{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction\n",
    "\n",
    "Use the {ref}`openpifpaf.predict <cli-help-predict>` tool on the command line to run \n",
    "multi-person pose estimation on images.\n",
    "To create predictions from other Python modules, please refer to {doc}`predict_api`.\n",
    "First we present the command line tool for predictions on images, \n",
    "{ref}`openpifpaf.predict <cli-help-predict>`. Then follows \n",
    "a short introduction to OpenPifPaf predictions on videos with \n",
    "{ref}`openpifpaf.video <cli-help-video>`.\n",
    "\n",
    "\n",
    "## Images\n",
    "\n",
    "Run {ref}`openpifpaf.predict <cli-help-predict>` on an image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "python -m openpifpaf.predict coco/000000081988.jpg --image-output --json-output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This command produced two outputs: an image and a json file.\n",
    "You can provide file or folder arguments to the `--image-output` and `--json-output` flags.\n",
    "Here, we used the default which created these two files:\n",
    "\n",
    "```sh\n",
    "coco/000000081988.jpg.predictions.jpeg\n",
    "coco/000000081988.jpg.predictions.json\n",
    "```\n",
    "\n",
    "Here is the image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import IPython\n",
    "IPython.display.Image('coco/000000081988.jpg.predictions.jpeg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Image credit: \"[Learning to surf](https://www.flickr.com/photos/fotologic/6038911779/in/photostream/)\" by fotologic which is licensed under [CC-BY-2.0].\n",
    "\n",
    "[CC-BY-2.0]: https://creativecommons.org/licenses/by/2.0/\n",
    "\n",
    "\n",
    "And below is the json output. The json data is a list where each entry in the list corresponds to one pose annotation. In this case, there are five entries corresponding to the five people in the image. Each annotation contains information on `\"keypoints\"`, `\"bbox\"`, `\"score\"` and `\"category_id\"`.\n",
    "\n",
    "All coordinates are in pixel coordinates. The `\"keypoints\"` entry is in COCO format with triples of `(x, y, c)` (`c` for confidence) for every joint as listed under {ref}`coco-person-keypoints`. The pixel coordinates have sub-pixel accuracy, i.e. 10.5 means the joint is between pixel 10 and 11.\n",
    "In rare cases, joints can be localized outside the field of view and then the pixel coordinates can be negative. When `c` is zero, the joint was not detected.\n",
    "\n",
    "The `\"bbox\"` (bounding box) format is `(x, y, w, h)`: the $(x, y)$ coordinate of the top-left corner followed by width and height.\n",
    "\n",
    "The `\"score\"` is a number between zero and one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "python -m json.tool coco/000000081988.jpg.predictions.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optional Arguments:\n",
    "\n",
    "* `--show`: show interactive matplotlib output\n",
    "* `--debug-indices`: enable debug messages and debug plots (see {ref}`Examples <example-debug>`)\n",
    "\n",
    "Full list of arguments is available with `--help`: {ref}`CLI help for predict <cli-help-predict>`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Videos\n",
    "\n",
    "```sh\n",
    "python3 -m openpifpaf.video --source myvideotoprocess.mp4 --video-output --json-output\n",
    "```\n",
    "\n",
    "Requires OpenCV. The `--video-output` option also requires matplotlib.\n",
    "Replace `myvideotoprocess.mp4` with `0` for webcam0 or other OpenCV compatible sources.\n",
    "The full list of arguments is available with `--help`: {ref}`CLI help for video <cli-help-video>`.\n",
    "\n",
    "In v0.12.6, we introduced the ability to pipe the output to a virtual camera.\n",
    "This virtual camera can then be used as the source camera in Zoom and other\n",
    "conferencing softwares. You need a virtual camera on your system, e.g.\n",
    "from [OBS Studio](https://obsproject.com) (Mac and Windows) or\n",
    "[v4l2loopback](https://github.com/umlaeute/v4l2loopback#distributions) (Linux)\n",
    "and need to install `pip3 install pyvirtualcam`. Then you can use the \n",
    "`--video-output=virtualcam` argument.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Debug\n",
    "\n",
    "Obtain extra information by adding `--debug` to the command line. It will\n",
    "show the structure of the neural network and timing information in the decoder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "python -m openpifpaf.predict coco/000000081988.jpg --image-output --json-output --debug"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can enable debug plots with `--debug-indices`. \n",
    "Please refer to {ref}`the debug outputs in the Examples <example-debug>` and \n",
    "some further {ref}`debug outputs in the prediction API <predict-fields>`."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "ea6946363a43e80d241452ab397f4c58bdd3d2517da174158e9c46ce6717422a"
  },
  "kernelspec": {
   "display_name": "Python 3.9.4 64-bit ('venv3': venv)",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": ""
  },
  "metadata": {
   "interpreter": {
    "hash": "ea6946363a43e80d241452ab397f4c58bdd3d2517da174158e9c46ce6717422a"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}