{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-cell"
    ]
   },
   "outputs": [],
   "source": [
    "import openpifpaf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Models\n",
    "\n",
    "For now, see Method section in our publication {cite}`kreiss2019pifpaf` for an architecture overview."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Base Networks\n",
    "\n",
    "The following types of base networks have implementations and can be used with the `--basenet` command line argument for training. Not all base networks are well optimized and you can consult the {ref}`introduction` for benchmark results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(openpifpaf.BASE_FACTORIES.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pretrained Models\n",
    "\n",
    "The current list of pretrained models is below and you can use them with the `--checkpoint` argument. Not all pretrained models are well optimized and you can consult the {ref}`introduction` for benchmark results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[c for c, u in openpifpaf.CHECKPOINT_URLS.items() if isinstance(u, str)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Headmeta\n",
    "\n",
    "Headmeta (see {ref}`api:headmeta`) is a class that holds configuration data about a head network. It is instantiated in a DataModule (above) and used throughout OpenPifPaf to configure various other parts. For example, the `cocokp` head meta instances are: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "openpifpaf.plugins.coco.CocoKp().head_metas[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "openpifpaf.plugins.coco.CocoKp().head_metas[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When a new network is created, information from the head metas will be used to create the appropriate torch graph for the heads. It will use the type of the head meta (`openpifpaf.headmeta.Cif`, `openpifpaf.headmeta.Caf`, ...) and information like the number of keypoints in Cif or the number of skeleton connections in Caf to know how many feature maps to create.\n",
    "\n",
    "Similarly, the decoder will look for heads that are of type `headmeta.Cif` and `headmeta.Caf` to instantiate the CifCaf decoder.\n",
    "\n",
    "To get started, have a look how the head metas are created in `CocoKp` for new pose datasets and in `CocoDet` for new detection datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(coordinate-system)=\n",
    "## Coordinate System\n",
    "\n",
    "When adding your own datasets or transformations, it is good to keep in mind the coordinate system that OpenPifPaf assumes. The short summary is, everything is done in such a way that annotation coordinates only need to be multiplied by the image rescaling factor without offset. Therefore, the (0, 0) feature map needs to be aligned with the (0, 0) pixel.\n",
    "\n",
    "![coordinate system](images/coordinate_system.png)\n",
    "\n",
    "This also shows that training image sizes must be a multiple of the network stride _plus one_, i.e., in the above image, an input image size of $5\\times5$ would be optimal for a feature map of size $2\\times2$ because the stride is 4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": ""
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}