{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Drawing Labels on Samples\n",
    "\n",
    "This recipe demonstrates how to use FiftyOne to render annotated versions of image and video [samples](https://voxel51.com/docs/fiftyone/user_guide/using_datasets.html#samples) with their [label field(s)](https://voxel51.com/docs/fiftyone/user_guide/using_datasets.html#labels) overlaid."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "If you haven't already, install FiftyOne:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install fiftyone"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this recipe we'll use the [FiftyOne Dataset Zoo](https://voxel51.com/docs/fiftyone/user_guide/dataset_creation/zoo_datasets.html) to download some labeled datasets to use as sample data for drawing labels.\n",
    "\n",
    "Behind the scenes, FiftyOne uses either the\n",
    "[TensorFlow Datasets](https://www.tensorflow.org/datasets) or\n",
    "[TorchVision Datasets](https://pytorch.org/vision/stable/datasets.html) libraries to wrangle the datasets, depending on which ML library you have installed.\n",
    "\n",
    "You can, for example, install PyTorch as follows (we'll also need `pycocotools` to load the COCO dataset, in particular):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install torch torchvision\n",
    "!pip install pycocotools"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Drawing COCO detections\n",
    "\n",
    "You can download the validation split of the COCO-2017 dataset to `~/fiftyone/coco-2017/validation` by running the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Split 'validation' already downloaded\r\n"
     ]
    }
   ],
   "source": [
    "!fiftyone zoo datasets download coco-2017 --splits validation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's load the dataset, extract a [DatasetView](https://voxel51.com/docs/fiftyone/user_guide/using_datasets.html#datasetviews) that contains 100 images from the dataset, and render them as annotated images with their ground truth labels overlaid:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Split 'validation' already downloaded\n",
      "Loading 'coco-2017' split 'validation'\n",
      " 100% |█████| 5000/5000 [14.8s elapsed, 0s remaining, 339.4 samples/s]      \n",
      "Writing annotated images to '/tmp/fiftyone/draw_labels/coco-2017-validation-anno'\n",
      " 100% |███████| 100/100 [7.3s elapsed, 0s remaining, 11.9 samples/s]        \n",
      "Annotation complete\n"
     ]
    }
   ],
   "source": [
    "import fiftyone as fo\n",
    "import fiftyone.zoo as foz\n",
    "import fiftyone.utils.annotations as foua\n",
    "\n",
    "# Directory to write the output annotations\n",
    "anno_dir = \"/tmp/fiftyone/draw_labels/coco-2017-validation-anno\"\n",
    "\n",
    "# Load the validation split of the COCO-2017 dataset\n",
    "dataset = foz.load_zoo_dataset(\"coco-2017\", split=\"validation\")\n",
    "\n",
    "# Extract some samples\n",
    "view = dataset.limit(100)\n",
    "\n",
    "#\n",
    "# You can customize the look-and-feel of the annotations\n",
    "# For more information, see:\n",
    "# https://voxel51.com/docs/fiftyone/user_guide/draw_labels.html#customizing-label-rendering\n",
    "#\n",
    "config = foua.DrawConfig({\n",
    "    \"per_object_label_colors\": True\n",
    "})\n",
    "\n",
    "# Render the labels\n",
    "print(\"Writing annotated images to '%s'\" % anno_dir)\n",
    "view.draw_labels(anno_dir, config=config)\n",
    "print(\"Annotation complete\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's list the output directory to verify that the annotations have been generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 51976\r\n",
      "drwxr-xr-x  202 Brian  wheel   6.3K Jul 27 18:36 .\r\n",
      "drwxr-xr-x    5 Brian  wheel   160B Jul 27 15:59 ..\r\n",
      "-rw-r--r--    1 Brian  wheel   115K Jul 27 18:36 000001-2.jpg\r\n",
      "-rw-r--r--@   1 Brian  wheel   116K Jul 27 12:51 000001.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel   243K Jul 27 18:36 000002-2.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel   243K Jul 27 12:51 000002.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel   177K Jul 27 18:36 000003-2.jpg\r\n",
      "-rw-r--r--@   1 Brian  wheel   177K Jul 27 12:51 000003.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel   101K Jul 27 18:36 000004-2.jpg\r\n"
     ]
    }
   ],
   "source": [
    "!ls -lah /tmp/fiftyone/draw_labels/coco-2017-validation-anno | head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an example of an annotated image that was generated:\n",
    "\n",
    "![coco-2017-annotated](images/draw_labels_coco2017.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Drawing Caltech 101 classifications\n",
    "\n",
    "You can download the test split of the Caltech 101 dataset to `~/fiftyone/caltech101/test` by running the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Split 'test' already downloaded\n"
     ]
    }
   ],
   "source": [
    "!fiftyone zoo datasets download caltech101 --splits test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's load the dataset, extract a [DatasetView](https://voxel51.com/docs/fiftyone/user_guide/using_datasets.html#datasetviews) that contains 100 images from the dataset, and render them as annotated images with their ground truth labels overlaid:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Split 'test' already downloaded\n",
      "Loading 'caltech101' split 'test'\n",
      " 100% |█████| 9145/9145 [4.8s elapsed, 0s remaining, 1.9K samples/s]      \n",
      "Writing annotated images to '/tmp/fiftyone/draw_labels/caltech101-test-anno'\n",
      " 100% |███████| 100/100 [2.6s elapsed, 0s remaining, 37.4 samples/s]        \n",
      "Annotation complete\n"
     ]
    }
   ],
   "source": [
    "import fiftyone as fo\n",
    "import fiftyone.zoo as foz\n",
    "import fiftyone.utils.annotations as foua\n",
    "\n",
    "# Directory to write the output annotations\n",
    "anno_dir = \"/tmp/fiftyone/draw_labels/caltech101-test-anno\"\n",
    "\n",
    "# Load the test split of the Caltech 101 dataset\n",
    "dataset = foz.load_zoo_dataset(\"caltech101\", split=\"test\")\n",
    "\n",
    "# Extract some samples\n",
    "view = dataset.limit(100)\n",
    "\n",
    "#\n",
    "# You can customize the look-and-feel of the annotations\n",
    "# For more information, see:\n",
    "# https://voxel51.com/docs/fiftyone/user_guide/draw_labels.html#customizing-label-rendering\n",
    "#\n",
    "config = foua.DrawConfig({\n",
    "    \"font_size\": 36\n",
    "})\n",
    "\n",
    "# Render the labels\n",
    "print(\"Writing annotated images to '%s'\" % anno_dir)\n",
    "view.draw_labels(anno_dir, config=config)\n",
    "print(\"Annotation complete\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's list the output directory to verify that the annotations have been generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 17456\r\n",
      "drwxr-xr-x  182 Brian  wheel   5.7K Jul 27 18:37 .\r\n",
      "drwxr-xr-x    5 Brian  wheel   160B Jul 27 15:59 ..\r\n",
      "-rw-r--r--@   1 Brian  wheel    13K Jul 27 18:37 image_0001-2.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel    41K Jul 27 15:59 image_0001.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel   197K Jul 27 18:37 image_0002.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel   5.9K Jul 27 18:37 image_0003.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel    19K Jul 27 18:37 image_0004-2.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel    33K Jul 27 15:59 image_0004.jpg\r\n",
      "-rw-r--r--    1 Brian  wheel    18K Jul 27 18:37 image_0005-2.jpg\r\n"
     ]
    }
   ],
   "source": [
    "!ls -lah /tmp/fiftyone/draw_labels/caltech101-test-anno | head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an example of an annotated image that was generated:\n",
    "\n",
    "<img src=\"images/draw_labels_caltech101.jpg\" width=\"350\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Drawing labels on videos\n",
    "\n",
    "FiftyOne can also render frame labels onto video samples.\n",
    "\n",
    "To demonstrate, let's work with the (small) video quickstart dataset from the zoo:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset already downloaded\n",
      "Loading 'quickstart-video'\n",
      " 100% |█████████| 10/10 [15.9s elapsed, 0s remaining, 0.6 samples/s]     \n",
      "Name:           quickstart-video\n",
      "Media type      video\n",
      "Num samples:    10\n",
      "Persistent:     False\n",
      "Info:           {'description': 'quickstart-video'}\n",
      "Tags:           []\n",
      "Sample fields:\n",
      "    media_type: fiftyone.core.fields.StringField\n",
      "    filepath:   fiftyone.core.fields.StringField\n",
      "    tags:       fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)\n",
      "    metadata:   fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)\n",
      "    frames:     fiftyone.core.fields.FramesField\n",
      "Frame fields:\n",
      "    frame_number: fiftyone.core.fields.FrameNumberField\n",
      "    objs:         fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)\n"
     ]
    }
   ],
   "source": [
    "import fiftyone.zoo as foz\n",
    "\n",
    "# Load a small video dataset\n",
    "dataset = foz.load_zoo_dataset(\"quickstart-video\")\n",
    "\n",
    "print(dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the dataset contains frame-level detections in the `objs` field of each frame.\n",
    "\n",
    "Let's make a [DatasetView](https://voxel51.com/docs/fiftyone/user_guide/using_datasets.html#datasetviews) that contains a couple random videos from the dataset and render them as annotated videos with the frame-level detections overlaid:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing annotated videos to '/tmp/fiftyone/draw_labels/quickstart-video-anno'\n",
      "Rendering video 1/2: '/tmp/fiftyone/draw_labels/quickstart-video-anno/0587e1cfc2344523922652d8b227fba4-000014-video_052.mp4'\n",
      " 100% |████████| 120/120 [19.0s elapsed, 0s remaining, 6.7 frames/s]      \n",
      "Rendering video 2/2: '/tmp/fiftyone/draw_labels/quickstart-video-anno/0587e1cfc2344523922652d8b227fba4-000014-video_164.mp4'\n",
      " 100% |████████| 120/120 [27.2s elapsed, 0s remaining, 4.3 frames/s]      \n",
      "Annotation complete\n"
     ]
    }
   ],
   "source": [
    "import fiftyone.utils.annotations as foua\n",
    "\n",
    "# Directory to write the output annotations\n",
    "anno_dir = \"/tmp/fiftyone/draw_labels/quickstart-video-anno\"\n",
    "\n",
    "# Extract two random samples\n",
    "view = dataset.take(2)\n",
    "\n",
    "#\n",
    "# You can customize the look-and-feel of the annotations\n",
    "# For more information, see:\n",
    "# https://voxel51.com/docs/fiftyone/user_guide/draw_labels.html#customizing-label-rendering\n",
    "#\n",
    "config = foua.DrawConfig({\n",
    "    \"per_object_label_colors\": True\n",
    "})\n",
    "\n",
    "# Render the labels\n",
    "print(\"Writing annotated videos to '%s'\" % anno_dir)\n",
    "view.draw_labels(anno_dir, config=config)\n",
    "print(\"Annotation complete\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's list the output directory to verify that the annotations have been generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 34832\r\n",
      "drwxr-xr-x  4 Brian  wheel   128B Oct  7 23:57 \u001b[34m.\u001b[m\u001b[m\r\n",
      "drwxr-xr-x  3 Brian  wheel    96B Oct  7 23:57 \u001b[34m..\u001b[m\u001b[m\r\n",
      "-rw-r--r--  1 Brian  wheel   7.5M Oct  7 23:57 0587e1cfc2344523922652d8b227fba4-000014-video_052.mp4\r\n",
      "-rw-r--r--  1 Brian  wheel   8.5M Oct  7 23:58 0587e1cfc2344523922652d8b227fba4-000014-video_164.mp4\r\n"
     ]
    }
   ],
   "source": [
    "!ls -lah /tmp/fiftyone/draw_labels/quickstart-video-anno"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a snippet of an annotated video that was generated:\n",
    "\n",
    "![quickstart-video-annotated](images/draw_labels_quickstart_video.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleanup\n",
    "\n",
    "You can cleanup the files generated by this recipe by running the command below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "!rm -rf /tmp/fiftyone"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}