{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", " Try in Google Colab\n", " \n", " \n", " \n", " \n", " Share via nbviewer\n", " \n", " \n", " \n", " \n", " View on GitHub\n", " \n", " \n", " \n", " \n", " Download notebook\n", " \n", "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Comparing YOLOv4 and EfficientDet\n", "\n", "This example compares the [YOLOv4](https://arxiv.org/abs/2004.10934) and [EfficientDet](https://arxiv.org/abs/1911.09070) object detection models on the [COCO dataset](https://cocodataset.org/#home) using FiftyOne.\n", "\n", "For more information check out the [YOLOv4 blog post](https://medium.com/voxel51/fifteen-minutes-with-fiftyone-yolov4-180cf66923a9?source=friends_link&sk=71922580c58ef371fbbb80f7356a872d) and [EfficientDet blog post](https://medium.com/voxel51/fifteen-minutes-with-fiftyone-efficientdet-d84b60ffff28?source=friends_link&sk=cf3f54dc3b8544e45fc70c07ae0eea30)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup\n", "\n", "If you haven't already, install FiftyOne:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pip install fiftyone" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load dataset\n", "\n", "First, let's load the validation split of COCO-2017 from the [FiftyOne Dataset Zoo](https://voxel51.com/docs/fiftyone/user_guide/dataset_creation/zoo.html):" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "import fiftyone as fo\n", "import fiftyone.zoo as foz" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Split 'validation' already downloaded\n", "Loading 'coco-2017' split 'validation'\n", " 100% |███████████████| 5000/5000 [41.7s elapsed, 0s remaining, 121.3 samples/s] \n", "Dataset 'coco-2017-validation' created\n" ] } ], "source": [ "dataset = foz.load_zoo_dataset(\"coco-2017\", split=\"validation\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, let's add some pre-generated YOLOv4 and EfficientDet predictions to the dataset.\n", "\n", "You can download the predictions from [this Google Drive link (72MB)](https://drive.google.com/file/d/1wJJy8F25ixPSfrXcYsZz6VJ3rtsVIck5)." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# Path to the downloaded JSON file\n", "DATASET_PATH = \"/path/to/yolo_edet_dataset.json\"" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " 100% |███████████████| 5000/5000 [4.6m elapsed, 0s remaining, 18.6 samples/s] \n" ] } ], "source": [ "# Load the predictions\n", "data_dir = os.path.dirname(dataset.first().filepath)\n", "predictions = fo.Dataset.from_json(DATASET_PATH, rel_dir=data_dir)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# Merge the predictions into `dataset`\n", "dataset.merge_samples(predictions)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's launch the [FiftyOne App](https://voxel51.com/docs/fiftyone/user_guide/app.html) and qualitatively compare the predictions of the various models:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", " \n", "
\n", " \n", "
\n", "\n", "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "session = fo.launch_app(dataset)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate and explore\n", "\n", "You can evaluate any of the predictions with respect to the ground truth labels.\n", "\n", "For example, let's evaluate the YOLOv4 predictions:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Evaluating detections...\n", " 100% |███████████████| 5000/5000 [1.4m elapsed, 0s remaining, 68.4 samples/s] \n" ] } ], "source": [ "results = dataset.evaluate_detections(\n", " \"yolov4\",\n", " gt_field=\"ground_truth\",\n", " eval_key=\"yolov4\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[With dataset views](https://voxel51.com/docs/fiftyone/user_guide/using_views.html), you can easily identify samples of interest. For example, let's view the samples where YOLOv4 had the most false positives:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", " \n", "
\n", " \n", "
\n", "\n", "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "session.view = dataset.sort_by(\"yolov4_fp\", reverse=True)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "session.freeze() # for notebook sharing" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" } }, "nbformat": 4, "nbformat_minor": 4 }