{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quickstart: In-App Labeling in 15 Minutes\n",
    "\n",
    "This quickstart shows you how to annotate a **multimodal 2D/3D dataset** directly in the FiftyOne App. By the end, you'll know:\n",
    "\n",
    "1. How grouped datasets work (synchronized 2D + 3D data)\n",
    "2. How to switch between camera and point cloud views\n",
    "3. How to draw 2D bounding boxes on images\n",
    "4. How to explore 3D point cloud data\n",
    "\n",
    "> **Note:** This track is standalone. The Full Loop track uses a cloned dataset (`annotation_tutorial`) so you can do both independently."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -U fiftyone"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import fiftyone as fo\n",
    "import fiftyone.zoo as foz\n",
    "from fiftyone import ViewField as F"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the Multimodal Dataset\n",
    "\n",
    "We'll use `quickstart-groups`, a subset of the KITTI self-driving dataset with:\n",
    "- **Left camera** images (2D)\n",
    "- **Right camera** images (stereo pair)\n",
    "- **Point cloud** data (3D LiDAR)\n",
    "\n",
    "All synchronized per scene."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load directly from the zoo (changes are ephemeral)\n",
    "dataset = foz.load_zoo_dataset(\"quickstart-groups\")\n",
    "\n",
    "print(f\"Dataset: {dataset.name}\")\n",
    "print(f\"Media type: {dataset.media_type}\")\n",
    "print(f\"Group slices: {dataset.group_slices}\")\n",
    "print(f\"Default slice: {dataset.default_group_slice}\")\n",
    "print(f\"Num groups (scenes): {len(dataset.distinct('group.id'))}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understand the Grouped Structure\n",
    "\n",
    "A **grouped dataset** links related samples together. Each group represents one scene with multiple views:\n",
    "\n",
    "| Slice | Content | What you'll see |\n",
    "|-------|---------|----------------|\n",
    "| `left` | Left camera image | 2D RGB image with existing detections |\n",
    "| `right` | Right camera image | Stereo pair |\n",
    "| `pcd` | Point cloud | 3D LiDAR visualization |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "# Look at one group\ngroup_ids = dataset.distinct(\"group.id\")\nprint(f\"Total groups (scenes): {len(group_ids)}\")\n\n# Examine first group\nexample_group = dataset.get_group(group_ids[0])\nprint(f\"\\nSamples in first group:\")\nfor slice_name, sample in example_group.items():\n    print(f\"  {slice_name}: {sample.filepath.split('/')[-1]}\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Launch the App"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = fo.launch_app(dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the Grouped Data\n",
    "\n",
    "In the App, you will notice the **group icon** in the grid. Each cell represents a group (scene) with multiple slices.\n",
    "\n",
    "1. **Click on any sample** to open the expanded view\n",
    "2. **Click the Annotate tab** (right sidebar)\n",
    "3. Use the **Annotating Slice** dropdown to switch between `left` (camera), `right` (camera), and `pcd` (point cloud)\n",
    "\n",
    "Try switching to `pcd` to see the 3D point cloud visualization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enter Annotate Mode (2D)\n",
    "\n",
    "Let's label some objects on the camera images:\n",
    "\n",
    "1. **Click on a sample** to open the modal\n",
    "2. **Select the `left` slice** (camera view)\n",
    "3. **Click the \"Annotate\" tab** (pencil icon in the right sidebar)\n",
    "\n",
    "You're now in annotation mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Label Field\n",
    "\n",
    "Before drawing boxes, create a field to store them:\n",
    "\n",
    "1. In Annotate mode, click the **Add schema** button\n",
    "2. Click **\"New Field\"**\n",
    "3. Enter name: `my_labels`\n",
    "4. Select type: **Detections** (bounding boxes)\n",
    "5. Add classes: `Car`, `Pedestrian`, `Cyclist`\n",
    "6. Click **Create**\n",
    "\n",
    "![Create Label Field](https://cdn.voxel51.com/getting_started_annotation/notebook1/create_label_field.webp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Draw 2D Bounding Boxes\n",
    "\n",
    "1. **Click the Detection button** (square icon)\n",
    "2. **Click and drag** on an object to draw a box\n",
    "3. **Select a label** from the class list in the sidebar\n",
    "4. Your label saves automatically!\n",
    "\n",
    "### Tips:\n",
    "- **Resize**: Drag corners or edges\n",
    "- **Move**: Click inside and drag\n",
    "- **Delete**: Select and press Delete\n",
    "\n",
    "**Try it:** Draw 3-5 boxes on different objects, then move to a few more samples.\n",
    "\n",
    "![Draw Bounding Boxes](https://cdn.voxel51.com/getting_started_annotation/notebook1/draw_bounding_boxes.webp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore 3D Data\n",
    "\n",
    "Now let us look at the point cloud:\n",
    "\n",
    "1. **Switch to the `pcd` slice** using the Annotating Slice dropdown in the Annotate tab\n",
    "2. **Switch to the Explore tab** to see the existing `ground_truth` labels as 3D cuboids\n",
    "3. The 3D visualizer loads the point cloud\n",
    "4. **Navigate:**\n",
    "   - **Rotate**: Left-click and drag\n",
    "   - **Pan**: Right-click and drag (or Shift + left-click)\n",
    "   - **Zoom**: Scroll wheel\n",
    "   - **Reset view**: Press `1`, `2`, `3`, or `4` for preset angles\n",
    "\n",
    "![Explore 3D Data](https://cdn.voxel51.com/getting_started_annotation/notebook1/explore_3d_data.webp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Verify Your Labels Saved\n",
    "\n",
    "Run this after labeling some samples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reload to see changes\n",
    "dataset.reload()\n",
    "\n",
    "LABEL_FIELD = \"my_labels\"\n",
    "\n",
    "# Check if field exists and has labels\n",
    "if LABEL_FIELD in dataset.get_field_schema():\n",
    "    # Get left camera slice (where we annotated)\n",
    "    left_view = dataset.select_group_slices([\"left\"])\n",
    "    labeled = left_view.match(F(f\"{LABEL_FIELD}.detections\").length() > 0)\n",
    "    \n",
    "    total_dets = sum(\n",
    "        len(s[LABEL_FIELD].detections) \n",
    "        for s in labeled \n",
    "        if s[LABEL_FIELD] is not None\n",
    "    )\n",
    "    \n",
    "    print(f\"Samples with labels: {len(labeled)}\")\n",
    "    print(f\"Total detections: {total_dets}\")\n",
    "    \n",
    "    if len(labeled) > 0:\n",
    "        sample = labeled.first()\n",
    "        print(f\"\\nExample from {sample.filepath.split('/')[-1]}:\")\n",
    "        for det in sample[LABEL_FIELD].detections:\n",
    "            print(f\"  {det.label}\")\n",
    "else:\n",
    "    print(f\"Field '{LABEL_FIELD}' not found.\")\n",
    "    print(\"Create it in the App: Annotate tab -> Schema -> New Field\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Key Concepts\n",
    "\n",
    "### Grouped Datasets\n",
    "- Link related samples (same scene, different sensors)\n",
    "- Each **group** has multiple **slices** (left, right, pcd)\n",
    "- Use the slice selector to switch views\n",
    "\n",
    "### Slice Selector in Annotate Tab\n",
    "- Shows which slice you're annotating\n",
    "- Switch to `left` for 2D camera annotation\n",
    "- Switch to `pcd` for 3D point cloud viewing\n",
    "\n",
    "### Ephemeral vs Persistent\n",
    "- This quickstart works on the zoo dataset directly (changes are ephemeral)\n",
    "- For persistent work, clone the dataset (covered in Full Loop)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "You learned how to:\n",
    "1. Load a multimodal grouped dataset (KITTI)\n",
    "2. Navigate between 2D and 3D slices\n",
    "3. Create annotation fields and draw 2D bounding boxes\n",
    "4. Explore point cloud data in the 3D visualizer\n",
    "\n",
    "**That's multimodal annotation in FiftyOne.**\n",
    "\n",
    "---\n",
    "\n",
    "## What's Next?\n",
    "\n",
    "For production annotation workflows, continue to the **Full Loop Track**:\n",
    "\n",
    "- **Step 2: Setup Splits** - Create proper train/val/test splits at the group level\n",
    "- **Step 3: Smart Selection** - Use diversity sampling to pick high-value scenes\n",
    "- **Step 4: 2D Annotation** - Disciplined labeling with QA checks\n",
    "- **Step 5: 3D Annotation** - Annotate cuboids on point clouds\n",
    "- **Step 6: Train + Evaluate** - Train a model and analyze failures\n",
    "- **Step 7: Iteration** - Use failures to drive the next labeling batch\n",
    "\n",
    "The Full Loop clones the dataset to `annotation_tutorial` for persistent work."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}