{ "cells": [ { "cell_type": "markdown", "id": "0", "metadata": {}, "source": [ "# Create a Semantic Segmentation Table from ADE20K\n", "\n", "Ingest a small ADE20K subset into a 3LC table for semantic segmentation — the simplest case: paired images and grayscale label-map PNGs plus an id-to-name mapping.\n", "\n", "![](../../images/ade20k-semseg.png)\n", "\n", "\n", "\n", "Semantic segmentation labels every pixel with a class. This is the minimal ingest path: point the\n", "documented front door `Table.from_semantic_segmentation` at the image files and their single-channel\n", "mask PNGs (whose pixel values are the class ids), together with the ADE20K label map. No decoding or\n", "remapping is needed — the front door reads the label-map PNGs directly. In ADE20K, id 0 marks the\n", "*unlabeled* region (pixels outside the 150 classes); we tag it as `void` so it is treated as an\n", "ignore / don't-care region and excluded from segmentation metrics.\n", "\n", "For richer variants — a custom trimap split, or a 21-class problem with a training companion — see the\n", "`create-oxford-pets-semseg-table` and `create-pascal-voc-semseg-table` notebooks in this folder." ] }, { "cell_type": "markdown", "id": "1", "metadata": {}, "source": [ "## Project setup" ] }, { "cell_type": "code", "execution_count": null, "id": "2", "metadata": { "tags": [ "parameters" ] }, "outputs": [], "source": [ "# Parameters\n", "PROJECT_NAME = \"3LC Tutorials - Semantic Segmentation ADE20k\"\n", "DATASET_NAME = \"ADE20k_toy_dataset\"\n", "TABLE_NAME = \"train\"\n", "DOWNLOAD_PATH = \"../../transient_data\" # dataset root; point at an existing copy to skip the download\n", "INSTALL_DEPENDENCIES = True" ] }, { "cell_type": "markdown", "id": "3", "metadata": {}, "source": [ "## Install dependencies" ] }, { "cell_type": "code", "execution_count": null, "id": "4", "metadata": {}, "outputs": [], "source": [ "if INSTALL_DEPENDENCIES:\n", " %pip install -q 3lc huggingface-hub requests matplotlib" ] }, { "cell_type": "markdown", "id": "5", "metadata": {}, "source": [ "## Imports" ] }, { "cell_type": "code", "execution_count": null, "id": "6", "metadata": {}, "outputs": [], "source": [ "import io\n", "import json\n", "import zipfile\n", "from pathlib import Path\n", "\n", "import requests\n", "import tlc\n", "from huggingface_hub import hf_hub_download" ] }, { "cell_type": "markdown", "id": "7", "metadata": {}, "source": [ "## Download the dataset\n", "\n", "The ADE20K toy subset is a small zip of paired images and grayscale mask PNGs. We download it only when\n", "it is missing, so a pre-seeded copy under `DOWNLOAD_PATH` is used as-is." ] }, { "cell_type": "code", "execution_count": null, "id": "8", "metadata": {}, "outputs": [], "source": [ "DATASET_ROOT = (Path(DOWNLOAD_PATH) / \"ADE20k_toy_dataset\").resolve()\n", "\n", "if not DATASET_ROOT.exists():\n", " print(\"Downloading ADE20K toy dataset...\")\n", " response = requests.get(\"https://www.dropbox.com/s/l1e45oht447053f/ADE20k_toy_dataset.zip?dl=1\")\n", " response.raise_for_status()\n", " zipfile.ZipFile(io.BytesIO(response.content)).extractall(DOWNLOAD_PATH)\n", "print(\"Dataset at\", DATASET_ROOT)" ] }, { "cell_type": "markdown", "id": "9", "metadata": {}, "source": [ "## Define the class universe\n", "\n", "We pull ADE20K's 150-class `id -> name` map from the Hugging Face Hub. That file is 0-indexed\n", "(`\"0\" -> \"wall\"`, ...), but the mask PNGs use id 0 for the *unlabeled* region and 1..150 for the\n", "classes, so we shift the class ids up by one and add `unlabeled` at id 0. Passing `void=0` to the front\n", "door tags that region as a don't-care / ignore class — it stays in the value map but is excluded from\n", "segmentation metrics like mean IoU. (ADE20K has no separate background class; every labelled pixel is\n", "one of the 150 classes.)" ] }, { "cell_type": "code", "execution_count": null, "id": "10", "metadata": {}, "outputs": [], "source": [ "with open(\n", " hf_hub_download(\n", " repo_id=\"huggingface/label-files\",\n", " filename=\"ade20k-id2label.json\",\n", " repo_type=\"dataset\",\n", " )\n", ") as f:\n", " id2label = json.load(f)\n", "\n", "# Shift the 0-indexed HF map up to the PNGs' 1-indexed class ids, and add the unlabeled (void) region at id 0.\n", "VOID_ID = 0\n", "classes = {VOID_ID: \"unlabeled\", **{int(k) + 1: name for k, name in id2label.items()}}\n", "print(f\"{len(classes)} classes incl. unlabeled/void (e.g. 1={classes[1]}, 2={classes[2]})\")" ] }, { "cell_type": "markdown", "id": "11", "metadata": {}, "source": [ "## Locate the image/mask pairs\n", "\n", "The images and masks share a filename stem across parallel `images/` and `annotations/` folders. We\n", "sort both so they line up positionally, then hand the file paths straight to the front door." ] }, { "cell_type": "code", "execution_count": null, "id": "12", "metadata": {}, "outputs": [], "source": [ "image_paths = sorted(DATASET_ROOT.glob(\"**/images/training/*.jpg\"))\n", "mask_paths = sorted(DATASET_ROOT.glob(\"**/annotations/training/*.png\"))\n", "assert image_paths and len(image_paths) == len(mask_paths), \"expected paired image/mask files\"\n", "print(f\"{len(image_paths)} image/mask pairs\")" ] }, { "cell_type": "markdown", "id": "13", "metadata": {}, "source": [ "## Write the table\n", "\n", "`Table.from_semantic_segmentation` reads each mask PNG as a dense `(H, W)` label map and writes an\n", "`image` column plus an RLE-backed `mask` column in one call." ] }, { "cell_type": "code", "execution_count": null, "id": "14", "metadata": {}, "outputs": [], "source": [ "table = tlc.Table.from_semantic_segmentation(\n", " images=image_paths,\n", " masks=mask_paths,\n", " classes=classes,\n", " void=VOID_ID,\n", " project_name=PROJECT_NAME,\n", " dataset_name=DATASET_NAME,\n", " table_name=TABLE_NAME,\n", " if_exists=\"overwrite\",\n", ")\n", "print(f\"Wrote {len(table)} rows -> {table.url}\")" ] }, { "cell_type": "markdown", "id": "15", "metadata": {}, "source": [ "## Inspect a sample\n", "\n", "Reading a row back hands you the source image and the dense mask. We overlay them to sanity-check the\n", "alignment." ] }, { "cell_type": "code", "execution_count": null, "id": "16", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "\n", "sample = table[0]\n", "seg = sample[\"mask\"]\n", "print(\"Classes present:\", [classes[c] for c in seg.present_class_ids.tolist()])\n", "\n", "image = sample[\"image\"]\n", "fig, axes = plt.subplots(1, 2, figsize=(10, 4))\n", "axes[0].imshow(image)\n", "axes[0].set_title(\"image\")\n", "axes[1].imshow(image)\n", "axes[1].imshow(seg.mask, alpha=0.5, cmap=\"tab20\", interpolation=\"nearest\")\n", "axes[1].set_title(\"segmentation\")\n", "for ax in axes:\n", " ax.axis(\"off\")\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "17", "metadata": {}, "source": [ "## Next steps\n", "\n", "The table is now in your 3LC project — open it in the [3LC Dashboard](https://dashboard.3lc.ai) to\n", "browse the images and their segmentations.\n", "\n", "For an end-to-end training example on the same sample type, continue with the Oxford-IIIT Pets pair\n", "(`create-oxford-pets-semseg-table` + `pytorch-oxford-pets-unet-training`)." ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.12.10" } }, "nbformat": 4, "nbformat_minor": 5 }