{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Lesson 40: Object Detection II — From Classification to Regression\n",
"\n",
"Lesson 39's sliding-window detector classified thousands of fixed-size windows and merged the survivors with NMS. That works, but it is fundamentally a *classification* approach bolted onto a search — the network never predicts a box directly, only \"face or not, at this exact window.\" Modern detectors instead treat localization as a **regression** problem: given an image, directly predict box coordinates. This lesson builds the simplest possible version of that idea, then surveys how real detectors (R-CNN, YOLO, SSD) scale it up."
]
},
{
"cell_type": "code",
"id": "a6d0f365",
"source": "import numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as patches",
"metadata": {},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"id": "dbe19032",
"source": "## One object per image: predict a box directly\n\nThe simplest possible regression detector: one circular blob per image, at an unknown location and size. Instead of a class label, the network's target is now four numbers — `(x0, y0, width, height)` of the bounding box, normalized to `[0, 1]` by image size.",
"metadata": {}
},
{
"cell_type": "code",
"id": "f2b18fc7",
"source": "SIZE = 32\n\ndef make_scene(rng, size=SIZE, obj_size=8):\n scene = np.zeros((size, size), dtype=np.float32)\n cx = rng.integers(obj_size, size - obj_size)\n cy = rng.integers(obj_size, size - obj_size)\n yy, xx = np.mgrid[0:size, 0:size]\n scene[((xx - cx) ** 2 + (yy - cy) ** 2) <= (obj_size * 0.5) ** 2] = 1.0\n scene = np.clip(scene + rng.normal(0, 0.05, scene.shape), 0, 1).astype(np.float32)\n box = (cx - obj_size // 2, cy - obj_size // 2, obj_size, obj_size) # x0, y0, w, h\n return scene, box\n\nrng = np.random.default_rng(9)\nN = 400\nscenes, boxes = [], []\nfor _ in range(N):\n s, b = make_scene(rng)\n scenes.append(s); boxes.append(b)\nscenes = np.array(scenes, dtype=np.float32)\nboxes = np.array(boxes, dtype=np.float32)\n\nsplit = int(0.85 * N)\nXtr, Btr = scenes[:split], boxes[:split] / SIZE\nXte, Bte = scenes[split:], boxes[split:] / SIZE\n\nfig, axes = plt.subplots(1, 4, figsize=(9, 2.5))\nfor ax, im, b in zip(axes, Xtr[:4], boxes[:4]):\n ax.imshow(im, cmap='gray')\n ax.add_patch(patches.Rectangle((b[0], b[1]), b[2], b[3], edgecolor='lime', facecolor='none', linewidth=2))\n ax.axis('off')\nplt.show()",
"metadata": {},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"id": "f532b10a",
"source": "## The model and the IoU metric\n\nThe network is a CNN backbone (Lesson 33's pattern) followed by a 4-output regression head with a sigmoid, so every prediction lands in `[0, 1]` — a valid normalized box coordinate. It's trained with plain MSE loss against the true box, and evaluated with **IoU** (Lesson 39's intersection-over-union), the metric that actually matters for detection: how much the predicted and true boxes overlap, not how close the four numbers are in isolation.",
"metadata": {}
},
{
"cell_type": "code",
"id": "65976fc1",
"source": "class Detector(nn.Module):\n def __init__(self):\n super().__init__()\n self.conv = nn.Sequential(\n nn.Conv2d(1, 16, 5, padding=2), nn.ReLU(), nn.MaxPool2d(2),\n nn.Conv2d(16, 32, 5, padding=2), nn.ReLU(), nn.AdaptiveMaxPool2d(1),\n )\n self.fc = nn.Linear(32, 4) # x0, y0, w, h, normalized\n\n def forward(self, x):\n return torch.sigmoid(self.fc(self.conv(x).flatten(1)))\n\ndef iou_batch(pred, target):\n px0, py0, pw, ph = pred[:, 0], pred[:, 1], pred[:, 2], pred[:, 3]\n tx0, ty0, tw, th = target[:, 0], target[:, 1], target[:, 2], target[:, 3]\n px1, py1, tx1, ty1 = px0 + pw, py0 + ph, tx0 + tw, ty0 + th\n ix0, iy0 = torch.maximum(px0, tx0), torch.maximum(py0, ty0)\n ix1, iy1 = torch.minimum(px1, tx1), torch.minimum(py1, ty1)\n inter = (ix1 - ix0).clamp(min=0) * (iy1 - iy0).clamp(min=0)\n union = pw * ph + tw * th - inter\n return inter / union.clamp(min=1e-8)\n\ntorch.manual_seed(0)\nmodel = Detector()\nopt = torch.optim.Adam(model.parameters(), lr=0.005)\nXt = torch.tensor(Xtr).unsqueeze(1); Bt = torch.tensor(Btr)\nfor _ in range(400):\n opt.zero_grad()\n loss = F.mse_loss(model(Xt), Bt)\n loss.backward()\n opt.step()\n\nwith torch.no_grad():\n pred_te = model(torch.tensor(Xte).unsqueeze(1))\n ious = iou_batch(pred_te, torch.tensor(Bte))\n\nprint(f'mean IoU on test set: {ious.mean().item():.3f}')\nprint(f'fraction of test boxes with IoU > 0.5: {(ious > 0.5).float().mean().item():.1%}')",
"metadata": {},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"id": "b84319ee",
"source": "fig, axes = plt.subplots(1, 4, figsize=(9, 2.5))\nfor i, ax in enumerate(axes):\n ax.imshow(Xte[i], cmap='gray')\n tb = Bte[i] * SIZE\n pb = pred_te[i].numpy() * SIZE\n ax.add_patch(patches.Rectangle((tb[0], tb[1]), tb[2], tb[3], edgecolor='lime', facecolor='none', linewidth=2, label='true'))\n ax.add_patch(patches.Rectangle((pb[0], pb[1]), pb[2], pb[3], edgecolor='red', facecolor='none', linewidth=1.5, linestyle='--', label='pred'))\n ax.set_title(f'IoU={ious[i]:.2f}', fontsize=9)\n ax.axis('off')\naxes[0].legend(fontsize=6, loc='upper left')\nplt.show()",
"metadata": {},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"id": "fc21329f",
"source": "## Scaling this up: two families of real detectors\n\nThis lesson's detector only handles exactly one object per image, because a fixed-size output vector (4 numbers) can only ever describe one box. Real scenes have a variable, unknown number of objects. Two different fixes became the two dominant families of detector:\n\n**Two-stage (R-CNN family):** first generate a modest number of *region proposals* — candidate boxes likely to contain something, via a cheap, class-agnostic method (the original R-CNN used classical segmentation; **Faster R-CNN** (Ren et al., 2015★) learns a small \"region proposal network\" instead) — then run a classifier-plus-box-regressor (this lesson's whole architecture) on each proposal independently, exactly like running the sliding-window classifier from Lesson 39 but only at a handful of promising locations instead of every window. Accurate, but only as fast as (proposals) x (one forward pass) allows.\n\n**Single-stage (YOLO, SSD):** skip proposals entirely. **YOLO** (You Only Look Once, Redmon et al., 2016★) divides the image into a coarse grid of cells, and has each grid cell directly predict (as this lesson's network does) a fixed number of boxes plus a class label plus a confidence score, all in one forward pass. To let a single cell describe objects of different aspect ratios, single-stage detectors use **anchor boxes**: several predefined box shapes (tall, wide, square) per cell, with the network predicting an *offset* from each anchor rather than a box from scratch. Faster to run, historically somewhat less accurate than two-stage methods, though the gap has narrowed considerably.\n\nBoth families end with the same postprocessing step this lesson skipped by only ever predicting one box: NMS (Lesson 39) to merge the overlapping candidate boxes any real multi-object scene produces.\n\n### Exercise\n\n1. Change `obj_size` in `make_scene` from a fixed `8` to a random value (e.g. `rng.integers(4, 12)`) so objects vary in size, and retrain. Does mean IoU hold up, get worse, or barely change — and why would variable object scale be harder for a single fixed-size regression head than variable position?\n2. The loss function here is plain MSE on `(x0, y0, w, h)`, but the metric that matters is IoU. Replace the loss with `1 - iou_batch(pred, target).mean()` (directly optimizing IoU) and compare final mean test IoU to the MSE-trained version. Real detectors (e.g. Faster R-CNN, YOLO variants) do exactly this with generalized IoU losses — can you see why MSE loss and IoU metric might disagree on which of two similar predictions is \"better\"?\n3. This lesson's detector has no anchor boxes and only ever handles one object. Sketch (in words) how you would modify the architecture's *output* to handle up to 3 objects per image using a 3x3 grid where each grid cell predicts one box and a \"there's an object here\" confidence — the core idea behind YOLO's grid.",
"metadata": {}
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}