{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# Lesson 38: Visualizing and Understanding CNNs\n\nA trained CNN is a black box in the sense that its millions of weights don't have obvious individual meanings. But *where in the input image* a prediction comes from is answerable, and answering it is often what separates \"the model got the right answer\" from \"the model got the right answer for the right reason.\" This lesson builds two visualization tools from scratch: **saliency maps** (<a href=\"../references.html#simonyan-2013-saliency\">Simonyan et al., 2013</a>) and **Grad-CAM** (<a href=\"../references.html#selvaraju-2017\">Selvaraju et al., 2017</a>)."
  },
  {
   "cell_type": "code",
   "id": "9e84d27c",
   "source": "import numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport matplotlib.pyplot as plt",
   "metadata": {},
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "54fe351b",
   "source": "## Setup: the plus-vs-circle CNN from Lesson 33, with feature maps exposed\n\nThe only change from Lesson 33's `CNNClassifier` is that `forward` now also returns the last convolutional layer's feature map (before global pooling), so both visualization methods below can get at it.",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "654c4b97",
   "source": "def make_image(shape_type, cx, cy, size=16):\n    img = np.zeros((size, size), dtype=np.float32)\n    if shape_type == 'plus':\n        img[cy-1:cy+2, cx-3:cx+4] = 1.0\n        img[cy-3:cy+4, cx-1:cx+2] = 1.0\n    else:\n        yy, xx = np.mgrid[0:size, 0:size]\n        img[((xx-cx)**2 + (yy-cy)**2) <= 9] = 1.0\n    return img\n\ndef make_dataset(rng_local, n, position_range=(4, 12)):\n    imgs, labels, positions = [], [], []\n    for _ in range(n):\n        shape_type = rng_local.choice(['plus', 'circle'])\n        cx, cy = rng_local.integers(*position_range), rng_local.integers(*position_range)\n        imgs.append(make_image(shape_type, cx, cy))\n        labels.append(0.0 if shape_type == 'plus' else 1.0)\n        positions.append((cx, cy))\n    return np.array(imgs, dtype=np.float32), np.array(labels, dtype=np.float32), positions\n\nrng = np.random.default_rng(4)\nX_train, y_train, _ = make_dataset(rng, 300)\nX_test, y_test, pos_test = make_dataset(rng, 50)\n\nclass CNN(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv1 = nn.Conv2d(1, 8, 5, padding=2)\n        self.conv2 = nn.Conv2d(8, 16, 5, padding=2)\n        self.gpool = nn.AdaptiveMaxPool2d(1)\n        self.fc = nn.Linear(16, 1)\n\n    def forward(self, x):\n        f1 = F.relu(self.conv1(x))\n        f2 = F.relu(self.conv2(f1))  # last conv feature map, full 16x16 resolution\n        feat = self.gpool(f2).flatten(1)\n        return self.fc(feat).squeeze(-1), f2\n\ntorch.manual_seed(0)\nmodel = CNN()\nopt = torch.optim.Adam(model.parameters(), lr=0.01)\nXt = torch.tensor(X_train).unsqueeze(1); yt = torch.tensor(y_train)\nfor _ in range(300):\n    opt.zero_grad()\n    out, _ = model(Xt)\n    loss = F.binary_cross_entropy_with_logits(out, yt)\n    loss.backward()\n    opt.step()\n\nwith torch.no_grad():\n    out, _ = model(torch.tensor(X_test).unsqueeze(1))\n    acc = ((out > 0).float() == torch.tensor(y_test)).float().mean().item()\nprint(f'test accuracy: {acc:.1%}')",
   "metadata": {},
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "4c3ef306",
   "source": "## Saliency maps\n\nThe idea (<a href=\"../references.html#simonyan-2013-saliency\">Simonyan et al., 2013</a>): take the gradient of the predicted class *score* with respect to every input pixel. A pixel with a large-magnitude gradient is one where a small change would most change the prediction — i.e., a pixel the network is \"looking at.\"",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "7061eacf",
   "source": "def saliency_map(model, img):\n    x = torch.tensor(img[None, None]).float()\n    x.requires_grad_(True)\n    score, feat = model(x)\n    score.backward()\n    return x.grad[0, 0].abs().numpy(), feat\n\nidx = 3\nsaliency, _ = saliency_map(model, X_test[idx])\ncx, cy = pos_test[idx]\n\nfig, axes = plt.subplots(1, 2, figsize=(7, 3.2))\naxes[0].imshow(X_test[idx], cmap='gray')\naxes[0].scatter([cx], [cy], c='red', marker='x', s=60, label='true center')\naxes[0].set_title('input image'); axes[0].legend(fontsize=7); axes[0].axis('off')\naxes[1].imshow(saliency, cmap='hot')\naxes[1].set_title('saliency map')\naxes[1].axis('off')\nplt.show()",
   "metadata": {},
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "9b441f5c",
   "source": "## Grad-CAM\n\nRaw saliency maps are pixel-level and tend to be noisy. **Grad-CAM** (<a href=\"../references.html#selvaraju-2017\">Selvaraju et al., 2017</a>) instead works on the last convolutional layer's feature maps, which are lower-resolution but far more semantically meaningful: \n\n1. Take the gradient of the class score with respect to each channel of the last conv feature map.\n2. Average each channel's gradient spatially to get one importance weight per channel.\n3. Form a weighted sum of the feature-map channels using those weights, then apply ReLU (only positive evidence for the class matters).\n\nThe result is a coarse heatmap, the same spatial size as the last conv layer, that can be upsampled back to the input resolution.",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "511bcc79",
   "source": "def grad_cam(model, img, out_size=16):\n    x = torch.tensor(img[None, None]).float()\n    x.requires_grad_(True)\n    score, feat = model(x)\n    feat.retain_grad()\n    score.backward()\n    weights = feat.grad[0].mean(dim=(1, 2))  # (channels,) importance per channel\n    cam = F.relu((weights[:, None, None] * feat[0]).sum(dim=0))\n    cam_up = F.interpolate(cam[None, None], size=(out_size, out_size), mode='bilinear', align_corners=False)\n    return cam_up[0, 0].detach().numpy()\n\ncam = grad_cam(model, X_test[idx])\n\nfig, axes = plt.subplots(1, 3, figsize=(10, 3.2))\naxes[0].imshow(X_test[idx], cmap='gray')\naxes[0].scatter([cx], [cy], c='red', marker='x', s=60)\naxes[0].set_title('input image'); axes[0].axis('off')\naxes[1].imshow(saliency, cmap='hot')\naxes[1].set_title('saliency map'); axes[1].axis('off')\naxes[2].imshow(cam, cmap='hot')\naxes[2].set_title('Grad-CAM'); axes[2].axis('off')\nplt.show()",
   "metadata": {},
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "7e83ec59",
   "source": "## Do these maps actually point at the shape?\n\nA single example is a nice picture but not evidence. Check quantitatively: for every test image, find each map's peak pixel and measure its distance to the shape's true center, and compare against the distance a *random* guess would get.",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "dc05d65a",
   "source": "sal_dists, cam_dists, rand_dists = [], [], []\nrand_rng = np.random.default_rng(0)\nfor i in range(len(X_test)):\n    sal, _ = saliency_map(model, X_test[i])\n    cam_i = grad_cam(model, X_test[i])\n    cx_i, cy_i = pos_test[i]\n\n    peak_sal = np.unravel_index(sal.argmax(), sal.shape)       # (row, col) = (y, x)\n    peak_cam = np.unravel_index(cam_i.argmax(), cam_i.shape)\n    sal_dists.append(np.hypot(peak_sal[1] - cx_i, peak_sal[0] - cy_i))\n    cam_dists.append(np.hypot(peak_cam[1] - cx_i, peak_cam[0] - cy_i))\n\n    rx, ry = rand_rng.integers(0, 16), rand_rng.integers(0, 16)\n    rand_dists.append(np.hypot(rx - cx_i, ry - cy_i))\n\nprint(f'{\"method\":>18} {\"mean peak distance to true center\":>36}')\nprint(f'{\"saliency map\":>18} {np.mean(sal_dists):>33.2f} px')\nprint(f'{\"Grad-CAM\":>18} {np.mean(cam_dists):>33.2f} px')\nprint(f'{\"random baseline\":>18} {np.mean(rand_dists):>33.2f} px')",
   "metadata": {},
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "08743d8a",
   "source": "Both methods land far closer to the true shape center than a random guess would, confirming they're picking out genuinely relevant image regions rather than something spurious — and here, with full spatial resolution preserved in the last conv layer, Grad-CAM is actually the more precise of the two.\n\nIn practice these tools matter most for catching a specific failure: a network that gets the right answer on a training or validation set for the *wrong* reason — for example, learning to recognize a background watermark that happened to correlate with one class, rather than the object itself. A saliency map or Grad-CAM overlay on such a network would show it \"looking\" at the watermark, not the object — a bug that overall accuracy alone would never reveal (echoing Lesson 37's point that a single accuracy number hides a lot).\n\n### Exercise\n\n1. Grad-CAM here uses the *last* conv layer. Modify `grad_cam` to instead use the intermediate feature map after `conv1` (before `conv2`). Does the resulting heatmap get sharper (closer to pixel-perfect, like the saliency map) or coarser, and why would an earlier layer behave that way?\n2. Deliberately corrupt the dataset: add a small fixed bright square in the same corner of *every* \"plus\" training image (but no circle images), retrain, and check whether Grad-CAM on a test plus image lights up on the shape or on the corrupted corner.\n3. The saliency-map gradient in this lesson is taken with respect to the raw logit (`score`), not the sigmoid probability. Try computing it with respect to `torch.sigmoid(score)` instead — does the resulting map look meaningfully different, and can you explain why using the chain rule?",
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}