{ "cells": [ { "cell_type": "markdown", "id": "99700935", "metadata": {}, "source": [ "# Lesson 11: Smoothing and Gaussian Pyramids\n", "\n", "Lesson 10 introduced Gaussian blur as one convolution kernel among several. Here we look at *why* blurring matters beyond just \"softening\" an image: it's the key ingredient that makes downsampling safe. That leads directly to the **Gaussian pyramid** — a stack of progressively smaller, blurrier versions of an image, used throughout computer vision for multi-scale analysis." ] }, { "cell_type": "code", "execution_count": null, "id": "2a456224", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import cv2\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "id": "8ca03b1b", "source": "## Why downsampling needs blurring first\n\nNaively shrinking an image by keeping every $k$-th pixel (\"nearest-neighbor downsampling\") can produce **aliasing**: fine periodic detail that oscillates faster than the new pixel spacing can represent folds into a completely different, fake low-frequency pattern.\n\nWe demonstrate with a photo of a brick wall — the repeating rows of bricks and mortar lines are exactly the kind of fine, regular detail that aliases badly.", "metadata": {} }, { "cell_type": "code", "id": "62da61e5", "source": "brick_wall = cv2.imread('../img/brick_wall.jpg', cv2.IMREAD_GRAYSCALE)\n\nfactor = 4\nnaive_downsample = brick_wall[::factor, ::factor] # subsample directly\nsafe_downsample = cv2.GaussianBlur(brick_wall, (0, 0), sigmaX=factor / 2)[::factor, ::factor] # blur first\n\nfig, axes = plt.subplots(1, 3, figsize=(10, 3.5))\naxes[0].imshow(brick_wall, cmap='gray')\naxes[0].set_title('Original')\naxes[1].imshow(naive_downsample, cmap='gray')\naxes[1].set_title('Naive: subsample only\\n(aliased, noisy-looking)')\naxes[2].imshow(safe_downsample, cmap='gray')\naxes[2].set_title('Blur, then subsample\\n(correctly soft)')\nfor ax in axes:\n ax.axis('off')\nplt.tight_layout()\nplt.show()", "metadata": {}, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "id": "8b920f3d", "source": "

Image source: publicdomainpictures.net

", "metadata": {} }, { "cell_type": "markdown", "id": "2fbba6f1", "source": "The naive version turns the regular brick pattern into fine, noise-like speckle — a classic aliasing artifact (the same effect that makes a car's wheels look like they're spinning backwards on camera). The blurred version stays recognizably a brick wall, because the mortar lines were smoothed away *before* subsampling, rather than being randomly kept or dropped pixel by pixel.", "metadata": {} }, { "cell_type": "markdown", "id": "5de3d67a", "source": "## Choosing sigma and kernel size\n\nBlurring correctly before downsampling needs two things to actually work: the right `sigma`, and a kernel *large enough to represent that sigma*. `cv2.GaussianBlur`'s `sigma` sets how far the bell-curve weighting spreads: larger sigma averages over a wider neighborhood, removing finer detail. Concretely, blurring a single pixel with a 3x3 Gaussian kernel is nothing more than a **weighted average of its 3x3 neighborhood**: multiply each of the 9 neighboring pixel values by the matching kernel weight and sum. `cv2.getGaussianKernel(3, sigma)` gives the 1D weights; their outer product gives the 2D kernel.", "metadata": {} }, { "cell_type": "code", "id": "482d3eb6", "source": "rng = np.random.default_rng(0)\nnoise_img = rng.integers(0, 255, (20, 20)).astype(np.uint8) # single-channel, for a simple example\n\nsigma = 1.0\nk1d = cv2.getGaussianKernel(3, sigma)\nkernel2d = k1d @ k1d.T\nprint('3x3 Gaussian kernel (sigma=1.0):\\n', np.round(kernel2d, 3))\nprint('weights sum to:', kernel2d.sum())\n\ny, x = 10, 10\nneighborhood = noise_img[y - 1:y + 2, x - 1:x + 2].astype(np.float64)\nmanual = (kernel2d * neighborhood).sum()\n\nblurred_full = cv2.GaussianBlur(noise_img, (3, 3), sigmaX=sigma)\n\nprint(f'\\n3x3 neighborhood around pixel ({x}, {y}):\\n{neighborhood}')\nprint(f'\\nweighted average (manual): {manual:.2f}')\nprint(f'cv2.GaussianBlur pixel: {blurred_full[y, x]}')", "metadata": {}, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "id": "3a67b5db", "source": "The manual weighted average matches `cv2.GaussianBlur`'s output for that pixel (up to integer rounding) — every blurred pixel really is just this 9-number dot product, repeated at every location in the image. Increasing sigma while keeping the kernel at 3x3 barely changes anything, though, because a 3x3 window only has room for 3 taps — to actually see a wider sigma's effect, the kernel has to grow too.", "metadata": {} }, { "cell_type": "code", "id": "a7e1e9c7", "source": "img = np.zeros((150, 150, 3), dtype=np.uint8)\ncv2.circle(img, (75, 75), 55, (255, 120, 30), -1)\ncv2.rectangle(img, (20, 20), (60, 60), (30, 200, 255), -1)\n\nsigmas = [0, 1, 3, 8]\nfig, axes = plt.subplots(1, len(sigmas), figsize=(12, 3.5))\nfor ax, s in zip(axes, sigmas):\n # ksize=(0, 0) tells OpenCV to pick a kernel size automatically, big enough for this sigma\n blurred = img if s == 0 else cv2.GaussianBlur(img, (0, 0), sigmaX=s)\n ax.imshow(blurred)\n ax.set_title(f'sigma = {s}')\n ax.axis('off')\nplt.tight_layout()\nplt.show()", "metadata": {}, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "id": "47dec8f1", "source": "### Kernel size has to keep up with sigma\n\n`ksize=(0, 0)` above told OpenCV to auto-size the kernel for each sigma. What happens if the kernel is fixed too small instead? `cv2.getGaussianKernel` always renormalizes its weights to sum to 1, no matter how few taps it's given — so a too-small kernel doesn't just clip the Gaussian's tails, it silently reshapes the whole kernel into something close to a plain box average, throwing away the bell-curve shape entirely.", "metadata": {} }, { "cell_type": "code", "id": "7ef7c537", "source": "sigma = 5.0\nk3 = cv2.getGaussianKernel(3, sigma).ravel()\nprint(f'a 3-tap kernel for sigma={sigma}, renormalized to sum to 1:', np.round(k3, 3))\nprint('(nearly identical to a plain 3-tap box average [0.33, 0.33, 0.33] -- the wide bell curve got squashed away)')\n\ntoo_small = cv2.GaussianBlur(img, (3, 3), sigmaX=sigma) # kernel far too small for this sigma\nauto = cv2.GaussianBlur(img, (0, 0), sigmaX=sigma) # OpenCV auto-sizes the kernel to fit sigma\n\nfig, axes = plt.subplots(1, 3, figsize=(9, 3.5))\nfor ax, im, title in zip(axes, [img, too_small, auto],\n ['Original', 'ksize=3\\n(too small for sigma=5)', 'ksize=0\\n(auto-sized for sigma=5)']):\n ax.imshow(im)\n ax.set_title(title, fontsize=9)\n ax.axis('off')\nplt.tight_layout()\nplt.show()\n\nprint(f'mean abs difference, too-small vs. correctly-sized kernel: {np.abs(too_small.astype(int) - auto.astype(int)).mean():.2f}')", "metadata": {}, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "id": "04234e5c", "source": "The two results are visibly different — `ksize=3` barely blurs the image at all, despite asking for `sigma=5`, since a 3-tap window simply can't represent a bell curve that wide. This is exactly why the pyramid code below (and the aliasing fix above) passes `ksize=(0, 0)`: letting OpenCV choose a kernel wide enough for the requested sigma, rather than risking a silently-too-small one.", "metadata": {} }, { "cell_type": "markdown", "id": "b6b5c026", "metadata": {}, "source": [ "## Building a Gaussian pyramid\n", "\n", "A **Gaussian pyramid** repeats \"blur, then downsample by 2\" over and over, producing a stack of images each half the width and height of the previous one. Each level is a properly anti-aliased, coarser view of the same scene — not just a smaller crop." ] }, { "cell_type": "code", "execution_count": null, "id": "26f34a1f", "metadata": {}, "outputs": [], "source": [ "def gaussian_pyramid(image, num_levels, sigma=1.0):\n", " pyramid = [image]\n", " current = image\n", " for _ in range(num_levels - 1):\n", " blurred = cv2.GaussianBlur(current, (0, 0), sigmaX=sigma)\n", " current = blurred[::2, ::2]\n", " pyramid.append(current)\n", " return pyramid\n", "\n", "photo = np.zeros((256, 256, 3), dtype=np.uint8)\n", "photo[:] = (40, 40, 40)\n", "cv2.circle(photo, (128, 128), 90, (255, 120, 30), -1)\n", "cv2.rectangle(photo, (30, 30), (110, 110), (30, 200, 255), -1)\n", "\n", "pyramid = gaussian_pyramid(photo, num_levels=5)\n", "\n", "fig, axes = plt.subplots(1, len(pyramid), figsize=(13, 3))\n", "for ax, level in zip(axes, pyramid):\n", " ax.imshow(level)\n", " ax.set_title(f'{level.shape[1]}x{level.shape[0]}', fontsize=9)\n", " ax.axis('off')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "c05b9fca", "metadata": {}, "source": [ "### Comparison to OpenCV's built-in `cv2.pyrDown`\n", "\n", "OpenCV provides `cv2.pyrDown`, which does the same blur-then-downsample idea using a fixed, carefully designed 5-tap binomial kernel (approximating a Gaussian) instead of an arbitrary `sigma`. The output sizes match ours exactly; pixel values are close but not identical, since the kernels differ slightly." ] }, { "cell_type": "code", "execution_count": null, "id": "0f7f0829", "metadata": {}, "outputs": [], "source": [ "cv_pyramid = [photo]\n", "current = photo\n", "for _ in range(4):\n", " current = cv2.pyrDown(current)\n", " cv_pyramid.append(current)\n", "\n", "for ours, cvs in zip(pyramid, cv_pyramid):\n", " assert ours.shape == cvs.shape\n", " diff = np.abs(ours.astype(int) - cvs.astype(int))\n", " print(f'{ours.shape[1]:>4}x{ours.shape[0]:<4} mean abs diff = {diff.mean():.2f}')" ] }, { "cell_type": "markdown", "id": "99344537", "metadata": {}, "source": [ "## Pyramids lose information\n", "\n", "Downsampling is not reversible: upsampling a lower pyramid level back to the original size (`cv2.pyrUp`) cannot recover detail that blurring/subsampling discarded. This gap between an upsampled coarse level and the original is exactly what a **Laplacian pyramid** captures at each level — a topic for a future lesson — but we can already see the information loss directly." ] }, { "cell_type": "code", "execution_count": null, "id": "8d599d12", "metadata": {}, "outputs": [], "source": [ "level1 = pyramid[1] # 128x128, one pyrDown from the original\n", "reconstructed = cv2.pyrUp(level1) # back up to 256x256\n", "\n", "diff = cv2.absdiff(photo, reconstructed)\n", "\n", "fig, axes = plt.subplots(1, 3, figsize=(9, 3.5))\n", "for ax, im, title in zip(axes, [photo, reconstructed, diff],\n", " ['Original', 'pyrDown then pyrUp', 'Difference (lost detail)']):\n", " ax.imshow(im)\n", " ax.set_title(title, fontsize=9)\n", " ax.axis('off')\n", "plt.tight_layout()\n", "plt.show()\n", "\n", "print(f'mean absolute reconstruction error = {diff.astype(np.float64).mean():.2f} gray levels')" ] }, { "cell_type": "markdown", "id": "a9cca866", "metadata": {}, "source": [ "The sharp edges of the circle and square come back soft and shifted-looking — the fine detail was permanently discarded when the image was blurred and subsampled, and `pyrUp` (interpolation) can only guess, not restore it." ] }, { "cell_type": "markdown", "id": "674259a6", "metadata": {}, "source": "### Exercise\n\n1. Repeat the brick-wall experiment with `factor = 2` instead of `4`. Does naive subsampling still show visible aliasing? Why might a smaller downsampling factor alias less?\n2. Gaussian pyramids are used for coarse-to-fine search (e.g. quickly finding an approximate object location at a small pyramid level, then refining at larger levels). Why would skipping the blur step and just subsampling directly break this strategy?\n3. Build a 6-level pyramid of a real photo-like image and measure how many levels it takes before the image becomes too small to recognize any shape at all. What real-world resolution would that correspond to for, say, a 1920x1080 photo?" } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.x" } }, "nbformat": 4, "nbformat_minor": 5 }