{ "cells": [ { "cell_type": "markdown", "id": "61d027b0", "metadata": {}, "source": [ "# Lesson 26: Image Formation\n", "\n", "Up to now we have been processing images without thinking about where they come from. This lesson unpacks that mystery by considering: **geometry** (how the projection matrix is derived from a pinhole camera model), **optics** (what a real lens does that a pinhole camera fails to model — focus and blur), and **sensing/color** (how a real sensor captures only one color per pixel and needs to reconstruct the rest, along with the two classic pitfalls — gamma and white balance — that trip up naive image processing)." ] }, { "cell_type": "code", "execution_count": null, "id": "52c56bd8", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import cv2\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "id": "1267a998", "metadata": {}, "source": [ "## The pinhole projection matrix, derived\n", "\n", "A pinhole camera only lets through rays of light passing through a particular point (the *center of projection*). By similar triangles, a 3D point $(X, Y, Z)$ (camera-centered coordinates) lands on the image plane at\n", "\n", "$$x = f\\frac{X}{Z}, \\qquad y = f\\frac{Y}{Z}$$\n", "\n", "where $f$ is the distance from the pinhole to the image plane. Converting to pixel units (accounting for pixel size and the origin location) requires the **intrinsics** $K$; and incorporating the camera's position and orientation relative to the world requires the **extrinsics** $[R | t]$. Put together, in homogeneous coordinates:\n", "\n", "$$\\underbrace{\\begin{bmatrix}u\\\\v\\\\1\\end{bmatrix}}_{\\text{pixel}} \\;\\propto\\; \\underbrace{\\begin{bmatrix}f_x&0&c_x\\\\0&f_y&c_y\\\\0&0&1\\end{bmatrix}}_{K}\\underbrace{\\begin{bmatrix}R & t\\end{bmatrix}}_{\\text{extrinsics}}\\begin{bmatrix}X\\\\Y\\\\Z\\\\1\\end{bmatrix}_{\\text{world}}$$\n", "\n", "This is exactly the $P = K [R | t]$ that Lessons 27-29 (Camera Calibration, Epipolar Geometry, and Structure from Motion) will use. Every symbol can be traced back to a physical cause: $f_x, f_y$ to focal length and pixel size; $(c_x, c_y)$ to where the optical axis hits the sensor (known as the *principal point*); $R, t$ to the camera's pose." ] }, { "cell_type": "code", "execution_count": null, "id": "375d4c1c", "metadata": {}, "outputs": [], "source": [ "K = np.array([[500, 0, 320], [0, 500, 240], [0, 0, 1]], dtype=np.float64)\n", "R, t = np.eye(3), np.array([0, 0, 5.0])\n", "P = K @ np.hstack([R, t.reshape(3, 1)])\n", "\n", "cube_corners = np.array([[x, y, z] for x in (-1, 1) for y in (-1, 1) for z in (-1, 1)], dtype=np.float64)\n", "homogeneous = np.hstack([cube_corners, np.ones((8, 1))])\n", "projected = (P @ homogeneous.T).T\n", "pixels = projected[:, :2] / projected[:, 2:3]\n", "\n", "edges = [(0, 1), (0, 2), (0, 4), (1, 3), (1, 5), (2, 3), (2, 6),\n", " (3, 7), (4, 5), (4, 6), (5, 7), (6, 7)]\n", "\n", "plt.figure(figsize=(5, 4))\n", "for i, j in edges:\n", " plt.plot([pixels[i, 0], pixels[j, 0]], [pixels[i, 1], pixels[j, 1]], color='tab:blue')\n", "plt.scatter(*pixels.T, color='red', zorder=5)\n", "plt.gca().invert_yaxis()\n", "plt.title('A 3D cube, projected through P = K[R|t]')\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "2ae15d4a", "metadata": {}, "source": [ "## Real lenses: focus and defocus blur\n", "\n", "A pinhole camera lets in so little light that it needs absurdly long exposures. Therefore, real cameras use a lens to gather much more light. Lenses, however, focus sharply only for objects at a particular distance (the **focal plane**). Points nearer or farther than that spread their light over a small disk on the sensor (the **circle of confusion**) instead of at a point — exactly the blurring convolution kernels from Lessons 9-10, just applied with a kernel size that depends on depth." ] }, { "cell_type": "code", "execution_count": null, "id": "c15723f5", "metadata": {}, "outputs": [], "source": [ "scene = np.zeros((200, 300, 3), dtype=np.uint8)\n", "cv2.rectangle(scene, (20, 20), (90, 180), (255, 120, 30), -1) # near object\n", "cv2.rectangle(scene, (110, 20), (190, 180), (30, 200, 255), -1) # object at the focal plane\n", "cv2.rectangle(scene, (210, 20), (280, 180), (120, 255, 60), -1) # far object\n", "\n", "depths_m = {'near': 2.0, 'mid': 5.0, 'far': 9.0}\n", "focus_depth_m = 5.0\n", "\n", "def defocus_kernel_size(depth, focus, strength=15.0):\n", " radius = strength * abs(depth - focus) / focus\n", " ksize = int(radius) * 2 + 1\n", " return max(1, ksize)\n", "\n", "def blur_with_context(image, y0, y1, x0, x1, ksize, pad):\n", " \"\"\"Blur a region using a padded crop, so the blur can pull in real neighboring\n", " pixels (including the background) instead of only the region's own solid color.\"\"\"\n", " if ksize <= 1:\n", " return image[y0:y1, x0:x1].copy()\n", " yy0, yy1 = max(0, y0 - pad), min(image.shape[0], y1 + pad)\n", " xx0, xx1 = max(0, x0 - pad), min(image.shape[1], x1 + pad)\n", " blurred = cv2.GaussianBlur(image[yy0:yy1, xx0:xx1], (ksize, ksize), 0)\n", " return blurred[y0 - yy0:y0 - yy0 + (y1 - y0), x0 - xx0:x0 - xx0 + (x1 - x0)]\n", "\n", "defocused = scene.copy()\n", "regions = {'near': (20, 180, 20, 90), 'mid': (20, 180, 110, 190), 'far': (20, 180, 210, 280)}\n", "for name, (y0, y1, x0, x1) in regions.items():\n", " k = defocus_kernel_size(depths_m[name], focus_depth_m)\n", " print(f'{name:>5} object: depth={depths_m[name]}m, blur kernel size={k}')\n", " defocused[y0:y1, x0:x1] = blur_with_context(scene, y0, y1, x0, x1, k, pad=k)\n", "\n", "fig, axes = plt.subplots(1, 2, figsize=(9, 3.5))\n", "axes[0].imshow(cv2.cvtColor(scene, cv2.COLOR_BGR2RGB))\n", "axes[0].set_title('All in focus (pinhole idealization)')\n", "axes[1].imshow(cv2.cvtColor(defocused, cv2.COLOR_BGR2RGB))\n", "axes[1].set_title(f'Focused at {focus_depth_m}m\\n(near/far objects blurred)')\n", "for ax in axes:\n", " ax.axis('off')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "73b046c2", "metadata": {}, "source": [ "This explains why a wider aperture gathers more light (at the expense of a shallower depth of field), while a narrow aperture produces a sharp image (but lets in less light) — the classic photographic aperture/depth-of-field tradeoff. This is the reason portrait photos often blur the background while keeping the subject sharp." ] }, { "cell_type": "markdown", "id": "2783ccbb", "metadata": {}, "source": [ "## The Bayer color filter array: one color per pixel\n", "\n", "To keep costs down, most camera sensors don't measure red, green, and blue separately at every pixel. Instead, a single monochrome photosensor array sits under a mosaic of tiny color filters, so each individual pixel physically records only *one* of the three colors. The most common arrangement (RGGB) — the **Bayer filter** (Bayer, 1976) — repeats in a 2x2 tile pattern: one red, one blue, and *two* green filters, since the human eye is most sensitive to green. Every pixel in the raw sensor output is therefore missing two-thirds of its color information by construction." ] }, { "cell_type": "code", "execution_count": null, "id": "b67bb0b3", "metadata": {}, "outputs": [], "source": [ "rng = np.random.default_rng(0)\n", "bayer_scene = np.zeros((120, 160, 3), dtype=np.uint8)\n", "cv2.rectangle(bayer_scene, (10, 10), (70, 100), (40, 180, 220), -1)\n", "cv2.circle(bayer_scene, (110, 60), 40, (200, 90, 40), -1)\n", "bayer_scene = np.clip(bayer_scene.astype(np.float64) + rng.normal(0, 4, bayer_scene.shape), 0, 255).astype(np.uint8)\n", "\n", "def make_bayer_rggb(img):\n", " # (row, col) parity: (even,even)=R, (even,odd)=G, (odd,even)=G, (odd,odd)=B\n", " H, W = img.shape[:2]\n", " mosaic = np.zeros((H, W), dtype=np.uint8)\n", " mosaic[0::2, 0::2] = img[0::2, 0::2, 2] # R (img is BGR, so channel 2 = R)\n", " mosaic[0::2, 1::2] = img[0::2, 1::2, 1] # G\n", " mosaic[1::2, 0::2] = img[1::2, 0::2, 1] # G\n", " mosaic[1::2, 1::2] = img[1::2, 1::2, 0] # B\n", " return mosaic\n", "\n", "def colorize_bayer_rggb(mosaic):\n", " \"\"\"Put each pixel's raw reading into its OWN filter's color channel, zeroing the\n", " other two -- this is what actually lets you see the RGGB pattern, since a plain\n", " grayscale view of the mosaic just shows intensities with no color information at all.\"\"\"\n", " H, W = mosaic.shape\n", " colored = np.zeros((H, W, 3), dtype=np.uint8)\n", " colored[0::2, 0::2, 0] = mosaic[0::2, 0::2] # R filter\n", " colored[0::2, 1::2, 1] = mosaic[0::2, 1::2] # G filter\n", " colored[1::2, 0::2, 1] = mosaic[1::2, 0::2] # G filter\n", " colored[1::2, 1::2, 2] = mosaic[1::2, 1::2] # B filter\n", " return colored\n", "\n", "bayer = make_bayer_rggb(bayer_scene)\n", "bayer_colored = colorize_bayer_rggb(bayer)\n", "\n", "y0, y1, x0, x1 = 40, 56, 20, 36 # a 16x16 crop, fully inside a colored shape\n", "\n", "fig, axes = plt.subplots(1, 3, figsize=(11, 3.5))\n", "axes[0].imshow(cv2.cvtColor(bayer_scene, cv2.COLOR_BGR2RGB))\n", "axes[0].add_patch(plt.Rectangle((x0, y0), x1 - x0, y1 - y0, edgecolor='white', facecolor='none', linewidth=1.5))\n", "axes[0].set_title('true scene (each pixel: R, G, and B)')\n", "axes[1].imshow(bayer[y0:y1, x0:x1], cmap='gray')\n", "axes[1].set_title('raw mosaic\\n(grayscale readout -- no color info visible)')\n", "axes[2].imshow(bayer_colored[y0:y1, x0:x1])\n", "axes[2].set_title('same patch, colorized by filter\\n(the actual RGGB pattern)')\n", "for ax in axes:\n", " ax.axis('off')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "aed569ab", "metadata": {}, "source": [ "### Demosaicking: reconstructing RGB from the mosaic\n", "\n", "**Demosaicking** (or \"debayering\") fills in each pixel's two missing colors by interpolating from same-color neighbors — the simplest version is bilinear: average the nearby known samples of a color, weighted by distance, exactly Lesson 9's bilinear interpolation, just interpolating across a sparse, patterned grid of known samples instead of a dense downsampled image." ] }, { "cell_type": "code", "execution_count": null, "id": "eef25d6d", "metadata": {}, "outputs": [], "source": [ "def demosaick_bilinear_rggb(mosaic):\n", " H, W = mosaic.shape\n", " R = np.zeros((H, W)); G = np.zeros((H, W)); B = np.zeros((H, W))\n", " R[0::2, 0::2] = mosaic[0::2, 0::2]\n", " G[0::2, 1::2] = mosaic[0::2, 1::2]; G[1::2, 0::2] = mosaic[1::2, 0::2]\n", " B[1::2, 1::2] = mosaic[1::2, 1::2]\n", "\n", " Rmask = np.zeros((H, W)); Rmask[0::2, 0::2] = 1\n", " Gmask = np.zeros((H, W)); Gmask[0::2, 1::2] = 1; Gmask[1::2, 0::2] = 1\n", " Bmask = np.zeros((H, W)); Bmask[1::2, 1::2] = 1\n", "\n", " def fill(channel, mask):\n", " # weighted average of known same-color neighbors within a 3x3 window\n", " kernel = np.array([[0.25, 0.5, 0.25], [0.5, 1.0, 0.5], [0.25, 0.5, 0.25]])\n", " weight_sum = cv2.filter2D(mask, -1, kernel, borderType=cv2.BORDER_REFLECT)\n", " value_sum = cv2.filter2D(channel * mask, -1, kernel, borderType=cv2.BORDER_REFLECT)\n", " filled = channel.copy()\n", " empty = mask == 0\n", " filled[empty] = value_sum[empty] / np.clip(weight_sum[empty], 1e-6, None)\n", " return filled\n", "\n", " R_full, G_full, B_full = fill(R, Rmask), fill(G, Gmask), fill(B, Bmask)\n", " return np.clip(np.stack([B_full, G_full, R_full], axis=-1), 0, 255).astype(np.uint8)\n", "\n", "manual_demosaick = demosaick_bilinear_rggb(bayer)\n", "cv2_demosaick = cv2.cvtColor(bayer, cv2.COLOR_BayerBG2BGR)\n", "\n", "manual_err = np.abs(manual_demosaick.astype(np.float64) - bayer_scene.astype(np.float64)).mean()\n", "cv2_err = np.abs(cv2_demosaick.astype(np.float64) - bayer_scene.astype(np.float64)).mean()\n", "print(f'manual bilinear demosaick MAE vs ground truth: {manual_err:.2f}')\n", "print(f'cv2 demosaick MAE vs ground truth: {cv2_err:.2f}')\n", "\n", "fig, axes = plt.subplots(1, 3, figsize=(9, 3.5))\n", "for ax, im, title in zip(axes, [bayer_scene, manual_demosaick, cv2_demosaick],\n", " ['ground truth', 'manual bilinear\\ndemosaick', 'cv2 demosaick']):\n", " ax.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))\n", " ax.set_title(title, fontsize=9)\n", " ax.axis('off')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "67ab4eb1", "metadata": {}, "source": [ "Both methods reconstruct the ground truth closely (the residual error is mostly just the sensor noise already present in the mosaic, not a demosaicking artifact) — a from-scratch bilinear fill and OpenCV's built-in demosaicking land within a fraction of a pixel value of each other. Real demosaicking algorithms are more sophisticated than pure bilinear (edge-aware interpolation that avoids blurring across object boundaries, since naive bilinear demosaicking is exactly what causes the color-fringing \"zipper\" artifacts visible along sharp edges in cheap cameras), but the core idea — interpolate each missing color from its nearest same-color neighbors — is the same.\n", "\n", "### The pitfall: getting the mosaic pattern wrong\n", "\n", "The R/G/B arrangement matters, and there's more than one convention (RGGB, BGGR, GRBG, GBRG, depending on the sensor). Demosaicking with the *wrong* assumed pattern doesn't fail gracefully — it silently produces a plausible-looking but badly wrong-colored image, since every pixel gets a value, just the wrong one." ] }, { "cell_type": "code", "execution_count": null, "id": "471d4959", "metadata": {}, "outputs": [], "source": [ "wrong_demosaick = cv2.cvtColor(bayer, cv2.COLOR_BayerRG2BGR) # wrong pattern assumption\n", "wrong_err = np.abs(wrong_demosaick.astype(np.float64) - bayer_scene.astype(np.float64)).mean()\n", "print(f'correct pattern (BayerBG2BGR) MAE: {cv2_err:.2f}')\n", "print(f'WRONG pattern (BayerRG2BGR) MAE: {wrong_err:.2f}')\n", "\n", "fig, axes = plt.subplots(1, 2, figsize=(6, 3.5))\n", "axes[0].imshow(cv2.cvtColor(cv2_demosaick, cv2.COLOR_BGR2RGB)); axes[0].set_title('correct pattern', fontsize=9)\n", "axes[1].imshow(cv2.cvtColor(wrong_demosaick, cv2.COLOR_BGR2RGB)); axes[1].set_title('wrong pattern', fontsize=9)\n", "for ax in axes:\n", " ax.axis('off')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "96accf51", "metadata": {}, "source": [ "The error jumps by more than an order of magnitude if the wrong pattern is used — every red sample gets treated as blue and vice versa, producing a strongly color-shifted image. This is the practical reason RAW image pipelines always need to know a camera's exact sensor layout." ] }, { "cell_type": "markdown", "id": "96fecbeb", "metadata": {}, "source": [ "## Gamma correction: pixel values are not linear light\n", "\n", "Cameras and displays don't store brightness proportionally to physical light intensity (**linear radiance**). Instead, values are **gamma-encoded**: `encoded = linear ** (1/gamma)` (typically gamma ≈ 2.2, close to the sRGB standard), which allocates more encoded levels to darker tones — matching human vision's greater sensitivity to shadows than highlights, and historically matching how CRT displays responded to voltage. Decoding reverses it: `linear = encoded ** gamma`." ] }, { "cell_type": "code", "execution_count": null, "id": "f118999a", "metadata": {}, "outputs": [], "source": [ "def encode_gamma(linear, gamma=1 / 2.2):\n", " return np.clip(linear, 0, 1) ** gamma\n", "\n", "def decode_gamma(encoded, gamma=2.2):\n", " return np.clip(encoded, 0, 1) ** gamma\n", "\n", "linear_values = np.linspace(0, 1, 100)\n", "plt.plot(linear_values, linear_values, '--', color='gray', label='identity (no gamma)')\n", "plt.plot(linear_values, encode_gamma(linear_values), label='gamma-encoded (stored pixel value)')\n", "plt.xlabel('linear light intensity')\n", "plt.ylabel('encoded value')\n", "plt.legend(fontsize=8)\n", "plt.title('Gamma encoding curve')\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "aef09f74", "metadata": {}, "source": [ "### The pitfall: averaging encoded pixels is not averaging light\n", "\n", "Blending, resizing, and blurring (Lessons 8-10) all implicitly assume a linear quantity is being averaged. Pixel values usually aren't linear — they're gamma-encoded — so naively averaging two encoded pixel values gives the *wrong physical brightness*, sometimes badly so." ] }, { "cell_type": "code", "execution_count": null, "id": "d1d98534", "metadata": {}, "outputs": [], "source": [ "black_encoded, white_encoded = encode_gamma(0.0), encode_gamma(1.0)\n", "\n", "# WRONG: average the encoded (stored) pixel values directly\n", "naive_blend_encoded = (black_encoded + white_encoded) / 2\n", "naive_blend_as_linear_light = decode_gamma(naive_blend_encoded)\n", "\n", "# RIGHT: decode to linear light first, average there, then re-encode for storage/display\n", "correct_blend_linear = (decode_gamma(black_encoded) + decode_gamma(white_encoded)) / 2\n", "correct_blend_encoded = encode_gamma(correct_blend_linear)\n", "\n", "print(f'true 50% linear-light gray, correctly encoded: {correct_blend_encoded:.3f}')\n", "print(f'naive average of encoded black/white: {naive_blend_encoded:.3f}')\n", "print()\n", "print(f'naive result represents only {naive_blend_as_linear_light:.1%} of the true linear brightness '\n", " f'(should be 50%)')\n", "\n", "swatch = np.zeros((80, 240), dtype=np.float64)\n", "swatch[:, :80] = naive_blend_encoded\n", "swatch[:, 80:160] = correct_blend_encoded\n", "swatch[:, 160:] = 0.5\n", "plt.imshow(swatch, cmap='gray', vmin=0, vmax=1)\n", "plt.title('naive blend | correct blend | encoded 0.5 for reference')\n", "plt.axis('off')\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "88725330", "metadata": {}, "source": [ "The naive blend is visibly, dramatically darker than the physically correct 50%-gray result — a real and common bug when image-processing code (blurring, mipmap generation, alpha blending) operates directly on gamma-encoded pixel values instead of linearizing first." ] }, { "cell_type": "markdown", "id": "384f4ceb", "metadata": {}, "source": [ "## White balance: color depends on the light source\n", "\n", "A camera records the *product* of a surface's reflectance and the illuminant's color, not the surface's \"true\" color alone — a white sheet of paper looks orange under warm tungsten light and blue-ish under overcast sky, even though our visual system usually (mostly) compensates for this automatically (color constancy). Cameras have to do the same correction deliberately: **white balancing**." ] }, { "cell_type": "code", "execution_count": null, "id": "56be6a79", "metadata": {}, "outputs": [], "source": [ "rng = np.random.default_rng(0)\n", "true_scene = np.zeros((150, 150, 3), dtype=np.uint8)\n", "cv2.rectangle(true_scene, (20, 20), (130, 130), (150, 150, 150), -1) # a neutral gray card\n", "cv2.circle(true_scene, (75, 75), 30, (60, 120, 200), -1)\n", "true_scene = np.clip(true_scene.astype(np.float64) + rng.normal(0, 5, true_scene.shape), 0, 255).astype(np.uint8)\n", "\n", "tungsten_gains = np.array([1.3, 1.0, 0.6]) # BGR: boosts red, cuts blue -- a warm cast\n", "cast_scene = np.clip(true_scene.astype(np.float64) * tungsten_gains, 0, 255).astype(np.uint8)\n", "\n", "fig, axes = plt.subplots(1, 2, figsize=(6, 3.5))\n", "axes[0].imshow(cv2.cvtColor(true_scene, cv2.COLOR_BGR2RGB))\n", "axes[0].set_title('True colors')\n", "axes[1].imshow(cv2.cvtColor(cast_scene, cv2.COLOR_BGR2RGB))\n", "axes[1].set_title('Under warm tungsten light')\n", "for ax in axes:\n", " ax.axis('off')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "18087b89", "metadata": {}, "source": [ "### The gray-world assumption\n", "\n", "A simple, classic white-balancing algorithm assumes that, averaged over an entire real-world scene, colors roughly cancel out to neutral gray. Under that assumption, any systematic difference between a photo's per-channel averages reveals the illuminant's color cast — so rescaling each channel to equalize the averages should approximately undo it." ] }, { "cell_type": "code", "execution_count": null, "id": "8c2e046a", "metadata": {}, "outputs": [], "source": [ "def gray_world_white_balance(image):\n", " image_f = image.astype(np.float64)\n", " channel_means = image_f.reshape(-1, 3).mean(axis=0)\n", " target_gray = channel_means.mean()\n", " gains = target_gray / channel_means\n", " return np.clip(image_f * gains, 0, 255).astype(np.uint8)\n", "\n", "corrected_scene = gray_world_white_balance(cast_scene)\n", "\n", "error_before = np.abs(cast_scene.astype(int) - true_scene.astype(int)).mean()\n", "error_after = np.abs(corrected_scene.astype(int) - true_scene.astype(int)).mean()\n", "print(f'mean abs pixel error before correction: {error_before:.2f}')\n", "print(f'mean abs pixel error after correction: {error_after:.2f}')\n", "\n", "fig, axes = plt.subplots(1, 3, figsize=(9, 3.5))\n", "for ax, im, title in zip(axes, [true_scene, cast_scene, corrected_scene],\n", " ['True colors', 'Color cast', 'Gray-world corrected']):\n", " ax.imshow(cv2.cvtColor(im, cv2.COLOR_BGR2RGB))\n", " ax.set_title(title, fontsize=9)\n", " ax.axis('off')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "e773a4db", "metadata": {}, "source": [ "The correction substantially reduces the error, but not to zero — the gray-world assumption is only approximate here (this small scene isn't perfectly neutral on average, since it's dominated by the colored circle), which is exactly why real cameras often combine it with other cues (an actual detected gray/white reference patch, learned scene statistics, or user-specified presets) rather than relying on the gray-world assumption alone." ] }, { "cell_type": "markdown", "id": "957d3388", "metadata": {}, "source": [ "### Exercises\n", "\n", "1. Change `strength` in `defocus_kernel_size` to make the depth-of-field effect much shallower (larger `strength`) or much deeper (smaller `strength`). At `strength=0`, what should happen, and does the code produce that?\n", "2. Modify `make_bayer_rggb` and the `Rmask`/`Gmask`/`Bmask` assignments in `demosaick_bilinear_rggb` to build and decode a BGGR mosaic instead of RGGB (swap which corner is red vs. blue). Find the matching OpenCV code by trying each of `cv2.COLOR_BayerBG2BGR`, `cv2.COLOR_BayerGB2BGR`, `cv2.COLOR_BayerGR2BGR`, `cv2.COLOR_BayerRG2BGR` and checking which one drops the MAE back down to the noise floor — OpenCV's naming convention for these codes is notoriously easy to get backwards, which is itself a small demonstration of this lesson's point about silent pattern-mismatch errors.\n", "3. Repeat the gamma-blending experiment, but blend `0.2` and `0.8` (instead of pure black and white) in both the naive and correct ways. Is the discrepancy between them larger or smaller than the black/white case, and why might that make sense given the shape of the gamma curve?\n", "4. Modify the white-balance demo so the scene is dominated by a large *saturated red* region instead of a neutral gray card (i.e. make gray-world's core assumption clearly false). Does `gray_world_white_balance` still improve the result, make no difference, or actively make it worse?" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.x" } }, "nbformat": 4, "nbformat_minor": 5 }