{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Sampling and Spatial Resolution\n", "stough 202-\n", "\n", "One of the core properties of any digital image is its *resolution*. But this is a very loaded word in that it is used in a variety of ways for different purposes. Very generally, the light from the [external world](https://en.wikipedia.org/wiki/Solipsism) comes at our digital camera from a virtually uncountably infinite number of directions, with each direction specifying a \"color\" among a virtually uncountably infinite number of color distributions and intensities. Resolution broadly refers to the finite sampling of this infinite world. In the case of *spatial resolution* that is simply the number of *pixels* or \"picture elements\" that our camera can possibly obtain (see \"internal image plane\" below). Beyond spatial resolution, there is also [color resolution](./color_quantization.ipynb), which we get to elsewhere. \n", "\n", "\n", "\n", " \n", "\n", "In this notebook we're going to look at the spatial resolution of a digital image." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Imports\n", "We'll use the [`rescale`](https://scikit-image.org/docs/stable/api/skimage.transform.html#skimage.transform.rescale) method in the powerful [`scikit-image`](https://scikit-image.org/) module in order to change image resolution. We'll also be using some widgets to put together an [interactive visualization](../NumpyAndVisualization/interactive_vis.ipynb). Lastly, we're starting to import common functionality from our own `../dip_utils/` utilities folder. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%matplotlib widget\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "from skimage.transform import rescale\n", "from ipywidgets import VBox, IntSlider, AppLayout\n", "\n", "# For importing from alternative directory sources\n", "import sys \n", "sys.path.insert(0, '../dip_utils')\n", "\n", "# Importing from our own utilities\n", "from matrix_utils import arr_info\n", "from vis_utils import vis_image, vis_pair" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "I = plt.imread('../dip_pics/grandCanyon.jpg')\n", "arr_info(I)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "vis_image(I, figsize=(5,3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see the image matrix `I` is 2160 rows by 3840 columns, with three color channels (red, green, blue) in the integer range $[0,255]$. By the way this is literally a \"4K\" image.\n", "\n", "- \"4K\" refers to the approximate number of columns, or the left-right sampling of the image. At an aspect ratio of 16(width, columns):9(height, rows), a \"1080\" high definition image or screen is 1920 pixels across, about 2K. So a 4K is a 2x2 grid of such 2K screens. It has 4 times the number of pixels in a 2K (or 1080). It's more [complicated](https://www.cnet.com/news/4k-1080p-2k-uhd-8k-tv-resolutions-explained/) than that, but not by much.\n", "\n", "If we think of an image as a grid of pixels, where the center of each pixel represents that pixel's coordinates, then changing the resolution of an image involves resampling the space between those pixel coordinates. We'll avoid most of the math here by relying on `skimage`, but take a very simple example: If we wanted to halve the number of pixels both in width and height without cropping, we could just take the average of every 2x2 pixels. Similarly, if we wanted to double the number of pixels in both directions, we could simply make a 2x2 grid of copies of each pixel. \n", "\n", "We'll start by doing this halving of resolution. We'll first ask [`rescale`](https://scikit-image.org/docs/dev/api/skimage.transform.html?highlight=rescale#rescale) to give us an image with 1/4 the number of pixels. But then we'll *upsample* that image by four. The `order` and `anti-aliasing` parameters are used to define the level of \"smoothing\" that takes place, i.e. how many neighboring pixels to consider when deciding the color of the new sample coordinate." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# The image is so large, take a tiny piece of it to test the order, anti-aliasing parameters.\n", "# I = I[800:930, 1900:2050, :]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sI = rescale(I, .25,\n", " order=0, # use bilinear sampling\n", " anti_aliasing=False,\n", " channel_axis=-1)\n", "\n", "# rescale back up\n", "reI = rescale(sI, 4,\n", " order=0, # use nearest neighbor sampling\n", " anti_aliasing=False,\n", " channel_axis=-1)\n", "\n", "vis_pair(I, reI, first_title='Original', second_title='Quarter Res')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you're displaying this notebook in anything less than 8K full screen it's likely that the original and quarter-resolution images look identical. Use the zoom function to look more closely at particular areas to observe the difference. As you zoom in you should start to see the pixels that make up both images, and you will see the difference in spatial resolution. In this display, the reason I upsample the just downsampled image is so that the display could `sharex` and `sharey`.\n", "\n", "Below is an interactive visualization that allows you to select the degree of downsampling in the right-hand image. First, we'll define a quick function that does the resampling." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def downandup(I, factor):\n", " # downscale\n", " sI = rescale(I, 1/factor,\n", " order=0, # use nearest neighbor\n", " anti_aliasing=False,\n", " channel_axis=-1)\n", "\n", " # rescale back up\n", " reI = rescale(sI, factor,\n", " order=0, # use nearest neighbor sampling\n", " anti_aliasing=False,\n", " channel_axis=-1)\n", " return reI" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "plt.ioff()\n", "\n", "# Create a slider object\n", "slider = IntSlider(\n", " orientation='horizontal',\n", " description='Exponent:',\n", " value=0,\n", " min=0,\n", " max=10\n", ")\n", "\n", "slider.layout.margin = '0px 30% 0px 30%'\n", "slider.layout.width = '40%'\n", "\n", "\n", "f, ax = plt.subplots(1,2, figsize=(10,3), sharex=True, sharey=True)\n", "f.canvas.header_visible = False\n", "ax[0].imshow(I, interpolation='none')\n", "ax[0].set_title('Original')\n", "\n", "rdisp = ax[1].imshow(I, interpolation='none')\n", "rtext = ax[1].set_title(rf'Factor: $2^{slider.value:d}$, {2**slider.value:d}')\n", "\n", "\n", "# A function that will be called whenever the slider changes.\n", "def update(change):\n", " Ire = downandup(I, factor=2**slider.value)\n", " rdisp.set_data(Ire)\n", " rtext.set_text(rf'Factor: $2^{slider.value:d}$, {2**slider.value:d}')\n", " f.canvas.draw()\n", " f.canvas.flush_events()\n", "\n", "# Connecting the slider object to the update function above.\n", "# This is event-handling.\n", "slider.observe(update, names='value')\n", "\n", "plt.tight_layout()\n", "# Creates an application interface with the various \n", "# pieces we already instantiated inside of it. \n", "AppLayout(\n", " center=f.canvas,\n", " footer=slider,\n", " pane_heights=[0, 6, 1]\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that I'm using the `slider.value` as an exponent to give a little more effect. Think of the printed \"Factor\" ($2^{slider.value}$) as the number of pixels to combine in each dimension. So at an exponent of 3, we're scaling the image by 8 in each dimension; i.e., each pixel in the downsampled image represents 64 pixels in the original.\n", "\n", "Note you could also modify the parameters we send to `rescale`, such as turning on anti-aliasing, and seeing what that does to the output. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.11" } }, "nbformat": 4, "nbformat_minor": 4 }