{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# N-dimensional image processing\n", "This tutorial covers how to access and manipulate image sample values directly using [ImgLib2](https://imagej.net/ImgLib2)'s primary image structures." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Added new repo: scijava.public\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "method": "display_data" }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9187a75e-394c-4293-aafc-76a53df0d192", "version_major": 2, "version_minor": 0 }, "method": "display_data" }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "ImageJ v2.0.0-rc-71 is ready to go." ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%classpath config resolver scijava.public https://maven.scijava.org/content/groups/public\n", "%classpath add mvn net.imagej imagej 2.0.0-rc-71\n", "ij = new net.imagej.ImageJ()\n", "\"ImageJ v${ij.getVersion()} is ready to go.\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Image samples" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Images in ImageJ are backed by data structures from the [ImgLib2](https://imagej.net/ImgLib2) library. There are several important interfaces, but the two most crucial to understand initially are `IterableInterval` and `RandomAccessibleInterval`.\n", "\n", "As you can infer from their names, both of them are `Interval`s, which means finite, discrete point samplings in\n", "$\\mathbb{Z}^{n}$\n", "bounded in each dimension. For example, a `uint8` image with dimensions $1024 \\times 768$ is an interval in $\\mathbb{Z}^{2}$ bounded in X (the first dimension) by $[0, 1023]$ and bounded in Y (the second dimension) by $[0, 767]$, with each integer coordinate inside the interval possessing some value $v \\in \\mathbb{Z} : 0 \\leq v \\leq 255$.\n", "\n", "* An `IterableInterval` is an image which can be [_iterated_](https://en.wikipedia.org/wiki/Iteration#Computing) in some order. That is, you can loop over its samples, although the _iteration order_ may vary depending on the type of image—that is, you cannot rely on a particular sample order _a priori_. The good news is that an `IterableInterval` _does_ know its dimensional position at each iteration. An `IterableInterval` is essentially a [stream](https://en.wikipedia.org/wiki/Stream_(computing)).\n", "\n", "* A `RandomAccessibleInterval` is an image which can be _inspected_ at will at arbitrary positions within the interval. In other words, it provides [_random access_](https://en.wikipedia.org/wiki/Random_access) to the image samples.\n", "\n", "If you are familiar with the `java.io` package of the Java standard library, `IterableInterval` is to `java.io.InputStream` as `RandomAccessibleInterval` is to `java.io.RandomAccessFile`.\n", "\n", "Let's dive into the API for each of these types of images!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Iterating images" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is a demonstration of image iteration:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The image is a net.imglib2.img.array.ArrayImg\n", "image instanceof IterableInterval? true\n", "Each sample is a net.imglib2.type.numeric.real.DoubleType\n", "Sample values = 0.0 1.0 2.0 3.0 4.0 1.0 2.0 3.0 4.0 5.0 2.0 3.0 4.0 5.0 6.0\n" ] }, { "data": { "text/plain": [ "null" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import net.imglib2.IterableInterval\n", "\n", "// Create a tiny image.\n", "image = ij.op().run(\"create.img\", [5, 3])\n", "println(\"The image is a \" + image.getClass().getName())\n", "println(\"image instanceof IterableInterval? \" + (image instanceof IterableInterval))\n", "println(\"Each sample is a \" + image.firstElement().getClass().getName())\n", "\n", "// Populate it with a diagonal gradient.\n", "ij.op().image().equation(image, \"p[0]+p[1]\")\n", "\n", "// Iterate over the image samples!\n", "print(\"Sample values =\")\n", "// A foreach loop in Groovy uses \"in\" to seperate arguments whereas a Java foreach loop uses \":\" to seperate arguments\n", "for (v in image)\n", " print(\" \" + v)\n", "println()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The actual type of the created image object is `ArrayImg`, which is an _image container_ backed by one big array—in this case, a `double[]` because the samples are of type `DoubleType`.\n", "\n", "More generally: an `Img` is an object which is both an `IterableInterval` and a `RandomAccessibleInterval`. Other kinds of image containers include `PlanarImg` (one array per 2D slice) and `CellImg` (one array per N-dimensional block).\n", "\n", "You might be wondering: what if I need to know the dimensional position during iteration? The solution is to use a _cursor_:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Samples by position:\n", "\t(0, 0) = 0.0\t(1, 0) = 1.0\t(2, 0) = 2.0\t(3, 0) = 3.0\t(4, 0) = 4.0\n", "\t(0, 1) = 1.0\t(1, 1) = 2.0\t(2, 1) = 3.0\t(3, 1) = 4.0\t(4, 1) = 5.0\n", "\t(0, 2) = 2.0\t(1, 2) = 3.0\t(2, 2) = 4.0\t(3, 2) = 5.0\t(4, 2) = 6.0" ] }, { "data": { "text/plain": [ "null" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(\"Samples by position:\")\n", "c = image.localizingCursor()\n", "while (c.hasNext()) {\n", " v = c.next()\n", " xPos = c.getLongPosition(0)\n", " yPos = c.getLongPosition(1)\n", " if (xPos == 0) println()\n", " print(\"\\t(\" + xPos + \", \" + yPos + \") = \" + v)\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each cursor is a pointer into the image somewhere, which knows its position. It is possible to run multiple cursors over an image simultaneously.\n", "\n", "Note that for performance, we use a _localizing cursor_ above by calling `localizingCursor()`, because we knew we would query the position every time. If we were going to query the position only rarely, `cursor()` would be better—a vanilla cursor still knows its position, but does less bookkeeping and hence is faster to iterate in cases where you don't query very often." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Iteration orders" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the positions above, we see that the iteration order here happens to be nice and organized, with X moving fastest, followed by Y. This so-called _flat iteration order_ is what you get with `ArrayImg`, but may differ with other image containers. Here is an example of how a _cell image_ with $2 \\times 2 \\times 2$ blocks differs from an _array image_ of the same size:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Array image iteration order =\n", "\t(0, 0, 0)\t(1, 0, 0)\t(2, 0, 0)\t(3, 0, 0)\n", "\t(0, 1, 0)\t(1, 1, 0)\t(2, 1, 0)\t(3, 1, 0)\n", "\t(0, 0, 1)\t(1, 0, 1)\t(2, 0, 1)\t(3, 0, 1)\n", "\t(0, 1, 1)\t(1, 1, 1)\t(2, 1, 1)\t(3, 1, 1)\n", "\n", "Cell image iteration order =\n", "\t(0, 0, 0)\t(1, 0, 0)\t(0, 1, 0)\t(1, 1, 0)\n", "\t(0, 0, 1)\t(1, 0, 1)\t(0, 1, 1)\t(1, 1, 1)\n", "\t(2, 0, 0)\t(3, 0, 0)\t(2, 1, 0)\t(3, 1, 0)\n", "\t(2, 0, 1)\t(3, 0, 1)\t(2, 1, 1)\t(3, 1, 1)\n", "\n" ] }, { "data": { "text/plain": [ "null" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "// Create an array image and a cell (2x2x2) image.\n", "import net.imglib2.img.array.ArrayImgs\n", "import net.imglib2.img.cell.CellImgFactory\n", "long[] dims = [4, 2, 2]\n", "arrayImg = ArrayImgs.unsignedBytes(dims)\n", "cellImg = new CellImgFactory(2).create(dims, arrayImg.firstElement())\n", "\n", "def printPositions(ii, width) {\n", " c = ii.localizingCursor()\n", " col = 0\n", " while (c.hasNext()) {\n", " v = c.next()\n", " xPos = c.getLongPosition(0)\n", " yPos = c.getLongPosition(1)\n", " zPos = c.getLongPosition(2)\n", " print(\"\\t(\" + xPos + \", \" + yPos + \", \" + zPos + \")\")\n", " if (++col == width) { col = 0; println() }\n", " }\n", " println()\n", "}\n", "\n", "println()\n", "println(\"Array image iteration order =\")\n", "printPositions(arrayImg, 4)\n", "\n", "println(\"Cell image iteration order =\")\n", "printPositions(cellImg, 4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Random Access Images" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Accessing image samples directly" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sometimes you need to access samples in an order other than the natural one. This is possible as long as the image implements the `RandomAccessible` interface.\n", "\n", "By using a `RandomAccess` accessor, we can get access to any pixel of the image in any order:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Array image instanceof RandomAccessible? true\n", "Cell image instanceof RandomAccessible? true\n" ] }, { "data": { "text/html": [ "
ArrayImg | |
CellImg |