// @OpService ops // @UIService ui // Script parameters for ops etc. Has to be at the very top for the SciJava framework to get them /* Convolution - Deconvolution Demo for Fiji/ImageJ * * Exponential freq. chirp wave image, with automated convolution / deconvolution demos. * Shows the point of doing image restoration a.k.a. deconvolution * as well as differences between simple inverse filters and constrained iterative methods. * * * Daniel J. White 2016 * * * License: GPL v3. * * Thanks: * * For code/tools - the authors of the plugins used here * and all the contributors to imageJ and Fiji. * * For ideas, late Jan 2016 - Richard Cole, * Director: Advanced Light Microscopy & Image Analysis Core * Wadsworth Center Dept. of Biomedical Sciences * School of Public Health State University of New York * * * Requires: * * some plugins, see below: * Put the .class files in the ImageJ plugins folder * As of jan 2016, compiling the .java source code files * on the fly is broken for java 1.8 (but works for java 1.6) * * Iterative Deconvolution 3D plugin from Bob Dougherty * http://www.optinav.info/Iterative-Deconvolve-3D.htm * http://www.optinav.info/download/Iterative_Deconvolve_3D.class * * Gaussian PSF 3D from Bob Dougherty * http://www.optinav.info/download/Gaussian_PSF_3D.class * or Diffraction PSF 3D also from Bob Dougherty * http://www.optinav.info/Diffraction-PSF-3D.htm * http://www.optinav.info/download/Diffraction_PSF_3D.class * * RandomJ for Poisson noise generation * Turn on Erik's http://imagej.net/ImageScience imageJ update site * if you dont have it already. * * * What's it useful for: * Showing why image restoration by deconvolution is necessary * before quantitative analysis of high N.A. fluorescence microscopy images. * Showing/Simulating the effect of lens induced blurring (convolution) * on contrast vs. spatial frequency, then fixing the introduced error, * within the bandwidth of system, by deconvolution. * * see also http://imagej.nih.gov/ij/macros/DeconvolutionDemo.txt * * Showing a simulation of the systematic error effect * of the microscope objective lens OTF (similar to a Gaussian blur) on the * contrast of different spatial frequencies in a (fluorescence) light microscopy image. * Showing why optical microscope images should be deconvolved, * to correct the systematic error of contrast vs. feature size: * Contrast of smaller and smaller features is attenuatted more and more, * up to the band limit and/or noise floor, after which deconvolution can no longer * recover information since it is lost. * * In raw images, measurements of small object intensities * are lower than they should be compared to larger objects, * making quantitative analysis problematic. * * Deconvolution greatly improves the situation, to the resolution and/or noise limit. * * Why this approach? * Biologists are often unfamiliar with frequency space and the Fourier Theorem. * The general trivial description of blur caused by the lens describes * the resolution limit to some intuitive extent, but does not highlight * the main problem that deconvolution fixes: That the contrast of resolved * features is attenuated as a function of feature size - the OTF. * Meaning, quantification of intensity in different sized objects is likely * to be far from close to the true values until the image is deconvolved, * thus restoring the contrast and intensity of the smaller features. * The trick here is to show frequency space in real space: Not having to * explain frequency space so we don't lose most of the audience. * * How to do it: * open this imageJ javascript in the script editor, select language javascript and run. * You might resize and reposition some windows. * * How to do interactive exploration of different blur widths with live line profile plot: * 1) Run the script to generate the increasingly finely striped image. * 2) Select all, then run plot profile (Ctrl-A, Ctrl-K) * 3) In plot profile, select live mode. * 4) Run the Gaussian Blur tool, set Sigma (Radius) to 5, turn on Preview. * 5) Toggle preview on and off to see the effect. * 6) Try different Sigma values to simulate different lens numerical apertures. * Higher Sigma corresponds to lower N.A., and more blur. * 7) See the effect of noise on resolution by adding noise to the image. * Use a line selection, not select all, for the plot profile. * 8) Note the bandwidth (resolution limit) imposed by the blur. * 9) Note that high spatial frequency features (smaller objects) close to the resolution limit * have their contrast more strongly attenuated than lower spatial frequency (large) features. * * What is iterative constrained deconvolution - an algorithm explination: * from DAVID A. AGARD et al. meth cell biol 1989 * http://dx.doi.org/10.1016/S0091-679X(08)60986-3 * .... significantly improve performance for three-dimensional optical sectioning data. * The method is stable, and shows substantially accelerated con-vergence compared to previous methods. * Convergence is reached in 5-10 cycles (instead of 20-50) and since most of the improvement occurs in the first 5 cycles, * iterations can be terminated early with little degradation. * The strategy is to develop a positively constrained solution, g, that, * when convolved with the known smearing function s, will regenerate the observed data o. * The pixel-by-pixel differences between the convolved guess and the observed data are used to update the guess. * Two different update schemes can be used: an additive method initially developed by van Cittert * and later modified by Jannson (Jannson et al., 1970) and by us (Agard et al., 1981), * and a multiplicative method developed by Gold (1964): * (a) o^k = i^k * s * eq 12 (b) i^k+1 = i^k + y(o)(o — o^k) * (c) if i^k+1 < 0 then i^k+1 = 0 (non negativity constraint) * (d) k = k + 1 * * y = 1 — [o^k — A]^2 / A^2 * where * i is what we want to know - the restored image * i^k is the current guess at i * o is observed blurry, noisy image * o^k is the current guess i^k blurred with s (a) * A is a constant set to (the maximum value of o)/2. * * For the Gold method, the update equation on line b is changed to * (b') ik(ook) * * In three dimensions these schemes suffer from rather slow convergence, although the Gold method is faster. * The problem stems from inadequate correction of the high-frequency components. * This can be seen by the following argument performed only with the additive method for simplicity. * At each cycle we desire to correct our current guess with a function, 6, so that * o ~ (g, + 8) * s or in Fourier space: 0 (G, + 6)S. Rearranging, we get * 8 = (0 — G,S)IS (13) * From this we can see that a better approximation to the update is to use an inverse filtered version * of the difference between the observed data and the convolved guess. * In practice, we use a Wiener filter to minimize effects of noise and only perform the inverse filtered update * for the first two cycles. After that, we switch to the modified Van Cittert update described above [Eq. (12)]. * At each cycle, the new guess is corrected to maintain positivity and any other desired real-space constraints, * such as spatial boundedness. Every five cycles or so, the guess is smoothed with a Gaussian filter to .... */ // imports first please importClass(Packages.ij.IJ); importClass(Packages.ij.WindowManager); importClass(Packages.ij.gui.WaitForUserDialog); importClass(Packages.ij.plugin.Duplicator); importClass(Packages.ij.plugin.ImageCalculator); importClass(Packages.ij.process.ImageStatistics); importClass(Packages.net.imglib2.img.display.imagej.ImageJFunctions); importClass(Packages.net.imglib2.FinalDimensions); importClass(Packages.net.imagej.ops.Op); // end of imports // Main code execution block // running functions defined below and ImageJ functions. //Part 1 - convolution and deconvolution in 2D // generate exponential (or linear) chirp stripey image by running the drawTestImageFunction with expoChirpFunction as argument drawTestImageFromFunction(512, 512, expoChirpFunction, "Chirp"); //drawTestImageFromFunction(linearChirpFunction, "Chirp"); horizLinePlot(); messageContinue("Notice", "The pattern has the same contrast, 1-10000 photons,\n" + "regardless of spacing or width of the stripes!\n" + " Next - Generate Point Spread Function \n" + " (PSF = image of a point light source) \n" + " for convolution and deconvolution, Continue?"); //Test ij log output IJ.log("Iteration is a cool way to do deconv"); // generate squared value (quasi-confocal) 2D Diffraction model PSF // for blurring (de-sharpening) by convolution and deblurring (re-sharpening or restoring) by deconvolution IJ.run("Diffraction PSF 3D", "index=1.520 numerical=1.42 wavelength=510 " + "longitudinal=0 image=10 slice=200 width,=512 height,=512 depth,=1 " + "normalization=[Sum of pixel values = 1] title=PSF"); IJ.selectWindow("PSF"); IJ.run("Square"); // make the pixel values large, so we can add a realtively small amount of noise later IJ.run("Multiply...", "value=1000000000000"); IJ.resetMinAndMax(); // add a little white noise to avoid divide by zero in inverse filter. IJ.selectWindow("PSF"); IJ.run("Duplicate...", "title=PSFwithNoise"); IJ.selectWindow("PSFwithNoise"); IJ.run("Add Specified Noise...", "standard=0.00000002"); messageContinue("Notice", "The PSF is generated from a diffraction model. \n" + "It is about 20 pixels wide, and simulates a confocal PSF.\n" + " Next - blur the stripey image using the PSF, Continue?"); // Use Fourier domain math to do the convolution // needs power of 2 sized images in x and y. IJ.selectWindow("Chirp"); IJ.run("FD Math...", "image1=Chirp operation=Convolve " + "image2=PSFwithNoise result=Chirp-blur32bit do"); // rescale intensities to same range as original image. scaleIntensities10k("Chirp-blur32bit", "Chirp-blur-scaled"); horizLinePlot(); messageContinue("Notice", "The smaller the features are,\n" + "the more their contrast is attenuated (lost): \n" + "The smaller features have the most wrong pixel values! \n" + " Next - Inverse Filter with the PSF \n" + " to restore the image intensites, Continue?"); // Fourier domain math deconvolve: inverse filter with PSFwithNoise. // A little noise in PSF avoids divide by zero // see https://imagej.nih.gov/ij/macros/DeconvolutionDemo.txt // needs square power of 2 sized images!!! so 1024x1024 or 512x512 here I guess. // Above, we used FD math to do the convolution as well. IJ.run("FD Math...", "image1=Chirp-blur32bit operation=Deconvolve " + "image2=PSFwithNoise result=InverseFiltered do"); IJ.selectWindow("InverseFiltered"); horizLinePlot(); messageContinue("Notice", "The inverse filter gives a perfect result \n" + "because the PSF is known perfectly and there is no noise! \n" + "Sadly this is not a realistic situation, there is always noise... \n" + " Next - Generate blurred, slightly noisy image, Continue?"); // Generate more realistic test image that contains noise // so we can demo how to deal with that. IJ.selectWindow("Chirp-blur-scaled"); // Add noise with a certain SD //IJ.run("Add Specified Noise...", "standard=0.2"); //renameImage("Chirp-blur-scaled", "Chirp-blur-noise"); // Add Poisson modulatory noise, like photon shot noise. The "mean" parameter is ignored in modulatory mode. IJ.run("RandomJ Poisson", "mean=10.0 insertion=Modulatory"); renameImage("Chirp-blur-scaled with modulatory Poisson noise", "Chirp-blur-noise"); IJ.selectWindow("Chirp-blur-noise"); horizLinePlot(); messageContinue("Notice", "This is more realistic: \n" + "The image is blurred and contains Noise \n" + "Thus, a simple inverse filter will not work, \n" + "because amplified noise will kill the real features! \n" + " Next - Inverse filtering a noisy image, Continue?"); // Fourier domain math deconvolve of noisy image: // inverse filter with PSF without noise - should amplify noise over features. IJ.run("FD Math...", "image1=Chirp-blur-noise operation=Deconvolve " + "image2=PSF result=InverseFilteredNoise do"); IJ.selectWindow("InverseFilteredNoise"); horizLinePlot(); messageContinue("The inverse filtered image:", "Inverse filtering is defeated by noise! \n" + "The pixel intensity values of the smaller and smaller features \n" + "have a lower and lower signal : noise ratio \n" + "The inverse filtered image has this noise amplified: \n" + "so much that the image features are lost. \n" + " Next - Iterative, Non-negativity Constrained Deconvolution \n" + " using the generated PSF on the noisy blurred image, Continue?"); // Perform iterative, non negative constrained, deconvolution // on the noisy image with the slightly noisy PSF // to simulate a real sitiuation. // IJ builtin commands implemetation of non negative constrained additive iterative deconv algorithm // Take blurred noisy image as first guess at model image // we will need an image calculator to get differnce image ic = new ImageCalculator(); imgStats = new ImageStatistics(); // set needed variables then loop the following to do iterations until run out of iterations temp = new Duplicator().run(WindowManager.getImage("Chirp-blur-noise")); temp.setTitle("temp"); temp.show(); ChirpBlurNoiseimp = WindowManager.getImage("Chirp-blur-noise"); var iterations = 76; // multiples of 5 plus 1 please, as we smooth every 5 iterations and 0th iteration. for (i=0; i