{ "cells": [ { "cell_type": "markdown", "id": "2d922667-7bf9-49d2-88b9-c49d1ca12a68", "metadata": {}, "source": [ "# Vane Calibration\n", "This notebook shows how to calibrate data that uses a vane (or a hot load) to determine the system temperature.\n", "In the case of the GBT this applies mainly to [Argus](https://gbtdocs.readthedocs.io/en/latest/references/receivers/argus.html), although the [Q-band receiver](https://gbtdocs.readthedocs.io/en/latest/references/receivers/q-band.html) also has a vane.\n", "For the background on the calibration please refer to [Frayer et al. 2019](https://ui.adsabs.harvard.edu/abs/2019nrao.reptE...1F/abstract).\n", "\n", "## Dysh commands\n", "\n", "The following dysh commands are introduced (leaving out all the function arguments):\n", "\n", " filename = dysh_data()\n", " sdf = GBTFITSLoad()\n", " sb = sdf.getfs()\n", " v = sdf.getvane()\n", " ta = sb.timeaverage()\n", " ta.baseline()\n", " ta.average()\n", " ta.plot()\n", " ta1.oshow()\n", "\n", "\n", "## Loading Modules\n", "We start by loading the modules we will use for this example. \n" ] }, { "cell_type": "code", "execution_count": 1, "id": "34883649-57da-4253-b62d-fa2d0f78ba83", "metadata": {}, "outputs": [], "source": [ "# These modules are required for the data reduction.\n", "from dysh.fits import GBTFITSLoad\n", "from astropy import units as u\n", "from dysh.log import init_logging\n", "\n", "# These modules are used for file I/O\n", "from dysh.util.files import dysh_data\n", "from pathlib import Path" ] }, { "cell_type": "markdown", "id": "35caed50-0264-41a9-ba24-715f56207b5b", "metadata": {}, "source": [ "## Setup\n", "dysh uses a logger to communicate. If you are working in the command\n", "line, then the logging is setup for you. If you are working in a\n", "jupyter lab instance, then you need to set it up. You can do so using\n", "the init_logging function imported above. As an argument, init_logging\n", "takes a number, the verbosity level. level 0 is for error messages\n", "only, 1 for warning, 2 for info and 3 for debug. Here we set it to\n", "level 2." ] }, { "cell_type": "code", "execution_count": 2, "id": "74d688a7-01ad-4660-930e-ce7d1abacd8a", "metadata": {}, "outputs": [], "source": [ "init_logging(2)\n", "\n", "# also create a local \"output\" directory where temporary notebook files can be stored.\n", "output_dir = Path.cwd() / \"output\"\n", "output_dir.mkdir(exist_ok=True)" ] }, { "cell_type": "markdown", "id": "b4daeab5-7ca1-4017-af89-d3ea1ad17ff3", "metadata": {}, "source": [ "## Data Retrieval\n", "\n", "Download the example SDFITS data, if necessary." ] }, { "cell_type": "code", "execution_count": 3, "id": "49f46581-d217-4560-a0ed-e6206c0761da", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:09.754 I Resolving example=vane -> fs-Argus/data/AGBT20B_295_02.raw.vegas/AGBT20B_295_02.raw.vegas.A.fits\n", "18:42:09.755 I url: http://www.gb.nrao.edu/dysh//example_data/fs-Argus/data/AGBT20B_295_02.raw.vegas/AGBT20B_295_02.raw.vegas.A.fits\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Odd-1, did not find /home/teuben/GBT/dysh_data/example_data/fs-Argus/data/AGBT20B_295_02.raw.vegas/AGBT20B_295_02.raw.vegas.A.fits\n", "AGBT20B_295_02.raw.vegas.A.fits already downloaded\n" ] } ], "source": [ "filename = dysh_data(example=\"vane\")" ] }, { "cell_type": "markdown", "id": "57d5074d-868d-499b-a71a-4bd0677678cc", "metadata": {}, "source": [ "## Data Loading\n", "\n", "Next, we use \n", "[GBTFITSLoad](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#dysh.fits.GBTFITSLoad)\n", "to load the data, and then its \n", "[summary](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#dysh.fits.GBTFITSLoad.summary)\n", "method to inspect its contents." ] }, { "cell_type": "code", "execution_count": 4, "id": "ee66b9b7-b900-4b7f-9c2e-40315f88154d", "metadata": {}, "outputs": [], "source": [ "sdfits = GBTFITSLoad(filename)" ] }, { "cell_type": "code", "execution_count": 5, "id": "92f55874-3482-47cc-8f07-fa4917fe819b", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SCANOBJECTVELOCITYPROCPROCSEQNRESTFREQDOPFREQ# IF# POL# INT# FEEDAZIMUTHELEVATION
10VANE65.0Track193.17377793.17377711252166.987843.5400
11SKY65.0Track193.17377793.17377711252166.987543.5399
12G24.78965.0Track193.17377793.173777111512167.436343.6122
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sdfits.summary()" ] }, { "cell_type": "markdown", "id": "a8578090-d080-4fcd-b0e3-dbec40f8ec89", "metadata": {}, "source": [ "This is a frequency switched observation with Argus.\n", "The first two scans, 10 and 11, are observations of the vane and the \"cold\" sky.\n", "The next scan, 12, are the observations of the target using frequency switching." ] }, { "cell_type": "markdown", "id": "ead38c7d-3e61-4ab0-883a-4167a12daab3", "metadata": {}, "source": [ "## Data Reduction\n", "To calibrate data using a vane we can use the same methods as with any other observations, with the difference that we must specify the ``vane`` argument.\n", "This argument can be an integer, with the scan number of the vane observations (in this case scan 10), or it can be a \n", "[VaneSpectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.vane.VaneSpectrum)\n", "object.\n", "\n", "### Calibration with Vane\n", "We will show how to calibrate the data providing a scan number for the ``vane`` argument.\n", "We call \n", "[getfs](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#dysh.fits.GBTFITSLoad.getfs)\n", "with ``vane=10``.\n", "If working from one of the GBO data reduction hosts, this will initialize a \n", "[VaneSpectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.vane.VaneSpectrum)\n", "and determine the zenith opacity and atmospheric temperature from the CLEO weather forecast scripts.\n", "If working from elsewhere, then by default these values are not known and the vane calibration will use an approximation to determine the system temperature (equation (23) in [Frayer et al. 2019](https://ui.adsabs.harvard.edu/abs/2019nrao.reptE...1F/abstract)).\n", "\n", "For this data set we know that feeds 10 and 8 are available, we use ``fdnum=10``, and only a single spectral window and polarization are available, so we use ``ifnum=0`` and ``plnum=0``." ] }, { "cell_type": "code", "execution_count": 6, "id": "17102cc6-1ecb-4757-9f56-1ff5886c5e3e", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:09.908 I Vane calibrated data will be calibrated to Ta* units by default.\n", "18:42:10.418 I Weather forecast not available.\n", "18:42:10.506 I Ignoring 1 blanked integration(s).\n", "18:42:10.711 I Vane temperature (twarm): 400.15 K\n", "18:42:10.712 I No zenith opacity nor atmospheric temperature available. Will approximate the calibration temperature to the vane temperature 400.15 K\n", "18:42:10.712 I Mean calibration temperature (tcal): 400.15 K\n" ] } ], "source": [ "ta = sdfits.getfs(scan=12, ifnum=0, plnum=0, fdnum=10, vane=10).timeaverage()" ] }, { "cell_type": "markdown", "id": "a67f00af-f040-47b4-a83e-13dd9c050a9c", "metadata": {}, "source": [ "The above messages tells us what values for the different parameters required to determine the system temperature were adopted.\n", "\n", "Next, we plot the calibrated spectrum." ] }, { "cell_type": "code", "execution_count": 7, "id": "671fb1ac-da2e-45b2-a8aa-91b2e37d29a6", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9cd3a2e76ddf4e4e845d897208b32df0", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot = ta.plot(xaxis_unit=\"GHz\")" ] }, { "cell_type": "markdown", "id": "1942bf4e-a9c0-4b85-9bac-1dd41fa6f3ff", "metadata": {}, "source": [ "We see a clear signal at about 93.150 GHz, and it's \"ghost\" at 93.055 GHz.\n", "\n", "#### Changing default values\n", "\n", "The parameters required to determine the system temperature using a vane are: the zenith opacity (``zenith_opacity``), vane temperature (``t_warm``), atmospheric temperature (``t_atm``) and background temperature (``t_bkg``). \n", "From these a calibration temperature (``t_cal``) is derived. \n", "These can be modified by providing values for them as arguments to the calibration method.\n", "\n", "In the following code, we modify the first four parameters used to derive the calibration temperature.\n", "We set ``zenith_opacity=0.01`` in nepers, ``t_warm=583`` in K, ``t_atm=260`` in K and ``t_bkg=3``.\n", "We use a higher vane temperature so we can see the difference when comparing the results." ] }, { "cell_type": "code", "execution_count": 8, "id": "456fe5d7-f73a-4cd2-8aa0-4249bce770ab", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:11.943 I Vane calibrated data will be calibrated to Ta* units by default.\n", "18:42:12.230 I Weather forecast not available.\n", "18:42:12.298 I Ignoring 1 blanked integration(s).\n", "18:42:12.417 I Vane temperature (twarm): 583.00 K\n", "18:42:12.418 I Mean calibration temperature (tcal): 584.97 K\n" ] } ], "source": [ "ta2 = sdfits.getfs(scan=12, ifnum=0, plnum=0, fdnum=10, vane=10,\n", " zenith_opacity=0.01,\n", " t_warm=583,\n", " t_atm=260,\n", " t_bkg=3,\n", " ).timeaverage()" ] }, { "cell_type": "markdown", "id": "c7a593d9-89b9-4840-88b4-089139425153", "metadata": {}, "source": [ "Now we plot both results on top of each other, with the increased vane temperature spectrum in blue and the original spectrum in orange." ] }, { "cell_type": "code", "execution_count": 9, "id": "ec5697c7-4796-4e04-960c-899a46b53c58", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "eea9f7c1e67643129448728efbed0f97", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot2 = ta2.plot(xaxis_unit=\"GHz\")\n", "plot2.oshow(ta)" ] }, { "cell_type": "markdown", "id": "10b5d29f-ecaa-4d21-89a2-82cabe0d0614", "metadata": {}, "source": [ "The effect of having a larger vane temperature is to increase the derived system temperature, so the resulting spectrum is scaled up.\n", "\n", "We can check the system temperature by inspecting the \"TSYS\" item of the ``Spectrum.meta`` dictionary." ] }, { "cell_type": "code", "execution_count": 10, "id": "58c35905-c663-462d-9f22-56fc123b3291", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "173.0724666317342 253.01146158520933\n" ] } ], "source": [ "print(ta.meta[\"TSYS\"], ta2.meta[\"TSYS\"])" ] }, { "cell_type": "markdown", "id": "ab4623af-b556-47cb-b4ae-8465140d9985", "metadata": {}, "source": [ "Alternatively, we can directly provide a value for the calibration temperature, and get the same result.\n", "From the output of the previous example, we had a calibration temperature of 584.97 K." ] }, { "cell_type": "code", "execution_count": 11, "id": "ee3dd6dc-bd69-4c81-a2d0-4c2ac16cd71c", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:13.741 I Vane calibrated data will be calibrated to Ta* units by default.\n", "18:42:13.981 I Weather forecast not available.\n", "18:42:14.044 I Ignoring 1 blanked integration(s).\n" ] } ], "source": [ "ta3 = sdfits.getfs(scan=12, ifnum=0, plnum=0, fdnum=10, vane=10,\n", " t_cal=584.97\n", " ).timeaverage()" ] }, { "cell_type": "markdown", "id": "324743cd-03b4-4680-8c39-65b86fa3eff0", "metadata": {}, "source": [ "We plot this and the previous result on top of each other to confirm that they are the same." ] }, { "cell_type": "code", "execution_count": 12, "id": "97c7f69d-1c7b-43d6-81b1-257bc6734d63", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "3d7538340cec4d2e9ed473392faaa5be", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot3 = ta3.plot(xaxis_unit=\"GHz\")\n", "plot3.oshow(ta2)" ] }, { "cell_type": "markdown", "id": "06da0089-b76d-4cd9-b95a-aa2cfc014cf7", "metadata": {}, "source": [ "## Creating a VaneSpectrum Object\n", "\n", "For more control, or developing alternative data reduction approaches, one could create a \n", "[VaneSpectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.vane.VaneSpectrum)\n", "object directly.\n", "There are two ways of doing so, using \n", "[getvane](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#dysh.fits.gbtfitsload.GBTFITSLoad.getvane),\n", "or directly instantiating a \n", "[VaneSpectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.vane.VaneSpectrum)\n", "object by directly providing the input values, or from a \n", "[Spectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.spectrum.Spectrum)\n", "object using \n", "[VaneSpectrum.from_spectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.vane.VaneSpectrum.from_spectrum).\n", "Here we will show how to use \n", "[getvane](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#dysh.fits.gbtfitsload.GBTFITSLoad.getvane).\n", "\n", "The arguments are the same as those provided for the calibration." ] }, { "cell_type": "code", "execution_count": 13, "id": "c943cb71-e238-42e8-ab0c-c891db7e25b3", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:15.705 I Weather forecast not available.\n" ] } ], "source": [ "vane = sdfits.getvane(scan=10, ifnum=0, plnum=0, fdnum=10, \n", " zenith_opacity=0.01,\n", " t_warm=583,\n", " t_atm=260,\n", " t_bkg=3)" ] }, { "cell_type": "markdown", "id": "1e3b7261-2c63-490a-af50-96d10dfcbd20", "metadata": {}, "source": [ "The calibration temperature can be obtained using the\n", "[VaneSpectrum.get_tcal](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.vane.VaneSpectrum.get_tcal) \n", "method.\n", "This takes as input a reference \n", "[Spectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.spectrum.Spectrum),\n", "For example:" ] }, { "cell_type": "code", "execution_count": 14, "id": "1225e0fa-3100-4ff7-a7eb-70ae5e5152cf", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:15.741 I Using TSYS column\n" ] } ], "source": [ "ref = sdfits.gettp(scan=11, ifnum=0, plnum=0, fdnum=10).timeaverage()" ] }, { "cell_type": "code", "execution_count": 15, "id": "75485722-4dfd-432d-a231-2bd068837448", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:15.893 I Vane temperature (twarm): 583.00 K\n", "18:42:15.894 I Mean calibration temperature (tcal): 584.98 K\n" ] }, { "data": { "text/plain": [ "np.float64(584.9782633701202)" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vane.get_tcal(ref)" ] }, { "cell_type": "markdown", "id": "0d50ad86-b643-46ed-af0a-1969e752e34c", "metadata": {}, "source": [ "And the system temperature using the \n", "[VaneSpectrum.get_tsys](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.vane.VaneSpectrum.get_tsys) \n", "method, which also takes as input a reference \n", "[Spectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.spectrum.Spectrum)." ] }, { "cell_type": "code", "execution_count": 16, "id": "4b40a181-17ca-4a5a-898f-135634f93c7f", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:15.904 I Vane temperature (twarm): 583.00 K\n", "18:42:15.905 I Mean calibration temperature (tcal): 584.98 K\n" ] }, { "data": { "text/plain": [ "np.float64(250.1676896502541)" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vane.get_tsys(ref)" ] }, { "cell_type": "markdown", "id": "7fe1a86b-a186-4218-a2b9-7c58e838d1b5", "metadata": {}, "source": [ "As mentioned earlier, it is also possible to directly provide a \n", "[VaneSpectrum](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.vane.VaneSpectrum) \n", "to the calibration methods. In this case, the additional parameters (``t_warm``, ``t_atm``, ``t_bkg`` and ``t_cal`` are ignored).\n", "For example:" ] }, { "cell_type": "code", "execution_count": 17, "id": "d9c06ee8-9b3b-4599-8e65-6194e95611a9", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:15.915 I Vane calibrated data will be calibrated to Ta* units by default.\n", "18:42:15.916 I Will use a zenith opacity of 0.01 nepers. Taken from vane.\n", "18:42:16.004 I Ignoring 1 blanked integration(s).\n", "18:42:16.137 I Vane temperature (twarm): 583.00 K\n", "18:42:16.138 I Mean calibration temperature (tcal): 584.97 K\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "11ed30f0f71f4accbf8bea9e9fa1de3e", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ta4 = sdfits.getfs(scan=12, ifnum=0, plnum=0, fdnum=10, vane=vane).timeaverage()\n", "plot4 = ta4.plot(xaxis_unit=\"GHz\")" ] }, { "cell_type": "markdown", "id": "35c22b7b-4635-4586-a1d9-aecf35ae8c00", "metadata": {}, "source": [ "We also note that when providing the ``vane`` argument during calibration, the system temperature argument ``t_sys`` is ignored." ] }, { "cell_type": "markdown", "id": "06b8af6b-202b-427d-8fc8-f9adb1d47bee", "metadata": {}, "source": [ "## Final Stats\n", "\n", "Finally, at the end we compute some statistics over a spectrum, merely as a checksum if the notebook is reproducible." ] }, { "cell_type": "code", "execution_count": 18, "id": "f6fd00aa-3540-42d3-b5fd-7fda936d871b", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:17.371 I rms is OK \n" ] } ], "source": [ "ta4.check_stats(2.05290008 * u.K)" ] }, { "cell_type": "code", "execution_count": 19, "id": "d38c4325-255f-46be-9823-38e4c3703828", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "18:42:17.429 I Note: found 1 NaN (masked) values\n" ] }, { "data": { "text/plain": [ "np.float64(0.9100977688428993)" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ta4[1500:8500].radiometer()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.9" } }, "nbformat": 4, "nbformat_minor": 5 }