{ "cells": [ { "cell_type": "markdown", "id": "f2ff2011-ba60-4a00-886d-f8a1efd950b0", "metadata": {}, "source": [ "# Saving Data to External Files\n", "This notebook gives examples of how to write out selected data from\n", "[GBTFitsLoad](https://dysh.readthedocs.io/en/latest/modules/dysh.fits.html#module-dysh.fits.gbtfitsload) \n", "and how to save \n", "[Spectrum](https://dysh.readthedocs.io/en/latest/modules/dysh.spectra.html#module-dysh.spectra.spectrum), \n", "[Scan](https://dysh.readthedocs.io/en/latest/modules/dysh.spectra.html#module-dysh.spectra.scan), \n", "and \n", "[ScanBlock](https://dysh.readthedocs.io/en/latest/modules/dysh.spectra.html#dysh.spectra.scan.ScanBlock) \n", "to different formats. It also introduces the \n", "[dysh_data](https://dysh.readthedocs.io/en/latest/modules/dysh.util,html) \n", "function to obtain filename references to various test and example data.\n", "\n", "## Dysh Commands\n", "\n", "The following dysh commands are introduced (leaving out all the function arguments):\n", "\n", " filename = dysh_data()\n", " sdf = GBTFITSLoad()\n", " sb = sdf.getps()\n", " ta = sb.timeaverage()\n", " ta.average()\n", " ta.plot()\n", " ta.write()\n", " Spectrum.read()\n", "\n", "## Loading Modules\n", "We start by loading the modules we will use for this example. \n", "\n", "For display purposes, we use the static (non-interactive) matplotlib backend in this tutorial. However, you can tell `matplotlib` to use the `ipympl` backend to enable interactive plots. This is only needed if working on jupyter lab or notebook." ] }, { "cell_type": "code", "execution_count": 1, "id": "a51699fb-4012-4663-a447-a8290adca502", "metadata": {}, "outputs": [], "source": [ "# These modules are required for loading and reading data.\n", "from dysh.fits.gbtfitsload import GBTFITSLoad\n", "from dysh.spectra.spectrum import Spectrum\n", "from astropy import units as u\n", "from dysh.log import init_logging\n", "\n", "# We will use matplotlib for plotting.\n", "import matplotlib.pyplot as plt\n", "\n", "# These modules are used for file I/O\n", "from dysh.util.files import dysh_data\n", "from pathlib import Path" ] }, { "cell_type": "markdown", "id": "939b0703-f97b-448f-9ff6-71382bbc4571", "metadata": {}, "source": [ "## Setup\n", "We start the dysh logging, so we get more information about what is happening.\n", "This is only needed if working on a notebook.\n", "If using the CLI through the ``dysh`` command, then logging is setup for you.\n", "\n", "We also create a directory where output files from the notebook get stored." ] }, { "cell_type": "code", "execution_count": 2, "id": "61632226-51fc-46bf-b532-f7da3ba87eb5", "metadata": {}, "outputs": [], "source": [ "init_logging(2)\n", "\n", "# also create a local \"output\" directory where temporary notebook files can be stored.\n", "output_dir = Path.cwd() / \"output\"\n", "output_dir.mkdir(exist_ok=True)" ] }, { "cell_type": "markdown", "id": "24b6bd02-19a4-4886-8fd3-2ed663713e1f", "metadata": {}, "source": [ "## Data Retrieval\n", "\n", "Download the example SDFITS data, if necessary.\n", "\n", "The actual full path filename will depend on your dysh installation, it could be local, or it will need to be downloaded.\n", "\n", "The \n", "[dysh_data](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.util.html#dysh.util.files.dysh_data)\n", "function has a few mnemonics, to avoid having to remember long filenames. Data via `test=` are available for developers, since they come with the code. Data via `example=` will need to be downloaded, unless you have downloaded the example data into `$DYSH_DATA/example_data`.\n", "\n", "Here is an example how to find the aliases available with `test=`" ] }, { "cell_type": "code", "execution_count": 3, "id": "c55bf7d8-c462-4e63-8108-537d3b9e6db2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# dysh_data::test\n", "# ---------------\n", "getps AGBT05B_047_01/AGBT05B_047_01.raw.acs/\n", "getps2 TGBT21A_501_11/TGBT21A_501_11.raw.vegas.fits\n", "getfs TGBT21A_504_01/TGBT21A_504_01.raw.vegas/TGBT21A_504_01.raw.vegas.A.fits\n", "subbeamnod2 TRCO_230413_Ka\n" ] } ], "source": [ "dysh_data(test=\"?\")" ] }, { "cell_type": "markdown", "id": "4c6d38eb-ab58-4197-8271-d48aa5a13cf5", "metadata": {}, "source": [ "For many example notebooks we use the `test=\"getps\"` dataset. In the next cell there are multiple ways to get at the same data:" ] }, { "cell_type": "code", "execution_count": 4, "id": "a95701ab-fb3b-4d6f-885d-c3e66fa0b9e4", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "14:57:34.831 I Resolving test=getps -> AGBT05B_047_01/AGBT05B_047_01.raw.acs/\n" ] } ], "source": [ "# filename = dysh_data(example=\"positionswitch/data/AGBT05B_047_01/AGBT05B_047_01.raw.acs/AGBT05B_047_01.raw.acs.fits\")\n", "# filename = dysh_data(example=\"getps\")\n", "filename = dysh_data(test=\"getps\")" ] }, { "cell_type": "markdown", "id": "5864a5e2-101c-4eef-ae56-0dced46a985e", "metadata": {}, "source": [ "## Data Loading\n", "\n", "Next, we use `GBTFITSLoad` to load the data, and report some basic `info` and use the `summary` method to inspect its contents." ] }, { "cell_type": "code", "execution_count": 5, "id": "eb5f2170-fb32-4d18-9de7-e46fd0164c6d", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Filename: /home/teuben/GBT/dysh/testdata/AGBT05B_047_01/AGBT05B_047_01.raw.acs/AGBT05B_047_01.raw.acs.fits\n", "No. Name Ver Type Cards Dimensions Format\n", " 0 PRIMARY 1 PrimaryHDU 12 () \n", " 1 SINGLE DISH 1 BinTableHDU 229 352R x 70C ['32A', '1D', '22A', '1D', '1D', '1D', '32768E', '16A', '6A', '8A', '1D', '1D', '1D', '4A', '1D', '4A', '1D', '1I', '32A', '32A', '1J', '32A', '16A', '1E', '8A', '1D', '1D', '1D', '1D', '1D', '1D', '1D', '1D', '1D', '1D', '1D', '1D', '8A', '1D', '1D', '12A', '1I', '1I', '1D', '1D', '1I', '1A', '1I', '1I', '16A', '16A', '1J', '1J', '22A', '1D', '1D', '1I', '1A', '1D', '1E', '1D', '1A', '1A', '8A', '1E', '1E', '16A', '1I', '1I', '1I'] \n" ] } ], "source": [ "sdfits = GBTFITSLoad(filename)\n", "sdfits.info()" ] }, { "cell_type": "code", "execution_count": 6, "id": "bf42e295-5389-45c0-b7b6-08d52427d3c4", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SCANOBJECTVELOCITYPROCPROCSEQNRESTFREQDOPFREQ# IF# POL# INT# FEEDAZIMUTHELEVATION
51NGC52914386.0OnOff11.4204051.42040512111198.343118.6427
52NGC52914386.0OnOff21.4204051.42040512111198.930618.7872
53NGC52914386.0OnOff11.4204051.42040512111199.330518.3561
54NGC52914386.0OnOff21.4204051.42040512111199.915718.4927
55NGC52914386.0OnOff11.4204051.42040512111200.304218.0575
56NGC52914386.0OnOff21.4204051.42040512111200.890618.1860
57NGC52914386.0OnOff11.4204051.42040512111202.327517.3853
58NGC52914386.0OnOff21.4204051.42040512111202.919217.4949
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sdfits.summary()" ] }, { "cell_type": "markdown", "id": "9351def9-e459-46f0-be5f-cc5e9113213a", "metadata": {}, "source": [ "### Calibrate a position switched scan \n", "\n", "The `getps` function returns a `ScanBlock` containing one [`PSScan`](https://dysh.readthedocs.io/en/latest/modules/dysh.spectra.html#dysh.spectra.scan.PSScan) with 11 integrations for the ON position and 11 integrations for the OFF position." ] }, { "cell_type": "code", "execution_count": 7, "id": "b8ea0bfc-6a50-43f3-861a-28383bc312ef", "metadata": {}, "outputs": [], "source": [ "ps_scan_block = sdfits.getps(scan=51, ifnum=0, plnum=0, fdnum=0)" ] }, { "cell_type": "code", "execution_count": 8, "id": "357fdb65-d0d0-42f7-8ff2-fb2f51789eef", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of integrations = 22\n" ] } ], "source": [ "print(f\"Number of integrations = {ps_scan_block[0].nrows}\")" ] }, { "cell_type": "markdown", "id": "95eaa2b4-dfb0-4cbc-bd33-61574487fd1f", "metadata": {}, "source": [ "Get the time average of the calibrated data. This method returns a `Spectrum`." ] }, { "cell_type": "code", "execution_count": 9, "id": "191dce67-3ca7-40e7-9610-371002934880", "metadata": {}, "outputs": [], "source": [ "ta = ps_scan_block.timeaverage()" ] }, { "cell_type": "code", "execution_count": 10, "id": "fa82c57a-9346-4c65-9b52-84d29b38024d", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "192da8ecf8ef4d97a55ca5cb80f767ad", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ta.plot();" ] }, { "cell_type": "markdown", "id": "1e3fe713-24f2-458e-a8a3-88cb63191fb7", "metadata": {}, "source": [ "In the \n", "[Position-Switched Data Reduction](https://dysh.readthedocs.io/en/latest/users_guide/positionswitch.html)\n", "notebook we will go in more depth on this particular data. Here we continue with some examples of how to write and read data." ] }, { "cell_type": "markdown", "id": "5e7285c7-51c1-400e-845e-0590ae656e9a", "metadata": {}, "source": [ "## Reading and Writing Individual Spectra" ] }, { "cell_type": "markdown", "id": "8e5d964f-53ac-4591-a70a-f4cf4b2a3b43", "metadata": {}, "source": [ "### Inputs and Outputs\n", "dysh supports output to text files in a variety of [formats familiar to users of astropy](https://docs.astropy.org/en/stable/io/ascii/index.html#id1):\n", "* basic\n", "* commented_header\n", "* ECSV\n", "* fixed_width\n", "* IPAC\n", "* MRT\n", "* votable\n", "\n", "The following lines of code define some of the available formats in a list, and then loop over them saving the calibrated data in each format.\n", "We use the `overwrite=True` parameter to avoid errors if the files already exist on disk." ] }, { "cell_type": "code", "execution_count": 11, "id": "0fbf4377-ec18-45c6-9978-9cc66b4262cc", "metadata": {}, "outputs": [], "source": [ "# Define the formats in a list.\n", "fmt = [ \n", " \"basic\",\n", " \"commented_header\",\n", " \"ecsv\",\n", " \"fixed_width\",\n", " \"ipac\",\n", " \"mrt\",\n", " \"votable\",\n", "]\n", "\n", "\n", "# Loop over formats writing the calibrated spectrum.\n", "for f in fmt:\n", " file = output_dir / f\"testwrite.{f}\"\n", " ta.write(file, format=f, overwrite=True)" ] }, { "cell_type": "markdown", "id": "9f7976e1-8843-449b-a691-f679b8dc6155", "metadata": {}, "source": [ "We can also write a `Spectrum` to FITS format \n", "\n", "NOTE: this is actually NOT the SDFITS format, but a BINTABLE erroneously labeled as \"SINGLE DISH\"." ] }, { "cell_type": "code", "execution_count": 12, "id": "17fcc3f7-b768-4e82-912a-d8da3f580981", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: Attribute `QD_XEL` of type cannot be added to FITS Header - skipping [astropy.io.fits.convenience]\n", "WARNING: Attribute `QD_EL` of type cannot be added to FITS Header - skipping [astropy.io.fits.convenience]\n", "WARNING: Attribute `TWARM` of type cannot be added to FITS Header - skipping [astropy.io.fits.convenience]\n", "WARNING: Attribute `TCOLD` of type cannot be added to FITS Header - skipping [astropy.io.fits.convenience]\n" ] } ], "source": [ "ta.write(output_dir / \"testwrite.fits\", format=\"fits\", overwrite=True)" ] }, { "cell_type": "markdown", "id": "014ebca5-5e6c-4400-bb35-3463e11f852a", "metadata": {}, "source": [ "We can read spectra in FITS and a few formats. [As noted in astropy, ECSV ](https://docs.astropy.org/en/stable/io/ascii/ecsv.html#ecsv-format) is the only ASCII format that can make a lossless output-input roundtrip and thus reproduce an original spectrum.\n", "\n", "We use the `Spectrum.read` method to read the saved data." ] }, { "cell_type": "code", "execution_count": 13, "id": "f9b097e4-27c7-4460-955f-06e7bcccd99d", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e73d3cf01eba44a3947d22fc5a057288", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "s1 = Spectrum.read(output_dir / \"testwrite.fits\", format=\"fits\")\n", "s1.plot();" ] }, { "cell_type": "code", "execution_count": 14, "id": "b6c51e2d-2be7-4326-ae12-3ed05f1f73b3", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "5e7ce62c77174a77bd91e441e215621c", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "s2 = Spectrum.read(output_dir / \"testwrite.ecsv\", format=\"ecsv\")\n", "s2.plot();" ] }, { "cell_type": "markdown", "id": "537058ef-b76d-4b4f-b175-61103cb46767", "metadata": {}, "source": [ "### GBTIDL ASCII format\n", "dysh can read text files created by GBTIDL's `write_ascii` function. However, those files do not provide sufficient metadata to fully recreate the spectrum. " ] }, { "cell_type": "code", "execution_count": 15, "id": "dee9791c-9036-443d-9951-d4bfcc78df0c", "metadata": {}, "outputs": [], "source": [ "# \"https://www.gb.nrao.edu/dysh/example_data/onoff-L/gbtidl-data/onoff-L_gettp_156_intnum_0_HEL.ascii\"\n", "filename_ascii = dysh_data(example=\"onoff-L/gbtidl-data/onoff-L_gettp_156_intnum_0_HEL.ascii\")\n", "s3 = Spectrum.read(filename_ascii, format='gbtidl')" ] }, { "cell_type": "code", "execution_count": 16, "id": "0b8102b6-3c8f-4c33-a972-32b81d09bc36", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Spectrum (length=32768)\n", "Flux=[3608710. 3553598. 3604808. ... 3523171. 3545982. 3474847.] ct, mean=493422767.43425 ct\n", "Spectral Axis=[1.42009237 1.42009166 1.42009094 ... 1.39665654 1.39665582\n", " 1.39665511] GHz, mean=1.40837 GHz \n", " {'SCAN': 156, 'OBJECT': 'NGC2782', 'DATE-OBS': '2021-02-10 00:00:00.000', 'RA': np.float64(119.42083333333332), 'VELDEF': 'None-HEL', 'POL': 'YY'}\n" ] } ], "source": [ "print(s3, \"\\n\", s3.meta)" ] }, { "cell_type": "markdown", "id": "37c77152-bdb3-4dc4-bd5d-3e705b669908", "metadata": {}, "source": [ "dysh can even read compressed ASCII files. Note these data have velocity on the spectral axis." ] }, { "cell_type": "code", "execution_count": 17, "id": "2f054bdf-cfd1-486e-8e2d-04c541e6244e", "metadata": {}, "outputs": [], "source": [ "filename_ascii_gz = dysh_data(example=\"onoff-L/gbtidl-data/onoff-L_getps_152_RADI-HEL.ascii.gz\")\n", "s4 = Spectrum.read(filename_ascii_gz, format='gbtidl')" ] }, { "cell_type": "code", "execution_count": 18, "id": "72ff1f59-7363-4168-af22-b5b217a1b132", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Spectrum (length=32768)\n", "Flux=[-0.1042543 0.05250004 0.00432693 ... -0.05038555 0.03408394\n", " 0.06139921] K, mean=0.17345 K\n", "Spectral Axis=[1281.15245599 1281.30342637 1281.45439674 ... 6227.69690405\n", " 6227.84787443 6227.99884481] km / s, mean=3754.57565 km / s \n", " {'SCAN': 152, 'OBJECT': 'NGC2415', 'DATE-OBS': '2021-02-10 00:00:00.000', 'RA': np.float64(114.65624999999999), 'VELDEF': 'RADI-HEL', 'POL': 'YY'}\n" ] } ], "source": [ "print(s4, \"\\n\", s4.meta)" ] }, { "cell_type": "markdown", "id": "fa602f35-2e39-4f90-a505-8e00e26d8f36", "metadata": {}, "source": [ "#### Plot \n", "To plot the spectrum contained in the ascii files you have to use `matplotlib`." ] }, { "cell_type": "code", "execution_count": 19, "id": "a088d9c3-43a2-496b-bb47-ff61ab6d7562", "metadata": {}, "outputs": [], "source": [ "# set the rest frequency to the HI line\n", "s4.rest_value = 1420405751.786 * u.Hz" ] }, { "cell_type": "code", "execution_count": 20, "id": "0b6de133-2d3b-41e2-8241-e76250e48143", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/teuben/GBT/anaconda3/lib/python3.13/site-packages/traitlets/traitlets.py:1385: DeprecationWarning: Passing unrecognized arguments to super(Toolbar).__init__().\n", "NavigationToolbar2WebAgg.__init__() missing 1 required positional argument: 'canvas'\n", "This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.\n", " warn(\n" ] } ], "source": [ "plt.figure()\n", "plt.xlabel(f\"Velocity ({s4.meta['VELDEF']}) [{s4.spectral_axis.unit}]\")\n", "plt.ylabel(f\"$T_A$ [{s4.flux.unit}]\")\n", "plt.plot(s4.spectral_axis, s4.flux);" ] }, { "cell_type": "markdown", "id": "216f4479-770c-4c74-878c-8c1f28a06e9a", "metadata": {}, "source": [ "Apart from the signal around 3800 km/s, there are a handfull of positive-negative instrumental spikes That is for another notebook." ] }, { "cell_type": "markdown", "id": "a335b2f6-0b1d-4686-aa6b-b66a97e8c609", "metadata": {}, "source": [ "## Writing Multiple Calibrated Spectra to SDFITS\n", "You can write the calibrated data from a `ScanBlock` to the SDFITS format.\n", "If there are multiple scans in the `ScanBlock`, they will all be written to the same SDFITS (useful for `gbtgridder`)." ] }, { "cell_type": "code", "execution_count": 21, "id": "6f99a6b0-199a-45db-b328-3609baab6b57", "metadata": {}, "outputs": [], "source": [ "ps_scan_block.write(output_dir / \"scanblock.fits\", overwrite=True)" ] }, { "cell_type": "markdown", "id": "3b6b6fd7-899a-4da7-a785-a3edaa38fc2b", "metadata": {}, "source": [ "## Reading Calibrated SDFITS\n", "\n", "To load the saved data, we use the same function we used to load the raw data, `GBTFITSLoad`." ] }, { "cell_type": "code", "execution_count": 22, "id": "505b441e-e8a7-4c44-b56d-69847754a628", "metadata": {}, "outputs": [], "source": [ "ps_scan_block_read = GBTFITSLoad(output_dir / \"scanblock.fits\")" ] }, { "cell_type": "markdown", "id": "b29e1434-8578-4a0d-8d5e-81f71b1b615d", "metadata": {}, "source": [ "This is treated the same was as the raw data, so the same methods are available." ] }, { "cell_type": "code", "execution_count": 23, "id": "c0a6106c-d698-4175-96be-392bc0c42a8a", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SCANOBJECTVELOCITYPROCPROCSEQNRESTFREQDOPFREQ# IF# POL# INT# FEEDAZIMUTHELEVATION
51NGC52914386.0OnOff11.4204051.42040511111198.343118.6427
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ps_scan_block_read.summary()" ] }, { "cell_type": "markdown", "id": "301550f7-a7da-4025-abf2-4ce16adc01b6", "metadata": {}, "source": [ "However, since the data is already calibrated, trying to fetch the data using calibration methods like `getps` or `getnod` will result in errors.\n", "Instead, we can use the `gettp` method or `getspec`.\n", "\n", "### Using `gettp`\n", "\n", "If we use `gettp`, then we will get all of the integrations as a `ScanBlock` object." ] }, { "cell_type": "code", "execution_count": 24, "id": "1d358fa9-d915-4186-b323-db64b160bb03", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "14:57:41.496 I Using TSYS column\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Number of integrations: 11\n" ] } ], "source": [ "tp_read = ps_scan_block_read.gettp(scan=51, ifnum=0, plnum=0, fdnum=0)\n", "print(f\"Number of integrations: {tp_read[0].nint}\")" ] }, { "cell_type": "markdown", "id": "d4d1ef53-73d1-4ced-bcae-bee91ecd8f3d", "metadata": {}, "source": [ "We can access individual integration through the `getspec` method." ] }, { "cell_type": "code", "execution_count": 25, "id": "595efc4f-9807-4dfa-8d0e-eaadd9730575", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "a9d8503fdf024392a1171fcd80981f9b", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ta_read_0a = tp_read[0].getspec(0)\n", "ta_read_0a.plot();" ] }, { "cell_type": "markdown", "id": "6c67f780-cf93-4db7-84d5-62e057bbbce8", "metadata": {}, "source": [ "### Using `getspec`\n", "\n", "Now we do the same using `getspec`. This method takes as input the row number we want to retrieve." ] }, { "cell_type": "code", "execution_count": 26, "id": "a41c4788-a80d-4964-a1fc-50d2d041fae6", "metadata": {}, "outputs": [], "source": [ "ta_read_0b = ps_scan_block_read.getspec(0)" ] }, { "cell_type": "code", "execution_count": 27, "id": "5190e455-3ce3-44df-b691-71727bc457bd", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "f324987638d543368bb795b4995b8829", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ta_read_0b.plot();" ] }, { "cell_type": "markdown", "id": "29fbfbb6-e7a9-4960-a987-e325460b182b", "metadata": {}, "source": [ "If we take the difference, it is zero." ] }, { "cell_type": "code", "execution_count": 28, "id": "4a043563-62e7-45a2-bd5d-5e8e1d865554", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "np.float64(0.0)" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "(ta_read_0a.data - ta_read_0b.data).sum()" ] }, { "cell_type": "markdown", "id": "edb47869-587b-47ff-9e66-23a59657fd1e", "metadata": {}, "source": [ "## Writing Out Selected Data from `GBTFITSLoad`\n", "The `write` method of `GBTFITSLoad` supports down-selection of data. \n", "Data can be selected on any SDFITS column.\n", "\n", "In the following call to `GBTFITSLoad.write` we will select a single polarization (`plnum=1`), and only the first five integrations of each scan (`intnum=range(5)`). We also set `overwrite=True` to avoid errors if the file already exists, and request that the output be saved into a single file (`multifile=False`)." ] }, { "cell_type": "code", "execution_count": 29, "id": "0c8a42c1-77e5-4aa9-88b5-ad88f7be0694", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG PLNUM INTNUM # SELECTED\n", "--- --------- ----- ---------- ----------\n", " 0 01f161d54 1 range(0,5) 80\n" ] } ], "source": [ "sdfits.write(output_dir / \"mydata.fits\", plnum=1, \n", " intnum=range(5), overwrite=True, multifile=False)" ] }, { "cell_type": "markdown", "id": "3c59504e-9393-45ae-afb9-da513191ff55", "metadata": {}, "source": [ "These data, can of course, be read back in." ] }, { "cell_type": "code", "execution_count": 30, "id": "4a0bcaa6-97d7-4e4c-94ef-50761abc1857", "metadata": {}, "outputs": [], "source": [ "sdfits2 = GBTFITSLoad(output_dir / \"mydata.fits\")" ] }, { "cell_type": "code", "execution_count": 31, "id": "0955e660-5ecc-4291-bc5f-d7adb57baeb8", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SCANOBJECTVELOCITYPROCPROCSEQNRESTFREQDOPFREQ# IF# POL# INT# FEEDAZIMUTHELEVATION
51NGC52914386.0OnOff11.4204051.4204051151198.232718.6739
52NGC52914386.0OnOff21.4204051.4204051151198.820018.8193
53NGC52914386.0OnOff11.4204051.4204051151199.220718.3889
54NGC52914386.0OnOff21.4204051.4204051151199.805818.5264
55NGC52914386.0OnOff11.4204051.4204051151200.195218.0919
56NGC52914386.0OnOff21.4204051.4204051151200.781518.2213
57NGC52914386.0OnOff11.4204051.4204051151202.220117.4228
58NGC52914386.0OnOff21.4204051.4204051151202.811717.5334
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sdfits2.summary()" ] }, { "cell_type": "markdown", "id": "80442074-dd89-44d4-9aa3-b3544a5dcf57", "metadata": {}, "source": [ "### Writing SDFITS to Multiple Files \n", "If the data came from multiple files, for instance VEGAS banks, then by default they are written to multiple files, so\n", "```\n", " sdfits.write('mydata.fits')\n", "```\n", "would write to mydata0.fits, mydata1.fits, ... mydataN.fits.\n", "\n", "If you want them organized for `GBTFITSLoad`, create a new directory first (and/or make sure this directory is clean):\n", "```\n", " import os\n", " os.mkdir(\"my\")\n", " sdfits.write(\"my/data.fits\")\n", "```\n", "which would write my/data0.fits, my/data1.fits, ... my/dataN.fits. This way `GBTFITSLoad` can more easily read them as a related group:\n", "```\n", " mysdf = GBTFITSLoad(\"my\")\n", "```\n", "See also the \n", "[Merging SDFITS Files](https://dysh.readthedocs.io/en/latest/users_guide/merge_sdfits.html)\n", "notebook how to deal merging SDFITS files.\n" ] }, { "cell_type": "markdown", "id": "0e888ef0-9a5e-4d56-bc1b-3ae5ea558e31", "metadata": {}, "source": [ "## Final Stats\n", "\n", "Finally, at the end we compute some statistics over a spectrum, merely as a checksum if the notebook is reproducible.\n" ] }, { "cell_type": "code", "execution_count": 32, "id": "5123eac9-179a-427b-a459-c7a480f2a29f", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "14:57:41.990 I rms is OK \n" ] } ], "source": [ "ta_read_0a.check_stats(0.26165166 * u.K)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.9" } }, "nbformat": 4, "nbformat_minor": 5 }