{ "cells": [ { "cell_type": "markdown", "id": "858af913-07e5-4b3f-8462-20ddb3fe6680", "metadata": {}, "source": [ "# Selecting Data\n", "\n", "This notebook shows how to select data in dysh.\n", "By selecting data, you can narrow down which scans or integrations the subsequent calibration routines will operate on.\n", "We call such narrowing down a \"selection rule.\"\n", "You create selection rules through methods of [``GBTFITSLoad``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#module-dysh.fits.gbtfitsload.GBTFITSLoad), which uses an instance of [``Selection``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.util.html#dysh.util.selection.Selection) as an attribute called ``selection``.\n", "You can create multiple selection rules that will be logically ANDed to create a final rule at calibration time.\n", "\n", "## Dysh commands\n", "\n", "The following dysh commands are introduced (leaving out all the function arguments):\n", "\n", " filename = dysh_data()\n", " sdf = GBTFITSLoad()\n", " sdf.select()\n", " sb = sdf.getps()\n", " ta = sb.timeaverage()\n", " ta.baseline()\n", " ta.average()\n", " ta.plot()\n", " ta_plt.savefig()\n", "\n", "## Loading Modules\n", "We start by loading the modules we will use in this notebook. " ] }, { "cell_type": "code", "execution_count": 1, "id": "126a61c2-8730-4194-a39a-3728f611dbd7", "metadata": {}, "outputs": [], "source": [ "# These modules are required for the notebook (no plotting is done)\n", "import astropy.units as u\n", "from astropy.time import Time\n", "from dysh.log import init_logging\n", "from dysh.fits.gbtfitsload import GBTFITSLoad\n", "from dysh.log import init_logging\n", "\n", "# These modules are used for file I/O\n", "from dysh.util.files import dysh_data\n", "from pathlib import Path" ] }, { "cell_type": "markdown", "id": "ab4b2cbd-7987-4a68-9f6f-ffea3c0ecafb", "metadata": {}, "source": [ "## Setup\n", "We start the dysh logging, so we get more information about what is happening.\n", "This is only needed if working on a notebook.\n", "If using the CLI through the dysh command, then logging is setup for you." ] }, { "cell_type": "code", "execution_count": 2, "id": "1b386a84-da2c-4d00-94ca-f980c9e49675", "metadata": {}, "outputs": [], "source": [ "init_logging(2)\n", "\n", "# also create a local \"output\" directory where temporary notebook files can be stored.\n", "output_dir = Path.cwd() / \"output\"\n", "output_dir.mkdir(exist_ok=True)" ] }, { "cell_type": "markdown", "id": "42421919-e8e3-42b9-8080-4b68dcef38f4", "metadata": {}, "source": [ "## Data Retrieval\n", "\n", "Download the example SDFITS data, if necessary.\n", "\n", "The code below will download an SDFITS file from [](http://www.gb.nrao.edu/dysh/example_data) and put it in a data directory.\n", "\n", "\n", "The code below will download an SDFITS file from http://www.gb.nrao.edu/dysh/example_data and put it in a data directory.\n", "The data directory must exist where this notebook is being run from, otherwise the downloaded SDFITS will be named data.\n", "The example will work either way, but be aware if you find a new file named data after running it." ] }, { "cell_type": "code", "execution_count": 3, "id": "41e76282-545a-4573-9bb7-38af50aa2a89", "metadata": { "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "19:59:30.771 I Resolving example=survey -> hi-survey/data/AGBT04A_008_02.raw.acs/AGBT04A_008_02.raw.acs.fits\n" ] } ], "source": [ "# hi-survey/data/AGBT04A_008_02.raw.acs/AGBT04A_008_02.raw.acs.fits\n", "filename = dysh_data(example=\"survey\")" ] }, { "cell_type": "markdown", "id": "493fadee-e596-4d2d-af68-86793f2148f9", "metadata": {}, "source": [ "## Data Loading\n", "\n", "Next, we use [``GBTFITSLoad``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#module-dysh.fits.gbtfitsload.GBTFITSLoad) to load the data, and then its [``summary``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#module-dysh.fits.gbtfitsload.GBTFITSLoad.summary) method to inspect its contents.\n", "We add the UTC column to the ``summary``." ] }, { "cell_type": "code", "execution_count": 4, "id": "9efd592a-c289-4219-82f1-a62cbfc72f04", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SCANOBJECTVELOCITYPROCPROCSEQNRESTFREQDOPFREQ# IF# POL# INT# FEEDAZIMUTHELEVATIONUTC
2203C2860.0OffOn11.4000001.4000001261185.280682.02462004-04-22 04:51:41.035000192
2213C2860.0OffOn21.4000001.4000001261187.213681.99802004-04-22 04:52:56.034999936
2223C2860.0OffOn11.4000001.4000001261193.833181.84132004-04-22 04:57:18.034999936
2233C2860.0OffOn21.4000001.4000001261195.676681.77882004-04-22 04:58:33.035000192
2243C2860.0OffOn11.4000001.4000001261195.518280.29102004-04-22 05:00:06.034999936
2253C2860.0OffOn21.4000001.4000001251199.935881.60052004-04-22 05:01:31.028000128
2263C2860.0OffOn11.4000001.4000001261200.833380.02652004-04-22 05:04:25.035000192
2273C2860.0OffOn21.4000001.4000001261205.947181.26092004-04-22 05:05:57.035000192
228B1328+2540.0OffOn11.4000001.4000001261207.525773.98442004-04-22 05:17:26.034999936
229B1328+2540.0OffOn21.4000001.4000001261210.960075.15842004-04-22 05:18:54.034999936
230B1345+1250.0OffOn11.4000001.40000012181193.273862.07032004-04-22 05:27:20.118333440
231B1345+1250.0OffOn21.4000001.40000012181195.679463.32442004-04-22 05:30:43.118333440
244B1345+1250.0OffOn11.4000001.40000012181200.954360.92842004-04-22 05:45:21.118333440
245B1345+1250.0OffOn21.4000001.40000012181203.531162.05872004-04-22 05:48:43.118333440
246B1345+1250.0OffOn11.4000001.40000012181204.340860.39302004-04-22 05:52:26.006111104
247B1345+1250.0OffOn21.4000001.40000012181206.976361.46552004-04-22 05:55:48.006111232
248B1345+1250.0OffOn11.3700001.37000012181208.040859.69832004-04-22 06:00:27.118333440
249B1345+1250.0OffOn21.3700001.37000012181210.721560.70602004-04-22 06:03:49.118333440
250B1345+1250.0Track11.3700001.3700001231215.243059.61192004-04-22 06:08:15.013333504
251B1345+1250.0Track11.3700001.3700001231215.969259.41752004-04-22 06:09:57.013333120
263U8091213.0OffOn11.4204051.42040512301241.133950.63932004-04-22 06:31:44.201666816
264U8091213.0OffOn21.4204051.42040512301240.979550.73952004-04-22 06:37:07.201666688
265U8091213.0Track11.4204051.4204051231243.480749.82342004-04-22 06:40:42.013332992
266U82492541.0OffOn11.4204051.42040512101241.706550.28012004-04-22 06:45:34.062999936
267U82492541.0OffOn11.4204051.42040512301242.689249.63432004-04-22 06:49:17.011333376
268U82492541.0OffOn21.4204051.42040512301242.545149.73332004-04-22 06:54:41.011333248
269U82492541.0Track11.4204051.4204051231244.943548.07872004-04-22 06:58:18.013332992
270U85034676.0OffOn11.4204051.42040512301270.174160.24852004-04-22 07:05:18.201666816
271U85034676.0OffOn21.4204051.42040512301269.939660.54492004-04-22 07:10:40.201666816
272U8091213.0OffOn11.4204051.42040512301253.578041.44202004-04-22 07:21:01.011333376
273U8091213.0OffOn21.4204051.42040512301253.458641.55172004-04-22 07:26:24.011333632
274U8091213.0Track11.4204051.4204051231255.579339.55412004-04-22 07:31:11.013333504
275U82492541.0OffOn11.4204051.42040512301266.546946.83772004-04-22 07:36:10.011333376
276U82492541.0OffOn21.4204051.42040512301266.373847.03882004-04-22 07:41:31.011333376
277U82492541.0Track11.4204051.4204051231267.942645.18632004-04-22 07:45:11.013333504
278U99654524.0OffOn11.4204051.42040512301221.558267.71822004-04-22 07:52:57.011333248
279U99654524.0OffOn21.4204051.42040512301221.207067.80702004-04-22 07:58:27.011333632
280U99654524.0Track11.4204051.4204051231222.926267.36572004-04-22 08:01:49.013333120
281U10351891.0OffOn11.4204051.42040512301221.295577.46682004-04-22 08:09:01.011333376
282U10351891.0OffOn21.4204051.42040512301220.492477.59222004-04-22 08:14:38.011333248
283U10351891.0Track11.4204051.4204051231227.903276.26442004-04-22 08:18:33.013333120
284U90074618.0OffOn11.4204051.42040512301248.663238.34972004-04-22 08:26:53.011333376
285U90074618.0OffOn21.4204051.42040512301248.554938.44332004-04-22 08:32:14.011333248
286U90074618.0Track11.4204051.4204051231250.557436.67872004-04-22 08:36:03.013333504
287U90075257.0OffOn11.4204051.42040512301234.295648.02092004-04-22 08:41:26.011333248
288U90075257.0OffOn21.4204051.42040512301234.166948.09372004-04-22 08:46:50.011333376
289U98035257.0OffOn11.4204051.42040512301264.961057.99972004-04-22 08:56:18.011333120
290U98035257.0OffOn21.4204051.42040512301264.727658.24832004-04-22 09:01:40.011333504
291U98035257.0Track11.4204051.4204051231266.522156.28842004-04-22 09:05:54.013332992
292U10351891.0OffOn11.4204051.42040512301252.164467.08472004-04-22 09:11:13.011333248
293U10351891.0OffOn21.4204051.42040512301251.813467.30382004-04-22 09:16:38.011333248
294U10351891.0Track11.4204051.4204051231255.376464.91842004-04-22 09:23:31.013333504
295U106292980.0OffOn11.4204051.42040512301233.947266.85512004-04-22 09:29:00.011333248
296U106292980.0OffOn21.4204051.42040512301233.599166.98462004-04-22 09:34:28.011333504
297U106292980.0OffOn11.4204051.42040512301238.358465.06142004-04-22 09:40:00.011333632
298U106292980.0OffOn21.4204051.42040512301238.044165.20062004-04-22 09:45:27.011333504
299U106292980.0Track11.4204051.4204051231241.440563.61312004-04-22 09:49:02.013332992
300U110174644.0OffOn11.4204051.42040512301235.460576.20742004-04-22 09:54:06.011333248
301U110174644.0OffOn21.4204051.42040512301234.772676.38792004-04-22 09:59:39.011333376
302U110174644.0OffOn11.4204051.42040512301241.519774.35142004-04-22 10:05:11.011333504
303U110174644.0OffOn21.4204051.42040512301240.959074.54712004-04-22 10:10:43.011333632
304U110174644.0Track11.4204051.4204051231242.664173.94252004-04-22 10:14:13.013333120
305U110174644.0Track11.4204051.4204051211242.935973.84082004-04-22 10:14:48
306U110174644.0Track11.4204051.4204051231246.051172.57382004-04-22 10:16:10.013332992
307U114613122.0OffOn11.4204051.42040512301168.053159.89752004-04-22 10:25:22.011333120
308U114613122.0OffOn21.4204051.42040512301167.889459.88432004-04-22 10:30:53.011333376
309U114613122.0OffOn11.4204051.42040512301173.469860.24432004-04-22 10:36:22.011333248
310U114613122.0OffOn21.4204051.42040512301173.311160.23762004-04-22 10:41:54.011333120
311U114613122.0Track11.4204051.4204051231178.120760.38022004-04-22 10:45:45.013333120
312U115784601.0OffOn11.4204051.42040512301156.357358.80362004-04-22 10:52:22.011333248
313U115784601.0OffOn21.4204051.42040512301156.179658.77312004-04-22 10:57:50.011333248
314U115784601.0Track11.4204051.4204051231160.446959.44642004-04-22 11:01:18.999999872
315U115784601.0Track11.4204051.4204051231161.004059.52312004-04-22 11:02:30.013332992
316U116274864.0OffOn11.4204051.42040512301157.806555.33722004-04-22 11:07:12.011333632
317U116274864.0OffOn21.4204051.42040512301157.654955.31072004-04-22 11:12:39.011333504
318U116274864.0OffOn11.4204051.42040512301162.462556.06552004-04-22 11:18:07.011333248
319U116274864.0OffOn21.4204051.42040512301162.319956.04652004-04-22 11:23:36.011333376
320U116274864.0Track11.4204051.4204051231166.798356.58022004-04-22 11:28:00.013333376
321U119923592.0Track11.4204051.4204051231124.476254.32422004-04-22 11:32:01.013333120
322U119923592.0OffOn11.4204051.42040512301125.632054.91282004-04-22 11:35:30.011333120
323U119923592.0OffOn21.4204051.42040512301125.456554.82512004-04-22 11:40:55.011333504
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sdfits = GBTFITSLoad(filename)\n", "sdfits.summary(add_columns=[\"UTC\"])" ] }, { "cell_type": "markdown", "id": "2a1ce21a-2dbb-4672-aadf-0774f2b3e8db", "metadata": {}, "source": [ "## Using Selection\n", "\n", "Now we show various ways in which [``GBTFITSLoad.selection``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#dysh.fits.gbtfitsload.GBTFITSLoad.selection) can be used to select data.\n", "\n", "### Select by column value\n", "\n", "One way of selecting data is by specifying a value for an SDFITS column name. (The column name case insensitive, but the value is not). \n", "For example, we can select data which has OBJECT=\"U8249\" or OBJECT=\"U8249\" using the following.\n", "\n", "**Note** Selecting values in a list will logically OR those values. So `object=[\"U8249\",\"U11017\"]` will select scans with either object. But multiple keywords (columns) will be logically ANDed (see below)." ] }, { "cell_type": "code", "execution_count": 5, "id": "72e0e07f-ccaa-4a02-8685-7641136fafc4", "metadata": {}, "outputs": [], "source": [ "sdfits.select(object=[\"U8249\",\"U11017\"])" ] }, { "cell_type": "markdown", "id": "61173765-201b-4079-8ca4-5294a2a8f089", "metadata": {}, "source": [ "We can view the contents of the selection using its `show` method.\n", "This displays the selection as a table.\n", "The `# Selected` column gives the number of records (integrations) selected by each selection rule. \n", "Each time we create a new selection, it is assigned a unique id and tag." ] }, { "cell_type": "code", "execution_count": 6, "id": "fd4d1a79-6c10-49f8-a10c-0012ce93d7cb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG OBJECT # SELECTED\n", "--- --------- ------------------ ----------\n", " 0 4d8020b72 ['U8249','U11017'] 572\n" ] } ], "source": [ "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "5c974128-e99d-4967-b404-5d8f29ec1723", "metadata": {}, "source": [ "We can also specify the tag name to have a more meaningful value. " ] }, { "cell_type": "code", "execution_count": 7, "id": "c5e0ca48-6b76-414d-8dac-50daab9ddbae", "metadata": {}, "outputs": [], "source": [ "sdfits.select(proc=\"OffOn\", tag='proc onoff')" ] }, { "cell_type": "code", "execution_count": 8, "id": "fc0e2a0f-0011-4189-9d5f-2352891b43f6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG OBJECT PROC # SELECTED\n", "--- ---------- ------------------ ----- ----------\n", " 0 4d8020b72 ['U8249','U11017'] 572\n", " 1 proc onoff OffOn 3540\n" ] } ], "source": [ "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "75e62980-d381-4358-90a1-2c6789aa5d40", "metadata": {}, "source": [ "### Combining Selections\n", "\n", "Once we have multiple selection rules in the ``Selection`` object, we can combine them into a single selection using the ``final`` property. \n", "This will return a [``pandas.DataFrame``](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html)." ] }, { "cell_type": "code", "execution_count": 9, "id": "454b8b20-5bb2-4146-9da5-85afc639e4e8", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTBANDWIDDATE-OBSDURATIONEXPOSURETSYSTDIM7TUNIT7CTYPE1CRVAL1...SITELATSITEELEVEXTNAMEFITSINDEXUTCCHANPROCOBSTYPESUBOBSMODEINTNUM
0U824912500000.02004-04-22T06:44:49.005.0054.7794881.0(32768,1,1,1)CountsFREQ-OBS1.408421e+09...38.43312824.595SINGLE DISH02004-04-22 06:44:49.000NoneOffOnPSWITCHOFFTPWCAL0
1U824912500000.02004-04-22T06:44:49.005.0054.7794881.0(32768,1,1,1)CountsFREQ-OBS1.408421e+09...38.43312824.595SINGLE DISH02004-04-22 06:44:49.000NoneOffOnPSWITCHOFFTPWCAL0
2U824912500000.02004-04-22T06:44:49.005.0054.7794881.0(32768,1,1,1)CountsFREQ-OBS1.408421e+09...38.43312824.595SINGLE DISH02004-04-22 06:44:49.000NoneOffOnPSWITCHOFFTPWCAL0
3U824912500000.02004-04-22T06:44:49.005.0054.7794881.0(32768,1,1,1)CountsFREQ-OBS1.408421e+09...38.43312824.595SINGLE DISH02004-04-22 06:44:49.000NoneOffOnPSWITCHOFFTPWCAL0
4U824912500000.02004-04-22T06:44:59.015.0054.7794881.0(32768,1,1,1)CountsFREQ-OBS1.408421e+09...38.43312824.595SINGLE DISH02004-04-22 06:44:59.010NoneOffOnPSWITCHOFFTPWCAL1
..................................................................
515U1101712500000.02004-04-22T10:12:48.0210.0009.5967091.0(32768,1,1,1)CountsFREQ-OBS1.398806e+09...38.43312824.595SINGLE DISH02004-04-22 10:12:48.020NoneOffOnPSWITCHONTPNOCAL27
516U1101712500000.02004-04-22T10:12:58.0210.0009.5967091.0(32768,1,1,1)CountsFREQ-OBS1.398806e+09...38.43312824.595SINGLE DISH02004-04-22 10:12:58.020NoneOffOnPSWITCHONTPNOCAL28
517U1101712500000.02004-04-22T10:12:58.0210.0009.5967091.0(32768,1,1,1)CountsFREQ-OBS1.398806e+09...38.43312824.595SINGLE DISH02004-04-22 10:12:58.020NoneOffOnPSWITCHONTPNOCAL28
518U1101712500000.02004-04-22T10:13:08.0210.0009.5967091.0(32768,1,1,1)CountsFREQ-OBS1.398806e+09...38.43312824.595SINGLE DISH02004-04-22 10:13:08.020NoneOffOnPSWITCHONTPNOCAL29
519U1101712500000.02004-04-22T10:13:08.0210.0009.5967091.0(32768,1,1,1)CountsFREQ-OBS1.398806e+09...38.43312824.595SINGLE DISH02004-04-22 10:13:08.020NoneOffOnPSWITCHONTPNOCAL29
\n", "

520 rows × 94 columns

\n", "
" ], "text/plain": [ " OBJECT BANDWID DATE-OBS DURATION EXPOSURE TSYS \\\n", "0 U8249 12500000.0 2004-04-22T06:44:49.00 5.005 4.779488 1.0 \n", "1 U8249 12500000.0 2004-04-22T06:44:49.00 5.005 4.779488 1.0 \n", "2 U8249 12500000.0 2004-04-22T06:44:49.00 5.005 4.779488 1.0 \n", "3 U8249 12500000.0 2004-04-22T06:44:49.00 5.005 4.779488 1.0 \n", "4 U8249 12500000.0 2004-04-22T06:44:59.01 5.005 4.779488 1.0 \n", ".. ... ... ... ... ... ... \n", "515 U11017 12500000.0 2004-04-22T10:12:48.02 10.000 9.596709 1.0 \n", "516 U11017 12500000.0 2004-04-22T10:12:58.02 10.000 9.596709 1.0 \n", "517 U11017 12500000.0 2004-04-22T10:12:58.02 10.000 9.596709 1.0 \n", "518 U11017 12500000.0 2004-04-22T10:13:08.02 10.000 9.596709 1.0 \n", "519 U11017 12500000.0 2004-04-22T10:13:08.02 10.000 9.596709 1.0 \n", "\n", " TDIM7 TUNIT7 CTYPE1 CRVAL1 ... SITELAT SITEELEV \\\n", "0 (32768,1,1,1) Counts FREQ-OBS 1.408421e+09 ... 38.43312 824.595 \n", "1 (32768,1,1,1) Counts FREQ-OBS 1.408421e+09 ... 38.43312 824.595 \n", "2 (32768,1,1,1) Counts FREQ-OBS 1.408421e+09 ... 38.43312 824.595 \n", "3 (32768,1,1,1) Counts FREQ-OBS 1.408421e+09 ... 38.43312 824.595 \n", "4 (32768,1,1,1) Counts FREQ-OBS 1.408421e+09 ... 38.43312 824.595 \n", ".. ... ... ... ... ... ... ... \n", "515 (32768,1,1,1) Counts FREQ-OBS 1.398806e+09 ... 38.43312 824.595 \n", "516 (32768,1,1,1) Counts FREQ-OBS 1.398806e+09 ... 38.43312 824.595 \n", "517 (32768,1,1,1) Counts FREQ-OBS 1.398806e+09 ... 38.43312 824.595 \n", "518 (32768,1,1,1) Counts FREQ-OBS 1.398806e+09 ... 38.43312 824.595 \n", "519 (32768,1,1,1) Counts FREQ-OBS 1.398806e+09 ... 38.43312 824.595 \n", "\n", " EXTNAME FITSINDEX UTC CHAN PROC OBSTYPE \\\n", "0 SINGLE DISH 0 2004-04-22 06:44:49.000 None OffOn PSWITCHOFF \n", "1 SINGLE DISH 0 2004-04-22 06:44:49.000 None OffOn PSWITCHOFF \n", "2 SINGLE DISH 0 2004-04-22 06:44:49.000 None OffOn PSWITCHOFF \n", "3 SINGLE DISH 0 2004-04-22 06:44:49.000 None OffOn PSWITCHOFF \n", "4 SINGLE DISH 0 2004-04-22 06:44:59.010 None OffOn PSWITCHOFF \n", ".. ... ... ... ... ... ... \n", "515 SINGLE DISH 0 2004-04-22 10:12:48.020 None OffOn PSWITCHON \n", "516 SINGLE DISH 0 2004-04-22 10:12:58.020 None OffOn PSWITCHON \n", "517 SINGLE DISH 0 2004-04-22 10:12:58.020 None OffOn PSWITCHON \n", "518 SINGLE DISH 0 2004-04-22 10:13:08.020 None OffOn PSWITCHON \n", "519 SINGLE DISH 0 2004-04-22 10:13:08.020 None OffOn PSWITCHON \n", "\n", " SUBOBSMODE INTNUM \n", "0 TPWCAL 0 \n", "1 TPWCAL 0 \n", "2 TPWCAL 0 \n", "3 TPWCAL 0 \n", "4 TPWCAL 1 \n", ".. ... ... \n", "515 TPNOCAL 27 \n", "516 TPNOCAL 28 \n", "517 TPNOCAL 28 \n", "518 TPNOCAL 29 \n", "519 TPNOCAL 29 \n", "\n", "[520 rows x 94 columns]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sdfits.selection.final" ] }, { "cell_type": "markdown", "id": "5666395f-3b8a-420e-9e15-a69c57f82f90", "metadata": {}, "source": [ "In this particular case, we wind up with 520 integrations. This is a selection of objects U8249 or U11017 AND proc OnOff. Because keywords in the same selection rule are logically ANDed, this could also have been accomplished via\n", "\n", "`sdfits.select(object=[\"U8249\",\"U11017\"], proc=\"OffOn\")`\n", "\n", "(Try it yourself)." ] }, { "cell_type": "markdown", "id": "9571b2b8-ffb2-4cf8-b058-09c898389189", "metadata": {}, "source": [ "You can also see a summary of the selected data, using the ``selected`` argument of [``summary``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#module-dysh.fits.gbtfitsload.GBTFITSLoad.summary):" ] }, { "cell_type": "code", "execution_count": 10, "id": "1521b0f9-7b93-4502-9031-da49dc99a147", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SCANOBJECTVELOCITYPROCPROCSEQNRESTFREQDOPFREQ# IF# POL# INT# FEEDAZIMUTHELEVATION
266U82492541.0OffOn11.4204051.42040512101241.706550.2801
267U82492541.0OffOn11.4204051.42040512301242.689249.6343
268U82492541.0OffOn21.4204051.42040512301242.545149.7333
275U82492541.0OffOn11.4204051.42040512301266.546946.8377
276U82492541.0OffOn21.4204051.42040512301266.373847.0388
300U110174644.0OffOn11.4204051.42040512301235.460576.2074
301U110174644.0OffOn21.4204051.42040512301234.772676.3879
302U110174644.0OffOn11.4204051.42040512301241.519774.3514
303U110174644.0OffOn21.4204051.42040512301240.959074.5471
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sdfits.summary(selected=True)\n" ] }, { "cell_type": "markdown", "id": "6f4cb3ef-f1ab-42bf-8a29-cfd29929cca9", "metadata": {}, "source": [ "### Remove selection rules\n", "You can remove a selection rule by `id` or `tag`.\n", "Multiple rows with the same tag will all be removed." ] }, { "cell_type": "code", "execution_count": 11, "id": "43fe5ffb-2821-4ee4-9f29-ec245168ae55", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG PROC # SELECTED\n", "--- ---------- ----- ----------\n", " 1 proc onoff OffOn 3540\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING: AstropyDeprecationWarning: Calling `Table.loc/iloc/loc_indices[index_id, item]` to select `item` from index\n", "`index_id` is deprecated. Instead select the index using the syntax\n", "`Table.loc/iloc/loc_indices.with_index(index_id)[item]`.\n", " [astropy.table.index]\n" ] } ], "source": [ "sdfits.selection.remove(id=0)\n", "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "b6b25468-efca-4bd1-91ef-a34e6eb502f5", "metadata": {}, "source": [ "To remove all selection rules use [``clear_selection``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#dysh.fits.GBTFITSLoad.clear_selection).\n", "After using it, the ``Selection`` will show no selection rules." ] }, { "cell_type": "code", "execution_count": 12, "id": "a771e05c-9532-4139-b4a8-3b7f25c639e1", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG OBJECT BANDWID DATE-OBS ... SUBOBSMODE FITSINDEX CHAN UTC # SELECTED\n", "--- --- ------ ------- -------- ... ---------- --------- ---- --- ----------\n" ] } ], "source": [ "sdfits.clear_selection()\n", "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "97ca64ec-ca06-47c6-a0a4-caf09db3a440", "metadata": {}, "source": [ "### Select by Range\n", "\n", "It is also possible to define a selection given a range of values using [``select_range``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.util.html#dysh.util.selection.Selection.select_range).\n", "In this case the selection must be specified using either a list, ``[]``, or a tuple, ``()``, with a start and an end value. Ranges are considered inclusive of both ends.\n", "Lower limits are give by ``(value,None)`` or ``(value,)``.\n", "Upper limits are given by ``(None,value)``, since ``(,value)`` is not valid Python.\n", "For coordinates the default unit is taken to be degrees.\n", "Other units can be explicitly given.\n", "Both ``()`` and ``[]`` are valid for indicated ranges, but only tuples can be a lower limit ``(value,)``.\n", "\n", "For example to select only rows where the right ascension is greater than 114 degrees:" ] }, { "cell_type": "code", "execution_count": 13, "id": "6c627654-19bc-4151-b2de-222efb3e4398", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG CRVAL2 # SELECTED\n", "--- ----------- ------------------- ----------\n", " 0 RA>=114 deg [np.float64(114.0)] 3766\n" ] } ], "source": [ "sdfits.select_range(ra=(114,), tag=\"RA>=114 deg\")\n", "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "eda04fe5-98f3-40c6-8f03-2658c023216d", "metadata": {}, "source": [ "(Right Ascension is the FITS CRVAL2 column).\n", "\n", "To select rows where the elevation is below 80 degrees:" ] }, { "cell_type": "code", "execution_count": 14, "id": "50faf345-a76d-4b45-a743-d9610565122d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG CRVAL2 ELEVATIO # SELECTED\n", "--- ----------- ------------------- --------- ----------\n", " 0 RA>=114 deg [np.float64(114.0)] 3766\n", " 1 EL<80 [None,80] 3582\n" ] } ], "source": [ "sdfits.select_range(elevation=[None,80], tag=\"EL<80\")\n", "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "4c387a31-c0cb-45f3-b512-36b6af576cd2", "metadata": {}, "source": [ "(Note elevation column is ELEVATIO because the FITS standard only allow 8 characters for column names).\n", "\n", "We can check that the selections were applied properly by inspecting the final result and a subset of its columns. " ] }, { "cell_type": "code", "execution_count": 15, "id": "f19448b1-3eda-49af-bebe-b38b0f1213f0", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTCRVAL2ELEVATIO
03C286202.78450879.997019
13C286202.78450879.997019
23C286202.78450879.997019
33C286202.78450879.997019
4B1328+254202.06632474.023075
............
3577U11992335.19751755.157387
3578U11992335.19752055.183819
3579U11992335.19752055.183819
3580U11992335.19754655.210211
3581U11992335.19754655.210211
\n", "

3582 rows × 3 columns

\n", "
" ], "text/plain": [ " OBJECT CRVAL2 ELEVATIO\n", "0 3C286 202.784508 79.997019\n", "1 3C286 202.784508 79.997019\n", "2 3C286 202.784508 79.997019\n", "3 3C286 202.784508 79.997019\n", "4 B1328+254 202.066324 74.023075\n", "... ... ... ...\n", "3577 U11992 335.197517 55.157387\n", "3578 U11992 335.197520 55.183819\n", "3579 U11992 335.197520 55.183819\n", "3580 U11992 335.197546 55.210211\n", "3581 U11992 335.197546 55.210211\n", "\n", "[3582 rows x 3 columns]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sdfits.selection.final[[\"OBJECT\",\"CRVAL2\",\"ELEVATIO\"]]" ] }, { "cell_type": "markdown", "id": "b7313f50-6a0b-450a-a714-f23b6c0cdb99", "metadata": {}, "source": [ "It is also possible to use units during selection.\n", "For example" ] }, { "cell_type": "code", "execution_count": 16, "id": "2e187bff-c9eb-4391-a709-d80f60a90f1a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG CRVAL2 ... ELEVATIO # SELECTED\n", "--- ----------------- ------------------- ... --------- ----------\n", " 0 RA>=114 deg [np.float64(114.0)] ... 3766\n", " 1 EL<80 ... [None,80] 3582\n", " 2 14.23<=DEC<=14.25 ... 132\n" ] } ], "source": [ "sdfits.select_range(dec=[854, 855] * u.arcmin, tag=\"14.23<=DEC<=14.25\")\n", "sdfits.selection.show()" ] }, { "cell_type": "code", "execution_count": 17, "id": "25f790c7-ddd3-4825-b6ec-e03635de6833", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SCANOBJECTVELOCITYPROCPROCSEQNRESTFREQDOPFREQ# IF# POL# INT# FEEDAZIMUTHELEVATION
321U119923592.0Track11.4204051.4204051231124.476254.3242
322U119923592.0OffOn11.4204051.42040512301125.632054.9128
323U119923592.0OffOn21.4204051.42040512301125.456554.8251
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sdfits.summary(selected=True)" ] }, { "cell_type": "markdown", "id": "b84e82b3-f17d-4ca0-a9d1-fa63f4207c1e", "metadata": {}, "source": [ "### Select Within a Range\n", "\n", "It is also possible to specify the midpoint and a range to make a selection.\n", "In this case we use [``select_within``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.util.html#dysh.util.selection.Selection.select_within) and specify the mean value and the +- range.\n", "\n", "For example to select between elevation of 50-10 and 50+10 we would use" ] }, { "cell_type": "code", "execution_count": 18, "id": "9c0b64e5-0f7c-4c92-a103-c650938d4d53", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG CRVAL2 ... ELEVATIO # SELECTED\n", "--- ----------------- ------------------- ... --------- ----------\n", " 0 RA>=114 deg [np.float64(114.0)] ... 3766\n", " 1 EL<80 ... [None,80] 3582\n", " 2 14.23<=DEC<=14.25 ... 132\n", " 3 EL=50+/-10 ... [40,60] 1694\n" ] } ], "source": [ "sdfits.select_within(elevation=(50,10), tag=\"EL=50+/-10\")\n", "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "e6c8562e-3a1f-46f1-ba32-d7b0b1f2ad33", "metadata": {}, "source": [ "Which shows a selection between 40 and 60 degrees of elevation." ] }, { "cell_type": "markdown", "id": "a4b5d4e3-fe74-49d4-8a96-0cf6bad8d038", "metadata": {}, "source": [ "Now, when you do a [``getps``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#dysh.fits.GBTFITSLoad.getps) it will operate only on the selected data.\n", "\n", "(Currently, you can't preselect [``ifnum``](https://dysh.readthedocs.io/en/latest/reference/glossary.html#term-ifnum), [``plnum``](https://dysh.readthedocs.io/en/latest/reference/glossary.html#term-plnum), or [``fdnum``](https://dysh.readthedocs.io/en/latest/reference/glossary.html#term-fdnum); they must be provided as method arguments)." ] }, { "cell_type": "code", "execution_count": 19, "id": "f624409e-1862-4e04-beff-33a51ce386bc", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "19:59:32.055 I Using TSYS column\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "3c9ba15dbbe74dca87ce7f6393ad0211", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sb = sdfits.getps(ifnum=0, plnum=0, fdnum=0)\n", "sb.timeaverage().plot(ymin=-.1, ymax=.1, xmin=1.404E9, xmax=1.406E9);" ] }, { "cell_type": "markdown", "id": "1c7c7215-fe5e-4792-9f95-9589aabda054", "metadata": {}, "source": [ "### Using Aliases\n", "\n", "[``Selection``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.util.html#dysh.util.selection.Selection) knows about certain aliases for column names.\n", "For example, the SDFITS column ELEVATIO can also be selected using ELEVATION.\n", "The aliases are defined in the ``aliases`` attribute of ``Selection``." ] }, { "cell_type": "code", "execution_count": 20, "id": "28b372be-4e93-49b5-85c8-eafbf2a0d881", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'FREQ': 'CRVAL1',\n", " 'RA': 'CRVAL2',\n", " 'DEC': 'CRVAL3',\n", " 'GLON': 'CRVAL2',\n", " 'GLAT': 'CRVAL3',\n", " 'GALLON': 'CRVAL2',\n", " 'GALLAT': 'CRVAL3',\n", " 'ELEVATION': 'ELEVATIO',\n", " 'SOURCE': 'OBJECT',\n", " 'POL': 'PLNUM',\n", " 'SUBREF': 'SUBREF_STATE'}" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sdfits.selection.aliases" ] }, { "cell_type": "markdown", "id": "ca77655a-b48c-48b9-9f26-a3697a9d723b", "metadata": {}, "source": [ "It is also possible to add your own aliases.\n", "For example to use target and az as aliases for OBJECT and AZIMUTH we would use" ] }, { "cell_type": "code", "execution_count": 21, "id": "59eccb75-8ce7-45fb-9ada-ee65128c5fb7", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "{'FREQ': 'CRVAL1',\n", " 'RA': 'CRVAL2',\n", " 'DEC': 'CRVAL3',\n", " 'GLON': 'CRVAL2',\n", " 'GLAT': 'CRVAL3',\n", " 'GALLON': 'CRVAL2',\n", " 'GALLAT': 'CRVAL3',\n", " 'ELEVATION': 'ELEVATIO',\n", " 'SOURCE': 'OBJECT',\n", " 'POL': 'PLNUM',\n", " 'SUBREF': 'SUBREF_STATE',\n", " 'TARGET': 'OBJECT',\n", " 'AZ': 'AZIMUTH'}" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sdfits.selection.alias({'target':'object','az':'azimuth'})\n", "sdfits.selection.aliases" ] }, { "cell_type": "markdown", "id": "16cd927d-fa71-4e1d-bed8-68a4a3eef9d1", "metadata": {}, "source": [ "Then you can select using your aliases:" ] }, { "cell_type": "code", "execution_count": 22, "id": "d9d3ebf7-1b21-49b1-b44f-f9ec1e78e89d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG OBJECT ... ELEVATIO # SELECTED\n", "--- ----------------- ------ ... --------- ----------\n", " 0 RA>=114 deg ... 3766\n", " 1 EL<80 ... [None,80] 3582\n", " 2 14.23<=DEC<=14.25 ... 132\n", " 3 EL=50+/-10 ... [40,60] 1694\n", " 4 9eba1ed60 U8249 ... 304\n" ] } ], "source": [ "sdfits.select(target=\"U8249\")\n", "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "7d387de3-4cf8-40cc-a846-5b216454cd43", "metadata": {}, "source": [ "Notice that this will only affect the aliases for this particular instance of a [``GBTFITSLoad``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#module-dysh.fits.gbtfitsload.GBTFITSLoad).\n", "Any new [``GBTFITSLoad``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.fits.html#module-dysh.fits.gbtfitsload.GBTFITSLoad) objects will not know about these aliases." ] }, { "cell_type": "markdown", "id": "7279270f-8cc5-458c-817a-803dd808045f", "metadata": {}, "source": [ "### Empty Selections\n", "\n", "Any selection that results in no data being selected is ignored.\n", "You will get a warning message in this case." ] }, { "cell_type": "code", "execution_count": 23, "id": "012adfe6-6900-438c-b45e-681cabd62993", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "19:59:32.771 W Your selection rule resulted in no data being selected. Ignoring.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " ID TAG OBJECT ... ELEVATIO # SELECTED\n", "--- ----------------- ------ ... --------- ----------\n", " 0 RA>=114 deg ... 3766\n", " 1 EL<80 ... [None,80] 3582\n", " 2 14.23<=DEC<=14.25 ... 132\n", " 3 EL=50+/-10 ... [40,60] 1694\n", " 4 9eba1ed60 U8249 ... 304\n" ] } ], "source": [ "sdfits.selection.select(target='foobar')\n", "sdfits.selection.show()" ] }, { "cell_type": "markdown", "id": "3a716b3e-d624-486e-b7b2-2fe8fd967312", "metadata": {}, "source": [ "### Time Selections\n", "\n", "UTC time ranges can be selected with [``astropy.time.Time``](https://docs.astropy.org/en/stable/api/astropy.time.Time.html#astropy.time.Time) objects.\n", "This checks against the UTC timestamp column.\n", "For LST, use ``select_range(lst=[number1,number2])``.\n", "\n" ] }, { "cell_type": "code", "execution_count": 24, "id": "093a7b37-935c-4f42-b5ff-599b74bbf6d6", "metadata": {}, "outputs": [], "source": [ "# clear the selection for this demonstration\n", "sdfits.clear_selection()" ] }, { "cell_type": "code", "execution_count": 25, "id": "48ffada1-3ef7-422c-8e2f-59ace6161f57", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG ... # SELECTED\n", "--- ---------- ... ----------\n", " 0 time range ... 216\n" ] } ], "source": [ "sdfits.select_range(utc=(Time(\"2004-04-22T05:27:20.12\", scale=\"utc\"),\n", " Time(\"2004-04-22T05:48:43.12\", scale=\"utc\")),\n", " tag=\"time range\")\n", "sdfits.selection.show()" ] }, { "cell_type": "code", "execution_count": 26, "id": "17cca444-0498-4bd4-b52f-986f014f19c7", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SCANOBJECTUTCLSTPLNUMIFNUMFDNUM
0230B1345+1252004-04-22 05:27:25.13051035.015413100
1230B1345+1252004-04-22 05:27:25.13051035.015413100
2230B1345+1252004-04-22 05:27:25.13051035.015413000
3230B1345+1252004-04-22 05:27:25.13051035.015413000
4230B1345+1252004-04-22 05:27:35.14051045.056730100
........................
211245B1345+1252004-04-22 05:48:28.10052301.445514000
212245B1345+1252004-04-22 05:48:38.11052311.486833100
213245B1345+1252004-04-22 05:48:38.11052311.486833100
214245B1345+1252004-04-22 05:48:38.11052311.486833000
215245B1345+1252004-04-22 05:48:38.11052311.486833000
\n", "

216 rows × 7 columns

\n", "
" ], "text/plain": [ " SCAN OBJECT UTC LST PLNUM IFNUM \\\n", "0 230 B1345+125 2004-04-22 05:27:25.130 51035.015413 1 0 \n", "1 230 B1345+125 2004-04-22 05:27:25.130 51035.015413 1 0 \n", "2 230 B1345+125 2004-04-22 05:27:25.130 51035.015413 0 0 \n", "3 230 B1345+125 2004-04-22 05:27:25.130 51035.015413 0 0 \n", "4 230 B1345+125 2004-04-22 05:27:35.140 51045.056730 1 0 \n", ".. ... ... ... ... ... ... \n", "211 245 B1345+125 2004-04-22 05:48:28.100 52301.445514 0 0 \n", "212 245 B1345+125 2004-04-22 05:48:38.110 52311.486833 1 0 \n", "213 245 B1345+125 2004-04-22 05:48:38.110 52311.486833 1 0 \n", "214 245 B1345+125 2004-04-22 05:48:38.110 52311.486833 0 0 \n", "215 245 B1345+125 2004-04-22 05:48:38.110 52311.486833 0 0 \n", "\n", " FDNUM \n", "0 0 \n", "1 0 \n", "2 0 \n", "3 0 \n", "4 0 \n", ".. ... \n", "211 0 \n", "212 0 \n", "213 0 \n", "214 0 \n", "215 0 \n", "\n", "[216 rows x 7 columns]" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sdfits.selection.final[[\"SCAN\",\"OBJECT\",\"UTC\", \"LST\",\"PLNUM\",\"IFNUM\",\"FDNUM\"]]" ] }, { "cell_type": "markdown", "id": "88044ad7-11d2-43f4-834f-cd7aba37787b", "metadata": {}, "source": [ "### Channel Selection\n", "\n", "You can selection a contiguous range of channels using [``select_channel``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.util.html#dysh.util.selection.Selection.select_channel), and the integrations will be trimmed to that channel range during calibration. The final spectrum will have the input channel range. As with [``select_range``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.util.html#dysh.util.selection.Selection.select_range), channel ranges are inclusive at both ends." ] }, { "cell_type": "code", "execution_count": 27, "id": "91871580-d678-44ea-828d-c823e92c8746", "metadata": {}, "outputs": [], "source": [ "sdfits.select_channel([2000,6000], tag=\"channels\")" ] }, { "cell_type": "code", "execution_count": 28, "id": "dfbb03ba-4eb0-40f0-b5e3-f2ce4ba23c8b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " ID TAG ... # SELECTED\n", "--- ---------- ... ----------\n", " 0 time range ... 216\n", " 1 channels ... 3766\n" ] } ], "source": [ "sdfits.selection.show()" ] }, { "cell_type": "code", "execution_count": 29, "id": "9fadaa78-52c4-4bd7-be63-5dd84a0a7478", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "19:59:33.096 I Ignoring 9 blanked integration(s).\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "4f1cb2a74d584676baee87d078155068", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='Clear All Regions', style=ButtonStyle(), tooltip='Clear all …" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sb = sdfits.getps(ifnum=0, plnum=1, fdnum=0)\n", "sb.timeaverage().plot(xaxis_unit=\"channel\");" ] }, { "cell_type": "markdown", "id": "663681b2-c3e4-46d1-ba6c-44b8374b54e3", "metadata": {}, "source": [ "The resulting [``Spectrum``](https://dysh.readthedocs.io/en/latest/reference/modules/dysh.spectra.html#dysh.spectra.spectrum.Spectrum) only has 4000 channels, as specified by the channel selection.\n", "\n", "Note that you can only have one channel selection rule at a time." ] }, { "cell_type": "code", "execution_count": 30, "id": "d3588cb3-83aa-4521-a325-2c9b8bb032a1", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "You can only have one channel selection rule. Remove the old rule before creating a new one.\n" ] } ], "source": [ "try: \n", " sdfits.select_channel([60,70])\n", "except Exception as e:\n", " print(e)" ] }, { "cell_type": "markdown", "id": "86608116-e842-4b16-8b4c-b1157574527a", "metadata": {}, "source": [ "## Final Stats\n", "\n", "Finally, at the end we compute some statistics over a spectrum, merely as a checksum if the notebook is reproducible.\n", "\n", "This particular case has a mixed number of scanblocks, so we just pick the first.\n" ] }, { "cell_type": "code", "execution_count": 31, "id": "c72974d1-aa5e-4abf-ba39-ebc7d2a4f681", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "19:59:33.635 I rms is OK \n" ] } ], "source": [ "sb.timeaverage().check_stats(0.05038713 * u.K)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.9" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 5 }