{"cells":[{"cell_type":"markdown","id":"5f7ef32b","metadata":{"id":"5f7ef32b"},"source":["**CRIB SHEET RULES OF THE ROAD:**\n","\n","This crib sheet is provided to support access, utilization, and plotting of UCalgary optical datasets. It is intended as a base set of code that a user may edit and manipulate to serve their own needs. Crib sheets contains UCalgary verified and validated procedures for plotting and manipulating UCalgary ASI data for common use cases. Use of this crib sheet does not require acknowledgment, it is freely distributed for scientific use. Please also remember to perform due diligence on all data use. We recommend comparison with verified data products on [data.phys.ucalgary.ca](https://data.phys.ucalgary.ca) to ensure that any user output does not contradict operational summary plots. Data use must be acknowledged according to the information available for each data set - please see [data.phys.ucalgary.ca](https://data.phys.ucalgary.ca). If you encounter any issues with the data or the crib sheet, please contact the UCalgary team for support (Emma Spanswick, elspansw@ucalgary.ca). Copyright © University of Calgary."]},{"cell_type":"markdown","id":"ded55fbe-326e-4964-aadc-dd4ea9af3db9","metadata":{"id":"ded55fbe-326e-4964-aadc-dd4ea9af3db9"},"source":["---\n","# **Download and read ASI raw data using PyAuroraX**\n","---\n"]},{"cell_type":"markdown","id":"35209183-611f-4f6c-8771-45744678fbbd","metadata":{"id":"35209183-611f-4f6c-8771-45744678fbbd"},"source":["Data can be downloaded from the UCalgary Space Remote Sensing Open Data Platform using the one of the following methods:\n"," - PyAuroraX (for all-sky imager data only) <-- we'll explore this method in this crib sheet\n"," - FTP (ftp://data.phys.ucalgary.ca)\n"," - Rsync (rsync://data.phys.ucalgary.ca)\n"," - HTTP via browser (https://data.phys.ucalgary.ca).\n"," - Directly using the API (https://api.phys.ucalgary.ca)\n","\n","Please note that the API is currently under development and we will do our best to keep this crib sheet up-to-date with the latest changes. If you have any questions, please reach out to the UCalgary Team (Emma Spanswick, elspansw@ucalgary.ca).\n","\n","### **Crib Sheet Summary**\n","\n","Below, we'll go through how to download and read data using PyAuroraX, the recommended library for working with All-Sky Imager (ASI) data that we provide.\n","\n","\n","\n","---\n","\n",""]},{"cell_type":"markdown","id":"3690a92b-4c77-4f46-98d1-f760b064b42b","metadata":{"id":"3690a92b-4c77-4f46-98d1-f760b064b42b"},"source":["## **Install dependencies**\n","\n","Here we'll install [PyAuroraX](https://github.com/aurorax-space/pyaurorax), and import it.\n","\n","Some helpful links:\n"," - [PyAuroraX documentation](https://docs.aurorax.space/code/overview)\n"," - [PyAuroraX API Reference](https://docs.aurorax.space/code/pyaurorax_api_reference/pyaurorax)\n"," - [Jupyter notebook examples](https://github.com/aurorax-space/pyaurorax/tree/main/examples/notebooks)"]},{"cell_type":"code","execution_count":null,"id":"e2a0f526","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":42564,"status":"ok","timestamp":1718809954918,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"e2a0f526","outputId":"768f5bfb-2fe8-45ef-e318-41f1b4bd91ca"},"outputs":[{"name":"stdout","output_type":"stream","text":["Looking in indexes: https://test.pypi.org/pypi/, https://pypi.org/simple\n","Collecting pyaurorax==1.0.0-rc1\n"," Downloading https://test-files.pythonhosted.org/packages/1b/6a/7032956187fa0b6c7bdbc4c2de8b0e1d70a130947ac2acc13083cc380065/pyaurorax-1.0.0rc1-py3-none-any.whl (193 kB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m193.3/193.3 kB\u001b[0m \u001b[31m5.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting aacgmv2<3.0.0,>=2.6.2 (from pyaurorax==1.0.0-rc1)\n"," Downloading aacgmv2-2.6.3.tar.gz (1.6 MB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m9.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n"," Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n"," Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n","Collecting cartopy<0.24.0,>=0.23.0 (from pyaurorax==1.0.0-rc1)\n"," Downloading Cartopy-0.23.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.6/11.6 MB\u001b[0m \u001b[31m22.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: click<9.0.0,>=8.1.3 in /usr/local/lib/python3.10/dist-packages (from pyaurorax==1.0.0-rc1) (8.1.7)\n","Requirement already satisfied: humanize<5.0.0,>=4.4.0 in /usr/local/lib/python3.10/dist-packages (from pyaurorax==1.0.0-rc1) (4.7.0)\n","Collecting matplotlib<4.0.0,>=3.9.0 (from pyaurorax==1.0.0-rc1)\n"," Downloading matplotlib-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.3/8.3 MB\u001b[0m \u001b[31m77.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting numpy<2.0.0,>=1.26.4 (from pyaurorax==1.0.0-rc1)\n"," Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m18.2/18.2 MB\u001b[0m \u001b[31m105.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: pyproj<4.0.0,>=3.6.1 in /usr/local/lib/python3.10/dist-packages (from pyaurorax==1.0.0-rc1) (3.6.1)\n","Requirement already satisfied: python-dateutil<3.0.0,>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pyaurorax==1.0.0-rc1) (2.8.2)\n","Collecting pyucalgarysrs<2.0.0,>=1.0.0 (from pyaurorax==1.0.0-rc1)\n"," Downloading pyucalgarysrs-1.0.4-py3-none-any.whl (69 kB)\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m69.2/69.2 kB\u001b[0m \u001b[31m11.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: requests<3.0.0,>=2.28.1 in /usr/local/lib/python3.10/dist-packages (from pyaurorax==1.0.0-rc1) (2.31.0)\n","Requirement already satisfied: termcolor<3.0.0,>=2.0.1 in /usr/local/lib/python3.10/dist-packages (from pyaurorax==1.0.0-rc1) (2.4.0)\n","Collecting texttable<2.0.0,>=1.6.4 (from pyaurorax==1.0.0-rc1)\n"," Downloading texttable-1.7.0-py2.py3-none-any.whl (10 kB)\n","Requirement already satisfied: shapely>=1.7 in /usr/local/lib/python3.10/dist-packages (from cartopy<0.24.0,>=0.23.0->pyaurorax==1.0.0-rc1) (2.0.4)\n","Requirement already satisfied: packaging>=20 in /usr/local/lib/python3.10/dist-packages (from cartopy<0.24.0,>=0.23.0->pyaurorax==1.0.0-rc1) (24.1)\n","Requirement already satisfied: pyshp>=2.3 in /usr/local/lib/python3.10/dist-packages (from cartopy<0.24.0,>=0.23.0->pyaurorax==1.0.0-rc1) (2.3.1)\n","Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib<4.0.0,>=3.9.0->pyaurorax==1.0.0-rc1) (1.2.1)\n","Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib<4.0.0,>=3.9.0->pyaurorax==1.0.0-rc1) (0.12.1)\n","Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib<4.0.0,>=3.9.0->pyaurorax==1.0.0-rc1) (4.53.0)\n","Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib<4.0.0,>=3.9.0->pyaurorax==1.0.0-rc1) (1.4.5)\n","Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.10/dist-packages (from matplotlib<4.0.0,>=3.9.0->pyaurorax==1.0.0-rc1) (9.4.0)\n","Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib<4.0.0,>=3.9.0->pyaurorax==1.0.0-rc1) (3.1.2)\n","Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from pyproj<4.0.0,>=3.6.1->pyaurorax==1.0.0-rc1) (2024.6.2)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil<3.0.0,>=2.8.2->pyaurorax==1.0.0-rc1) (1.16.0)\n","Requirement already satisfied: h5py<4.0.0,>=3.1.0 in /usr/local/lib/python3.10/dist-packages (from pyucalgarysrs<2.0.0,>=1.0.0->pyaurorax==1.0.0-rc1) (3.9.0)\n","Requirement already satisfied: joblib<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from pyucalgarysrs<2.0.0,>=1.0.0->pyaurorax==1.0.0-rc1) (1.4.2)\n","Requirement already satisfied: opencv-python<5.0.0,>=4.4.0 in /usr/local/lib/python3.10/dist-packages (from pyucalgarysrs<2.0.0,>=1.0.0->pyaurorax==1.0.0-rc1) (4.8.0.76)\n","Requirement already satisfied: scipy<2.0.0,>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from pyucalgarysrs<2.0.0,>=1.0.0->pyaurorax==1.0.0-rc1) (1.11.4)\n","Requirement already satisfied: tqdm<5.0.0,>=4.61.2 in /usr/local/lib/python3.10/dist-packages (from pyucalgarysrs<2.0.0,>=1.0.0->pyaurorax==1.0.0-rc1) (4.66.4)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.28.1->pyaurorax==1.0.0-rc1) (3.3.2)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.28.1->pyaurorax==1.0.0-rc1) (3.7)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.28.1->pyaurorax==1.0.0-rc1) (2.0.7)\n","Building wheels for collected packages: aacgmv2\n"," Building wheel for aacgmv2 (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n"," Created wheel for aacgmv2: filename=aacgmv2-2.6.3-cp310-cp310-linux_x86_64.whl size=1679867 sha256=eef80c8edc69bab013f0ada56160ff2903e55b6827fcbe297809cb76be53db1f\n"," Stored in directory: /root/.cache/pip/wheels/e9/d6/3f/10a359ebb903b2dbe5aa94f7024b5284059bb0daef65a7a0b4\n","Successfully built aacgmv2\n","Installing collected packages: texttable, numpy, aacgmv2, pyucalgarysrs, matplotlib, cartopy, pyaurorax\n"," Attempting uninstall: numpy\n"," Found existing installation: numpy 1.25.2\n"," Uninstalling numpy-1.25.2:\n"," Successfully uninstalled numpy-1.25.2\n"," Attempting uninstall: matplotlib\n"," Found existing installation: matplotlib 3.7.1\n"," Uninstalling matplotlib-3.7.1:\n"," Successfully uninstalled matplotlib-3.7.1\n","Successfully installed aacgmv2-2.6.3 cartopy-0.23.0 matplotlib-3.9.0 numpy-1.26.4 pyaurorax-1.0.0rc1 pyucalgarysrs-1.0.4 texttable-1.7.0\n"]}],"source":["!pip install pyaurorax"]},{"cell_type":"code","execution_count":null,"id":"55027f86-b09b-445f-b3c8-345c17a1e5c6","metadata":{"id":"55027f86-b09b-445f-b3c8-345c17a1e5c6"},"outputs":[],"source":["import pprint\n","import datetime\n","import pyaurorax\n","\n","aurorax = pyaurorax.PyAuroraX()"]},{"cell_type":"markdown","id":"28282e83","metadata":{"id":"28282e83"},"source":["## **Explore datasets**"]},{"cell_type":"markdown","id":"72042df5","metadata":{"id":"72042df5"},"source":["All data available are organized by unique 'dataset' identifier strings, for example, 'THEMIS_ASI_RAW'. There are a few functions available for exploring and finding more information about the datasets. Let's take a look at these."]},{"cell_type":"code","execution_count":null,"id":"e8aaae17","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":828,"status":"ok","timestamp":1718809979383,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"e8aaae17","outputId":"95b90810-529e-4b3f-a967-7307233811da"},"outputs":[{"name":"stdout","output_type":"stream","text":["Name Provider Level DOI Details Short Description \n","================================================================================================================================================================================================\n","REGO_CALIBRATION_FLATFIELD_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers Flatfield calibration data (IDL save format) \n","REGO_CALIBRATION_RAYLEIGHS_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers Rayleighs calibration data (IDL save format) \n","REGO_DAILY_KEOGRAM_JPG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers daily keogram summary product (JPG format) \n","REGO_DAILY_KEOGRAM_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers daily keogram summary product (PGM format) \n","REGO_DAILY_KEOGRAM_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers daily keogram summary product (PNG format) \n","REGO_DAILY_MONTAGE_JPG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers daily montage summary product (JPG format) \n","REGO_DAILY_MONTAGE_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers daily montage summary product (PGM format) \n","REGO_DAILY_MONTAGE_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers daily montage summary product (PNG format) \n","REGO_HOURLY_KEOGRAM_JPG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers hourly keogram summary product (JPG format) \n","REGO_HOURLY_KEOGRAM_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers hourly keogram summary product (PGM format) \n","REGO_HOURLY_KEOGRAM_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers hourly keogram summary product (PNG format) \n","REGO_HOURLY_MONTAGE_JPG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers hourly montage summary product (JPG format) \n","REGO_HOURLY_MONTAGE_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers hourly montage summary product (PGM format) \n","REGO_HOURLY_MONTAGE_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers hourly montage summary product (PNG format) \n","REGO_RAW UCalgary L0 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers 3-sec raw data \n","REGO_SKYMAP_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers skymap data (IDL save format) \n","THEMIS_ASI_DAILY_KEOGRAM_JPG UCalgary L2 None THEMIS All Sky Imagers daily keogram summary product (JPG format) \n","THEMIS_ASI_DAILY_KEOGRAM_PGM UCalgary L2 None THEMIS All Sky Imagers daily keogram summary product (PGM format) \n","THEMIS_ASI_DAILY_MONTAGE_JPG UCalgary L2 None THEMIS All Sky Imagers daily montage summary product (JPG format) \n","THEMIS_ASI_DAILY_MONTAGE_PGM UCalgary L2 None THEMIS All Sky Imagers daily montage summary product (PGM format) \n","THEMIS_ASI_HOURLY_AVERAGE_JPG UCalgary L2 None THEMIS All Sky Imagers hourly average summary product (JPG format) \n","THEMIS_ASI_HOURLY_AVERAGE_PGM UCalgary L2 None THEMIS All Sky Imagers hourly average summary product (PGM format) \n","THEMIS_ASI_HOURLY_KEOGRAM_JPG UCalgary L2 None THEMIS All Sky Imagers hourly keogram summary product (JPG format) \n","THEMIS_ASI_HOURLY_KEOGRAM_PGM UCalgary L2 None THEMIS All Sky Imagers hourly keogram summary product (PGM format) \n","THEMIS_ASI_HOURLY_MONTAGE_JPG UCalgary L2 None THEMIS All Sky Imagers hourly montage summary product (JPG format) \n","THEMIS_ASI_HOURLY_MONTAGE_PGM UCalgary L2 None THEMIS All Sky Imagers hourly montage summary product (PGM format) \n","THEMIS_ASI_RAW UCalgary L0 None THEMIS All Sky Imagers 3-sec raw data \n","THEMIS_ASI_RAW_ROW2 UCalgary L0 None THEMIS All Sky Imagers raw row2 data \n","THEMIS_ASI_RAW_WIDE UCalgary L0 None THEMIS All Sky Imagers raw wide data \n","THEMIS_ASI_SKYMAP_IDLSAV UCalgary L3 None THEMIS All Sky Imagers skymap data (IDL save format) \n","THEMIS_ASI_THUMB32 UCalgary L1A None THEMIS All Sky Imagers thumb32 data \n","THEMIS_ASI_VEC1024 UCalgary L1A None THEMIS All Sky Imagers vec1024 data \n","TREX_BLUE_CALIBRATION_FLATFIELD_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers Flatfield calibration data (IDL save format)\n","TREX_BLUE_CALIBRATION_RAYLEIGHS_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers Rayleighs calibration data (IDL save format)\n","TREX_BLUE_DAILY_KEOGRAM_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers daily keogram summary product (PGM format) \n","TREX_BLUE_DAILY_KEOGRAM_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers daily keogram summary product (PNG format) \n","TREX_BLUE_DAILY_MONTAGE_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers daily montage summary product (PGM format) \n","TREX_BLUE_DAILY_MONTAGE_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers daily montage summary product (PNG format) \n","TREX_BLUE_HOURLY_KEOGRAM_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers hourly keogram summary product (PGM format) \n","TREX_BLUE_HOURLY_KEOGRAM_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers hourly keogram summary product (PNG format) \n","TREX_BLUE_HOURLY_MONTAGE_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers hourly montage summary product (PGM format) \n","TREX_BLUE_HOURLY_MONTAGE_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers hourly montage summary product (PNG format) \n","TREX_BLUE_RAW UCalgary L0 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers 3-sec raw data \n","TREX_BLUE_SKYMAP_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers skymap data (IDL save format) \n","TREX_NIR_CALIBRATION_FLATFIELD_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers Flatfield calibration data (IDL save format) \n","TREX_NIR_CALIBRATION_RAYLEIGHS_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers Rayleighs calibration data (IDL save format) \n","TREX_NIR_DAILY_KEOGRAM_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers daily keogram summary product (PGM format) \n","TREX_NIR_DAILY_KEOGRAM_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers daily keogram summary product (PNG format) \n","TREX_NIR_DAILY_MONTAGE_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers daily montage summary product (PGM format) \n","TREX_NIR_DAILY_MONTAGE_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers daily montage summary product (PNG format) \n","TREX_NIR_HOURLY_KEOGRAM_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers hourly keogram summary product (PGM format) \n","TREX_NIR_HOURLY_KEOGRAM_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers hourly keogram summary product (PNG format) \n","TREX_NIR_HOURLY_MONTAGE_PGM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers hourly montage summary product (PGM format) \n","TREX_NIR_HOURLY_MONTAGE_PNG UCalgary L2 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers hourly montage summary product (PNG format) \n","TREX_NIR_RAW UCalgary L0 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers 6-sec raw data \n","TREX_NIR_SKYMAP_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers skymap data (IDL save format) \n","TREX_RGB_DAILY_KEOGRAM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/4p8e-1k65 TREx RGB All Sky Imagers daily keogram summary product \n","TREX_RGB_DAILY_MONTAGE UCalgary L2 https://commons.datacite.org/doi.org/10.11575/4p8e-1k65 TREx RGB All Sky Imagers daily montage summary product \n","TREX_RGB_HOURLY_KEOGRAM UCalgary L2 https://commons.datacite.org/doi.org/10.11575/4p8e-1k65 TREx RGB All Sky Imagers hourly keogram summary product \n","TREX_RGB_HOURLY_MONTAGE UCalgary L2 https://commons.datacite.org/doi.org/10.11575/4p8e-1k65 TREx RGB All Sky Imagers hourly montage summary product \n","TREX_RGB_RAW_BURST UCalgary L0 https://commons.datacite.org/doi.org/10.11575/4p8e-1k65 TREx RGB All Sky Imagers 3Hz raw burst data \n","TREX_RGB_RAW_NOMINAL UCalgary L0 https://commons.datacite.org/doi.org/10.11575/4p8e-1k65 TREx RGB All Sky Imagers 3-sec raw data \n","TREX_RGB_SKYMAP_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/4p8e-1k65 TREx RGB All Sky Imagers skymap data (IDL save format) \n"]}],"source":["# list all datasets\n","datasets = aurorax.data.list_datasets()\n","\n","# print them out in a table\n","aurorax.data.list_datasets_in_table()"]},{"cell_type":"code","execution_count":null,"id":"04c5ab88","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":761,"status":"ok","timestamp":1718810622630,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"04c5ab88","outputId":"d33f6600-abe0-47c8-ca2c-df6cb9bc48de"},"outputs":[{"name":"stdout","output_type":"stream","text":["Object representation:\n","----------------\n","Dataset(name=TREX_RGB_RAW_NOMINAL, short_description='TREx RGB All Sky Imagers 3-sec raw data', provider='UCalgary', level='L0', doi_details='https://commons.datacite.org/doi.org/10.11575/4p8e-1k65', ...)\n","\n","\n","Object as a dictionary:\n","----------------\n","{'citation': 'Spanswick, E., & Donovan, E. Transition Region Explorer - RGB '\n"," 'Dataset [Data set]. University of Calgary. '\n"," 'https://doi.org/10.11575/4P8E-1K65',\n"," 'data_tree_url': 'https://data.phys.ucalgary.ca/sort_by_project/TREx/RGB/stream0',\n"," 'doi': 'https://doi.org/10.11575/4P8E-1K65',\n"," 'doi_details': 'https://commons.datacite.org/doi.org/10.11575/4p8e-1k65',\n"," 'file_listing_supported': True,\n"," 'file_reading_supported': True,\n"," 'level': 'L0',\n"," 'long_description': 'Transition Region Explorer (TREx) full-color RGB All Sky '\n"," 'Imager array. More information can be found at '\n"," 'https://data.phys.ucalgary.ca',\n"," 'name': 'TREX_RGB_RAW_NOMINAL',\n"," 'provider': 'UCalgary',\n"," 'short_description': 'TREx RGB All Sky Imagers 3-sec raw data'}\n"]}],"source":["# get the TREx RGB raw dataset\n","dataset = aurorax.data.list_datasets(\"TREX_RGB_RAW_NOMINAL\")[0]\n","\n","# view Dataset object in a few different ways\n","print(\"Object representation:\\n----------------\")\n","print(dataset)\n","\n","print()\n","print()\n","\n","# and now as a dictionary\n","print(\"Object as a dictionary:\\n----------------\")\n","pprint.pprint(dataset.__dict__)"]},{"cell_type":"code","execution_count":null,"id":"8c9e1d86","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":202,"status":"ok","timestamp":1718810626489,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"8c9e1d86","outputId":"3dcda0fd-7904-436e-97de-3489cf10d0e4"},"outputs":[{"name":"stdout","output_type":"stream","text":["Dataset:\n"," citation : Spanswick, E., & Donovan, E. Transition Region Explorer - RGB Dataset [Data set]. University of Calgary. https://doi.org/10.11575/4P8E-1K65\n"," data_tree_url : https://data.phys.ucalgary.ca/sort_by_project/TREx/RGB/stream0\n"," doi : https://doi.org/10.11575/4P8E-1K65\n"," doi_details : https://commons.datacite.org/doi.org/10.11575/4p8e-1k65\n"," file_listing_supported : True\n"," file_reading_supported : True\n"," level : L0\n"," long_description : Transition Region Explorer (TREx) full-color RGB All Sky Imager array. More information can be found at https://data.phys.ucalgary.ca\n"," name : TREX_RGB_RAW_NOMINAL\n"," provider : UCalgary\n"," short_description : TREx RGB All Sky Imagers 3-sec raw data\n"]}],"source":["# there is also a pretty_print function to show all information in a different way\n","dataset.pretty_print()"]},{"cell_type":"markdown","id":"8c001a93","metadata":{"id":"8c001a93"},"source":["## **Explore observatories**\n","\n","Each ASI array has a set of observatories where instruments are deployed. We can easily view observatory information using the `list_observatories()` functions. We show some examples below."]},{"cell_type":"code","execution_count":null,"id":"8711d351","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":658,"status":"ok","timestamp":1718810628769,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"8711d351","outputId":"5c93e77d-0c57-4761-ad43-b26caf5d1180"},"outputs":[{"name":"stdout","output_type":"stream","text":["UID Full Name Geodetic Latitude Geodetic Longitude Provider\n","====================================================================================\n","atha Athabasca, AB, Canada 54.6 -113.64 UCalgary\n","chbg Chibougamau, QC, Canada 49.81 -74.42 UCalgary\n","ekat Ekati, NWT, Canada 64.73 -110.67 UCalgary\n","fsim Fort Simpson, NWT, Canada 61.76 -121.27 UCalgary\n","fsmi Fort Smith, NWT, Canada 60.03 -111.93 UCalgary\n","fykn Fort Yukon, AK, USA 66.56 -145.21 UCalgary\n","gako Gakona, AK, USA 62.41 -145.16 UCalgary\n","gbay Goose Bay, NL, Canada 53.32 -60.46 UCalgary\n","gill Gillam, MB, Canada 56.38 -94.64 UCalgary\n","inuv Inuvik, NWT, Canada 68.41 -133.77 UCalgary\n","kapu Kapuskasing, ON, Canada 49.39 -82.32 UCalgary\n","kian Kiana, AK, USA 66.97 -160.44 UCalgary\n","kuuj Kuujjuaq, QC, Canada 58.11 -68.41 UCalgary\n","mcgr McGrath, AK, USA 62.95 -155.6 UCalgary\n","nrsq Narsarsuaq, Greenland 61.16 -45.44 UCalgary\n","pgeo Prince George, BC, Canada 53.76 -122.85 UCalgary\n","pina Pinawa, MB, Canada 50.26 -95.87 UCalgary\n","rank Rankin Inlet, NU, Canada 62.82 -92.11 UCalgary\n","snap Snap Lake, NWT, Canada 63.56 -110.87 UCalgary\n","snkq Sanikiluaq, NU, Canada 56.54 -79.23 UCalgary\n","talo Taloyoak, NU, Canada 69.54 -93.56 UCalgary\n","tpas The Pas, MB, Canada 53.99 -100.94 UCalgary\n","whit Whitehorse, YK, Canada 60.62 -135.01 UCalgary\n","yknf Yellowknife, NWT, Canada 62.52 -114.31 UCalgary\n"]}],"source":["# list all observatories\n","#\n","# we must supply an 'instrument array' parameter to this function. View the\n","# documentation or integrated type hinting in VSCode to see possible instrument\n","# array choices.\n","observatories = aurorax.data.list_observatories(\"themis_asi\") # some choices are: themis_asi, rego, trex_nir, trex_rgb, trex_blue\n","\n","# print them out in a table\n","aurorax.data.list_observatories_in_table(\"themis_asi\")"]},{"cell_type":"code","execution_count":null,"id":"622a606d","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":2,"status":"ok","timestamp":1718810630151,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"622a606d","outputId":"4483ad90-3bc9-497d-93c0-2024b7eb6cfb"},"outputs":[{"name":"stdout","output_type":"stream","text":["Observatory(uid=atha, full_name='Athabasca, AB, Canada', geodetic_latitude=54.6, geodetic_longitude=-113.64, provider='UCalgary')\n","\n","{'full_name': 'Athabasca, AB, Canada',\n"," 'geodetic_latitude': 54.6,\n"," 'geodetic_longitude': -113.64,\n"," 'provider': 'UCalgary',\n"," 'uid': 'atha'}\n"]}],"source":["# view the first dataset\n","print(observatories[0])\n","\n","print()\n","\n","# show all values in the Dataset class\n","pprint.pprint(observatories[0].__dict__)"]},{"cell_type":"code","execution_count":null,"id":"0cbddb8f","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":194,"status":"ok","timestamp":1718810631670,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"0cbddb8f","outputId":"2ac2db33-0bfd-4ce6-b9d5-a012e158941c"},"outputs":[{"name":"stdout","output_type":"stream","text":["Observatory:\n"," full_name : Athabasca, AB, Canada\n"," geodetic_latitude : 54.6\n"," geodetic_longitude : -113.64\n"," provider : UCalgary\n"," uid : atha\n"]}],"source":["# there is also a pretty_print function to show all information in a different way\n","observatories[0].pretty_print()"]},{"cell_type":"markdown","id":"f70e3013-c1ff-49a7-a413-d4abbe77e497","metadata":{"id":"f70e3013-c1ff-49a7-a413-d4abbe77e497"},"source":["## **Download the data**\n","\n","We are going to download an hour of THEMIS data from the camera in Athabasca, AB.\n","\n","The `download()` function provides the ability to download a timeframe of data for a specific dataset. As a dataset can have data from many different sites, the site is an optional flag. There are also a few more options to choose from, including disabling or customizing the progress bar, forcing the data to be redownloaded even if it exists locally already, or adjusting the number of parallel download streams. Explore the API reference documentation for the function to learn more.\n","\n","By default, PyAuroraX will save data to the `aurorax.download_output_root_path` variable, which is your home directory. If you want to change this, you can edit this attribute like so:\n","\n","```python\n","aurorax.download_output_root_path = \"some other path\"\n","```\n","\n","Note: If you are running this notebook from within VSCode and you want the progress bar to be **dark theme**, insert the HTML markup [found here](https://github.com/microsoft/vscode-jupyter/issues/7161#issuecomment-1616627670) at the top of the notebook."]},{"cell_type":"code","execution_count":null,"id":"umtREGOlGPiV","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":202,"status":"ok","timestamp":1718811260645,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"umtREGOlGPiV","outputId":"94a7367b-f6b9-4193-f3f0-3e3c554e6946"},"outputs":[{"name":"stdout","output_type":"stream","text":["PyAuroraX(download_output_root_path='/content/ucalgary_data', read_tar_temp_path='/content/ucalgary_data/tar_temp_working', api_base_url='https://api.aurorax.space', api_headers={'content-type': 'application/json', 'user-agent': 'python-pyaurorax/1.0.0-rc1'}, api_timeout=10, api_key='None', srs_obj=PyUCalgarySRS(...))\n"]}],"source":["# We can set the path of where we want to save data to. By default,\n","# PyAuroraX saved data to your home directory. Since we can be running\n","# this crib sheet on Google Colab, we're going to do a special case\n","# and set the path if running in Colab. We'll leave the default if\n","# running in a different environment.\n","import sys\n","if (\"google.colab\" in sys.modules):\n"," aurorax.download_output_root_path = \"/content/ucalgary_data\"\n"," aurorax.read_tar_temp_path = \"/content/ucalgary_data/tar_temp_working\"\n","print(aurorax)"]},{"cell_type":"code","execution_count":null,"id":"2110ebdb","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":49,"referenced_widgets":["72eb63256b694a1591e7a1adcb75502f","f38db7bba0334aa2a26b7f1bf492e59c","cec64f22539d4489b1acf4051b9f8434","91fb2c1915db4c659e27db90f1b4b56f","69b5e1311c584fa0a756b2c107c735ef","ed4e7902712744c7ab895c71a354083f","1128ded3ad7f4267936c0c45c8a70438","b0eed76b8a6f4a3d94497ce2a07499ee","53d46b7d5ae14372b2770d7a57c05aa9","e2413c5c741548d69a344875552af3bf","39dab9940bde4df6bf5a0ad4335add4b"]},"executionInfo":{"elapsed":11049,"status":"ok","timestamp":1718811290144,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"2110ebdb","outputId":"2af64798-35bc-4f8b-aa4d-a2a0c29b5e4d"},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"72eb63256b694a1591e7a1adcb75502f","version_major":2,"version_minor":0},"text/plain":["Downloading THEMIS_ASI_RAW files: 0%| | 0.00/128M [00:00f4)\n"," full_elevation : array(dims=(256, 256), dtype=>f4)\n"," full_map_altitude : array(dims=(3,), dtype=>f4)\n"," full_map_latitude : array(dims=(3, 257, 257), dtype=>f4)\n"," full_map_longitude : array(dims=(3, 257, 257), dtype=>f4)\n"," generation_info : SkymapGenerationInfo(...)\n"," get_precalculated_altitudes: \n"," imager_uid : themis19\n"," project_uid : themis\n"," site_map_altitude : 0.0\n"," site_map_latitude : 56.376724\n"," site_map_longitude : 265.35632\n"," site_uid : gill\n"," version : v02\n"]}],"source":["# Now that we know which one we'll use, we can read it in.\n","#\n","# You can also read in all of them and choose later using the resulting Data object.\n","skymap_data = aurorax.data.ucalgary.read(dataset, r.filenames[0])\n","\n","print(skymap_data)\n","print()\n","skymap_data.pretty_print()\n","\n","print()\n","skymap_data.data[0].pretty_print()"]},{"cell_type":"markdown","id":"f56ac9c1","metadata":{"id":"f56ac9c1"},"source":["### **Automatically choosing a skymap**\n","\n","You can also let the library choose the skymap for you using the `download_best_skymap()` function."]},{"cell_type":"code","execution_count":null,"id":"aeba2b9e","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":480,"status":"ok","timestamp":1718811560342,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"aeba2b9e","outputId":"7d971177-4484-4e6e-e5dc-56f67413eb0a"},"outputs":[{"data":{"text/plain":["[PosixPath('/content/ucalgary_data/THEMIS_ASI_SKYMAP_IDLSAV/gill/gill_20210308/themis_skymap_gill_20210308-+_v02.sav')]"]},"execution_count":34,"metadata":{},"output_type":"execute_result"}],"source":["# set params\n","dataset_name = \"THEMIS_ASI_SKYMAP_IDLSAV\"\n","site_uid = \"gill\"\n","dt = datetime.datetime(2021, 11, 4)\n","\n","# get the recommendation\n","r = aurorax.data.ucalgary.download_best_skymap(dataset_name, site_uid, dt)\n","r.filenames"]},{"cell_type":"code","execution_count":null,"id":"1d137010","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":187,"status":"ok","timestamp":1718811563249,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"1d137010","outputId":"233984d2-03d4-4125-e4c1-c9a09f0faca6"},"outputs":[{"name":"stdout","output_type":"stream","text":["Data(data=[1 Skymap object], timestamp=[], metadata=[], problematic_files=[], calibrated_data=None, dataset=Dataset(name=THEMIS_ASI_SKYMAP_IDLSAV, short_description='THEMIS All Sky Im...))\n","\n","Data:\n"," data : [1 Skymap object]\n"," timestamp : []\n"," metadata : []\n"," problematic_files : []\n"," calibrated_data : None\n"," dataset : Dataset(name=THEMIS_ASI_SKYMAP_IDLSAV, short_description='THEMIS All Sky Im...)\n","\n","Skymap:\n"," filename : /content/ucalgary_data/THEMIS_ASI_SKYMAP_IDLSAV/gill/gill_20210308/themis_skymap_gill_20210308-+_v02.sav\n"," full_azimuth : array(dims=(256, 256), dtype=>f4)\n"," full_elevation : array(dims=(256, 256), dtype=>f4)\n"," full_map_altitude : array(dims=(3,), dtype=>f4)\n"," full_map_latitude : array(dims=(3, 257, 257), dtype=>f4)\n"," full_map_longitude : array(dims=(3, 257, 257), dtype=>f4)\n"," generation_info : SkymapGenerationInfo(...)\n"," get_precalculated_altitudes: \n"," imager_uid : themis19\n"," project_uid : themis\n"," site_map_altitude : 0.0\n"," site_map_latitude : 56.376724\n"," site_map_longitude : 265.35632\n"," site_uid : gill\n"," version : v02\n"]}],"source":["# now that we have the skymap file, we'll read it\n","skymap_data = aurorax.data.ucalgary.read(dataset, r.filenames)\n","\n","print(skymap_data)\n","print()\n","skymap_data.pretty_print()\n","\n","print()\n","skymap_data.data[0].pretty_print()"]},{"cell_type":"markdown","id":"c35464fa","metadata":{"id":"c35464fa"},"source":["## **Calibration data**\n","\n","Calibration data is used for converting data to Rayleighs, or applying corrections such as a flatfield. More information about why this is important can be found in the other crib sheets."]},{"cell_type":"code","execution_count":null,"id":"e95eb22d","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":674,"status":"ok","timestamp":1718811567193,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"e95eb22d","outputId":"ea931075-d294-4112-99c5-bf4910d8bb61"},"outputs":[{"name":"stdout","output_type":"stream","text":["Name Provider Level DOI Details Short Description \n","================================================================================================================================================================================================\n","REGO_CALIBRATION_FLATFIELD_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers Flatfield calibration data (IDL save format) \n","REGO_CALIBRATION_RAYLEIGHS_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/z7x6-5c42 REGO All Sky Imagers Rayleighs calibration data (IDL save format) \n","TREX_BLUE_CALIBRATION_FLATFIELD_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers Flatfield calibration data (IDL save format)\n","TREX_BLUE_CALIBRATION_RAYLEIGHS_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/80pf-0p02 TREx Blueline All Sky Imagers Rayleighs calibration data (IDL save format)\n","TREX_NIR_CALIBRATION_FLATFIELD_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers Flatfield calibration data (IDL save format) \n","TREX_NIR_CALIBRATION_RAYLEIGHS_IDLSAV UCalgary L3 https://commons.datacite.org/doi.org/10.11575/98w7-jp47 TREx NIR All Sky Imagers Rayleighs calibration data (IDL save format) \n"]}],"source":["# search datasets for calibrations\n","aurorax.data.list_datasets_in_table(name=\"CALIBRATION\")"]},{"cell_type":"markdown","id":"e68243b3","metadata":{"id":"e68243b3"},"source":["When selecting calibration files to use for converting ASI image counts to Rayleighs, we have two methods available to us:\n","\n","1. choosing manually\n","2. using the `download_best_flatfield_calibration()` or `download_best_rayleighs_calibration()` function to choose automatically\n","\n","Cameras are calibrated before they are deployed to the field, and after any in-house repairs are performed. There exist flatfield and Rayleighs calibration files, for each specific camera detector. A detector can live at multiple sites thoughout the years of operating the instrument array. Hence, why they are not associated with a specific site at all. We use the data device UID value to know what calibration files to use for the data we're processing."]},{"cell_type":"markdown","id":"593abdcb","metadata":{"id":"593abdcb"},"source":["### **Choosing a calibration file manually**"]},{"cell_type":"code","execution_count":null,"id":"b1fb9555","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1597,"status":"ok","timestamp":1718811587666,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"b1fb9555","outputId":"9c1bb6aa-c991-45bb-e841-6212d87fd03c"},"outputs":[{"name":"stdout","output_type":"stream","text":["/content/ucalgary_data/REGO_RAW/2021/11/04/resu_rego-655/ut06/20211104_0600_resu_rego-655_6300.pgm.gz\n","\n","rego-655\n"]}],"source":["# download a minute of REGO data\n","dataset_name = \"REGO_RAW\"\n","start_dt = datetime.datetime(2021, 11, 4, 6, 0)\n","end_dt = datetime.datetime(2021, 11, 4, 6, 0)\n","site_uid = \"resu\"\n","r = aurorax.data.ucalgary.download(dataset_name, start_dt, end_dt, site_uid=site_uid, progress_bar_disable=True)\n","\n","# determine the device uid\n","#\n","# you can either inspect the URLs and determine it by the filename, or you can read the\n","# data and inspect the device UID field of the metadata\n","print(r.filenames[0])\n","print()\n","\n","data = aurorax.data.ucalgary.read(r.dataset, r.filenames)\n","print(data.metadata[0][\"Imager unique ID\"])"]},{"cell_type":"code","execution_count":null,"id":"8a987ef3","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":478,"status":"ok","timestamp":1718811589809,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"8a987ef3","outputId":"ba342108-bfe8-4e44-cdcb-0cd02026aab3"},"outputs":[{"name":"stdout","output_type":"stream","text":["['https://data.phys.ucalgary.ca/sort_by_project/GO-Canada/REGO/calibration/REGO_Rayleighs_15655_20141002-+_v01.sav']\n","['https://data.phys.ucalgary.ca/sort_by_project/GO-Canada/REGO/calibration/REGO_flatfield_15655_20141002-+_v01.sav']\n"]}],"source":["# now that we know the device UID we are interested in, we can get a list\n","# of all the flatfield and rayleighs calibration files, and then choose which\n","# one we want to download\n","start_dt = datetime.datetime(2014, 1, 1, 0, 0)\n","end_dt = datetime.datetime.now()\n","device_uid = \"655\"\n","r_rayleighs = aurorax.data.ucalgary.get_urls(\"REGO_CALIBRATION_RAYLEIGHS_IDLSAV\", start_dt, end_dt, device_uid=device_uid)\n","pprint.pprint(r_rayleighs.urls)\n","\n","r_flatfield = aurorax.data.ucalgary.get_urls(\"REGO_CALIBRATION_FLATFIELD_IDLSAV\", start_dt, end_dt, device_uid=device_uid)\n","pprint.pprint(r_flatfield.urls)"]},{"cell_type":"code","execution_count":null,"id":"a537fb05","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1281,"status":"ok","timestamp":1718811593278,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"a537fb05","outputId":"d27f94d1-3acc-4f1e-9029-5ff491913fe6"},"outputs":[{"name":"stdout","output_type":"stream","text":["[PosixPath('/content/ucalgary_data/REGO_CALIBRATION_RAYLEIGHS_IDLSAV/REGO_Rayleighs_15655_20141002-+_v01.sav')]\n","[PosixPath('/content/ucalgary_data/REGO_CALIBRATION_FLATFIELD_IDLSAV/REGO_flatfield_15655_20141002-+_v01.sav')]\n"]}],"source":["# this is simple as there is only one to choose from\n","#\n","# now let's download the data\n","d_rayleighs = aurorax.data.ucalgary.download_using_urls(r_rayleighs, progress_bar_disable=True)\n","d_flatfield = aurorax.data.ucalgary.download_using_urls(r_flatfield, progress_bar_disable=True)\n","\n","print(d_rayleighs.filenames)\n","\n","print(d_flatfield.filenames)"]},{"cell_type":"code","execution_count":null,"id":"89388fe3","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":208,"status":"ok","timestamp":1718811594262,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"89388fe3","outputId":"bad14ec4-7fb7-475f-eb60-b4cb314e3840"},"outputs":[{"name":"stdout","output_type":"stream","text":["Data(data=[1 Calibration object], timestamp=[], metadata=[], problematic_files=[], calibrated_data=None, dataset=Dataset(name=REGO_CALIBRATION_RAYLEIGHS_IDLSAV, short_description='REGO All...))\n","Data(data=[1 Calibration object], timestamp=[], metadata=[], problematic_files=[], calibrated_data=None, dataset=Dataset(name=REGO_CALIBRATION_FLATFIELD_IDLSAV, short_description='REGO All...))\n","\n","Calibration:\n"," dataset : Dataset(...)\n"," detector_uid : 15655\n"," filename : /content/ucalgary_data/REGO_CALIBRATION_RAYLEIGHS_IDLSAV/REGO_Rayleighs_15655_20141002-+_v01.sav\n"," flat_field_multiplier : None\n"," generation_info : CalibrationGenerationInfo(...)\n"," rayleighs_perdn_persecond : 10.399999618530273\n"," version : v01\n","\n","Calibration:\n"," dataset : Dataset(...)\n"," detector_uid : 15655\n"," filename : /content/ucalgary_data/REGO_CALIBRATION_FLATFIELD_IDLSAV/REGO_flatfield_15655_20141002-+_v01.sav\n"," flat_field_multiplier : array(dims=(512, 512), dtype=>f8)\n"," generation_info : CalibrationGenerationInfo(...)\n"," rayleighs_perdn_persecond : None\n"," version : v01\n"]}],"source":["# now that we have the calibration files, we'll read them\n","cal_rayleighs_data = aurorax.data.ucalgary.read(d_rayleighs.dataset, d_rayleighs.filenames)\n","cal_flatfield_data = aurorax.data.ucalgary.read(d_flatfield.dataset, d_flatfield.filenames)\n","\n","print(cal_rayleighs_data)\n","print(cal_flatfield_data)\n","print()\n","\n","cal_rayleighs_data.data[0].pretty_print()\n","print()\n","cal_flatfield_data.data[0].pretty_print()\n"]},{"cell_type":"markdown","id":"e2f17a6c","metadata":{"id":"e2f17a6c"},"source":["### **Automatically choosing a calibration file**\n","\n","You can also let the library choose the calibration for you using the `download_best_flatfield_calibration()` and `download_best_rayleighs_calibration()` functions."]},{"cell_type":"code","execution_count":null,"id":"961b1c56","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1440,"status":"ok","timestamp":1718811601415,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"961b1c56","outputId":"cb9f8a50-0a6e-42df-823f-80ff6633aa5a"},"outputs":[{"name":"stdout","output_type":"stream","text":["[PosixPath('/content/ucalgary_data/REGO_CALIBRATION_RAYLEIGHS_IDLSAV/REGO_Rayleighs_15654_20210806-+_v02.sav')]\n","[PosixPath('/content/ucalgary_data/REGO_CALIBRATION_FLATFIELD_IDLSAV/REGO_flatfield_15654_20210806-+_v02.sav')]\n"]}],"source":["# set params\n","device_uid = \"654\"\n","dt = datetime.datetime(2021, 11, 4)\n","\n","# get the recommendations\n","r_rayleighs = aurorax.data.ucalgary.download_best_rayleighs_calibration(\"REGO_CALIBRATION_RAYLEIGHS_IDLSAV\", device_uid, dt)\n","r_flatfield = aurorax.data.ucalgary.download_best_flatfield_calibration(\"REGO_CALIBRATION_FLATFIELD_IDLSAV\", device_uid, dt)\n","\n","# show results\n","print(r_rayleighs.filenames)\n","print(r_flatfield.filenames)"]},{"cell_type":"code","execution_count":null,"id":"215d8c64","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":219,"status":"ok","timestamp":1718811607826,"user":{"displayName":"Darren Chaddock","userId":"00063457193488446753"},"user_tz":360},"id":"215d8c64","outputId":"5ab56aec-d21a-4b0b-bba8-6e86d995c45e"},"outputs":[{"name":"stdout","output_type":"stream","text":["Calibration:\n"," dataset : Dataset(...)\n"," detector_uid : 15654\n"," filename : /content/ucalgary_data/REGO_CALIBRATION_RAYLEIGHS_IDLSAV/REGO_Rayleighs_15654_20210806-+_v02.sav\n"," flat_field_multiplier : None\n"," generation_info : CalibrationGenerationInfo(...)\n"," rayleighs_perdn_persecond : 10.137431837782506\n"," version : v02\n","\n","Calibration:\n"," dataset : Dataset(...)\n"," detector_uid : 15654\n"," filename : /content/ucalgary_data/REGO_CALIBRATION_FLATFIELD_IDLSAV/REGO_flatfield_15654_20210806-+_v02.sav\n"," flat_field_multiplier : array(dims=(512, 512), dtype=>f8)\n"," generation_info : CalibrationGenerationInfo(...)\n"," rayleighs_perdn_persecond : None\n"," version : v02\n"]}],"source":["# now that we have the calibration file, we'll read it\n","cal_rayleighs_data = aurorax.data.ucalgary.read(r_rayleighs.dataset, r_rayleighs.filenames)\n","cal_flatfield_data = aurorax.data.ucalgary.read(r_flatfield.dataset, r_flatfield.filenames)\n","\n","cal_rayleighs_data.data[0].pretty_print()\n","print()\n","cal_flatfield_data.data[0].pretty_print()"]},{"cell_type":"code","execution_count":null,"id":"66b62dd7","metadata":{"id":"66b62dd7"},"outputs":[],"source":[]}],"metadata":{"colab":{"provenance":[]},"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.9"},"widgets":{"application/vnd.jupyter.widget-state+json":{"0e0a5c4c49044a32860f3dacecbd8f56":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1128ded3ad7f4267936c0c45c8a70438":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1a2b44558d3c45ce915040b5b905f139":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_310eb958a94f4428a531d758cc4e9bd0","placeholder":"","style":"IPY_MODEL_541ca1480b8549ce8d3712d486b5a1c3","value":" 7.08M/7.08M [00:01<00:00, 5.40MB/s]"}},"1caec0937e214f3f9f941f72cbb227de":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"227484f221fe4d2b9cabca199d2fdb9f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_368ed11b6d1146dd805a5070438c3de4","placeholder":"","style":"IPY_MODEL_7e133bbac2374fc486062b6e85272a09","value":"Downloading THEMIS_ASI_RAW files: 100%"}},"310eb958a94f4428a531d758cc4e9bd0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"33b2d71153914c3797fec14665aba030":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_abda2076d56e4d508dea4dea7036a4cd","placeholder":"","style":"IPY_MODEL_96911d307db048f0bce325f58bda1cda","value":"Downloading THEMIS_ASI_SKYMAP_IDLSAV files: 100%"}},"368ed11b6d1146dd805a5070438c3de4":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"39dab9940bde4df6bf5a0ad4335add4b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"53d46b7d5ae14372b2770d7a57c05aa9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"541ca1480b8549ce8d3712d486b5a1c3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5551efbbac39416b8328e882b237a8b9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_75d634237213406d932157c1aea65002","max":7078784,"min":0,"orientation":"horizontal","style":"IPY_MODEL_0e0a5c4c49044a32860f3dacecbd8f56","value":7078784}},"68ff9cf78dc540169898a5db082e391d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_33b2d71153914c3797fec14665aba030","IPY_MODEL_5551efbbac39416b8328e882b237a8b9","IPY_MODEL_1a2b44558d3c45ce915040b5b905f139"],"layout":"IPY_MODEL_eb6c24fe28c2440ea0f301ffd41944d5"}},"69b5e1311c584fa0a756b2c107c735ef":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6e2c51bcf4cd44a2a0ac7cad58508997":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"72eb63256b694a1591e7a1adcb75502f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_f38db7bba0334aa2a26b7f1bf492e59c","IPY_MODEL_cec64f22539d4489b1acf4051b9f8434","IPY_MODEL_91fb2c1915db4c659e27db90f1b4b56f"],"layout":"IPY_MODEL_69b5e1311c584fa0a756b2c107c735ef"}},"75d634237213406d932157c1aea65002":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7898810590f24650825cd7782d5cc454":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7e0160831fa143ecb77ee3f001b359fa","placeholder":"","style":"IPY_MODEL_6e2c51bcf4cd44a2a0ac7cad58508997","value":" 136M/136M [00:00<00:00, 3.00GB/s]"}},"7e0160831fa143ecb77ee3f001b359fa":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7e133bbac2374fc486062b6e85272a09":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"86ea5093032449b09387ab543ffc4cf6":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_1caec0937e214f3f9f941f72cbb227de","max":135927273,"min":0,"orientation":"horizontal","style":"IPY_MODEL_d29040d9e36b4958b8afa11eea9c8c7b","value":135927273}},"91fb2c1915db4c659e27db90f1b4b56f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e2413c5c741548d69a344875552af3bf","placeholder":"","style":"IPY_MODEL_39dab9940bde4df6bf5a0ad4335add4b","value":" 128M/128M [00:10<00:00, 14.4MB/s]"}},"96911d307db048f0bce325f58bda1cda":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"abda2076d56e4d508dea4dea7036a4cd":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b0eed76b8a6f4a3d94497ce2a07499ee":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cec64f22539d4489b1acf4051b9f8434":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_b0eed76b8a6f4a3d94497ce2a07499ee","max":127920163,"min":0,"orientation":"horizontal","style":"IPY_MODEL_53d46b7d5ae14372b2770d7a57c05aa9","value":127920163}},"cff9f426985b4abb9b58975caa80a02c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d29040d9e36b4958b8afa11eea9c8c7b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"e2413c5c741548d69a344875552af3bf":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"eb34fb4612c6441d9a5f4b42c2b73f17":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_227484f221fe4d2b9cabca199d2fdb9f","IPY_MODEL_86ea5093032449b09387ab543ffc4cf6","IPY_MODEL_7898810590f24650825cd7782d5cc454"],"layout":"IPY_MODEL_cff9f426985b4abb9b58975caa80a02c"}},"eb6c24fe28c2440ea0f301ffd41944d5":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ed4e7902712744c7ab895c71a354083f":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f38db7bba0334aa2a26b7f1bf492e59c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ed4e7902712744c7ab895c71a354083f","placeholder":"","style":"IPY_MODEL_1128ded3ad7f4267936c0c45c8a70438","value":"Downloading THEMIS_ASI_RAW files: 100%"}}}}},"nbformat":4,"nbformat_minor":5}