{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Welcome to vCDAT\n", "This is your personal JupyterLab vCDAT interface.\n", "\n", "Just press **shift-enter** to follow this getting started guide." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Resources\n", "A small introduction video to JupyterLab." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from IPython.display import YouTubeVideo\n", "YouTubeVideo('ctOM-Gza04Y')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using Your Own Data\n", "\n", "Ensure that your NetCDF data files are **located in the same directory from which you launched the JupyterLab interface or in a subdirectory** (like sample_data). Once your own data files are in this directory, they should automatically show up in the JupyterLab interface and be listed in the left panel. If you do not see them immediately, refresh the browser. If you still do not see the files, click on the grey file folder in the top left corner of the interface to make sure the list of files is showing.\n", "\n", "If you need to access data outside of your *top* directory, you can create a symbolic link to them.\n", "\n", "For example assuming you would like to access data in `/scratch/some_data` (and assuming you can access this directory), simply go to the *top* directory of your user space (usually `$HOME`) and run this command\n", "\n", "```bash\n", "ln -s /scratch/some_data my_link_to_some_data\n", "```\n", "\n", "Now in the jupyter interface you should see a directory called `my_link_to_some_data` simply click on it and you are where you want to be." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Download the Sample Data\n", "\n", "To download some test data we've provided, enter the code below within a cell in a Jupyter notebook and run the cell by holding down shift and pressing enter while that cell is highlighted. It might take a few moments for the commands to run.\n", "\n", "The data will be stored within a subfolder called \"sample_data\" in the same location from which you launched your JupyterLab interface. If you look at the left panel, you should see a folder titled \"sample_data\". If you double click on the sample_data folder, you will see many test files in a variety of formats." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading: 'BlueMarble.ppm' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/BlueMarble.ppm\n", "Downloading: 'clt.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/clt.nc\n", "Downloading: 'geo.1deg.ctl' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/geo.1deg.ctl\n", "Downloading: 'geo.1deg.gmp' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/geo.1deg.gmp\n", "Downloading: 'geo.1deg.grb' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/geo.1deg.grb\n", "Downloading: 'meshfill.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/meshfill.nc\n", "Downloading: 'navy_land.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/navy_land.nc\n", "Downloading: 'rlut_CERES_000001-000012_ac.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/rlut_CERES_000001-000012_ac.nc\n", "Downloading: 'sampleCurveGrid4.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/sampleCurveGrid4.nc\n", "Downloading: 'sampleGenGrid3.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/sampleGenGrid3.nc\n", "Downloading: 'sftbyrgn.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/sftbyrgn.nc\n", "Downloading: 'sftlf_10x10.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/sftlf_10x10.nc\n", "Downloading: 'sftlf_visus.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/sftlf_visus.nc\n", "Downloading: 'so_Omon_ACCESS1-0_historical_r1i1p1_185001-185412_2timesteps.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/so_Omon_ACCESS1-0_historical_r1i1p1_185001-185412_2timesteps.nc\n", "Downloading: 'so_Omon_GISS-E2-R_historicalNat_r5i1p1_185001-187512_2timesteps.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/so_Omon_GISS-E2-R_historicalNat_r5i1p1_185001-187512_2timesteps.nc\n", "Downloading: 'so_Omon_HadGEM2-CC_historical_r1i1p1_185912-186911_2timesteps.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/so_Omon_HadGEM2-CC_historical_r1i1p1_185912-186911_2timesteps.nc\n", "Downloading: 'so_Omon_MPI-ESM-LR_1pctCO2_r1i1p1_185001-185912_2timesteps.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/so_Omon_MPI-ESM-LR_1pctCO2_r1i1p1_185001-185912_2timesteps.nc\n", "Downloading: 'swan.four.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/swan.four.nc\n", "Downloading: 'ta_ncep_87-6-88-4.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/ta_ncep_87-6-88-4.nc\n", "Downloading: 'tas_ccsr-95a.xml' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ccsr-95a.xml\n", "Downloading: 'tas_ccsr-95a_1979.01-1979.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ccsr-95a_1979.01-1979.12.nc\n", "Downloading: 'tas_ccsr-95a_1980.01-1980.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ccsr-95a_1980.01-1980.12.nc\n", "Downloading: 'tas_ccsr-95a_1981.01-1981.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ccsr-95a_1981.01-1981.12.nc\n", "Downloading: 'tas_ccsr-95a_1982.01-1982.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ccsr-95a_1982.01-1982.12.nc\n", "Downloading: 'tas_ccsr-95a_1983.01-1983.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ccsr-95a_1983.01-1983.12.nc\n", "Downloading: 'tas_ccsr-95a_1984.01-1984.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ccsr-95a_1984.01-1984.12.nc\n", "Downloading: 'sftlf_ccsr.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/sftlf_ccsr.nc\n", "Downloading: 'tas_dnm-95a.xml' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_dnm-95a.xml\n", "Downloading: 'tas_dnm-95a_1979.01-1979.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_dnm-95a_1979.01-1979.12.nc\n", "Downloading: 'tas_dnm-95a_1980.01-1980.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_dnm-95a_1980.01-1980.12.nc\n", "Downloading: 'tas_dnm-95a_1981.01-1981.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_dnm-95a_1981.01-1981.12.nc\n", "Downloading: 'tas_dnm-95a_1982.01-1982.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_dnm-95a_1982.01-1982.12.nc\n", "Downloading: 'tas_dnm-95a_1983.01-1983.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_dnm-95a_1983.01-1983.12.nc\n", "Downloading: 'tas_dnm-95a_1984.01-1984.12.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_dnm-95a_1984.01-1984.12.nc\n", "Downloading: 'sftlf_dnm.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/sftlf_dnm.nc\n", "Downloading: 'taylor.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/taylor.nc\n", "Downloading: 'test_anim.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/test_anim.nc\n", "Downloading: 'test_col.asc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/test_col.asc\n", "Downloading: 'testgrib2.ctl' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/testgrib2.ctl\n", "Downloading: 'testgrib2.grib2' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/testgrib2.grib2\n", "Downloading: 'testgrib2.idx' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/testgrib2.idx\n", "Downloading: 'testpp.pp' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/testpp.pp\n", "Downloading: 'thermo.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/thermo.nc\n", "Downloading: 'vertical.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/vertical.nc\n", "Downloading: 'tas_ukmo_con.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ukmo_con.nc\n", "Downloading: 'tas_gavg_rnl_ecm.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_gavg_rnl_ecm.nc\n", "Downloading: 'tas_ecm_1979.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_ecm_1979.nc\n", "Downloading: 'tas_cru_1979.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_cru_1979.nc\n", "Downloading: 'psl_6h.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/psl_6h.nc\n", "Downloading: 'ts_da.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/ts_da.nc\n", "Downloading: 'tas_mo.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_mo.nc\n", "Downloading: 'tas_mo_clim.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_mo_clim.nc\n", "Downloading: 'tas_6h.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/tas_6h.nc\n", "Downloading: 'th_yr.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/th_yr.nc\n", "Downloading: 'th_yr.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/th_yr.nc\n", "Downloading: 'th_yr.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/th_yr.nc\n", "Downloading: 'geos5-sample.nc' from 'https://cdat.llnl.gov/cdat/sample_data/' in: sample_data/geos5-sample.nc\n" ] } ], "source": [ "import vcs\n", "import cdms2\n", "import cdat_info\n", "import pkg_resources\n", "vcs_egg_path = pkg_resources.resource_filename(pkg_resources.Requirement.parse(\"vcs\"), \"share/vcs\")\n", "path = vcs_egg_path+'/sample_files.txt'\n", "cdat_info.download_sample_data_files(path,\"sample_data\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "***WARNING***\n", "\n", "The JupyterLab file browser allows you to double click on a file to open it. Unfortunately this means loading the whole file into the browser. For netCDF files this is not practical (these files can be several Gb), so we kill the transfer to prevent the file from loading. Unfortunately this brings up the following error message. You can ***safely ignore*** the error message and move on. The creators of Jupyterlab are aware of the problem and are implementing a solution for big files.\n", "\n", "![File Loading Message In VCDAT](https://cdat.llnl.gov/cdat/vcdat/images/file_load_error_vcdat.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Software\n", "\n", "VCDAT Jupyter-lab comes with the following core tools:\n", "\n", "* python >3\n", "* cdat_info\n", "* cdms2\n", "* cdutil\n", "* genutil\n", "* vcs \n", "* dv3d\n", "* libcdms >=3.1.2\n", "* vtk-cdat >8.1\n", "* requests\n", "* numpy\n", "* ghostscript\n", "* nodejs\n", "* jupyterlab \n", "* pip \n", "* nb_conda \n", "* nb_conda_kernels \n", "* plumbum \n", "* jupyterhub\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Next Steps\n", "\n", "For more details on the user interface, see the [User Interface](https://github.com/CDAT/jupyter-vcdat/wiki/User-Interface) wiki page.\n", "\n", "\n", "Additional details on accessing data are included on the vCDAT [Access Data](https://github.com/CDAT/jupyter-vcdat/wiki/Access-Data) help page. CDAT (and vCDAT) can open any OPeNDAP file so if you wanted to get the latest CMIP6 data or other climate data you could use the Earth System Grid Federation (ESGF) data warehouse interface to access the OPeNDAP URL for that data. [This page](https://www.earthsystemcog.org/projects/cog/doc/opendap) provides details on browsing and downloading data from ESGF using OPeNDAP.\n", "\n", "Once you have a sample or \"real\" dataset, you might want to [Load Variables](https://github.com/CDAT/jupyter-vcdat/wiki/Load-Variables) and plot them (see the [Plot Variables](https://github.com/CDAT/jupyter-vcdat/wiki/Plot-Variables) page)." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.7" }, "selected_variables": [], "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": false, "sideBar": false, "skip_h1_title": true, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false }, "vcdat_file_path": "", "vcdat_loaded_variables": [], "vcdat_variable_info": {} }, "nbformat": 4, "nbformat_minor": 2 }