{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Building ERDDAP Datasets\n", "\n", "This notebook documents the process of creating XML fragments\n", "for nowcast system run results files\n", "for inclusion in `/opt/tomcat/content/erddap/datasets.xml`\n", "on the `skookum` ERDDAP server instance.\n", "\n", "The contents are a combination of:\n", "\n", "* instructions for using the\n", "`GenerateDatasetsXml.sh` and `DasDds.sh` tools found in the\n", "`/opt/tomcat/webapps/erddap/WEB-INF/` directory\n", "* instructions for forcing the server to update the datasets collection\n", "via the `/results/erddap/flags/` directory\n", "* code and metadata to transform the output of `GenerateDatasetsXml.sh`\n", "into XML fragments that are ready for inclusion in `/opt/tomcat/content/erddap/datasets.xml`" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from collections import OrderedDict\n", "\n", "from lxml import etree" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**NOTE**\n", "\n", "The next cell mounts the `/results` filesystem on `skookum` locally.\n", "It is intended for use if when this notebook is run on a laptop \n", "or other non-Waterhole machine that has `sshfs` installed \n", "and a mount point for `/results` available in its root filesystem.\n", "\n", "Don't execute the cell if that doesn't describe your situation." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "!sshfs skookum:/results /results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `metadata` dictionary below contains information for dataset\n", "attribute tags whose values need to be changed,\n", "or that need to be added for all datasets.\n", "\n", "The keys are the dataset attribute names.\n", "\n", "The values are dicts containing a required `text` item\n", "and perhaps an optional `after` item.\n", "\n", "The value associated with the `text` key is the text content\n", "for the attribute tag.\n", "\n", "When present,\n", "the value associated with the `after` key is the name\n", "of the dataset attribute after which a new attribute tag\n", "containing the `text` value is to be inserted." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "metadata = OrderedDict([\n", " ('coverage_content_type', {\n", " 'text': 'modelResult',\n", " 'after': 'cdm_data_type',\n", " }),\n", " ('infoUrl', {\n", " 'text': \n", " 'https://salishsea-meopar-tools.readthedocs.org/en/latest/results_server/index.html#salish-sea-model-results',\n", " }),\n", " ('institution', {'text': 'UBC EOAS'}),\n", " ('institution_fullname', {\n", " 'text': 'Earth, Ocean & Atmospheric Sciences, University of British Columbia',\n", " 'after': 'institution',\n", " }),\n", " ('license', {\n", " 'text': '''The SalishSeaCast NEMO model results are copyright 2013 – present\n", "by the SalishSeaCast Project Contributors and The University of British Columbia.\n", "\n", "They are licensed under the Apache License, Version 2.0. http://www.apache.org/licenses/LICENSE-2.0''',\n", " }),\n", " ('project', {\n", " 'text':'SalishSeaCast NEMO Model',\n", " 'after': 'title',\n", " }),\n", " ('creator_name', {\n", " 'text': 'SalishSeaCast Project Contributors',\n", " 'after': 'project',\n", " }),\n", " ('creator_email', {\n", " 'text': 'sallen@eoas.ubc.ca',\n", " 'after': 'creator_name',\n", " }),\n", " ('creator_url', {\n", " 'text': 'https://salishsea-meopar-docs.readthedocs.org/',\n", " 'after': 'creator_email',\n", " }),\n", " ('acknowledgement', {\n", " 'text': 'MEOPAR, ONC, Compute Canada',\n", " 'after': 'creator_url',\n", " }),\n", " ('drawLandMask', {\n", " 'text': 'over',\n", " 'after': 'acknowledgement',\n", " }),\n", "])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `datasets` dictionary below provides the content\n", "for the dataset `title` and `summary` attributes.\n", "\n", "The `title` attribute content appears in the the datasets list table\n", "(among other places).\n", "It should be `<`80 characters long,\n", "and note that only the 1st 40 characters will appear in the table.\n", "\n", "The `summary` atribute content appears\n", "(among other places)\n", "when a user hovers the cursor over the `?` icon beside the `title`\n", "content in the datasets list table.\n", "The text that is inserted into the `summary` attribute tag\n", "by code later in this notebook is the\n", "`title` content followed by the `summary` content,\n", "separated by a blank line.\n", "\n", "The keys of the `datasets` dict are the `datasetID` strings that\n", "are used in many places by the ERDDAP server.\n", "They are structured as follows:\n", "\n", "* `ubc` to indicate that the dataset was produced at UBC\n", "* `SS` to indicate that the dataset is a product of the SalishSeaCast NEMO model\n", "* `n` to indicate that the dataset is from a nowcast run;\n", "other possibilities include `f` for forecast,\n", "`f2` for forecast2 (aka preliminary forecast),\n", "`ng` for nowcat-green,\n", "`a` for atmospheric forcing.\n", "* a description of the dataset variables; e.g. `PointAtkinsonSSH` or `3DuVelocity`\n", "* the time interval of values in the dataset; e.g. `15m`, `1h`, `1d`\n", "* the dataset version; e.g. `V1`\n", "\n", "So:\n", "\n", "* `ubcSSnPointAtkinsonSSH15mV1` is the version 1 dataset of 15 minute averaged sea surface height values at Point Atkinson from `PointAtkinson.nc` output files\n", "* `ubcSSn3DwVelocity1hV2` is the version 2 dataset of 1 hr averaged vertical (w) velocity values over the entire domain from `SalishSea_1h_*_grid_W.nc` output files\n", "* `ubcSSnSurfaceTracers1dV1` is the version 1 dataset of daily averaged surface tracer values over the entire domain from `SalishSea_1d_*_grid_T.nc` output files\n", "\n", "The dataset version part of the `datasetID` is used to indicate changes in the variables\n", "contained in the dataset.\n", "For example,\n", "the transition from the `ubcSSn3DwVelocity1hV1` to the `ubcSSn3DwVelocity1hV2` dataset\n", "occurred on 24-Jan-2016 when we started to output vertical eddy viscosity and diffusivity\n", "values at the `w` grid points.\n", "\n", "All datasets start at `V1` and their `summary` ends with a notation about the variables\n", "that they contain; e.g.\n", "```\n", "v1: wVelocity variable\n", "```\n", "When the a dataset version is incremented a line describing the change is added\n", "to the end of its `summary`; e.g.\n", "```\n", "v1: wVelocity variable\n", "v2: Added eddy viscosity & diffusivity variables ve_eddy_visc & ve_eddy_diff\n", "```" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false }, "outputs": [], "source": [ "datasets = {\n", " 'ubcSSnBathymetry2V1' :{\n", " 'type': 'geolocation bathymetry',\n", " 'title': 'SalishSeaCast NEMO model Grid, Geo-location and Bathymetry, v1',\n", " 'summary':'''Longitude, latitude, and bathymetry of the SalishSeaCast NEMO model grid.\n", "The bathymetry values are those calculated by NEMO from the input bathymetry file.\n", "NEMO modifies the input bathymetry to remove isolated holes, and too-small partial steps.\n", "The model grid includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "\n", "v1: longitude, latitude and bathymetry variables''',\n", " 'fileNameRegex': '.*SalishSea2_NEMO_bathy\\.nc$'\n", " \n", " },\n", " \n", " 'ubcSSn2DMeshMask2V1': {\n", " 'type': 'geolocation bathymetry',\n", " 'title': 'SalishSeaCast NEMO model Grid, 2D Mesh Mask, v1',\n", " 'summary':'''NEMO grid variable value for the u-v plane of the \n", "SalishSeaCast NEMO model Arakawa-C grid.\n", "The values are those calculated by NEMO from the input coordinates and bathymetry files.\n", "The variable names are those used by NEMO-3.4,\n", "see the NEMO-3.4 book (http://www.nemo-ocean.eu/Media/Files/NEMO_book_V3_4.pdf) for details,\n", "or the long_name attributes of the variables for succinct descriptions of the variables.\n", "The model grid includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "\n", "v1: e1t, e2t, e1u, e2u, e1v, e2v, e1f, e2f, glamt, gphit, glamu, gphiu, glamv, gphiv, \n", "tmaskutil, umaskutil, vmaskutil, fmaskutil, ff, mbathy variables''',\n", " 'fileNameRegex': '.*mesh_mask_SalishSea2\\.nc$',\n", " },\n", " \n", " 'ubcSSn3DMeshMask2V1': {\n", " 'type': 'geolocation bathymetry',\n", " 'title': 'SalishSeaCast NEMO model Grid, 3D Mesh Mask, v1',\n", " 'summary':'''NEMO grid variable value for the SalishSeaCast NEMO model Arakawa-C grid.\n", "The values are those calculated by NEMO from the input coordinates and bathymetry files.\n", "The variable names are those used by NEMO-3.4,\n", "see the NEMO-3.4 book (http://www.nemo-ocean.eu/Media/Files/NEMO_book_V3_4.pdf) for details,\n", "or the long_name attributes of the variables for succinct descriptions of the variables.\n", "The model grid includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "\n", "v1: e3t, e3u, e3v, e3w, gdept, gdepu, gdepv, gdepw, tmask, umask, vmask, fmask variables''',\n", " 'fileNameRegex': '.*mesh_mask_SalishSea2\\.nc$'\n", " },\n", " \n", " 'ubcSSnPointAtkinsonSSH15mV1': {\n", " 'type': 'tide gauge',\n", " 'title': 'Nowcast, Point Atkinson, Sea Surface Height, 15min, v1',\n", " 'summary': '''Sea surface height values averaged over 15 minute intervals from\n", "SalishSeaCast NEMO model nowcast runs. The values are calculated at the model grid point\n", "closest to the Point Atkinson tide gauge station on the north side of English Bay,\n", "near Vancouver, British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: ssh variable''',\n", " 'fileNameRegex': '.*PointAtkinson\\.nc$',\n", " },\n", " \n", " 'ubcSSnCampbellRiverSSH15mV1': {\n", " 'type': 'tide gauge',\n", " 'title': 'Nowcast, Campbell River, Sea Surface Height, 15min, v1',\n", " 'summary': '''Sea surface height values averaged over 15 minutes intervals from\n", "SalishSeaCast NEMO model nowcast runs. The values are calculated at the model grid point\n", "closest to the Campbell River tide gauge station at the north end of the Strait of Georgia,\n", "near Campbell River, British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: ssh variable''',\n", " 'fileNameRegex': '.*CampbellRiver\\.nc$',\n", " },\n", "\n", " 'ubcSSnCherryPointSSH15mV1': {\n", " 'type': 'tide gauge',\n", " 'title': 'Nowcast, Cherry Point, Sea Surface Height, 15min, v1',\n", " 'summary': '''Sea surface height values averaged over 15 minutes intervals from\n", "SalishSeaCast NEMO model nowcast runs. The values are calculated at the model grid point\n", "closest to the Cherry Point tide gauge station in the southern Strait of Georgia,\n", "near Birch Bay, Washington.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: ssh variable''',\n", " 'fileNameRegex': '.*CherryPoint\\.nc$',\n", " },\n", "\n", " 'ubcSSnFridayHarborSSH15mV1': {\n", " 'type': 'tide gauge',\n", " 'title': 'Nowcast, Friday Harbor, Sea Surface Height, 15min, v1',\n", " 'summary': '''Sea surface height values averaged over 15 minutes intervals from\n", "SalishSeaCast NEMO model nowcast runs. The values are calculated at the model grid point\n", "closest to the Friday Harbor tide gauge station at San Juan Island in Haro Strait,\n", "near Friday Harbor, Washington.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: ssh variable''',\n", " 'fileNameRegex': '.*FridayHarbor\\.nc$',\n", " },\n", "\n", " 'ubcSSnNanaimoSSH15mV1': {\n", " 'type': 'tide gauge',\n", " 'title': 'Nowcast, Nanaimo, Sea Surface Height, 15min, v1',\n", " 'summary': '''Sea surface height values averaged over 15 minutes intervals from\n", "SalishSeaCast NEMO model nowcast runs. The values are calculated at the model grid point\n", "closest to the Nanaimo tide gauge station on the west side of the central Strait of Georgia,\n", "near Nanaimo, British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: ssh variable''',\n", " 'fileNameRegex': '.*Nanaimo\\.nc$',\n", " },\n", "\n", " 'ubcSSnNeahBaySSH15mV1': {\n", " 'type': 'tide gauge',\n", " 'title': 'Nowcast, Neah Bay, Sea Surface Height, 15min, v1',\n", " 'summary': '''Sea surface height values averaged over 15 minutes intervals from\n", "SalishSeaCast NEMO model nowcast runs. The values are calculated at the model grid point\n", "closest to the Neah Bay tide gauge station on the south side of the west end of the Juan de Fuca Strait,\n", "near Neah Bay, Washington.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: ssh variable''',\n", " 'fileNameRegex': '.*NeahBay\\.nc$',\n", " },\n", "\n", " 'ubcSSnVictoriaSSH15mV1': {\n", " 'type': 'tide gauge',\n", " 'title': 'Nowcast, Victoria, Sea Surface Height, 15min, v1',\n", " 'summary': '''Sea surface height values averaged over 15 minutes intervals from\n", "SalishSeaCast NEMO model nowcast runs. The values are calculated at the model grid point\n", "closest to the Victoria tide gauge station on the north side of the east end of the Juan de Fuca Strait,\n", "in the Victoria Inner Harbour, near Victoria, British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: ssh variable''',\n", " 'fileNameRegex': '.*Victoria\\.nc$',\n", " },\n", "\n", " 'ubcSSnSandHeadsSSH15mV1': {\n", " 'type': 'tide gauge',\n", " 'title': 'Nowcast, Sand Heads, Sea Surface Height, 15min, v1',\n", " 'summary': '''Sea surface height values averaged over 15 minutes intervals from\n", "SalishSeaCast NEMO model nowcast runs. The values are calculated at the model grid point\n", "closest to the Sand Heads light station on the east side of the central Strait of Georgia,\n", "near Steveston, British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: ssh variable''',\n", " 'fileNameRegex': '.*Sandheads\\.nc$',\n", " },\n", " \n", " 'ubcSSn3DTracerFields1hV1': {\n", " 'type': '3d fields',\n", " 'title': 'Nowcast, Salish Sea, 3d Tracer Fields, Hourly, v1',\n", " 'summary': '''3d salinity and water temperature field values averaged over 1 hour intervals\n", "from SalishSeaCast NEMO model nowcast runs. The values are calculated for the entire model grid\n", "that includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: salinity (practical) and temperature variables''',\n", " 'fileNameRegex': '.*SalishSea_1h_\\d{8}_\\d{8}_grid_T\\.nc$',\n", " },\n", " \n", " 'ubcSSnSurfaceTracerFields1hV1': {\n", " 'type': 'surface fields',\n", " 'title': 'Nowcast, Salish Sea, Surface Tracer Fields, Hourly, v1',\n", " 'summary': '''2d sea surface height and rainfall rate field values averaged over 1 hour intervals\n", "from SalishSeaCast NEMO model nowcast runs. The values are calculated for the surface of the model grid\n", "that includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: sea surface height and rainfall rate variables''',\n", " 'fileNameRegex': '.*SalishSea_1h_\\d{8}_\\d{8}_grid_T\\.nc$',\n", " },\n", " \n", " 'ubcSSn3DuVelocity1hV1': {\n", " 'type': '3d fields',\n", " 'title': 'Nowcast, Salish Sea, 3d u Velocity Field, Hourly, v1',\n", " 'summary': '''3d zonal (u) component velocity field values averaged over 1 hour intervals\n", "from SalishSeaCast NEMO model nowcast runs. The values are calculated for the entire model grid\n", "that includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: uVelocity variable''',\n", " 'fileNameRegex': '.*SalishSea_1h_\\d{8}_\\d{8}_grid_U\\.nc$',\n", " },\n", " \n", " 'ubcSSn3DvVelocity1hV1': {\n", " 'type': '3d fields',\n", " 'title': 'Nowcast, Salish Sea, 3d v Velocity Field, Hourly, v1',\n", " 'summary': '''3d meridional (v) component velocity field values averaged over 1 hour intervals\n", "from SalishSeaCast NEMO model nowcast runs. The values are calculated for the entire model grid\n", "that includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: vVelocity variable''',\n", " 'fileNameRegex': '.*SalishSea_1h_\\d{8}_\\d{8}_grid_V\\.nc$',\n", " },\n", " \n", " 'ubcSSn3DwVelocity1hV1': {\n", " 'type': '3d fields',\n", " 'title': 'Nowcast, Salish Sea, 3d w Velocity Field, Hourly, v1',\n", " 'summary': '''3d vertical (w) component velocity field values averaged over 1 hour intervals\n", "from SalishSeaCast NEMO model nowcast runs. The values are calculated for the entire model grid\n", "that includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "Geo-location and depth data for the SalishSeaCast NEMO model grid are available in the ubcSSnBathymetry2V1 dataset.\n", "\n", "v1: wVelocity variable''',\n", " 'fileNameRegex': '.*SalishSea_1h_\\d{8}_\\d{8}_grid_W\\.nc$',\n", " },\n", " \n", " 'ubcSSaSurfaceAtmosphereFieldsV1': {\n", " 'type': 'surface fields',\n", " 'title': 'HRDPS, Salish Sea, Atmospheric Forcing Fields, Hourly, v1',\n", " 'summary': '''2d hourly atmospheric field values from the\n", "Environment Canada HRDPS atmospheric forcing model that are used to force the SalishSeaCast NEMO model.\n", "The model grid includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "Geo-location data for the atmospheric forcing grid are available in the ubcSSaAtmosphereGridV1 dataset.\n", "Atmospheric field values are interpolated on to the SalishSeaCast NEMO model grid (ubcSSnBathymetry2V1 dataset)\n", "on-the-fly by NEMO.\n", "\n", "v1: atmospheric pressure, precipitation rate, 2m specific humidity, 2m air temperature,\n", "short-wave radiation flux, long-wave radiation flux, 10m u wind component, 10m v wind component variables''',\n", " 'fileNameRegex': '.*ops_y\\d{4}m\\d{2}d\\d{2}\\.nc$',\n", " },\n", "}\n", "\n", "datasets['ubcSSn3DwVelocity1hV2'] = datasets['ubcSSn3DwVelocity1hV1']\n", "datasets['ubcSSn3DwVelocity1hV2'].update({\n", " 'title': datasets['ubcSSn3DwVelocity1hV1']['title'].replace(', v1', ', v2'),\n", " 'summary': datasets['ubcSSn3DwVelocity1hV1']['summary'] + '''\n", "v2: Added eddy viscosity & diffusivity variables ve_eddy_visc & ve_eddy_diff''',\n", "})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `dataset_vars` dictionary below is used to rename\n", "variables from the often cryptic NEMO names to the names\n", "that appear in the ERDDAP generated files and web content.\n", "\n", "The keys are the NEMO variable names to replace.\n", "\n", "The values are dicts that map the variable names to use in ERDDAP\n", "to the `destinationName` attribute name." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "dataset_vars = {\n", " 'sossheig': {'destinationName': 'ssh'},\n", " 'vosaline': {'destinationName': 'salinity'},\n", " 'votemper': {'destinationName': 'temperature'},\n", " 'vozocrtx': {'destinationName': 'uVelocity'},\n", " 'vomecrty': {'destinationName': 'vVelocity'},\n", " 'vovecrtz': {'destinationName': 'wVelocity'},\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A few convenient functions to reduce code repetition:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def print_tree(root):\n", " \"\"\"Display an XML tree fragment with indentation.\n", " \"\"\"\n", " print(etree.tostring(root, pretty_print=True).decode('ascii'))" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def find_att(root, att):\n", " \"\"\"Return the dataset attribute element named att\n", " or raise a ValueError exception if it cannot be found.\n", " \"\"\"\n", " e = root.find('.//att[@name=\"{}\"]'.format(att))\n", " if e is None:\n", " raise ValueError('{} attribute element not found'.format(att))\n", " return e" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def replace_yx_with_lonlat(root):\n", " new_axes = {\n", " 'y': {'sourceName': 'nav_lon', 'destinationName': 'longitude'},\n", " 'x': {'sourceName': 'nav_lat', 'destinationName': 'latitude'},\n", " }\n", " for axis in root.findall('.//axisVariable'):\n", " if axis.find('.//sourceName').text in new_axes:\n", " key = axis.find('.//sourceName').text\n", " new_axis = etree.Element('axisVariable')\n", " etree.SubElement(new_axis, 'sourceName').text = new_axes[key]['sourceName']\n", " etree.SubElement(new_axis, 'destinationName').text = new_axes[key]['destinationName']\n", " axis.getparent().replace(axis, new_axis)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we're ready to produce a dataset!!!\n", "\n", "Use the `/opt/tomcat/webapps/erddap/WEB-INF/GenerateDatasetsXml.sh` script\n", "generate the initial version of an XML fragment for a dataset:\n", "```\n", "$ cd /opt/tomcat/webapps/erddap/WEB-INF/\n", "$ bash GenerateDatasetsXml.sh EDDGridFromNcFiles /results/SalishSea/nowcast/\n", "```\n", "The `EDDGridFromNcFiles` and `/results/SalishSea/nowcast/` arguments\n", "tell the script which `EDDType` and what parent directory to use,\n", "avoiding having to type those in answer to prompts.\n", "Answer the remaining prompts,\n", "for example:\n", "```\n", "File name regex (e.g., \".*\\.nc\") (default=\"\")\n", "? .*SalishSea_1h_\\d{8}_\\d{8}_grid_W\\.nc$\n", "\n", "Full file name of one file (default=\"\")\n", "? /results/SalishSea/nowcast/28jan16/SalishSea_1h_20160128_20160128_grid_W.nc\n", "\n", "ReloadEveryNMinutes (e.g., 10080) (default=\"\")\n", "? 10080\n", "```\n", "Other examples of file name regex are:\n", "\n", "* `.*PointAtkinson.nc$`\n", "* `.*SalishSea_1d_\\d{8}_\\d{8}_grid_W\\.nc$`\n", "\n", "The output is written to `/results/erddap/logs/GenerateDatasetsXml.out`\n", "\n", "Now, we:\n", "\n", "* set the `datasetID` we want to use\n", "* parse the output of `GenerateDatasetsXml.sh` into an XML tree data structure\n", "* set the `datasetID` dataset attribute value\n", "* re-set the `fileNameRegex` dataset attribute value because it looses its `\\` characters during parsing(?)\n", "* edit and add dataset attributes from the `metadata` dict\n", "* set the `title` and `summary` dataset attributes from the `datasets` dict\n", "* set the names of the grid `x` and `y` axis variables\n", "* rename data variables as specified in the `dataset_vars` dict" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def update_xml(root, datasetID, metadata, datasets, dataset_vars):\n", " root.attrib['datasetID'] = datasetID\n", " root.find('.//fileNameRegex').text = datasets[datasetID]['fileNameRegex']\n", "\n", " for att, info in metadata.items():\n", " e = etree.Element('att', name=att)\n", " e.text = info['text']\n", " try:\n", " root.find('.//att[@name=\"{}\"]'.format(info['after'])).addnext(e)\n", " except KeyError:\n", " find_att(root, att).text = info['text']\n", " \n", " title = datasets[datasetID]['title']\n", " find_att(root, 'title').text = title\n", " find_att(root, 'summary').text = '{0}\\n\\n{1}'.format(title, datasets[datasetID]['summary'])\n", "\n", " for axis_name in root.findall('.//axisVariable/destinationName'):\n", " if axis_name.text in ('x', 'y'):\n", " axis_name.text = 'grid{}'.format(axis_name.text.upper())\n", " if datasets[datasetID]['type'] == 'tide gauge':\n", " replace_yx_with_lonlat(root)\n", " \n", " for var_name in root.findall('.//dataVariable/destinationName'):\n", " if var_name.text in dataset_vars:\n", " var_name.text = dataset_vars[var_name.text]['destinationName']" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [], "source": [ "parser = etree.XMLParser(remove_blank_text=True)\n", "tree = etree.parse('/results/erddap/logs/GenerateDatasetsXml.out', parser)\n", "root = tree.getroot()\n", "\n", "datasetID = 'ubcSSn3dTracerFields1hV1'\n", "\n", "update_xml(root, datasetID, metadata, datasets, dataset_vars)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Inspect the resulting dataset XML fragment below and edit the dicts and\n", "code cell above until it is what is required for the dataset:" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 10080\n", " 10000\n", " /results/SalishSea/nowcast/\n", " true\n", " .*SalishSea_1h_d{8}_d{8}_grid_T.nc$\n", " last\n", " 20\n", " false\n", " false\n", " \n", " \n", " Grid\n", " modelResult\n", " CF-1.6, COARDS, ACDD-1.3\n", " https://salishsea-meopar-tools.readthedocs.org/en/latest/results_server/index.html#salish-sea-model-results\n", " UBC EOAS\n", " Earth, Ocean & Atmospheric Sciences, University of British Columbia\n", " data, density, deptht, local, ocean, oceans,\n", "Oceans > Salinity/Density > Salinity,\n", "practical, salinity, sea, sea_water_practical_salinity, seawater, source, temperature, time_counter, vosaline, votemper, water\n", " GCMD Science Keywords\n", " The SalishSeaCast NEMO model results are copyright 2013 – present\n", "by the SalishSeaCast Project Contributors and The University of British Columbia.\n", "\n", "They are licensed under the Apache License, Version 2.0. http://www.apache.org/licenses/LICENSE-2.0\n", " CF Standard Name Table v29\n", " Nowcast, Salish Sea, 3d Tracer Fields, Hourly, v1\n", "\n", "3d salinity and water temperature field values averaged over 1 hour intervals \n", "from SalishSeaCast NEMO model nowcast runs. The values are calculated for the entire model grid \n", "that includes the Strait of Juan de Fuca, the Strait of Georgia, Puget Sound, \n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "\n", "v1 salinity and temperature variables\n", " \n", " Nowcast, Salish Sea, 3d Tracer Fields, Hourly, v1\n", " SalishSeaCast NEMO Model\n", " SalishSeaCast Project Contributors\n", " sallen@eoas.ubc.ca\n", " https://salishsea-meopar-docs.readthedocs.org/\n", " MEOPAR, ONC, Compute Canada\n", " over\n", " \n", " \n", " time_counter\n", " time\n", " \n", " \n", " null\n", " null\n", " Statistics\n", " \n", " \n", " \n", " deptht\n", " depth\n", " \n", " \n", " null\n", " Location\n", " depth\n", " \n", " \n", " \n", " y\n", " gridY\n", " \n", " \n", " Location\n", " Y\n", " \n", " \n", " \n", " x\n", " gridX\n", " \n", " \n", " Location\n", " X\n", " \n", " \n", " \n", " vosaline\n", " salinity\n", " float\n", " \n", " \n", " null\n", " 37.0\n", " 32.0\n", " null\n", " Salinity\n", " sea_water_practical_salinity\n", " PSU\n", " \n", " \n", " \n", " votemper\n", " temperature\n", " float\n", " \n", " \n", " null\n", " 40.0\n", " -10.0\n", " null\n", " Temperature\n", " null\n", " \n", " \n", "\n", "\n" ] } ], "source": [ "print_tree(root)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Extra processing step are required for some types of datasets.\n", "See:\n", "\n", "* [Surface Field Datasets](#Surface-Field-Datasets)\n", "* [Model Grid Geo-location and Bathymetry Datasets](#Model-Grid-Geo-location-and-Bathymetry-Datasets)\n", "* [EC HDRPS Atmospheric Forcing Datasets](#EC-HDRPS-Atmospheric-Forcing-Datasets)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Store the XML fragment for the dataset:" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": true }, "outputs": [], "source": [ "with open('/results/erddap_datasets_xml/{}.xml'.format(datasetID), 'wb') as f:\n", " f.write(etree.tostring(root, pretty_print=True))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Edit `/opt/tomcat/content/erddap/datasets.xml` to include the\n", "XML fragment for the dataset that was stored by the abave cell.\n", "\n", "Run `/opt/tomcast/webapps/erddap/WEB-INF/DasDds.sh` to:\n", "\n", "* confirm that the XML fragment is correct\n", "* create the `.das` and `.ddl` files for the dataset\n", "\n", "`DasDds.hs` takes a `datasetID` as input from the command-line\n", "and generates a ridiculous amount of output on the screen\n", "and in `/results/erddap/logs/DasDds.log`\n", "and `/results/erddap/logs/DasDds.out`.\n", "\n", "If `DasDds.sh` finishes without errors and produces\n", "metadata that looks sensible for the dataset,\n", "create a flag file to signal the ERDDAP server process to load the dataset:\n", "```\n", "$ cd /results/erddap/flag/\n", "$ touch \n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Surface Field Datasets\n", "\n", "The `/opt/tomcat/webapps/erddap/WEB-INF/GenerateDatasetsXml.sh` script produces and XML\n", "fragment that uses all of the dimensions that it finds in the sample file it parses,\n", "and includes only the variables that have all of those dimensions.\n", "To produce an XML fragment for surface fields we need to do some additional work:\n", "\n", "* Delete the depth axis\n", "* Delete all of the `dataVariable` elements\n", "* Add `dataVariable` elements for the surface variables" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": true }, "outputs": [], "source": [ "for axis in root.findall('.//axisVariable'):\n", " if axis.find('.//destinationName').text == 'depth':\n", " axis.getparent().remove(axis)\n", " break\n", "\n", "for var in root.findall('.//dataVariable'):\n", " var.getparent().remove(var)\n", " \n", "var = etree.SubElement(root, 'dataVariable')\n", "etree.SubElement(var, 'sourceName').text = 'sossheig'\n", "etree.SubElement(var, 'destinationName').text = 'ssh'\n", "etree.SubElement(var, 'dataType').text = 'float'\n", "attrs = etree.SubElement(var, 'addAttributes')\n", "etree.SubElement(attrs, 'att', name='_ChunkSize').text = 'null'\n", "etree.SubElement(attrs, 'att', name='coordinates').text = 'null'\n", " \n", "var = etree.SubElement(root, 'dataVariable')\n", "etree.SubElement(var, 'sourceName').text = 'rain_rate'\n", "etree.SubElement(var, 'destinationName').text = 'rain_rate'\n", "etree.SubElement(var, 'dataType').text = 'float'\n", "attrs = etree.SubElement(var, 'addAttributes')\n", "etree.SubElement(attrs, 'att', name='_ChunkSize').text = 'null'\n", "etree.SubElement(attrs, 'att', name='coordinates').text = 'null'\n", "\n", "find_att(root, 'keywords').text = (\n", " 'model results, height, local, sea, sea surface height, sossheig, source, surface, time_counter')" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": true }, "outputs": [], "source": [ "print_tree(root)" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": true }, "outputs": [], "source": [ "with open('/results/erddap_datasets_xml/{}.xml'.format(datasetID), 'wb') as f:\n", " f.write(etree.tostring(root, pretty_print=True))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Model Grid Geo-location and Bathymetry Datasets\n", "\n", "Model grid geo-location and bathymetry datasets require a lot of hand editing\n", "because they are not model generated.\n", "Here is an example of a finished one:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 10080\n", " 10000\n", " /results/nowcast-sys/NEMO-forcing/grid/\n", " false\n", " .*SalishSea2_NEMO_bathy\\.nc$\n", " last\n", " 20\n", " false\n", " false\n", " \n", " \n", " Grid\n", " modelResult\n", " CF-1.6, COARDS, ACDD-1.3\n", " https://salishsea-meopar-tools.readthedocs.org/en/latest/results_server/index.html#salish-sea-model-results\n", " UBC EOAS\n", " Earth, Ocean & Atmospheric Sciences, University of British Columbia\n", " Dept of Earth, Ocean & Atmospheric Sciences, University of British Columbia\n", " bathymetry, bottom, data, model results, depth, floor, latitude, longitude, nemo, ocean, oceans,\n", "Oceans > Bathymetry/Seafloor Topography > Bathymetry, salishsea, sea, sea_floor_depth, seafloor, topography\n", " GCMD Science Keywords\n", " The SalishSeaCast NEMO model results are copyright 2013 – present\n", "by the SalishSeaCast Project Contributors and The University of British Columbia.\n", "\n", "They are licensed under the Apache License, Version 2.0. http://www.apache.org/licenses/LICENSE-2.0\n", " CF Standard Name Table v29\n", " SalishSeaCast NEMO model Grid, Geo-location and Bathymetry, v1\n", "\n", "Longitude, latitude, and bathymetry of the SalishSeaCast NEMO model grid.\n", "The bathymetry values are those calculated by NEMO from the input bathymetry file.\n", "NEMO modifies the input bathymetry to remove isolated holes, and too-small partial steps.\n", "The model grid includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "\n", "v1: longitude, latitude and bathymetry variables\n", " SalishSeaCast NEMO Model Grid, Geo-location and Bathymetry, v1\n", " RSalishSeaCast NEMO Model\n", " SalishSeaCast Project Contributors\n", " sallen@eoas.ubc.ca\n", " https://salishsea-meopar-docs.readthedocs.org/\n", " MEOPAR, ONC, Compute Canada\n", " over\n", " \n", " \n", " y\n", " gridY\n", " \n", " \n", " Y\n", " \n", " \n", " \n", " x\n", " gridX\n", " \n", " \n", " X\n", " \n", " \n", " \n", " longitude\n", " longitude\n", " float\n", " \n", " \n", " 180.0\n", " -180.0\n", " longitude\n", " \n", " \n", " \n", " latitude\n", " latitude\n", " float\n", " \n", " \n", " 90.0\n", " -90.0\n", " latitude\n", " \n", " \n", " \n", " bathymetry\n", " bathymetry\n", " float\n", " \n", " \n", " 450.0\n", " 0.0\n", " OceanDepth\n", " null\n", " sea_floor_depth\n", " \n", " \n", "\n", "\n" ] } ], "source": [ "parser = etree.XMLParser(remove_blank_text=True)\n", "tree = etree.parse('/results/erddap_datasets_xml/ubcSSnBathymetry2V1.xml', parser)\n", "root = tree.getroot()\n", "\n", "print_tree(root)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## EC HDRPS Atmospheric Forcing Datasets\n", "\n", "### Atmospheric Forcing Grid Geo-location Dataset\n", "\n", "Use the `/opt/tomcat/webapps/erddap/WEB-INF/GenerateDatasetsXml.sh` script\n", "generate the initial version of an XML fragment for the dataset:\n", "```\n", "$ cd /opt/tomcat/webapps/erddap/WEB-INF/\n", "$ bash GenerateDatasetsXml.sh EDDGridFromNcFiles /results/forcing/atmospheric/GEM2.5/operational/ ops_y\\d{4}m\\d{2}d\\d{2}.nc$ /results/forcing/atmospheric/GEM2.5/operational/ops_y2016m03d07.nc 10080\n", "```\n", "\n", "Like the model grid geo-location and bathymetry dataset,\n", "the atmospheric forcing grid dataset requires a lot of hand editing.\n", "Here is the finished dataset:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 10080\n", " 10000\n", " /results/forcing/atmospheric/GEM2.5/operational/\n", " false\n", " ops_y2016m03d07.nc$\n", " last\n", " 20\n", " false\n", " false\n", " \n", " \n", " Grid\n", " modelResult\n", " CF-1.6, COARDS, ACDD-1.3\n", " null\n", " Mon Mar 7 10:07:34 2016: ncks -4 -L4 -O /results/forcing/atmospheric/GEM2.5/operational/ops_y2016m03d07.nc /results/forcing/atmospheric/GEM2.5/operational/ops_y2016m03d07.nc\n", " created by wgrib2\n", " https://salishsea-meopar-tools.readthedocs.org/en/latest/results_server/index.html#salish-sea-model-results\n", " UBC EOAS\n", " Earth, Ocean & Atmospheric Sciences, University of British Columbia\n", " Dept of Earth, Ocean & Atmospheric Sciences, University of British Columbia\n", " atmosphere, model results, HRDPS, latitude, longitude, salishsea\n", " GCMD Science Keywords\n", " The SalishSeaCast NEMO model results are copyright 2013 – present\n", "by the SalishSeaCast Project Contributors and The University of British Columbia.\n", "\n", "They are licensed under the Apache License, Version 2.0. http://www.apache.org/licenses/LICENSE-2.0\n", "\n", "This dataset is derived from a product of the Environment and Climate Change Canada HRDPS\n", "(High Resolution Deterministic Prediction System) model. The Terms and conditions of use of\n", "Meteorological Data from Environment Canada are available at\n", "http://dd.weather.gc.ca/doc/LICENCE_GENERAL.txt.\n", " CF Standard Name Table v29\n", " HRDPS, Salish Sea, Atmospheric Forcing Grid, Geo-location, v1\n", "\n", "Longitude and latitude of the Environment and Climate Change Canada HRDPS\n", "atmospheric forcing model grid that is used to for the SalishSeaCast NEMO model.\n", "The model grid includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "\n", "v1: longitude and latitude\n", " HRDPS, Salish Sea, Atmospheric Forcing Grid, Geo-location, v1\n", " SalishSeaCast NEMO Model\n", " SalishSeaCast Project Contributors\n", " sallen@eoas.ubc.ca\n", " https://salishsea-meopar-docs.readthedocs.org/\n", " Environment and Climate Change Canada, MEOPAR, ONC, Compute Canada\n", " over\n", " \n", " \n", " y\n", " gridY\n", " \n", " Y\n", " \n", " \n", " \n", " x\n", " gridX\n", " \n", " X\n", " \n", " \n", " \n", " nav_lon\n", " longitude\n", " float\n", " \n", " longitude\n", " \n", " \n", " \n", " nav_lat\n", " latitude\n", " float\n", " \n", " latitude\n", " \n", " \n", "\n", "\n" ] } ], "source": [ "parser = etree.XMLParser(remove_blank_text=True)\n", "tree = etree.parse('/results/erddap_datasets_xml/ubcSSaAtmosphereGridV1.xml', parser)\n", "root = tree.getroot()\n", "\n", "print_tree(root)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Atmospheric Forcing Model Fields\n", "\n", "* Change the value of the `recursive` element to `false` so that the `/results/forcing/atmospheric/GEM2.5/operational/fcst/` directory is excluded\n", "* Add Environment Canada acknowledgement and terms & conditions of use to `license` element\n", "* Add Environment Canada to `acknowledgement` element" ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "collapsed": true }, "outputs": [], "source": [ "root.find('.//recursive').text = 'false'\n", "find_att(root, 'license').text += '''\n", "\n", "This dataset is derived from a product of the Environment Canada HRDPS (High Resolution Deterministic Prediction System)\n", "model. The Terms and conditions of use of Meteorological Data from Environment Canada are available at\n", "http://dd.weather.gc.ca/doc/LICENCE_GENERAL.txt.'''\n", "find_att(root, 'acknowledgement').text += ', Environment Canada'" ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "collapsed": true }, "outputs": [], "source": [ "for axis in root.findall('.//axisVariable'):\n", " axis_name = axis.find('.//sourceName').text\n", " if 'time' not in axis_name:\n", " attrs = axis.find('.//addAttributes')\n", " etree.SubElement(attrs, 'att', name='grid_spacing').text = 'null'\n", " etree.SubElement(attrs, 'att', name='units').text = 'null'\n", " etree.SubElement(attrs, 'att', name='long_name').text = axis_name.upper()\n", " etree.SubElement(attrs, 'att', name='standard_name').text = axis_name" ] }, { "cell_type": "code", "execution_count": 48, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " 10080\n", " 10000\n", " /results/forcing/atmospheric/GEM2.5/operational/\n", " false\n", " .*ops_y\\d{4}m\\d{2}d\\d{2}\\.nc$\n", " last\n", " 20\n", " false\n", " false\n", " \n", " \n", " Grid\n", " modelResult\n", " CF-1.6, COARDS, ACDD-1.3\n", " null\n", " Thu Mar 10 10:11:37 2016: ncks -4 -L4 -O /results/forcing/atmospheric/GEM2.5/operational/ops_y2016m03d10.nc /results/forcing/atmospheric/GEM2.5/operational/ops_y2016m03d10.nc\n", "created by wgrib2\n", " https://salishsea-meopar-tools.readthedocs.org/en/latest/results_server/index.html#salish-sea-model-results\n", " UBC EOAS\n", " Earth, Ocean & Atmospheric Sciences, University of British Columbia\n", " atmosphere,\n", "Atmosphere > Atmospheric Water Vapor > Humidity,\n", "atmospheric, atmpres, component, data, downward, flux, humidity, level, local, long, long-wave, mean, msl, precipitation, pressure, qair, rad, radiation, rain, rainfall, reduced, sea, seawater, short, short-wave, solar, source, specific, specific_humidity, tair, temperature, therm_rad, time_counter, total, u-component, u_wind, v-component, v_wind, vapor, water, wave, wind\n", " GCMD Science Keywords\n", " The SalishSeaCast NEMO model results are copyright 2013 – present\n", "by the SalishSeaCast Project Contributors and The University of British Columbia.\n", "\n", "They are licensed under the Apache License, Version 2.0. http://www.apache.org/licenses/LICENSE-2.0\n", "\n", "This dataset is derived from a product of the Environment and Climate Change Canada HRDPS\n", "(High Resolution Deterministic Prediction System) model.\n", "The Terms and conditions of use of Meteorological Data from Environment Canada are available at\n", "http://dd.weather.gc.ca/doc/LICENCE_GENERAL.txt.</att>\n", " CF Standard Name Table v29\n", " HRDPS, Salish Sea, Atmospheric Forcing Fields, Hourly, v1\n", "\n", "2d hourly atmospheric field values from the\n", "Environment and Climate Change Canada HRDPS atmospheric forcing model that are used to force the SalishSeaCast NEMO model.\n", "The model grid includes the Juan de Fuca Strait, the Strait of Georgia, Puget Sound,\n", "and Johnstone Strait on the coasts of Washington State and British Columbia.\n", "Geo-location data for the atmospheric forcing grid are available in the ubcSSaAtmosphereGridV1 dataset.\n", "Atmospheric field values are interpolated on to the SalishSeaCast NEMO model grid (ubcSSnBathymetry2V1 dataset)\n", "on-the-fly by NEMO.\n", "\n", "v1: atmospheric pressure, precipitation rate, 2m specific humidity, 2m air temperature,\n", "short-wave radiation flux, long-wave radiation flux, 10m u wind component, 10m v wind component variables\n", " HRDPS, Salish Sea, Atmospheric Forcing Fields, Hourly, v1\n", " SalishSeaCast NEMO Model\n", " SalishSeaCast Project Contributors\n", " sallen@eoas.ubc.ca\n", " https://salishsea-meopar-docs.readthedocs.org/\n", " MEOPAR, ONC, Compute Canada, Environment and Climate Change Canada\n", " over\n", " \n", " \n", " time_counter\n", " time\n", " \n", " \n", " null\n", " time\n", " \n", " \n", " \n", " y\n", " gridY\n", " \n", " \n", " null\n", " null\n", " null\n", " Y\n", " y\n", " \n", " \n", " \n", " x\n", " gridX\n", " \n", " \n", " null\n", " null\n", " null\n", " X\n", " x\n", " \n", " \n", " \n", " atmpres\n", " atmpres\n", " float\n", " \n", " \n", " null\n", " null\n", " \n", " \n", " \n", " precip\n", " precip\n", " float\n", " \n", " \n", " null\n", " 200.0\n", " 0.0\n", " null\n", " \n", " \n", " \n", " qair\n", " qair\n", " float\n", " \n", " \n", " null\n", " 128.0\n", " 0.0\n", " null\n", " specific_humidity\n", " \n", " \n", " \n", " solar\n", " solar\n", " float\n", " \n", " \n", " null\n", " 500.0\n", " -500.0\n", " null\n", " \n", " \n", " \n", " tair\n", " tair\n", " float\n", " \n", " \n", " null\n", " 313.0\n", " 263.0\n", " null\n", " \n", " \n", " \n", " therm_rad\n", " therm_rad\n", " float\n", " \n", " \n", " null\n", " null\n", " \n", " \n", " \n", " u_wind\n", " u_wind\n", " float\n", " \n", " \n", " null\n", " null\n", " \n", " \n", " \n", " v_wind\n", " v_wind\n", " float\n", " \n", " \n", " null\n", " null\n", " \n", " \n", "\n", "\n" ] } ], "source": [ "print_tree(root)" ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "collapsed": true }, "outputs": [], "source": [ "with open('/results/erddap_datasets_xml/{}.xml'.format(datasetID), 'wb') as f:\n", " f.write(etree.tostring(root, pretty_print=True))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.1" } }, "nbformat": 4, "nbformat_minor": 0 }