{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Basic Info\n",
"\n",
"**Please note that [nwm package](https://github.com/gantian127/nwm) is deprecated.** This package now only serves as an example to demonstrate how to implement Basic Model Interface ([BMI](https://bmi.readthedocs.io/en/latest/))\n",
"for research datasets as the [CSDMS Data Component](https://csdms.colorado.edu/wiki/DataComponents). \n",
"\n",
"Suggested citation: Gan, T. (2020). Jupyter Notebook for the nwm Python package, HydroShare, http://www.hydroshare.org/resource/87fa0749a0b944228e3c613dc7d8899b"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Quick Start Tutorial "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook will help you get started using the nwm package to download the National Water Model (NWM) datasets. \n",
"\n",
"This tutorial includes the following sections:\n",
"\n",
"1. [Brief Introduction](#section1)\n",
"\n",
" This section provides basic information about nwm package. \n",
"
\n",
" \n",
"2. [Start with Examples](#section2)\n",
" \n",
" This section provides two examples to demonstrate how to use nwm to download datasets for visualization.\n",
"
\n",
" \n",
"3. [Write Your Own Code](#section3)\n",
"\n",
" This section provides guide to write your own code and explore the NWM datasets for hurricane events. \n",
"
\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## 1. Brief Introduction"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"nwm package provides a set of functions that allows downloading of the National Water Model ([NWM](https://water.noaa.gov/about/nwm)) datasets for data analysis and visualization. These functions were implemented using the API of the HydroShare National Water Model Web App. A HydroShare [account](https://www.hydroshare.org/sign-up/) is required to access [this app](https://hs-apps.hydroshare.org/apps/nwm-forecasts/).\n",
"\n",
"nwm package also includes a Basic Model Interface ([BMI](https://bmi.readthedocs.io/en/latest/)), which converts the NWM dataset into a reusable, plug-and-play data component for [PyMT](https://pymt.readthedocs.io/en/latest/?badge=latest) modeling framework developed by Community Surface Dynamics Modeling System ([CSDMS](https://csdms.colorado.edu/wiki/Main_Page)) \n",
"\n",
"\n",
"To install nwm package, you can use the following command:\n",
"```\n",
"$ pip install nwm\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## 2. Start with Examples"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In nwm package, NwmHs class is designed for users to download datasets. BmiNwmHs class is designed to convert NWM dataset as a data component for the [PyMT](https://pymt.readthedocs.io/en/latest/?badge=latest) modeling framework. The following examples demonstrate how to download the same dataset using NwmHs and BmiNwmHs for data visualization. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Example 1: use NwmHs class to download data (Recommended method)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Import NwmHs class and download data with **get_data( )** method. This example downloads short range (18 hours) forecast of streamflow at a river channel during a hurricane event. You can check the details of the [parameter settings](https://nwm.readthedocs.io/en/latest/#parameter-settings) for get_data( ) method to better understand the parameter values used in the example."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"from nwm import NwmHs\n",
"\n",
"# download streamflow data\n",
"nwm_data = NwmHs()\n",
"dataset = nwm_data.get_data(archive='harvey', config='short_range', geom='channel_rt',\n",
" variable='streamflow', comid=[5781915], init_time=0, \n",
" start_date='2017-08-23')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The downloaded NWM dataset is stored as a self-described xarray object. With this data object, you can check the metadata and make time series plot."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# show metadata\n",
"dataset.attrs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# plot data\n",
"plt.figure(figsize=(9,5))\n",
"dataset.plot()\n",
"plt.xlabel('Year 2017')\n",
"plt.ylabel('{} ({})'.format(dataset.variable_name,dataset.variable_unit))\n",
"plt.title('Short range streamflow forecast for Channel 5781915 during Harvey Hurricane Event')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Example 2: use BmiNwmHs class to download data (Demonstration of how to use BMI)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Import BmiNwmHs class and instantiate it. A configuration file (yaml file) is required to provide the parameter settings for data download. An example configure_file.yaml file is provided in the same folder with this Jupyter Notebook file. You can also skip this example and try section 3 to use NwmHs class for data download."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import cftime\n",
"\n",
"from nwm import BmiNwmHs\n",
"\n",
"# initiate a data component\n",
"data_comp = BmiNwmHs()\n",
"data_comp.initialize('config_file.yaml')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use variable related methods from BmiNwmHs class to check the variable information of the NWM dataset. This data component stores a flow forecast variable. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# get variable info\n",
"var_name = data_comp.get_output_var_names()[0]\n",
"var_unit = data_comp.get_var_units(var_name)\n",
"print(' variable_name: {}\\n var_unit: {}\\n'.format(var_name, var_unit))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use time related methods of BmiNwmHs class to check the time information of the NWM dataset. The time values are stored in a format which follows [CF convention](http://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.pdf). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# get time info\n",
"start_time = data_comp.get_start_time()\n",
"end_time = data_comp.get_end_time()\n",
"time_step = data_comp.get_time_step()\n",
"time_unit = data_comp.get_time_units()\n",
"time_steps = int((end_time - start_time)/time_step) + 1\n",
"print(' start_time:{}\\n end_time:{}\\n time_step:{}\\n time_unit:{}\\n time_steps:{}\\n'.format(start_time, end_time, time_step, time_unit, time_steps))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Loop through each time step to get the flow and time values. stream_array stores flow forecast values. cftime_array stores the numerical time values. time_array stores the corresponding Python datetime objects. get_value( ) method returns the flow forecast value at each time step. update( ) method updates the current time step of the data component."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# initiate numpy arrays to store data\n",
"stream_value = np.empty(1)\n",
"stream_array = np.empty(time_steps)\n",
"cftime_array = np.empty(time_steps)\n",
"\n",
"for i in range(0, time_steps):\n",
" data_comp.get_value(var_name, stream_value)\n",
" stream_array[i] = stream_value\n",
" cftime_array[i] = data_comp.get_current_time()\n",
" data_comp.update()\n",
" \n",
"time_array = cftime.num2date(cftime_array, time_unit, only_use_cftime_datetimes=False, only_use_python_datetimes=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# plot data\n",
"plt.figure(figsize=(9,5))\n",
"plt.plot(time_array, stream_array)\n",
"plt.xlabel('Year 2017')\n",
"plt.ylabel('{} ({})'.format(var_name, var_unit))\n",
"plt.title('Short range streamflow forecast for Channel 5781915 during Harvey Hurricane Event')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## 3. Write Your Own Code "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"NwmHs class can be used to download datasets for several hurricane events, including [Harvey (2017)](https://en.wikipedia.org/wiki/Hurricane_Harvey), [Irma (2017)](https://en.wikipedia.org/wiki/Hurricane_Irma), and [Florence (2018)](https://en.wikipedia.org/wiki/Hurricane_Florence). Try with the following instructions and write your own code to explore the data for Hurricane Harvey. You can check the [parameter settings](https://nwm.readthedocs.io/en/latest/#parameter-settings) for get_data( ) method to download various climate forcing and forecast datasets."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Hurricane Harvey \n",
"\n",
"Hurricane Harvey made landfall on Texas and Louisiana in August 2017. It caused catastrophic flooding and many deaths. In this use case, we will explore the datasets for Nederland, Texas, an area impacted by flooding during the hurricane event."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"from nwm import NwmHs\n",
"\n",
"nwm_data = NwmHs()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Step1: Explore climate forcing data**\n",
"\n",
"Download the rain rate data of a grid cell (comid=[833, 2596]) which locates in the Nederland area. The data is **short range** forecast for 2017-08-28 with model initiation time as 00:00."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# download rain rate data\n",
"rain_rate = "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Double-click __here__ for the solution.*\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the rain rate data. Heavy rainfall is usually more than 0.30 in/hr. What is the max rain rate for the grid area on that day?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# plot rain rate data\n",
"rain_rate.plot()\n",
"plt.ylabel('{} ({})'.format(rain_rate.variable_name,rain_rate.variable_unit))\n",
"plt.title('Rain rate for grid cell [833, 2596]')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Download the air temperature data for the same grid cell (comid=[833, 2596]). Let's get this climate forcing data for **medium range** forecast starting from 2017-08-28 with model initiation time as 00:00."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# download air temperature data\n",
"air_temp = "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Double-click __here__ for the solution.*\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the air temperature data. What is the max and min air temperature for the grid area during 2017-08-28 to 2017-09-27?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# plot air temperature data\n",
"air_temp.plot()\n",
"plt.ylabel('{} ({})'.format(air_temp.variable_name,air_temp.variable_unit))\n",
"plt.title('Air temperature for grid cell [833, 2596]')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Step2: Explore streamflow forecast results**\n",
"\n",
"Download the streamflow data for a river channel (comid=[1112323]) which locates in the grid cell area. First try to get the **short range** forecast for 2017-08-28 with model initiation time as 00:00. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# download short range streamflow forecast \n",
"short_range = "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Double-click __here__ for the solution.*\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Download the streamflow data for the same river channel (comid=[1112323]). Let's get the **medium range** and **long range** forecast datasets starting from 2017-08-28 with model initiation time as 00:00. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# download medium range streamflow forecast\n",
"medium_range = \n",
"\n",
"# download long range streamflow forecast\n",
"long_range = "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Double-click __here__ for the solution.*\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Download **analysis and assimilation** streamflow data from 2017-08-28 to 2017-09-10 for the same river channel (comid=[1112323]). The analysis and assimilation configuration produces a real-time analysis of the streamflow. The stream-gauge observations are assimilated from the USGS."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# download analysis and assimilation streamflow data\n",
"analysis_assim = "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Double-click __here__ for the solution.*\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's plot all the streamflow datasets. From this plot, what do you find from the results produced by the four configurations? "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# plot the streamflow forecast data\n",
"plt.figure(figsize=(14,7))\n",
"short_range.plot()\n",
"medium_range.plot()\n",
"long_range.plot()\n",
"analysis_assim.plot()\n",
"\n",
"plt.ylabel('{} ({})'.format(short_range.variable_name,short_range.variable_unit))\n",
"plt.title('Streamflow forecast and analysis for Channel 1112323')\n",
"plt.legend(labels=['short range','medium range','long range','analysis assimilation'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Step3: Explore other forecast results**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Aside from streamflow data, NWM provides forecasts for other variables (e.g., soil moisture and groundwater runoff). We will download and visualize some of these datasets.\n",
"\n",
"Download the near surface soil saturation data for the same grid cell (comid=[833, 2596]). The data is **short range** forecast for 2017-08-28 with model initiation time as 00:00."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# download near surface soil saturation data\n",
"soil_sat = "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Double-click __here__ for the solution.*\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the soil saturation data. It can be found that the near surface soil is highly saturated during heavy rainfall period (check with the rain rate plot above). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# plot near surface soil saturation data\n",
"soil_sat.plot()\n",
"plt.ylabel('soil saturation (m^3/m^3)')\n",
"plt.title('Near surface soil saturation for grid cell [833, 2596]')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Download the accumulated groundwater runoff data for the same grid cell (comid=[833, 2596]). The data is **long range** forecast starting from 2017-08-28 with model initiation time as 00:00."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# download accumulated groundwater runoff data\n",
"groundwater_runoff = "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Double-click __here__ for the solution.*\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the accumulated groundwater runoff data. How much groundwater runoff was generated during Aug 28-30th? "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# plot accumulated groundwater runoff data\n",
"groundwater_runoff.plot()\n",
"plt.ylabel('{} ({})'.format(groundwater_runoff.variable_name,groundwater_runoff.variable_unit))\n",
"plt.title('Accumulated groundwater runoff for grid cell [833, 2596]')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}