{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Angular data\n",
"Sometimes we are interested in evaluating angular forecasts, such as a wind direction forecast against wind direction observations. \n",
"\n",
"Several metrics in `scores` can be used to evaluate angular forecasts expressed in degrees. If the `is_angular` argument is available in a metric, then that function can be used with angular data and it will handle the discontinuity that occurs between 360 degrees and 0. Simply set `is_angular=True`. It does this by calculating the length of the smaller of the two explementary angles between the two sources of data (e.g., forecast and observed).\n",
"\n",
"If your data is in radians, you can convert it to degrees my multiplying your data by 180/pi (you can use `numpy.pi`)."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"from scores.continuous import mae\n",
"\n",
"import numpy as np\n",
"import xarray as xr\n",
"\n",
"np.random.seed(0) # set the seed to make notebook reproducible"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"fcst = xr.DataArray(\n",
" data=np.random.random_sample((1000, 1000)) * 360, \n",
" dims=[\"space\", \"time\"], \n",
" coords=[np.arange(0, 1000), np.arange(0, 1000)]\n",
")\n",
"obs = xr.DataArray(\n",
" data=np.random.random_sample((1000, 1000)) * 360, \n",
" dims=[\"space\", \"time\"], \n",
" coords=[np.arange(0, 1000), np.arange(0, 1000)]\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's calculate MAE with `angular=True`"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"
<xarray.DataArray ()>\n",
"array(89.9521687)
xarray.DataArray
89.95
array(89.9521687)
"
],
"text/plain": [
"\n",
"array(89.9521687)"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mae(fcst, obs, is_angular=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since the forecast and observed values are random taken from a uniform distribution on the interval [0, 360), we would expect the MAE to be around 90. Let's see what happens when we don't use the `is_angular=True`."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"
<xarray.DataArray ()>\n",
"array(120.03938622)
xarray.DataArray
120.0
array(120.03938622)
"
],
"text/plain": [
"\n",
"array(120.03938622)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mae(fcst, obs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The value is far higher, since we didn't use the smaller length of the two explementary angle.\n",
"\n",
"## Other things to try:\n",
"- Try out some of the other metrics set up to work with angular data including MSE, RMSE, and the Flip-Flop Index.\n",
"- Read up on the Circular Flip-Flop Index which handles angular data. [Griffiths, D., Loveday, N., Price, B., Foley, M. and McKelvie, A., 2021. Circular Flip-Flop Index: quantifying revision stability of forecasts of direction. Journal of Southern Hemisphere Earth Systems Science, 71(3), pp.266-271.](https://doi.org/10.1071/ES21010)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "scoresenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}