{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<style>div.container { width: 100% }</style>\n",
    "<img style=\"float:left;  vertical-align:text-bottom;\" height=\"65\" width=\"172\" src=\"../assets/holoviz-logo-unstacked.svg\" />\n",
    "<div style=\"float:right; vertical-align:text-bottom;\"><h2>Tutorial 9. Advanced Dashboards</h2></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point we have learned how to build interactive apps and dashboards with Panel, how to quickly build visualizations with hvPlot, and add custom interactivity by using HoloViews. In this section we will work on putting all of this together to build complex and efficient data processing pipelines, controlled by Panel widgets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import colorcet as cc\n",
    "import pandas as pd\n",
    "import holoviews as hv\n",
    "import numpy as np\n",
    "import panel as pn\n",
    "import xarray as xr\n",
    "\n",
    "import hvplot.pandas # noqa: API import\n",
    "import hvplot.xarray # noqa: API import\n",
    "\n",
    "pn.extension()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we get started let's once again load the earthquake and population data and define the basic plots, which we will build the dashboard around."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "df = pd.read_parquet('../data/earthquakes-projected.parq')\n",
    "df.time = df.time.astype('datetime64[ns]')\n",
    "df = df.set_index(df.time)\n",
    "\n",
    "most_severe = df[df.mag >= 7]\n",
    "\n",
    "ds = xr.open_dataarray('../data/raster/gpw_v4_population_density_rev11_2010_2pt5_min.nc')\n",
    "cleaned_ds = ds.where(ds.values != ds.nodatavals).sel(band=1)\n",
    "cleaned_ds.name = 'population'\n",
    "\n",
    "mag_cmap = cc.CET_L4[::-1]\n",
    "\n",
    "high_mag_points = most_severe.hvplot.points(\n",
    "    x='longitude', y='latitude', c='mag', hover_cols=['place', 'time'],\n",
    "    cmap=mag_cmap, tools=['tap'], selection_line_color='black')\n",
    "\n",
    "rasterized_pop = cleaned_ds.hvplot.image(\n",
    "    rasterize=True, cmap='kbc', logz=True, clim=(1, np.nan),\n",
    "    height=500, width=833, xaxis=None, yaxis=None).opts(bgcolor='black')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building Pipelines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous sections we built a little function to cache the closest earthquakes since the computation can take a little while. An alternative to this approach is to start building a pipeline in HoloViews to do this very thing. Instead of writing a function that operates directly on the data, we rewrite the function to accept a Dataset and the index. This function again filters the closest earthquakes within the region and returns a new Dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from holoviews.streams import Selection1D\n",
    "\n",
    "def earthquakes_around_point(ds, index, degrees_dist=0.5):\n",
    "    if not index:\n",
    "        return ds.iloc[[]]\n",
    "    row = high_mag_points.data.iloc[index[0]]\n",
    "    half_dist = degrees_dist / 2.0\n",
    "    df = ds.data\n",
    "    nearest = df[((df['latitude'] - row.latitude).abs() < half_dist) \n",
    "                 & ((df['longitude'] - row.longitude).abs() < half_dist)]\n",
    "    return hv.Dataset(nearest)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we declare a HoloViews ``Dataset``, an ``Selection1D`` stream and use the ``apply`` method to apply the function to the dataset. The most important part is that we can now provide the selection stream's index parameter to this apply method. This sets up a pipeline which filters the Dataset based on the current index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = hv.Dataset(df)\n",
    "index_stream = Selection1D(source=high_mag_points, index=[-3])\n",
    "\n",
    "filtered_ds = dataset.apply(earthquakes_around_point, index=index_stream.param.index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The filtered Dataset object itself doesn't actually display anything but it provides an intermediate pipeline stage which will feed the actual visualizations. The next step therefore is to extend this pipeline to build the visualizations from this filtered dataset. For this purpose we define some functions which take the dataset as input and then generate a plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hv.opts.defaults(\n",
    "    hv.opts.Histogram(toolbar=None),\n",
    "    hv.opts.Scatter(toolbar=None)\n",
    ")\n",
    "\n",
    "def histogram(ds):\n",
    "    return ds.data.hvplot.hist(y='mag', bin_range=(0, 10), bins=20, color='red', width=400, height=250)\n",
    "\n",
    "def scatter(ds):\n",
    "    return ds.data.hvplot.scatter('time', 'mag', color='green', width=400, height=250, padding=0.1)\n",
    "\n",
    "\n",
    "# We also redefine the VLine\n",
    "def vline_callback(index):\n",
    "    if not index:\n",
    "        return hv.VLine(0)\n",
    "    row = most_severe.iloc[index[0]]\n",
    "    return hv.VLine(row.time).opts(line_width=1, color='black')\n",
    "\n",
    "temporal_vline = hv.DynamicMap(vline_callback, streams=[index_stream])\n",
    "\n",
    "dynamic_scatter = filtered_ds.apply(scatter)\n",
    "dynamic_histogram = filtered_ds.apply(histogram)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have defined our visualizations using lazily evaluated pipelines we can start visualizing it. This time we will use Panel to lay out the plots:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pn.Column(\n",
    "    rasterized_pop * high_mag_points,\n",
    "    pn.Row(\n",
    "        dynamic_scatter * temporal_vline,\n",
    "        dynamic_histogram))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise\n",
    "\n",
    "Define another function like the ``histogram`` or ``scatter`` function and then ``apply`` it to the ``filtered_ds``. Observe how this too will respond to changes in the selected earthquake.\n",
    "\n",
    "<details><summary><i><u>(Solution)</u></i></summary><br>\n",
    "\n",
    "```python\n",
    "def bivariate(ds):\n",
    "    return ds.data.hvplot.bivariate('mag', 'depth')\n",
    "    \n",
    "filtered_ds.apply(bivariate)\n",
    "```\n",
    "    \n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connecting widgets to the pipeline\n",
    "\n",
    "At this point you may be thinking that we haven't done anything we haven't already seen in the previous sections. However, apart from automatically handling the caching of computations, building visualization pipelines in this way provides one major benefit - we can inject parameters at any stage of the pipeline. These parameters can come from anywhere including from Panel widgets, allowing us to expose control over any aspect of our pipeline. \n",
    "\n",
    "You may have noticed that the ``earthquakes_around_point`` function takes two arguments, the ``index`` of the point **and** the ``degrees_dist``, which defines the size of the region around the selected earthquake we will select points in. Using ``.apply`` we can declare a ``FloatSlider`` widget and then inject its ``value`` parameter into the pipeline (ensure that an earthquake is selected in the map above):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dist_slider = pn.widgets.FloatSlider(name='Degree Distance', value=0.5, start=0.1, end=2)\n",
    "\n",
    "filtered_ds = dataset.apply(earthquakes_around_point, index=index_stream.param.index,\n",
    "                            degrees_dist=dist_slider)\n",
    "\n",
    "pn.Column(\n",
    "    dist_slider,\n",
    "    pn.Row(\n",
    "        filtered_ds.apply(histogram),\n",
    "        filtered_ds.apply(scatter)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the widget value changes the pipeline will re-execute the part of the pipeline downstream from the function and update the plot. This ensures that only the parts of the pipeline that are actually needed are re-executed.\n",
    "\n",
    "The ``.apply`` method can also be used to apply options depending on some widget value, e.g. we can create a colormap selector and then use ``.apply.opts`` to connect it to the ``rasterized_pop`` plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmaps  = {n: cc.palette[n] for n in ['kbc', 'fire', 'bgy', 'bgyw', 'bmy', 'gray', 'kbc']}\n",
    "\n",
    "cmap_selector = pn.widgets.Select(name='Colormap', options=cmaps)\n",
    "\n",
    "rasterized_pop_cmapped = rasterized_pop.apply.opts(cmap=cmap_selector)\n",
    "\n",
    "pn.Column(cmap_selector, rasterized_pop_cmapped)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise\n",
    "\n",
    "Use the ``.apply.opts`` method to control the style of some existing component, e.g. the ``size`` of the points in the ``dynamic_scatter`` plot or the ``color`` of the ``dynamic_histogram``.\n",
    "\n",
    "<details><summary><i><u>(Hint)</u></i></summary><br>\n",
    "\n",
    "Use a ``ColorPicker`` widget to control the ``color`` or a ``FloatSlider`` widget to control the ``size``. \n",
    "    \n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary><i><u>(Solution)</u></i></summary><br>\n",
    "\n",
    "```python\n",
    "color_picker = pn.widgets.ColorPicker(name='Color', value='#00f300')\n",
    "size_slider = pn.widgets.FloatSlider(name='Size', value=5, start=1, end=30)\n",
    "    \n",
    "color_histogram = dynamic_histogram.apply.opts(color=color_picker.param.value)\n",
    "size_scatter = dynamic_scatter.apply.opts(size=size_slider.param.value)\n",
    "    \n",
    "pn.Column(\n",
    "    pn.Row(color_picker, size_slider),\n",
    "    pn.Row(color_histogram, size_scatter)\n",
    ")\n",
    "```\n",
    "    \n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connecting panels to streams\n",
    "\n",
    "At this point we have learned how to connect parameters on Panel objects to a pipeline and we earlier learned how we can use parameters to declare dynamic Panel components.  So, this section should be nothing new;, we will simply try to connect the index parameter of the selection stream to a panel to try to compute the number of people in the region around an earthquake.\n",
    "\n",
    "Since we have a population density dataset we can approximate how many people are affected by a particular earthquake. Of course, this value is only a rough approximation, as it ignores the curvature of the earth, assumes isotropic spreading of the earthquake, and assumes that the population did not change between the measurement and the earthquake. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def affected_population(index, distance):\n",
    "    if not index:\n",
    "        return \"No earthquake was selected.\"\n",
    "    sel = most_severe.iloc[index[0]]\n",
    "    lon, lat = sel.longitude, sel.latitude\n",
    "    lon_dist = (np.cos(np.deg2rad(lat)) * 111.321543) * distance\n",
    "    lat_dist = 111.321543 * distance\n",
    "    hdist = distance / 2.\n",
    "    mean_density = cleaned_ds.sel(x=slice(lon-hdist, lon+hdist), y=slice(lat+hdist, lat-hdist)).mean().item()\n",
    "    population = (lat_dist * lon_dist) * mean_density\n",
    "    return 'Approximate population around {place}, where a magnitude {mag} earthquake hit on {date} is {pop:.0f}.'.format(\n",
    "        pop=population, mag=sel.mag, place=sel.place, date=sel.time)\n",
    "\n",
    "def bounds(index, value):\n",
    "    if not index:\n",
    "        return hv.Bounds((0, 0, 0, 0))\n",
    "    sel = most_severe.iloc[index[0]]\n",
    "    hdist = value / 2.\n",
    "    lon, lat = sel.longitude, sel.latitude \n",
    "    return hv.Bounds((lon-hdist, lat-hdist, lon+hdist, lat+hdist))  \n",
    "\n",
    "dynamic_bounds = hv.DynamicMap(bounds, streams=[index_stream, dist_slider.param.value])\n",
    "bound_affected_population = pn.bind(affected_population, index=index_stream.param.index, distance=dist_slider)\n",
    "pn.Column(pn.panel(bound_affected_population, width=400), \n",
    "          rasterized_pop * high_mag_points * dynamic_bounds, dist_slider)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The full dashboard\n",
    "\n",
    "Finally let us put all these components together into an overall dashboard, which we will mark as ``servable`` so we can ``panel serve`` this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "title = '## Major Earthquakes 2000-2018'\n",
    "logo = pn.panel('../assets/usgs_logo.png', width=200, align='center')\n",
    "widgets = pn.WidgetBox(dist_slider, cmap_selector, margin=5)\n",
    "\n",
    "header = pn.Row(pn.Column(title, pn.panel(bound_affected_population, width=400)),\n",
    "                pn.layout.Spacer(width=10), logo, pn.layout.HSpacer(), widgets)\n",
    "\n",
    "dynamic_scatter = filtered_ds.apply(scatter)\n",
    "dynamic_histogram = filtered_ds.apply(histogram)\n",
    "temporal_vline = hv.DynamicMap(vline_callback, streams=[index_stream])\n",
    "rasterized_pop_cmapped = rasterized_pop.apply.opts(cmap=cmap_selector.param.value)\n",
    "dynamic_bounds = hv.DynamicMap(bounds, streams=[index_stream, dist_slider.param.value])\n",
    "\n",
    "body = pn.Row(\n",
    "    rasterized_pop_cmapped * high_mag_points * dynamic_bounds,\n",
    "    pn.Column(dynamic_scatter * temporal_vline, dynamic_histogram),\n",
    ")\n",
    "\n",
    "pn.Column(header, body).servable()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "If you have gone through all the tutorials and exercises, you should now have a very good idea of the power of the HoloViz ecosystem, and how each of the tools fit together. You can see many examples of HoloViz apps at [examples.pyviz.org](https://examples.pyviz.org), though do note that each of them was written at a particular stage of HoloViz development and may not be using the best-practice recommendations as outlined in these tutorials. Have fun working with the tools, and feel free to chime in at our [Discourse](https://discourse.holoviz.org) site if you have usage questions that others in the community can answer!"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}