{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we discovered in the [Introduction](Introduction.ipynb), HoloViews allows plotting a variety of data types. Here we will use the sample data module and load the pandas and dask hvPlot API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import hvplot.pandas  # noqa\n",
    "import hvplot.dask  # noqa"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we learned the hvPlot API closely mirrors the [Pandas plotting API](https://pandas.pydata.org/pandas-docs/stable/visualization.html), but instead of generating static images when used in a notebook, it uses HoloViews to generate either static or dynamically streaming Bokeh plots. Static plots can be used in any context, while streaming plots require a live [Jupyter notebook](http://jupyter.org), a deployed [Bokeh Server app](https://bokeh.pydata.org/en/latest/docs/user_guide/server.html), or a deployed [Panel](https://panel.pyviz.org) app.\n",
    "\n",
    "HoloViews provides an extensive, very rich set of objects along with a powerful set of operations to apply, as you can find out in the [HoloViews User Guide](http://holoviews.org/user_guide/index.html). But here we will focus on the most essential mechanisms needed to make your data visualizable, without having to worry about the mechanics going on behind the scenes.\n",
    "\n",
    "We will be focusing on two different datasets:\n",
    "\n",
    "- A small CSV file of US crime data, broken down by state\n",
    "- A larger Parquet-format file of airline flight data\n",
    "\n",
    "The ``hvplot.sample_data`` module makes these datasets Intake data catalogue, which we can load either using pandas:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from hvplot.sample_data import us_crime, airline_flights\n",
    "\n",
    "crime = us_crime.read()\n",
    "print(type(crime))\n",
    "crime.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or using dask as a ``dask.DataFrame``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights = airline_flights.to_dask().persist()\n",
    "print(type(flights))\n",
    "flights.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The plot interface"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ``dask.dataframe.DataFrame.hvplot``, ``pandas.DataFrame.hvplot`` and ``intake.DataSource.plot`` interfaces (and Series equivalents) from HvPlot provide a powerful high-level API to generate complex plots. The ``.hvplot`` API can be called directly or used as a namespace to generate specific plot types."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The plot method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The most explicit way to use the plotting API is to specify the names of columns to plot on the ``x``- and ``y``-axis respectively:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.line(x='Year', y='Violent Crime rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you'll see in more detail below, you can choose which kind of plot you want to use for the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot(x='Year', y='Violent Crime rate', kind='scatter')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To group the data by one or more additional columns, specify an additional ``by`` variable. As an example here we will plot the departure delay ('depdelay') as a function of 'distance', grouping the data by the 'carrier'. There are many available carriers, so we will select only two of them so that the plot is readable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flight_subset = flights[flights.carrier.isin(['OH', 'F9'])]\n",
    "flight_subset.hvplot(x='distance', y='depdelay', by='carrier', kind='scatter', alpha=0.2, persist=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have specified the `x` axis explicitly, which can be omitted if the Pandas index column is already the desired x axis. Similarly, here we specified the `y` axis; by default all of the non-index columns would be plotted (which would be a lot of data in this case). If you don't specify the 'y' axis, it will have a default label named 'value', but you can then provide a y axis label explicitly using the ``value_label`` option.\n",
    "\n",
    "Putting all of this together we will plot violent crime, robbery, and burglary rates on the y-axis, specifying 'Year' as the x, and relabel the y-axis to display the 'Rate'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot(x='Year', y=['Violent Crime rate', 'Robbery rate', 'Burglary rate'],\n",
    "             value_label='Rate (per 100k people)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The hvplot namespace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead of using the ``kind`` argument to the plot call, we can use the ``hvplot`` namespace, which lets us easily discover the range of plot types that are supported. Use tab completion to explore the available plot types:\n",
    "\n",
    "```python\n",
    "crime.hvplot.<TAB>\n",
    "```\n",
    "\n",
    "Plot types available include:\n",
    "\n",
    "* <a href=\"#area\">``.area()``</a>: Plots a  area chart similar to a line chart except for filling the area under the curve and optionally stacking \n",
    "* <a href=\"#bars\">``.bar()``</a>: Plots a bar chart that can be stacked or grouped\n",
    "* <a href=\"#bivariate\">``.bivariate()``</a>: Plots 2D density of a set of points \n",
    "* <a href=\"#box-whisker-plots\">``.box()``</a>: Plots a box-whisker chart comparing the distribution of one or more variables\n",
    "* <a href=\"#heatmap\">``.heatmap()``</a>: Plots a heatmap to visualizing a variable across two independent dimensions\n",
    "* <a href=\"#hexbins\">``.hexbins()``</a>: Plots hex bins\n",
    "* <a href=\"#histogram\">``.hist()``</a>: Plots the distribution of one or histograms as a set of bins\n",
    "* <a href=\"#kde-density\">``.kde()``</a>: Plots the kernel density estimate of one or more variables.\n",
    "* <a href=\"#the-plot-method\">``.line()``</a>: Plots a line chart (such as for a time series)\n",
    "* <a href=\"#scatter\">``.scatter()``</a>: Plots a scatter chart comparing two variables\n",
    "* <a href=\"#step\">``.step()``</a>: Plots a step chart akin to a line plot\n",
    "* <a href=\"#tables\">``.table()``</a>: Generates a SlickGrid DataTable\n",
    "* <a href=\"#groupby\">``.violin()``</a>: Plots a violin plot comparing the distribution of one or more variables using the kernel density estimate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Area\n",
    "\n",
    "Like most other plot types the ``area`` chart supports the three ways of defining a plot outlined above. An area chart is most useful when plotting multiple variables in a stacked chart. This can be achieve by specifying ``x``, ``y``, and ``by`` columns or using the ``columns`` and ``index``/``use_index`` (equivalent to ``x``) options:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.area(x='Year', y=['Robbery', 'Aggravated assault'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also explicitly set ``stacked`` to False and define an ``alpha`` value to compare the values directly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.area(x='Year', y=['Aggravated assault', 'Robbery'], stacked=False, alpha=0.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another use for an area plot is to visualize the spread of a value. For instance using the flights dataset we may want to see the spread in mean delay values across carriers. For that purpose we compute the mean delay by day and carrier and then the min/max mean delay for across all carriers. Since the output of ``hvplot`` is just a regular holoviews object, we can use the overlay operator (\\*) to place the plots on top of each other."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "delay_min_max = flights.groupby(['day', 'carrier'])['carrier_delay'].mean().groupby('day').agg([np.min, np.max])\n",
    "delay_mean = flights.groupby('day')['carrier_delay'].mean()\n",
    "\n",
    "delay_min_max.hvplot.area(x='day', y='amin', y2='amax', alpha=0.2) * delay_mean.hvplot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Bars\n",
    "\n",
    "In the simplest case we can use ``.hvplot.bar`` to plot ``x`` against ``y``. We'll use ``rot=90`` to rotate the tick labels on the x-axis making the years easier to read:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.bar(x='Year', y='Violent Crime rate', rot=90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to compare multiple columns instead we can set ``y`` to a list of columns. Using the ``stacked`` option we can then compare the column values more easily:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.bar(x='Year', y=['Violent crime total', 'Property crime total'],\n",
    "                 stacked=True, rot=90, width=800, legend='top_left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Scatter\n",
    "\n",
    "The scatter plot supports many of the same features as the other chart types we have seen so far but can also be colored by another variable using the ``c`` option. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.scatter(x='Violent Crime rate', y='Burglary rate', c='Year')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Anytime that color is being used to represent a dimension, the ``cmap`` option can be used to control the colormap that is used to represent that dimension. Additionally, the colorbar can be disabled using ``colorbar=False``."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step\n",
    "\n",
    "A step chart is very similar to a line chart but instead of linearly interpolating between samples the step chart visualizes discrete steps. The point at which to step can be controlled via the ``where`` keyword allowing `'pre'`, `'mid'` (default) and `'post'` values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.step(x='Year', y=['Robbery', 'Aggravated assault'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### HexBins\n",
    "\n",
    "You can create hexagonal bin plots with the ``hexbin`` method. Hexbin plots can be a useful alternative to scatter plots if your data are too dense to plot each point individually. Since these data are not regularly distributed, we'll use the ``logz`` option to map z-axis (color) to a log scale colorbar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights.hvplot.hexbin(x='airtime', y='arrdelay', width=600, height=500, logz=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Bivariate\n",
    "\n",
    "You can create a 2D density plot with the ``bivariate`` method. Bivariate plots can be a useful alternative to scatter plots if your data are too dense to plot each point individually."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.bivariate(x='Violent Crime rate', y='Burglary rate', width=600, height=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### HeatMap\n",
    "\n",
    "A ``HeatMap`` lets us view the relationship between three variables, so we specify the 'x' and 'y' variables and an additional 'C' variable. Additionally we can define a ``reduce_function`` that computes the values for each bin from the samples that fall into it. Here we plot the 'depdelay' (i.e. departure delay) for each day of the month and carrier in the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights.compute().hvplot.heatmap(x='day', y='carrier', C='depdelay', reduce_function=np.mean, colorbar=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tables\n",
    "\n",
    "Unlike all other plot types, a table only supports one signature: either all columns are plotted, or a subset of columns can be selected by defining the ``columns`` explicitly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.table(columns=['Year', 'Population', 'Violent Crime rate'], width=400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Distributions\n",
    "\n",
    "Plotting distributions differs slightly from other plots since they plot only one variable in the simple case rather than plotting two or more variables against each other. Therefore when plotting these plot types no ``index`` or ``x`` value needs to be supplied. Instead:\n",
    "\n",
    "1. Declare a single ``y`` variable, e.g. ``source.plot.hist(variable)``, or\n",
    "2. Declare a ``y`` variable and ``by`` variable, e.g. ``source.plot.hist(variable, by='Group')``, or\n",
    "3. Declare columns or plot all columns, e.g. ``source.plot.hist()`` or ``source.plot.hist(columns=['A', 'B', 'C'])``\n",
    "\n",
    "#### Histogram\n",
    "\n",
    "The Histogram is the simplest example of a distribution; often we simply plot the distribution of a single variable, in this case the 'Violent Crime rate'. Additionally we can define a range over which to compute the histogram and the number of bins using the ``bin_range`` and ``bins`` arguments respectively:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.hist(y='Violent Crime rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we can plot the distribution of multiple columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns = ['Violent Crime rate', 'Property crime rate', 'Burglary rate']\n",
    "crime.hvplot.hist(y=columns, bins=50, alpha=0.5, legend='top', height=400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also group the data by another variable. Here we'll use ``subplots`` to split each carrier out into its own plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flight_subset = flights[flights.carrier.isin(['AA', 'US', 'OH'])]\n",
    "flight_subset.hvplot.hist('depdelay', by='carrier', bins=20, bin_range=(-20, 100), width=300, subplots=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### KDE (density)\n",
    "\n",
    "You can also create density plots using ``hvplot.kde()`` or ``hvplot.density()``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.kde(y='Violent Crime rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Comparing the distribution of multiple columns is also possible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns=['Violent Crime rate', 'Property crime rate', 'Burglary rate']\n",
    "crime.hvplot.kde(y=columns, alpha=0.5, value_label='Rate', legend='top_right')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ``hvplot.kde`` also supports the ``by`` keyword:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flight_subset = flights[flights.carrier.isin(['AA', 'US', 'OH'])]\n",
    "flight_subset.hvplot.kde('depdelay', by='carrier', xlim=(-20, 70), width=300, subplots=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Box-Whisker Plots\n",
    "\n",
    "Just like the other distribution-based plot types, the box-whisker plot supports plotting a single column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot.box(y='Violent Crime rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It also supports multiple columns and the same options as seen previously (``legend``, ``invert``, ``value_label``):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns=['Burglary rate', 'Larceny-theft rate', 'Motor vehicle theft rate',\n",
    "         'Property crime rate', 'Violent Crime rate']\n",
    "crime.hvplot.box(y=columns, group_label='Crime', legend=False, value_label='Rate (per 100k)', invert=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, it also supports using the ``by`` keyword to split the data into multiple subsets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flight_subset = flights[flights.carrier.isin(['AA', 'US', 'OH'])]\n",
    "flight_subset.hvplot.box('depdelay', by='carrier', ylim=(-10, 70))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Composing Plots\n",
    "\n",
    "One of the core strengths of HoloViews is the ease of composing\n",
    "different plots. Individual plots can be composed using the ``*`` and\n",
    "``+`` operators, which overlay and compose plots into layouts\n",
    "respectively. For more information on composing objects, see the\n",
    "HoloViews [User Guide](http://holoviews.org/user_guide/Composing_Elements.html).\n",
    "\n",
    "By using these operators we can combine multiple plots into composite plots. A simple example is overlaying two plot types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crime.hvplot(x='Year', y='Violent Crime rate') * crime.hvplot.scatter(x='Year', y='Violent Crime rate', c='k')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also lay out different plots and tables together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(crime.hvplot.bar(x='Year', y='Violent Crime rate', rot=90, width=550) +\n",
    " crime.hvplot.table(['Year', 'Population', 'Violent Crime rate'], width=420))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Large data\n",
    "\n",
    "The previous examples summarized the fairly large airline dataset using statistical plot types that aggregate the data into a feasible subset for plotting.  We can instead aggregate the data directly into the viewable image using [datashader](http://datashader.org), which provides a rendering of the entire set of raw data available (as far as the resolution of the screen allows). Here we plot the 'airtime' against the 'distance':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights.hvplot.scatter(x='distance', y='airtime', datashade=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Groupby\n",
    "\n",
    "Thanks to the ability of HoloViews to explore a parameter space with a set of widgets we can apply a groupby along a particular column or dimension. For example we can view the distribution of departure delays by carrier grouped by day, allowing the user to choose which day to display:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flights.hvplot.violin(y='depdelay', by='carrier', groupby='dayofweek', ylim=(-20, 60), height=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This user guide merely provided an overview over the available plot types; to see a detailed description on how to customize plots see the [Customization](Customization.ipynb) user guide."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}