{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Live Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 'Getting Started' guide has up until this point demonstrated how HoloViews objects can wrap your data and be given a rich, useful representation. All of the visualizations assumed that the data was already available in memory so that it could be used to construct the appropriate object, and all of the resulting visualizations can be viewed in static HTML pages, no longer requiring Python when users interact with them.\n",
    "\n",
    "In many important scenarios, the assumption that the data is immediately available in memory does not hold. The data of interest may exist on some remote server, making it unavailable locally until it is fetched. In other situations, the data may exist on the local disk, but be too large to fit into memory. Perhaps the data doesn't even exist yet: it may be the result of some computation yet to be performed or the outcome of some live process with the corresponding measurement not yet made.\n",
    "\n",
    "All these scenarios are examples of *live data* that can be made available to HoloViews using the appropriate Python process. In this section, we will see how HoloViews allows you to build visualizations that can update dynamically to newly available data and that can respond to live user interaction.\n",
    "\n",
    "<p><center><div class=\"alert alert-info\" role=\"alert\"><b>Note: </b>To work with live data, you need a live Python server, not a static web site, which is why the outputs shown below are GIF animations. If you run this notebook yourself, you will be able to try out your own interactions and compare them to the displayed GIF animations.</div></center></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A computational process\n",
    "\n",
    "Let us start by importing NumPy and HoloViews and loading the Bokeh extension:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import holoviews as hv\n",
    "from holoviews import opts\n",
    "import numpy as np\n",
    "hv.extension('bokeh')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many possible examples of live data, including financial data feeds, real-time scientific measurements, and sophisticated numerical simulations. Here we will consider the path traced by two very simple equations:\n",
    "\n",
    "$$x_{n+1} = \\sin(ay_n) + c \\cos(ax_n)$$\n",
    "$$y_{n+1} = \\sin(bx_n) + d \\cos(by_n)$$\n",
    "\n",
    "These equations define the 'Clifford Attractor' described in the  book \"Chaos In Wonderland\" by [Clifford Pickover](https://en.wikipedia.org/wiki/Clifford_A._Pickover). Now let's write a simple Python function to iterate these two equations starting from position ``(x0,y0)``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clifford_equation(a,b,c,d,x0,y0):\n",
    "    xn,yn = x0,y0\n",
    "    coords = [(x0,y0)]\n",
    "    for i in range(10000):\n",
    "        x_n1 = np.sin(a*yn) + c*np.cos(a*xn)\n",
    "        y_n1 = np.sin(b*xn) + d*np.cos(b*yn)\n",
    "        xn,yn = x_n1,y_n1\n",
    "        coords.append((xn,yn))\n",
    "    return coords"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we run this function now, we'll get a list of 10000 tuples, which won't be very informative. To visualize this data, we can set one suitable visual defaults for the ``Curve`` and `Points` elements we will be using in the rest of the notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "opts.defaults(\n",
    "    opts.Curve(color='black'),\n",
    "    opts.Points(color='red', alpha=0.1, width=400, height=400))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now pass the output of our ``clifford`` function to the ``Points`` element, making it trivial to define a function that when called gives us a visualization:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def clifford_attractor(a,b,c,d):\n",
    "    return hv.Points(clifford_equation(a,b,c,d,x0=0,y0=0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then view the output for some combination of values for ``a,b,c,d``, starting from the origin:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clifford_attractor(a =-1.5, b=1.5, c=1, d=0.75 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This HoloViews element gives us a snapshot for the four chosen values, but what we really would like to do is to interact with the four-dimensional parameter space directly, even though that parameter space is too large to compute all possible combinations feasibly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Live parameter exploration\n",
    "\n",
    "To dynamically explore these parameters, we can start by declaring a ``DynamicMap``, passing in our function instead of the dictionary of ``Image`` elements we saw in the [Introduction](1-Introduction.ipynb). We declare the four arguments of our function as ``kdims``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clifford = hv.DynamicMap(clifford_attractor, kdims=['a','b','c','d'])\n",
    "clifford"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see from the error message, HoloViews does not yet have the information needed to give us a visualization--it has no way to guess any value to use for the 'a','b','c', and 'd' dimensions.  Since we know what suitable values look like, we can easily specify appropriate ranges using the ``redim`` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# When run live, this cell's output should match the behavior of the GIF below\n",
    "clifford.redim.range(a=(-1.5,-1),b=(1.5,2),c=(1,1.2),d=(0.75,0.8), x=(-2,2), y=(-2,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='https://assets.holoviews.org/gifs/guides/getting_started/5-Live_Data/live_data_1.gif'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These ranges supplied with ``redim.range`` are semantic specifications of suitable values for each of the parameters and they are used to define suitable ranges for the interactive sliders above. Note how the HoloViews options system described in the [Customization section](2-Customization.ipynb) continues to work with the ``DynamicMap``."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Live interaction\n",
    "\n",
    "The live visualizations above are indistinguishable from standard HoloViews visualization, apart from the speed and memory usage. With a live Python server and the Bokeh backend, HoloViews can also be used to build highly customized interactive displays using ``DynamicMap`` and the *streams system*. A HoloViews stream is simply a parameter of a corresponding stream class configured to track some variable, typically reflecting some user interaction. For instance, let's write a function that accepts an initial ``x`` and ``y`` value and computes a more complex version of the above plot, showing the ``x``,``y`` starting point as a dot along with a line segment indicating the first step taken from that point when computing the attractor, and some text showing the starting point numerically:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def interactive_clifford(a,b,c,d,x=0,y=0):\n",
    "    coords = clifford_equation(a,b,c,d,x0=x,y0=y)\n",
    "    \n",
    "    points = hv.Points(coords).opts(color='green')\n",
    "    start  = hv.Points(coords[0]).opts(color='black', size=10, alpha=1)\n",
    "    step   = hv.Curve(coords[:2], group='Init')\n",
    "    text   = hv.Text(0,1.75, 'x:{x:.2f} y:{y:.2f}'.format(x=coords[0][0],y=coords[0][1]))\n",
    "    \n",
    "    return points * start * step * text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All we have done is create an ``Overlay`` as described in the [Introduction](1-Introduction.ipynb) containing our Clifford attractor and a few other HoloViews elements parameterized accordingly, including ``Points`` and the ``Text`` annotation. Passing this function to ``DynamicMap`` together with a `PointerXY` stream that grabs the x,y locations of the mouse (in data space) creates an explorable visualization that you can interact with directly.  The plot now shows the attractor (in blue) and the starting point and first step (in red), with the starting point following the mouse position (slowly, as it has to draw 10,000 points each time the pointer moves!):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from holoviews.streams import PointerXY\n",
    "\n",
    "# When run live, this cell's output should match the behavior of the GIF below\n",
    "iclifford = hv.DynamicMap(interactive_clifford, kdims=['a','b','c','d'], streams=[PointerXY(x=0,y=0)])\n",
    "iclifford.redim.range(a=(-1.4,-1),b=(1.6,1.8),c=(1,1.5),d=(0.7,0.8), x=(-2,2), y=(-2,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='https://assets.holoviews.org/gifs/guides/getting_started/5-Live_Data/live_data_2.gif'></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By exploring with the mouse, see if you can find the fixed-point location (where the next step maps you to the same position) located at ``x=0.18,y=0.65`` with parameters ``a=-1.4, b=1.6, c=1`` and ``d=0.7``.\n",
    "\n",
    "To learn more about the streams system please consult the [user guide](../user_guide/07-Live_Data.ipynb) and check out our [Streams gallery](../reference/index.html#streams).  And to explore attractors like the above in much more detail, check out the Datashader-based [notebook](https://anaconda.org/jbednar/clifford_attractor) (which uses a HoloViews DynamicMap) and the corresponding [app](https://anaconda.org/jbednar/datashaderattractors)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tradeoffs using live data\n",
    "\n",
    "``DynamicMap`` and ``Streams`` allow specification of exciting, dynamic visualizations that let you build full-featured interactive applications and simulations with very little code (particularly when combined with a declarative widget library like [Panel](https://panel.pyviz.org)). The way these dynamic visualizations work is that HoloViews runs JavaScript in your browser, which then communicates with a running Python server process that may be running in the Jupyter notebook server or in the [Bokeh server](http://bokeh.pydata.org/en/latest/docs/user_guide/server.html). This Python process may be running locally on your machine or on some remote internet or local-network server. Regardless of where it is running, this Python process executes the callback you supply to ``DynamicMap``, allowing HoloViews to update your visualization whenever the parameters change.\n",
    "\n",
    "This architecture is powerful and fully general, as you can always make static content in memory into dynamic output generated by a function (see the [User Guide](../user_guide/07-Live_Data.ipynb) to learn more). These dynamic visualizations can then be deployed as [dashboards](../user_guide/17-Dashboards.ipynb) or deployed as [Bokeh Apps](../user_guide/Deploying_Bokeh_Apps.ipynb).\n",
    "\n",
    "Using live data is not always recommended, however, because using purely static content also has some important advantages:\n",
    "\n",
    "### Reasons to use live data\n",
    "\n",
    "* Your data is inherently coming from a live source and your visualization needs to reflect this in real time.\n",
    "* You wish to explore a large parameter space and statically sampling this space adequately is prohibitive in memory or computation time.\n",
    "* Your data is too big to fit in memory and you only need to explore a portion of it that you can stream in from disk.\n",
    "* You want an open-ended visualization that keeps updating indefinitely.\n",
    "\n",
    "### Reasons to use static data\n",
    "\n",
    "* You wish to archive or record your visualization in such a way that it exists independently of code execution in a potentially changing codebase.\n",
    "* You wish to share visualizations in a static HTML file that does not require running a live server (e.g a file that can be e-mailed and immediately viewed or placed on an HTML server).\n",
    "\n",
    "The general recommendation is to visualize your data with ``HoloMap`` (as in the introduction to this guide) when you have a small amount of data (typically a few megabytes) that can be quickly computed and can reasonably be embedded into an HTML file. Otherwise, you can use ``DynamicMap`` that you can sample from to generate a ``HoloMap`` from when you wish to share your results (see the [user guide](../user_guide/07-Live_Data.ipynb) for more information on how to turn your ``DynamicMap`` objects into ``HoloMap``s).\n",
    "\n",
    "Now that you have explored the basic capabilities of HoloViews, you should try it out on your own data, guided by the [user guide](../user_guide/) and following examples in the component [reference gallery](../reference/) and other demos in the [gallery](../gallery/)."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}