{ "cells": [ { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "# Low Level Widget Tutorial" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## How do they fit into the picture?" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "One of the goals of the Jupyter Notebook is to minimize the “distance” the user is from their data. This means allowing the user to quickly view and manipulate the data. " ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "| ![](images/inputoutput.PNG) | ![](images/widgets.PNG) |\n", "|-------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------|\n", "| Before the widgets, this was just the segmentation of code and results from executing those segments. | Widgets further decrease the distance between the user and their data by allowing UI interactions to directly manipulate data in the kernel. |" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## How?" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Jupyter interactive widgets are interactive elements, think sliders, textboxes, buttons, that have representations both in the kernel (place where code is executed) and the front-end (the Notebook web interface). To do this, a clean, well abstracted communication layer must exist." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Comms" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "This is where Jupyter notebook “comms” come into play. The comm API is a symmetric, asynchronous, fire and forget style messaging API. It allows the programmer to send JSON-able blobs between the front-end and the back-end. The comm API hides the complexity of the webserver, ZMQ, and websockets." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/transport.svg)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Synchronized state" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Using comms, the widget base layer is designed to keep state in sync. In the kernel, a Widget instance exists. This Widget instance has a corresponding WidgetModel instance in the front-end. The Widget and WidgetModel store the same state. The widget framework ensures both models are kept in sync with eachother. If the WidgetModel is changed in the front-end, the Widget receives the same change in the kernel. Vise versa, if the Widget in the kernel is changed, the WidgetModel in the front-end receives the same change. There is no single source of truth, both models have the same precedence. Although a notebook has the notion of cells, neither Widget or WidgetModel are bound to any single cell." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/state_sync.svg)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Models and Views" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "In order for the user to interact with widgets on a cell by cell basis, the WidgetModels are represented by WidgetViews. Any single WidgetView is bound to a single cell. Multiple WidgetViews can be linked to a single WidgetModel. This is how you can redisplay the same Widget multiple times and it still works. To accomplish this, the widget framework uses Backbone.js. In a traditional MVC framework, the WidgetModel is the (M)odel, and the WidgetView is both the (V)iew and (C)ontroller. Meaning that, the views both display the state of the model and manipulate it. Think about a slider control, it both displays the value and allows the user to change the value by dragging the slide handle." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "deletable": true, "editable": true, "scrolled": true }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9863258038944c80b155e81633469a84" } }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9863258038944c80b155e81633469a84" } }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from ipywidgets import *\n", "from IPython.display import display\n", "w = IntSlider()\n", "display(w, w)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false, "deletable": true, "editable": true, "scrolled": true }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9863258038944c80b155e81633469a84" } }, "metadata": {}, "output_type": "display_data" } ], "source": [ "display(w)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/assoc.svg)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Code execution" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "The user code required to display a simple FloatSlider widget is:\n", "\n", "```python\n", "from ipywidgets import FloatSlider\n", "from IPython.display import display\n", "slider = FloatSlider()\n", "display(slider)\n", "```\n", "\n", "In order to understand how a widget is displayed, one must understand how code is executed in the Notebook. Execution begins in the code cell. A user event triggers the code cell to send an evaluate code message to the kernel, containing all of the code in the code cell. This message is given a GUID, which the front-end associates to the code cell, and remembers it (**important**)." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/execute.svg)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "subslide" } }, "source": [ "Once that message is received by the kernel, the kernel immediately sends the front-end an “I’m busy” status message. The kernel then proceeds to execute the code." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/busy.svg)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Model construction" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "When a Widget is constructed in the kernel, the first thing that happens is that a comm is constructed and associated with the widget. When the comm is constructed, it is given a GUID (globally unique identifier). A comm-open message is sent to the front-end, with metadata stating that the comm is a widget comm and what the corresponding WidgetModel class is. " ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/widgetcomm.svg)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "subslide" } }, "source": [ "The WidgetModel class is specified my module and name. Require.js is then used to asynchronously load the WidgetModel class. The message triggers a comm to be created in the front-end with same GUID as the back-end. Then, the new comm gets passed into the WidgetManager in the front-end, which creates an instance of the WidgetModel class, linked to the comm. Both the Widget and WidgetModel repurpose the comm GUID as their own." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/widgetcomm2.svg)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "subslide" } }, "source": [ "Asynchronously, the kernel sends an initial state push, containing all of the initial state of the Widget, to the front-end, immediately after the comm-open message. This state message may or may not be received by the time the WidgetModel is constructed. Regardless, the message is cached and gets processed once the WidgetModel has been constructed. The initial state push is what causes the WidgetModel in the front-end to become in sync with the Widget in the kernel." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/state.svg)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Displaying a view" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "After the Widget has been constructed, it can be displayed. Calling `display(widgetinstance)` causes a specially named repr method in the widget to run. This method sends a message to the front-end that tells the front-end to construct and display a widget view. The message is in response to the original code execution message, and the original message’s GUID is stored in the new message’s header. When the front-end receives the message, it uses the original messsage’s GUID to determine what cell the new view should belong to. Then, the view is created, using the WidgetView class specified in the WidgetModel’s state. The same require.js method is used to load the view class. Once the class is loaded, an instance of it is constructed, displayed in the right cell, and registers listeners for changes of the model." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![](images/display.svg)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Widget skeleton" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false, "deletable": true, "editable": true, "scrolled": true }, "outputs": [ { "data": { "application/javascript": [ "this.model.get('count');\n", "this.model.set('count', 999);\n", "this.touch();\n", "\n", "/////////////////////////////////\n", "\n", "this.colorpicker = document.createElement('input');\n", "this.colorpicker.setAttribute('type', 'color');\n", "this.el.appendChild(this.colorpicker);" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%%javascript\n", "this.model.get('count');\n", "this.model.set('count', 999);\n", "this.touch();\n", "\n", "/////////////////////////////////\n", "\n", "this.colorpicker = document.createElement('input');\n", "this.colorpicker.setAttribute('type', 'color');\n", "this.el.appendChild(this.colorpicker);" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Since widgets exist in both the front-end and kernel, they consist of both Python (if the kernel is IPython) and Javascript code. A boilerplate widget can be seen below:\n", "\n", "Python:\n", "```python\n", "from ipywidgets import DOMWidget\n", "from traitlets import Unicode, Int\n", " \n", "class MyWidget(DOMWidget):\n", "\t_view_module = Unicode('mywidget').tag(sync=True)\n", "\t_view_name = Unicode('MyWidgetView').tag(sync=True)\n", "\tcount = Int().tag(sync=True)\n", "```\n", "\n", "JavaScript:\n", "```js\n", "define(['jupyter-js-widgets'], function(widgets) {\n", "\tvar MyWidgetView = widgets.DOMWidgetView.extend({\n", " \trender: function() {\n", " \tMyWidgetView.__super__.render.apply(this, arguments);\n", " \tthis._count_changed();\n", " \tthis.listenTo(this.model, 'change:count', this._count_changed, this);\n", " \t},\n", " \n", " \t_count_changed: function() {\n", " \tvar old_value = this.model.previous('count');\n", " \tvar new_value = this.model.get('count');\n", " \tthis.el.textContent = String(old_value) + ' -> ' + String(new_value);\n", " \t}\n", "\t});\n", " \n", "\treturn {\n", " \tMyWidgetView: MyWidgetView\n", "\t}\n", "});\n", "```\n", "\n", "Describing the Python: \n", "\n", "The base widget classes are `DOMWidget` and `Widget`.\n", "\n", "`_view_module` and `_view_name` are how the front-end knows what view class to construct for the model.\n", "\n", "`sync=True` is what makes the traitlets behave like state.\n", "\n", "A similarly named `_model_module` and `_model_name` can be used to specify the corresponding WidgetModel.\n", "\n", "`count` is an example of a custom piece of state.\n", "\n", "Describing the JavaScript: \n", "\n", "The `define` call asynchronously loads the specified dependencies, and then passes them in as arguments into the callback. Here, the only dependency is the base widget module are loaded.\n", "\n", "Custom views inherit from either `DOMWidgetView` or `WidgetView`.\n", "\n", "Likewise, custom models inherit from `WidgetModel`.\n", "\n", "The `render` method is what is called to render the view’s contents. If the view is a `DOMWidgetView`, the `.el` attribute contains the DOM element that will be displayed on the page.\n", "\n", "`.listenTo` allows the view to listen to properties of the model for changes.\n", "\n", "`_count_changed` is an example of a method that could be used to handle model changes.\n", "\n", "`this.model` is how the corresponding model can be accessed.\n", "\n", "`this.model.previous` will get the previous value of the trait.\n", "\n", "`this.model.get` will get the current value of the trait.\n", "\n", "`this.model.set` followed by `this.save_changes();` changes the model. The view method `save_changes` is needed to associate the changes with the current view, thus associating any response messages with the view’s cell.\n", "\n", "The dictionary returned is the public members of the module.\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Serialization of widget attributes\n", "\n", "Widget trait attributes tagged with `sync=True` are synchronized with the JavaScript model instance on the JavaScript side. For this reason, they need to be serialized into `json`.\n", "\n", "By default, basic Python types such as `int`, `float`, `list` and `dict` are simply be mapped to `Number`, `Array` and `Object`. For more complex types, serializers and de-serializers mustbe specified on both the Python side and the JavaScript side.\n", "\n", "\n", "### Custom serialization and de-serialization on the Python side\n", "\n", "In many cases, a custom serialization must be specified for trait attributes. For example\n", "\n", " - if the trait attribute is not json serializable\n", " - if the trait attribute contains data that is not needed by the JavaScript side.\n", " \n", "Custom serialization can be specified for a given trait attribute through the `to_json` and `from_json` metadata. These must be functions that take two arguments\n", "\n", " - the value to be [de]serialized\n", " - the instance of the underlying widget model.\n", " \n", "In most cases, the second argument is not used in the implementation of the serializer. \n", "\n", "**Example**\n", "\n", "For example, in the case of the `value` attribute of the `DatePicker` widget, the declaration is\n", "\n", "```python\n", "value = Datetime(None, allow_none=True).tag(sync=True, to_json=datetime_to_json, from_json=datetime_from_json)\n", "```\n", "\n", "where `datetime_to_json(value, widget)` and `datetime_from_json(value, widget)` return or handle json data-structures that are amenable to the front-end.\n", "\n", "**The case of parent child relationships between widget models**\n", "\n", "When a widget model holds other widget models, you must use the serializers and deserializers provided in ipywidgets packed into the `widget_serialization` dictionnary.\n", "\n", "For example, the `HBox` widget declares its `children` attribute in the following fashion:\n", "\n", "```python\n", "from .widget import widget_serialization\n", "\n", "[...]\n", "\n", "children = Tuple().tag(sync=True, **widget_serialization)\n", "```\n", "\n", "The actual result of the serialization of a widget model is a string holding the widget id prefixed with `\"IPY_MODEL_\"`.\n", "\n", "### Custom serialization and de-serialization on the JavaScript side\n", "\n", "In order to mirror the custom serializer and deserializer of the Python side, symmetric methods must be provided on the JavaScript side.\n", "\n", "On the JavaScript side, serializers are specified through the `serializers` class-level attribute of the widget model.\n", "\n", "They are generally specified in the following fashion, extending the dictionnary of serializers and serializers of the base class. In the following example, which comes from the `DatePicker`, the deserializer for the `value` attribute is specified.\n", "\n", "```JavaScript\n", "static serializers = _.extend({\n", " value: {\n", " serialize: serialize_datetime,\n", " deserialize: deserialize_datetime\n", " }\n", "}, BaseModel.serializers)\n", "```\n", "\n", "Custom serializers are functions taking two arguments: the value of the object to [de]serialize, and the widget manager. In most cases, the widget manager is actually not used." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Widget Messaging\n", "\n", "The protocol for\n", "\n", "- instantiating jupyter widgets \n", "- synchronizing widget state between the front-end and the back-end companion objects\n", "- sending custom messages between these objects\n", "\n", "Is entirely based upon the `Comm` section of the Jupyter kernel protocol.\n", "\n", "For more details on comms *per se*, we refer to the [relevant section of the specification for the Jupyter kernel protocol](http://jupyter-client.readthedocs.io/en/latest/messaging.html).\n", "\n", "### Implementation of a backend for the Jupyter widgets protocol.\n", "\n", "Jupyter widget libraries built upon ipywidgets tend to have a large part their code-base in JavaScript, since this is where the logic for drawing and rendering widgets resides. The Python side mostly consists in a declaration of the widget model attributes.\n", "\n", "A byproduct of the *thin backend* of widget libraries is that once the widget protocol is implemented for another kernel, all the widgets and custom widget libraries can be reused in that language.\n", "\n", "Therefore, in this documentation, we concentrate on the viewpoint of a kernel author implementing a jupyter widget backend.\n", "\n", "#### The `jupyter.widget` comm target\n", "\n", "Jupyter interactive widgets define two `comm` targets\n", "\n", " - `jupyter.widget`\n", " - `jupyter.widget.version`\n", " \n", "The first one is the target handling all the widget state synchronization as well as the custom messages. The other target is meant for a version check between the front-end and the backend, and can be ignored from now.\n", "\n", "#### Instanciating widgets from the front-end and the backend\n", "\n", "** Reception of a `comm_open` message **\n", "\n", "Upon reception of the `comm_open` message for target `jupyter.widget`\n", "\n", "```python\n", "{\n", " 'comm_id' : 'u-u-i-d',\n", " 'target_name' : 'jupyter.widget',\n", " 'data' : {\n", " 'widget_class': 'some.string'\n", " }\n", "}\n", "```\n", "\n", "The type of widget to be instanciated is determined with the `widget_class` string.\n", "\n", "In the python implementation, this string is actually the key in a registry of widget types. In the case where the key is not found, it is parsed as a `module` `+` `class` string.\n", "\n", "In the Python implementation of the backend, widget types are registered in the dictionary with the `register` decorator. For example the integral progress bar is registered with `register('Jupyter.IntProgress')`.\n", "\n", "** Emmission of the `comm_open` message upon instanciation of a widget **\n", "\n", "Symmetrically, when instanciating a widget in the backend, a `comm_open` message is sent to the front-end.\n", "\n", "```python\n", "{\n", " 'comm_id' : 'u-u-i-d',\n", " 'target_name' : 'jupyter.widget',\n", " 'data' : {\n", " '[serialized widget state]'\n", " }\n", "}\n", "```\n", "\n", "The type of widget to be instanciated in the front-end is determined by the `_model_name`, `_model_module` and `_model_module_version` keys in the state, which respectively stand for the name of the class that must be instanciated in the frontend, the javascript module where this class is defined, and a semver range for that module.\n", "\n", "** Sending updates of the state for a widget model **\n", "\n", "```python\n", "{\n", " 'comm_id' : 'u-u-i-d',\n", " 'data' : {\n", " 'method': 'state',\n", " 'state': '[serialized widget state or portion of the serialized widget sate]',\n", " 'buffers': '[optional list of keys for attributes sent in the form of binary buffers]'\n", " }\n", "}\n", "```\n", "\n", "Comm messages for state synchonization optionally contain a list binary buffers. If this list is not empty, a corresponding list of strings must be provided in the `data` message providing the names for these buffers.\n", "\n", "The front-end will unpack these buffer and insert them in the state for the specified keys.\n", "\n", "** Sending custom messages **\n", "\n", "In the Python implementation, the base widget class provides a means to send raw comm messages directcly. `Widget.send(content, buffers=None)` will produce a message of the form\n", "\n", "```python\n", "{\n", " 'comm_id' : 'u-u-i-d',\n", " 'data' : {\n", " 'method': 'custom',\n", " 'content': 'the specified content',\n", " 'buffers': 'the provided buffers'\n", " }\n", "}\n", "```\n", "\n", "** Receiving data synchronization messages **\n", "\n", "Up on updates of the JavaScript model state, the front-end emits widget state patches messahes\n", "\n", "```python\n", "{\n", " 'comm_id' : 'u-u-i-d',\n", " 'data' : {\n", " 'method': 'backbone',\n", " 'sync_data': 'the patch to the data',\n", " 'buffers': 'optional buffer names list'\n", " }\n", "}\n", "```\n", "\n", "The `sync_data` contains the serialized state of the changed model attributes in the form of a dictionnary.\n", "\n", "Optionally, the message may specify a list of buffer names. When provided, the corresponding binary buffers in the zmq message should be appended in the `sync_data` dictionary with the keys specified in the `buffers` list.\n", "\n", "** State requests **\n", "\n", "In the case of a front-end connecting to a running kernel where widgets have already been instanciated, it may send a request state message, of the form\n", "\n", "```python\n", "{\n", " 'comm_id' : 'u-u-i-d',\n", " 'data' : {\n", " 'method': 'request_state'\n", " }\n", "}\n", "```\n", "\n", "The expected response to that message is a regular `update` message as specified above containining the entirety of the widget model state." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Installation" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Because the API of any given widget **must exist in the kernel**, the kernel is the natural place for widgets to be installed. However, **kernels, as of now, don’t host static assets**. Instead, static assets are hosted by the webserver, which is the entity that sits between the kernel and the front-end. This is a problem, because it means widgets have components that need to be **installed both in the webserver and the kernel**. The kernel components are easy to install, because you can rely on the language’s built in tools. The static assets for the webserver complicate things, because an extra step is required to let the webserver know where the assets are." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Static assets" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "In the case of the classic Jupyter notebook, static assets are made available to the Jupyter notebook in the form of a Jupyter extensions. JavaScript bundles are copied in a directory accessible through the `nbextensions/` handler. Nbextensions also have a mechanism for running your code on page load. This can be set using the install-nbextension command." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Distribution" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "A template project is available in the form of a cookie cutter: https://github.com/jupyter/widget-cookiecutter\n", "\n", "This project is meant to help custom widget authors get started with the packaging and the distribution of Jupyter interactive widgets.\n", "\n", "It produces a project for a Jupyter interactive widget library following the current best practices for using interactive widgets. An implementation for a placeholder \"Hello World\" widget is provided." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": { "656d04c01fd640a99b97c0b3b961685f": { "model_module": "jupyter-js-widgets", "model_module_version": "~2.0.30", "model_name": "SliderStyleModel", "state": { "_model_module_version": "~2.0.30", "_view_module_version": "~2.0.30" } }, "9863258038944c80b155e81633469a84": { "model_module": "jupyter-js-widgets", "model_module_version": "~2.0.30", "model_name": "IntSliderModel", "state": { "_model_module_version": "~2.0.30", "_view_module_version": "~2.0.30", "layout": "IPY_MODEL_c6aa61369e9b4891abdde51e84bf4e57", "style": "IPY_MODEL_656d04c01fd640a99b97c0b3b961685f" } }, "c6aa61369e9b4891abdde51e84bf4e57": { "model_module": "jupyter-js-widgets", "model_module_version": "~2.0.30", "model_name": "LayoutModel", "state": { "_model_module_version": "~2.0.30", "_view_module_version": "~2.0.30" } } }, "version_major": 1, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 1 }