{ "metadata": { "name": "", "signature": "sha256:51ca2640929c4aea4addb291eac93429243e4831f1ff02eaf6aa4422201740ae" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "
The IPython Protocol

Frontends and Kernels
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



\n", "

\"Living the Dream!\"

\n", "




\n", "\n", "

with [Paul Ivanov](http://twitter.com/ivanov) and [Thomas Kluyver](https://twitter.com/takluyver)\n", "

[https://github.com/ipython/front-to-back](https://github.com/ipython/front-to-back)\n", "

\n", "




\n", "


\n", "\n", "(thanks to [Jonathan Frederic](https://twitter.com/GooseJon) for last minute [presentation shortcuts](https://gist.github.com/jdfreder/e2bf6c6c6beadf0a4477))\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "The logistics of this presentation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![ ](/files/images/horse.jpg)" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "What is IPython?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![ipython universe](/files/images/ipython-uni.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "The punchline..." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![ipython universe](/files/images/ipython-uni.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"Why should I care?\"\n", "\n", "Implement one piece - and get the others for free.\n", "\n", "(free as in BSD)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "This talk" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " \n", " \n", "![ipython universe](/files/images/ipython-oos.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Analogy \n", "\n", "(imperfect, but useful)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", "\n", "
The WebIPython
\n", " Browser\n", "
\n", " ![](/files/images/nextstation.jpg)\n", " 		\n", " Client\n", "
\n", " ![](/files/images/computer.png)\n", "
\n", "
\n", "

speaks **HTTP** with

\n", "
\n", "\n", "
\n", "\n", " 		\n", "
\n", "\n", "

speaks \"ICCP\" with

\n", "
\n", "\n", "
\n", " Webserver\n", "
\n", " ![](/files/images/next.jpg)\n", " 		\n", " Kernel\n", "
\n", " ![](/files/images/pdp12.jpg)\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "ICCP?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**I**nteractive **C**omputing **C**ommunication **P**rotocol\n", "\n", "I wanted \"IPython Protocol\" \n", "- but IP was already taken :\\\n", "\n", "\"Interactive Computing Protocol\" \n", "- also taken (\"[Internet Cache Protocol](http://en.wikipedia.org/wiki/Internet_Cache_Protocol)\", RFCs\n", "[2186](http://tools.ietf.org/html/rfc2186) and\n", "[2187](http://tools.ietf.org/html/rfc2187))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Kernel" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "Computational Engine" ] }, { "cell_type": "heading", "level": 3, "metadata": {}, "source": [ "- accepts messages\n", "

\n", "- performs actions\n", "

\n", "- sends out results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "One-to-Many (Kernel to Clients)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "

console qtconsole vim-ipython vim-ipython bipython \n", "





\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "kernel\n", "

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Anatomy of a Message" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In brief, read [the messaging\n", "spec](http://ipython.org/ipython-doc/dev/development/messaging.html) for more\n", "details.\n", "\n", "Essential fields:\n", "\n", " header:\n", " msg_id:\n", " msg_type: execute_request, execute_reply, object_info_request,\n", " complete_request, complete_reply, history_request, ...\n", "\n", " parent_header: (see above)\n", "\n", " content: structure depends on message type\n", "\n", " example: `execute_request`\n", " - `code` (str)\n", " - `silent` (bool)\n", " - `store_history` (bool)\n", "\n", " example: `execute_reply`\n", " - `status` ('ok', 'error', or 'abort')\n", " - `execution count` (prompt numbers)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Client" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 2, "metadata": {}, "source": [ "- connects to exactly one kernel\n", "


\n", "- send messages to kernel based on user actions\n", "


\n", "- displays kernel messages when appropriate" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Why have many clients?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Differences in\n", "\n", "\n", "- **multiline editing**\n", " - qtconsole\n", " - vim-ipython\n", " - notebook\n", "- **display capabilities**\n", " - images in qtconsole\n", " - images, video, javascript in notebook\n", "- **persistence** \n", " - no save and rerun in console, qtconsole\n", " - .ipynb files for notebook, EIN\n", " - files for vim-ipython\n", "- **user interface** \n", "

I have a serious illness: chronic CLI. It's terminal.

— Paul Ivanov (@ivanov) May 3, 2014
\n", " \n", "- **client-side requirements**\n", " - python for console, qtconsle, vim-ipython, bipython\n", " - modern webbrowser for notebook\n", "- **interactivity** (bipython)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Clients" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## ship with ipython:\n", "\n", "`ipython qtconsole` - the original decoupling, uses Qt4\n", "\n", "`ipython console` - terminal version, decoupled \n", "\n", "- (different from the classic single process `ipython`)\n", "\n", "both of these actually start a kernel and a client, \n", " pass `--existing` flag to not start a *new* kernel \n", " and instead simply connect to one that already\n", " exists\n", "`ipython notebook` ♦ ♦ - more on this later" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3rd party\n", "[vim-ipython](https://github.com/ivanov/vim-ipython)\n", "\n", "[bipython](http://bipython.org)\n", "\n", "[EIN - Emacs IPython Notebook](http://tkf.github.io/emacs-ipython-notebook/) (OOS)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "[vim-ipython](https://github.com/ivanov/vim-ipython)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[vim](http://vim.org) plugin first written during the sprints at [SciPy\n", "2011](http://conference.scipy.org/scipy2011/)\n", "\n", "[demo #1](files/images/demo1.mp4) [demo #2](files/images/demo1.mp4)\n", "\n", "


\n", "## Live demo?\n", "


\n", "\n", "### PSA: vim-ipython common problem (e.g. OS X)\n", "\n", "Vim needs to be compiled with Python support (`vim --version | grep +python`)\n", "AND the Python linked must have IPython with ZeroMQ support\n", "\n", "HOWEVER: the Python linked **does not** need to be your preferred Python.\n", "\n", "Solution: either install IPython and PyZMQ into system Python, or compile vim\n", "against a Python which has IPython and PyZMQ (Python that comes with Anaconda,\n", "for example)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "[bipython](http://bipython.org)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[![bipython logo](/files/images/bipython_logo.png)](http://bipython.org)



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "ripped out the messaging code from vim-ipython, \n", "\n", "and slapped them on top of [bpython](http://bpython-interpreter.org/)'s [urwid](http://urwid.org/) interface\n", "\n", "...in 2 days." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Live Demo?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Embed is you friend!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In any Python code you have lying around, do this:\n", "\n", "```python\n", "import IPython; IPython.embed_kernel()\n", "```\n", "\n", "When that code runs, it will prompt something like this:\n", "\n", " NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.\n", "\n", " To exit, you will have to explicitly quit this process, by either sending\n", " \"quit\" from a client, or using Ctrl-\\ in UNIX-like environments.\n", "\n", " To read more about this, see https://github.com/ipython/ipython/issues/2049\n", "\n", "\n", " To connect another client to this kernel, use:\n", " --existing kernel-5979.json\n", "\n", "(demo if time permits)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Take away for clients" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "You favorite tool can talk (bi-directionally) to Python!\n", "

\n", "\n", "![takeout](/files/images/takeout.jpg)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "



" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Bonus slides" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Lies" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "ICCP over zermoq happens over multiple channels (3) not just one.\n", " - stdin\n", " - Shell ( action requests: execution, object introspection, code completion )\n", " - IOPub ( \"broadcast channel\" )" ] }, { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "What about the notebook? (analogy)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "see [sequence diagram](/files/notebook-as-proxy.html)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import IPython.display as d" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 8 }, { "cell_type": "code", "collapsed": false, "input": [ "d.IFrame(\"/files/notebook-as-proxy.html\", 800, 600)" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "\n", " \n", " " ], "metadata": {}, "output_type": "pyout", "prompt_number": 10, "text": [ "" ] } ], "prompt_number": 10 } ], "metadata": {} } ] }