{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Magics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Magics are escape hatch syntax that allow you insert non-python syntax into a python cell withotu using string.\n",
    "\n",
    "You can find two kinds of magics: \n",
    "\n",
    "- line magics (start with a single percent `%magic`)\n",
    "- cell magics (start with a double percent `%%magic`)\n",
    "\n",
    "Line magic will get the content of the **physical line** after the magic (up until the first following `\\n`)\n",
    "Cell magics will get the content of the line following them and the rest of the cell."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A simple tic-toc magic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we'll earn how to define our own IPython magic. Magics are usualy function (for stateless magics) or methods (usually for stateful magics or magics that needs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll want to time things later, so let's see if we can make a simple line magic that mesure the execution time of a function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from IPython.core.magic import register_line_magic, register_line_cell_magic\n",
    "import inspect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(register_line_magic.__doc__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inspect.signature(compile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%load tictoc_solution.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def pyfib(n):\n",
    "    if n < 2:\n",
    "        return 1\n",
    "    else:\n",
    "        return pyfib(n-1) + pyfib(n-2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dt = %tictoc pyfib(25)\n",
    "dt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's just graph the time it takes as a fucntion of of the input parameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pytimes = []\n",
    "pyX = list(range(30))\n",
    "for i in pyX:\n",
    "    t = %tictoc pyfib(i)\n",
    "    pytimes.append(t)\n",
    "py = {\"x\":pyX, 'y':pytimes, 'label':'py'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compare(*args):\n",
    "    fig, ax = plt.subplots()\n",
    "    _min = min(args[0]['y'])\n",
    "    for arg in args:\n",
    "        _min = min(min(arg['y']), _min)\n",
    "        ax.scatter(**arg)\n",
    "    ax.set_yscale('log')\n",
    "    ax.set_ylim(_min/10)\n",
    "    ax.set_xlabel('n')\n",
    "    ax.set_ylabel('time (s)')\n",
    "    ax.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "compare(py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise:\n",
    "\n",
    " - Transform the above to a function\n",
    " - Observe that %tictoc does not have access to locals. \n",
    " - You need to wrap it in `IPython.core.magics:@needs_local_scope` decorator, and make the funciton a class magic. \n",
    " - Look at what `timeit` does, with the `-n` and `-r` and `-o` flags."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Going faster \n",
    "\n",
    "As a case study for cross language integration we'll attempt to speed the above fibonacci function. \n",
    "From Physics we know that there is an absolute speed limit so let's use C. \n",
    "\n",
    "Here is the necessary header and source to get a working Fibonacci function in C. \n",
    "\n",
    "We'll fist do that manually, and then write a magic that automatically make this easier for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "header = \"int cfib(int);\"\n",
    "source = \"\"\"\n",
    "int cfib(int n){\n",
    "    if (n < 4)\n",
    "        return 1;\n",
    "    else\n",
    "        return cfib(n-1) + cfib(n-2);\n",
    "}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We don't want to work in vacuum, so let's use FFI, which allow us to easily compile C code on the fly. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cffi import FFI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An `FFI` object need to have at least `cdef()` and `set_source()` called with the right values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename = 'mycfibfile'\n",
    "ffi = FFI()\n",
    "ffi.cdef(header)\n",
    "ffi.set_source(filename, source)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's compile the above into a shared object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "shared_object = ffi.compile()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "let's import our newly created module, and look at what's inside"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cfib_module = __import__(filename)\n",
    "dir(cfib_module.lib)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we finally get out c-based fibonacci function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cfib = cfib_module.lib.cfib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(cfib.__doc__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now compare its speed to the Python Fibbonacci we had before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ctimes = []\n",
    "cX = list(range(42))\n",
    "for i in cX:\n",
    "    t = %tictoc cfib(i)\n",
    "    ctimes.append(t)\n",
    "c = {\"x\":cX, 'y':ctimes, 'label':'C'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "compare(py, c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Write a magic class. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Staful Magics, or magics which needs to have access to IPython internals (like for example injecting variables in to namespaces) need to be full classes which methods will be called when the user invoke magics.\n",
    "\n",
    "Your class needs to derive from `IPython.core.magic:Magics`, and should be decorated with `IPython.core.magic:@magics_class`. Each methods on this class will can be different magic when decorated with `@cell_magic` and similar. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will find below the skeletton of a magic to define inline C function. We've given you two useful pieces of code:\n",
    "\n",
    " - Generating a random filename (shared objects cannot be reloaded)\n",
    " - And how to inject a function with a given name in the user namespace. \n",
    " \n",
    "### Exercise:\n",
    "\n",
    "Complete the code bellow using previsous section in order to compile a C function into a loadable Python module, and inject it into the user namespace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from IPython.core.magic import magics_class, cell_magic, Magics\n",
    "import string\n",
    "from random import choice\n",
    "\n",
    "@magics_class\n",
    "class CFFI(Magics):\n",
    "    \n",
    "    @cell_magic\n",
    "    def c(self, line, cell):\n",
    "        rname = '_cffi_%s' % ''.join([choice(string.ascii_letters) for _ in range(10)])\n",
    "        self.shell.user_ns['fib'] = lambda x:x\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to manually register the class with IPython."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_ipython().register_magics(CFFI)        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%c int d(int); \n",
    "int d(int n){ return n*2;}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "compare(c, py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## About correctness "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check that out implementation is correct, hopefully we should get the same values for both the Python and C based Fib functions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[(pyfib(i), cfib(i)) for i in range(10)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the values differ, check that that changing the C source code and rerunning the function gives you the right value. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementation note:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We definitively did not choose the smartest algorithme for the fibonacci function. You can get much faster resuls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def ffib(n):\n",
    "    if n < 2:\n",
    "        return 1\n",
    "    a,b = 1,1\n",
    "    for i in range(n-2):\n",
    "        a,b = b, a+b\n",
    "    return b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t = %timeit -n1 -r1 -o ffib(23)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "fftimes = []\n",
    "ffX = list(range(100))\n",
    "for i in ffX:\n",
    "    t = %tictoc ffib(i)\n",
    "    fftimes.append(t)\n",
    "ff = {\"x\":ffX, 'y':fftimes, 'label':'C'}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = %time 1+1 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "compare(c, py, ff)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "36",
   "language": "python",
   "name": "36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}