{
 "metadata": {
  "name": "",
  "signature": "sha256:178c82ec6e17ae009ab45fcc2abf547df823f501947a1c6a73a3a7054c7a611c"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<h1 align=center>An intro to <a href=\"http://cs.mcgill.ca/~jpineau/comp598/\">Applied Machine Learning</a> in Python</h1>\n",
      "<h2 align=center>Pierre-Luc Bacon</h2>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# How to Python at McGill\n",
      "\n",
      "Python is great because it has so many packages. However, you will quickly discover that installing packages without administrative privileges can become challenging. There is [virtualenv](http://www.virtualenv.org/en/latest/) that lets you install packages locally but it can be a nightmare when you have C librairies to build. \n",
      "\n",
      "If you want to do scientific computing with Python, I advise you to go with the [Anaconda](http://continuum.io/downloads) Python distribution. It will save you (and the SOCS sysadmins) a great deal of time. To install Anaconda on Linux for 64 bits machines, it would look like:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash\n",
      "wget http://repo.continuum.io/archive/Anaconda-2.0.1-Linux-x86_64.sh\n",
      "chmod +x Anaconda-2.0.1-Linux-x86_64.sh\n",
      "./Anaconda-2.0.1-Linux-x86_64.sh\n",
      "conda update conda\n",
      "conda update ipython"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "IPython notebook can then be started from the command line with `ipython notebook --pylab=inline` in a directory where you want to store your notebooks."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# The magic of IPython\n",
      "\n",
      "If you miss the interactive style of the Matlab prompt, it's a good idea to use the [IPython](http://ipython.org/) shell or the web-based [IPython notebook](http://ipython.org/notebook.html). Notebooks embody the idea of [literate programming](http://en.wikipedia.org/wiki/Literate_programming) interleaving text, math rendering, executable code, plots and other forms of medias. It's also a great way to embrace reproducibility ! Download this notebook and see for yourself.  \n",
      "\n",
      "Although IPython has the *python* suffix in its name, its modular design allows for a great deal of interoperability with other languages. For example, you can call bash through the so-called *cell magics*. For example:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%pwd"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 2,
       "text": [
        "u'/Users/pierrelucbacon/workspace/notebooks'"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In order to capture the output of a bash cell into Python variables, you could also do:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash --out output --err error\n",
      "echo $RANDOM"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print output"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "22358\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can obtain the list of supported cell magics through `%lsmagic`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%lsmagic"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "json": [
        "{\"cell\": {\"prun\": \"ExecutionMagics\", \"file\": \"Other\", \"!\": \"OSMagics\", \"capture\": \"ExecutionMagics\", \"timeit\": \"ExecutionMagics\", \"script\": \"ScriptMagics\", \"pypy\": \"Other\", \"system\": \"OSMagics\", \"perl\": \"Other\", \"HTML\": \"Other\", \"bash\": \"Other\", \"python\": \"Other\", \"SVG\": \"Other\", \"javascript\": \"DisplayMagics\", \"writefile\": \"OSMagics\", \"ruby\": \"Other\", \"python3\": \"Other\", \"python2\": \"Other\", \"latex\": \"DisplayMagics\", \"sx\": \"OSMagics\", \"svg\": \"DisplayMagics\", \"html\": \"DisplayMagics\", \"sh\": \"Other\", \"time\": \"ExecutionMagics\", \"debug\": \"ExecutionMagics\"}, \"line\": {\"psource\": \"NamespaceMagics\", \"logstart\": \"LoggingMagics\", \"popd\": \"OSMagics\", \"loadpy\": \"CodeMagics\", \"install_ext\": \"ExtensionMagics\", \"colors\": \"BasicMagics\", \"who_ls\": \"NamespaceMagics\", \"lf\": \"Other\", \"install_profiles\": \"DeprecatedMagics\", \"ll\": \"Other\", \"pprint\": \"BasicMagics\", \"lk\": \"Other\", \"ls\": \"Other\", \"save\": \"CodeMagics\", \"tb\": \"ExecutionMagics\", \"lx\": \"Other\", \"pylab\": \"PylabMagics\", \"killbgscripts\": \"ScriptMagics\", \"quickref\": \"BasicMagics\", \"magic\": \"BasicMagics\", \"dhist\": \"OSMagics\", \"edit\": \"KernelMagics\", \"logstop\": \"LoggingMagics\", \"gui\": \"BasicMagics\", \"alias_magic\": \"BasicMagics\", \"debug\": \"ExecutionMagics\", \"page\": \"BasicMagics\", \"logstate\": \"LoggingMagics\", \"ed\": \"Other\", \"pushd\": \"OSMagics\", \"timeit\": \"ExecutionMagics\", \"rehashx\": \"OSMagics\", \"hist\": \"Other\", \"qtconsole\": \"KernelMagics\", \"rm\": \"Other\", \"dirs\": \"OSMagics\", \"run\": \"ExecutionMagics\", \"reset_selective\": \"NamespaceMagics\", \"rep\": \"Other\", \"pinfo2\": \"NamespaceMagics\", \"matplotlib\": \"PylabMagics\", \"unload_ext\": \"ExtensionMagics\", \"doctest_mode\": \"KernelMagics\", \"logoff\": \"LoggingMagics\", \"reload_ext\": \"ExtensionMagics\", \"pdb\": \"ExecutionMagics\", \"load\": \"CodeMagics\", \"lsmagic\": \"BasicMagics\", \"autosave\": \"KernelMagics\", \"cd\": \"OSMagics\", \"pastebin\": \"CodeMagics\", \"prun\": \"ExecutionMagics\", \"cp\": \"Other\", \"autocall\": \"AutoMagics\", \"bookmark\": \"OSMagics\", \"connect_info\": \"KernelMagics\", \"mkdir\": \"Other\", \"system\": \"OSMagics\", \"whos\": \"NamespaceMagics\", \"rmdir\": \"Other\", \"automagic\": \"AutoMagics\", \"store\": \"StoreMagics\", \"more\": \"KernelMagics\", \"pdef\": \"NamespaceMagics\", \"precision\": \"BasicMagics\", \"pinfo\": \"NamespaceMagics\", \"pwd\": \"OSMagics\", \"psearch\": \"NamespaceMagics\", \"reset\": \"NamespaceMagics\", \"recall\": \"HistoryMagics\", \"xdel\": \"NamespaceMagics\", \"xmode\": \"BasicMagics\", \"cat\": \"Other\", \"mv\": \"Other\", \"rerun\": \"HistoryMagics\", \"logon\": \"LoggingMagics\", \"history\": \"HistoryMagics\", \"pycat\": \"OSMagics\", \"unalias\": \"OSMagics\", \"install_default_config\": \"DeprecatedMagics\", \"env\": \"OSMagics\", \"load_ext\": \"ExtensionMagics\", \"config\": \"ConfigMagics\", \"profile\": \"BasicMagics\", \"pfile\": \"NamespaceMagics\", \"less\": \"KernelMagics\", \"who\": \"NamespaceMagics\", \"notebook\": \"BasicMagics\", \"man\": \"KernelMagics\", \"sx\": \"OSMagics\", \"macro\": \"ExecutionMagics\", \"clear\": \"KernelMagics\", \"alias\": \"OSMagics\", \"time\": \"ExecutionMagics\", \"sc\": \"OSMagics\", \"ldir\": \"Other\", \"pdoc\": \"NamespaceMagics\"}}"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 5,
       "text": [
        "Available line magics:\n",
        "%alias  %alias_magic  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %install_default_config  %install_ext  %install_profiles  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode\n",
        "\n",
        "Available cell magics:\n",
        "%%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%latex  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile\n",
        "\n",
        "Automagic is ON, % prefix IS NOT needed for line magics."
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Another useful cell magic is also `%timeit`, giving you an estimate of the time spent executing a given function. For example:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%timeit np.linalg.eigvals(np.random.rand(50,50))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "1000 loops, best of 3: 1.12 ms per loop\n"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can learn more about various cell magics in this [notebook](http://nbviewer.ipython.org/github/ipython/ipython/blob/1.x/examples/notebooks/Cell%20Magics.ipynb)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Talking to R\n",
      "\n",
      "If you have R installed, IPython is able to execute a cell in R and return the result to Python. This can be useful if you find that a particular library was written only for R while your project uses mainly Python. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%bash\n",
      "pip install rpy2"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%load_ext rpy2.ipython"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = np.linspace(0,10,50)\n",
      "y = 0.9*x + np.random.randn(50)\n",
      "plt.plot(x,y,'.')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "[<matplotlib.lines.Line2D at 0x10b355dd0>]"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAEACAYAAABBDJb9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADmhJREFUeJzt3X+IZWUdx/H3tmM/piWniHYzF8Y2jIjYlClKjE5lYItU\n/ywYUWbg/lGRFpRakPNX9IMwgvpjCm2lxhIL0WittTr6h2JWauav1a3wF2poUxRESbc/njs7d9r5\nde4595znPM/7BcPcO5575pmL+znnfp/vOQ9IkiRJkiRJkiRJkiRJkiS17grgKeCekZ+9DDgMHAF+\nDsx0MC5JEvC8LW53JXD2//3sEkKYnwr8YvhckhS5WVafmT8A7Bw+3jV8LknqwFbPzNeyk1B6Yfh9\n5wbbSpImqE6YjxoMvyRJHZiq8dqnCOWVJ4FXAk+vtdGePXsGR48erfFrJClLR4HXbHXjOmfm1wPn\nDR+fB1y35miOHmUwGPg1GHDZZZd1PoZYvnwvfC98L9b/uuCCAcCeKoG81TC/GrgVeC3wKHA+8CXg\n3YTWxHcOn0uSajpypPprtlpm+cA6Pz+r+q+UJG1kerr6a5qaANUWFEXR9RCi4Xuxwvdihe9FsLhY\n/TXbmh/GcQaDgY0uklTFtm3boEJGe2YuSQkwzCUpAYa5JCXAMJekBBjmkpQAw1ySEmCYS1ICDHNJ\n2sSBA1AUsG8fLC11PZq1GeaStIkjR+Dmm+HQoRDsMTLMJWkTy/dKmZuDhYVux7IeL+eXpE0sLYUz\n8oUFmGlp6fqql/Mb5pIUIe/NIkkZMswlKQGGuSQlwDCXpAQY5pKUAMNckhJgmEtSAgxzSUqAYS5J\nCTDMJSkBhrkkJcAwl6QEGOaSlADDXJIS0ESYXwrcC9wDLAIvaGCfkpS0ppeiqxvms8AFwOnAG4Dt\nwLk19ylJvTFuKDe9FF3dMP878B9gGpgafn+87qAkCfqxkPK4odz0UnR1w/xZ4GvAI8ATwBJwU91B\nSRL0YyHlcUN5cRH274fDh5tZiq5umO8BLiKUW04CdgAfrLlPSQL6sZDyuKE8MwPXXNPcmqJTNV8/\nB9wKPDN8/mPgDOD7oxvNz88fe1wUBUVR1Py1knKwuNj+QspVLYdyXWVZUpbl2K+vu6DzXkJwvwn4\nF/Bd4NfAN0e2cUFnKUIHDoQyxvR0CM1Yw7Jpffm7217Q+W7gKuA3wO+HP4v0w5CkUX2oR09Cqn93\n3TILwFeGX5J6pA/16ElI9e+uW2bZCsssUoSWluKvR09CX/7uqmUWw1ySiK+W3nbNXJKS0PdaumEu\nSfS/lm6ZRZKIr5ZuzVySIlGnDm/NXJIi0WYd3jCXpAlpsw5vmUXquUm01MXWptdXderw1sylzBRF\n+CgP4e59Tdz0aRL7VDXWzKXMTOKjfJvlgT4sQNEHnplLPTeJlro22/T8FLA2yyySemXfvtDtMTfX\n3Ko7KTDMJfVKbBfrxMIwl6QEOAEqSRkyzCUpAYa5tAHb5tQXhrm0gb7f41r5MMylDfT9HtfKh90s\n0gZsm1NXbE2UpATYmiitw8lMpcwwVzaczFTKDHNlw8lMpcyaubLhZKb6xAlQSZ1xhaLmOAEqqTPO\nS3THMJfUGOclutNEmM8A1wL3A/cBb2lgn5J6aHExrBbkIhPta6JmfhC4GbgCmAJeDPxt5L9bM5ek\nitqeAD0RuBN49QbbGOaSVFHbE6CnAH8BrgR+B3wbmK65T6kXcr2iNNe/O3ZTDbz+dOATwB3A14FL\ngC+MbjQ/P3/scVEUFEVR89dK3Vvu3IAQcLmsKp/r3z1pZVlSluXYr69bZtkF3EY4Qwc4kxDm54xs\nY5lFScp1Vflc/+62tV1meRJ4FDh1+Pws4N6a+5R6IdfOjVj+bss9qzXRzbIX+A7wfOAocD52s0ia\nsKJYKffs359euafqmXndmjnA3cCbGtiPJG2ZFyit5r1ZpJZ5/5JmpH7jNG+0JUUu9fJADFI4YHqj\nLSlylgcmL8cbfhnmUsti6QZJWY4HTMssUk+kUDpoSwr1dGvmUqKstefFmrmylvKFJDmWDrR1hrmS\nkvLEV59r7SkfZGPRxEVDUjRSPnudmelvacWbc02eZ+ZKSp/PXlOW8kE2Fk6AShNg58lqKXSXtM1u\nFikCdp6oLrtZpAhYVlDbPDOXJsCyguqyzCJJCbDMIkkZMswlKQGGuSQlwDCXIuJl7xqXYa7eSTnw\nUr63jCbLMFfvpBx49qdrXIa5eiflwPPeMhqXfebqHS/IUQ68aEiSEuBFQ5KUIcNcnUm5K0Vqm2Gu\nzsTUleKBRX1nmKszMXWlxHRgaZIHqXw0FebbgTuBGxranzIQUxteTAeWJqV6kNLxmgrzC4H7ANtW\neq7NM7nlBYq7DnKI68DSpFQPUjpeE2F+MrAP+A7ttDpqgnI9k4vpwNKkVA9SOt5UA/u4HPgM8JIG\n9qWOeSaXluWDlNJXN8zPAZ4m1MuL9Taan58/9rgoCopi3U3VscXF5q+udKV6aXNlWVKW5divr1sW\n+SLwIeA54IWEs/MfAR8e2cYrQDPnSvVSdW1fAfo5YDdwCnAu8EtWB7lk6UZqQdN95p6C6zhOwkmT\n5422JClC3mhLkjJkmEtSAgzzhnkvDEldMMwblusVlJK6ZZg3zDY8SV2wm6Vhrk8pqQmuASpJCbA1\nUZIyZJhLUgIMc0lKgGGeIXvhpfQY5hnqQy+8BxypGsM8Q5PohW86fPtwwJFiYphnaBK3pG06fL34\nSqrGPnM1Yt++EORzc80cJLz4SrnzoiF1wvCVmmWYS1ICvAJUkjJkmEtSAgzzSNhXLakOwzwS9lVL\nqsMwj4R91ZLqsJslErb2SRpla6IkJcDWxAQ5OSppM4Z5Dzg5KmkzhnkPODkqaTPWzHvAyVEpP21P\ngO4GrgJeAQyABeAb/7eNYS5JFbUd5ruGX3cBO4DfAu8H7h/ZxjCXpIra7mZ5khDkAP8ghPhJNfcp\nSaqoyQnQWeA04PatbGy73WT5/kp5mWpoPzuAa4ELCWfoq8zPzx97XBQFRVEca7eDEDzXXNPQSATg\n+yv1TFmWlGU59uub6GY5AfgJcAj4+hr/fc2aedPLjGk131+p39qeAN0GHASeAT61zjZrhrntdpPl\n+yv1W9thfiZwC/B7QmsiwKXAjSPbRN3NcuBAKElMT4dV6w0+STHwRlsVFcVKbXn/fmvLkuLgjbYq\n8lJ5SSnI/szc2rKkGFlmkaQEWGbJjBcHSQLDvPe817kkMMx7zwlcSWDNvPecwJXS5ASoJCXACdAG\nObkoqS8M8w04uSipL5IJ80mcRTu5uJqfVKR4JRPmG51FjxtCi4vhfi1N3UK272HoJxUpXsmE+UZn\n0eOG0MxMuPFWU10ifQ9DP6lI8epVmG90ZrvRWXQsIRTLOMbV9CcVSc3pVWviuLerjaUXO5ZxSIpf\n0n3mLoUmKRdJh7lntpJykXSYS1IuvAJUkjLUSpj3ta9akvqilTILDFwsWZIqiLJmPjc3sPtEkiqI\nMsz/+teBQS5JFUQZ5nazSFI1drNIUoYMc0lKgGEuSQkwzCUpAU2E+dnAA8BDwMUN7E+SVFHdbpbt\nwIPAWcDjwB3AB4D7R7axm0WSKmq7m+XNwMPAn4H/AD8A3ldzn5KkiuqG+auAR0eePzb8mSSpRXXD\nvPH6Sd8XPZakLkzVfP3jwO6R57sJZ+erzM/PH3tcFAVFUay7w+VFjyEEuzfnkpSDsiwpy3Ls19ed\nAJ0iTIC+C3gC+DU1J0BdGk6S2p8AfQ74BPAz4D7gh6wO8spcAV6SqvNGW5IUIW+0JUkZMswlKQGG\nuSQlwDCXpAQY5pKUAMNckhJgmEtSAgxzSUqAYS5JCTDMJSkBhrkkJcAwl6QEGOaSlADDfAyuhiQp\nNob5GJZXQzp0KAS7JHXNMB/D9HT4PjcHCwvdjkWSwMUpxrK0FM7IFxZcDUnSZFRdnMIwl6QIudKQ\nJGXIMJekBBjmkpQAw1ySEmCYS1ICDHNJSoBhLkkJMMwlKQGGuSQlwDCXpATUCfOvAvcDdwM/Bk5s\nZESSpMrqhPnPgdcDe4EjwKWNjChhZVl2PYRo+F6s8L1Y4Xsxvjphfhj47/Dx7cDJ9YeTNv9HXeF7\nscL3YoXvxfiaqpl/FPhpQ/uSJFU0tcl/PwzsWuPnnwNuGD7+PPBvYLHBcUmSKqh7P/OPABcA7wL+\ntc42DwN7av4eScrNUeA1bfyis4F7gZe38cskSeurc2b+EPB84Nnh89uAj9UekSRJkqRmnQ08QDiD\nv7jjsXRpN/ArQknqD8Anux1OFLYDd7IyiZ6rGeBawsV39wFv6XY4nbqU8G/kHkIzxQu6HU6rrgCe\nIvzty15GaEA5Qrimp7Ol47cTJj5ngROAu4DXdTWYju0C3jh8vAN4kHzfi2WfBr4PXN/1QDp2kNDW\nC6GzLNerqGeBP7IS4D8EzutsNO17G3Aaq8P8K8Bnh48vBr7U9qCWvRW4ceT5JcMvwXWE7p9cnQzc\nBLyDvM/MTyQEmMJZ6IPASwkHtRuAszodUftmWR3mDwA7h493DZ9vaFI32noV8OjI88eGP8vdLOEI\nfHvH4+jS5cBnWLl6OFenAH8BrgR+B3wbmO50RN15Fvga8AjwBLBEOODnbCeh9MLw+84NtgUmF+aD\nCe23z3YQ6qMXAv/oeCxdOQd4mlAvr3uNQ99NAacD3xp+/yf5fnrdA1xEONk5ifBv5YNdDigyA7aQ\nqZMK88cJE3/LdhPOznN1AvAj4HuEMkuuzgDeC/wJuBp4J3BVpyPqzmPDrzuGz68lhHqO5oBbgWeA\n5wh3YT2j0xF17ylWrr5/JeEkqBNThKuXZgm96DlPgG4jBNblXQ8kMm8n75o5wC3AqcPH88CXuxtK\np/YSOr1eRPj3chD4eKcjat8sx0+ALncBXkKHE6AA7yFMajxM3rfHPZNQH76LUF64k9C2mbu3YzfL\nXsKZuWsChM6N5dbEg4RPs7m4mjBX8G/CXOP5hEnhm4igNVGSJEmSJEmSJEmSJEmSJEmSJElSR/4H\n4p4HX+pwSzYAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10b1799d0>"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%Rpush x y\n",
      "%R lm(y~x)$coef"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "<FloatVector - Python:0x10b17dfc8 / R:0x102e54240>\n",
        "[0.412314, 0.830338]"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can learn more about the rmagic in this [notebook](http://nbviewer.ipython.org/github/ipython/ipython/blob/3607712653c66d63e0d7f13f073bde8c0f209ba8/docs/examples/notebooks/rmagic_extension.ipynb). "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Talking to Matlab\n",
      "\n",
      "The [pymatbridge](https://github.com/arokem/python-matlab-bridge) module and cell magic allows you to spawn matlab in the background and feed it commands from IPython. This extension is not part of the core IPython and you will also need to install it manually."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%load_ext pymatbridge"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Starting MATLAB on http://localhost:53704\n",
        " visit http://localhost:53704/exit.m to shut down same\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "MATLAB started and connected!\n"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%matlab \n",
      "x = linspace(0.01,6*pi,100);\n",
      "plot(sin(x))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGwCAIAAADOgk3lAAAACXBIWXMAAAsSAAALEgHS3X78AAAA\nIXRFWHRTb2Z0d2FyZQBBcnRpZmV4IEdob3N0c2NyaXB0IDguNTRTRzzSAAAQdklEQVR4nO3c3Xaj\nSLaFUTjD7//KnAvaKqWEJfQDEWvHnBc13FaONmUEX8QWlfOyLBMApPm/1gcAAO8QMAAiCRgAkQQM\ngEgCBkAkAQMgkoABEEnAAIgkYABEEjAAIgkYAJEEDIBIAgZAJAEDIJKAARBJwACIJGAARBIwACIJ\nGACRBAyASAIGQCQBAyCSgAEQScAAiCRgAEQSMAAiCRgAkQQMgEgFAzbPc+tDAOBwP60P4JukC2Ac\npXZgy7Isy9L6KAA4Q6kd2L2me7JlmuwIE61rIOcukYvum/rfDxQP2NToHMzztCzTPF9+9NzDW2Ge\n2x9GD8fw4DDWEzdN/zt3Rx9pD7+NHo7h88P4yomr8av44mG0PoTn6gfsfJdr6fImvCoZ/bqcuGm6\n3A0Pbxif2zxx0/HrD5or9RlYD/645UWsZoa2eeKWZXLiOvfXiWMEBQPWcPdtwR7qwYnTsJ45cYPr\nYth6nDOnyU/rJW99cuJyPT41TtwnOvko7rGCO7AmXCqFWcv36elF58SVJ2DncTl1yMqjNhddbQIG\nz7kP9sbKg0nAvmL/teQ+CCdz0RUmYIzrpVW8+2A/bL9YCdinXr2W3AfhZC66qgSMQVnFQzoB+4ib\n4FAs5HvgouNCwBpwH2zOTRAKEDAgxtsrD6vGkgTsfVbxA3IfhH4IWBvugw1ZeUANAgYMwaqxHgGD\n17gPtmLrzA0Be9Pn15L7IMAnBIyxWMWPzKqxGAGDl7kPns/Kg3sC9o5vXUvugwBvEzBgIFaNlQgY\nAzGGgkoEDN5hIX8mKw82CdjLvnstuQ8CvEfAGIVVPBQjYMBYjD3KEDB4k/vgOWyd+YuAvca1BNCJ\nn9YH8LL5d9G7bJVkfXXzpW6tC/moQ87jNwz1hAVsnudLnK6/vvnO/UsAF1aNNRghwvt8DHY0meGB\nsB3YU/cjxPnfG8wnOzPXElDVHLgWqxOwv6aLEbNEAw2grftPZFodyX5GiNRnccA9498CwnZgy7Lc\nP4W47rc2XwKgqrCATVtxypoWUozx73H8YnnMCHGvo68lAw2AlwgYAJEEjOKMofiLsUc6AYNPuQ9C\nEwIG9MjWmacEbJdzriULeYD9BAwYl1VjNAGjMmMoKEzA4Ass5OF8AgZ0x9aZPQTsuTOvJQt5gJ0E\nDIBIAkZZxlDsYeyRS8CAvlh5sJOAwXdYyMPJBOwJi0GAPglYdyzkAfYQMGqydWY/q8ZQAgZAJAGD\nr7GQ/5ytM/sJGACRBOyRVotBC3mApwQMgEgCRkE+R+FVxh6JBAyASAIG32Qh/wlbZ14iYABEErA/\ntV0MWsgDPPbT+gBeNv/e15e7vDx4CeCxddXo5hEkLGDzPF/idP319Fuv9Ts3LzEU9yAYRFjAnrrO\n2PV3LoSNo1nIk2gO/NCiVMA2N2eKBRFUv62bW2VEzzzEsc21BNA5AeuXBxEBHggbIS7Lcv+o4Tow\n3HwJgKrCAjZtxcnHXVyY/fIJD+BkMUIEIJKAwff5/PJV9j28QcA2uJYA+idgXbOQB/iLgFGHrTMM\nRcAA/mPsEUTAAIgkYHAIC/n9zH55j4ABEEnAblkMAkQQsN6ZRAFsEjCKsHXmW6waUwgYAJEEDIBI\nAgZHMYnaw+yXtwnYP1xLACkELICFPMA9AQO4ZdUYQcCowOwXBiRgAEQSMDiQSdRjts58QsAAiCRg\n/+l5MWghD3BDwACIJGDE63nrTC5jj/4JGACRBAyASHkBm389+ANnHg88ZhL1F7NfPvTT+gBeM8/z\n8vuWv/76+g+8+//sWgJIkrcDe2AzaWVYyANcC9uBveFmT1a4cABvS/zwpU7A1t/+5Z+XUClWbWa/\nHGcdewzyBru5VUb0rE7AHn82BkAxYQFbluWyLlAsyDXOzobjhAVs2hoJ3nxHzOjNUJMoOE2ppxDf\n5uYCEEfAkniSHuBCwACIJGAEM/vlaMYePRMwACIJGACRBAzOYBJ1zeyXrxAw1xJAJAELYyEPsBIw\nACIJGKnMfjmHsUe3BAyASAIGQCQBg5OYRK3MfvkWAQMg0ugBS1wMWsgDTAIGQCgBI1Li1plcxh59\nEjAAIgkYAJEEDM5jEmX2yxcJGACRhg6YxSBArqEDlsskCkDAyGPrDEwCBrCHsUeHBAyASD+tD+Bl\n8+8qaLmbIj14CTqxLuTHfIcO+y/OQcICNs/zJU7XX1+s39l8CYBKSo0QX4qWxSBAtLAd2B4326/5\n3w9ey+zMRp5EAV83Bz6jUipg6wm4SVSZYgEc5+ZWGdGzUiPESa4GYN9JK56k703YDmxZlvtHDdeZ\n4fp9DyICDCIsYNNWmdbvKBYpfH4JX1FthAj0SbP5OgEDINKgAauxGPSRMjCyQQMGQDoBI0mNrTO5\njD26ImAARBIwACIJGDQw2iTK7JcjCBgAkUYMmMUgQAEjBqyS0SZRABcCBkAkASOG2S89MPboh4AB\nEEnAoI1xFvK2zhxEwACINFzALAYBahguYPWMM4kCuCZgAEQSMDKY/dIPY49OCBgAkQQMgEgCBs2M\nMIky++U4AgZApLECVnUxOMJCHuDGWAEDoAwBA3iZsUcPflofwDfNv2+opeSgcGBVZ7/AJ+oEbJ7n\nS7euvwagpLFGiKJGb2pPomydOVSdHdhf5n9vD3ZmAPfm25VUwK2yfsAUC+Cpm1tlxGBgrBFiYbUn\nUcCZ5nmapoAbSp0d2LIsnkIEGEedgE26VZQHAejTOvbw5mzICBGASAIGQCQBg8aqPoBjvMbRBAyA\nSAJWR9WFPMAmAQPgP0GzXwEDeJOxR1sCRteCFoPAyQQMgEgCBu3Vm0TZOnMCAQMgkoABEEnASqk3\niQLOlDX7FTAAIgkY/cpaDDImY4+GBAyASAIGfJmtM+cQMOiCSRS8SsAAiCRg1VjIA4MQMACmKfDD\nSwGjU3HXEsMy9mhFwACIJGAARBIw6EWNSZTZL6cRMAAiCRgAkX5aH8DL5t8hy3I3p3jw0lDWSdTY\nvwPgNYk3jbCAzfN8idP11xfrdzZfAqCSUiNE0SojcTHIyGo8gBMnbAe2x832a/73bSVyAJvmtAh3\nHbBX27P++Zs/plgESf/8MvrgebD071PXAXujPXIFMIiuA3ZvWZb7Rw3XmeH6fQ8iAgwiLGDTVpnW\n7yjWtfRJFHCm0NtFqacQARiHgNGd0MUgg/Mk/fkEDIBIAgZ8h60zJxMw6ItJFOwkYABEErCyLOSB\nPXJnvwIGQCQBoy+5i0Ew9jiZgAEQScAAiCRg0J3ESZTZL+cTMAAiCRjAuKK3zgJWWeIkCmAnAaMj\n0YtBmKwazyVgAEQSMAAiCRj0KGsSZfZLEwIGQCQBAxhU+tZZwIrLmkQB7Cdg9CJ9MQgrq8bTCBgA\nkQQMgEgCBp1KmUSZ/dKKgAEQScDqS1nIA7wkL2Dzrwd/4MzjAUhUYPb70/oAXjPP8/L7K7/++voP\nnH5QfEGBawku1rGHt/TRwgL22Jq0m4bd/M/75gGQuPovFbBNigXw1M2tMqJnXQfspc3T+ocv/9Qt\nCuh/EtX54VFb1wF7KUKPPxsDoJiuA3bv+iMuxdqv/4U8cKYaN4SwgE1b27Kb74gZwAjy/jsw6qmx\nGIRr/gKBEwgYAJEEDIBIAgZd63kSZfZLWwIGQCQBG0XPC3ngTGW2zgIGQCQBo7Eyi0G4YexxNAED\nIJKAAe+wdaY5AYPemUTBJgEDIJKADcRCHqg0+xUwgKNYNR5KwGip0mIQOJmAARBJwCBAb5MoW2d6\nIGAARBKwsfS2kAfOVGzrLGAARBIwmim2GIRNxh7HETAAIgkYZOhnIW/rTCcEDIBIAjacfhbywJnq\nbZ0FDIBIAkYb9RaD8Bdjj4P8tD6Al82/b4Tl7v734CUAigkL2DzPlzhdfz391mv9zs1LUMO6kG/7\n1m5+AHARFrCnrjMGQGGlAra5OZv/nT1rGzCgp1vnOfBjuq4D9pX2KNa9HiZRQFdubpURPes6YNpT\nlXwyGqvGI3QdsHvLstw/argODDdfAqCqsIBNW3G6fEe3KK/tQt4egq74D5kBiCRgAMVV3ToL2KD8\n3TZAOgHjbFUXg/CYVePXCRgAkQQMwrRayNs60xsBAyCSgI3LRB5GUHjrLGAAJ7Fq/C4B41SFF4PA\nyQQM8ljIwyRgwB62znRIwIZmIQ/kEjCAsmpvnQWM89S+lmAPY48vEjAAIgkYRDpzIW/rTJ8EDIBI\nAjY6E3moqvzWWcA4SflrCXayavwWAQMgkoBBqnMW8rbOdEvAAAoaYeUhYJjIA5EEjDOMsBiE/awa\nv0LAAIgkYBDs6IW8rTM9+2l9AC+bf6/XZevCWl/dfIkH1vugXxvUMMjlHBaweZ4vcbr++uY79y8B\nUIwRIocbZDEIL/Ecx+fCdmBP3Y8Q53/fI3ZmFHPc+NfKYyhzYE67DthL7flruqhYAE/dfyLT6kj2\n6zpg2nMmz3FADeNcyF0H7N6yLPdPIa77rc2XaG6cawleZdX4obCATVtxMi1kcO6DjMlTiMAGRaR/\nAsZ/PNcL6YZaeQgYBxrqWoI3WDV+QsAAiCRgUMF3F/K2zkQQMP5hoAG5Rlt5CBhHGe1aAk4mYMA/\nrDxOZuzxNgGDItwHGY2Acct9EBINuHUWMA4x4LUEb7NqfI+AQR2f3wetPAgiYGywHoQsY648BIzv\nG/Nagk9YNb5BwKAU90HGIWBscx8ckK0zWQSML3MThPe8vWoc9qITMKjmvfvgsDdBcgkYfzJFhP6N\nvPIQML5p5GsJPmfV+BIBg4JevQ9aeZBIwHjkpfugmyCcbPCLTsCgpv2Lj8Fvgr0xRdxPwHjC5QQn\n23nRWXkIGN/hWuqQxQe1CRjPuQ8WZuXRJxfdHgJ2hrmPd+Inh/H4ctp/Eyzwq/iiEw7j6YmbplF+\nFRHHMO0+DCuPqWTAOnkXjsOFFMqJ69yDxYdztyoVsHme1es4m5eTC6l/Tlw05+6Bn9YH8E3Lskx2\nYEe6vhWuX7uQIjhxodbT5Nz9ZV7K/TLm+b9/KTE70tLJxye8yIljl/7rkLoDuynTX7/o/k9AOL/e\nUE4cFaQGTJkABlfqIQ4AxlHwMzAARmAHBkCk1M/A9rg86NFwl7n5SOSZx3PzQ3s4hlaHcfnRDX8V\n07/PHy3LMuwZuX8Oq+0Z6eo6bXIYDy6NHu6lm8oG7KYcTW6UnRzP9Zuy+TGs96lWh3E5KW3fHps/\n+rTDuH4nNDwj1z9l/SsImpyR67t2q1/F/Q89+TCub1bND+YlRohHWZalhzPtGC76ufZ6+CtjbprR\n9kh6OIyRdXKzekPZHRjXLiuphgfQ6kd3qPlfGdPtgrqJm+FE8yNhPwErrpOLs/kte/3R1/9spfm5\n6ErbgnYyHLssLtuuMhMZIdbX9o7ZyQW5/JpaP9TT6kfTrbWdVjZvqDxA6OHJmbZPIXbyoFdXDzX1\n8xSi50JvNj2D/yqaH0biU4iVAwZAYUaIAEQSMAAiCRgAkQQMgEgCBkAkAQMgkoABEEnAAIgkYABE\nEjAAIgkYAJEEDIBIAgZAJAEDIJKAARBJwACIJGAARBIwACIJGACRBAyASAIGQCQBAyCSgAEQScAA\niCRgAEQSMAAiCRgAkQQMgEgCBkAkAQMgkoABEEnAAIgkYABEEjAAIv0/uC0socxAfvgAAAAASUVO\nRK5CYII=\n"
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Numpy basics\n",
      "\n",
      "Numpy is a Python library which provides matrix manipulation capabilities and other standard numerical algorithms. You can load it under the `np` namespace with: "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Creating arrays\n",
      "\n",
      "You can create a vector from an existing tuple:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = np.array([1,2,3,4])\n",
      "print x\n",
      "print x.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[1 2 3 4]\n",
        "(4,)\n"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Similarly, you can define a numpy matrix as:\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X = np.array([[1,2], [3,4]])\n",
      "print X\n",
      "print X.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[1 2]\n",
        " [3 4]]\n",
        "(2, 2)\n"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can also easily generate points on an interval through [linspace](http://docs.scipy.org/doc/numpy/reference/generated/numpy.linspace.html) or [logspace](http://docs.scipy.org/doc/numpy/reference/generated/numpy.logspace.html):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = np.linspace(0, 10, 25)\n",
      "print x"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[  0.           0.41666667   0.83333333   1.25         1.66666667\n",
        "   2.08333333   2.5          2.91666667   3.33333333   3.75         4.16666667\n",
        "   4.58333333   5.           5.41666667   5.83333333   6.25         6.66666667\n",
        "   7.08333333   7.5          7.91666667   8.33333333   8.75         9.16666667\n",
        "   9.58333333  10.        ]\n"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The standards matrices can be generated with:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.zeros((5,5))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 17,
       "text": [
        "array([[ 0.,  0.,  0.,  0.,  0.],\n",
        "       [ 0.,  0.,  0.,  0.,  0.],\n",
        "       [ 0.,  0.,  0.,  0.,  0.],\n",
        "       [ 0.,  0.,  0.,  0.,  0.],\n",
        "       [ 0.,  0.,  0.,  0.,  0.]])"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.ones((5,5))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        "array([[ 1.,  1.,  1.,  1.,  1.],\n",
        "       [ 1.,  1.,  1.,  1.,  1.],\n",
        "       [ 1.,  1.,  1.,  1.,  1.],\n",
        "       [ 1.,  1.,  1.,  1.,  1.],\n",
        "       [ 1.,  1.,  1.,  1.,  1.]])"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.eye(5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 19,
       "text": [
        "array([[ 1.,  0.,  0.,  0.,  0.],\n",
        "       [ 0.,  1.,  0.,  0.,  0.],\n",
        "       [ 0.,  0.,  1.,  0.,  0.],\n",
        "       [ 0.,  0.,  0.,  1.,  0.],\n",
        "       [ 0.,  0.,  0.,  0.,  1.]])"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Indexing\n",
      "\n",
      "As opposed to Matlab, Numpy is 0-indexed. Indexing can be performed efficiently through *slicing*."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X = np.random.randn(4,4)\n",
      "print X"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[-0.3391406   0.62073092 -0.2864898   1.05243832]\n",
        " [ 0.92636541  0.78939298  0.00748023  0.8718698 ]\n",
        " [ 0.01819851 -0.15747068 -0.07151959  0.58040635]\n",
        " [-1.76144683  0.1659408  -0.9622996  -0.65804837]]\n"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Selecting all rows except the first and all columns:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X[1:,:]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "array([[ 0.92636541,  0.78939298,  0.00748023,  0.8718698 ],\n",
        "       [ 0.01819851, -0.15747068, -0.07151959,  0.58040635],\n",
        "       [-1.76144683,  0.1659408 , -0.9622996 , -0.65804837]])"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Select columns 2 and 0 and all rows:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X[:,[2,0]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 22,
       "text": [
        "array([[-0.2864898 , -0.3391406 ],\n",
        "       [ 0.00748023,  0.92636541],\n",
        "       [-0.07151959,  0.01819851],\n",
        "       [-0.9622996 , -1.76144683]])"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`-1` (minus one) stands for the last element in a given dimension"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X[-1,:]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 23,
       "text": [
        "array([-1.76144683,  0.1659408 , -0.9622996 , -0.65804837])"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "A powerful indexing feature can also be achieved through *logical indexing*. You can create a boolean array from an ndarray:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X < 0.5"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 24,
       "text": [
        "array([[ True, False,  True, False],\n",
        "       [False, False,  True, False],\n",
        "       [ True,  True,  True, False],\n",
        "       [ True,  True,  True,  True]], dtype=bool)"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The boolean ndarray can then be used for indexing:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X[X < 0.5]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 25,
       "text": [
        "array([-0.3391406 , -0.2864898 ,  0.00748023,  0.01819851, -0.15747068,\n",
        "       -0.07151959, -1.76144683,  0.1659408 , -0.9622996 , -0.65804837])"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Numpy for Matlab users\n",
      "\n",
      "If you know Matlab already, make sure to look at the cheatsheet [Numpy for matlab users](http://mathesaurus.sourceforge.net/matlab-numpy.html)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Numpy I/O\n",
      "\n",
      "\n",
      "You can import and export data from a variety of formats. For example, you can save a numpy array to a text format using [np.savetxt](http://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X = np.random.normal(size=(5,5))\n",
      "np.savetxt('data.txt', X)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 26
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%cat data.txt"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "8.156678852559566817e-01 -3.131124534131186632e-01 8.337257398597310853e-01 9.214720052339393508e-01 1.085875976924086217e+00\r\n",
        "-5.035982362246711475e-01 -1.786739101987857847e-01 -8.520783372461552263e-01 2.078405838941234929e-01 1.096214780850903514e+00\r\n",
        "-6.567926252898417250e-01 -7.454343453128293717e-01 -1.143679949305732579e+00 3.589055086963070518e-01 1.382764023257532227e+00\r\n",
        "4.030407399756600895e-01 -4.347724112286717180e-01 -3.709554624484467933e-02 -1.868833136961473906e+00 1.845541086337965653e-01\r\n",
        "-9.785313966063215185e-02 8.348666702977869392e-01 -1.781600107757391305e+00 -2.223242216704917340e-01 1.715788698981550509e+00\r\n"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X = np.loadtxt('data.txt')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 28
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Unless you want your data files to be human readable in text form, it is usually preferable to use [http://docs.scipy.org/doc/numpy/reference/generated/numpy.save.html](http://docs.scipy.org/doc/numpy/reference/generated/numpy.save.html) rather than `np.savetxt`. The array will then be saved in a more compact Numpy (`.npy`) format."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Dealing with matfiles\n",
      "\n",
      "If you can export Matlab matrices in versions v4 (Level 1.0), v6 or v7 to 7.2, then you would be able to use [scipy.io.loadmat](http://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html) to load your data in numpy."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%matlab\n",
      "x = magic(4);\n",
      "save('test.mat', 'x')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 29
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from scipy.io import loadmat\n",
      "print loadmat('test.mat')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{'x': array([[16,  2,  3, 13],\n",
        "       [ 5, 11, 10,  8],\n",
        "       [ 9,  7,  6, 12],\n",
        "       [ 4, 14, 15,  1]], dtype=uint8), '__version__': '1.0', '__header__': 'MATLAB 5.0 MAT-file, Platform: MACI64, Created on: Mon Sep 22 23:21:18 2014', '__globals__': []}\n"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The corresponding [scipy.io.savemat](http://docs.scipy.org/doc/scipy/reference/generated/scipy.io.savemat.html) is also available."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Scikit-learn\n",
      "\n",
      "Scikit-learn offers a variety of classification, regression and clustering algorithms under a unified interface. If you decided to install Anaconda, it should already be available. [Standard datasets](http://scikit-learn.org/stable/datasets/) are shipped with Scikit. They can be quite useful for debugging your algorithm. We will load the *iris* dataset with:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn import linear_model, datasets\n",
      "\n",
      "iris = datasets.load_iris()\n",
      "X = iris.data[:,:]\n",
      "Y = iris.target"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 31
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Scikit-learn provides different [cross-validation](http://scikit-learn.org/stable/modules/cross_validation.html) techniques. In the following `StratifiedKFold` is an iterator which outputs pairs of indices for the train and test instances. You then use these indices to slice your dataset. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.cross_validation import StratifiedKFold\n",
      "from sklearn.metrics import mean_squared_error\n",
      "\n",
      "skf = StratifiedKFold(Y, 3)\n",
      "logreg = linear_model.LogisticRegression(C=1e5)\n",
      "\n",
      "for train, test in skf:\n",
      "    ypred = logreg.fit(X[train], Y[train]).predict(X[test])\n",
      "    print mean_squared_error(Y[test], ypred)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "0.0196078431373\n",
        "0.117647058824\n",
        "0.0\n"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The above loop can also be written more succinctly using [cross_val_score](http://scikit-learn.org/stable/modules/cross_validation.html#computing-cross-validated-metrics). By specifying `n_jobs=k`, you can compute the cross-validation loop in parallel in `k` processes. If `k=-1`, all CPUs are used."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.cross_validation import cross_val_score\n",
      "cross_val_score(logreg, X, Y, cv=skf, scoring='mean_squared_error', n_jobs=-1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 33,
       "text": [
        "array([-0.01960784, -0.11764706, -0.        ])"
       ]
      }
     ],
     "prompt_number": 33
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Model selection\n",
      "\n",
      "Using cross-validated scores, you can perform model selection efficiently through [GridSearchCV](http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.GridSearchCV.html#sklearn.grid_search.GridSearchCV). In the following, we will load the digits dataset and performe grid search over the parameters of an SVM classifier."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.svm import SVC\n",
      "from sklearn.cross_validation import train_test_split\n",
      "from sklearn.grid_search import GridSearchCV\n",
      "from sklearn.metrics import classification_report\n",
      "\n",
      "digits = datasets.load_digits()\n",
      "n_samples = len(digits.images)\n",
      "X = digits.images.reshape((n_samples, -1))\n",
      "y = digits.target\n",
      "\n",
      "X_train, X_test, y_train, y_test = train_test_split(\n",
      "    X, y, test_size=0.5, random_state=0)\n",
      "\n",
      "tuned_parameters = [{'kernel': ['rbf'], 'gamma': [1e-3, 1e-4],\n",
      "                     'C': [1, 10, 100, 1000]},\n",
      "                    {'kernel': ['linear'], 'C': [1, 10, 100, 1000]}]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 34
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`GridSearchCV` takes an estimator (regression, classification or clustering), a dictionary of parameters to try for that particular estimator, a cross-validation iterator and a scoring function. Once again, the parameter search can be performed in parallel with `n_jobs=-1`. The `fit` function of `GridSearchCV` then scores every possible combination of the `tuned_parameters` and saves the best one in the `best_estimator_` attribute."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "skf = StratifiedKFold(Y, 5)\n",
      "clf = GridSearchCV(SVC(C=1), tuned_parameters, cv=skf, n_jobs=-1)\n",
      "clf.fit(X_train, y_train)\n",
      "print clf.best_estimator_\n",
      "print \"Grid scores on train set:\"\n",
      "for params, mean_score, scores in clf.grid_scores_:\n",
      "    print(\"%0.3f (+/-%0.03f) for %r\"%(mean_score, scores.std() / 2, params))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "SVC(C=10, cache_size=200, class_weight=None, coef0=0.0, degree=3,\n",
        "  gamma=0.0001, kernel='rbf', max_iter=-1, probability=False,\n",
        "  random_state=None, shrinking=True, tol=0.001, verbose=False)\n",
        "Grid scores on train set:\n",
        "0.933 (+/-0.018) for {'kernel': 'rbf', 'C': 1, 'gamma': 0.001}\n",
        "0.767 (+/-0.018) for {'kernel': 'rbf', 'C': 1, 'gamma': 0.0001}\n",
        "0.933 (+/-0.018) for {'kernel': 'rbf', 'C': 10, 'gamma': 0.001}\n",
        "0.940 (+/-0.012) for {'kernel': 'rbf', 'C': 10, 'gamma': 0.0001}\n",
        "0.933 (+/-0.018) for {'kernel': 'rbf', 'C': 100, 'gamma': 0.001}\n",
        "0.940 (+/-0.012) for {'kernel': 'rbf', 'C': 100, 'gamma': 0.0001}\n",
        "0.933 (+/-0.018) for {'kernel': 'rbf', 'C': 1000, 'gamma': 0.001}\n",
        "0.940 (+/-0.012) for {'kernel': 'rbf', 'C': 1000, 'gamma': 0.0001}\n",
        "0.940 (+/-0.012) for {'kernel': 'linear', 'C': 1}\n",
        "0.940 (+/-0.012) for {'kernel': 'linear', 'C': 10}\n",
        "0.940 (+/-0.012) for {'kernel': 'linear', 'C': 100}\n",
        "0.940 (+/-0.012) for {'kernel': 'linear', 'C': 1000}\n"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The classification performance of the best estimator can then be reported with [classification_report](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html#sklearn.metrics.classification_report)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "y_true, y_pred = y_test, clf.predict(X_test)\n",
      "print(classification_report(y_true, y_pred))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "             precision    recall  f1-score   support\n",
        "\n",
        "          0       1.00      1.00      1.00        89\n",
        "          1       0.96      1.00      0.98        90\n",
        "          2       1.00      0.99      0.99        92\n",
        "          3       0.99      0.99      0.99        93\n",
        "          4       1.00      1.00      1.00        76\n",
        "          5       0.97      0.97      0.97       108\n",
        "          6       0.99      0.99      0.99        89\n",
        "          7       0.99      1.00      0.99        78\n",
        "          8       0.99      0.96      0.97        92\n",
        "          9       0.99      0.98      0.98        92\n",
        "\n",
        "avg / total       0.99      0.99      0.99       899\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 36
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Dealing with large parameter sets\n",
      "\n",
      "It might happen that the set of parameters over which you would hope to run `GridSearchCV` would be too large. We could then decide to put a prior or the parameters and perform a stochastic search. [RandomizedSearchCV](http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.RandomizedSearchCV.html#sklearn.grid_search.RandomizedSearchCV) supports any of the standard distribution from [scipy.stats](http://docs.scipy.org/doc/scipy/reference/stats.html) as priors. In this example, we will set an exponential priors over the `C` and `gamma` parameters."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import scipy.stats\n",
      "from sklearn.grid_search import RandomizedSearchCV\n",
      "\n",
      "tuned_parameters = {'C': scipy.stats.expon(scale=100), 'gamma': scipy.stats.expon(scale=.1),\n",
      "  'kernel': ['rbf'], 'class_weight':['auto', None]}\n",
      "\n",
      "clf = RandomizedSearchCV(SVC(C=1), tuned_parameters, n_iter=10, n_jobs=-1)\n",
      "clf.fit(X_train, y_train)\n",
      "print clf.best_estimator_\n",
      "y_true, y_pred = y_test, clf.predict(X_test)\n",
      "print(classification_report(y_true, y_pred))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "SVC(C=77.5702758756, cache_size=200, class_weight='auto', coef0=0.0, degree=3,\n",
        "  gamma=0.00989451451095, kernel='rbf', max_iter=-1, probability=False,\n",
        "  random_state=None, shrinking=True, tol=0.001, verbose=False)\n",
        "             precision    recall  f1-score   support\n",
        "\n",
        "          0       1.00      0.78      0.87        89\n",
        "          1       1.00      0.63      0.78        90\n",
        "          2       1.00      0.57      0.72        92\n",
        "          3       1.00      0.72      0.84        93\n",
        "          4       0.19      1.00      0.32        76\n",
        "          5       1.00      0.27      0.42       108\n",
        "          6       1.00      0.78      0.87        89\n",
        "          7       1.00      0.91      0.95        78\n",
        "          8       1.00      0.29      0.45        92\n",
        "          9       1.00      0.68      0.81        92\n",
        "\n",
        "avg / total       0.93      0.65      0.70       899\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Fancier model selection\n",
      "\n",
      "[Hyperopt](https://github.com/hyperopt/hyperopt) goes beyond the randomized and grid search methods and provides fancier *hyperparameters* optimization. This package is quite popular in the neural net communities. "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Plotting the confusion matrix\n",
      "\n",
      "The confusion matrix is a useful visual tool to asess the classification performance. In the ideal case, you would see only entries on the diagonal. In this case, every label would have been predicted perfectly."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.metrics import confusion_matrix\n",
      "cm = confusion_matrix(y_true, y_pred)\n",
      "plt.matshow(cm)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 38,
       "text": [
        "<matplotlib.image.AxesImage at 0x10dfdb410>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAPYAAAD7CAYAAABZjGkWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAACZZJREFUeJzt3V+MXGUZgPFnaEv/uLEYb0ylcRqURBMDNIhNlXRruMCG\nQExMrIFINDFeVCmaoOAN20tNSDUREgUkbShiLIiSYIzE7qKSFJC2QlvUYkdomwZSoGS5IC0dL84h\n3bXbPWd25ptv5t3nl5zs7PbkzJtmnz3z5+y3IEmSJEmSJEmSJKmGa4GXgH8DP8g8Sx0rgV3AfuBF\n4Ja849S2ANgDPJ57kJouAnYCB4EDwJq849RyB8X3xQvAQ8DivOPkswA4BDSBRcBe4JM5B6rhI8Dl\n5e0R4J8M/swA3wN2AL/PPUhN24BvlLcXAsszzlJHE/gPZ2P+NXBztmnO44I+3c9VFGG3gFPAw8AN\nfbrvuTpO8QMIYJLijLIi3zi1XAxsAO4DGplnqWM5cDXwy/Lz08DJfOPU8jbF9/Ayih9Ey4CjWSea\nQb/C/ijw6pTPj5RfGxZN4Apgd+Y5qmwFbgPO5B6kplXA68ADwPPAvRShDLI3gLuAV4BjwFvAk1kn\nmkG/wm736X5SGKF4DriZ4sw9qK4DXqN4fj0MZ2soznirgXvKj+8At2edqNolwK0UP+xXUHx/3Jhz\noJn0K+yjFC9GvW8lxVl70C0CHgEeBB7LPEuVtcD1wGHgV8AXgO1ZJ6p2pNyeLT/fSRH4ILsSeBo4\nQfHU4VGK//t5aSHwMsVPuQsZjhfPGhRhbM09yBysY3heFX8KuLS8PQb8KN8otVxG8S7JUorvkW3A\npqwTZfZFileWD1G8XTDoPk/xXHUvxcPbPRRv2Q2DdQzPq+KXUZyx91Gc/Qb9VXGA73P27a5tFI/s\nJEmSJEmSJElSFl1fobRuMe2Jd3sxiqTOfQz47zkd9+LSw3Z7Vf2dx96EsQ9V79c4fOfcJ+qpcWA0\n8wzA2Fj9fXeNwfqa+3dy3KTGGYj/59rGGYx5t8AMHffrklJJfWTYUkB9D3t0Sb/vsVvN3AN0rjma\ne4I5aOYeoEPN3APMqv9hL+33PXarmXuAzq0azT3BHDRzD9ChZu4BZlUn7GFbq0ya96rCXgD8jCLu\nTwFfZfB/3VKa96rCHsa1yqR5ryrsYV+rTJqXqsIe5rXKpHlrYcW/11qrbOzNs7dHlwzjK9/SsGiV\n2+yqwn4O+ATFa/vHgK9QvIA2TZ1LRCX1QpPpb7VNzLhXVdingW8Df6R4hfx+ioXzJQ2wqrAB/lBu\nkoaE14pLARm2FJBhSwEZthSQYUsBGbYUUE/WPIPer0/W/t2Wnh/zfY0bBmU9tfquaq9PctxnGruS\nHFf94ppn0rxh2FJAhi0FZNhSQIYtBWTYUkCGLQVk2FJAhi0FZNhSQIYtBWTYUkCGLQVk2FJAhi0F\nZNhSQIYtBWTYUkCGLQVk2FJAhi0FZNhSQHX+2mYWKZcIPnMizdLGF3w43cwbeCLJcZ9haZLjKi/P\n2FJAhi0FZNhSQIYtBWTYUkCGLQVUJ+yVwC5gP/AicEvSiSR1rc772KeA7wJ7gRHg78CfgIMJ55LU\nhTpn7OMUUQNMUgS9ItlEkrrW6XPsJnAFsLv3o0jqlU4uKR0BdgKbKc7cU4xPud0sN0m91yq32dUN\nexHwCPAg8Ni5/zxa8zCSutNk+olzYsa96jwUbwD3AweAn3Q5laQ+qBP254CbgPXAnnK7NuVQkrpT\n56H4X/FCFmmoGKwUkGFLARm2FJBhSwEZthSQYUsBDewqpSmlWk20/dk0q58CNPa1Ex15LNFxlZNn\nbCkgw5YCMmwpIMOWAjJsKSDDlgIybCkgw5YCMmwpIMOWAjJsKSDDlgIybCkgw5YCMmwpIMOWAjJs\nKSDDlgIybCkgw5YCMmwpIMOWApqXyw+n0tidZlljgIcvbyQ57kbSzax8PGNLARm2FJBhSwEZthSQ\nYUsBGbYUUN2wFwB7gMcTziKpR+qGvRk4AKT6W66SeqhO2BcDG4D7gDRXSUjqqTphbwVuA84knkVS\nj1RdUnod8BrF8+vR8+82PuV2s9wk9V6r3GZXFfZa4HqKh+JLgA8C24GvTd9ttNPpJM1Jk+knzokZ\n96p6KP5DYCWwCtgI/JlzopY0aDp9H9tXxaUh0MmvbU5wvvO+pIHilWdSQIYtBWTYUkCGLQVk2FJA\nhi0F5CqlQ+JgouOOTG5KdGSYHLk72bE1O8/YUkCGLQVk2FJAhi0FZNhSQIYtBWTYUkCGLQVk2FJA\nhi0FZNhSQIYtBWTYUkCGLQVk2FJAhi0FZNhSQIYtBWTYUkCGLQVk2FJArlI6JLaMJfpDpyNjaY4L\ntFdtSXLcxuE7kxw3Es/YUkCGLQVk2FJAhi0FZNhSQIYtBVQn7IuAnRR/F+4AsCbpRJK6Vud97J8C\nTwBfLvf/QNKJJHWtKuzlwNXAzeXnp4GTSSeS1LWqh+KrgNeBB4DngXuBZamHktSdqrAXAquBe8qP\n7wC3px5KUneqHoofKbdny893MmPY41NuN8tNUu+1ym12VWEfB14FLgX+BVwD7D93t9GORpM0V02m\nnzgnZtyrzqvi3wF2ABcCLwNf724wSanVCXsf8JnUg0jqHa88kwIybCkgw5YCMmwpIMOWAjJsKSDD\nlgJy+eFhMXYk9wQdS7VM8MnFaZY1Blj+boyljT1jSwEZthSQYUsBGbYUkGFLARm2FJBhSwEZthSQ\nYUsBGbYUkGFLARm2FJBhSwEZthSQYUsBGbYUkGFLARm2FJBhSwEZthSQYUsBuUrpkGh/aWWS4zZ+\nO3yrciZdSfRQomN/PN3KqjPxjC0FZNhSQIYtBWTYUkCGLQVk2FJAdcK+A9gPvAA8BCxOOpGkrlWF\n3QS+CawGPg0sADYmnklSl6ouUHkbOAUsA94rPx5NPZSk7lSdsd8A7gJeAY4BbwFPph5KUneqztiX\nALdSPCQ/CfwGuBHYMX238Sm3m+Umqfda5Ta7qrCvBJ4GTpSfPwqs5ZywRzsaTdJcNZl+4pyYca+q\nh+IvAWuApUADuAY40PVskpKqCnsfsB14DvhH+bVfJJ1IUtfq/Nrmj8tN0pDwyjMpIMOWAjJsKSDD\nlgIybCkgw5YCMmwpoEYPjtGG4VvCdtiMTG5KctzJkbuTHFfTtb+VZvnhxs+LD///dc/YUkCGLQVk\n2FJAhi0FZNhSQIYtBWTYUkCGLQVk2FJAhi0FZNhSQIYtBWTYUkCGLQVk2FJAhi0FZNhSQBnCbvX/\nLrvSyj1Ax04/9bfcI8xBK/cAHWrlHmBWhl2plXuAjr33F8NOr5V7gFn5UFwKyLClgHqxSuk4sK4H\nx5HUuQlgNPcQkiRJkiRJkiTl8z+agjHt0pU8ygAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10dce9510>"
       ]
      }
     ],
     "prompt_number": 38
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Generating an ROC curve"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Wee can compute the Receiver Operating Characteristic (ROC) curve to assess the quality of a binary classifier. In the following, we will use a synthetic binary classification task from the Hastie textbook. It consists of 12000 instances, each with 10 Gaussian features. We will make use of the probabilistic interpretation of logistic regression in order to produce the ROC curve. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.datasets import make_hastie_10_2 \n",
      "from sklearn.linear_model import LogisticRegression\n",
      "\n",
      "X, y = make_hastie_10_2()\n",
      "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=0)\n",
      "\n",
      "tuned_parameters = [{'penalty': ['l1', 'l2'],\n",
      "                     'C': [1, 10, 100, 1000]}]\n",
      "\n",
      "logreg = LogisticRegression()\n",
      "clf = GridSearchCV(logreg, tuned_parameters)\n",
      "clf.fit(X_train, y_train)\n",
      "clf.best_score_"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 39,
       "text": [
        "0.49083333333333334"
       ]
      }
     ],
     "prompt_number": 39
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The [roc_curve](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html) function takes as inputs the true labels for the binary classification as well as the probability estimate of the dominant class. We use `np.max` with `axis=1` to compute the max across the columns (the second axis)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.metrics import roc_curve\n",
      "y_pred = clf.predict_proba(X_test)\n",
      "y_score = np.max(y_pred, axis=1)\n",
      "fpr, tpr, th = roc_curve(y_test, y_score)\n",
      "plt.plot([0, 1], [0, 1], 'k--')\n",
      "plt.xlabel('False Positive Rate')\n",
      "plt.ylabel('True Positive Rate')\n",
      "plt.plot(fpr, tpr)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 40,
       "text": [
        "[<matplotlib.lines.Line2D at 0x10df08f10>]"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VGXax/EvhCC9Y6OpiPiqCwtKUVGDshRdRVmKVFHE\nwtoBRV2BdQEbICJroQi4urAC7msEBRHNK6voCpnQFKQu0hSBBQTCJGTeP+4ZM8SUSTJnzpTf57py\nMeXMmdsjnDtPux8QERERERERERERERERERERERERERGJWW8APwBrCzlmMrAJWA20iERQIiLinquw\nm31BieF64AP/4zbAl5EISkRE3HUOBSeG14BeQc83AGc4HZCIiBSsrMvfXw/4Puj5TqC+S7GIiAju\nJwaAMnme+1yJQkREACjn8vfvAhoEPa/vf+0UjRs39m3ZsiViQYmIxIktwPnF/ZDbLYZUYID/cVvg\nv9gsplNs2bIFn8+nH5+PUaNGuR5DtPzoWuhaJPq1yMnx8cEHPgYN8nHGGScAH8nJh/noox/w+XwA\njUtyY3a6xTAHuAaog40ljAKS/e+9js1Iuh7YDBwFbnc4HhGRuHD8OFSqZI/PP38Hx449x4QJ1/LQ\nQ90oW7Zqqc7tdGLoHcIx9zkcg4hIXHn+eXjsMahdG374AVas2MF55z3J2WefHZbzu92VJMWUkpLi\ndghRQ9cil65Frni+FgcOQPv2lhRuuw327IGkJGjXrl3YkgL8ekZQtPL5+8tERBKKzwfTpsGLL8KG\nDfbavHnQvXvRny1TpgyU4D6vxCAiEoUOHoQHH4S//c2ed+2aQ8OGL3HuufDwww+HdI6SJgZ1JYmI\nRJHFi6FuXahVC/7+d3jlFVi1ysO2bS3YunUZvXr1KvokpaTEICISBbKzYdAg6NIFLrsMtmyBw4e9\n7N07is6dOzF06FDef//9sI4lFMTtBW4iIgntxx9h9Gh49VV7/tZb0LevPR4y5CF27NhBRkZGRBJC\ngMYYREQibMUKSwYffWTPa9eGhx6CJ5+EMkF35SNHjlClSpXAWEGxlXSMQS0GEZEI8Xph8GB48024\n/HJLDO3aQcWK+R9ftWrpFqqVlBKDiEgEbN0Kjf0FKt55B3r0yH3P6/Vy5MgRateu7U5weWjwWUTE\nQYcOQc+elhROO81WKgcnBY/HQ6tWrXjllVfcCzIPJQYRkTA7edK6iW67DWrUsAVpEydafaPTT7dj\nvF4vo0aNolMnm3H0pz/9yd2gg6grSUQkDFauhNmzYfVqWL7cXrvxRli6FK677tRBZY/Hw8CBA2nQ\noEHEZxyFQrOSRERK4cABuP12SE2F6tVh5Ei44gpo3RrKFtAnM3HiROrUqUP//v1LPOMoFCqJISIS\nQd9+a+sNPB6oXBkWLoRoq9+nkhgiIhGQk2PjBRddZPWMPB74+efoSwqlocQgIhKC776DCy+0MtdD\nh8L998O2bfDb3xb8GY/Hw6effhq5IMNEiUFEpAg//ghNm0JmJqxda3WNJk8u+PjgGUf79++PXKBh\nollJIiIF+OEHG1j+8EMoVw42b7Y/CxPtM45CoRaDiEg+jh+HJk3gyy9tGqrXW3RSmDJlyi/rEiJV\nCdUJmpUkIuJ38qS1Ej7/3FYrJyfDiROnrkEozKpVqzjrrLOiJiFouqqISCm89Rb072+Pa9aEP/wB\nXn+94LUIsUDTVUVESuDAAWsd9O8PDzxgA8wHDtg+y7GcFEpDg88iknCys6FNG1uD4PNBvXowa5bV\nNiqK1+tl7NixlC1bllGjRjkeqxuUGEQkoezaZSUrduywQndXXQUVKoT22eAZR1OnTnU2UBclaENJ\nRBJJZqbtjlapEtSvbzOMNm2C3/0utKSQtxJqLM84CoVaDCISt3Jy4L33YNQoW5g2ciQ88YTti1Ac\nTz75JN9++23MrksoLs1KEpG4c+QILF4MkybBF19A794wfDi0aFGy8x0/fpwKFSo4WgnVCdrzWUQS\nWk4OvP02jB5t22jWrm37ICxfbvsql0bFgjZljlNKDCIS01JTrWUQqFWXkgLvvgvNmoW+MC3A6/Vy\n4MABzjzzzLDHGUs0+CwiMWnRImjVCrp2tVIV775rrYZPP4XmzYufFAJ7L08urDpeglCLQURiyo4d\ncP31sH69lbxevBg6dSr5+QLrEl599VXGjx9P/8Dy5wSmxCAiMSEry8YLOnSwRWkff2xjCKURD5VQ\nnRArQ+yalSSSgHw+GDMGpk6FnTutREXnzjBzJpx+eunPP336dMqXL+/43stuURE9EYkLPp/NKlq6\nFO69114bP96K2p1zjquhxRwlBhGJee+/DzffbIPIjRvDDTdYUkhOdjuy2KTqqiISs0aNsllEN91k\nM41++sl2S3vppfAkBY/Hw8KFC0t/ogThdGLoDGwANgGP5fN+HWAxkAGsAwY6HI+IRBGfD555Bp5+\n2vZDOHnSdkyrXTs85w+ucXT06NHwnDQBODkrKQmYAnQAdgFfA6nAt0HH3Ad4gMexJLEReAvIdjAu\nEXHZTz/BsmXwyivw2WcweTL07Rve79CMo5JzssXQGtgMbAeygLlA1zzH7AGq+R9XA/ajpCASlxYt\nslXJZ5wBdevaxjiNGtm4wv33h/e7pk6dmjCVUJ3gZIuhHvB90POdQJs8x0wDPgF2A1WBng7GIyIu\n2LYNuneH9HTo0sVqGd14o5W/dkq7du3USigFJxNDKNOInsDGF1KAxsBSoDlwJO+Bo0eP/uVxSkoK\nKSkpYQhRRJzw3//abKKxY+35pZdCRoaVqoiEiy66KDJfFGXS0tJIS0sr9XmcnK7aFhiNDUCDjSPk\nAM8FHfMBMBb43P98GTZIvTLPuTRdVSTKHTgAEyfCnDm2DgHgscdgxAioUcO57/X5fHG5OC0corHs\n9kqgCXAO1lXUC+id55gN2OD058AZQFNgq4MxiUgYHTwI8+fbZjiLFlkxuxEjrOvI6dZBoMbRkSNH\nmDhxorNflmCcTAzZ2KyjJdgMpRnYjKS7/e+/DowDZgKrsYHwR4EDDsYkIqWUlQW33w5//7tNNwXb\nInPOHCtX4WTrICBR9l52S6y0v9SVJBIF3nvPViYDvPEGDBgASUmR+/78KqGqG6lg0diVJCJxYvNm\nuO8+WLIEBg+GKVOgfPnIxzFu3DhWrVqlGUcOi5VUqxaDiEseeshKU7RsCU89ldticIPX6yU5OVmt\nhBCpiJ6IhNWyZXDrrbZKefLk8C9CE+epiJ6IhMWePbYzWocOtm/yhg2RTwper5cdO3ZE9kvlF0oM\nIgLA999bIjj7bPB6be/kZcugadPIxhHYe3nSpEmR/WL5hQafRRLc+vU23XTPHqhZE1autJXKkaa9\nl6OHWgwiCWrnTtsV7ZJLLCHs3Gmrl91ICoFWQmDG0YABAzTA7KJYufIafBYJk6wsaN3aahc1bgwv\nvAC33OJuTO+88w6ZmZlalxBmmpUkIoXy+WDGDFuHAJCaalVOJX5pVpKIFGj5cluHMHgwdOsG2dlK\nClIwJQaROLZnj+2lfPXVULkyeDywYEFky1gE83g8zJ07150vl5ApMYjEof37YdYsm3oKNq7wr3/Z\n+gQ3BO+9nJOT404QEjIlBpE4smYNDBkCderAHXdY+esNG6wctlvyzjjq06ePe8FISJQYROLA9u22\nUrl5c1i3zjbM8Xph3rzIL1ALNmvWLO29HIM0K0kkxv3lLzBypHUbzZ5tCSJabN26lQoVKighuETT\nVUUSSE4OPPggfPCBbaP55pughcKSl/ZjEEkQJ0/a1NM1a6y1MHgwnHGG21Fp7+V4osQgEkP27YM2\nbWDbNvjqK1vB7LZAjaNdu3Yxffp0t8ORMNDgs0iUy862rqLOneH00+HQIfjyy+hICsEzjp5++mm3\nw5EwKU5iqORYFCKSr4wMqF4dbrsNzjsPvvvO1ii0aeNuXMHrEjTjKP6E0pV0BTAdqAo0AH4L3AUM\ncTAukYS2ejVMmpS7SG3/fqhQwe2ocr388svaezmOhTJS9G+gO/Ae0ML/2nrgYqeCyodmJUlCOHoU\nRo2CCROgbVu46y4YONDKWkST7OxskpKSNNgc5ZyelZR3j73s4n6RiBTs4EFLAvPnQ40a8OSTMGaM\n21EVrJybS6nFcaGMMewArvQ/Lg8MA751LCKRBHL8OPTrB7Vq2Vaac+ZYkoiWpOD1etm0aZPbYUiE\nhZIY7gX+CNQDdmHdSX90MiiReObzQWamDShXqgRvv22lK378EW691e3ocgVmHL344otuhyIRFkp7\n8AIgb9WrK4HPwx+OSHz7/nto1gz++197Pm4cPPqoe2Ww8+P1ehkzZgyvvfYaEyZMoF+/fm6HJBEW\nSothSoiviUg+fD5YtQq6doWGDW2P5WPH7PXHH4+upODxeLjsssvweDxkZGRoq80EVViL4XJsqmpd\n4BFyR7arooVxIiFZtQquusrGElq1ip7VygXZu3cvw4cPp1+/fkoICaywxFAeSwJJ/j8DDmPTV0Uk\nj6wseOst+PhjSE+3vRBq1bKFafXrux1d0bp06eJ2CBIFQvmV4Bxgu7NhFEnrGCSq/fwzfPgh3Hkn\nHD5sA8tXXAHXXOPufgiS2Jxcx3AMGA9cBFT0v+YDri3ul4nEm61b4aabYP162yXthhtg5kyoWdPt\nyAqXnp5Oeno6d955p9uhSBQKZazgbWADcB4wGms9rHQuJJHY4PPZHsrr11u3UVYW/O//RndS8Hq9\njBw5ks6dO1OxYsWiPyAJKZQWQ22sVtIDwP/5f5QYJKEdOGA1jE6csKTQokXRn3Fbeno6AwcOpFGj\nRqpxJIUKpcXg9f+5F/g90BKI4t+JRJxz+DA8/TTUrm2Dyfv2xUZSePvtt+ncuTPDhw8nNTVVSUEK\nFcqgxI3Acqyy6stANaxLKdW5sH5Fg8/iOp8PTjsNypaFl1+2ndNixe7duwGUEBJMpPd8bo1VXS1K\nZ2ASNuV1OvBcPsekAC8CycBP/ud5KTGIazIz4YUXrAz2gQO2arl6dbejEimaE4mhLHAL0BhYB3wA\nXAaMA07H9mUoTBKwEeiA1Vj6GujNqQX4amClNToBO4E6WHLIS4lBImrTJpg71/ZUzsqCatVg9Gjo\n3RvOPNPt6AqXk5ND2bJagyolTwyF/e2Zim3GUxP4E7AAmA28Qu6+DIVpDWzGZjFlAXOBrnmO6eM/\n707/8/ySgkhE/fnPcMEF8M47ti5h/37bTvPhh6M7KQRmHPXpk7e0mUjxFDYrqS3QDMgBKmCDz42B\n/SGeux7wfdDznUDeDQmbYF1In2Krq18C/hbi+UXC4vBhGDkSUlNh2zZ7bcYMuOMOd+MqjsCMo4YN\nGzJ16lS3w5EYV1iLIQtLCgCZwDZCTwpgi+CKkozNcroe6056CksWIo7LyYGxY228IDUVHnjASlfk\n5MROUgjsvdy5c2eGDRumvZclLAprMVwIrA163jjouQ9rTRRmFzaTKaABuV1GAd9j3UfH/T+fAc2B\nX+0MMnr06F8ep6SkkJKSUsTXixTsr3+F++6zx888AyNGuBtPSb3xxhvae1l+kZaWRlpaWqnPU9ig\nxDlFfHZ7Ee+XwwafrwN2Y7OY8g4+X4iV8O4EnAZ8BfQCvslzLg0+S6nt3AkffQTvvWcthD59rOBd\nLBcRzcnJoUyZMqqEKvlyolbS9pIG45cN3AcswWYozcCSwt3+91/HSm0sBtZg3VbT+HVSECmVzEy4\n7DIrXVGvnhW2++QTaN/e7chKT7OPxAmx8muGWgxSIvPnQ48e9vjrry1BxKLA3ssXX3yx26FIDHFi\nuqpIzPrkE6hQwZLC9ddbTaNYTQoZGRm0bt2aiRMnuh2KJIhQE0MlQFXlJSZMnw7XXWdrEXbtgkWL\noHx5t6MqvsCMo44dO/LII48wffp0t0OSBBFKYrgJ8GBjBWCL2yJZJ0kkJBkZVvF08GCberpmjT2P\nRWvWrKF169a/zDgaMGCABpglYkL5m5aObcrzKbkrntcBlzgVVD40xiC/snq13fy3bIHZs2H7drj6\nanjpJdsnIZYtX76cbdu20b9/fyUEKTEni+h9ha1Y9pCbGNZQ9DqGcFJiEAD27oUnnrBd0gDatYMm\nTaBSJXjwQXssIsbJrT3XA339xzbBNuz5orhfJFIaOTnw+OPw/PP2/I03oG/f2Bw7EIl2oYwx3A9c\nDJwA5gCHgYecDEokwOeDUaOseN3zz1v5a58Pbr89PpJCRkYGkyZNcjsMkVOEkhiaAk9gJbcvA57E\naieJOGrXLihXznZM69ULdu+GYcPcjio8gmcc1a5d2+1wRE4RSlfSROBMYB7wD2zgWcQRPp9thtO8\nuSWGpCTYsQMaNCj6s7EiIyODgQMHUr9+fdU4kqgUSoshBWiPFbt7HSuk95SDMUkC2rbNBo/LloU6\ndSwprF8P2dnxlRQWLFjwy7oEVUKVaFXc0erfAI9hhe6Swx9OgTQrKY7985/QrRvUqmXjCYMGQeXK\nbkfljP3793PixAklBIkIJ6erXgT0BLpj+zH8A5gP/FjcLysFJYY4dPQoVKlij2+5BaZNA3W3i4SP\nk4nhS2xbznnYHgtuUGKII5mZMH48PPWUdR0dPx4fM4zyOnnyJElJSW6HIQnMySJ6bYFJuJcUJE74\nfPDqq1CxoiWFbt3g2LH4SwqBGUc33XST26GIlEhhiWGe/8+1+fyscTguiSOHD8PDD1vrYMgQmDrV\nFqwtWACnneZ2dOEVqIS6atUqpk2b5nY4IiVSWBPjbGzntUb5HOcD/uNUUPlQV1KMysqyFkGNGlbK\n4p57oGpVt6MKP6/Xy9ixY3n11VcZP368ahxJVHCiJMZu/59DsJlIwZ7L5zWRU+zaBfXr2+P//Aeq\nVXM3HifNmzdPey9L3AglkwQXzwtYi01djRS1GGKIzwcjR8KYMfY8K8tWMMezwN9PtRIkmjgx+Hwv\nlgCacur4wnY0xiD52LYNmja1sYQxY+CZZxIjKYD9A1RSkHhR2N/k6kBN4Fms2yhw7BFsPUMkqcUQ\nxY4dgw8/hO7dbV3CunXWhRSPMzW9Xi/r1q2jZcuWbociUiQnWgw+rHXwRywZHPb/+IBaxY5Q4s6+\nfTBpkq1S7tcPUlJg505o1Cg+k4L2XpZEUVgmWQTcgCWH/H5dP9eJgAqgFkOUOXrUBpObNoUePeDP\nf3Y7IudoxpHEKidmJd3g//OcEsQjcWz4cFu53LgxfPON29E4a+3atfTv31+VUCWhhJJJrgRWAz8D\n/bEZSi+hdQwJZ+9eOOssezx7tu2gFo9dRsEyMjJYs2aNWgkSk5yslbQWaI5NT50FzAB6ANcU98tK\nQYnBZUuXQseO9njvXjjjDHfjEZGiOVkrKRvIAW4G/gpMAeJw7aoUZNIkSwrXXQc//aSkIBLvQkkM\nR7CtPfsBC4EkIrsXg7jo+eetztG4cfDxx/FbFjsjI4O//OUvbochEhVCSQy9gBPAHcBeoB7wgpNB\nSXR46SV47DG46y54/HG3o3FG8N7LjRo1cjsckagQat/TmUArbNrqv4nsJj2gMYaI8vng5Zdtq817\n7rFS2fEoeO/lqVOnasaRxB0nxxh6Al9hA849scTQo7hfJNEtOxtWrYJrr7WSFg8+CPfeG79JYdGi\nRdp7WaQAoWSSNUAHclsJdYFlQDOngsqHWgwOmjIF7r/fHrdvDy+8AJdcEn97JQQ7cuQIR44cUUKQ\nuObEArdfzg3sC3q+vyRfJNFn/Xrba3nTJvjjH637KFGm6letWpWq8bgxhEgYhJIYFgNLgL9jCaEX\n8KGTQUlkXHKJ/blmDfwmkkXUIywrK4vkZE2kEwlVKGMMw4HXsK6j3wCvA486GZQ4a8ECW5MAcPBg\n/CaFwIyjDh06oK5IkdAV1mK4AJuWej42zjAc2BmJoMQZP/4IN9wAK1fCZZfBkiW25WY8Cp5xNGfO\nHJWzECmGwv61/AuYDSwHbgQuB7pFIqh8aPC5FLxeK2nx+9/b89WroVkkpw5EkCqhiuRyYrpqFWAa\nsAFrOZSkzHZn/+c3Ufge0a2w0htuJZ641qyZJYVrrrFqqPGaFACWLFnyy97LAwYMUFIQKYHC/tVs\nAPoEHfe2/3kZbKFbehHnTgI2YlNddwFfA72Bb/M5bilwDJgJLMjnXGoxlMDf/ga33w4nT1o3Ut26\nbkfkPO29LJLLiemqe4EJhTxvX8S5WwObsY1+AOYCXfl1YrgfmI+1GiRM/vlPGDAA2rSBf/0rMfZd\nBiUEkXAo7HaRUspz1wO+D3q+E2iTzzFdgWvJLbkhpdSsGaxda6uY586Nz6Tg9XpZuXIlV1xxhduh\niMQdJ28ZodzkJwEj/MeWQQvnSm3ZMksKu3ZBvC7qDcw4atKkCZdffrlaCSJh5mRi2AU0CHregF9P\nd70U62ICqAN0AbKA1LwnGz169C+PU1JSSElJCV+kccDns+02H30ULr00PpOCZhyJFC4tLY20tLRS\nn8fJf1XlsMHn64DdWPG9/AafA2YC7wPv5vOeBp8L8fnnVhr7m2/gvvtg7FioVs3tqMLrm2++oU+f\nPqqEKlIMTtZKKgv0xaarPg00xMpw/7uIz2UD92HlNJKwLUG/Be72v/96cYOVUx07Bm3bWtdRq1bw\n3XfQpInbUTmjfPnyPPLII2oliERAKP/CXsO29rwWuBCoBXwEXOZgXHmpxZDHDz9YAbwVK2DhQlvR\nLCISzMkWQxugBeDxPz+AtvZ0jddrCeGDD+z54sXQqZO7MYlIfAmliJ4X6woKqIu1ICTCJk+2PRI+\n+MDGE3y++EsKGRkZDB8+XEXvRFwUSmJ4GfgncDowDvgceMbJoOTXXnnFdlXr2xcOHYL/+R+3Iwqv\n4L2XfxOv5V5FYkQoXUlvAauw2UWQ/+plcYjPB6+/bhvpzJoFt93mdkThF1wJNSMjQzOORFwWyqBE\nwzzHBtr4O8IfToEScvB5zhzo469WNWYMPPmku/E4YdmyZfTu3VvrEkQcUNLB51A+sI7cZFABm7a6\nEbi4uF9WCgmTGA4csFbB+vWwbRv06gXTp0OVKm5H5owTJ06wf/9+tRJEHOBkYsirJfBHYFAJPltS\nCZEY5s2Dnj0hORleeAF69IjPFcwiEhmRTAxgrYhLSvjZkojrxLBli7UKnn3WSlo8+yzEY49KZmYm\nFSpUcDsMkYThZGIYGvS4LNZiqAVEcqJk3CaG4cOtxtEFF1gLYcwYtyMKv0CNo0WLFvH1119rHEEk\nQpxc4Bbcu50NLCT/zXSkmJYssaTwwgswbJjb0TgjeMZRamqqkoJIDCgqMSQB1Ti11SBhMHEiDB1q\nm+kMjcOrq0qoIrGrsMRQDmshXEnudp5SSj4fXH217ar29NPw1FNuR+SMFStWkJ6ernUJIjGosF/h\n0rHxhNeAs4F52L7MYEkiv/LYTomLMYasLLjwQti6FVautH0TRESc4sQYQ+BkFYD9WHXVYJFMDDHv\n6FGoWxeOH7cy2ZdEck6XiEgxFJYY6gKPAGsjFEtc69XLksKaNfGVFLxeL8uXL+e6664r+mARiQmF\nJYYkoGqkAolXmZnQuDHs3m21juKpPlxgxtG5555L+/btKVs2lJqMIhLtCut78mD7MESDmBxj8Pkg\nKcn+XLHCdluLB5pxJBIbnFzHICV0002WFHbuhHr13I4mPDZs2MCtt96qSqgicaywTFIbG3SOBjHV\nYti3D84/Hw4fjr/ZR7t372bZsmX069dPrQSRKBfpWkmRFhOJweezGUc9esB331kNpPPOczsqEUlU\nJU0MGi0Mk0OHoGxZaN7cBpjT05UURCQ2KTGEwWefQY0a9jgnB+bPhxbRMmxfQhkZGdxzzz3k5Gh7\nb5FEo8RQSkuXwjXXWHXUH36I/XLZwXsvX3HFFRpHEElAmpVUQjNnwpAhtk7hD3+wVkKs097LIgJq\nMRTbiRNw771wxx3w0EO2mjkeksIXX3xBx44deeSRR3j//feVFEQSWKz0E0TFrKTJk+HBB+3x229D\nnz7uxhNOJ0+eZN++fZx55pluhyIiYaLpqg67+26YOhXGjoURI2wGkohINNPKZ4f4fNCli+22tnw5\ntGvndkSld/ToUSpXrux2GCISpfR7byGOHYPKlS0prF0b+0khMOOodevWnDx50u1wRCRKKTEUYMYM\nSwrHj1uCiPVS2RkZGbRu3ZpVq1axdOlSkpKS3A5JRKKUEkM+Bg2CO++Enj1tbULFim5HVHLB6xI0\n40hEQqExhjxmzIA33oDnnoNHH3U7mtJbu3YtGRkZWpcgIiHTrKQg2dmQnAwdO9q4gohILNN01VL6\n+Weo6t+vLicn9ktbiIioumoppKbmJoXDh2MzKXi9XhYuXOh2GCISBxI+MQwbBl27Qvfu1lKoGoO7\nXAdmHE2dOpXs7Gy3wxGRGBeJxNAZ2ABsAh7L5/2+wGpgDfA50CwCMQFw660wYQIMHAjz5sVeSyHv\njKP33nuPcuU0n0BESsfpu0gSMAXoAOwCvgZSgW+DjtkKXA0cwpLIVKCtw3Exbx784x8wbZpNTY01\nmzdvpnv37qqEKiJh5/TvyJcDo7AbPsAI/5/PFnB8TWAtUD/P62EdfP7sM9tD4YEH4KWXwnbaiDp4\n8CAffvghvXv31p4JIpKvaJ2V1B3oBAz2P+8HtAHuL+D4YcAFwF15Xg9bYsjKgvLlbQtOjyf2uo9E\nREIVrUX0inM3bw/cAVyZ35ujR4/+5XFKSgopKSklCmj2bPtz4UIlBRGJL2lpaaSlpZX6PE7fGtsC\no8ntSnocyAGey3NcM+Bd/3Gb8zlPWFoMq1fDb39rxfCWLy/16SIiIyOD8ePHM3PmTJKTk90OR0Ri\nSLSuY1gJNAHOAcoDvbDB52ANsaTQj/yTQlhkZkLfvjYT6ZNPnPqW8AmecdSxY0fNNhKRiHH6bpMN\n3AcswWYozcBmJN3tf/91YCQ26Pyq/7UsoHU4g9i719Yq/Oc/sHixlb2IZtp7WUTcFCu97CXuStq3\nD+rVs0HnL76Ayy8Pc2Rh5vF46NSpE+PHj6d///6acSQiJRats5LCpUSJweeDIUPgvfdg48bYWNXs\n8/n46acoX2VMAAAKbUlEQVSfqFu3rtuhiEiMi9ZZSa4aPhxeew3mz4+NpAD2P1JJQUTcFLctBp8P\nqle3kheDBxd9vBsOHTpE9erV3Q5DROJUtM5KckV2NnTqBEeORGe5i8CMo5YtW+L1et0OR0TkFHHX\nlZSVBRdcANu3w6pV0beIzePxMHDgQBo0aMDy5cspX7682yGJiJwirloMx47B+edbUti+HVq2dDui\nXIFWQqdOnRg2bJj2XhaRqBU3LYZvvoGLL7bHmzdDo0buxpPXli1bWLdundYliEjUi7KOlgIVOvg8\nezY89ph1G61caesWREQSXcIOPm/ZYhvt9O4N336rpCAiUloxnRg2b7Yxhfbt4cUXoUYNtyOysYR5\n8+a5HYaISInFdGK44w6oVCl6iuJ5PB5atWrFm2++yYkTJ9wOR0SkRGI2MRw7ZqWzoyEp5J1xlJqa\nymmnneZ2WCIiJRKzs5LeegsuugjatHE3jm3btnHzzTfTsGFDzTgSkbgQs7OSrrzSKqWOH+9SRH5H\njx5l4cKF9OzZU5VQRSSqJFx11bp1bW+FSy91KSIRkSiXUInh6FGoUsXKX2hjMxGR/CXUOoZx46zF\nEMmk4PF46NatG5mZmZH7UhERF8RcYvB6LTHcf3+kvi93xtEtt9yi2UYiEvdiriNm7lyoVQtGjHD+\nuwKVUDXjSEQSScwlhoUL4a67IDnZ2e/ZuHEjnTp1YsKECfTr108zjkQkYcRcYvjuO7jxRue/p2nT\npmzcuJGaNWs6/2UiIlEkpsYYMjJg9Wpo1Soy36ekICKJKFb6R3w+n49bb7UpqgsWhPfk+/fvp3bt\n2uE9qYiIyxJiuqrHA4MGhe98gRlHLVq04NixY+E7sYhIDIuZxHDokI0vXHtteM4XqISanp7Ol19+\nSaVKlcJzYhGRGBczieHee6FCBfspjfwqoWoaqohIrpiZlbRgAXz2WenPs2fPHjZs2KB1CSIiBYiZ\nwecqVXwcPKjaSCIioYr7wefrr1dSEBGJhJhJDJdcUrzjvV4vs2fPJm+5bhERKVzMJIby5UM/NjDj\naP78+ZqGKiJSTDGTGHr2LPqY/GYcVa5c2fngRETiSMz02p97buHv79y5kxtuuEGVUEVESilmZiUV\nNVbg9XpZuHAht9xyiyqhioiQYFt7iohI0aJ1umpnYAOwCXisgGMm+99fDbRwOB4RESmCk4khCZiC\nJYeLgN7A/+Q55nrgfKAJcBfwalEn9Xg8dOnShcOHD4c32hiRlpbmdghRQ9cil65FLl2L0nMyMbQG\nNgPbgSxgLtA1zzE3AbP9j78CagBn5Hey4BlHffr0oWrVqo4EHe30lz6XrkUuXYtcuhal5+SspHrA\n90HPdwJtQjimPvBD3pO1atVKM45ERCLAycQQ6mhx3oGRfD83dOhQ+vfvrxlHIiIOc/Iu2xYYjY0x\nADwO5ADPBR3zGpCGdTOBDVRfw69bDJuBxg7FKSISr7Zg47hRoxwW1DlAeSCD/AefP/A/bgt8Gang\nRETEHV2Ajdhv/I/7X7vb/xMwxf/+aqBlRKMTEREREZHYogVxuYq6Fn2xa7AG+BxoFrnQIi6UvxcA\nrYBsoFskgnJBKNchBfAA67Dxu3hV1LWoAyzGurDXAQMjFlnkvYGNy64t5JiYvW8mYV1K5wDJFD0m\n0Yb4HZMI5VpcDlT3P+5MYl+LwHGfAAuBP0QquAgK5TrUANZjU77Bbo7xKJRrMRp4xv+4DrCfGCoa\nWkxXYTf7ghJDse+b0VR2O6wL4mJcKNdiBXDI//grcm8G8SaUawFwPzAf2BexyCIrlOvQB1iArQcC\n+ClSwUVYKNdiD1DN/7galhiyIxRfpC0HDhbyfrHvm9GUGPJb7FYvhGPi8YYYyrUINojc3wjiTah/\nL7qSW1IlHisuhnIdmgC1gE+BlUD/yIQWcaFci2nAxcBurPvkwciEFpWKfd+MpqZVWBfExbji/De1\nB+4ArnQoFreFci0mASP8x5YhdqoGF0co1yEZm9l3HVAJa1V+ifUtx5NQrsUTWBdTCrYGainQHDji\nXFhRrVj3zWhKDLuABkHPG5DbJC7omPr+1+JNKNcCbMB5GjbGUFhTMpaFci0uJXeRZB1smnQWkOp4\ndJETynX4Hus+Ou7/+Qy7GcZbYgjlWlwBjPU/3gJsA5piLalEE9P3TS2IyxXKtWiI9bO2jWhkkRfK\ntQg2k/iclRTKdbgQ+BgbnK2EDUZeFLkQIyaUazERGOV/fAaWOGpFKD43nENog88xed/UgrhcRV2L\n6diAmsf/8+9IBxhBofy9CIjXxAChXYdh2MyktcADEY0usoq6FnWA97H7xFpsYD5ezcHGUrxYq/EO\nEve+KSIiIiIiIiIiIiIiIiIiIiIiIiIiIlIcJ8ldl+HBFvEV5OcwfN8sYKv/u1ZRssWC07CFZWBl\nGIJ9XuLIThW4LmuAd4EqRRzfHJvnLyIS84pTxyYcNW+CF8P9Dlv8UxpO1eEJPu8sYGgRxw8EXnYo\nFkkA0VRdVSSvyliJh1XYb8s35XPMWVhNIA+2wrWd//WOwBf+z77jP1d+AsXFlpO7afoj/nOtJbcq\nZ2VgEVZ+YS3Qw/96Glar6Vmgoj+Ov/nfC7Rq5mJlCQJmYQmpLPACtmp9NXBXATEGW4EVhQMrP/0F\nkI61Ti7ASkQ8DfTyx9LDH/sbWMnldPK/jiIiUSmb3G6kBVjNn6r+9+pwajG4wG/RQ8ntwimLdbPU\nAf4Pu1GD7fD1VD7fN5PcTX16YDfdllgSqojdUNcBv/UfNzXos4Fa/5+SW2Igb4sh8PxmLBmA3bh3\nAKdhieBJ/+unAV9jNW/yCpwnCbsuQ/zPq/pfA+iA7UcBcBu2Y1fAOGzHP7Ba/BuxWkoi+Yqm6qoi\nxzl128FkbBeuq4Ac4GzgdODHoGP+jf02nAz8L/abdwpWPO4L/zHlgx4HK4P9xv4n/zkHYV1K7/pj\nwf/4KmybyPFYy2Ah8K9i/HctBl7yx9EFS1onsFbNb4Du/uOqYa2W7Xk+H2iJ1PO/95r/9RrAm/7P\n+Mj995y39HhH4EasjhJYEmqAJQiRX1FikGjWF/vtvyU2ALsNqJDnmOXYjfv32G/lE7ES5EspunCa\nD7tZvhv0WgdOvamW8R+3CUtaNwBjgGXAX0L878jEupw6AT2xomcB9/ljLUwgYVYElmCbEv3T//3L\ngFuARhS+x3M34q/8tjhEYwwSzaphv8mfxDYkapTPMQ2x7Tyn+39aYGWFryS3L74ytrtZfvJuYLIc\n6/oJdCXd7H/tLOwG/zbWcshvQ/UsCv5l6x9Y1ctA6wPsJj8k6DMXUHgXz3GsYupYf9zVsKqaALcH\nHXeY3C64wPcEV1qNqc3gRSSxHc7zvDbWBbQG6y5aT+4U1sCxt2GDwelYF00gebQnd1B3NdaiyKug\nEt0Pkzv4HLihdvSfJ1DiPDCuEDzG8CzwDbmDz8H/PeWwMukzgl4rg93k1/i/axm5YxfB8l6XVGxw\nuS3WHZSOtR62+t+v6Y8xMPhcAet+WoONmcTTBkYiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIiIS\nL/4f+XFCA+alvckAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10df08c90>"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This is a pretty bad ROC curve."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Plotting the learning curve \n",
      "\n",
      "[sklearn.learning_curve.learning_curve](http://scikit-learn.org/stable/modules/generated/sklearn.learning_curve.learning_curve.html#sklearn.learning_curve) is another one-liner which might be useful for illustrating the learning behavior of a given a model (for a fixed choice of parameters). Building upon our last example, we will plot the learning curve of the best model that we found through grid search. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.learning_curve import learning_curve\n",
      "from sklearn.cross_validation import ShuffleSplit\n",
      "\n",
      "cv = ShuffleSplit(digits.data.shape[0], n_iter=10,\n",
      "                                   test_size=0.2, random_state=0)\n",
      "\n",
      "train_sizes, train_scores, test_scores = learning_curve(clf.best_estimator_, X, y, cv=cv,\n",
      "                                                        train_sizes=np.linspace(.1, 1.0, 5))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 41
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can use the [fill_between](http://matplotlib.org/examples/pylab_examples/fill_between_demo.html) plotting function of Matplotlib to show the standard deviation in the score estimates. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "train_scores_mean = np.mean(train_scores, axis=1)\n",
      "train_scores_std = np.std(train_scores, axis=1)\n",
      "test_scores_mean = np.mean(test_scores, axis=1)\n",
      "test_scores_std = np.std(test_scores, axis=1)\n",
      "plt.grid()\n",
      "plt.fill_between(train_sizes, train_scores_mean - train_scores_std,\n",
      "                     train_scores_mean + train_scores_std, alpha=0.1,\n",
      "                     color=\"r\")\n",
      "plt.fill_between(train_sizes, test_scores_mean - test_scores_std,\n",
      "                     test_scores_mean + test_scores_std, alpha=0.1, color=\"g\")\n",
      "plt.plot(train_sizes, train_scores_mean, 'o-', color=\"r\",\n",
      "             label=\"Training score\")\n",
      "plt.plot(train_sizes, test_scores_mean, 'o-', color=\"g\",\n",
      "             label=\"Cross-validation score\")\n",
      "\n",
      "plt.legend(loc=\"best\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 42,
       "text": [
        "<matplotlib.legend.Legend at 0x10fa93650>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEACAYAAAC+gnFaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvXl4VOX5//86s2UnCSQsAZIgWqtWwdYiopCoCBTcl4po\n+0Otl7YiSLVVcUFR6/KtFfm0tqiItkptXVpRyiKVEBQVraLWKsqShCTImn2fmfP745kzmUlmkoFM\nZs5J7td1nStz1nnPyczzPs99PwsIgiAIgiAIgiAIgiAIgiAIgiAIgiAIgiAIgiAIgiAIQhDTgK+A\nb4DbwhxTCHwC/BcoCtheAnzm27eltwQKgiAIvYcd2A7kA05gK3Bch2MygC+AEb71rIB9u4CBvStR\nEARB6Am2bvaPQxlBCdAGvARc0OGYWcCrQLlv/UCH/VrPJAqCIAi9SXdGMBzYHbBe7tsWyDGop/4N\nwEfATwL26cB63/breqRUEARB6BUc3ezXI7iGE/g+cDaQDLwHvI/KKZwBVALZwFuoXMOmIxUrCIIg\nRJ/ujKACGBmwPpL2EJDBblQ4qMm3FANjUEZQ6TtmP/APVKgpyAhycnL0yspKBEEQhMNiB3B0NC7U\nXWjoI1ToJx9wAZcDKzsc8zrqyd+OqhGcCvzP9zrNd0wKMAX4vOMbVFZWouu66ZeFCxfGXYPoFJ1W\n1Sg6o78Ao4+k0A9FdzUCNzAHWIsq6JcBXwLX+/YvRYV71qCaiXqBp1FGcBTwWsD7vAisi5bwWFNS\nUhJvCREhOqOLFXRaQSOITjPTnREArPYtgSztsP5b3xLITmDsEeoSBEEQYkR3oSHBx+zZs+MtISJE\nZ3Sxgk4raATRaWbM0MZf98W7BEEQhAjRNA2iVIZLjSBCioqK4i0hIkRnaAYOHIimabLIYrll4MDe\nH5whkhyBIFieqqoqpOYpWBFN6/3AjYSGhH6BpmliBIIlCffd9RmEhIYEQRCEniNGECESe48uVtEp\nCP0BMQJB6CNMnz6dv/zlL1E/Vuj7SI5A6BeYNUeQmprqTwY2NDSQmJiI3W4H4KmnnuKKK66IpzzB\nBMQiRyBGIPQLwv2YiletYt2SJThaWnAnJDBl7lwmzZhxWNeOxjUARo0axbJlyzjrrLM67XO73Tgc\n0sivP96HWBiBGdBjTkuLrh88qOseT8SnbNiwoff0RBHRGZpQ37ONb76pLxg9WtfBvywYPVrf+Oab\nEV83GtcwyM/P1//973/ruq7uz/Dhw/VHHnlEHzp0qP7Tn/5Ur6qq0mfMmKFnZ2frmZmZ+rnnnquX\nl5f7zy8oKNCfeeYZXdd1ffny5frpp5+u33rrrXpmZqY+atQoffXq1Ud07M6dO/WJEyfqaWlp+uTJ\nk/Vf/OIX+lVXXRXyM+zfv1+fMWOGnpGRoQ8cOFCfOHGi7vV6dV3X9bKyMv2iiy7Ss7Oz9UGDBulz\n5szRdV3XPR6Pfv/99+t5eXn64MGD9Z/+9Kd6TU2Nruu6vmvXLl3TNH3ZsmV6bm6uXlBQoOu6ri9b\ntkw/7rjj9MzMTH3q1Kl6aWnpYd9vqxCujCSyaQIion/mCDwe2LMHysqgtTXeaoQ4sW7JEh7csSNo\n24M7dvDW//1fTK8Rjr1791JVVUVZWRlLly7F6/Vy7bXXUlZWRllZGUlJScyZM8d/vNEByWDLli18\n97vf5eDBg/z617/m2muvPaJjZ82axfjx4zl06BD33nsvL7zwQti27Y899hgjR47kwIED7Nu3j4ce\neghN0/B4PJx77rmMGjWK0tJSKioq/GGv5557jueff56ioiJ27txJfX190OcCKC4u5quvvmLNmjW8\n/vrrPPTQQ/zjH//gwIEDTJw4UUJofYAY+6uu642Nuv7117q+c6eub9um69XVsdcgxJRQ37OFBQVB\nT/LGsjDEtnBLuGMX+p5cD4eONQKXy6W3tLSEPf6TTz7RMzMz/euFhYX6smXLdF1XT/lHH320f19D\nQ4OuaZq+d+/ewzq2tLRUdzgcelNTk3//VVddFbZGcM899+gXXHCBvn379qDtmzdv1rOzs3VPiFr4\nWWedpf/xj3/0r2/btk13Op26x+Px1wh27drl3z9t2jS/dl1XNYrk5GS9rKws7L2yMuHKSKRGECUS\nEiA5GfbuhYoKcLvjrUiIIe6EhJDbPVOnRmwF7ilTQl8jMbHH+rKzs3G5XP71xsZGrr/+evLz80lP\nT6egoICampqwSfChQ4f6XycnJwNQX19/WMdWVlYycOBAEgM+z8iRIzudb/CrX/2Ko48+milTpjB6\n9GgeeeQRAHbv3k1eXh42W+ciZ8+ePeTl5fnXc3Nzcbvd7N27N+R7lpaWMm/ePDIzM8nMzGTQoEEA\nVFRUhNUldE3/NgIAmw1SU6G5GUpLoaEh5GFWafcuOiNnyty53Dk6eG6PBaNHc85NN8X0GuHoGH55\n7LHH+Prrr9myZQs1NTVs3LgxcJKSXmHYsGEcOnSIpqYm/7aysrKwx6empvLb3/6WHTt2sHLlSn73\nu9/x9ttvk5ubS1lZGR6Pp9M5OTk5QXMAlJWV4XA4GDJkiH9b4L3Izc3lqaeeoqqqyr80NDQwfvz4\nHn7a/osYgUFSErhcUF4O+/eD1xtvRUIvM2nGDKY+8QR3T53KvQUF3D11KtOeeOKwWvxE4xqRUl9f\nT1JSEunp6Rw6dIj77rsv6u/Rkby8PE455RTuvfde2traeO+993jzzTfD5ghWrVrF9u3b0XWdAQMG\nYLfbsdvtjBs3jmHDhnH77bfT2NhIc3MzmzdvBuCKK67g8ccfp6SkhPr6ehYsWMDMmTND1h4Abrjh\nBn7zm9/wv//9D4Camhpefvnl3rkB/YT+1Q6rO+x2SEuDmhqor4ecHBU+AgoLC+OrLUJE5+ExacaM\nHhfa0bhGKDoWtjfffDOzZs0iKyuL4cOH88tf/pKVKzvOHNt+bsfzwxXe3R374osvMnv2bAYNGsS4\nceO4/PLLQz7ZA3zzzTfMmTOH/fv3k5mZyY033khBQQEAb7zxBnPnziU3NxdN07jyyiuZMGEC11xz\nDZWVlUyaNInm5mamTZvG/wUk2ztqu/DCC6mvr2fmzJmUlpaSnp7OlClTuOyyy0JqErrHDG1Q9d6s\n2oakqUk9+aekhD+mtVUt2dmQkQExGAFQ6D3M2qHMilx++eUcf/zxLFy4MN5S+gUy6Fw8cbmUUezf\nD+XlFK1fH29FEWGG2HskWEWnAB999BE7duzA6/WyevVqVq5cyYUXXhhvWUIUkdBQV2haeyJ5714V\nLkpNjbcqQYgp3377LRdffDEHDx5k5MiR/OlPf2LMmDHxliVEETPEO8wZGuqIxwONjSpMlJWl8gmC\nZZDQkGBVJDRkJoxEcl2damba3BxvRYIgCFFBjCBCinxN3UhOVqZQWgqHDqmORSbCKrF3q+gUhP6A\nGMGR4HSqXMHBg7B7N7S1xVuRIAjCESM5gp7S3KzyB0OGwIABPb+e0CtIjkCwKpIjsAKJiapX8p49\nagnT0UYQBMGsiBFEiD9HEAqbTSWSGxtV7iBgXJZYY5XYu1V0CodHSUkJNpsNr2+Ilq6mxOx47OHy\n0EMPcd111x2xVqEdMYJokpQEDoea5+DAARmvSIiYFStWcMopp5CWlkZOTg7Tp0/n3XffjbesHvOv\nf/2Ln/zkJz2+TlFRUadRT++44w6efvrpHl9bECOImMIJEyI70OFQtYOqqrhMfGOWMXy6wyw6V721\niqlXT6VwdiFTr57KqrdWxfwav/vd75g/fz533XUX+/btY/fu3dx4441hxxEKN86PYA3cJhzuXoyg\nt0hJUU1LS0rUIHaC6Vj11irm/WEe6/LXsXHURtblr2PeH+YdVkHe02vU1NSwcOFCnnzySS688EKS\nkpKw2+3MmDHDP5b/vffey6WXXspPfvIT0tPTef7556msrOT8889n0KBBHHPMMTzzzDP+a27ZsoVT\nTjmF9PR0hg4dyi233AJAc3MzV111FVlZWWRmZjJu3Dj27dvXSdPf/vY3fvjDHwZte/zxx7ngggvU\nZ161ipNPPpn09HRyc3O7HAW1sLCQZcuWAcrAbr31VrKzsxk9ejSrVgXfo+XLl3P88cczYMAARo8e\nzVNPPQVAQ0MDP/rRj6isrCQtLY0BAwawZ88e7r333qDaxsqVKznhhBPIzMzkzDPP5KuvvvLvy8/P\n57HHHmPMmDFkZGQwc+ZMWlpaQmrevn07BQUFZGRkkJ2dzcyZM/37vvjiC8455xwGDRrE0KFDeeih\nhwBoaWnh5ptvZvjw4QwfPpz58+fT6nsILCoqYsSIETz66KMMGzaMa6+9Fl3Xefjhhzn66KPJysri\n8ssvp6qqKux97G3ECCKkyxxBOOIw8Y1VYu9m0LlkxRJ2nBw8zeSOk3fwf3+NfJrJnl7jvffeo7m5\nmYsuuqjL41auXMlll11GTU0Ns2bNYubMmeTm5rJnzx5eeeUVFixYwIYNGwCYN28e8+fPp6amhp07\nd3L55ZcD8Pzzz1NbW0t5eTmHDh1i6dKlJCUldXqv888/n23btrF9+3b/thUrVnDllVcCas6BF154\ngZqaGlatWsUf//hHXn/99ZC6A0c2ffrpp1m1ahVbt27lo48+4pVXXgkaWXTIkCGsWrWK2tpali9f\nzvz58/nkk09ISUlhzZo15OTkUFdXR21tLcOGDQs69+uvv2bWrFksWbKEAwcOMH36dM477zz/07em\nabz88susXbuWXbt28dlnn/Hcc8+F1Hz33Xczbdo0qqurqaioYO7cuQDU1dUxefJkpk+fzp49e9i+\nfTtnn302AA8++CBbtmzh008/5dNPP2XLli088MAD/mt2nHZ0yZIlrFy5kuLiYvbs2eMfqTVeiBH0\nNoET35SUhJ34Rog9LXroJ8K1O9ei3adFtKzbtS7kNZq9kfU8P3jwIFlZWWHH3jeYMGEC559/PgD7\n9+9n8+bNPPLII7hcLsaMGcPPfvYz/vznPwPgcrn45ptvOHDgAMnJyYwbN86//eDBg3zzzTdomsbJ\nJ59MWlpap/dKSkriggsu4K9//Sughpbetm2b//0LCgo44YQTADjxxBOZOXMmGzdu7Paz/v3vf2f+\n/PkMHz6czMxMFixYENQscvr06YwaNQqASZMmMWXKFDZt2gQQsvlk4La//e1vnHvuuZx99tnY7XZu\nvfVWmpqa/HMeAMydO5ehQ4eSmZnJeeedx9atW0PqdLlclJSUUFFRgcvlYoIvLPzmm2+Sk5PD/Pnz\ncblcpKam+u/tihUruOeee8jKyiIrK4uFCxcGJcltNhv33XcfTqeTxMREli5dygMPPEBOTg5Op5OF\nCxfyyiuvHHHivKeIEURIxDmCcCQlqRpCL098Y5bYe3eYQWeCFnqqyqlHTUVfqEe0TBkVeqrKRFtk\nU1UOGjSIAwcOdFsAjBgxwv/amD4yJaAfTG5urn+qxmXLlvH1119z3HHHMW7cOH8I5ic/+QlTp05l\n5syZDB8+nNtuuw23282mTZtIS0sjLS2NE088EVAT1htGsGLFCi666CL/dJUffPABZ555JoMHDyYj\nI4OlS5dy8ODBbj/rnj17ghK+ubm5QftXr17N+PHjGTRoEJmZmfzrX/+K6LrGPQm8nqZpjBw5Mmj6\nysDpOJOSksJO2/noo4+i6zrjxo3je9/7HsuXLwfUdJtHHXVU2PfvON1mZWWlf73jtKMlJSVcdNFF\n/uk2jz/+eBwOR9D0nLFEjCCWBE58U1oKYWKUQmyYO2suoz8JnmZy9MejuemKyKeZ7Ok1TjvtNBIS\nEvjHP/4R9piOE8fk5ORw6NChoIKsrKzMbxZHH300K1asYP/+/dx2221ceumlNDU14XA4uOeee/ji\niy/YvHkzb775Jn/+85+ZOHEidXV11NXV8fnnnwMwefJk9u/fz6effspLL73ErFmz/O81a9YsLrzw\nQsrLy6muruaGG26I6El22LBhQdNcBr5uaWnhkksu4de//jX79u2jqqqK6dOn+5/6w02qYzB8+HBK\nS0v967qus3v3boYPHx72noZjyJAhPPXUU1RUVLB06VJ+8YtfsGPHDnJzc9m5c2fIc0JNt5mTkxP2\n/XJzc1mzZk3QdJuNjY0MGzasy8/ZW4gRRMgR5QjCkZyshrguLVWti6LY49UMsfdIMIPOGefM4Ikb\nn2Bq6VQKdhUwtXQqT8x5ghnnRD7bWE+vkZ6ezqJFi7jxxht5/fXXaWxspK2tjdWrV3PbbbcBncMi\nI0eOZMKECdxxxx20tLTw2Wef8eyzz3LVVVcB8MILL7B//37/9TVNw2azsWHDBj7//HM8Hg9paWk4\nnU7sYUbRdTqdXHbZZdx6661UVVVxzjnn+PfV19eTmZmJy+Viy5YtrFixotuCGuDHP/4xS5YsoaKi\ngqqqKh5++GH/vtbWVlpbW/1hstWrV7NuXXvYbciQIRw8eJDa2tqQ177ssstYtWoVb7/9Nm1tbTz2\n2GMkJib6wzod6aqX+csvv0x5eTkAGRkZaJqG3W7n3HPPZc+ePTzxxBO0tLRQV1fHli1bADXd5gMP\nPMCBAwc4cOAAixYt6rLZ7A033MCCBQv8Zrh///6wrcRigcxHEC9cLjVm0f79ap6DoUPVuhBTZpwz\n47AK/t64xi9/+UuGDh3KAw88wJVXXklaWhqnnHIKd955JxB6Ksm//vWv3HDDDeTk5JCZmcmiRYs4\n66yzAFi7di233HILjY2N5Ofn89JLL5GQkMDevXv5+c9/Tnl5OampqcycObPLwmrWrFlMmjSJG2+8\nMSiH8eSTT3LLLbcwZ84cCgoKuPzyy6murvbvD2cK1113HV9//TVjxowhPT2dW265xf9AkJaWxpIl\nS/jxj39MS0sL5513nr+VEsB3v/tdrrjiCo466ii8Xi9ffPFF0H059thjeeGFF7jpppuoqKjg5JNP\n5o033sDhCF3EhbqnBh999JE/2T5kyBCWLFlCfn4+AG+99Rbz5s3jvvvuIyEhgfnz5zNu3Djuuusu\namtrOemkkwBlenfddVfYezJv3jx0XWfKlClUVlYyePBgZs6c6c/DxJpIxqmYBiwG7MAzwCMhjikE\nHgecwAHfeqTnWnusoWjQ3KxyBkOHysQ3vYSMNSRYlViMNdTdRezANmAyUAF8CFwBfBlwTAbwLjAV\nKAeyUGYQybkQYyMoXrWKdYsX46ipwZ2czJRrrmHS5Mkxe/+wyMQ3vYoYgWBVYmEE3YWGxgHbgRLf\n+kvABQQX5rOAV1EmAMoEIj03phSvWsXaefN4cEd7u+87fQme7sygaPPmnrcc6orAiW8aGiAnRw1o\nd5gUFRWZokVOd1hFpyD0B7pLFg8Hdgesl/u2BXIMMBDYAHwEGEHHSM6NKeuWLAkyAYAHS0t5y9c8\nzBSYfOIbQRD6Ht3VCCIphZzA94GzgWTgPeD9CM8FYPbs2f5kTEZGBmPHjvU/LRrJpGisO1paKPK9\nZ6HvbxGw+9tv/VqM1kHG03/c1k87DQ4epOitt2DQIAp9NZbuPq+xrTfun5XXBcHqFBUV+XtDG+Vl\ntOguvjQeuBeV9AW4A/ASnPS9DUjyHQcqKbwGVQPo7lyIYY7grqlTeWBd556gdycmcv9TT4Gvu7ip\nkIlvooLkCASrYoaJaT5ChX7yARdwOdCxsevrwBmo5HAycCrwvwjPjSlT5s7lztHBnX8W5OVxzpw5\ncM89cN11akygEES1H8HhcJgT31jlCdgqOgWhP9BdaMgNzAHWogr6Zahk7/W+/UuBr1A1gM9QT/xP\no4yAMOfGjUkzVFvvuxcvxl5TgyclhWlXX60SxT//OfzhDzB1KsyZA9dea552/R0nvhk2TJmDEDGZ\nmZkRdXoSBLORmZnZ6+9hhl+GufoR7NwJd96pOno99BB0GI437rjdSv+gQTBwoDIJQRD6HbHsRxAL\nzGUEShG88Qbcdx8UFipjGDgwphK7paFB1VhyclQvZUEQ+hUyeX1vo2lw/vlQVKTM4swzKbr/fnNN\nPRlm4hurxN5FZ/SwgkYQnWZGjKAr0tJg0SJ48UVYvx4uugj+97/uz4sVcZj4RhCEvoeEhiLF64UV\nK+DRR+GSS+CWW8w1LlBTk/o7dKh5xlASBKHXkNBQPLDZ4Kqr4O231dDRhYWwapV5ev4mJalcQS9P\nfCMIQt9DjCBC/P0IsrJg8WL4/e/ht7+Fn/5UNek0A3Y7RZ9/bomJb6wSh7WCTitoBNFpZsQIjpTx\n42HdOjjtNJgxAx5/3DwFby9OfCMIQt9DcgTRoLxc9Uz+5hv4zW9g4sToXLen6LpqZpqUJBPfCEIf\nQ/oR9JTemphm3Tq4+2445RRlDEOGRPf6R4pMfCMIfQ5JFseBiMYamjIFNmyAESNg8mR49tluxwaK\nNiF1JiaqpqYVFaqpaYw1hcIqcVgr6LSCRhCdZkaMINokJ8Mdd8Crr8K//qXyB1u3xltV8MQ3paWq\nliAIgoCEhnr3fXRdGcKDD8K0aXD77ZCe3rvvGQltbcoIsrMhM1MllgVBsBQSGrIKmgaXXqrCRboO\nZ56pjCHerXicTpUrOHgQdu9WxiAIQr9FjCBCejQfQUYGPPwwLFsGTz8Nl12mWhj1AhHr1DRVI3K7\n1XhFhw7FtPmrVeKwVtBpBY0gOs2MGEEsOflk1Rt5+nS4+GI1zLUxNES8MCa+qapSuYNdu6C6WmoJ\ngtCPMENwuO/mCLpi7141zPXHH8P998M558RPSyAej6oZeL2qpVFmpkqAO7qbw0gQhFgi/Qh6ihmM\nwKC4GBYsgGOPVSOdDh8eb0XtuN3Q2qpyGklJKsSVlKRaIAmCEFckWRwHem3O4kmT1BDX3/uemibz\nySd7FJaJqk6HQ9UGjFxCZaWawa2yUvVY7sHAdlaJw1pBpxU0gug0M2IEZiAxEebPhzffhM2blSF8\n8EG8VQXjcqmWRikpKnRUUQE7dqgQV1NT/FtCCYJwxEhoyGzoukoo33uvGrPorrvU/MRmRNeVKXg8\napju9HRlFomJ8VYmCH0eCQ31ZTQNzj1XTZOZkQFnnaVmSDPj/AKapgr9lBT1t6YGyspU+CjGzVEF\nQThyxAgipNdyBOFITYWFC9WsaH/7G1xwAfz3v92eFnOdBpqmEsmpqSqMFNgctaamU97DKnFYK+i0\ngkYQnWZGjMDsnHAC/POfcMUVcOWVyhzq6+OtqmtstnZTcDjUjGm7dqnaQm2tzK0sCCajf+YImptV\noeR0qrbyVuHgQTVu0caNKodw7rnWGieora29OWpysjRHFYQeIP0IokFzMxw4oJpBJiSocIZV2LJF\njXA6dCg88ACMGhVvRYdPa6syBl1Xo6Kmp6s8g00qqYIQCZIsjgaJiWregNxcVfjU1XXZfj9usfdQ\njBsHa9aoVkXnnQe/+51/WGlT6eyCoo8+Uknm1FSlvaJCJZn37TNVc1QrxIutoBFEp5npv0ZgkJSk\nzGDkSBVmqa+3xjg7TifccAOsXQv/+x+cfbbqpWxFEhKUKSQlKUPevVuZwoEDMm+CIMSA/hsaCkdj\no3oqbW1VtQarjLGzfr2aJnPsWJVQHjo03op6htfb3kfB4VBjHqWkWCuEJwi9iOQIehtdbzeEtjbr\nJDSbmmDJEnjhBZg3D2bPto6RdYXHo4zZ61U1IWMgPKcz3soEIW5IjqC3Mcbqz8+HnBxoa6Po7bdN\nMddvlyQlUTRxIvzjHypkNGOGGt3UhBxWLsNuV2ackqLyOfv2tTdHravr1eaoVogXW0EjiE4zI0bQ\nFZqmkpmjRqmn0NZWlUMwYy/fQI4+Gv7+d7j+evjZz+C229QcA30Bh6M9yazr8O23Kp9QUdHjgfAE\nob8ioaHDwetVT6D797cPzWz25o41NfDoo2r8ogUL1OxoVup7ECmBzVEHDFCLNEcV+jCSI4g3Xq8q\nYA8cUOvJyeYvXLduVX0PkpPhN79R8x/0VZqbVbjIZlOGkJamTMHs/yNBOAwkRxAHguKGNpsKFR11\nlBoZtLFRLSYwtLCx97Fj1TDX550Hl16qzKCxkeL167lr1izuveQS7po1i+L16+OrMxokJqrQUcfm\nqAcPHvZAeFaIF1tBI4hOM9MHmpTEEbsdBg5UT53V1WqgNWOcHTNit6uWRNOnw6JFFI8fz1q7nQf3\n7fMfcmdJCQCTJk+Oj8ZoYgyEB6oWV12tzECaowpCEJFUK6YBiwE78AzwSIf9hcDrwE7f+mvA/b7X\nJUAt4AHagHEhrm+90FA43G5lBlVV7S1dTMxdP/oRD3z2WaftdxcWcv+LL8ZBUYwInJfZ5ZLmqIIl\niWZoqLsagR34PTAZqAA+BFYCX3Y4biNwfojzdZRRHOqRSqvgcEB2thpMrapKPYE6HKadqMWRnBxy\nu722NsZKYozdrgp+UOa9b58K6yUmqhqelToSCkIU6C5HMA7YjnqybwNeAi4IcVxXrtQnMnSHFTd0\nOmHwYNUPITlZNTmN0VAJhxN7d4cZedXz2Wdq2OuXX1ZNMnsB04yJ1LE56p49Kp/gm5u56I031D0w\nOrSZEKvEtEWneenOCIYDuwPWy33bAtGBCcCnwL+A4zvsWw98BFzXI6VWxOVSQz3k5amnzLo6U83a\nNeWaa7gzLy9o24K8PM754x+VEaxaBaecAnPmwIYNfX8eAaez3RRcLlX419ZCZSWUlKg5mnftUvM0\n19SontxGk1VBsDDdPa1fgsoRGIX4VcCpwE0Bx6ShcgCNwI+AJ4Dv+PYNA/YA2cBbvvM2dXiPvpMj\n6I7mZpWsrK9XBY0J5kIoXr+et5Yvx97cjCcxkXOuvjo4UXzwIKxcCa++quZ5vuACuOQSOPHE/tkc\n0+NRhujxKAPQdXUfXC5V+0tMVIbicEh4SehVYpkjqABGBqyPRNUKAqkLeL0aeBIYiMoL7PFt3w/8\nAxVq6mgEzJ49m/z8fAAyMjIYO3YshYWFQHs1rU+sJyZS9M030NJC4fHHQ10dRZ98Ag4HhRMmqON9\nIZNYrXuTkzn7xhuD9hdt3ty+vm0bHHsshW++CTt2ULRkCcyeTeGAAXDxxRTl5UF2dtz0x3z9gw9C\n7x83Tv1TkvSfAAAgAElEQVQ/165tX7fZKPr4Y0hIoPDss8HppOidd8BuN8f3UdYttV5UVMRzzz0H\n4C8vo0V3buIAtgFnA5XAFuAKgpPFQ4B9qDDQOODvQD6QjEo21wEpwDrgPt/fQCxRIygqKvL/c6JG\nU5PqpdzcrGoHUWi1EliI9xq6Dh99BK+9Bm+8oTqnXXyxmjEtPd08OqNAj3Tquqo9uN3B+QW7XdUc\nkpJUTcKoQRxhL+he+W72AqIzusSyRuAG5gBrUYX6MpQJXO/bvxS4FPi579hGYKZv31BUU1LjfV6k\nswn0b4y5EIyRTuvrrdFiRdPghz9Uy333qfzBK6/A/feryXIuvRTOPFPa6GuaKuQ7GrzXq/IPRidE\nI7zkcLSHl1yu9vBSfwzBCTHFDN8wS9QIeh2rDn0dSE2NSjC/+ips26ZqCJdcohLOUph1j+QfhMNA\nxhrqy+i6aq64b58qFKxoCKASy6+9pkyhrU2Fji6+WA3LIRweRnjJGAZd11UYKSFBGYQRVnQ4rPld\nEY4IGWsoDsSsbXHg0NdDhx720NemaZ8/YgTMnQtFRfCnP6mms0YeYflyitasibfCiDDF/TQ6Jaak\ntDdvTUryD5tR9M9/qrkZAvo/UFVluv4PVmmfbxWd0UTql2ZF09QYRqmp1hv6OhBNg5NOUsvdd8Om\nTaqWsHYtTJigQkfnnGP64ThMR2D+wTAJkPyDcESY4VsgoaFI8HpV5yZj6OukJGv/iOvrYfVqFT76\n7DOYNk3VGE47zVpGZxUk/9DnkBxBf8bjUUnZQ77hm6xuCKBmGfvnP5UpHDrUnk/47nfjrazvE2n+\nwW5Xi9W/a30IyRHEAdPEDY2hr0eNUn8bG1V/BJ+ZmiKmHQFBOocOhRtugHXr4IUX1LarroIpU1R+\nYe/e+IjEGvezRxq7yT+wZ097/mH7djXExp49yrDr6w9rmA3T/Ia6wSo6o4nUAa1Kx7kQDh3qGy1G\nvvtdNaXm7bfDe++pfMKZZ8KYMaqWMH16ezxc6B3C9X+A9iG8Gxvbk9BGmMluVzUIl0uZi93eHmqS\ncJ+pMUM9L+ahoTZPGw2tDaQlpGG39YHCE9RTmQWGvj4imppUbeG112DLFjj7bJVknjhR4tlmwutt\nz0UYeQjDJDQt2CQMg5Amr0eM5Ah6SFNbE7uqduG0O8lKzmJAwoC+Ywitrap2UFOjfmR9rTWODIJn\nTXS93SS83s6hJJer3SQC+0RIy6awSI4gCjjtTpKcSRxsOsiuql1UN1Xj1cO3t7ZK3LBo82YVczfm\nQqiri9lcCIfDEce1Bw2Cq69W8y+/+qqamP6GG1T4aMkSZQ5m0BlDrKARTaNoyxZV0Ccnt+ckjMVm\nU9/TQ4dUDmL3bjX0t5GXqKhQDwF1daqG2Iv9I6zyW48m/bpebdNsJDuT8epe9jfu50DjAbJTsklL\nSMOmWdwjExJg2DCVRzhwQP2AXOYY+jpqjB4Nt94Kt9yiBsF79VXVDPXYY1UtYcaMiAfBE+KM0Sop\nFF6vCn02N3cu/O329r4UxgB+Rk1CQk4RY4Y6V1xCQ+W15aS4gpOOXt1LU1sTGlrfMQSD5mZlCI2N\n7dXwvkhrK7z9tjKFTZtkELy+jjHCq8cT3ATWyEsEmoTLFWwSFg85SY6gh4QzAgPDEGyajazkrL5l\nCL0w9LVpqa5uHwTv66/hvPNUyyMZBK//EGgSgclraG9UYdSUA03CAq2cJEfQy9g0GymuFBIcCexr\n2Meuql28ue7NLnMIZqHb+KYx9PWIEepHUVcXlykoYxLXzsiAK69UrY1Wr1ahsltvhdNPh9/+VrWN\nN4POHmIFjRAnnQ5He+c4o5+EkZdwOFRTWKO/hC8vUbRihfpulJerh6a6OlWTbm1tr3X0Mfp1jqA7\nDEPw6l6qmqrYVbWL7ORsUhNSrV9DSE5WCWVjpNPm5vYJU/ri0/LIkWoQvJtugs8/V/MnXHyxMsRL\nLoHzz4dBgyhev551zz6Lo6UFd0IC2aedZokJdIQjIFxeIjFRmYfbrYZ1qalRuQnjdzFiRJ9rjWeG\nX7zpQkPh8Hg9NLubsWk2spNVDkHrC4WmrisjqK1VC7Qn3voybjcUF6saw7//TfGoUaytrOTB/fv9\nh9yZl8fURYuC53EW+i8NDTB8uHqQijOSI+ghR2oEBoYh2DW7P4fQJwwB1JNPY6PqnNbU1D7uTF9v\ngVFfz13nn88D27Z12nX3wIHcP2mSaqo6YID6G/g6PT14PTXVEjFm4Qjoo0YgoaEI2bxpMxMmqhCB\n3WYnxZWCx+vh2/pvOdB4gMGpg0lxpsTdEHo836rNpgqy1FTVZK++vt0Uohg6Mt2cxampODIzO20u\nAuzZ2ao3c22tihfX1al27XV17bUo47URT05JCW0cxtLRTDr+PQwzMd29DIPoNC9iBD3AbrOTmpCK\nx+uhsrYSh93B4BRzGEJUcDohM1MtRuiopkaFkvpgE1R3mD4WnmHDVD4hUjweZaCB5lBT024ixn3c\nvTt4W6CpNDUpMwhXCwnctnevOr7jcUZHrSOkY75kyjXXSIisj2KG0spyoaFweLwemtqacNqdZKdk\n9x1DCMTrVYVOVZV68tW09gHGLE7x+vWsveceHiwt9W9bkJfHtHjkCDyeYJPo+LfjtlDHhTKTCGsk\nxZ99xtrHH+fBsjK/JMmX0GdDQ2YopfqMERi4vW5a3C04bI6+awigkq0NDWpYgNbW9nbZFv6sxevX\n89by5dibm/EkJnLO1Vdbt+BzuzubRHeG4vt7V2kpD7S2drrk3Wlp3D92bHv40GiOaRhOYBNNI8QV\nuM3qDwx91AgkNBQhgTmC7nDYHDhcDtxeN5V1lThtTganDCbZmdzrhtDjHMHh4HCoRGl6ugod1der\nNtm63t5+O5xOk8ZhJ02eHFTwW6GNfth76XC0h/YOE8cll8D773fabs/NVWM71derQjEwBFZZqdaN\nxdjvW4oaGylMSgo2CsMkujOWUPt7KTQZ7n4Wr1/PuqeewqFpuJOTmTJ3LpNmzOgVDbFGjKAXcdgc\npLpScXvdVNRV4LK5yE7JjokhxJzERLUMHKhMobpaFQDG8MMyXLSlCJsvyc6GI33QePddNa9EXV0n\nkwgyj7q69olvujAWHI6uayAdzSOcsaSmdluTDRU2vHPHDoA+YQZmKI36XGgoHG6vm2Z3s98QYv3+\nMccIHVVVqdCRMXGJNK00PabKl4TC6PtiGEd3xhJqf+A2t7tzGCvAWO56910eqKzsJOPuqVO5f82a\nONwACQ1ZFqOG0OZpo7y2nAR7AoNTVcioTxIYOmppaQ8deTx9b/KcPoZR2N8dkC+ZZqZ8iaap3r1J\nSZCV1fPrGU2lDePoYBSOLVtCnmY34RDvR4IYQYQcTo6gO5x2J067UxlCTTkuuytqhhDTHMHhkJCg\nloEDoamJotWrKTzppPZ9Jg0dmTWXEUhvaeyYL+kppr6XAU2lQ+l0r1wJAbUjA08feZiROnoccdqd\npCakomka5TXllNWU0djWGG9ZvYumqRYXgwbBUUepSXR0vX0i9F6abEQQesKUa67hzry8oG0LRo/m\nnJtuipOi6CI5AhPR5mmjxd1CojORrOSsvhsyCkVra3voyO2W0JFgOorXr+etp5/GDnhSUjjnppvi\nmiiWfgQ9xKxGYNDqaaXF3UKSM4ns5GySnH1rpMMuMZKANTXtA+D19XkTBOvQR/sRSGgoQjZvil17\ncpfdRVpCGl7dy+7a3eyu2U1TW1NE51plvtWwOo0k4NChairKnBy1rb5e9WSOcejIKv0IrIDoNC/m\nzNAJgDIEl91Fq6eV3bW7SXIkkZWc1X9qCHZ76AHw2tr6RC9mQTALZvgVSWgoQlrcLbR6WklxppCV\nkkWiox/G0HVdNUXt4wPgCSalj4aGpEZgIRIcCSQ4Emhxt1BaXdo/DcEY5C4xUbUfNwbAM3ox95EB\n8AQhlkiOIEJimSPojgRHAmkJabR52yirLqOitoJmt+rYYvkcweFgs6keoCNGwKhRMHiwanFkNEWN\nQk3TCvFiK2iEGOrUdZVL8njU98HtVuHE1lZVm2xuVt+PpiaVd2psVE/6vqXo7bfbO5MFbKehQV27\nD4YjpUZgYTrWEFJdqbR6Oo8Y2S9wOoMHwDPmAPB61b4uBsATegldbzdjXVcFc1tb+/bA/R2PN9Yh\nfMEbar+mqQcEY5vNpha7vf21sQTuN87RNFXTHD68fb3jYtLOjz3BDNYmOYIo0exups3TRlpCGoOS\nBpHg6OeFnzF3QnW1epqz2VQuoQ/+kI8Yr7e9ADZeGy2zImmhFe4J2SiQA193VxB3LJCNc8Mtofb3\nI2Ldj2AasBiwA88Aj3TYXwi8Duz0rb8KPBDhuSBGEHUMQxiQMICBSQPFECD0AHhWbnXUseAOVYB3\nFcbQ9c5Pyw5H579dFcRdFdZCrxNLI7AD24DJQAXwIXAF8GXAMYXAL4Hzj+BcsIgRRHOsod4kUGez\nuxm3102aK810hhDXMZFaWlToqLq629BR1MfHCSy4O77uGBoxjte04ELdeML2FeBF779P4aRJwYW4\n3R4cJun4Nw4FtmnHweqAVXTGstXQOGA7UOJbfwm4gM6FeSgxkZ4bU1a9tYrFLy6mpq2GZHsy11x6\nDZPPNMmIilHGaE3U2NZIbUstAxIGMCh5EC57P29qaQyAN2hQcOjI2BcqdBQqfBKqAO8umWgU0A6H\net3xKdzY310hHsiOHTBkSHTujdAv6c5NLgWmAtf51q8CTgUCR1oqAF4DylFP/rcC/4vwXIhhjWDV\nW6uY94d57Dh5h39b3n/yWPSzRX3WDAJpamvC7XWTnpDOwOSBYgiBuN2q9Ygx7WbHp/DAAjyw4O74\n+nAKcEHoAbGsEURSQn8MjAQagR8B/wS+00NdvcKSFUuCTACg9AelLH91eb8wAqNHckNbA7VVtaQn\nppOZlCmGAKowNyZ2b2vr/FQuCH2Y7oygAlXIG4xEPfkHUhfwejXwJDDQd1x35wIwe/Zs8vPzAcjI\nyGDs2LH+GJ3R3jwa6y16C+zyveko399d8OnXn7JuxzomjJzAZx98BuCPswf2H5gwcYJ/veN+s6w/\n/YenOeGkEyI6vqG1gXX/XkeKK4Vpk6fhsDnYtHETdpu9V+5/4LqxrbeuH631xYsX99r3MVrrW7du\n5eabbzaNnnDrHf/38dYTbt2s97OoqIjnnnsOwF9eRovuHnUcqITv2UAlsIXOCd8hwD5U7WEc8Hcg\nP8JzIYahoalXT2Vd/rpO24/65ChyLsjhkz2fcOLgE5mUP4nCvEK+N/h72G2ql6oVk8WR0tymksqB\n8ygb4xy57C4SHAnYNTt2mx2HzYFN63k/RKsk5Kyg0woaQXRGm1g3H/0R7U1AlwEPAdf79i0FbgR+\nDrhR4aFfAu93cW5H4psj+CiPRdepHEFjWyPvl7/PxtKNFJcWs79hPxPzJlKQV8CkvEnkpOXERKcZ\n8Hg9eHQPHq8Hr66aI+q6jqZpaJqG0+Yk0ZGIy+7CaXfisDn8ZhENoxAEoWtkPoIesOqtVSxesZia\n1hpSHClcfcnVYfMDlXWVFJcWs7F0I5tKN5GVnEVBfgEFeQWMHzG+f00cE4Cu60EmYRiFsc9hc+C0\nO3HZXSQ6EpVJ+GoTds0eVPMQBOHIECPoIUfSj+Cdje+QdmwaRaVFFJcU8/m+zxk7dCyF+YVMypvE\n8dnHm+JJ2AwhLMMo3F43Xt2Lruv+2gSAXbPzn/f+Q2Fhob9GYdQmzGYUVggTWEEjiM5oI6OPxgGb\nzcaYoWMYM3QM806dR31rPZt3b2ZjyUZuePMG6lrrmJQ7iYJ8FUYanDI43pLjhqZpODQHDlvor5dX\n9+LRPdS11uHxevzbDbNw2Bz+2oTL7gqqTRg5G6Fn6LqOju43aq/uRUcPeh1Y4zPuvaZpaKjwoE2z\n+V9r+NYD9gvWwQz/LUvUCLqjrKbMH0Z6t+xdctJyKMgroCC/gHHDx/WvoaJ7iFf34va68Xg9/oIp\nXCLbZXcFhZ7MUCvrbfy1rBAFubFu1MY6/vV4PXhRhbvWzc/fKNQBv0kYBNbwjP0dr6dpGnZN5YyM\nxfgfBS52m71LQwncJrQjoaEe0ttjDbm9brZ+u9VvDF/u/5If5vzQ3xrpO4O+I1/qHmDVRHZ3hbfx\n9B1YaHt0T9h8TLj3MO5Dxyd3ozCN1T0wPm/Hzx1oKh0/TyhDCUTTNGzY/P/LrgymK0MJvDdWRYyg\nh8R6rKGa5hre3f0uG0s3srFkI22eNiblT6Igr4CJuRMZlDzoiK4bbZ2xpLd0RjuRbcSLuyq8jXWP\n1xOy8Daewo3vuVFYG4Ve4NN1uIIrMAwTq3sZbaKls6OhBIa1oHPtBTrXYAK3GeZihCXff+d9zph0\nhqqtBDxAGP+LZGeyKQxEcgQWIz0xnenHTGf6MdPRdZ2S6hI2lm7k9W2vc/v62xmVOYpJecoYTsk5\nRXr69oDu8hOGUTS0NVDXWhcyke20O/0mUlFbwTcHv+my8DYwCofAwtum2XA4HFIDjCI2zRb1R9ig\n3IjuocXT4jccaDeXNm8b+Rn5fa7FoBm+nZaoEfQWbZ42Pt7zsb810vaq7Zw6/FR/a6TRmaOlEIkh\ngbWIrp7Chf5JQ2sDwwcMN4URSGioh5jJCDpyqOkQ75S9Q3FpMUUlRdg0mz/pfEbuGWQkZsRboiD0\nW/qqEcQ/0GURYjVn8cCkgZx/7Pn8dspv+fC6D3nx4hc5NutY/vbF3zj1mVM5d8W5/L93/x8fVnxI\nm6ctbjp7iuiMHlbQCKLTzEiOwMRomsYxg47hmEHH8LPv/4wWdwsfVn5IcWkxd224i7KaMiaMmOBv\njZSXkRdvyYIgWBAJDVmYA40H/E1Ui0uLSXYk+1sjnT7ydNIS0uItURD6FH01NCRG0EfQdZ2vDnyl\nmqiWbuQ/lf/hhMEn+FsjjRkyRnrlCkIP6atGIDmCCDF73FDTNI7LPo6Tmk7ir5f8lU9v+JSbT72Z\nupY6frXuV5z0p5O4/s3rWfH5CipqK/znrd+wnlk3zuKSn1/CrBtnsX7D+pjoNfv9NLCCTitoBNFp\nZiRH0EdJciapkVLzC6AAvq3/luLSYopLi3n4nYfJSMzgqJqj+HTzp+wbv89/XskzJQD9YsY2QRAU\nEhrqh3h1L1/s+4Kf3/Jzdn1/V6f93/n0Ozxw/wPkZ+YzLHWYKXpRCoIZ6KuhIakR9ENsmo0Th5zI\nkLQh7KKzEVS1VPHYe49RWl1KdXM1I9NHkp+R32kZMWBE2B68giBYB/kVR0hfHM8lwZYQcvsJWSfw\n4uUvAtDY1khpdSkl1SWU1pTy5YEvWb19NaXVpexr2Mew1GF+Y8jLyCM/I59RGaMYmT6yyxFX++L9\njBdW0AjW17l+w3qe+vtTaDaNZHsyc2fNZcY5M+KgMPqIEfRjrrn0GkqeKaH0B6X+bXkf5XH1dVf7\n15OdyRyXfRzHZR/X6fwWdwu7a3f7jaKkuoRNZZsoqS6horaCQcmD/MaQl65MIj8zn/z0/Fh8PEGI\nGus3rOeeZ+4J+q3s+IOa8rYvmIHkCPo56zesZ/mry2n2NJNoT+xy6s7Dwe11U1lX6TeIkuqSdsOo\nKSHNlRZUkxiVMUq9Ts8jMykzCp9M6CnrN6zn2VeepcXbQoItgWsuvca0jQiMORhaPa20eFpo87TR\n6mml1dtKq7uVNm9b8HbfYqx3d87ap9by7anfdnrfqaVTWfPsmjh8YskRCFFk8pmTe+XH7bA5yE3P\nJTc9l0l5k4L2eXUve+v3UlqjjGFX9S7WbF/jNwy7Zg8KNflrFRl5ZCdnyyBwMSDUE3DJMyW4PW7O\nmHRGUEEaqoBt9XRd+LZ4WoLP97bR6vYVwgEFdKgl3LF2mx2nzUmCPcE/1LjTHrwe+Nplc/mPCZzo\nyDgnyZnkf12cUBzyPjV7m2P1L+lVxAgixOrxTTNh02zs2rqLCRMnMH7E+KB9uq5T1VzFrqpdQeGm\nv3z2F0qqS2h2NytzSM/vVKMYlhb9Fk5WuJ+hNOq6TrO7mYa2Bupb66lvraehtf11fWs99W0htvmO\n++9L/6V+Yn3QNUt/UMq1i68l+cvk4IKzQ+EbWKAGFr6HvjxE7km5QYVtsjM5/Dk2Jy5HcIEd6n0M\nDdHqMLl502YmnBZ8P9elrQvZsCLR1jdmHhQjEEyFpmkMTBrIwKSB/CDnB53217bUUlpdyq5qZRQf\n7/mY1758jZLqEqqbqxmRPiK4FuHLTYwYMAKn3dnt+3cMh5x27GkxMwKP1+MvuP0FtK+wrmutCyq0\nG1obqG9Tr8s/K8dR7vDvN/bZNBuprlRSXamkOFPaX7tSSHWmkpqQSqozlezkbPIz8v37U12pLHpr\nEZ/zeSeN40eO59WbXj2iz7c52fymGo5Q+bTRH4/mpjk3xVFV9DBDHVtyBEJUaGpr8oebgnITNaXs\nrd/L0NShIfMSuem5JDoSQ4ZD8v6Tx6KfLQobPmtxt6hJblrqOj1hdyrAwz2B+wr/ZnczSY4k0lxp\nqrA2Cm1XalDB7d8WolAPPKcnExzNunEWG0dv7LS9cGchL/7+xSO+rpVZv2E9T7/8NGiQ4kjhpitu\nimuiWMYa6iFiBP2PVk8ru2t3U1JVEpSbKK0upby2nIFJA2lc20jN6TWdzs16P4vvXf499QTeUu9/\nEm9obQAIepI2CubAwrpTod1xm++1WaZAhNA5gryP8lh0XXhT7A9Ih7J+jhVixSA6w+GyuxidOZrR\nmaM77fN4PVTWVXL1+1dTQwcj2AWZyZlc+/1rg57UjXBLgiN0X4xY0hv30ijsg1qUXdezFmXy3TQv\nYgRCv8duszMyfSSDkwbzJV922j88dThnjTorDsriS2+1KBPMh4SGBMGHhEOE7pDQkCD0cXojHCII\nVsAcmak44NW9uL1uPF4PXt1Ld7USq4xRLjp7xuQzJ/Pi71/k1T++you/f5FkR/yf/LrDrPeyI6LT\nvPTLGoHD5iAtIc1vAl6vMgIvXry6Fy1Ebau5rZmG1gZlGBr+YzRN6/Taptn8rwP/CoIgmBEzlE4x\nzxF0ha7r6Ogh/3p1b8htgYvH68Gje4K2GSZjfE4dPaTZBOlAV4bSwWQC/xr7xWQEITZIjqCf4H/C\n76Wy1TCGSMymo8l4dS8e3eOvybR52zqZjEEkZmO0WTdMxWFzmKYduyAIsUOMIEKKioooLCzs8XVs\nmq3XTEbXdTYUbWBSwaRuzabj0upppcXdgtvr9tcwNDRsNmUQds0e1ZqHVdpqW0GnFTSCNXR6dS/v\nFr/LuNPH+X8bgeh69w9YVkSMoA8R+GR/pBhJdLfX7R/VsdndTJO7KSi0ZdNs2DW7MokoDfYlCL2F\nEbYNDNfqdA5J2zU7GhqJjkRcdhcOm8P/HTe+833x+24GazNVjkAIj8fr8ZuEYRDN7mbavG1BT0mG\nGTlsDslfCL1KYF7OqO2C78k94LtnfB+Nwt1ld6mC3WbHrqlC3mjkYRVkrCHBVBiTgri9bjy6hxZ3\nC83uZlrcLf4fKKgvrt1m9/8oBSEcxtO70fBC1/VOhbumadg1e1Dh3vHp3Sjs+yKxNoJpwGLADjwD\nPBLmuB8C7wGXA8Y4tSVALeAB2oBxIc6zhBFEK0fQ25hNZ8dQk1GLeKf4Hcad3v51MH6wZktYWyGu\nbQWNAO8Wv8v4M8YHt6rzetE0zd+4Qdd1lZfSHP55Bxw29bpj4d5b3xOz/YbCEctWQ3bg98BkoAL4\nEFgJnQZksaMMouOcbTpQCBzqqVDBmtg0m3/yEJyQTjoAuwbsYlTmKH+4KTDU5Pa6/fFbo1+GhJrM\ni67r/sLd4/UENU4wCndN02j1tOLVvf6JZFx2V1BoxijczfQg0F/o7ld1GrAQVSsAuN339+EOx90M\ntKJqBW/SXiPYBZwCHOziPSxRIxBiR2Coye11+8NMrZ5WPLrHf5wRGpCEdXTpqhlz4DHGk7xDU/ff\nKOCddqe/ZhcYphETjy6xrBEMB3YHrJcDp4Y45gLgLJQRBJbqOrAeFRpaCjzdE7FC/0DTNJx2p39G\nsbSENP++cKGm5tbmoFyEWUNN0aS7zo7G36Bz0EEnqFAO3KZpGjZsnfqWBLag6ZhklQLe+nRnBJE8\nqi9G1RR0lDsFfitOB/YA2cBbwFfApo4XmD17Nvn5+QBkZGQwduxYf4yuqKgIIO7rxjaz6Am3vnjx\nYlPev2jdz+KNxe3rTvik6BP/utvr5u0Nb+Pxejj1jFNpdjezsWijfx1gyztbQIMzJp2Bw+bgvXfe\nA/DH2I1xZoz1p//wNCecdELY/ZGs67rOaRNPQ9d1tY7Oqaefio7Oe5vU+xv5kg/e+QCAU884FV3X\n+eDdD9DQ/Po/eOcDdNT1bNj44N0P+PK/X3LdL67zfx4bNiYWTMSm2di8aTM2zcakgklomsamjZvQ\nNI3CwkI0NP/9PPPMM7Fptl79/3f830f7+tFa37p1KzfffLNp9BjrRUVFPPfccwD+8jJadGfl44F7\naQ8N3QF4CU4Y7wy4ThbQCFyHyiUEshCoBx7rsN0SoSGrJJBEZ2dChZqa3c20eFr84Q5jqI7AduJG\nwW0UyqCenr26t9NTtXF8qKdjIyxixL8Dn7RDLR2HE+lu7Cr5n0cXq+iMZashB7ANOBuoBLYAV9A5\nWWywHHgDeA1IRiWR64AUYB1wn+9vIJYwAqFvEtg3os3TRotHNX2NpNDubvwnGQdK6E1imSNwA3OA\ntahCfRnKBK737V/axblDUYZgvM+LdDYBQYgrdpuqASQQ/yknBSFeRJJFWw0cCxwNPOTbtpTQJnA1\n7YX/TmCsb/lewLmWJDC+aWZEZ3Sxgk4raATRaWb6ZnMKQRAEIWLMEMCUHIEgCMJhEs0cgdQIBEEQ\n+jliBBFilbih6IwuVtBpBY0gOs2MGIEgCEI/R3IEgiAIFkRyBIIgCELUECOIEKvEDUVndLGCTito\nBCR5npAAAAYHSURBVNFpZsQIBEEQ+jmSIxAEQbAgkiMQBEEQooYYQYRYJW4oOqOLFXRaQSOITjMj\nRiAIgtDPkRyBIAiCBZEcgSAIghA1xAgixCpxQ9EZXayg0woaQXSaGTECQRCEfo7kCARBECyI5AgE\nQRCEqCFGECFWiRuKzuhiBZ1W0Aii08yIEQiCIPRzJEcgCIJgQSRHIAiCIEQNMYIIsUrcUHRGFyvo\ntIJGEJ1mRoxAEAShnyM5AkEQBAsiOQJBEAQhaogRRIhV4oaiM7pYQacVNILoNDNiBIIgCP0cyREI\ngiBYEMkRCIIgCFFDjCBCrBI3FJ3RxQo6raARRKeZESMQBEHo50iOQBAEwYJIjkAQBEGIGpEYwTTg\nK+Ab4LYujvsh4AYuOYJzTY9V4oaiM7pYQacVNILoNDPdGYEd+D2qQD8euAI4LsxxjwBrjuBcS7B1\n69Z4S4gI0RldrKDTChpBdJqZ7oxgHLAdKAHagJeAC0IcdxPwCrD/CM61BNXV1fGWEBGiM7pYQacV\nNILoNDPdGcFwYHfAerlvW8djLgD+6FvXA7Z3d64gCIIQZ7ozgkia8ywGbvcdq9Gexe5TTYFKSkri\nLSEiRGd0sYJOK2gE0Wlmumt6NB64FxXnB7gD8KLyAQY7A66TBTQC1wH7IjgXVPho9GErFwRB6N/s\nAI6OxRs5fG+WD7iArXSd8F0OXHyE5wqCIAhxwNHNfjcwB1iLagW0DPgSuN63f+kRnCsIgiAIgiAI\ngqAwS4ezkcAG4Avgv8Bc3/aBwFvA18A6ICPgnDtQur8CpsRMqcIOfAK84Vs3o84MVJPiL4H/Aaea\nVOcdqP/758AKIMEkOp8F9vp0GRyJrh/4rvEN8EQMNP4/1P/8U+A1ID3OGsPpNLgFlbscaGKdN6Hu\n6X8JzrHGS2dUsaMSxfmAk/jmEIYCY32vU4FtPi2PAr/2bb8NeNj3+niUXidK/3ZiO1zHL4EXgZW+\ndTPqfB64xvfagSoQzKYzH9XYIcG3/jfg/zOJzonAyQQXCoejy2jAsQXVpwfgX7Q33ugtjefQfk8e\nNoHGcDpBPQCuAXbRbgRm03kmyvydvvVsE+iMKqcR3BP5dt9iBv4JTEY57RDftqG+dVBOHFiDWYNq\nYRULRgDrUV8Qo0ZgNp3pqAK2I2bTORBl+pkos3oDVZCZRWc+wYXC4eoaRnBebibwp17WGMhFwAu+\n1/HUCKF1vgycRLARmE3n34GzQhwXVZ3xHHTOrB3O8lGu/AHqR7fXt30v7T/CHJReg1hqfxz4Fao6\na2A2naNQvcyXAx8DTwMpJtR5CHgMKAMqgWrU05fZdBocrq6O2yuIrd5rUE+khNASb40X+N73sw7b\nzabzGGAS8D5QBJzi2x5VnfE0AjN2OEsFXgXmAXUd9ul0rTkWn+dcVP+MTwjfB8QMOh3A94EnfX8b\n6FzbM4PO0cDNKPPPQf3/rwqhI946w72vGX9DBncCrai8i9lIBhYACwO2mWFI/lA4UDXW8agHwL/3\nxpvE0wgqUDE6g5EEO1mscaJM4C+o0BCop66hvtfDUIUwdNY+wrett5kAnI+qyv4VVWX8iwl1lvuW\nD33rr6AM4VvMpfMUYDNwENXc+TVUyNJsOg0O5/9c7ts+osP2WOidDUwHrgzYZiaNo1Hm/ynqtzQC\n+A+qhmUmnfje+zXf6w9RkYAszKfziDFThzMN+DMq7BLIo7TH4W6nc+LLhQqD7CD2TxQFtOcIzKiz\nGPiO7/W9Po1m0zkG1RIjyfd+zwM3mkhnPp2TxYer6wNUiy2N3kkcdtQ4DdUKK6vDcfHUGEpnIKGS\nxWbReT1wn+/1d1BhTDPojCo/QiXrtqOSH/HiDJTTbkWFXT5B3byBqMRsqOZ6C1C6vwKmxlKsjwLa\nWw2ZUecY1BNMYDNCM+r8Ne3NR59H1QzNoPOvqLxFKyqXdvUR6jKaEm4HlvSyxmtQTRZLaf8dPRln\njYE6W2i/l4HsJLj5qJl0OlG1/s9RtZZCE+gUBEEQBEEQBEEQBEEQBEEQBEEQBEEQBEEQBEEQBEEQ\nBEEQBEEQBEEQrMr/D4cvKVcoGfzDAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10f812410>"
       ]
      }
     ],
     "prompt_number": 42
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# One-hot encoding of categorical features\n",
      "\n",
      "Scikit-learn also offers convenient preprocessing functions for the most common cases. Let say that you have categorial data of the following kind:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "measurements = [\n",
      "     {'month': 'January', 'temperature': 33.},\n",
      "     {'month': 'March', 'temperature': 12.},\n",
      "     {'month': 'May', 'temperature': 18.},\n",
      "     {'month': 'January', 'temperature': 12.},\n",
      " ]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 43
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[DictVectorizer](http://scikit-learn.org/stable/modules/feature_extraction.html#loading-features-from-dicts) can be used to transform lists of dictionaries. Categorial variables are transformed using one-hot encoding. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.feature_extraction import DictVectorizer\n",
      "vec = DictVectorizer()\n",
      "vec.fit_transform(measurements).toarray()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 44,
       "text": [
        "array([[  1.,   0.,   0.,  33.],\n",
        "       [  0.,   1.,   0.,  12.],\n",
        "       [  0.,   0.,   1.,  18.],\n",
        "       [  1.,   0.,   0.,  12.]])"
       ]
      }
     ],
     "prompt_number": 44
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Vector quantization\n",
      "If you need to discretize a continuous variable into a smaller set of discrete values, a possible approach is to use K-means. The `fit_predict` functions computes the clusters and returns the closest cluster centroid for every input point. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.cluster import KMeans\n",
      "\n",
      "clst = KMeans(n_clusters=4)\n",
      "centroids = clst.fit_predict(iris.data).reshape(iris.data.shape[0], 1)\n",
      "print iris.data[-10:]\n",
      "print centroids[-10:]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[ 6.7  3.1  5.6  2.4]\n",
        " [ 6.9  3.1  5.1  2.3]\n",
        " [ 5.8  2.7  5.1  1.9]\n",
        " [ 6.8  3.2  5.9  2.3]\n",
        " [ 6.7  3.3  5.7  2.5]\n",
        " [ 6.7  3.   5.2  2.3]\n",
        " [ 6.3  2.5  5.   1.9]\n",
        " [ 6.5  3.   5.2  2. ]\n",
        " [ 6.2  3.4  5.4  2.3]\n",
        " [ 5.9  3.   5.1  1.8]]\n",
        "[[2]\n",
        " [2]\n",
        " [0]\n",
        " [2]\n",
        " [2]\n",
        " [2]\n",
        " [0]\n",
        " [0]\n",
        " [2]\n",
        " [0]]\n"
       ]
      }
     ],
     "prompt_number": 45
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can further transform this representation into one-hot vectors if needed."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.preprocessing import OneHotEncoder\n",
      "enc = OneHotEncoder()\n",
      "enc.fit_transform(centroids).toarray()[-10:]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 47,
       "text": [
        "array([[ 0.,  0.,  1.,  0.],\n",
        "       [ 0.,  0.,  1.,  0.],\n",
        "       [ 1.,  0.,  0.,  0.],\n",
        "       [ 0.,  0.,  1.,  0.],\n",
        "       [ 0.,  0.,  1.,  0.],\n",
        "       [ 0.,  0.,  1.,  0.],\n",
        "       [ 1.,  0.,  0.,  0.],\n",
        "       [ 1.,  0.,  0.,  0.],\n",
        "       [ 0.,  0.,  1.,  0.],\n",
        "       [ 1.,  0.,  0.,  0.]])"
       ]
      }
     ],
     "prompt_number": 47
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Scaling"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Certain algorithms require the data to be distributed more or less like the normal distribution. The process of transforming the data so that it has unit variance and zero mean is called *standardization*. We can apply standardization in one line with Scikit-learn: "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.preprocessing import scale\n",
      "\n",
      "mu1 = np.array([0.5, 1.5])\n",
      "sigma1 = np.array([[1.0, 0.5], [0.5, 1.0]])\n",
      "X = np.random.multivariate_normal(mu1, sigma1, 1000)\n",
      "plt.plot(X[:,0], X[:,1], '.')\n",
      "\n",
      "Xs = scale(X)\n",
      "plt.plot(Xs[:,0], Xs[:,1], '.')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 48,
       "text": [
        "[<matplotlib.lines.Line2D at 0x10fded290>]"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXt8VPWZ/99zyySTy5zJjZAGQlREKGgCw6U/aElbrJJK\nk9VOa3GL2paxN7XWQm23u9rfutt2+XXb3V6saS20XVMbsNKim20TlahsxUtRUChRkCjEEC4zuV8m\nyfn98Z1zm0syCUMucD6vV15JZs75fr/ne855vs/3eT7P84AJEyZMmDBhwoQJEyZMmDBhwoQJEyZM\nmDBhwoQJEyZMmDBhwoQJEyZMmDBh4iKFJUntHAM6gCEgBCxLUrsmTJgwYeI84y0ge7IHYcKECRMX\nE6xJbCtZ2rwJEyZMmEgAyRLgMtAAvARsTFKbJkyYMGFiAjAz/DsPeAV4/ySOxYQJEyYuCtiT1M67\n4d+ngMcQTsxnAS699FL5yJEjSerGhAkTJi4aHAEuG+mAZJhQXEBm+O904CPAAXUER44gy/KU/7n3\n3nsnfQzmOM1xTtcxmuNM/g9w6WjCNxka+AyE1q209zDw5yS0a8KECRMmRkAyBPhbQGkS2jFhwoQJ\nE2NAMmmE0xrl5eWTPYSEYI4zuZgO45wOYwRznJOBieBuy2F7jgkTJkyYSBAWiwVGkdGmBm7ChAkT\n0xSmADdhwoSJaQpTgJswYcLENIUpwE2YMGFimsIU4CZMmDAxTWEKcBMmTJiYpjAFuAkT0wB+P5SX\nQ0UFBIOTPRoTUwWmADdhYhqgqQkaG6GuTghzEybAFOAmTEwLuFzit9cL1dWTOxYTUwdmJKYJE9MA\nwaDQvKurQZImezQmJgKJRGKaAtyECRMmpiDMUHoTJkyYuIBhCnATJkxcsLjQ2TumADdhwsQFiwud\nvZMsAW4D9gG7ktSeCRMmTJwzLnT2TrIE+J3AQcD0VpowYWLKoKYGfD6or78w2TvJEOBFQAXwCyaG\n1WLChIlxYrrZhM91vJIEtbUXpvCG5AjwHwCbgOEktGXChInziOlmE55u451onGtR4+uANoT9uzze\nQffdd5/6d3l5+QVVk86EiemE6WYTnm7jPRfs3r2b3bt3j+mcczV5/CvwaWAQSAWygEeBDbpjzEAe\nEyaSCL9faKYul7DxjsU8MN0iOqfbeJOJiY7EXA18DVgX8bkpwE2YSCLKy4VZAYSDrrZ2Uodj4jxh\nMiIxTUltwsR5hmJWyMiAQGB6OCNNnB+YuVBMmJggnIvpQ49gEC6/HE6dEv9PpBaerGswMToS0cDP\n1YlpwsRFh/EKMYVRobQxXqG7eTMMDoq/S0sn1rm3axe0toq/b7kFdu6cuL5NRMMU4CZMjBFjEcR6\nYe9wiM/OlVHR1CRMJwBz5ogFZKI04/5+7W+LGfUx6TBzoZgwMUaMhdqm5zGnpycnKlBvA+/qEiaV\nieJLL1kifpeVwdat568fE4nBtIGbuOgxVu11LNS2igohVL3e5IVzx7KBd3Ulv594fY+X1mfaz8eG\nRGzgEwHZhImpjNWrZRnEj88X+5iNG8Vxa9fKciCQeNuBgGhzLOckgrVrxXi9XtH2+eonmUhknk1o\nIAFWn6mBm7jokYiWPNW414omnJYGzc3TQ6udNQuOH4esLNi/H4qLJ3tEUxtmRR4TJhJAIhnrplpI\nt5Kkqbl5+uQKUQR2Rwds2jS5Y7lQYApwExc9EslYN1XTkk70wnIu2QGzssTvqbIIXggwTSgmTExj\nnK9cIfEcjnpTUm4uLF2qfT+ak/JizmsyHphV6U1MW0w0Y2Gs/U11RoV+fHl5Y7eTx7P5K/4ChcKo\n/z7yHEma2nM01WGyUExMW0w0Y2Gs/U11RoV+fHl5Yx9rJMtFgcJ2WbPG+P3GjbLs8YjPSkvFZ1N9\njqY6SICFYtrATUxJTLRtd6z9JXr8ZFXA0Qf7DA2Jv8cyl3l5wkQSqTUr/oLt240+gV27tOjQwkLx\nmTKG3Fxoabl4qgBdaJjshczENMRE85rH2l+ix4+mhY6XXx4PSntr1shyTo7Wd1HR2NqPHPdo41S0\nb5DlqirxmTJHK1dOL0080Z1Dsu9dJJgi2V2Tf2UmTEwTxDNFKEi2sCgo0NrLz4/uO9F2lHFnZIjF\nQC+E8/Kiz1dMKmVl0e2ONgdTDYmO93ybiDAFuAkTE4N4gnE0TT3ZwkKvCVdURPcdq51YYw8EZDk3\nVztWWRgyMmKPY6TrnA5RonokOt7zvTBhCnATJiYGesFYUiIEnscjNNORXu5kCwtFE1YciYm0ox97\nRoYsu91CeCufe72yfOxYbOflZOF8my8SwflemJgAAZ4K7AVeAQ4C3zEFuImLEXrBqDc3JGt7HSks\nYgmwjRtF3wUFQuAm0k7k2B0Oo9bt88nyhg1aX4ogn2xt+mJguEyEAAcI+5qxA88Dq0wBbmIyMJla\nmV4wKgIxUhMeaXyR3412LbEE2HiFmn7sdrvRBHMu7UZeW1GRWGCScX8S3ZFMBU19vJgoAa4X5C8C\nC0wBbmIyMBlaWTz7cVWVLFdWjm5/jvfdaNcSS4Alwyar9HvllWNrdyRBqb+WZN2fZLGApjImSoBb\nESaUTuDfYnw/2fNgYoojWVrSZLAdIm3fynVs2LFRXr11tbz2v9bKgd7AqOOL/G60a4klwJJhkx1P\nuxs3Crt5PEGpXEtW1sTfn+nGgNEjEQGezDBNN/An4B5gt16A33vvveo/5eXllJeXJ7FbE9MdyUrV\nOlHFBvTHhkLQ0CCCZJxO2LNHHJO7qZzT6eKifAt81PpqRxxf5HejXYsyhiNHRJa/rKzJC1fX3z+P\nB44eNY5DuZYtW0QWwonMhTKd8q/s3r2b3bt3q/9/+9vfhgkOpf9H4GsRn032QmZiimMqaElj2Wrr\nudZ6qp7+OtZsXStzH7K32qtq4OeCyF1KsswSo+1+EtkdKdft8cR3npoYO5gAE0ouoKxracAzwIdN\nAW5iLJgKPOGxLCKxog5l2Xgdgd6A7Kv1yYHeQFJMRJELTDLMEvFMH/rx6hk1ehPRSCaW8VzvVHA2\nns8xjKftiRDgi4C/Imzg+4FYadqTOxMmLmqMha0xlpdmLIvISFGHsZAMR1q8EmrnQuvTj8tu17Rn\n/efKbiOSHjnSdYznehMNMDqfOJ8Oz/G0PRECPBEkdyZMTComW1Maia1RUBCf9REr/Hu8GKvGmQwT\nUSKORNWBuiGatqfnckeySyIFi368+gUi0esYz/WOFmA0EQyS82nKG0/bmALcRLIx2bSseGyNWGPS\n5/M4n2MebU5iCd+xLISJHKsfgz4EPtZnyhgDAaOGradBKuPV952otj8ek1ggIMsld2yUV/5cY+4o\n9y83VyxEyeSRJ2vc57NtTAFuQsF0purpEfkixBNC+mMTCf/euDHx8PdIjGdOxpKlMBHTRVGRZhNX\n2lbs216vLBcWat/rHY2jCZbRdjFjnbcR+eJbV8vch8x9CP9BIDqb4WQoD5O168QU4CYUJEtzngoO\nx0iMNqZExhzJ6hhvJGOiGEuWwngLlB56IVdVJcZz441Ce12zRpaXLx/ftY22ixnrvI30HK79r9jM\nHWUM+gUpGbuWRDFZu05MAW5CwWRrzucb5/rC6k0xiTonxzqGkcwRkefGs0PHa0/ZZSjpXyOphoks\nAvGiSkfaxcRLGxAPIz2HeuaO4fNxOGyTKXQn693BFOAmFCRbc55sZ2YkzuWFVZJA5ecLXnfk9cyb\np2XoG4nnHI9JoZgYJEn7PpKSF3mucr9iRXQqc5+Sop1TURFdOi2RRSCR8SsFIvSpAfSfV1TIcvGX\njfbreJioHVwyhe5k7ToxBbiJ84XJdmZG4lxe2NGuRc+VLioSn8VawEZjUig/sSh5kQ47VbBH2IXj\ntVlZGZ9qGEvLH4mVMhITRGlDz4X3+WS54BvaOKsenvwHYiqa+sYKTAFu4nxhqplkNmzQ7L1jHU+s\na9ELO6U0mcsl7MoFBcasfXpWR7xUrSDLixYJ+/SGDdEFgGM57Hy+2HZhpc3MzNhtjJWpojgnjx0T\nOwP9AhI5N5GLh9stzvPcLsbJRq9c+cnxPxBTbWc3mcAU4Bc3zufLMNU0nLHsCCLnJda16NurqBCa\n9yWXyLLNZhRgHk+0Nqtv/8aHN8o5d6+W87+6Vj7WKjqIDMXXI0qLjmEXHm8Qj17LV8w56enGeYtn\nyonU0vXn5eXJcnZhQObjPrl0ReCcnomptrObTGAK8IsbF9PLECn4Ek1vGjkvynmKhqrXyvXFDkCj\n5OkFcnGx0eSSuynaBKI3P1RWGvtXbd8xTByRSITCpz9m9WrRn1IrUxG++utU6IgOh2CuxAub1ztN\nlbZcrnPnaU+1nd1kAlOAX9yYKi/DWHcC49k5RGqKI9LVdPMSKSj150VWctebTUDLg6IXyHrHoiTF\nTmqlcLIzM8UCEMs2HWljluVoZ2oiFL60tOhj4tXN3LhRy62i/8nNjQ6iicVOSUb1+am2s5tMYArw\nixsT/TLEE7xj3Qmcj9wheujnJV6SKK83mgGiP1ZPNdTXodQzTSoqNBPIht9vUNtavjqgHlNQYBR8\nkVGUTqcQqpGfp6UZBXE8Cp/lYxtlblktc9Na2ZYekDds0HYICxfGv0/KYqXXsGPdE/1cThWF4Xxg\nMmzzmALcxEQinuAd64ud7NwhG/8YTcWL19eGDcKssGaNLK/8+eqoyMDKSmEmiaWNBgKxIx43/nGj\n7P6OW22r4HafQSDqOdr6xaCqKrZGrBewIPqMaT7540aZb2j9Xl3tM9yj4uLY3HOPR5ZffdWoYScS\nRDMS6+V8Cb2JEqyTYY7EFOAmJhLxBO9YGSLJ3jnEouLF6yvtE5rGyoY1Mvchl/3UGBk40sscy4yg\n79/zXY98rDVgENp69kdhodC6JUnMl54BY7XGt12Pdt3uf/UY8ovEMnkEAtEslFgO00QzQI5kUkmW\n4J0owToZuwtMAW5iIhFP8J7Pl2w0brMsxw/RjtUOt2pCj09UynzcF0WL0+cd0Qf2zJunXWdGhvad\n0j//ZJelu1bLx1oD8tyvbpTtG1fLOXcIdkosbjdhO3xKihibIhAXLhw9klPfr+e7HvlYQAxmNJNH\nIvdqpIAfvVlnpOjPRJ+J0QR9vHuRbEyGbR5TgJuYCkiW9mIQ1mH7tOf2tTKpgSj7scFOGydEWw9V\noNyk8ZlJDcQUDPE0Sz37RPlu40ZZXvnhgMzmPHVhKLrLJ7vvXG34XxFE+p/SUmNflZXxhYheICpR\nnmuuC8jFX/PJKz8ciJmEauVKIWQVbrvHI8vZ2WFb/K1aZOUGfyBumH+8gCV99GeiQUMj3pc4gj4Z\njtOpCkwBbiLZSAZDZLwwBKLo6Hl83CeXlhoZEYqAL7pnbUwBFgmXS5xrdQXkvC/6ZKkgEFcwxBM+\n+gVk4ULZ4DBUFoa0O7zysdaAnHun+N/1FfF/ZMY9q1UIJz2dcSRqYTzTiN5Wrgj2oiKjbT3SQVpU\nZLT/2z+l2ewLCmJr/4rmvWiRMeQ+8r4pOdsTfSZGE/SjfT+dA4OYIAE+C3gaeB14DbjDFOAXLiaT\nW65/WRV6nqIpV1VFsEt09l8+7ht1vHqBVlQ08tY8nvA5dkzYsJV8KgatNFUEuqR9SrBRVj+0Ri68\nq1IN7tFHbKalGbMHKnTGkeY+lmlEzyBJSYntEE1JMTJnFDZL0T1ifm2f96o7nHh968elLzEXed8i\nhXgiiDXXY8lRPp1jISZKgBcApeG/M4DDwHxTgF+YmCyqmH7Lf+yYMIsU3O6TSQ1EhcCnpWkaryKA\n9AE+irmgsFBz2OmdhceOjb41H0uxX+XH5ZLl5Q+sjulQVcLrnU7BANELYcX5m+jcK0JPYcRYLLHp\ngHa7LC9Zov1fWKhp+Y5PbJDZlCvz6TUyqQHZYom/oI02rkDAGOykLByR7SSqLY9FKE9nauNECfBI\n7MRY2Hiy58HEOWI8VVmSjVgvbSztTD0urPGSGjBECM6YES3IQAuXVwJrInOVjDSeeBqlMr5XXxVt\n33ijljNkNGZLIGDMLmi1ijB+h0O0NxKURUrPAY/UopUfJbpU4bWr47hFW2gKbveNmE88EXNILCGu\nJAaLNwfxMBahPJ0DgyZDgM8BmhGauCnALxCc723oWLTZROsx6rVevXNRHw6v2Icj2xyJKx0Zah9r\nXkYMaAovLJWf1KrVFxVpZgz9ghHLFKJor7HyrkQ5FHVCmI/7jJGk64zCvbJSsGjUY24yJqeKly1x\nLPd45UpZ1eSVnc547vF0FspjQSIC3J5E4Z0B7ADuBLr0X9x3333q3+Xl5ZSXlyexWxPnGy6X+O31\nQnV18ttvaoLGRvG33w+1tdHH1NSI76qrQZLAv8tP05kmXA4XNTfUIKVK+P3Q0QF2OwwOivN6erQ2\n7HYhvgDcbnjmGbj/fq3NWNfrdBrH1tam/W+1wvCwaGvLltGv58gRoE/C+vtagu+Hvzwt2tNjzhxt\nLMo1BwLQ0CA+s1hgYADq6iAvT3w2uNYPOU3Mv9/FoftrcLnCDYTEhUiOXGwlLSx8oIJD/1xD29sS\nthlNDBWJQXo2+Nn2vVrmzNHmjUdr4Do/7mer2fY3Sb2WlhbYs0e7NkkS1+tyifHq51GPpibtvNRU\nOHgQiouNx0Te43iQJPFTVTV6v/Hg90ePO9ZnE4ndu3eze/fuie00DAfwJ+ArMb6b7IXsosL58Lon\nqvGMFPE40vhi2XtHu45E8mQrmndZmSw7Px6tccaDPhozshKN4tx0u432Y32+kFiJsGQ5dm1Hg0Z8\n62rZ/cW1ctr6DbJj42rZfvNaefU1AfnYMWHiKSyMnQtcr2mXbPapxSmuXheQqx72ySsfWmnI1e3z\naU5gz2bBgtn4R8FL56a1siVNmFs8ntG1ZL1ZRJnTRHOlx32OxljEOd6ucKwJzaaaw5MJMqFYgF8D\nP4jz/WTPw0WFyXwIR4p4HGl8gUA0h3u06xgpT3ZZmRAmenu956va2BzrfXHNABs3Gk0uSn1J5Ti9\nEFaEVywHodMp0s/qk0/pzTtR5+jNHZvyDKaPyNwjihlC/bmrSND9vpUlL//Isah5i5yreT+aJ9v/\nKUu2/GOKvPL6V4Ug/obWv+0buXL+XWvlG28JRAnAyMVcH7ijMFBG8lckkmUxkWd47Vqx6GXcvlpe\nszW2wpBoQrPxLDITgYkS4KuAYeAVYF/451pTgE8Okv0QjkWjTyTiMd74Ij/Xa7qxIuxGypMda5yK\nxilt8hqSSUW+2AV+TVO3ugJR4f+uG8X3lr9fK1+1XITEx4ui1NucU1KEBu3xCO1YOUfNLx4Wwtzj\nlrl5tWp/XrQ0Or92ZFZEPrPS4HCMnF81oVY4IMf2LS0/CncVyVVVuoIM38iIyf+OLAGnzusabdFM\nRBAmLJxHeYYDgdipeuO1E5mYLNazMtVs6xMlwEfDZM/DRYVkPoSR2uhoGn2kUB2pSG7k+CI/TyTC\nLjKMXk8PzM83UgXXXCfMCRv8AVVrzMkxOuQ2bpRl2+dWG7TfyP6z7oj+3uPR0sjqs/gpDlN9DhPl\np6JClkvu2ChLd4XNOp9dHhXC78gMyIWfj641qReC6emy7LhVCN+Mf8mQVz+0Ri6+c4O88uer5aJ/\nL5JXPrQyOpvi13JFP990ybiPyRUVYn74uE/QBu/T+PWKAIx3P8YqCBMVzgkF+YyiMMSLC4i3O5xq\nwBTgJs4FekHh8Yx9UTgXc04iL7q+/SizQowfQ7GFdRtl++dWqzZx1WwTZl/Yv+CN4pjLsqxGUOoF\nnPKj0BDz8mSV5WH5tIgEjSXAC7YUaEL77vzY7d5iFDwb/ygEev5da2XPzPBxqQE55Vu56nG539P+\n1p+rhutLx4TG7z6mjiUQEGan3CKNfrlwoawGSI1Wdu5cAnPGi0RSJChIZHc41UACAjyZLBQTFxgU\nNobHA/v2aV55vbc+Lw+am2N77s+FvaJnJGzeHJsdoLQPQtwpsNlgaEj8rTBFADo7ob09fMyMJgbD\nLIyU2xfzqKWX4ZJ+OHklWS1VPHv3Vu4/LUUxIl76Rg2X3+1n4NFq6JNUxovXCwsWwM03h8eS0wRz\nGpGBvGv9FLxeS2ur1k5KCvQP9WsftHghlA6PV2MLSYSHj112MQjkDnhpebCahquqCGQ1ghtSNyyG\nwV6szn7SM6wM9IG30EvLUQlSG7ANuhmyt+Mt9JLWUM3Jk+FGg8Xwg3cMY5Ek2LkTbr5Z4oknarFl\nwcKF8O67sH49PPAAbNpkZIgkwh6CaMZQbW1y6B1SqkStL06nEai5oQb/Lj/V66qRUieYXjLNMdkL\nmYlxIp62pNd89cEmkQEto4VBx/t8pCo5kdt3xZGohIm73SLQpapKaJTKufpcKTabZvO1f8ErZ921\n0qCtpn469nYhkgNusYiqOvn5IkhHb25yftao8QUCWimzsjJhk1VzhN+2UCY1ILvdYtzKOFNSZHnJ\nyoCcusEnZ+aFte2btHbzvjfDMG7Xt4rklR8OyO4ZYS3afUwuussXVYxCb+rR264j763TGe2g1CPh\nhFTT0HwxFYBpQjFxPmDISbLGKBjiObsURApjhXqozyoYKTjibd8LCoTQzM/XChBE9jl3rhBWOTni\nGJXtkhqQU24SpgI1aOU+ZPyl8pKVsZNf6ceuL50GxqhHqysgr75G2NxVf0DY9FFwdzh9rD5Xyyer\nDJS9DRvEHEQWUAZZds/Q2vV816O2IX1H0kw1YfON/ea18quHjQwd/SIXa76KviDOzfrCWjkrXzPl\nxKJdjtVWnftNb9zEYolQUC82YApwE+cD+hdXrwUnUhcxUhhHJp3yeo3JlZTMdnFD5nULR6wCv3qt\nuKhIE1C2DWtlR6YQUFn5AXnVj6tk5y2V8pKVmtAqucMoVNRoxFt1zkfFXh0R9Rh5/ZFaqCLUyn7q\nlWdfHjA4U+OxWtxuoekrC2TB5rDD8R5JXllxTF1MrZ/V+iq6y6fes8pKzaYdKTCV3YV0l3au/Zu5\nMjetlRfGYMKM6XkJ26r1voCohFimlh4FTBu4ifOBzc/6aatoYv0TwqZ56JCk2qvXrxfHxLN7R0bb\nuRzCkJ3R7sX1QjWX/R944w1x7MKFsG2bFnk3fz7090NaGnR3a22WlkJ6uhbp19Agjj10CPr6xGdW\nKzzxBHzxhSaOn2hkCBi6dTF0zKYj5OIvX6/hg++TIE0bv3NRE43Nwsi7+F4/3S/X4nBAKKuJdrew\nQ3OdH3bUkpHqogvI7PTS+Xg1Xq8YZ3m5sNU7PiGu01vopXqdmBjFJlv1O0m1JS9eDMGgdm16e/4H\nPyhs0sqx9me3w7V+eLyaPX0SlZXg88FT6S7OAK6gl+fuEX0pNm7/Lj9VO5vYf3I/gb6AOo62plrR\nbpEL3GAfzmAw5TTMrWP2Gj+SlJitORYUW3XFVm1uI58N5TnQz4+J0WGd7AGYmH5oOiMEW92bdfh3\n+dm8WYSEK84unw/q641Oz/JyqKgQ/9fWat/lPVOD/bCPrgfqaXtb4sknNUfjZZcZHWatrSKsvKUl\nfMw6Pw5/Obl3VpDmCeqHSGur6Lc0nCdzeFiEzWelhT2fJ7zQVQhzGmFuHUNr/TQ0QEYGlJSIEPrD\nr4ljnWe8NP+4mrY2CIWg/bT4PKfPi/RcNampMO9ADQVnfDznr6dkpoTTCbt2CWFbVwdNr+SR58pD\ncooLUoSalCoZnL2FheIaQfz9wQ9q323danTcDnZJsKMW+iRKS8ViV1sLV3fUkPKGj6WH6vmnzRIz\nZ4rr8Xhg2+Pi3inCuyxfCEyl3dIjNVTN9ZERXKHOk+N/kiNQa2qinw31uxtq8C3wUf/petPJOMUw\n2TsRE0lGJCVrNLrgSJn7IqmKkaHrap9ro00KGbdr2+6qh31yVZXmKFTC8teskWW+NE/mHrds+Xqu\nfNVHXpXzv+STK24IaMErYerelVdGmDBSA3LRXT6DLRhkeX5ZOEQ9Bj0wLU2WHdcbw/a9XlnO/7o2\nVuc/5sqe29fKhZcI08maNWK8WVkad1y5/kjzkd5kpbfDr1ypmUbcd2p9653Meico/jKZT1Sq5eIi\n+1F44aUrzs18osd0Lq4wGSABE4plggT4BHRjYqIQ7AsaKFkVFULLzMiAFStg+3ZjcqDXX4fTp7Xz\nfT6Ncqacq1AV3e7YCY2CQZg3T2j6ixbBJZdAd1UFDc11eAu9quYWDMLll8OpU+K8ykr4wwIJnGG1\nvr0IfvAOJSWQPzvI3hl++KOgBFZWQn4+/Oqsn4HMJhy4WPxWDS88Ixloijk5sGyZ0MaVJFMG3FIu\nNHug4IyPQ/+3lkv+qYJATh30Z4AznOvtNZ/QoImmPh49Gp3sSaHjOXDR9J0ajr8pCYrkOpHMqiDH\nxXBKB21OYUuy9echdXs5U10DfWIyM3KDdJULs0vZfIlF/+CnubOJI4ddFL9YQ/NhieJiYf5JT9dM\nWLHGoU8kNhKU52D/fm13oX8GxoOxjmGqtD0WWCwWGEVGmwLchIqxZGPTH/vAA0K49odpzZWVwt5a\nXh62167zY8ltQh5wUXqkhqfrJJXb7XBogmLzs7FfHKUv/bGSFL2QKFAWBa9XbNfzt+QRSjkNAy74\nyUG8c4txOjWbOUBurriGgwchUFmuCmC9kI1ERQX89a/CXKPwwR0OCH2iAubWwQkvlR317HxE4up1\nQRpS/VgzAgzPaYATXqwP1zPcI8adkiKyDCqIJeBmfn8mrV2CTG7vz2fwB4eFYNYtGI7+AkLOVsNC\nURjwUfpGLa+/DgUF8NZbYm5yc+EPnnLaPbGvtaQEZs+Ofh7Kt5WrvgHfAt+oXGz1OQhDuS/nku1v\nrGOYKm2PBYkIcNOJeREgUY0i0cCMyGM3bRIvuSLALeFHTrGrZsxpoitHHBxYspj5D/VyKr+fob4l\nsH07BZLEnXfCHzxNqjDx7/KrL86uXahBMDk5wtb+5pvQ1ibhcNTSvgwknbYa6Sh96baXWP7gKuo/\n9Rz/+UYxaZ/080h9E8xxYfl9DbkZYj5UgR5Ow2o76WXocaP9V5+q1uEQjlK/X6ST3bQJ3n4b9kak\nYr3iCniXqG6VAAAgAElEQVT3XQk6ahlODYJ/MQw7ka9fDztqyLBL1NXBmjViDnNz4Rm3H+fnm3DZ\nXXiba9j+G4n+QS3wZ9DZpjpQLUMuZMDd5WXhwR3scW0iMz9AZ36D2J18vRop1ShI09NFAFb7kAs8\niECix6vJyhIpeWOl0lWeh1gOx5GeMeU5KCsTC4LbfW6pYOONIVkwHapGTK4hyUTCFK2R6kDKstGG\nGWmrjpXUSLGrKomkvNVeQ2pTPeXO4dDssynfyjXk8NBnvFN+9AmdXK6x2VZj1cvUF3rQV/PRh+gv\nWmQMDIqb60NXLMEzMxCVfCr9y7r+7yiRuWW1nPbNInn5gyvlgrvXimRbEbREn0+W1/x6jfqZ555S\n2ZYeMI63aoPs+argmr/afEykltXxrmMmDNNda2GhMYNjPP79yg+LcnZKPc/IOc39Xq6Bzx2ZiVB/\nP8dSH1OPsYTRK0jUBj9S2xNpx8fkgZuQ5cTzQIzG4dbnfp41SwTFKJzrkYI69C+EMhbVkabP+xEO\nrsm8UxPyeV/0qfUqDT/rNsrcXSCz2SNnfnGN2k4ilXHUMYSdl3rBHOunsFCWMz61Uc66Y7Xs/IwQ\nynquuZ5Tfaw1IPJqRywQ+h8l+RQbvYZMgoaMgrqKOKUrAvKGDUJw5t9RJVf8ujKmAzWyDJr+fpaU\niHJylo8JJ+earWsNWRmJEayzYcdGOXeTMV1rPIe1MqcZ/6JlMyzYUhC3bJz+J1lpj0cLBkpGquWJ\nTNdMAgLcpBFOYfh3+SnfVk7FwxUE+4KjnxAHiVK0srLE73gc7n5d6o6eHuGYbGjQKrPU1gonpiRp\nOVLASJmruaGGynmV5J+tgl8/hX1QjKesDCqvkVh6rJbO02IgtpNeTv2ymjNnhI1YgdsNGcVNkNkK\nLmEq4Dq/YdzKll+h8fn9xvnIafXBb+qhT+LSS4XNvqBAfJ+ZqR1bWiqcsLb8JjqyG+mfXUdgpV+9\nbr8fap/UaJWbnvEjKTaDE14sT0RPZOi3NVjbS7BbnThmHhbX5HSLuS/0Ur4yjZzCDlJDBXgadpCb\nIXHkCOx5UqLtPx/jha/u5PCrYt48Hli5UrRrGw73251La3cLL86rgNSgSk88ucyPvKCWdk8jDc11\nvFqsTcr8+cK3oEdzZxOn08Wx/l3i2Hj5bZRnbMWsFepnrd2t6nn6c8vKhLNYaSftk8l5ziPprZFI\nRmWp812dairi/C5TFzAmOjpttNBovZkkHt0vMvIxHhTtbvXP18qVn9SoamvXCk3c9imfmm1PH7Jv\nswktKOcOTZN/73+WGdqQ5eiw95UrxXhmzBBb+JwcocXbN66WpS+tlddcF5AzMzXzTHm5FgUqy9FZ\nCNPTxZhWrtRMP57NYodzrFWYGFKlgJqjRf+TkSGL3UN4/PnfK5SPBY7JJT8skVc+tNIQIq9o8FHF\nI9ZtlFNuE9rxsdaASFmrmEN0Wn3RXT7NHKLT0KXveAwaeMxcJzF2bqM9I4HegJplMXLHFxnBm+xU\nr2NJLzteTGTOcCbIhPJL4CRwwBTgycVUS4EZ7wXUQ8k1EqtorR7xXtpAwFidp6hI60/9fN1G2XP3\nSjnlH/Llq7dVGELB164Vi0PKbcIGbUsPxDbBIMsOfwxTh840w12FMp9dKbu/uFZeWXFMy52iHHfL\najnlM2tl3Mdkj99oE5Zl3SKi5CbZXCR7vrZSLD5flzRe+C2VUXPCfcjur4nFIjdXlpcvlw3FI/Qc\neF+tT8ubsm6jGPt9yPYvl8rLVwsb+CWXaAtN1r945GOBY6MmoxqPnXk85yXrOR/veKcqEhHgyaAR\nvh9RxPjXwKI4AjwJ3Vx8iEeTGy8mgt/a3AyrVsFzz0XzmPWoeLiCujfryB3wMm9vPVkpkspIiKQB\nKiwFlW9+e7nKainp8dH761pOn9YV5L17pjCvAByqwvH7xwiFoseQf1cFbW5B97PV1DPUbaTkGXC2\nBDpnw4CLjD/V0HVdlXpcUbuPA/9UG5VuV+G/p3+5nO7ciDY7cyHzNLSU8uzGp1nlldQ5yR4opXso\ngHOgiFSykLvzODXQDCEXFb01pNskAhVGDvxlRRJnzoDlM+XIsxvVa6cnDy7fBTaRKpdeicKXtnHi\niODMJ1JEOBGcS0HgZD/nFwomkgc+B9iFKcCnNKYKvxW0l7blwWr2PCleWoX7HE+wKJ/rhdfb94sQ\nfAM2Z4NLRIxktVRSdngnjVl+XLObCHW7CD1SQ+kVEjv/J8hXnvTzzKZqzraE27gpzOEG6MuE1E7o\nzgXbAKR2AJB21MegtYvQnDpyUnO5Im8eWc4sam6oYfMdErW1WjqAoiK44p/FeBmygU1E69iOVjDU\nI/J/+9aJcwxzMktbIByhXEIOEQlV3OVjzou1vPFOkP6P+Ck7Xs3230i0t4uF89J7K2g8UafxzD9R\nZViQbH/z4Vsn8e6AcSE/14rseopiPP64ibHBFODTFOdLU1Y0PH3k4rjGd44vO2jX+PorLk4/WIN3\noRQV3BGvH73GdkmhJKL7wtGIhFxg74GSRujKJsXpQHYGCA0NgUUIz5Q3fHzgZK0aMapo9gCu7CA9\na24RZPb/+SFcvQmyWmB2mCTe62H580fZuxe4zk9OSQtn0sV3JT0+zj5Yqwpvj0dEVN759SA1nX4G\nU9rEuFpK4ddPQ5+E2w2vvmrcrVRUQF22WEhyUnMZHB6kfSCIx5nN0Mn5dJzKgkfD0ZXr/ORe0USq\nzcVATQ39/ZByg5+lJ6vpPi3RWKQtSJaBTJ751AG+9dLNUQu5XgDHi5KMvB/6wKvQIzU0PC6p/HGF\nU3+uEZcXMxIR4CYLZQpiNG86GBNEBRN03I83YVBkX0oQT12dYC/E6185b9YsoR3qx6pc42lPHUVf\n8FNfH81gUfux+5n7rxpLQc9qSQtnDyS3SU1MRb9HRBU6exhwnCQ0PKAKb3vIw0B3Gg1F5cy/v4Kb\nd97Myx+cieNb2Tg/dzV2B1C7E373GLQXQ78EuQdFH30Sz/79PrJdEvRJeI/VUrZAMGa8hV4KX65W\nhbfDAddcIwJWdm2XGPxtrWivOxd6c9U5am8XCav081tTA5WDNWQ0+wgcmUf7gJi0nh4LHdIecY3X\niefCNkMwRY6n1tG2zE/7SYlTP60l3Sqxc6dox+PMBkBO6WTDkx9k/8n9AKQ70gn0BQj2BRNiV+jv\nu99vfE4zPuVXE1WNxmYykTxMSCTmfffdp/5dXl5OeXn5RHQ7bZFIJNhYoiYVxCtBFUvT1e8COo7W\nsKdRCPy5c43nKln/lP71bXV0aJrY8ePGseqvUYkUPPpeP6H3CS16flkNqYg+rXlCSNW9KcYl1deq\nfYSu9cNwE+S8DoCnx8vub29lzSqJU/MdhrFmpWRRtm8fjZetgMxWWoHfveYUpc3sQFED/e/3q+Hk\ndjsM5jSp5hiGUrhujZvSK4Rg3roV7tydh/UtJ/ua38Ra7IPU7XjSJPbtE+XV1BDydX644g+Q2g6X\nNqhRlJmZglKnpBwo+H+7kJ1ncZW56D3sZbgvvEINZDCIDDZEJsXHq7F8zI8lXwhjRyiHUFYL3FTB\nwqYaqqslkUL2EYmKh5erOy+nzclbwbcA6A5103C0Af8uPzU1taPawyOF/PontHu49fpqpHAq4chI\nWBOJYffu3ezevXtM55gmlCmIRJw68Rx9Y4UiUBXNUdny6u3lKW/4GHi4lowM6ArnYXI6BS9c6V/Z\nTu//q4vAz8X2vqBACHi3W7SvT3ZFavQ1OvzlDL5H9Gk/7BNaK6h2aU+Pl3XvW8AfdjeLlK6P1uDY\nUEWoUJxTmFHE6186oCXYem8OZJxVr7VqXhVbr32M/O9nE7ILoeywOAjJwsNpC7kZ+v4xyuZLzJ4N\n//EfsOJHFbRm1mkT9poP+iVyr2hi6VUuXn5dSx4FkHrUx7v/UcvmZ/384sX/QiYEshWbxcmQrVMc\n1OOB/zyqJphyOERirJjOzoOVUPy/kB7OztVexKLnDnDiiMTZWzVnreT0EOwX12QZTCOrZzFLF2Wx\n/cYacZ/Dc73+0fXUvVlHljOLjv6OMZnTIn0TpvPx/GKiTCi/Bf4XuBx4B7g1CW1e1NCbCOJhpNzK\nY8Eui5/2vyuHmypwzwiqW15FQ+aEl4FHqykqEsIXRK6O0lIR+LJjh+hf2U4HcurUoJrnnxdjfPVV\ncU5Xlwj8ufxyWH+9RPXVxmtcuUz0aT/pZfAxMRC7HXi0Bk+Lj31frefPLzSLfClz63B/2s//WSrO\nKcv38vqXDrD5DonyciEQC5/4K5YhJwDWwQye2dtFe3+Q1XOXAFBaUEpOeg4ANouN3Tc/Q8lMkZ97\nYEAsPB/prMHRH47w6c+AtACuOQfDO4I6AhxRx289WUZh2SvM+ZnE1n1bkW29YBsE+4AmvAcd8OA+\nLTtgBipDpieoS/YN0OOGATdYBb3mvZ5SKlsO8MyfJZYvRzBLwrBbtd2GbO+lPWuPGoATGUjlW+Bj\n/+f3j9mcpgRrKc9bIs+pHskKTJvsPqYSzGyEFzmy7y4XVc6BimIfT9witN5gX5D5m/20/rxadTCC\n0MBaWqKdVIqDtCzfy+zGerb9TDIsLMqOQV8lPtLBpWh0f769mvaT4uSrrxYCIy1N2MX3zKlgsESw\nLCrO1tP0gaUcaz/G0KCVjM4yes9mM/g7sQNwOCBkC8Ltl6sabFG7jwP/Ws3ie/0UvlzNa1ddR3uW\nuBjfAh/P3qFVj6+sFFpn494g3D4X0gUTJN9VQFtPK95CL8e/v4PWq+4Eq4Vn797KdQ1zaO9vjz3Z\nPR54cB+2rmLe/36RFbC9XfRhsYDsDMKX5kFmmxDeD74Kf3ezjo2Sw4yhZRS/WEOaRaLhqvdAZgv0\nZXL1O89xoPQaWrtboc8Nqe2U5Xt56tZ6NtdvjusUV0xlR84eoVgqVtk0Iwnl8TqxJ4IFNZWYVucK\nM52siVFx9TZBccvp83LFixofe/Ozfg6ebOLI31w8v6mG4hnx07Umsp1W83SvEGwRu+zize8Z21XH\ndLXQ1MvK4KmnRPsqSyI1CNf5yXqmmv0vSFz1WylaYEakRbXfLIS+K+jl4D31FM8QWnpjlh+u+hXY\nB8hMyeTAFw5QdkmxxmrJbcIhC4ZF1q3r6ZgheOvU7uDMdR8k01JIr6NZOEmtIfKHljCUv48zvWdI\ns6fROzAA1nCS7/ZCeOB1lTnimNlEhlMzN6kIX19qQzXDPRIDvjCLJEYecfttqxicKRafqrk+tl5f\nza2/9zPw+BZSrtsk7NKpUpRQk1Ilzb/R38Ged3R5dYkWfJGsqKprpVEZK7Ew699ncbzzOFnOLPZ/\nfj/F0giBAuNEsphWUwEmC8XEqNh+Yw0lPT5Cv6xnz5OSgWGw50QjrZkiv4cescw38bbTypZ20ffC\nW9ocwRYZLKnjg48sjrnd3b49XNvxKa19NTWtXYIdtXQs34z3J+X0hHqMFzRkJ/uvWwBh5qmqgo99\nOI+UwVyWLpLUghH79yMCXOwiCXfnQCeb6jdprJacJihuJDSnjtRP+KkoFyXRQl0Sp4+7kW29dLj3\nEHIdh4w2cAVoy2wgzZ5Gii0Fd9cyrMdXibZaSkl56HU17ws5TYQKw+Ym/2Jsny2HmyrIyg/ynls2\nY5Pa6PvoegasQRx/rBECuy9M7ehzQ/2W8N/is7J8ryqs83bX0t1STP9/1aoLQ6RTXM8eOXJWmICy\nnFmGY9T7F5Hrxb/LH5exMpr5QhHYHf0dbKrfFPV9MnCxlWYzNfCLAKPxyvUcYIW7vP6JxDSZmAwW\nPRPl+nL2nNAVDHB2wdw67G1ellzlZO+7mvliJIaMw4HqRG1oAMu3XMj2XnGQbAGL9oxVzfXh2Fmr\nOttmfrOcVmej+l2gOlzAVxfwYw+5+fiJYzzasZlQyS5IOw32QdKCZRy65ynmf7uK3hm667ikQWOn\nhJFuT2eg3yHMNgCBYkjphtZS2L6d3AyJ06ch4/MVdBXUkdHupa/HqWrRxVklBPvPqjsKfUWddP91\ndGeHNeWQU/DdT16JNSTxvpPb1J3T/K/7aR0QTJ7KwRp2PiJF7Y70mvCztz7L/c/cj+3JLTxp24SD\nNErKmslK02nbRZoT+ei3RQKwWCyT0cwXo2nHU6USzlSBqYGbABLP0qaUNZOkxDWZSG5w5GdH/iYa\nd5wS1De686A7l8FOib++INTdyMIA5dvKmfWNClatCbJjh2iroUEIcUU7tzp05Ws6ZkBngdrW1uur\nkSShfVdUwJlWnUP291oB38xu4cykV2Lwx6/yu7ObCV1eK5gdduE4XHXVTIpnSIR6tDZ4vJqUs0sM\n82DBQvdgtya8T3ihs0jYzS9twLPBz0sviSjF9x6qwXJQFHIe7BKab+6Al8DbhZo5SLYx5DzFmew6\nUu6+nNRCkbWQgQxw9IvFo6QRq5zCHtdm6maUU7hlFm35v1P58H+aO59Va4JRDuOB05omfO+T91Pr\nq+XdvxVz6qe1tPQ0s+eEeFbm/3g+jsygwYkspUpRzkz1OYpT6EHRyh/46AMjPlOJxD+YMMIU4BcB\nlBcro8NL4NfVUYE3iklEX4dRSpWQ6muZXyKRnS3s0sp5+sAeR5j8oN9OHwkTM6xWuPQVUeV88WtC\nc0NqVoVaqDOdonbxQm+u38zMb5bzy707aGwWgSl7PH61hqLSvio8wgLAOpQGv3ye7OBHyEnNVT/X\nLyLy9rAZ4jf1OAYl9XoP/ON2Cs744D/egvZi5OwmwdPWIT1V5LG1Pqa14bJKvO+d7TisKer8fqD4\nAwBkdpfBwUoWvlJPvqSlxb2yuRq3W4SY722UkGvDJo5HRTX7eXvr6bCE8+/K4LCJV9NmsTHgOMWZ\nvtMUZRbhlLO0wb27ULB1wmap3pTjDNs71K/77K3s8fipqxPmJOWeDXSE2zjhRf6juGnKopaVpjFh\nWrtb2b0sn/wv+9j3f6tj+isMz1GMRV8vlDfVbxqRtWJWwhk7TBPKRYBgX5DL7/Zz6peieG+ijqfI\nWoYqR1z3eWWlyNWt306vWhVmqeiK7S48LEKtM79QQecMwSIpPVDPlV/bzJ+P7VJrParoz4DjK6Cz\nEMvcP+PO7sf7niUUZhTS3N4MMrxx9g3qfP/LyvcW0/PJcoZnadv3rq21qqM1PV04LNOLm3jfEhfb\nb9S25zffDL/9raDyKaYNhu1gHeTKvDIaP/MUUqrEihWI8HkF6/xYC/cj57/CR1te4AX79wnk1rFs\n9lXkPrWdbT+TINU477nhAEx9gWdlDgcGoG7mKi1kPwIZjgxWFK0g0NPJyyfDAwkUkz40hwH364Qc\np3E73bT3t2PBgowM/Vnw0/3QXkxODpw5I07Lnx2kbZmf0uPVPF0nNGqF473lR0FW/Ga+YLPoUDWv\nisdufEz9P1Fzx1iciiav3AiThXIRYaQ8FTU31LD+emnMgT/6HCGlpfD00yNnC4w6T5fZT7FLb/mR\nSCAl/7GabT+TmP9vmn1aRSgFwiYSy6AT2a7xnfNceZzqEZRAxc6acoOf0GU7hFmhN5uVl88nzZpF\n+v/UCEEKzP3Xck6ni35y03JZ+p6l5D1Twx9+J6lBTBXXB3mxwM+ph7fA1Zsofk8ac0qbDbk+VOiu\nLeUNHwOONvX/DEcG/UP9yLJMRnAlwQd2kmGXooKgAKQNft77/ib2HTtCz1CnugOwWqwMy8NR9yTF\nmsLA8ABl+V4Onnibfkeb+nl2z/sIdUoMprbSnhkW8q/5KH2zltxcYYbyegV3f9Om+JGSwb4g839s\nFOKV8yrZeeNO9f9E6XqmUB4/TAF+ESEyGVFbhfEFq7569FDpSASDcOutInu2Ugke4OZH/fz33iZK\nFxi1Wf15t9wCf7m0gras+NrXFZv8HLbtgDSdM3DYIjjQGUGkbi+y5001F4gtJOHu9nJWajC0abm1\nXBWe9lAOgw6hauaf8nHtByWaO5t4/dTrnO45jRUrwwjBmNfm49RPa9WdggMXy96pYc+T4aRMt2kO\n2OIuH283ScjZTViyj2BN62LIESStvZRlB5+mMXc9zBU8+CPBJjoGNFNGymAumR1LOfN2HkjN2GUX\nq07W4HZKnL5O5+QNIzVUyPLL5qr3LzMlk86BTvX7oswiPlTyIWoO1DAoDxrO1TuKy/K9nDm8gFkL\nm0mzu3jjlTyKFmoOyriO6V1+Dp46yAvHXyAkhygtKOXpm582HH8h0fWmKsyq9BcRRspTIbQfo9kk\nkWAMSYLHHos+Z/+SJgJZjTQ0G6vH68/buROCfTUjal+toSbICAvvQQfYQ2CVISNIms1FSv7bnO0R\naqst5Gbox69wtt+NtdKP7elq1j8mkZcHDGkOxsE+SeQaOeGl7TfV/LerStW8Fe0VhG253b0HPrNK\nJKtyBQgBLw/5KSmpxemEl44eAafQht9OfRy5tB+sw8hAmN1NWv8c3jggYVmeh9yTy7HDErYC7bVy\n2V30cJoz2XWi+rsFBoHnZT+9v6ql4mExdtugmyF7u8pVd7vhlp23YMHCD6/9ISt+sYLW7lZVYFY9\nUhUlvO0hD4OPV1NaCnMq/Gy9vpqqR6rUhSC3MJc9J4T9JtZ9U9B0pknlhhdlFkUJbxD27kQ169HM\nLeP53mSsCJgCfAogGQ9jZAKh0V6weMmwRkpstT/VRWBvDRS5IAtyXbm0dLYw6xsVFL9YQ/Plmyle\n3ETzEZf4/7BEcXEt67fGXiQchAVvixf7oMTg7Ibw/6W43Om0DWn2YMeQm6G/uxlCLoYfrWFvmOOc\nmwusyhPslj4J/vgLkQL28WpKr5Bom3EEukTNydKCUhqbG7FZbAzJQww5W2B2izagE172fquaa1aH\nbfjzimH2cWHKUCiLegzbOdv/LnxyFjg7ILWDgKuBglMVFBa+QmlBKa+cfIWezjBXXdGlhi3UbxJc\n7jyX4JdbTi/g1BEJ+1Pb+NxrEtu3o5os/Lv8XJp9KQA7fDuQUiXV4We32hkcHsST6mH3rfu4v0kK\nPwPihuodgy2d4lqznFlsuXqL2nbks2dINBZHu1Z4/yM9u5ELvtJf5MKhODrH8v1o51wsMFkoUwDJ\noE+NNU+FXmNPS9MYCgcPxqAFRuQ5KT0imCXzcuax5509gjHy3sUcd9ey54TGIDl+XAjCyKLCCl76\nRg2ut3xkPlbPYM12UUHmYCXuPzyNE41tsSi3lKtKZmnpYteJxkpLIeXj4Sx/6afg0gbsFZu4ur2W\nymsknn4aSjyCVtPe385rba9RkF7AqtmrjAMZskFnPkVzz/KB38+h9bOp8NkVMPMl9RCrxRZ9AdZB\nmLUX3MfVYg/WAQ/Pb36YE3ef4ImbnqBEKolxnoyvehPBIDS3N3Oq5xRtrkbkUAodJyW1YLICRSNu\n7W5l6T9sYtYX/Zzt7iDNnsZVM66iIL2Afbft48ev/gttFeWsf0ILpNEzQ5Sx6ANpYj17yjkL8hZQ\n9UjViHlFRnp2FSUhcHJkdslo7JNY35uMFQFTA58CGO/DGM8MkohGr9fYq6o0bVypzK6nBSrjK8v3\nMttWzbY6CUmqpeLhCkBwvEO9TsgWaUrp9cDj1WRmQmen1tYVV4jshA4HvPQSFBeLKvSN4UR7/O4x\ntcDBTZ+toSVXGOBb9mzD8vH14AknutolBhZY5aersBYGwl7IIRuD9gCu7CA7HxHX3BzUqHlneoVt\nfFHuYgrSC2jtbsWGjSHbEGS20dp7VpglHAjBrMOH53yIQ6cPcWn2parmp0Cxq9tDbl6+bZ+BbqdE\nOBrQ46H159X4W8FVKebW3eWl/XFxXWVlYr7UohenRKpc62AGpzoD4GjheDgA6uV3Xwbg8h9dTtnM\nMvaeEONe/OBiZrtnG56BWNGWsZ49ZfHXOyrjabkjPbuKklB6pEY16cR8FkfZLcb6fiwmnAsZphNz\nCmC8nvpYVVT8u/zUvl6rBoQkktBHzyq57C4/Da8YHZTxxqd8/vZPqtlbJJx4SsIm2oupqBAUPsWs\nk3KDn1CWiBIs/EsN3Rvn0N7XBfIw9Hqwp8isvGQJxdmF7Hq2mcBJF/b+PAYzmmHIgTXvDTLlItrb\nsig9UkP6Z6s0B6CM+jRXzfXx2Hpxzat+uSoq14djMIfVl5eR7kjnxVe7aUltwDaUgcXez6AcMjg6\nARi2s/rVN9m5rRhSg+RtyWNwOMJ5COSk5bDsPcsMi2awL8i8H82jradNbYufvYx39pUiQVg4re6W\nD1Rz520SFovIM755M2zLnUnIKZggKjUQsPflM5japqaEVeC0idzmSt5v5bqV/CcHTx3kyNkjPP+5\n59Ww9pGevViOyshF2D0j/vnJrLl5McJkoVzgiEXn02tNnlQPR+88OmpmuYMHRfDN88/DzU9HJD7S\nFU+I5+ycNQuOnw7Cx/zwR8F51tMOFejzfeelFnCmr80oKMNQBBGgcrIBUc0mnBGwOKuEd7tOMDA8\ngN1iZ0nBcva+u0fNwKdcsyKEkK1gGcYy6EK296jX1/JgNXsWa9kKU+2p7P3cXq75zTVGLvTBSnzy\nTvZ/4ArePPsmQ/IQLodLy8WiW0D0i6bfDwePBtm74jKVHeN6y0fZm7VkZcWf0/JyaHy/U83Vohfg\nWaevxmWR+NPXtrD0V5czMDyAy+HiL5/9C/c/c78h77fe6TnWLH2xhLskGet9vvPOqM2YGCdMFsoF\njliVT5QtrSfVw77b9qkvnt6skufKo7m9Wau2s0ccs2mTtqVXtsRVPzE6OyUJdu0SPOYlS0Ro+8AA\nwoFYW0tBASxfDu5P+6naaTTjrFzmovGEaOtUXyuWGM9maUEpx4LHNAGuCO8eDwyJyEe3001BZj7N\nHcJkMygPUpCVi8/ji9IEa26oYe5/zuV0rxD8uZkZnOrtwVvoZd8LabxVUiV4kggT0aJX6rnjkxIL\nMw9xemk+g+FiD26PhervwpyftTIkCw7K4FB4bLpixZ5UD9Xrqo2O3z01ULAM5oar4jxbzZ5sQV0s\n3D9G6PsAACAASURBVHKExZdGp3E9ssAP1pB6HXoB3nEqnY7f1VJl8VO2qIxXWl/h2suu5Y66O9T7\nH2liGI+ZLlYFJyXy1uWC555LqBkT5xGmBn6BId6WWK+Z64NhUgYKGPj3Q1rO74hKORUVoiZlxpwm\nVix20ftfNWoVeaWg7pl3XchdguOcJ7lo+k6NQeNzHvHheryWK5cF2bt0Pn32VtxdXn57yw+4fufV\n1H+6nu//7/eRkdlWtQ1frY+Gtxo0/vOwDZpXkebuozdb2HgVGzYQk6fs94tiFf0ZTVjzX+dM32ns\nVjsuu4tURyofKvkQ2/fVMeQQ6mTqQBHv/uMBQ6rU/K9dTVtGA1JvGa/c9RTFMyTytuRxuuc0LoeL\na9/5C7s67idkC8ClDTisKSyeWcaJjhN09HdoXPDXfJQer2bO7cIOvGiuxPE15YZq8WDUjFf9wsgP\ndwzkEUo5RWZ3GZ0/egrvQsnAU9ff04L0Ag59+VBMc1c8M91ogWDKOc3NItL2ueeMhZhNJB8Tlczq\nWuBvwBvA15PQnolzQDz2iV4Du2rGVernAymtpNx9OdKXKiA1GHV+TQ3kXtFEV04jDc11QjMMI71Y\n1KqUL6uDRTUwp5FTUjjlqC7/Sv/2agKr/DTOrKKv1wrdObSflHjwnxfS+w+9rJq9isdufIydN+5k\nxS9W8MKJF0ixpvDfN/03uWm5Iqd2SSMhl3BIluV7hR23twpnbzHH3kjH98h6A1OiqQlaBwR97Uzf\naVJsKQwOD9Ix0EFbdxuP7N+hCm+GbSy/9AoxTzp2zodObyevzYf39af4p80ih7j97TU4rCksLVzK\nyTdmE+qUwNGLdSiVVLuTvSf2crzzuCq8JaeHSpsIWX9svZjXgQFENkHAMixUWrfTrVL7wJiThNYr\nCf3kRVLe8LF4/1NUXiMWW+WYjA4vQy3aPW3tbo1ihIzGShqpYLG+reJiYTYxhffUwLlq4DbgMLAG\nOAG8CHwKOKQ7xtTApwD0GhighkpnpGTQNSCCZSJtt2rh4E+Iog/eQi87PlbPVz4vIcvQXSU+tw5m\nMGwXbUhOD2995ahoY5efwK+rRQi6LvRcgdPqxJXiIs2eRomnhOY3smix71ETMhVlFrFoxiLq3gz3\n8c4SkN4iO2UWy6/K4uVv1tD2oSpDuH7ebmGzfz7fT3+JCK+/Mq+M/IwcGt5qiJ4YnfkjMmJVz87J\ny4NTpzBcR8EZH62dbVHXpcBhceDuW8xQVzZpw3mc9fyZweF+Bo8vgT8Ivrq7+G017F0//8G+ILfs\nvIW//MVCW/VWQyh+yR1+Zpc14bA62PdCBmce2irm8+759DtEsM+Cl+ppPiwlXDUn0p+SaDphE+cP\nE+HEfB9wL0ILB7gn/Pu7umNMAT6FoObXzgyS8Sk/XYMBGt5qiHpRDQmrPhkk5QY/aQ3VqlDI+6yf\nI+2C1TAsQ1tPK3aLnZdve5krZ1yp2oBffPMI/aeKGcp7CRz92Cx2huRB0u0ZdA92RQ9QyYMiW1ld\n/H5+9Xe/YunPl6rmAYYtIloToDsfWq9UIy8rO+oJtoZNIDpB6xjI56XbXuDq2mW0dYfZIO8uws2l\nlC7rovGdBqyDGdiHsiibU8KJt7IofrGGl56T6O+HrCxB7WtshKwvVNAxQwi2y7IW8OhruwjZA6q5\nRwkSArAP5DCYEs4gpUSaKlBylNypLY5KVka9r+LImWaO/M3FvNdqaPxTdIh/wRkfrcckMuY0saTU\ngeRKZ1vVtjFXzTELFk89TIQA/zhwDbAx/P/fA8uB23XHmAJ8CkEVzGH7dekiB+kO8dLrX9RZX/Rz\nvLeJrDQX+++tMZYhy2nCUbRfreyek5ajcqwVLVJvc49EUWYRV+RdQcPRsEbcnwnOTpFD+w8PwW1L\nwTagttc10EXdm3XYhjIYskUI/UNVMONVXHIh/2dxFjxaQ0PxCsh5UytnhqiF+aEPpPH44cfpDvWQ\n1nEV3gXZ/KLqAS7bspTBlFPGdiPKslVVCQeeLT3Iky4/V71dTW+lRmPMC1Swclk6XUMBGo42YG31\nMtwdDuuPgCWUTk7P+3jp69txuwVnuzCzUKUFKvQ/veMy31XA+14+xLafSQbteMfH6vF+X0sXoMz/\naAnHTEx9TIQN3JTMSYQ+z3Zkzu5kQbHx2guE/brhaAOW4ZQoLat4scgx3TGjjhW/mU+wLyjODeee\nVoS3t9BLWUGZ+nf1umpRsuyv4Y6GjRGM3kIvB754gMLntpNzqor8s5XYjn5UDYUvcM3GNZwPCPvw\n0TNvExoKUTWvig9etsLQ1ntzr6QitJWclNn05Igq7Bmf8mPPbjUI77T2Up77ejXNwWbO9p2lf6iP\nYPpeGprrmP+T+QzZg4axOgZyIasF2wbhF8i91c+pj5bzl/fN4r+l6zjV3kXDnGXsOR7ml7deyakH\nH8axs5btPmE3H95WLyg64UITHmc2luEUkEF2dHPa3cCmZ0TF+Nnu2ex5Z4+hxBmgCm8QO5yUG/xR\nxTaKZ0gsvSqaYRKr7N1YcLFVd5+uOFca4Qlglu7/WcDxyIPuu+8+9e/y8nLKy8vPsdsLE/HykyQT\nCvVwZ3ZYUPRlMfD4Flgf7jds+jgceF09R3GK1dTUMv9+F60I5scc9xy2Vm1Vz0trqKbqYYn9+yHQ\nWwPX+bG52xia1YizfREfLruEh31C028+DGcaRaYs+8ZyNRR+xbV+nnqxGKTjyNaQmvu6RCqhMLMQ\ni2xHtgj6XnFWCe+5ZTPDB/dDH2SnZXOqr4UhmxbcAuDMe4cvNKzHYXVEzUfvYK+mxliHKMwoYlZm\nMXtT9jAEWO6cS8A2xJ4T4aRb0nGQEPz08DjI+RulpWHzQ6qE961a6vrCbf7kEEVf8DMwsxHZqlUR\ncgxqdMP9J/cDUFZQxmOffIzLfxTmdttduFJcnO45bRDOm+s309bdxvpH15OXnkdHfwcF6QVqnhQQ\nQlupSjSW6vEq/fHkfgJ9AfWzizXXyERi9+7d7N69e0znnKsJxY5wYn4YaAFewHRijhvj2faOJRGW\n/tg9+87SEXae5bsKOHz7oagK5vrIPsU+GyuaT4GhAMQ6P/aCJlwpDvq60rnuw27ODDVz5LBIdHX4\nVVEf0uuF9NsqaDxRpwbhzL1nPac9dVgH3AyntEdFFgJwwsts1wLaC/6gRp3qTTkqhlEFdMXcCv7a\n8ldau4W9flAexGF1EBoO26bfXUhl8FkG1okgmAxHBl2haDu9Y9BDRoaVQJ/WV9VcH3mZotq7A2F2\nantbUu/lJQ9mqwKRIQevfu4NrpxdbJhvJed2c7CZVb9cxariVbzT/g5Hzh6hvKScdzvfjaokn5uW\nq3LcS6QSQ/j8eKrHR5q+TCfm5GEiAnkGgS8Df0IwUh7CKLxNjAGxAnNGw1iysumPzZ9RQEc4iLCt\np5Vbf+/nsfW1BrrhDt8ONtVvUh1Zu5q0yjkf2PoBSjwlxix2Su6LUjh05S76Ha10ALhyeOzNTqGB\npsLxnFvgffmkFDYhrXDxi6oHDP289I0aVn3PzxNf2cL9e8XnRf+ySDytw1boyUGSLyM44w90KAWA\nLTaVTWOAbEc8pvDSXgcvfPMQm57xs+XqLWyq38Tb7W+r+UPcXIbbt5n/eetlHFYHDpsDQkBLGeQf\nhHBhiWUlC/jpR3/Ksp8vo3+oXwj6oQCnTrWogjV/nZ/K47VqHvU0exoBAjBs57q567jjqZtxOVzq\nrsBb6GVb1TZAVG9/56vvUL6tXG3vyaNPqo5c5ZzSglJyXbk0HG1QFzn9s+ByiWchsnr8SFDz3hSU\nMds9G7fTTdUjVROattVMFZs4zECeaQ59hfH9n98fpRXrX4bQUEhlnOzw7WDuv60QuTbCDI5YFcz1\ncP6z05BPW2FbKI4zPZPBoHFG4lAVGfkBunISD+1237WKDknTwHNScznTd3qEMyCrdxGLLssWjsaW\nUvj10/jWSQZNVAu1B5vVTrrDZSzGYHWy5tBhzpTfxN53tf59C0TUpz7KUw0uOuGF39Qb+prx/2ao\nDBhF+wdRqsxhc5DmSKM52GwQWvpcJC0dLbR06VLfInYUD1//sHq/IsPn73xi84iFN2JBuf/KePSm\nlFgBQucDiVb7GQkXwiJgVqW/CKAIbH2KUD12Ne1SAzJefvdl1VZaLBWz+sAhQWc7UK+WHhsp4MOV\nogWXKMLbk+ohzZ5G+TaRxjRt/c1U7SzXcoSEoYTNZzgyqbD+kBWLR65gHuk4S9EV87WFMrki571x\n58TtdJM+OAv6s3jp6BvYBnKgN5fsbGhpMTqJa26owW6xgwWG5EGD8AYYGO7nr4tXkJmapn6W2V3G\nlg+IBW7pe5aq1/H8554XRZJPLSBjYxV75lSwak1QFBIe1CiEivD2pHrISs2irbuNXYd3GQJnrvjx\nFex5ew8p1hQe+thDlHii09I6rA7D/YosKtzcGXZUNyeeplhprznYLFII6xbhWAFC5wPJSBV7sVS4\nNzXwKYxEquaMVNrK74df5mcz5DBqwnqNefG9frqLdhEa7mfJzCVs/8R2LX9KRP++P15tCHFX8q3c\nvPNmreqLziarwGF1cOWMK9XUp4r2qtf0Y2VRlFIlVYv67vsf4KqfLoE0YXcu7KhieObztHa34hjO\nJGTtJKP7Sj6wqISHfduYc08V7R4jjdFzspLA8Xy10PKh+4VmpoTHj4SctBzOnrEhv+OFxx6m8hop\nXHUorLHa02hub8aBi30HOziTHtbWz5ZAx2wc7zlAyHGWrJQsOgY6sFlsrJq9isHhQYNtX7mPc344\nR52LoswiAI53Hld3PtkDpczf+zRZKdK4no3RoJxbVlDGic4TtHW3TZg9PBkc9Auh5JupgU9zRIY3\nx0Kk1hV5/tDbSwCwD2UCEXmfJZhd1kRbTyuBPhHQo9dWIvvf/ont+Bb4OPCFA/gW+Dh651GKpWKV\n+pblzOK9+UIzdjvdgChjtnjmYlFJXtd/pKbfdKZJFVj2kAfbcBq1r9eqWpT3ZyuwnxJ0RWsog7nv\n7aJ7oBuAEL3wxtV0/aiR9F07kVIlrdqPjsbYmf0czN8Bcxr5/+2dfXQc9XnvP9K+SFrJ0spIQpZl\ny4CNsY5t5LAOburE4jSi1JBIic+2ueZwwZyyiUkoubT2JaH3QlpoaZ0XQmg5cXMwSY73gmwu5gpw\nqASWiLm8lFcbbGwju8LGEbZkS7Ks95f+MZrRzOzM7kq70u5Kz+ccDqvdnZ1nZr3P/Ob7vLXOGV+Z\nvX3H25TklJCRnjG2jfFnkePKob23nVHPGbi8ATI6SRv7WWkr1k5lxdrQshdXkXI+8rp90F2ipF26\nzlE6p5QDmw+QnpbO8OgwTS1NHGk7Aiiac/XSau17dDkUndvj8rD/9v3andbw6DAZ/aUseW0fr73s\nnfS/jUio275y6ysc+d6RSX/OZIhU9h8NsRx7KiEr8CQm1mKM9eth774O8v97gMb7xgOCVsNpIbQp\nVLj961fn52rWavqwqumqQcLTF04b5isevPNgSIfE5vPNXOi/oDjwYQd8uhZ36WFl0rueI+tJW/gf\njGaZim5AyTYZdSvd8oayyOqooL8jn4o/Os+bnzcZyv3BOrvC3Dt8RUEFV1yyiO7B7vGiIyDtQilf\neuOkoR2sfsW3eO5iXj7+MuWXXM3hw3Amx1jp6vw7pyZBZTmzyHZlUzGvgpKcElo6W2g+30zvYC/n\nes/xpQVf4vmNz2v6tqqvF3u9tLZKoc5MRvqBpzixNsQPt72+pN7955twuUZDqjHN26vbNJcH6HId\npavdA88EKf7uRlrnWN+uWt3KBuoC1B2po62nLXSq+hiuNBeDo4OG5/L+UM3qNQNK/xXz0AUrLhZQ\nNFLBcF8O7V3dcIXSWXBdxULDseon37T1tI1nYGTm8e9vtdB70UWn92WlhH80natff48P/n0loKTn\neW8JGNIr9ZJS9dJq3A634cJZ8M8FtPe2k+XMYuWlK7UsGEMfdB2q5LRsa4DWf9uOb7mX3buV9r+T\n+bcxEwJ8swFx4IItVtN8ot5G12fEfczPqlPbaVkZ4I37thvGiUGonmnWug2MDUVYVeQjNzObpk+b\ntFzsvJ4KPrhnH3l5GDI/GFWmxoc4c92Ahay+y+g9U4KzqJl3Nr/ByoWKHKG2nD1bVMuwc1xvPnjn\nQdb8ao02uAGAnrngOacd88DOWm31W7PHmDWhlv77SnyUF5Qrq+pzzZR5lb7fD3/1YW7ceSP7b9/P\n5hc2a+891n4s5LysKl7FK7e+okxGGrugZv1FgJYLk3fA+iwPc+64OPPkQQY6CBrmVZfHo/xQc3Lg\n/HlltR1pJafmeedmeegC8nt8XHlqO282eaGpli2tymrUvLrTp4Hpte4Q0pS+3K9sqtds3la1jet+\nfR0lC7LZ3LCR4IYgq+evVuSEnnxov5IR0/xK9bNAyfS4cl4Jb2a+xhDw4JtbqF2o2FOXFqB1bi2M\nOe/8zHwO3nmQrfVbjc57FHArejt9eQw8v43S0nHpwiprQr1oqV0fQQlCAnzc9jFDI0Os+uUqVhat\npGZpDTtqduDf5afheAMri1ZSmluK2+FmR80OQ3VlbS1UPhk59z/cKltvrzl3XCouUwsJYs5QzCl5\n+rSqZY8t4/EdHRQUQHc3hinogboA8348j7n/NJeq31QZ0vnU/hoH7lcCRMd/VM/Ba9fAvV4cPyjk\nb3/cEjF9Sw14OtOcVF1WxbqyddpraYPZWl9u1fGXecsMvUICdQGCG4JKut6jx3Fd8pntOXCmO3nv\n2+/x2QklBTGv28e2r4ynpfXnHIXMscBpuos/Xfyn1DxVw+5Du43OOw2tiIfMTvi2j95vrqezXzk3\nasCss6+TRY8sYskvluBId1DzVE1IdouvxEfJnBJaLyqB46ZPm3A5lHTAXX4lSNy0qYkXbn6BZ7/1\nrHU6ZxRpduG+B32Az2rQsZA6iIQyzcRLf4z0Ofrb5IKsAkjD4Ez85X66d9Sydy8UbAqw9EtK50F9\nmbb6vnCrsrx/9NI1NhXe4/QwNDLEwMgAc9xzOLj5oJY9odr7QesHdIw5PlXbvXnXJuqP/J7BsZmR\nl/X4WfhmrZa+aNWbWpUTPv3qeADVme4kLyOP9t523OlucjNyGR4dZvgPK+lqzYfnduD/mle7S/jw\nc2VSjzcjn/e/Y0yHBCX9caQ/i2GXLjd8JB3SFbmmtNPPyZ+Onxvvw17t7sKd7taKnkC5IDnSHErj\nrzS0oKjdNKFw6aPRpNlFm0YnbWOTF9HAk5B4VZnZTZ43B+T0/TzMvU3o8xIIwOnrdf2ldaPK5mbN\nZVnBspB5jXrUHGpHmgOPy6OMQBtDb1ekHht6h5PxdL02ts3vh+2/iTw5Xc1Jz8vMI1AXMGS/APCh\nH99/1lJ+b4DnjtUacqzVzBh97rNaRr7znefGJ/fo8HT4OHRvPQ+9Nd6/+63P3qK9tx2Py8PqktU0\ntTRpn9XW06bZU720WmsVm5eZF1KBOZn4hBlxzKmPaOBJSLyqzFQHpA7R1b+mOkp93+0CTwFX5F9B\nS0fLeNe6TMU5rN9p7H/y/d99n1FGDU7nyl9cia/EF+LI377jbcr/pRxHusPgvCuKKwzDfT86+5H2\nvNrFUJ8F0tXfRZYzC0eag7e/uALKlIDftvuV/Xnra6n5l9BVaXBDUOunvfmFzRR6Cjlz8QxH2o9o\ntqwoqOByx3aerIeaPcZzp09rVD/L4/IwMDxA8/nmEOedl5FPT3c6ufNPc9W/zaNvuE97zZ2uDF12\noOSeq9q2N9PLgp8qTTtzM3L5+Q0/1+5M9Bc2VYPWj3WLtoeJGauBxMLMQ1bg00w8q8zUVae+/4n5\n1hkIWZGaV/52NqmflZ6WzsjoiOW2YHRCznQn119+PTs37GRr/VbDnYJdHri+34YZbUBE5fgwCX0V\nJcC8n8zTmmzpOxKW5JSwev5qQ8rggp8t4FTXKZzpTt4JKNOD7I5FrXp0pDn48oIvc6LzBOd7z4eU\n29uhz/A413tOSxfUn0MrqSPW9FE9kjKYuoiEMkMxl2/rf5whaXtjeupHFetpy59YaXFHXwdX/uJK\nrQtefmY+x+8+HlHGsFpdOtOdfHLXJ4aLjd3UHme6k6GRIYOt69fD3ksrtfRFvRPUN9lKG3UxmjZI\nXrePD/5GkTjqjtTRP6y0CrgwcMHSkYLi7HYf2s35vvNUFFfw4ecfannqJTklLLlkie2UIbVEXmWO\new6Zzkzt3BVlF1mWo0+F1KHX0Lu+OS6PTVayExKDlNLPULyZXryZXp478lxIpkFIifpYOXzbL4OU\ndk6stHhr/VY6+8bateKg8bZGy23VrAa1tF5FlYsAhkaGQppt6VuXFnmUKTz5mfm8E3jHUAYdqAvQ\n9c1K3PMVGcYsP+kn14ymDUJnKZ2P1bPlLqWXiprx0XCigZaOFsvPUOMK6p3A6QvGzn8VxRWavc50\nRXnMdedSnF3MtfOv1Zy3K00pgb8wcMGQwTM4PBgydCFQF6DmqRrrNriTQM082p25nqY3O9i7F5o/\njl2yE5IXWYGnKPrVq3llbJgoP6ikCapFJ1t/b31LbXWrbV4hR9tOVP0sV7qLA58f4EzPmZBKzKPt\nR3E5xudxqttt/9p2w2Df4IYgNU/VaHZ4nB5WzVtFbkYuhZ5CWjpb2N+yn2HGBgkPeRn68QkyvruG\nkbzj48MaUJzwnr/YY+g9Hk7GUVvWrihawap5q2g+10zzuWZeuuUlHnz1Qe0z9DKIN8Ortew91XlK\nCwir6GUVfcbPRFbHdrKI4fsaC9rufr6DLa9KMDMVkSDmDEZdDaqyhf7HqR/NVl09lskxpqfaDYCw\nel4/nxGUdqI377qNF27ZE2KP3qnoHZPaG0XvQPT7UrsOBuoC2piwd06/w5kepQ/Kpj2bQgpP1M82\nTNNBaaD16qb3efCIl5cuaaVrYPy1kpwSLV3PXFhkkEWGneAYgs98XNu9m8PXXkduRi5PHXxKk2nu\n33c/hdmF2qCDx28cH0ihnovtX9vOTcGbDA7ckebgfN95TnScAJQLonpcakteK63a7LDtvkPtjqbI\nx8KxoK3XK8HMmUwsDtwPPABcBawG3o2HQUJ0BDcELbXTQF2A1/+4Dq7tZ073Nfz8f++i7FJvSDaI\n+ZZaddbpaemcuXiGjr4OyrxlWvWgyvvvp8EtGPZ3tP0or598XXNwqhziK/EZsk1UrDJx9E5JP7ty\nlFHDsW58ZqP2muq81VQ9NVhZWwuZD45nh3hcHsoLyy3Po2pLRXEFJw8uor01Ey5vwDnk5V9/lset\nexeG6N4DwwMGe7fUbzE4SfWxWiQD4wFRVVYxTzzS32Use2wZ1y++Xksv1F8QA3UB20wmw7+JzbLa\nng3E4sAPAt8AfhknWwQT4TII7NLEjrYfZcDdCm644FEmn9f6aw0OxzNUSsbT9Wx8dryXtOqsR0ZH\naGppIlAX0ByQY2gOw84LZHWs4v/fu8Owv7ojdSEyga/ER7Y72/a2/Vj7MZzpTo6fP05nX6cyjk3n\nlLKdSh+UiuKKcac8dqyFnkJt5b2yaCUdfR1a2p+eiuIKLVjZM9hDw4kGljy6hNXzV2vSi3n1vPGb\nXiVQmt3GUHYDvl8vsbyBdTlceIisLQc3BLltz22kkaZ1NAy52JhWz6Dc6bx49EXDpB/zvqwu3pI6\nOPuIxYF/HDcrZimRUrwmMu9SpfmIBzKVxysKKrQfvEGG0BXKBAJKLrh+tbiqeJXBUfzttdu48ZEt\n7L93u6FwJbghGNI9r6K4gp0bdobVW8/2nGVoZIhzvedY+8RaTt5z0rB6VPdrdXfx3JHntJX3ZfmX\n0dHXYXmO5mbNNezTkeagrbeNvZ/spdBTqGWH6FfPwSAse9BDK0oPcNWB6rsELi9azpM1T3L37+6m\nIKsg7HF6M73s+ZYiN1llm+i//8dvfJw1v1pD68VWRUvP9GqzLs2zSQFx1AIgGnhCieSgJ1P0U/Yf\nQU4VbILRUS5PexLvd8eLVDQZ4lnlOX2hiH61qJc9VJvUknH9invTnk1cM+8aGk40sLxoOYvzF4cU\n6FhdoPqGxuWNgZEBLv3xpQwOD3LNvGu4+25oOeJVBvJWoV2M1POlb4S1/9P9LC9abnmO9Ct1b4aX\nq4uvpqmlKSTQuP1r2w1B3zd2BNnyakDJWjneQDrpeFwevlL2FTwuj7Zybulooa23jYbjDVFdXK1W\nx2YZ5vD3DttexMRhC1ZEcuD1QLHF8z8E6qLdyQMPPKA9rqyspLKyMtpNZzSRHLRZ546mKCPX7YWn\nn8XngyfrrZ1oMBhaKKJfLYZDv+J+9w/vsiBvAcXZxTz/354PGaisd1BF24pYV7aOXX++yyBvqIN+\nARpONFDoDHC2aawtQMBYRq6XGQDae9s51n4Mf7mfLGeWYXp6S2eLtlJft2gdT9Y8yRd++QUyHBkc\najuEO93NJ+2f4N/lp/f4Ll5rUk7Elru81NbW0tHXQeE/FzI0OsT5vvMcPnuYk/ecDLEllvQ882eY\nHbU47dlFY2MjjY2NE9omHmmE+4C/xj6IKWmENky0iMOuj4rhVvyrQbbc5dWcczx6r+ip+o0yF3Nu\n1lwu9F/QnGT10uqQC4B+2o+Kv9zP6ydfDwmOgiK/FDy/j4bnvZaTZsyFRY40B81/pfTZ1ldjVi+t\nZmB4IKTC0a5wqLjdT+svakP2qfZ58bg8HLrzkOECdeueW3n6w6fJcmbhm+9jl39XVN+hnmTqVyIV\nm8nHdBbyTEc++YxjorP/7FZ9+tahW14NUFs77oTstjG3mw3UBZj3k7E2sr+tCpkKr6LOxVxWsMyQ\nwpdGWshnBjcEtQAcKNWJ26q2GRyhI81B1eVVVC+tZt+t+9j1Wy9+v/WYMG+mF1+JD1AyVd799rva\nZ/UPjd8ZpJFmORNRPRfqvE5Q9P437ttuuc+373ib0jmlIc4boKWjhf7hfjr6OzQZxYz5fJiJP9Z/\n1wAAD7JJREFUx+zHeDFbprjPNGJxvN8AHgUKgE7gPeDPLN4nK3Adsax09Cs2fbHL4PCgpumaKy3t\nVnn6FWuGI4PhkWHDeDN/uZ8Dnx+gtbsVl8PF23e8TZm3LKTbIcCKohW8uulVQyqcWvQDsPjRxVp/\nEv3EGv2+oh1KYHc8Vb+t0rI81Ak2dudvW9U27v7d3SF6vxV235f+7sJun/G++5lKZsIU95mG9EKZ\nBJF6McdKvH7U+s+xKpaJxNx/mmvbQEp1SIseWWRoRHXynpOG/ZqbRZklk0JPIb4SH4Mjg1pGhdpg\nS51UE85hWJ0rO4dqd3GLVQ6w+746+joMQV+rfSbaKV712FUhF2A7kknOERSkF8okUKsY9+4dn1IT\nT+IR/ArUBTjw+QFA0Y131Oww3IpHunUHZRo6KBKGSq47l/WL12urSZfDpdm8//b9IfZ/9N2P2POt\nPYZ2rKpk4kx3crbnLHs/2Ys73W2QM7yZXg5/73CIxBHNubK71dfLEZHkgGjOTzgb1P3t+dYebWqO\n1T6DG4Jc5r2MDEcGG5/ZGHFf8aa1u5XO/k7aetpY+8TasO9NJjlHiB5x4Cbi0Ys5HFba7EQ52n5U\nWz0vyltkmz+uOpOrHrsK78NeCrcVas2cVLlkeHSY4uxiqpdW0/I/Wnjh5he0z7PSgMPZr3fM2a5s\n7Xm3wx3iHKJxGFb7aj6vVIzmZeSxrWqb5XaRLpLhHLyVjh+NE7bapzfTGzIObjqxugALMwuRUEzE\nsxfzVBHp1tz8upUUom/Bun7xel64+QVt+0BdwNCCtWROSUjb2khEo0nriTY2sPaJtSENoMzbqp9n\nVQik1++tzp+VZGLOcLFKt7Rr8bvxmY0Jk1FaOlpY+8Ra9t++P6x8IiQn0sxqEqiTv5MZuz4odq/b\nSSED/YoDdzvchu3VFqwwlputq1yMtiK0ZE4JBVkFXOK5RHsunJOOVNSkbqtO2rHro6JODir0FBry\nwvUyBygXMiuHarWSPtdzTntdn3mjR72j0F8Aljy6hIp5FVQvrdbiBPFI14v2M8q8ZYbcdWHmIRJK\nCmInP6i3/xuf2Whw7lZSiG++ko63qngVO2qM/U30BTMVxRVcfenVyjYRUhG15wPwXKOxUhGspQv1\nM+yabKmo27b1tIU4X9XeHHeOprvv/WRvyL70zlk/GUhPoaeQQk8hp7tOc1PwJtbvXE+WK0t7Xd9o\nywrNlrFS/IbjDbgd7pBOjFbnIBpN3u4zhNmJrMBTgGhXXFarWHXbFZeuIC9zPP95l3+XrczQ1d9F\nUXYRvnk+dm7YqT1v7uOhH5dmaE17FDpLPZAP+T3jDll1bulp6bx84mVaOlpCVsXlheUhK2f9tlZS\nhOp01eETuRm5lBeU0/Rpk+GCEO7Oxaov+OluZaiD2l1Rba4VDnUfaim++YIUqRNjNHc48QiECzMD\nWYGnANGuuCabtaGn7kgdr518jTMXz2grR6v3hhus7PEAzwTJP+3nvXvGnW1wQxBnupOR0RGtkZV5\nVdzS0WJprz6gubV+q2HF2tLZwtmes5qm39XfRX5WPv5yv3ZBWL9zvXbc6t/qale9GDW1NGnOW23u\n5Svx8dYdb+Ev92v9xMOhnqtd/l2Wwd5wBUbROuR4BMKFmYEEMWNkOkqQ1aBkgaeApZcsJTcj13Jf\nVrm80eYiq8fx2qevaRkqdgE7MA4H/vrSr9Pe0z4eROzz2gaCzeXpeZl5BpujsdccaFQLg3Izcunq\n77Itn/eX+zlz8UxIkFL/nryMPK5bdB2P3PAI1/36OkrmlNie73ghOdiCFVLIMw1MR7Wd+gMPN1k+\n0raRnIO5T0hFcUXIilN/sdJPWS/IKtBar1oNCtZf4Dr7Oi0zI/Rj2HLcOWErJBf8dAGnLpwiNyOX\nA985oF0EtlVtC3G65iwQq6wQ9aLhSHOwduFaLbd9OisppReJYEYKeaaB6dAj1dty/W19tPuKlG9t\nDiKuKl6l9SVRsyZUueLQ2UOavKEfDlwxr8LWLrOEo2ZG2HUubDjRgMvhCuvA1G27+rvYUr9FO8Yy\nb1lI3rVZbrCSH4IbghRkFTA8OqwNs4Dp1ZolMClMBnHgMTKdeuRU7Muc3bGiaAUdfR1a0Yresahj\n13wlPt74yzc0W+z0XojeCU7EWYa7kNm1aFXtMv8dqFMmw6vrHHPQc7q+WwlMCpNBJJQUI9632mbN\nWd+QSq8v202GiWSnfvJ8uG0m0ssknCwUS4ve0jmltumFU43o4IIZ0cBnIPHWZc2Ow+zQwbqqcSJ2\nFmQVsHr+6rAXHP2FST/E9zLvZSzMWxj2gmV3UYvmYpfohlOCYIc48BQkktOJ5HBiXaHHayWo2pnj\nyqF7sBuwH0IR3BAMaUWrdirMcGREDNzaXdQmUgIf7y6GZiRIKUwUCWImiIlW1umJFMyKpMvGGgyb\naFc6u2NVi2synBlA+CEUgbqAQQPW6+vRBG7t9GPzkIdIxzuVgUQJUgpTgTjwKSCWH2ukYFYkBzvd\nwTC7Y1WLa9p72y37jujtzHJl0dXfRXF2Mbv9u3no9w9x5uIZNj6zkcdvfDxiINHuonZNyTWAdbsA\nK+J97vQXN7UEX4KUQjyJVULZBtwEDADNwCaU6Tx6Zp2EEouuGk7CiOY23Ly9fpvCbKWdbDxv4+2O\nNdI50NtpDpxaFdtEi2E+6I2Ps6V+S0iHQLvjjncgMdahG8LsZjo08CrgZWAEeHjsuXtN75l1Dnyq\nMgomE8A0BxPVght1Wk6sjtzuWK2ejzSeLFyxTbToj1cNgOr7m2gtaKd48pLVcYnjFibCdGjg9SjO\nG+BNoDTGz5sRTFRHjhZ9p7vzfeej0tf1soBacKPv2jcZPVYvDQCWx2rXPyVSnxO7YhvzfqMZrFAy\np8TQ38TQI2aKJy9ZHZcgxJt4auC3Ay/G8fMEE2rFYPdgt+0kdKttzAU3a+avASavx05W4w83nsxc\nbGPVdCqa/eqPVw2AZruyKcouYrd/93gDqSmevGR1XIIQb6JpJ1sPFFs8/0OgbuzxfSg6eNDqAx54\n4AHtcWVlJZWVlROxURjDm+ll9fzV2m15NM5XdSIqtf7amCUesyOONkUu0iAKPVYtVqMJMm6t32oI\ngPq2+2jrbePi4EW21G/RzkUwmPyTl4TZRWNjI42NjRPaJh554LcBdwB/AvRZvD7rNPCpJBkq9sw2\nTEXTJyv9OJpjt+tUOFU6tOR3C1PFdAQxbwB+AqwD2mzeIw58hmMXrIvFuU32QhWvStJomc6OhcLs\nYjoc+DHADahDA18H7jS9Rxz4DMfO2SbCuU33HYpkmghThZTSCwklGueW6hJEMkhawsxEHHiKkupO\nTWUymvVslCBmyvctxBfphZKizJS+GdGk0cVavh5L35lkYaZ838L0I1PpkwT9KszliL1vRqqs6iaS\nWmjFRCe6JyMyzEGYLCKhJAl6KaF6aTVuhzsmXXW2SBPqcOW8jDw++M4HIaPaUgHR0QUropFQZAWe\nJOhXYVbTaya6op4tq7qyvDJOdZ2is7/TUKiTSpiLrQQhWkQDTxLi3ed7tvThmMygZ0GYKcgKPAnQ\nr67tmOiKeiav6qxaxor8IMxGRANPAqLRq5NZJ53ugOls0feF2Y2kEaYI0ayuk7mz3XSnwc0WfV8Q\nIiEOPAlIdb16uh1qqp8vQYgXIqEIMZPM8o4gpCpSSi8IgpCizKo88FSpPBQEQYgXM0YDl34SgiDM\nNmaMA5fMBEEQZhszRgOXQJogCDOJqQ5i/j3wdWAUaEeZjXnS4n0SxBQEQZggU+3A5wAXxh7fBVwN\n/KXF+8SBpwASBBaE5GKqKzEv6B7nYD/UWEgBJAgsCKlHrGmEDwG3AD3AmtjNERKFBIEFIfWIJKHU\nA8UWz/8QqNP9fS+wFNhk8d7R+++/X/ujsrKSysrKiVkpTDkSBBaExNLY2EhjY6P2949+9COYpkrM\nhcCLwHKL10QDFwRBmCBTrYEv0T2uBt6L4bMEQRCECRLLCnw3imwyDDQDm4EzFu+TFbggCMIEkWZW\ngiAIKYoMdBAEQZjBiAMXBEFIUcSBC4IgpCjiwAVBEFIUceCCIAgpyoyZyCPMfKThliAYkRW4kDJI\nwy1BMCIOXEgZpOGWIBiRQh4hZZCGW8JsQioxBUEQUhSpxBQEQZjBiAMXBEFIUcSBC4IgpCjiwAVB\nEFIUceCCIAgpSjwc+F8DI8DcOHyWIAiCECWxOvAFQBXQEgdbEop+mGgyI3bGl1SwMxVsBLEzEcTq\nwH8KbI2HIYkmVb5UsTO+pIKdqWAjiJ2JIBYHXg2cAg7EyRZBEARhAkTqRlgPFFs8fx/wA+B63XPT\nUdUpCIIgjDFZp7sceBnoGfu7FPgM+CKhk+k/Aa6Y5H4EQRBmK83A4unY0QkkC0UQBGFaiVceuHSr\nEgRBEARBEIRkJNmLfv4e+AB4H0XjX5BYc2zZBhxGsfX/AnmJNccSP/ARMAx8IcG2WHED8DFwDPif\nCbbFjieAz4GDiTYkAguAfSjf94fAXyXWHFsygTdRft+HgH9MrDlhcQDvAXWJNkRlAfA7klsvn6N7\nfBfwq0QZEoEqxuWvh8f+SzauAq5E+WEnmwN3oATXFwEulB/0skQaZMOXgVUkvwMvBirGHucAR0jO\n8wngGfu/E3gDWJtAW8JxD7AT+H/h3jSdvVBSoejngu5xDtCWKEMiUI9yJwPKiqI0gbbY8TFwNNFG\n2PBFFAf+n8Ag8BRKXUOy8XvgfKKNiIJWlIsgQDfK3WFJ4swJi5o550a5kJ9LoC12lALrURaQSTHQ\nIZWKfh4CPgVuJTlXtmZuB15MtBEpxnzgpO7vU2PPCbGzCOWu4c0E22FHOsrF5nOUu8NDiTXHkp8B\nWxhfpNkSqZBnIqRK0Y+dnT9E0ZvuG/vvXpQTuWn6TDMQyU5Q7BwAgtNllIlobExGJGtqasgBdgN3\no6zEk5ERFLknD3gJqAQaE2iPmZtQamneQ7Et4SxHudqdGPtvEOXWtSiBNkXDQpSATLJyG/AaSmAm\nmUlGDXwNSjxG5QckbyBzEcmvgYMSS3gJ+H6iDZkA/wv4m0QbYeIfUO4OTwB/AC4Cv0moRSaSOYi5\nRPf4LuC3iTIkAjegRPwLEm1IFOwDrkm0ESacKFVui1C00GQNYkJqOPA0FCfzs0QbEoECwDv2OAt4\nFfiTxJkTkXUk4Z3scZLXge9G+bG8DzxD8t4lHENp4fve2H//mlhzLPkGykqiFyXItTex5oTwZyjZ\nEp+grMCTkf8DnAb6Uc5louS8SKxFkSbeZ/zf5A0JtciaFcC7KHYeQNGZk5l1RMhCEQRBEARBEARB\nEARBEARBEARBEARBEARBEARBEARBEARBEARBEARhGvgveVHfWybyRrwAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10fd4b790>"
       ]
      }
     ],
     "prompt_number": 48
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Whitening \n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It is sometimes necessary to transform the data so that the covariance is the identity, a decorrelation procedure called *whitening*. For example, given some points generated from a multivariate distribution:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mu1 = np.array([0.0, 0.0])\n",
      "sigma1 = np.array([[1.0, 0.5], [0.5, 1.0]])\n",
      "X = np.random.multivariate_normal(mu1, sigma1, 1000)\n",
      "plt.plot(X[:,0], X[:,1], '.')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 49,
       "text": [
        "[<matplotlib.lines.Line2D at 0x10ff1c190>]"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXuMXNd937/L3SW1y8c+uKTo7YqPyAqjRGJJc1G6oA0u\nULKQFkl3k5YAYbRbKe0u0ABO3LRS/EBBokaaAEZrBAViQG0j1kGZQFEboYLNBJSxqzoCIiAtJTuK\nHFqsV1Wq0orskeUXnEia/nH28P7ub86599zH3Htn5vsBBpydnTn33Dvc7/nd3/k9AEIIIYQQQggh\nhBBCCCGEEEIIIYQQQgghhJBc3AHgeQAvAPgzAL9W73QIIYRkYXzr3xEAfwzgQzXOhRBCBoZtJYzx\ng61/twMYBvDtEsYkhBCSQhkCvg3GhfJNAOswrhRCCCE9xASMC2Wh5nkQQshAMFLiWN8B8AUA8wA2\n7It33313++bNmyUehhBCBoKbAN6f9IaiLpQZAJNbz8cAnANwPTaDmzfRbrcb9bh48WLtc+iVeXFO\nnNMgzKuJcwJwd5oAF7XA3wfgP8MsBNsA/DaALxUckxBCSABFBfyrAD5QxkQIIcVYWwNu3ADGx4Er\nV+qeDamCMn3gPcPCwkLdU3DSxHlxTmE0YU43bgDPPmuer60Bv/ALC7XOx0cTrpWmiXMKYaiCY7S3\n/DmEkC6yuAhcvQrMzwPXrgGTk+mfIc1laGgISNFoCjghfcJbbxnL+7HHKN79AAWcEEJ6lBABLyMT\nkxBCSA1QwAkhpEehgBNCSI9CASeEkB6FAk4IIT0KBZyQHmdtDVhYMHHgb71V92xIlVDACelxbAbm\n1atGzMngQAEnpMcZ32pqOD9vknjI4MBEHkJ6nDoyMHXhLGZ+lg8zMQkhXWFhISqcdf488MQTtU6n\nL2EmJiE10e8bi3TbNANa4IR0gX6zUF21xsty29Ad4ybEAh/IeuCEdJt+s1B1rfEnnihvUXKN7YNi\nH4cuFEISyOsKuXLFWN5V1eVeWwPe9z5geho4d658t003F6QsYzNksnrahPQqZ86024B5nD9f92z8\nyHl2Y66tlhmz1XL/fnXVzOHBB/3vyTu25MEHzfnNz2c/Tq8BINX3TAuckAR6xRVi5wkAx4+XP9fJ\nSePa8N1NFLGM08aWVH1n03S4iUlIAmXEWFfht33rLeDhh439ffly9eLGdm7lwzhwQhpAv0WkuGA7\nt/JhFAohDaBX3DBFsG4QUi1l+MDvArAO4CUAfwrgF0sYk5DG0UsRKVUkEfV7slIvUIYL5cDW4wUA\nuwD8TwDLAF7e+j1dKKQv6LYrpCxfedo8qzoOKUZVqfS3YMQbAL4HI9yzJYxLSKPotiukrBjntHn6\njpPVom6Sa4h3A+VwGMCrMJa4pc5QSkJKI0u8ch6KxDjLOOzNzeR5+o6TNea929cjC3LuR47kj0lv\nEgiIAy9zE3MXgCcB/BKMJX6bS5cu3X6+sLCAhYWFEg9LSPeRboduceVKZySHPO6+fcCrr7pdHzId\n/ZFHkt0mn/uceY+OGMlqUXdr4zKPi0fOfceO8NT8JrGxsYGNjY1ajj0K4A8BfMzxu7oXMkIKo63T\nIpmHeY87M+O3MtOs9xDruikWdZ7sVzn3fsnWRIAFXgZDAD4P4LOe39d9HQgpjBaFqlLs5XHPno2e\nnz4dP36a+PaSqBWda1MWoqJUJeAfAvAezEbm9a3HAxRw0k9oUfCJTNmWuTxuESuzl0Stl+baTUIE\nnJmYhOTAZh6OjcX90svL1YTW+Y4fGhJYRXq/Psajj7IUbBaYSk9Il9Gx0N/7nr8mSFmiKcd5+23g\nueei44cuGFXEcOtjvPEG48azECLgVVDvfQghXWRuzrgyJiZM+F7S7f+BA5Hfemkp/zGl/92OmdVf\nrF0wLtdPUXeQPkZWt8/qqjm/qSnj/x80lwoq8oGnUfd1IKQ0tKicOhW2mbm62m4PD0fvXV7OPwcp\nhGkx3z70QuPalC26UauPkdW33e0a500HFHBCykWLSqgFLD83Odlur6yY1+bmTERJFis3RAizWM+r\nq2ZBAtrtEyfSN2rLxjdXe3yg3T5+nBZ4XdR9HQgpDS0qoRaw/dzUlPmMXgjKziAMtZ5XV437x+Xa\nqSoaxDfXVqvdPnSo3d67ly6UOqn7OhBSGq2WcX8sLWUTFCmG0uLdvdsf212EUOtZiufUlN8f3k2S\n5torLe26AdhSjZBymZwEfv/3gaeeiqe7pxVSmpw0j+Vl4MkngVbLvP7hD0elZvfsMa/NzACvv16s\nMJMtYfuTP2mO6RvLpqBPTQHXr5s5Zimqpc89T1GppHK7TSqYNajUvZAR0lVCrUTtNtEW58qKSZef\nnEweL4uFnDY3l5ski+9bj1+2xTzIST2gC4UUoepb6apIOq8855wkeHI8mwp/4oTbBZMm8K73WZFM\n2wjMshFZpEt8L6XsNx1QwEkR+tX/mHReRQspJR1raSmsXsnx48bPnqWuSdJGoO+YZSzQRUMFiR9Q\nwEkR+tWaSjqvss85y3ih4YGnT5vwxc3NSIRtpcIs8+6lBbpf7waTAAWcFKFframk8yrrnK3gnD2b\nbE1nJcnnPDeXLYqklxboXlpsygIsZkVIPXSr1sjiYrzWykc+Yn6emQGOHjWRLLo+io0s0TVYbNSI\nbuzQBHTdGHuerhoz/QproRBSE92ybn0+ZxlDrrNDe9F61XPu17vBJEAXCiHVkaUvZcgYeYtTyWP7\n0uS7OZcy6CX3TrcABZyQfOQRrzIs3bxj+CxUHQXjw3W+uuphlUI6iBa3BszEJCQfTz8dZSPefXdY\nZqHMGhwbM37wu+4C7rwTmJ4Gzp3rHENnLrrGCDm2bTCclMl4+bL/867sS9nA+dat9KzMMvGdD6me\nuhcyQjJj3Q7ykWYRS6vRVawqLe78yJF4iKBuaBxaWfDMmXZ7fLzd3rPHFIJaXEyu8S3dLLLqX6uV\nv944KQ5ogZN+IU+NjSKMjZl/h4fNv7o+iasGyPKy6cgDRNarrW8CAMePd9bzkBby7KyJHrl1C3jk\nkeh3u3YBb77prk2i52Et6R/8wESjfOtbwFe+ElmyLkv7xo2oNsvhw9F7JyeBl1/21ykh9TNS9wQI\nCcEKD2CEp9vtuI4cMYL97rvA3Bxw6FAUmre2Bnz5y0ZoAeChh4x42vndc48R66Ul4Dd+A/jYx4wd\nfflyvADWjRvA6KgR/scfN6FyQLRYjI2ZMb7/feCZZ9wFnfR1saI/MgK88475+Y/+KHq/qziUfO3x\nx+PjW1eGnDN7Wg4Wdd+JkD6g6qiEtBof0sWyvBy1VpNdd0ILW9n3uUIC00Lo9Lzse1980cxJZmv6\nomNCNwx7MRyxlwGjUEi/0O2oBO0bTqvxIQtTtVpx0XWF7Onxk3pS2rF37UpvZGDnZTv8uPzkZfXi\nZGhftYACTogbLahZrUtfgwafUMrxd+wwJWP37zcWsf799HS7PTqabT56/kePmk47MzNmM1PeLfiu\nQdL10eccyiDWMCkLVCTgvwXgmwC+SgEnVZNXILTgWetSWr2hY4eUgbUulm3b3FEp8vhJY4WWjZVt\n0uwxd++OFgw97337/DHgRdwlTXK79NpiUpWAfxjACQo4qYM8AuHKTmy1jIj5CkUlNVawlQB9db5X\nV+NWsH3cd1+ndZulZrgUXW0d2zmNj0fP9Z2Ba9GwfTnzVDfU10W6g5rgdvF9n00V9qoEHAAOU8BJ\nHeTxy/qyE7M2J3BVApRYYZDuFbnJKd0ZljQ3hZ2TtOTPn+8Uoc3NaBNTb7jqY83Omt/t2dNunzrl\nPidf/LhL+OR1WV7u3t5FVuH1fZ9NukuQUMBJ35PHL+v7Q87anMA3jku4rXhv326eT0zE3Rn6s3Nz\nZmPUipMsT7t3bzSmbUTs69Jz4EC7PTJiXj92zH0ucgPWzi9E5HzCV9VmZ5F9C0lTN2cbI+AXL168\n/VhfX6/7upCMNPUWMy9lRbTocaxgSis76aHri6yuxn3XPneOFdmpqXb7wgW/y0P75n0RKFbAku4m\nxsfN74aG2u2TJ5PdI1XVMUmK5Mly7KbUXVlfX49pZWMEnPQ2TbzFDP1jLfK+LG6DdtufPp/0kNdT\nft5azdqdI635PXui97mEV34mqRJhSMq8y4ef1iIu9DrnRQtvE/+fFoECTkqhibeYeTrBJzUAlu8b\nHTXWpXQt2M096Raxr+nYbu3rHh2Nxjp2zIQP6uspN1YnJkwijnbnSJHVgjoz027feacZw0bRtFpG\nZEM6AqVZodbCt773kBj1tOtctsg28f9pEaoS8N8B8DqAHwF4DcDDFPD+wuUqqNulEvLH6oo20W4K\nHcanXRz2GDpRR2/42etjBdO6F0ZGzLWyGZD33GM+u327EWmL3vhzIb8HK6hjY6ZYlZ5fFnEM+T7t\npuiLL8ajWrIunkVENm2eTXGFlEVVAp5G3deBlEwTblVD/lhd0SbyNbsBaMeTGYvHj8fTzl0hd2Nj\nkbV95kynH1iGJdpqgtJy3rEj+kxWYZNRJvLzSe4SnwDK8967Nz2xJzQ00HVORUS2Cf/vqgQUcNIN\n8lhRWbP+ujVP+9rUVGcUSKtlrF9X/LWO056fj1vgVtBl5IhL9O0GpBailRUj+GluiXbbnyW5tNRu\nHzoUn4PEJ4Cu0rnaPZQUGuj7/sq2iPvNRZIGKOCkG+T5w0yynrphWa2uxmtrW+TcfcKTtKDIz7vc\nLvI8VlaM5W3D/mZmTATH0FCnEOl6JXoOMjV+x47ovYuL6ddSJxxp/7VdlHbvdruMZKbq/Hxn3ZWq\nLON+c5GkAQo4aQpJ1lM3LCstKmkbajarcWUl7iPfsSO+MSixFrvdlLTuEVczYRvXLX+W4+lkGyno\ny8vu8ELAbCq6rP6kcEIttlYYXS4jXeXQJdhJ4t5kmrCXkwQo4KQpJFlPSb/L80fm6jCTtKEmXRzS\nb63juY8cMcKqBb3VMmPs3h3fnMyS2amrG0pXy9RUZLVv2+YXc93Rx+I6T+n/z/p96boxm5vRseVC\n1HQ/ddN96qCAk16irLAzV0RH0oaa9Gvb58PDcTGanu4M3ZN+Yvm7ublo/F27zO9mZjpDAyVaMH0i\nbd0mVtClJS5jwuW10ufp8v9nodWKu3GWlsIKejWNpvvUQQEndZPFgtYlV6emIqGQqedpY4ZEP/ia\nHMgQPcDUCVlactf7dqWgj49H2ZE+YdfXZufOSOTtOVqxtZ+X7pmVFVOOFmi3f+qnOu8UfIJkffIh\nG6VpaJePvebT08bnX8Yxuk3TfeqggJO6yWJB+0qqams6bcysIYZ6jKTolYmJqEmwfJ/tgHPhQtx6\ntpby2FjnAuSK/rChhdovLRcZOfeDByPLe2goWgRci5z0q+/fHyZcvsVSunxWViLXjY6NJ/kBBZzU\nTZbbVH2rb10D8lY9y5hJUSY6wcc1j6SQOF+UixRXaRnPzEQbfC7hlo804bO1xScmOkMZ7eddC5Q+\nrqvTvbxmekNXRrXIc5cLg11MZHd7kg9QwEnd5LlNtTHNy8uREElBCB3TZ2W7Enyybpb62pTJOHPt\nF5auGb042UeI8EnXjZ2HjoDRi9zRo/EFZefO+BiuPQc5X7npqa+ra0HyZZOScEABJ72KLJ/qa5Ig\nBdclwD5LPS3ULqTYf1qNbesGsdmae/ZEx3C5iPbsiVwzaddF3j24XCxyHtbql+J9xx3xuxx7znLs\nnTuj/YeRkXjav75+rjhyWt/FAQWchNK0mNg0P7f+vev9Pkvd9XpIsX/ZAUeH/Vlkws3mZtxaXl42\nkSvWLSGjRnRCTsh18ZWHld+l3nwdHzfzarU6KxAmVVOUsfRnz8YLZLniyElxQAEnoXQrGzLvopDm\n506LsdZCmjQnnz+73Y78zVrMfIuD9BnPzXXOy1UyFjCbimlJMD7ffZLf2or05KSJlJE1W/Q52Lla\nd4wdZ2am92K8+wFQwEkoZcfE6qp/ummufF9aHY2QDumtlrFubUlV6TKQ4XtSQG0ct0ya0b5bbcGm\nJcDIXpTW0pXnYY+1e3fn2DK2enm587x91rcuhSvnaq1inRrvwoYZnjljxnd9li6S6gAFnIRSdkys\nz9LUBZ9cgprUYzG0fKl2GVjkQqWFSQujtHitRZqUALO66o5b981PpszPz8cXvP37k+uRuNw/eqHx\nnbfP0tdx+Nu3mznZWi6+RsukO4ACTuoiKRrDPmZm4q3AfFZi1trfgElwmZ11VxzU9T6sy0BGgOio\nDdvkITS23C4evoYP1gUi56M3FrWPWifiSJ+0rcfiS6SRx/EVvLLXT2d52rsYCne1gAJO6kJHY1gx\nslamjMSw4uAT6qyJObOzYWKTtPnmS2XXDYNlXRQp0GNj8RhtK7yLi/4OOXpjUc9LnuOBA+4NUpn5\nKQt0ybnahcL6tl2bnfKR1I6NdA9QwElT0GLpEpELF/KneqdZ6Vk3VHXMth5bW9vW6j13zljep07F\ny7e67ixWV43QDw8bl8WpU50RHq5zlIuEvYPY3PS7rfS57N8fr6JoX5ML7OSkORc7l6ZFKQ0CoICT\nUKr+A7WbjtpidAldEr7QNk3WKJvNzXgavI7R9tUCHxuLn9P4eFS3RFuyMhlIu05cESYy63F+Pn69\nZD2S4eHomLJAl76TkC6nxcXkUMCmV+7rR0ABJ6HU8Qeq25uFtuqSJDVCeP/7o1BCeyw7dpY+kK6N\ny1arsw64fkxNxV0TOm47KaXeuj/0voB0J8noGbvASFG3rqlWK25x22swO2t+3r278xz19ckbpRSS\ncEXcgAJOQqmjtKZub5YnEkZnRPrcCKOj8TjvIguWtvovXHCL8MmT8U1afV7aMrbzlS4Xu0Dt2mUE\nV0bwSD99Utlci76+SaGF+vrkjVIKSbjS15XibgAFnIRSdhhhVcfUVqQUMFc0hauZQ1afe5IouR7W\nEtYCpfcF9P6A3ciU8eEuV8399/vj4zV2DnNzcdeODi0sa0HP0tSCbpo4oICTKgi1nMqysGSW5cmT\nfkvR1uawD10fXG7u+QQjpMaK/Vkm0ehj6sSmJIHSIqwbIetHlsJRrsVGN2QoYnGnnUvSuE1vsFA1\noICTKsiTaJP0vjShl0JprVMZzWI/YwV6bMzdNEFb4dYKtf0r5+bi/m1ZY8W2EJMNIaRLYmIinvSi\n/f0hC52dh3QFWTG3i0FWsZMbndb6LtPiLkIdd4FNBhUJ+AMAvgbg6wB+hQI+eIT+8Ye6LdKEXgra\nxERnIwEbxSE3IXWNkna7c9PP5aqwj23b4guAbjrcbsfrppw+7Y5acbUz03VbfC6Z48f91Qc1vkVw\nZSV+/ezGr68WTMiYpDtUIeDDAF4BcBjAKIAXANxLAR8sQi0nLZg+KzxtQbACp2tau3zdFl2jpN3u\nLA6lXSAyE1PXVpGbp3v3mjnr5gpyDjqLUoq2dPXI3pYyQzQpRNJFSC10eydQ9h2UhYJfjCoE/G8D\n+APx88e3HhRw4iTEWk9bEOzv9WafTj2XuEICpSDt3x93l4yNRVb7sWMmRV0uAPbYcqNULgD6LkOL\nn6vtmt503bs3X+0RXddbzsPeJcga31lLFYRmZnJTshhVCPg/APAfxM//EMC/p4D3HlVZSysrxgrP\n2/TW14y43c7uQ9XJONbyPXasM35bLwCtlt/lIjcdR0bM++ziYMXP3hFs22Ys99lZt+skpLmExpUg\nZOt5+3z6WUoV+OqQ+64vNyXzESLgIwUFPGiFuHTp0u3nCwsLWFhYKHhYUjY3bgDPPmuer60BTzzh\nft/amnnv+Dhw5QowOZntOK++CvzlXwLPPJN8nJB5PvKI+byc0yuvAIcPA6OjwNmzwMYG8KMfASdP\nAr/3e/H5XrkC3HsvcOsWMDICvPOOef3b3wZee808P3ECuHzZfM6+Bpifx8fN2JKpKWBsDHj9dfPz\nO++Yh33fwYPms3/yJ8CP/Rjw3nvA888DS0vmOEePAm+8ER17bAxYWDDHevtt4LnnzO/ktdPfiZ7T\n/Dzw2GPA8rIZAzDn+5nPROeS9j2Mj0djXb4c9v/gyhXzvscey/7/ZBDZ2NjAxsZGpcf8IOIulE+g\ncyOz7oWMBBBqLenb4qyWuz6OrQcyMmLcFBcuJI+X1g5Nbzwm+cXb7SiiRL7Xlk9NszZtDLp87N7d\nWQjLjq2vrc6kbLfNHcrevcalozc0fS4i/Z1Y985998V95/qO49Ch8O9OW+llRRQRP6jAhTIC4CbM\nJuZ2cBOzZwl1PyR1mJGV+nx/tElCoN0PPsH1tUOTj/HxuLtgcjIs6sXWwJb+Y1/97KQKftYVMzZm\nYtVtdcK5OdOTUkaCyIVCX095vX2RJ9avbYta6TBH6WKSfvYiHXbyLvgknCoEHAAeBPDnMNEon3D8\nvu7rQEpEC2iWBsEupPgODUWiKMdLs+JWViJh2rkzCoezlujkZLJlL0P8dCQJEPd1j4y4y8dqX/r0\ntHlfmsgD8Trk+nr6xFjiSon3fQe6aUSICId0REr7fukHz05VAp5G3deBBJLndjfJItZ/tD4hsBEe\n1hLV42VpcAxEhaA2NyPxS7I25TnYuVu3x86d8RBCOYYOi/SJuRxPPoaGOqscuq6n3JR0uT1c19u+\nphOc7KImY8rTvusiVnQ3k3P63T0DCjjJQlH/tsX3R5vWCUZaopK0BCCfJazbkllBnZz0R8HICBHf\nQ87DdWyX4Fuht4WqRkejML405OIjXS9JUST2NVnz2/rDswpqU63ofnfPgAJOshDi33YRKvRp7hZd\n08OOOzsbF0WXBb283OlXlm6CEyfim5PWNSI76uh52mPef7+xlBcXOxsPt1rRMeymoT0nm6Cza1dn\no+CVlaiZw969yRmQdrNUWvFp6fgWKf6+Ddm07y9E9Ouwhpu6sJQFKOAkCyH+bRehQp/F3aLHDREu\nl+VsGxVoa1m7NI4cidqOnTkTF3u5sLgEUZ+X/XlzM+5imZlxN3MGovR+F/ouwpWO78OKv93gdFHG\nnVcd1nA33TNNABRwUoQqNqqSjmHHtRtv27ebTUafsGxuGqvWbmju2hUXrZUVt3vj/vs7RdIXsid9\nyFnCJuWCIaNL7O/OnPHvF8hj6pT6NLFNqvmt55j1zitpDFIcUMBJWWQJDcwzhgsbgXHqVGfBqvPn\nO4tAtdudlq2vDoh8TE/HrffRURP65yrulHaurprfcuzt242o2v6ZvrBJOdfpaX+hKVdRLUme0gV5\nxLjfreE6AAWclEXeW2QpaCHWoEYKlA570xUG5Yao9Xu73DXSVy4tY1c971CXgn2PK9pF1h+R9ctn\nZiJx15urcuM26Zql+bhdwuo6n6QSBaQeQAEfPMreTLLjJbUGSyItkzBtvq7Gu/Z9usKgTsjRm5PS\nN720FPdzy56cutZ2mo94dbVT/OU5yoVLtkiTc9WLhr378DVEtvhcOknX1bUY93tERy8CCvjgUfYf\nohzPtgbLgrwdv3Chs5CVHN/Gb4f6nHWBKVdIX1LkhV0cJifjPTll/Lj0QVtRHhuLLyraNTM7G1+g\n7HH27o3cQbLJsuyNuXdvdNykhsiyL6erYmHS/4OkuHH6sJsDKOCDR9Y/xDQLuOgftryFl6Jy5Ejc\nsve5CpJ8qy5/s06skX5h+f4774yLsB5XumcOHowvPHIjdHY2vnCMj/sXKF1jxZ5Xq2V+Xl52W+t5\n+kdmaW6cdp1JPYACPnhk/UNME4Iy/7ClqEihmpvrtHLzzl2K6bFj8bHk+6XLQ2/+Sb/7xESn794K\n8diYuas4fdr456W7Z98+c172OHv2xK1u3zn6ap/oxSptYfV9b/2evdhPgAJO0vClXHcDKSqumh+h\nC4XPL29bg+3f35mi3m5Hm4mjo+6O7hYpxFNTncfZ3DSW9qlT8fda4XelzWurO+QaSbH1ZVT6Cm3p\na+WKP6evu9mAAk7SsEKQJ0KkjONK0XGFBbqQIrRjhzuE0LpopLC5Ckvt2GE2EWUEiKvbjvb/66YJ\n1mL21UbZvTveys3ObWUlSiDSm66+OwbpD89SJ0ZXN6R13mxAASca3x9pGZtYRQXA1XjYhd6s1O4T\n7aLRv7fWtyv13i4qMnbblf0pLe/x8ejuxVUjXC8qvtomWoTl+Vgh1pu5ad+b707HZbnrBSNv1yRS\nDqCAE43PYivD11309tzVeNiF3EgcHo6KQiW5aOTvL1xwt0OT7hRZYtZlOUtBlYuFHFfXLtF3AHLj\nFugMFZTn4/t+0r433+/T9g/oZqkfUMCJppvhYkXHlmGBada8bNYwPp5crzrJDywfcjPTJXy6L6S1\nZGU1RRu3DUTuFLsI2Ouzc6fxaVs3zMiIycrsprUbsgnaasVdQ6HlAkh3AAWcaLKE5RUZu+hYackz\n0lqXfmdX3LfPDyxdGDq5yOWXdm34Sqv60KFIwI8dczddlq4ZWQd9dNS4X+680+0Pz4L2sbsyRH3/\nD3RLN1IfoICTLJQZoVB0rLQCSzISRPqRQxoeyLhrV9q4ttCt7/rsWRMTLq1/GastxXz/fvcmobTW\nZQKP6yGvW5YFUc5fV2hkC7TeARRwkgUbZjcxUdz6ct2i+0QopGVXWi1x60OWIiqF15WtmDZ315jS\n2h8ZMf537UrxCaCufa5dFrZ+uMsfnqdNnczyPHEi7Bp0w8XG6JZ8gAI+WBT9QykzlDDNh+yrvOc7\nrh3vnnuiUEO5kWgF0ReJImtxS3yLh7XQ9ZjaanbVEvEJoM/vvLwcNXuQx037rO885F2F63soo7Jk\nFmjV5wMU8MGibLdF2fjGDz3u6mpnGrsOiZPi5arFrYXcd820INraKGfPdlZFlK6RY8f8lm5o8o2L\nNGHN8t1XLaiss5IPUMAHizLrlnSDVsvdYb3VMiF1e/b4k3h0fZLh4eQEnrk54x+XCTCutmy+a6ZF\nTv6sa6Po6JQ00mp4ZylfG5paL6laULv9/6pfAQV8sGjqH4qvJri0hqU479jReQ5S9IaG4g2BZe1s\nOY582FZpLheGyyqWY549G6/Vol1NWe4gzpyJb7qmRc0cOOAeUy8wWb77pv4/IXFAASdpVLHB5KoJ\nLpNYjhx3llDpAAAPNElEQVRJzkhstzvrgktarc4Udmltb98eLxebJoi2+42M4JBWty68FSqI2k9u\nE4d02F/aZmi7XdyKdoUacpOxWaDLAn4ewEsA3gXwAQp4b9JNf6ir6JStCW4r+mmLVouSHcPGTPuS\nS6yg2Q1NLZYHDiQLlY3A8VnWco7Ly9ksWBtXrhcpa30nhf35mhEXtaJ1HfbQ/wNZIolIMbot4D8B\n4McBrFPAe5cyLbmkdG1bDMr1mp3D9LQRdl89bVm4Sh7b+rtl30gdopckVKur8dhuW/tEiqQV+KTu\n7j5cmZ+uDdATJyLrfufOzrmWKZKuyJqQ/wNFIolINrot4BYKeA9TpiXn6/xi/ci+zTY7h6QCVGmi\n4RK706ejCJSJiej9cj56nNFRd+0T3VA5FN2n8/7741Eq8tjT02Zey8tuUS1TJOX3nuX/QNFIIhIO\nBZx0nbTOL9rqTRKLtPocPtGw1nNSko91WbiscFm46sIF94ar7VupxT8NOQ9ZtVAnKOmORK7r1ASR\n9H1/3BgtnzIE/BqArzoeP5NFwC9evHj7sb6+Xvd1IQ7y3J5bK1e6LjQ+0XHVG8kqDvZ1Vzq8Pa7s\niCMLStl6JnNz5vnoqDl/V0szLeJJFnBSeJ/rDsOeQ4gbgyLZ36yvr8e0sgwBD4EWeB8QktCS5ONO\nSoJxiY4Wx1BBDMWKnXZ9uNw1LpG2G652AZD+6aRmCLpo1MqKP3pFz9cVI1/kGpDepkoBP0kB7218\nrdVCfNxpSTAupNBNTvoFy1XbJIuQyTnKKBQrpnLz0j5sV3mdoBMahiivSVICUNIYeTcIKfb9Q7cF\n/GcBvAbghwBuAbhKAe9dfBuJaT5ul6CF+GqtgE5OdrpfXHHjPhdEEtrF46rnvblpNg1l+7Tt280C\nY0MXs2Q3yqJRuvpg2vzL2CBkNEj/0G0BD6Xu60Ay4GvBlcd9kfQZ13uS4sbTXBASnzsjKRTQl8Hp\n2ngMPR8ppvv3p8+/jA3CJmx0knIABZxkpa6NMl3rxBU3HtLVvd32uzN0Z3eJFVeZwTkxUaxCow6B\ndCUAle3y4EZn/wAKOGkyPl+3bCKc1aJcXY2H/C0uRp+TFrmuQWKFz4q/rYmuj59FcJNCIC10eRAf\noIATSVFrr2xr0eXrlk2E2+1wi/Lo0XjlQZcoWis7qdejjAZxZXhmFdy0+dPlQXyAAk4kRa29tM9n\nFXgZ+aKFMisuH3ZSl3cf2pWjz7dswaXLg/gABZxIiopP2ufzWKdHjsRD+fK6EWQRqN27466TLMhz\ncDU8rkpwfY2VyeAACjiRFBWfpM/rokx5Us2l7zsrm5smfjuvcFtkWr3td5l3vNA7Etf79EYs/eOD\nBwIEfKgiAa/gMESztgbcuAGMjwNXrgCTk9071sIC8Oyz5vnSEvDUU2GfW1wErl4FpqaA69eBQ4e6\nNsUY+to8+qj5eXQU2LULePzxzuuV9XqOjwM//KF5vrgIfOEL7vfJa3fkCHDwIPDSS8Cbb5rXjh8H\n1tejOZb9fVb5/4SEMzQ0BFSj0YnUvZANLFVGOOTtaJ/lrqDIJqr+bFLLtPPnzaaobZ6cdwNT9+/0\n4auXMjvbbh86FMWhl9l0WsJImGYCulAGmyojHHzi4hLdvEJcRGj0Z/W10T/Ljczt2+Pp97rcgI8d\nO8z7t21rt3/u58I6wet5+LJSy/w+GQnTTEABH2zSrNtuNQiQY8miVa4ONFmEuIjQ6DsEfW30z3ZT\nVKbYLy93brom1WeRhbRkiGPSOet5yHP2FQYrCiNhmgko4CSJbjUIkMjkGZv9mFeI5TGyLj5Z3Q+b\nm0b0dSNkvekqx923z13rWz7ynLOvSiHpb0ABJ0mUfevsElXrdpCRKWVYfK7FJ0nUy1g05Dg24cjX\nkMF+1t6ByCJXZZwr6X9AASdJlH3r7BKabt2euwQ5SejKmofP9eIrVGV/X6TzO33Ugwko4KRKqhQa\nLaS+OPQQV4sr4qTofDRFrGj6qAcTMA6cVMlbb5mY4sceS44lTos7zhOX7ItDl6+fPw888UTnZycn\nge98xzyfmwNeey39eFmx8e7z88C1a4y1JukwDnyA6JVOLLrWiN74078PtVaLNEOwESfj4/kt8DRo\nRZOsgC6UwaFXNrpctUbknPOm1vsEMkQ4bcSJT7x7ZXEk/QUCBHykAgEnFTA+bv6dnzcujKZi5zk1\nBezYAdy6BQwPA2+8YVww8vfXrxtXQ4hLZXKy0z0iP5fEoUPJbpMbNyI3zNqa2w3jOibT0kk/UPdC\nNhA04RY9pIKenKfuDC+7xsvP5r27KOuuhD0pSR2AFvjg4LJAu4XPyrxxw1jUAPDMM25rVc5zz57o\n9elp4PXXgY98pNNytRb0zIx5z+JieDEpwH9X4jsP+fq+fcDbbwMHDgBPPln8mIT0GnUvZMRBEb+u\ntDJlKrmNhQaSu95YWi2Tnbm0lJwpaa3yrNmUeUP75OuyzngZxyQkFHATk/gocqvvq563tBQJcjeS\nVcqOMw+JXEnrJE9ItwAFnPgoIoZJ1fPyEmK5lm3dhkSu0KImdYEKEnk+A+CnAfwVgJsAHgbwHYeA\nFzwMKZvQpJuqxnHBiA4yyIQk8hQV8HMAvgTgPQC/vvXax9V7KOAkFyFZlHngwkB6gRABLxqFck08\nfx7A3y84HiG36VZEh4zrvvde4OWXqxVxLiCkLLaVONbPA/hiieORLrK2ZizcxUXjBmniHK5cMZa3\nq3ZIkfnLxJ5bt8xYVWIXkKtXqz826S9CLPBrAA44Xv8kgKe3nn8Kxg9+xTXApUuXbj9fWFjAwsJC\nljmSLpAlu7AstOWZNoek2PYi879yxVjet26lW/chseJZrWjGihMXGxsb2NjYqPy4DwF4DsAdnt/X\nu5VLnHSz9KsvxjytL2UWis4/NLokJFb8/PlscfWMbCEhoIIwwgcAvARgJuE9dV8HIrBCc/asidnu\nRr9Mn+hp0S0iZFWJYGiVQ6bQk7KpQsC/DuBVANe3Hr9JAW82oUJTVqKPqztNt9q3dYPQKofsmkPK\npgoBD6Hu60AEoUJTVqJPt8jaE7PbZD1nlqglaYACTjShQtN0P23WnphNo5fmSuoBbKlG+hVXBmgv\ntS3rpbmSeqgiEzMECjiphG6m9ZdNL82V1AMFnHQVZhQS0j1CBLzMTEwyYDCjkJB6oYCT3DCjkJB6\noQuF5IZ+XEK6B33ghBDSo9AHTgghfQwFnBBCehQKOCEZaEIddUIsFPABhUKUD4ZOkiZBAR9QKET5\nYOgkaRIU8AGFQpSPpDZvhFQNwwgHlKQYbqbIE1I/jAMnuVhYiPpNnj9fTb9MQkgcxoGTXNC9Qkhv\nQAucdMAUeULqhy4UQgjpUQbehcJYZ0JIP9PXAs5YZ0JIP9PXAs7NOEJIP9PXPnBuxhFCepVub2J+\nGsDfA9AG8C0ADwF4zfE+bmISQkhGui3guwF8d+v5RwH8TQD/1PE+CjghhGSk21Eo3xXPdwF4s8BY\nhBBCMjJS8PO/CuAfAfgBgA8Wnw4hhJBQ0gT8GoADjtc/CeBpAJ/aenwcwGcBPOwa5NKlS7efLyws\nYGFhIftMCSGkj9nY2MDGxkamz5QVhXIQwBcB3Of4HX3ghBCSkW77wO8Rz5cAXC8wFiGEkIwUscCf\nBHAUwLsAbgL4ZwDecLyPFjghhGSExawIIaRHGfhiVoQQ0s9QwEkjYSVJQtKhgJNGwkqShKRDASeN\nhJUkCUmHm5ikkbCSJBl0GIVCCCE9CqNQCCGkj6GAE0JIj0IBJ4SQHoUCTgghPQoFnBBCehQKOCGE\n9CgUcEII6VEo4IQQ0qNQwAkhpEehgJNgWCGQkGZBASfBsEIgIc2CAk6CYYVAQpoFi1mRYFghkJDq\nYDVCQgjpUViNkBBC+hgKOCGE9ChlCPi/APAegOkSxiKEEBJIUQG/C8A5AK+WMJfK2NjYqHsKTpo4\nL84pDM4pnCbOq4lzCqGogP87AI+WMZEqaeqX1cR5cU5hcE7hNHFeTZxTCEUEfAnAXwD4SklzIYQQ\nkoGRlN9fA3DA8fqnAHwCwN8Vr1URkkgIIWSLvKJ7H4AvAfjB1s9zAP4vgL8F4A313lcA3J3zOIQQ\nMqjcBPD+Kg70DTAKhRBCKqWsOHCmWhJCCCGEENIkLsFErFzfejxQ62ziNCkR6dMAXgTwAswew131\nTgcA8BkAL8PM678BmKh3Orc5D+AlAO8C+EDNc3kAwNcAfB3Ar9Q8FwD4LQDfBPDVuiciuAvAOsx3\n9qcAfrHe6QAA7gDwPMzf258B+LV6pxNjGEYrn657IgBwEcAv1z0JB3cB+AM0x4e/Wzz/KID/WNdE\nBOcQudp+fevRBH4CwI/DiEKdAj4Ms1F/GMAojBjcW+N8AODDAE6gWQJ+AMDxree7APw56r9OALBV\nJBkjAP4YwIdqnIvklwH8FwD/PelNVdZCaWKYYdMSkb4rnu8C8GZdExFcg7lDAYy1MlfjXCRfA3Cj\n7knARF69AmATwF8D+F2YHIk6+TKAVs1z0NyCWdwA4Hswd3Wz9U3nNjaSbjvMYvztGudimQOwCGPA\nNaYa4UdhbsP/E4AmVJNuaiLSrwL4PwD+MZpj7Vp+HsAX655Ew/gbAF4TP//F1mvEz2GYO4Tna54H\nYDTwBRiX0zqMK6VuPgvgEUSGk5e0RJ4sJCX9fA7Av976+dMA/i2Af1LisfPMqa5EJN+cPgnj7/rU\n1uPjMF/kww2YE7bm9FcArlQwH0vIvOqGEVjZ2AXgSQC/BGOJ1817MK6dCQB/CGABwEaN8/lpmFya\n61tzaRyHUb9v7j6YFfcbW4+/hrkF3l/jnDQHYTZ7msBDAJ6D2fRpGnX7wD8Is49i+QSasZF5GPX/\nnWlGYUTyY3VPxMO/AvAva57Dv4G5o/sGgP8H4PsAPl/rjAC8Tzz/56jWiguhKZuY94jnHwXw23VN\nRPAATOTATN0T8bAO4GSNxx+ByZg7DONHbcImJtA8AR+CEaLP1j0RwQwid+4YgP8B4O/UN50OzqAh\nd5qfh/E1vwjgKQB31judDv43miHgT8L80b0A4L+iGXcEX4cpF2xDQH+z3unc5mdhLJUfwmyQXa1x\nLg/CRFW8AmOB183vAHgdwI9grlEVbrg0PgTjrngBzQknvh/A/4KZ01dg/M5N4gxSolAIIYQQQggh\nhBBCCCGEEEIIIYQQQgghhBBCCCGEEEIIIRXw/wF/uGEi7dnKtAAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10fab8810>"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can *sphere* or *whiten* then data through PCA:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.decomposition import PCA\n",
      "pca = PCA(whiten=True)\n",
      "Xwhiten = pca.fit_transform(X)\n",
      "plt.plot(Xwhiten[:,0], Xwhiten[:,1], '.')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 50,
       "text": [
        "[<matplotlib.lines.Line2D at 0x110083050>]"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnX+MXNd137/U/qC4WopLcklstyuRiuwwcmlXtIhYhZxy\ngZCGtHBAOikBw2i3UdAlUARO0h9y4hgFJQRpAgRoCxSIAbWN3aZlCkNNjAoVE7AG1xWUxIVRUVEs\n27QIL+tUlR0no9quAzWwX/+4ezVn7pz74/1+b+b7AQa7M/PmvfPezPvec88991yAEEIIIYQQQggh\nhBBCCCGEEEIIIYQQQgghpBB3AvgcgBsAXgHwK+2aQwghJA8Lu39nAfwhgPe2aAshhEwNd1Swj+/u\n/p0HMAPgzyvYJyGEkAhVCPgdMCGUrwO4DhNKIYQQ0iMOwIRQ1lu2gxBCpoLZCvf1fwD8FwCnAWzb\nF++///7s1q1bFR6GEEKmglsA3hbaoGwIZRnA0u7/+wCcA/DiiAW3biHLss4/Ll++3LoNtJN29tVG\n2ln9A8D9MQEu64H/FQD/FqYhuAPAbwL4TMl9EkIISaCsgL8M4N1VGEIIISQfVWShTATr6+ttm5AE\n7ayWPtjZBxsB2tkGexo4RrYbzyGEEJLInj17gIhG0wMnhJCeQgEnhJCeQgGfci5dAtbXgY0N4I03\n+rNvQggFvDG6KmY3bwKf/Sxw9aqxsS/7JoRQwBujDTFLaTQWdmtJnj4NPP10tcevc9+EEAp4Y7Qh\nZimNxpUrwMWLwLVrwNKSvk1R6tx31XS1h0RICKYRNsQbbxiRePrp5sRsY8OI9+nT/RDRNllfN40d\nYBqdT32qVXMISUojpIBPMG00Gn2FjR3pGhRwQhJhY0e6BgWc5ObSJRM7X1gwMWyKGSHtwJmYJDdM\n/asGDoqSJqCAkxGY+lcNbAhJE1DAyQh9Sv3rMmwISRMwBk6mkrpj/RwUJWXhICYhHpj3TboOBzEJ\n8cAQB5kE6IGTqYQhDtJ1GEIhhJCewhAKqRTmNk8m/F77y9QIOH+k5Wkzt5nfX30wZ72/TI2A80da\nHjnwt29fvYLqCnZb3980NBwc0J1u7gFwHcAXAPwxgJ9x3s+6wGOPZRmQZadPZ9lg0LY1/WQwyLKL\nF83fM2fM9QTMa1Xj7r+t76/u8+wC8nsl3QFAdPCwCg/8LwH8AwB/DcDDAH4awAMV7LdSOMMwjZDH\nubRk8qWXlvJ7bXk9Wdfb/9a3gJUV4Jlnmv3+psE7ld8rIZ8G8KPiedsNGclBqseZ12vL68k26e2n\n2LG5aex47DF6qqQZ0JAHLjkO4BSAz1W8X9IQqR5nXq8trydbxtuvEmvH7dscQyHdY7bCfS0CeAbA\nzwL4jnzjySeffOv/9fV1rK+vV3jYflC09kbsc1XU9JD7+PjHgSee0Ce4lDnWlSv5J87Y483NARcu\nAJ/4RHvd/GkIpZB22d7exvb2divHngPwewB+Tnmv7Z5IJygaBoh9rorwwsrKcB/nzxe3pWrc421t\ntRfG4EAfaRokhFCq8MD3APg3AF4B8C8q2F9u+rCKTFEPLva5KjzDN98c/v8Hf2AGGbVr2LQXao+3\nuAgMBsBrrwEvvGBeu3Sp2QJUNpRCqqEP9+y08F4A3wdwA8CLu49Hxfu1t1R9SPUq6sHFPleFZ3j2\n7PD6ha5h017oYJBly8tDu2xPISWVsG5vvc3ewCTQh3u2bZDggTdB7SfKHO9yDAb5xLFJ5He7s5Pe\ngMQEoqwAU4DKwXs2DqZFwCcxPtm0h9fVa+jalXpdYgJRVoApQOXo6u+tS2BaBHwSadPD63J4oKo8\n9bICTAEidYMW8sBJIrGZiW2mrTVVd6RInZHQdZH7A8J56mVn5nL2IpkW2m7IOknMk2zTw9O8U+uV\nr61l2SOPVOOdF+llhK6L3N/ycjd7EISkgmkNoXQ5BGDpcgxVE0kpjlWFdqq+BnZ/i4scYCT9B9Ma\nQulD6VjbhX/HO8wsw7zlSn3hh5SwRGwbLTxgQxd3323+Li+bvGy7jyLHrbrAmN3fww+b5zZ/fFLL\nwBLSBI23XDHPrkseetUzNLXX3fMtkmJnvXKbyvfII6P7SDmPouea9/saDLLsyBF64aTfYFpDKLH4\ncZdyeIuGEXyf017PW1s75fq4+3Cfa6Kbcq7a51K/r60tk89+8GCWHT4ctyeFLjX2ZLrAtAp4jC7F\nn6ueoam97p5vnhQ7XxlVdx/uc010U85V+1zq9yVrugBmwDVvI5Bijw+KPakSUMB1pi2HN3a+rvBU\nUYu7yp5F6vd18ODQ1qWleM8kjz2Li6bkQN7Gh5CigAKen755UXns9W0bEp6iwpciuqFYe5Frb2u6\nLC2ZWH1eezS7dnbS4+ld6tmR/gMKeH764EVJgXEHE0P4zi0kPHX2VqQ9991XvuEM2ZqnoSuyHufW\nlvkuVlbGGw9CigAKuB/fDd0HL0oKTJ4iVL5zazqkZK+9rTR4+nS+hqgIeRrmvGMGefdPSAqggPvx\n3XB9iI8XrdDXlXOT135+3oQ+bPijrobTXrPl5fhM0iLXqUjD37dwHWkWUMD99MHT9tGUEJcRGN9n\nt7aGg4133TUU8gsXTBjFJ67u/orkhrv56ysrxa9haOA3FXrtJAQo4H664o12GZ/ApIhnykSjo0dH\nvWKZReIKmru/stkxZYWzCvHtsxNB6gcUcJJKnok3RSb6aK9rszp9gubmpluxP3Uqf3ZMFYtXVCG+\ndCJICFDASZYV95h9ApMiXkUmGp06ZRZVtu+56Xw2xCI99dAizL5rUUW2SJXiy1g40QAFnGRZWrpe\nHo+yas/Rt79Q2KSo95t3pfsmxJWxcKKBBAGvYlV60nFsJcHlZeDP/gz46lfN80uXTBXAmzeBuTng\n/Hngk5+MVwb8yEeAb3wD+NCHqllR3Lfiu7t4w4c+ZJ4/+CBw/DjwiU/kP7a7zwsXTOVKQF/p/tln\ngddfN/8//jjwO7+T73hFbCKkS7TdkPWaKmZaahkYBw+OxoPzhCNSKh66nDiRZQcOmAHLUOjCDZuE\n6q0Uwd3H2po5jwMHdLvKhGuK2kRIljGE0knydsnzdPlDXXGZvifFSgrUhQtpNmuVBw8cCIcB5Ptr\na+P7t1UE5+aG2+3da15bXR1PL6xqhSCbCeMTaJuf/uCDFFjSLKCAd4+88U5XLEPer5zZ6OZeSwE9\nenT4/uqqeW3/ft0D1cTZ9RilTdazd7G2LSyMH0db7cf3sOecskJQSmMZa8Dq9I45eElCoCEB/w0A\nXwfwMgVcR96o1qNLqW6XZeMCEqv3LUuoWkJiF5vCniLO1qaDB7Psgx/URWlnx9imvS9zs+1yaPv3\nD1+z/8tztp+5+27/gGZKY2m/j7zpiJKiQszBSxKiKQH/EQCnKOB+5I16771ZNjtb/MYNpeH5FiG2\nHrQ9riaEvowOKc6+2LW0KSZKWsx9MDDe7/nzw7j3zo55fuGCXi7AHjNUSqBMumMemi65S6aDpgQc\nAI5Pk4Cnelyxok2uR6vtN+VYmghpXvfGhl8IqxK4mCjNz4/aUyd5bC8Tzqiz5C6ZXijgNZHqccnt\nFhaMeFsx18INcvvl5fFysSm1O9xGw3rfTXl5MVFaWhqNOTcZBy46AByDQkzqIEXAG8kDf/LJJ9/6\nf319Hevr600ctjZS83bldnv3Ai+8YJ6vrQEvvzyeg2y3X1wEvvlN4OpVYGVluL/XXwdOnAC+/GV/\n/rPMW15dBX7/94EnnjB2Li2Z49y8Cdy6BRw7ZlaZL5vLre1Tvvfss8CbbwL79hmJBICDB4E//VNz\n/oPBcFstH7wM1raFBeBb3xp+Bw88ALzvfcDt2+a9uTnzemouttzvlSvV202mj+3tbWxvb7dy7OOY\nIg/c9bhi+dda6l2W+etOy9KqOzujqXWxbAuZMaKlxYUGNPN6w3Z7mcmRki0CDBcdto+6egha7XTZ\ny5G9gTxeNAcgSd2AIZRmSLmZrTjLRYJ9A3Cbm2YZL5ulYgXdCp+b96xV+NPylmUu+B13DLezNoWq\nAcZWi/eFa2SGiW2IDhwYfvbBB4141hV+cItnyUJWWg3y1HEIDkCSumlKwH8LwGsA3gTwNQCPT5uA\nh25m9+Z3PULt5ncbhMHAZK8cPjwaQ77vPrMPm11y6lQ4K0Pzhi9cCNcY0bxsa7edxbh/vxmQ9GWL\n2AyT97xntHdQddw4ZY1N+Tw2+Kv1ImKFvgipiqYEPEYrJ9/k4FjoZvatr+gLX6ytDUVaetGayLpl\nWGNTvX250/Z1zRv2hUDcqfkphaHckrBVfz+hntCJE6YHMDtrtvMdU5tlahuvumZjckIP0cA0C3hX\nYpSuaD3yyDCVzjcZxz6OHRtOMbcxYymysjE4dCh9qbA8NUbsMQ4dGvX0tbh+7JrnyReXpAqcJr72\nc7bRSg11ad+LNlNTlgFImZgVC0Uxnk4smGYBD93MTXo5PtFyZ0xq3rHrYbufkeGJuhYFtvbL/csJ\nONos0ZRZpilhJ1vnJBSbl5955JEs27fPhGrcFEyZf37yZHV57W7DG7v2mlgznk40MM0CHvKkigpc\n2UYgdKO63rFcdcbXfdem6NchAjKMEJpyPhiYwdeU67y5abJANKH3hW1CjbH2GdnTeeklU/flnnvy\nFb+KxbplLyglxKL9BkLHYHhlesE0C7hLFV5O3sUAXPIMfMljra7Gvb9jx0z1vqWltK58iNDAa2qc\nXa7+rsW7tUlL9j07ODozM2w05Eo92gIVNiXQDZXYVe+t8MeqJuZF9oJSwid5VwNieGV6AQV8SBVZ\nA3ljvllWfDX1lAZHbuOGW8rc7L6B11jjJwVKZpy4XrlvZXor5PKzWnEu33mvrY2mCtrCWFbopXj7\nCnPVSREx7kookD2B5gEFvFp8Md+QsIWWBQvdxO6xYilysitfprKee1524DXFa9QmzWj51nK7PXvG\nQx82l913XbXz1kIS8rhS6Ofmyq2HWZQivcA6QoFFYE+geUABr5cUr969aYuGcmKr4MjqfTJkUSRd\nr2i2iA193H23iTnbRYjPnh1NT3RTKd3HwYOjg5FaI2bP6+xZf/hCnof09J9/fnzbsh5m0aJjeWlr\nwJMDrc2Dvgl4EzdR3bgi43qvRW5iXy6yT1zd+LK2TZFQTqgx2NoajT+7vQ0bHnFTKd3HwsL4a9Zu\nORVeTsVPGY+QtmkrApWNjZfxUPP8bos2AmXvjSoaH5IP9E3Ay3bTutDNkzbEMjJSbypfLrLPK5ID\nidqkIHef7iCiJNUbl+/NzpoGS6YVatdk717z1w5WyvCJTKe0DYeso+6GWWI9FCv4CwvDKpBammLR\n2HgZD7Xs7zbld9SFe4PkA30T8LLdtC5086QNNgY7O6tnh6TeVL7z8nlFg4EJX0iv0wq/W25Wimus\nkQmlKmozTLVYtFuo6777RhsZuaCD1nC427lhGZn9IuPeR46YbJ6dHX+aom/RiirDI3XUVUn5HXXh\n3iD5QN8E3ApP0UVqu9DNkzYMBqMhDPcGS72p7D7f9jbT1b/jDlN/ZO/e8TixRd7U0quUYYjV1Xj+\nuFsoKxRvlgOXNmNiZcUI9NGjRhxDg3JuI2PPS071P3Zs9Pdht7Vxdt/kHXnt7f5s2MTObnVDRFod\nmDLeqy9UU/Z3m/I76sK9QfKBvgl4lhXr6p04YW6M5eV2sgt8yNg1MJ4dkqfB2toaDTW4D1vYyk7p\ntsIsvUp3H1YMV1ay7Ad+YHgN5UITshKi+524QhsSZ9/3GfOc7aQmW53RfS9Wa8aGabQsFdeLd8NK\nbgNw993lfl++RjXvdHwXivNkgj4KeJGunvRq3AGqNsk7GSfUYGndflsSVssDP3x4PPVP7uPAgfHP\naJ7rnXcOU/3kyvWuN3nnneMNUWyWoi9vfN++UcH0pSdqWT1uT8AV6VA4RIvZuw+tHkpsv+7+3VBN\n3un4ZDpAHwW8iDdhwxQLC93ywPNMxpEeqLat3W5uLsuee840VC+9pOeBu3FtN+5tBUQTrJkZf4xY\nikvKNpubpiGx4ZMsG61zIheqkIOSUszdiUQphbhCv6HQbFq7b9vr2L9/tKEE/DNRUxpin11V5vDX\nTRcyvaYF9FHAi7CzM5yFZ+nCDy2lMdIGHGOV/Hz7sXngq6tDL9v1YO3anJpgzc2ZRkF6sdIu60XL\n0JDMDLENgA0DaKLmE3654LLmVbvnXiTsoKVjaja6IRa7TajWSdFBQtsLOXrUXIOuiyKzWZoD0yLg\nGn36oWmxUbciny/3Wmuk3DixFBfr6UpvMuTF7uyYWPn8vGkQ3Dj0xsb4km9235qouVUXgSx717v0\nAcnU1MrU79gdMM1TpCulIS4Sh+7T7zTLmM3SJJhmAW/6h1bG49dio5qnGsq9DmW3SHGRg6qpq8K7\ntrhxaOvBy1CDHCCVPSPZMNgeg5Yhk1o0y/WMU2LcWu547Hh1YW3ypZp2DQ6YNgemWcCb/qHl8aRi\n2RtZNp7qptXYzpsfnmVDsbUep1bZzxU/KZaHDpmMkLk5s6203+5rcXF06bfQRCF5TWSGjG+wUJ6j\nVgUw9D1YO226oM1QKZtdUoZYqimZXjDNAt40mphqtTu0NDkNmzp35sz4tHGLK0hFpmL7Kvu5x9EW\njtC2kzMtXa88NctmaUmv5xIKJ1nyFhjrgmjGsnXIdIJJFvAuDFJKNK/XN61eljv1dZlDqXOhbVPL\n2mp2y2JUPo9Uy5jQZms++OBowxObou6GkXzXLnae7vegnbdbb7zteK6vNyHp2u+d1A8mWcD7MPij\nTau3qXC+IlPaZ7XV3jXRdBsEuU2K15+yjcx2sZ6xG1d3s0lmZkwjFPKeQz0De37uYszuOaY2bnIg\n15ef3zX68Hsn1YJJFvA+jIZLUXLDHVbAU2pea8gb2p14YmPZUlilF+8LuaR44D4bbLjk8OHRyotu\nfDc2YCgzUaxHalMt3/Oe8OQkTdjsOdmUyiwbb3BSadMLdr+borbQk+8PmGQBr3qQsqkfthQcbbWZ\nVGQDJlPy5ubG49SuF+8buHRX0UktNXvq1Gg++Pz8MHatlY0NiaZm29raeJ68VpRLC0e5Ba3W1oYD\nw+98p79XE/PmfQO+KRT5rcVKCKRCT74/oCEBfxTAlwB8BcDPNyXgqRQp2VpnPecyPQdt1qD0cmdm\nsuzHf3zoYb7znXpc1TdwKdP4NG/ZzU2X+dqap22zPOTDTsf3TcTx2ebG0n0TglZW9AwabXr84cOj\n31nsN5Ay4JtCkd+a+7sp+jvqQ8+VGJoQ8BkArwI4DmAOwA0AD+QV8Dq937IlW6s8RpaFV2PPexx7\n3c6c0Uul+jxd2XtxY+3ugOH8fFpu+s7OqLd98OBorRT3M76JOJpt1vuWefJu0SptX3Z/cj1MOz3e\nrVEeGkuIXTftewzNFC3yW3N7nEV7oMzj7g9NCPjfAPC74vkv7D5yCXid3bq8JVvdrIqUH7oWZ/UR\nO9eiE1FsrnWsPKzkxAkjjvPzZgq9PIZ82AJhrqD6FpKwQisrGdqCWDZNLlb/Qxa6kjVfLIPBaINh\nQzja4r/uzFFZPkAK+fnz8UU45PFTxyi0RqUqEWVMe3JpQsD/FoB/JZ7/bQD/Mq+A19mtK3Kz5G1Q\n8nSnfeeq1Z52wwG+TA2Zay2zQGJoVRxtlokVR1kgzB5fy4rJsmHvYnXVXJPVVXM+Mj5uZ3/G6n+k\nfAc7O+YYGxvh0rBaGqY9F7fBC/0WY2KpZQbJRkujrAAzpj25NCHgP5Ei4JcvX37rcf369TFDm+jW\n1RmnjomyPKbvXH0xXRsq0brjbuw573JgoSqOWoGwGFoMXT60AlKykUpd/cc3QcoXpvA1ONLLT1mz\n1DeI6UunTMntLrsWJ2Pak8P169dHtLIJAX8YoyGUj2J8ILPt65JlWT5PJW+DkiLKoWPKokrveld4\nvUd3X3IR4Oefz9dQ5RXp2L6lgGkPG5N3wzRWxN34vW+xi9Dixhb3O9Fsz+u9+gYx3XGDWGlgi1bE\nLOU6SxjTnlyaEPBZALdgBjHnUXAQswma9FTcFLfYMd2iSr5Qidsd39oaDZ+srRXrUhfJ1JErANmQ\nyaFD5r2TJ0eF1RWowWDcW19eHr1em5t+71Q2FLaxC81qdW23+0r5Tfgyf7SJRnZpttSQmlbEzGdr\n3TCW3j2aEHAAeAzAl2GyUT6qvN/2dciyrFlPJTXX24ZGbIzYFysdDOKFmwCTx21FMNULdPfjCoYv\nrOFbzceer2x0lpbGvfzNzWEZ2rvuGv+8651ubg4bDLkI8gc/OBpj9wmeJtYpvwnZ0MisHvlZX4Ob\nZ+A8Zmsd5J2pS5qlKQGP0fZ1aJxYYSutC2/DAfJ97TOaoM7NZdlDDw33Yxc7Tg3f2AHL/ftH18PU\nwhquBxryst1ysRK5X60Bs5k9s7MmC8W9Vj6h9127PA24/Kysqphacrass9CUs5Gn3g5pHlDA20G7\nAUNdeCli0tPTBs1k6py2io17Q4bqo2jxdTkourIybCSkR7+5af6/886h3a6XHeuSaxNsZAPmW5TC\nFfo8YQjf4Kdb7VBeY3s9JrFKYMpAL2kPUMC7g68Lb71UzdOLzUqUHqGMKy8vjw+sWRGzCwbbRsPG\nra2ASjvsMS5eHD2+u4yafc02FCnZFW4an7uOpz1323AsLJiZpnbZN3c/KWEI3+CnNmNUayTrpuk4\nNAdAuw0mQcAnZXAldrPIgTAtRmsFyc4k1DxCu70r9jK0IBdNsI99+4yguTMntaXQpNDayTluQ6HF\nr33f4WBgehe24bCTenznYsMnMbQZr9rg5+nT442WvcbuBCPfb9GOZchl54r8VpnTTSSYBAGflh91\nTOCt0GkV+VxkuMGdHepmhwCjXrhtHNzYtesxHzw4GneXgueGNXxLpWmTl9w4sxZrTxFJ7Xdjbb/r\nLvO+HRS2r588ac7bF07w/RZDJQZSyJu1RKYDTIKAd2miQtW9gdggpfuaFLpQtUBf1kqWGXGSixDP\nzAxDCPv3h1dGd6fey4HGc+eG4RN3coxvHU43b1z7jgcDvVCWJpKxiUDuvmz+eWoowfdbdBuZ0ALJ\nmr3yOpSpUJlyrL73ZKcJTIKAdylOl5pul5JT7OZCW89TO4br4WkDnj40u6zgaOEUWSTLPRcp/Db3\nO2Sr6/kePmw+4wr3oUOmEfEt6mvtlSEeTSTl9TxyJFyJMeV83euoLdKcZcMGc2PDn3Wj4X6vdTop\n09KTnSQwCQLeJu6NHeoNpNwg7jZuTNZ3DPvaXXeZ7WRt7NjiCNpgohsOcYtT+c5FNhwbG6O1WOxC\nDq79Vvj27RvGlqUIx9bZzDITz96719i5vOzvJbgNg9bIyiyT2Pn6vruUWukp2Gtlr0OdTkqXerIk\nDVDAy+He2KHeQEoRJDfGKQsehY7hdv3tAFwstc2XIy33K4tTxVYLsvuzCyEMBqMDmXZ5Ml9RKftY\nWsqye+81Yrq2NjqIqJ1TqJSAbGSlB37y5PhAo1u8y20gQ6VhreinLtKcQpO9yy71ZEkaoICXI4/X\nErpBfDMzZRgldowiObu+HGkfmp2ykJYc+LO4XrnPBiue8/PGg9XCN771KX054K7NcoDWDS2FGrOY\nuGmNkFa2luJIqgQU8HL4buxQsX6NUEMgjxESg5AtvkFPua5kCpqdIe9Xvu96vK7dttFxwyVuHW+N\nzU0jznaGqRbPX1wcCrg2A9b2KlIbM+262PM8dmw4qckdB6Cgk6oABXxIlTdWTNRcUruvRQaatNma\nMhacp5u/uWm8Y22JMxurTW1YfLizT59/3l91UDtH91zPnh31vN0sjtS6NFnm/424WT2hKejud0hB\nJ0UBBXxIFaPwrjeXEod2Pxu6kYsMNMkZi3Jw0+eJajW0T5wwIQ6tKNRgEK5pkpfBYLwErK9glHuO\ncrDVjgPIMQHtuuW5prHp91rM3A1nucfL87uj2BMJKOBDqhiFlzfj6mq+8ETKjVxkoMl+RnblDxwY\nF1w3i0La4s7AnJsbD1VUiTtNXjYcWlaNG4YZDEbrlZw7N3rdbIO0vKwvxxazyxdCko2ar9fgfodl\nGxAyvWAaBTzUDS47Cl+mEZCfDU0tL2ubL8ZrF36YmRmKvT0P26OQGRZFhCTVg9QaHV9vRhtvcOug\n2xRMi7ZUXArab8T3nfvEtkwVRKb6EQmmUcDr9GJSb8bYUmplbCzaQMnwyurq6LZ2ZR5rl2/R4hhu\nidjl5bQp/6E8aG28QYuJy9BLaKm4vPiuq69BLlNXm6l+RIJpFPAueDExgS5jo9y3XYnetw8p9jbu\nPDMzGh7RVpzR0hTzxPDdVYJ8pAiWna4PmOXmBgO9PooMvRRZzzNv/NnXILdVV5vx88kD0yjgdXsx\nVQxGapXyUo4l186Uq9ikzB7c2DCi5i70kDpwpw002u1sHe3V1dHSrHv2jJZ+zXsds2zUo7UiLcM+\ntlEq62nHBlJDdqfm6NcpsoyfTx6YRgGvmyoGI1NvNnc7+TxlLUjrvd5991DgZN702bPjRZ980++1\nioFaKCM1hq5dAzn46Nq7vGzO2a7B6ZaA1Y7jNjD2ryagoYqIMbvrTBNNpQs9T1ItoIDnJ3Ulmbw3\nSqxSnkZIbHd2xrNJXKT3asMt7ufOnx/NqJCfmZ8fvm7Fx7fsmSbeoRRL7Tpqg4++AU/58F3HUAPj\nXi+tHnuq3SGKfO9FYPx88gAFPD8xLyl2o/gaALlfu8pN7GYbDMbFNk+KmhaTlqvdaLnKNpRw8OBo\nuEWuh2mRompDJwcOmLS+UIqlr7JfaPBRxr3lohah/HR3Gr/9G5sRGyKvUBb53gnJMgp4Icp2RX0N\nQNH9pk7Dl8ip9HLSjF0dxxVPGaaQC0akNhB511T0XaPQ4ONgYMT63nvTFrWQ16fs4GwZuhra4KBn\n9wEFPD9lu6K+GzYmtqHFGVJqpch9yXiu9ahnZsz/Mp3QjeG6KXChCStlrpV2jVIFJSWO7Ma+Q/us\ne/Av5Rq1IaYc9Ow+mCYBT13dpm7KdLFjN1JsW5lJYWPnZ87oi/Zq5WVtTFsuwyaPOTeXvt6jHZDU\nClBp10jBu/jQAAAPjklEQVQeZ35+ePw8Ndm1fcmGrKoxjap/V23UT+lqz4AMQc0CfhHAFwB8D8C7\n2xZwTdy66mUUHdjSFkuQN7r0vOVUc3fx4T17zHqWrkBok1C0nOuUa+lOz3c/5xNm+7CDmO53mDIG\nYa+DWxtGs9tN6UwRz6p/V2XqpxSFg57dp24B/yEAPwjgehcEXPMoyngZdXr0eQa23Ik2voyRixeH\njcHi4vhCxXZw0H1N5jxraYeDwahnn1q8y52eH5uKvrk5bGj27RseX7Mp9dpubMQn1oRSNX3iWbX3\n6oopvWOSZc2FUDoh4JpHUcbLqNOjL1PgKDTrz81asaETO1XdCrxcTMHmPG9t6fHxLBsOIMaKd8nG\nxi567Csm5V4D2UicOzfcLjQ1PTapxl6T0G/A3T7lu6nbey2zfw5OTg5TJeBVU7VHL8lzg4YERsuq\nCL1vj6vldLsxaG3wT5toI8nTwLnXQIZ/Dh/WY95uEbAyk2p8dsQGbpsgVYS17boaNiT5qULArwF4\nWXn8WB4Bv3z58luP69evt31dkijj0VfpBWkCE7IhxUZtG1nN0J1ub5Fx7YWFuPdrSbketnfglgiQ\ntrriVFeoIU+mS5vT4rXtGH7pL9evXx/RyioEPIWJ9MDL0EcvSAqlTwTkRBtN5H2NR57yA9rUfiuU\nq6uj8fAqQxl5B5a7MC1e246Dk5NDkwL+EAV8SF+9ICtiq6smjOGmDMqJNnnOMc+2rgBJoZTpkKGC\nU/JcUj1keRxt1mmZc8pLqghTrCebugX8AwC+BuAvALwO4CoF3CBvrBQhKdsdT53gk2eSjOZdygky\nsdmQWpnaPOfmLl93+rTJQ5cZJiF8udW+yT15BZniSeqmbgFPpe3rUCsxcUzpapftjssMDllJT5uZ\naYtaaasCWRHzLegQE3jfMcue0+qqsUNWIIx54KHcal+mTZ5Gl5C6AQW8etybOya+KZ5dKNMiBV8G\nhytaMn1Qq2RoRczNN3cFXhaFCmWG+M5by2YJTUqyjZKNTZ86lR5eePvbzbGs956y2lAfxzDI5AEK\nePXEMiGKrIkYyrRIQU7g0WyTr1tbYwN1oRQ9GRLxXY/QMmkym2V+PjwpSaY6FglbyGPZBSbajG+T\nbtLFXhco4NUTmyhSRIDL1oz2ZXDYnGYrYidPDsuvxsSwaCZEisjauLac4m+n/ufZTwpy5Z7U76SN\n+HYXBWSa6GKvCxTw6qlK+CRV1YzWbHOzK4rsS1sZ3lJkeTg7S9MNt6ytVT+Iu7MzzF4pkp/eFF0U\nkGmii70uUMCbp4j3VuePp2x8PcvGhVYKTJlJJ4NBNYsAF12Eo0ui2UUBmSa6mFUECng7lFnhvOpj\n231vbuprXabYLSsF7t8/mjpYZtJJllVz7kXFrw3R9F3jLgoIaRdQwNtBenZHjqQLuby5U7zlPLUw\n5OtaLfDQZweD0ZV9fOl3PpvybFeEFPHTjte0aPoWjCZEAxTwdtCyP1JuVnfxBPu/zd12s1xC9btd\nr1LWOvFNvgl5pKneapmQSp10IVyS0ogSYgEFvDyhATwfvqyQGNriCW4jEConK4/tHi9vOmPKeynl\nXGPn2lT4ogsx5pRGlBALKODlCQ3gxcjbRZeDenbFdbcRiJWTTaHs1PvQbMsiIRXfNlWEWeQCz7F6\n5trnqsxQYZyb5AEU8PJIrzhlBmBZ3Js89rwIsXBCnvd9Xm2eKfUpcfuiYY+ipWG7EHIh0w0o4OUZ\nDIznZifATAKxcELq+6HZlikiHztemfriqefi2ppaVztWHIuQsoAC3j+aEAY5+UbLdol5+Sm9ACuA\nNhRUJPZeNn97a8tcw1DVRGlrnrraKYW9CCkDKOD9wk0zq0sY3DRH9zhFFnQuUgOmKFVnxJSZfBUr\njtWl2Z6kX4AC3j55bmApOLOz9WVNSAHUMmU04fOJYdHysWWEzR0A9WUJNbHoQmwgmbF0UhRQwNsn\nzw0s08xSquYVZTAYlovVsjO06fdyYQW5bZ5Yt6SosIXK+br7yutZ1+EtdyF9kfQTUMDbp8xyYjGK\nCk5sRqC0QwqkVmwqZUBTI+W6pGSHVJklVIe3vLlpwlSyd8CwCkkBFPB89C33t6jg5JkRGBPaoueX\nMjEoJTukyiyhOrzlPOEoDYr99AIKeD76Fq8sW8QpZUZgk5NPYotlWHvkakFFKyxq1HGu2jnk+d76\n9psk1QEKeD76Fq+s0vttm62t4UCoXYUnJY1QLthQpcCFPN88XrF2Dnmuf99+k6Q6QAHPRxeFbRpw\nY/JHj6aHdeRycqm1alIIeb5NesX8TU4vSBDwPQ0JeAOHmSwuXQJu3gQWFoArV4ClpfDrfWZ9Hfjs\nZ0dfW1kBvvhF/fzeeMNch6efHm775pvm//PngU9/On7M2HXc2ACuXgVOnwauXRt9P/QeIVWxZ88e\nIKLRdzRjCsnLzZtG1K5eNWITe72LXLpkxHljw4iuj4UF83d2dvja66/7z29pCfjUp8zfpaXh5wFg\nT6JLEruOV64AFy/qAm3fe8c7gAsX4udHSF2UFfBfA/BFAC8B+G0AB0pbRAAMRen06aGn6b5+44YR\nlyNHgNu3m7cxRmpjYwXx1VeNNw2Mn3eIhx4yfxcXge98J01MfdfXIhsJ33u3b/enMSVE4xyGjcCv\n7j5c2g4l9RI39qmVRZVx47U1/77aSkUrMgDnZpmkfG4w0EsCxD7T1lJuhKSAhgcxPwDg31PA60Eb\nOLMZGAsL4XTAtlLRiopkEXvbEFMOMJI6SRHwKmPgPwXguQr3RwRal//znwfW1oBXXgGOHcv32arR\n4t2hMESIIvYeOQIsLwOvvQa8//3NxKWLnh8hVZEy5HMNwIry+i8CeHb3/48BeDeAn1C2yy5fvvzW\nk/X1dayvr+ezsgK6lL1RxBaZeZHX9jKfTUVmkly8aIStKEXs1TJZytpBSJNsb29je3v7redPPfUU\n0ECm4E8CeAHAnZ732+6JZFnWrRlteW3pw3TqtuPB9vh2XKDrcek+fKekXdBADPxRAF8AsBzYpu3r\nkGVZ+wJTxpYuNT4+2o4Hp5Z3LUOVotuH75S0SxMC/hUAtwG8uPv49a4KeNsCU8aWLjU+ddJ1r7RK\n0Z2W75QUJ0XAOROzB9QRw+7SmIAlNY7elu1VzsBsYlyC9JuUmZgU8CmlykHHqkgVyLZsp+iSJuFU\n+gkjdWp6Ck2kFuYlNH1d0pbtTBskXYMeeI9oO1WvK/TZdkJSYQilI1QVs522KnhdjNMT0hQMoXSE\nMhUEZdjk4x9PCzG0RZUhHqBflRcJaYPZ+CakLGVitlbEAOCJJ7ox2OhD2nrpUnlb24p10/MnfYEe\neAOkDs5pdHGw0UfVtpa5bmWg50/6AmPgHadPA3Z9sjXEtI01kG7CQUxCCjApDRHpNxRwQgjpKcxC\nIYSQCYYCTgghPYUC3kGqzqcmhEwmFPAOwjS2ZmBDSfoOBbyD9Cn3u8+woSR9hwLeQdqawNI3ynrQ\nbChJ32EaIektZaszMt+bdJmUNELWQiG9pawHbet7E9JX6IGT3kIPmkwynIlJCCE9hTMxCSFkgqGA\nE0JIT6GAE0JITykj4L8E4CUANwB8BsA9lVhESCKcSUmmnTKDmPsBfHv3/w8D+OsA/p6yHQcxSS2U\nzQMnpMvUPYj5bfH/IoBvltgXIbnhTEoy7ZRNI/xlAH8HwHcBPAxA68jSAye1wDxwMslUkQd+DcCK\n8vovAnhWPP8FACcAPK5sm12+fPmtJ+vr61hfX48clhBCpovt7W1sb2+/9fypp54CGprIcy+A5wCc\nVN6jB04IITmpOwb+dvH/eQAvltgXIYSQnJTxwJ+BCZt8D8AtAH8fwDeU7eiBE0JITlgLhRBCegpr\noRBCyARDASeEkJ5CASeEkJ5CASeEkJ5CASeEkJ5CASeEkJ5CASeEkJ5CASeEkJ5CASeEkJ5CASeE\nkJ5CASeEkJ5CASeEkJ5CASedgYsUE5IPCjjpDDdvmkWKr141Yk4ICUMBJ52BixQTkg/WAyedgYsU\nEzKECzoQQkhP4YIOhBAywVDACSGkp1DACSGkp1DACSGkp1DACSGkp1Qh4P8IwPcBHKpgX4QQQhIp\nK+D3ADgH4HYFtrTK9vZ22yYkQTurpQ929sFGgHa2QVkB/2cAPlKFIW3Tly+VdlZLH+zsg40A7WyD\nMgJ+HsCfAPijimwhhBCSg9nI+9cArCivfwzARwG8T7zWxKxOQgghuxQV3ZMAPgPgu7vP1wD8LwA/\nDOAbzravAri/4HEIIWRauQXgbU0c6KtgFgohhDRKVXngrFZFCCGEEEJIl/glAC8BuAETO7+nXXO8\n/BqAL8LY+tsADrRrjpeLAL4A4HsA3t2yLS6PAvgSgK8A+PmWbfHxGwC+DuDltg2JcA+A6zDf9R8D\n+Jl2zfFyJ4DPwdzfrwD4lXbNiTID4EUAz7ZtSIAdmAy/FwH893ZNAfaL/z8M4F+3ZUiEcxiGlX51\n99FFfgjAD8Lc3F0S8BmYQevjAOZgbugH2jTIw48AOIXuC/gKgAd3/18E8GV083oCwO56SpgF8IcA\n3tuiLTH+IYD/AOA/t21IgKRxxaZqoXxb/L8I4JsNHTcv12DKAgDGo1hr0ZYQXwJws20jFH4YRsB3\nAPwlgP8IM1+gazwPYNC2EQm8DtMIAsB3YHqHq+2ZE8RmpM3DNOR/3qItIdYAbMA4kV1PfY7a12Qx\nq18G8D8B/F1017OV/BSA59o2omf8VQBfE8//ZPc1Up7jML2Gz7Vsh487YBqbr8P0DF9p1xwv/xzA\nExg6al0lA/BfAXwewJZvoyoF/BpMl9R9/Nju+x8DcC+AT8JcxLaI2QkYW/8fgCuNWzckxc6uwWyk\nelgE8AyAn4XxxLvI92HCPWsA/iaA9Vat0Xk/zDyVF9F97/sRmAb7MQA/DRP26wT3wgzIdJWfBPAC\nzMBM1+laDPxhAL8rnn8U3R3IPI7ux8ABM5bwewB+rm1DcvBPAPzjto1Q+KcwPcSvAvjfAP4vgH/X\nqkVpXIap+toabxf/fxjAb7ZlSIRHYUb8l9s2JJHrAB5q2wjBLMzsseMwsdCuDmIC/RDwPTAC02aP\nNYVlAEu7/+8D8N8A/Gh75iRxBt3NQlnAMPHjLhiH8n3+zevnGZib5QaA/wTgaJvGBPgKTGncF3cf\nv96uOV4+AONJ/AXMQNfVds0Z4TGYbIlXYTzwLvJbAF4D8CbMdXy8XXO8vBcmNHEDw9/ko61apPNO\nAP8Dxs4/gokxd50z6G4Wyn0w1/IGTLSiq/cRIYQQQgghhBBCCCGEEEIIIYQQQgghhBBCCCGEEEII\nIdPB/weSpuQ9p1PjeAAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x10ff33850>"
       ]
      }
     ],
     "prompt_number": 50
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Counting word frequencies\n",
      "\n",
      "When working with text data, you often have to estimate empirical probabilities. [CountVectorizer](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html) allows you to write a one-liner for this task. We will use the [20 Newsgroup](http://qwone.com/~jason/20Newsgroups/) dataset to illustate this usecase. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.datasets import fetch_20newsgroups\n",
      "twenty_train = fetch_20newsgroups()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 51
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.feature_extraction.text import CountVectorizer\n",
      "\n",
      "vectorizer = CountVectorizer()\n",
      "X_train_counts = vectorizer.fit_transform(twenty_train.data)\n",
      "vectorizer.vocabulary_.get(u'algorithm')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 52,
       "text": [
        "27366"
       ]
      }
     ],
     "prompt_number": 52
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "By default, `CountVectorizer` only extracts frequencies for single tokens. When looking at pair or triples of tokens, we then talk about n-grams models. `CountVectorizer` takes a `ngram_range` argument specifying the minimum and maximum order of the n-grams to extract. \n",
      "\n",
      "The scikit-learn documentation has a [dedicated page](http://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html) on text data. Make sure to take a look !\n",
      "\n",
      "Oh, and here's a fun exercise ! Using `CountVectorizer`, estimate the empirical probabilies of some text corpus and sample sequences of words starting from an initial word. This would give you a primitive Markov Chain approach to text generation."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Learning more\n",
      "\n",
      "* [NLTK](http://www.nltk.org/) for more advanced Natural Language Processing (NLP) tools in Python\n",
      "* [statsmodels](http://statsmodels.sourceforge.net/) for autoregressive models\n",
      "* [Pandas](http://pandas.pydata.org/) for data gymnastics on structured datasets\n",
      "* [matplotlib](http://matplotlib.org/gallery.html) to learn how to generate fancy plots\n",
      "* [networkx](https://networkx.github.io/) for representing and operating on graphs\n",
      "* [Starcluster, Statsmodels and Pandas](http://nbviewer.ipython.org/github/pierrelux/notebooks/blob/master/Starcluster%20and%20IPython%20tutorial.ipynb) another tutorial that I wrote on using Starcluster on Amazon EC2 in combination with statsmodels and Pandas\n",
      "* [Diving into Open Data with IPython Notebook & Pandas](http://nbviewer.ipython.org/github/jvns/talks/blob/master/pyconca2013/pistes-cyclables.ipynb) on using Pandas on the Bixi dataset. (Btw, Julia is a former student of Prakash).\n",
      "* [How to convert notebooks to Latex](http://ipython.org/ipython-doc/1/interactive/nbconvert.html) A great way to submit your assignments !"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}