{
 "metadata": {
  "name": "",
  "signature": "sha256:d10bef3b7dd63acdac4af9ddc623448eaf443e0e14cc07baf690bf1d1a129cbf"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Validation and Model Selection"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "- *Jake VanderPlas*\n",
      "- *ESAC statistics workshop, Oct 27-31 2014*\n",
      "- *Source available on [github](http://github.com/jakevdp/ESAC-stats-2014)*"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Outline\n",
      "\n",
      "In this section, we'll look at *model evaluation* and the tuning of hyperparameters: this will be reminiscent of the model evaluation information we heard about earlier this week.\n",
      "Just as a quick review, there are a few ways of doing model evaluation:\n",
      "\n",
      "- Information criteria such as the AIC or BIC (approximate analytical approach)\n",
      "- Computation of Bayesian Evidence & Odds Ratios (probabilistic approach)\n",
      "- Hold-outs & Cross Validation (engineering approach)\n",
      "\n",
      "In Machine learning contexts, the most common type of model validation is the third: using hold-out sets and/or cross-validation.\n",
      "\n",
      "By the end of this section, you will:\n",
      "\n",
      "- Know several metrics by which models can be evaluated\n",
      "- Know the difference between *bias* and *variance*, and why there is a tradeoff\n",
      "- Know how to use *(cross) validation curves* to evaluate & improve your model\n",
      "- Know how to use *learning curves* to further evaluate & improve your model\n",
      "\n",
      "Recall our previous discussion of the importance of splitting data into a *training set* and a *testing set*: we'll be extending that here."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%matplotlib inline\n",
      "import numpy as np\n",
      "import matplotlib.pyplot as plt"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Exploring Validation Metrics"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "How can you evaluate the performance of a model?\n",
      "\n",
      "The simplest way might be to count the number of matches and mis-matches. But this is not always sufficient.  For example, imagine you have a situation where you'd like to identify a rare class of event from within a large number of background sources (in my field, an example of this is finding variable stars from the background of non-varying stars).  This \"number of matches\" metric can break down:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Generate an un-balanced 2D dataset\n",
      "np.random.seed(0)\n",
      "X = np.vstack([np.random.normal(0, 1, (950, 2)),\n",
      "               np.random.normal(-1.8, 0.8, (50, 2))])\n",
      "y = np.hstack([np.zeros(950), np.ones(50)])\n",
      "\n",
      "plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='none',\n",
      "            cmap=plt.cm.Accent);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvWl0FOe57/urntXd6tbQmmeBBiTMYKYQ2xgbbGzwmNix\nHds4sb0z7+x5r3Xuh3uy117n7LXXOufcnNx7TvZJgsfYeIyDCeABbIyJBzCDQYBGNM9jSz13V9X9\n0OlCpW5JDRIWmPp5eS1VdQ2vhPSvt573ef6PIMsyGhoaGhpXH7qFHoCGhoaGxqWhCbiGhobGVYom\n4BoaGhpXKZqAa2hoaFylaAKuoaGhcZWiCbiGhobGVcq8CLggCHpBEE4IgrB7Pq6noaGhoTE78zUD\n/xvgLKAllWtoaGh8RcxZwAVBKAS2Ar8DhDmPSENDQ0MjKeZjBv7/AP8ESPNwLQ0NDQ2NJJmTgAuC\ncBcwIMvyCbTZt4aGhsZXijAXLxRBEP4r8DgQASyAA3hTluXtk47R4uIaGhoal4AsyzNOjOc0A5dl\n+f+SZblIluUy4GHgg8niPem4K/7///yf//OCj0EbpzbOq3WM2jjn//9kmO88cG22raGhofEVYZiv\nC8my/BHw0XxdT0NDQ0NjZrRKzL+wcePGhR5CUmjjnF+uhnFeDWMEbZwLwZwWMZO6gSDIl/seGhoa\nGl83BEFAvpyLmBoaGhoaC4cm4BoaGhpXKZqAa2hoaFylaAKuoaGhcZWiCbiGhobGVYom4BoaGhpX\nKZqAa2hoaFylaAKuoaGhcZWiCbiGhobGVYom4BoaGhpXKZqAa2hoaFylaAKuoaGhcZWiCbiGhobG\nVYom4BoaGhpXKZqAa2hoaFylaAKuoaGhcZWiCbiGhobGVcqcBFwQBIsgCJ8LgnBSEISzgiD823wN\nTENDQ0NjZubU1FiW5YAgCLfIsuwTBMEAHBYE4UZZlg/P0/g0NDQ0NKZhziEUWZZ9f/nSBOiBkble\nU0NDQ0NjduYs4IIg6ARBOAn0Ax/Ksnx27sPS0NDQ0JiN+ZiBS7IsrwAKgQ2CIGyc86g0NDQ0NGZl\nTjHwyciy7BYEYQ+wGjg4+bNf/OIXytcbN25k48aN83VbDQ0Nja8FBw8e5ODBgxd1jiDL8iXfUBAE\nFxCRZXlMEIQU4F3gX2RZPjDpGHku99DQ0NC4FhEEAVmWhZmOmesMPA94XhAEHdFwzIuTxVtDQ0ND\n4/Ixpxl4UjfQZuAa1wiyLCMIM06YNDSS5quYgWtoXPNIssTbrW9zavgUDqODBxc/SIG9YKGHpXEN\noJXSa2jMkVPDpzg5dBJJlhgLjfHW+bcWekga1wiagGtozBFv2DvjtobG5UITcI3LxnBgmFebXuXF\nhhdpcbcs9HAuGzUZNaToU5TtVdmrFnA0GtcS2iKmxmVBkiV+9eWvGAuNAaAX9Pzkup+Qaclc4JFd\nHsaCYzS5m3AYHVSlVy30cDS+BmiLmBoLhi/iU8QbQJRFBvwDX1sBTzOnsSZ7zUIPQ+MaQxNwjcuC\n1WDFZXExFBgCwKQzkWfNm9d77O/cT91wHWnmNO4tv5d0c/q8Xl9D40pHC6FoXDbGQ+N81P0RISnE\nupx1FNoL5+3ap4ZO8Yfzf1C2i+3FPFnz5LxdX0NjodFCKBoLisPk4O6yuy/LtUeCIzNua2hcC2hZ\nKBpXPO6Qm/rReoYDw8q+yrRK9IJe2V6SvmQhhqahsaBoIRSNK5oebw/P1z9PUAyiF/Q8UvEIi9MW\nA9Dt6aZ+tB6n2cn1WdejE2afj4iySN1wHWEpTG1GLSmGlFnP0dBYCLQQisZVz+f9nxMUg0BUfP/c\n92dFwAvsBRddsv5a02s0jDUA8GnfpzxU8RDjwXGyrdk4TI75HbyGxmVGE3CNKxqTzjTj9sXgCXsU\n8YZoodFv6n5DRI5g1pvZXrVd8zDRuKrQYuAaVzQb8jeQlZIFgNPkZHPR5ku+lklnwqgzqvZF5AgA\nQTHIn/v+fOkD/Qu93l5+Xfdr/vuJ/85H3R/N+XoaGjOhxcCvISRZ4vP+zxnyD1GRVkF1evVCDykp\nJFnCF/GRYkhRLVzOxsnBk7zb8S4AW4q3sCJrBfWj9exu3U1ICuE0OZU8dYBlmcv41qJvzXjNiBTB\nF/FhN9oTxtx/+eUvGQteKGB6rPIxJeRzsbS4W2gdbyXPlkdtRu0lXUPj6kWLgWuoeL/zfT7t+xSA\nY4PHeKTikaui7Fsn6LAb7Rd1jjvk5u22t5FkCYBdrbsoc5RRnV5NdXo1voiP35z5jXK8RWdhVdaq\nGT29+3x9/L7h93jCHrJTstlevV01LlmWcQfdqnMmV6MmQ9NYE8cGjxEUg7SOtyr7J4on+EbuNy7q\nWhpff7QQyjVEs7tZtd0y/vU1mPKFfYp4A8jI+CI+Zfv00GnVTDkgBXi2/llebnwZURITXvO9jvfw\nhD0ADPgH+HOvOuQiCIJqpmzRW1jkXJT0mPt8fexs2kn9aL1KvAHOjp5N+joa1w7aDPwaIjslm0H/\noGr760p2SjYFtgK6vd0AFNgKlFg6gEGX+Fe/yd3EmZEzLHMtA6BhtIE+Xx+ljlLCUlh17NRtgPsX\n3U+poxRv2MvSzKUXVd7f4+1RPXQmk2HOSPo6GtcOmoBfQ9xVehc6QafEwFdlfX1tT/U6PU9UP8Gp\n4VNANL49WbSXu5ZTN1IXN9OFCwubR/qPsLd9LwBCt8CG/A30eHsQZZEUfQprs9fG31fQszp79SWN\nOd+Wj07QJRTxZZnLLumaGl9v5ryIKQhCEfACkA3IwG9kWf7VpM+1Rcw5cm7kHE3uJrJSsliXsy6p\nghWN2ZFkifHQOCeHTnKw+yAAOSk5PFnzJGa9mR1nd9Dp6VSOX+FawU35NzEcGCbPmkeqKXXex9Q0\n1sQXA1/Q4m5RHiQAGws2srFg47zfT+PK5ataxAwDfyfL8klBEOzAMUEQ3pdl+dw8XPuap360nleb\nX1W2PWEPtxXdtoAjmpkvh75kX/s+ZGRuK7rtkmejyeCP+Dk1dAqdoGO5azkm/cXliOsEHWnmNDYW\nbIwubIZ9FNoLles4TU46uSDgTpMTs96MN+yl19d7WQS8Iq2CirQKnj33LO0T7cp+LYSikYg5C7gs\ny31A31++9giCcA7IBzQBnwemLjw2u5svi4AP+gc50HWAsBTmhtwbKHeWX/Q1PGEPu1p3KSGAPW17\nKHeUk2GZf/EJS2GePfcsA/4BINqX8vtLvn/Jbye51ty4fXeU3IEv4qPP10e5o5wVrhX8n7r/w0R4\nAoD1uevZUrzl0r+JGbi//H52te5iLDhGbUatEpPX0JjMvMbABUEoBVYCn8/nda9lpi40JrvwKMsy\nZ0bO4Iv4qE6vnrFMPCJFeLHhRcZD4wC0jbexpXgLFWkVCRfhfGEfQ4EhMi2Z2Iy2C/sj8Zkf3oiX\nDOZfwHu9vYp4A3R6OhkJjOBKcc3bPexGO9urtyvbRweOKuIN0Rj5rYW3xhUHzQdp5jSeqH5i3q+r\n8fVi3gT8L+GTN4C/kWXZM/mzX/ziF8rXGzduZOPGjfN12689a7LX4Al7aBqLxsDvLLkzqfPebnub\nE4MnADjUc4gf1v5w2ld+b8SriDdEPUf2tu/F2Glke9V2ilKLlM96vb28UP8CftGPRW9he/V28m35\nALgsLopTi+mY6AAgz5qnNHEYD43T5+vDZXFd9Iz8k95PODF0glRjKneV3kWGJQO70Y6AgEx0fUUv\n6C+7MZXVYFVty7LMf/niv5BjzeHRykc1LxWNOXHw4EEOHjx4UefMSyWmIAhG4E/APlmWfznlM20R\n8ytGkiX+9ei/KuIGcE/ZPVyfdX3C44NikH8//u8Jsx+WZizlgcUPKNuvNb/G2ZELOclL0pfwUMVD\nynZYCnN6+DSyLLM0cylmvZleby/P1T9HUAxiEAw8UvlI0vnRTWNNvNT4krKdZ83jh0t/CMDxwePs\n79yPTtBxZ8mdl71aUZZldrft5sTgCfSCXrXIuMK1gvvK77us958Pjg4c5XDPYUx6E3eV3kVJaslC\nD0ljGr6SRUwhWra2Azg7Vbw1FgadoMNqtOINe5V9M1UyesKeafOPzXqz+tpTar+mVi0adca4B8Vk\nR8GIHOFw7+GkBXxyqTtEC2h2t+5mcdpirs+6ftqHUjIExSBvt75Nl6eLInsR95TdgyAIuINuHCZH\n3KKoIAjcU3YP20q38Xrz69SP1iuf+SP+Sx7HV0Wvt5c9bXuU7Z2NO/mnlf+EXpe8PYHGlcV8hFBu\nAB4DTgmCcOIv+/6TLMvvzMO1NS6R+8vu563WtwhGgqzJWUNlWuW0x6YaU7EarKpKRYCslCxuLrhZ\nte/mgptpm2jDE/ZgN9qTSm2bWjQzEZpQzp+Nckc5BsGgzHZFWeTY4DGODR7j4YqHZ/VzifmIJyrc\nOdB1gDMjZwBwj7gx6oycHz+PO+TGZrSxvWo7OdacuPP0gp412WtoGmtClEVl+0pnall/QAwQEAPY\ndLZpztC40tHMrL6GDPgHeKH+BTxhDw6Tgyeqn5i1G3yvt5cDXQcQZZGb8m4ix5qD1WBN6AsSFIOM\nBcdIM6fFzdAT0eXp4ndnf6faZ9ab+V7198izzd7ouMvTRd1wHXUjdUopO8Dq7NXcVXpXwnNi4Y7j\ng8cxCAbuX3S/KsTSPtHOy40vK28GEE0TdIcueJlUp1fzcMXD045rwDdAj6+HfGs+2dYrv6rVG/by\nH3X/oSzElqaW8r0l31vYQc1C23gbo8FRyhxlpJnTFno4XymamdU1yoddHypCF2ssPJvLXp4tj8eq\nHkvq+ma9OeHMdDoiUiRuXyx88UT1E1gMlhnPL7QXUmgvxBvxcnr4tLJ/poycZnczxwePR+8vR/jj\n+T+yJH0JOkFHIBKIE2+I9vCcLOCTPVG+GPiCLk8XhfZCJbc925pNtjUbWZapG67DG/GyJH3JFbuY\naTPaeLrmaU4OncSkN13WHP354NO+TxU3SYvewlM1T6nsEDQ0Af9aMjWePV18+6si15YbN7sF6PX1\n8sy5Z/ir2r9KKhVva8lWBAQG/YMsdi6eUYCminNYCvNZ32ecHj6NWW+O+3xz4WYWpy3m+XPP4xf9\nGAQDNqONhtEGRoIjipCcHDpJRIqonAGfr3+etok2AD7o+oCfXvfTK1bEnWYnpY5S/tDyBw50HmBV\n9qqkM5u+SvwRP3/uuWAWFhADnBo+xabCTQs4qisPTcC/htyUfxNtE20ExSAWvYUb8m64pOtIskSn\npxODYEiqU40v4uP5c88zGBjEYXTwvSXfI82cpsyePur+iFPDp1QmUAP+Abo8XRzuPUyXpwuXxcUD\nix9ImH+eYkiZ9U0iRkVaBdkp2UqueGVaJe91vqd8Pjmu7jA5WJ29GovBws+W/YzP+z/nUM8hTg6d\n5OTQybiZ/vnx84qAHx84rog3RB8cxwaOcUvhLUmNcyF4s+VNJW308/7PFZvdKwVP2MNvz/wWT0SV\njRyXxqmhCfjXkkJ7IT+77mcMBYbISsm6aC9tiIr3S40v0eKOWs6uzV7L1tKtM57zh+Y/0O/vB6IL\nZs+de46/XfG3QFQk7y67m5vyb+JXX/4KiehbgYDA7vO7GQmNANDt7VZCK3PBrDfzVM1TtLhbsBgs\n9Hp7aRxrVD7XCTpuzL0RSZZYm7NWCePYjDZODp1UXWtyNg+gCh990vdJ3L0X+o1nJmRZjvt+Jq8r\nXAmcHj4d97ZWmVZ5VSwUf9VoAv41JdWUOievjvaJdkW8AY4MHOGm/JtmvOZIcES1PR4ejzsmzZzG\ntxd/m33t+5BkidXZqznUc0h1zGhg9JLHPRmz3kxNRg0QjaFOdvpbnLZ42vZsUwXOG/GSZ81DJ+go\nshdxc/7NRKQI58fPM3WBXifoWJsT71J4pSAIAiuzVvLFwBdANL10pgylhWBq31Onycl3K7+7QKO5\nstEEXCMhiVqXzdbObEn6ElVfyZyUxAudRfYiHqt6DJfFRb+vP07ASx2l1I/Wk2vNnbfMg3xbPtur\ntlM3UkeqMZWKtApebHiRsBhmfe56mt3NnBw6SaoxlRRdCh5RPSvt9fXy/SXfpyS1BFESeaHhBcVs\nSocOCYlUYyrfrfzuZTG5mk+2lWyj3FGOJ+yZ1WZhIVjuWk79aD1N7ibMejP3lN2z0EO6YtHSCDWm\nZVfrLqUc/7ai25KKpe/v3E/9aD2ZlkzuK78vrrz91NAp/tj6RyRZwqQzEZJCigBCVNx7vb1E5Agm\nnYknqp+4qE7x/b5++nx9FNgKpvVFEWWRX578pZJON7kkfya+Vf4tlrmW0TbexnP1z6k++1Htj8i1\nxRtiaVw6vrAPk940bfONrztaGqHGjEyEJnivM9ombKVrZZzj3b1l97IxfyN6nT7pOPrmos0zdo5/\nt/NdJYwRkkIAingDDAeGlcXFkBTis/7P+Lb920ndu3GskVeaXkGSJQyCgceqHqPUURp3nD/iV5lS\nJSPeAId7DlObWRtfoYkwL7NuSZaYCE1gNVovi0HWVGbq/5nMuW0TbUSkCGWOsssislajtmg5G5qA\nXyOIkshHPR9F24OllrI+dz2vNb+mNCxoHW/FaXbGeWM4zc75HcgsWjk1Z3xqPHQmjvQfUR4OETnC\n0YGjCQXcarCSY82h3xddcDXrzOgEHX5x5nL4gcAA46Fx5a0EouJ9R8kdKlfGS8Ef8fNC/Qv0+npJ\n0afwSOUjFKcWz+maMSJShPc636NzopNCeyG3Fd3G3va9nBo+hd1o5zuLv0OhvfCirrmrdZey2FuS\nWsL2qu1aSf4CoAn4Fcqgf5CIFCHXmnvJs6TJ7O/ar3SkbxxrxKAz0OPtUR3T6+1VCXivt5dmdzMZ\nloyLMopqcbdwsPsgAgKlqaVkWbOoSa9Br9Nze/Htimd4otDFssxlNLmbcIfcuCwupZRflETe73yf\nDk8HBbYCbi++PW6WatGrC4KmcyfUCTq2V23ncO9hQmKIFVkrODN8hpbxFlXP0ETj29e+T5XNMpcW\napP5rO8zen29APhFP39o+QNbSrawJH3JnK/9Uc9HHOk/AkRj+WPBMZrcTUC00OsPLX/g58t/nvT1\nYl2MYrRPtNPuaafccfEe8hpzQxPwK5ADnQf4uPdjIFrO/Z3F37nkRgWBSICdTTtV3V0g6p9dmlqq\ndKaPZVjE6PJ08ey5ZxHlaDXiYMFgUr4n46FxdjbtVGbSHZ6otewp5ym+W/ldlruWU5paiifswWKw\n0OPtIRgJMhoaxWVxscK1AgkJX9iHzWhTvu9DPYf4rP8zINr816AzxDVT2Fy0mX5/P4P+QfKsearx\nekIe+v39ZKdkk2pKxWa0Kee/3PiySpRjpBpT8UV8KtfBqcdF5AiiJKLXz232OfkeEE3DfLXp1Rnt\nApJlwDeg2h4NqrN8pnrgzIZBMMQ93C7mTUlj/tAE/ArDG/Yq4g3RlmrtE+2UOcou6Xof934cJ94Q\nXSxclrmMj3o+YiI0wQrXCtVi4dmRs4p4Q3TxMRkBHwmMJCydj82q08xpOM1OJTSTyKNFjz4upjy5\neQPEixJEUxR/et1PCYkhVZz6xOAJdrXuUrbvKr1LNWue2vUoRqI0SIjmtMcKYVZnr77oVm6TOdp/\nlA+7P0RASNjQ+NjAMbaWbJ1TH9RFzkU0jDUo27UZtRwdOKoI98W+QViNVu4ouYN32t9BRuYbOd+Y\nMQTT6ekkKAYpTS29ZhckLxfaT/MKYiI0we623XH7BS49hDLV5tSit7AhfwNrstcgCMK0LcGmxr4n\nb3d7upkIT1CSWhIXpkg1piYMPQgIeMKeadMCx4JjeMNecqw5Cf/Iy53lnBs9p9qejqmCuqd9j2p7\nb/ters+6XhHF7JRs+nx9yueTs2ISUZtei91kx2KwsCpr1bTHTWU4MEy/r588Wx7p5nQG/ANxY5uK\njEzDWIMSSgmJIQb8AzhNzqQXTtfmrMWoM9LpicbAr8+6nuuzr6dprAmHyXFJeeDrctaxwrUCURZn\nrJB8t+NdJXRXaCvkiSVPfCULtNcKmoBfQfzh/B9oHW9V7avNqJ2T6f7KrJWcGj5FRIogILC1ZGtS\n/RXzrHmUO8rp8HRg1pm53hX13f6k9xOlJD3dnM7TNU+rFvCGAkMJszpkZF5ufJkfL/1xnPAcGzjG\nn9r+hIxMga2AJ6qfiBPhNdlrMAgGJQae7KxRluW4Ypups9yHKh5ib9tevBGvco+Z+LT/U+Vrd9DN\nrYW3zjqOFncLLze+jCiLGHXRTkeTLQVmYtf5XSxZtYSJ0ATPnHuG0eAoBsHAdyq+k7T4rsxaycqs\nlcq20+Scc+x+NifKoBhUxBugy9tFi7vliirbv9q59PcyjXknlhURY232Wh5Y9MCcFjGL7EX8qPZH\n3Ft2L0/XPJ2UeB/oOsAz557h/Ph5IlIEb8TLW61v0ePtURXdjAZHVe6AABmWjGnfGHwRH7/88pec\nG1H3u36v8z1F9Lu93XHXjLEyayX3lt3LqqxVjIfGCYmhWb8XQRASCuwL9S+wv3M/g/5B0s3pPFr1\nKD+o/UHCrJuZCpgO9RyKq9xMxKd9nyohqbAU5vP+zymwF6hCSJmWzLiGGRBNp5RkiSP9R5T4dUSO\ncKDrwKz3XUh0gi4u9KPNvucXbQZ+BbHIuUgRLwGBpZlL5yUDxZXiSrrZryiJHO45HLdfkiXaxtsw\n6U0ExICyPzZTrhuu47O+zzAbzNxWdBuf9n2qyrVWri+L/LH1jyzJmD67YqbvOSyFebnxZVrHWzHp\nTHxn8XdYnLZ4xu/phrwbyDRn8krzK8q+tok22ibaODpwlB8t/ZFinrUuZ11c/D/NnIbL4lLFkSdz\nduQsa3Jm9umY+kZh1psVv5ZTQ6cw6Awsdy1HL+jxhX08V/+c0o3oxrwbo0I45ccSe1AGIgE+7/+c\nsBRmVfaqhEZgC4FRZ2RbyTb2tO9BkiWWu5ZrmSrzjCbgSRISQ3jCHpxm56wl5ZfKPWX34LK4cIfc\n1GbUzlse8MUgCAJ6nT7hQmR2SjZ3l97N682vE5JCVDgrWJa5jD5fH2+2vKnMoof9w5Q5yjg1fCrh\nPcJSGEmWlNnZHcV3sLttN5IsUWQv4rrM66Yd34nBE0qYKSSF+FP7n/jbtL+d9ftKtyQWtaAYpG28\njfSs6OeF9kKernmanY07GQ+Po0PHjXk3ssy1jHMj5wiLYRrcDap2anva9+CNeGdc5N1cuJleby+j\nwVFVpyOrwaqypgWwm+w8Xfs0beNtWA1W5fcg9nAZDgxj1Bm5reg2JFnihYYXlJTQk0Mn+cnSn1wx\nRTCrsldRm1FLWApf8RYDVyOagCdBx0QHLze+TEAMkJOSwxPVTyT9ByLJEp6wB6vBOusKvFFnjGth\n9lWjE3TcU3oPu1p3IcoiqcZU7EY7K7NWKjPdf77+nwmKQSX2PegfVMW9x0JjrLGugeEL17XoLcrM\n/YbcG1Sv1iuzVrLIuQhfxEdWShYhMcS5kXNYDBYqnBXKjNwX9sW9HcTiyKIs0jjaiIxMZVql8rOu\nH6nn3c6ol3exvThhfDvDkqHazrPlUe4s5+TQSSQk3ul4B0mWWJUdXbAsdZTSMNqg+p7PjZxLKOD+\niB9JlsiwZPDXy/4af8Qf1+koLIXZ27ZXWWTcWrIVi94SFyu2G+38aOmPGA4Mk2qMpkK6Q25VPr8n\n7KHb201FWkXcWBYKi8GChZmbdmhcGpqAJ8G7He8q4tPv7+ez/s+SWrjyhD28UP8CA/4B7EY7j1U9\nRq71yvfLWOZaRlV6FSExlHDWZNAZVA+jQnuh4msCUGAr4Ju532QsOMaxgWNISFh0Fm4vup3MlEzC\nUpj3Ot4jx5rDctdyIJqa5zA5CEQC/O7s7xgORNW/3FFOZVolFWkVHBs4Fpfad3P+zUiyxM7GnUo6\nYGlqKY9XP854cFwVNhkNjvLgogcJikGODhwlJIVYm7M2bpFYlES+HPpS2Q6KQXa37WY0OMrmos30\n+friFmoTGUL9uffPvN/5PgDrc9ezpXgLvoiPN1rewB/xsy5nHSuzVvJh14ecGIpWdw4FhkgxpEyb\nHWTUGZXfofFQ9A0hRZ+iVJEKCNdc67FrmfnoSv8MsA0YkGV5+nffq5jJ8VBI3CIsEYd7Div5y56w\nh/c63mN79fZ5H9/lIBajTQa9oGdtzlp6vb3kWHO4Kf8mBEFgwD+gpOONhccYDAxiM9rY2bRTOdcd\ncrMhf4Oy3exuVsQbos0Tzo+f50DnASrS1bNKu9FO01gTI4ERVS5320Qb/b5+hvzqjvYQFb0vh7+k\nz9eH1WCl0BafvyzKYsxISLX/y6Ev2Vy0mWxrtirV0KAzcE/ZPXjCHoYDw7gsLiRZUsQboouYyzKX\n8WrTq0pz4V2tuwiKQSXWHWPy958ISZZ4rfk16kfro77meTfSNNZERI6wIX+D1nbsGmI+ZuDPAv8v\n8MI8XOuK5Ob8m3mj5Q1EWcRutM+6YBVjanVdsmljVxPjoXF+c+Y3SlOADEsGKYYUxkPjcRV/kizF\nLQTWj9arBHy68vewHCYshpWZfiyvvMndpJSFT8ait1A3Uhe3v93TruR8+yI+3u18lyeXPKl8fmzg\nGHva9iTMA/dH/ATEAJmWTB6pfISPez5WZstjoTF+3/B7gmIw2jmoPL5zkD/ij+sM/07HO6zLXqfa\nFxSDRKTItCG3+tF6JQYvyRKf9n3Kf1r1nxIW+xzuPczhnsOKLesi56KE19S4OpmzgMuy/LEgCKVz\nH8qVy5KMJfzU+lPGgmPk2nKTbu20NnstZ4bP4Bf96AU9N+bdeJlHevkIikHMerPiYDfoH2Rf+z6G\nA8Oqji7HB4+z0rWSZ849o3qAWfVWUgwpdIyqY9CT0+jCYpj28XbSzelx4g/RFMOfXPcTur3dfNL7\nCd3ebuWznJQcBgODyLLM5qLNGHSGhOXx7ePqqtSweOGh6ov4lHz0RETkCL+r+x06XbQx8nh4HIPO\nwHBgmKMDR5U+m/6In1PDp1ieuZwvh6OhmEXORZQ6ShPG4U+PnGaxYzHN49G3iPaJdg71HJo2TDf1\nDVCUxYRQVHH9AAAgAElEQVRj7vJ0sb9zPxDtKfla82v888p/Tsp0KigGlTTLyrRKvpn3zVnP0fjq\n0WLgSZJhyYhb7JqNbGs2P7nuJ/T6enFZXBd9/pXAcGCYlxpeYiQ4gl7QI8uy4ok9dTYJ0ZZkn/V/\nFvf2kWPN4WD3QWU7ll1xZ/GdtI63Uj9Sz5GBIyohWuRYpHi1QLQox26047K44manK7JWsDZ7LTIy\nBp1h2txsv+hXYsY6QcdN+TcB0SrYfl//rNayQ0F1uCMiRXjr/FtxoqoTdNxffj8rs1YiyiKljlJ0\ngo5Hqx7lf5z4HwSlC02VfREfYVn9djbZVGsq1enV5FnzFPOrDfkbEmZGTYTUaZxBMUhQCmLVzT4B\n2du2V3n4tE1Es2FWZK2Y9bxLpcvTxWhwlJLUkiuuwcSVzFci4L/4xS+Urzdu3MjGjRu/itteEcy1\ntdlC817He0qrtNhawNSekXDBE/vbi77NqaH49MGpOeHFqcU8XPEwb7e+zfHB4wnvbTFYSDOlKQ8K\nv+jnf375PwmIAVU4KtWYyorMFaqZpc1oY3PhZvZ37VddUy/o+d6S7+EOucm0ZJJpyeT08GneOv8W\nkiypFgSTZaoZlMPkYEP+BmUmPhYcwx1yc33W9Zj1Zh6vfpwdZ3coDwuXxUVVWpXKs2amLBKT3sST\nNU/SMdGB1WAlz5YXd4wkS4iyiM1gwxuJPswq0yqTfnvs8fXEba9gBUExSKenk1Rjqqo36Fw4OnCU\nPW1RS4EUQwpP1zyd0CPn687Bgwc5ePDgRZ0zLx15/hJC2Z1oEfNK78gTkSJ83v857qCb2sy5la0n\ncy9vxEuqMXVO5kSXG1/Yh1/0k25O5/n65xOaYU3GpDPx4+t+rBSQuENunjn7jNKYtsxRRq41V1VW\nXZpaisPkmDZXHGBb6TaWZizl1aZXVZ3fE7G5aLMSohoLjvHW+bfomOjAoDNQnV5N63ir4t1dm1FL\n63grh3sPoxf0dEx0qIqTVmevVkJfszHV98WsM3Nzwc181v8Z3rBXtQB+X9l9yiz27MhZvhj4AofR\nwS1Ft+A0OTk5dJJuTzclqSUszVw6672nQ5ZlXml6RVlvcFlcrM9dzwrXiqQ9u/e07eHowFFl+zuL\nv0Opo5QdZ3coi6x3FN8Rl8N+Kfzqy1+p+qnelH8Tmwo3zfm6VztaR54keLv1bUVEvhj8gqeWPHVR\nLbySpd/Xz+8bfs9EeIKslCy2V23/ymfmoixypP8I7qCbmoyahIVCdcN1vHX+rehrf2opq7NW0zHR\nMW1oQUDge0u+p6r+c5qc/Hz5z6OWsToLZoOZiBRBkiW6Pd34I/4ZBdmsN3Nj3o1KF/JksmFikwR/\nxM9vzvxGmRWHpTBnRs7wjyv+UcndHwuO8VLjS9NmE0mypBLvFF0KfilezJe7lrM0YymvNL2iCHVx\narHiFTOVD7s/ZEXWCpX3S5G9SJkVr3CtYIVr7mGKQf+garF4KDBEni2PDk+H4tO+qXATRalF015j\nS/EWbEYbQ/4hKtIqqMmo4Uj/EVWGzMHug/Mi4BaDBYKTtvVazniyzEca4U7gZiBTEIRO4P+WZfnZ\nOY/sK6Jp7EIGgyRLnB8/f1kEfH/nfiWMMOgf5OPej9lasnXe7zOZkBji2OAxQmIomm/c/aHSTebI\nwBHlYSXLMi3uFkRZZG/7XkWM2ibaWOZaxg+X/pD3Ot7j/Pj5hPdxGB280vQKPd4eSlJLuLv0bkx6\nE07TBV8Rg87AnSV3AvBvx/5NdX5sFrvYuZgVrhVUpVepPDMWOxfHZa9MzjvPtGRyfdb1dEx0cG7k\nXFxIQ5IlAmJAEfAj/UemFe9ca25c+mCqOZWgP6jKTCl3lHNX6V0YdUa+v+T7nB05S6opWvSUKCsG\nojHojokOxYYVolar73e+P6+/C4nsbUVJ5OXGl5XQ00uNL/E3y/9m2qwfg84QV5g0dd1hvqxh7yq9\ni5cbX8YT9lDuKGdtztp5ue61wHxkoTwyHwNZKFwpLqWtWGz7cjA1hfBypxTKssxLjS8p4Y9jg8eI\niBdES5Ilmt3NFNgLeLPlTSXlbqoRlSRL2Aw21c9oMkadkV2tuxTROj18GqfJOWNfzOyUbNX1Yq/n\nicQkKAbp8/ep9unQ4TQ58Ua8pOhTWJ4Z7WKeyIo3Roo+eu2x4JgqlDMZvaDn6Zqn+WLgC9X+NFNa\nnB/5HcV3KA+ZQnuh4oftCXuwG+2qzJwYYSnMM+eeidt/bOAYmws3xwmvN+ylYayBFH0K1enVSfvi\npJnTuK3oNvZ37kdGJs+ax+8bf6/6nQuIAcZD49MKeAxZlonIEYw6I8syl3F66DStE63oBf2cG03E\nyLfl8w8r/oGQFEq69kAjyjUfQnlg8QPsaduDO+hmmWvZvLSwSsQNeTfQ6elElEUsegvrctbNftIc\n8IQ9qtj1eGicrJQsfP4Ls1NXioux4JgqX3pyqCQnJYelmUt57txz0z5wQlIoriFCohTAyTy4+EH2\nte9jPDTOctfyGY2tXm9+XXV9k85ERIowGIhmafgiPj7o/mDWzIVdbbt4uOJhJsIT04aDRFnk5NBJ\n1uasxR1y0+xuJsuSxTdzv0mjuzHu2ETYjXaernmaU8On0Ak6PKFocc94aJx+f3/Cc0RZJCSpm1D4\nIj5+e/a3jAWjC7grXSu5t/zeGb/HydyQdwNrstdwdOCoqqAoRpo5bdasqC5PF680vYIn7KHCWcH9\n5fcr2UWJnAbngiAImnhfAte8gDtNTr5b+d3Lfp+KtAp+et1Po/FIa95lj38nEpgtRVv4rP8z3CE3\nSzOWUptRizfsjVuIe2jxQ1iNVvJt+QCqZgeJmCqINRk1Mx7vMDl4qOKhpL6PqWGbmoyahFkwU8Me\nk0MsAA2jDUiyRK41l6yUrGnT9Do9nazOXs2qrFWkmaIiV5hayArXCuW+2SnZHOg6QIGtgJsLblYJ\nWb+vn9eaX2MsOEZNRg33ld+HXtDzWvNrKgE36ozKQ7EyrRK70a4aR/NYsyLeACeGTmAxWKYtsU+E\nSW+KS6c0680sdy3nhtwbZrV2fbv1beVNosndxO623cqbU1gKs7d97yU1g9CYP655AZ9PwlKYXm8v\nNqMtYRrUpeSSXyqTY/sQDQ8sTlscZ71qM9q4s+RO9rXvQ0bmxrwb42bEkzu46wQdedY8VRFNvi2f\n2vRaDvUcIigFOT18mqq0KlWMdLouPvWj9exu3U1YCrMkfQnd3m4EQeC2otuoTKuM65bjDyfODFmf\nu55jg8cYDgyzyLGIFVkreLPlTeVzu9GuzBofWPQAv677dcLr6AQd593n2dm0UxHYTYWbuK/8Ptbm\nrOVo/1FODJ1gwD9As7uZupE6wlKYstQy7iq7i12tu5SFvtPDpylOLWZN9hoWOxdzduSscp8NeRtI\nMaZg1BnjMk72d+5XZYDE+LTvU2wGGznWHDIsGUml2tVm1PJ5/+fKA3197vq42HZADKBDp3oDODZw\nLC5sFCtUiiFKid9CptLsbqbX20txavFlzfK6FtEEfJ4IiAGePfss/f5+BAS2lW6bl27ll8rU19HJ\n+b9hKayafa3NWcvKrJVIspTwNfbRykf5oOsD/BE/q7NXU5JawoGuA3ROdFKUWsSmwk08e+5ZpTil\nfrSeLwa+UDIUPu75WGk+kG5O55HKR2h1t2LQGdjXvk95LY8VjgC81vwaj1Y+qixIWvQWlmYujRMN\ni87ClpItrMxayTfzvokoi0pRy5B/iKMDR0kxpHB/+f3KOYmKXkyCiZAc4sTgCb4c+lLVtefE4Alu\nyr+JTHOmyuQKLviWfDn8JWnmtLjYd2wGfN6tfpNonWhV+eKEpTAHOg/QOt46bagF4MOuDxER0Qk6\nHlz0YNzDVpIlBvwDmPVm0s3pFNgLeLrmaZrdzWRYMqjNqFUdH2t5FuvWtCZnDSExFNfqzaw3c2vh\nrYy1jDESHEFA4JbCW6YdZ4zJ/UgFBB6qeEjryDOPaAJ+iYwERniz5U1Gg6PUZtTisriUPzwZmQOd\nB+ZVwJvGmhjwD1DmKFNCG9MhyRICglKtZ9abua/8Pvp9/bzc+DLukJtFjkU8VPGQMuua6XXaYXJw\nX/l9qn2xjJIYU3tvTt4+3HvBAnY0OMozZ59R5V0nIiJF2Nu+V2keHBADfDHwBXr0yiKhgEBQCnJi\n8ASVaZXYjDaVON9SeEtCkcm0ZFKdXq3y9A7JF8ItU1uujQZH+dej/zpt3HvycauyVvFh94dANIxT\nm1FLj7cnrioy9kAdDgzTPtFO42gj9WP1cdecioiojPFQzyGVgIuSyEuNLylhpy3FW1ifu548W17C\nYp9uT7eyoCsjs7d9L7WZtQgIcT+DO0vupNBeyA+X/pBubzcOoyOpBf/Jef4yMqeHT2sCPo9oAn6J\nvHX+LSWMcHTgKMsy1a3K5nOB50j/Efa271Wuu71qO6WO0mmP3926W7EntRvs/HDpD0k1pbLj7A6l\nuKZlvIXP+z9XSskTIcoivrAPm9E26/ezNmetkv9s0VtUTRmmnjudeE/uyh7zBY8bEyI5KTkU2AqU\n1MIOTwcfdn+YdFaEIAgU2gpVAj4TMvKs4g1Rz5wl6UvIs+UxGhxlsXMxw4FhdjbuVK0TZFqilrrP\nnXtOWdi+FKY+dBvHGlVrBu93vs/a7LVxxTv+iJ+6kTpGA+rFZhmZiBTBYXIoC6AQzbJZmhEN85j1\n5ovqqjM5lTTRtsbc0AT8EokJYQyHyaEYFekFfdwMdS5MXrSTZInTw6enFfBYJkUMT8RDs7uZlVkr\n44TzaP9RqtOrE9qPDvmHeLHhRdwhNy6Li+3V22fM9Phm3jfJt+UzGhylzFGmeFLvadujmo0X2grp\n8napzq1Nr6XUWUpZahl1I3UICKzJXsOxwWMJ+z5aDBZ8YXWud5935oXWRN/ffBOKRB84kxf23u98\nXyXei52LGfAPTNueLVmsBiu3F99+0ecN+gd59tyzqtBU7PdiWeYy5d94W+k2lmYuJSSGKHOUXXLO\n921FtzEeGqfH20Opo3TBG5Z83dAE/BK5LvM6/tz7ZwAMgoHajFpuLbyV4cAwVoNV1al9rqQa1Rkr\nM2Ww6AU9VoNV8b8AlAyHdTnr+FPbn5T94+Fxdjbu5OfLf67sOz18mj1tewiJIaVwZSgwxKGeQ7PO\ncEsdpZRSCkDreCvesDduMW5DwQZeb35dlZbY5e3iwYpoo4Ul6UtIM6dh0BkotBeypWgLETnCsYFj\njIXGSDGksDZnLUf6j6ium6j/5nR4w17Ojp6d8Ri9oL/omfGutl10ebtYmrlUWaybml1i0pmUsFAi\npmYEJWJ55nJ6fD281/ke20q2kWPNQZREzo6cVZ1/W9Ftqtm3P+JXiTdE34buKbsHp8kZN7OejwVH\nm9HGg4sfxKw3X9H2EVcrmoBfIrcV3UauNZfR4CiVaZVKl5TLYaa/tXQr3mYvA74BFjkXcUPeDdMe\n64/4KXeWK2lza3PWkmPN4dlzzzLkHyI7JVuVXTAaHKXF3UJxajEhMcQfz/8xoXAl0wE+xq7zu5QQ\nzlRMOhMmnUkl4EadkUH/IM/XPx9tP6e3kmHJUGbq12ddT6mjlJNDJ/FH/Lze/Hpc38zJi6/dnm66\nPF3k2/ITlosPBYZUKYZTERCYRUMB4oyvJFni6MBRjg0eY2vJVpZmLuWWglsY8A/QOdFJgb2ALcVb\n6PZ2x73BxbAb7TM+jGwGm2qx9+XGl/m7FX/HkYEjnB45rewvthezPnc9EI2z7+/cjzvkjqtS1aOn\nJr0mWs4+A56wB0/YQ5YlK2k/lYgU4ZWmV2h2N5OiT+Hhyoe1LJR5RhPwOTBT891kkWSJfl8/Zr15\n2hRDp8nJ0zVPz3gdb9jLyaGTUa+Tv4hDij6F9bnr2d22Wynq8Ua8qlizjMyLDS+SZ83j3rJ7E4q3\nWW9mXc46AmKAj3s+xhPysCJrBWWOsrhjJ0IT04p3TIhvLbxVqZrUoePu0rs51HNIyeDwiT583gtC\nM9Wt0BP24DQ5lXxuo87I7UXRcELjWKMScxYQeGDxA9iNdprHmjHpTRTZi6b1KomxzLUs2pdz9Ny0\nx6QYUthaslWVqhhDkiX+1PYnDnYd5AdLf8CTS55kPDhO60QrI8ERnqh+goPdB4lIERrGGlQ/86mh\noalMfrOCaCgvIkXiZvWxB4skS7xY/2JC61+Ae8vvnVG8A2KA44PHOdB5AFEWybfl80T1E0kV3Rwb\nPKYUYflFP7tbd/OzZT+b9TyN5NEEfAERZZFXGl9RytA3FW6adlGx19vLudFzOE1OVmatVL2OBsUg\nO87uUDm6QfSPZse5HQQi6tj31AwDgF5fL+92vKtqOOCyuNhYsJGi1CKcJicv1r+o+HOfHjnNX9X8\nVVx2g0FnSBgGsBvsLHZGc9BXZa8iz5bHkH+I4tRi0sxpCfOeZ6LL08XtRbfjMDlINaYqPicnBk8o\n95aR2XV+14yz7cnEwiZfDn1JtiV72uN0go6S1BJsBht/v/zvaRhr4MjAkbjiIE/Ew9H+o6zOXs1v\nz/5WeUDdUnAL31oU7djzRvMbqkrYWJZJspQ7yjHooiG8I/1HlIdBbFHdF/HFibdJZyLTksmW4i3K\nWspIYIQTgycw6U2szVmLWW+m29PN7xt+r3rL6PH2cHLwJOty1+GP+NnVuisa304t5e6yu1ULq1Pz\nxqdua8wdTcAXkBZ3i8r46EDXAdblrIvzxOjz9bHj7A4lX7rX16uKR3d5uuLEO8bkaj6YOcbaOtFK\nnjWP+8ruQ5RFlmYuVc20Widala8lWaJjoiNOwGMz06l5xJ6Ih9ebX+epJU9RmFpIvi1flQ55Q94N\nNLubCYgBLHoLdoM9rnnCZOpG6qgbqWNV1iruLrtb2T815jybeJt0JopTi2l2N6tmwgOBgWnPkWSJ\n+tF6GkYbeKL6CXKtuXwj5xu80/FOnOXA6eFoWGNyfvinfZ/iDrk5M3IGp8lJdVo1Y6ExKpwVtLhb\n4ry4NxVs4qOej+KaZAgIlKSWIMkShfZCnq55mpbxFlwWl5Kql8jZz2q08sOlP1S2PWEPO87uUGb3\njWONPFXzFAe6DiS01I2tjbzX8Z6SyXNq+BROk5NNRRdsYJdnLudI/xHle58p9KdxaWgCfhUQa1gb\n48zIGZWAT13kTEROSg41GTUYdcYZQwi9vl5qMmoSOtrlWnPp8V4Ql1xbbsJrZKZkkmHOiHuoyMi8\n0PACj1c/TnZKNmdGzqAX9CxyLOLt1rcJiAFMumiYY6qj33QPnmODx9hctFmp7ryl8BaGAkO0jbfN\nuhgIUYG/1JmhjMz+zv1xWTWTGQuNqWLWEJ3Bx8JCA/4BhgJDSLJESAxhFOLz8a9zXccN+TdwYvCE\nyrBLRubD7g/5vP9zVmev5tbCW+MeqJ/1fRZ3PZdFnb/dOdGpCs10ejo57z6f8E0tJyWHla6VQLzn\nzdRtp9nJj5b+iPaJdpwmp2L4pTF/aAK+gJQ5yqhwVihilciRDlBS8mJM9t4Gkgo/LHMtU2ZAJr1J\ncQ2syajhtabXlFmVy+JSjaF9op0uTxeF9kIWORYx4BsAIVqSnWhBajwUzWyZyfxqb9te1YLa5B6Y\nISkUJ97Zlmz0gp5ef2/c9XSCTpXiZjVYlRjz5BZuMzGd02IyzCTeMcZD49Sk13B29Cw2g42KtIq4\n1FAg4VuUzWBT/v2vy7yO44PHVTYGEA2THOo5RKYlk+Wu5arPpv5uGHSGuKKsdEt63APyxYYX2VS0\niW5vN2EpjNVgZVvJNirTK5UwSU1GjcrXPZEpmd1oj6v+1Jg/NAFfIPa07eGLgS8w68xsLdnKYufi\naRcxr8u8jn5fP6eHTyesipyuq82yzGUYdUYK7AVcn3W9sn919mpVlehDFQ9xpP8IFoOF24puU/af\nGTnD682vx19Yjnpl3Fpwa5zF6UhgZFar3FgvxxizuRcadUbFfXAyekHP3aV3J6wivTHvRiZCE9SP\n1sct/M2EVW9FEISkzlmasVQVv56O4tRivlPxHaXTfJ+vj9PDp5NKU5xsgWDSm/j+ku/zbvu7HB2M\nf2gPBeJDTimGFFXGyy0Ft8SFmXKtudxTdg972/cq/3YyMn3ePn6+7OeMBEfITsmOs55dm7OWVGMq\nPb6oD3xsjUPjq0MT8AWgcaxRmRkFpAAfdH0wq4n95qLNise2J+yhxd1CpiWTNHMaDpNDtYBWm1HL\nssxlVKVXJTWeqvSqhMdO9f2YjC/ii/pET3nlz7HmXFJfycnugS6LSyVGvb5eVTMFiGbY/P3Kv5/W\nAsCgM3B32d0sdy1P6MENUbH2ieqsj+nEO9+aT7+/XxFdo84Y1+F+KrGCpFg5f+xNIVGp+nQMBgb5\nqPsjpQDGoDNwZ+md2Ew2jg8cZzw8rlyzy9PFfzvx38gwZyhrF2uz1/Jh94dMhCeoSqua1sZ4ZdZK\nzo+fV2L2EJ09z9bTdUnGkhntgK8mBnwDuENuCu2Fs/qkXyloAr4ATM3FDYpBRElMKr92wDfAs/XP\n4o/4MegMfLfiu3x70bd5q+UtxsPjLM9czh0ld6jOGfQPMhYco8BekHRTW4iPrRt0BqWTTXV6dULx\nTDGkcFP+TQnj7CX2Eto9iUVv8mJjopnkVERZpHOik/rRerJSsliZtTJhtWBxajH3lN7D221vx302\nVbwhPk0vxoB/gG2l2zjaf5TR4GhcY+VEGAUjW0vjO+0M+geTis/H+Kj7I8JSmKAYZHX2anKsOWSn\nZGM32THoDBTYoh2kYnngkxdM2yfa+el1P0VGxmlyztgUojS1lPPu8/gjfkodpXGuhV9njg8eZ3fr\nbuXn9FTNU7N6zF8JaAK+AFQ6K0kzpykZIquyVyX0qzg1dAq9Ts9y13JFLD/r/0wpTY9IEQ71HOJ7\nS77Hj6/7ccJ7nRw6ya7zu5CRcZgcPFXzVNJ+FJuKNjEaHKXL00WRvYjbi2+ncayRFEOKspCViOsy\nr4srIQemFe9EOI1OAmKAoBSMm31DVPBfaHhB2T4+eJyna55O+BBMZOR0sUTkCK3jrdxddje/OfOb\npM4JySGeO/ccj1Y9qvz7+cI+Wsdb0aFL+H0lQkZWDMFOD5/m4YqHeaPlDWUWHxSD0xbIhKQQ/1H3\nH/hFP/m2fB6vejzh7LJuuE61QFqSWjJrcc9cCYpBuj3dpJpSL0sB3MVwqPuQ8vvqDrk5OXSSDfkb\nFnRMyaAJ+AJgNVr5Qc0PaHQ3kqJPiQtfhMQQO87uUGaip4dP80T1E+gEXZwV6mweFZN/McdD4xwf\nOJ6UDShE46+TLU8hGiKZjogU4bXm1+j19lJoL5zT4qAn7KHEUaIyZ5opBbLX18sfz/+Re8rvAeBw\nz2FGgiOExJDiZX4xJLpXl6cLHbOXg08+t22ijS8GvmB97npkOZqFM7VBhtPoxB1WV2bmpuTS5+9D\nj16VGx4QAzSPNatCMN6IN+o3nsAeQIdOCWf1eHv4pO8TNhVuosXdQv1oPRmWDNblrItbOG52N19W\n3xJv2KvULggI3FV6F6uyV122+81GXL9P4eqQxvloanwH8EtAD/xOluV/n/OorgGsRuu0Hci7vd2q\nMEL7RDtjwTEyLBnclH8TLe4WRoIj2Iy2GXtPQrxjXaxs/f3O9wmJIdbnrk86Vj6ZidAEXd4uJkIT\nhKUw5Y5y9nXso2MiWgQ0EZ6gLLWMLm9XUv0/pwqmSW+iKq1q2kbKiTg9cpoUQzT+PjmWeykkelBM\n9ZhJ9tygGMQf8bO/c3/C7kbusFu1BhCzAY5hM9iU+woILEpbxPGh40ooLs0ULYTKteZSaCskz5an\nuBz2entVv0shMUTreCu/b/i9Ms6RwEjcDPhy9YaN8eXQl0rWTSwdciEFfGvpVl5tepWgGKTYXryg\nXv4Xw5wEXBAEPfD/AZuBbuCoIAhvy7I8fQ3yNcx593kaxhrIsGSwJnvNtOY+dqNdJWgGwaC89jpM\nDn5y3U8YD42TakqdtS3WnSV3srNpJ0ExSKGtkKWZS3nm7DPK4lenp5MfL/3xRf3BDgeG2XF2hyqW\nrxf0cQtzw8HhhF1bXBYXoiQyGopmn5h0JlwWl1LAEpuRFacWc3b0LMP+YSJShIA0s4c4RJtJJBua\nuFi6vd282PAiqcbUpM2zYml0//v0/57xnJAUIs+ax7bSbaryfBGRyrRKxkJjBMUg38z9JmWOMp5c\n8iRHB44SEkOcGDqhVFv6Ij62lW5TxLBuuI43W95ERiZFn8Lq7NWcHDqpesg0uZv462V/jSfkoXW8\nlRxrzkW1brsU4t4kF3jGW+4o5x9W/AMBMaB0b7oamOtPbS3QLMtyG4AgCK8A9wKagE+hdbyVFxte\nVM16prOczUrJYmvpVj7o+gCDYGBryVZV3NKgMyTdmq3UUco/rvxH/BE/xweP88svf6n6XJRFBv2D\nKgHv9fbS4+2hwF6gmHRN5tjAsbiFWFEWMQgGVcGRUTAmFFNJllQCEpJCqurDDfkbqM2s5bdnfhuX\n8zwbMjI51hw87viu8MmQZkqb1jckxkR4ghtyo02qB/wD0/qblznKeGDRAwz6B5MS/EH/IIX2QlJN\nqarUylxrblxDY1eKiztL7qRuuE7lPTMeGqfX24s34iXfls/SzKVkp2QzHBym0Ba9dpZFPdvOsmSh\nF/Rxi9/TIcoin/d9zkhwhOr06ktKH1yZtZIzI2fo8HRg1CVe7P2qMelNCeswrmTmKuAFwORAZxdw\nedutX6U0jTWpZz1jTTN6hq/JXsOa7DVJX1+SJF5//XXOnDlDXl4ejz/+OHZ7NN/XqDMyFB5KWNhi\n1ptVJe0Now3sbNqpbH+r/Fssc6mbVUz3S16SWkKnp5OQFKLAVkB1enVCP++R4MiMM5x+Xz/vdbw3\no3gbBAOiLMaFK8od5dxefDvvdLzDSCD6it7j7Uk666MktYRl5mWcd5+nx9czbbrfscFjKuG26q34\nRdBQMgoAACAASURBVL9yn0JbIQ8uehCr0RrXrWg6InKEPm8fEfHCQ7DMUcbqnOlf5wvthZj1ZqWa\nNMOcwe/O/Q5JlrAb7Ty55EmyrdlkWy94u6zIWsFYaIxzI+fIsGSwrXRbUuOLsa99H18MfAFEH+bb\nq7dH11cGj2M32tlSvGXaDI5+Xz/72vcREAOsy1nH/YvuJ8WQkrDkX2N25irgSf1V/OIXv1C+3rhx\nIxs3bpzjba8+poYo5jvG+MEHH/DBBx8A0N/fj16v5wc/+AEQfVi82vRq3DlVaVXcXHAzTvOFrJRY\nO7AY73S8Eyfg38j5BnXDdXHpfrGZvCzLbCrcRHFqMf2+fhpGGwjLF+LgNqNt2ubEAA1jDbMK7lRf\nkBjr89ZjM9r49qJv0znRyanhU/gj/mm9YqbijXi5pfAWjg8eV8TbordQlV6l5MXnWnPjYtnyX/6L\n4UpxKQZbrhQXS9KXxLkbJvIc39m0U1V40zHRwa9P/5rhwDCZlkweqnhIFa9OM6fx5JInOTZ4DAGB\nxrFGZdyesIejA0cThkM2Fmy8qDTBz/o+49O+T7EYLKpO9zIyxwePq9Yc3EE3T9dG3TPbJ9ppG28j\nz5ZHhbOClxpfUpwT3259m7+q/au4yuJkmAhN8EbLG/T5+ihzlPGt8m9ddbPnqRw8eJCDBw9e1Dlz\nFfBuYLLhchHRWbiKyQJ+rbLStZLR4Cj1o/VkWjKTbv+VLAMDA3HbZ0bOcKDzAKPB0ThBrEqr4uGK\nh+PygmVZfVyiBUiLwcLGgo280fKGav94eFyJre9s2snfLv9bHlj8ABCN/3/c+zEGIepW2OhuVJ2b\nnZKNjEy6KV31WaJskKmhmhiZlkyyLFk0u5vZ1bKLiciFsIXNYCMoBqcV/hjN7mZebHhRlUsdEAOK\neNsMNjLNmXECPrVwKZYvH+P+8vsZPTeqOk+URZVnjM1gi/MJF2VReVAOBYZ4+/zb3FN+Dw6TQynW\nybHmUOms5JXmV+LuWz9az+aizQkbOc+GLMu0T7TT7+/nnY53gGiK3dRrCah/h2ILsJOtfSHao3Oy\n7a2MzHBgeNYer4l4t+NdxSK5frSej3s+VhlpXY1Mndz+y7/8y6znzFXAvwAqBEEoBXqAh4BH5njN\nryWCILCpcBObCi/ul6ynpweDwUB2dmJ705AYwqQ3sWzZMg4dOqQIcGVtJW80v5FwJluZVslDFQ8l\nLOq4tfBWVQilJqMm4X0XOxersiOmEpbCjIfGsegt6HV6yp3llDvLaZ9o59lzz8YdH2syMTXckGj8\nOdYcVXglz5pHkb2IG/NvREbmtabX4lwIA2Ig6erH4cDwtCmL3oiXM6NnZr1G3Ugdjg6H0vbMpDdx\na+GtvNL4impdYCQ4QlVaFaPBUVWjjeno9nbzv07/L6wGK49XPa7kuO/v2h8n3hC1KWgYbZj233Em\n3jr/VkKbBlEWqUqrwh1ysyR9CRVpFdSN1Ck/35hFbd1wnepnWD9aT3FqsZKpZNabKbLHN9xIhthE\nQdmeocvR15k5CbgsyxFBEH4GvEs0jXCHloEyP8iyzI4dOzh6NFpyf+edd3LffRc8UIb8Q7zU+BKj\nwVGK7EU8Wv0oP//5z5UYeP51+ZxtuJAXHBmPIOgE9HY9ESkybQy6Kr2K7y/5PudGzpFuSZ82Dm8x\nWPjZsp/x69O/jvtjguhi4AddH9DkbsJpcvJI5SPkWnNn/UPzhD2syV5Ds7uZoBiMWyytTq9mQ/4G\nXmp8CW/YS5G9iMeqHsOgM7CnbQ/nRs4ltJC9mBZpOkHHxvyNfND9QVLHT8cnfZ9QZC9iScYSWtwt\ncc2NYzSMNSR0IUxETPx9ER9vtryJO+SOmwFPZeo9uz3d0XS51GL0gp7DvYeV7JNNhZsw6AyMBEam\n9dhZ7FzMI5XReZo76Ob44HGWZy4nJIVwmpxsKIgWwCRqaLytdBuf9H5CQAxwfdb1cUZtEE27DImh\nGUv4q5xVyoNAQJiX5ipXI3PO3ZFleR+wbx7GojGJlpYWRbwB9u3bxy233ILTGf2j2NexT8lU6PR0\ncrj3MJtrNlNTE51pBSLRdChP+P9n773D3KrvfP/XUZdmNL33Xt1twBiDjQvYxhiMgQCht1CS7G42\nySXZe58nezf3PvfufX7Zlk0hkAVjWsB0bGNwrxj3cZnm6b1KozLq5/eHouM5kqbauKHXX5Z0JJ2R\nZz76ns/3/Xm/rZh2mRiu969so2dHY1owtsoi15g7oegrvUrP3JS5sr55SVwJyfpk1IKanR07Af9l\n94aaDcxPm0+SLkmWCGRQGWRFWqPQcGPajWRFZ2FxW9jWuk0qQEqUtFpaMbvM/P2sv5c2hrvt3ezp\n2BMyjDKSPGMebtFN45Df0zzcJKQCBWqFGkEQ2Nu5l7L4Mv/YuyhidponHbYA8G79u9xbeC+t1tZR\n+/pqhZp8Y35IW2kkAgI5xhypbQATsxwA/5dpgK0tW9nftR+A7OhsKuIrpI3mhiG/hezK3JXhbQmi\ncySr2YNdB5mROINXzrwiKWyyorNYV7hOWhzcnHEzA84BGocaSTOkcXvO7WiV2jEHyar6q6RYv/L4\ncu4rui9ksWFxWTjYfd4mtyKhgqK476aR1tUxbhQhhOCUnWApm06l49nKZ3lz35t01p8fCrEes9Jb\n3kuXvQudUseAY4BUQ+qUQ5hvybiFKHUUXfYuCmIKpEv1YPWJ1W3lq9avQvTilfGVMme96YnT+cOp\nP+DyuRAQWJSxCJ/oY3fnbrx4sXlsbKzfyMzEmRzpOzLmuakVaskKNSMqg/KEcra3bcfusTMtYRq7\nOnbJvjx8+PCIHkm7Xj1YzeNlj5MXkycFAo/V5lALatlmbYAPzn0w6oahAgXJ+mRqzbUICCgFZdg+\nvYhIojZRVsDDoVVocfrk/uaBAuj0OqXiDYT9Ugm0pmI0MSzLXsZXrV8B/ji8QNqRiMiWli1olBqZ\nPLLN2obVbZUUKBqlhvuK7hvzfGU/oyjyaeOn0pXS2cGzVA9Wh7R/qk3VsvetGayZ8Htca1wdavXv\nIIWFhcyde34ybcWKFdLqG/wJ84FLZ41CI7OLDRCjiWFZVphJTZ9/U/G3Vb9lfc16/rPqP0PiwMLR\nZm1jZ/tOTg+cpt3aTr25Ho/oYV7KPFbnrfYHJtv9BW5m4syw0rDgNsaJfn98mVqhJjMqExFRaoGI\niJzoPxESTOwRPeMWb/Dr6cviyrB77Ozs2MnLp1+mxlRDq7WVzS2bw3qkBJ+f2enfVLS4LdyYdiM3\npt5Iii6FKKX8Cy/DkMHTFU9jUIaahfnwSWlH4R4LhGSIiKNushpUhhDbXa0iNJcyMzqT61POO1sW\nxBRQ1V/Fib4TKARFyGo2OLEpVu3/HXN6nVII9tzkucxLmRdS7HUKnax9o1KoqB2sDTu8NRHC/fzh\nNtGDDdkCap/vIkKw6uCiv4EgiN/2e1yriKIobWKmpoZ6kHTYOugd7iU7OnvUwR6fz8cf//hHjh/3\nBwhEVUYxbfk0EJGyL4GQaLJgmi3NvF79eshGYLohnSfKn2BH6w4O9BwAoCK+guU5y6kZrKHL3iUL\nLxiPrKiskJCEkrgS+h399Dv6J/w6MLZ3SoDMqExp1TnSYCzA85XPY/PYeLP2TbyiF4Wg4IHiByiO\nLWZH+w7Omc+RrE9mRe4K2ixtbGrexJBrKKQQGdVGbB7bhDdSR1IWX8ayrGXs79ovC3iemzKXDEMG\nTZYmBh2DxGpjWZm7kmh1NL3DvdSb6vmi9Qvp+GXZy9ApdXze9DkiIjMSZ1A9WC3bMwjksn7W9Jmk\n9Qa/j3iTpUlqQeUZ83ik7BGq+qvY1b6LIdeQ9OVXElfCQyUPTfrnBNjRtoNdHbsA/2b1k+VPhgQo\ni6LIZ02fcazvGAaVgXsL75U2Tq8lBEFAFMUxNzgiBfw7gCiKNDU30TncSUJaAkWxRbxV+5bMZ+SG\n1BvGHCza3LyZr7u/DrnfM+TBucPJUN8QmgwN8UvjUWgUkglTYIOpqr9qUhaqwTw37Tn+cOoPU37+\naNyScQu5xlzcPjdfd30ty/3UKDQszVrKno49WD3nZYXFscV8v/T70m2zy8y21m2c6j816hh/fky+\nVPwmg1JQ8qMZPyJOGydtXLZb28mOzmZd4boxHQM/bPhQ5umeHZ3NUxVPMewZxu1zE6OJ4d26dyV9\nukJQ8FT5U2RGZ/JfZ/9L1q6ZnjCduwrukjIwy+LLpD559WA179S9I3vvn83+2ZTbcu3WdoY9w+QY\nc8bUdvtE31Uz8j4VJlLAIz3w7wDNlmYOew6j0+mYET0DlULFsuxlbKjZgN1jJ1GXOG7g7GgWtEMH\nhnD2+Xuurg4XtiobxrlGacNPRKTN2hZSvGM1sXhED5lRmdSaRt+8A38vO1GXyNKspWEnOy+EAccA\nC9IWcGbwjKx4g3/Ef3NL6P78SFsDn+jjldOvjDsq32ef2IajgEC0OhqVoMItupmdNFtSagSkgxPB\n5DSFBE4Ezluv0qPH/+97Cu9hb8deLG4L0xKnkRnt9xYvii2SFfAzg2coN5UzLXFayHsF+8ZrFJqQ\nVfNkCJzDSMwuM1+1fsWwZ5h5KfMoiy+7pov3RImswK9x+ob7+MOpP0iX9OmGdCmR3OV1YXFbMKqN\n/n62z0NZfJm06vH6vFjcFslc67Omz6g11+LxeaTR7b5P+nD3nu9TGsoMxN4UWuxHqj5iNbE8XfG0\nJBP7fdXv6R4Ob/mqVWhJMaRIl/w1gzWcGzp3/v3CpOpMFpWgCvEfGQ2j2sjTlU9LX2hNQ028Vv3a\nBb1/giaBWF0sN6XdRGFsIVa3lZdPvyx9KSxIW8BtObfRam3F5DSRa8wdN2zgtbOvyfIqwa/ieXba\ns2NaAo/ki5YvONB1QLqtUqh4ac5LYRUqezv3sqdjDxqFhjX5ayiOK57Qe0yUkb8jCkHBMxXPXBSf\n9yuZyAr8O4BP9LG/az8dtg5yo3O5PvV62YBOu61d1o/ttHdKwz8apYYERQLv1L1Djcm/k5+kS+IH\n036AzW3j9erXGXQOEqOJ4ZHSRyRDpU+bPuVIj38T0VBmwNz71+lBJeiLw0dR+fCRpEsiThtHvbme\nfznxL9yWfRvz0+ZzU/pNfNDwQdjnOX1OyVe8zdoWkud4ocUb/JuiEyne4A/jONl3knabv40xUqIX\nYE7yHFqtrdLG8Hi9+EHXIE9WPin9bMEqi0Pdh4jVxrK52X81YFAZeLri6TENzYJ7+eB3NmwYaphw\nAc+JzuEA5wu4x+fBI3pQhSkbC9MXsjB94YRed7J4fB7ZF7xP9NFl77rmC/hEiBTwq5zdHbslk6oz\nA/7BnRvSzvuJpRnSZLprjULDu3XvMugcpCi2iPlp86XiDX5t8Vs1bxGjiZGK2pBriO1t21mTv4aX\nT78sK3aGEgPpaenY+mw4Ehyo4vy/UuEGZ3QqHfXmesD/R7ilZQtl8WXMSJpBvbl+1MGRkYwXqCAg\n8EjJI7xZ96bs/aNV0bI+9pjvMeLzCsaHT2rjVA9WY1AZZAU6UZfIHbl3gACHuw9jdpnptHfK+t/B\nn42IyLa2bdyVf5f0+Ej0Kr1sJWz32Pmw4UOWZS8bVa9fkVAhkwwGSNQlTuQjAKAgtoBkfbL0RTQr\nadZlMZ1SKVRkRGVIah2loJRi5L7rRAr4VUCbtY0Pzn2Aqc+EfY8dt8XNvHnzeOCBB0J0wc2WZlkB\nTzWkcnv27VIv1+VzSS2IQz2HiFJHhRSsRktjyHSfx+dhc/PmsCtVRbKC1HT5ePutmbdysv8kfcN9\n+PChUWjINGTSZpUrTHqHe4lSR4UYPY3GeL3mvJg8CuIKyDXK03wWZS7ii5YvxvVCmZ08m8WZi9nd\nvhuX18WspFns7NjJsGdY6pWPJKAjVwpKZifPpt3azj8d/ieyorJ4oOQBotXRIRt82dHZNFuaZavy\ngEGU3W1nd/tu6X6VQkVZfBmnBk7JXqPV2sprZ1/jsbLHwiowbky7ka+7v5a+KAKa+pK4kpBj3T43\n58znUClUFMYUSldwWqWWpyqeonawFq1SG/a5l4rvl3yfHe07GPYMMzdlrsxd8btMpIBfBbxX/x5m\nl5m+Hef7zTt37iQ3N5eM7AzZ6i6cMdBYG0pDriGWZC7hq7avZPcHX/J3D3fj9oZP1hl0DkqFXUDg\nhtQbWJC+gIUZC7F77FQPVPNNzzcc6Q3Vbp8eOE2dqW5CqT3hzgv8boGZUZmkGlKlHMPgL5pver4Z\nt3hrlVquT7meWE2sTFJZGFco/Xs0i1uv6MXsNEtGTm22NjY3b6YgpoDMqEwazA24fC50Sh39jv6Q\nnyOg4z87eFbmR+71efmm5xvCISJSY6oJW8AHnYMhq/xw6fEen4fXz74uSTdnJM7gnsJ7pMd1Sl2I\nG+XlIEodddEN4K4FIgX8CkcURWnV6bXJWxKDg4OsmL8CAYF2Wzu5xlwWpC8IeY3M6MxRHfxK40sp\niSuhZ7gnpIUhIKBX6rF77RM2CxIROdh9kAHnAA8UP4BBZWBH+45RV86jacQLYwqZmTSTnuEeuuxd\nUusF/JugI137HF4HzdZm1uSvkWR1+TH5DPaeL+ITMYpak7dmzL5q33CfTJYX3Ns+Zz4nO/7swFlO\nD/iNr/KN+dySeQvJ+mT+9bg8VGNB2gLK4suAsb9sAfRKvcz50OFxcKr/FKXxpbJ0phR9imSlAP5x\n+nC2rc2WZpnu/mT/SZZnLx/ThyTClUOkgF/hCILArKRZHO09ir5Qj63Kf6mtVCspqSxBqVCOm4uZ\nrE/m4dKHOdJ7BL1ST6IuEbPLTEFsgZSmkh6VHlLApydOD9uXTtWnkhuTS4ulJWzGI/itROvN9eQZ\n8yYcPxYgIyqD75d+X5KJWd1W3ql7R2q/BAycRhZPj8/D1tatUpjA0qylGNVGTC4T2VHZfNb82bjv\n+03PNxzvO05FQgWzk2eHPN5oaRxzFR+sAR95u9HSSKm9lPyYfGYlz5KGZAwqfzbqO7Xv0D3cTb4x\nn4qECs4MnEGj0JCgTaBr+PxnHCjeaoUag8rAsb5jHOs7RnZ0No+VPSYpRHQqHU+UP8GBrgMICNyU\nflNYTXXwl45CUIwblB3hyiHyP3UVsDpvNXnGPPoz+tmevJ3hoWG02Vq+snxFoeiXnQU2ueanzpcF\nNATIi8kLe6kdMMYPbBAFiNPGsTpvNeeGzskM/AUE7iq4i4yoDKxuKzvadjDkGsLkNNHrCB3H1yg1\nYYdYRnMHLIsv4/6i+2Ua32h1NLdn386rZ1+V7hMRQ17j9MBpWiwtVCZUcrD7IAICy3OWj9qCGImA\nIMnu6sx1RKmjZD3fHnsPPp+8QI/mWzIaB7oOUBJXwh25d5BnzMPqtlIWX8abtW9KG4WDzkEWZSzi\npbkv4fa6+V3V78K+ltvnll2FtFpb6bB1kGPMke47Zz5Hj72HOG1c2OzUkb83AW5Ov1mmc49wZRMp\n4FcBCkHBjKQZ1JnqUOYricYvN+se7mbAMSDZyoJfifLC9Bdkl+I2t41D3YcQEbk+9XqZFG9LyxbZ\nRqhOqSPVkMqa/DVolBoeLX2UL1q+oMfeg1FjZGHGQjKiMhBFkdMDp/GJPqYlTmNm0kz2dOyRFBoZ\nURkUxvp7xw8WP8j+zv2c7D8phReMLLwCAitzV1IcV0ycJi6sT3m4zdM4bRw2t01m5GVxn3eqExHZ\n2rJ11M81ShXFsGeYOG0cDq9DZmzVZm2TCvjO9p2S0idBm4CAgMPrCOuFnqpPHVXTbnaZ+W3Vb1lX\nuE4aiOm0dYb40HTYOvD5fGxu3hwSFBEgYNQ1kpGFt3qwmk3NmwC/ZYLdY+fh0odlx7u8rpBefOD/\n7Epl2DOM3WMnXhsfGeQhUsCvKhJ0CSFWrMPeYVlxM7vMUjgu+Fdq/3X2vyTr0VP9p3hu2nPS5XRw\nb7ssvozFmYvZ2roVu9vOdanXkdeTx/GPjqNQKJj34DxI8MsXAzayx/qO4RN9zE2ey+Eev3Suw9bB\n9rbtLM9ejkapYXHWYgpjC2WraIAlmUsojC0MO303kryYvJD+r91tHzVQeCLYPDYq4itYV7iOjQ0b\nJRkmIAUNuLwuWZZooLe/v2s/Nsv5Ah4Itwgu3sEKH5/oY0fbDkriSjg7cFbK7ZQhwobaDSFXRSqF\nCp/Ph1qpZm3BWmxuG5uaNyEisiRziSxqLbi1FdhcHUm8Np6y+DJpPD7HmHNFy/OqB6t5/9z7eHwe\nsqOzeaT0kas+Ru1CiRTwq4hEXSL3F93P7o7dqBVqbsu+jXhtPBqFRjIkEhDotndLBbzf0S/zjR5w\nDtAz3CM9PiNphjQoIyAwLXEab9e+LRWi+rZ6ejf2Sumnf/7znyktLZVNQ4LfS1pElF3W7+vcx7B7\nmDvy7kCpUJKkT5JtrMVqYimNL6Xf0Y/BaZA22Tw+j99aVeHXQ1tcFgYcAzxc+jDrq9dLdqmjrU5H\nkqJPweQ0hQ15AP+IuKHFwMqclRiUBixuC5UJldIkoSAIIf12haCgMqFSChQAMGqMIStylULFusJ1\ndNm6JIMm8Lde3qh5Q3p+8Gq6YaghxHs8TZ/G3YV3k6xPlunEZyfPltpJI8kz5snOO9+YH/KzC4JA\nuiGdbns3Uaoo1uavRalQ0jvcy0cNHzHkGmJm0sxx91guFZubN0upQ63WVo71HeOG1O92hnqkgF8B\neL1e3nzzTY4dO0ZycjJPP/30qBFqZfFlkmIhwEMlD/H+ufexuq2IiHza9Cl6lZ6KhAqMaqOsQKgE\nlczX5LqU64jTxNFl7yIvJo+MqAz51NuwTxZd7fF4sNlspOpTabG0MNw4jLvbzXDZMI5poavho31H\nSdAnsDB9IXqVnsfLHmd/1348Pg/ttnZ+f+r3gL+IPVL6CE2WJna07UAQBFbmrCTFkMKbNW/i8rkw\nqAwTStWJUcXg9Dlx+pz0DPdQEFPA9MTp1JvrJVXISE71n+JIzxFERBamL5TJ5lxeFyqFSvb5pehS\nKIkrIUYdQ6e9k7yYPI73HpetemcnzWZl7ko0Sg15xjxqTH5nRq1Sy3Wp1/FZ0/lN1eBWiIgoc0ZU\nCArWFq4NO0E5WhshLyaPB4of4MzAGWK1sdyccXPIMSf7TkpXUYPOQf7t5L9REV9Bn6NPUu3s7dxL\nelQ6lQmV4T/sS0jw//1UbWuvJSIF/Apgz5497Nu3D4Dm5mbWr1/PT3/60wk/Py8mjyh1lCyIt2Go\ngYqECjyih+XZyznccxif6GNG4oyQvmdxXLHMuyInOkeymlUnqVHFq/AM+lc+xcXFJCcns1xcTvOJ\nZk5s98vq9pzew4muE0SVhTrQDTrOt3iS9EmsyV/Du3Xvyuxh3T43uzp2SaoIURTZ1LyJHGOOtHoO\njlcbDavXKmtbNAw18L3i7zE7eTY3WG7gw3MfMug6f04j2zB7O/dSmVApyQk7bZ2yAusRPfzHyf/g\n/uL7KU8ol7TV8Zp46kx1DHuH0Sg03JB6g3R5r1fpeabyGcxOM1Fqf999rPF6AYGHSx7mi5YvcPqc\nzE+dP+Hx93ZrOyanCYfHgVFrZG3h2lGPDSetPDN4BpUgLwvhxvIvB0uylvBp46f+cAtdIrOSZ13u\nU7rsRAr4FcDg4OCYt0fi9rk51H0Ih8fBjKQZUt8zVZ9Kt/38yjnVkMrxvuN80vgJPtFHij4Fu8fO\n9vbt7Oncw/dLvj+qh/KDJQ+yq30XR3uP4lK5SFydyHD9MAszF7Jq8SoUCgUaNBA00+JsdoYt4CVx\nJXzc+DE1gzUkaBPIjM6U+q4jCVZ5iIiMUuPGJHgMXq/USyqMHGMOfzPrb9jctJmve0LtcQHpC8Ph\ndfBFyxchj3vxsqVlC6XxpdJ9pwdOSy0dl8/FV21fyTYNlYJS8i7RKrWsyl3FF61fIIpiyMrShw+j\nxiizrA2m3lRPtamaRF0iN6TegEJQsK1tG3s69siOuz71elblrgr7GoWxhezr3BfyRZKsT5Z65mqF\n+rJOYI5kTvIcco25WFwWMqIyvvP9b7iAAi4Iwn3Ar4Ay4DpRFI+O/YwIozF37ly2bduG2+1f6c2f\nP3/UY/9S9xcp+/Gbnm94btpzxGnjWJW3CrxwrvYcOfE5zEuexz8f+2epmI1cbQVWu6MVcL1Kz4rc\nFcxNmcum5k3YPXauX3U9c1Pmyo4LDplQxYb+OlUmVGJymjjWewzwr6KDAxsCRKmiZKnlJXElTE+Y\nToe9Y8KTmsFoFBq+V/w9qdVgcVk40ntk1NVvnjEPh8fBoe5DWFyWsNJI8F8hjCSgrglgcY2tfb8u\n9TrmpcwD4N9P/rtsI1pEpNPWOer/zznzOTbUbpBuDzoGuS3nNvZ27A059nDPYVbkrAjbahERSdWn\nMugalNwltUotawvW0mZrY8g1RHl8uWxz9HKTqEuclJ/Ltc6FrMCrgLXAHy/SuXxnycnJ4Re/+AWn\nTp0iOTmZOXNC49HAv7k3MrjX4XXQZGlilnYWSp+S2g9qaWxspJ56PE0eCJ2clgje9ApHsj6Zx8oe\nG/XxO++8E5vNRn19Pb5EH8p5oa8pIMg2NsdEgEdLH6XGVINSUGJz29jYsHFCT43XxKNQKEJSe+am\nzCXXmMvpgdOYHCYO9RySzken1Entk6LYIq5PvZ4TvSd4u+5tYHTjLAFB2tjz+Dy8Xft2yKbuzKSZ\ntFvbabI0kWpIlQamZK/zV7nkvYX38sqZV6QvFYWgkNnF+kQf3fZuNEoNibrEkOGbenM9twu3oxAU\nIat5BaExauCPinur9i1pU1Cn1HFr5q2UxJcQr40f1WvkZN9Jjvcdx6gxsjx7eYg7ZIRLy5QLuCiK\n1UBYzW6EieETfZzoO4HNY6MyqZLbb799zONVChUxmhiZ9C9B678sr66uprHx/LDM7t27eWLhE2zt\n2Sq1UFxeFyaXCYPKwNKspRd8/hqNhkcffRTwbyhV9VexuWWztJpD9E9zRqmjwjrjjUStUHN9abm7\n4AAAIABJREFU6vWoFCppw+zfTvyb7BiloGRJ1hLUqNnUukn22KzkWdyQdgP/evxfZT3tjKgMNjdv\n5lDPoZD3dHgd3F90P/HaeNKj0mmxtHB68PwmZ7h0HYWg4LHSx8iN8bsAnug7EVK847XxpBnSePXs\nq9IV0Oq81dKKO5jM6EweKnmILS1b8IpelmQukdotPtHHO3XvSKEXxbHFstYNIClT7sy7k0+aPpG1\nkDyih6ahppDVfL+jXyregc+iKK4o7Lh9gKahJpntr8lp4onyJ0Y9PsK3T6QHfhn5uOFjTvT7NwH3\nde7jB5U/kNJXRuPB4gf5rOkzhj3D3JB2gzR5p9XKPTQUCgWzUmdRnl6OzW0jRe8PRTA7zRg1xjE9\nN/od/Ti9TsmKdiIoFUrabG04vU58Th+D2wZxdbr4S9ZfePHFFymLK6PadL7vrRJUzEqexYyEGVg8\nFtIN6SH+1sHv7RW9lMWXoVPqQgp4nCYOnVLHs5XP8nHjx5hdZqYnTGd64nQ+PPdh2HPWKPxTooEB\nmGDddTh8oo/6oXqpgIeTJ3p8Hqr6q2SF9FD3ISoTKqkerEar1IYkygRvJAdoGGqQJRbVmetwep0s\nylhEjamGBF2C374W/5dYXkwe/3pC7rUS7hxTDanoVXqGPf6+fYI2YdTUpQDBn89EPq/RcPvc/pg9\nUWR64vRIP3uKjFnABUH4EkgL89AvRVH8dKJv8qtf/Ur69+LFi1m8ePFEn3rNIoqizGdk2DNMvbl+\n1FVagPSodJ6pfCbkfkeCA0O5AftZOwhw5313otPp0KGT/WEm6ZPGfP2R05SFMYU8VPrQhNottaZa\nagb9vuLW41Zcnf6i0dbWxnvvvcezzz7L27Vv02xtxqg28kjpI2FH/kdyU9pNfNL0iey+DmsHaqWa\nopgi6of8Blc5xhxJ+hetjpa+rFQKVUivGkAtqInTxXF7zu1olVo+a/qMk30nMagNY6pDpOcr1LRb\n2znccxiloJSGeALMTZl7/irkr/QN98m81EvjSnmw5MEx3wcIsfUF/2Sl2WUmVhPLkswlsuzJOG0c\n81LmSV4rmVGZ5MeEasCj1FE8Uf4EB7sOohSULMxYOK4HSo4xR/b5jOZFPh5e0cv66vXS/MGR3iM8\nWf7kd96DZefOnezcuXNSzxnzExNFcfmFnFCAkQU8gh9BEIjRxMj6w+OtgMZif9d+YhfEYpxrBAUo\nsiY/Zuz2udnetl26fW7oHHWmuhDdudVtxeq2kqRLQqVQ0W3v5p26d6QVp88pbz1YLBYEQeCh0oc4\nM3CGfZ37+KjxIxZnLOZI7xFMLhOVCZVMT5xOramWqr4quoe7sbtDZYO7OnZJg0nx2njuL7pfkvyd\n6DvB/q79khqnxdKCXqUPKcjxunhemP4CAMd7j0vFzuV0Ea+NJyMqA6/PK7tiCJAZlYleqedPZ/4k\n3adVaFmatRSHx0FeTB7FccW0WFpkPiM+fLKNyhpTDd/0fMN1KdeF/b8IkB+TT54xLyQezewyY3aZ\nef/c+zw37TnZY6vzVjMtYRoun4v8mPywPijgH3Jak79mzPcfSVZ0Fg+WPMiJvhMY1UYWZy6e8HNH\n0jfcJxVv8K/ku+3d407jXusEL27/8R//cdznXKyvvO90I9zj8/BFyxe0WlvJis7i9pzbR/2jGcn9\nRffzYcOH2D125qXMu6AcwUBLRKFVyG5fKMErwJrBGt6rfw+P6CFFn8Lj5Y/Tbe+WtQv0xXrcjW48\nHg+CIHDLLX6P7t7hXt4/9750bIulRdp0a7G0sLt9d1h/kZGMnCoddA7SbGkmPSqdD859ENY5sdPe\nSaw2VqZl1iv1nOg7QWVCZYhTotPr5L6i+zjcc5hB5yAmlwmj2siSrCVkRWcRrY4OMZhy+pwYNUbZ\nsEyqIVW2X2FQGUJ07Dvbdo5bwBWCgsfKHmNH+w5Jyz+yxx92FB9GVbAEaBpqwuF1kB+TP6nflZK4\nkguWFRpUBpnFgICAQW24oNf8rnIhMsK1wL8DScDngiAcE0Vx5UU7sysEi8vCJ42fMOgcpDy+nCVZ\nS0I2bne175Ic77rsXWgUGm7LuW3c186MzuSHM354Uc5zZe5K3qx5E4vbQlZ0Fjem3Tip5/cO9+L0\nOlmWvYwvW78E/BtmwV8qW1u3Sg58PcM9HOw6yIzEGTK/8dzCXNb8wxrq6+vJzs4mP99/Cd833Ccr\n9MGKifGKd7wmHovbInMA3N62nSR90qhxbHnGPJZkLWFD9QaGXEP48NFsbabZ2szxvuOszFnJ3s69\nUstjVtIszgyckU1K5kTnUJFQId0OFysWnI2pVWp5vOxx9nbulSY8P2/6XJYSZPPY+F/f/C9yjDnM\nSZkz6rSjIAgsyVrCkqwl9Az38PLpl6UNyHAhDeOxpXmLZPiVok/hqYqnLtoX/kQwaoysyV/DluYt\niIgsz14+5uZphNGJpNKPwxs1b8hkW3fn3x0yAfZ27duyXMni2OIxhzC+LXyiD6fXOWk70JFuewUx\nBdyRdwcen4dkfXLIRuJ/nPyPEKnebdm3kRGVweGew+hUOhZnLg4rLxtyDfG7qt9dkAHVrKRZISEQ\n0xOmUzVQJbsvVZ/K/LT5Ml/vZksz/3X2v2TH/WjGjxBFkVpTLbHaWCoTKvmy9Uv2de6THfd42ePS\nqrbD1sH66vU4vA6UgpJFGYu4JfOWcc/d6XXyyulXRtWWh/vdCke7rZ1trdsQRZFbs26VWciOh8fn\n4deHfy27797CeyV3xAhXDhNJpY/4MY5D/7C8WI28hA8QrPG9XJacCkEx6eLdYG6Que01DDXQN9xH\nqiE1rAJlWdayEH301tatJOuTubfoXlbnrSZaHS257v35zJ/Z1LQJt89NjCaGJ8qfoDx+YqvGcBt4\n6YZ09Er5zxicFxmliuLZac+GhDIYVPLLdIWgQKfUkaRPYkH6AmkFnBMdWhBHRs5lRGXw8zk/5+dz\nfs7/uO5/hC3eVrc15IpDq9SOWehHavzH4kjPERqGGmi0NPJ69euTUoMoBEVIe+9yKkCcXic1gzW0\nWlrHPzhCCN/tbd8JUBpfytfd/pFrASFsn/q61OtQK9W0Wvw98HBpLpeSqt4q6obqiNfGszBj4aj9\n+IahBtbXrA+5fyzpYHlCOeuK1vFe/Xuy+4ODDQ52HZQc+FqsLSDAqtxVpBpSWZGzghpTTdjkdwGB\newruweQykWZI4936d6V2gUahkVaKgZBmOJ+TmaRLIk4b53c/DFLOeH1evun+xm/B6xlGrVBzR94d\nMgVHgNL4UtSCGrd4fvpzZKhF4DMK/kIIcLz3uKTHLogp4KGShySFRa4xF61CKzkqjiRJN7ZCKMBI\nGwKv6KXeXB82CzUcCkHBXfl38VHDR3hED7OSZlEcO/W9lwvB4XHw6tlXJT/0RRmLuDXr1styLlcr\nkQI+Drfn3E6iLpEBxwBl8WWjSqdmJc1iVtKFm+u0t7ezf/9+DAYDy5YtC9F3j4XL5eI3//EbGmsb\nUUYriV8ej8llYm1BeEOj/Z2hwzVl8WUUxhZicpqoNdUSrY6W9X8DxxTEFEj93DnJc0IUNE1DTbLb\nXbbzTn2x2ljWFqzlq1b/qlaBQjKXujXrVqYnTZeOfXH6ixzoPIAg+FPVDWoDFQkV7GjfEdKK6XP0\n0efo463at3iy/EnZ1ciujl2yYZ6MqAxmJs0M+7mAX64ZMPQCJlXkNjVvkr6cGoYaeLv2bZZmLyUj\nKoMYTQxPVz7Noa5DDDgHcHgduLwucmNywzoGhiNRl4jdapfdngzTEqdRFl+G2+eeUvpOi6WFnuEe\nco25FzRmf3bwrCzMYk/nHhZlLooENUyCSAEfB4Wg4PrU6y/Je/X39/PP//zPOBz+wlRTU8NPfvKT\nCT9/+/btNNb6pzG9Vi/m/Waa0poAv0RwZ/tO+ob7KIkrYW7K3BAFhl6p53tF38PsMvPy6Zcl1cQN\nqTewMvf8/rRSUPL90u/TYmlBKShDerBNQ02SRjtAdnQ2HzZ8SIetg1xjLityVjA9cTo1gzWYnCai\n1FEk6ZNIM8jHDuK18X6flxEYNUaeLH+SP535U1iPlN7hXk72n5R5RY8MRQZkKUThWJO/hrdq32LA\nOUB+TD7LspdxeuA0bdY2sqKzRt1wFEUxZILz3NA5Ws628GzlsyTrk0nWJ3NH/h2jvrfH56Hf0U+0\nOjrsFcK6wnV80viJf1gpcfqUrF5VCtWUdNfHe4/zUeNH/tcQVDxW9hjZxuxJvw6Etm7UCnWkeE+S\nSAG/gqirq5OKN/gLuMvlQqOZWI+yu1ueBuOz+aSCuLl5M0d7/X5jNaYaf1alMV/mYFgQU4AgCFQP\nVsskb8d6j8kKOPiL+MgBEbPLTL2pHq1SS82gvD2SoE3A5XNJie69w720WlrJi8mT2lNR6iierXx2\n1J/N5XVxvO84XtHLzMSZmFymMQ2uglsoKfoUWa84XKHotneztWUrLp+LhekL+fHMH+MVvSgFJUd7\nj/JJ4/mhIkeeI8TcC/wbT8uylrGlZYvsfrfPTZOladwV67BnmNeqX6Pb3o1KUHFf0X0ho/Nx2jge\nLfNbGAy5hvhL3V+wuq3MTp59Qe07t8/Nsd5juH1uZibNDLsRPTJf1CN6ON53fMoFvDy+nMqESk4P\nnEalUHFX/l1TPvfvKpECfgWRmpoa2HkGICEhYcLFG8Bmk/dpdToddxfcDSAbnAjcXpq1VNJSZ0Rl\nSCvd4D/ckatAURQ5fPgwQ0NDzJ49m4SEBAadg/zx1B+llkawn7RKoQqJGuse7pbdZ3PbqB6sDpuw\n4hN9bKjdILkUHu45TEV8RchxgZDjHGNOSHvkjrw7qDXVSl9MwcNJXp+XDTUbpKuSd+vf5YVpL0iT\nq4Ep0wA1ppqwBRxgftp8imKL2FCzAZPrvP58Ij3uI71HpC9Vj+ifLwgu4CN5t+5d2m1+X98Wawtx\n2riwk5fjIYoib9W+JYVPf9P9DT+Y9oOQFkvwFcFU9Ntd9i6ah5pJNaRyX9F9rHKvQqPUTGh2IoKc\nSAG/gsjPz+fhhx9m+/btGAwGHnxw/FHrkURHywtvQXqB9AeYFZUl6zdmRWWhVWrDjnNXJlTSZGny\nu86pjawrXCc99tZbb7F7924ANm/ezC9/+UtOO07L+tHBG5o9wz0Tcq2LUoW2C8C/uh8ZX9bv6JdW\n7gEMKgM/nPFDhj3DssBbURTZ27mXOnOd7Kri9MBpljiWSP1ju8cuayn5RB99jj6pgAf7tIzXd07S\nJ/FI2SN83vQ5NreNuSlzyY/Jp83ahsfnIceYE97iNUhyO95Yf3D2Zbe9e0oF3OK2SMUbwOQy0Wpt\nDRnaWZGzApPTJCUdLUxfOKn3abY0s756vTQDcGfenaN+EUYYn0gBv8JYuHAhCxdO7o8iwMqVKzl9\n+jQmk4moqCjWrDk/Jr0qbxU6lU7qgY+MDQtGEARW561mdd7qkMf27z+/8WmxWKiqqiK6fPzibHVb\nUaHCgyfs43OSRx9k0Sv1qBQqmXtewKBJgYL8mHzWFKzBoDKEKEO+7v5a8nYJxuKySIU4Sh0lS5TX\nKXUyZcetmbf6vcytbWRFZXFrplwtYXVb8fg8MjOyRF2i1OoA+LTxU470HgH80tOHSh4KKeJzkudw\nvO84/Y5+FIJi3DzK/Jh8qb+vEBQc7T3KtrZtFMQUsK5w3YQlgjqlTpatCsgsbQMk6BJ4YfoL+ETf\nlPrVJ/pOyAa4jvUeixTwCyBSwK8hdu/ejcViQa/X89BDD5GTc35z0ew0UxJXwuLMxaNO3bndbtTq\nsS9jY2JiGBg4P74dGxvL9OTpnOg7IVNtBCMgjFq8E7QJY3py6FQ67iu8j03Nm3B4HTKjKB8+vlf8\nvVELVaC9EI7Pmz5nTf4anD4nWdFZPFr2KHs79+Lyurg+9XpZAdMoNWOqeba2bgX8aqRA2yqA0+uk\n1doqFW/wb6q2WFpCRt6j1FH8oPIH9Az3YFQbxzX8uq/oPvZ07MHqtjLgGJD+D2pMNezt3MuSrCVj\nPj+AWqGmPL6c0wOnpcnP4A3lkUx1s3Gs9lyEyRMp4NcINTU1bN3qLyLDw8OsX7+e2bNno1QqOdJz\nhM+aPpOyBJ8qf0rWu6wfqOePf/gjQ81DGGON/O2P/5asrKyw7/Pss8/y6quvMjQ0xMKFC5k1yy+d\nfLLiSQ50HuCLVnkEWYw6Bh8+WV5nAL1KT5IuiQeKHxj35yuNL6U0vhSr28ofTv1Ber3SuNIxV5m5\nxlyq+qvCPtbr6OXVs68C/i+Rpyqe4vacsT3Zg7F77FLxBjjed5zZybMluWm7tZ0NNRukuLWRjKYC\n0Sg1ZEWH//yD0Sq10ir9lTOvyB4LVhmNxTc930jWxoh+24MLxSt68fg8sgXDwvSFdNo6aRhqIEWf\nErI5HmFyRAr4NYLVKi+QTqcTt9uNUqlkR/sOqZfa7+jneN9xFqQv8B/ndfLKh68w1Ow3XbKYLbz+\nxuv8wy/+Iez75Ofn8+tf/zrsY8HyOY1Cw09m/4QDXQdk2ZJqhZrnpj1Hoi6R3uFe7B77hFdi0epo\nnql4hqqBKtosbbRZ2/j9qd9zV/5dYYdZ5qXMwyf6aBxqJD0qnX0d+8IO0Qw4BzjZf3LSHjLhhpEC\naekd1g5ePfNq2GCIeSnzJlykJ8qspFm0Wf1xdQpBwYzE0dtkwQQyMANciNc3+IeNNp7bKCla7s6/\nG0EQ0Cg1l8Vm4lolIrq8RigvLycl5XwM1vz589Hp/KZLwZI6peL8bZvbhsshN/y32kJXyxOhJK5E\npiSYnugfyCmPL5f1pq9LuY5EXSKfN33Of1b9J/9Z9Z+8tuc1Tpw4IZNRjkasNpYMQwbVpmqsHivd\n9m7erHlz1PT0PGMeaoWafkc/S7KWSCqZcGqZYAadgzIb2GCi1dEy5UxhTCG5MblY3VZeq34tpHgn\n6hL5yayfcF3KdXzc8DGfNX2G2TnByLlxmJcyj0dKH+G27Nt4uuLpSW1m5hnzZLc77Z1sad4S/uAJ\n8FHDR5LM80TfibDWvBEunMgK/BrBYDDw0ksvcfz4cfR6vdTaAP8I+/vn3sftc5Mdnc3spPNa4VhN\nLNnTsqmprkF0+VfpS5ZMrG8aTLI+macrnubs4FliNDHSZGqcNo4fTPsBdaY6jGojpfGl9A33SZpi\nyxELB44f4AAHSE9P57/9t/+GXj/2hOBIeR74nf3+7cS/cV/RfVQkVGBxWbB77ESro3mt+jVJgdKg\nbuBvZ/4tIiJ/PvtnWXEOxNMF2NS0SZreXJC2YFSHyZW5K5mZNFP6fBWCglZra9gknBR9CkpByWtn\nX5PaKg3mBl6Y/sJFCTQojC2ckhfPzKSZeH1ePmv6TPrSOdh9kJK4EgpiCyb1Wj7RF/KzBwdcRLg4\nRAr4ZeDgwYN0dHRQXl5Oefnk7UABmpubcbvdFBQUoFD4L6SioqK46aabQo5tOtREz6Ye1Bo18x6Z\nJ+sZKxVKfnDTD9ieuJ2u5i7mFcxjVvnULQFSDamkGlJD7o/VxDI7aTZbWrbwZeuXkn2oKIpYT55f\n8Xd2dnL06NGwP4fD66DB3IBepacgpgC9Ui/rLYuIbG3ZitPr5NOmT/GJ/kGmkfJBi9uCzWMj1ZAa\nErbc7+iXil+PvUc2er+/az/zUuaFyAkDBLdv4rXxYdN95qXMo9veLTvvAecAQ66hUV/7UjEtcVpI\nAlKwh/lEUAgK5qfOl3JQE7QJlMaNrmWPMHUiBXwcvF4v33zzDW63m7lz52IwXJjx/KZNm/j4448B\n2Lp1Kz/84Q+ZNm1yVp7vvvsu27f7k3OmTZvGiy++KBXxYJqamvjgA38QrcPh4JVXXuE3/99vUKvV\nmJwm6s31xGpiWT1tNXzLjqL7uvZJq+4+Rx9JuiT6HH0IKkFa/QNs2LABp9MpuxIY9gzzyplXJCvb\nBWkLeLryaT5t+JQma5N0nMllknmRdNm7ZF7lBpVBUnbkx+RLVsHBlgDhLG+D+90NQw102jrJNeaG\n9LPTDGmsyV/D3s69eHweco25zE2ZS64xF7PLjFqhlloMUeoojBrjJD7JbweNUsPspNkc6zsG+Ns9\nU3XWvC3nNorjirF77P4v2yl4rkQYn0gBHwNRFPn9739PVZVfxbBt2zZeeuklqbc8EQ4dOsRXX32F\nwWDg/vvv59ixY7LXP3HixKgFXBRFzp49i8vlorKyErVajdlsloo3wKlTp6irq6O0NPwKZ9As79+6\nnC4sNgvo4E9n/iSF2t6SccuEJWdTJdiaV6/S8+L0Fzn1/VNs3LARt9tf0Hw+H3/5y1+YMWMGSUl/\nnYQ01ch8yA90HaCqvwqL2xKy0g0Oirgx7Ua6h7sRELg161YpkOG+wvt4vfp1v2RPY5Tsa+0eOx83\nfix7jdnJs2V5osd6j0nHCAg8VPKQ5FTZZe/C5rZRkVARdrQ9VhPLQyUPsadjD0pBydLspdLegcfn\nuazZkGvy11AWX4bD66AkruSCCu9UBooiTI5IAR8Ds9ksFW/wX97X19dPeMXc0tLCn//8Z2m67re/\n/S15eXm0tJzXSycnh3pj1JvrOdxzmNottbSf8uuY8/Pz+fu//3uUSqVs3B5AqfRvSra3t/OnP/2J\ngYEB5s2bx8MPP0xSdhLKGCXeIX9R02ZrsSltNA80M+wZxmPyYNpjYqN9I5ZbLNx117fnR1ESX3Je\nqoZ/0zNZn8yt82+lIL2A//2//7f0mCiK2O3nL9/DadcDMrngNkVhTCH15npERDKjMrk54+YQqaHb\n5+bzps8l9YXJaeKDcx/w/PTnOdV/SvZloVaoQ3w6Ar4ugfd/u/ZtMqIzKIwplGx0k/XJIa6IAfJj\n8smJzuFY3zHqTHUMOgb5vPlzrG4rZfFl3Fd4n2yz+VIhCMKYo/sRriwiBXwMdDodarVaWhmCf5Bl\nonR1dckKbX9/Pz/5yU9wu910dHRQUVHB0qVLZc/ptnfzVu1buO1uek71SPc3NjZSW1tLZWUld999\nNx999BGiKHLTTTdRVFSE3W7nlVdeobPTX5D27dtHYWEhM6+fSfKaZOzn7AhqAUOhgSh1lKQDH9wx\niGfA317YtGkT2dnZzJkzZ/If1gSoTKhEWaykcaiRNEOabHWanZ1NaWkpNTV+z5Hi4mKik6I51H0I\no9pIWXwZM5NmcqLvBGqFGo1CI4tgi1ZHM+wZpjC2kHsL78XqtmL32EkzpIVd0f6l7i8hAQqB/Mpg\nT45wXx7BLQ8fPtqsbXRYz8vveod7OdV/iutSw+devnfuPcnbe+RVRPVgNUf7jo6blxkhQqSAj4FO\np+Opp57ijTfewO12c8cdd8imG8ejsLAQnU4nSeMKCwtJSkrixRdfHPU5nbZOfKIPQSn4o6JHLC4D\nrZsVK1Zw44034vF4SExM5MSJE/zpT3+SfdEAmEwmotXR3FtxL5ujNuMTfSzPXk6CLoE4bRxNQ01s\ntW6VPae3N3zc18WiLL4sxEgKQKFQ8OMf/5hjx44hiiIFFQW8Uv2KtIk2P3U+awvWsip3FSqFikPd\nhyRtuU6p4/Gyx2UtjgRlAgmE3xT0+Dxh028CBljTE6dzeuA09eZ61Ao1d+bdGXLs7Tm3M+QaotXa\nGlYLHmC0dojb55YFMwRfRQRaWxEijMUFFXBBEP4fsBpwAeeAJ0RRvDii1iuE2bNnM3v21Cw6ExMT\n+elPf8revXvR6/Xcfvv4U37pUen+MWUNxN4Uy9D+IUSfyLJlyygsPL+hFBt7fsT6nXfeCSneOp1O\nWklPT5wuabIDKAQF9xTeg/U6K/v2+fMfNRoN06efP87hcUzZN3okXfYu3D43mVGZY45gq1QqrrvO\nv+o82HVQpoA42nuUFbkrpNXwjWk3khGVwaBjkLyYPJkHyXioFCpiNbEyFcqijEUszlwM+MfvZybN\nZHnWcuJ18WEnPaPV0TxR/gSNQ428UfOGVMRnJc/iZN9JPKKHwpjCkM9dOgdBRZQqKmyQs0FlGPV5\nESKM5IJCjQVBWA5sE0XRJwjC/wEQRfGloGOu6lDjy0GdqY7DPYfRq/QsSl2EXqkfc+P0Zz/7GUND\nQ9LtmTNnsm7dOlJTQ+V8wfh8Pvbu3YvJZGLOnDlkZWUhiiKfNH7Csb5jKAUldxfcPeWCMjIBPUGb\nMGG986n+U7x/7n3pdrw2nr+Z+TejHm9z2+hz9JGoS5yQ82GnrZPPmj5j2DPM9anXMz9tPv2Oft6p\ne0dybYxWR/NM5TMhaUPBdNg6aBpqIsWQQlFskd+vxeMkRhODIIyeSdtqaeXjxo9xeB3ckHoD2dHZ\nmF1m8mPywxpJRfhuMZFQ44uWSi8IwlpgnSiKDwfdHyngE6Cnp4e9e/ei0+lYsmTJpJQue/fuZcOG\nDYiiSEpKCvPnz6exsZGsrCzuvPNOaZNzotSZ6niz9k3ptkqh4pdzfzlpAyOb28b/O/b/ZPfNS5kX\n1uUwGFEU+bz5c471HsOoNnJv0b2jjp532jp5vfp1HF4HOqWOR8senXBGZACf6OPfT/57yDTnbdm3\nSbYDESJcSiZSwC9mD/xJ4O2L+HrfGcxmM//3//5fyc/k9OnT/OxnP5vw8xcuXEhxcTEmk4mTJ0/y\nySf+YYyqqio8Hg/33nvvpM4neGrO4/PgFb2TLuDhVp8DDr+TodfrZefOnfT39zN79myKi4tDnjua\npW0wezr3SLpth9fBno49fK/4e5M612HPcNhRfJ1q4l+k3xbbWrdxsv8ksdpY7sq/a9IZmMF027tx\nep1kRmVeFqVLhIvHuAVcEIQvgXC+kr8URfHTvx7zD4BLFMW3wr3Gr371K+nfixcvZvHixVM512uW\nc+fOycyo6uvrsdlsREVN3GozNTWVoaEhtm2Te183NDSM+1yv18vGjRupq6sjJyeHu++9W+aNfUPq\nDVNKSzGoDOQZ82iyNEn3lcf7J0/ffvtt9uzZA8COHTv4+c9/Tn7+1HTDiiBLn6lYnRrwLnigAAAT\nJ0lEQVRUBpL1ybLQi6KYojGDjy8Fp/pPsafT/zmZXWY2nts4ZvTceOxs38nO9p0A5BhzeLT00cuq\nO49wnp07d7Jz585JPWfc/zlRFJeP9bggCI8Dq4Clox0zsoBHCCU5OVmm7TYajeN6gYTjm2++CUl0\nKSgY38diy5YtUuFvaWlBrVbz1H1P0TDUgE6pC/GsngyPlj3KtrZtdNu7KYkrkSR1J0+elI7x+Xyc\nOnVqygV8ceZimi3NWNwWotXR0mbkZBAEgUdLH2VXxy6cHidzU+Ze0M99sQg20hrLWGs8AsHWAVos\nLdSZ66Qv1QiXl+DF7T/+4z+O+5wLVaGsAH4GLBJFcXwbuQhhyc7O5tFHH2Xr1q1otVoefPDBUUfj\nxyI+Pl52OyUlhbVrw4cQjKS9XR560NHRgUapCSv3mywKQcHy7NA1QGpqKmbzeRVIWtro4QHjkaRP\n4kczfoTJZSJOEzfhFJpgjBrjhFo2l5LiuGJ2te+SrADCZYFOFAEBhaCQyR6DnSojXF1cqAqlDtAA\ngYiWA6IovhB0TGQT8xLhdrt5/fXXOX36NOnp6TzzzDMhRT0c+/btY/369dLttWvXsmLFim/zVBkc\nHGTDhg309/czd+5c7rwzVGs9EpPTxNfdXyMgcGPajVeEd8ilotPWKTk8zkmeM+U0HIBD3YfY3LwZ\nEZGy+DLuL7r/gl4vwrfHJVWhjHESkQJ+FfD1119TW1tLbm4ut9xyyyV/f6/Py4cNH1I9WE2CLoH7\ni+6XBnOcXie/q/qdpNtO0Cbw/PTnIynmU8TqtuLyuvyOiWPIHC8noihSZ67D5XVRElcy5auqq5lI\nAb+EHDx4kOrqarKzs7n11lun1AL5LnOw6yBbWs4HCOQYc3iy/EnAr5cORJ8FeH7a82FtayNcG3xw\n7gNO9vv3SdIMaTxZ/uR3rohfahnht8bw8DAffPABfX19zJkzh5tvvvlyn5KMAwcO8Nprr0n/ttvt\n47YEIsgJzsy0us7fjtPGyVLptUrtJW+h2D12uuxdJGgTJjX1GWHy2Nw2qXiDf5K3ydJESVzJZTyr\nK5OrooCvX7+eo0ePAnDmzBmMRqMsceZyc/bs2ZDbkQI+OaYlTuPr7q8lj+yRRldGjZEHih9gR9sO\nBASWZS+TRbR92ww4Bvjz2T9jdVtRCSq+V/w9yTo2wsVHrVCjFJQyW+CABXAEOVdFAW9sbJTdbmpq\nuqIKeGZmpuz2aInuEfx88MEH7Nmzh5iYGFavXo3RaKSgoIBnK5/lnPkcCbqEkNVWUWwRRbFFl+V8\nv+7+WrpC8IgedrbvjBTwbxGNUsNd+XfxSeMneEQPC9IWyMI2IpznqijgxcXFHDp0Pt6qqOjy/CGP\nxvLly7Hb7dTU1JCdnT3pycfvEidOnOCLL/wuggELXPB/Cf7sZz9jftr8y3l6YRkrFDrCt8OMpBlU\nJlbiE32RzeoxuCoK+COPPEJcXBx9fX3Mnj170hFk3zYKhWJCeuuLhSiKtLW1oVarL0g/faF0dHTw\n2Wef4fV6WbVqFbm5ueM+x2QKnxzf3t7O4cOHr7j9DYAF6QuoNdXS5+hDr9RzW3b4cOMIFxeloIzo\n1MfhqijgGo2GdevWXe7T+NapqqqiqqqK9PR0Fi1aFFbJ4vP5+OMf/8jx48cBWLlyJXffffelPlUc\nDgf/8i//Irkg1tbW8j//5//EaBx7c3HatGlERUVhs4XaqKpUV+avY7Q6muemPYfZZSZaHR024CFC\nhMvBlfkX8x2kqqqK3/72t9LtwcFB7rnnnpDj6urqpOINsHnzZpYuXTpu4bzYDAwMyCxs7XY73d3d\n455HYmIiv/jFLzh27Bj9/f3s3bsXj8dDRUWF5AV+JaJSqCZkIuXwOPCJPinxKIDJaaJ3uJdUQ2rE\nKjbCRSNSwK8QRmZvgt+RMFwBDzd4cTmGMRITE4mLi5NaIlFRURNu5yQnJ3Pbbf42xF133YXdbich\nIeGq184f6DrA1patiIjMT53Pilz/NGvTUBNv1r6J2+dGq9TyaOmjZEZnjvNqoTQONXKq/xSx2lgW\npC2ImFBFiBTwi4HP5+PIkSO4XC7mzJkzJSOq9PR02e3RimFxcTFz587lyJEjAKxevZro6PEDDC42\nWq2Wv/u7v+Pzzz/H6/WycuXKKZ2HwWDAYLh0ksBvC5vbJhVvgIPdB5meOJ3M6Ez2de6T5JFOr5MD\nXQe4t2hyG91t1jZZ8k/vcC/rCq/9tmKEsYkU8IvAyy+/zLFjxwD48ssveemllyYVyACwaNEiTCYT\np06dIi0tjYceeijscYIg8Mwzz7B69WrUanXYVPuLgdfr1+COFQaRlpbGU0899a28/9WGx+cJybUM\nFO3glfJUVBUNQw0yE6pz5nNTOMsI1xqRAn6BDA0NScUboLOzk7q6Oilb0mazMTw8TGJi4pitjoCS\nZSJqFkEQyMiYXOLMZPjqq6/YuHEjAPfeey9Ll47qFBzhr8RqY5mZNJMTfScAyI/JJ9uYDcDSrKW0\n29oZcg2RoE1gUeYi2XNFUWRLyxZODZwiThPHPYX3hPTbU/QpstvJ+m/nizvC1UWkgF8gOp0OtVot\nCxUObOQdPHiQ9evX4/V6mT59Os8///yk480uNX19fbz//vuSr/h7773HrFmzSEy8sBSY7wJrC9Yy\nO2k2HtFDfky+JIFL0ifx4xk/xuaxEa2ODpHGHe87ztfdXwP+VsyHDR/ydMXTsmPK4su4Ped2qvqr\niNHEsCp31aX5oSJc0UQK+AWi0Wh48skneeONN3C5XKxatYq8vDxEUeStt96SWhFVVVUcPXr0ilZa\ngF9NMtJ8TBRFhoeHL+MZXV2MFgKhUqhGDUcOuCwGMDlNbG/bjlf0Mi9lHvFavyXwjWk3cmPajRf1\nfCNc3UQK+EVgzpw5zJkzB1EUpTaJKIpS8Q7g8Xgu2nu2tbXR3NxMXl5eyCj/ZHG5XDgcDmJiYsjK\nyqKsrIzq6moAysvLv9V2zdWK2+dmd8duTE4TlQmVFxR+URZfxt6OvVJog0/0sbtjNwAn+07y/PTn\nL6n3S4Srh4id7LfIli1b+PDDDwHIycnhpz/9KVrthQ+BVFVV8bvf/Q6fz4dSqeSHP/whFRVTS2o5\ndeoUL7/8Mk6nk4qKCl544QUEQZAiz2bOnHnFt30uB+/Xv8+pgVPS7cfKHiM/ZmqRcOB33KsZrEGt\nULO1davssUdLH6UgdvxovAjXFteMneyVhN1uZ9euXfh8Pm6++WZiYkYfylixYgXTp0/HarVSUFCA\nWn1xPB327NmDz+dXJHi9Xnbt2jXlAv7mm2/idPpT6M+cOcPBgwe5+eabmTNnzkU514nSNNTEp02f\n4vQ6WZi+8Ir0RBnJyKBmgGZL8wUV8DRDGmmGNFxeF7s7duPw+hMKFYIiYl8bYVQiBXwSeL1efvOb\n39Da2gr4vb//+3//72NKBi+0vRGOYN30hejAA8U7gMvlmvJrTRWvz8s7de9IRWtLyxayo7OnNOxy\nqUg3pFNnrpNupxkujieNRqnhoZKH+KLlCzyih8UZi0nQJVyU145w7THl0TdBEP5JEIQTgiAcFwRh\nmyAI2RfzxK5Eent7peIduN3W1nbJz2Pt2rXk5PjtNXNzc7nrrrum/FqrVp1XMyQlJXH99ddf8PlN\nFqfPKRXvAMEbe1caawvWMjNpJrnGXO7IveOiBEAHyDHm8EzlMzw/7XnKEyKJ8RFGZ8o9cEEQjKIo\nWv767x8BM0VRfDrMcddMD9xms/GLX/xCWrUqlUr+6Z/+6bJJ7Dwez0UxgGppacFsNlNUVDSlKdKL\nwYaaDdSb6wEwqo08N+05otRRl+VcIkS4ErhkmZiCIPwCiBVF8aUwj10zBRz8HiXvvfceXq+Xu+++\nm7lz517uU7om8Pg8HO09isvrYkbSjIjhU4TvPN96ARcE4X8BjwB2YL4oiiFmz9daAb9Y9PT08MYb\nb2A2m1mwYAErVqy43Kd01eETfSiEq9sAK0KE0bjgAi4IwpdAuN2ZX4qi+OmI414CSkVRfCLMa0QK\neBh+/etfy/rpL774IjNmzLiMZ3TxEUWRjRs3cvToUZKTk3nsscdISLjwDbl6Uz0bz23E4XVwXep1\nkanECNckFywjFEVx+QTf6y1g02gP/upXv5L+vXjxYhYvXjzBl7126enpGfP2tcD+/fv58ssvAejv\n7+f111/n7/7u70KOE0WR/v5+dDrduIoaURR5/9z70qbnoe5DFMUWRRLLI1z17Ny5k507d07qOVPe\nARMEoVgUxYCO6i7g2GjHjizgEfzMnDlTyvlUq9VT1nFfyfT29o55G/zSzN///+3db4hVeR3H8fdn\nhsZtWFqNoaxWHAdUbBd3XHPJpgcusiFLKIKyBVLWsyhcYu3f+iAkwiioJGmeZEHEWlAyJFusOrtD\nQaAYzjibs7uW3XIWWxEXTTIbvd8enLPDODN37tW5c3/3OJ8XDMy598zlwx3Ol9855/f9nd5ehoeH\naWlpYceOHfT09FT8zHKUuXn7zqmPN2651d+Kb/Lgdu/evVX/ZjZTGPZJWgncBv4GfGEWnzXv7Ny5\nk6VLl3L16lXWrVt3X7arr169mqNHj44vKbBmzZop+wwNDY0/zKJcLnPo0CHWr19f8eEOrS2trH3f\nWk5dOgXAwgULWf6QnxBv85Nb6W1OnT9/njNnztDR0UFPT8+UJXVPnjzJwYMHx7dbWlo4cOBA1fb9\n199+nRu3brBi4Yopjy8zux+4ld6S6+rqoqur8joe3d3ddHZ2UiqVANi8eXNNa6+sXLSyXhHNCssj\ncEtubGyMUqlEe3v7nCw9YFZEDWvkqRLCBdzM7C7VUsDdBWFmVlAu4GZmBeUCbmZWUC7g1nDXrl2j\nVCpNWYvczO6OpxFaQ42MjNDb28vNmzfp6Ohg9+7dLFq0aPz9sbExBgcHkUR3d3ddlss1u1/56LCG\n6uvrGx95X758mf7+frZt2wZkbfX79+/n3LlshYZVq1axa9euil2ZZvOdjwxrqMmdmBO3R0dHx4s3\nZKP1ixcvNiybWdG4gFtDbd26lQULFgDZI9w2btw4/l57e/sdBV1SsicEmRWBG3ms4a5fv86VK1dY\nvHgxbW1td7x3/PhxDh8+jCS2b9/upYdt3nInphVSuVwG8LVvm9e8mJUVkgu3WW18pJiZFZQLuJlZ\nQbmAm5kVlAu4mVlBuYCbmRXUrAu4pOcklSW9tx6BzMysNrMq4JKWAE8B/6hPnHQGBgZSR6iJc9ZX\nEXIWISM4ZwqzHYF/H/hqPYKkVpR/qnPWVxFyFiEjOGcK91zAJW0BRiPiTB3zmJlZjWbsxJR0DFg8\nzVt7gG8An5i4ex1zmZlZFfe0FoqkR4F+4D/5Sw8DbwJPRMSlSft6IRQzs3vQkMWsJP0dWBsRV2b9\nYWZmVpN6zQP3KNvMrMHmfDlZMzObGw3txGz2ph9J35I0JGlQUn8+z73pSPqepJE862FJD6XONJmk\n7ZL+Ium2pMdT55lM0iZJr0k6J+lrqfNMR9JPJb0laTh1lplIWiLplfz//aqkXakzTUfSA5JO5Mf3\nWUn7UmeqRFKrpNOSjsy0X8MKeEGafr4bEY9FRDfQB3wzdaAKjgKPRMRjwBtkM4KazTCwFfhD6iCT\nSWoFDgCbgA8Dn5a0Km2qaf2MLGOzGwO+HBGPAB8FvtiM32dE/Bd4Mj++VwNPSvp44liVPAucpcrl\n6UaOwJu+6Sci/j1h80HgcqosM4mIYxFRzjdPkM0CaioR8VpEvJE6RwVPAH+NiFJEjAG/BLYkzjRF\nRPwReDt1jmoi4l8RMZj/fh0YAT6YNtX0IuKdmXNtQCvQdBMvJD0MPA38hCrTsxtSwIvU9CPp25L+\nCXwW+E7qPDX4PPC71CEK5kPAhQnbo/lrNkuSOoE1ZAOLpiOpRdIg8BbwSkScTZ1pGj8AvgKUq+1Y\nt0eqFaXpZ4acz0fEkYjYA+yR9HWyL/JzDQ2Yq5Yz32cP8L+IeKGh4XK1ZGxSvnM/ByQ9CPwaeDYf\niTed/My1O79v9JKkDRExkDjWOEmfBC5FxGlJG6rtX7cCHhFPVQj0KLAMGJIE2en+nyVNafpphEo5\np/ECCUe21XJK2kl2mrWxIYGmcRffZbN5E5h4g3oJ2Sjc7pGkdwG/AX4REX2p81QTEVclvQh8BBhI\nHGeijwGbJT0NPAC8R9LPI+Iz0+0855dQIuLViHh/RCyLiGVkB8rjKYp3NZKWT9jcApxOlWUmkjaR\nnWJtyW/MNLtmW2bhFLBcUqekNuAZ4LeJMxWWspHZQeBsRPwwdZ5KJHVIWpj//m6ySRVNdYxHxPMR\nsSSvlZ8CXq5UvCHNAx2a+fR1n6Th/BrZBuC5xHkq+RHZTdZj+VSjH6cONJmkrZIukM1KeFHS71Nn\nekdE3AK+BLxEdqf/VxExkjbVVJIOAX8CVki6ICnJ5bwa9AA7yGZ1nM5/mnH2zAeAl/Pj+wRwJCL6\nE2eqZsZ66UYeM7OC8iPVzMwKygXczKygXMDNzArKBdzMrKBcwM3MCsoF3MysoFzAzcwKygXczKyg\n/g9QhnVNQgUm0wAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1037355d0>"
       ]
      }
     ],
     "prompt_number": 2
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Quick Exercise #1"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Use a *Support Vector Machine*, and see how well the model can classify this data.  You can use what we did above as a template:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.svm import SVC\n",
      "# First instantiate the \"Support Vector Classifier\" (SVC) model\n",
      "\n",
      "\n",
      "# Next split the data (X and y) into a training and test set\n",
      "\n",
      "\n",
      "# fit the model to the training data\n",
      "\n",
      "\n",
      "# compute y_pred, the predicted labels of the test data\n",
      "\n",
      "\n",
      "# Now that this is finished, we'll compute the classification rate\n",
      "# print(\"accuracy:\", np.sum(ytest == ypred) * 1. / len(y_test))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Did you do well?  Would you say that this is a good classification scheme for the problem?"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "The Problem with Simple Validation"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The problem here is that we might not care how well we can classify the **background**, but might instead be concerned with successfully pulling-out an uncontaminated set of **foreground** sources.  We can get at this by computing statistics such as the **precision**, the **recall**, and the **f1 score**:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.cross_validation import train_test_split\n",
      "from sklearn import metrics\n",
      "from sklearn.svm import SVC\n",
      "\n",
      "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)\n",
      "clf = SVC().fit(X_train, y_train)\n",
      "y_pred = clf.predict(X_test)\n",
      "\n",
      "print(\"accuracy:\", metrics.accuracy_score(y_test, y_pred))\n",
      "print(\"precision:\", metrics.precision_score(y_test, y_pred))\n",
      "print(\"recall:\", metrics.recall_score(y_test, y_pred))\n",
      "print(\"f1 score:\", metrics.f1_score(y_test, y_pred))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "accuracy: 0.968\n",
        "precision: 0.833333333333\n",
        "recall: 0.625\n",
        "f1 score: 0.714285714286\n"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "heading",
     "level": 4,
     "metadata": {},
     "source": [
      "What do these mean?"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "These are ways of taking into account not just the classification results, but the results **relative to the true category**.\n",
      "\n",
      "$$ {\\rm accuracy} \\equiv \\frac{\\rm correct~labels}{\\rm total~samples} $$\n",
      "\n",
      "$$ {\\rm precision} \\equiv \\frac{\\rm true~positives}{\\rm true~positives + false~positives} $$\n",
      "\n",
      "$$ {\\rm recall} \\equiv \\frac{\\rm true~positives}{\\rm true~positives + false~negatives} $$\n",
      "\n",
      "$$ F_1 \\equiv 2 \\frac{\\rm precision \\cdot recall}{\\rm precision + recall} $$\n",
      "\n",
      "The **accuracy**, **precision**, **recall**, and **f1-score** all range from 0 to 1, with 1 being optimal.\n",
      "Here we've used the following definitions:\n",
      "\n",
      "- *True Positives* are those which are labeled ``1`` which are actually ``1``\n",
      "- *False Positives* are those which are labeled ``1`` which are actually ``0``\n",
      "- *True Negatives* are those which are labeled ``0`` which are actually ``0``\n",
      "- *False Negatives* are those which are labeled ``0`` which are actually ``1``\n",
      "\n",
      "\n",
      "We can quickly compute a summary of these statistics using scikit-learn's provided convenience function:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(metrics.classification_report(y_test, y_pred,\n",
      "                                    target_names=['background', 'foreground']))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "             precision    recall  f1-score   support\n",
        "\n",
        " background       0.97      0.99      0.98       234\n",
        " foreground       0.83      0.62      0.71        16\n",
        "\n",
        "avg / total       0.97      0.97      0.97       250\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This tells us that, though the overall correct classification rate is 97%, we only correctly identify 67% of the desired samples, and those that we label as positives are only 83% correct!  This is why you should make sure to carefully choose your metric when validating a model."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Quick Exercise #2"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Compare the *K Neighbors Classifier*, the *Gaussian Naive Bayes Classifier*, and the *Support Vector Machine Classifier* on this toy dataset, using their default parameters.  Which is most accurate?  Which is most precise?  Which has the best recall?\n",
      "\n",
      "Imagine that you are a medical doctor, and here the features are biometric measurements, and the labels are\n",
      "\n",
      "- 0 = no disease\n",
      "- 1 = potential infection\n",
      "\n",
      "For your patients, which classifier might you choose? (That is, would you be more concerned with accuracy, precision, or recall?)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 6
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Cross-Validation"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Using the simple train/test split as above can be useful, but there is a disadvantage: **Your fit is ignoring a portion of your dataset**.  One way to address this is to use cross-validation.\n",
      "\n",
      "The simplest cross-validation scheme involves running two trials, where you split the data into two parts, first training on one, then training on the other:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X1, X2, y1, y2 = train_test_split(X, y, test_size=0.5)\n",
      "print(X1.shape)\n",
      "print(X2.shape)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "(500, 2)\n",
        "(500, 2)\n"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "y2_pred = SVC().fit(X1, y1).predict(X2)\n",
      "y1_pred = SVC().fit(X2, y2).predict(X1)\n",
      "\n",
      "print([metrics.precision_score(y1, y1_pred),\n",
      "       metrics.precision_score(y2, y2_pred)])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[0.6470588235294118, 0.94117647058823528]\n"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This is known as **two-fold** cross-validation, and is a special case of *K*-fold cross validation.\n",
      "\n",
      "Because it's such a common routine, scikit-learn has a K-fold cross-validation scheme built-in:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.cross_validation import cross_val_score\n",
      "\n",
      "# Let's do a 2-fold cross-validation of the SVC estimator\n",
      "print(cross_val_score(SVC(), X, y, cv=2, scoring='precision'))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[ 0.73684211  0.875     ]\n"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It's also possible to use ``sklearn.cross_validation.KFold`` and ``sklearn.cross_validation.StratifiedKFold`` directly, as well as other cross-validation models which you can find in the ``cross_validation`` module."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Quick Exercise #3"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The ``SVC`` classifier takes a parameter ``C`` whose default value is ``1``.  Using 5-fold cross-validation, make a plot of the precision as a function of ``C``, for the ``SVC`` estimator on this dataset.  For best results, use a logarithmic spacing of ``C`` between 0.01 and 100."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 10
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Grid Search"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This exercise you just completed is an example of a **grid search** for model evaluation.  Again, because this is such a common task, Scikit-learn has a grid search tool built-in, which is used as follows.  Note that ``GridSearchCV`` has a ``fit`` method: it is a meta-estimator: an estimator over estimators!"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.grid_search import GridSearchCV\n",
      "\n",
      "clf = SVC()\n",
      "Crange = np.logspace(-2, 2, 40)\n",
      "\n",
      "grid = GridSearchCV(clf, param_grid={'C': Crange},\n",
      "                    scoring='precision', cv=5)\n",
      "grid.fit(X, y)\n",
      "\n",
      "print(\"best parameter choice:\", grid.best_params_)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "best parameter choice: {'C': 0.10608183551394483}\n"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stderr",
       "text": [
        "/Users/jakevdp/anaconda/envs/py3k/lib/python3.3/site-packages/sklearn/metrics/metrics.py:1771: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples.\n",
        "  'precision', 'predicted', average, warn_for)\n"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "scores = [g[1] for g in grid.grid_scores_]\n",
      "plt.semilogx(Crange, scores);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEFCAYAAAAG45eHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFuJJREFUeJzt3XuwXWV5x/HvkwMJBBqQYoMNsbEYgViJF4opom6hloBC\nOmDFcCnlMoZxgtahBRGF0/EyxVGrllbRgnXUIVrLWGihlLHuKUpFaFWwEieBAZOQcEuCAgkk5O0f\n6wDHzTn7utZZa+3z/cycSfZe71774WXnt9/zvusSKSUkScNpRtkFSJKKY8hL0hAz5CVpiBnykjTE\nDHlJGmKGvCQNsY4hHxFXR8SDEXFXmzafi4g1EfGTiHhNviVKkvrVzUj+y8DSyTZGxPHAy1NKC4F3\nA5/PqTZJ0oA6hnxK6RZgS5smJwJfGWt7G7BvRMzNpzxJ0iDymJOfB6wb93g9cGAO+5UkDSivhddo\neey1EiSpAnbLYR8bgPnjHh849tyviQiDX5L6kFJqHUh3LY+R/HXAnwJExBJga0rpwYkappQK/bns\nsssKf22ndu22T7Ztoudbn5vo8atelfiXf0ksWpR473sTO3fan/08N8h/51T0Zy+v67c/8/5sVrUv\ne3ltN+2moj8H1c0hlNcAtwIHR8S6iDg7IlZExIqx4L4BuDci1gJXAu8ZuKo+NRqNwl/bqV277ZNt\nm+j51ucmarNlCyxeDN//Pvz0p3DSSfDEE23L60kd+3OQ54rW73v28rp++zPvz2bRqvDZbNemSv0Z\neXxTdPVGEWmq3ms6GB0d5VOfGmXDBpgzB55+GlasyML++uvhgAPKrrA+RkdHGR0dLbuMoWF/5isi\nSCVP16gERx3VYNs22Hvv7PHMmXD11XDiibBkCfzf/5VbX52UMRIdZvZntTiSr6lHHoGDD4ZHH33h\ntq99DS64AK65Bo4+euprk5SfQUfyeRxdoxJs3Qr77jvxttNPh/nz4Z3vhAsvhJe/vP2+3vhG2G+/\n/GuUVD5H8jV1++1w3nnwP/8zeZvVq+HSS+GppyZv8/DD8Fu/Bd/+dv41ShqcI/lpautWeNGL2rc5\n5BD45jfbt9m2LRvp33EHHH54fvVJqgYXXmtqy5bJp2t6seeecMkl2Yhf0vAx5Guqm5F8t845Jzsa\n59Zb89mfpOow5Guq3cJrr2bNgg9/OPuRNFwM+ZrKa7rmWWeeCfffD81mfvuUVD5DvqbynK4B2H13\nuOyybDTvQVDS8DDkayrvkTzAqadmJ1ndfHO++5VUHkO+pvIeyQOMjMBf/RV86EOO5qVhYcjXVJ4L\nr+O94x3ZyVP/+q/571vS1DPka6qI6RqAGTOy0fyll8KuXfnvX9LUMuRrqojpmmctW5ZN3Vx7bTH7\nlzR1vHZNDaWUHdv+y1/CHnsU8x433gh/8Rdw551Z4Esqh9eTn4a2bcuCt6iAB1i6FPbZB77xjeLe\nQ1LxDPkaKmrRdbwI+MhHYHQUdu4s9r0kFceQr6EtW4qbjx/v6KNh3jz46leLfy9JxfBSwzU0FSN5\neH40f+qp2eMlS7K7Uc1waCDVhiFfQ0UdPjmRo47Kgv4//gM++tHsdoNHHAGvf/3zPy9+8dTUIql3\njslqqMjDJydy5pnw9a/DPffAmjXw3vdmz3/2s7BwIRx0EKxcCffdN3U1SeqOIV9DUzVdM5EXvxje\n/vZsdH/zzbB5M1x/PcyZA697HZx9dvZFIKkaDPkamqqF127MmAGLFsHHP56F++/8Dhx5JJx2WnYj\nEknlMuRrqMyRfDv77Zddrviee+BVr8qOzjn5ZPjf/y27Mqmennxy8H248FpDW7bAoYeWXcXk5syB\nD3wAzj8fvvQlOOEEWLwYfu/32r/uoIPg3e/Ojuopyg9/CN/6Vvs2M2ZkX1JvfCO89KXF1TLsUsp+\nu7vhhuznllvg6aeLf98998w+SwsXvvBn7txiP195uu02eNe7Bt+PlzWooZNOyqZDTj657Eq6s317\ndubsgw+2b/f1r2dfCB/9aDF13HZbtv/3vAdmz5683dNPw49+lIXSHntkYX/UUdmfixZ5CGk727Zl\ndxe78cYs2Ldvh+OOg+OPz36z23vv4mt4/HFYuzb7gmn9eeqpLOznzCm+jlmz4M//PPvv70VK8OlP\nw+WXwxe+ACefPNhlDQz5Gjr6aLjkEjjmmLIrydfDD8Ob35x9gV1ySb77/vGP4dhj4eqr4W1v6+41\nz45Eb7kFvve97M/Nm+ENb8hG+l7T53m7dmV9fMst8OpXZ6F+/PFZP1Vp5LxlS/YF8MQTxb/Xpk3Z\nndZe+Ur4zGdgwYLOr3n0UfizP4OHHsoGRgsWDH7tGkO+hl7zGrjqKnjta8uuJH8bN8Kb3pSNtt//\n/nz2+bOfZV+IV1wx+G8/Gzdmgb96dT61DZNXvAL+6I+qc1BAFWzfDp/8ZBby73sf/OVfTn7Nqe99\nLzvx8JRT4GMfg5kzs+cN+WnoZS+D73wHfvd3y66kGL/4RTaiv+giOO+8wfa1Zg285S3Zr76nnZZP\nfVKv7r8/G7TceWd2fsn43yZ37co+n5/9bDZ4a/1N05CfhvbdF+69NzuaZVjdcw80Gtn8/Jln9reP\n++/Pviw+/GE455xcy5P6ctNN2QEJhxySje732gvOOCNby7jmGjjwwBe+xksNTzO7dsGvfpVdBniY\nHXRQdrLVxRfDN7/Z++s3bMimaC64wIBXdRx7LNx1F/zBH2SXB1m8OPvzu9+dOODz4Ei+ZrZuzU44\neuyxsiuZGnfemc3zfvGLcOKJ3b3moYeyEfxZZ8GFFxZbn9Svdeuygw06ra05kp9mqnS261Q47LDs\npuLnnptdJK2TzZvhrW/NFq8MeFXZ/PlTc/CEJ0PVTFXPdi3S4YfDt78Nf/zH2XVz2h2n/oMfZAtX\nl102dfVJVWbI18xUXma4So48Mpujv/329u2OOy47WaxKx2ZLZeoY8hGxFPgMMAL8Q0rp8pbt+wNf\nAw4Y298nU0r/mH+pgqm/zHCVLF6c/UjqXts5+YgYAa4AlgKLgOUR0XrVlJXAj1JKrwYawKciwt8Q\nCjIdp2sk9a/TwusRwNqU0n0ppR3AKmBZS5uNwLNXgpgDPJpS8tbPBZluC6+SBtNpxD0PWDfu8Xrg\n9S1tvgT8Z0Q8APwG8M78ylMrR/KSetEp5Ls5sP2DwI9TSo2IOAi4OSIWp5R+1dpwdHT0ub83Gg0a\njUYPpQqykfzBB5ddhaSiNJtNms1mbvtrezJURCwBRlNKS8ceXwzsGr/4GhE3AB9LKX1/7PF3gItS\nSne07MuToXJw+unZWXNnnFF2JZKmQtEnQ90BLIyIBRExEzgFuK6lzWrgD8eKmQscDNzbb0Fqz+ka\nSb1oO12TUtoZESuBm8gOobwqpXR3RKwY234l8HHgyxHxE7IvjQtTSpsLrnvacuFVUi+8dk3NvPKV\n2c0EOt1KT9Jw8No104wjeUm9MORrxjl5Sb0w5Gvkqadgx472N6GWpPEM+Rp59ro1XnxLUrcM+Rpx\nqkZSrwz5GnHRVVKvDPkacSQvqVeGfI1M1xuGSOqfIV8j0/mGIZL6Y8jXiNM1knplyNeIC6+SemXI\n14gjeUm9MuRrxIVXSb0y5GvEhVdJvTLka8TpGkm9MuRrxIVXSb0y5GvEkbykXnlnqJpICXbfHbZt\ny/6UND14Z6hp4vHHYY89DHhJvTHka8KpGkn9MORrwkVXSf0w5GvCkbykfhjyNeFIXlI/DPmacCQv\nqR+GfE0Y8pL6YcjXhNM1kvphyNeEI3lJ/TDka8KRvKR+GPI14UheUj8M+Zow5CX1w5CvCadrJPXD\nkK8JR/KS+mHI14QjeUn9MORrYOdOePJJ2HvvsiuRVDeGfA089hjssw/M8P+WpB51jI2IWBoRqyNi\nTURcNEmbRkT8KCJ+GhHN3Kuc5pyqkdSv3dptjIgR4ArgD4ENwO0RcV1K6e5xbfYF/g44NqW0PiL2\nL7Lg6chFV0n96jSSPwJYm1K6L6W0A1gFLGtpcyrwzyml9QAppUfyL3N6cyQvqV+dQn4esG7c4/Vj\nz423ENgvIr4bEXdExBl5FihH8pL613a6Bkhd7GN34LXAMcBs4L8j4gcppTWDFqeMIS+pX51CfgMw\nf9zj+WSj+fHWAY+klLYB2yLiv4DFwAtCfnR09Lm/NxoNGo1G7xVPQ07XSNNHs9mk2Wzmtr9IafLB\nekTsBvycbJT+APBDYHnLwushZIuzxwKzgNuAU1JKP2vZV2r3XprcBz8Ie+0Fl1xSdiWSplpEkFKK\nfl/fdiSfUtoZESuBm4AR4KqU0t0RsWJs+5UppdUR8e/AncAu4EutAa/BbNkCBx5YdhWS6qjtSD7X\nN3Ik37fly+GEE+DUU8uuRNJUG3Qk7zmUNbB1q3PykvpjyNfAli0eXSOpP4Z8DXgIpaR+GfI14CGU\nkvplyFdcSo7kJfXPkK+47duzSwzvsUfZlUiqI0O+4lx0lTQIQ77inKqRNAhDvuJcdJU0CEO+4hzJ\nSxqEIV9xnu0qaRCGfMW58CppEIZ8xTldI2kQhnzFufAqaRCGfMU5kpc0CEO+4lx4lTQIQ77iXHiV\nNAhDvuKcrpE0CEO+4lx4lTQIQ77iHMlLGoQ38q6wXbtg993h6adhZKTsaiSVwRt5D7Ff/hL23tuA\nl9Q/Q77CnKqRNChDvsJcdJU0KEO+whzJSxqUIV9hjuQlDcqQrzBH8pIGZchXmNetkTQoQ77CvG6N\npEEZ8hXmdI2kQRnyFebCq6RBGfIV5khe0qAM+Qpz4VXSoAz5CnPhVdKgDPkKc7pG0qAM+Qpz4VXS\noDqGfEQsjYjVEbEmIi5q0+73I2JnRJyUb4nT01NPwY4dMHt22ZVIqrO2IR8RI8AVwFJgEbA8Ig6d\npN3lwL8DfV/cXs977LFsFB/2pqQBdBrJHwGsTSndl1LaAawClk3Q7nzgW8DDOdc3bbnoKikPnUJ+\nHrBu3OP1Y889JyLmkQX/58ee8h5/OXDRVVIeOoV8N4H9GeADYzdwDZyuyYWLrpLysFuH7RuA+eMe\nzycbzY/3OmBVZJPH+wPHRcSOlNJ1rTsbHR197u+NRoNGo9F7xdPEgw/C3LllVyFpqjWbTZrNZm77\ni2wAPsnGiN2AnwPHAA8APwSWp5TunqT9l4HrU0rXTrAttXsv/bq//mvYvBk+8YmyK5FUpoggpdT3\nDEnbkXxKaWdErARuAkaAq1JKd0fEirHtV/b7xmpv40ZYsKDsKiTVXafpGlJKNwI3tjw3YbinlM7K\nqa5pb9MmWLKk7Cok1Z1nvFbUxo3wkpeUXYWkujPkK2rTJkNe0uAM+YrauBEOOKDsKiTVnSFfQY8/\nDs88A3PmlF2JpLoz5Cvo2akar1sjaVCGfAU5VSMpL4Z8BbnoKikvhnwFOZKXlBdDvoI8Rl5SXgz5\nCnK6RlJeDPkKcrpGUl4M+QpyukZSXgz5CnK6RlJe2l5PPtc38nryXdm5E/bcE7Zvh5GRsquRVLZB\nryfvSL5iHnoI9t/fgJeUD0O+Ylx0lZQnQ75iXHSVlCdDvmJcdJWUJ0O+YpyukZQnQ75inK6RlCdD\nvmI2bXIkLyk/hnzFOJKXlCdDvmJceJWUJ894rZCUYPZsePTR7E9J8ozXIbJ1K8ycacBLyo8hXyFO\n1UjKmyFfIR4jLylvhnyFeGSNpLwZ8hXiMfKS8mbIV4gjeUl5M+QrxIVXSXkz5CvEhVdJeTPkK8Tp\nGkl5M+QrxIVXSXkz5Cti+3Z44gn4zd8suxJJw8SQr4hNm2DuXIi+r1AhSS/UVchHxNKIWB0RayLi\nogm2nxYRP4mIOyPi+xFxWP6lDjenaiQVoWPIR8QIcAWwFFgELI+IQ1ua3Qu8KaV0GPAR4It5Fzrs\nXHSVVIRuRvJHAGtTSvellHYAq4Bl4xuklP47pfTY2MPbgAPzLXP4eYy8pCJ0E/LzgHXjHq8fe24y\n5wA3DFLUdOQx8pKKsFsXbbq+00dEvAU4G3jDRNtHR0ef+3uj0aDRaHS766G3cSMcfnjZVUgqW7PZ\npNls5ra/jneGioglwGhKaenY44uBXSmly1vaHQZcCyxNKa2dYD/eGaqNE06Ac8+FZcs6t5U0fUzF\nnaHuABZGxIKImAmcAlzXUsRLyQL+9IkCXp258CqpCB2na1JKOyNiJXATMAJclVK6OyJWjG2/ErgU\neBHw+cgO9N6RUjqiuLKHjyEvqQjeyLsCdu2CWbOyM15nziy7GklV4o28h8Ajj8A++xjwkvJnyFeA\nx8hLKoohXwEeIy+pKIZ8BbjoKqkohnwFeHEySUUx5CvAkbykohjyFWDISyqKIV8BTtdIKoohXwGO\n5CUVxZCvAEfykopiyJfs8cfhmWdgzpyyK5E0jAz5kj07VeMNvCUVwZAvmVM1kopkyJfMRVdJRTLk\nS+bFySQVyZAvmRcnk1QkQ75kTtdIKpIhXzIXXiUVyZAvmSN5SUUy5EtmyEsqkjfyLtGOHTB7Nmzf\nDiMjZVcjqYq8kXeNPfQQ7L+/AS+pOIZ8iVx0lVQ0Q75EzsdLKpohXyJDXlLRDPkSOV0jqWiGfIkc\nyUsqmiFfIkNeUtEM+RI5XSOpaIZ8iRzJSyqaZ7yWJCXYc0/YvDk761WSJuIZrzW1dSvMmmXASyqW\nIV8Sp2okTQVDviQuukqaCoZ8SRzJS5oKHUM+IpZGxOqIWBMRF03S5nNj238SEa/Jv8zhY8hLmgpt\nQz4iRoArgKXAImB5RBza0uZ44OUppYXAu4HPF1TrUBl0uqbZbOZWy3RnX+bL/qyWTiP5I4C1KaX7\nUko7gFXAspY2JwJfAUgp3QbsGxFzc690yAw6kvcfUn7sy3zZn9XSKeTnAevGPV4/9lynNgcOXlrv\nBvlwdfvaTu3abR+/bfxIfqLXtD5Xxj+cOvVnHs8Vrd/37OV1/fZnL8/72ezcpkr9uVuH7d2evdR6\noP6Er7vggi731qdbb21y5JGNQl/bqV277eO33XXX8yP5ZrNJo/Hrr2l9bqI2RRvkPbt9bad27bZ3\n02+9PFe0ft+zl9f125+9PO9ns3ObKvVn2zNeI2IJMJpSWjr2+GJgV0rp8nFtvgA0U0qrxh6vBt6c\nUnqwZV+e7ipJfRjkjNdOI/k7gIURsQB4ADgFWN7S5jpgJbBq7Etha2vAD1qkJKk/bUM+pbQzIlYC\nNwEjwFUppbsjYsXY9itTSjdExPERsRZ4Ajir8KolSV2ZsguUSZKmnme8StIQM+QlaYiVGvIRsSwi\nvhgRqyLirWXWMgwi4mUR8Q8R8U9l11JnEbFXRHxl7LN5atn11J2fy3z1mpuVmJOPiH2BT6aUzi27\nlmEQEf+UUvqTsuuoq4g4A9icUvq3iFiVUnpX2TUNAz+X+eo2N3MZyUfE1RHxYETc1fJ8x4ubjfkQ\n2TVyRC79qRY99un4s7ifmdJCa8LPaL767M+ucjOv6Zovk13EbHxxE17cLCLOiIi/iYjfjszlwI0p\npR/nVMsw6Ks/S6izTrruU7JLc8wfa+a61cR66U911su/+Z5yM5cPcErpFmBLy9MTXtwspfTVlNL7\nU0oPAOcDxwDvePbYe/XfnxGx39gZyK92FPXreulT4Frg5Ij4e7KT/dSil/70c9lZj5/PlfSQm53O\neB3ERBcue/34BimlzwGfK7CGYdJNf24GzpvKompuwj5NKT0JnF1OSbU2WX/6uezPZP15PvC33e6k\nyF9Fy1/RHS72Z/7s03zZn/nKpT+LDPkNPD+vydjf1xf4fsPO/syffZov+zNfufRnkSH/3MXNImIm\n2cXNnN/sn/2ZP/s0X/ZnvnLpz7wOobwGuBV4RUSsi4izUko7yRYIbgJ+BnwjpXR3Hu837OzP/Nmn\n+bI/81Vkf1biZChJUjE8BliShpghL0lDzJCXpCFmyEvSEDPkJWmIGfKSNMQMeUkaYoa8JA0xQ16S\nhtj/A3Qj6X8tlw8iAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x103735e10>"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Grid search can come in very handy when you're tuning a model for a particular task."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Quick Exercise #4"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Use ``GridSearchCV`` with the nearest neighbors estimator (``sklearn.neighbors.KNeighborsClassifier``) to determine the optimal number of neighbors (``n_neighbors``) for classifying digits (use accuracy as your metric)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.neighbors import KNeighborsClassifier\n",
      "from sklearn.datasets import load_digits\n",
      "digits = load_digits()\n",
      "X, y = digits.data, digits.target\n",
      "\n",
      "# construct the K Neighbors classifier\n",
      "\n",
      "\n",
      "# Use GridSearchCV to find the best accuracy given choice of ``n_neighbors``\n",
      "\n",
      "\n",
      "# Plot the accuracy as a function of the number of neighbors.\n",
      "# Does this change significantly if you use more/fewer folds?"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 14
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Overfitting, Underfitting and Model Selection"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now that we've gone over the basics of metrics, validation, and cross-validation, it's time to go into even more depth regarding model selection.\n",
      "\n",
      "The issues associated with validation and \n",
      "cross-validation are some of the most important\n",
      "aspects of the practice of machine learning.  Selecting the optimal model\n",
      "for your data is vital, and is a piece of the problem that is not often\n",
      "appreciated by machine learning practitioners.\n",
      "\n",
      "Of core importance is the following question:\n",
      "\n",
      "**If our estimator is underperforming, how should we move forward?**\n",
      "\n",
      "- Use simpler or more complicated model?\n",
      "- Add more features to each observed data point?\n",
      "- Add more training samples?\n",
      "\n",
      "The answer is often counter-intuitive.  In particular, **Sometimes using a\n",
      "more complicated model will give _worse_ results.**  Also, **Sometimes adding\n",
      "training data will not improve your results.**  The ability to determine\n",
      "what steps will improve your model is what separates the successful machine\n",
      "learning practitioners from the unsuccessful."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Illustration of the Bias-Variance Tradeoff"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For this section, we'll work with a simple 1D regression problem.  This will help us to\n",
      "easily visualize the data and the model, and the results generalize easily to  higher-dimensional\n",
      "datasets.  We'll explore a simple **linear regression** problem.\n",
      "This can be accomplished within scikit-learn with the `sklearn.linear_model` module.\n",
      "\n",
      "We'll create a simple nonlinear function that we'd like to fit"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def test_func(x, err=0.5):\n",
      "    y = 10 - 1. / (x + 0.1)\n",
      "    if err > 0:\n",
      "        y = np.random.normal(y, err)\n",
      "    return y"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 16
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now let's create a realization of this dataset:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def make_data(N=40, error=1.0, random_seed=1):\n",
      "    # randomly sample the data\n",
      "    np.random.seed(1)\n",
      "    X = np.random.random(N)[:, np.newaxis]\n",
      "    y = test_func(X.ravel(), error)\n",
      "    \n",
      "    return X, y"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 17
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X, y = make_data(40, error=1)\n",
      "plt.scatter(X.ravel(), y);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE/VJREFUeJzt3X2QZFdZx/HvQ17KwRRJrSkTIJEgArEojQmlgsaiLTIz\nC+LCZi0pFBhEWdBS0BrCEilkqiyNgFsCvlbC21jyohIWF4zbO6gNrIYAsomRZIGoYAATkACFVYNJ\nzOMf3Rs7k3npl9t9u09/P1Vd1X3nzu1nerp/c+bcc86NzESSNP0eUncBkqRqGOiSVAgDXZIKYaBL\nUiEMdEkqhIEuSYXoKdAj4q0RcWdE3Ny17fURcWtE3BQR742IM0dXpiRpJ7220N8G7N6w7SjwhMy8\nCPgMcGWVhUmS+tNToGfmR4Cvbdi2lpn3dR7eAJxXcW2SpD5U1Yf+QuC6io4lSRrA0IEeEa8C7s7M\nd1ZQjyRpQKcO880R8QLg6cBTt9nHxWIkaQCZGf3sP3ALPSJ2A1cAz8zMb+1Q1NTeXvOa19RewyzW\nbv3136y/3tsgeh22+C7gH4HHR8TtEfFC4PeBM4C1iDgeEX80UAWSpEr01OWSmc/ZZPNbK65FkjQE\nZ4ruoNFo1F3CwKa5drD+uln/9IlB+2p6foKIHPVzSFJpIoIc10lRSdJkMdAlqRAGuiQVwkCXpEIY\n6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQpSnRbDZZWNjHwsI+ms1m3eVoAjn1X5oCzWaTvXuXWF9/\nLQBzcwc4dGiVxcXFmivTqAwy9d9Al6bAwsI+1tb2AEudLavMzx/m6NFr6yxLI+RaLpI0w4a6BJ2k\n8Vhe3s+xY0usr7cfz80dYHl5td6iNHHscpGmRLPZ5ODBq4F2wNt/Xjb70CWpEPahS9IMM9AlqRAG\nuiQVwkCXpEIY6JL64hIEk6unUS4R8VbgJ4AvZ+b3dbbtAv4ceBTwOeCnM/Prm3yvo1ykQrgEwfiM\ncpTL24DdG7a9EljLzMcBf9t5LKlgBw9e3QnzJaAd7CfHxqt+PQV6Zn4E+NqGzXuAk1PVVoFnVViX\nJKlPw/Shn5OZd3bu3wmcU0E9kibY8vJ+5uYO0G7DrXaWINg/lue2735nlazlkpkZEVt2lK+srNx/\nv9Fo0Gg0qnhaSWO2uLjIoUOrXUsQjKf/fGPf/bFjS8X13bdaLVqt1lDH6Hnqf0RcALy/66ToCaCR\nmXdExMOBv8/MCzf5Pk+KShrKLC4fPO6p/4f5/1d3CXjfEMeSJA2ppy6XiHgX8BTg7Ii4HfgN4HeA\nv4iIn6czbHFURUqabS4f3BtXW5Q0NsMsATxrywe7fK6kieWkpP64fK5UiBKH6DkpafS8BJ00YWZh\niJ5Gwxa6tEHdreNSW7J1TkqaFbbQpS62jkenrklJs8STolKXSZjAUtLJw1kbmVKlQU6K2kKXJkwp\nLVn/2xk/W+hSl5Jax3W75JJLOX78f4FHAPuBO4qfrl8lhy1KQzrZOp6fP8z8/OGJCvO6T9b2o9ls\nctNNtwAvob3S9hJwc71FzYLMHOmt/RTS4I4cOZLz85fn/PzleeTIkbrLqcWRI0dybu6chLcnvD3n\n5s4Z2WtRxes9P395p9bs3N6eD3nId8zs728QnezsL2/7/Ya+n8BA1xDGGWSTbLOAnJ+/vPLnqer1\n3qzeiy9+SuX1lmyQQPekqCbaA8dkw/p6e9ukdIOUpqrXe7PFtK66ysW0Rs1Al6bAtK02WMpInWlj\noGuiTVuQjdKFF34Pn//8b/KoR53HVVeNJiCrfL0XFxcN8TFzlIsm2smW3sUXX8OuXb/JhRc+6KJY\nxTs5lPL48Rdx112v5sSJEyN7rqpH+UzTyJwi9Nvp3u8NT4pqSLN+YnRcJ0SrNuu/t2ExwElRW+ia\neKUuVjUudbWS/b2Nn4Guoflv9WgNs0rhye6atbU9rK3tYe/eJX9HJeu3Sd/vDbtcijaOf6v9133w\nyT51dteM6vc2KxPNcGKRMsf7hh/nhJdZ+BBXre7+96p/b7P0x91A19jf8HUHhrZXWgDO0vttkEB3\nHHphxj2z0nHik80JPrPFQNdQDIzJV9IEHxsQ2xt6PfSIuBJ4LnAf7fUxfy4z/6fr6znsc6h3ruc9\nm2bpykCz8rMOsh76UIEeERcAfwd8b2b+T0T8OXBdZq527WOgj9msvOHV5h/xMtUR6LuA64EnAd8E\nDgFvzMwPdu1joEtD2u6P9CRcB1XVG/sVizLzLuAg8B/Al4Cvd4e5VIVZn7jk5CD1aqiTohHxGOBX\ngQuAbwB/GRE/m5nv6N5vZWXl/vuNRoNGozHM02qGeKHhnUcueaKwDK1Wi1arNdxB+h3n2H0Dng28\nuevx84A/3LBPtYMzNXFGOelnlsYdb6WX18CJV+WhhnHoJ4BXR8Qc8C3gMuBjQx5TU8QW9Oj10gIv\naWiiBlfFsMVX0P5f8D7gk8AvZOY9XV/PYZ9Dk2vUJ+QcwdHmyKXZM8hJ0aEnFmXm64DXDXscaTN1\nTlzaGKJAbaFqC1y9GLqFvuMT2EIvWqkt6I0/1+mn/ypwGnff/XqgnJ9Tk2vs49B7egIDvXgldgc8\nuCvpycBLcKy3xqWWLhdpHN0BJf7RkKpmoGukqgjiOkbSbBxZcvrpJ4AruPvu9mPHemsS2eWikRm0\nf33jH4GDB6+uZWr7JJ0U1eyxy0UTZZC12TdrjV944YXjKPdBNutKMsQ1yQx0TZTN/gjANczNHXBq\nu7QDA10jU9UaI2effQ6HDr3ai2hIO7APXSPV70nRUse1S/1yHLqK4BBFyUCXpGKM/QIXkqTJYaBL\nUiEMdEkqhIEuSYUw0DWQWb9wszSJHOWivjlWXBo9hy1qLEZ92TlJDluUpJnmWi7qW1VrtEiqll0u\nGojT86XRsg9dkgphH7okzbChAz0izoqI90TErRFxS0Q8qYrCJEn9qeKk6BuB6zLzpyLiVODbKzim\nJKlPQ/WhR8SZwPHM/O5t9rEPXZL6VEcf+qOBr0TE2yLikxFxTUQ8dMhjSpIGMGyXy6nAJcAvZ+bH\nI+INwCuB3+jeaWVl5f77jUaDRqMx5NNqI4cRStOt1WrRarWGOsawXS7nAtdn5qM7jy8FXpmZz+ja\nxy6XEXNtFak8Y+9yycw7gNsj4nGdTZcBnxrmmHqwnVY2PHjw6k6YLwHtYD/ZWpc0O6oY5fIrwDsi\n4nTgX4Gfq+CY6tjY+j52bMnWt6RNOVN0wvWysuFWXS6A/erSlBqky8XFuQqwuLjIoUOrXeHdDnNb\n9tJssYU+4QY94ema5dJ0s4VeoM1a37ayJW3GFvqYjWu8uEMZpenm8rkTbtwh62QjaXoZ6BPOfm1J\nvXI99Bmy02QjSbPHQB+j5eX9zM0dAFaB1c61OPdvuf9WoX2y62ZtbQ9ra3vYu3fJUJdkl8u49dqv\nvV1/u103UvkctjgFFhcXezo5+cD1WWB9vb3NE5uStmKXy5hU2efdb9eNpNlgl8sYDDJccafvcUii\nVDaHLU6oB/d5v5xdu97HE5940Y796Ia2NJsM9An1wEBvAs8FfhdwBqekzXlSdEItL+/n2LEl1tcB\n/oR2mHuyU1K1PCk6BicX2JqfP8yuXV+puxxJhbLLZcxcNEtSL+xDnxKe7JS0EwNdkgrh4lySNMMM\ndEkqhIEuSYUw0CWpEJUEekScEhHHI+L9VRxPktS/qlroLwNuARzOIkk1GTrQI+I84OnAm4G+hthI\nkqpTRQv994ArgPsqOJYkaUBDLc4VEc8AvpyZxyOisdV+Kysr999vNBo0GlvuOvGc5SlpFFqtFq1W\na6hjDDVTNCJ+G3gecC/wbcDDgGsz8/ld+xQzU9R1WCSNS61T/yPiKcDLM/MnN2wvJtC9OLOkcZmE\nqf9lJLckTaHKLnCRmR8CPlTV8SbRAy9UQefizKv1FiVJHa622CdPikoaB5fPlaRCTEIfuiSpJga6\nJBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtS\nIQx0SSqEgT6EZrPJwsI+Fhb20Ww26y5H0ozzAhcDajab7N27xPr6a4H25egOHVr1CkaSKuEVi8Zo\nYWEfa2t7gKXOllXm5w9z9Oi1dZYlqRBesUiSZtipdRcwrZaX93Ps2BLr6+3Hc3MHWF5erbcoSTPN\nLpchNJtNDh68GmgHvP3nkqoy9j70iDgf+FPgO4EErs7MN23Yp4hAN7wljVMdgX4ucG5m3hgRZwD/\nBDwrM2/t2mfqA90RLZLGbewnRTPzjsy8sXP/v4FbgUcMc8xJdPDg1Z0wXwLawX6ytS5Jk6KyUS4R\ncQFwMXBDVceUJPWuklEune6W9wAv67TUH2BlZeX++41Gg0ajUcXTjo0jWiSNWqvVotVqDXWMoUe5\nRMRpwAeAv8nMN2zy9anvQwdPikoarzpOigawCnw1M39ti32KCHRJGqc6Av1S4MPAP9MetghwZWYe\n6drHQJekPrmWiyQVwrVcJGmGGeiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjo\nklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5J\nhTDQJakQQwd6ROyOiBMR8dmIOFBFUZKk/kVmDv7NEacAnwYuA74IfBx4Tmbe2rVPDvMckjSLIoLM\njH6+Z9gW+g8Bt2Xm5zLzHuDdwDOHPKYkaQDDBvojgdu7Hn+hs23qNZtNFhb2sbCwj2azWXc5krSj\nU4f8/p76UlZWVu6/32g0aDQaQz7taDWbTfbuXWJ9/bUAHDu2xKFDqywuLtZcmaRStVotWq3WUMcY\ntg/9ScBKZu7uPL4SuC8zX9u1z9T1oS8s7GNtbQ+w1Nmyyvz8YY4evbbOsiTNkDr60D8BPDYiLoiI\n04FnA4eHPKYkaQBDdblk5r0R8ctAEzgFeEv3CJdptby8n2PHllhfbz+emzvA8vJqvUVJ0g6G6nLp\n6QmmsMsF2v3oBw9eDbQD3v5zSeM0SJeLgS5JE6iOPnRJ0oQw0CWpEAa6JBXCQJekQhjoklQIA12S\nCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQ\nBrokFcJAl6RCGOiSVIiBAz0iXh8Rt0bETRHx3og4s8rCJEn9GaaFfhR4QmZeBHwGuLKakiZLq9Wq\nu4SBTXPtYP11s/7pM3CgZ+ZaZt7XeXgDcF41JU2WaX5TTHPtYP11s/7pU1Uf+guB6yo6liRpAKdu\n98WIWAPO3eRLv56Z7+/s8yrg7sx85wjqkyT1KDJz8G+OeAHwIuCpmfmtLfYZ/AkkaYZlZvSz/7Yt\n9O1ExG7gCuApW4X5IAVJkgYzcAs9Ij4LnA7c1dl0fWb+UlWFSZL6M1SXiyRpclQ+UzQidkXEWkR8\nJiKORsRZm+xzfkT8fUR8KiL+JSJeWnUd/YiI3RFxIiI+GxEHttjnTZ2v3xQRF4+7xu3sVH9E/Gyn\n7n+OiH+IiO+vo86t9PL6d/b7wYi4NyIuH2d9O+nx/dOIiOOd93trzCVuq4f3z9kRcSQibuzU/4Ia\nytxURLw1Iu6MiJu32WeSP7vb1t/3ZzczK70BrwNe0bl/APidTfY5F/iBzv0zgE8D31t1LT3Wewpw\nG3ABcBpw48ZagKcD13Xu/zDw0TpqHaL+JwNndu7vnrb6u/b7O+ADwL666+7z9T8L+BRwXufx2XXX\n3Wf9K8BVJ2sHvgqcWnftnXp+DLgYuHmLr0/sZ7fH+vv67I5iLZc9wGrn/irwrI07ZOYdmXlj5/5/\nA7cCjxhBLb34IeC2zPxcZt4DvBt45oZ97v+ZMvMG4KyIOGe8ZW5px/oz8/rM/Ebn4aRNAuvl9Qf4\nFeA9wFfGWVwPeqn/Z4BrM/MLAJn5X2OucTu91P+fwMM69x8GfDUz7x1jjVvKzI8AX9tml0n+7O5Y\nf7+f3VEE+jmZeWfn/p3Ati9eRFxA+y/UDSOopRePBG7vevyFzrad9pmUUOyl/m4/z2RNAtux/oh4\nJO2Q+ePOpkk68dPL6/9YYFenm/ETEfG8sVW3s17qvwZ4QkR8CbgJeNmYaqvCJH92+7XjZ3egYYvb\nTDh6VfeDzMztxqFHxBm0W10v67TU69BrOGwcfjkpodJzHRHx47Rn9f7o6MrpWy/1vwF4Zef9FDz4\nd1GnXuo/DbgEeCrwUOD6iPhoZn52pJX1ppf6fx24MTMbEfEYYC0iLsrMb464tqpM6me3Z71+dgcK\n9Myc3+aJ74yIczPzjoh4OPDlLfY7DbgW+LPMfN8gdVTki8D5XY/Pp/1XfLt9zutsmwS91E/nZMo1\nwO7M3O5f1HHrpf4nAu9uZzlnA0+LiHsy8/B4StxWL/XfDvxXZq4D6xHxYeAiYBICvZf6fwT4LYDM\n/NeI+Hfg8cAnxlLhcCb5s9uTfj67o+hyOQwsde4vAQ8K604r6y3ALZn5hhHU0I9PAI+NiAsi4nTg\n2bR/hm6HgecDRMSTgK93dSvVbcf6I+K7gPcCz83M22qocTs71p+Z352Zj87MR9P+j+4XJyTMobf3\nz18Bl0bEKRHxUNon524Zc51b6aX+E8BlAJ3+58cD/zbWKgc3yZ/dHfX92R3BWdtdwAdpL6l7FDir\ns/0RwF937l8K3Ef7jPrxzm13jWean0Z7pM1twJWdbS8GXty1zx90vn4TcEldtQ5SP/Bm2iMTTr7W\nH6u75n5f/6593wZcXnfNA7x/Xk57pMvNwEvrrrnP98/ZwPs77/2bgZ+pu+au2t8FfAm4m/Z/Qi+c\nss/utvX3+9l1YpEkFcJL0ElSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIK8X+Q2CSp\n+ZKJzgAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x108ab3210>"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now say we want to perform a regression on this data.  Let's use the built-in linear regression function to compute a fit:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X_test = np.linspace(-0.1, 1.1, 500)[:, None]\n",
      "\n",
      "from sklearn.linear_model import LinearRegression\n",
      "model = LinearRegression()\n",
      "model.fit(X, y)\n",
      "y_test = model.predict(X_test)\n",
      "\n",
      "plt.scatter(X.ravel(), y)\n",
      "plt.plot(X_test.ravel(), y_test)\n",
      "print(\"mean squared error:\", metrics.mean_squared_error(model.predict(X), y))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "mean squared error: 1.78514645061\n"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHntJREFUeJzt3XuUVOWZ7/Hvo4B0xqCjDCLeMIkKY44eM8aEI4aK0g1q\ngmDnMsaQNhpbTbiMiySoWZP0rKwcRzOsYxS8ACKNipcDtrbGULTRSuwTxTt4QQVN1KjgXSFprv2c\nP97CaRu6u+67atfvs1Ytqqp37f10U/X0289+3nebuyMiIpVvt6gDEBGRwlBCFxGJCSV0EZGYUEIX\nEYkJJXQRkZhQQhcRiYmMErqZLTCz9Wb2dJfnfm1mq81spZndYWZ7FS9MERHpS6Yj9BuA8d2eWw4c\n6e5HAy8CFxcyMBERyU5GCd3dHwTe7/Zcm7t3ph+uAA4scGwiIpKFQtXQzwbuLdC+REQkB3kndDP7\nGbDF3RcXIB4REclRv3xebGZnAacAJ/WyjRaLERHJgbtbNtvnPEI3s/HAT4DT3H1TH0FV7O0Xv/hF\n5DFUY+yKP/qb4o/2lotM2xZvAf4EHGFmr5nZ2cBVwJ5Am5k9aWZX5xSBiIgUREYlF3c/YxdPLyhw\nLCIikgfNFO1DIpGIOoScVXLsoPijpvgrj+Vaq8n4AGZe7GOIiMSNmeGlOikqIiLlRQldRCQmlNBF\nRGJCCV1EJEPu0NYGEybAiy9GHc3O8popKiJSDTZsgEWLYPZs6NcPpk6FA8twOUIldBGRHqxdG5L4\njTdCIgHXXANjxoBl1XtSOiq5iIh00dkJy5bBqafCqFEwcCA88QQsXRqSerkmc9AIXUQEgI8+goUL\nw4j8U5+CadNgyRKoqYk6sswpoYtIVXvhhZDEb74Zxo6F66+H0aPLeyTeE5VcRCpEMpmkrq6eurp6\nkslk1OFUtM5O+O1vYdw4+MpXYNAgWLUKbr8dTjihMpM5aOq/SEVIJpNMmtRAR8dlANTUzKSlpZlx\n48ZFHFll+eADuOEGmDMH9torlFW+/e1QJy83uUz9V0IXqQB1dfW0tU0AGtLPNFNb28ry5UujDKti\nPPdcKKvccguMHx/aDkeNKu+ReC4JXTV0EYml7dvhnnvgqqvgmWegsRGefRaGDYs6suJRQhepADNm\nNNLe3kBHR3hcUzOTGTOaow2qTL3/fjixOWcODBkSRuPf/CbssUfUkRWfSi4iFSKZTDJr1lwgJHjV\nzz/pmWfCaPz220MP+dSp8KUvRR1V7lRDF5Gqsm0btLaGRP7CC3DeeeE2dGjUkeVPNXQRqQrvvgvz\n58PVV8MBB4TReH09DBgQdWTRUkIXkYqxcmUYjS9dGlY8XLoUjj026qjKhxK6iJS1bdugpSUk8pdf\nhgsuCOWVIUOijqz8KKGLSFl6+22YNy+scDh8eCirTJoE/ftHHVn50tR/EclKsZcgePxxOOssOOyw\nsHxtays8+CB861tK5n3JaIRuZguAU4G33P1/pJ/bB7gNOAT4C/Atd/+gSHGKSBnovgRBe3tDQZYg\n2Lo11MOvugpeew1++MOQzAcPLkTU1SOjtkUzOwHYCCzqktAvB95x98vNbCbwj+5+0S5eq7ZFkZgo\n9BIE69fD3Llw7bVhRD51Kpx2WrgqULXLpW0xo5KLuz8IvN/t6QnAjqlqzcDEbA4sItXr0Udh8mQY\nMQJefRV+9ztIpULroZJ57vKpoe/n7uvT99cD+xUgHhEpYzNmNFJTM5MwhmtOL0HQmNFrt2wJa45/\n+cthKv5RR8FLL4UTn0cd1ffrtXxw3wryu9Dd3cx6rKs0NTV9fD+RSJBIJApxWBEpsXHjxtHS0txl\nCYK+6+dvvgnXXRdu//zPcNFF8PWvw+67Z37cYtXuy0kqlSKVSuW1j4yn/pvZcODuLjX054GEu68z\ns/2BB9x9xC5epxq6SJVxhxUr4MorQznlX/8VpkyBI4/MbX/VuHxw0WroPWjlv3+6DcCdeexLRGJg\n82ZYtAiOOw7OPDPM4nz55dBLnmsyl8xl2rZ4CzAGGGxmrwE/B/4TuN3MziHdtlisIEWkvL3+euhU\nmTsXjj4afv5zOOWU7MoqvdHywZnRaosikhN3+NOfQlll+fIwIp8yJXSu9CSfJYCrbflgLZ8rIkW3\naVO4lNtVV8GGDSGJn3VWuEZnb3Rd1OyUuoYuIkVSji16r70Gl1wCBx8cLiLxq1+FRbKmT+87mQPM\nmjU3ncwbgJDYd4y4pTDUwi9SZsqpRc89rKNy5ZVw//1hMlB7Oxx+eMlDkQwooYt0E3Wt9pMjWejo\nCM+VMo6//x0WLw5llc2bQ1nlhhvg05/OfZ86sVl8SugiXZTT6DgKr7wSrgK0YEG4Huevfw1jx8Ju\nBSjO5jIpSbLk7kW9hUOIVIba2tMdFnooNrjDQq+tPb2kMSxbtsxravZLx7HQa2r282XLlhXteJ2d\n7r//vfvEie777ON+4YXua9cWZt/Lli3z2trTvbb29KJ+D3GUzp1Z5VuN0EXKTKlGsn/7G9x0Uyir\ndHaGlQ5vvBH23LMw+6/2v3aioLZFkS6qobXu5ZdDWWXhQjj+eJg2DU48ESyrBrm+feELo3nyye3A\nMKARWBf76fqFpLZFkTztGB3X1rZSW9taVsk8n1ZGd2hrCxdWPu64kLwffRTuugtOOqnwyTyZTLJy\n5XPA+YSVthuApwt7ENlZtjWabG+ohi55Uh0297r6hg3uc+a4jxzp/vnPu193nfvGjX0fK9+f967O\nRey2275V+/+XC3KooSuhS1kr9QnCcpXtydo1a9ynTw8nOSdNcn/ggXDysy+F+nnvKt5jjhmT9X6q\nWS4JXSdFpayVQ092pejsDGuqXHUVPPIInHMOPPEEHHJI5vso1M97Vz3nl16qnvNiU0IXqQC9Tcr5\n6CNobobZs6GmJpzkXLIk3I+Kes6joYQuZU2zC//biBGf45VXfskhhxzIpZc2M3z4OKZODZd1GzsW\n5s+H0aPzO8FZyJ/3uHHjlMRLTAldytqOkd7FF/+SV15ZxyGH9LI2a0x1b6XcuLGNn/3sX3j1VTj3\nXFi5Eg46qDDHKvTIOuplFKpOtkX3bG/opKjkqdpPjIYTjDd3OcH4jh955JXe0RF1ZL2r9v+3fJHD\nSVH1oUvZq+ZlV597DlavPheo7/LsPQwblmLgwMz2EdVSvNX8/xYVJXTJWzmu3V3Jtm+H1laorQ0z\nOL/ylcMYOPA4oBloTte1GzPa145yTVvbBNraJjBpUoP+j+Is2yF9tjdUcom1UvxZXS1/ur/3nvt/\n/Zf7oYe6f/GL7jfe6L5pU/harpN9olxsrFj/b9Uy0QxNLBL30r7hS5Uw4vwhfvpp98ZG9733dj/z\nTPeHHy7cvqNePbLQ/2/V8svdXQldvPRv+KgTRqXautX9jjvcv/pV9/33d29qcn/zzcIfJ24JsJre\nb7kkdLUtxkypZ1aqTzw7774L118Pc+bAAQeEJWvr62HAgOIcTxN8qosSuuRFCSMzK1eGKflLl4YV\nD5cuhWOPLc2x4zTBRwOI3uW9HrqZXQx8F+gkrI/5fXff3OXrnu8xJHPVsJ53pdi2De68MyTyl16C\n88+HxkYYMqTwx6qmCTzV8r3msh56XgndzIYD9wMj3X2zmd0G3OvuzV22UUIvsWp5w5ert9+GefPg\nmmtg+PBQVpk0Cfr3L87x9Es8nqJI6PsADwFfBjYALcBv3P2+LtsooUtVeOKJMBq/886QwKdOhWOO\nKcy+e/slXVdXT1vbjotIADTrykAxUPIrFrn7e8As4FXgDeCDrslcpBDKeeLS1q1w221hUayJE+GI\nI2DNGliwoLDJXJODJBN5nRQ1s88C/wYMBz4E/q+ZnenuN3fdrqmp6eP7iUSCRCKRz2GlipTrhYbX\nr4e5c+Haa+Gww+DCC+G006BfEdoM+upc0onCeEilUqRSqfx2km2fY9cb8G1gfpfHk4E53bYpbHOm\nlJ1iTvopt77jRx5xnzw5TAL6wQ/cV64s/jEz+RnEeeJVtSKCPvTngX83sxpgEzAWeCTPfUoFKdcR\ndCFt2RIuGHHllbBuHfzoR3DFFbDPPqU5fiYj8Di1Jkoesv0N0P0G/BR4ltCy2Az07/b14v8qk8gU\newQd5UzHN95w/8Uv3IcOdT/xRPeWFvdt20py6J1oBF59iGKmqLtfDlye735EdqXUE5fcYcWK0K3S\n2rqVvfa6n0MP/R0//enJAJx8cjTtoBqBSybynljU5wHUthhrcemB3rwZbr89lFXeew9OOul5brzx\nNDZtugSAAQP+DejPli2/Bir3+5TKUfI+9IwOoIQee5U8ken110Onyrx5cNRRoXf8lFPg5JO793aP\nAs5Hvd5SKrkkdK3lInkrRTmgkL803OFPfwplleXL4Ywz4IEHYOTIQkUrEg0ldCmqQiTiQnXSbNoE\nt94ayiobNsCUKXDddbDXXjtv272zZMCA54GfsGVLeKxebylL2Z5FzfaGulyqVq4dKt07OvLtpHn1\nVfeLL3b/p39yHz/e/be/dd++PbP4u8ahThMpJbQeupSTXNZm39VofMSIEVkf2x0efDCUVX7/e5g8\nGdrb4fDDM9/HrkpJlXR+QKqPErqUlV39EoB51NTMzGhqe0cHLF4cEvmmTaGssmABfPrTJQlfJFJK\n6FI0hVpjZPDg/Whp+fdee9FfeQWuvjok7y99CS67DGprYbe8lp8TqSxqW5SiyvakaDZ97e6QSoWT\nnH/8IzQ0wA9/CJ/7XMG/DZGSUx+6xEJfvwT+9je46SaYPRu2bw+945Mnw557RhGtSHEooUus/fnP\n4eLKCxfC8cfDtGlw4olgWb3lRSpDyS9wIVJs7nDffeHCyl/8Ykjejz4Kd90FJ52kZC7SlU6KSlna\nuBEWLQplld13D2WVW26Bf/iHqCMTKV9K6FJW1q4NZZVFi2DMmNC5MmaMRuIimVDJRSLX2QnJJJx6\nKowaBXvsES64fMcdkEgomYtkSiN0yUkh1mj56CNobg5llZqaUFZZsiTcF5HsqctFspbvGugvvhiS\n+E03hROb06bB6NEaiYt0peVzpSRyWaOlsxOWLQtT8p94An7wA1i5Eg46qERBi1QBJXQpqg8/hBtu\nCCc6Bw0KZZWWFhg4MOrIROJHCV2ylskaLatXh7LK4sUwblyolY8apbKKSDGphi452dVJ0e3b4d57\nw9oqTz8NjY1w/vkwbFjEwYpUIE39l0i8/35Y5XDOHBg8OJzk/OY3Q/uhiORGJ0WlpJ55JpRVbrst\n9JDfcktYulZEopF3QjezvYH5wJGAA2e7+8P57lfK0/bt0NoaulWefx7OOy/Uy4cOjToyESnECP03\nwL3u/g0z6wdotY0Yeu89mD8/TMXff/9QVqmvhwEDoo5MRHbIq4ZuZnsBT7r7Z3rZRjX0CrZqVRiN\nL1kSVjycOhWOPTbqqETiL4oa+qHA22Z2A3A08Dgw3d3/nud+JULbtsGdd4ZE/tJLoVPlhRdgyJCo\nIxOR3uSb0PsBXwCmuPujZnYFcBHw864bNTU1fXw/kUiQSCTyPKx0V4i1Vd55B+bNg2uugYMPDmWV\nSZOgf/9CRysi3aVSKVKpVF77yLfkMhR4yN0PTT8eDVzk7l/rso1KLkWW79oqTz4ZRuMtLSGBT50K\nxxxTzIhFpC8lv2KRu68DXjOzw9NPjQWezWefsrNkMkldXT11dfUkk8mdvv7JtVVCYt8xWu/J1q2h\n3XD0aDjtNDjsMFizJvSTK5mLVKZCdLlMBW42swHAS8D3C7BPSes++m5vb8hq9N3dW2/B3Llw7bXw\nuc/BhReGhN5PMxJEKp5mipa5urp62tomsGNlQ2imtraV5cuXfrxNTyUX4OOR+te+9mMee2wUd98N\n3/hGKKscdVQpvxMRyYZmilapcePG0dLS3OWkaEjmEyf+gE2bbgZGcN99mzn77BdYu/YI9t03wmBF\npGiU0MtcJisbQkjqO8ow69bB8cffyqZNLwCfAsB9Ea++ehf77rt0p9eKSDzomqJlbsfou7a2ldra\n1l7r5ytWwJlnwsiRsHnzPwJtXb6qspdI3KmGXmKF6BfvavNmuP320Hb4zjswZQp8//vwyCP5tTKK\nSLS0fG6Zy7dfvKs33gidKnPnhpObU6fCKafA7rt/8niF/OUhIqWjhF7mMulY6Y07PPRQuIDE8uVw\nxhlhRD5yZNFCFpGIlHxikZTGpk2wcGFYFKuhIVzKbf7837NmTT3Tp+96spGIVB8l9BKaMaORmpqZ\nQDPQnO5Yaexx+5tuSvGZzyxl0KAPmTPnbX75y7BI1ogRSb773TNpa5tAW9sEJk1qUFIXEZVcSq2v\nurY7tLfDJZeso719D2Ad8Dw1NRd8XG/Pt3QjIuVPE4sqQNd+8a46OmDx4tCtsmkTwD2EHvLvACPp\n6PiIWbPm6sSmiPRIJZcS6WmBrVdegZkzw3K1LS1w2WXw3HNw8MG/A7bucl/Zlm5EpDqo5FIC3dsV\nBw6cyX/8RysPP3wcf/gDfO978KMfhcWyenpN9xZHtSSKxJvaFsvUzjXvN9ltt79x+OH3c+mln2Hi\nxLG7fJ2Stkj1UkIvUyeccAHt7VOAI4G3gAuACYBmcIrIrqkPvYy4w333hbXGV636Df36/QFYCpxG\nSOaZX4xCRCQTSugFtnFjuCbn5z8fLh5x6qnwxhsDuOeez1Jbu5h99nk76hBFJKZUcimQtWthzhxY\ntAjGjAlrqyQSYN3+YCrkei4iEl+qoZdYZye0tYXe8RUr4Jxz4IIL4JBDen+dTnaKSF+U0EtkwwZo\nbg6JvKYmjMa/851wX0SkEDRTtMhefBFmz4abboKTToL582H06J3LKiIiUVBC70NnJyxbFkbjjz8O\n554LK1fCQQdFHZmIyCcpoffgww/DkrWzZ8OgQaGs0tICAwdGHZmIyK4poXezenVI4rfcAnV1oVY+\napTKKiJS/gqS0M1sd+Ax4K/u/vVC7LOUtm+He+8NZZVVq6CxEZ55BoYNizoyEZHMFWqEPh14Dvh0\ngfZXEh98AAsWhP7xffcNZZVvfQv22CPqyEREspf3TFEzOxA4BZgPVERh4tln4fzz4dBDw4nOxYvh\nkUdg8mQlcxGpXIUYof8f4CfAoALsq2i2b4e77w5lldWr4bzzwr9Dh0YdmYhIYeSV0M3sa8Bb7v6k\nmSV62q6pqenj+4lEgkSix00L7r334PrrQ1ll//1h2jSor4cBA3Lbn2Z5ikgxpFIpUqlUXvvIa6ao\nmf1vYDKwDRhIGKUvdffvddkmkpmiq1aF0fiSJTBhQqiPH3tsfvvUOiwiUiqRTv03szHAj7t3uZQy\noW/bBnfdFRL5mjVhXZXGRhgypDD718WZRaRUymHqfySLtrzzTpiGf/XV4dqcU6fC6adD//5RRCMi\nEo2CJXR3/wPwh0LtLxNPPhlG4y0tMGlSGJ0fc0zxjjdjRiPt7Q10dITH4eLMzcU7oIhIFip6tcXz\nzoPhw8P6KoMHF+UQO9FJUREpBS2fKyISE7qmqIhIFVNCFxGJCSV0EZGYUEIXEYkJJXQRkZhQQhcR\niQkldBGRmFBCFxGJCSV0EZGYUEIXEYkJJXQRkZhQQhcRiQkldBGRmFBCFxGJCSV0EZGYUELPQzKZ\npK6unrq6epLJZNThiEiV0wUucpRMJpk0qYGOjsuAcDm6lpZmXcFIRApCVywqobq6etraJgAN6Wea\nqa1tZfnypVGGJSIxoSsWiYhUsX5RB1CpZsxopL29gY6O8LimZiYzZjRHG5SIVDWVXPKQTCaZNWsu\nEBK86uciUiglr6Gb2UHAImAI4MBcd7+y2zaxSOhK3iJSSlEk9KHAUHd/ysz2BB4HJrr76i7bVHxC\nV0eLiJRayU+Kuvs6d38qfX8jsBoYls8+y9GsWXPTybwBCIl9x2hdRKRcFKzLxcyGA8cAKwq1TxER\nyVxBulzS5ZYlwPT0SP0TmpqaPr6fSCRIJBKFOGzJqKNFRIotlUqRSqXy2kfeXS5m1h+4B/idu1+x\ni69XfA0ddFJUREoripOiBjQD77r7hT1sE4uELiJSSlEk9NHAH4FVhLZFgIvdfVmXbZTQRUSypLVc\nRERiQmu5iIhUMSV0EZGYUEIXEYkJJXQRkZhQQhcRiQkldBGRmFBCFxGJCSV0EZGYUEIXEYkJJXQR\nkZhQQhcRiQkldBGRmFBCFxGJCSV0EZGYUEIXEYkJJXQRkZhQQhcRiQkldBGRmFBCFxGJCSV0EZGY\nUEIXEYkJJXQRkZhQQhcRiYm8E7qZjTez581sjZnNLERQIiKSPXP33F9stjvwAjAWeB14FDjD3Vd3\n2cbzOYaISDUyM9zdsnlNviP044C17v4Xd98K3Aqcluc+RUQkB/km9AOA17o8/mv6uYqXTCapq6un\nrq6eZDIZdTgiIn3ql+frM6qlNDU1fXw/kUiQSCTyPGxxJZNJJk1qoKPjMgDa2xtoaWlm3LhxEUcm\nInGVSqVIpVJ57SPfGvqXgSZ3H59+fDHQ6e6Xddmm4mrodXX1tLVNABrSzzRTW9vK8uVLowxLRKpI\nFDX0x4DDzGy4mQ0Avg205rlPERHJQV4lF3ffZmZTgCSwO3B91w6XSjVjRiPt7Q10dITHNTUzmTGj\nOdqgRET6kFfJJaMDVGDJBUIdfdasuUBI8Kqfi0gp5VJyUUIXESlDUdTQRUSkTCihi4jEhBK6iEhM\nKKGLiMSEErqISEwooYuIxIQSuohITCihi4jEhBK6iEhMKKGLiMSEErqISEwooYuIxIQSuohITCih\ni4jEhBK6iEhMKKGLiMSEErqISEwooYuIxIQSuohITCihi4jEhBK6iEhM5JzQzezXZrbazFaa2R1m\ntlchAxMRkezkM0JfDhzp7kcDLwIXFyak8pJKpaIOIWeVHDso/qgp/sqTc0J39zZ370w/XAEcWJiQ\nykslvykqOXZQ/FFT/JWnUDX0s4F7C7QvERHJQb/evmhmbcDQXXzpEne/O73Nz4At7r64CPGJiEiG\nzN1zf7HZWcC5wEnuvqmHbXI/gIhIFXN3y2b7XkfovTGz8cBPgDE9JfNcAhIRkdzkPEI3szXAAOC9\n9FMPufsPCxWYiIhkJ6+Si4iIlI+CzxQ1s33MrM3MXjSz5Wa29y62OcjMHjCzZ83sGTObVug4smFm\n483seTNbY2Yze9jmyvTXV5rZMaWOsTd9xW9mZ6bjXmVm/8/Mjooizp5k8vNPb/dFM9tmZqeXMr6+\nZPj+SZjZk+n3e6rEIfYqg/fPYDNbZmZPpeM/K4Iwd8nMFpjZejN7updtyvmz22v8WX923b2gN+By\n4Kfp+zOB/9zFNkOB/5m+vyfwAjCy0LFkGO/uwFpgONAfeKp7LMApwL3p+18CHo4i1jziHwXslb4/\nvtLi77Ld/cA9QH3UcWf5898beBY4MP14cNRxZxl/E3DpjtiBd4F+UceejucE4Bjg6R6+Xraf3Qzj\nz+qzW4y1XCYAzen7zcDE7hu4+zp3fyp9fyOwGhhWhFgycRyw1t3/4u5bgVuB07pt8/H35O4rgL3N\nbL/ShtmjPuN394fc/cP0w3KbBJbJzx9gKrAEeLuUwWUgk/i/Ayx1978CuPs7JY6xN5nE/yYwKH1/\nEPCuu28rYYw9cvcHgfd72aScP7t9xp/tZ7cYCX0/d1+fvr8e6PWHZ2bDCb+hVhQhlkwcALzW5fFf\n08/1tU25JMVM4u/qHMprElif8ZvZAYQkc036qXI68ZPJz/8wYJ90mfExM5tcsuj6lkn884AjzewN\nYCUwvUSxFUI5f3az1ednN6e2xV4mHP2s6wN399760M1sT8Koa3p6pB6FTJND9/bLckkqGcdhZl8l\nzOo9vnjhZC2T+K8ALkq/n4yd/y+ilEn8/YEvACcBnwIeMrOH3X1NUSPLTCbxXwI85e4JM/ss0GZm\nR7v7hiLHVijl+tnNWKaf3ZwSurvX9nLg9WY21N3Xmdn+wFs9bNcfWArc5O535hJHgbwOHNTl8UGE\n3+K9bXNg+rlykEn8pE+mzAPGu3tvf6KWWibx/wtwa8jlDAZONrOt7t5amhB7lUn8rwHvuHsH0GFm\nfwSOBsohoWcS//8CfgXg7i+Z2Z+BI4DHShJhfsr5s5uRbD67xSi5tAIN6fsNwE7JOj3Kuh54zt2v\nKEIM2XgMOMzMhpvZAODbhO+hq1bgewBm9mXggy5lpaj1Gb+ZHQzcAXzX3ddGEGNv+ozf3T/j7oe6\n+6GEv+guKJNkDpm9f+4CRpvZ7mb2KcLJuedKHGdPMon/eWAsQLr+fATwckmjzF05f3b7lPVntwhn\nbfcB7iMsqbsc2Dv9/DDgt+n7o4FOwhn1J9O38RGeaT6Z0GmzFrg4/dx5wHldtpmd/vpK4AtRxZpL\n/MB8QmfCjp/1I1HHnO3Pv8u2NwCnRx1zDu+fHxM6XZ4GpkUdc5bvn8HA3en3/tPAd6KOuUvstwBv\nAFsIfwmdXWGf3V7jz/azq4lFIiIxoUvQiYjEhBK6iEhMKKGLiMSEErqISEwooYuIxIQSuohITCih\ni4jEhBK6iEhM/H8u4OGHInwHAAAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x106e92d90>"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We have fit a straight line to the data, but clearly this model is not a good choice.  We say that this model is **biased**, or that it **under-fits** the data.\n",
      "\n",
      "Let's try to improve this by creating a more complicated model.  We can do this by adding degrees of freedom, and computing a polynomial regression over the inputs. Scikit-learn makes this easy with the ``PolynomialFeatures`` preprocessor, which can be pipelined with a linear regression.\n",
      "\n",
      "Let's make a convenience routine to do this:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.preprocessing import PolynomialFeatures\n",
      "from sklearn.linear_model import LinearRegression\n",
      "from sklearn.pipeline import make_pipeline\n",
      "\n",
      "def PolynomialRegression(degree, **kwargs):\n",
      "    return make_pipeline(PolynomialFeatures(degree),\n",
      "                         LinearRegression(**kwargs))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we'll use this to fit a quadratic curve to the data."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model = PolynomialRegression(2)\n",
      "model.fit(X, y)\n",
      "y_test = model.predict(X_test)\n",
      "\n",
      "plt.scatter(X.ravel(), y)\n",
      "plt.plot(X_test.ravel(), y_test)\n",
      "print(\"mean squared error:\", metrics.mean_squared_error(model.predict(X), y))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "mean squared error: 0.919717192242\n"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH4dJREFUeJzt3Xl8VPW9//HXR4Qa9ao/REBEFHtdUFsWF1y4dQomQWtR\nwOp1qWlriwu13l7koqg0tWqLberW1j5cGy1V20IsKDKJ1ii4oCioILhrtRVURK8Xwv75/XEGEkOW\n2c/Mmffz8ZhHZibfOeeTycxnvvM53+/3mLsjIiLFb7uwAxARkexQQhcRiQgldBGRiFBCFxGJCCV0\nEZGIUEIXEYmIpBK6md1pZivM7OUW9/3SzJaa2YtmNsPMds1dmCIi0plke+h3ASNb3VcPHOLuA4HX\ngMuyGZiIiKQmqYTu7nOBVa3ua3D3zYmb84G+WY5NRERSkK0a+veA2VnaloiIpCHjhG5mlwPr3f1P\nWYhHRETStH0mDzaz7wAnAiM6aKPFYkRE0uDulkr7tHvoZjYSmAic7O5rOwmqaC8/+clPQo+hFGNX\n/OFfFH+4l3QkO2zxXuAp4EAze8/MvgfcDOwMNJjZQjP7XVoRiIhIViRVcnH3M9q4+84sxyIiIhnQ\nTNFOxGKxsENIWzHHDoo/bIq/+Fi6tZqkd2Dmud6HiEjUmBmer4OiIiJSWJTQRUQiQgldRCQilNBF\nRCJCCV1EJCKU0EVEIkIJXUQkIpTQRUQiQgldRCQilNBFikQ8HqeiYiwVFWOJx+NhhyMFSFP/RYpA\nPB5n9OgqmpqmAlBWNom6uloqKytDjkxyJZ2p/0roIkWgomIsDQ2jgKrEPbWUl8+kvn56mGFJDmkt\nFxGREpbRKehEJD8mTBjHvHlVNDUFt8vKJjFhQm24QUnBUclFpEjE43Fqam4FggSv+nm0qYYuIhIR\nqqGLiJQwJXQRkYhQQhcRiQgldBGRiFBCF5GUaAmCwpXUKBczuxP4BvChu38lcV934H5gH+Ad4DR3\n/7SNx2qUi0hEaAmC/MnlKJe7gJGt7rsUaHD3A4BHE7dFJMJqam5NJPMqIEjsW8bGS/iSSujuPhdY\n1eruUcCWqWq1wClZjEtERFKUSQ29l7uvSFxfAfTKQjwiUsAmTBhHWdkkgj5cbWIJgnF52bdq953L\nylou7u5m1m6hvLq6euv1WCxGLBbLxm5FJM8qKyupq6ttsQRBfurnrWv38+ZVRa5239jYSGNjY0bb\nSHrqv5ntC8xqcVB0GRBz9+VmtifwmLsf1MbjdFBURDJSissH53vq/0yan90q4IEMtiUiIhlKquRi\nZvcCxwE9zOw9YArwC+DPZnYuiWGLuQpSREqblg9OjlZbFJG8yWQJ4FJbPljL54pIwdKkpNRo+VyR\niIjiED1NSso9nYJOpMCUwhA9yQ310EVaCbt3HNWebJiTkkqFeugiLah3nDthTUoqJTooKtJCIUxg\n6ezgoTusXg1r1kBTU/Bzy/VNm5q3Y9b8s6wMdtwRdtqp+bLDDs1tcvm3lNLIlGxK56CoeugiBWLD\nBnj/fdhhh0rGj6+nrm4xa9fuTv/+i/j5z3szYQJ8/DGsXAlduwZJeccdm5N1WRlsn3hHt+xDbd7c\nnPi3fBCsXg0bN0L37rDHHl+89O4N++wD++4b/Nxrr+btpkLfdvJPPXSRFnI9tM4dVqyApUuDy7Jl\nzT9XrGhOpv36BZe+faFnT+jRo/my++7QrVvmsWzYEHxAfPTRFy8ffADvvhtc3nkHPvwQ9twT9t8f\nDj4YBgxo/rnHHu338ocMGcbChZuAPsA4YHnkp+tnk8ahi2RBtsoE7vD22/D88/DCC80/IUiGAwbA\nQQc1/9x77457wmGVL9avh/feg9dea/4geuWV4Od228HgwXD44XDYYcHPffaB+vo4J554Bps3X5/Y\nyiTgbMrL31ZCT5ISukRSsdRh166F556DuXODy/z5QSnksMOaL0OGBL3dVOVzUk6yz7c7LF/e/GG1\nYEFwWb8e4AVWrgQYkmhdy3bbTWD27GkF+/8rNOkkdNw9p5dgFyLpmTNnjpeV9XL4g8MfvKysl8+Z\nMyfssNzdfe1a90cfdb/sMvdhw9x32sn98MPdf/xj9xkz3Jcvz96+ysvHJJ4DT1z+4OXlY7K3g4Rs\nPN///Kf7wIG/cFjcIt4PvFevWn/00eB5k84lcmdK+VYHRaWgfXFMdnBwr6bm1tB6eW+8AXPmQDwO\nTzwRlEoqKmDKFDj6aNh551DCyppsPN99+sDUqYMYPXpEYltd6dZtJrHYtUyeHJRqjjsOTjgBRo6E\n/v1z87eUIiV0kQ5s3gzPPAMzZsADDwQjRCor4eyz4Q9/CA5Q5kOxrTa47ZjzcVRW7gcEB2IfeST4\nYPzpT2HXXeGb34QxY+Coo4K6vKQp1S59qhdUcpEMhFFyWb/ePR53P/9899693Q891H3KFPcXXnDf\nvDmnu27XnDlzfPDgY7179y/74MHH5ew5yPfzvWlT8LxOmeJ+yCHuffq4jx8flLI2bMjZbosCaZRc\nlNCl4OUjmW3e7P7kk+4XXOC+++7uQ4e6T53q/tprWd9VyvKdZOfMmePl5WO8vHxMxvtJdVvLlrlf\ne21wLKJHj+D/8dRT4X2QhkkJXSIplwlt6VL3K65w79/ffcAA92uucX/77axsOmvydUA02zL9v731\nlvvVV7sfeKD7l7/s/pOfuL/+eu7iLTTpJHRVq6TgZXuxqtWr4a67goOYX/96UBefPh2WLIHJk4MZ\nklES1mJjmf7f+veHyy8PDqLedx+sWgXHHgvHHBP8/9asyVnoRUsJXTIW9uqEyXrxRRg/PpjAM2NG\nkLzfew9qaoLJMble1yRdmaxSuGX8ekPDKBoaRjF6dFVB/4/aYhZMWLrxxmBphEsvDT6A994bLroI\nFi8OO8ICkmqXPtULKrlEWj7qu5nsY90693vucT/ySPe993avrnb/xz+yGl5epFvXDrNck6vXxpbn\nYtiwcX7mmW94nz7uxxwT/J/XrctC4AUC1dDFPbsHtTqTzwkvqfxNH33k/rOfue+5p/vxx7vPnOm+\ncWPWwyp4Ydffs/1abOtD4qGH4l5X5z58eDBK5ppr3D/+OAvBh0wJXfI+IiLshNHa4sXu3/+++267\nuZ97rvtLL4UWSkEo5Jm26ejs9bZokft3vhP8/88/Pxg1U6zSSeiqoUdMvs92UyhnoZk/H04+GY4/\nPlil8NVX4fbb4StfyXsoBWXLBJ/y8pmUl8+M/PK1AwcGB0yXLg1Wqfza1+CUU4I1dkpCqp8AqV5Q\nDz2vwugx57PE09LmzcEElBEj3Pv1c//Nb9zXrMnb7iUEqX7jWLPG/eabg+MnFRXujz+ex2AzRBgl\nF+AyYAnwMvAn4Eutfp/7v1y2itpX7LZs3uz+4IPuRx3lfsAB7nfdFczuLGVhfaiGIZ2/dd0699tv\nD8azDxvm/vDDhT9ZKe8JHdgXeGtLEgfuB6pcCT1UUX5zP/pokMgPOcT9/vtL80Bna6XwIZ4tGza4\nT5vmfvDBwciYxx4LO6L2hZHQuwOvAv+PYKGvWcDxroQuWfbUU8Eohn//9+ANWWqJvKMP6UI7MF0M\nNm4Mhjnut597ebn7s8+GHdG20knoGR0UdfdPgBrgH8C/gE/d/ZFMtinS0ksvwdChHzJ8+EcsX/47\nbrihnjPPhC5dwo4sf6IwOajQdOkSrJi5bBmMHQujRweXYp+klNHyuWb2ZeC/CEovnwF/MbOz3H1a\ny3bV1dVbr8diMWKxWCa7lRLwwQdw5ZUwffo6Vq++kQ0bDuKVV3biW986J/IjNVrrbI3yYltat5B0\n7QrnnQfnnAO33AIjRgSjpa66Kji/az41NjbS2NiY2UZS7dL7F8sppwO3t7j9beC3rdrk9nuJhC6b\nNfvVq92vuipY8XDiRPdY7OySLyckU1KJ8nGTfFq1yv2SS4LX3zXXhDtqihBq6AOBxUAZYASDkce3\napOHP13Ckq0Dcps2ud99t3vfvu6nnRastOeu+rC7DnqG4Y033E89NRgOO21a8PrMt7wn9GCf/A/N\nwxZrga6tfp/7v1xCk42E++yzwfrXQ4cGa5K3pGQWUA88HHPnuh9xRDCyasGC/O47lITe6Q6U0CMt\nk4S+cqX7eecFZwWqrW2/FxRWMmu9XyXV0rRpk/udd7r36hUsJ7ByZX72q4QueZdOD3rTJvc77gje\nIOPHB3XLQtP67+rWbTfv1m2Pkv+mUMo++SR4vfbs6X7rrbkvw6ST0C14XO6Ymed6HxKueDze6mTA\n7Y9AWbQILrwwOPny734HQ4bkK8rUVFSMpaFhFFtGlsDRwPktbgfro9TXTw8lPgnPokXwox8Fr99D\nD83dfswMd09plf6Mhi2KQLAAVGfDCNesgepqqK2Fq6+Gc89N7ezuqXxoiOTSoEHwxBNhR9E2JXTJ\nqXg8zuWXz+OVVy7kyCPh5Zf3pGfP1LcxenRVYiw2zJtXlfOx6K3HdnfrtgyYyPr1wW2N9ZZCpJKL\n5MyMGY9y2mnvs2nTqcB8ysrOTCoRt+6N19Tc2qr8kZ9yR+s4AH1LkLxJp+SihC458dBDMHbsx6xb\n9wlwQOLezhNx6954WdkkDjroIBYu/C6qX0spUQ1dQvfZZ8GJe598Eg499Gaef34/mhN659qa5g63\nUVY2SVPbRTqhMxZJ1jz2GHz1q7DzzsGiWtdcc0xWzmbUo0evkjrrjki6VHKRjK1dC5Mnw/33B6d9\nO+GE5t+lOjqlrZKLEriUItXQJe8WLgyWIT34YPj972H33TPfpoYoiiihSx5t2gTXXQe//jVcfz2c\ndRZYSi89EemIDopKXixfHvTK16+H55+Hfv3CjkhEQAdFJUX19cF0/WOOgb//XclcpJCohy5J2bAB\npkyBu++GP/4Rhg8POyIRaU0JXTr17rtwxhmw667BQdBUp+6LSH6o5CIdeughOPJIGDMmuL4lmcfj\ncSoqxlJRMVYnLBYpEBrlIm3atCk4Ue4dd8Cf/xzUzLfQWHGR3NMoF8mKlSuDYYhr18KCBdue/byz\ns9CLSDhUcpEveP55OPxw+MpX4JFHtk3mIlK41EOXre68EyZNgltugVNPbb9d67XCtViWSGFQDV3Y\nsAEuvjhYXGvGDBgwoPPHaHq+SG5p6r+kbOVK+Na3YKedYNo02GWXsCMSEUgvoauGXsKWLAmGJB5x\nBDzwgJK5SLHLOKGb2W5m9lczW2pmr5jZUdkITHLrwQfh61+Hn/4Upk6FLl3CjkhEMpWNg6I3ArPd\n/VQz2x7YKQvblBxxD1ZJvOkmmDULhg4NOyIRyZaMauhmtiuw0N3366CNaugFYv16+MEPglLLAw9A\n375hRyQi7Qmjht4f+MjM7jKzF8zsNjPbMcNtSg6sWgWVlfD55/DEE0rmIlGUaclle2AI8EN3f87M\nbgAuBaa0bFRdXb31eiwWIxaLZbhbaa2jYYTvvAMnnggjR8Ivf6l6uUghamxspLGxMaNtZFpy6Q08\n7e79E7eHAZe6+0kt2qjkkmMdra3y3HNw8slw2WVw0UUhByoiScv7Wi7uvtzM3jOzA9z9NeB4YEkm\n25RtdTaJp721Vdatq+T734fbbguSuohEWzZGuVwETDOzbsCbwHezsE1JaN37njevKqmVDf/xjxM4\n//xgydsjjshHpCISNs0ULXAVFWNpaBjFlt431FJePpP6+ulb27RO+ttv/xm9e3+Pq69+gWnTbgQ0\nPV+k2GimaImqrKykrq6WESMepE+fvuy3XxU1Nc9ywQWn0dAwioaGUYweXaUTUYhEnHroBS7Zk0k0\nNQWniVuzJlhga8yYznv2IlK41EOPoC297/LymZSXz2wzmX/2WTAkcYcdgin9O+8cUrAiEir10PMs\n28vOLl8eJPNhw4Lp/Ntt17wfnSZOpHhp+dwCl+0k++abUFEBVVVw5ZVgrf71WrNcpHgpoRe4ZEas\nJGvx4mAq/xVXwAUXZDVMESkAqqGXiIUL4WtfW0f37tdTVzdWo1dEBFBCz6sJE8ZRVjYJqAVqE+fi\nHNdu+3g8TkXFWCoqmpP2/PkwfPg6Vq++kMWLu2tIoohspZJLniVb126r3n7VVTO57roj2Wuva1i0\nqC8akigSXXlfy0VSV1lZmdTByW3XZ9mTyZMP5OGHYerUFwCtfysiX6SSS560VT5JzX8wcOAvGTEi\n9dKNiJQGlVzyIJ3his2PuRc4ii99aQx/+9t/bX2MhiSKRJuGLRaobYcrXkL37g9w2GEDO0zGV165\niOuu24chQ66murpCSVukhKiGXhTiQC2ffPIrGhraXw63rg5uu20Q8+fDoEE14YQqIkVFCT0PJkwY\nx7x5VTQ1Afwe+BWtT0bRMqH/7W/BZKHZs2HQoDAiFpFipIOiedByga3u3T/qsO2sWTBuXHBiiiFD\n8hSgiESCauh51tEB0tmz4bvfDVZM1FmGREqbDooWibZGqMyZA+ecE/TQhw4NOUARCZ0SepGqr4ez\nzw5q50cfHXY0IlIItDhXEZo7N0jmdXVK5iKSGSX0EC1YAGPHwr33wrHHhh2NiBQ7JfSQLFkCJ50E\nt90GI0aEHY2IRIESegjefDM4OUVNDZx8ctjRiEhUZCWhm1kXM1toZrOysb0oe/99OP744JRxZ50V\ndjQiEiXZ6qFfDLwCaDhLBz76CMrL4cIL4bzzwo5GRKIm44RuZn2BE4HbgZSG2JSSTz8NyiynngoT\nJ4YdjYhEUTZ66NcDE4HNWdhWJDU1wTe/CcOGwVVXhR2NiERVRotzmdlJwIfuvtDMYu21q66u3no9\nFosRi7XbtOClug75xo1wxhnQrx/ccAOYvsOISBsaGxtpbGzMaBsZzRQ1s2uBbwMbgR2AXYDp7n5O\nizaRmSma6okq3INa+TvvBOuzdOuWx2BFpKiFOvXfzI4DLnH3b7a6PzIJfdsTVXR8cuYpU4IlcB97\nDP7t3/IWpohEQCGc4CIamTsLbrklmAH65JNK5iKSH1lL6O7+OPB4trZXiL54ogoSJ2eu3abd9Olw\n9dXBOi09e+Y5SBEpWVptMUWdHRRtbITTToN4HAYPDiFAEYkELZ8bspdeCmaB3ncfDB8edjQiUsy0\nfG6I3n8fvvENuOkmJXMRCYcSehZ8/nmwcuIPfwj/+Z9hRyMipUollwxt3BjMAu3XD37/e00cEpHs\nUMklz9yDXjnAb3+rZC4i4cr2OPSS8qtfwdNPB8MTt9czKSIhUxpK01/+EhwAffpp2GWXsKMREVFC\nT8tTTwVrmjc0QN++YUcjIhJQDT1Fb74ZnNj57rth0KCwoxERaaaEnoJPPw2GJ06ZAiecEHY0IiJf\npGGLSdq4MUjm++8PN98cdjQiEnUatphDEyfCpk1w/fVhRyIi0jYdFE3C7bcH65o/84yGJ4pI4VLJ\npROPPx6snjh3LhxwQNjRiEipUMkly956C04/HaZNUzIXkcKnhN6O//3fYI2WK68MlsQVESl0Suht\n2LQJzjwTjjsOxo9vv108HqeiYiwVFWOJx+P5C1BEpA2qobfh0kvh2WeDsw517dp2m3g8zujRVTQ1\nTQWC09HV1dVucwYjEZF0FMJJooven/8M998Pzz3XfjIHqKm5NZHMqwBoagruU0IXkbAoobfw8stB\niaW+Hnr0CDsaEZHUKKEnrFoFo0fDjTcmd3LnCRPGMW9eFU1Nwe2ysklMmFCb2yBFRDqgGjrBQdCT\nToIBA+DXv07+cfF4nJqaW4EgwavcIiLZkk4NPaOEbmZ7A3cDPQEHbnX3m1q1KfiEfvnlwZK4DQ3t\nzwRV8haRfAojofcGerv7IjPbGXgeOMXdl7ZoU9AJffp0+O//hgULYI892m6jES0ikm95H+Xi7suB\n5Ynr/2dmS4E+wNIOH1ggliyBCy6Ahx9uP5mDRrSISHHI2sQiM9sXGAzMz9Y2c+nTT+GUU6CmBg47\nLOxoREQyl5VRLolyy1+Bi939/1r/vrq6euv1WCxGLBbLxm7TtnkznH02nHgifPvbnbfXiBYRybXG\nxkYaGxsz2kbGo1zMrCvwIPCwu9/Qxu8LroZ+7bXBcriPPdbx5KGWdFBURPIpjIOiBtQCK939x+20\nKaiE/ve/w1lnBQdB99or7GhERNoWRkIfBjwBvEQwbBHgMnef06JNwST0f/0LDj8c7rkHRowIOxoR\nkfblPaEntYMCSegbNsDw4VBZCVdcEXY0IiIdU0LvwMSJwTDFBx+E7bRosIgUOK222I66umAVxRde\nUDIXkeiKfA/9zTfh6KNh1iwYOjS0MEREUqJzirbS1ASnngpTpiiZi0j0RbqH/oMfwOefw733gqX0\nOSciEi7V0Fu45x6YOzc485CSuYiUgkj20F99FYYNg0cfha9+Na+7FhHJCtXQgbVr4fTT4Wc/UzIX\nkdISuR76RRfBBx/AX/6iUouIFK+Sr6HX1QUThxYuVDIXkdITmR76u+/CEUfAzJlw1FE5352ISE6V\nbA19wwY44wy45BIlcxEpXZHooU+eHEzrnz1bU/tFJBpKsobe0AC1tUHdXMlcREpZUSf05cuhqiqY\nRNSzZ9jRiIiEq6hLLqNGwcCBwZhzEZEoKbn10Jcuhf33h+2L+nuGiMi2Si6hi4hEVckOWxQRESV0\nEZHIUEIXEYkIJXQRkYhQQhcRiYiME7qZjTSzZWb2uplNykZQIiKSuoyGLZpZF+BV4Hjgn8BzwBnu\nvrRFGw1bFBFJURjDFo8E3nD3d9x9A3AfcHKG2xQRkTRkmtD3At5rcfv9xH1FLx6PU1ExloqKscTj\n8bDDERHpVKaT5pOqpVRXV2+9HovFiMViGe42t+LxOKNHV9HUNBWAefOqqKurpbKyMuTIRCSqGhsb\naWxszGgbmdbQjwKq3X1k4vZlwGZ3n9qiTdHV0CsqxtLQMAqoStxTS3n5TOrrp4cZloiUkDBq6AuA\n/c1sXzPrBpwOzMxwmyIikoaMSi7uvtHMfgjEgS7AHS1HuBSrCRPGMW9eFU1Nwe2ysklMmFAbblAi\nIp3QaovtiMfj1NTcCgQJXvVzEcknLZ8rIhIRWj5XRKSEKaGLiESEErqISEQooYuIRIQSuohIRCih\ni4hEhBK6iEhEKKGLiESEErqISEQooYuIRIQSuohIRCihi4hEhBK6iEhEKKGLiESEErqISEQooYuI\nRIQSuohIRCihi4hEhBK6iEhEKKGLiESEErqISESkndDN7JdmttTMXjSzGWa2azYDExGR1GTSQ68H\nDnH3gcBrwGXZCamwNDY2hh1C2oo5dlD8YVP8xSfthO7uDe6+OXFzPtA3OyEVlmJ+URRz7KD4w6b4\ni0+2aujfA2ZnaVsiIpKG7Tv6pZk1AL3b+NVkd5+VaHM5sN7d/5SD+EREJEnm7uk/2Ow7wA+AEe6+\ntp026e9ARKSEubul0r7DHnpHzGwkMBE4rr1knk5AIiKSnrR76Gb2OtAN+CRx19PufmG2AhMRkdRk\nVHIREZHCkfWZombW3cwazOw1M6s3s93aaLO3mT1mZkvMbLGZ/SjbcaTCzEaa2TIze93MJrXT5qbE\n7180s8H5jrEjncVvZmcl4n7JzJ40s6+GEWd7knn+E+2OMLONZjYmn/F1JsnXT8zMFiZe7415DrFD\nSbx+epjZHDNblIj/OyGE2SYzu9PMVpjZyx20KeT3bofxp/zedfesXoDrgP9JXJ8E/KKNNr2BQYnr\nOwOvAgOyHUuS8XYB3gD2BboCi1rHApwIzE5cHwo8E0asGcR/NLBr4vrIYou/Rbu/Aw8CY8OOO8Xn\nfzdgCdA3cbtH2HGnGH818PMtsQMrge3Djj0Rz38Ag4GX2/l9wb53k4w/pfduLtZyGQXUJq7XAqe0\nbuDuy919UeL6/wFLgT45iCUZRwJvuPs77r4BuA84uVWbrX+Tu88HdjOzXvkNs12dxu/uT7v7Z4mb\nhTYJLJnnH+Ai4K/AR/kMLgnJxH8mMN3d3wdw94/zHGNHkon/A2CXxPVdgJXuvjGPMbbL3ecCqzpo\nUsjv3U7jT/W9m4uE3svdVySurwA6fPLMbF+CT6j5OYglGXsB77W4/X7ivs7aFEpSTCb+ls6lsCaB\ndRq/me1FkGRuSdxVSAd+knn+9we6J8qMC8zs23mLrnPJxH8bcIiZ/Qt4Ebg4T7FlQyG/d1PV6Xs3\nrWGLHUw4urzlDXf3jsahm9nOBL2uixM99TAkmxxaD78slKSSdBxm9nWCWb3H5i6clCUT/w3ApYnX\nk7Ht/yJMycTfFRgCjAB2BJ42s2fc/fWcRpacZOKfDCxy95iZfRloMLOB7v55jmPLlkJ97yYt2fdu\nWgnd3cs72PEKM+vt7svNbE/gw3badQWmA3909wfSiSNL/gns3eL23gSf4h216Zu4rxAkEz+Jgym3\nASPdvaOvqPmWTPyHAfcFuZwewAlmtsHdZ+YnxA4lE/97wMfu3gQ0mdkTwECgEBJ6MvEfA1wD4O5v\nmtnbwIHAgrxEmJlCfu8mJZX3bi5KLjOBqsT1KmCbZJ3oZd0BvOLuN+QghlQsAPY3s33NrBtwOsHf\n0NJM4BwAMzsK+LRFWSlsncZvZv2AGcDZ7v5GCDF2pNP43X0/d+/v7v0JvtFdUCDJHJJ7/fwNGGZm\nXcxsR4KDc6/kOc72JBP/MuB4gET9+UDgrbxGmb5Cfu92KuX3bg6O2nYHHiFYUrce2C1xfx/gocT1\nYcBmgiPqCxOXkSEeaT6BYKTNG8BlifvOA85r0eY3id+/CAwJK9Z04gduJxiZsOW5fjbsmFN9/lu0\nvQsYE3bMabx+LiEY6fIy8KOwY07x9dMDmJV47b8MnBl2zC1ivxf4F7Ce4JvQ94rsvdth/Km+dzWx\nSEQkInQKOhGRiFBCFxGJCCV0EZGIUEIXEYkIJXQRkYhQQhcRiQgldBGRiFBCFxGJiP8PNMgmdr+I\nLBYAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x106dd3a50>"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This reduces the mean squared error, and makes a much better fit.  What happens if we use an even higher-degree polynomial?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "model = PolynomialRegression(degree=30)\n",
      "model.fit(X, y)\n",
      "y_test = model.predict(X_test)\n",
      "\n",
      "plt.scatter(X.ravel(), y)\n",
      "plt.plot(X_test.ravel(), y_test)\n",
      "plt.ylim(-4, 14)\n",
      "print(\"mean squared error:\", metrics.mean_squared_error(model.predict(X), y))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "mean squared error: 0.375474625154\n"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEACAYAAABMEua6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUFOXV+PHvZR8EQQVZRAQRiRBUjEGNWwvONBiDAtlc\nURDzxmg0jILE5Oe4xS2oMRrfI74IGhEjCpIYbUZM4xZ3NllEVBBUQEBBYWAGuL8/apppZrqqt+qV\n+zlnDr1UV99puu/cvvXU84iqYowxpjg0ynUAxhhj/GNJ3RhjiogldWOMKSKW1I0xpohYUjfGmCJi\nSd0YY4pIQkldRCaJyDoRWRTjvnIR2S0iB/ofnjHGmGQkWqk/Cgyqf6OIHAqUAqv8DMoYY0xqEkrq\nqvoq8HWMu+4BxvoakTHGmJSl3FMXkXOANaq60Md4jDHGpKFJKg8SkZbA73FaL3tu9iUiY4wxKUsp\nqQM9gG7AAhEB6AK8JyL9VXV99IYiYpPLGGNMClQ16WI5pfaLqi5S1Q6q2l1VuwNrgOPqJ/So7Qvq\np7paadzYuXzjjTfmPJ50fix+iz8fY3/6aWXYsOQf96c/Kddf71xu105Zv744X3vV1GvhRIc0Pgm8\nARwpIqtF5NL6eTvlCPKQCKTxmhpj4ti5E5o2Te2x9tn0llD7RVXPi3P/4f6Ekx8sqRuTWTU10CSF\n5q/Ykbu47IzSGKKTeiAQyGks6bL4c6uQ489k7Nmo1Av5tU+HJfUYoquBQn9jWPy5VcjxZzL2bFTq\nhfzap8OSegyRN461YIzJjHQq9Qj7fMZmSd2DvWmMyYxUK3XY+3NpPfaGLKm7sIOlxmROqpW6JfH4\nLKm7aNTIkroxmeJXpW4asqTuQgR27851FMYUJ6vUM8eSugtrvxiTOVapZ44ldReW1I3JHKvUM8eS\nugtL6sZkTjqVeoR9PmOzpO7CkroxmePXGaVWuTdkSd1Fo0Z2oNSYTLG5XzLHkroLq9SNyZyaGpul\nMVMsqbuwcerGZM7OnVapZ4oldRfWfjEmc9Kp1I03S+ou7OQjYzIn1Uod7Bt0PAkndRGZJCLrRGRR\n1G13i8hSEVkgIs+KSJvMhJl91n4xJnNSrdSj2y/2+YwtmUr9UWBQvdtmA31U9RhgOTDer8Byzdov\nxmSODWnMnISTuqq+Cnxd77ZKVY2kvreALj7GllPWfjEmc2xIY+b42VMfCfzbx/3llLVfjMkcW3g6\nc9I8UdchIjcA1ao6Ndb9FRUVey4HAoGCWGbK2i/GZE51tVXq9YXDYcLhcNr7STupi8glwFnAQLdt\nopN6obCkbkzmVFdD8+a5jiK/1C94b7rpppT2k1ZSF5FBwHXA6aq6PZ195RvrqRuTOekkdWu/eEtm\nSOOTwBtALxFZLSIjgb8CrYBKEZknIn/LUJxZZz11YzJnxw5o1iz5x9mQxvgSrtRV9bwYN0/yMZa8\nYu0XY5KzdSuMHQunnw4//7n3tn5V6sXcY0+VnVHqwtovxaGmBp57Dp54Ar79NtfRFLebboJPPoFf\n/xrWrPHe1o9K3cRmSd2FtV8K3+bNcMopcPfd8I9/QO/e8MEHuY6qOG3dCo88Ag8/DEOHwpNPem9v\nB0ozx5K6C2u/FDZVuOQS6NcPXn3VqdbvuAN+/GMn2Rt/Pf889O8Phx4Kw4bBv+OcsZJqpQ5WbMVj\nSd2FtV8K29SpsGoV/OUvdV/ZL7gAysrghhtyG1sx+te/4JxznMsnnADvvgu7drlvn2qlbu2X+Cyp\nu7D2S+GqqYGKCpgwoWHiuOsuePppWLw4J6EVrZdfdv5gAhx0EHTu7P0aW6WeOZbUXVj7pXBNnQpd\nu8IZZzS874AD4OqrneRu/LF6tVN5H3543W3HHON9/MKPSt2Se2yW1F1Y+6VwPfQQ/O537vdfcYXT\nLvjss+zFVMzefBNOPHHvhHvUUbB0aeztd+50/m3cOLXnsyGN3iypu7D2S2FauNAZTjeo/iTRUdq2\nhfPPh0kFeJZFKBSirGw4ZWXDCYVCuQ4HqEvq0Xr3hiVLYm9fXZ1668WSeHyW1F1Y+6UwPfoojBwZ\nf7KoUaOcbQvp/zgUCjF06AgqK4dQWTmEoUNH5EVif/NNOOmkvW/r1Qs+/DD29jt22HDGTLKk7sKS\neuHZvds5CHperHOf6zn2WOeA3pw5mY/LLxMmPExV1Z3ACGAEVVV3MmHCwzmNqboa5s+H44/f+/bD\nDnNGH8X6tptOpQ72DToeS+ourKdeeN56C9q0cfq5ibjwQnjqqczGVOwWLIAjjoDWrfe+vW1b5zMU\n65yAdCp1a7/EZ0ndhfXUC8/06fCznyW+/bBhzklJkQN3+a68/HJKSsYBU4AplJSMo7z88pzGFKuf\nDk7y7drVqdbrs0o9syypu7D2S+H55z/rToBJRLduTpvglVcyFpKvgsEgM2ZMobR0FqWls5gxYwrB\nYDCnMbkldXBe21gjjNKZIsCGNMbny8pHxcjaL4Xl44+dCbuOPTa5xw0fDs88AwMGZCYuvwWDwZwn\n8mhvvgl/+EPs+9wq9XROPKrP2jENWaXuwtovhSUUgmAw+Q/58OHw7LP2BzwV69fDxo3OSJdYMlGp\ng30u47Gk7sLaL4XlxRe9x6a7OfJIZxTM22/7H1MxC4VCBIO306TJfCorYw+rzESlbpV5fAkldRGZ\nJCLrRGRR1G0HikiliCwXkdki0jZzYWaftV8Kx44dEA5DaWlqjz/rLHjhBV9DKmqR8fLz5/+YjRtx\nHS/ftatV6rmQaKX+KFC/DroeqFTVI4E5tdeLhrVfCsdrrzlnMB50UGqPHzzYknoy6sbLHw0c6zpe\nPjJWvT6r1DMroaSuqq8CX9e7eQjO2Cpq/z3Xx7hyztovheOll1Kv0gFOPhmWL3d6xCZR8bNrp07w\n1VcNh4xu3w4lJak/sxVb3tLpqXdQ1XW1l9cBHXyIJ29YUi8cc+dCIJD645s1g4EDnYOtJr7y8stp\n3vxRYDNe4+WbNHG+PdX/Y7ltW2pJPRQKcd99D/HCC5WEQiFL7i58GdKoqioiri9xRUXFnsuBQIBA\nOp/ALLGeemHYutWZxKv+3CPJGjzYWa3noov8iauYBYNBRo06lJkz36VPn1mUl7uPl+/UCb74wplf\nPaKqClq2TO45I338qqpngXYMHXoaImsQKZ5R2eFwmHA4nPZ+0nlF1olIR1VdKyKdANcvr9FJvVBY\nT70wvPGGMzY92SRR3+DBMG6cs1pPqlPC7kvWr+/N7bf35uKLB3pu17kzfPnl3relUqnX9fF/BEBV\n1Z00blxNMZ1qU7/gvemmm1LaTzrtl1k4MwtR++/MNPaVd6z9UhjmzoXTT09/P4ccAl26OPPHGG+q\nib/unTrtndRDoRAPPvh3Zs2akRczTBajRIc0Pgm8AfQSkdUicilwB1AqIsuBAbXXi4a1XwpDuv30\naDYKJjFLl8J++zmjW+KJtF+groWyYkVfVq7sltTUwXXz3rwBfERJyTiaN/fptNQik+jol/NUtbOq\nNlPVQ1X1UVXdpKpnquqRqlqmqt9kOthssvZL/tu2DebNgx/9yJ/9DR7snMSUD/JxMYyIOXMS/0Ma\n3X6pa6EcA/RLaurgyLw3vXvPp3Pnz5gxYwqNGxdP68VP9qq4sPZL/nvzTejb16ka/XDSSfDRR85o\njXnzQnsSTnn55Vmdb6XuoOCdALz22oi8mLwrYsYM+O1vE9u2Uyf/vv0Eg0HGjHGOo+TJS5GXbJoA\nF9Z+yX9+tl7AGdp4xhkwYcLCnK4wlI+LYURs2ADvvQdlZYltH91+qWuhrABeT3nq4Mg3aPsmHZsl\ndRfWfsl/4bA/B0mjDRoEjz/+dd4m1Vz75z+dE70SHW300Udz+eCDjZSVDQdgxowpdOiwjr5930rp\n20f9M0rtDNOGLKm7sPZLftu+3akYTz7Z3/0OGgQbNiQ5f6/P8nExjIhHHklsuUBw2kgjR/6MHTva\nUFm5liFDfglA//4nc8stY1JuJ1mx5c2Sugtrv+S3t95y5nupv4xaug47DDp1akzz5lPJVVINBoNM\nnz6Fvn03cvDBR9K16zImTgxyww3zKS39Wc4Onr7xhnPQ89wEJwQZP/4WamoU2An8lurqJowffwvb\ntqV+XoFV5vHZgVIX1n7JL6HQ3gcu33wz6Gs/Pdrw4a3YuPEBvvzy+trny+5Byk8/hZtuCtK6dZDr\nroMePeDZZxdx++27UP0b8GrGD57Wf73LyoLcdhuMGZP4yVmrVq0F/gy0AH4BbGfVqlsoKbG5XzLJ\nkroLa7/kj1ijQY48cjG33ZbitIxxDBoEN9/ck9deeyYj+/fywQfOQcjrroNrrqmrTCsqKlAdAvwA\nGEZVVTMmTHg4I0k91ut92WVzWLmyD6NHJ76fww7rwqZNDW+zSj2zLKm7sKSeP/YeDQJVVY344IOW\nnHJKZp7vtNOc+WS+/hoOOCAzzxHLZ585Q/X+/Gc4//x4Ww/ku+9m+/K8NTXw0EMwebIzpLO6uj/V\n1QuIzNFXVXUyEye2YeHC5OZBv/328QwZchHV1ScD62nW7Dpuv/1xrr46vUrdeLOeugvrqeez9uy3\n3+e0aZOZvbdoAaee6kzpmy1VVTBkiNPeiJXQTz/9OBo1+h2RPn/TpmNZseJO1qxJ73lXroT+/Z3J\nzO65B1avhhNOGAvMj9rqG048sZyePZPbdzAYZNasx+nWbQE9eqxi1qzHCQaDbN2a3rkFNqTRmyV1\nF9ZTzx91o0GuBU5CZAFHH/1VRp9z0KDsnl06fjz07Okk9fpCoRC33fZXdu8eCfwvjRqVc+ONHSkv\nL+EnP4HvvkvtOZctg1NOgUsucU4QCgSgbVu44YafUlIygroDxWdz/fUXpPy7NW/+NTt21H3l+eYb\n53lSYUMaE6CqGf1xnqLwXHSR6pQpuY7CRNx6663aqNEBCpMVvtRmzc7TF198MWPPt3y5aufOqrt3\nZ+wp9giFVLt0Ud24Mfb9paXDan9vrf2ZrKWlw3T3btWRI1WHDFHdudN9/y+++KKWlg7T0tJhe16z\nVatUDz1UdfLkxB+TrBdffFFLSjoovKywSktKOujzz4e0USPVXbtS2qVOnux8NlVV99tP9dtvU9tP\nIajNnUnnXKvUXVj7JXHZmKdk7tz32b37Xpy+ekeqq3+c0ROCevZ0+r6LFsXfNh3btsHll8OkSXDg\ngck9VsTphX/7rTNtcCyRg57RZ8c+9dTLlJXB734HI0bEflwwGGT27GeYPfuZlA/G1h0LOQPoSlXV\nndx99xO0aeN8E06FVebxWVJ3Ye2XxMRKGtkZQ70z/iZpGjQInn8+s89x551wwgney/F5nYzUrBlM\nn+6c6TlxYsPHNpxyYAKjR3dl2DAnqWdbTc1+vhwLCYVCbN9exZAhF+TdhGc5l0p5n8wPBdp+GTlS\n9ZFHch1Favz46pwot9aA3+q+ys9TWKglJR0y/ru99JJq//6Z2/8nn6gedJDqZ5/F3zbe/+ny5aod\nOqhWVu59e8P/ny/0kENCWWkr1f2fPaawU1u06Kh//evreuyxqe9zyhTVAQM+r91vtcLfs/JeyAVS\nbL9YUncxapTqww/nOork1X2QJitMzvgbPltJXdX53dq2Xaz9+t2clQ9xdXXiSTcV556reuut/u1v\n7lzVdu1Un3mm7ra698OTCmu1cePp+vzz2UuAkT9GTZps0WnT5uicOaqBQOr7e+wx1U6dwll7z+VS\nqkndxqm7KNT2S8Mx3WTsJBVwWgOvvTaCqirnutMamJKR5zr11CA1NfDKK71p1SojT7GXpk3hJz+B\nmTPhqqv83ffs2U6//skn/dvnaac5I3aGD4enn4ZRo6BduyAXXhhmypQOHHLIHB58sDWDB2fv7Nhg\nMEgwGKRPHzjqqAGsWJH6yJeIQvxcZlPaSV1ExgMXAruBRcClqroj3f3mmp18lJjI4gV1p5Rn7tT1\nV1911iPNRkKPGDbMGb/tZ1KvrnbmI7/3XmdMvB+iT+v/y19+zbJlZ3Ljjc5wx+OP/x6vvw7HH/9T\nf54sBV26OGPg0xnOCM6B0qOO6snmzb+gqup84B8ZLSQKUVpJXUS6AaOBo1R1h4g8BfwS54hOQSvU\npJ7NyjkiUo1l2gsvOKsTZVNpKVx0kbNwxsEH+7PP+++Hww+Hs89O/DH152KJfr0bntZ/ITNmTGHc\nuPxZSeLww+GTT5w/aOlW6p06dWbGjCmcddYuAoHnGTs2fxYQyQfpVupbgBqgpYjsAloCn6cdVR4o\n1CGN2aycs+2FF2DatOw+Z4sWTgvmySfh6qvT39+XX8IddzgzHiY6PC/eSkjZbrmlokcP+Phj5zPV\nvXvq+4m8ZsFgkBYtYNasqb6tfFUs0hrSqKqbgAnAZ8AXwDeqmsWTqzOnUHvq4M8Y43zz6KOvsHLl\n14wdm/1pZy+5xJkXxQ9jx8Lo0XDkkYk/Jp9XQkpUJKl/+ml6Sd3El277pQdwDdAN2Aw8LSIXqOoT\n0dtVVFTsuRwIBAhkas5UHxVq+yUXolsDY8ZczjffBHnkEefEmqFDnQq3WRoLv4dCIX71q7nU1HTi\npZeG8Prr2V2z84wzYONGmD/f6emn6vXXndWali71LTQgNy23ZPXq5fzeLVqkn9QLtdiKJxwOEw6H\n099RKkNmIj84kyQ/EnX9IuDBettkcNBP5lx9teo99+Q6ivxXfwhlkyZ/0a5dv9WZM1Vffln1rLNU\njztO9auvUn8OZ9jk6pwOYfvDH1SvuSb1x1dXq/btqzptWvKPTWSYajbPTUjFrl2qbds6/3+bN6e+\nnyeeUD3vPOdyy5aq333nT3z5iBwNaVwG/FFESoDtwJnA22nuMy8Ucvslm+r3c3fu/Jru3X/DOec4\nlWIgADfcAGee6YxeSWWlop07SwCfjlImIfobyHnnXcO4cadyxhmVPPDA/wIND1h6uf9+6NgRfv7z\n5ONI5DhJtg5Wp6pRI+dg6caNsP/+6e0r8rm0z2dsaSV1VV0gIo8B7+IMaXwfKKxmnwtrv6QqTLNm\nddMGisBttzmjRy67zDnQmez8HT/84Vjmzg2ze/eXQHbaCw0PTv6Mww57hZ/+dDY1NUNqb0usDbR6\nNdx+O/z3v6nPXZLvSTsR//lP+kM4bZbG+NIep66qdwF3+RBLXinU0S/ZVtfPDQAbKCn5dYOEKwIP\nPAAnneRMXDVqVHLPsWzZ0Qwd+in/+c8tAIwZc1XGE1ysESWff15BTc1DQJs9t8UbZaIKV17pjEtP\ndj7yYpNuhW4SYxN6ubD2S2KCwSB33TWD5s1bMWDAn2NWrqFQiCFDhtOixRjKy6tZvTrx/W/aBC+9\nVMPzz1/Lpk1/ZNOmP3LbbX/NySROTZq8jdNlTNzEiU6l7jaLokmefS692TQBLqz9kriXXjqJP/8Z\nrryy4Tnv9dsYTZrcy7BhI3n77fauX52je9mHH34b+++/nrVr/0A2x2HHHlFyFTffPIrq6qeAGXHb\nQMuWOccTXnkluWXgjDtrt8RnSd2FtV8S8/HHzlC9qVNj39/wQOpjfPjht0ya1D5mG6b+HwGRRhxx\nxHTWrv1Bhn6D2NwOTh5/fIhLLnmPpk0PYOJE937611/Duec6vfSjjspm5MXPKnVvltRdWPslMQ8+\nCCNHJrM6vNKnz1+5/vp7GTgQunXb+976fwRUv2G//d6npOQfWR+HHevgZDAYZPFiOOYY9sRT3+bN\nzhQAgwc7B4eNf6xSj8+Sugtrv8RXXQ2PPQbvvOO+Taw2RkXFFObPh0svhTlz4q2Cs4D27Ttwxx1/\nzJupDw48EGbMgLPOcl6D6GGKS5bAeec5C1dPmJCzEIuaDWmMI5XB7cn8UKAnH914o/Nj3M2cqXrq\nqfG3i3VizM6dqqec0vA1rjvRJqywUVu06JiXJ9Ooqr7zjmqvXs7vMWaMs1boAQc48/BnYxGKfdFT\nT6n+7GfO5RYtVLdty208mYTNp+4vEdi1K9dR5Le//x0uvDD+drHaGI0bO3N+9+8PXbs6LZzItn/7\n2z+4/PJj6NfvNm6+eXLejs8+/nhYsABCIecU+H79nNcklROsjPGLJXUXjRrBzswvg1mwvvnGWejh\n4TRONevYESorIRiEefOc8dwrV8LNN5/GfffBFVfk/+kPzZvDkCHOj8kOa7t4s6Tuwnrq3qZPd079\nP+CA9PbTqxe8/Tb86U9QVgYdOsDddzur9xhTnx0ojc+Sugsb0ujt73+Ha67xZ18HHwz33ef8GBOP\nVere7IxSFzak0d1nn8G8edU88MAvKCvL/vzmZt9llXp8Vqm7sPaLuxtvXM7Wrf9lzpyzgMQntjLG\nT1Z0xWaVuotCbr9s3AhjxsCAAXDttbBhg3/7VoWnny5h1672FPJKPKZwRSdzq9wbsqTuolDbLx9/\nDCecADt2wPXXw/btzrDBlSv92f/8+bBrVzPgK392aEwSLInHZ+0XF4XYfoksH3fVVXWLJJeVOdPe\nnnkmvP9++tOfPv44DB36HTNnjsvr5dNM8SrEYiub0q7URaStiEwXkaUiskRETvQjsFwrxPbL738P\n3/++M3d3tCuvhIED4Te/SW//O3fCk0/CjTf2YMaMKZSWzqK0dJb1003WWKUenx+V+l+Af6vqT0Wk\nCbCfD/vMuUKr1JcsgSeecM5sjPXGv/de6NsXXnrJqdpTMWcOHHqoM7a8V6/CX4nHFCar1L2lVamL\nSBvgVFWdBKCqO1V1sy+R5Vih9dTHjXPm7m7XLvb9LVs6E0xdfXXqZ8pOngwXX5xyiMakzSr1+NJt\nv3QHvhKRR0XkfRGZKCIJT8KazwqpUl+4EN57D/7nf7y3O+cc6NQJHnoo+efYtAleeAHOPz+1GI3x\nWyEVXdmUblJvAhwH/E1VjwO2AtenHVUeKKSe+p13Omd3zp0boqxsuOsJQSLOWZu33OIs4pCMqVOd\n+cEPPNCnoI1JkQ1p9JZuT30NsEZVIzNqTydGUq+oqNhzORAIEAgE0nzazCuU9ssnnzizBA4f/tJe\nKwa5nRD0/e87K/L86U/OHCuJmjTJ+eNhTC4VcxIPh8OEw+G09yOaZuYSkVeAy1R1uYhUACWqOi7q\nfk33OXJh4kRnoqmJE3Mdyd5rdpaXX75Xor7iCmdSrXfeGU5l5RAiKwaBMzpl9uxnGuzvyy+d5P7u\nu9C9e/znf/ttZyGIjz92psw1JldmzoRHH4XnnnNmyNyypXjXfxURVDXpP2N+nHx0FfCEiCwAjgb+\n5MM+cy5f2i+RNTsrK4dQWTmEoUNH7GmtrFsH06bVjUmvb8OGdTHbMZ06OY/5/e8Ti+Gee5ztLaGb\nXCvmSt0vaQ9pVNUFwA99iCWvZOtA6aJFTlujXTtn9Er79nvfX3/Nzqoq57ZgMMh99zlLpx18cMNl\n45o1u4bFi5tSXT0aaNiOKS+HI4+Et95yzkB1s3KlM+d5OvOmG2Oyx6YJcNG4ceZXPlq9GkpL4bjj\nnOsnnQTr1yf22E2bnEQ7dqxzPRgM7nVCUJ8+x1BdfTdu87Pst59zwPTaa72PHfy//+e0eNI9E9UY\nvxRgNzerbJoAF5lK6tH98erqB7jiik6MGePc16qVcxAzHIZmzZzbYi3cXF4+hfvvd7Y97LC6fUcv\nG1dWFn+ViREjnCkEHnkERo9ueP/77ztV+vLlqf62xvgruv1iyd1FKgubJvNDgS48PXWq6i9/6e8+\n6xZVnqzwtMIGffbZyj33796tevbZqmPHNnxc9MLNmzertmun+tFHiT7XZC0p6RBzAefFi1UPOkh1\n4cK9b//2W9XvfU/1scfS+Y2N8ddzzzmfEVXVpk1Vd+zIbTyZRIoLT1tSdzFtWt2q5X4pLR1Wm2S1\n9meplpYO22ub9etVO3dWrax02Ymqjh+vOmJEw2RfX7z7I6ZNU+3USfW995zrW7Y4H5xLL03ltzQm\nc2bNsqQe78faLy6y0VOHTxvc0r49TJnitEbmz4f33997OGP37kEefhjuv/8/ccelR7djvPziF87v\nGwxCjx7O2Pdzz3VaM8bkG2u7eLOk7iITSb2uP94CGEqLFpdRXj6pwXZnngkXXgiDB29g8eLRbN9+\nCwCvvvprDjlkPjfdtD+TJz/gOiomFT/9KQwaBPPmQdeue/fqjckXNqQxPhv94sLvpB45QPq9732P\nww5bSvv285k5c5JrEr71Vli9egXbt79BZATL9u2vsm3be1xxhX9xRWvVCk491RK6MYXMKnUXjRv7\nN049cgJRpFXSuHETRo9uTTDY3/UxTZtC3773MGfOH4EutbduonfvhxA5w3VUjDHFztov3iypu2jU\nyL9Kvf4JRLt2fUs4fAVlZduAhqf+R1x33SjeeKN0zx+DkpJxXHedk7gj49Lr+u22UIUpfjakMT5L\n6i4ye6C0MR9++DzLlt0LuE++FS9xJ3og1JhiYrM0erOk7sLPpL53q+QQoCmq95LIQU5L3MbUsSQe\nnx0odeFnUo8+hb9Hj684+ODP/dmxMcbUY5W6Cz8PlEJdxX3uudCnz3zuvXeQHeQ0JgXWS/dmSd2F\nnwdKo737Ltx777Gcdpod5DQmWdZ+ic+SuotMHChdvx62bYNu3aB7d+uVG5MKq9S9WU/dRSaS+uLF\nzopDVm0Ykxob0hifL0ldRBqLyDwR+acf+8sHmUjqS5ZA797+7tOYfY0NafTmV6V+NbAEKJq/nX4f\nKAVYuhSOOsrffRqzL7EkHl/aSV1EugBnAY8ARfOSZ+JAqVXqxphM86NSvxe4DsiDZZr9Y+0XY/KT\n9dK9pZXUReRsYL2qzqOIqnTwTuqhUIiysuGUlQ0nFAoltL9Nm5wzRzt39jFIY/Yx1n6JL90hjT8C\nhojIWUALYH8ReUxVL47eqKKiYs/lQCBAIBBI82kzzy2p159x0W3elvoi/XR7UxqTnmKt1MPhMOFw\nOO39iPr0ConI6cC1qvqTererX8+RTZ9+CgMGOP9GKysbTmXlECLztoBz+v/s2c947m/iRPjvf2FS\nwzUxjDEJqqyEu+5y/m3cGHbsgCZFeraNiKCqSZeBfo9TL7zs7cLvA6VLltjIF2P8Zt98G/Ltb5yq\nzgXm+rWibFMbAAAOZklEQVS/XHNrv6S6OMWSJc4ydcaY9BTgF/+sKtIvLulzS+qpLk6xdKmNfDEm\nXVaZx2dJ3YXX6JdgMMjatUF+8APntP94tmyBjRtt7U9j/GCVujdL6i68zihdvhwuuQR69oQPP4xf\nPSxbBr16OX16Y0zqrFKPz9KMC68DpU89Bddc4ywO/cYb8fdlB0mN8Y9V6t4sqbvwar/Mnw8nnACD\nB8PLL8ff15Il0KePv/EZsy+yWRrjs6Tuwiupf/AB9O0LgQDMTWC8j03kZUxmWDumIeupu3BL6tu3\nw2efwZFHQocOzkpGu3d798ttzhdj/GMVujer1F24HShds8aZv6VpU2jXDvbfv+FZp9G2bYMvvoAe\nPTIXqzH7CqvM47Ok7sLtQOmaNXDIIXXXjzsO3n/ffT8ffghHHFG8pzIbk21WqXuzpO6iUSPnzVP/\nDfT559ClS931eEnd+unG+Mcq9fgsqbsQiV2tJ1upWz/dGJNNltQ9xDpYumZN7Erd7SuhJXVj/BX5\nrFkbJjZL6h5iHSxdtw46dnQuh0IhRowYzpYt3/DEE+GY+7Ckbox/6rdfrB3TkB2+8xCr/bJhgzPq\nZe/FMr5j1KhJtG+/Y6/JvaqrYdUqZzoBY4w/rEL3ZpW6h1jtl0hSnzDh4dqEPgLoQnX1mD0zN0Ys\nWwbdukHz5tmK2JjiZpV5fJbUPXgl9YYOanDLu+/C8cdnJDRj9llWqXtLO6mLyKEi8h8RWSwiH4jI\nb/0ILB/UT+qqTlI/6CBnsYySknHAFOAZoDHl5Zfv9fh33rGkboyfrFKPz49KvQb4nar2AU4EfiMi\nRTEyO/pAaSgUYuDAC9m9u4q5c0N7FssoLZ3FmWdOpVWr9vTrt/diGVapG2OyLe0Dpaq6Flhbe/k7\nEVkKdAaWprvvXIscKK07KPogsJOhQ0cwY4az4lHkwOiAATBvHkSOk+7YAYsXQ79+uYvfmGJk7Rdv\nvvbURaQb0A94y8/95kqk/VJ3UHQ40JqqqjsbHBStfxLSokXO9AAtW2Y1ZGOKmg1pjM+3IY0i0gqY\nDlytqt9F31dRUbHnciAQIBAI+PW0GeU1/W59xx0H06fXXQ+H4ZRTMhKWMfu0Yq3Uw+Ew4XA47f2I\n+vAKiUhT4F/AC6p6X7371I/nyIXDD4fKSlixItJ+eRI4nJKSE/a0XyLWrnXmeFm/3pnBsbQUrrwS\nzjknd/EbU2xefx3GjnX+FSneBA8gIqhq0t9F/Bj9IsD/AUvqJ/RCFzlQGjko2rv3Ejp3/rhBQgfn\nLNOePZ2VkNaudUa+DBiQo8CNMfssP9ovJwMXAgtFZF7tbeNV9UUf9p1T0WeUBoNBLrssyKpVdQdD\n6xs9Gu6+22nFDB8OrVtnL1Zj9hXFXJ37wY/RL69RpCcx1e+pb94Mbdq4b3/xxfD00/Dcc4ktc2eM\nSY4dGI3P5n7xUD+pb9my97S79TVvDrNnZz4uY/ZlsdY5MHWKssL2S7KVujEms6xSj8+SuoemTWHn\nzrrrltSNyT2r0r1ZUvfQtKkzfW6EJXVjcssq9fgsqXto2hRqauquW1I3xuQ7S+oeYiX1/ffPXTzG\nGGu/xGNJ3UOzZtZ+MSafWPslPkvqHqz9Ykz+sSGN3iype4hO6tXVzmWbddGY3Imu1K1qj82Suofo\npL5li1Ol2xvJGJPPLKl7iO6pW+vFmPxgrRdvltQ9RFfqNvLFmNyzb8rxWVL3UD+pW6VuTO5Zpe7N\nkroHS+rG5Ber1OOzpO4huqe+ZYu1X4zJBzak0ZsldQ+xRr8YY3LHhjTG58dydoNEZJmIfCQi4/wI\nKl9YUjfGFJq0krqINAYeAAYBvYHzROQoPwLLB82a2egXY/KNtV68pVup9wdWqOpKVa0BpgHnpB9W\nfoieetd66sbknrVc4kt3ObtDgNVR19cAJ6S5z7wQCoWYNOlLtm8/iIEDm7FlS9CSujF5wCp1b+km\n9YRe3oqKij2XA4EAgUAgzafNrFAoxNChI6iqeg5ow9ChAY4+eiH7739wrkMzZp9WzJV6OBwmHA6n\nvR/RNP7siciJQIWqDqq9Ph7Yrap3Rm2j6TxHLpSVDaeycggwovaWKbRtezwzZ/bh9NNzGZkx+7Z5\n82DkSHjnHWeh9+g1hIuNiKCqSf8ZS7en/i7QU0S6iUgz4BfArDT3mZd27mxp7Rdj8kCkRizmqj0d\naSV1Vd0JXAmEgCXAU6q61I/Acqm8/HJKSsYBrwIfU1IyjlatOlhSNybHLJHHl/Y4dVV9QVV7qeoR\nqnq7H0HlWjAYZMaMKfTt+w4dOqxlxowp1NRYpW5MPiiwbm7WpXugtGgFg0G2bg3y+ONQVmZDGo3J\nB1apx2fTBHiInFG6Ywc0auQcmDHG5JZV6t4sqXuIJHU7m9SY/GCVenyW1D1Ekrq1XozJL1atu7Ok\n7iEy9a5N5mVM/rAhjd4sqXto2RK2brVK3Zh8YYk8PkvqHlq3hm+/taRuTD6x1os3S+oeIkndDpQa\nkx+sUo/PkrqH6ErdeurGmEJgSd3Dfvs5Y9Q3bbJK3Zh8Ye0Xb5bUPYg4if2zz+CAA3IdjTEm0n6x\nxO7OknocrVvDihXQsWOuIzHGgA1pjMeSehyRpN6hQ64jMcZYIo/PknocrVvD559bUjcmX1jrxZsl\n9That3b+taRuTO5ZpR6fJfU4WrZ0/m3XLrdxGGNMItKaT11E7gbOBqqBj4FLVXWzH4Hli/POgy++\ngCY287wxecHaL97SrdRnA31U9RhgOTA+/ZDyywUXwPvv5zoKYwzYkMZEpLtGaaWq7q69+hbQJf2Q\njDHGnQ1p9OZnT30k8G8f92eMMXuxRB5f3E6xiFQCsU69+b2q/rN2mxuAalWdGmsfFRUVey4HAgEC\ngUAqsRpjTNEKh8OEw+G09yOaZnNKRC4BRgMDVXV7jPs13ecwxhiADz+En/wEFi1y5mPasSPXEWWO\niKCqSX83SXf0yyDgOuD0WAndGGP8ZO2X+NLtqf8VaAVUisg8EfmbDzEZY4wr++LvLa1KXVV7+hWI\nMcbEY0Ma47MzSo0xBcWGNHqzpG6MKRiWyOOzpG6MMUXEkroxpqBYP92bJXVjTMGw9kt8ltSNMQXF\nKnVvltSNMQXDhjTGZ0ndGFOQrBUTmyV1Y0xBsSrdmyV1Y0zBsOo8PkvqxpiCYpW6N0vqxpiCYZV6\nfJbUjTEFxSp1b5bUjTEFw4Y0xmdJ3RhTkKwVE1vaSV1EykVkt4gc6EdAxhjjxap0b2kldRE5FCgF\nVvkTTv7xYyHYXLL4c6uQ48/H2JOpzvMx/mxIt1K/BxjrRyD5qtDfGBZ/bhVy/Pkae6KVer7Gn2kp\nJ3UROQdYo6oLfYzHGGNcWR89Ps81SkWkEugY464bgPFAWfTmPsZljDExWU/dm2gKr5CIfB+YA2yr\nvakL8DnQX1XX19vW/guMMSYFqpp0sZxSUm+wE5FPgR+o6qa0d2aMMSZlfo1Tt2rcGGPygC+VujHG\nmPzg+xmlInKgiFSKyHIRmS0ibWNsc6iI/EdEFovIByLyW7/jSJaIDBKRZSLykYiMc9nm/tr7F4hI\nv2zH6CVe/CJyQW3cC0XkdRE5OhdxxpLIa1+73Q9FZKeIDMtmfPEk+N4JiMi82vd7OMshekrgvdNO\nRF4Ukfm18V+SgzBjEpFJIrJORBZ5bJPPn1vP+FP63Kqqrz/AXcDY2svjgDtibNMROLb2civgQ+Ao\nv2NJIubGwAqgG9AUmF8/HuAs4N+1l08A3sxVvCnGfxLQpvbyoHyJP5HYo7Z7GfgXMDzXcSf52rcF\nFgNdaq+3y3XcScZfAdweiR3YCDTJdey18ZwK9AMWudyft5/bBONP+nObiblfhgBTai9PAc6tv4Gq\nrlXV+bWXvwOWAp0zEEui+gMrVHWlqtYA04Bz6m2z5/dS1beAtiLSIbthuoobv6r+V1U31159C2fE\nUj5I5LUHuAqYDnyVzeASkEj85wPPqOoaAFXdkOUYvSQS/5fA/rWX9wc2qurOLMboSlVfBb722CSf\nP7dx40/lc5uJpN5BVdfVXl4HeL6AItIN5y/VWxmIJVGHAKujrq+pvS3eNvmSGBOJP9oo4N8ZjShx\ncWMXkUNwEs1DtTfl04GgRF77nsCBtS3Hd0XkoqxFF18i8U8E+ojIF8AC4OosxeaHfP7cJiuhz63n\nyUdu4pyUtIeqqtc4dRFphVN9XV1bsedKokmi/pjRfEkuCcchImcAI4GTMxdOUhKJ/T7g+tr3k5Bf\nJ7olEn9T4DhgINAS+K+IvKmqH2U0ssQkEv/vgfmqGhCRHkCliByjqt9mODa/5OvnNmHJfG5TSuqq\nWurx5OtEpKOqrhWRTsB6l+2aAs8Af1fVmanE4aPPgUOjrh+K8xfda5vICVf5IJH4qT3IMhEYpKpe\nX1mzKZHYfwBMc/I57YDBIlKjqrOyE6KnROJfDWxQ1SqgSkReAY4B8iGpJxL/j4DbAFT149rzUnoB\n72YlwvTk8+c2Icl+bjPRfpkFjKi9PAJokLBrq63/A5ao6n0ZiCFZ7wI9RaSbiDQDfoHze0SbBVwM\nICInAt9EtZlyLW78ItIVeBa4UFVX5CBGN3FjV9XDVbW7qnbH+Wb36zxJ6JDYe+c54BQRaSwiLXEO\n2C3JcpxuEol/GXAmQG0/uhfwSVajTF0+f27jSulzm4GjuQcCLwHLgdlA29rbOwPP114+BdiNc6R9\nXu3PoBwfhR6MMwpnBTC+9rZfAb+K2uaB2vsXAMflMt5k4wcewRm1EHm93851zMm89lHbPgoMy3XM\nKbx3rsUZAbMI+G2uY07yvdMO+Gft+34RcH6uY46K/UngC6Aa5xvRyAL73HrGn8rn1k4+MsaYImLL\n2RljTBGxpG6MMUXEkroxxhQRS+rGGFNELKkbY0wRsaRujDFFxJK6McYUEUvqxhhTRP4/K7+KbFTY\n0AgAAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x108ad8f90>"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "When we increase the degree to this extent, it's clear that the resulting fit is no longer reflecting the true underlying distribution, but is more sensitive to the noise in the training data. For this reason, we call it a **high-variance model**, and we say that it **over-fits** the data."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Just for fun, let's use IPython's interact capability (only in IPython 2.0+) to explore this interactively:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from IPython.html.widgets import interact\n",
      "\n",
      "def plot_fit(degree, Npts):\n",
      "    X, y = make_data(Npts, error=1)\n",
      "    X_test = np.linspace(-0.1, 1.1, 500)[:, None]\n",
      "    \n",
      "    model = PolynomialRegression(degree=degree)\n",
      "    model.fit(X, y)\n",
      "    y_test = model.predict(X_test)\n",
      "\n",
      "    plt.scatter(X.ravel(), y)\n",
      "    plt.plot(X_test.ravel(), y_test)\n",
      "    plt.ylim(-4, 14)\n",
      "    plt.title(\"mean squared error: {0:.2f}\".format(metrics.mean_squared_error(model.predict(X), y)))\n",
      "    \n",
      "interact(plot_fit, degree=[1, 30], Npts=[2, 100]);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEKCAYAAADticXcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGPZJREFUeJzt3Xu4XHV97/H3h+yAuUAujSbkAkEqSAUCtAKmejItF8NF\nsLTWe0GPba0e8bQgGNCy26e2Vmu19fT4PEXFYFup5aLACYagDFAFIkpIQoIBJUC4hFsIBBJIsr/n\nj7V2spjM3ntm1uw9k18+r+dZz6zb/NZ3Jns+85vfmslSRGBmZmnYq9MFmJlZ+zjUzcwS4lA3M0uI\nQ93MLCEOdTOzhDjUzcwS4lA3G4CkiqRHOl2HWTMc6mYJkXSCpPskvSjpR5IOGGTfyZKukbRJ0lpJ\n7221LeseDnXbY0jq2N977bEl9TR5/yH3lzQFuAq4GJgE3AX85yB3+RdgC/A64P3A1yT9RottWZdw\nqCcm73GdL2m5pBckfUPSVEk3SNooaYmkiYX9j5f0E0kbJC2TNK+w7UOSVkl6XtIvJf1JYVtF0jpJ\nfyFpvaTHJJ0zSF3n5G08L+lXkt6Xrx8l6R8kPZVv/7ikvv4QzB/PCYV2eiV9u7D8X5Iel/ScpFv6\nQynf9i1JX5O0SNImoCJpuqSrJD2Z1/GJwv5j8vs8K+le4M1DPNdvzJ/PZ/Ie7bsGOfbv5I/lAknL\ngRfyx36GpHvz5/9mSW+s+bcs7j/U6/UsYGVEXBURrwC9wBxJh9SpfVy+/2cj4qWI+DHwfeCDzbZl\nXSYiPCU0AQ8CPwFeC0wH1gM/B+YA+wA/BP4y33cG8DQwP18+MV/+tXz5VOCgfP5/AC8CR+fLFWAr\n2Yt9FHBKvn1CnZrGARuBN+TLU4HfyOc/CqzOa5kE3AxsB/YqPJ7fLbR1CfDtwvI5efujgS8Ddxe2\nfQt4DnhLvjwG+BnwGaAHOAj4JXByvv3zwC3ARGAmsBJ4eIDneRzwCHA2WefoKOAp4LABjr1P/lh+\nnj/WfYBDgE3ACflz+CngfqAnv8/a4v75unuA9wxQ0z8B/1KzbjlwVp19jwZerFn3F8C1zbblqbsm\n99TT9NWIeCoiHgNuA26PiHsi4mXgGrIXNMAHgEUR8QOAiLiJ7GP2afnyooh4MJ+/FbgReFvhOFuB\nv46I7RFxA1lAHTpATX3AEZLGRMT6iFiVr/9D4MsR8WhEbAD+FlCjDzQivhURL0bEVuCvyHqT+xZ2\n+V5E3J7PHwlMiYi/iYht+WP7OvCefPu7gM9FxHMRsY4s2Aaq5XTgwYhYGBF9EbEMuDpvY5dj5889\nwD/nj/Vl4N3A9RHxw4jYDvwD2RvP3P6HV7M/ETEnIq4YoKZxwPM1654HxtfZd3ydfV8A9h1k+0Bt\nWRdxqKdpfWF+c83yFna+MA8E3pV/9N8gaQPw28A0AEmnSLojH17YQNZz/7VCW89ERF9h+SXqvOgj\n4kWyAPso8Jik6yX1h//+ZD3efg83+iDz4YvPS3pA0kaynjDAlP5DA+sKdzkQmF7zeBeQjSlD9smm\n0VoOBI6raet9ZJ9C+o9d75szxXX7F48REf33mTHA/kPZBOxXs24CWVg3um9/kL/QRFvWRRzqe4aB\nepsPkw1lTCpM+0bEFyTtQ3ai7AvA6yJiErBokLYGFRE3RsTJZG8Y9wGX5pseB4rfqqj9hsWLZD3Q\nfvuTBSZkIXoGcEJETCAbTqGmxuJ/Q/owWe+6+Hj3i4jTG6yl6GHgljrP3ccHuU9tPY+RvTlkRUsC\nZgGPDrD/UO4lG2brb28ccHC+vtYaoEfSrxfWzSns20xb1kUc6nu2fwPeIenkvNf7mvwE6Axg73x6\nGuiTdApwcisHkfQ6SWfmwbCVLKi355u/C5wraYakScCneXWQLQPeI6lH0m8Bv1/YNh54GXg2b/tv\naw9ds7yU7ITjBflJ0VGSDs/b7a9lgaSJkmYCn2Bg1wOHSPqApNH59ObCic5G3vy+C5wm6XcljQbO\nI/sk9ZMG7lvPNcDhks6S9Bqy8w/LImJN7Y75p6ergb+WNFbSW4F3AN9uti3rLg71PUPUzAdAPm58\nJnAR8CRZ7/M8QBHxAnAuWfA8C7yX7NsRA7U7mL2APyfrgT5DNi7/Z/m2S4HFZCcA7yL7dFAMxM+S\n9RA3kJ2U/ffCtsuBh/J2VwK3D/RY88fbRzYWfhTwK7ITm//KzmGGv8rbexD4Qd5+3ccYEZvI3uTe\nkx//ceDvyN4Idzn2AG2sITuv8dW8ltOAd0TEtoHuI2mlar5PXmjvabI3vc+R/Zv9FjvPFyDpIkmL\nCnf5GNkY/pNkb/AfjYjVjbRl3UvZMN4QO0nfJPuDezIijqjZdh7wRbITUM8OS5W2x5A0myxwe2rG\n682sAY321C8D5teulDQLOImsd2NmZh3WUKhHxG1kH39r/SNwQVsrMmvu5KCZFTT1U+UiSWcC6yJi\neXbS3qy8iFhL9kMcM2tBS6EuaSzZybWTiqvbUpGZmbWs1Z76wcBs4J68lz4T+JmkYyPiyeKOkvxR\n2sysBRHRdGe5pa80RsSKiJgaEQdFxEFkv9o7pjbQC/vvttMll1zS8Rpcf+fr2BPr351rT6H+VjUU\n6pK+Q/aDiEMkPSLpQ7W53XIFZmbWNg0Nv0RE3R87FLa/vj3lmJlZGf5F6RAqlUqnSyjF9XfW7lz/\n7lw77P71t6qhX5SWOoAUw30MM7PUSCJG6kSpmZl1J4e6mVlCHOpmZglxqJuZJcShbmaWEIe6mVlC\nHOpmZglxqJuZJcShbmaWEIe6mVlCHOpmZglxqJuZJcShbmaWEIe6mVlCGg51Sd+UtF7SisK6L0pa\nLekeSVdLmjA8ZZqZWSOa6alfBsyvWXcj8KaImAOsARa0qzAzM2tew6EeEbcBG2rWLYmIvnzxTmBm\nG2szM7MmtXNM/cPAoja2Z2ZmTWrowtNDkXQx8EpE/Ee97b29vTvmK5XKHnvtQDOzgVSrVarVaul2\nmrpGqaTZwHURcURh3TnAHwMnRMSWOvfxNUrNzJrU6jVKS/XUJc0HPgXMqxfoZmY2shruqUv6DjAP\nmAKsBy4h+7bL3sCz+W63R8THau7nnrqZWZNa7ak3NfzSCoe6mVnzWg11/6LUzCwhDnUzs4Q41M3M\nEuJQNzNLiEPdzCwhDnUzs4Q41M3MEuJQNzNLiEPdzCwhDnUzs4Q41M3MEuJQNzNLiEPdzCwhDnUz\ns4Q41M3MEuJQNzNLSEOhLumbktZLWlFYN1nSEklrJN0oaeLwlWlmZo1otKd+GTC/Zt2ngSURcQjw\nw3zZzMw6qKFQj4jbgA01q88AFubzC4F3trEuMzNrQZkx9akRsT6fXw9MbUM9ZmZWQk87GomIkDTg\n1aV7e3t3zFcqFSqVSjsOa2aWjGq1SrVaLd2OIgbM4lfvKM0GrouII/Ll+4BKRDwhaX/g5oh4Y537\nRaPHMDOzjCQiQs3er8zwy7XA2fn82cD3SrRlZmZt0FBPXdJ3gHnAFLLx878Evg98FzgAWAv8YUQ8\nV+e+7qmbmTWp1Z56w8MvrXKom5k1rxPDL2Zm1mUc6mZmCXGom5klxKFuZpYQh7qZWUIc6mZmCXGo\nm5klxKFuZpYQh7qZWUIc6mZmCXGom5klxKFuZtakRx+FX/yi01XU51A3MxvCtm3w4x/DxRfDUUfB\nkUfCDTd0uqr6/L80mpnV6OuDlSuhWoWbb4Zbb4WZM+G007LpuOOgpy3XjRuY/+tdM7MWbdkCy5bB\nnXdmAX7LLTB5MlQq2TRvHsyYMbI1OdTNzBqwfTvcfz8sXZqF+NKlsGoVHHooHHssvO1tWZCPdIjX\n6lioS1oAfADoA1YAH4qIlwvbHepm1hFPPw0rVsDy5TunVatg6tQswI89NhtKOfpoGDu209W+WkdC\nPb8Y9Y+AwyLiZUn/CSyKiIWFfRzqZjZstm+Hhx6CNWtePd17L2zalJ3UPPJIOOKI7Pbww2G//Tpd\n9dBaDfWyQ/3PA1uBsZK2A2OBR0u2aWb2Khs3wtq1WXg/9FA2/8ADWXg/+CBMmwaHHLJzOv10OOww\nOOAAUNOxuHsrFeoR8aykLwEPA5uBxRFxU1sqM7M9wgsvwOOPw2OP7bx9+OGdIb52bdYbP/DAbJo9\nO7udOzcbBz/4YBgzpsMPoouUCnVJBwP/G5gNbAT+S9L7I+Lfi/v19vbumK9UKlQqlTKHNbMu99JL\n2Xj2U0/tvH3iiV3D+/HHs68PTp8O+++/83b27OxkZX+QT56cfo+7Wq1SrVZLt1N2TP3dwEkR8ZF8\n+YPA8RHx8cI+HlM3G2aLFy/mS1/6VwDOO+9PePvb396WdrduzYY+nnvu1dMzz2RhXRvc/eu2b4fX\nvhamTNl5O23aruE9fTrsu2/6gd2KTo2p3wd8VtIYYAtwIrC0ZJtm1oTFixfze793Nps3/z0A//3f\nZ3PllZczd+7JbNqUDW9s2rRz6l9+/vldw7p22rIFJkyAiRNfPU2alIX1rFnZN0f6g7s/xMeNc1B3\nSju+0ngBcDbZVxp/DnwkIrYWtrunbqUNV0+0E/r6YPPmbIhi8+Zdp4HWF7cVw3np0vvYuHEqMCk/\nwjYksd9+oxg/PusJjx9P3flJk3YN7OLkcO4c//jIklXbEx0z5kKuuWbhkMHe15cNH2zblt32z7/8\nMrzySnZbnB9sXbP790/1gnnr1uzEXu00dmzj64rh/JnPXMRPf3occGb+yL/NiSd+jyVLrhrefxgb\nVp0afrEOisiCa/v21qey92+mjf5wrb2tt654e/fds9i8eTnwOgA2bz6Ts856ialTB79vBIwenU09\nPTtv99knm/be+9W3ja4bN67x/esF8z77tLf3GzEvf9N7Dsje9M4/f+EQ97JU7dY99TvugCuuyEKl\nr29nyLVjuZ1ttftYxUmCUaNan/baq9z9m2mnNlzr3dZbd+GFn+Wuu94CnJr/y3+fuXNv4vLLvzrg\nfXp6smPuKVIanrLMHjn8snIl3HRTFihSdts/lVluZ1vDdaz+IN0TxjtbHX4x253tkaFuew73RG1P\n41A3M0tIq6HuKx+ZmSXEoW5mlhCHuplZQhzqZmYJcaibmSXEoW5mlhCHuplZQhzqZmYJcaibmSXE\noW5mlpDSoS5poqQrJa2WtErS8e0ozMzMmteO/0/9n4BFEfEHknqAcW1o08zMWlD2wtMTgLsj4vWD\n7OP/0MvMrEmd+g+9DgKeknSZpJ9LulTS2JJtmplZi8qGeg9wDPB/I+IY4EXg06WrMjOzlpQdU18H\nrIuIn+bLV1In1Ht7e3fMVyoVKpVKycOamaWlWq1SrVZLt1P6IhmSbgU+EhFrJPUCYyLiwsJ2j6mb\nmTWpY1c+kjQH+DqwN/BL4EMRsbGw3aFuZtYkX87OzCwhvpydmZk51M3MUuJQNzNLiEPdzCwhDnUz\ns4Q41M3MEuJQNzNLiEPdzCwhDnUzs4Q41M3MEuJQNzNLiEPdzCwhDnUzs4Q41M3MEuJQNzNLiEPd\nzCwhbQl1SaMk3S3puna0Z2ZmrWlXT/2TwCrAlzgyM+ug0qEuaSZwKtl1Spu+9JKZmbVPO3rqXwY+\nBfS1oS0zMyuhVKhLOh14MiLuxr10M7OO6yl5/7nAGZJOBV4D7Cfp8oj4o+JOvb29O+YrlQqVSqXk\nYc3M0lKtVqlWq6XbUUR7zm1KmgecHxHvqFkf7TqGmdmeQhIR0fQISLu/p+70NjProLb11Ac8gHvq\nZmZN65aeupmZdZBD3cwsIQ51M7OEONTNzBLiUDczS4hD3cwsIQ51M7OEONTNzBLiUDczS4hD3cws\nIQ51M7OEONTNzBLiUDczS4hD3cwsIQ51M7OEONTNzBJSOtQlzZJ0s6R7Ja2UdG47CjMzs+aVvvKR\npGnAtIhYJmk88DPgnRGxOt/uKx+ZmTWpY1c+iognImJZPr8JWA1ML9uumZk1r61j6pJmA0cDd7az\nXTMza0xPuxrKh16uBD6Z99h36O3t3TFfqVSoVCrtOqyZWRKq1SrVarV0O6XH1AEkjQauB26IiK/U\nbPOYuplZk1odU2/HiVIBC4FnIuLP62x3qJuZNamTof5W4FZgOdDf2IKI+EG+3aFuZtakjoX6kAdw\nqJuZNa1jX2k0M7Pu4VA3M0uIQ93MLCEOdTOzhDjUzcwS4lA3M0uIQ93MLCEOdTOzhDjUzcwS4lA3\nM0uIQ93MLCEOdTOzhDjUzcwS4lA3M0uIQ93MLCEOdTOzhJQOdUnzJd0n6X5JF7ajKDMza02pKx9J\nGgX8AjgReBT4KfDeiFhd2MdXPjIza1Knrnx0LPBARKyNiK3AFcCZJds0M7MWlQ31GcAjheV1+Toz\nM+uAnpL3b2hcpbe3d8d8pVKhUqmUPKyZWVqq1SrVarV0O2XH1I8HeiNifr68AOiLiL8v7OMxdTOz\nJnVqTP0u4A2SZkvaG3g3cG3JNs3MrEWlhl8iYpuk/wUsBkYB3yh+88XMzEZWqeGXhg7g4Rczs6Z1\navjFzMy6iEPdzCwhDnUzs4Q41M3MEuJQNzNLiEPdzCwhDnUzs4Q41M3MEuJQNzNLiEPdzCwhDnUz\ns4Q41M3MEuJQNzNLiEPdzCwhDnUzs4Q41M3MElIq1CV9UdJqSfdIulrShHYVZmZmzSvbU78ReFNE\nzAHWAAvKl2RmZq0qFeoRsSQi+vLFO4GZ5UsyM7NWtXNM/cPAoja2Z2ZmTeoZagdJS4BpdTZdFBHX\n5ftcDLwSEf9Rr43e3t4d85VKhUql0kqtZmbJqlarVKvV0u0oIso1IJ0D/DFwQkRsqbM9yh7DzGxP\nI4mIULP3G7KnPsRB5wOfAubVC3QzMxtZpXrqku4H9gaezVfdHhEfq9nHPXUzsya12lMvPfwy5AEc\n6mZmTWs11P2LUjOzhDjUzcwS4lA3M0uIQ93MLCEOdTOzhDjUzcwS4lA3M0uIQ93MLCEOdTOzhDjU\nzcwS4lA3M0uIQ93MLCEOdTOzhDjUzcwS4lA3M0tI6VCXdJ6kPkmT21GQmZm1rlSoS5oFnAQ81J5y\nuk87LgTbSa6/s3bn+nfn2mH3r79VZXvq/whc0I5CutXu/ofh+jtrd65/d64ddv/6W9VyqEs6E1gX\nEcvbWI+ZmZXQM9hGSUuAaXU2XQwsAE4u7t7GuszMrAUtXXha0uHAD4GX8lUzgUeBYyPiyZp9fdVp\nM7MWtHLh6ZZCfZdGpAeB34yIZ0s3ZmZmLWvX99TdGzcz6wJt6ambmVl3aPsvSiVNlrRE0hpJN0qa\nWGefWZJulnSvpJWSzm13Hc2SNF/SfZLul3ThAPv8c779HklHj3SNgxmqfknvz+teLunHko7sRJ31\nNPLc5/u9WdI2SWeNZH1DafBvpyLp7vzvvTrCJQ6qgb+dKZJ+IGlZXv85HSizLknflLRe0opB9unm\n1+2g9bf0uo2Itk7AF4AL8vkLgc/X2WcacFQ+Px74BXBYu2tpouZRwAPAbGA0sKy2HuBUYFE+fxxw\nR6fqbbH+twAT8vn53VJ/I7UX9vsRcD3w+52uu8nnfiJwLzAzX57S6bqbrL8X+Lv+2oFngJ5O157X\n8zbgaGDFANu79nXbYP1Nv26H4/9+OQNYmM8vBN5Zu0NEPBERy/L5TcBqYPow1NKoY4EHImJtRGwF\nrgDOrNlnx+OKiDuBiZKmjmyZAxqy/oi4PSI25ot3kn1jqRs08twDfAK4EnhqJItrQCP1vw+4KiLW\nAUTE0yNc42Aaqf9xYL98fj/gmYjYNoI1DigibgM2DLJLN79uh6y/ldftcIT61IhYn8+vBwZ9AiXN\nJnununMYamnUDOCRwvK6fN1Q+3RLMDZSf9H/BBYNa0WNG7J2STPIguZr+apuOhHUyHP/BmByPuR4\nl6QPjlh1Q2uk/kuBN0l6DLgH+OQI1dYO3fy6bVZDr9tBf3w0kCF+lLRDRMRg31OXNJ6s9/XJvMfe\nKY2GRO13RrslXBquQ9LvAB8Gfnv4ymlKI7V/Bfh0/vckuuuHbo3UPxo4BjgBGAvcLumOiLh/WCtr\nTCP1XwQsi4iKpIOBJZLmRMQLw1xbu3Tr67ZhzbxuWwr1iDhpkIOvlzQtIp6QtD/w5AD7jQauAv4t\nIr7XSh1t9Cgwq7A8i+wdfbB9+n9w1Q0aqZ/8JMulwPyIGOwj60hqpPbfBK7I8pwpwCmStkbEtSNT\n4qAaqf8R4OmI2AxslnQrMAfohlBvpP65wOcAIuKX+e9SDgXuGpEKy+nm121Dmn3dDsfwy7XA2fn8\n2cAugZ33tr4BrIqIrwxDDc26C3iDpNmS9gbeTfY4iq4F/ghA0vHAc4Vhpk4bsn5JBwBXAx+IiAc6\nUONAhqw9Il4fEQdFxEFkn+z+rEsCHRr72/k+8FZJoySNJTtht2qE6xxII/XfB5wIkI9HHwr8akSr\nbF03v26H1NLrdhjO5k4GbgLWADcCE/P104H/l8+/FegjO9N+dz7N7/BZ6FPIvoXzALAgX/enwJ8W\n9vk/+fZ7gGM6WW+z9QNfJ/vWQv/zvbTTNTfz3Bf2vQw4q9M1t/C3cz7ZN2BWAOd2uuYm/3amANfl\nf/crgPd1uuZC7d8BHgNeIftE9OHd7HU7aP2tvG794yMzs4T4cnZmZglxqJuZJcShbmaWEIe6mVlC\nHOpmZglxqJuZJcShbmaWEIe6mVlC/j+xQQpIRydGqQAAAABJRU5ErkJggg==\n",
       "text": [
        "<matplotlib.figure.Figure at 0x109fdde50>"
       ]
      }
     ],
     "prompt_number": 45
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Detecting over-fitting\n",
      "Clearly, computing the error on the training data is not enough (we saw this previously).  But computing this *training error* can help us determine what's going on: in particular, comparing the training error and the validation error can give you an indication of how well your data is being fit.\n",
      "\n",
      "Let's do this:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "degrees = np.arange(1, 30)\n",
      "\n",
      "X, y = make_data(100, error=1.0)\n",
      "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)\n",
      "\n",
      "training_error = []\n",
      "test_error = []\n",
      "mse = metrics.mean_squared_error\n",
      "\n",
      "for d in degrees:\n",
      "    model = PolynomialRegression(d).fit(X_train, y_train)\n",
      "    training_error.append(mse(model.predict(X_train), y_train))\n",
      "    test_error.append(mse(model.predict(X_test), y_test))\n",
      "    \n",
      "# note that the test error can also be computed via cross-validation\n",
      "plt.plot(degrees, training_error, label='training')\n",
      "plt.plot(degrees, test_error, label='test')\n",
      "plt.legend()\n",
      "plt.xlabel('degree')\n",
      "plt.ylabel('MSE');"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEPCAYAAABCyrPIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VPW9//HXJyEkbAHCEpYE2RdFC0pdQGsUa1HRan/e\nVmstqEVtUdtatdZrK73XLl5QXLDWBcS91hVRFFRI3UBUVpFdUJB9D4SELN/fH2cCISZhJpnJmTnz\nfj4e55EzZ87M+ZwMzCff3ZxziIiIpPgdgIiIxAclBBERAZQQREQkRAlBREQAJQQREQlRQhARESCG\nCcHMcs1slpktMbPPzeyGas7JM7PdZjY/tN0eq3hERKR2jWL43iXAb51zC8ysOfCZmb3tnFta5bz/\nOOcuiGEcIiIShpiVEJxzm5xzC0L7e4GlQKdqTrVYxSAiIuFrkDYEM+sKDAQ+rvKUAwab2UIzm2Zm\nRzdEPCIi8m2xrDICIFRd9CLw61BJobJ5QK5zrtDMzgFeBXrHOiYREfk2i+VcRmaWBrwOvOmcuzeM\n89cAJzjndlQ5rgmXRETqwDkXdrV8LHsZGTAR+KKmZGBm2aHzMLMT8RLUjurOdc4Fdrvjjjt8j0H3\npvvT/QVvi1Qsq4yGAD8DFpnZ/NCx24AuAM65h4GLgV+aWSlQCFwSw3hERKQWMUsIzrkPOEIJxDn3\nIPBgrGIQEZHwaaRyHMjLy/M7hJgJ8r2B7i/RBf3+IhXTRuVoMTOXCHGKiMQTM8NF0Kgc826nIpLc\nQv1GJMai8UezEoKIxJxK+LEVraSrNgQREQGUEEREJEQJQUREACUEEZE6++Uvf8mdd94Z9XP9om6n\nIhJToa6PfodRra5duzJp0iTOPPNMv0Opl5p+x5F2Ow1UCWH6quks37bc7zBEJEHUlqxKS0sbOBr/\nBSohvLz0Zd5d867fYYhIArj88sv5+uuvOf/882nRogVjx44lJSWFSZMmcdRRR3HWWWcB8F//9V90\n7NiRVq1acfrpp/PFF18cfI+RI0fyxz/+EYD8/HxycnK45557yM7OplOnTkyePLlO527fvp3zzz+f\nli1bcuKJJ3L77bdz2mmnxfx3EqiEkNsyl3W71/kdhogkgKeeeoouXbrw+uuvU1BQwI9//GMA3nvv\nPZYtW8b06dMBOO+881i1ahVbt27l+OOP57LLLjv4HmZ22BiAzZs3s2fPHjZs2MDEiRMZPXo0u3fv\njvjc0aNH06JFCzZv3swTTzzBk08+2SAD/IKVEDJzWbdHCUEkkZhFZ6uviqqjMWPG0KRJE9LT0wHv\nL/tmzZqRlpbGHXfcwcKFCykoKPjW6wDS0tL405/+RGpqKueccw7Nmzdn+fLlEZ1bVlbGyy+/zJ//\n/GcyMjLo168fI0aMaJB2mEAlhC4tu/D17q/9DkNEIuBcdLZoyc3NPbhfXl7OrbfeSs+ePWnZsiXd\nunUDYNu2bdW+tk2bNqSkHPpabdq0KXv3Vl0osvZzt27dSmlp6WFx5OTk1OuewhWohJDbUiUEEQlf\nddUwlY8988wzvPbaa7z77rvs3r2bNWvWAIf/pR9JVU4457Zr145GjRqxbt2h77LK+7EUqISQk5nD\nhoINlJWX+R2KiCSA7OxsVq9eXePze/fuJT09naysLPbt28dtt9122PORrEwW7rmpqan86Ec/YsyY\nMezfv59ly5bx1FNPqQ0hUhmNMmiV0YrN+zb7HYqIJIA//OEP3HnnnWRlZfHSSy9960v35z//OUcd\ndRSdO3emf//+nHLKKYedU7WhuLYv7UjOnTBhArt376ZDhw6MGDGCSy+9lMaNG9flFiMSuIFpgx4Z\nxIPnPshJOSfFOCoRCUc8D0xLFL///e/ZsmULjz/+eLXPa2BaDdSwLCKJbvny5SxatAjnHHPnzmXS\npElcdNFFMb9u4NZDUNdTEUl0BQUFXHrppWzYsIHs7GxuuukmLrjggphfN3gJQYPTRCTBDRo0iJUr\nVzb4dYNZZbRHVUYiIpEKXELIzVQJQUSkLgKXELq07KI2BBGROghcQujQvAPbC7dTXFrsdygiIgkl\ncAkhNSWVji068k3BN36HIiKSUAKXEEBjEURE6iKQCUENyyISjq5duzJz5sx6vcfkyZMbZPGahhDI\nhKCGZREJh6bVOFwgE0JuZq6qjESkVlWX0Bw3bhxz5sxh8ODBtG7dmgEDBvCf//zn4PmTJ0+mR48e\nZGZm0r17d5599lmWLVvGtddey+zZs2nRogVZWVk+3lEUVEzJGs+bF2b4piyb4s595tyIXiMisRHp\n/9+G1LVrV/fuu+8655xbv369a9OmjXvzzTedc869/fbbrk2bNm7btm1u7969LjMz061YscI559ym\nTZvckiVLnHPOTZ482Z166qn+3EBITb/j0PGwv2sDN3UFhKqM1IYgkhDsz9GZ59/dUb+qn6effppz\nzz2XYcOGAXDWWWcxaNAg3njjDS6++GJSUlJYvHgxOTk5ZGdnk52d7V03QFVOgUwIqjISSRz1/SKP\nlq+++ooXXniBqVOnHjxWWlrKmWeeSdOmTXn++ecZN24cV111FUOGDOHuu++mT58+PkYcfYFsQ8hq\nksWBsgPsKd7jdygiEscqL1LTpUsXLr/8cnbu3HlwKygo4JZbbgHg7LPPZsaMGWzatIm+ffsyatSo\nb71HogtkQjAzVRuJyBFVXkLzsssuY+rUqcyYMYOysjKKiorIz8/nm2++YcuWLUyZMoV9+/aRlpZG\ns2bNSE1NPfge69evp6SkxM9biYpAJgQITYOtrqciUouKJTRbt27NCy+8wJQpU/jrX/9K+/bt6dKl\nC3fffTfOOcrLyxk/fjydO3emTZs2vP/++zz00EMADB06lGOOOYYOHTrQvn17n++ofgK3hGaFq6Zc\nxck5JzPqhFExikpEwqG+/rGnJTSPILelGpZFRCIR3ISgpTRFRCIS2ISg6StERCIT2ISgKiMRkcgE\nNyFk5rJ+z3o1ZomIhCmwCaFZ42Y0TWvK1sKtfociIpIQAjl1RYWKdRHaN0vsvsEiiS5Io3mDLLAl\nBFDDskg8iGS2zaBvB0oP8Pj8x+k7oS/ffeS7vPzFy5SVl0VrVuh6C3wJQQ3LIuK3wpJCHpv3GOM+\nGkfvNr2ZcM4Ezux2ZtyVnIKdEFpqKU0R8c/O/Tt58JMHeWDuAwzJHcKLP36REzuf6HdYNYpZlZGZ\n5ZrZLDNbYmafm9kNNZx3v5mtNLOFZjYwmjGoykhE/LCxYCO3vH0LPR/oyaodq8gfkc/LP3k5rpMB\nxLYNoQT4rXPuGOBkYLSZ9at8gpmdC/R0zvUCrgYeimYAqjISkYZ290d3c8w/jqGotIh5V89j8oWT\n6deu35FfGAdiVmXknNsEbArt7zWzpUAnYGml0y4Angid87GZtTKzbOfc5mjEoBKCiDSkuz64i4nz\nJ7Lol4vIyczxO5yINUgvIzPrCgwEPq7yVGeg8jf2eiBqv8VOLTqxee9mSstLo/WWIiLVGvfROB6b\n/xizRsxKyGQADdCobGbNgReBXzvn9lZ3SpXH1fafGjNmzMH9vLw88vLyjnjttNQ02jVrx4aCDXRp\n2SXckEVEInLP7Ht4+LOHmTViFp0zO/sWR35+Pvn5+XV+fUzXQzCzNOB14E3n3L3VPP9PIN8596/Q\n42XA6VWrjOqyHkKFUyaewrjvj2NIlyF1er2ISG3unXMvE+ZOYNaIWeS2zPU7nMPEzXoI5nWwnQh8\nUV0yCHkN+Hno/JOBXdFqP6ighmURiZX7P76f+z++n5kjZsZdMqiLWFYZDQF+Biwys/mhY7cBXQCc\ncw8756aZ2blmtgrYB1wR7SDUsCwisfDg3AcZP2c8+SPyA1MlHcteRh8QRgnEOXddrGIAr4SwcsfK\nWF5CRJLMPz75B2M/Gkv+yHyOanWU3+FETaDnMoLQaGWVEEQkSh7+9GHu+vAuZo2YRddWXf0OJ6oC\nPXUFhKqMNH2FiETBo589yl/e/wuzRsyiW+tufocTdYFPCGpUFolfzjn2l+5nx/4d1W479++ksKSQ\n9s3a0zmzM51adDq4tc5o3aCTw02cN5H/fe9/mTliJj2yejTYdRtS4BNCu2bt2HtgL4UlhTRNa+p3\nOCJJ75lFz/D3D//O9sLt7Ni/A4A2TduQ1STr0JZxaL99s/Zs2beFWWtnsaFgA9/s+YYNBRsoKi06\nLEF0atGJjs070iStCY1TG5Oemk7j1MaHbemNDh1LS0mjtLyUotIiisuKKSotOrgVlx7+ePO+zbyy\n7BVm/nwmPbN6+vwbjJ3AJ4QUSyEnM4f1e9bTu01vv8MRSVrlrpw7Zt3B04ufZvIPJ9MzqydZTbJo\nktakTu+378A+Nu7dyIaCDQe3jQUbKSot4kDZAQ6UH/B+lh2guLT44H7lLS01jfTUdDIaZRy2VT3W\nrmk78kfkB7ZkUCHwCQG8huWvd3+thCDik/0l+xk5ZSTrdq/j4198HJVVDJs1bkbPrJ6B/ou9oQW+\nlxGoYVnET5v2biLviTxSLZWZI2ZqSds4lhQJQQ3LIv5YtHkRJz12Euf1Oo9nfvQMGY0y/A5JapEc\nVUaZuXyy4RO/wxBJKm+seIORU0bywDkPcEn/S/wOR8KQFCUETV8h0nCcc9w35z5GTR3F1EunKhkk\nkOQoIbRUlZFIQygpK+GGN2/gg3UfMPuq2YGa1iEZJEVCqGhUds416EAWkWSyq2gXP37hxzRKacSH\nV35IZnqm3yFJhJKiyigzPZPUlFR2Fe3yOxSRQFq7ay2DJw6mX9t+vHbpa0oGCSopEgKop5FIrBSV\nFnHhvy5k5ICR3HfOfTRKSYqKh0BKmoSghmWR2Lhx+o30btObmwff7HcoUk9Jk8pVQhCJvheWvMD0\n1dOZd/U8tc8FQNIkBI1WFomu1TtWM3raaKZdNo2WGS39DkeiIGmqjLRQjkj0FJcW85MXf8Lt37ud\nQZ0G+R2OREnyJARVGYlEze/f+T25LXO5/sTr/Q5Foii5qoxUQhCpt1eXvcqry15l/jXz1W4QMEmT\nEHIyc9hQsIGy8jJSU1L9DkckIX216yuuef0aplwyhdZNWvsdjkRZ0lQZpTdKp1VGKzbv2+x3KCIJ\nqaSshEteuoSbB9/MyTkn+x2OxEDSJARQTyOR+vjvmf9NVpMsbjzlRr9DkRhJqoSghmWRupm2chrP\nff4cT1z4BCmWVF8bSSVp2hBADcsidbF+z3qunHIlL/zXC7Rt2tbvcCSGkirV52bmqspIJAKl5aX8\n9KWfcsNJN3DaUaf5HY7EWHIlhJa5fL1HVUYi4RqTP4aMRhnceuqtfociDSD5qoxUQhAJyztfvsPj\nCx5n3tXz1G6QJJLqU87N1PQVIuFwznHj9Bv553n/JLt5tt/hSANJqoTQoXkHduzfQXFpsd+hiMS1\n6aunY2YM7z3c71CkASVVQkhNSaVj8458U/CN36GIxLWxH43lplNu0tQUSSapEgKEGpY1FkGkRp9t\n+IyV21dySf9L/A5FGljSJQQ1LIvUbuxHY/nNyb8hLTXN71CkgSVdQlDDskjN1uxcwztfvsOo40f5\nHYr4IFAJoawMDhyo/ZwuLbuoykikBuPnjOcXx/+CFukt/A5FfBCohDByJPzrX7WfoxKCSPW2F27n\n6UVPc8NJN/gdivgkUAmhVy9Yvrz2c9SoLFK9f3zyDy7qexGdWnTyOxTxSaASQp8+sGxZ7eeoUVnk\n2/aX7OfBTx7kpsE3+R2K+ChwCeFIJYTWGa0pKS9hT/GehglKJAE8ufBJvtv5u/Rr18/vUMRHgUoI\nvXrB6tVe43JNzEyznopUUlZext2z7+aWwbf4HYr4LFAJoVkzaNcOvvqq9vO0LoLIIVOWTyGrSRan\ndjnV71DEZ4FKCAB9+4bRsKwSggjgTWI39qOx3Dz4Zk1TIcFLCOG0I2gsgojnw3Ufsq1wGxf2vdDv\nUCQO1JgQzOxnlfaHVHnuulgGVR/h9DTKbamxCCLgTVNx48k3kpqS6ncoEgdqKyH8rtL+hCrPXRWD\nWKIinBJCbqbGIogs3bqUOevnMHLASL9DkTiRtFVGKiFIsrt79t2M/u5omqQ18TsUiROBW0IzJwd2\n74Y9eyAzs/pzclvmsn7PepxzakiTpLSxYCMvL32ZFdev8DsUiSO1lRD6mtliM1sM9KnYr3gczpub\n2SQz2xx6TXXP55nZbjObH9pur8M9HCYlBXr3hhW1/DtvmtaUZmnN2Fq4tb6XE0lI9398Pz899qe0\nbdrW71AkjtRWQojGkMXHgQeAJ2s55z/OuQuicK2DKhqWBw2q+Zzcll7X0/bN2kfz0iJxr6C4gEfn\nPcrcUXP9DkXiTI0lBOfc2sobsBc4HmgTenxEzrn3gZ1HOC3qdTbhNiyrHUGS0WPzHmNo96F0b93d\n71AkztTW7fQNM+sf2u8IfA5cATxlZr+N0vUdMNjMFprZNDM7OhpvqrEIItUrKSth/Jzx3Dz4Zr9D\nkThUW5VRV+fc56H9K4AZzrmfm1kL4CNgfBSuPw/Idc4Vmtk5wKtA7+pOHDNmzMH9vLw88vLyanzT\ncBJCr6xeLNmyJOKARRLZ80uep0dWDwZ1qqU+VRJWfn4++fn5dX69Oeeqf8JsgXNuQGh/JvCoc+65\n0OOFzrnvhHUBs67AVOfcsWGcuwY4wTm3o8pxV1Oc1dm7F7KzoaDAa2SuzpItSxj+3HC+vOFL9TSS\npDHokUH8Oe/PnNf7PL9DkQZgZjjnwv6Cq62X0Xozu97MfgQMBN4KXaApUequambZFvo2NrMT8RLU\njiO87IiaN4fWrWFdLU0ER7c7mtLyUlZsV7c7SQ4LNi1ga+FWhvUc5ncoEqdqSwhXAf2BEcBPnHMV\njcMn4fUeOiIzew6veqmPma0zsyvN7BozuyZ0ysXAYjNbANwLXFKXm6jOkaawMDOG9RjGW6veitYl\nReLapPmTGPmdkZqmQmpUY5VRPIm0ygjgV7/yZj69oZblYV/64iUem/8Yb172Zj0jFIlvRaVF5NyT\nw6dXf0rXVl39DkcaSKRVRjVW/ZjZVLxeQNW9mYv22IFoC6dheWj3oYycMpL9Jfs1fF8CbcqyKQzo\nMEDJQGpVW1vAycB64Dng49CxiuQQ98WKPn1g6tTaz2mV0YoBHQbw3lfv8YOeP2iYwER8MGnBJK4a\nGLdzUkqcqK0NoSNwG147wr3A94Gtzrl859x/GiK4+ghnoRxA7QgSeF/t+orPNnymNQ/kiGobqVzq\nnHvTOfdzvNLCKuA/8bwWQmVdusD27V4X1NoM6zmMt1YrIUhwPbHwCS7pf4mqReWIap3+2swyzOz/\nAU8Do4H7gFcaIrD6SkmBnj1rn+QOYGDHgWwv3M7aXWsbJC6RhlTuynl8weNcOfBKv0ORBFDb1BVP\n4XUZHQj8j3Puu865/3XOfdNg0dVTOA3LKZbCD3r+gOmrpjdMUCINaNaaWbTKaMXxHY/3OxRJALWV\nEC4DegG/Bj4ys4JK256GCa9+wkkIEGpHULWRBNDE+RO5coBKBxKe2toQUpxzLWrYalh6Jr6E27B8\ndo+zmbVmFiVlJbEPSqSB7Ny/k2krp3HZcZf5HYokiMAtoVlZuCWEds3a0atNL2avnx37oEQayLOL\nn2VYz2FkNcnyOxRJEEmREMrLj3yuup9K0GjsgUQq0AkhM9PbvgmjGXxYTyUECY4FmxawrXAbQ7sP\n9TsUSSCBTggQfrXRSTknsXbXWjbt3RT7oERibNL8SVwx4ApSLPD/xSWKAv+vJdyG5UYpjRjafSgz\nVs+IfVAiMVRUWsSzi59l5ICRfociCSbwCSHcEgKoHUGCYcqyKQzsOFAT2UnElBAq+UHPHzBj9QzK\nystiG5RIDGnsgdRVUiSE2hbKqSwnM4cOzTvw2cbPYhuUSIx8tesr5m2cx0X9LvI7FElAgU8IXbvC\nli1QWBje+eptJIls8oLJXNL/EjIaZfgdiiSgwCeE1FTo3h1WrgzvfCUESVQVE9lp7IHUVeATAoTf\n0wjg1C6n8vmWz9mxf0dsgxKJsplrZtK6SWsGdhzodyiSoJIiIUTSsJzRKIPvHfU93vnyndgGJRJl\nk+ZrZLLUT9IkhHAblkHVRpJ4Kiay++mxP/U7FElgta2pHBh9+sADD4R//rCew/jr+3/FOYeZHfkF\nIvXgnGNb4TZWbF/B8u3LWbF9BdsKt3F2j7M5t9e5NG/c/Ijv8eziZzmn1zmayE7qJWkSwooV4ByE\n8/3eM6snTdOasnjLYo7LPi72AUpSKCwpZOX2lazYvuKwL//l25djGH3a9qF3m970adOHnMwcHl/w\nOKOmjmJot6FcfPTFDO89nMz06meenzh/IneddVcD35EEjTnn/I7hiMzM1TfO7GyYPx86dQrv/Oum\nXUeXll24Zcgt9bquJB/nHBsKNrBg0wJv2+z9XL9nPT1a9zj4pd+7Te+DSaBNkzbVlkZ37t/Ja8tf\n48WlL/LeV+9x+lGnc/HRF3NBnwtoldEKgPkb53Ph8xey5tdrNHeRHMbMcM6FXc2RNAnhe9+DP/8Z\nzjgjvPNfX/E698y+h5kjZtbruhJspeWlLN+2/Ftf/oYxsONABmQPYEAHb+vVpheNUupeKN9dtJvX\nV7zOi0tfZOaamQzJHcLFR1/MB19/QJeWXRiTNyZ6NyaBoIRQg1Gj4IQT4Nprwzt/74G9dLy7Ixtu\n3ECL9Bb1urYEy8rtK5m6Yiqvr3idOevnkJOZc/BLf0CHAQzsMJAOzTvEtP2poLiAaSunHSw5zP3F\nXI5qdVTMrieJSQmhBuPGwfr1cO+94b/mrCfP4oaTbuCCPhfU69qS2ErKSvhw3YdMXT6V11e+TkFx\nAcN7D2d47+Gc0fUM/cEgcSvShJAUjcrgNSy/+25kr6nofqqEkHy2F27nrVVvMXXFVGasnkGPrB4M\n7zWcZ3/0LAM7DlRdvQRS0iSESEYrVxjWcxgXPHeBup8miZ37d/LUoqd44YsXWLR5EWd0PYPhvYcz\n/gfj6diio9/hicRc0iSEbt1g40YoKoKMMOf9OqbdMRwoO8CqHavo1aZXbAMUXzjn+GjdRzwy7xFe\nW/4a5/Y6l9tOvY0zup2hCeIk6SRNQmjUyJv5dNUq6N8/vNeY2cFqIyWEYKkoDTzy2SOUlJdwzQnX\ncPfZd9O2aVu/QxPxTVJVhEY6hQWE2hFWaxqLIHDOMXvdbEa+OpLu93dnzvo5TDh3AstGL+PGU25U\nMpCklzQlBIhskrsKZ3U/iyunXElRaZGqEBLUjv07eHbxszzy2SMUlRZx9QlXM/b7Y2nXrJ3foYnE\nlaRLCPn5kb2mVUYrjss+jve/ep/v9/h+TOKS6CguLWbZtmUs3rKYz7d8fvDn9sLtDO89nPuG3Ude\n1zx1EBCpQVIlhL594eGHI3/dsJ7DeH7J85zV/Sx9mUSgrLyMhZsXUu7KSbGUg1uqpR72uPLmcFSM\nOanYr+5YSXkJK7evPOyLf82uNXRv3Z3+7ftzbPtjGXX8KI5tfyzdWndTN1GRMCTNwDSA7duhRw/Y\nuTO8Se4qrN+znh/+64dkNcni4eEP071193rHEnSb9m7iZy//jLW71tIqoxXlrvzgVubKDntc7sop\nK/eOmRmGHUy8FftVj6WmpNIzqyf92/X3EkD2sfRp04f0Rul+3rZIXNFI5SNo2xaWLPEmu4tEaXkp\n9865l79/8HduPfVWfnPyb+o1L02Qvb36bUa8OoJRx4/ij6f/Ub8nEZ8oIRzBkCHw17/C6afX7fWr\nd6zmmtevYWfRTh47/zEtV1hJaXkpd8y6g8kLJ/PURU9xZrcz/Q5JJKlFmhCSrmK1Lj2NKuuR1YO3\nL3+b60+8nmHPDOP3b/+ewpLC6AWYoNbtXkfe5Dw+3fgp86+Zr2QgkoCSLiHUZQqLqsyMkQNGsuja\nRXy952uOe+g43v0ywomSAmTq8qkMenQQ5/c+nzcve5P2zdr7HZKI1EHSVe726QPvvRed98puns1z\n/+853ljxBle+diVDuw1l3NnjkmYZwwNlB7j1nVt5eenLvPKTVxicO9jvkESkHpKuhFDfKqPqnNf7\nPD7/5ec0b9ycY/5xDA998hCz1sxiyZYlbN23lbLysuheMA6s3rGaIZOG8OXOL5l3zTwlA5EASLpG\n5QMHIDMTdu+G9Bj0UJyzfg7j54xn095NbNm3hS37trC7aDdtmrahfbP2tG/WnnZN2x3c79C8A73b\n9KZf234JM3L230v+zXXTruP2793O9Sder7EZInFKvYzC0Ls3vPoqHH101N6yViVlJWwr3MbWwq0H\nk0TFtqFgA8u3L2fp1qU0SmlEv3b9OLrt0fRr149+bfvRr10/cjNzj/ilW1xazK6iXUfcdhfvrvZ4\ncVnxt/r81/Qzu1k2z1/8PCd0OqFhfoEiUidaICcMFQ3LDZUQ0lLT6NiiY61z6jvn2LR3E0u3LWXp\n1qUs3baUqSumsnTrUgoOFNCnTR/6tO1DWXlZtV/qZeVltMpodXBrmdGS1hmtDzuW2zL3sMcHz01v\nSXqj9MNGBVf8LHfl3zrWrHEzjS0QCaCk/F8di3aE+jKzg0mjapfNXUW7WLp1KSt3rKRRSqNqv9Sb\nNGqiqhsRqZekTQgffuh3FOFrldGKU3JP4ZTcU/wORUQCLKa9jMxskpltNrPFtZxzv5mtNLOFZtYg\nw37jsYQgIuK3WHc7fRwYVtOTZnYu0NM51wu4GngoxvEAhxbKSYD2dBGRBhPThOCcex/YWcspFwBP\nhM79GGhlZhFOOxe5du28ZLBtW6yvJCKSOPwemNYZWFfp8XogJ9YXNYvOFBYiIkESD43KVbvGVFuR\nM2bMmIP7eXl55OXl1euiFe0Ip55ar7cREYkb+fn55Ee6LGQlMR+YZmZdganOuWOree6fQL5z7l+h\nx8uA051zm6ucF9WBaQD33ef1NPr3v6P6tiIicSPRpr9+Dfg5gJmdDOyqmgxi5Re/gNmz4YMPGuJq\nIiLxL6YlBDN7DjgdaAtsBu4A0gCccw+HzpmA1xNpH3CFc25eNe8T9RICwLPPwj33wNy5kOJ3ahQR\niTLNZRSNcda8AAAMXElEQVQB57wV1EaNgiuuiPrbi4j4SgkhQp98Aj/8oTcuITMzJpcQEfFForUh\n+O6734Wzz/bWWRYRSWZJX0IA2LgRjj0WPv4YevSI2WVERBqUSgh10LEj/O53cNNNfkciIuIflRBC\nioq89REefRSGDo3ppUREGoRKCHWUkQHjxsFvfgOlpX5HIyLS8JQQKrnoImjbFh55xO9IREQanqqM\nqli40Ot1tGwZtG7dIJcUEYkJjUOIgmuv9aqQ7r23wS4pIhJ1SghRsHWr18D83nvQr1+DXVZEJKrU\nqBwF7drBbbfBb3+rVdVEJHkoIdRg9GhYuxamTfM7EhGRhqGEUIPGjb2ZUG+8EQ4c8DsaEZHYU0Ko\nxbnnelNZTJjgdyQiIrGnRuUjWLYMTjsNliyB9u19CUFEpE7UqBxlffvCZZd5aybs2eN3NCIisaOE\nEIa//AU6dIDjjoN33/U7GhGR2FCVUQTeestbi/nCC+Guu6BZM78jEhGpmaqMYmjYMFi8GHbvhu98\nBz780O+IRESiRyWEOnrlFfjVr+Dyy+F//seb6kJEJJ6ohNBALroIFi2CL7+EE06ATz/1OyIRkfpR\nQqiHdu3ghRfg9tvhvPPgT3/SIDYRSVxKCPVkBpdeCvPnw7x5cNJJ3hTaIiKJRgkhSjp1gqlT4frr\n4ayzvGU4n3vOW5pTRCQRqFE5BoqLYcoUeOwxr9Tw05963VWPO87vyEQkmWg9hDizdi08/jhMmgQd\nO8JVV3lVTJmZfkcmIkGnhBCnysrg7be9UsM773i9lH7xCxg82GuHEBGJNiWEBLBlCzz5JEycCCUl\n3iC37t29mVUrti5doFEjvyMVkUSmhJBAnPPGMixfDqtXe9uXX3o/N22C3NzDE0X37l5X11atDm3N\nm6uEISLVU0IIiOJir/2hcqL48kvYvh127Tq0FRUdniAqb+npkJrqlTRSUw9t1T1OiaC/WVqa1waS\nmQktW377Z3q6kpRIPFBCSDIlJd7cShUJYufOQ/vFxV7bRWmp97PyVvlYaWlka0cfOOBNBb5nj3ft\nqj+dO5Qgmjf3EkijRtVvlZ9LTfUSScUGhz+u7lhKSu0/K/abNPFiCWdr0kQJTYJBCUF8V1x8KEEU\nFBxKOqWlXgKr2K9uc+5QcqrYr+2Yc1BefuSfRUWwd2/1W0HB4ftlZdC69eFbVlb1+506QZ8+6jUm\n8UkJQaSeiou9EtaOHV6Jq2Kr7vH69bBihVdF17cv9Ot3+M9OnVTaEP8oIYg0sPJyWLfOW2516dLD\nf+7f7yWGvn3hqKO8KrKUlMPbcKpuKSmHqtMqtsaNq99PS/Oqubp18/u3IPFICUEkjuzY4SWGZcu8\npFFa6iWQqm06FVvFcxXVa5W3Aweqf7xlC5x6Kowf7yUdkQpKCCJJpqgIxo6F++6DG2+E3/3O6+kl\novUQRJJMRgb88Y/wyScwdy4ceyxMn+53VJKIVEIQCZg33oAbboCBA+Gee7xR75KcVEIQSXLnnQdL\nlniz6x5/PPztb17PKZEjUUIQCaCMDG8Fv7lzYfZsLznMmOF3VBLvVGUkkgSmToVf/9pb//v++72p\n2CX4VGUkIt9y/vleNVKfPl7bwiuv+B2RxCOVEESSzOzZ8LOfwRlnwL33egPbJJhUQhCRWp1yCixY\n4A2CGzgQPv7Y74gkXqiEIJLEXnoJfvUrGD0abrtNizIFjUYqi0hENmyAkSO9mV6fftpbjEmCIa6q\njMxsmJktM7OVZvb7ap7PM7PdZjY/tN0ey3hE5Ns6dYK33oJLL4WTT4ZJkyJbH0OCI2YJwcxSgQnA\nMOBo4FIz61fNqf9xzg0MbXfGKp54lp+f73cIMRPke4Pg3F9Kije6edYsb06kiy/2VucLyv3VJOj3\nF6lYlhBOBFY559Y650qAfwE/rOa8pJ8tPsj/KIN8bxC8++vf3xvM1r07fOc7cNtt+dx5J0yYAE89\n5Y1neP99by3wr7/2FkEqL/c76roL2udXX7FsQuoMrKv0eD1wUpVzHDDYzBYC3wA3Oee+iGFMInIE\n6ene7KkXXghjxnizqS5bdmhp1spLtu7a5a350KIFNG3qlTQq1nuo2K9uq7wkamU1LSZU9Xjlx9U9\nV91W+boV25o1XoILJ460NG8EeG1berr3MzX18OnMK095XvVYJNVzbdvCddeFf36kYpkQwrnNeUCu\nc67QzM4BXgV6xzAmEQnTkCHeNmZM7eeVlnolhcJC70su3K2qmr4Yqx6v/Li652rbKpZVrdieeAJG\njDhyHM55a08UFXnzQhUVfXvbsePQ86Wlhy94VPln1WMpcdT5P2a9jMzsZGCMc25Y6PEfgHLn3F21\nvGYNcIJzbkeV42riEhGpg0h6GcWyhPAp0MvMugIbgJ8Al1Y+wcyygS3OOWdmJ+IlqB1V3yiSGxIR\nkbqJWUJwzpWa2XXAdCAVmOicW2pm14Sefxi4GPilmZUChcAlsYpHRERqlxAD00REJPbiqDnj2440\nsC3RmdlaM1sUGpQ31+946svMJpnZZjNbXOlYlpm9bWYrzGyGmbXyM8b6qOH+xpjZ+kqDK4f5GWNd\nmVmumc0ysyVm9rmZ3RA6HojPr5b7C8rnl2FmH5vZAjP7wsz+Fjoe0ecXtyWE0MC25cBZeF1SPwEu\ndc4t9TWwKKqpET1RmdlpwF7gSefcsaFj/wdsc879Xyipt3bO3epnnHVVw/3dARQ45+7xNbh6MrMO\nQAfn3AIzaw58BlwIXEEAPr9a7u/HBODzAzCzpqEem42AD4CbgAuI4POL5xJCuAPbEl1gGsydc+8D\nO6scvgB4IrT/BN5/woRUw/1BAD5D59wm59yC0P5eYCneWKJAfH613B8E4PMDcM4VhnYb47Xb7iTC\nzy+eE0J1A9s613BuonLAO2b2qZmN8juYGMl2zm0O7W8Gsv0MJkauN7OFZjYxUatUKgv1DBwIfEwA\nP79K9zcndCgQn5+ZpZjZArzPaZZzbgkRfn7xnBDisy4ruoY45wYC5wCjQ1USgRWasjZon+tDQDdg\nALARuNvfcOonVJ3yEvBr51xB5eeC8PmF7u9FvPvbS4A+P+dcuXNuAJADfM/Mzqjy/BE/v3hOCN8A\nuZUe5+KVEgLDObcx9HMr8ApeNVnQbA7V32JmHYEtPscTVc65LS4EeIwE/gzNLA0vGTzlnHs1dDgw\nn1+l+3u64v6C9PlVcM7tBt4ATiDCzy+eE8LBgW1m1hhvYNtrPscUNWbW1MxahPabAWcDi2t/VUJ6\nDaiYHGAE3vQkgRH6T1bhIhL0MzQzAyYCXzjn7q30VCA+v5ruL0CfX9uK6i4zawJ8H5hPhJ9f3PYy\nAgjNb3Qvhwa2/c3nkKLGzLrhlQrAGyD4TKLfn5k9B5wOtMWrr/wTMAX4N9AFWAv82Dm3y68Y66Oa\n+7sDyMOrbnDAGuCaSnW2CcPMTgXeAxZxqFrhD8BcAvD51XB/t+HNnhCEz+9YvEbjlND2lHNurJll\nEcHnF9cJQUREGk48VxmJiEgDUkIQERFACUFEREKUEEREBFBCEBGRECUEEREBlBBEDgpNhfw7v+MQ\n8YsSgsgh9R6UE5p6WCQhKSFIUjOz/zaz5Wb2PtAndKyHmb0ZmoX2PTOrfHxOaFGjO82sIHQ8z8ze\nN7MpwOehWSfHmtnc0CyaV1e63s2Vjo/x4ZZFaqS/ZiRpmdkJeHNkfQdIA+bhLZzyMHCtc26VmZ0E\n/AMYCtwHjHfOPW+htcErGQgc45z7KpQAdjnnTjSzdOADM5sB9AZ6ho6nAFPM7LTQOgsivlNCkGR2\nGvCyc64IKDKz14AMYDDwgjcfGuAtOAJwMt6CIwDPAeMqvddc59xXof2zgWPN7OLQ40ygV+j42WY2\nP3S8GdATUEKQuKCEIMnM8e3VslLw/rofGOF77avy+Drn3NuVD5jZD4C/OeceifC9RRqE2hAkmb0H\nXBhaoLwFcD5QCKyp+OvePMeFzp8DVPzVf0kt7zsd+FVFA7OZ9TazpqHjV4amO8fMOptZu6jflUgd\nKSFI0nLOzQeeBxYC0/CmenbAZcBVoeUIP+dQNdFvgBtDx3sAuyu/XaX9x4AvgHlmthhvVa7UUInh\nWWC2mS3Cm5a4eYxuTyRimv5aJExm1sQ5tz+0fwnwE+fcRT6HJRI1akMQCd8JZjYBr91hJ3Clz/GI\nRJVKCCIiAqgNQUREQpQQREQEUEIQEZEQJQQREQGUEEREJEQJQUREAPj/NRuXYyDIfAQAAAAASUVO\nRK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1088c1490>"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This is a typical bias/variance plot.\n",
      "\n",
      "On the **Left Side** of the plot, we have a **high-bias** model, characterized by the training and test data showing equally bad performance.  This shows that the model is **under-fitting** the data, because it does equally poorly on both known and unknown values.\n",
      "\n",
      "On the **Right Side** of the plot, we have a **high-variance model**, characterized by a divergence of the training and test data.  Here the model is **over-fitting** the data: in other words, the particular noise distribution of the input data has too much effect on the result.\n",
      "\n",
      "The optimal model here will be around the point where the **test** error is minimized."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Quick Exercise # 5"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's appy this learning curve idea to a **Decision Tree** regression on this dataset. ``sklearn.tree.DecisionTree`` takes a number of hyper-parameters, but one that can really control the over-fitting, under-fitting is the ``max_depth`` parameter especially controls the tradeoff between bias and variance.\n",
      "\n",
      "Generate a dataset like the one above, but containing 500 points, and use a Decision Tree regressor to explore the effect of the ``max_depth`` parameter on the bias/variance tradeoff.  What is the optimal value?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from sklearn.tree import DecisionTreeRegressor"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 24
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Quick Exercise #6"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "One popular and effective way to address over-fitting is to use **Ensemble methods**.  ``sklearn.ensemble.RandomForestRegressor`` uses multiple randomized decision trees and averages their results.  The ensemble of estimators can often do better than any individual estimator for data that is over-fit.  Repeat the above experiment using the Random Forest regressor, with 10 trees.  What is the best ``max_depth`` for this model?  Does the accuracy improve?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Illustration of Learning Curves"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The exact turning-point of the tradeoff between bias and variance is highly dependent on the number of training points used.  Here we'll illustrate the use of *learning curves*, which display this property.\n",
      "\n",
      "The idea is to plot the mean-squared-error for the training and test set as a function of *Number of Training Points*"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "X, y = make_data(200, error=1.0)\n",
      "degree = 3\n",
      "\n",
      "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)\n",
      "\n",
      "N_range = np.linspace(15, X_train.shape[0], 20).astype(int)\n",
      "\n",
      "def plot_learning_curve(degree=3):\n",
      "    training_error = []\n",
      "    test_error = []\n",
      "    mse = metrics.mean_squared_error\n",
      "    \n",
      "    for N in N_range:\n",
      "        XN = X_train[:N]\n",
      "        yN = y_train[:N]\n",
      "    \n",
      "        model = PolynomialRegression(degree).fit(XN, yN)\n",
      "        training_error.append(mse(model.predict(XN), yN))\n",
      "        test_error.append(mse(model.predict(X_test), y_test))\n",
      "      \n",
      "    plt.plot(N_range, training_error, label='training')\n",
      "    plt.plot(N_range, test_error, label='test')\n",
      "    plt.plot(N_range, np.ones_like(N_range), ':k')\n",
      "    plt.legend()\n",
      "    plt.title('degree = {0}'.format(degree))\n",
      "    plt.xlabel('num. training points')\n",
      "    plt.ylabel('MSE')\n",
      "    \n",
      "plot_learning_curve(3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEZCAYAAAB8culNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VOX5//H3TQhhS9gNO0F2cQFkURZNFRVF2VwpbtVS\n61e0tfpFtLaitX7lp2jrRbVqWVSUWhdWBaVqBAU3BEQxYFgT2beQsIQs9++PZxKGbITJnMyS+3Vd\nc+XMnDPn3BPI+cx5nuecI6qKMcYY469GqAswxhgTfiwcjDHGlGDhYIwxpgQLB2OMMSVYOBhjjCnB\nwsEYY0wJFg4mqojIDBH5S6jrMCbSWTiYaKO+R1QSkU9EZJeIHBSRH0VkbKhrMtGpZqgLMMYDEvQV\nitRU1bxgrzcA9wCpqporIn2BJSKyRFXXhbowE13syMFENBHpKSLf+r5J/xuoXWz+lSKySkT2i8jn\nInKW37xeIrLS997/iMibhU1SIpIsIhkiMl5EtgNTxZkgImkisse3fCO/9Z0nIst821olIhcG+/Oq\n6hpVzfV7KRs4GOztGGPhYCKWiNQC5gCvAI2At4Cr8TUriUhPYCowFmgMvAjME5FY33tnA9N8750F\njODEJqlE37y2wB24b+3DgAuAFsB+4B++bbUCFgCPqWoj4H7gHRFpWkbtC3whUtpj3kk+9wIROQKk\nALep6vaK/s6MqSixayuZSCUiFwCzVLWV32ufAx+p6p9F5AVgt6r+2W9+KvAb39M3VLW137ylwCe+\n9yYDHwDxqnrMN38tME5VP/Y9bwFsAergwqC7qt7st75Fvm286sFnjwFGAi8BPVR1a7C3Yao3O3Iw\nkawl8HOx17b4TbcD7vP/Vg60xn3rL+296cWe7y4MBp8kYLbfutYCebgjjHbAtcW2NQBoHvjHK5uq\n5qvq28CXuJAwJqisQ9pEsu1Aq2KvtQPSfNNbgb+q6hPF3+jrDyj+3rZ+74WSo562Ar9S1eWlrG8r\n8Jqq/qb4vNKIyEJgYBmzl6jq0IqsB4gFDlVwWWMqzI4cTCRbBuSJyD2+foRRQB+/+S8DvxWRvr7O\n5HoiMlRE6vvemy8i40SkpogML/be0vwTeEJE2gKISDMRGeabNxO4SkQuFZEYEant69QuHkAAqOrl\nqhpfxqPUYBCRLiJyuYjU8X3eG4HewIcV/H0ZU2EWDiZi+UbtjAJuBfYC1wHv+M1fgeuMngLsA34C\nbi723ttxHctjcB3K/s1IxY8c/g7MAz4UkYPAcqCvb30ZwHDgIWAX7ijjPoL7NybAI8BOYAfwa2Co\n9TcYL3jaIS0i04ChwC5VPauU+U1x37ia45q4nlbVGZ4VZEw5RORL4HlVfSXUtRgTal4fOUwHhpQz\nfxywUlV7AMnAZBGxfhBTJUTkAhFp7mtWugU4E1gU6rqMCQeehoOqLsUdspdlO5Dgm04A9obJWaim\neugCrML9H70XuEZVd4a2JGPCg+fnOYhIEjC/jGalGsDHQGcgHrhOVRd6WpAxxpiTCnWH9EPAKlVt\nCfQA/iEi8SGuyRhjqr1Qt+/3B/4KoKobRGQT7lD/G/+FRMRO4zbGmACoakAXogz1kUMqMBhARBJx\nwbCxtAVVNWIfjzzySMhrqK71R3LtVn/oH5Fef2V4euQgIrOAC4GmIpKOG6MdC6CqLwJPANNFZDUu\nqMar6j4vazLGGHNynoaDqo4+yfw9wFVe1mCMMebUhbpZqVpITk4OdQmVEsn1R3LtYPWHWqTXXxkR\nccluEdFIqNMYY8KJiKABdkiHerSSMSbKiQT9rq2mFMH+Am3hYIzxnB35e8uLALY+B2OMMSVYOBhj\njCnBwsEYY0wJFg7GGFMJd955J48//njQlw01G8pqjPGUbzhlqMsoU1JSEtOmTeOiiy4KdSkBK+t3\nXJmhrHbkYIyp1soLr7y86nt7GQsHY0y1ddNNN7F161auuuoq4uPjeeqpp6hRowbTpk2jXbt2DB48\nGIBrr72WFi1a0LBhQy688ELWrl1btI5bb72VP/3pTwCkpKTQunVrnnnmGRITE2nZsiUzZswIaNm9\ne/dy1VVX0aBBA/r27cvDDz/MoEGDvP+l+Fg4GGOqrddee422bduyYMECsrKyuO666wBYsmQJqamp\nfPDBBwAMHTqUtLQ0du/eTa9evRgzZkzROkTkhPMMdu7cycGDB9m2bRtTp07lrrvuIjMz85SXveuu\nu4iPj2fnzp288sorvPrqq1V6QqGFgzEm5EQq/wiGwualiRMnUqdOHeLi4gD3jb9evXrExsbyyCOP\nsHr1arKyskq8DyA2NpY///nPxMTEcPnll1O/fn3WrVt3Ssvm5+fz7rvv8uijj1K7dm26devGLbfc\nUqV9NxYOxpiQU638I5jatGlTNF1QUMCECRPo2LEjDRo0oH379gDs2bOn1Pc2adKEGjWO71rr1q1L\ndnb2KS27e/du8vLyTqijdevWlfpMp8rCwRhTrZXWVOP/2uuvv868efP46KOPyMzMZNOmTcCJRwCn\n0txTkWWbNWtGzZo1SU9PL3rNf7oqWDicoqVblrLr0K5Ql2GMCZLExEQ2bNhQ5vzs7Gzi4uJo3Lgx\nhw4d4qGHHjph/qncda2iy8bExDBq1CgmTpzIkSNHSE1N5bXXXrM+h3C2dOtSznz+TKatnBbWY7eN\nMRXz4IMP8vjjj9O4cWPeeeedEjvgm2++mXbt2tGqVSvOPPNMzj///BOWKd7JXN4O/FSWnTJlCpmZ\nmTRv3pxbbrmF0aNHU6tWrUA+YkDsJLgArNqxirHzx1Ivth4vXvkiXZp2CXVJxoStcD8JLlI88MAD\n7Nq1i+nTp5eYZyfBhYkezXvwxe1fMKrbKAZMG8Bjnz5GTl5OqMsyxkSRdevW8d1336GqfPXVV0yb\nNo2RI0dW2fYtHAIUUyOGe/rdw8o7VrJi+wp6vNiDpVuWera9LQe28I+v/kHqnlTPtmGMCR9ZWVlc\nffXV1K9fnxtuuIH777+fYcOGVdn2rVkpCFSV2amzuWfhPVzR6QomDZ5EozqNKr3ONbvWMCd1DnNS\n55B+MJ32DdvTo3kPXrrqpSBVboz3rFnJe140K1k4BFHm0Uwe+ughZqfO5pnLnuH67tef0uiCvII8\nlqUvKwoEgBFdRzCi6wj6t+lPemY65009j21/2EZMjRivPoYxQWXh4D0LhwixPH05v1nwG9oktOH5\noc+T1DCpzGUP5x5m8YbFzFk3hwXrF9AmoU1RIJx12lklwqXniz15bshzDGpXdddYMaYyLBy8F3Hh\nICLTgKHALlU9q4xlkoFngVhgj6oml7JMRIUDwLH8Y0xeNpnJyyczYeAEfn/e76lZw92ye+/hvSxY\nv4A56+bw8aaP6d2yNyO6jGBYl2G0a9iu3PU+9ulj7D+yn2eHPFsVH8OYSrNw8F4khsMgIBt4tbRw\nEJGGwOfAZaqaISJNVbXEOemRGA6F0val8dsFv2XfkX3ccOYNLExbyLfbv2Xw6YMZ0WUEQzsPpXGd\nxhVe35qda7hq1lVs+t2mKj0hxphAWTh4L+LCAUBEkoD5ZYTD/wDNVfXPJ1lHxIYDuM7lmd/N5PP0\nzxnaaSiDTx9Mndg6Aa+r85TOvHnNm/Rq0SvIlRoTfBYO3ovGcChsTuoOxAN/V9XXSlkuosMh2MYv\nHk9cTBx/uegvoS7FmJOycPCeF+FQs9JVVU4s0Au4GKgLLBeRL1T1p+ILTpw4sWg6OTmZ5OTkKiox\n/IzqNorb591u4WBMEATjNqEzZsxg6tSpLF3q3blOFZGSkkJKSkpQ1hXqcEjHdUIfAY6IyBLgHKDc\ncKju+rbqy/4j+1m3Z51dusOYSoqmI5viX5wfffTRgNcV6jOk5wIDRSRGROoC/YC1J3lPtVdDajCy\n60hmp84OdSnGRLTitwl9+umn+eKLL+jfvz+NGjWiR48efPrpp0XLz5gxgw4dOpCQkMDpp5/OG2+8\nQWpqKr/97W9Zvnw58fHxNG5c8QEmYa3wErJePIBZwDbgGO4o4TbgDuAOv2XuB34A1gD3lLEeNSda\nvGGx9n25b6jLMOakwv3vNykpST/66CNVVc3IyNAmTZrowoULVVV18eLF2qRJE92zZ49mZ2drQkKC\nrl+/XlVVd+zYoT/88IOqqs6YMUMHDhwYmg+gZf+Ofa8HtP/2tFlJVUdXYJmngae9rCMaXdjuQtL2\npZFxMIPWCVV7hyhjgk0erfywbH2k8k1DM2fO5IorrmDIkCEADB48mN69e/Pee+9xzTXXUKNGDdas\nWUPr1q1JTEwkMTHRbTtKmqX8hbrPwQQoNiaWKztfyZzUOYzrOy7U5RhTKcHYsQfDli1beOutt5g/\nf37Ra3l5eVx00UXUrVuXN998k6effprbb7+dAQMGMHnyZLp0ic5+v1D3OZhKGNV1FO/++G6oyzAm\novmfTNq2bVtuuukm9u/fX/TIyspi/PjxAFx66aV8+OGH7Nixg65duzJ27NgS64gWFg4R7NIOl7Ji\n+wr2HC79RufGmJPzv03omDFjmD9/Ph9++CH5+fkcPXqUlJQUfv75Z3bt2sXcuXM5dOgQsbGx1KtX\nj5iYmKJ1ZGRkkJubG8qPElQWDhGsTmwdLjn9Euavm3/yhY0xpSq8TWijRo146623mDt3Lk888QSn\nnXYabdu2ZfLkyagqBQUFPPvss7Rq1YomTZqwdOlSXnjhBQAuvvhiunfvTvPmzTnttNNC/ImCw67K\nGuFe/+513vzhTeaNnhfqUowpVTSdRxCu7DahpoShnYeSsjmFrJysUJdijIkiFg4RrmHthvRv059F\naYtCXYoxJopYOESBUd1G8W6qjVoyxgSPhUMUGN5lOIvSFpGTlxPqUowxUcLCIQok1k+ke7PufLTp\no1CXYoyJEhYOUWJUt1HM/tEuxGeMCQ4byholNu3fRL9/9WP7fduJqRET6nKMKRKNZw+Ho2APZbVr\nK0WJ9o3a0yqhFZ+nf84F7S4IdTnGFLEvdpHJmpWiyMiuI+1aS8aYoLBwiCKjuo1idups+6ZmjKk0\nC4co0r1Zd2rF1OLb7d+GuhRjTISzcIgiIsKorqM8u32oHZEYU31YOESZkd2C3++QfSyby1+/nN8t\n+l1Q12uMCV8WDlGmb6u+ZOZkkronNSjr23dkH4NfHUyTOk2Y9f0s1u5eG5T1GmPCm4VDlKkhNRjR\nZURQTojbnrWdC2dcyMC2A3lt5GtMGDCBB/77QBCqNMaEOwuHKFQ4aqkyNu7fyMDpAxl95mieuuQp\nRIRxfcfx/a7v+XjTx0Gq1BgTriwcotAF7S5gw/4NpGemB/T+73d9zwXTL+D+8+/noUEPFZ3hGlcz\njicvfpL7P7yfAi0IZsnGmDBj4RCFYmNiuarzVcxJnXPK7/0i4wsufvVinrrkKe7sc2eJ+dd1v45a\nMbWY+d3MYJRqjAlTnoaDiEwTkZ0isuYky/URkTwRGeVlPdXJyK4jT/keD4s3LGbYrGFMHz6d0WeN\nLnUZEWHypZP548d/5HDu4WCUaowJQ14fOUwHhpS3gIjEAJOARYBdoStILu1wKd9u/5Y9h/dUaPl3\n1r7DmHfH8M5173BFpyvKXXZA2wH0a9WPv33xt2CUaowJQ56Gg6ouBfafZLG7gbeB3V7WUt3Uia3D\nJadfwrx180667LSV07h74d18cOMHDGo3qELrf3Lwkzyz/Bl2Zu+sbKnGmDAU0j4HEWkFDAde8L1k\np+AG0ciuI086amnyssk89uljpNyaQs8WPSu87o6NO3LT2TcxMWViJas0xoSjUF+y+2/ABFVVcUNi\nymxWmjhxYtF0cnIyycnJnhcX6YZ2Hsqd791JVk4W8XHxJ8xTVR7++GHeTX2Xpb9aSpsGbU55/Q9f\n8DBd/9GVu/vdzRnNzghW2caYAKWkpJCSkhKUdXl+sx8RSQLmq+pZpczbyPFAaAocBsaq6rxiy9nN\nfgI0ZOYQbut5G9d1v67otQItYNz74/jq569YOGYhzeo1C3j9k5dNJmVLCvNHzw9GucaYIKrMzX5C\n2qykqqerantVbY/rd7izeDCYyhnVbdQJ11rKzc/lxndvZO3utXx8y8eVCgaAcX3H8cOuH+zEOGOi\njNdDWWcBy4AuIpIuIreJyB0icoeX2zXHDe8ynEVpiziad5TDuYcZ8eYIso9ls3DMQhLiEiq9/ria\ncTw5+Enu+/A+OzHOmChi95CuBgZNH8Sdve/khW9eIKlhEtOGTSM2JjZo61dVzp96Pv/T53+4+Zyb\ng7ZeY0zlRGyzkqkaI7uO5KbZN9GzeU9eGfFKUIMB3H/AZy57xk6MMyaK2JFDNbD38F4WrF/Azefc\nXHSdJC9c+9a19EjswR8v+KNn2zDGVFxljhwsHEzQpO1Lo9+/+rH2f9aSWD8x1OUYU+1ZOJiwce+i\nezmad5QXrnzh5AsbYzxl4WDCxr4j++gypQuf3vqpnRhnTIhZh7QJG43rNObBgQ8yfvH4UJdijKkE\nCwcTdHf1uYu1u9fy0caPQl2KMSZAFg4m6ApPjLt/sd0xzphIZeFgPHHtGddSu2Ztu2OcMRHKOqSN\nZ5alL+P6t69n3bh11I2tG+pyjKl2rEPahKX+bfpzXuvzmLxscqhLMcacIjtyMJ7auH8jF864kDOa\nncGEARNITkr29CxtY8xxdp6DCWs5eTm8vuZ1Jn0+iUa1GzFh4ASGdRlGDbEDV2O8ZOFgIkJ+QT5z\nUufwf5/9H4dyD/HAgAf45Vm/pFZMrVCXZkxUsnAwEUVV+XjTx/zfZ//H+r3rue/8+/h1r19Tr1a9\nUJdmTFSxcDAR6+ufv2bS55NYsmUJd/W5i3F9x9GkbpNQl2VMVLBwMBFv3Z51/L/P/x+zU2dza49b\n+cP5f6B1QutKrTMnL4etmVvZkrmFLQe20KlJJ/q36U/NGjWDVLUx4c3CwUSNjIMZPLv8Waavms7I\nriMZP2A8XZp2KXXZo3lH2Zq5lc0HNhc9tmRuKZrec3gPreJbkdQwidYJrVmzaw3pmekM7TyU4V2G\nc2mHS6lfq34Vf0Jjqo6Fg4k6+47sY8pXU5jy1RQGtRvE4PaDXRBk+kLgwBb2HtlLm4Q2JDVMIqlh\nEu0atDs+3bAdLeNbljhK2Jq5lXnr5jF33Vy+zPiSC9pdwPAuw7my85W0iG8Rok9rjDcsHEzUOnTs\nEFNXTuX7Xd+fsPNPaphE8/rNiakRE/C6M49msjBtIXPXzWVR2iK6NOnC8C7DGd51ON2adrPzMUzE\ns3AwppKO5R9jyZYlzE2dy9x1c4mrGcewzsMY3nW49VOYiGXhYEwQqSqrd64uCood2Tu4tcet/LrX\nrzm90emhLs+YCrNwMMZDP+7+kX99+y9e/e5VejTvwdheYxnRdYSdvGfCXtiGg4hMA4YCu1T1rFLm\njwHGAwJkAXeq6nelLGfhYEIuJy+H2amzeWnFS/yw+wduPvtmxp47ls5NOoe6NGNKFc7hMAjIBl4t\nIxzOB9aqaqaIDAEmqup5pSxn4WDCyk97f2LqyqnMWDWDrk27MrbXWK4+42pq16wd6tKMKRK24QAg\nIknA/NLCodhyjYA1qlrizCcLBxOujuUfY/66+bz07Uus2LaCG8++kbG9xtL9tO6hLs2YqAmH+4HO\nqvqbUuZZOJiwt2n/JqaunMq0ldNo36g9v+n1G67tfm2ZNzoq0AJ2HdrFzwd/JuNgBhkHM/g56+cT\nfubk5XBO83Po3aI357Y8l3NbnEti/cQq/mQmUkV8OIjIL4B/AANUdX8p8/WRRx4pep6cnExycnLQ\nazUmGPIK8nhv/Xu8/O3LLM9Yzg3db6BD4w4n7PQzDmawI3sHDeIa0CqhFa0TWtMq3v30n65Zoyar\ndqzim23fsGL7ClZsX0G92Hr0btmbc1uca4FhTpCSkkJKSkrR80cffTRyw0FEzgbeBYaoaloZy9iR\ng4lI6ZnpzFg1g71H9p6w02+V0IqW8S1PuY9CVdm4f6MLim0rigKjfq36LizCIDCO5R9jR/YOtmdt\n52DOQWJqxBAjMQH9rFmjJk3qNLETEgMUsUcOItIW+Bi4UVW/KGcdFg7GlOFkgdGpcSca12lM4zqN\naVSnUdF04SO+VnyFdr55BXnsOrSLbVnbyn3sP7qfxHqJtIxvSUJcAvmaT35BfkA/c/NziasZx2Ud\nLmNIxyFccvoldtXeUxC24SAis4ALgabATuARIBZAVV8UkX8BI4GtvrfkqmrfUtZj4WDMKfAPjM0H\nNrPvyD72HdnH/qP7i6b3HdnH/iP7OZx7uERoNKrdiNo1a7Pz0E62Z21nW9Y2dh/eTdO6TWlRvwUt\n41uW+WhWt1mlLmtS3IZ9G/hgwwcsSltEyuYUzmh2BkM6DuHyjpfTu2XvoG4r2oRtOASLhYMx3snN\nzz0hNPYfcdOHcw/TvH7zop1+Yv3EkF9GJCcvh8/TP2dR2iIWpS1iW9Y2LulwCUM6DOGyjpfRvH7z\nkNYXbiwcjDHVUsbBDD5I+4BFGxbx343/JalhEkM6DGFIxyH0bNGT3PxccvJzyMnLKffn0byjpc6r\nFVOLhLgEGtRuQIO4BiWmE+ISwvrIxcLBGFPt5RXk8WXGl+6oYsMi1u5eS62YWsTFxFG7Zm3iasYR\nFxNX6s/aNWu76WKv5+bnkpmT6R5HMzmYc7BoOjMnk+xj2dSNretCI64BDWq70GhYuyEN4xrSsHZD\nGtVp5H7WblTq89iYWM9+JxYOxoSZggL45ht4/31YtAhat4Y//AH69w91ZSaYCrSA7GPZRWFxMOcg\nmUczOXD0AJk5mew/sp8DRw+w/6j76T9dOC+uZlypwREXE0eBFpCv+RRoQcnpgpO//umvPg1+OIjI\njao60zc9QFU/95s3TlWnBPTbDICFg4kE+/bBhx8eD4TTToMrroAhQ2DtWnj2WffafffByJEQE76t\nEaaKqCrZx7JLDY5j+ceoITWoITWIqRFzfFpiKvz6L9r/wpNwWKmqPYtPl/bcaxYOJhypwqpVLgze\nfx/WrIHkZBcIl18O7dqduHx+PsydC5Mnw/bt8Pvfw69+BfHxISnfRDFV2L0bEhM9aFaycDCmpMxM\n+O9/XRgsXAj167swuOIKuOACqF3Bc9q++MKFxCefwO23w913u6YnYypCFfbvh02bYPPmkj83b4Y6\ndWDvXgsHYzyh6pqECo8OvvkGBgw4fnTQqVPl1r9pE/z97/DqqzB0qGty6tEjOLWbyHbwYMkdv/+0\nCLRv7x5JSSV/xsd71CEtIkeAwstZdAA2+M3uoKqlX03MAxYOJlT+/GeYOhWGD3eB8ItfQL16wd/O\ngQPw0kvw3HPQubMLicsvhxo1gr8tE3rZ2ZCeDhkZ7lHa9LFjpe/4C6cbNTr5drwKh6Ty3qiqmwPZ\nYCAsHKq3n35y36LOPbdqt/vWW3D//fD1164juSocO+a2O3kyHDkCo0e7jutjx9wjN/f4dFmvFX/e\nvDkMHOge/fp5E27+srJc/8t330HDhnDNNVCzCs+dO3wYnngCUlKgTx84/3z3aNOmarafmwtpacd3\n8mXt+Nu0cU2JhT+LTzdu7I4OKqNKhrKKSFPgAmCLqq4IZGOBsnCovv79b9ceD/DKK+7be1VYtQou\nucSNPupZZQ2ox6m6nduiRW7HWqvW8UdsbMWfx8bC1q3w2WewdCmsXg1nngmDBrmwGDAAmjULvMbN\nm10IrF59/LFtG5xxBpxzDqxfDzt3wsMPwy9/6W1IqMK8efC738F557nO/pUrYfly96hVy4VE//7u\nZ8+eEBdX+W1u2OC+QHz1lXusXg0tW0LbtmXv/Bs1qvyOvyK8OnJ4D3hAVb8XkRbASuBrXBPTy6r6\nbKAFn3KRFg7VTk6OOy/ggw/cN+mcHBg2DF54Aa6+2ttt797tvnFOmgTXX+/ttqrakSNuR7Z0qQuM\n5cuhRQsXFIWB0b59yR3X4cPw/fcnhsB337l27bPPdkFQ+OjU6XgIFIbcxIkuNP74R7jxxuCHxIYN\ncM89sHEjTJkCF1984nxVN2/ZsuNhsX69q7fwyOL886FVq/K3s2vX8RD46iv3u6xXD/r2Pf4499zw\nGYHmVTj8oKrdfdMPAV1V9WYRiQeWnezmPcFk4VC9bNoE113nvmVNnw4NGrjXV650Rw5PPeV2MF7I\nzXVHDP37u6aJaJef75qACsNi6VIXDAMHQpcukJrqgiA9Hbp2dTvTwjA4+2xo2rTi20pJgUcfdUcy\nDz/s/g1jK3ly8JEjLsSnTIH//V+49153hFAR2dlu514YFsuXQ926J4ZFTs6JYZCZ6b44FAZBnz4u\nXMOVV+GwSlV7+KY/xh0tzPI9X62q5wRa8CkXaeFQbcyfD7/+NTz4oGseKP4Ndu1auPRS11H8mxL3\nDKy8u+5yO6+5c6tnZ7CqC+fPPnN9Pd26uRDo0qXyO/JCS5a4kNi0yR1J3HxzYOtesMAdLfTq5U4w\nrGyfgqrrKygMii++cEHTr9/xMOjYMbL+X1QmHFDVUh/AAuBuYBSwH2jke70u8ENZ7/Pi4co00Sw3\nV3X8eNU2bVSXLSt/2Z9+Um3XTvWZZ4Jbw4svqnbtqnrgQHDXa0q3ZInqxRerJiWpvvSSak5Oxd63\ncaPqsGGqnTqpLlrkbY2RzrfvDGi/W14G3g6cCdwCXK/Hb9/ZD5geUBIZU4qff4aLLnLNF99+6w7n\ny9Oxo/v2+fzz8Pjj7htfZX32GfzpT+6IobAZy3hr0CB3QuHMma5fqXNnePFFN5KnNEePwl/+Ar17\nu2/za9bAZZdVbc3VSqCpUpUP7Mghai1erNq8uepf/qKan39q7922TbV7d9UJE1QLCgKvYcsW1RYt\n7FtoqH3+ueqll6q2bav6wguqR48en7dwoWrHjqojRqhu2hSyEiMOlThyKK/PYT6gQGntVaqqwzxL\nrJK1aFl1mshUUOC+9f/zn+6b40UXBbaePXvct8cBA+Bvfzv19uDDh9032NGj3TkNJvS++ML1Sfzw\ngzsZcMkSd1T53HNVN5Q5WnjVIb0byABmAV8Wvuz7qar6aSAbDISFQ3TZvduNVDl6FGbNcmPCK+PA\nAbfT6NaukPeHAAAS/ElEQVTNnWVc0audqh4fe//qq1Uz7txU3JdfwtNPw1lnwfjxFb9ulTnOq3Co\nCVwCjAbOAt4DZqnqD4EWGigLB2fbNrdjjY+HhAT3s7In8VS1ZcvghhtgzBjXfhys8e7Z2e48iObN\n3clyFRn9MmkSvP22+2Zap05w6jAmnHh+hrSIxOFC4mlgolbhvRx826+24XDwILzzjmt6WbnSnV2Z\nleVez8py33b9w8J/2v9ns2buzNizznKn5Vc1VTfccNIkd62iK68M/jaOHHGXaoiNhTffLD84338f\nxo51307taqgmWnkWDiJSGxgK3AAkAfOAaar6cyAbC1R1C4fcXHfZhJkz3c9f/AJuusldtbP4oXVO\nzvGgKOtnVpa7f8D337sRHg0auLHr/o/OnYM3jr24AwfgttvcNWX+8x930TCvHDvmmoqysmD2bHdS\nU3Gpqe7y2nPm2J3ZTHTzqlnpNaA78D7wpqquCbzEyqkO4aDqvsXOnOm+9Xbp4trlr70WmjQJ3nYK\nCmDLFnfpA//H1q1um8VDIzHx5G3xubmuY3jXLtfstWvX8cfu3W644lVXuTObq6IZLC/PhdHmze5E\nqYSE4/MOHHDDIB94wC1jTDTzKhwKgENlvE9VNaGMeUEXzeHw00/w+usuFGJiXCCMGQOnn161dRw+\n7M4+9g+M1atdTWef7ZqjEhJKD4CsLBdgp512/NGs2fHpbt3c5RiqUkGBO9v522/dTXkaN3aXirjy\nSnftn+eeq9p6jAmFKrkqa0ArF5mGa5bapWVci0lEngMuBw4Dt6rqylKWiapw2L3bNa/MnOkuBnbD\nDS4UevcOrxEzqq45qjAsDh06MQAKH40aheclBVTd8NT//hcWL3aXwf7mG9dU51UTmjHhJJzDYRCQ\nDbxaWjiIyBXAOFW9QkT6AX9X1fNKWS7iwyE/342MmTnTjY4ZOtQFwiWX2I7KS6rwyCOuEzwuzl1o\nLZjNdMaEs8qEg6e34FDVpSe5adAw4BXfsl+KSEMRSVTVnV7WFQr/+Y/bSf3xj/DGG+FzSd9oJwKP\nPeaa6fr1s2AwpqKq8P5MpWoFpPs9zwBaA1EXDt9/7/oSbrkl1JVUT7feGuoKjIksoQ4HKHl5jlLb\njyZOnFg0nZycTHJysncVnURhLafyc/Zs+NOfTv199tN+2k/7WdGfKSkpRc8ru4/0tM8Biu5FPb+M\nPod/Aimq+m/f81TgwuLNStHQ53D22TBjhrv2vDHGVIXK9DmEeozJPOBmABE5DzgQjf0NBQXuJiKd\nO4e6EmOMqZiaXq5cRGYBFwJNRSQdeASIBVDVF1X1fRG5QkTScOdU/MrLekIlPd0N96xfP9SVGGNM\nxXgaDqo6ugLLjPOyhnCwbp07+9gYYyJFqJuVqgULB2NMpLFwqAIWDsaYSGPhUAXWr7fOaGNMZLFw\nqAJ25GCMiTSen+cQDJF8nsPhw+6KoIcOVfz2lcYYEwyRfJ5D1EtLc9f1sWAwxkQSCwePWZOSMSYS\nWTh4zMLBGBOJLBw8tn69hYMxJvJYOHhs3TobxmqMiTwWDh5StWYlY0xksnDw0K5dbpRS06ahrsQY\nY06NhYOH7MxoY0yksnDwkDUpGWMilYWDhywcjDGRysLBQzaM1RgTqSwcPGTDWI0xkcouvOeR3FyI\nj4cDB6B27VBXY4ypjuzCe2Fo0yZo2dKCwRgTmSwcPGLDWI0xkczCwSM2UskYE8ksHDxi4WCMiWQW\nDh6xYazGmEjmaTiIyBARSRWRn0TkgVLmNxWRRSKySkS+F5FbvaynKtkwVmNMJPNsKKuIxADrgMHA\nz8DXwGhV/dFvmYlAnKo+KCJNfcsnqmpesXVF1FDWgwfdSKWDB6GGHZsZY0IkXIey9gXSVHWzquYC\n/waGF1tmO5Dgm04A9hYPhki0bh106mTBYIyJXDU9XHcrIN3veQbQr9gyLwMfi8g2IB64zsN6qowN\nYzXGRDovw6Ei7UAPAatUNVlEOgCLReQcVc0qvuDEiROLppOTk0lOTg5WnUFnI5WMMaGQkpJCSkpK\nUNblZZ/DecBEVR3ie/4gUKCqk/yWeR/4q6p+7nv+EfCAqn5TbF0R1edw/fUwbBiMGRPqSowx1Vm4\n9jl8A3QSkSQRqQVcD8wrtkwqrsMaEUkEugAbPaypStgwVmNMpPOsWUlV80RkHPABEANMVdUfReQO\n3/wXgSeA6SKyGhdU41V1n1c1VYWCAutzMMZEPrsqa5Clp0PfvrB9e6grMcZUd+HarFQtWZOSMSYa\nWDgEmZ0ZbYyJBhYOQWbDWI0x0cDCIcgsHIwx0cDCIcisz8EYEw1stFIQHT0KDRtCVhbExoa6GmNM\ndWejlcJEWhokJVkwGGMin4VDEFmTkjEmWlg4BJENYzXGRAsLhyCykUrGmGhh4RBEFg7GmGhh4RBE\n1udgjIkWFg5BsmcP5OdDs2ahrsQYYyrPwiFICpuUJKARxcYYE14sHILEmpSMMdHEwiFIbBirMSaa\nWDgEiY1UMsZEEwuHILFwMMZEE7vwXhDk50O9erB/P9SpE+pqjDHGsQvvhdjmzZCYaMFgjIkeFg5B\nYE1KxphoY+EQBDaM1RgTbSwcgsCGsRpjoo2n4SAiQ0QkVUR+EpEHylgmWURWisj3IpLiZT1esWYl\nY0y0qenVikUkBpgCDAZ+Br4WkXmq+qPfMg2BfwCXqWqGiDT1qh4vWbOSMSbaeHnk0BdIU9XNqpoL\n/BsYXmyZXwLvqGoGgKru8bAeT2Rnw7590KZNqCsxxpjg8TIcWgHpfs8zfK/56wQ0FpFPROQbEbnJ\nw3o8sX49dOwINaz3xhgTRTxrVgIqctZaLNALuBioCywXkS9U9afiC06cOLFoOjk5meTk5OBUWUnW\n32CMCRcpKSmkpKQEZV2enSEtIucBE1V1iO/5g0CBqk7yW+YBoI6qTvQ9/xewSFXfLrausD1D+tFH\nITcXHn881JUYY8yJwvUM6W+ATiKSJCK1gOuBecWWmQsMFJEYEakL9APWelhT0NkwVmNMNPIsHFQ1\nDxgHfIDb4b+pqj+KyB0icodvmVRgEfAd8CXwsqpGXDhYs5IxJtrYhfcqQRUSEiA9HRo2DHU1xhhz\nonBtVop627dD3boWDMaY6GPhUAnW32CMiVYWDpVg/Q3GmGhl4VAJdtkMY0y0snCoBGtWMsZEKwuH\nSrBmJWNMtLKhrAE6dswNYz14EGrVCnU1xhhTkg1lDYENG6BtWwsGY0x0snAIkPU3GGOimYVDgKy/\nwRgTzSwcAmTDWI0x0czCIUDWrGSMiWYWDgGyZiVjTDSzcAjAvn2QkwPNm4e6EmOM8YaFQwAK+xsk\noNHDxhgT/iwcAmD9DcaYaGfhEAAbqWSMiXYWDgGwzmhjTLSzcAiANSsZY6KdXXjvFOXnQ/36sGcP\n1KsX6mqMMaZsduG9KpSeDk2bWjAYY6KbhcMpsv4GY0x14Gk4iMgQEUkVkZ9E5IFylusjInkiMsrL\neoLB+huMMdWBZ+EgIjHAFGAIcAYwWkS6lbHcJGAREPanlQUyjDUlJcWTWqpKJNcfybWD1R9qkV5/\nZXh55NAXSFPVzaqaC/wbGF7KcncDbwO7PawlaFq2hD59Tu09kf4fLJLrj+TaweoPtUivvzJqerju\nVkC63/MMoJ//AiLSChcYFwF9gPAYklSOhx4KdQXGGOM9L48cKrKj/xswwTdOVYiAZiVjjKkOPDvP\nQUTOAyaq6hDf8weBAlWd5LfMRo4HQlPgMDBWVecVW1fYH1EYY0w4CvQ8By/DoSawDrgY2AZ8BYxW\n1R/LWH46MF9V3/WkIGOMMRXmWZ+DquaJyDjgAyAGmKqqP4rIHb75L3q1bWOMMZUTEZfPMMYYU7XC\n+gzpip5EFy5EpI2IfCIiP4jI9yJyj+/1xiKyWETWi8iHItIw1LWWR0RiRGSliMz3PY+Y+kWkoYi8\nLSI/ishaEekXYfU/6Pv/s0ZE3hCRuHCtX0SmichOEVnj91qZtfo+20++v+lLQ1P1cWXU/5Tv/85q\nEXlXRBr4zQv7+v3m3SciBSLS2O+1U6o/bMOhoifRhZlc4F5V7Q6cB9zlq3kCsFhVOwMf+Z6Hs98B\nazk+4iyS6v878L6qdgPOBlKJkPpFJAkYC/RS1bNwzbE3EL71T8f9ffortVYROQO4Hve3PAR4XkRC\nvf8prf4Pge6qeg6wHngQIqp+RKQNcAmwxe+1U64/1B+uPBU9iS5sqOoOVV3lm84GfsSd7zEMeMW3\n2CvAiNBUeHIi0hq4AvgXx0eSRUT9vm95g1R1Grh+L1XNJELqBw7ivmDU9Q3oqIsbzBGW9avqUmB/\nsZfLqnU4MEtVc1V1M5CG+xsPmdLqV9XFqlrge/ol0No3HRH1+zwDjC/22inXH87hUNpJdK1CVMsp\n830L7In7D5aoqjt9s3YCiSEqqyKeBf4XKPB7LVLqbw/sFpHpIvKtiLwsIvWIkPpVdR8wGdiKC4UD\nqrqYCKnfp6xaW+L+hgtFwt/zbcD7vumIqF9EhgMZqvpdsVmnXH84h0PE9pSLSH3gHeB3qprlP893\nwl9YfjYRuRLYpaorKeOExHCuHzf6rhfwvKr2Ag5RrAkmnOsXkQ7A74Ek3B9zfRG50X+ZcK6/uArU\nGrafQ0T+CBxT1TfKWSys6heRusBDwCP+L5fzlnLrD+dw+Blo4/e8DScmX1gSkVhcMLymqnN8L+8U\nkea++S2AXaGq7yT6A8NEZBMwC7hIRF4jcurPwH1r+tr3/G1cWOyIkPp7A8tUda+q5gHvAucTOfVD\n2f9Xiv89t/a9FnZE5FZc0+oYv5cjof4OuC8Wq31/w62BFSKSSAD1h3M4fAN0EpEkEamF60yZd5L3\nhJSICDAVWKuqf/ObNQ+4xTd9CzCn+HvDgao+pKptVLU9riP0Y1W9icipfweQLiKFF1UfDPwAzCcC\n6sd1np8nInV8/5cG4wYGREr9UPb/lXnADSJSS0TaA51wJ8aGFREZgmtWHa6qR/1mhX39qrpGVRNV\ntb3vbzgDN7hhJ4HUr6ph+wAux51lnQY8GOp6KlDvQFxb/Spgpe8xBGgM/Bc3+uFDoGGoa63AZ7kQ\nmOebjpj6gXOAr4HVuG/eDSKs/vG4QFuD69CNDdf6cUeX24BjuP7BX5VXK67JIw0XgpeFYf23AT/h\nRvkU/v0+HwH15xT+/ovN3wg0DrR+OwnOGGNMCeHcrGSMMSZELByMMcaUYOFgjDGmBAsHY4wxJVg4\nGGOMKcHCwRhjTAkWDqbaEJFzROTyAN7XUkTeqsBy74lIQmDVVU5Fti0it/jOWjbmpOw8B1Nt+C6L\ncK6q3l3KvJrqLlkRtUTkE+B+VV0R6lpM+LMjBxMSvsui/CgiL4m7MdIHIlLbNy9FRM71TTf1XScG\nEblVROb4biKzSUTGicj9viuwLheRRuVsrxbwGHC9uBsZXSciE0XkNRH5DHhFRNqJyBIRWeF7nO9X\n6xq/Gt4VkYXibmgzyW8bm8Xd7Ka8z9ZHRL7z1fBUGTdqSfbVscB3Y5YXfJfTQERG+96/RkSerOi2\nReQa3LWbXvf9vmqLyJPibiy0WkSequQ/qYkyFg4mlDoCU1T1TOAAcLXv9fKu5tkdGAn0Af4KHFR3\nBdblwM1lbUhVjwF/Av6tqj1V9T++WV2Bi1V1DO4icZeo6rm4a0s9V8bqzgGuA87ChU3hpY/9ay7r\ns00HxqpqTyCvnM/ZBxiHuzlLB2CUiLQEngR+AfQA+oi7RPNJt62qb+OuV/ZL3++rHjBCVQtvbPOX\nMuow1ZSFgwmlTXr8uvMrcFeUPJlPVPWQqu7B7fjm+15fU4H3Cydewlhx14/K8T2vBfxLRL4D/oPb\nMZfmI1XN8r1vLdCulGVKfDZxNyOqr6pf+l5/o1g9/r5Sd6OrAtw1dAbivvmnqLtqaz7wOnBBRbbt\nN69we5nAURGZKiIjgSNl1GGqKQsHE0o5ftP5uNtigvtGXfh/s3Y57ynwe16Au59DeUr7ln7Yb/pe\nYLuqno3bEdcqYz3F6y5tuxVZpqLX2hdKr72s18v6vRat19e/0hd3WfMrgUXl1GKqIQsHE04Kd5ab\ncTtngGtO8b3lyQLiy5mfAOzwTd/MiTvVSlN3y9IsESm8PeMN5Sze19d/UAPXhLUUd4nlC0Wkibh7\nrN8AfFqBTRf+brJwnxFxd8hrqKoLgT/gmsqMKWLhYEKp+LfewudPA3eKyLdAE7/Xi/dFFJ9WABG5\nQ0TuKGV7nwBnFHZIl7KO54FbRGQV0AXILmVbFb0TW1mf7XbgZRFZibtHdGYZ7/0amIJrttqoqrPV\n3a9igu9zrAK+UdX5fu852bZnAP/0/V7jgfkishoXPPdW4DOZasSGshpThUSknqoe8k1PwN1z+d5i\nyyQD96nqVSEo0Rjg5G20xpjgGioiD+L+9jYDt5ayTMTcJ9pELztyMMYYU4L1ORhjjCnBwsEYY0wJ\nFg7GGGNKsHAwxhhTgoWDMcaYEiwcjDHGlPD/AeiMFJp5lm1MAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x108a88490>"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This shows a typical learning curve: for very few training points, there is a large separation between the training and test error, which indicates **over-fitting**.  Given the same model, for a large number of training points, the training and testing errors converge, which indicates potential **under-fitting**.\n",
      "\n",
      "It is easy to see that, in this plot, if you'd like to reduce the MSE down to the nominal value of 1.0 (which is the magnitude of the scatter we put in when constructing the data), then adding more samples will *Never* get you there.  For $d=3$, we've already converged.  What about for $d=1$?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plot_learning_curve(2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEZCAYAAAB8culNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VFX6wPHvCyRACh1D6MWOdHFRVKIUXezYVxSVx11R\n17K6iv4UUBF1xZV1VSxUC6wKFkBQEA2iIirSpDdpEooESGhp7++PMwlDJgkpczMz4f08zzy5c++Z\ne98JZN4559xzjqgqxhhjjL9KoQ7AGGNM+LHkYIwxJoAlB2OMMQEsORhjjAlgycEYY0wASw7GGGMC\nWHIwFYqIjBORp0MdhzGRzpKDqWjU96hwRKS+iEwUka0iskdEvhWRs0Idl6mYLDmYikiCfkKRKsE+\nZynEAfOBjkBtYDzwmYjEhjQqUyFZcjARTUQ6iMgvIrJPRP4HVMt3/FIRWSQiqSLynYi08TvWUUQW\n+l77gYi8n9skJSJJIrJFRB4WkW3AaHEGishaEdnlK1/b73xdROR737UWiUi3YL5XVd2gqiNUdbs6\nbwHRwMnBvI4xYMnBRDARiQY+wX2Drg18CFyNr1lJRDoAo4E7gDrAG8AUEYnyvfZjYIzvtROBKzm6\nSSrBd6wp8DfgXuBy4HwgEUgFXvVdqxEwDXhKVWsDDwGTRaReIbFP8yWRgh5Tivn+2+OSw9rilDem\nJMTmVjKRSkTOByaqaiO/fd8Bs1V1kIiMBHaq6iC/4yuBv/qeTlDVxn7H5gJf+16bBHwBxKtqhu/4\ncuAeVf3K9zwR2AhUxyWD1qp6i9/5Pvdd420P3nsN4DvgXVV9PtjnNyYc2lGNKa2GwNZ8+zb6bTcD\nbhGRv/vti8J965cCXrs53/OduYnBpznwsYjk+O3LwtUwmgHXishlfseqAF8V432UiIhUB6YC31ti\nMF6x5GAi2TagUb59zTjSzLIJeEZVh+V/oa8/IP9rm3J0E03+avUm4DZVnVfA+TYB76jqX/MfK4iI\nzADOLeTwN6p6SSGvq4prStukqn8rzrWMKQ3rczCR7HsgS0Tu9fUj9AE6+x1/C7hTRM7ydSbHisgl\nIhLne222iNwjIlVE5Ip8ry3I68AwEWkKebeWXu479i5wmYj0EpHKIlLN16mdPwEBoKp/VtX4Qh6F\nJYYoYBJwALi1WL8hY0rJkoOJWKqaCfTBfVD+AVwHTPY7vgDXGf0KsBtYA9yS77X9cR3LN+E6lP2b\nkfLXHP4DTAFmisg+YB5wlu98W4ArgMeAHbhaxoME92/sHOASoCewR0TSfI+uQbyGMYCHHdIi0gR4\nGzgB90f2pqq+nK9MPdw3rga4Jq7hqjrOk4CMOQYRmQ+8pqrjQx2LMaHmZXJoADRQ1UW+avwC4EpV\nXeFXZghQVVUf9SWKVUCCqmZ5EpQxfnx3O60GduFqDq8BLVV1e0gDMyYMeNaspKopqrrIt50OrMDd\nXeJvG1DDt10D+MMSgylHpwCLcM1KDwDXWGIwximXcQ4i0hyYg7sPPN1vfyXcrX4nA/HAdao6w/OA\njDHGFMnzDmlfk9Ik4D7/xODzGLBIVRsC7YFXRSTe65iMMcYUzdNxDr5b7ybjRnF+UkCRc4BnAFR1\nnYhswFX1f853HhvGbYwxpaCqpZqI0rOag4gIbl6b5ao6opBiK4EevvIJuMSwvqCCqhqxj8GDB5f5\nHD9t/QmGwNA5QyMu9kj/3Vv8Fn+kPsrCy5pDV6AvsEREFvr2PYYbhYqqvgEMA8aKyGJconpYVXd7\nGFPEev675+nSuAu/pPwS6lCMMccBz5KDqn7LMWomqroLuKyoMgZW/7Ga5N+S+aLvF/R5v0+owzHG\nHAdshHQ5SEpKKtPrh38/nLs73037Bu3ZfXA3fxz4IziBFUNZYw81iz+0LP7IFRFTdouIRkKcXtiW\nto3Wr7Vm9d9XUy+mHt3GdePx8x6nZ6ueoQ7NGBPmRAQtZYe0zcoa5kb8MIKb295MvRi3ZkzHBh1Z\nsG2BJQcTMdy9KcZrwf4CbckhjO05tIdRC0fxy1+PdEJ3atiJT1d9GsKojCm547XmX168SMDW5xDG\nXv/5dS456RKa1WqWt69TYid+2WZ3LBljvGU1hzB1MPMg/5n/H2bdPOuo/SfXPZkd+3eQejCV2tVr\nF/JqY4wpG6s5hKnxi8dzZsMzOeOEM47aX7lSZdoltLPagzHGU5YcwlBWThYvfP8Cj3R9pMDj1rRk\nTHgYMGAAQ4cODXrZcGDNSmFo8vLJJMYlcm7TgpcY7tSwE9PXTC/nqIypeJo3b86YMWO48MILS/X6\nkSNHelI2HFjNIcyoKs9/93yhtQaAjokdreZgTBD4xgEUeCwr6/heWsaSQ5iZtX4WGdkZXHJygWvM\nA3BqvVP5Pe139h7aW46RGVOx3HzzzWzatInLLruM+Ph4XnjhBSpVqsSYMWNo1qwZPXr0AODaa68l\nMTGRWrVq0a1bN5YvX553jltvvZUnnngCgOTkZBo3bsy///1vEhISaNiwIePGjStV2T/++IPLLruM\nmjVrctZZZ/H4449z3nnnef9L8WPJIczk1hoqSeH/NFUqVaFtQlsWpiwstIwxpmjvvPMOTZs2Zdq0\naaSlpXHdddcB8M0337By5Uq++OILAC655BLWrl3Lzp076dixIzfddFPeOUTkqDEG27dvZ9++ffz+\n+++MHj2au+++m71795a47N133018fDzbt29n/PjxvP322+U+mNCSQxj5aetPrN29lhvOuOGYZa1p\nyVQUIsF5lFVu89KQIUOoXr06VatWBdw3/tjYWKKiohg8eDCLFy8mLS0t4HUAUVFRDBo0iMqVK/Pn\nP/+ZuLg4Vq1aVaKy2dnZfPTRRzz55JNUq1aN0047jX79+pX7QEJLDmHk+e+e58GzHySqctQxy3ZK\n7MSCbQvKISpjvKUanEewNGnSJG87JyeHgQMHcuKJJ1KzZk1atGgBwK5duwp8bd26dalU6cjHakxM\nDOnp+RfALLrszp07ycrKOiqOxo0bl+k9lYYlhzCxatcqvtn4Df079C9W+U4NO7Hgd0sOxpRFQU01\n/vvee+89pkyZwuzZs9m7dy8bNmwAjq4BlKS5pzhl69evT5UqVdi8eXPePv/t8mLJIUzkTssdGx1b\nrPKn1TuNzfs2k3Y47diFjTEFSkhIYN26dYUeT09Pp2rVqtSpU4f9+/fz2GOPHXW8JCuuFbds5cqV\n6dOnD0OGDOHgwYOsXLmSd955p+L0OYhIExH5WkSWicivInJvIeWSRGShr0yyV/GEs637tjJ5xWTu\nOeueYr8mqnIUZ5xwBotSFnkYmTEV26OPPsrQoUOpU6cOkydPDvgAvuWWW2jWrBmNGjXijDPO4Oyz\nzz6qTP5O5qI+wEtS9pVXXmHv3r00aNCAfv36ceONNxIdHV2at1hqnq3nICINgAaqukhE4oAFwJWq\nusKvTC3gO+AiVd0iIvV8q8PlP1eFXs/hnzP/SWZOJiMuLmyp7YLd9dldnFz3ZO7vcr9HkRlTdkWN\nJTDF88gjj7Bjxw7Gjh1b4PHCfsdlWc/Bs5qDqqao6iLfdjqwAmiYr9hfgMmqusVXruBengos9WAq\nYxaN4R9n/6PEr7U7loypmFatWsWSJUtQVX788UfGjBnDVVddVa4xlEufg4g0BzoA8/MdOgmo42t+\n+llEbi6PeMLJyJ9HcunJl9K0ZtMSv9buWDKmYkpLS+Pqq68mLi6OG264gYceeojLL7+8XGPwfG4l\nX5PSJOA+Xw3CXxTQEegOxADzROQHVV2T/zxDhgzJ205KSqoQa7sezDzIy/NfZvYts0v1+tYntGZD\n6gb2Z+wvdke2MSb8nXnmmaxZE/AxeEzJyckkJycHJQZP15AWkShgGjBDVQMa1EXkEaC6qg7xPR8F\nfK6qk/KVq5B9DiN/GsmMtTOYcuOUUp/jzDfP5OU/v8w5Tc4JYmTGBI/1OXgvovocxHXFjwaWF5QY\nfD4FzhWRyiISA/wJWF5I2QolKyeL4fOGM/DcgWU6T6dEG+9gjAk+L5uVugJ9gSUikjsJ0GNAUwBV\nfUNVV4rI58ASIAd4S1WPi+QwafkkGsY3LPM3/k4NO/H95u+DFJUxxjieJQdV/ZZi1ExUdTgw3Ks4\nwlHutNxDLyj7wh8dEzvyyo+vBCEqY4w5wkZIh8DMdTPJysmi90m9y3yuNie0Ye3utRzMPBiEyIwx\nxrHkEAK503IHYzh81SpVObXeqSzevjgIkRljjGPJoZzN3zKf9anrub719UE7pw2GM6Z0mjdvzldf\nfVWmc4wbN67cF+IpD5YcyllJpuUuLrtjyZjSsdtsC2fJoRyt3LWSbzd9y+0dbg/qeTs17MQvKVZz\nMKYk8i8TOnz4cH744QfOOeccateuTfv27ZkzZ05e+XHjxtGqVStq1KhBy5YtmTBhAitXruTOO+9k\n3rx5xMfHU6dOnRC+oyDLnUY2nB8uzMh3+ye365PJTwb9vAcyDmj1odX1YObBoJ/bmLIK57/f5s2b\n6+zZs1VVdcuWLVq3bl2dMWOGqqrOmjVL69atq7t27dL09HStUaOGrl69WlVVU1JSdNmyZaqqOm7c\nOD333HND8wZ8Cvsd+/aX6nPX8+kzjLN131Y+Xvkxa/5e8iHxx1I9qjon1T2JpduX0rlR56Cf3xgv\nyZPBWadAB5eteejdd9+ld+/eXHzxxQD06NGDM888k88++4xrrrmGSpUqsXTpUho3bkxCQgIJCQnu\nuhW0WcqSQzn513f/ol+7ftSNqevJ+TslduKXbb9YcjARp6wf6sGyceNGPvzwQ6ZOnZq3Lysriwsv\nvJCYmBjef/99hg8fTv/+/enatSsvvvgip5xySggj9pb1OZSD3/b8xrtL3+XR8x717BodEzvaDK3G\nlJD/7eRNmzbl5ptvJjU1Ne+RlpbGww8/DECvXr2YOXMmKSkpnHrqqdxxxx0B56hILDmUg0FfD+Ke\nzvdwQuwJnl3Dpu82puT8lwm96aabmDp1KjNnziQ7O5tDhw6RnJzM1q1b2bFjB59++in79+8nKiqK\n2NhYKleunHeOLVu2kJmZGcq3EnSWHDy2dPtSvlj3BQ+e86Cn12nXoB0rdq4gIzvD0+sYU5HkLhNa\nu3ZtPvzwQz799FOGDRvGCSecQNOmTXnxxRdRVXJycnjppZdo1KgRdevWZe7cuYwcORKA7t2707p1\naxo0aMAJJ3j3BbC8eTpld7BE8pTdl0+8nAtbXFguS3me8doZvH3V23RM7Oj5tYwpLhtL4L2ImrLb\nwLebvmXJ9iUMOHNAuVyvU0MbDGeMCQ5LDh5RVQZ+OZAnk56kapWq5XLN3DuWjDGmrCw5eOSzNZ+x\n59Ae+rbtW27XtDuWjDHBYsnBA9k52Tw6+1GGdR9G5UqVy+267Ru0Z9nOZWRmV6y7Jowx5c/LZUKb\niMjXIrJMRH4VkXuLKNtZRLJEpI9X8ZSnCUsnUKNqDS47+bJyvW5cdBzNajZj+c7jYjE9Y4yHvKw5\nZAIPqGproAtwt4iclr+QiFQGngc+ByJ+NMnhrMMMSh7Ec92fC8ngGGtaMsYEg2fJQVVTVHWRbzsd\nWAE0LKDo34FJwE6vYilPby54k9Prn855zUIzv7tN323CkYjYw8OHF8plbiURaQ50AObn298IuAK4\nEOgMRPTN0GmH03hm7jN80feLkMXQqWEnPlj+Qciub0x+NsYhMnmeHEQkDlczuM9Xg/A3Ahioqiou\n/RWaAocMGZK3nZSURFJSUvCDLaOXfniJHi170K5Bu5DF0L5Be5ZsX0JWThZVKtm8isYcT5KTk0lO\nTg7KuTwdIS0iUcA0YIaqjijg+HqOJIR6wAHgDlWdkq9c2I+Q3rl/J6e9eho/3vEjLWu3DGksp7xy\nCpOunUSbhDYhjcMYE1phOULaVxMYDSwvKDEAqGpLVW2hqi1wtYsB+RNDpBg2dxg3nnFjyBMD2GA4\nY0zZeXm3UlegL3CBiCz0Pf4sIn8Tkb95eN1yt3HPRt5e8jaPn/94qEMB7I4lY0zZedYorarfUoLk\no6q3eRWL1wYnD+auM+8iIS4h1KEArubwycpPQh2GMSaCWY9lGf2641dmrJ3B6ntWhzqUPB0SO7B4\n+2Kyc7LLdYS2MabisOkzyuj/vvo/Hun6CDWr1Qx1KHlqVatFQmwCq/5YFepQjDERypJDGXy/+XsW\npSzirs53hTqUAJ0aWqe0Mab0LDmUUu6U3EO6DaFalWqhDieAjZQ2xpSFJYdSmrF2BrsO7OLmdjeH\nOpQC2R1LxpiysORQCjmakzcld7iOQu6Y2JFFKYvI0ZxQh2KMiUCWHEph4tKJxETFcMUpV4Q6lELV\nqV6HejH1WPPHmlCHYoyJQJYcSigjO4Mnvn4iZFNyl0Snhp2sackYUyqWHErozQVvckq9U+jWvFuo\nQzmmjg062h1LxphSseRQQtPXTGfYhcNCHUaxWM3BGFNans7KGizhNCurqoZ9c1KuXQd20erlVqQ+\nkkolse8BxhxvwnJW1ooqUhIDQL2YetSqVov1qetDHYoxJsJYcqjgbDCcMaY0LDlUcJ0Srd/BGFNy\nlhwquI6JdseSMabkLDlUcLkT8IVLh74xJjJ4mhxEpImIfC0iy0TkVxG5t4AyN4nIYhFZIiLfiUhb\nL2M63pwQewKx0bFs2LMh1KEYYyKI1xMDZQIPqOoiEYkDFojILFVd4VdmPXC+qu4VkYuBN4EuHsd1\nXMltWiru+taqyv7M/aQdTiMtI420w2nsz9xP54adqR5V3eNojTHhwNPkoKopQIpvO11EVgANgRV+\nZeb5vWQ+0NjLmI5HnRI7MeqXUSzfufzIB77vQz/3Z3pG+lGJoFqVasRHxxNfNZ746HgOZR2ifYP2\nTLh6QqjfjjGmHJTblKIi0hzogEsAhekPTC+PeI4n17e+ntSDqWRmZ1I/tj4ta7fM+9D3/xkXHUd8\ntPuZf3nRA5kHaP96ez5e8TFXnXZViN6JMaa8lMsIaV+TUjIwVFULXPleRC4AXgW6qmpqvmM6ePDg\nvOdJSUkkJSV5Fq8p2LebvuW6D69j6YCl1I2pG+pwjDH5JCcnk5ycnPf8ySefLPUIac+Tg4hEAdOA\nGao6opAybYGPgItVdW0Bx8Nm+ozj3QOfP8COAzt4r897oQ7FGHMMYTt9hri5JkYDy4tIDE1xiaFv\nQYnBhJdnuj/Dj1t/5JOVBVYAjTEVhKc1BxE5F/gGWALkXugxoCmAqr4hIqOAq4BNvuOZqnpWvvNE\nfM1hzx44eBASE8v3uqrw3//C5s3wr39BMKaGmrtxLtdPut6al4wJc2WpOdisrOWkb1+YMgVeeglu\nvz04H9LHkpEBd90FP/7onv/1r3DPPcE59/2f38+uA7t4t8+7wTmhMSbowrZZyTjr18Pnn7vHq6/C\npZfC7797e82dO6FHD9i1C77/Hj75BIYOhW++Cc75h3Ufxvyt8/l05afBOaExJqxYcigHzz8PAwbA\nOefA/PnQuTN06AATJrhmn2BbuhT+9Cc47zz46COIi4OWLeGdd+CGG1wTU1nFRMUw5vIx3DX9LnYf\n3F32Expjwoo1K3ls61Zo0wZWr4Z69Y7sX7AA+vWDU0+FkSOhfv3gXG/KFOjfH0aMgJtuCjz+wgvw\nwQcwdy5Uq1b269034z52H9rNO1e9U/aTGWOCypqVwtiLL8Kttx6dGAA6dYKff4ZWraBtW/j447Jd\nR/VIDWXatIITA8BDD7lrDhgQnFrLsO7DmLd5HlNWTSn7yYwxYcNqDh7atQtOPtk18zRqVHi57793\ntYguXeDll6F27ZJd59Ah19m8bBl8+ik0PsYEJPv3uyauO+4ITgf1Nxu/4cbJN7J0wFLqVK9T9hMa\nY4LCag5h6j//gWuuKToxgPugXrTIJYW2bV3HdXGlpMAFF7gEMXfusRMDQGysq6kEq4P6/Gbnc81p\n13D/5/eX/WTGmLBgNQeP7NvnOoHnz3fNOMX11VfuVtdevVyTVHx84WUXLoQrr3Tln3gCKpUw1c+a\n5Wos8+dDkyYle21++zP20/b1toy4aASXnXJZ2U5mjAkKqzmEoZEj4aKLSpYYAC68EJYsgZwcV4vw\nmyblKJMnuwQyfDgMHlzyxADQsyc88AD06eMG6JVFbHQsYy4fw52f3Wl3LxlTAVjNwQMHD0KLFvDl\nl3DGGaU/z/Tpri/h6qvh2WchJsZ1Ig8dCm+95cYudOxYtlhV4S9/gapVYezYsg/Ou3fGvew5tIe3\nr3q7bCcyxpSZ1RzCzOjRrnO5LIkBoHdvV4vYtcuNi/j6a7jxRnc30vz5ZU8M4JLBqFGuieqVV8p+\nvme7P8t3m79j6qqpZT+ZMSZkrOYQZBkZcNJJ8OGHcNZZxy5fXJMnw513uqakUaOgepAXZNuwAc4+\nG95/H7p1K9u55vw2h5s+uomlA5ZSu3oJb70yxgSNza0URsaOdSOfZ80K/rkPH4boaO/mZfryS7j5\nZlcradq0bOf6+/S/sy9jH+OvHB+c4IwxJWbNSmEiOxueew4ee8yb81et6u2EfT16wIMPBqeD+tke\nz/Ltpm+ZtnpacIIzxpQrSw5BNHky1K0LkbxI3YMPuoF7d95ZthHUcdFxjL58NHdOu5PUg6nHfoEx\nJqxYcggSVRg2zNUaymM6bq/kdlAvXlz2Duqk5klceeqV9P24L9vTtwcnQGNMufAsOYhIExH5WkSW\nicivInJvIeVeFpE1IrJYRDp4FY/Xpk93CeKSS0IdSdnFxLgR1M88A3PmlO1cL/R8gdPrnc4ZI8/g\nv/P/S1ZOVnCCNMZ4yrMOaRFpADRQ1UUiEgcsAK5U1RV+ZXoD96hqbxH5E/AfVe1SwLnCukNaFbp2\nhfvug+uvD3U0wRPMDurlO5dzz/R72H1wN6/2fpWuTbsGJ0hjTKHCskNaVVNUdZFvOx1YATTMV+xy\nYLyvzHyglogkeBWTV+bMcWMRrrkm1JEEVzA7qE+vfzqzb5nNwHMHcv2k67n1k1utqcmYMFYufQ4i\n0hzoAMzPd6gR4L/0zBagGFPHhZdhw2DgQKhcOdSRBF+wOqjBfYu54YwbWHH3CurH1LemJmPCmOfJ\nwdekNAm4z1eDCCiS73n4th8V4KefYOVKt0Z0RZTbQb1kiRudvXJl2c8ZXzWeF3q9wJxb5/Dxyo85\n880z+W7Td2U/sTEmaKp4eXIRiQImA++q6icFFNkK+M8H2ti3L8CQIUPytpOSkkgKk/tFhw2Df/7T\nDU6rqGJi3NTe//0vnH++G6X9+ONuFbuyyG1qen/Z+1w/6Xp6tOzB8z2eJyEu4loWjQkLycnJJBc2\nW2cJedkhLbj+hD9U9YFCyvh3SHcBRkRSh/SyZdC9O6xf7z5Ajwf79rlbXEeMCF6SAEg7nMZTc55i\n3OJxDDp/EAM6D6BKJU+/uxhT4YXl9Bkici7wDbCEI01FjwFNAVT1DV+5V4CLgf3Abar6SwHnCsvk\n0Levm1xv4MBQR1L+vEoSdleTMcETlskhmMIxOaxf7ybWW7cOatYMdTSh40WSUFXeX/Y+D818yJqa\njCkDT25lFZG+fttd8x0LwsrDke1f/3J38BzPiQGgRg03KnztWjj9dNcn0bdv2TquC7qr6f7P7+eD\nZR+wee/mY5/AGFNmhdYcRGShqnbIv13Qc6+FW81h61Zo0wZWr4Z69UIdTXjxoiaxctdKPln5CfO2\nzGPe5nlEV47m7CZnc3Zj9+iY2JGqVaoG5w3gai5b07ayOGUxS7YvoUbVGtzY5kbqVK8TtGsYUx48\naVay5FC4Bx90y3i+9FKoIwlf/kniwgvh7rvh3HPLPu+UqrI+dX1eovhh6w+s3LWStglt85JFl8Zd\naFKzeItiH8o6xPKdy/MSweLti1m8fTFVKlWhXUI72iW0Y2vaVqavmU7vk3rTv0N/LmhxAZXEpiUz\n4c+SQznatcsNClu6FBo1CnU04W/fPrfGxciRUKUKDBjgpuSoUSN419ifsZ+ff//ZJYwiahe7D+5m\n8Xa/JJCymHWp6zixzol5iaBdg3a0TWhLg7gGR11j98HdvLfkPUYvHM2+w/u4vcPt3Nr+VhrXiLgx\nm+Y44lVyOAis9T1tBazzO9xKVcvt5s1wSg6DBkFKCrz5ZqgjiSyqkJzsksSsWXDddS5RtG/vxbWO\nrl3M2zKPZTuXER8dT7sG7fISQduEtpxe//QSNUmpKr9s+4VRv4zi/WXvc3aTs+nfoT+Xnnwp0ZUr\n8GAXE5G8Sg7Ni3qhqv5WmguWRjglh9tuc+3orVqFOpLItW2bW2f7jTegcWOXJK67DqpV8+6aGdkZ\nRFWKQoI4n/qBzANMXj6ZUQtHsXLXSm5pewv9O/bn1HpBuKfXmCAol1tZRaQecD6wUVUXlOZipRVO\nycEET1YWfPaZq00sWAD9+rk7wE48MdSRldzqP1YzZuEYxi8eT6varejfoT/Xtr6WuOi4UIdmjmNe\n1Rw+Ax5R1V9FJBFYCPyEa2J6S1XLrTvWkkPFt26dq0mMG+eamgYMgMsuc/0UkSQzO5Ppa6YzeuFo\n5m6ay7WnX0v/Dv1pm9CWalWqBbXmYsyxeJUclqlqa9/2Y8CpqnqLiMQD36tqm1JHXNIgLTkcNw4d\ngkmTXG1i40a44w73aJh/svcI8Hva74xfNJ7xi8ezce9GDmcdJiYqhpioGGKjY93PqNiCn/t+NqnZ\nhO4tutOidotQvx0TgbxKDotUtb1v+ytcbWGi7/liVW1X2oBLHKQlh+PS4sXw+uvw/vtuipKHHw51\nRGWTnZPNwayD7M/Yz4HMA+zP9P0s4vna3Wv5cv2XxEbH0rNlT3q27MkFLS6wMRemWLxKDtOAL3Cz\npI4GWqpqqojEAD/l1irKgyWH49u2bdClixuVXpFW2isuVeXXHb8ya/0sZq2fxXebvuPUeqfSs2VP\nerTswTlNzgnqIEBTcXiVHBKAp4AGwKuqOtO3/wKgk6oOL2W8JQ/SksNxb8kStzLd1Knwpz+FOprQ\nOpx1mHlb5jFr3Sy+3PAlK3auoGvTrvRo0YOerXrS5oQ21rdhAJt4zxwnpk2Dv/4V5s2DZs1CHU34\nSD2YyleL0vwQAAAVtklEQVQbvuLL9V8ya/0s0jPS6d6ye17NwgbqHb+8qjlMxU21XdCJVVUvL80F\nS8OSg8n10ktuxPW33wZ3lHVFsiF1Q16imL1hNnsP7T3qeP5ahfj9ifsfqyyViY2OJS46jtioWGKj\nY/N+5u3L/zxfmejK0QiCiFBJKpV4u5JUom71utSpXsdqQ6XgVXLYiVvTeSJH1n7OvYiq6pzSXLA0\nLDmYXKruNtfNm2HKlIq5bncwqSrZmn3U86OO+63Km/9YZk4mBzIPkJ6Rzv6M/ezP3J/3M/++9Iz0\nwOOZ+8nIzkBVUZQczcnbVvU9P8Z2tmaz68AuDmQeIDEukcT4RBLjEmkY3zDweXwi9WLqHdfzXh3I\nPMD29O2kpKeQkp5Cn9P7eJIcqgA9gRuBNsBnwERVXVbawEvLkoPxl5kJvXu7hZZs8sPjw8HMg6Sk\np/B72u9sS9/GtrRtR7bTfdtp29h3eB8JcQkkxiVyQuwJ1KpWq8BH7Wq1j3pes1rNsF15MDsnm50H\ndrItbVveh37eY//Rzw9nHaZBXIO8x6c3fuptn4OIVMUlieHAEFV9pVgnFxkDXALsKGhchG/U9bu4\nTu8qwHBVHVdAOUsO5ih79sDZZ8N997lR1caA66zfvn87v6f9zs79O9l7eC+pB1PZc2jPkcfhPUc9\nTz2Yyt7De4mJiglIIjWq1iA+Ot49qsYTFx2Xt13Yz+pVqiMiZGZnkp6RTlpGGukZ6Uc90g4XsM+v\nXFpGGn8c+IOU9BT+OPgHdarXOepDv0Gs+5kYn3jU/ppVax7V/OZZh7SIVMN9uN8ANAemAGNUdWux\nTi5yHpAOvF1IchgCVFXVR32JYhWQoKpZ+cpZcjAB1q2Drl3hnXegZ89QR2MiWY7mkJ6RHpA00jLS\nSDucVvBP33buh3nu/szsTCpXqkx2TnZeQsl9xEcf/byo/XVj6pIYl0j92PqlrtWUJTkUekUReQdo\nDUwHnlLVpSU9uarOPcYEftuAtr7tGsAf+RODMYVp1Qo++ACuuQbmzIHTTgt1RCZSVZJK1KhagxpV\na9C0ZtMynSszO5NszaZq5aoR3YleVJ9DDrC/kNepqhbrXhFfcphaSM2hEvAVcDIQD1ynqjMKKGc1\nB1Oo8ePhqafghx+gfv1QR2NM+PCk5qCq5dHl/xiwSFWTRKQVMEtE2qlqWjlc21QQ/frBqlXQpw98\n+SVUtcHCxpRZqLvnzwGeAVDVdSKyATgF+Dl/wSFDhuRtJyUlkZSUVD4RmogwdKhbE+KOO1xNIoJr\n88aUWnJyMsnJyUE5l+cjpI/RrPRvYK+qPumbrmMB0FZVd+crZ81K5pgOHIBu3eCqq+Cxx0IdjTGh\nF7bTZ4jIRKAbUA/YDgwGogBU9Q3fHUpjgaZAJeBZVZ1QwHksOZhi2bbNzb304otw7bWhjsaY0Arb\n5BAslhxMSSxa5G5tnT4dOncOdTTGhE5ZksPxO87cVFjt27s1qq+8EjZtCnU0xkSmUHdIG+OJyy+H\nNWvcUqPffgvx8aGOyJjIYs1KpsJSdVN8p6TAJ5/YJH3m+GN9DsYUIjMTLr4Y2raFf/7TzclU0kf1\n6m6Sv9xHmzZw6qlQrVqo350xRbPkYEwRUlNdB/XWrVCrVuGP2rUL3p+WBr/+6h5Ll7qf69ZB8+aB\nSaNVK6uhmPBhycGYcpaR4UZl508aKSmuVtGmzdFJo7EtxmZCwJKDMWEiPR2WLTs6aSxa5KYXf+UV\nW97UlC9LDsaEsYwMeOEFtzDRI4/A/fdDVFSoozLHA0sOxkSAdevgrrtc09Mbb0CXLqGOyFR0lhyM\niRCq8P778I9/wBVXwLBhriPcGC/YCGljIoQI3HADLF/unrduDRMmuKRhTDixmoMxIfTDD/C3v0FC\nArz2Gpx4YqgjMhWJ1RyMiVBdusDPP0OvXm776afh8OFQR2WMJQdjQi4qCh56CBYsgJ9+chMHzpkT\n6qjM8c4m3jMmTDRrBp9+6uaB6tsXevSABx90xw4fdo+MjCPbxX1etSrUqeM6vnN/+m/XqAGVKtjX\nxMxM9/7j4kIdSfnKyYHffnPja5YsKdu5rM/BmDCUlgaDBsG0aRAd7T7gcx8leR4dDYcOuSlEdu92\nP/23d+92K+jVrHl0wqhTB+rXh5YtoUWLIz8j4cN29Wq45hrYsMHNznvbbXDhhRUvAaamHkkCuT+X\nLXP/lm3bupH5zz8fpreyisgY4BJgR0HLhPrKJAEv4VaI26WqSQWUseRgjEeystwEg/4JIzUVtm93\nH7Dr17ufGza45NCy5dFJI3e7cWOoEuK2iA8/dGNJhg6Fq692d4KNHeveU79+cOutLt5IkjtVS/5E\nsGePm54lNxHk/vS/NTpsxzmIyHlAOvB2IWtI1wK+Ay5S1S0iUk9VdxVQzpKDMSGm6gbw5SaM3KSR\nu71jh0sQ/gmjVy/o2NH72DIy4OGHYcoUlyA6dTr6+KJFLklMmOBuH77tNle7iI0NXgz79sHXX8PM\nme7DvHLlI48qVQK3C9rnv715s0sEq1dD06buw98/ETRvfuzaUNgmBwARaQ5MLSQ53AU0UNVBxziH\nJQdjwtzhw7Bx45GksXatG/DXvj088YRb29sLmzfDdde5ZrDx44seVJiR4Zrqxo51i0D16eMSRdeu\nbgxKSWRnuzvNZs50j0WL3B1nvXq5D29VVyvLznaPkmxnZUFiojvP6adDTEzpfjeRnBxym5NaA/HA\nf1T1nQLKWXIwJgIdOuQ+iJ991n3IPfGE+yAOli++cM1F//iHu+OrJP0K27bBu++6+DIzXZNTv35F\nz6C7ceORZDB7NjRq5JJBr15w3nml/xD3SiQnh1eAjkB3IAaYB1yiqmvyldPBgwfnPU9KSiIpKcm7\noI0xQZWR4b7VDxvmmpwGDYJu3Up/vuxseOopGDXKNRWV5Vyq8OOPLkl88AF07uxqE1de6ZJGcvKR\nhJC7NkivXu5nw4alv64XkpOTSU5Oznv+5JNPRmxyeASorqpDfM9HAZ+r6qR85azmYEwFkJnpvq0/\n84z71j1okLuTqCRNOjt3wl/+4ppeJk6EBg2CF9/Bg/Dxxy5R/Pyzu8ZZZ8FFFx1pLoqku54iueZw\nKvAKcBFQFZgPXK+qy/OVs+RgTAWSlQX/+5+7q6huXZckevU6dpL4/nu4/nq4+WZXc/Dy7qiUFIiP\nD26ndXkL2+QgIhOBbkA9YDswGNfHgKq+4SvzEHAbkAO8paovF3AeSw7GVEDZ2e7uoqefdrfJDhoE\nvXsHJglVGDECnnsORo+GSy8NTbyRJmyTQ7BYcjCmYsvJgY8+ckmiShXXcX355a4JZ+9euP121xn8\n4YfuFllTPJYcjDEVQk6OG6vw9NOu6WnAAHjxRdf5+9JLbtS3KT5LDsaYCkUVpk+HV1+Fm25yD1Ny\nlhyMMcYEsPUcjDHGBJUlB2OMMQEsORhjjAlgycEYY0wASw7GGGMCWHIwxhgTwJKDMcaYAJYcjDHG\nBLDkYIwxJoAlB2OMMQEsORhjjAlgycEYY0wASw7GGGMCeJocRGSMiGwXkaXHKNdZRLJEpI+X8Rhj\njCker2sOY4GLiyogIpWB54HPgVJNLWuMMSa4PE0OqjoXSD1Gsb8Dk4CdXsZijDGm+ELa5yAijYAr\ngJG+XbaijzHGhIEqIb7+CGCgqqqICEU0Kw0ZMiRvOykpiaSkJM+DM8aYSJKcnExycnJQzuX5MqEi\n0hyYqqptCji2niMJoR5wALhDVafkK2fLhBpjTAmVZZnQkNYcVLVl7raIjMUlkSlFvMQYY0w58DQ5\niMhEoBtQT0Q2A4OBKABVfcPLaxtjjCk9z5uVgsGalYwxpuTK0qxkI6SNMcYEsORgjDEmgCUHY4wx\nASw5GGOMCWDJwRhjTABLDsYYYwJYcjDGGBPAkoMxxpgAlhyMMcYEsORgjDEmgCUHY4wxASw5GGOM\nCWDJwRhjTABLDsYYYwJYcjDGGBPAkoMxxpgAniYHERkjIttFZGkhx28SkcUiskREvhORtl7GY4wx\npni8rjmMBS4u4vh64HxVbQs8DbzpcTzGGGOKwdPkoKpzgdQijs9T1b2+p/OBxl7GY4wxpnjCqc+h\nPzA91EEYY4yBKqEOAEBELgBuB7oWVmbIkCF520lJSSQlJXkelzHGRJLk5GSSk5ODci5R1aCcqNAL\niDQHpqpqm0KOtwU+Ai5W1bWFlFGv4zTGmIpGRFBVKc1rQ9qsJCJNcYmhb2GJwRhjTPnztOYgIhOB\nbkA9YDswGIgCUNU3RGQUcBWwyfeSTFU9q4DzWM3BGGNKqCw1B8+blYLBkoMxxpRcxDYrGWOMCU+W\nHIwxxgSw5GCMMSaAJQdjjDEBLDkYY4wJYMnBGGNMAEsOxhhjAlhyMMYYE8CSgzHGmACWHIwxxgSw\n5GCMMSaAJQdjjDEBLDkYY4wJYMnBGGNMAEsOxhhjAniaHERkjIhsF5GlRZR5WUTWiMhiEengZTzG\nGGOKx+uaw1jg4sIOikhv4ERVPQn4KzDS43hCIlgLfodCJMcOFn+oWfyRy9PkoKpzgdQiilwOjPeV\nnQ/UEpEEL2MKhUj+DxbJsYPFH2oWf+QKdZ9DI2Cz3/MtQOMQxWKMMcYn1MkBIP/6prZYtDHGhJio\nevtZLCLNgamq2qaAY68Dyar6P9/zlUA3Vd2er5wlDGOMKQVVzf8FvFiqBDuQEpoC3AP8T0S6AHvy\nJwYo/ZszxhhTOp4mBxGZCHQD6onIZmAwEAWgqm+o6nQR6S0ia4H9wG1exmOMMaZ4PG9WMsYYE3nC\noUO6UCJysYis9A2SeyTU8RyLiDQRka9FZJmI/Coi9/r21xGRWSKyWkRmikitUMdaFBGpLCILRWSq\n73nExC8itURkkoisEJHlIvKnSIlfRB71/d9ZKiITRKRqOMde0CDXouL1vb81vr/pXqGJ+ohC4n/B\n939nsYh8JCI1/Y6Fffx+xx4UkRwRqeO3r0Txh21yEJHKwCu4QXSnAzeKyGmhjeqYMoEHVLU10AW4\n2xfzQGCWqp4MzPY9D2f3Acs5cudYJMX/H2C6qp4GtAVWEgHx+27cuAPo6Lt5ozJwA+Ede0GDXAuM\nV0ROB67H/S1fDLwmIqH+/Cko/plAa1VtB6wGHoWIih8RaQL0BDb67Stx/KF+c0U5C1irqr+paibw\nP+CKEMdUJFVNUdVFvu10YAVuLEfeYD/fzytDE+GxiUhjoDcwiiO3GUdE/L5veeep6hgAVc1S1b1E\nRvz7cF8uYkSkChAD/E4Yx17IINfC4r0CmKiqmar6G7AW9zceMgXFr6qzVDXH93Q+R8ZdRUT8Pv8G\nHs63r8Txh3NyKGiAXKMQxVJivm+CHXD/wRL87sLaDoTzKPCXgH8COX77IiX+FsBOERkrIr+IyFsi\nEksExK+qu4EXgU24pLBHVWcRAbHnU1i8DXF/w7ki4e/5dmC6bzsi4heRK4Atqrok36ESxx/OySFi\ne8pFJA6YDNynqmn+x9TdARCW701ELgV2qOpCAgcnAuEdP+7uu47Aa6raEXcH3FHNMOEav4i0Au4H\nmuP+kONEpK9/mXCNvTDFiDds34uI/B+QoaoTiigWVvGLSAzwGO6u0LzdRbykyPjDOTlsBZr4PW/C\n0ZkvLIlIFC4xvKOqn/h2bxeRBr7jicCOUMV3DOcAl4vIBmAicKGIvEPkxL8F963pJ9/zSbhkkRIB\n8Z8JfK+qf6hqFvARcDaREbu/wv6v5P97buzbF3ZE5FZc0+pNfrsjIf5WuC8Xi31/w42BBb756koc\nfzgnh5+Bk0SkuYhE4zpTpoQ4piKJiACjgeWqOsLv0BSgn2+7H/BJ/teGA1V9TFWbqGoLXGfoV6p6\nM5ETfwqwWURO9u3qASwDphL+8a8EuohIdd//ox64mwIiIXZ/hf1fmQLcICLRItICOAn4MQTxFUlE\nLsY1q16hqof8DoV9/Kq6VFUTVLWF7294C+4Gh+2UJn5VDdsH8GdgFa7z5NFQx1OMeM/FtdUvAhb6\nHhcDdYAvcXc/zARqhTrWYryXbsAU33bExA+0A34CFuO+fdeMlPhxnYjLgKW4ztyocI4dV7v8HcjA\n9Q/eVlS8uCaPtbhEeFEYxn87sAZ3l0/u3+9rERD/4dzff77j64E6pY3fBsEZY4wJEM7NSsYYY0LE\nkoMxxpgAlhyMMcYEsORgjDEmgCUHY4wxASw5GGOMCWDJwRw3RKSdiPy5FK9rKCIfFqPcZyJSo3TR\nlU1xri0i/Xyjlo05JhvnYI4bvmkROqnq3ws4VkXdtBUVloh8DTykqgtCHYsJf1ZzMCHhmxZlhYi8\nKW5hpC9EpJrvWLKIdPJt1/PNE4OI3Coin/gWkdkgIveIyEO+GVjniUjtIq4XDTwFXC9uIaPrRGSI\niLwjIt8C40WkmYh8IyILfI+z/WJd6hfDRyIyQ9yCNs/7XeM3cYvdFPXeOovIEl8MLxSyUEuSL45p\nvoVZRvqm1EBEbvS9fqmIPFfca4vINbj5m97z/b6qichz4hYXWiwiL5Txn9RUMJYcTCidCLyiqmcA\ne4CrffuLms2zNXAV0Bl4BtinbgbWecAthV1IVTOAJ4D/qWoHVf3Ad+hUoLuq3oSbJK6nqnbCzS31\nciGnawdcB7TBJZvcqY/9Yy7svY0F7lDVDkBWEe+zM3APbnGWVkAfEWkIPAdcALQHOoubovmY11bV\nSbj5yv7i+33FAleqau7CNk8XEoc5TllyMKG0QY/MO78AN6PksXytqvtVdRfug2+qb//SYrxeOHoK\nY8XNH3XY9zwaGCUiS4APcB/MBZmtqmm+1y0HmhVQJuC9iVuMKE5V5/v2T8gXj78f1S10lYObQ+dc\n3Df/ZHUzt2YD7wHnF+fafsdyr7cXOCQio0XkKuBgIXGY45QlBxNKh/22s3FLY4L7Rp37f7NaEa/J\n8Xueg1vPoSgFfUs/4Lf9ALBNVdviPoijCzlP/rgLum5xyhR3rn2h4NgL21/Y7zXvvL7+lbNw05pf\nCnxeRCzmOGTJwYST3A/L33AfzgDXlPC1RUkD4os4XgNI8W3fwtEfqmWmbsnSNBHJXZ7xhiKKn+Xr\nP6iEa8Kai5tiuZuI1BW3xvoNwJxiXDr3d5OGe4+IWyGvlqrOAP6BayozJo8lBxNK+b/15j4fDgwQ\nkV+Aun778/dF5N9WABH5m4j8rYDrfQ2cntshXcA5XgP6icgi4BQgvYBrFXc1tsLeW3/gLRFZiFsn\nem8hr/0JeAXXbLVeVT9Wt17FQN/7WAT8rKpT/V5zrGuPA173/V7jgakishiXeB4oxnsyxxG7ldWY\nciQisaq637c9ELfm8gP5yiQBD6rqZSEI0Rjg2G20xpjgukREHsX97f0G3FpAmYhaK9pUTFZzMMYY\nE8D6HIwxxgSw5GCMMSaAJQdjjDEBLDkYY4wJYMnBGGNMAEsOxhhjAvw/rLuk2DlUrlIAAAAASUVO\nRK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x1088a7550>"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We see that it hasn't converged yet: you still might expect to decrease the test error by adding more points.\n",
      "\n",
      "What about the other extreme?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plot_learning_curve(5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEZCAYAAAB8culNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNX5+PHPkxBIwk7YZF8FZRWVoqJGUGRxpQoiIKK1\nYrVVqxW1VePSVr9al59WtFXABaVFUVll0yAV3GURBAKI7DuGBAiQzPP740zCkI0sczNLnvfrdV+5\n29z7zMDcZ+45554jqooxxhgTKCbUARhjjAk/lhyMMcYUYMnBGGNMAZYcjDHGFGDJwRhjTAGWHIwx\nxhRgycFEFRGZKCKPhzoOYyKdJQcTbdQ/RSUR2Sgih0Qkwz99HOqYTHSqEuoAjPGABP2AIlVUNTvY\nxy0DBS5T1U9CHYiJbnbnYCKaiJwhIt+JyAERmQzE59t+mYgsFZH9IvK5iHQJ2NZDRL73v/a/IvKf\n3CIpEUkWkS0icp+IbAdeF+d+EVknInv8+9cNOF4vEVnsP9dSEbnQq7ft0XGNyWPJwUQsEakKfAi8\nAdQFpgC/xl+sJCJnAK8DtwD1gFeBaSIS53/tB8B4/2vfBa7ixCKpRv5tLYBbgT8AVwAXAKcA+4F/\n+s/VFJgBPKaqdYF7gfdFpH4Rsc/wJ5HCpmkneeuTRGSXiMwRka4l/byMKQ2xvpVMpBKRC4B3VbVp\nwLrPgQWq+rCIjAN2q+rDAdtXA7/1L76jqs0Cti0CPvW/NhmYA9RU1aP+7auAO3KLdETkFOBnIAGX\nDDqp6g0Bx/vYf443g/iezwG+w/2wu9M/dVTV9GCdwxiwOwcT2ZoAW/Ot+zlgviVwT+CvcqAZ7ld/\nYa/dnG95d25i8GsFfBBwrFVANu4OoyVwbb5znQc0LvvbK0hVl6jqEVU9rKpPAr8A5wfzHMaAVUib\nyLYdaJpvXUtgnX9+E/BXVf1b/hf66wPyv7ZFwGuhYKunTcBoVV1SyPE2AW+p6m/zbyuMiMwGehex\n+TNVHVSS4xQSozFBYXcOJpItBrJF5A/+eoTBwNkB2/8NjBGRnv7K5OoiMkhEavhfmyMid4hIFRG5\nMt9rC/MK8DcRaQEgIg1E5Ar/treBy0Wkn4jEiki8v1I7fwICQFUHqGrNIqZCE4OINBeR80Skqv/4\nfwKSgM9L+HkZU2KWHEzEUtVjwGDgRmAvMAR4P2D7t7jK6JeAfUAacEO+196Mq1gejqtQDixGyv+r\n/AVgGjBXRA4AS4Ce/uNtAa4EHgR24e4y7iG437GawMv+97IF6AcMUNX9QTyHMYDHFdIiMh4YBOxS\n1S5F7JMMPAfEAXtUNdmzgIwphoh8Cbysqm+EOhZjQs3rO4cJQP+iNopIHVxTwMtVtTNwjcfxGJNH\nRC4Qkcb+YqVRQGfAnjg2Bo8rpFV1kYi0KmaX64H3/bfkqOoeL+MxJp8OwH+B6sB64BpV3RnakIwJ\nD6FurdQeiBORT3HlqS+o6lshjslUEqr6b1yltTEmn1AnhzigB9AXSASWiMgXqpoW2rCMMaZyC3Vy\n2IyrhD4MHBaRz4BuuFYleUTE2nIbY0wZqGqZ+uIKdVPWj4De/nbhicCvcE+dFqCqETs98sgj5T7G\n9ozt1Pp7LWr9vRbpWekRF38kf/YWv8UfqVN5eHrnICLvAhcC9UVkM/AIrigJVX1VVVf7+59ZDviA\nf6tqocmhspuVNotL217K0ZyjTP1xKjd2vzHUIRljopjXrZWGlWCfZ4BnvIwjGsxMm8mVHa4kMS6R\nV755xZKDMcZToS5WqhSSk5PL9foj2UeYv2E+A9oN4LJTL+O77d+x9UD+PuO8U974QymSYweLP9Qi\nPf7yiIguu0VEIyFOr8zfMJ+HPn2IJTe7/t5+M+03dKzfkXvPvTfEkRljwpmIoBFaIW1KYMbaGQxq\nf7wvthFdR/D28rdDGJExJSciNlXAFGyhbspqTkJVmbF2Bu8NeS9v3QUtL2Dv4b2s2LmCLo0K7bLK\nmLBSme/8K4IXycHuHMLc2r1rycrOolujbnnrYiSG4V2GM2nFpBBGZoyJZpYcwtzMtJkMaj+owC+D\nEV1HMGnFJHzqC1FkxphoZskhzM1YO4PLTr2swPrODTuTlJDEZz9/FoKojDHRzpJDGEvPSuebbd/Q\np3WfQrdbxbQxoXfbbbfxxBNPBH3fULOmrGFsysopTFg6gVnDZxW6feuBrXQZ14Vt92wjvkp8BUdn\nTMn4m1OGOowitWrVivHjx9OnT+E/wiJBUZ+xNWWNUrn1DUVpWqspPU7pwYy1MyowKmOiS3HJKzs7\nu4KjCR+WHMKUT33MSpvFoFOLTg5gRUvGlMfIkSPZtGkTl19+OTVr1uTpp58mJiaG8ePH07JlSy6+\n+GIArr32Wk455RTq1KnDhRdeyKpVx7uAu/HGG3nooYcASE1NpVmzZjz77LM0atSIJk2aMHHixDLt\nu3fvXi6//HJq165Nz549+ctf/sL555/v/YfiZ8khTH299WsaVm9Iqzqtit1v8GmD+XTjp+w9tLdi\nAjMmirz11lu0aNGCGTNmkJGRwZAhQwD47LPPWL16NXPmzAFg0KBBrFu3jt27d9OjRw+GDx+ed4z8\nD6Ht3LmTAwcOsG3bNl5//XVuv/120tPTS73v7bffTs2aNdm5cydvvPEGb775pifPMxTFkkOYKqqV\nUn61qtViQLsBTFk1pQKiMsYbIuWfgiG3eCklJYWEhASqVasGuF/81atXJy4ujkceeYRly5aRkZFR\n4HUAcXFxPPzww8TGxjJgwABq1KjBmjVrSrVvTk4OU6dO5dFHHyU+Pp7TTjuNUaNGVWjdjSWHMDUj\nbUax9Q2BrGjJRDrV8k/B1Lx587x5n8/H/fffT7t27ahduzatW7cGYM+ewoe8T0pKIibm+KU1MTGR\nzMzMUu27e/dusrOzT4ijWbNm5XpPpWXJIQxtPbCVTembOKf5OSXa/9K2l7Jm7xo27N/gcWTGRJ/C\nimoC102aNIlp06axYMEC0tPT+emnn4AT7wBKU9xTkn0bNGhAlSpV2Lx5c966wPmKYMkhDM1Mm8ml\nbS+lSkzJur6Ki41jaKehvLPiHY8jMyb6NGrUiPXr1xe5PTMzk2rVqlGvXj0OHjzIgw8+eML20oy6\nVtJ9Y2NjGTx4MCkpKRw+fJjVq1fz1ltvWZ1DZTczbWaJ6hsC5RYthXN7cmPC0QMPPMATTzxBvXr1\neP/99wtcgG+44QZatmxJ06ZN6dy5M+ecc84J++SvZC7uAl6afV966SXS09Np3Lgxo0aNYtiwYVSt\nWrUsb7FM7CG4MJOVnUXDpxuy8a6N1EuoV+LXqSrtX2zP5Gsmc1aTszyM0JjSCfeH4CLF2LFj2bVr\nFxMmTCiwzR6CqwRSN6bSrXG3UiUGcP8JrGLamOixZs0ali9fjqry1VdfMX78eK6++uoKO78lhzAz\nY+0MLmtfuiKlXMO7DGfyD5PJ9lXepzqNiRYZGRn8+te/pkaNGlx33XXce++9XHHFFRV2fhvsJ4yo\nKjPTZjJ92PQyvb59Unta1WnF/A3z6d+uf5CjM8ZUpLPOOou0tLSQnd/uHMLIqt2rUFU6NehU5mNY\n0ZIxJhgsOYSRogb2KY2hnYYyY+0MMo8W/tCNMcaUhKfJQUTGi8hOEVlxkv3OFpFsERnsZTzhrqRd\nZhSnQfUG9G7Rmw9XfxikqIwxlZHXdw4TgGILv0UkFngK+BiouCc8wsy+w/tYumMpya2Sy30sK1oy\nxpSXp8lBVRcB+0+y2++B94DdXsYS7uasm0Nyq2QS4hLKfawrOlzBl1u/ZEfmjiBEZoypjEJa5yAi\nTYErgXH+VZX2SZmTDexTGolxiVzZ4Uom/zA5KMczxlQ+oW7K+jxwv6qquFrYIouVUlJS8uaTk5NJ\nTk72PLiKku3LZva62Tx58ZNBO+aIriO4f/793NXrrqAd05hoFIxhQidOnMjrr7/OokWLghhZ6aWm\nppKamhqUY4U6OZwJTPa3zqkPDBCRY6o6Lf+Ogckh2nyx5Qta1G5Bs1rB65L3olYXsS1jGz/u/pHT\nGpwWtOMaE22iqXuP/D+cH3300TIfK6TFSqraRlVbq2prXL3DbYUlhmg3c23wipRyxcbEcn2X65m0\nYlJQj2tMNMk/TOgzzzzDF198wbnnnkvdunXp3r07CxcuzNt/4sSJtG3bllq1atGmTRveeecdVq9e\nzZgxY1iyZAk1a9akXr3SdX0TtnK7kPViAt4FtgFHgc3ATcCtwK2F7DsBGFzEcTSadX65sy7ZvCTo\nx/1++/fa6vlWmuPLCfqxjSmpcP/+tmrVShcsWKCqqlu2bNGkpCSdPXu2qqrOmzdPk5KSdM+ePZqZ\nmam1atXStWvXqqrqjh07dOXKlaqqOnHiRO3du3do3oAW/Rn715fp+u1psZKqDivFvqO9jCVc/fzL\nz+zM3MnZTc4O+rG7NepG9bjqLN68mN4tegf9+MYEizxa/lbs+kj5i4befvttBg4cSP/+rgX+xRdf\nzFlnncXMmTO55ppriImJYcWKFTRr1oxGjRrRqFEjd+4oKZYKFOo6h0pvZtpMBrQfQGxMbNCPHdhT\nqyUHE86CcWEPhp9//pkpU6Ywffrx/s2ys7Pp06cPiYmJ/Oc//+GZZ57h5ptv5rzzzuMf//gHHTp0\nCGHE3rHuM0IsmE1YCzO8y3DeW/UeR7KPeHYOYyJZYHc1LVq0YOTIkezfvz9vysjI4L777gOgX79+\nzJ07lx07dtCxY0duueWWAseIFpYcQujQsUMs+nkR/dr28+wczWs3p0ujLsxeN9uzcxgTyQKHCR0+\nfDjTp09n7ty55OTkkJWVRWpqKlu3bmXXrl189NFHHDx4kLi4OKpXr05sbGzeMbZs2cKxY8dC+VaC\nypJDCH3y0yec2eRM6sTX8fQ8I7qM4LXvXvP0HMZEqtxhQuvWrcuUKVP46KOP+Nvf/kbDhg1p0aIF\n//jHP1BVfD4fzz33HE2bNiUpKYlFixYxbpx7frdv37506tSJxo0b07BhwxC/o+CwYUJLacrKKfRs\n2pOWdVqW+1hjZoyhfb323HPuPUGIrGiHjh2ix6s9eOTCRxjWpcRtBIwJimh6jiBc2TChYWDLgS30\ne7sfuw+Wryso9Q/sM+hU7+obciXGJTJp8CTu/PhONqVv8vx8xpjIZ8mhlO4+526uOe0aBkwaQMaR\njDIfZ/nO5VSLrUaHpIpp6XBmkzO5u9fdjPpwFD71Vcg5jTGRy5JDGTzR5wnOPOVMrvrPVWRlZ5Xp\nGDPTZnLZqZdVaCuH+867j2xfNs8uebbCzmmMiUyWHMpARHh50MvUS6jH8KnDyfHllPoYM9bO8LQJ\na2FiY2J56+q3eOrzp1i2Y1mFntsYE1ksOZRRbEwsb1/9NulZ6YyZMaZUFW57Du1h1e5VXNDyAg8j\nLFyrOq14tt+zDJ86nMPHDlf4+Y0xkcGSQzlUq1KND4Z+wLKdy3hwwYMlft3stNn0ad2HalWqeRhd\n0UZ0HUGnhp14YMEDITm/MSb8WfcZ5VSzWk1mDZ/F+RPOp35i/RI1S52RVv6xostDRBg3aBzdXunG\nwPYDPX0IzxiIzieIo5095xAkm9M303tCbx5LfoxR3UcVud+xnGM0fKYhP97+I41rNK7ACAtasGEB\noz4cxbIxy0hKTAppLMaY4LPnHMJA89rNmTNiDmPnj2XamqKHpPh88+e0q9cu5IkBoG+bvgztNJTf\nzvitPaRkjDmBJYcg6li/I9OHTefmaTfz2c+fFbqPFwP7lMdf+/6VtL1pvLHsjVCHYowJI5Ycguzs\npmfz7q/f5Zr/XsPSHUsLbA91fUN+8VXimTR4En+a9yc27N8Q6nCMMWHCkoMHLm5zMS8PepmBkwaS\ntjctb/36fev5JesXepzSI4TRFdSlURce7P0gIz8YSbYvO9ThGGPCgCUHj1xz+jWkJKdw6duXsi1j\nG+Ceih7YbiAxEn4f+5297iShSgJP/u/JUIdijAkD4XeViiK/PfO33NLjFi59+1L2H96f12VGOIqR\nGCZeNZEXv3qRr7Z+FepwjDEhZk1ZPaaq3DP3HpZsWcLKXSvZ+set1KxWM9RhFWnKyin8+ZM/8/2t\n31O9avVQh2OMKYfyNGW15FABfOrjpo9uYtfBXcwaPivU4ZzUqA9HkVAlgVcueyXUoRhjysGSQwRQ\nVY7kHCG+SnyoQzmpA0cO0O2Vbrw44MWwLQYzxpycPQQXAUQkIhIDQK1qtXjzqje5Zfot7MzcGepw\njDEh4GlyEJHxIrJTRFYUsX24iCwTkeUi8rmIdPUyHlNy57c8n5u638Rvpv/Gnp42phLy+s5hAtC/\nmO0bgAtUtSvwOPAvj+MxpfBI8iNsz9jOv761fxZjKhvP6xxEpBUwXVW7nGS/usAKVW1WyLaIr3OI\nVKv3rOb8Cefzv9H/o0P9ihnS1BgTHNFS53AzEP5NeSqZjvU78ljyY4z4YATHco6FOhxjTAUJi/Ec\nROQi4CbgvKL2SUlJyZtPTk4mOTnZ87iMM+asMcxIm8HY+WN59lIbf9qYcJWamkpqampQjhXyYiV/\nJfRUoL+qritiHytWCrF9h/dx7uvn8vuev+f2nreHOhxjTAmUp1gppHcOItIClxhGFJUYTHiol1CP\n2cNnc97482heuzlXdLgi1CEZYzzk6Z2DiLwLXAjUB3YCjwBxAKr6qoi8BlwNbPK/5Jiq9izkOHbn\nECa+3vo1A98ZyKzrZ3F207NDHY4xphj2hLSpUNPWTGPMjDF8ftPntK7bOtThGGOKELHFSiYyXdHh\nCjanb2bApAEsvnkx9RLqhTokY0yQ2Z2DKbM/zf0TX279krkj50ZM1yDGVCZWrGRCwqc+rnvvOmJj\nYpk0eFJYDmJkTGUWLQ/BmQgTIzG8efWbbE7fzIMLHgx1OMaYILLkYMolvko8H133EVN/nMor39j4\nD8ZEC6uQNuWWlJjE7OGz6T2hN81qNbMxIIyJAnbnYIKibb22fDD0A0Z/NJpvt30b6nCMMeVkycEE\nTa9mvfjXZf/iislXsPGXjaEOxxhTDlasZILq6tOuZlP6JgZOGsjnN31O3YS6oQ7JGFMGdudggu7O\nXndyadtLGfzfwRzJPhLqcABYv289B44cCHUYxkQMe87BeCLHl8OQ94YQXyWet69+G5EyNbUul/Ss\ndCb/MJkJSyewYf8Gsn3Z3NDtBv7wqz/Qpm6bCo/HmIpmzzmYsBMbE8vbV7/Nhv0buGPWHaTtTauQ\nsah96mP+hvkMnzqcls+3ZP5P83n4wofZds82lo5ZSrXYavT8d08G/2cwi35eZONjG1MEu3Mwntp9\ncDdj549lzvo5VI2tyiVtLuGSNpfQt03foPbJtGH/BiYuncgby96gXkI9RncfzfVdrqd+Yv0C+2Ye\nzeSNpW/wwpcvUKtaLe7qdRdDOg2hamzVoMVjTDiw7jNM2FNVVu1exbwN85i3YR6Lfl5Ex/od6de2\nH5e0uYRzmp9T6ovzwaMHef/H9xn//XhW7l7J9Z2vZ/QZo+neuHuJXu9THzPXzuT5L59n9Z7V3H72\n7dx65q0kJSaV5S0aE3YsOZiIcyT7CEu2LGHe+nnM3TCXNXvWcEHLC7ikzSX0a9uPjvU7FlpPoaos\n3ryYCUsn8P6P73Ne8/MY3X00l516GdWqVCtzPMt2LOP5L5/nw9UfMuT0IdzV6y5Oa3Baed6iMSFn\nycFEvL2H9rLgpwV5ySLHl8MlbS+hX5t+XNzmYo7mHOXNZW8ycdlEBGF099GM7DaSJjWbBDWOHZk7\nGPf1OF759hV6nNKDu351F/3a9gtJhbox5WXJwUQVVSVtX1peoli4cSEiwrWnX8vo7qPp1ayX5xfr\nrOws3lnxDs998RyZRzO5uPXF9G3Tl4taXUSjGo08PbcxwWLJwUS1YznH8KmvXMVGZaWqrNy9kk9+\n+oQFPy1g4caFNK/dnD6t+tC3TV8uaHkBdeLrVHhcxpSEJQdjKki2L5vvtn+Xlyy+2PIFpzc4nT6t\n+tCndR/Oa3EeiXGJZTp2ji+HzQc2k7Y3jbR9aaTtTWPP4T3Ui69Hg+oNqJ9Yv8CUlJBEXGxckN+l\niRaWHIwJkdyK9dxksWzHMs5qchZ9W/elZ8M+bPmyJx+8F0fjxnDvvXBqBx9bD2zNu/in7TueCH76\n5SeSEpJon9Se9vXc1LB6Q/Zn7WfPoT3sObSH3Yd2583vObSHfYf3UT2uesHkkXB8vnGNxrSo3YLm\ntZtTu1ptqz+pRCw5GBMmduzP4Ln3F/H+d5/wkywgJmk9nWr25siheNL2pkHd9dSJr83pjf0JIOn4\n33b12pX6rsOnPtKz0gskjcBksj1jO5sPbGZT+iYAlyhqNXdT7ebHl2u7dQlxCV58NCYELDkYE0JZ\nWfDxxzB5MsyeDeeeC9ddB1ddBcfi9rBw40JyNIdmie1ZPKMd/3y2Js2bw9ixMGAAxFRQPwWqSvqR\ndDanb85LFrnzuctbD2ylZrWaNK/VnFNqnkKVmCrESqz7GxNLrMQe/xs4X8RfESHHl0O2L5sczSHH\nl3PC32LX+3LwqY+mNZvStVFXujXuRpeGXagdX7tiPrAoYMnBmAp27BgsWOASwrRp0K2bSwi//jXU\nL/hQ9gmys2HKFPi//3PHue8+GDYM4sKg6sCnPnYf3M2m9E3sPLiTbF923oW6sIv4yf6qauGJJSY2\nL/Hk3xa4f4zEsCl9E8t2LmP5zuX8sOsH6ifWp1vjbnRt2JWujdzUrl47YmNiQ/3xlcrRo5CeXvj0\nyy9w8CAcOVL8lJVV/Pa9e8M0OYjIeGAQsEtVuxSxz/8DBgCHgBtV9ftC9rHkYEIuJwcWLXIJ4f33\noV07lxCuvRaalOFxC1WYN88libVr4e674ZZboEaN4MdekVRh/Xr3WSUkwDXXQJUgDQ7gUx8b9m9g\n+c7lLNuxjOW7lrN853J2ZO6gU4NOecmia6OuNK/VnKzsLA4dO1TyKfsQh48dzluOkRhqVK1R4ilO\na7Dy+xqsXl6dg+kJHEyPJyM9jvRfpEACOHoUatd2U506x+dzp+rVIT4eqlUregrcXqVqDlrlMBp7\nGF/MIXJiDtG74+lhmxzOBzKBNwtLDiIyELhDVQeKyK+AF1S1VyH7WXIwIaEKX37pEsJ//wuNGrmE\nMGQItG4dvPN88w08/TR88gnceiv84Q/QsGHwju+lnBxYscIlg9wpNhbOPx+2bYOtW+GBB2DkSKjq\nUfdVGUcyWLFrBct3Ls+btmZsJTEusdApoUpCkdtytyfEJaCqZB7NLHo6lsnO/Zls2uH+HjicSVyN\nTGLjD5Ith8kmC8VH1Zh44mMTSKiSQHxcPNWrJlC9qptPqJJAfJV4EuISjs9XcfU+uQnr0LETk9bh\n7MMnJLXDxw5zNOcoCXEJJ7yHNb9fE57JAUBEWgHTi0gOrwCfqup//MurgQtVdWe+/Sw5mJB46il4\n5RUYPRqGDoUOHbw937p18OyzLhkNHQo33+wSUu3a7o6iouonipOVBV9/Df/7n0sEixdD48YuGeRO\nrVpBbqOozz6DJ56ANWtcEdrNN7tfvF754guXaL/6Ctq3d/9mgVPLli55ldXhw7Bwoatfmj0bMjKg\nf39Xf3TJJVA33/hW2b5ssrKzOHzssPubffiE+dxtgfNZ2VkoSvW46u5Cn++if0Ii82+rFlutQEu0\nsK5zOElymA78XVUX+5fnA2NV9dt8+1lyMBVu9Wp3ofvmG3dBqUi7dsGLL8IHH7jy5/R0OHQIatYs\nWPxwsqlWLZdYatZ0f+Pjj1+4SyI93SWA3LuC77+Hjh2PJ4LevUt2l/Pll/DXv7rP89573R1S9epl\n/4wC+XwwY4ZLClu2wB//6C7WGza4pBQ47d4NbdsWTBodOhS8sOdat+54Mvjf/1wd04ABburWLTyS\ndmHKkxzCYZjQ/IEXmgVSUlLy5pOTk0lOTvYuopPIjcX+Ru9fnw8WLEghJQUmTAhNHI8/nsLjjx9f\nfuihFA4cgEcfTSErC66/PoX0dBff/v1w3nkp7NsHU6akcOQING/utq9bl8LRoxAXl0JGBhw5kkLV\nqlC/fgo1akBGhlvu3Nktr13rls88M4XFi2HlyhSaNIEbbkjh4Yfhk09SqFatbO9r2jQYMyaF116D\np55K4a674MCBsh/vyBG49loXZ8uWKdx3n4t3715o1y6Fdu1g8eIU6teHl15yr3vwQfc5XXRRCmvW\nwDPPuP0zMlJITIT4+BSSkmDYsBS2bIG33nKf35AhKdx4I3TpkkJCgjtOKP5fFPc3NTU1b7m818hQ\n3zm8AqSq6mT/shUrmbDw4ouujmHhwvD9VVhWx465ljCZmW7KyDg+H7hOBHr1gjPP9KauYOVK+Pvf\nYc4cuP12V89Sr4RDfOzf74r7XnzR/XL/05/gootKd0eUnyps337iXUaDBu7uoHv38h07VCK5WCmw\nQroX8LxVSJtQ27gRzj7bFR94XcdgIC0NnnwSPvwQfvtb12qrqGKqTZvg+edh4kS47DJXPNW1a4WG\nG1HCdphQEXkXWAx0EJHNInKTiNwqIrcCqOosYIOIrANeBX7nZTzGnIyqa056772WGCpK+/bw+uvw\n7beufqNjR5cgtm07vs+yZTBiBJxxhruTW7YM3nzTEoOX7CG4CLJwoauU7NevfK0tTNEmTICXXnKV\np1XCoUauEtq6FZ55Bt54wz1DsnEj/PAD3Hmnq8SubQ9Il1hYFysFgyUHeOEF97BU48awb5/7ktx0\nU+S0hY8E27e78ut589xfE1q7dsG4cdCiBVx/vXvQy5SOJYcolpPjmuXNnw8zZ7r2419/7b40U6fC\nwIFw222uOWEkVpiFC1UYPBg6d4bHHw91NMYEhyWHKHXwoPvFlJnpumuok29Mmf373a33uHGuX57b\nbnNPodaqFZp4I9mUKfDII64Nv/1CNdHCk+QgIiNU9W3//Hmq+nnAtjtU9aUyRVsGlTE57NjhWmN0\n6QKvvlp8U0JV+PRTlyTmz3ddO9x2m2t+VxqqsHevaz2SluYe/AmcF3GVhR07usra3Pk2bbzrFqEi\n7N3r7hhSh368AAAUZklEQVSmToVzzgl1NMYEj1fJ4XtVPSP/fGHLXqtsyWHlShg0CH7zG/jzn0tX\nXLRtm2v58a9/QbNmLklce63rAA1cAti3r+gEoOpaj+RO7dodn/f5XNvv1avdlDu/ebN7gjgwYeTO\nJyUVHavP57oiOHzYVbTnzudfPnQITj/dNS/1wsiRrifV557z5vjGhIolhygyf74rSnruORg+vOzH\nyc52dRTjxrkmghdc4C7iaWnuolxUAkhKKn3dxZEjrhfO/Elj9Wp3R9GiheuBMn8COHrUdeWQmOiS\nV0LCifOBywsWuPqVp54q+YNSJTFrFtxxh+s4LlhdORgTLiw5RIkJE+D++92TuRdeGLzjrl/vOiNr\n3dolgPr1K6byWhV27nRJKT6+4AW/WrWSx5GeDn/5y/FxEEaOLP97OHDAFSdNmAB9+5bvWMaEI6+S\nw2FgnX+xLbA+YHNbVS3bKOplEO3JQRUefhjeecf9krWHr4r2zTeuGW+tWu6uqGPHsh/rd79zdy+v\nvRa8+IwJJ151vHdaGeMxpXDkiHteYcMG9+u+QYNQRxTezjrLdcX8z3+65ru33QYPPni8TqWkFi50\nI7j98IM3cRoT6YrsPkNVNwZOuEF7egBJ/mVTTnv3uv7fjxxxg7xYYiiZ2FjXSduyZa5eo3Nn13lb\nSR065Cr7X365YPNgY4xTZHIQkZki0tk/fwrwAzAaeEtE7q6g+KLW+vVuIPpevVwdQ2l/+Rpo2tTV\nQbz4oruDuO4695TzyaSkuJ5Gr7jC8xCNiVjFdbzXSlVzb7pHA3NV9XLgV8BNnkcWxZYscUUid9/t\nKlejrUvoijZwoCseatPGdcT20kvuyfLCfP2167DtxRcrNkZjIk1xl6VjAfMXA7MBVDUD8HkZVDSb\nMgWuvBLGj4cxY0IdTfRITIS//Q1SU92dWK9e8N13J+5z9KgbovLZZ60Iz5iTKS45bBGR34vIYOAM\n4GMAEUkkPEaQiyiq7i7hj3+EuXPdACIm+Dp1cpXNv/ud+4zvvNM1WQU3ZkCLFjBsWGhjNCYSFJcc\nbgY6A6OAoaq637/+V8AErwOLNjNmuCeXlywpfbcWpnREYPRo96R5ZqZ7uvq551xR0iuvWAeFxpSE\ndbxXQR57DLKyXNGHqViffebuIH7/e9ds2JjKwpPnHERkOqBAYQdWVbW2HqWwdq1rtmoq3gUXuN5W\njTElV1zdQS9gC/Au8KV/XW6iiOyf8SGwZo3rw8cYYyJBcd1nVAEuAYYBXYCZwLuqurLiwsuLJaKL\nlVTd0IYbNwa30zhjjClOeYqVintCOltVZ6vqDbi7iHXAQhGx37+ltGOH62TOEoMxJlIU2yRVROKB\nQcB1QCvgBeAD78OKLmvXWmd6xpjIUlyF9FtAJ2AW8JiqrqiwqKLMmjWWHIwxkaW4O4fhwEHgTuBO\nObFxuKqqjVRcQmvWwKmnhjoKY4wpueLqHGJUtWYRU4kSg4j0F5HVIpImImML2V5fRD4WkaUi8oOI\n3FiO9xK27M7BGBNpPOvyTURigZeA/sDpwDARyT9GxB3A96raHUgG/uFvJRVVrM7BGBNpvOwPtCew\nzj8exDFgMnBlvn22A7l3IbWAvaqa7WFMFe7oUdi0yfUYaowxkcLLX+lNgc0By1tw/TIF+jfwiYhs\nA2oCQzyMJyQ2bIBmzVxTVmOMiRReJoeSPLX2ILBUVZNFpC0wT0S6+bsFP0FKSkrefHJyMsnJycGK\n01NWpGSMqSipqamkpqYG5ViedbwnIr2AFFXt719+APCp6lMB+8wC/qqqn/uXFwBjVfWbfMeK2Cek\nn37ajU727LOhjsQYU9l48oR0EHwDtBeRViJSFRgKTMu3z2rcQEKISCOgA7DBw5gqnDVjNcZEIs+S\ng79i+Q5gDrAK+I+q/igit4rIrf7d/gacJSLLgPnAfaq6z6uYQsGasRpjIpGN5+CxRo1cd9FNmoQ6\nEmNMZROuxUqV3i+/wKFDcMopoY7EGGNKx5KDh3LrG2xYSmNMpLHk4CFrxmqMiVSWHDxkldHGmEhl\nycFD1ozVGBOpLDl4yO4cjDGRypqyesTngxo1YNcu99cYYyqaNWUNQ5s3uzGjLTEYYyKRJQePWH2D\nMSaSWXLwiNU3GGMimSUHj9gzDsaYSGbJwSNWrGSMiWSWHDxixUrGmEhmTVk9cPiwa6mUmQmxsaGO\nxhhTWVlT1jCTlgZt2lhiMMZELksOHrD6BmNMpLPk4AGrbzDGRDpLDh6wZqzGmEhnycEDVqxkjIl0\nlhyCTNWKlYwxkc+SQ5Dt3u1aKdWvH+pIjDGm7Cw5BJndNRhjooElhyCz+gZjTDTwNDmISH8RWS0i\naSIytoh9kkXkexH5QURSvYynItidgzEmGniWHEQkFngJ6A+cDgwTkdPy7VMH+Cdwuap2Bq7xKp6K\nYs1YjTHRwMs7h57AOlXdqKrHgMnAlfn2uR54X1W3AKjqHg/jqRB252CMiQZeJoemwOaA5S3+dYHa\nA/VE5FMR+UZERnoYj+eOHYONG6Ft21BHYowx5VPFw2OXpBvVOKAH0BdIBJaIyBeqmpZ/x5SUlLz5\n5ORkkpOTgxNlEP30EzRpAvHxoY7EGFMZpaamkpqaGpRjedZlt4j0AlJUtb9/+QHAp6pPBewzFkhQ\n1RT/8mvAx6r6Xr5jRUSX3TNmwD//CbNnhzoSY4wJ3y67vwHai0grEakKDAWm5dvnI6C3iMSKSCLw\nK2CVhzF5ypqxGmOihWfFSqqaLSJ3AHOAWOB1Vf1RRG71b39VVVeLyMfAcsAH/FtVIzo5dO8e6iiM\nMab8bCS4IEpOhocegr59Qx2JMcaEb7FSpWPNWI0x0cKSQ5AcOAAZGa61kjHGRDpLDkGyZg20bw8x\n9okaY6KAXcqCxLrNMMZEE0sOQWLNWI0x0cSSQ5BYZbQxJppYcggSK1YyxkQTe84hCHw+qFkTtm+H\nWrVCHY0xxjj2nEOIbd3qkoIlBmNMtLDkEARW32CMiTaWHILA6huMMdHGkkMQ2J2DMSbaWHIIAnvG\nwRgTbSw5BIEVKxljoo01ZS2nrCyoUwcyM6GKl4OuGmNMKVlT1hBatw5atbLEYIyJLpYcyskqo40x\n0ciSQzlZfYMxJhpZcignu3MwxkQjSw7lZM1YjTHRyJJDOajanYMxJjpZciiHvXtdgmjQINSRGGNM\ncFlyKIfcIiUpUytiY4wJX54mBxHpLyKrRSRNRMYWs9/ZIpItIoO9jCfYrEjJGBOtPEsOIhILvAT0\nB04HhonIaUXs9xTwMRBRv8GtGasxJlp5eefQE1inqhtV9RgwGbiykP1+D7wH7PYwFk/YnYMxJlp5\nmRyaApsDlrf41+URkaa4hDHOvyo8O1AqgjVjNcZEKy97BCrJhf554H5VVRERiilWSklJyZtPTk4m\nOTm5vPGVS3Y2bNgA7duHNAxjjMmTmppKampqUI7lWa+sItILSFHV/v7lBwCfqj4VsM8GjieE+sAh\n4BZVnZbvWGHXK+v69dC3L2zcGOpIjDGmcOXpldXLO4dvgPYi0grYBgwFhgXuoKptcudFZAIwPX9i\nCFdW32CMiWaeJQdVzRaRO4A5QCzwuqr+KCK3+re/6tW5K4LVNxhjopmnoxCo6mxgdr51hSYFVR3t\nZSzBtnYtdOoU6iiMMcYb9oR0GVmxkjEmmllyKCMrVjLGRDMbQ7oMMjKgUSM3bnSMpVdjTJiyMaQr\nWFqae77BEoMxJlrZ5a0MrL7BGBPtLDmUgdU3GGOinSWHMrA7B2NMtLPkUAbWVbcxJtpZa6VSUoVa\ntWDzZqhTJ9TRGGNM0ay1UgXatg0SEy0xGGOimyWHUrIiJWNMZWDJoZSsMtoYUxlYcigla8ZqjKkM\nLDmUUk4OdOkS6iiMMcZb1lrJGGOilLVWMsYYE1SWHIwxxhRgycEYY0wBlhyMMcYUYMnBGGNMAZYc\njDHGFGDJwRhjTAGeJwcR6S8iq0UkTUTGFrJ9uIgsE5HlIvK5iHT1OiZjjDHF8zQ5iEgs8BLQHzgd\nGCYip+XbbQNwgap2BR4H/uVlTKGQmpoa6hDKJZLjj+TYweIPtUiPvzy8vnPoCaxT1Y2qegyYDFwZ\nuIOqLlHVdP/il0Azj2OqcJH+HyyS44/k2MHiD7VIj788vE4OTYHNActb/OuKcjMwy9OIjDHGnFQV\nj49f4g6RROQi4CbgPO/CMcYYUxKedrwnIr2AFFXt719+APCp6lP59usKTAX6q+q6Qo5jve4ZY0wZ\nlLXjPa/vHL4B2otIK2AbMBQYFriDiLTAJYYRhSUGKPubM8YYUzaeJgdVzRaRO4A5QCzwuqr+KCK3\n+re/CjwM1AXGiQjAMVXt6WVcxhhjihcR4zkYY4ypWGH9hPTJHqALNyLSXEQ+FZGVIvKDiPzBv76e\niMwTkbUiMldE6oQ61uKISKyIfC8i0/3LERO/iNQRkfdE5EcRWSUiv4qw+B/w//9ZISLviEi1cI1f\nRMaLyE4RWRGwrshY/e8tzf+d7heaqI8rIv6n/f93lonIVBGpHbAt7OMP2HaPiPhEpF7AulLFH7bJ\noYQP0IWbY8DdqtoJ6AXc7o/5fmCeqp4KLPAvh7M7gVUcb20WSfG/AMxS1dOArsBqIiR+f93cLUAP\nVe2CK4q9jvCNfwLu+xmo0FhF5HRcnePp/te8LCKhvv4UFv9coJOqdgPWAg9ARMWPiDQHLgF+DlhX\n6vhD/eaKc9IH6MKNqu5Q1aX++UzgR9xzHVcAb/h3ewO4KjQRnpyINAMGAq8BuQ0BIiJ+/6+881V1\nPLg6L/8DlhERP3AA9wMjUUSqAIm4hhxhGb+qLgL251tdVKxXAu+q6jFV3Qisw33HQ6aw+FV1nqr6\n/IuBD+VGRPx+zwL35VtX6vjDOTmU9gG6sOL/FXgG7j9YI1Xd6d+0E2gUorBK4jngT4AvYF2kxN8a\n2C0iE0TkOxH5t4hUJ0LiV9V9wD+ATbik8IuqziNC4vcrKtYmuO9wrkj4Pt/E8YdyIyJ+EbkS2KKq\ny/NtKnX84ZwcIramXERqAO8Dd6pqRuA2dS0AwvK9ichlwC5V/Z7jdw0nCOf4ca3vegAvq2oP4CD5\nimDCOX4RaQvcBbTCfZlriMiIwH3COf78ShBr2L4PEfkzcFRV3ylmt7CKX0QSgQeBRwJXF/OSYuMP\n5+SwFWgesNycEzNfWBKROFxieEtVP/Sv3ikijf3bTwF2hSq+kzgXuEJEfgLeBfqIyFtETvxbcL+a\nvvYvv4dLFjsiJP6zgMWquldVs3HP/5xD5MQPRf9fyf99buZfF3ZE5EZc0erwgNWREH9b3A+LZf7v\ncDPgWxFpRBniD+fkkPcAnYhUxVWmTAtxTMUS96DG68AqVX0+YNM0YJR/fhTwYf7XhgNVfVBVm6tq\na1xF6CeqOpLIiX8HsFlETvWvuhhYCUwnAuLHVZ73EpEE//+li3ENAyIlfij6/8o04DoRqSoirYH2\nwFchiK9YItIfV6x6papmBWwK+/hVdYWqNlLV1v7v8BZc44adlCV+VQ3bCRgArMFVnjwQ6nhKEG9v\nXFn9UuB7/9QfqAfMx7V+mAvUCXWsJXgvFwLT/PMREz/QDfgaWIb75V07wuK/D5fQVuAqdOPCNX7c\n3eU24CiufnB0cbHiijzW4ZLgpWEY/01AGq6VT+739+UIiP9I7uefb/sGoF5Z47eH4IwxxhQQzsVK\nxhhjQsSSgzHGmAIsORhjjCnAkoMxxpgCLDkYY4wpwJKDMcaYAiw5mEpDRLqJyIAyvK6JiEwpwX4z\nRaRW2aIrn5KcW0RG+Z9aNuak7DkHU2n4u0U4U1V/X8i2Kuq6rIhaIvIpcK+qfhvqWEz4szsHExL+\nblF+FJF/iRsYaY6IxPu3pYrImf75+v5+YhCRG0XkQ/8gMj+JyB0icq+/B9YlIlK3mPNVBR4Dhoob\nyGiIiKSIyFsi8j/gDRFpKSKfici3/umcgFhXBMQwVURmixvQ5qmAc2wUN9hNce/tbBFZ7o/h6SIG\nakn2xzHDPzDLOH93GojIMP/rV4jIkyU9t4hcg+u7aZL/84oXkSfFDSy0TESeLuc/qYkylhxMKLUD\nXlLVzsAvwK/964vrzbMTcDVwNvBX4IC6HliXADcUdSJVPQo8BExW1TNU9b/+TR2Bvqo6HNdJ3CWq\neiaub6n/V8ThugFDgC64ZJPb9XFgzEW9twnALap6BpBdzPs8G7gDNzhLW2CwiDQBngQuAroDZ4vr\novmk51bV93D9lV3v/7yqA1epau7ANo8XEYeppCw5mFD6SY/3O/8trkfJk/lUVQ+q6h7chW+6f/2K\nErxeOLELY8X1H3XEv1wVeE1ElgP/xV2YC7NAVTP8r1sFtCxknwLvTdxgRDVU9Uv/+nfyxRPoK3UD\nXflwfej0xv3yT1XXa2sOMAm4oCTnDtiWe750IEtEXheRq4HDRcRhKilLDiaUjgTM5+CGxQT3izr3\n/2Z8Ma/xBSz7cOM5FKewX+mHAubvBraralfchbhqEcfJH3dh5y3JPiXta18oPPai1hf1ueYd11+/\n0hPXrfllwMfFxGIqIUsOJpzkXiw34i7OANeU8rXFyQBqFrO9FrDDP38DJ15Uy03dkKUZIpI7PON1\nxeze019/EIMrwlqE62L5QhFJEjfG+nXAwhKcOvezycC9R8SNkFdHVWcDf8QVlRmTx5KDCaX8v3pz\nl58BbhOR74CkgPX56yLyzyuAiNwqIrcWcr5PgdNzK6QLOcbLwCgRWQp0ADILOVdJR2Ir6r3dDPxb\nRL7HjRGdXsRrvwZewhVbbVDVD9SNV3G//30sBb5R1ekBrznZuScCr/g/15rAdBFZhks8d5fgPZlK\nxJqyGlOBRKS6qh70z9+PG3P57nz7JAP3qOrlIQjRGODkZbTGmOAaJCIP4L57G4EbC9knYsaJNtHL\n7hyMMcYUYHUOxhhjCrDkYIwxpgBLDsYYYwqw5GCMMaYASw7GGGMKsORgjDGmgP8P0WAn+qLaCZ4A\nAAAASUVORK5CYII=\n",
       "text": [
        "<matplotlib.figure.Figure at 0x108a225d0>"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For $d=5$, it's converged, but to a better value than for $d=3$.\n",
      "\n",
      "Thus you can **bring the two curves closer together** by adding more points, but you can **bring the convergence level down** only by adding complexity to the model.\n",
      "\n",
      "This can be very useful, because it gives you a hint about how to improve upon results which are sub-par.  To make this more concrete, imagine an astronomy project in which the results are not robust enough.  You must think about whether to spend your valuable telescope time observing *more objects* to get a larger training set, or *more attributes of each object* in order to improve the model.  The answer to this question is important, and can be addressed using these metrics."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Final Exercise"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's return to our digits data.  Using what you've learned here, explore several classifiers and find the optimal classifier for this dataset.  What is the best **accuracy** you can attain on a 5-fold cross-validation over the data?  Are there indications that more training data would be helpful in improving this accuracy?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 29
    }
   ],
   "metadata": {}
  }
 ]
}