{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Model selection\n",
    "\n",
    "Up to now, we have seen many ways to use models from the `sklearn` library to help solve your machine learning problems. Once you know how to use these models, an important question is that of model selection.\n",
    "\n",
    "In this section of the tutorial, we will deal with two important questions related to model selection:\n",
    "1. How to select a model that achieves a reasonable trade-off between underfitting (aka bias) and overfitting (aka variance);\n",
    "2. How to perform parameter tuning with `sklearn`.\n",
    "\n",
    "## Underfitting _vs_ overfitting\n",
    "\n",
    "Let us consider the problem of explaining variable `y` using covariate `X` in the following case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x10e1f5710>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH3pJREFUeJzt3Xt8VOW97/HPbxIFSXtqLbtyMeo5XlBpvFUBFTRtVaQ5\nyKVCROIF260Wa3d7jm3VpgWqvuruZWtV6oZubbWgIMol2ZFGeslWvIC4pUWtl16sEBG3Wrs1CCUz\nv/3HWkMmyUwcyGQma+b7fr3mRbJmzaxnkuGbZ571e55l7o6IiERPrNANEBGRvaMAFxGJKAW4iEhE\nKcBFRCJKAS4iElEKcBGRiMoqwM2s0sx+Y2bPmdmzZvblcPsBZrbGzF4ys4fNbP++ba6IiCRZNnXg\nZjYEGOLuG83sQ8DTwGRgFvCmu3/PzL4BfNTdr+nTFouICJBlD9zdX3f3jeHX7wG/B4YD5wJ3h7vd\nTRDqIiKSB3s8Bm5mhwInAOuAA919W3jXNuDAnLVMRER6tEcBHg6fPAj8k7u/m3qfB2MxmpcvIpIn\n5dnuaGb7EIT3z919Zbh5m5kNcffXzWwo8EaaxynURUT2grtbT/dnW4ViwJ3A8+5+S8pdDcDF4dcX\nAyu7PjZsRNHe5syZU/A26PXpten1Fd8tG9n2wE8D6oDfmdkz4bZrgZuA+83s88ArwPQsn09ERHop\nqwB397Vk7q2fmbvmiIhItjQTs5eqq6sL3YQ+Vcyvr5hfG+j1lYKsJvL06gBm3tfHEBEpNmaG5+Ik\npoiI9D8KcBGRiFKAi4hElAJcRCSiFOAiIhGlABcRiSgFuIhIRCnARUQiSgEuIhJRCnARkYhSgIuI\nRJQCXEQkohTgIiIRpQAXEYkoBbiISEQpwEVEIkoBLiISUQpwEZGIUoCLiESUAlxEJKIU4CIiEaUA\nFxGJKAW4iEhEKcBFRCJKAS4iElEKcBGRiFKAi4hElAJcRCSiygvdgL2VSCRoampi0aLlANTVTaWm\npoZYTH+TRKQ0mLv37QHMPNfHSCQSnHfehTz88HO0tV0GQEXFAsaPr2LZsnsU4iISeWaGu1tP+0Qy\n6ZqamsLwfhKYDcymrW0dzc2baGpqKnTzRETyIpIBvmjR8rDnPTBl60Da2i5j8eIVhWqWiEheRTLA\nRUQkogFeVzeViooFwI6UrTuoqFjIzJlTCtUsEZG8yirAzewuM9tmZptSts01sy1m9kx4O6fvmtlZ\nTU0N48dXUVExGpgPzKeiYjTjx1dRU1OTr2aIiBRUVlUoZjYOeA+4x92rwm1zgHfd/V8+4LE5r0KB\njjLC5Jj3zJlTVEYoIkUjmyqUrMsIzexQoLFLgL/n7j/8gMf1SYCLiBSzfJQRXmVmvzWzO81s/14+\nl4iI7IHezMS8A/hO+PX1wA+Bz6fbce7cubu/rq6uprq6uheHFREpPi0tLbS0tOzRY/Z6CGUP7tMQ\niojIHurTIRQzG5ry7RRgU6Z9RUQk97IaQjGz+4AzgMFmthmYA1Sb2fGAA38GLu+zVoqISDeRXMxK\nRKTYFe1iViIiogAXEYksBbiISEQpwEVEIkoBLiISUQpwEZGIUoCLiESUAlxEJKIU4CIiEaUAFxGJ\nKAW4iEhEKcBFRCJKAS4iElEKcBGRiFKAi4hElAJcRCSiFOAiIhGlABcRiSgFuIhIRCnARUQiSgEu\nIhJRCnARkYhSgIuIRJQCXEQkohTgIiIRpQAXEYkoBbiISEQpwEVEIkoBLiISUQpwEZGIUoCLiESU\nAlxEJKIU4CIiEaUAFxGJKAW4iEhEZRXgZnaXmW0zs00p2w4wszVm9pKZPWxm+/ddM0VEpKtse+A/\nBc7psu0aYI27Hwn8KvxeRETyxNw9ux3NDgUa3b0q/P4F4Ax332ZmQ4AWdz8qzeM822NESSKRoKmp\niUWLlgNQVzeVmpoaYjGNSolI75kZ7m497tOLAP+ru380/NqAt5Pfd3lc0QV4IpHgvPMu5OGHn6Ot\n7TIAKioWMH58FcuW3aMQF5FeyybAc5I0YUIXV0r3oKmpKQzvJ4HZwGza2tbR3LyJpqamQjdPREpE\neS8eu83Mhrj762Y2FHgj045z587d/XV1dTXV1dW9OGzhLVq0POx5D0zZOpC2tstYvHgFEydOLFTT\nRCSiWlpaaGlp2aPH9CbAG4CLgX8O/12ZacfUABcRke66dm7nzZv3gY/JtozwPuBxYISZbTazWcBN\nwFlm9hLw6fD7klBXN5WKigXAjpStO6ioWMjMmVMK1SwRKTFZn8Tc6wMU6UnMadMuorl5U8pJzIU6\niSkiOZPTKpReNKLoAhw6yggXL14BwMyZU1RGKCI5owAXEYmovJURiohI/inARUQiSgEuIhJRCnAR\nkYhSgIuIRJQCXEQkohTgIiIR1Zu1UERE+r1iXrtfE3lEpGhFee1+TeQRkZLWee3+K4BK2tqOY9Wq\nXzNnzhwSiUShm9grCvA8SyQSNDY2Uls7i9raWTQ2Nkb+TSTSX3Ws3b8vcCHwLWAM8Xg9N920jGnT\nLor0/z+NgedRuo9zy5d/jZNOuoVHH22mvFy/DpG+0QQ8BzxJ8kIs7e2X0tw8mqampshehEU98DxK\ndym29vaNPPnkq5x++pmR7gmI9Ecda/cvAzJfRSuqFOB5lOlSbPAV1q9/SdfTFMmxmpoaxo+voqzs\nl4VuSp9QgPcT8fhhke4JiPRHsViMZcvu4dprP095+a0U21W0FOB5VFc3lbKy7m8iWAiMLEyjRIpc\nLBZj3rx5nHvuSVRUjAbmA/OpqBjN+PFV1NTUFLqJe0114HmUSCQ47bSzePLJV4GvhFsXAscwaNBz\nLFlyY2RPpoj0d1G7ipauyNMPtbe3c/rpZ7J+/UvE44cBIxk06HHOOefYfj+xQETyRwHeT0WtJyAi\n+acAFxGJqGwCXDNH+oFiXmxHRPqOeuAFFuXFdkSk72gxqwhINzuzrW0dDz20kcbGxkI3T0T6MQV4\ngWWanbljxxe58sprNb1eRDJSgPdjr7/+jqbXi0hGCvAC62l2Zjx+rqbXi0hGCvACq6mpYcgQA04k\nOcUXRgNVaHq9iPREAV5gsViM+fNvYsCA94H1wNPADcBCKir+LdIL7Yjkgi6CkpnKCPuBRCLBtGkX\n0dy8KaWUcKFKCaXklXKZrWZiRki66fUTJkxg9erVuyf4XHDBZAAWL17B1q1bgDKGDh3KhRd+ThN/\npCg1NjYyY8a3wjLbZKXWDioqRnPffTcU9eJvCvAIS9fzKCv7Ee7/TSJxCPAucCVQOj0SKT21tbO4\n//6TCeZIpJpPbe3TLFlyVyGalReaSh9hnSf4BD2PePxS4JPANuD3u7e3tUX/2n4isufUXeunMl9+\nbTZQ2W171K/tJ5JOxzUti+tKOrmiABeRfit5Tctiu5JOrvR6DNzMXgH+G4gDu9x9VJf7NQa+F4KT\nN/W0ta0j9eRNMISyndQhlFI5qSPFJdtVOEt1/fy8nMQ0sz8Dn3T3tzPcrwDfC+lKC8vKbsX9byQS\nhxL8zUyexFTJoURLKZcHZiufAX6Su7+V4X4F+F7q2vOYMWMSAPfeu5LXXtsMlDNs2FCtHy6R01Ee\n+DjwK2A5EGfAgEdZuvQWJk6cWPJr5OcrwP8E/I1gCGWBu/+ky/0KcBHpJCgP/CTwBPAccFl4z+0M\nH57goIOGsmHDVuLxLwOl2TvPVxnhae6+1cz+AVhjZi+4+6M5eF4RKWrPhbfUSTqX0tr6CVpbXwT+\nhEple9brAHf3reG//2VmK4BRQKcAnzt37u6vq6urqa6u7u1hRSTC6uqm8uCDlxOP19O9VParwNJu\n25OlssUa4C0tLbS0tOzRY3o1hGJmg4Ayd3/XzCqAh4F57v5wyj4aQhGRThKJBAcfPJLW1qtIN8sy\nCPBHum0v9tmXqfJxSbUDgUfNbCOwDvj31PAWEUknuQrnwIE/pvta+LcBm7ttLy+/TZN3utBaKEVK\nV7qX/i5dqezAgXfw97+/SSJRTTDXIXly8xbGjDmYxx5bUzLvYS1mVaKSNbbNzc+yffupwLOUlf2R\nUaOO5JFHfkl5uZbAkf4hXans3XffT3PzpvC9+1zJvncV4CWqsbGR88+vZ/v2o4EX6OjF3FxyvRiJ\nnlKdedmVArxEBTW2AwlqbFNLtLYTix3JqaeOZNiwYRpWEenHtJxsSXuWoOedDO8E8I8kEh9h7dpg\nRmdTUz3jxy8tqckRIsVE/2uLUHCl+z922dpEMGniaYKyrdm0ta2juXkTTU1NeW+jiPSeArwI1dTU\nMGrUkcDNdJRiLadzjxx6WkdcF5IV6f80hFKEYrEYjzzyS8aNG8+GDcfT3n4V8Afg5Kwen26lOA23\nSE9Utlog7t6nt+AQUgjxeNwbGhq8svIYh2McRjq87+Dh7X0vLx/hDQ0NnR7X0NDgFRXHddu3ouLY\nbvtKaYvH475y5UofPnyEl5WNcJjvMN8rKo71qVNnejweL3QTIyvMzh7zVVUoJSAoK/wm27ePBJ6n\n6+SIRx9tZvXq1bt7T6+9tpm1aycDX+ryTKU1lVl61t7ezrhxn2H9+udJJMqB24EpBCOzushIb6kK\nRYBgTPycc5byi1/8LpwcsWT35IiWltVMn35xp+GS8vIngMUEJzv1EVi6a29v5xOfOJkXX2wD5oVb\nvwOsAO6hFBaf6g/0v7MExGIxli27hyVLbqS29u/U1h7BihULWLv21zQ3N4fh/STJ6pT29o3AmwT/\nGZO2M2DAD2htbdVJzRKXSCQYO/YzvPjie8DvSL5vguWQNhFUPEk+aAilxAWTfk4m3YpwZWU3Eo9/\nE0hQVnYjsH9JL7AvgVWrVjF58qXA9aRfSfBp4McaQumlfKxGKEVs//0HMH36BsaOXcU++wwmHt+I\nasjlBz+4Ddi3hz1eZuDAk3Tl+DxQgJe4urqpGZb0XMh77wX3DxtWyY4ds0lXQ37llddoOKXEvPLK\nVmAysIDu75ubOeCAV1iy5EZ9OssD/XRLXE1NDR/7WBw4keDj73xgNFDFzp3/L+0kn1SbN3+MGTPq\nmTbtIoV4iTj00IOAkUAVwXsl+b45kQ9/eDvbtv2ZSZMmKbzzQD/hEheLxTj11NHAGQRjl08DNxBU\nEgRvjwsumExZ2Y/o3tu6FRhHW9txrFr1a+bMmaMQLwFXXz2bsrLbgIUE75WngfXEYn/j7rvnl9SS\nr4Wmk5hCY2MjM2bU09a2jo5hko463kQiwec+dwXx+MfpqCFfCLwKDAOuBKC8/FZOOmk4lZWVmJlm\n4xWp5Ezd1at/y44dXwSCCzFMmHAcDzzwc/2+c0TLyUpW0l0ZpaJi4e4qkxkzPs/9938SOIRgTZXN\nwDvAFoKPzh2TN4KP1WcCVapUKWJas7vvKcAlaz39h+woNbwCuJBgVcNkT3wBQWgnh1ySZWR3odl4\nIntPAS450djYyJQpXyMev5Gg9jf1IhE7CE5k3QBMpHOAg6bf939aiKp/UoBLTiQSCQ4+eCStrX8F\nvk1Pkzc6wryGYEbe96isfJv5829SKPRDwfmNOlavfoKdOw8CYMCALXz2s6dqPLvAFOCSM6tWrWLq\n1C+QSMwjfYAvAV4HTgJ+DlxM6lBLpvFw9f7yq+vP+4gjhvPd7/4bicQQUofFysre4MEH/5VJkyYV\nrK2lTgEuOROsf/FpnniilWC9i44hlPLy4xkz5iDi8XZ++9u3wwWzHiXolXevakmOh6dbd1wnPnOn\na1hfcMFkfvazpaxZ8/zun7fZDbhX0PV3CicyduwwHn30l4VoupBdgGs9cMnarl27fMyYT3t5+QiH\n2x1u77Tuc+f1x+enrCWevN3u06dfsvv5tO5434nH4z5lygXhzzdYo3vgwJFeVjbEoS3l531hxt9V\nZeXIQr+MkkYW64GriyNZKy8v57HH1rB8+feprX2a2tqnue++G3b3lmOxGBMnTuSUU0ZlfI7HHlu3\ne7LPokXLw55gdpd5k+w1NTV1W2Vyx44NxOMfBX6VsmdZxuc45JCD+riV0lsKcNkjyZBesuQuliy5\ni4kTJ3Yb6uhpfZU333x/jxbA0rU5906mP47BpKsVQAJoJKjpv4Wuv6uyslu5+uov5qu5spc0Bi45\n11G1YiRnaQYzN4O1M8aObeI//uMhmpqa0swA3c6AASM5+eQjGTp0KJs3v8qmTW9rjHwP9bRMcLBu\nd5zgJPMXCC7e8SbwFQBisR8xadLJqkIpMJ3ElIKZPv0Sli3bD9gZbplCUFo4n1hsDkOHHsgpp4yi\ntfUv4YnPy+m87viXCMoQ/xMYD0wLH/93TQ7KQqblEcrKjsPsHdrbB9NxkjlB0Cu/ilisneuuu5x5\n8+YpvAtMAS4Fkz5AtgP/BxhMsmdYUbGAqqrBHHzwwbz22mY2bHidHTvWA/9IEDBfDh+bOuPzDk0O\n+gCZlkc4++xP8MYb/8Vjj00mXe98+PD5vPrqswrvfkDXxJSCqampYfz4pTQ0HE97+1Xh1h8A+wMb\nSIZ6W9ulbNx4EuXlxiuvbGXHjtOBNQQf7zfSEf6XEkwSSj9+Xqz15Hv7upKX0eu8PMIN1NTUMGPG\n5zM+7rTTRkf+Z1ZK1AOXPpNIJDjjjLNZu/bvwOFAKzCJ9OOyS4HzCa5s7sBVGfZbT0XFxqzqyc8+\n+xNcfPF07r13JRC9UM/2dV1wwWSArF/nB60+qaGp/kF14FJwQa33sWGt9yUZa45h1u46cBjaw34f\n9zFjPu3xeLzLMbrXk5eVHekDBhy6uw46tWY9nWQd+/Tpl/j06Zd4Q0NDxn37UrIdY8ee7WVlwxwe\ncIhnfF1lZUeG9d0/crjCzQ70/fY70K+77jrftWtX2uefOnVm+HvpXs8v/QNZ1IErwKVPdQ6Lyx2O\n7ha0cKxDQ8q2yx2OSLPf4Q4n+LRpF3c6xvTpPf1huCirSULxeNwnT57hAweO3B2MsdgRPmbMp9KG\nYLrH5yL8003ACX4+M1NCvPvrgiqHTzmkPu5wHzbsyIwh3tDQ4LW1s7y2dlbB/lhJZgpw6RdSw234\n8KPCkLw9vB3dJZzc4VY3+0jYEx8b3g5yGOUw3Pfdd7Cfd95Fu0On5wCf1W1bbe2sbm1cuXKll5WN\nSPNH4zAfM+b0HsMtGbqDBh3rcIXDWDc70EeMONZ37ty5Rz+rTJ8mOv+RS76ueLjtkvDneGTaP3r1\n9fV7/DuTwssmwDUGLnmVuu54a+trPPXUi+zc+RwwKNxjB4MGjaKysowXX3wP+Gq4fT6wC9gPuBzo\nqAmvq/scM2bUs3Pnd4B/D/f/v0A98D2CZW7Z/TzTp2+grm4qixYtx90ZMaKSO+5YzFtvXU26cfdY\nbB4rV96ZcWy4sbGR88+vZ/v2o4EX6FgU6mYOOGAn27b9KevLjHWUXyYn1kwlKJ+8g44VH08Evgvc\nT8eCYfcBM9K2v7LyDl599dmsji/9h8bApV/LNBZ7yilnZOiFHh6OB3ceEjniiGMc/ld4f8fwAezv\n8G6n/QcNqvIxYz7VZYjicIfKHnrxQzqt4ZLa/oaGBj/ooGPCnne6Nh+WdQ84Ho/78OEjHI7x7sMn\ntzqM9YEDjwnHu5d1OV7mTyFa0ySayMcQCnAOQbfjZeAbae7Py4uVaEo3Fjtt2sV7NCQCHwlDr3t4\n7rPPwbv/OAwaVOUjRozMMFQy3OGoNNtHOAz14cOP6jSMEizsdbrHYkMdBjsc3OsAbWhoCIeX0g2f\nDPVY7KM+ZszpPnnyDC8r63qityHcT0MoxSKbAO9VPZWZlRHUfZ0DHAPMMLOje/OcUlrSra1i1vOn\nxu6SF1buuu7HV4nH32f48PlMm/YUxx47mD/84a/E419Os+83gK0Ek4Xmh7cTCa79eRhbt77DGWdM\noLGxkfb2dsaNO5snn3yNROIQ4KPAwRlb9/77bT2u35Jc72X27GvYsWN2mrZdBuxPIrGFTZve4ZJL\najnllKouz1ITtv3ElPZXMWxYjDlz5mQ8tkRbbwtiRwF/cPdX3H0Xwar+WgFeeqWubioVFQvovhjW\nLQRBlbptPlCZ8bkSiQm8884+jBhRyaZNbxOPfybDnjGCsHyDYK2QpwnGmc8AtpFIfIu1aycxY0Y9\n48aNZ/36VuAm4F3gd8DXCWaLdm/zW2+9w9ixn6a9vT1N+4Ja7xkzvsWWLQdkfB3BJKZBtLVdxn33\nreLrX/9Sl59RDFjIPvu0MXjwD6isvIP6+vP5y1+ey3r8XSLog7roPd2A84CfpHxfB9zWZZ88fNiQ\nYpJubBwOczggHCpJbhvhMMZhZYbhg5Hh0MLtKWuUZxpqONaD8fWjvaPaoyHtuHawHvoVXcad4+FY\ndWqbDw/bd5vD4d3q1927Vp301LaOCpTa2lmq5S4BZDGE0ts/zZ7NTnPnzt39dXV1NdXV1b08rBSz\nrtPAg/fyJ3nggWdIJL5Dx3T6KcAy4Kzw39F0VID8CDiZjgqOpBqCWZ9VJFff61gpcQrBZeFWEFSu\nLA+fr/OQRrA0wAKCFf1OTraaYJ2W4DqgQXXIT8LnjAFfYMOG42lqaupUzdJ52ddk21JfR7JtNQSz\nJRcyc+YNPU6Vj8pMU+mspaWFlpaWPXvQByV8TzdgDPCLlO+vpcuJTNQDlxxI1+McNKjKhw07Mtx2\nqwcTgD7ucKjDirBXHFSq1NfXp8wIjTsc7zDOg5OiDZ5+kkxP9eUfD3vh6SYmHRHe1/1xXWvQu9ew\nJ2u7x/p++w0JZ1zeqh52CSIPPfANwBFmdijwGlBLUIwqklPpe5w3MmHCBFavXr27p75589Hh8rSt\nwB1UVCxk/Pgq5syZw/PP/5nm5tFhj/dk4NfAw3S+FuTNdIytDwi/v7TLPrcQ1GNPAS6ie4+5DBiZ\n9nUE/y871NVNpampnra25DFiwFlUVNSzePG/EovF1MOWjHo9kcfMJhC8o8uAO939u13u994eQyRb\nqROFAGbOnLI79FLv27Kllccf34D7EDomv/yYoB8yBPgH4DDgVYKLHSQvAnwL7oOBxwjCNkEwbPIl\n4BDga+G2bxOcDE0N/irq68/n+uuv79TedMu+6qIVovXARTJIVn889NDj7NwZXPtxwIAtnHDCwTz1\n1MvE4/UEwZ4M6BXAywwe/BfefPPrBIGdahwdMyETBD3zTXTumQ9i+vSjWLr0p93akumPjpQuBbhI\nD9IF54QJEzj99DN54olWggDuvNzqV796Ljff3NBtKdZ99hnBrl37pjym4yo3cCxBnfor1NY+owtR\nSFYU4CJ7IZioM54NG1p3X4wiOayxdOnPqK29pNuQx1lnjWTdug1s3ep0r265B10KTvaUAlxkL2U7\nlp56XyKRYN68edx993LefPMt2tsHsGvX1UBM49qyxxTgIgWicW3pLQW4iEhEZRPg6g6IiESUAlxE\nJKIU4CIiEaUAFxGJKAW4iEhEKcBFRCJKAS4iElEKcBGRiFKAi4hElAJcRCSiFOAiIhGlABcRiSgF\nuIhIRCnARUQiSgEuIhJRCnARkYhSgIuIRJQCXEQkohTgIiIRpQAXEYkoBbiISEQpwEVEIkoBLiIS\nUQpwEZGIUoCLiESUAlxEJKIU4CIiEaUAFxGJKAW4iEhEKcBFRCJqrwPczOaa2RYzeya8nZPLhomI\nSM960wN34F/c/YTw9otcNSpKWlpaCt2EPlXMr6+YXxvo9ZWC3g6hWE5aEWHF/iYq5tdXzK8N9PpK\nQW8D/Coz+62Z3Wlm++ekRSIikpUeA9zM1pjZpjS3c4E7gP8NHA9sBX6Yh/aKiEjI3L33T2J2KNDo\n7lVp7uv9AURESpC79zhMXb63T2xmQ919a/jtFGDT3jRARET2zl4HOPDPZnY8QTXKn4HLc9MkERHJ\nRk6GUEREJP/yMhPTzK4Pq1U2mtmvzKwyH8fNBzP7vpn9Pnx9y83sI4VuUy6Z2TQze87M4mZ2YqHb\nkytmdo6ZvWBmL5vZNwrdnlwys7vMbJuZpR3WjDozqzSz34Tvy2fN7MuFblOumNlAM1sXZuXzZvbd\nHvfPRw/czD7s7u+GX18FHOfuX+jzA+eBmZ0F/MrdE2Z2E4C7X1PgZuWMmR0FJIAFwP939/8scJN6\nzczKgBeBM4FW4Clghrv/vqANyxEzGwe8B9yTrrAg6sxsCDDE3Tea2YeAp4HJRfT7G+Tu282sHFgL\nXO3ua9Ptm5ceeDK8Qx8C3szHcfPB3de4eyL8dh1wUCHbk2vu/oK7v1ToduTYKOAP7v6Ku+8ClgCT\nCtymnHH3R4G/FrodfcXdX3f3jeHX7wG/B4YVtlW54+7bwy/3BcqAtzPtm7fFrMzsRjN7FbgYuClf\nx82zS4GHCt0I+UDDgc0p328Jt0nEhCXMJxB0noqCmcXMbCOwDfiNuz+fad/eVKF0PegaYEiau65z\n90Z3/ybwTTO7BrgZmJWrY/e1D3pt4T7fBP7u7vfmtXE5kM3rKzI6c18EwuGTB4B/CnviRSH8RH98\neD6t2cyq3b0l3b45C3B3PyvLXe8lYr3UD3ptZnYJ8FngM3lpUI7twe+uWLQCqSfSKwl64RIRZrYP\n8CCwyN1XFro9fcHd/2ZmTcBJQEu6ffJVhXJEyreTgGfycdx8CJfR/Rowyd13FLo9faxYJmVtAI4w\ns0PNbF+gFmgocJskS2ZmwJ3A8+5+S6Hbk0tmNji5rpSZ7QecRQ95ma8qlAeAEUAc+CPwRXd/o88P\nnAdm9jLByYbkiYYn3H12AZuUU2Y2BbgVGAz8DXjG3ScUtlW9Z2YTgFsIThLd6e49lmtFiZndB5wB\nfAx4A/i2u/+0sK3KHTMbCzwC/I6O4bBri2FJazOrAu4m6FzHgJ+7+/cz7q+JPCIi0aRLqomIRJQC\nXEQkohTgIiIRpQAXEYkoBbiISEQpwEVEIkoBLiISUQpwEZGI+h9hVdPs1A0cEQAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ba531d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "np.random.seed(0)\n",
    "X = np.random.randn(100, 1)\n",
    "y = 2 * X ** 2 - X + .3 * np.random.randn(100, 1)\n",
    "plt.scatter(X, y, s=40)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, if we want to fit a linear regression model to this data, we do not get great results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAD7CAYAAABOi672AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8lOWZ//HPPRMlMOIRK6cgVgUCom5rBSpqVqvR5hcR\nKkQED9huW2V11/391vVAi662Hrr760HRlW21RZCTBZp0tAHbZusBz1oRInioCnioB7QaSEjmufaP\nZyaZTCYhCUlmnpnv+/V6XswpM/dD4Jr7ue/rvm5nZoiISPYLZboBIiLSOQrYIiIBoYAtIhIQCtgi\nIgGhgC0iEhAK2CIiAVHQW2/snFO+oIhIN5iZS/d4r/awzQwzY/78+c23c/nIh/PMh3PMl/PMh3MM\n4nl2pNd62H3F8zyi0SiLF68CYPbsaZSVlREKabRHRHJLoAO253mcd96FrF27kbq6bwMQjc6jtHQ5\nK1cuUtAWkZzSJxGtpKSkV943Go3Gg/WTwOXA5dTVPUV19Qai0WivfGZHeus8s0k+nCPkx3nmwzlC\nbp2n29OYSbff2DnrrfdOqKiYw4oVX8EP1skWUFHxHMuW3durny8i0tOcc1gmJh1FRKTnBDpgz549\njUjkHqA+6dF6IpGFzJo1NVPNEhHpFYEeEvE8j+nTL6K6ekPzpGMkspDS0vGadBSRQOpoSCTQARta\n0vqWLFkNwKxZU5XWJyKBldMBW0Qkl3R70tE5d69z7n3n3Iakx25wzm1zzr0QP87q6QaLiEhbexo3\nuA9IDcgG/H8z+7v48bveaZqIiCTrMGCb2aPAjjRPpe2ui4hI7+nuzNwVzrk/O+d+4Zw7sEdbJCIi\naXWnlsjdwL/Hb98E/CfwzXQvvOGGG5pvl5SU5NQSURGRnlBTU0NNTU2nXrvHLBHn3EigyszGd/E5\nZYmIiHRRjy5Nd84NSbo7FdjQ3mtFRKTndDgk4pxbCpwKDHLObQXmAyXOuePxs0X+Anyn11spIiJa\nOCMikk1UrU9EJAcoYIuIBIQCtohIQChgi4gEhAK2iEhAKGCLiASEAraISEAoYIuIBIQCtohIQChg\ni4gEhAK2iEhAKGCLiASEAraISEAoYIuIBIQCtohIQChgi4gEhAK2iEhAKGCLiASEAraISEAoYIuI\nBIQCtohIQChgi4gEhAK2iEhAKGCLiASEAraISEAoYIuIBIQCtohIQChgi4gEhAK2iEhAKGCLiASE\nAraISEAoYIuIBIQCtohIQChgi4gEhAK2iEhAKGCLiASEAraISEAoYIuIBIQCtohIQChgi4gEhAK2\niEhAKGCLiASEAraISEAoYIuIBIQCtohIQChgi4gEhAK2iEhAKGCLiASEAraISEAoYIuIBIQCtohI\nQChgi4gEhAK2iEhAKGCLiASEAraISEAoYIuIBESHAds5d69z7n3n3Iakxw52zq1zzm1xzq11zh3Y\n+80UEZE99bDvA85KeewaYJ2ZjQJ+H78vIiK9zJlZxy9wbiRQZWbj4/dfAU41s/edc4OBGjMbk+bn\nbE/vnSs8zyMajbJ48SoAZs+eRllZGaGQRpxEpGucc5iZS/tcNwL2DjM7KH7bAR8n7qf8XF4EbM/z\nOO+8C1m7diN1dd8GIBK5h9LS8axcuUhBW0S6pKOAXbA3b2xm5pxrNyrfcMMNzbdLSkooKSnZm4/L\nStFoNB6snwQKAairu5Tq6glEo1HKy8sz20ARyWo1NTXU1NR06rXdHRIpMbP3nHNDgD/m85BIRcUc\nVqz4CnB5yjMLqKh4jmXL7s1Es0QkoDrqYXfner0SuDh++2JgTXcbJiIinbentL6lwBPAaOfcVufc\nHOBW4Azn3BbgtPj9vDV79jQikXuA+qRH64lEFjJr1tRMNUtEctAeh0S6/cZ5MiTieR7Tp19EdfWG\npEnHhZp0FJFu2asskb340LwI2NCS1rdkyWoAZs2aqrQ+EekWBWwRkYDo6UlHERHJAAVsEZGAUMAW\nEQkIBWwRkYBQwBYRCQgFbBGRgFDAFhEJCAVsEZGAUMAWEQkIBWwRkYBQwBYRCQgFbBGRgFDAFhEJ\nCAVsEZGA2KtNeEVEckWirv3ixasAfzepbKtrr3rYIpL3PM/jvPMuZO3ajUk7R92TkZ2jVA9bRKQD\n0Wg0HqyfBC4HLqeu7imqqzcQjUYz3bxmCtgikvcWL14V71kXAh5QBVxGXd3+3H77T/E8L7MNjFPA\nFhFp5gEXAt8DvgLM5MkntzF9+kVZEbQVsDPI8zyqqqqoqJhDRcUcqqqqsuIfhUi+mT17GpHIPcBq\nYCPQMjTS1PRi1gyNaNIxQ9JNchQU/IzDDnMsWHAr5eXlWTU7LZLLPM9j+vSL+M1v/kAsNg8/WCdb\nQEXFcyxbdm+vt0WTjlko3SRHU9OLbN/uqKj456y5BBPJB6FQiJUrFzFp0vhMN6VDCtgZ0nqSI6EQ\nmEtDwylZcwkmki9CoRBXX/2P8aGR+qRn6olEFjJr1tRMNa2ZAnZWClNX922WLFmd6YaI5JWysjJK\nS8cTiUwAFgALiEQmUFo6nrKyskw3TwE7U1omOVp/k8NCIPPf5CL5KDE0snTpzVRUPEdFxXMsXXpz\nny+eaY8mHTMkMcnx0EMvUl9/WfzRhcB4YCGRyCSWLr2Z8vLyDLZSRPpaR5OOCtgZlEjrmzv3Wt57\n7xNisXOAcUQiP8/IklgRyTwF7CyXKDqTGLOeNWtq1hWdEZG+oYAtIhIQysMWEckBCtgiIgGhgC0i\nEhAK2CIiAaEtwrJQELYqEpG+pyyRLJNNWxWJSN9TlkiAtLdV0UMPvUhVVVWmmyciGaSAnWXaq+JX\nX38Zc+deq5KrInlMATtA3nvvE5VcFcljCthZZvbsaYTDPyNdFb9Y7ByVXBXJYwrYWaasrIzBgx3w\nJRL1eGECfhW/cZlsmohkmAJ2lgmFQixYcCv9+u0CngaeA27GL7n686zY9UJEMkMBOwuVl5dTVnYS\nkciLwJeBt4lEJmXNrhcikhnKw85SKrkqkp9UXlVEcl6urBBWwBaRnJZLK4S10lFEclp7K4Srqzfk\n1NoFFX8KiHSXe2effTYPP/ww99//a959dxsQZsiQIVx44TcCeSko0l3trRCuq/s2S5aszpnNrBWw\nAyDd5V40Oo/99/+/fPJJIbt2RYC/AXMBeOiheZSWLg/cpaCIdEz/mwOgvcu9d981du0qA3bh52vn\n7qWgSEdmz55GJHIPqSuEI5GFObV2QQE7ANq73IN/BiqB9i8FRfJBWVkZpaXjiUQmkFghHIlMyLm1\nCxoSEZHAC4VCrFy5KGXtws05N5ejtL4AqKqqYubMedTVPUVLT7oev77IecBDQOvnIpEJLF16c85M\ntojkC+VhB5zneUyffhHV1RuSckwXcsAB9ezYsS+7du0HfEpi0jESWRjI/FMRUcDOCemWqifS+hYv\nXsU772wFChg6dEhgV3iJSC8FbOfcm/i5ZDGg0cxOTHleAVtE9kquLDfvit4K2H8BvmxmH7fzvAK2\niHRbLi0374reXJqe9k1FRPZWy/qDJ4Ai4Bnq6o4jGn28eUNqz/OoqqqiomIOFRVzqKqqyul9T/em\nh/0G/kxXDLjHzP475Xn1sEWk2yoq5rBixZeB9cBG/PUGAHcybJhxxx0/5IorruW99yAWuxLIjR54\nRz3svcnDPsnM3nXOHQqsc869YmaPJr/ghhtuaL5dUlJCSUnJXnyciOSfjfHjSVrSVi/lnXfGM2PG\nd2lqGoS/ytd/rq7uUqqrJxCNRgOT0lpTU0NNTU2nXtsjWSLOufnA52b2n0mPqYctIt1WVVXF1Knf\nIRabh192IdkC4C78VNa2z1VUPMeyZff2RTN7XI+PYTvnBjjnBsZvR4AzgQ3db6KISGv+htQHZLoZ\nWaW7gzyHAY86517EX2L3WzNb23PNEpF8l9iQurDwLlKLOvk97GlA7hd8SqaFMyKStdKt8i0svJvG\nxo+IxV4Fvot/cd/y3Ne/fnzOTjoqYItIVktd5Ttz5hR+9asVrF37MnV13wI2Eg5XMnjwgSxYcAvl\n5eWBDdaggC0iOSZdqYZcWQGpgC1Afi7zFQkaBWxpXuZbXf0yO3d+FXiZcPh1TjxxFH/60yMUFKg0\nukg2yNyu6Q88AC++CLt29erHyJ5Fo9F4sC7GXzk2k1hsHuvXb+fkk0tzejmvSK7o3R729OlQWwuv\nvQZDhsDYsVBc3Po48MBe+XxpzV/mW4gfrJNXjdVTUHA8q1b9KDArw0RyWW8tTd+zFSv8P5ua4I03\n/OC9cSP88Y9w113wyiuw//4tgTz5z0MPBafaUj3rZdLt/9jUdAVLlqxWwBbJcpkdw/Y82LrVD+S1\ntbBpU8ttaOmFJwfyoiIF8m5of5mvB1xOUdGjTJp0oiYiRTIseJOOZvDBBy1BPHHU1sLf/tZ2WKW4\nGI48EjRx1i7P85g8+TTWr9+Ov9CgED9YXwA8j78De25UOxMJsuAF7I588klLLzy5Z/7uu37QTh4n\nHzsWRo2CwsI9v28eaGpq4uSTS3n22e00NV2BH7j/ALyENvAVyQ65FbDbs3MnbNnSelhl0yZ/7Lyo\nqO3wSnExDBzYd+3LEolc7NtvX8Bjjz0D3ESuVTsTCbLMTTr2pQED4Pjj/SNZYyO8/npLIF+3Dn76\nU9i8GQ4+uO0YeXExDBqUmXPoA6FQiPLy8vjimbGd/jktuhHJvNzpYXeV58Fbb7XtkdfWwj77tB1a\nKS6GYcNyZsKzJc3vCfyCi+2n+eXr3nrSs/Sl3zn5MSTSU8z88fDUcfLaWn/YZcyY1pOdY8fCEUdA\nOJzplndJVVUV559/PTt3jgM20bL90k+YOHEEjz++rvk/UlVVFTNnfo+6utb52xrrls7Sl37nKWD3\nlB070vfI338fjj667dDK0UdDv36ZbnVaibKVv/vdS/Gl6hvbXaru98a/gsa6pTs8z2P+/PnccstK\nYrEX0Zd+x/JjDLsvHHQQnHSSfySrq/MXASUC+QMP+MH8rbfg8MPbTnaOGQP77ZeZc4gLhUKsXLko\nqeLZ0cyadTVlZWWA36tOXLq+885W4MuZa6wElp+ZdDpPPvky/pzJOqAMvypGIXV139airS5QD7s3\n7d4Nr77atle+ZQt84Qvpx8kPPjijTU536VpYeFe8YPzrwID4K9U7kvZ5nkdVVRWXXnolH3/cj0Se\nv79DzHhgEX7Q1lVaKg2JZJtYDP7yl7arO2troX//9Ev1Bw/ukwnP9sarw+HjKCjYTUPD/wMgElmo\n8UdJK/GlH40+we7d+5Ca5w8TgJuBM/Sln4YCdlCYwfbt6RcG7d7ddnXn2LEwciT0YMDsaLx68uQq\nhg0bCvi7fgA88MAaQDP+0sKf0J7Hzp0NwJWk3/F8GQUFH3DOOSfoSz+FxrCDwjkYPtw/zjij9XMf\nftg6gD/yiH/7ww9h9Oi24+RHHQX77tujzRs2bCjLlt2bdtgkGp1Haely/ecTFi9eFZ/IruzgVZu4\n5prLufHGG/XvpQvUww66zz5rmfBMHl55+20/3TB1nHz0aH+RUTv8IZF51NW1zs0uLDyBZct+wJQp\nU5TmJx3yr9JeA44hXZ4/HMOkScN57LE/KFinoSGRfFRf33rCM/Hna6/54+Gp4+Tx2uSJ3nNl5bPE\nYlfG32wB4fAOzjnnNB588H5mzvym0vykXS2VIa/Dr73esqs5LODgg3fy/vuvaZejdmhIJB8VFsL4\n8f6RLLk2eW0t1NTA3Xf7t/ffn1BxMf9RWMhw9yEbqGQTB/NXbiUW+xpr104iGo12+LFPPPEUFRVz\nNKadx8rKyjjxxFGsX/9T4M/A74HVQIx+/XZx770/VbDuJvWwxed5sG0b1Nbyq2u+R/2L+zIWRzG1\nhPCopZhajH7HNTHq3K8z5/blvLLrOSwpzc9P1/oaMF6r2PJc28qQyizqLA2JSJekZooM4gPGsoli\n7mbK0S9x1oihfPz4E/Srb2Azw6hlMJt4nVqGsImlvM7rxFhFOPwI1177TU0s5alE7RB/YRbMmjVV\nV12doIAtXdLexGPypGJTUxNnTTqVnc9vYbR3EMUMYCxvUEw9Q2niDQ5jE0PY7N5mwAlj+Od77iBU\nXKza5CJ7oIAtXZKoM1JdvSGpUE/ry9n0mSK/Bq6lkKcYzVsUU0sxGzg2fBenD92fgX/9q5+ymDrZ\nWVzs7+0pIgrY0nV7upxNv8BmDpA+e6So6C4mTziBfzhtIqceeiihzZtbslc2b/brtKRbqn/ooX1w\ntiLZQ1ki0mWJjQ46l1PtAVHgaeA1oIiWAj++rVsPYenWCVQ+/F9tJ548z88bTwTwZ5+F++/3b4fD\n6TdjHj48Z2qTi3SWetjSLX6u7b8Siz0P/AOwkZZc2+QCP7tpqR1RTpcW2Jj5pWtTFwVt2gSff+5X\nPUzNJ//iFwNXm1wkmYZEpMd5nseIEePYvv1zYH/gOVqvZvsScArwp/jtRHU26JEFNjt2tK1LXlsL\n773n1yFP7ZFncW3ybKedYvqWhkSkx4VCIRYsuJVp076F511LS7Amfnsu/fvfzK5d5+IX++nh/9wH\nHQRf/ap/JKur88fEE4F86VL/9ptvwogRbcfJs6A2eTZT3Zjsoh62dFtLL/sK2qvu98IL7yalB3rA\nasLhK5g0aTxXX/2PfddTS9QmT+2Vb9niT2ymGyfPcG3ybNBSee/7wG/jj/4fBgy4kWXLfqC6Mb1A\nQyLSa37zm99w/vnXU1//LKk520uW/DuLFq2Mpwd+C3gA+IhEMfv2VkP26SV4LObvDJQcyBPBvLAw\nfQri0KF5M+E5Y8YlrFy5GdhF6zmKAcyYMYbly+/LXONylAK29Jo95WwDRKNRbr/9pzz55Daamjre\n0y9rNmtNbMacOtm5aRM0NLStS15c7NcmD/iEZ+qX5YYNz1Nb20S6OYrJk4fy6KOPZKiluUsBW3pV\nZ5Ygd3Yj30CUbv3oo/SZKx9+CKNGtR0n74Xa5D0hNThfcMG5/PKXy1m3blPzl2UodBOe9z3aG/J6\n9NHf9W2j84AmHaVXdS1nu2OLF6+KB4vWk5hZtVnrIYfA5Mn+kezzz/3a5Ime+P33t9QmHzmybQri\nmDEd1ibvTemuZCorr2+zd6fnPdnuewwdOqQvmipJFLClT8yePY1odB51dZfSemOEu5k58weZbFrP\n2W8/OOEE/0iWqE2e6I1XVsJtt/m1yQ87rO1kZ7w2eW+KRqPxYN1yJVNffyl+Cubv8XPmAaYD/wak\n/t7uYvbsW3u1jdKWhkSkT3RmY4SWGiUdF57KGek2Y960ye+lRyJtJzuLi3tsM+aOhqj88epEjrwH\nnATsAK5ofs2wYcbbb29UWl8v0JCIZFwoFOLii2fw0ENPEIs9DYSB24jFTm/eGKG8vJyysjJKS5dT\nXT2hzSRmWVlZRs+hx4XD/vj2UUdB8heRWXNtcmpr4aWXYPlyP5jHYulTEEeM6MHNmGO0lBtYCbyJ\nvwjqWcABp3DSSQ0K1hmgHrb0mc5OPHZmEjNvV9998EH6Cc8dO/z9OlOHVo48EvbZp83btHclEw4f\nRzjcwO7dQ4C/AXPjzyXKDSwkEpmUe1c7WUQ9bAmUPU1i5vXqu0MP9Y9TTmn9+Keftt6M+d57/dvb\ntvlBOyWfvOy00ygtHd/mSubMM09g7NgjuO22B1NSMP3x7X79xlFaelLuXe0EhHrY0mc66tV94xvH\nY9Yf51yb3nJqb3rUqOH8+MdV2Z36ly127fJXc6b2yl9/HRs6lPcPOYSnP9vF9oEHcsyMKZz0rW8x\n8ztXtXslNHlyFf/zPw/l9pdihikPW7JCukU2cCf+pXeEdCsggZTetEco9H08bxxwFDCNllKu2rW9\n05qa4PXX2y7Vf+UVPvKM53eNoJYz4nt5FrOJsXzACioqntffby9TwJas4Xke8+fP55Zb7iUW+xow\nAqgC0veWgaSFNPsCF+JnMSQyTZJLud6tgL23PI9H7ruPBXNv5oiG71LMa/H9PGsx/oYrHsXBkye3\nHmIpKsqbpfp9QWPYkjVCoRBbtmwjFrse/5J7Dn6NirYLZebOvQYzqKs7BT9YR/HrbqeOrU4AVhOJ\nLGTWrJtbfV7eTk52VyjEaXPmcPdDf2Rh9QPxq5rjiQy4hxklR/Pzf7m8pRrib3/r98w/+6xt5kqi\nNnmBQkxP0t+mZK2tWw8BzscfNrkI/59r2+AO3yYcvoLS0tNaTYbl+uRkZ7+MuvqlFQqFWLlyUUqm\nzg9afub001v/wCeftB5WWbjQv/3ee37KYupS/VGjVJu8mzQkIn2u9eTjOmAe0HoiMnWXGv/+YGAK\n6SbDxo79JRs2PNUqCHVUl+Sqq85hy5ZtQDB73e0VyTrzzGO4+OIZPPDAGiB9fZA+K6a1c6ffG09N\nQXzzTX+Lt3RL9QcO7L32BITGsCWrtJ58TJRd/ZDEpKO/2q7tLjX+AfA8nakc11Hedzj8w/iwTIaq\nAe6F1vMAp+MvHy8DdhMOH0dBwW4aGv4VgMLCu5LqgxSSWAwTDj/Ctdd+kxtvvLHvz7mx0V+Wnzrh\nuXmzX6clXSXEQYP6to0ZpDFsySqpl9xmYxg1ajhbtjzL+vVPs3XrqaTbpaaw8GPq6/cBTgAuiz96\nB1CHmeF5XqeDjz/h6QfyurpLqa6e0LzaMp1sGQtvWeL/XPMXjn+FshxYRCx2ZXwlqX9uLfVB1gEr\nSOy9GYtN5Ic//AkbN77RXBagz+yzT0tATuZ5fu87Ecifegp++Uv/9j77tO2RFxfDsGH5NeFpZr1y\n+G8t0jWVlZUWiRxrsMv8NdpmsMsikWNt3rx5NmDAeIOVBmMMhhp81+AOKywcZ9OmzbJYLLbH94Ji\ng8qkx8zgTquomJO2TbFYzM49d6YVFo4zWGCwwEKho23ixL+3xsbGPZ5TLBazyspKmzHjEpsx4xKr\nrKxs1c6u//0cl+acjo2f050Gc9qcG5xu0PbnwuFRtmbNmm61pc94ntn27WaPPGJ2xx1ml11mVlJi\ndthhZgMHmp14otkll5jddptZZaXZa6+ZNTVlutXdFo+d6eNqe0/s7aGALd0Ri8Vs2rRZ8UB7p8Gd\nFokca9OmzbKGhgabNOlUC4UGGQw2eNAgFj8etFBosJ100hnNATHdexUUjDaYGP+ZzgXsNWvWWDg8\nOk2QPNImTjylw+Abi8Vs6tQL4kHWD/YFBaNt4sTTOhXsU82YcUn8faxN++Gidr+M/MfT/9zkyaVd\nbkfW+Ogjs8ceM1u40Oyqq8xKS81GjDDr39/s2GPNzj/f7MYbzVauNHv5ZbOGhky3eI8UsCVQEj3S\nioo5VlExxyorK62xsbFN4PN7lRcYzIz3Hv3HEwE+FotZY2OjzZs3z4qKxlpR0VibMWOG9e9/TNoe\nfGVlZZs2zJhxiQ0adES7wS4UOrTVz6Vq6RHXxQPpJQYXGgzZY7BPp6OAHQoNtnB4cPyzYvHPu9Dg\nCwYHtvtzRUXjuv27ylqffWb2zDNmixaZXXut2ZQpZqNGmfXrZzZ6tNm555pdd53Z/febPfec2eef\nZ7rFzRSwJfDaHwoYZzAybQBetWqVTZz49/HecUswHzp0VHxopXUPvrGx0SorK2369Itt2LAx1q/f\nSIM7DCZ30Ksd0W7P3CwRYO+If7Ekf9kUGxzUpeGIWCxm8+bNs3B4SDwQV8YD8y4rKBht1113nU2d\nekH83CYajG3+POeGGByV5u+v2CZPPr0nfkXBUF9vtmGD2YoVfs/7/PP9nnj//maHH2521ll+T33h\nQr/n/tFHfd7EjgK2skQkEDqu37wc+FObxwcOvJHPPjsA2EC6tL5XX90O+NUAzz77bGbMuLhVmlzL\nKsrpwPdpm3r4pXjbJrZZXZmYpLz88mvYtu0UYD2pqznhWCZPHtGpfRHTpfH5+ekHEA5/zFe+MpxH\nH60mFAoxf/58br11ZUrxpp3A4cAXkv4OFxIO/5Vf//q/mDJlyh7bkNNiMX/CM7kueWLyc8CA9CVt\ne6g2eapeyRJxzp0F/AS/sPHPzey27r6XSG/47LMm4CrSraJ89dXWS9irqqra7MDir6I8HngXGIQf\noBPlRn8GfEA4XMDMma2DXVNTEyeffDpPP/0qnhfG/0K5uU074J946627O3Uu6XaI8dt3LLHYkbz0\n0nYqKi5h5cpFbNmyjaamK1M+bwDwfUKh+XjeMgD69fsbX//6aSqWBX5t8iOP9I/U2uTbt7cE8Zdf\nhhUr/PuNjel3Czr88B6sTd5atwK2cy6M//X+NWA78IxzrtLManuycSIJ7W0x5vcZ6uK3Ux//crvv\nt337O81pgJ7ncfvtd6bdS9LfZeX2+P0m/BS6EHAifvBewK9+tYLy8vLm9zr55DN58sl38Hu0HwMj\n223H4YcP79T5t7fXJfwT8Bw7d65uTk1sX4ivfvVEhg0bCsCsWVcHbsFQn3POX+QzfDiccUbr5xK1\nyRPH2rX+nx9/7NcmT01BPOqotLXJu6S9sZKODmAS8Luk+9cA16S8pg9GeyRfpMv4CIdHxSfTJsQn\nIO+MH4lMkDXxx1PHbY+2fv1GNo9bT516gYXDQzsYpz7ZYJTBofHx6MSE5yyDulYTlpWVlfEx8wfj\n4+u74mPNbdvRlZS6jrND5jTfTkzStpca2dEEqfSQTz81e+ops/vuM7v6arPycrMjj/QnPIuLzb7x\nDbN588yWLDF74QWznTtb/TgdjGF3d0hkGLA16f42/LXDIr0iXX2LmTNv50c/WsDjj5+D35tdHX/1\ncOBc/GXtK/H/aSbGfX8MHE5DQxXV1ZO48cYbWbt2I7HYz4B/J3WzWViIP5xxRvx9Do+/76X4vexb\nqKvbn7lzrwHg/vt/Hd+z8rf4Y8WF+KsQl6e04ycMHhwG6HDBT2Is/J13thIOrycWa699LfJqm7Vs\ntP/+cOKJ/pGsvr51bfI1a+CWW/xVn0OGtPTIO9JeJO/oAL4B/HfS/dnAHSmv6dUvMRGz9hbHPJiU\nEZFIb7vI/PS2cQaNzT3SoqJEJkUs3mNO7qknetExa9ubjcV78a0zUIYNGxPvhaf2iBPtmGwwyGCg\nwTgLh4dbjA+TAAAGV0lEQVTYpEmnps3Jbp3DfUf8845qp32te9DpUiO7u1hHelljo9nmzWZr1pj9\n8Ie90sPeDhQl3S/C72W3csMNNzTfLikpoaSkpJsfJ5Jeut7kgAH3cOCBIT79dEJSRsV64BigmvRT\nNyH82iVR/HHh4fg918TmCKmi+BsvtGRi1NVdSix2Av36/ScNDf8BzKelxx7C76X/C3AI/lz95cRi\nsH79jzn55FIef3xdq55224nGy4HVhMNXcOCB+/L5546GhgnA3W160HvaZk2yR81jj1FTU9Op13Yr\nrc85VwBsBk4H3gGeBmZa0qSj0vqkr6TbtPfss8/m4YcfZsmS1Wzf/g7PPLOZhoaN+NkSkJze9+Mf\nV6ZsW/Yg/rTMy7RfQfAiYCLp0gyHDVvAjh0F7NwZAT7C/wIAfzhmN9Af+HOr9y4oOJ5Vq37UKsB2\nlMo4Y8azzJ49rcONiiWYejytz8yanHP/iN9dCQO/MGWISIa015tMPNZSHXBSmzHd+fPns2nTX1r1\n0P0EqBB+lkkiWP4E+BR4Ez/3uxo/YLc1cmQRRxwBb765lYED+wP3sXnzG3jefvg77FxAarZHU9MV\nLFmyutM9YuecetB5qNt52Gb2MPBwD7ZFpFekL8h/c3OPNPk5vzdeT0PDBuCP+BOZMfbdt5EvfnEo\nr7yyAr8ULPiBvfUkYDj8M559tqW8aSRyD+PHRzAbBLxES5XBtlKvSNtLZUy3s47kB610FEmSbqPg\nRG88HC5g5cr++OPhT+BnfGwgkfmxzz534nmfxGtPtwy9hEJH4Hnfw++tV5F+w4bxzJt3PjfddFOn\n2hKU2t3SdaqHLdJJHfXGo9Eoq1Z9h1hsHn5ATkxSrgZe5YAD6vjww+tpCdYAhXjeUUn306X4LQQG\nNe+A05m2KFjnJ/WwRTrJ8zxGjBjH9u1XkG4isKjoLrZunZvmue8Cf8AfEikEPPwgfwVwLP6KyTep\nqHhBO75Lhz1sfU2LdFIoFGLBglspLLwLfxgjwR9XvvjiaUQi97R5rl+/RwmFPsbvVS8A7sZfpHMa\n8BBwBpHIz5k1a2ofnYkElXrYIl3Q0bjy8uW/pKLikjbPnXnmMZgZDz+8noaGYcDb+HVJrgVCGpeW\nVjK+CW9NTU1eLJrJh/PMh3OEjs8zXd53Yly5veeApD0sLb6H5TaccxnLodbvMjt1FLD7ZAOD+fPn\n7/3yzQDIh/PMh3M0y4/zzIdzNAveedLB0nRdf4mIBIQCtohIQPTqGHavvLGISI6zvp50FBGRnqUh\nERGRgFDAFhEJiD4L2M65m5xzf3bOveic+71zrmjPPxUszrkfOedq4+e5yjl3QKbb1Bucc9Odcxud\nczHn3Jcy3Z6e5Jw7yzn3inPuVefcv2W6Pb3BOXevc+5959yGTLeltzjnipxzf4z/O33ZOXdlptvU\nE/qyh327mR1nZscDa/C348g1a4FxZnYcsAV/KVsu2gBMBf6U6Yb0JOdcGL9m6lnAWGCmc644s63q\nFffhn2MuawSuMrNx+IXL5+bC77LPAraZfZZ0dz9aigrnDDNbZ2Ze/O5T+PtM5Rwze8XMtmS6Hb3g\nROA1M3vTzBqBZcCUDLepx5nZo8COTLejN5nZe2b2Yvz250AtMDSzrdp7fVpe1Tn3A+BCYCftbdeR\nOy4Flma6EdIlw4CtSfe34VdskgBzzo0E/g6/ExVoPRqwnXPrgMFpnrrOzKrM7HrgeufcNfgb3M3p\nyc/vC3s6x/hrrgd2m9kDfdq4HtSZ88xBynHNMc65/fA36fyneE870Ho0YJvZGZ186QP4dSUDZ0/n\n6Jy7BPg6/gbFgdWF32Uu2Q4kT4YX4feyJYCcc/sAvwYWm9maTLenJ/RllsjRSXenAC/01Wf3Fefc\nWcC/AlPMrH5Pr88R6auKBdOzwNHOuZHOuX2BCqAyw22SbnDOOeAXwCYz+0mm29NT+mylo3PuQWA0\nEANeBy4zs7/2yYf3Eefcq8C+wMfxh9abWer2I4HnnJsK/AwYhL+V+AtmdnZmW9UznHNn42+RHgZ+\nYWa3ZLhJPc45txQ4FTgE+CvwfTO7L7Ot6lnOucn4WUwv0TLUda2Z/S5zrdp7WpouIhIQWukoIhIQ\nCtgiIgGhgC0iEhAK2CIiAaGALSISEArYIiIBoYAtIhIQCtgiIgHxvziTkkFcQVOjAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10a477048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_regression(regressor, data, y, tf=None):\n",
    "    plt.scatter(data[:, 0], y, s=40)\n",
    "    X = np.linspace(data[:,0].min() - .5, data[:,0].max() + .5, 100).reshape((100, 1))\n",
    "    if tf is None:\n",
    "        y_pred = regressor.predict(X)\n",
    "    else:\n",
    "        y_pred = regressor.predict(tf.fit_transform(X))\n",
    "    plt.plot(X, y_pred, color=\"r\")\n",
    "    plt.xlim(data[:,0].min() - .5, data[:,0].max() + .5)\n",
    "    plt.ylim(y.min() - .5, y.max() + .5)\n",
    "\n",
    "regressor = LinearRegression()\n",
    "regressor.fit(X, y)\n",
    "plot_regression(regressor, X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are in a typical case of underfitting: our model is not complex enough to explain our data.\n",
    "\n",
    "Let us see what happens if we now feed our linear regressor with polynomial features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 3)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAD7CAYAAABOi672AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4k2X3wPHvnRRaCMoQkS0oQ0YBRaDIqgoUyFsVFGpp\nZamIoDheN0VQcCM4qAq+oiLKqAxbCxYcVV5ALCgKiICIyvCnAiovgZY2uX9/PGlJS7rTPk1yPtfV\nyzZJk/MAnpzc49xKa40QQoiqz2J2AEIIIUpGErYQQvgJSdhCCOEnJGELIYSfkIQthBB+QhK2EEL4\niZCKemKllKwXFEKIMtBaK2+3V2iFrbVGa820adPyvvfZ19Gj6Nq1jf/6+rnL+FUh11nFvoLhGoPl\nOoPhGv3xOotSYRV2hatXD4YMwbVoEaktW7Jo0QoA4uOHYbfbsVhktEcIEVj8N2EDrptv5tdhNxCb\n0xzHydsASE1NICpqKUlJCyVpCyECSqVktMjIyAp53tQTJ3D+7yTtT84FJgITcTg2k5a2ndTU1Ap5\nzaJU1HVWJcFwjRAc1xkM1wiBdZ2quDGTMj+xUrqinjtXTMxYLlp2jItowHhe97gnkZiYrSxZsqBC\nX18IIXxNKYU2Y9KxMrxJBDfwPudw3OxQhBCiQvl1wo6PH8YJ2xI+pR8jec99ayY223zi4oaaGpsQ\nQviaXw+JuFwuhg8fRXbqRh7LyuYyHsRme52oqHCZdBRC+KWihkT8OmGDkbRTU1LoHhfPiz0j6Tl5\nvCzrE0L4rYBO2Hmeegp++glef734xwohRBVV5klHpdQCpdTvSqntHrdNV0odVEp94/4a5OuAy2Ts\nWHj/fTguk49CiMBU3LjBm0DBhKyB2VrrS91fH1VMaKXUsCFcdRW8917xjxVCCD9UZMLWWq8H/vJy\nl9dy3XS33Qbz5kFlDsUIIUQlKevM3J1KqW+VUm8oper4NKLy6N8f/vkHMjLMjkQIIXyuLL1EXgUe\nd38/A3geuNnbA6dPn573fWRkZMVvEbVYjCr71Vehe/eKfS0hhPCB9PR00tPTS/TYYleJKKVaACla\n6/BS3le5q0Ry/fkntG4N+/dD3bqV//pCCFEOPt2arpRq5PHjUGB7YY81xfnnw5Ah8PbbZkcihBA+\nVWSFrZRaDPQD6gO/A9OASKALxmqR/cBtWuvfvfyuORU2wPr1cOutsGsXqKo5PyqEEN4Ex8YZT1pD\neDi8/DJceaU5MQghRBkEdLc+r5SCCROMyUchhAgQgVlhg7G8r0ULY1ikYUPz4hBCiFIIvgoboHZt\nGD4c/vMfsyMRQgifCNwKG2DbNoiONpb4hfj18ZVCiCARnBU2QJcu0Lw5JCebHYkQQpRbYCdsgEmT\nIDHR7CiEEKLcAntIBCArCy68ED77DNq1MzsaIYQoUvAOiQCEhsItt8gSPyGE3wv8ChvgwAFjPPuX\nX6BWLbOjEUKIQgV3hQ3QrBn06weLFpkdiRBClFlwJGwwJh/nzpXDDYQQfit4EvZVV4HLBSXsOyuE\nEFVN8CRspeDOO42GUEII4YeCY9Ix14kTxhK/r782/iuEEFWMTDrmqlULRo+GV14xOxIhhCi14Kqw\nAfbtg4gIY4lfzZpmRyOEEPlIhe3p4ouNhP3ee2ZHIoQQpRJ8CRvOTD5WxU8AQghRiOBM2AMGwOnT\n8PnnZkcihBAlFpwJWymYPBlefNHsSIQQosSCb9Ixl8NhLO376iu46CKzoxFCCEAmHb2z2eDmm2Uj\njRDCbwRvhQ3w669w6aXGEWLnnmt2NEIIIRV2oZo3h/794a23zI5ECCGKFdwVNsCmTXDTTbB7N1it\nZkcjhAhyUmEXJSIC6tWD1FSzIxFCiCJJwlYK7rkH5swxOxIhRLB75pki75YhEYDsbGPL+qpVcNll\nZkcjhAhGx49Dy5aoY8dkSKRI1aoZG2lmzzY7EiFEsFqwwNiFXQSpsHP9/bexgea776BpU7OjEUIE\nk5wcaNUKli1D9eghFXax6tSBUaNkI40QovKtWGEcFt69e5EPkwrb0/790K0b/PyzcdiBEEJUNK2N\n1WoPPwzXXSfL+kqsZUu48kpjLEkIISrDxo1w9ChERxf7UKmwC/rySxg5EvbsgZAQs6MRQgS6YcPg\n6qth0iRANs6UTkQENG5sjCkJIURF+vFHWL8exowp0cMlYXtz//3w3HNyIo0QomI9/zxMmGB0Dy0B\nGRLxxuWC9u3htdcgMtLsaIQQgeiPP6BtW6OPUYMGeTfLkEhpWSzw73/Ds8+aHYkQIlDNnQsxMfmS\ndXGkwi5MZqaxamTdOujY0exohBCBxOGAFi2MFSKtW+e7SyrssggLM05XnzXL7EiEEIFmwQLo2/es\nZF0cqbCL8tdfRlMo2a4uhPCV3G3oS5YYq9IKkAq7rOrWhdGjpfWqEMJ3kpKMbeheknVxpMIuzoED\n0LmzsV6yXj2zoxFC+DOtoUsXePJJsNu9PkQq7PJo1gyuvRYSE82ORAjh7z76yEjaQ4aU6delwi6J\nXbugXz+jOVQJF7gLIcRZ+vY1NsqMHFnoQ6TCLq927aB3b2kKJYQouw0b4OBBGDGizE8hFXZJbd5s\n/EH/+KNxQo0QQpTGNdfA4MFw++1FPkwqbF/o0cNY4rdkidmRCCH8zY4dkJFR4iZPhZGEXRoPPwxP\nPWX0GhFCiJJ65hm46y6oUaNcTyMJuzT69zcmHVetMjsSIYS/+OknWLOm2KGQkpCEXRpKwZQp8MQT\n0npVCFEyzzxjJOvatcv9VDLpWFouF3TqZPQYGTTI7GiEEFXZwYNGvtizB+rXL9GvyKSjL1ks8Mgj\nRpUthBBFmTULxo0rcbIujlTYZZGTY6zNfuMNYyG8EEIU9McfcMklsHMnNGpU4l+TCtvXQkLgoYek\nyhZCFG7OHLjxxlIl6+JIhV1Wp08bLRKTkow12kIIkeuvv4z8sHWrcVBBKZS5wlZKLVBK/a6U2u5x\nWz2l1Dql1B6l1FqlVJ1SRRMoqlc3quwZM8yORAhR1bz4otE0rpTJujhFVthKqT7ACWCh1jrcfduz\nwBGt9bNKqQeBulrrh7z8bmBX2ABZWcbuxw8+gK5dzY5GCFEV/P23UV1v3mzkh1IqqsIudkhEKdUC\nSPFI2D8A/bTWvyulGgLpWutLvPxe4CdsgJdfRq9bx4e33sqiRSsAiI8fht1ux2KRKQIhgs6MGUbP\nobffLtOv+zph/6W1ruv+XgHHcn8u8HtBkbBdDgd/1z+ff9GUTZl3A2CzzSMqKpykpIWStIUIJseP\nG1X1hg3Qpk2ZnqKohB1Snti01lopVWhWnj59et73kZGRREZGluflqqTUTz9lvasu951ux/VMBMDh\nGEdaWg9SU1OJjo42OUIhRKV5+WWIiipVsk5PTyc9Pb1Ejy3rkEik1vr/lFKNgM+CeUgkJmYsKcs6\ns49niCKN7XRy35NITMxWliyRHtpCBIX//c+orr/4wlh/XUa+XoedDIx2fz8aCPpOSKeozizu41Ee\nNzsUIYRZEhPh6qvLlayLU9yyvsXARqCtUuqAUmos8DQwQCm1B7jK/XPQio8fhs02j1cZSy820Jlt\nQCY223zi4oaaHZ4QojIcPw6zZ8Ojj1boy8jGmXJyuVwMHz6KtLTt3OJoTT/2cpMNmXQUIpjMnAk/\n/ACLFpX7qcq1SqQcLxoUCRuMpJ2amsqyt5N4cfVydjwxk9533SXJWohg8Pff0Lp1uVaGeJKEXZnm\nzoW0NEhJMTsSIURlmD4dfvkF3nzTJ08nCbsyZWYa77YrVkC3bmZHI4SoSMeOGVX1V1/BRRf55Cml\nW19lCgsz+mVX8OSDEKIKmD0bhg71WbIujlTYFeH0aeNd9913oVcvs6MRQlSEP/80lvCVoSNfUWRI\nxAxvvWV8ffaZcRakECKw/PvfRgO4uXN9+rSSsM2QkwMdOxpbVQcMMDsaIYQvHTwInTvDjh0+PaAA\nZAzbHCEh8Pjjxnh2ML9xCRGIZs6EW27xebIujiTsinTDDZCdbfTLFkIEhn374P334YEHKv2lJWFX\nJIvFeCeeOhWcTrOjEUL4wvTpMHkynHdepb+0JOyKZrfDOefA4sVmRyKEKK+dO2HtWrj7blNeXiYd\nK8MXX8Do0UavgdBQs6MRQpTVtddCv35w770V9hIy6Wi2vn2hfXuYN8/sSIQQZfXf/8K2bTBxomkh\nSIVdWb77DgYOhL17jSESIYT/0Br69IFbbzU+LVcgqbCrgk6djPXYzz9vdiRCiNL68EP45x+Ijzc1\nDKmwK9P+/XD55bBrFzRoYHY0QoiScDqNTTJPPw3/+leFv5xU2FVFy5bGO/SMGWZHIoQowOVykZKS\nQkzMWOOs1pQUXC4XvPMO1K1rrPgymVTYle3PP6FdO9i0yWjDKoQwncvl4oYbbmLt2p04HOMBsNnm\nEd2/He99vRG1ZAlccUWlxCK9RKqap56CLVtg+XKzIxFCACkpKcTGTsXh+BIIc9+ayaPVWnJbt4to\nvGFDpcUiQyJVzd13Q0aGcaSQEMJ0ixatcFfWYYALSKE+Y7kz+wj3ZmljaKQKkIRthho1jC3r998v\njaGEqFJcwE3AVKZxmHfpzfJvjzF8+KgqkbQlYZslPh596hRbHn747EkOIUSlio8fhs02D1gJ7KQ1\n7xDD98wgiZycbaSlbSc1NdXsMCVhm8UFPGarR91n57By2aUsW9aNYcPup3nzDnzwwQeSuIWoRHa7\nnaiocKzWO4HxPM00ZnEfR6kPhOFwjOfdd1eaHaYkbLOkpqYya9tRftBXcQc5wERycrZx6JAiJubu\nKvMRTIhgYLFYSEpaSM+e4fRhL13ZyktMNjuss0jCNknuJMd9zOZhnuI8jmBMeEwiK6tvlfkIJkSw\nsFgsPHDfRF60zONBZpBJDfc9mdhs84mLG2pqfCAJ23Q/0I4l3Mh0pnvcaq0yH8GECCb2o0epUcfG\nhzVnAYlAIjZbD6KiwrHLxpngZaz7TMDh2Ew9HPzAJfRjLbsYA8wEfiUmZitLliwwOVIhgsSJE9C2\nLa7ly0n988+8gikubih2ux2LpXLqW9k4UwW5XC6GDx/F6tXbyMy8nbv5lAGsw841wHxstp4sXjyT\n6Ohos0MVIjhMnQo//2xsRTeRJOwqKrd3waRJD3P0t7/41nWKu4hlvW0jUVHhJCUtrLR3dSGC2i+/\nQNeuRr/rpk1NDUUSdhXncrlITU1l13MvEbctg2/eWsCQ666TZC1EZYmJMQ4ZmTbN7EgkYfsNrWHI\nEOjfH/79b7OjESI4pKfDmDHw/fdQs6bZ0UjC9iu7d0OvXrBjBzRsaHY0QgS2nBy47DJ49FG44Qaz\nowGk+ZN/adsWxo6FRx4xOxIhAt+8eVC/Plx/vdmRlIhU2FXR8eNwySWwahV07252NEIEpiNHjHHr\nTz6B8HCzo8kjQyL+6K234NVXjYMOZPJRCN+7/XaoVg1eesnsSPKRIRE/43K5SKlbl9379jOvRx/p\n4ieEr23ZAitXwmOPmR1JqUiFXcV4HlXUyjGYNBLpVrMZ3QZdKuuyhfAFlwsiIowKe+xYs6M5i1TY\nfiQ1NdV9rtyXfMtTLGUMU072ZPXqbaSkpJgdnhD+7z//MYZCRo82O5JSkwq7iomJGcuyZd2AiQDU\n5m++pz3XEcfhJqn8+usOqbKFKKsjR6BDB1i7Fjp3Njsar6TC9mP/UIcHeYZXWMofv/0lLVeFKI+H\nHoLY2CqbrIsjCbuKiY8fhtX6EpCZd9sibsDBMca7LpaWq0KU1aZNsGaN3000epKEXcXY7XYaNlTA\nZeT244UIJnAl09hG3VMnzQ1QCH+UnQ3jx8Ps2VC7ttnRlJkk7CrGYrGQmPg0oaGngK+ArcBMfmAp\nb1azMfXIAZMjFMIPzZ5tdOEbMcLsSMpFEnYVFB0djd3eC5ttG9AV+BWbrSffDLmSRr//DjKOLUTJ\n/fQTPPccvPIKKK9zeX5DVolUUbktV8869eKTT4yPdjt2gM1mcpRCVHFaw+DBcOWV8OCDZkdTIrI1\nPdDExUHjxkbVIIQAzhQ5ixatAIwJfPuJE1iefBK+/tpYe+0HJGEHmj/+MJrVrF5tnJIhRJDz3CHs\ncIwHoFnNRL51/kTtTz7G0quXyRGWXFEJO6SygxE+0KCBUV3ffDNkZPhN5SBERfHcIQxhADx+chOL\nQ36n2bFjBMrJqJKw/cRZH/fihmJv0IDdN9/MtEwLv/12ELDSqFEjbrrp+ko95VkIsy1atMJdWRvJ\neiBpRLKejjmP8K93VwbMYdaSsP2At497qalTaV/zOKv//JRtdGQv2cAkAFavTiAqaqk0ixJBycYJ\nXmMCtzEPBz+aHY5Pyf/NfiD/x72JwEQcjs1k/BnCDCJ5nR9RZOS7Ly1tu2xjF0EjPn4YNts8IJOZ\nJLCePqylHzbbfOLihpodns9IwvYDBT/uGcKAu5nLYUK4gIksyHefwzFetrGLoGG324mKCufqsI7E\n8Cb30AGbrQdRUeHY7Xazw/MZGRLxcy4U44hnA9NZw2B+4mKzQxKi0lksFpLefo2Tbdsyr2VXBjTd\nTVzczICby5FlfX4gJSWF2NgEHI7NnKmyM4Fw4AZgNfcSyzWs4Uo+Q3Mam60HixfPDJjJFiGKde+9\n8NtvsHix2ZGUi6zD9nMul4vhw0eRlrY9b9LRZptP7dqZ/PVXdU6dqoWFv/mC0yymK2/ZdhMVFS6T\njiJ4rF9v9AnZvt04Bd2PScIOAN62qg8ePJg1a9awaNEKDh8+QLOTWby2I4OMl1/iyltukWQtgsPJ\nk0Z/62efhaH+P8FYIQlbKfUzcBxwAtla6+4F7peEbYY5c2D5cvj8c7BazY5GiHLxut284Lj05Mlw\n9Ci8+65JUfpWRSXs/UBXrfWxQu6XhG0Glwv694eBA43TNYTwU972H9hs8/IP961da+z4/e47qFvX\n5Ih9oyIT9uVa66OF3C8J2yy//gqXX278Y+7SxexohCgTY7J9Kg7HRuATYAXgJDR0PUuXvsC1ffqg\nO3fmy1tv5YWd+4FCKnA/U1EJ+yfgH4whkXla69cL3C8J20wLFxr9RjIyICys+McLUcUYB1J3BTYB\nO4Hx7nvm0qSJZlPL+qz7ZgfjMy/A6ZwMeKnA/VBFNX/qpbX+TSl1PrBOKfWD1nq95wOmT5+e931k\nZCSRkZHleDlRKjfdBKtWQUICzJpldjRClNFO99eZpk4wjr6HWnDi0F7uoDVOvsm7z+EYR1paD1JT\nU/1mSWt6ejrp6ekleqxPVokopaYBJ7TWz3vcJhW22Y4cMYZE3nrLGNcWwo+kpKQwdOhtOJ0JGG0X\nDM35hS10ZDDns5X78t1nSCQmZitLlizAHxVVYZfpM4NSqqZS6hz39zZgILC97CGKClG/Prz5JowZ\nYyRvIfyIcSB1/gNzreSwiHhmcTVbqWFSZOYp6yDPBcB6pdQ2YDPwodZ6re/CEj4zYADExhoz6fKJ\nR/iR3AOpw8JewdjZCw/zFKcJ4Tl2A8OAeXn3GTIDruGTJ9k4EwxOn4aICOMsyAkTzI5GiBLz3OXb\nydGflbzOFaGN+CXnOE7nXmACxod7Y0IyLOxVhgzpErCTjpKwg8Xu3dC7N3z2GXTsaHY0QpSYy+Ui\nbdkyLrvlVhZe2p02903m7beXsXbtDhyOW4CdWK3JNGxYh8TEp4iOjvbbZA2SsEWut94ytu9mZMiJ\n68J/aA3Dh8MFF0BiIuC9VYO/r7/OJQlbAMY/8kP9+/Pj/gO81r13QGwyEEEgMRHeeAM2bgyKPQWS\nsEXeNt//fvQtn586xJM0YLH1f3Tv3oYvvviYkBBpjS6qoK+/hkGDjGTdqpXZ0VQKny/rE/4nNTWV\ntLQd/HmqIzE0YDaHaeW8hU2bDtGnTxQul8vsEIXI7/hxo2Xq3LlBk6yLIwk7SCxatIKTJ68AfmA7\n3/Iwc1jOcmxsYsuWQ3L+o6hatDaWog4YYCRtAUjCDjI7MJY/hfEGN7OZHsxnMjk5d8j5j6JqmT0b\nfvkFXnjB7EiqFEnYQSI+fhhW6z6PWxSTSKQdu5jEEjZu3ExMzFhSUlJkeESY64svjMZl778PoaFm\nR1OlyKRjkHC5XPTufRWbNh3C2GgQBrhoyTVs4iOu426+5KKA6HYm/Njhw9Ctm9FSYeBAs6MxhUw6\nCiwWC1988TEREc0JCekCJAIT2c8expHEMpbSkKE4HJtJS9suY9qi8p0+bYxX33570Cbr4kjCDiIh\nISFs2LCOFSueo3fvFCAJuJvVDGU+41nO9VRH4XCMlzFtUfkmT4bzzoNHHjE7kipLEnaQsVgsREdH\n07hxI6B93u1PMIX/oyFzuQM4eyjL5XKRkpJCTMxYGesWvvfaa8bJ5++8AzIUVyjZLRHUOmJ0OxuH\nJozRvM0mIrjDkszAuP/kPcrb2XqpqQlERS2VsW5RYoUeqLthA0ybBhs2wLnnmhxl1SYJO0jFxw/j\nww+ncPJkB6AHMJ4TwLU4yLD+RZ2aNfMem5qa6k7WZ0798MeTPYR5CnvTj+uzgNe+3YxauFA2x5SA\nlEZBym63M2hQJ2rW3An0BJZgtT7BBT1bUPvDD7GMHAl79wLGphvjfzLPPg5hMtYtSsTlcjFt2jSS\nk7e63/QnAhNRjo+5M+0jvh88GKKizA7TL0iFHaQsFgtJSQs9Op61Ji7uAex2OwDfDh/O+V0vJ+Eq\nO4eP/gF0NTVe4Z9ycnLo0+dqvvxyB8acyTrAjgXNYsayQXflsxMulpgcp7+QddgiH8+Pro85zuNS\nDnJtaAinco7hdO4DcodKMrHZerB48UwZEhFnyZ2kHjduMseOhQJ3u++ZB4Qzh/PowPcMIZrrY7b5\n7fmLFaGiTk0XAchzvPoBqrGSobyQVY/bLBsJDe1AVtZ9ANhs84mKCs+ryIXIlfumn5q6kdOnQ4Hv\n8Dzx/HYuJgoLPckg1BZFXNxME6P1L5KwRT6e49UuYCTvkU4kj7ja8Fm3HJo02QpAbOzj7v/eDCC9\ntUWe3M6QRrKejOfcxzWkkcAJ+tAaR0gk10RdLm/6pSAJWxTJQS3spLKJDjTKasftSxbIMj9RpDOd\nIZPz3d6DL/kPtzCE2/iJN0h4aCKPPfaY/HspBfmTEvnExw/DZst/EvUf1Oa66nUZu3cHpKUVWOZn\nzPjLlnaR3w7gGnJPNW/NHlZxHaN5nS2soGfPcEnWZSB/WiIfu93OwIEdsVo7Y/QbSQQu43vnCR7v\n1A0dH8+mF+fJMj9RqDOdITsA4TTkMtbQiylczRoeoV69HL744mNJ1mUgf2IiH4vFwujRIwgJOQ18\nBWwFnsHp3MdLW38nY8IEHtjwCe34zevvS5tWYbfb6d69DfAidXiGNE7yJs1YQAihoadYsOBFOZKu\njGRZnzhLTMxYli3rhjHc4SmRmJitPNC4Dg1emEtvvZ1faOu+LxMIB/oD4dKmNcjl5OQw4Ir+PJmx\nmY1cwX0MxWZ7Xf5NlIC0VxU+1WXWLNLCO/Gx6kQDnsIYNukINAVeBJrhcHTmgw8+Zdq0aVJpB6EQ\np5NP61SnydW9yBjRnJiYr1m8eKYk63KSClucJSUlhdjYBByOzZwZp86/USYnJ4c3W7Yi4uAhruRy\njtIF+C+QhbG5xlg9EhLyEtdcc7n8jxpMsrMhJsb4ftkykOGPUpEKW5SK3W4nKiocm60HuROPNluP\nfBtl1qxZwz3HavMhd7OOTOrwJDDd/QxnVo/k5GyT1SPBJCcH4uONwwgWL5Zk7WNSYQuvclth5q76\niIsbmm9jzJlx7tuZxX30YT0DaM1xeuFt7LtZs1fo2bO7bLAJZE4njB4Nf/4JH3wAYWHF/444S1EV\ntiRsUSb5JyadvMS1XM6nDKIzx3kEsHPmA1wisBS4USYjA5XTCbfcAr/+Ch9+CDVqmB2R35IhEeFz\nxlrbl4CTwCgmc4AMuvMxh6nLQ8AowIWxemQ+cD+ywSZA5eQYlfWvv0JysiTrCiQJW5SJ3W6nYUMF\ntAW2AZu5i8/4nOF8QjXO4yuM6vsyjOV+uf0iZIONvynyeLjsbIiNhaNHjcraZjM32AAnCVuUicVi\nITHxaSyWTGASxmoSxf08xxqGkM4JWoatAPoBC5F/av4pt29MbOxUli3rxrJl3YiNTWD48FG4Tp2C\nG24wJhhXrZLKuhLI/0WizKKjo2nUqH6BWxVTeIIldGU9J+hQ4zPgtPs+F7Acq/UJDh06JLsh/UBu\n5z2HYyqQAWTgcDzKhjXfcDQiwphYTEqC0FCzQw0KMukoyuWDDz7gxhunkJm5hYJrtjfE96Tp2wsZ\nopry1ak7gfeAo+Q2sy9sArLQw1plkrLSjRgxhqSk3cApctfWn08ia/iFoxc3ZeDunWC1mhpjoJFV\nIqLCuFwuhg8fRVra9rxWq7mHGyQlLYSlS8m+/XbubX4x83c5yMnZRmGbcXKfr2DrVllZUnkKvllu\n3/41u3blYPSUCeNCfmYtA1jC36zr1Yn1//3E1HgDUVEJG611hXwZTy2CgdPp1MnJyTomZqyOiRmr\nk5OTtdPpPPOAtDT9d2iYHsE4DbrA11wdEzM276HJycnaZuus4ZTHY05pm62TTk5ONuHqAlfu39uI\nEWP0iBFj9KpVq/R118W6//wTNSRqi6Wh+3utu5KhD9JY38mLGubq3r2jzL6EgOTOnV7zqmxDEuVm\nsViIjo4u/GzHgQN5ot9AZq1dQQva8CwPAN4LiOJOaJfzI33D2yeZ5OQpZGcfzXd2p8v1JQDRJPMG\nNzOe+axiKJBI48aNTIo+eMnnS1Ep+txxC1fXaMhI3uU1JhBCNpBJWNirxMZea3Z4QcfbIRSZmVtw\nOusCnsMcN3Anj/Mat2En1Z2sMwkLe4X4+GGmxB7MJGGLSmG32+k46DL6WU7RhA2soz316UR29lHe\nfntZ3moRbyfeGGPd84mLG2pK7IGosE8yxhJNY418NU4zjxRu4zi9qEEGGeQeaHHeeS45i9EEkrBF\npcg9GCGzWg7XcimbqMFX/EN75yrWrt2Rt/OxJI2nREVz0oDf+JQuNOAjIjiHn+kGbMGYfOxLr14R\nMgFsAlk2iBRuAAAQI0lEQVQlIipNwYMRbmQxLzGZO4hGxcCSJQuA4htPeT5Glv6VTWEtdK3WzvRQ\nx1mcc4w3OZfHmIbGgnE2YzgwH5utZ76VPcK3ZFmfqBK8nWTThW94n4H82Ko+UTu2lWgDhiz9Kz+v\nyzFrzmNOq+rE/riLMZl1WO7ah2cyh8sIDT2F3d5L/pwrkCRsUSUUVtXVs3RkTaNM6p3M5IUrIom6\nbXS+arlgNd2mTVPmzElxT5gVvqZbFM3zk0yN7NPM/OMXGjsc3NWoJS+vvhpvbXJ7907h889XS7Ku\nQLIOW1QJTqdTDxsWp222Thrmur8u0dBYQyt9D8P079TSo0Ob62HD4rTT6dROp1MPHTrSY23wy9pi\nqauht4YxGpI1OL2u6RYltHmz1q1aaT1+vNanTukRI8bkrb0uas28qBgUsQ5b3iZFpbFYLCQlLeSe\ne67Ban0S41T2G4Dzge3MYTmDSefhrFCGJa8mLSmpwPKzCcAmXK4GQCzQDUjgTCtXUSo5OTBjBkRH\nw9NPw7x5EBYmK3WqMNk4IyqVxWJhz56DOJ1TMD5yj8XoUWEMbXxNVy7jG2bnRNEn7ib+XbcxDsdg\noDqQCuzEaOeaOxQyDugBrHQnlJn5Xk8mJwuxdy+MHWs0b/r6a2jSJO8uY6XOUtLSepzVbkBW6phL\nEraock5iYwKx2J1/8dqRw0SylIc4xglq4JncDWHAeKzWO4mKuipfQvE2OZmamkBU1NKAmDQr6ZuR\n5+OUy8XUc6y0T05GJSTA5MlQ4PG5n4Tyr9SZKW90VUFhYyXl/ULGsEUhjH4hndz9QpI15H5/pneI\ncVuyrs1f+g1G6f1U04O4tNCx1fbtL8/fv0QX3ZckISEhr4fGWb1P/MDZY/uJ2mbrpIcOHalXrVrl\ntT9IRx7RX3KhTrfY9O0Dr/G7aw4WFDGGLQlbVLr8k48vaYjQ0MpjIrKdhjiPyUStBzBJ76Wafp9z\ndFP2FEju7XTv3lef9TpFTZ5ZrY3zJbrcSU5/4HQ6dUJCgvsabvKYeD2lrdY2OjS0Rd61hYV10HUs\nDfTz3KF/53w9nolaEa+t1kY6ISHBb645mBSVsGVIRFS6gh+5tb6ENm2asmfPFjZt+ooDB/ph7HI8\n8/F7He24PLQud2Vl8g3teJ4hzOEqsngNcKC1xuVylfgju9PZn9xlaw7HONLSepCamlroksCqMhae\nO8yTnLzVPQ8AxsTrUmAhTudknE7jeDaFi6GZNXmOW/mIXXSgD0fYAIzH6ezJk0++wM6dP/H+++/I\nUIe/KCyTl/cLqbBFGeQfLjl7GKNmzXB9ES/rFdTSP2PVI7lKK17UYWEdzqqSC3suo4JPLvGSNafT\nqa+7LlaHhXXwaDvaWkdEXKmzs7OLvaaCbUzLMwRT2DBP7hCS8QllrO5Lut5MN51BV92TezVcreHs\n37Na2+hVq1aVKRZRMZAhEeEvvK3Vzh2yyMrK0j179tMWS30NDXVvHs9LSlFM0RZ1ge7Va0BeQvT2\nXCEhbd1DMM4SJ+xVq1Zpq7WtlyR5sY6I6Ftk8vU21hwS0lZHRFxVomRfUFHDPDBKh9NSf0B3/RMt\n9I28pxVOj2Em778nfa2rFknYwq94OxAhOzv7rMQHnbQiVg+nl95JqN5ISz2QSdpWMzyv2s7OztYJ\nCQm6WbP2ulmz9nrEiBG6Ro2OXit4zwMSPKvi+vVbFprsLJbzizxY4UxF7HBXwGPc486Nik323hSW\nsDvzkF5BqD6MRd/DU7o6J92vd5OGBhrqFHoNzZp1KPPflfA9SdjC7xU+FNBBQwtt4YQewRK9k3Z6\nM131qNDmemVSko6IuNJdHZ+ZYGzcuI2uWTP8rAo+OztbJycn6+HDR+smTS5xT969rI1dlYVVtc2L\n3P1nJNiXNYx0D0nkvtm001C3VMMRZyYbG7kT8Qe6L5/qD7DrQ1j1h/3765joEe5ri9DQPu/1lGqk\njYnds4eHvE3YCvNIwhZ+r+ihgD55Pyuc+hpW6c+5WP+sLPoeztN1OXRWNZ2QkFCiCt5YrbJKe196\n2E5DO68JO7dCb9q0vYYJ2tv4MbQucbL0HFoJ5QUdx2i9hTC9izA9QTXQ/br109nZ2XlJ3Rj68Xw9\nh4b67iSeuxqnk7ZaG8oYdhVTIQkbGAT8AOwFHvRyf6VdoAh8JU3Ynrd34xy9iMv1X9TWixip+5Ku\nwaW9jVcXXsG31XCV+6udR7Jro6GutlrPPyvhZWdn64iIvtpiaaShqYa65R6OSE5O1t1qtNFzmKT/\noL5OY4AewgqtaKVhkK7pMQxU+J/VSx59WHrr0NAWeujQkbK0r4opKmGXaVmfUsoKzAX6A4eADKVU\nstZ6V5mWqghRjPj4YaSmJuBwjCN/y88XAIf7+/y3Z9CDeIZSjxjiWUQik6jFCZbQht27T+ByOrFY\nrbhcLp59dm4hJ7DcCTzr/jkHYwmdBeiOcTpLIm+/vYzo6GgsFgsul4s+fQby5ZeHgQuBY0CLQq/r\nwgubFn3h+/fDsmV0fPpZVpxysYB6dOcrfqal+wGHga2cPLkyb2li4SxccUV3mjRpDEBc3AOye9Hf\nFJbJi/oCegIfefz8EPBQgcdU1huSCALeVnxYrW20MZnWwz1kkVv95q4EKTiU4dLhfKWfoK7+iRB9\n2FZLO++4Q0+/4kpd09KomAq+jYbztTEe7Tlc4sg3YZmcnOweM39fG+Prhe/m9LqkLidH602btJ46\nVeuuXbU+/3ytJ0zQ0yOjtIWXC4lvbN73uUM8hS2NlJPnqz6KqLDL+tbaBDjg8fNB921CVIjczTaL\nF88kJmYrMTFbWb78WXr16gbEAzMxjq/aCjQF4oBojFNSco8be4XtxDKFS7mIvxnmbMSnO3Yw8MvN\n/Ob6m495kGkkcBWfUIv/YVTq84H7gW+BRhhV80RgM/Ad8BQOx7lMmvQQKSkpvPPOcpzOycCH7seF\nAfYCcSQC4TRsqFDZ2bg2bYLnn4ehQ+GCC+DWWyErC2bNwnXwIClDhvBxTg7KOpeCHfSM+PJ30JNj\n1gJXmQ4wUEpdDwzSWt/q/jke6KG1vtPjMboszy1EaXg/FGE5xoe+7Zzp8vc+8BFGK9dtGH3PEmnW\n7BUOHJhEHWLoxfX04Xv6UIvOHOI3YBsXsJ1x7KMV+/iG/RziD97D+JfdC/gLmAwYp97UqXOaQ4cm\nYbxx5J6uo6nLUVryDi2Zz0X8Sjin6YyV1mRxsKaNi0ePwtK3L/TuDU2NYZL8zatuAd4FjgB3u69z\nPsYbwULgdL4DHEpyzJqomnx+4oxSKgKYrrUe5P75YcCltX7G4zF62rRpeb8TGRlJZGRkqV9LiKJ4\nO+qqZs151KmTxT//hOXdBi8CzYA0zjSpPJOwjcTqwkjud2GlCW24kc7UoSPfcxE/cTGbuIj/ow45\nHKMWf5LJP3QhixpkEcppQgi1fE51nUOIbkFt9lOfutTnKKeowX5asJ997Kc6OwjjW0bxPQ3JJJGI\niOZs2LAuX0I13oymepys4wJWYrXeSZ061TlxQpGVdS9gyWt/GghdCINNeno66enpeT8/9thjPk/Y\nIcBu4GqMWY+vgFjtMekoFbaoLN6qycGDB7NmzRrefXclhw4dJiNjN1lZO4Ga7t8yjhS7555rmDMn\nuUCF/j5Ghb6D/BOZPYCZhDCI8xjJ+bThXIYQShZhZFKNbLJJ49zzUjl2IoTfs2wc4QRHmUwW1YA5\nwGmgBsYQy5nnDgnpwooVz+XrZeLtDExDIiNGbCE+fphU0AGoqAq7TKtEtNY5Sqk7MMoVK/CGlhUi\nwiQWi4Xo6OizGjfl3namCu95VkP+adOm8f33+/M16zcWQFmArpxJli8A/wA/k8N8fucLfudKjGER\nTwfpdck+lIJjPx+g9jnnUJtF7N79Ey5XLaA5MJKCq1Fycu7k3XdXlvg8SqWU12sWga3M3fq01muA\nNT6MRYgKUVxDfs/7jGo8k6ys7cBnwErASfXq2Vx0UWN++GEZxjgyGIk9/zJDq/Ultmw5TVbW/YAx\nrh0ebkPr+hiTlLcXGmfBT6SFLWX0drKOCA5yaroQHryNiedW41ZrCElJNYBNwEaM02+2u/8L1arN\nxeX6G6dzH55DLxZLS1yuqRjVegrGWu78J8dDOAkJNzJjxowSxSJj1YHL50MiQgSqoqrx1NRUVqy4\nDaczASMhL8SYpFwJ7KV2bQdHjkzhTLIGCMPlauXxsx2jd3UPchO9sdqjPnv2HCxxLJKsg5NU2EKU\nkMvlonnzDhw6dCfeJgLzrzjxNAH4FGNI5MxqD2MXZSeMHZM/ExPzDUuWLKjQaxBVX1EVtrxNC1FC\nFouFxMSnCQt7hYIbWGy2+YwePQybbd5Z94WGrsdiOcaZjTOvAo8DVwGrgQHYbP8hLi7/BhghCpIK\nW4hSKGpceenSt4iJGXPWfQMHdkRrzZo1m8jKagL8itGX5GFkDbUoyOcbZ0r4onkJOz09PSg2zQTD\ndQbDNULR11nULsLC7gM8zrDU7jMsD6KUMm0NtfxdVk1FJewyt1ct7guP5k/Tpk0rax8UvxIM1xkM\n16h1cFxnMFyj1v53nVRA8ychhBCVTBK2EEL4iQodw66QJxZCiACnK3vSUQghhG/JkIgQQvgJSdhC\nCOEnKi1hK6VmKKW+VUptU0p9opRqVlmvXVmUUs8ppXa5r3OFUqq22TFVBKXUcKXUTqWUUyl1mdnx\n+JJSapBS6gel1F6l1INmx1MRlFILlFK/K6W2mx1LRVFKNVNKfeb+d7pDKTXZ7Jh8oTIr7Ge11p21\n1l2AVcC04n7BD60FOmitOwN7MLayBaLtGAcJfmF2IL6klLJi9EwdBLQHYpVS7cyNqkK8iXGNgSwb\nuEdr3QGIACYFwt9lpSVsrfX/PH6sxZmmwgFDa71Oa+1y/7gZ4zTYgKO1/kFrvcfsOCpAd+BHrfXP\nWutsYAlwrckx+ZzWej3GYZQBS2v9f1rrbe7vTwC7gMbmRlV+ldpeVSn1BHATcBLjXS+QjQMWmx2E\nKJUmwAGPnw9idGwSfkwp1QK4FKOI8ms+TdhKqXVAQy93PaK1TtFaTwGmKKUewjjgbqwvX78yFHeN\n7sdMAU5rrd+r1OB8qCTXGYBkjWuAUUrVwjik8y53pe3XfJqwtdYDSvjQ9zD6Svqd4q5RKTUGGIJx\nQLHfKsXfZSA5hHG0eq5mGFW28ENKqWrAcmCR1nqV2fH4QmWuEmnt8eO1wDeV9dqVRSk1CLgfuFZr\nnVnc4wOE965i/mkL0Fop1UIpVR2IAZJNjkmUgVJKAW8A32utXzA7Hl+ptJ2OSqn3gbaAE9gH3K61\n/qNSXrySKKX2AtWBY+6bNmmtCx4/4veUUkOBl4D6GEeJf6O1HmxuVL6hlBqMcUS6FXhDa/2UySH5\nnFJqMdAPOA/4A3hUa/2muVH5llKqN8Yqpu84M9T1sNb6I/OiKj/Zmi6EEH5CdjoKIYSfkIQthBB+\nQhK2EEL4CUnYQgjhJyRhCyGEn5CELYQQfkISthBC+AlJ2EII4Sf+H2aMYvD5PDojAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e24c0f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tf = PolynomialFeatures(degree=2)\n",
    "X_deg2 = tf.fit_transform(X)\n",
    "print(X_deg2.shape) # Col. 0: constant, Col. 1: X, Col. 2: X^2\n",
    "\n",
    "regressor_deg2 = LinearRegression()\n",
    "regressor_deg2.fit(X_deg2, y)\n",
    "plot_regression(regressor_deg2, X, y, tf=tf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, it seems the model fits pretty well our data. And if we had used higher degree polynomial representation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAD7CAYAAABOi672AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucjHX7wPHPd2bZZRSFcgwVQg4pLUVtUkvzrLJhYp0f\nh9L5eX4q2qLooJ4i2XqSSLawJO22WOppSyUiSoqKlHOJDpZduzPf3x/3zNpds+eZuWd2rvfrtS+7\nc7jv67Zc872v70lprRFCCBH8LGYHIIQQomwkYQshRIiQhC2EECFCErYQQoQISdhCCBEiJGELIUSI\niPDXgZVSMl5QCCEqQGutvD3u1xa21hqtNZMnT87/vip/hcN1hsM1hst1hsM1mnad77+Pjomp0HtL\n4rcWdqC4XC7S09NJTl4OwJAh8djtdiwWqfYIIUxy6BA0aODzw4Z0wna5XPTvP5Q1a7aTlTUWgPT0\nRGJjl7B06RuStIUQ5vBTwg5IRouJifHLcdPT093J+nNgPDCerKwNZGRsIz093S/nLIm/rjOYhMM1\nQnhcZzhcI5h0nX5K2Kq0mkmFD6yU9texPRyOkWxKaU5XLiaDWH6nnvuZJByOzSxePM+v5xdCCK+G\nDoVevWD48HK/VSmFNqPT0d8uOnqET3ieoSxkFxfxITHcwUsoXGaHJoQIZ1LDLmLNGiZv/Jihkeew\nNGcFUWiu5wOmMJnG1fbTIeFVsyMUQoSrgwehYUOfHzY0W9hLl8KQIVRLTcVp747NFk0280jnZwbW\nOMnYiL+xZ2ebHaUQIlxJDdstJweaNIFVq+CKK/KH9b355jsAJCT0w96wIZY+fWDtWujUyfcxCCFE\ncU6dApsNsrPBai3320uqYYdewl68GF59FT74oOTXpaTAhAmwcSOcf77v4xBCCG/27YMrr4QDByr0\n9gp3Oiql5imlDiulthV4bIpSap9Saov7q3eFoqqoV1+FMWNKf93AgTBgADz4oP9jEkIIDz+VQ6D0\nGvZ8oGhC1sDzWuvL3F+r/RKZN7t2wddfQ79+ZXv9o48apZNt20p/rRBC+IJZCVtrvQ445uUpr811\nv5s7F4YNg8jIsr3+7LNh0iSYONG/cQkhhIeJLezi3K2U+kop9ZpSqo5PIypObi7Mnw+jR5fvfbff\nDtu3w0cf+ScuIYQoyE9D+qBi47BfBh53fz8VeA74p7cXTpkyJf/7mJiYyk0RTUuDVq2gTZvyvS8y\nEqZNM2rZ69eDMufmQAgRJg4dgksuKfPLMzMzyczMLNNrSx0lopRqDqRprduX8znfjhLp0wcGDzam\nfJaXywWdOxs17fh438UkhBBF3Xor3HabMeihAnw6NV0pVbCt3w/wf4/ewYOwYQP071+x91ss8PTT\n8Mgj4Of1TYQQYc6sGrZSahHwGdBaKbVXKTUKmK6U+lop9RVwLXC/XyIr6NNPoXt3qFGj4seIjTUG\nsa9d67u4hBCiKD8m7BJr2FrrQV4eDvwSeJ9/DtHRlTuGUnDffTBzJtx4o2/iEkKIgrQOylEigbVh\nA3TtWvnjDB4MmzfDjh2VP5YQQhR1/LjxZ61afjl88Cfs3FzYsgW6dKn8saKiYNw4mDWr8scSQoii\nPEP6/DQaLfgT9rZt0Ly5MQnGF+64w1iP5Ji3+UBCCFEJfiyHQCgkbF/Urwtq2BD+8Q9j1qQQQvhS\n2CdsX9WvC7r3XnjxRcjL8+1xhRDhLewTtq9b2ACXXw4XXADvvefb4wohwltYJ+yjR40ifrt2vj/2\nmDFSFhFC+FZYJ+yNG43WcAV2bShV//7w2WfGYuNCCOELhw75beEnCPaE7Y/6tYfNBg4HvP66f44v\nhAg/Bw+GcQvbH/XrgkaPhtdeMxaHEkKIygrbkojWRknEnwm7c2eoU6f0/SGFEKI0TiccOQLnnee3\nUwRvwv7hB2N6px/rQSglnY9CCN84csRoAFar5rdTBG/C9mf9uqDBgyEjw/jLFkKIivJzOQSCOWFv\n2WKMEPG3OnWgb1944w3/n0sIUXWFdcLetQtatgzMuUaONEaLyOYGQoiK8vOQPgjmhP3jj3DRRYE5\n17XXwl9/wdatgTmfEKLq8fOQPgjWhO1ywU8/wYUXBuZ8FouxV6SURYQQFXXgQJi2sA8eNJZT9dMi\n4F4NGwZvvWWsvy2EEOW1Z4+xFLQfBWfC3rUrcOUQj5Yt4eKLYfXqwJ5XCFE17NkDLVr49RTBm7Av\nvjjw5x02DBYsCPx5hRChTeswbmEHssOxIIfD2FX96NHAn1sIEbqOHTP6wurU8etpgjNhm1ESAeMv\nu3dvYwsxIYQoqwC0rkES9pmGD5eyiBCifCRhm5Swb7wRfv4Zdu405/xCiNATtgn76FFj1at69cw5\nf0SEsb7IwoXmnF8IEXp++ilME7anda2UeTEMHQrJybJOthCibMK2hW1mOcSjUydj0s4nn5gbhxAi\nNEjCNpFSxphsmaouhChNgMZggyTs4g0eDMuXw8mTZkcihAhmR48afV9+HoMNwZiwf/zRnFmORTVp\nYqzHnZZmdiRCiGAWoNY1BGPCDpYWNsgKfkKI0oVtwj550ri9aNzY7EgM8fFGx+Ovv5odiRAiWIVt\nwt69G5o1A6vV7EgMtWpBXBwsWmR2JEKIYBW2CdusVfpKMmyYTKIRQhQvbBO2Wav0laRnT2NDhe3b\nzY5ECBGMwjZhB1OHo4fVCkOGSCtbCHGmAI7BBknYZeOZqu50mh2JECKY/P47VKsGtWsH5HSSsMvi\n0kvh/PPhww/NjkQIEUwC2LqGYErYWsP+/dC0qdmReCdT1YUQRYVtwv77b2N6p81mdiTeDRoEqalw\n/LjZkQghgkXYJuyDB6FBA7OjKN5550GPHsb6IkIIAWGcsA8dCu6EDUZZ5PXXzY5CCBEsJGEHsbg4\n+Ppr45ckhBCSsINYVBTcdpt0PgohjB2pfvoJWrQI2CklYZfXyJFGWUS2DxMivP30k7H37FlnBeyU\nkrDLq3NnYyTLunVmRyKEMNP27dCuXUBPKQm7vJQyWtnz55sdiRDCTGGdsA8ehIYNzY6ibBISYMUK\nGZMtRDgL64QdKi1sMKapX3stLF1qdiRCCLOEbcJ2Oo1FVOrXNzuSshsxQsoiQoQrpxN27oQ2bQJ6\n2uBI2L/9Bueea0xNDxV2u/EL++EHsyMRQgTa7t3GnXatWgE9bXAk7FAqh3hUr24su/raa2ZHIoQI\ntO3boW3bgJ9WEnZljB4NCxZAbq7ZkQghAsmE+jVIwq6cSy4x9qB87z2zIxFCBJIk7BBM2ABjxsCr\nr5odhRAikMI6YYfSGOyi+veHDRtg716zIxFCBEJeHnz/fcBHiEApCVspNU8pdVgpta3AY+cqpdYq\npb5XSq1RStWpdBSh3MKuWdNYEGrePLMjEUIEwq5dRgPThM1WSmthzwd6F3nsIWCt1roV8IH758oJ\n5YQNRllk3jzZpFeIcPDtt6aUQ6CUhK21XgccK/JwX2CB+/sFwC2VjiLEE7arQwf+qF6dJ6/rjcMx\nkrS0NFyymp8QVZNJ9WuoWA37fK31Yff3h4HzKx1FCCdsl8tF//5DSfwlm7brfiUlpQuDBiUyYMAw\nSdpCVEUmJuxKTS3UWmullC7u+SlTpuR/HxMTQ0xMzJkvOnECcnKgdu3KhGKa9PR01qzZjj61lcdo\nRRPi2Jc1ioyMaNLT04mLizM7RCGEL23fDhMm+OxwmZmZZGZmlum1Suti863xAqWaA2la6/bun3cA\nMVrrQ0qphsCHWutLvLxPl3ZswFgE/LrrQnbbLYdjJCkpXYDxzORe/uJsHmUqkITDsZnFi6UzUogq\nIy/P2LDg99+NAQd+oJRCa628PVeRkkgqMNz9/XBgRUUDA0K6HFLUf7md0cylGqfMDkUI4Q8//giN\nG/stWZemtGF9i4DPgNZKqb1KqZHA08ANSqnvgZ7unysulMdgA0OGxGOzvQJks4M2fEcb+rEEm20O\nCQn9zA5PCOFLX38Nl15q2ulLrGFrrQcV81Qvn0UQ4i1su91ObOwSMjKiycoay0tcyL2WO8iLvQW7\n3W52eEIIX1q/Hrp1M+305s90DPGEbbFYWLr0DRYtmobDsZnIAXlcUSeSpVMexGIx/69XCOFDn31m\nasIutdOxwgcua6fj2LFw+eUwbpxf4jDF5MlGp8Ts2WZHIoTwlZMnjV3Sf/vNrzVsX3c6+laIt7C9\nGjsW3noL/vzT7EiEEL6yaZMx/tqkDkeQhO0fjRtDbKxsISZEVWJy/RokYfvPfffBrFmyvogQVcVn\nn8FVV5kagrkJW2s4fNjYG62qiY42ristzexIhBCVpbUkbI4eNZYojIoyNQy/ufdemDnT7CiEEJW1\ne7exj2vTpqaGYW7CrqrlEI9bbzVmRm3ZYnYkQojKCILWNQRDwq6K5RCPatXgrrvghRfMjkQIURmS\nsDHGKterZ2oIfjdmDLz7rvHhJIQITUEwQgTMTtjHjsE555gagt/VrQuDBsGLL5odiRCiIv76C374\nAS67zOxIgqDTsaonbIB//xteeQX+/tvsSIQQ5bVxI3TubHQ6msz8Fva555oaQkBcdBFcfz28+qrZ\nkQghysvk9UMKMj9hh0MLG+CBB2DGDDgla2ULEVI+/hiuvtrsKACzE/bRo+HRwgZjgavWrWHRIrMj\nEUKU1Z9/GiWR6683OxLA7IQdTi1sgAcfhGeeAdmcV4jQsHo1dO8OtWqZHQlQyU14Ky3cEnavXhAZ\nCenpIJvzChFUXC4X6enpJCcvB4zdpP6RmooKov+r5q6H3awZfPQRNG/ulxiC0pIlRi17/XpQXpe8\nFUIEmMvlon//oaxZs52srLEAnF3zv+zL24nthx+wXHBBwGIJ3vWww62FDdC/v1EXe/99syMRQril\np6e7k/XnwHhgPJ1OPMeuPAvpX31ldnj5zEvYubnGDg5nn21aCKawWuGRR+Cxx4wVwIQQpktOXu5u\nWUcBLiCNOO5nhas+zzzzAq4g6XcyL2H/8QfUrh2eZQGHw9hm6MMPzY5ECFGICxgKPEJffiONwXz+\n+T4GDBgWFEnbvIQdjuUQD6sVEhPRjz1GWloaDsdIHI6RpKWlBcU/CiHCzZAh8dhsrwDvANtpxRvY\niOBLniIvbysZGdtIT083O0wTE3Y4jcH2wuVwcHDzlyQNvJ+UlC6kpHQhPn4CF1zQjnfffVcStxAB\nZLfbiY1tj9V6NzCWONaQRhyggCiyssby5pvvmByltLBNk56RwWO55/BAdhM8nRx5eVvZv1/hcNwX\nNLdgQoQDi8XC0qVv0K1bewD6kkoqfU2O6kySsE2SnLyceacmcAH7uI7/uR+NAu4kJ+eaoLkFEyJc\nWCwWHnjgLlrUnE1HvuJ/9HQ/k43NNoeEhH6mxgdSEjFVHlYe5XGeZBJQcMSINWhuwYQIJ3a7ncdb\nRLIiwkoOrwFJ2GzRxMa2x263mx2etLDN4unkWMwt1OQEcaQB2cAcwPxPciHCkQVIOPkXFz79MA7H\nZhyOzSxaNI2lS9/AYjF32gqYOTX92DFo0sS005vN6ORYwsqVV/Jwdnee5HbSqYeLDsD12GzdSEiY\nZnaYQoSXDz5AnXUWPf71L3oE4ZBjc0siYdzC9nRyLF78BFsafcRxjnAb9YFobLZuQXMLJkRYmTMH\nxo0L2vkhUhIxkcVi4eabb+aXvdvRT0zhedsXDO6/KahuwYQIG4cOGUtGJCSYHUmxzCuJSKdjPovF\nwlWTJsFHH/FmzJWykp8QZpg/H269NaiXy5AWdjB55hmYOtVYHEoIETgul7GF37hxZkdSIknYwaRj\nR7jpJnjqKbMjESK8vP8+1KkDV1xhdiQlMm897Bo14PffoWZNv5w/ZB04AO3bw+bN4bVOuBBm0Rpi\nYmDUKBg+3OxognA97Oxs4xakRg1TTh/UGjWCu++GSZPMjkSI8LB2LRw+HNSdjR7mtLAPHoTOnY0/\nxRlcf/9NTvPmTL2sG7vq1mfIkHjsdruMGhHC17SGK6+ECRNg4ECzowGCsYUd5mOwS+Jyueg//Hb+\ndbwm9g++IyXlCgYNSpTFoITwh3ffNTZT6d/f7EjKxJyELR2OxfJsVTTn1A6qcy5DOJusrA2sXLmV\ntLQ0s8MToupwOo3dn6ZNgxC5ezUvYcsYbK88WxW5sHEXs5nOg5zFKbKz7+DOOydKK1sIX1myBGrV\nghCaUSwlkSC2kWhW0YdHeRyAQ4f+kCVXhfCFEyeM1vUTTwTtNHRvpCQSZIYMicdqnYWxch9M5CmG\ns4A2zMLp7CtLrgrhC1OmGJ2NPXuW+tJgYl4LW0oiXtntdho0UEBnIInfWMpUqjOLv4C2JkcnRBWw\naRO88Qa88ILZkZSbtLCDjMViISnpaSIjTwIbgc28RBL1qcfwyOeCYtcLIUJWbi7885/wn//AeeeZ\nHU25SadjEIqLi8NuvxqbbStwOU72c1/USZ5Xv2G/6iqzwxMidD3zDDRuHBKTZLwxZ+LMTTfBnXeG\nVO9soLlcLtLT0/Nr1gkJ/fjHqlUolwv++1+ToxMiBH3zDVx3nVESadbM7GiKVdLEGXMSdrdu8Nxz\nIK3F8vnzT2jbFlJS4OqrzY5GiKDiaeQkJy8HKDxD+O+/oUsXmDgxKNYLKUnwJexLLoEVK4w/Rfks\nXQqPPQZffgnVq5sdjRBBweVy0b//UNas2U5W1lgAbLZXiI1tz9KUBViGDAGbDebONTnS0gVfwj7v\nPNi2Dc4/3y/nrtK0NjY46NoVEhPNjkaIoJCWlsagQY+QlfU5EOV+NBubLZrPhl5Fh/XrYf36kFhw\nrqSEHfgdZ7SGP/6QUSLlVPB2ry41mPHss1jj4lj1yy8sXPg2Bw/uA6w0bNiQoUNvlcWiRFjxzBA+\nnawBorgkK5Zm81+EbV+HRLIuTeATdlaWcSsvt/Nl5u12z1p9PaO6RDPY2pLj2bWAv4A7AVi5MpHY\n2CWyL6QIaw05wHLmMveKbvy7ZUuzw/GJwP9vlmnp5eZZEMq43RsPjGf2qR/4PdfCHdn1gJPA5vzn\nsrI2kJGxTaaxi7AxZEg8NtsreGYI1ySLVOKYVy2SVg/eb25wPhT4hC1jsMvN++1eDUbzEBNYR2v6\nUvRWMCtrrExjF2HDbrcTG9semy0axYskcxU7In5i2z96Yq9Cw4elhR3CfqYuU6jPPJKx4DQ7HCFM\nY7FYWLr0DRYtmsaKS+ZwSf3DnLNkLkuXLaxSZUFzWtiSsMul6O2eIRuYycsM5yS/8QBPFnrOZpsj\n09hFWLFYLMTt3UtfZw5tvv0Ge3x8lUrWYEano5REys243VtCRkZ0gTGmc6hd28KxY+mMONmSzTzG\narLYSlNstjnExravUreCQpTqnXeM5VLXrYN69cyOxi8Cn7ClJFJuntu9wlPVp9GnTx9WrVpFcvJy\n/rv1FMv3zeaRPv1wDJ8mw/pEePnkExg3DlavhgsvNDsav6nwxBml1B6MsWROIFdrfWWR571PnJk0\nyZhx9PDDFTqvKIbW4HBAkybw/PNmRyOET5Q43dzj22+NNUKSk+GGG0yK1Hf8NXFGAzFa66Pletex\nY9C0aSVOK7xSCl5+GTp2NBbX6tXL7IiEqBRv8w/S04vMMdi7F3r3NtYmqgLJujSVvWcu/946x45B\nnTqVPK3wqm5deP11Y3Gb334zOxohKuX0/IPPgKbAF2RldSQ9/VNjQ+qjR9GxsWy/4QYcaR/gcIwk\nLS2tSu97WpmSyG7gT4ySyCta61eLPO+9JCJLq/rfQw8Za7W8915I7VcnREEOx0hSUi4H1gPbgbHu\nZ2ZzUUMnG85WpOz/lbtPnofTeQ9QYMGnEJ7l66+SyNVa64NKqfrAWqXUDq31uoIvmDJlSv73MTEx\nxMTEGMscnnVWJU4rSjV1KnTvDrNmwb33mh2NEJWw3f11elEnK8OYcfB8Vh5ycadugebL/OeyskaR\nkRFNeno6cXFxZgVdLpmZmWRmZpbptT5ZrU8pNRk4rrV+rsBj3lvYnTrB/Plw2WWVPq8owa5dxop+\na9bI37UISWlpafTrNw6nMxFj2QUAzauMoQmfEQfkcVeB5zyScDg2s3jxvIDG6ysltbArdM+glKqp\nlDrL/b0NuBHYVqY3Sws7MC66yNhk1OGAv/4yOxohys3YkLp2occm8xid2Ep/RpNXgS60UFfRIs/5\nwDql1FZgA/Ce1npNmd4pCTtwBg82hjuNGWMM+xMihHg2pI6KegnIZgxzGMpC7LxNFnOBeODMGcBV\neZZv4DcwqFEDfv8datb0y3lFEdnZxpZsY8bA+KK3jkIEN5fLxYABw1Dpn/JizlGu4T72Rb1Nbu7v\nOJ0/ALdj3NwbHZJRUS9z002dqmynY2ATdl4eREUZW83L6IXA+fFHY//MlSvhiivMjkaIcnF98QW5\nvXoxtUsPfqx3HoMG3cyCBSmsWfMNWVmjge1Yrak0aFCHpKSniIuLC9lkDcGUsI8dgxYtjB1nRGAt\nWwYPPGDsGC1ruYhQ8fPPRmNj9mzod7rM4ZkBeXqphn5VZjmG4EnYv/xi7Pa9d69fzilKpu+/n98+\n/ph7LrwUbbF4n+YrRLD44w8jX4wdG1bDU30+SqTCjh+XDkeTuFwuBv50iB1bd9J22XZSUn6kX79x\ndO/ek7y8PLPDE6Kw3Fy49VZjiYUwStalCWzClhEipklPT2fl2m8Z4OrFCL4mjnY4nYmsX7+fHj1i\nq/R0XhFitDY6yGvWlIXMigh8wq5VK6CnFIbk5OWcOHEVv7KHAbzPXJbTiuuBbWzatF/2fxTB47nn\n4Isv4K23wGo1O5qgIi3ssPINMJaNXMMknuRdbuZscsjLu1v2fxTBYcUKmDkT0tIkV3ghCTtMDBkS\nj9W6K//n1xjNGm5kEbdh4Ss++2xDWKx2JoLYli3GfIEVK2QJ5mIEdpRIUhJs3w4vveSXc4riuVwu\nunfvyfr1+zEmGkRh5RSruYAvOcWDTAOqxmpnIgQdOgTR0fCf/8CAAWZHY6rgGSUiLWzTWCwWPv74\nfbp2vYCIiE5AEk7uwYGNeM5hKLWA8WRlbSAjY5vUtEXgZGfDLbfAqFFhn6xLIwk7jERERPDpp2tZ\nvvxZundPA5ZylH/TlzT+w/9xFZ8CUWRljZWatggMrWH0aGjWDB591Oxogp4k7DBjsViIi4ujUaOG\nQFsAvqMtQ1nIMvpzIbu8vs/lcpGWlobDMVJq3cJ3pk+HHTuMJZdluYpSBXbXdEnYQeZSjNXORrGG\nWB7nUdK5iR5WFwkJp8e/lmlvPSFKUXRD3X+3akKX+fNRGzbIYnBlJAk7TA0ZEs977z3MiRPtgGhg\nLP8FWnKYVTZF5wIbmp7eW+/0rh+huLOHME/RD/22HKAFTzExphdPNmwY4Fv90CUlkTBlt9vp3bsD\nNWtuB7oBi7Fan2B51450jrkGy9ix+WtoJycvd7esowocQWrdomxcLheTJ08mNXUzWVmfcy4OUlnE\nv5jD7C8OSgd3OUgLO0xZLBaWLn2jwIpnLUlIeAC73Q4nTnD0iiv4qG1HFne4nAMH9gKXmx2yCEF5\neXn06HE9n3/+DdCWCFaRwmyWE08yoyDrJG+++Y7cpZVRYBP28eMyNT2IeDogC/5ncblc9B82js17\nI1hz4jDn7jhJatQhrNYncDpHAZ5ao2dnj2mmxC6Cm6eTetSoezh6NBKYCsDzjCGHSB5itbkBhihp\nYYtC8uvVJzbRhwOsowcHs2fwnvURIiPbkZPzfwDYbHOIjW1vtMiFKMBTr05P/4xTpyKBr4EoxjCH\nG6hLNNVwsRq4QT70y0kStiikYL36Jy4kjjRW05uBzkE4u+2kcePNAAwa9Lj7z38CyNraIl96ejoZ\nGd+4k/U9QBQ9+JipPEIP1vEXa4FniIiYQGzsFfKhXw6SsEWJttCZgaSQQhwzalzHU4vnyTA/USLP\nypCQCkAz9rAEB0NZyA+0AtYC3/LQQ+N57LHH5N9LeWit/fJlHLqAnBytIyK0drm0CF6pqanaZuug\n4aQ2holoDSf1gGpN9clzztF6xw73azqe8RqbrYNOTU01+xKEyQYOHKGhu4bbtY12+isu1fcwM//f\nCVyku3W7VjudTrNDDUru3Ok1rwbuo83TupbZTEHNbrdz442XYrV2BJLcX51Z7splbotW6NhYVr/y\nhgzzE8XyrAypaEMyf7KBA8zCiuff0rnn5vHxx+9Ly7oCAp+wRVCzWCwMHz6QiIhTwEZgMzAdp3MX\nD+08yfYbbuCRjzJoiPeNlGWZVmG327nyylZMYwrn0ow7eRX4EthIZORJ5s17gYiIwFZjqwpJ2OIM\nb721gpycCcACYB4QB9QkK2ss0/52ciy+Hx+qx6nPLwXelQ3MZO/ea0hJ6cKgQYkMGDBMknYYslgs\nrBs3kuGRJ3FYD5HLQeBybLat2O1Xy5jrSpCELcqt9fz5fNW6FR9YWnMu0zFudS8FmgAvAE3JyurI\nu+/+j8mTJ0vSDjeffIJ1wgQabtzAnHdm4HBsxuHYzKJF06RTupICt4FBRoaxV9uaNX45n/CdtLQ0\nBg1KJCtrA6fr1NnYbNEsWjSNuLg48nJzeav5RbQ/cJBeXMFROgGfADkYk2uM0SMREbPo2/cK+Y8a\nLnbvhquvhtdfh9hYs6MJScGxgYG0sEOG3W4nNrY9Nls0no5Hmy260ESZVatXM/6Pc1jNfXzASeoy\nFZjiPsLnwHhgPHl5W2VDhHDxxx9gt8Mjj0iy9pPAtbDnz4ePPjI+eUXQ8yyF6Rn1kZDQr9DEGIdj\nJCkpXYA7mEYicaRxPe04Qg+MZF1QEk2bvkS3blfKBJuq6tQpI1m3bQsvvGB2NCGtpBZ24Lpq//5b\n1hEJId7WGfFOkcjjuPiRTN6hF7s5RFPATsEbuL1767J3bxeZYFMVaW1snlujhlH2FH4TuP8xx49L\nSaQKMcbazgJOAMN4lJ0kE8s6dtOMB4BhgAtj9MgcYAKyZ2QVNXkyfPcdLFoEMlzPr6SGLSrEbrfT\noIECWgNbgc95mneZyWQ+5jitWY9RGukMtMdocYNMsAk9JW4PN3cuvPkmvPce2GzmBhoGAlsSadQo\nYKcT/mWVdk4uAAATs0lEQVSxWEhKepr4+NG4XBPxjCZJ4i6OU4sPuZf+kcv4LGcARsellD9CUYnr\nxoxwYElMhI8/hvPOMznS8CAtbFFhcXFxNGxY74zHFzCCO3CQ7vyLvpEZwCn3My7gbazWJ9i/f7/M\nhgwBnpX3srIeAb4AviAr61H+Sl9P7tCh8O670KqV2WGGDUnYosI8reyoqJcwatUe2bxv28C2qY+z\ngIP8s3or4EXgamAiTmcin3xyc7GzIWWH9uCxcOHbnDhRE2MDgi5AFzowkeScvTzbMRqio02OMMwU\ntypUZb8oulpfr15aZ2T4fGUrYS6n06nj4xPcK/zN1jBb22wddHx8gnY6ndq5bZvOql9f/7fZRTrC\n2qrUFf6cTqfu12+wezXAJA1JhY4n/MvpdOrU1FQ9cOAIPXDgCN2mTQcNbfN/bxfyo95HQ92fxrp7\n9+vNDrdKooTV+gJbw5YWdpVz5t6QkJAw7fRY60svpeaWLdzQviNznC0Zh4Xc/Hef7oD0DB+UHdoD\nxzPWPjl5OQCDB9/C668vYe3ab/Pr1RbLauARIIoL+Jn36cVjTGEZuXQnzbzgw5QkbFFppY7ZbtyY\nKdf15pbl37CWG4hnOUep6/Wlpe3QLgnbN7x1JqamPkxu7u84nbvw7N3pcn0OQGP28T96MoP7eZWx\nQBKNGjU0KfrwJTVsERADRjgYXtPFerqwgWja8Q2QTVTUywwadLPZ4YWdwncyxjIC2dmbcDrPAT4o\n8MoBNGQG/+M6XmI8L3IPxu/tJYYMiTcl9nAmCVsEhN1u54bY9iRa05hMdz6kGwO4mNzc31mwICW/\nU3HIkHhstlco2olpbNbaz5TYq6Li7mTgTuD0GPlGdOJ/7GM+f/A8UXg2Iahb1yV7MZogMAlba5ma\nHuYKbozwFpob6cl0cnnSeRsfFJj5WJaFp4S/OQEXzXmVj2nL60TwND2BTRgbWlzD1Vd3laUFTBCY\nxZ9OnoQ6dSAnxy/nEqHh9IJRxuJQdTnCWwwmkp9YHNeZl1OXAKUvPFXwNZ4OM1lUqnyKW0LXau2I\n1ZpDi1N1WMN2phPPS1wLvIIxY3UONlu3/GV2he+VtPhTYIb1HT6sdd26fhsGI0KDsTlrUoFhfVpb\nyNOT+Ic+GlVD65Ury3QcGfpXecUNx+zXb7Ce/c9/6v1Y9XDmFBqCCW10ZGRz+Xv2M0oY1heYFvbu\n3XD99fDTT345lwgNJbXqEq9pzF2fb2B90xZYnnqcPrfckt9aLtqabtWqCTNmpBUa+ld0gwVROq93\nMhYLx2+9lZE5Q1nOq0XekUT37ml89NFKuZPxI/Nb2Fu3at2+vd8+kURo8Naqg0s0NNLQUtdlul5G\nJ/2NitL397zJmHhzRmv6RW2xnKOhu4YRGlI1ON2twNna4Rhp9mWGrjlztD7/fJ3Y86Yz7oTk7zdw\nKKGFHZiPSRkhIjg9yeb++/titT6JsSt7f6A+8DW/8wD9+ZLpejaT/pfBzhEjWJmaWmD42e3Aelyu\n84BBGFOlEzm9lKuoEKcTJk6E6dNh3TquvO92GakTpAIzcUYStnCzWCx8//0+nM6HMTofR2Ls/+gp\nbSgW8k8+4gDJKdNptngprXLtbKE6kA5sx1jO1fP6UUA08I47oUwrdD7pnCzFsWMweLAxMGD9eqhf\nH/tFFxEbu4SMjOj8STU22xwZqRMEJGGLoPQL53JNzmUMowWreIuFXMpkLuME3sYOj8VqvZvY2J6F\nEkqJS4NWgR1vyvphVOzrvvkG+vWDvn3hmWegWjWgDMsNCPMUVyup7BcFa9hz52o9UmpfwpCamuqu\nY59016A93xcckdDB/ZzW9flZJ1NH/0yUHsgoDa4zaqtt215xxsgF4zwdvS44lZiYmL/AUWpqasiN\neihupEy/foP1ihUr8q9txYoV+pZbBhV+Xc32OqlztHbVq6d1crLZlyKKoIQadmAS9owZWt9zj18v\nUoSOwp2PszR01XBxgY7INhoSCnQmGkm5B830FiJ1Jt11B7YWGm7mbeU4b8MIPceyWhuF7JBAp9Op\nExMT3dcwtEDH60lttbbSkZHN868tKqqdtlobaMjS4NTnkqyXcYHeSoSeNW5cyFxzOCkpYUtJRARc\n0VturS+hVasmfP/9Jtav38jevdfibZeaL6KyuTy7HmPYxRp6sIZLeJQj7CEXrTUul6vMt+xOZy88\nE3jKshpgsNTCPWWe1NTN7n4AMDpelwBv4HTeg9O5Ec+1ZWePwtimbS03MpO5fMpSejCYf5H3ahIf\n/jqUZcsWSqkjVBSXySv7RcEW9oQJWj/1lD8/lEQVUbhccmYZo2bN9hqW6rNoqSdTSx8hUr/AtbpF\nZKszWsnFHctowaeWecia0+nUt9wySEdFtctvuVosLXXXrtfp3NzcUq+p6BrTlSnBFFfmOV1Cmq1h\nZKFrq89TOpnz9W6q6168V+h9VmsrvWLFigrFIvwD00sit9+u9ezZfr1IUTWUtCFCTk6O7tbtWm2x\n1NPQQMMyXZ+D+nnu1b9TS79MDX3r5d3zE6K3Y0VEtHaXYJxlTtgrVqzQVmtrL0nyIt216zUlJl9v\nteaIiNa6a9eeZUr2RZVU5oFhhT6MLOTpUczVhzhLT+dcXZPnvb6ve/fYcsch/Mf8hJ2QoPWCBX69\nSFF1eFqkDsdI7XCM1KmpqTo3N/eMxGe0KgdrGKTr01Y/wY36CDa9yFpHT4jprZ1Op87NzdWJiYm6\nadO2umnTtnrgwIG6Ro1Lvbbgi+5842kV16vXotgkabHUL/S+ok63iLPciXSEu+7csNRk701JCdti\naZBfr76RlXorzfUn1NcdOUdDnWLf17Rpuwr/roTvmZ+w+/bVevlyv16kqNqKLwW009A8//HaHNP/\n5km9R1XTR1q31okXt9U1LC11wQ7GRo1auUsrhVvwubm5OjU1VQ8YMFw3bnyJu/PuRW3MqiyuVXtB\nibP/jAT7ovuDpeCHTRsN55SrHHG6s7GhLtrZGBHRWk+aNEk/eO2N+n1LLb2TKH0LTdwxJmmlGmqj\nY/fM8pBs9RVczE/Y112n9dq1fr1IUbWVXAroccbjVl7Qg6PO1u9TQ/9KPT2Te9wjS07Xw8vWgk/Q\nsEJ7H3rYRkMbrwnb00Jv0qSthtvdybro+1uWOVl6K60Y5++qrZaW+p5LOmrntddqV7Nm+t0+fXTU\nGftnZmmop439GT2jcTpoq7WB1LCDTEkJu8KjRJRSvYGZgBWYq7WeXuyLjx+XUSIioJxYeSvbyls8\nRXP6MILXSSOOP6lNSlYzjn/5LYvT385/fVpa2hl7SRqzKDsBB4F6GKMt7nQ/Nwv4Das14owdc/Ly\n8ujR43o2bvwBl8uKMYJjGmdO+LmXn39+uUzX422vy9rEk0AHxrqyqbbzFLNrdeaunTt5c9jtZDv/\nUeR8NYFHsVgm43ItBiAy8i9uuqmnLJYVQio0lkcpZQVmA72BtsAgpVSbYt8gw/pEJRW3E43RZvix\nmMcvB2APLZjCYzRnD3fwMvU5TuLa99AdOsDEibjWreM/018sZgeWu93H/xH4E2MI3aPAhcAU4JxC\nO+a4XC569LiRzz8/gMvVDCNRtiv2upo1a1Km6/fsEFMNC7GsZj4j2MMlXENT/sVVXKqPMum7LNLX\nrCnhKBauuupKHI6WOBwtWbp0lgzpCzXFNb1L+gK6AasL/PwQ8FCR15xu4zdurPUvv/j7TkJUYd5G\nfFitrdydadHukoXnVt8zEqS4UkZLXaN6Mz3x2hu186GH9O7adfQRlH6bjvpuXtDt+UqrAisAGiWX\nVhrqu+vRBcslWYU6LFNTU90jSpa56+vFz+Ys85C6I0f0c91i9Hyi9RHO1Z/STd/Pc7o+h3XhYXyz\n80s8xQ2NLKmDVAQH/FASaQzsLfDzPowVeLyTFraoJG/rWwwa9AzPPpvEp5/2BZpxei/CJsAtQByw\nFOOf5lj3czOAZpw8lcasTd2w9ohgRl4zzuZOYphKDF9xF7M5n8Ns5jK+4Fs2MZLt3MYPDCePZu7j\njgKuBJ4iK+ts7rzzIQAWLnwbp/Me4D2MyStRgB2jLFIwjpk0aGAFKDzhR2vYtQs2bYIvvkBnZpL3\n3Xd0i6rJEmUlUW9gPxe7j5ENzMEot5xmbLMmizdVRRXawEApdSvQW2s9xv3zECBaa313gdcYHxZa\nQ0SEsT1YRGAmVorw4X1ThLcxbvq2Qf4qf8uA1RhLuW7FWPcsiaZNX2Lv3jsxlm4d5n7PWOpynCt4\nhS6cRWda0I5vacoedmNjFz34iebsIZV95PIrA/mVszhZ820i6uSx68BdwJcYy7+Od8fkAt4jiqc5\ni52cRzYNqE8j9Qfdm5zD6GuuxrJrF+zYAWefDZdfjqtzZxLXfMTLmw/zx4mxwJvAEeA+9zHnYGzb\n9QZwqtAGDmXZZk0Ep5I2MKhoBt0PNC3wc1OMVnYhU6ZMgVOnQCliPvmEmJiYCp5OCO+8tSZr1nyF\nOnUs/Pnn6cdgPXApkIH3f/YWjMSXDtzL7zQhgxfIwI6nqyeKGbTiIy5kOM1Jpzl/050e1Gcz5/Er\n9U/8Rq0Tx4jgHk4QRS4LMWreEEEeNrI4heY4Fn4lgkNEcki35Je9P/Lk+s1Mmv8KljZtoH59ANLT\n0pj19DKyTmzE+DAaD7yD1Xo3depU5/hxRU5ONPDyGS1oi8VCXFycdCiGgMzMTDIzM8v02oq2sCOA\nncD1wAGMlegHaa2/K/Aao4X922/Qpg0cOVLu8whRFt5ak3369GHVqlW8+eY77N9/gC++2ElOznaM\nTkDwbCl2//19mTEjtUgLfRlGC/0bCm5BZpQ0pmGURIYBXTndgvZI4oJGszl1zELOyZrAUWAcTixk\n8TJOcoEawFeFjh0R0Ynly58tlGCLblpc8BwDB25iyJB4aUFXQT5vYWut85RSd2E0V6zAawWTdSEn\nTkDNml6fEsIXimtNeh5zuVwMGDCMjIxuZ9R0J0+ezLff/lSohW4MgLJgjDLxJMuZGKNE9mAsTJWB\nkbDP1LTFBagLYc+evZx1Vi0ghZ07d+Ny1QIuAAZTdDRKXt7dvPnmO2VuESulpAUdhipcVNZarwJW\nlfrCkyclYQtTlbYgf8HnjNZ4Njk524APMToynVSvnsuFFzZix44UjDoyGIl9FIU3FJ7Fpk2nyMmZ\nAIDN9grt29vQuh7wNXBHsXEWvdsdMiSe9PREsrIKn8PbzjoiPPh/1/Qvv4TRo40/hQhyp1vj285o\njVutESxdWgOjHv4ZxoiPbXhGflSrNhuX6w+czl0ULL1YLC1wuR7BaK2nYdS1C+8cD+1JTLyNqVOn\nlimWqrBjjvDOH52OZSclERFCSmqNp6ens3z5OJzORIyE7OmkfAf4gdq1szhy5GFOJ2uAKFyuiwv8\n7G2I3xygHt9/X7jfXrbqEkUFJmHXqOH30wjhK8XVxO12Ow0a1Gb//vxXYnRAxgFJ1KjxEt4nD7fD\nqIF7ShtvYCT5u4EOGB2Ze1BqS5ljEeHJ/x/T0sIWVYTFYiEp6Wmiol6i6FR4m20Ow4d7nz4fGbkO\ni+UoRqs6CXgZeBzoCawEbsBmm0tCQr8AXYkIVf5P2NLpKKqQuLg4brqpEzabJ/kmYbNF5484iY1t\nf8ZzN93Uib59byQy8i9gETAd+A0jgb+c/36ZhShK4/9Ox9deI/Ptt4lZudIv5wkmmZmZVX5yUDhc\nI5R8nSXNIizuOaDAHpbavYflPpRSpo2hlt9lcDK90zHzyBFi/H4i84XaP4yKCIdrhJKvs6S6cknP\nBVstWn6XoScwNexq1fx+GiGEqOokYQshRIjwaw3bLwcWQogqrrgatt8SthBCCN+S6VJCCBEiJGEL\nIUSICFjCVkpNVUp9pZTaqpT6QCnVtPR3hRal1LNKqe/c17lcKVXb7Jj8QSk1QCm1XSnlVEp1Njse\nX1JK9VZK7VBK/aCUetDsePxBKTVPKXVYKbXN7Fj8RSnVVCn1ofvf6TdKqXvMjskXAtnCfkZr3VFr\n3QlYAUwO4LkDZQ3QTmvdEfgemGhyPP6yDegHfGx2IL6klLJirJnaG2gLDFJKtTE3Kr+Yj3GNVVku\ncL/Wuh3GwuV3VoXfZcASttb67wI/1uL0osJVhtZ6rdba5f5xA8ZusFWO1nqH1vp7s+PwgyuBH7XW\ne7TWucBi4GaTY/I5rfU64JjZcfiT1vqQ1nqr+/vjwHdAI3OjqryA7oqrlHoCGAqcoLjtOqqOURgL\nR4jQ0RjYW+DnfRgLfogQppRqDlyG0YgKaT5N2EqptUADL09N0lqnaa0fBh5WSj0EzABG+vL8gVDa\nNbpf8zBwSmv9VkCD86GyXGcVJGNcqxilVC2MTTrvdbe0Q5pPE7bW+oYyvvQtjHUlQ05p16iUGgHc\nhLFBccgqx++yKtkPFOwMb4rRyhYhSClVDXgbSNZarzA7Hl8I5CiRlgV+vBk4c7X2EKeU6g1MAG7W\nWmeX9voqwuuMrBC1CWiplGqulKoOOIBUk2MSFaCUUsBrwLda65lmx+MrAZvpqJRaBrQGnMAu4A6t\n9a8BOXmAKKV+AKoDR90Prddajy/hLSFJKdUPmAXUw9hKfIvWuo+5UfmGUqoPxvYwVuA1rfVTJofk\nc0qpRcC1QF3gV+BRrfV8c6PyLaVUd4xRTF9zutQ1UWu92ryoKk+mpgshRIiQmY5CCBEiJGELIUSI\nkIQthBAhQhK2EEKECEnYQggRIiRhCyFEiJCELYQQIUISthBChIj/B+4i8xVsHYdzAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e2b9cc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tf = PolynomialFeatures(degree=10)\n",
    "X_deg10 = tf.fit_transform(X)\n",
    "\n",
    "regressor_deg10 = LinearRegression()\n",
    "regressor_deg10.fit(X_deg10, y)\n",
    "plot_regression(regressor_deg10, X, y, tf=tf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, it is quite clear that we are overfitting the data. \n",
    "\n",
    "To see this, we can have a look at the $R^2$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.98683319423\n",
      "0.988025974709\n"
     ]
    }
   ],
   "source": [
    "print(regressor_deg2.score(X_deg2, y))\n",
    "print(regressor_deg10.score(X_deg10, y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the $R^2$ is not able to show that degree 10 polynomial regression is overfitting.\n",
    "\n",
    "To better assess whether a fitted model tends to underfit or overfit the data, it is necessary to split our data into a train set and a validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 -0.150174605536\n",
      "2 0.989327576712\n",
      "3 0.98875729718\n",
      "4 0.989462491585\n",
      "5 0.980318724099\n",
      "6 0.913716903047\n",
      "7 0.799959199313\n",
      "8 0.255051844284\n",
      "9 0.444334215095\n",
      "10 -6.52795449077\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=0)\n",
    "\n",
    "for degree in range(1, 11):\n",
    "    tf = PolynomialFeatures(degree=degree)\n",
    "    X_train_poly = tf.fit_transform(X_train)\n",
    "    X_test_poly = tf.fit_transform(X_test)\n",
    "\n",
    "    regressor = LinearRegression()\n",
    "    regressor.fit(X_train_poly, y_train)\n",
    "    print(degree, regressor.score(X_test_poly, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get more reliable results, it is recommended to use $k$-fold cross validation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 -0.0473348727\n",
      "2 0.97998648803\n",
      "3 0.979919207038\n",
      "4 0.980795367286\n",
      "5 0.966331415499\n",
      "6 0.951232294194\n",
      "7 0.640549302997\n",
      "8 -2.36809778361\n",
      "9 -0.837808309871\n",
      "10 -46.5247428935\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score, KFold\n",
    "\n",
    "for degree in range(1, 11):\n",
    "    tf = PolynomialFeatures(degree=degree)\n",
    "    X_poly = tf.fit_transform(X)\n",
    "    scores = cross_val_score(LinearRegression(), X_poly, y, cv=KFold(n_splits=3))\n",
    "    print(degree, scores.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This indicates that we should pick a degree between 2 and 6.\n",
    "\n",
    "## Parameter tuning\n",
    "\n",
    "We now turn our focus on setting adequate values for hyperparameters in diverse machine learning problems. In the following, we will consider the case of Support Vector Machines for Classification, but what we will see can be straight-forwardly extended to other `sklearn` models.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVC(C=0.01, cache_size=200, class_weight=None, coef0=0.0,\n",
      "  decision_function_shape=None, degree=3, gamma='auto', kernel='linear',\n",
      "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
      "  tol=0.001, verbose=False)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.datasets import make_blobs, make_circles\n",
    "\n",
    "X, y = make_blobs(n_samples=50, n_features=2, centers=2, cluster_std=1.3, random_state=0)\n",
    "dict_params = {\"C\": [.01, .1, 1., 10.]}\n",
    "clf = GridSearchCV(estimator=SVC(kernel=\"linear\"), param_grid=dict_params, cv=KFold(n_splits=3))\n",
    "clf.fit(X, y)\n",
    "\n",
    "print(clf.best_estimator_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The value $C=0.01$ seems the best when using cross-validation on the training set. A similar scheme can also be used to pick the adequate kernel for SVM:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
      "  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',\n",
      "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
      "  tol=0.001, verbose=False)\n"
     ]
    }
   ],
   "source": [
    "X, y = make_circles(n_samples=100, random_state=0, noise=.1, factor=.5)\n",
    "dict_params = {\"C\": [.01, .1, 1., 10.], \"kernel\": [\"linear\", \"rbf\"]}\n",
    "clf = GridSearchCV(estimator=SVC(kernel=\"linear\"), param_grid=dict_params, cv=KFold(n_splits=3))\n",
    "clf.fit(X, y)\n",
    "\n",
    "print(clf.best_estimator_)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}