{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "***\n",
    "***\n",
    "# 计算传播与机器学习\n",
    "\n",
    "***\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](./img/machine.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 1、 监督式学习\n",
    "\n",
    "工作机制：\n",
    "- 这个算法由一个目标变量或结果变量（或因变量）组成。\n",
    "- 这些变量由已知的一系列预示变量（自变量）预测而来。\n",
    "- 利用这一系列变量，我们生成一个将输入值映射到期望输出值的函数。\n",
    "- 这个训练过程会一直持续，直到模型在训练数据上获得期望的精确度。\n",
    "- 监督式学习的例子有：回归、决策树、随机森林、K – 近邻算法、逻辑回归等。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 2、非监督式学习\n",
    "\n",
    "工作机制：\n",
    "- 在这个算法中，没有任何目标变量或结果变量要预测或估计。\n",
    "- 这个算法用在不同的组内聚类分析。\n",
    "- 这种分析方式被广泛地用来细分客户，根据干预的方式分为不同的用户组。\n",
    "- 非监督式学习的例子有：关联算法和 K–均值算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 3、强化学习\n",
    "\n",
    "工作机制：\n",
    "- 这个算法训练机器进行决策。\n",
    "- 它是这样工作的：机器被放在一个能让它通过反复试错来训练自己的环境中。\n",
    "- 机器从过去的经验中进行学习，并且尝试利用了解最透彻的知识作出精确的商业判断。 \n",
    "- 强化学习的例子有马尔可夫决策过程。alphago\n",
    "\n",
    "> Chess. Here, the agent decides upon a series of moves depending on the state of the board (the environment), and the\n",
    "reward can be defined as win or lose at the end of the game:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src = './img/mlprocess.png' width = 800>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 线性回归\n",
    "- 逻辑回归\n",
    "- 决策树\n",
    "- SVM\n",
    "- 朴素贝叶斯\n",
    "---\n",
    "- K最近邻算法\n",
    "- K均值算法\n",
    "- 随机森林算法\n",
    "- 降维算法\n",
    "- Gradient Boost 和 Adaboost 算法\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn做线性回归\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# 线性回归\n",
    "- 通常用于估计连续性变量的实际数值（房价、呼叫次数、总销售额等）。\n",
    "- 通过拟合最佳直线来建立自变量X和因变量Y的关系。\n",
    "- 这条最佳直线叫做回归线，并且用 $Y= \\beta *X + C$ 这条线性等式来表示。\n",
    "- 系数 a 和 b 可以通过最小二乘法获得"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.17.1\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "from sklearn import datasets\n",
    "from sklearn import linear_model\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.preprocessing import scale\n",
    "\n",
    "import sklearn\n",
    "print sklearn.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# boston data\n",
    "boston = datasets.load_boston()\n",
    "y = boston.target\n",
    "X = boston.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'__class__ __cmp__ __contains__ __delattr__ __delitem__ __dict__ __doc__ __eq__ __format__ __ge__ __getattr__ __getattribute__ __getitem__ __gt__ __hash__ __init__ __iter__ __le__ __len__ __lt__ __module__ __ne__ __new__ __reduce__ __reduce_ex__ __repr__ __setattr__ __setitem__ __setstate__ __sizeof__ __str__ __subclasshook__ __weakref__ clear copy fromkeys get has_key items iteritems iterkeys itervalues keys pop popitem setdefault update values viewitems viewkeys viewvalues'"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "' '.join(dir(boston))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD',\n",
       "       'TAX', 'PTRATIO', 'B', 'LSTAT'], \n",
       "      dtype='|S7')"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston['feature_names']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:          boston.target   R-squared:                       0.741\n",
      "Model:                            OLS   Adj. R-squared:                  0.734\n",
      "Method:                 Least Squares   F-statistic:                     108.1\n",
      "Date:                Sun, 08 May 2016   Prob (F-statistic):          6.95e-135\n",
      "Time:                        20:14:27   Log-Likelihood:                -1498.8\n",
      "No. Observations:                 506   AIC:                             3026.\n",
      "Df Residuals:                     492   BIC:                             3085.\n",
      "Df Model:                          13                                         \n",
      "Covariance Type:            nonrobust                                         \n",
      "===================================================================================\n",
      "                      coef    std err          t      P>|t|      [95.0% Conf. Int.]\n",
      "-----------------------------------------------------------------------------------\n",
      "Intercept          36.4911      5.104      7.149      0.000        26.462    46.520\n",
      "boston.data[0]     -0.1072      0.033     -3.276      0.001        -0.171    -0.043\n",
      "boston.data[1]      0.0464      0.014      3.380      0.001         0.019     0.073\n",
      "boston.data[2]      0.0209      0.061      0.339      0.735        -0.100     0.142\n",
      "boston.data[3]      2.6886      0.862      3.120      0.002         0.996     4.381\n",
      "boston.data[4]    -17.7958      3.821     -4.658      0.000       -25.302   -10.289\n",
      "boston.data[5]      3.8048      0.418      9.102      0.000         2.983     4.626\n",
      "boston.data[6]      0.0008      0.013      0.057      0.955        -0.025     0.027\n",
      "boston.data[7]     -1.4758      0.199     -7.398      0.000        -1.868    -1.084\n",
      "boston.data[8]      0.3057      0.066      4.608      0.000         0.175     0.436\n",
      "boston.data[9]     -0.0123      0.004     -3.278      0.001        -0.020    -0.005\n",
      "boston.data[10]    -0.9535      0.131     -7.287      0.000        -1.211    -0.696\n",
      "boston.data[11]     0.0094      0.003      3.500      0.001         0.004     0.015\n",
      "boston.data[12]    -0.5255      0.051    -10.366      0.000        -0.625    -0.426\n",
      "==============================================================================\n",
      "Omnibus:                      178.029   Durbin-Watson:                   1.078\n",
      "Prob(Omnibus):                  0.000   Jarque-Bera (JB):              782.015\n",
      "Skew:                           1.521   Prob(JB):                    1.54e-170\n",
      "Kurtosis:                       8.276   Cond. No.                     1.51e+04\n",
      "==============================================================================\n",
      "\n",
      "Warnings:\n",
      "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
      "[2] The condition number is large, 1.51e+04. This might indicate that there are\n",
      "strong multicollinearity or other numerical problems.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import statsmodels.api as sm\n",
    "import statsmodels.formula.api as smf\n",
    "\n",
    "# Fit regression model (using the natural log of one of the regressors)\n",
    "results = smf.ols('boston.target ~ boston.data', data=boston).fit()\n",
    "\n",
    "print results.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "lm = regr.fit(boston.data, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(36.491103280361614,\n",
       " array([ -1.07170557e-01,   4.63952195e-02,   2.08602395e-02,\n",
       "          2.68856140e+00,  -1.77957587e+01,   3.80475246e+00,\n",
       "          7.51061703e-04,  -1.47575880e+00,   3.05655038e-01,\n",
       "         -1.23293463e-02,  -9.53463555e-01,   9.39251272e-03,\n",
       "         -5.25466633e-01]),\n",
       " 0.7406077428649428)"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm.intercept_, lm.coef_, lm.score(boston.data, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "predicted = regr.predict(boston.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEYCAYAAAB7twADAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmcXFWZ97+nu6urqveENGFPB5KQQAJJkO1FpMMmvjMK\n+g4googEZDEksggBZDOyBGSCiCGEiYZBsgiOCjNoQ7RbJ2LoDKvSrDIdISzdyiJZINvz/nHO7bpV\ndau7qrv2fr6fz/2k6lbde8+thvO7z3Kex4gIiqIoijJUKgo9AEVRFKU8UEFRFEVRsoIKiqIoipIV\nVFAURVGUrKCCoiiKomQFFRRFURQlK5SEoBhjGo0xDxhjXjDGPG+MOdQYM8IY86gx5iVjTJsxprHQ\n41QURRnOlISgAN8HHhGRScCBwIvAXGCViOwL/Ba4ooDjUxRFGfaYYl/YaIxpAJ4WkX0S9r8IHCUi\n7xhjdgE6RGRiQQapKIqilISFMhb4mzHmx8aYp4wxi40xNcBoEXkHQETeBnYu6CgVRVGGOaUgKFXA\ndOCHIjId2Ih1dyWaVsVtaimKopQ5VYUeQBq8AbwuIv/j3v8MKyjvGGNG+1xePUEHG2NUaBRFUQaB\niJhMvl/0Fopza71ujJngdh0DPA88BJzp9n0V+GU/5yjb7dprry34GPT+9P6G4/2V872JDO45vBQs\nFIDZwP3GmBDwGvA1oBL4qTHmLGAdcEoBx6coijLsKQlBEZFngYMDPjo232NRFEVRgil6l5fSP62t\nrYUeQk7R+yttyvn+yvneBkvRr0MZKsYYKfd7VBRFyTbGGKTcgvKKoihKaaCCoiiKomQFFRRFURQl\nK6igKIqiKFlBBUVRFEXJCiooiqIoSlZQQVEURVGyggqKoiiKkhVUUBRFUZSsoIKiKIqiZAUVFEVR\nFCUrqKAoiqIoWUEFRVEURckKKiiKoihKVlBBURRFUbKCCoqiKIqSFVRQFEVRlKyggqIoiqJkBRUU\nRVEUJSuooCiKoihZQQVFURRFyQpVhR5AOhhjuoEPgB3AVhE5xBgzAlgJjAG6gVNE5IOCDVJRFGWY\nUyoWyg6gVUSmicghbt9cYJWI7Av8FriiYKNTFEVRSkZQDMljPRG4172+FzgpryNSFGVYIiK8++67\nhR5GUVIqgiLAY8aYtcaYs92+0SLyDoCIvA3sXLDRKYoybJg3bx4nnXQSIlLooRQdJRFDAY4QkbeM\nMc3Ao8aYl7Ai4yflX/e6667re93a2kpra2suxqgoyjBg5syZzJ8/nwcffJCTTz650MPJGh0dHXR0\ndAzpHKbUVNYYcy2wATgbG1d5xxizC9AuIpMCvi+ldo+KohQ38+bNY8mSJbz44otEIpFCDycnGGMQ\nEZPJMUXv8jLG1Bhj6tzrWuB44E/AQ8CZ7mtfBX5ZkAEqilJ2iAjLly/n2WefDfz80ksv5bjjjuMf\n//hHnkdW3BS9hWKMGQv8HOvSqgLuF5GbjTEjgZ8CewLrsGnD7wccrxaKoihp88QTT3DRRRfxxz/+\nkaOOOor29naMyehBvSwYjIVS9IIyVFRQFEVJhzfeeIMrrriCn/zkJ3H7f/azn/GFL3yhQKMqHCoo\nAaigKIoyEFu2bGHs2LG8+eabSZ9NnDiR559/noqKoo8QZJWyjKEoiqLkmurqai6++OKk/SeeeCIP\nP/zwsBOTwaIWiqIoCtZK2X///Xn11VeZMmUKCxYs4Jhjjin0sAqGurwCUEFRFMVj/fr13HPPPVxz\nzTWBVsevf/1r1q1bx9lnn01lZWUBRlg8qKAEoIKiKMqmTZv43ve+x/z589m0aRP33XcfX/7ylws9\nrKJGBSUAFRRFGb5460kuv/xy3njjjb79u+++Oy+99BK1tbUFHF1xo0F5RVEUHw888ACnn356nJiA\ndX0tWbKkQKMqX9RCURSlbNm2bRtTp07l+eef79s3atQovvvd72qcZADUQlEURfFRVVXFggULAAiF\nQlx66aW8+uqrnHvuuSomOUAtFEVRShovTlJVVcUpp5wS+J2bbrqJf/mXf2H8+PFZuWZvby/d3d20\ntLTQ3NyclXMWGxqUD0AFRVHKlyeeeIJvfvObrFmzhp133plXXnmFhoaGnF5z+fKVzJx5AdXVLWzZ\n0s2SJQs57bRTc3rNQqCCEoAKiqKUH6nqbl122WXMnz8/Z9ft7e1lzJiJbN7cDhwAPEc0OoN1614s\nO0tFYyiKogwLTj/99CQxAVi4cCEffPBBzq7b3d1NdXULVkwADiAUGkN3d3fOrllKqKAoilJy3HDD\nDUn7TjzxRJ5++mkaGxtzdt2WFuvmgufcnufYunUdLS0tObtmKaGCoihKyfHJT36yLwA/ZcoUVq1a\nxS9+8QvGjRuX0+s2NzezZMlCIpFWotEpRCKtLFmysOzcXYNFBUVRlKLkjTfe4Otf/zp///vfAz+f\nP38+ixYt4umnn85rEcfHH/8jH320mc2b/8FHH23m8ccfz9u1ix0NyiuKUlRs2rSJW2+9lVtuuYVN\nmzZx4YUXcscddxR6WAC88MIL7LffQcAavKA8HEZX15NMmjSpsIPLMhqUVxSlZBERli1bxr777st1\n113Hpk2bABto7+rqKvDoLJ2dndiu47GgPOzh9isqKIqiFAV//vOfA+tubd++nYULF+Z1LL29vaxd\nu5be3t64/YcccgjwOv6gPLzh9isqKIqiFAVTpkzhjDPOiNvX3NzM3Xffzfe///28jWP58pWMGTOR\n4447jzFjJrJ8+cq+zyZNmsTxx7cChwHjgcM4/vjWsnN3DRaNoSiKUjS8+eabjB8/nq1btzJnzhy+\n/e1v5zQNOJGBFi7GPl8CfAA0Eo3O1IWNDrVQFEXJG16c5Nvf/nbg57vtthv33nsvXV1d3HrrrXkV\nEwhauLgrFRWjePrppxM+Pwn4KnCSLmz0oRaKoih5obOzkzlz5rBmzRqMMTz55JNMmzat0MOKI95C\neQE4H9iJaPRdlixZyLHHHq0WSj+UjIVijKkwxjxljHnIvR9hjHnUGPOSMabNGJPfRxlFUdJi/fr1\nnHHGGRx66KGsWbMGsJbKRRddRK4f9lIF11MRW7h4FHAW0AG8wubN7cyceQEAn/zkocCXgBuAL3Hk\nkYeWnZgMlpIRFGAO4M8dnAusEpF9gd8CVxRkVIqi9MtNN93Efffdl7T/8ccfj2t8lW36C673x2mn\nncovf7mS2trxJNbsam9v57HHOrDrUF4G1vDoox288MILObmHUqMkBMUYswfwf4F/8+0+EbjXvb4X\n69RUFKXIuPbaa5NKyp944ok8//zzTJ48OSfX7O3tZebMC9i8uZ0PPniyz8JI11KZNm0aO3bEpwdv\n3bqOV199FdiD+HUou7Nq1aqs30MpUhKCAiwAvgX47ePRIvIOgIi8DexciIEpitI/zc3NXHPNNQBM\nnjy5r+5WtppdBTHUqsCe6ysanUFDw3Si0RksWbLQ1Qp7g/h1KOsZPXp0lu+gNKkq9AAGwhjzT8A7\nIvKMMaa1n6+mdMZed911fa9bW1tpbe3vNIqiZMr69eu54oorOP/88zn88MOTPr/wwgsZNWoUp59+\nOlVVmU87mXZIjK8KbNN/M60KfNppp3LssUfHXde6trYDrUAL0A1sZ8qUKRneUfHR0dFBR0fH0E4i\nIkW9ATcCfwVeA94CNgD3YVMwRrvv7AK8kOJ4URQlN2zcuFGuv/56qampEUAOPfRQ2b59e1avsWzZ\nColGR0pj43SJRkfKsmUr+v1+T0+PdHZ2yqJFiyUaHSkNDdPSOi4dOjs7JRqdItAj0CnQI9HoZOns\n7BzyuYsNN3dmNl9nekAhN+Ao4CH3+hbgcvf6cuDmFMdk4adVFMXPjh07ZNmyZbLnnnsK1jvQt913\n331Zu05PT49EoyMFnhUQgWclGh0pPT09gd9PFJ9FixZLZ2dnyu/nejylzHATlJHAKuAl4FGgKcUx\n2fhtFUXx8f7778tOO+2UJCaAnHDCCVm7TmdnpzQ2TneTt90aGqbFWQSeRdLV1ZWXyd4TrWxaPsVI\n2QvKYDYVFEXJDQsXLowTklGjRsmiRYtk69ataZ/DE4NUk/5AFoHfIgmHG5w7KrX4ZIuBxl0OqKCo\noAxbhsP/4IUi1W+7detW2X///SUUCskll1wi7733XkbnTTc2kmgReG6sZIukXSA6LNxR+UAFRQVl\nWJJp0FZJjx07dsisWRdKRUUopXtn7dq18vLLL2d87nRjEX53lj/Qbi2SJolGx8ZZJJFIi4TDTUNy\nR6XzcDIcHmBUUFRQhh3DKUiaT5544gn5xCc+4XNp3ZPV3zYoNlJfP1WWLl0a6M7yLJPEv7W1SNrj\n/vae+AxmnOk8nAyXB5isC4oLgoczPWkxbSoo5U06QVslfd544w35yle+EhBs31ngg6z9tkEPAhCV\n+vopKcUjHG6S+vr4GEk0OlnC4YasBMjTeTgZTg8wgxGUgVYYtQMfG2PWAr9z2x9EZLP/S8aYsdha\nW28Di0Tk/QHOqyhZIRsL2IYziQsGH3vsscC6W/Ae8O9Z+229legzZ86gqmovPvzwJeA6PvzwMuA5\n5sw5kurqffCvdK+ubmHLltfw/63hTZ5+eg0bNmxIe9FjKrzV9Zs3J6+u985rV9rvTnzpld3ivjOs\n6U9tgHXAN4Dl2HoDO4CPgT9gFxx+Gqj1fb8ZuDtTVcvlhlooZc9wSeMUCfbdD9afH+S62b59uxx0\n0EFx1klFRUjq6vbLyW/b09MjS5cuTbI86uomSzjclGQJ5GKxon8sA1kfXV1dSYF/iEpXV1fWxlEs\nkAOX120J7/d2ArMR6HUCswVbevNm4BzgN5kOIpebCsrwYDgESYMEYNmyFRKJNElt7b4SiTSlPcEG\nTZ6RSJO0tbXJQw89JIBMmTJFVq1alfK3zdZvnmoiTyUeufxbD/RwYlfKjxVoEpgg0CSRSEtZuliz\nLiiBB8BdwJ7u9STgAuCnwDtYu/iMTM+Zy00FRSkHUk26VVW1AiMEpgjUS1VVbb9rOtra2uQnP/mJ\nHHPMMRKJ7OGzClYI1Eht7YESjY6UuXOv6Hc9SbYD06km8myLx1AzuHp6eiQUqneCcqBAk4RCdWX5\nIJMvQbkpxf4o8CNgn0zPmctNBUUpVjKZLIOSD2pq9hOoEZgvMNJNcFGZO/eKpOOXLVshoVCD+54/\n2P47sXWpRqQdaM5VYDrX6brZEMGenh6prm6Mu/fq6kYVlCEIygpg1xSfVQG3Z3rOXG4qKEoxMpiC\nh8lZTw0Ce7mn5dj+UKihz6ff09Mjv/71ryUUqhUYnZS9VVlZLbW1EwTGpZ0pN1BmXa5cUkMRhGyJ\n4HDKKsyXoByNrfp7ClAZ8PltmZ4zl5sKilJs9De59TcZJ7qFzjrr6wJh58sX3zZewuEGmTVrtkSj\nI6WmZoJAJElMABk3bpz813/9V+B4Uq3n6G/8uVqjMVRByJYQxMbRLrbacLumDQ9FUOx1mOmC8X8D\nlgGzsR0Tvwk8PJhz5mpTQVGKjVST27x5Nww4GScXQmx3bix/1pG331v01yNQlyAmFQIhWbhwkYgk\ni9WsWXP6HUtQzCOXazSGKgjZHNusWXPcbztBICqzZs3O+BylQN4ExV6LA4H/BLa6bK8dwNPAuMGe\nMxebCopSbKTKsOpvwku0XOIn2BVORMYJNLr3IjDePUV736l0YnKGwHspr5Gqam+ixdL/mDKf9DP9\nzTIVhGykl+vCxhwJSt8JoBE4CBg/1HPlYlNBUQrBQHGExMlt3rwbUk7G3ne9DKxga+C3AlUCewq8\nJUFlSWwAf8KAE36QMHgr0geynnI52WZLEIYS39EYSpYFBagFPguMSth/LhDN9Hy53lRQlGySzoSU\nbhzBf65Uk/HDDz8slZX1cfu9rCLvOjU1E8QYz/pAQqGdnNtqtlscuI/YLK7FEpTNFWR5pFMzK504\nTzr3nu3fP5eohZJ9QVkKfAt4GKjz7d8b+FGm58v1poKiZIt0hGIoE07iZHz88Z8RG3QfH/dEDPtI\nW1ubrF+/Xj772c/2CYm3GWPkN7/5jYjYld2VlVHnCpsmUCu2ZtbUuFhJff00CYebZNGixXFjqa09\nILCqb39P5elaZ6VaXHG4VGbIl6Dc5v49GDg74bMjga9les5cbiooSjYITtttSiq50dnZ6cqI2H7j\nmU6+3vvVq1eLzcy6S5KD7jXS1tYmN998S5KYWAslJIsXL+67Rmw1/QSJRJr66SdiLZFFixbHrcAP\nhxuS1l4M9qm8XJ7wu7q6ZOnSpWVZcsUj7wsbg1KEgQWZnjOXmwqKkg2CfOdeeq6/b/miRYude+hA\nJwTzAyfMnp6efrO6zjvvAhfzmC42Q6vRWSqNEgrV+cTg5DgxOeGEE+SVV15JGn+Q1WDFb1rCPR0g\n4XCDRCKJa1vqsvJUXg4xiFK3sNIlX4JyN7CLe31rwOfzMz1nLjcVFCUbBMcVRgpc7VxI0yQcbnCr\n0eOf+G+99ba4ydx7+reCkfyk3tXVlWQRQINAtYTDDbJs2QrfxNzjxGaKwK4yb94NGd1TYgFGGCk1\nNftJbe2+SZN+W1vbkOMXpW6hlPr4MyFfgjLVpQcfHiQewPJMz5nLTQVFyRbLlq1wE/B4JyaLfe6o\nFQL1khjviET2l3C4QWpqbCzi1ltvcxPS/c76iJ+05827wa2A97KxXhf4srNSqmXlypUBQfznBJ4S\nGCGRSFNGk1vMojogzqJKtFCKLVurUJSDhZUu+VzYeKZb2Pg2sBC4FJgLdARZLYXcVFCUbNLV1eUm\n/HaxcZIDnZWQapFhYupuVCKRvXzH+GMyja7Y48+d1XG+s2I8l1Y4LnV43rwb3OfT3LlWDGpyW7Ro\nsYTDDVJXN7nv3Lme9AudrTVY1ELJgaDYa3E40OZb2Pg+MJ+AciyF3FRQlGzjTbZ1dZOdYPitDW+R\n4Xjn/tol7mnWpvBWO5FZITaNdx+pqKhx5xovNhMr7BMSb5sssF28hIDVq1c7S+J+J1DpTW5Bk3m6\n+5TStrAyoVALG6uA0cUmJL7xDfV3VcqEgcqSZzJ5et9ftGhxQDykXaqr62TOHK9ER3yGlhWZemdZ\nNEpFRSThe0EteBEbJ3lLvISA6uo6OfnkUyUSaep3cvPf22DWyCjJDIffJ18xlPMD9u0OjM70XGle\nLww84eI2zwM3uv0jgEeBl5yl1Jji+Oz9wkrJYt06TVJfnzzxDjVrx5+x1dAwTUKheqmqqhdbCiXq\nLI6pzo3lWSJd4qUWRyLjJD728p7YWluekOwkEBIbJ/GEqd6dd6xEIiNk3rwb+l1o2Ng4XSKRprTS\nf4dLFpPSPzkRFGBPoNr3/pGA7zQC1wCXZjqAtAYJNe7fSmx3yCOce+0yt/9y4OYUx2bxJ1ZKkVjg\nOXkNSTZ94j09toFVJBK/Gt1eu1qgxb0Ou/0fCzzr4iaJlkxIbCmVSwVWSyhU50sIqJd0U5Nj99Yj\n8B2B/hcoFkuMYDhYAMVOrgRlPbAJaAeuB37vF5iE734CuCTTQaQ9WKgBOoH9gBc9qwjYBXgxxTHZ\n/I2VEqOrq8v1Atk/biL11pAE1dCCfTJKv/XT2dkptbUHJp3Pxjk8sWgQCIkxVVJVFXEdAL0mWQcI\nROVTn5oh4XBjnCvLphPXuePjU33r6ib3U5PLi+tMc0I0P6VYFEMWk1pIxUGuBGVvbFn6B11W1w5g\nsxOWecCxngXhvn9LpoNIYwwVzuX1D+/8wHsJ33k3xbHZ/ZWVkiGW5jshaSK1TanaJRJpCrAoMk+/\n9Qher1Ij3qp5WCP+zC1jvPjJCvedTqmtndz3dJ74lG4zu+KbYXmLEYMsFBvfSbaY/BldA40/nxZK\noa+vxMhXDGUN8HVgObbR1g7gYxfnWAn8d6bnzODaDcAfgdZEAQH+nuIYufbaa/u29vb27P3iSlES\ncz0lLtqLOpdPROA2gTapqdlbTjnlVMlG+q3HsmUrXKxiH3feaoEOsetJgoLt17hJf+BMrVSFG70a\nXIkECVB9/VRZunRp2pWQ82khFIOFNFxpb2+PmyvzJSi/Sni/HzAL2xr4AeCATM+Z4fWvxq57eSHB\n5fVCiu9n7QdXihd/1pVX6t1O5it8k9M+Ars7YfEHzUMSS/+1k3pQn/BM/PqeqLW1tblMsBFiTFWA\nmIQEbhDYR2prJ6Q1gXsTfn391LiCjqnGMZgn/kLFMNRCKR7yJSinZHrMUDZgFC6DC4g6V9sxLih/\nuduvQflhTGyCTYwRtDvhWO3EotG9rpdYJ0Nv/2L37+4CDRIK1cVNYpmm23Z1dcnKlSvljjvu6Av+\nX3PNNQliMkPglb5Js62trW8V/ECTeaqikkMpKV8slNp4y5V8CUoY2w9lt4T9R5CDtSjAFOApF0N5\nFpdJBowEVmHThh8FmlIcn83fWCkyUtfY8sqijHciM1piqbYT3OtGsQsSawS+6Pt+k0QiLX1ulnSf\nmr2JMBJp8Vk99vqzZs2W7du3y0EHHSRTpkyRK6+8KnDSHExAOt2y+qWUNVVq4y1H8iUo17vg+NqE\n/ZOAOzM9X643FZTyJrgK8H5ig+5+kWl0VskIsWtA/IHqdklO2432lSYPukZd3YFJ6bY2ZnOc2DUk\nkcDzrV+/XrZt29Z3zECNrQZy92TDRTQcSrErmTMYQakgc8LAPwHf8e8UkReAXxpjvjiIcyrKoGhp\naWHLlm7gObfnOeAvQDNwgNt3ANZzugVoAX4O1AMhoBdYB+wR9/1odB82bNiQ8hobNrzM3XcvBmDT\npk1cddVVfPTRB8Bj2DyV+oTr70FnZye77bYblZWVADQ3N3PwwQfT3NwMQHd3N9XVLXHHhUJj6O7u\nTnn/gznGz4UXfpP99juIM8+8kf32O4gLL5yT1nGKEkimCgTcPpTP872hFkrZM2vWbGdh2CD7QQcd\nLIml4e37u5zlEHNF2f37J1kooVB8Gu6tt94miVV5ISLf+ta3ZLfddkuIjXjb7wItnlTk20Lp6urq\n1zJThjfkyUJpNMZM7ufzjwdxTkXpl97eXtauXUtvb2/S/oUL78HO4e8C23nyyV5gG7aB6HTgKGwN\n028CBpv5/rL7N4xds3sdcBgwDjicrVs/4j/+4xd912lu3gnYFfg37JraMUAFt9++gjfffDPFqI8H\nxgOHMWvWOUyaNKnfe2xubmbJkoVEozNoaJhONDqDJUsW9lkw2TrGo7OzE1sII9mSUpRBkakCYYPk\n64HTARPw+eJMz5nLDbVQSp7+gs4rV650T9lXS2I5EmuNjBCISCg0SaBSbOqw+LZpYmtqidg1KjVi\nA/q2PIv3pB//NN/ji8HsEFuny1olO+20k8ydO1f+9Kc/xWV5ZcJgAtKDOUYtFKU/yEdQ3l6HU4CP\ngG7gDuCrwBeA24GlgzlnrjYVlNJmIJfOHXfc4YQgufOgrZ0V9olAgySvGm8Sb+1JfE+THqmvn9oX\neN+4caOcf/4FbgLeM0GY1goYOf300+W9994r5M+VMTF3YSwbTVFEBicog3F5ISI/xfZDeQX4BvBj\nbGmWicDswZxTUYIYKOh87LHHAm8Cu2G9rb14rhv7elf3vhvrzroLmIF1hR0ObMRmvM/A9oprxbqz\nHmPbtr8yZswYli9fzsSJE9lrrz3p6nqSO+74FpHIu8SC9NVEIo0sWLCApqamnP0WueAHP/g+XV1P\nsnTpVXR1PckPfvD9Qg+pJEjlgh32ZKpAiRs2feYQoGWo58rFhlooJUX/qbR2IaK/zlZPT49Mn35w\ngLsrKjBHYsF5v5vKW9AYkVCoViorayW+q2KNRCJN8p3vfFcOP/zwPndWbW2tPPLII3F9RQa7+E7X\nWZQuw6V4JflyeZXSpoKSPXI9Cab6H3XZshWuIm+NwDiprm7sa1Ob3NzKq9l1mxOJkNhFjJPFLmqM\nSqzGVlg+/enPSChU58RmqkCDVFSE5XOf+1yfkPi3UGinvrEN1OUw1e/lX9k/UOkUpbgYTqVhsi4o\nwMXAMQOeBMYCtwKTMx1ArjcVlOyQi6eyxMk31f+oQZ/FqgT72+96m1ezq1pi6cRekcaIeKvhPUsm\nEpnkBKjGxV0i0tLSEiAoVQLfEnhmwMZU1dWNEgrVJf1etid9ctFKFZXSYDgVr8yFoLwCPJmw71Tg\nGWAZcKxvfwibl/mVTAeRy00FZejk4qksUaCC+pJ4/6MG/U8cDu/thKPHubn8E3SDGOOJSaLl8nOx\nWV3tEgvcV0isVMo4scH7KjHG9IlJVVWjwMspJ5HgEjDxFYRt18gGie/OKJKq/LxSfPT09CR1vQwq\nJFoO5EJQRpJcs+thYA52SfB24DXgSmBX9/ltmQ4il5sKytDJ9lNZKovDNo9qT8tCsZaG96S/wk3e\n1pVVUeF1RUycuPdxYjHdiclezurwhGSEO9ez7jxVMmnSJHnwwQf7FdSenh5ZunSpK07pv14sJbmu\nbrKEQg3u/pIz0oIaZCnFR09Pj89FOk1gRFIh0XIhLzEU4E7c+hNs860bsWk2W4EngfZMz5nLTQVl\n6GTbQkkWqBUCNRIO2xXrkUhLklst1hN+qnMZ7SLgT3mNCITln//5n11L3Vv6sVBE4FGJ9W3/ddzk\nbq2KfSQSaZHHH3/cd/2GpMZUsXjIlIDrxSwUK5ZeXxKvJXFs1X25+uHLjdh/uz3uYaFHXV5DFJRJ\n2NzL43zCUomt73WRZ6kUy6aCkh2yWVI8OXMrfm2Iv9+7v8dJff0UCYcbZPbsOW5CHuGe+D0XVo2z\nTKJiC0SGnUXiTdy7CPxe4Hrxd02E0xKsivsFon1uqFRB9GShtTGZ+vqpfTEU7/eaO/dKiU8euFog\nLNHopLLOFCo3NCg/BEHB1qwIB+yvAk4G9sz0gvneVFCyRzazvLxJOhYL8Sb0HgmH95a5c6/09TiJ\nCMwSWyX4fve0H7TqfbLYjK5YoywrKru7/VUCO/uExL/9QWI1v+oFRsu8eTf0O4EEVyGe3NcNMTHp\nINFVUlVV29cDpdB/DyV9hsti0FwISlD/+GjA98ZiV8nPJUVfkkJtKijFS1dXl4RCXiqvFwtpckLh\nNcry4iPG96FZAAAgAElEQVRed8WxTiTCkuxiqnGbFyOZ7d4f6M4RSiEmlQJ7OCG5QqC9X9HwXByZ\nPq16ac61tRMkEmkaklUyXNZCFBuxv3m7eJaxWijpC8o67Er45cAbxPrH/8HFTo4Han3fbwbuznQQ\nudxUUIqXzs5Oqa3dV2KlU/yLEO8XqHP7Y//zWqFodJ97x3kWSIMkx0zaEwTH34Z3hMAigVUCYams\nrE1y6Q0kGpm6ArNhVQwnt0uxoWnDQxOU2xLe7+0EZiO2rsUObJOJNcDNwDnAbzIdRC43FZTC0t9C\nv1hTqhFig+jjnEUyUmKdFBvcNkVimVn7OtGpd4Kyv7NYvKD3RoF33PtO3//8+4hdb7KTO3eTxOIr\no6WyMiqf//z/k9WrV8fdw7JlK1yqqM0iC4Xq4oQj366n4TSpFRvDSczzFZS/y4uduAD9BcBPgXeA\n94AzMj1nLjcVFEu+J72enh6ZN++GPrdMKFQv1dWNKVbB10lsEWJi8caoE4wmgZOdcNQKfMm9bnff\nbRcba7lJrPvq6AALxVtBv05iHRw9yycqqfziMeG7X7w4jr/8S74ZTpNaMTJcet7nS1BuSrE/CvwI\n2CfTc+ZyU0HJn7/dn5UVXxIlOZMrcR2Hzdyq8lkZ3naAwA0SW2cSFRvz8Cb/JrFpuE9IcozEi8/4\nm2I1uvF49b7GuX3ptACOt57mzbshJ79jOgyXSa1YGQ4JEfkSlBWpUoNd9pd2bCwi8vE061kjkUiT\ny8qKis3K8twynZJYHsXvoomVI7k6YGJvkuSV8ImWR2WCkHjbOLHpw51ORERgH6mtnSDR6Ei59dbb\n3Mr1ywUmJAjZeFm6dGnf/cVcc8VjFQyHSU0pHPkSlKOBt7A9USoDPteV8kVErv3t3pNy8krzkRJb\nEZ7aQlm0aLFUVUXFBta7xFYI9go4JgqTt40TaPMJRVAqcEjggqTrQlRWrlzZNwkvWrQ4IdMs9j1/\no6l5826QROtJ4xZKOZMXQbHXYaYLxv8NW9NrNnAStpbXw4M5Z6624S4oubRQgkuieCvN411VoVCd\nVFc3xrloFi3yVoz7izfuLjYWcr9YV1ZD4GRv3VTT3flrJbbq3ROTh8W/2NArbR+JtPSJgH/Boi3X\nknptgcYtlOFG3gTFXosDgf/EllzZ4bangXGDPWcutuEuKCK587cHWT920d5dAmGprZ0okUhT3wLB\nxIyv5Kq7I5yA7O87X5cTmhECE52QVKcQGOOsFS/A71k5V0vimgHrZmsQf7HISKSp35a9GrdQhhOD\nERSvdMqgMcY0Ylvh/UNEXhnSyYLPvwfw78BorGjdIyJ3GGNGACux7fW6gVNE5IOA42Wo91gO9Pb2\n0t3dTUtLC83NzUn76+rq2LBhQ9+/id9Ldc4xYyayeXM7tivic9hea4aqql2pqPg7d9zxPc4995yk\n4x555BHOOutGdux4yffJdGxG+tvAf7tzLsPWHj0Y2xR0HvY/h5d9x43DVv/pATZRUVHJ8uVL+cMf\nHufOO+9mxw4D7EZ19d9YuvRuAL72tfP4+OMabGLiOGA9kUgDv//9Tzn44IMz/h0VpdwwxiAiJqOD\nMlWgfG/ALsBU97oOeAnbang+cJnbfzlwc4rjh6jT5Yv3xB2N2sKGodBogahEo1P6fQLv6emRtrY2\naWtrk1tvvU1imVRNzl01os8dVVERDazLVVMzMcCVNcKdY6bYlOAWSQ647+muEVyEEWqkpmZvufzy\nKxLGNVMiEVsjLLbSOTnYn8o6SQcNkivlBHkKytcCnwVGJew/l4CyLNnegF9gS8C8CIyWmOi8mOL7\n2fp9y4rg+Ed89lSqJlL+7olVVbWuQVWnwD2S3D2xRioqwlJd3Sh1dVMTXFBjnKvKc01ViY1/7OQE\nxb+q3b8dLrH1KTViEwFEvCyuUKg2sIlVOLynLF261LnpkjPPotHBlZD3Z7lpKRSlXBiMoFRkZgQB\n8ENnIfzYGFPn2/+Y+yxnGGNagKnYlfmjReQdABF5G9g5l9cuJzyXU1XV7li3Eu7fPbDPC/Z9KDSG\n7u7uuOO+9rWvs3UrwH8Br7Bt23/y0UfdQBhb9m3XhHPuyo4dlWzZ8ns2bHga+6f7Htal9R5wBraq\nTwjrvawCNmDLx+0eMHqDTTQcB/wr1tU1yX32HPAm559/NqFQS8I4xvPxxz2MGzeOLVu6sa61bndM\n7NiWlpY0fsHY7/Hd797IXnvty9VX/5iPPjJ88MFlbN7czsyZF9Db25v2uRSlHKgaxDF/F5FbjTEd\nwBeBfwMQkdeMMT82xnxNRH6czUECOPF6EJgjIhtcNz0/KQMl1113Xd/r1tZWWltbsz28kmH58pXM\nnHkBVVVj+PDDV4FbgMuwE+ob2IkW4Dm2bl1HS0tLX9zgvvvu5+OPt2GF5/8BC4FTqa5uxpijqKgY\nwebNb7tzeTGVN7EVe/yT+1hsXARsXkcFsZjJc8DhxLpKn+IbfTOwyY05hDVM78EWxd4JeJNweDR3\n3/1jtm3bDnQArX33FomMpbq6miuvvISrr/4n7DPIYcA+wGtceeVVacdFli9fyVlnncdHH20B/ugb\n+wzgxT4x1jiLUip0dHTQ0dExtJNkatLgWylPwJoTYEGm50zjmlXAr7Fi4u17gXiX1wspjh2y6Vcu\npHJzRSI2nlFV1exiKJP7srO8mEckMlliFYC9Y20swlskaOtd7SqxfiQjJVa4Mb5niE3PrRe4VGxt\nLhG/28qmDe9wbqkqgdVxbrRTTvmiS/mdKtXVDVJZ6bnrEqsT7yWJTayGWkol9jsG9bO3/VQ0pVgp\ndchTDOVuYBf3+taAz+dnes40rvnvwL8mXge43L3WoLyjv8BwUJpvff1UWbp0qXR1dUlnZ6d0dXX1\n1eAK7kLorTMR8WIol156mVRV+UWjXWz6789djCPiYiITfOfbIbbulnFbp+8ajRKLrVSLTReOFxyv\nj0hnZ6e0tbX5uuglC2Zt7cSkuMZQUoDju/YlXq9myKXpFaUYyJegTMWuNzk8SDyA5Zmec4DrHYHt\nXf+Mu+5TwAnYfversFlfj5KiD8twEpSBanbF1l60i1ciPvHJPN6K6RS7INA/mR/g9j8roVCDHH74\nkRLcv32c2PLzX5T4Hie7CqwROEyIC7R7dbyiAqPdv2GxgfvkhY1e18T4MSdbDJ5gBgnsYLOy4n+j\nWD/7SGRE35obRSl18iIo9jqciV0p/zbWkX4ptrlWR5DVUsit2AUlW6mm6fbtsCnCYfGytEKhhjjh\n6ezsdPW4JOUTv03nrZGKihqJZYYlf6+y0mt45d+fKnOr0gnQdLGLE/0i4rnJJovnvqquboz7zbzm\nVYnXy5XryW/h+BdvKkq5kDdBsdficKCN2Er5950bKqm+VyG3YhaUbFYBTr+zYHJdLf/kHCuH0u4s\nkat9k3lUbFqv12L3fom14fUq8Y4TGxuZL9XVdU7AxLftnkJQQj6rJqi971iBa8Rf5LGtrS3uN+jp\n6ZGTTz5V8tWeVdedKOVMXgWl7wQ2YD662ITEN76h/q45Idu1odLvfZ64/qJHYPe+gon2HJ+Om5Rt\nb5E6iQXcV/iO9VsE7WKtH1tHq7p6dwlyV9mYiV9Ibvcdu7fEFkcmHvdz3/uaQEEJas/qxYd04leU\n9MlXDOUTwB3AkZkeW4itWAUlW1WA/U/JqQLNqS2UmP8/HLZuGxuIT3RfhcXGQfzFGO3KdGO8oope\n/OMId9xZThj2klhdLa8a8YNOVGrFVg3ukVjNrkonZJ7FM01irXtHuPcjJBSqSxKIoN80HN5LQqE6\nqa/vf/W/oijx5EtQfoJdEfZUpscWYitWQcmGhRLkMkvlholV1p3qJu66JOsiHG5wpdz9gfhEK8Tr\nzT5WvOD46tWr5ZprrpFrr71eIpEmqapqdNbH1xMsl9W+83pCc4Akdku04uHvLR+R0077slRXN0hN\nzYSUWVTJv6nnrjtQElOHFUXpn3wJys0uw2qPTI8txFasgiIycOpqfz76VO6d/iZLfz2t6uo6iWVm\nedbABKmsrHUTerv7zB8n8bZ9BO6QurpYqZKNGzfK9ddfLzU1NRJzZ0XEttv1jql112qXmCusS4Ld\nYmGJ9ZH/ktj038kSDjfEZXil+k3r6iYHnHdk3JgVRUlNvgTlDOCITI8r1FbMgiKSWjQGCth3dnZK\nNLq3xFrSxvf6GOgasS6J7eItULT9Sxol5r5qEevqqk+YmBvjBKy3t1f22GMPn5D4t9N8Vs1d7l/P\nBSYCSyWoWyKMErhDYKXEGnXZ66cjnEuXLnW1w/znPUDC4Qa1UBQlDfIlKFHgYeB6YHKmx+d7K3ZB\nCSIdd1hXV/CTfWK13P6Eadas2c4S2FViGVr+rov1Yt1UNm23qmpfia0licoxxxzXd73Pfe5zAWJS\n6TtnnVgX1mRnqXhutFQWSsgJyYGSWAAyKNaUKJqpqgL0Z90oihIjX4LyC2wzis1uwWEP8ADwDWD/\nTM+X660UBSWdgL21UOJTchOr5fYnTLHPlga6huzkP04gLKFQnVx77fWuVe81AieKdWfFUnNffvll\nCYVCAkhTU5PMmHG0E6vdnUXjbw28nxOMGomthvcH9kNSUVGbMKZYifpEcU0lmv64UTjcpGKiKBmQ\nE0EBLnBB+HedZfIbtz8EfAq41i1o/MgJzJpMB5HLrRQFJR0LJZ3v9CdMsc86xcYqxLfZelSeKywU\napCqqjqxgfiwEwB/Ta+orF69Wj71qaOksjIsDQ0HuhjNWIErnfgcIF5gPFbCPhZ0r6kZL1VVUZk9\ne460tbX5Fld62z4CuydZWQP9DrpWRFEGR9YFBVtNeAfQiy1DuwP4AFuo8USg3vfdCHAM8OVMB5HL\nrRQFRcQuMAyHG6SubnLKdNd0gvoDWyjtElSPyrqbPDfT3gIXCOwhNmDf4LYu9/k4CYXqJN5l1i6x\n3vC3iXWftUgo1CDh8F4JYhEr5+KtGwlyV82de2VaqcKDSb9WFCWeXAjKH7GtdcE2ojgMWAH8BTgO\nOCTTC+Z7K0VBiblqpgW6ahJ7s3vdE4Oewm2cJHjluHedyso94lxOxoQlluV1n0CFxMdGGsV2Tmzw\nWRztvsl/pMBi8cq7QI1UVdXKvHk3SFdXl6/Sr9dlsUm89SieGKTrrsr2AtHBoFaQUo7kQlCeTbF/\nPnBqphcrxFZqgpJuTS4vXjBr1uy494sWLe4TmNWrV/syuZJTi5ctW+GKRYbFrkK33wmF6iQaHSnh\ncKrMLcQG1tudmOyRYHFMdqKTfA/Llq1wZe7HOcEJOWGyiyZDobq4ysfpTNRDqRw8VLJZPkdRiolc\nCMqjKfbXAUsyvVghtlITlPRrckmfKyhmHcwXf+HHWEZWf+fyV+jtEeiU2trJMnfulS4O4i+T4i+X\n8iN3zIFOEBJX1o/rO59nebS1tSWM3xOk2LGVlbWDmqALYSUUg3WkKLliMIIyUAvgbUE7RWSDC8Ar\nWaC3t5e1a9fS29tLS0uLa1Eba03rdU7s7u6murqF4La9vcBNQA3WU/kKtt3u+9iciVTnmobtAHAp\nMAE4m40b/8LNN9/Gli0PA1f5RjoVmzX+KPA1N8bXgTnAYdTXTyMancHcuZcQCr0D7AucB+zL5s2v\nACSMvxbY0/d+V7ZvFzZvbueDD57MqJVuc3MzBx98cF47JAb9PRLbJivKcGIgQenv80CxUTJj+fKV\njBkzkeOOO48xYyayatVvWbJkIdHoDGprDyQancGSJQtpbm4OFJtY295ubIvcsSS32/0ssBfV1Uey\nYMHNdHd3U1dXx8aNLwGHYOt73gn8DngWK0SVwOexvduPx/aK7yUUqqS6+iQikcnAYYTDtUSjP2TR\nou/zm98sZt26F7n44m9iTAVWyJ4EOjCmkj333DNh/BuxguS9fwzYjVKZoPsTf0UZlvRnvgAbsKVW\nPgM0JHy2MFNzqBAbRezySuUyWbRosUQiTVJbu29c3aply1a4bCq7fqO6urEvhlJbO1FsOm9jgvup\nXrzV76FQg4RCddLYOF1CoXqxWVhNvpjIDp97bKr4U4dhpNTU7C1tbW19GWi2FEpywLw/t50Xc4gV\ni5wtsQZVTS6+UjoupELGbxQll5CDGMoOt23HWiTPAD/AphP/JMUxl2Q6iFxuxSwoqVry2kB5/KQa\nn0ob323RTvBNvr7vYTdhRyS5sOMIgSfdd8IJsZE7E4TIW9xYJ15hxaCU3kzXyLS1tUlt7b4S623S\nI7W1E6Stra0kJ2jN8lLKkVwIyrPA/sD52HTh9T6B2Y7t2LjSfT7RHdOe6SByuRWzoARNvOFwkysh\nH/90v3Tp0sCn/uRAtxeov0VsHazERYtTnagEZW5V+D6PZXHV1Izrm9zj+6nHAu6J6z76EwZdjKgo\nxU8uBGVZwL5xwFnAvcD/Jlgx7wCbMx1ELrdCCko6E2PixLto0eLAyTaVZdDW1pYkNNHo5L5Fkcll\nVRqd5ZIoJtVi+5R0ir/syllnnRN3D11dXa61b6MEpfomWiqp7r8ULRFFGU5kXVDSOgHsBXwZuAe7\n4HH7UM+Zza1QgpLJ+oTEiTfVZBu0P/lp3/Y1Wb16dV+peq+ce1VVrVRURJxYjHFCMkpgJycy8VbO\n2WefE3hPiUUkvVTfVAsx071vRVGKh4IIStIJ4flsn3OI4xnarzoIsrE+IdVkG7Tfm+gjEdv0Khqd\nIpHICDnnnK/Ljh07+mIs9fXTJBIZIRUVUbHlUC4R+G8JhRqksjLirBmvB8n8JDdUsmttpNjyK4lx\nGq3qqyilTrEISuBiyEJthRCUVMH2pUuX5uxpPNbf5BmBZQK7CCALFixIEWNpFH873blzrxRbzNFz\necXXxAq6J3v8d5zF4t9fPH1Hgsraq1WkKANTLIKya7bPOcTxDO1XHQSpVrTnsq95Z2eny5w6PC42\nMmrUKGloSGw0NV5s7az+VrEnB8qT76nGtQ1OXClfHJ0RBypTo3EbRUlNUQhKLjZgiQv4P+fbNwK7\nZPsloA1oTHFsVn7cTInv4R4Vf7n3XKytWL58eZyQ+DfbxyRVuZb4Olv9BcoTP58374a+tOVYf/ji\n6N0+cJma4l/joiiFpJwF5ZPYuh9+QZkPXOZeXw7cnOLYbPy2g8JrRRuUBjzQ0/tA/eQTP/v4449l\nl112iROSyspKueSSS+See5bECYH3pJ4qpbc/l1Cqz9Mpt59Pgl10451Vlv7fQVGGK2UrKPbeGJMg\nKC8Co93rXYAXUxyXhZ928AxmzUV/GWL9ffbQQw/1ickJJ5wgL7/8ctw4ch1LGKwY5QK1UBRlaAw3\nQXk34fN3Uxw35B92qAyUBuwXh4GaYtlV9AsDJ8UdO3bIrFmzZNWqVYW83UAKUeY98XfvzzJTFCWe\nwQiKsccVP8aYMcDDInKAe/+uiIz0ff53Edkp4Di59tpr+963trbS2tqahxHH09vbS3d3Ny0tLTQ3\nN9Pb28uYMRPZvLkdWwzxOaLRGfziF8s55ZQr+OCDJ/uObWiYzrJl8/jhD3/Ir371K2wBxZeAOhoa\nprNq1d0cfPDBeRn3YM8RdK/r1r2Y8+rAQb/7UO9HUcqRjo4OOjo6+t5ff/31iIjJ6CSZKlChNpIt\nlBeId3m9kOK4ocl0jkhVQDE52+pXUlkZlmg0mhBsvzptt81gXVHZsiq0Ta+ilB6UucurBfiT7/18\n4HL3uiiD8v3Rn2srfqFiYvtdbzNxlYhTMZAopPo8fnzxxSizea+KohQnZSsowDLgTeBj4K/Y7k4j\ngFVY38+jQFOKY7P08w5MJkHnnp4emTfvBlem/oCkyT62UHFxkphMmjRJ7rzzzrQsk/5Eob+JPmZV\nrHCpwNMFamTevBsG9dto7S5FKS3KVlCGsuVLUDJxD3nfjUanCESlunq3JGsj3k30OQHEmCqZO3eu\nbN26Na0xDSQKA7UbjkSaxNbsyo5loavUFaV0UEEpkKBk4tIJTmcdIXBOP9bDywJfzdjlNJAoDDTu\nefNukMSyKhr7UJThwWAEZaAWwEoaZNJbPP67AvwZ2xjzHnbsqO47ZtWq37Jt2xbgcOAzVFf/gh/9\naFFGmUnNzc1cddW3gJ0Cx9bc3NzXbrihYXpcu2GAc889h2j0XbTFraIoaZGpApXaRtFaKPcJHJYU\nH1mzZk3GAfGBVtWn02FR+5YoiuIHdXkVRlBEYhNvUIA9kX/919tTZG4hZ5xxRkZptv7S9KmuO1RR\n0NiHogw/VFAKLCg2Y2vftNJ5P//5z8cJSSgUkksvvVTef//9tC2UWFHG9CyjdERBxUNRFBEVlIIJ\nymDWWXR3d0skYlvxnnjiifLKK6/Efb5s2QoJherFNq8aJ9XVjUlFHG0ZlgPjLBnYZ8ipvVreXVGU\nwQhKyZReGSzGGMn1Pa5du5bjjjsvqVzK7bdfyIcffsjs2bMDj7v77rsZN24cBxxwQFI5kIHKlaxd\nu5ZjjpnJhx+uB2LfgU8RiRj++teXMwrgF7I8iqIoxYcxBsmw9IpmeWWBlpYWtmzpJpYN9RgbNz7P\nWWedxcUXX8zzzz8feNy5555LT8/fGDNmIscddx5jxkxk+fKVwMCZYy0tLWzbth5bJGAGcCBwGHAr\n1dVjAzPM+iOTTDVFUZRAMjVpSm0jrzGUERIOx/ckAeT444+XHTt2JB0zUGXhgdxonosKWgTqxa6q\nH9ziQy2PoiiKH3QdSuE47bRTOfPMU/n447eTPnv00UdZu3Zt0v6BrIIrr7yESOSowDUi3jXXrXuR\nefPOIRKppKHhrsDvpcNAa1IURVEGQmMoWeStt95iwoQJbNiwoW/flClTWLBgAcccc0zS91PFLRYs\nuJmLLppLdbV1pV155SWce+45/U7u2SrLruXdFUWBwcVQCu6SyvVGnqsN33jjjQLIqFGjZNGiRbJt\n27Z+v5+4RmTRosVF5XrSNGJFGZ4wCJdXVQ6EbVhz0UUXsXXrVubMmUNjY+OA3z/ttFM59tij+6yC\n7u5uqqp2xxZW7iWxVEo+Wb58JTNnXtBnKS1ZspDTTjs1r2NQFKV0UJdXkXH33fdw3nlzgAnA68Dl\nhMM38fTTjzNp0qS8jUPTiBVleKNpwyVOb28vF100F1gDPINdX3IdxjRw0EGf7EspzgeaRqwoSqao\noBQRQZM4jOOjjx5k8+Z2Zs68gN7e3ryMJXltjVYaVhSlf1RQioigSRzWY7sf59dC0DRiRVEyRWMo\nRYYXCK+q2osPP3wJuA64jELFMDSNWFGGJ4OJoaigFCHeJP7UU89w0UVzCYXGsHXrOs2yUhQlb6ig\nBFCKguKnnCyEcroXRSl3VFACKHVBKRd0TYuilBYqKAGooGRGLqwIXdOiKKWHrkNRhsTy5SsDS+kP\nFV3ToijDg5K3UIwxJwC3Y8VxiYjMT/hcLZQ0yKUVoRaKopQew85CMcZUAHcCnwb2B04zxkws7KhK\nk1xaEbqmRVGGByVtoRhjDgOuFZHPuPdzsRUy5/u+oxZKGuTDitAsL0UpHQZjoZR6teHdsRUUPd4A\nDinQWEoaz4qYOXNG3LqXbE78zc3NKiSKUsaUuqCkxXXXXdf3urW1ldbW1oKNpZhJLKWvk7+iDB86\nOjro6OgY0jnKweV1nYic4N6ry0tRFCULDLugPLAWGGeMGWOMqQa+CDxU4DEpiqIMS0ra5SUi240x\ns4BHiaUNv1DgYSmKogxLStrllQ7q8lIURcmc4ejyUhRFUYoEFRRFURQlK6igKIqiKFlBBUVRFEXJ\nCiooiqIoSlZQQVEURVGyggpKgejt7WXt2rX09vYWeiiKoihZQQWlAOSqkZWiKEoh0YWNeUabTSmK\nUgrowsYSQNvhKopSrqig5JmWlha2bOkGnnN7nmPr1nW0tLQUblCKoihZQAUlz2g7XEVRyhWNoRQI\nbYerKEoxM5gYigqKoiiKkoQG5RVFUZSCoYKiKIqiZAUVFEVRFCUrqKAoiqIoWUEFRVEURckKKiiK\noihKVlBBURRFUbKCCoqiKIqSFYpaUIwx/2KM+bMxZrsxZnrCZ1cYY14xxrxgjDm+UGNUFEVRLEUt\nKMCfgM8Dv/PvNMZMAk4BJgGfARYaYzJa0VkudHR0FHoIOUXvr7Qp5/sr53sbLEUtKCLykoi8AiSK\nxYnAChHZJiLdwCvAIfkeXzFQ7v9R6/2VNuV8f+V8b4OlqAWlH3YHXve9X+/2KYqiKAWiqtADMMY8\nBoz27wIEuEpEHi7MqBRFUZRMKYlqw8aYduASEXnKvZ8LiIjMd+9/DVwrIk8EHFv8N6goilKEZFpt\nuOAWSgb4b+wh4H5jzAKsq2sc0Bl0UKY/iKIoijI4ijqGYow5yRjzOnAY8J/GmF8BiEgX8FOgC3gE\nuECbniiKohSWknB5KYqiKMVPUVsoQ8UYc4Ix5kVjzMvGmMsLPZ6hYoxZYox5xxjznG/fCGPMo8aY\nl4wxbcaYxkKOcbAYY/YwxvzWGPO8MeZPxpjZbn+53F/YGPOEMeZpd483uv1lcX8expgKY8xTxpiH\n3PuyuT9jTLcx5ln3N+x0+8rp/hqNMQ+4xeLPG2MOzfT+ylZQjDEVwJ3Ap4H9gdOMMRMLO6oh82Ps\n/fiZC6wSkX2B3wJX5H1U2WEbcLGI7A8cDnzD/b3K4v5E5GNghohMAw4AjjbGHEGZ3J+POVhXtEc5\n3d8OoFVEpomIt+6tnO7v+8AjIjIJOBB4kUzvT0TKcsPGXX7lez8XuLzQ48rCfY0BnvO9fxEY7V7v\nArxY6DFm6T5/ARxbjvcH1GCTSPYrp/sD9gAeA1qBh9y+crq//wV2SthXFvcHNAB/Cdif0f2VrYVC\n8uLHNyjPxY87i8g7ACLyNrBzgcczZIwxLcBUYA32P+ayuD/nDnoaeBvoEJtcUjb3BywAvoVdR+ZR\nTvcnwGPGmLXGmLPdvnK5v7HA34wxP3Yuy8XGmBoyvL9yFpThSklnWRhj6oAHgTkisoHk+ynZ+xOR\nHQd5AdUAAAcXSURBVGJdXnsARxpjWimT+zPG/BPwjog8Q3KpJD8leX+OI0RkOvB/sS7ZIymTvx92\nCcl04IfuHjdivToZ3V85C8p6YC/f+z3cvnLjHWPMaABjzC5AT4HHM2iMMVVYMblPRH7pdpfN/XmI\nyD+w6e6foHzu7wjgc8aY14Dl2BjRfcDbZXJ/iMhb7t9erEv2EMrn7/cG8LqI/I97/zOswGR0f+Us\nKGuBccaYMcaYauCL2AWRpY4heZHnme71V4FfJh5QQvwI6BKR7/v2lcX9GWNGeRkyxpgocBzwNGVy\nfyJypYjsJSJ7Y/9f+62IfAV4mDK4P2NMjbOeMcbUAsdjq6GXy9/vHeB1Y8wEt+sY4HkyvL+yXodi\njDkBm7lQASwRkZsLPKQhYYxZhg147gS8A1yLfVJ6ANgTWAecIiLvF2qMg8VlPP0e+z+puO1KbPD6\np5T+/U0B7sU+DFRgrbDvGWNGUgb358cYcxS2VNLnyuX+jDFjgZ9j/7usAu4XkZvL5f4AjDEHAv8G\nhIDXgK8BlWRwf2UtKIqiKEr+KGeXl6IoipJHVFAURVGUrKCCoiiKomQFFRRFURQlK6igKIqiKFlB\nBUVRFEXJCiooiqIoSlZQQVHKFmPMva53xQ5jzDZjTL/FQY0xhxtjNhhjthhjHjfGLMnXWIsVY8xt\nxph17jc8qtDjUYobFRSlbBGRrwL/AvwBu0J9QqrvujpinwfC2AJ5/0dEZuZloEWMiFwC3AJ8jK3+\nrCgpUUFRyp0jsY3JDLB3P987G9v7oRLbSEiJ8UlgrdgmYYqSEhUUpdw5AlvB+CNSCIrrv7IJa8Hs\nAP47T2MrFY4EflfoQSjFjwqKUu40uHLx3aS2UL4iIv8OfAp4vlSL++UCY8zewG6ooChpUFXoAShK\nrnBB+L+6t68RICjGmJOAh1xJ+YOAxQHfmQqcD7wLVGOrPX9DRDb6vnM08HWscDVj3WbfFJGDA863\nJ3A1tonRR267RUQ2uwn8IqAF+ImIrPQdNxv4jIh8xrfvC8AMbDXY04CTsa2F9wBu9/W3yPQ+/hdo\nAv4MbMXGoRSlfwrdy1g33XK1AV8CPute/wDoTfi8HrjYvT4G6+46OeE7Z2L7luzi2/ct77zu/Uzg\nLWBX934MsBn4VcCYIsBLwHj3fm+gF/gcNs5zFzaOMxt4JuHY/wHu9b2vBm52r1/G9h45AhiJFbY7\nhngfe2JF7w+F/lvqVhqburyUcuaTxOIhrwEjjTH1vs/PA+52rz+F7XXxe+9DY8zhWIvlArH9tDHG\nHOy++0f3/kCsCMyRWEe/dcAGgmMxx2EtmNfd+/exvdgfA44GHhWR7cAJWJHwxlIHHEi866kV+KMx\nJoSd/J8SkT8ANVgr5IEh3sfr2NiSuruUtFCXl1LOjJBYPOR/3b97A88aYw4CXpSYu+dI4FWxnes8\nrsNOzCc511IF8AJwmth+9wA3AP/AtkwFwBgzEetOChKU93GuJGPMI8BPReRGd1wX0GuM2Q3bEfAL\nvuM+6a7/u4Rz/RE4HGutPAAgIm9g27cO9T4muftQQVHSQgVFKUuMMTtju1p6vIZLHTbG/An4JxH5\njvtuCDgUWOY7vgprMdwjIpenuEYj8GngAWdVeMzArtt4IvEYEflvY8zlwIXAN4BZxphviMhdnmVg\njDkV+BD4le/QI4G3ROQvvnN1uu/PAP4uIn8OGONQ72MbsDroOEVJRF1eSrnyKXzuK+ItlK9h16Z4\nHAxEiX8S3wkby3itn2uMc99JXPB3FHbdxhbXOjYOEblVRPYCJmNF5xsJXzkeaBeRrUH3E3DOGaS2\nIoZyH61YN9rGoPtQlERUUJRy5Uh8giIiHwJ/d/urXHzAw4uf+Cflv2FdQElWvDFmb2PMue5ziGWS\n4bLFWollRc3xffYfxphnfGN6AfgJyZP9XtjAvf+cn/DdzyW+zyLAYUBH4jizcB9H+c47B0UZABUU\npexw7q5jsALi53+BscA9CfuPBt7wi4xz/dyNDY77z30k8G3g30XkFeBZbIqv5176AbZ8yzpjzE7Y\nDC6Pg4BVCeP8EvCdhPG8gs3U8rgKa0F0G2P2x8Y/PI4AQqQQlCHcxw+xcZm/GGOagbeDzq8ofoyI\nFHoMipIV3AT+M2xAuhabOvs9EbnLfb4QO4GucZPmQ8BoYCp2LUgn8B8i8gP3/UrgZuyajr9iJ9jn\nROTHvmuOw2ZpvYyd9H+AneTPxArDFSLyrvvukdin/ijQgM3Gukt8a0Xc98ZiM65exMZ97gE+g7V8\nXgMuEZEt7rtfBmaJyGH9/C5DuY+vAq8Cl4su+FQGQAVFURRFyQrq8lIURVGyggqKoiiKkhVUUBRF\nUZSsoIKiKIqiZAUVFEVRFCUrqKAoiqIoWUEFRVEURckKKiiKoihKVlBBURRFUbKCCoqiKIqSFf4/\n3B55OhWmhZIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11c2ff350>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.scatter(y, predicted)\n",
    "ax.plot([y.min(), y.max()], [y.min(), y.max()], 'k--', lw=4)\n",
    "ax.set_xlabel('$Measured$', fontsize = 20)\n",
    "ax.set_ylabel('$Predicted$', fontsize = 20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 训练集和测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  6.32000000e-03,   1.80000000e+01,   2.31000000e+00, ...,\n",
       "          1.53000000e+01,   3.96900000e+02,   4.98000000e+00],\n",
       "       [  2.73100000e-02,   0.00000000e+00,   7.07000000e+00, ...,\n",
       "          1.78000000e+01,   3.96900000e+02,   9.14000000e+00],\n",
       "       [  2.72900000e-02,   0.00000000e+00,   7.07000000e+00, ...,\n",
       "          1.78000000e+01,   3.92830000e+02,   4.03000000e+00],\n",
       "       ..., \n",
       "       [  6.07600000e-02,   0.00000000e+00,   1.19300000e+01, ...,\n",
       "          2.10000000e+01,   3.96900000e+02,   5.64000000e+00],\n",
       "       [  1.09590000e-01,   0.00000000e+00,   1.19300000e+01, ...,\n",
       "          2.10000000e+01,   3.93450000e+02,   6.48000000e+00],\n",
       "       [  4.74100000e-02,   0.00000000e+00,   1.19300000e+01, ...,\n",
       "          2.10000000e+01,   3.96900000e+02,   7.88000000e+00]])"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "Xs_train, Xs_test, y_train, y_test = train_test_split(boston.data,\n",
    "                                                      boston.target, \n",
    "                                                      test_size=0.2, \n",
    "                                                      random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "lm = regr.fit(Xs_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(30.288948339368776,\n",
       " array([ -1.12463481e-01,   3.00810168e-02,   4.07309919e-02,\n",
       "          2.78676719e+00,  -1.72406347e+01,   4.43248784e+00,\n",
       "         -6.23998173e-03,  -1.44848504e+00,   2.62113793e-01,\n",
       "         -1.06390978e-02,  -9.16398679e-01,   1.24516469e-02,\n",
       "         -5.09349120e-01]),\n",
       " 0.75088377867329148)"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm.intercept_, lm.coef_, lm.score(Xs_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "predicted = regr.predict(Xs_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEYCAYAAAB7twADAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VPXZ8PHvHbINS4JI3CoSLLKoKMGCWLUEAWuf51V4\nvCpK3VpxX8AFH8DSilJUhL5KWzFoI6Uoi7Z9LL5XFaQStT5CIiJaFoEikaCQWBWVRJLA/f5xziST\nyWSZyZnMkvtzXedi5pwz5/wOgbnz2+6fqCrGGGNMW6XEugDGGGOSgwUUY4wxnrCAYowxxhMWUIwx\nxnjCAooxxhhPWEAxxhjjiYQIKCKSLSIviMhWEdksImeLyFEislpEPhSRVSKSHetyGmNMR5YQAQWY\nD/xNVQcCZwLbgGnAGlXtD7wGTI9h+YwxpsOTeJ/YKCJZwEZV/W7Q/m3ACFXdLyLHAUWqOiAmhTTG\nGJMQNZQ+wGciskhE3hWRp0SkM3Csqu4HUNV9wDExLaUxxnRwiRBQUoEhwBOqOgQ4iNPcFVy1iu+q\nljHGJLnUWBegFcqAPar6jvv+zzgBZb+IHBvQ5FUe6sMiYoHGGGMioKoSzvlxX0Nxm7X2iEg/d9co\nYDOwEvipu+9a4K/NXCNpt/vvvz/mZbDns+friM+XzM+mGtnv4YlQQwGYBDwnImnALuBnQCfgeRG5\nDigFxsewfMYY0+ElREBR1U3A0BCHRrd3WYwxxoQW901epnn5+fmxLkJU2fMltmR+vmR+tkjF/TyU\nthIRTfZnNMYYr4kImmyd8sYYYxKDBRRjjDGesIBijDHGExZQjDHGeMICijHGGE9YQDHGGOMJCyjG\nGGM8YQHFGGOMJyygGGOM8YQFFGOMMZ6wgGKMMcYTFlCMMcZ4wgKKMcYYT1hAMcYY4wkLKMYYYzxh\nAcUYY4wnLKAYY4zxhAUUY4wxnrCAYowxxhMWUIwxxnjCAooxxhhPpMa6AK0hIruBA8ARoEZVh4nI\nUcAKoDewGxivqgdiVkhjjOngEqWGcgTIV9U8VR3m7psGrFHV/sBrwPSYlc4YY0zCBBShcVnHAovd\n14uBce1aImNMh6SqfP7557EuRlxKlICiwKsiUiIi17v7jlXV/QCqug84JmalM8Z0GLNmzWLcuHGo\naqyLEncSog8FOFdVPxWRHGC1iHyIE2QCNfnTnTlzZt3r/Px88vPzo1FGY0wHMHHiRObMmcOf/vQn\nLrvsslgXxzNFRUUUFRW16RqSaFFWRO4HvgGux+lX2S8ixwFrVXVgiPM10Z7RGBPfZs2aRWFhIdu2\nbSMzMzPWxYkKEUFVJZzPxH2Tl4h0FpGu7usuwIXAB8BK4KfuadcCf41JAY0xSUdVWbZsGZs2bQp5\nfMqUKYwZM4avvvqqnUsW3+K+hiIifYD/wWnSSgWeU9VHRKQH8DzQCyjFGTb8ZYjPWw3FGNNq69ev\n56677uLtt99mxIgRrF27FpGwflFPCpHUUOI+oLSVBRRjTGuUlZUxffp0nn322Qb7//znP3PppZfG\nqFSxYwElBAsoxpiWVFdX06dPHz755JNGxwYMGMDmzZtJSYn7HgJPJWUfijHGRFt6ejp33313o/1j\nx47lpZde6nDBJFJWQzHGGJxaymmnncbOnTsZNGgQjz32GKNGjYp1sWLGmrxCsIBijPHbu3cvTz/9\nNL/85S9D1jpeeeUVSktLuf766+nUqVMMShg/LKCEYAHFGFNZWcm8efOYM2cOlZWVLFmyhKuuuirW\nxYprFlBCsIBiTMfln08ydepUysrK6vZ/5zvf4cMPP6RLly4xLF18s055Y4wJ8MILL3DllVc2CCbg\nNH0VFhbGqFTJy2ooxpikVVtby+DBg9m8eXPdvp49e/KrX/3K+klaYDUUY4wJkJqaymOPPQZAWloa\nU6ZMYefOndx0000WTKLAaijGmITm7ydJTU1l/PjxIc95+OGH+fGPf8wpp5zSzqVLXNYpH4IFFGOS\n1/r167nzzjtZt24dxxxzDDt27CArKyvWxUoK1uRljOkQysrKuPrqqxk+fDjr1q0DoLy8nNmzZ8e4\nZB2b1VCMMQlnxIgRvPHGG432d+3albKyMrKzs2NQquRiNRRjTIcQqiYyduxYNm7caMEkhiygGGMS\nznnnnVfXAT9o0CDWrFnDiy++SN++fWNcso7NAooxJi6VlZVx44038u9//zvk8Tlz5lBQUMDGjRs7\ndBLHeGJ9KMaYuFJZWcncuXN59NFHqays5I477uA3v/lNrIvVQEVFBbt37yY3N5ecnJxYFycqrA/F\nGJOwVJWlS5fSv39/Zs6cSWVlJQALFixgy5YtMS5dvWXLVtC79wDGjLmZ3r0HsGzZilgXKW5YDcUY\nExc++OADzjjjjJDHbrvtNn73u9+1c4kaq6iooHfvAVRVrQXOAN7H5xtJaem2pKupWA3FGJOwBg0a\nxDXXXNNgX05ODgsXLmT+/PkxKlVDu3fvJj09FyeYAJxBWlpvdu/eHbtCxRELKMaYuPHwww/TuXPn\nurxbO3bs4MYbb2z3vFsVFRWUlJRQUVHRYH9ubi7V1buB990971NTU0pubm67li9eWUAxxrQbfz/J\njBkzQh4/4YQTWLx4MVu2bGHu3LkxmVPSXB9JTk4OhYUL8PlGkpU1BJ9vJIWFC5KuuStS1odijGkX\nxcXFTJ48mXXr1iEibNiwgby8vFgXq4HW9pHYKK/QEqaGIiIpIvKuiKx03x8lIqtF5EMRWSUiNj3W\nmDi0d+9errnmGs4+++y6vFuqyl133UW8/bJnfSRtkzABBZgMBI4dnAasUdX+wGvA9JiUyhjTrIcf\nfpglS5Y02v+///u/DRa+iget6SOxYcPNUNW434ATgVeBfGClu28bcKz7+jhgWxOfVWNM7JSXl2tW\nVpYCddvYsWN1+/btsS5aSEuXLlefr4dmZeWpz9dDly5dXnesvLxcfb4eCpsUVGGT+nw9tLy8PIYl\njg73uzOs7+pEqaE8BtyL84/R71hV3Q+gqvuAY2JRMGNM83JycvjlL38JwOmnn16Xdyucxa6aGnUV\nDRMmXE5p6TbWrFlIaek2Jky4vO6YNYk1LzXWBWiJiPwnsF9V3xOR/GZObbIxdubMmXWv8/Pzyc9v\n7jLGmHDt3buX6dOnc8stt3DOOec0On7HHXfQs2dPrrzySlJTw/vaWbZsBRMn3kp6utMcVVi4oMGX\nfDTk5OSE7Gxv2CTmdNony7DhoqIiioqK2naRcKs07b0BDwEfA7uAT4FvgCXAVho2eW1t4vNe1P6M\nMSEcPHhQH3jgAe3cubMCevbZZ+vhw4c9u348NjE11ySWTIigySuhhg2LyAjgHlW9REQeBf6tqnNE\nZCpwlKpOC/EZTaRnNCYRqCrLly9n6tSp7Nmzp8GxJUuWcNVVV3lyn5KSEsaMuZkDBzbU7cvKGsKa\nNQsZOnSoJ/eIhA0bbuIzifRlGxRQegDPA72AUmC8qn4Z4jMWUIzx2IEDB/jud78bMrX8RRddxMsv\nv+zJfTpS7qx4k9TzUABU9XVVvcR9/bmqjlbV/qp6YahgYoyJjuzsbGbNmtVgX8+ePSkoKOCll17y\n7D42Mz2xJFQNJRJWQzEmOmpraxk8eDDbt29n0qRJzJgxg+7du0flXh2hiSneJH2TVyQsoBgTGX8/\nydKlS3nxxRdDJmh85513yM7ODmsIsEkMFlBCsIBiTPiKi4u58847efvttwF4+umnuf7662NcKtOe\nPA8obif4OlU91NbCxYoFFGNazz+fJDhVyjHHHMOOHTvIysqKUclMe4tGp/xa4EsReUNEZonIaBHx\nhbhxHxF5XESmiUh0GlGNMVH36quvhsy79cUXX/Dmm2/GoEQmkbRUQykFHgXOA84HTgBqgHeA193t\nH6p60D0/B/iVqt4U5XK3mtVQTDKJduf0kSNHGDZsGBs21M/7GDt2LPPmzaNv376e38/Er2jUUP6k\nqk+o6gRVPRHoC9wNDAZuAF4GvhCRdSLyCDDOPccY47H2yHKbkpLC448/DjhL8vrzblkwMa0Rdqe8\niDwJPKSqe0RkIDASJwvwCCAdmKyqf/S6oJGyGopJBl5O8CsrK+O+++5j6NCh3HHHHSHPWb16NRdc\ncEHYebdM8miviY1fquoeAFXdqqoLVHU8kAv8D/BWBNc0xjTDiyy3lZWVPPjgg/Tv358lS5Zw//33\nh5zpDnDhhRdaMDFhiySg9BGR44N3qmoVcCMQ+lceY0zEWrPwU1NUnXXc+/fvz/33309lZSXgdLQ/\n8MAD0Sqy6YAiCShPAe+KyHgRaTDTSVVrgcOelMwYU6etKUgWLFhAWVlZo/1vvfUWNTU1XhfXdFAR\nTWwUkYnAk8BXwGpgHU6K+VxglKpe7GEZ28T6UEwyiXSU1zvvvNMgO69IKqmp6Sxa9HuuvHJCNIpq\nEly7zpQXkTOB2cAPAX9NZRNwmarujOiiUWABxXQkqopI6O+AK664ghUrVgDXAPOBjy1zr2lSu2Yb\nVtVNqvp/gJ7AUKC/qubFUzAxpqPw95Ocfvrp7Nu3L+Q51157LV27ngosBrpjy9car4UdUESki4hc\nLCI9AVT1gKpuAC4INYveGBNd69ev5/vf/z5XXnklW7ZsYcaMGSHP+973vsfhw/uIpGPfmNaIpIby\nBDAAWCQiXQP2v+oeM8a0g08++YSrr76a4cOHs27durr9zzzzDBs3bmx0vq0tYqItkomNv1bVe0Rk\nKHCmqv4+4Nj5QF9VXeRxOSNmfSgmFtpj/Y4PPviAwYMHc+TIkQb709LSeOKJJ7jhhhtiVjaT+Nqr\nD6UaQFVLgIGBB1T1TepnXhnTIS1c+DS9evVj1KibopYiBZzUKDfeeGODfWPHjmXLli1NBhNwaipD\nhw61YGI8F0lA6SEix7mvj4Q4Xt2G8hiT0BYufJqbb57MoUOv8/XX71JVtZbrrruZ1atXU1FREfF1\nDx8OPb3rwQcfJDs72/JumbgQSZPXYGARcCswTlWnBh1fpqpxM7DdmrxMe6moqKBXr74cOtQHeM/d\nuwK4ji5dTuHIkT0UFi5gwoTLW3299evXs3jxYtLT03nuuedCnvfBBx9w6qmnhlxR0ZhItds8FBH5\nKc6M+c+BvwC7gFTgIqBEVe8N+6JRYgHFtJeSkhJGjZrI11/vxVlK6HigP1BEuAkd//CHP3LDDTdT\nW3sIf0PAW2+9xfe///2wy2V9JiYS7TYPRVX/gJNdeBNOGvtHgWnAevdPYzqc3Nxcamv3AlNxknB/\nH+hBuAkdn376aa677mfU1lYR2Kp8xx13NOqAb0l7pLw3xq8tExvfVtUfAj6cX8WOVtWpqmq5vEyH\nVD8sdw5du55Aevo+0tIqaO28j4qKCkpKSnjllVdQbRw4vvrqK8rLy1tdnoqKCiZOvJWqqrUcOLCB\nqqq1TJx4a5v6coxpTiQTG28JfO8mhEzFmTHvORHJEJH1IrJRRDaLyEPu/qNEZLWIfCgiq0QkOxr3\nNyYcEyZcTmnpNl577RnKynaxePFTrZr3EViT+NvfXgs6ehRpaZ158803Oe644xp9tilNpbzfuHEj\nJSUlFliM51rsQxGRXsB+Va123/9NVf8j6JxsYDJQqarzPC+kSGdVrXSzG78F3ANcAvxbVR8VkanA\nUaraqLnN+lBMsPbuU6ioqKibaJiXl1d3z+rqatLT00MuntWp09kcPvwt6enHkJJSzTPPFLS6Mz/w\nvsHXTUs7j9TUNNLTnXT44QwSMB1LJH0oqGqzG7AXqMTpZXwAeANIb+Lc7wH3tHTNSDegM1AMnAps\nA4519x8HbGviM2qM39Kly9Xn66HZ2UPU5+uhS5cub/d7PvfcMn3uuee0V69e+tprr2lxcbFmZw9R\n0LqtW7fBumjRIi0uLtby8vI23zsrK08zM7trenq2wib3PpvU5+vRpuub5OV+d4b3Hd3iCXAycCfw\nJ2AfTi9hlRtYZgGjgc4B5z8abiFaUYYUYCNOuvxH3X1fBJ3zeROf9fZv2SSs8vJy9fl6tOsXauN7\nLlGRTgoooGeeeaZ++umnUS1XeXm5FhcX66pVqxoFrqysPC0uLvbkPia5RBJQWlzjU1V3AY+7GyKy\nDngGZxjL9cDPgRoReQ/YDZzQyspRq6nTQ5knIlnAKhHJd/9DNjitqc/PnDmz7nV+fj75+fleF9Ek\ngI0bN5KS0otQo65ycnKi0hTm78eoqjoBuBp4lsAW2E2bNvHSSy9RWLiAiRNHkpbWm5qaUk9zbOXk\n5NQ9X/2qj04TmCWHNH5FRUUUFRW17SLhRiDg5aD3pwK3A8uBF4Azwr1mmPf/BTAF2ErDJq+tTZzv\nUbw2iWzp0uWamdldoXPImkC0msLqayhvKxxfVzPxb2lpaTp79uy6c9vaxNWSwCaw9mryM4mJCGoo\nkcyUH6+qz7ctjIV1v55AjaoecNPjr8Lpy7kQp5lrjnXKm+Y07JzeCtwC9MDn+4LCwgWMHn1Bo85r\nLxeeWrZsBRMn3opqV7799uO6/WPHjmXevHntnirFJjqa1ohKp3zwBmQAFwMnBO0/F+gU7vVacb9B\nwLs4fSibgCnu/h7AGuBDnGWIuzfxeS+CtUlgjTu9y7VLl366atWqJo63rW/hm2++abSvvLxc161b\np2eeeaYOGjRI16xZ06ZnMibaaKcaygPAXcCHqjo0YP9A4DZVvT2sC0aZ1VBMqOGzgTWQlo63VllZ\nGdOnT2fDhg1s2rSJtLS0Rud88sknHHvssZZ3y8S99kq9kgH8J/Bg4E5V3Qr8VUSuiOCaxkRN4MJS\nXbsOIiPjfB577JG6YNHWhacqKyt54IEH6NevH88++yxbt25lwYIFIc894YQTLJiYpBVJDeVxVb0z\n0uPtzWooHU9TfQQLFz7N5Mn/TXp6H2prSxtN6mtqAmJzVq5cyW233UZZWVmD/d27d2fnzp0cffTR\nHj2VMe2rvWoo2SJyejPHD0VwTWM80VQyxIqKCu66a1qDdUqC81qtWfMa48ZNYPz46a1OpFhVVdUo\nmAD06tWLffv2efdgxiSCcDtdcDrJ9wJX4tZwgo4/Fe41o7lhnfIdRnMTF1vqeI900uORI0f03HPP\nrRsG3LNnTy0oKNDa2tr2eGRjooYIOuXDrqGo6gc4nfKFwEci8hsRuVZELhWRx4F0LwKdMeFqKhmi\nv/mrflIfBE/qa+6z4PSTlJWVNUqqKCI8/vjjZGRkcM8997Bjxw5uuummFvtJ/JmFLUGjSSrhRiCt\n/80/D3gVOIyTjuUI8AqQFek1o7FhNRTPtMfEu7ZoqZYRPKlv1qzZdce2bNmiGRlZCmsbfHb//v26\ndOlSPfroozU11dfkxMdw/k5ikU/MmHARjVxeLV7ASVs/DMht67WisVlA8UaifAmGmgkeGAjLy8t1\n1qzZmpnZve5Zbr99svp8PdTnG6Tg08zMXPX5euiDD/5KzznnnICZ7Z0VPm1Trq1Y5BMzJhIxCSjx\nvllAabtE+xIMDCDBgXDu3F9rRkb3gGdZq+Bz35crPKdpad30kksuaZQmxdkmtmnio9eTKI2JlkgC\nSrN9KCJyt4iMaqnZTET6iMjcFkZ/mQTVUv9CrDTVD5GTk8PQoUP57LPP+NnPbm6wYuG9987g0KFs\nnMGIFUAXoBdOSpYBwK+pqall8+bNIe6YipOgYVPESRVb6ssxJqE1F22AHcCGoH2XA+8BS4HRAfvT\ncNLcXx1uVIvmhtVQ2iweaygtNcEtXbrc7RPp16A2AAMVMhTOVOih8AuFTIWjGjxfRkaWpqSk1NVM\nzjrre5qRke1JUkVL0Jj44r0/0Qt43eSF8+tYcM6ul3BWZ/R3yO8C7gOOd4//OtxCRHOzgOKNePoS\nrA9waxWKFdY2CHANjzcMhE7z1tqA95l68cXjFPo2aoa69NJL9fTTT6/Lu+Xll0hH+EJKVonSn9hW\nngeUkB+A31E/w/5k4CHgE6AG2ACsDfea0dwsoHgnXr4Ei4uL1ec72Q0WQxR6aGZmbl3Z/vCHP2i3\nboPc4PCUG0TOUOiu0NvdX6ZwtUKKLlmyJGQNbNeuXVpTUxPTZzXxJR5r69ESSUBpcYGtEJ4AFojI\nX4A1qnqfiPwCuAjoh7MuiklC/oWaYsWfUqW6upqqqk+BdfiTOX777XBef/1NRoy4iNTU3nz99U7g\nUZx14E4Gfg90xUmKfSuwGGdla1i+fHnIBa769OkTg6c08ax+wbTQi7R1dM3m8hKREcA6VT0UtD8V\n+C/32J7oFrFtLJdXcvCvKZKensu33+5E9TtUV2+pO56ZeTqqezh06E38QQaG06VLbw4eLMUJPl8A\nY4EDja4/c+YD3HrrLbZOiGmWV5mpE0EkubxaCihHcIbDlACvu9tbqloVdF4fnH6VfUCBqn4ZZtmj\nxgJKYglM7AjOb4Rdu3blrLPOC/hPXAT8B4E1lIyMEaSn9+Lrr9+vu1a3bnn89rd38u231dx11zRS\nUo7l4MGtIe7al4yMcvbs2Zl0XwrGe/5fbgJrs4FJRpOF5wtsAaXAbcAyoAxnNvwh4C2cvpMLgS4B\n5+cAC8Ntd4vmhvWhJAx/Z2e3bnmalpalqaldNDt7iKand9WMjNMadJpnZuZqRkb3ukECBQVPNdu2\n7e//ufnmmwPmlPRUKFCosbkgJizx0p8YTXi9wJaI/FpV7wl4fzLwI5zG6UrgaKAWZ0XFIuBfwBWq\n2uLclfZiNZTEEKopAYYDVwHP44z5eJvAZoY//OFJwEkVn5eXx5o1rzFx4q2kpvaipqaUZ54pYPTo\nCxo0Y3355ZcMHDiQzz77ktraV4Hz6q63YcM/+Oabb6zJyxjabwngJ4Fe7uuBOD2czwP7cRqprwn3\nmtHcsBpKQiguLtZu3fKC5owMcueMbFJY7s4V+a6mpWVpWlrXulQpPl8f9fl66HPPLdMnn3xSe/bs\nqeeee25drSV4eGdlZWWjYdD+9CvJPhTUmNainZYAflhVp4fY78MZATZbVf8V1kWjyGooiaGiooJe\nvfpx6NDr1NdQRuBMhfL/c6rA5xvO4cOfUV0d2Pk+EvgVInegejjgqunALOC/CdV56u+vadxHk7wd\nrca0VnstsNVHRI4P3qlOR/2NwB0RXNN0cDk5Ocyf/yhOM9eZOEFiIs7SO0XuWZ9y5MhnZGR8l8A0\nMCDArUHBBCAXeAQnxUrjdDH+FC3ffPNNXKaWMSbRRBJQngLeFZHxItJg0QdVrcWZPW9Mq/lzcl16\n6TgKCuaTkbGbjIyuwAJSU/sA/0FmZh98vpHMnz+PQ4d2EZgLCw6GuGoacAnwHWA3zeXMsvxaxngj\nkgW2XgNmAM8C+0VkqYhMEpFxInInzuRGY1oleMnerKwsNm5cB3wFrKO2diuwDtUv2bDhH/zgB+dx\n+HA18AOcxUOHA91wail+I4EtwNXAv+ja9Tp8vpEUFi4I2YSVk5NDYeECfL6RZGUNafZcY0zTwu5D\nqfugyJnAbOCHgL+msgm4TFV3elO8trM+lOYFzvvw6gu0tddsapLYiy8uY/z46Rw4sKHu3KysIVx7\n7Q8oKHiGmpqjcALOvcB/0aXLOG655RJWrVrFxRdfwmOPPVk3R+DBB39OTs7RDBs2jIEDB3pSbmM6\ngnYZ5RW8AdnAWcApbb1WE9c/EXgN2Ax8AExy9x8FrAY+BFYB2U18vi0DHZJaNJLchXPNWbNmh0zK\nuGrVqqA5Jas1JSXNnTvS1c3fdZT72kkMuW/fvrq8W/45Ak2N8go8J5nnERjTFiTjAlvAccBg93VX\nN4AMAOYA/+3unwo80sTnvfnbTTLRSHLX+JprNSMjS7ds2RLy3MzM7hqcNj4z8yidNGmyimS4Q4Z7\nBExERCFXodr9TGdNT+8aMmg193wdJVusMW3RLgEFZ0Wii4GeQftvAnzhXi+C+78IjAa2AcdqfdDZ\n1sT5Xv39JpVorBzY8JrL3WDQTzMyujf60q4/139enkJnN5D4FHopSFAw8W+Pu/f4rq5YsSKs52tc\n+0nebLHGtEUkASWSUV5PuDWERSLSNWD/q+6xqBGRXGAwThKnY1V1P4Cq7gOOiea9k000RjbVX7MI\nZ77rWuBDDh16nYkTb22wsmL9uQNxfje4ETiMaibOj/dDoNHodOAUnGHF75Oe/hkjR44M6/kAGyJs\nTJREkr7+36o6V0SKgCtw8oKjqrtEZJGI/ExVF3lZSAA3eP0JmKyq34hIcE97kz3vM2fOrHudn59P\nfn6+18VLOP6RTcEp29vaGX3fffcwa9bFVFefQKgv7ZycnLrO7wcf/DkzZowgPT2X6updwIkcOpQV\n8LnHgfHua+EnP/kJf/nL3+jUaTJHjpRRWLiwyfI29Xx5eXkBgcYZCGBDhI2BoqIiioqK2naRcKs0\nwMMBrxutzgg8Fu41W3HPVOAVnGDi37eVhk1eW5v4bJurfokk3M5mLzqny8vLddas2XX9EpmZ3TUt\nLavZ/ovMTGcp3rS04zUtrYtOmjQ5RJ/KewopmpKSpr///TOePV88rT5pTLyinfpQFgLHua/nhjg+\nJ9xrtuKefwT+b/B9gKnua+uU19gsTbp06XI3EHRuEECcXFsNv7TrO8rnuIHjJLefpLuCT1NTu2in\nTp3da31XU1O76dSp06LSv2GjvIxpXiQBJZJcXoOBRTiN5ONUdWrQ8WWqOiGsizZ/v3OBN3CGDPs7\nZu8DinGSUvbCSbM/XkOsw9JR5qHEYuGf+nv+Fvg1zgrQjqysIbzwwiMcddRRdfM6SkpKGDVqIl9/\nXYqTFuX9gKsdC/xfMjNvY/HihXUZhG0+iDGxEck8lLD7UFT1PRGZj7PY1uci0g3Y5V7rIpzFuDyj\nqm9RP3Ey2Ggv75XIorE0aUsT/ervOQYnhVvDfonggJCbm0tl5YdANQ2DCTjJqs8gPb0Pffr0YejQ\noWGXxxgTW5GM8kJV/4CTCnYTcAPO+ijTgPXun6adeT1qa+HCp+nVqx+jRt1E794DWLZsRTP3/BRY\nAOQDfZtNXTJu3Ngm7nghcKDJMgenaAlVHmNMjIXbRha84dRMjgU6tfVa0djogH0obe1sLih4yp0L\n0vJcjcAkMdJ6AAAYRElEQVR7ZmZ211mzZjc4L3jWelZWnoqk1M0pEUnVTp18zZY5GpMwjTHNo536\nUL4HXAO8oKpvehncoqGj9KH4tbVZyFmXpC+HDvUB3qvb361bHn//+1Pk5uY2un6oexYXF7N581Zu\nu+1uUlN78/XX24CZOGuT/BmRy3n44dlcd911AGzcuBGAXr16NVo1saSkhDFjbm6U22vNmoUhm8aM\nMW3XXis2PouTD/zdcD8bi40OVEPxgrNy4iB39np9jSAjo3uzubH89uzZo1dddZUCmpqa2eAazjXL\nFVS7dRtcNyvfX8vx+U5WZwXGQXXrxBcXF+uWLVushmJMO6Odhg0/grOM3onhfjYWmwWU8DQc2ttD\n4QwFn86d++tmv9QPHjyoDzzwgHbu3DkgRYoorKwLIs61iht8tv5+a4OC2BwFn3br5l+id5LNHTGm\nHUUSUCKZKb8FGKjO6CuThO677x5mz36Y1NQTqanZzfz58xkyZLA7out4nIF8uXWjyESEvLw8ysrK\ngq6kOK2jKThThXbQtet1HD78SV2nfUlJiXvdLjhDic/AWWFxDrCOr792Ro0VFo5kw4Z/NGoOM8bE\nj0gCygvA8yLyLk4/yj89LpOJkWXLVjBx4q2kp+ciksLUqRO46aYb6tKlVFbuAPoDfYCPqKqqITc3\nl549ezJkyJAQAeUU4EkgBxjO3Lm/YsSI8xsEhPqRYgfxr6wIh3CmF9UPga6qymL+/N8ya9YDFkyM\niVfhVmlwsv1uB6pwlvstxwkytwGnhXu9aG9Yk1ertDSSqry8XNPTsxscT0/Prju+fft2TUtz1izp\n3r27Zmb2Uqity/Qb2GcSrD4dS66CTzMzBwSNMpvjvu9rzV3GtBOikW1YRG4Vkd0i8rmIvAR0U9V+\nQBbOWqtP4PwK+mvgfRFZ53XQM9Hnn6TYVBbe3bt3k5nZB2cConM8M/PkuuOnnHIKU6ZMYcqUKbzz\nzjuIHMRZEw3gfWprP25yTsyECZdTWrqNN954ni1bNvDGG39k2rQ7gXNwlvmdiZOBeAdVVWsbZS42\nxsSJ5qINTjbhIziN2gfd1wdwEjWOxQku/nMzgVHAVeFGtWhuWA2lVZqroRw5ckSffPJJd/5IqsKO\nFkda3X77JLdWcYqCT2+/fVLY5XFyhD2ozlopWre1dd0WY0zL8HqUF/A2To4sAAGGA8uBfwFjgGHh\n3rC9NwsorbN06XJNS+tal5gxLS1LCwqe0nXr1unw4cMDRm6hqanZzTY9NRy5Vaz+ZXrDHebbVOJJ\nGzJsTPRFElBa6pTvrKrP+7+VcdodrhCROUAPVX21jRUkE2MVFRVs3LiR6667mZqat3Cy59xDTU1P\n7rjjTmpqKht9prb2ACtW/JlLL7005DXrc3zl1+0LJ6+Yf6Lk6NEX8PHH21m48GkeesjbdVuMMVHQ\nXLQBVjexvytQGG70isVGEtZQvEq97u8M79LlTLcW8FTQXJCXG9RMnC1VU1MzdefOnc2WL9KJiE2l\n4Ld088a0L6LQ5PW3Zo49Fe7NYrElW0Dxas2TUF/6kK0wqEF/RVpaj4BgMlZhe6v6MCLJK2Y5u4yJ\nH5EElJaavJobBVYbYaXIRKiiooKJE2+lqmqtm6b+fSZOHMno0ReE3QTUMN19MZAN9MQZEV6fhj4l\n5TApKakcObIAJ7F067IYT5hwOYMHn0FxcTHDhg1j4MCBYZYJvEjBb4xpPy0NGz5PRB4RkR+JSFa7\nlMg0qaWhveHIzc3l0KFdwMXA2cDPyMz8nGnT7sHnG0lW1hB8vpEsWrSQZ599Fp9vWt2+1vRhLFu2\ngrPOOo/Jk3/DWWedFzLdfEVFBSUlJXVDgL1OwW+MaWfNVV9whgkfwZnAWIuTfva3OMOJn23iM/eE\nW02K5kYSNXl51STkz7uVkZHRoH/k3nun1t0nuL8inD6M1pSzqaa7RFzv3fp3TDIiCn0om4DTgFtw\nhgvvDQgwh4F9wAr3+AD3M2vDLUQ0t2QKKKpt/8Ktra3VU089NURnO9q/f3+tqalpcxmLi4s1O3tI\nk3NHWjMrP1G+oL3q0zIm3kQjoCwNsa8vcB2wGPgoqBazH6gKtxDR3JItoKi2/Qt39uzZjYLJ6aef\nrmvWrPGsfM0FjJYCTqKwQQQmmXkeUFp1ATgJuAp4GmfC4+G2XtPLLRkDSltVVlZqr169FNCjjz5a\nCwoKPKmZBGquJpUsX8TJEhiNCSWSgBLRmvKBVPVjVX1WVW9Q1e8C29p6TdN2lZWVFBYW+oNqAy++\nuJL9+78gNTWHgwcPk5XVndTUSBJPN82fn2vNmoWUlm5jwoTL647l5ORQWLigQed/Ik5WtEEExjQU\n9hLALV5QZLWqXujpRdugoy0BXF5ezlNPPUVBQQF79+7lhRde4Mc//nHd8YqKCk488RSqq9/APzQ4\nPf0HlJXtaPcv9LYuVxwP/Cn/A2fxBwZPYxJVJEsARyOgHK+qn3p60TboSAFl1qzZzJw5kyNH6qcI\n5ebmsnXrVjIzMwFYvXo1P/zhbcCOgE/2ZdWqBVx4Ydz8HpBQkiEwGhMskoDS5iavYNEIJiJSKCL7\nReT9gH1HichqEflQRFaJSLbX900ky5cv55e/nNEgmIAzd+V3v/td0NmfENhMA3ET/xNSTk4OQ4cO\ntWBiOjzPA0qULAJ+GLRvGrBGVfsDrwHT271UceSkk04iJSUjaK9w5ZVXcv3119ftycvLIy0tBcgH\nhgD5pKWlkJeX136FNcYkpYQIKKr6D+CLoN1jcYYu4/45rl0LFWdOOeUUUlPTAvaMJCMji8cee4zu\n3bvX7c3JyWHx4t+Tmal06XKQzExl8eLf22/Xxpg287wPJVpEpDfwkqqe4b7/XFV7BBxv8D5gf1L1\noRQXF/PRRx9x8sknN2qzX7p0Oddc81MyMnqh+nmzHcTW7m+MaU4kfSjejhWNrSajxsyZM+te5+fn\nk5+f3w7F8dbevXuZPn06S5YsAYSsrDOoqdnTIGj85CdXMGbMqFYFipycHAskxpg6RUVFFBUVteka\niVxD2Qrkq+p+ETkOJ+VLo5S28VpDaW0NobKyknnz5jFnzhwqKwMXu/oF8GN8vpGUlm6z4GCM8VRc\njPKKInE3v5XAT93X1wJ/be8CRWrZshX07j2AMWNupnfvASEz8fpddtll3H///UHBBGAucHzE2YaN\nMcZrCVFDEZGlOMOSjsbJF3Y/8CLwAtALKAXGq+qXIT4bVzWUiooKevceQFXVWvwTC5urZfz9739n\n9OjRQXv7AgVAjtVQjDFRkbQ1FFX9iaqeoKoZqnqSqi5S1S9UdbSq9lfVC0MFk3gU7pomo0aN4pJL\nLgGgZ8+eTJx4PZmZ/yYr696ETVlijElOydQpnxAa5n9yaijV1btZuXIl/fr1Izu78fzMefPm0a9f\nP2bMmEF2djYPP/yQjdAyxsSdhGjyaot4a/KC+vxPqakn8e23O+jWLYPPP/+cKVOmMHfu3FgXzxhj\n4iOXV7yJx4AC8Morr3Dvvffyz3/+s25fWloaW7ZsoW/fvjEsmTHGWEAJKR4Dys6dO+nXr1/I1PLX\nXHMNixcvDvEpY4xpP0nbKZ8oKioqKCkpoaKiotnz+vbty+WXN5zBnpaWxpQpU/jNb34TzSIaY0zU\nWEDxSDhzSwAeeeSRupTyY8eOZcuWLcydOzdkp3xzWhvEjDEm2qzJywNNzS35y1+eY/v27UyaNCnk\n5xYuXEjfvn0ZNWpURPet79z/DtXVpcyfP4+bbroh8gcxxhiX9aGE0B4BpaSkhDFjbubAgQ3unjLS\n0s6gpuYLOnXqxKZNmzjttNM8vWd9EJsKzMGZ37mdgoL5FlSMMW1mfSgxUj+3ZD3wIHAKNTVOtv3D\nhw9z9913h+yAb4vdu3eTmvodnGCyFngPWMfkyf9tzV/GmJiwgOKBnJwcCgsXkJqaj5MV5tsGx1ev\nXk1JSYmn93SCWClOzaR+1n16eq7l9jLGxIQFFI9MmHA5GzeW0Llz5wb7Bw0axJo1axg2bJin98vJ\nyWH+/HnAdgKX862t/Zjc3FxP72WMMa1hAcVDp59+OjNmzACcvFsFBQVs3Lgx4k73ltx00w0UFMwn\nI2ME3brlWW4vY0xMWae8x7799lseffRRJk+eHPYQ4EjZ6ovGGK/ZKK8Q4nGmvDHGxDsb5WWMMSZm\nLKAYY4zxhAUUY4wxnrCAYowxxhMWUIwxxnjCAooxxhhPWEAxxhjjCQsoxhhjPGEBxRhjjCcSPqCI\nyEUisk1EtovI1FiXxxhjOqqETr0iIik46XZHAZ8AJcAVqrot4BxLvWKMMWHqiKlXhgE7VLVUVWuA\n5cDYGJfJNKGiooKSkhJbAMyYJJXoAeU7wJ6A92XuPhNnli1bQe/eAxgz5mZ69x7AsmUrYl0kY4zH\nUmNdgPYwc+bMutf5+fnk5+fHrCwdUUVFBRMn3kpV1Vqqqs4A3mfixJGMHn2Bpds3Jk4UFRVRVFTU\npmskeh/KcGCmql7kvp8GqKrOCTjH+lBirKSkhDFjbubAgQ11+7KyhrBmzUKGDh0aw5IZY5rSEftQ\nSoC+ItJbRNKBK4CVMS6TCZKbm0t19W4ClyquqSm1pYqNSTIJHVBU9TBwO7Aa2AwsV9WtsS2VCZaT\nk0Nh4QJ8vpFkZQ2xpYqNSVIJ3eTVGtbkFT9sqWJjEoctARyCBRRjjAlfR+xDMcYYEycsoBhjjPGE\nBRRjjDGesIBijDHGExZQjDHGeMICijHGGE9YQDHGGOMJCyjGGGM8YQHFGGOMJyygGGOM8YQFlBix\n1QuNMcnGAkoM2OqFxphkZMkh21lFRQW9ew+gqmot4Kxe6PONpLR0m2XgNcbEDUsOmQB2795Nenou\nTjABOIO0tN7s3r07doUyxhgPWEBpZ7Z6oTEmWVlAaWe2eqExJllZH0qM2OqFxph4Zis2hhCvAcUY\nY+KZdcobY4yJGQsoxhhjPGEBxRhjjCfiOqCIyI9F5J8iclhEhgQdmy4iO0Rkq4hcGKsyGmOMccR1\nQAE+AP4LeD1wp4gMBMYDA4EfAQtEJKzOo2RRVFQU6yJElT1fYkvm50vmZ4tUXAcUVf1QVXcAwcFi\nLLBcVWtVdTewAxjW3uWLB8n+j9qeL7El8/Ml87NFKq4DSjO+A+wJeL/X3WeMMSZGUmNdABF5FTg2\ncBegwM9V9aXYlMoYY0y4EmJio4isBe5R1Xfd99MAVdU57vtXgPtVdX2Iz8b/AxpjTBwKd2JjzGso\nYQh8sJXAcyLyGE5TV1+gONSHwv0LMcYYE5m47kMRkXEisgcYDvw/EXkZQFW3AM8DW4C/AbdafhVj\njImthGjyMsYYE//iuobSViJykYhsE5HtIjI11uVpKxEpFJH9IvJ+wL6jRGS1iHwoIqtEJDuWZYyU\niJwoIq+JyGYR+UBEJrn7k+X5MkRkvYhsdJ/xIXd/Ujyfn4ikiMi7IrLSfZ80zyciu0Vkk/szLHb3\nJdPzZYvIC+5k8c0icna4z5e0AUVEUoDfAT8ETgMmiMiA2JaqzRbhPE+gacAaVe0PvAZMb/dSeaMW\nuFtVTwPOAW5zf15J8XyqeggYqap5OMt1XiAi55IkzxdgMk5TtF8yPd8RIF9V81TVP+8tmZ5vPvA3\nVR0InAlsI9znU9Wk3HD6XV4OeD8NmBrrcnnwXL2B9wPebwOOdV8fB2yLdRk9es4XgdHJ+HxAZ5xB\nJKcm0/MBJwKvAvnASndfMj3fR8DRQfuS4vmALOBfIfaH9XxJW0Oh8eTHMpJz8uMxqrofQFX3AcfE\nuDxtJiK5wGBgHc4/5qR4Prc5aCOwDyhSZ3BJ0jwf8BhwL848Mr9kej4FXhWREhG53t2XLM/XB/hM\nRBa5TZZPiUhnwny+ZA4oHVVCj7IQka7An4DJqvoNjZ8nYZ9PVY+o0+R1InC+iOSTJM8nIv8J7FfV\n92icKilQQj6f61xVHQL8B06T7Pkkyc8PZwrJEOAJ9xkP4rTqhPV8yRxQ9gInBbw/0d2XbPaLyLEA\nInIcUB7j8kRMRFJxgskSVf2ruztpns9PVb/CGe7+PZLn+c4FLhGRXcAynD6iJcC+JHk+VPVT988K\nnCbZYSTPz68M2KOq77jv/4wTYMJ6vmQOKCVAXxHpLSLpwBU4EyITndB4kudP3dfXAn8N/kACeQbY\noqrzA/YlxfOJSE//CBkR8QFjgI0kyfOp6n2qepKqnozzf+01Vb0aeIkkeD4R6ezWnhGRLsCFONnQ\nk+Xntx/YIyL93F2jgM2E+XxJPQ9FRC7CGbmQAhSq6iMxLlKbiMhSnA7Po4H9wP04vym9APQCSoHx\nqvplrMoYKXfE0xs4/0nV3e7D6bx+nsR/vkHAYpxfBlJwamHzRKQHSfB8gURkBE6qpEuS5flEpA/w\nPzj/LlOB51T1kWR5PgARORP4PZAG7AJ+BnQijOdL6oBijDGm/SRzk5cxxph2ZAHFGGOMJyygGGOM\n8YQFFGOMMZ6wgGKMMcYTFlCMMcZ4wgKKMcYYT1hAMUlLRBa7a1ccEZFaEWk2OaiInCMi34hItYj8\nr4gUtldZ45WI/FpESt2/wxGxLo+JbxZQTNJS1WuBHwNv4cxQ79fUuW4esf8CMnAS5H1fVSe2S0Hj\nmKreAzwKHMLJ/mxMkyygmGR3Ps7CZAKc3Mx51+Os/dAJZyEhU+88oESdRcKMaZIFFJPszsXJYPwt\nTQQUd/2VSpwazBHgzXYqW6I4H3g91oUw8c8Cikl2WW66+N00XUO5WlX/CPwA2Jyoyf2iQUROBk7A\nAopphdRYF8CYaHE74T923+4iREARkXHASjel/FnAUyHOGQzcAnwOpONke75NVQ8GnHMBcCNO4MrB\naTa7U1WHhrheL+AXOIsYfetuj6pqlfsFfheQCzyrqisCPjcJ+JGq/ihg36XASJxssBOAy3CWFj4R\neDxgfYtwn+MjoDvwT6AGpx/KmObFei1j22yL1gb8BLjYff1boCLoeDfgbvf1KJzmrsuCzvkpzrol\nxwXsu9d/Xff9ROBT4Hj3fW+gCng5RJkygQ+BU9z3JwMVwCU4/TxP4vTjTALeC/rsO8DigPfpwCPu\n6+04a4+cC/TACWy/aeNz9MIJem/F+mdpW2Js1uRlktl51PeH7AJ6iEi3gOM3Awvd1z/AWeviDf9B\nETkHp8ZyqzrraSMiQ91z33bfn4kTBCZr/Yp+pcA3hO6LGYNTg9njvv8SZy32V4ELgNWqehi4CCdI\n+MvSFTiThk1P+cDbIpKG8+X/rqq+BXTGqYW80Mbn2IPTt2TNXaZVrMnLJLOjtL4/5CP3z5OBTSJy\nFrBN65t7zgd2qrNynd9MnC/mcW7TUgqwFZigznr3ALOBr3CWTAVARAbgNCeFCihf4jYlicjfgOdV\n9SH3c1uAChE5AWdFwEsDPneee//Xg671NnAOTm3lBQBVLcNZvrWtzzHQfQ4LKKZVLKCYpCQix+Cs\naum3C3fosIh8APynqj7onpsGnA0sDfh8Kk6N4WlVndrEPbKBHwIvuLUKv5E48zbWB39GVd8UkanA\nHcBtwO0icpuqPumvGYjI5cDXwMsBHz0f+FRV/xVwrWL3/JHAv1X1nyHK2NbnqAX+EepzxgSzJi+T\nrH5AQPMVDWsoP8OZm+I3FPDR8Dfxo3H6MnY1c4++7jnBE/5G4MzbqHaXjm1AVeeq6knA6ThB57ag\nUy4E1qpqTajnCXHNkTRdi2jLc+TjNKMdDPUcxgSzgGKS1fkEBBRV/Rr4t7s/1e0f8PP3nwR+KX+G\n0wTUqBYvIieLyE3ucagfSYY7Wiyf+lFRkwOO/UVE3gso01bgWRp/2Z+E03EfeM3vBTzPPQHHMoHh\nQFFwOT14jhEB152MMS2wgGKSjtvcNQongAT6COgDPB20/wKgLDDIuE0/C3E6xwOvfT4wA/ijqu4A\nNuEM8fU3L/0WJ31LqYgcjTOCy+8sYE1QOX8CPBhUnh04I7X8fo5Tg9gtIqfh9H/4nQuk0URAacNz\nPIHTL/MvEckB9oW6vjGBRFVjXQZjPOF+gf8Zp0O6C87Q2Xmq+qR7fAHOF+g690tzJXAsMBhnLkgx\n8BdV/a17fifgEZw5HR/jfMG+r6qLAu7ZF2eU1nacL/3f4nzJ/xQnMExX1c/dc8/H+a3fB2ThjMZ6\nUgPmirjn9cEZcbUNp9/naeBHODWfXcA9qlrtnnsVcLuqDm/m76Utz3EtsBOYqjbh07TAAooxxhhP\nWJOXMcYYT1hAMcYY4wkLKMYYYzxhAcUYY4wnLKAYY4zxhAUUY4wxnrCAYowxxhMWUIwxxnjCAoox\nxhhPWEAxxhjjif8PjAUCIYIdJmsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11c3e40d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.scatter(y_test, predicted)\n",
    "ax.plot([y.min(), y.max()], [y.min(), y.max()], 'k--', lw=4)\n",
    "ax.set_xlabel('$Measured$', fontsize = 20)\n",
    "ax.set_ylabel('$Predicted$', fontsize = 20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 交叉验证"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-1.5787701857179994"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "\n",
    "regr = linear_model.LinearRegression()\n",
    "scores = cross_val_score(regr, boston.data , boston.target, cv = 3)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEACAYAAABbMHZzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNXZB/DfM7LIoiyyg0SMVRAXFEGBKkktiFBxKW4E\nbKz2tVpT37eh6isNSYz7y9tqsbb6Vo3VuFVEAUWBYkSkCCqICwgmmWEVUBQMAkLmef+4k0kms965\nd9b7+34++ZC5c+eeM9f4zJlznnOOqCqIiMhZXKmuABERJR+DPxGRAzH4ExE5EIM/EZEDMfgTETkQ\ngz8RkQPZEvxFZJyIrBeRDSJyW4jnR4vItyLyoe/nD3aUS0RE8Wll9QIi4gLwMIDzAWwDsEpEXlXV\n9S1OXaqqE62WR0RE1tnR8h8OYKOqelT1EIDnAVwc4jyxoSwiIrKBHcG/L4DNzR5v8R1raYSIrBGR\n10TkZBvKJSKiOFnu9onRBwD6q+r3InIhgFcAnJiksomIqAU7gv9WAP2bPe7nO+anqvXNfl8gIo+I\nSFdV3d3yYiLCxYaIiExSVVNd63Z0+6wCcIKI5IhIGwBXAZjb/AQR6dns9+EAJFTgb6SqGfPjrq1F\nWUEBZuTloaygAO7aWtuuXVpamvL3lw4/vA+8F7wXkX/iYbnlr6oNInIzgIUwPkweV9V1InKD8bQ+\nBmCSiNwI4BCA/QCutFpuOvDU1WHWmDEor6lBBwD7AJSuWIGiRYuQM2BAqqtHRBSWLX3+qvoGgJNa\nHHu02e9/AfAXO8pKJ5UlJf7ADwAdAJTX1GBmSQlKn3kmlVUjIoqIM3wt8G7Z4g/8jToA8G7bZsv1\n8/LybLlOpuN9aMJ70YT3whoG/3ht2wbX+vXY1+LwPgCuPn1sKYJ/3Abehya8F014L6xJVqpnxvPU\n1aGypATerVvhUkXhJ5+gcMoUlM6bh/La2qY+f5cLRVdfnerqxiTgPfXti8KKCo5VEDmExDtSnCgi\noulWp5ADu336oGjZMgBG37932za4+vRB4dChyPnrX4EPPgCOOiriNVMZeEO+p9xcDlYTZSARgZpM\n9Ux5ilKIlCVNN2UFBVoPqDb7qQe0rKAg9Auuv1518mRVrzfk0+7aWi3OzfVfsx7Q4txcddfWJvBd\nBDL9nogobfnipqlYyz7/aPbvh3fpUnMDuw89BKxdCzz+eMinw2UJVZaU2FTp6LxbtyZ0sJqI0hv7\n/JsJ6oqZMAE5M2bAJYJ9QECwjDiw27498OKL8Iwcico5c+D9/vumrp1OneBdtSrlgdfVt6+590RE\n2cXsV4VE/yBF3T4hu2KOOELdjz0WVzeNu7ZWi3v0CHxN+/bq7tBBy/r3T3mXi7u2Vovbt09p1xMR\n2QNxdPukPNgHVShFwT9aH7i7tlbLCgp0Rn6+lhUURA2SYa93xRVp0eevy5eru2dPLbvySp0xaJCW\n9e/PwE+UoeIJ/uz28YnWB54zYICpWbthr7drF3IGDEDRokWYWVIC7wcfwLVnT3KzbFSB225Dzr33\novTaa4HaWuDcc4HjjktO+USUcgz+Pnb3gUe7nv/DZP9+oH9/wOuNr+LxeO01YPdu4JprjMcDBhjl\nezz8ACByCGb7+BRWVKC0d2//jN3GvPfCior4r5ebG/167doBhYXAX/8aX8XNamgAbr8duPde4Igj\njGMiwKhRwPLlyakDEaUcJ3k145k6FZWffAJvly7GhC2LE6/82UONE8DCXa+2Fhg+HNi0ycgUSqTK\nSiMFdelSI+g3+uMfgZoa4C+Zsf5eqifJEaWTeCZ5Mfg38nqN7pd//Qs46aTo59ttwgRg0iTg2msT\nV8b+/cZ7e+EFYMSIwOfeew+44QZgzZqol0l14OXsZKJAnOFrxcqVqgMHpqZsVdX581WHDg07K9gK\nf6ZSbq6W9esXOqvn4EHVDh1U9+yJeq1UZypxdjJRIHCGrwWvvAJccknqyh83zhiEXbnS1ss2tpKn\nVVWhvKYG07ZswawxY+Cpqws8sU0b4MwzjW8AEXB2MlF2YPBv9OqrqQ3+RxwB3Hij7X3upoL1qFHA\nu+9GvF46BN7GTKrmODuZyBwGfwDYuNFodQ8bltp6XHstMHcu8NVXtl3Su25d7MF65MioGT9hA2+P\nHhFf56mrQ/mUKSjNz0f5lCnB3zxMKKyoQGnXroGZVAMGxJ2ZReREzPMHjFb/xImAK8Wfhd26Gd8+\nHn8cuO02Uy8NGoS97DLkPPYYXJ9/Hvv8hZEjgSlTjHTQxjTQFgorKlA6ezbKDxxoGmzt2BFFH3wA\nz4IFqKyqChoItnuv45z+/VHUti1mjhsH78GDcG3YgKLrr+dgL5EZZgcJEv2DVAz4jhqlumBB8ssN\nZeVKdffpo2VXX60z8vJiWkoi7LpEFRXqXr/e3ADtwIGqq1eHL6yuTt2dOmnZVVc1LXVRU6PuBx7Q\nYpcrZDllkyfbO0C7YIExON5o0SLVU05JyGA5USYA1/aJw44dqp07qx44kNxyw3DX1mpx27amsmls\nXZfol79Uffjh8M/feafqTTfFXod27XSGSMDxxp8Z+fkx35cAl16q+re/NT32elUHDVJdsiS+6xFl\nuHiCP7t95s0Dxo4F2rZNdU0A+AZoDx4MGqCdWVISdm0hW9clGjXKmOvwm98EP6cK/OMfQFVV7HUY\nMgSu/v2x74UXgrueNm0CNm6Ep1Wr2OcNbN8OvPWWMVmtkQjw298Cf/4zkJ8f2/skcjgO+KY6xbOF\neLJpbM1+iZTxs3w50KpVyIHxsHU4/ngU3ntv8FIXxx2HwgkT4Bk+HLNOPdVIRa2uxrSqqtCpqI0q\nK4Gf/xw4+ujA41OnAu+8A7jdMb9VIkcz+1Uh0T9IZrfPd9+pHnWU6rffJq/MKOKZwOT+4gstbtXK\nnolXXq/qMceobtkS/NyvfqV6zz2h6xBl8le4rqeyK66I/f02NKgef7zqe++Frntxservf2/+PRNl\nOLDP36TZs1XHjEleeTGIawbt/PnqHjxYyyZPjnm/gYguukj1xRcDj33/vWqXLqqbN0esu5k9D1RV\nZ+TlxT4esGiR6mmnhR/YrakxPrjq66OWaxf/e45xcJ4oEeIJ/s7u80+zLh8AgWv9v/suXC5X9JTI\nhx5Czq23orRxiWarGrt+Lr+86dirrwJDhwL9+kWsu5k9DwCTS2n/3/8B//EfgQvSNXf88Ubdq6qM\n8xLM7hRWoqQy+2mR6B8kq+X/ww+qXbtGbMmm3O7dRh0jtSY//VS1Vy97s5WWLlU966zAY+PGqT79\ntH1l+IT8ptOqlbpXrgw8cedO1U6dVL/5JvIFFy9WHTw4KWmfXGOI0gXY8o/OPxnq00/hAlB46BBy\nUl2pcLp0MVbavO8+4NFHQ5/z5z8b59iZrXTWWcBnnwH79gEdOhgZNitWAC+9ZF8ZPgHfdHxLXxd1\n7oyc//ovI+uo8X099ZTxLa1z58gX/MlP4Dl4EJVjxsDb0JDQVUfTYakLoriZ/bRI9A8S2PJPhxUp\nTdu1y+hr37Qp+LmvvzbmKGzfbn+5Z5+t+tZbxu//8z+q115rfxnhNDSoXnaZ6jXXGC14r1f1xBNV\nly2L+lJ3ba0Wd+tm+r9xPH33ZZdfHlfLn+MEZDdwwDeyjP2aPm2a6s03Bx+//37VqVMTU+bvfqd6\n111G4D3lFNXq6sSUE059vbpPPlnLhgzRGUOGaNnRR6u7pibqy+LKloqzUeC+8UYtPuqowNf17x/x\ndRnZAKG0x+AfhanMknSyfbvR+t+2renYoUOqxx6r+v77iSlz9mzV8eNVP/hA9bjjjNZ4Erlra7U4\nJ8d0kIznv3FcjYJNm1S7dlX3u+82ZTgNGaLuH/844nhDxjZAKK3FE/wdNckrY5cC7tXLmMQ0c2bT\nsTlzjJ3Hhg5NSJGefv1QvngxSidORHnHjvB4PAkpJ5zKkhKUezym9w2I579xXH33JSXAr3+NnJEj\nUfrMMyhfsgSlK1ciZ+9e4Pnn7S2LKBHMflok+gfs8w9tyxaj9b9zp/F41KjgXHybpMN9ivdbWjx1\nN90aX7NGtWfP0JMD//1v1d69jUytUGUNHx66rKuuivi+iCIBu32ic9fWallens7o0iXjBtvckydr\n2ckn64yhQ7WsfXt1b9iQkHLSoWvCSh38A6ojRmhZmzbq/vDDqOcXd+8e+IHRpo26164N/YKxY1Vn\nzQp/wZtuUr3hhuDjDz6o7j59tLh//8CyOnZU95lnqn71VdT3RhRKPMHfmRu4z54NPPus8W+G8NTV\nYVZeHso3bUr4puWl+fkor64OfXzJElvLCse2Tdqvu87oHistDX/OwYPwHH88KgcPhvfwYbh690ah\n14ucnTuB114Djjyy6dyFC4GbbwY+/RRo3Tr09fbsgedHP0LlkCHwHjpkpJv27o2cOXOAf/0LHq/X\nSDf2pbYWlpcj59FHgblz4XnsMVQ+9lhsi9wR+XAD91hVViYuSyZBktkaT4eWv2p8y0UE2bBBtVs3\n1b17w5/z4IOqEyYEHjt8WHXSJCPl9PDhpmOnnab60ktR613cs2dg6751a3UvXx75dWVlWnzEEZnZ\nLalMYU0lpKrbB8A4AOsBbABwW5hz/gxgI4A1AIZEuFbCbpDfX/6i+utfJ74cGyUzUykd+vxtdfXV\nRlpsKHv3Gv33H30U/NyBA6rnn6/uK6801k0aOFDLunWLmnIa74dnunzoxiPr/mYyTDzB3/IMXxFx\nAXgYwPkAtgFYJSKvqur6ZudcCCBXVX8kImcD+BuAc6yWHbf6eqBjx5QVHw9Ta+BYFHLWbSZ3P9xx\nB/DTnwJFRUC7doHPPfggcP75wGmnBb+ubVt4/vQnzBo2zL/Hwj4ApWPHRux+ijejJ5MzgSpLSvxd\ndEBs+1BQatmR6jkcwEZV9ajqIQDPA7i4xTkXA/gHAKjqewA6iUhPG8qOT+OyBRmksKIieE383NyE\nbVreuEhb+ZIlKH3mmcwN/ABwyinAiBHA3/8eePyrr4CHHgLuvDPsSyvvvz/k5jqRUk7jTSnO2FRk\nZPYHl1PZEfz7Atjc7PEW37FI52wNcU7yZGDw97fGCwpQmp+PmQUFXD3SjOnTgQceAH74oenYffcB\nV1wB5OaGfVk8QS3eD+pkf8DbKZM/uJwqLRd2Kysr8/+el5eHvLw8ewuor4/4P3y6imfJZPI56yxg\n8GBjG8rrrwe2bAGefBL45JOIL4unuy3ebjP/66ZPh/eFF+CaNAlF992XER/whRUVKJ03D+V79wZm\nZ2XAB1cmqq6uRnWIjDwzLKd6isg5AMpUdZzv8e0wBh/ub3bO3wC8paov+B6vBzBaVXeEuJ5arVNU\nU6cCY8YAdq1/T5nhnXfgmTwZleeeC291NVxduqBw/vyIwdW2lFOzRo0C7rorc/Yk9nrh6d0blSef\nbOxDcfnlKLzrroz44MoG8aR62tHyXwXgBBHJAbAdwFUArm5xzlwAvwHwgu/D4ttQgT9p9u3LuAFf\nss7Trx9mffUVyp97zgjk27ejdMyYiIE8ZYPfw4YBq1ZlTvB//33kdO2K0iVLgJwcY14FA39asxz8\nVbVBRG4GsBDGGMLjqrpORG4wntbHVPV1ERkvIl/AaDxda7VcS+rrM67Pn6yrLClB+YEDpjNSUtLd\nNmyYsdNcppgzB7j0UmOXtfPOA5YuBU48MdW1oghs6fNX1TcAnNTi2KMtHt9sR1m2yMABX7IuozJS\nhg0zBqkTyL+xkdXZxKrAyy8b22cCTcH/+uvtrTDZKi0HfBMuA/P8ybpkzpWw7IQTgG+/BXbtArp3\nt/3ytu4/vG4dsH9/0wqzo0cD99xjd5XJZo5a0tmPLX9HyqhUSpfLCKarViXk8uEmZUVbMjukl19u\n6vIBjO6e/fuBJC8DTuYw+JNjZNxcicZB3wTwfvSRfV1gjf39jZr3+1PaYrcPOUpGzZUYNsyYi2C3\nRx6B64sv7OkC83iAzZuBH/848Hhj8J861WJlKVGc1/JXBb7/ni1/Sn/DhgHvv2/8zUbgqatD+ZQp\nxpLbU6bAU1cX+kRVY52jBx9E4Rtv2NMFNmcOcNFFQKsW7Ui2/NOe89bz378f6NIFOHAgcWUQ2UHV\n2MLz/feBY48NeUq0SWj+jJ4tW+DatAmFRx+NnMWLgW7dmp57+WW4Ro9G4SOPmO8CGz0auPVWYMKE\nwOMNDUC3bsZgcK9ecb19ih3X84/Fzp2qxxyT2DKI7DJhgurs2WGfDrsM9Omnq3vWLC3u1y9wmeUB\nA4KXWf7d71TLy83XbccO1U6dVPfvD/38z36WsK1GKRC4gXsMONhLmSTKoG/YuQu7dxsrkm7ZEpjR\n42vtBxg/Hnj9dfN1mzsXuOCCwJ3Omhs9Gnj7bfPXpaRwZvDnYC9liijBP+xqmuedB+8JJ8SW0XPu\nuUb3zK5d5ur28svAZZeFf579/mnNecGfSztQJmkc9PV6Qz5dWFGB0i5dQg7cxrzMcps2xoY2b74Z\ne7327AGWLQMuvDD8OWecAbjdwO7dsV+XksZ5wZ/dPpRJuncHOncGvvgi5NM5/fujqG1bzLzwwqC5\nC6YmtY0fb2xWH6vXXzda9kcfHf6c1q2Bc84xPiQo7Tgvz585/pRpGrt+Qi2UtmQJcnr3RmmIPntT\nK5JeeCFw223A4cPBaZvN+DOEFi6E6/jjUVhXFzlDqLHrZ+LEWN4pJZHzgj9b/pRpzjrLCP4FBcHP\nPfEE8Mtfhn1pzJPa+vY10knfe8/YSyCEoLTSXbuiLomN884Dpk2LXj4lHbt9iNJduEHfb74BFiwA\nJk+2p5woWT9xrQc0fDjw2WfAd9+FfDrmCWpkO+cFf3b7UKYZOhT46COjS6a5554zUi27drWnnCjB\nP64lsY880qj/8uVBTzV+k5hWVYXy6mpMq6rCrDFj+AGQJM4L/mz5U6bp1Ano1w/49NPA408+GbHL\nx7RzzgE2bQK2bg35dNybtIdJ+bR1ZVEyzZnBny1/yjQtu37WrgW+/BL46U/tK6NVK2DsWOCNN0I+\nXfjb36LU5TK/HlCY4J9Rm+tkIecN+NbXAz17proWROY05vs37o715JPAL34BHHGEveWMH29sH3nd\ndUFP5dx/P4puuAEz9+41t5/xiBHAhx8a62q1a2ccO3QIrq+/zpzNdbKQ84I/u30oEw0bBjz1lPH7\nDz8YWyaG6Ee37IILgKIio4w2bZqOv/QS8NlnyKmqQmm45RzC8Ozahcp27eAdORKuwYNROHUqcqZP\nR2GXLijNyUG5x9O0KF2bNii65RZb31JMdbRrS8tMYnYxoET/INELu11xhepzzyW2DCK7ff+9art2\nxiJqs2ernnde4soaPlx1yZKmx199pdq7t+ry5aYv5a6t1eLc3MDF5Vwudd99t6rXq+7aWi0rKNAZ\n+flaNnmyuouKVAcNUt22zcY3FEcdc3ODF8BLY4hjYbeUB/ugCiU6+E+YoDp3bmLLIEqEIUNUV6ww\nVsusrExcOWVlqsXFTY+nTlW95Zb4LhVu1dGCgvAvuvtu1R/9SN3vvmt8MOTlaVlBQcKCcVx1TDPx\nBH92+xBlAE9dHSq/+w7eK66Aa/t2FN53H3ISVdj48cZ4wsyZRurnsmXAxx/Hdam4BnXvuAOe+nrM\nGj0a5YcPW99gPlodm618GnMds4Dzsn2Y508Zxp8PX1OD8k2bMO3QIcy6+OKE5cN7unZFeW0tSocN\nQ/nll8Nz551xN5jiTQ+t3LTJH/iBxKaBNg48m61jxjP7VSHRP0h0t8+gQaqffJLYMohslMxuCbv7\nv+O93oy8vID32/gzIz8/rnqE9c476u7WTYuPO85xff7Oa/mz24cyTDLz4e2eeOVfXK6gIGjV0Uji\nnlBmxp49wNSpyHniCRQtWYKZ+fko7dw55jpmOuf1+bPbhzJMYyBMRj58Ij5oYl5crpnCigqUrlgR\nvDex2Q3mI7n5ZmDcOOCii5ADoPTZZ4GTTwaefhoQc9vhZiK2/InSnKl1+S1KSos7BgHfGHr0wMzh\nw+1tjT/7rDFj+n//t+lYr17G3IYtW+wpI82J0V2UPkREE1anw4eBtm2Nfx3wyU7Zwz8JyTezNlGT\nkIKWbYavxZ3KbpAXXzRmNC9YYOky/ntYUwPX6tUo/Oc/kXPRRYEnNU5y+9nPLJWVbCICVTUV1JwV\n/PfsMdYs37s3MdcnygLJ+qCJ2XffGfsNbNpk7GoWh5g/1G691VhIb/p0W6qeLAz+0WzbZiwvu317\nYq5PRIkxcSJw5ZWhN7SJQfmUKZhWVRU0bjKzoCBwPKKqyljb6J//tFTdZIsn+Durz5+DvUSZ6bLL\ngJdfjvvlMQ9kn366sXeCAzgr+HOwlygzXXQRsHgx8P33cb3cdcwxsQ1kn3SSMeC7r+XZ2YfBn4jS\n3zHHGCubvvmm+dd6vSjctQulnTtHz5hq3RoYNCju5SwyibOCP7t9iDJXvF0/DzyAnIYGFL33XmyT\nzRzS9eOsSV5s+RNlrksuAf7wh+C9BiJ55x3gwQeBVauQc+yxsU02O/10YM0aa3XNAJZa/iLSRUQW\nisjnIvKmiHQKc55bRD4SkdUistJKmZZwC0eizNWnDzBwIPDWW7Gdv2sXMHky8MQTRop3rIYMcUTL\n32q3z+0AFqvqSQCWAPjvMOd5AeSp6hmqOtximfGrr2fLnyiTRen68dTVoXzKFJTm56P89NPhmTDB\nWKLajNNOM/r8vV6LlU1vVoP/xQB8e8vhKQCXhDlPbCjLOnb7EGW2Sy818vAbGoKe8i99XVWF8upq\nTNu+HbMWLTK/9HWXLkDXrkBtrU2VTk9WA3IPVd0BAKr6JYAeYc5TAItEZJWI/MpimfHjgC9RZsvN\nBXr3Drl/ccgVSWtr41uR1AGDvlGDv4gsEpG1zX4+9v07McTp4abmjlLVMwGMB/AbEfmxlUrHjS1/\noswXpuvH1h25HDDoGzXbR1XHhHtORHaISE9V3SEivQDsDHON7b5/d4nIHADDASwLd92ysjL/73l5\necjLy4tWzdgw+BNlvssuAyZMAP74x6YFGmtq4Pr4Y/uWvh4yBHjqqejnpUh1dTWqq6stXcPS2j4i\ncj+A3ap6v4jcBqCLqt7e4pz2AFyqWi8iHQAsBFCuqgvDXDNxa/tMnQqMGQNcc01irk9ECeeprUXl\nkCHwnngiXAMHonDYMOTcfTc8N96IWVVV9qxI+sUXwPnnAx5PIt5CSP4F9bZuhatvX1ML6sWzto/V\nLRe7AlgM4HMYQb2z73hvAPN9vw8AsAbAagAfA7g9yjUtbmgWwaWXqr70UuKuT0QJFXJbyFat1D1v\nnv/5soICnZGfr2UFBfFvxdjQoNqxo+ru3TbWPjyr22cijm0cU75nb1CFEhn8x45VXbAgcdcnooRK\n5n7GOmKE6ltv2X/dEKK9L/+HWl5eyA+1eIK/s2b4MtuHKKMlcz9jf8aPXWOOEYR9X6tXw1NdjVnX\nXx/YnbViheUNdlKfe59MHPAlymhJ3WYyiTN9w76v775D5fjxwSmsNTXxpbA2L9PSqzMNW/5EGS2Z\n+xknM9e/sKICpUcdFfy+3n4b3rPPTsi3HWd1+7DlT5TR/Bu7N9tmsihR20yeeiqwbh1w6JCx1LMJ\nZjN3clRRJIKZkybB+/XXAe+r8VuBLSmszThrG8ejjwY2bzb26CQiiubEE40JZaecEvNLYt4vuLnr\nrjP2Kb7zzriuxz18I1EFWrUCDh40/iUiiubyy42lpE3sHRzzfsGNamuB4cOBDRuMNYVC8H+T8H3b\naflNIp7g75woeOCA8dWNgZ+IYtU46Gsi+JvOSLrnHuDGG8MGfsDo7oppLwITnBMJ2d9PRGadfjow\na5apl7jq62Pvo6+tBebMATZutFLLuDgn24eZPkRkltmMn7lzUejxoLR//8DMne7dQ2ck3XMPcNNN\nEVv9icKWPxFROP36GdtGfvkl0KtX5HNXrQKuuw45r72Gou7dmzKSOnZE0fLlyKmpAZoP+NbVpazV\nDzgp+LPlT0QmedxuVLZuDe8FF8B16qnhUzbr6oCLLwYefxwYPhw5QGAf/dKlwKRJwNtvA4MGGcdi\n6OtPJOcEf7b8icgEf4rlzp3osHMn9q1dG7Csgj8Dx+2Ga+1aFBYXI2diqG1OAJx3HjBzJjxjx6Ly\n7LON/P8PP0Thv/+NnOS+rSZmFwNK9A8StbDb3LmqEyYk5tpElHXCLraWn6/uhQu1eMAAU6twumtr\ntbhz57hX7owEcSzsxgFfIqIQwqZsrlmDyp//HOV1dabW26ksKUH5t9/avkZPvJwT/NntQ0QmhF1s\nbfx4eIcONb3eTlJXJI0Bgz8RUQiRFpGLZ3XRpK5IGgPnLO9w993GB8A999h/bSLKSuGWVYhn/Z64\n1vyJEdf2ieSOO4yW//Tp9l+biBwn2no7dr0mFgz+kdxyizHB4j//0/5rExGlUDzB3zl9/sz2ISLy\nc07w54AvEZGfc4I/W/5ERH7OCf5s+RMR+TH4ExE5kHOCP7t9iIj8nBP82fInIvJj8CciciDnBH92\n+xAR+Tkj+B8+DBw6BBx5ZKprQkSUFpwR/Bu7fMTU7GcioqzlrOBPREQAGPyJiBzJGcGfg71ERAGc\nEfzZ8iciCsDgT0TkQM4I/uz2ISIKYCn4i8gkEflERBpE5MwI540TkfUiskFEbrNSZlzY8iciCmC1\n5f8xgEsBvB3uBBFxAXgYwAUABgO4WkQGWizXnPp6Bn8iomZaWXmxqn4OACIRZ08NB7BRVT2+c58H\ncDGA9VbKNmXfPnb7EBE1k4w+/74ANjd7vMV3LHnY7UNEFCBqy19EFgHo2fwQAAUwXVXnJapitqqv\nB3r2jH4eEZFDRA3+qjrGYhlbAfRv9rif71hYZWVl/t/z8vKQl5dnrQZs+RNRFqmurkZ1dbWla4iq\nWq6IiLwFYJqqfhDiuSMAfA7gfADbAawEcLWqrgtzLbWjTgGuuQY4/3zgF7+w97pERGlARKCqplau\ntJrqeYmIbAZwDoD5IrLAd7y3iMwHAFVtAHAzgIUAPgXwfLjAnzDM8yciCmA12+cVAK+EOL4dwM+a\nPX4DwEloF0iPAAAH/UlEQVRWyrKE3T5ERAGcMcOXwZ+IKIAzgj+7fYiIAjgj+LPlT0QUwBnBny1/\nIqIAzgj+bPkTEQXI/uCvyuBPRNRC9gf/AweA1q2BVpayWomIskr2B3+2+omIgjD4ExE5UPYHf2b6\nEBEFyf7gz5Y/EVGQ7A/+3MKRiChI9gd/buFIRBTEGcGfLX8iogDZH/w54EtEFCT7gz9b/kREQRj8\niYgcKPuDP7t9iIiCZH/wZ8ufiCgIgz8RkQNlf/Bntw8RUZDsD/5s+RMRBcn+4M+WPxFRkOwP/mz5\nExEFYfAnInKg7A/+7PYhIgqS/cGfLX8ioiAM/kREDpTdwb+hAfjhB6Bdu1TXhIgorWR38N+3D2jf\nHhBJdU2IiNJKdgd/buFIRBRSdgd/buFIRBRS9gd/tvyJiIJkd/Bnjj8RUUjZHfzZ8iciConBn4jI\ngSwFfxGZJCKfiEiDiJwZ4Ty3iHwkIqtFZKWVMk1htw8RUUitLL7+YwCXAng0ynleAHmq+o3F8sxh\ny5+IKCRLwV9VPwcAkaizqASp6GJinj8RUUjJCsgKYJGIrBKRXyWpTOb5ExGFEbXlLyKLAPRsfghG\nMJ+uqvNiLGeUqm4Xke4wPgTWqeqycCeXlZX5f8/Ly0NeXl6MxbSwbx/QvXt8ryUiSlPV1dWorq62\ndA1RVcsVEZG3ABSr6ocxnFsK4DtV/WOY59WOOgEAbrwROPVU4Kab7LkeEVEaEhGoqqlFzOzs9glZ\nsIi0F5GOvt87ABgL4BMbyw2PA75ERCFZTfW8REQ2AzgHwHwRWeA73ltE5vtO6wlgmYisBrACwDxV\nXWil3Jgx+BMRhWRLt4+d7Or28dTVoXLkSHh79IDr1FNRWFGBnAEDbKghEVF6iafbJyuDv6euDrPG\njEF5TQ06ANgHoDQ3F0WLFvEDgIiyTqr7/NNGZUmJP/ADQAcA5TU1qCwpSWW1iIjSRlYGf+/WrWjZ\n098BgHfbtlRUh4go7WRl8Hf17Yt9LY7tA+Dq0ycV1SEiSjtZGfwLKypQmpvr/wBo7PMvrKhIZbWI\niNJGVg74Ar5sn5ISeLdtg6tPH2b7EFHWYrYPEZEDMduHiIhiwuBPRORADP5ERA7E4E9E5EAM/kRE\nDsTgT0TkQAz+REQOxOBPRORADP5ERA7E4E9E5EAM/kREDsTgT0TkQAz+REQOxOBPRORADP5ERA7E\n4E9E5EAM/kREDsTgT0TkQAz+REQOxOBPRORADP5ERA7E4E9E5EAM/kREDsTgT0TkQAz+REQOxOBP\nRORADP5ERA7E4E9E5ECWgr+IPCAi60RkjYjMFpGjw5w3TkTWi8gGEbnNSplERGSd1Zb/QgCDVXUI\ngI0A/rvlCSLiAvAwgAsADAZwtYgMtFiuI1RXV6e6CmmB96EJ70UT3gtrLAV/VV2sql7fwxUA+oU4\nbTiAjarqUdVDAJ4HcLGVcp2Cf9wG3ocmvBdNeC+ssbPP/5cAFoQ43hfA5maPt/iOERFRirSKdoKI\nLALQs/khAApguqrO850zHcAhVX02IbUkIiJbiapau4BIIYBfAfiJqh4M8fw5AMpUdZzv8e0AVFXv\nD3M9axUiInIgVRUz50dt+UciIuMA/B7AeaECv88qACeISA6A7QCuAnB1uGuafQNERGSe1T7/WQA6\nAlgkIh+KyCMAICK9RWQ+AKhqA4CbYWQGfQrgeVVdZ7FcIiKywHK3DxERZZ60meHr5IlgIvK4iOwQ\nkbXNjnURkYUi8rmIvCkinVJZx2QRkX4iskREPhWRj0Xkt77jjrsfItJWRN4TkdW++3GP77jj7gVg\nzBny9TDM9T125H0AABFxi8hHvr+Nlb5jpu5HWgR/TgTDkzDee3O3A1isqicBWIIQE+iy1GEAv1PV\nwQBGAPiN72/BcffDN46Wr6pnADgNwE9EZBQceC98bgHwWbPHTr0PAOAFkKeqZ6jqcN8xU/cjLYI/\nHD4RTFWXAfimxeGLATzl+/0pAJcktVIpoqpfquoa3+/1ANbBmDzo1Pvxve/XtjD+f/0GDrwXItIP\nwHgAf2922HH3oRlBcPw2dT/SJfhzIliwHqq6AzACIoAeKa5P0onIcQCGwJg93tOJ98PX1bEawJcA\nqlX1MzjzXvwJRmZh80FKJ96HRgoj0WaViFzvO2bqflhK9aSkctTIvIh0BPASgFtUtT7E/A9H3A/f\n8iln+BZNfFNE8hD83rP6XojIBAA7VHWN7/2Hk9X3oYVRqrpdRLoDWCgin8Pk30W6tPy3Aujf7HE/\n3zEn2yEiPQFARHoB2Jni+iSNiLSCEfifVtVXfYcdez8AQFX3AngdwFlw3r0YBWCiiNQCeA7G2MfT\nAL502H3wU9Xtvn93AXgFRte5qb+LdAn+/olgItIGxkSwuSmuU7KJ76fRXACFvt9/AeDVli/IYk8A\n+ExVH2p2zHH3Q0S6NWZsiEg7AGMArIbD7oWq3qGq/VX1eBixYYmqTgUwDw66D41EpL3vmzFEpAOA\nsQA+hsm/i7TJ8/fNFn4IxgfS46p6X4qrlDQi8iyAPADHANgBoBTGp/k/ARwLwAPgClX9NlV1TBZf\nNstSGH/M6vu5A8BKAC/CQfdDRE6FMXDXOLj3tKrOFJGucNi9aCQiowEUq+pEp94HERkAYA6M/zda\nAahS1fvM3o+0Cf5ERJQ86dLtQ0REScTgT0TkQAz+REQOxOBPRORADP5ERA7E4E9E5EAM/kREDsTg\nT0TkQP8PPqvEe2XQT0IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11c33d650>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "scores = [cross_val_score(regr, data_X_scale, boston.target, cv = int(i)).mean() for i in range(3, 50)]\n",
    "plt.plot(range(3, 50), scores,'r-o')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.453848713596956"
      ]
     },
     "execution_count": 176,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X_scale = scale(boston.data)\n",
    "scores = cross_val_score(regr, boston.data, boston.target, cv = 7)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 使用天涯bbs数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>link</th>\n",
       "      <th>author</th>\n",
       "      <th>author_page</th>\n",
       "      <th>click</th>\n",
       "      <th>reply</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>【民间语文第161期】宁波px启示:船进港湾人应上岸</td>\n",
       "      <td>/post-free-2849477-1.shtml</td>\n",
       "      <td>贾也</td>\n",
       "      <td>http://www.tianya.cn/50499450</td>\n",
       "      <td>194675</td>\n",
       "      <td>2703</td>\n",
       "      <td>2012-10-29 07:59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>宁波镇海PX项目引发群体上访 当地政府发布说明(转载)</td>\n",
       "      <td>/post-free-2839539-1.shtml</td>\n",
       "      <td>无上卫士ABC</td>\n",
       "      <td>http://www.tianya.cn/74341835</td>\n",
       "      <td>88244</td>\n",
       "      <td>1041</td>\n",
       "      <td>2012-10-24 12:41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         title                        link   author  \\\n",
       "0   【民间语文第161期】宁波px启示:船进港湾人应上岸  /post-free-2849477-1.shtml       贾也   \n",
       "1  宁波镇海PX项目引发群体上访 当地政府发布说明(转载)  /post-free-2839539-1.shtml  无上卫士ABC   \n",
       "\n",
       "                     author_page   click  reply              time  \n",
       "0  http://www.tianya.cn/50499450  194675   2703  2012-10-29 07:59  \n",
       "1  http://www.tianya.cn/74341835   88244   1041  2012-10-24 12:41  "
      ]
     },
     "execution_count": 207,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('/Users/chengjun/github/cjc2016/data/tianya_bbs_threads_list.txt', sep = \"\\t\", header=None)\n",
    "df=df.rename(columns = {0:'title', 1:'link', 2:'author',3:'author_page', 4:'click', 5:'reply', 6:'time'})\n",
    "df[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# 定义这个函数的目的是让读者感受到：\n",
    "# 抽取不同的样本，得到的结果完全不同。\n",
    "def randomSplit(dataX, dataY, num):\n",
    "    dataX_train = []\n",
    "    dataX_test = []\n",
    "    dataY_train = []\n",
    "    dataY_test = []\n",
    "    import random\n",
    "    test_index = random.sample(range(len(df)), num)\n",
    "    for k in range(len(dataX)):\n",
    "        if k in test_index:\n",
    "            dataX_test.append([dataX[k]])\n",
    "            dataY_test.append(dataY[k])\n",
    "        else:\n",
    "            dataX_train.append([dataX[k]])\n",
    "            dataY_train.append(dataY[k])\n",
    "    return dataX_train, dataX_test, dataY_train, dataY_test, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variance score: 0.78\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# Use only one feature\n",
    "data_X = df.reply\n",
    "# Split the data into training/testing sets\n",
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplit(np.log(df.click+1), \n",
    "                                                                   np.log(df.reply+1), 20)\n",
    "# Create linear regression object\n",
    "regr = linear_model.LinearRegression()\n",
    "# Train the model using the training sets\n",
    "regr.fit(data_X_train, data_y_train)\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "print'Variance score: %.2f' % regr.score(data_X_test, data_y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "y_true, y_pred = data_y_test, regr.predict(data_X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAEACAYAAABBDJb9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEMNJREFUeJzt3V+MnNV9xvHncbwL4zjrm6xIhMVM2grVRUoAVaaV2zJN\nug4lLbRqqwY1ikotroJACoqgiSovN1xUaiPapheILWqieJFCExXSpFoaMlQmCmxiuwEbArQaxyQB\nDxJyirzqLuHXix2v1svszDue43lnz3w/0sjvjM+c89u1/fjd37x/HBECAGxt28ouAAAwOMIcADJA\nmANABghzAMgAYQ4AGSDMASADA4e57SttH7V9pP3rGdt3pCgOAFCMUx5nbnubpFckXRcRp5JNDADo\nKnWb5Xck/TdBDgDDlTrM/1TSfOI5AQA9JGuz2J6Q9BNJvxIRrSSTAgAK2Z5wrt+V9P3Ngtw2F4EB\ngAsQEe41JmWb5Rb1aLFExMg/Dh48WHoN1EmN1Emd5x5FJQlz2zu0+uHnV1PMBwDoT5I2S0SclTSd\nYi4AQP84A3SDer1edgmFUGc6W6FGiTpT2yp1FpX0pKGuC9kxrLUAIBe2FUP+ABQAUBLCHAAyQJgD\nQAYIcwDIAGEOABkgzAEgA4Q5AHTRarW0uLioVmu0rx9ImAPAJubn51WtVjUzM6Nqtar5+dG9wjcn\nDQFAB61WS9VqVUtLS2uvVSoVnTx5UtPTw7t6CScNAcAAms2mJicnz3ttYmJCzWaznIJ6IMwBoINa\nrabl5eXzXltZWVGtViunoB4IcwDoYHp6WnNzc6pUKpqamlKlUtHc3NxQWyz9oGcOAF20Wi01m03V\narVSgrxoz5wwB4ARxgegADBGCHMAyABhDgAZSHVD5122v2L7edvHbV+XYl4AQDFJbugs6X5J34iI\nP7G9XdKORPMCAAoY+GgW21OSjkbEL/YYx9EsANCnYR7N8gFJr9t+yPYR2w/YriSYFwBQUIow3y7p\nWklfiIhrJZ2VdE+CeQEABaXomb8i6VREfK/9/BFJd3caODs7u7Zdr9dVr9cTLA8A+Wg0Gmo0Gn2/\nL8kZoLaflHRbRLxo+6CkHRFx94Yx9MwBoE9DPZ3f9ockPShpQtL/SLo1Is5sGEOYA0CfuDYLAGSA\na7MAwBghzAEgA4Q5AGSAMAeADBDmAJABwhwAMkCYA0AGCHMAyABhDgAZIMwBIAOEOQBkgDAHgAwQ\n5gCQAcIcADJAmANABghzAMgAYQ4AGSDMASADhDkAZGB7iklsNyWdkfS2pJWI2JtiXgBAMUnCXKsh\nXo+INxLNBwDoQ6o2ixPOBQDoU6oADkmP2160fVuiOQEABaVqs+yLiJ/antZqqD8fEYc3DpqdnV3b\nrtfrqtfriZYHgDw0Gg01Go2+3+eISFqI7YOS/jci/nbD65F6LQDInW1FhHuNG7jNYnuH7Z3t7XdL\n2i/puUHnBQAUl6LNcpmkr9mO9nxfjoiFBPMCAApK3mbZdCHaLADQt6G1WQAA5SPMASADhDkAZIAw\nB4AMEOYAkAHCHAAyQJgDQAYIcwDIAGEOABkgzAEgA4Q5AGSAMAeADBDmAJABwhwAMkCYA0AGCHMA\nyABhDgAZIMwBIAOEOQBkIFmY295m+4jtR1PNCQAoJuWe+Z2STiScDwBQUJIwt71b0o2SHkwxHwCg\nP6n2zD8v6TOSItF8AIA+bB90Atsfk/RaRByzXZfkzcbOzs6ubdfrddXr9UGXB4CsNBoNNRqNvt/n\niMF2pm3fJ+kTkt6SVJH0HklfjYhPbhgXg64FAOPGtiJi053ktXEpA9b29ZLuioibOvweYQ4AfSoa\n5hxnDgAZSLpn3nUh9swBoG/smQPAGCHMASADhDkAZIAwBxJptVpaXFxUq9UquxSMIcIcKKhbWM/P\nz6tarWpmZkbValXz8/MlVIhxxtEsQAHz8/M6cOCAJicntby8rLm5Od1yyy2SVkO+Wq1qaWlpbXyl\nUtHJkyc1PT1dVsnIBEezAIm0Wi0dOHBAS0tLOnPmjJaWlnTgwIG1PfRms6nJycnz3jMxMaFms1lC\ntRhXhDnQQ6+wrtVqWl5ePu/3V1ZWVKvVhlQhQJgDPfUK6+npac3NzalSqWhqakqVSkVzc3O0WDBU\n9MyBAs71zCcmJrSysnJez/ycVqulZrOpWq1GkCOZUi601XUhwhxbHGGNMhDmAJABjmYBgDFCmANA\nBghzoCSc/o+UCHOgBJz+j9T4ABQYMk7/Rz/4ABQYUZz+j4uBMAeGjNP/cTEMHOa2L7H9tO2jto/b\nvi9FYUCuOP0fF0OSnrntHRFx1va7JD0l6a6IeGrDGHrmwDqcUYoiivbMt6dYLCLOtjcv0ere/hsp\n5gVyNj09TYgjmSQ9c9vbbB+V9KqkRkScSDEvAKCYVHvmb0u6xvaUpAXb10fEkxvHzc7Orm3X63XV\n6/UUywNANhqNhhqNRt/vS36cue2/knQ2Iv5mw+v0zAGgT0M7ztz2e23vam9XJM1IOjbovACA4lK0\nWd4v6Z9tW6v/OXwpIr6VYF4AQEGczg8AI4zT+QFgjBDmAJABwhwAMkCYA0AGCHMAyABhDgAZIMwB\nIAOEOcYON1JGjghzjBVupIxccQYoxgY3UsZWxBmgwAbcSBk5I8wxNriRMnJGmGNscCNl5IyeOcYO\nN1LGVlK0Z06YA8AI4wNQABgjhDkAZIAwB4AMEObAFsQlCbDRwGFue7ftJ2wft/2s7TtSFAagMy5J\ngE4GPprF9vskvS8ijtneKen7km6OiBc2jONoFmBAXJJg/AztaJaIeDUijrW335T0vKTLB50XwDtx\nSQJsJmnP3HZN0tWSnk45L4BVXJIAm9meaqJ2i+URSXe299DfYXZ2dm27Xq+rXq+nWh4YC+cuSXDg\nwAFNTExoZWWFSxJkptFoqNFo9P2+JGeA2t4u6euSvhkR928yhp45kAiXJBgfQz2d3/YXJb0eEZ/u\nMoYwB4A+DS3Mbe+T9J+SnpUU7cdnI+LfN4wjzAGgT1xoCwAywIW2AGCMEOYAkAHCHAAyQJgDQAYI\ncwDIAGEOABkgzAEgA4Q5AGSAMAeADBDmAJABwhwAMkCYA0AGCHMAyABhDgAZIMwBIAOEOQBkgDAH\ngAwQ5jhPq9XS4uKiWq3WSMwDoBjCHGvm5+dVrVY1MzOjarWq+fn5UucBUFySe4DanpP0e5Jei4gP\nbjKGe4COsFarpWq1qqWlpbXXKpWKTp48qenp6aHPA2DVsO8B+pCkjyaaCyVoNpuanJw877WJiQk1\nm81S5gHQnyRhHhGHJb2RYi6Uo1araXl5+bzXVlZWVKvVSpkHQH/omUOSND09rbm5OVUqFU1NTalS\nqWhubq7v1kiqeQD0J0nPXJJsVyU91q1nfvDgwbXn9Xpd9Xo9ydpIp9VqqdlsqlarDRTAqeYBxk2j\n0VCj0Vh7fu+99xbqmQ81zPkAFAD6M+wPQCXJ7QcAYMiShLntQ5K+I+lK2z+yfWuKeQEAxSRrs/Rc\niDYLAPStjDYLAKAkhDkwRFyzBhcLYQ4MCdeswcVEzxwYAq5ZgwtFzxwYIVyzBhcbYQ4MAdeswcVG\nmANDwDVrcLHRMweGiGvWoF9Fe+aEOQCMMD4ABYAxQpgDQAYIcwDIAGEOABkgzAEgA4Q5AGSAMAeA\nDBDmAJABwhwAMpDqHqA32H7B9ou2704xJwCguIFP57e9TdKLkj4i6SeSFiV9PCJe2DCO0/kBoE9F\nT+ffnmCtvZJeioiT7YUflnSzpBe6vmuLS33BpH7nOzd+eXlZL7/8svbu3as9e/ZsOm7nzp168803\n+56/2/innnpKCwsL2r9/v/bt29f7iwRw8UTEQA9JfyTpgXXPPyHp7zqMi1wcOnQoKpVK7Nq1KyqV\nShw6dGio850bPzk5GZLWHrfffnvHcZVKJSStbRedv1s9MzMz5629f//+/r9wAD21s7N3FhcZ1HWC\nMQvz06dPr4XjuUelUonTp08PZb5O49c/Tpw40XNcv/NvHH/48OGO8x4+fPiCvgcANlc0zFO0WX4s\n6Yp1z3e3X3uH2dnZte16va56vZ5g+eE6d/uv9fdyPHf7rwtpt/Q7X6fx6z3zzDPas2dP13H9zr9x\n/MLCQse1FxYWaLcAA2o0Gmo0Gv2/sUjid3tIepeklyVVJU1KOiZpT4dxw/hP7KJjz5w9c2CYNKw2\ny+paukHSDyW9JOmeTcZc/K96SM71lKemppL2zIvOd6E980svvbSv+bvVs3//fnrmwBAUDXPuNHSB\nOJqFo1mAYeC2cQCQAW4bBwBjhDAHgAwQ5gCQAcIcADJAmANABghzAMgAYQ4AGSDMASADhDkAZIAw\nB4AMEOYAkAHCHAAyQJgDQAYIcwDIAGEOABkgzAEgA4Q5AGSAMAeADAwU5rb/2PZztn9u+9pURQEA\n+jPonvmzkv5Q0pMJahkJjUaj7BIKoc50tkKNEnWmtlXqLGqgMI+IH0bES5J63mx0q9gqf8DUmc5W\nqFGiztS2Sp1F0TMHgAxs7zXA9uOSLlv/kqSQ9LmIeOxiFQYAKM4RMfgk9rcl3RURR7qMGXwhABhD\nEdGzld1zz7wPXRcrUgwA4MIMemjiH9g+JenXJH3d9jfTlAUA6EeSNgsAoFxDPZrF9l/bft72Mdv/\nYntqmOsXNconQ9m+wfYLtl+0fXfZ9XRie872a7Z/UHYt3djebfsJ28dtP2v7jrJr6sT2Jbaftn20\nXet9Zde0GdvbbB+x/WjZtXRju2n7v9rf02fKrqcT27tsf6WdmcdtX9dt/LAPTVyQdFVEXC3pJUl/\nOeT1ixrJk6Fsb5P0D5I+KukqSbfY/uVyq+roIa3WOOrekvTpiLhK0q9L+tQofj8j4v8k/XZEXCPp\ng5I+bHtfyWVt5k5JJ8ouooC3JdUj4pqI2Ft2MZu4X9I3ImKPpA9Jer7b4KGGeUT8R0S83X76XUm7\nh7l+USN8MtReSS9FxMmIWJH0sKSbS67pHSLisKQ3yq6jl4h4NSKOtbff1Oo/lsvLraqziDjb3rxE\nq/9uR+77a3u3pBslPVh2LQVYI3yeTbtr8ZsR8ZAkRcRbEfGzbu8p84v5C0l8YNqfyyWdWvf8FY1o\n+Gw1tmuSrpb0dLmVdNZuXxyV9KqkRkSM4t7v5yV9RqvnoYy6kPS47UXbt5VdTAcfkPS67YfabasH\nbFe6vSF5mNt+3PYP1j2ebf/6++vGfE7SSkQcSr1+yjoxHmzvlPSIpDvbe+gjJyLebrdZdkv6LdvX\nl13TerY/Jum19k861uj9VLvRvoi4Vqs/SXzK9m+UXdAG2yVdK+kL7TrPSrqn1xuSioiZbr9v+8+1\n+g38cOq1+9GrzhH1Y0lXrHu+u/0aLpDt7VoN8i9FxL+WXU8vEfEz2/8m6Vc1Wp/p7JN0k+0bJVUk\nvcf2FyPikyXX1VFE/LT9a8v217TawjxcblXneUXSqYj4Xvv5I5K6HvAw7KNZbtDqj2E3tT/U2QpG\naQ9jUdIv2a7anpT0cUmjetTAVtg7k6R/knQiIu4vu5DN2H6v7V3t7YqkGUnHyq3qfBHx2Yi4IiJ+\nQat/L58Y1SC3vaP905hsv1vSfknPlVvV+SLiNUmnbF/Zfukj6vHB8rB75n8vaadWe1VHbP/jkNcv\nZFRPhoqIn0u6XatHBR2X9HBEdP2Euwy2D0n6jqQrbf/I9q1l19RJ+4iQP9Pq0SFH238nbyi7rg7e\nL+nb7Z75dyU9GhHfKrmmrewySYfXfT8fi4iFkmvq5A5JX7Z9TKtHs3Q9JJWThgAgAyN7aA4AoDjC\nHAAyQJgDQAYIcwDIAGEOABkgzAEgA4Q5AGSAMAeADPw/GIotn4E+xIoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b85e450>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(y_pred, y_true,  color='black')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEACAYAAACnJV25AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHlNJREFUeJzt3Xt8nFW97/HPr23STFtaqoTKdeYg0LRAbSmXjQWMYBHh\ngHpkK8iBo7gRkavcyp2KUnUDAoIbL6RsQBo2oMWCFClCqCCX2AuttBQsptx6GRGRXmjSZp0/1qTJ\nkyZNMnmSNfPM9/165dVhOfPML4P9suY3a9Yy5xwiIpIcA0IXICIi8VKwi4gkjIJdRCRhFOwiIgmj\nYBcRSRgFu4hIwgyK4yJm1gC8DzQDTc65g+K4roiI9FwswY4P9Grn3HsxXU9ERPIUVyvGYryWiIj0\nQlxh7IA5ZlZvZqfHdE0REclDXK2YSc65lWZWiQ/4pc65Z2K6toiI9EAswe6cW5n7M2tmM4GDgEiw\nm5k2pRERyYNzznpy/163YsxsiJkNy90eChwF/KWT4gr+55prrgleg+pUjapTdbb85COOGfsoYGZu\nRj4IuNc593gM1xURkTz0Otidc38DxsdQi4iIxEBLFNuprq4OXUK3qM74FEONoDrjVix15sPy7eH0\n+InMXH89l4hIUpgZrr8/PBURkcKiYBcRSRgFu4hIwijYRUQSRsEuIpIwCnYRkYRRsIuIJIyCXUQk\nYRTsIiIJo2AXEUkYBbuIJEY2m6W+vp5sNhu6lKAU7CKSCLW1taTTaSZPnkw6naa2tjZ0ScFoEzAR\nKXrZbJZ0Os2GDRu2jKVSKVasWEFlZWXAynpPm4CJSElqaGigvLw8MlZWVkZDQ0OYggJTsItI0ctk\nMjQ2NkbGmpqayGQyYQoKTMEuIkWvsrKSmpoaUqkUw4cPJ5VKUVNTU/RtmHypxy4iiZHNZmloaCCT\nySQm1PPpsSvYRUQKmD48FRERBbuISNIo2EVEEkbBLiKSMAp2EZGEUbCLiCSMgl1EJGEU7CIiCRNb\nsJvZADObb2az4rqmiIj0XJwz9vOAJTFeT0RE8hBLsJvZrsAxwB1xXE9ERPIX14z9JuBiQJvBiIgE\n1utgN7NjgdXOuYWA5X5ERCSQQTFcYxJwvJkdA6SA7czsbufcqe3vOHXq1C23q6urqa6ujuHpRUSS\no66ujrq6ul5dI9Zte83sU8CFzrnjO/jftG2viEgPadteERHRQRsiIoVMM3YREVGwi4gkjYJdRCRh\nFOwiIgmjYBcRSRgFu4hIwijYRUQSRsEuIpIwCnYRkYRRsIuIJIyCXUQSZcOG0BWEp2AXkUR46y34\n93+H446DUt+WSsEuIkWtqQmuvx6qquDBB+EPf4D77w9dVVgKdhEpWk8/DRMmwCWXwLp1rePz54er\nqRDEcYKSiEi/WrUKLr4YfvWr6Pi++8J//RccdliYugqFZuwiUjQ2b4bbboPRo6OhPmwY3Hijn6mX\neqiDZuwiUiSefx6+/W1YsCA6/pWv+FDfZZcwdRUizdhFpKC9+y5885twyCHRUN97b5gzB+67T6He\nnoJdRApSczPccYdvu/zyl63jqRRcdx0sWgSf+Uy4+gqZWjEiUnAWLoQzz/Ttl7aOPx5uuQUymSBl\nFQ3N2EWkYLz/Ppx7LkycGA31TAZmzYLf/lah3h2asYtIcM7BjBlw4YWwenXreHm5X6N+2WUwZEi4\n+oqNgl1EglqyBM46C+rqouNHHQW33uo/JJWeUStGRIJYuxamTIFPfCIa6rvs4rcEeOwxhXq+NGMX\nkX7lHPzmN3D++X7jrhYDB8J3vgNXXw3bbReuviRQsItIv/nrX+Gcc/xsvK3DDvNbAey7b5i6kkat\nGBHpcxs2wDXX+OBuG+o77gh33+0381Kox0czdhHpU48+6mfpr7/eOmbmtwf4/vdh++3D1ZZUCnYR\n6RMrVvg++kMPRccPOsi3XSZODFNXKeh1K8bMBpvZC2a2wMxeNrNpcRQmIsWpsRF++EMYMyYa6iNH\nws9/Ds89p1Dva72esTvnNprZp51z681sIPCsmU1yzj0bQ30iUkSefNKvSX/llej4aaf5sK+sDFNX\nqYmlFeOcW5+7ORj/LuC9OK4rIsVh5Ur/rdHa2uj4uHFw++3wyU+GqatUxbIqxswGmNkCYBVQ55xb\nEsd1RaSwbdoEN9/sd2BsG+rbbefH581TqIcQ14y9GZhgZsOBx83sU865p9vfb+rUqVtuV1dXU11d\nHcfTi0gAzz7rV7YsWhQd/+pX4YYbYKedwtRV7Orq6qhrv79CD5lzLp5qWi5odhWw3jl3Y7txF/dz\niUj/y2b9VgB33hkdr6qCn/4UjjgiTF1JZWY456wnj4ljVcwOZjYidzsFTAYW9va6IlJYNm/2q1pG\nj46G+pAh/oPRl15SqBeKOFoxOwF3mZnh/0Nxj3PuDzFcV0QKxLx5/uCL+vro+Be/6Hvpu+8epi7p\nWOytmE6fSK0YkaLz3ntw5ZV+ZUvbv7577OG31D3mmHC1lYp8WjH65qmIbMU5v4fLxRf7nnqLwYPh\n0kt9jz2VClefbJuCXUQiFi/2q12eeSY6fvTRfpa+555h6pLu0+6OIgLABx/4LxlNmBAN9d12g1//\n2m/mpVAvDpqxi5Q45+CBB/whF++80zo+aJAP+quugqFDw9UnPadgFylhy5bB2WfDE09Ex6ur/Zr0\nsWODlCW9pFaMSAlav96vdtlvv2iojxoF997rN/NSqBcvzdhFSsysWXDuuX6/9BYDBviZ+7XXwogR\n4WqTeCjYRUrE3/7mA/2RR6LjhxziD74YPz5MXRI/tWJEEm7jRn8E3dix0VD/6Efhjjv8ChiFerJo\nxi6SYI8/7lssr73WOmYGp58O06b5cJfkUbCLJNBbb8EFF/hljG1NmOC3Bzj44DB1Sf9QK0akl7LZ\nLPX19WTbfve+Dx7THU1Nfi/0qqpoqI8YAbfd5jfxUqgnn4JdpBdqa2tJp9NMnjyZdDpNbfuz4WJ6\nTHfMnetn5BdfDOvWtY6fcopfr37WWTBwYCxPJQVOuzuK5CmbzZJOp9mwYcOWsVQqxYoVK6js5NTm\nfB7TldWrfZjfc090fJ99/GqXww/P67JSIIIctCFSqhoaGigvL4+MlZWV0dDQEOtjOrN5s/926OjR\n0VAfOhSuvx4WLFColyp9eCqSp0wmQ2NjY2SsqamJTCYT62M68sILfgfG+fOj4yecADfdBLvu2qPL\nScJoxi6Sp8rKSmpqakilUgwfPpxUKkVNTc02Wyr5PKatd9+FM87wXypqG+p77gmPPeY/MFWoi3rs\nIr2UzWZpaGggk8l0O6B7+pjmZn/O6JQpPtxbVFTA5Zf7HntFRb6/gRSyfHrsCnaRArdwoW+7PPdc\ndPzYY+EnP/HH1Ely6cNTkQR5/3047zyYODEa6uk0PPQQPPywQl06pg9PRQqMc1Bb6w+5WLWqdbys\nzLdcrrgChgwJV58UPgW7SAFZutR/keipp6LjRx7pvzlaVRWmLikuasWIFIB16+DSS2HcuGio77QT\n3HcfzJmjUJfu04xdJCDnYOZMOP98ePPN1vGBA/3e6VOnwvDhwcqTIqVgFwlk+XI45xyYPTs6PmmS\n3wpg3LgwdUnxUytGpJ99+CF897t+L5e2oV5Z6deqz52rUJfe0YxdpB/Nnu1n6cuXt46Zwbe+Bddd\nByNHhqtNkkPBLtIP3njD99FnzoyOH3CAb7sceGCYuiSZet2KMbNdzexJM3vZzBab2blxFCaSBI2N\n8KMfwZgx0VDffnsf6M8/r1CX+PV6SwEz+xjwMefcQjMbBswDPu+ce6Xd/bSlgJSUp57ya9KXLo2O\nf+1rPux33DFIWVJkgmwp4Jxb5ZxbmLu9FlgK7NLb64oUq5Ur4eST4YgjoqG+337wxz/6D0gV6tKX\nYl0VY2YZYDzwQpzXlWTqq3M/Q9m0CW65xX+RaMaM1vFhw+DHP/bb7B56aLj6pHTE9uFprg3zIHBe\nbua+lalTp265XV1dTXV1dVxPL0WmtraWb3zjG5SXl9PY2EhNTQ0nnXRS6LLy9txzcOaZ8NJL0fET\nT4Qbb4Sddw5TlxSfuro66urqenWNWLbtNbNBwCPAbOfcLZ3cRz12Afrm3M9Q/v53v0f69OnR8dGj\n/bF1Rx4Zpi5JjpDb9k4HlnQW6iJtxXnuZyjNzfCLX/gAbxvqqRRMm+Zn7gp1CaXXrRgzmwScDCw2\nswWAAy53zj3W22tLMsV17mco8+b5gy9efDE6/vnPw803Q5H8GpJgcayKedY5N9A5N945N8E5t79C\nXbalt+d+hvLPf8LZZ/t1521DPZPxh1489JBCXQqDjsaTYPI5KzQE5+Cee/whF2vWtI6Xl/v++mWX\n+RaMSF/QmaciMfvLX/yXjObOjY4fdZQ/+GKvvcLUJaVDZ56KxOSDD+Cii2D8+Gio77ILPPAAPPaY\nQl0KlzYBE2nDOXjwQfjOd+Dtt1vHBw3yY1df7b9wJFLIFOwiOa++6j8cnTMnOn744X7Drn32CVOX\nSE+pFSMlb/16uOoqv5dL21AfNcp/aFpXp1CX4qIZu5S0hx/2Z4u2/W7UgAF+nfr3vue31xUpNgp2\nKUkNDXDeeTBrVnT84IN922X//YOU1WPFsmRU+pdaMVJSNm70R9CNHRsN9Y98xG8R8Kc/FU+o19bW\nkk6nmTx5Mul0mtra2tAlSYHQOnYpGU884dekv/pqdPw//gN+8APYYYcwdeUjSRupybZpHbtIB95+\nG77yFZg8ORrq48f7Gfovf1lcoQ7J2EhN+o6CXRKrqckfcFFVBfff3zo+fDj85CdQXw+HHBKuvt4o\n9o3UpG8p2CWRnnkGJk6ECy+EtW2OfTn5ZFi2DM45x3/pqFgV60Zq0j/UY5dEWbMGLrkE7rorOj52\nrD/4ImmHdmlVTPJpEzApWZs3+1Utl1/ut9dtMWQIXHMNnH++341RpNjkE+xF/GZUxKuv918o+vOf\no+Nf+hLcdBPstluYukRCUY9ditY//uEPkD744Giof/zjMHu238xLoS6lSDN2KTrNzb6Hfskl/jDp\nFoMH+1bMJZdARUW4+kRCU7BLUVm0yLddnn02On7MMX4J48c/HqYukUKiVowUhX/9y++Hvv/+0VDf\nfXeYORMeeUShLtJCM3YpaM7B//wPXHABrFzZOl5W5teoX3klDB0arj6RQqRgl4L1yit+b5cnn4yO\nH3GEX5NeVRWmLpFCp1aMFJx16/yHoOPGRUN9p52gttZv5qVQF+mcZuxSMJyD3/7W75P+xhut4wMH\n+i0Avvtdv8+LiGybgl0Kwuuv+5OMfve76PgnP+kPvvjEJ8LUJVKM1IqRoD780B9Bt88+0VDfYQeY\nPh3++EeFukhPacYuwfz+93D22fDXv7aOmcE3vwnTpvlTjUSk5xTs0u/efNOvSf/1r6PjEyfC7bfD\ngQeGqUskKWJpxZhZjZmtNrNFcVxPkqmpCa6/HsaMiYb6iBG+j/7CCwp1kTjEsm2vmR0KrAXuds6N\n6+Q+2ra3hD39tN8KYMmS6Pipp8J//ieMGhWmLpFCF+zMU+fcM8B7cVxLkmXVKjjlFH/ARdtQ33df\nmDvXb+alUBeJl1bFSJ/YtAluuw1Gj4Zf/ap1fNgwuOEGmD8fDjssXH0iSdavH55OnTp1y+3q6mqq\nk3ZOmQDw/PO+7bJgQXT8y1/2h0vvskuYukSKQV1dHXV1db26RmxH45lZGnhYPfbS9e67cOmlcMcd\n0fG99vJ7u0yeHKYukWIWrMfe8vy5Hykxzc0+zPfeOxrqFRXw/e/D4sUKdZH+FNdyxxnAn4C9zewN\nM/t6HNeVwrdgAUyaBKef7o+qa3Hccf7D0iuu8CcbiUj/ia0V0+UTqRWTKO+/D1dd5Vsszc2t45mM\nP8nouOOClSaSKPm0YvTNU+kR5+Dee+Gii2D16tbx8nK4+GK/3e6QIeHqExEFu/TAyy/7gy+efjo6\nPnmyX9q4995h6hKRKK1jly6tXQtTpsD48dFQ33lnuP9+v5mXQl2kcGjGLp1yDn7zGzj/fHjrrdbx\ngQP92DXXwHbbhatPRDqmGXsRyWaz1NfXk81m+/y5XnsNPvc5OOGEaKgfdphfCXPDDdFQ76i2/qxX\nRFop2ItEbW0t6XSayZMnk06nqa2t7ZPn2bDBz8T33de3WFpUVsJ//7dvxey3X9e19Ve9IrI1LXcs\nAtlslnQ6zYYNG7aMpVIpVqxYQWVlZWzP8+ij/mzR119vHTODM8/0XzQaObL7tTnn+PDDD/u0XpFS\nEPqbp9JHGhoaKC8vj4yVlZXR0NAQy/VXrIAvfhGOPTYa6gceCC++6NeqdxTqndU2YMAABg4c2Gf1\nisi2KdiLQCaTobGxMTLW1NREJpPp1XUbG+GHP/QHXzz0UOv4yJHws5/Bc8/BAQf0vLbm5mY2b94c\ne70i0j0K9iJQWVlJTU0NqVSK4cOHk0qlqKmp6VVb48kn/SHRl13m++otvv51WLYMzjjDr37Jt7bp\n06fHWq+IdJ967EUkm83S0NBAJpPJOyRXroQLL4T2n2WOG+ePp5s0Kb7a4qhXpNTl02NXsJeIloMv\nrr4aPvigdXy77eB73/PfKB2kbzWIFBztFSMdevZZf/DFonZHjZ90kl+PvvPOYeoSkb6hHnuCZbNw\n2mlw6KHRUK+qgieegBkzFOoiSaRgT6DNm+HnP/fnjd55Z+v4kCHwgx/ASy/BkUeGq09E+pZaMQkz\nb57/QlF9fXT8C1+Am2+GdDpMXSLSfzRjT4j33vMfgB54YDTU99gDfvc7mDlToS5SKjRjL3LOwd13\n+0Mu2u61NXiwP1h6yhRIpcLVJyL9T8FexBYv9qtdnnkmOn700XDrrbDnnmHqEpGw1IopQh984L9k\nNGFCNNR33RUefNBv5qVQFyldmrEXEef8iUUXXADvvNM6PmiQH7vqKhg2LFx9IlIYFOxFYtkyOPts\nv/68repqv/vi2LFByhKRAqRWTIFbvx6uuMIfbtE21EeNgnvv9Zt5KdRFpC3N2AvYrFlw7rl+v/QW\nAwb4mfu118KIEeFqE5HCpWAvQH/7mw/0Rx6Jjh9yiN+Bcfz4MHWJSHFQK6aAbNzod1ocOzYa6h/9\nKNxxh18Bo1AXka5oxl4gHn/ct1hee611zAxOPx2mTfPhLiLSHQr2wN56yy9VfOCB6PiECXD77XDw\nwWHqEpHiFUsrxsyONrNXzOxVM5sSxzWTrqnJ74VeVRUN9REj/IEY9fUKdRHJT69PUDKzAcCrwJHA\nO0A9cKJz7pV299MJSjlz5/qtAF5+OTp+yilw/fV+KaOICOR3glIcM/aDgNeccyucc03AfcDnY7hu\nn8pms9TX15Nts3NWR2OdPW7p0qWd3rez66xeDaeeCp/6VDTUq6o28bOfLeXGG7N9Eurd+b16co2e\nvE7t75Pv6y4iPeCc69UP8CXgF23++f8CP+ngfq5QzJgxw6VSKTdixAiXSqXcjBkzOhzr7HGpVMoB\nW263vW9H19m0ybnbbnNuxAjn/MYA/mfoUOe++tX5rqJiu20+b9y/a2+uUV5e7srKyrr1OrW/T76v\nu0gpy2Vnz3K5pw/Y6gJFFuxr1qzZEswtP6lUylVUVGw1tmbNmm0+rv19O7rP4MGHu3HjGiOBDs6d\ncIJzCxf+vcNa2j5vX/yuPbn+tn7v7r5OqVTKLVmyJK/XXaTU5RPscayKeRvYvc0/75ob28rUqVO3\n3K6urqa6ujqGp++ZhoYGysvL2bBhw5axAQO27kiVlZXR0NBAZWVlp49rf1+gzX0+Akxj48bTWbSo\n9fp77uk/HP3sZ6G+/vWtrtn+eeP+XXt6/W393h1dr7PnfPHFF/N63UVKTV1dHXV1db27SE//S9D+\nBxgI/BVIA+XAQmBMB/fr8/+ydUdfz9grKoY4OM1BNjJDr6hw7tprnduwoetaNGPXjF2kBSFaMf55\nORpYBrwGXNrJffr41+++lr7u8OHDt+r1th3r7HEtAVVRURG574IFzu2115qt2i7HHuvc8uXdr6Wv\nf9feXKOlx96d16n9ffJ93UVKWT7B3uvljt1VaMsds9ksDQ0NZDKZLW/7Oxrr7HHDhg1j7dq1ZDIZ\nyssrufpq32Jpbm697267bebWWwdy/PH+W6Q9qSVOcVy/7TWAbr9O7e+T7+suUqryWe5YssEeB+eg\nttafZrRqVet4WRlcdBFceSUMGRKuPhEpfvkEu7YUyNPSpXDWWfDUU9HxI4/0M/eqqjB1iYhod8ce\nWrcOLr0Uxo2LhvpOO8F998GcOQp1EQlLM/Zucg5mzoTzz4c332wdHzjQ750+dSoMHx6sPBGRLRTs\n3bB8OZxzDsyeHR2fNMkffDFuXJi6REQ6olbMNnz4oZ+J77NPNNQrK+HOO/1mXgp1ESk0mrF3YvZs\nP0tfvrx1zAy+9S247joYOTJcbSIi26Jgb+eNN3wffebM6PgBB/i2y4EHhqlLRKS71IrJaWyEH/0I\nxoyJhvr22/uTjJ5/XqEuIsVBM3b8ssWzzvJr09v62td82O+4Y5CyRETyUtLBvnKl/4bojBnR8f32\n822XQw8NU5eISG+UZCtm0ya4+WYYPToa6sOGwY9/DPPnK9RFpHiV3IzdOTjjDJg+PTp+4olw442w\n885h6hIRiUvJbALW9qkXLvSrXJqb/az9pz/1e7yIiBQa7e7YAef8+vOWP1tMmeJXvFxwAQwe3O9l\niYh0i4K9A+0DXUSkmOQT7In/8FShLiKlJvHBLiJSahTsIiIJo2AXEUkYBbuISMIo2EVEEkbBLiKS\nMAp2EZGEUbCLiCSMgl1EJGEU7CIiCaNgFxFJmF4Fu5mdYGZ/MbPNZrZ/XEWJiEj+ejtjXwx8EXg6\nhloKQl1dXegSukV1xqcYagTVGbdiqTMfvQp259wy59xrQGL2UCyWf9mqMz7FUCOozrgVS535UI9d\nRCRhujzz1MzmAKPaDgEOuMI593BfFSYiIvmJ5QQlM3sKuNA5N38b9wl34KmISBHr6QlKXc7Ye2Cb\nT9zTwkREJD+9Xe74BTN7E/g34BEzmx1PWSIikq9+O8xaRET6R5+vijGzXc3sSTN72cwWm9m5ff2c\nPWVmg83sBTNbkKtzWuiatsXMBpjZfDObFbqWzphZg5m9lHtNXwxdT2fMbISZPWBmS3P/7g8OXVN7\nZrZ37nWcn/vz/QL9e3RZ7jVcZGb3mll56Jo6Ymbn5bKo4PLIzGrMbLWZLWozNtLMHjezZWb2ezMb\n0dV1+mO54ybgAufcPsAhwFlmVtUPz9ttzrmNwKedcxOAccARZjYpcFnbch6wJHQRXWgGqp1zE5xz\nB4UuZhtuAR51zo0BPgEsDVzPVpxzr+Zex/2BicA6YGbgsiLMLA2cDkxwzo3Df353YtiqtmZm+wDf\nAA4AxgP/28z2CFtVxJ3AZ9uNXQo84ZwbDTwJXNbVRfo82J1zq5xzC3O31+L/4uzS18/bU8659bmb\ng/Gvy3sBy+mUme0KHAPcEbqWLhgF/j0JMxsOHOacuxPAObfJOfevwGV15TPAcufcm6ELaedfQCMw\n1MwGAUOAd8KW1KExwAvOuY3Ouc3AXOD/BK5pC+fcM2ydPZ8H7srdvgv4QlfX6de/eGaWwf9X8oX+\nfN7uyLU3FgCrgDrnXKHOiG8CLsZ/l6CQOWCOmdWb2emhi+nE/wL+bmZ35tocvzCzVOiiuvAVoDZ0\nEe05594DbgTeAN4G/umceyJsVR36C3BYrr0xBD9J2i1wTV3Z0Tm3GvxEGdixqwf0W7Cb2TDgQeC8\n3My9oDjnmnOtmF2Bw83sU6Fras/MjgVW594BGYW9lcOkXOvgGHz77dDQBXVgELA/8NNcrevxb3sL\nkpmVAccDD4Supb1cO+M7QBrYGRhmZl8NW9XWnHOvAD8C5gCPAguAzUGL6rkuJ3X9Euy5t2YPAvc4\n537bH8+Zr9xb8d/he3CFZhJwvJm9jp+1fdrM7g5cU4eccytzf2bx/eBC7LO/BbzpnPtz7p8fxAd9\nofocMC/3mhaaA4BnnXP/yLU4fgN8MnBNHXLO3emcO8A5Vw38E3g1cEldWW1mowDM7GPAmq4e0F8z\n9unAEufcLf30fD1iZju0fNKceys+GVgYtqqtOecud87t7pzbA//B1JPOuVND19WemQ3JvUPDzIYC\nR+HfAheU3NvbN81s79zQkRT2h9InUYBtmJxlwL+ZWYWZGf61LLgPogHMrDL35+743WlnhK1oK+3f\njc8Cvpa7/f+ALifHcX7ztEO51SUnA4tzPWwHXO6ce6yvn7sHdgLuyv0fcgD+ncUfAtdUzEYBM3Pb\nSAwC7nXOPR64ps6cC9yba3O8Dnw9cD0dyvWDPwN8M3QtHXHOvZR79zgP39pYAPwibFWd+rWZfQRo\nAr5dSB+Ym9kMoBr4qJm9AVwD/BB4wMxOA1YAX+7yOvqCkohIshT0cjQREek5BbuISMIo2EVEEkbB\nLiKSMAp2EZGEUbCLiCSMgl1EJGEU7CIiCfP/AdHnHniwWh+0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112a1bf50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot outputs\n",
    "plt.scatter(data_X_test, data_y_test,  color='black')\n",
    "plt.plot(data_X_test, regr.predict(data_X_test), color='blue', linewidth=3)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coefficients: \n",
      "[ 0.68250313]\n"
     ]
    }
   ],
   "source": [
    "# The coefficients\n",
    "print 'Coefficients: \\n', regr.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Residual sum of squares: 0.42\n"
     ]
    }
   ],
   "source": [
    "# The mean square error\n",
    "print \"Residual sum of squares: %.2f\" % np.mean((regr.predict(data_X_test) - data_y_test) ** 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "df.click_log = [[df.click[i]] for i in range(len(df))]\n",
    "df.reply_log = [[df.reply[i]] for i in range(len(df))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variance score: 0.95\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "Xs_train, Xs_test, y_train, y_test = train_test_split(df.click_log, df.reply_log,test_size=0.2, random_state=0)\n",
    "\n",
    "# Create linear regression object\n",
    "regr = linear_model.LinearRegression()\n",
    "# Train the model using the training sets\n",
    "regr.fit(Xs_train, y_train)\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "print'Variance score: %.2f' % regr.score(Xs_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEACAYAAABLfPrqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUFOW57/HvgzA4ykVULgoyQ0QEjNGgosn2xN4qXnJB\nV/Y6BDVRI7rcUbdG94oyxiOz197nRKKJmosmJqOiISDGJEI0gKgdEy9IJARlECeSQUDBERVEEAZ4\nzh9VQDNdDdPTPdW332etWVS/Vd39FA39m/d962LujoiISDa6FLoAEREpPQoPERHJmsJDRESypvAQ\nEZGsKTxERCRrCg8REclau8PDzBrMbK2ZLU5p+76ZLTWzRWb2mJn1SllXZ2ZN4fqzUtpHmdliM3vD\nzO7K366IiEhcsul5PACc3aZtLnCMux8PNAF1AGY2EhgHjADOBe4xMwufcy8wwd2HAcPMrO1riohI\nkWt3eLj7X4AP2rTNc/cd4cOXgEHh8lhgurtvc/dmgmAZbWYDgJ7uviDc7iHg/BzqFxGRAsjnnMdl\nwJPh8kBgZcq61WHbQGBVSvuqsE1EREpIXsLDzL4LtLr7tHy8noiIFLeuub6AmV0KfBE4PaV5NXBE\nyuNBYVum9kyvrQtviYh0gLvbvrfquGx7Hhb+BA/MzgG+A4x19y0p280ExptZlZkNAYYCL7v7GmC9\nmY0OJ9AvBh7f2xu6e9n+TJo0qeA1aN+0f9q/8vuJQ7t7Hmb2ayABHGJmbwGTgJuBKuCp8GCql9z9\nKndvNLMZQCPQClzlu/foauBBYH/gSXefnad9ERGRmLQ7PNz9wojmB/ay/feA70W0vwIc2973FRGR\n4qMzzAsokUgUuoROU877Btq/Ulfu+xcHi2t8rCPMzIu5PhGRYmRmeJFNmIuIiCg8REQkewoPERHJ\nmsJDRCQPFi6Eo4+GX/6y0JXEQxPmIiI5uuKKPUOjpQUOPbRw9cQxYZ7z5UlERCrV+vVw0EHp7Zs2\nxV9L3DRsJSLSAbNmRQfH5MkweHD89cRNPQ8RkSy4w+mnQzKZvu6NN+Coo2IvqSAUHiIi7bR6NQwa\nlN4+YgQsWQLWqbMMxUXDViIi7XDvvdHB8dBD0NhYWcEB6nmIiOzV9u1BaKxZk77u3Xehb9/4ayoG\n6nmIiGTw2mvQtWt6cJx3XjD3UanBAQoPEZFIdXVwbMTNI556Cn7/+/jrKTYathIRSbF5MxxwQPS6\njRvhwAPjradYqechIhJKJqOD44YbgmEqBcdu6nmIiADjxsGjj6a3L1oExx0Xfz3FTuEhIhXtvfei\nJ74PPRTeeSeYMG+vlpYWmpubqa2tpW+Zz6Zr2EpEKta0adHB8aMfBRc3zCY4pk2bRk1NDWPGjKGm\npoZp06blr9AipKvqikjFcYfjj4fFi9PXvfUWHHFEdq/X0tJCTU0Nmzdv3tVWXV3NihUrCtID0W1o\nRUTybPly6NIlPTg+/3nYsSP74ABobm6mqqpqj7Zu3brR3Nzc8UKLnMJDRCrG7bfDkUemtz/2GDz/\nfMcvMVJbW8vWrVv3aGttbaW2trZjL1gC2h0eZtZgZmvNbHFKWx8zm2tmy8xsjpn1TllXZ2ZNZrbU\nzM5KaR9lZovN7A0zuyt/uyIiEq21Faqr4cYb09d98AF89au5vX7fvn1paGigurqaXr16UV1dTUND\nQ1lPmrd7zsPMTgU2Ag+5+2fCtsnAOnf/vpndBPRx94lmNhKYCpwEDALmAUe5u5vZfOAad19gZk8C\nd7v7nAzvqTkPEcnJggUwenR6+yWXwIMP5ve9iuVoq6Ka83D3vwAftGk+D5gSLk8Bzg+XxwLT3X2b\nuzcDTcBoMxsA9HT3BeF2D6U8R0Qkr771rejgeP75/AcHBD2Qk046qax7HDvlep5HP3dfC+Dua8ys\nX9g+EHgxZbvVYds2YFVK+6qwXUQkbzZsgN69o9d98gl07x5vPeUo3ycJ5n2Mqb6+ftdyIpEgkUjk\n+y1EpIw88QR8+cvp7f/1X3DrrfHXE4dkMkky6taGnSir8zzMrAaYlTLnsRRIuPvacEjqWXcfYWYT\nAXf3yeF2s4FJwIqd24Tt44HT3P1bGd5Pcx4i0m5jxsC8eentr78ORx8dfz2FUlRzHiELf3aaCVwa\nLl8CPJ7SPt7MqsxsCDAUeNnd1wDrzWy0mRlwccpzREQ65O23g8Ns2wbH0KHBzZwqKTjiks2hur8G\nXgCGmdlbZvZN4DZgjJktA84IH+PujcAMoBF4ErgqpQtxNdAAvAE0ufvsfO2MiFSeX/wCBkbMnN5/\nPzQ1BScESv7p8iQiUpJ27IAhQ4LLibS1Zg307x9/TcWiGIetREQKrrER9tsvPTi+9KXgulWVHBxx\nUXiISEm55RY45pj09jlz4A9/iL+eSqX7eYhISfjkk+ASI1E++gh69Ii3nkqnnoeIFL3nnosOjmuv\nDYapFBzxU89DRIraRRfBr3+d3r5wIXz2s/HXIwGFh4gUpfffh0MOSW/v3Tu4dWw2d/mT/NOwlYgU\nnRkzooPjhz+EDz9UcBQDfQQiUjTc4cQTgyGptpqboaYm9pIkA/U8RKQoNDcHZ4O3DY7Ro4MTAhUc\nxUXhISIF98MfBmeLt/XoozB/fsdvDyudR8NWIlIwra1w8MGwcWP6unXrgnVSnNTzEJGCWLgQqqrS\ng+PCC4O5DwVHcVN4iEjsrr0WTjghvf2552Dq1Pjrkexp2EpEYrNxI/TsGb1u82bYf/9465GOU89D\nRGIxe3Z0cNx6azBMpeAoLep5iEinO/fcIDzaamyEESPir0dyp/AQkU6zZg0cdlh6e20tvPmm7vJX\nyvTRiUinuP/+6OD4xS/gn/9UcJQ69TxEJK927ICjjoLly9PXvf12dKBI6VH2i0jeLFsW3B62bXCc\nfXYwKa7gKB8KDxHJi/p6GD48vf2JJ6Iny6W0adhKRHKyZUvmw2w3bMh8XoeUNvU8RKTDnn8+Ojiu\nvjoYplJwlK+8hIeZ1ZnZEjNbbGZTzazKzPqY2VwzW2Zmc8ysd5vtm8xsqZmdlY8aRCReF18Mp56a\n3r5gAfzkJ/HXI/Eyd8/tBcxqgGeB4e6+1cweAZ4ERgLr3P37ZnYT0MfdJ5rZSGAqcBIwCJgHHOUR\nhZhZVLOIFNAHH0RftPCAA4K7/HXrFn9Nsiczw9079UL2+eh5bAC2AgeaWVegGlgNnAdMCbeZApwf\nLo8Fprv7NndvBpqA0XmoQ0Q62W9/Gx0ct98OH3+s4KgkOU+Yu/sHZvYD4C1gEzDX3eeZWX93Xxtu\ns8bM+oVPGQi8mPISq8M2ESlS7vD5z8NLL6WvW748+kZOUt5yDg8z+xRwPVADrAceNbOLgLbjTR0a\nf6qvr9+1nEgkSCQSHapTRDrmrbeibwE7ahT89a+6y18xSCaTJJPJWN8zH3Me44Ax7n5F+PgbwCnA\n6UDC3dea2QDgWXcfYWYTAXf3yeH2s4FJ7j4/4rU15yFSQHffDd/+dnr7tGkwfnz89Uj7xDHnkY/z\nPJYB/8fM9ge2AGcAC4CNwKXAZOAS4PFw+5nAVDO7k2C4aijwch7qEJE82bYN+vULJsfbeu89OOSQ\n+GuS4pLzhLm7/x14CHgF+DtgwH0EoTHGzJYRBMpt4faNwAygkeCorKvUvRApHn//ezDx3TY4xo0L\n5j4UHAJ5GLbqTBq2EonXDTfAnXemtyeTcNppsZcjHVQqw1YiUuI+/hh69Ihet2kTVFfHW48UP12e\nRKTCPfVUdHDU1QXDVAoOiaKeh0gFGzsWZs1Kb3/tNTjmmPjrkdKh8BCpQO++C/37p7cPGgTNzcE9\nOUT2RsNWIhVmypTo4PjZz2DlSgWHtI96HiIVYscOGDkyuNtfW6tWwUBdJEiyoJ6HSAVoagp6FG2D\n4/TTg1BRcEi2FB4iZe5//geGDUtvnzkTnn5a16aSjtGwlUiZ2roVunePXrd+PfTqFW89Ul7U8xAp\nQy++GB0cV14ZnLuh4JBcqechUmYuuwweeCC9ff58GK3brkmeKDxEysT69XDQQentVVXw0UfBnyL5\nomErkTLw+99HB8f3vgdbtig4JP/U8xApYe7B1W7//Of0df/4Bxx5ZPw1SWVQeIiUqFWr4Igj0tuP\nPTa4J4cOwZXOpGErkRL0059GB8evfgWLFys4pPOp5yFSQrZvh8MPDy5s2Na770LfvvHXJJVJPQ+R\nEvHqq9C1a3pwfPWrwdyHgkPipPAQKQE33gif+Ux6+9NPw2OPxV+PiIatRIrYpk1w4IHR6z7+GA44\nIN56RHZSz0OkSD3zTHRwfOc7wTCVgkMKST0PkSL0b/8Gv/1tevvixcGhuCKFpvAQKSItLdCvX3r7\ngAHBeR26y58Ui7wMW5lZbzN71MyWmtkSMzvZzPqY2VwzW2Zmc8ysd8r2dWbWFG5/Vj5qECl1U6dG\nB8dPfgLvvKPgkOJi7p77i5g9CPzJ3R8ws67AgcDNwDp3/76Z3QT0cfeJZjYSmAqcBAwC5gFHeUQh\nZhbVLFJW3IMjqV57LX3dypUwaFD8NUlpMzPcvVNPFc2552FmvYD/5e4PALj7NndfD5wHTAk3mwKc\nHy6PBaaH2zUDTYAuFC0V6c03oUuX9OD4wheC28MqOKRY5WPYagjwnpk9YGYLzew+MzsA6O/uawHc\nfQ2ws0M+EFiZ8vzVYZtIRbntNhg6NL39d7+DP/1JlxiR4paPCfOuwCjganf/q5ndCUwE2o43dWj8\nqb6+ftdyIpEgkUh0rEqRIrF1K/TsGfzZ1ocfQu/e6e0ie5NMJkkmk7G+Z85zHmbWH3jR3T8VPj6V\nIDyOBBLuvtbMBgDPuvsIM5sIuLtPDrefDUxy9/kRr605DykrL78MJ5+c3n7ZZdDQEH89Up5KYs4j\nHJpaaWbDwqYzgCXATODSsO0S4PFweSYw3syqzGwIMBR4Odc6RIrdlVdGB8eLLyo4pPTk6zyPa4Gp\nZtYNWA58E9gPmGFmlwErgHEA7t5oZjOARqAVuErdCylnGzZED0V16QKbN+suf1Ka8nKobmfRsJWU\nulmzYOzY9Pb//m+45Zb465HKEMewlc4wF+kE7nDmmcH1qdp64w046qj4axLJJ4WHSJ69/TYMjDj4\nfPhwWLIkGK4SKXX6ZyySRz//eXRwTJkCS5cqOKR8qOchkgfbt0NNDaxenb5u7droa1aJlDL9HiSS\noyVLgtvDtg2Or3wlmPtQcEg5UniI5ODmm+HTn05vnzsXZs6Mvx6RuGjYSqQDNm/OfCe/jRsz3zpW\npFyo5yGSpWQyOjiuvz4YplJwSCVQz0MkC1/7GsyYkd6+aBEcd1z89YgUisJDpB3WrYNDD01vP/jg\n4GiqrvqfJBVGw1Yi+zB9enRw3H13ECoKDqlE+mcvkoE7jBoVDEm1tWIFDB4cf00ixUI9D5EIy5cH\nZ4O3DY7PfS64PayCQyqdwkOkjdtvhyOPTG//zW/ghRd0e1gR0LCVyC6trXDQQbBpU/q699+HPn3i\nr0mkWKnnIQK88kpwU6a2wXHxxcHch4JDZE8KD6l4V18NJ56Y3v6XvwRXwxWRdBq2kor10UfQq1f0\nuk8+ge7d461HpJSo5yEV6ckno4Ojvj4YplJwiOydeh5Scc4+O7jqbVuvvw5HHx1/PSKlSOEhFeOd\nd+Dww9PbjzwyuK+47vIn0n767yIV4Ze/jA6O+++Hf/xDwSGSLfU8pKzt2BH0LJqb09e98w4MGBB7\nSSJlIW+/b5lZFzNbaGYzw8d9zGyumS0zszlm1jtl2zozazKzpWZ2Vr5qEEm1dCnst196cJx7bjAp\nruAQ6bh8dtavAxpTHk8E5rn70cAzQB2AmY0ExgEjgHOBe8x0wQfJr1tvhZEj09tnzw6OtBKR3ORl\n2MrMBgFfBP4vcEPYfB5wWrg8BUgSBMpYYLq7bwOazawJGA3Mz0ctUtk++QSqq6PXffQR9OgRbz0i\n5SpfPY87ge8AntLW393XArj7GqBf2D4QWJmy3eqwTSQnf/5zdHD8x38Ew1QKDpH8ybnnYWZfAta6\n+yIzS+xlU9/Luozq6+t3LScSCRKJvb2FVKqvfx2mTk1vf+WV4J4cIuUsmUySTCZjfU9z79B3+u4X\nMPt/wNeBbUA10BP4HXAikHD3tWY2AHjW3UeY2UTA3X1y+PzZwCR3Txu2MjPPtT4pb++/D4cckt7e\ns2ewTnf5k0pkZrh7p84l5zxs5e43u/tgd/8UMB54xt2/AcwCLg03uwR4PFyeCYw3syozGwIMBV7O\ntQ6pPI8+Gh0cP/gBbNig4BDpTJ353+s2YIaZXQasIDjCCndvNLMZBEdmtQJXqXsh2XCHk0+GBQvS\n1/3zn1BbG3tJIhUn52GrzqRhK2lrxYrocDjxRHj5Zd3lTwRKZNhKJC533RUdHI88EvRCFBwi8dGo\nsBS9bduCuY0NG9LXrVsHBx8cf00ilU49Dylqf/sbdOuWHhwXXhjMfSg4RApD4SFF69vfjj5H47nn\nos/pEJH4aNhKis7GjcF5GlE2b4b994+3HhFJp56HFJU5c6KD45ZbgmEqBYdIcVDPQ4rGl78MTzyR\n3t7YCCNGxF+PiGSm8JCCW7s2+t4agwcHJ/3pLn8ixUf/LaWgLr88Ojjuuy84IVDBIVKc1POQgti+\nPfO1p95+Gw47LN56RCQ7+r1OYjdrVnRwnHlmMCmu4BApfup5SKy6dg16HW3953/CHXfEX4+IdIzC\nQ2KxYQP07h29bvVqOPzweOsRkdxo2Eo63R13ZA4OdwWHSClSz0M6VaYr3d57L/z7v8dbi4jkj8JD\nOkVzMwwZEr3u44/hgANiLUdE8kzDVpJ33/hGdHD06xcMUyk4REqfeh6SN+6ZT+qbNw/OOCPeekSk\n8yg8JC+efx5OPTV63fbtOlNcpNzov7TkbNiw6OA477y990ZEpHSp5yEdtmVL5kukL1sWhIqIlCf9\nTigd8vDDmYPDXcEhUu4UHpI1M7j44vT2SZOC4BCR8pdzeJjZIDN7xsyWmNmrZnZt2N7HzOaa2TIz\nm2NmvVOeU2dmTWa21MzOyrUGiUdLS+aT/tatg/r6WMsRkQLKR89jG3CDux8DfA642syGAxOBee5+\nNPAMUAdgZiOBccAI4FzgHrNMX0lSLOrqgvM0orjDwQfHW4+IFFbO4eHua9x9Ubi8EVgKDALOA6aE\nm00Bzg+XxwLT3X2buzcDTcDoXOuQzmMGt92W3v7IIxqmEqlUeT3aysxqgeOBl4D+7r4WgoAxs52/\ntw4EXkx52uqwTYrMkiXw6U9Hr9u6Fbp1i7ceESkeeQsPM+sB/Aa4zt03mlnb30k79DtqfcpAeiKR\nIJFIdLREycJZZ8FTT6W3H3ccLFoUfz0iklkymSSZTMb6nuZ5GHcws67AH4A/uvvdYdtSIOHua81s\nAPCsu48ws4mAu/vkcLvZwCR3nx/xup6P+qT9duyA/faLXjd/PozWAKNI0TMz3L1T55Lzdaju/UDj\nzuAIzQQuDZcvAR5PaR9vZlVmNgQYCrycpzokB3/8Y+bg2LFDwSEiu+Xc8zCzfwGeA14lGJpy4GaC\nQJgBHAGsAMa5+4fhc+qACUArwTDX3AyvrZ5HTA48EDZtSm+/4gq477746xGRjouj55GXYavOovDo\nfBs3Qs+e0etWroRBg+KtR0RyV0rDVlKC7r47c3C4KzhEJDNdGLFCZTot88c/hmuuibcWESk9Co8K\ns3IlDB4cvW7jxmDuQ0RkXzRsVUEmTIgOjl69gmEqBYeItJd6HhVgbzdkmj0bzj473npEpPQpPMrc\n/PlwyinR63R7WBHpKH11lLFjj40OjnPOKa7bw7a0tLBgwQJaWloKXYqItFORfH1IPm3dGhxN9dpr\n6euWLAnOJC8W06ZNo6amhjFjxlBTU8O0adMKXZKItINOEiwz06fDBRdEryu2v8qWlhZqamrYvHnz\nrrbq6mpWrFhB3759C1iZSGnTSYKSFbPo4Lj55uILDoDm5maqqqr2aOvWrRvNzc2FKUhE2k0T5iWs\npaWF5uZmevUawvDhh2bYBg6NXlVwtbW1bN26dY+21tZWamtrC1OQiLSbeh4FlMtE8c65gs99blvG\n4HAv3uAA6Nu3Lw0NDVRXV9OrVy+qq6tpaGjQkJVICdCcR4FMmzaNCRMmUFVVxdatW7nzzjsZNWoU\ntbW1+/zy3D1XEHEZXOBXv4KLLuqMqjvHzh5Ue/ZdRPZNV9Ut0/CImigG6NmzJ9u2baOhoYELMs16\nA7ff3sSNNx4VuW7LFmgzjSAiFUbhUabhsWDBAsaMGcP69esj1+/tiKNMFzQEmDNnLp/97Gf127tI\nhdPRVmUqaqI4VdQRR9u3Zw6OLl2uoqqqO+PGjdO5EiISC4VHAaROFPeMuKFGa2srPXr02DWZftNN\n0DXDcXGzZ8+le/cH2bp1K+vXr2fz5s1MmDBBZ2uLSKfSoboFcsEFF3DmmWfS3NzMwoULuf766+nW\nrRutra1MmDCBE044gaqqKtav/zDja7jDggV9qKqq2mP+ZGfPRcNXItJZNOdRJHYecdSjRw9OOOEE\nNm8+EIjuPfzpT/CFL+x+ns7SFpFUmvOoIH379uWkk05i48aN4SG40cHhvjs4dj5P50qISNzU8ygy\nmSbFDztsO2+/vV/G5+lcCRHZSYfqlnl4pH7hL17clzPPjN5u8uSH+dd/Ha5gEJF2UXiUcXjccccd\n1NXVYWa0tmY+bPdnP/s5119//a4z0fd1AqGISFmHh5mdA9xFMO/S4O6TI7Ypy/C4/PLLaWhoCB9F\n79/pp8P06ZoMF5Hsle2EuZl1AX4CnA0cA1xgZsMLUUvcli5dGgaHkyk4PvkEnn5alywXkeJVqKOt\nRgNN7r7C3VuB6cB5BaolVpdffjmZQgPgRz/6Md27B8u6ZLmIFKtChcdAYGXK41VhW9l74YX9M6z5\nGWBs27ZtV4sOwxWRYlX0Z5jX19fvWk4kEiQSiYLVkh8vRLTtHprs2uY6JKlnoutoKxGJkkwmSSaT\nsb5nQSbMzewUoN7dzwkfTwS87aR5OU6YmxnBcQLX7WzZY31jYyMjRoyIuywRKSNlO2EOLACGmlmN\nmVUB44GZBaolVkEYfpsgNPb8bK+55hoFh4iUhIIMW7n7djO7BpjL7kN1lxailkJw97AHErjyyiu5\n7rrrFBwiUjJ0kqCISJkp52ErEREpYQoPERHJmsJDRESypvAQEZGsKTxERCRrCg8REcmawkNERLKm\n8BARkawpPEREJGsKDxERyZrCQ0REsqbwEBGRrCk8REQkawoPERHJmsJDRESypvAQEZGsKTxERCRr\nCg8REcmawkNERLKm8BARkawpPEREJGsKDxERyVpO4WFm3zezpWa2yMweM7NeKevqzKwpXH9WSvso\nM1tsZm+Y2V25vL+IiBRGrj2PucAx7n480ATUAZjZSGAcMAI4F7jHzCx8zr3ABHcfBgwzs7NzrKFk\nJZPJQpfQacp530D7V+rKff/ikFN4uPs8d98RPnwJGBQujwWmu/s2d28mCJbRZjYA6OnuC8LtHgLO\nz6WGUlbO/4DLed9A+1fqyn3/4pDPOY/LgCfD5YHAypR1q8O2gcCqlPZVYZuIiJSQrvvawMyeAvqn\nNgEOfNfdZ4XbfBdodfdpnVKliIgUFXP33F7A7FLgCuB0d98Stk0E3N0nh49nA5OAFcCz7j4ibB8P\nnObu38rw2rkVJyJSodzd9r1Vx+2z57E3ZnYO8B3gCzuDIzQTmGpmdxIMSw0FXnZ3N7P1ZjYaWABc\nDPwo0+t39s6LiEjH5NTzMLMmoApYFza95O5XhevqgAlAK3Cdu88N208AHgT2B5509+s6XICIiBRE\nzsNWIiJSeWI9wzyfJxWaWZWZTQ+f86KZDU5Zd0m4/TIzuzi+PWwfMzvHzF4Pa7yp0PVkYmaDzOwZ\nM1tiZq+a2bVhex8zmxv+/c4xs94pz8nb5xgXM+tiZgvNbGb4uGz2z8x6m9mjYb1LzOzkctm/sNYl\nYV1Tw1pKet/MrMHM1prZ4pS2WPYp6+9Nd4/tBzgT6BIu3wZ8L1weCfyNYA6mFvgHu3tF84GTwuUn\ngbPD5W8B94TLXyM4rwSgD/Am0Bs4aOdynPu5j7+DLuH+1QDdgEXA8ELXlaHWAcDx4XIPYBkwHJgM\n3Bi23wTclu/PMeb9vB74FTAzfFw2+0cwRPzNcLlr+P+i5Pcv/P+zHKgKHz8CXFLq+wacChwPLE5p\n6/R9ogPfm7H+J23zl3Q+8HC4PBG4KWXdH4GTCb68GlPaxwP3hsuzgZPD5f2Ad9tuEz6+F/haofYz\nYr9PAf6Y8niPfS/mH+D3BL8AvA70D9sGAK/n8XNsiXmfBgFPAQl2h0dZ7B/QC3gzor3k9y/8sns9\n/LMrwUE6ZfFvkyAYU8OjM/epw9+bhbwwYq4nFe56jrtvB9ab2cF7ea1i0ba+kjhR0sxqCX4jeong\nH/JaAHdfA/QLN8vH5/hh+DnG5U6CIwZTJ//KZf+GAO+Z2QPhsNx9ZnYAZbB/7v4B8APgrbDO9e4+\njzLYtwj9OnGfOvy9mffwMLOnwrG2nT+vhn9+JWWbzjipUIf1dhIz6wH8huCouY3s+UVLxOOc3i6P\nr7X3NzL7ErDW3Rft431Lcv8IfiMfBfzU3UcBHxP8tlryn5+ZfYpguLEGOBw40Mwuogz2rR2KYp/y\nHh7uPsbdP5Pyc2z4586z0S8FvghcmPK01cARKY8HhW2Z2vd4jpntB/Ry9/fD9sEZnlMMir2+PZhZ\nV4LgeNjdHw+b15pZ/3D9AODdsD2fn2Mc/gUYa2bLgWnA6Wb2MLCmTPZvFbDS3f8aPn6MIEzK4fM7\nEXje3d8Pf4P+HfB5ymPf2opjn7L+Xor7aKudJxWO9fSTCseHRwIMYfdJhWsIulWjzcwITip8POU5\nl4TL/xt4JlyeA4wJjzLpA4wJ24rFAmComdWYWRXBWOPMAte0N/cTjJ/endI2E7g0XL6EPT+TfH2O\nnc7db3b3we7+KYLP4Rl3/wYwi/LYv7XASjMbFjadASyhPD6/ZcApZrZ/WNMZQCPlsW/Gnj2COPYp\n++/NOCZvHZUzAAAA00lEQVSAUiZhmgguUbIw/LknZV0dwdECS4GzUtpPAF4Nn3t3Snt3YEbY/hJQ\nm7Lu0rD9DeDiOPexnX8P5xD8428CJha6nr3U+S/AdoIjwv4WfmbnAAcD88J9mAsc1BmfY8z7ehq7\nJ8zLZv+A4wh+YVkE/JbgaJqy2D+CX0SXAIuBKQRHL5b0vgG/Bt4GthDM53yT4KCATt8nsvze1EmC\nIiKSNd2GVkREsqbwEBGRrCk8REQkawoPERHJmsJDRESypvAQEZGsKTxERCRrCg8REcna/wdFUsk+\n/fA3FQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11d3f22d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot outputs\n",
    "plt.scatter(Xs_test, y_test,  color='black')\n",
    "plt.plot(Xs_test, regr.predict(Xs_test), color='blue', linewidth=3)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.21630869764168115"
      ]
     },
     "execution_count": 226,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "\n",
    "regr = linear_model.LinearRegression()\n",
    "scores = cross_val_score(regr, df.click_log, df.reply_log, cv = 3)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 228,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.17723080221577134"
      ]
     },
     "execution_count": 228,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "scores = cross_val_score(regr, df.click_log, df.reply_log, cv = 4)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn做logistic回归\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- logistic回归是一个分类算法而不是一个回归算法。\n",
    "- 可根据已知的一系列因变量估计离散数值（比方说二进制数值 0 或 1 ，是或否，真或假）。\n",
    "- 简单来说，它通过将数据拟合进一个逻辑函数(logistic function)来预估一个事件出现的概率。\n",
    "- 因此，它也被叫做逻辑回归。因为它预估的是概率，所以它的输出值大小在 0 和 1 之间（正如所预计的一样）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$$odds= \\frac{p}{1-p} = \\frac{probability\\: of\\: event\\: occurrence} {probability \\:of \\:not\\: event\\: occurrence}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$$ln(odds)= ln(\\frac{p}{1-p})$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$$logit(x) = ln(\\frac{p}{1-p}) = b_0+b_1X_1+b_2X_2+b_3X_3....+b_kX_k$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/logistic.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "repost = []\n",
    "for i in df.title:\n",
    "    if u'转载' in i.decode('utf8'):\n",
    "        repost.append(1)\n",
    "    else:\n",
    "        repost.append(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[194675, 2703], [88244, 1041], [82779, 625]]"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X = [[df.click[i], df.reply[i]] for i in range(len(df))]\n",
    "data_X[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.61241970021413272"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "df['repost'] = repost\n",
    "model = LogisticRegression()\n",
    "model.fit(data_X,df.repost)\n",
    "model.score(data_X,df.repost)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def randomSplitLogistic(dataX, dataY, num):\n",
    "    dataX_train = []\n",
    "    dataX_test = []\n",
    "    dataY_train = []\n",
    "    dataY_test = []\n",
    "    import random\n",
    "    test_index = random.sample(range(len(df)), num)\n",
    "    for k in range(len(dataX)):\n",
    "        if k in test_index:\n",
    "            dataX_test.append(dataX[k])\n",
    "            dataY_test.append(dataY[k])\n",
    "        else:\n",
    "            dataX_train.append(dataX[k])\n",
    "            dataY_train.append(dataY[k])\n",
    "    return dataX_train, dataX_test, dataY_train, dataY_test, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variance score: 0.40\n"
     ]
    }
   ],
   "source": [
    "# Split the data into training/testing sets\n",
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplitLogistic(data_X, df.repost, 20)\n",
    "# Create logistic regression object\n",
    "log_regr = LogisticRegression()\n",
    "# Train the model using the training sets\n",
    "log_regr.fit(data_X_train, data_y_train)\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "print'Variance score: %.2f' % log_regr.score(data_X_test, data_y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "y_true, y_pred = data_y_test, log_regr.predict(data_X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1],\n",
       " array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]))"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_true, y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.00      0.00      0.00        10\n",
      "          1       0.50      1.00      0.67        10\n",
      "\n",
      "avg / total       0.25      0.50      0.33        20\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_true, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "Xs_train, Xs_test, y_train, y_test = train_test_split(data_X, df.repost, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variance score: 0.60\n"
     ]
    }
   ],
   "source": [
    "# Create logistic regression object\n",
    "log_regr = LogisticRegression()\n",
    "# Train the model using the training sets\n",
    "log_regr.fit(Xs_train, y_train)\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "print'Variance score: %.2f' % log_regr.score(Xs_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logistic score for test set: 0.595745\n",
      "Logistic score for training set: 0.613941\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       1.00      0.03      0.05        39\n",
      "          1       0.59      1.00      0.74        55\n",
      "\n",
      "avg / total       0.76      0.60      0.46        94\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print('Logistic score for test set: %f' % log_regr.score(Xs_test, y_test))\n",
    "print('Logistic score for training set: %f' % log_regr.score(Xs_train, y_train))\n",
    "y_true, y_pred = y_test, log_regr.predict(Xs_test)\n",
    "print(classification_report(y_true, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.53333333333333333"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logre = LogisticRegression()\n",
    "scores = cross_val_score(logre, data_X, df.repost, cv = 3)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.62948717948717947"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logre = LogisticRegression()\n",
    "data_X_scale = scale(data_X)\n",
    "# The importance of preprocessing in data science and the machine learning pipeline I: \n",
    "scores = cross_val_score(logre, data_X_scale, df.repost, cv = 3)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn实现贝叶斯预测\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Naive Bayes algorithm\n",
    "\n",
    "It is a classification technique based on Bayes’ Theorem with an assumption of independence among predictors. \n",
    "\n",
    "In simple terms, a Naive Bayes classifier assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature. \n",
    "\n",
    "why it is known as ‘Naive’? For example, a fruit may be considered to be an apple if it is red, round, and about 3 inches in diameter. Even if these features depend on each other or upon the existence of the other features, all of these properties independently contribute to the probability that this fruit is an apple."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "贝叶斯定理为使用$p(c)$, $p(x)$, $p(x|c)$ 计算后验概率$P(c|x)$提供了方法："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$$\n",
    "p(c|x) = \\frac{p(x|c) p(c)}{p(x)}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- P(c|x) is the posterior probability of class (c, target) given predictor (x, attributes).\n",
    "- P(c) is the prior probability of class.\n",
    "- P(x|c) is the likelihood which is the probability of predictor given class.\n",
    "- P(x) is the prior probability of predictor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/Bayes_41.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Step 1: Convert the data set into a frequency table\n",
    "\n",
    "Step 2: Create Likelihood table by finding the probabilities like:\n",
    "- p(Overcast) = 0.29, p(rainy) = 0.36, p(sunny) = 0.36\n",
    "- p(playing) = 0.64, p(rest) = 0.36\n",
    "\n",
    "Step 3: Now, use Naive Bayesian equation to calculate the posterior probability for each class. The class with the highest posterior probability is the outcome of prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Problem: Players will play if weather is sunny. Is this statement is correct?\n",
    "\n",
    "We can solve it using above discussed method of posterior probability.\n",
    "\n",
    "$P(Yes | Sunny) = \\frac{P( Sunny | Yes) * P(Yes) } {P (Sunny)}$\n",
    "\n",
    "Here we have P (Sunny |Yes) = 3/9 = 0.33, P(Sunny) = 5/14 = 0.36, P( Yes)= 9/14 = 0.64\n",
    "\n",
    "Now, $P (Yes | Sunny) = \\frac{0.33 * 0.64}{0.36} = 0.60$, which has higher probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'ABCMeta  BaseDiscreteNB  BaseEstimator  BaseNB  BernoulliNB  ClassifierMixin  GaussianNB  LabelBinarizer  MultinomialNB  __all__  __builtins__  __doc__  __file__  __name__  __package__  _check_partial_fit_first_call  abstractmethod  binarize  check_X_y  check_array  check_is_fitted  in1d  issparse  label_binarize  logsumexp  np  safe_sparse_dot  six'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import naive_bayes\n",
    "'  '.join(dir(naive_bayes))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- naive_bayes.GaussianNB\tGaussian Naive Bayes (GaussianNB)\n",
    "- naive_bayes.MultinomialNB([alpha, ...])\tNaive Bayes classifier for multinomial models\n",
    "- naive_bayes.BernoulliNB([alpha, binarize, ...])\tNaive Bayes classifier for multivariate Bernoulli models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#Import Library of Gaussian Naive Bayes model\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "import numpy as np\n",
    "\n",
    "#assigning predictor and target variables\n",
    "x= np.array([[-3,7],[1,5], [1,2], [-2,0], [2,3], [-4,0], [-1,1], [1,1], [-2,2], [2,7], [-4,1], [-2,7]])\n",
    "Y = np.array([3, 3, 3, 3, 4, 3, 3, 4, 3, 4, 4, 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4 3]\n"
     ]
    }
   ],
   "source": [
    "#Create a Gaussian Classifier\n",
    "model = GaussianNB()\n",
    "\n",
    "# Train the model using the training sets \n",
    "model.fit(x[:8], Y[:8])\n",
    "\n",
    "#Predict Output \n",
    "predicted= model.predict([[1,2],[3,4]])\n",
    "print predicted"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# cross-validation \n",
    "    \n",
    "k-fold CV, the training set is split into k smaller sets (other approaches are described below, but generally follow the same principles). The following procedure is followed for each of the k “folds”:\n",
    "- A model is trained using k-1 of the folds as training data;\n",
    "- the resulting model is validated on the remaining part of the data (i.e., it is used as a test set to compute a performance measure such as accuracy)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[41 16  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0]\n"
     ]
    }
   ],
   "source": [
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplit(df.click, df.reply, 20)\n",
    "# Train the model using the training sets \n",
    "model.fit(data_X_train, data_y_train)\n",
    "\n",
    "#Predict Output \n",
    "predicted= model.predict(data_X_test)\n",
    "print predicted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.65000000000000002"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.score(data_X_test, data_y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.49403904714780522"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "\n",
    "model = GaussianNB()\n",
    "scores = cross_val_score(model, [[c] for c in df.click], df.reply, cv = 5)\n",
    "scores.mean()    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn实现决策树\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# 决策树\n",
    "- 这个监督式学习算法通常被用于分类问题。\n",
    "- 它同时适用于分类变量和连续因变量。\n",
    "- 在这个算法中，我们将总体分成两个或更多的同类群。\n",
    "- 这是根据最重要的属性或者自变量来分成尽可能不同的组别。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/tree.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/playtree.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 在上图中你可以看到，根据多种属性，人群被分成了不同的四个小组，来判断 “他们会不会去玩”。\n",
    "### 为了把总体分成不同组别，需要用到许多技术，比如说 Gini、Information Gain、Chi-square、entropy。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn import tree\n",
    "model = tree.DecisionTreeClassifier(criterion='gini')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.91722595078299773"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplitLogistic(data_X, df.repost, 20)\n",
    "model.fit(data_X_train,data_y_train)\n",
    "model.score(data_X_train,data_y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict\n",
    "model.predict(data_X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.33461538461538459"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# crossvalidation\n",
    "scores = cross_val_score(model, data_X, df.repost, cv = 3)\n",
    "scores.mean()   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn实现SVM支持向量机\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/svm.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- 将每个数据在N维空间中用点标出（N是你所有的特征总数），每个特征的值是一个坐标的值。\n",
    "   - 举个例子，如果我们只有身高和头发长度两个特征，我们会在二维空间中标出这两个变量，每个点有两个坐标（这些坐标叫做支持向量）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/xyplot.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- 现在，我们会找到将两组不同数据分开的一条直线。\n",
    "    - 两个分组中距离最近的两个点到这条线的距离同时最优化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/sumintro.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 上面示例中的黑线将数据分类优化成两个小组\n",
    "- 两组中距离最近的点（图中A、B点）到达黑线的距离满足最优条件。\n",
    "    - 这条直线就是我们的分割线。接下来，测试数据落到直线的哪一边，我们就将它分到哪一类去。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import svm\n",
    "# Create SVM classification object \n",
    "model=svm.SVC() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'LinearSVC LinearSVR NuSVC NuSVR OneClassSVM SVC SVR __all__ __builtins__ __doc__ __file__ __name__ __package__ __path__ base bounds classes l1_min_c liblinear libsvm libsvm_sparse'"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "' '.join(dir(svm))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.90156599552572703"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplitLogistic(data_X, df.repost, 20)\n",
    "model.fit(data_X_train,data_y_train)\n",
    "model.score(data_X_train,data_y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict\n",
    "model.predict(data_X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# crossvalidation\n",
    "scores = []\n",
    "cvs = [3, 5, 10, 25, 50, 75, 100]\n",
    "for i in cvs:\n",
    "    score = cross_val_score(model, data_X, df.repost, cv = i)\n",
    "    scores.append(score.mean() ) # Try to tune cv\n",
    "      "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEYCAYAAAB2qXBEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt0VPW5xvHvGylVQ0WFiiRICqEqVWpB5aZgBFFsreVY\nL0BrxePxsNqK9mZFPVlA057ai1YLthWLoq2KxxuXSpVbUxBBELCCgiDEIKFoERSJFkLynj/2xExC\nrjOT2ZOZ57PWLDN7dmbebGMef9dt7o6IiEisssIuQERE2jYFiYiIxEVBIiIicVGQiIhIXBQkIiIS\nFwWJiIjEJSWCxMxGmtkmM9tsZrc0cE6Bma0zsw1m9rdk1ygiIvWzsNeRmFkWsBkYDuwEVgOj3X1T\n1DkdgReBC929zMw6u/vuUAoWEZFaUqFF0h/Y4u6l7l4BzAK+VuecscBT7l4GoBAREUkdqRAkucDb\nUc93RI5FOxk43sz+ZmarzezqpFUnIiKNahd2Ac3UDugHDAOygRVmtsLd3wy3LBERSYUgKQO6Rz3v\nFjkWbQew293/DfzbzJYCZwC1gsTMtHGYiEgM3N1i/d5U6NpaDfQyszwzaw+MBubWOWcOcK6ZHWFm\nRwMDgI31vZm76+HOpEmTQq8hVR66FroWuhaHP7Zte4v8/B8C++P+Ix56i8TdK83sBmABQbDNcPeN\nZjY+eNmnu/smM3seeBWoBKa7++shli0i0qZNnDiTrVunEIwWxCf0IAFw9+eAU+ocu6/O818Dv05m\nXSIi6aKyEtasgYULg8eyZVUkIkQgNbq2pBUUFBSEXULK0LWooWtRIxOuRUkJTJ8OV1wBJ5wA114L\n//oX/PjHcPnlWUB5Qj4n9AWJiWRmnk4/j4hIS3zwAfztb7BgQdDq2LcPLrgALrww+Gdu1MKKkpJS\nRoyYGune6oDHMdiuIBERaaMqKmDVqprgWL8eBg2CESOC8OjTB7Ia6XcqKSmlsHAmjzwyWUFSTUEi\nIunMHbZsCUJjwQL4+9+hR48gOEaMgHPPhaOOavn7mpmCpJqCRETSzXvvweLFNeFx6FDQ2hgxAoYP\nhy5d4v8MBUkUBYmItHUHDsCLL9bMrnrjDRg6tKbV0bs3WMx/8uunIImiIBGRtsYdXnutJjheeAFO\nPbWm1TFoELRv37o1KEiiKEhEpC3YtQsWLaoJj09/umaAfNgwOP745NajIImiIBGRVPTxx7BsWc3s\nqtJSOP/8mlZHfn7iu6taQkESRUEiIqmgqgr+8Y+aAfKXXoIzzqgZ5+jfH9qlxL4iAQVJFAWJiIRl\nx46arqpFi+C442qCo6AAOnYMu8KGKUiiKEhEJFn274fi4prwePfdYDpudXjk5YVdYfMpSKIoSESk\ntVRvelg9zrFmTdBFVR0cffvCEUeEXWVsFCRRFCQikkglJTXBsWQJ5OTUDJAPHQrZidk8N3QKkigK\nEhGJx/vv1970cP/+2pse5uSEXWHrUJBEUZCISEtUVAQzqqpnV23YAIMH17Q6+vQJd1pusihIoihI\nRKQx7rB5c80AeXFxsIajejHgOefAkUeGXWXyKUiiKEhEpK7du2s2PVy4MFjjUT1APnx4cMOnTKcg\niaIgEUlv1ffPKCurIjc3i6KicfToUXue7YEDsHx5TXBs3gznnVfT6jjllMzormoJBUkUBYlI+qp9\nR79soJz8/EksWDCBjz7K+2SAfPly+MIXalodAwe2/qaHbZ2CJIqCRCR9ffObU3jkkR8RhEi1co48\n8td07TrpkwHyYcOCVeXSfPEGSQrt9iIimezQoWB1+K5d9T8WLKiidogAZPOlL1WxYkUYFUs1BYmI\ntBp32Lu34XCIfuzdC506wYkn1n707BlMyd2zJ4vFi8up2yLJz2/kpuSSFOraEpEW++ij5oXDO+/A\n0UcfHg5duhx+rHPnxnfEbWiMZOHCCYcNuEvLaIwkioJEJHZNdS1FPw4ehK5dDw+D+gIjkesyqmdt\n7dxZRU5O/bO2pOUUJFEUJCK1tbRrqXPnpsPhxBPhmGM0hTadKEiiKEgkFTVn7UNLxdu1VN+jc+e2\nu3utxEdBEkVBIqmmJf36LelaqqhoXjgkumtJ0pOCJIqCRFJNQ2sfevf+NWedNUldS5IStI5EJIXt\n2FH/2oeDB6u44ILa4dCpk7qWpG1SkIi0kl27YNOmLODwtQ8DB2bxrW+FVJhIgmklj0grWLQI+vWD\nK64YR8+ekwjCBKrHSIqKxoVXnEiCaYxEJIEOHYIpU+CBB+Dhh4NtyrX2QVKdBtujKEgkTGVlMHZs\nsNPsn/8czJgSaQviDRJ1bYkkwPz5cOaZwf0unn9eISKZRYPtInGoqIDbb4fHHoMnnoAhQ8KuSCT5\nFCQiMXrrLRgzBo4/HtatC9aAiGQidW2JxOCZZ6B/f7j8cpg3TyEimU0tEpEWOHAAbr45CI9582DA\ngLArEgmfgkSkmd58E666CvLyYO1a3c5VpJq6tkSa4fHHYdAguPZaeOophYhItJRokZjZSOBugmCb\n4e6/qPP6ecAcYFvk0NPu/tPkVimZ6OOP4XvfgyVLgmm9/fqFXZFI6gk9SMwsC5gGDAd2AqvNbI67\nb6pz6lJ3vzTpBUrG2rgx6Mo67TRYsybYcVdEDpcKXVv9gS3uXuruFcAs4Gv1nKdNsyVpHnoIhg6F\nG2+ERx9ViIg0JvQWCZALvB31fAdBuNQ1yMxeAcqAm9399WQUJ5ll/3644QZ46aWgO6tPn7ArEkl9\nqRAkzbEG6O7uH5nZxcBs4OT6Tpw8efInXxcUFFBQUJCM+iQNrF8PV14JAwfCyy9Ddt3biIikieLi\nYoqLixP2fqFv2mhmA4HJ7j4y8nwi4HUH3Ot8TwlwprvvqXNcmzZKi7nDH/8It90Gd96J7hMiGScd\n7pC4GuhlZnnAP4HRwJjoE8ysi7u/E/m6P0EA7jnsnURaaN8+GD8eXnsNli2DU08NuyKRtif0wXZ3\nrwRuABYArwGz3H2jmY03s/+OnHa5mW0ws3UE04SvCqlcSSNr1wbTeTt2DMZEFCIisQm9ayuR1LUl\nzeEO06ZBURFMnRpM8RXJZOnQtSWSNHv3wnXXwfbtsGIF5OeHXZFI2xd615ZIsqxcGXRlde8Oy5cr\nREQSRS0SSXtVVXDXXfDLX8L06TBqVNgViaQXBYmktd274ZprYM8eWL062LlXRBJLXVuStpYuhb59\n4fTTg68VIiKtQy0SSTuVlfDznwczsx58EC6+OOyKRNKbgkTSyq5dcPXVcPBgsGNvbm7YFYmkP3Vt\nSdpYtAjOPBMGD4bFixUiIsmiFom0eYcOwZQp8MAD8PDDMHx42BWJZBYFibRpZWUwdiy0bx9sedKl\nS9gViWQedW1JmzV/ftCVdeGFwW1wFSIi4VCLRNqcigq4/XZ47DF44gkYMiTsikQym4JE2pS33oIx\nY6BTJ1i3Djp3DrsiEVHXlrQZs2dD//5w+eUwd65CRCRVqEUiKe/AAfjxj4PwmDcPBgwIuyIRiaYg\nkZT25pvB/UI+97lgVtZxx4VdkYjUpa4tSVmPPw6DBsG118KTTypERFKVWiSScj7+GL73PViyJJjW\n269f2BWJSGPUIpGUsmlTMAayb1+wV5ZCRCT1KUgkZTz8cLAm5MYb4dFH4Zhjwq5IRJpDXVsSuvJy\n+O53YdWqoDurT5+wKxKRllCLREK1fj2cdRaYBXcwVIiItD0KEgmFO9x/PwwbBrfeGtyAKjs77KpE\nJBbq2pKk27cPxo+H11+HZcvg1FPDrkhE4qEWiSTV2rXBjr0dO8LKlQoRkXSgIJGkcIepU2HkSPjp\nT+EPf4Cjjgq7KhFJBHVtSavbuxeuuw62b4cVKyA/P+yKRCSR1CKRVrVyZbCosHt3WL5cISKSjtQi\nkVZRVQV33QW/+hXcdx+MGhV2RSLSWhQkknC7d8M118CePcEiw7y8sCsSkdakri1JqGXLoG9fOP10\nWLpUISKSCdQikYSorIQ77oBp0+CBB+Dii8OuSESSRUEicdu1C66+Gg4ehJdfhtzcsCsSkWRS15bE\nZfHiYIHh4MHB1woRkcyjFonE5NAh+MlPYMaMYPv34cPDrkhEwtLiIDGzbGAYsMLdd0cdHw887O4f\nJ7A+SUFlZTB2LLRvH2x50qVL2BWJSJhi6dq6FzgVeNDMOkQdXxh5TdLY/PlBV9ZFFwW3wVWIiIi5\ne8u+wexOd/+hmZ0NnOHuf4x6bQjQy90fTHCdza3NW/rzSPNUVMDtt8OsWfDII8GdDEUkPZgZ7m6x\nfn8sLZKDAO6+Gugd/YK7LwO+GGsxkppKS2Ho0GDb97VrFSIiUlssg+3Hm9mJ7r4LqKrn9YNx1iQh\nKykppbBwJmVlVVRVZbFhwzhuuy2P738fsjTPT0TqiCVIfg/81cy+Q/1B0r2lb2hmI4G7CVpIM9z9\nFw2cdzbwInCVuz/d0s+RppWUlDJixFS2bp0CZAPl5OZO4rLLJpCVpWXqInK4Fv//pbu/AtwD/B24\nxsx+Z2Y/MrOJZlYM7GjJ+5lZFjANuAg4DRhjZofd7ihy3h3A8y2tWZqvsHBmVIgAZFNWNoXCwpkh\nViUiqSymdSTuPtPM3gAmA9cDRwD7gPuA21r4dv2BLe5eCmBms4CvAZvqnDcBeBI4O5aapXk2b66i\nJkSqZbNzZ32NTxGROBYkuvsK4CIzawd0Ana7e2UMb5ULvB31fAdBuHzCzHKAUe5+vpnVek0SZ9Ei\nePXVLKCc2mFSTk6OBkdEpH4t/utgZmeZ2W8jU31x90Pu/k6MIdJcdwO3RJfRip+Vke6/H775TXjo\noXHk508iCBOAcvLzJ1FUNC684kQkpcXSIvkecG7k0S8BNZRRe4C+W+RYtLOAWWZmQGfgYjOrcPe5\ndd9s8uTJn3xdUFBAQUFBAkpMX1VVMHEizJ4dbAH/+c/n0b//BAoLf83OnVXk5GRRVDSBHj000C6S\nLoqLiykuLk7Y+8WyIPEO4JfA0e7eooH1Bt7vCOANYDjwT2AVMMbdNzZw/oPAvPpmbWlBYst89FGw\na+/u3fD009CpU9gViUgYwliQ+DrQOxEhAhDpErsBWAC8Bsxy941mNt7M/ru+b0nE52a6XbugoACO\nPhoWLFCIiEjsYmmRHAX8H7AWeMLdN7RGYbFQi6R5NmyASy6B666D//kfMI04iWS0eFsksQTJbOAL\nwElAe+A9gjUlxUCxu78WazHxUpA07fnng+6se+6BMWPCrkZEUkEYXVtvu/vJwDHA+QQ7/n4WuBN4\n1cxWxlqMtK4//AHGjYNnnlGIiEjixNIiuYxgnccKYIm7fxg5fiRwDtDV3f+c6EKbWZtaJPWorIRb\nboF58+DZZ6FXr7ArEpFUkvSurciHtgcKgPfdfVWsH55oCpLDlZcH60Pefx+eegqOPz7sikQk1YTR\ntYW7HwQ+APLM7KxYP1xa186dcN55cOyxwdiIQkREWkMsK9tPNLPVBLvwzgJeMrM3zOyihFcnMXv1\nVRg0CC67DB54ILgtrohIa4hljGQWsBJYSjDgPgD4FnAKMMHdf5/oIltQm7q2gL/+Fa65BqZOhauu\nCrsaEUl1YUz/nebuN9Rz/EJgOnC5u78ca0HxUJDA734HRUXBeMjgwWFXIyJtQbxBEsteW/XeAdHd\nF5jZMOAnwDdjLUhiU1kJP/oRPPccLF8OPXuGXZGIZIpYgqSDmfV193V1X3D3bWa2JwF1SQvs3w9j\nxwYztFasCAbXRUSSJZZZW3cBz5rZNZG7FtZ1VJw1SQuUlcHQofDZzwZjIwoREUm2WNeRfA14DPgX\nMAdYA3wIDAU+4+7XJbLIFtSVUWMkr7wCl14K3/lOsOBQe2aJSCxCWZAY+eA+wK+AEdTcaOp54Cp3\n3xdrQfHIpCB59lm49lq491644oqwqxGRtiyMWVvtgYuANe6+08w6AfkEGziuaOU7JTZVW0YEydSp\n8POfB3tmDRgQdjUi0taFsbL9duARgi4t3P29yDYp7wH3xFqINK2yEm68Mdh88cUXFSIikhpimbX1\naeArQK1h3cjNqOaY2Wh3n5WQ6uQTH34Y7Nh74EAwvVeD6iKSKmJpkRzp7svcfV7dF9x9ITAw/rIk\n2o4dMGQI5OTA/PkKERFJLbEESUczO72R1w/EWowcbu3aYM+sb3wD7rsPPvWpsCsSEakt1nUkz5vZ\nN8zqnXB6XJw1ScTcuXDRRcHdDG++WdN7RSQ1xbqO5ErgYWAXMJfa60iOdfdxCayxJXWlxawt9yA8\nfvUrmD0bzj477IpEJJ2FuY6kL/BLYBg160gWAFdqHUnsDh2Cm26CpUvhL3+BvLywKxKRdNeqQWJm\n8whaGouAxe5eWs85nYGewLvu/lashSRCWw+Sfftg9GioqoLHH4eOHcOuSEQyQWsHSTawnmAA/U2C\nBYf/G/X6MUCWu78fawGJ1JaDZPt2uOQSOOecYMFhu1gmZouIxKC1FyR+F/hfd+/t7l+tDhEzO87M\nngX2AO+Z2T8i+29JDF5+Obh3yLhxwf1EFCIi0pY09SdrgLt/vZ7j9wAXA1uBBwgWJ95vZqdFt1ik\nabNnw/XXw/33w6hRYVcjItJyTQXJYWtCzKwd8B8EN7ga4u67IsfvBP5iZqsjCxOlEe5w113wm98E\nN6M688ywKxIRiU1TQfKZeo4NALKBpdUhAuDu75rZaOBeQEHSiIoKmDAhuAnVihVw0klhVyQiErum\nxkhWmNn4OscuBhz4W92T3X0bNVOBpR4ffBAMqm/fDi+8oBARkbavqRbJncBKM8sHngJ6ATdGXpvd\nwPd8OkG1pZ3SUvjKV6CgAO6+W4PqIpIeGm2RuPsB4ALgdGAF8CegA3Cnu/+j7vmR6cCHWqHONm/V\nqmBm1vXXa3qviKSXZq9sj2zU2BvY5O7rGzjn9wQLF59MXInNl6rrSJ56Cr79bZgxA7761bCrERGp\nLbQtUqIKuAUYBPQhGDs5K6wFiqkWJO7BfllTpwYbMPbtG3ZFIiKHCz1IIkVkA0MI9t3q4u7XxP2m\nsdURapCUlJRSWDiTsrIqunbNorJyHJs35zFvHnTrFlpZIiKNSokgSRVhBklJSSkjRkxl69YpBLOj\nyzn66Em89NIETj9dOy+KSOoK457tUo/CwplRIQKQzUcfTeGOO2aGWJWISOtTkCRIWVkVNSFSLZud\nO6vCKEdEJGkUJAmSm5sFlNc5Wk5Oji6xiKQ3jZEkSElJKV/60lT27asZI8nPn8TChRPo0UNjJCKS\nujTYHiXMIDl0CE48sZTBg2eyf38VOTlZFBWNU4iISMpTkEQJM0gWLoTbbw9WsIuItCWatZUiHn0U\nxo4NuwoRkeRLiSAxs5FmtsnMNkdWytd9/dLIXRjXmdnLZjYsjDob8vHHwQ2qrroq7EpERJIv9K0D\nzSwLmAYMB3YCq81sjrtvijptkbvPjZzfB3iGYCfilPDss8GNqbp2DbsSEZHkS4UWSX9gi7uXunsF\nMAuodf93d/8o6mkHYHcS62uSurVEJJOlQpDkAm9HPd8ROVaLmY0ys43AfGruiRK699+HxYvhssvC\nrkREJByhd201l7vPBmab2bkE90U5pb7zJk+e/MnXBQUFFBQUtGpdTz8NF1wAxx7bqh8jIpIwxcXF\nFBcXJ+z9Qp/+a2YDgcnuPjLyfCLg7v6LRr5nK9Df3d+rczzp038vuCC418jXv57UjxURSZh0mP67\nGuhlZnlm1h4YDcyNPiFyq9/qr/sB1A2RMPzzn7BmDXz5y2FXIiISntC7tty90sxuABYQBNsMd99o\nZuODl3068HUz+xZwkGBDq5SYaPv44zBqFBx1VNiViIiEJ/SurURKdtdW//7ws5/BiBFJ+0gRkYRL\nh66tNmnLFti+Hc4/P+xKRETCpSCJ0WOPBSvZ24XeOSgiEi4FSQzctQhRRKSagiQG69ZBRUUwRiIi\nkukUJDGobo1YzENTIiLpQ7O2WqiyErp3h0WLoHfvVv0oEZGk0KytJFu2DE44QSEiIlJNQdJCGmQX\nEalNXVstcOAA5OTAK6/ASSe12seIiCSVuraS6Lnn4PTTFSIiItEUJC2gbi0RkcOpa6uZPvwQunWD\nbdugU6dW+QgRkVCoaytJZs+G885TiIiI1KUgaSZ1a4mI1E9dW83w7rtw8slQVgbZ2Ql/exGRUKlr\nKwmeeAIuuUQhIiJSHwVJM6hbS0SkYeraakJJSbDL786d8KlPJfStRURSgrq2WtmsWXDFFQoREZGG\nKEiaoG4tEZHGKUgasX497NsHgweHXYmISOpSkDTi0UdhzBjI0lUSEWmQBtsbUFUFPXvC3LnwxS8m\n5C1FRFKSBttbyYoV0KED9OkTdiUiIqlNQdIA3ZddRKR51LVVj4oKyM2Fl16CHj0SUJiISApT11Yr\nWLgQevVSiIiINIeCpB5aOyIi0nzq2qqjvDzo1tq8GU44IUGFiYikMHVtJdi8eTBokEJERKS52oVd\nQKooKSmlsHAmzz1XxcknZ1FSMo4ePfLCLktEJOWpa4sgREaMmMrWrVOAbKCc/PxJLFw4QWEiImlP\nXVsJUFg4MypEALLZunUKhYUzQ6xKRKRtUJAAZWVV1IRItWx27qwKoxwRkTZFQQLk5mYB5XWOlpOT\no8sjItIU/aUEJk4cxxFHTKImTIIxkqKiceEVJSLSRmiwHSgqgtWrSznmmJns3FlFTk4WRUWatSUi\nmSHewfaMD5J//Qt694ZVq4Jt40VEMo2CJEosQXLTTeAOv/1tKxUlIpLiFCRRWhok27bB2WfDxo1a\nyS4imUvrSOJQWBi0SBQiIiKxS4kgMbORZrbJzDab2S31vD7WzP4RebxgZnHft3DdOliyBH7wg3jf\nSUQks4W+15aZZQHTgOHATmC1mc1x901Rp20Dhrr7B2Y2ErgfGNjSz6reT6usrIpNm7L49rfH0aGD\nZmaJiMQj9DESMxsITHL3iyPPJwLu7r9o4PxjgfXuflI9rzU4RlLfflo9e05i0SLtpyUimS0dxkhy\ngbejnu+IHGvIfwF/bemH1Lef1rZt2k9LRCReoXdttYSZnQ9cC5zb0DmTJ0/+5OuCggIKCgoA7acl\nIlKtuLiY4uLihL1fKgRJGdA96nm3yLFazOyLwHRgpLvvbejNooMkWs1+WtFhov20RCTzRP9PNsCU\nKVPier9U+Cu6GuhlZnlm1h4YDcyNPsHMugNPAVe7+9ZYPqSoaBz5+dpPS0Qk0UIfbIdg+i9wD0Gw\nzXD3O8xsPMGg+3Qzux+4DCgFDKhw9/71vE+jCxKrZ21pPy0RkRpa2R4l1k0bRUQyWTrM2hIRkTZM\nQSIiInFRkIiISFwUJCIiEhcFiYiIxEVBIiIicVGQiIhIXBQkIiISFwWJiIjERUEiIiJxUZCIiEhc\nFCQiIhIXBYmIiMRFQSIiInFRkIiISFwUJCIiEhcFSZoqLi4Ou4SUoWtRQ9eihq5F4ihI0pT+I6mh\na1FD16KGrkXiKEhERCQuChIREYmLuXvYNSSMmaXPDyMikkTubrF+b1oFiYiIJJ+6tkREJC4KEhER\niUvaBImZjTSzTWa22cxuCbueZDKzbma2xMxeM7P1ZnZj5PhxZrbAzN4ws+fNrGPYtSaDmWWZ2Voz\nmxt5nqnXoaOZPWFmGyO/GwMy+FrcGrkGr5rZI2bWPpOuhZnNMLN3zOzVqGMN/vyR67Ul8rtzYVPv\nnxZBYmZZwDTgIuA0YIyZnRpuVUl1CPiBu58GDAK+G/n5JwKL3P0UYAlwa4g1JtNNwOtRzzP1OtwD\nzHf33sAZwCYy8FqYWR5wPdDX3b8ItAPGkFnX4kGCv4/R6v35zewLwJVAb+Bi4Hdm1uhAfFoECdAf\n2OLupe5eAcwCvhZyTUnj7rvc/ZXI1/uBjUA3gmvwUOS0h4BR4VSYPGbWDfgy8Meow5l4HY4Bhrj7\ngwDufsjdPyADrwWwDzgIZJtZO+AooIwMuhbu/gKwt87hhn7+S4FZkd+Zt4AtBH9jG5QuQZILvB31\nfEfkWMYxs88BXwJWAl3c/R0IwgY4IbzKkuY3wM1A9HTETLwOPYDdZvZgpJtvupkdTQZeC3ffC9wJ\nbCcIkA/cfREZeC3qOKGBn7/u39Mymvh7mi5BIoCZdQCeBG6KtEzqzu1O67neZvYV4J1I66yxpnha\nX4eIdkA/4F537weUE3RlZNTvBICZ9QS+D+QBOQQtk2+QgdeiCTH//OkSJGVA96jn3SLHMkakyf4k\n8Cd3nxM5/I6ZdYm8fiLwblj1Jck5wKVmtg14DBhmZn8CdmXYdYCgVf62u78cef4UQbBk2u8EwFnA\ncnff4+6VwDPAYDLzWkRr6OcvA06KOq/Jv6fpEiSrgV5mlmdm7YHRwNyQa0q2B4DX3f2eqGNzgXGR\nr68B5tT9pnTi7re5e3d370nwO7DE3a8G5pFB1wEg0mXxtpmdHDk0HHiNDPudiHgDGGhmR0YGjYcT\nTMbItGth1G6pN/TzzwVGR2a29QB6AasafeN0WdluZiMJZqlkATPc/Y6QS0oaMzsHWAqsJ2ieOnAb\nwb/8/yP4v4tS4Ep3fz+sOpPJzM4Dfujul5rZ8WTgdTCzMwgmHXwK2AZcCxxBZl6Lmwn+aFYC64D/\nAj5DhlwLM3sUKAA6Ae8Ak4DZwBPU8/Ob2a3AdUAFQVf5gkbfP12CREREwpEuXVsiIhISBYmIiMRF\nQSIiInFRkIiISFwUJCIiEhcFiYiIxEVBIiIicVGQiIhIXBQkIiISFwWJiIjERUEiIiJxUZCIiEhc\n2oVdgEi6MbM+wA+AD4B/AweAnxHcM+cm4HPAn9398cj544CfAye7+4chlCwSFwWJSAKZ2ZUEt/u9\nyN03mNkQ4FlgDTASuCHyuBV4vPrbCG5z2psm7vsgkorUtSWSIGbWF3gY+LG7b4gcPpbgxkodgL+6\nexVBoGyu/j53fxB4Cdid3IpFEkP3IxFJEDNbCHwe6OF1/sMys64EtzI9keAmQpe6+/yo16e5+w3J\nrFckUdQiEUkAM+sMDAPm1g0RAHf/Z+R+4d8iGDt5rs4pe1u/SpHWoSARSYyeBGMda5o4bxhQHOni\nAsDMhgM5dnDRAAAAtUlEQVQrW7E2kValIBFJjHcj/zxs1pWZ9TazcyJPc4EtdU75CjAfkTZKQSKS\nAO7+FrAAGBp93MxGAN+lpsXxKpAT9fp/AnPq6w4TaSs02C6SIGZ2DPBb4CDBmMengZXuPivqnK7A\nNGAXwRqTpe4+J4RyRRJGQSIiInFR15aIiMRFQSIiInFRkIiISFwUJCIiEhcFiYiIxEVBIiIicVGQ\niIhIXBQkIiISFwWJiIjERUEiIiJx+X9CPNUfsa/GfAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b99ebd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(cvs, scores, 'b-o')\n",
    "plt.xlabel('$cv$', fontsize = 20)\n",
    "plt.ylabel('$Score$', fontsize = 20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "\n",
    "> # 泰坦尼克号数据分析\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "#Import the Numpy library\n",
    "import numpy as np\n",
    "#Import 'tree' from scikit-learn library\n",
    "from sklearn import tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "train = pd.read_csv('/Users/chengjun/github/cjc2016/data/tatanic_train.csv', sep = \",\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  PassengerId  Survived  Pclass  \\\n",
       "0           0            1         0       3   \n",
       "1           1            2         1       1   \n",
       "2           2            3         1       3   \n",
       "3           3            4         1       1   \n",
       "4           4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  app.launch_new_instance()\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:9: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:10: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:11: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n"
     ]
    }
   ],
   "source": [
    "train[\"Age\"] = train[\"Age\"].fillna(train[\"Age\"].median())\n",
    "#Convert the male and female groups to integer form\n",
    "train[\"Sex\"][train[\"Sex\"] == \"male\"] = 0\n",
    "train[\"Sex\"][train[\"Sex\"] == \"female\"] = 1\n",
    "\n",
    "#Impute the Embarked variable\n",
    "train[\"Embarked\"] = train[\"Embarked\"].fillna('S')\n",
    "#Convert the Embarked classes to integer form\n",
    "train[\"Embarked\"][train[\"Embarked\"] == \"S\"] = 0\n",
    "train[\"Embarked\"][train[\"Embarked\"] == \"C\"] = 1\n",
    "train[\"Embarked\"][train[\"Embarked\"] == \"Q\"] = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.12629524  0.31274009  0.2387992   0.32216546]\n",
      "0.977553310887\n"
     ]
    }
   ],
   "source": [
    "#Create the target and features numpy arrays: target, features_one\n",
    "target = train['Survived'].values\n",
    "features_one = train[[\"Pclass\", \"Sex\", \"Age\", \"Fare\"]].values\n",
    "\n",
    "#Fit your first decision tree: my_tree_one\n",
    "my_tree_one = tree.DecisionTreeClassifier()\n",
    "\n",
    "my_tree_one = my_tree_one.fit(features_one, target)\n",
    "#Look at the importance of the included features and print the score\n",
    "print(my_tree_one.feature_importances_)\n",
    "print(my_tree_one.score(features_one, target))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  app.launch_new_instance()\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:6: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:7: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:12: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:13: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "/Users/chengjun/anaconda/lib/python2.7/site-packages/ipykernel/__main__.py:14: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n"
     ]
    }
   ],
   "source": [
    "test = pd.read_csv('/Users/chengjun/github/cjc2016/data/tatanic_test.csv', sep = \",\")\n",
    "# Impute the missing value with the median\n",
    "test.Fare[152] = test.Fare.median()\n",
    "test[\"Age\"] = test[\"Age\"].fillna(test[\"Age\"].median())\n",
    "#Convert the male and female groups to integer form\n",
    "test[\"Sex\"][test[\"Sex\"] == \"male\"] = 0\n",
    "test[\"Sex\"][test[\"Sex\"] == \"female\"] = 1\n",
    "\n",
    "#Impute the Embarked variable\n",
    "test[\"Embarked\"] = test[\"Embarked\"].fillna('S')\n",
    "#Convert the Embarked classes to integer form\n",
    "test[\"Embarked\"][test[\"Embarked\"] == \"S\"] = 0\n",
    "test[\"Embarked\"][test[\"Embarked\"] == \"C\"] = 1\n",
    "test[\"Embarked\"][test[\"Embarked\"] == \"Q\"] = 2\n",
    "\n",
    "# Extract the features from the test set: Pclass, Sex, Age, and Fare.\n",
    "test_features = test[[\"Pclass\",\"Sex\", \"Age\", \"Fare\"]].values\n",
    "\n",
    "# Make your prediction using the test set\n",
    "my_prediction = my_tree_one.predict(test_features)\n",
    "\n",
    "# Create a data frame with two columns: PassengerId & Survived. Survived contains your predictions\n",
    "PassengerId =np.array(test['PassengerId']).astype(int)\n",
    "my_solution = pd.DataFrame(my_prediction, PassengerId, columns = [\"Survived\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Survived\n",
      "892         0\n",
      "893         0\n",
      "894         1\n"
     ]
    }
   ],
   "source": [
    "print my_solution[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(418, 1)\n"
     ]
    }
   ],
   "source": [
    "# Check that your data frame has 418 entries\n",
    "print my_solution.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Write your solution to a csv file with the name my_solution.csv\n",
    "my_solution.to_csv(\"/Users/chengjun/github/cjc2016/data/tatanic_solution_one.csv\", index_label = [\"PassengerId\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.905723905724\n"
     ]
    }
   ],
   "source": [
    "# Create a new array with the added features: features_two\n",
    "features_two = train[[\"Pclass\",\"Age\",\"Sex\",\"Fare\", \"SibSp\", \"Parch\", \"Embarked\"]].values\n",
    "\n",
    "#Control overfitting by setting \"max_depth\" to 10 and \"min_samples_split\" to 5 : my_tree_two\n",
    "max_depth = 10\n",
    "min_samples_split = 5\n",
    "my_tree_two = tree.DecisionTreeClassifier(max_depth = max_depth, min_samples_split = min_samples_split, random_state = 1)\n",
    "my_tree_two = my_tree_two.fit(features_two, target)\n",
    "\n",
    "#Print the score of the new decison tree\n",
    "print(my_tree_two.score(features_two, target))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.979797979798\n"
     ]
    }
   ],
   "source": [
    "# create a new train set with the new variable\n",
    "train_two = train\n",
    "train_two['family_size'] = train.SibSp + train.Parch + 1\n",
    "\n",
    "# Create a new decision tree my_tree_three\n",
    "features_three = train[[\"Pclass\", \"Sex\", \"Age\", \"Fare\", \"SibSp\", \"Parch\", \"family_size\"]].values\n",
    "\n",
    "my_tree_three = tree.DecisionTreeClassifier()\n",
    "my_tree_three = my_tree_three.fit(features_three, target)\n",
    "\n",
    "# Print the score of this decision tree\n",
    "print(my_tree_three.score(features_three, target))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.939393939394\n",
      "418\n",
      "[0 0 0]\n"
     ]
    }
   ],
   "source": [
    "#Import the `RandomForestClassifier`\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "#We want the Pclass, Age, Sex, Fare,SibSp, Parch, and Embarked variables\n",
    "features_forest = train[[\"Pclass\", \"Age\", \"Sex\", \"Fare\", \"SibSp\", \"Parch\", \"Embarked\"]].values\n",
    "\n",
    "#Building the Forest: my_forest\n",
    "n_estimators = 100\n",
    "forest = RandomForestClassifier(max_depth = 10, min_samples_split=2, n_estimators = n_estimators, random_state = 1)\n",
    "my_forest = forest.fit(features_forest, target)\n",
    "\n",
    "#Print the score of the random forest\n",
    "print(my_forest.score(features_forest, target))\n",
    "\n",
    "#Compute predictions and print the length of the prediction vector:test_features, pred_forest\n",
    "test_features = test[[\"Pclass\", \"Age\", \"Sex\", \"Fare\", \"SibSp\", \"Parch\", \"Embarked\"]].values\n",
    "pred_forest = my_forest.predict(test_features)\n",
    "print(len(test_features))\n",
    "print(pred_forest[:3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.14130255  0.17906027  0.41616727  0.17938711  0.05039699  0.01923751\n",
      "  0.0144483 ]\n",
      "[ 0.10384741  0.20139027  0.31989322  0.24602858  0.05272693  0.04159232\n",
      "  0.03452128]\n",
      "0.905723905724\n",
      "0.939393939394\n"
     ]
    }
   ],
   "source": [
    "#Request and print the `.feature_importances_` attribute\n",
    "print(my_tree_two.feature_importances_)\n",
    "print(my_forest.feature_importances_)\n",
    "\n",
    "#Compute and print the mean accuracy score for both models\n",
    "print(my_tree_two.score(features_two, target))\n",
    "print(my_forest.score(features_two, target))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 阅读材料\n",
    "机器学习算法的要点（附 Python 和 R 代码）http://blog.csdn.net/a6225301/article/details/50479672\n",
    "\n",
    "The \"Python Machine Learning\" book code repository and info resource https://github.com/rasbt/python-machine-learning-book\n",
    "\n",
    "An Introduction to Statistical Learning (James, Witten, Hastie, Tibshirani, 2013) : Python code https://github.com/JWarmenhoven/ISLR-python\n",
    "\n",
    "BuildingMachineLearningSystemsWithPython https://github.com/luispedro/BuildingMachineLearningSystemsWithPython"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 作业\n",
    "https://www.datacamp.com/community/tutorials/the-importance-of-preprocessing-in-data-science-and-the-machine-learning-pipeline-i-centering-scaling-and-k-nearest-neighbours"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  },
  "latex_envs": {
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 0
  },
  "toc": {
   "toc_cell": false,
   "toc_number_sections": true,
   "toc_threshold": 6,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}