{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "***\n",
    "***\n",
    "# 计算传播与机器学习\n",
    "\n",
    "***\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](./img/machine.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 1、 监督式学习\n",
    "\n",
    "工作机制：\n",
    "- 这个算法由一个目标变量或结果变量（或因变量）组成。\n",
    "- 这些变量由已知的一系列预示变量（自变量）预测而来。\n",
    "- 利用这一系列变量，我们生成一个将输入值映射到期望输出值的函数。\n",
    "- 这个训练过程会一直持续，直到模型在训练数据上获得期望的精确度。\n",
    "- 监督式学习的例子有：回归、决策树、随机森林、K – 近邻算法、逻辑回归等。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 2、非监督式学习\n",
    "\n",
    "工作机制：\n",
    "- 在这个算法中，没有任何目标变量或结果变量要预测或估计。\n",
    "- 这个算法用在不同的组内聚类分析。\n",
    "- 这种分析方式被广泛地用来细分客户，根据干预的方式分为不同的用户组。\n",
    "- 非监督式学习的例子有：关联算法和 K–均值算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 3、强化学习\n",
    "\n",
    "工作机制：\n",
    "- 这个算法训练机器进行决策。\n",
    "- 它是这样工作的：机器被放在一个能让它通过反复试错来训练自己的环境中。\n",
    "- 机器从过去的经验中进行学习，并且尝试利用了解最透彻的知识作出精确的商业判断。 \n",
    "- 强化学习的例子有马尔可夫决策过程。alphago\n",
    "\n",
    "> Chess. Here, the agent decides upon a series of moves depending on the state of the board (the environment), and the\n",
    "reward can be defined as win or lose at the end of the game:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src = './img/mlprocess.png' width = 800>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 线性回归\n",
    "- 逻辑回归\n",
    "- 决策树\n",
    "- SVM\n",
    "- 朴素贝叶斯\n",
    "---\n",
    "- K最近邻算法\n",
    "- K均值算法\n",
    "- 随机森林算法\n",
    "- 降维算法\n",
    "- Gradient Boost 和 Adaboost 算法\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn做线性回归\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# 线性回归\n",
    "- 通常用于估计连续性变量的实际数值（房价、呼叫次数、总销售额等）。\n",
    "- 通过拟合最佳直线来建立自变量X和因变量Y的关系。\n",
    "- 这条最佳直线叫做回归线，并且用 $Y= \\beta *X + C$ 这条线性等式来表示。\n",
    "- 系数 $\\beta$ 和 C 可以通过最小二乘法获得"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:22.109042Z",
     "start_time": "2019-04-22T08:22:20.811040Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import sklearn\n",
    "from sklearn import datasets\n",
    "from sklearn import linear_model\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.preprocessing import scale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:24.400103Z",
     "start_time": "2019-04-22T08:22:24.390296Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# boston data\n",
    "boston = datasets.load_boston()\n",
    "y = boston.target\n",
    "X = boston.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:25.362696Z",
     "start_time": "2019-04-22T08:22:25.356162Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD',\n",
       "       'TAX', 'PTRATIO', 'B', 'LSTAT'], dtype='<U7')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston['feature_names']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:28.261135Z",
     "start_time": "2019-04-22T08:22:26.683836Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:          boston.target   R-squared:                       0.741\n",
      "Model:                            OLS   Adj. R-squared:                  0.734\n",
      "Method:                 Least Squares   F-statistic:                     108.1\n",
      "Date:                Mon, 22 Apr 2019   Prob (F-statistic):          6.72e-135\n",
      "Time:                        16:22:28   Log-Likelihood:                -1498.8\n",
      "No. Observations:                 506   AIC:                             3026.\n",
      "Df Residuals:                     492   BIC:                             3085.\n",
      "Df Model:                          13                                         \n",
      "Covariance Type:            nonrobust                                         \n",
      "===================================================================================\n",
      "                      coef    std err          t      P>|t|      [0.025      0.975]\n",
      "-----------------------------------------------------------------------------------\n",
      "Intercept          36.4595      5.103      7.144      0.000      26.432      46.487\n",
      "boston.data[0]     -0.1080      0.033     -3.287      0.001      -0.173      -0.043\n",
      "boston.data[1]      0.0464      0.014      3.382      0.001       0.019       0.073\n",
      "boston.data[2]      0.0206      0.061      0.334      0.738      -0.100       0.141\n",
      "boston.data[3]      2.6867      0.862      3.118      0.002       0.994       4.380\n",
      "boston.data[4]    -17.7666      3.820     -4.651      0.000     -25.272     -10.262\n",
      "boston.data[5]      3.8099      0.418      9.116      0.000       2.989       4.631\n",
      "boston.data[6]      0.0007      0.013      0.052      0.958      -0.025       0.027\n",
      "boston.data[7]     -1.4756      0.199     -7.398      0.000      -1.867      -1.084\n",
      "boston.data[8]      0.3060      0.066      4.613      0.000       0.176       0.436\n",
      "boston.data[9]     -0.0123      0.004     -3.280      0.001      -0.020      -0.005\n",
      "boston.data[10]    -0.9527      0.131     -7.283      0.000      -1.210      -0.696\n",
      "boston.data[11]     0.0093      0.003      3.467      0.001       0.004       0.015\n",
      "boston.data[12]    -0.5248      0.051    -10.347      0.000      -0.624      -0.425\n",
      "==============================================================================\n",
      "Omnibus:                      178.041   Durbin-Watson:                   1.078\n",
      "Prob(Omnibus):                  0.000   Jarque-Bera (JB):              783.126\n",
      "Skew:                           1.521   Prob(JB):                    8.84e-171\n",
      "Kurtosis:                       8.281   Cond. No.                     1.51e+04\n",
      "==============================================================================\n",
      "\n",
      "Warnings:\n",
      "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
      "[2] The condition number is large, 1.51e+04. This might indicate that there are\n",
      "strong multicollinearity or other numerical problems.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import statsmodels.api as sm\n",
    "import statsmodels.formula.api as smf\n",
    "\n",
    "# Fit regression model (using the natural log of one of the regressors)\n",
    "results = smf.ols('boston.target ~ boston.data', data=boston).fit()\n",
    "\n",
    "print(results.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:29.198868Z",
     "start_time": "2019-04-22T08:22:29.179869Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "lm = regr.fit(boston.data, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:30.210025Z",
     "start_time": "2019-04-22T08:22:30.203639Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(36.45948838508965,\n",
       " array([-1.08011358e-01,  4.64204584e-02,  2.05586264e-02,  2.68673382e+00,\n",
       "        -1.77666112e+01,  3.80986521e+00,  6.92224640e-04, -1.47556685e+00,\n",
       "         3.06049479e-01, -1.23345939e-02, -9.52747232e-01,  9.31168327e-03,\n",
       "        -5.24758378e-01]),\n",
       " 0.7406426641094095)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm.intercept_, lm.coef_, lm.score(boston.data, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:31.110418Z",
     "start_time": "2019-04-22T08:22:31.107129Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "predicted = regr.predict(boston.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:32.479326Z",
     "start_time": "2019-04-22T08:22:31.916490Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAETCAYAAADH1SqlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJztnXl8VNXZ+L9PhiEkbGETMYogVaiUYgTUSqtiX6VqrVQFte5tpe1rrW0RReuCllYofevys30trX1LXXGlWNoiCm5YFyCgIqCioEYRRMKWQJZ5fn/cmWSWe2fuJLMleb6fz3wmc+659565Sc5zzrOKqmIYhmEYySjK9wAMwzCMwseEhWEYhpESExaGYRhGSkxYGIZhGCkxYWEYhmGkxISFYRiGkRITFoZhGEZKTFgYhmEYKTFhYRiGYaSkU74HkCn69u2rgwYNyvcwDMMw2hQrVqz4TFX7perXboTFoEGDWL58eb6HYRiG0aYQkU1++pkayjAMw0iJCQvDMAwjJSYsDMMwjJSYsDAMwzBSYsLCMAzDSEnevaFEZCOwC2gEGlR1tIj0BuYBg4CNwCRV3Z6vMRqGYXR0CmVnMU5Vj1DV0eHP04BnVPVQ4JnwZ8MwDCPMli1beOGFF3J2v0IRFvGcAcwN/zwXmJDHsRiGYRQMu3btYvr06QwZMoRJkyaxZ8+enNy3EISFAk+JyAoRmRxu66+qn4R/3gz0dztRRCaLyHIRWb5169ZcjNUwDCMv1NXVcddddzFkyBBuvvlmdu/ezebNm7n99ttzcn9R1ZzcyHMAIuWqWiUi+wGLgSuABapaFtVnu6r2Snad0aNHq0VwG4bRHtm9ezcVFRW8++67Cce6d+/Oe++9R9++fVt0bRFZEWUC8CTvOwtVrQq/bwGeAI4CPhWRAQDh9y35G6FhGEZ+6datG6NHJ87nffv2ZcaMGXTv3j3rY8irsBCRriLSPfIzcDLwJrAAuDjc7WLg7/kZoWEYRmEwY8YMOnVyHFi7du3KjTfeyIYNG/jJT35CcXFx1u+fb9fZ/sATIhIZywOq+m8ReQ14WES+B2wCJuVxjIZhGDlhw4YNAAwZMiTh2JAhQ/jxj39MXV0dN9xwA/vvv39Ox5Z3m0WmMJuFYRhtlU8//ZQZM2Zw9913c8opp7BgwQLXfqpKeHGdMdqMzcIwDKOjEu0Ge9ddd9HQ0MCTTz7pGT+RaUGRDiYsDMMwcky8G2x8rMQ111xDoWl98m2zMAzD6DCEQiEefvhhfvGLX/Dee++59hk0aBCXX355VlROrcGEhWEYRg5YvHgx11xzDZWVla7H+/Tpww033MAPf/jDnHg3pYsJC8MwjCyyYsUKpk2bxtNPP+16vLS0lClTpnDVVVfRo0ePHI/OPyYsDMMwssTVV1/N7NmzXY8FAgEmT57MjTfemHM32JZgwsIwDCNLVFRUuLZPnDiRGTNmcNhhh+V4RC3HvKEMwzCyxDnnnBMjMMaNG8err77Kww8/3KYEBZiwMAzDaBV1dXWsXLnS9VhRURGzZs1i5MiR/Pvf/+aZZ55hzJgxOR5hZjA1lGEYRguIdoPdtm0b7733Hr17907od9JJJ7Fy5UqKitr22rxtj94wDCMPPP3004wZM4bzzjuP9957jx07djBz5kzP/m1dUIAJC8MwDN+sXLmSk08+uWm3EM2dd97Jhx9+mLOxzK+sYuzMJQyetpCxM5cwv7Iqq/czNZRhGEYKNmzYwPXXX89DDz3kerxTp05897vfpUuXLjkZz/zKKq59/A1q6xsBqKqu5drH3wBgQkV5Vu5pOwvDMAwPtmzZwhVXXMGwYcM8BcWkSZN46623+MMf/kC/fv1yMq7Zi9Y3CYoItfWNzF60Pmv3tJ2FYRhGHLt27eJ3v/sdv/3tb9m9e7drn3HjxjFr1qy8eDd9XF2bVnsmMGFhGIYRxfPPP8/EiRPZssW9mvPIkSOZNWsWJ598ct4S/R1QVkKVi2A4oKwka/c0NZRhGEYUQ4cOTUgZDk422Pvuu4+VK1cyfvz4vGaEnTp+KIGi2PsHioSp44dm7Z4mLAzDMKLo378/V111VdPnvn37cscdd7Bu3TrOP//8gnCDXb7pcxpDsfUuGkPK8k2fZ+2e+f/WhmEYeeCtt97yLDA0ZcoUBg8ezA033MCGDRv4yU9+UlBpwx98xd1F16s9E5iwMAyjQ7FhwwbOO+88hg8f7lnrunv37qxfv55bbrmlINOGN3oIOa/2TGDCwjCMDoGbG+y1115LQ0ODa/9gMJjL4TXhJ9gu4GEv8WrPBCYsDMNo1+zatYvp06czZMgQ7rrrrhjhsHbtWubOnZvH0cUSCbarqq5FaQ62ixcY5x19kOv5Xu2ZwISFYRjtkrq6Ou666y6GDBnCzTff7BovMXLkSA455JA8jM4dv8F2ow/unTB5F4Xbs4UJC8Mw2hWhUIiHHnqIL37xi1xxxRVs3bo1oU+0G+y4cePyMEp3/AbbzV60nlBcn1C4PVuYsDAMo90Qnw02nkJ0g43GK6iuSCRGFZWPCO7CelKGYRgtYPfu3Z7ZYAFKS0sL1g02mqnjh1ISDCS0N6rG2C68hEo2I7gt3YdhGG2erl27EgrFK2acbLCXXXYZN954I/vvv3/WxzG/sorZi9bzcXUtB5SVMHX80LSywEb6Tnl4dYIbbMR2MaGinEF93NN9DOrTztN9iEhARCpF5B/hz4NF5BUReVdE5olI53yP0TCMwkVEEooPRWeDzZWg8OPJlIoJFeWEPOIlImqm/7znHqnt1Z4JCkJYAFcCa6M+zwJuU9UvANuB7+VlVIZhFBS7du3is88+cz02evRozjnnHMaNG8err77KvHnzOPTQQ3M2tkymDU+lZgp5xN55tWeCvAsLETkQOA34c/izACcCj4a7zAUm5Gd0hmEUAtFusFdffbVnv//7v//jmWeeafNpw91sFyXBQFYTBaYi78ICuB24Gpo8wfoA1aoaiZz5CMhO6SfDMAoaNzfYuXPnsmbNGtf+JSUlLc4G29oypZk0Ok+oKOfWM0dQXlaCAOVlJdx65oisVcHzQ14N3CLyTWCLqq4QkRNacP5kYDLAwIEDMzw6wzDyydNPP80111yT4N0UCoW47rrr+Pvf/56xe2WiTOnU8UNjrgGt2w1MqCjPq3CIJ9/eUGOBb4nIqUAXoAdwB1AmIp3Cu4sDAVcRr6pzgDkAo0ePzqK2zjCMXLFy5UqmTZvG4sWLXY+XlpYycuRIQqFQxuIkktkbkk3Y8d5PZ40qZ+m6rS32hvJLuUfxo/L26jqrqtcC1wKEdxZXqer5IvIIcDbwEHAxkLklhGEYBcmGDRu4/vrrPWtdZ9MNtiX2BrfdyGMrqnKiLsr0LsYP+d5ZeHEN8JCIzAAqgXvyPB7DMLLEli1b+OUvf8ndd9/tmQF20qRJzJgxI2veTX7LlEbvJIpEksZCZJPI9VsT05EuBSMsVPVZ4Nnwz+8BR+VzPIZhZJ877riD66+/3jXJH8C4ceOYNWtWi7yb0gmQ87NSj99JeNWOyGbKjWhybdMoGGFhGEb7J34CP3z7Ts9ssLNmzeLkk09ukXdTugZrPyt1N7uGG9lMuZFPTFgYbZ7WplgwcoPbBL4tcAT9ywfyadUHgJMNdsaMGZx33nmtMl63xGAdv1KPuNJG/q7c1FTx5DsWIpuYsDDaNJlweTSyz5YtW1wn8L2hInp89UIan/kjN9xwAz/4wQ8ykuTPSxVUVV3L4GkLYxYVbosNIOHvSgA3xVNAhJBqRhYq6Sx8cr1IEq+C5W2N0aNH6/Lly/M9DCPHjJ25xNOFcNm0E/MwIiOaiBvs2rVrKZp0BxJ0EQQa4vVfHJ/RWtdefxfRlAQDnDWqnMdWVCXYKoo7FVFdW59wTrzAKAkGMub9FL/wSXb9+ZVVTH10NfWNzaMJBoTZZ49MeywiskJVR6fqVwgR3IbRYvKR19+IxS3yecOGDXznO99h1KhRLF68mI8++ghZu8j1/PJeXTMqKMA71Xc0tfWNPPjKh67qKjdBAY6gyFZUdTq5pW5+ck2MoACob1RuftI9sj0TmBrKaNP4dXk0skP8aviDqk+4dPLt7Fr1Lxrj3GA/e3EeBxz+X9R1Km1qy5aOP95g7aU/8fJo8iKbO9Z0Fj7ba9yFmVd7JjBhYbRp8hGc1Jbw0mtnSt8dWQ2H9tWw87X57HztCbTOfdI7avSRTDq+nHvX7M2Jnj3aYO2llgq4xEoA9CoNsrc+lNO/q0Jf+JiwMNo0+QhOait4Gf+Xb/o8Rk/fGqeAqm072bV6EdXLHiJUU+3aJ94N9oene483W79Hr0WFl83iptOHA7n9u0pn4VNWEnRVlZWVBLM2PhMWRpun0BKuFQpeOvAHX/nQNfJ4+oI1vvMgDehRzLFF7/DpX2aw9/OPXfsHevZnvxMu4sYbrmD8qIOSjjXbXm3JFhWjD+7tKRQyZbj2I3TSWfh8c+QA7nv5A9f2bGHeUIbRThk8baGnrt6L2885wnVyml9ZxdRHVlMfUuo+fY9t/76Tus3vul6jqKQHPY89h+5HnIp0CvrS8xeSV1smdzjpeDilQyafl19vKNtZGEaB09LJy28gWTReQWvTF6yhPlyGTTp3oW7L+wl9JFhMjzHfpsdRZ1JU3GzE9uOZVihebZne4bQ0m20q8vG8TFgYRgHTmsnLTQceLJKmSd+Nqupaxs5c0qQnd/MmCvY6gG4jv8HuyoUABAIBJk+ezGtl49gaKk24ph8DbSrjbq4C0DI9uWdrUs+HMdxTWIjIo8AKYCWwUlW3Zm0UhmG4ksr3PtkE6qYDr6lrSOleWVVdy9RHV9NYu4dQ58TJH6Ds2HPZs2YJJYNH8ZuZv+byCV/zVLmMG9YvJm2G20SfzLibyyj9TE/u2ZrUxw3r52qzGDesX6uum4xkO4szwy8FEJGPaRYeK3AEyCdZG5lhGEnTVviZQOON/4OnLUx5z9C+GraG3WD3m3gzXQ48PKFPoFsvyifPIdC1F3et2EP5wVWuwmncsH6+PK+SGXfHzlySFVWOG5me3LPl2r10nfva3as9EyQTFgOBUcCRUe/fApoc30RkC4kC5MOsjdYwOhhek1dAJOkE6qW2SWbH0MZ6dse5wVY/+1f6nz/LNfNroGuvhPvGCyevif7mJ9e4js9t8s+lfj4bpVEh8y64BWWzUNWPgI+IqlInInOA7wNrgW3AIcCpwClRfT5T1f7ZGrBhdCS8Ji+vVNkfV9cmVdu4XQ8NsWfdi1Q/fy8N1bHKgn1Vb1H77quUHnp00nGmO3ltr6lvUoelUivlUj+fjck9G67dBWWziEdErgbOB45T1Rej2o8EfgF8G/gQ98SMhtGuSdcA21rf+9mL1rtOFmWlQaY8vNqzglvErTJyvdLP3uLzpf/HZ++tdR1nUUkPtD71arVIJCGbK/j3yEqmVkp3td9aY3hbiNvJR+YC33EWIrIReFpVv+9x/BrgRmCsqq7K2Ah9YnEWRr5I15c+E773btcIFAmNSTydADbOPA1ozga7ePFi137FXUr41oWTeaXbWMTDyJ2MspIg07/lREHHZ0f1QoD3w+OLx68AyFZcQyGSKQ+xbGSd7Y+jenJFVWcBbwLXpXFNw2jzeHksTXl4NfMrq2La51dWMeXh1b6zi0afF53ZFeDWM0c0ZUDt2jmQUlAERNiwYQPHnTKhKRtsQp9AgB/96Ee8/94GHp5zO3dceCzBolh7RRFO7iQJX9ON6tr6ptQifnUNyVQoEyrKWTbtRN6feRrLpp3oOSmmk7nVSI904izeBv4rRZ9ngYtaPBrDaIN46eUbVWN08ZFVr9/azZGVY3zhnYiO/9YzR7Bs2onMr6ziZ/OSb+ZD9Xv5/Lm5DPvtv2locHednThxIjNmzOCwww5rakulw0/mXeWVWgTc60JkQoVSKMF92SYfRb/SERZ/AW4TketU9dceffYHerd+WIbRdkiml6+tb+Sn81Yxe9F6auoaktZwjl5Zx08G8dNt9Gp5ysOrUy7epVNnQh+vdRUUxQNHMHD893l4zk9cz02mw09lk/ASjJG6EJkOsiv0zK2ZIluR4clIRw11F/A08EsRmS8iY6IPishpwLk4Rm7D6DD4KbRTVV2bNBhOaI6ejuwokgmWyDWT7VSiKe0c5Krrb45pC/YbxH4Tb6b/ub+mrveQBJWZH1J9dy81VSSHUSq1UibG0x5T1heU62w8qtoYFgh/AL4HnC4i24APgP2Acpy/+TuyMVDDKFQiE52bF5JfolVMP02hUorgFmuhGoLGBqRT55j2LsEiRh5zHD2HHMnurR9RdtyFdD38eESa14stWZVG+t/85JoEYZgsBXi2Ju+OkrI+HzuoFmWdFZGjgCuAk3AERSPwDnCrqt6b0RH6xLyhjHzj5omTLYIBSfAwqt24iurn/kqXg0fS64RLE84pCQY4eXCQ+W/tQjol1j1I5o3kh2wXWjKayaTXl19vqFanKBeRYqBRVRtSds4iJiyMQiDaKJ0tSoNF1NSHmj7v2/wu1c/NZe/GSqchEKR88hw69UjMExQQoXMnoTbq/Ah+01vb5F8Y5Np1Nm1hISLDgKNx7B0bVXVp2qPLAiYsjEyRiX/CbO0yykqC7KitR4H66s1UP38vNWufS+jXdcRJ9D31StdrBAMCSkz2Wb+r0nRWtCZU2gYZr2chjnLzHppdYwUIRa4hIqLtpZKS0a5JNollyiUxWnce7/raGqpr62ncU82O/8xjV+W/IOS+odeGfaiGYmwSESLqq0j96fIUQW7xWWv9eOHkw7XTyC7puM5eA1wMvATMBU7GyUob4VgRuR+41O9uQ0S6AM8DxeGxPKqqN4nIYOAhoA9OgsILVbUujbEahivXz3+D+1/+ICFmAZxJLJMuidEup5lQT4X21bAznA1W69yvUzzwy/Q64RKKBxzmejyaRtUmY7OfaOhkY4/3wsmHa6eRXdJxnb0UWA8cr6p/wonWbkJVlwENwKQ0rrkPOFFVRwJHAN8QkWOAWcBtqvoFYDuO95VhtIr5lVUxgiJCJBbCq1QlOBNlS1xLI0QikN0dSZOjjfXsWvkPPp4zmR3LHnAVFMH9BofdYH/VJCjKSoKerqsRkkU3+3HfjRDvhdMWguPio+Jb8/vtCKSzszgY+L2qJvvrWQEc6/eCYbXV7vDHYPilwInAd8Ltc4HpwP+mMVbDaMLvqj6VuigTapR0S53WrH+J7c/+X0I22AiBnv0p+9oFdB9+PBq19isJBppyM/1s3qqkKrDWTuxurrCFHhxnarL0SWdnsRNHXZSMKmBAOgMQkYCIrAK2AIuBDUB1lHfVRzgxHIaRNpFJwe8EreC5+s9EjiE/AXzRNGx931VQFJX0oNfXL6P8+3fTbfg4bjvnyKY8UeVlJU0G5wkV5SltJV4TeLKJPSCScK9oCj04znJIpU86O4vXgP8SkSJVTfS7cwgBPdMZQHincoSIlAFPAMP8nisik4HJAAMHDkzntkY7JtooWxQ24qZDS1bhfok3fAeSjC8gwoHHTWTHyoWEancCIMFieoz5Nj2OOpOiYicbbHlZSdKUHOVJdjPJJvCp44d6BgiGVJPGZBR6cFxbUJMVGukIi3uAR4BbgOs9+nyZJJlpk6Gq1SKyFPgKUCYincK7iwNxdixu58wB5oDjOtuS+xrtA6+key2NqPYifrXdEvfQ+Il9fmUV0x5dxd6ohW7EHfVn81bR89hz2b7kz3Q74hv0PPZcOnXrHdMv1WrdteARzWnEvcY7oaLcNTIb/KmTCrkuRKGryQqRdNJ9PCYi84BrReSLwN7o4yLybZystI/6vaaI9APqw4KiBCcifBawFDgbxyPqYqKq9RlGPKmS7mUKgZiJuaV672gB07fTXvpvWEhd5asccNFtfLKzLkbozF60ntARp1ByyCiCvWOvWR6ucT170Xp+Nm+Vp7DyWuVH2pKde9Ppw3NeZCcX5KN4UFsnnZ0FOJXytgE/ijSEdwN9gcOBepzJ3i8DgLkiEsCxnzysqv8QkbeAh0RkBlCJs6sxDFfS8dppDUqsEEjXPXR+ZRXTF6yhura+yQ12U5Qb7PBPXual/3dDzDlNk1qn5utFdh2Ab2Hltpvxc26hq5NaSnv9XtmkpbmhjqY5N1Qkp8DrwFRVdS+9lWUsgrvjMnjawpzU8g2IsOHWU33dd2OcPj8yOdfs3cvu1YuoXvYQoZrq2Ot378uUP/2TWeeMSTjXbVLzcvX1k7ajNeca7YtsRHCfDzyrqlWq+grwSri9GEfo7E16AcPIEum6o7aUePtHsvvOr6yKWaX+5l9r+ez1pVQ/f6+nGyxSxAOLlycICy/df2uMtJk28F4//42mQkcBEc47+iBmTBjRomsZhUk6rrP34hIcp6r7TFAY+WTq+KFOvqMsUx5n/Eym356+YE3Tz08//TQr7vwhny34jbcb7ImXUX7ZHwn09e/V1xKX10ycG8/189/gvpc/aBKmjarc9/IHXD//jbSvZRQu6QiLlIjI2SLyy0xe0zD8EJ+uOxts37MvJso3mX67uraeAy65g7JDR3PSSSdR9+mGhD4SLKbnV86h/Ad/oseYM5BOqSOuo2lNLEMm4yAefMW93plXu9E2SaqGEpFzgeWq+q7P6w0HrgNuSNXRMPySyj312sdfz8k4aupDvrydGnZ8yvbn/uaaDRYAKaLbyPH0HHtejBsswHlHH+R7PK0x0mbSwOvlnpxpt2Ujv6SyWTwAqIjsxnEGOUlEPgZWAm+oarwDdlecfE+GkRH8eO241WbIFvHeTr1KgwlxCI01OzwFRZ8Rx9PlmO9w8CFfYFCfEl5+b3ur9PytiWXIVByEV2BhOrsko/BJJSymAEcCo3Aiq8fSnPupPuziuhLHvXUncB6wKTtDNToihZi9NNoIfNPpw5n66OoYNVjxgMMoHfY1ata90NTWZeAInnvkzxx11FE5HWsuOO/og7jv5Q9c2432Q1Jhoaq3RX4WkRDwV+A/OALkSGAETrbYaH6c2SEaHRk/XjtFAqEcajwiRuBQKJSgzomkFyn72gXUvP0SwT4H0euESxlSMbZdCgqgaTdk3lDtm3SC8mYBK1S1KUI7HEx3OI7A6BM+/oLH+YaRNn7SMnzlkN4s2/B5TsZTEgww5aRDmTdvHjfeeCNPPPEEEyoOTyieRO9y9j//N3QecCilnYNc/Q3fKc/aJDMmjDDhkGNyXYmw1TW4CwULymubpPqDn19ZxdRHVseUAA0WCbMnjgTwlXo8HQICXo5V5WUljC/byoI/z2bFihUAHHXCeDqNv9o1jUZr/4mtLKnhRTrlbVOR8RrcIvItHHvF71W14HziTFi0nlxPTn7+4OdXViXYBIIB4ZwxB/HYiqqcpPkAqPt0AyM+XshTTz2VcKz/+bPpcuAXXccfTfzzHTesH0vXbfVV3jVCquR/RscgkxH4GY/gBv4bx05xi8cNrQZ3GybbxWDcBJEf4/XsResTYijqG7VJP55t6qs3U/3CvdS89RyucddSRN3mt5uEhZfx3e35RhuF/ZR3BSd+w4r0GPlIsZ6OsPgy8JSq1ngc/4KIPAtMUdWHWj0yI6dk0+vISxB57Qqi/+C9/vizLSgaa3aw46WH2FX5Lwg1uPYpHfpVyo67MCEbrNuY/SQ7jDzv5Zs+T6pay7c3mJF/ylxctiPt2SIdYdGbJG6xqvqOiFQBl+CkFjfaENlcqXgJIi///Gjjda7yPkUI1dWy87X57Hz1cdda1wA9DhnJ4gfn8LMlu5Ma36N3U35FW/yOwwsr0tOx8VorZXMNlU66j62kLplaieNOa7QxMpkrKJ50dgfxKSfc0lJkI9RLGxvYtXIhVX+8jB0v3u8qKIL9BrHfxJvpdfYMjjrqqKQpM6LLuWbj/9eK9HRsdtQm7iqStWeCdITFS8Bp4SJFXlTj1LYw2hjZrJnsd2IrKwm6GoeLOzX/mfYqDWYnHbkqO159PCFtOECgZ3/6fHMKAy69k5JDRlHeq7Rp1xDZIUFsPeqW1NjwW5vbivQY2VzceZGOsPhfnNoVfxURL/XVcGBXq0dl5JwJFeXceuYIystKEGInvtbiJojcEHHGMb+yirEzlzBo2kJ+Om8V1VGrpb31Ibp29jeppoN0ClL2tQti2opKetDr65dR/v276TZ8HCJFlAQDjBvWr2nXAM4OKTKBR55XMjVR5PlecMzAhOedLEVGpn8vRtvFLdNyMCBZXUSkU1b1WRG5E/gJcLCITFPVZyPHRWQicArwz4yP0sgJ2aqZHO3ZlMz+sL2mnvP/9B9WfrDDc1WeCVdZVUVcJuWuhx/Pzlcfp2H7x/QYPYEeR59JUXHXpuMBEW49c4QvZwAvW0sq18blmz53tVlccMzAjAS9WexGOyJ+i51l58C0UpSr6k+BmcBo4BkR+UxEVojIhzhG7VD4uGHEMKGinGXTTkyoCRHPsg2fZy12or56M1sXzGbXa0+4Hhcpou9pP+eAyX+i7LgLYwRFSTDA/0wayYSKcl/OAC1V682YMIILjhnYtMMIiGRUUETbUSJeadFp1422wexF62MCVQHqQ8rsReuzds90a3CjqteJyCPAz4GTgYrwoTeBaaq6LIPjM9oZ+fDiadxTzY7/zGtyg937/gq6fvlkAl26JfTtvN/ghLbyuBW4nxQkrUkBnq3UGYWYlNFoGQUVZ5GsloWqVgIXhvtZWVUDiFVx9CwJIgLVNfUxE2UuXWFD+2ocN9jXnojxbgrt3c3Olx+l1wmX+LrOJztqWb7p86YJder4oa5xInv2NcSUU82WWq+l5GOCMbKDnwVLpkm2s3gACInI4ar6tojMwklHvlJV34l0UlWrX5EHCkX3HBlHVXUtQrPaNNooHR2d7DXRZhJtrGf36kVUL3vI1bsJYO+Hb6AaQiS1JjakNNkRZkxoNi7f/OSamMCoQo+uzscEY2QHt/+jbHvJJRMWV+HUsYhEbE8lPBeEiyGtormWxUrgLVXNXRWaDky2U3P4ub+bgEhmX4uoOyLG3UwnAARQDVGz7kWqn7/XtdY1QKDHfpQddyFdDz/el6CI5sFXPmxSD0XcY+OjaGvrG5m+YE1BCot8TDDqsZNMAAAgAElEQVRGdshkpUO/eAoLVf1dXNOJNNexOBKnENLXaJ4j9orIGzhpyi/PwliNMPnUPccLqnQcMCLqjoh65vr5b/iKVvZD7cZVVD/3V+o2u1cALirpQc+vnEP3ilORTokpEYJFQoNq0gjY+CBCL/VNdW19jDqqUMjHBGNkj1yrOdNynQWejXwWkVJgJLECpAIYA5iwyCL50D1H7yZaygFlJcyvrEpQ37SGuk83sP3Zv7J3Y6XrcQkWu7rBRhMxYANJVWTxMRDJ7C+FajQuNDuK0XZI2xsqQjih4H/CLwBEpDPwpQyMy0hCrnXPXumy0yESzBafbry17Fr1b3dBIUV0GzmenmPPo1O33p7nR8c9zK+sokuwyPN7xpcJnTp+KD+dt8q1rxmNjfaGp9JWRB4VkWtFZLyI9PNzMVWtU9WVmRue4UY2U3O4kU7qisjau6wkSK/SYEzU8dJ1WzMqKADKxn4HCRbHtJUO/SoHfP9/6TP+8qSCApon9YhAdNvxeMU6TKgop5dHlk8zGhvtjWQ7izPDr4hR+2NgBY4xewWOV5S7FdHIKrnWPadaJUeM3PHxCPH8zGMV3hoC3XrRY8y32fHSQxQPHEGv4y+h+AD/QrNnSdCzkAykjri+6fThZjQ2OgTJhMVAHG+oI6PevwWcHukgIltIFCAFV0WvPZJK95xJ19pUsRGKk+DP6x6RsbRkTxFxg9276XX6TrjWNU1Hj6POpLj8i3QZfGTC8bKSIPWNIfbUJe6MgkXCnrqGGDffeFLZaMxobHQU0qrBLSJzgO8Da4FtwCHAAcQ6xXymqv19Xu8g4G9A//A15qjqHSLSG5gHDAI2ApNUdXuya1lZ1WbcbAwCHDukNxu31aac1KKN2V41J9wIFAn/E1Ub++PqWspKg+ze25CQmiAVbm6w/c66gdIvHO37GmUlQfbUNbiqviLqo1SG9oAIG249NY2RG0bbIuNlVUXkauB84DhVfTGq/UjgF8C3gQ9Jz5uyAaey3koR6Q6sEJHFOAWUnlHVmSIyDZgGXJPGdTs0bjYGxcm7FMErNiNe0KRTka4xpPxs3io6BaRpgm6J15OXG2z1c3MpOWQ0UuQv66wInjYSVX+5/1tTkS96d1dWGmy6p+0+jLZIujW4H4wWFABhg/ZZInINcCNO/IUvwjaPT8I/7xKRtUA5cAZwQrjbXByXXRMWPvHrieMWm9GSOgzRKN4TdCpSucE27PiU+q0b6dx/SMprde0coDqJoKqurafcR+qRVIkP3ZhfWcX0BWti1FvRQjPXQZSGkQnSERb9cVRPrqjqLBE5E7gOmJTuQERkEE6cxitA/yjj+ebwvd3OmQxMBhg4cGC6t2xXRK9ii9JQHVVV1zJ25pIm1VQuS5hGqK/eTPUL91Lz1nPuHaSIbkd8g57HnpvSuylCSBWR5GUm/aQeSddQ7dfN2BL4GW2NdITF28B/pejzLHBRuoMQkW7AY8BPVXVntJFSVVVEXP/lVXUOMAccm0W6920vtEZ1JDQbcXMtKBprdrDjpYeassG6UTr0q5QddyHB3ulNqrX1yTPP9CoNNk3UUx5e7frMykqCaU/m6ezMLBbDaEukIyz+AtwmItep6q89+uwP+Fv6hRGRII6guF9VHw83fyoiA1T1ExEZAGxJ55odjdaojtKRsEU4BUtaS6iu1skG++rjrrWuAV9usOVlJU07oj37kns1RRMMCDedPhxoVgO5ub9O/9Zwv1+piXQEgMViGG2JdITFXcBpwC9F5CjgV6r6WuSgiJwGnItj5PaFOFuIe4C1cbmoFgAX4xRSuhj4exrjbDf4dX9NNkFFT6jjhvXjH6s/8T2pxpOpLJH7PnqLHS/e73osuN9geh1/iasbbDS9SoMx8Q+Dpy30dW+3WJBMur/6VeVZLIbR1kgnN1RjWCD8AfgecLqIbAM+APbDMUwLcEca9x+LUxfjDRGJRGxdhyMkHhaR7wGbaIENpK3jJ7NsqvgFt4Cypeu2tlhYZIoug4+keOCX2ffB601tgZ79KfvaBb6zwe6IS9bnZ5JOFmCXqZxJXnaQrp0DBANF5g1ltFnSyg2lqvXAZSLyJ+AK4CScYL1GYD1wq6rem8b1XqQ5Q0Q8X09nbLkm2/UkUmWW9WNIHdQnUc2RDwN2PCJCrxMuYfPffu5kgz32HLof4Z4N1ouQOvUkUhUkipCrlbwF6RntlRYlElTVV4mtlNeoqu4WynZILupJpMos68dO8dKGz2NW39mstRxd1wIcN9jqF++n98n/TafufRP6Fw84jL6nT6VkyBiKiktbdM9od1SvgkSRsZ01KnfZVi2zq9EeSa/6CyAiw0TkYhG5VETGqeq+jiQoIPmqP1N4GT8j7X4MqQpNY4oIuGwRERT11ZvZ+uRsPvnrldS++yo7XnzA85yuhx/fYkHhxoSKcko7J65/FPjH6k8YO3MJg6ctZOzMJVkVnIbRHkkngrsIxxgdcY0VHJtnp/Bx0XRyh7RhclFPIlVVM7+G1Krq2iaVWTZLmXq5we5+42l6jPk2wb4HJTm7ZZSVJKqtkhUkithqLCjOMNInnZ3FNTieSf8Bfojj7hptbzhWRDaKyLgMjq8gSbXqbwnzK6tiVr4At545gvKykpg039E6+vg05V5c+/gbWbNVhOpqqV72IFV//D67VjyZGC+hIfas9Qi2S4OiOMtWsEhcXVv9/g4yvRM0jPZOOjaLS3GM2MeHPaMOwElhDoCqLhORBhzPpaWZHWZhkelaxl42kFvPHJHUeweaDaklwSJqPALRWrujKAkGYgTV/Moqpj1Sydbl/6T6pQcJ7al2PS/YbxC9TriULoOPbNX9ITYSu1dpkJtOH+66K/ATlR3BguIMwz/pCIuDgd+rarL/whXAsa0bUuHj1+PFr8dUS2tqxxtS51dWeVZuaw3RgiIUClH3zjJ23ncNn3+40bV/oMd+lB13oW83WD9E6zf3JonOjv/dJEt9YkFxhuGfdITFTqA4RZ8qoN2rocBfPQm/HlNeKiK39mQCaEJFeavrZMdTXlbSdP1nnnmGadOm4ZUKvqikBz2/cg7dK9Jzg02XVII0+neTLFjPguIMwz/pCIvXgP8SkSJV9VrahYCerR9W2yed3YJXzYhAXASzHwGUjhrGD1XVtQyathBp2MeWP32fmp2JZUWkUzHdx0yg59FnUlTcNSP3TYVfFZKXI0B0bijDMFKTjo7gHuBQ4JYkfb5Mksy0HYl0PKa81CTx7X5cdidUlHPWqPQmQQHGDulN187eBnPtVEzxmLNjG4uK6H7EKRzwgz/R67gLMyIovBN8xBKtQop3Doh2i/WqVx7JDWUYhj98CwtVfQynet21IvIYcFj0cRH5Nk5W2hddTu9wpOMx5VUzIb7drwBaum6rnyE2ocBbn+yiJsVupPsRpxLo6WSLLx06lgO+97/0Hn+577ThXghwwTED2TjzNF/9o50JIrutqupalObdVkRgTKgoT+pVZhiGP9KN4D4fZ+fwo0iDiCwF+gKHA/XArIyNrg2TjseU375eKpV4AdQSL5/tNfVN2WCDvQ6g6+HHJ/SRTkH6jP8xRcWlSbPBpovSLOBSxY/EJwL0o+6ziGrDaD3pBOV9EVinqj8WkXtpzg0VmVVeB6aq6orMD7Nt0iVY1DSRlZUEmf6t4THup9GG6rNGlbN03daknlNTxw9l6qOrYyrRBQPiW6h4oY0N7F69qMkNNtC9LyWHHkNRMNGfoWRwhe/rgrNjWLpua8rxRAScm+AU4PxjBjJjwgjP8/y2G4bRMtLZWawB7gMuUtVXcCraRXJDiaruzcL42iRuSf72NYQ8j1dV1/LYiipuPdOZDGcvWs/P5q1i9qL1jBvWr0mIlJUGaYwvWepi7nATKhGCRQLh2tSqIWrWLaP6hb/RsP2Tpj6Nuz5j18qF9Dz6zITz06FXaZDRB/dm4eufpOzbMxyNnW4iPr+7rXyQ7WSThpFL0hEW23GpVaGq+zI3nPZBKtWI1/Gbn1zD3vpQjBC57+UPmvrEJ8gDqA+puxupR+KV+pBSVhJk9/uVfLjoHuo2v+Pab/fqf9NjzBlIkb8ocTe219T79szaubeeilueoromvRTemQ6QzBS5SDZpGLkkHWHxAjAsWwNpT6RSjXgddxMGLbnf7EXrqQ+5S4u6Tzew/tm/sndjpevxaDfYVIIikmk2meuvm6AoEifFeDQhbf7+6UyshZoSvKWBloZRqKQjLH4FvCgiY6Ir5BmJpFKNpGtT8HO/aNyEUX31ZqpfuJeatzzyNEkR3UaOp+fY85J6NwVECKnGTMrXz38jZgcUwcsl2EOOxZDOxFqIBmyzpRjtjXSExURgCfC0iPxEVedmaUxtCje9dCrViNfx4k5FLapiF69y6VkSbLqOVzbYaEqHjqXsuIsI9nYm3GCR0LlTEXvqYlfGwSJh9sSRCROzl6uu147DL215Yi1kW4phtIR0hMVVOFoHAf4iIjOBhTiG7uXAGx2troWXXvqsUeUUd2r2hIpPfDehopzlmz7nwVc+pFGVgAhnjSpn9MG9046+Lg0WNRnEI3W2d+5tFjjbl/yZPWvc8zoWH/Qlep1waYIb7OyJIwESjeQeEXNek3qjKiXBQIujydvyxFqothTDaCnpRHCfCEwFHsDJPtsP+C5wN46w2CUir4rIHzI+ygLFSy99/8sfxOwQ4hPfza+s4rEVVU2r7kZVHlvhBJGdNao8Ic1HMuobNSYg7b6XP4hR8/Q89lyIS+YX7DeI/c6eTv/zbk0QFJFcULMXrU/wpqpvVNe03l6TeiQALhmC41YcDMR+57Y+sVowoNHe8L2zUNVngWcjn0WkFBiJU4M78hoJjAL+O5ODLFS8VtTxipd4/buXkJm+YA37GkIxqpv4cqXRCHgasiMEe5fTbeR4dq/6l5MN9msXONlgXYzX0RN0Ojr3ZKvoZMkNy8tKmlKwt0c300K0pRhGS2lRDW4AVa3BKYT0n0ibiHQGvpSBcbUJ0jFUR0+yyaq5xaM4aqxol1ogRr1Tu2k1ezetptdxFyWcD9Bz7HkEex+YNBtsfGR0Ojr3VB5JflQyNrEaRmGTUliIyHTgB0AfYBMwF5ilqgkzm6rWASszPMacke7q1iva2G2tHz3JpusNVV1Tz23nHNE0tp4lQURgx0dvsz3KDbZ0yBiKy7+YcH6nbr3pMeYM12uPHdKb+y/7StPnyDPwGt+gPiWMnbkk4Rklm+wL1b3VMAz/SLKy2SLyXeDPcc0K/ENV3WefPDF69Gj1qrPgB7eo6/gKcV7nRU+C44b147EVVUmv43WvLsEi11iLgAj/M8nxQppfWcWUe55i89K5CW6wxQcOp/93ZiJp2DziVUGpDOzxwjBZKg7DMAofEVmhqqNT9Uu1s/ghUIdTUvU5nKC8WcA3RWSiqj7S6pEWCJmqVgcw+uDeSVfRXittwHWyblTl2sffYMf2bVx7w3Q+edml1jWw76M11G1+h+IBzQmBS4JF1CapLBetEnN7BvHELy0UuP/lDxh9cG/bKRhGOyaVsBgCPKqqD4Y/fywiJwHvAhcB7UZYZDKIyo/+PVmfKQ+vjjFyh+pq+WTZfL77m8cJ1bmPp8kNNkpQBIuEs0YdmLDTiSZaPdbSuAaFNheZ3B4N6oaRTVIJi144gqEJVa0WkYU4tSvaDV52hLLSoKuOPltMqCjnZ+E62vHZYN0I9htEr+MvocshoxLUT/UhZem6rdx65gimL1iTYECPNzKnsqUk88wq5AC6VKpCy9tkGKnxE2fhpsP4AMfg3W5wq6gWDAi79zZ4FtbJFgN6FLNn7Qt8fM+P+Hzx/7oKikCP/ehz2s8ZcMkdlAwZ7Wmn+Li6lgkV5ay66WRuP+eIpH7/bs8gctXyshLOP2agZyW7Qg2gcyuOdP/LH6SsOGgYRiwtdZ1tANx9MNNARP4CfBPYoqpfCrf1xqnINwjYCExS1cTCzxnGzY6wZ19Dwmo8F8ngTij9kP8scK8h1adPH44847u83etYiHKD9eOFlUo95tdr6f6XP4i5VyEH0LnZYdri7sgw8k0qb6gQ0AisA17FSe3xKnAWcJ2qtjx/tXP944DdwN+ihMVvgM9VdaaITAN6qeo1qa7VWm8oNwZPW+g6sQjwfpISoH714V79QqEQQ744go1vv9XUt3OXLkydMoWpU6fSs2fPFnlhZYq2pO/3+h26Ee0ZZhgdhUx5Qz2NE5k9PPy6JO4mvwVWhV9vqaq3240Lqvq8iAyKaz4DOCH881ycqPGUwiIbJAtM85ow/dYxSNXvD7f/llNPPZVAIMBll13GjTfeyIABA5rOb4kXVqbwY8AvFIHi9TuM34kV8u7IMAqBpDuLpk4ig4HRUa8jgZ7hw5EL7APeBCpV9Qe+B+AIi39E7SyqVbUs/LMA2yOfk5GNnYVXPMRZo8o9V/F+UlsAHHX946xffD89vzKJoi7dEvqpKrfccgvnnnsuQ4e2rUmspTEruRyLnzK2htERyNTOAgBVfR94nyhXWRE5lFgBUhF+H4UT8d1qVFVFxFOaichkYDLAwIEDM3HLGLx0+MliMlK54O7evZvf/e53LJ89E62rhaIAvY6/OKGfiHDTTTdl/DtFk63VfyEV/rHoccPIDK3JDfUO8A7wIDTtAobhCIzW8KmIDFDVT0RkALAlyRjmAHPA2Vm08r6uuKlcIq6t8UQmI7edRaixgUPOuJLtyx6keltz/YddyxfQ/cjT6NS9L5BZr6JkwiCbZT8LrfCP5Z0yjNaTTorypKjDWlW9t5WXWgBEltoXA39v5fUyjteEHpmQo91PVUPsWfs8H9/zI95fcGeMoADQhn3sWr4A8K83n19ZxdiZSxg8bSFjZy5xdeV1cxmNdvtNlvm2tSR7PoZhtE0yJixagog8iJO1dqiIfCQi3wNmAieJyDs4gX8zs3V/P5OuG+OG9UuIN4hOyR2pY1C7cRWb//ZzPlvwGxq2f5Jwne5lvRh02o8o+9oFvusdpBICEZKpgiB55tvWxpG4xWuYAdkw2jYtVkNlAlU9z+PQ17N975aqYSKFi+J1XkcO7Nl03sFsodvSWWx56inXa0inYq675qomN9h0SLYjiB53KlVQsmjt1toWzE5gGO2PvAqLfNJSI6xXsr2XNnzO3U++xAsP/Z4HHnjA/WQpotvIkxl2yqXMmDGpReNOtSPwW49i6vih/DSJ7aW1mJ3AMNoXeVVD5ZOWGmE9603X7+XHk8Z7CorSoWM54Ht/4MBvXskvJo5Nb7BRJNP7R6erSKUKmlBRTq9S9yB8sy0YhhFPhxUWLTXCeh0vCnahW0ViVHfxQV9i/wv/h34TrqVznwM5a1TrVtzJ9P7RgsxPDeibTh9utgXDMHzRYdVQfkp9ep33s3mrXFNIDD35O2x4+2m2bdtG6f6D6fbVi2OywSqwdN1WlzObSRX7MKGinJufXONaJClekGUqF5RhGEaHFRYTKspZvulzHnzlQxpVCYikXPWHQiHq3lnG2NKdLKsZkJAu4tozjmDnIb9DRLjhjZ5QlJg6K5may6/R/abTh7dI0LlhtgXDMPzQYdVQEa+mSJGhRlUeW1Hl6Ta6ZMkSjj76aM455xxWP3IHvz3bXcVz0UUXceGFF1Leu5vrdbzUWPMrq5jy8GpfqbP9qJgMwzAySYfdWfj1hlq1ahXTpk1j0aJFTW1vvvkmu9csZdm0i/HCTc0FUFPXEOO1BM07ikaPPF1uu5GW7AgKJbmfYRhtjw67s0jlDfX+++9zwQUXUFFRESMoItx88800NnrXq46s/stKYj2OttfUJwTRpap9XebhtZQOfoP5DMMw3OiwwsJLHdSv0z6uvPJKhg4dyv333+/a5+yzz+Zf//oXT76+OWkE+ISKcroWJ27e4lVLqdx1fSQGTkmqiG7DMIxkdFhhER+HEKqrZc/L83jz9ou58847qa9P9DY6/vjjefnll3nkkUdYW9PN10rdTzxHKnfdHbWJY0mXQkvuZxhG26LDCouImmhA9yC7Kv/J5j9N5rPn7qV2z+6EviNGjOCf//wnS5cu5eijjwb8r9T9xHO4BdD5uUY6WHI/wzBaQ4cVFuAIjMPevp/Pn/oD9bsTy3wffPDB/O1vf6OyspJTTjmlKV4CUq/UI0kKq6prPZMORo/j1jNHuEZUZypIzpL7GYbRGjq0sAC4/PLLE9r69OnDbbfdxvr167nwwgsJBBJX/clW6tHGZHCC8SICw8vNdUJFOZU3nszt5xyRFZdYc7c1DKM1+Cqr2hZoTVnVs88+m8cee4ySkhJ+/vOf+8oGm6x0qN/SqoZhGPkmo2VV2zu/+tWv6Nu3LzfddBMDBgzwdU6yVBnJKukVEhZ3YRiGX0xYAEOHDuXuu+9O+zy3wLj5lVUUibgG2BWSMTmbZVUNw2h/dHibRSZJFoktOBNyOhX5sonFXRiGkQ4mLDJIskjsiPgolMhpi7swDCMdTFhkEL8TbSGs4C3uwjCMdDBhkUHSmWjzvYK3uAvDMNLBhEUGcZuA4wPyIuR7BW9xF4ZhpIN5Q2UQN3faccP68diKqowUKso0VvjIMAy/mLDIMG4T8OiDe3e4eAaL4TCM9oUJixzQ0VbwFsNhGO0Ps1kYGcdiOAyj/WE7iw5ErlRDFsNhGO0P21l0EHJZVtViOAyj/WHCooOQS9WQxXAYRvujYIWFiHxDRNaLyLsiMi3f42nr5FI1ZDEchtH+KEibhYgEgN8DJwEfAa+JyAJVfSu/I2u7HFBW4lpjI1uqoY7mAWYY7Z1C3VkcBbyrqu+pah3wEHBGnsfUpjHVkGEYraFQhUU58GHU54/CbTGIyGQRWS4iy7du3ZqzwbVFTDVkGEZrKEg1lF9UdQ4wB5yyqnkeTsFjqiHDMFpKoe4sqoCDoj4fGG4zDMMw8kChCovXgENFZLCIdAbOBRbkeUyGYRgdloJUQ6lqg4j8GFgEBIC/qOqaPA/LMAyjw1KQwgJAVf8J/DPf4zAMwzAKVw1lGIZhFBAmLAzDMIyUmLAwDMMwUmLCwjAMw0iJCQvDMAwjJSYsDMMwjJSYsDAMwzBSUrBxFoVMrsqTGoZhFAomLNIkUp40UnUuUp4UMIFhGEa7xdRQaZLL8qSGYRiFggmLNMlleVLDMIxCwYRFmniVIc1WeVLDMIxCwIRFmlh5UsMwOiJm4E6TiBHbvKEMw+hImLBoAVae1DCMjoapoQzDMIyUmLAwDMMwUmLCwjAMw0iJCQvDMAwjJSYsDMMwjJSIquZ7DBlBRLYCm/I9jlbSF/gs34MoIOx5NGPPIhZ7Hs209lkcrKr9UnVqN8KiPSAiy1V1dL7HUSjY82jGnkUs9jyaydWzMDWUYRiGkRITFoZhGEZKTFgUFnPyPYACw55HM/YsYrHn0UxOnoXZLAzDMIyU2M7CMAzDSIkJizwhIn8RkS0i8mZUW28RWSwi74Tfe+VzjLlCRA4SkaUi8paIrBGRK8PtHfV5dBGRV0Vkdfh53BxuHywir4jIuyIyT0Q653usuUJEAiJSKSL/CH/uyM9io4i8ISKrRGR5uC3r/ysmLPLHX4FvxLVNA55R1UOBZ8KfOwINwBRVPRw4BrhcRA6n4z6PfcCJqjoSOAL4hogcA8wCblPVLwDbge/lcYy55kpgbdTnjvwsAMap6hFRLrNZ/18xYZEnVPV54PO45jOAueGf5wITcjqoPKGqn6jqyvDPu3AmhXI67vNQVd0d/hgMvxQ4EXg03N5hnoeIHAicBvw5/FnooM8iCVn/XzFhUVj0V9VPwj9vBvrnczD5QEQGARXAK3Tg5xFWu6wCtgCLgQ1Atao2hLt8hCNQOwK3A1cDofDnPnTcZwHOwuEpEVkhIpPDbVn/X7HiRwWKqqqIdChXNRHpBjwG/FRVdzoLSIeO9jxUtRE4QkTKgCeAYXkeUl4QkW8CW1R1hYickO/xFAhfVdUqEdkPWCwi66IPZut/xXYWhcWnIjIAIPy+Jc/jyRkiEsQRFPer6uPh5g77PCKoajWwFPgKUCYikQXegUBV3gaWO8YC3xKRjcBDOOqnO+iYzwIAVa0Kv2/BWUgcRQ7+V0xYFBYLgIvDP18M/D2PY8kZYR30PcBaVf1d1KGO+jz6hXcUiEgJcBKOHWcpcHa4W4d4Hqp6raoeqKqDgHOBJap6Ph3wWQCISFcR6R75GTgZeJMc/K9YUF6eEJEHgRNwMkZ+CtwEzAceBgbiZNCdpKrxRvB2h4h8FXgBeINmvfR1OHaLjvg8voxjpAzgLOgeVtVbROQQnNV1b6ASuEBV9+VvpLklrIa6SlW/2VGfRfh7PxH+2Al4QFV/JSJ9yPL/igkLwzAMIyWmhjIMwzBSYsLCMAzDSIkJC8MwDCMlJiwMwzCMlJiwMAzDMFJiwsIwDFfC2YBVRB5P3dto75iwMNoMIvJUePJSEflJkn73RPX7Sy7H2M4YFX5fmddRGAWBCQujLTEKJ505wJfdOoRTeV8KNIabludgXO2VI8PvK/I6CqMgMGFhtAnCkau9gVdxIt4ThIWIFAG/B7bSLCRMWLQc21kYTZiwMNoKkSIvK3DSOwwPC4dofoCzGr4aGALUA6vjLyQiZ4nIv0TkMxGpC1cXu05EAi59vyMi94vI2yKyS0S2i8hrInKp2yBF5Gsi8riIbBCRveFqiK+KyK1x/a4Nq8nOdLnGwPCxJ6LaxoXbfisiR4nI30Xk83Db8FZ8v04icqWIvB4e7yYRuSacr+tI4GNV/dTtuxodCxMWRlshWlisBEpxBAIAItIX+BXwEvAcTs6tN6PzBYVrRDyIUzTnC8AjwB9w8lH9Coixb4QTtv0NOAQnd9VdwOPAYOAvInJNXP/rgOdxVuTPAL8DngSKgfFx3yeyanfb+YwJv0ev6CMqoS8BL4bH/EfgAWBdC79fZ+CfOPUiGoH/Fx73dGAOsD+mgjIiqIVqhpcAAASpSURBVKq97FXwL2AJTtGX4TjZRhU4K+r4n3EmvArgrPDxOXHXuCvcfivQKao9CCwLHzs8qr0bsL/LWAYAu4B1UW39cewpLwCdXc7pG/f5fWCrx3e9NTyWb0a13R9u2wUc43Feut/vT+G2GwjniQu3HxduV2B6vn/39iqMl+0sjIInSiVSg7OKjqy4vxw+fjTwXeCPqlqJy6o93Oe/gb+rk/Y6YihHVetpLkl5dFT7blXdHD8edSqSfYxjQ4kwDCdL7NuqWudyzmdRY+kNDMJ71R7ZRbntLK5U1ZfjT0j3+4nIUcD3gX+o6i9VVaP6P09zvWvbWRiAVcoz2gaHAj2Bl1S1UUTeB3YAX44yam8DfhHuH5lso1U8VwAC1IjIdJd7fCn83rSAEpFewI9x6j8PBXoQq7qtjPp5TXhM3xWRfjg7gadUdbvLvSITv5fxfRROdbiPw+PoChyGU9Dmrx7npPv9rgi//8rjetvC72bcNgATFkbbIGbyV1UVkUpgBDAZZ3K9LGpiHgXsw6mPEeHk8Pt5Ke61CZpqSjyFo156Fad2wjYcVdNg4CKijOeq+lm4LsdNwKnA6UCjiCwGfqGq0ZNuZOeTsGoPe331Av4d1TwSZ5JfqKqh+HNa8v3C/bfh1Axx4xAcgdVhKtAZyTFhYbQFoo3bESqB44FfA6/hVNpDRAbjqIdeC6tfEJEuQD/geVU93uc97wXKgHGq+mz0ARG5JfxjzM5AVd8EJoYNx8fhCLKJwBgRKddmY3tF+N1t1X5q1PeLENmJuE7s6X6/cP/9gMpo9VPU8SOBA4B/pbqW0XEwm4XRFnATFitx1C49gcujJj03LyMJv/f1czMROQjHHvKsi6Aowwn6ix9PE6pap6pPq+okHM+lPjg7lAjDgHpV3RR9nogU47j/Rr5fhFRqq7S+H44jQCOOwHAjos4zFZTRhAkLo6AJ2yQqgD00G13Bcfn8NvB1VX0tqj3BXqGqtcDrwOFucQ3h+3w1Kg5hb/j9EBEJRvXpA8wDDsRRR60Kt1eIyBDiEJEv4NgKPgA+ijpUBwRF5NCovl1xXFsjtoX4nUUdsWq1JtL9fuEd1ztAuYicHtfvGiByDTNuG02YGsoodIbhuLAui9bXq1NfeL5Lf6/4hanAQuAxEXkaZ3ItAsrD5wRVdWD42ltFZAlwIvBKuP8A4BRgKU7cwluqGhEqVwCXiMjL4fvuxtH5fyt8/LtxtoZFOLEUz4cD77oBXw+P6ROcGJL3oGm3cTjwupuXVUu+X5hbcTykHhORh4DNODXhRwAfAgdhOwsjmnz77trLXsleOIZkBe7w2X8bjottwOXYGJyAtc040d2f4azW/4izQ4nu2w8nIG8rTmzDS+GxVITHc09U3zNwbBzrgZ04u4CNOLEfh7qMowtwB477bQ2OgPkBjo0khKP+ivQdFb7fH318d9/fL9z/ShyhVI+TQuUxnF3MJuCzfP/u7VVYL1FNsG8ZhmEYRgxmszAMwzBSYsLCMAzDSIkJC8MwDCMlJiwMwzCMlJiwMAzDMFJiwsIwDMNIiQkLwzAMIyUmLAzDMIyUmLAwDMMwUmLCwjAMw0jJ/wdaCN/mbA4l1gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.scatter(y, predicted)\n",
    "ax.plot([y.min(), y.max()], [y.min(), y.max()], 'k--', lw=4)\n",
    "ax.set_xlabel('$Measured$', fontsize = 20)\n",
    "ax.set_ylabel('$Predicted$', fontsize = 20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 训练集和测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:36.365683Z",
     "start_time": "2019-04-22T08:22:36.360788Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[6.3200e-03, 1.8000e+01, 2.3100e+00, ..., 1.5300e+01, 3.9690e+02,\n",
       "        4.9800e+00],\n",
       "       [2.7310e-02, 0.0000e+00, 7.0700e+00, ..., 1.7800e+01, 3.9690e+02,\n",
       "        9.1400e+00],\n",
       "       [2.7290e-02, 0.0000e+00, 7.0700e+00, ..., 1.7800e+01, 3.9283e+02,\n",
       "        4.0300e+00],\n",
       "       ...,\n",
       "       [6.0760e-02, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9690e+02,\n",
       "        5.6400e+00],\n",
       "       [1.0959e-01, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9345e+02,\n",
       "        6.4800e+00],\n",
       "       [4.7410e-02, 0.0000e+00, 1.1930e+01, ..., 2.1000e+01, 3.9690e+02,\n",
       "        7.8800e+00]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boston.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:48.265456Z",
     "start_time": "2019-04-22T08:22:48.261247Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "Xs_train, Xs_test, y_train, y_test = train_test_split(boston.data,\n",
    "                boston.target, \n",
    "                test_size=0.2, \n",
    "                random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:51.873960Z",
     "start_time": "2019-04-22T08:22:51.869286Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "lm = regr.fit(Xs_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:52.561738Z",
     "start_time": "2019-04-22T08:22:52.555669Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(30.24675099392396,\n",
       " array([-1.13055924e-01,  3.01104641e-02,  4.03807204e-02,  2.78443820e+00,\n",
       "        -1.72026334e+01,  4.43883520e+00, -6.29636221e-03, -1.44786537e+00,\n",
       "         2.62429736e-01, -1.06467863e-02, -9.15456240e-01,  1.23513347e-02,\n",
       "        -5.08571424e-01]),\n",
       " 0.7508856358979673)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lm.intercept_, lm.coef_, lm.score(Xs_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:53.518402Z",
     "start_time": "2019-04-22T08:22:53.515220Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "predicted = regr.predict(Xs_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:22:54.585839Z",
     "start_time": "2019-04-22T08:22:54.380438Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAETCAYAAADH1SqlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8VOXVwPHfyQIEEAKCCkEFFKEoyhJFRXCtiKIiWHDBXdRXsVoU2ZR9s/iW2lqt+EqliEgVxK0uKC5UK0KIgoC4oAhBJAgRkABZzvvHvRMmyZ3MTDJbkvP9fPKZmXufufeZUe6Z+yznEVXFGGOMqUhSvCtgjDEm8VmwMMYYE5QFC2OMMUFZsDDGGBOUBQtjjDFBWbAwxhgTlAULY4wxQVmwMMYYE5QFC2OMMUGlxLsCkdKsWTNt3bp1vKthjDHVSlZW1g5VbR6sXI0JFq1bt2blypXxroYxxlQrIrIplHLWDGWMMSYoCxbGGGOCsmBhjDEmKAsWxhhjgrJgYYwxJqi4j4YSke+BPUARUKiqmSLSFFgAtAa+Bwaq6q541dEYY2q7RLmzOFdVO6tqpvt6JPCuqrYD3nVfG2OMcf3444+89957MTtfogSLsi4H5rjP5wD94lgXY4xJGD///DMjRozguOOOY9CgQezduzcm502EYKHA2yKSJSK3uduOVNUf3efbgCO93igit4nIShFZmZubG4u6GmNM3Kgq559/Pn/84x/Jz88nNzeXRx99NCbnToRgcZaqdgX6AHeJSC//naqqOAGlHFWdpaqZqprZvHnQ2erGGFOtiQj33ntvqW0zZsxg167od+nGPVioao77uB14CTgN+ElEWgC4j9vjV0NjjEkcgwcPpn379gAcddRRTJ06lQYNGkT9vHEdDSUiDYAkVd3jPr8QmAi8AtwATHcfX45fLY0xJnaKiop47rnnKCws5Kabbiq3PyUlhRkzZvDll19y1113Ub9+/ZjUS5xWnvgQkbY4dxPgBK7nVHWKiBwO/As4BtiEM3R2Z0XHyszMVEskaIyprlSVRYsWMXbsWNatW0fTpk357rvvaNSoUVTPKyJZfiNRA4prM5SqblTVU9y/E1V1irv9Z1U9X1XbqeoFwQKFMcZUV6rKm2++yamnnsqVV17JunXrANi5cyczZ86Mc+0OiXufhTHG1FbLli3j7LPPpk+fPmRlZZXb/9e//pX9+/fHoWblWbAwxpgYy8rKok+fPvTq1Ytly5aV2y8iXH/99Xz66afUq1cvDjUsL+7pPowxprZYt24dY8eOZeHChQHLDBgwgIkTJ9KxY8cY1iw4CxbGGBNlGzduZPz48cybN4/i4mLPMhdddBGTJ0+mW7duMa5daCxYGGNMlD3yyCPMnTvXc1/Pnj2ZMmUKPXv2jHGtwmN9FsYYE2Vjxowp1/fQrVs33nzzTT744IOEDxRgwcIYY6IuIyODO++8E4Df/OY3LFy4kBUrVtC7d29EJM61C401QxljTBXt27ePv/3tb3z88ccsWrTIMwCMHDmSzp07c80115CcnByHWlaNBQtjjKmkgwcP8tRTTzF58mS2bdsGwNtvv03v3r3LlW3evDnXXXddrKsYMdYMZYwxYSosLOSZZ56hffv2DB06tCRQgNM/Ec80StFiwcIYY0JUXFzMCy+8wEknncRNN93E999/X67M119/zVdffRX7ykWZBQtjjAlCVfn3v/9Nt27dGDhwIBs2bChXJi0tjREjRrBx48aSFOI1ifVZGGNMBT744ANGjx7Nxx9/7Lk/NTWV22+/ndGjR9OiRYsY1y52LFgYY4yHzz//nOHDh7NkyRLP/UlJSdxwww2MHTuW1q1bx7ZycWDBwhhjPGzdujVgoBg4cCATJkygQ4cOMa5V/FifhTHGeLjooovo0aNHqW19+/YlOzubBQsW1KpAARYsjDG13K+//uq5XUSYMmUKAOeccw4ff/wxr776Kp07d45l9RKGNUMZY2ql7du3M23aNJ555hnWrl1Ly5Yty5U5++yzWbFiBd26dUu4tByLs3OY8dYGtubl0zI9jeG929OvS0bUzmd3FsaYWiUvL48HH3yQtm3b8uc//5m8vLySOwgvmZmZCRkoRi1aQ05ePgrk5OUzatEaFmfnRO2cFiyMMbXCr7/+yrRp02jTpg1Tpkwp1fw0a9YsvvvuuzjWLjwz3tpAfkFRqW35BUXMeKv8/I9IsWYoY0yNtn//fp588kmmTp3K9u3bPcu0aNGCH374gTZt2sS4dpWzNS8/rO2RYMHCGFMj+fI3TZw4kc2bN3uWOeKIIxg9ejS33357wqx1HUpfRMv0NHI8AkPL9LSo1cuaoYwxNUpxcTHz58+nY8eODBkyxDNQpKenM3XqVL799lvuueeehAoUofRFDO/dnrTU0mnO01KTGd47emlG7M7CGFNjrF+/nquuuorVq1d77m/QoAH33nsv999/P+np6TGuXXAV9UX43134nsdyNJQFC2NMjdGyZUu2bNlSbnudOnW48847GTVqFEcccUQcahaacPoi+nXJiGpwKMuaoYwxNUbjxo0ZMWJEyevk5GSGDBnCN998w8yZMxM6UEDgPgev7Yuzc+gxfSltRr5Oj+lLozpsFhIkWIhIsohki8hr7us2IrJcRL4RkQUiUifedTTGJI4vvviCoqIiz31Dhw6lZcuWXHPNNaxfv55Zs2Zx9NFHx7iGlRNqX0RtnmdxD7De7/XDwExVPR7YBdwSl1oZYxLKhg0buOqqq+jUqRPz5s3zLFO/fn2+/PJL5s2bR7t27WJcw6rp1yWDaf07kZGehgAZ6WlM69+pXHNTPOZZSLyX/xORVsAcYAowDLgUyAWOUtVCETkDGK+q5Re19ZOZmakrV66Men2NMbG3adMmJkyYwJw5cyguLgagTZs2fPnll9SpE5mGh1inz6iKNiNfx+vKLcB30y8J61gikqWqmcHKJcKdxZ+BB4Bi9/XhQJ6qFrqvtwCe/8VE5DYRWSkiK3Nzc6NfU2NMTG3bto27776bdu3a8Y9//KMkUAB89913PP300xE5TzyadaoinL6NSIlrsBCRvsB2Vc2qzPtVdZaqZqpqZvPmzSNcO2NMvOzcuZORI0fStm1bHnvsMQoKCsqV6dy5MyeccEJEzhePZp2qqI3zLHoAl4nIxUA9oBHwKJAuIinu3UUrIDHDuzEmovbs2cOf//xnHnnkEXbv3u1Zpn379kyaNIkBAwaQlBSZ37vxSJ9RFbVunoWqjgJGAYjIOcD9qnqtiLwAXAk8D9wAvBy3Shpjoi4/P58nnniCadOmsWPHDs8yxx57LOPHj2fw4MGkpET20hWP9BlVFet5FvG+swhkBPC8iEwGsoHINEwaYxJOXl4eJ510Ejk53g0IRx11FA8++CC33nordevWjUodhvduz6hFa0o1RUW7Wae6SZhgoarvA++7zzcCp8WzPsaY2EhPT6d79+4sWrSo1PamTZsyYsQIhg4dSv369aNah3g061Q3cR86Gyk2dNaY6mvt2rV06tQJVaVhw4YMGzaMYcOG0bhx43hXrcYLdehswtxZGGNqLlXl7bffpnXr1rRvX75p58QTT2TIkCE0atSIESNG0KxZszjU0lTEgoUxJqqWLVvGmDFjWLZsGQMGDODFF1/0LPfkk0/GuGYmHIkwKc8YUwNlZWXRp08fevXqxbJlywBYuHAhWVmVmlZl4syChTEmotatW8eVV15JZmYmb775Zrn9Dz74YBxqZarKmqGMqcFime9o48aNTJgwgWeffbZUWg5/ffr0YfLkyVE5v4kuCxbG1FC+fEe+uQO+fEdARAPG1q1bmTx5Mk899RSFhYWeZXr27MnUqVM566yzInZeE1vWDGVMDRXtfEc7duzg/vvv57jjjuOJJ57wDBTdunXjzTff5IMPPrBAUc3ZnYUxNVQ08x2pKueff37Ata47duzIpEmTuOKKKxCRKp/PxJ/dWRhTQ0UzjbWIMGzYsHLb27Zty9y5c1m9ejX9+/e3QFGDWLAwpoaKdhrrwYMH06FDBwAyMjL4+9//zpdffsngwYNJTk4O8m5T3VgzlDE1VFXzHRUWFjJ37lxUlZtvvrlku/8Iq3qnDebGiw/y+OSRpKUlboZWU3WWG8oYU0pxcTEvvvgiY8eOZcOGDTRt2pTvvvuORo0alRthBc7ditc60aZ6qPKyqiLyooiMEpHeImLL0BlTw6kqr7/+Ot26dWPQoEFs2OCMmtq5cyd/+tOfgOq3opyJnIr6LPoDU4B/A9tEZLOILBaRsSJyiYi0iE0VjTHR9v7779OjRw/69u3LZ599Vm7/Y489xv79+6vdinImcirqszgG6AZ09Xu8DLjUV0BEtgNZwCrfo6pujlptjTER9emnnzJmzBjeeecdz/1JSUnccMMNjB07lnr16lXLFeVMZAQMFqq6BdiC35KmIjILuBVYD/wMtAUuBvr4ldmhqkdGq8LGmKpbs2YNDz30EC+/HHjF4kGDBjFhwoRSKcVtRbnaK+TRUCLyAHAt0EtV/+O3vSswBrgC2AzUjB5zY6Ioljmb/H3zzTeMGzeO+fPnE2hwS9++fZk0aRKdO3cut89WlKu9Qh4NJSLfA++o6q0B9o8AxgI9VLV8o2eU2WgoU13Ec0TRXXfdxeOPP+6575xzzmHq1KmcccYZUa2DSSxVHg3l4UicpidPqvow8AUwOoxjGlOrLM7O4b5/fR63EUVjxoyhXr16pbaddtppLFmyhKVLl1qgMAGFEyy+Ai4IUuZ9oGela2NMDea7oygKcDefk5dPj+lLWZydE7U6tGzZkqFDhwLQqVMnXn75ZT755BMuuOACS81hKhTODO7ZwEwRGa2qUwOUOQpoWvVqGVPzeM1RKKuyacR9fSBbtu+CdW9w5P7NLH/vTc8AMGLECLp27cqgQYNISrKMPyY04QSLx4BLgEkichowRVVX+HaKyCXAVTid3MaYMkKdi+Brkgo1WCzOzmHEv7LIXfE6v/z3XxTvy+MHYNzf5jFx6OBy5Zs1a8bVV18d8rGtM9tAGM1QqlqEEyxm48y3+EREtovIShH5AXgFSAUejUpNjanmwpmLEGpgKSws5P7JM/n2b7ey691ZFO/LK9n3p2kTA454CoWv2SwnLx/l0F1PNJvJTOIK6x5UVQtUdQhwOvAcUIwzWa8FsAG4QVX/GvFaGlMDeGWBDdRLECywFBcX8/zzz9OxY0e+XfS/FO3JLVdm389b+eqrrypbXUvtYUqpVNZZVf0UuA5AROoCRarqvZ6iMQbwnqNwbofmLMzKCXmSm6ry2muv8fv7RvD91+s9y0hqXQ7LvJwOv72m1IS6cAW6u/F1xFvTVO0SdrAQkQ5Ad5y7ku9V9b3KnlxE6gEfAnXduryoquNEpA3wPHA4ThqR61T1YGXPY0yi6Nclo9yFNfPYpkx4dS279hUAUDfF+4Z/6dKljB49muXLl3sfPDmFwzpfTOMzfkfD9GaM6tepSnUNlNpDoGR7tNb1NoknnBncScDTwPW+TTjNUCnuftHwG0gPAOep6l4RSQX+IyJvAMOAmar6vIj8HbgFeCLMYxsTVKJ04O4vKC55npdfUOoCnJ2dzf3338/SpUu93yxJNOx0AU17XE1So+YR+xxeqT2E8ikawu2QN9VTOHcWI4AbgI+BOcCFOJlpfc4UkXnATaHebbjBZa/7MtX9U+A84Bp3+xxgPBYsTISVnUkdr1/JFfUN9OuSwU8//RQgUAj1f9OL9LOuIbVpBgJ8N/2SiNXLq9nM604DLOtsbRBOsLgJpxP7bFUtEpGW+AULVf1IRAqBgUDITVMikozT1HQ88DfgWyDPrw9kC2A/WUzEBbtIx0qwtN+9e/emZ8+eLFu2rGRf2vHdSe85mDpHtCnZFo3Mr2WbzXpMX2pZZ2upcEZDHQu84Q6hDSQLODOcCqhqkap2BloBpwEdQn2viNzmDt1dmZtbfjSIMV4WZ+cEvOiBc5H2lWkz8vXoz6p2L7TFB/M9t4sIU6ZMAeCCCy7g4TmvcOxV40sFilhlfo32ut4mcYUTLHbjdERXJAdnGG3YVDUP547kDCBdRHx3Pa3c43q9Z5aqZqpqZvPmtpifCc5/7kAgjdNSYzq/4NZuTdi9dBZbHr+Rwj07gPIX4J49e7Jq1SqWLFnCA9dfyrT+nchIT0OAjPS0mC1r2q9LRtzObeIrnGaoFcAFIpKkqsUByhQDjUM9oLtca4Gq5olIGvBb4GGcoHElzoioG/BbU8OYqgiWciMtNRkRYtI8tXPnTmbMmMFf/vIX9u3bB8DujxfQadD9nh3UXbp0KXnuNaoqVuJ5bhM/4dxZPA20AyZWUOZkKshM66EF8J6IrMYJRktU9TWczvRhIvINzvDZp8M4pjEBVdQR6/uVnOcOYQ3nveHYs2cPkydPpk2bNkyfPr0kUADkf7GEuQNb28XYJJyQ7yxUdaGILABGichvgP3++0XkCpystC+GcczVQBeP7Rtx+i+MiahAI3oy0tP4aOR5gHP3EY1O3P379/PEE08wdepUduzY4VkmIyODLVu20LZt2yqdy5hICzfl5LU4Q1j74SQNRETeE5E1OEGiAKcZyZiEFEoHbaQ7cQsKCnjyySc5/vjjGTZsmGegOPLII/nrX//Khg0b6NWrV6XOY0w0hTWD2+2rGCoic4G7cfoYznZ3rwaGq2pWZKtoTPgCTbbzNe+Mf2UteflOc1O91NK/mfznF+Tk5ZMsUionUqhNREVFRcyfP59x48axceNGzzJNmjRhxIgRDB06lAYNGlTqsxoTC+HM4L4WeF9Vc1R1ObDc3V4XZ3nW/RUewJgYCWWy3YHCQ2M0du0rKLff91jZSXvr1q1j4MCBrF271nN/w4YNGTZsGMOGDaNx45DHhBgTN+E0Q83FSbtRiqoesEBhEkmwbKmhZlOtStbVVq1asW3btnLb69Wrx3333cfGjRuZMGGCBQpTbUR0mSwRuVJEJkXymMaEK9iM6GD7Qz1ORRo1asTIkSNLXqekpHDHHXfwzTff8Mgjj2Dzgkx1U2GwEJGrROT4MI53IjC6alUypmoCjVrybQ+2P9TjAMxcsIQzpyzxnOl911130apVK6677jo2bNjAE088QUZGeEulxmoWuTHBBOuzeA5QEdmLk+DvtyKyFVgFrFHVsgPSG+BkkjXVUKJkYK0qr2yp/qOZAmVT9a3TcG6H5rz3ZS45efnlsqz6jrN+/Xpu+f1w/vvO6xx+8b007HRBuT6NtLQ01q9fT8OGDcP+DImS5NAYH6koq7iI/AFnJbxuHMrZ5HtDAbAOJ3Bk46QDmQrsVdXfRKvCgWRmZurKlStjfdoao+zFCZwLY3VN5VA28PkCQNnXXgGhLN/+jPQ0rj8pjY9f+DvPPvssxcVOJ3ly4yPJGPJ3JDkVKD1no7IC5a6KxLGN8SciWaqaGaxchXcWqjrT74DFwDPAf3ECSFegE9C5zNuGhltZE3+JkoE1UvyHyXr9Sl+YlcO0/p0CTsDzp0CDgt2c8sPr3D32/ygoKH1DXfTLT+z9/G0O6+qkB4/ETO+q9JcYEw3hzLN4GMhS1ZIZ2m568Y44AeNwd/+yAO83CawmXpx8dxdewSC/oIg//Oszgi3XVbTvF3YvX8gPq15jXaH3Yo11jjqe1MOPLnkdiXTdgWaaWypwEy/hpPsY5bGtCFjj/plqrDpenCrqY/FqViurokBRfGAfu1e8xO4Vi9GD3gGzVZt2aLdBJLftjogAkUvXHazfxZhY9zGGPHRWRC4TkekicnTw0qa6qW7rFDy4eA1/WPBZwDTiwbLLBlJcsJ9fli8k5++38MtH8z0DRZs2bfjnP//J91+v57HRd9CqSf2Ip+u2VOCmIv6p9mORRh+CdHCXKijyJk4/RWtV3eexvzJrcEeMdXBXXXUZDbU4O4c/LPjMs1M6w633vQs+C/u4xfv3svXpOynau9Nzf51Gh/Pow5O5+eabqVOnTtjHNyZSIjkAIiId3GWcDLztFShcx4vI+8B9qvp8GMc1CaK6rFMw460NAUcv+Q8xDVdSvYY0bn0iO78o3e2WlNaIZj0G8ZeJDzDojHCmHRkTHfHoYwwnWDQFNgXaqapfi0gOcCPOokXGRIz/XU9Ft6++pH+VVbf71bD2I9Bikus24LBT+9H+gqsYeVmXahFITe0Qjz7GcIJFLsGXTM0G+la+OsZ4z5FYmJUTNAgIUFRBS2iT+qns/PUg+zeuJCX9qFIjmHzqNDuGhp0vok5aA2ZOHsv1555U1Y9jTMTFYwBEOMHiY+ASEUlT1UD3OnlAs6pXy9QW/sNbk0UoUi01SS4nL595n/xQ4d0EOIHi2tOPKZloV1ZGehp9mu1k6oRx5G9ZR9oJZ3DEFWNIS02mbkpSSbpygMMvvBOAJ5dv5/pzI/IxjYko/zT6sepjDKeD+xxgKfACcK2qFnqUeQ04XVVjHjCsg7v6CWV4azACpf6heB4z9xvS1y3i808+LPXek+96nAm3XBaws9zr+MbUNKF2cIc8dFZV3wf+AvwO+I8bPPxP+DugD84Mb2OCquzwVn/p9VP59UAhf1jwGT2mLwUoGXJakPs9e16dxqbZ95YLFAAtvnmZfl0yKmznjdWwRGMSXbgr5d0rIvuAB4B3RWQXTqf3EUBLoBiYHvFamhopEiM3du071Hzku6if27KIrxbMZMdnSwmU9Smz5wVMn+78r+rV/ltWdU59YkwkhBUsAFR1tIi8AAwDLgS6uLu+AEaq6kcRrJ+pwQKN6Kiswt25bPn4eZ5YvQS02LNM3WM60aTX9dQ5sSudOztpzcq2/wZqkqrOqU+MqaqAwUJErgJWquo3ZfepajZwnVvOllU1lRLKL/pQFP2axy+fvMCe7H9DUdms+Y46LU4gvdf11Dv2FESk3IXff45JoAlPiZz6xJhoq+jO4jmgWEQ6qupXIvIwTjryVar6ta+Qqtr6FSYs/kNjG6elUi81iV37CjxHQ/lLS00mSeDXg4eCi6ry04IHKcj93vNcqc2OJb3X9aQdf1pJ/iao+MJveZmMKa+iDu77gQWAb8b2cJwA8qWI/CIiH4jITBG5XkROEpGILtFqaqayOW3y8gvYX1DMnwd15ttpF5ORnuYZKJJFmNa/E6nJpf83ExEanXZFufIpTVrQ7NLhtLj5r9Rv171UoAh24be8TMaUF/DOQlX/VGbTeRxax6Ir0APoyaEfgftFZA1OmvK7olBXUwMEWzcjUL9AsSorN+0sNR/Cp0HHc9j9yYsU/LyZRs2Oon73QdT5zblIsvO/d3KSUFysKE7QGdAteFqT6pL6xJhYCSdF+fvA+77XIlIfOIXSAaQLcCpgwSLOYpEUsDLnCNSh7QsSXp3eWlzEgXXv8sTnymGn9C73XklK5vi+/8PtXepz++238+b6n0vqlV4/lb37C/GFpyJVFmblkHlsUwsGxoQh5El5IR1MpA5wkqquCrH80cA/gSNx7lBmqeqjItIUpwmsNfA9MFBVd1V0LJuUd0gslkgNNKGuSf1Uxl16oud5KsoWmyxCsSopSVDgDmRSLWbf+mXk/Wcehbu2klSvIRl3PE1S3Qal3lvRZwuWnbO6ZNo1JlqqPClPRF4UkVEi0ltEmodyUlU9GGqgcBXiZKntCJwO3CUiHYGRwLuq2g54131tQlRRU080zwHOvIdAE9gqyhZbpE4zUUGx02m97+vl/PiP37Pj1RkU7toKOCnEd3+6uNx7KwqCFWXnjMeaAMZUVxV1SvcHpgD/BraJyGYRWSwiY0XkEhEJllQwKFX90RdcVHUPsB7IAC4H5rjF5gD9qnqu2iQW6YsrOlagwBTK+fO//4xtc+8nd9EkzxFOe7JfR/2WN00WqfBOINCop5bpaTEJqsbUFBX1WRwDdMPpi/A9XgZc6isgItuBLJwhtVk4w2o3V6YiItIap89jOXCkqv7o7tqG00xlQhSL9MXBJtR5BYaK3nMgZz27PpzLgR9Wex9Qkmh48m9pfOZVSMqhhYeu7l7xwo0VDYP9Q4AFkmzynTHlVTQaaguwBXjZt01EZgG34twB/Ay0BS7GyQnlK7NDVcO6uItIQ2AhcK+q7vYf5qiqKiKerRcichtwG8AxxxwTzilrtGjOE/DPEhtoPgSUDkyLs3OY8OraUqk5fAq2b2TXh3PJ/3ZFgCMJDTr24o9TJ7FV05m/fDNFqiSLcHX3o5ncr1OF9a0oO6fvc1RUd2OMI+TRUCLyAHAt0EtV/+O3vSswBrgC2Ezg60eg46biBIp5qrrI3fyTiLRQ1R/d5q7tXu9V1VnALHA6uMM5b3VQ2c7XaKQvXpydw/hX1pYauhroC/cPTIuzcxj+4ucUFJUuXfDzFvZ9Mp+8Lz4IeM60dqfTpOdgbrq0F3de7gSFYMHBS6BhsDb5zpjQhZMb6k5gvn+gAHD7HAaIyAhgLM78i5CIcwvxNLC+zLyOV4AbcJIS3oDf3U1tUXa0kf9yoaEGjGiPfPJJT0ulQd0Uz8A0460N5QIFwJ5Vr7InQKCo17oL6T0Hc+RxJzH+Mu+RVZEQjzUBjKmuwgkWR+I0PXlS1YdFpD8wGhgY4jF74OSYWiMivgbk0ThB4l8icgtOVttQj1djBJu8Fu+6+Pslv4DPxl3ouS9Q+3+jMwaxd/U7aOGhbDGHHXsi9c+8luNO7h6zi7ZNvjMmNOEEi6+AC4KUeR+4PtQDuncpEmD3+aEepyaK5oimcJu3gp2zojb+Fo3rsfWX8jkmUxo25bCul7D700WkHtGW4/rcwrp/jCqVlsMYkzjCCRazgZkiMlpVpwYocxTQtOrVMtEa0VS2DyEnL5/hL34OBG7eqmgUU6A2/vn/2cCIiQ+zY+MXNB8w1jMINDr9Suoc1Y76HXqwX5IqDBQ2ec6Y+Aon+d9jwDvAJHe+xan+O0XkEuAqnE5uU0XDe7cnLTW51LZIdL5OeHVt+c7mImXCq2vDqgs4s7X9J8Qtzs7hjElv0vT8IQy+8DQ2L/kH+d+uYP/GLM/jJqc1osFveiKSVGEQtMlzxsRfOLmhityA8DhwC3CpiPwM/ICzUl4GTpPSo9GoaG0Trc5Xr+GrFW0PtS4vfvo9d0+oQYaiAAAZN0lEQVT4E7kfPkfR3tJdW3nL5lKvbVdSkpI9U5AHC4KJ1H9jTG0V7rKqBcAQEXkKuBv4Lc5kvSJgAzBNVedGvJa1VKw7Xytq6glUl6KiIp5//nlu/f0D7N+51fO4hb/8ROGuH0lqmsH30y8pNwz3QGER9y74jBlvbfAMiLGYkW6MqVjYy6oCqOqnlF4pr0hVCyNZMRMd6Wmpnmm+01KTwhqqq6osXryYhx56iLVrvZuwpE4ajTIvp9FpV5BUt0GppqYDhYeWPS12bzN851y5aSfvfZlbErQaB6izTZ4zJnbCDhYi0gHojtPf8b2qvhfxWpmoGX/ZiQx/4XMKig81BKUmCfVSk8s1RXk19agqS5YsYcyYMQTM8pucymFdL6Hx6b8juX5joHRTU0VDcfMLipj3yQ8lzVQ5efmkJgupSVKqzjZ5zpjYCmcGdxLOBDrf0FgBin3HEBHRSOY7N1ERqP8hUJ4k/1FQWVlZ3HfffXzwQYBZ10nJNDz5QhqfOYiUw5qVbM4o06QVrPmo7P9EBUVKgzrJHFG/jo2GMiZOwrmzGIEzm/pjnEywF+JkpvU5U0TmATfZ3UZiKtsnMXNQ51LBw2t4rLjv69clgx07dngHChEadDyHxj2uIbXJoWTETeqnkj22/GS9YEkIvfx6sIhfD+aXCzzGmNgIZ+jsTTid2Ger6lPAF/47VfUjnPUpat1s6+og2PDT4b3be86OVChJ2X3hhRfSq1evUvv79+9Py5seo1nf+0oFCoC8ACOsAg3FhcAzNH1s2Kwx8RFOsDgWeENVA+d9cNKUn1m1KploCLZ2Q78uGSXNP8UH9pUq52s2EhGmTJkCQO/evVmxYgULFy6kdbsOnucM1AHdr0sG0/p3IsPdn+xOxstIT+Pa04NnD7Y1J4yJvXCaoXYDdYOUyQHOrXx1TDQszs4JuvY1QPOkX/nyzX/y69qltLzlCVIaOf0O/hf9s846i88//5yTTz65ZNvw3u3LZZZNTZYKO6ArGhb82uc/eo5+ClRvY0z0hXNnsQK4wO3oDqQYaFy1KplI8jU/BdIyPY0dO3bQ7/o7yJpxHXuzX0cP5vPLf58HvEcd+QeKEmV7pasw1GH8ZScGbKbyr7cxJnbCCRZPA+2AiRWUOZkKMtOa2KtomKoe2EfyZy9wTOs2vDz3yVLLle5dvYT6+3MrXN/a/xz+w1oBCoq10k1FZZupyvZj2LBZY2IvnHQfC0VkATBKRH4DlEolKiJX4GSlfTGyVTSV4b+iXVnFBfvZs+p1dn/yIj/s3+P5/pTDmpGcnxfSqKNIzLD2mj3+0cjzAu6z0VDGxFa4k/Kuxblz+B/fBhF5D2gGdAQKgIcjVrsaLJoXwECLFWlRAXs/f5tf/ruAor07Pd+b3LApjc+8ioYn/5a9yakhna+qGXKDLfRka04YE38hN0O5dxOqqkNxRjzNB3YAZwMnAmuAvqrqnWLUlIh2FtWyTU9aXMTeNe+Q89Qd7FzyhGegSEprRPo5N9Pytqc4rMvFSHJqyBf7qmbIDTZSyxgTf+HcWawFngWuV9XlwHIoyQ0lqlp+hRvjKdpZVP2bfw7mbiJ38TQKd27xLCt10jjizCtpdOrlHEyqV7I9nIt9vy4ZrNy0k/nLN1OkSrIIA7qFfjdgiQKNSXzhBItdeKxVoaoHPMqaCkT74ujfLJTSqDnF+bvLlZGUuhzWrS9H9hjIHwc7y6ZXtllscXYOC7NyKHKzvRSpsjArh8xjm5Za6yLQ8aO10JMxJnLCCRbLAO/ZVyYs0b44Du/dvqQPIKlufRqffiW73psNQEpKKs0yLyal2wCOaZVRLg15ZQS7UwrWJ+FfX5+aMOLJOuZNTRLO0NkpQN+yK+SZ8EVjFbxVq1ZRWOhkifcfeirACedeSbOjWnLjjTfy9ddf8eN/F7P5sev4aOR5EW/28toeyuxx//pmpKeFNGQ3kdnqfqamCefO4nfAUuAdEfm9qs6JUp1qvEiugrd27VoeeughXnrpJWbPns1NN91Ucg7/4/067CsaNGgQmQ9QRrA7pVCa3WraiCdb3c/UNOEEi/tx5uUKMFtEpgOv43R0rwTW2AJIoavqxfHbb79l/PjxzJs3D19m+PHjx3PNNddQt275rCzRChRA0Gak2tgnYZ32pqYJJ1ich7OEqu/vBOBmnGy0AAdFZA2wUlXvjGgtTYktW7YwadIkZs+eXdLs5PPDDz+Qcem9NOzal2QRru5+NJP7dYp6nYLdKdXUPomK1MYAaWq2cGZwvw+873stIvWBUygdQE4BugEWLCIsNzeXadOm8fjjj3PggPcAtDot2pHS7FjAGZH07Cc/AMQsYAS6U4pks1t1URsDpKnZJJKL24lIHeAkVV0VsYOGKDMzUwMu81kN+UbSbN6Wi65+jR2fLGJ//j7PsieddBK5J/Sj7vHdESmdSSlZhG+nXRyLKpsybDSUqQ5EJEtVM4OVC3pnISLjgduBw4FNOKvkPayq5XJIq+pBIOaBoqZZnJ3DiOdXsH35YnYvX0jx/r2e5Y4//ngmTJjAoEGDOG7Mm55limyl27ipaZ32pnarMFiIyM3AWL9NxwETgFOBy6NYr1pt2stZfPu3myn6dZfn/latWjF27FhuvPFGUlOd/E3JIp6BIVmCrT1njDHBBZtncQdwECeBYCucrLKrcOZb/K6qJxeR2SKyXUS+8NvWVESWiMjX7mOTqp6nutm+P5m6rU4stz2pfmNmzpzJ119/zZAhQ0oCBcDV3Y/2PFag7cYYE45gweI44EVVna+qW1V1KfBbnNQf10fg/M8AF5XZNhJ4V1XbAe+6r2uVlulppPe8Ftx1pqRuA9J7Xc+pD8zj3nvvpV69euXeM7lfJwaffkzJnUSyCINPPyYmndvGmJovWJ9FE+Ab/w2qmicir+PcZVSJqn4oIq3LbL4cOMd9PgdnBNaIqp4r0agqDz46h3//oOTVOaJUB6gzkuYge7r0Qeo2oNFp/Wl4WGNGXlbxhX9yv04WHIwxURHK0Nlij20/4HR4R8ORqvqj+3wbcGSggiJyG3AbwDHHHBOl6kTe0qVL+Z977uerL7JJa3c6R/R/sFy+JIAZDYbZSBpjTEIId/Ejn0IgtJVxqkBVVUQCDudR1VnALHCGzka7Pj6VHRL5ySefMGbMGJYuXVqyLf/rTziwdQN1W7YvlQ7CRtIYYxJJKIkEHxKRNSLytIjcJiKdqXyQCcVPItICwH3cHsVzha0yCeJWr17NZZddxhlnnFEqUPjkLXu25LmlgzDGJKJgweId4BeclfBuAp4AsoDRACLyiIgMFpGTRCScDLYVeQW4wX1+A/ByhI4bEeGs6vbVV19x9dVXc8opp/Dqq696Hi/t+O40OffmkteWDsIYk4gqvENQ1QsBRKQNkOn31xVoDAzDSS4IcMAdAputqreHcnIRmY/Tmd1MRLYA44DpwL9E5BacSYADw/xMURVKgrhNmzYxceJE5syZQ1FRkWf5U7r3JK9jfziiXck2SwdhjElUITUnqep3wHfAC75tItKO0gGki/vYDWfGdyjHvTrArvNDeX88VJQgbtu2bUydOpUnn3ySgwcPer7/jDPOYMqUKZx77rmWDsIYU21Uuu9BVb8GvgbmA4iTlKgDTsCosQIliLv/whPo3bs3q1ev9nzfKaecwpQpU7j44otL8jdZJ7YxprqIVD8D6livqnMjdcxEFGhVtyu6tmL48OHlyp9wwgksWLCAVatWcckll5RL9GeMMdVBRLPOxlMiZJ0tKiri5JNPZt26dRx77LGMGzeO6667jpSUaA4eM8aYyotY1llzSEFBAbNnz0ZEuO2228rtT05O5pFHHuHbb79lyJAhnivWGWNMdWTBIgRFRUXMnz+fcePGsXHjRpo0acLAgQNJT08vV7ZPnz5xqKExxkRXxPosqqPF2Tn0mL6UNiNfp8f0peUm1qkqL730EqeccgrXXXcdGzduBGDXrl387//+bzyqbIwxcVFr+yx8M7HLjmqa1r8Tl3duyZIlSxgzZgyBjtmsWTM2b97smQG2MnzDaHPy8kvWpsiw4bTGmCgLtc+i1t5ZBJqJ/dCTL3LOOefQu3dvz0CRkpLCHXfcwWeffRbRQOFLIQKHVrcLJZWIMcbEQq3tsyg7E/vAtm/IWzaXTRuzPMuLCIMHD2b8+PG0bds2onXxClw+/skFjTEmXmptsPDNxC7YsZm8/zzLvg0fBSzbv39/Jk6cyIknll+9LhKCJQ+05ILGmHirtc1Qw3u3Jy01mT3ZrwcMFBdddBErV65k4cKFUQsUEDx5oCUXNMbEW60NFr6Z2O0vugFJKT0f4qyzzuLDDz/kjTfeoFu3blGviy9webHkgsaYRFBrgwU4AWPFlCsZPuweALp27cobb7zBhx9+SM+ePWNaD18KEaBkHW1fKhHrrzDGxFutHTrr7+eff+b999+nf//+lrvJGFOrWLqPMBx++OEMGDAg3tUwxpiEVauboYwxxoTGgoUxxpigLFgYY4wJyoKFMcaYoCxYGGOMCcqChTHGmKAsWBhjjAnKgoUxxpigLFgYY4wJyoKFMcaYoCxYGGOMCSphg4WIXCQiG0TkGxEZGe/6GGNMbZaQiQRFJBn4G/BbYAuwQkReUdV18a2ZCdXi7BxmvLWBrXn5tExPY3jv9pZq3ZhqLFHvLE4DvlHVjap6EHgeuDzOdTIhWpydw6hFa8jJy0eBnLx8Ri1aw+LsnHhXzRhTSYkaLDKAzX6vt7jbShGR20RkpYiszM3NjVnlTMVmvLWB/IKiUtvyC4qY8daGONXIGFNViRosQqKqs1Q1U1UzmzdvHu/qGNfWvPywthtjEl+iBosc4Gi/163cbaYaaOkuDxvqdmNM4kvUYLECaCcibUSkDnAV8Eqc62RCNLx3e9JSk0ttS0tNZnjv9nGqkTGmqhJyNJSqForIUOAtIBmYrapr41wtEyLfqCcbDWVMzSGqGu86RERmZqauXLky3tUwxphqRUSyVDUzWLlEbYYyxhiTQCxYGGOMCcqChTHGmKAsWBhjjAnKgoUxxpigLFgYY4wJyoKFMcaYoCxYGGOMCcqChTHGmKAsWBhjjAnKgoUxxpigLFgYY4wJyoKFMcaYoBIyRXmiW5ydY+m3jTG1igWLMC3OzmHUojUla0zn5OUzatEaAAsYxpgay5qhwjTjrQ0lgcInv6CIGW9tiFONjDEm+ixYhGlrXn5Y240xpiawYBGmlulpYW03xpiawIJFmIb3bk9aanKpbWmpyQzv3T5ONTLGmOizDu4w+TqxbTSUMaY2sWBRCf26ZFhwMMbUKtYMZYwxJigLFsYYY4KyYGGMMSYoCxbGGGOCsmBhjDEmKFHVeNchIkQkF9gU73pUUTNgR7wrkUDs+zjEvovS7Ps4pKrfxbGq2jxYoRoTLGoCEVmpqpnxrkeisO/jEPsuSrPv45BYfRfWDGWMMSYoCxbGGGOCsmCRWGbFuwIJxr6PQ+y7KM2+j0Ni8l1Yn4Uxxpig7M7CGGNMUBYs4kREZovIdhH5wm9bUxFZIiJfu49N4lnHWBGRo0XkPRFZJyJrReQed3tt/T7qicinIvK5+31McLe3EZHlIvKNiCwQkTrxrmusiEiyiGSLyGvu69r8XXwvImtE5DMRWelui/q/FQsW8fMMcFGZbSOBd1W1HfCu+7o2KATuU9WOwOnAXSLSkdr7fRwAzlPVU4DOwEUicjrwMDBTVY8HdgG3xLGOsXYPsN7vdW3+LgDOVdXOfkNmo/5vxYJFnKjqh8DOMpsvB+a4z+cA/WJaqThR1R9VdZX7fA/ORSGD2vt9qKrudV+mun8KnAe86G6vNd+HiLQCLgH+z30t1NLvogJR/7diwSKxHKmqP7rPtwFHxrMy8SAirYEuwHJq8ffhNrt8BmwHlgDfAnmqWugW2YITUGuDPwMPAMXu68Opvd8FOD8c3haRLBG5zd0W9X8rtvhRglJVFZFaNVRNRBoCC4F7VXW38wPSUdu+D1UtAjqLSDrwEtAhzlWKCxHpC2xX1SwROSfe9UkQZ6lqjogcASwRkS/9d0br34rdWSSWn0SkBYD7uD3O9YkZEUnFCRTzVHWRu7nWfh8+qpoHvAecAaSLiO8HXisgJ24Vi50ewGUi8j3wPE7z06PUzu8CAFXNcR+34/yQOI0Y/FuxYJFYXgFucJ/fALwcx7rEjNsG/TSwXlX/5Lertn4fzd07CkQkDfgtTj/Oe8CVbrFa8X2o6ihVbaWqrYGrgKWqei218LsAEJEGInKY7zlwIfAFMfi3YpPy4kRE5gPn4GSM/AkYBywG/gUcg5NBd6Cqlu0Er3FE5CxgGbCGQ+3So3H6LWrj93EyTidlMs4Pun+p6kQRaYvz67opkA0MVtUD8atpbLnNUPerat/a+l24n/sl92UK8JyqThGRw4nyvxULFsYYY4KyZihjjDFBWbAwxhgTlAULY4wxQVmwMMYYE5QFC2OMMUFZsDDGeHKzAauILApe2tR0FixMtSEib7sXLxWR31dQ7mm/crNjWccappv7uCqutTAJwYKFqU664aQzBzjZq4CbyvsmoMjdtDIG9aqpurqPWXGthUkIFixMteDOXG0KfIoz471csBCRJOBvQC6HgoQFi8qzOwtTwoKFqS58i7xk4aR3ONENDv5ux/k1/ABwHFAAfF72QCIyQETeEJEdInLQXV1stIgke5S9RkTmichXIrJHRHaJyAoRucmrkiLSU0QWici3IrLfXQ3xUxGZVqbcKLeZrL/HMY5x973kt+1cd9sjInKaiLwsIjvdbSdW4fOliMg9IrLare8mERnh5uvqCmxV1Z+8PqupXSxYmOrCP1isAurjBAQARKQZMAX4GPgAJ+fWF/75gtw1IubjLJpzPPAC8DhOPqopQKn+DTdh2z+Btji5qx4DFgFtgNkiMqJM+dHAhzi/yN8F/gS8CtQFepf5PL5f7V53Pqe6j/6/6H1NQicB/3Hr/CTwHPBlJT9fHeDfOOtFFAF/des9HpgFHIU1QRkfVbU/+0v4P2ApzqIvJ+JkG1VggN/+/8O54HUBBrj7Z5U5xmPu9mlAit/2VOAjd19Hv+0NgaM86tIC2AN86bftSJz+lGVAHY/3NCvz+jsgN8BnnebWpa/ftnnutj3A6QHeF+7ne8rd9hBunjh3ey93uwLj4/3f3v4S48/uLEzC82sS2YfzK9r3i/tkd3934GbgSVXNxuNXu1vmTuBlddJe+zrKUdUCDi1J2d1v+15V3Va2PuqsSLYVpw/FpwNOltivVPWgx3t2+NWlKdCawL/afXdRXncW96jqJ2XfEO7nE5HTgFuB11R1kqqqX/kPObTetd1ZGMBWyjPVQzugMfCxqhaJyHfAL8DJfp3aPwNj3PK+i61/E8/dgAD7RGS8xzlOch9LfkCJSBNgKM76z+2BRpRuus32e77WrdPNItIc507gbVXd5XEu34U/UOd7N5zV4ba69WgAnICzoM0zAd4T7ue7232cEuB4P7uP1rltAAsWpnoodfFXVRWRbKATcBvOxXWI34W5G3AAZ30Mnwvdx6uDnGsTlKwp8TZO89KnOGsn/IzT1NQGuB6/znNV3eGuyzEOuBi4FCgSkSXAGFX1v+j67nzK/Wp3R301Ad7023wKzkX+dVUtLvueynw+t/zPOGuGeGmLE7BqzQp0pmIWLEx14N+57ZMNnA1MBVbgrLSHiLTBaR5a4Ta/ICL1gObAh6p6dojnnAukA+eq6vv+O0Rkovu01J2Bqn4B/M7tOO6FE8h+B5wqIhl6qLO9i/vo9av9Yr/P5+O7E/G8sIf7+dzyRwDZ/s1Pfvu7Ai2BN4Idy9Qe1mdhqgOvYLEKp9mlMXCX30XPa5SRuI/NQjmZiByN0x/yvkegSMeZ9Fe2PiVU9aCqvqOqA3FGLh2Oc4fi0wEoUNVN/u8Tkbo4w399n88nWLNVWJ8PZyBAEU7A8OJrzrMmKFPCgoVJaG6fRBfgVw51uoIz5PMK4HxVXeG3vVx/harmA6uBjl7zGtzznOU3D2G/+9hWRFL9yhwOLABa4TRHfeZu7yIix1GGiByP01fwA7DFb9dBIFVE2vmVbYAztNXXt1D2zuIgpZvVSoT7+dw7rq+BDBG5tEy5EYDvGNa5bUpYM5RJdB1whrB+5N9er876wos9ygeavzAceB1YKCLv4Fxck4AM9z2pqnqMe+xcEVkKnAcsd8u3APoA7+HMW1inqr6gcjdwo4h84p53L06b/2Xu/pvL9DW8hTOX4kN34l1D4Hy3Tj/izCHZCCV3Gx2B1V6jrCrz+VzTcEZILRSR54FtOGvCdwI2A0djdxbGX7zH7tqf/VX0h9ORrMCjIZb/GWeIbbLHvlNxJqxtw5ndvQPn1/qTOHco/mWb40zIy8WZ2/CxW5cubn2e9it7OU4fxwZgN85dwPc4cz/aedSjHvAozvDbfTgB5nacPpJinOYvX9lu7vmeDOGzh/z53PL34ASlApwUKgtx7mI2ATvi/d/e/hLrT1TL9W8ZY4wxpVifhTHGmKAsWBhjjAnKgoUxxpigLFgYY4wJyoKFMcaYoCxYGGOMCcqChTHGmKAsWBhjjAnKgoUxxpigLFgYY4wJ6v8B8SdJZgkdXu4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.scatter(y_test, predicted)\n",
    "ax.plot([y.min(), y.max()], [y.min(), y.max()], 'k--', lw=4)\n",
    "ax.set_xlabel('$Measured$', fontsize = 20)\n",
    "ax.set_ylabel('$Predicted$', fontsize = 20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 交叉验证"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# cross-validation \n",
    "    \n",
    "k-fold CV, the training set is split into k smaller sets (other approaches are described below, but generally follow the same principles). The following procedure is followed for each of the k “folds”:\n",
    "- A model is trained using k-1 of the folds as training data;\n",
    "- the resulting model is validated on the remaining part of the data (i.e., it is used as a test set to compute a performance measure such as accuracy)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:23:06.421218Z",
     "start_time": "2019-04-22T08:23:06.407755Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-1.5841985220997412"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "regr = linear_model.LinearRegression()\n",
    "scores = cross_val_score(regr, boston.data , boston.target, cv = 3)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:03.323654Z",
     "start_time": "2019-04-22T08:24:01.612164Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xu8VHW9//HXh5uKaYBcRK7mJYPSbW5Qyk4JaEonL5llUZFHH+Q5lZl2UTk/TYsuVl6qkye8pB12qGkUJkpIntQ8qWAoAipIXkAUxGuAKO7P74/vmvbM7LXmumb2Hub9fDzmMTPfdd1LWZ/5fj/r+/2auyMiIpLRo6tPQEREuhcFBhERyaHAICIiORQYREQkhwKDiIjkUGAQEZEcCgwiIpJDgUFERHIoMIiISI5eXX0ClRg4cKCPHj26q09DRKShLFmy5AV3H1RsvYYMDKNHj2bx4sVdfRoiIg3FzJ4qZb1UmpLM7Ggze8zMVpvZOTHLP29mG81safQ6LWvZNDNbFb2mpXE+IiJSuaprDGbWE/gv4EhgLfCAmc1z9xV5q97g7l/K23YAcAHQCjiwJNr2pWrPS0REKpNGjWE8sNrd17j7G8D1wHElbvthYKG7vxgFg4XA0Smck4iIVCiNwDAMeCbr+9qoLN+JZvawmd1kZiPK3BYzm25mi81s8caNG1M4bRERiVOvx1VvAUa7+4GEWsF15e7A3We5e6u7tw4aVDSpLiIiFUojMKwDRmR9Hx6V/ZO7b3L3bdHXq4BDSt22IbW1wejR0KNHeG9r6+ozEhEpWRqB4QFgPzPb28z6ACcD87JXMLOhWV+PBVZGnxcAR5lZfzPrDxwVlTWutjaYPh2eegrcw/v06QoOItIwqg4M7r4d+BLhhr4SuNHdl5vZRWZ2bLTaGWa23MweAs4APh9t+yLwbUJweQC4KCprXDNmwJYtuWVbtoRyEZEGYI0453Nra6t32w5uPXqEmkI+M2hvr//5iIhEzGyJu7cWW09jJaXpugI59ZEj63ceIiJVUGCoVHaCeeRIOPxw+Pzn4YADYJddctfdeWeYObMrzrI8SpqLCAoMlclPMD/zDPzlL/Cxj8GyZXDllTBqVGg+6tEDBg6EE0/s6rMuTElzEYkoMFQiLsEMsGQJ9OwJU6fCk0+GnMItt8DatXDuuYX32dW/1pU0F5GIAkMlnn669PIpU+CLX4TLLoOFC+O36w6/1sv5m0Rkh6bAUA53mDMneXlSgvnii+Fd7wo5iE2bOi8/77yu/7WedO5Kmos0HQWGQrKbd0aMgAkT4NOfhn32CQnlbH37JieY+/YN+3ruubCfTHPRz34G3/529/i1fuGFISeSrdDfJCI7LAWGJPnNO2vXwn33wUknwcqVcNVVHQnmUaNg1qyQW0iyYkXIP2zd2tFc9OUvw/nndw4yGfX8tf7GG+G8+vUL3wcOLP43icgOSR3ckoweHW7e+UaNConltPa3116hqWn69NzmpL5963djfvNN2H9/GDQoPF319rfD6afDJZfU/tgiUjfq4FattJt3krZbvz7c/GfNCkEHQi3k8svr92v9f/4nBLvzz4feveHgg+GBB+pzbBHpdhQYkqSdjC22v8wjrn/9a2jSqVdN7s03Qx7hkEPgIx8JZePGwYMPwvbt9TkHEelWFBiSzJxZXoK5lP317Vt8f+PHw5gx8MtfVnaccv3617BmTagtZJLP48aFZq0V+bOzikgzUGBIMnUqHBfNUFpqgrnY/jLNRYX2ZwannAL/938hyV1L27fDd74DLS3w0Y92lI8bF94bpTmpqzsHiuxgFBgKee45OPDA0IP5ySerb/PP7hFdaH+f/Wx4gunaa6s7XjFz5sDq1bm1BYB99w1PJzVCYOgOnQNFdjAKDElefTU8oXPMMfU/9pAhob3/V7+qXTv/W2+F2sKBB3bUjDJ69IDW1tICQ1f/WtdQHiKpSyUwmNnRZvaYma02s3Nilp9lZivM7GEzW2Rmo7KWvWVmS6PXvPxtu8wdd4SbclcEBoB/+7dQY7n99nT3m7mR9+oFjz8OH/pQuKnnGzcOHn4YXn+98L66+td6d+gcKLKDqTowmFlP4L+AY4AxwKfMbEzean8DWt39QOAm4OKsZVvdvSV6HUt3cdttsPvu8L73dc3xp0yBwYPhmmvS22f2jTzjqqvib+TjxoXAuHRp8v66w691DeUhkro0agzjgdXuvsbd3wCuB3LaJtz9TnfP3EH+CgxP4bi14x4Cw+TJ4bn+rtC7d8g13HILbNyYzj7LuZGXkoDuDr/WZ86EPn1yy3bZRUN5iFQhjcAwDHgm6/vaqCzJqcBtWd93NrPFZvZXMzs+hfOp3vLlsG5d1zUjZZxySvjVPnt29fvati2+5zXE38iHDYM99ywcGJJ+le+yS8jRJEkzLzF1ajiP3r07Euif+YyG8hCpQl2Tz2b2GaAV+GFW8aioi/angcvMbJ+EbadHAWTxxrR+QSe5LYpbRx9d2+MUM3Zs6Nfwy1+W3+Et++Y7aBAML1BJi7vBm4VjFwoMM2eGXEW23r3DeFDvfW8YIDA/AKSdl1i9Ory+/e2QUH/nO9X/QqRa7l7VC5gALMj6fi5wbsx6k4GVwOAC+7oW+HixYx5yyCFeU0cc4f7ud9f2GKU65ZTQD9rMfdQo99mzi28ze7Z7376Z/tMd23/0o53L+/ZN3ue3vx3Wefnl+OXt7e6DB7vvvHPu+d19t/uAAbnHAfeddnLfbbfO5RC2rcQFF4RjP/NM+P6DH4T9PfpoZfsT2YEBi72U+3opKxXcAfQC1gB7A32Ah4CxeescDDwB7JdX3h/YKfo8EFgFjCl2zJoGhldfde/d2/3rX6/dMUo1e7b7LruUfiPPGDUq+eY7e3Z4LyXQ3H572G7RovjlS5aE5bNmdV42fHj8OSS9zEq7Jtna29332cd90qSOsvXr3Xv2dP/GN8rfn8gOrm6BIRyLKcDj0c1/RlR2EXBs9PkO4HlgafSaF5W/D1gWBZNlwKmlHK+mgWHu3HBZ/vSn2h2jVIVu8IWYpXPzfeGFsN33vx+//BvfcO/VK6xX6jkUep11lvvPf1564PrLX8J2116bW37sse5Dhri/8UZ5f6/IDq6ugaHer5oGhunT3d/2Nvdt22p3jFJVeoMfObKygBLnHe9wP/HEzuXt7e6jR7sffXT8dklBbY89Ojdn7bKL+wc+EL9+oRrS6aeHbV99Nbf8d78L286bV/7fK7IDKzUwqOdzNs96TDX/EciuUOkz+p/4ROeySgcATEpA339/GNbjk5+M3y5p0MDLL+88ZtSVV8Jdd4W5KfIlPU67bRvccAOccALstlvusilTQu/xq68u6U8UkVwKDNlWrIBnnun6x1Qz4m6uO+1U/Ab/+OPhZjlyZPUDAI4bFx5nff753PIbbgjB8/iEJ4wLDRqYNGbU+vXx+4p7nHb+fHjppdDXI1/v3vC5z8Ef/hB6j9dDVw8NIpKmUqoV3e1Vs6akH/4wNEE8/XRt9l+J7GRxz57Fn5Zau9a9Rw/3c85J5/h33RWuyS23dJS99Zb7sGGhLT9N5eRUTjgh5BHefDN+XytXhm0vvjjdc4wT9xRYKQ8JiNQZakqqwG23hb4DI0Z09Zl0yP51/Z3vwCOPhDGMklx9dVj3tNPSOf573xt+BWc3J917b+gAmNSMVKm4GhKEcaOyvfhiqA18+tOd+1FkHHBAGM7kmmtqP+lRdxgaRCRFCgwZ//gH3H1392lGivOFL8CuuybPxfzWW2Hso8mTYZ/YfoLl23XXMHFQdmC4/vowiVH2HA5pyG9+Gj4c9tgj3NxffLFjvRtvDDPPxTUjZTv1VHj00TC3RS11h6FBRFKkwAChPfgd7wg3m1/9qvu2D/fvH349//rX8OyznZcvWBByJF/4QrrHHTcuJJvdQ/C56aYwLHh+0jcN2TWkZ54JuYRnn4Vp00IZhP9GY8eGCYYKOemkkAc56qjatv1rID/ZwSgwZIZoyAyzsWFD957o5cwzw835Zz/rvOwXvwgjsh6b8iC148fDpk3hhv3nP4dEdNrNSIWOfcklHU1Hw4aFGsC6dSFAFjJvXrhWmzeXN/xGuYnkmTM7D13es2dpT4EpaS3dUSmJiO72SjX5XGknsq504onu/fu7v/ZaR1naSedsixeHa3LDDaGfx667um/enP5xkrS3ux96aOf/RsUSvJX8t60kkXz33WG9fv3CQwKZYT+WLi38dylpLXWGOriVKK1ewvV0773hHH/yk46yCy8MZatXp3+8bdvc+/RxP/PM0EHt5JPTP0YxI0aUf5Ov5L9tJcFk8uQwZlQmWL74YhgratKkENTSPJZIFUoNDGpKasT24QkTwhM3l10WmkpqkXTO1qdPSARffnloUlq0qP5NHmvXxpcXSvBW8t+23ETyPfeE2f6++c2OJ6r694dvfStcp1tvTe9YInWiwDBzZpg/IFulvYTr6eyzYc0a+N3vapd0zmhrCzcrjx773Lix/nmYSm7ySb2vC/23Lfc4F14Y8jqnn55bfvrpsP/+8LWvhYca8q1Y0TF/RKnHEqmXUqoV3e2Vege3yy/PrcY3Qhvv9u2h+aJPn3DePXp0HkwuLd2hyaPS9vhMB8HMNpdcUnz9Hj06Nz1deWXndTO5hR//OH5f8+aF5T/9aW75E0+477WX++67hyHL86/r1KmFz1GkQijHUIZHHw2Xoq0t3f3W0uzZYXjweiQuu0seppwhw/OtWxeu1xlnFF7v/vs9J5E8eHB4nzIlBONskyaF3tdJifj2dveJE0Ne5sUXQ9natWHwwQED3Jcty/2bRo50HzcuHP/kkyv/W0USKDCU48EHw6WYOzfd/dZSPX/Fd4caQxo++9nwRNVLLyWvc/zxISi88kpH2RVXhL/3zDM7yjJDhRSrgSxdGtbbbbdwk+/VK0xY9MAD8eu/+ab7YYd1vtZ6WklSUGpgUI4BOoYz2HXXrj2PctQzcVlJW3139NWvhj4NV10Vv3z58pCzOeMM2H33jvLTTw/9Ry67LHQwHD0a/uVfQt+Dfv0KH/ORR0KfhtdeC7f47dtD+WOPxa/fq1d858VGGmJDfTMaXynRo5QXcDTwGLAaOCdm+U7ADdHy+4DRWcvOjcofAz5c7Fip1xgWLAi/yu65J9391lK9f8VX04zTnRxxRHj0NW4Sn6lTQ40ibuKh7dvdW1rK/yVfyX+n7tJ0Vwn1zejWqPMMbj0Js7e9g47pPcfkrfMfwH9Hn08Gbog+j4nW34kwPegTQM9Cx0s9MGRmbXvwwXT3W0v6B1iZTEJ4zpzc8ieeCEnns89O3rY796XoLhr53JtAqYEhraak8cBqd1/j7m8A1wPH5a1zHHBd9PkmYJKZWVR+vbtvc/e/RzWH8SmdV2kyTUlxI3t2V4XmO5BkH/kI7LdfGGbDvaP8Bz8IzThnn528bb36UjRy0536ZuwQ0goMw4Bnsr6vjcpi13H37cArwB4lbltbjZhjgOQJbyRZjx4hX/DAA2H4cAjjLl17bcgfDB2avG29bvKZoJ/Z79ve1jhBvxE7jEonDZN8NrPpZrbYzBZvzAx4l5bNm8N7I9UYpHLTpoXeyZdeGr7/+Meh9/g3vlF4u2pu8uXW7KZODYP+HX44HHRQYwQFCD2+8zVKbUf+Ka3AsA7Int1meFQWu46Z9QLeDmwqcVvcfZa7t7p766BBg1I67UgjNiVJ5XbdNfQSv/nmUEO49NIwv0SmBpGkmpt8pTW7lhZ46KGOIce7u0yNa6edwvvw4Y1T25F/SiswPADsZ2Z7m1kfQnJ5Xt4684Bp0eePA3+KkiHzgJPNbCcz2xvYD7g/pfMqzZYt4R965n9m2fENi1orM3NCb95c2jAf9W6+a2kJk0itWVPb46Rl/vwwxMzNN4fvV16poNCAUgkMUc7gS8ACYCVwo7svN7OLzCwzOcDVwB5mtho4Czgn2nY5cCOwArgd+KK7v5XGeZVsy5bwKzJp7BrZ8fzoR53LumNfgYMPDu9/+1vXnkep5s+HiRNDPw8z+Otfu/qMpAKp5Rjcfb677+/u+7j7zKjsfHefF31+3d1Pcvd93X28u6/J2nZmtN073f22tM6pZJs3qxmp2TTK0zNjxoSnpZYurd0x0uqQtmoVrF4NU6aE2f3e/e7aT6sqNdEwyeea2rJFgaHZNMrTMzvvDO96V+0CQ2YGw6eeCo/vljrLXZz588N7Zt70CRPgvvsaJz8i/6TAAAoMzaiR+gocfHDtmpJmzOh4+CKj0ia1+fNDENt77/B9wgR45RV49NHqz1PqSoEBOnIM0jwaqYNgSwusXx/m2k5bWk1qmzfD//5vaEbKOOyw8K7mpIajwADKMTSrRukg2NIS3h96KN39PvFEyCvEKbdJ7U9/gjfeyA0M++8f+osoMDQcBQZQU5J0b5nAUKw5qZwk8qpV8MEPhhzGzjvnLqukSW3+/NBD+/DDO8p69Ai1Bj2Z1HAUGEBNSdK99e8fmroKJaCLJZGzg8awYTBuHGzbFjr1XXVVR8e0PfYov0nNPQSGI48M84NnmzAhTGP6yitl/cnStRQYQE1J0v21tBQODElJ5PPO6xw0nn023Ki/9jU48MAQBNatgz33hKOOKr9JbcWKkJPIbkbKOOywcMz77itvn9KlFBhATUnS/bW0hMl9MuN65SuURD7llM5BA+CKKzo+m8GkSbBoUe6os6W49dbwnnlMNduhh6qjWwNSYAAFBun+Dj443LCXLYtfnpQs3n13ePPN+GX5wWTSJNiwIcw6V47588NAf5lhRvKPP3asEtANRoHBXTkG6f4yCeik5qSZM+OTyD//echPxMkPJpMmhfdFi0o/r1degXvuiW9GysgkoNXRrWEoMLz+eggOqjFIdzZyZEhCJz2ZNHVqR1NOfr+MUjvzjRwZJjG6447Sz2vhwjBkeaHAMGECvPxy8jzX0u0oMGjIbWkEZoUT0O6hCWjixM79MsrpzDd5Mvz5z8nNT/nmz4d+/To6s8WZMCG8K8/QMBQYFBikUbS0wMMPw/btnZc9+GDom/CpT8VvW2pnvsmTwzDf95cw8n17O9x2G3z4w2GgvyTvfGcIHsozNAwFhkad1lOaT0tLaPp8/PHOy+bMgd694WMfq+4YH/pQqFUUa05qawvJ5ueeC+sW6kzXo0d4OkmBoWEoMGhaT2kUmbkZ8puT2tvhhhvCL/cBA6o7xoABcMghhRPQmX4RmUmONm0qPiLrhAmwfDm8+mp15yd1ocCgpiRpFAccEHoW5weGv/wF1q5NbkYq16RJ4df9P/4Rv7ySEVkzHd2SmqjSmhNCUlFVYDCzAWa20MxWRe/9Y9ZpMbP/M7PlZvawmX0ya9m1ZvZ3M1savVqqOZ+KqClJGkXv3mHym/wnk+bMCdNpHnts/Hblmjw55DHuvjt+eSUjsh56aHiPa05Kc04ISUW1NYZzgEXuvh+wKPqebwvwOXcfCxwNXGZm/bKWf93dW6JXDaepSqAagzSSgw8ONYZM7+Tt2+E3v4GPfjQMYpeG978/zH+elGcYPjy+vNCIrP36hdno4p5MSnNOCElFtYHhOOC66PN1wPH5K7j74+6+Kvr8LLABGFTlcdOjHIM0kpYWeOGFMN4RhFzACy+k14wEofbx/vcn5xkOOqhzWSkjsk6YEAJD/pAbjTLNahOpNjAMcff10efngCGFVjaz8UAf4Ims4plRE9OlZrZTgW2nm9liM1u8cePGKk87i2oM0kjyh+CeMwfe/vb4cYqqMXlymP9hw4bc8kcfhdtvD0N2lzvJ0WGHwYsvdjxVtW0bfPnLyWMz1XuaVeU5/qnAw8eBmd0B7BmzKKee5+5uZomjb5nZUOB/gGnunukbfy4hoPQBZgHfBC6K297dZ0Xr0NraWuYoXwUoxyCNJPNrfenScPOeOxdOPDE0/aQpMzzGnXfCJ6O0oHu4ke+6K9x4IwweXN4+N20K7wccAHvtFXImTz0Vgtqf/9y5Oelzn6vubyhHJs+ROYdMngO67wROteTuFb+Ax4Ch0eehwGMJ6+0OPAh8vMC+PgT8oZTjHnLIIZ6a737XHdxffz29fYrU0r77up94ovtvfxv+312wIP1jbN/u/va3u592WkfZzTeH4/3kJ+Xvb/Zs9759w/bZr7PO6lg+apS7mfvw4e577um+++7uS5ak8ucUNWpU53ODUL4DARZ7CffYapuS5gHTos/TgN/nr2BmfYC5wK/c/aa8ZUOjdyPkJ8oc1jEFmzeHqmP+BCMi3VVLS2hKmjMn/GqfODH9Y/TsCUcc0ZFn2LIFvvrVMH/Dv/97+fuLSzAD3HxzeM/umf3MM+Gx1n79QpPVXnvVvnlHeY4c1QaG7wNHmtkqYHL0HTNrNbOronU+AfwL8PmYx1LbzGwZsAwYCHynyvMpX2bIbbO6H1qkYmvWhKeRtmwJndtqYfJk+Pvfw7G+971wk/zZzwoPf5Gk3BvviBFwxhmhL8X69bV/jHXEiPjyeuc5uokK/gt3cPdNwKSY8sXAadHn2cDshO1r8FOnTBpyWxpJWxvcckvH93/8o3Zt4Zlf+PvsE97f9z74wAcq29fIkeHGHlee5Kc/jT+nGTPS/1unTIH//u/cskrmvt5BqOezpvWURjJjRniaJ1stnvlva4NvfSu37G9/q/zXeqlDf2erV/POxo2h9rXvvmFocwjjQJU79/UORIFBs7dJI6nXzTIuJ7B1a+UBqJyhvzOSahNpN++cdVYYw+l3v+vIefzyl00bFECBQYFBGku9bpa1CEClDv2dUUkto1wLFsDs2XDOOWEK0rFjQ/ny5ekdowEpMCjHII2kHjdLqF8AKiS7lpFxwQXp/ZLfvBlOPz3MF3HeeaFs8GAYOFCBoatPoMspxyCNpJImmUrUKwAVk6llZGoqPVK8ZV1wQdj3lVfmzpc9dqwCQ1efQJdTU5I0mnKbZCo9Rj0CUKlGjID3vCdMJVqN7GEvfvzj0Fcj/0mrTGBIGqqjCSgwqClJJF49AlA5pkwJQ4FXOtlP/vDeEAb1y3/SauzYcIx166o73wamwKAag0hjmDIlDDNebNrRJKU+aaUEtAKDcgwiDWLChDCSbKXNSaU+aaXA0OSBwV01BpFG0bs3HHUU3HZbZe3/pQ57MXBgeDpJgaFJvf56eFeOQaQxHHNMmKTo4YfL3/aQQzqXJT1p1eRPJjV3YNAkPSKN5eijw3u5zUkPPADz5sHhh5f2pNXYsbBiRdM+mdTcgUHTeoo0lqFD4b3vLS8wbN0K06aFbW+5pbQnrcaMgddeC0OAN6HmDgyqMYg0nilT4N574aWXSlv/P/8TVq6Ea64JczyUoskT0AoMoByDSCOZMiX84v/jH4uve9ddcOml8B//AUceWfoxFBiqY2YDzGyhma2K3vsnrPdW1kQ987LK9zaz+8xstZndEM34Vh+qMYg0nvHjYcCAws1JbW3haaMPfjDMRheXeC5kjz1gyBAFhiqcAyxy9/2ARdH3OFvdvSV6HZtV/gPgUnffF3gJODWFcyqNcgwijadnz5CEvu22UHPIl+nhnMkPbN8OX/5y+XNJNPGTSWkEhuOA66LP1xHmbi5JNNfzRCAzF3RZ21dNTUkijWnKlDDBzpIlnZedd17nHs6VTGaUeTIpLvjs4NIIDEPcfX30+TlgSMJ6O5vZYjP7q5llbv57AC+7+/bo+1pgWArnVBo1JYk0pg9/ODxymt+c9NJL6c0lMXZsaFVIexKkBlBSYDCzO8zskZjXcdnrubsDSQ/+jnL3VuDTwGVmtk85J2pm06PAsnjjxo3lbJpMTUkijWngQDj00NzAsGJFyD8kKXcuiSZOQJcUGNx9sru/O+b1e+B5MxsKEL1vSNjHuuh9DfC/wMHAJqCfmfWKVhsOxA5p6O6z3L3V3VsHDRpUxp9YgGoMIo1rr73g/vvDENqDB4f+Da+9Buefn85cEgoMVZkHTIs+TwN+n7+CmfU3s52izwOB9wMrohrGncDHC21fM8oxiDSmtraO2oJ7yDe8+WbII1x4YTpzSfTvHzrF1TMwZM8XMXp0+QnzlKQRGL4PHGlmq4DJ0XfMrNXMrorWeRew2MweIgSC77v7imjZN4GzzGw1IedwdQrnVJotW8ITDr171+2QIpKCGTM6xjrLaG8Pk+9AenNJ1PPJpPz5Ip56KnzPBIc6Bo1exVcpzN03AZNiyhcDp0Wf7wXek7D9GqBAw2ANZYbcNuuSw4tIhdJKMBczdmyY+rO9Pd1pRePEzRexZQucdVYIgmec0bE8EzSgJhMoqeez8gsijScpkVxugrmYsWPDfeLJJ9Pdb5ykoLZhA5x2WjqP4JZIgUH5BZHGM3NmOgnmYuqZgE4KaoMHJ29To0dpFRhUYxBpPFOnppNgLmbMmPBeSWAoNydw2mmdy/r2hUsuCX9fnLRrSJGqcwwNTdN6ijSuqVNr0r6eo18/GDas/MCQSSSXkxNYvhx22inUENauDTf9mTM71s/eH9SmhhRRjUGBQUQKyQyNUY6kRHJSTuDJJ+E3vwljOj39dOenqepVQ4ooMCjHICKFjB0b5nMoZ8ykcp+auvzycMM/44zkfab1CG4JFBhUYxCRQsaODbPA/f3vpW8zJGHIuLicwEsvhUdiTz4ZRoyo7BxT1tyBQTkGESmm3CeTXngB3nijc/+onXeOzwn84hfhXvS1r1V3nilq7sCgpiQRKaacJ5O2bw+//DdvDkNzZHICPXuGe80xx+Suv20b/OQnYXa5gw5K/9wrpMCgGoOIFLL77qGJp5TAcN55sGgRXHEF/L//15ETuPdeePVVOOWUMNxFxpw5sH59t6otQDMHBncFBhEprq0tDNKX6ZeQ1B/hxhvhhz8M80ufckrusvHj4eKLYd68kGiGcA/60Y/gwAPLm4+6Dpo3MGzdGt4VGEQkSaY/QmbAvkID2518Muy3H1x6afy+vvIVOO44OPvsMGprz56hFvK+93W78dqaNzBoyG0RKSapP8IXvxhqBqee2jEaqnvomPab38TvyyxMSeoOzz3X0aT0q1912fDaSRQYVGMQkSRJ/Q5eeSXkEbZtyy3furXwwHbf/W5ujgFqOhhepZoTG+DZAAALv0lEQVQ3MGhaTxEpJmksohEjkpt/Cg1sV6/hwqtUVWAwswFmttDMVkXv/WPWOcLMlma9Xjez46Nl15rZ37OWtVRzPmVRjUFEikkaxfV736ts6O96DRdepWprDOcAi9x9P2BR9D2Hu9/p7i3u3gJMBLYAf8xa5euZ5e6+tMrzKZ1yDCJSTKExiioZ+rtew4VXqdrAcBxwXfT5OuD4Iut/HLjN3bcUWa/2VGMQkVIkjVFUycB2dR4Mr1LVDrs9xN3XR5+fAxIGCPmnk4FL8spmmtn5RDUOd9/WebMaUI5BRKpVydDf9RguvEpFA4OZ3QHsGbMoJ43u7m5mHrNeZj9DCfM+L8gqPpcQUPoAs4BvAhclbD8dmA4wMo32ODUliYjEKhoY3H1y0jIze97Mhrr7+ujGv6HArj4BzHX3N7P2naltbDOzXwKJ/cLdfRYheNDa2poYgEqmpiQRkVjV5hjmAdOiz9OA3xdY91PAnOyCKJhgZkbITzxS5fmUToFBRCRWtYHh+8CRZrYKmBx9x8xazeyqzEpmNhoYAfw5b/s2M1sGLAMGAt+p8nxKpxyDiEisqpLP7r4JmBRTvhg4Lev7k8CwmPUmVnP8qmzZAr16QZ8+XXYKIiLdUfP2fNbIqiIisRQYREQkR/MGBk3rKSISq3kDg6b1FBGJ1dyBQTUGEZFOmjcwqClJRCRW8wYG1RhERGI1d2BQjkFEpJPmDgyqMYiIdNK8gUE5BhGRWM0bGNSUJCISqzkDQ3s7bN2qGoOISIzmDAyvvx7eFRhERDppzsCgIbdFRBI1Z2DQtJ4iIomaOzCoxiAi0knVgcHMTjKz5WbWbmatBdY72sweM7PVZnZOVvneZnZfVH6DmdV+5hwFBhGRRGnUGB4BPgbclbSCmfUE/gs4BhgDfMrMxkSLfwBc6u77Ai8Bp6ZwToUpxyAikqjqwODuK939sSKrjQdWu/sad38DuB44zswMmAjcFK13HXB8tedUlHIMIiKJ6pVjGAY8k/V9bVS2B/Cyu2/PK+/EzKab2WIzW7xx48bqzkZNSSIiiXqVspKZ3QHsGbNohrv/Pt1Tiufus4BZAK2trV7VzhQYREQSlRQY3H1ylcdZB4zI+j48KtsE9DOzXlGtIVNeW5kcg5qSREQ6qVdT0gPAftETSH2Ak4F57u7AncDHo/WmAbWvgajGICKSKI3HVU8ws7XABOBWM1sQle9lZvMBotrAl4AFwErgRndfHu3im8BZZraakHO4utpzKkqBQUQkUUlNSYW4+1xgbkz5s8CUrO/zgfkx660hPLVUP5s3Q69e0Lt3XQ8rItIImrfns/ILIiKxmjcwqBlJRCSWAoOIiORozsCgaT1FRBI1Z2BQjkFEJFHzBgbVGEREYikwiIhIjuYMDJs3qylJRCRBcwYG1RhERBIpMIiISA4FBhERydF8gaG9HbZuVY5BRCRB8wWGrVvDu2oMIiKxmi8waMhtEZGCFBhERCRHVYHBzE4ys+Vm1m5mrQnrjDCzO81sRbTuV7KWfcvM1pnZ0ug1JW4fqdK0niIiBVU7Uc8jwMeAXxRYZztwtrs/aGa7AUvMbKG7r4iWX+ruP6ryPEqnGoOISEFVBQZ3XwlgZoXWWQ+sjz6/ZmYrgWHAisSNakmBQUSkoLrmGMxsNHAwcF9W8ZfM7GEzu8bM+hfYdrqZLTazxRs3bqz8JDJNSQoMIiKxigYGM7vDzB6JeR1XzoHM7G3AzcCZ7v5qVHwFsA/QQqhV/Dhpe3ef5e6t7t46aNCgcg6dK1NjUI5BRCRW0aYkd59c7UHMrDchKLS5+2+z9v181jpXAn+o9lhFqSlJRKSgmjclWUhAXA2sdPdL8pYNzfp6AiGZXVsKDCIiBVX7uOoJZrYWmADcamYLovK9zGx+tNr7gc8CE2MeS73YzJaZ2cPAEcBXqzmfkuhxVRGRgqp9KmkuMDem/FlgSvT5HiD2sSV3/2w1x69Ipsawyy51P7SISCNozp7PvXuHl4iIdNKcgUH5BRGRRM0XGDStp4hIQc0XGFRjEBEpSIFBRERyKDCIiEiO5gsMyjGIiBTUfIFBNQYRkYIUGEREJEdzBgY1JYmIJGq+wLB5s2oMIiIFNF9gUFOSiEhBzRUY2tvh9dcVGERECmiuwKDZ20REimrOwKAag4hIomon6jnJzJabWbuZtRZY78loQp6lZrY4q3yAmS00s1XRe/9qzqcoBQYRkaKqrTE8AnwMuKuEdY9w9xZ3zw4g5wCL3H0/YFH0vXYUGEREiqoqMLj7Snd/rIpdHAdcF32+Dji+mvMpStN6iogUVa8cgwN/NLMlZjY9q3yIu6+PPj8HDKnpWajGICJSVNE5n83sDmDPmEUz3P33JR7ncHdfZ2aDgYVm9qi75zQ/ububmRc4j+nAdICRI0eWeNg8CgwiIkUVDQzuPrnag7j7uuh9g5nNBcYT8hLPm9lQd19vZkOBDQX2MQuYBdDa2poYQApSYBARKarmTUlmtquZ7Zb5DBxFSFoDzAOmRZ+nAaXWQCqjHIOISFHVPq56gpmtBSYAt5rZgqh8LzObH602BLjHzB4C7gdudffbo2XfB440s1XA5Oh77ajGICJSVNGmpELcfS4wN6b8WWBK9HkNcFDC9puASdWcQ1kUGEREimqens9tbXDRReHze94TvouISCdV1RgaRlsbTJ/eUWN4+unwHWDq1K47LxGRbqg5agwzZnQEhYwtW0K5iIjkaI7A8PTT5ZWLiDSx5ggMSR3iKu0oJyKyA2uOwDBzZucnkfr2DeUiIpKjOQLD1KkwaxaMGgVm4X3WLCWeRURiNMdTSRCCgAKBiEhRzVFjEBGRkikwiIhIDgUGERHJocAgIiI5FBhERCSHuVc2501XMrONwFNdfR51NhB4oatPopvQtQh0HQJdh6CU6zDK3QcV21FDBoZmZGaL3b21q8+jO9C1CHQdAl2HIM3roKYkERHJocAgIiI5FBgax6yuPoFuRNci0HUIdB2C1K6DcgwiIpJDNQYREcmhwNANmdk1ZrbBzB7JKhtgZgvNbFX03r8rz7EezGyEmd1pZivMbLmZfSUqb6prYWY7m9n9ZvZQdB0ujMr3NrP7zGy1md1gZn26+lzrwcx6mtnfzOwP0fdmvQ5PmtkyM1tqZoujslT+bSgwdE/XAkfnlZ0DLHL3/YBF0fcd3XbgbHcfAxwGfNHMxtB812IbMNHdDwJagKPN7DDgB8Cl7r4v8BJwaheeYz19BViZ9b1ZrwPAEe7ekvWYair/NhQYuiF3vwt4Ma/4OOC66PN1wPF1Paku4O7r3f3B6PNrhJvBMJrsWnjwj+hr7+jlwETgpqh8h78OAGY2HPgIcFX03WjC61BAKv82FBgaxxB3Xx99fg4Y0pUnU29mNho4GLiPJrwWUfPJUmADsBB4AnjZ3bdHq6wlBM0d3WXAN4D26PseNOd1gPDj4I9mtsTMpkdlqfzbaJ6JenYg7u5m1jSPk5nZ24CbgTPd/dXwIzFolmvh7m8BLWbWD5gLHNDFp1R3ZvavwAZ3X2JmH+rq8+kGDnf3dWY2GFhoZo9mL6zm34ZqDI3jeTMbChC9b+ji86kLM+tNCApt7v7bqLgprwWAu78M3AlMAPqZWebH3XBgXZedWH28HzjWzJ4Eric0IV1O810HANx9XfS+gfBjYTwp/dtQYGgc84Bp0edpwO+78FzqImo/vhpY6e6XZC1qqmthZoOimgJmtgtwJCHfcifw8Wi1Hf46uPu57j7c3UcDJwN/cvepNNl1ADCzXc1st8xn4CjgEVL6t6EObt2Qmc0BPkQYLfF54ALgd8CNwEjCyLKfcPf8BPUOxcwOB+4GltHRpnweIc/QNNfCzA4kJBJ7En7M3ejuF5nZOwi/nAcAfwM+4+7buu5M6ydqSvqau/9rM16H6G+eG33tBfza3Wea2R6k8G9DgUFERHKoKUlERHIoMIiISA4FBhERyaHAICIiORQYREQkhwKDiIjkUGAQEZEcCgwiIpLj/wPiXBik9DBJmQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "scores = [cross_val_score(regr, boston.data,\\\n",
    "                          boston.target,\\\n",
    "                          cv = int(i)).mean() \\\n",
    "          for i in range(3, 50)]\n",
    "plt.plot(range(3, 50), scores,'r-o')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:34.174960Z",
     "start_time": "2019-04-22T08:24:34.155764Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.45059442471362826"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X_scale = scale(boston.data)\n",
    "scores = cross_val_score(regr,data_X_scale, boston.target,\\\n",
    "                         cv = 7)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 使用天涯bbs数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:46.198546Z",
     "start_time": "2019-04-22T08:24:46.171912Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>link</th>\n",
       "      <th>author</th>\n",
       "      <th>author_page</th>\n",
       "      <th>click</th>\n",
       "      <th>reply</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>【民间语文第161期】宁波px启示:船进港湾人应上岸</td>\n",
       "      <td>/post-free-2849477-1.shtml</td>\n",
       "      <td>贾也</td>\n",
       "      <td>http://www.tianya.cn/50499450</td>\n",
       "      <td>194675</td>\n",
       "      <td>2703</td>\n",
       "      <td>2012-10-29 07:59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>宁波镇海PX项目引发群体上访 当地政府发布说明(转载)</td>\n",
       "      <td>/post-free-2839539-1.shtml</td>\n",
       "      <td>无上卫士ABC</td>\n",
       "      <td>http://www.tianya.cn/74341835</td>\n",
       "      <td>88244</td>\n",
       "      <td>1041</td>\n",
       "      <td>2012-10-24 12:41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         title                        link   author  \\\n",
       "0   【民间语文第161期】宁波px启示:船进港湾人应上岸  /post-free-2849477-1.shtml       贾也   \n",
       "1  宁波镇海PX项目引发群体上访 当地政府发布说明(转载)  /post-free-2839539-1.shtml  无上卫士ABC   \n",
       "\n",
       "                     author_page   click  reply              time  \n",
       "0  http://www.tianya.cn/50499450  194675   2703  2012-10-29 07:59  \n",
       "1  http://www.tianya.cn/74341835   88244   1041  2012-10-24 12:41  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('../data/tianya_bbs_threads_list.txt', sep = \"\\t\", header=None)\n",
    "df=df.rename(columns = {0:'title', 1:'link', 2:'author',3:'author_page', 4:'click', 5:'reply', 6:'time'})\n",
    "df[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:47.185301Z",
     "start_time": "2019-04-22T08:24:47.169337Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# 定义这个函数的目的是让读者感受到：\n",
    "# 抽取不同的样本，得到的结果完全不同。\n",
    "def randomSplit(dataX, dataY, num):\n",
    "    dataX_train = []\n",
    "    dataX_test = []\n",
    "    dataY_train = []\n",
    "    dataY_test = []\n",
    "    import random\n",
    "    test_index = random.sample(range(len(df)), num)\n",
    "    for k in range(len(dataX)):\n",
    "        if k in test_index:\n",
    "            dataX_test.append([dataX[k]])\n",
    "            dataY_test.append(dataY[k])\n",
    "        else:\n",
    "            dataX_train.append([dataX[k]])\n",
    "            dataY_train.append(dataY[k])\n",
    "    return dataX_train, dataX_test, dataY_train, dataY_test, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:48.122580Z",
     "start_time": "2019-04-22T08:24:48.081523Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variance score: 0.42\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# Use only one feature\n",
    "data_X = df.reply\n",
    "# Split the data into training/testing sets\n",
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplit(np.log(df.click+1), \n",
    "                                                                   np.log(df.reply+1), 20)\n",
    "# Create linear regression object\n",
    "regr = linear_model.LinearRegression()\n",
    "# Train the model using the training sets\n",
    "regr.fit(data_X_train, data_y_train)\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "print('Variance score: %.2f' % regr.score(data_X_test, data_y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:49.133689Z",
     "start_time": "2019-04-22T08:24:49.129343Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[12.179091917198399], [11.387872315966666], [11.323941765302724]]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X_train[:3]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:50.276495Z",
     "start_time": "2019-04-22T08:24:50.273286Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "y_true, y_pred = data_y_test, regr.predict(data_X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:51.151351Z",
     "start_time": "2019-04-22T08:24:50.992991Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAADT1JREFUeJzt3UGIo/d5x/HfT7MbEjWBwM4cjNfzvimUQJtDzAqXYujBEFhMaHvoIUH1KUVgCDi0UGp0ykHXkLMgpinzkhBwDsWkBEM3BEPiROPaIfamJQ3W1CWwE0JIFkFLsk8PO7vZXc+MXs3onVeP9P2AYEd6X73P/r3+8vJKIzkiBADIo9P2AACAxRBuAEiGcANAMoQbAJIh3ACQDOEGgGQINwAkQ7gBIBnCDQDJXGriSbe3t6MsyyaeGgDW0v7+/i8iYqfOto2EuyxLTSaTJp4aANaS7WndbblUAgDJEG4ASIZwA0AyhBsAkiHcAJAM4QaAZAg3AJxDVVUqy1KdTkdlWaqqqsaP2cj7uAFgE1RVpcFgoNlsJkmaTqcaDAaSpH6/39hxOeMGgDMaDof3o33PbDbTcDhs9LiEGwDO6ODgYKH7l4VwA8AZ7e7uLnT/stQOt+0t2/9u+5UmBwKALEajkbrd7kP3dbtdjUajRo+7yBn3C5JuNjUIAGTT7/c1Ho9VFIVsqygKjcfjRl+YlCRHxPyN7KuSvippJOnvIuLTp23f6/WCTwcEgPps70dEr862dc+4vyzpHyTdOfNUAIClmBtu25+WdCsi9udsN7A9sT05PDxc2oAAgIfVOeN+WtJf2H5X0tclPWN779GNImIcEb2I6O3s1PoSBwDAGcwNd0S8GBFXI6KU9BlJ/xYRf9P4ZACAY/E+bgBIZqHPKomI70j6TiOTAABq4YwbAJIh3ACQDOEGgGQIN9CwNj5oH+uNL1IAGtTWB+1jvXHGDTSorQ/ax3oj3ECD2vqgfaw3wg00qK0P2sd6I9xAg9r6oH2sN8INNKitD9rHeqv1RQqL4osUAGAxTXyRAgBgRRBuAEiGcANAMoQbAJIh3ACQDOEGgGQINwAkQ7gBIBnCDQDJEG4ASIZwA0AyhBsAkiHcAJAM4QaAZAg3ACRDuAEgGcINAMkQbgBIhnADQDKEGwCSIdwAkAzhBoBkCDcAJEO4ASAZwg0AyRBuAEiGcANAMoQbAJKZG27bH7T9A9tv2X7b9hcvYjAAwPEu1djmfyU9ExG3bV+W9Jrtf42I7zc8GwDgGHPDHREh6fbRj5ePbtHkUACAk9W6xm17y/abkm5JejUiXm92LADASWqFOyJ+FxGflHRV0lO2P/HoNrYHtie2J4eHh8ueEwBwZKF3lUTEryTdkHT9mMfGEdGLiN7Ozs6y5gMAPKLOu0p2bH/06M8fkvQpST9pejAAwPHqvKvkMUlftb2lu6H/RkS80uxYAICT1HlXyY8kPXkBswAAauA3JwEgGcINAMkQbgBIhnADQDKEGwCSIdwAkAzhBoBkCDcAJEO4ASAZwg0AyRBuAEiGcANAMoQbG6mqKpVlqU6no7IsVVVV2yMBtdX5WFdgrVRVpcFgoNlsJkmaTqcaDAaSpH6/3+ZoQC2ccWPjDIfD+9G+ZzabaTgctjQRsBjCjY1zcHCw0P3AqiHc2Di7u7sL3Q+sGsKNjTMajdTtdh+6r9vtajQatTQRsBjCjY3T7/c1Ho9VFIVsqygKjcdjXphEGo6IpT9pr9eLyWSy9OcFgHVlez8ienW25YwbAJIh3ACQDOEGgGQINwAkQ7gBIBnCDQDJEG4ASIZwA0AyhBsAkiHcAJAM4QaAZAg3ACRDuAEgGcINAMkQbgBIhnADQDKEGwCSIdwAkMzccNt+wvYN2+/Yftv2CxcxGADgeJdqbPNbSX8fEW/Y/oikfduvRsQ7Dc8GADjG3DPuiPh5RLxx9OffSLop6fGmBwMAHG+ha9y2S0lPSnq9iWEAAPPVDrftD0t6WdIXIuLXxzw+sD2xPTk8PFzmjACAB9QKt+3LuhvtKiK+edw2ETGOiF5E9HZ2dpY5IwDgAXXeVWJJX5F0MyK+1PxIAIDT1DnjflrSc5Kesf3m0e3ZhucCAJxg7tsBI+I1Sb6AWQAANfCbkwCQDOEGgGQI94aoqkplWarT6agsS1VV1fj+5z0mgBNExNJv165dC6yOvb296Ha7Ien+rdvtxt7eXmP7n/eYwKaRNImajfXd7Zer1+vFZDJZ+vPibMqy1HQ6fd/9RVHo3XffbWT/8x4T2DS29yOiV2tbwr3+Op2OjvvvbFt37txpZP/zHhPYNIuEm2vcG2B3d3eh+5ex/3mPCeBkhHsDjEYjdbvdh+7rdrsajUaN7X/eYwI4Rd2L4YvceHFy9ezt7UVRFGE7iqJY+EXCs+x/3mMCm0S8OAkAuXCNGwDWGOEGgGQINwAkQ7gBIBnCDQDJEG4ASIZwA0AyhBsAkiHcAJAM4QaAZAg3ACRDuJPh68AAXGp7ANRXVZUGg4Fms5kkaTqdajAYSJL6/X6bowG4QJxxJzIcDu9H+57ZbKbhcNjSRADaQLgTOTg4WOh+AOuJcCfC14EBkAh3KnwdGACJcKfS7/c1Ho9VFIVsqygKjcdjXpgENgxfXQYAK4CvLgOANUa4ASAZwg0AyRBuAEiGcANAMoQbAJIh3ACQDOEGgGQINwAkQ7gBIJm54bb9ku1btn98EQMBAE5X54z7nyRdb3gOAEBNc8MdEd+V9MsLmAUAUAPXuAEgmaWF2/bA9sT25PDwcFlPCwB4xNLCHRHjiOhFRG9nZ2dZTwsAeASXSgAgmTpvB/yapO9J+rjt92x/rvmxAAAnuTRvg4j47EUMAgCoh0slAJAM4QaAZAg3ACRDuAEgGcINAMkQbgBIhnBvgKqqVJalOp2OyrJUVVVtjwTgHOa+jxu5VVWlwWCg2WwmSZpOpxoMBpKkfr/f5mgAzogz7jU3HA7vR/ue2Wym4XDY0kQAzotwr7mDg4OF7gew+gj3mtvd3V3ofgCrj3CvudFopG63+9B93W5Xo9GopYkAnBfhXnP9fl/j8VhFUci2iqLQeDzmhUkgMUfE0p+01+vFZDJZ+vMCwLqyvR8RvTrbcsYNAMkQbgBIhnADQDKEGwCSIdwAkAzhBoBkCDcAJEO4ASAZwg0AyRBuAEiGcANAMoQbAJIh3ACQDOEGgGQINwAkQ7gBIBnCDQDJEG4ASIZwA0AyhBsAkiHcAJAM4QaAZAg3ACRDuAEgGcINAMnUCrft67b/w/ZPbf9jE4NUVaWyLNXpdFSWpaqqWujxZRzjtO23t7e1vb39vn2XMVfTf486+7Sxvpug6TVhzTdURJx6k7Ql6b8k/aGkD0h6S9Ifn7bPtWvXYhF7e3vR7XZD0v1bt9uNvb29Wo8v4xh1tn903+eff/7ccy3qLGuxiuu7CZpeE9Z8vUiaxJwe37vVCfefSfr2Az+/KOnF0/ZZNNxFURwbx6Ioaj2+jGPU3f7B29bW1rnnWtRZ1mIV13cTNL0mrPl6WSTcvrv9yWz/taTrEfG3Rz8/J+lPI+Lzj2w3kDSQpN3d3WvT6fTU531Qp9PRcXPY1p07d+Y+voxj1N2+jkXmWtRZ1mIV13cTNL0mrPl6sb0fEb062y7txcmIGEdELyJ6Ozs7C+27u7t76v3zHl/GMc7y3FtbW+eea1FnmXcV13cTNL0mrPkGm3dKrgu4VLKK12C5xs017vPiGjcWoSVf474k6WeSPqbfvzj5J6fts2i4I+7+IyyKImxHURTv+8c37/FlHOO07a9cuRJXrlx5377LmKvpv0edfdpY303Q9Jqw5utjkXDPvcYtSbaflfRl3X2HyUsRMTpt+16vF5PJZO7zAgDuWuQa96U6G0XEtyR961xTAQCWgt+cBIBkCDcAJEO4ASAZwg0AyRBuAEim1tsBF35S+1BS/d95X23bkn7R9hArjjWqh3WqZ1PXqYiIWr923ki414ntSd33Vm4q1qge1qke1mk+LpUAQDKEGwCSIdzzjdseIAHWqB7WqR7WaQ6ucQNAMpxxA0AyhPsEF/EFydnZfsn2Lds/bnuWVWb7Cds3bL9j+23bL7Q906qx/UHbP7D91tEafbHtmVYZl0qOYXtL0n9K+pSk9yT9UNJnI+KdVgdbMbb/XNJtSf8cEZ9oe55VZfsxSY9FxBu2PyJpX9Jf8e/p92xb0h9ExG3blyW9JumFiPh+y6OtJM64j/eUpJ9GxM8i4v8kfV3SX7Y808qJiO9K+mXbc6y6iPh5RLxx9OffSLop6fF2p1otR98lcPvox8tHN84qT0C4j/e4pP9+4Of3xP9oWALbpaQnJb3e7iSrx/aW7Tcl3ZL0akSwRicg3MAFsf1hSS9L+kJE/LrteVZNRPwuIj4p6aqkp2xz+e0EhPt4/yPpiQd+vnp0H3AmR9dtX5ZURcQ3255nlUXEryTdkHS97VlWFeE+3g8l/ZHtj9n+gKTPSPqXlmdCUkcvvH1F0s2I+FLb86wi2zu2P3r05w/p7hsDftLuVKuLcB8jIn4r6fOSvq27LyR9IyLebneq1WP7a5K+J+njtt+z/bm2Z1pRT0t6TtIztt88uj3b9lAr5jFJN2z/SHdPnF6NiFdanmll8XZAAEiGM24ASIZwA0AyhBsAkiHcAJAM4QaAZAg3ACRDuAEgGcINAMn8P7Lcj2jEg96EAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(y_pred, y_true,  color='black')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:52.301659Z",
     "start_time": "2019-04-22T08:24:52.130224Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAH0hJREFUeJzt3Xmck9W9x/HPGZAlKKACbkiiVGQRRZiLO3UvLtXe2lbtXHu11lGLG6AIjnUfFfdal3YqIu1ErbhUq7hdUVyLDgIC4oLLUFALVAFl2GR+948zYyYgkGSSefIk3/frlZdwnuTJLxG+nDnPec5xZoaIiIRHSdAFiIhIehTcIiIho+AWEQkZBbeISMgouEVEQkbBLSISMgpuEZGQUXCLiISMgltEJGRa5+KkXbp0sVgslotTi4gUpGnTpi0xs66pPDcnwR2LxaipqcnFqUVECpJzrjbV52qoREQkZBTcIiIho+AWEQkZBbeISMgouEVEQkbBLSISMgpuEZFmisfjxGIxSkpKiMVixOPxnL5fTuZxi4gUi3g8Tnl5OXV1dQDU1tZSXl4OQFlZWU7eUz1uEZFmqKio+C60G9XV1VFRUZGz91Rwi4g0w/z589NqzwYFt4hIM/To0SOt9mxIObidc62cc9Odc0/mrBoRkZCprKwkEokktUUiESorK3P2nun0uM8H5uaqEBGRMCorK6OqqopoNIpzjmg0SlVVVc4uTAI4M9v8k5zrDkwAKoERZnbspp5fWlpqWh1QRCR1zrlpZlaaynNT7XHfBowC6jfxpuXOuRrnXM3ixYtTPK2IiKRrs8HtnDsWWGRm0zb1PDOrMrNSMyvt2jWltcBFRCQDqfS4DwCOc859CjwIHOqcq85pVSIislGbDW4zG2Nm3c0sBpwETDaz/8l5ZSIi8r00j1tEJGTSWqvEzF4CXspJJSIikhL1uEVEQkbBLZJjLb3kpxQ+LesqkkNBLPkphU89bpEcCmLJTyl8Cm6RHApiyU8pfApukRwKYslPKXwKbpEcCmLJTyl8Cm6RHApiyU8Jzrp1LfM+KS3rmi4t6yoixeRf/4JRo2DrreGuuzI7RzrLumo6oIhIhlauhBtvhOuv978uKYEzz4S99srt+2qoREQkTWYwcSL07g2XX+5DG6C+Hp56Kvfvrx63iEgaZs6E88+HKVOS2wcMgNtvh4MOyn0N6nGLiKRgyRI46ywYODA5tLt0gaoqqKlpmdAG9bhFRDZp7Vp/wfGKK2Dp0kR769Zw7rlw2WXQuXPL1qTgFhHZiOeegwsugLlzk9t/9CO49Vbo0yeYujRUIiKynnnz4PjjfUA3De0f/AD+8Q94+ungQhsU3CIi3/n6a7j4YujbF554ItG+1VZ+2t+cOXDsseBccDWChkpERKivh7/8BcaMgS++SLQ7B6edBpWVsP32wdW3PgW3iBS1f/4TzjsP3noruX2//fz0vtKU7mVsWRoqEZGi9NlncMopPqCbhvaOO0J1Nbz2Wn6GNqjHLSJFZtUquOUWuPZaWLEi0d62LVx4IYweDVtuGVx9qVBwi0hRMIO//x1GjoRPPkk+dsIJ/uLjLrsEU1u6FNwiUvBmz/bzsV94Ibl9jz3g97+HQw8Npq5MaYxbRArWl1/COef41fqahvY228Cdd8L06eELbVCPW0QK0Lffwp/+5G9H//LLRHurVnD22XDllT68w0rBLSIFZfJkv3rf7NnJ7YcdBrfd5odHwk5DJSJSED75xF9kPOyw5NDeZRd47DF4/vnCCG1Qj1tEQu6bb+C66+Dmm2H16kR7hw5QUQHDh0O7dsHVlwsKbhEJJTOIx/3aIp99lnzslFP8dmI77hhMbbmm4BaR0HnrLT+O/cYbye2DB/vpffvuG0xdLUVj3CISGl984Rd9Gjw4ObS33x7uu8+3FXpog3rcIhICq1f7nvQ11/ilVxu1aePHsCsq/NKrxULBLSJ5ywyefBJGjPCbGzR13HH+guQPfhBMbUHabHA759oBLwNtG57/sJldnuvCRKS4zZ3re9PPPpvc3qePn4995JHB1JUPUhnjXg0camZ7AQOAoc65IhhFEpEgLF3q1xXp3z85tDt39sMlM2cWd2hDCj1uMzPgm4bfbtHwsFwWJSLFZ906uOceuPRSWLIk0V5SAuXlcNVV0LVrcPXlk5TGuJ1zrYBpwA+AO81sak6rEpGi8vLLfheamTOT23/4Q9/L3muvYOrKVylNBzSzdWY2AOgODHbObXDjqHOu3DlX45yrWbx4cbbrFJECVFsLJ57oA7ppaEejMHEivPiiQvv7pDWP28yWAi8CQ7/nWJWZlZpZaVf9PCMim1BXB5dfDr17w0MPJdrbt/dDInPnws9+Fvxu6vkqlVklXYG1ZrbUOdceOAIYm/PKRKTgmMHf/gajRsG//pV87OSTYexY2HnnYGoLk1TGuHcAJjSMc5cAD5nZk7ktS0QKzdtv+9vUX301uX3gQD+OfeCBwdQVRqnMKnkH2LsFahGRArRokb+zcdw43+Nu1K2b37D31FP9BgeSOt05KSI5sWYN3HGH321m+fJEe+vWvuf9u99Bp07B1RdmCm4Rybqnn/Z3Pb7/fnL70UfDLbfA7rsHU1ehUHCLSNZ88IFfV+Spp5Lbe/WCW2/1wS3Np2VdRaTZli2DCy/0W4M1De2OHf1CULNmKbSzST1uEclYfT2MHw+XXOIvQjZyDk4/3S/Dut12wdVXqBTcIpKR117zFxmnTUtuP+AAP71v0KBg6ioGGioRkbQsWABlZX7eddPQ7t4d7r8fXnlFoZ1r6nGLSEpWrvTj1ddd529Zb9Sunb8TctQov7O65J6CW0Q2yQwefdRffPz00+RjP/853HADxGJBVFa8FNwislHvvOPHsV96Kbl9zz39OPbBBwdRlWiMW0Q2sGQJ/Pa3sPfeyaG97bZw991+3RGFdnAU3FKU4vE4sViMkpISYrEY8Xg86JLywtq1cPvtsNtuPqDr6317q1a+5/3hh3DWWVpbJGgaKpGiE4/HKS8vp67hClttbS3l5eUAlJWVBVlaoJ5/3u/1+O67ye1HHOE35+3bN5i6ZEPqcUvRqaio+C60G9XV1VFRURFQRcH66CP4yU/8BrxNQ7tnT3j8cb9hr0I7vyi4pejMnz8/rfZC9fXXMGaMD+XHH0+0b7ml39Bgzhw47jjtQpOPFNxSdHr06JFWe6Gpr4e//MWv0Hf99X751UannuoXiho1Ctq2DaxE2QwFtxSdyspKIpFIUlskEqGysjKgilrO1Kmw337wv/8Ln3+eaN9nH39s/HjYYYfg6pPUKLil6JSVlVFVVUU0GsU5RzQapaqqqqAvTH72mQ/rffeFN99MtO+wg+99v/46DB4cXH2SHmdN9xLKktLSUqupqcn6eUUkPatW+Rkh11wDK1Yk2tu08XdCjhnjx7QleM65aWZWmspzNR1QpACZwRNP+E0NPv44+dhPfuLXHNl112Bqk+ZTcIsUmDlz/Hzs//u/5PZ+/Xzv+/DDg6lLskdj3CIF4ssv4bzzYK+9kkN7663hD3+AGTMU2oVCPW6RkPv2W/jzn/2u6f/5T6K9pATOPtvvsr7ttsHVJ9mn4BYJsRdf9MMi77yT3H7IIX71vv79g6lLcktDJSIh9Mkn8LOfwaGHJod2LAaPPAIvvKDQLmTqcYuEyIoV/m7HG2+E1asT7ZGI37B35Ei/I40UNgW3SAiYwQMP+FvRFy5MPlZW5tcW2WmnYGqTlqfgFslzNTV+LezXX09uLy3149j77x9MXRIcjXGL5KkvvoDTT/e3ojcN7e22g3vv9WuLKLSLk3rcInlmzRq/C81VV/mlVxttsYWfQXLppdCxY3D1SfAU3CJ5wgwmTYLhw/0WYU0de6y/Tb1Xr2Bqk/yi4BbJA++95wP7mWeS23v3hltvhaFDg6lL8pPGuEUCtHSpXwiqf//k0O7UyQf2O+8otGVD6nGLBGDdOn+BsaICFi9OtDsHZ5wBV18N3boFV5/kt80Gt3NuZ+AvwHaAAVVm9vtcFyZSqF55xU/vmz49uf2gg/z0vr33DqYuCY9Uhkq+BUaaWV9gX2CYc057Poukaf58OOkkGDIkObR33hkefBCmTFFoS2o22+M2s8+Bzxt+/bVzbi6wE/BujmsTKQh1df4W9bFjYeXKRHu7dnDxxf5uyPW2wBTZpLTGuJ1zMWBvYGouihEpJGYwcSJcdJHvbTd14olwww1QJBvLS5alHNzOuS2BR4ALzGz59xwvB8oBeuhPoxS56dP9OPYrryS3Dxjgx7GHDAmmLikMKU0HdM5tgQ/tuJk9+n3PMbMqMys1s9KuXbtms0aR0Fi8GM48EwYNSg7tLl2gqsqvO6LQluZKZVaJA8YBc83sltyXJBI+a9fCnXfCFVfAsmWJ9tat4dxz4bLLoHPnwMqTApPKUMkBwCnALOfcjIa2S8xsUu7KEgmPZ5/1a4i8915y+9Ch/iaa3r2DqUsKVyqzSl4FXAvUIhIqH37o73p88snk9t1284F9zDHB1CWFT7e8i6Rp+XI/ha9fv+TQ3moruOkmmD1boS25peAuEvF4nFgsRklJCbFYjHg8nvPXN/c98019PYwf71fou/FGP64N/jb1X//a98BHjoQ2bYKtU4qAmWX9MWjQIJP8UV1dbZFIxPBLFhhgkUjEqqurc/b65r5nvnn9dbPSUjM/Ozvx2H9/s7feCro6KQRAjaWYsc4/P7tKS0utpqYm6+eVzMRiMWprazdoj0ajfPrppzl5fXPfM18sXAijR0N1dXL7Tjv5G2hOPtn3uEWayzk3zcxKU3qugrvwlZSU8H3/n51z1NfX5+T1zX3PoK1a5TcuuPZaf8t6o7Zt/Z2Qo0dDhw7B1SeFJ53g1hh3EdjYnayp3uGayeub+55BMYNHH4U+ffwWYU1D+4QT/JS/q69WaEuwFNxFoLKyksh6qxhFIhEqKytz9vrmvmcQZs2Cww/3Ad10NKd/f5g8GR5+GGKxoKoTaSLVwfB0Hro4mX+qq6stGo2ac86i0WjaFwkzeX1z37OlLFliNmyYWUlJ8oXHbbc1u/tus7Vrg65QigG6OCmyed9+C3/8o78d/auvEu2tWsFvf+tvX99mm8DKkyKTzhi3ti6TovTCC/429dmzk9sPPxxuu83fXCOSrzTGLUXl44/hpz/1Ad00tHfdFf7+d3juOYW25D/1uKUofPMNXHedn+K3enWivUMHP3tk+HA/1U8kDBTcUtDq6yEe91uEff558rFf/cqH+Y47BlObSKYU3FKw3nzT70Lzz38mtw8eDLffDvvsE0xdIs2lMW4pOJ9/Dqed5oO5aWhvvz1MmABvvKHQlnBTj1sKxurVfj/Hq6/2Y9qN2rTx62ZfcolfelUk7BTcEnpmfl3sESNg3rzkY8cf7y9I9uwZTG0iuaDgllCbO9fPx37uueT2vn39fOwjjgimLpFc0hh3yBTa5gSZ+uorH9j9+yeHdufO/sLjjBkKbSlc6nGHSDwep7y8nLqGJetqa2spLy8HoKysLMjSWsy6dXDPPX7u9ZIlifaSEjjzTLjqKujSJbj6RFqC1ioJkULZnCBTU6b46X0zZya3H3ywvyi5556BlCWSFVqPu0DNnz8/rfZCUVsLv/iFD+imoR2N+qVWJ09WaEtxUXCHSFg3J8hUXR1cfjn07g0TJybaIxE/5W/uXL92trYOk2Kj4A6RMG5OkAkzePBB2H13P2a9alXi2C9/Ce+/78e427cPrkaRICm4Q6SsrIyqqiqi0SjOOaLRKFVVVQV1YfLtt2HIEL8J74IFifaBA+HVV/26I927B1efSD7QxUnJC4sWQUUFjBvne9yNunXzC0GdeqqfOSJSqLSRgoTGmjVwxx1w5ZWwfHmivXVrP4Pkd7+DTp2Cq08kHym4JTBPP+3XwX7//eT2Y47xt6nvvnswdYnkOwW3tLgPPvDrijz1VHJ7r17+NvWjjgqmLpGw0KihtJhly+Cii2CPPZJDu2NH38OeNUuhLZIK9bgl5+rr4b77YMwYfxGykXNw+ulQWekvQopIahTcklOvvw7nnQfTpiW3H3igv0194MBg6hIJMw2VSE4sWABlZXDAAcmh3b07PPAAvPyyQlskU+pxS1atXAk33QTXX+9vWW/Urh2MGuUfHToEV59IIdhscDvn7gWOBRaZ2R65L0nCyAweeQQuvNAvCtXUz38ON9wAsVggpYkUnFSGSu4Dhua4DgmxmTPhkEN8QDcN7b32gpdegoceUmiLZNNmg9vMXga+bIFaJGSWLIGzz/Zj1VOmJNq33Rb++Ec/tv3DHwZXn0ih0hi3pG3tWrj7br/k6tKlifZWreCcc3z71lsHV59IoctacDvnyoFyKNz1oQWef97v9fjuu8ntRx4Jt97qN+kVkdzK2nRAM6sys1IzK+3atWu2Tit5Yt48OP54H9BNQ7tnT3jiCXjmGYW2SEvRPG7ZpK+/htGjoV8/H9CNttwSxo6FOXPgxz/WLjQiLWmzwe2cewB4A9jdObfAOXd67suSoNXXw4QJfuGnsWP98quNTj0VPvzQz8lu2zawEkWK1mbHuM3s5JYoRPLH1Kn+NvU330xu33dfuP12+K//CqYuEfE0VCLf+ewz+NWvfEA3De0dd4S//hVee02hLZIPNB1QWLXKr4N9zTWwYkWivU0bfyfkmDF+TFtE8oOCu4iZweOPw8iR8PHHycf++7/9miO77hpMbSKycQruIjVnjt/T8YUXktv79fPLrR52WDB1icjmaYy7yHz5JZx7rl9HpGlob72137R3xgyFtki+U3AXgXg8TjTaE+eG0bXrV9xxB6xb54+VlMCwYX5637Bhfnd1Eclv+mta4OLxOKefXs3q1Y8Ce1Ffnzh2yCF+WKR//8DKE5EMKLgL2CefQHn51qxe/fT6R+ja9XpeeOFPuuNRJIQ0VFKAVqyASy+FPn2gru7opkeACqAPS5b8WaEtElLqcRcQM7j/frj4Yli4cP2jfwVGA58B0KNHtIWrE5FsUXAXiJoaP73v9deT23fddQkLF/6c1atf+q4tEolQWVnZsgWKSNZoqCTkvvgCfv1rGDw4ObS32w7Gj4cPP+zCuHG/IRqN4pwjGo1SVVVFWVlZcEWLSLM4M8v6SUtLS62mpibr55WENWv8jJCrr/ZLrzbaYgsYPhwqKqBjx+DqE5H0OOemmVlpKs/VUEnImMFTT8GIEX7udVM//jHcfDPstlswtYlIy1Bwh8h77/ne9DPPJLf36eO3DfvRj4KpS0Ralsa4Q2DpUt/D7t8/ObQ7dfKr+s2cqdAWKSbqceexdevg3nv9ePXixYl25+CMM/wyrNreU6T4KLjz1Cuv+Ol906cntx90kN+FZsCAYOoSkeBpqCTPzJ8PJ50EQ4Ykh/bOO8Pf/gZTpii0RYqdetx5oq4ObrzRb8y7cmWivX17fyfkRRdBJBJcfSKSPxTcATODiRN9MM+fn3zsxBPhhhugR49gahOR/KTgDtCMGX4c++WXk9sHDPDj2AcdFExdIpLfNMYdgMWL4ayzYNCg5NDu0gWqqvy6IwptEdkY9bhb0Nq1cOedcMUVsGxZor11a7+d2GWXQefOgZUnIiGh4G4hzz4LF1zg735sauhQf9dj797B1CUi4aOhkhz78EM47jgf0E1De7fd4MknYdIkhbaIpEfBnSPLl/tpfP36wT/+kWjfais/7W/2bDjmGLQLjYikTUMlWVZfDxMmwJgx8O9/J9qdg9NOg2uv9Wtli4hkSsGdRW+8Aeed52eFNLX//n7t7NKUVtoVEdk0DZVkwcKFcMopPqCbhvZOO0E8Dq++qtAWkexRj7sZVq2CW27xwx8rViTa27b1d0KOHg0dOgRXn4gUJgV3Bszgscdg5Ej49NPkYyecADfdBLFYEJWJSDFQcKdp1iw/H3vy5OT2/v39OPYhhwRTl4gUD41xp+g//4Fhw/w6Ik1De5tt4K674O23Fdoi0jJSCm7n3FDn3PvOuXnOudG5KCQejxOLxSgpKSEWixGPx9M63tzzb+w10WhPnDuHLl2+5K67/HQ/gFat/G3q1147kbFjY7Rpk1ldmcj0s+Ty+83WOQpNrr8TfedFysw2+QBaAR8BuwJtgJlA3029ZtCgQZaO6upqi0QiBnz3iEQiVl1dndLx5p5/Y69p2/Zog1nmR7UTj5KSyXbddf9odl2ZyPSz5PL7zdY5Ck2uvxN954UFqLHN5HHjI5Xg3g94tsnvxwBjNvWadIM7Go0m/eFrfESj0ZSON/f86/voI7P27Z/ZILBhnsHx3722uXVlIpP3zPX3m61zFJpcfyf6zgtLOsHt/PM3zjn3M2Comf2m4fenAPuY2TnrPa8cKAfo0aPHoNra2k2et6mSkhK+rw7nHPX19Zs93tzzN/rmGz+17+abYc2aps/8BrgGuBVY891rgWbVlYlMvotcf7+Z1lXocv2d6DsvLM65aWaW0h0fWbs4aWZVZlZqZqVd09x6vMdGtnhpbN/c8eaev74e/vpX6NULrrtu/dCeAPQCxtIY2o2vbW5dmcjkPXP9/WbrHIUm19+JvvMitrkuOS0wVBLkGPfUqWb77rv+kIhZz56LrW3bId/7o2jjazXG3by6Cp3GuCUdZHmMuzXwMbALiYuT/Tb1mnSD28z/IYxGo+acs2g0usEfvs0dT/f8f/jDw3bqqRsG9vbbm02YYLZuXeI1gLVq1eq78cOm793cujKRyXvm+vvN1jkKTa6/E33nhSOd4N7sGDeAc+5o4Db8DJN7zaxyU88vLS21mvVXWsoTq1fDbbfBNdf4Me1GbdrAiBFwySV+6VURkZaUzhh3SndOmtkkYFKzqgqYmd+4YMQImDcv+djxx/sLkj17BlObiEg6iuKW97lzYfhwv31YU337+t73EUcEU5eISCYK+pb3r77y64r0758c2p07+3VFZsxQaItI+BRkj3vdOrjnHrj0UliyJNFeUgJnnglXXQVdugRXn4hIcxRccE+ZAuefDzNnJrcffLDvZe+5ZyBliYhkTcEMldTWwi9+4QO6aWhHo/Dww35FP4W2iBSC0Pe4V6yAG27wj1WrEu2RiN+wd+RIaN8+uPpERLIttMFtBg8+CKNGwYIFycd++UsYOxa6dw+mNhGRXAplcL/9tt9N/bXXktsHDoTbb4cDDgimLhGRlhCqMe5Fi+CMM/yO6U1Du1s3GDcO3npLoS0ihS8UPe41a+COO+DKK2H58kT7Flv4GSSXXgqdOgVXn4hIS8r74J40yd/1+MEHye3HHAO33OKXYhURKSZ5G9zvv+/XFZm03gopvXr529SPOiqYukREgpZ3Y9zLlvkpfHvskRzaHTv6haBmzVJoi0hxy5se97p1MH68X1Z18eJEu3Pwm9/4ZVi7dQuuPhGRfJE3wX3BBf4CZFMHHuhvUx84MJiaRETyUd4MlZx1FrRq5X+9887+5pqXX1Zoi4isL2963P36JW5PHzXK37IuIiIbypvgBn+buoiIbFreDJWIiEhqFNwiIiGj4BYRCRkFt4hIyCi4RURCRsEtIhIyCm4RkZBxZpb9kzq3GKjN8mm7AEuyfM4gFMrnAH2WfFQonwOK77NEzaxrKifLSXDngnOuxsxKg66juQrlc4A+Sz4qlM8B+iyboqESEZGQUXCLiIRMmIK7KugCsqRQPgfos+SjQvkcoM+yUaEZ4xYRES9MPW4RESHPg9s5184596ZzbqZzbo5z7sqga2ou51wr59x059yTQdfSHM65T51zs5xzM5xzNUHXkynnXGfn3MPOufecc3Odc/sFXVMmnHO7N/y/aHwsd85dEHRdmXLODW/4Oz/bOfeAc65d0DVlwjl3fsNnmJPN/x95PVTinHNABzP7xjm3BfAqcL6Z/TPg0jLmnBsBlAIdzezYoOvJlHPuU6DUzEI9z9Y5NwF4xczucc61ASJmtjTouprDOdcKWAjsY2bZvp8i55xzO+H/rvc1s5XOuYeASWZ2X7CVpcc5twfwIDAYWAM8A5xlZvOae+687nGb903Db7doeOTvvzSb4ZzrDhwD3BN0LQLOuU7AEGAcgJmtCXtoNzgM+CiMod1Ea6C9c641EAE+C7ieTPQBpppZnZl9C0wBfpqNE+d1cMN3QwszgEXA82Y2NeiamuE2YBRQH3QhWWDAc865ac658qCLydAuwGJgfMPw1T3OuQ5BF5UFJwEPBF1EpsxsIXATMB/4HFhmZs8FW1VGZgMHOee2dc5FgKOBnbNx4rwPbjNbZ2YDgO7A4IYfP0LHOXcssMjMpgVdS5YcaGYDgaOAYc65IUEXlIHWwEDgbjPbG1gBjA62pOZpGO45DpgYdC2Zcs5tDRyP/4d1R6CDc+5/gq0qfWY2FxgLPIcfJpkBrMvGufM+uBs1/Aj7IjA06FoydABwXMPY8IPAoc656mBLylxDrwgzWwQ8hh/HC5sFwIImP8U9jA/yMDsKeNvM/h10Ic1wOPCJmS02s7XAo8D+AdeUETMbZ2aDzGwI8BXwQTbOm9fB7Zzr6pzr3PDr9sARwHvBVpUZMxtjZt3NLIb/UXaymYWuFwHgnOvgnNuq8dfAkfgfC0PFzL4A/uWc272h6TDg3QBLyoaTCfEwSYP5wL7OuUjDBIXDgLkB15QR51y3hv/2wI9v35+N8+bVLu/fYwdgQsNV8hLgITML9TS6ArEd8Jj/O0Vr4H4zeybYkjJ2LhBvGGL4GDgt4Hoy1vCP6BHAmUHX0hxmNtU59zDwNvAtMJ3w3kX5iHNuW2AtMCxbF7/zejqgiIhsKK+HSkREZEMKbhGRkFFwi4iEjIJbRCRkFNwiIiGj4BYRCRkFt4hIyCi4RURC5v8BFawCipP+jrAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot outputs\n",
    "plt.scatter(data_X_test, data_y_test,  color='black')\n",
    "plt.plot(data_X_test, regr.predict(data_X_test), color='blue', linewidth=3)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:53.326537Z",
     "start_time": "2019-04-22T08:24:53.321437Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('Coefficients: \\n', array([0.68623605]))"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The coefficients\n",
    "'Coefficients: \\n', regr.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:55.007412Z",
     "start_time": "2019-04-22T08:24:55.002637Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Residual sum of squares: 0.98'"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The mean square error\n",
    "\"Residual sum of squares: %.2f\" % np.mean((regr.predict(data_X_test) - data_y_test) ** 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:24:55.875656Z",
     "start_time": "2019-04-22T08:24:55.846855Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/datalab/Applications/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:1: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access\n",
      "  if __name__ == '__main__':\n",
      "/Users/datalab/Applications/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:2: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access\n",
      "  from ipykernel import kernelapp as app\n"
     ]
    }
   ],
   "source": [
    "df.click_log = [[np.log(df.click[i]+1)] for i in range(len(df))]\n",
    "df.reply_log = [[np.log(df.reply[i]+1)] for i in range(len(df))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:25:13.823742Z",
     "start_time": "2019-04-22T08:25:13.811227Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Variance score: 0.62'"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "Xs_train, Xs_test, y_train, y_test = train_test_split(df.click_log, df.reply_log,test_size=0.2, random_state=0)\n",
    "\n",
    "# Create linear regression object\n",
    "regr = linear_model.LinearRegression()\n",
    "# Train the model using the training sets\n",
    "regr.fit(Xs_train, y_train)\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "'Variance score: %.2f' % regr.score(Xs_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:25:18.210290Z",
     "start_time": "2019-04-22T08:25:18.010690Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAHFVJREFUeJzt3XtwXVW9B/Dv7+TRNk15mERohezTKnKpvLQZLspjhIKilId3qFMmXHl4PZCgoniHATPcOzJmuHeGAYRR9AwIOD0iUGUuUsVyAa3CnWIKiIVWaWsSWoGkOLS0AUp7fvePnYTk5OzHOXud/Trfz8wZkrPXWWtlt3y7s9bae4mqgoiI0iMTdQeIiMgsBjsRUcow2ImIUobBTkSUMgx2IqKUYbATEaUMg52IKGUY7EREKcNgJyJKmcYoGm1vb9dsNhtF00REibV+/fodqtrhVS6SYM9msxgYGIiiaSKixBKRIT/lOBRDRJQyDHYiopRhsBMRpQyDnYgoZQIHu4gcKSLPT3ntEpFvmOgcERFVLvCqGFX9C4DjAUBEGgBsB/BQ0HqJiKg6podilgLYoqq+luQQEdWDQqGAbDaLTCaDbDaLQqFQ0/ZMr2NfAeC+cgdEJAcgBwCdnZ2GmyUiiqdCoYBcLoexsTEAwNDQEHK5HACgu7u7Jm2KqT1PRaQZwN8BfExVX3cr29XVpbxBiYjqQTabxdDQzEEMy7IwODhYUV0isl5Vu7zKmRyK+RyAZ71CnYiongwPD1f0vgkmg/1COAzDEBHVK6eh51oOSRsJdhGZC+BMAL8wUR8RUVr09/ejpaVl2nstLS3o7++vWZtGgl1V96hqm6ruNFEfEVFadHd3I5/Pw7IsiAgsy0I+n6/ZxClgcPK0Epw8JSKqXBSTp0REFAMMdiKilGGwExGlDIOdiChlGOxERCnDYCciShkGOxFRyjDYiYhShsFORJQyDHYiopRhsBMRpQyDnYgoZRjsREQpw2AnIkoZBjsRUcqY2kHpIBFZJSKbRGSjiHzSRL1ERFS5RkP1fA/Ao6p6gYg0A2jx+gAREdVG4GAXkQMBnArgEgBQ1b0A9gatl4iIqmNiKGYhgFEAd4vIcyJy5/jm1kREFAETwd4I4BMA7lDVjwPYA+Da0kIikhORAREZGB0dNdAsERGVYyLYtwHYpqrrxr9fBTvop1HVvKp2qWpXR0eHgWaJiKicwMGuqq8BeEVEjhx/aymAl4LWS0RE1TG1KuZrAArjK2K2ArjUUL1ERFQhI8Guqs8D6DJRFxERBcM7T4mIUobBTkSUMgx2IqKUYbATEaUMg52IKGUY7EREKcNgJyJKGQY7EVHKMNiJiFKGwU5ElDIMdiKilGGwExHV2KZNQFcX0NsLjIzUvj1TT3ckIqISW7cCH/7w+9+vXw+oAnfcUdt2GexERIa98gqQzQLF4sxjxx5b+/Y5FENEZMjrrwMHHgh0dpYP9YsuAq64ovb94BU7EVFAb7wBHH008Npr5Y+fey6wahXQ1BROfxjsRERV2rnTnhTdvLn88aVLgdWrgVmzwu2XkWAXkUEAbwHYD2CfqnI3JSJKrd27gZNOAl54ofzxE08EnngCmDMn3H5NMHnFfpqq7jBYHxFRrLz9NnDGGcDTT5c/fuyxwFNPAa2t4farFIdiiIg87N0LLFsGPPZY+eOLFtlLGQ86KNx+OTG1KkYBrBGR9SKSM1QnEVGk9u0DvvAFe4y8XKgfcggwOgps2RKfUAfMXbGfrKrbReSDAB4TkU2qunZqgfHAzwFAZ2enoWaJiMzbvx/40peAn/60/PHWVuCvfwXmzw+3X34ZuWJX1e3j/x0B8BCAE8qUyatql6p2dXR0mGiWiMgoVeDyy4HGxvKhnskAw8PAW2/FN9QBA8EuInNFZN7E1wA+A2BD0HqJiMKiClx9tR3c+Xz5Mps321fyhx8ebt+qYeKK/RAAfxCRPwF4BsBqVX3UQL1ERGUVCgVks1lkMhlks1kUCoWq67r+ejvQb7ml/PGXXrKDf+ozX+Iu8Bi7qm4FcJyBvhAReSoUCsjlchgbGwMADA0NIZez12x0d3f7rufGG4Fvf9v5+PPPA8clNNlEVUNvtKurSwcGBkJvl4iSL5vNYmhoaMb7lmVhcHDQ8/O33QZcdZXz8XXrgBNmzBLGg4is93MDKNexE1GiDA8PV/T+hDvvBL7yFefja9cCp5wSpGfxwac7ElGiOC2Xdnq/UABEnEN9zRp7DD0toQ4w2IkoYfr7+9HS0jLtvZaWFvT390977+c/twP9oovK1/Pww3agn3lmrXoaHQY7ESVKd3c38vk8LMuCiMCyLOTz+cmJ01/9yg70Cy4o//kHHrAD/ZxzQux0yBjsRHXO5NLBsHR3d2NwcBDFYhGDg4Po7u7Gk0/agX722eU/c889dqAvXx5qVyPByVOiOmZq6WCUnn7afoSukx/8AOjpCa8/ccArdkq8JF5xxkVfX99kqE8YGxtDX19fRD3yb/16+wrdKdRvusm+Qq+3UAcY7JRwE1ecQ0NDUNXJK06Guz/VLh2M0v3324He5bCa+4Yb7ED/1rfC7VecMNgp0eJwxZnk3xgqXToYpdWr7UBfsaL88WuvtTeQvv76cPsVRwx2SrSorziT/huD36WDUfrtb+1AX7as/PHLL7cD/cYb7XLEYKeEi/qKMw6/MQThtXQwSs88Ywf1aac5l9m3D/jhDxnopfisGEq00lUdgH3FGVY4ZTIZlPt/SERQLBZr3n4abdgAHHOMe5l33wWam8PpT5z4fVYMr9gp0aK+4oz6N4Y02bzZvvJ2C/U9e+yJ0XoM9Uow2Cnxyt2sEpYkjFFXKuzJ4G3b7EA/4gjnMjt32oFecqrJAYOdKICof2MwLczJ4JERO9DddiQaHbUD/YADjDefahxjJ6JJQZ917sebbwIHH+xeZvt2YMECI82lSuhj7CLSICLPicgjpuokonDVcvnonj32FbpbqG/ZYl+hM9SDMTkUcxWAjQbrI6KQ1WIy+N137UBvbXUu8+KLdqAvWlR1MzSFkWAXkcMAnA3gThP1EZEt7IlMk5PB+/bZgT57tnOZgQE70Bcvrrh6cmHqiv1WANcAcFy4KyI5ERkQkYHR0VFDzRKlVxR3tZqYDC4W7UBvanIus3atHehLlhjoNM0QePJURJYB+Lyq9orIpwH8u6o63Pxr4+QpkbcwJjJNUgUyHpeKv/41cNZZ4fQnjcKcPD0JwLkiMgjgZwBOF5GVBuolqmtRPwenEiLuof7gg3bwM9TDETjYVfU6VT1MVbMAVgB4QlUddhkkIr+ScFdrR4f7c1ruussOdKdt6qg2eIMSUUzF+a7Wo46yA33HjvLHb73VDvTLLgu3X2QzGuyq+luv8XUi8ieOd7Wecood6Js2lT8+scnFVVeF2y+ajneeEpGn884DHn7Y+fg3vwncfHN4/alXfLojEQV26aX2FbpTqF92mX2FzlCPl8aoO0BE8XP11cAttzgfP/984KGHwusPVYbBTkSTTj8dePJJ5+Onngr87nfh9Yeqw6EYIsKKFfaQi1OoL15sD7kw1JOBV+xEdezkk4GnnnI+/sEPAq+/Hl5/yAxesRPVoeXL7St0t1BXZagnFYOdqI58/et2oK9a5VymWLRDnZKLwU5UB777XTvQb7/ducy+fXaguz0igJKBY+xEKfajHwFXXOFe5p13gFmzwukPhYPBTpRCDz4IfPGL7mV27QLmzQunPxQuBjtRijz2GPCZz7iXGR0F2tvD6Q9Fg8FOlAJ//CNwwgnuZYaGgBg98ZdqiMFOlGCbNtmP0HXz0kveZShduCqGKIFeecVeveIW2OvW2atcGOr1h8FOlCA7dtiB7jaksmaNHeheQzOUXoGDXURmi8gzIvInEXlRRL5jomNE9L7du+1A7+hwLnP//Xagn3lmeP2ieDJxxf4ugNNV9TgAxwM4S0RONFAvUd175x070N2WJV5yyTOwrCxWrMggm82iUCiE10GKpcCTp2pvwbR7/Num8RdvSCYKYP9+oNHj/84bbgAWLSogl8thbGwMADA0NIRcLgcAkW6hR9EyMsYuIg0i8jyAEQCPqeo6E/USeSkUCshms8hk0nG1OnFLv1uoX3mlXe7664G+vr7JUJ8wNjaGvr6+GveU4szIckdV3Q/geBE5CMBDInK0qm6YWkZEcgByANDJxbRkQKGQrqtVr2e0LFsG/PKX098bHh4uW9bpfaoPxjezFpH/ADCmqjc5leFm1mRCNpvF0NDQjPcty8Lg4GD4HaqSV6DPmmWPtZeTlnNA/oS2mbWIdIxfqUNE5gA4E8CmoPUSeUn61aqId6irOoc6APT396OlpWXaey0tLejv7zfQQ0oqE2Ps8wE8KSIvAPgj7DH2RwzUS+TKaUgv7kN9fgPdzy/T3d3dyOfzsCwLIgLLspDP5xM5FEXmGB+K8YNDMWRC6Rg7YF+txjXY/DznnBtckJvQhmKITKhmdUtSrlZNXqET+cFgp8hNXHkPDQ1BVSdXt/gN98HBQRSLRQwODlYV6rVaMslAp6hwKIYiF+XKjloM53DIhWrF71AMg50il8lkUO7voYigWCzWtG2T/6gw0KnWOMZOiRHl6hYTSyY55EJxw2CnyIWxFttpHD3IPyqzZzPQKZ4Y7BS5Wq9ucZucreYflWOOsQP93Xed2ywWGegUHY6xU+p5jaMXCgX09fVheHgYnZ2d6O/vL/uPyjnnAI943Hq3dy/Q1GSq50TTcYw9pfwszUvbEw+rMfUclAt14P1xdK8lk1deaV+hu4X67t32FTpDnWJBVUN/LVmyRKlyK1eu1JaWFoX9vHsFoC0tLbpy5cqKyqRduXNQ7mVZlms9/f0TI+TOr5GRcH4mIlVVAAPqI2M5FJMgfpbm8Wl/zudgKre16nffDVx2mXsbW7cCCxcG6SVR5TgUk0J+luYl/YmHJrj9rG6Ts6tX20MubqH+3HP2tTpDneKMwZ4gfpbmJfWJhyY5/ayWZZUdR3/ySTvQly1zrvOJJ+xAP/54070lMo/BniB+lubx+dz+z8Ezz9iBfvrpznU98IAd6KedVoueEtWIn4F40y9OnlZv5cqValmWiohallV2UtRPmbRzOwcvv+w9KXr77RF2nsgBOHlKNN1rrwHz57uXufZa4MYbw+kPUaU4eUqRCXMdvZ+2du+2h1zcQv2YY+xrdYY6pUFj0ApE5HAAPwFwCOz1wXlV/V7QeimZSh+DO3H7PgDjG2B4tbVvn78bhnjrP6VN4KEYEZkPYL6qPisi8wCsB3C+qr7k9BkOxaRXmOvondrq7LQwPOzdFgOdksbvUEzgK3ZVfRXAq+NfvyUiGwF8CIBjsFN6ed2+b1L5OhVeTTHQKe2MjrGLSBbAxwGsM1kvJUOhUIA4PMe2Fuvop9c58aQAZ3yELtULY8EuIq0Afg7gG6q6q8zxnIgMiMjA6OioqWYpRvr6+hx3QqrFOnq7TgY6USkjyx1FpAnAIwB+o6o3e5XnGHs6OW1xB8Dx/WpxGzqqR6EtdxT7d++7AGz0E+oUTK2XEk6tv729He3t7b7bcruV31S/uQ0dkQ9+7mJyewE4Gfbvwi8AeH789Xm3z/DO0+rU+pG8Xo+79WrLqX89PT2B++11pyhg4gwQxRt83nnKRwokiGVZVT1XPGj9lbRV7lb+IP1moBO9z2+w85ECCeI0hi0iKBaLNas/aFvV9Jtj6EQz8ZECKVTrR/L6qaeatirpN8fQiYJjsEekmsnEWj+St1z9Xm35+Tn89DuqQOf+sJRKfsZrTL/qfYw9yCRorR/JO7X+trY2bWtrc2yrkp/Dqd9+xtCLRaM/YlX9J4oDcIw9vtKyL2mQn8PPGPp77wGNgR964Swtfw5UP/yOsTPYI1DrSdCw1GpSdOdO4IADgvbOW1r+HKh+cPI0xtKyL6npSdGtW+3BlzBCHUjPnwNRKQZ7BNKyL6mpSdE//MEO9IULa9FLZ2n5cyCawc9AvOlXvU+eqgafBK1kkrOWfXH67KGHek+K3ntvRd2sCe4PS0kC3nmaXkFv/feqK8jKkJNO8g7073ynqqqJ6p7fYOfkaQI5reaYyu/KDlMrQ666CrjtNvcyy5cDDzzgu0oiKhHaDkoUPj+7EfndscipnN/P33abHepuFi0CtmzxVR0RGcDJ0wQyeet/tStD1qyxJ0W9Ql2VoU4UNgZ7AlVz638ldbl9fuNGO9A/+1n3eidG1IkofAz2BOru7kY+n4dlWRARtLW1oa2tDSICy7KQz+fR3d1dVV1On3/jDTvQFy92r4+BThQ9Tp4mXKFQQF9fH4aHh9HZ2Yn+/n7foe7H3r3ArFne5RjmRLXHydM6UCgUkMvlMDY2BgAYGhpCLpcDgMDhrgpkfPw+x0Anih8jQzEi8mMRGRGRDSbqI3/6+vomQ33C2NgY+vr6AtUr4h3qHHIhii9TY+z3ADjLUF3kU9CliqW4yQVROhgJdlVdC+AfJuoi/0w9xIqBTpQuXBWTYEEfYsVAJ0qn0IJdRHIiMiAiA6Ojo2E1m2p+lyqWYqATpZux5Y4ikgXwiKoe7VWWyx2j4WeTC4Y5UXxxuSNNYqAT1RdTyx3vA/B/AI4UkW0i8mUT9VIwHHIhqk+mVsVcqKrzVbVJVQ9T1btM1JsUhUIB2WwWIoLGxkaICLLZLHp7e9He3g4RgYigvb0dhUKhJm1nMhlks1kUCgVfgb5yZQGWNf1zYSjXXz/HvT5HRFP4eWi76VeaNtrw2vSi9NXU1GRsl56ZbXtvcuHU5yCba1Tf3+ntOh3v6emJpL9EcQNutBEOP5telKp0Ewvvtr3/DIvF96/iTW2uUSmvdp2ONzQ0YP/+/Y6fI6oXfidPGewBZTIZVHoORQTFYjFw234mRd9+G5g9e/p7Tn021S8nXu1Wei5r3V+iuPEb7LxBKaBK7/Ks9jNT+RlDf/VVe/ClNNTd2g/aLy9e7Todb2hoqKg+onrHYA/Ia9OLUk1NTb7vDC3lJ9Bnz/4UVq4s4NBDncsEvWO1Wl7tOh3P5XKR9JcosfwMxJt+pWnyVNWe9LMsSwFoQ0ODAlDLsrSnp0fb2tomJ/za2tqqmvDzMykKLFPLsnzXP9FnEanoc0F5tet0PKr+EsUJOHmafH7G0P/7v4Frrql9X4goehxjT7CPftQ71C+80L5OZ6gTUSk+UiBGLrkEuPde9zIdHcDISCjdIaKEYrDHwPe/D3z1q97leOs/EfnBYI/Q6tXAsmXe5RjoRFQJBnsENm0CjjrKuxwDnYiqwWAP0ZtvAgcf7F2OgU5EQTDYQ/Dee0Bzs3c5BjoRmcBgrzFuckFEYWOw1wgDnYiiwmA3jIFORFEztTXeWSLyFxHZLCLXmqizVLU76Pjdkae3t3dyF6TSV2trK+bMmTP5fSaTQW9v77R6/Dygq6GhEcDM+p1eZ5xxBgCgt7cXmUxmWvtTy2UyGcybN2/az9jb2zu5m1NjYyN6e3vL/sxTd3hqbW1Fe3s7MpkM2tvbJ7+u9ny3t7ejtbXVdQep0vLVtOlUn98dmrg7E6WOnwfKuL0ANADYAmARgGYAfwKw2O0zlT4ErNodfyrZkaea19KlS30+oKv6NhYsWFDxZyYeRFb6ymQyVfej2vNd+mpubnbdManSNiv5e1LueHNzszY1NQVqlygs8PkQMBPB/kkAv5ny/XUArnP7TKXBPvHkxNKXZVlVfc4p+Cp71TbQ4/iq9nw71eOnvFeblfw98du/StslCovfYA/8dEcRuQDAWar6b+Pf/yuAf1bVr5aUywHIAUBnZ+eSSraTq3bHn2p2N/Lmpz4fA+0JZOp8V7JjUiW7JJncoYm7M1Ecxe7pjqqaV9UuVe3q6Oio6LPV7vhT6Y487iYu5twI0hrqQPXn26mcn/KV7JJU7Q5NQdslihsTwb4dwOFTvj9s/D1jqt3xp5IdeZxFG+gLFiyo+DNO/3BlMtX/cVd7vks1Nze77phUaZte7Xvt0NTc3IympqZA7RLFjp/xGrcX7CWTWwEsxPuTpx9z+0w1OyhVu4OO3x15enp6SsZgvcfQp9YDH+O2lY7tL126VFVVe3p6VEQm35/69cT3ra2t037Gnp6eyfYaGhq0p6en7M88dYenuXPnaltbm4qItrW1TX5d7flua2vTuXPnTtZfbgep0vLVtOnnz9vtOHdnoqRAmDsoicjnAdwKe4XMj1XV9XInzjsoLVoE/O1v7mWMD9sTEfkQ6hi7qv5KVT+qqh/2CvW4uvlmex26W6gXiwx1Ioq/ur/z9J57gEsvdS/z3ntAY92fKSJKirqNq8cfB8Zv7HT09tvA7Nnh9IeIyJS6C/Y//xk49lj3Mrt2AfPmhdMfIiLT6ibYN28GjjjCvczOncABB4TTHyKiWkl9sO/aBRx4oHuZkRGgwnumiIhiK7Q7T8O2Zw+wZIl7qO/YYa9yYagTUZqkLtjfeQc49VSgtRV49tnyZbZvtwO9rS3cvhERhSE1wb53L/C5zwFz5gC//335Mlu32oFexV36RESJkfhg37cPWL4cmDULePTRmcc7OuwxdFVg4cLw+0dEFLbEBnuxCFx8MdDUBKxaNfN4S4s95MKJUSKqN4kLdlWgpwdoaAB+8pPyZQYH7clTDrkQUT1K1HLHp54CTj7Z+fjLLwMf+Uh4/SEiiqNEXbFfckn591980b6SZ6gTESUs2D/wgenfP/usHeiLF0fTHyKiOErUUMyjjwJr1wLZLHDccVH3hogonhIV7AcfDJx3XtS9ICKKt0BDMSKyXEReFJGiiHju6kFERLUXdIx9A4B/AbDWQF+IiMiAQEMxqroRAETETG+IiCiwRK2KISIib55X7CLyvwAOLXOoT1X/x29DIpIDkAOAzs5O3x0kIqLKeAa7qnrsDOqPquYB5AGgq6tLTdRJREQzcSiGiChlRLX6i2cR+QKA2wF0AHgTwPOq+lkfnxsFMFR1w+nSDmBH1J2IEZ6PmXhOpqvn82GpqufzagMFOwUnIgOqynsAxvF8zMRzMh3PhzcOxRARpQyDnYgoZRjs0ctH3YGY4fmYiedkOp4PDxxjJyJKGV6xExGlDIM9QiLSICLPicgjUfclDkTkIBFZJSKbRGSjiHwy6j5FSUS+Of701A0icp+IzI66T2ETkR+LyIiIbJjy3gdE5DEReXn8vwdH2cc4YrBH6yoAG6PuRIx8D8CjqvpPAI5DHZ8bEfkQgK8D6FLVowE0AFgRba8icQ+As0reuxbA46p6BIDHx7+nKRjsERGRwwCcDeDOqPsSByJyIIBTAdwFAKq6V1XfjLZXkWsEMEdEGgG0APh7xP0JnaquBfCPkrfPA3Dv+Nf3Ajg/1E4lAIM9OrcCuAZAMeqOxMRCAKMA7h4fnrpTROZG3amoqOp2ADcBGAbwKoCdqrom2l7FxiGq+ur4168BOCTKzsQRgz0CIrIMwIiqro+6LzHSCOATAO5Q1Y8D2IM6/hV7fNz4PNj/4C0AMFdELoq2V/Gj9rI+Lu0rwWCPxkkAzhWRQQA/A3C6iKyMtkuR2wZgm6quG/9+Feygr1dnAPibqo6q6nsAfgHgUxH3KS5eF5H5ADD+35GI+xM7DPYIqOp1qnqYqmZhT4g9oap1fTWmqq8BeEVEjhx/aymAlyLsUtSGAZwoIi1ib1G2FHU8mVziYQAXj399MQDf+0LUi0Bb4xEZ9jUABRFpBrAVwKUR9ycyqrpORFYBeBbAPgDPoQ7vuBSR+wB8GkC7iGwD8J8A/gvAAyLyZdhPif1idD2MJ955SkSUMhyKISJKGQY7EVHKMNiJiFKGwU5ElDIMdiKilGGwExGlDIOdiChlGOxERCnz/5nKxu1/eSW3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot outputs\n",
    "plt.scatter(Xs_test, y_test,  color='black')\n",
    "plt.plot(Xs_test, regr.predict(Xs_test), color='blue', linewidth=3)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:25:26.241798Z",
     "start_time": "2019-04-22T08:25:26.227633Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.6837007391943056"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "regr = linear_model.LinearRegression()\n",
    "scores = cross_val_score(regr, df.click_log, \\\n",
    "                         df.reply_log, cv = 3)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-22T08:25:30.245410Z",
     "start_time": "2019-04-22T08:25:30.227128Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.7188149722820985"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "scores = cross_val_score(regr, df.click_log, \n",
    "                         df.reply_log, cv =5)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn做logistic回归\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- logistic回归是一个分类算法而不是一个回归算法。\n",
    "- 可根据已知的一系列因变量估计离散数值（比方说二进制数值 0 或 1 ，是或否，真或假）。\n",
    "- 简单来说，它通过将数据拟合进一个逻辑函数(logistic function)来预估一个事件出现的概率。\n",
    "- 因此，它也被叫做逻辑回归。因为它预估的是概率，所以它的输出值大小在 0 和 1 之间（正如所预计的一样）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$$odds= \\frac{p}{1-p} = \\frac{probability\\: of\\: event\\: occurrence} {probability \\:of \\:not\\: event\\: occurrence}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$$ln(odds)= ln(\\frac{p}{1-p})$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$$logit(x) = ln(\\frac{p}{1-p}) = b_0+b_1X_1+b_2X_2+b_3X_3....+b_kX_k$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/logistic.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:46:50.277195Z",
     "start_time": "2018-04-29T07:46:50.272229Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "repost = []\n",
    "for i in df.title:\n",
    "    if u'转载' in i:\n",
    "        repost.append(1)\n",
    "    else:\n",
    "        repost.append(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:47:06.292994Z",
     "start_time": "2018-04-29T07:47:06.270715Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[194675, 2703], [88244, 1041], [82779, 625]]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X = [[df.click[i], df.reply[i]] for i in range(len(df))]\n",
    "data_X[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:47:45.269303Z",
     "start_time": "2018-04-29T07:47:45.259792Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.61241970021413272"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "df['repost'] = repost\n",
    "model = LogisticRegression()\n",
    "model.fit(data_X,df.repost)\n",
    "model.score(data_X,df.repost)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:47:59.648431Z",
     "start_time": "2018-04-29T07:47:59.633936Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def randomSplitLogistic(dataX, dataY, num):\n",
    "    dataX_train = []\n",
    "    dataX_test = []\n",
    "    dataY_train = []\n",
    "    dataY_test = []\n",
    "    import random\n",
    "    test_index = random.sample(range(len(df)), num)\n",
    "    for k in range(len(dataX)):\n",
    "        if k in test_index:\n",
    "            dataX_test.append(dataX[k])\n",
    "            dataY_test.append(dataY[k])\n",
    "        else:\n",
    "            dataX_train.append(dataX[k])\n",
    "            dataY_train.append(dataY[k])\n",
    "    return dataX_train, dataX_test, dataY_train, dataY_test, "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:48:27.726443Z",
     "start_time": "2018-04-29T07:48:27.710922Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Variance score: 0.45'"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Split the data into training/testing sets\n",
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplitLogistic(data_X, df.repost, 20)\n",
    "# Create logistic regression object\n",
    "log_regr = LogisticRegression()\n",
    "# Train the model using the training sets\n",
    "log_regr.fit(data_X_train, data_y_train)\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "'Variance score: %.2f' % log_regr.score(data_X_test, data_y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:48:56.873331Z",
     "start_time": "2018-04-29T07:48:56.870219Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "y_true, y_pred = data_y_test, log_regr.predict(data_X_test)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:39:12.344043Z",
     "start_time": "2018-04-29T07:39:12.338223Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],\n",
       " array([0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]))"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_true, y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:39:13.175680Z",
     "start_time": "2018-04-29T07:39:13.171386Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.50      0.17      0.25         6\n",
      "          1       0.72      0.93      0.81        14\n",
      "\n",
      "avg / total       0.66      0.70      0.64        20\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_true, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:51:43.039620Z",
     "start_time": "2018-04-29T07:51:43.034812Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "Xs_train, Xs_test, y_train, y_test = train_test_split(data_X, df.repost, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:51:47.690742Z",
     "start_time": "2018-04-29T07:51:47.683127Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Variance score: 0.60'"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create logistic regression object\n",
    "log_regr = LogisticRegression()\n",
    "# Train the model using the training sets\n",
    "log_regr.fit(Xs_train, y_train)\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "'Variance score: %.2f' % log_regr.score(Xs_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:51:55.780061Z",
     "start_time": "2018-04-29T07:51:55.771924Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logistic score for test set: 0.595745\n",
      "Logistic score for training set: 0.613941\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       1.00      0.03      0.05        39\n",
      "          1       0.59      1.00      0.74        55\n",
      "\n",
      "avg / total       0.76      0.60      0.46        94\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print('Logistic score for test set: %f' % log_regr.score(Xs_test, y_test))\n",
    "print('Logistic score for training set: %f' % log_regr.score(Xs_train, y_train))\n",
    "y_true, y_pred = y_test, log_regr.predict(Xs_test)\n",
    "print(classification_report(y_true, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:52:53.880925Z",
     "start_time": "2018-04-29T07:52:53.866672Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.53333333333333333"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logre = LogisticRegression()\n",
    "scores = cross_val_score(logre, data_X, df.repost, cv = 3)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T07:53:26.825100Z",
     "start_time": "2018-04-29T07:53:26.810871Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.62948717948717947"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logre = LogisticRegression()\n",
    "data_X_scale = scale(data_X)\n",
    "# The importance of preprocessing in data science and the machine learning pipeline I: \n",
    "scores = cross_val_score(logre, data_X_scale, df.repost, cv = 3)\n",
    "scores.mean()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn实现贝叶斯预测\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Naive Bayes algorithm\n",
    "\n",
    "It is a classification technique based on Bayes’ Theorem with an assumption of independence among predictors. \n",
    "\n",
    "In simple terms, a Naive Bayes classifier assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature. \n",
    "\n",
    "why it is known as ‘Naive’? For example, a fruit may be considered to be an apple if it is red, round, and about 3 inches in diameter. Even if these features depend on each other or upon the existence of the other features, all of these properties independently contribute to the probability that this fruit is an apple."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "贝叶斯定理为使用$p(c)$, $p(x)$, $p(x|c)$ 计算后验概率$P(c|x)$提供了方法："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$$\n",
    "p(c|x) = \\frac{p(x|c) p(c)}{p(x)}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- P(c|x) is the posterior probability of class (c, target) given predictor (x, attributes).\n",
    "- P(c) is the prior probability of class.\n",
    "- P(x|c) is the likelihood which is the probability of predictor given class.\n",
    "- P(x) is the prior probability of predictor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/Bayes_41.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Step 1: Convert the data set into a frequency table\n",
    "\n",
    "Step 2: Create Likelihood table by finding the probabilities like:\n",
    "- p(Overcast) = 0.29, p(rainy) = 0.36, p(sunny) = 0.36\n",
    "- p(playing) = 0.64, p(rest) = 0.36\n",
    "\n",
    "Step 3: Now, use Naive Bayesian equation to calculate the posterior probability for each class. The class with the highest posterior probability is the outcome of prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Problem: Players will play if weather is sunny. Is this statement is correct?\n",
    "\n",
    "We can solve it using above discussed method of posterior probability.\n",
    "\n",
    "$P(Yes | Sunny) = \\frac{P( Sunny | Yes) * P(Yes) } {P (Sunny)}$\n",
    "\n",
    "Here we have P (Sunny |Yes) = 3/9 = 0.33, P(Sunny) = 5/14 = 0.36, P( Yes)= 9/14 = 0.64\n",
    "\n",
    "Now, $P (No | Sunny) = \\frac{0.33 * 0.64}{0.36} = 0.60$, which has higher probability.\n",
    "\n",
    "$P(No | Sunny) = \\frac{P( Sunny | No) * P(No) } {P (Sunny)}$\n",
    "\n",
    "Here we have P (Sunny |No) = 2/5 = 0.4, P(Sunny) = 5/14 = 0.36, P( No)= 5/14 = 0.36\n",
    "\n",
    "Now, $P (Yes | Sunny) = \\frac{0.4 * 0.36}{0.36} = 0.4$, which has lower probability.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'ABCMeta  BaseDiscreteNB  BaseEstimator  BaseNB  BernoulliNB  ClassifierMixin  GaussianNB  LabelBinarizer  MultinomialNB  __all__  __builtins__  __doc__  __file__  __name__  __package__  _check_partial_fit_first_call  abstractmethod  binarize  check_X_y  check_array  check_is_fitted  in1d  issparse  label_binarize  logsumexp  np  safe_sparse_dot  six'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import naive_bayes\n",
    "'  '.join(dir(naive_bayes)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- naive_bayes.GaussianNB\tGaussian Naive Bayes (GaussianNB)\n",
    "- naive_bayes.MultinomialNB([alpha, ...])\tNaive Bayes classifier for multinomial models\n",
    "- naive_bayes.BernoulliNB([alpha, binarize, ...])\tNaive Bayes classifier for multivariate Bernoulli models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:02:37.644606Z",
     "start_time": "2018-04-29T08:02:37.635952Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#Import Library of Gaussian Naive Bayes model\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "import numpy as np\n",
    "\n",
    "#assigning predictor and target variables\n",
    "x= np.array([[-3,7],[1,5], [1,2], [-2,0], [2,3], [-4,0], [-1,1], [1,1], [-2,2], [2,7], [-4,1], [-2,7]])\n",
    "Y = np.array([3, 3, 3, 3, 4, 3, 3, 4, 3, 4, 4, 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:02:52.828101Z",
     "start_time": "2018-04-29T08:02:52.818463Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([4, 3])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Create a Gaussian Classifier\n",
    "model = GaussianNB()\n",
    "\n",
    "# Train the model using the training sets \n",
    "model.fit(x[:8], Y[:8])\n",
    "\n",
    "#Predict Output \n",
    "predicted= model.predict([[1,2],[3,4]])\n",
    "predicted"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# cross-validation \n",
    "    \n",
    "k-fold CV, the training set is split into k smaller sets (other approaches are described below, but generally follow the same principles). The following procedure is followed for each of the k “folds”:\n",
    "- A model is trained using k-1 of the folds as training data;\n",
    "- the resulting model is validated on the remaining part of the data (i.e., it is used as a test set to compute a performance measure such as accuracy)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:04:04.297675Z",
     "start_time": "2018-04-29T08:04:04.273413Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([41,  2,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,\n",
       "        0,  0,  0])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplit(df.click, df.reply, 20)\n",
    "# Train the model using the training sets \n",
    "model.fit(data_X_train, data_y_train)\n",
    "\n",
    "#Predict Output \n",
    "predicted= model.predict(data_X_test)\n",
    "predicted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:04:34.184513Z",
     "start_time": "2018-04-29T08:04:34.178511Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.65000000000000002"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.score(data_X_test, data_y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:05:04.297453Z",
     "start_time": "2018-04-29T08:05:04.249311Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/datalab/Applications/anaconda/lib/python3.5/site-packages/sklearn/cross_validation.py:516: Warning: The least populated class in y has only 1 members, which is too few. The minimum number of labels for any class cannot be less than n_folds=7.\n",
      "  % (min_labels, self.n_folds)), Warning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.53413410073295453"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "\n",
    "model = GaussianNB()\n",
    "scores = cross_val_score(model, [[c] for c in df.click],\\\n",
    "                         df.reply, cv = 7)\n",
    "scores.mean()    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn实现决策树\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# 决策树\n",
    "- 这个监督式学习算法通常被用于分类问题。\n",
    "- 它同时适用于分类变量和连续因变量。\n",
    "- 在这个算法中，我们将总体分成两个或更多的同类群。\n",
    "- 这是根据最重要的属性或者自变量来分成尽可能不同的组别。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/tree.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/playtree.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 在上图中你可以看到，根据多种属性，人群被分成了不同的四个小组，来判断 “他们会不会去玩”。\n",
    "### 为了把总体分成不同组别，需要用到许多技术，比如说 Gini、Information Gain、Chi-square、entropy。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:10:20.871345Z",
     "start_time": "2018-04-29T08:10:20.855125Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn import tree\n",
    "model = tree.DecisionTreeClassifier(criterion='gini')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:10:49.988277Z",
     "start_time": "2018-04-29T08:10:49.973060Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.91275167785234901"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplitLogistic(data_X, df.repost, 20)\n",
    "model.fit(data_X_train,data_y_train)\n",
    "model.score(data_X_train,data_y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:11:12.730866Z",
     "start_time": "2018-04-29T08:11:12.725782Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0])"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict\n",
    "model.predict(data_X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:11:28.411441Z",
     "start_time": "2018-04-29T08:11:28.397481Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.33461538461538459"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# crossvalidation\n",
    "scores = cross_val_score(model, data_X, df.repost, cv = 3)\n",
    "scores.mean()   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> # 使用sklearn实现SVM支持向量机\n",
    "***\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/svm.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- 将每个数据在N维空间中用点标出（N是你所有的特征总数），每个特征的值是一个坐标的值。\n",
    "   - 举个例子，如果我们只有身高和头发长度两个特征，我们会在二维空间中标出这两个变量，每个点有两个坐标（这些坐标叫做支持向量）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/xyplot.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- 现在，我们会找到将两组不同数据分开的一条直线。\n",
    "    - 两个分组中距离最近的两个点到这条线的距离同时最优化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![](./img/sumintro.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## 上面示例中的黑线将数据分类优化成两个小组\n",
    "- 两组中距离最近的点（图中A、B点）到达黑线的距离满足最优条件。\n",
    "    - 这条直线就是我们的分割线。接下来，测试数据落到直线的哪一边，我们就将它分到哪一类去。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:17:29.788250Z",
     "start_time": "2018-04-29T08:17:29.785022Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn import svm\n",
    "# Create SVM classification object \n",
    "model=svm.SVC() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:17:31.035310Z",
     "start_time": "2018-04-29T08:17:31.030713Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'LinearSVC LinearSVR NuSVC NuSVR OneClassSVM SVC SVR __all__ __builtins__ __cached__ __doc__ __file__ __loader__ __name__ __package__ __path__ __spec__ base bounds classes l1_min_c liblinear libsvm libsvm_sparse'"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "' '.join(dir(svm))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:17:41.872379Z",
     "start_time": "2018-04-29T08:17:41.849759Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.90380313199105144"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_X_train, data_X_test, data_y_train, data_y_test = randomSplitLogistic(data_X, df.repost, 20)\n",
    "model.fit(data_X_train,data_y_train)\n",
    "model.score(data_X_train,data_y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:17:47.661313Z",
     "start_time": "2018-04-29T08:17:47.655841Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predict\n",
    "model.predict(data_X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:18:00.419986Z",
     "start_time": "2018-04-29T08:17:58.671257Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# crossvalidation\n",
    "scores = []\n",
    "cvs = [3, 5, 10, 25, 50, 75, 100]\n",
    "for i in cvs:\n",
    "    score = cross_val_score(model, data_X, df.repost,\n",
    "                            cv = i)\n",
    "    scores.append(score.mean() ) # Try to tune cv\n",
    "      "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-29T08:18:05.493658Z",
     "start_time": "2018-04-29T08:18:05.359658Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEXCAYAAACK4bLWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X2YVWW9//H3BxUVRU0BAeWh9Ij5kA9MSWhCYoDaycfQItOwsCxPKmoeOVbWhT9NOZdHT9ZFppiSZaAdFRVUBBUfckjUk6Wh8ijgIJopKjDz/f1x7zlsxhmYPew9a8/en9d1zTWse62157tA/LDue637VkRgZma2pTplXYCZmVUGB4qZmRWFA8XMzIrCgWJmZkXhQDEzs6JwoJiZWVE4UMzMrCgcKGZmVhQOFDMzK4qtsy6gPXXr1i369++fdRlmZh3KvHnzVkVE980dV1WB0r9/f2pra7Muw8ysQ5G0qDXHucvLzMyKwoFiZmZFUTaBImmUpNckLZA0ppn9P5a0RNJCSYNzbUMkvZw7b3z7V21mZo3KYgxFUldgIjAIqAfmS7onIupy+8cANcA+wAfAtpIE3AicDLwCPCtpekTMz+IazMyqXbncoYwA5kTEsohYAcwChuXtPx84LyLej+QDYCCwMiKej4j3gKnAyHav3MzMgPIJlD5A/lMES4FeAJK2AXoCYyS9JOkuSbtt6px8ksZKqpVUW1dXV7ILMDMrR1OmQP/+0KlT+j5lSul+VrkESmegIW+7gdT1BdAN+BjwCLAvsBgYv5lz/k9ETIqImoio6d59s49Rm5lVjClTYOxYWLQIItL3sWNLFyrlEijLgT3ytvcEluR+vQp4NyIejLRe8f8AAzZzjplZ1bv0UlizZuO2NWtgfIkeYSqXQJkBjJDUQ1JPYDAwEyAi1gFPS2ocH/ki8AzwFDBA0gBJOwAnAXe2f+lmZuVjzRqYPh2++11YvLj5Y1pq31Jl8ZRXRKzMPfb7ZK5pHDBc0l4RcQ3wHeBWSf9NCpPLImKtpLOAe0jdX1dFRKve5jQzqyQLF6YQue8+mDULPvgAunSB7beH99//6PF9+5amjrIIFICImAxMbmHfq8DhzbQ/QHqU2MysaqxbB3PnpgCZPh1efDG177VXGiM57jg48kiYNi1t53d7dekCEyaUpq6yCRQzM2vZypXwwAMpQGbOhH/8A7bZJgXHN7+ZQmSfJv+8Hj06fR8/PnVz9e2bwqSxvdgcKGZmZaihAebN23AX8swzqb1XLzjllBQgRx8NXbtu+nNGjy5dgDTlQDEzKxP/+Ac8+GAKkPvvT3clEhx2GPz0pylEDj44tZUjB4qZWUYi4G9/SwEyfTo8/jisXw+77AIjR6YAGTkSunXLutLWcaCYmbWj99+H2bM3hMjChan9wAPhwgtTiAwaBFt3wP87d8CSzcw6lsWLNwTIrFkpVLp0gWHD4JJL4NhjoU+frKvccg4UM7MiW78ennhiQ4j85S+p/ROf2PBE1pAhsN122dZZbA4UM7MiqKtLA+nTp8OMGWmAfeut02O9Y8aku5ABA8p3QL0YHChmZm3Q0ADPPrvhLuSZZ9Ige8+ecPLJGx7r3WmnrCttPw4UM7NWeuedjR/rXbEi3XF85jNw+eUbHuvtVC6zJLYzB4qZWQsi4KWXNsyT9dhjadqTXXaBESM2PNbrlTESB4qZWZ4PPtjwWO9998Grr6b2Aw6ACy5IIfLZz3bMx3pLzb8lZlb1lizZECAPP5wmU9x++/RY70UXwTHHQL9+WVdZ/hwoZlZ11q+HJ5/cME/WCy+k9o9/fMMTWUOHplCx1nOgmFlVWLUqDaTfd196rPett1K31ec+B9dck0Jk330r+7HeUnOgmFlFikiP9TbehTz9dGrbfXc44YQUIF/4Auy8c9aVVo6yCRRJo4CrgHrgioi4KW/fZOALQOPaY0dFxGJJs4H+wPpc+4CIqG+vms2svPzzn/DQQxvGQ5YvT3ccn/40/OhHaUD90EOr97HeUiuLQJHUFZgIDCIFynxJ90REXd5hoyNidjOnD42IhaWv0syyMmVK84tERcDLL2+4C3n00fRY7847p8d6jz02Daj36JH1FVSHsggUYAQwJyKWAUiaBQwDfpdpVWaWuSlTNl7GdtEiOOssuPVWWLAAXnklte+/P5x/fgqRwYPTaobWvsolUPoAi/K2lwK98rbXAbdIehe4KSIm5to/BB6R9DYwMSJua/rBksYCYwH69u1bitrNrMg+/DB1V73+Opx33sZrojfunzEjdWFdcEEKkf79MynV8pRLoHQGGvK2G0hdXwBExLcAJPUBHpT0XEQ8FBEjcu37AQ9LeiYiXsr/4IiYBEwCqKmpidJehpltytq1abqS11/fEBjNfa1evfnPkuDee0tfs7VeuQTKcmBo3vaewNNND4qIJZLuBQ4AHsprf1HSXOCTwEtNzzOz0lq/Pi1X2zQYmoZGXd1Hz91qq7ROeu/esPfeaXbe3r03tI0Zkz6nKXc4lJ9yCZQZwP+T1APoBAwGzm7cKWnviFggaTdgZOO+vPZ+wGHAhe1fulnlqq+HN97Y/B3FG2+kAfJ8nTqlmXd7907/8x80KP266Ve3bpt+6urqqzceQ4G0ONWECaW5Zmu7sgiUiFgpaTzwZK5pHDBc0l4RcQ1wXa5b60Pg+oiYmzvuLkk7AmuAC/y0l1nrNDSkF/2aC4f84FixIh2bT0pPTTUGQk3Nhl833lX07p2O2WqrLa919Oj0vbmnvKy8KJr+s6KC1dTURG1tbdZlmJVMBLz5ZstdTvlBsX79R8/v3v2jwdD0q0cPP0FVbSTNi4iazR1XFncoZtWipfcpNicC3n5783cUy5enge+mdt11QyDst1/zdxQ9e0LnzsW/ZqseDhSzdtLc+xRjx8L778Phh2/6jmL58jStelO77LIhGBoHs5t+9exZeWuXW3lyl5dZO+nfP4VIa3Tt2nw45N9R9OqVBqfNSs1dXmZlZvHilvf99rcbB8WOO7ZfXWbF4kAxawdPP50eja1vZurSfv3gK19p/5rMis1zbpqV2E03pfGNj33so2MZfp/CKokDxaxE1q6Fc85JExkOGQIvvQQ33pjuSKT0fdIkv09hlcNdXmYlsGIFfPnL8PjjaU3yK65IqwOOHu0AscrlQDErsj/9CU46KS0x+7vfwamnZl2RWftwl5dZEd10U1qjfJtt4IknHCZWXRwoZkWwbh1873tpvOTII6G2Fg46KOuqzNqXA8VsC61cCcOGwc9/nsZL7r8fdtst66rM2p/HUMy2QON4yerVcPvtcNppWVdklh3foZi10c03p+6txvESh4lVOweKWYHWrYNzz00rCR5xRBovOfjgrKsyy54DxawAjeMl//3fcOGF8MADHi8xa1Q2gSJplKTXJC2QNKbJvsmSluX2LZDUN9c+RNLLufPGZ1O5VYtnnkmrE9bWpqnor746vaxoZklZ/HWQ1BWYCAwC6oH5ku6JiLq8w0ZHxOy8cwTcCJwMvAI8K2l6RMxvv8qtWtxyC5x9dlpb5Ikn3MVl1pxyuUMZAcyJiGURsQKYBQzbzDkDgZUR8XxEvAdMBUaWuE6rMuvWwb/9G5x5ZloEy+MlZi0rl0DpA+QvPbQU6JW3vQ64RdJfJI1r5TkASBorqVZSbV1dXdPdZi164w04+mi4/nq44AKYMQO6dcu6KrPyVRZdXkBnoCFvu4HU9QVARHwLQFIf4EFJz23unLxzJwGTIK3YWPTKrSLV1qb3S1atSuMlX/1q1hWZlb9yuUNZDuyRt70nsKTpQRGxBLgXOKC155gV6je/SY8Dd+oEc+c6TMxaq1wCZQYwQlIPST2BwcDMxp2S9s593400TvIM8BQwQNIASTsAJwF3tnvlVjHWrYPvfx/OOGPDeMkhh2RdlVnHURZdXhGxMvfY75O5pnHAcEl7RcQ1wHWS9gM+BK6PiLkAks4C7iF1f10VEYua+XizzXrjDRg1CubMgfPPh5/9zI8EmxWqbP7KRMRkYHIL+45tof0BYJ/SVWXVYN48OPFEqKuD227zAlhmbVUuXV5mmfjNb1L3lpTGSxwmZm3nQLGqtG4dnHdeGi8ZPDiNlxx6aNZVmXVsZdPlZdZe6urSeMns2SlUPIWKWXH4r5FVlT//OY2XvPFG6u46/fSsKzKrHO7ysqpx221pvATSeInDxKy4HChW8davT48Cn346DBrk8RKzUnGgWEWrq4Phw+Haa9NLizNnQvfuWVdlVpk8hmIVy+MlZu3LdyhWkaZMSeMlEfD44w4Ts/bgQLGKsn59mmr+a1+Dww5L4yUDB2ZdlVl1cJeXVYxVq+DUU2HWrLQo1jXXwDbbZF2VWfVwoFhFePbZNF6yYkVarvfrX8+6IrPq4y4v6/B++9s0XlJfn8ZLHCZm2XCgWIe1fj2MG5cmdPz0p9OswTU1WVdlVr3c5WUd0qpVcNpp8PDDcO65MHGix0vMsuZAsQ5n/nw44YQ0XnLzzXDmmVlXZGZQRl1ekkZJek3SAkljWjjm15IW5G3PlrQwd84CSVu1X8WWhdtvT9PN19fDY485TMzKSVncoUjqCkwEBgH1wHxJ90REXd4xnwd6NnP60IhY2C6FWmbWr4dLLkldW5/7HPzhD7D77llXZWb5yuUOZQQwJyKWRcQKYBYwrHGnpO2AnwKXZVSfZejNN2HkyBQm3/teGjdxmJiVnzbdoUjaFzgMELA4ImZtYR19gEV520uBXnnbPwR+Aaxuct6HwCOS3gYmRsRtzdQ6FhgL0Ldv3y0s09rbc8+l8ZLlyz1eYlbuCgoUSZ2AXwONT/oLaGj8HEmKiGhDHZ1zn9OogdT1haQDgYMi4lJJ/fNPiogRuWP2Ax6W9ExEvNTkmEnAJICampq21GYZ+d3vYMwY2HXXNF7y6U9nXZGZbUqhXV4/AM4AngS+DUwjhUqjwblB8s8X+LnLgT3ytvcEluR+fQawt6T5wH1AH0m/zz85Il4E5gKfLPDnWhlavx4uugi+8pX0Xsm8eQ4Ts46g0ED5BvASMCQifgX8b/7OiJgLrAdGFfi5M4ARknpI6gkMBmbmPvPCiBgQEQcDxwJLIuJUAEl75773I3XBzS/w51qZefNNOOaYNA/Xd78LDz3k8RKzjqLQMZR+wM8jon4Tx8wjBUKrRcRKSeNJdz4A44DhkvaKiGs2cepdknYE1gAX+Gmvju2559J8XMuWwU03wTe+kXVFZlaIQgPlHWDbzRyzDCi0y4uImAxM3swxC4G987YPLPTnWHn6/e9TgDSOl3zmM1lXZGaFKrTL6xng6NzgfEsagJ3bXpJVk/p6uPjiNI3KwIFp/RKHiVnHVGig/Br4F+AnmzjmU8Cbba7Iqsbq1Wm85Oqr4Zxz0vslPZt7ddXMOoSCurwiYlruCat/l/RJ4IP8/ZJOBI4GphavRKtEzz+f3i9ZtgxuvBHOOivrisxsS7XlxcbRpDuQ7zQ2SHoE6AbsB6wDripKdVaR7rgjjZfssgs8+mhaqtfMOr6Cp16JiIaI+B7pSa7bgVXAEGB/4AXgixExr6hVWkWor4cf/CAt03vIIen9EoeJWeUo9E35TwJ/i+Rp4Olc+7aAIuKDTX6AVa3Vq9OLijNnwne+A9deC507Z12VmRVToXcofwFuadoYER86TKwlL7yQ3nSfPTuNl9xwg8PErBIVGihvsWFKFLPNuuMOGDQIPvgA5szx4LtZJSs0UB4D9i1FIVZZ6uvT+iWnngoHH5zGSwYNyroqMyulQgNlAnCcJE/VZy1avRqOOw6uugq+/W145BG/X2JWDQoNlFNIi189JOmMEtRjHVzjeMkjj8CvfgW/+IXHS8yqRaHvoVwEBGnK+pskXQlMJz3tVQu8EBHri1uidRR/+EN6v2SnndJ4ibu4zKpLoYFyFHBo3tc+wBjStPYAayW9ANRGxDlFq9LKWn09/Md/wJVXwuDBMHUq9Oq1+fPMrLIUOvXKbGB247akLsBBbBwyBwEDAQdKhZoyBcaPh8WLYY890gzBzz8PZ58N113nLi6zatWmNeUbRcQa0homjeuYIKkzcMAW1mVlasoUGDsW1qxJ20uXpq+zzoJf/jLb2swsW20KFEk7k+5GtgIWR8TLjfsiYi3w5+KUZ+Vm/PgNYZLvoYfavxYzKy8Fz+Ul6RLSGvAPkZbu/auk5ZImSOra1kIkjZL0mqQFksa0cMyvJS3I2x4i6eXceePb+rOt9RYvLqzdzKpHQYEi6evAFaQld28FrgV+T1pU69+BZyV9vNAickE0ETgi93WFpO5Njvk80DNvW8CNpEeZDwDOkHRwoT/bCrPLLs239+3bvnWYWfkp9A7lPNLdyYCIODMixkXEV4E+wLeA3YEHJe1Q4OeOAOZExLKIWEF612VY405J2wE/BS7LO2cgsDIino+I90hrsIws8OdaK0XAj38Mb70FW2218b4uXWDChEzKMrMyUmig7AvcFREbrciYm9L+18AJwMeBcQV+bh9gUd72UiD/wdMfAr8AVhdwDgCSxkqqlVRbV1dXYFkG0NAA554Ll18OY8bAzTdDv34gpe+TJsHo0VlXaWZZK3RQfg1NVmnMFxEPS5oBnMymlwluqjOp26xRA1APIOlA4KCIuFRS/9ac06SmScAkgJqamiigJgPWroUzz4Tbb4eLLkrTqUhw+ulZV2Zm5abQO5T/Ja8rqgXPke5SCrEc2CNve082zGp8BrC3pPnAfUCf3DLEmzrHiuC99+D441OYXHUV/OxnKUzMzJpTaKBMBg6S9INNHNO7DXXMAEZI6iGpJ2k1yJkAEXFhRAyIiIOBY4ElEXEq8BQwQNKA3JjNScCdbfjZ1oy33oLhw9OCWL/6FVx8cdYVmVm5KyhQImIy8CDpKazbmz5VJekoYBTwpwI/dyUwnvSC5FzSGMxwSRdu4py1wFnAPaSFv66PiEUtHW+t9/rrcOSRUFub5uf65jezrsjMOgJFFDaskHvi6nbgeNJEke8AC4GPkQbKG4CjIuKxolZaBDU1NVFbW5t1GWVtwYJ0Z1JXB//zP3DUUVlXZGZZkzQvImo2d1zBLzZGxAcRcSLwJdJMwyLN37Un6c5kRDmGiW3ec8/BEUfAO+/ArFkOEzMrTJvn8oqIe4F7ASTtCHwYEeuKVZi1r8ceg3/9V+jaNa39vq/X5TSzAhX6pvyXJF0pqU9+e0S86zDpuO69N3Vz9eoFTzzhMDGztim0y+sc0vonbza3MzcdinUgt94KJ5wABxyQ7lL69Nn8OWZmzSk0UD4FzMxNW9+cvSUtk3TaFtZl7eC//gu+/nUYMiSNmXTrlnVFZtaRFRoou7LxdCcbiYi/k6ZAOXMLarISi4Af/hDOOw9OOgnuuy+NnZiZbYlCA6WOZubLamI+cGDbyrFSq6+Hc86Bn/40vV9yxx2w7bZZV2VmlaDQQHkCOE7S9ps45m3AnSdlaO3aNInjL38JP/hBmtSx6czBZmZtVWig/ALoDkyW1NIjx/sD/9yiqqzo3nsvPRb8+9/D1VfDlVd6Xi4zK65Cp16ZDVwHfBl4XNLQ/P2SvgwcQ94a85a91avh6KPTMr033QQXtjihjZlZ2xX8YmNEnCdpDXAx8LCkt0gD9T1IE0M2AFcWtUprs2XLYMSINKXKtGnpEWEzs1IoeOoVgIi4FKgBfktag+QQ0lTyfwGOj4i5RavQ2uzvf4fDD0/rvd9/v8PEzEprS6ZemQ+cDiBpW9JEky0uvmXt69lnYeTItNriI4/AwIFZV2RmlW6zdyiSzpO0ODc1fUu6OEzKx5w5MHQobLcdPP64w8TM2kdruryGkB4D/nPTHZLOyo2hrJJUJ+lST7+SrbvvTmMme+wBc+fCgAFZV2Rm1aI1gbI/8FhEvJ3fmFtcaxKwM2kK+92AnwL/VewirXVuuSW9+X7QQWlerj33zLoiM6smrQmU3YEFzbSfTQqS3wB9gUHAC8B3JR1WaCGSRkl6TdICSWOa7LtS0t9yXW8X57XPlrQwd84CSVX7mt5//ieceWZaw+Thh2G33bKuyMyqTWsCpTPwbjPtxwDrgPMiYmlE/Im0rnsDUNCisZK6AhOBI3JfV0jqnnfIdRGxL3AocFnu+EZDI2Lv3Fd9IT+3EkTA+PEwbhx8+ctwzz2w445ZV2Vm1ag1gbIC6JffIGkP0l3J0/ldYRHxKvAQ8LkC6xgBzImIZRGxApgFDMv73Ndzv+xNeuflvQI/vyLV18O3vw1XXAFnnw233+55ucwsO60JlCeBY5rcFYzIfZ/TzPEvk5YDLkQfNp7FeCl5k1BKGippCfAocGFENOR2fQg8IulZSV9r7oMljZVUK6m2rq6uwLLK14cfwmmnpfm4Lr0UfvELz8tlZtlqTaBMAroCt0jqLuljwHeBAGY0c/y7rfzcfJ1JXWWNGkgvTAJpypeI6AN8FvilpE/k2kdExMeB0cDVkj7yTFNETIqImoio6d69e9PdHdK778IXvwhTp8LEiTBhguflMrPsbfZ//Ln5u24ETiB1f60CDgZeiYjHmzmlD7CywDqWk960b7QnsKSZWv4KPE4aS8lvfxGYC3yywJ/b4bz5Jgwbll5WnDwZLrgg64rMzJLW3kl8G/gx8Bbpya5lwNdbOPZIYHGBdcwARkjqIaknMBiYCSBpO0kDc7/uQXqabH5ue+/c937AYY3tlWrpUvjc5+C55+DOO+GMM7KuyMxsg1ZNvZIbs/gJ8BNJO0XEO80dJ2kYabD+lkKKiIiVksazYZbiccBwSXsBPwdukLQ7aTD+RxHR+BjzXZJ2BNYAF0TEwkJ+bkfy0kswfDi8/TbMmJGW7TUzKydtmW242TDJ6QXMBu5tw+dOBia3sLvZ91oioipWhpw3D445Jo2TzJ4NhxySdUVmZh/VptmGWxIRt0XEURHxTDE/t5o98gh8/vPQpUual8thYmblqqiBYsX1xz+mGYP79k3zcv3Lv2RdkZlZyxwoZermm+Hkk+HQQ+HRR9Nkj2Zm5cyBUoauuQbGjNmwbO+uu2ZdkZnZ5jlQykgEXHIJXHQRnHpqmpdrhx2yrsrMrHXavGKjFVfjvFw33gjf+Q5cf72nUjGzjsV3KBmZMgX694dOndKg+6BBKUwuuwx+/nOHiZl1PL5DycCUKTB2LKxZk7aXLElfX/sa/OQn2dZmZtZWvkPJwPjxG8Ik32OPtX8tZmbF4kDJwOIWZjprqd3MrCNwoGSgb9/C2s3MOgIHSgYmTIDtt9+4rUuX1G5m1lE5UDIwejQcf3z6tQT9+qWVF0ePzrYuM7Mt4ae8MvLqqzBwINTWZl2JmVlx+A4lA4sXw5/+BKecknUlZmbF40DJwJ13pu8nn5xtHWZmxVQ2gSJplKTXJC2QNKbJvisl/U3SYkkX57UPkfRy7rzx7V9120ybBp/6lKejN7PKUhaBIqkrMBE4Ivd1haTueYdcFxH7AocCl0nqKknAjcApwAHAGZIObufSC/b662ltE9+dmFmlKYtAAUYAcyJiWUSsAGYBwxp3RsTruV/2BhaR1pYfCKyMiOcj4j1gKjCyfcsu3F13pVmFPX5iZpWmXAKlDykoGi0lrU8PgKShkpYAjwIXRkTD5s7JO3espFpJtXV1dSUpvhDTpsG++8J++2VdiZlZcZVLoHQGGvK2G4D6xo2ImB0RfYDPAr+U9InNnZN37qSIqImImu7duzfd3a7q6mDOHN+dmFllKpdAWQ7kL3K7J7Ck6UER8VfgcdJYSqvOKSd//CM0NDhQzKwylUugzABGSOohqScwGJgJIGk7SQNzv+4BDALmA08BAyQNkLQDcBJwZybVt9LUqbDXXukJLzOzSlMWb8pHxMrcY79P5prGAcMl7QX8HLhB0u6kwfgfRcQCAElnAfeQur+uiohFH/308rB6NcyaBePGpelWzMwqTVkECkBETAYmt7D7sBbOeQDYp0QlFdXdd8P69X5c2MwqV7l0eVW8adPS9PQ1NVlXYmZWGg6UdvDOOzBzZro7cXeXmVUqB0o7uPdeWLvWT3eZWWVzoLSDqVOhd28YNCjrSszMSseBUmLvvgv33w8nnQSd/LttZhXM/4srsfvvhw8+cHeXmVU+B0qJTZ0KPXrAEUdkXYmZWWk5UEro/fdh+nQ48UTYaqusqzEzKy0HSgnNmAHvveeXGc2sOjhQSmjaNNh1Vxg6NOtKzMxKz4FSIh9+mKZbOf542GabrKsxMys9B0qJPPxwekPeT3eZWbVwoJTI1Kmw004wbNjmjzUzqwQOlBJYty4tpvWlL8G222ZdjZlZ+3CglMDs2fDWW+7uMrPq4kApgalTYYcdYPjwrCsxM2s/ZRMokkZJek3SAkljmuz7vqS/Sloo6VZJW+faZ+faFuS+Mn99sL4e7roLvvhF2H77rKsxM2s/ZREokroCE4Ejcl9XSOqed8g/gYOAvYHdgVPz9g2NiL1zX/XtVXNLHnsM6ur8MqOZVZ+yCBRgBDAnIpZFxApgFvB/z0dFxE0RsTYi1gPPA7tmVOdmTZuW7kyOOSbrSszM2le5BEofYFHe9lKgV9ODJHUBjgPuyTV9CDwi6VlJX2vugyWNlVQrqbaurq7IZW+soSEFysiRsOOOJf1RZmZlp1wCpTPQkLfdAGzUfSWpE/Ab4PqIWAgQESMi4uPAaOBqSQOafnBETIqImoio6d69e9PdRfXUU7B8uZ/uMrPqVC6BshzYI297T2BJ44YkATcCL0bEDU1PjogXgbnAJ0tc5yZNnQqdO6cBeTOzalMugTIDGCGph6SewGBgZt7+G4AVEfHD/JMk7Z373g84DJjfTvV+RETq7ho+PL0hb2ZWbbbOugCAiFgpaTzwZK5pHDBc0l7AU8DZwKuSRuX2XxYRtwN3SdoRWANc0NgVloXaWli8GC6/PKsKzMyyVRaBAhARk4HJLexu9k4qIg4sVT2FmjoVtt46TbdiZlaNyqXLq0Nr7O4aNiytf2JmVo0cKEXw3HPwyit+mdHMqpsDpQimTYNOneCEE7KuxMwsOw6UIpg6FYYMgRK/5mJmVtYcKFtgyhTo3Rv+9jeYPz9tm5lVq7J5yqujmTIFxo6FNWvS9ltvpW2A0aOzq8v7dyDmAAAINUlEQVTMLCu+Q2mj8eM3hEmjNWtSu5lZNXKgtNHixYW1m5lVOgdKG/XtW1i7mVmlc6C00Te+8dG2Ll1gwoT2r8XMrBw4UNroqafSmid9+oAE/frBpEkekDez6uWnvNrgqafggQfgqqvg4ouzrsbMrDz4DqUNLr8cunWDc87JuhIzs/LhQClQ493JRRd5mV8zs3wOlAL57sTMrHkOlAL47sTMrGVlEyiSRkl6TdICSWOa7Pu+pL9KWijpVklb59qHSHo5d17J31H33YmZWcvKIlAkdQUmAkfkvq6QlD937z+Bg4C9gd2BUyUJuBE4BTgAOEPSwcWubcoU6N8/TU//wANpES3fnZiZfVRZBAowApgTEcsiYgUwCxjWuDMiboqItRGxHnge2BUYCKyMiOcj4j1gKjCymEU1TgC5aFFalRHg7rs9q7CZWXPKJVD6AIvytpcCvZoeJKkLcBxwTwHnjJVUK6m2rq6uoKKamwDy/fc9AaSZWXPKJVA6Aw152w1Aff4BkjoBvwGuj4iFrTkHICImRURNRNR0L3AFLE8AaWbWeuUSKMuBPfK29wSWNG7kjZe8GBE3tOacYvAEkGZmrVcugTIDGCGph6SewGBgZt7+G4AVEfHDvLangAGSBkjaATgJuLOYRU2YkCZ8zOcJIM3MmlcWc3lFxMrcY79P5prGAcMl7UUKjrOBVyWNyu2/LCJul3QWaTylM3BVRCxq+tlbonGix/HjUzdX374pTDwBpJnZRykaH1+qAjU1NVFbW5t1GWZmHYqkeRFRs7njyqXLy8zMOjgHipmZFYUDxczMisKBYmZmReFAMTOzoqiqp7wk1bHxdC2NugGr2rmccuFrrz7Vet3ga2/rtfeLiM1ONVJVgdISSbWteSSuEvnaq+/aq/W6wdde6mt3l5eZmRWFA8XMzIrCgZJMyrqADPnaq0+1Xjf42kvKYyhmZlYUvkMxM7OicKCYmVUQSdtL2ieLn131gSJplKTXJC2QNCbrekpF0naSJkl6SdIiSefn2r8vaXGu/Zis6ywlSZ0lvSjpxtx2VVy7pJ0l/U7SMkmv5H4fquXaL5D099zf8e/m2iry2iXtJOmPwErg4rz2Zq9X0pWSlkp6QdLAohQREVX7BXQlrfK4B9ATWAF0z7quEl3rbsDJgEgvOK0EhgAv534f9gNeB7bJutYS/h78GLiPtPrnXtVy7aSls/8j92e/XbVcO9AfWAjskPvv/x/A/pV67cCOwDDgm8CNubZm/6yBo4DHSWtifQGYX4waqv0OZQQwJyKWRcQKYBbpD6TiRMSbETEtklWkID0SuCMi/hkRL5L+8hXnXyplRtIngU8Dd+SaTqQKrj1vBdQrcn/2H1Al1w6sy31vIP2P813gWCr02iPi3Yh4GFif19zSn/VJwOSIWB8RDwLdc/+tbJFqD5Q+bDwVy1KgV0a1tBtJB5D+pdqNKrh+SQKuA76f11wtf/b7A68B03JdHtdQJdceEctId6VPAQ8BXwH2pAquPU9Lf9ZN25dRhN+Hag+UzqR/vTRqAOozqqVdSOoG3Ap8g+q5/m8DsyNiQV5btVx7D1JXx7nAocDhwJeogmuXtBPwVdI/JP4TuJDq+XNv1NL1luT3oSzWlM/QcmBo3vaewNPZlFJ6kj4G3ANcGhHP5Abo9sg7ZE9SV1ilOR3oKunLwK6kPvXrqI5rfwOYFxFLASQ9SPofRzVc+9eA5yNiNjBb0omkcdJquPZGy2n+epu29ybdvWyRar9DmQGMkNQjr695ZsY1lUTuX2t3AxMi4v5c83TgNEldcmMMuwLzs6qxVCJicEQcGBEHAz8E7gLupQqundTds5+k3pK2BY4mjSVUw7V/ABwsaRtJXYF9SF1f1XDtjVr6Oz4dOEPSVpK+ALwcEau39IdV9R1KRKyUNB54Mtc0LiLey7KmEvo3UpfHtZKuzbUNB24D/kL6y/fNyD0aUukiYp6kir/2iHhP0rnAg8C2pIHYiblwqehrJ/23fRTwKvA+cEtEzK3UP/dcaD5LeqJrO0lDgW/RzN9xSXeRnvJ8FXiT1DW45TVUyO+lmZllrNq7vMzMrEgcKGZmVhQOFDMzKwoHipmZFYUDxczMisKBYmZmReFAMTOzonCgmJlZUThQzMysKBwoZmZWFA4UMzMrCgeKWTuRNFTS73PreH8oabmkGZJOkPTvkiI3kWNz534id86fcguGmZWdqp5t2Ky95GZ4/j6wijR1+HKgH2nG58HA7NyhB7TwEVeRFkU6v1Jmx7XK40AxKzFJV5DCZBpwRv4SCZJ2BHZjw2p5+zdz/uHAKaS1weeWvmKztvH09WYlJOkQoBb4M3B4RKzdxLFvAp0i4mN5bSItknUQsG9ELCxtxWZt5zEUs9K6gPT37JJNhUnO88AukvbMa/sK8BngWoeJlTsHillpjQDeAh5pxbHP5b7vDyBpO+AK0rrwV5SkOrMicqCYlUguELoDiyKioRWnPJ/73jgwfx5p4P6yiHinBCWaFZUDxax0Gh/v7dHK4xvvUA6Q1AP4d+B/gV8XuzCzUnCgmJVIRLxPCoTekkY13S9pH0lb5TX9hfS01/7A5cBOwAURUd/0XLNy5Ke8zEpI0nHA3aS7lftJobELcDDQJyJ6NTn+ReATpEf6H4iIL7ZvxWZt5zsUsxKKiOnAUFKYfBY4H/hX4G3gwmZOeQ7YFogW9puVLd+hmJlZUfgOxczMisKBYmZmReFAMTOzonCgmJlZUThQzMysKBwoZmZWFA4UMzMrCgeKmZkVhQPFzMyKwoFiZmZF4UAxM7Oi+P/gdhYMizDZCAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(cvs, scores, 'b-o')\n",
    "plt.xlabel('$cv$', fontsize = 20)\n",
    "plt.ylabel('$Score$', fontsize = 20)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "\n",
    "> # 泰坦尼克号数据分析\n",
    "\n",
    "王成军\n",
    "\n",
    "wangchengjun@nju.edu.cn\n",
    "\n",
    "计算传播网 http://computational-communication.com"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:31:28.492497Z",
     "start_time": "2018-05-29T07:31:28.488728Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn import tree\n",
    "import warnings  \n",
    "warnings.filterwarnings(\"ignore\")  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-06T07:02:49.855926Z",
     "start_time": "2018-06-06T07:02:49.705773Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "train = pd.read_csv('../data/tatanic_train.csv', \n",
    "                    sep = \",\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-06T07:02:52.803564Z",
     "start_time": "2018-06-06T07:02:52.759733Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  PassengerId  Survived  Pclass  \\\n",
       "0           0            1         0       3   \n",
       "1           1            2         1       1   \n",
       "2           2            3         1       3   \n",
       "3           3            4         1       1   \n",
       "4           4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:28:58.070575Z",
     "start_time": "2018-05-29T07:28:57.897862Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "train[\"Age\"] = train[\"Age\"].fillna(train[\"Age\"].median())\n",
    "train[\"Fare\"] = train[\"Fare\"].fillna(train[\"Fare\"].median())\n",
    "#Convert the male and female groups to integer form\n",
    "train[\"Sex\"][train[\"Sex\"] == \"male\"] = 0\n",
    "train[\"Sex\"][train[\"Sex\"] == \"female\"] = 1\n",
    "#Impute the Embarked variable\n",
    "train[\"Embarked\"] = train[\"Embarked\"].fillna('S')\n",
    "#Convert the Embarked classes to integer form\n",
    "train[\"Embarked\"][train[\"Embarked\"] == \"S\"] = 0\n",
    "train[\"Embarked\"][train[\"Embarked\"] == \"C\"] = 1\n",
    "train[\"Embarked\"][train[\"Embarked\"] == \"Q\"] = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:28:08.358884Z",
     "start_time": "2018-05-29T07:28:08.346226Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.12294397  0.31274009  0.23680307  0.32751287]\n",
      "0.977553310887\n"
     ]
    }
   ],
   "source": [
    "#Create the target and features numpy arrays: target, features_one\n",
    "target = train['Survived'].values\n",
    "features_one = train[[\"Pclass\", \"Sex\", \"Age\", \"Fare\"]].values\n",
    "\n",
    "#Fit your first decision tree: my_tree_one\n",
    "my_tree_one = tree.DecisionTreeClassifier()\n",
    "my_tree_one = my_tree_one.fit(features_one, target)\n",
    "#Look at the importance of the included features and print the score\n",
    "print(my_tree_one.feature_importances_)\n",
    "print(my_tree_one.score(features_one, target))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:28:15.915998Z",
     "start_time": "2018-05-29T07:28:15.705994Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "test = pd.read_csv('../data/tatanic_test.csv', sep = \",\")\n",
    "# Impute the missing value with the median\n",
    "test.Fare[152] = test.Fare.median()\n",
    "test[\"Age\"] = test[\"Age\"].fillna(test[\"Age\"].median())\n",
    "#Convert the male and female groups to integer form\n",
    "test[\"Sex\"][test[\"Sex\"] == \"male\"] = 0\n",
    "test[\"Sex\"][test[\"Sex\"] == \"female\"] = 1\n",
    "\n",
    "#Impute the Embarked variable\n",
    "test[\"Embarked\"] = test[\"Embarked\"].fillna('S')\n",
    "#Convert the Embarked classes to integer form\n",
    "test[\"Embarked\"][test[\"Embarked\"] == \"S\"] = 0\n",
    "test[\"Embarked\"][test[\"Embarked\"] == \"C\"] = 1\n",
    "test[\"Embarked\"][test[\"Embarked\"] == \"Q\"] = 2\n",
    "\n",
    "# Extract the features from the test set: Pclass, Sex, Age, and Fare.\n",
    "test_features = test[[\"Pclass\",\"Sex\", \"Age\", \"Fare\"]].values\n",
    "\n",
    "# Make your prediction using the test set\n",
    "my_prediction = my_tree_one.predict(test_features)\n",
    "\n",
    "# Create a data frame with two columns: PassengerId & Survived. Survived contains your predictions\n",
    "PassengerId =np.array(test['PassengerId']).astype(int)\n",
    "my_solution = pd.DataFrame(my_prediction, PassengerId, columns = [\"Survived\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:28:18.081288Z",
     "start_time": "2018-05-29T07:28:18.074414Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>892</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>893</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>894</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Survived\n",
       "892         0\n",
       "893         0\n",
       "894         1"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_solution[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:25:44.488717Z",
     "start_time": "2018-05-29T07:25:44.484381Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(418, 1)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check that your data frame has 418 entries\n",
    "my_solution.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Write your solution to a csv file with the name my_solution.csv \n",
    "my_solution.to_csv(\"../data/tatanic_solution_one.csv\", \n",
    "                   index_label = [\"PassengerId\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:28:26.996353Z",
     "start_time": "2018-05-29T07:28:26.982601Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.905723905724\n"
     ]
    }
   ],
   "source": [
    "# Create a new array with the added features: features_two\n",
    "features_two = train[[\"Pclass\",\"Age\",\"Sex\",\"Fare\",\\\n",
    "                      \"SibSp\", \"Parch\", \"Embarked\"]].values\n",
    "\n",
    "#Control overfitting by setting \"max_depth\" to 10 and \"min_samples_split\" to 5 : my_tree_two\n",
    "max_depth = 10\n",
    "min_samples_split = 5\n",
    "my_tree_two = tree.DecisionTreeClassifier(max_depth = max_depth, \n",
    "                                          min_samples_split = min_samples_split, \n",
    "                                          random_state = 1)\n",
    "my_tree_two = my_tree_two.fit(features_two, target)\n",
    "\n",
    "#Print the score of the new decison tree\n",
    "print(my_tree_two.score(features_two, target))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:28:28.033226Z",
     "start_time": "2018-05-29T07:28:28.018293Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.979797979798\n"
     ]
    }
   ],
   "source": [
    "# create a new train set with the new variable\n",
    "train_two = train\n",
    "train_two['family_size'] = train.SibSp + train.Parch + 1\n",
    "\n",
    "# Create a new decision tree my_tree_three\n",
    "features_three = train[[\"Pclass\", \"Sex\", \"Age\", \\\n",
    "                        \"Fare\", \"SibSp\", \"Parch\", \"family_size\"]].values\n",
    "\n",
    "my_tree_three = tree.DecisionTreeClassifier()\n",
    "my_tree_three = my_tree_three.fit(features_three, target)\n",
    "\n",
    "# Print the score of this decision tree\n",
    "print(my_tree_three.score(features_three, target))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:28:32.678968Z",
     "start_time": "2018-05-29T07:28:32.465958Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.939393939394\n",
      "418\n",
      "[0 0 0]\n"
     ]
    }
   ],
   "source": [
    "#Import the `RandomForestClassifier`\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "#We want the Pclass, Age, Sex, Fare,SibSp, Parch, and Embarked variables\n",
    "features_forest = train[[\"Pclass\", \"Age\", \"Sex\", \"Fare\", \"SibSp\", \"Parch\", \"Embarked\"]].values\n",
    "\n",
    "#Building the Forest: my_forest\n",
    "n_estimators = 100\n",
    "forest = RandomForestClassifier(max_depth = 10, min_samples_split=2, \n",
    "                                n_estimators = n_estimators, random_state = 1)\n",
    "my_forest = forest.fit(features_forest, target)\n",
    "\n",
    "#Print the score of the random forest\n",
    "print(my_forest.score(features_forest, target))\n",
    "\n",
    "#Compute predictions and print the length of the prediction vector:test_features, pred_forest\n",
    "test_features = test[[\"Pclass\", \"Age\", \"Sex\", \"Fare\", \"SibSp\", \"Parch\", \"Embarked\"]].values\n",
    "pred_forest = my_forest.predict(test_features)\n",
    "print(len(test_features))\n",
    "print(pred_forest[:3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-05-29T07:26:25.602062Z",
     "start_time": "2018-05-29T07:26:25.572689Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.14130255  0.17906027  0.41616727  0.17938711  0.05039699  0.01923751\n",
      "  0.0144483 ]\n",
      "[ 0.10384741  0.20139027  0.31989322  0.24602858  0.05272693  0.04159232\n",
      "  0.03452128]\n",
      "0.905723905724\n",
      "0.939393939394\n"
     ]
    }
   ],
   "source": [
    "#Request and print the `.feature_importances_` attribute\n",
    "print(my_tree_two.feature_importances_)\n",
    "print(my_forest.feature_importances_)\n",
    "\n",
    "#Compute and print the mean accuracy score for both models\n",
    "print(my_tree_two.score(features_two, target))\n",
    "print(my_forest.score(features_two, target))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 阅读材料\n",
    "机器学习算法的要点（附 Python 和 R 代码）http://blog.csdn.net/a6225301/article/details/50479672\n",
    "\n",
    "The \"Python Machine Learning\" book code repository and info resource https://github.com/rasbt/python-machine-learning-book\n",
    "\n",
    "An Introduction to Statistical Learning (James, Witten, Hastie, Tibshirani, 2013) : Python code https://github.com/JWarmenhoven/ISLR-python\n",
    "\n",
    "BuildingMachineLearningSystemsWithPython https://github.com/luispedro/BuildingMachineLearningSystemsWithPython"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 作业\n",
    "https://www.datacamp.com/community/tutorials/the-importance-of-preprocessing-in-data-science-and-the-machine-learning-pipeline-i-centering-scaling-and-k-nearest-neighbours"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python [conda env:anaconda]",
   "language": "python",
   "name": "conda-env-anaconda-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 0,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "780px",
    "left": "1279px",
    "top": "168.667px",
    "width": "341px"
   },
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}