{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 内容概要\n",
    "这一节我们介绍以下几个内容：\n",
    "- 我们该怎样选择模型用于监督学习任务？\n",
    "- 我们该如何选择调整得到最好的模型参数？\n",
    "- 我们该如何对测试数据进行预测估计？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 使用整个数据集进行训练和测试\n",
    "- 这里我们使用手中的整个数据集来训练模型\n",
    "- 使用同样的数据集来测试模型，然后评估预测的结果和真实结果的差别"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "iris = load_iris()\n",
    "\n",
    "# create X(features) and y(response)\n",
    "X = iris.data\n",
    "y = iris.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logistic regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted response:\n",
      "[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 2 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression()\n",
    "\n",
    "logreg.fit(X, y)\n",
    "y_pred = logreg.predict(X)\n",
    "print \"predicted response:\\n\",y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "150"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 分类准确率\n",
    "现在我们需要使用一种度量方式来评价我们的模型的运行情况，我们使用正确预测的比例来作为评估的度量(evaluation metric)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.96\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "print metrics.accuracy_score(y, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上说明对于训练的数据，我们有96%的数据预测正确。这里我们使用相同的数据来训练和预测，使用的度量称其为**训练准确度**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### KNN(K=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.966666666667\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "knn5 = KNeighborsClassifier(n_neighbors=5)\n",
    "knn5.fit(X, y)\n",
    "y_pred = knn5.predict(X)\n",
    "print metrics.accuracy_score(y, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### KNN(K=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "knn1 = KNeighborsClassifier(n_neighbors=1)\n",
    "knn1.fit(X, y)\n",
    "y_pred = knn1.predict(X)\n",
    "print metrics.accuracy_score(y, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面我们得到了训练准确度为100%的模型，貌似得到了最好的模型和参数。但我们回想一下KNN算法的原理，KNN算法寻找训练数据中的K个最近的数据，它使用指向最多的那个类别来作为预测的输出。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "这下我们就明白了为什么当K=1的时候KNN算法的训练准确度为1，KNN会查找在训练数据集中的最近的观测，训练得到的模型会在相同的数据集中找到相同的观测。换句话说，KNN算法已经记住了训练数据集，因为我们使用同样的数据作为测试的数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###小结\n",
    "我们要明确训练模型的目的是得到对于未来的观测数据的良好的预测性能，如果我们只是试图最大化训练的准确率，很有可能得到的是针对训练数据的过于复杂的模型，这种模型不具有很好的泛化能力，也就是说对于未知数据的预测得不到很好的性能。\n",
    "\n",
    "构建没有必要的过于复杂的模型被称作过拟合。这样的模型过分地拟合噪声数据，胜过去拟合信号数据。在KNN模型中，小K值的模型会生成复杂度很高的模型，因为它跟随数据中的噪声。\n",
    "\n",
    "下面这个是过拟合的一幅图像：\n",
    "![](http://blog.kaggle.com/wp-content/uploads/2015/05/05_overfitting-300x300.png)\n",
    "这里绿色的分割线努力去学习噪声的规律，而不像黑色的分割线去学习数据信号的规律，这样对比之下，黑色的线的泛化能力应该更好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 分别设置训练集和测试集\n",
    "- 将数据集分成两部分：训练集和测试集\n",
    "- 使用训练集对模型进行训练\n",
    "- 使用测试集进行模型的测试并评估性能如何"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(150, 4)\n",
      "(150,)\n"
     ]
    }
   ],
   "source": [
    "print X.shape\n",
    "print y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 第一步：将X和y分割成训练和测试集\n",
    "from sklearn.cross_validation import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=4)\n",
    "# 这里的random_state参数根据给定的给定的整数，得到伪随机生成器的随机采样"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://blog.kaggle.com/wp-content/uploads/2015/05/05_train_test_split-e1431626237154.png)\n",
    "上面这个图告诉我们train_test_split函数的功能，将一个数据集分成两部分。这样使用不同的数据集对模型进行分别的训练和测试，得到的测试准确率能够更好的估计模型对于未知数据的预测效果。\n",
    "\n",
    "这里测试数据的大小没有统一的标准，一般选择数据集的20%-40%作为测试数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(90, 4)\n",
      "(60, 4)\n"
     ]
    }
   ],
   "source": [
    "print X_train.shape\n",
    "print X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(90,)\n",
      "(60,)\n"
     ]
    }
   ],
   "source": [
    "print y_train.shape\n",
    "print y_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr',\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 第二步：使用训练数据训练模型\n",
    "logreg = LogisticRegression()\n",
    "logreg.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.95\n"
     ]
    }
   ],
   "source": [
    "# 第三步： 针对测试数据进行预测，并得到测试准确率\n",
    "y_pred = logreg.predict(X_test)\n",
    "\n",
    "print metrics.accuracy_score(y_test, y_pred)\n",
    "# 可以尝试一下，对于上面不同数据集分割，得到的测试准确率不同"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 使用KNN算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.966666666667\n"
     ]
    }
   ],
   "source": [
    "# K=5\n",
    "\n",
    "knn5 = KNeighborsClassifier(n_neighbors=5)\n",
    "knn5.fit(X_train, y_train)\n",
    "y_pred = knn5.predict(X_test)\n",
    "print metrics.accuracy_score(y_test, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.95\n"
     ]
    }
   ],
   "source": [
    "# K=1\n",
    "\n",
    "knn1 = KNeighborsClassifier(n_neighbors=1)\n",
    "knn1.fit(X_train, y_train)\n",
    "y_pred = knn1.predict(X_test)\n",
    "print metrics.accuracy_score(y_test, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们能找到一个比较好的K值吗？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 测试从K=1到K=25，记录测试准确率\n",
    "k_range = range(1, 26)\n",
    "test_accuracy = []\n",
    "\n",
    "for k in k_range:\n",
    "    knn = KNeighborsClassifier(n_neighbors=k)\n",
    "    knn.fit(X_train, y_train)\n",
    "    y_pred = knn.predict(X_test)\n",
    "    test_accuracy.append(metrics.accuracy_score(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x71f1df0>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEPCAYAAACDTflkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xv0XGV97/H3h0CAcAkB03ALbOwBDCwUpCe1Xo5RaU+0\nVQStSF2VCqVpl6BVzyqW0yWptqdggZoWS7GiYIuiVbHRVgE9hOppy8WkIUASAbNDEiBc5BYuQsj3\n/LGfITvD7zIzv9mz5/J5rZX1m31/ZtjMd57nu5/nUURgZmbWiZ3qLoCZmQ0uBxEzM+uYg4iZmXXM\nQcTMzDrmIGJmZh1zEDEzs45VGkQkLZS0RtJdks4ZY/ssSddIWinpJklHl7Z9RNLtklZJ+rKkXdP6\nxZI2SlqR/i2s8j2Ymdn4KgsikqYBlwALgaOAUyXNa9rtXGB5RLwKeD+wJB17EHA2cHxEHANMA96b\njgng4og4Lv37XlXvwczMJlZlTWQ+cHdE5BHxPHA1cGLTPvOAGwAiYi2QSZqdtu0MzJC0MzAD2FQ6\nThWW28zMWlRlEDkI2FBa3pjWla0ETgaQNB84FDg4IjYBFwH3AvcBj0XE90vHnZ2awC6XtE9Vb8DM\nzCZWZRBpZTyV84F9JK0AzgJWAC9ImgW8A8iAA4E9Jb0vHXMpcBhwLHA/RbAxM7Ma7FzhuTcBc0vL\ncylqIy+KiCeB0xvLktYBPwXeCqyLiEfS+m8CrwWuiogHS/t/Hvj2WBeX5EHBzMzaFBFtpQuqDCK3\nAodLyiiapE4BTi3vIGkm8ExEPCfpTODGiNgiaT3wGkm7A88CJwA3p2MOiIj70ylOAlaNV4B2P4xh\nJWlxRCyuuxx18+ewnT+L7fxZbNfJj+/KgkhEbJV0FnAtxdNVl0fEakmL0vbLKJ7auiIV/HbgjLTt\nZklfB5YDW9Pfz6VTXyDpWIrmsnXAoqreg5mZTazKmggR8V3gu03rLiu9/g/gyHGOXQwsHmP9+7ta\nSDMz65h7rI+GZXUXoE8sq7sAfWRZ3QXoI8vqLsAg07BOSiUpnBMxM2tdJ9+bromYmVnHHETMzKxj\nDiJmZtaxSp/OssEksT9wTN3lsKGxLoK7q76IxD7AtAgeqfpatp2DiI3lz4DXUAwrYzYVewEvAK/r\nwbU+CuyZ/lqPOIjYWF4OfCSC6+suiA02iYOBW3p0uV8E9ujRtSxxELGxZEBecxlsONwP7CuxewTP\nVHytDAeRnnNi3XYgsTPFkP331l0WG3wRvEBxLx3Sg8tl6Z/1kIOINTsQeCiCn9ddEBsaORV/uUvs\nCrwMmJYS7NYjDiLW7DDclGXdlVPcV1U6lGKqiXU9uJaVOIhYswwHEeuudVTfzJSl6/TiWlbiIGLN\nMor/Ec26Jac3QSTv0bWsxEHEmmW4JmLdleMgMrQcRKyZcyLWbTnV5yka920vrmUl7idizTIcRKy7\nHgD2lpgRwdMVXSOjaIZ9CtdEeso1EXtR6iNyALCh7rLY8IhgG0VfkUMrvExGqTlLwnMJ9YiDiJUd\nDGyO4Lm6C2JDJ6eiGoLE7sAs4P4IHgO2pWXrAQcRK3M+xKqSU12u4lBgQ6rxVH0ta+IgYmUZDiJW\njSr7b2Ts+Fi6+4r0kIOIlWW4j4hVI6faIJL36FrWxEHEyjJcE7Fq5FTXxJThIFKbSoOIpIWS1ki6\nS9I5Y2yfJekaSSsl3STp6NK2j0i6XdIqSV+WtGtav6+k6yX9RNJ1kjzYWvc4J2JVyanui735vs1x\nTqRnKgsikqYBlwALgaOAUyXNa9rtXGB5RLwKeD+wJB17EHA2cHxEHANMA96bjvk4cH1EHAH8IC1b\nd2S4OcuqsRnYQ2LPCs6d4ZxIbaqsicwH7o6IPCKeB64GTmzaZx5wA0BErAUySbPTtp2BGZJ2BmYA\nm9L6dwBXptdXAu+s7i2MDonpwByKkVDNuiqCANZTTV+RjB1rIutxX5GeqTKIHMSOndY2pnVlK4GT\nASTNp7jBDo6ITcBFFB2U7gMej4jvp2PmRMTm9HozxRefTd3BFM/Zb627IDa0crrczCQxA9ib4rsA\ngAgeB54D9uvmtWxsVQ57Ei3scz6wRNIKYBWwAnhB0iyKGkcGPA78k6T3RcRVO1wgIiSNex1Ji0uL\nyyJiWVvvYLQ4H2JVy+l+M1MG3FvqI1K+1mHAw12+3lCRtABYMJVzVBlENgFzS8tzaWoqiYgngdMb\ny5LWAT8F3gqsi4hH0vpvAq8FrgI2S9o/Ih6QdADw4HgFiIjF3XkrIyHD+RCrVhW5ioyx79vGtW7p\n8vWGSvphvayxLOm8ds9RZXPWrcDhkjJJ04FTgKXlHSTNTNuQdCZwY0RsoWjTfI2k3SUJOAG4Mx22\nFDgtvT4N+FaF72GUZLgmYtXKqSaI5D26lo2hsiASEVuBs4BrKQLAVyNitaRFkhal3Y4CVklaA/xP\n4MPp2JuBrwPLgdvSvp9Lf88HflXST4A3p2WbugwHEatWTvcfvc1wEKmVIlpJXQweSRERfjqjRRI/\nAv53BDfWXRYbThKzgTUR3Ut4S/wT8I0Irm5a/3bgDyJ4W7euNQo6+d50j3VryHBOxKr1MLCbxN5d\nPGfGxDkRq5iDiCGxKzCb4nFqs0qkviI53e0rkjF2c5b7ivSIg4hB8eTcJvcRsR7I6VJeJPV+34Mx\nntCM4EngaYofR1YhBxED9xGx3ulmM1MGrE81nLHkeAytyjmIGDgfYr2T090gMtF967xIDziIGPjx\nXuudnO4GkbxH17JxOIgYOIhY7+R0r4kpw0Gkdg4iBs6JWO90s4lpsvs2xzmRyjmIGDgnYr3zM2Bn\niW5MJpfhnEjtHERGnMRuFENm3193WWz4lfqKZF04XcbENZH1wKHuK1ItBxE7BNgQwQt1F8RGRs4U\ng0jq9b4bEwz1HsFTwJN4zqFKOYiY8yHWa91oZsqAfII+Ig05zotUykHEMpwPsd7K6U4QaeW+dV6k\nYg4iluGaiPVWztRrBxmt3bc5DiKVchCxDAcR662cLjVn9ehaNgEHEXNOxHptHVMfYfcwWm/Ock6k\nQg4iluGciPXWY+nvVPqKZLgm0hccREaYxO4U/yM/UHdZbHSU+opMpYaQ0VoQWQ8cIvm7rir+YEdb\nBtwbwba6C2Ijp+OnplJv950per9PKIJnKGo+B3RyLZucg8hoy3A+xOqR03kzUwasa6GPSIMf862Q\ng8hoy3A+xOqRM7UgkvfoWjYJB5HRluGaiNUjp/OcSIaDSN9wEBltfrzX6jKVJqaM9oOIH/OtSKVB\nRNJCSWsk3SXpnDG2z5J0jaSVkm6SdHRaf6SkFaV/j0v6UNq2WNLG0raFVb6HIZfhIGL1WE/nfUVa\n7SPS4JxIhXau6sSSpgGXACcAm4BbJC2NiNWl3c4FlkfESZKOBD4LnBARa4Hj0nl2Ssdfk44J4OKI\nuLiqso+QDOdErAYRPCaxFdgXeKTNwzPcnNU3qqyJzAfujog8Ip4HrgZObNpnHnADQAocmaTZTfuc\nANwTERtK6zw/wBRJ7AHsBWyuuyw2snLabGZKNZeM9oLIvcBciWntXMtaU2UQOQgof/FvTOvKVgIn\nA0iaDxwKHNy0z3uBLzetOzs1gV0uqRszpI2iDFjfxmOSZt3WSTNT4//3xybcqySCZylqOwe2eS1r\nQWXNWdDSl9P5wBJJK4BVwArYPjmSpOnA24FyPuVS4JPp9aeAi4Azxjq5pMWlxWURsazFso+CDOdD\nrF457QeRw2ivj0hDI2BtmGS/kSJpAbBgKueoMohsAuaWludS1EZeFBFPAqc3liWtA35a2uWtwI8j\n4qHSMQ+W9v888O3xChARizss+yjIcD7E6pUDR7Z5TEZnP37ydOwPOzh2aKUf1ssay5LOa/ccVTZn\n3QocLilLNYpTgKXlHSTNTNuQdCZwY0RsKe1yKvCVpmPKwxecRFGDsfZluCZi9cppvyaSMbUgYl1W\nWU0kIrZKOgu4FpgGXB4RqyUtStsvA44CrpAUwO2UmqUk7UGRVD+z6dQXSDqWorlsHbCoqvcw5A6j\nCPRmdekkJ5KxY2tFq3LgVzo4ziahiOHMq0qKiPBTXOOQuBX4YAQ31V0WG00SewP3A3u2muOQ+Dbw\n+Qj+uc1rnQCcG8Gb2y/p6Ojke9M91kdXhnMiVqMIngCeBZof659Ihpuz+oqDyAiS2AuYATw02b5m\nFctp8cu91EdkfQfX2QAcJFX6MNFIchAZTRmQu4+I9YF28iL7AVsjWu8j0hDBzyl+NDX3VbMpmjSI\nSLq4MaaVDY0MP5ll/SGn9SCSMbUmWI+hVYFWaiKrgc9JulnS70uaWXWhrHIZzodYf8hpfeiTjKn9\n+MlxEOm6SYNIRPx9RLwOeD/Ff4BVkr4s6U1VF84qk+GaiPWHdmoHGQ4ifaelnEgakfcVFAMmPkQx\n5tVHJX21wrJZdTyPiPWLnNa/2Kd63+Z4XpGuayUn8lfAWuBtwJ9HxPERcUFEvB04tuoCWiUyHESs\nP6wHDm1xXpEM50T6TiuPu90G/ElEPDXGtl/ucnmsNzKcE7E+EMEWiaeAOcADk+ye4easvtNKc9bj\nwC6NBUn7SHonQES0/aid1UtiJjCd9icCMqvKpDWEKfYRadgIHOC+It3VShA5rxws0uvFlZXIqpbh\nPiLWX3ImryHMBp5Nvdw7EsFzFJOwzZ1sX2tdK0FkrLZKzxA2uDKcD7H+kjN5EMnoThOs8yJd1koQ\n+XHqcPiLkv5bSrT/uOqCWWUynA+x/pIz+VNTGd358ZPjINJVrQSRs4Hnga9SzJP+LPDBKgtllcpw\nTcT6Syu1gwwHkb40aYIpTRJ1zmT72cA4DPhR3YUwK8mZ/Iv9MOCOLl3LHaW7aNIgIukXgD+imEBq\n97Q6IsLj8g+mDNdErL80+orsFMG2cfbJgO904VrrgA904TyWtNKcdRWwBng5xVNZOZ4Rb5BlOCdi\nfSSCpym6Euw/wW4Zbs7qS60Ekf0i4vPAcxFxY0R8ADw72CCS2Ifiv/mjdZfFrMm4eZHUR+RQptZH\npGEjMEdiehfOZbQWRJ5Lfx+Q9BuSXg3MqrBMVp3DcB8R608549cQ5gBPRbBlqheJYCvFlLzuK9Il\nrfTc/DNJ+wAfA/4G2Bv4SKWlsqpkOB9i/Sln/Md8M7rbBNuo9dzTxXOOrAmDSBq994iI+A7wGLCg\nF4WyymQ4H2L9KQeOH2dbRnd//OQ4L9I1EzZnRcQLwKk9KotVL8M1EetPE/UVyXAQ6Vut5ER+JOkS\nSW+Q9GpJx6e8iA0ezyNi/Spn/C/2bt+3OZ5XpGtaCSLHAUcDnwQuAi5MfyclaaGkNZLukvSSDouS\nZkm6RtJKSTc15nKXdKSkFaV/j0v6UNq2r6TrJf1E0nUpX2OtyXBzlvWn9cBcacxx+TKqyYlYFyii\nmgd1Uj5lLXACsAm4BTg1IlaX9vlL4ImI+JSkI4HPRsQJTefZKR0/PyI2SPo08HBEfDoFplkR8fEx\nrh8R0cpENyMhPSb5OHBIBB7C3/qOxH3A/Ag2Nq1fC7wzgtVjH9n2dQ4B/j2Cg7txvmHSyfdmKz3W\nzwOCYjTfFyNORHxykkPnA3dHRJ7OczVwIuxwI8wDzk/nWyspkzQ7Ih4q7XMCcE9EbEjL7wDemF5f\nCSwDXhJE7CVmAdscQKyP5RQ1hBeDiMROwCF0p49Iw33AbIldI/h5F887klppznoq/dsCbKOYJjdr\n4biDgA2l5Y1pXdlK4GQASfMpOhQ1/zp4L/Dl0vKciNicXm+meIbcJud8iPW7nJd+t+wPPJF6tXdF\n6iuyiSI42RS1MgDjheXl1AR1XQvnbqWd7HxgiaQVwCpgBfBC6VrTgbczzgCQERGSxr2OpMWlxWUR\nsayFMg2rDOdDrL/lvDThnVHNfdvIi9xVwbkHhqQFTLHrRifTRO7BS2sUY9nEjr1C58KObZ0R8SRw\nemNZ0jrgp6Vd3gr8uKl5a7Ok/SPiAUkHAA+OV4CIWNxCOUdFhmsi1t/WAa9pWpdRzX2b4+Q66Yf1\nssZySl+0ZdLmLEmrSv/uoEiWL2nh3LcCh6c8x3TgFGBp07lnpm1IOhO4MQ0933Aq8JWm8y4FTkuv\nTwO+1UJZzEHE+l/OS7/YMxxE+lorNZG3l15vBTZHxPOTHRQRWyWdBVxLMZ3u5RGxWtKitP0yiuHl\nr0hNUrcDZzSOl7QHRVL9zKZTnw98TdIZFDfCe1p4D1Y0E3y/7kKYTSDnpV/sh1HNTKo5RUuHTdGk\nj/hKeg1wZ0Q8kZb3BuZFxE09KF/H/IjvjiRuB34rgtvqLovZWCR2BZ4AZkQUuVGJ64ELI7i2y9d6\nPfDpCF7bzfMOuk6+N1t5OuvvYIfRM59K62xApD4iGd19TNKsq9Ljtg+xY841w81Zfa2VIEJEbCu9\nfgHG7FVq/Ws/4LkIHq+7IGaTyElf7qmPyFzg3gqucz+wn8RuFZx7pLQSRNZJ+pCkXSRNl/RhdnyC\nyvqf+4jYoMjZ/pjvgcCjETzT7Yuk5rINFH3TbApaCSK/D7yO4pHdjRSP4P1elYWyrstwHxEbDDnb\nm5kyqr1vPYZWF7TS2XAzxeO5NrgyXBOxwbAOeH16nVHtfZvjIDJlrfQT+VJ5pNw08u4Xqi2WdVmG\ng4gNhpwdayJ5j65lHWqlOeuVEfHioH0R8Sjg+UQGi3MiNihytudEqr5vy9eyDrUSRCRp39LCvvjp\nrEGT4ZyIDYYNwIESO+OcyEBopcf6RcB/SPoaxXDwvwn8eaWlsq5xHxEbJBE8J/EAxWjeGW7O6nut\nJNa/JOnHwJspRuY9KSLurLxk1i2zgacjeLLugpi1KAd+kSKQVNFHpOEBYB+J3at4jHhUtNrZ8I6I\n+Bvge8C70kCMNhicD7FBk1N0K3i4ykmjIthGEaSyqq4xClp5OusgSR+VdAvFIInTKCaKssGQ4XyI\nDZZ1wJvozX3rvMgUjRtEJC2StIxirPn9KEbYvT8iFkfEqt4Uz7ogwzURGyw58Cv05r7NcRCZkoly\nIpcA/wH8VkTcAiB5UNwBlFHUIM0GRQ7sioPIQJioOesAigmhLpT0E0l/BuzSm2JZFzknYoMmb/pb\n9bXcV2QKxg0iEfFwRFwaEW8E3gI8RjE17RpJ/6dnJbSpynBOxAbLBuAFnBMZCJNOSvWSA6QjgPdG\nxCerKVJ39POkVBLfAn6jR5d7FpgTwVM9up7ZlEncBLwrgo0VX+dlFAPLttJnri7PAFkEj1R9oU6+\nN9sOIoOiz4PIeoqpf3vxSysas8SZ2UtJTKPoSN2v/hP4YASVzybbyfdmP0ffoSSxC7A/kEewte7y\nmI26fv+RJbGOIm/Tl1OSt9TZ0LpqLnB/BM/XXRAzGwh9nbeZtCYi6XiK4U7KHgfWR4R/Sbcvw09L\nmVnrcuCVdRdiPK00Z30WOB64LS0fA9wBzJT0BxFxbVWFG1IZDiJm1roceEfdhRhPK81Z9wHHRsTx\nEXE8cCzFHOu/Cny6ysINKffbMLN25PRxX5ZWgsiREfHigItpBN9XRMQ9vLSZaweSFqZ+JXdJOmeM\n7bMkXSNppaSbJB1d2raPpK9LWi3pTkm/nNYvlrRR0or0b2HL77Y/ZLjfhpm1LgcOkfozh91Kc9Yd\nki4FrqZ4DO49wJ2SdoXxk8OSplEMnXICsAm4RdLSiFhd2u1cYHlEnCTpSIqmsxPStiXAv0bEuyXt\nDOyR1gdwcURc3PK77C8ZromYWYsieFriCWAOcH/d5WnWSmT7HeAe4A+BD1M0ZZ1GEUDePMFx84G7\nIyKPiOcpgtCJTfvMA24AiIi1QCZptqSZwBsi4gtp29aIeLx0XD8/0z2ZDAcRM2tPTp82aU0aRCLi\n6Yi4MCJOSv8uTOu2RcREEx0dRDF8QcPGtK5sJXAygKT5wKEUE9EcBjwk6YuSlkv6e0kzSsednZrA\nLpe0Twvvsy9I7Ar8AkXNzMysVTl9+phvK4/4vh44j+INNPaPiHj5JIe20hX+fGCJpBXAKmAFxZg5\n04FXA2dFxC2SPgN8HPgEcCnQGHLlUxTT954xTtkXlxaXRcSyFspUpbnAJncyNLM2VdJXRNICYMFU\nztFKTuRyiqas5dBWz85NFF+aDXNhx3FwUk3m9MaypHUUzWV7AhsbQ9ADX6cIIkTEg6X9Pw98e7wC\nRMTiNsrbCxluyjKz9uUUP6y7Kv2wXtZYlnReu+doJSfyWER8NyI2p5F9H46Ih1s47lbgcEmZpOnA\nKcDS8g6SZqZtSDoTuDEitkTEA8CGNNgjFMn2O9J+B5ROcRJFDWZQZDiImFn7cvo0J9JKTeQGSX8J\nfBO2z3ccEcsnOigitko6C7iWYkrdyyNitaRFaftlwFHAFZKCYuKkcrPU2cBVKcjcA3wgrb9A0rEU\nzWXrgEUtvId+4T4iZtaJnD7NiUw6im+aIvclO0XEmyoqU1f04yi+ElcB34vgH+oui5kNDondgUeB\nGRFsq+46FYziGxELOi6RNctwTcTM2hTBMxKPUsw421dPd44bRCT9dkT8g6SPsWNNRBRPZw1qZ786\nZTiImFlncoom8cEIIkCjX8ZetPa4rk1AYjfgZRRjkZmZtSun+CH6o3qLsaNxg0hKfAN8PyJ2KHTq\nO2LtOQTY0O8T4JhZ3+rLeUVaecT3b8ZY99fdLsgIyHBTlpl1LqcPg8hEOZFfAV4LzJb0UbaPV7UX\nxSO71p4MBxEz61xO0d+ur0yUE5nO9oCxV2n9E8C7qyzUkHIfETObir5szmqln8ihEbE+vZ4G7Nk0\nom5f6rd+IhJfAb4TwVV1l8XMBk8awPUJir4ileRWO/nebCUn8heS9pa0B8UQI3dK+qOOSjjaMlwT\nMbMORfBz4GHgwLrLUtZKEDk6Ip4A3gl8l+LL8LerLNSQynAQMbOpyemzMbRaCSI7S9qFIoh8O00w\n5X4jbUhDFsyiD2clM7OB0nd5kVaCyGUU0W9P4N8kZUDf50T6zKHAvVWOeWNmIyFn0IJIRPx1RBwU\nEW+NiG3AeqCvB1/sQxluyjKzqcsZtOYsSfunaWi/l1bNo5hj3Vrnx3vNrBtyBq0mAlwBXMf2JwLu\nAj5SVYGGVIaDiJlN3eDkRCQ1OiK+LCK+SpoaNyXWPUd4ezKK//hmZlOxAThQamlCwZ6YqCZyc/q7\nRdLLGislvQYn1tuV4ZqImU1RBM8BDwIH112WhomiWaPX4seAfwZeLunfgdl42JN2OSdiZt2S00c/\nTCcKIuWBF68B/jW9/jnwFmBl9cUbfBJ7UIw9trnuspjZUOirvMhEQaR54MWGGWOss/EdCqx3HxEz\n65KcAQkiD0TEn/asJMMro0+qnWY2FHLgjXUXoqGVR3xtapwPMbNuyumjmshEQeSEnpViuGU4iJhZ\n9/RVTmTcIBIRj0z15JIWSloj6S5J54yxfZakayStlHSTpKNL2/aR9HVJqyXdmR4tRtK+kq6X9BNJ\n10naZ6rlrFiG+4iYWfdsBPaX2KXugkCFzVlpAqtLgIXAUcCpkuY17XYusDwiXgW8H1hS2rYE+NeI\nmAe8Elid1n8cuD4ijgB+kJb7WYZrImbWJRE8DzwAzK27LFBtTmQ+cHdE5KmX+9XAiU37zANuAIiI\ntUAmabakmcAbIuILadvW0myK7wCuTK+vpBiivp85J2Jm3dY3TVpVBpGDKLroN2xM68pWAicDSJpP\n8TjswRRfvA9J+qKk5ZL+XlLj0eI5EdHoc7EZmFPVG5gqib0oHol+sO6ymNlQyemTIFLl+CutTFx1\nPrBE0gqKqXdXUIzRNR14NXBWRNwi6TMUzVaf2OECESFp3OtIWlxaXBYRy9p6B1N3KJBHeBIvM+uq\nnC4EEUkLgAVTOUeVQWQTO7bZzaWojbwoIp4ETm8sS1oH/JRiAqyNEXFL2vQNoJGY3yxp/4h4QNIB\nTPArPyIWT/VNTFGGm7LMrPtyipFDpiT9sF7WWJZ0XrvnqLI561bgcEmZpOnAKcDS8g6SZqZtSDoT\nuDEitkTEA8AGSUekXd8C3JFeL2X7fCanAd+q8D1MlfMhZlaFvsmJVFYTiYitks4CrqUYQuXyiFgt\naVHafhnFU1tXpCap24EzSqc4G7gqBZl7gA+k9ecDX5N0BsUX9Huqeg9dkOEgYmbdl9MnQUQRw9lc\nLykiQpPvWWUZ+AZwdQT/VGc5zGy4pPlEngL2juDn3Ttv+9+bHvakWhmuiZhZl0WwFbiPPugr4iBS\nLedEzKwqfZEXcRCpiMRMYFfg4brLYmZDKcdBZKgdCqxzHxEzq0hO0dpRKweR6mS4KcvMqpPjmshQ\ncz7EzKrknMiQy3AQMbPq5DiIDLUMzyNiZtW5D3iZxG51FsJBpDoZromYWUUieIFiPMJD6iyHg0h1\nnBMxs6rVnhdxEKmAxD4U44X9rO6ymNlQy3EQGUoZ7iNiZtXLqbmviININTLclGVm1XNz1pByPsTM\neiHHQWQoZTiImFn1chxEhlKG+4iYWfXuB2ZJ7F5XARxEqpHhmoiZVSyCbcC9FAO+1sJBpMskhHMi\nZtY7OTU2aTmIdN8sIIDH6i6ImY2EnBof83UQ6b4M9xExs97JcU1kqGS4KcvMeqfWviIOIt3nfIiZ\n9VKOg8hQyfDjvWbWOznDmhORtFDSGkl3STpnjO2zJF0jaaWkmyQdXdqWS7pN0gpJN5fWL5a0Ma1f\nIWlhle+hAxmuiZhZ72wG9pLYo46L71zViSVNAy4BTgA2AbdIWhoRq0u7nQssj4iTJB0JfDbtD8UT\nTgsionkk3AAujoiLqyr7FGU4iJhZj0SwTWI9RV+RO3t9/SprIvOBuyMij4jngauBE5v2mQfcABAR\na4FM0uzSdo1z7vHW18p9RMysJjk15UWqDCIHARtKyxvTurKVwMkAkuZTRNKD07YAvi/pVklnNh13\ndmoCu1zSPt0vesf2A56L4PG6C2JmIyWnprxIZc1Z0FI/ifOBJZJWAKuAFcALadvrI+K+VDO5XtKa\niPghcCnwybTPp4CLgDPGOrmkxaXFZRGxrO130Z4M10LMrPdyOqiJSFoALJjKhasMIpuAuaXluRS1\nkRdFxJMCxd2FAAAKQUlEQVTA6Y1lSeuAn6Zt96W/D0m6hqJ57IcR8WBp/88D3x6vABGxeMrvoj0Z\nDiJm1nvrgOPbPSj9sF7WWJZ0XrvnqLI561bgcEmZpOnAKcDS8g6SZqZtpCarGyNii6QZkvZK6/cA\nfo2ipoKkA0qnOKmxvk84H2JmdcipKSdSWU0kIrZKOgu4lmK+8csjYrWkRWn7ZcBRwBWSArid7c1S\nc4BrJDXKeFVEXJe2XSDpWIrmsnXAoqreQwcyYE3dhTCzkZNTU05EEcM5xJOkiIiePsUl8S/A30WM\n38RmZtZt6cnQp4A5ETzZ+Xna/950j/XuynBzlpn1WBrwNaeGeUUcRLok/RLIcBAxs3rk1JAXcRDp\nntnAM1OpSpqZTUFODXkRB5HuyXAtxMzqU8uQ8A4i3ZPhIGJm9clxEBlo7iNiZnXKcXPWQMvwPCJm\nVp8c10QGWoZrImZWn4eB6RIze3lRB5HuyXAQMbOa1NVXxEGkC0p9RNbXXBQzG205Pc6LOIh0xxzg\nyQi21F0QMxtpOT3OiziIdEeGm7LMrH497yviINIdGQ4iZla/HAeRgeQ+ImbWD3KcExlIGe4jYmb1\ny3FNZCBluCZiZvX7GbCTxD69uqCDSHe4OcvMalfqK5L16poOIlMksRNwCO4jYmb9IaeHeREHkanb\nH3gsgqfrLoiZGT1+zNdBZOoy3JRlZv0jx0FkoDgfYmb9JMdBZKBkOIiYWf/IGZaciKSFktZIukvS\nOWNsnyXpGkkrJd0k6ejStlzSbZJWSLq5tH5fSddL+omk6yT17FG2cWS4j4iZ9Y91QJYGhq1cZUFE\n0jTgEmAhcBRwqqR5TbudCyyPiFcB7weWlLYFsCAijouI+aX1Hweuj4gjgB+k5Tpl9HlNRNKCusvQ\nD/w5bOfPYrth+ywieAzYBszqxfWqrInMB+6OiDwingeuBk5s2mcecANARKwFMkmzS9vHiqTvAK5M\nr68E3tnVUrdvEHIiC+ouQJ9YUHcB+siCugvQRxbUXYAK5PSoSavKIHIQsKG0vDGtK1sJnAwgaT7F\nZCoHp20BfF/SrZLOLB0zJyI2p9ebKYZhr4XENGAu7iNiZv0lp0fJ9Z0rPHe0sM/5wBJJK4BVwArg\nhbTt9RFxX6qZXC9pTUT8cIcLRISkVq4zKYnfAd7V5mG7AI9E8Gw3ymBm1iU/BRan77VKKaIr38Ev\nPbH0GmBxRCxMy38MbIuICyY4Zh1wTERsaVp/HvBkRFwsaQ1FruQBSQcAN0TEK8Y4VzVvzMxsiEVE\nWwn5KmsitwKHS8qA+4BTgFPLO0iaCTwTEc+lJqsbI2KLpBnAtIh4UtIewK8Bf5oOWwqcBlyQ/n5r\nrIu3+0GYmVn7KgsiEbFV0lnAtcA04PKIWC1pUdp+GcVTW1ekWsPtwBnp8DnANZIaZbwqIq5L284H\nvibpDIp2v/dU9R7MzGxilTVnmZnZ8Bu6HuuTdXAcJeN12BwFkr4gabOkVaV1/dZRtSfG+SwWS9qY\n7o0VkhbWWcZekTRX0g2S7pB0u6QPpfUjd29M8Fm0dW8MVU0kdXBcC5wAbAJuAU6NiNW1Fqwm6UGF\n4yPiZ3WXpdckvQHYAnwpIo5J6z4NPBwRn04/MGZFRN2dVSs3zmfx4sMqtRauxyTtD+wfEf8laU/g\nxxR9zT7AiN0bE3wW76GNe2PYaiKtdHAcNSP5gEF6HPzRptX91lG1J8b5LGAE742IeCAi/iu93gKs\npui/NnL3xgSfBbRxbwxbEGmlg+MoGa/D5qjqm46qfeLsNG7d5aPQfNMsPTl6HHATI35vlD6L/0yr\nWr43hi2IDE/bXHe8LiKOA94KfDA1axhFR1VG+365lGJYjGOB+4GL6i1Ob6Xmm28AH46IJ8vbRu3e\nSJ/F1yk+iy20eW8MWxDZRDEMScNcitrISIqI+9Pfh4BrKJr7Rtnm1A5M6qj6YM3lqU1EPBgJ8HlG\n6N6QtAtFAPmHiGj0MxvJe6P0Wfxj47No994YtiDyYgdHSdMpOjgurblMtZA0Q9Je6XWjw+aqiY8a\neo2OqjBBR9VRkL4oG05iRO4NFZ3PLgfujIjPlDaN3L0x3mfR7r0xVE9nAUh6K/AZtndw/Iuai1QL\nSYdR1D5ge4fNkfksJH0FeCPwMoo27k8A/wx8DTiE1FE1Ih6rq4y9MsZncR7FyLXHUjTbrAMWlXIC\nQ0vS64F/A25je5PVHwM3M2L3xjifxbkUI4u0fG8MXRAxM7PeGbbmLDMz6yEHETMz65iDiJmZdcxB\nxMzMOuYgYmZmHXMQMTOzjjmI2ECT9H8l/VrTuj+U9LcTHLNM0vEVl+sraeyhDzetXyzpY+n1bmn4\n8U+McfxvSrpT0g+mUIYtpddvk7RW0iGpDE9Jmj3OvtskXVha/l9p1F+zl3AQsUH3FeC9TetOAb48\nwTGVjo2Uhs/4pYh4VUQsGevaaUSFbwC3RMQnxzjNGcDvRsRbWrzmWLOURtr2FmAJsDAi7k3bHgY+\n1rxv8hxwkqT9xthmtgMHERt03wB+vfElmkYjPTAifiTpUkm3pAl3Fo91cNMv8HdL+mJ6PVvS1yXd\nnP69doxjd5P0RRUTfy2XtCBtug44KE3o8/oxLrsLxTQFayPi3DHO+wngdcAXJF0gadexriPpdyQt\nTbWV68d5f/8D+Bzw6xGxLq0O4AvAKeOM0Pp8OuYjY53TrMxBxAZamnDrZuBtadV7ga+m1+dGxH8H\nXgW8UdIxY51inNdLgL+KiPnAuykGomv2QeCFiHglxVARV6YaxtuBeyLiuIj4UdMxAv4I+HlEfHSc\n9/RJinHgfisizgHOGuM6u6bdjwPeFRFvGuNUu1EMfXNiRPykadsWikDyh2OVAfhb4H2S9h5nuxng\nIGLDodykdUpahuKX9o+B5cDRwLw2znkCcImkFRRjbu0laUbTPq8D/hEgItYC64EjmHhCnwB+BLxW\n0uEtlmW86wRw/QRjPD0H/D/gd8cpx18Dp6WhwHfcWAyP/iXgQy2W0UaUg4gNg6XAWyQdB8yIiBVp\nAMqPAW+OiFcB/0Lxy7xZufaxe+m1gF9OtYnjImJuRDw9xvGdzA74bxRNRd9tDD/egvGu89QEx2yj\nmOp0vqQ/bj5fRDxOkTs6a5zjP0ORm9mjxTLaCHIQsYGXJtK5Afgi2xPqe1N8wT4haQ7FxFxj2Szp\nFZJ2ohj2uhFUrqP0K1zSsWMc+0PgfWn7ERQjwK5tsczfBC4Evidp5iS7j3WdNbQQwCLiWeDXKZqm\nTh9jl4uBRRQjPTcf+yjFyLZn4OS6jcNBxIbFV4Bj0l8iYiWwguLL9iqKJqSxfBz4DkWzz32l9R8C\nfik9pnsH8HtjHPu3wE6SbqNIlJ8WEc+nbRN96UYq499R5CyWlnIcYxnvOpM9Zda4zqPAQuBPJL29\nadsjwDeB6c3HJRdRDCFvNiYPBW9mZh1zTcTMzDrmIGJmZh1zEDEzs445iJiZWcccRMzMrGMOImZm\n1jEHETMz65iDiJmZdez/A5q1YIKD3xApAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x71ee230>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(k_range, test_accuracy)\n",
    "plt.xlabel(\"Value of K for KNN\")\n",
    "plt.ylabel(\"Testing Accuracy\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 模型参数的选择\n",
    "在这里，我们选择K的值为11，因为在K=11时，有较高的测试准确率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 这里我们对未知数据进行预测\n",
    "knn11 = KNeighborsClassifier(n_neighbors=11)\n",
    "knn11.fit(X, y)\n",
    "\n",
    "knn11.predict([3, 5, 4, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 小结\n",
    "- KNN的模型复杂度主要由K的值决定，K值越小，复杂度越高\n",
    "- 训练准确度随着模型复杂度更加复杂而升高\n",
    "- 测试准确率在模型过于复杂和过于简单的时候都比较低\n",
    "\n",
    "这种分割训练和测试数据集的模型评估流程的缺点是：**这种方式会导致待预测数据(out-of-sample)估计准确率的方差过高。**因为这种方式过于依赖训练数据和测试数据在数据集中的选择方式。一种克服这种的缺点的模型评估流程称为K折交叉验证，这种方法通过多次反复分割训练、测试集，对结果进行平均。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. K折交叉检验\n",
    "K折将数据集分成K个部分，其中K-1组数据作为构建预测函数的训练之用，剩余的一组数据作为测试之用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import KFold\n",
    "import numpy as np\n",
    "\n",
    "def cv_estimate(k, kfold=5):\n",
    "    cv = KFold(n = X.shape[0], n_folds=kfold)\n",
    "    clf = KNeighborsClassifier(n_neighbors=k)\n",
    "    score = 0\n",
    "    for train, test in cv:\n",
    "        clf.fit(X[train], y[train])\n",
    "        score += clf.score(X[test], y[test])\n",
    "        #print clf.score(X[test], y[test])\n",
    "    score /= kfold\n",
    "    return score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 测试从K=1到K=25，记录测试准确率\n",
    "k_range = range(1, 26)\n",
    "test_accuracy = []\n",
    "\n",
    "for k in k_range:\n",
    "    test_accuracy.append(cv_estimate(k, 5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.92666666666666653, 0.90666666666666662, 0.90666666666666662, 0.90666666666666662, 0.91333333333333333, 0.90666666666666662, 0.92000000000000015, 0.91333333333333333, 0.92000000000000015, 0.92000000000000015, 0.91333333333333344, 0.89333333333333331, 0.90666666666666662, 0.90000000000000002, 0.90000000000000002, 0.88666666666666671, 0.88000000000000012, 0.86666666666666659, 0.88666666666666671, 0.86666666666666681, 0.86666666666666681, 0.86666666666666681, 0.86666666666666681, 0.84666666666666668, 0.85999999999999999]\n"
     ]
    }
   ],
   "source": [
    "print test_accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7371530>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEPCAYAAABcA4N7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4nFW5/vHvTehKlSIlGLqgQCgGBJQAAYOoICCQw0Ga\nGH5HBAuKeCyxBlQUPCiJdBQBRarSgiYIKIGQkBAhGDChBCmCIoggkPv3x1qbTHZmz35n73n3zOx5\nPteVi3nrejIZ5pl3VdkmhBBC6MlSzQ4ghBBCa4tEEUIIoaZIFCGEEGqKRBFCCKGmSBQhhBBqikQR\nQgihplIThaTRkuZImivp5CrHV5N0laSZkqZKekfev3zevlfSbEnjyowzhBBCz1TWOApJQ4AHgVHA\nAuBuYIztByrO+S7wT9vfkLQ58CPbo/KxFW2/JGlp4HbgRNtTSwk2hBBCj8p8ohgBPGR7vu1XgcuA\n/bqdswUwGcD2g8AwSWvm7ZfyOcsCywALS4w1hBBCD8pMFOsBj1VsP573VZoJHAAgaQTwNmD9vD1E\n0r3AU8DNtu8uMdYQQgg9KDNRFKnTOhVYVdIM4HhgBvA6gO3XbQ8nJY4du9ovQgghDKylS7z3AmBo\nxfZQ0lPFG2y/ABzdtS1pHvCXbuc8L2kyMBr4U+UxSTFRVQgh9IFtFT23zEQxDdhU0jDgCeAQYEzl\nCZJWAf5t+z+SjgVutf2ipDWA12z/Q9IKwF6kp48l1POXHcwkjbM9rtlxtIJ4LxaJ92KReC8WqfdH\ndmmJwvZrko4HbgKGAOfZfkDS2Hx8IrAlcGEOejZwTL58HeCi3HNqKeBy29eXFWsIIYSelflEge0b\ngBu67ZtY8fqPwOZVrrsP2K7M2EIIIRQTI7MHjynNDqCFTGl2AC1kSrMDaCFTmh1AuyptwN1AkORo\nowghhPrU+90ZTxQhhBBq6rhEIfFNiQ2bHUcIIbSLjksUwAbA+5odRAghtItOTBSTSOMyQgghFNBx\njdkS65BGeK9pp+lCQgihk0Rjdi9s/kqaXmSHZscSQgjtoOMSRXYzsHezgwghhHbQqYki2ilCCKGg\njmujSNexImmdi3VtXmh8ZCGE0LqijaIAm5dIS7Pu1uxYQgih1XVkosii+imEEAqIRBFCCKGmTk4U\nM4C1pLRGdwghhOo6NlHkwXa/A0Y1O5YQQmhlpScKSaMlzZE0V9LJVY6vJukqSTMlTZX0jrx/qKTJ\nkv4kabakE0oIbxIxniKEEGoqtXtsXsr0QdKv9gWknkZjbD9Qcc53gX/a/oakzYEf2R4l6a3AW23f\nK+nNwD3A/t2u7dd6FBLDgLtSOSzs631CCKGdtFr32BHAQ7bn234VuAzYr9s5WwCTAWw/CAyTtKbt\nJ23fm/e/CDwArNvI4GzmA/8Atm7kfUMIYTApdc1sYD3gsYrtx4Edu50zEzgAuF3SCOBtwPrAM10n\nSBoGbAtMLSHGrt5P95Zw70FHYilAAzGhosRywFv6cOk/8liZEEIDlJ0oitRrnQqcKWkGcB+pN9Ib\nX0K52ukK4MT8ZLEYSeMqNqfYnlJnjJOA/wG+W+d1nerLwCiJUTavlFWIxErAHcBaUHe14H8kRtg8\n3fjIQmg/kkYCI/t8fcltFDsB42yPztunAAttn1bjmnnAVrZflLQM8GvgBttnVDm332tmS6xCetJZ\n0+bl/txrsJNYBngE+AtwPzDWLvRjoN5yliL9OPhbX8qQ+Drpf4pRNv9pdHwhtLtWa6OYBmwqaZik\nZYFDgGsrT5C0Sj6GpGOBW3OSEHAecH+1JNEoNs+TnmR2LauMQeRDwEPAPsDOwHEllfMlYG3g+D4m\nonHA34EzGxlUCJ2q1ERh+zXgeOAm0i/Qy20/IGmspLH5tC2B+yTNIS1RemLevwvw38DukmbkP6NL\nCjVGaRdzHDAhT6S4HzBO4r2NLEBif+BjwIF9fRrIPdgOB3aT+Hgj4wuhE3Xk7LFL3oddgR/abNeA\nsAYliU2APwBDu9omJPYCLgZ2tHm0AWW8g9QD7v020xpwv02B20lJ5/b+3i+EwaLVqp7axVRgY4m1\nmh1IC/s4cFFlA7bNJFIngKvz1O19JrE6cDXw2UYkiRzfXOAI4BcSQxtxzxA6UTxRvHEvrgUutbm0\nEfcbTHI31ceAXfKXb+UxkZ4qlgb+qy9tChJLA78BZtt8tgEhd7//50jtY++x+Xej7x9Cu4knir67\nmWin6MkBwKzuSQIgJ4aPA5sAn+vj/ccDApaY4qVBvgf8GTgnJ7YQQh0iUSwyCdgrvkiqOg6Y0NPB\n/Cv9w8CJEvvUc2OJw/K1h9q81q8oe47PpAbyLYDPlFFGCINZJIpF/kwaILh5swNpJRJbApsB19Q6\nz+Zx4GDgQonNCt57e+AHwP42z/U31l7ie4mUkE6SeF+ZZYUw2ESiyPKvzugmu6SPA+fbvNrbiTZ3\nkMZAXCOxcq1zJdYGriINqJvdkEh7j+9RUjK7OPfiCiEUEIlicZEoKuSeTIcD5xS9xuYcUhfXn+UR\n1tXuuyzwK1ICuqoRsdYR323AV0nJbKWBLDuEdhW9nha7H2uSRh6vUeQX9GAncSTwEZt967xuWVLS\n/b3Nl6scn0AaeX1gs6Z3l5iYYzggppgPnSZ6PfWDzTPAw8BOzY6lRRwHTKz3ojyi+iPA4RIHVR6T\nOA54D/DRJn9BfxJYA/hKE2MIoS1EolhSVD8BEsNJ08Rf35fr88ytBwBnS2m9D4n3AF8D9svTgDRN\nTmYHAkdLHNDMWEJodT0mCknXS/rvPM13J4lEkYwFzulPl1Wb6cAJpJHbw4HLgcNtHmpQjP1i8xQp\nmU2UeGez4wmhVdV6ovgJ8AFgnqRfSPpw1yyvg9ztwDslVm12IEVJrCdxTAPvtxJwKGn23n7JI91/\nSZpJ+Ps2N/f3no2Upwv5NKlxu9N+FIVQSI+JwvbVtg8FhgFXkubMeUzSBZL2HqD4Blxek+IPwO7N\njqUIiRVIcyR9v4HJYgwwxWZBg+73RdIYhtMbdL+GsvkZMB/Ys8mhhNCSem2jsP0v25fZ3h/Ym7Qk\n6Q2lR9ZcbVH9lEeRTyQ1wO8IjJfYuQH3rDkSu142r9tcV8YiRw10E23wbx5CM/SaKCS9VdIJkv5A\n+uV6Iwz66bjbIlEAnwK2Ao62mQMcCfxSYr1+3PNdwKqk96CTtMu/eQgDrsdxFJI+TqqnfjtpcNSl\nwB/dQgMvGj2OYtF9WQr4K2mdhfmNvn8jVKwFsZPNIxX7TyFV87y3L0u7SpwHzLU5tWHBtoH8b/4k\nsEMj1tYIoZU1chzFTqRZPYfa/qTtP/QlSUgaLWmOpLmSlpgdVNJqkq6SNFPSVEnvqDh2vqSnJN1X\nb7n9kfv330KL/sKU2Bj4GWkivUe6HT6VtKb1hHonOMwN+AcCFzQk0DbS6v/mITRTrUTxC2BV269X\n7pR0kKRC/zNJGgKcBYwmLXk6RtIW3U77IjDd9jbAR1l8neML8rXNMInUJtNSco+ka4Cv29za/Xhu\nBzgGGM6iZWWLOhy4MXcb7UQt+W8eQrPVShRfgSW/iPK+bxS8/wjgIdvzbb8KXEZaa7nSFqS5gbD9\nIDBM0pp5+zbg7wXLarRJwB4SQ5pU/hJy9chFwJ3Aj3s6z+ZfwP7AyVKxnjz56WMsfRiJPYhMAvbs\naY6qEDpVrf8hlrP9dPedtp8B3lTw/uuRVkbr8njeV2kmadATkkYAbwPWL3j/0uSuoU/RWg33XwLe\nCnyitx5EuW1lDHCJxEYF7r0LsAwwpZ8xtq08VfozpJ59IYRs6RrHVpK0TH4SeIOkZYDlC96/SJvG\nqcCZkmYA9wEzgNdrX7JYPOMqNqfYnlL02gK6esLc3cB79onEfsCxwIjKdatrsZki8U3SYLJ327xY\n4/TjgAkt3oV1IHT9m9/T7EBCaBRJI4GRfb6+Rq+nU0mza37S9ot530qkNoRnbPe6bKWknYBxtkfn\n7VOAhbZPq3HNPGCrijKHAdfZ3qrKuaX0elp0f/YFTrKbO/hO4h2kX/r72txV57UCzgVWAQ6uNhGf\nxBqkWXM3KnsBoVYn8UHgU3YMvguDVyN7PX2ZVPUyX9J0SdOBeaRH8y8VvP80YFNJw/L0H4cA13YL\neJWuqUEkHQvc2pUkWsCtwA5S4aq2hpNYjTR+5bP1Jgl4o3H7f0hVfv/bw2lHANd2epLIpgAj8loc\nIQQKrEchaUV4YzWwh2y/VFcB0j7AGcAQ4Dzb4yWNBbA9UdK7gQtJ1VSzgWNsP5+vvRTYDXgL8DTw\nFdsXVNy71CeKVAa3AqfaAz8aXWJp4DfA/Taf7ue91gHuIrVvXFuxX8CDwJE2f+hPGYOFxG3AN21u\nanYsIZSh3u/OWLio1zL4ErC6zWfKLKeHsr9Lalgd3Z9ZXCvutyNwHTDS5v68bw9SdeLW0T6RSHwZ\nWMXmpGbHEkIZYuGixmtK33qJw0i9wQ5pRJIAsJkKfI7UuL1a3h2N2EuK8RQhVIgnil7LYAipXead\nNk+UWVZFmduT5tTaw6bho9IlzgA2Jw3M+xMwzOb5RpfTrnKV3zPAFjZPNjueEBqt3u/OHrvHStqe\n1G4gqnRztT29TxG2GZvXJSYD+0j8fACKfAtpWvfjykgS2Umk2VJvBX4VSWJxNq/lf/NRpKlSGkpC\n8QQX2kmt7rFTSAliBWB7YFY+tDUwzfa7ByLAWgbiiSKVw2GkLqYD5Vs23yyzAIm3kJ5ajrW5t8yy\n2pHE/yNNuHhEg++7CfA7YC+bBxt57xCKanhjtqQrga/avi9vvxP4mu0D+xVpAwxUogidJ3+h/x5Y\nr5G//iW+Q2r/WJ40O3E8zYUBV0Zj9tu7kgSA7dmk+ZlCGMweBl4hTWbZEBLLkdYM+QhpptpLWmku\nsRB6UiRRzJJ0rqSRknaXdA5pfqYQBq38FNHoxYwOBGbazCWt0/1m4OsNvH8IpSiSKI4C7idNWX1C\nfn1UmUGF0CIanSjemJ3X5lXSk8VhEgc3sIwQGi66x4bQA4nVgfnAmkUnYqxxry2B3wIb5CTRtX9b\n4GZglB1P6mFgNLJ7bK2umba9dV2RhdBmbJ6TmAO8m/5Pvz4WOK8ySeQyZkgcD1wt8S6bv/WznBAa\nrlb32GG1LrQ9v/Hh1CeeKELZJL4FYPc4oWKRe6wIPApsX2Xp2q5zxgM7Au/rnkxCaLSG9XrKq9LN\nzwnh38BWwDuBl1ohSYQwQBrRTnEwMLWnJJF9CXgZOL2fZYXQcL02Zks6mDTr6EdIH/i7JH2k7MBC\naBF/BN6e2yv66jhgQq0TbF4H/gt4n8TR/SgrhIYrMuBuFjCqa1nUvJ71b1uhjSKqnsJAkLgeON/m\nij5cO5y0BsuGORn0dv7bSQP9PmRzZ93BhlBAGQPuRJogrcuzeV8InaI/1U9jgXOLJAkAmznA0cAV\nEuv2scwQGqrWmtldbgRukvRzUoI4BAZ+EZ8QmmgS8Ml6L5JYCTiU1LZXmM2vJc4GrpQYafNyvWWH\n0Eg9PlFIWh7A9udIg4S2JjVoT7T9+SI3lzRa0hxJcyUtsca2pNUkXSVppqSpkt5R9NoQBtCfgOUl\nNq7zujHAZJsFfSjz26SeUmfnVQhDaJpa3WOn295O0k9tH173jaUhpCU2RwELgLuBMbYfqDjnu8A/\nbX9D0ubAj2yPKnJtvj7aKMKAkLgIuNPm7ILnC7gHOKWvS6rmtdr/QBp/8cO+3COEaho24A5YTtJh\nwC6SDqgsgzTg7spe7j2CtMb2/BzYZcB+QOWX/RbAqaQbPihpmKS1gI0LXBvCQJpEWnGwUKIA3gWs\nmq/rE5t/SewP/FFits3v+nqvEPqjVqI4DjgMWAX4YJXjvSWK9YDHKrYfJw0oqjST9D/f7ZJGAG8D\n1i94bQgD6Rbg/ySWLrg07Vhgos3C/hRqM09iDHCpxESqLCLWizvsvierEKB2ophq+zZJ02wvtmiP\npI0K3LvIB/pU4ExJM4D7gBnA6wWv7YplXMXmFNtTil4bQlE2T0o8BuwAtbutSqxKmil2swaVPVni\nGNJTSj2WJU1lPrS/c1WF9iZpJDCyr9fXShTXSNq/SpLYhtQv/G293HsBMLRieyjpyeANtl+ARYOL\nJM0jrQOwQm/XVtxjXC9xhNAoXd1kexvfcDhwo83TjSrY5jfAb+q9TuJdpKQ1EMv4hhaVf0BP6dqW\n9NV6rq81juIe4HpJK1bcfCTpw/qxAveeBmya2x2WJXWrvbbyBEmr5GNIOha41faLRa4NoQl6HU+R\nG7HH0stI7AE0gRRPCH1Wa66nLwGTSWMo3pwbtC8G9rfda52n7deA44GbSGtYXG77AUljJXV9cLcE\n7pM0B3gfac2LHq/t618yhAb5PbBtHh/Rk11IT+q3DkxIvboW2CxPcx5CnxSZwuOzLPpFsq/tuaVH\nVVB0jw0DTeJ3wPdtft3D8Z8B99j8YGAj65nEN4GV7PRDLIR6vztrjaO4rmJzV2Au8FTetu0P9TnK\nBolEEQaaxCnAW6t96UqsATwEbGTz3IAH1wOJt5GqkjeweanZ8YTma+Q4iq7pjp1fm0VzPLXvsngh\n9M/NwM96OHYEcG0rJQkAm0ckppJmf76wyeGENhRLoYZQB4mlgKeB4fainni5EftB4EibPzQrvp5I\nfBD4X5udmh1LaL4yZo8NIWR5AN1vWbL30+6khYf+OOBBFXM9sG6e9jyEukSiCKF+1brJHkcaid2S\nj+h5mvNzia6yoQ9qzR770/zfTw1cOCG0hUnAqFwNhcTapMTRU9tFqzgPOLSX7r0hLKHWE8X2ktYF\njpa0evc/AxVgCK0mr339D9LU+5AXGrJ5vnlR9S5Pdz6ZNP15CIXV6vU0gVQXuxGpa10l5/0hdKpJ\nwF4Ss4CPAwc1OZ6iJgCnSpzTqtVkofXUGpn9Q9tbABfY3rDbn0gSodN1tVPsDTxrL/FjqlXdQpoR\nut4JBkMHK9Q9Nk8E+F7Sk8RttmeWHVgR0T02NIvEKqSJKu8gVTud28slLUPiZGAzm2OaHUtojoZ3\nj5V0InAJsCawNvAzSSf0PcQQ2l9uj5gFvBu4rMnh1OsC4IA8HXoIvSoy19N9wE62/5W33wTcaXur\nAYivpniiCM0k8RlgbZu2W9Nd4jLSokb/1+xYwsBr5BQelRb28DqETtYyE//1wQTgLImzolE79KZI\norgAmCrpStJcT/sD55caVQhtoM2/YG8l/f+/C3B7k2MJLa5oY/b2pBlkuxqzZ5QdWBFR9RRC30l8\nGtje5r+bHUsYWA2bZrxBwYwGzgCGAOfaPq3b8VVIo1mHkn7dfM/2hfnYiaSV9AScY/vMKvePRBFC\nH0msDvwF2MTmb82OJwyclpkUUNIQ4CxgNGkluzGStuh22ieA2baHkxb+Pl3S0pLeSUoS7wK2AT4g\naeOyYg2hE+Xp0K8hTY8eQo/KnBRwBPCQ7fm2XyV1Idyv2zkLgZXz65WBZ4HXgS2AqbZftv06qT71\ngBJjDaFTTQTGds1bFUI1RcZRnCBptT7cez3gsYrtx/O+SmcBW0p6ApgJnOhUF3Yf8J48r9SKwL7A\n+n2IIYRQ2x9J06Pv3uxAQusq8itibeBuSb+QNFpS0XqtIo0fo4HpttcFhgM/kvRm23OA00irid0A\nzCC65YbQcLnn1gRi+vFQQ6/dY23/r6Qvk+a0ORI4S9IvgPNsP1zj0gWkRuouQ2HRimDZkcD4XM7D\nkuYBbwem2T6f3A1X0reBR6sVImlcxeYU21N6+zuFEBZzCfAtibfaPNnsYELjSRpJagfu2/VFez1J\nGg4cRXoK+B2wE3CL7c/1cP7SpKUh9wSeAO4Cxth+oOKcHwNP2f6apLVJs9Rubfs5SWvZflrSBsBN\nwI62/9mtjOj1FEIDSJwDzLP5drNjCeVrePfY3E31o6SG5nOBq2y/KmkpYK7tHnsjSdqHRd1jz7M9\nXtJYANsTJa1DWux9HVI32PG2f56v/T3wFuBV4NO2J/f3LxtCqE5ie+AKUlfZ15sdTyhXGYnia8D5\nth+pcmxL2/fXH2ZjRKIIoXEkpgFftrmh2bGEcpUxjuJG4O8VBawsaUeAZiaJEELDTSCt/R3CYoo8\nUdwLbGd7Yd4eQmps3nYA4qspnihCaByJN5M6jWxtL9HxJAwipYzM7koS+fXrpDaHEMIgYvMicClp\nVoQQ3lAkUczLg+6WkbRsbtz+S9mBhRCaYiLwMYllmx1ITyR2zSsMhgFSJFEcR5qKeAFpHMROpMXk\nQwiDjM0sYCqwxCScLeRiYEyzg+gkpc4eW7Zoowih8SRWBu4EzrSZ2Ox4KklsDDwEXGlzYLPjaVcN\nX+FO0grAMaQZYJfv2m/76D5FGEJoaTb/lNgfuF3iT3ZLLWy0F2lqn90lhsSYj4FRpOrpp6T5nkaT\nZnEdCrxYZlAhhOay+TNpoO0vpMWm4mm2vUhr2DwO7NDkWDpGoe6xtodLmmV7a0nLALfb3nFgQqwZ\nW1Q9hVAiic8DhwC72vy7ybEMAZ4B3gGcBPzd5pvNjKldldE99j/5v89L2gpYFVizL8GFENrOd0lz\ntp0j0ewfZTsAC2z+CkwiPV2EAVAkUfxE0urAl4BrgfuB75QaVQihJeRpyD9GaqP8bJPD2ZuUIAB+\nD2wnsVIT4+kYNRuz88R/L9h+jtQ+seGARBVCaBk2L+XG7akS99nc1KRQ9uKNZQl4SeJuYDfg102K\np2PUfKLII7I/P0CxhBBalM2jwMHAxRKbDHT5+clhO9KTRJeofhogRaqeJkk6SdLQvDTp6rkqKoTQ\nQWxuA8YB1zShymc34C6bf1Xsu5lIFAOiSK+n+VRZ1tR206uhotdTCAMrN2hPIHWZP8AemCWKJc4E\nnrRT1VPetxTwNDA8JjGsT8PXo2hlkShCGHh5HqjfAbfYjBugMu8HPmozrdv+y4EbbC4ciDgGizJG\nZh9B9SeKiwtcO5pFK9yda/u0bsdXIQ2eGZpj+Z7tC/OxT5NGhBu4DzjK9iu9lRlCKJfNfyQOBO6W\nmGVzZZnlSawPrAXMqHK4q53iwjJj6HRFqp7OYlGiWAHYA5hu+6BerhtC6n89ijSh4N0suWb2F4GV\nbJ8iaY18/tr5z23AFrZfkXQ5cL3ti7qVEU8UITSJxA7ADcAeNveVWM6RwPttDq5y7G3AXcA6A1UN\nNhg0/InC9vHdClgVuLzAvUcAD9men6+7DNgPeKDinIXAyvn1ysCztl+T1BXbipJeB1YkJZsQQouw\nmSbxaeBqiXfZPFdSUXuxaPxE9xgekXge2Bq4t6TyO16hhYu6eYli4ynWAx6r2H4876t0FrClpCeA\nmcCJALYXAKeTVtt6AviH7Vv6EGsIoUQ2PwOuAi6Xev/hWa/cYN1josiim2zJek0Ukq6r+PMbUvXQ\nVQXuXaSVfDSpGmtdYDjwI0lvlrQa8CFgGLAu8GZJhxW4Xwhh4H2B9P97GSO3twaet5lf45xIFCUr\n8gvg9IrXrwGP2H6sp5MrLIDFZp0cCkt0YTuSN0Za+mFJ84AtSAlinu1nASRdCewMXNK9EEnjKjan\n2J5SILYQQoPYvCZxMml8xfcaPPV317TitUwGfiqxvM3LDSx70JA0EhjZ1+uLJIpHgb/a/ncucAVJ\nw7raHmqYBmwqaRip+ugQllyV6lFSY/cdktYGNgceBgTslNfCeDmfc1e1QmyPK/B3CCGUyGaGxBPA\nPjR2So29gB/3UvbzErOAXYGooq4i/4Ce0rUt6av1XF+kjeKXsNgvhIXAFQUCew04HriJNJHg5bYf\nkDRW0th82jeAnSXNIv0Df972c7bvymVMB2blc39S5C8UQmiaiaSlkxtCYnng3aQnht5E9VOJCq9H\n0W3fTNvblBpZAdE9NoTWIbEiqQPLdjaPNOB+o4Cv2+xc4NxdgP+z2a6/5XaCMtaj+Juk/SoK2A/4\nW1+CCyEMXjYvkQbQfqxBt+ytt1Olu4CNpFgrpwxFnig2ITUir5t3PQ4cbvuhkmPrVTxRhNBaJLYk\nVSO/zebVft5rOnBC0TW7Ja4BLrO5tD/ldoLS5nqStBKA7Rf6GFvDRaIIofVI/B44oz9Te0isBcwF\n1iiacCSOJ1V7Hd3XcjtFw6ueJI2XtKrtF2y/IGk1SbFObQihJxPof6P2nsCtdT6VTAL2aoElWwed\nIm0U+9j+R9eG7b8D+5YXUgihzf0KGN7PBY6KjJ/o7s+kXpmb96PcUEWRRLGUpOW7NvLYhmXLCymE\n0M5sXgEuAo7ty/X5iaCehuyuck10ky1FkURxCfBbScdI+hipoarXKcZDCB3tJ8BREsv14drNSVOC\n/LkP10aiKEGhxmxJ+5BGRxuYZLtZi6svJhqzQ2hdErcA59XbC0nik6RV647pQ5lrAA8Ba/a319Vg\nVsY4CmzfYPuzpPVy18qTA4YQQi0TgLG9nrWkuqudutj8jTQN0I59uT5UV6TX03KSDpD0S9KcTXuS\nPgAhhFDLNcDmElsUvUBiGWA34Lf9KHcSsHc/rg/d9JgoJL1P0oXAX4ADgZ8Cz9k+0vZ1AxRfCKFN\n5aqf86nvqWJH4GGbZ/pRdLRTNFiPbRSSFpKWIz3K9l/yvnm2iyxaNCCijSKE1iYxjDST9FCbfxc4\n/+vAcjYn96PM5YFncpn/6O38TtTINortgDuBSZIm5R5PQ/obYAihc+QFh6bCkutd96Av4ye6l/ky\ncAewe3/uExbpMVHYvtf2ycAmpEbs4cAykm6Q9PEBii+E0P4KTT8usSrwTtKXfH9F9VMD9dqY7eQO\n28cD6wM/AHYqPbIQwmBxPbC+RG9LE+wO/KFBq9RFomigQt1ju9h+3fbNtmPSrRBCITavAefQe6N2\nn7vFVnEfsFJuIwn9VFeiqJek0ZLmSJoraYnGKUmrSLpO0r2SZks6Mu/fXNKMij/PSzqhzFhDCKU6\nDzhU4s01zmlYosjTedxCPFU0RGmJQtIQ4CxgNLAlMEZS9/7UnwBm5xX0RgKnS1ra9oO2t7W9LbA9\n8BJwVVmxhhDKZbMAuBUYU+14/uW/MulJoFFiPEWDFEoUkt4j6aj8ek1JRbrIjgAesj3f9qvAZcB+\n3c5ZSPqhotHsAAASiUlEQVRwkP/7bF5ru9Io4GHbjxWJNYTQsiYA/6+HacD3Am6xWdjA8m4B9pCi\nt2Z/FRmZPQ74PHBK3rUsabnD3qxHWj+3y+N5X6WzgC0lPQHMBE6scp9DgZ8XKC+E0NomAasCO1Q5\ntjeNa58A3niKeRJiHe3+WrrAOR8GtgXuAbC9oGu1u14UWTpvNDDd9u6SNiaN2dimaxU9ScsCH4Se\nB9/kRNZliu0pBcoNIQwwm4USPyE1at/dtT//4t8D+FQJxd5Melq5u7cTBzNJI0nV+31SJFG8Ynuh\npK4C31Tw3guAoRXbQ0lPFZWOBMYD2H5Y0jzSFMPT8vF9gHts9zic3/a4gvGEEJrvAmCOxEkVo6a3\nA57MTwCNNgn4HPDtEu7dNvIP6Cld25K+Ws/1RdoofilpIrBqHmj3W+DcAtdNAzaVNCw/GRwCXNvt\nnEdJbRBIWpuUJP5ScXwMxELpIQwWNk+RfuX/d8XuRnaL7e5WYAeJoj9wQxVF16PYm0W9B26yXegf\nNa9jcQZp6o/zbI+XNBbA9kRJ6wAXAusAAsbb/nm+9k3AI8CGXVVRVe4fcz2F0GYkdgd+CGxtY4nJ\nwPdsSlm+QGIKcJrNDWXcvx3V+91ZKFG0qkgUIbSf3OtpDnA0cC+pwXkdmxdLKu9/gbfYfKaM+7ej\nhi9cJOmFKn8el3SVpI36F24IodPkwXBd8z+9F5heVpLIYjxFPxVpzD6T1M21q63gUGBjYAZprvmR\npUQWQhjMLgK+QhpLVVb7RJd7gHUl1rV5ouSyBqVeq54kzbK9dbd999oeLmmm7d4m+ipNVD2F0L4k\nLgYOB3aymVpyWVcA19pcXGY57aLe784iTxQvSToE+GXePgjemN2xfRs4QgjNdjapSmhabyc2wI3A\n96W+L4g0AGbbHNLsIKop8kSxMan6qWtq8TtJA2MWANvbvr3UCGvHFk8UIbQxiRVtXhqAcpYGNoOq\n04e0AgG3A5vnLsTlFha9nkIIof1IXA380uaS8stqcNWTpBWAY0gzwC7ftT/WpAghhIbqWmyp9ERR\nryIjs38KrE2al+lW0lQcZXZlCyGETjQJ2KuH2XWbqkii2MT2l4EXbV8EvB/YsdywQgih48wFXgW6\nr9vTdEUSxX/yf5+XtBVpmuA1ywsphBA6Tx6I2JJrfRdJFD+RtDrwJdKkfvcD3yk1qhBC6EwtmShq\n9nqStBTwEduXD1xIxUWvpxDCYCLxFmAesIb9Rm1OCeU0cK4n2wtJq9uFEEIomc2zwJ+Bdzc7lkpF\nqp4mSTpJ0lBJq3f9KT2yEELoTC1X/VRkZPZ8qkzVYXvDkmIqLKqeQgiDjcQewHi7vN6lMTI7hBDa\nmMRywN+ADWz+Xk4ZjV+P4k2SvizpnLy9qaQPFAxmtKQ5kuZKWmIyLkmrSLpO0r2SZks6suLYqpKu\nkPSApPsl7dT9+hBCGGxsXgHuAPZodixdirRRXEAaS7Fz3n4C+FZvF0kaApxFGtG9JTBGUveBJJ8A\nZtseTlrX4nRJXdOKnAlcb3sLYGvggQKxhhDCYHAzLdROUSRRbGz7NPLAO9v/KnjvEcBDtufbfhW4\nDNiv2zkLgZXz65WBZ22/JmkV4D22z89lvmb7+YLlhhBCu2upBu0iieKVPDEg8Ma0468UuG490sp4\nXR7P+yqdBWwp6QlgJnBi3r8h8IykCyRNl3SOpBULlBlCCIPBbGBFiY0bfWNpie/hXhVZuGgcadGP\n9SX9HNgFOLLAdUVayUcD023vnhPQJEnb5Li2A463fbekM4AvkJZOXIykcRWbU2xPKVBuCCG0LBtL\n3EJ6qni4v/eTNJI3lq0eU/eTSq+JwvbNkqazaOGiE20/U+DeC0gzzXYZSnqqqHQkMD6X87CkecDm\n+bzHbd+dz7uClCiqxTeuQCwhhNBuJpGq6yf090b5B/SU3KPqf+DSuq4v0uvpOtJyhZNt/7pgkoC0\nvOGmkoZJWhY4hDRXVKVHgVG5nLVJSeIvtp8EHpO0WT5vFPCnguWGEMJgcAuwu8SQBt7zAFI1f12K\ntFGcDrwHuD93Vz1I0vK9XWT7NeB44CbSRIKX235A0lhJY/Np3wB2ljSL9KZ83vZz+dgngUskzST1\nevp2XX+zEEJoYzZPkHqZ7tDA2x4HTKz3osID7nK31d2BY4HRtlfu5ZLSxYC7EMJgJvED4Fmbbzbg\nXlsCvwU2AP2noQPuUgFaATiQlI3eBVzUl0BDCCHUpZHjKT4OnGfzar0XFpnr6RekFe1uJI2FuDXP\nKtt08UQRQhjMJN4EPAmsY/d9CWqJFUltwtvbPNLwKTyA84GNbI+1PRnYRdKP+hhvCCGEgmz+ReoY\ntFs/b3UwMNXmkb5c3GuisH0jsI2k70p6hNQAPacvhYUQQqhbI0Zpj6Uf3Wx7HEchaXNgDHAo8Cxw\nOamqamRfCwshhFC3SfSjXVhiOGlWjOv7fI+e2igkLQRuAz5me27eN68V1qHoEm0UIYTBLo+jeBrY\nxl5i0HKR688G/mrz9UX7GtdGcQCpEWVynmtpFBBfyiGEMIBsXgd+Rx6cXA+JlUi1Quf1J4YeE4Xt\nq20fArwdmAJ8ClhT0tmS9u5PoSGEEOrS13aKMcBkmwX9KbyuFe7yWtkHAYfabvqiGlH1FELoBBIb\nAneSuskWGp4gIeAe4BSbmxY/FkuhhhDCoCMxFzjILjZXk8QI0ti3TbonlzLGUYQQQmi+equfxgIT\niz6B1BKJIoQQ2kPhRCGxKmnapQsaUXAkihBCaA+TgZ0lep29GzgcuNHm6UYUHIkihBDagM0/SEuk\n7lrrvNyI3a+R2N1FogghhPZRpPppF9KsG7c2qtBIFCGE0D6KJIrjSI3YDevSWmqikDRa0hxJcyWd\nXOX4KpKuk3SvpNmSjqw4Nl/SLEkzJN1VZpwhhNAm7gQ2lliz2kGJNYAP0OA1g0pLFJKGAGcBo4Et\ngTGStuh22ieA2baHAyOB0/NKegAGRtre1vaIsuIMIYR2kRcduhXYs4dTjgCusXmuh+N9UuYTxQjg\nIdvzbb9KGvixX7dzFgJdS6quDDyb19ruEoPpQghhcVWrnyoaseteE7s3ZSaK9YDHKrYfz/sqnQVs\nKekJYCZwYsUxA7dImibp2BLjDCGEdjIJ2Csnhkq7Ay8Df2x0gT2uR9EARRpSRgPTbe8uaWNgkqRt\nbL8A7GL7r5LWzPvn2L6t+w0kjavYnGJ7SiOCDyGEFvUgqbZls/y6y3HAhGqN2JJGkqr3+6TMRLEA\nGFqxPRSWmEv9SGA8gO2HJc0DNgem2f5r3v+MpKtIVVlLJArb4xoeeQghtCgbS0wC9iYnCom3kqqj\nqta+5B/QU7q2JX21njLLrHqaBmwqaZikZYFDgGu7nfMoeY51SWuTksRfJK0oaaW8/02kN+S+EmMN\nIYR20r2d4ijgCpvnyyis1NljJe0DnAEMAc6zPV7SWADbEyWtA1wIrEN6lBpv++eSNgKuzLdZGrjE\n9vgq94/ZY0MIHUdiLWAusAapU9BDpJll7yl2fUwzHkIIg57EdOAEYCXgGzY7FL+2vu/OMtsoQggh\nlKer+mkbGjivUzXxRBFCCG1IYhTwY1L10wY2Lxa/NhYuCiGETnA7qTfpz+tJEn0RVU8hhNCGbF6W\n+DZp1otSRdVTCCF0mKh6CiGE0FCRKEIIIdQUiSKEEEJNkShCCCHUFIkihBBCTZEoQggh1BSJIoQQ\nQk2RKEIIIdQUiSKEEEJNkShCCCHUVGqikDRa0hxJcyWdXOX4KpKuk3SvpNmSjux2fIikGZKuKzPO\nEEIIPSstUUgaApwFjAa2BMZI2qLbaZ8AZtseTlr4+3RJlRMVngjcD0suFh4WlxdPD8R7USnei0Xi\nvei7Mp8oRgAP2Z5v+1XSDIf7dTtnIbByfr0y8Kzt1wAkrQ+8HziXtExqqG1kswNoISObHUALGdns\nAFrIyGYH0K7KTBTrAY9VbD+e91U6C9hS0hPATNITRJcfAJ8jJZMQQghNUmaiKFJdNBqYbntdYDjw\nI0krSfoA8LTtGcTTRAghNFVp61FI2gkYZ3t03j4FWGj7tIpzfg2Mt31H3v4t8AXgw8DhwGvA8qRq\nqV/Z/mi3MqLtIoQQ+qCe9SjKTBRLAw8CewJPAHcBY2w/UHHOj4GnbH9N0trAPcDWtp+rOGc34CTb\nHywl0BBCCDWVthSq7dckHQ/cBAwBzrP9gKSx+fhE4BvAhZJmkaqYPl+ZJCpvV1acIYQQamvrpVBD\nCCGUr21HZvc2mK+TSJovaVYenHhXs+MZSJLOl/SUpPsq9q0uaZKkP0u6WdKqzYxxoPTwXoyT9Hj+\nbMyQNLqZMQ4USUMlTZb0pzyY94S8v+M+GzXei8KfjbZ8osiD+R4ERgELgLvp1v7RSSTNA7bvodpu\nUJP0HuBF4GLbW+V93wH+Zvs7+UfEara/0Mw4B0IP78VXgRdsf7+pwQ0wSW8F3mr7XklvJrV/7g8c\nRYd9Nmq8FwdT8LPRrk8URQbzdZqO7EZs+zbg7912fwi4KL++iPQ/xaDXw3sBHfjZsP2k7Xvz6xeB\nB0jjuDrus1HjvYCCn412TRRFBvN1EgO3SJom6dhmB9MC1rb9VH79FLB2M4NpAZ+UNFPSeZ1Q1dKd\npGHAtsBUOvyzUfFe3Jl3FfpstGuiaL/6snLtYntbYB/gE7kKIgBOdaud/Hk5G9iQNKD1r8DpzQ1n\nYOWqll8BJ9p+ofJYp3028ntxBem9eJE6PhvtmigWAEMrtoeSnio6ku2/5v8+A1xFqprrZE/lelkk\nrQM83eR4msb2085I86Z1zGdD0jKkJPFT21fn3R352ah4L37W9V7U89lo10QxDdhU0jBJywKHANc2\nOaamkLSipJXy6zcBewP31b5q0LsWOCK/PgK4usa5g1r+MuzyYTrksyFJwHnA/bbPqDjUcZ+Nnt6L\nej4bbdnrCUDSPsAZLBrMN77JITWFpA1JTxGQBlBe0knvhaRLgd2ANUh1zl8BrgF+AWwAzAcOtv2P\nZsU4UKq8F18lzZg6nFTFMg8YW1FHP2hJ2hX4PTCLRdVLp5BmiOioz0YP78UXgTEU/Gy0baIIIYQw\nMNq16imEEMIAiUQRQgihpkgUIYQQaopEEUIIoaZIFCGEEGqKRBFCCKGmSBSh5Un6naS9u+37VF4h\nsadrpkjavuS4Ls3z5JzYbf84SZ/Nr5fP01p/pcr1H5F0f14CuK8xvFjx+v2SHpS0QY7hX5LW7OHc\nhZK+V7F9Up5pNoQlRKII7eBS4NBu+w4Bfl7jmlLn8cnTQOxgexvbZ1YrO88a8Cvgbttfr3KbY4CP\n2d6zYJnVVqR0PrYncCYw2vaj+djfgM92Pzf7D/BhSW+pciyExUSiCO3gV8C+XV+UeQbMdW3fLuls\nSXfnBVnGVbu42y/pgyRdkF+vKekKSXflPztXuXZ5SRcoLQw1XdLIfOhmYL284MuuVYpdhjT9/YO2\nv1jlvl8BdgHOl3SapOWqlSPpSEnX5qeOST38/d4L/ATY1/a8vNvA+cAhPcwK+mq+5tPV7hlCpUgU\noeXlBZnuAt6fdx0KXJ5ff9H2u4BtgN0kbVXtFj28PhP4ge0RwEGkidG6+wTwuu2tSVMeXJSfFD4I\nPGx7W9u3d7tGwOeBV2x/poe/09dJc5b9l+2TgeOrlLNcPn1b4EDbu1e51fKkKVz2s/3nbsdeJCWL\nT1WLAfgxcJiklXs4HgIQiSK0j8rqp0PyNqRfzPcA04F3AFvUcc9RwFmSZpDmh1pJ0ordztkF+BmA\n7QeBR4DNqL3gi4HbgZ0lbVowlp7KMTCpxnxE/wHuAD7WQxw/BI7IU0wvfjBNu30xcELBGEOHikQR\n2sW1wJ6StgVWtD0jT4j4WWAP29sAvyH9wu6u8ilihYrXAnbMTwXb2h5q+6Uq1/dlhbjfk6p1buia\n1rqAnsr5V41rFpKWtBwh6ZTu97P9PKkt5/gerj+D1FbypoIxhg4UiSK0hbzQymTgAhY1Yq9M+hL9\np6S1SQs3VfOUpLdLWoo0nXJX4riZil/TkoZXufY24LB8fDPSrKMPFoz5SuB7wI2SVunl9GrlzKFA\nkrL9MrAvqRrp6CqnfB8YS5pduPu1fyfNpnoM0aAdehCJIrSTS4Gt8n+xPROYQfpCvYRU3VPNF4Bf\nk6ponqjYfwKwQ+7i+ifg41Wu/TGwlKRZpMbpI/I67VD7i9U5xgmkNoRrK9ocqumpnN56b3WV83dg\nNPAlSR/sduxZ4Epg2e7XZaeTpiYPoaqYZjyEEEJN8UQRQgihpkgUIYQQaopEEUIIoaZIFCGEEGqK\nRBFCCKGmSBQhhBBqikQRQgihpkgUIYQQavr/f87Y3hCuff0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x71f9590>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(k_range, test_accuracy)\n",
    "plt.xlabel(\"Value of K for KNN\")\n",
    "plt.ylabel(\"Average Accuracy of Kfold CV\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}