{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# KNN（K最近邻分类）算法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果有一个数据集中，有N类数据。输入没有标分类的数据集后，我们可以将预测集中的数据，和训练集的数据相比较，提取和预测数据最相似（距离最近）的K个数据，选择这K个数据中出现次数最多的标签，作为新数据的分类。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNN算法的思想非常简洁直观：\n",
    "\n",
    "1、计算测试数据与各个训练数据之间的距离； \n",
    "    \n",
    "2、按照距离的递增关系进行排序；  \n",
    "   \n",
    "3、选取距离最小的K个点；  \n",
    "   \n",
    "4、确定前K个点所在类别的出现频率； \n",
    "   \n",
    "5、返回前K个点中出现频率最高的类别作为测试数据的预测分类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### KNN算法的优点：\n",
    "1、简单，易于实现； \n",
    " \n",
    "2、因为找的是最近邻的数据点，因此当某些点数量稀少时，划分越准确，适合对稀有点分类； \n",
    " \n",
    "3、使用多分类问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 算法实现"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们利用一个案例，按照KNN算法的思想，逐步实现算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### KNN案例：优化约会网站的配对效果\n",
    "\n",
    "\n",
    "<b>项目概述</b>\n",
    "\n",
    "海伦使用约会网站寻找约会对象。经过一段时间之后，她发现曾交往过三种类型的人:\n",
    "\n",
    "- 1：不喜欢的人\n",
    "- 2：魅力一般的人\n",
    "- 3：极具魅力的人\n",
    "\n",
    "\n",
    "她希望：\n",
    "\n",
    "- 不喜欢的人则直接排除掉\n",
    "- 工作日与魅力一般的人约会\n",
    "- 周末与极具魅力的人约会\n",
    "\n",
    "\n",
    "现在她收集到了一些约会网站未曾记录的数据信息，这更有助于匹配对象的归类。\n",
    "\n",
    "<b>开发流程</b>\n",
    "\n",
    "海伦把这些约会对象的数据存放在文本文件 datingTestSet2.txt 中，总共有 1000 行。海伦约会的对象主要包含以下 3 种特征：\n",
    "\n",
    "- `Col1`：每年获得的飞行常客里程数 \n",
    "- `Col2`：玩视频游戏所耗时间百分比 \n",
    "- `Col3`：每周消费的冰淇淋公升数 \n",
    "\n",
    "文本文件数据格式如下：\n",
    "```python\n",
    "40920\t8.326976\t0.953952\t3\n",
    "14488\t7.153469\t1.673904\t2\n",
    "26052\t1.441871\t0.805124\t1\n",
    "75136\t13.147394\t0.428964\t1\n",
    "38344\t1.669788\t0.134296\t1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 298,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "data = pd.read_csv('datingTestSet2.txt',sep = '\\t',header = None)\n",
    "X = np.array(data.iloc[:,:-1]) \n",
    "y = np.array(data.iloc[:,-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 切分数据\n",
    "\n",
    "我们可以直接调用sklearn的函数将数据集切分为训练集和测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 229,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split  \n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算测试集数据和训练集间的距离，进行分类。 \n",
    " \n",
    "我们先用最简单的思想分类：将想要预测的样本，和训练集中每个样本的特征直接相减的绝对值之和作为距离，将距离最近的训练样本的标签标记为预测样本的标签。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "metadata": {},
   "outputs": [],
   "source": [
    "class KNN:\n",
    "    def __init__(self):\n",
    "        pass\n",
    "    def train(self,X_train,y_train):\n",
    "        #读取训练集\n",
    "        self.X_train = np.array(X_train) \n",
    "        self.y_train = np.array(y_train)\n",
    "    def predict(self,X_test):\n",
    "        (m,d) = np.shape(X_test) #测试集的数量和特征数\n",
    "        y_pred = np.zeros((m)) #将预测的标签初始化为0\n",
    "        for i in range(m):    \n",
    "            distance = np.sum(np.abs(self.Xtrain - X_test[i,:]),axis = 1) #求距离的绝对之和\n",
    "            min_index = np.argmin(distance) #找到最近点的索引\n",
    "            y_pred[i] = self.y_train[min_index] #将最近点的分类给新数据标记\n",
    "        return y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以将这个算法称为“最近邻算法“，直接取找最近的一个数据进行分类标记，我们将这个算法扩展到K近邻算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以扩展的方向：\n",
    "* 选择不同的距离公式\n",
    "* 选择不同的K值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 选择不同的距离公式：\n",
    "上一个算法中用的距离公式为曼哈顿距离，将参数特征相减的绝对值求和，即L1距离。我们还可以用L2距离，"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "曼哈顿距离：$$d_1(I_1,I_2) = \\sum_P|I_1^p - I_2^p|$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "欧式距离：$$d_2(I_1,I_2) = \\sum_P\\sqrt{(I_1^p - I_2^p)^2}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "打个比方来说，当你搜索地图上的两个点，欧式距离就是将两个点用直线相连的空间距离；曼哈顿距离衡量的是你从A点开车到B点的距离，因为你不能穿过大楼和墙壁，所以衡量的是横向路线和纵向路线的的加总距离。  \n",
    "  \n",
    "KNN算法中，欧式距离用的更多，因为我们一般衡量变量特征的在多维空间中的距离，这时候不需要“开车绕墙”。 \n",
    " \n",
    "如有兴趣，可自行学习其他距离公式，添加到我们后面的算法中。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 选择不同的K值\n",
    "我们不再是选取排序后距离最近的一个训练数据打标签，而是选择距离最近的前K个训练数据，找到大多数近邻归属的类别，将预测值归为此类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "排序和计数我们可以直接调用argsort函数和Counter函数 \n",
    " \n",
    "按照以上思想，我们重新改写KNN算法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import  Counter\n",
    "class KNN:\n",
    "    def __init__(self,k=1,metric ='euclidean'): #默认距离算法为欧式距离，默认最近邻\n",
    "        self.metric = metric\n",
    "        self.k = k\n",
    "    def train(self,X_train,y_train):\n",
    "        self.X_train = np.array(X_train)\n",
    "        self.y_train = np.array(y_train)\n",
    "    def predict(self,x_test):\n",
    "        (m,d) = np.shape(x)#测试集的数量和特征数\n",
    "        y_pred = np.zeros((m))#将预测的标签初始化为0\n",
    "        #============================= show me your code =======================        \n",
    "        \n",
    "        \n",
    "        #============================= show me your code =======================\n",
    "        return ypred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*可能你会问，如果两个分类刚好数量相等怎么办？可以有多种方法进行处理，如随机分类，如比较两类的距离总长度，我们这里不做更多处理，按Counter函数默认给出的分类。*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 选择K值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "那么到底如何选择K值呢？我们可以选择在测试集中表现最好的K值。 \n",
    " \n",
    "本任务中我们直接调用sklearn中的kFold函数，将数据集进行k折验证，取每次验证的评分平均值作为此K值的误差评分。（这两个k表示的意思不一样，请留意）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如何定义测试结果的评分呢？可以直观地将分类正确的比例作为衡量指标。定义准确度的函数为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def score(ypred,ytest):\n",
    "    return sum(ypred == ytest)/len(ytest)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将我们自己撰写的分类器中添加评分函数，这就是一个相对完整的分类器了，我们可以将他和sklearn的结果做比较"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 317,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import  Counter\n",
    "class KNN:\n",
    "    def __init__(self,k,metric ='euclidean'):\n",
    "        pass\n",
    "        self.metric = metric\n",
    "        self.k = k\n",
    "    def train(self,X,y):\n",
    "        self.X_train = np.array(X)\n",
    "        self.y_train = np.array(y)\n",
    "    def predict(self,x_test):\n",
    "        x = np.array(x_test)\n",
    "        (m,d) = np.shape(x)\n",
    "        ypred = np.zeros((m))\n",
    "        #============================= show me your code =======================        \n",
    "        \n",
    "        \n",
    "        #============================= show me your code =======================\n",
    "        return ypred\n",
    "    def score(self,ypred,ytest):\n",
    "        return sum(ypred == ytest)/len(ytest)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 和sklearn的KNeighborsClassifier算法做比较"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 338,
   "metadata": {},
   "outputs": [],
   "source": [
    "#数据标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "ss = StandardScaler()\n",
    "\n",
    "X_ = ss.fit(X)\n",
    "X_std =ss.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 339,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8ldW18PHfykwGMhEgJEAIg4omgKDiPFBbFSttr7ZaB7RW9FqrXjvZ4fp6fXvvW3vbil6tLYpDaxXHXrFgVVCrVWaEMAmEMIYQIIGQABnPev84z8FjPCEnyXnOkKzv55NPznmmsx8jWdl77b0eUVWMMcaYUIuLdAOMMcb0ThZgjDHGuMICjDHGGFdYgDHGGOMKCzDGGGNcYQHGGGOMKyzAGGOMcYUFGGOMMa6wAGOMMcYVCZFuQCQNGDBAi4qKIt0MY4yJKStWrNivqnmdHdenA0xRURHLly+PdDOMMSamiMj2YI6zITJjjDGusABjjDHGFRZgjDHGuMLVACMil4jIRhEpF5F7A+xPFpEXnf1LRKTIb1+piCwSkXUiskZEUpzt7zvXXOV8DezsWsYYY8LPtQAjIvHAY8ClwFjgGhEZ2+6wm4EDqjoKeAh40Dk3AXgOuE1VTwYuAFr8zrtWVcc7X3uPdy1jjDGR4WYP5nSgXFUrVLUZmANMa3fMNOBZ5/UrwBQREeDLQJmqrgZQ1RpVbevk8zq6ljHGmAhwM8AUADv93u9ytgU8RlVbgTogFxgDqIi8JSIrReTH7c572hke+3e/INLRtYwxxkRAtCb5E4BzgGud718XkSnOvmtVtQQ41/m6visXFpEZIrJcRJbv27evW41bvq2WB//+Kfa4aWOM6ZibAaYSGOr3vtDZFvAYJ++SCdTg7e18oKr7VfUIMB84FUBVK53v9cDzeIfijnetz1HVWao6SVUn5eV1uhA1oLWVdTz+/haq6hq7db4xxvQFbgaYZcBoERkhIknA1cDcdsfMBaY7r68E3lVvt+AtoEREUp1gcT6wXkQSRGQAgIgkApcDazu5VsiVDs0CoGxXnRuXN8aYXsG1AOPkQe7AGyw2AC+p6joReUBErnAOmw3kikg5cA9wr3PuAeB3eIPUKmClqs4DkoG3RKTM2V4JPHG8a7lhbH5/EuKEsl0H3foIY4yJea7WIlPV+XiHt/y33ef3uhG4qoNzn8M7Vdl/22FgYgfHd3itUEtJjGfMoAzWVFoPxhhjOhKtSf6oV1qYSdmuOkv0G2NMByzAdFNpYRZ1R1vYUXsk0k0xxpioZAGmm0oLMwFYbYl+Y4wJyAJMN40ZlEFSQhxrLNFvjDEBWYDppqSEOE7K7289GGOM6YAFmB4YV5jJuso62jyW6DfGmPYswPRASUEmh5vbqNjXEOmmGGNM1LEA0wPjbEW/McZ0yAJMD4zMSyc1Kd5W9BtjTAAWYHogPk44ZUgmZbai3xhjvsACTA+VFGayfvchWto8kW6KMcZEFQswPVRamElTq4dN1fWRbooxxkQVCzA9VFroTfSvsUS/McZ8jgWYHirKTSUjJcEWXBpjTDsWYHpIRCgtzGRNpc0kM8YYfxZgQqC0MItPq+ppbGmLdFOMMSZqWIAJgdKCTFo9yqd7LNFvjDE+rgYYEblERDaKSLmIfOERxiKSLCIvOvuXiEiR375SEVkkIutEZI2IpIhIqojME5FPne2/8jv+RhHZJyKrnK/vunlv/kqPrei3YTJjjPFxLcCISDzwGHApMBa4RkTGtjvsZuCAqo4CHgIedM5NwPu45NtU9WTgAqDFOec3qnoiMAE4W0Qu9bvei6o63vl60qVb+4IhmSnkpiVZyRhjjPHjZg/mdKBcVStUtRmYA0xrd8w04Fnn9SvAFBER4MtAmaquBlDVGlVtU9Ujqvqes60ZWAkUungPQfEl+q0HY4wxn3EzwBQAO/3e73K2BTxGVVuBOiAXGAOoiLwlIitF5MftLy4iWcBXgYV+m/9FRMpE5BURGRq6W+lcSWEW5XsbONzUGs6PNcaYqBWtSf4E4BzgWuf710Vkim+nM4T2AvCIqlY4m98AilS1FHiHz3pGnyMiM0RkuYgs37dvX8gaPK4wE4/Cut2HQnZNY4yJZW4GmErAvxdR6GwLeIwTNDKBGry9nQ9Udb+qHgHmA6f6nTcL2KyqM30bnGG0Juftk8DEQI1S1VmqOklVJ+Xl5XX75torKcwELNFvjDE+bgaYZcBoERkhIknA1cDcdsfMBaY7r68E3lVVBd4CSpxZYwnA+cB6ABH5Jd5AdLf/hUQk3+/tFcCGEN/PcQ3MSCE/M8US/cYY40hw68Kq2ioid+ANFvHAU6q6TkQeAJar6lxgNvBnESkHavEGIVT1gIj8Dm+QUmC+qs4TkULg58CnwErvfAAedWaM3SkiVwCtzrVudOveOlJSkMkaK91vjDGAiwEGQFXn4x3e8t92n9/rRuCqDs59Du9UZf9tuwDp4PifAj/tYZN7ZNzQLN5eX03d0RYy+yVGsinGGBNx0Zrkj0klBd48zFrrxRhjjAWYUCp1Ev2rLdFvjDEWYEIpKzWJYTmp9mwYY4zBAkzIeVf0W4AxxhgLMCFWWphJ5cGj7G9o6vxgY4zpxSzAhJg9QtkYY7wswITYKQWZiGDDZMaYPs8CTIilJycwMi/dSsYYY/o8CzAuKC3IpKyyDm/VG2OM6ZsswLigtDCTffVN7DnUGOmmGGNMxFiAcUFJoe8RypaHMcb0XRZgXHDykP7Ex4nlYYwxfZoFGBekJMYzZlCG9WCMMX2aBRiXjCv0lu63RL8xpq+yAOOSksJMDh5pYWft0Ug3xRhjIsICjEvGOYl+q6xsjOmrLMC4ZMygDJLi4+wJl8aYPsvVACMil4jIRhEpF5F7A+xPFpEXnf1LRKTIb1+piCwSkXUiskZEUpztE5335SLyiDjPTRaRHBF5R0Q2O9+z3by3ziQlxHHSkP6s3mk9GGNM3+RagBGReOAx4FJgLHCNiIxtd9jNwAFVHQU8BDzonJuA93HJt6nqycAFQItzzuPALcBo5+sSZ/u9wEJVHQ0sdN5HVGlBJmsr6/B4LNFvjOl73OzBnA6Uq2qFqjYDc4Bp7Y6ZBjzrvH4FmOL0SL4MlKnqagBVrVHVNhHJB/qr6mL1Ts/6E/C1ANd61m97xJQWZnK4uY2K/Q2RbooxxoSdmwGmANjp936Xsy3gMaraCtQBucAYQEXkLRFZKSI/9jt+VwfXHKSqVc7rPcCgUN1Id5Xain5jTB+WEOkGdCABOAc4DTgCLBSRFXgDUKdUVUUk4LiUiMwAZgAMGzYsNK3twKiB6SQlxLFxT72rn2OMMdHIzR5MJTDU732hsy3gMU7eJROowdsz+UBV96vqEWA+cKpzfGEH16x2htBwvu8N1ChVnaWqk1R1Ul5eXg9ur3PxcUJuWhI1h5td/RxjjIlGbgaYZcBoERkhIknA1cDcdsfMBaY7r68E3nVyK28BJSKS6gSe84H1zhDYIRGZ7ORqbgBeD3Ct6X7bIyo7NYkDFmCMMX2Qa0NkqtoqInfgDRbxwFOquk5EHgCWq+pcYDbwZxEpB2rxBiFU9YCI/A5vkFJgvqrOcy59O/AM0A940/kC+BXwkojcDGwHvunWvXVFTloSB45YgDHG9D2u5mBUdT7e4S3/bff5vW4Erurg3OfwTlVuv305cEqA7TXAlB42OeSy05KoPGjlYowxfY+t5HdZdmoitTZEZozpgyzAuCw7NYm6oy20tnki3RRjjAkrCzAuy0lLAqDuaEsnRxpjTO9iAcZl2U6AsUS/MaavsQDjsuzURABqD1sPxhjTt1iAcVl2qrcHY4l+Y0xfYwHGZb4czEEbIjPG9DEWYFx2rAdjAcYY08dYgHFZv6R4UhLjrFyMMabPsQATBjmpSRw4Ykl+Y0zfYgEmDLLTrOClMabvsQATBtmpSZaDMcb0ORZgwsB6MMaYvsgCTBjkpCZaDsYY0+dYgAmD7DQreGmM6XsswISBby3MQSt4aYzpQyzAhMGxgpeWhzHG9CGuBhgRuURENopIuYjcG2B/soi86OxfIiJFzvYiETkqIqucrz842zP8tq0Skf0iMtPZd6OI7PPb9103760rclJ9FZWtB2OM6Ttce2SyiMQDjwEXA7uAZSIyV1XX+x12M3BAVUeJyNXAg8C3nH1bVHW8/zVVtR44tk1EVgCv+R3yoqreEfq76ZmsYxWVrQdjjOk73OzBnA6Uq2qFqjYDc4Bp7Y6ZBjzrvH4FmCIiEszFRWQMMBD4METtdU2OPRPGGNMHuRlgCoCdfu93OdsCHqOqrUAdkOvsGyEin4jIP0Tk3ADXvxpvj0X9tv2LiJSJyCsiMjQkdxEC2akWYIwxfU+0JvmrgGGqOgG4B3heRPq3O+Zq4AW/928ARapaCrzDZz2jzxGRGSKyXESW79u3z4Wmf1G/pHj6JcZbkt8Y06e4GWAqAf9eRKGzLeAxIpIAZAI1qtqkqjUAqroC2AKM8Z0kIuOABGcfznE1qtrkvH0SmBioUao6S1UnqeqkvLy8ntxfl2SnJtpTLY0xfYqbAWYZMFpERohIEt4ex9x2x8wFpjuvrwTeVVUVkTxnkgAiUgyMBir8zruGz/deEJF8v7dXABtCdichkJ2WZENkxpg+xbVZZKraKiJ3AG8B8cBTqrpORB4AlqvqXGA28GcRKQdq8QYhgPOAB0SkBfAAt6lqrd/lvwlc1u4j7xSRK4BW51o3unRr3ZJjAcYY08e4FmAAVHU+ML/dtvv8XjcCVwU471Xg1eNctzjAtp8CP+1Je92UnZrEztojkW6GMcaETbQm+Xsdbw7GejDGmL7DAkyYZKclcaix1QpeGmP6DAswYeJbbGkFL40xfYUFmDDJSrWCl8aYvsUCTJj4Cl5aHsYY01dYgAmT7DRvwUurqGyM6SsswISJFbw0xvQ1nQYYEYkXkd+EozG9WbYNkRlj+phOA4yqtgHnhKEtvVpKohW8NMb0LcGu5P9EROYCLwOHfRtV9bWOTzHtecvFWA7GGNM3BBtgUoAa4CK/bcrnnyZpOpGdlmg5GNNtvkcfBflMPmMiLqgAo6o3ud2QviA7NclyMKZbmlrbOO/X7/HDL5/AVZOi5ll6xhxXULPIRKRQRP4qInudr1dFpNDtxvU22alWUdl0z9b9h6k+1MTf1+6JdFOMCVqw05SfxvvsliHO1xvONtMFOWlJluQ33bKpugGApVtrafNoJ0cbEx2CDTB5qvq0qrY6X88A4XscZC+RlZrIocZWWqzgpemizdX1ANQ3tbJud12EW2NMcIINMDUicp2zJiZeRK7Dm/Q3XXCs4KXNJDNdtKm6nlzn/5/FFfZPz8SGYAPMd/A+RXIPUIX38caW+O8i32LLg5aHMV20ubqB04pyGJmXxqItFmBMbOh0FpmIxAPfUNUrwtCeXs3Xg7GZZKYrGlva2FZzmMtL88lNT+L1VbtpbfOQEG+Vnkx0C3Yl/zXdubiIXCIiG0WkXETuDbA/WURedPYvEZEiZ3uRiBwVkVXO1x/8znnfuaZv38DjXSuaZKX6Cl5agDHBq9h3GI/C6EEZnDkyl4amVtbtPhTpZhnTqWAXWn4kIo8CL/L5lfwrOzrB6fk8BlwM7AKWichcVV3vd9jNwAFVHSUiVwMPAt9y9m1R1fEdXP5aVV3ebtvxrhUVPuvBWA7GBG/zXm+Cf8ygjGP/Dy2qqGHc0KxINsuYTgXbxx4PnAw8APzW+eqsAObpQLmqVqhqMzAHmNbumGnAs87rV4Ap0v1lyqG8lit8ORjrwZiu2FRdT0KcMGJAGnkZyYwemG6JfhMTgsnBxAGPq+pLXbx2AbDT7/0u4IyOjlHVVhGpA3KdfSNE5BPgEPALVf3Q77ynRaQNeBX4pXpraHR0rf3t7mcGMANg2LBhXbylnklJjCc1yQpemq7ZVN1A0YA0khK8fw9OLs7ltZW7aGnzkGh5GBPFgsnBeIAfh6Et/qqAYao6AbgHeF5E+jv7rlXVEuBc5+v6rlxYVWep6iRVnZSXF/6lPNmpSdRaD8Z0webqesYMSj/2fnJxLoeb21hbaethTHQL9s+fBSLyQxEZKiI5vq9OzqkE/IsmFTrbAh4jIglAJlCjqk2qWgOgqiuALcAY532l870eeB7vUFyH1wry/sImOy3RejAmaI0tbWyvPcLogRnHtp1R7P2nt7iiNlLNMiYowQaYbwHfAz4AVjhf7ZPs7S0DRovICBFJAq7GW27G31xguvP6SuBdVVURyXMmCSAixcBooEJEEkRkgLM9EbgcWHu8awV5f2HjrUdmSX4TnPK9DajCaL8ezID0ZMYMSmeR5WFMlAu2mvKIrl7YyYPcAbwFxANPqeo6EXkAWK6qc4HZwJ9FpByoxRuEAM4DHhCRFsAD3KaqtSKSBrzlBJd4YAHwhHNOR9eKKtmpSeyoPRLpZpgY4T+DzN/k4lxeWWF5GBPdjhtgROTHqvpr5/VVqvqy377/UtWfHe98VZ0PzG+37T6/143AVQHOexVvAr/99sPAxA4+K+C1ok1OmpXsN8HbVN1AQpxQlJv2ue1nFufyp0XbKdtVx8Th2RFqnTHH19mfPv69gJ+223dJiNvSJ2SnJlFvBS9NkDZX1zPCbwaZzxnF3smWNl3ZRLPOAox08DrQexOEnDTvan4reGmCsam64QvDY+DtCZ84OMMCjIlqnQUY7eB1oPcmCFm22NIE6WhzGzsPHPlcgt/f5OJclm87QHOr9YZNdOoswIwTkUMiUg+UOq9970vC0L5exwpemmD5ZpAF6sEATC7O4WhLG2sqD4a5ZcYE57gBRlXjVbW/qmaoaoLz2vc+MVyN7E2itWT/ln0NHG5qjXQzjJ9N1b4ZZIF7MGeM8OVhbD2MiU42vzHMsp0cTDQVvDzS3Mrlj/yTx94rj3RTjJ9Ne+tJjBeGt5tB5pPt5GHs+TAmWlmACbNoLHi5YvsBjra0sXLHgUg3xfjZXN1A8YD0465zmVycy/LttZaHMVHJAkyY+QpeRlMOxvcX8LrKQ3g8NncjWmyqru8wwe9z5shcGls8rN5leRgTfSzARIC3XEz0BBjfVNf6pla21Rzu5GgTDoebWtl14GiHCX6fM0bkIAKLbZjMRCELMBEQTQUvDze1UrarjotOHAjAGqvQGxXK9zYAHSf4fbJSkzhpcH8Wb7UAY6KPBZgI8Jbsj44k//LtB2j1KNdPHk5yQhxrdlmAiQa+GWSjO+nBwGfrYZpa29xuljFdYgEmAnLSkqJmmvLiihoS44UzinM4Kb8/ZdaDiQrlextIio9jeE5qp8dOLs6hqdXD6p32szPRxQJMBGSnRk/By0VbahhXmEVqUgKlhZmsq6yzRH8U2FRdT3FeGglBVEo+Y0QuIth0ZRN1LMBEQLQUvGxoamVNZR2TncKJpxRkcri5jYr9luiPtI5qkAWSmZrI2Pz+VpfMRB0LMBHgK3gZ6Zlky7fV0ubRYwGmtDAToNc8iteN5821tHlo86gr1/Y53NRK5cGjnSb4/U0uzmXljgM0tlgexkSPoB44ZkIrO81XLqaFgRkpEWvHIif/4nueyKi8dFIS4yjbVcfXJhRErF2h8OzH25j1QQVPTp/ESfn9e3w9j0e5b+5anlu849i2hDgh3u/L+z6OhDghMUH4P5efzJfGDuryZ212ZpAFk+D3ObM4l9n/3MqqnQeP/cFgTKRZgIkA32r+SOdhFlfUMn5oFv2S4gFIiI9jbH7/mO/BHGps4bdvb+RQYyvffmIxz333DE4ektnt67V5lJ+8WsYrK3Zx5cRChuWk0upR2jwe7/c2pU2VNo8ee7/w0708u2hbtwLMZzXIgg8wp43IIU68kzYswJho4WqAEZFLgIfxPt74SVX9Vbv9ycCf8D6lsgb4lqpuE5EiYAOw0Tl0sareJiKpwMvASKANeENV73WudSPw30Clc86jqvqke3fXfcfKxUQwwNQ3trC2so7vXTDyc9tLC7N4aflO2jxKfFxsPvLn6X9u41BjK499+1T+c956vv3EEv7y3TM4paDrQabNo/zo5dW89kkld39pNHd/aUxQ5/3mrY08/o8t1DQ0kZue3KXP3FxdT3JCHMOCmEHmk9kvkZOHZLJoSw13f6lLH2eMa1zLwYhIPPAYcCkwFrhGRMa2O+xm4ICqjgIeAh7027dFVcc7X7f5bf+Nqp4ITADOFpFL/fa96HdOVAYX8CvZH8EczPJtBz6Xf/E5pSCTI81tbN3fEKGW9Uzd0Rae/GcFF48dxNTSfF689UzSkxP49hOLWb2za+VUWts83PPSKl77pJIfXDwm6OACcFlJPm0e5a111V29BTZVNzAyL73LAX5ycQ6f7DxoeRgTNdxM8p8OlKtqhao2A3OAae2OmQY867x+BZgiIh3+q1LVI6r6nvO6GVgJFIa85S7LSo38Uy0XV9SQFB/Hqe2e5+5L9JfF6ILLp/65lfrGVu7+0mgAhuakMmfGZDJTE7nuySVBF/RsafNw14ureH3Vbn58yQl8f8roLrXjpPwMigekMX9NVZfvYXMQNcgCmVycS3Orh092WF0yEx3cDDAFwE6/97ucbQGPUdVWoA7w/Uk9QkQ+EZF/iMi57S8uIlnAV4GFfpv/RUTKROQVERkaqFEiMkNElovI8n379nXrxnoqGgpeLqqoYfywLFIS4z+3fWReOv0S42MywNQdaeGpf27lKycP+lzOxRtkziQnPYkbZi9lxfbjPz+lpc3DnS98wryyKn522YncfsGoLrdFRJhams/HW/ZT09AU9Hn1jS3srmvsUv7Fx5eHWWTTlU2UiNZpylXAMFWdANwDPC8ix6YCiUgC8ALwiKpWOJvfAIpUtRR4h896Rp+jqrNUdZKqTsrLy3P1Jo4nOzUpYjmYQ07+JVAyOD5OOHlIbCb6Z/+zgvqm1oBDWQVZ/ZgzYzJ5GcncMHspS7cGDjLNrR6+95eVvLl2D/9++VhmnDcy4HHBuKwkH4/C39ftCfqcYzPIBna9B9M/JZFTCjJtPYyJGm4GmErAvxdRyGcJ+C8c4wSNTKBGVZtUtQZAVVcAWwD/3xqzgM2qOtO3QVVrVNX3p+KTeCcORK2ctMhVVF62tRaPeqe2BlJSmMm63Ydoi6EV/QePNPPUR9u4rGRwh9OS8zO9QWZQZgrTn1r6hZXvTa1t3P6XFby9vpr/uOJkbj5nRI/adOLgDIrz0phXFvww2eZuzCDzd2ZxLqt2WB7GRAc3A8wyYLSIjBCRJOBqYG67Y+YC053XVwLvqqqKSJ4zSQARKQZGAxXO+1/iDUR3+19IRPL93l6BdxZa1MpOi1zBy8UVNSQlxDFhWFbA/SUFmRxtaWPLvthJ9D/54VYON7dy15TjJ+IH9U9hzozJFGT346ZnlvJR+X4AGlvauO3PK1iwYS//92unMP2soh63SUS4vCSfxRU17A9ymGxTdQPJCXEM7cIMMn+Ti3NpbvOwcrs9PM5EnmsBxsmp3AG8hfeX/Uuquk5EHhCRK5zDZgO5IlKOdyjsXmf7eUCZiKzCm/y/TVVrRaQQ+DneWWkrRWSViHzXOedOEVknIquBO4Eb3bq3UMhOjVzJ/sUVtUwY+sX8i0+sJfoPHG7m6Y+2cllJPicM7vwv/4EZ3iAzPCeN7zyzjHfWVzPjzyt4b+M+/t83Srh+8vCQte2yUmeYbG1ww2SbqusZNbDrM8h8JhVlEx8nNkxmooKr62BUdT4wv922+/xeNwJXBTjvVeDVANt3AQH/5anqT4Gf9rDJYROpHEzd0RbW7a7jzuPMihoxIJ3UpHjWVtZx5cTon6T3xIcVHGlp4+4uzPQakJ7M87ecwbVPLuGWPy1HBH79L6V887SAc0O67YRBGYx0hsmuCyJwba5u4MyR3V8omeHkYd7ZsJeT8vsTFyfEiRAf5+1RxYv3fVwcxImQGB9HSUEmSQnRmo41scxW8kdITloS9U3egpfHe+Z6qPnyL8db7R0fJ5wyJJOyGHgMb+3hZp75eBuXlw7pUmkVgNz0ZF64ZTI//981fOXkwUwbH/ryON7ZZEN49N3N7KtvIi+j40WXdUdb2HOosVtTlP2dPyaPRxZu5l//sjKo4289v5ifXnpSjz7TmEAswERIdupnBS/DWY/Ml38ZPzRw/sXnlIJMnl+6ndY2T1Al4yPljx9s4WhLG3dN6fpUYvDmwn5/rbvzQaaW5PPIws38fd2e4w6/le91EvwDu5fg97lrymi+WppPq0fxqOLxgEe95WxUlTbnvcejPLtoG89+vI3vnlN83OBnTHdYgIkQX8HLA4fDW/ByUUUNE4dld5h/8SktzOSpjzyU72vgxME9Lxbphv0NTfzp4+1cMW4Io3r4S9lNYwalM2pgOvPKdh83wGyq9j0muWf3Eh8nQffm8rP6sWDDXv7wjy38++XtC20Y0zPR+6dpL5cTgYKXdUdaWF91KKhiiL66XdGc6J/1QQVNrW3HzSdFAxFhakk+S7fWsre+scPjNlXX0y8xnsLsfmFr24gBaXx9QgHPLd5O9aGO22ZMd1iAiZDPSvaHL8As2VqDKkElkYsHpJHmJPqj0b76Jv60aBvTxhcwMq9nOYtwmOrMJnvrOLPJNlc3MGpgOnFhLjJ650WjafUoj7+/Jayfa3o/CzARcqxkfxgDzOKKWpIT4hg3tPOqwnFxwikFmVHbg/njP7bQ3Orh+xd1L/cSbmMGZTB6YDp/O86iy03drEHWU8NyU7lqYiHPL91BVd3RsH++6b0swESIr+BlOKcqL66oYeLwbJITjp9/8SkpyGRD1aGIP9q5vb31jTy3ZDtfm1BAcQz0XnymluazdFstewMMRdUdaWFvfVOP8y/d9b0LR+HxKL9/z3oxJnQswERISmI8aUnxHAjTav6DR5rZsOdQh+VhAikpzKSp1cPm6uha0f+H9ytoaVPuvCi6cy/tTS3JRzuoTbbJN4MsAj0Y8BYE/eZpQ5mzbAeVB60XY0LDAkwEZaeFb7Hlkq21qMLkLiziK3ES/dGUh9l7qJG/LNnONyYUUDQgLdLN6ZLRgzIYMyjwMJnvKZajIzgb7nsXjkIQHn23PGJtML2LBZiAQj+xAAAfFElEQVQIyk5NClsOZtGWGlIS4xhXePz1L/6KctPISE6grDJ6Flz+/v0ttHmU78dY78VnaskQlgUYJttc3UBqUjwFWeGbQdZeQVY/rj59KC8v38nO2iMRa4fpPSzARFA4ezCLK2qYNDynSyVB4uKEkwv6s6bykIstC96hxhaeX7qDb5xawLDc7hWDjLSppYNRhTfbzSbbVF3P6AjMIGvv9gtGERdnvRgTGhZgIignNTEsOZgDh5v5dE89k4tzunxuaWEWG6oO0dwa+UT/O+uqaW71cPXpwyLdlG4bNTCDEwZlfKGE/6bqhi6XunHD4MwUvn36MF5ZuYvtNYcj3RwT4yzARFBWmApeLtnqrazbnSKKJQWZNLd6juUIImnemioKsvoxoZMyN9Fuamk+y7bXsqfOO0x24HAz+xuaIpbgb+/2C0aSECc8stB6MaZnLMBEkK/gpdu9g8UVtfRLjKekoOu/mKMl0V93pIUPN+/jspLBiER2GKmnLnNmk7251tuLOfYUyyjowQAM7J/C9ZOH89dPdrF1v/ViTPdZgImgcK3mX1xRw6Si7G6VZB+em0pGSgJlEQ4wb6/fQ0ubMrV0SETbEQqjBqZz4uAM5q/xBphNPXyKpRtuPX8kSQlxPLJwc6SbYmKYBZgI8tUjczMPU9PQ5ORfuveMERGhpCAz4j2Y+c7w2LjCzqsQxIKpJfks23aAPXWNbK6uJy0pniGZ4St62pm8jGSmn1nE66sqKd8bXeugTOywABNBvpL9bha8XLq1Fjj+8186U1KYyadV9RFL9HuHx/ZzeWl+zA+P+VxW6n3C9/w1VWyqbmDUoIyou7cZ5xWTkhhvvRjTba4GGBG5REQ2iki5iNwbYH+yiLzo7F8iIkXO9iIROeo8EnmViPzB75yJIrLGOecRcf5VikiOiLwjIpud79lu3lsoHCvZ7+IQ2aKKGlKT4o89Brk7SgoyaW6LXKL/rfV7aPUoU51fyr3ByLzPhsk2761nzMDoSPD7y01PZvpZRbxRtjsqJnmY2ONagBGReOAx4FJgLHCNiLR/4MTNwAFVHQU8BDzot2+Lqo53vm7z2/44cAsw2vm6xNl+L7BQVUcDC533US0nDAHGm3/J6dFTM0udyQGRKnw5r6yKoTn9jk046C0uL81n+fYD7G9ojqr8i78Z5xaTmhjPwwusF2O6zs0ezOlAuapWqGozMAeY1u6YacCzzutXgClynHECEckH+qvqYlVV4E/A1wJc61m/7VHL7YKX+xua2FTd0K31L/6G5vQjs18iayKQhzlwuJmPyvcztWRI1A0h9dRlJZ/1yCJRRTkY2WlJfOecEcxbU8WGquhYcGtih5tPtCwAdvq93wWc0dExqtoqInWAL1kwQkQ+AQ4Bv1DVD53jd7W7pu9B6oNU1bd6bQ8wKFCjRGQGMANg2LDILthLTvAWvKw97E6Sf0mFN//SlQKXgfgS/WsiUDLmbWd47PJeNDzmU5yXzkn5/dlQdShqezAA3z2nmGc+2sbMBZv44/WTIt2c41q98yB//aQyqGPjRLjp7CKG5sRmVYhYEK2PTK4ChqlqjYhMBP5XRE4O9mRVVRHRDvbNAmYBTJo0KeAx4ZSdluTaENmcZTsYkJ507OmUPVFSmMmTH3qfIBlsuf9Q+FtZFcNzUzl5SHQ+trmnbjxrOC8s3Ul+FM0gay8zNZEbzhrO79/fwt5DjQzsH51tXbq1lhufXkqbRzt9JDhAQ1Mruw4cYdYN0R00Y5mbAaYSGOr3vtDZFuiYXSKSAGQCNc7wVxOAqq4QkS3AGOf4wg6uWS0i+apa5Qyl7Q31Dbkhx6UAs3xbLR9u3s/PLzupR/kXn5KCTFralI176intQsHMnqg93MzHW2q49bziXjc85vOt04bxrdOiv/TN1ycU8Nh7W5i/poobzx4R6eZ8wfJttdz09FIGZ6Yw55bJQQXBhxds5qEFm1hbWReSP8LMF7mZg1kGjBaRESKSBFwNzG13zFxguvP6SuBdp/eR50wSQESK8SbzK5whsEMiMtnJ1dwAvB7gWtP9tkc1t8rFzFywmQHpSVw7OTS/vHwJ9nAm+t9at4e2XjZ7LFaNGpjBiYMzmLem4ydyRsqK7bVMf2opg/oHH1wAbjqniP4pCcy0CQyucS3AqGorcAfwFrABeElV14nIAyJyhXPYbCBXRMqBe/hs5td5QJmIrMKb/L9NVWudfbcDTwLlwBbgTWf7r4CLRWQz8CXnfdTLSU0Mecn+Zdtq+Wf5fm47fySpSaHppBZm9yMrNTGsCy7nlVUxYkAaY/N75/BYrPFfHBotVu44wPSnljGwfwovzAg+uAD0T0nku+cWs2BDNWui9NHgsc7VdTCqOl9Vx6jqSFX9T2fbfao613ndqKpXqeooVT1dVSuc7a+q6snOFOVTVfUNv2suV9VTnGve4Qynoao1qjpFVUer6pf8AlJU85bsD22Sf+aCTd7eyxnDQ3ZNX6I/XD2YmoYmPt6yn6klvWdxZazz9SSjpRezaudBps9eSm56Ei/cMplB3cgN3XR2EZn9Enl44SYXWmhsJX+E5aQm0RDCgpdLt9byUXkNt50/kn5JoU3GlxZmsqm6nsaWtpBeN5C/r9uDR7HhsShSnJfO2Pz+zCvbHemmsHrnQa6fvYTsNG9wGdzNSRIZKYnccu4IFmzYS9mu6HmwXm9hASbCskJc8NLbe0kOae/Fp6Qgk1aP8uke91d1zyurojgvjRMHR+/03b5oamk+K3ccpPLg0Yi1oWzXQa6bvYSs1ERemDGZIT18Cuj0s4rISk20XIwLLMBEmK/gZSjyMEsqavh4Sw23nV8c8t4LQIkze2yNy3/p7atvYnFFDZfb8FjUmeosDn0zQsNkayvruO7JJWT2S+SFWyaH5BHT3l5MMe9+updVO60XE0oWYCIsO823mr/neZiZCzaTl5HMdZND33sBGJKZQk5akusr+j8bHov90vy9TdGANE4p6M8bZeEPMGsr67j2ySVkpHiDS2F26BZITj+riOzURB5eYLmYULIAE2Ghqke2uKKGRRXe3Eswi8y6I1yJ/nlluxk1MD1qnvBoPu/y0iGs3nmQnbVHwvaZ63cf4rrZS0hLimfOjMkhX32fnpzALecV897GfXyy40BIr92XWYCJsGzfEFkP18LMXLCJvIxkrj3D3UV74woz2by3gao6d8bg99Y3smRrrc0ei2K+YbL5YRomO3ikmetmL6FfYjxzZpzpWmmXG8709mIsFxM6FmAiLBQFLxdtqWFxRS3/6mLvxeeqSUMR4PfvbXHl+n9fuwe12WNRbWhOKuMKM/lbmIbJnviwggNHmpk9/TSG5bpXNyw9OYEZ543kH5v2sWK79WJCwQJMhCUnxJOenNCjp1o+vHATAzOS+bbLvRfw/nL55mlDmbNshyszif5WVsWYQelRXfzReP8AWFNZx/aaw65+Tu3hZp75aBuXleQzNgz16G44czg5aUk8bA9ZCwkLMFEgKzWx2zmYY72XC9zvvfjcceEoBOHRd0P7j7D6UCPLttUytcSS+9HO96gBtxddzvqggiMtbdw9ZbSrn+OTlpzArecV84H1YkLCAkwUyElL6nYOZuYCb+/lmtPDVzBxSFY/rjl9KC8v38WOmtAlet9cU+UMjw0O2TWNOwqzU5kwLIt5Lg6T1TQ08adF2/hq6RBGh7FHe/2Zw8lNS2KmzSjrMQswUSA7tXsVlT/esp8lW2u5PYy9F5/bLxxFfJzwPyHsxcxbU8WJgzMYNdCGx2LB1JJ81u0+xNb97gyTzfqggsaWNu4MU+/FJzUpgVvPL+bDzftZvi0mKk5FLQswUaA7JftVlZkLNjOofzJXh7H34jOofwrXTR7Oa59UhuQXzJ66RpZtO3BshpKJfseGyVwoHbOvvolnF21j2vgCRg0M/3T16yYPZ0B6Eg9ZL6ZHLMBEgazUxC4vtFxUUcPSrbXcfsGosPdefG47fyRJ8XE8EoKEqG/K62U2eyxmDMnqx6Th2a7MJvvjP7bQ3Orh+xeNCvm1g5GalMBt54/ko3LvvzPTPRZgokBXC16qKjPf8fZevnXa0M5PcEleRjI3nDWc11dVUr63Z/XJ5q2p4qT8/ozMs8WVsWRqaT6f7qmnfG9DyK6591Ajf168na9PKKQ4gv8/XHvGcAakJ1supgcswESB7C4WvFy0pYal2yLbe/G59byR9EuM79HitN0Hj7Ji+wEut95LzLn0lHxECGmy//F/bKHVo9w5JTK9F59+SfHcdn4xH2+pYUlFTUTbEqvcfGSyCZKvXEztkeZOH5jU0ubhN29vZHD/lIj2Xnxy0pK48ewifv/+Fu7Yc4gTB3dtrYKqMuuDCuCzMX0TOwZnpnDa8BzmrdnNXV/qeTK++lAjf1myg29MKGB4bloIWtgz100ezh8/qOCel1YzcmA6qkqbR/Go4vGAR5U2VTwKHo8SJ/DzqWM5fUROpJseFawHEwV8q/k7m6rc0ubh7jmrWLnjID/6ygkR77343HJuMelJCTzcxV6MqvLg3zfyzMfbuH7ycEYMiPwvFNN1l4/LZ1N1A5uqe/4Yh9+/V47Ho3z/ovDOHOtISmI8910+lgHpSRw62sLhplZa2jx4FOLiIDkxjvTkBLL6JTIgPYldB47yy3nrcZ6D2Oe52oMRkUuAh4F44ElV/VW7/cnAn4CJQA3wLVXd5rd/GLAeuF9VfyMiJwAv+l2iGLhPVWeKyP3ALcA+Z9/PVHW+KzcWYscKXh4n0d/qBJd5a6r4xdST+JeJheFqXqeyUpP4zjkjeHjhZtbtruPkIZmdnqOq/L83P2XWBxVcN3kY/3HFyWFoqXHDJacM5v/MXcffyqq45+LuTzGvqjvKC0t3cuXEQldLwnTVV8cN4avjglv8++KyHfzk1TW8t3EvF504yOWWRT/XejAiEg88BlwKjAWuEZGx7Q67GTigqqOAh4AH2+3/HfCm742qbnQeozweb1A6AvzV7/iHfPtjJbjAZ8+E6Wiqcmubh7te9AaXn192Et89tziczQvKzeeOoH9KAg+903kvRlX55bwNzPqggulnDuf/TjuFuDgrbBmrBmakcMaIHOaV7e7RX+6/f28LivK9CyObe+mJb5xayLCcVGYu2Gy9GNwdIjsdKFfVClVtBuYA09odMw141nn9CjBFnBK6IvI1YCuwroPrTwG2qOr2kLc8zLJ8ASbAEFlrm4e7X1zFvLIqfnbZidxyXvQFF4D+KYnMOK+YBRuqj/voWVXlgb+tZ/Y/t3LT2UXcf8XJVjW5F5haOoQt+w6zsZvDZJUHjzJn2Q6umjTUtWrJ4ZAYH8cdF42ibFcdCzfsjXRzIs7NAFMA7PR7v8vZFvAYVW0F6oBcEUkHfgL8x3GufzXwQrttd4hImYg8JSLZgU4SkRkislxElu/bty/QIWGXlOAdx23/VMvWNg//9tJq/lZWxU8vPZEZ542MUAuDc+PZI8hOTeR37wSe1qmq3D93HU9/tI2bzxnBfZePteDSS1x6ymDiejCb7LH3ygFiuvfi452gkMrMhZv6fC8mWpP89+Md7go4uV5EkoArgJf9Nj8OjATGA1XAbwOdq6qzVHWSqk7Ky8sLaaN7Ijst8XM9mNY2D/e8tJo3Vu/m3ktP5Nbzozu4gLfc+a3nj+T9jV8sFOjxKPe9vo5nF21nxnnF/GLqSRZcepEB6cmcOTKXv5VVdfmX6s7aI7y8fCdXnzYsJI9AjrSE+DjuuHAUaysP8c766kg3J6LcDDCVgP882kJnW8BjRCQByMSb7D8D+LWIbAPuBn4mInf4nXcpsFJVj/30VLVaVdtU1QM8gXeILmbkpCYdK9nf2ubhBy+vZu7q3fzkkhO5LQaCi88NZ3pLbPgvTvN4lF+8vpY/L97ObeeP5KeXnmjBpReaWjKErfsPs77qUJfOe+y9cgTh9gtj5//zznx9QgFFuZaLcTPALANGi8gIp8dxNTC33TFzgenO6yuBd9XrXFUtUtUiYCbwX6r6qN9519BueExE/BdRfB1YG7pbcV+WU/CyzaP84OXVvL5qNz++5AT+9YLY+kfnK7Hx4eb9LN1ai8ej/Oyva3h+yQ5uv2AkP7nkBAsuvdQlpwwmPk66NEy2o+YIr6zYxbfPGEZ+Zuz3XnwS4uP4/kWjWV91iLf7cC/GtQDj5FTuAN4CNgAvqeo6EXlARK5wDpuNN+dSDtwD3NvZdUUkDbgYeK3drl+LyBoRKQMuBP4tRLcSFjlpSeyvb+KHTnD50VdO4PYLYnM8+rrJw8nLSOa3b2/k3tfKmLNsJ9+/aBQ/+ooFl94sJy2Js7o4TPY/724mPk5i7g+pYEwbP4QRA9KYuWAzHk/f7MW4ug7GmSo8v922+/xeNwJXdXKN+9u9PwzkBjju+p60NdKyU5PYXdfIXz+p5EdfOSGmk50pifF874KR3P/GepZsreWuKaO5+0ujLbj0AZeX5vOTV9fw5IdbGdg/mYS4OOLjhIQ4Id758r1uaGrltU8queHM4QzqpIJFLPL2YkZxz0ureXv9Hi45peuVKjweZe7q3Zw1KpeBGbH338hKxUSJvIxkAH745TExHVx8rj59GG+vr+ac0QNitidmuu4rJw/mgTfW85/zNwR1fEpiHP8aQznGrrpi3BAefbecmQs28+Wxg7u03svjUe59rYyXlu9ixIA05syYHHOBWPpyAmrSpEm6fPnySDcDgLqjLaytrOPsUQMi3RRjeqTuSIs3n+jU7Wptc757PLR59NhXq0cZkpXS6x8w97+fVHL3i6t4/NpTuTTIensej/LT19bw4vKdXDWxkPlrqhjUP4UXoiTIiMgKVZ3U6XEWYKIjwBhjeqc2j3LxQ/8gMS6ON+86t9NejG9izJxlO7nzolH828VjWLnjADfMXsqg/inMmTG506K4bgs2wETrOhhjjOkV4uOEu6aMZmN1PW+u3XPcYz0e5ef/u+bYxJh/u3gMIsLE4Tk8+53TqT7UyNVPLGbvocYwtb5nLMAYY4zLLi8dwqiB6Ty8cFOHM8q8wWUtLyzdyR0XjuIeJ7j4TCrK4ZnvnM6eukaueWIxe+ujP8hYgDHGGJfFxwl3ThnNpuoG5q354joh32LkF5bu4HsXjuQHXx4TcNblaUU5PHPT6VTVNXLNrOgPMhZgjDEmDKaW5DN6YDqPLNxMm18vxuNR/v31tccWI//wy8dfL3b6iByevvE0quoa+fYTS9hX3xSO5neLBRhjjAkDXy9m897PejGqyn1z1/KXJTv41wtGBr0Y+YziXJ6+8TQqDxzlmicWR22QsQBjjDFhMrUknzGD0nl4wSZa2zz8++treW7xDm47fyQ/7mKlizOKc3n6Jm+Q+XaUBhkLMMYYEyZxccJdU8awZd9hrvrjIp5bvINbzy/udo2+ycW5PHXjaexygsz+hugKMrYOxtbBGGPCyONRLn34QzZW13PrecXcG4Lq4ou21HDTM0tRhX5J8cSJECdCfBzHXsfFQfyx196p08E+Crq9YNfBWKkYY4wJo7g44ZFrJlC26yBXTiwMSY2+M0fm8sItk5m7ejcej9Kmike9wcyjSpvHm+/x356VmhiCuzk+CzDGGBNmJwzO4ITBoS2RM2FYNhOGBXyQb8RYDsYYY4wrLMAYY4xxhQUYY4wxrrAAY4wxxhWuBhgRuURENopIuYh84XHIIpIsIi86+5eISFG7/cNEpEFEfui3bZvzaORVIrLcb3uOiLwjIpud79GV7TLGmD7GtQAjIvHAY8ClwFjgGhEZ2+6wm4EDqjoKeAh4sN3+3wFvBrj8hao6vt087HuBhao6GljovDfGGBMhbvZgTgfKVbVCVZuBOcC0dsdMA551Xr8CTBFnUriIfA3YCqwL8vP8r/Us8LUetN0YY0wPuRlgCoCdfu93OdsCHqOqrUAdkCsi6cBPgP8IcF0F3haRFSIyw2/7IFX11cHeAwzq+S0YY4zprmhdaHk/8JCqNgRY5XqOqlaKyEDgHRH5VFU/8D9AVVVEAtbAcYKSLzA1iMjGdocMAPb3+A6iR2+7H+h999Tb7gd63z31tvuBnt3T8GAOcjPAVAJD/d4XOtsCHbNLRBKATKAGOAO4UkR+DWQBHhFpVNVHVbUSQFX3ishf8Q7FfQBUi0i+qlaJSD6wN1CjVHUWMKujRovI8mBq7MSK3nY/0PvuqbfdD/S+e+pt9wPhuSc3h8iWAaNFZISIJAFXA3PbHTMXmO68vhJ4V73OVdUiVS0CZgL/paqPikiaiGQAiEga8GVgbYBrTQded+vGjDHGdM61HoyqtorIHcBbQDzwlKquE5EHgOWqOheYDfxZRMqBWrxB6HgGAX91hs0SgOdV9e/Ovl8BL4nIzcB24JshvyljjDFBczUHo6rzgfnttt3n97oRuKqTa9zv97oCGNfBcTXAlB4016fD4bMY1dvuB3rfPfW2+4Hed0+97X4gDPfUp58HY4wxxj1WKsYYY4wrLMA4OitrE4s6KqsTS0TkKRHZKyJr/bbFbFmgDu7nfhGpdH5Oq0Tkski2sStEZKiIvCci60VknYjc5WyP5Z9RR/cUkz8nEUkRkaUistq5n/9wto9wSnSVOyW7kkL+2TZEdqyszSbgYrwLQpcB16jq+og2rIdEZBswSVVjdv6+iJwHNAB/UtVTnG2/BmpV9VfOHwPZqvqTSLYzWB3cz/1Ag6r+JpJt6w5nSUC+qq50ZniuwFtF40Zi92fU0T19kxj8OTnVUdKcdYWJwD+Bu4B7gNdUdY6I/AFYraqPh/KzrQfjFUxZGxMBziLa2nabY7YsUAf3E7NUtUpVVzqv64ENeCt0xPLPqKN7iknO0o8G522i86XARXhLdIFLPyMLMF7BlLWJRR2V1Yl1vbEs0B0iUuYMocXMcJI/pxr6BGAJveRn1O6eIEZ/TiISLyKr8C5AfwfYAhx0SnSBS7/zLMD0bueo6ql4K1p/zxme6VXUO8Yb6+O8jwMjgfFAFfDbyDan65z6ga8Cd6vqIf99sfozCnBPMftzUtU2VR2Pt6LK6cCJ4fhcCzBewZS1iTn+ZXUAX1md3qDaGSf3jZcHLAsUK1S12vkF4AGeIMZ+Ts64/qvAX1T1NWdzTP+MAt1TrP+cAFT1IPAecCaQ5ZToApd+51mA8QqmrE1M6aSsTqzrVWWBfL+IHV8nhn5OTgJ5NrBBVX/ntytmf0Yd3VOs/pxEJE9EspzX/fBOZtqAN9Bc6Rzmys/IZpE5nCmHM/msrM1/RrhJPSIixXh7LfBZWZ2YuycReQG4AG/l12rg/wD/C7wEDMMpC6SqMZE47+B+LsA77KLANuBWv/xFVBORc4APgTWAx9n8M7w5i1j9GXV0T9cQgz8nESnFm8SPx9upeElVH3B+R8wBcoBPgOtUtSmkn20BxhhjjBtsiMwYY4wrLMAYY4xxhQUYY4wxrrAAY4wxxhUWYIwxxrjCAozp1ZyquF9pt+1uETluUT8RaTje/hC0K8+pZPuJiJzbbt/7IjLJeT3CqUj8lQDX+G+nOu5/d7MNF4jI3/ze/1JE/i4iyU4blvvtmyQi7/udpyLyVb/9fxORC7rTDtN7WYAxvd0LfPFR3Fc72yNpCrBGVSeo6oeBDhCRQuDvwA9U9a0Ah8wASlX1R8F8oN+q7UD7fgGcDXzdby3EQBG5tINTdgE/D+ZzTd9lAcb0dq8AU33PunCKFw4BPhSRdBFZKCIrxfvcnC9U0A7wV/6jInKj83qiiPzDKSb6VruV3r7ji0TkXadA4kIRGSYi44FfA9PE+1yRfgHanQ+8DfxcVb9QVUJE5gLpwAoR+Vagz3GOe0ZE/iAiS5zP/AIR+QHeenVfVdWjfrv+m46DyGqgTkQu7mC/MRZgTO/mrB5fivcXKHh7Ly85BRgb8f7FfipwIfBbp0xIp5xaVf8DXKmqE4GngECVEv4HeFZVS4G/AI+o6irgPuBFVR3f7pe6z7PAo6r6SoB9qOoVwFHn/BcDfY7f4YXAWap6T4BLnQ3cBlzqV9LdZxHQLCIXBmqDc7+/6GCfMRZgTJ/gP0zmPzwmwH+JSBmwAG+58mDLyp8AnAK845RB/wXeX+TtnQk877z+M3BOkNdfAFwnIqlBHn+8z3lZVds6OK8c73+Hjnoiv6SDIOI828ZXWsWYL7AAY/qC14EpInIqkKqqK5zt1wJ5wESnlHk1kNLu3FY+/+/Et1+AdU4PYryqlqjql0PY5l/jLcL68vFyJ0E6fJx91cBlwMxAPRVVfRfoB0zu4HzrxZgOWYAxvZ4z9PMe3mEs/+R+JrBXVVucX67DA5y+HRjrzKzKwpucB9gI5InImeAdMhORkwOc/zGf9Z6uxVtEMVh3A4eA2UEM3XX7c1R1E/AN4DknP9TeL4Efd3Du20A2UBrs55m+wwKM6SteAMbx+QDzF2CSiKwBbgA+bX+Squ7EWxV4rfP9E2d7M95S5w+KyGpgFXBWgM/9PnCTMwx3Pd5noQfFyRNNx5vwD5igD8XnOJ+1DLgJmCsiI9vtmw/sO87p/8nnn6dkDGDVlI0xxrjEejDGGGNcYQHGGGOMKyzAGGOMcYUFGGOMMa6wAGOMMcYVFmCMMca4wgKMMcYYV1iAMcYY44r/D9K/M56gcgTMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.model_selection  import cross_val_score\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "k_range = range(1, 31)\n",
    "k_error = []\n",
    "#循环，取k=1到k=31，查看误差效果\n",
    "for k in k_range:\n",
    "    knn = KNeighborsClassifier(n_neighbors=k)\n",
    "    #cv参数决定数据集划分比例，这里是按照5:1划分训练集和测试集\n",
    "    scores = cross_val_score(knn, X_std, y, cv=5, scoring='accuracy')\n",
    "    k_error.append(1 - scores.mean())\n",
    "\n",
    "#画图，x轴为k值，y值为误差值\n",
    "plt.plot(k_range, k_error)\n",
    "plt.xlabel('Value of K for KNN')\n",
    "plt.ylabel('Error')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用我们自己撰写的K近邻算法测试数据，用同样的作图法输出每个K值的误差结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl4VOXZ+PHvnWWyA0kIAdmSsKgoyKaAgKKoRVGpdYOigFXRqm19fd/62r31Z/tW2yputbWitdYFl7ZSoCKLW6sCYQuGLQFRlhBCCGHPev/+mBMc4ySZJHNmJsn9ua65cuY5zznzHAdz59lFVTHGGGOCLSrcBTDGGNM+WYAxxhjjCgswxhhjXGEBxhhjjCsswBhjjHGFBRhjjDGusABjjDHGFRZgjDHGuMICjDHGGFfEhLsA4dS1a1fNysoKdzGMMaZNWb169X5VzWgqX4cOMFlZWeTm5oa7GMYY06aIyGeB5LMmMmOMMa6wAGOMMcYVFmCMMca4wgKMMcYYV1iAMcYY4wpXA4yITBKRLSJSKCL3+TkfJyLznPMrRCTL59wQEflIRPJFZIOIxDvp7zr3XOe8ujV1L2OMMaHnWoARkWjgSeBSYBAwTUQG1ct2M1Cmqv2BR4AHnWtjgL8Ct6vqGcAEoMrnuumqOtR57WvsXsYYY8LDzXkw5wCFqrodQEReAaYAG33yTAF+7hy/DjwhIgJcAuSp6noAVS0N4PP83ktd2BN61Y4DfLC1JKC8IsLVw3vRJz0x2MUwxpiI5maA6Qns9Hm/CxjVUB5VrRaRciAdGAioiCwGMoBXVPUhn+ueE5Ea4A3gASeINHSv/b4fKCKzgdkAffr0adGDrfmsjMffKQworyocqajmJ5fXr7wZY0z7Fqkz+WOAccDZwDFgmYisVtVleJvHdotICt4AcyPwl0BvrKpPA08DjBw5skW1m9vO78dt5/cLKO/4h5Zz4GhlSz7GGGPaNDc7+XcDvX3e93LS/OZx+l06A6V4azvvq+p+VT0GLAKGA6jqbufnYeAlvE1xjd0rrFITPRZgjDEdkpsBZhUwQESyRcQDTAXm18szH5jpHF8DLHeauxYDg0Uk0QkW5wMbRSRGRLoCiEgscDnwSRP3CqvURA8Hj1mAMcZ0PK41kTn9IHfhDRbRwLOqmi8i9wO5qjofmAu8ICKFwAG8QQhVLRORh/EGKQUWqepCEUkCFjvBJRpYCvzJ+Ui/9wq3tCQP2/cfCXcxjDEm5Fztg1HVRXibt3zTfupzfAK4toFr/4p3qLJv2lFgRAP5G7xXOKUmeig7WtV0RmOMaWdsJr/L0pJiOVJRTUV1TbiLYowxIWUBxmWpSR4ADh6zWowxpmOxAOOytERvgLGRZMaYjsYCjMu6OAGmzAKMMaaDsQDjsjSniazMmsiMMR2MBRiXpSbFAnDA5sIYYzoYCzAuS7UmMmNMB2UBxmWx0VGkxMdYJ78xpsOxABMCaUkeyqyJzBjTwViACQFb8NIY0xFZgAmB1MRYq8EYYzocCzAhkJpk65EZYzoeCzAhkJZofTDGmI7HAkwIpCZ5OFZZw4kqW/DSGNNxWIAJgS9m81stxhjTcViACYFUW/DSGNMBWYAJgZM1GOvoN8Z0IK4GGBGZJCJbRKRQRO7zcz5OROY551eISJbPuSEi8pGI5IvIBhGJF5FEEVkoIpud9F/75J8lIiUiss553eLmszVHmq1HZozpgFwLMCISDTwJXAoMAqaJyKB62W4GylS1P/AI8KBzbQze7ZJvV9UzgAlA3Z//v1XV04BhwFgRudTnfvNUdajzesalR2s2W7LfGNMRuVmDOQcoVNXtqloJvAJMqZdnCvC8c/w6MFFEBLgEyFPV9QCqWqqqNap6TFXfcdIqgTVALxefISi6JHhrMNbJb4zpSNwMMD2BnT7vdzlpfvOoajVQDqQDAwEVkcUiskZE7q1/cxHpAlwBLPNJvlpE8kTkdRHp7a9QIjJbRHJFJLekpKSlz9YsMdFRdE6ItRqMMaZDidRO/hhgHDDd+XmViEysO+k0ob0MPKaq253kfwJZqjoEWMIXNaMvUdWnVXWkqo7MyMhw8xm+JC3JwwHbdMwY04G4GWB2A761iF5Omt88TtDoDJTire28r6r7VfUYsAgY7nPd00CBqs6pS3Ca0Sqct88AI4L4LK2Wmmg1GGNMx+JmgFkFDBCRbBHxAFOB+fXyzAdmOsfXAMtVVYHFwGBn1FgMcD6wEUBEHsAbiO72vZGI9PB5eyWwKcjP0yppSbaisjGmY4lx68aqWi0id+ENFtHAs6qaLyL3A7mqOh+YC7wgIoXAAbxBCFUtE5GH8QYpBRap6kIR6QX8CNgMrPGOB+AJZ8TYd0XkSqDaudcst56tJVITPeTvORTuYhhjTMi4FmAAVHUR3uYt37Sf+hyfAK5t4Nq/4h2q7Ju2C5AG8v8A+EEri+yaVKcGo6o4gdEYY9q1SO3kb3dSEz1UVNdy3Ba8NMZ0EBZgQqRuNn+ZjSQzxnQQFmBCJNVm8xtjOhgLMCFSt+CljSQzxnQUFmBCJNX2hDHGdDAWYEIkzfaEMcZ0MBZgQqRTQixRYn0wxpiOwwJMiERHCZ0TYm1PGGNMh2EBJoRSkzw2TNkY02FYgAmhtESPNZEZYzoMCzAhlGoLXhpjOhALMCGUluixYcrGmA7DAkwIpSZ5KDtahXdHAmOMad8swIRQWlIslTW1HK20BS+NMe2fBZgQ6mLrkRljOhALMCFks/mNMR2JBZgQsvXIjDEdiasBRkQmicgWESkUkfv8nI8TkXnO+RUikuVzboiIfCQi+SKyQUTinfQRzvtCEXlMnO0hRSRNRJaISIHzM9XNZ2uJNAswxpgOxLUAIyLRwJPApcAgYJqIDKqX7WagTFX7A48ADzrXxuDdLvl2VT0DmADUTYF/CrgVGOC8Jjnp9wHLVHUAsMx5H1G+aCKz2fzGmPbPzRrMOUChqm5X1UrgFWBKvTxTgOed49eBiU6N5BIgT1XXA6hqqarWiEgPoJOqfqzesb5/Ab7u517P+6RHjJT4GKKjxDr5jTEdgpsBpiew0+f9LifNbx5VrQbKgXRgIKAislhE1ojIvT75dzVwz0xVLXKO9wKZ/golIrNFJFdEcktKSlr2ZC0UFSWkJtqCl8aYjiEm3AVoQAwwDjgbOAYsE5HVeANQk1RVRcTvbEZVfRp4GmDkyJEhn/HYxdYjM8Z0EG7WYHYDvX3e93LS/OZx+l06A6V4aybvq+p+VT0GLAKGO/l7NXDPYqcJDefnvqA+TZCkJdp6ZMaYjsHNALMKGCAi2SLiAaYC8+vlmQ/MdI6vAZY7fSuLgcEikugEnvOBjU4T2CERGe301cwA3vRzr5k+6RElNSnWRpEZYzoE1wKM06dyF95gsQl4VVXzReR+EbnSyTYXSBeRQuAenJFfqloGPIw3SK0D1qjqQueaO4BngEJgG/AvJ/3XwMUiUgBc5LyPOGm2J4wxpoNwtQ9GVRfhbd7yTfupz/EJ4NoGrv0r3qHK9dNzgTP9pJcCE1tZZNelOn0wqoozhccYY9olm8kfYmlJHqprlcMV1eEuijHGuMoCTIil2oKXxpgOwgJMiNUtF2MjyYwx7Z0FmBDrkhgL2Hpkxpj2zwJMiH1Rg7GRZMaY9s0CTIjVLdl/0Gowxph2zgJMiKXExRATJdYHY4xp9yzAhJiIkJrksT4YY0y7ZwEmDGw9MmNMR2ABJgxSk2Ips05+Y0w7ZwEmDFITPbYnjDGm3bMAEwapSbYnjDGm/bMAEwZpid5O/trakO93ZowxIWMBJgxSkzzUKhw+YQteGmPaLwswYZCW5F0uxvphjDHtmQWYMKhbUdmGKhtj2jMLMGFQtx6ZdfQbY9ozVwOMiEwSkS0iUigi9/k5Hyci85zzK0Qky0nPEpHjIrLOef3BSU/xSVsnIvtFZI5zbpaIlPicu8XNZ2uNkzUYayIzxrRjrm2ZLCLRwJPAxcAuYJWIzFfVjT7ZbgbKVLW/iEwFHgSud85tU9WhvvdU1cPAyTQRWQ38zSfLPFW9K/hPE1ypVoMxxnQAbtZgzgEKVXW7qlYCrwBT6uWZAjzvHL8OTJQAN6oXkYFAN+CDIJU3ZJI80Xiio6wGY4xp19wMMD2BnT7vdzlpfvOoajVQDqQ757JFZK2IvCci4/3cfyreGovvZJKrRSRPRF4Xkd7+CiUis0UkV0RyS0pKWvBYredd8DKWg7ZcjDGmHYvUTv4ioI+qDgPuAV4SkU718kwFXvZ5/08gS1WHAEv4omb0Jar6tKqOVNWRGRkZLhQ9MLZcjDGmvXMzwOwGfGsRvZw0v3lEJAboDJSqaoWqlgKo6mpgGzCw7iIROQuIcc7h5CtV1Qrn7TPAiOA+TnCl2XIxxph2zs0AswoYICLZIuLBW+OYXy/PfGCmc3wNsFxVVUQynEECiEgOMADY7nPdNL5ce0FEevi8vRLYFLQncUFqktVgjDHtm2ujyFS1WkTuAhYD0cCzqpovIvcDuao6H5gLvCAihcABvEEI4DzgfhGpAmqB21X1gM/trwMuq/eR3xWRK4Fq516zXHq0oEhLjKwazOelx8hIiSPBEx3uohhj2gnXAgyAqi4CFtVL+6nP8QngWj/XvQG80ch9c/yk/QD4QWvKG0qpibEcPF5FTa0SHRXQwDnX7Dl4nIsfeY/bzu/HPRcPbPoCY4wJQKR28rd7qUkeVKH8ePhHkj35TiEV1bXk7y4Pd1GMMe2IBZgwqVsuJtzrke0qO8arud7R5Fv3HQ5rWYwx7YsFmDCpWy7mYJg7+p98ZxuCMO2cPuw8cJyjFbaFgDEmOCzAhEkk1GB2HjjGa7k7uf7s3pw/0DsnqHDfkbCVxxjTvliACZOT65GFsQbz5DuFREUJd1zQj4GZyQBsLbZmMmNMcDQZYEQkWkR+G4rCdCRpJ/eECU8n/+elx3ht9S6+eU4fenROoG96Ep6YKAqsBmOMCZImA4yq1gDjQlCWDiXBE018bFTYajCPLy8gJkr49oR+AERHCf0ykq0GY4wJmkDnwawVkfnAa8DRukRV/VvDl5impCZ6wtIHs2P/Uf62djczxvQls1P8yfSBmcnk7igLeXmMMe1ToAEmHigFLvRJU768F4tpptQwzeZ/fHmht/Zyfr8vpQ/MTOHNdXs4UlFNcpyrc3CNMR1AQL9FVPUmtwvSEaUleULeRPbp/qP8fe0ubhqbTTef2gvAgG7ejv6C4sMM65Ma0nIZY9qfgEaRiUgvEfm7iOxzXm+ISC+3C9fepSZ5KDsW2k7+x5cV4ImJ4vZ6tRfw1mAACoqto98Y03qBDlN+Du/Kx6c4r386aaYV0hJjQ9oHs63kCP9Yt5sbR/clIyXuK+d7pyUSFxNlHf3GmKAINMBkqOpzqlrtvP4MhG+3rnYiNclD+fEqqmtqQ/J5jy0rIC4mmtv81F7AO5Ksf7dkttpQZWNMEAQaYEpF5AZnTky0iNyAt9PftELdbP6DIVjwsnDfYeav38OMc/vSNfmrtZc6AzNTKLAajDEmCAINMN/CuwfLXrzbGV8DWMd/K3VxJluGYiTZo8sKSYiN5rbz/Nde6gzITKao/ASHToR/lWdjTNvW5CgyZ2fJb6jqlSEoT4fyxWx+dwPM1uLDLMjbw+3n9ztZa2rIwG5fdPSP6GsjyYwxLRfoTP5pIShLh5OaFAu4vx7Zo8sKSIyNZvb4r+zT9hVfjCSzZjJjTOsE2kT2HxF5QkTGi8jwuldTF4nIJBHZIiKFInKfn/NxIjLPOb9CRLKc9CwROS4i65zXH3yuede5Z925bo3dK5KlnVzw0r3mqC17D7NoQxGzxmadXGCzMb1SE0iIjWarDVU2xrRSoNO1hzo/7/dJU748s/9LnKa1J4GLgV3AKhGZr6obfbLdDJSpan8RmQo8CFzvnNumqkPxb7qq5tZLa+xeESk1BE1kjy7bSpInhlsDqL0ARDkjyQps8zFjTCsFsppyFPCUql5Q79VgcHGcAxSq6nZVrQReAabUyzMFeN45fh2YKCIt3aA+mPcKifjYaBI90a518m8qOsSiDXu5aWzWyQEFgRiQaYteGmNaL5A+mFrg3hbcuyew0+f9LifNbx5VrQbKgXTnXLaIrBWR90RkfL3rnnOax37iE0Qau9dJIjJbRHJFJLekpKQFjxVcqYkeDrjUBzNn6VZS4mK4ZVxgtZc6AzNTKD5UQXmIVxkwxrQvgfbBLBWR/xGR3iKSVvdysVxFQB9VHQbcA7wkIp2cc9NVdTAw3nnd2Jwbq+rTqjpSVUdmZIR/rmhqUqwrNZj8PeUszi/mW+Oy6ZwY26xrT24+Zs1kxphWCDTAXA/cCbwPrHZe9ftA6tsN9PZ538tJ85tHRGKAzkCpqlaoaimAqq4GtgEDnfe7nZ+HgZfwNsU1eK8Any9svDWY4NcU5iwtICU+hm+Ny272tQOcocrWTGaMaY2AAoyqZvt5NdXusgoYICLZIuIBpuJdz8zXfGCmc3wNsFxVVUQynEECiEgOMADYLiIxItLVSY8FLgc+aexegTxfOKUlBX/J/k92l7NkYzG3jMuhc0Lzai8APbskkOiJtkUvjTGt0miAEZF7fY6vrXfuV41d6/SD3AUsBjYBr6pqvojcLyJ1kzbnAukiUoi3KaxuKPN5QJ6IrMPbYX+7qh4A4oDFIpIHrMNba/lTE/eKaG7sCTNn6VY6xcdw07isFl0fFSUM6GYd/caY1mlqmPJU4CHn+Ad4d7SsMwn4YWMXq+oiYFG9tJ/6HJ8ArvVz3RvAG37SjwIjGvgsv/eKdGlJHg5XVFNVU0tsdKAtlg3L23WQpZv28d8XD6RTfPNrL3UGZKbw7pbwD4IwxrRdTf1GkwaO/b03LZB6crJlcGoxc5YW0CUxllljs1p1n4GZyew/UhGWHTeNMe1DUwFGGzj29960QNrJBS9b39G/9vMylm/ex63jc0hpRe0FvDUYsI5+Y0zLNdVEdpaIHMJbW0lwjnHexzd8mQlU3XpkwZjNP2dpAamJscw8N6vV9zq1LsDsO8KonK9MJzLGmCY1GmBUNTpUBemo6paLaW0T2erPynhvawn/O+k0kuMCXQGoYT06x5MSF2OLXhpjWqz1vcqmVeoWvGxtDWbO0q2kJXmYMaZvMIqFiNDflowxxrSCBZgw6+LMsm9NZ/rqzw7wQcF+bjsvh6Qg1F7qDOyWYnNhjDEtZgEmzOJiokmOi2nVkv2PLCmga7KHG4NUe6kzIDOZ0qOVlB6pCOp9jTEdgwWYCJCaFNviPphVOw7w78L93HZePxI9wau9wBebj9neMMaYlrAAEwHSEj0t7oN5ZMlWuibHccPo4NZewGd3S1v00hjTAhZgIkBqkqdFNZiPt5fy4bZSvj2hHwme4A/4y+wUR0p8jHX0G2NaxAJMBEhtYQ3mkSVb6ZYSx/RRfVwolXck2cDMFGsiM8a0iAWYCNCSBS8/3LafFZ8e4NsT+hEf6950pYGZyRQUH6YNLExtjIkwFmAiQFpSLEcrazhRVRNQflVlzpICMjvFMe0cd2ovdQZ0S6HsWBX7j9iaZMaY5rEAEwHqFrw8GOBQ5Q+3lbJyxwHumNDf1doL+I4ks34YY0zzBHdcq2mRNJ/lYrp3bnyJt6qaWn779hZ6dI7n+rN7N5o3GE5un1x8mLH9u7r+ecGSt+sgf/rgU2qD3LTXJy2R/7poIJ4Y+9vMmKZYgIkAJ5fsb6IfprK6lrteWsPazw/yyPVnuV57AchIiaNzQmyb6+h/Ynkh720toVdqQtDuqcDCvCIKig/z5PThxMXYUn3GNMYCTAQ4uR5ZI0OVK6truePFNSzdVMzPrxjEVcN6haRs3pFkyW1q0cvDJ6p4d2sJ00f14WdXnBHUe7/w0Q5+8mY+d/x1Db+/wYKMMY1xtZ4vIpNEZIuIFIrIV7YwFpE4EZnnnF8hIllOepaIHBeRdc7rD056oogsFJHNIpIvIr/2udcsESnxueYWN58tmE6uqNxADaaiuoY7XlzN0k3F3D/lDGaNzQ5l8RiQmcLWNjSSbNmmfVRW13L5kB5Bv/eNY7J44OtnsmzzPm5/YXXAAzOM6YhcCzAiEg08CVwKDAKmicigetluBspUtT/wCPCgz7ltqjrUed3uk/5bVT0NGAaMFZFLfc7N87nmmaA/lEvqFrw84GfTsYrqGr791zUs3bSP//f1M5kxJivEpYOB3ZI5dKKafYfbxppkC/L20KNzPMN6p7py/xtG9+VXVw3mnS0l3GZBxpgGuVmDOQcoVNXtqloJvAJMqZdnCvC8c/w6MFFEGtyKWVWPqeo7znElsAYITVuRi2Kjo0iJj/nKbP4TVTXc9sJqlm/exy+vOpMbXVgOJhBtaSRZ+fEq3t+6n8mDexAV5d6u3t8c1Ydff2Mw7xeUcOtfci3IGOOHmwGmJ7DT5/0uJ81vHlWtBsqBuu0Ts0VkrYi8JyLj699cRLoAVwDLfJKvFpE8EXldRPwOsRKR2SKSKyK5JSUlLXowN6QlfXk2/4mqGma/sJp3t5Twf98YzPRR4Qku4Lt9cuR39C/dWExlTS2TXWgeq2/qOX148BtD+HfhfgsyxvgRqWMti4A+qjoMuAd4SUQ61Z0UkRjgZeAxVd3uJP8TyFLVIcASvqgZfYmqPq2qI1V1ZEZGhqsP0RypiV+sR3aiqoZb/5LLBwUlPHj1YNcnUzala7KH1MTYNtHRv3BDET27JDC0d5eQfN51Z/fmoau9Qebm51dxvNKCjDF13AwwuwHfWkQvJ81vHidodAZKVbVCVUsBVHU1sA0Y6HPd00CBqs6pS1DVUlWt6yR4BhgRxGdxXZqz4OXxyhpueT6Xfxfu58Grh3D92eENLuAdSVbX0R/Jyo9V8UFBCZOH9KCRltagu3Zkb357zVl8uK2Ub/15Fccqq0P22cZEMjcDzCpggIhki4gHmArMr5dnPjDTOb4GWK6qKiIZziABRCQHGABsd94/gDcQ3e17IxHxbRO5EtgU5OdxVWqih+JDFdz8/Cr+s20/v7nmLK4b6f5EykB5hyofieiRZG9v3EtVjTJ5sPvNY/VdPaIXD193Fis+tSBjTB3XAozTp3IXsBjvL/tXVTVfRO4XkSudbHOBdBEpxNsUVjeU+TwgT0TW4e38v11VD4hIL+BHeEelrak3HPm7ztDl9cB3gVluPZsb0pJiKTlcwUfbS/ndtWdxzYjIGrswMDOFwxXV7D10ItxFadDCDUX0Sk1gSK/OYfn8q4b14uHrhrLy0wPMem4VRyuCG2R2HzxOZXVtUO9pjJtcnWipqouARfXSfupzfAK41s91bwBv+EnfBfht+1DVHwA/aGWRw+aULglECfzuurNCNomyOQZ0+6Kjv0fn4M2OD5aDxyr5d8F+bh6fHdLmsfq+PqwnIvBf89bx0Fub+cWUM4Ny3x37j3LJI+8ztE8Xnpt1NklxNkfaRL5I7eTvcKaP6st7378gIoMLfLEmWaR29L+dX0x1rXL54FPCXRSmDO3JdSN78/LKnRSVHw/KPR9fXghA7o4D3ORC7cgYN1iAiRCemCh6pyWGuxgNSk+OIz3JE7Ed/Qs2FNEnLZEze3ZqOnMI3HlBf2pVefKdwlbfa3vJEf6+dhczxvTl0anDWP15GTOfXckRCzImwlmAMQEbkJkckXNhyo5W8p/C/SEfPdaY3mmJXDuyN/NW7WT3wdbVYh5fXognJorbzu/HFWedwmNTh7F250FmPruSwycC2+LBmHCwAGMCNjAzhcJ9kTeSbHH+Xmpq1ZW1x1rjrgv7A7SqFrOt5AhvrtvNzDFZZKTEATB5SA+emDaM9TsPMuPZlRyyIGMilAUYE7ABmSkcqahmT3lkjSRbuKGI7K5JDOoRGc1jdXp2SeD6s3vzWu5OdpUda9E9HltWQHxsNLPPy/lS+qWDe/DEN4ezYVc5M+ZakDGRyQKMCdjAbs7mY3sjpx+m9EgFH24rZfLgyGke83XnBf0RpEW1mMJ9h5m/fg8zxmSRnhz3lfOTzuzO76cPJ39POTfOXUn5cQsyJrJYgDEBi8RFL99ymsdCsfZYS/TonMDUc3rzWu4udh5oXi3m0WWFJPqpvfi65IzuPDV9BBv3lHPj3BWUB7jttjGhYAHGBCw1yUPX5LiI6uhfmFdETkYSp3VPCXdRGnTHhP5ERQlPLA+8FrO1+DAL8vYw89yskxvSNeSiQZn84YYRbC46zPS5H3OwkY3rjAklCzCmWQZmJlOwLzJqMCWHK/h4eymXR2jzWJ3uneP55jl9eH3NLj4vDawW8+jSApI8Mdw6vuHai6+Jp2fyxxtHsHXvEaY/s6LJ7beNCQULMKZZBmamUFB8hOqa8C9Z8lb+XmoVJg8J/+TKpnx7Qj9iooTHlxc0mXfz3kMs3FDErHOzSG2i9uLrgtO68fSMERTsO8JNf14VcaP9TMdjAcY0y6jsNI5X1bDy0wPhLgoL8/bQv1vyyVUGIllmp3imj+rL39buZsf+o43mfXRpASlxMdwyvvlbY084tRv3TTqNdTsP8lmAtSVj3GIBxjTLhFO7keiJZsGGorCWY9/hE6z49EDEjh7z5/YJOcRGC481UovZuOcQ//pkLzeNzaJLYuC1F1/nn+rd5+ij7aUtut6YYLEAY5olwRPNxNMzeeuTvWFtJnvrk72oErGjx/zplhLPDaP68o+1u9le4n+gxKPLtpISH8PN4wLre/Enp2sSGSlxfLTNAowJLwswptkmD+7BgaOVfLw9fM1kC/KKGJiZfHLodFtx2/n98MREnVy80tcnu8tZnF/Mt8Zm0zkxtsWfISKMyUnn4+2l1g9jwsoCjGm2CadmkOSJZuGGPWH5/OJDJ1i14wCXt4HO/foyUuKYMSaLN9ftZlu9WsycpQWkxMfwrXHN73upb0y/dPYdrmB7E/09xrjJAoxptvjYaC4a5G0mqwpDM9m/NhShCpeFYefKYJh9Xg5xMdE8tuyLvpgNu8pZuqmYW8bl0Dmh5bWXOqNz0gH42PphTBhZgDEtMnlwD8qOVfFhGNr5F24o4rTuKfTvFvmjx/zpmhzHjHP7Mn/9HgqdOUVzlm6lc0IsN43LCspnZKUMCxcMAAAaJ0lEQVQn0r1TvPXDmLByNcCIyCQR2SIihSJyn5/zcSIyzzm/QkSynPQsETnubIm8TkT+4HPNCBHZ4FzzmDhDiEQkTUSWiEiB8zPVzWfr6M4bmEFyXAwL80LbTLa3/ASrdpRF3MrJzXXbef1IjI3m0WWFrN95kGWb93Hr+Gw6xbe+9gLefpjROWl8vP2A9cOYsHEtwIhINPAkcCkwCJgmIoPqZbsZKFPV/sAjwIM+57ap6lDndbtP+lPArcAA5zXJSb8PWKaqA4BlznvjkvjYaC4elMni/OKQ7hO/0Bke3Vabx+qkJXmYeW4WC/L28MO/b6BLYiwzz80K6meM6ZfO/iMVX+nrMSZU3KzBnAMUqup2Va0EXgGm1MszBXjeOX4dmCiNTGoQkR5AJ1X9WL1/lv0F+Lqfez3vk25ccvmQHpQfr+I/2/aH5PNUlQV5exjUoxM5GW2zeczXreNzSPLEkL/nELeOzyElSLWXOnX9MMFoJisqP877W0tafR8TfqrKY8sKQrL9uZsBpiew0+f9LifNbx5VrQbKgXTnXLaIrBWR90RkvE/+XQ3cM1NV62b/7QUy/RVKRGaLSK6I5JaU2P8wrTFuQFdS4mNYmOf+pEtV5Xdvb2Xt5we5ekQv1z8vFFKTPHx7Qj9O6Rwf9NoLQJ+0RE7pHB+U4eS/WrSZmc+tPNlnZNquDbvLeXjJVtbuPOj6Z0VqJ38R0EdVhwH3AC+JSMC7STm1G78Nz6r6tKqOVNWRGRkZwSltBxUXE80lg7qzOH+vq81kqspvFm/hiXcKmXZOH25y4ZdxuNx5QX8+/MFEkuNign5vbz9M6+fDHK+sYdmmYlS9WwiYtm1hXhGx0cLXBnV3/bPcDDC7gd4+73s5aX7ziEgM0BkoVdUKVS0FUNXVwDZgoJPf989X33sWO01odU1p+4L6NMavy4f04PCJav5d6E5tUFX59Vub+f2725g+qg+//PqZREW1jaVhIsHofumUHq2kYF/L+2He3bKPY5U1nJ2VyoK8PRG1H5BpHm8zcxHj+ndt1WTeQLkZYFYBA0QkW0Q8wFRgfr0884GZzvE1wHJVVRHJcAYJICI5eDvztztNYIdEZLTTVzMDeNPPvWb6pBsXje3flU7xMSxwoZlMVfm/f23mj+9t58bRfXnAgkuzjQlCP8yCDUV0Tfbw1A0jvCPflja9IrSJTOt2HmT3weMhm6TsWoBx+lTuAhYDm4BXVTVfRO4XkSudbHOBdBEpxNsUVjfy6zwgT0TW4e38v11V6xqS7wCeAQrx1mz+5aT/GrhYRAqAi5z3xmWemCi+dkZ3luQXU1FdE7T7qiq/XLiJp9/fzswxfbl/yhltZlHLSNI7LZGeXRJaPOHyWGU1yzftY9KZ3emaHMdNY7NZuKGIzXsPBbmkJhQW5hXhiY7iokF+u6iDLvgNvz5UdRGwqF7aT32OTwDX+rnuDeCNBu6ZC5zpJ70UmNjKIpsWmDykB6+t3sUHW/cH5R+uqnL/go08958dzDo3i59dMciCSyuMzkln+eZiamu12TXA5Zv3cbyqhsmDvX/x3jI+m+c/3MGjSwt46oYRbhTXuKS2Vlm0oYjzBnYNymoRgYjUTn7Thozt7/0HuzAIS/irKr/4pze4fGtstgWXIBjTL52yY1VsaUHfycK8Iromx3FOdhoAXRI93DQ2i399speNe6wW05as3XmQPeUnQroCuQUY02qx0VFMOqM7SzYWc6Kq5c1kqsrP5ufz5w93cMu4bH5y+ekWXIJgdI43ODS3mexoRTXLN+/jssHdifap+dw8LoeU+BgeXbY1qOU07lqYV4QnJoqLTg9N8xhYgDFBMnlID45UVLd4Ml5trfKTNz/hLx99xuzzcvjRZAsuwdIrNZHeaQnN7uhftnkfFdW1TK63akLnxFi+NTabxfnF5O8pD2ZRjUvqmsfOH5gR9Am9jbEAY4JiTL90UhNb1kxWW6v8+M1P+OvHn3Pb+Tn84NLTLLgE2ZicdFZ8eoDa2sDnwyzM20O3lDhGZqV95dy3xmWTEh/DHBtR1ias+byMvYdOhHwNPwswJihio6OYdGYPljazmay2VvnRPzbw0orP+faEftw3yYKLG0bnpFN+vIpNAY7+OlJRzTtbSrhscI8vNY/V6ZwQy63jc1iysZgNu6wWE+kWOM1jE0PYPAYWYEwQXT6kB0cra3h3S2BzXGtrlR/8bQMvr9zJHRP6ce/XTrXg4pIv9ocJbNmYZZu8i5g21iF809gsOifEMmep9cVEsrrmsQtOzXBlxYjGWIAxQTMqO430JE9Aky5rapV738hjXu5Ovnthf75vwcVVp3RJoG96YsD9MAvyiujeKZ4RfRre9SIlPpZbx2ezbPM+1odgXSvTMrmflbHvcAWTw7ADrAUYEzQx0VFMOrM7yzbt43hlw81kNbXKva/n8frqXXxv4gDuucSCSyh4+2FKqWmiH+bwiSrec5rHmpo3M/PcLLokWi0mki3M20NcTBQTT+sW8s+2AGOCavKQHhyvquGdBprJamqV77+2njfW7OLuiwbwXxcPDHEJO67ROekcPlHNpqLG+2GWbiqmsqbx5rE63lpMDu9sKWHt52XBKqoJkppaZdEne7nwtG4khbh5DCzAmCAblZ1O12SP3yX8q2tquefVdfxt7W7uuXggd19kwSWUAt0fZmFeEad0jmdY7y4B3XfmuVmkJsbaiLIItGrHAUoOV4Rs7bH6LMCYoIqOEi49swfLNhdzrLL6ZLo3uKznzXV7+P7XTuW7EweEsZQdU/fO8WR3TWp0wmX58Sre37o/oOaxOslxMcw+rx/vbS1h9WdWi4kkC/OKSIiN5oLTwrM1iQUYE3STh/TgRFUtyzd7m8mqa2q5e9465q/fw72TTuXOC/qHuYQd1+icdFZ+eoDqGv/79yzdGHjzmK8ZY/qSnuSxvpgIUlOr/OuTIi48vRuJntA3j4EFGOOCs7PSyEiJY2FeEVU1tXzvlXUsyCvivktP444JFlzCaUy/dA5XVLOxgX6YBXl76NklgaEBNo/VSYqL4bbzc/igYD+5O1q/g6ZpvRWflrL/SCWXDw7t5EpfFmBM0EVHCZed2Z3lm/dx54trWLihiB9ddjq3n98v3EXr8EY7i1b664cpP1bFBwX7mTykR4tG9d0wui9dkz08YrWYiLAwr4hETzQTTg396LE64ak3mXZv8pBTeP6jz3h7YzE/nnw6t4zPCXeRDNCtUzz9MpL4aHspt9UL+Is37qW6Vr+y9ligEj0x3H5+Px5YuIkV20sZ5QwqiETHK2uYs3QrxYdOBJS/f7dkvj2hv99VDdx0rLKaJ5YXcvmQUxh0SsC7xlNdU8tbn+xl4umZJHiiXSxh4yzAGFeM7JvK5UN6MDonnRtG9w13cYyP0Tnp/GPtbqpraomJ/qIRY2FeEb1SExjSq3OL7z19VF/++P525iwt4OXZkRlgjlVWc/Ofc/n401L6pCU2mb+mVvnHuj1sLznKb649K2RB5mhFNTc9t4qVOw7w148/48VbRjM4wO/m4+0HKD1a2eI/FoLFAoxxRVSU8MQ3h4e7GMaPMf3SeXHF52zYXc4wZ6Z+2dFK/lO4n5vHZ7dq0muCJ5rbz+/H/1uwkY+3l54cGh0pjlV6f2mv2nGAh687i6uG9QrouseWFfDwkq3UqvK764a6HmSOVFRz03MrWf1ZGT+efDp//nAH05/5mBduHsVZAfSPLdywhyRPNBNODc/osTqu9sGIyCQR2SIihSJyn5/zcSIyzzm/QkSy6p3vIyJHROR/nPenisg6n9chEbnbOfdzEdntc+4yN5/NmLZqVPZX1yV722keu3xw6+dLTB/Vh24pcTyyJLL6Yo5WVDPLCS6PXD804OAC8N2JA/ifSwbyj3V7uOfVdQ2OwguGwyeqmPnsStZ8fpDHpg3jlvE5vDJ7NJ0SYrlh7grWNbEsT5XTPHbRoEziY8PXPAYuBhgRiQaeBC4FBgHTRGRQvWw3A2Wq2h94BHiw3vmHgX/VvVHVLao6VFWHAiOAY8DfffI/Unfe2a7ZGFNPRkocA7ol85HPfJgFeUX0SUvkzJ6Bt/M3JD42mjsm9GPFpwf4cNv+Vt8vGI5UVDPruZXk7jjAnKnDmDK0Z7PvcdeFA7h30qm8uW4P//XqeleCTF1wWbfzII9PG3ZygmSv1ETm3TaG1EQPNz6zotFVEz7aVkrZsaqwN4+BuzWYc4BCVd2uqpXAK8CUenmmAM87x68DE8Wpn4vI14FPgfwG7j8R2KaqnwW95Ma0c6Nz0sndcYCqmloOHK3kw22lLR495s/Uc/rQvVM8jyzZimrge9C44UhFNbOcGsGjU4dx5Vktr6XdMaE/9116Gv9cv4fvzVtHVRCDzKETVcx4diV5u8p5YtowLqsXIHp2SeCV2aNJS/Zw49yVDU5qXZhXREpcDOcNDG/zGLgbYHoCO33e73LS/OZR1WqgHEgXkWTgf4FfNHL/qcDL9dLuEpE8EXlWRPwuAysis0UkV0RyS0patvuiMW3dmH7pHKusIW9XOYvz91JTq0HdjCo+Npo7LujHqh1l/KeweTtpBlNdjWDtzoM8NnUYV7QiuNS5/fx+/PCy01iYV8T3XlkblCBTfryKG+euZMOucp745nAubaD2cYoTZLome5j57EpWf/blOUdVNbW8lb+XiyOgeQwidx7Mz/E2dx3xd1JEPMCVwGs+yU8B/YChQBHwO3/XqurTqjpSVUdmZIQ/whsTDqOc+TAfby9lYV4R2V2TGNSj9c1jvq4/uzc9OsfzyNLw1GLqagTrdx7kiWnDmr06QWNmn9ePH08+nUUb9vKdl1oXZMqPVzFj7go27inn99OHM+nM7o3m79E5gVdmjyEjJY4Zc1eyymdi638K91N+vCqoz9oabgaY3UBvn/e9nDS/eUQkBugMlAKjgIdEZAdwN/BDEbnL57pLgTWqWlyXoKrFqlqjqrXAn/A20Rlj/EhPjuPUzBQWbSjiw237mTw4eM1jdeJiornjgv6s/qyMDwpC2xcTaI2gNW4Zn8NPLh/EW/l7ufPFNVRWNz/IlB+r4oZnVrCx6BBPTR/BJWc0HlzqdO8czyuzR5PZKZ6Zz65k5afeILMwr4iU+BjGDeja7LK4wc0AswoYICLZTo1jKjC/Xp75wEzn+BpguXqNV9UsVc0C5gC/UtUnfK6bRr3mMRHx/Rd0FfBJ8B7FmPZnTL908vccolZx7S/e60b24pQQ12KaWyNojZvHZfPzKwbx9sZi7mhmkDl4rJLpcz9my97D/OGGEVw0qHnbGWd28gaZ7p3jmfXcSv5dsJ/F+Xu5ZFB34mLC3zwGLs6DUdVqp9axGIgGnlXVfBG5H8hV1fnAXOAFESkEDuANQo0SkSTgYuC2eqceEpGhgAI7/Jw3xvgYnZPGnz/cQU5GEqd1T3HlM+Jiornzwv786O+f8N7WkhYvW/Lp/qOUHK5oMl+tKr9atIlNRYf4/fQRXNzMX9otMWtsNlFRwk/fzOeOF1cz+7yml0SqqVUeWLiRguIj/PHGEVzQws3AujlBZtrTHzPj2RXUKkHtS2stCfcIj3AaOXKk5ubmhrsYxoRF2dFKRv3fMu6c0J/vXeTe9gmV1bVc8Nt36ZoSxz/uOLfZTXGvr97F919fT6C/qjzRUTx1w3Amnu5+cPH1wkc7+MmbDQ16/SpPTBRP3zgiKGuFlRyu4Jt/+piyY5V8eN9EPDHudq+LyGpVHdlkPgswFmBMx/VZ6VF6dE5w/RfSKys/576/beC5WWc366/1V3N38r9v5DGuf9eAF0vtnZpIn/Sml4Bxw9biwwHVtAD6pCXSO4ClagJ1vLKGwyeq6NYpPmj3bEigAcaWijGmA+ubnhSSz7l6RC+efLeQR5ZuZcKpGQHVYuat8galcf278qcZIyNi2G1TBmamMDDTnebGpiR4osO6sKU/kTpM2RjTjsRGR/GdCwaQt6v85EZ0jXlpxef87xsbOG9ARpsJLuarLMAYY0LiquE96ZOWyJylBY2OKHtxxWf88O8buODUDP544wgLLm2YBRhjTEjERkfxnQv7s2F3OUs3+a/FvPDRDn7090+48LRu/MGCS5tnAcYYEzJXDetJVnqi3zXKnv/QOwrrotO78dQNwyNmLodpOQswxpiQiYmO4jsXDmBj0SEW559ciIPn/vMpP5ufz8WDMvn99BEWXNoJCzDGmJCaMvQUsrsmMWfpVmprlWc+2M4v/rmRr52RyZPfHO76kGkTOvZNGmNCKiY6iu9NHMDmvYf59oureWDhJi49sztPWHBpd+zbNMaE3BVnnUK/jCQW5xdz2eDuPDZtGLHR9uuovbGJlsaYkIuOEn5z7Vm8v7WEOy/ob8GlnbIAY4wJi+F9Uhnex+++gKadsD8bjDHGuMICjDHGGFdYgDHGGOMKCzDGGGNcYQHGGGOMKyzAGGOMcYUFGGOMMa6wAGOMMcYV0tjGP+2diJQAn9VL7grsD0Nx3NLengfa3zO1t+eB9vdM7e15oHXP1FdVM5rK1KEDjD8ikquqI8NdjmBpb88D7e+Z2tvzQPt7pvb2PBCaZ7ImMmOMMa6wAGOMMcYVFmC+6ulwFyDI2tvzQPt7pvb2PND+nqm9PQ+E4JmsD8YYY4wrrAZjjDHGFRZgHCIySUS2iEihiNwX7vIEg4jsEJENIrJORHLDXZ6WEJFnRWSfiHzik5YmIktEpMD52WY2FWngeX4uIrud72mdiFwWzjI2h4j0FpF3RGSjiOSLyPec9Lb8HTX0TG3yexKReBFZKSLrnef5hZOeLSIrnN9580TEE/TPtiYyEJFoYCtwMbALWAVMU9WNYS1YK4nIDmCkqrbZ8fsich5wBPiLqp7ppD0EHFDVXzt/DKSq6v+Gs5yBauB5fg4cUdXfhrNsLSEiPYAeqrpGRFKA1cDXgVm03e+ooWe6jjb4PYmIAEmqekREYoF/A98D7gH+pqqviMgfgPWq+lQwP9tqMF7nAIWqul1VK4FXgClhLpMBVPV94EC95CnA887x83j/528TGnieNktVi1R1jXN8GNgE9KRtf0cNPVObpF5HnLexzkuBC4HXnXRXviMLMF49gZ0+73fRhv9B+VDgbRFZLSKzw12YIMpU1SLneC+QGc7CBMldIpLnNKG1meYkXyKSBQwDVtBOvqN6zwRt9HsSkWgRWQfsA5YA24CDqlrtZHHld54FmPZtnKoOBy4F7nSaZ9oV9bbxtvV23qeAfsBQoAj4XXiL03wikgy8Adytqod8z7XV78jPM7XZ70lVa1R1KNALb4vNaaH4XAswXruB3j7vezlpbZqq7nZ+7gP+jvcfVntQ7LST17WX7wtzeVpFVYudXwC1wJ9oY9+T067/BvCiqv7NSW7T35G/Z2rr3xOAqh4E3gHGAF1EJMY55crvPAswXquAAc6oCg8wFZgf5jK1iogkOR2UiEgScAnwSeNXtRnzgZnO8UzgzTCWpdXqfhE7rqINfU9OB/JcYJOqPuxzqs1+Rw09U1v9nkQkQ0S6OMcJeAczbcIbaK5xsrnyHdkoMocz5HAOEA08q6q/DHORWkVEcvDWWgBigJfa4jOJyMvABLwrvxYDPwP+AbwK9MG7GvZ1qtomOs4beJ4JeJtdFNgB3ObTfxHRRGQc8AGwAah1kn+It8+irX5HDT3TNNrg9yQiQ/B24kfjrVS8qqr3O78jXgHSgLXADapaEdTPtgBjjDHGDdZEZowxxhUWYIwxxrjCAowxxhhXWIAxxhjjCgswxhhjXGEBxrRrzqq4X6uXdreINLqon4gcaex8EMqV4axku1ZExtc7966IjHSOs50Vib/m5x6/cVbH/U0LyzBBRBb4vH9ARN4SkTinDLk+50aKyLs+16mIXOFzfoGITGhJOUz7ZQHGtHcv450462uqkx5OE4ENqjpMVT/wl0FEegFvAf+tqov9ZJkNDFHV7wfygT6ztv2d+zEwFrjKZy5ENxG5tIFLdgE/CuRzTcdlAca0d68Dk+v2unAWLzwF+EBEkkVkmYisEe++OV9ZQdvPX/lPiMgs53iEiLznLCa6uN5M77r8WSKy3FkgcZmI9BGRocBDwBTx7iuS4KfcPYC3gR+p6ldWlRCR+UAysFpErvf3OU6+P4vIH0RkhfOZXyEi/413vborVPW4z6nf0HAQWQ+Ui8jFDZw3xgKMad+c2eMr8f4CBW/t5VVnAcYTeP9iHw5cAPzOWSakSc5aVY8D16jqCOBZwN9KCY8Dz6vqEOBF4DFVXQf8FJinqkPr/VKv8zzwhKq+7uccqnolcNy5fp6/z/HJ3gs4V1Xv8XOrscDtwKU+S7rX+QioFJEL/JXBed4fN3DOGAswpkPwbSbzbR4T4FcikgcsxbtceaDLyp8KnAkscZZB/zHeX+T1jQFeco5fAMYFeP+lwA0ikhhg/sY+5zVVrWngukK8/x0aqok8QANBxNnbpm5pFWO+wgKM6QjeBCaKyHAgUVVXO+nTgQxghLOUeTEQX+/aar78/0ndeQHynRrEUFUdrKqXBLHMD+FdhPW1xvpOAnS0kXPFwGXAHH81FVVdDiQAoxu43moxpkEWYEy75zT9vIO3Gcu3c78zsE9Vq5xfrn39XP4ZMMgZWdUFb+c8wBYgQ0TGgLfJTETO8HP9h3xRe5qOdxHFQN0NHALmBtB01+LPUdWtwDeAvzr9Q/U9ANzbwLVvA6nAkEA/z3QcFmBMR/EycBZfDjAvAiNFZAMwA9hc/yJV3Yl3VeBPnJ9rnfRKvEudPygi64F1wLl+Pvc7wE1OM9yNePdCD4jTTzQTb4e/3w76YHyO81mrgJuA+SLSr965RUBJI5f/ki/vp2QMYKspG2OMcYnVYIwxxrjCAowxxhhXWIAxxhjjCgswxhhjXGEBxhhjjCsswBhjjHGFBRhjjDGusABjjDHGFf8fZLVRULccn+IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "kf = KFold(n_splits=5,shuffle=False) #将数据集分为互斥的5等份，用作测试\n",
    "k_errors = [] #建立初始的误差列表\n",
    "for k in k_range:\n",
    "    knn = KNN(k=k)\n",
    "    scores = []\n",
    "    for train , test in kf.split(X_std,y):\n",
    "        knn.train(X_std[train],y[train])\n",
    "        ypred = knn.predict(X_std[test])\n",
    "        score = knn.score(ypred,y[test])\n",
    "        scores.append(1-score)\n",
    "    k_errors.append(np.mean(scores))\n",
    "\n",
    "plt.plot(k_range, k_errors)\n",
    "plt.xlabel('Value of K for KNN')\n",
    "plt.ylabel('Error')\n",
    "plt.show()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "观察到，算法在$k=21$的时候表现良好，取K值为21，来预测一个新数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 321,
   "metadata": {},
   "outputs": [],
   "source": [
    "knn = KNN(k=21)\n",
    "knn.train(X_std,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 347,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "每年获得的飞行常客里程数?38300\n",
      "玩视频游戏所耗时间百分比?1.6\n",
      "每周消费的冰淇淋公升数?.13\n",
      "这个人属于:  不喜欢的人\n"
     ]
    }
   ],
   "source": [
    "# 定义类别对应的标签\n",
    "resultList = ['不喜欢的人', '魅力一般的人', '极具魅力的人']\n",
    "#输入数据\n",
    "ffMiles = float(input(\"每年获得的飞行常客里程数?\"))\n",
    "percentTats = float(input(\"玩视频游戏所耗时间百分比?\"))\n",
    "iceCream = float(input(\"每周消费的冰淇淋公升数?\"))\n",
    "inArr = np.array([[ffMiles, percentTats, iceCream]])\n",
    "#用之前的fit的标准化数据来转换数据\n",
    "x_new = ss.transform(inArr)\n",
    "#预测数据\n",
    "ypred = knn.predict(x_new)\n",
    "print(\"这个人属于: \", resultList[int(ypred) - 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 参考资料"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://baike.baidu.com/item/%E9%82%BB%E8%BF%91%E7%AE%97%E6%B3%95/1151153?fromtitle=knn&fromid=3479559&fr=aladdin"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://www.cnblogs.com/midiyu/p/10786765.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://www.cnblogs.com/listenfwind/p/10685192.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://blog.csdn.net/m0_38056893/article/details/102990001"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://people.csail.mit.edu/dsontag/courses/ml12/slides/lecture10.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://cs231n.github.io/classification/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://www.csd.uwo.ca/courses/CS4442b/L3-ML-knn.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture02.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://blog.csdn.net/FrankieHello/article/details/79659111"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://www.cnblogs.com/jyroy/p/9427977.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}