{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# K-Nearest Neighbors\n", "
KNN is a form of instance, or memory based learning wherein we don't learn a function $f(X)$ to estimate $E[Y|X]$. Rather, to make a classification for a given instance $X_i$, we search the training data for the $k$ nearest neighbors, as defined by some distance metric $d(X_i,X_j)$. The estimate of $E[Y|X]$ is then given by:
\n",
"\n",
"
Given a feature set and a distance function, the paramater that controls complexity is the number of neighbors used $k$. Like with all complexity parameters, the optimal value is dependent on the data, and is generally the one that exploits the best bias-variance tradeoff.
\n",
"\n",
"The following is an example grid search to find the optimal $k$. We also create 2 other variants of the data to show the effect of each on accuracy.
\n",
"\n",
"
One major challenge with kNN is that the expected distance between any two points in $p$-dimensional space increases as $p$ increases. The following is a simulation that illustrates this concept.
\n",
"\n",
"We generate $n$ uniformly distributed samples in a $p$-dimensional hyper-cube. We then show the average distance between instances increases as we consider more dimensions.\n",
"\n",
"
Now we plot this histogram of distances, and not just the mean distance by dimension $p$.
" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/briand/anaconda/envs/py35/lib/python3.5/site-packages/matplotlib/cbook/deprecation.py:107: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.\n", " warnings.warn(message, mplDeprecation, stacklevel=1)\n", "/Users/briand/anaconda/envs/py35/lib/python3.5/site-packages/matplotlib/axes/_axes.py:6571: UserWarning: The 'normed' kwarg is deprecated, and has been replaced by the 'density' kwarg.\n", " warnings.warn(\"The 'normed' kwarg is deprecated, and has been \"\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAEWCAYAAABG030jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmYHGW5/vHvPUkgMQnZWCQTIGEN\n4Aho4mFVFnMAwbAIGpU1ogIuEA9q9IAG3MIRhQP8jooHBBRBBREEQkBZBRECAkGSI5CEbCxJIAlL\ngBCe3x9vTexMpnuqZ7qnp2fuz3XNNbXXU9VVT7/1dtVbigjMzKx+NNQ6ADMzK48Tt5lZnXHiNjOr\nM07cZmZ1xonbzKzOOHGbmdWZmiZuSf+QtG8tY6g1SUdIWiDpVUm7VXC5n5Z0WwWWs2UWW69KxFXm\nurvd8SFppKSQ1LuT1rePpP/rjHXlJWmapOM7YT0hadus+6eSzqr2OrN1nSLphey8GVaVlUREVf6A\necCHWww7AfhLmcsZCQTQu1qx1vIPeAY4rMT4AF4DXgUWAT8GetU67g5uc/Nn+mr29wJwEzCuA8uq\ni+OjkvECU4DVwCvZ3z+Bi4HNa72dXeEv28/btjJ8X2BhldbZB1gF7FJk/G7AysK4gPcDy4GRedfT\n46tKOqvkU8JWwD/amGaXiBgAHAB8CvhsR1bYBba52eBsu3YBbgeul3RCbUOqO7+JiIHAUOAI4N3A\nw5I2r21YPdZmQF+KnNMR8XfSl+vPlfQBLgO+FRHzcq+lit9282ijxF04DfABYAbp2+gF4MfZ8Pms\nWzrbg1TFcybwLPAicCUwqGC5x2XjlgFntVjPFOBa4FfZuk7K1v1X0rfec9mO3aDFN/epwFOkks13\ngG2A+7Nl/LZw+hbb3GqswIbZ9jSXqJ/JU2oAfgdcnHVPJpXYXwGeBI4osa8D+EK2DXOBs4GLCkoJ\nrwE/zPr7AW+QksFICkqI2XLnZOucC3y6YB0TgVnAy8B0YKsi27TOMguGn5F99g0dPD62Ae7IPv+l\nwFWkL4nC4+4M4HFgBfAboG/B+MOAR7N1PQMclA0fBFyaHSOLgO+SXf0A2wJ3Z8tbSkqopbb9c8Di\nbFlnZOPeDbwODCuY/n3AEqBPK8uaAvyqxbBewGPAedFK6TLb9q9m2/5atj2bAdOyz/RPwJCC6Xcn\nHefLs+XuWzDuLtK5cF82723Axtm4vqRzbFk270PAZgXznVTq/Gixr47PPuelwH8WrD/Pebtt1n15\n9nn1J5WI3+Ffx8zwMvf7hsAF2ee3OOveENg+26fNx+MdRY6BDYHZwOeBb2f7r6Gs/NqFEvdfgWOz\n7gHA7sVOclKCeBrYOpv298Avs3E7ZTttb2AD4DzS5WRh4l4NHJ4dNP1Ilyq7A72z9c0CTm9xANwA\nbATsDLwJ/Dlb/yBS0jy+yH4oGmtribmV+QsPvp2A54HPZP1HZwddA/CJ7KDZvMi+DlKpdmi2zfsD\nM7Nxe5IS1N+y/v2Bx1ruf9JBvxLYIRu3ObBzQbJ7Gtgxm/ZM4P42klfLxL11NnzHDh4f2wLjSCfI\nJsA9wAUtjrsHs303NPu8Ty5IBiuy+RuARmB0Nu564GfZftg0W8bns3FXA/+ZzdMX2LuNbb86W04T\nKUE0b+ctwCkF059P9gXbyrKm0CJxZ8PPKfgs92X9xP0AKVk3kpLlI6RL+L6kL7xvZ9M2khLvR7Lt\nGpf1b5KNv4t03GxPOqbuAqZm4z4P/BF4F+nL5P3ARgXzNSfuUudy8776ebb8XUjnXvPxkee8XSdx\nt7ZP2rHfz8n24aak4+t+4Dulju1WlrEX6QtnJdnxVVZ+LXeG3AtOB8irWXDNf69TPHHfQyoFbtzW\nSU5KmqcW9O9ASsa9gW8BVxeMexfwFusm7nvaiP104PoWB8BeBf0PA18v6P8RBYmhxbKKxtry4Coy\nf2Qf7sukk+S7FPl2JpUSD8u6T2D9xL1/QX9zqXoYqeT+TWAh6eQ5G7iw5f4nJZrlwMeAfi3WPY3s\nCyXrb8g+761aibPVg5uUONbu6/YeH62s73Dg7y2Ou2MK+v8L+GnW/TPg/FaWsRkpafQrGPZJ4M6s\n+0rgEmBEG8dWc7yjW6z/0qz7E8B9WXcv0hf1B4osawqtJ+6Tgaey7n1ZP3EXXiVdB/ykoP9LwB+y\n7q9TUMjIhk0nK6SQEvCZBeNOBW7NuieSEtp7W4nvLv6VuEudy837akTB+AeBCWWct3kTdzn7/Rng\nIwX9BwLz8h6P2XSDSFcQ95Warthfteu4D4+Iwc1/pA+2mM+QvrlnS3pI0qElph1OurRq9izpg94s\nG7egeUREvE4qJRRaUNgjaXtJN0l6XtJK4PvAxi3meaGge1Ur/QPaEWte74uIIRGxTUScGRHvZHEf\nJ+lRScslLQfe00rchQr3yypS1cOHgA+SLvPvJ5UEPpT1ryMiXiMd4CcDz0m6WdLobPRWwH8XxPIS\nIFKpLa/maV9qZVzu40PSZpKukbQo+zx/xfr75fmC7tf51+e3BenEbGkrUpXScwXb+DNSqQvga6Tt\nfTC7G2Zi0a1MCo/BZ0nHCaQru50kjSKVcFdExINtLKulRlrfh83yHstbAUc3b2+2zXuTrrSaFduP\nvyQl+WskLZb0X1l9bkt5zo9W15HzvM2rnP3eWszDi0xbzI9I59gISRPKDbbL/DgZEU9FxCdJJ8K5\nwLWS+pO+vVpaTDqomm0JvE06AJ8DRjSPkNSPVKpcZ3Ut+n9CqnPaLiI2IpU+1f6tyR1ru0nainQJ\n+UVS3dxg4AlKx91yu+8mVYvsRqqDvJtUevgAqYS7/gIipkfEONLJOzuLAVIi+nzhF3VE9IuI+8vY\nrCNIl+7r3b5W5vHx/Wx4U/Z5HkP+z3MBqY68teFvkkr8zdu3UUTsnMX3fER8NiKGk6oJ/qf5VrQi\ntijo3pJ0nBARb5B+MzkGOJaUAHOT1AB8FLi3nPmKWEAqcRd+pv0jYmpbM0bE6og4OyJ2IlXFHUr6\n7amljpwf7T1v1ztmytzvrcW8OMd6AZD0YWA86Tg5hVTgGZp3fuhCiVvSMZI2yUqTy7PB75Dq/94h\n1YE1uxqYJGmUpAGkE/U3EfE26YfHj0raU9IGpMvJtj7MgaTqiFezEuQpldquNmLtiOaktQRA0omk\nEnc57iadTE9GxFtkl7DA3IhY0nLirCR7WJYw3yRVhb2Tjf4p8A1JO2fTDpJ0dJ4gsuV+kfRDzTea\nryhaTFPO8TEwi22FpEbSj3F5XQqcKOkASQ2SGiWNjojnSD++/UjSRtm4bSR9KIvvaEnNBYaXSZ/N\nettR4CxJ78r214mkH0ibXUmq6hpPzsQtqbekHUnH27tJt4121K9I59KBknpJ6itp34LtLBXPfpKa\nsvv/V5KqP1rbHx05P9p73r4ADJM0qMXwvPv9auBMSZtI2phUPfurPCvOzp1LgEkRsTQibiH99nR+\nztiBLpS4gYOAf0h6FfhvUj3Wqqyq43vAfdnl2u6k22d+SSoVziXV1X4JICL+kXVfQyp9v0oqxb1Z\nYt1nkG6ze4VUgvxNiWnLVTTWjoiIJ0mXW38lHYhNpF+ny3E/qa67uXT9ZBZfq6Vt0vHyFVLp4iVS\nlcopWTzXk0rC12SXrU8AB7ex/uWSXgNmkn4AOzoiLisybTnHx9mkuwJWADeTfvDKJbs8PpF0Iq0g\nfbk1l66OI/3g/SQpOV/Lv6oNxgJ/y+K7ETgtIuaUWNXdpB/l/ky6A2Ttw1IRcR8pyT0SEc8Wmb/Z\nJ7J1rsjWuwx4f0TkLgEWExELSD86f5P0BbmA9CWYJ2+8m7R/VpJ+NLyb1pNhR86Pdp23ETGblHzn\nZMfM8Gx43v3+XVI14+OkY/eRbFge3wdmR8RVBcNOBw6WNC7nMlBWUd5tZd/iy0mXU3NrHY9ZHpLu\nAH4dEf9b61h6knrZ712pxF0xkj6aXYb2J90OOJP0a7pZlydpLOmKoZJXftaGetrv3TJxky7vmm+O\n3450Wd29Ly2sW5B0BekhmNMj4pVax9NT1Nt+7/ZVJWZm3U13LXGbmXVbVWlsaOONN46RI0dWY9Fm\nZt3Www8/vDQiNmlruqok7pEjRzJjxoxqLNrMrNuS1Nbtn4CrSszM6o4Tt5lZnXHiNjOrM13lTShm\nVsdWr17NwoULeeONN2odSl3o27cvI0aMoE+f1hpMbJsTt5l12MKFCxk4cCAjR45EqlTDmt1TRLBs\n2TIWLlzIqFGj2rUMV5WYWYe98cYbDBs2zEk7B0kMGzasQ1cnTtxmVhFO2vl1dF85cZuZ1RnXcZtZ\nxe019Q4WLV9VseU1Du7HfZP3LzlNr169aGpqYvXq1fTu3ZvjjjuOSZMm0dDQwIwZM7jyyiu58MIL\nKxbTVVddxbnnnktEMHDgQH7yk5+wyy67VGz5pfTcxH1+E6yYn3vyA7fcksW9ylvF8P7DmX7U9DID\nM6t/i5avYt7UQyq2vJGTb25zmn79+vHoo48C8OKLL/KpT32KlStXcvbZZzNmzBjGjBlTsXgARo0a\nxd13382QIUOYNm0an/vc5/jb3/5W0XUU03MT94r5MGVF7skXX9HEzONnlrWKpiuayo3KzCpg0003\n5ZJLLmHs2LFMmTKFu+++m/POO4+bbrqJKVOmMHfuXObMmcP8+fM5//zzeeCBB5g2bRqNjY388Y9/\nzHWb3p577rm2e/fdd2fhwoXV3KR1uI7bzLqlrbfemjVr1vDiiy+uN+6ZZ57hjjvu4MYbb+SYY45h\nv/32Y+bMmfTr14+bb06l+0mTJrHrrruu9zd16vrvSr700ks5+OC23tRXOT23xG1mPdbBBx9Mnz59\naGpqYs2aNRx00EEANDU1MW/ePADOPz/f+3vvvPNOLr30Uv7yl79UK9z1OHGbWbc0Z84cevXqxaab\nbsqsWbPWGbfhhhsC0NDQQJ8+fdbentfQ0MDbb6cXzE+aNIk777xzveVOmDCByZMnA/D4449z0kkn\nMW3aNIYNG1bNzVmHE7eZdTtLlizh5JNP5otf/GK775luq8Q9f/58jjzySH75y1+y/fbbt2sd7eXE\nbWYV1zi4X647QcpZXltWrVrFrrvuuvZ2wGOPPZavfOUrFYuhpXPOOYdly5Zx6qmnAtC7d+9Oew9B\nVd45OWbMmOjyL1KYMqisu0qa2nlXSbnzmNWjWbNmseOOO9Y6jLrS2j6T9HBEtHnfou8qMTOrM07c\nZmZ1xonbzKzOOHGbmdUZJ24zszrjxG1mVmd8H7eZVV6ZrW+2adCWMKn0rbWd3azr7NmzOfHEE3nk\nkUf43ve+xxlnnLF23K233sppp53GmjVrOOmkk9Y+aVkpTtxmVnlltr7ZpimD2pyks5t1HTp0KBde\neCF/+MMf1hm+Zs0avvCFL3D77bczYsQIxo4dy/jx49lpp50qtm5XlZhZt9PcrOvFF19MRHDXXXdx\n6KGHAjBlyhSOP/549tlnH7baait+//vf87WvfY2mpiYOOuggVq9enXsdY8eOXa8J2AcffJBtt92W\nrbfemg022IAJEyZwww03VHT7nLjNrFvqzGZdCy1atIgttthibf+IESNYtGhRRbfNVSVm1uNUslnX\nWnDiNrNuqTOadW1NY2MjCxYsWNu/cOFCGhsbO7w9hZy4zazb6YxmXYsZO3YsTz31FHPnzqWxsZFr\nrrmGX//61+1aVjFO3GZWeYO2zHUnSFnLa0NnN+v6/PPPM2bMGFauXElDQwMXXHABTz75JBtttBEX\nX3wxBx54IGvWrGHixInsvPPOFV23m3XNyc26mhXnZl3L52Zdzcx6ECduM7M648RtZlZnnLjNzOqM\nE7eZWZ1x4jYzqzO+j9vMKu7Aaw9k8WuLK7a84f2HM/2o6SWn6exmXW+44QbOOussGhoa6N27Nxdc\ncAF77703AFdccQXf/e53ATjzzDM5/vjjK7ZecOI2sypY/Nriij7D0HRFU5vTdHazrgcccADjx49H\nEo8//jgf//jHmT17Ni+99BJnn302M2bMQBLvf//7GT9+PEOGDKnYul1VYmbdTmc06zpgwIC1j9O/\n9tpra7unT5/OuHHjGDp0KEOGDGHcuHHceuutFd0+J24z65Y6o1nX66+/ntGjR3PIIYdw2WWXAW7W\n1cysKirVrOsRRxzBEUccwT333MNZZ53Fn/70p2qGvZZL3GbWLRU269pS3mZd875I4YMf/CBz5sxh\n6dKlbtbVzKw9OqNZ16effpptttkGSTzyyCO8+eabDBs2jAMPPJBvfvObvPzyywDcdttt/OAHP2hX\nDMXkStyS9ga2i4hfSNoEGBARcysaiZl1G8P7D891J0g5y2tLZzfret1113HllVfSp08f+vXrx29+\n8xskMXToUM466yzGjh0LwLe+9S2GDh1a0XW32ayrpG8DY4AdImJ7ScOB30XEXsXmcbOu7Z/HrB65\nWdfyVbtZ1yOA8cBrABGxGBjYjjjNzKwC8iTutyIVywNAUv/qhmRmZqXkqeP+raSfAYMlfRaYCPy8\numH1XE/vfwCrF5f3qHCf4cPZ9o4/VykiM+tq2kzcEXGepHHASmAH4FsRcXvVI+uhVi9ezI6zZ7U9\nYYFZo123aNaTtJm4JY0C7m1O1pL6SRoZEfOqHZyZma0vTx3374B3CvrXZMPMzKwG8tRx946It5p7\nIuItSRtUMSYzq3Pt+a2mlDy/43R2s67NHnroIfbYYw+uueYajjrqKKBrNOu6RNL4iLgRQNJhwNKK\nRmFm3Up7fqspJc/vOJ3drCvAmjVr+PrXv86///u/rx3WVZp1PRn4pqT5khYAXwc+X7EIzMwqrDOa\ndQW46KKL+NjHPrZOeyhdolnXiHgmInYHdgJ2jIg9I+LpikZhZlZh1W7WddGiRVx//fWccsop6yy7\nSzTrKmlD4GPASKB3c4MtEXFORSMxM+sklWjW9fTTT+fcc8+loaHzG1nNU8d9A7ACeBh4s7rhmJlV\nRmGzrrNmrVvfnrdZ1zvvvHO95U6YMIHJkyczY8YMJkyYAMDSpUu55ZZb6N27N42Njdx1111rp1+4\ncCH77rtvRbctT+IeEREHVXStZmZV1BnNus6d+68GUk844QQOPfRQDj/8cF566aUu0azr/ZKaIsLN\n3JlZLn2GD6/oE719hne9Zl2L6YxmXfMk7r2BEyTNJVWVCIiIeG9FIzGzbqMWbeesWbOm6Lh99913\nbXXFlClT1hn36quvru1uOS6vyy+/fJ3+iRMnMnHixHYtK488ifvgqq3dzMzKlqeRqWcBJG0K9K16\nRGZmVlKb97FIGi/pKWAucDcwD5hW5bjMrM609TYt+5eO7qs8NyB+B9gd+GdEjAIOAB7o0FrNrFvp\n27cvy5Ytc/LOISJYtmwZffu2vwIjTx336ohYJqlBUkNE3Cnpgnav0cy6nREjRrBw4UKWLFlS61Dq\nQt++fRkxYkS758+TuJdLGgDcA1wl6UWy9092FXtNvYNFy1eVNc8819abVUyfPn0YNWpUrcPoMfIk\n7sOAVcAk4NPAIODsagZVrkXLVzFv6iHlzTSlKqGYmVVdnjrub0XEOxHxdkRcEREXkloINDOzGsiT\nuMe1Msz3dpuZ1UjRqhJJpwCnAttIerxg1EDgvmoHZmZmrStVx/1r0v3aPwAmFwx/JSJeqmpUZmZW\nVNGqkohYkb3J/Uzg+ewJylHAMZIGd1J8ZmbWQp467uuANZK2BS4BtiCVxs3MrAbyJO53IuJt4Ejg\nooj4KrB5dcMyM7Ni8iTu1ZI+CRwH3JQN61O9kMzMrJQ8iftEYA/gexExV9Io4JfVDcvMzIrJ06zr\nk8CXC/rnAudWK6D2PL7eOLhflaIxM+t6St3H/duI+LikmcB6TX5V6w047Xp83cysBylV4j4t+39o\nZwRiZmb5FE3cEfFc1rkC2C7r/mdErKh6VGZmVlSpqpINgZ8Bh5PefiNgK0nXAydHxFudE6KZmRUq\ndVfJmaTb/raIiN0iYldgS1KyP6szgjMzs/WVStxHAJ+NiFeaB2Tdp2bjzMysBkol7nci4vWWAyPi\nVVq5y8TMzDpHqbtKQtIQUt12S+9UKR4zM2tDqcQ9CHiY1hO3S9xmZjVS6nbAkZ0Yh5mZ5ZSnrRIz\nM+tCnLjNzOqME7eZWZ1ps3VAAEm9gM0Kp4+I+dUKyszMimszcUv6EvBt4AX+dRtgAFVpHdDMzErL\nU+I+DdghIpZVOxgzM2tbnjruBaQWAs3MrAso1TrgV7LOOcBdkm4G3mweHxE/rnJsZmbWilJVJQOz\n//Ozvw2yP/CTk2ZmNVPqycmzASQdHRG/Kxwn6ehqB2ZmZq3LU8f9jZzDzMysE5Sq4z4Y+AjQKOnC\nglEbAW9XOzAzM2tdqTruxcAMYDyplcBmrwCTqhlUd3Hx/7zNrB/sWNY8fYYPr1I0ZtZdlKrjfgx4\nTNJVEeESdjtsugJ2nD2r1mGYWTdTqqrktxHxceDvkta7iyQi/OSkmVkNlKoqOS37f2hnBGJmZvmU\nqip5Luv8MHBPRDzVOSGZmVkpedoq2RL4maSRpB8p7wHujYhHqxiXmZkV0eZ93BHx7YjYH9gZuBf4\nKuveZWJmZp0oT7OuZwJ7AQOAvwNnkBK4mZnVQJ6qkiNJD9zcDNwN/DUi3iw9i5mZVUueqpL3kX6g\nfBAYB8yU9JdqB2ZmZq3LU1XyHmAf4EPAGFL73K4qMTOrkTxVJVNJd5JcCDwUEaurG5KZmZXSZuKO\nCD+AY2bWheRp1tXMzLoQJ24zszpTVuKW1CBpo2oFY2ZmbWszcUv6taSNJPUHngCelPTV6odmZmat\nyVPi3ikiVgKHA9OAUcCxVY3KzMyKypO4+0jqQ0rcN2a3A/ot72ZmNZIncf8UmAf0B+6RtBWwsppB\nmZlZcSXv45bUALwQEY0Fw+YD+1U7MDMza13JEndEvAN8rcWw8DsozcxqJ09VyZ8knSFpC0lDm/+q\nHpmZmbUqT1sln8j+f6FgWABbVz4cMzNrS562SkZ1RiBmZpZP0cQtaf+IuEPSka2Nj4jfVy8sMzMr\nplSJ+0PAHcBHWxkXgBO3mVkNFE3cEfHt7P+JnReOmZm1Jc+Pk0g6hPSW977NwyLinGoFZWZmxeVp\nZOqnpDtLvgQIOBrYqspxmZlZEXnu494zIo4DXo6Is4E9gO2rG5aZmRWTJ3Gvyv6/Lmk4sBrYvHoh\nmZlZKXnquG+SNBj4IfAI6Y6Sn1c1KjMzKyrPAzjfyTqvk3QT0DciVlQ3LDMzK6ZoVYmk7STdIOkJ\nSVdLaoyIN520zcxqq1Qd92XATcDHSFUkF3VKRGZmVlKpqpKBEdFcl/1DSY90RkBmZlZaqcTdV9Ju\npHu3AfoV9keEE7mZWQ2UStzPAT8u6H++oD+A/asVlJmZFVeqrRK/nszMrAvK8wCOmZl1IU7cZmZ1\nptR93Htl/zfsvHDMzKwtpUrcF2b//9oZgZiZWT6l7ipZLekSoFHShS1HRsSXqxeWmZkVUypxHwp8\nGDgQeLhzwjEzs7aUuh1wKXCNpFkR8VgnxmRmZiXkuatkmaTrJb2Y/V0naUTVIzMzs1blSdy/AG4E\nhmd/f8yGmZlZDeRJ3JtGxC8i4u3s73JgkyrHZWZmReRJ3EslHSOpV/Z3DLCs2oGZmVnr8iTuicDH\nSY1MPQccBZxYzaDMzKy4PK8uexYY3wmxmJlZDm6rxMyszjhxm5nVmTYTt6RReYaZmVnnyFPivq6V\nYddWOhAzM8un6I+TkkYDOwODJB1ZMGojoG+1AzMzs9aVuqtkB1JDU4OBjxYMfwX4bDWDMjOz4ko1\nMnUDcIOkPSLCbXKbmXUReeq4T5Y0uLlH0hBJl1UxJjMzK6HNB3CA90bE8uaeiHhZ0m5VjKlbabqi\nqazph/cfzvSjplcpGjPrDvIk7gZJQyLiZQBJQ3POZ8DM42eWNX25id7Mep48CfhHwAOSfpv1Hw18\nr3ohmZlZKXnaKrlS0gxg/2zQkRHxZHXDMjOzYkrdx90XOBnYFpgJ/DQi3u6swMzMrHWl7iq5AhhD\nStoHA+d1SkRmZlZSqaqSnSKiCUDSpcCDnRNSFzZlUJkzDK9KGGbWs5VK3KubOyLibUmdEE4XN2VF\nedNfs2N14jCzHq1U4t5F0sqsW0C/rF9ARMRGVY/OzMzWU+qR916dGYiZmeXjFymYmdUZJ24zszrj\nxG1mVmecuM3M6owTt5lZnekerfyd3wQr5pc1y8LYmBFVCsfMrJq6R+JeMb/sh2P2nnwz86oTjZlZ\nVbmqxMyszjhxm5nVGSduM7M648RtZlZnusePk+3Qf5upNF0xOff0w/u7iVYz6xp6bOJu2GB52S/y\nNTPrClxVYmZWZ5y4zczqjBO3mVmdceI2M6szTtxmZnXGidvMrM44cZuZ1RknbjOzOuPEbWZWZ5y4\nzczqjBO3mVmdceI2M6szTtxmZnXGidvMrM44cZuZ1RknbjOzOuPEbWZWZ5y4zczqjBO3mVmdceI2\nM6szTtxmZnXGidvMrM70rnUA1nF9hg9n1ugdy5p+2zv+XMWIzKyanLi7gXKTcDlJ3sy6nh6duEdO\nvjn3tI2D+3Hf5P2rGI2ZWT49OnHPm3pI7mnLSfJmZtXkHyfNzOqME7eZWZ3p0VUl5bh8+veY9Ycz\nypqnz7verlI0ZtaTOXHntNmql9lx9qzyZpoyqDrBmFmP5qoSM7M6o4io/EKlJcCz7Zx9Y2BpBcOp\nJW9L19RdtqW7bAd4W5ptFRGbtDVRVRJ3R0iaERFjah1HJXhbuqbusi3dZTvA21IuV5WYmdUZJ24z\nszrTFRP3JbUOoIK8LV1Td9mW7rId4G0pS5er4zYzs9K6YonbzMxKcOI2M6szXSpxSzpI0v9JelrS\n5FrH016StpB0p6QnJf1D0mm1jqkjJPWS9HdJN9U6lo6QNFjStZJmS5olaY9ax9RekiZlx9YTkq6W\n1LfWMeUl6TJJL0p6omDYUEm3S3oq+z+kljHmVWRbfpgdY49Lul7S4Eqvt8skbkm9gP8HHAzsBHxS\n0k61jard3gb+IyJ2AnYHvlDH2wJwGlDm8/5d0n8Dt0bEaGAX6nSbJDUCXwbGRMR7gF7AhNpGVZbL\ngYNaDJsM/DkitgP+nPXXg8ssRlg5AAAGBUlEQVRZf1tuB94TEe8F/gl8o9Ir7TKJG/gA8HREzImI\nt4BrgMNqHFO7RMRzEfFI1v0KKUE01jaq9pE0AjgE+N9ax9IRkgYBHwQuBYiItyJieW2j6pDeQD9J\nvYF3AYtrHE9uEXEP8FKLwYcBV2TdVwCHd2pQ7dTatkTEbRHR3MLcA8CISq+3KyXuRmBBQf9C6jTZ\nFZI0EtgN+FttI2m3C4CvAe/UOpAOGgUsAX6RVfv8r6T+tQ6qPSJiEXAeMB94DlgREbfVNqoO2ywi\nnsu6nwc2q2UwFTQRmFbphXalxN3tSBoAXAecHhErax1PuSQdCrwYEQ/XOpYK6A28D/hJROwGvEb9\nXI6vI6v/PYz0ZTQc6C/pmNpGVTmR7lGu+/uUJf0nqdr0qkovuysl7kXAFgX9I7JhdUlSH1LSvioi\nfl/reNppL2C8pHmkqqv9Jf2qtiG120JgYUQ0X/lcS0rk9ejDwNyIWBIRq4HfA3vWOKaOekHS5gDZ\n/xdrHE+HSDoBOBT4dFThYZmulLgfAraTNErSBqQfW26scUztIkmkutRZEfHjWsfTXhHxjYgYEREj\nSZ/HHRFRlyW7iHgeWCBph2zQAcCTNQypI+YDu0t6V3asHUCd/tBa4Ebg+Kz7eOCGGsbSIZIOIlUv\njo+I16uxji6TuLPK/C8C00kH4W8j4h+1jard9gKOJZVQH83+PlLroIwvAVdJehzYFfh+jeNpl+yq\n4VrgEWAm6Tyum0fGJV0N/BXYQdJCSZ8BpgLjJD1FuqKYWssY8yqyLRcDA4Hbs3P/pxVfrx95NzOr\nL12mxG1mZvk4cZuZ1RknbjOzOuPEbWZWZ5y4zczqjBO3dZikNdltT09I+p2kd7Ux/S3tbTEte1S9\nQw12STpB0pLs0fenJE2XtGfB+HMkfbjE/IfXeaNhVud8O6B1mKRXI2JA1n0V8HC5Dx5lD5IoIqre\nJkr2VNuYiPhi1r8fcDWwX0S0+SCLpMuBmyLi2mrGaVaMS9xWafcC2wJI+oOkh7N2oz/XPIGkeZI2\nljQya3/9SuAJ4FhJP86mOU3SnKx7a0n3Zd13SRqTtRF+eVbKnylpUjZ+G0m3Zuu9V9LotgKOiDtJ\nD7B8LlvG5ZKOyrqnKrWr/rik87KS+Xjgh9lVxjaSPivpIUmPSbqu+YojW86Fku6XNKd5mdm4r2dx\nPyZpantjt56pd60DsO4ja2L0YODWbNDEiHhJUj/gIUnXRcSyFrNtBxwfEQ9Iejfp6VmAfYBlWdvT\n+wD3tJhvV6Axa4+agqqXS4CTI+IpSf8G/A+wf47wHwE+32J7hgFHAKMjIiQNjojlkm6koMQtaXlE\n/Dzr/i7wGeCibDGbA3sDo0mPdV8r6WBSI1H/FhGvSxrawdith3HitkroJ+nRrPtesjavgS9LOiLr\n3oKUpFsm7mcj4gFI7YlIGiBpYDb9r0ltaO9Dakip0Bxga0kXATcDtym1xrgn8LtU8wLAhjm3Qa0M\nWwG8AVyq9PafYm8Aek+WsAcDA0jNNjT7Q1b986Sk5qZKPwz8orkdi+zLrSOxWw/jxG2VsCoidi0c\nIGlfUoLaIytV3gW09nqt11r03w+cCPwf6UtgIrAH8B+FE0XEy5J2AQ4ETgY+DpwOLG8ZS0670aKh\npoh4W9IHSI04HUW6GmitBHw5cHhEPJbVn+9bMO7Ngu7WvhyaNdD+2K2HcR23Vcsg4OUsaY8mvcIt\nj3uBM0hVI38H9gPejIgVhRNJ2hhoiIjrgDOB92Vtns+VdHQ2jbLkXpKkD5Hqt3/eYvgAYFBE3AJM\nIr3uDOAVUiNCzQYCzyk15fvpHNt4O3BiQV340PbGbj2TS9xWLbcCJ0uaRSo9P5BzvntJ1ST3RMQa\nSQuA2a1M10h6m01z4aP5vX6fBn4i6UygD6kd8cdamf8TkvYmvfZrLvCxVu4oGQjcoPQiXgFfyYZf\nA/xc0pdJJfGzSG84WpL9H0gJEXGrpF2BGZLeAm4BvllG7NbD+XZAM7M646oSM7M648RtZlZnnLjN\nzOqME7eZWZ1x4jYzqzNO3GZmdcaJ28yszvx/OfLjv81bGzwAAAAASUVORK5CYII=\n", "text/plain": [ "