{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"![](\"data/cover.jpg\")
\n",
"*This notebook contains an excerpt from the book [Machine Learning for OpenCV](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv) by Michael Beyeler.\n",
"The code is released under the [MIT license](https://opensource.org/licenses/MIT),\n",
"and is available on [GitHub](https://github.com/mbeyeler/opencv-machine-learning).*\n",
"\n",
"*Note that this excerpt contains only the raw code - the book is rich with additional explanations and illustrations.\n",
"If you find this content useful, please consider supporting the work by\n",
"[buying the book](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv)!*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"< [Measuring-Model-Performance-with-Scoring-Functions](03.01-Measuring-Model-Performance-with-Scoring-Functions.ipynb) | [Contents](../README.md) | [Using Regression Models to Predict Continuous Outcomes](03.03-Using-Regression-Models-to-Predict-Continuous-Outcomes.ipynb) >"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Understanding the k-NN Classifier\n",
"\n",
"The $k$-NN algorithm is arguably one of the simplest machine learning algorithms. The\n",
"reason for this is that we basically only need to store the training dataset. Then, in order to\n",
"make a prediction for a new data point, we only need to find the closest data point in the\n",
"training dataset-its **nearest neighbor**.\n",
"\n",
"In a nutshell, the $k$-NN algorithm argues that a data point probably belongs to the same\n",
"class as its neighbors.\n",
"Of course, some neighborhoods might be a little more complicated. In this case, we would\n",
"not just consider our closest neighbor (where $k=1$), but instead our $k$ nearest neighbors.\n",
"\n",
"That's all there is to it."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Implementing k-NN in OpenCV\n",
"\n",
"Using OpenCV, we can easily create a $k$-NN model via the function `cv2.ml.KNearest_create`. Building the model then involves the following steps:\n",
"- Generate some training data.\n",
"- Create a k-NN object for a given number k.\n",
"- Find the k nearest neighbors of a new data point that we want to classify.\n",
"- Assign the class label of the new data point by majority vote.\n",
"- Plot the result.\n",
"\n",
"We first import all the necessary modules: OpenCV for the $k$-NN algorithm, NumPy for\n",
"data munging, and Matplotlib for plotting. If you are working in a Jupyter Notebook, don't\n",
"forget to call the `%matplotlib inline` magic:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"import cv2\n",
"\n",
"import matplotlib.pyplot as plt\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"plt.style.use('ggplot')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Generating the training data\n",
"\n",
"The first step is to generate some training data. For this, we will use NumPy's random\n",
"number generator. As discussed in the previous section, we will fix the seed of the random\n",
"number generator, so that re-running the script will always generate the same values:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"np.random.seed(42)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can pick a single data point with `0 <= x <= 100` and `0 <= y <= 100`:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([51, 92])"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"single_data_point = np.random.randint(0, 100, 2)\n",
"single_data_point"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As shown in the preceding output, this will pick two random integers between 0 and 100.\n",
"We will interpret the first integer as the data point's $x$ coordinate on the map, and the\n",
"second integer as the point's $y$ coordinate. Similarly, let's pick a label for the data point:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"single_label = np.random.randint(0, 2)\n",
"single_label"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Turns out that this data point would have class 0.\n",
"\n",
"Let's wrap this process in a function that takes as input the number of data points to\n",
"generate (that is, `num_samples`) and the number of features every data point has (that is,\n",
"`num_features`):"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def generate_data(num_samples, num_features=2):\n",
" \"\"\"Randomly generates a number of data points\"\"\"\n",
" data_size = (num_samples, num_features)\n",
" train_data = np.random.randint(0, 100, size=data_size)\n",
" labels_size = (num_samples, 1)\n",
" labels = np.random.randint(0, 2, size=labels_size)\n",
" \n",
" return train_data.astype(np.float32), labels"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's put the function to test and generate an arbitrary number of data points, let's say\n",
"eleven, whose coordinates are chosen randomly:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 71., 60.],\n",
" [ 20., 82.],\n",
" [ 86., 74.],\n",
" [ 74., 87.],\n",
" [ 99., 23.],\n",
" [ 2., 21.],\n",
" [ 52., 1.],\n",
" [ 87., 29.],\n",
" [ 37., 1.],\n",
" [ 63., 59.],\n",
" [ 20., 32.]], dtype=float32)"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_data, labels = generate_data(11)\n",
"train_data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As we can see from the preceding output, the `train_data` variable is an 11 x 2 array, where\n",
"each row corresponds to a single data point. We can also inspect the first data point with its\n",
"corresponding label by indexing into the array:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(array([ 71., 60.], dtype=float32), array([1]))"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_data[0], labels[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This tells us that the first data point is a blue square (because it has class 0) and lives at\n",
"location $(x, y) = (71, 60)$ on the town map. If we want, we can plot this data point on\n",
"the town map using Matplotlib:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAENCAYAAADgwHn9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X1UVHXiBvBnmAEnMF5mBmFnyYjA1N01Wt/I15XF3WNW\n23Y6ZW2tgKYypGVa1GqL5wjukHbYfIuyQNPWVneV7NV25CgmyAq+UbKmLZgxFi9DYMCAM9zfH/38\nLiDUlZh7GXs+f83cuXe+DxyPD/f1q5EkSQIREREAH7UDEBHRwMFSICIigaVAREQCS4GIiASWAhER\nCSwFIiISWApERCSwFIiISGApEBGRwFIgIiJBp3aAvrDb7WpH6JHJZEJdXZ3aMWTxlqzekhNgVk9h\n1v5hNptlrcc9BSIiElgKREQksBSIiEhgKRARkcBSICIiQbGrj5qbm5GTk4Pz589Do9EgJSUFR48e\nRWlpKTQaDYKCgmCxWGAwGJSKRERE3ShWCnl5eYiNjcWSJUvgcrnQ1taGiIgIzJo1CwDw3nvv4R//\n+AfmzZunVCQiIupGkcNHLS0tqKioQHx8PABAp9MhICAA/v7+Yp22tjZoNBol4hARUS80SszRXFVV\nhZdffhkRERE4d+4coqKikJiYCL1ej+3bt6OwsBD+/v5IT09HYGDgFdvbbDbYbDYAgNVqRXt7u6cj\n94lOp4PL5VI7hizektVbcgLM6inM2j/8/PxkradIKXz22WdYtmwZVq5ciZiYGOTl5eG6664Th44A\nYPfu3bh06RLuv//+7/0+3tH8w3lLVm/JCTCrpzBr/xhQdzQbjUYYjUbExMQAAOLi4lBZWdllnUmT\nJqGkpESJOERE1AtFSiE4OBhGo1H8hV9eXo6IiAhcuHBBrFNaWiq7yYiIyDMUu/ooOTkZa9euhcvl\nwpAhQ2CxWJCTk4MLFy5Ao9HAZDLxyiMiIpUpVgqRkZGwWq1dli1dulSp4YmISAbe0UxERAJLgYiI\nBJYCEREJLAUiIhJYCkREJLAUiIhIYCkQEZHAUiAiIoGlQEREAkuBiIgElgIREQksBSIiElgKREQk\nsBSIiEhgKRARkcBSICIigaVAREQCS4GIiASWAhERCSwFIiISWApERCSwFIiISGApEBGRwFIgIiJB\np9RAzc3NyMnJwfnz56HRaJCSkoKSkhKUlZVBp9MhLCwMFosFAQEBSkUiIqJuFCuFvLw8xMbGYsmS\nJXC5XGhra8OoUaPw0EMPQavVYtu2bdi9ezcefvhhpSIREVE3ihw+amlpQUVFBeLj4wEAOp0OAQEB\nuPXWW6HVagEAw4YNg8PhUCIOERH1QpE9hZqaGgQGBmLjxo04d+4coqKikJiYCL1eL9YpKCjAhAkT\netzeZrPBZrMBAKxWK0wmkxKxr5pOpxuw2brzlqzekhNgVk9hVmUpUgputxuVlZVITk5GTEwM8vLy\nkJ+fj1mzZgEAdu3aBa1Wi8mTJ/e4fUJCAhISEsT7uro6JWJfNZPJNGCzdectWb0lJ8CsnsKs/cNs\nNstaT5HDR0ajEUajETExMQCAuLg4VFZWAgD279+PsrIyLFq0CBqNRok4RETUC0VKITg4GEajEXa7\nHQBQXl6OiIgIHD9+HG+99RbS0tIwaNAgJaIQEdF3UOzqo+TkZKxduxYulwtDhgyBxWLBs88+C5fL\nhZUrVwIAYmJiMG/ePKUiERFRN4qVQmRkJKxWa5dl69atU2p4IiKSgXc0ExGRwFIgIiKBpUBERAJL\ngYiIBJYCEREJLAUiIhJYCkREJLAUiIhIYCkQEZHAUiAiIoGlQEREAkuBiIgElgIREQksBSIiElgK\nREQksBSIiEhgKRARkcBSICIigaVAREQCS4GIiASWAhERCSwFIiISWApERCTolBqoubkZOTk5OH/+\nPDQaDVJSUlBfX4+dO3eiuroaq1atws0336xUHCIi6oHsUvjiiy9w+PBhfP3115g7dy6qq6vhcrlw\n4403yto+Ly8PsbGxWLJkCVwuF9ra2uDv74+lS5filVde6fMPQERE/UfW4aPi4mKsWLECDocDBw8e\nBAA4nU68/vrrsgZpaWlBRUUF4uPjAQA6nQ4BAQGIiIiA2WzuY3QiIupvsvYUduzYgeXLlyMyMhLF\nxcUAgBtvvBFVVVWyBqmpqUFgYCA2btyIc+fOISoqComJidDr9X0OTkRE/U9WKTQ2Nl5xmEij0UCj\n0cgaxO12o7KyEsnJyYiJiUFeXh7y8/Mxa9YsWdvbbDbYbDYAgNVqhclkkrWd0nQ63YDN1p23ZPWW\nnACzegqzKktWKURFRaGwsBBTp04Vyw4dOoTo6GhZgxiNRhiNRsTExAAA4uLikJ+fLztkQkICEhIS\nxPu6ujrZ2yrJZDIN2GzdeUtWb8kJMKunMGv/kHuoXlYpJCUlISMjAwUFBWhra0NmZibsdjuWL18u\na5Dg4GAYjUbY7XaYzWaUl5cjIiJC1rZERKQcjSRJkpwV29raUFZWhrq6OhiNRowePfqqzglUVVUh\nJycHLpcLQ4YMgcViwalTp5Cbm4umpiYEBAQgMjISy5Yt+97vstvtssdV0kD+K6E7b8nqLTkBZvUU\nZu0f/bqnkJubi+TkZEyYMKHL8s2bNyMxMVHWQJGRkbBarV2WjRs3DuPGjZO1PREReZ6sS1IPHDjQ\n4/LCwsJ+DUNEROr6zj2FgoICAN9ePXT59WU1NTW4/vrrPZeMiIgU952lcPlGNZfLJV5fFhQUhNTU\nVM8lIyIixX1nKaSnpwMA3nzzTdn3FBARkfeSdaK5cyFIkoTOFyz5+PBBq0RE1wpZpeBwOPDaa6+h\noqICzc3NXT77+9//7pFgRESkPFl/5r/yyivQ6XT485//DL1ej6ysLIwZMwaPPvqop/MREZGCZJXC\np59+ipSUFERGRkKj0SAyMhIpKSl45513PJ2PiIgUJKsUfHx8oNVqAQABAQFoamrCoEGD4HA4PBqO\niIiUJeucQnR0NI4dO4Zx48bh1ltvRXZ2Nvz8/DhTGhHRNUZWKSxcuFBccZSYmIg9e/bA6XRi5syZ\nHg1HRETKklUKAQEB4rWfnx/uu+8+jwUiIiL1yCoFl8uF/fv3o6qqCk6ns8tnjz32mEeCERGR8mSV\nwvr163Hu3DmMHj0aQUFBns5EREQqkVUKJ06cwPr167scRiIiomuPrEtSTSYTLl265OksRESkMll7\nClOmTMHq1asxY8YMBAcHd/ns5z//uUeCERGR8mSVwgcffAAA2L59e5flGo0G69ev7/9URESkClml\nsGHDBk/nICKiAYDPvSYiIqHXPYXFixcjOzsbAJCSktLrF7z00kv9n4qIiFTRaynMnz9fvF64cKEi\nYYiISF29lsLw4cPF65EjRyoShoiI1NVrKcidUe2BBx7otzBERKSuXkuhvr5evG5vb0dJSQmio6Nh\nMplQV1eHs2fPYvz48bIHam5uRk5ODs6fPw+NRoOUlBSYzWZkZ2ejtrYWoaGhWLx4MQYPHvzDfiIi\nIuqzXkvBYrGI13/961/x+OOPIy4uTiwrKSlBcXGx7IHy8vIQGxuLJUuWwOVyoa2tDbt378YvfvEL\n3HPPPcjPz0d+fj4efvjhPv4oRMqLjQ1Dba220xIzACA01I3jx79SJxTRDyDrktTLE+x0NnbsWBw7\ndkzWIC0tLaioqEB8fDwAQKfTISAgAEeOHMHUqVMBAFOnTsWRI0euJjuR6roWwvcvJxroZJVCeHi4\nuKv5sr179yI8PFzWIDU1NQgMDMTGjRvx9NNPIycnB06nE42NjQgJCQEAhISEoKmp6SrjExFRf5J1\nR/OCBQuwZs0a7NmzBwaDAQ6HA1qtFkuWLJE1iNvtRmVlJZKTkxETE4O8vDzk5+fLDmmz2WCz2QAA\nVqsVJpNJ9rZK0ul0AzZbd96S1Vty9mQg5/am3yuzKktWKdx444148cUXcebMGTQ0NCA4OBjDhg2D\nTidrcxiNRhiNRsTExAAA4uLikJ+fj6CgIDQ0NCAkJAQNDQ0IDAzscfuEhAQkJCSI93V1dbLGVdrl\nk/DewFuyDvyc5l4/Gci5B/7v9X+YtX+Yzb3/W+3sew8fdXR04JFHHoEkSRgxYgQmTJiAkSNHyi4E\nAAgODobRaITdbgcAlJeXIyIiAmPGjMGBAwcAAAcOHMDYsWNlfycREfW/7/2f3cfHB2azGRcvXoTB\nYOjzQMnJyVi7di1cLheGDBkCi8UCSZKQnZ2NgoICmEwmPPnkk33+fiI1hIa6ezypHBrqViEN0Q+n\nkSRJ+r6V3nrrLRQVFWHGjBkwGo3QaDTiMzXmU7i8xzHQDORdx+68Jau35ASY1VOYtX/IPXwk6xjQ\nhx9+CADYuXNnl+WcT4GI6NrC+RSIiEiQfbbY7Xbj9OnTcDgcMBqNGDZsGLRa3qBDRHQtkVUK1dXV\nyMrKQnt7O4xGI+rr6+Hr64u0tDRERER4OiMRESlEVim8+uqrSEhIwF133SVOMu/ZswevvfYa0tPT\nPRqQiIiUI+sxF1VVVbjzzju7XHU0c+ZMVFVVeSoXERGpQFYpGAwGnDp1qsuyiooK8dwiIiK6Nsg6\nfPTggw8iKysLo0ePFtfhHj16lNN0EhFdY2SVwpgxY5CVlYXi4mI0NDTghhtuwP333y/7ZggiIvIO\nsi9JNZvN+P3vf4/GxkYEBQXBx0fWkSciIvIiskqhpaUFubm5KCoqgtvthlarxYQJE5CcnAx/f39P\nZyQiIoXI+nM/Ly8PTqcTa9aswbZt27BmzRq0t7cjNzfX0/mIiEhBskrh+PHjWLhwIcxmM3x9fWE2\nm2GxWHDixAlP5yMiIgXJKgU/P78rpspsamq6qjkViIho4JP1v3p8fDwyMjIwc+ZMhIaGora2Fu++\n+26X2dCIiMj7ySqFe++9FyEhITh06BAcDgcMBgN+97vfYdq0aZ7OR0RECpJVChqNBvHx8YiPj/d0\nHiIiUpGscwq5ubk4ffp0l2WnT5/G5s2bPZGJiIhUIqsUDh06hJtvvrnLsqioKHz00UceCUVEROqQ\nVQoajQYdHR1dlnV0dEDG9M5ERORFZJXC8OHD8eabb4pi6OjowM6dOzF8+HCPhiMiImXJOtGclJQE\nq9WK+fPni6ekhoSEIC0tzdP5iIhIQbJKwWg0IisrC2fPnkV9fT2MRiOio6P5UDwiomuM7FuSfXx8\nMGzYME9mISIilSn2nIrU1FTo9Xr4+PhAq9XCarWiqqoKmzZtgtPpRGhoKBYtWsSnrhIRqUjRhxel\np6cjMDBQvH/55ZfxyCOPYOTIkSgoKMCePXswa9YsJSMREVEnqp4UsNvtGDFiBABg1KhRKCkpUTMO\nEdGPnqxS2LJlC6qqqn7wYJmZmUhLS4PNZgMA3HDDDSgtLQUAHD58GPX19T94DCIi6juNJOMOtNzc\nXBQXFyMwMBCTJ0/G5MmTYTQar2qgyw/Sa2xsREZGBpKSkhAUFIS8vDxcvHgRY8aMwfvvv9/jxD02\nm00UidVqRXt7+1WNrRSdTgeXy6V2DFm8Jau35ASY1VOYtX/4+fnJWk9WKQDf3rB27NgxHDx4EEeP\nHkVMTAymTJmC8ePHQ6/XX1W4HTt2QK/X4+677xbL7HY71q1bh7/85S/fu73dbr+q8ZRy+R4Ob+At\nWb0lJ8CsnsKs/cNsNstaT/Y5BR8fH4wePRpPPPEEMjMz0dTUhI0bN+LRRx9FTk4OHA5Hr9s6nU60\ntraK1ydPnsTQoUPR2NgI4NvC2bVrF6ZPny43DhEReYDsq49aWlpw+PBhHDx4EOfOncP48eMxZ84c\nmEwmvPPOO1i1ahXWrFnT47aNjY3iM7fbjUmTJiE2Nhbvvfce9u7dCwAYN24c52cgIlKZrFJ44YUX\ncOLECYwYMQLTp0/H2LFj4evrKz7/4x//iMTExF63DwsLw+rVq69Yfscdd+COO+64+tREROQRskoh\nJiYGc+bMQXBwcI+f+/j4YNOmTf0ajIiIlCerFDqfEO7NoEGDfnAYIiJSF59oR0REAkuBiIgElgIR\nEQksBSIiElgKREQksBSIiEhgKRARkcBSICIigaVAREQCS4GIiASWAhERCSwFIiISWApERCSwFIiI\nSGApEBGRwFIgIiKBpUBERAJLgYiIBJYCEREJLAUiIhJYCkREJLAUiIhI0Ck1UGpqKvR6PXx8fKDV\namG1WlFVVYVNmzahvb0dWq0Wc+fORXR0tFKRiIioG8VKAQDS09MRGBgo3m/btg333XcfbrvtNhw9\nehTbtm3DihUrlIxERESdqHr4SKPRoLW1FQDQ0tKCkJAQNeMQEf3oKbqnkJmZCQCYPn06EhISMHv2\nbGRmZmLr1q3o6OhARkaGknGIiKgbjSRJkhIDORwOGAwGNDY2IiMjA0lJSTh8+DBGjhyJuLg4FBUV\nYd++fXjuueeu2NZms8FmswEArFYr2tvblYh81XQ6HVwul9oxZPGWrN6SE2BWT2HW/uHn5ydrPcVK\nobMdO3ZAr9fjn//8JzZv3gyNRgNJkpCYmIgtW7Z87/Z2u12BlFfPZDKhrq5O7RiyeEtWb8kJMKun\nMGv/MJvNstZT5JyC0+kU5w6cTidOnjyJoUOHwmAw4NSpUwCAjz/+GOHh4UrEISKiXihyTqGxsRFr\n1qwBALjdbkyaNAmxsbHQ6/XIy8tDR0cHfH19MX/+fCXiEBFRLxQphbCwMKxevfqK5cOHD0dWVpYS\nEYiISAbe0UxERAJLgYiIBJYCEREJLAUiIhJYCkREJLAUiIhIYCkQEZHAUiAiIoGlQEREAkuBiIgE\nlgIREQksBSIiElgKREQksBSIiEhgKRARkcBSICIigaVAREQCS4GIiASWAhERCSwFIiISWApERCSw\nFIiISGApEBGRwFIgIiJBp9RAqamp0Ov18PHxgVarhdVqRXZ2Nux2OwCgpaUF/v7+WL16tVKRiIio\nG8VKAQDS09MRGBgo3i9evFi8fv311+Hv769kHCIi6mZAHD6SJAnFxcWYOHGi2lGIiH7UFN1TyMzM\nBABMnz4dCQkJYnlFRQWCgoLwk5/8RMk4RETUjUaSJEmJgRwOBwwGAxobG5GRkYGkpCSMHDkSALBp\n0yaEh4fjrrvu6nFbm80Gm80GALBarWhvb1ci8lXT6XRwuVxqx5DFW7J6S06AWT2FWfuHn5+frPUU\nK4XOduzYAb1ej7vvvhtutxsLFiyA1WqF0WiUtf3lk9MDjclkQl1dndoxZPGWrN6SE2BWT2HW/mE2\nm2Wtp8g5BafTidbWVvH65MmTGDp0KACgvLwcZrNZdiEQEZHnKHJOobGxEWvWrAEAuN1uTJo0CbGx\nsQCAQ4cO8QQzEdEAoUgphIWF9Xr/QWpqqhIRiIhIhgFxSSoREQ0MLAUiIhJYCkREJLAUiIhIYCkQ\nEZGgys1rREQ0MHFPoR8988wzakeQzVuyektOgFk9hVmVxVIgIiKBpUBERIJ2xYoVK9QOcS2JiopS\nO4Js3pLVW3ICzOopzKocnmgmIiKBh4+IiEhQdOa1a0lzczNycnJw/vx5aDQapKSkwM/PD5s2bUJ7\nezu0Wi3mzp2L6OhotaN+Z1an04nQ0FAsWrRI9Tmy7XY7srOzxfuamhrcf//9mDp1KrKzs1FbW4vQ\n0FAsXrwYgwcPVjFp71kNBgN27tyJ6upqrFq1CjfffLOKKb/VW1aHw4GysjLodDqEhYXBYrEgICBA\nxaS9Z7148SJKS0uh0WgQFBQEi8UCg8Ew4HLOnDkTALBnzx5s27YNr776apd56b2CRH2ybt06yWaz\nSZIkSZcuXZK++eYbaeXKldLRo0clSZKksrIyKT09XcWE/9NT1meeeUb65JNPJEmSpH379knbt29X\nM+IV3G63NHfuXKmmpkbaunWrtHv3bkmSJGn37t3S1q1bVU7XVees58+fl6qrq6X09HTp7Nmzake7\nQuesx48fl1wulyRJkrR169YB/Xttbm4Wy999913p5ZdfVjFZV51zSpIk1dbWShkZGVJKSorU2Nio\ncrqrx8NHfdDS0oKKigrEx8cD+HYKvoCAAGg0GjGZUEtLC0JCQtSMKXL0lNVut2PEiBEAgFGjRqGk\npETNmFcoLy9HeHg4QkNDceTIEUydOhUAMHXqVBw5ckTldF11zhoRESF7his1dM566623QqvVAgCG\nDRsGh8OhcrquOmftvBfb1tYGjUajYrKuOucEgC1btuAPf/jDgMp4NXj4qA9qamoQGBiIjRs34ty5\nc4iKikJiYiJmz56NzMxMbN26FR0dHcjIyFA7aq9Zb7jhBpSWlmLs2LE4fPgw6uvr1Y7aRefJlxob\nG0XBhoSEoKmpSc1oV/CmiaJ6y1pQUIAJEyaokKh33bNu374dhYWF8Pf3R3p6uorJuuqcs7S0FAaD\nAZGRkeqG+gG4p9AHbrcblZWV+M1vfoPnn38egwYNQn5+Pj788EPMnj0bL730EmbPno2cnBy1o/aa\nNSUlBXv37kVaWhpaW1uh0w2cvw9cLhfKysoQFxendpTvdS1k3bVrF7RaLSZPnqxSsiv1lPXBBx/E\nSy+9hEmTJuGDDz5QMd3/dM7Z1taGXbt24YEHHlA71g/CUugDo9EIo9GImJgYAEBcXBwqKytx4MAB\njB8/HgBw++234+zZs2rGBNB71p/+9KdYvnw5srKyMHHiRISFhamc9H+OHTuGm266CcHBwQCAoKAg\nNDQ0AAAaGhoG1Im77lkHsp6y7t+/H2VlZVi0aNGAOtzxXb/XSZMmDZjDnZ1zfvXVV6ipqcFTTz2F\n1NRU1NfXIy0tDV9//bXaMa8KS6EPgoODYTQaYbfbAXx7TDEiIgIGgwGnTp0CAHz88ccIDw9XMyaA\n3rM2NjYCADo6OrBr1y5Mnz5dzZhddD9sMGbMGBw4cAAAcODAAYwdO1ataFfw5kNHx48fx1tvvYW0\ntDQMGjRIxWRX6p71woUL4nVpaemAOW/TOefQoUPx6quvYsOGDdiwYQOMRiOysrK84g+GzgbOMQMv\nk5ycjLVr18LlcmHIkCGwWCwYO3Ys8vLy0NHRAV9fX8yfP1/tmAB6zlpYWIi9e/cCAMaNG4dp06ap\nnPJbbW1tOHnyJObNmyeW3XPPPcjOzkZBQQFMJhOefPJJFRP+T09Z//3vfyM3NxdNTU2wWq2IjIzE\nsmXLVEz5rZ6yvvbaa3C5XFi5ciUAICYmpsvnaukp6xtvvIELFy5Ao9HAZDIN2JzXAt7RTEREAg8f\nERGRwFIgIiKBpUBERAJLgYiIBJYCEREJLAUiD9qwYQPefPNNAEBFRQUef/xxlRMRfTeWApFCRowY\ngRdffLFfvmvFihXYt29fv3wXUWcsBaJ+4Ha71Y5A1C94RzN5pS+//BLPPvssnnvuOURFRcHhcOCp\np57Ck08+iZ/97GdXrF9XV4fNmzejoqICkiRh4sSJmDNnDjo6OrB7927s27cP7e3tiI2NRXJysnhU\nc2lpKf72t7/B4XAgMjISc+fORUREBAAgNTUV06dPx0cffQS73Y6tW7fi888/R05ODi5cuIDbbrut\ny/OEPvnkE6xbt048KDE1NRW//e1vUVhYiNraWsTGxiI1NRV+fn745ptvsH79epw5cwYdHR245ZZb\n8Oijj8JoNGL79u2oqKjAmTNnsHnzZvzqV7/CnDlzUF1djdzcXPz3v/9FYGAgHnjggQH35FPyAirP\n50DUZ//617+kJ554QnI6nVJGRoa0ZcuWHtdzu93S0qVLpby8PKm1tVVqa2uTKioqJEn6doKhxx57\nTPryyy+l1tZWafXq1dLatWslSZKk6upq6eGHH5ZOnDghXbp0ScrPz5cee+wx6dKlS5IkSZLFYpGW\nLl0q1dbWSm1tbdKlS5eklJQU6e2335YuXbokFRcXS7NmzRITGH388cfS/PnzRS6LxSI988wzUn19\nvXTx4kXpiSeekPbu3StJkiQ1NTVJxcXFktPplFpaWqQXXnhBysrKEtump6eLiZMkSZJaW1ulBQsW\nSAUFBZLL5ZI+++wzKTk5Wfr888/78TdOPwY8fEReKyEhAeHh4fjTn/6EhoYGPPjggz2ud/bsWTgc\nDjzyyCPQ6/Xw8/PD8OHDAQAfffQR7rzzToSFhUGv1+Ohhx5CUVER3G43ioqKcNttt2HUqFHQ6XS4\n66670N7ejtOnT4vvnjFjBkwmE/z8/PDpp5/C7XZj5syZ0Ol0iIuL+97pOGfMmAGDwYDBgwdj9OjR\nqKqqAgBcf/31iIuLw6BBg3Ddddfh3nvvRUVFRa/fc/ToUYSGhmLatGnQarWIiorC+PHjcfjw4av8\nrdKPHQ8fkVf79a9/jeeffx7z5s2Dr69vj+vU1dUhNDRUzDLWWUNDg5gxCwBMJhPcbjcaGxuv+MzH\nxwcmk6nLDGUmk6nLdxkMhi6HjDp/3pPOT9D08/MT393W1oYtW7bg+PHjaG5uBgC0traio6MDPj5X\n/i1XW1uLM2fOIDExUSxzu92YMmXKd45P1B1LgbyW0+nEli1bEB8fj507dyIuLg6DBw++Yj2TyYS6\nujq43e4riiEkJAS1tbXifV1dHbRaLYKCghASEoLPP/9cfCZJEurq6nqdMD4kJAQOhwOSJIliqK+v\n79Mj1N9++23Y7XasWrUKwcHBqKqqwtNPPw3p/59f2X3uA6PRiJEjR+K555676rGIOuPhI/JaeXl5\nuOmmm7BgwQL88pe/xCuvvNLjetHR0QgJCcEbb7wBp9OJ9vZ2/Oc//wEATJw4Ee+++y5qamrgdDqx\nfft23H777dBqtZgwYQKOHTuG8vJyuFwuvP322/D19cUtt9zS4zjDhg2Dj48P3n//fbjdbpSUlPR5\noiWn0wk/Pz/4+/vjm2++wc6dO7t8HhQUhK+++kq8Hz16NC5cuIDCwkK4XC64XC6cPXsWX3zxRZ/G\npx8vlgJ5pSNHjuD48ePiWfazZ89GZWUlDh48eMW6Pj4+SEtLw5dffgmLxYKUlBQUFRUBAKZNm4Yp\nU6YgPT0dqamp8PX1RXJyMgDAbDZj4cKFyM3NxZw5c1BWVoa0tLRepy7V6XRYunQp9u/fj6SkJBQV\nFWHcuHHrdhuzAAAAe0lEQVR9+vnuuOMOtLe3Y86cOVi2bBliY2Ov+LykpARJSUnIzc3Fddddh+XL\nl+PQoUOYP38+5s2bhzfeeAMul6tP49OPF+dTICIigXsKREQksBSIiEhgKRARkcBSICIigaVAREQC\nS4GIiASWAhERCSwFIiISWApERCT8H9ALRZMZBvP8AAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(train_data[0, 0], train_data[0, 1], 'sb')\n",
"plt.xlabel('x coordinate')\n",
"plt.ylabel('y coordinate')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"But what if we want to visualize the whole training set at once? Let's write a function for\n",
"that. The function should take as input a list of all the data points that are blue squares\n",
"(`all_blue`) and a list of the data points that are red triangles (`all_red`):"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def plot_data(all_blue, all_red):\n",
" plt.figure(figsize=(10, 6))\n",
" plt.scatter(all_blue[:, 0], all_blue[:, 1], c='b', marker='s', s=180)\n",
" plt.scatter(all_red[:, 0], all_red[:, 1], c='r', marker='^', s=180)\n",
" plt.xlabel('x coordinate (feature 1)')\n",
" plt.ylabel('y coordinate (feature 2)')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's try it on our dataset! First we have to split all the data points into red and blue sets. We\n",
"can quickly select all the elements of the `labels` array created earlier that are equal to 0,\n",
"using the following command (where `ravel` flattens the array):"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([False, False, False, True, False, True, True, True, True,\n",
" True, False], dtype=bool)"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"labels.ravel() == 0"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"All the blue data points are then all the rows of the `train_data` array created earlier,\n",
"whose corresponding label is 0:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"blue = train_data[labels.ravel() == 0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The same can be done for all the red data points:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"red = train_data[labels.ravel() == 1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, let's plot all the data points:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmQAAAF6CAYAAAC3JUTKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8U3W+//F3mrSUgnRJWCwgUAqyuLBTYQYE6swVXBgf\nLhcFqeBGuYDKoAwj4jysIwhYRIrVYSkuA8o8xl7Xq4M8QGWzQst4AbFlUTbtklIotJQ2+f3hr7lU\n2nJaSE5IXs+/mpyc5N1+H9E3n3NyYnG73W4BAADANCFmBwAAAAh2FDIAAACTUcgAAABMRiEDAAAw\nGYUMAADAZBQyAAAAk1HIAAAATEYhAwAAMBmFDAAAwGQUMgAAAJNRyAAAAExmMztAYxw9erRR+zkc\nDhUWFl7iNLgUWBv/xvr4L9bGf7E2/s1X6xMbG2vocUzIAAAATEYhAwAAMBmFDAAAwGQUMgAAAJNR\nyAAAAExGIQMAADDZZXnZCwAAqs2YEan9+xv+v7O4uErNn1/ihURAw1HIAACXtf37bdq6tYnZMYCL\nwiFLAAAAk1HIAAAATEYhAwAAMBmFDAAAwGQUMgAAAJNRyAAAAExGIQMAADAZhQwAAMBkFDIAAACT\nUcgAAABMRiEDAAAwGYUMAADAZHy5OADgshYXV+nT/QBvoJABAC5r8+eXmB0BuGgcsgQAADAZhQx+\nwfLll2ZHAADANBQymM6amyvb3XfLmpdndhQAAExBIYPpIlNSZHE6FZmSYnYUAABMQSGDqay5uQrN\nyZEkhebkMCUDAAQlChlMFZmSImthoSTJWlDAlAwAEJQoZDDNudOxakzJAADBiEIG05w7HavGlAwA\nEIwoZDBFbdOxakzJAADBhkIGU9Q2HavGlAwAEGwoZPC5+qZj1ZiSAQCCic++y/LDDz/U+vXrZbFY\n1L59eyUnJ+v48eNatGiRSktL1alTJ02ZMkU2G1+vGejqm45Vq56SOTMyfBMKAAAT+WRC5nQ69ckn\nn2ju3LlauHChXC6XNm/erLfeekujRo3S4sWL1axZM61fv94XcWAiI9OxakzJAADBwmeHLF0ulyoq\nKlRVVaWKigpFRUVp165dSkhIkCTdeOONysrK8lUcmMTIdKwa55IBAIKFT44PxsTE6NZbb9WkSZMU\nFham66+/XnFxcYqIiJDVavU8xul0+iIOTGLNzVVodnaD9gndsUPWvDxVxcd7KRUAAObzSSErLS1V\nVlaW0tLSFBERoZdeekk5Bg9bSdK6deu0bt06SdLcuXPlcDgalcNmszV6X1w8y4ED0j33qKq2bSEh\ncrtcte4XY7XKzbqZiveO/2Jt/Bdr49/8bX18Usi+/fZbtWrVSi1atJAkDRw4UHv37tXp06dVVVUl\nq9Uqp9OpmJiYWvdPTExUYmKi53ahwUNev+ZwOBq9Ly6BTp2kP/+51k0XXBvWzVS8d/wXa+O/WBv/\n5qv1iY2NNfQ4n5xD5nA4lJubqzNnzsjtduvbb79Vu3bt1LNnT23dulWStGHDBvXr188XcQAAAPyK\nTyZkXbp0UUJCgp566ilZrVZ17NhRiYmJ6tOnjxYtWqQ1a9aoU6dOGj58uC/iAAAA+BWL2+12mx2i\noY4ePdqo/Rgf+y/Wxr+xPv6LtfFfrI1/C8pDlgAAAKgbhQwAAMBkFDIAAACTUcgAAABMRiEDAAAw\nGYUMAADAZBQyAAAAk1HIAAAATEYhAwAgwIT+/68lxOWDQgYAQACx5uYqZuJEWfPyzI6CBqCQAQAQ\nQCJTUmQ9flyRKSlmR0EDUMgAAAgQ1txchebkSJJCc3KYkl1GKGQAAASIyJQUWf//F2ZbCwqYkl1G\nKGQAAASAc6dj1ZiSXT4oZAAABIBzp2PVmJJdPihkAABc5mqbjlVjSnZ5oJABAHCZq206Vo0p2eWB\nQgYAwGWsvulYNaZk/o9CBgDAZay+6Vg1pmT+j0IGAMBlysh0rBpTMv9GIQMA4DJlZDpWjSmZf6OQ\nAQBwGbLm5io0O7tB+4Tu2MGUzE/ZzA4AAAAazlJaqrLbb2/4fidPeiENLhaFDACAy1Bl79460bu3\n2TFwiXDIEgAAwGQUMgAAAJNRyAAAAExGIQMAADAZhQwAAMBkFDIAAACTUcgAAABMRiEDAAAwGYUM\nAADAZBQyAAAAk1HIAAAATEYhAwAAMBmFDAAAwGQUMgAAAJNRyAAAAExGIQMAADAZhQwAAMBkFDIA\nAACTUcgAAABMRiEDAAAwGYUMAADAZBQyAAAAk1HIAAAATEYhAwAAMBmFDAAAwGS2Cz1g//792rFj\nh3744QedPn1aERER6tChg3r37q3OnTv7IiMAAEBAq7OQ7dy5U6tXr1ZZWZl69Oihq6++Wk2bNlVZ\nWZmOHDmixYsXKzw8XGPGjFGvXr18mRkAACCg1FnI/vWvf+nBBx9UfHx8nTvn5eXpv//7vylkAAAA\nF6HOQvbHP/7xgjvHx8dr+vTplzQQAABAsOGkfgAAAJPVW8jKy8u1bNkyzZgxQy+//LJ++umnGtuZ\njgEAAFy8egvZG2+8oYKCAt11112KjIzUrFmztGvXLs/2goICrwcEAAAIdPVe9mL79u1KTU1VRESE\nBgwYoD59+ig1NVXJycnq06ePrzICAAAEtHonZBUVFbLZ/q+zXXfddXryySf16quvasuWLbJYLF4P\nCAAIbqFbt5odAfC6egtZ+/bttWfPnhr3de3aVbNmzdKKFSt05swZr4YDAAQ3a26uYiZOlDUvz+wo\ngFfVW8hGjx6t0tLS8+7v1KmTZs+erSFDhngtGAAAkSkpsh4/rsiUFLOjAF5V7zlk9Z0ndtVVVyk5\nOdnwC506dUrp6ek6dOiQLBaLJk2apNjYWKWmpqqgoEAtW7bU448/rubNmxtPDwC4oBkzIrV//wW/\nKe88cXGVmj+/xAuJjLHm5io0J0eSFJqTI2tenqrquVg5cDlr+Du0kVauXKlevXpp+vTpqqys1Jkz\nZ/Tee+/p2muv1ejRo5WZmanMzEyNHTvWV5EAICjs32/T1q1NzI7RYJEpKbIWFkqSrAUFikxJkTMj\nw9xQgJf45MKwp0+f1p49ezR8+HBJks1mU7NmzZSVlaWhQ4dKkoYOHaqsrCxfxAEA+Llzp2PVqqdk\nQCDySSHLz89XixYttHTpUj355JNKT09XeXm5SkpKFB0dLUmKjo7WiRMnfBEHAODnzp2OVauekgGB\nyCeHLKuqqnTgwAFNmDBBXbp00cqVK5WZmWl4/3Xr1mndunWSpLlz58rhcDQqh81ma/S+8C7Wxr+x\nPv7LyNqEhjbuP/WhoaHmrPt33yn03/+udVOTnTvlKCqSrr7ax6EajveNf/O39TH0Lj179qz+8Y9/\naNOmTTp58qRWrVqlnTt36tixY/qP//iPC+5vt9tlt9vVpUsXSVJCQoIyMzMVGRmp4uJiRUdHq7i4\nWC1atKh1/8TERCUmJnpuF/7qX01GORyORu8L72Jt/Bvr47+MrM3Zs3ZJDT+H7OzZsyosLGpkssaL\nmT5dlvz8WrdZ8vPlmj79sjiXjPeNf/PV+sTGxhp6nKFDlqtWrdKhQ4c0depUz8Vg27dvr88++8zQ\ni0RFRclut+vo0aOSpG+//Vbt2rVTv379tHHjRknSxo0b1b9/f0PPBwAITLWdO/ZrnEuGQGRoQvb1\n119r8eLFCg8P9xSymJgYOZ1Owy80YcIELV68WJWVlWrVqpWSk5PldruVmpqq9evXy+Fw6Iknnmjc\nbwEACAi1nTv2a3ziEoHIUCGz2WxyuVw17jtx4oSuuOIKwy/UsWNHzZ0797z7n3nmGcPPAQAIXEam\nY9W4LhkCjaFDlgkJCVqyZIny//8x/eLiYi1fvlyDBg3yajgAQPAwMh2rxicuEWgMFbJ7771XrVq1\n0vTp03X69GlNnTpV0dHRuuuuu7ydDwAQBKy5uQrNzm7QPqE7dnAuGQLGBQ9Zulwufffdd7rvvvuU\nlJTkOVRZfS4ZAAAXy1JaqrLbb2/4fidPeiEN4HsXLGQhISF68cUX9cYbb0hSnZemAACgsSp799aJ\n3r3NjgGYxtAhy+7du+v777/3dhYAAICgZOhTli1bttQLL7ygfv36yW631zhcec8993gtHADg4sXF\nVfp0PwANZ6iQVVRUeC7a2pBrjwEAzDd/fonZEQBcgKFClpyc7O0cAAAAQctQIfv555/r3Na6detL\nFgYAACAYGSpkU6dOrXPbO++8c8nCAAAABCNDhezXpev48eNau3atunfv7pVQAAAAwcTQZS9+LSoq\nSklJSfr73/9+qfMAAAAEnUYVMkk6evSozpw5cymzAAAABCVDhyyfeeaZGtceO3PmjA4dOqQ777zT\na8EAAACChaFCNnz48Bq3w8PD1aFDB1155ZVeCQUAABBMDBWyJk2a6IYbbjjv/q1btyohIeGShwIA\nAAgmhs4hS09Pr/X+11577ZKGAQAACEb1TsiqLwjrcrmUn58vt9tdY1tYWJh30wEAAASBegvZuReE\nnTJlSo1tUVFRuuuuu7yTCgAAIIjUW8iqLwg7Z84c/eUvf/FJIAAAgGBj6BwyyhgAAID3GPqUZVVV\nlT799FPt3r1bJ0+erLGNsgYAAHBxDE3IVq1apXXr1qlHjx7av3+/Bg4cqJKSEvXs2dPb+QAAAAKe\noUK2bds2zZo1SyNHjpTVatXIkSM1Y8YM7dq1y9v5AAAAAp6hQlZRUSG73S5JCgsL05kzZ9S2bVsd\nPHjQm9kAAACCgqFzyNq2bat9+/YpPj5ecXFxWrt2rZo2baqYmBhv5wMAAAh4hiZkSUlJCgn55aHj\nx4/XgQMHtH37dj388MNeDQcAABAMDE3I4uPjPT9feeWVmj17ttcCAQAABBtDhUyS/v3vf2vTpk0q\nKSnRzJkztW/fPpWVlemaa67xZj4AAICAZ+iQ5SeffKK//e1vuvLKK7Vnzx5Jv5zcv2bNGq+GAwAA\nCAaGCtnHH3+s2bNna/To0Z5zydq2baujR496NRwAAEAwMFTIysrK5HA4atxXWVkpm83wEU8AAADU\nwVAh6969uzIzM2vc98knn3ClfgAAgEvAUCGbMGGCvv76a02ePFnl5eWaNm2atm7dqvHjx3s7HwAA\nQMCr85jjwYMH1bFjR0lSdHS0XnjhBe3bt08FBQWy2+2Kj4/3nE8GAACAxquzUc2ZM8fz89SpU2Wx\nWBQfH68bbrhBXbt2pYwBAABcInVOyCIiIrR9+3a1a9dOxcXFys/Pl9vtPu9xrVu39mpAAACAQFdn\nIXvggQeUkZGhwsJCuVwuTZkypdbHvfPOO14LBwAAEAzqLGQDBgzQgAEDJEn333+/3njjDZ+FAgAA\nCCaGTgRbsWKFt3MAAAAErToL2apVq3T8+HFJqvMCsMePH9eqVau8kwwAACBI1HnIMjY2Vn/605/U\nrl07de/eXbGxsWratKnKysp07Ngx7d69W0ePHtUdd9zhy7wAAAABp85CdtNNN2nYsGH65ptvlJ2d\nraysLJ0+fVrNmjXTVVddpZtuukl9+/aV1Wr1ZV4AAICAU++XUdpsNiUkJCghIcFXeQAAAIIOV3cF\nAAAwGYUMAADAZBQyAAAAk1HIAAAATFbvSf3nOnz4sLZu3arjx4/rwQcf1JEjR1RZWakOHTp4Mx8A\nAEDAMzQh27Jli5599lk5nU59+eWXkqTy8nK+TgkAAOASMDQhe/fdd/X000+rY8eO2rJliySpQ4cO\nOnjwoDezAQAABAVDE7KSkpLzDk1aLBZZLBavhAIAAAgmhgpZXFycvvjiixr3bdq0SfHx8V4JBQAA\nEEwMHbJ84IEHlJKSovXr1+vMmTN6/vnndfToUT399NPezgcAABDwDBWytm3batGiRdq+fbv69u0r\nu92uvn37Kjw83Nv5AAAAAp6hQ5YrVqxQkyZNNGjQIN12220aPHiwwsPDlZGR4eV4AAAAgc9QIdu4\ncWOt9//6vDIAAAA0XL2HLNevXy9Jqqqq8vxcLT8/X1dccYX3kgEAAASJegtZ9UVgKysrPT9Xi4yM\n1OTJk72XDAAAIEjUW8jmzJkjSVqzZo3+8z//0yeBAAAAgo2hT1meW8bcbrfcbrfndkiI8e8nd7lc\nmjlzpmJiYjRz5kzl5+dr0aJFKi0tVadOnTRlyhTZbIa/XhMAACAgGGo/TqdTy5cv1549e3Tq1Kka\n29555x3DL/bxxx+rbdu2KisrkyS99dZbGjVqlAYPHqzXX39d69ev1+9+97sGxAcAALj8GRpvvf76\n67LZbHrmmWcUHh6uefPmqV+/fnrooYcMv1BRUZF27NihESNGSPpl0rZr1y4lJCRIkm688UZlZWU1\n4lcAAAC4vBkqZN9//70mTZqkjh07ymKxqGPHjpo0aZI+/PBDwy+UkZGhsWPHer7/8uTJk4qIiJDV\napUkxcTEyOl0NuJXAAAAuLwZOmQZEhLiKU7NmjXTiRMn1LRpU8MFavv27YqMjFRcXJx27drV4JDr\n1q3TunXrJElz586Vw+Fo8HNIks1ma/S+8C7Wxr+xPv6LtfFfrI1/87f1MVTI4uPjlZ2drQEDBuj6\n669XamqqwsLC1LlzZ0MvsnfvXn3zzTfKzs5WRUWFysrKlJGRodOnT6uqqkpWq1VOp1MxMTG17p+Y\nmKjExETP7cLCQkOv+2sOh6PR+8K7WBv/xvr4L9bGf7E2/s1X6xMbG2vocYYK2ZQpUzyfrExKStL7\n77+v8vJyjRo1ytCL3Hvvvbr33nslSbt27dIHH3ygqVOn6qWXXtLWrVs1ePBgbdiwQf369TP0fAAA\nAIHEUCFr1qyZ5+ewsDDdeeedl+TF77vvPi1atEhr1qxRp06dNHz48EvyvAAAAJcTQ4WssrJSGzZs\n0MGDB1VeXl5j23/913816AV79uypnj17SpJat26tF154oUH7AwAABBpDhWzJkiX64Ycf1LdvX0VG\nRno7EwAAQFAxVMh27typJUuW1Dh0CQAAgEvD0HXIHA6Hzp496+0sAAAAQcnQhGzIkCGaP3++br75\nZkVFRdXYds0113glGAAAQLAwVMj+53/+R5K0evXqGvdbLBYtWbLk0qcCAAAIIoYKWVpamrdzAAAA\nBC1D55ABAADAe+qckD3++ONKTU2VJE2aNKnOJ3j11VcvfSoAAIAgUmche+SRRzw/T5kyxSdhAAAA\nglGdhaxbt26en3v06OGTMAAAAMGozkL2zjvvGHqCe+6555KFAQAACEZ1FrKioiLPzxUVFdq2bZvi\n4+PlcDhUWFiovLw8DRw40CchAQAAAlmdhSw5Odnz86JFizRt2jQlJCR47tu2bZu2bNni3XQAAABB\nwNBlL7KzszVgwIAa9/Xv31/Z2dleCQUAABBMDBWyNm3aeK7WX+3TTz9VmzZtvBIKAAAgmBi6Uv+j\njz6qBQsW6P3331dMTIycTqesVqumT5/u7XwIEpYvv5S6dzc7BgAApjBUyDp06KCXX35Zubm5Ki4u\nVlRUlLp27SqbzdDuQL2submy3X23rO+9p6r4eLPjAADgcxc8ZOlyuTRu3Di53W51795dgwYNUo8e\nPShjuGQiU1JkcToVmZJidhQAAExxwUIWEhKi2NhYnTx50hd5EGSsubkKzcmRJIXm5Mial2dyIgAA\nfM/QmOs3v/mN5s2bp5tvvll2u10Wi8Wz7ZprrvFaOAS+yJQUWQsLJUnWggJFpqTImZFhbigAAHzM\nUCH77LPPJElr166tcb/FYtGSJUsufSoEhXOnY9Wqp2ScSwYACCaGCllaWpq3cyAInTsdq8aUDAAQ\njAyfmV9VVaW9e/fK6XTKbrera9euslqt3syGAFbbdKwaUzIAQLAxVMiOHDmiefPmqaKiQna7XUVF\nRQoNDdVTTz2ldu3aeTsjAlBt07FqTMkAAMHGUCFbtmyZEhMTdeutt3pO6H///fe1fPlyzZkzx6sB\nEXjqm45VY0oGAAgmhgrZwYMHNXv27Bqfrhw1apTee+89rwVD4KpvOlaNKRkANM6MGZHav7/h1wqN\ni6vU/PklXkgEIwytWExMjHbv3l3jEhd79uxRdHS014IhMBmZjlVjSgYADbd/v01btzYxOwYayFAh\nGzNmjObNm6e+ffvK4XCosLBQO3bs0JQpU7ydDwHGyHSsGlMyAECwuOCV+iWpX79+mjdvntq3b6/y\n8nK1b99ec+fOVf/+/b2dDwHEmpur0OzsBu0TumMHV+8HAAQ8wweZY2Nj9Yc//EElJSWKjIxUSIih\nLgd4WEpLVXb77bVuaxoerrLy8tr342u7AAABzlAhO336tFasWKHNmzerqqpKVqtVgwYN0oQJExQR\nEeHtjAgQlb1760Tv3rVuC3M4dMLgoUwAAAKNoTHXypUrVV5ergULFuitt97SggULVFFRoRUrVng7\nHwAAQMAzVMhycnI0ZcoUxcbGKjQ0VLGxsUpOTtbOnTu9nQ8AACDgGSpkYWFhOnHiRI37Tpw4IZut\n4dc5AQAAQE2GGtXw4cOVkpKiUaNGqWXLliooKNBHH32kxMREb+cDAAAIeIYK2R133KHo6Ght2rRJ\nTqdTMTExuv322zVs2DBv5wMAAAh4hgqZxWLR8OHDNXz4cG/nAQAACDqGziFbsWKF9u7dW+O+vXv3\nKoMrqAMAAFw0Q4Vs06ZN6ty5c4374uLi9NVXX3klFAAAQDAxfMjS5XLVuM/lcsntdnslFAAAaJy4\nuEqf7odLw1Ah69atm9asWaOxY8cqJCRELpdLa9euVbdu3bydDwAANMD8+SVmR0AjGCpkDzzwgObO\nnatHHnlEDodDhYWFio6O1lNPPeXtfAAAAAHPUCGz2+2aN2+e8vLyVFRUJLvdrvj4eL5gHAAA4BIw\nfKn9kJAQde3a1ZtZAAAAghIjLgAAAJNRyAAAAExGIQMAAAEtdOtWsyNckKFCtmrVKh08eNDLUQAA\nAC4ta26uYiZOlDUvz+wo9TJUyKqqqvT8889r+vTpyszMVFFRkbdzAQAAXLTIlBRZjx9XZEqK2VHq\nZehTlhMmTFBSUpKys7P15Zdf6p///Ke6dOmiIUOGaODAgQoPD/d2TgAAgAax5uYqNCdHkhSakyNr\nXp6q4uNNTlU7w+eQhYSEqG/fvnrsscf0/PPP68SJE1q6dKkeeughpaeny+l0ejMnAABAg0SmpMha\nWChJshYU+PWUzPB1yE6fPq2tW7fqyy+/1A8//KCBAwdq4sSJcjgc+vDDD/XXv/5VCxYs8GZWAAAA\nQ86djlXz5ymZoUK2cOFC7dy5U927d9dNN92k/v37KzQ01LP9/vvvV1JSkrcyAgAANMi507Fq1VMy\nZ0aGOaHqYaiQdenSRRMnTlRUVFSt20NCQvS3v/3tkgbzthkzIrV/v+EBoUdcXCVf3AoAgB+rbTpW\nrXpKJofDx6nqZ6iR3HbbbRd8TJMmTS46jC/t32/T1q2XV2YAAHBhtU3HqnnOJfvwQx+nqh8XhgUA\nAAGjvulYtdCcHGnvXh8lMoZCBgAAAkZ907Fq1oIC2f70Jx8lMoZCBgAAAoKR6Vg1S1aWX129n0IG\nAAACgpHpWDVLfr5fXZes4R8zbITCwkKlpaXp+PHjslgsSkxM1MiRI1VaWqrU1FQVFBSoZcuWevzx\nx9W8eXNfRAIAAAHEmpur0OzsBu0TumOH31yXzCeFzGq1aty4cYqLi1NZWZlmzpyp6667Ths2bNC1\n116r0aNHKzMzU5mZmRo7dqwvIgEAgABiKS1V2e23G3580/BwlZWXy3LypBdTGeeTQhYdHa3o6GhJ\nUtOmTdW2bVs5nU5lZWXp2WeflSQNHTpUzz77LIUMAAA0WGXv3jrRu7fhx4c5HDph8PCmL/j8HLL8\n/HwdOHBA8fHxKikp8RS16OhonThxwtdxAAAATOeTCVm18vJyLVy4UElJSYqIiDC837p167Ru3TpJ\n0ty5c+Vo5NV1bTabZ9/Q0Mb96qGhoY1+fdTt3LWB/2F9/Bdr479YG//mb+vjs0JWWVmphQsX6re/\n/a0GDhwoSYqMjFRxcbGio6NVXFysFi1a1LpvYmKiEhMTPbcLGzlidDgcnn3PnrVLaviV+s+ePavC\nwqJGvT7qdu7awP+wPv6LtfFfrI1/89X6xMbGGnqcTw5Zut1upaenq23btrrllls89/fr108bN26U\nJG3cuFH9+/f3RRwAAAC/4pMJ2d69e/XFF1/oqquu0owZMyRJY8aM0ejRo5Wamqr169fL4XDoiSee\n8EUcAAAAv+KTQtatWze9++67tW575plnfBHhPHFxlT7dDwAAoC4+Panfn8yfX2J2BAAAAEl8dRIA\nAIDpKGQAAAAmo5ABAACYjEIGAABgMgoZAACAyShkAAAAJqOQAQAAmIxCBgAAYDIKGQAAgMkoZAAA\nACajkAEAAJiMQgYAAGAyChkAAIDJKGQAAAAmo5ABAACYjEIGAABgMgoZAACAyShkAAAAJqOQAQAA\nmIxCBgAAYDIKGQAAgMkoZAAAACajkAEAAJiMQgYAAGAyChkAAIDJKGQAAAAmo5ABAACYjEIGAABg\nMgoZAACAyShkAAAAJqOQAQAAmIxCBgAAYDIKGQAAgMkoZAAAACajkAEAAJiMQgYAAGAyChkAAIDJ\nKGQAAAAmo5ABAACYjEIGAABgMgoZAACAyShkAAAAJqOQAQAAmIxCBgAAYDIKGQAAgMkoZAAAACaj\nkAEAAJiMQgYAAGAyChkAAIDJKGQAAAAmo5ABAACYjEIGAABgMgoZAACAyWxmBwBwacyYEan9+xv+\nlo6Lq9T8+SVeSHR54O8GwB9QyIAAsX+/TVu3NjE7xmWHvxsAf8AhSwAAAJNRyAAAAExGIQMAADAZ\nhQwAAMBkpp/Un5OTo5UrV8rlcmnEiBEaPXq02ZEAAAB8ytQJmcvl0vLlyzVr1iylpqZq06ZNOnz4\nsJmRAAAAfM7UQpaXl6c2bdqodevWstlsGjRokLKyssyMBAAA4HOmFjKn0ym73e65bbfb5XQ6TUwE\nAADge6aeQ+Z2u8+7z2KxnHffunXrtG7dOknS3Llz5XA4GvV6Nput0fvCu1ibixca2ri3c2ho6AX/\n9oG8Pt5PeyJvAAAQkklEQVT8u/lCIK/N5Y618W/+tj6mFjK73a6ioiLP7aKiIkVHR5/3uMTERCUm\nJnpuFxYWNur1HA5Ho/eFd7E2F+/sWbukhl9x/uzZsyosLKr3MYG8Pt78u/lCIK/N5Y618W++Wp/Y\n2FhDjzP1kGXnzp117Ngx5efnq7KyUps3b1a/fv3MjAQAAOBzpk7IrFarJkyYoOeff14ul0vDhg1T\n+/btzYwEAADgc6Zfh6xPnz7q06eP2TGAy15cXKVP9wsU/N0A+APTCxmAS2P+/BKzI1yW+LsB8Ad8\ndRIAAIDJKGQAAAAmo5ABAACYjEIGAABgMgoZAACAyShkAAAAJrO4a/tCSQAAAPhMUE3IZs6caXYE\n1IG18W+sj/9ibfwXa+Pf/G19gqqQAQAA+CMKGQAAgMmszz777LNmh/CluLg4syOgDqyNf2N9/Bdr\n479YG//mT+vDSf0AAAAm45AlAACAyWxmB/CFnJwcrVy5Ui6XSyNGjNDo0aPNjhTUCgsLlZaWpuPH\nj8tisSgxMVEjR45UaWmpUlNTVVBQoJYtW+rxxx9X8+bNzY4blFwul2bOnKmYmBjNnDlT+fn5WrRo\nkUpLS9WpUydNmTJFNltQ/OfDr5w6dUrp6ek6dOiQLBaLJk2apNjYWN43fuLDDz/U+vXrZbFY1L59\neyUnJ+v48eO8d0yydOlS7dixQ5GRkVq4cKEk1fn/GbfbrZUrVyo7O1tNmjRRcnKyzw9nBvyEzOVy\nafny5Zo1a5ZSU1O1adMmHT582OxYQc1qtWrcuHFKTU3V888/r08//VSHDx9WZmamrr32Wi1evFjX\nXnutMjMzzY4atD7++GO1bdvWc/utt97SqFGjtHjxYjVr1kzr1683MV3wWrlypXr16qVFixZp/vz5\natu2Le8bP+F0OvXJJ59o7ty5WrhwoVwulzZv3sx7x0Q33nijZs2aVeO+ut4v2dnZ+umnn7R48WI9\n/PDDWrZsmc/zBnwhy8vLU5s2bdS6dWvZbDYNGjRIWVlZZscKatHR0Z5/eTRt2lRt27aV0+lUVlaW\nhg4dKkkaOnQo62SSoqIi7dixQyNGjJAkud1u7dq1SwkJCZJ++Y8ca+N7p0+f1p49ezR8+HBJks1m\nU7NmzXjf+BGXy6WKigpVVVWpoqJCUVFRvHdM1KNHj/OmxXW9X7755hsNGTJEFotFXbt21alTp1Rc\nXOzTvAE/N3U6nbLb7Z7bdrtdubm5JibCufLz83XgwAHFx8erpKRE0dHRkn4pbSdOnDA5XXDKyMjQ\n2LFjVVZWJkk6efKkIiIiZLVaJUkxMTFyOp1mRgxK+fn5atGihZYuXaoffvhBcXFxSkpK4n3jJ2Ji\nYnTrrbdq0qRJCgsL0/XXX6+4uDjeO36mrveL0+mUw+HwPM5ut8vpdHoe6wsBPyGr7UOkFovFhCT4\ntfLyci1cuFBJSUmKiIgwOw4kbd++XZGRkX71UXD8oqqqSgcOHNDvfvc7vfjii2rSpAmHJ/1IaWmp\nsrKylJaWptdee03l5eXKyckxOxYM8oeuEPATMrvdrqKiIs/toqIinzZe1K6yslILFy7Ub3/7Ww0c\nOFCSFBkZqeLiYkVHR6u4uFgtWrQwOWXw2bt3r7755htlZ2eroqJCZWVlysjI0OnTp1VVVSWr1Sqn\n06mYmBizowYdu90uu92uLl26SJISEhKUmZnJ+8ZPfPvtt2rVqpXn7z9w4EDt3buX946fqev9Yrfb\nVVhY6HmcGV0h4CdknTt31rFjx5Sfn6/Kykpt3rxZ/fr1MztWUHO73UpPT1fbtm11yy23eO7v16+f\nNm7cKEnauHGj+vfvb1bEoHXvvfcqPT1daWlpeuyxx3TNNddo6tSp6tmzp7Zu3SpJ2rBhA+8hE0RF\nRclut+vo0aOSfikA7dq1433jJxwOh3Jzc3XmzBm53W7P+vDe8S91vV/69eunL774Qm63W99//70i\nIiJ8XsiC4sKwO3bs0KpVq+RyuTRs2DDdcccdZkcKat99952eeeYZXXXVVZ6R8JgxY9SlSxelpqaq\nsLBQDodDTzzxBB/fN9GuXbv0wQcfaObMmfr555/P++h+aGio2RGDzsGDB5Wenq7Kykq1atVKycnJ\ncrvdvG/8xLvvvqvNmzfLarWqY8eOevTRR+V0OnnvmGTRokXavXu3Tp48qcjISN19993q379/re8X\nt9ut5cuXa+fOnQoLC1NycrI6d+7s07xBUcgAAAD8WcAfsgQAAPB3FDIAAACTUcgAAABMRiEDAAAw\nGYUMAADAZBQyAH4lLS1Na9askSTt2bNH06ZNMznR//nXv/6ljIwMz+2vv/5akyZN0rhx43TgwAHz\ngnnBggULuNI84EMUMgB+q3v37nr55ZcvyXM9++yz+vzzzxu9f2Vlpf75z3/qtttu89z35ptvasKE\nCXrzzTfVqVOni8p3991366effrqo52iI4uJizZs3T4888ojuvvtu5efn19g+evRorV692md5gGBH\nIQNgiqqqKrMjNEhWVpZiY2NrfPVNQUGB2rdvb2Kq/+NyuRr0eIvFol69emn69Om1bo+Pj1dZWZn2\n7dt3KeIBuICA/y5LAL/46aef9Kc//UmzZ89WXFycnE6nZsyYoSeeeEI9e/Y87/GFhYXKyMjQnj17\n5Ha7NXjwYE2cOFEul0vvvfeePv/8c1VUVKhXr16aMGGC5wviv/nmG/3973+X0+lUx44d9eCDD6pd\nu3aSpMmTJ+umm27SV199paNHj+rNN9/Ujz/+qPT0dB07dky9e/eu8YW+u3bt0iuvvKL09HTP/r//\n/e/1xRdfqKCgQL169dLkyZMVFham0tJSLVmyRLm5uXK5XLr66qv10EMPyW63a/Xq1dqzZ49yc3OV\nkZGhG2+8URMnTtSRI0e0YsUK7d+/Xy1atNA999yjQYMG1fr3y87OVo8ePSRJZ8+e1YQJE+RyuTRj\nxgxFRUXplVdekdPp1IoVK7Rnzx6Fh4dr1KhRGjlypCQpLy9PK1eu1JEjRxQWFqaBAwdq/Pjxstls\nmjNnjiRpxowZkqRJkyapoqJCn3/+uZ577jlPhrvvvluLFy9WmzZtlJaWprCwMBUWFmr37t2aMWOG\nunfvrtWrV2vLli2qrKxU//79lZSUpLCwsPN+n6ioKP3+97+vtxj36NFDO3bs8PkVy4FgxIQMCBJt\n2rTRfffdp1deeUVnzpzRq6++qqFDh9Zaxlwul+bNmyeHw6G0tDSlp6dr8ODBkn75Pr4NGzZozpw5\nWrJkicrLy7V8+XJJ0tGjR/Xyyy8rKSlJy5YtU+/evTVv3jxVVlZ6nnvTpk2aOXOmMjIy5Ha7NX/+\nfP32t7/VihUrdMMNN2jbtm31/h5btmzRrFmzlJaWph9//FEbNmyQ9Mt3pN54441aunSpli5dqrCw\nME+uMWPGqHv37p7DixMnTlR5eblSUlL0m9/8RsuWLdO0adO0fPlyHTp0qNbXPXTokGJjYyVJoaGh\nevPNNyVJ8+fP1yuvvOL5m3Xs2FGvvfaannnmGX388cee87BCQkI0fvx4LV++XCkpKfrf//1fffrp\np5Kkv/zlL57nevPNN+sshb/21Vdf6Q9/+INWrVqlbt266e2339axY8c0f/58LV68WE6nU//4xz8M\nPVdt2rVrpx9++KHR+wMwjkIGBJHExES1adNGs2bNUnFxscaMGVPr4/Ly8uR0OjVu3DiFh4crLCxM\n3bp1k/RLCbjlllvUunVrhYeH695779XmzZtVVVWlzZs3q3fv3rruuutks9l06623qqKiQnv37vU8\n98033yyHw6GwsDB9//33qqqq0qhRo2Sz2ZSQkHDBaczNN9+smJgYNW/eXH379tXBgwclSVdccYUS\nEhLUpEkTNW3aVHfccYf27NlT5/Ps2LFDLVu21LBhw2S1WhUXF6eBAwd6vgj6106dOqWmTZvW+Xz7\n9u3TiRMndOedd8pms6l169YaMWKENm/eLEmKi4tT165dZbVa1apVKyUmJmr37t31/q4X0r9/f3Xr\n1k0hISEKDQ3V559/rvHjx6t58+aev8GmTZsa/fzh4eE6derURWUEYAyHLIEgM2LECL344ot6+OGH\n6/yS48LCQrVs2VJWq/W8bcXFxWrZsqXntsPhUFVVlUpKSs7bFhISIofDIafTWePx5z5XTExMjcOU\n526vTVRUlOfnsLAwz3OfOXNGq1atUk5OjqdElJWVyeVyKSTk/H97FhQUKDc3V0lJSZ77qqqqNGTI\nkFpft1mzZiorK6szV0FBgYqLi2s8n8vlUvfu3SX9Mj184403tG/fPlVUVKiqqkpxcXH1/q4XYrfb\nPT+fOHFCZ86c0cyZMz33ud3uBp9bdq7y8nI1a9bsojICMIZCBgSR8vJyrVq1SsOHD9fatWuVkJCg\n5s2bn/c4h8OhwsJCVVVVnVfKoqOjVVBQ4LldWFgoq9WqyMhIRUdH68cff/Rsc7vdKiwsrHEi/K+f\ny+l0yu12e0pZUVGR2rRp0+Df7YMPPtDRo0f117/+VVFRUTp48KCefPJJud1uSapR+qRfykyPHj00\ne/ZsQ8/foUMHHTt2rM7tDodDrVq10uLFi2vdvmzZMnXs2FHTpk1T06ZN9dFHH9U5jZOkJk2aqKKi\nwnP7+PHj5z3m3N/piiuuUFhYmF566aU6/94NdfjwYXXo0OGSPBeA+nHIEggiK1euVKdOnfToo4+q\nT58+ev3112t9XHx8vKKjo/X222+rvLxcFRUV+u677yRJgwcP1kcffaT8/HyVl5dr9erVuuGGG2S1\nWjVo0CBlZ2fr22+/VWVlpT744AOFhobq6quvrvV1unbtqpCQEH3yySeqqqrStm3blJeX16jfrby8\nXGFhYYqIiFBpaanWrl1bY3tkZKR+/vlnz+2+ffvq2LFj+uKLL1RZWanKykrl5eXp8OHDtT5/7969\n6z3EGB8fr6ZNmyozM1MVFRVyuVz68ccfPb9PWVmZIiIiFB4eriNHjuizzz6rN1+HDh106NAhHTx4\nUBUVFXr33Xfr/f1DQkI0YsQIZWRkqKSkRJLkdDrrvZZYRUWFzp49K+mXy3qcWwClX64D17t373pf\nF8ClQSEDgkRWVpZycnL08MMPS5LGjx+vAwcO6MsvvzzvsSEhIXrqqaf0008/KTk5WZMmTfKcCzVs\n2DANGTJEc+bM0eTJkxUaGqoJEyZIkmJjYzVlyhStWLFCEydO1Pbt2/XUU0/JZqt9GG+z2fTHP/5R\nGzZs0AMPPKDNmzdrwIABjfr9Ro4cqYqKCk2cOFF//vOf1atXr/O2b9u2TQ888IBWrFihpk2b6umn\nn9amTZv0yCOP6OGHH9bbb79d4wMI5+rbt6+OHDlS4/BrbX+zgwcPavLkyZo4caJee+01nT59WpI0\nbtw4ffXVV7r//vv12muvnXfi/l133aW0tDQlJSVp8+bNio2N1Z133qnnnntO06ZN85zDV5/77rtP\nbdq00Z///GeNHz9ezz33nI4ePVrn48eOHav7779fkvTYY49p7Nixnm15eXlq0qSJ4uPjL/i6AC6e\nxV09zwcA1GvdunU6fPhwjfPEAtWCBQs0fPhw9enTx+woQFCgkAEAAJiMQ5YAAAAmo5ABAACYjEIG\nAABgMgoZAACAyShkAAAAJqOQAQAAmIxCBgAAYDIKGQAAgMn+H7DBLn6fH83bAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_data(blue, red)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Training the classifier\n",
"\n",
"Now it's time to train the classifier.\n",
"\n",
"As all other machine learning functions, the $k$-NN classifier is part of OpenCV 3.1's `ml` module. You can create a new classifier using the following command:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"knn = cv2.ml.KNearest_create()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We then pass our training data to the train method:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"knn.train(train_data, cv2.ml.ROW_SAMPLE, labels)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, we have to tell knn that our data is an $N \\times 2$ array (that is, every row is a data point).\n",
"Upon success, the function returns True."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Predicting the label of a new data point\n",
"\n",
"The other really helpful method that `knn` provides is called `findNearest`. It can be used to\n",
"predict the label of a new data point based on its nearest neighbors.\n",
"\n",
"Thanks to our `generate_data` function, it is actually really easy to generate a new data\n",
"point! We can think of a new data point as a dataset of size 1:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 91., 59.]], dtype=float32)"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"newcomer, _ = generate_data(1)\n",
"newcomer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Our function also returns a random label, but we are not interested in that. Instead, we\n",
"want to predict it using our trained classifier! We can tell Python to ignore an output value\n",
"with an underscore (`_`).\n",
"\n",
"Let's have a look at our town map again. We will plot the training set as we did earlier, but\n",
"also add the new data point as a green circle (since we don't know yet whether it is\n",
"supposed to be a blue square or a red triangle):"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmQAAAF6CAYAAAC3JUTKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4U3WixvE3TbpQkC4JiwUESkHAjVUqjoBQZ67gwvi4\nXBSkghvl4sagDKPiPNYRRCwiRXRYisuAMs/IuF4d5AEUrFbaMl5EbFkUKNolbaHQhTa5f3ibS6Ut\np4XkhOT7+avJOSd5k99z9OV3Ts6xuN1utwAAAGCaELMDAAAABDsKGQAAgMkoZAAAACajkAEAAJiM\nQgYAAGAyChkAAIDJKGQAAAAmo5ABAACYjEIGAABgMgoZAACAyShkAAAAJrOZHaA1CgoKWrWdw+FQ\ncXHxWU6Ds4Gx8W+Mj/9ibPwXY+PffDU+cXFxhtZjhgwAAMBkFDIAAACTUcgAAABMRiEDAAAwGYUM\nAADAZBQyAAAAk52Tl70AAKDerFlR2ru35f87i4+v1YIF5V5IBLQchQwAcE7bu9emzMxws2MAZ4RD\nlgAAACajkAEAAJiMQgYAAGAyChkAAIDJKGQAAAAmo5ABAACYjEIGAABgMgoZAACAyShkAAAAJqOQ\nAQAAmIxCBgAAYDIKGQAAgMm4uTgA4JwWH1/r0+0Ab6CQAQDOaQsWlJsdAThjHLIEAAAwGYUMfsHy\n2WdmRwAAwDQUMpjOmpcn2623ypqfb3YUAABMQSGD6aJSU2VxOhWVmmp2FAAATEEhg6mseXkKzc2V\nJIXm5jJLBgAIShQymCoqNVXW4mJJkrWoiFkyAEBQopDBNCfPjtVjlgwAEIwoZDDNybNj9ZglAwAE\nIwoZTNHY7Fg9ZskAAMGGQgZTNDY7Vo9ZMgBAsKGQweeamx2rxywZACCY+Oxelu+//742btwoi8Wi\nbt26KSUlRWVlZVq0aJEqKirUs2dPzZgxQzYbt9cMdM3NjtWrnyVzZmT4JhQAACbyyQyZ0+nURx99\npHnz5mnhwoVyuVzatm2b3njjDY0bN06LFy9W27ZttXHjRl/EgYmMzI7VY5YMABAsfHbI0uVyqaam\nRnV1daqpqVF0dLR27typxMRESdKoUaOUlZXlqzgwiZHZsXqcSwYACBY+OT4YGxur66+/XtOmTVNY\nWJguu+wyxcfHKzIyUlar1bOO0+n0RRyYxJqXp9CcnBZtE5qdLWt+vuoSEryUCgAA8/mkkFVUVCgr\nK0vp6emKjIzUCy+8oFyDh60kacOGDdqwYYMkad68eXI4HK3KYbPZWr0tzpxl3z7ptttU19iykBC5\nXa5Gt4u1WuVm3EzFvuO/GBv/xdj4N38bH58Usm+++UYdO3ZU+/btJUnDhg3T7t27dfz4cdXV1clq\ntcrpdCo2NrbR7ZOSkpSUlOR5XGzwkNevORyOVm+Ls6BnT+lPf2p00WnHhnEzFfuO/2Js/Bdj4998\nNT5xcXGG1vPJOWQOh0N5eXmqrq6W2+3WN998o65du+qiiy5SZmamJGnTpk0aMmSIL+IAAAD4FZ/M\nkPXu3VuJiYl67LHHZLVa1aNHDyUlJWnQoEFatGiR1q5dq549e2r06NG+iAMAAOBXLG632212iJYq\nKCho1XZMH/svxsa/MT7+i7HxX4yNfwvKQ5YAAABoGoUMAADAZBQyAAAAk1HIAAAATEYhAwAAMBmF\nDAAAwGQUMgAAAJNRyAAAAExGIQMAIMCE/t9tCXHuoJABABBArHl5ip06Vdb8fLOjoAUoZAAABJCo\n1FRZy8oUlZpqdhS0AIUMAIAAYc3LU2huriQpNDeXWbJzCIUMAIAAEZWaKuv/3TDbWlTELNk5hEIG\nAEAAOHl2rB6zZOcOChkAAAHg5NmxesySnTsoZAAAnOMamx2rxyzZuYFCBgDAOa6x2bF6zJKdGyhk\nAACcw5qbHavHLJn/o5ABAHAOa252rB6zZP6PQgYAwDnKyOxYPWbJ/BuFDACAc5SR2bF6zJL5NwoZ\nAADnIGtenkJzclq0TWh2NrNkfspmdgAAANBylooKVd54Y8u3O3rUC2lwpihkAACcg2oHDtSRgQPN\njoGzhEOWAAAAJqOQAQAAmIxCBgAAYDIKGQAAgMkoZAAAACajkAEAAJiMQgYAAGAyChkAAIDJKGQA\nAAAmo5ABAACYjEIGAABgMgoZAACAyShkAAAAJqOQAQAAmIxCBgAAYDIKGQAAgMkoZAAAACajkAEA\nAJiMQgYAAGAyChkAAIDJKGQAAAAmo5ABAACYjEIGAABgMgoZAACAyShkAAAAJrOdboW9e/cqOztb\nP/zwg44fP67IyEh1795dAwcOVK9evXyREQAAIKA1Wch27NihNWvWqLKyUv3799eFF16oNm3aqLKy\nUocOHdLixYsVERGhCRMmaMCAAb7MDAAAEFCaLGT/+te/dPfddyshIaHJjfPz8/XPf/6TQgYAAHAG\nmixkf/jDH067cUJCgmbOnHlWAwEAAAQbTuoHAAAwWbOFrKqqSsuXL9esWbP04osv6qeffmqwnNkx\nAACAM9dsIXvttddUVFSkW265RVFRUZozZ4527tzpWV5UVOT1gAAAAIGu2ctebN++XWlpaYqMjNTl\nl1+uQYMGKS0tTSkpKRo0aJCvMgIAAAS0ZmfIampqZLP9f2e79NJL9eijj+rll1/WF198IYvF4vWA\nAIDgFpqZaXYEwOuaLWTdunXTrl27GjzXp08fzZkzRytXrlR1dbVXwwEAgps1L0+xU6fKmp9vdhTA\nq5otZOPHj1dFRcUpz/fs2VNPPPGERowY4bVgAABEpabKWlamqNRUs6MAXtXsOWTNnSd2wQUXKCUl\nxfAbHTt2TMuWLdOBAwdksVg0bdo0xcXFKS0tTUVFRerQoYMefvhhtWvXznh6AMBpzZoVpb17T3un\nvFPEx9dqwYJyLyQyxpqXp9DcXElSaG6urPn5qmvmYuXwPpfbpc0HN2vpjqXKLsxWdV21wq3hGtRx\nkFIuS9HIriMVYuGKWq3R8j20lVatWqUBAwZo5syZqq2tVXV1td555x1dcsklGj9+vNavX6/169dr\n4sSJvooEAEFh716bMjPDzY7RYlGpqbIWF0uSrEVFikpNlTMjw9xQQWxP2R5N+GiCyqrLdOzEMc/z\nVXVV2nZ4m3YU71B0eLTWXLtGvaK513VL+aTGHj9+XLt27dLo0aMlSTabTW3btlVWVpZGjhwpSRo5\ncqSysrJ8EQcA4OdOnh2rVz9LBt/bU7ZH4/45TgUVBQ3K2MmOnTimgooCjfvnOO0p2+PjhOc+nxSy\nwsJCtW/fXkuXLtWjjz6qZcuWqaqqSuXl5YqJiZEkxcTE6MiRI76IAwDwcyfPjtWrnyWDb7ncLk34\naIIqairklrvZdd1yq6KmQrd/dLtcbpePEgYGnxyyrKur0759+zRlyhT17t1bq1at0vr16w1vv2HD\nBm3YsEGSNG/ePDkcjlblsNlsrd4W3sXY+DfGx38ZGZvQ0Nb9pz40NNSccf/uO4X++9+NLgrfsUOO\nkhLpwgt9HKrlAmW/+XjPxyqvKT9tGavnlltlNWXKPZqr38b/1svpWs/fxsfQXnrixAn9/e9/19at\nW3X06FGtXr1aO3bs0OHDh/Uf//Efp93ebrfLbrerd+/ekqTExEStX79eUVFRKi0tVUxMjEpLS9W+\nfftGt09KSlJSUpLncfGv/tVklMPhaPW28C7Gxr8xPv7LyNicOGGX1PJzyE6cOKHi4pJWJmu92Jkz\nZSksbHSZpbBQrpkzz4lzyQJlv5n32TxV1Jx6xYXmVNRU6Nktz2pQe/+9iLyvxicuLs7QeoYOWa5e\nvVoHDhzQAw884LkYbLdu3fTJJ58YepPo6GjZ7XYVFBRIkr755ht17dpVQ4YM0ebNmyVJmzdv1tCh\nQw29HgAgMDV27tivcS6Zb2UXZvt0u2BlaIbsq6++0uLFixUREeEpZLGxsXI6nYbfaMqUKVq8eLFq\na2vVsWNHpaSkyO12Ky0tTRs3bpTD4dAjjzzSuk8BAAgIjZ079mv84tK3qutadxH41m4XrAwVMpvN\nJper4cl5R44c0XnnnWf4jXr06KF58+ad8vyTTz5p+DUAAIHLyOxYPa5L5jvh1nBV1VW1ajsYZ+iQ\nZWJiopYsWaLC/zumX1paqhUrVmj48OFeDQcACB5GZsfq8YtL3xnUsXXngbV2u2BlqJDdfvvt6tix\no2bOnKnjx4/rgQceUExMjG655RZv5wMABAFrXp5Cc3JatE1odjbnkvlAymUpahvatkXbtLW11fTL\npnspUWA67SFLl8ul7777TnfccYeSk5M9hyrrzyUDAOBMWSoqVHnjjS3f7uhRL6TByUZ2Hano8Ggd\nP3Hc0KUvLLIoJiJGI7pyv+uWOG0hCwkJ0XPPPafXXntNkpq8NAUAAK1VO3CgjgwcaHYMNCLEEqI1\n167RuH+OO+3FYS2yqF1YO/3t2r9xT8sWMvRt9evXT99//723swAAAD/UK7qXPrjxA8W1i2vy8GXb\n0Lbq0q6LPrjxA+5l2QqGfmXZoUMHPfvssxoyZIjsdnuDw5W33Xab18IBAM5cfHytT7dDYOoV3UuZ\n/5mpLQe3KH1HurILs1VdV61wa7gGdRyk6ZdN14iuI5gZayVDhaympsZz0daWXHsMAGC+BQvKzY6A\nABFiCdGobqM0qtsos6MEHEOFLCUlxds5AAAAgpahQvbzzz83uaxTp05nLQwAAEAwMlTIHnjggSaX\nvfXWW2ctDAAAQDAyVMh+XbrKysq0bt069evXzyuhAAAAgkmrfgoRHR2t5ORk/e1vfzvbeQAAAIJO\nq3+bWlBQoOpq7uQOAABwpgwdsnzyyScbXHusurpaBw4c0M033+y1YAAAAMHCUCEbPXp0g8cRERHq\n3r27zj//fK+EAgAACCaGCll4eLiuuOKKU57PzMxUYmLiWQ8FAAAQTAydQ7Zs2bJGn3/llVfOahgA\nAIBg1OwMWf0FYV0ulwoLC+V2uxssCwsL8246AACAINBsITv5grAzZsxosCw6Olq33HKLd1IBAAAE\nkWYLWf0FYefOnas///nPPgkEAAAQbAydQ0YZAwAA8B5Dv7Ksq6vTxx9/rG+//VZHjx5tsIyyBgAA\ncGYMzZCtXr1aGzZsUP/+/bV3714NGzZM5eXluuiii7ydDwAAIOAZKmRffvml5syZo7Fjx8pqtWrs\n2LGaNWuWdu7c6e18AAAAAc9QIaupqZHdbpckhYWFqbq6Wl26dNH+/fu9mQ0AACAoGDqHrEuXLtqz\nZ48SEhIUHx+vdevWqU2bNoqNjfV2PgAAgIBnaIYsOTlZISG/rDp58mTt27dP27dv17333uvVcAAA\nAMHA0AxZQkKC5+/zzz9fTzzxhNcCAQAABBtDhUyS/v3vf2vr1q0qLy/X7NmztWfPHlVWVuriiy/2\nZj4AAICAZ+iQ5UcffaS//vWvOv/887Vr1y5Jv5zcv3btWq+GAwAACAaGCtmHH36oJ554QuPHj/ec\nS9alSxcVFBR4NRwAAEAwMFTIKisr5XA4GjxXW1srm83wEU8AAAA0wVAh69evn9avX9/guY8++ogr\n9QMAAJwFhgrZlClT9NVXX2n69OmqqqrSgw8+qMzMTE2ePNnb+QAAAAJek8cc9+/frx49ekiSYmJi\n9Oyzz2rPnj0qKiqS3W5XQkKC53wyAAAAtF6TjWru3Lmevx944AFZLBYlJCToiiuuUJ8+fShjAAAA\nZ0mTM2SRkZHavn27unbtqtLSUhUWFsrtdp+yXqdOnbwaEAAAINA1WcjuuusuZWRkqLi4WC6XSzNm\nzGh0vbfeestr4QAAAIJBk4Xs8ssv1+WXXy5JuvPOO/Xaa6/5LBQAAEAwMXQi2MqVK72dAwAAIGg1\nWchWr16tsrIySWryArBlZWVavXq1d5IBAAAEiSYPWcbFxemPf/yjunbtqn79+ikuLk5t2rRRZWWl\nDh8+rG+//VYFBQW66aabfJkXAAAg4DRZyK655hpdffXV+vrrr5WTk6OsrCwdP35cbdu21QUXXKBr\nrrlGgwcPltVq9WVeAACAgNPszShtNpsSExOVmJjoqzwAAABBh6u7AgAAmIxCBgAAYDIKGQAAgMko\nZAAAACZr9qT+kx08eFCZmZkqKyvT3XffrUOHDqm2tlbdu3f3Zj4AAICAZ2iG7IsvvtBTTz0lp9Op\nzz77TJJUVVXF7ZQAAADOAkMzZG+//bYef/xx9ejRQ1988YUkqXv37tq/f783swEAAAQFQzNk5eXl\npxyatFgsslgsXgkFAAAQTAwVsvj4eG3ZsqXBc1u3blVCQoJXQgEAAAQTQ4cs77rrLqWmpmrjxo2q\nrq7WM888o4KCAj3++OPezgcAABDwDBWyLl26aNGiRdq+fbsGDx4su92uwYMHKyIiwtv5AAAAAp6h\nQ5YrV65UeHi4hg8frhtuuEFXXnmlIiIilJGR4eV4AAAAgc9QIdu8eXOjz//6vDIAAAC0XLOHLDdu\n3ChJqqur8/xdr7CwUOedd573kgEAAASJZgtZ/UVga2trPX/Xi4qK0vTp072XDAAAIEg0W8jmzp0r\nSVq7dq3+8z//0yeBAAAAgo2hX1meXMbcbrfcbrfncUiI8fuTu1wuzZ49W7GxsZo9e7YKCwu1aNEi\nVVRUqGfPnpoxY4ZsNsO31wQAAAgIhtqP0+nUihUrtGvXLh07dqzBsrfeesvwm3344Yfq0qWLKisr\nJUlvvPGGxo0bpyuvvFKvvvqqNm7cqN/+9rctiA8AAHDuMzS99eqrr8pms+nJJ59URESE5s+fryFD\nhuiee+4x/EYlJSXKzs7WmDFjJP0y07Zz504lJiZKkkaNGqWsrKxWfAQAAIBzm6FC9v3332vatGnq\n0aOHLBaLevTooWnTpun99983/EYZGRmaOHGi5/6XR48eVWRkpKxWqyQpNjZWTqezFR8BAADg3Gbo\nkGVISIinOLVt21ZHjhxRmzZtDBeo7du3KyoqSvHx8dq5c2eLQ27YsEEbNmyQJM2bN08Oh6PFryFJ\nNput1dvCuxgb/8b4+C/Gxn8xNv7N38bHUCFLSEhQTk6OLr/8cl122WVKS0tTWFiYevXqZehNdu/e\nra+//lo5OTmqqalRZWWlMjIydPz4cdXV1clqtcrpdCo2NrbR7ZOSkpSUlOR5XFxcbOh9f83hcLR6\nW3gXY+PfGB//xdj4L8bGv/lqfOLi4gytZ6iQzZgxw/PLyuTkZL377ruqqqrSuHHjDL3J7bffrttv\nv12StHPnTr333nt64IEH9MILLygzM1NXXnmlNm3apCFDhhh6PQAAgEBiqJC1bdvW83dYWJhuvvnm\ns/Lmd9xxhxYtWqS1a9eqZ8+eGj169Fl5XQAAgHOJoUJWW1urTZs2af/+/aqqqmqw7L/+679a9IYX\nXXSRLrroIklSp06d9Oyzz7ZoewAAgEBjqJAtWbJEP/zwgwYPHqyoqChvZwIAAAgqhgrZjh07tGTJ\nkgaHLgEAAHB2GLoOmcPh0IkTJ7ydBQAAICgZmiEbMWKEFixYoGuvvVbR0dENll188cVeCQYAABAs\nDBWy//7v/5YkrVmzpsHzFotFS5YsOfupAAAAgoihQpaenu7tHAAAAEHL0DlkAAAA8J4mZ8gefvhh\npaWlSZKmTZvW5Au8/PLLZz8VAABAEGmykN13332ev2fMmOGTMAAAAMGoyULWt29fz9/9+/f3SRgA\nAIBg1GQhe+uttwy9wG233XbWwgAAAASjJgtZSUmJ5++amhp9+eWXSkhIkMPhUHFxsfLz8zVs2DCf\nhAQAAAhkTRaylJQUz9+LFi3Sgw8+qMTERM9zX375pb744gvvpgMAAAgChi57kZOTo8svv7zBc0OH\nDlVOTo5XQgEAAAQTQ4Wsc+fOnqv11/v444/VuXNnr4QCAAAIJoau1H///ffr+eef17vvvqvY2Fg5\nnU5ZrVbNnDnT2/kQJCyffSb162d2DAAATGGokHXv3l0vvvii8vLyVFpaqujoaPXp00c2m6HNgWZZ\n8/Jku/VWWd95R3UJCWbHAQDA5057yNLlcmnSpElyu93q16+fhg8frv79+1PGcNZEpabK4nQqKjXV\n7CgAAJjitIUsJCREcXFxOnr0qC/yIMhY8/IUmpsrSQrNzZU1P9/kRAAA+J6haa7f/OY3mj9/vq69\n9lrZ7XZZLBbPsosvvthr4RD4olJTZS0uliRZi4oUlZoqZ0aGuaEAAPAxQ4Xsk08+kSStW7euwfMW\ni0VLliw5+6kQFE6eHatXP0vGuWQAgGBiqJClp6d7OweC0MmzY/WYJQMABCPDZ+bX1dVp9+7dcjqd\nstvt6tOnj6xWqzezIYA1NjtWj1kyAECwMVTIDh06pPnz56umpkZ2u10lJSUKDQ3VY489pq5du3o7\nIwJQY7Nj9ZglAwAEG0OFbPny5UpKStL111/vOaH/3Xff1YoVKzR37lyvBkTgaW52rB6zZACAYGKo\nkO3fv19PPPFEg19Xjhs3Tu+8847XgiFwNTc7Vo9ZMgBonVmzorR3b8uvFRofX6sFC8q9kAhGGBqx\n2NhYffvttw0ucbFr1y7FxMR4LRgCk5HZsXrMkgFAy+3da1NmZrjZMdBChgrZhAkTNH/+fA0ePFgO\nh0PFxcXKzs7WjBkzvJ0PAcbI7Fg9ZskAAMHitFfql6QhQ4Zo/vz56tatm6qqqtStWzfNmzdPQ4cO\n9XY+BBBrXp5Cc3JatE1odjZX7wcABDzDB5nj4uL0+9//XuXl5YqKilJIiKEuB3hYKipUeeONjS5r\nExGhyqqqxrfjtl0AgABnqJAdP35cK1eu1LZt21RXVyer1arhw4drypQpioyM9HZGBIjagQN1ZODA\nRpeFORw6YvBQJgAAgcbQNNeqVatUVVWl559/Xm+88Yaef/551dTUaOXKld7OBwAAEPAMFbLc3FzN\nmDFDcXFxCg0NVVxcnFJSUrRjxw5v5wMAAAh4hgpZWFiYjhw50uC5I0eOyGZr+XVOAAAA0JChRjV6\n9GilpqZq3Lhx6tChg4qKivTBBx8oKSnJ2/kAAAACnqFCdtNNNykmJkZbt26V0+lUbGysbrzxRl19\n9dXezgcAABDwDBUyi8Wi0aNHa/To0d7OAwAAEHQMnUO2cuVK7d69u8Fzu3fvVgZXUAcAADhjhgrZ\n1q1b1atXrwbPxcfH6/PPP/dKKAAAgGBi+JCly+Vq8JzL5ZLb7fZKKAAA0Drx8bU+3Q5nh6FC1rdv\nX61du1YTJ05USEiIXC6X1q1bp759+3o7HwAAaIEFC8rNjoBWMFTI7rrrLs2bN0/33XefHA6HiouL\nFRMTo8cee8zb+QAAAAKeoUJmt9s1f/585efnq6SkRHa7XQkJCdxgHAAA4CwwfKn9kJAQ9enTx5tZ\nAAAAghJTXAAAACajkAEAAJiMQgYAAAJaaGam2RFOy1AhW716tfbv3+/lKAAAAGeXNS9PsVOnypqf\nb3aUZhkqZHV1dXrmmWc0c+ZMrV+/XiUlJd7OBQAAcMaiUlNlLStTVGqq2VGaZehXllOmTFFycrJy\ncnL02Wef6R//+Id69+6tESNGaNiwYYqIiPB2TgAAgBax5uUpNDdXkhSamytrfr7qEhJMTtU4w+eQ\nhYSEaPDgwXrooYf0zDPP6MiRI1q6dKnuueceLVu2TE6n05s5AQAAWiQqNVXW4mJJkrWoyK9nyQxf\nh+z48ePKzMzUZ599ph9++EHDhg3T1KlT5XA49P777+svf/mLnn/+eW9mBQAAMOTk2bF6/jxLZqiQ\nLVy4UDt27FC/fv10zTXXaOjQoQoNDfUsv/POO5WcnOytjAAAAC1y8uxYvfpZMmdGhjmhmmGokPXu\n3VtTp05VdHR0o8tDQkL017/+9awG87ZZs6K0d6/hCUKP+PhabtwKAIAfa2x2rF79LJkcDh+nap6h\nRnLDDTecdp3w8PAzDuNLe/falJl5bmUGAACn19jsWD3PuWTvv+/jVM3jwrAAACBgNDc7Vi80N1fa\nvdtHiYyhkAEAgIDR3OxYPWtRkWx//KOPEhlDIQMAAAHByOxYPUtWll9dvZ9CBgAAAoKR2bF6lsJC\nv7ouWct/ZtgKxcXFSk9PV1lZmSwWi5KSkjR27FhVVFQoLS1NRUVF6tChgx5++GG1a9fOF5EAAEAA\nseblKTQnp0XbhGZn+811yXxSyKxWqyZNmqT4+HhVVlZq9uzZuvTSS7Vp0yZdcsklGj9+vNavX6/1\n69dr4sSJvogEAAACiKWiQpU33mh4/TYREaqsqpLl6FEvpjLOJ4UsJiZGMTExkqQ2bdqoS5cucjqd\nysrK0lNPPSVJGjlypJ566ikKGQAAaLHagQN1ZOBAw+uHORw6YvDwpi/4/ByywsJC7du3TwkJCSov\nL/cUtZiYGB05csTXcQAAAEznkxmyelVVVVq4cKGSk5MVGRlpeLsNGzZow4YNkqR58+bJ0cqr69ps\nNs+2oaGt++ihoaGtfn807eSxgf9hfPwXY+O/GBv/5m/j47NCVltbq4ULF+qqq67SsGHDJElRUVEq\nLS1VTEyMSktL1b59+0a3TUpKUlJSkudxcSunGB0Oh2fbEyfsklp+pf4TJ06ouLikVe+Ppp08NvA/\njI//Ymz8F2Pj33w1PnFxcYbW88khS7fbrWXLlqlLly667rrrPM8PGTJEmzdvliRt3rxZQ4cO9UUc\nAAAAv+KTGbLdu3dry5YtuuCCCzRr1ixJ0oQJEzR+/HilpaVp48aNcjgceuSRR3wRBwAAwK/4pJD1\n7dtXb7/9dqPLnnzySV9EOEV8fK1PtwMAAGiKT0/q9ycLFpSbHQEAAEASt04CAAAwHYUMAADAZBQy\nAAAAk1HIAAAATEYhAwAAMBmFDAAAwGQUMgAAAJNRyAAAAExGIQMAADAZhQwAAMBkFDIAAACTUcgA\nAABMRiEDAAAwGYUMAADAZBQyAAAAk1HIAAAATEYhAwAAMBmFDAAAwGQUMgAAAJNRyAAAAExGIQMA\nADAZhQwAAMBkFDIAAACTUcgAAABMRiEDAAAwGYUMAADAZBQyAAAAk1HIAAAATEYhAwAAMBmFDAAA\nwGQUMgDGkXDrAAASEUlEQVQAAJNRyAAAAExGIQMAADAZhQwAAMBkFDIAAACTUcgAAABMRiEDAAAw\nGYUMAADAZBQyAAAAk1HIAAAATEYhAwAAMBmFDAAAwGQUMgAAAJNRyAAAAExGIQMAADAZhQwAAMBk\nFDIAAACTUcgAAABMRiEDAAAwGYUMAADAZBQyAAAAk1HIAAAATEYhAwAAMJnN7AAAzo5Zs6K0d2/L\nd+n4+FotWFDuhUTnBr43AP6AQgYEiL17bcrMDDc7xjmH7w2AP+CQJQAAgMkoZAAAACajkAEAAJiM\nQgYAAGAy00/qz83N1apVq+RyuTRmzBiNHz/e7EgAAAA+ZeoMmcvl0ooVKzRnzhylpaVp69atOnjw\noJmRAAAAfM7UQpafn6/OnTurU6dOstlsGj58uLKyssyMBAAA4HOmFjKn0ym73e55bLfb5XQ6TUwE\nAADge6aeQ+Z2u095zmKxnPLchg0btGHDBknSvHnz5HA4WvV+Nput1dvCuxibMxca2rrdOTQ09LTf\nfSCPjze/N18I5LE51zE2/s3fxsfUQma321VSUuJ5XFJSopiYmFPWS0pKUlJSkudxcXFxq97P4XC0\nelt4F2Nz5k6csEtq+RXnT5w4oeLikmbXCeTx8eb35guBPDbnOsbGv/lqfOLi4gytZ+ohy169eunw\n4cMqLCxUbW2ttm3bpiFDhpgZCQAAwOdMnSGzWq2aMmWKnnnmGblcLl199dXq1q2bmZEAAAB8zvTr\nkA0aNEiDBg0yOwZwzouPr/XpdoGC7w2APzC9kAE4OxYsKDc7wjmJ7w2AP+DWSQAAACajkAEAAJiM\nQgYAAGAyChkAAIDJKGQAAAAmo5ABAACYzOJu7IaSAAAA8JmgmiGbPXu22RHQBMbGvzE+/oux8V+M\njX/zt/EJqkIGAADgjyhkAAAAJrM+9dRTT5kdwpfi4+PNjoAmMDb+jfHxX4yN/2Js/Js/jQ8n9QMA\nAJiMQ5YAAAAms5kdwBdyc3O1atUquVwujRkzRuPHjzc7UlArLi5Wenq6ysrKZLFYlJSUpLFjx6qi\nokJpaWkqKipShw4d9PDDD6tdu3Zmxw1KLpdLs2fPVmxsrGbPnq3CwkItWrRIFRUV6tmzp2bMmCGb\nLSj+8+FXjh07pmXLlunAgQOyWCyaNm2a4uLi2G/8xPvvv6+NGzfKYrGoW7duSklJUVlZGfuOSZYu\nXars7GxFRUVp4cKFktTk/2fcbrdWrVqlnJwchYeHKyUlxeeHMwN+hszlcmnFihWaM2eO0tLStHXr\nVh08eNDsWEHNarVq0qRJSktL0zPPPKOPP/5YBw8e1Pr163XJJZdo8eLFuuSSS7R+/XqzowatDz/8\nUF26dPE8fuONNzRu3DgtXrxYbdu21caNG01MF7xWrVqlAQMGaNGiRVqwYIG6dOnCfuMnnE6nPvro\nI82bN08LFy6Uy+XStm3b2HdMNGrUKM2ZM6fBc03tLzk5Ofrpp5+0ePFi3XvvvVq+fLnP8wZ8IcvP\nz1fnzp3VqVMn2Ww2DR8+XFlZWWbHCmoxMTGef3m0adNGXbp0kdPpVFZWlkaOHClJGjlyJONkkpKS\nEmVnZ2vMmDGSJLfbrZ07dyoxMVHSL/+RY2x87/jx49q1a5dGjx4tSbLZbGrbti37jR9xuVyqqalR\nXV2dampqFB0dzb5jov79+58yW9zU/vL1119rxIgRslgs6tOnj44dO6bS0lKf5g34eVOn0ym73e55\nbLfblZeXZ2IinKywsFD79u1TQkKCysvLFRMTI+mX0nbkyBGT0wWnjIwMTZw4UZWVlZKko0ePKjIy\nUlarVZIUGxsrp9NpZsSgVFhYqPbt22vp0qX64YcfFB8fr+TkZPYbPxEbG6vrr79e06ZNU1hYmC67\n7DLFx8ez7/iZpvYXp9Mph8PhWc9ut8vpdHrW9YWAnyFr7EekFovFhCT4taqqKi1cuFDJycmKjIw0\nOw4kbd++XVFRUX71U3D8oq6uTvv27dNvf/tbPffccwoPD+fwpB+pqKhQVlaW0tPT9corr6iqqkq5\nublmx4JB/tAVAn6GzG63q6SkxPO4pKTEp40XjautrdXChQt11VVXadiwYZKkqKgolZaWKiYmRqWl\npWrfvr3JKYPP7t279fXXXysnJ0c1NTWqrKxURkaGjh8/rrq6OlmtVjmdTsXGxpodNejY7XbZ7Xb1\n7t1bkpSYmKj169ez3/iJb775Rh07dvR8/8OGDdPu3bvZd/xMU/uL3W5XcXGxZz0zukLAz5D16tVL\nhw8fVmFhoWpra7Vt2zYNGTLE7FhBze12a9myZerSpYuuu+46z/NDhgzR5s2bJUmbN2/W0KFDzYoY\ntG6//XYtW7ZM6enpeuihh3TxxRfrgQce0EUXXaTMzExJ0qZNm9iHTBAdHS273a6CggJJvxSArl27\nst/4CYfDoby8PFVXV8vtdnvGh33HvzS1vwwZMkRbtmyR2+3W999/r8jISJ8XsqC4MGx2drZWr14t\nl8ulq6++WjfddJPZkYLad999pyeffFIXXHCBZ0p4woQJ6t27t9LS0lRcXCyHw6FHHnmEn++baOfO\nnXrvvfc0e/Zs/fzzz6f8dD80NNTsiEFn//79WrZsmWpra9WxY0elpKTI7Xaz3/iJt99+W9u2bZPV\nalWPHj10//33y+l0su+YZNGiRfr222919OhRRUVF6dZbb9XQoUMb3V/cbrdWrFihHTt2KCwsTCkp\nKerVq5dP8wZFIQMAAPBnAX/IEgAAwN9RyAAAAExGIQMAADAZhQwAAMBkFDIAAACTUcgA+JX09HSt\nXbtWkrRr1y49+OCDJif6f//617+UkZHhefzVV19p2rRpmjRpkvbt22deMC94/vnnudI84EMUMgB+\nq1+/fnrxxRfPyms99dRT+vTTT1u9fW1trf7xj3/ohhtu8Dz3+uuva8qUKXr99dfVs2fPM8p36623\n6qeffjqj12iJ0tJSzZ8/X/fdd59uvfVWFRYWNlg+fvx4rVmzxmd5gGBHIQNgirq6OrMjtEhWVpbi\n4uIa3PqmqKhI3bp1MzHV/3O5XC1a32KxaMCAAZo5c2ajyxMSElRZWak9e/acjXgATiPg72UJ4Bc/\n/fST/vjHP+qJJ55QfHy8nE6nZs2apUceeUQXXXTRKesXFxcrIyNDu3btktvt1pVXXqmpU6fK5XLp\nnXfe0aeffqqamhoNGDBAU6ZM8dwg/uuvv9bf/vY3OZ1O9ejRQ3fffbe6du0qSZo+fbquueYaff75\n5yooKNDrr7+uH3/8UcuWLdPhw4c1cODABjf03blzp1566SUtW7bMs/3vfvc7bdmyRUVFRRowYICm\nT5+usLAwVVRUaMmSJcrLy5PL5dKFF16oe+65R3a7XWvWrNGuXbuUl5enjIwMjRo1SlOnTtWhQ4e0\ncuVK7d27V+3bt9dtt92m4cOHN/r95eTkqH///pKkEydOaMqUKXK5XJo1a5aio6P10ksvyel0auXK\nldq1a5ciIiI0btw4jR07VpKUn5+vVatW6dChQwoLC9OwYcM0efJk2Ww2zZ07V5I0a9YsSdK0adNU\nU1OjTz/9VE8//bQnw6233qrFixerc+fOSk9PV1hYmIqLi/Xtt99q1qxZ6tevn9asWaMvvvhCtbW1\nGjp0qJKTkxUWFnbK54mOjtbvfve7Zotx//79lZ2d7fMrlgPBiBkyIEh07txZd9xxh1566SVVV1fr\n5Zdf1siRIxstYy6XS/Pnz5fD4VB6erqWLVumK6+8UtIv9+PbtGmT5s6dqyVLlqiqqkorVqyQJBUU\nFOjFF19UcnKyli9froEDB2r+/Pmqra31vPbWrVs1e/ZsZWRkyO12a8GCBbrqqqu0cuVKXXHFFfry\nyy+b/RxffPGF5syZo/T0dP3444/atGmTpF/ukTpq1CgtXbpUS5cuVVhYmCfXhAkT1K9fP8/hxalT\np6qqqkqpqan6zW9+o+XLl+vBBx/UihUrdODAgUbf98CBA4qLi5MkhYaG6vXXX5ckLViwQC+99JLn\nO+vRo4deeeUVPfnkk/rwww8952GFhIRo8uTJWrFihVJTU/U///M/+vjjjyVJf/7znz2v9frrrzdZ\nCn/t888/1+9//3utXr1affv21ZtvvqnDhw9rwYIFWrx4sZxOp/7+978beq3GdO3aVT/88EOrtwdg\nHIUMCCJJSUnq3Lmz5syZo9LSUk2YMKHR9fLz8+V0OjVp0iRFREQoLCxMffv2lfRLCbjuuuvUqVMn\nRURE6Pbbb9e2bdtUV1enbdu2aeDAgbr00ktls9l0/fXXq6amRrt37/a89rXXXiuHw6GwsDB9//33\nqqur07hx42Sz2ZSYmHja2Zhrr71WsbGxateunQYPHqz9+/dLks477zwlJiYqPDxcbdq00U033aRd\nu3Y1+TrZ2dnq0KGDrr76almtVsXHx2vYsGGeG0H/2rFjx9SmTZsmX2/Pnj06cuSIbr75ZtlsNnXq\n1EljxozRtm3bJEnx8fHq06ePrFarOnbsqKSkJH377bfNftbTGTp0qPr27auQkBCFhobq008/1eTJ\nk9WuXTvPd7B169ZWv35ERISOHTt2RhkBGMMhSyDIjBkzRs8995zuvffeJm9yXFxcrA4dOshqtZ6y\nrLS0VB06dPA8djgcqqurU3l5+SnLQkJC5HA45HQ6G6x/8mvFxsY2OEx58vLGREdHe/4OCwvzvHZ1\ndbVWr16t3NxcT4morKyUy+VSSMip//YsKipSXl6ekpOTPc/V1dVpxIgRjb5v27ZtVVlZ2WSuoqIi\nlZaWNng9l8ulfv36Sfpl9vC1117Tnj17VFNTo7q6OsXHxzf7WU/Hbrd7/j5y5Iiqq6s1e/Zsz3Nu\nt7vF55adrKqqSm3btj2jjACMoZABQaSqqkqrV6/W6NGjtW7dOiUmJqpdu3anrOdwOFRcXKy6urpT\nSllMTIyKioo8j4uLi2W1WhUVFaWYmBj9+OOPnmVut1vFxcUNToT/9Ws5nU653W5PKSspKVHnzp1b\n/Nnee+89FRQU6C9/+Yuio6O1f/9+Pfroo3K73ZLUoPRJv5SZ/v3764knnjD0+t27d9fhw4ebXO5w\nONSxY0ctXry40eXLly9Xjx499OCDD6pNmzb64IMPmpyNk6Tw8HDV1NR4HpeVlZ2yzsmf6bzzzlNY\nWJheeOGFJr/vljp48KC6d+9+Vl4LQPM4ZAkEkVWrVqlnz566//77NWjQIL366quNrpeQkKCYmBi9\n+eabqqqqUk1Njb777jtJ0pVXXqkPPvhAhYWFqqqq0po1a3TFFVfIarVq+PDhysnJ0TfffKPa2lq9\n9957Cg0N1YUXXtjo+/Tp00chISH66KOPVFdXpy+//FL5+fmt+mxVVVUKCwtTZGSkKioqtG7dugbL\no6Ki9PPPP3seDx48WIcPH9aWLVtUW1ur2tpa5efn6+DBg42+/sCBA5s9xJiQkKA2bdpo/fr1qqmp\nkcvl0o8//uj5PJWVlYqMjFRERIQOHTqkTz75pNl83bt314EDB7R//37V1NTo7bffbvbzh4SEaMyY\nMcrIyFB5ebkkyel0NnstsZqaGp04cULSL5f1OLkASr9cB27gwIHNvi+As4NCBgSJrKws5ebm6t57\n75UkTZ48Wfv27dNnn312yrohISF67LHH9NNPPyklJUXTpk3znAt19dVXa8SIEZo7d66mT5+u0NBQ\nTZkyRZIUFxenGTNmaOXKlZo6daq2b9+uxx57TDZb45PxNptNf/jDH7Rp0ybddddd2rZtmy6//PJW\nfb6xY8eqpqZGU6dO1Z/+9CcNGDDglOVffvml7rrrLq1cuVJt2rTR448/rq1bt+q+++7Tvffeqzff\nfLPBDxBONnjwYB06dKjB4dfGvrP9+/dr+vTpmjp1ql555RUdP35ckjRp0iR9/vnnuvPOO/XKK6+c\ncuL+LbfcovT0dCUnJ2vbtm2Ki4vTzTffrKeffloPPvig5xy+5txxxx3q3Lmz/vSnP2ny5Ml6+umn\nVVBQ0OT6EydO1J133ilJeuihhzRx4kTPsvz8fIWHhyshIeG07wvgzFnc9fP5AIBmbdiwQQcPHmxw\nnligev755zV69GgNGjTI7ChAUKCQAQAAmIxDlgAAACajkAEAAJiMQgYAAGAyChkAAIDJKGQAAAAm\no5ABAACYjEIGAABgMgoZAACAyf4XMbjFi/uUq2cAAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_data(blue, red)\n",
"plt.plot(newcomer[0, 0], newcomer[0, 1], 'go', markersize=14);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you had to guess based on its neighbors, what label would you assign the new data pointblue\n",
"or red?\n",
"\n",
"Well, it depends, doesn't it? If we look at the house closest to it (the one living roughly at $(x,\n",
"y) = (85, 75)$), we would probably assign\n",
"the new data point to be a red triangle as well. This is exactly what our classifier would\n",
"predict for $k=1$:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Predicted label:\t [[ 1.]]\n",
"Neighbor's label:\t [[ 1.]]\n",
"Distance to neighbor:\t [[ 250.]]\n"
]
}
],
"source": [
"ret, results, neighbor, dist = knn.findNearest(newcomer, 1)\n",
"print(\"Predicted label:\\t\", results)\n",
"print(\"Neighbor's label:\\t\", neighbor)\n",
"print(\"Distance to neighbor:\\t\", dist)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, knn reports that the nearest neighbor is 250 arbitrary units away, that the neighbor\n",
"has label 1 (which we said corresponds to red triangles), and that therefore the new data\n",
"point should also have label 1. The same would be true if we looked at the $k=2$ nearest\n",
"neighbors, and the $k=3$ nearest neighbors.\n",
"\n",
"But we want to be careful not to pick arbitrary\n",
"even numbers for $k$. Why is that? Refer to page 64 for the answer."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, what would happen if we dramatically widened our search window and classified\n",
"the new data point based on its $k=7$ nearest neighbors (circled with a solid line in the figure\n",
"mentioned earlier)?\n",
"\n",
"Let's find out by calling the `findNearest` method with $k=7$ neighbors:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Predicted label:\t [[ 0.]]\n",
"Neighbor's label:\t [[ 1. 1. 0. 0. 0. 1. 0.]]\n",
"Distance to neighbor:\t [[ 250. 401. 784. 916. 1073. 1360. 4885.]]\n"
]
}
],
"source": [
"ret, results, neighbor, dist = knn.findNearest(newcomer, 7)\n",
"print(\"Predicted label:\\t\", results)\n",
"print(\"Neighbor's label:\\t\", neighbor)\n",
"print(\"Distance to neighbor:\\t\", dist)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Suddenly, the predicted label is 0 (blue square). The reason is that we now have four\n",
"neighbors within the solid circle that are blue squares (label 0), and only three that are red\n",
"triangles (label 1). So the majority vote would suggest making the newcomer a blue square\n",
"as well.\n",
"\n",
"For $k=6$, there is a tie:"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Predicted label:\t [[ 0.]]\n",
"Neighbors' labels:\t [[ 1. 1. 0. 0. 0. 1.]]\n",
"Distance to neighbors:\t [[ 250. 401. 784. 916. 1073. 1360.]]\n"
]
}
],
"source": [
"ret, results, neighbors, dist = knn.findNearest(newcomer, 6)\n",
"print(\"Predicted label:\\t\", results)\n",
"print(\"Neighbors' labels:\\t\", neighbors)\n",
"print(\"Distance to neighbors:\\t\", dist)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternatively, predictions can be made with the `predict` method. But first, need to set `k`:"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(0.0, array([[ 0.]], dtype=float32))"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"knn.setDefaultK(7)\n",
"knn.predict(newcomer)"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(0.0, array([[ 0.]], dtype=float32))"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"knn.setDefaultK(6)\n",
"knn.predict(newcomer)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you can see, the outcome of $k$-NN changes with the number $k$. However, often we do not\n",
"know beforehand which number $k$ is the most suitable. A naive solution to this problem is\n",
"just to try a bunch of values for $k$, and see which one performs best. We will learn more\n",
"sophisticated solutions in later chapters of this book."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"< [Measuring-Model-Performance-with-Scoring-Functions](03.01-Measuring-Model-Performance-with-Scoring-Functions.ipynb) | [Contents](../README.md) | [Using Regression Models to Predict Continuous Outcomes](03.03-Using-Regression-Models-to-Predict-Continuous-Outcomes.ipynb) >"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.3"
}
},
"nbformat": 4,
"nbformat_minor": 1
}