{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Logistic Regression With Linear Boundary Demo\n", "\n", "_Source: 🤖[Homemade Machine Learning](https://github.com/trekhleb/homemade-machine-learning) repository_\n", "\n", "> ☝Before moving on with this demo you might want to take a look at:\n", "> - 📗[Math behind the Logistic Regression](https://github.com/trekhleb/homemade-machine-learning/tree/master/homemade/logistic_regression)\n", "> - ⚙️[Logistic Regression Source Code](https://github.com/trekhleb/homemade-machine-learning/blob/master/homemade/logistic_regression/logistic_regression.py)\n", "\n", "**Logistic regression** is the appropriate regression analysis to conduct when the dependent variable is dichotomous (binary). Like all regression analyses, the logistic regression is a predictive analysis. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables.\n", "\n", "Logistic Regression is used when the dependent variable (target) is categorical.\n", "\n", "For example:\n", "\n", "- To predict whether an email is spam (`1`) or (`0`).\n", "- Whether online transaction is fraudulent (`1`) or not (`0`).\n", "- Whether the tumor is malignant (`1`) or not (`0`).\n", "\n", "> **Demo Project:** In this example we will try to classify Iris flowers into tree categories (`Iris setosa`, `Iris virginica` and `Iris versicolor`) based on `petal_length` and `petal_width` parameters." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# To make debugging of logistic_regression module easier we enable imported modules autoreloading feature.\n", "# By doing this you may change the code of logistic_regression library and all these changes will be available here.\n", "%load_ext autoreload\n", "%autoreload 2\n", "\n", "# Add project root folder to module loading paths.\n", "import sys\n", "sys.path.append('../..')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import Dependencies\n", "\n", "- [pandas](https://pandas.pydata.org/) - library that we will use for loading and displaying the data in a table\n", "- [numpy](http://www.numpy.org/) - library that we will use for linear algebra operations\n", "- [matplotlib](https://matplotlib.org/) - library that we will use for plotting the data\n", "- [logistic_regression](https://github.com/trekhleb/homemade-machine-learning/blob/master/homemade/logistic_regression/logistic_regression.py) - custom implementation of logistic regression" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Import 3rd party dependencies.\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "# Import custom logistic regression implementation.\n", "from homemade.logistic_regression import LogisticRegression" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Load the Data\n", "\n", "In this demo we will use [Iris data set](http://archive.ics.uci.edu/ml/datasets/Iris).\n", "\n", "The data set consists of several samples from each of three species of Iris (`Iris setosa`, `Iris virginica` and `Iris versicolor`). Four features were measured from each sample: the length and the width of the sepals and petals, in centimeters. Based on the combination of these four features, [Ronald Fisher](https://en.wikipedia.org/wiki/Iris_flower_data_set) developed a linear discriminant model to distinguish the species from each other." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
sepal_lengthsepal_widthpetal_lengthpetal_widthclass
05.13.51.40.2SETOSA
14.93.01.40.2SETOSA
24.73.21.30.2SETOSA
34.63.11.50.2SETOSA
45.03.61.40.2SETOSA
55.43.91.70.4SETOSA
64.63.41.40.3SETOSA
75.03.41.50.2SETOSA
84.42.91.40.2SETOSA
94.93.11.50.1SETOSA
\n", "
" ], "text/plain": [ " sepal_length sepal_width petal_length petal_width class\n", "0 5.1 3.5 1.4 0.2 SETOSA\n", "1 4.9 3.0 1.4 0.2 SETOSA\n", "2 4.7 3.2 1.3 0.2 SETOSA\n", "3 4.6 3.1 1.5 0.2 SETOSA\n", "4 5.0 3.6 1.4 0.2 SETOSA\n", "5 5.4 3.9 1.7 0.4 SETOSA\n", "6 4.6 3.4 1.4 0.3 SETOSA\n", "7 5.0 3.4 1.5 0.2 SETOSA\n", "8 4.4 2.9 1.4 0.2 SETOSA\n", "9 4.9 3.1 1.5 0.1 SETOSA" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Load the data.\n", "data = pd.read_csv('../../data/iris.csv')\n", "\n", "# Print the data table.\n", "data.head(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plot the Data\n", "\n", "Let's take two parameters `petal_length` and `petal_width` for each flower into consideration and plot the dependency of the Iris class on these two parameters." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEXCAYAAACpuuMDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl4VeW1x/HvIiBBBKkEC4JIBAEZNGjqhANK61AoWK+oaK1oW3rrtdqrpYPSihW1DtfZW0sdkCsXFScsVKwXaYtW0DAosyMKSBVxYBQhWfePsxNzTk6SfXLmnN/nefIk593v3nudoGdl7/2u9zV3R0REpFqLbAcgIiK5RYlBRESiKDGIiEgUJQYREYmixCAiIlGUGEREJIoSgxQ8MzvPzP6a7ThEcoUSgzR7ZrbGzL5Z33Z3n+ruJyd4zOVmtjX4qjSzL2q9vjL5qEWyp2W2AxDJJjNr6e67E93P3fvXOsbfgIfd/b5UxiaSLbpikIJiZmPM7CUzu83MNgETgrYXg+0WbPvIzDab2VIzG5DgOYrN7DMzO7hWWxcz225mHc3sm8FVzG/NbJOZvWtm58Tsf6uZrTWzD83sv82sONi2r5n9JTj+J2b2jxT9akRqKDFIIToSeAf4OnBdzLaTgeOB3sDewFnApkQO7u5fAI8B36vVfC7wnLtXH6sb0A7YD/gB8ICZ9Qq23QyUAocABwE9gKuCbeOC2DsBnYHxicQmEoYSgxSiD9z9Lnff7e47YrbtIvKB3Rcwd1/p7huacI6HgHPNzILX5wP/U2t7FXC1u+909xeA2cAoM2sB/Aj4mbt/6u6bgRuA6iuKXUSSSXd3/9LddcUgKafEIIVobX0bgg/pu4F7gI/MbJKZtU/0BO7+ErAbODa4FdUdmFWryyZ3317r9XtEPvA7A62B14LbRZ8BM4F9g36/D/rOMbO3zWxcorGJNEaJQQpRg1MKu/ud7n440I/ILaWmfvhOIXI76XzgMXffWWtbRzNrU+t1d+AD4EPgS6CPu3cIvvZ2972D2Da7+3+6ew/gdOCXZnZCE+MTiUuJQaQWM/uGmR1pZq2AbcAXRG77NMX/AGcSeb4wJWZbCyIPvvcwsyHAacDj7l4J3Afcbmadgofh3czs5CC+75hZz+AW1edAZRLxicSlxCASrT3wJ+BTIrdsNhF5GJwwd18DLAV2uvs/YzavI5J4NhB5HvFDd38z2HZFcO5XiHz4/5XIQ2iAPsALwFbgJeAOd5/XlPhE6mNaqEckfcxsCvCOu0+o1fZN4L7gdpBIzlGBm0iamNmBwEhgYLZjEUmEbiWJpIGZ3QC8Blzv7u9nOx6RROhWkoiIRNEVg4iIRMnLZwwlJSXeo0ePbIchIpJXFi5c+LG7d2qsX14mhh49elBRUZHtMERE8oqZvRemn24liYhIFCUGERGJosQgIiJR8vIZQzy7du1i3bp1fPHFF9kOpeAVFxfTrVs3WrVqle1QRKQJmk1iWLduHe3ataNHjx58NQW+ZJq7s2nTJtatW0dpaWm2wxGRJmg2t5K++OILOnbsqKSQZWZGx44ddeUmkseaTWIAlBRyhP4dRPJbWm8lmdn+ROah/zqRxVEmufsdMX2GADOAd4OmJ939d+mMS0Tyz6x3ZnHHojv417Z/0bltZy477DKGHTgsqf2BpI7ZXKX7GcNu4Ap3X2Rm7YCFZva8u6+I6TfP3YenOZaMuO666/jf//1fioqKaNGiBX/84x/55S9/yYYNG2jTJrJgV69evRg0aBDTp08HYOnSpQwcGJmA86KLLuLSSy9l0qRJ3HrrrQC0b9+eW2+9lWOPPRaAmTNn8pvf/Iaqqip27drFZZddxo9//OOaGMrKyujbty+PPPJIJt+6SNrMemcWE/45gS8qI7coN2zbwIR/TgAI9UEeb//xL47HzNhVtatJx2zO0poYgkXUNwQ/bzGzlUBXIDYxNAsvv/wyM2fOZNGiRbRu3ZqPP/6YL7/8EoCpU6dSXl4e1f+qq64CYK+99mLJkiU17TNnzuSPf/wjL774IiUlJSxatIjTTz+dV155hY4dOzJ27FheeeUVunXrxs6dO1mzZk3NvitXrqSyspJ58+axbds22rZtm/43LpJmdyy6o+ZDvdoXlV9wx6I7Qn2Ix9t/t++us8hrIsdszjL2jMHMegCDgAVxNh9tZq+Z2bNm1r+e/ceaWYWZVWzcuDHpeJ5evJ7Bv3+B0l/NYvDvX+DpxeuTPuaGDRsoKSmhdevWAJSUlLDffvslfJwbb7yRm2++mZKSEgAOO+wwLrjgAu655x62bNnC7t276dixIwCtW7emT58+NftOmzaN888/n5NPPpkZM2Yk/Z5EcsG/tv0rofam9ku0b3OVkcRgZnsBTwA/c/fNMZsXAQe4+6HAXcDT8Y7h7pPcvdzdyzt1anQOqAY9vXg9v35yKes/24ED6z/bwa+fXJp0cjj55JNZu3YtvXv35uKLL+bvf/97zbbzzjuPsrIyysrKGDeu4bXlly9fzuGHHx7VVl5ezvLly9lnn30YMWIEBxxwAKNHj2bq1KlUVX215O+jjz7KOeecw+jRo5k2bVpS70ckV3Ru2zmh9qb2S7Rvc5X2xBAsqv4EMNXdn4zd7u6b3X1r8PNfgFZmVpLOmG5+bjU7dlVGte3YVcnNz61O6rh77bUXCxcuZNKkSXTq1Imzzz6byZMnA5FbSUuWLGHJkiXcfHOTlhCucd999zFnzhyOOOIIbrnlFi666CIAKioqKCkpoXv37gwdOpTFixfzySefJHUukVxw2WGXUVxUHNVWXFRc8wC5Kfu3tJa0ahFdhJnIMZuzdI9KMuB+YKW731pPn87Ah+7uZnYEkWS1KZ1xffDZjoTaE1FUVMSQIUMYMmQIAwcO5KGHHkr4GP369WPhwoWcdNJJNW0LFy6kf/+v7rINHDiQgQMHcv7551NaWsrkyZOZNm0aq1atonpK8s2bN/PEE0/wox/9KOn3JZJN1ff8mzqCqL79kzlmc5buUUmDgfOBpWZW/XT1SqA7gLvfC5wJ/MTMdgM7gHM8zcvK7dehDevjJIH9OrRJ6rirV6+mRYsWHHTQQQAsWbKEAw44gGXLliV0nF/84hf88pe/ZPbs2XTs2JElS5YwefJkFixYwNatW6moqGDIkCFR56iqquKxxx5j6dKlNc815s6dy7XXXqvEIM3CsAOHJfWhXd/+SgR1pXtU0otAg9VO7n43cHc644g17pQ+/PrJpVG3k9q0KmLcKX0a2KtxW7du5ac//SmfffYZLVu2pFevXkyaNIkzzzyT8847r2a4aklJCf/3f/9X73FGjBjB+vXrOeaYYzAz2rVrx8MPP0yXLl3YsmULN910Ez/+8Y9p06YNbdu2ZfLkycybN4+uXbtGPew+/vjjWbFiBRs2bKBLly5JvTcRKRx5ueZzeXm5xy7Us3LlSg4++ODQx3h68Xpufm41H3y2g/06tGHcKX04fVDXVIdasBL995D8kWyhWRgT509k+hvTqfIqWlgLRvUexfijxqf0HIXIzBa6e3lj/ZrNJHqJOn1QVyUCkQQlW2gWxsT5E3l09aM1r6u8qua1kkNmNKu5kkQkvRoqNEuV6W9MT6hdUk+JQURCS7bQLIwqr0qoXVJPiUFEQku20CyMFhb/Y6m+dkk9/aZFJLRkC83CGNV7VELtknoF+/BZRBKXbKFZGNUPmDUqKYvcPe++Dj/8cI+1YsWKOm2ZNGTIEJ89e3ZU22233eannnqqFxcX+6GHHlrz9dBDD7m7+wEHHOADBgzwgQMH+vHHH+9r1qyp2XfixIner18/HzhwoB966KE+f/58d3c/4YQT/NVXX3V39y1btvjYsWP9wAMP9MMOO8xPOOGEmn5r1671ESNGeK9evfzAAw/0Sy+91Hfu3Onu7nPnzvVhw4bVeQ87d+70yy67zHv27Om9evXyESNG+Nq1a2u2t2jRwg899FDv37+/Dx8+3D/99NN6fx/Z/vcQkbqACg/xGasrhhQZPXo0jzzyCKecckpN2yOPPMJNN93E2rVro6bVrm3u3LmUlJRw9dVXM3HiRP70pz81OH13bT/84Q8pLS3lzTffpEWLFrz77rusWLECd+eMM87gJz/5CTNmzKCyspKxY8dy1VVXNThP05VXXsmWLVtYvXo1RUVFPPjgg5xxxhksWLAAM6NNmzY176N6ttfqqcOlcIRd8CZsW9irjbD1E4nUWWSiJiNZ2YixcBPD64/BnN/B5+tg724w9LdwyFlNPtyZZ57J+PHj+fLLL9ljjz1Ys2YNH3zwAfvvv3+o/Y8++mjuvPNOIP703bHefvttFixYwNSpU2nRIvKoqLS0lNLSUubMmUNxcTEXXnghEJm/6bbbbqO0tJRrrrkm7vm3b9/Ogw8+yLvvvktRUREAF154IQ888AAvvPACQ4cOrRPv66+/Huq9SfMRdsGb37z0G9w9suZBA/3C1kCErZ9IpM4iEzUZycpWjIX58Pn1x+DPl8LnawGPfP/zpZH2Jtpnn3044ogjePbZZ4HI1cJZZ52FmfH222/XTLldVlbGvHnz6uw/e/ZsTj/9dKDh6burLV++nLKyspoP8dhtsdN2t2/fnu7du/PWW2/Fjf+tt96ie/futG/fPqq9errv2iorK5kzZw4jRoxo4DcizVF9C95Uf9hX21W1qyYpNNQvbA1E2PqJROosMlGTkaxsxViYiWHO72BXzCR6u3ZE2pNQfTsJIolh9OjRAPTs2bNmyu0lS5Zw3HHH1exz4okn0rVrV5599tma/g1N351NO3bsoKysjM6dO/Phhx/yrW99K9shSYalYxGbMMcMWz+RSJ1FJmoykpWtGAszMXy+LrH2kEaOHMmcOXNYtGgR27dvr/NXezxz587lvffeo6ysjKuvvrqmvXr67muuuYa7776bJ554Imq//v3789prr1FZWRl7yJppu2vbvHkz77//Pr169YobR8+ePXn//ffZsmVLVHvt6b6rnzG89957uDv33HNPo+9Pmpd0LGIT5phh6ycSqbPIRE1GsrIVY2Emhr27JdYe0l577cWJJ57IRRddVPPXfxgtW7bk9ttvZ8qUKXzyySesXr2aN998s2Z79dTatfXs2ZPy8nKuvvpqPJgIcc2aNcyaNYuhQ4eyfft2pkyZAkRu/VxxxRWMGTOGPffcM24Mbdu25YILLuDyyy+vSTZTpkxh+/btUetCAOy5557ceeed/Nd//Re7d++OdzhppsIueNOqRStaWstG+4WtgQhbP5FInUUmajKSla0YCzMxDP0ttIpZe6FVm0h7kkaPHs1rr70WlRhinzFUP2SurUuXLowePZp77rmHrVu3csEFF9CvXz8OOeQQVqxYwYQJE+rsc9999/Hhhx/Sq1cvBgwYwJgxY9h3330xM5566immT5/OQQcdRO/evSkuLub666+v2XfOnDl069at5uvll1/mhhtuoLi4mN69e3PQQQcxffp0nnrqKSLrLUUbNGgQhxxyiJYPLTDDDhzGhGMm0KVtFwyjS9suTDx2ItcOvjaq7drB1zLx2ImN9ptwzIRQD1HjnTfevmH7Jdo3W7IVY8FOu53qUUkSTdNui+QeTbvdmEPOUiIQyVHpqFnIJbked+EmBhHJSemoWcgl+RB3YT5jEJGclY6ahVySD3ErMYhITklHzUIuyYe4lRhEJKeko2Yhl+RD3EoMIpJT0lGzkEvyIW4lhhQ58cQTee6556Labr/9dk477TQGDBgAwN/+9jf23ntvysrK6Nu3Lz//+c+j+s+ePZsjjjiCvn37UlZWxtlnn837778PwJgxY3j88ccBGDJkCOXlX404q6ioYMiQITXnGD58eM22Z599lvLycvr168egQYO44ooros5ZVlbGOeeck5pfgkgKpKNmIZfkQ9walZQiDU27ffHFF9e0HXfcccycOZMdO3YwaNAgvvvd7zJ48GCWLVvGT3/6U5555pma8f/PPPMMa9asoXv37nXO99FHH/Hss89y2mmn1RvTsmXLuOSSS5g1axZ9+/alsrKSSZMm1WxfuXIllZWVzJs3j23bttG2bdtU/CpEkjbswGGhC99y6QM1rFyPu2ATQ6rHESc67XabNm0oKytj/fr1ANx4441ceeWVUUVhDc1eOm7cOK677roGE8NNN93EVVddRd++fYHI/Es/+clParZPmzaN888/n5UrVzJjxgzOPffchN6zSKIysZZDrtUI5Fo8YRRkYkjHOOLa026PHDkyatrteD799FPefPNNjj/+eCAyVXbsraWGHH300Tz11FPMnTuXdu3axe2zbNmyOreOanv00Ud5/vnnWbVqFXfddZcSg6RVvP/vwq7bEK9fPtQ25Fo8YRXkM4Z0jSOub9rt2ubNm8ehhx5K165dOeWUU+jcue5IhE2bNlFWVkbv3r255ZZb6j3f+PHjmThxYpNiraiooKSkhO7duzN06FAWL17MJ5980qRjiYQR7/+7sOs2xOuXD7UNuRZPWAWZGNI1jjjMtNvHHXccr732GsuXL+f++++vWSqzf//+LFq0CICOHTuyZMkSxo4dy9atW+s930knncSOHTuYP39+3O39+/evM/12tWnTprFq1Sp69OhBz5492bx5c52pvUVSKRNrOeRajUCuxRNWQSaGdI0jTmTa7dLSUn71q19x4403AvCLX/yC6667jpUrV9b02b59e6PnHD9+PDfddFPcbePGjeP666/njTfeAKCqqop7772XqqoqHnvsMZYuXcqaNWtYs2YNM2bM0EypklaZWMsh12oEci2esAoyMaRzHHG8abfr8+///u/84x//YM2aNQwcOJA77riD73//+/Tp04fBgwezcuXKRu/7f/vb36ZTp05xtx1yyCHcfvvtjB49moMPPpgBAwbwzjvvMG/ePLp27cp+++1X0/f4449nxYoVbNiwIbE3LBJSvP/vwq7bEK9fPtQ25Fo8YRXstNv5OFIgn2jabYlHo5KyG0/YabcLNjFIeunfQyT35MR6DGa2PzAF+DrgwCR3vyOmjwF3AN8GtgNj3H1ROuMSkWhh/5IfduCwhPo2Z8lcCeTSVUQ8ab1iMLMuQBd3X2Rm7YCFwOnuvqJWn28DPyWSGI4E7nD3Ixs6bn1XDH379q23bkAyx91ZtWqVrhjyROxYe4jc069dNwCRe+Mje41kxlszovq2tJZRdQfVfXNtmodUivc7C/uek9k3WWGvGNL68NndN1T/9e/uW4CVQNeYbiOBKR4xH+gQJJSEFBcXs2nTJvLx1lhz4u5s2rSJ4uLixjtLTghbX/BF5RdMf2N6nb7x6g7yYax+MpKpT8iH2oaMVT6bWQ9gELAgZlNXYG2t1+uCtqjhMWY2FhgLxJ07qFu3bqxbt46NGzemLGZpmuLiYrp165btMCSkRMbUV3lVWo6bb5KpT8iH2oaMJAYz2wt4AviZu29uyjHcfRIwCSK3kmK3t2rVitLS0qTiFClEndt2ZsO2cMOUW1iL0Mkh18fqJ6O+31mY95zMvpmS9joGM2tFJClMdfcn43RZD9Seaa5b0CYiGRC2vqC4qJhRvUfV6Ruv7iAfxuonI5n6hHyobUj3qCQD7gdWuvut9XR7BrjEzB4h8vD5c3dXlZVIhlQ/8Aw70mjQvoMKflRSfb+zsFOFN3XfTEn3qKRjgXnAUqD6+vNKoDuAu98bJI+7gVOJDFe90N0r4hyuRrxRSSIi0rCcqGNw9xeBBsePeiQz/Uc64xCRxNU31n7i/IlMf2M6VV5FC2vBqN6jGH/U+ND755J8iDEbmk3ls4ikTn1j7cs6lTH/X3Vn8z27z9lRySGbY/XDyocYUy0n6hhEJD/VN9Y+XlIAmP7G9FD759JY/XyIMVuUGESkjkTH1McOYc2Hsfr5EGO2KDGISB2JjqlvYdEfJfmwDkE+xJgtSgwiUkd9Y+2P6nxU3P6jeo8KtX8ujdXPhxizJWNTYohI/mhorH2YUUn5MFY/H2LMFo1KEhEpEDlRxyAiuWfizDFM/7iCKiL3kkeVlENJr1C1CZD6sf/xrkDiVVcnu1pbc14/IdV0xSBSQCbOHMOjH1dA7XVL3KNfB2JrEyD1Y/8nzp/Io6sfrdNuGM5Xn03xzpFILPm6fkKqqY5BROqYHpsUIG5SgLq1CZD6sf/xzgFEJYX6zpFILM19/YRUSygxmFlbMytKVzAikl7hV1OIv/ZCqsf+J7O+QyKxNPf1E1KtwcRgZi3M7Fwzm2VmHwGrgA1mtsLMbjazXpkJU0RSIZG/BGNrEyD1Y//jnaM+sedIJJZk4i7EeofG/lXmAj2BXwOd3X1/d98XOBaYD9xoZt9Lc4wikiKjSsojzxRqq+c5Y2xtAqR+7H+8c0DkGUNj50gklua+fkKqNTYq6Zvuviu20d0/IbL4zhPBQjwikgfGD58MSYxKSvXY/+pzNGVUUiKxNPf1E1It9KgkM/sakZXWapKJuy9KU1wN0qgkEZHEpbSOwcyuBcYAb0PNcAEHTmpqgCLSdBkZV//6YzDnd/D5Oti7Gwz9LRxyVmrPITkpbIHbWUBPd/8yncGISONix9Vv2LaBCf+cAJC65PD6Y/DnS2HXjsjrz9dGXoOSQwEIOyRgGdAhnYGISDgZGVc/53dfJYVqu3ZE2qXZC3vFcAOw2MyWATurG919RFqiEpF6ZWRc/efrEmuXZiVsYngIuBFYSmI1MiKSYp3bdmbDtg1x21Nm726R20fx2qXZC3srabu73+nuc93979VfaY1MROLKyLj6ob+FVm2i21q1ibRLsxf2imGemd0APEP0raSsDFcVKWQZGVdf/YBZo5IKUqg6BjObG6fZ3T0rw1VVxyAikriU1jG4+4nJhyQiKRO2xiAdtQg5VN9QaOskZEqoZwxmdr2Zdaj1+mtmNjF9YYlIvaprDD5fC/hXNQavP9a0fuk4dwZU13Ns2LYBx2vqOWa9MyvjsTQ3YR8+n+bun1W/cPdPgW+nJyQRaVDYGoN01CLkUH1DIa6TkClhE0ORmbWufmFmbYDWDfQXkXQJW2OQjlqEHKpvKMR1EjIlbGKYCswxsx+Y2Q+A54nUNohIptVXSxDbHrZfOs6dAYW4TkKmhEoM7n4jMBE4OPi61t1vSmdgIlKPsDUG6ahFyKH6hkJcJyFTGhyVZGbmwXhWd58NzG6oj4hkQNgag3TUIuRQfUMhrpOQKQ3WMZjZ34gsyDPD3d+v1b4HkVXcLgDmuvvk9IYZTXUMIiKJS1Udw6nARcA0MysFPgPaELkF9Vfgdndf3EAQDwDDgY/cfUCc7UOAGcC7QdOT7q7pG6X5S7YW4Ja+sLXWfEl7dYGTf1f3mBDuPDMvh4WTwSvBiuDwMczqNzTuX+OqHWj+ElnBrRVQAuyoPXS1kX2OB7YCUxpIDD939+GhI0ZXDJLnYtc6gMh9+u/cGS45xCaF+rRoBWZQWWsZlXjnmXk5VNwfteustnsyYd99+aLWnJnFRcWM7DWSGW/NiBomWlxUzIRjJig55IGwVwxhRyXh7rvcfUPYpBDs8w/gk7D9RQpCsrUAYZICQNWu6KRQ33kWTq6z6x1f6xCVFCBSIzD9jemqHSgAoRNDGh1tZq+Z2bNm1r++TmY21swqzKxi48aNmYxPJLWyXQsQex6vrNPlXy2L4u5a5fFn3VftQPOS7cSwCDjA3Q8F7gKerq+ju09y93J3L+/UqVPGAhRJuWzXAsSex+omgc676yYLgBYW/yNDtQPNS1YTg7tvdvetwc9/AVqZWUk2YxJJu2RrAfbqEq5fi1ZQtEfj5zl8TJ1dL/v0M4pjPh6Ki4oZ1XuUagcKQNhJ9M4wszfN7HMz22xmW8xsc7InN7POZmbBz0cE8WxK9rgiOe2QsyIPgPfeH7DI97APngF+vqpuctirC5zxp+hjnv7fMPKexs8z/FYo/8FXVw5WxLCDRzPhuOvp0rYLhtGlbRcmHDOB8UeNZ8IxE+q068Fz8xJ2PYa3gO+4+8qEDm42DRhCZDTTh8DVQCsAd7/XzC4BfgLsBnYAl7v7Pxs7rkYliYgkLqXrMQAfJpoUANx9dCPb7wbuTvS4IiKSPo1NiXFG8GOFmT1K5OFw7aU9n0xjbCK5LVML1sQpPmP4reHieX9+3X27H5UTU1o0REV02dXYlBgPNrCvu/tFqQ+pcbqVJFmXbJFaWHGKz4DIM4HaySFePC2KoCrO6KLY9nTEnYTqBXhURJd6YW8lhX3GMNjdX2qsLVOUGCTrbhsQrGIWY+/94T+Xpe481+wTt84AK4Kra9WO1hdPWKmOOwknP34yG7bVLeLr0rYLfz3zr1mIqPlIdeXzXSHbRApDporU4iWFeO3JnjcLC+3URwvwZF9jzxiOBo4BOpnZ5bU2tQfil0aKFIK9u9VzxZDiIjUrqv+KIUw8YWVhoZ36dG7bOe4Vg4roMqexK4Y9gL2IJJB2tb42A2emNzSRHJapBWviFJ/FbY8XT4t6/naLbc/SQjv10QI82dfgFYO7/x34u5lNdvf3MhSTSO7L1II11Q+YGxuVVF88eTgqSQvwZF9jo5L+DNTbwd1HpCOoxujhs4hI4lJV4HZL8P0MoDPwcPB6NJFKZpH8lupahIdGwLt//+p16QnQsVfdv9oh/lVAvJqFeH/hQ9MX5RFpRNjhqhWxWSZeW6boikFSItW1CLFJIVElfeHjVXXbYx9AF+0B7pH1FqqFXZRHClqqh6u2NbMDax28FGjb1OBEckKyC+bESiYpQPykAHVHJVV+GZ0UIPyiPCIhhJ0r6T+Bv5nZO4ABBwA/TltUIpmQ7QVzMqE5vRfJmFCJwd1nm9lBQN+gaZW772xoH5Gcl6lahGxqTu9FMqbBW0lmdlLw/QxgGNAz+BpWa4I9kfyU6lqE0hOSi6ekb/z22GK2oj0izxRqC7soj0gIjT1jqP4v/TtxvoanMS6R9Et2wZxYFzxTNzmUnlBnERzKfxC/7ZIF8du/e290jCPviSzC05RFeURCCDUqKddoVJKISOJSulCPmb0NzAfmAfPcfXmS8Ynkv3g1EBCuliCR+olkai0ytWaENCth6xhaA0cCxwGDgT7A6+7+3fSGF5+uGCTr4q5/ELIOm7DQAAAR6ElEQVSWIJH6iWRqLTK1ZoTkjVTXMVQCu4LvVcBHwZdIYYpXAxG2liCR+olkai1SXachBSNsHcNmYClwK/And9+UvpBE8kAi9QGxfROpn0im1qIQ6jQkLcJeMYwG/gFcDDxiZteY2dD0hSWS4xKpD4jtW9++8doT6ZvKfaWghUoM7j7D3ccRqXb+CzAGmJnGuERyW9z1D0LWEiRSP5FMrUWm1oyQZidUYjCzJ8zsLeAOYE/g+8DX0hmYSE6LVwMRtpYgkfqJZGotUl2nIQUj7KikcmCxe/wFaM3sW+7+fKqDq49GJYmIJC6ldQzu3tin8I1AxhKDNDP5MNY+mZoFkTwTdlRSYyxFx5FCEzvW/vO1kdeQOx+y8WKc8R/RayLkYtwiTRR2VFJj8m9eDckN+TDWPl6M8dZEyLW4RZooVYlBpGnyYax9MjULInkoVYlhTYqOI4UmH8baJ1OzIJKHGnzG0NiaC+7+ZPBdazNI0wz9bfz5fHJprH28GOOtu5xrcYs0UWMPn7/TwDYHnkxhLFKIqh/U5vLonvpijNeWS3GLNJHWYxARKRAprWMIDjgM6A8UV7e5e4NDMMzsASIrvX3k7gPibDci1dTfBrYDY9x9UdiYRJh5OSycDF4ZWfHs8DEw/Nbk+qZ6/QPQlYXklbAL9dxLZCqME4H7gDOBV0LsOhm4G5hSz/bTgIOCryOBPwTfRRo383KouP+r11751evYD/ywfZOpq4i379MXR6/RoHoHyQNhRyUd4+7fBz5192uAo4Heje3k7v8APmmgy0hgikfMBzqYWZeQMUmhWzg5fHvYvqle/yDsGg0iOSRsYqj+r327me1HZNGeVHyAdwXW1nq9Lmirw8zGmlmFmVVs3LgxBaeWvBd/6q747WH7pmP9g2T7imRY2MQw08w6ADcDi4jULUxLV1DxuPskdy939/JOnTpl8tSSq6wofHvYvulY/yDZviIZFjYx3OTun7n7E8ABQF9gYgrOvx7Yv9brbkGbSOMOHxO+PWzfVK9/EHaNBpEcEjYxvFz9g7vvdPfPa7cl4Rng+xZxFPC5u29IwXGlEAy/Fcp/8NVf/VYUeR1vpFHYvqle/yDsGg0iOaTBOgYz60zknv/DwLl8NYtqe+Bed+/b4MHNpgFDgBLgQ+BqoBWAu98bDFe9GziVyHDVC0NM8a06BhGRJkhVHcMpRJbx7AbU/tNqM3BlYwd399GNbHfgPxo7joiIZE6DicHdHwIeMrN/C54viIhIMxf2GcNLZna/mT0LYGb9zOwHaYxLRESyJGxieBB4DtgveP0G8LO0RCQiIlkVNjGUuPtjQBWAu+8G6qkYEhGRfBY2MWwzs44ES3hWDy1NW1QiIpI1YWdXvZxIzUFPM3sJ6ERkIj0REWlmQiUGd19kZicAfYjUMqx2912N7CYiInko7LTbxcDFwLFEbifNM7N73f2LdAYnIiKZF/ZW0hRgC3BX8Ppc4H+AUekISkREsidsYhjg7v1qvZ5rZivSEZCIiGRX2FFJi4KRSACY2ZGAJisSEWmGwl4xHA7808zeD153B1ab2VIiUx4dkpboREQk48ImhlPTGoWIiOSMsMNV30t3ICIikhvCPmMQEZECocQgIiJRlBhERCSKEoOIiERRYhARkShKDCIiEkWJQUREoigxiIhIFCUGERGJEnZKDEnC04vXc/Nzq/ngsx3s16EN407pw+mDumY7LBGRuJQY0uzpxev59ZNL2bGrEoD1n+3g108uBVByEJGcpFtJaXbzc6trkkK1Hbsqufm51VmKSESkYUoMafbBZzsSahcRyTYlhjTbr0ObhNpFRLJNiSHNxp3ShzatiqLa2rQqYtwpfbIUkYhIw/TwOc2qHzBrVJKI5Aslhgw4fVBXJQIRyRtpv5VkZqea2Woze8vMfhVn+xgz22hmS4KvH6Y7plzx9OL1DP79C5T+ahaDf/8CTy9en+2QRETSe8VgZkXAPcC3gHXAq2b2jLuviOn6qLtfks5Yco3qG0QkV6X7iuEI4C13f8fdvwQeAUam+Zx5QfUNIpKr0p0YugJra71eF7TF+jcze93MHjez/eMdyMzGmlmFmVVs3LgxHbFmlOobRCRX5cJw1T8DPdz9EOB54KF4ndx9kruXu3t5p06dMhpgOqi+QURyVboTw3qg9hVAt6Cthrtvcvedwcv7gMPTHFNOUH2DiOSqdCeGV4GDzKzUzPYAzgGeqd3BzLrUejkCWJnmmHLC6YO6csMZA+naoQ0GdO3QhhvOGKgHzyKSdWkdleTuu83sEuA5oAh4wN2Xm9nvgAp3fwa41MxGALuBT4Ax6Ywpl6i+QURykbl7tmNIWHl5uVdUVGQ7DBGRvGJmC929vLF+qnwOKexiO+f96WVeevuTmteDe+7DqPLucfcNe0wt9CMimaQrhhBii9Eg8qA49plAbFKoZkDt33KbVkX82+FdeWLh+kaPGfbcIiKNCXvFkAvDVXNe2GK0eEkBopNC9b7TFqwNdUwVwolIpikxhJCOYrTKeq7UYo+pQjgRyTQlhhDSUYxWZBbqmCqEE5FMU2IIIWwx2uCe+8TdPzYFtGlVxOgj9w91TBXCiUimKTGEELYYbeqPjq6THAb33Ifbzi6rs+/E0weGOqYK4UQk0zQqSUSkQKiOIcXGP72UaQvWUulOkRmjj9yfdzduDV2zEI/qE0QkF+mKIYTxTy/l4fnvh+obr2Yh3q0f1SeISKapjiGFpi1Y23inQLyahXg1B6pPEJFcpcQQQn01B2HFqzlQfYKI5ColhhDqqzkIK17NgeoTRCRXKTGEMPrIuKuNxhWvZiFezYHqE0QkVykxhDDx9IF876juNVcORWZ876juoWsW4j1MVn2CiOQqjUoSESkQqmOII2zdQLyahQXvbOLNj7bV9Dlo37a8u3Ebu2vl1ZYGe+5RxOadX402at+6iDZ7FPHhli9r2r7ebg8WXPUtrccgIjmpYK4YwtYNJFKzkIz2rYvYVYXWYxCRjFEdQ4ywdQOJ1CwkY/POSq3HICI5qWASQ9i6gWRrFpKl9RhEJNsKJjGErRtItmYhWVqPQUSyrWASQ9i6gURqFpLRvnWR1mMQkZxUMIkhbN1AfTULB+3bNqrfQfu2pWXMxUVLi3zg19a+dRFfb7dHVNvX2+3B69ecqvUYRCQnFcyoJBGRQqc6hpCSqRGIt+89c9+sU+/w/OVD0hS9iEjqFfQVQzI1AvH2rY+Sg4jkAtUxhJBMjUC8fetT+wpCRCTXFXRiSKZGQHUEItJcFXRiSKZGQHUEItJcFXRiSKZGIN6+9Ykd6ioikssKOjEkUyMQb9/bzy6LW++gB88ikk8KelSSiEghyZlRSWZ2qpmtNrO3zOxXcba3NrNHg+0LzKxHumMSEZH6pTUxmFkRcA9wGtAPGG1m/WK6/QD41N17AbcBN6YzJhERaVi6rxiOAN5y93fc/UvgEWBkTJ+RwEPBz48DQ82yPMWpiEgBS3di6ArUXvlmXdAWt4+77wY+BzrGHsjMxppZhZlVbNy4MU3hiohI3oxKcvdJ7l7u7uWdOnXKdjgiIs1WuhPDeqD2Agfdgra4fcysJbA3sCnNcYmISD3SPbvqq8BBZlZKJAGcA5wb0+cZ4ALgZeBM4AVvZAztwoULPzaz95KIqwT4OIn9c4neS27Se8ldzen9JPpeDgjTKa2Jwd13m9klwHNAEfCAuy83s98BFe7+DHA/8D9m9hbwCZHk0dhxk7qXZGYVYcby5gO9l9yk95K7mtP7Sdd7Sft6DO7+F+AvMW2/rfXzF8CodMchIiLh5M3DZxERyYxCTQyTsh1ACum95Ca9l9zVnN5PWt5LXs6VJCIi6VOoVwwiIlIPJQYREYlSUInBzB4ws4/MbFm2Y0mWme1vZnPNbIWZLTezy7IdU1OZWbGZvWJmrwXv5Zpsx5QsMysys8VmNjPbsSTDzNaY2VIzW2JmeT3XvZl1MLPHzWyVma00s6OzHVNTmFmf4N+j+muzmf0specopGcMZnY8sBWY4u4Dsh1PMsysC9DF3ReZWTtgIXC6u6/IcmgJCyZNbOvuW82sFfAicJm7z89yaE1mZpcD5UB7dx+e7XiayszWAOXunvcFYWb2EDDP3e8zsz2APd39s2zHlYxgBuv1wJHunkzRb5SCumJw938QKaLLe+6+wd0XBT9vAVZSd4LCvOARW4OXrYKvvP2Lxcy6AcOA+7Idi0SY2d7A8UQKanH3L/M9KQSGAm+nMilAgSWG5ipY3GgQsCC7kTRdcOtlCfAR8Ly75+17AW4HfgFUZTuQFHDgr2a20MzGZjuYJJQCG4EHg1t895lZc1iM/RxgWqoPqsSQ58xsL+AJ4Gfuvjnb8TSVu1e6exmRiRaPMLO8vNVnZsOBj9x9YbZjSZFj3f0wIott/UdwOzYftQQOA/7g7oOAbUCdFSXzSXA7bAQwPdXHVmLIY8H9+CeAqe7+ZLbjSYXg8n4ucGq2Y2miwcCI4N78I8BJZvZwdkNqOndfH3z/CHiKyOJb+WgdsK7WlejjRBJFPjsNWOTuH6b6wEoMeSp4YHs/sNLdb812PMkws05m1iH4uQ3wLWBVdqNqGnf/tbt3c/ceRC7zX3D372U5rCYxs7bBwAaC2y4nA3k5os/d/wWsNbM+QdNQIO8GasQYTRpuI0EGJtHLJWY2DRgClJjZOuBqd78/u1E12WDgfGBpcG8e4Mpg0sJ80wV4KBhh0QJ4zN3zephnM/F14Klgpd2WwP+6++zshpSUnwJTg1sw7wAXZjmeJgsS9beAH6fl+IU0XFVERBqnW0kiIhJFiUFERKIoMYiISBQlBhERiaLEICIiUZQYREQkihKDNDtmNsbM9gvRb7KZndnA9r+ZWXmKY+tgZhfXej0k7NTcZnZ7KqakMLNbzOykZI8jzZcSgzRHY4BGE0OWdAAubrRXDDPrCBwVzBCcrLvI83mCJL2UGCSnmVmPYGGVqcHiKo+b2Z7BtsPN7O/BzJ/PmVmX4AqgnEiF6xIza2NmvzWzV81smZlNCqYTSTSOk83sZTNbZGbTg8kLqxeyuSZoX2pmfYP2Tmb2fLDw0H1m9p6ZlQC/B3oGsd0cHH6vWgvITK0nvn8DaqqOzewbZvZPiyxu9IqZtQuulJ4OzrvGzC4xs8uD2UTnm9k+AMEUzR3NrHOivwcpDEoMkg/6AP/t7gcDm4GLgwkE7wLOdPfDgQeA69z9caACOM/dy9x9B3C3u38jWJypDZDQwjnBB/p44JvBTKMVwOW1unwctP8B+HnQdjWReZL6E5mwrXvQ/isi8+eXufu4oG0Q8DOgH3AgkelOYg0mshhT9ayajxJZzOhQ4JvAjqDfAOAM4BvAdcD2YDbRl4Hv1zreonrOI1JYcyVJ3lrr7i8FPz8MXErkr+cBwPPBH9hFwIZ69j/RzH4B7AnsAywH/pzA+Y8i8qH9UnCuPYh80Farntl2IZEPZYBjge8CuPtsM/u0geO/4u7rAIJ5r3oQWcWuti5E1hOASKLc4O6vBsffHOwLMDdYuGmLmX1e630uBQ6pdbyPyN3bbZJlSgySD2In9HLAgOXu3uC6vWZWDPw3keUp15rZBKA4wfMbkcWDRtezfWfwvZKm/T+1s9bP9R1jB+Hirn2sqlqvq2KOW8xXVxkiUXQrSfJBd/tq4fZzifw1vRroVN1uZq3MrH/QZwvQLvi5+sP04+C5QL2jkBowHxhsZr2Cc7U1s96N7PMScFbQ/2Tga3FiS8RKoFfw82qgi5l9Izh+OzNLNCH1Jk+n0Jb0U2KQfLCayOphK4l8wP7B3b8k8iF/o5m9BiwBjgn6TwbuDW7L7AT+RORD8Dng1URP7u4biYx0mmZmrxO5jdS3kd2uAU42s2XAKOBfwBZ330TkltSyWg+fw5hFZMp4gvd+NnBX8N6fJ4GroOD5TC8iz0pE6tC025LTLLKe9czgwXHeMLPWQKW77w6uav4QLF2azDFfBIYnu4i9mX0XOMzdf5PMcaT50jMGkfToDjxmZi2AL4EfpeCYVwTHTSoxEPn//r+SD0eaK10xSMEzs6eA0pjmX7r7c9mIRyTblBhERCSKHj6LiEgUJQYREYmixCAiIlGUGEREJMr/A3XldMGr8dFdAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# List of suppported Iris classes.\n", "iris_types = ['SETOSA', 'VERSICOLOR', 'VIRGINICA']\n", "\n", "# Pick the Iris parameters for consideration.\n", "x_axis = 'petal_length'\n", "y_axis = 'petal_width'\n", "\n", "# Plot the scatter for every type of Iris.\n", "for iris_type in iris_types:\n", " plt.scatter(\n", " data[x_axis][data['class'] == iris_type],\n", " data[y_axis][data['class'] == iris_type],\n", " label=iris_type\n", " )\n", "\n", "# Plot the data. \n", "plt.xlabel(x_axis + ' (cm)')\n", "plt.ylabel(y_axis + ' (cm)')\n", "plt.title('Iris Types')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prepara the Data for Training\n", "\n", "Let's extract `petal_length` and `petal_width` data and form a training feature set and let's also form out training labels set." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# Get total number of Iris examples.\n", "num_examples = data.shape[0]\n", "\n", "# Get features.\n", "x_train = data[[x_axis, y_axis]].values.reshape((num_examples, 2))\n", "# Get labels.\n", "y_train = data['class'].values.reshape((num_examples, 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Init and Train Logistic Regression Model\n", "\n", "> ☝🏻This is the place where you might want to play with model configuration.\n", "\n", "- `polynomial_degree` - this parameter will allow you to add additional polynomial features of certain degree. More features - more curved the line will be.\n", "- `max_iterations` - this is the maximum number of iterations that gradient descent algorithm will use to find the minimum of a cost function. Low numbers may prevent gradient descent from reaching the minimum. High numbers will make the algorithm work longer without improving its accuracy.\n", "- `regularization_param` - parameter that will fight overfitting. The higher the parameter, the simplier is the model will be.\n", "- `polynomial_degree` - the degree of additional polynomial features (`x1^2 * x2, x1^2 * x2^2, ...`). This will allow you to curve the predictions.\n", "- `sinusoid_degree` - the degree of sinusoid parameter multipliers of additional features (`sin(x), sin(2*x), ...`). This will allow you to curve the predictions by adding sinusoidal component to the prediction curve." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Theta 1Theta 2Theta 3
SETOSA30.291853-8.404206-12.313125
VERSICOLOR-2.8589041.546523-3.107221
VIRGINICA-45.3495405.76629410.458614
\n", "
" ], "text/plain": [ " Theta 1 Theta 2 Theta 3\n", "SETOSA 30.291853 -8.404206 -12.313125\n", "VERSICOLOR -2.858904 1.546523 -3.107221\n", "VIRGINICA -45.349540 5.766294 10.458614" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Set up linear regression parameters.\n", "max_iterations = 1000 # Max number of gradient descent iterations.\n", "regularization_param = 0 # Helps to fight model overfitting.\n", "polynomial_degree = 0 # The degree of additional polynomial features.\n", "sinusoid_degree = 0 # The degree of sinusoid parameter multipliers of additional features.\n", "\n", "# Init logistic regression instance.\n", "logistic_regression = LogisticRegression(x_train, y_train, polynomial_degree, sinusoid_degree)\n", "\n", "# Train logistic regression.\n", "(thetas, costs) = logistic_regression.train(regularization_param, max_iterations)\n", "\n", "# Print model parameters that have been learned.\n", "pd.DataFrame(thetas, columns=['Theta 1', 'Theta 2', 'Theta 3'], index=['SETOSA', 'VERSICOLOR', 'VIRGINICA'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Analyze Gradient Descent Progress\n", "\n", "The plot below illustrates how the cost function value changes over each iteration. You should see it decreasing. \n", "\n", "In case if cost function value increases it may mean that gradient descent missed the cost function minimum and with each step it goes further away from it.\n", "\n", "From this plot you may also get an understanding of how many iterations you need to get an optimal value of the cost function. " ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAELCAYAAAA2mZrgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Xl8VOX1+PHPmUnCZCNhCSQhLBHCEhaDRlqrIi5QtRbUUgVtK9WKtl+sW92ta21darULv1ZcqlYLtbZW6oJFpQrWLWzKIrLIEvadhJBt5vz+mJtxEiYLITeTZM779ZpX5j73zr3nOpiT+9z7nEdUFWOMMQbAE+0AjDHGtB2WFIwxxoRYUjDGGBNiScEYY0yIJQVjjDEhlhSMMcaEuJoUROQsEVklImtE5JYI6x8VkSXO6wsR2edmPMYYYxombo1TEBEv8AUwFigGPgEmq+qKera/Ghipqpe5EpAxxphGuXmlMApYo6rrVLUSmAVMaGD7ycBMF+MxxhjTCDeTQi9gU9hysdN2GBHpC+QC77gYjzHGmEbERTsAxyTgJVX1R1opIlOBqQDJycnHDx48uDVjM8aYdm/hwoW7VDWjse3cTAqbgd5hyzlOWySTgP+rb0eqOgOYAVBYWKhFRUUtFaMxxsQEEdnQlO3c7D76BMgTkVwRSSD4i3923Y1EZDDQBfjAxViMMcY0gWtJQVWrgWnAm8BK4EVVXS4i94rI+LBNJwGz1Mq1GmNM1Ll6T0FVXwder9N2Z53lu92MwRhjTNO1lRvNxph2rKqqiuLiYsrLy6MdSszz+Xzk5OQQHx/frM9bUjDGHLXi4mJSU1Pp168fIhLtcGKWqrJ7926Ki4vJzc1t1j6s9pEx5qiVl5fTrVs3SwhRJiJ069btqK7YLCkYY1qEJYS24Wi/h9hJCju/gLfuBnvIyZgO6/7772fo0KGMGDGCgoICPvroI8aMGcOgQYMoKCigoKCAiRMncv/994eWvV5v6P3vfvc7AGbMmMHgwYMZPHgwo0aNYsGCBaFjvPrqq4wcOZJjjz2W/Px8Hn/88VoxFBQUMGnSpFY975YUO/cU1syFBY9CWg6c8KNoR2OMaWEffPABr776KosWLaJTp07s2rWLyspKAF544QUKCwtrbX/77bcDkJKSwpIlS0Ltr776Ko8//jgLFiyge/fuLFq0iPPOO4+PP/6Ybt26MXXqVD7++GNycnKoqKhg/fr1oc+uXLkSv9/P/PnzOXjwIMnJye6feAuLnSuFr/0YBpwJc26D7cujHY0xpoVt3bqV7t2706lTJwC6d+9Odnb2Ee/nwQcf5OGHH6Z79+4AHHfccVx66aVMnz6dkpISqqur6datGwCdOnVi0KBBoc/OnDmT73//+4wbN45XXnmlBc6q9cVOUvB44Lw/gi8NXroMKsuiHZExpgWNGzeOTZs2MXDgQH7yk5/w7rvvhtZdcskloS6iG2+8scH9LF++nOOPP75WW2FhIcuXL6dr166MHz+evn37MnnyZF544QUCgUBou7/97W9MmjSJyZMnM3Nm+yz6HDvdRwApPeCCx+Ev58Obt8K3fxvtiIzpcO7593JWbDnQovvMz+7MXd8e2uA2KSkpLFy4kPnz5zNv3jwuuugiHnjgASBy91FzPfnkk3z22We89dZb/PrXv2bu3Lk888wzFBUV0b17d/r06UOvXr247LLL2LNnD127dm2R47aW2LlSqNH/dDjpWlj4DCx/OdrRGGNakNfrZcyYMdxzzz384Q9/4B//+McR7yM/P5+FCxfWalu4cCFDh36VlIYPH851113H3LlzQ8eYOXMmn3/+Of369aN///4cOHCgWcePtti6Uqhx+h2wfj7Mvgayj4MufaMdkTEdRmN/0btl1apVeDwe8vLyAFiyZAl9+/Zl2bJlR7Sfm266iZtvvpk5c+bQrVs3lixZwjPPPMNHH31EaWkpRUVFjBkzptYxAoEAL774Ip999lnoPsa8efO47777uOKKK1r0PN0Wm0nBGw/feQoeHw3/+BH88A3wxuZ/CmM6itLSUq6++mr27dtHXFwcAwYMYMaMGUycOJFLLrmExMREIHgD+q233qp3P+PHj2fz5s184xvfQERITU3l+eefJysri5KSEh566CGuvPJKEhMTSU5O5plnnmH+/Pn06tWr1o3t0aNHs2LFCrZu3UpWVpbr599SXJuj2S0tOp/CZy/BPy6HU34GZ/y8ZfZpTAxauXIlQ4YMiXYYxhHp+xCRhara6I2V2LunEG74RBj5PZj/CKx7t/HtjTGmg4vtpABw9kPQPQ/+ORUO7op2NMYYE1WWFBKSYeLTcGgP/OsnVgbDGBPTLCkAZA6Hcb+A1W/Ch3+MdjTGGBM1MZMU1u1fx7/W/Kv+DUZNhUHnwNw7YcuS+rczxpgOLGaSwnub3uPn7/+cksqSyBuIwITpkJwRLIOxey3sL4YDW6BkG5TugIO7oWwPHNoH5QegohT81a17IsYY46KYSQqZyZkAbDu4rf6NkrrCd56APevg98fBo0PhN0PgkUHw6zx4+Bh4KBce7AsP9IZf9Qq2r3m7lc7CGBPJaaedxptvvlmr7bHHHuPss88mMTExVPeooKCA5557DoB+/foxfPhwRowYwamnnsqGDRtCn41UghtgzJgx1DwSX1paypVXXkn//v05/vjjGTNmTGi74uJiJkyYQF5eHv379+eaa64JVWz973//y7nnnnvYOVRWVnLttdcyYMAA8vLymDBhAsXFxaH1NSW+hw0bxre//W327dvXgv8FvxIzI7ZqksL2su3kdcmrf8N+J8Pl/4Gdq0ADdV56eNvSWfDCRPjmL+FrVwWvOIwxrWry5MnMmjWLb37zm6G2WbNm8dBDD7Fp06ZapbHDzZs3j+7du3PXXXfxi1/8gieeeKLBEtzhfvSjH5Gbm8vq1avxeDx8+eWXrFixAlXlggsu4Mc//jGvvPIKfr+fqVOncvvtt/Pwww/Xew633XYbJSUlrFq1Cq/Xy5///GcuuOACPvroI0SExMTE0HnUVG2tKf/dklxNCiJyFvBbwAs8qaoPRNjmQuBuQIGlqnqxG7H0TOoJNHKlUKP3qOCrKQovg5evhDm3BEtyf+sRiOt0FJEaY47UxIkTueOOO6isrCQhIYH169ezZcsWevfu3aTPn3jiiaEJdiKV4K5r7dq1fPTRR7zwwgt4PMEOl9zcXHJzc3n77bfx+Xz88Ic/BIJ/4T/66KPk5uZyzz33RDx+WVkZf/7zn/nyyy/xer0A/PCHP+Tpp5/mnXfe4Ywzzjgs3k8//bRJ53akXOs+EhEvMB04G8gHJotIfp1t8oBbgZNUdShwrVvxZCRl4BFP05LCkeiUAhf+BUbfCIv/As+Oh9KdLXsMY0yDunbtyqhRo3jjjTeA4FXChRdeiIiwdu3aWt1H8+fPP+zzc+bM4bzzzgMaLsFdY/ny5aFZ2yKtq1t6u3PnzvTp04c1a9ZEjH/NmjX06dOHzp0712qvKdkdzu/38/bbbzN+/PgG/os0n5tXCqOANaq6DkBEZgETgBVh21wBTFfVvQCqusOtYOI8cXRP7N7ySQGCczWcfgf0GBIc6/DEaTB5ZvBRV2NizRu3wLbPWnafmcPh7MM6Gmqp6UKaMGECs2bN4qmnngKgf//+9XYfnXbaaezZs4eUlBTuu+8+oP4S3FOmTGnRUzpShw4doqCggM2bNzNkyBDGjh3rynHcvNHcC9gUtlzstIUbCAwUkfdF5EOnu8k1mcmZbC/b7t4Bhn0HLpsDAT88NQ5WzHbvWMaYWiZMmMDbb7/NokWLKCsrO+yv9UjmzZvHhg0bKCgo4K677gq1N1aCe+jQoSxduhS/33/YPiOV3j5w4AAbN25kwIABEePo378/GzdupKSk9tOR4SW7a+4pbNiwAVVl+vTpjZ5fc0T7RnMckAeMAXKA90RkuKrWuq0uIlOBqQB9+vRp9sEykzL5Yu8Xzf58k2SPhKnzYNYl8OL3YcxtcOpNdgPaxI5G/qJ3S0pKCqeddhqXXXYZkydPbvLn4uLieOyxxxg+fDh33HEHO3fujFiCO1z//v0pLCzkrrvu4r777kNEWL9+PcuXL+ecc87hlltu4bnnnuMHP/gBfr+fG264gSlTppCUlBQxhuTkZC699FKuv/56/vSnP+H1ennuuecoKyvj9NNPr7VtUlISv/vd7zjvvPP4yU9+Qlxcy/4ad/NKYTMQfpcnx2kLVwzMVtUqVf0S+IJgkqhFVWeoaqGqFmZkZDQ7oJ7JPdleth3XK8OmZsKU12DEJPjvL+HvU2z6T2NaweTJk1m6dGmtpFD3nkLNDeVwWVlZTJ48menTp1NaWsqll15Kfn4+I0aMYMWKFdx9992HfebJJ59k+/btDBgwgGHDhjFlyhR69OiBiPDyyy/z97//nby8PAYOHIjP5+OXv/xl6LNvv/02OTk5odcHH3zAr371K3w+HwMHDiQvL4+///3vvPzyy0iEPyhHjhzJiBEjXJny07XS2SISR/CX/BkEk8EnwMWqujxsm7OAyap6qYh0BxYDBaq6u779Hk3p7OeWP8fDRQ+zYNIC0jqlNWsfR0QV/vc7mHuXU0rjPuh1PHRKdf/YxrQiK53dthxN6WzXuo9UtVpEpgFvEnwk9WlVXS4i9wJFqjrbWTdORFYAfuDGhhLC0QofwNYqSUEETroGMoYE5214bgIgkDEYco4PJohehdAj3yb5Mca0Ca7+JlLV14HX67TdGfZegeudl+vCB7AN6jqoNQ4ZNHAcXLcMij+B4oWwuQg+fx0WPx9cH58EWQXQ6zjIKYTMEcHR1Z3Sgk82GWNMK4mpP0+bVOrCLb40GHBm8AXBrqW962HzQiguCiaKj5+AD/4Q9iEJfi6xCySmB3/60msvx/mC04t64p2fccGf3oSv3nuc5fhEiPcFk1Cc89MbbzfBjTEhMZUUuvm6ESdx0UkKdYlA19zga/jEYFt1JWxfFiyxcWhv8FW+76v3h/bB3g1ftWmgBeLwOskiEeKcpOE5kn8WEpZUBKS+9rqJJ2y5oXUtpcmJr50kyBMuh2MnRTsK0wHFVFLwerxkJGW0jaQQSVxCsAup13GNbxsIQGUJVFeAvwoCVcGKrYEq8FeGvXfWVVdC9SGoKoeqMqh2flaVQ9UhZ92hYFtTk034QwqqBCuVNPA+0ucaXNdSmrjP9jTBkjc+2hGYDiqmkgIEu5C2lbXRpHAkPJ5g15IxxrSgmLuLmZmUyfaDLo5qNsa0uoZKZw8bNgwIlqxOS0ujoKCAwYMH87Of/azW9nPmzGHUqFEMHjyYgoICLrroIjZu3AjAlClTeOmll4Bg+ezCwq+e7CwqKmLMmDGhY4SXxX7jjTcoLCwkPz+fkSNHcsMNN9Q6ZkFBAZMmta1uwJhLCq02gM0Y02pq6h6FmzVrFrfeemuttlNOOYUlS5awePFiXn31Vd5//30Ali1bxtVXX82zzz7L559/zpIlS7jkkktYv359xOPt2LEjVHyvPsuWLWPatGk8//zzrFixgqKiolplLlauXInf72f+/PkcPHiwGWftjphLCpnJmVT4K9hbsTfaoRhjWsjEiRN57bXXQvMeNFY6u2binc2bg0UWHnzwQW677bZaA77Gjx/P6NGjI37+xhtv5P77728wpoceeojbb7+dwYMHA8F6Sj/+8Y9D62fOnMn3v/99xo0bxyuvvNL0k3VZ7CWFpCg+lmqMcUVDpbMj2bt3L6tXrw790l++fDnHHdeEBzwcJ554IgkJCcybN6/ebZYtW9ZgUb6//e1vTJo0icmTJ7tSrqK5YvJGM8D2g9vJ75bfyNbGmCP14McP8vmez1t0n4O7DubmUTc3uE19pbPDzZ8/n2OPPZbVq1dz7bXXkpmZedg2u3fv5owzzqCsrIypU6cedu+hxh133MEvfvELHnzwwSM+n6KiIrp3706fPn3o1asXl112GXv27KFr165HvK+WFnNXCj2TnRnYOsITSMaYkKaUzj7llFNYunQpy5cv56mnngrNszB06FAWLVoEQLdu3ViyZAlTp06ltLS03uOdfvrpHDp0iA8//DDi+qFDhx5WQrvGzJkz+fzzz+nXrx/9+/fnwIEDh5XnjpaYu1Lo6utKnKeNDGAzpgNq7C96txxJ6ezc3FxuueUWHnzwQWbOnMlNN93E+eefz9e//vXQfYWyssYrG99xxx1cddVVHHPMMYetu/HGG7ngggs4+eSTGThwIIFAgBkzZjB16lRefPFFPvvsM7Kzs4HgvA733XcfV1xxRTPOvGXF3JWCRzz0TOppScGYDihS6ez6XHXVVbz33nusX7+e4cOH89vf/pYf/OAHDBo0iJNOOomVK1dy8cUNTxl/zjnnUF85/xEjRvDYY48xefJkhgwZwrBhw1i3bh3z58+nV69eoYQAMHr0aFasWMHWrVuP7IRd4FrpbLccTensGlPmTEFVefbsZ1soKmNim5XObluOpnR2zF0pQCtMy2mMMe1UTCaFnknBAWyBligoZ4wxHUhMJoXM5EyqA9XsKd8T7VCMMaZNic2kYAPYjGlx7e3+ZEd1tN9DbCaFsAFsxpij5/P52L17tyWGKFNVdu/ejc/na/Y+Ym6cAoTNwGYD2IxpETk5ORQXF7Nz585ohxLzfD4fOTk5zf58TCaF9E7pdPJ2su4jY1pIfHw8ubm50Q7DtICY7D4SERvAZowxEbiaFETkLBFZJSJrROSWCOuniMhOEVnivH7kZjzhMpMzLSkYY0wdriUFEfEC04GzgXxgsohEKkv6N1UtcF5PuhVPXTaAzRhjDufmlcIoYI2qrlPVSmAWMMHF4x2Rnkk92VG2A3/AH+1QjDGmzXAzKfQCNoUtFzttdX1HRD4VkZdEJPI0SS7ITM7Er352HdrVWoc0xpg2L9o3mv8N9FPVEcBcIGKFOhGZKiJFIlLUUo+82WOpxhhzODeTwmYg/C//HKctRFV3q2qFs/gkEHHuOlWdoaqFqlpYX5naI9UzyZlsx242G2NMiJtJ4RMgT0RyRSQBmATMDt9ARLLCFscDK12MpxYb1WyMMYdzbfCaqlaLyDTgTcALPK2qy0XkXqBIVWcDPxWR8UA1sAeY4lY8dXVO6ExiXKJ1HxljTBhXRzSr6uvA63Xa7gx7fytwq5sx1McGsBljzOGifaM5qjKTM637yBhjwsR8UrDuI2OM+UrMJ4Vdh3ZRHaiOdijGGNMmxHRS6JnUk4AG2Flm5X6NMQZiPCnYADZjjKkttpOCTctpjDG1xHZSsAFsxhhTS8wkhZkfb+SkB96h2h8ItaUkpJAcn2zdR8YY44iZpACwed8htpdU1GrLTLLJdowxpkbMJIWsNB8AW/cdqtVuM7AZY8xXYiYpZKcnArBlf3mtdpuBzRhjvhIzSaG+K4WeyT3ZfWg3Vf6qaIRljDFtSswkhVRfPKm+OLbU7T5KykRRu1owxhhiKCkAZKclHtZ91DPZJtsxxpgaMZUUstJ9bN1/+I1msFHNxhgDsZYU0hLZuq/OjeYkG8BmjDE1Yiop9Er3sftgJeVV/lBbUnwSqQmp1n1kjDHEWFLISgs+lro1wmOp1n1kjDGxlhTS6xnAlmQzsBljDMRYUshOq38Am3UfGWOMy0lBRM4SkVUiskZEbmlgu++IiIpIoZvxZDZQ6mJvxV4q/BWRPmaMMTHDtaQgIl5gOnA2kA9MFpH8CNulAtcAH7kVSw1fvJfuKQmHj1VICo5VsC4kY0ysc/NKYRSwRlXXqWolMAuYEGG7+4AHgfII61pcVlri4aOak22yHWOMAXeTQi9gU9hysdMWIiLHAb1V9TUX46glK80GsBljTH2idqNZRDzAb4AbmrDtVBEpEpGinTt3HtVxs9MPH8Bm3UfGGBPkZlLYDPQOW85x2mqkAsOA/4rIeuDrwOxIN5tVdYaqFqpqYUZGxlEFlZXmo6SimpLyr6qi+uJ8pHdKt+4jY0zMczMpfALkiUiuiCQAk4DZNStVdb+qdlfVfqraD/gQGK+qRS7GFJpXwQawGWPM4VxLCqpaDUwD3gRWAi+q6nIRuVdExrt13MZkOwPYNkcYwGZXCsaYWBfn5s5V9XXg9Tptd9az7Rg3Y6kRKnVR975Cck8W7VjUGiEYY0ybFVMjmgF6pHbCI0R8AulA5QHKqsqiFJkxxkRfzCWFOK+Hnp19bKnvCSSbgc0YE8NiLilAI2MV7L6CMSaGxWRSyE63Uc3GGBNJzCaFrfvLUdVQm3UfGWNMjCaFrDQfFdUB9hysDLUleBPo5utmVwrGmJjWpKQgIn9pSlt7Ud8MbD2Te9oANmNMTGvqlcLQ8AWnLPbxLR9O66gZwHbYfQWbgc0YE+MaTAoicquIlAAjROSA8yoBdgCvtEqELqgpdRHpZrN1HxljYlmDSUFVf6WqqcDDqtrZeaWqajdVvbWVYmxx3ZITSIjzRKx/VFpVSmllaZQiM8aY6Gpq99GrIpIMICLfE5HfiEhfF+NylYiQlearfwY2ewLJGBOjmpoU/giUicixBOc/WAs851pUrSArzRdxrmawsQrGmNjV1KRQrcGH+icAf1DV6QTnQ2i3stMSI3YfgSUFY0zsamqV1BIRuRX4PnCKM2tavHthuS87PZFtB8rxBxSvRwDISMpAEHss1RgTs5p6pXARUAFcpqrbCM6i9rBrUbWCrHQf/oCyo+Srq4V4TzwZiRn2WKoxJmY1KSk4ieAFIE1EzgXKVbVd31PITqt5LDXCADbrPjLGxKimjmi+EPgY+C5wIfCRiEx0MzC3ZTkD2CJVS7XuI2NMrGrqPYXbgRNUdQeAiGQAbwEvuRWY2+qdgS2pJws2L0BVEZFohGaMMVHT1HsKnpqE4Nh9BJ9tkzr74kjpFHf4XM3JmRyqPkRJVUmUIjPGmOhp6pXCHBF5E5jpLF9EnbmX25uaAWx1u496JgcHsG07uI3OCZ2jEZoxxkRNg0lBRAYAPVX1RhG5ADjZWfUBwRvP7VpWeoSxCklfjVUY2GVgNMIyxpioaawL6DHgAICq/lNVr1fV64GXnXUNEpGzRGSViKwRkVsirL9KRD4TkSUiskBE8ptzEs2VnXb4XM02gM0YE8saSwo9VfWzuo1OW7+GPuiU154OnA3kA5Mj/NL/q6oOV9UC4CHgN00NvCVkpSWyq7SCimp/qC0jMYM4iePTnZ+2ZijGGNMmNJYU0htYl9jIZ0cBa1R1napWArMIlskIUdUDYYvJgNKKauZV2BbWheT1eJk0eBKvrH2F19a91prhGGNM1DWWFIpE5Iq6jSLyI2BhI5/tBWwKWy522uru6/9EZC3BK4WfNrLPFvXVvAq1u5CuL7ye43ocx93/u5tVe1a1ZkjGGBNVjSWFa4Efish/ReQR5/UucDlwTUsEoKrTVbU/cDNwR6RtRGSqiBSJSNHOnTtb4rBAsFIqHD6ALd4TzyNjHqFzQmeumXcN+yv2t9gxjTGmLWtskp3tqvoN4B5gvfO6R1VPdEpfNGQz0DtsOcdpq88s4Lx64pihqoWqWpiRkdHIYZuuvrmaAbondueRMY+wvWw7N793M/6A/7BtjDGmo2lq7aN5qvp75/VOE/f9CZAnIrkikgBMAmaHbyAieWGL3wJWN3HfLSIxwUuXpPjDpuWsUdCjgFtH3cr7W95n+pLprRmaMcZERVMHrx0xVa0WkWnAm4AXeFpVl4vIvUCRqs4GponImUAVsBe41K146pOdnlhvUgD47sDvsnz3cp747AmGdhvKGX3PaMXojDGmdbmWFABU9XXqjHxW1TvD3rfIfYmjkZWWSPHesnrXiwi3fe02vtjzBbctuI2ZaTM5Jv2YVozQGGNaT7uuX9QSstN9DV4pAHTyduLR0x7FF+fjmnnXUFpZ2krRGWNM64r5pJCVlsiB8moOVlQ3uF1mcia/PvXXbCrZxG0LbiOggVaK0BhjWk/MJ4XseuZViOSEzBO4ofAG5m2axxOfPuF2aMYY0+piPinUPJa6ed/hj6VG8r0h3+Oc3HOYvmQ684vnuxmaMca0uphPCqErhUbuK9QQEe7+xt0M7DKQm+ffzMYDG90MzxhjWlXMJ4WenX2IwJYIA9jqkxiXyKOnPYogTHtnGst2LXMxQmOMaT0xnxTivR56pHZq8pVCjd6pvXl0zKPsr9jP5Ncmc+v8W63ctjGm3Yv5pADB+wqRSl00ZlTWKF47/zWuGH4F/1n/H859+Vx+v/j3lFXVP+7BGGPaMksKNG2sQn1SElL46XE/5d/n/5sz+pzBjE9n8K2Xv8U/V//T6iUZY9odSwpAdloiW/YfQrX50zlkp2Tz4OgHef6c5+mV0ou7/ncXF756IR9s+aAFIzXGGHdZUiA4V3N5VYB9ZVVHva9jM47lL2f/hYdPfZiDVQeZOncq096exrr961ogUmOMcZclBYJzNQNsacIAtqYQEc7qdxavnPcK1x1/HQu3L+SCVy5gzvo5LbJ/Y4xxiyUFglcKAFubOICtqTp5O3HZsMt47YLX6NO5D8+veL5F92+MMS3NkgItf6VQV1dfV8495lyW7lxqj60aY9o0SwpA95ROxHvlsLmaW9LYvmMBeHvj264dwxhjjpYlBcDjETLTfE0qitdcuWm5DEgfwH/W/8e1YxhjzNGypODISkts8XsKdY3rN47FOxazs2ynq8cxxpjmsqTgyE7zuXZPoca4vuNQlLc2vuXqcYwxprksKTiy0hPZtr8cf6D5A9ga0z+9P8ekHcPcDXNdO4YxxhwNSwqO7PREqgPKrtIKV48ztu9YFm5fyK5Du1w9jjHGNIclBUfosdRm1kBqqrF9xxLQAO9sfMfV4xhjTHO4mhRE5CwRWSUia0TklgjrrxeRFSLyqYi8LSJ93YynITUzsDWnWuqRGNhlIP069+M/G+wpJGNM2+NaUhARLzAdOBvIByaLSH6dzRYDhao6AngJeMiteBpTMwOb21cKIsLYvmMp2lbEnvI9rh7LGGOOlJtXCqOANaq6TlUrgVnAhPANVHWeqtZMPvAhkONiPA1KS4wnMd7r+pUCBLuQ/Opn3sZ5rh/LGGOOhJtJoRewKWy52Gmrz+XAGy7G0yAROapnsfiCAAAV6UlEQVR5FY7E4K6DyUnJsaeQjDFtTpu40Swi3wMKgYfrWT9VRIpEpGjnTvcGfmWnJx7RXM3NJSKM6zeOj7Z+xP6K/a4fzxhjmsrNpLAZ6B22nOO01SIiZwK3A+NVNeLzoKo6Q1ULVbUwIyPDlWABstJ8RzxXc3ON6zuOaq22p5CMMW2Km0nhEyBPRHJFJAGYBMwO30BERgKPE0wIO1yMpUmy0hLZWVpBZXXA9WPld8snOznbupCMMW2Ka0lBVauBacCbwErgRVVdLiL3ish4Z7OHgRTg7yKyRERm17O7VpGd7kMVth9onS6ksX3H8sHWDzhQecD14xljTFO4ek9BVV9X1YGq2l9V73fa7lTV2c77M1W1p6oWOK/xDe/RXTVjFVrjZjMEC+RVB6p5d9O7rXI8Y4xpTJu40dxWZKe3zgC2GsO7DyczOdPKaRtj2gxLCmFCA9hcrpZaQ0Q4s8+ZvL/lfUorS1vlmMYY0xBLCmGSEuJIS4x3fV6FcOP6jaMqUMW7xdaFZIyJPksKdWS5PANbXcdmHEuPxB72FJIxpk2wpFBHdnoim1vxSsEjHs7seyYLNi+grKqs8Q8YY4yLLCnUkZ3eulcKEKyFVOGv4L3i91r1uMYYU5clhTqy0hLZV1bFoUp/qx1zZI+RdPN1s3Laxpios6RQR2s/gQTg9XitC8kY0yZYUqgjNNlOK95XgGAtpEPVh3h/y/utelxjjAlnSaGO7FYe1VzjuJ7H0dXX1QayGWOiypJCHZlpPkRg097W7caJ88Rxep/Tebf4XcqrW/cqxRhjalhSqCMhzkNh3y68vHgz1X73q6WGG9t3rHUhGWOiypJCBFNH96d47yFe+2xrqx73hMwTSO+UbgPZjDFRExftANqiMwb3YECPFP707jrGH5uNiLTKceM98Zze53ReXv0y84vn4/P6SPAm4Itzfnpr/0yMS+TiIReT3y2/VeIzxnR8lhQi8HiEqaOP4aaXPmX+6l2MHujebG91TR0xlfRO6RyqPkSlv5Jyf3nwZ3Xw56HqQ+yr2EeFv4KdZTtZsHkBL377RXok9Wi1GI0xHZeoarRjOCKFhYVaVFTk+nEqqwOc8tA79M9I4a9XfN314zXHmr1ruPj1ixncdTBPffMp4j3x0Q7JGNNGichCVS1sbDu7p1CPhDgPl5+cy//W7ubT4n3RDieiAV0GcPeJd7N4x2IeXfhotMMxxnQAlhQaMHlUH1J9cTz+7rpoh1Kvc445h4sHX8xfVvyFOevnRDscY0w7Z0mhAam+eL739b68sWwr63cdjHY49fpZ4c84NuNY7nr/Ltbta7sJzBjT9llSaMQPv9GPOI+HJ+a33V+28d54fn3qr/HF+bjuv9dZ/SRjTLO5mhRE5CwRWSUia0TklgjrR4vIIhGpFpGJbsbSXD06+/jO8b34+8JidpZURDucemUmZ/LQ6IdYf2A9d/3vLtrbAwTGmLbBtaQgIl5gOnA2kA9MFpG6D9RvBKYAf3UrjpZwxSnHUOUP8Oz/1kc7lAZ9LetrXD3yauasn8MLK1+IdjjGmHbIzSuFUcAaVV2nqpXALGBC+Aaqul5VPwVat57EETomI4Vv5mfy3AfrOVhRHe1wGnT5sMs5rfdpPFL0CIt3LI52OMaYdsbNpNAL2BS2XOy0tUtXnnoMB8qrmfnxxmiH0iAR4f6T7yc7JZsb/nsDuw7tinZIxph2pF3caBaRqSJSJCJFO3fujEoMI/t04Wu5XXlqwZdUtXKhvCOVmpDKb8b8hpLKEm567yaqA2376sYY03a4mRQ2A73DlnOctiOmqjNUtVBVCzMyWq/kRF1XndqfrfvLmb1kS9RiaKpBXQdx54l38sm2T/jd4t9FOxxjTDvhZlL4BMgTkVwRSQAmAbNdPJ7rxgzKYFDPVB5/b227eLrn2/2/zYUDL+TPy/7MG1++wb7yfRyoPMDBqoOUV5dT5a/CH/C3i3MxxrQO1wriqWq1iEwD3gS8wNOqulxE7gWKVHW2iJwAvAx0Ab4tIveo6lC3YjpaIsKVpx7D9S8uZd6qHZw+uGe0Q2rUzaNuZsXuFdz03k0NbucVLx7x4BUv8d74UCXWTt5OdPJ2qvW+Zjk1IZX+6f0ZkD6AAekD6OLr0kpnZYxxixXEO0JV/gCnPjSPnK5JvHjliVGL40jsK9/H3I1zqfRXEtAA/oAfv/oJaIBqra7V5g/4qQpUUeGvoNJfWetn+KvSX8ne8r2UVJWEjtPV15W89LxgougSTBT90/vTOaFzFM/eGANNL4hnpbOPULzXw+WnHMN9r65g0ca9HNen7f91nO5L57sDv9vi+1VVdh7ayZq9a1izL/hau28t/1rzL8qqvxpV3SOpB3ld8hiYPjD4s8tActNySfAmtHhMxpijY1cKzXCwoppvPPAOXz+mK49/v9HEG3MCGmDbwW2hRLFm7xpW71vN2n1rqQpUARAncfRL6xdKEgO7DCQvPY/M5MxWm9TImFhiVwouSu4Uxw9O7Msf5q1h7c5S+mekRDukNsUjHrJTsslOyWZ0zuhQe1Wgio0HNrJ672q+2PsFX+z9gqU7lvLGl2+EtonzxBEncaH7GyISut9R86pp90jTn5MQDk80kZJPUlwSKQkppManBn8mpJKakEpKfAqdEzqH2hLjEiPus6nHjsTj8ZAUl0RyfDLJ8ckkxiUe0Tka0xLsSqGZdpVWcNID7/CtEVn84rxhJCVYfm2uksoS1uxbwxd7vmDLwS3BexzOPY+al1+DT0nVtPvV3+T9R/o3rkRoU+VQ9SFKKksoqSqhtLKUksoSSqtKj+r8mksQEuMSQ0kiKd5JGHHJeD3eJu3DIx7iPfGHPSwQ742v9dBAgieBeE88Xo+XOIkjzhMXfO+JwyveULL2erx4pWnHdkssX0n2SOxBui+9WZ+1KwWXdU/pxIWFvfnLhxv456LNdPbFkZnmIzMtkczOnchMSyQrzUdmZx+ZaT6y0nykJ1kfeiSpCamM7DGSkT1GRjuUiPwBPwerD4aSREllCYeqDzXps5GST32qA9WUVZdRVlXGwaqDoVdZ9VfLZVVlbCvb1uSkGAgEqAwEHxSo8n/1AEG12oDG9ujnX/85Fw660NVjWFI4Cj8/N5/Cfl3YvO8Q2/eXs3V/OdsPlPP51gPsLK2g7h+oJ/Trwg3jBvH1Y7pFJ2DTLF6Pl84JnTvUU1TVgWoq/ZXBV6CSiuoKqrWa6kB1rafQ/OoPtgX8ofUBjd6I/iNJsh3R4K6DXT+GJYWjkBDnYUJB5HJOVf4AO0sqQoli3c5SnvtgA5NmfMhJA7px/dhBHN+37T+5ZDqmOE+wiygpPinaoZg2xu4ptKLyKj/Pf7iBP727ll2llYwZlMENYwcxPCct2qEZYzq4pt5TsKQQBWWV1Tz7vw08/t5a9pVVMS6/J9eNHciQrI7TPWGMaVssKbQDJeVVPL1gPU/OX0dJRTXfGpHFdWfmMaBHarRDM8Z0MJYU2pF9ZZU8MX8df35/PeVVfs4fmcPt3xpC12R7WskY0zKamhRsZEwbkJ6UwI3fHMz8m07j8pNzmb10M2N/8y5vfLY12qEZY2KMJYU2pFtKJ27/Vj6zp51MZpqPH7+wiKtnLmbPwcpoh2aMiRGWFNqgIVmd+df/ncT1YwcyZ9lWxj36LnOW2VWDMcZ9lhTaqHivh5+ekcfsaSfTs7OPq563qwZjjPssKbRxka8atkU7LGNMB2VJoR0Iv2rokerjqucXcvXMxey1qwZjTAuzMhftyJCszrwy7ST+37y1/P6d1Xywdhfnjsgmp0sivbsmkdMlkZwuSaQlxkc7VGNMO2VJoZ2J93q45sw8xub35J5/L+fvRZs4WFm7YmaqL46cLsEk0dv52b9HCl/L7YovPrplj40xbZslhXYqP7szf7vyRFSV/Yeq2LTnEMV7yyje+9XPDbsP8v6aXZQ5SSMpwcvovAzG5vfk9ME96GKD44wxdVhSaOdEhPSkBNKTEiIW1lNV9pZV8WnxPt5auZ25K7YzZ/k2vB7hhH5dGJufybj8nvTuatUyjTEul7kQkbOA3wJe4ElVfaDO+k7Ac8DxwG7gIlVd39A+O2KZi9YUCCifbd7P3BXb+c+KbXyxPTir2ODMVMbm92Rsfk+GZafh8cTu7FbGdERRr30kIl7gC2AsUAx8AkxW1RVh2/wEGKGqV4nIJOB8Vb2oof1aUmhZG3YfdBLEdorW7yGgwW6mQZmpDMnqzJCszuRnpTIoszMpnezC0pj2qi0khROBu1X1m87yrQCq+quwbd50tvlAROKAbUCGNhCUJQX37DlYyX9X7eDT4v2s3HqAlVsPcKD8q2kb+3ZLYkhmZydZpJLbPbnWFUXda4vwuXTjPILXI2E/PXi9weWatliee9cYt7WFOZp7AZvClouBr9W3japWi8h+oBuwy8W4TD26JidwwXE5XHBcDhC8H7FlfzkrthwIJYmVWw/w5opth0012hK8TnLwiuAR8IQtiwheD8F1HsHjbFNXfYklYmukz9cTW939xnmEOdeObvB8jGmP2kV/gIhMBaYC9OnTJ8rRxA4RoVd6Ir3SExmb3zPUfrCims+3lVC8t6zez4YnDUXxB8AfCFAdUPwBpcqvXy37leqAUu0sq4Lf2S6gwZc/ELwf4neWA4HDZ+utL1FFao50MVpvnouwwu65mI7KzaSwGegdtpzjtEXaptjpPkojeMO5FlWdAcyAYPeRK9GaJkvuFMfxfbvYHNPGdEBulrn4BMgTkVwRSQAmAbPrbDMbuNR5PxF4p6H7CcYYY9zl2pWCc49gGvAmwUdSn1bV5SJyL1CkqrOBp4C/iMgaYA/BxGGMMSZKXL2noKqvA6/Xabsz7H058F03YzDGGNN0ViXVGGNMiCUFY4wxIZYUjDHGhFhSMMYYE2JJwRhjTIirVVLdICI7gQ3N/Hh3Ol4JjY52Th3tfKDjnVNHOx/oeOcU6Xz6qmpGYx9sd0nhaIhIUVMKQrUnHe2cOtr5QMc7p452PtDxzulozse6j4wxxoRYUjDGGBMSa0lhRrQDcEFHO6eOdj7Q8c6po50PdLxzavb5xNQ9BWOMMQ2LtSsFY4wxDYiZpCAiZ4nIKhFZIyK3RDueoyUi60XkMxFZIiLtcn5SEXlaRHaIyLKwtq4iMldEVjs/282kDfWcz90istn5npaIyDnRjPFIiUhvEZknIitEZLmIXOO0t8vvqYHzabffk4j4RORjEVnqnNM9TnuuiHzk/M77mzOFQeP7i4XuIxHxAl8AYwlOC/oJMFlVV0Q1sKMgIuuBQlVtt89Wi8hooBR4TlWHOW0PAXtU9QEneXdR1ZujGWdT1XM+dwOlqvrraMbWXCKSBWSp6iIRSQUWAucBU2iH31MD53Mh7fR7kuBcscmqWioi8cAC4BrgeuCfqjpLRP4ELFXVPza2v1i5UhgFrFHVdapaCcwCJkQ5ppinqu8RnEcj3ATgWef9swT/h20X6jmfdk1Vt6rqIud9CbCS4Nzq7fJ7auB82i0NKnUW452XAqcDLzntTf6OYiUp9AI2hS0X087/IRD80v8jIgudOaw7ip6qutV5vw3o2dDG7cQ0EfnU6V5qF90skYhIP2Ak8BEd4Huqcz7Qjr8nEfGKyBJgBzAXWAvsU9VqZ5Mm/86LlaTQEZ2sqscBZwP/53RddCjO1KztvX/zj0B/oADYCjwS3XCaR0RSgH8A16rqgfB17fF7inA+7fp7UlW/qhYAOQR7RgY3d1+xkhQ2A73DlnOctnZLVTc7P3cALxP8h9ARbHf6fWv6f3dEOZ6joqrbnf9hA8ATtMPvyemn/gfwgqr+02lut99TpPPpCN8TgKruA+YBJwLpIlIzu2aTf+fFSlL4BMhz7sYnEJwLenaUY2o2EUl2bpIhIsnAOGBZw59qN2YDlzrvLwVeiWIsR63mF6fjfNrZ9+TcxHwKWKmqvwlb1S6/p/rOpz1/TyKSISLpzvtEgg/UrCSYHCY6mzX5O4qJp48AnEfMHgO8wNOqen+UQ2o2ETmG4NUBBOfZ/mt7PB8RmQmMIVjRcTtwF/Av4EWgD8FquBeqaru4eVvP+Ywh2CWhwHrgyrC++DZPRE4G5gOfAQGn+TaC/fDt7ntq4Hwm006/JxEZQfBGspfgH/ovquq9zu+JWUBXYDHwPVWtaHR/sZIUjDHGNC5Wuo+MMcY0gSUFY4wxIZYUjDHGhFhSMMYYE2JJwRhjTIglBdNuiUhPEfmriKxzyn18ICLnH+U+7xaRnznv7xWRM5u5n4L6Km2KSJKIvOBUuV0mIgtEJEVE0kXkJ0cTvzFHy5KCaZecQUj/At5T1WNU9XiCgxJzImwbV7etKVT1TlV9q5khFgD1lV++BtiuqsOdaqqXA1VAOmBJwUSVJQXTXp0OVKrqn2oaVHWDqv4eQESmiMhsEXkHeNv5S/xtEVnk/IUeqpIrIreLyBcisgAYFNb+jIhMdN4fLyLvOlckb4aVePiviDzo1LP/QkROcUbN3wtc5NTmv6hO7FmElRxQ1VXOoKIHgP7OZx529n+jiHziFGqrqZPfT0Q+d642VorISyKS5Kx7QIJzBXwqIu2uDLSJvmb9BWVMGzAUWNTINscBI1R1j3O1cL6qHhCR7sCHIjLb2WYSwb/s45x9LgzfiVMr5/fABFXd6fySvx+4zNkkTlVHOd1Fd6nqmSJyJ8H5LqZFiOtpghVuJwJvA8+q6mrgFmCYU9gMERkH5BGswyPAbKfw4UaCyetyVX1fRJ4GfiIifyZYomGwqmpN6QNjjoQlBdMhiMh04GSCVw8nOM1zw0ovCPBL55dqgGAZ4Z7AKcDLqlrm7CdSTaxBwDBgbrDXCi/BSpo1aorELQT6NRarqi5xShCMA84EPhGRE4FDdTYd57wWO8spBJPERmCTqr7vtD8P/JRgGZdy4CkReRV4tbFYjKnLkoJpr5YD36lZUNX/c64AwqcmPRj2/hIgAzheVaskOHOdr4nHEmC5qp5Yz/qaejJ+mvj/lDMpyj+Bf4pIgOD9h39EOO6vVPXxWo3BeQDq1qdRVa0WkVHAGQQLoU0j2M1mTJPZPQXTXr0D+ETkx2FtSQ1snwbscBLCaUBfp/094DwRSXQqz347wmdXARnOX/OISLyIDG0kvhIgNdIKETlJnElcnPsP+QSLytX9zJvAZRKs/Y+I9BKRHs66PjXxABcDC5zt0lT1deA64NhGYjTmMJYUTLvkTOxyHnCqiHwpIh8TrBRZ3zzBLwCFIvIZ8APgc2c/i4C/AUuBNwiWWa97rEqCf3k/KCJLgSXANxoJcR6QX8+N5v7Au04siwle3fxDVXcD7zuPqT6sqv8B/gp84Gz7El8ljVUEJ1daCXQhOElMKvCqiHxKcJ7e6xuJ0ZjDWJVUY9oZp/voVedxVmNalF0pGGOMCbErBWOMMSF2pWCMMSbEkoIxxpgQSwrGGGNCLCkYY4wJsaRgjDEmxJKCMcaYkP8PkkkFTWmoSgUAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Draw gradient descent progress for each label.\n", "labels = logistic_regression.unique_labels\n", "plt.plot(range(len(costs[0])), costs[0], label=labels[0])\n", "plt.plot(range(len(costs[1])), costs[1], label=labels[1])\n", "plt.plot(range(len(costs[2])), costs[2], label=labels[2])\n", "\n", "plt.xlabel('Gradient Steps')\n", "plt.ylabel('Cost')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Calculate Model Training Precision\n", "\n", "Calculate how many flowers from the training set have been guessed correctly. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Precision: 96.0000%\n" ] } ], "source": [ "# Make training set predictions.\n", "y_train_predictions = logistic_regression.predict(x_train)\n", "\n", "# Check what percentage of them are actually correct.\n", "precision = np.sum(y_train_predictions == y_train) / y_train.shape[0] * 100\n", "\n", "print('Precision: {:5.4f}%'.format(precision))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Draw Decision Boundaries\n", "\n", "Let's build our decision boundaries. These are the lines that distinguish classes from each other. This will give us a pretty clear overview of how successfull our training process was. You should see clear distinguishment of three sectors on the data plain. " ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEXCAYAAACpuuMDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzs3Xd4VMXXwPHv7KYnJEACpNA7hBIgIp0o0gQBEQREKRbsWCkqKiIdRVD5vYqIiCJVFAQBFUERBKSE3kMgQKiB9Lo77x8bUjfJJluTzOd58iQ7O/fes4Hs2XvvnBkhpURRFEVR7tLYOwBFURTFsajEoCiKouSiEoOiKIqSi0oMiqIoSi4qMSiKoii5qMSgKIqi5KISg1LuCSGGCyF+s3cciuIoVGJQyjwhRKQQ4oGCnpdSLpNS9ijmPo8JIRIyv3RCiJQcj982P2pFsR8newegKPYkhHCSUmYUdzspZXCOfWwHvpdSLrJkbIpiL+qMQSlXhBCjhBA7hRCfCCFuAZMz2/7JfF5kPnddCBEnhDgihGhWzGO4CSHuCCGa5GgLEEIkCSF8hRAPZJ7FvCeEuCWEOC+EGJpn+7lCiCghxDUhxP+EEG6Zz1UVQvyauf8YIcTfFvrVKEoWlRiU8uheIAKoBkzL81wPoAvQEPABHgVuFWfnUsoUYBXweI7mx4AtUsq7+6oOVAACgaeAxUKI+pnPzQHqAC2ABkBt4J3M58Zlxl4F8AcmFSc2RTGFSgxKeXRFSvmZlDJDSpmc57l0DG/YjQEhpTwhpYwuwTG+BR4TQojMx08A3+V4Xg+8L6VMlVL+CWwGBgshNMAzwKtSyttSyjhgBnD3jCIdQzKpKaVMk1KqMwbF4lRiUMqjqIKeyHyT/hxYAFwXQiwUQngX9wBSyp1ABtAp81JUTWBjji63pJRJOR5fwPCG7w+4AocyLxfdATYAVTP7zczsu1UIcU4IMa64sSlKUVRiUMqjQqcUllJ+KqVsAzTFcEmppG++SzFcTnoCWCWlTM3xnK8Qwj3H45rAFeAakAY0klJWzPzykVL6ZMYWJ6V8TUpZGxgATBBCdC1hfIpilEoMipKDEOIeIcS9QghnIBFIwXDZpyS+AwZhuL+wNM9zGgw3vl2EEGFAb2CNlFIHLALmCSGqZN4Mry6E6JEZ30NCiHqZl6hiAZ0Z8SmKUSoxKEpu3sBXwG0Ml2xuYbgZXGxSykjgCJAqpdyV5+lLGBJPNIb7EU9LKc9kPvdG5rH3Ynjz/w3DTWiARsCfQAKwE5gvpdxRkvgUpSBCLdSjKNYjhFgKREgpJ+doewBYlHk5SFEcjipwUxQrEULUBfoDze0di6IUh7qUpChWIISYARwCpkspL9o7HkUpDnUpSVEURclFnTEoiqIouZTKewx+fn6ydu3a9g5DURSlVNm/f/9NKWWVovqVysRQu3Zt9u3bZ+8wFEVRShUhxAVT+qlLSYqiKEouKjEoiqIouajEoCiKouRSKu8xGJOens6lS5dISUmxdyjlnpubG9WrV8fZ2dneoSiKUgJlJjFcunSJChUqULt2bbKnwFdsTUrJrVu3uHTpEnXq1LF3OIqilECZuZSUkpKCr6+vSgp2JoTA19dXnbkpSilWZhIDoJKCg1D/DopSuln1UpIQogaGeeirYVgcZaGUcn6ePmHAOuB8ZtNaKeUUa8alKErpszFiI/MPzOdq4lX8Pf15pfUr9Knbx6ztAbP2WVZZ+x5DBvCGlPKAEKICsF8I8buU8niefjuklH2tHItNTJs2jR9++AGtVotGo+HLL79kwoQJREdH4+5uWLCrfv36tGrVitWrVwNw5MgRmjc3TMD55JNPMnbsWBYuXMjcuXMB8Pb2Zu7cuXTq1AmADRs28O6776LX60lPT+eVV17h2WefzYohJCSExo0bs2LFClu+dEWxmo0RG5m8azIpOsMlyujEaCbvmgxg0hu5se0n/TMJIQTp+vQS7bMss2piyFxEPTrz53ghxAkgCMibGMqEf//9lw0bNnDgwAFcXV25efMmaWlpACxbtozQ0NBc/d955x0AvLy8CA8Pz2rfsGEDX375Jf/88w9+fn4cOHCAAQMGsHfvXnx9fRkzZgx79+6levXqpKamEhkZmbXtiRMn0Ol07Nixg8TERDw9Pa3/wm3gdkwM5xPiqeKipYZ/dXuHo9jY/APzs97U70rRpTD/wHyT3sSNbZ8hM/It8lqcfZZlNrvHIISoDbQC9hh5ur0Q4pAQYpMQIriA7ccIIfYJIfbduHHD7Hh+PniZjjP/pM7EjXSc+Sc/H7xs9j6jo6Px8/PD1dUVAD8/PwIDA4u9n1mzZjFnzhz8/PwAaN26NSNHjmTBggXEx8eTkZGBr68vAK6urjRq1Chr2+XLl/PEE0/Qo0cP1q1bZ/ZrcgRpqek82n8mL+/6PzbeGc74FaO4c+uOvcNSbOhq4tVitZe0X3H7llU2SQxCCC/gR+BVKWVcnqcPALWklC2Bz4Cfje1DSrlQShkqpQytUqXIOaAK9fPBy7y19giX7yQjgct3knlr7RGzk0OPHj2IioqiYcOGvPDCC/z1119Zzw0fPpyQkBBCQkIYN67wteWPHTtGmzZtcrWFhoZy7NgxKleuTL9+/ahVqxbDhg1j2bJl6PXZS/6uXLmSoUOHMmzYMJYvX27W63EUugwdlfy16G87cT2lAsFtTrP40mC+Xb7Q3qEpNuLv6V+s9pL2K27fssrqiSFzUfUfgWVSyrV5n5dSxkkpEzJ//hVwFkL4WTOmOVtOkZyuy9WWnK5jzpZTZu3Xy8uL/fv3s3DhQqpUqcKQIUNYsmQJYLiUFB4eTnh4OHPmlGgJ4SyLFi1i69attG3blo8++ognn3wSgH379uHn50fNmjXp1q0bBw8eJCYmxqxjOQJ3TzdWrZ7B5OphbPgklM2RzfBwTUPfZgnj1w/jysVL9g5RsbJXWr+Cm9YtV5ub1i3rBnJJtncSTjhrchdhFmefZZlVE4MwjFv8GjghpZxbQB//zH4IIdpmxnTLmnFduZNcrPbi0Gq1hIWF8cEHH/D555/z448/FnsfTZs2Zf/+/bna9u/fT3Bw9lW25s2b89prr/H7779nHWP58uWcPHmS2rVrU69ePeLi4kp0fEfV86EH+GfhO3j84M/iX8M4m1CV4CYX+DF+BPMWz0ItOlV29anbh8kdJhPgGYBAEOAZwOQOk02+F2Bs+6mdpvJhxw9LvM+yzNqjkjoCTwBHhBB3766+DdQEkFJ+AQwCnhdCZADJwFBp5b/wwIruXDaSBAIrupu131OnTqHRaGjQoAEA4eHh1KpVi6NHjxZrP+PHj2fChAls3rwZX19fwsPDWbJkCXv27CEhIYF9+/YRFhaW6xh6vZ5Vq1Zx5MiRrPsa27Zt48MPP+SZZ54x63U5EhdXZ77+ZgoHd+/npRl6jo2M4YF6x6nUeR3v/LGXkYFTaRTcxN5hKlbQp24fs960C9peJYL8rD0q6R+g0GonKeXnwOfWjCOvcT0b8dbaI7kuJ7k7axnXs1EhWxUtISGBl19+mTt37uDk5ET9+vVZuHAhgwYNYvjw4VnDVf38/Pjjjz8K3E+/fv24fPkyHTp0QAhBhQoV+P777wkICCA+Pp7Zs2fz7LPP4u7ujqenJ0uWLGHHjh0EBQXlutndpUsXjh8/TnR0NAEBAWa9NkfTql0b/l4RwsTXZ7PkYDW69jxMs1pXWHvlDcalr8fJuczM9qIoNlcq13wODQ2VeRfqOXHiBE2amP5J8eeDl5mz5RRX7iQTWNGdcT0bMaBVkKVDLbeK++9hjqljZ/KDt4Zhg/8iyO0OzW5+RNuO7W1y7PLI3EIzU0zdPZXVp1ejl3o0QsPghoOZ1G6SRY9RHgkh9kspQ4vqV24/Vg1oFaQSQRnRb1gv1s/bwMkYf4KC7hDuO4H1i0N497GPcHVzsXd4ZYq5hWammLp7KitPrcx6rJf6rMcqOdhGmZorSSmfWrQP4ec5T3J9fhWW7W9PXIYbDTrvY86hh/njt032Dq9MKazQzFJWn15drHbF8lRiUMqEwJqB/P7zTB69EcwPX3bmn6v18a8YS2StaUz47mkS4+LtHWKZYG6hmSn0Ul+sdsXyVGJQypRnx45m2wdjuTEviG93duZ6agWatjvOx7uHk5aSZu/wSj1zC81MoRHG35YKalcsT/2mlTLHp7IPP/84gwfCvVm2piuRSZWpVecmUefMn/akvDO30MwUgxsOLla7YnkqMShlVu8B3XC7AFEJlQFYr3uGRd//n52jKt3MLTQzxaR2kxjSaEjWGYJGaBjSaIi68WxLUspS99WmTRuZ1/Hjx/O12VJYWJjcvHlzrrZPPvlE9urVS7q5ucmWLVtmfX377bdSSilr1aolmzVrJps3by67dOkiIyMjs7adOnWqbNq0qWzevLls2bKl3L17t5RSyq5du8r//vtPSillfHy8HDNmjKxbt65s3bq17Nq1a1a/qKgo2a9fP1m/fn1Zt25dOXbsWJmamiqllHLbtm2yT58++V5DamqqfOWVV2S9evVk/fr1Zb9+/WRUVFTW8xqNRrZs2VIGBwfLvn37ytu3bxf4+7D3v8dd2zZvk/cOf08O3jxG/u9EF7n4VHs5/pdH5cWIC/YOTVFsDtgnTXiPVWcMFjJs2LB86x+sWLGCt956i3r16mXNkxQeHs6IESOy+mzbto3Dhw8TFhbG1KlTgdzTdx8+fJg//viDGjVq5Dvm008/TeXKlTlz5gz79+/nm2++4ebNm0gpGThwIAMGDODMmTOcPn2ahISErGm+C/L2228THx/PqVOnOHPmDAMGDGDgwIFZU024u7sTHh7O0aNHqVy5MgsWLDD312Z1YT3D2LF4EpVWB7D4lzBOx1ejaaMo1iWN5OPF09U0GiWwMWIjPdb0oMW3LeixpgcbIzaa1WbOcc3pV9y+9mKPGMttHQOHV8HWKRB7CXyqQ7f3oMWjJd7doEGDmDRpEmlpabi4uBAZGcmVK1eMvqEb0759ez799FPA+PTdeZ07d449e/awbNkyNBpDfq9Tpw516tRh69atuLm5MXr0aMAwf9Mnn3xCnTp1+OCDD4wePykpiW+++Ybz58+j1WoBGD16NIsXL+bPP/+kW7du+eI9fPiwSa/N3pxdnPly0WQO/RfOizN1HHsiiO4NjuHXeQPvrDnElAHLVKW0iUxd8Obdne8ipTSseVBIP1NrIEytnyhOnYUtajLMZa8Yy+cZw+FV8MtYiI0CpOH7L2MN7SVUuXJl2rZty6ZNhnHzK1as4NFHH0UIwblz57Km3A4JCWHHjh35tt+8eTMDBgwACp+++65jx44REhKS9Sae97m803Z7e3tTs2ZNzp49azT+s2fPUrNmTby9vXO1353uOyedTsfWrVvp169fIb8Rx9PynhD+XjmFGmsqsviXbkQlVaJRSBTnDl2wd2ilRkEL3tx9s78rXZ+elRQK62dqDYSp9RPFqbOwRU2GuewVY/lMDFunQHqeSfTSkw3tZsh5OWnFihUMGzYMIN+lpM6dO2dtc9999xEUFMSmTZuy+hc2fbc9JScnExISgr+/P9euXaN79+72DqnYNBoNo0cPQHPMiYuJlQD4IeV19u7eZefISgdrLGJjyj5NrZ8oTp2FLWoyzGWvGMtnYogtYP7+gtpN1L9/f7Zu3cqBAwdISkrK96ndmG3btnHhwgVCQkJ4//33s9qLmr47ODiYQ4cOodPp8u7S6LTdcXFxXLx4kfr16xuNo169ely8eJH4+NyFYDmn+757j+HChQtIKUvFPQZj2oS1YUh9T/b/3Ij/btSiVtWbhFecwNvfvEiqqnUolDUWsTFln6bWTxSnzsIWNRnmsleM5TMx+BSwZnBB7Sby8vLivvvu48knn8z69G8KJycn5s2bx9KlS4mJicm6+XvX3am1c6pXrx6hoaG8//77WTdQIyMj2bhxI926dSMpKYmlS5cChks/b7zxBqNGjcLDw8NoDJ6enowcOZLXX389K9ksXbqUpKQk7r///lx9PTw8+PTTT/n444/JyMgwtjuHJoRg/Lsvs+nVMZyZXYvv/utIbLoHjTod5OMjA9iyaYO9Q3RYpi5446xxxkk4FdnP1BoIU+snilNnYYuaDHPZK8bymRi6vQfOedZecHY3tJtp2LBhHDp0KFdiyHuP4e5N5pwCAgIYNmwYCxYsICEhgZEjR9K0aVNatGjB8ePHmTx5cr5tFi1axLVr16hfvz7NmjVj1KhRVK1aFSEEP/30E6tXr6ZBgwY0bNgQNzc3pk+fnrXt1q1bqV69etbXv//+y4wZM3Bzc6Nhw4Y0aNCA1atX89NPP5G5jlIurVq1okWLFqV6+dCqgdXY8vNMRsS1YMUXHfk7uj5VveO4VHcmb3/9rNGzsfLO1AVvPuz4IVM7TbXYwjim1k8Up87CFjUZ5rJXjOV22m1Lj0pScrPltNuWEB8bxxMjpnGjn5Z+bfdTzTWe5lfm0Sasrb1DUxSLMXXa7fJ5xgCGJPDaUZh8x/BdJYVyrYKPN9PfGknan96cjq2GELD69ifcvHbD3qGVS9aoWXAkjh53+U0MipJHk3ubEKJJ4OCeekQk+NGk2QW+vz6ML5d+qgrhbOju2P3oxGgkMmvsft43T1P7OZrSELdKDIqSSQjBV99NZX7I/WyZ04oN51rg4pSBa7sVTNw0hAvnzts7xHLBGjULjqQ0xK0Sg6Lk0blbZ/759l2qrQ3km/X3cTLOnyb1L7EhdTSzvv4AvV6tC2BN1qhZcCSlIW6VGBTFCCdnJz5f+B5Luj/CP9ObsuZEKDo0+HfZwnt/PcKRg+H2DrHMskbNgiMpDXGrxKAohQhu3YwdP35I6921+HZ5GAdjqlMv6Dr/uLzKsf+O2Du8MskaNQuOpDTErRKDhdx3331s2bIlV9u8efPo3bs3zZo1A2D79u34+PgQEhJC48aNefPNN3P137x5M23btqVx48aEhIQwZMgQLl68CMCoUaNYs2YNAGFhYYSGZo8427dvH2FhYVnH6Nu3b9ZzmzZtIjQ0lKZNm9KqVSveeOONXMcMCQlh6NChlvkllFFCCKbMeIPFvQaw8/NW7LheHw+XNHaHq/WkrcEaNQuOpDTEraaUtJC78yT17Nkzq23FihXMnj2bF154Iautc+fObNiwgeTkZFq1asXDDz9Mx44dOXr0KC+//DLr16/PGv+/fv16IiMjqVmzZr7jXb9+nU2bNtG7d+8CYzp69CgvvfQSGzdupHHjxuh0OhYuXJj1/IkTJ9DpdOzYsYPExEQ8PT0t8asosxqGNMQzagUXbviRUfUssst6Ji6J5P3BH+HuabyiXCmZPnX7mFz45khvqKZy9LjL7RmDpccRDxo0iI0bN5KWZphrp6hpt93d3QkJCeHyZcNyk7NmzeLtt9/OVRTWr18/unTpYnT7cePGMW3atEJjmj17Nu+88w6NGzcGDPMvPf/881nPL1++nCeeeIIePXqwbt06019sOeXu6caPK17D5VtnvtvdkVtpnjTpGM78E4+wcePP9g6vVLDFWg6OViPgaPGYolwmBmuMIy5s2m1jbt++zZkzZ7Le+I8dO0br1q1NPl779u1xcXFh27ZtBfY5evRooRP5rVy5kqFDhzJs2LBSPbWFLfn6V2HTzzN5Kr01q//Xgb+uNMS3QgJX689h3PJRxMbE2jtEh2Xs7+7dne8y6Z9Judom/TOJd3e+W2S/0lDb4GjxmKpcJgZrjSMuaNrtnHbs2EHLli0JCgqiZ8+e+PvnH4lw69YtQkJCaNiwIR999FGBx5s0aVLWqm/FtW/fPvz8/KhZsybdunXj4MGDxMTElGhf5dETTw1l+6w3iPs8gCXbu3AlpSLNQk+z6Pxgzh0/U/QOyiFjf3emrttgrF9pqG1wtHhMVS4Tg7XGEZsy7Xbnzp05dOgQx44d4+uvvyY83DDsMTg4mAMHDgDg6+tLeHg4Y8aMISEhocDj3X///SQnJ7N7926jzwcHB+ebfvuu5cuXc/LkSWrXrk29evWIi4vLN7W3Ujgvby/WrJ7BW5U6svaLzvx7ow5+3gls3a5+j8bYYi0HR6sRcLR4TFUuE4O1xhEXZ9rtOnXqMHHiRGbNmgXA+PHjmTZtGidOnMjqk5SUVOQxJ02axOzZs40+N27cOKZPn87p06cB0Ov1fPHFF+j1elatWsWRI0eIjIwkMjKSdevWqctJJdSj3wNUOBbPmZvVAEjvupkFS+aqaTTysMVaDo5WI+Bo8ZiqXCYGa44jNjbtdkGee+45/v77byIjI2nevDnz589nxIgRNGrUiI4dO3LixAkee+yxQvfx4IMPUqVKFaPPtWjRgnnz5jFs2DCaNGlCs2bNiIiIYMeOHQQFBREYGJjVt0uXLhw/fpzo6OjivWAFFzcXZk99iKQfvFh3piVowKvjGt7+7VEiThtfSrU8MvZ3Z+q6Dcb6lYbaBkeLx1TldtrtjREbmX9gPlcTr+Lv6c8rrV9x6OFjpU1pm3bbEnQZOl5/ZQY7q+u4v/thmnpHk5ThQuyuzkx48gM0mnL5OSwXY393QInbjP3NOtrftiPFY+q02+U2MSjWVZ7/PU4dPsEz73+H14hEejY6SkXnZCKiqvJc/f8RUDOw6B0oipU4xHoMQogaQohtQojjQohjQoh850/C4FMhxFkhxGEhhOljNhXFATVq0YS/1k6j3YHaLFl5P4fvBFG3xnVWrf0/e4dWoOLUDZhTY1CWmPOaHf33ZdUzBiFEABAgpTwghKgA7AcGSCmP5+jzIPAy8CBwLzBfSnlvYfst6IyhcePGBdYNKLYjpeTkyZPl9owhp64d30GMS2V08C5uxHvRNv4tunS9z95h5XJ3rH3OYZXOGmeklLmGiLpp3ehfvz/rzq7L1ddJOCGEyDXE1E3r5nDTPFiSsd+Zqa/ZnG3N5RBnDFLKaCnlgcyf44ETQFCebv2BpdJgN1AxM6EUi5ubG7du3VIjQexMSsmtW7dwc3MrunM58NSIpsT/UYmd1+pSyTOJU/7vMXHpsyQlFD3izFZMrS9I0aWw+vTqfH2N1R2UhrH65jCnPqE01DbYbK4kIURtoBWwJ89TQUBUjseXMttyDY8RQowBxgBG5w6qXr06ly5d4sYNtRSjvbm5uVG9enV7h+EQRjw7nL43Yhjx7By+7VedB1uH06T9ET47PZDGl56if7/B9g6xWGPq9dL0tSgcfay+OcypTygNtQ02SQxCCC/gR+BVKWVcSfYhpVwILATDpaS8zzs7O1OnTh2z4lQUa6hcpTIb1s5g5bdrmPO5O82GRdIh8Cy3Gs9j/MoNTOrxKd6VfOwWn7+nP9GJpg1T1giNycnB0cfqm6Og35kpr9mcbW3F6uPnhBDOGJLCMinlWiNdLgM5Z5qrntmmKGXKkJGD+OvjN0n+IoBv/uxKVFIlgluf4ctvSzatiaWYWl/gpnVjcMPB+foaqzsoDWP1zWFOfUJpqG2w6hmDMNwJ/ho4IaWcW0C39cBLQogVGG4+x0opVZWVUiZ5VvBk5crpdGs3gS1+zXg25G/SWh8nOuoyATXy3n6zjbs3PE2tG2hVtVWJawzKioJ+Z6ZOFV7SbW3F2qOSOgE7gCPA3fPPt4GaAFLKLzKTx+dALyAJGC2l3Gdkd1mMjUpSlNJk2oS5rHRO5cEB/9HE5yqxqW7Ivb0ZO/pNNbJOsZpyV+CmKKXNvl37eGXeOqqMjKF73eN4OaVyKjKQUUHTaNi0kb3DK7Bid+ruqaw+vRq91KMRGgY3HMykdpNM3t6RlIYYLUklBkUpBXQ6HRNem8X2KhmE9TpMsM8VknUu3NrZgbdHf4hWq7VLXAWNtQ+pEsLuq/ln8x3SaEiu5GDPsfqmKg0xWppKDIpSipw9foan31mM++PJ9GxyhErOyURcq0Jf7UTuad/e5vH0WNPD5JFKYBitdGjEoSK3D/AM4LdBv1kkRnOVhhgtzSEK3BRFMU39pg3YtnY6YUfr8t3SMP67WYtaVW9ysNJ4vl+6sMjtLa24Y+rzDmEtDWP1S0OM9qISg6I4CCEEE94fy6aXnyF8SkOWn2iLi0bHWc1Wm8dS3DH1GpH7raQ0rENQGmK0F5UYFMXBVA3yp3pqPFcu+ZGoc6FW2ygmfPcMifEFr+ZnaQWNtW/n385o/8ENc1dwl4ax+qUhRntRiUFRHNAXq9+j/p8JrPi3PdEpPjRtd4wFZx9hzdqVNjl+n7p9mNxhMgGeAQgEAZ4BTO4wma96fsWQRkOyzhA0QpPvxnNh2zvSTd3SEKO9qJvPiuLA1i5fx/St4TR57AKdA8/ipNFx4lBdJnb5BN9qfvYOTyll1KgkRSkjkhOTefKZqUR2FPTscIg6Xre4nexBo8iX6N1vQLH3N3XDKFbf3IcewyWDwX6h4FffpNoEsPzYf2N1Ecaqq81drc2cuMtKvYNKDIpSxvz9+w7eXLqd2iOj6FXzGJH7Apny2Opi7WPqhlGsvLkPclZXS5n7cSZjl4gsPfZ/6u6prDyV//KYQCDJfm8ydozixFJa10+wNDVcVVHKmC7dO1P/ym2OnaxNks6Fmm2i+fjrGcVag2R13qQARpMCwOrT+ZOOpdcSMHYMIFdSKOgYxYmlrK+fYGnFSgxCCE8hhH1KMRVFYcr856m8NYnNZ5qTrHfGr8svTPrzEU4cPmrS9qavpmB87QVLj/03Z32H4sRS1tdPsLRCE4MQQiOEeEwIsVEIcR04CURnruE8RwhR3zZhKooCUL9ZA/76cQrNtgexZNV9HL4dRP0aV/nL+UWmLBpPRnpGodsX55Ng3toEsPzYf2PHKEjeYxQnFnPiLo/1DkX9q2wD6gFvAf5SyhpSyqpAJ2A3MEsI8biVY1QUJQeNRsPMuRNYM+xxDk6vz4rD95Ksd6ZW13+YtvdhDu7ZW+C2g/1CDfcUcirgUlTe2gSw/Nh/Y8cAwz2Goo5RnFjK+voJllbozWchhLOUMr3ADib2sTR181lRDKSUfDz9fyyLjaHDgBO08bvAhcPV+GCQsTWxDNSoJDUqqch+pt64EkJUwrDSWtbiPlLKAyWO0AwqMShKbqN7vcmenr7Go0cfAAAgAElEQVQ82/MPXMmg7qXx9OhR+t64FOuy6KgkIcSHwGHgU+DjzK+PzIpQUZQS2xixkR5retDi2xb0WNOD0BG18NyXwd6rdXDS6oiqNYPxy54iITa+5Ac5vAo+aQaTKxq+H15luRegODSTzhiEEKeA5lLKNOuHVDR1xqCUZwWNq58QPIFl0/Zxo5egd9tDVHe/w41ELwLODmf4oJHFO8jhVfDLWEhPzm5zdoeHPoUWj1rolSi2Zuk6hqNARfNCUhTFEgoaV7/w3EJ+WjOTCRU78fO8tvx+sQkV3FJIa7GQcWuHcz36uukH2Told1IAw+OtUyzwChRHZ2pimAEcFEJsEUKsv/tlzcAURTGuqHH1Dw3qwz8L3kLzTQCLfw8jItGXZs3Ps+Df50w/SOyl4rUrZYqpieFbYBYwk+x7DB9bKyhFUQpmyrh6V3dXln4/ldZ/prLmtw7cSPPCv9EtoiIiTTuIT/XitStliqmJIUlK+amUcpuU8q+7X1aNTFEUo4ozrv6hYV1wPys5F1cFD5d01qeMZvbXU9Dri6g47vae4Z5CTs7uhnalzDP15vNcIBVYn/kdUMNVFcVeijOu/siBwzw/dRWVRsbRo8FRvJ1TOHupGkMqTaZZSMuCD3J4leGeQuwlw5lCt/fUjedSzqJ1DEKIbUaapZTy/pIEZy6VGBSlePR6Pe+//Qm/uiTSuc9RWla6TKpOS/Sue5j0xEycXZztHaJiA2rabUUpy0z9NJ+n38XGYxn9ZQTaYen0anYYX5dELtz05f70N+nUpatlj20DZaUi2VYsXeA2XQhRMcfjSkKIqeYEqChKCd2tMYiNAqTh+y9j8xegGelX88B7/PFBK/pcaMSyxV3493odgnxvc6LaJKZ+Md5yx7aBu/Uc0YnRSCTRidFM3jWZjREbbR5LWWPqzefeUso7dx9IKW8DD1onJEVRCmVqjUEB/cSfH/LK+DH8PuEFIqZU59v/OpKsc6bCvftJTsxdH1HiY9tAeVwnwVZMTQxaIYTr3QdCCHfAtZD+iqJYi6k1BkX0q1zVl0c61ycmvArXUytQ0SuZ99Y9T9ydOPOPbQPlcZ0EWzE1MSwDtgohnhJCPAX8jqG2QVEUWzO1xsCEfiNfH0HT8zf5c09zLiZVotk9p/jqwiC+W7nYvGPbQHlcJ8FWTEoMUspZwFSgSebXh1LK2dYMTFGUAphaY2BCPzcPV1atmcm7/p34ZW4omy80xdM1hYxWixj/82NcvXSlZMe2gfK4ToKtFLUeg5BFDFsypY+lqVFJSrlXwlFJhY0gSk1J47kxUzjeWkP3LodoUOEGsaluuO4fwLMjx5Zon9amRiUVj0WGqwohtgM/AuuklBdztLtgWMVtJLBNSrnE3ICLQyUGRbGePf/8x2uf/UK1kTfpXuc4+nTB8Go/UamamkeztDM1MTgV8Xwv4ElguRCiDnAHcMdwCeo3YJ6U8mAhQSwG+gLXpZTNjDwfBqwDzmc2rZVSqukblbLP3E/dHzWGhOjsx14B0GNK/n2CacfZ8DrsXwJSx71CywvVBjD79yY0evwqTbyv8vLXTzP00VH0rd9XfUovB4qzgpsz4Ack5xy6WsQ2XYAEYGkhieFNKWVfkyNGnTEopZy5ax3kTQoF0TiDEKDLsYyKseNseB32fZ1r0x81Ffj876HwjODh5vvxcU7h7JUqpEU15E+251sLYnKHySo5lAKWXo8BKWW6lDLa1KSQuc3fQIyp/RWlXDC3FsCUpACgT8+dFAo6zv4l+Tb9MrACNx/fgtOmCJb+EMaBWzWoE3CTuqF7aHKjBmRk91W1A2WPyYnBitoLIQ4JITYJIYIL6iSEGCOE2CeE2Hfjxg1bxqcolmXvWoC8x5G6fF2uOmlBwNX7TqOv8hPh06uz7EB7EjJc6db3Ko8G18Q3yie7v6odKFPsnRgOALWklC2Bz4CfC+oopVwopQyVUoZWqVLFZgEqisXZuxYg73GENl8X/4zsZJFeMYOYR36DHcf5YVEXdl6rR0ClWAb3d0YTY7hNqWoHyha7JgYpZZyUMiHz518BZyGEnz1jUhSrM7cWwCvAtH4aZ9C6FH2cNqPybfrK7Tu45Xl7iGt/g85Vr7NneQuOxQXg5ZZKtejKqnagDDJ1Er2BQogzQohYIUScECJeCFFI3bxphBD+QgiR+XPbzHhumbtfRXFoLR413AD2qQEIw3dTbzwDvHkyf3LwCoCBX+Xe54D/Qf8FRR+n71wIfSr7zEFo6dNkGJM7TyfAMwCBIMAzgMkdJjNp9Pt4n43j5NUAUvVahj6dQmdtQzr6dDTzl6I4ElPXYzgLPCSlPFGsnQuxHAjDMJrpGvA+4AwgpfxCCPES8DyGW1nJwOtSyl1F7VeNSlIU+0mMT2T0U1O5dJ+WXu3DqeUZw61kTyodH8Tooc/aOzylEJZeqGenlNJhPhKoxKAo9rf11z95e/nf1H7iCmE1T+GmSefkyVqMbTmHoFpqbWhHZKnK54GZP3YF/DHcHM65tOdaM+MsEZUYFIdgq6khchSfIbSGewJ955oWz8Xd+bet2c5icaelpvPiC1M51BQeCDtEI+/rxMR7MLzqD1StXrXEL1kV0VmHpRLDN4VsK6WUT5YkOHOpxKDYnblFaqYyUnwGGO4J5EwOxuLRaEGffyhqvnYLxL3h+1+Y9O8Jwkbuo1WlS1Q7MoFeA/uXaF93F+BRRXSWZ5ECNynlaCnlaGDR3Z9ztBn536oo5YStFqwxUnxmtN1YPMaSgrF2C8R9733t8DifTOQdP/QSzjeZy7RF76DTFRBDIdQCPPZn6nDVz0xsU5TywVZFakaKz4y2m3tcM7evElSFJXOGcvv/fFh1tC2p0onqXbcxZddA9u/9r1j7Ugvw2F+hiUEI0V4I8QZQRQjxeo6vyUD+qhhFKS9sVaRmpPjMaLu5x7VA3A2DG7F97XQ6H67Ld9+Fse9mTWpVu8l+79d5Z/HLpKakFb0T1AI8jqCoMwYXwAvDLKwVcnzFAYOsG5qiODBbLVhjpPjMaLuxeDQFJJW87RaMWwjB25PHsuGFpzk1sw7f/9eB2Ax3Gnbez0eHH+b3334tch9qAR77M3W4ai0p5QUbxGMSdfNZcQhqVFKRFn62hC9OR9Hm0TPcWzUCAZz9tylTHluAs4tzgdupUUnWYalRSb8ABXaQUvYrWXjmUYlBUUqP2JhYRjw9g1t9NPRte4AAtzh8D7xB3yGP2Du0csdSC/V8lPl9IIY6hu8zHw/DUMmsKKWbpT/1f9sPzv+V/bhOV/Ctn/9TOxg/CzB2dmDsEz6UfFEeG/Op7MPCT95gwCtfcrBODQJqHGO3+wraX++Kb1U1NZojMvVS0r68WcZYm62oMwbFIixdi5A3KRSXX2O4eTJ/u9DmHoWkdQEpDest3GXqojx29PyI99jd3In+3f+jjuctYpI98Dz8MM8Mf4HMKdMUK7P0Qj2eQoi6OXZeB/AsaXCK4hAsXYtgTlIA40kB8g9N1aXlTgpg+qI8dvR/S6cwr2VXNn/Umg0RzXFzScel7TIm/jqUqAiHuYWpYHpieA3YLoTYLoT4C9gGvGq9sBTFBuy9YI4tONhr6dqjKzsWT8J3dQCL19/HqbhqNG0YxbrkUcxZ/CF6vd7eISqYmBiklJuBBsArwFigkZRyizUDUxSrs/eCObbggK/F2cWZLxZNZnG3geyYGczqE6FIIajaeRPrlq+0d3gKRRe43Z/5fSDQB6iX+dUnxwR7ilI6WboWoU5X8+Lxa2y8PW8xm9bFcE8hJ1MX5XEgzdu0YNNXE4n5sgp/XjK89vD0zegyij+NhmJZRZ0x3P2f/pCRr75WjEtRrM/cBXPyGrk+f3Ko0zXfIjiEPmW87aU9xtsf/iJ3jP0XGBbhKcmiPA7Gq6InQRmxXIioSkyaB3U7nuHDPQ+zZ9dOe4dWrpk0KsnRqFFJilJ2SCn5ePoClsfept2AE7Txu4BOr+H8vy14f+hcXN1d7R1imWHphXrOAbuBHcAOKeUx80MsOZUYFIdgrAYCTKslKE79hDm1FraqzraAG1eu8cQLn5A6SE/vloeo5hbPlVgfWl1/gV4PPmTv8MoESycGV+BeoDPQEWgEHJZSPmxuoCWhEoNid0bXPzCxlqA49RPm1FrYas0IC1uycBmfhZ+j5ZBztPePQIOkyqGx9HvUcWMuLSxdx6AD0jO/64HrmV+KUj4ZXf/AxFqC4tRPmFNrYas1Iyxs1JjhbJ/xGidn1+OX8y1w0ug5cut3e4dVrpiaGOKAecB5YKSUsr2UUq36rZRfxakPyNu3OPUT5tRalOI6jQo+FagUe4fLV31J1jkT2O0Y4358nJtXb9g7tHLB1MQwDPgbeAFYIYT4QAjRzXphKYqDK059QN6+xamfMKfWopTXafz423Sa/q7nm9+7cj7Rj2YtIvj+xlD+79v5lMZBM6WJqQVu66SU44BngV+BUcAGK8alKI7N6PoHJtYSFKd+wpxaC1utGWElbh5ufLdsKnOb9+T3OSH8crYlLk463Nuv5K3NQ4g8e97eIZZZJiUGIcSPQoizwHzAAxgBVLJmYIri0IzVQJhaS1Cc+glzai0sXadhJx3DOvLP0vcJXB/A4nX3cSLOn8b1LrExdTR/btps7/DKJFNHJYUCB6U0vgCtEKK7lNJmd4fUqCRFKZ+OHTzGU9N/ovqzUTxU6whRvwcz6YWv7B1WqWHRUUlSyn0FJYVMs0yOTFHyOrwKPmkGkysavh9eZe+I8jMWY2mIu4wJbhVM5agYIi5XI02vxb/bCSYvGkdGeoa9QytTLFL5LIQ4KKVsZYF4TKLOGMqQ0jDW3liMxtZEcLS4y6gLZyN58s0v0T6WSq/gw/i6JBF5w5fuuvF06NzZ3uE5NEvXMRRFDRFQSqY0jLU3FqOxNREcLe4yqlb92vzx03R6RzTk+2+6sPt6Har73eZY1beZuOQ5khOT7B1iqWepxKAoJVMaxtqbU7OgWIUQgtcmPsfv414gck4Nlu7uSEyaJ006Hmb+yYGs/2WtvUMs1SyVGCIttB+lvCkNY+3NqVlQrMq3mh+bfp7Jc7pQVn/enm2XG+LrlcjNRh8zfsVI7ty6Y+8QS6Wi1mMYWNjX3X5SSrU2g1IypWGsvbEYja2J4GhxlyPDRg9m+5w3SPxfAEu2d+FSUiWC25xh/vIJ9g6tVHIq4vnCpjSUgDpfU8xz90atI88AWlCMxtocKe5yxsvbi9WrZtC7/RuscbmH1zpuwb3NBa5duUa1wGr2Dq9UUesxKIpSpsz/cAFfJ8TTa/B/BPtEE5vqhua/Prw46nWEEPYOz64sOu125g77AMGA2902KWWhQzCEEIsxrPR2XUrZzMjzAkM19YNAEjBKSnmgqFjMSQy3Y2I4eegoGg00b90GjwqeJdqP4iA2vA77l4DUGVY8azMK+s41r6+l1z8AdWZhY3t27Oa1z3/Ff8RNutU9jpdTGicjAnmq1nTqN25o7/DsxtLrMXyBYSqM+4BFwCBgr5TyqSK26wIkAEsLSAwPAi9jSAz3AvOllPcWFY85ieH5Fe/Tro2hSPtGohcBp4cx/NHRJdqXYmcbXod9X+dvD30q/xu+qX0tvf6BqWs0KBan0+l489WZ/O2v4/4eh2jqE01yhgu3d3Vk4ugpaLXaondSxlg6MRyWUrbI8d0L2CSlLLKaRAhRG9hQQGL4EtgupVye+fgUECaljC5sn+Ykhoc/eYeKrS/j5pROiG8ULpoMbid7Isk+xbx+wZeXms+iRp2aJTqGYiMfVDZ8+s9LaOH9mJL1/aQZxEbl7+dTA147Wng8BW1rjCn7Uyzi1NETjHn3OzyfSKRn46NUdE7mwuYQ3nv5f/YOzeYsXeB29yNQkhAiEMOiPQElDS6HICDnX9KlzLZ8hBBjhBD7hBD7btwo+Zzsr9/Tl4xP/bkyowaLt4RxIi6AZOFMinAiRTiRodHQtMkF1iWN5K01w3lr1ePM+3oGer2+xMdUrKSgWVqMtZva1xrrH5jbVzFLo2ZN2L52GnzszOJ/O5Oq1+LcPJL0tPSiNy6nihqVdNcGIURFYA5wAMOIpEVWi8oIKeVCYCEYzhhKup/OndrTuVN7wHAdcuIUHYme2YuN6900BD58lQfqH6dxy7vT+kbw/t97Cb7cBxenrFssNA9uQ4NmTUoaimIuoS34LKCkfX2qF3DGYOL6ByafMah6B1sSQtAmpDIXL2m5muJNraDbzDzwMJ2TXyXsvgfsHZ7DMXnNZyll6t2fMdyATrnbVsS2tXGgS0lF0ev1TBo/h00iA72/ROuSQZd7jtOy8mU0IvfvKlWnJXJXCyY9+hEurtlj2rVO2nI/+sEm1D0GpZgWzP2axVFXCH3kFG2rRCKBs3ua8O7D8/AsBwNRLH2P4YCUsnVRbQVsW5uCE0Mf4CWybz5/KqVsW9Q+bTFcNS4mjqtRV0lIiGf8nDWkD9JRrWps1vNaIWntf4GqbvGk6JzQ6bOvyt1O8KTZ1VH07zfYqjEqqFFJSrHdvnmbEc/MIrY/9G59iED3WK7Ge9Pw4lM8PKBs/81aJDEIIfwxXPP/HngMsu7QegNfSCkbFxHEciAM8AOuAe8DzgBSyi8yh6t+DvTCMFx1tJSyyHd8W9cxSCn5YfFK9u0+ndWWkp7BXk8nmg+OIMg7u+xeCElNjxichJ7zl6qiS3VC3vThzR4z8a3mZ7OYFUUp3Orv1zLrryMEPxZJx8CzOAk913/vxMQXy+4qApZKDCMxLOMZCuR8J44DvpVS2qXy2VEK3OLuxDLmmZlcSc2+jKRHoOuto0f7QwS6Gc4w3LQZ3E7xIHlPa3xExay+AT51GfDIEHXZSVHsJCkhif593yfhaWdGhO7k9mUv3u620d5hWY2lLyU9IqX80SKRWYCjJIaCbNu8nbe+3UZSY8NN7Ub3XuS+WifxdErL1/fU+UCGVnqXwMDsm5Ee3h54VHDP11dRFMsbP2oyG+q780T/7QS6xXLseC1ebTWHwJplb4CApRODPzANCJRS9hZCNAXaSymN3M2zPkdPDGC4iR17Iw69XvLe+5+xu7akWtMYcp4bNAm6TLDPFTQCdDL7mZQMZ67uas27I2bh7OKcf+eKolhMWmo6Lz7/IYeaCbqHHaJhhevEpbmh29OTV0aPL1Nn9JZODJuAb4B3pJQthRBOGNaAbm5+qMVXGhJDXmeOnuKX1Vu4+/vWI9l45Bpuj6fQpPplcv7fq+55G1+XRK7FVyAp0Y30JGe6OT/L/fd3t1P0ilL2Hdy9n5c+/hm/kTF0r3ecCk6pnIoMYGTgVBoFl41h6ZZODP9JKe/JuYSnECJcShligViLrTQmBmOklHw0dQF/7L6EzDpjENxs6k7bh0/RpFI0AomnNs0wrC68DhViq2Rt7yo8eeaRN/Gp7GOX+BWlrNHpdEx8fTbb/NLp2vMwzSpeIUXnjPi3P889+bq9wzObpRPDduAR4HcpZWshRDtglpSyq9mRlkBZSQwFibkRw4gxc4iq4Y3UgE/LOHq3CSfIPTZf35tJnlQ6+jAdmmb/U7i6u1O7UZ0ydQqsKLYUcfIso978Bo+nExnQ+CB3rnkwsesme4dlNksnhtbAZ0Az4ChQBRgkpTxsbqAlUdYTw113/23WrljPjD/Dce+SjFaTPTVHdf9bdAo6g7s2I9d2egknz9TgpSazqFG3ti1DVpQyY9IzH7K2uguPD9xOkFssZ3aE8t7wj3BxdbF3aCVmjWm3nYBGGGoZTkkp7TbRSHlJDDklJybz+7rfSEvJ/rX/8udBTrRxokXoeZxyJAxf9wQaVrhOcoYzdxI90Ok1aA63ZsKTH6DRqGW+FcUUt2/EMOzJ2aQM1dO3ZThVXBO4dLsS98a+xAPde9s7vBKx9BmDG/AC0AnDPEk7MBS4pZgbaEmUx8RQkL07/+PjOWtIzzEN0FUXF3xHxNK+zhmcNHo8tal4OaVx/pof6RHZ9yiETstDjUZyb4eOdohcUUqHLz/9hi/PXqLN4NPcW/U8Aji9tzHv9f8ET+8K9g6vWCydGFYB8RgqoMFQBV1RSmmX+nGVGAqn1+t5/61P2HIpCZ2LBllLT+deR2lZ+RLaPPM9pem0ROxuxv1+D+KsdaZd1664eagaCkXJKTYmlhHPzCSmD/QOPUSQ+x2uJ1SgZsQIhjwy3N7hmczSieG4lLJpUW22ohJD8VyOvMSoVxZwK8wNjVv2JaeKlRPo0fQo1dzis9qi47xpcuUZ+j/0iD1CVRSHtm7lej78/SBNHrtIp8AzOGt0nDxch/GdP8GvWpWid2Bnlk4M3wOfSyl3Zz6+F3hRSjnC7EhLQCWGkjlx4AQJcdkjm/btP8Kic9HUfigaD9c0vFxTaO13ESehJy41e3pxKQVXTvszvtMn+Pk7/n9+RbGm5MRknh4zlYh2Gnp0OkRdr5tcjPDl7W4/o3Vy7FXhLJ0YTmC48Xwxs6kmcArIAKSUsoUZsRabSgyWEx8bz4dvzSfmdjLXEjK4+aAzXe45jkeO6TtctRkEuMVxJ9Wdq2erQo4q7aA7rXlx1BtqaKxS7kwa8yE/Brgx/JFtBLjGcZ9cQt3G9e0dVqEsnRhqFfa8lPJCMWIzm0oM1vP7hq1MW/gnKTnWl8jwcqL2o5fpWvskHtrshKEBNEJy+kIADS93RSOcaX9PN+o3Kb+LrSvlx64tOxm7fActnz5Ol2pniU9zJXXPA7w26i2HHf1n8eGqjkQlBtvSZeh47eVp/O2tRVbKbnfyyCCs3RGaVTTM9wSQnOFM9M57ePORyWi02X8cHhXc1VmFUuYc+i+cF2evofITd+he/zjezimcjvLnMd8PCG5plxmDCqUSg2JxcTHxxN3KvlF968YNXpm9AgbpqFw5AScnHe2CIvB1SSRNp0WfY8rA67e9aZ/wYqkd/60oBdHr9bw74SM2e6bQtfdRmle6RKrOmas72zJp5AycnE1dQdn6VGJQbEJKyfJFKzh2JJLYxFR2VdLQqt9ZqngkZPXRaPTU9byJAC5c8UPqss8k0q9UZmKfj6joW9HI3hWl9LhwNpLRb36J0/BUejc9TGWXJM4cDeSDh1Y6zE1plRgUu7h17SbPPf8RN1OzPyVlOAnol0GP0MP4uiRmtQsh8dCmcyvJk+T/WuAm3ann15z+Ax5Vl52UUklKyWuj3+OPhh4M7buDILc7dBdrqN4g0N6hASoxKA5m3epfmL5uPyk1sueZkVpo2iWSsJqncNcapvrQSzh5ugYjAt7Fxzv7LKJilYp4VfSyedyKUlw7Nv7NK2t3ce/Th2nnF8nlOxVpE/MiPXv2sXdoKjEojkev15OckD2Lii5Dx2tvzCG8sRbferFohKRV7Ugae1/Lt21Cmiux/3ZkwugP0God47RcUQryzf99z+dHIwgZcpb21SIME8ztb8i7feZTwcd+02ioxKCUGicPHefv33aRnp7O8oOXqfh4HHWqXc96XgD1vG9Q0TmZa/EVSE3LHkqbEudKH49X6dCpsx0iV5SCxd2JZdTTM7jeS0Ove8Kp4XGHG4leBJx5jOGDR9klJpUYlFJJr9czZdI89h7KTgxSwrVWnnTse5w63jdyLY/q7ZyMXmqICK+Na3wFPGRlXhr2Nh5eHrYPXlGM2LD2Vz74ZS8Nhl+ia/XTuGh0nN7ZkGkjv7Z5vYNKDEqZcvnCZUaP/YwbAT45C6+p2P4OvVsdwt8tLqvtarw3NU4PpkFQ9lRePhUr0bB5Y1uGrChZUpNTGTn8PSIecGNg5z0EuMXRPv47GrauZ9M4VGJQyoXvvl7B/L2ncLk3DaGRNKgdTYeAs7hqdLn66aXg+LHajGs3l6oB1ewUrVKe/fvbv7y04i/aPmO4KX3iTHWebzSLWvXq2CwGlRgcyM8HLzNnyymu3EkmsKI743o2YkCrIHuHVWYkJSTxz5a/ydDpWbJmB1H3aWnc9BIakT2TbFCFO9TzuklShjMJKdkTBGboNLgc7szro99SQ2QVq5JS8spzU9lZW/JA93CaeF8lMcOFhH/vY9zod21yWUklBgfx88HLvLX2CMk5VtJxd9YyY2BzlRysZNvm7Sz+ajMZOU4aIt1dCXziOq2CLuZKGD7OyXg5pRERXYW0yxUR6U4Ma/4yzVuF2CFypTw4duAoz05dTsWR8fRocBQf5xTOXq7Goz7vW/3/nUoMDqLjzD+5fCc5X3tQRXd2TrzfDhGVTzqdjtfHzmB3jB69NvvMQN9I0rWnYb6nu4sYJWc4cWVXK0K922f1c3Z24/5eD+LqVnrX+1Uch16vZ/I7n7DRKZHOfY/SstJlUnVarv/TjnefnmW1s1eVGBxEnYkbMfYbFsD5mfYveCnvIs9E8OS4r7jV0RPhoqdq0B16NTmSq0L7rruFSt3zzPfkqDNpKo4vKuIio15bgBieQZ/gQ1R0Tqb1jc9o2bG1VY6nEoODUGcMjk9KSeSJ86Qkp7Bp83aW3YqhdvfLuDhlX4uq6JFIG78LaIUkKd0lK9lLKbh4rAZvd5tHJb9Kxg+gKIU48s8RRn+xkVYvHKNTlXOc+rcJ7z7yGe6elh9yrRKDg1D3GEqfmBsxzPng/4hPyl57IvJWKnceEnQMOY1L5vQdAJ5OaQS4xXE7xYPrFysj9YK6sV14evgL6ma2YhKdTsfQ/m9x8SFXBnbYi79bPNGxPjS7Ooa+fR+26LFUYnAgalRS2fDLmo3MXbKLdKfsCQJTKjlRb1gUnWqcwVWTgVbo0SI5da46taNDEZnleALBA50HUqdBXXuFrzi4775ewfx9p2gx9Dzt/c+hFXpOHmjAOz3m41PZxyLHUIlBUWwgLTWdF1+Yyt7KruANLpVS6dbuME188s/3lJjuwu1/O/J8nzdznU34VPFW9ykUABLiEhj99DSudNPQ895D1PKM4WaSJ34nhjBy6NNm799hEoMQoh1J090AABSdSURBVBcwH9ACi6SUM/M8PwqYA1zObPpcSrmosH2WlcSgziTKjsS4JJLjk4k8d55X5/2M8yOpeFXInjDQzTWNjjXOUsk5GZ0UyBzl25duVqKb7k06delqj9AVB7Rl/W+8++Mu6j1+ma41ThOf7soLdX/C09PTrP06RGIQQmiB00B34BLwHzBMSnk8R59RQKiU8iVT91sWEoO691B26fV6ln+1gqjIq1ltV27GsdNPS6sHz+DjmpTV7qzV0aDCDfRScPl6JaRekH4+kHce+QjPCua9CSilW1pqOs8/O4UjLTW4u6Wy9ZmpODmZtxqcqYnB2mvOtQXOSikjMoNaAfQHjhe6VTkwZ8upXEkBIDldx5wtp1RiKOU0Gg3Dn30sX3t0VDQvj51PZGqOFeyctPz7SBrdQ47i65eAFj2egTdYcHYgafuboNVn/4k2qdaOAf0H2eQ1KPbn4urM10s+ZP+ufURcjDI7KRSHtY8UBETleHwJuNdIv0eEEF0wnF28JqWMyttBCDEGGANQs2ZNK4RqW1eMDGEtrF0p/QJqBLDmp5n52lcsWc3c2R6k+juDC7TscZZOgWdw7bo3V7/bchfjf17L40Hj8HDLHspYNSgA78reVo9fsY82HUJp06HID/kW5QirVP8CLJdSpgohngW+BfIN8JdSLgQWguFSkm1DtLzAiu5G6xsCK7rbIRrFnoaOGsyQkRJdho6UpBSeeW4GC++piXdAdpGdRqOnXYOzBAdHcpAXc20fH+1Kys/deG3kW2oRI8UirJ0YLgM1cjyuTvZNZgCklLdyPFwEzLZyTA5hXM9GRu8xjOvZyI5RKfYihMDJ2QkvHy+WL5/G8f1HOfTfkaznk5KS+XqtB4eH3ybI93aO7SRNKl6lSudfmXv4bzIytMTf8mRo1Xdo0bqVPV6KUgZY++azE4bLQ90wJIT/gMeklMdy9AmQUkZn/vwwMEFK2a6w/ZaFm8+gRiUpxaPT6Zg8cS5HTmR/ltJJuBLqQafex6judRsBVHZJJF2vJfJQLbQ5ZpKtkFaVVx9/D1d3VztErzgChxiVlBnIg8A8DMNVF0sppwkhpgD7pJTrhRAzgH5ABhADPC+lPFnYPstKYlAUS7hwNpKn31jIrSqeIKBSt9s82OwQfq7G53uqfWYggZWzT+SrVguiaatmtgxZsROHSQzWoBKDohTsf3O/ZtH5Kzg3z57SQwLNGl2kvX8ELnkWMdJJwYnwerx9/zwq+VW2cbSKLanEYGGmXvYZ/tW/7DwXk/W4Y73KDA6taXRbU/epLjkpxZUYl8jBXXf/RgR6qWf+os1E99RSv0E05Jjzt26lm9TxvEVyhjPJ6c6kpjlT8Vhvnh3xsprvqYxRicGCTC1Gy5sU7hKQa+ptd2ctj7QJ4sf9l4vcpyqEUyzpt/W/sWLZ9lz/H487u1DriWiCq11GCImfSyIe2jTOXalK+s3sIjuR6sKokPE0Cm5i+8AVi1CJwYJMnTq79sSNJu9TKwQ6I7/7vPtU03Yr1paels7YF6ZxMBbQCnSt4P5uh2jofY2cMzhphCQpw5nrO1vT0C04q93DrQIP9huIs4uzzWNXisdRKp/LBGsUoxlLCv/f3p1HV12feRx/f3KzEEIA2bewKwhoQUBQrFhQ1GrBHXEclTOOWnWqQ7XWDm6t1vboWGes4jiouFAVwa3qSD1T21EryKKUzci+qYQAhoQl6zN/3B+5SUjkJjfJzU2e1zkc7v3mtzw/OLnP/f2+3+/zre6YPhHONbSU1BRmzb63/P3qz1Zz/a+KeO/kkViFaRE9BuRyzqC/03f8YopYXN5eBPxm+Qt8/8AMxow5lVByiNQ0TxKJzBNDFBpiMlpNdwxVj+kT4VxjGzpiKB8uuJ8dG76ipKSkvP2VV97khUUdyBr/DaFQZN3srm3zGNlxK5s6zOSLrSmUliWxdfkA7p7yOzLaZsbjElyMPDFEIdrJaOMGdIi5j6HqMX0inIsHSfQaWLkP6457fsL0r3by+ENzOFQUWaxo5bZ2rLqoN2OGriM5qYz2qQcYMnYNT266iL07I6U6ykqTGJR3dr2Uj3YNy/sYouSjkpyr2bznF/BfryynRKKgZxpDpm7m5O4bSVbkziJFpYRURnZ2b7p9MxiC0uNJCnHBhCvpPaBvnKJvObzz2TkXFwf3H+Ta6+5nVfcMLDLxmlbdCjnr1BUcl5lzxD4FxWkUfHIGV024gaSkJDpndfTFixqAJ4Z6NvONlby0eBulZoQkpo3JYtOugqjvDqrjdwKuOTt0oJDiwsgjpzUrVzPj92+TMqWQVumRyXeZGQf5ftY62qWEFzYqM9j0dRcuzJjJiFGNW1W0ufPEUI9mvrGSFxdtjWrb6voTqptz4PMTXEtUWlrKgjkLyM2JfKHK3pzDR12NEWdtICO1kPTkIo7LzKGkLImde9phwbrZGBRm92Hm1IdIz/DBF3XhiaEeDbjz3RqHl0ajujkHPj/BuYhtG7fyrz99goIicSA5RMq0Q0wcsprWyZE7i2SVkZlcyNd57Shc0R9VGEt7UtcJnP+jC+IRekLxeQz1KJakANXPOfD5Cc5FZPXvXWkRoycffZZn3m5LUdvIR1RZJoz64Zec0m0Dqad/Vmn/HFvG7fMXcHn3HxMKpTBw8GDatPehsnXliSEKNc05iFZ1cw58foJzNbvh1uncUKUtPy+f6df+midP6Ut650Pl7anJpZw++AuGfW8Dq7gNgL9uSyflzR/x46tu8XpPdeCJIQrTxmTF1MdQ3ZwDn5/gXO1ktstk/qsPsmbZajZkry9v37M7j1kPt6brtN10abePUFIpJ3bYQetT5vHIincotcjopr3ftOXqXvcy+ISh1Z3CBbyPIUo+Ksm5pqu4qJh77nyE9ev3UFQK28e1ZvzEVXROzy/fRhhd0vI5VJrC1lVZqCiZdgf6MOPqmS2mzpN3PjvnWqy1K1Zz811zyWvXurzNkqHTebs5+/iVdEw9UN6+Jbcjx26eQtu0Y8rb+vY7jqEjTmjUmBuDJ4ZqRPsNvbq7g8Ubd7MuJ7Ii1rFdMti0az8lFf75kgWtU0PsK4w8HmqbFiI9NcTO/Mjoiq6ZqSz+t7N85rNzjcjMeOTBJ3hx5140sAQlwejh6xjdeUulGdoAJWVJfLlkEHdPfrRZ1XvyxFBFtPMGajNnIRZt00IUl+HrMTjXyAryCti4ZgMlpUXc/fBr5E2GrN67IhsIhnb+il7p33KoJJmi0khX7P5DaXRddyHTp10Xh8hj54mhimjnDcQ6ZyFWvh6Dc43rrZff4n/eiXyelFLG8pQUBl2xjQEdK5fv6JGeR1pSMZt2dKEkP42ygtbcNPZesvr3aeyw68TnMVQR7byBeCYF8PUYnGtsky+fzOTLJ1dqKzxYyA3X/4r/ze8J5aNdRfFpxpnjV9Cvx24AUpNKefPgNXw7eyQ9Qr3K92+f0YkpF08jFAqRiFpMYoh23kCscxZi5esxOBd/aelpPPv8/Ue0f7ZoGTc/WMY7w9MxGX1O2MmkgavIGv+3StvlA/d9/Co/TLqD4wcPK29PbZVCepum/7vbYhJDtPMGajNnIRY19TH4egzONV0jxo7ko1dPInfHHqysjP94/DnmLOtEz5E5qEIx2D6dchnebRtfcBsrcyN3DSVlSWz59HjuufQR0jNaV3OGpqHF9DGAj0pyztW/rRu2MHf2fEpKwyObzODDL/dQOrWYEQM3k6TIh0TnVgV0bZVP7v425O1tTWlRiFEHp3LhBZc1Sqze+eycc3H01H/OYd7CbMqI3Erk9UnjxMs2MqLLVpJktEoqJklG9qo+dMjtXb5dMqlcef5NdO3RtV5j8sQQpVi+jVe37+MfrDvizuL9GWfUS6zOucRWsK+A6dc+wJfd20IaZPTfz6SxK+iXsfuIbfMKW6El53DemItp0y6TLj27xHx+TwxRiGWOQHX71sSTg3OuopLiEspKy/jog4/5xdy/kHZ2ISmpkUWNOhxTwPisbNoEZcezc7rx63GvxXxeH64ahYcWZh/xwX6wuJSHFmYfNTFUt29NKt5BOOdcckoypMCEc3/AhxNP449/eIuCXZEyHYv+tJ5n+vdgxOkbSAsVk5PbvnHja9SzNTGxzBHweQTOufqQkprCRddcXKntKiB71VruumsOB4uhz4C2MKXxYmrRiSGWOQI17eucc/Vh0LDjmff6b+Ny7qSjb9J83X72INJTKs9MjHaOQHX71uTYLhl1is855+KhRSeGC0b05MGLTqBn+3REuP5QtMXpqtv30anDj0gC3vHsnEs0LXpUknPOtSTRjkpq8DsGSedIypa0XtLPq/l5mqRXgp8vltS3oWNyzjlXswZNDJJCwOPAucAQYJqkIVU2+ydgr5kNBH4HxKe3xTnnHNDwdwwnA+vNbKOZFQEvc+SgqynAc8Hr+cBEScI551xcNHRi6Alsq/B+e9BW7TZmVgLkAR2rHkjSdZKWSlq6a9euqj92zjlXTxJmVJKZPWVmo8xsVOfOneMdjnPONVsNnRh2AFkV3vcK2qrdRlIy0A44sqKUc865RtHQM5+XAMdK6kc4AVwOXFFlm7eAq4FPgEuAP9tRxtAuW7YsV9KWGOLqBOTGsH9T4tfSNPm1NF3N6Xpqey1RLU7doInBzEok3QwsBELAM2a2WtIvgaVm9hbwNPCCpPXAHsLJ42jHjelZkqSl0YzlTQR+LU2TX0vT1Zyup6GupcFrJZnZu8C7VdrurvD6EHBpQ8fhnHMuOgnT+eycc65xtNTE8FS8A6hHfi1Nk19L09WcrqdBriUhayU555xrOC31jsE551wNPDE455yrpEUlBknPSMqRtCrescRKUpakDyStkbRa0i3xjqmuJLWS9KmkFcG13BfvmGIlKSTpM0lvxzuWWEjaLGmlpM8lJXSte0ntJc2X9IWktZJOiXdMdSFpUPD/cfjPPkm31us5WlIfg6TTgQLgeTMbFu94YiGpO9DdzJZLygSWAReY2Zo4h1ZrQdHEDDMrkJQCfATcYmaL4hxanUmaAYwC2prZ+fGOp64kbQZGmVnCTwiT9BzwoZnNlpQKtDazb+MdVyyCCtY7gDFmFsuk30pa1B2Dmf0f4Ul0Cc/Mvjaz5cHrfGAtRxYoTAgWVhC8TQn+JOw3Fkm9gPOA2fGOxYVJagecTnhCLWZWlOhJITAR2FCfSQFaWGJoroLFjUYAi+MbSd0Fj14+B3KA980sYa8FeBT4GVAW70DqgQF/krRM0nXxDiYG/YBdwLPBI77ZkprDYuyXAy/V90E9MSQ4SW2ABcCtZrYv3vHUlZmVmtlwwoUWT5aUkI/6JJ0P5JjZsnjHUk9OM7OTCC+2dVPwODYRJQMnAbPMbASwHzhiRclEEjwOmwy8Wt/H9sSQwILn8QuAuWb2WrzjqQ/B7f0HwDnxjqWOxgGTg2fzLwMTJL0Y35Dqzsx2BH/nAK8TXnwrEW0Htle4E51POFEksnOB5Wa2s74P7IkhQQUdtk8Da83skXjHEwtJnSW1D16nA2cBX8Q3qroxszvNrJeZ9SV8m/9nM7syzmHViaSMYGADwWOXSUBCjugzs2+AbZIGBU0TgYQbqFHFNBrgMRI0QhG9pkTSS8AZQCdJ24F7zOzp+EZVZ+OAfwRWBs/mAX4RFC1MNN2B54IRFknAPDNL6GGezURX4PVgpd1k4A9m9l58Q4rJvwBzg0cwG4HpcY6nzoJEfRZwfYMcvyUNV3XOOXd0/ijJOedcJZ4YnHPOVeKJwTnnXCWeGJxzzlXiicE551wlnhicc85V4onBNTuSrpHUI4rt5ki65Dt+/hdJo+o5tvaSbqzw/oxoS3NLerQ+SlJIeljShFiP45ovTwyuOboGOGpiiJP2wI1H3aoKSR2BsUGF4Fg9RoLXCXINyxODa9Ik9Q0WVpkbLK4yX1Lr4GcjJf01qPy5UFL34A5gFOEZrp9LSpd0t6QlklZJeiooJ1LbOCZJ+kTSckmvBsULDy9kc1/QvlLS4KC9s6T3g4WHZkvaIqkT8BtgQBDbQ8Hh21RYQGZuDfFdDJTPOpY0WtLfFF7c6FNJmcGd0hvBeTdLulnSjKCa6CJJHQCCEs0dJXWr7b+Daxk8MbhEMAh4wsyOB/YBNwYFBB8DLjGzkcAzwANmNh9YCvyDmQ03s4PA781sdLA4UzpQq4Vzgg/0mcCZQaXRpcCMCpvkBu2zgNuCtnsI10kaSrhgW++g/eeE6+cPN7Pbg7YRwK3AEKA/4XInVY0jvBjT4aqarxBezOh7wJnAwWC7YcBFwGjgAeBAUE30E+CqCsdbXsN5nGtZtZJcwtpmZh8Hr18EfkL42/Mw4P3gC3YI+LqG/X8g6WdAa6ADsBr4Yy3OP5bwh/bHwblSCX/QHna4su0ywh/KAKcBFwKY2XuS9n7H8T81s+0AQd2rvoRXsauoO+H1BCCcKL82syXB8fcF+wJ8ECzclC8pr8J1rgROrHC8HJru4zYXZ54YXCKoWtDLAAGrzew71+2V1Ap4gvDylNsk3Qu0quX5RXjxoGk1/Lww+LuUuv1OFVZ4XdMxDhJd3BWPVVbhfVmV47YicpfhXCX+KMklgt6KLNx+BeFv09lA58PtklIkDQ22yQcyg9eHP0xzg36BGkchfYdFwDhJA4NzZUg67ij7fAxcFmw/CTimmthqYy0wMHidDXSXNDo4fqak2iak40jQEtqu4XlicIkgm/DqYWsJf8DOMrMiwh/yv5W0AvgcODXYfg7wZPBYphD4b8IfgguBJbU9uZntIjzS6SVJfyf8GGnwUXa7D5gkaRVwKfANkG9muwk/klpVofM5Gu8QLhlPcO1TgceCa3+fWtwFBf0zAwn3lTh3BC+77Zo0hdezfjvoOE4YktKAUjMrCe5qZgVLl8ZyzI+A82NdxF7ShcBJZnZXLMdxzZf3MTjXMHoD8yQlAUXAP9fDMX8aHDemxED49/7fYw/HNVd+x+BaPEmvA/2qNN9hZgvjEY9z8eaJwTnnXCXe+eycc64STwzOOecq8cTgnHOuEk8MzjnnKvl/THNtCsfRwi4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Get the number of training examples.\n", "num_examples = x_train.shape[0]\n", "\n", "# Set up how many calculations we want to do along every axis. \n", "samples = 150\n", "\n", "# Generate test ranges for x and y axis.\n", "x_min = np.min(x_train[:, 0])\n", "x_max = np.max(x_train[:, 0])\n", "\n", "y_min = np.min(x_train[:, 1])\n", "y_max = np.max(x_train[:, 1])\n", "\n", "X = np.linspace(x_min, x_max, samples)\n", "Y = np.linspace(y_min, y_max, samples)\n", "\n", "# z axis will contain our predictions. So let's get predictions for every pair of x and y.\n", "Z_setosa = np.zeros((samples, samples))\n", "Z_versicolor = np.zeros((samples, samples))\n", "Z_virginica = np.zeros((samples, samples))\n", "\n", "for x_index, x in enumerate(X):\n", " for y_index, y in enumerate(Y):\n", " data = np.array([[x, y]])\n", " prediction = logistic_regression.predict(data)[0][0]\n", " if prediction == 'SETOSA':\n", " Z_setosa[x_index][y_index] = 1\n", " elif prediction == 'VERSICOLOR':\n", " Z_versicolor[x_index][y_index] = 1\n", " elif prediction == 'VIRGINICA':\n", " Z_virginica[x_index][y_index] = 1\n", "\n", "# Now, when we have x, y and z axes being setup and calculated we may print decision boundaries.\n", "for iris_type in iris_types:\n", " plt.scatter(\n", " x_train[(y_train == iris_type).flatten(), 0],\n", " x_train[(y_train == iris_type).flatten(), 1],\n", " label=iris_type\n", " )\n", "\n", "plt.contour(X, Y, Z_setosa)\n", "plt.contour(X, Y, Z_versicolor)\n", "plt.contour(X, Y, Z_virginica)\n", " \n", "plt.xlabel(x_axis + ' (cm)')\n", "plt.ylabel(y_axis + ' (cm)')\n", "plt.title('Iris Types')\n", "plt.legend()\n", "plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0" } }, "nbformat": 4, "nbformat_minor": 2 }