{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression (Regularised)\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In this example, we will implement regularized logistic regression\n",
    "to predict whether microchips from a fabrication plant passes quality assurance (QA). During QA, each microchip goes through various tests to ensure it is functioning correctly.\n",
    "\n",
    "Suppose you are the product manager of the factory and you have the\n",
    "test results for some microchips on two different tests. From these two tests,\n",
    "you would like to determine whether the microchips should be accepted or\n",
    "rejected. To help you make the decision, you have a dataset of test results\n",
    "on past microchips, from which you can build a logistic regression model.\n",
    "\n",
    "## Visualizing the data\n",
    "\n",
    "Before starting to implement any learning algorithm, it is always good to\n",
    "visualize the data if possible.\n",
    "\n",
    "The file 'data/ex2data2.txt' contains the dataset for our Logistic regression problem.\n",
    "\n",
    "Here we will load the data and display it on a 2-dimensional plot, where the axes are the two exam scores, and the positive and\n",
    "negative examples are shown with different markers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initial imports\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib notebook\n",
    "import seaborn as sns\n",
    "\n",
    "# setting graph properties\n",
    "plt.rcParams['figure.dpi'] = 300  # setting figure dpi for better quality graphs\n",
    "plt.rcParams['figure.figsize'] = [10,8]\n",
    "sns.set(context=\"notebook\", style=\"white\")  # graph styling using seaborn\n",
    "%config InlineBackend.figure_format = 'pdf'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports from my models designed for these examples\n",
    "from models.data_preprocessing import add_bias_unit, map_feature, feature_normalize\n",
    "from models.logistic_regression import cost_function, predict, gradient_descent, gradient_function, sigmoid\n",
    "from models.plotter import plot_decision_boundary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data ...\n"
     ]
    }
   ],
   "source": [
    "print('Loading data ...')\n",
    "data = np.loadtxt('data/ex2data2.txt', delimiter=',')\n",
    "X = data[:, :-1]  # (118, 2)\n",
    "y = data[:, -1, np.newaxis]  # (118, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Plotting data with + indicating (y = 1) examples and o indicating (y = 0) examples.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7ff2c54efa58>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAEMCAYAAAAWDss+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5wVdf348deygYjoNyEFFVBMedNDNMUgjXSBRLyGoYakYV/5YpppfmvLNk39ekGKNorUoNSARCuSIPEGsi6S/fISkKHwFk25yCVclEuILLvz+2PmLMPh7NlzdmfmzJzzfj4e++DszJyZ9w7nzHs+l/l8yhzHwRhjjAlDu0IHYIwxpnhZkjHGGBMaSzLGGGNCY0nGGGNMaCzJGGOMCc3HCh1AXIjIAcAAYAPQUOBwjDEmKcqBI4CXVfWj9JWWZPYaACwudBDGGJNQZwB/SV9oSWavDQAzZ86ke/fuhY7FGGMSYePGjVx++eXgXUPTWZLZqwGge/fu9OjRo9CxGGNM0mRsZrCGf2OMMaGxJGOMMSY0lmSMMcaExpKMMcaY0FiSMcYYExpLMsYYY0JjXZiNaUFDQwM1NTUsX76cfv36MXToUMrLywsdljGJYEnGmCwaGhoYO3YMGzcqFWceRnX1TGbOFB58cIYlGmNyYEnGmCxqamrYuFF5/M/DaN++HZWVjVxw4XxqamoYNmxYocMLlJXYTBisTcaYLJYvX07FmYfRvr37VWnfvh2DKw7ntddeK3BkwUqV2Kqrq9j14Xyqq6sYO3YMDQ02VqxpG0syxmTRr18/Fj2/mfr6RgDq6xupXfRvTjjhhAJHFix/ia2qqj+P/3kYGzaspKamptChmYSzJGNMFkOHDqV7d+GCC+dzzz1LuODC+RxxRF+GDh1a6NACVSolNhM9SzLGZFFeXs6DD86gsnICB3YaTmXlhKJs9C+VEpuJnjX8G9OC8vJyhg0bVnQN/X5Dhw5l5ky3xDa44nBqF/27KEtsJnqWZIwxTSW2mpoaXnvtNSorT7DeZSYQlmSMMUBplNhM9GKVZETkJ8DFwDHAiaq6PMM25cBk4BzAASao6gMtrTPGGBO9uDX8zwHOBFZn2eZy4DjgeOB04HYROSaHdaaVGhoaWLBgAZMmTWLBggX27EQL7HwZs1eskoyq/kVV17aw2Sjg16raqKqbcRPTpTmsM61Qig/ptSVJlOL5MiabWCWZHPVi35LOGqBnDutMK5TaQ3ptTRKldr6MaUkSk4yJUBwf0hs8eDCDBw8OZd9tTRJxPF/GFFISk8wa4Gjf772AtTmsM61Qag/ptTVJlNr5MqYlSUwys4BxItJORA4DLgL+mMM60wqlMqxKSluTRKmdL2NaErcuzJOBkUB34FkRqVPVE0TkSeBWVX0F+C3wWWCV97Y7VPVt73W2daYV4vKQnr96bNGiRfstq62tDeQ4bX3yPS7ny5i4KHMcp9AxxILX1fnthQsX0qNHj0KHY9JkSjIVFRVNy4JKMrB3XpXXXnuNE05ITpKw+WBMIaxbt44vfOELAL1V9Z309bEqyZjiFMTFz59EUgknyMTiF9WT70EmBZvB08RVEttkTILYcyOZBX1erOu0iStLMmY/QT6xbhe/zII+L9Z12sSVJRmzj6DvsMO4+NXW1oZWVRaVoM+LdZ1umQ33UxiWZMw+gr7DtotfZkGfF+s6nZ1V2xaONfybfWS7w25NQ7hNhpVZ0OfFuk5n5795at++HZWVjVxw4XxqampsaoOQWZIx++jXrx/V1TOprGykfft2TXfYlZWtu8O2i19mYZwXmw+meUHfPJncWZIx+wij5GEXv8zsvEQn6JsnkztLMmYfVvIwxciqbQvHkozZj91hm2JjN0+FY0nGmFYKe+SBJIrz0DZ281QYlmRMybHkEA4b2sZkYs/JGGMCYaM7mEysJGNMHqKaciCJrJuwycSSjIm1oKq2LDmEz7oJm0wsyRiTh6imHAh6GoAoGuOtm7DJxJKMKQlRzkfTVkE2oEfZGG/dhE0msUoyItIHmA50BeqAMaq6Km2bGcBJvkUnARep6p9F5HbgG8B6b90Lqnpd6IGbQJV61VaQ42xFPWZXUN2E49wV2uQnVkkGmALcp6oPi8gVwFRgn7K2qo5JvRaRTwM1wDO+TWaoamUUwZrSFlayC7IBPYmN8dYVurjEpguziBwO9Ace9RY9CvQXkcOyvG0sMFNVPwo7PhOd1HwxtbW1VFRUUFFRsc+yoPYfV0FOA5DEqRasK3RxiU2SAXoC76pqA4D373pv+X5EpAPwFeChtFWXicirIjJfRE4PM2BjwhDk3DBJnGcmqAndbJKyeIhTksnXRcAaVV3mWzYF6K2qJwETgbki0rUg0RnTSqkG9MrKCRzYaTiVlRNaXVUU5L6iEkTpyyYpi484tcmsBY4SkXJVbRCRcuBIb3kmV5FWilHVjb7XC0RkLdAPWBRSzIkWZuNqUPsOslorbr3KssUT5DhbSRuzK4iu0DZJWXzEpiSjqv8GlgGjvUWjgaWqujl9WxHpAZwBzExbfpTv9cnAMYCGFHKihXmnV+i7yMGDB+/TGy0u+0qqqM9BEKWvoKrcTNvFJsl4rgGuF5E3gOu93xGRJ0XkM77trgQeV9X3094/XkSWi8g/gF8DX/WXbpImzDrlMBtXreHWtFWq9HXjjTcybNiwvEvBSezwUKziVF2Gqq4EPpth+Xlpv9/dzPuvDCm0yIXdjTPMrq1x6jYbt2du4hZPIURRbWmjD8RHrJKM2SvsOuUwx5kqxBhWzV28ly1z+4WcfPLJbd5XSiklAkjmObDRB+LDkkxMhV0aCPNOL053kankUltby8c//vGm14WSpOFtki4OHR5s5AJLMrEVdmkgzDu9QtxFBnnxtkQQ/DlIesmoNWzkApclmZiKojSQ751ePhebONxFGtOSMEsa1o3aZUkmpqxOORj+u+WtW7futywuVWfFLo6lwyR3rkkSSzIxZqWB1gny4lXoC2EhpN/dL1y4sChvbpLcuSZJLMmYrJJelx7HO+g4K6V2hCR3rkmSFh/GFJFTRORaETkzw7obwwnLBMGeVjf5iuJB2riMgh32A5tJHDcuDFlLMiJyGe6gky8Cd4jI87hP0e/0NrkD+Fm4IZpCspJAacl0d3/mGZ/gO9/5Dq+++mqBo8tNro35cexcU4xaKsncApyvqsOB3kA9sEBEDvbWl4UZnDFBissddFuEXTrNdHf/zPy1dOjQIbRjBimfcfOspBGNltpkeqrqCwCqugN3rpZfALUicjbghB2gyU/S21BMYWW6u3///Ua6dDmo0KHlJN/GfCtphK+lksxmEfmkf4GqXg88CywG2ocVmImfYigJ5KvU2rVSd/d1dWU8+rt3qasrY/ly5fnnn286F3E+Hzb6cvy0VJJ5AnfE41v9C1X1JhHZBfwwrMBM61gbSv7ifp6iLp2Wl5fTuXPnQPcZFes2HD9Zk4yqfivLuttEJONoyMaY6ISRJJN6s2LdhuOnTc/JqOruoAIxLhtQr/Di1q7V0gU/l+qrJCWKtrCRMuLHHsaMkaAfhCv2C0pbZEsky5Yty2tqABMv1pgfL5ZkYqTYB9RLyt30ySef3BRjXGOOsrQVt7/dJEuskoyI9AGmA12BOmCMqq5K2+Z24BvAem/RC6p6nbeuE/Ab4FRgD1CpqvOiib7tbEC96CS1zSE9+WUTt2q/YmNV27nJKcmIyHpVPTLD8jWq2ivAeKYA96nqwyJyBTAVyNRiN0NVKzMsrwS2qepxInI8sFhEjvOe8Ym9MHrG2BehOCU1SQYhDn9vKY3x1la5lmQOTl8gIh8DAuvnKCKHA/2B1C37o8C9InKYqm7OcTejcLtco6qrROQV4FxgVlBxhinonjFx+CIEdTddqAtLki/cpZyIwlbsVdtBamnssgW4T/UfICLz01b3xB3TLCg9gXdVtQFAVRtEZL23PD3JXOaNOLARuE1V/5+3vBew2rfdGu/9iRB0z5jWfhFK7YJUKn+nCY5VbeeupZLMH71/zwAe8y13gE3A02EE1YIpwN2qWi8iw4C5IvIpVa0rQCyBC7JnTBy+CHY3Hb5SOJ9xa1+yhz5z19LDmFMBRORFVV0WcixrgaNEpNwrxZQDR3rL/TFt9L1eICJrgX7AItySy9HsLfn0Ap4LOe7YSvoXIW4XliSzc9V6mW6O7KHP3OXaJvNJEflQVdUby+x+oBH4pqq+FUQgqvpvEVkGjAYe9v5dmt4eIyJHqeq73uuTgWMA9VbPAr4OvOI1/A/w9lOS8vki2AXdxFncSsT20Gfuck0yPwI+772uBt4AduBWXQVZ73INMF1EbgXeB8YAiMiTwK2q+gowXkROBRqA3bjz26RKNxOBaSLyprf+alXdHmB8iRK3L0K+F4W4XViM8bOHPnOTa5I5XFU3isgBQAVwBO7cMrn2+sqJqq4EPpth+Xm+11dmef9/gEuDjCnpcv0i2AXdmL2sZB+cXJNMnYgcA5wI/F1Vd4nIgeQwfbMxxgTJLvDJkmuSGQ8sxe1Vdrm3bAiQjPlYTeIl7cJipcFks5J9cHJKMqr6axGZ5b3+wFu8jL0JxxQR+zIZY4KST3WXAwwTkdQcM3tw22WMMcaYjHIdu+xzwBzgddzBJ3+O+2zKDcBFoUVnTIJYY3Fxsv+3tsm1JDMZd0TkwbglGID/B5wWRlDGtFXc56I3plTk/DCmqqaGkHG8fz8COgQfkjHJZI3FJmjFMIp6riUZFZEhacsGA68FG44xxhjYO4p6dXUVuz6cT3V1FWPHjqGhoaHQoeUl15LMd4E/ichjQEcR+TnuQ48XhxaZyVup3z1bm4gpJsUynUBOJRlVXYzb4P8u7jwv7wODfEPsG2N8amtrLamZNsk2inqStDSfzBOqej6Aqq4G7ogkKlPUwipxWZuIKSZJH0U9paXqsjMiiaIIFKqBzqqIjClOxTKdQK5tMiaLOExzbIwpLnEbRb21WkoyHUVkRrYNVHVMgPEkUmsa6IIq+SSliijqElccz4Ex+SqG6QRaSjIOEMikZMUs32mOreRjTLDifINV6lpKMh+p6v9FEkmC5dtAVyxdE/ORlBKXMUEqhocp26qlJFMWSRQJl28DXb4ln1zZRduY+LAaC1dLSebhSKLwiEgfYDrQFajDHS9tVdo2PwQuw51euR74gao+462bBpwFvOdtPktV7w477nwb6Iqla6IxhRT3npWlWGORSdYko6rXRhWIZwpwn6o+LCJXAFOB9OLAS0C1qu4UkU8Di0TkCFX90Fs/QVXvjTBmIL8GumLpmthahf7yGxOFsGoskiY2XZhF5HCgP5A6+48C94rIYaq6ObVdqtTieRW3Sq8rsC6qWNuqWLomGlNIcW/nsxoLV2ySDNATeFdVGwBUtUFE1nvLNzfznjHAW6rqTzDfFpGv4/aKq1LVFWEG3Vpx7ZrYmi9rHL/gxhRaqddYpMQpyeRFRCqAO9lb8gG4Gdigqo0iMgZ4WkSOTSUuY4yJitVYuHJOMiJyFTAaOBJYD/wOeEhVnaxvzN1a4CgRKfdKMeXesdZmiOV03E4JI1RVU8tV9V3f6xkiMgnoAawOKEZjTAzFtRQd1xqLKOU6/fKPgRHAz3Av2L2ASkCA7wURiKr+W0SW4Sayh71/l/rbY7xYBgC/By5R1SVp645KJRoRGY7bA+1dTFat6aUT9549xph4yLUk8zWgv7/tQ0SeAJYQUJLxXANMF5FbcacTGOMd60ngVlV9BbgfOBCYKiKp931VVf/pvbcb0AhsA76oqnswJcfaiYyJh1yTzHbvJ33ZtiCDUdWVwGczLD/P93pAlvefFWQ8paI1vXTi3rPHGBMPuSaZnwGzRWQCblfhnrizZU4SkWNTG6nqv4IP0Rhj2sZuhAon1yTzc+/fIWnLvwBM9l47QGl1mzCxYu1ExsRPTklGVXOaptkkX2suxHbxNsY0J7HPyRiTztqJjJ+VbOOh2SQjIk+r6jne68W41WH7UdUzQ4rNGGNMwmUryfhnxHwg7ECMMSZIVrKNh2aTjKo+4ns9PZpwjAmGXUyMiYd8hpU5GzgZ6Oxfrqq3Bh2UMcaYcEU1a2euw8rcC3wZeA7YGXgUxhgTomIr2bY1QUQ5a2euJZmvAJ9W1f0GqzTGGBOdIBJElLN25vr8y3vAB4Ee2RhjTN78CaKqqj+P/3kYGzaspKamJud9ZJu1M2jNJhkROTb1A1QDM0XkdP9y/5AyxhhjwhdEgujXrx+Lnt9MfX0jQNOsnSecEPysndlKMm8Cq7x/fwlcALzg/f6mb70xxpiIBJEghg4dSvfu7qyd99yzhAsunB/arJ3ZujDbUDIFFlXvjzC15fkEe7bBmP0FMa1zlLN25tq77Chgp6q+71t2KHCgqq4PPCoTae8PY0xyBJUgopq1M9feZXOAq3AnEkvpgTsSwH7zv5i2i7L3hzGFYCXV1kvStM65Jpk+3syTTVT1nyLSN8hgRKQPMB3oCtQBY1R1Vdo25bjTC5yDO57aBFV9oKV1SZOtcS/fD1bUX+a2DExogxoas78kV53n2u6yWUSO8y/wfq8LOJ4pwH2q2ge4D5iaYZvLgeOA44HTgdtF5Jgc1iVKlL0/jDHxlao6r66uYteH86murmLs2DE0NDQUOrSc5FqSeQh4TERuBv4FfBK4kwAHzhSRw4H+QOo2/VHgXhE5TFU3+zYdBfxaVRtxk98c4FJgYgvrEiWIxr1CacvAhG0d1NCqYOLNSqr5S3rVea5JZgJQD/wEd+rltbgJ5qcBxtITeFdVGwBUtUFE1nvL/UmmF7Da9/sab5uW1iVKWxv37MtsTHEIsuq8EHKdGbMRtzSQuBJBkiWpcc+YXBRq+P0kt2n069eP6uqZVFY20r59u6aq88rKZFSd5zMK82BgDHAU8C7wW1V9LsBY1gJHiUi5V4opB470lvutAY4GXvZ+95desq0rKXGZS6Mtx8z1vVZqM9kk/XGAJFedQ44N/yLyP8AfgI3AbGAD8KiIjAsqEFX9N7AMGO0tGg0sTWuPAZgFjBORdiJyGHAR8Mcc1hljSlAQY30VUqrqvLJyAgd2Gk5l5YTEJEjIvSTzPWCYqv4jtUBEfg88Bvw6wHiuAaaLyK24z+SM8Y71JHCrqr4C/Bb32ZxU1+Y7VPVt73W2dSaBcqnmiEupzeQnqv+jpLdpQLKrznNNMl2B19OWKdAlyGBUdSUZHu5U1fN8rxuAa5t5f7PrSkn6hXnhwoWJuevxS3o1h4mHpLdpJF2uz8n8BfipiHQCEJGDcDsB/DWswEzrJL1PvV/SqzlMPEQ5GKTZX64lmWuA3wFbRWQLbgnmr+xtPzEBCKIHTNL71Pu1pprDqslMuigHg8xVknu75avFkoyIlAEHAl8AegMXAr1VtcIGxwxOUCWQKCcjCpuNemCCkmrTuPHGGxk2bFjBE0yx1DbkosUko6oO8E+gUVXXqepLqrou/NBKS1BVQ8V0YbZqDlOMSq0aONc2maVAnzADKXVBlUCK6cKc9K6bxmRSTLUNuci1TaYWeFpEpuE+HOmkVqjqQ8GHVXqC6gETx/rntkhy101jMim13m65JplBwNtARdpyB3fwTNNGQT7VG+aFuZQaLE1xKvSzVEl/gj9fZY7jtLxVCfCmBHh74cKF9OjRoyAxpC7gr732GiecEL8SSPpzK4ue30z37vbcikmWQicZiP93PR/r1q3jC1/4Argdwt5JX5/r9MtnA++o6hu+ZX2Ao1V1QUCxlry4Vw0VU/fouIjDBc9EL+7f9SDlWl12H3Bm2rId3nLrEFAiimF4DlOabBDVwsm1d9nhqrohbdkGoHvA8ZgYK6bu0a01ePDgfS5OJjhxP7dxjy+uci3J/EtEhqqqvyP3YNzOAKZERNlgWczVSHZXHT0bRLVwck0ytwOzReRB4C3c6Zf/2/sxJaLYukcbY8KX68yYc73G/6uA83GflRmuqi9nf6cpNnFssAz7zjTokofdVe8V91Jd1PEV4yMCOc+MqaovAS+FGIvJUTF+ECH+FxxTHOL6OSrWqS2aTTIicrOq3u29vqO57VT11jACM5kV6wcxzkqt5BHl3xj3cxtlfMX6iEC2koz/icSeYQdiclOsH0TI7wtdLKWepMRZLOKYyFKK9RGBZpOMql7rex1qA783GdpvgFOBPUClqs7LsN0I4FbgAKAMeEhVq711XwN+Brzjbf62qn4pzLgLoVg/iMaUumId0yxrm4yI9GppB6q6JoA4KoFtqnqciBwPLBaR41R1R9p2G4ELVXW9iPwX8HcReUlVF3vrn1XVSwKIJ7aK9YOYr0JVs8TxDjgIcSgZxv3chh1fsY5p1lLD/zvsHXG5LMN6BwiiIWAUcCWAqq4SkVeAc4FZ/o1U9UXf660isgI4GlhMQuXbiB/3D6L/gt+WDgpRXnDiXIVi2i4OCTQXxfqIQEtJ5h+4s2JOBx4GwpoJsxew2vf7GlpoBxKRvsBpwNd9iytEZBmwDfiRqj4RdKBBak0jflI+iNZBIVni3gBfKuL4iEBbZU0yqnqKiPTDLWW8AKwAZgCzVfXDXA8iIktwE0km3XLdj29/RwBzgW/4poCeB/xeVT8UkVOAp0RkiKquyHf/UWltI34SPohRdlCwi6HJxhJoYbX4nIyqLge+KyI3AcOArwH3ecPMLMnlIKraP9t6EVmDW+212VvUC3iumW0PB54FfqyqTdVpqvqe7/VSEXkBGIibGGOpWBrxM1VH1NXVcdmoI2P7tyWlCsWYpMt1gEyA43EnLTsddzrm9wOMYxZetZfX8D8AeDp9IxHpCiwA7lXVB9PWHeV7fTRuVdqrAcYYuCQMONnQ0MCCBQuYNGkSCxYsoKGhIaf3HXDAATwzf12s/zaTWW1trSVZE5isk5aJSBdgNG512cHAb4GHA+pR5j/OQcA04BSgAfieqs711t0BrFfVKSIyEfgmoL63/1xVfyMi44ERuF2gAX6qqtPziOEYIp60LNVusWHDyn0a8ePSbtGaScpSpYGFCxfG+m/zsyoUY1qvpUnLWkoyu3BHWv4t8LdM26SNzJxYhZoZM84z5C1YsIDq6qqmdpX6erddpbJyQrNVXpl6l8Xxb/OzJGNM67V1ZsyNQEdgnPeTzgGObWOMJS3OjfhtbTOK899mjIlGS73LjokoDhNDrXnwM4mlgSTGbExS5DwKsyk9cX/w0xgTf5ZkTLOS8uCnMSa+LMmYrIq1XaVY5+Qx8VaKnztLMqbk2JA3phBK9XOXz8OYxhQF/5A3VVX9efzPw9iwYSU1NTUMHjx4nyf/jQlKts9dMbMkY0pOtq7ZYYhL4opLHKUq6s9dXFiSMYFr7VA0UUnCcD6m+JTq587aZIpYIRoZk1DvnN41+6mnV/P++43cddddPP/884ANlhmFUmsEL9VHAizJFKlCXeyjHOK/tdK7Zu/c+TBduhxEWVmmeflaJy6jPMcljnRJuBkJWqk+EmBJpkgV6mKflOkL/F2zb7zxxqblNo5ZNJJwMxKGYn0kIBtLMkWqUBf71gxFU0jpVTaO4wRSoonLRFlxiSNdUm5GTNtZw3+RKlQj49ChQ+ne3a13vueeJVxw4fzY1junqmyqq6vY9eF8qqur2LJlE9lGJjfBKNVG8FJkJZkiVahGxiTVOzdXZVNZeUuhQyt6pdoIXoqyzidTSgo1n0yYkjKfS6FMmjSJXR/Op6pq7+zg99yzhAM7Dd+nncaEwz6fxaGt88mYBCvFRsZ8JK39qNjY57M0xCLJiEgn4DfAqbjTJ1eq6rwM2w0GngTe8BZ9pKqf9a3/IfA179dpqnpniGGbhLMqG2PCF4skA1QC21T1OBE5HlgsIsep6o4M276uqp9JXygiZwKXAv28RS+KyCJVfT68sE2SJan9COLVO8yYXMUlyYwCrgRQ1VUi8gpwLjArz33MUNUPAURkhrfMkoxpllXZJIMl2OSKS5LpBaz2/b4G6NnMtn1EZAlQD9yvqtN9+6hN28eZAcdpjAlQHJJH2DGU2vA56SJJMl5S6NXM6m557GoJ0FNVt4pIb+BZEXlXVZ9tc5BZ1NfXs27dOnbt2hXmYQzQsWNHevToQfv27QsdSizEdVgYk5tSHD4nXSRJRlX7Z1svImuAo4HN3qJewHMZ9rPN9/ptEZkDDAKexS25HO3bvBewtm2Ru9atW8fBBx/MMcccE+j4VmZfjuNQV1fHunXr6N27d6HDMQVWDAm2VIfP8YtLddks4OvAK17D/wBgdPpGInIEsFFVHRHpApwN3OLbx2QRuc/7fQxwfRDB7dq1yxJMBMrKyujatSubN29ueeMSEddhYZqTS4xxSB5hxJCpWsyGz4lPkpkITBORN4EG4GpV3Q4gIncA61V1CnAxcK2I1OPGPl1V5wKoaq2IzAZSMwDNUNVFQQVoCSYadp5NSpISbHPVYqNHj2HSpNJ+FisWSUZV/4Pb/TjTult9r+8F7s2yn9uB2wMOL5a2bt3KGWecwZe//GVuuSXaYVBWrFjB22+/zXnnnZf3e9etW8fFF1/Miy++GEJkJgnikDyCjqG5ajGgaSy/Un0WKxZJpliF+QWaN28en/70p3niiSf43ve+R4cOHQI/RnNWrFhBbW1tq5JMsShEj6HU56jQvZXSj3/XXXc1lUCT2nbSVs1Vi61YsSJRz2KFwZJMQj322GN897vfZerUqSxcuJBzzz2X3bt3M2nSJBYvXky7du3o2bMn993nNlFNnTqVefPmUVZWRqdOnXjkkUdo164df/rTn3jkkUdoaGigc+fO3H777Rx77LHMnj2bxx9/nAMOOIA1a9bwiU98gokTJ9KhQwcmT57Mjh07GDFiBAMGDOCWW27hH//4Bz/5yU/4z3/+A8ANN9zQdKGZOXMm06ZNo3PnzlRUVBTqlAWmkD2Gcj12WDc4mY6/ZcsOunTpFnhVpz+Z3XzzzbG++882RFHJP4vlOI79OA59+vQ5pk+fPs7atWuddK+//vp+y3JRUVHhVFRUtOq92axYscIZMmSI09jY6MydO9cZO3as4ziO84tf/MK57rrrnI8++shxHMepq6tzHMdxZs+e7Xz5y192tm/f7rozULwAABP7SURBVDiO42zZssVxHMd5+eWXnXHjxjVtX1tb64waNcpxHMd57LHHnBNPPNF56623mvZ9/fXXN61LvXYcx9m6daszYsQIZ9OmTY7jOM6mTZucM844w9m6dauzYsUKZ9CgQc7mzZsdx3Gc2267zRk4cGDWv6+15zsq8+fPd4YPP9XZ/dH3HafxB87uj77vnH12f2f+/PmxOXZYn72Wjh/Ucffs2eNceeVXnOHDT3XG332OM3z4qc6VV37F2bNnT5v3HYZUvGef3d8Zf/c5ztln9491vEFau3at06dPH6dPnz7HOBmurVaSCVgUPWf++Mc/MmLECMrKyjj77LO566672LRpE8899xzf//73m6rOunTpAsBzzz3H6NGj6dy5MwCHHnoo4NYjr1y5kksvdZvDHMdh27amXuKceuqpHHvssQBceumlXHjhhRnjWbp0KevWrWPcuHFNy8rKyli9ejVLly5l8ODBfOITnwBg1KhRPPXUU20+B4VUyB5Dhe6tFNXxk9b1N2lDFEXJkkzC7N69m3nz5tGhQwfmzp0LuA+Lzp49O+99OY7DxRdfzLe+9a02xeQ4DiLCzJkz91u3dOnSNu07jgo5enO2Y0dxgxPV317oZNoaJV8t1gybGTNgtbW1TT8VFRVUVFTss6ytFi5cSO/evXn++eepqamhpqaGhx56iD/96U8MGTKE6dOns3v3bgC2bNkCwJAhQ3j00UfZscMdb/T9998H3FGI586dy8aNGwG3Dnz58uVNx1qyZAnvvPMO4LYBnXbaaQB07tyZ7du3N213yimnsHr1av72t781LXv11VdxHIeBAweyaNEi6urqALcUlnSFnP2z0DOPtnT8oD7nNnNm8bCSTMI89thj+1VbnXLKKTQ2NjJw4EC2b9/ORRddRPv27Tn66KOZPHkyF110EZs2bWLUqFF87GMfo1OnTsycOZMBAwZw4403cu2119LQ0EB9fT3nnHMO/fq5A1n379+fH/3oR6xevbqp4R/g9NNP56GHHuKLX/wiAwcO5JZbbuH+++9n4sSJjB8/nvr6enr27MmUKVPo27cv11xzTVN13ZlnBjucXCF6WhWyaiTbsaPoGhzV327TMBQPmxnTk21mzBUrVvCpT30q733G/QGybGbPnk1tbS2TJ0+O/Ni5nu/0nk6Lnt9M9+6lNS5Uc5L82UuxmTOTwWbGLKAkf8GTIGmNwyY/1sZRHCzJmIxGjhzJyJEjCx1GVklsHI6K3eCYuLCGf5NY1jhsTPxZScYkljUOGxN/lmRMYtkDcMbEnyUZk2jWOGxMvFmbjDHGmNBYkkmgoUOHcs455/DFL36Rc889l1mzZkUew4svvshf/vKXVr837j3XjDHBsOqyhJo8eTJ9+vThjTfeYOTIkZx55pl069YtsuO/9NJL7Ny5k89//vORHdOYbAo9z47JLBZJRkQ6Ab8BTgX2AJWqOi/DdjcAV/kWHQs8oKrfFpHBwJPAG966j1T1s6EGnkVUH/g+ffpwyCGHsGnTJrp168avfvUr5s+fT0NDA926dePOO+/ksMMOY/v27fzgBz9g1apVdOvWjW7dutG1a1duuummpnloXn75ZXbv3o2IcPvtt3PQQQexfft2xo8fz/LlyykrK+Mzn/kMo0aN4ne/+x2NjY389a9/5fzzz+fqq69m0aJF/PKXv2T37t20b9+eqqoqTj75ZAAmTZrEk08+ySGHHMLAgQMDPw+mtBVyjh+TXSySDFAJbFPV40TkeGCxiBynqjv8G6nqZGAygIi0B94FHvFt8rqqfiaqoJsT5Qf+73//O4ceeih9+/Zl7ty5rF27lj/84Q+0a9eORx55hAkTJlBdXc19993HIYccwtNPP80HH3zAyJEjGT58OAAPPPAABx98cNPglRMnTuRXv/oV//u//8v48ePp1KkTc+fOpV27dmzZsoUuXbpw2WWXsXPnTm666SYA1qxZw/3338+DDz5I586dWbVqFePGjaO2trZpIM85c+bQsWNHrrvuukDPQbGwO/HWs9Ef4isuSWYUcCWAqq4SkVeAc4FsjQ0XAhtU9ZUI4stLFB/4G264AcdxWLNmDT//+c/p0KFD0wXqS1/6EkDTbJfgtoPccsstAHz84x/nrLPO2ifeHTt28MwzzwDudAJ9+/YF3LloZs+eTbt2bvNdao6adIsXL2bNmjVcfvnlTcv27NnDe++9x4svvsh5553HQQcdBMAll1zC/fffH8h5KBZxuBNPcpKz0R/iKy5Jphew2vf7GqBnC++5CreKza+PiCwB6oH7VXV6cCHmLooPfKpN5qmnnqKqqor+/fvjOA7XXnstl1xySV77chyH2267jdNPP71NMZ1xxhn8+Mc/btM+SlWh78TjkOTaopBz/JjsIuldJiJLROS9Zn7y/gSLyBHAUOBh3+IlQE9V7Q9cBtwqImdlen/Yohzu5Nxzz2XQoEFMnTqVoUOH8sgjj7B161bALZGsXLkSgIEDBzZNcrZt2zYWLlzYtI+hQ4cybdo0du3aBcCOHTt46623AHcumgcffJDUaN2pOWrS55QZNGgQixcvZtWqVU3LXn31VQBOO+00nnrqKXbu3ElDQwOPPfZY4Och6bLdmETBn+Sqqvrz+J+HsWHDSmpqaiI5flsVep4d07xISjLehb9ZIrIGOBrY7C3qBTyX5S1XAk+q6nu+Y2zzvX5bROYAg4BnWxt3a0U93Ml3vvMdRo4cybhx4/jggw+44oorALeEMnr0aPr27ct1111HVVUV55xzDocddhj9+vVrqkq7+uqruffee7nkkksoKyujrKyMb37zm3zyk5+kqqqK8ePHc8EFF1BeXt40f8xZZ53FnDlzGDFiRFPD/8SJE7n55pvZtWsX9fX19O/fn5NOOokhQ4awbNkyRowY0dTwv2nTplDORVIV+k486dVNNvpDfMViPhkRuR04SlXHeQ3/fwGOU9XtzWyvwP+q6pO+ZUcAG1XVEZEuwCLgFlWdm2MMxxDgfDJxmwujvr6exsZGDjjgAHbs2MHo0aOpqqric5/7XMFiak5r5+9JslR11YYNK/e5MYmqumrBggVUV1c1VdfV17vVdZWVExKRZEzhJGU+mYnANBF5E2gArk4lGBG5A1ivqlO83wcBnYFn0vZxMXCtiNTj/l3Tc00wYYjbcCfbtm1j3LhxNDQ08NFHH3HBBRfEMsGUqkLfidtgoyYssSjJxEEYM2Oa1rHzXRhxK32bZEhKScYYU2BxK32b4mBjl+XISnzRsPNsTHGxJJODjh07UldXZxfAkDmOQ11dHR07dix0KMaYgFh1WQ569OjBunXr2Lx5c8sbmzbp2LHjfm1ixpjksiSTg/bt29O7d+9Ch2GMMYlj1WXGGGNCY0nGGGNMaKy6bK9ygI0bNxY6DmOMSQzfNTPjQ1WWZPY6AthnqHpjjDE5OwJ4K32hJZm9XgbOADbgDm1jjDGmZeW4CeblTCttWBljjDGhsYZ/Y4wxobEkY4wxJjSWZIwxxoTGkowxxpjQWJIxxhgTGksyxhhjQmNJxhhjTGjsYcwIiEgn4DfAqcAeoFJV52XY7gbgKt+iY4EHVPXbIjIYeBJ4w1v3kap+NuL4ssYgIj8Evub9Ok1V74wwthHArcABQBnwkKpWe+u+BvwMeMfb/G1V/VIbYuoDTAe6AnXAGFVdlbZNOTAZOAdwgAmq+kBL64KQY3w/BC7DffC4HviBqj7jrZsGnAW8520+S1XvjjC224FvAOu9RS+o6nXeupw+DyHHNwM4ybfoJOAiVf1zttgDiO0nwMXAMcCJqro8wzYF+9w1x0oy0agEtqnqccCFwAMi0jl9I1WdrKonq+rJwABgF/CIb5PXU+uDSjD5xJctBhE5E7gU6Of9XOotiyq2jcCFqtoP+BxwrYic4Vv/rC/uVicYzxTgPlXtA9wHTM2wzeXAccDxwOnA7SJyTA7rgpBLfC8BA1T1JNwbm9+LyIG+9RN85yuQBJNHbAAzfMf3X6Tz+ayGEp+qjvF9T68E3geeySH2tpoDnAmszrJNIT93GVmSicYovA+rd1f0CnBuC++5ENigqq+EHBu0Lr5M+5ihqh+q6ofADG9ZJLGp6ouqut57vRVYARwdwPH3ISKHA/2BR71FjwL9ReSwDHH/WlUbVXUz7gXi0hzWRRKfqj6jqju9X1/FLf11DSKGtsbWgiA+q0HGNxaYqaofBRFDNqr6F1Vd28JmBfncZWNJJhq92PfuYw3Qs4X3XIVbLeDXR0SWiMiLInJlgeJrLobW/I1BxwaAiPQFTgNqfIsrRGSZiDwvIue3IZ6ewLuq2gDg/bs+Q0zZ4g7rXOUTn98Y4C1VXedb9m0R+aeIzBGRTxUgtstE5FURmS8ip/uWx+bciUgH4CvAQznGHoVCfe6aZW0yARCRJbj/gZl0a8X+jgCGsrd9A2AJ0FNVt4pIb+BZEXlXVZ+NML5WxxBBbKn9HQHMBb6RKtkA84Dfq+qHInIK8JSIDFHVFa0KuoiISAVwJzDMt/hm3FJ0o4iMAZ4WkWNTF98ITAHuVtV6ERkGzBWRT6lqXUTHz9VFwBpVXeZblpTYI2NJJgCq2j/behFZg1t1s9lb1At4LstbrgSeVNVUwyuqus33+m0RmQMMAlq8wAcVXwsxpPaR0gtoqWgf6LnzqjueBX6sqrN8x/Cfx6Ui8gIwELdKLV9rgaNEpFxVG7zG1CPZ/29NxZ0amdZ/F5ltXVvlGh/eXfbDwAhV1dRyVX3X93qGiEwCegQQY06xqepG3+sFIrIWt51vEXvPXa7fpcDj87mKtFJMC7FHoVCfu2ZZdVk0ZgFfBxCR43Eb9Z/Osv1/k/bhFZEjRKTMe90FOBtYluG9ocXXQgyzgDEicqDXgDwG+EOEsXUFFgD3quqDaeuO8r0+Grcq7dXWBKOq/8b9m0d7i0YDS7067vS4x4lIO69O/yLgjzmsa5Nc4xORAcDvgUtUdUnaOv/5Go7bA+1d2iiP2PzHPxm3N1UqCeb7XQo8Pu/YPXCnBpmZR+xRKMjnLhsryURjIjBNRN7E/cJerarbAUTkDmC9qk7xfh8EdGbf3irgdl28VkTqcf/fpqvq3IjjazYGVa0VkdnAa94+Z6hqEHdvucb2faAP8HUR+br33p+r6m+A68Tt4rzHW/4DVV3ahpiuAaaLyK24PYvGePE8Cdzqddb4LfBZINX99Q5Vfdt7nW1dEHKJ737gQGCqiKTe91VV/af33m5AI7AN+KKq7iEYucQ2XkROxf3/3u3FlSohNPt5iDA+cGsbHlfV99Peny32NhGRycBIoDtuVXWdqp4Qo89dRjafjDHGmNBYdZkxxpjQWJIxxhgTGksyxhhjQmNJxhhjTGgsyRhjjAmNJRljfERkirgjFEd5TEdEjmtm3eUiMj/KeIwJknVhNiVBRN7BfXr7SP8IACKyFDgZ6K2q7xQoNgc4XlXfDHCfU4ArvF874A6AmRrEcbGqtmpQSRG5BvcBzrOybHM58E3c87pIVc9pzbFMcbCHMU0peRv3Ke5fAIjIiUCntu7UGwWhTFUb27qvoKjqNbgPFqbmZzlOVa/I+qbg1AHVwCm4876YEmZJxpSS3+I+wf0L7/crcackuCu1gbgTdq1T1Vu830cA/4c7gdxm4DpVfVpEaoEXgMG4w8OfKCI7cQdI/DywBfiRqv7a2085cBPu0PCH4078dpFv6PazROQp4DDcoUq+qaqOuBOu/Y+qft7bjwN8C7gROAR3pO6bWpPgxJ1v5yeAAP8CrlfVF7x143AHyuzq/d03AW/iTv72MRHZAexQ1e7p+1XVp7197LfOlB5LMqaU/A34qrhD17+BOzPkIHxJxk9EBuImoUuAhcARwMG+Tb6KO5eJ4lZHLQSW41bL9QUWiMhbqloDfBu3FHWed+yTgJ2+fV2AOw7XIcDfgcdpfkyuLwGfwR1+6Fnv+HnNcCjuZFVzcOcYqcGdLXGOuDNDgjt8y6mq+paIHAn8l6quEJEbaaG6zBg/SzKm1KRKM4twR2HONvDjWNxpnBd4v6dvO01VXwMQkZ64Cet8Vd0FLBORB7xj1QD/A3zPN9rxP9L2NUFVPwA+EJHncNszmksyP1LVLcAWEfkZbvLKdxrdK4HZvmkanhSR13EHPU11NOgn7lQO69k7nbAxebHeZabU/BZ3oqmv4ZZSsukJvJVlvX8I+COBLWmDNa4GUqPytrQv/yCKO3FLKbkcd7V37HwdDVwhIh+kfnBLR0d6gz5eDtwAbBSRPzfX+82YllhJxpQUVV0tIm/jVluNbWHztcAns6z3d81cD3QRkYN9iaYXe0s/qX0tzz/q/fRk72jXvWhdKWMt8ICqXp9ppao+ATwhIp2AHwO/xJ3YzLqjmrxYkjGlaCxwqKr+R0SyfQceBOaLyDzcibGOAA5W1ZXpG6rqWhH5K3CPiFTiTjswFrdEAG511p1eldSbwIm4U/22ZsbE74rIi7ilnW8BP23FPqYDfxV34rla3G7On8NNXuW41XXP4XZ73oE77D/AJqCniLRX1fpMO/Y6ObTHvb60E5GOwJ4ApwswCWLVZabkqOpbvnlBsm33Eu4EcpOArbjtOEdnecto3Emq1gN/Am7ztXn8FHcSt/m4c7Q8iDufS2vMxe0csAx4wttXXlT1X7jzA/0f8B5utdu3cK8J5bjz82zE7Y48APe5F3Dbid4B/i0i65rZ/TjgQ9zzNsx7fW++MZriYA9jGpMgYTy4aUyYrCRjjDEmNJZkjDHGhMaqy4wxxoTGSjLGGGNCY0nGGGNMaCzJGGOMCY0lGWOMMaGxJGOMMSY0lmSMMcaE5v8DhqS4yfSdi5gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Plotting data with + indicating (y = 1) examples and o indicating (y = 0) examples.')\n",
    "\"\"\"\n",
    "Example plotting for multiple markers\n",
    "x = np.array([1,2,3,4,5,6])\n",
    "y = np.array([1,3,4,5,6,7])\n",
    "m = np.array(['o','+','+','o','x','+'])\n",
    "\n",
    "unique_markers = set(m)  # or yo can use: np.unique(m)\n",
    "\n",
    "for um in unique_markers:\n",
    "    mask = m == um \n",
    "    # mask is now an array of booleans that van be used for indexing  \n",
    "    plt.scatter(x[mask], y[mask], marker=um)\n",
    "\"\"\"\n",
    "fig, ax = plt.subplots()\n",
    "y_slim = y.ravel()\n",
    "# plotting y=1 values\n",
    "ax.scatter(x=X[y_slim == 1, 0], y=X[y_slim == 1, 1], marker='+', c='black', s=50, label='Accepted')\n",
    "# plotting y=0 values\n",
    "# X[y_slim == 0, 0] is logical indexing with rows with y=0 only\n",
    "ax.scatter(x=X[y_slim == 0, 0], y=X[y_slim == 0, 1], marker='o', c='xkcd:light yellow', s=25, label='Regected', edgecolor='k')\n",
    "\n",
    "# labels\n",
    "ax.set_xlabel('Microchip Test 1')\n",
    "ax.set_ylabel('Microchip Test 2')\n",
    "\n",
    "# Specified in plot order\n",
    "ax.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Figure shows that our dataset cannot be separated into positive and\n",
    "negative examples by a straight-line through the plot. Therefore, a straightforward application of logistic regression will not perform well on this dataset\n",
    "since logistic regression will only be able to find a linear decision boundary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Mapping\n",
    "\n",
    "One way to fit the data better is to create more features from each data\n",
    "point. In the function map_features(), we will map the features into\n",
    "all polynomial terms of $x_{1}$ and $x_{2}$ up to the sixth power.\n",
    "\n",
    "$$\n",
    "mapFeature(x, 6) = \n",
    "\\begin{bmatrix} \n",
    "1 \\\\\n",
    "x_{1} \\\\\n",
    "x_{2} \\\\\n",
    "x_{1}^{2} \\\\\n",
    "x_{1} x_{2} \\\\\n",
    "x_{2}^{2} \\\\\n",
    "x_{1}^{3} \\\\\n",
    "\\vdots \\\\\n",
    "x_{1}x_{2}^{5} \\\\\n",
    "x_{2}^{6}\\\\\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "As a result of this mapping, our vector of two features (the scores on\n",
    "two QA tests) has been transformed into a 28-dimensional vector. A logistic\n",
    "regression classifier trained on this higher-dimension feature vector will have\n",
    "a more complex decision boundary and will appear nonlinear when drawn in\n",
    "our 2-dimensional plot.\n",
    "While the feature mapping allows us to build a more expressive classifier,\n",
    "it also more susceptible to overfitting. In the next parts of the example, we\n",
    "will implement regularized logistic regression to fit the data and also you will be able to see how regularization can help combat the overfitting problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adding Polynomial Features\n",
    "\n",
    "# Note that mapFeature also adds a column of ones for us, so the intercept term is handled\n",
    "X = map_feature(X, degree=6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cost Function and Gradient\n",
    "\n",
    "The **regularized** cost function for logistic regression is:\n",
    "\n",
    "$$\n",
    "J(\\theta) = \\frac{1}{m}\\sum_{i=1}^{m}\\left[ -y^{(i)} \\log\\left( h_{\\theta} \\left( x^{(i)} \\right) \\right) - \\left( 1 - y^{(i)} \\right) \\log \\left( 1 - h_{\\theta} \\left( x^{(i)} \\right) \\right) \\right]+\\frac{\\lambda}{2m} \\sum^{n}_{j=1}\\theta_{j}^{2}\n",
    "$$\n",
    "\n",
    "Note that you should not regularize the parameter θ0. In Octave/MATLAB, recall that indexing starts from 1, hence, you should not be regularizing\n",
    "the theta(1) parameter (which corresponds to θ0) in the code. The gradient\n",
    "of the cost function is a vector where the j\n",
    "th element is defined as follows:\n",
    "\n",
    "$$\\begin{align}\n",
    "\\frac{\\delta J(\\theta)}{\\delta \\theta_{j}} = \\frac{1}{m} \\sum_{i=1}^{m} \\left( h_{\\theta} \\left( x^{(i)} \\right) - y^{(i)} \\right)x_{j}^{(i)} && for j=0\\\\\n",
    "\\frac{\\delta J(\\theta)}{\\delta \\theta_{j}} = \\left(\\frac{1}{m} \\sum_{i=1}^{m} \\left( h_{\\theta} \\left( x^{(i)} \\right) - y^{(i)} \\right)x_{j}^{(i)}\\right)+\\frac{\\lambda}{m}\\theta_{j} && for j >= 1 \\\\\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "Note that while this gradient looks identical to the linear regression gradient, the formula is actually different because linear and logistic regression have different definitions of h$_{θ}$(x)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost at initial theta (zeros): [[0.69314718]]\n",
      "Expected cost (approx): 0.693\n",
      "Gradient at initial theta (zeros) - first five values only:\n",
      "[8.47457627e-03 7.77711864e-05 3.76648474e-02 2.34764889e-02\n",
      " 3.93028171e-02]\n",
      "Expected gradients (approx) - first five values only:\n",
      "[0.0085 0.0188 0.0001 0.0503 0.0115]\n",
      "Cost at test theta (with lambda = 10): [[3.16450933]]\n",
      "Expected cost (approx): 3.16\n",
      "Gradient at test theta - first five values only: [0.34604507 0.19479576 0.24438558 0.18346846 0.19994895]\n",
      "Expected gradients (approx) - first five values only:\n",
      "[0.3460 0.1614 0.1948 0.2269 0.0922]\n"
     ]
    }
   ],
   "source": [
    "# initial sizes\n",
    "m, n = X.shape\n",
    "\n",
    "# Initialize fitting parameters\n",
    "initial_theta = np.zeros([n, 1])\n",
    "\n",
    "# Compute and display initial cost and gradient\n",
    "cost = cost_function(initial_theta, X, y, regularized=False)\n",
    "grad = gradient_function(initial_theta, X, y, regularized=False)\n",
    "print('Cost at initial theta (zeros): {}'.format(cost))\n",
    "print('Expected cost (approx): 0.693')\n",
    "print('Gradient at initial theta (zeros) - first five values only:')\n",
    "print(grad[:5])\n",
    "print('Expected gradients (approx) - first five values only:')\n",
    "print('[0.0085 0.0188 0.0001 0.0503 0.0115]')\n",
    "\n",
    "test_theta = np.ones([X.shape[1], 1])\n",
    "cost = cost_function(test_theta, X, y, lamda=10, regularized=True)\n",
    "grad = gradient_function(test_theta, X, y, lamda=10, regularized=True)\n",
    "\n",
    "print('Cost at test theta (with lambda = 10): {}'.format(cost))\n",
    "print('Expected cost (approx): 3.16')\n",
    "print('Gradient at test theta - first five values only: {}'.format(grad[:5]))\n",
    "print('Expected gradients (approx) - first five values only:')\n",
    "print('[0.3460 0.1614 0.1948 0.2269 0.0922]')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning  parameters using scipy.minimize\n",
    "\n",
    "In earlier examples, we found the optimal parameters of a linear regression model by implementing gradent descent. I also wrote a cost function and calculated its gradient, then took a gradient descent step accordingly.\n",
    "\n",
    "This time, instead of taking gradient descent steps, we will use an function called minimize from scipy.optimize module.\n",
    "\n",
    "The scipy's minimize() function provides a common interface to unconstrained and constrained minimization algorithms for multivariate scalar functions in scipy.optimize.\n",
    "\n",
    "For logistic regression, we need to optimize the cost function J(θ) with parameters θ. Concretely, we are going to use minimize to find the best parameters θ for the logistic regression cost function, given a fixed dataset (of X and y values).\n",
    "You will pass to minimize() the following inputs:\n",
    "- Our predefined cost_function which returns cost while taking X and theta as arguments.\n",
    "- A gradient_function which returns the derivatives of the $\\theta$ values passed to it.\n",
    "- The initial values of the parameters we are trying to optimize.\n",
    "- A function that, when given the training set and a particular θ, computes the logistic regression cost and gradient with respect to θ for the dataset (X, y)\n",
    "- A callbak function which is called after each iteration and the $\\theta$ value obtained after each iteration is passed to it, we are using this callback function to store theta values for each iteration.\n",
    "\n",
    "The minimize() function returns a OptimizeResult object which contains final theta values, function end status, final cost etc.\n",
    "more info about the minimize function can be refered to the documentation <a href=\"https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html\">here</a>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost at theta found by Gradient descent: [0.52900273]\n",
      "theta: [ 1.27271026  1.18111686 -1.43166929 -0.17516292 -1.19271298 -0.45645981\n",
      " -0.92467487  0.62529965 -0.91743189 -0.35725404 -0.27469165 -0.29539514\n",
      " -0.14389149 -2.01987399 -0.36553118 -0.61558557 -0.27778948 -0.32742404\n",
      "  0.12393227 -0.05098418 -0.04466178  0.01555759 -1.45817009 -0.20603302\n",
      " -0.29244866 -0.2421784   0.02779373 -1.04319154]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(<Figure size 432x288 with 1 Axes>,\n",
       " <matplotlib.axes._subplots.AxesSubplot at 0x7ff2c54487f0>)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEMCAYAAADqG+D0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfXRcd33n8fedkSzJz08ayY/xo77jJLbskUOgSQr0hMddWCCw0M0hsLslhLT1OaXsZjekhOVs2BTS0oSH054AJdBAz6al6W7LAk0hEIeEA352gr+24/jZjizZjh8lS5rZP+6VPFb0ONbMnZE+r3Mc3fu79879SrHno/u7d36/IJfLISIiMlqJuAsQEZHKpAAREZGCKEBERKQgChARESmIAkRERApSFXcBpWBmNcANwDGgJ+ZyREQqRRKYB/zK3Tv7b5wQAUIYHs/EXYSISIW6BdjYv3GiBMgxgMcff5zGxsa4axERqQjHjx/n9ttvh+g9tL+JEiA9AI2NjSxcuDDuWkREKs2AXf+6iS4iIgVRgIiISEEUICIiUhAFiIiIFEQBIiIiBVGAiMiY0fQQE8tEeYxXRIbQ3ZPlQkc3Fzq6uNjZzfmLXVzo7O5rG/hrNxc6r2yrqU5y4/WN3Ny8gOaV9VRX6XfU8UwBMgJH287R3Z0lCIIr2oOAvrag7z8QENBvV4IgoK8pCPfpfY38rwPtO9Br9p03IG/fy7UMuG/0n2QiIAgCEkFwxfcglacnm+Ni51Bv8tHXKBR69z3f0c3FaNv5jm4udQ0/wk8iETCltorJtdVMjr7Onl7Lwvre9Sraz3Tw3I5j/OuvDjGlrprXK0zGNQXIMH6x/Sj/67FfxV1GUSWiEElEwZJMkBcwQRQ44RtI3z7B5bb++wy5PQgIEuFrJ4KAROLyuXvbggR52/POnb+eF4C95+h7/b5gvLze+z3mLycCCBL9ttNbz0D7Xw7c3mPz2wLoqz9//yuPe23N5Ije2MPf6M9f7OZiZ++bexcXo68XOrqvXO7s4mLnCN74A6jrfdOvCd/4Z0yZxLw5U/qCYEptFXW1VUyuqWZKXfi1rraKKXXVTK4Jt9VUJ0f0y0ZXdw9bdp/g2W1HeT4vTG68rpGbm+eztimlMBknFCDDuOHaRu796Ovo7s4CkCPs483loK+3N5frWw67gHN5y69tu3Lf6FV7983bkMvfZ4C2HJdfbLDz558DIJsL+6mz2RzZHGSzuXA9r61vezZqv+KYXHQMedv7rQ+wvSfb//gc2Sx5xw9QU+/6ANt7z5nr3da3PPL/t5Wgrqaq702+97f8ubPqmFyT/+Ze3XdlUFdb9ZqrhNpJI3vjHyvVVUled20jr7u2ka7uHrbuPsHGbUf55c5j/OTXh5hSW8WN18/jpub5rGuqp7oqWbLaZGwpQIZRXZXgDavnxV2GjFAuP1ii9WwOcv1Cpy8U89uy/Y+7HIS9r5u94mvechSGkLctO8T+eTVAFBQ11UyOfvufXFtFXU0ViURldy9WVyW54dpGbri2ka7uLNv2nGDjtiM8v/N4X5i87rqwm2udKUwqjQJExpW+biEq+413PKquSrB+VQPrVzX0hcmz247y/M5j/HTTYSb3hsma+ayzFJOqFSblTgEiIiWXHyZ3dzezfe8JNm4Nw+TpTYepq6nixusaual5PhmFSdlSgIhIrKqrErSkG2hJN/D7Pc1XXJk8vTkMk9dd28jNaxUm5UYBIiJloyp5OUzufn8z2/e0RfdMjvGzLZfD5Kbm+WTSKWoUJrFSgIhIWapKJsikU2TSqTBM9rbx7LajPLejN0zCG/Q3N88nk25QmMRAASIiZa8qmSBjKTKW4hO3rWHH3jae3X6UX2w/xs+3HAnDZFV4ZdKySmFSKgoQEakoVckE6yzFOkvxifetYcdLbWyMrkx+vvUItZPCK5ObmufTkk5RO0lvc8Win6yIVKxkMsHaphRrm8Iw2flSOxu3H+W5HUd5JgqT9asauLl5AS2rFCZjTT9NERkXkskEzU31NDfVc9d7V7NzX3vfPZON245S0xcm81mfbqC2Rm9/V0s/QREZd5LJBM0r62leWc/H37eGF/ZF3Vzbj/Fsb5ikG7ipeT43rFKYFEo/NREZ15KJgDUr6lmzop6Pv3cNL+5rZ+O2I/xixzGe3X6USdVJblilMCmEflIiMmEkEwGrV8xl9Yq53PneNbz4cjsbt14ZJutXpbh5zQJuvL5RH1ochgJERCakZCJg9fK5rF5+OUye3XaUX0SPB7+5ZSGf/A8tcZdZ1hQgIjLh5YfJx96zmof/djPP7zxOd0+WqqTmLhmMfjIiInmSiYA3rJ7Hxc5udu0/GXc5ZU0BIiLSz5oV9SQTAZu9Ne5SylpJurDM7CHgNmAJsNrdd5rZEuDJvN1mAtPdffYAxyeBR4C3E06t96C7f73YdYvIxDSlrpr0ktls2tXKHe+8Nu5yylap7oE8CTwMPNPb4O77gbW962b2F0PUczuwAlgJzAG2mNlT0WuIiIy5lnSKb//gN5w628GsabVxl1OWStKF5e4b3f3QYNvNbBJhSHxzkF0+CDzq7ll3P0EYSB8Y+0pFRELrLAXAFj8RcyXlq1zugbwbOOLumwfZvhg4kLd+EFhU9KpEZMJaNn8GM6fWsHmX7oMMplwC5D8x+NWHiEjJJRIB66yezd5KTzYXdzllKfYAMbMFwBuBx4fY7SBwTd76YmDQLjERkbGQSTdw9sIlXjp8Ou5SylLsAQJ8BPhnd28fYp8ngI+ZWcLM6oH3AH9XkupEZMJa11RPEKDHeQdRkgAxs0fM7DCwEHjKzF7I2/xRBui+MrMfmNn6aPU7wD5gD/A88Dl3f7m4VYvIRDdjag0rFs7UfZBBlOQxXnffAGwYZFvTIO3vzFvuAT5RnOpERAaXsRRP/Otuzl24xNTJk+Iup6yUQxeWiEjZyqRTZHOwbU9b3KWUHQWIiMgQbPEsptRWsWnXK3GXUnYUICIiQ+idd32zt5LL6XHefAoQEZFhZNIp2l/t4OArZ+MupawoQEREhpGJhjXR01hXUoCIiAxj7sw6FjdOU4D0owARERmBjKXYua+djs7uuEspGwoQEZERaEmn6O7JsuMlPc7bSwEiIjIC1y6dQ82kpLqx8ihARERGYFJ1ktXL52pcrDwKEBGREcpYiqNt5znWdj7uUsqCAkREZIRa0tHjvLoKARQgIiIjNm/uFBrnTNZ9kIgCRERkhIIgIGMptu89QVd3Nu5yYqcAEREZhZZ0Ax2XevjN/qHmwJsYFCAiIqOwesVcqpKBurFQgIiIjEpdTRXXLp3DJgWIAkREZLQylmL/sTO0v3ox7lJipQARERmlTPQ475YJ/jivAkREZJSWzJvO7Ok1E74bSwEiIjJKQRCwzlJs3X2CnuzEnaVQASIiUoAWa+DcxS72HDoVdymxUYCIiBSguameRDCxZylUgIiIFGD6lEmsXDxLASIiIqPXYil2HzrFmfOX4i4lFlWlOpGZPQTcBiwBVrv7zqi9FvgScCvQATzn7ncOcPxngbuBo1HTs+7++8WvXERkYJl0iu/+2Nm2+wS3rFsQdzklV7IAAZ4EHgae6df+BcLgaHL3nJk1DPEa33b3TxWrQBGR0VixaBbTJlezyV9RgBSTu28EMLO+NjObCtwBLHT3XLTfK6WqSUTkaiQTAWubUmze1UoulyMIgrhLKqm474EsB9qB+83s12b2tJndPMT+HzKz7Wb2YzN7Q4lqFBEZVMZSnDrbyf5jZ+IupeTiDpAksAzY4u7rgXuA75vZ9AH2/UtgqbuvAb4I/KOZzSldqSIir9U7rMlE/FR63AFyEOgGvgfg7r8E2oCm/ju6+3F374qW/wU4BFxfulJFRF5r9vRals6fPiEf5401QNy9Dfgp8BYAM2sCUsDe/vua2YK85bWET3N5SQoVERlCxlL8Zn87Fzq64i6lpEoWIGb2iJkdBhYCT5nZC9Gmu4B7zWwH8LfAh939dHTMD8xsfbTf581sp5ltAx6N9jteqvpFRAaTSafo7smxY29b3KWUVCmfwtoAbBigfR/wpkGOeWfe8keKVpyIyFVYtWQOtZOSbPJWbrx+XtzllEzc90BERCpedVWC5pX1bIoe550oFCAiImMgk07RevICR9vOx11KyShARETGQMbCx3kn0tNYChARkTHQOGcK8+dOYfMEmuZWASIiMkYy6RTb97Zxqasn7lJKQgEiIjJGWtINXOrq4YV97XGXUhIKEBGRMXL9sjlUVyUmTDeWAkREZIzU1lRx3bI5ChARERm9jKU4ePwsJ05djLuUolOAiIiMod7ReSfCVYgCRERkDC1umMbcGbVsngBz4ylARETGUBAEZNINbN19gu6ebNzlFJUCRERkjGXSKS50dOMHTsVdSlEpQERExljzynoSiYAt4/w+iAJERGSMTa2rxhbPYpMCRERERqslnWLvodOcPtsZdylFowARESmC3sd5t+4ev1chChARkSJYvmAm06dMGtfdWAoQEZEiSCQCMpZiq58gmx2fsxQqQEREimSdpTh9rpN9R1+Nu5SiUICIiBTJOqsHxu8shQoQEZEimTWtluULZ4zbcbEUICIiRZSxFL/Zf5LzF7viLmXMKUBERIqoJd1ANptj254TcZcy5kYcIGb2qUHaPzl25YiIjC92zSzqaqrGZTdW1Sj2/Qzw0ADt9wF/PtSBZvYQcBuwBFjt7juj9lrgS8CtQAfwnLvfOcDxSeAR4O1ADnjQ3b8+itpFRGJRlUywtqmezd5KLpcjCIK4SxozwwaImf1OtJg0szcD+d/9MuDsCM7zJPAw8Ey/9i8QBkeTu+fMrGGQ428HVgArgTnAFjN7yt33j+DcIiKxyliK53Yc43DrORY1TIu7nDEzkiuQb0Rfa4Fv5rXngFeAPxzuBdx9I4CZ9bWZ2VTgDmChu+ei/QabgeWDwKPungVOmNmTwAeAL46gfhGRWGUsHNZk067WiRUg7r4UwMy+7e53jOG5lwPtwP3Rlc054L7esOlnMXAgb/0gsGgMaxERKZrU7MksapjK5l2v8J43Lo+7nDEz4pvo/cPDzN5sZr99FedOEnaBbXH39cA9wPfNbPpVvKaISFlaZyle2NdOZ1dP3KWMmdE8hfUzM7spWr4H+Fvge2Z2b4HnPgh0A98DcPdfAm1A0yD7XpO3vhg4VOB5RURKrsUauNSdZedLbXGXMmZG8zmQ64Hno+WPAW8GXg/cVciJ3b0N+CnwFgAzawJSwN4Bdn8C+JiZJcysHngP8HeFnFdEJA7XLZ/DpKrEuBrWZDQBkgByZrYcCNz9RXc/BMwa7kAze8TMDgMLgafM7IVo013AvWa2g/CK5sPufjo65gdmtj7a7zvAPmAPYYh9zt1fHkXtIiKxqqlOcv2KuWwaRwEyms+BbAS+AswD/gEgCpNhr8fcfQOwYYD2fcCbBjnmnXnLPcAnRlGriEjZabEUj/7jTo63n6dxzpS4y7lqo7kC+ShwGtgOfDZqSxN+vkNERIbRO0vhlnHyqfQRX4G4eztwb7+2fx7zikRExqkF9VNJzapjs7fyjt9aGnc5V23EAWJm1YTDlnwYmA8cJbw38YC7XypOeSIi40cQBGTSDfxs82G6urNUV1X2eLajqf4LhGNW3QU0R19/B/jTItQlIjIuZSzFxc5udh04GXcpV200N9E/ADRHXVkAbmabgW3AH415ZSIi41DzyrkkEwGbd7WyevncuMu5KqO5AhlsCMnxM7SkiEiRTa6tZtXS2ePi8yCjuQJ5Avi/ZvY/uPzJ8PuidhERGaGMpfj2D37DqTMdzJpeG3c5BRvNFch/BZ4CvgpsAr4M/AT4L0WoS0Rk3OodnXfL7sq+ChnJfCA3Ae9293sIJ5X6TN62PwUyXB7iREREhrF0/gxmTqth065Wfmf94rjLKdhIrkDuBX4+yLafAp8eu3JERMa/RCIgYym2+Al6srm4yynYSAJkLfDDQbY9BbSMXTkiIhNDxlKcvXCJlw6fjruUgo0kQKYDkwbZVg2Mn+m1RERKZG1TPUEAmyt4WJORBMgu4K2DbHtrtF1EREZhxtQaViycWdGP844kQL4E/JWZvc/MEgDRvBzvA/4S+PNiFigiMl5l0in8wEnOXajM0aCGDRB3/y7hMCaPAR1mdhToiNa/6O7fK26JIiLjU4s1kM3B1j0n4i6lICP6HIi7/zmwAHgX8Kno64KoXURECtC0eCZT6qorthtrNMO5nwF+VMRaREQmlGQywdqmejbtaiWXyxEElTUyVGWPJSwiUuEyluLkmQ4OHj8bdymjpgAREYlR77AmlThXugJERCRGc2fWcU3jNDb7K3GXMmoKEBGRmGXSDbyw7yQXO7vjLmVUFCAiIjFrsRTdPVl2vNQWdymjogAREYnZtctmUzMpyZYKuw+iABERiVl1VZLVy+eyqcLGxVKAiIiUgZZ0imNt5znadi7uUkZsNFPaXhUzewi4DVgCrHb3nVH7fsKhUTqiXe9x99d8YNHMvgXcCvR2Ej7h7g8UtWgRkRLJpKNZCne1Mv/mqTFXMzIlCxDgSeBh4JkBtr2/N1CG8aC7f2VsyxIRid/8uVOZN2cKm7yVf3PzsrjLGZGSdWG5+0Z3P1Sq84mIVJpMOsX2vW10dffEXcqIlMs9kMfNbLuZfc3MZg6x3yfNbIeZPWlmq0pWnYhICWQsReelHl58+WTcpYxIOQTILe7eDNwABMBgXVSfBla4+2rg+8APzSxZohpFRIpu9Yq5VCWDihmdN/YA6e3WcvdO4GvATYPsd8Tds9Hyt4GpwMJS1SkiUmx1NVVcu3ROxUxzG2uAmNkUM5sRLQfAh4Ctg+y7IG/5bUAPcKQUdYqIlEpLOsX+Y2dof/Vi3KUMq2QBYmaPmNlhwquGp8zsBaABeNrMtgM7gSbg7rxjtprZ/Gj1sej+xzbgPuDd7l5ZA8eIiAwjk24AqIhurJI9xuvuG4ANA2xaN8Qxa/OWby1GXSIi5eSaxmnMnl7LZm/lLTdeE3c5Q4r9HoiIiFwWBAEZS7F19wl6erJxlzMkBYiISJnJpFOcu9jFnkOn4y5lSAoQEZEys7apnkRQ/rMUKkBERMrMtMmTaFo8q+xnKVSAiIiUoUy6gT2HTvPquc64SxmUAkREpAxlrJ5cDrbtORF3KYNSgIiIlKEVi2YxbXJ1Wd8HUYCIiJShZCJgXVOKzd5KNpuLu5wBKUBERMpUJp3i9NlO9h87E3cpA1KAiIiUqXUWzlK4aVd5Po2lABERKVOzp9eydP50tnh53khXgIiIlLGMpXjx5XYudHTFXcprKEBERMpYS7qBnmyO7Xvb4i7lNRQgIiJlLL1kNnU1ybIc3l0BIiJSxqqrEqxZUc8mbyWXK6/HeRUgIiJlLpNO0XryAkfbzsddyhUUICIiZS5Tpo/zKkBERMpc45wpLKifUnb3QRQgIiIVIJNuYMdL7XR29cRdSh8FiIhIBchYiktdPbywrz3uUvooQEREKsD1y+dQXZUoq24sBYiISAWonVTFdcvmsNkVICIiMkot6RSHXjlL66kLcZcCKEBERCpG7+O8W8rkKqSqVCcys4eA24AlwGp33xm17wc6oj8A97j7jwY4fjLw10AL0A18yt3/qeiFi4iUiUUN05g7s45Nu1p52+uXxF1O6QIEeBJ4GHhmgG3v7w2UIXwKOOPuK8xsJfCMma1w93NjXaiISDkKgoCWdIpnth6huydLVTLeTqSSnd3dN7r7oat4iQ8CfxW91h7g18A7xqI2EZFKsc5SXOjoxg+ciruUsrkH8riZbTezr5nZzEH2WQwcyFs/CCwqfmkiIuWjeWU9iURQFk9jlUOA3OLuzcANQAB8JeZ6RETK1tS6atLXzGJzGYyLFXuA9HZruXsn8DXgpkF2PQhck7e+GLiaLjERkYqUSafYe/hVTp/tjLWOWAPEzKaY2YxoOQA+BGwdZPcngI9H+64kvGL5YSnqFBEpJy3WAMCW3fF2Y5UsQMzsETM7DCwEnjKzF4AG4Gkz2w7sBJqAu/OO2Wpm86PVLwIzzWwv8E/Ane5+tlT1i4iUi2ULZjBj6qTYhzUp2WO87r4B2DDApnVDHLM2b/k88IEilCYiUlESiYB1TSm27G4lm82RSATx1BHLWUVE5Kpk0ilePXeJfUdeja0GBYiISAVa1xTNUujxPY2lABERqUAzp9WwYuGMWO+DKEBERCpUJt3ArgOnOH+xK5bzK0BERCpUxlJkszm27TkRy/kVICIiFcqumcXk2qrYhjVRgIiIVKiqZILmlfVs2tVKLpcr+fkVICIiFawlnaLt9EUOvVL6z1UrQEREKti6aJbCOLqxFCAiIhUsNWsyixqmxvI4rwJERKTCZayBnfva6bjUXdLzKkBERCpcJp2iqzvLzpfaS3peBYiISIW7ftkcJlUnS34fRAEiIlLhJlUnWb18TslnKVSAiIiMA5l0iiMnznO8/XzJzqkAEREZBzLR47xbStiNpQARERkHFtRPJTV7MptK+DivAkREZBwIgoAWS7F97wm6urMlOacCRERknMikU1zs7GHX/pMlOZ8CRERknFizYi7JRMCmEj2NpQARERknJtdWs2rpbLZ4aeYHUYCIiIwjGUux7+irnDzTUfRzKUBERMaRlnQDUJrHeRUgIiLjyNL505k1raYko/MqQERExpEgCFhnKbbsbqUnW9xZCquK+up5zOwh4DZgCbDa3Xf2234/8NmBtkXbvwXcCrRFTU+4+wNFLFlEpCJlLMVPfn2IvYdOYdfMLtp5ShYgwJPAw8Az/TeYWQZ4PXBgmNd40N2/UoTaRETGjbVN9QQBbPYTRQ2QknVhuftGdz/Uv93MaoCvAp8oVS0iIuPZjKk1rFw0s+ij85bDPZDPAX/j7vtHsO8nzWyHmT1pZquKXJeISMXKWAO7D57i7IVLRTtHrAFiZm8A1gNfG8HunwZWuPtq4PvAD80sWcz6REQqVUs6RTYHW3cX70OFcV+BvBFYBbxsZvuBhcCPzOyt/Xd09yPuno2Wvw1MjfYXEZF+Vi6aydS66qI+zlvKm+iv4e4PAg/2rkch8m8HeQprgbsfiZbfBvQAR0pTqYhIZUkmEzQ31bPZW8nlcgRBMObnKNkViJk9YmaHCa8anjKzF0ZwzFYzmx+tPhbd/9gG3Ae82927i1iyiEhFa7EUJ890cOD42aK8fsmuQNx9A7BhmH2W9Ftfm7d8a3EqExEZn1pWNVBXk6Tt9EWWzJs+5q8faxeWiIgUz+zptXzrM29jcm11UV4/7pvoIiJSRMUKD1CAiIhIgRQgIiJSEAWIiIgURAEiIiIFUYCIiEhBFCAiIlKQifI5kCTA8ePH465DRKRi5L1nDjhw7UQJkHkAt99+e9x1iIhUonnAS/0bJ0qA/Aq4BThGOAijiIgML0kYHr8aaGOQyxV30nURERmfdBNdREQKogAREZGCKEBERKQgChARESmIAkRERAqiABERkYIoQEREpCAT5YOEBTOzJuAxYA7QDtzh7nvirWr0zOwh4DZgCbDa3XfGW1FhzGwO8B1gOXAJ2AN83N1PxFpYgczsSWApkAXOAX/o7lvjrapwZnY/8Fkq++/YfqAj+gNwj7v/KLaCroKZ1QJfAm4l/H6ec/c7x+r1dQUyvL8EvuruTcBXgb+KuZ5CPQn8NnAg7kKuUg74grubu68mHF7hwZhruhofcfdmd18HPAR8M+6CCmVmGeD1VP7fMYD3u/va6E9FhkfkC4TB0RT9e/mTsXxxBcgQzCwFZIDvRU3fAzJmVh9fVYVx943ufijuOq6Wu59096fzmp4HrompnKvm7q/mrc4gvBKpOGZWQ/gL1ifirkVCZjYVuAP4E3fPAbj7K2N5DnVhDW0RcMTdewDcvcfMjkbtFdllMp6YWYLwDev/xF3L1TCzrwNvBQLg7TGXU6jPAX/j7vvNLO5axsLjZhYAG4F73f103AUVYDlht/v9ZvZmwi7S+9x941idQFcgUsm+TPiP4itxF3I13P333H0xcC/wxbjrGS0zewOwHvha3LWMkVvcvRm4gTDUK/XvVxJYBmxx9/XAPcD3zWz6WJ1AATK0Q8ACM0sCRF/nR+0So+ihgJXAB929Irt9+nP37wBvjh4UqCRvBFYBL0c3oBcCPzKzt8ZZVKF6u3rdvZMwFG+Kt6KCHQS6ibrg3f2XQBvQNFYnUIAMwd1bga3A70ZNv0uY5uq+ipGZfR5oAd4T/SOvSGY21cwW5a2/CzgZ/akY7v6gu8939yXuvgQ4DLzN3X8cc2mjZmZTzGxGtBwAHyJ8D6g47t4G/BR4C/Q9UZoC9o7VOXQPZHh3AY+Z2WeAU4Q3pSqOmT0CvA9oBJ4ys3Z3vy7mskbNzK4D/juwG/hF1N/+sru/N9bCCjMFeMLMphDOU3MSeFfvDU+JRQPw91FvQxJ4Ebg73pKuyl3AN83sz4Au4MNjeT9H84GIiEhB1IUlIiIFUYCIiEhBFCAiIlIQBYiIiBREASIiIgVRgIiUITM7Z2bL4q5DZCh6jFdkANEnqn+P8FPVv+fuNxfxXE8TjiP19WKdQ6QYdAUiUkRmpg/ryrilKxCRAURXIH9GOLhhNXAR6Hb3mdHQ5Q8A/x6oAf4B+CN3v2hmbwL+hnCgxz8C/gXYQDgJ1o2Eoz88C9zl7ofN7AHgvxF+Srgb+Ja7/4GZ5YCV7r43Glrjy8A7gAvAo8Dn3T1rZh8lvFJ6HvjPwGngbnf/f9H38VHgM0A94ThI97n748X4mcnEoysQkcH9hnAoiOfcfaq7z4zaHyQckG4tsAJYQPgm3asRmE04T8mdhP/O/jpaX0wYRl8BcPdPA88AfxCd4w8GqOPLhHOFLCMcuPAO4D/mbb8RcGAu4QRC3zCzIBoi5RHgHe4+DfgtKnRcJylPurwWGYVogL07gTXufjJq+zzwXcIxuiCcFOr+vIEeLwJ/n/caDxAOcjeS8yUJB/Rb6+5ngbPRuEYfBr4R7XbA3R+N9n+McATZBuBsVMv1ZnbQ3Y8Bxwr6xkUGoAARGZ16YDKwKW/ipIBw4L1eJ9y9dz5tzGwy4bzUbwdmRc3TzBgAIloAAAFZSURBVCzZO1nZEOYSdqHlTxN7gPCqp9fx3gV3vxDVNdXdj5vZB4FPEV6VPAv8sbvvGtF3KjIMBYjI0PrfJGwjvKK4zt2PjPCYPwYMuDF6U18LbCEMnoH273++LsLurxejtsXAYOe+QjSf94/MrA74n4T3T24ZybEiw9E9EJGhvQIsNLNJANHkVY8CXzKzFICZLTCztw3xGtMIQ+e0mc0G7h/gHAN+5iO6QvnfwANmNs3MrgE+SXijfkhm1mBm/y66F9JJOHvjuJh8S8qDAkRkaD8BXgCOm1lb1HYP4aQ8z5vZGeApwiuMwfwFUEd4NfE88MN+2x8G3m9mp6J5W/r7Q+A8sI9wju7vAt8cQe0JwrA5SjjXyBsJ55AXGRN6jFdERAqiKxARESmIAkRERAqiABERkYIoQEREpCAKEBERKYgCRERECqIAERGRgihARESkIAoQEREpyP8HeSMWc0tGaTsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEMCAYAAAD5zKAAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xUZfb48U8SQ40oHaUoiDwoQRG7CIRQpAmIsIC4uC6i8hXR3Y26WVnlp1JcZFFERRRXcFEsNEVRQkIQdUURUINwAEUg9CIl1DiZ3x93JgxhkrmT3ClJzvv1mpfDnTt3Tka4J087T4zb7UYppZRyWmykA1BKKVU2aYJRSikVEppglFJKhYQmGKWUUiGhCUYppVRInBPpAMLJGFMRuBbYCbgiHI5SSpUWccAFwLcictLum8pVgsFKLssjHYRSSpVSbYEv7J5c3hLMToBZs2ZRr169SMeilFKlwq5duxg8eDB47qF2lbcE4wKoV68eDRo0iHQsSilV2gQ1tKCD/EoppUJCE4xSSqmQKG9dZEqpUiYvL4/s7GyOHj0a6VDKvPj4eOrUqUO1atUcuZ4mGKVUVNu3bx8xMTEYY4iN1U6XUHG73Rw/fpzt27cDOJJk9P+WUiqqHTx4kLp162pyCbGYmBiqVKlC/fr12bNnjyPX1P9jSqmo5nK5iI+Pj3QY5UblypXJzc115FqaYJRSUS8mJibSIZQbTn7XUTMGY4ypCbwFXAKcAjYC94nI3gLnVQH+A1wN/A6kiMjCMIerlCrHDh06RNu2bfnDH/7AqFGjwvrZ69atY/PmzXTv3j3o92ZnZ3P77bezYsWKEER2tmhqwbiBf4mIEZGWwM/AeD/npQCHRaQpcCvwujEmIYxxKqVKiaSkJJKSkhy/7sKFC7nyyiv5+OOPOXXqlOPXL8q6dev49NNPw/qZxRU1LRgROQBk+hz6Ghju59QBwF2e92w0xqwEugHvhzpGpZQCmDNnDo888givvvoq6enpdOvWjVOnTjFp0iSWL19ObGwsDRs25KWXXgLg1VdfZeHChfkD6W+//TaxsbHMmzePt99+G5fLRUJCAqNHj6ZJkybMnTuXjz76iIoVK7J161Zq1arFhAkTqFChApMnTyYnJ4fevXtz7bXXMmrUKL7//nuee+65/KncI0eOzE+ss2bN4s033yQhIYH27duH9XuKmgTjyxgTi5VcPvTzciNgi8+ftwINwxGXUkqtX7+egwcPcsMNN7B3717mzJlDt27dmDZtGtu2bWPu3LlUqFCBAwcOADBv3jwyMjJ45513SEhI4LfffiM2NpaVK1eyaNEiZs2aRYUKFVi2bBn/+Mc/mD17NgDfffcd8+fPp0mTJkyZMoUxY8YwefJkRo4cSWZmJpMnTwbg8OHDPPnkk0ybNo06deqwZ88e+vXrx8KFC9mxYwevvPIK8+fPp1atWowePTqs31VUJhjgRSAHmBLpQJRSpYtvl9iyZcvOOpaZmVmi63/wwQf07t2bmJgYunTpwjPPPMPu3btZunQpf//736lQoQIANWrUAGDp0qUMGjSIhASrJ7969eoAZGRksH79evr37w9Y61AOHz6c/zlXX301TZo0AaB///7ceuutfuNZvXo12dnZDBs2LP9YTEwMW7ZsYfXq1SQlJVGrVi0ABgwYwKJFi0r08wcj6hKMMeY54FLgVhHJ83PKVuAiwDv43whYGqbwlFLl2KlTp1i4cCEVKlRgwYIFAOTm5jJ37tygr+V2u7n99tt56KGHShST2+3GGMOsWbPOem316tUlunZJRdMgP8aYsVizw/oUsanN+8B9nvMvxdrjpXSMeCmlQi4zMzP/0b59e9q3b3/GsZJIT0+ncePGfP7552RkZJCRkcEbb7zBvHnz6NChAzNmzMgf9Pd2kXXo0IF33nmHnJwcAH777TcAkpOTWbBgAbt27QKs9T5ZWVn5n7Vq1Sp+/fVXwBrzueGGGwBISEjgyJEj+eddddVVbNmyha+//jr/2A8//IDb7ea6665j2bJl7N+/H7BaX+EUNS0YY0wLIBXYAHxljAHYLCK3GWPWAN1FZAcwAXjTGLMJq3T0vSJypLDrKqWUU+bMmXNWV9VVV11FXl4e1113HUeOHKFPnz7Ex8dz0UUXMXnyZPr06cPu3bsZMGAA55xzDlWqVGHWrFlce+21PPzwwwwfPhyXy0Vubi5du3YlMTERgNatW/Pss8+yZcuW/EF+gBtvvJE33niDXr16cd111zFq1ChefvllJkyYwNixY8nNzaVhw4ZMnTqV5s2bc//99+d30bVr1y6s31eM2+0O6wdGkjHmYmBzenq67gejVCmxbt06LrvssmK91zv2UtKWS7jNnTv3jIH8cCv4nWdnZ9OxY0eAxiLyq93rRE0LRimlnFbaEktZowlGKaWiTN++fenbt2+kwyixqBrkV0opVXZoglFKKRUSmmCUUkqFhCYYpZRSIaEJRimlVEhoglFKqSAkJyfTtWtXevXqRbdu3Xj//fAXcl+xYgVffPFFsd8brhlqOk1ZKVUmuVwuMjIyyMrKIjExkeTkZOLi4hy59uTJk2nWrBkbNmygb9++tGvXjrp16zpybTu++eYbjh07xs033xy2zywOTTBKqTLH5XIxdOgQdu0S2rerzcSJs5g1yzB9+kzHkgxAs2bNqFatGrt376Zu3bpMmzaNxYsX43K5qFu3Lk8//TS1a9fmyJEj/OMf/2Djxo3UrVuXunXrUrNmTR577LH8fWS+/fZbTp06hTGG0aNHU7VqVY4cOcLYsWPJysoiJiaGa665hgEDBjB79mzy8vL46quv6NGjB/feey/Lli3jlVde4dSpU8THx5OamkqrVq0AmDRpEp988gnVqlXjuuuuc+znD0QTjFKqzMnIyGDXLuGjDzsTHx9LSkoePW9dTEZGBp07d3bsc7777juqV69O8+bNWbBgAdu2beO9994jNjaWt99+m/HjxzNx4kReeuklqlWrxqeffsrBgwfp27cvt9xyCwCvv/465557bn4hygkTJjBt2jT+8pe/MHbsWKpUqcKCBQuIjY3lwIED1KhRg4EDB3Ls2DEee+wxALZu3crLL7/M9OnTSUhIYOPGjQwbNozMzMz8opzz58+nUqVKPPDAA479/IFoglFKlTlZWVm0b1eb+HhrmDk+Ppak9nVYu3atIwlm5MiRuN1utm7dygsvvECFChXyu+Nuu+02gPxdKsEa9xg1ahQA559/Pp06dcq/VkZGBjk5OXz22WeAtSVA8+bNAWsvmblz5xIba/0c3j1mClq+fDlbt25l8ODB+cd+//139u3bx4oVK+jevTtVq1YFoF+/frz88ssl/g7s0ASjlCpzEhMTmThxFikpecTHx5Kbm0fmsj2kpLRw5PreMZhFixaRmppK69atcbvdDB8+nH79+gV1LbfbzZNPPsmNN95Yopjatm3Lv/71rxJdw2k6i0wpVeYkJydTr56h562LGTduFT1vXcwFFzQnOTnZ0c/p1q0bbdq04dVXXyU5OZm3336bQ4cOAVZLZP369QBcd911+RuUHT58mPT09DNiffPNNzlx4gQAOTk5/Pzzz4C1l8z06dPxVr337jFTcE+YNm3asHz5cjZu3Jh/7IcffgDghhtuYNGiRRw7dgyXy8WcOXMc/Q6Koi0YpVSZExcXx/TpM8nIyGDt2rWkpLRwdBaZr7/97W/07duXYcOGcfDgQe68807AapkMGjSI5s2b88ADD5CamkrXrl2pXbs2iYmJ+d1n9957L1OmTKFfv37ExMQQExPDiBEjuOSSS0hNTWXs2LH07NmTuLi4/P1fOnXqxPz58+ndu3f+IP+ECRN4/PHHOXHiBLm5ubRu3ZorrriCDh06sGbNGnr37p0/yL97927Hvwd/dD8YpVRUK8l+MNEiNzeXvLw8KlasSE5ODoMGDSI1NZWbbrop0qH5pfvBKKVUKXH48GGGDRuGy+Xi5MmT9OzZM2qTi5M0wSilVIjVrFmTuXPnRjqMsNNBfqVU1CtPXfmR5uR3rQlGKRXV4uLiyM3NjXQY5cbx48eJj4935FqaYJRSUe38889n9+7d5OXlRTqUMs3tdnPs2DG2b99OnTp1HLmmjsEopaJarVq1yM7ORkQiHUqZFx8fT926dalWrZoj19MEo5SKarGxsTRq1CjSYahi0C4ypZRSIaEJRimlVEhEVReZMeY54HbgYqCliGT5OWc08H/ADs+hL0UkfPWnlVJK2RJVCQaYD7wALA9w3kwRSQlDPEoppYopqhKMiHwBYIyJdChKKaVKKKoSTBAGGmO6ALuAJ0Xkf5EOSCml1JlK4yD/VKyKnlcAE4AFxpiaEY5JKaVUAaUuwYjILhHJ9TxPA7YBiZGNSimlVEGlLsEYY+r7PG+FNeNMl/gqpVSUiaoxGGPMZKAvUA9YYozZLyItjDGfAE+IyEpgrDHmasAFnAL+KCK7Ihe1Ukopf6IqwYjISGCkn+PdfZ7fFdaglFJKFUup6yJTSilVOmiCUUopFRKaYJRSSoWEJhillFIhoQlGKaVUSGiCUUopFRKaYJRSSoWEJhillFIhETDBGGOuMsYMN8a08/Paw6EJSymlVGlXZIIxxgwElgJ9gDnGmDnGmCo+pzwVyuCUUkqVXoFaMKOAHiJyC9AYyAXSjDHnel6PCWVwSimlSq9ACaahiHwJICI5IjIQWAVkevZgcYc6QKWUUqVToASz1xhzie8BEXkQWAIsB+JDFZhSSqnSLVCC+Rg4q3qxiDwGvA9UDEVQSimlSr8iy/WLyENFvPakMWaM8yEppZQqC0q0DkZETjkViFJKqbJFF1oqpZQKCU0wSimlQiKqtkxWqjRxuVxkZGSQlZVFYmIiycnJxMXFRTospaKGrRaMMWZHIce3OhuOUqWDy+Vi6NAhTJyYyonji5k4MZWhQ4fgcrnCGkNaWhqTJk0iLS0trJ+tlB12u8jOLXjAGHMOkOBsOEqVDhkZGezaJXz0YWdSU1vz0Yed2blzPRkZGWH5/GhIcEoFUmQXmTEmDWu1fkVjzOICLzcEVoQqMKWiWVZWFu3b1SY+3vodLT4+lqT2dVi7di2dO3cO+ef7Jrj4+FhSUvLoeetiMjIywvL5StkRqAXzATAHcHn+6318APwdqwimUuVOYmIiyz7fS25uHgC5uXlkLttDixYtwvL5RSU4paJFoIWWrwIYY1aIyJrwhKRU9EtOTmbWLEPPWxeT1L4Omcv2cMEFzUlOTg7L5ycmJjJx4ixSUvKIj4/NT3ApKeFJcErZYXcW2SXGmOMiIp7aZC8DecAIEfk5dOEpFZ3i4uKYPn0mGRkZrF27lpSUFmGdRRbpBKeUHXYTzLPAzZ7nE4ENQA4wFXCsw9cY8xxwO3Ax0FJEsvycEwdMBrpijQ+NF5HXnYpBRa9omxYcFxdH586d/Y55hDrWSCc4peywm2DqiMguY0xFoD1wAdbeMHsdjmc+8AJWpebCDAaaApcCNYHVxpglIvKrw7GoKOKdNbVrl9C+XW0mTpzFrFmG6dNnhv2mGih5hCvWohKcUtHA7jTl/caYi4EuwHcicgKoEMT7bRGRL0RkW4DTBgCviUieiOzFSkr9nYxDRZ9QTQtOSkoiKSnJ9vl2pgdHegqzUtHCboIYC6wGZmB1kQF0AH4IRVABNAK2+Px5K9aUaVWGRcusKTvJI1piVSrSbCUYEXkNa8vkJiKyyHN4DVZ3lVIhF+lpwV52kke0xKpUpAVTi8wNdDHGXCgiLwC/Y80kC7etwEXAt54/F2zRqDLIyVlTvl1iy5YtO+tYZmZmoe+1Mz3YiVijbUKDUsVhK8EYY27CGuv4CbgaayA+ERhJ+Bdbvg8MM8bMxRrk7wO0DXMMKsyCmTUVypuzneRRWKwAaWlpAeOKpgkNSpVEjNvtDniSMWYlMEpEPjXG/CYi1Y0xlYHNIlLPqWCMMZOBvkA9YB+wX0RaGGM+AZ4QkZWeacpTsCYcADwrItNsXv9iYHN6ejoNGjRwKmwVYsEkjII352Wf76VevcJvzt6WS1GtlsLiWbt2LS1a2JseHExcaWlpTJyYml8GJjfXKgOTkjJeZ4ypiMjOzqZjx44AjYOZsWt7oaWIfOp57s1IJ7FmkjlGREZitYoKHu/u89wFDHfyc1X0Cva3+XDU6CrO9OBg4op0nTM7tAtP2WF3FpkYYzoUOJYE6LQYFVLBTvmN1hlcwcQV7ZMEtJKzsstugnkEeN8Y8ypQyRjzAvBf4NGQRaYUwSeMYG/OmZmZQXWPFVcwcSUnJ1OvnjXOM27cKnreujiqysDoOh9ll91pysuxBve3A+8AvwFtROR/IYxNqaATRrTenIOJyztJICVlPJWr3EJKyvioGuCP1laiij5FDvIbYz4WkR5hjCekdJC/9PF2x+zcuf6MWVtF3XCLMwjvhEATBiIVl9PjJToJofwp7iB/oARzWESqlTy86KAJpnQK5Y351Cn47beiH7NnfwhA7969AIiJsR6+zytVgg8+mM455+TwxBMPcf75cN551sP3ufd94RLsrLpgrhlM0lelW6hnkSkVMSUt6piXB1u3goj1ePbZeRw71oiqVa9m924rAVSv7v9x4YWQkLAJgCuuALfbesDp5243nDgBeXkVOHasEZ99BgcPwqFD1uPgQeuRmwv160ODBqcfDRue+ec6dZxNQqGYVaeVnJVdgRJMJWPMzKJOEJEhDsajVIm43bBxIyxZAp9/Dj/9BJs2QY0aYIz1qFw5m5o1/8eCBVfTqBEEui8uXGi1YIYP/+tZr/lWANi61aoKkJ3dHrCu+8MPmfmvHz0K27dDdrb12Lo1j08/3c7mzac4caI2v/12Li5XDM2bw2WXnflo1MjFsmXBd3OFasqzVnJWdgRKMG5ANxRTUS0vDz77DN57z0osAB07Qrdu8Oij0KwZJCScPj8paQ4AjRsXfs2SlJMpTNWqVizNmp3uZsrNFQbfcbrr6tlnZ7JhQxzr1sG6dZCRAevWudm+3UXlyhfTpEk8ublfc/nlC3n33X8HTDK686WKpEAJ5qSI/L+wRKJUkPbvh//8B155xermuvtu+PvfrRt4wW6mUCSMgu8LpipAYV1XP/xgdV219Sl+lJa2hGeffYoxz/yBdesuID2jC+++25hLLz3GwIHn0rMnXH+9/5ZYMHXRdPGkclqgBBPmIUmlAtu7F/75T3j3XejVC95+G667ztmxi+ImDruC6brKysqiY3IC11+/i+uv38Wf/rSa5mYNe/cNISZmAPffDzt3Qvfu0LMn3HILVPNMzbE7XqL1z1QoBFoH89+wRKGUDXl58NprkJgIVapYYy0zZli/vQdKLt4FlZmZmbRv35727dufcSzcglnf4+/cz5fvokePGowZAz/8ACtXWt/Df/5jTRy4/XaYM8eafOAdL3n44Yfp3LlzwBI7RS2edLlcpKWlMWnSJNLS0nT1vipSkS0YEdGaXyoiCnbX1KiRzIgRccTEwOLFcOWVgd9TnC6egq2VpKQk1qxZQ6tWrQK+106i8l4/PT3ddteVnW6uiy6C996zrv3rr5nMnQsvvwz33AN9+sAdd0CHDnBOIf/i7bSotJWjguXolsdKOcG31tWhg8sZNmwvN9+cw9135/HFF4Unl1DVx2rVqpXf5BHsdsu+glmtH+zK/urVYehQSE+HtWut6dX/+Ic1Dfqhh2DFitNTrb3stKi0RIwKlq6DUSEXbMsiIyODnTuFP/3pLzzySE86d97Ar7/2pXHjvxMb639abDDrPSLRJeZPMFN9izst+MIL4S9/sR4bN8I778CQIZCdvZkGDd7n++8fpVIle62k0lDlWUUXTTAqpIrTrbJq1Xq2bp3C2LEXM/udd7n55i2MG1ehyBtZSW5+/maYnX/++QAcOnTorHOCTVChmsEW7LUvvRSeeMKaINGq1YtkZ/+Biy+Gxx+H//u/wJMBnJ7yrLPWyj7bCcYY82dgEHAhsAOYDbwhIoF3LFPlVrAryd1uWLSoP4cOrWLTxo+oUoWzbmT+ZnWFa73HmjVr8j/f6WQRLjExUL36KqpXX8WLL2YycqQ1OWDq1KJbSU5uW63jOeWD3S2T/wX0Bp4HtgCNgBTAoCX7VRGCbVnMng179tSlY8f3uK3vj7ZvZCW5+RU1JdnbkvEd9A9WKKc8B3Ntf62dBx9Mwu2G48c70afPKHr1grFjrcoHBZV0K2hf4dgYTkWe3RbMn4DWIpLtPWCM+RhYhSYYVYRgWhY7dliD0J98EsNVV00PqtaVk/WxfFspRXWRhWJ9TLB8u5lycnKoWrVq0NeIiYG6dZfwv/+NYtQouPxyGD8e7rrr7OnfBceCitsS0fGc8sFugjnieRQ8dtjZcFRZY7dl0b59EllZ4xkx4gauuQbgzBuZnbGG8lYfq+DNvUqV4xw4kIPL5fJ7c7fT2pkyBf70J7j/fnj/fZg5E2rWLDyG4rZEtIRN+WA3wTwPzDXGjAeygYZYu1xOMsY08Z4kIr84H6Iqzey2LHbt6sGpUzUYNSpCgXoUvOGGskuroGA/6+ybeys6dlpQ4m6ma66B//0PUlOhdWurYsINN/gflC9uS8TJ8RwVvewmmBc8/+1Q4HhHYLLnuRvQ0Tl1lkAti82bYfPmYVx55V+Ij/+P33NCXbqlOCL9+f5u7rd0aehIN1N8PDz3HLRta5XjeeyxPH74YQi7d5/ZFTZo0BAmTQq+JaIl/8sHWwlGRHRBpnKcN1FkZT1Nbu5CVq58k6SkzfmvR/oGXpC/Vf6+fw43f91Mny3exj//GbibyW7MvXtbCzW7dj1CTs4gft70FZUqxeR3hQH5W0EH2xIpb12a5ZGug1ERlZtbjYMHrwK6RToUv0KdPAobW1qzZg0ABw8eLPS9ycnJ/PWvOXTstIBbujTkk0W/snXrQZ555hnGjBkDOBN/48bw5z/P5NVX2zFiRF+mTZuf3xW2bt06bYmoQhWaYIwxn4pIV8/z5VhdYGcRkXYhik2VcoEW0mVmZjJtmrXL45491+QfCyTaWjbhUrDFFBcXR40adTl69CjvvreTDRu2cPLkSS655DLHP7t16+Y0aTKCdevfYsSIW3n++fn5XWHaElGFKaoF47uT5euhDkSVLXanr86eDQ8+CC+8UMTFisGp7qtQr/IvbGzJ7nobb0wF3+80a1D+TbZtG8LCj6fx8SctSUraqoPyqkiFJhgRedvn+YxwBGOMaQbMAGoC+4EhIrKxwDmjgf/DqiYA8KWIPBCO+JR9dqav7twJq1dbO086nWBKI2+3WFJSkt+p2OHmm7B8B+VXrlzKG28MplatgcTG2hue1bIw5VMwpWK6AK2ABN/jIvKEg/FMBV4Skf8aY+4EXgX8/Yo0U0RSHPxc5TA701ffe88aRK5UySpfn5GRwaRJk8JyA7L7235Rs9fCMcjvTTrgTIupJHy7wu67D9q1g6ZNYXiATT20LEz5ZbdUzBTgD8BS4FgoAjHG1AFaA96O3HeAKcaY2iKyNxSfqULHzkK62bPhySft3YDs3MxDWVQyHHwH9P21XHy7w/wJ589XowbMmwdt2sC11+JZHOufloUpv+y2YO4ArhSRbSGMpSGwXURcACLiMsbs8BwvmGAGelpUu4AnReR/IYxLFUOghXS//QZZWdCxo96AihLu9T/BVmeeMsUq///jj1CwMeJ9X+/evbUsTDllN8HsAwqfLxleU4ExIpJrjOkMLDDGXCYi+yMdmDot0EK6bdusXRjj452rSxXoZlzSFk7B16O9RRQO/ftbSeatt6wSM/5oWZjyq6hpyk18/jgRmGWMGQfs9j3PwfIw24D6xpg4T+slDmtrgDNaTSKyy+d5mjFmG5AIFN1/oMKuqOmr2dnWDotQ+A1o//7/Mn/+fKB0dnmVRKR+tmBbTDExMG4cDB4MgwZBxYpnn6NlYcqvolowm7DWvvjWU+1Z4BzHysOIyB5jzBqsPWf+6/nv6oLjL8aY+iKy3fO8FXAxIE7EoMLHN8EUdgM6cSK76IsEKVLdTeGoYxZJbdpAy5YwdSrMm5eUf9z7S0HHjh1xu90cPRrj2fJZF2OWF0VNU45EeZj7gRnGmCeA34AhAMaYT4AnRGQlMNYYczXgAk4Bf/Rt1ajSwTfB2KlLFezNOlpvxmXVmDHQpQs0bVqZc845ftbrMTExJCQk8PDDD0cgOhUpdmeR1QeOichvPseqA5VFZEfh7wyOiKwHrvdzvLvP87uc+jwVOdnZcNNNp/+sq8GjTzBJ+oorrAkbzZsv4p//tI5Fulabijy7g/zzgT9jtSq8GmCt8D8rISgViG8LJhKcLJvv771Q/saNnnoKrr/eWhdTq1ako1HRwG43WDMR+dH3gOfPzZ0PSZUHR47AuefaPz8zM7NM35zLgksusRbOTp/u3DVdLhdpaWlMmjSJtLQ0XC6XcxdXIWe3BbPXGNNURDZ5DxhjmmKVc1EqaOedB4cPl71ulGjctyachg61Ho8+WvKfWysAlH52E8wbwBxjzOPAL8AlwNNoEUxVTOefD0VUondEMDf48ty15aQbbwSXC775xuouKwldgFv62e0iG481dfg54FtggufP40MUl4piTnRbhCPBqPCLibEWXP7H/8akQSlqAa4qHezuaJmHlVQmhDYcFe2c6LZISkril1/uZfHiHDZvjo7WQii6tsprq2fIELjySpg0CSpXLv51tAJA6RdMNeUkrHUp9YHtwFsisjREcakoFajbwm5Z9nPOyeH33xP8fELJaFdXaASTdBs0sIpfLlgAAwcW/zO1AkDpZ3cdzD3AWKwxlxVAI+AdY8w/ReS1EManokxR3RbJycm2WjeZmZlMnWrtBSMyLf+YKju83WRFJZhAv4zYWYCropvdFsyjQGcR+d57wBjzLjAH0ARTjhTVbRHMoOxFF8GcOc7H50RXlya7kuvVC+65B44f999NZrerVRfglm52E0xN4KcCxwSo4Ww4KtoV1W0xefJk21WRW7WyWjAtWlgDwyr6lKS7sWpVqz7ZihXgb1NOnSFWPtidRfYF8G9jTBUAY0xVrAH/r0IVmIpO3m6LlJTxnsKF4/N/60xMTGTZ53vJzc0DyJt/Df0AAByZSURBVG/dtGhx9qBsvXrW/iGzZgVeQJmUlFTk1sGBXleR0a4dLF/u/zWdIVY+2G3B3A/MBg4ZYw5gtVy+wqp4rMqZwrotghmUjYmBq66CNWtCVzJGu7pKpqTdjW3bwuTJ/l/TGWLlQ8AEY4yJASoDHYF6WHu07BARZ2upq1Iv2EHZVq2sBNOz4CYQqky4+Wa44w5r4WXBvwI6Q6x8CJhgRMRtjPkRONeTVDSxqEIFMyjbqhV88IH/1wL1//tyejpyeSzxEgrVq1tFL3/5xdpe2ZfOECsf7HaRrQaaAetDGIsqZ1q1gtTUSEeh7CiYbO0m4ZYt4ccfz04woDPEygO7CSYT+NQY8ybWFsZu7wsi8obzYanyoFkzaxrrL79AkyZnvhZM/7+2OKJXw4awfXuko1CRYjfBtAE2A+0LHHdjFcJUKmixsdC1KyxaBA88ENlYtAJAaNSuDXv3Bj5PlU12a5F1CHUgqnzq2hVmzYp8glGBFScJ164NWVkhDkxFLbulYroAv4rIBp9jzYCLRCQtVMGpsq9LF7j3XjhxAipV8n9OoNaDE62L8r6PS6hoC6Z8s9tF9hLQrsCxHM/xZo5GpMqVGjUgMdFakKdjvdGtOEk43AnGbrFVFR52V/LXEZGdBY7txFoXo1SJdO0Kn34a6ShUKDiZYALtQ+StbzZxYionji9m4sRUhg4dotssR5DdBPOLMabgCqgkrIF/pUqke3f46CNwuwOfGw6ZmYHL1yh7atWC/Q5srG4nefjWN0tNbc1HH3Zm5871ZGRklDwAVSx2u8hGA3ONMdOBn7G2TL7b81CqRK6+2kouK1fCtdeW7Fo6fhIedr/fuDjIyyv559kpjllUfTNdaxMZtlowIrIA6AJUBXp4/nuL57hSRQrUtRETA3/8I8ycGZ54tDhm6WOnOGYwxVZVeNje0VJEvgG+CWEsqgyyu+/HnXdC8+a/8f33/fn88yURjLh8KS0tPjvFMbW+WfQpNMEYYx4XkTGe508Vdp6IPOFUMJ6pzzOw9p/ZDwwRkY0FzokDJgNdsRZ6jheR152KQTnL7r4fTZpA5crbOHDguqA/w8lFkqXlhlueuFwu8vLyOHasIu3af0jPHg35fPnes5KH1jeLPkW1YHyLqDcMdSAeU4GXROS/xpg7gVeBgr9+DAaaApdiJaLVxpglIvJrmGJUQQimX7xu3cXs3t0lJHHoSv3SybcF3LVrbZZmHCM94yiPPTaWTp06nZU8tL5ZdCk0wYjIcJ/nIR/MN8bUAVoD3r8Z7wBTjDG1RcR3ouMA4DURyQP2GmPmA/2xNkBTUSZQ14bvTX7jxu+BX2nTpifx8TmAvRu/LpIMjhPJNlzrTQq2gB/xtIBjY2O1ZVIKFDkGY4xpFOgCIrLVoVgaAttFxOW5rssYs8Nz3DfBNAK2+Px5K+FrYakgBdcvfhBYxO7dXWjQYK6jcRSVhHwH/bV1E5jdcTWA3Fw4x/ZI79l0ZljpFuh//a+crpzsb+d0N6C/RqhCBeoXL3jjP3Tocw4ffpn09JFnbVLlFLfbzdGjR5k0aRKJiYm43W5iYvz99S6bStriszuuBrBvH9SsWfxYdefL0i1QgvkeazfLGcB/gR0hjGUbUN8YE+dpvcRh7Z65rcB5W4GLgG89fy7YolFRpqh+cd+ulpycHKpVW0vFijBvHvTrF/xnBbpRulwuDhzYTfXqcZ4Fe7No3Pj0b9/axRZYMK2Kffus1fzFpTPDSrci18GIyFVAP6AG8CXwCTAQqCAiLm93lhNEZA+wBhjkOTQIWF1g/AXgfWCYMSbWGFMb6AMUsi+iipRAa1+85/iuzq5S5TgHDuzmiSdcpKbCyZPOx5WRkUH9+gmkL+mlq72LKZj1Jnv32kswhf198baAU1LGU7nKLaSkjPfbFaeiU8CFliKSJSKPABcD/wZ6AjuNMa1DEM/9wIPGmA3Ag54/Y4z5xBhzjeect4BfgI3A18BTIqIla6KI3ZpQBUt7pC/pxYUXViU+PoPmzWHyZOdjs7NgrzwpTlmc5ORk6tWzWhXjxq2i562LC21V7N1rlYspSqC/L94W8MMPP0znzp01uZQiwQy/XYq14diNWFso/+Z0MCKyHrjez/HuPs9dwPCC56joYbePvqib/b//3Zkbb7RW+NdzsKRqoD597RoLLJj1Jna6yIIZ01GlS5EtGGNMDWPMA8aYb4D5WCX624lIB201qMLYbSUU1dVy6aXw5z9DaurZ17fT/VaYYH77VoXztirmz5/PmDFjCm1V2Oki01Zl2RWoi2wHMAIruTyA1SXV1BiT7H2EOkBV+tjtow90sx81CpYsgaVLT7+npCXZo6FPvzzVQtu2DerXL/ocrSFWdgXqItsFVAKGeR4FuYEmTgelSje7M38CdbVUqwavvAL33AM//ABVqzrTnaKrvcPnp5/gssuKPkdnipVdRSYYEbk4THGoMiSYPvpAN/uePeHdd+Hxx+H550u+8C4U05DLwtTmYH4Gu5UAjh2DzZtPcu+93fn88/RCr6c1xMquEqyxVapwTrYSXnjB2la5f//Su/CuPNZC+/prSEjYSGxs4O5LbVWWTZpgVNSrUQNeeska9P/uu9PjNk52p5SFVki42K0E8PnncN55PwR9/XDVOVOhpwlGhVVxbx633QazZ8NTTwXfnRKK1kOw1wxlQc7iXi9UrSrvNb7/fiIHD/6XbduW2b5uMHXOVPTTBKPCpqQ3jylToGVL6NtXu1OiQVJSEmvWrKFVq1ZnvZaXF8+RI5dhFQCxT9fElC2aYFTYlPTmUbs2TJ0KAwfCqlVW15kdhbUevM+TkpKC/g2+LGwR4MTP0KpVK7+ttXnzrOexsVcFdV2tnly2aIJRjiusG8yJm0efPlbf/pAh8OGHEOtZyeW9Qaanp4es/96pROJEIor2SQMzZ8Jdd8FbbwX3vtI6iUP5pwlGOaqobjCnbh7PPgvJyfDXv8KkSeCttO92u4PqgisLrZBws5PY9u2DjAyYMSP4BKNrYsoWTTDKUUV1gzl184iPh48+spLMP/4BY8dax48ePRqwCy4UCSQSScnp5OjkzzB7NvToYS2UDfa6uiambNEEoxwVqBusJDePgl1vlSq9yIsvPs8772SyZcsyEhISuK3PTY7230d7V1S42UlsM2bAM88U/zN0TUzZEbBcv1LBCFRXqril1/3VIDtyZAMtW/6VPXs6AY+Qm5vLZ4uzi1XTqjhl69XZfvoJtm+HTp0iHYmKBtqCUY4KVR96YV1vKSkPcPnljWja9CEuuOBimjT50tHPLg3jNNEUzxtvwJ13ErLtrlXpoglGOSpUfeiBut6uvPJhfvxxArVq3c8dd6Tz00/afx9qBRPbrl1WglmzJjLxqOijCUY5LhR96IFmoFWqtJdWrUby5Zcfsm9fZ157rTPx8c58tnfsZ//+/VSsWBGXy6VJy4/x460N4ho1Cu59Whqm7Ipxu92RjiFsjDEXA5vT09Np0KBBpMNRQfCOwezcuf6M7q+CU5CPHrWKYsbGwnvvQZUqznyud+rzss/3sn17DjVq1M0f9FfWvi+tWlljMHXr2n+fv++3Xj0tDRNtsrOz6dixI0BjEfnV7vt0kF+VCnY3CqtaFRYssFb5d+oEO3aU7HN9x35SU1vz0YedqV49lqNHj9q+hlMbjEXzRmVPPw333htccgH/3+/OnevJyMgITaAqrLSLTJUaRXW9FexmmT49mXHj4rj6apg+Hbp3L95n+hv7uaVLQ959b2dJfpQyZdMmmDsXNmwI/r1aGqZs0xaMijiXy0VaWhqTJk0iLS3N9vbHvu8vOIV52LAhPP64i/feg+HD4S9/gZMng4/N37TrTxb9yoYNG/JbFNHaqgikpN+71+jRMHKk/dpwvnS75LJNWzAqopwozx6oiOaaNda2y9dcA9OmwY032o/PO+26Y6cF3NKlIZ8t3sbWrQc5GSBbObVAM1QLPZ0qi//117BkCbz8crHC0NIwZZwmGBVRTpRnD9TNUr06fPCBNeh/++3QuzeMGwfnnx/42t6xnyVLlrBw4UKuuOJSjh79H02aNI+q9SfBcuJ7P3LEWvPy8stWWZji0NIwZZsmGBVRTvTB2ymiGRMDAwbALbdAaiq0aGEVyuzf/3SxzKK8887M/N/2q1Y9wYEDR4ucruzUAs1QLfQs6feelwcPPADt20PfviWLRUvDlF06BqMiyok++OTk09sojxu3ip63Li60m+X88+GVV+D9962ZT127wnffFX39gjOd0pf0onr12FI906kk3/upU9Z6l59/huefD3WkqjSLihaMMaYK8B/gauB3IEVEFvo5Lwn4BPDOVzkpIteHK07lPCf64IvTzXLTTdamZVOnQq9e1vjM6NFw1VVnn+vvt/1uXS9yfKZTOEvRFPd7P3zYarGce6419lK5cshDVaVYVCQYIAU4LCJNjTGXAsuNMU1FJMfPuT+JyDVhjk+FiFN98MXpZomPhwcftCYATJtmlZi/4QYr0VxxxenzSrqPjVMJw8nEE+h795fsdu6Ebt2s5Pzii1pvTAUWLV1kA4BXAURkI7AS6BbRiFTYFLfCslMqV4aHHrK6fNq2tcZp+vWDH3+0Xg+mC640iYuLY8yYMcyfPz/g975+vZVY+veHl146O7kUZ7q2U9OkVfSKlhZMI2CLz5+3Ag0LObeZMWYVkAu8LCIzQh2cKh8qV7bWy9x3nzVO06WLNRng/vvjePHFmXz9tfMznaJ9vxm3G157DR5/HP71L7j7bmeu69Q0aRXdwpJgPAmhsBJ4wRSXWAU0FJFDxpjGwBJjzHYRWVLiIJXyqFIF/vY3q/vs/ffh1VfhnnviuOWWzvTr15mbbipd3UN2x3bOTnYXUrPmN+Tmnkfz5uO4++43SxyLt+LCggUL2LDhezLSe1Cp0jnFmiatol9YEoyItC7qdWPMVuAiYK/nUCNgqZ/rHPZ5vtkYMx9oA2iCUY6rUAEGD7Ye+/bBvHnw+uvWmE2XLlZ3UffukJBQ/M+IxH4zhbWa1njq7Ccm3ggMB0ZTrdpCGjWaRWzs2d1Xwba+fFstbdvWolIlN/cPX8b01ztoiZgyKlq6yN4H7gNWegb5rwUGFTzJGHMBsEtE3MaYGkAXYFRYI1XlUq1aMGyY9di/H+bPt/Y+GTbMKqrZo0ce8fFfsm/fymKXnHe73Rw9epRJkyadcY1gE0/Bumxut5uYAIt93G5o0uQOdu7swdq1PahZczkXXfQ43333GvDnoH6OwhRc3PnoI1fR89aFZCzdTlL7+kFNnFClQ7QkmAnAm8aYTYALuFdEjgAYY54CdojIVOB2YLgxJhcr9hkisiBSQavyqWZNGDrUehw4APPm5fH00yvYseNyqla9lCpVvqFx42m89da9XHyx/S2hDxzYTfXqcZ56aqfHJILhbSX88ssabunSgKeffp3Nm/eRnb3Hb+uiTZte7N7dhf37R3DyJIwcmUeTJst45pkRxMQUvfdNsK0vf9O927W9kFdeyeK559aWiYkT6kxRkWBE5CjQv5DXnvB5PgWYEq64lAqkRg1o1CgdY1KR9Z3ZtKku6emNGTP2Ulq1clGzZhxJSdChg7XWpmlTOMfPv7qMjAzq10/wW7olGN5WQvqSXlYr4dHWJHWYy969h/LPcbtjSE+3uvtWrHibmjW/Yvp0aNPGxT33DGHZMmHggAv5bHE2Q4cOcWzg3d9076WZu7nssrb06tVLS8SUQVGRYJQqzby/mVeoEMvll+/l8sv3kpOzikqVs+jc+WEyM60utdGjYft2aNbMmp2WmAjNm8Mll8B334mf3+5r8be//Y0fPfOl7cwu820luN2wb995XHZZT7ZsOZeWLUezZg18/701ieGee2Dr1oHEx+fQvn0X0tIKdGE92trRgXd/izsbNGjB888/r4mljNIEo8q9km7ZW9RCzMREK5GMGGGde+wYrFsHa9dCVhbMmGGtv9m0aTh5eYOZ+dYpzj//JOdVO853q9oAx4FNwEG2b69DXNxx3O54XnoJTpywtiDw/vfkSdi0qT/ffdeC995vwaZNtahY8XdOndpAfPx2GjeG226DK6+0uvkARow4XTCjJPXJ7IwPaWHL8ke3TFblmhNb9trdzrkov//uYvDgEWRn76VlYiO++TaXhATDwIH/x3PPvc7vvyfQo8cd5ORYs9sqVoRKlc78b8WKUKFCHu+++wqnTq2gS+fjrPjmF3bsOGpri+e0tDQmTkzNb8Hk5lrddCkp43VmVzlX3C2TNcGocs2pm6q3FbR27VpatCjeb+aFXaO4s8iCjcWJRKnKpuImGO0iU+WaU1v2OlFy3qmy9cW9jnZhKadFSy0ypSKiNGzZm5mZGbaSMZGuC6fKFm3BqHJNt+xVKnQ0wahyTbuFlAodTTCq3NMtewMr6VRuVT5pglEqAkrTDVtL66vi0gSjVJg5dcMOV5IqWKRSS+sru3QWmVJh5nvDTk1tzUcfdmbnzvVB1R3zJqmJE1M9xTFTGTp0SEh2hSxqKrdSRdEEo1SYOXHDdiJJ2VUapnKr6KQJRqkwc+KGHc5WRXJyMvXqWVO5x41bRc9bF+tUbmWLJhilwsyJG3Y4WxXeqdwpKeOpXOUWUlLG6wC/skVrkSkVASWtXaZ1w1Q4aS0ypUqRkq690QWiqjTQBKNUKaULRFW00zEYpZRSIaEJRimlVEhoglFKKRUSmmCUUkqFhCYYpZRSIaEJRimlVEhoglFKKRUSmmCUUkqFhCYYpZRSIVHeVvLHAezatSvScSilVKnhc88MqhZReUswFwAMHjw40nEopVRpdAHws92Ty1uC+RZoC+wEnN/6TymlyqY4rOTybTBvKlfl+pVSSoWPDvIrpZQKCU0wSimlQkITjFJKqZDQBKOUUiokNMEopZQKCU0wSimlQkITjFJKqZAobwstHWeMuRN4FLgceFhEphRyXhLwCbDBc+ikiFwfliDPjMNWvJ5zhwGPATHAImCkiOSFJdDTMVQB/gNcDfwOpIjIQj/nJRGh79cY0wyYAdQE9gNDRGRjgXPigMlAV8ANjBeR18MRX0E24x0N/B+ww3PoSxF5IJxx+sTyHHA7cDHQUkSy/JwTTd+vnXhHEz3fb03gLeAS4BSwEbhPRPYWOM/Wv0Vf2oIpuTXAQOBtG+f+JCKtPI+wJxcPW/EaYxoDTwI3Apd6HneGPLqzpQCHRaQpcCvwujEmoZBzI/X9TgVeEpFmwEvAq37OGQw0xfoebwRGG2MuDluEZ7ITL8BMn+8zIjc/j/lAO2BLEedE0/drJ16Inu/XDfxLRIyItMQqBTPez3nB/FsENMGUmIhkichPQFh/sy+uIOLtB8wXkb2eVstrwICQB3i2AXhugJ7fslcC3SIQh1/GmDpAa+Adz6F3gNbGmNoFTh0AvCYieZ7fDOcD/cMXqSWIeKOGiHwhItsCnBYV3y/YjjdqiMgBEcn0OfQ1cJGfU4P+t6gJJryaGWNWGWNWGGPuinQwATTizN/AtgINozyOSHy/DYHtIuIC8Px3h58Yo+X7tBsvwEBjzA/GmMXGmBvDGWQxRMv3G4yo+36NMbHAcOBDPy8H/R3rGEwAxphVWF+sP3W9/1BtWAU0FJFDnu6nJcaY7SKyxJFAPRyMNywCxRvEpcLy/ZYjU4ExIpJrjOkMLDDGXCYi+yMdWBkRrd/vi0AOUOjYbDA0wQQgIq0dus5hn+ebjTHzgTaAozdAp+LF+u3Et5ncCHC82R8oXmOMNw7vgGMjYKmf64Tl+/VjG1DfGBMnIi7PYPOFnP1deX8ObzXagr8NhouteEVkl8/zNGPMNiARWBbWaO2Llu/Xlmj8fj2TEy4Fbi1kMo+tf4u+tIssTIwxFxhjYjzPawBdsAbco9UcoI8xpran2TwMeC8CcbwP3AdgjLkUuBb4tOBJkfp+RWSP53MGeQ4NAlYXnIGD9XMMM8bEesY7+gAfhDq+guzGa4yp7/O8FdaMKAlTmMURFd+vXdH2/RpjxmLNDusjIicLOc3Wv0VfWq6/hIwxg4AJQHWsKX5HgS4i8pMx5ilgh4hMNcaMwOrbzMVqOc4QkQnRGq/n3PuwpjQDLAZGhLuLzRhTFXgTuAprD59HRWSB57Wo+H6NMc2xpv1WB37DmvYrxphPgCdEZKWnpTAFK/EBPCsi08IRXzHjnYF1w3Fh/T15UkQ+iVC8k4G+QD1gH7BfRFpE8fdrJ95o+n5bAFlYU/yPew5vFpHbjDFrgO4isqOof4uF0QSjlFIqJLSLTCmlVEhoglFKKRUSmmCUUkqFhCYYpZRSIaEJRimlVEhoglHljjFmqjHmn2H+TLcxpmkhrw02xiwOZzxKhYNOU1ZlhjHmV6xV6ReKyD6f46uBVkBjEfk1QrG5gUtFZJOD15zK6QrXFbC2VfAuklsuIsUqCmqMuR/oJyKdijhnMDAC63tdJiJdi/NZqmzTUjGqrNmMtTr9RQBjTEugSkkv6qkSEBPu/XCKIiL3A/dD/v4iTUUkXFsq7AcmYi26uzpMn6lKGU0wqqx5CxiCJ8EAdwEzgWe8Jxhj3gSyRWSU58+9gf8HNMGqs/SAiHxqjMkEvgSSsErctzTGHMMqVHgzcABrxfhrnuvEYW3QNhSog7Uyuo9P6fZOxphFQG1gFlZlBLcx5k/APSJys+c6buAh4GGgGtYmT48VJ7kZY9oCzwEG+AV4UES+9Lw2DHgca+OxvZ7YNwHPA+cYY3KAHBGpV/C6IvKp5xpnvaaUlyYYVdZ8DfzRGHMZ1g1+IFbRy2f8nWyMuQ4rAfUD0oELgHN9Tvkj1p4XgtUFlY5VVuNCoDmQZoz5WUQygL9itZ66ez77CuCYz7V6YtVvqgZ8B3xE4bWcbgOuARKwCnYKENQOjZ4Nt+Zj7eORgbXb43zPjpZglQy6WkR+NsZcCJwnIuuMMQ8ToItMKTs0waiyyNuKWQasA7YXce5Q4A0RSfP8ueC5b4rIWgBjTEOsZNVDRE4Aa4wxr3s+KwO4B6s+k7do4fcFrjVeRA4CB40xS7HGLwpLMM+KyAHggDHmeazEFewWwHcBc322LPjEGPMTVr0u76SCRM+2Bjs4vX2vUo7QWWSqLHoLuAP4E1brpCgNsbaILYxvGfsLgQMicsTn2BbAWxk30LV2+Tw/htU6sfO5WzyfHayLgDuNMQe9D6xW0YUi8hvWNsMjgV3GmA8Lm+WmVHFpC0aVOSKyxRizGauramiA07cBlxTxuu80yx1ADWPMuT5JphGnWz3ea2UFH/VZGgJrfT6jOK2LbcDrIvKgvxdF5GPgY2NMFeBfwCtAZ878mZUqNk0wqqwaClQXkaPGmKL+nk8HFhtjFmJtnnQBcK6IrC94oohsM8Z8BYwzxqQAzTyfM9hzyuvA055uqE1AS6ztiYuzS+EjxpgVWK2ch4B/F+MaM4CvPJuvZWJNZb4JK3HFYXXRLcWa2pwDeCcR7AYaGmPiRSTX34U9Exrise4hscaYSsDvIvJ7MeJUZZR2kakySUR+FpGVNs77BrgbmAQcwhq3uaiItwzC2hxqBzAPax8P7xjHv7E2ZVsMHMZKXpWL+SMswJoIsAb42HOtoIjIL8DtWDPk9mF1tT2E9e8+Dvg7VrfdfqzJByM8b/0U+BXYY4zJLuTyw7D2DpmE1eo5jkPb7KqyQxdaKhVlQrEoU6lI0BaMUkqpkNAEo5RSKiS0i0wppVRIaAtGKaVUSGiCUUopFRKaYJRSSoWEJhillFIhoQlGKaVUSGiCUUopFRL/H/sR/BTlTcivAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_norm, mu, sigma = feature_normalize(X[:, 1:])\n",
    "X_norm = add_bias_unit(X_norm)\n",
    "\n",
    "from scipy.optimize import minimize\n",
    "\n",
    "theta_history = []  # for plotting cost vs iteration graph\n",
    "\n",
    "\n",
    "def theta_store(value, *args):\n",
    "    theta_history.append(value)\n",
    "\n",
    "\n",
    "initial_theta = np.zeros(n)\n",
    "op_result = minimize(fun=cost_function, x0=initial_theta, jac=gradient_function, args=(X, y, 1, True), method='tnc', callback=theta_store)\n",
    "\n",
    "\n",
    "print('Cost at theta found by Gradient descent: {}'.format(op_result.fun))\n",
    "print('theta: {}'.format(op_result.x))\n",
    "\n",
    "# converting theta_history into J_history\n",
    "J_history = (np.array(theta_history[::-1]) @ op_result.x)\n",
    "\n",
    "# plot J_history\n",
    "fig1, ax1 = plt.subplots()\n",
    "ax1.plot(range(J_history.size), J_history)\n",
    "ax1.set_xlabel('Iterations')\n",
    "ax1.set_ylabel('Cost')\n",
    "\n",
    "fig2, ax2 = plt.subplots()\n",
    "X_old = data[:, :-1]  # (118, 2)\n",
    "y_old = data[:, -1, np.newaxis]  # (118, 1)\n",
    "y_slim = y_old.ravel()\n",
    "# plotting y=1 values\n",
    "ax2.scatter(x=X_old[y_slim == 1, 0], y=X_old[y_slim == 1, 1], marker='+', c='black', s=50, label='Accepted')\n",
    "# plotting y=0 values\n",
    "# X[y_slim == 0, 0] is logical indexing with rows with y=0 only\n",
    "ax2.scatter(x=X_old[y_slim == 0, 0], y=X_old[y_slim == 0, 1], marker='o', c='xkcd:light yellow', s=25, label='Regected', edgecolor='k')\n",
    "\n",
    "# labels\n",
    "ax2.set_xlabel('Microchip Test 1')\n",
    "ax2.set_ylabel('Microchip Test 2')\n",
    "\n",
    "# Specified in plot order\n",
    "ax2.legend()\n",
    "\n",
    "theta = op_result.x[:,np.newaxis]\n",
    "plot_decision_boundary(theta=theta, X=X, y=y, hypothesis=sigmoid, precision=0.1, fig=fig2, ax=ax2, feature_map=(map_feature, 6))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Influence of labda values\n",
    "\n",
    "In this part of the example, we will get to try out different regularization parameters for the dataset to understand how regularization prevents overfitting.\n",
    "\n",
    "### No regularization (Overfitting) ($\\lambda = 0$)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mayank/PycharmProjects/machine_learning_implementations_with_numpy/models/logistic_regression.py:12: RuntimeWarning: overflow encountered in exp\n",
      "  g = 1.0/(np.exp(-1*z)+1)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(<Figure size 432x288 with 1 Axes>,\n",
       " <matplotlib.axes._subplots.AxesSubplot at 0x7ff2c51accc0>)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEMCAYAAAD5zKAAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydd3gU1drAf0kIHaQXkaJADggiTQQpCaFZ6FJF8Xq5IHbUXL18IjZEEBDFBgqKBSsgWFBBQhBREQRUEA8qvSO9E5L5/jgbCGGzO5vM7myy7+959mEzO3vm3SU577w9yrIsBEEQBMFpot0WQBAEQcifiIIRBEEQgoIoGEEQBCEoiIIRBEEQgoIoGEEQBCEoFHBbgFCilCoEXAXsBNJcFkcQBCGvEANUBpZrrU/ZfVNEKRiMclnithCCIAh5lNbAd3ZPjjQFsxNgxowZVKpUKagXatMGZsyAqlWdXbddO3jtNahZ09l1hdDy9dcwezZMmeL82pYFr7wCb70FL7wALVo4fw0hsti1axcDBgwAzx5ql0hTMGkAlSpV4pJLLgn6xSpWBKcvExUVnHWF0FKqFBQqFLz/x2eeMTcjt9wC994LDz8M0RJxFXJPQKEF+ZUTBBewrOBv+O3bw/Ll8Nln0K0bHDgQ3OsJQlZEwQiCC6SnG2s02FxyCaSkwGWXQdOmsGpV8K8pCBlEmotMEMKCUFgwGRQsaGIx11wDHTvCmDEwaFBoru0E6enpbNu2jWPHjrktSr4nNjaWChUqULJkSUfWEwUTJAoVgu+/N3eOwVi3Th1n1xVCR3o6LF0KhQuH9rp9+0KDBtCzJ5w5A7ffHtrr55R//vmHqKgolFJESyApaFiWxYkTJ9i+fTuAI0pG/reCxFtvwYgRJrh65oxz677xBowaBQ88AKmpzq0rhIYDB6BrV/j5Zxg9OvTXr1sXbr0VNm4M/bVzysGDB6lYsaIolyATFRVF0aJFqVKlCnv27HFkTfkfCxLNmsGKFbB6tQm27trlzLpNmph116+Htm3Bc7Mh5AFWrjT/f7Vrm7hIlSpuS5Q3SEtLIzY21m0xIoYiRYqQ6tDdqyiYIFKuHMybBwkJJsC6xKESzzJl4NNP4brrzLrJyc6sKwSPqVOhUycT/5g4EWS/DIyoUGRECICz33XYKBilVFml1DyllFZK/aaUmq2UKu/lvKJKqQ+VUn8ppf5QSnV2Q167xMTA44+bDaZ3bxg/3gR4c0t0NDzyCLzzDgwYYOoe0tNzv67gLCdOmID6c8+ZG4w+fdyWSHCCQ4cO0aBBA0aNGhXya69bt4558+bl6L3btm3j6quvdlii7AkbBQNYwLNaa6W1vgL4Gxjj5bwk4LDWuhbQBZiqlCoeQjlzxLXXwrJl8NFH0KsXHDrkzLqZax26d5dah3Bi40Zo2RKOHYOffpLEDDdISEggISHB8XU///xzrrzySr744gtOnz7t+Pq+WLduHV999VVIr5lTwkbBaK33a61TMh36Eaju5dS+wBTPe/4EVgDXBV1AB6he3dzFVqoEV10Fv/7qzLoZtQ6XXiq1DuHCvHnQvLkJqL//PhQP+1sgIRBmzZrFnXfeiVKKhQsXAnD69GnGjh1L586d6dq1K3fdddfZ86dMmUKXLl3o2rUr/fr1I93jbvjkk0/o3bs3PXv2ZODAgWzYsAGA2bNnc9tttzF06FCuv/56Bg4cyO7duzlw4ACTJk3i+++/p1u3bmctqF9++YVbbrmFnj170rNnT1JSUs5ee8aMGXTo0IEePXowc+bMEH1DHizLCrtHXFxcdFxc3DdxcXH3enntSFxcXPlMP78SFxf3gM11a8TFxVlbt2613ObNNy2rShXn1/3gA8sqV86ypk51fm3BHnv2WFbx4pa1ZInbknhn6lTLql/fsjZudFsSe/z+++85fm98fLwVHx/vnDCWZa1bt85q27atlZ6ebs2dO9caNGiQZVmW9eKLL1p33XWXderUKcuyLGvfvn2WZVnW7NmzrT59+lhHjhyxLMuy9u/fb1mWZS1fvtwaPHjw2fNTUlKsvn37WpZlWbNmzbKuuOIK6++//z679j333HP2tYznlmVZhw4dsrp162bt3r3bsizL2r17t9W6dWvr0KFD1rp166yWLVtae/futSzLsh577DGrWbNmfj9j1u9869atVlxcnBUXF1fDCmAvD9c6mBeBo8BLbgsSLHr0gPvuc37djFqHG2809TIvvQRFijh/HSF7TpyA0qWhVSu3JfHOv/9tXLRXXw1vvgnXX++2RM6S2SW2ePHiC45lvrvPCTNnzqRbt25ERUXRsWNHRo0axe7du1m0aBH/+9//KFiwIABlypQBYNGiRfTv35/iHjO2dOnSACQnJ/PHH3/Qu3dvwNzsHz58+Ox1mjRpwmWeQrrevXvTpUsXr/KsWrWKbdu2MXjw4LPHoqKi2Lx5M6tWrSIhIYFy5coB0LdvX7788stcff5ACDsFo5QaD9QGumitvYWtt2BcZ3s9P1cDFoVIvDxB3brG5z9kiKnenjlTui8L54iKMnVUV10F/fsbhfPYYyYhRfDN6dOn+fzzzylYsCBz584FIDU1ldmzZwe8lmVZ3HjjjdyXyztNy7JQSjFjxowLXlvlsr88bGIwAEqp0UAToLuPoTYfA7d7zq+NmfGSNyJeIaR4cTMuYNAg067900/dlkgIN1q3NjVVS5aYlPe9e/2/Jy+QkpJy9hEfH098fPx5x3LDwoULufTSS/n2229JTk4mOTmZN954g08++YS2bdvy1ltvnQ3679+/H4C2bdvy/vvvc/ToUQAOeDJxEhMTmTt3Lrs8RXJpaWmsWbPm7LVWrlzJpk2bABPzad68OQDFixfnyJEjZ89r1KgRmzdv5scffzx77Ndff8WyLJo1a8bixYvZt28fQMhjMGFjwSil6gHDgfXA90opgI1a6x5KqdXA9VrrHcA4YLpS6i9M6+ghWusj2a0byURFwd13m8B/377GZTZqFBQIm/91wW0qVYIFC0zXiSZNTJajZx8TvDBr1qwLXFWNGjUiPT2dZs2aceTIEbp3705sbCzVq1dn0qRJdO/end27d9O3b18KFChA0aJFmTFjBldddRXDhg3jjjvuIC0tjdTUVK699lrq168PQOPGjRk7diybN2+mXLlyjBs3DoAWLVrwxhtv0LVrV5o1a8aIESN45ZVXGDduHKNHjyY1NZWqVasyefJk6tSpw9ChQ8+66Nq0aRPS7yvKcqIoI4+glKoBbFy4cGFI5sH44tAhqFbNuXRlf+zda+plUlNNVlOQ561FNFu2mPjLli1uSxIYn34K//kPPPqouTEJl9rGdevWUbdu3Ry9NyP2klvLJdTMnj2blJQUJk2a5Mr1s37n27Zto127dgCXaq032V0nrFxkQvAoXx6+/NJM2mzSxDRbFITMdO0KP/wA06bBTTeBx6OTp3HCLSbkHFEwEURMDDzxBDz1lGnCKQhZqVnTKJkiRUw/vXXr3JYoMunZs6dr1ouTiIKJQOrWhbSABp8KkUSRIqZr94MPGov3gw/clkjIq4iCiVAiKPQm5JBBg2D+fNPz7r77IMQdUYR8gCiYCCRcgrdC+NOokUll3rjRdAXfts1tiYS8hCiYCEUsGMEupUvDnDnQpYspzvzmG7clEvIKomAiELFghECJjobhw03x7i23mHoqGQ8h+EMUTIRy8qRYMULgJCYal9mXX5q0Zk+xekSRmJjItddeS9euXbnuuuv4+OOPQy7DsmXL+O6773L83p49ezoskXdEwUQgdeuaO9J+/eCI9EAQAqRKFTMeIi7O1FT9/LPbEnknLS2NBQsWMHHiRBYsWECag6mTkyZN4tNPP+WFF17giSeeYPfu3Y6tbYeffvqJpXmgmE2ahkQgJUuatjH33GNqHWbNgssvd1sqIS8RG2umdLZoYYbpjR5tugCEi/s1LS2NQYMGsmuXJr5NeSZMmMGMGYpp094mxsGunnFxcZQsWZLdu3dTsWJFXnvtNebPn09aWhoVK1bkqaeeonz58hw5coT/+7//488//6RixYpUrFiRsmXL8vDDD3P69GkmTpzI8uXLOX36NEopHn/8cYoVK8aRI0cYPXo0a9asISoqiqZNm9K3b18++OAD0tPT+f7777nhhhsYMmQIixcv5tVXX+X06dPExsYyfPhwGjZsCMDEiROZN28eJUuWpFmzZo59fn+IgolQihQxY5zfeAPi4+HFF41FIwiB0Lv3ufEQS5fCK69A0aJuS2Va4e/apfns0w7ExkaTlJRO5y7zSU5OpkOHDo5d5+eff6Z06dLUqVOHuXPnsnXrVj766COio6N57733GDNmDBMmTODll1+mZMmSfPXVVxw8eJCePXvSqVMnAKZOnUqJEiXONqIcN24cr732Gvfffz+jR4+maNGizJ07l+joaPbv30+ZMmXo168fx48f52FPxfSWLVt45ZVXmDZtGsWLF+fPP/9k8ODBpKSknG3KOWfOHAoXLnzeILRgIwomwvn3v6FxYzPG+fvvYfx48IyzEARbKGXGgd9+u7FoZs2CWrXclWnNmjXEtylPbKyJAsTGRpMQX4G1a9c6omDuvfdeLMtiy5YtvPDCCxQsWJDk5GTWrFlDjx49AGNFZcyAWbZsGSNGjACgVKlStG/f/uxaycnJHD16lK+//howIwHqeOZrL1q0iNmzZxMdbT5HxoyZrCxZsoQtW7YwYMCAs8fOnDnDP//8w7Jly7j++uspVqwYAL169eKVV17J9XdgB1EwAg0bmsDtrbcaa+bjj80YZkGwS7Fi8M47MHmymUH02mvQvbt78tSvX58JE2aQlJRObGw0qanppCzeQ1JSPUfWnzRpEnFxcXz55ZcMHz6cxo0bY1kWd9xxB7169QpoLcuyeOyxx2jRokWuZGrdujXPPvtsrtZwGgnyCwCUKgWffGI2Bal1EHJCVBTccQd8/jkMGwYPPQRnzrgjS2JiIpUqKTp3mc8zz6ykc5f5VK5ch8TEREevc91119GyZUumTJlCYmIi7733Hoc8LdJPnz7NH3/8AUCzZs3ODig7fPgwCxcuPE/W6dOnc/LkSQCOHj3K33//DZhZMtOmTSOj633GjJmsM2FatmzJkiVL+PPPP88e+/XXXwFo3rw5X375JcePHyctLY1Zs2Y5+h34QiwY4SzR0aYJZrNmprX/nXfC//2fOS7YJ9LTv5s1MxbxzTeb5/7cZTVqwOOPOxu7iYmJYdq0t0lOTmbt2rUkJdUjMTHR0QB/Bg8++CA9e/Zk8ODBHDx4kJtvvhkwlkn//v2pU6cOd911F8OHD+faa6+lfPny1K9f/6z7bMiQIbz00kv06tWLqKgooqKiuPvuu6lZsybDhw9n9OjRdO7cmZiYmLPzX9q3b8+cOXPo1q3b2SD/uHHjeOSRRzh58iSpqak0btyYBg0a0LZtW1avXk23bt3OBvlDlfUm82BcItTzYAJl+3YzpKxUKXj7bcjG9St4Yd066NlTOhGnpZl6mePHfZ83Zw6sXZt97CY382DChdTUVNLT0ylUqBBHjx6lf//+DB8+nGuuucZt0bzi1DwYsWAEr1SpAosWGYumSROYOdP8K/jn4EG46CK3pXCfmBjo3Nn/eb17w6uvhkfsJlgcPnyYwYMHk5aWxqlTp+jcuXPYKhcnEQUjZEtGrcM115iZ7aNGweDB4VPrEK4cPGgsP8EeUVHGHdukCfTpY7IZR4/OX6O9y5Yty+zZs90WI+SId13wS69esGQJTJoEt93m3+UR6Rw6JAomJ1x9tekK8Msv0K4d7Nx57rVIcuW7jZPftSgYwRYZtQ5nzphah0zJKkIWxEWWc8qVg3nzoG1baNoUvv3WBOxTU1PdFi1iOHHiBLGxsY6sJQpGsE1GrcPQocZt9sknbksUnoiLLHfExJissmnTTHxm1apS7N69m3Rp3xxULMvi+PHjbN++nQoVKjiyZj7ycuYtUlKgcGG3pQicjFqHJk3MH/8PP+Q/f3luOHYMFi40/bmE3HHttfDTT9CvXzlOndpGkyZaUuaDTGxsLBUrVqRkyZKOrCfbQohJS4MnnjA9wDyth/IkzZoZf/mAAcZf/uGHUKmS21K5y/r1Jj25aVOjhIXcU706pKRE88AD1XjqKZPK3KCB21IJdpH7gRCyd6/JxlqyxBSitW7ttkS5I6u/fMkStyVyj9mzoVUruPdeePPN8Gj4mF8oVAheftm4zdq1M3VZQt5AFEyI+PFH41Zq3BgWLMg/d/sZ/vKpU0222YQJkVXJnp5uuh088AB88QUMGSJp3MFiwABTm/X006axpqezihDGhJWLTCk1HrgRqAFcobVe4+Wcx4E7gR2eQ0u11qHrPx0glgUvvQRPPWU24a5d3ZYoOGT4yzO6Mr/xRv7PpDp2zIwP/ucfY5GWK+e2RPmf+vVh+XIYNMhYjDNnmlYzQngSbhbMHKANsNnPeW9rrRt6HmGrXI4ehZtuMtkwP/yQf5VLBtWrw3ffQcWKpmHmb7+5LVHw2L4d2rQxw9sWLBDlEkpKloSPPjK9zq6+2rhphfAkrBSM1vo7rfVWt+VwgnXrTCC8aFGjXGrWdFui0FCokBk6NXKkmd/+zjtuS+Q8K1dC8+Ymi+7NN81nFkJLVJTp2Dx7tnFLjhhhLMl9+7J/HDzottSRR1i5yAKgn1KqI7ALeExr/YPbAmWlTx+TSXTPPW5L4g4332zmzHTtagaY9e3rtkTO0bs3jB1rrFPBXVq2NNmM//qXKQb2xcmT5vfyhRfyZolAXiSsLBibTMZ09GwAjAPmKqXKuizTBRw7Bjfc4LYU7lK/Prz3nrnT9IyxyBccPWqsMyE8qFjRdG32Zb3s22daz+zbZ2I3mza5LXVkkOcUjNZ6l9Y61fN8AbAVqO+uVEJ2NG9uAv///a/bkjiHZUmmWF6kZEkzrXXAAIndhIo8p2CUUlUyPW+IyTjTrgkk+GX0aBMIX7TIbUmcQRRM3iUqCu6/3xRsDhkCjz5qip+F4BBWCkYpNUkptQ24BPhGKbXWc3yeUqqp57TRSqk1SqlfgNeBW7TWu1wSWbBBiRImVfv22+HECbelyT2iYPI+rVqZ2M3SpSbFfu9etyXKn4RVkF9rfS9wr5fj12d6fmtIhRIcoWtXk1E2apQplBMEt6lYEebPN1ZMkyYm9bl5c7elyl+ElQUj5G8mTTITC3/5xW1JcodYMPmHAgXgmWeMhd21K7z4YmR1ogg2omCEkFG5spmQeeONcOCA29LknDNnpHt0fqNrV1Ov9sYbJv386FG3JcofiIIRQsott5g/5n798mZw1bLM5lOsmNuSCE5Ts6Zpc1SsmCmSXrfObYnyPqJghJDz7LOmSeTw4W5LEjinThnrxaGBf0KYUaSI6RmYlGRaAX3wgdsS5W1EwQghp0AB84c7a5YpxMxLHDsGxYu7LYUQbP79b5MA8MgjZgTD6dNuS5Q3EQUjuELZsjBnjqnyX7nSbWnsI+6xyKFRI5PKvHkzxMfDtm1uS5T3EAUjuMYVV8Crr0KPHrBnj9vS2EMsmMiiVCn45BPo3t10CF+wwG2J8haiYARXufFGGDjQtJPJC5k7hw+LBRNpREfDww8bd+6tt5pmmYI9/CoYpVQjpdQdSqk2Xl4bFhyxhEjiiSegTp3wz9yxLNNFuX17tyUR3KBtWzMe4PXX3ZYk7+BTwSil+gGLgO7ALKXULKVU5mnjTwZTOCEyiI42BZgPPmgydz780G2JvPPWW7BhgxkRLUQm4h4NDH8WzAjgBq11J+BSIBVYoJQq4Xld6pkFxxg0yGTuDB9uZukcO+a2ROfYuNF0hH73XRkwJgh28adgqmqtlwJorY9qrfsBK4EUzwwWaaogOEqjRrBqlVEujRrBsmVuS2QKQgcOhP/9zyQmCIJgD38KZq9S6rxhv1rre4BvgCWAlJsJjnPRRfD226bNf9euZvxyaqp78owfb2p37r/fPRkEIS/iT8F8AVzQvVhr/TDwMSDOAiFo9OoFq1fDihXQogX88UfoZfjuO5gwwcRfoiXnUhACwuefjNb6Pq31yGxeewyQydZe2LfPpLPKhpR7KleGL76A//wHWrc2bTxCxYIF0LOnibtUqxa66wrhS3S0adR68KDbkuQNcrUFaq2lgUIWli83syVuuw2qV3dbmvxBVBQMHWqsifHj4e67g+8yS083/aimToWOHYN7LSHvUKeOKQxu2tRY14Jv5B7bISzLVKXfcANMnAjjxsnMEKdRygT9N240m/6OHcG71pw5pqFlly7Bu4aQ94iONrNjnnwSOnSA6dPdlii8EQXjAMeOmSyjV181I1h79HBbovzLRRfBp59CQgI0aGCU+Zkzzl4jPd0Ufz72mNwkCN656SZISYExY2DwYDh50m2JwhNRMLlEa7j6anNn8+OPULu22xLlf2JizOa/dKmJzzRpYp47xdy5Jmusc2ff56WlpbFgwQImTpzIggULSMuLA26EHFOvnnGJHz4MLVsay1o4H1sKRinl1RmhlNrirDh5i5kzoVUr0857+nQoWtTvWwQHUcoE4v/v/6BPHxP32rs3d2tmWC+PP+7beklLS2PQoIFMmDCckyfmM2HCcAYNGhhSJSMKzn1KlDCjJwYOhObNzQ2PcA67FkyJrAeUUgWAiGyckJoKDzxgKru/+gqGDBFXiltERUHfvqaHWenS5q5yypScT8ucO9dYSP6sl+TkZHbt0nz2aQeGD2/MZ592YOfOP0hOTs7ZhQMkHBScYIiKgvvuM12Xhw41M2Tkv8Hgc7K4UmoBplq/kFJqfpaXqwJhUGcdeh54ANavN7MiypRxWxoBoGRJeO45+Ne/4M47TZzms88CTxUfPx5GjPB/w7BmzRri25QnNtZcIDY2moT4Cqxdu5YOHTrk7EMEQGYFFxsbTVJSOp27zCc5OTkk1xcu5JprzJ7Qp49RMmPGuC2R+/j785sJzALSPP9mPGYC/8M0wYw4tm+H228X5RKONGhggq9HjgT+B37qlGlT06mT/3Pr16/P4m/3kpqaDkBqajopi/dQr169wIXOAb4UnOAeFSqYnnrbt7stSXjg04LRWk8BUEot01pL1reQJyhQwMzuuPpqOHHCtJqJtdHU6LffoFYte7G0xMREZsxQdO4yn4T4CqQs3kPlynVITEzM/QewQf369ZkwYQZJSenExkafVXBJSaFRcIJgB58KJhM1lVIntNba05vsFSAduFtr/XfwxBOEnHHJJabFzKBBxnXx7rsmKcAXP/9sMtLsEBMTw7Rpb5OcnMzatWtJSqpHYmIiMTExuRfeBm4rOEGwg10FMxZo5Xk+AVgPHAUmA445fJVS44EbgRrAFVrrNV7OiQEmAddi4kNjtNYhbCAiuEVaWhrJycmsWbOG+vXr+93QM9rMTJ5s0kifeMLEZ7KLr6xYYSq07RITE0OHDh28xjwClTVQ3FZwgmAHuwqmgtZ6l1KqEBAPVMbMhsllUugFzAFewHRqzo4BQC2gNlAWWKWU+kZrvclhWYQwIiNratcuTXyb8kyYMIMZMxTTpr3tc1ONijKzZRIT4ZZb4PPP4Y03jPLJyooVpueZHVl8KY+cyhoovhScIIQDdnNs9imlagAdgZ+11ieBggG83xZa6++01lv9nNYXeF1rna613otRSr2dlEMIP3KbFqyUKcZs1szMmZk92xxPSEggISGBkydN0WyDBr7XsZMe7HYKsyCEC3YVxGhgFfAWxkUG0Bb4NRhC+aEasDnTz1swKdNCPsaJrKnYWOMmmznTjALYt+/cawcOmOSAIUNMLCY77CgPyfCKbC6+GJKT4aef3JbEfWwpGK3165iRyZdprb/0HF6NcVcJQtBxMi34wAEzX6Zs2XPHKleGTZvMxMoePUyHhpkzL+xzZkd5uJ3CLLhLu3amIWbnzvDKK6YRbqQSiIvLAjoope7z/HwGE4cJNVuAzI3wqwH+3GpCHicxMZFKlUzW1DPPrKRzl/k5zpq67bYl/PPPsyQkJLB48WIWL15MQkICPXsmMG9eAhs2mMrsiROhZk2TJJCxSdhRHk7IKm1g8jY9ehiX7OTJJvZ37JjbErmDrSC/UuoaTKzjd6AJJhBfH7iX0BdbfgwMVkrNxgT5uwOtQyyDEGICyZryFYTfswcOHWpInTqjs71WgQLQu7d5LF8Od91lugK8+aa99ODsZAVYsGCB38yyUCUJCMGldm3TAPeOO0xN1qxZ/lPl8xt2s8gmAQO11l8ppQ54jv0AvOekMEqpSUBPoBLwjVJqn9a6nlJqHjBSa70CeAe4GvjT87YntdbSxzQfk1Vh3HPPPdlutP4253ffhQEDSjB9uvH0JiQkAJCSkuJ1vauuMneijz0GDRvCm2/aU3RZM7wCURrSBib/ULSoaYT7+uvG7frqqyb+FynYLrTUWn/leZ7hUTyFySRzDK31vRirKOvx6zM9TwPucPK6QvgS6N28r825ffsOTJtm/sgDITYWRo82A6YGDoQ+fWIYPTqw9OBAlIbbfc7sEOw6n/xEVJRJHmnSxCiX77+HsWPtdZfI69iNwWilVNssxxIASYsRgkqgKb++NueffoLTp6F1Dh2qbduaMbkbNpjW7L8GkEMZSGZZuCcJSCfnnNGkiclQ1Nr8Lu3Z47ZEwceugvkv8LFSagpQWCn1AvAu8FDQJBMEAk/5zW5zLl++MTffDA8/fH4lf0pKSrbuMW+ULWtqaO64w1g0PXrYS0cNRGk4mdAQDKTOJ+eUKWPieZUqwYwZbksTfOymKS/BBPe3A+8DB4CWWusfgiibIAR8N+9tcy5RohWPPtqaO++0V6nvjwyXx8aNpkNAr14QH+89rdmXXNkpjYwkgaSkMRQp2omkpDFhFeCXOp/cER0N1aub4Xb5HX/zYL7QWt8AoLXeDDwZEqkEwUOgTR2zZnANGNCUkSNb8uCDUdxzj7OyFS0K99xjhkzNng2DBv3KzTdX4tFHKzB4sGndnp1c/nqHOdkGxul4iXRyFuziL8gv6b+Cq+SkqWPG5lyjRgfatTMjlYcOzbkM/jLNYmPNVM1XX72Xo0drsXHjVJSCLl2MArrqqvPlCmWgPhgpzxIUnDkAACAASURBVNLJWbCL3SwyQXCNnGzMu3ebiurHHjMt+zPjT2HkhuLF/2LqVHj2WZg2zUw3rFDBKJo+faCgo3mX/glGyrN0chbs4k/BFFZKve3rBK31QAflEYRck54Ot95qUoqzKhenyVBWAIsXL77g2F9/pTBvHjz/PDz+ODz9tCngjI4OzHWVUzdXsFKepZOzYAd/CsYCZKCYkKcYMwaOHjUbek7xpzjsWj8xMcZV1qULLFxostjGj4cnn0zjgw8Gsnu3f9dVbtxcEi8R3MSfgjmltX4iJJIIggM89xxMnQqLF5uWLxk4pTCykvl9/lxv7dqZlOaZM2Ho0BMcPvwQM979meuu+9un6yo3bq5A4iVSPCk4jT8Fk83sP0EIP0aPNv3CFi+Gqrkc4BCI4giE6GgTi9m6dSrLl58k6b//ZtTTJ3j8sWTi26z26rrKjZvLbrxE+p8JwcCfgnk3JFIIQi6wLOMO+/hj+PZb79Mqg6UwckqDBvVYsGA4K38+zJw5Dbhv2PXs3NmUPn0KsXIlXHmlca9B7t1cduIldq0ksXKEQPCpYLTW0vNLcAW7G5llwfDhMG8eLFyYxpo1yXzwQc43v6zKJyEhgdWrV9OwYUO/77WjsDLWX7hwITNmKLp1/5qE+FVUrTqBypX7UaDA/QwYYLLg4uNNIWd8fCIVK/p3c+VGcdqxksTKEQJF0pSFsMPuRmZZcP/9sGQJfPNNGg89FJzNr2HDhl437dxs6FldV//9b4brymzwO3fCokVmMuLzz8dw9Oi71K27m19+/Y2hQwvQtWsbRzd1O1aSdHkWAkUUjBB0AnWr2NnI0tPNnJZVq0x21vLl9jc/N11jmfHluqpcGW66yTwANm2KYtGiSsybV4nbboPrroN//Qvatz/nSguErMrRTjJAXujyLIQXomCEoJITt4q/jSw1FQYPhr//hvnzoWTJ3G1+3jLMSpUqBcChQ4cuOCdQBeVEBluNGnDbbeaxfz+8/z6MGAFdu+6lcuUvqFz5M378MTfp1P6TAZxOeZZ4Tv7HtoJRSv0b6A9cDOwAPgDe0FpH8MRpwR85cav42sgOHYIaNVYQE3OKzZtbUqyY//c4yerVq89u4E6mOwdCmTLGervrLmja9CF27uzKihXTgT7AS1jW+R2j7eIvGcDJFjESz4kM7I5MfhboBjwPbAaqAUmAQlr2Cz7IiWWR3UYWF5dI69ZQtOhWatV6kWLFkv2+x87m5yvDLMOSyRz0D5RgZrCtWPEmAIcOQdOmu9i+fQbbtlXjqqugUSNYsMD8W65c7i2p3I6CzozEcyIDuxbMv4DGWuttGQeUUl8AKxEFI/ggJ5aFt42sbNlEWrWK4b774LPPXrjgDt3J/liZrRRfLrJwSHfO7Ga66KIZVK78CS+/nMKqVSY+9fTTZkhaiRJw4sQoihf/i+LF/8RM3tjmb/kLyM0o6MxIPCcysKtgjngeWY8ddlYcIb9h17LIulln3sjmz4fmzQ9Qq9YkPv98Ed9+6/3uO9L6Y2Xd3IsWPcGBA0e5/PI0rrgihoGeLoHp6WZ2zapVrVi1qhUrV8Lff9/JqVPl+f33glSrZgpT77vP/Futmnk0bAiFC/uWIaeWiLSwiQzsKpjngdlKqTGY256qmCmXE5VSl2WcpLXe4LyIQl4mt5bFm2+aOpfLLx9JqVK/BVnaC60Rp60UX+sEeq0LN/eGtGs/94LNPToaatY0j169Mq41AMuK4sMPF7F1K2zZYh5bt8KPP8KmTebRvz/8+9/GzeYtKJ9TS0Ra/kcGdhXMC55/22Y53g6Y5HluARKdEy4gJ5ZFRnX+O++Y1i9KvXj2tXBwTYXD9b1t7p06VrXtZoqKsqhUyYzvzZhZk5nNm2H6dDMWukwZi4IF36FYsbdo367wWVdY//4DmTgxcEtEWv5HBrYUjNba1mhlQQiE7ILOaWmFWL8+ierVO/DDD1CxoksCZsFblX/mn0ONNzfT1/O38uij/t1MdmSuXt3M03n0URg3biXPP1+ZEyfmUe/yVbz+2iL+M3g2wNlR0IFaIpHm0oxEpA5GCCvS02P57bexFCq0l5QUM5bYTYKtPLJTsqtXrwbg4MGD2b43MTGRBx44Srv2c+nUsSrzvtzEli0HGTVqFE8//TTgXIPOggW/5d575jNo0M8880w8DRvdT/36tVm9+k+xRIRsyVbBKKW+0lpf63m+BOMCuwCtdZsgySbkcfwV0mVN37WsKKpUWUCFCvDRR2Zj84bbrim38JYIUaZMRY4dO8aHH+1k/frNnDp1ipo16zp+7XPW0hEmTpzHvfd+yzUtG7Ft23X06SOWiOAdXxZM5kmWU4MtiJC/yEn66saNg0hNNa1fslMudnHKfRXsKv/samTs1ttkyJT1/U7jLSjfseMaGjXqSIsW8O67pm2NIGQmWwWjtX4v0/O3QiGMUioOeAsoC+wDBmqt/8xyzuPAnZhuAgBLtdZ3hUI+wT6Bpq/u2NGFf/6JZ+VKKFLEBYHDgAy3WEJCgtdCyFCTWWFlH5SPpmFDk2324IPm4a2LgLSFiUwCaRXTEWgIFM98XGs90kF5JgMva63fVUrdDEwBvEUL39ZaJzl4XcFhAklfnTcPjhx5gFdeWco778wMyQZk927fVxV+KIL8GUoHnLGYckN2QfmEBFi2zGSbrVxpJopmjp1JW5jIxW6rmJcwjY4WAceDIYhSqgLQGMj47X0feEkpVV5rvTcY1xSCh91Cup9/hltvtWjW7HE++OCLbDcgO5t5sMYih4rMAX1vlktmd5g33Px81arBd9/B7bfDNdfAJ5/ApZea16QtTORi14K5CbhSa701iLJUBbZrrdMAtNZpSqkdnuNZFUw/j0W1C3hMa/1DEOUScoCdQro9e6B7d7jzzl9ZtuwL2YC8EOpJnLlR0kWKwFtvwaRJZljaJZf0pGDB/XTr1k3awkQodhXMP0D2+ZKhZTLwtNY6VSnVAZirlKqrtd7ntmDCOfwV0qWlmVknAwdCqVLJjmxA/jbj3Fo4WV8Pd4vIDaKiTMuZAwdg4sQnuPLKB6QtTATjK035skw/TgBmKKWeAXZnPs/B9jBbgSpKqRiP9RKDGQ1wntWktd6V6fkCpdRWoD7g238ghBxfhXRPPGGq9Z98EpKTvW9A+/a9y5w5c4C86fLKDW59NqcsppEj4eWXD/Dnn8OkLUwE48uC+QtT+5I5J6RzlnMcaw+jtd6jlFqNmTnzruffVVnjL0qpKlrr7Z7nDYEagHZCBiE0fPklvPGGib/ExGTvTjt5MvBuv75wy90Uij5m4ULmG4B//lkBrOTKKx+lbNltHDsWRZGinaQYM4LwlabsRnuYocBbSqmRwAFgIIBSah4wUmu9AhitlGoCpAGngVsyWzVCeLNli5nK+PHH51rA2OlLFehmnRc24/zPMeB2/vrrA0qVWk3x4lEMGzbMbaGEEGI3i6wKcFxrfSDTsdJAEa31juzfGRha6z+Aq70cvz7T81udup4QWtLS4Oab4YEHoHXr81+TvlThR06U9IVWokXNmhUpVmwekyZl9y4hv2I3yD8H+DfGqsjgEkyF/wUKQRC8MXYsFCgASWFQweRk23xv74XIixtlx7hxUK8eDBgAV8tucRYrAobN21UwcVrr84ZxaK1/U0rVCYJMQj7l5Zdz3gYmEjfm/EKZMnDPPfDhh4ErmPzaAeCaa+Cuu6BBA+jY0W1pgoddBbNXKVVLa/1XxgGlVC1MOxdB8Mvhw3DwIMTFnX/c7Zb3ThPqRIJwJvPnrlABNgSYb5qfOwDceCOULWtS9YcOhREjct9/Lxyxq2DeAGYppR4BNgA1gaeQJpiCTbQ2yiWUf0SBbPDi2gouJUuam4xAyO8dABISTCZl377www+mYWjZsm5L5Sx2/9zHYFKHxwPLgXGen8cESS4hjElLS2PBggVMnDiRBQsWkJaW5vc9WoNSIRBOCEtKlAhcwfjqZ5dfqFzZuI3r14cmTWD5crclcha7Ey3TMUplXHDFEcKdnLot/vgD6ngiduFoLQTDtSVWzzlyYsFESgeA2FiTCNGiBdxwgyk+vv12712p8xqBdFNOwNSlVAG2A+9orRcFSS4hTPHntsguKKs19OwZfPnCUXnlB3KrdHOiYCKtA0DPnsaS6dULli6FyZOhWDG3pcoddutg/gOMxsRclgHVgPeVUo9qrV8PonxCmOHLbZGYmJitdaN1zFkXmQTCIw9vCsZfhpidAtz8Rlwc/Pgj3HEHNG8OKSl5Oy5j14J5COigtf4l44BS6kNgFiAKJoLw5bbwZd1ERYUmKOuE8hJl5zypqcYVlIFdV2skFuAWLQrTp0PLlvDLL5CXDTa7CqYs8HuWYxoo46w4Qrjjy20xadKkbK2b2rU78Ndf0Lixyx9AsI2T7sa9e6F8+XM/5/cMsdwSFQWFC7stRe6xm0X2HfCcUqoogFKqGCbg/32wBBPCkwy3RVLSGE/jwjFn7zrr16/P4m/3kpqaDnDWuqlXrx61asGff164XkpKit+NKiEhwefoYH+vC+6TVcFEQoaYYN+CGQp8ABxSSu3HWC7fYzoeCxFGdm4LX9bN5s0mcBlKxNWVO5yMlWVVMJGSIRbp+FUwSqkooAjQDqiEmdGyQ2vtbC91Ic/jKyhbu7aZdihEJnv3Qrly536OtAyxSMWvgtFaW0qp34ASHqUiikXIluysm+xcZNnhz/+fGafTkSWzzXn27oVKlc79HIkZYpGIXRfZKiAO+COIsgj5mMqV4ciRC10lQt4gq7INVAnv3QtXXHH+sUjMEIs07CqYFOArpdR0zAjjs42mtdZvOC+WkN+IjoZbbjEVy88+6//8QPz/YnGEP1u3QtWqbkshhBq7CqYlsBGIz3LcwjTCFAS/jBxp7mLvuw+qVHFbmvORDgDBZeNGqFHDbSmEUGO3F1nbYAsi5H8uvhgGD4b/+z8J+OdFcqqEz5yBnTvFgolE7LaK6Qhs0lqvz3QsDqiutV4QLOGE/MeIEWbI0uefQ+fO9t7jz3pwwrqQ9jXBY+tWqFgRChZ0WxIh1Nh1kb0MtMly7KjneNyFpwuCd4oXhzffNIOWfv01b/dZijRyqoQ3bYJLLw2KSBeQXydg5lXsVvJX0FrvzHJsJ6YuRhACIj4e+vQxY3SF/I9T8Rd/c4gy+ptNmDCckyfmM2HCcAYNGmhrXpEQHOwqmA1KqawVUAmYwL8gBMzo0Waa3yefuC3JhdhpXyPYZ8sWqF49d2vYUR6Z+5sNH96Yzz7twM6df5CcnJzLTyDkFLsusseB2UqpacDfmJHJt3keghAwRYrA66/DgAHQrp1p5+4EEj8JDYF8v4cP5z5r0E5zTF/9zaTWxh1sWTBa67lAR6AYcIPn306e44Lgk+xcG23aQKdO8MgjoZVHmmOGlmPHcj84y05zTF/NVgV3sD3RUmv9E/BTEGUR8iH+5n48+yzUq2csmf/9LwEQ6yOUhMLic0LB2GmOKf3Nwo9sFYxS6hGt9dOe509md57WeqRTwnhSn9/CzJ/ZBwzUWv+Z5ZwYYBJwLabQc4zWeqpTMgjO4s+1UaYMPPccDBkCpUrFEB0deEDWySJJcbE5T24VTFpaGunp6Rw/Xog28Z/S+YaqfLtk7wXKQ/qbhR++LJhLMj0PVYnUZOBlrfW7SqmbgSlA1tuPAUAtoDZGEa1SSn2jtd4UIhmFALDjF+/XzxRerl/fh2rV3g+KHFKp7x65UTCZLeBrry3PouTjLEw+xsMPj6Z9+/YXKA/pbxZeZKtgtNZ3ZHoe9GC+UqoC0BjI+M14H3hJKVVea70306l9gde11unAXqXUHKA3ZgCaEGb4c21kbPInTlRi48aX2LhxbMAbvxRJBoYTyjaQepPcKJisFvB/PRZwdHS0WCZ5AJ8xGKVUNX8LaK23OCRLVWC71jrNs26aUmqH53hmBVMN2Jzp5y2EzsISAsSuX7xIkV3AaOBD0tIeJSbmpKNy+FJCmYP+Yt34x19c7cLzTbPTnCCZYXkbf0H+TZzrnBzl5XULkNsIIVv8+cUzb97x8Qlo3YHSpb/ik08gWDeolmVx7NgxJk6cSP369bEsi6gob7/e+ZPcWnx2UoYzU7ky7NiRM1ll8mXexp+C+QUzzfIt4F0gh78mttgKVFFKxXislxjM9MytWc7bAlQHlnt+zmrRCGGGL794ZlfLsWNHqV17HCdPXseNN8L06VCqVGDX8rdRpqWlsX//bkqXjvEU7M3g0kvP3X2Li80/gVoVVavCthyOKZTMsLyNT8NVa90I6AWUAZYC84B+QEGtdVqGO8sJtNZ7gNVAf8+h/sCqLPEXgI+BwUqpaKVUeaA7MNMpOQRn8NfWI+OczNXZRYue4ODBHcyZk0a1atC4MaxY4axcycnJVKlSnIXfdJVq7xwSaL3JJZeYhpe+yO73JcMCTkoaQ5GinUhKGpOtK04IP/x6RrXWa7TW/wVqAM8BnYGdSqnGQZBnKHCPUmo9cI/nZ5RS85RSTT3nvANsAP4EfgSe1FpLy5owwm5PqKytPRZ+05WLLy7G0qXJTJpkBpNdfz28+CJYVjYXCxA7BXuRRE7a4iQmJlKpkrEqnnlmJZ27zPdpVfizYPz9vmRYwMOGDaNDhw6iXPIQtgstMWnB8UALzAjlA04Lo7X+A7jay/HrMz1PA+7Ieo4QPtj10ftztfTqBY0aQe/e8O23MHUqXHRR7mTz59MX15h/Aq038WfBBBrTEfIOPi0YpVQZpdRdSqmfgDmYFv1ttNZtxWoQssOulWDH1VKzJnz/PZQvD02awKpV9txv2RHo3bfgnQyrYs6cOTz99NM+rQp/FoxYlfkXfy6yHcDdGOVyF8YlVUsplZjxCLaAQt7Dro/e7mZfuDC88go8/TR07GgRHz81xy3Zw8GnH2m90KpUgb174fhx769LD7H8iz8X2S6gMDDY88iKBVzmtFBC3sZu5k+grpa+feHw4R8YNuxabh5wJfff/yVJSacDdqdItXdoiY0FpWDtWrjqqgtfl0yx/ItPBaO1rhEiOYR8RCCKI9DN/ujRZSQ9+B3r/nicho3u5vXXPgmo8C4Yacj5IbU5kM8QSCeAjONXXpnCL794VzDSQyz/EkiQXxBsEywrISNI/9mn7/H55/Xpf1NfoqLK8OKL4T17OdJ7oTVoYEZkZ4dYlfkTUTBCnuJ8d8pq4uJeY/v2kdx9dwIA3bvnbN38YIWEipx0ArjySvj0U3vrB9LnTAhvRMEIISW3m0dWd8rw4fVITExg6dIoBg+Gd94xdTMXX3zuPcGwHgJdM5gNOXO6XrCsKm/rnj7djeXL3yU+vguLF2e/bqB9zoTwJoct6AQhcOwWYPrDW+Fdmzbwyy9w+eXQsKEZx5yeHqQPIgBGkaxevdrWuQULHiI6+hSnTlX0eV7W4lvptJC3EQtGCBnBLqgrXBieegr69IFBg+C992DatOyth4znCQkJAd/B54cRAU58hoYNG9q21jp2hPvv/9DnetI9OX8hFozgONkVQgazoC6jtiQtLY1duxbQt+/z1K69nmbNLKZMcabVjFP1Kzlpz5KdLBnKcfHixecdC0fq1oXff/d9jtTE5C/EghEcxZcPPdit1y3LOu/aW7a8S6tWibz++lhmz45i2rTzz88PVkioyU3c5vLLYfnybF8GpCYmvyEKRnAUX26wYG8ex44d83rtp55ayPLl7WncGCZMSOHmmx253FncUEpOK8dgfYbM69ata0Zj+0JqYvIXomAER/HnQ8/N5pE1A23UqFFnB4UtXryY4sWL06P7NRdcW+s1jBzZns6dYeBA+OQTmDwZKlTwf81Ir1/JSm4U2+WXw7p1xl3pa76b1MTkHyQGIziKPx96Tluve8tA279/N1am4Epqaipfz9+W7bUbN4aff4a4OFOXMXv2ufWdiIsIvilXDgoUgF273JZECBViweSAAgVgzRro2dNtScKPYLnBsnO9JSWNoEOHDiQkJGBZFpdeeonPaxcqBGPGQNeucOutMGsWPP+86dbsjbwQpwk3eXxRty788YcZoyzkf0TB5IAnnoAePUwL8kmTTHqsYAiWD91O+mpUVJTta19zDaxeDSNHwhVXwMSJ0K+fb9eNcD45UWwlS8KxY87LIoQn4iLLAXXrmmyYgwehZUvYKJNxziMYEwjtpq8Gcu1ixWDCBNPCZPRoY9V4m1uSkXa9b98+jh49GnBhqHCO6GjI+vXlZr6PEN6IgskhJUrAhx/CLbdA8+bwxRduS5S/8Tc7JjcxlGbNTGzmqqvMBM0pU851Acgc+7mp/yWUKwcNG9YjPj7eoU8WWcTEnN9hwanuDkJ4Ii6yXBAVBcOGmY2pXz+TofTkk+aPSHCWYKevFixo3GU33mi6ALz/vmk3s2nThbGfdu3nciwAP49TsZtwjQEFQlYLRsYl52/EgnGAli3NHfCPP0KnTrBnj9sS5U98ub+ccrPUqwdLl5quzC1awKRJhWjVsuJ5sZ9OHaty+vRpRz5TpJHVgpFxyfkbUTAOUaECzJ8PV19tZsd//73bEuUdcqscnHazxMQYy/Snn2D79nqMnzAWrS8CTOxn3pebWL9+fdi3ZvGHG7GP6OjzFYy0hsnfiIvMQWJizNz4Fi3guuvgr7+yT38VDE60Zw+Wm+Wyy+Cnn0pxzTXzqH/FYFq3/pyTJ19k69aDnDp1yud7nSrQDFahp1tt8Q8eNCOUM5DWMPkbUTBBoHNnKFsWjhwRBeMPJ5RDMDvwFigQww8/3MQ77yxl7Ng4Nm9eSOnSb9Ky5RekpHydq7XdxI3Yx9atJvty5sxzx6Q1TP5GFIzgKk4oh2A30QRISZlC9eqaazs15t0ZPVi27B2efz6dO+6IplAhb+ennH2em+B8sAo93WiLP2UKDBgAxYuff1xaw+RfJAYjuIoTPnh/Kcy5JfPd/sSJ5dmx/VuuuGIoH3ywj7g4mDoVUlMduVTICHXs49Qp8z3deWdQlhfClLCwYJRSRYE3gSbAGSBJa/25l/MSgHnAes+hU1rrq0Mlp+A8Tvjgg+1m8Xa337vXCYoUnUHz5sMYMQLGjYOxY6Fbt5x3AwhlGnKoYx8zZ0KDBlCnTlCWF8KUsFAwQBJwWGtdSylVG1iilKqltT7q5dzftdZNQyyfECScUg7BdLP4csE1bw7ffANffw1JSfDcczB+vCnezMApheGk4vH3vTut7F56Cf73P0eWEvIQ4aJg+gK3Amit/1RKrQCuAz52VSohJIS7D97O3X6nTtC+PUyfbvrUtWljMgovu8w9uf0RExPD008/DcCwYcNytZYvhfTzz7Bzp0l+yUzW8QsS3M9/hIuCqQZszvTzFqBqNufGKaVWAqnAK1prPyOMBCF32LWyYmJMF4B+/UyPs6uugsREuP9+k7ruzXWW3+fNrF1rvo+HHz6/w4VbadJCaAmJgvEohGrZvFwxgKVWAlW11oeUUpcC3yiltmutv8m1kILgg0CsrGLFTNuZ+++HN980/erKlYNevaBVK1OIW7BgCIT2YNfd5ZSy++sv+Oor4zZcuhReeMF8B3DOapk7dy7r1/9C8sIbKFy4gLSIyaeERMForRv7el0ptQWoDuz1HKoGLPKyzuFMzzcqpeYALQFRMELYUaIE3Hsv3HXXuQ33zjvNBty0KbRuDY88kkKLFiZ1N1RB/uwUyerVqwFo2LBhQOukpRXhu+9KANdSpMh20tMLU7r0T7zwwnVMn25qwsx556yW1q3LUbiwxdA7FjNtatuQpEkLoSdcXGQfA7cDKzxB/quA/llPUkpVBnZprS2lVBmgIzAipJIKghd8xRNiYuCGG8wD4NAh00poyRLTHHXlSjMCYteuu4iNXcMDD8ygbdtKXH99AjExMQErnqyyWJZ1drS0LzIUS0pKitdrWhbs3g2bNpnH5s23cPBgQ44cqQv8AHxNvXqPUqzYBqKioG/f685bP2tx50P/bUTnLp+TvGg7CfFVHK9dEtwnXBTMOGC6UuovIA0YorU+AqCUehLYobWeDNwI3KGUSsXI/pbWeq5bQgsCBB5PuOgi00roOs/+e/IkLFuWRt++mzl9+iref/9yXnjhYooX30NiYiU2bryNYsU2sm4d1Kp1fquV7GTZsGE1nTpewlNPTWXjxn/Ytm2PV3dXxrEvvkjhn39g1640Ro9eyV9/tcCyqjBkSDqbN0ezaRNs2WIGhtWoYR59+gyiVSto2xY6d37qvHW94S3du03ri3n11TWMH79WWsTkQ8JCwWitjwG9s3ltZKbnLwEvhUouQbBDbtuuFC4Mp08n07DhB541fuD4cejQUVOv3khWroS9exPp1s1YDiVLQpkyULq0+TfzY8+ev1i+/HL+M6gfqamxJCQUYMOGDcTEnGL9+uqkpxciPb0gnTrBP//A2rUfkpp6EeXKQfnyFseObSEq6hRVqnRg//6N/PrrdEaMuJWaNWOoVs3El3KKt3TvRSm7qVu3NV27dpUssnxIWCgYQcjLONF2JesaRYvC9dcd4cOPbmbLlt8AqFcvnkqVYjhzpjjTp89l/34ueKxde5yiRRuzYUMpihRJpXDhM1xetwDp6QcZObIrRYoYhVaypOmTd/fd9xEbe5Bvv/2Kb775hgkThp9VlKmpRlEWKlSVunVzHxfxlu59ySX1eP7550Wx5FNEwQgRT27rMZzoheZtja/nb6VgwaLnnRcdnUbBgoeIi/O+TsuW/zBhwqM899w5JdGu/VwuvrgoQ4c+dMH5P/74wdnnuVGUduJD0tgy8hAFI0Q0TtRjONF2xdsal13WiGnT3qZdu3aAvU3c2zoHDqRTpox/31YomoaGe1Gt4CyiYISIxom29U7cmTvZMien68hsFsFpRMEIEY1TbeuduDN36u4+p+uIC0twGmnXL0Q0eWFkb0pKSshaxmQop2HDhtGhQwdRLkKuEAtGiGjEXAH59wAACrZJREFULSQIwUMUjBDRiFtIEIKHKBgh4pHMJv9Ia30hJ4iCEQQXyEsbtrTWF3KKKBhBCDFObdihUlJOpHILkYlkkQlCiMm8YQ8f3pjPPu3Azp1/kJycbHuNDCU1YcJwTp6Yz4QJwxk0aCBpaWmOy+srlVsQfCEKRhBCjBMbthNKyi55IZVbCE9EwQhCiHFiww6lVZGYmEilSiaV+5lnVtK5y3xJ5RZsIQpGEEKMExt2KK2KjFTupKQxFCnaiaSkMRLgF2whQX5BCDFO1N6EukBUUrmFnCAKRhBcILcbthSICnkBUTCCkEcRq0IIdyQGIwiCIAQFUTCCIAhCUBAFIwiCIAQFUTBBolQpmDEDLMttSQRBENxBFEyQmDMHPv8cevSAgwfdlkYQBCH0iIIJEtWqwbffwiWXQNOm8MsvbkskCIIQWkTBBJFCheCll+DJJ6F9e5g+3W2JBEEQQocomBBw002QkgJjxsCQIXDypNsSCYIgBB9RMCGiXj1YvhwOHYKWLWHjRrclEgRBCC6RVskfA7Br1y7XBBg/3rjKrrkGxo2DhATXRBEEIYw5cwb27YNt29yW5Lw9M6BeRFFWBOXRKqVaAUvclkMQBCGP0lpr/Z3dkyPNglkOtAZ2As6P/hMEQcifxACVMXuobSLKghEEQRBChwT5BUEQhKAgCkYQBEEICqJgBEEQhKAgCkYQBEEICqJgBEEQhKAgCkYQBEEICqJgBEEQhKAQaYWWjqOUuhl4CLgcGKa1fimb8xKAecB6z6FTWuurQyLk+XLYktdz7mDgYSAK+BK4V2udHhJBz8lQFHgTaAKcAZK01p97OS8Bl75fpVQc8BZQFtgHDNRa/5nlnBhgEnAtYAFjtNZTQyFfVmzK+zhwJ7DDc2ip1vquUMqZSZbxwI1ADeAKrfUaL+eE0/drR97HCZ/vtyzwDlATOA38Cdyutd6b5Txbf4uZEQsm96wG+gHv2Tj3d611Q88j5MrFgy15lVKXAo8BLYDansfNQZfuQpKAw1rrWkAXYKpSqng257r1/U4GXtZaxwEvA1O8nDMAqIX5HlsAjyulaoRMwvOxIy/A25m+T1c2Pw9zgDbAZh/nhNP3a0deCJ/v1wKe1VorrfUVwN/AGC/nBfK3CIiCyTVa6zVa69+BkN7Z55QA5O0FzNFa7/VYLa8DfYMu4IX0xbMBeu6yVwDXuSCHV5RSFYDGwPueQ+8DjZVS5bOc2hd4XWud7rkznAP0Dp2khgDkDRu01t9prbf6OS0svl+wLW/YoLXer7VOyXToR6C6l1MD/lsUBRNa4pRSK5VSy5RSt7otjB+qcf4d2BagapjL4cb3WxXYrrVOA/D8u8OLjOHyfdqVF6CfUupXpdR8pVSLUAqZA8Ll+w2EsPt+lVLRwB3Ap15eDvg7lhiMH5RSKzFfrDcqZvyh2mAlUFVrfcjjfvpGKbVda/2NI4J6cFDekOBP3gCWCsn3G0FMBp7WWqcqpToAc5VSdbXW+9wWLJ8Qrt/vi8BRINvYbCCIgvGD1rqxQ+sczvR8o1JqDtAScHQDdEpezN1JZjO5GuC42e9PXqVUhhwZAcdqwCIv64Tk+/XCVqCKUipGa53mCTZfzIXfVcbnyOhGm/VuMFTYkldrvSvT8wVKqa1AfWBxSKW1T7h8v7YIx+/Xk5xQG+iSTTKPrb/FzIiLLEQopSorpaI8z8sAHTEB93BlFtBdKVXeYzYPBj5yQY6PgdsBlFK1gauAr7Ke5Nb3q7Xe47lOf8+h/sCqrBk4mM8xWCkV7Yl3dAdmBlu+rNiVVylVJdPzhpiMKB0iMXNCWHy/dgm371cpNRqTHdZda30qm9Ns/S1mRtr15xKlVH9gHFAak+J3DOiotf5dKfUksENrPVkpdTfGt5mKsRzf0lqPC1d5PefejklpBpgP3B1qF5tSqhgwHWiEmeHzkNZ6rue1sPh+lVJ1MGm/pYEDmLRfrZSaB4zUWq/wWAovYRQfwFit9WuhkC+H8r6F2XDSML8nj2mt57kk7ySgJ1AJ+AfYp7WuF8bfrx15w+n7rQeswaT4n/Ac3qi17qGUWg1cr7Xe4etvMTtEwQiCIAhBQVxkgiAIQlAQBSMIgiAEBVEwgiAIQlAQBSMIgiAEBVEwgiAIQlAQBSNEHEqpyUqpR0N8TUspVSub1wYopeaHUh5BCAWSpizkG5RSmzBV6Rdrrf/JdHwV0BC4VGu9ySXZLKC21vovB9eczLkO1wUxYxUyiuSWaK1z1BRUKTUU6KW1bu/jnAHA3ZjvdbHW+tqcXEvI30irGCG/sRFTnf4igFLqCqBobhf1dAmICvU8HF9orYcCQ+HsfJFaWutQjVTYB0zAFN01CdE1hTyGKBghv/EOMBCPggFuBd4GRmWcoJSaDmzTWo/w/NwNeAK4DNNn6S6t9VdKqRRgKZCAaXF/hVLqOKZRYStgP6Zi/HXPOjGYAW2DgAqYyujumVq3t1dKfQmUB2ZgOiNYSql/Af/RWrfyrGMB9wHDgJKYIU8P50S5KaVaA+MBBWwA7tFaL/W8Nhh4BDN4bK9H9r+A54ECSqmjwFGtdaWs62qtv/KsccFrgpCBKBghv/EjcItSqi5mg++HaXo5ytvJSqlmGAXUC1gIVAZKZDrlFszMC41xQS3EtNW4GKgDLFBK/a21TgYewFhP13uu3QA4nmmtzpj+TSWBn4HPyL6XUw+gKVAc07BTAwFNaPQM3JqDmeORjJn2OMcz0RJMy6AmWuu/lVIXAxdprdcppYbhx0UmCHYQBSPkRzKsmMXAOmC7j3MHAW9orRd4fs567nSt9VoApVRVjLK6QWt9ElitlJrquVYy8B9Mf6aMpoW/ZFlrjNb6IHBQKbUIE7/ITsGM1VrvB/YrpZ7HKK5ARwDfCszONLJgnlLqd0y/roykgvqesQY7ODe+VxAcQbLIhPzIO8BNwL8w1okvqmJGxGZH5jb2FwP7tdZHMh3bDGR0xvW31q5Mz49jrBM7193suXagVAduVkodzHhgrKKLtdYHMGOG7wV2KaU+zS7LTRByilgwQr5Da71ZKbUR46oa5Of0rUBNH69nTrPcAZRRSpXIpGSqcc7qyVhrTeBSX0BVYG2ma+TEutgKTNVa3+PtRa31F8AXSqmiwLPAq0AHzv/MgpBjRMEI+ZVBQGmt9TGllK/f8/9v745V4oqCOIx/ZFlICh/AQtJlKl/A17AKVhbphG0SsE+RbvcFTJEnECGCbGMqQUmRMgOJCAYhkhSKkCJbpJi7jSRiFg/K8v3qc8/uFneHM3cu/7fAOCLeU+FJi8BCZn6+vjAzTyPiAHgTES+BZ93nrHVLtoDXXRvqC7BMxRPPklL4KiIOqVPOABjOsMc74KALX/tAjTKvUIWrR7Xo9qnR5itgOkTwHViKiH5m/v7bxt1AQ5/6D3kUEY+BSWZOZviemlO2yDSXMvNrZn68xbojYB0YARfUc5unN1zynAqHOgO2qRyP6TOOIRXKNgYuqeL1ZMafsEMNAnwCdru9/ktmHgOr1ITcD6rVNqDu+x6wSbXtflLDBxvdpXvACXAeEd/+sf0LKjtkRJ16fnFHMbuaH75oKT0wLV7KlO6DJxhJUhMWGElSE7bIJElNeIKRJDVhgZEkNWGBkSQ1YYGRJDVhgZEkNWGBkSQ18Qe6HpHD1RgT0gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "initial_theta = np.zeros(n)\n",
    "op_result = minimize(fun=cost_function, x0=initial_theta, jac=gradient_function, args=(X, y, 0.0001, True), method='tnc')  # lambda is zero in the args tuple\n",
    "\n",
    "fig3, ax3 = plt.subplots()\n",
    "# plotting y=1 values\n",
    "ax3.scatter(x=X_old[y_slim == 1, 0], y=X_old[y_slim == 1, 1], marker='+', c='black', s=50, label='Accepted')\n",
    "# plotting y=0 values\n",
    "# X[y_slim == 0, 0] is logical indexing with rows with y=0 only\n",
    "ax3.scatter(x=X_old[y_slim == 0, 0], y=X_old[y_slim == 0, 1], marker='o', c='xkcd:light yellow', s=25, label='Regected', edgecolor='k')\n",
    "\n",
    "# labels\n",
    "ax3.set_xlabel('Microchip Test 1')\n",
    "ax3.set_ylabel('Microchip Test 2')\n",
    "\n",
    "# Specified in plot order\n",
    "ax3.legend()\n",
    "\n",
    "theta = op_result.x[:,np.newaxis]\n",
    "plot_decision_boundary(theta=theta, X=X, y=y, hypothesis=sigmoid, precision=0.1, fig=fig3, ax=ax3, feature_map=(map_feature, 6))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the changes in the decision boundary as you vary λ. With a small\n",
    "λ, you should find that the classifier gets almost every training example\n",
    "correct, but draws a very complicated boundary, thus overfitting the data. This is not a good decision boundary: for example, it predicts\n",
    "that a point at x = (−0.25, 1.5) is accepted (y = 1), which seems to be an incorrect decision given the training set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Too much regularization (Underfitting) ($\\lambda = 100$)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<Figure size 432x288 with 1 Axes>,\n",
       " <matplotlib.axes._subplots.AxesSubplot at 0x7ff2c5136080>)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEMCAYAAAD5zKAAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3hU5bX48W8SEyAE5A5yUwF58RAVUamXCiEICuKliFVqq+2xVD1a5demtbQcta0irVKUqkUrVq2opxYERVGQCNJa72CFygKVW7jJHcI1TOb3xzsThjDJzGT2nr1nZn2eZ54MOzt7VgbYa97benOCwSBKKaWU03K9DkAppVRm0gSjlFLKFZpglFJKuUITjFJKKVdoglFKKeWK47wOIJWMMY2Ac4CNQMDjcJRSKl3kAScAH4rIwXh/KKsSDDa5LPI6CKWUSlMXAv+I9+RsSzAbAaZNm0aHDh28jkUppdLCpk2buO666yB0D41XtiWYAECHDh3o3Lmz17EopVS6SWhoQQf5lVJKuUITjFJKKVdkWxeZUirNVFdXU1FRwd69e70OJePl5+fTrl07mjdv7sj1NMEopXxt69at5OTkYIwhN1c7XdwSDAbZv38/69evB3AkyejfllLK13bu3En79u01ubgsJyeHwsJCOnXqxNdff+3INfVvTCnla4FAgPz8fK/DyBpNmjShqqrKkWtpglFK+V5OTo7XIWQNJ99r34zBGGNaA38FugOHgJXATSKypdZ5hcBfgLOAw0CZiMxOcbhKqSy2a9cuLrzwQr797W8zbty4lL72559/zqpVqxg2bFjCP1tRUcFVV13F+++/70Jkx/JTCyYI/F5EjIicBnwJTIhyXhmwW0R6AJcBTxpjilIYp1IqTZSUlFBSUuL4dWfPns0ZZ5zBa6+9xqFDhxy/fn0+//xz3njjjZS+ZkP5pgUjItuBBRGH3gNuiXLqNcANoZ9ZaYz5CBgKvOR2jEopBTB9+nR+9rOf8fjjjzN//nyGDh3KoUOHmDRpEosWLSI3N5cuXbrw6KOPAvD4448ze/bsmoH0559/ntzcXF5++WWef/55AoEARUVF3HPPPXTr1o0ZM2bw6quv0qhRI9auXUubNm144IEHKCgoYPLkyVRWVnLFFVdwzjnnMG7cOD799FMefPDBmqnct99+e01inTZtGk8//TRFRUUMGDAgpe+TbxJMJGNMLja5vBLl212BNRF/Xgt0SUVcSim1fPlydu7cybnnnsuWLVuYPn06Q4cO5YknnmDdunXMmDGDgoICtm/fDsDLL79MeXk5L7zwAkVFRezYsYPc3Fw++ugj5syZw7Rp0ygoKGDhwoX88pe/5MUXXwTg448/ZubMmXTr1o1HHnmE++67j8mTJ3P77bezYMECJk+eDMDu3bu5++67eeKJJ2jXrh1ff/01I0eOZPbs2WzYsIE//elPzJw5kzZt2nDPPfek9L3yZYIB/ghUAo94HYhSKr1EdoktXLjwmGMLFixI6vp///vfueKKK8jJyWHIkCHce++9bN68mbfffptf/OIXFBQUANCqVSsA3n77bUaNGkVRke3Jb9myJQDl5eUsX76cq6++GrDrUHbv3l3zOmeddRbdunUD4Oqrr+ayyy6LGs/ixYupqKhg9OjRNcdycnJYs2YNixcvpqSkhDZt2gBwzTXXMGfOnKR+/0T4LsEYYx4ETgEuE5HqKKesBU4EwoP/XYG3UxSeUiqLHTp0iNmzZ1NQUMCsWbMAqKqqYsaMGQlfKxgMctVVV3HHHXckFVMwGMQYw7Rp04753uLFi5O6drL8NMiPMWY8dnbYlfVsavMScFPo/FOwe7ykx4iXUsp1CxYsqHkMGDCAAQMGHHUsGfPnz+fkk0/mnXfeoby8nPLycp566ilefvllBg4cyDPPPFMz6B/uIhs4cCAvvPAClZWVAOzYsQOA0tJSZs2axaZNmwC73mfp0qU1r/XJJ5+wevVqwI75nHvuuQAUFRWxZ8+emvPOPPNM1qxZw3vvvVdz7N///jfBYJB+/fqxcOFCtm3bBtjWVyr5pgVjjOkNjAVWAO8aYwBWici3jDFLgGEisgF4AHjaGPMFtnT0j0RkT13XVUopp0yfPv2YrqozzzyT6upq+vXrx549e7jyyivJz8/nxBNPZPLkyVx55ZVs3ryZa665huOOO47CwkKmTZvGOeecw5gxY7jlllsIBAJUVVVxySWXUFxcDEDfvn353e9+x5o1a2oG+QHOO+88nnrqKS6//HL69evHuHHjeOyxx3jggQcYP348VVVVdOnShSlTptCrVy9uvvnmmi66/v37p/T9ygkGgyl9QS8ZY04CVs2fP1/3g1EqTXz++eeceuqpDfrZ8NhLsi2XVJsxY8ZRA/mpVvs9r6ioYNCgQQAni8jqeK/jmxaMUko5Ld0SS6bRBKOUUj4zYsQIRowY4XUYSfPVIL9SSqnMoQlGKaWUKzTBKKWUcoUmGKWUUq7QBKOUUsoVmmCUUioBpaWlXHLJJVx++eUMHTqUl15KfSH3999/n3/84x8N/tlUzVDTacpKqYwUCAQoLy9n6dKlFBcXU1paSl5eniPXnjx5Mj179mTFihWMGDGC/v370759e0euHY8PPviAffv28c1vfjNlr9kQmmCUUhknEAhw443Xs2mTMKB/WyZOnMa0aYapU591LMkA9OzZk+bNm7N582bat2/PE088wdy5cwkEArRv357f/va3tG3blj179vDLX/6SlStX0r59e9q3b0/r1q258847a/aR+fDDDzl06BDGGO655x6aNm3Knj17GD9+PEuXLiUnJ4ezzz6ba665hhdffJHq6mreffddLr30Un70ox+xcOFC/vSnP3Ho0CHy8/MZO3Ysffr0AWDSpEm8/vrrNG/enH79+jn2+8eiCUYplXHKy8vZtEl49ZXB5OfnUlZWzfDL5lJeXs7gwYMde52PP/6Yli1b0qtXL2bNmsW6dev429/+Rm5uLs8//zwTJkxg4sSJPProozRv3pw33niDnTt3MmLECC6++GIAnnzySZo1a1ZTiPKBBx7giSee4P/9v//H+PHjKSwsZNasWeTm5rJ9+3ZatWrFtddey759+7jzzjsBWLt2LY899hhTp06lqKiIlStXMnr0aBYsWFBTlHPmzJk0btyYW2+91bHfPxZNMEqpjLN06VIG9G9Lfr4dZs7Pz6VkQDuWLVvmSIK5/fbbCQaDrF27locffpiCgoKa7rhvfetbADW7VIId9xg3bhwALVq04KKLLqq5Vnl5OZWVlbz55puA3RKgV69egN1LZsaMGeTm2t8jvMdMbYsWLWLt2rVcd911NccOHz7M1q1bef/99xk2bBhNmzYFYOTIkTz22GNJvwfx0ASjlMo4xcXFTJw4jbKyavLzc6mqqmbBwq8pK+vtyPXDYzBz5sxh7Nix9O3bl2AwyC233MLIkSMTulYwGOTuu+/mvPPOSyqmCy+8kN///vdJXcNpOotMKZVxSktL6dDBMPyyudx//ycMv2wuJ5zQi9LSUkdfZ+jQoVxwwQU8/vjjlJaW8vzzz7Nr1y7AtkSWL18OQL9+/Wo2KNu9ezfz588/Ktann36aAwcOAFBZWcmXX34J2L1kpk6dSrjqfXiPmdp7wlxwwQUsWrSIlStX1hz797//DcC5557LnDlz2LdvH4FAgOnTpzv6HtRHWzBKqYyTl5fH1KnPUl5ezrJlyygr6+3oLLJIP/3pTxkxYgSjR49m586dfPe73wVsy2TUqFH06tWLW2+9lbFjx3LJJZfQtm1biouLa7rPfvSjH/HII48wcuRIcnJyyMnJ4bbbbqN79+6MHTuW8ePHM3z4cPLy8mr2f7nooouYOXMmV1xxRc0g/wMPPMCvfvUrDhw4QFVVFX379uX0009n4MCBLFmyhCuuuKJmkH/z5s2Ovw/R6H4wSilfS2Y/GL+oqqqiurqaRo0aUVlZyahRoxg7diznn3++16FFpfvBKKVUmti9ezejR48mEAhw8OBBhg8f7tvk4iRNMEop5bLWrVszY8YMr8NIOR3kV0r5XjZ15XvNyfdaE4xSytfy8vKoqqryOoyssX//fvLz8x25liYYpZSvtWjRgs2bN1NdXe11KBktGAyyb98+1q9fT7t27Ry5po7BKKV8rU2bNlRUVCAiXoeS8fLz82nfvj3Nmzd35HqaYJRSvpabm0vXrl29DkM1gHaRKaWUcoUmGKWUUq7wVReZMeZB4CrgJOA0EVka5Zx7gP8BNoQO/VNEUld/WimlVFx8lWCAmcDDwKIY5z0rImUpiEcppVQD+SrBiMg/AIwxXoeilFIqSb5KMAm41hgzBNgE3C0i//I6IKWUUkdLx0H+KdiKnqcDDwCzjDGtPY5JKaVULWmXYERkk4hUhZ7PA9YBxd5GpZRSqra0SzDGmE4Rz/tgZ5zpEl+llPIZX43BGGMmAyOADsBbxphtItLbGPM6cJeIfASMN8acBQSAQ8D3RGSTd1GrsGAQDh6Eykr72Ls3+vN9++DAAXvugQPHPg4ehEOHoKrKPg4fjv61ujr2I5acHMjNPfqRl3fsn487rv5Hfj4UFNT/aNQIGje2j8jnkY8mTaCwEJo2tV8LC+3xnBz3//6UcpqvEoyI3A7cHuX4sIjnN6Q0qCwVDML27bBpE2zefOTr9u2wcyfs2BH9EQxCs2ZQVHTk0bTp0X9u0sQ+GjeG5s2hXbujb7zhR/jGXftr+HntRFD7kZMT+8ZcXW1jDiekQODYJHX4sD1++PCxj8gkGE6Khw4d+wgn3q1bj02m4ef799vHvn1HPw4ePJJ4CguPfn+jPW/WDI4/Hlq0iP7VoUK5SsXkqwSjUmfXLhCBL76AVatg9WrYuNEmkk2b4Ouv7c2qfXvo0ME+2rWDVq3glFOgZcsjjxYtjjxv0sTr3yzzVFcfSTyRLcE9e6J/Xb3a/v3u2mU/DER+3bXLtqZatoQ2baB16yOPyD+3aQMnnACdO9vn2oJSDaEJJsPt2AEffQRLl9qEsny5/bpnDxgDPXrAySfDWWdBx45Hkkn79rYVobyXm2tbgU2bQtu2yV0rGLRJaMcO2Lbt2MeaNfDJJ7altWEDVFTYpNa585FHly5HnnfrZh9Nmzrzu6rMogkmg+zfD0uWwAcfwIcf2q8bN0LfvnD66XDaaTByJPTqBZ066afSbJSTY7vQmjWDeAsU79tnE034sW6d/cAyZw589ZVtAbdsaT+sdO9uH+Hnp5xiW7gqO2mCSWNbt8LcufDOOzahLF9uk0e/fjBoEPziF3DqqXasQqmGKiyEnj3tI5rqali/Hr780na5fvklTJ9uv65cacfZTjvNPoqL7ddTT9Xu1GygCSaNVFfb7q45c+zj88+hpARKS+H734c+fexAuVKplJtru826dLH/HiMFg7bbbelS+OwzePNNePBBm4i6drXJpk8fOPdc+MY3bMtKZQ5NMD63ZYv9T/nGG/Zr+/YwdCjcdx9885s6TqL8LScHTjrJPoYPP3L80CFYscImncWL4de/tl+7d4fzz4fzzrNfu3fXrtx0pgnGh/bvh5kz4amnbNfXwIE2qYwfH3+/uVJ+VlBgu8uKi2HUKHvs0CE7hvivf8Frr8G4cXb69nnnQf/+MGyY7QLWhJM+NMH4yMqV8NBD8OKLcPbZ8MMfwquvareXyg4FBXb8sF8/uOMOe6yiAt59F8rL4eKL7fqnYcPsZJX+/W33nPKvmH89xpgzjTG3GGP6R/neGHfCyi5Ll8J3vmO7BFq3tl0Fb74J11yjyUVlt86d4dvfhilT7FjOK6/YGZB33AEnngh33mm72ZQ/1ZtgjDHXAm8DVwLTjTHTjTGFEaf8xs3gMt0nn8CIEXDRRXDGGXbK529+o91gSkWTk2O71MaOhU8/tRNdcnPh0kvtNPxHH7VTqpV/xGrBjAMuFZGLgZOBKmCeMSY810N7Qxtg1y646SY76Nm/v00sd96pM2iUSkRxMdx/v61cMHkyzJ9vJxPcc4+dHKO8FyvBdBGRfwKISKWIXAt8AiwI7cESdDvATDNzJvTubT+Nff45jBlj1xkopRomN9dOj54xAxYtsouLjYFbb7VrcZR3YiWYLcaY7pEHROTHwFvAIkDL5sVp0ya4+mrbUnn+edunfPzxXkelVGYxBh5/3H54a9nSrq359rftbEyVerESzGvAMdWLReRO4CVAV2HEEAza6cann25XQn/6qe0WU0q5p317uPdeW8bm/PPtrLOBA+G997yOLLvUO01ZRO6o53t3G2Pucz6kzLF1q53jv2MHzJtnB/KVUqnTrJnthr71Vpg2Da66yq4pe+ghWy1cuSupWeQicsipQDLN8uW2/MXZZ9tPTZpclPJOfr4tp7R8ud3b55xz7PIA5S5dpuSCZcvsoOO4cXaWy3G6nFUpX2jWDP7yFzsWOnCgfa7cownGYatXwyWXwB/+YD8xKaX85/vfhwULbOHN73/f7nmjnKcJxkGbN8OQIfDzn9uV+SqzBQIB5s2bx6RJk5g3bx6BQMDrkFQCeve2eyaB7TJbtszbeDJRXAnGGLOhjuNrnQ0nfe3aZQcPv/Md+PGPvY5GuS0QCHDjjdczceJYDuyfy8SJY7nxxutTmmQ0wSWvaVN4+mn7obCkxD5Xzom3BXPMGnNjzHGAzsPAVj++/HI7HfLuu72ORqVCeXk5mzYJr74ymLFj+/LqK4PZuHE55eXlKXl9PyS4TBLuMhs/3m4dENQl5I6od/jZGDMPu1q/kTFmbq1vdwHedyuwdPLDH9r97CdP1lLi2WLp0qUM6N+W/Hz7GS0/P5eSAe1YtmwZgwcPdv31IxNcfn4uZWXVDL9sLuXl5Sl5/UzUu7etBDBkiP1/fNddXkeU/mK1YP4OTAcCoa/hx9+BX2CLYGa1l16Cjz+2iym1dHj2KC4uZuE7W6iqqgagqqqaBQu/pnfv3il5/foSnGq49u1tJfM//xleftnraNJfrIWWjwMYY94XkSWpCSl9bN5sx1tmzdL9xbNNaWkp06YZhl82l5IB7Viw8GtOOKEXpaWlKXn94uJiJk6cRllZNfn5uTUJrqwsNQkuk3XoYOuaDRtmq2+k6DNDRop3hUZ3Y8x+EZFQbbLHgGrgNhHJynJywSDcfDP893/bekcqu+Tl5TF16rOUl5ezbNkyysp6U1paSl5eXkpe3+sEl+nOOccuNbjiCjvTrFUrryNKT/EmmN8B3ww9nwisACqBKYBjHb7GmAeBq4CTgNNE5Ji1tsaYPGAycAl2fGiCiDzpVAzxeu45W6n1xRdT/crZKxAIUF5eztKlSykuLk7pDT2avLw8Bg8eHHXMw+1YvU5w2eB737NbOF97Lbz+ui6Yboh437J2IrLJGNMIGACcgN0bxuldF2YCD2MrNdflOqAHcArQGlhsjHlLRFY7HEud1q+Hn/7U9tU20nKfKRGeNbVpkzCgf1smTpzGtGmGqVOfTflNNVbySFWs9SU45Yzf/c52lf3iF3ZRpkpMvMPS24wxJwFDgI9F5ABQkMDPx0VE/iEi62Kcdg3wZxGpFpEt2KR0tZNxxHLrrXDbbXDmmal81ezm1rTgkpISSkpK4j4/nunBXk9hVs457jjbSzFzpu21UImJN0GMBxYDz2C7yAAGAv92I6gYugJrIv68FjtlOiU+/thudXznnal6RQX+mTUVT/LwS6zKGa1a2Yk8Y8bY3WdV/OJKMCLyZ+yWyd1EZE7o8BJsd1VWmTLFDu5r11hqeT0tOCye5OGXWJVzeveGsjI7a1QXYcYvkWGrIDDEGNNRRB4GDmNnkqXaWuBEILxHXe0WjWt27YK//93ulqdSy8lZU5FdYgsXLjzm2IIFC+r82XimBzsRq98mNCj4yU/gr3+162NGjPA6mvQQV4IxxpyPHev4D3AWdiC+GLid1C+2fAkYbYyZgR3kvxK4MBUv/NxzMHiwnSevUiuRWVNu3pzjSR51xQowb968mHH5aUKDOqKgAB57DG64AYYPt39W9csJxtHeM8Z8BIwTkTeMMTtEpKUxpgmwSkQcu90aYyYDI4AOwFZgm4j0Nsa8DtwlIh+Fpik/gp1wAPA7EXkizuufBKyaP38+nTt3Tii2YNBue/zww6BLDVIrkYRR++a88J0tdOhQ98053HKpr9VSVzzLli2jd+/4pgcnEte8efOYOHFsTRmYqipbBqasbILOGPOBoUPt+pibb/Y6ktSpqKhg0KBBACcnMmM37oWWIvJG6Hk4Ix3EziRzjIjcjm0V1T4+LOJ5ALjFydeNx7/+BQcP2k2KVOok+mk+FTW6GjI9OJG4vK5zFo9s7sL79a9h5Ej4wQ90LDaWeGeRiTGm9q21BMiaaTFTpsBNN2kxy1RLdMqvX2dwJRKX3ycJZHsl53794LTT4MmUL+9OP/EmmJ8BLxljHgcaG2MeBp4Dfu5aZD5y4ICdpnj99V5Hkn0STRiJ3pwXLFiQUPdYQyUSV2lpKR062HGe++//hOGXzfVVGRhd5wO/+hU89JDOKIsl3mnKi7CD++uBF4AdwAUi8i8XY/ONhQuhuBjatvU6kuyTaMLw6805kbjCkwTKyibQpPBiysom+GqA36+txFQ67zy7CPOf//Q6En+rd5DfGPOaiFyawnhc1dBB/ttvtzPHfvlL10JTdQh3x2zcuPyoWVv13XAbMgjvhFgTBryKy+nxEp2EYP3+9yACU6d6HYn7GjrIHyvB7BaR5smH5w8NSTDBIPToYct3n3GGq+GpOnh1Yw6Ld6ZZQ2akuS3RWXWJXDORpJ+JNm6E//ovqKiwWy9nMrdnkWWtFSvs7LHTT/c6kuzldFFHPyYCt7gxq04rOVsnnAAXXGAXX99wg9fR+FOsBNPYGPNsfSeISEYPfb/2mq2mqrPHVDTJVAVIdH1PQ7q53JryrJWcrR/8AP74R00wdYmVYIJAVm4oFvb667b+kMouySSOeCSyvieZlf2686W7LrvMLrj86ivo1s3raPwnVoI5KCK/TkkkPlRVBe+9B9Onex2JSpZbCSPy5xLpekuk6yqZbq5E6qJl8+LJhioosHXJZs60tcrU0WIlmKzuGFq2DLp2heOP9zoSlWoNTRzxSqTrKplurnjHS7T+WcMNHQqPPqoJJppYCSart9j58EO7alelP7cTRqIS6bpKtpsrnvGSeFtJ2so51qBBdnvlykooKvI6Gn+pN8GISMprfvnJBx9ogvFKQ25kTtz8aiefkpISlixZQp8+fWL+bDwJK3z9+fPnx911FW83VzKJM55WkrZyomvWDM45B95+247JqCMc3fI402iC8UZDal25WR+rT58+UW/aiW63HCmR1fqpWNkfT8UELRFTt6FD7YQgdTRdB1OHvXvhiy90/YsTEm1ZNGRQO5Gf8cv6l0Sm+rq9FiieVlI6VHn2ytChdo+YYFCXNETSBFOHxYvtNqm6qVByGtKt0pAbWTI3v2gzzFq0aAHArl27jjkn0QTl5pRnp64dz2QAp6c8Z9J4Tu/eUF1tS8f06uV1NP4Rd4Ixxvw3MAroCGwAXgSeEpGMrCe6bJktya2S05DWSKwbWbSxhlSt91iyZEnN67uxPsZLsVpJTm5bnWnjOTk50L+/LX6pCeaIeLdM/j1wBfAQsAboCpQBhgwt2S8CxngdRfprSMuiITeyZG5+9c0wC7dkIgf9E+XmDLZErp1sayfZraAjpWJjuFQ791y7bu7GG72OxD/ibcF8H+grIhXhA8aY14BPyOAEM2CA11Gkv4a0LBpS68rJ+liRrZT6usj8MN05spupsrKSpi5XXazdymloSyQTx3POPRcef9zrKPwl3gSzJ/SofWy3s+H4h7ZgnNHQKba1b2TxfPrOtvpYtW/uhYX72b69kkAgEPXm7kZLqqEtkUwsYXP66bZkzJ49duqyij/BPATMMMZMACqALthdLicZY2oq8IjIV86HmHoHD9oS3FpbKHnpVnm39g3XzS6t2hJ9rWNv7n0YdNEs17qZog3KN7Ql4uR4jl8UFECfPnaBdhr/Go6KN8E8HPo6sNbxQcDk0PMg4M+7RoK+/NKWiNEZZM5womXht5X4fnj9aDf3i4d0caWbqa6usFGjrmfSpMRbIun2wSNe4XEYTTBWXAlGRLJqQeaKFdCzp9dRZD63KxY7Ldoq/8g/p1q0bqY3567jf/83djdTojHX1RUG1GwFnWhLJBO7NL/xDZg2zeso/EPXwUSxbh2ceKLXUSg/cDt51JVklyxZAsDOnTvr/NnS0lJ+8pNKBl00i4uHdOH1OatZu3Yn9957L/fddx/gXPx1dYV9/vnnGdkSaajTTrNLHJRVZ4IxxrwhIpeEni/CdoEdQ0T6uxSbZyoqIM4dlVU9Yi2ka2i3l99aNqkSbSJEq1bt2bt3L//3t42sWLGGgwcP0r37qY6/dn2D8pnYEmmoHj1g/XrYtw8KC72Oxnv1tWAid7J80u1A/KSiAoqLvY4ivXm9kM6p7iu3V/nXlWTjXW8Tjqn2zzstEwfl3ZCfb5PM8uXQt6/X0XivzgQjIs9HPH8mFcEYY3oCzwCtgW3A9SKystY59wD/g60mAPBPEbnVyTi0BZO8TFxI57Zwt1hJSUnUMalUi0xYyQ7KZ1JZmFh697bdZJpgEisVMwToAxy144GI3OVgPFOAR0XkOWPMd4HHgWgfkZ4VkTIHX/commCSl+j01fnz51NeXs6kSZNScgOK99N+fd14qRjkDycdcKbFlIyGdoV53ZpNtXCCUfGXinkE+DbwNrDPjUCMMe2AvkD4X+8LwCPGmLYissWN14wmGIQNG6BTp1S9YmZKZCFdPDegeG7m6TYrrbbIAf1oLZfI7rBo/Pr7ZVtr9tRT4bms3qrxiHhbMN8BzhCRdS7G0gVYLyIBABEJGGM2hI7XTjDXhlpUm4C7ReRfTgWxfTs0bqwDdMlKpM8+225AiUj1+h8nk3T456644oqMKwtTn+7d7Yp+FX+C2QrUPV8ytaYA94lIlTFmMDDLGHOqiGxz4uJbt0Lbtk5cKbsl0mfvVF2qWDfjZG+etb/v1xaD32RiWZj6dOliu9lV/dOUIwulTASmGWPuBzZHnudgeZh1QCdjTAGDceYAABzMSURBVF6o9ZKH3RrgqFaTiGyKeD7PGLMOKAbq7z+I09at0KaNE1dS8fbZ13UD2rbtOWbOnAmkZ5dXMrz63dxoMWXbDLTWre00ZZ2qXH8L5gvs2pfI/dmG1zrHsfIwIvK1MWYJds+Z50JfF9cefzHGdBKR9aHnfYCTAHEiBoBt2zTBpFpdN6ADB5z9GOhVd1Mq6pj5RbRW4qBBgwgGg+zdmxPa8jmzF2Pm5NhJQuvXwymneB2Nt+qbpuxFeZibgWeMMXcBO4DrAYwxrwN3ichHwHhjzFlAADgEfC+yVZOsrVvtJxCVOvF0pyV6s06Hm3E2ycnJoaioiDFjxngdSkp07my7yTTBxMEY0wnYJyI7Io61BJqIyIa6fzIxIrIc+EaU48Mint/g1OtFownGG7oa3H8akqT9WJTUC+EEk+3iHeSfCfw3tlUR1hm7wv+YhJDOtm/XBJMNnCybH+1nIfvGjdQRnTppgoH4E0xPEfks8oCIfGaMybjdp3ftsrNAlL/ojTk7pWsFgI4d4YsvvI7Ce/EmmC3GmB4iUvOWGWN6YMu5ZJS9e3U3ulTKtG4U7SI6ItnfO50rALRrB/9ybHVe+oo3wTwFTDfG/Ar4CugO/JYMLIJZWQlFRbHPU/6XyA1eu7b8J50X4LZtC19/7XUU3os3wUwAqoAHsSvr12GTyx9cisszlZXQtKnXUfhbunZbqPTi1AJcL7RrpwkG4t/Rshp4IPTIaHv3agumPk50W/ixteBG15a2epKTzhUA2raFLSmroOhfiVRTLsGuS+kErAf+KiJvuxSXZ7SLrH6xui28bt34MXllAi/Gk9K5AkCbNnbRdnU15GbVhvNHi3cdzA+B8dhusfeBrsALxpj/FZE/uxhfymkXWf3q67YoLS2Nq3WjA+EKYne1JrsHjZfy86F5c5tksrm2YbwtmJ8Dg0Xk0/ABY8z/AdOBjEow+/dr/aD61Ndt4YdBWSeSlyY798Xb1ZrOC3Bbt7br6jTBxNYa+E+tYwK0cjYc7x04YMv1q+jq67aYPHly2g7KqmO52d3ohw8jbmvZEnbsiH1eJou3d/AfwB+MMYUAxpim2AH/d90KzCuaYOoX7rYoK5sQKlw4oeZTZ3FxMQvf2UJVVTVATeumd++6B2UXLFgQ80ZVUlJS79bBsb6v/Ke+rtZMoQkm/hbMzcCLwC5jzHZsy+VdbMXjjBEMwsGD0KiR15H4W13dFn4blNWuruS4OVaWzjPE4tWiBez0yy5aHomZYIwxOUATYBDQAbtHywYRybhKO4cP2xkfaTCG6EvpPCirUstvH0bcoC2YOBKMiASNMZ8BzUJJJeMSS5h2jyXPqUHZWP3/kZweH9CZbe7Lhg8jLVtqCybeLrLFQE9guYuxeO7AAe0eUyqa2snWiSSczjPE4tGihd3+I5vFm2AWAG8YY57GlokJhr8hIk85H5Y3qqqgoMDrKBQk1v+vLQ7lR82bw1dObSifpuJNMBcAq4ABtY4HsYUwM0JVFRwXd20DlUm0AoByWrNmduF2Nou3FtlAtwPxg8OH7QpcpdSxNAknplkz2LPH6yi8FW+pmCHAahFZEXGsJ3CiiMxzK7hU0xaMP8W6cTlxY9PyNcpp2oKJv4vsUaB/rWOVoeM9HY3IQ9qCUapu6ZCEvS62GqmoSFsw8SaYdiKysdaxjdh1MRlDWzBK+Ves5OG3HTC1iyz+UjFfGWNqr4AqwQ78Z4zDhzXBqPjK16jUCiePiRPHcmD/XCZOHMuNN15PIBCoOSeyvtnYsX159ZXBbNy4nPLyck9i1gQTfwvmHmCGMWYq8CV2y+QfhB4Zo7paV/GnO7923WSaVL+/8RTH9NsOmIWFtjp7NourBSMis4AhQFPg0tDXi0PHM0a2bw7klkAgwLx585g0aRLz5s076lOnF7Q4ZvqJpzhmQ4qtuqlJE00wcXcIicgHwAcuxuI5TTDOS6RfXFsfqZcu73k8xTH9Vt+scWNbPDeb7yt1JhhjzK9E5L7Q89/UdZ6I3OVUMKGpz89g95/ZBlwvIitrnZMHTAYuwS70nCAiTzrx+tn8D8Etqdj3w8n1Gelyw80mgUCA6upq9u1rRP8BrzD80i68s2jLMcnDb/XNcnJskjlwIHs3MayvBdM54nkXtwMJmQI8KiLPGWO+CzwO1P74cR3QAzgFm4gWG2PeEpHVyb64Jhjn+aVfXBcJpqfIFvAll7Tl7fJ9zC/fy513jueiiy46Jnn4rb5Zkyawb58mmGOIyC0Rz10fzDfGtAP6AuF/GS8Ajxhj2orIlohTrwH+LCLVwBZjzEzgauwGaEnRBOO8WF0bTtz402F9hp848Z6nar1J7Rbwz0It4Nzc3LSovJzt4zD1jsEYY7rGuoCIrHUoli7AehEJhK4bMMZsCB2PTDBdgTURf16LQy2sYNA2a5Vz/NIvXl8Sihz019ZNbKlcb+KXFnBDaYKp32qOVE6OdusNAv7/GKE8E6tf3IvWRzAYZO/evUyaNIni4mKCwSA5WfTJItn3PBXjamHpvvNlo0Zw6JDXUXgnVoL5FLub5TPAc8AGF2NZB3QyxuSFWi952N0z19U6by1wIvBh6M+1WzRJCQZjn6MSU1+/eGRXS2VlJU2bNk3qtWLdKAOBANu3b6Zly7zQgr1pnHzykU/f2sUWWypbFX5pATdUQYGdSZat6h1xEJEzgZFAK+CfwOvAtUCBiATC3VlOEJGvgSXAqNChUcDiWuMvAC8Bo40xucaYtsCVwN+diCGLPsS6Lp61L7VXZxcW7mf79s2urpMpLy+nU6ci5r91uS9We6cjN9ab1PXvJdwCLiubQJPCiykrm+BZ6ZeGyPYWTMwhbRFZKiI/A04C/gAMBzYaY/q6EM/NwI+NMSuAH4f+jDHmdWPM2aFz/gp8BawE3gN+IyKOlazRFkzy4inrAceW9pj/1uV07NjU1Zt9PAv2sklDyuKUlpbSoYNtVdx//ycMv2xuUq2KWP9ewi3gMWPGMHjw4LRJLqAtmEQqb52C3XDsPOwWyjucDkZElgPfiHJ8WMTzAHBL7XOcoC0YZ8TbR+/FAG6sPn3tGovN6fUmqRzTSTVtwdTDGNPKGHOrMeYDYCa2RH9/ERnoZKtBZZZ4WwkN7WpJpvSM05++s1W4VTFz5kzuu+++pFoVmdyqzPYWTKwusg3Abdjkciu2S6qHMaY0/HA7wFTKybFrYVRy4k0cDbnZx9v9Vhc/9OlrLbSj+a2GmJMKCrK7BROri2wT0BgYHXrUFgS6OR2UV/LyNME4Id6ZPw3panGiO8Vvq72zXbrPFKtPXh54XNvVU/UmGBE5KUVx+EK2/2NwSiKJI9GbfbLjNm5MQ86Eqc2J/A6JVAKI57p+qyHmpOOOy+57im6vFSE3N7v/MTjJrVZCui6801po9cvUVuVxx9mNDLOVJpgI2kXmf251p2RCKyRV3K6+kKo6Z6mQl6cJRoVoF5n7kr15NKQ7xY3WQ6LXdPOm3NDrudWqSua6qaxzlgraRaZqaIJxl1M3j0ztTkk3JSUlLFmyhD59+jh2zUxbE3PccVBV5XUU3tEEEyHb+0vd5tXNo67WQ/h5SUlJwp+0M2GLACd+hz59+jjaWkv36sm15eZmd3UQTTARsv3ThlPq6gZz8+YRvpHNnz/ftf57pxKJE4koUycNpOskjvpoglEA5OdrCyZZ9XWDuX3zCAaDCXXBZUIrJNXcTmyZtiYmJ0cTjArJz9cWTLLq6wZz++axd+/emF1wbiQQL5KS08nRrd8h0etm2poYTTCqhiaY5MXqBkvm5lG76+3ee++t2Shs4cKFFBUV8a0rz3e0Cy5Tu6IaKhWtvkyaxJHtCUZ3oI+gYzDJi1VXqqGl16PVINu+fTPBiP+9VVVVvDm3okE1rRpStl6pWLI9wWgLJoK2YJLnVjdYXV1vZWXjGDx4MCUlJQSDQU4+ubOjr50O4zR+i0cdoQlG1WjUKLtLazvBrT70eGag5eTkZFT/vd9pYostGLRTlbOVJpgIBQW2BVNdnd3/KJLlRh96vDPQ3Hjt8NjPtm3baNSoEYFAQJOWgzKpNExt2X4v0QQTISfnyP4NjRt7HY2KFKvrza1P05HTrr8zqjML39lCnz69adWqfc2gv2q4TCsNU5smGHWUcDeZJhh/8Wr6arSxn0EXzWLv3r1xX8OpsRu/jgElI9NKw9SmCUYdRcdh/Ku+7i+3ulmijf1cPKQL//e3jUlfW2VeaZjasj3BZPGvHl3jxnDggNdRZJdAIMC8efOYNGkS8+bNi3v748ifT2Yb5fpEm3b9+pzVrFixombr43Td/jjZ990JmbxdMtgEE1qqlZW0BVOLtmBSy4k+eDe7WcJjP4MumsXFQ7rw5tx1rF27k4Mx/pE4tUDTrYWefhn7yLTSMLVlewtGE0wthYWwf7/XUWQPJ5KDm90s4bGft956i9mzZ3P66aewd++/6NatV1qPhfhl7CPTSsPUVlVl19dlK00wtTRpAvv2eR1F9nAiOaSiAu8LLzxb82m/adMDbN++t97pyk4t0HRroaefxj4yqTRMbdmeYLK48RadtmBSy4k++NLSUjp0sN0s99//CcMvm+toN0vkp/2xY/sy/63Ladkyl/Lyckeu74VMH/vwi2xPML5owRhjCoG/AGcBh4EyEZkd5bwS4HVgRejQQRH5hpOxFBZqCyaVnOiDd7ubJdqn/aGXnOj4p/1UTkPO9LEPv9AE4w9lwG4R6WGMOQVYZIzpISKVUc79j4ic7VYgTZpoCyaVnEoObnazJNsF51TCcDLxxHrfM3HNjReqquzi7WzllwRzDXADgIisNMZ8BAwFXkp1INqCST2/98Fn6qf9vLw87rvvPgDGjBmT1LUakpAyuURMmLZg/KErsCbiz2uBLnWc29MY8wlQBTwmIs84GUhhISSwSFtlATe74LJ1vxm/TJN226FDmmBcF0oIXev4dvsELvUJ0EVEdhljTgbeMsasF5G3kg4ypFkz2LPHqaupTOH3VlZ94m1dpCLZhVsts2bNYsWKTymffymNGx+XcSViwvbvt93u2SolCUZE+tb3fWPMWuBEYEvoUFfg7SjX2R3xfJUxZiZwAeBogtm9O/Z5SjnBi/1m6kokS5YsAaBPnz5JXSes9u8R2Wq58MI2NG4c5OZbFjL1yYEZVyImTBOMP7wE3AR8FBrkPwcYVfskY8wJwCYRCRpjWgFDgHFOBnL88fDll05eUWUDJ8YTgsEge/fuZdKkSUddI9HEUzuWYDBYs7V0fcKJZcGCBa4ku9qLO3/+szMZftlsyt9eT8mATo6vXfIDTTD+8ADwtDHmCyAA/EhE9gAYY34DbBCRKcBVwC3GmCps7M+IyCwnAzn+eNi1y8krqkznxHhCIBBg+/bNtGyZF6qnduQaDYnlq6+WcPGQzvz2t0+yatVWKiq+jtq6qJ1IwvXJ4tn7JtHWV7Tp3v0v7Mif/rSUBx9clhETJ2rTBOMDIrIXuLqO790V8fwR4BE3Y9EEoxLlRNmV8vJyOnUqinqNhsQy/63LbSvh530pGTiDLVti/6OOTJTXXtORN+dWcOON1zs28B5tuvfbCzZz6qkXcvnll2fkLDJNMOoommBUopwouxL9030bfvrTn/LZZ58B8Q24R7vOsKEnsWdPo5jdXcd0Yf28r6MD79Gme3fu3JuHHnoo4xJL2P792b23lJaKqaVlS9i+3esoVColW7beibIr0a7x5tx1FCS4Si/R6yxYsKAm8dSXKGOJvE5dwtO9y8om0KTwYsrKJmTctORIgYBdB5PNCUZbMLW0aQPbtnkdhUoVJ8ZPnFiIGe0a3bqdydSpzzJo0CAgvgH3aNfZsaOaVq2axvzZVBQNTefp3onasweKirRcv4rQujVs3QrBYHZvFJQtnBg/cWIhppMlcxp6nUytWOCV3buheXOvo/CWJphaCgshL8+u5i8q8joa5TanytY78cncqU/3Db1Opu/Nkmp79miCyeLGW93atLGtGJX50qFsfTzjG04JJ6cxY8YwePBgTS5J0BaMtmCiCieYk07yOhLlNu0WUm7ZvdtWBslmmmCiaNMGtmyJfZ5Kf9otpNyyZ48mGE0wUXToAJs3ex2FSpVsmtnUUNlQWt9p27dDq1ZeR+EtTTBRdOwIGzZ4HYXKZOl0w86W0vpO27rV9oZkMx3kj6JjR1i/3usoVKYK37AnThwbqjs2lhtvvD7hBZ7JLhCNV+RU7rFj+/LqK4PZuHF5wmVsso0mGE0wUWkLRrnJiRu2U0kqHsms8M9mmmA0wUTVqZMmGOUeJ27YqWxVpMNUbj/SBKMJJiptwSg3OXHDTmWrorS0lA4d7FTu++//hOGXzdWp3HHQBKMJJqrwLDKXurRVlnPihp3KVkW2Fal0ypYtmmBygsGg1zGkjDHmJGDV/Pnz6dy5c73ndukCixbpYkvljvAssmXLltG7d+Jrb8JjMBs3Lj9qgaje+P2hutqWndq+3X5NdxUVFeGiqyeLyOp4f06nKdehe3e7dbImGOWGZNfe6AJRf9u61dYyzITkkgxNMHUIJ5hQpXSlfEcXiPrXunW2FyTb6RhMHcIJRimlErVuHcTohc8KmmDqoAlGKdVQFRXaggFNMHXSBKOUaijtIrM0wdShRw/44gs7G0QppRKxdq0mGNAEU6cWLaBlS1i1yutIlFLpZvlyMMbrKLynCaYeZ5wBn37qdRRKqXRSXQ0rVkCvXl5H4j1NMPXo0weWLPE6CqVUOlm71vZ+ZPtmY6AJpl5nnKEJRimVmOXL4dRTvY7CHzTB1KNPH+0iU0ol5rPPQAtNW9m2kj8PYNOmTXGd3KgR7N4Ny5bB8ce7GpdSKkO8/z5ceKFdC5MpIu6ZCdUiyrZil98EFnkdh1JKpakLReQf8Z6cbS2YD4ELgY2AFuNXSqn45AEnYO+hccuqFoxSSqnU0UF+pZRSrtAEo5RSyhWaYJRSSrlCE4xSSilXaIJRSinlCk0wSimlXKEJRimllCuybaGl44wx3wV+DvwXMEZEHqnjvBLgdWBF6NBBEflGSoI8Oo644g2dOxq4E8gB5gC3i0hKt2AzxhQCfwHOAg4DZSIyO8p5JXj0/hpjegLPAK2BbcD1IrKy1jl5wGTgEiAITBCRJ1MRX21xxnsP8D/AhtChf4rIramMMyKWB4GrgJOA00RkaZRz/PT+xhPvPfjn/W0N/BXoDhwCVgI3iciWWufF9X8xkrZgkrcEuBZ4Po5z/yMifUKPlCeXkLjiNcacDNwNnAecEnp81/XojlUG7BaRHsBlwJPGmKI6zvXq/Z0CPCoiPYFHgcejnHMd0AP7Pp4H3GOMOSllER4tnngBno14Pz25+YXMBPoDa+o5x0/vbzzxgn/e3yDwexExInIa8CUwIcp5ifxfBDTBJE1ElorIf4C02Fw5gXhHAjNFZEuo1fJn4BrXAzzWNYRugKFP2R8BQz2IIypjTDugL/BC6NALQF9jTNtap14D/FlEqkOfDGcCV6cuUiuBeH1DRP4hIutinOaL9xfijtc3RGS7iCyIOPQecGKUUxP+v6gJJrV6GmM+Mca8b4y5wetgYujK0Z/A1gJe7DKeSBxevL9dgPUiEgAIfd0QJUa/vJ/xxgtwrTHm38aYucaY81IZZAP45f1NhO/eX2NMLnAL8EqUbyf8HusYTAzGmE+wb2w07cP/UePwCdBFRHaFup/eMsasF5G3HAk0xMF4UyJWvAlcKiXvbxaZAtwnIlXGmMHALGPMqSKyzevAMoRf398/ApVAnWOzidAEE4OI9HXoOrsjnq8yxswELgAcvQE6FS/200lkM7kr4HizP1a8xphwHOEBx67A21Guk5L3N4p1QCdjTJ6IBEKDzR059r0K/x7harS1Pw2mSlzxisimiOfzjDHrgGJgYUqjjZ9f3t+4+PH9DU1OOAW4rI7JPHH9X4ykXWQpYow5wRiTE3reChiCHXD3q+nAlcaYtqFm82jgbx7E8RJwE4Ax5hTgHOCN2id59f6KyNeh1xkVOjQKWFx7Bg729xhtjMkNjXdcCfzd7fhqizdeY0yniOd9sDOiJEVhNoQv3t94+e39NcaMx84Ou1JEDtZxWlz/FyNpuf4kGWNGAQ8ALbFT/PYCQ0TkP8aY3wAbRGSKMeY2bN9mFbbl+IyIPODXeEPn3oSd0gwwF7gt1V1sxpimwNPAmdg9fH4uIrNC3/PF+2uM6YWd9tsS2IGd9ivGmNeBu0Tko1BL4RFs4gP4nYg8kYr4GhjvM9gbTgD77+RuEXndo3gnAyOADsBWYJuI9Pbx+xtPvH56f3sDS7FT/PeHDq8SkW8ZY5YAw0RkQ33/F+uiCUYppZQrtItMKaWUKzTBKKWUcoUmGKWUUq7QBKOUUsoVmmCUUkq5QhOMyjrGmCnGmP9N8WsGjTE96vjedcaYuamMR6lU0GnKKmMYY1ZjV6V3FJGtEccXA32Ak0VktUexBYFTROQLB685hSMVrguw2yqEF8ktEpEGFQU1xtwMjBSRi+o55zrgNuz7ulBELmnIa6nMpqViVKZZhV2d/kcAY8xpQGGyFw1VCchJ9X449RGRm4GboWZ/kR4ikqotFbYBE7GL7s5K0WuqNKMJRmWavwLXE0owwA3As8C94ROMMU8DFSIyLvTnK4BfA92wdZZuFZE3jDELgH8CJdgS96cZY/ZhCxV+E9iOXTH+59B18rAbtN0ItMOujL4yonT7RcaYOUBbYBq2MkLQGPN94Ici8s3QdYLAHcAYoDl2k6c7G5LcjDEXAg8CBvgK+LGI/DP0vdHAr7Abj20Jxf4F8BBwnDGmEqgUkQ61rysib4Succz3lArTBKMyzXvA94wxp2Jv8Ndii17eG+1kY0w/bAIaCcwHTgCaRZzyPeyeF4LtgpqPLavREegFzDPGfCki5cBPsK2nYaHXPh3YF3Gt4dj6Tc2Bj4FXqbuW07eAs4EibMFOARLaoTG04dZM7D4e5djdHmeGdrQEWzLoLBH50hjTETheRD43xowhRheZUvHQBKMyUbgVsxD4HFhfz7k3Ak+JyLzQn2uf+7SILAMwxnTBJqtLReQAsMQY82TotcqBH2LrM4WLFn5a61oTRGQnsNMY8zZ2/KKuBPM7EdkObDfGPIRNXIluAXwDMCNiy4LXjTH/wdbrCk8qKA5ta7CBI9v3KuUInUWmMtFfge8A38e2TurTBbtFbF0iy9h3BLaLyJ6IY2uAcGXcWNfaFPF8H7Z1Es/rrgm9dqJOBL5rjNkZfmBbRR1FZAd2m+HbgU3GmFfqmuWmVENpC0ZlHBFZY4xZhe2qujHG6euA7vV8P3Ka5QaglTGmWUSS6cqRVk/4WksTj/oYXYBlEa/RkNbFOuBJEflxtG+KyGvAa8aYQuD3wJ+AwRz9OyvVYJpgVKa6EWgpInuNMfX9O58KzDXGzMZunnQC0ExEltc+UUTWGWPeBe43xpQBPUOvc13olCeB34a6ob4ATsNuT9yQXQp/Zox5H9vKuQP4QwOu8QzwbmjztQXYqcznYxNXHraL7m3s1OZKIDyJYDPQxRiTLyJV0S4cmtCQj72H5BpjGgOHReRwA+JUGUq7yFRGEpEvReSjOM77APgBMAnYhR23ObGeHxmF3RxqA/Aydh+P8BjHH7Cbss0FdmOTV5MG/gqzsBMBlgCvha6VEBH5CrgKO0NuK7ar7Q7s//s84BfYbrtt2MkHt4V+9A1gNfC1MaaijsuPxu4dMgnb6tmPQ9vsqsyhCy2V8hk3FmUq5QVtwSillHKFJhillFKu0C4ypZRSrtAWjFJKKVdoglFKKeUKTTBKKaVcoQlGKaWUKzTBKKWUcoUmGKWUUq74/0zelCRmkozgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "initial_theta = np.zeros(n)\n",
    "op_result = minimize(fun=cost_function, x0=initial_theta, jac=gradient_function, args=(X, y, 100, True), method='TNC')  # lambda is zero in the args tuple\n",
    "\n",
    "fig4, ax4 = plt.subplots()\n",
    "# plotting y=1 values\n",
    "ax4.scatter(x=X_old[y_slim == 1, 0], y=X_old[y_slim == 1, 1], marker='+', c='black', s=50, label='Accepted')\n",
    "# plotting y=0 values\n",
    "# X[y_slim == 0, 0] is logical indexing with rows with y=0 only\n",
    "ax4.scatter(x=X_old[y_slim == 0, 0], y=X_old[y_slim == 0, 1], marker='o', c='xkcd:light yellow', s=25, label='Regected', edgecolor='k')\n",
    "\n",
    "# labels\n",
    "ax4.set_xlabel('Microchip Test 1')\n",
    "ax4.set_ylabel('Microchip Test 2')\n",
    "\n",
    "# Specified in plot order\n",
    "ax4.legend()\n",
    "\n",
    "theta = op_result.x[:,np.newaxis]\n",
    "plot_decision_boundary(theta=theta, X=X, y=y, hypothesis=sigmoid, precision=0.1, fig=fig4, ax=ax4, feature_map=(map_feature, 6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With a larger λ, you should see a plot that shows an simpler decision\n",
    "boundary which still separates the positives and negatives fairly well. However, if λ is set to too high a value, you will not get a good fit and the decision boundary will not follow the data so well, thus underfitting the data."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}