{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Gentle Introduction to Machine Learning with Python and Scikit-learn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Guillermo Moncecchi, Diego Garat, Raúl Garreta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This **Python 3** IPython notebook is based on the planned version of the Second Edition of the \"Learning Scikit-learn: Machine Learning with Python\". We do not know if this version will finally come to light (since we are really short of free time). Anyway, the code for this version will be available here, with (we hope) useful comments. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_In this notebook, we show (using very simple examples) the general procedures for performing machine learning using scikit-learn. We will address classification, regression and clustering. We will use the Iris flower dataset, introduced in 1936 by Sir Ronald Fisher for classification and clustering, and the Boston houses dataset to for regression._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting the environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start by importing numpy, scikit-learn, and pyplot, the Python libraries we will be using. Show the versions we will be using (in case you have problems running the notebooks). The code here presented should run in Python 3 (it is time...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "%pylab inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import scikit-learn, numpy, scipy and pyplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python version: 3.3.5\n",
      "IPython version: 3.2.0\n",
      "numpy version: 1.9.2\n",
      "scikit-learn version: 0.16.1\n",
      "matplotlib version: 1.4.3\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import scipy as sp\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn\n",
    "import IPython\n",
    "import platform\n",
    "    \n",
    "print ('Python version:', platform.python_version())\n",
    "print ('IPython version:', IPython.__version__)\n",
    "print ('numpy version:', np.__version__)\n",
    "print ('scikit-learn version:', sklearn.__version__)\n",
    "print ('matplotlib version:', matplotlib.__version__)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every method implemented on scikit-learn assumes that data comes in a dataset. Scikit-learn includes a few well-known datasets. The [Iris flower dataset](https://en.wikipedia.org/wiki/Iris_flower_data_set) includes information about 150 instances from three different Iris flower species, including sepal and petal length and width. The natural task to solve using this dataset is to learn to guess the Iris species knowing the sepal and petal measures. Let's import the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "X_iris = iris.data\n",
    "y_iris = iris.target\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset includes 150 instances, with 4 attributes each. For each instance, we will also have a target class (in our case, the species). This class is a special attribute which we will aim to predict for new, previously unseen instances, given the remaining (known) attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(150, 4) (150,)\n",
      "Feature names:['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']\n",
      "Target classes:['setosa' 'versicolor' 'virginica']\n",
      "First instance features:[ 5.1  3.5  1.4  0.2]\n"
     ]
    }
   ],
   "source": [
    "print (X_iris.shape, y_iris.shape)\n",
    "print ('Feature names:{0}'.format(iris.feature_names))\n",
    "print ('Target classes:{0}'.format(iris.target_names))\n",
    "print ('First instance features:{0}'.format(X_iris[0]))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us display each instance in a 2d-scatter plot, using first sepal measures, and then petal measures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEZCAYAAACTsIJzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYVOXZ+PHvvewCIrsUQYRlAQM2bBEVkSKrrxFFIEgT\nNILlZ8VXLCCKbUU0QUkwSozYUCzYMIoJgq/KArGgBlQUMdKrFEGK9OX+/XHODDPD1J09U3bvz3XN\ntXP6PWdm557zPM95HlFVjDHGGJ+cdAdgjDEms1hiMMYYE8QSgzHGmCCWGIwxxgSxxGCMMSaIJQZj\njDFBLDGkmYhcKiLT0x2HARG5XERmJ7mPTiKyMMry50XkgSjLS0TkxWRiyAYiUioiV0VYlrZzEC2u\nqsQSg8dEZJmI/E+k5ar6sqp2Kcd+S0Vkp4hsFZEtIvKliAwXkeoJ7GO/iPwm0WMnKtHjiEh1Efmz\niKwUkW0islRExnoZY0VR1dmqemy0VdwHIlIsIivDLK8K/OchwjLPRUhA0eKqMiwxeC/iB01EqiW5\n38GqWgAcAdwG9AemJrgfSSIGr45zJ9AGOF1V84Fi4D9eBJUm0c5Fqt6PTGbnIM0sMaSQW1TxsYj8\nRUQ2AiWBxRfiGCsi69yrgG9E5PhouwRQ1Z2qOhPoAZwpIhe6+2srIp+KyGYRWSMij4tInrtslruP\nr91f5X1FpK6I/FNE1ovIJhF5V0QKQ+Jf7F6lLBGRSwKWXSkiC9ztpolIs0jHieNUnQa8rao/ua9v\nuaq+FHCsJiIy2Y1ziYj8b8CyEhF5U0RedeP8j4icFLD8DhFZ5C77TkR6xhEPIvKCiNzqPi90r4Ju\ncKdbisjP7vOgqwAROUVE5rrHexWoCaiI1ALeA5q452WriDTGSfjV3eNtFZFvReTUCDH9XUQeCZn3\njojc7D4fLiKr3P0sFJFzIuynq3sutrrr3xawrJuIfOV+hj4WkRMDli1zz+d37vv+nIjUcJdF/Swl\nQkTaicgnbgxfiUjngGWlIjJSRP7txj9dRA4LWD5QRJaLyEYRuceN+X9E5HycHyAXu+d/XsAhW0Ta\nX5Whqvbw8AEsBc5xn18O7AUG4yTlmu682e7yLsCXQIE7fQxwRIT9zgCuDDN/JvAn93kboK17rObA\nAmBIwLr7gd8ETNcHLnLjqg28DvzDXXYosAU4yp1uBLR2n/8e+NGNNwe4C/g40nHceZuB9hFe213A\ncuB64ERAApbl4Fw93A3kAkcCi4Hz3OUlwB6gF1AN50pqCVDNXd7Hd06BfsB2oFHA+zM7QkxXAFPc\n55cAi4BX3ekrA85TMbDSfV7dfR1D3Fh6u7GNdJd39q0bcJwSYCdwPk7ifwj4NEJMnYAVAdP1gB04\nV5DHACsCXmuz0PcgYLu1QAf3eR3gFPf5KcA64HQ3loE4n+c8d/ky4Bug0D32v4EHYn2Won1+A87B\ni+7zQmAjcL47fa47fZg7XYrz2WvlHmsG8Ed3WWtgG9AeyAMecc+/7//xPmBiyLFL3ff2oP1VpYdd\nMaTeGlX9m6ruV9VdIcv2AvnAcSKSo6o/qPurOZH94/yToqpzVfVz91jLgadwvozCUtVNqvoPVd2l\nqttxvpQC198PnCgih6jqOlVd4M6/Duef5wdV3Q/8EfitiBRFOVY9Vf0kwuI/AqOBS4EvgFUiMtBd\ndjrQQFVHqeo+VV0KPINTjObzpaq+paplwF9w/sHPdI/7ph64Enkd50vljEhxBpgFdBQRwflCfhjo\n4C7rjJOQQ7UDclX1r6papqqT3dfjE6nIZLaqTlPnm+ol4OQI6/0b5+qjkzvdB/jEfX1lQA3geBHJ\nU9UVqrokwn72uOsVqOoWVfX9er4GGK+qX6hjIrDbfV3gXN2MU9XVqroZeBAYAHF9luL1B2Cqqk5z\n9/sBzo+nCwNimKCqi9z/p9eB3wacjymq+omq7gXuJbhYVzj4PVDguQj7qzIsMaReaGWjn6p+BIwD\n/gasE5HxIpKf4P6bApsARORo93J+rYhswfnHjXhZLCK13GMuc9efCdQREVHVX4GLcZLAGne/x7ib\nNgf+6l7qbwZ+dueXq+jATWRPqGpHnF+wDwLPicix7rGa+I7lHu9O4PCAXawK2Je6043d1zhQROYF\nbHtCtHMSsJ/FwK84XxKdgH+65+Fo4CzCJ4YmwOqQectjnwHWBTzfAdQUkYP+V93X9irulzHOlczL\n7rJFwM04v77Xicgkt6gqnN5AV2CZWzTj++JvDtwWcq6buq/LJ/DzvMK3LNpnKY7XH6g50Dckhg44\nV0U+gT+eduJcoeDGEvhZ2MmBz2Y0kfZXZVhiSL2oLR5U9XFVPQ3nMvhoYFi8O3Z/obcBfE0u/45T\nfNRKVevgFNFEe89vc4/Z1l2/MwG/qlT1fVU9D+efciHwtLvdCuAa9yrA9zhUVT+LN/ZIVHW3qj6B\nU/R0nHuspSHHKlDVbgGb+a9U3C/Upjhf4s1xrpoGA/VVtR7wLZF/uYeaCfTFKUpZ405fjnOF9lWY\n9ddycHJsHvjywr3kOGPxmQT0cV9bW2Cyf0eqk1S1k3tMxbkKO/iAql+qak+gIfA2zq9kcM71gyHn\nuraqvhawebOQ575EGPWzlIAVOMVKgTHkq+rDcWy7Bue9B0BEDiH4R0CVb30UiSWGDCIip4nIGeJU\nEO8AduEUCUTcxN2ullsh9w4wR1V9LZNq45Sx7nB/bV8fsv06oGXAdG2cX0hbRKQ+ThmsL7bDReT3\nInIoTpHXrwGxPQmMEJHW7rp1JLiSOfQ4UYnIEBHpLCKHiEiuiAxyY5sHfA5sE5Hb3eXVROQEETkt\nYBenishFIpKL86t5F/AZTj2J4pRR54jIFThXDPGaCdyIU6wETnn0jThFP+G+ZD4F9onITSKSJyK9\ncIrCfNYBh4lIQeDLTyAeVPUrnNfzDDBNVbeC/2rxHLcyeDcRPktuXJeKSB236G1bwHpPA9eJ04hB\nRORQEblQRHy/oAW4QZzK+Po4Pzx8SSPiZynB1/oS0F1EznPf65riVPAHJtxI+5nsbnumOM24S0LW\n/Qmnojl0+yrfKsoSQ2qFa7oaOK8A5xftJpyKvY04FWaRjBORrTgf8LHAGziVlj5DcYoXtrr7fTXk\n+CXAC+4leh/gUeAQ97if4LSa8a2fA9yC84vwZ5zilOsBVPVtnF+jr7rFBvNxKtIjHQe3JUgHwtsB\n/BnnF/cG9zi9VXWZW4fRDadIZ4m7/Cn33OHG+w5OsdcmnHqKXm4Z/wJ3v5+65+wEnHJ6AraN9ity\nFs4Xni8xfOyer1kh66l7XnyV4JfjnLN+BP+iX4jzi3+JOC13fK2Swn1GonkFOMf961MDp65mA855\nbIBT5BbOH4Cl7nt3Dc45Q1X/A1yNU7y5Cac+ZmBAPOoe832cBgA/AqPcZdE+S7Fel/8cqOoqnMYN\nI4D1OFcQtxH85a0Rtv0O+F+cz/0anKS3HidRgvP/AvCziHwZa39ViYT/oVOBBxBZhvPFVAbsVdW2\nYdZ5DLgA5wvh8oDKL2MSIiL34RSdXZbuWCo7EVkKXOXWjWU890pnM87nI566niorNwXHUKBYVTeF\nWygiXXHeqKNE5AyccvF24dY1Jg5VvhjAHCAi3YEPcT4XY4BvLCnElqqipGj/rD2AFwBUdQ5QV0Qa\npSQqUxlVyUt/E1EPnOLP1Tj1XP2jr24gNUVJS3BujCrDaRP9dMjyd3HawH/iTn8ADHfLN40xxqRY\nKoqSOqjqWhFpCPyfiCxU1dAeLMPdZGKMMSYNPE8MqrrW/btBRP6B09Y6MDGsJqDdOU6746CbgkTE\nEoUxxpSDqiZc7+ZpHYPbvj7ffX4ocB5OU8ZAU3CawOHecfmLqq4LWSftfYeEPu677760x5ANMWVq\nXBaTxVQV4iovr68YGgH/cO8fyQVeVtX3ReRaAFUdr6pTxendcRHOTVNXeByTMcaYKDxNDOp0cHZQ\nB1SqOj5k+kYv4zDGGBM/u/O5nIqLi9MdwkEyMSbIzLgspvhYTPHL1LjKw/PmqhVBRDQb4jTGmEwi\nImg5Kp9T0VzVGFNJJN5rtkmVivzxbInBGJMQu3rPPBWdsK2OwRhjTBBLDMYYY4JYYjDGGBPEEoMx\nxpgglhiMMVXWsmXLyMnJYf/+/ekOJaNYYjDGVHnW0iqYJQZjTErULyhARPyP+gUFsTdK0OjRo2na\ntCkFBQUce+yxfPTRR6gqf/rTn2jVqhUNGjTg4osvZvPmzQCcddZZANStW5f8/HzmzJmDqjJq1Cha\ntGhBo0aNGDRoEFu3bgVg165d/OEPf6BBgwbUq1ePtm3bsn79egAmTJhA69atKSgooGXLljz11FMV\n/vpSJt29/8XZQ6AaY9Iv0v9ivfx838h5/ke9/PyDtw14VPT/9cKFC7WoqEjXrl2rqqrLly/XxYsX\n66OPPqpnnnmmrl69Wvfs2aPXXnutDhgwQFVVly1bpiKiZWVl/v08++yz2qpVK126dKlu375de/Xq\npZdddpmqqj755JPavXt33blzp+7fv1/nzp2rW7duVVXVf/3rX7pkyRJVVZ05c6bWqlVL586dW6Gv\nMZJI59Kdn/h3bnk2SvXDEoMxmSHqF1CML/5YiSGe5BLNjz/+qIcffrh+8MEHumfPHv/84447Tj/8\n8EP/9Jo1azQvL0/Lysp06dKlByWGc845R//+97/7p3/44QfNy8vTffv26XPPPaft27fXb775JmY8\nPXv21L/+9a9xx5+Mik4MVpRkjMkIm7dtIzQzbN62Le7tW7VqxaOPPkpJSQmNGjViwIABrFmzhmXL\nlnHRRRdRr1496tWrR+vWrcnNzWXduoOGfQFg7dq1NG/e3D/drFkz9u3bx/r167nsssvo0qUL/fv3\np7CwkOHDh7Nv3z4A3nvvPdq1a8dhhx1GvXr1mDp1Kj///HP5T0gaWWIwxqREvfx8BPyPevn5FX6M\nAQMGMHv2bJYvX46IMHz4cJo1a8a0adPYvHmz/7Fjxw4aN24ctiuJJk2asGzZMv/0ihUryM3NpVGj\nRuTm5nLvvffy3Xff8cknn/DPf/6TiRMnsnv3bnr37s3tt9/O+vXr2bx5M127dvWVeGQdSwzGmKSF\nfumH++LftHVrUHHFJrdCt6L897//5aOPPmL37t3UqFGDmjVrkpuby3XXXceIESNYsWIFABs2bGDK\nlCkANGzYkJycHBYvXuzfz4ABAxg7dizLli1j+/btjBgxgv79+5OTk0NpaSnz58+nrKyM/Px88vLy\nqFatGnv27GHPnj00aNCAnJwc3nvvPd5///0KfX2pZJ3oGWOSVhFf8vXy85GQoqNErip2797NnXfe\nyffff09eXh4dOnTgqaeeolGjRqgq5513HmvWrOHwww+nf//+9OjRg1q1anHXXXfRoUMH9u7dy/Tp\n07nyyitZs2YNZ511Frt27eL888/n8ccfB+Cnn37iuuuuY9WqVdSuXZv+/ftz2WWXkZOTw2OPPUa/\nfv3YvXs33bt35/e//33S5yRdbDwGY0zc3P790x2GCRHpfSnveAxWlGSMMSaIJQZjjDFBLDEYY4wJ\nYonBGGNMEEsMxhhjgnieGESkmojME5F3wywrFpEt7vJ5InK31/EYY4yJLhX3MQwBFgCRGiTPVNUe\nKYjDGGNMHDy9YhCRpkBX4BmcmyHDruZlDMYYYxLjdVHSWGAYEGl4JAXai8jXIjJVRFp7HI8xxiQl\nPz8/qC+l8mjRogUffvhhxQTkAc+KkkSkG7BeVeeJSHGE1eYCRaq6Q0QuAN4Gjg63YklJif95cXEx\nxcWRdmmMMd7ZlkCPr5H4BiuqaKWlpZSWlia9H8+6xBCRh4DLgH1ATaAAmKyqA6NssxQ4VVU3hcy3\nLjGMyQDJdImxfv16Jk6cyK87fqXn73ty8sknV3B0FaOsrIxq1ap5eowjjzySZ599lnPOOSeh7fbt\n20du7sG/57OmSwxVHaGqRap6JNAf+Cg0KYhII3HTpoi0xUlUm8LszhiT4b788kvGjBnD888/z+7d\nu4OW/fTTT5za9iSmLRjD1zue5JzfdazwopTRo0fTt2/foHlDhgxhyJAhbN26lauuuoomTZrQtGlT\n7rnnHvbvd0q4n3/+eTp06MCtt95KgwYNuP/++1m0aBGdO3embt26NGzYkP79+/v3mZOTw5IlSwDY\nuXMnt912Gy1atKBu3bp06tSJXbt2ATBlyhSOP/546tWrx9lnn83ChQvDxr17925uvvlmCgsLKSws\n5JZbbmHPnj2AcwXQtGlTHn74YRo3bsxVV11VoecsovKM7pPoA+gMTHGfXwtc6z4fDHwLfAV8ArSL\nsH18wxgZYzwV6X9x0quT9LBGtbX7kMZ6yrkN9cyOp+quXbv8y++6+0694IZGOllP1sl6st7+Vgs9\nvf3JQfv48ccftV3HNlpQt5a2aXuCzp8/P6HYli9frrVq1dJt27apquq+ffu0cePGOmfOHO3Zs6de\nd911umPHDl2/fr22bdtWx48fr6qqEyZM0NzcXB03bpyWlZXpzp07tX///vrQQw+pquru3bv1448/\n9h9HRHTx4sWqqnrDDTfo2WefrWvWrNGysjL99NNPdffu3frDDz/ooYceqh988IHu27dPH374YW3V\nqpXu3btXVVVbtGjhH1Xunnvu0TPPPFM3bNigGzZs0Pbt2+s999yjqqozZszQ3NxcveOOO3TPnj26\nc+fOhN4XbGhPY4zXIv0vHt64vo7+4iidrCfrm/tP0t+e01Bfeukl//Ibb7pOBz7S2J8Yxsw7Wo85\nvoV/+a5du7RFy0K9YmxTnbDheL3+mWbauGkD/3jK8erYsaNOnDhRVVXff/99bdWqla5bt05r1KgR\n9KX6yiuv6Nlnn62qTmJo1qxZ0H4GDhyo11xzja5ateqgY/gSQ1lZmR5yyCFhh/kcOXKkXnzxxf7p\n/fv3a2Fhoc6cOVNVgxNDy5Yt9b333vOvO336dG3Rwjk3M2bM0OrVq+vu3bujvu6KTgx257PJOPUL\nCvyVcyJC/YKCdIdkYvhl01aaHlcDcMq1Gx+TFzSsZY9uvZj+2HZ++PRXflqym5eGbqJH917+5YsW\nLWJ/tR10u/kwChrkcu5V9ajbJIf58+cnFMcll1zCpEmTAHjllVe45JJLWL58OXv37qVx48b+4T2v\nu+46NmzY4N+uqKgoaD8PP/wwqkrbtm054YQTmDBhwkHH2rhxI7t27aJly5YHLVu7di3NmjXzT4sI\nRUVFrF69+qB116xZc9BQomvWrPFPN2zYkOrVqydwFpJnicFknNCxfxMZ99ekx/+cV8yLQzfy6y9l\nfP/v7Xz25pagloO/+93vGD3qMZ4etJNRnTdw1kkX8+DIP/mX161bl1827GL7Zmf85F2/lvHz6p3U\nrVs3oTj69OlDaWkpq1ev5u233+aSSy6hadOm1KhRg59//tk/tOeWLVuCkk5oC6FGjRrx1FNPsXr1\nasaPH88NN9zgr1fwadCgATVr1mTRokUHxdGkSROWL1/un1ZVVq5cSWFhYdh1Q4cSbdKkScTYUsES\ngzEmaS89/xrV1p7E9UWLGH/Zdp576iVOOumkoHUGDbycJf9dxZqVG/nrX8aRl5fnX1ZYWMgVV1xF\nSac1vHTHOkrOWkPX83tw3HHHJRRHw4YNKS4u5vLLL+c3v/kNxxxzDI0bN+a8887j1ltvZdu2bezf\nv5/Fixcza9asiPt54403WLVqFeAkLREhJyf46zInJ4crr7ySW2+9lbVr11JWVsann37Knj176Nev\nH//617/46KOP2Lt3L3/+85+pWbMm7du3P+hYAwYMYNSoUWzcuJGNGzcycuRILrvssoRed4UrT/lT\nqh9YHUOVAjjVX+7D3v/M4eV7sX//fn377bf1gQce0Ndee033799frv28+OKLKiI6ZswY/7wtW7bo\n9ddfr02bNtU6deroKaecoq+99pqqqj7//PPaqVOnoH3cfvvtWlhYqLVr19aWLVvq008/7V+Wk5Pj\nr3zeuXOn3nzzzVpYWKh16tTRzp07++sy/vGPf2jr1q21Tp06WlxcrAsWLPDvI7COYdeuXXrTTTdp\n48aNtXHjxjpkyBB/ncKMGTO0qKgo5muO9L5QzjoGG9rTZJz6BQVBxUf18vMrfOB4Uz42tGdmquj7\nGCwxGGPiZokhM2XNDW7GGGOykyUGY4wxQSwxGGOMCWKJwRhjTBBLDMYYY4JYYjDGGBPEEoMxxpgg\nlhiMZ6wzPJMpunbtyosvvlju7RMZzrMihv5MN7vBzXhGRAh81wTs5qgsZze4ZSa7wc0Yk3XKyuDH\nHw9Mb9oE69enL55AZWVl6Q4h41hiMMYkbc0a2Lv3wPSKFcHLv/oKOnaEL75wksK558LLL1dsDNGG\n9jz77LN59tlngfBDeW7atInu3btTp04d2rZty913302nTp38+wkczvPyyy9n8ODBdOvWjYKCAtq1\naxfUJXesoT99w5727duXxo0bU7duXTp37syCBQsq9oQkwRKD8Uy9/HwE/I96+flpjsh4ZeRIGDDA\nSQ4ffACnnQaBY9Kceio88wy0bw+HHeYkhptvPrBcFYYOBbenazZscJbv2xd/DAMGDGDq1Kls374d\ncK4E3njjDS699FIgeFyDzz//nJYtW7J+/XpGjBjBDTfcQH5+PuvWreOFF15g4sSJUcdBeO211ygp\nKWHz5s20atWKu+66K+x6Q4cOZd68eXz66ads2rSJRx55xL/fCy+8kEWLFrFhwwbatGnjjzMjlKdL\n1lQ/sG6XjckIkf4Xd+1SvfBC1Tp1VOvWVZ016+B1fv75QG/qn39+8PJHHlFt2VJ17lzVE05Qvftu\n1UR73g43tKeqanFxsT777LOqevBQnvv27dO8vDz973//65939913a8eOHf3TgeM8X3755Xr11Vf7\nl02dOlWPPfbYg9aNNvRnqM2bN6uIJDyUqU+k9wUb2tOUh7UcMhWhRg24/nrYssV53q5d8HJf8dGw\nYTBlCnTr5hQrBRo6FHr3hjZtnCuOkSMh0cHLwg3tGU7gUJ4bNmxg3759QfOaNm0a9TiNGjXyPz/k\nkEP8VymBog39uX//fu644w5atWpFnTp1OPLIIxERNm7cGP0FpoglhirOhtE0FeGDD+CKK+D//s/5\nUvcVK/ls2gR9+sDo0dC9u1OstHJl8D42bICpU6FmTZg9O7goKl7hhvYMJ7CYqGHDhuTm5rIyIKCV\nocGVQ7ShP19++WWmTJnChx9+yJYtW1i6dGlgCUnaWWIwxiTtq69g8mTnqmDyZDjiCNi8+cDyVq1g\nxIgDVwDdu0OvXgeWqzrzevaEHTvguuugS5fE6hgg/NCesVSrVo1evXpRUlLCzp07WbhwIS+++GLE\nOoZ4v7yjDf25fft2atSoQf369fn1118ZMWJEQq/Ta5YYjDFJGzoUfI14atSAcePg8MPj314EXnnl\nQPHR0KHw7ruQm5t4LJdccgkffvhh1KuF0C/9cePGsWXLFo444ggGDRrEgAEDqF69etA20bYPXe4z\nZswYTjzxRE4//XQOO+ww7rzzTlSVgQMH0rx5cwoLCznhhBM488wzo1Z2p5rnN7iJSDXgS2CVqnYP\ns/wx4AJgB3C5qs4Ls45myiVWZWPDaJpEVJUb3IYPH8769euZMGFCukOJSzbe4DYEWAAcFLWIdAVa\nqepRwDXA31MQjwmwaevWoNYIlhRMVfTDDz/wzTffoKp8/vnnPPfcc1x00UXpDittPE0MItIU6Ao8\ng9OUPVQP4AUAVZ0D1BWRRmHWM8bPWlKZirZt2zZ69+5N7dq16d+/P0OHDqVHjx7pDittylGCl5Cx\nwDAg0n9uIRBY/b8KaAqs8zguk8V8Lal8xFpSmSSddtpp/BjYZ0cV51liEJFuwHpVnScixdFWDZkO\nW4BZUlLif15cXExxcbRdGmNM1VNaWkppaWnS+/Gs8llEHgIuA/YBNXGuGiar6sCAdZ4ESlX1VXd6\nIdBZVdeF7Msqn42f9dqaPlWl8jnbZE3ls6qOUNUiVT0S6A98FJgUXFOAgQAi0g74JTQpGBPK+mAy\nxlte1zEEUgARuRZAVcer6lQR6Soii4BfgStSGI/JUtZyKr0yqb298YYN1GOMMZVUxhUlmcqvekCT\nURGhuv2SNKZSSGVRkqlk9sJBlcDGmOxnVwzGGGOCWGIwxhgTxIqSTLnlEVx8lJeuQIwxFcquGEy5\n7QkZDnCPtRwzGWjlSmfgH5/Zsw8eJMiLbbOZJYYqIC+k9VBehrcesk7yTEVatswZFKi01Hn06uXM\n83rbbGb3MVQB2daFRLbFazJfaSmcfbbzfMYMSKSrtWS2TTe7j8EYY0yFsMRgjKnUZs+Gvn2dX/sz\nZjjPA+sNvNo2m1lRUhWQJ0LgmOq5wN4MPp823KipSCtXOvUCvjGpZ8+GFi2gqMjbbTNBeYuSLDEY\nY0wlZXUMxhhjKoQlhiogmeaf0ba1ZqXGVE5WlFQFJNP8M9q21qzUmMxmRUnGGGMqRMy+kkSkJtAb\naBGwvqrqSA/jMsYYkybxXDG8A/TA6X5/u/v41cugTMVKZozkaNva2MvGVE4x6xhE5FtVPSFF8USK\nweoYjMkC2d7uv7Lxso7hExE5qRwxVUnpaKljrYNMpqiqnc5VNhGvGERkvvu0GnAUsBTY7c5TVU1Z\nssimK4Z0tNSx1kEmk2Rzp3OVTXmvGKJVPnd3/yoHD+dr3zrGGFNJRUwMqroMQEReVNXLApeJyIvA\nZeG2M8ZUXYGdzoHz/K23DtQ5mOwQz9CeQRXPIpILnBrPzt2mrjOBGkB14B1VvTNknWKclk9L3FmT\nVXVUPPvPRPXy85GQDuAq4zGNCadFi+BE8NZbzjyTXaLVMYwA7gQOAXYGLNoLPKWqd8R1AJFaqrrD\nTSj/Boaq6r8DlhcDt6pqjyj7yJo6BmOMyRQV3ipJVR9S1XxgjKrmBzzqx5sU3P3scJ9Wx6nI3hRm\ntcweazLDVQ8ZurN6yNCd0ZZ72aLJWksZk52iXTG08T0lTGWzqs6N6wAiOcBcoCXwd1W9PWR5Z+At\nYBWwGudo4HA3AAAZ4ElEQVSKYkHIOnbFEEWsVknp6u/IWksZk15etEr6C05COASnTuEbd/5JwJfA\nmfEcQFX3A78VkTrAdBEpVtXSgFXmAkVucdMFwNvA0aH7KSkp8T8vLi6m2NrAGWNMkNLSUkpLS5Pe\nTzx3Pr8F3Keq893pE4D7VbV3wgcTuQfYqapjoqyzFDhVVTcFzLMrhijsisEYE46Xdz4f60sKAKr6\nLXBcnEE1EJG67vNDgN8B80LWaSTiFHqLSFucZBWuHsJEkAdBfRblJbDcy/6OrC8lY7JTPM1VvxGR\nZ4CXcP6/LwG+jnP/jYEX3HqGHOBFVf1QRK4FUNXxQB/gehHZB+wA+if4Gqq8PTF+hUdb7uVYyjZO\nszHZKZ6ipEOA6wHfLSqzcCqRd3kcW2AMVpRkKsT06dP521NjARh8zS106dIlru2idQ5nHceZTOVZ\nUZKq7lTVv6jqRe5jbCqTQrZJpolmrGan5d3Oq+E5yxtvukyfPp1LB/WheY8FNO+xgEsH9WH69Olx\nbRutczjrOM5UNtGaq76hqn1F5FsObq5qnehF4NUwmsls51Xlc7ZVLvfofT7Neyzg7EH1AZjxwiaW\nT2nNlMnT4to+Wudw1nGcyUReNFcd4v7tVr6QjDHGZKNoneitcZ+eC8xU1R9TE5Ix3hh8zS1cOqiP\nf3rS8M28/MItcW0brXM46zjOVDbxVD6PBDoCR+Lc2DYLmK2qX3kfnj+GrClKql9QwOaQDu3ibZ1T\nXYS9AdN5xG5xFM920WJKR7zpZJXPpiopb1FSzMQQcIBDgGuAoUATVa2W6MHKK5sSg6l6Vq6E11+f\nw8xP7gOgc/v76dfvDM8Tw5w58MUXcOONzvS4cXD66XDGGd4e12QPz1olicg9IvIe8D7QCrgNsN9C\nGSRWyyLrzM5br78+h2HDWlLtqAKqHVXAsGEtef31OZ4f94sv4Kab4NFHncdNNznzjElWPEVJ83C6\n2v4XTjHSJ6q6O+pGFcyuGKJLplWSSV6P3udT7agC3h79OgA9h/ej7Metcbd2Ssajj8ItbjXJ2LFw\n882eH9JkES/vYzgFpwL6c5wuLb4VkX9H38oYY0y2itklhoiciHPX81nAaTjdY8/yOC5jskbn9vcz\nbFhLeg7vB8A7Dz/BI48s9vy448bBrbc6VwrgPM/NPVDnYEx5xdNX0h+B2cBjwBequjfG+ibFYg3t\naUN/eqtfvzOAOcz8xGnN9cgji9153jr9dHjssQOJIDfXmWdMsuJulZROVsdg4lXe5qheitZqyaum\nrrH2G2u5F816Tep52e12lZNMK55o23rVt5C1OnIk0xeSl6K1WvKqn6VY+4223Ks+pUz2sCuG8Mfz\npP8gr1oHWasjR7J9IXkZV7RWS171sxRrv5GWe9mnlEktu2IwxhhTISJWPovIu1G2U1Xt4UE8xpRb\nMn0heSlaqyWv+lmKtd9oy73qU8pkj2jdbhdH21BVSz2IJ6xUFyUl039QtG296lsomXgrG6t8PnBM\nq3w2nveVlE7WKskYYxLnZV9JR4vImyLyvYgsdR9LyhemMVXPypXOL2ef2bOdeRWx36uvnkizVofT\nrNXhXH31xLj3O2cODB68gB69z6dH7/MZPHgBc7zv3slkiXgqnycAT+L0l1QMvAC87GFMWc2rpq55\nIU1d8zJ8GE1zgFdNOEeOnMgzz17AqX2KObVPMc88ewEjR06Ma9uJExfwxBPHsqlGOzbVaMcTTxzL\nxIkLkg/KVArxdKI3V1XbiMh8VT0xcF5KIiS7ipKyramrSQ0vmnA2a3U4p/YpDmoG+583S1mxaH3M\nbXv0Pp9NNdrx8aQSADoMKKH+7s/S3rTXVCwvhvb02SUi1YBFInIjsAY4NNEDGWOMyQ7xJIabgVrA\nTcADQAEwyMugjKlMvGrC2eXsMTzz8AX+ZrBvP/w3/t9V78W1bdERf+HdJ46lw4ASAD6edC833LAw\nuYBMpREzMajq5wAiIsBNqhpXO0gRqQnMBGoA1YF3VPXOMOs9BlwA7AAuV9V58YefeZLpsC7atrk4\nxUeB0yY7tGgRnAjeesuZl6x77x0ITGT6m6UA/L+r3nPnxTZwYGtgASt/+gyAG25Y6M4zBqecOtoD\nOB2YDyx3H18Dp8Xazt22lvs3F/gM6BiyvCsw1X1+BvBZhP2oyTzTpk3T7r26aPdeXXTatGkJbTtq\n1CgtatlQi1o21FGjRlXIflesUB0z5jP/tmPGfKYrVgQvnzXrwPSsWepfHmtbr7zzjuqQIQemhwxx\n5vlEOxfRYo72WpMV6zx6ddxsiykTuN+dMb+rQx/xfLnPBzoFTHcEvknoIE5R1BdA65D5TwIXB0wv\nBBqF2b5CT1a9/HwF/I96+fkJLfdKXsAxAc0LeN3RYkpHvNOmTdPDGtXWG58v0hufL9LDGtWO+0t8\n1KhRWqsgx79trYIcf3JIZr9jxnymIhu05/C+2nN4XxXZoGPGfOZfPmuWaoMGqjNmOI8GDQ58YcTa\n1itDhjj/hYMHOw84kChinYtoMUd7rcmKtm8vj5ttMWUCLxPDvDDz5sa1c6c57FfANuDhMMvfBdoH\nTH8AnBpmvYo/WQGP0P3HWu6VaMct7zKvdO/VRW98vkgn68k6WU/WG58v0u69usS1bVHLhgdtW9Sy\nYdL77d6ri/Yc3td/KnoO73vQtjNmHDhVM2Yktq1XfAnBlyACY4p2LmLFHOm1VoRo+/byuNkWU7qV\nNzHEU1Q9U0TGA5Pc6YvdeW3cb+y5kTZU1f3Ab0WkDjBdRIr14K40QptShW2DWVJS4n9eXFxMsXXZ\naIwxQUpLSyktLU1+R7EyB1AKzIj0iDcDAfcAQ0PmPQn0D5hOSVESdsWQNCtKqhhWlFR5Y8oEeFWU\nVN4H0ACo6z4/BGec6P8JWSew8rkdKap8tjqGimGVz8mzyufKG1MmKG9iiOfO5yOAB4FCVT1fRFoD\nZ6rqszG2OxGn+4wc9/Giqj4iIte63/Tj3fXGAecDvwJXaJiiqWy689kYYzKFlwP1PA+8DzRxp38E\nYnbOrqrzVbWNqv5WVU9S1Ufc+eN9ScGdvlFVW6nqyeGSgqmcpk+f7u/ALXTYyGjL0nncSMvj6SSv\nvK/Jy3ORDl51KGgqWKxLCuBL9++8gHlflefypLwPUlTG75OuopmqIlrZeTJ1DF4eN9ryWGXY5X1N\nXp6LdKnK5f3pgIdFSaVAb+ADVT1FRNoBo1W1s0e5KlwMGivOCj6edVjnoWhjCns5bnMyx421PFon\neeV9TZk6hnWybEzo1PGyE73bcO43+I2IfAI0BPpE38QYY0zWiueyAmcUyhPcR155Lk2SeZDioiTS\n1Fy1qrCiJCtKsqKk1KCim6sCbYHGAdODgCnAY0D98hysvI9UfzFbHYP3ojXDTKYZrJfHjbQ8nuaQ\n5X1NXp6LdKjKTUfTwYvEMM+XAICzgLU4dQ2jgDfLc7DyPuwXe+US68sh2j0OXn6xlPe4sWL67DPV\nxx8/MP344848H6++/LMtqXj13lblZORFYvg64PnfgJJwy1LxsMRQuUQrToh2V3SsbZORzHFjxfT4\n46oiqmPHOg+RA4nCq+KibCyG8uq9rcrFV14khm999QnAD0DngGXfledg5X1YYqh8InVqFq2DvVjb\nJiPZ48aKaezYA8vHjj0wP5lOA6Pxar9e86qzO+tEL7FHtFZJk3A6y9uIM4jObAAROQr4Jf7qbWOM\nMVklWtYAzgQuAg4NmHc00KY8Wai8D+yKoVKxoiQrSgrHipIqHpnWiV5FPiwxVC5W+WyVz+FY5XPF\nK29iiHnncyawTvS8M336dP721FgABl9zC126dMno486ZA6NGTePr752xjU8+biJ3330+Z5zh7XGj\nWbkSli07MKbz7NnOmM5FRfEtN8Yr5b3zOe1XA/E8sCsGT6SruCGZ43br9p5CmXYYcJ92GHCfQpl2\n6/ae58eNJlZRRVUuyjDphV0xmESlqy+eZI7brNXhNGt7Ax9PKgGgw4ASVnz+BCsWrff0uLHE6v/H\n+gcy6eBlt9vGGGOqkHg60TOV1OBrbuHSQQf6Q5w0fDMvvxBzqI20Hvfk4ybyz0nn0WFACQAfT7qX\nbt3aeX7caGbPhr59nSsBcJ6/9VZwnUK05cZkGitKquKs8tkqn03lVd6iJEsMxhhTSVkdg8k4Xg1n\n6cVwlzbkZOVl723iLDEYT0yfPp1LB/WheY8FNO+xgEsH9YnrSzzWduXdbyzLlkGvXk7rodJS5/my\nZUnv1mQAe28TZ0VJxhNeDWeZzianJntV1ffWipKMMcZUCGuuajxR3qahsbZLV5NTk73svU2cp0VJ\nIlIETAQOxxkm8ylVfSxknWLgHWCJO2uyqo4KWceKkrJQeZuGxtouHU1OTfaqyu9teYuSvL5i2Avc\noqpfiUht4D8i8n+q+n3IejNVtYfHsWStZD/Y6bpXIZpIMa1cCbVqdWHKZGfa14Ik8LV26dKlwl9D\nUVHwMRL5NblyJbz++hxmfnIfAJ3b30+/fmd4/sVTlb/wEpHMe1tVeVrHoKo/qepX7vPtwPdAkzCr\nJt77XxWSTKsKr1rxJHPcaMuysQXJ66/PYdiwllQ7qoBqRxUwbFhLXn99jufHzcZzZbJDyloliUgL\nYCZwvJskfPM7A28Bq4DVwFBVXRCybZUvSipvq4pM7CgvVkzZ1oKkR+/zqXZUAW+Pfh2AnsP7Ufbj\nVs/PMWTfuTKplalFSQC4xUhvAkMCk4JrLlCkqjtE5ALgbZxR4oKUlJT4nxcXF1Ns/wHGGBOktLSU\n0tLS5HdUnr66E3kAecB04OY4118K1A+Zl0gX5JVOMv35Z+KYC9GWZePYBWPGfKYiG7Tn8L7ac3hf\nFdmgY8Z8FnvDJGXjuTKpRSaOxyAiArwA/KyqYdsUikgjYL2qqoi0BV5X1RYh66iXcWa6bK18jnbc\naJXP2VahapXPJlNlZCd6ItIRmAV8g9NcFWAE0AxAVceLyGDgemAfsAO4VVU/C9lPlU4MkJkti2J5\n8MEHGT/hrwBce8UQ7rrrrjRHZEzVkpGJoaJU9cTga8UzYHQ9wHdT15sZnRwefPBBHnr4Xq58rBCA\n525azYjbR1pyMCaFLDFUYulqWZSMZq0O5/f31AyK+Z0HdsU1BKcxpmJYX0nGGGMqhPWVlAXSNQRn\nMq69YggP3XSvf9pXlGSMyXxWlJQlrPLZGJMoq2MwxhgTxOoYqjAvhrqsCJkalxeq0ms1lZ8lhgTV\nLyhARPyP+gUFaY0nXZ3kZWtcXqhKr9VUDVaUlCARITASAdIZW6Y2Zc3UuLxQlV6ryS5WlGSMMaZC\nWHPVLJepTVkzNS4vVKXXaqoGK0pKUP2CAjZv2+afrpefz6atW9MYUeY2Zc3UuLxQlV6ryR7WXNWY\nGJLtBdW+/E22sToGY2JIZghOa3lkqhK7YjBVRjJDcFrLI5ON7IrBGGNMhbBWSabK6Nz+foYNa0nP\n4f0AeOfhJ3jkkcVxbWstj0xVYkVJpsqwymdT1VirJGOMMUGsjsEYY0yFsMRgjDEmiCUGY4wxQSwx\nGGOMCWKJwRhjTBBPE4OIFInIDBH5TkS+FZGbIqz3mIj8KCJfi8gpXsZkjDEmOq+vGPYCt6jq8UA7\nYLCIHBe4goh0BVqp6lHANcDfPY6p0rFhJY0xFcnTxKCqP6nqV+7z7cD3QJOQ1XoAL7jrzAHqikgj\nL+OqTKxzN2NMRUtZlxgi0gI4BQjtzrIQWBkwvQpoCqxLSWBZ7m9PjWXA6Hr+zt188+yuXGNMeaUk\nMYhIbeBNYIh75XDQKiHTB93mXFJS4n9eXFxMcXFxBUZojDHZr7S0lNLS0qT343mXGCKSB/wTeE9V\nHw2z/EmgVFVfdacXAp1VdV3AOtYlRgS+oqQBo+sBvs7d3rQrBmNMZnaJISICPAssCJcUXFOAge76\n7YBfApOCia5Lly68/MKbLJ/SmuVTWltSMMYkzdMrBhHpCMwCvuFA8dAIoBmAqo531xsHnA/8Clyh\nqnND9mNXDMYYkyDrXdUYY0yQjCxKMsYYk30sMRhjjAliicEYY0wQSwzGGGOCWGIwxhgTxBKDMcaY\nIJYYjDHGBLHEYIwxJoglBmOMMUEsMRhjjAliicEYY0wQSwzGGGOCWGIwxhgTxBKDMcaYIJYYjDHG\nBLHEYIwxJoglBmOMMUEsMRhjjAliicEYY0wQSwzGGGOCWGIwxhgTxBKDMcaYIJ4mBhF5TkTWicj8\nCMuLRWSLiMxzH3d7GY8xxpjYvL5imACcH2Odmap6ivsY5XE8Faa0tDTdIRwkE2OCzIzLYoqPxRS/\nTI2rPDxNDKo6G9gcYzXxMgavZOKHIBNjgsyMy2KKj8UUv0yNqzzSXcegQHsR+VpEpopI6zTHY4wx\nVV5umo8/FyhS1R0icgHwNnB0mmMyxpgqTVTV2wOItADeVdUT41h3KXCqqm4Kme9tkMYYU0mpasLF\n9Wm9YhCRRsB6VVURaYuTqDaFrleeF2aMMaZ8PE0MIjIJ6Aw0EJGVwH1AHoCqjgf6ANeLyD5gB9Df\ny3iMMcbE5nlRkjHGmOyS7lZJBxGRau7Nbu9GWP6YiPzotmQ6Jd0xpeMmPRFZJiLfuMf7PMI6KT1P\nsWJK182MIlJXRN4Uke9FZIGItAuzTqrPVdSYUn2uROSYgGPNc499U5j1Unae4okpTf97d4rIdyIy\nX0ReEZEaYdZJx3dU1LgSPleqmlEP4FbgZWBKmGVdganu8zOAzzIgpuJw8z2OZylQP8rylJ+nOGJK\n+Xlyj/sCcKX7PBeokwHnKlZMaTlX7rFzgLU4rQXTep7iiCml5wloASwBarjTrwGD0n2e4owroXOV\nUVcMItIU58Q+Q/gb33rg/FOhqnOAum4FdjpjIsp8L0U7ZsrPUxwxxbO8QolIHaCTqj4HoKr7VHVL\nyGopPVdxxgTpu/HzXGCxqq4MmZ+uz1S0mCC152krsBeoJSK5QC1gdcg66ThP8cQFCZyrjEoMwFhg\nGLA/wvJCIPDDsQpomuaY0nGTngIfiMiXInJ1mOXpOE+xYkrHeToS2CAiE0Rkrog8LSK1QtZJ9bmK\nJ6Z03vjZH3glzPx0fKZ8IsWU0vOkTovJPwMrgDXAL6r6QchqKT9PccaV0LnKmMQgIt1wmq7OI3pm\nC13mWe15nDH5btI7GXgc5yY9r3VQ1VOAC4DBItIpzDopO09xxpSO85QLtAGeUNU2wK/AHWHWS+W5\niiemdJwrRKQ60B14I9IqIdOet1yJEVNKz5OItARuxim6aQLUFpFLw60aMu3peYozroTOVcYkBqA9\n0EOcm9wmAeeIyMSQdVYDRQHTTQl/yZSymFR1m6rucJ+/B+SJSH0PY0JV17p/NwD/ANqGrJLq8xQz\npnScJ5xfa6tU9Qt3+k2cL+VAqT5XMWNK07kCJ6n/x30PQ6X8MxUrpjScp9OAT1T1Z1XdB7yF8x0R\nKB3nKWZciZ6rjEkMqjpCVYtU9UicS8ePVHVgyGpTgIEAbkuOX1R1XTpjEpFGIiLu84g36VUUEakl\nIvnu80OB84DQbs1Tep7iiSnV5wlAVX8CVoqIr5uVc4HvQlZL9WcqZkzpOFeuATg/gMJJ6XmKJ6Y0\nnKeFQDsROcQ97rnAgpB10nGeYsaV6LlKd19J0SiAiFwLzg1xqjpVRLqKyCKcS/Ar0h0Tqb9JrxHw\nD/c9zgVeVtX303yeYsZE+m5m/F/gZbdIYjFwZQZ8pqLGRBrOlZvQzwWuDpiX1vMUKyZSfJ5U9Wu3\nxOBLnDrHucDT6T5P8cRFgufKbnAzxhgTJGOKkowxxmQGSwzGGGOCWGIwxhgTxBKDMcaYIJYYjDHG\nBLHEYIwxJoglBlMpiMhdIvKt2xfMPPcmnorcf7FE7nY9bBfxSR7v9yJyXMB0qYicWtHHMSacTL7B\nzZi4iMiZwIXAKaq6173V/6B+8rPMRcC7wPfutN1wZFLGrhhMZXAEsFFV94LT26Sv7yYROdX9tf2l\niEwTkSPc+aUi8qh7dTFfRE5357cVkU/E6fn044CuK2ISkUNF5DkRmeNu38Odf7mIvCUi74nIf0Vk\ndMA2V4nID+42T4nI426i6w484u7nN+7qfd31fhCRjhVx4owJxxKDqQzeB4rcL8y/ichZACKSh9OT\nZG9VPQ2YADzobqPAIW6PsDcAz7nzv8cZL6ENzhjlDyUQx13Ah6p6BnAOzhe7r0vtk4F+wInAxSJS\nKCJNgLtxBnTpABwLqKp+itPnzlBVbaOqS9x9VHP3fbMbmzGesKIkk/VU9Ve3/L0TcDbwmojcAfwH\nOB5nnAiAajj91ftMcrefLSIFIlIA1AEmikgrnOSRl0Ao5wHdRWSoO10DaObu50NV3QYgIgtwukhu\nCMxU1V/c+W8AgVcood03v+X+netub4wnLDGYSkFV9wMzgZkiMh8YhJMYvlPV0K6Ro3kA50v8IhFp\nDpQmGEovVf0xcIaInAHsDphVhvO/F1pvEKsff98+fNsb4wkrSjJZT0SOFpGjAmadAiwDfgAaut0f\nIyJ5Ejxy1cXu/I443SNvBQo4cFWRaM+Y0wH/gPVyYCD4cIM8KfAF0FlE6oozJGNvDiSDbW4sxqSc\nJQZTGdQGnheR70Tka5yy+hK3MroPMFpEvgLmAWcGbLdLROYCTwBXufMeBv7ozq9G8K/2cC2DNGD+\nAzgDoHwjIt8C94dZ58CGqmtw6jA+B/4NLAV84z+/CgwTkf8EVD6HHtcYT1i326ZKEpEZwG2qOjfN\ncRzq1pHk4tQhPKuq76QzJmPsisGY9CoRkXk4I94tsaRgMoFdMRhjjAliVwzGGGOCWGIwxhgTxBKD\nMcaYIJYYjDHGBLHEYIwxJoglBmOMMUH+P1tDp9ie9Nn4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1080b7810>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEZCAYAAABiu9n+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VPW9//HXJyRsGnZEVmPBpbTWYluqohK8XrcWVFwg\nrqi3StUfKHq1oFZUbl1qq1VvFXqBAhUX3EArYkUitFqq4kJFrFr2XUEW2ZPP749zMkyGmWQmmWQm\nyfv5eMwjc875nnM+c5Kcz5zv93u+x9wdERGReHIyHYCIiGQvJQkREUlISUJERBJSkhARkYSUJERE\nJCElCRERSUhJoo4ws4vMbFam42iozKzAzErNrFr/M2a21cwKEiwbYmbzKli30MxWVGf/2ayyY2xm\nS83sP7ItrvquQX7obFTZP4C7P+Hup1Vhu8VmtsPMtpjZZjN718xuMbPGKWyj1My+leq+U1Vb+wn3\nVeEJuaa4e767L02mbG0ej5pQA8fYw1eNCv8XT67p/dQVShLZI+E/gJk1quZ2r3X3FsDBwI3AYOCV\nFLdj1YghG/dTV+h41D5Hxz1CSSILhd/A/mZmvzWzL4HR0d/KLPCgma0Lrw4+MrPvVLRJAHff4e5v\nAgOA48zsJ+H2epvZ22a2ycxWm9kjZpYXLpsbbuPDsKrkfDNrZWYvm9l6M9toZi+ZWeeY+L8Ir17+\nbWYXRi27wswWheu9ambdEu0nheP0iJl9bWafRH8DNLOWZjY+/EwrzexuM8sxs28Dj4XHYKuZbQzL\n/8TM3g+P6XIzu6OyGML1LjezGVHTn5nZM1HTK8zse+H7yNWBmbU1sxnh/uYD3aPWSXg8zGxE+Ltf\nbWZDEsQ0yMzeiZl3g5lND9+faWYfh7+jlWZ2YzYf4zhxmZn9wsw+N7MvzexpM2sdLiurHrrUzJaZ\n2QYzGxW1bjMzmxT+DS4ys5strMYzsylAN+ClMO6bonZ7cbzt1XvurlcWvIAlwMnh+yHAHuBagkTe\nNJw3L1x+GvAu0CKcPgI4OMF25wBXxJn/JnBv+P4YoHe4r0OARcDwqLKlwLeiptsA54RxHQg8A7wQ\nLjsA2AwcFk53AHqG788CPgvjzQFuBf6WaD/hvE3A8Qk+W9lxGg40Ai4AvgZahctfIDhRNQPaA/OB\nq8Jll5Udz6jt9QW+E74/ClgLnBVOF4Tx5cSJ41BgU/i+E7AUWB5OfwvYGO8zAk+Fr2bAd4CVwNwK\njnth+HlHh5/3DOAboGWcmJoBW4AeUfPeAS4I368B+oTvWwK9svkYx/kfGQ68FR7vPOBxYGrMdsYC\nTYDvATuBI8Ll9xL8X7QEOgMflf2+YvdTyfaOzPR5ozZeGQ9Ar/AXsX+SWBazfAj7ksTJwKfAjxP9\nQ0WtlyhJPAmMTbDO9cDzUdP7nbxjyn+f8ERIkCQ2AQOBZjHlZkbHQpAovgG6JrOfOPsdAqyKmTcf\nuJggOe0EmkYtKwLeiD2eFWz/IeC34fvKTmDLgV4EVXljgb8TJMPLgRdjjyXBCXc3cHjUsv+Jjin2\neBAkie3RMQDrgN4JYpoC3B6+P4wgaTQNp5cBVxF+0agjxzj6f2QR5U/kHcPjmRO1nU4xMZclyC+A\n/4xadiWwIt5+YuKK3d6gZP9W6/JL1U3ZK2EvFnd/A3gU+F9gnZmNNbP8FLffBSirAjg8rD5aY2ab\nCU5WbROtaGbNw30uDcu/CbQ0M3P3b4BBwFBgdbjdI8JVDwF+Z0G11ibgq3B+5/12krxVMdPLCL5d\ndiP4hrkman+PE3zbTfS5fmxmcyyoRvsauJoKjkOMNwlO4ieG798k+NZ8Uvg+Vnsgl/K/5+VJ7Ocr\ndy+Nmt5OcDUXz1SCkzbAhQRXezvD6XOBM4GlFnRuOLaCfWbLMY5WALwQtd9FwF6CxFVmbdT76OPU\nifLHfWWS+4zd3gGpBFxXKUlkrwp7cbj7I+7+Q6AncDjw38lu2My6ElQxlfU8eYzgn6yHu7ckqAaq\n6G/jxnCfvcPyfQnaPcraPl5z91MJGsoXA38I11tOUBXROup1gLv/PdnY44hNMIcQnNRWALuAtlH7\naunuR4Xl4h3fqcCLQBd3b0Vwwkv2f+RNoB9BkihmX9LoS/wksYHgpNYtal63OOWq43WgvZkdTXCF\nM7Vsgbu/6+5nE5zQXySoMkwkW45xtOXA6TF/S83dfU0S664BukZNd41ZrqGxoyhJ1EFm9sPwG1ke\nwTeanUBJRauE6zU3s77AdGC+u5f1cDoQ2ApsN7MjgZ/HrL+OqEbVsPwOYLOZtQEijY9mdpCZnWVm\nBxDUZX8TFdvjwCgz6xmWbWnlG6hj95OMg8xsmJnlhds6EnjF3dcCrwG/NbP8sDG1u5mdFLWvLuEx\njP5cm9x9t5n1Jvj2newJoyxJNHX31cBfgdMJ2m/ejy3s7iXA8wSdEpqFx+SymGJVOR7R+9gDTAMe\nAFoDfwEIj9VFZtYyjGMrFf/9ZMsxjvY48Cvb1/GhvZkNSHLdZ4CRFnTA6AxcFxNDsse9QfSAUpLI\nTvG6w0bPawGMI6guWgp8Cfy6gu09amZbCC6XHyQ4cZwetfwmgn/WLeF2n4rZ/2hgUnhpfx5BPXKz\ncL9vEbQ1lJXPAW4g+Kb5FcE3658DuPuLwH3AU2E11UKCRvhE+ym7+axPBZ9tPkF9+wbgbuBcd98U\nLrsUaExwlbQx/NwHh8tmAx8Da81sfTjvGuCu8FjdDjwds6+EJzN3/4zgZDsvnN5CUPf9Nw8rseNs\n4zqCk+ZaYEL4qui4x/u7qMxU4D+AaTHVVBcDS8Lfw1XARRVsIyuOcYzfATOA18JtvU3Q+SKZ7dxF\nUMW0hCDJTSNozyhzD3BbeNxHVLC9BnHFYeX/fmtxx2ZNCb59NSH4I5vu7iPjlHuYoBfHdmCIu+/3\nrUwaJgu6f17p7idmOpb6qiEcYzP7OUGjdr9Mx5KNMnYlETag9XP37xN0KetnZidElzGzMwnqyQ8j\n+LbzWO1HKiL1iZkdbGZ9wuqxI4ARBF15JY6MVje5+/bwbWOCLoEbY4oMACaFZecDrcysAyKBqlS/\nSGrq4zFuTNCmsYWgSuxF4PcZjSiLZay6CcCCAbMWEDQSPebuN8csfwm4x93fCqdfB25x9/dqPVgR\nkQYo01cSpWF1UxfgJDMrjFMstgdBfftWIyKStXIzHQCAu282sz8DPyToY15mFeX7MHdh/xt7MDMl\nDhGRFLl7pd14M3YlYWbtzKxV+L4Z8J/s3598BkEXO8I7Qr9293XxtpfpW9cre91xxx0Zj0FxKk7F\nqTjLXsnK5JVER4I+4DkEyWqKu882s6sB3H2su79iwWiVnxPclHV5BuMVEWlwMpYk3H0hwdAQsfPH\nxkxfV2tBiYhIObrjupYUFhZmOoSkKM70UpzppThrX0a7wKZLMPho3f8cIiK1xczwJBqus6J3k4jU\nD2YNYsy7Oqc6X6KVJEQkrXRVn12qm7jVJiEiIgkpSYiISEJKEiIikpCShIiIJKQkISISx9KlS8nJ\nyaG0tLTywvWYkoSISAUaem8tJQkRybg2LVpgZuVebVq0SOs+7rvvPrp06UKLFi048sgjeeONN3B3\n7r33Xnr06EG7du0YNGgQmzYFj+8+6aSTAGjVqhX5+fnMnz8fd2fMmDEUFBTQoUMHLrvsMrZs2QLA\nzp07ufjii2nXrh2tW7emd+/erF8fPNp74sSJ9OzZkxYtWtC9e3fGjRuX1s9WozI9EmGaRjN0Ecm8\nRP+LrfPzy55wF3m1zs8vv17MK53/14sXL/auXbv6mjVr3N192bJl/sUXX/hDDz3kxx13nK9atcp3\n797tV199tRcVFbm7+9KlS93MvKSkJLKd8ePHe48ePXzJkiW+bds2HzhwoF9yySXu7v744497//79\nfceOHV5aWuoLFizwLVu2uLv7n//8Z//3v//t7u5vvvmmN2/e3BcsWJC2z1eRRMcxnF/5+TWZQtn+\nUpIQyQ4VnpAqSALJJInYRBOdZCrz2Wef+UEHHeSvv/667969OzL/29/+ts+ePTsyvXr1as/Ly/OS\nkhJfsmTJfkni5JNP9sceeywy/emnn3peXp7v3bvXJ0yY4Mcff7x/9NFHlcZz9tln++9+97uk46+O\n6iYJVTeJSJ2waevWcpcim7ZuTXrdHj168NBDDzF69Gg6dOhAUVERq1evZunSpZxzzjm0bt2a1q1b\n07NnT3Jzc1m3Lu5ja1izZg2HHHJIZLpbt27s3buX9evXc8kll3DaaacxePBgOnfuzC233MLevXsB\nmDlzJsceeyxt27aldevWvPLKK3z11VfVOBq1R0lCRDKudX4+BuVerfPz07qPoqIi5s2bx7JlyzAz\nbrnlFrp168arr77Kpk2bIq/t27fTsWPHuMNZdOrUiaVLl0amly9fTm5uLh06dCA3N5df/vKXfPzx\nx7z11lu8/PLLTJ48mV27dnHuuedy8803s379ejZt2sSZZ55ZVguS9ZQkRKTGVZYENm7Zsl81x8aw\nQTgd/vWvf/HGG2+wa9cumjRpQtOmTcnNzWXo0KGMGjWK5cuXA7BhwwZmzJgBQPv27cnJyeGLL76I\nbKeoqIgHH3yQpUuXsm3bNkaNGsXgwYPJycmhuLiYhQsXUlJSQn5+Pnl5eTRq1Ijdu3eze/du2rVr\nR05ODjNnzuS1115L22eraRrgT0RqXDpO+K3z87GoKqZUrjR27drFyJEj+eSTT8jLy6NPnz6MGzeO\nDh064O6ceuqprF69moMOOojBgwczYMAAmjdvzq233kqfPn3Ys2cPs2bN4oorrmD16tWcdNJJ7Ny5\nk9NPP51HHnkEgLVr1zJ06FBWrlzJgQceyODBg7nkkkvIycnh4Ycf5oILLmDXrl3079+fs846q9rH\no7boeRIikjbhMwoyHYZESfQ7SfZ5EqpuEhGRhJQkREQkISUJERFJSElCREQSUpIQEZGEMpYkzKyr\nmc0xs4/N7J9mNixOmUIz22xm74ev2zIRq4hIQ5XJ+yT2ADe4+wdmdiDwnpn9xd0/iSn3prsPyEB8\nIiINXsauJNx9rbt/EL7fBnwCdIpTtNJ+vCIiUjOyok3CzAqAXsD8mEUOHG9mH5rZK2bWs7ZjExFJ\nVn5+frmxnaqioKCA2bNnpyegNMj4sBxhVdOzwPDwiiLaAqCru283szOAF4HD421n9OjRkfeFhYUU\nFhbWSLwiIolsTWFk2kTKHrqUbsXFxRQXF6ceTyZvoTezPOBlYKa7P5RE+SXAD9x9Y8x8DcshkgWq\nMyzHnDlzmFM8hw4HdeDyyy+nefPmaY6u+kpKSmjUqFGN7uPQQw9l/PjxnHzyySmtt3fvXnJz9//e\nX2eH5bAgVY4HFiVKEGbWISyHmfUmSGob45UVkey1Z88epkyZwgMPPMDf//73/ZY/Pu4xii47i8X+\nB/702l2cdPJx7Ny5M237v++++zj//PPLzRs+fDjDhw9ny5YtXHnllXTq1IkuXbpw++23U1paCsAf\n//hH+vTpw4gRI2jXrh133nknn3/+OX379qVVq1a0b9+ewYMHR7aZk5PDv//9bwB27NjBjTfeSEFB\nAa1ateLEE0+MfKYZM2bwne98h9atW9OvXz8WL14cN+5du3Zx/fXX07lzZzp37swNN9zA7t27geDK\noEuXLtx///107NiRK6+8Mm3Hq5xknkxUEy/gBKAU+AB4P3ydAVwNXB2WuRb4Z1jmLeDYBNtK4vlM\nIlLT4v0v7t692wtP6ePf69ve+1/f0dt1PNAnTBwfWV5aWuotWx/gv1t0hD/nR/uzpd/zXicf5FOn\nTi23nf997FHv1LWdt2nfwq8ddnW5J8xVZtmyZd68eXPfunWru7vv3bvXO3bs6PPnz/ezzz7bhw4d\n6tu3b/f169d77969fezYse7uPnHiRM/NzfVHH33US0pKfMeOHT548GD/1a9+5e7uu3bt8r/97W+R\n/ZiZf/HFF+7ufs0113i/fv189erVXlJS4m+//bbv2rXLP/30Uz/ggAP89ddf97179/r999/vPXr0\n8D179ri7e0FBQeRpebfffrsfd9xxvmHDBt+wYYMff/zxfvvtt7u7+5w5czw3N9d/8Ytf+O7du33H\njh1J/06i5ld+rk6mULa/lCREqm/5cve5c/dNz50bzEtFvP/F5557znse186nlXzPn/Oj/aGPj/AW\nrQ7w0tJSd3cvKSnxvLxGPnX7Uf6cH+3P+dF+6pWdyj0mdPr06d7p0Bb+mw8O97HLv+3f/492fvPI\nG1OK7YQTTvDJkye7u/trr73mPXr08HXr1nmTJk3KnWCnTp3q/fr1c/cgSXTr1q3cdi699FK/6qqr\nfOXKlfvtoyxJlJSUeLNmzeI+yvSuu+7yQYMGRaZLS0u9c+fO/uabb7p7+STRvXt3nzlzZqTsrFmz\nvKCgwN2DJNG4cWPftWtXhZ+7ukkiK3o3iUjmLV0KAwdCcXHwGjgwmFddGzdupNMRjcnJCaq/Ox3e\nhO3bdlBSUgIEVTSnnvkfTLxuPV+u2M27L2/hnelbytXJvzzzBc644UAKjm5Gu66NKbqnNa+8OiOl\nOC688EKefPJJAKZOncqFF17IsmXL2LNnDx07dow8wnTo0KFs2LAhsl7Xrl3Lbef+++/H3enduzff\n/e53mThx4n77+vLLL9m5cyfdu3ffb9maNWvo1q1bZNrM6Nq1K6tWrdqv7OrVq/d7XOrq1asj0+3b\nt6dx48YpHIXUKUmICAAnngjTpkG/fsFr2rRgXvW3eyLvvbyFhW9s5ZvNJfzp5vX06XtsuUbWJyY9\nQ6sdx3Fb7zVMv7URzz49ncMP39eRsU3r9qz9rDQyveazXbRq1TqlOM477zyKi4tZtWoVL774Ihde\neCFdunShSZMmfPXVV5HHl27evJmFCxdG1ovtadShQwfGjRvHqlWrGDt2LNdcc02kHaJMu3btaNq0\nKZ9//vl+cXTq1Illy5ZFpt2dFStW0Llz57hlYx+X2qnTvtvJaqIXVCwlCRGpUUcccQRTp0xj4lU7\nGdr5M3Z+eiTPTH2hXJmWLVvyzNQXWL9mEx9/+Nl+PXuuHzaCD2c4j16ylonXr2PS8I3ce/dvU4qj\nffv2FBYWMmTIEL71rW9xxBFH0LFjR0499VRGjBjB1q1bKS0t5YsvvmDu3LkJtzNt2jRWrlwJQKtW\nrTAzcnLKn0pzcnK44oorGDFiBGvWrKGkpIS3336b3bt3c8EFF/DnP/+ZN954gz179vCb3/yGpk2b\ncvzxx++3r6KiIsaMGcOXX37Jl19+yV133cUll1yS0ueutmTqpLL9hdokRKpt7lz3du3c58wJXu3a\nlW+jSEZN/i9u2LDBH3roIb/33nv9k08+qdI2pkyZ4mbmDzzwQGTe5s2b/ec//7l36dLFW7Zs6b16\n9fKnn37a3d3/+Mc/+oknnlhuGzfffLN37tzZDzzwQO/evbv/4Q9/iCzLycmJNFzv2LHDr7/+eu/c\nubO3bNnS+/btG2n7eOGFF7xnz57esmVLLyws9EWLFkW2Ed0msXPnTh82bJh37NjRO3bs6MOHD4+0\nQcyZM8e7du1a6WdO9DshyTYJPb5URABYsSJogyirYpo3DwoKIKZKvkJ6fGn2qe59Ehm/41pEstO6\ndcGrLEk8+2zw87zzgp/z5kHjxrB7d3raLiQ7KUmICLCvd9O0acH0f/1X8LNdu/jT558Pd9wBd965\nbx2pf1TdJCIRxcVBzyaAOXOCnxVNFxaWXwdU3ZRt6uywHCIikv1U3SQiQNDGcP75+64Yzj47+Jlo\nOrq6KfYqQ+oPVTeJCLB/76bYhupkGq7Vuyn7VLe6SUlCRNJGSSL7qE1CRERqjJKEiDRoZ555JlOm\nTKny+qk8sjQdjzetbapuEpG0UXVT9lF1k4jUeUuWwK5d+6YTPKit1pUNZ96QKUmISI1bvnzf+5IS\niH10wr33Bl1qd+2C558PbtJbuzZ9+6/o8aX9+vVj/PjxQPzHlW7cuJH+/fvTsmVLevfuzW233caJ\nUeOQRD+ydMiQIVx77bX89Kc/pUWLFhx77LHlhhGv7PGmu8JMef7559OxY0datWpF3759WbRoUfoO\nRoqUJESkRm3YAD/6Ebz8cpAgLrsMRo4sX+bRRyE3F5o2hSuvhJkz4eCD9y1ftAjuugvKak2efhpe\nKD/aeIWKiop45ZVX2LZtGxBcIUybNo2LLroIKP9chn/84x90796d9evXM2rUKK655hry8/NZt24d\nkyZNYvLkyRU+x+Hpp59m9OjRbNq0iR49enDrrbfGLXfTTTfx/vvv8/bbb7Nx40Z+/etfR7b7k5/8\nhM8//5wNGzZwzDHHROLMiGSGis32FxoqXCQjYh95muh/cf5897Zt3Vu1cj/lFPdvvtm/zFNPuYN7\nly7uO3eWX/bVV+69ernfdJP7k0+6H3ywe5wng1Yo3uNL3d0LCwt9/Pjgmduxjyvdu3ev5+Xl+b/+\n9a/IvNtuu81POOGEyHT0c62HDBniP/vZzyLLXnnlFT/yyCP3K1vR401jbdq0yc3Mt2zZktoHDiX6\nnaDHl4pITYt95GkiP/gBtG0LX38NV10FzZuXX/788zB8OMyfH1x1lFU9lWnTBl5/HR54AIqK4LXX\n4KijUos13uNL44l+XOmGDRvYu3dvuXldunSpcD8dOnSIvG/WrFnk6iVaRY83LS0t5Re/+AU9evSg\nZcuWHHrooZgZX375ZcUfsIYoSYhIlcU+8jSesiqmbt2CRHLttUHVU7RVq4Iqpt69g6qknj3hm2/K\nl/nLX4I7vPPyYPLkfVVPyYr3+NJ4oquS2rdvT25uLitWrIjMi35fVRU93vSJJ55gxowZzJ49m82b\nN7NkyZLoWpNapyQhIjVq82Zo0QKmT4e+fYME8eGH5cv8v/8HvXoF7/PygobsNm32LX/7bbj+enj3\n3aBBe/bsoB0jFfEeX1qZRo0aMXDgQEaPHs2OHTtYvHgxU6ZMSdgmkeyJvKLHm27bto0mTZrQpk0b\nvvnmG0aNGpXS50w3JQkRqbLoQQHLBvmL1aYN/P73+6qYeveGBG25CfXuHSSKo47aV/V08cWpx3vh\nhRcye/bsCq8iYhPAo48+yubNmzn44IO57LLLKCoqonHjxuXWqWj92OVlHnjgAY466ih+9KMf0bZt\nW0aOHIm7c+mll3LIIYfQuXNnvvvd73LcccdV2FBe0zJ2M52ZdQUmAwcBDoxz94fjlHsYOAPYDgxx\n9/fjlPFMfQ6Rhix2UMCGcDPdLbfcwvr165k4cWKmQ0lKXb6Zbg9wg7t/BzgWuNbMvh1dwMzOBHq4\n+2HAVcBjtR+mSP2wYkXwzb/Ms8/uG9k1mel584JtROvatf4/uvTTTz/lo48+wt35xz/+wYQJEzjn\nnHMyHVatydjzJNx9LbA2fL/NzD4BOgGfRBUbAEwKy8w3s1Zm1sHd19V6wCJ1XKqPJ433uNLnn9/3\nzOuGYuvWrRQVFbF69Wo6dOjATTfdxIABAzIdVq3JirGbzKwAeBP4jrtvi5r/EnCPu78VTr8O3OLu\n78Wsr+omkSSk+njS2OnCwoq33xCqm+qa6lY3ZfzJdGZ2IPAsMDw6QUQXiZmO+xc4evToyPvCwkIK\nK/trFhFpQIqLiymu6GaWBDJ6JWFmecDLwEx3fyjO8seBYnd/KpxeDPSNrW7SlYRI5ebNK1/dVPY4\n0hdfTG66rLqpojYIXUlknzp7JWFBn67xwKJ4CSI0A7gOeMrMjgW+VnuESNUUFJQ/yf/f/wU/yy66\nK5t+/vlgG9KwZLIL7AnAXOAj9lUhjQK6Abj72LDco8DpwDfA5e6+IM62dCUhkgUy2Z9fEqvOlURW\nNFxXl5KESOpi73EYOxbWr4fbbw+mH300GEfpxz+OX37evODKorZ7O2VLHHVd1lc3iUhmxXaJvekm\n2LYN8vOD6REj4OGH9yWJ2PKZ6hKbLXE0FLqSEGnAYrvEfvAB3HBDMP3gg8F4SRWVz1QnwmyJoy6r\nC3dci4hIllN1k0gDFT04H0D//kF104MPBtMjRgRPi7vuuvjlk+kSWxtxZyqOhkJJQqSBiu0S+8AD\nQcN1WRVTbm7QcJ2ofKa6xGZLHA2FkoRIPTR/PkyevIgVa0cA0PXg33LMMT3ZsWPflcH06eWTwJln\nBo3CZY4+Gjp12jfdtWv5xuF439znz4d33tm3j7vvhvbtYejQYDq2J1IyPZViyyxdWj4pxMah3k/p\npSQhUg9NnryI3//+SPoUHQvAS78/khNOWMPf/taRvXuDMjXRe+mdd2DYMCL7uOOO4DkSRx4Zf5vJ\n7DPVuNT7Kc2SeRB2tr9I8KBvkYaq/8DTvE/RHR485NO9T9Ed3n/gaf7ggx6Z9+CD+683Z86+5XPm\nVG3fsfuobJvJ7DPVuNLxOeq78LxZ6flVvZtERCSxZDJJtr/QlYRIOddc87FDifcpuiO8oijxE05Y\n7WbBt/sHH3Q3c3/kkX3rzJ3r3q5d8M17zpzg/dy5qe33kUd8v30ccEDibSazz1TjSsfnaAhI8kpC\nbRIi9dCll/YEFrFi7d8BuOaaxRxzTE8GDdrXqFwTvZd+9KOgnaNsH1u3Bg3XiQYJTGafqcal3k/p\npTuuRUQaIN1xLVKPxD6fOt7zpqti1qxZDDj3dAacezqzZs2q/gal3lGSEKkDyrp1FhcHr4EDy9/T\nUBWzZs3iosvO45ABizhkwCIuuuw8JQrZj6qbROqIdA9qN+Dc0zlkwCL6XdYm2OakjSyb0ZMZz71a\nvQ1LnaDqJhERqTb1bhKpA2piULtrr7qBiy47LzL95C2beGLSDdWMVOqbSqubzKwpcC5QwL6k4u5+\nV82GljxVN0l9V1PjEc2aNYv/HRcM+3rtVTdw2mmnVW+DUmek7fGlZjYL+Bp4Dygpm+/uv6lukOmi\nJCENQWUn9HSc8Gs7aWgwvsxJNkkkczfzP5O5Ky+TL3THtdRzr776qrftcKBf98euft0fu3rbDgf6\nq6++mvTydOyjJuju6MwhyTuuk7mSGAc86u4fVT931QxdSUh9V1lPpHT0VMpUbyc9ijQzkr2SSNhw\nbWYLw7cbILZjAAAVYklEQVSNgMvNbAmwK5zn7v696ocpIiLZrKLeTf3Dnw7EZht9bRepRZX1REpH\nT6VM9HbSo0izXzLVTVPc/ZLK5lVp52YTgJ8A6939qDjLC4HpwL/DWc+5+5g45VTdJPWeGq4lnapd\n3RTluzEbzgV+UNXAYkwEHgEmV1DmTXcfkKb9iWSFqpyMR44cyWfLgqbBlUvWA5TbRjL+53/+h7ET\nfwfAoHNuY8CAYZET9LPPApzGjOeCWMrGh6rJE3Yyj0SVDEvUog2MArYCe8OfZa+NwL3JtIon8yK4\n/2JhgmWFwEtJbCMNbf0itaMqvYh69erlzVrkRNZp1iLHG+VZZDq/VVNv1b5ZhdscM2aMN4/aRpPm\nJ3nz5tsiPYtatgxe6mnUMJDG3k33uvsvaiA/lW2/IEwE8aqb+gLPAyuBVcBN7r4oTjmv7HOIZIuq\n9CLKb5vLkN92KrfOH0esZtJXwYX+yOM+49ShbSvcZrceB3HW7U3LlXl61DFsWP1iMB22C6inUcOQ\njt5Nx4Rvp0W9j3D3BdWIL1kLgK7uvt3MzgBeBA6PV3D06NGR94WFhRTqr1tEJKK4uJji4uKU16uo\nTeK3BL2YmhG0QZTdJ/E94F3guJT3liJ33xr1fqaZ/d7M2rj7xtiy0UlCJJtVpRfRYYd8j/HDPoxM\njx+2it07nDmTgn+FNYtL+dN/f1nhNq++fDi/GvbLyPQfrjmURjwRuYI4++zgp3oa1U+xX57vvPPO\npNZLprrpeeAOd18YTn8XuNPdz61qsDHbLyBxdVMHgp5Pbma9gWfcvSBOOVU3SZ1SlYbrY445JtJw\nfdgh3+Oee+7Zr+G6sm1W3nAN54X5Sz2N6rd0jt20yN17VjavKszsSaAv0A5YB9wB5AG4+1gzuxb4\nOUHj+XZghLv/Pc52lCRERFKQziTxFLAN+BPBTXUXAge6e1E6Ak0HJQnJtJq4vyD6W//Vlw/n1ltv\nrXCf7777brnyP/zhDzM+wqvug8he6RzgrxkwAnghfN0ANE2m61RtvVAXWMmgmhgYL7a7avMWOT5m\nzJiE+2zRtok3aW5R3VvNW7RtUquD9cWjAfyyF+nqAlsX6EpCMqkmBsaL1111+t07Wf75+oT7fO3x\nr7jn7cOA5LrE1hYN4Jed0tEFdpq7n29m/2T/sZrcNcCfiEi9V1EX2OHhz5/WRiAidVVNDIwX2111\nwrBVjLp538MgY/c5+cb17NqxO9IldtlHO5h84/q0xlQVGsCv7kum4fq/CMZP+qx2Qkqdqpsk09Rw\nHZ8arrNXOns33QWcABxKcBPdXGCeu3+QjkDTQUlC0ikdJ/whQ4YwfebTAJx1xiAOO+ywcidwYL8E\nEJsUYk/ysP+AfplOAlJ3pS1JRG2wGXAVcBPQyd0bVS/E9FGSkHSZNWsWF112HkX3tQbKqmmeTekE\nPGTIEJ55YQpXPtwZgHHXrKRRrkWmxw9bRele52e/7wIEVUnH9z6Zt/7xBleEZSYMW4VZLpf/rgMA\nf/rvLynZ41z2UHsAJl2/gUZ5xsW/blflOKVhS9tQ4WZ2O3A8cCDwAXAj8NdqRyiShf533IMU3dc6\n0iuobF4qJ9/pM5/myoc7R7bx2uNfletpVDYvenrSjcVcEbVOvDLR0/G2mWqcIslI5nkSA4E9wJ8J\nqprecvddFa8iIiL1QaVJwt17mVkLoA/wn8A4M1vn7ifUeHQitSwdPZXOOmMQ44dNiUwv/WgH44et\nikyXVTeV9UQqq26aMOyNSJmy6qayMmXVTakM6CeSDsk0XB8FnAicBPyQ4NkOc939lxWuWIvUJiHp\npIZraQjS2bvpZWBe+HrH3fekJ8T0UZKQTItNLFDxCTyZRJRqsqrt51NL3Za2sZvqwguN3SQZFDuO\nUmWPEk1mrKdUx4OqifGjpH5DYzeJ1I7YcZQqGzcpmbGeUh0PqibGj5L6LdkriZzaCEZEROqmZLrA\nikgFYntEVdbzKJkeVKn2sqqJ8aNEoIKGazN7qYL13N0H1ExIqVN1k2SaGq6lrql27yYzK6xoRXcv\nrlJkNUBJQkQkNWkfuymbKUk0HNnybTnVKweRbJPO+yQOB34FfAdoGs52d/9WtaNMEyWJhiEdg+/V\nRByxg+9psD2pC9I2wB8wEbgD+C1wOnA5kDUjwErDkY7B92oqjtjB+DTYntQXyXSBbeburxNcdSxz\n99HAT2o2LBERyQbJXEnsNLNGwOdmdh2wGjigZsMS2V+2dPOMjSN28D11P5X6JJk2iR8Bi4FWwN1A\nC+B+d/97tXZsNoHgimS9ux+VoMzDwBnAdmCIu7+foJzaJBoINVyLpEfaxm4CLkhmXqovgpFlewEL\nEyw/E3glfP9j4O8VbCulMUukYRkzZox37d7eu3Zv72PGjEl5+auvvur9B57m/QeelvR4SLHrVGUb\nIjWJJMduSuZk/n4y86ryAgoqSBKPA4OiphcDHRKUTfPhk/pizJgx3rxFTmTgu+YtcsolgsqWV2Xg\nvFQH/BPJhGSTREU3050RfpsfBDwFlF2W5AM93b13Mpc0FTGzAuAlj1PdFN7xfY+7vxVOvw7c4u7v\nxSnriT6HNGzdehzEWbc3LTfw3fS7d7L88/VJLa/KwHmpDvgnkgnp6AK7GngPOCv8aYADW4HaapWL\n/QAJM8Ho0aMj7wsLCyksLKyZiERE6qDi4mKKi4tTXi9hknD3D4EPzewJIA/o5u6Lqxxh6lYBXaOm\nu4Tz4opOEiJlrr58OL8atu8hihOGrWLUzXclvbwqPapSHfBPpDbEfnm+8847k1uxsvooYADwKbA0\nnO4FzEimLiuJbReQXMP1sajhWqpIDdci+yNdDx0yswXAycAcd+8Vzvunu383uTSUcLtPAn2BdsA6\ngru688Iz/tiwzKMEd3l/A1zu7gsSbMsr+xwiIrJPOofl2OPuX5uV21ZplSMLuXtREmWuq+5+RESk\n6pJJEh+b2UVArpkdBgwD3qrZsEREJBskM3bTdQQjwO4CngS2ANfXZFAiIpIdEl5JmFkzYCjQA/gI\nOM7d99RWYCIiknkVXUlMAn4ALCQYP+mBWolIRESyRkV3XC/08E5oM8sF3inr3ZRt1LtJRCQ1yfZu\nquhKYm/ZG3ffW0E5ERGppyq6kighGKK7TDNgR/je3b1FDceWNF1JiIikptr3Sbi7HlEqItLAJdMF\nVkREGiglCRERSUhJQkREElKSEBGRhJQkREQkISUJERFJSElCREQSUpIQEZGElCRERCQhJQkREUlI\nSUJERBJSkqgn2rRogZlFXm1aZM34iyJShyUcBbYu0Siw4YiO0dNAQz8mIpJYOp4nISIiDZyShIiI\nJJTRJGFmp5vZYjP7zMxuibO80Mw2m9n74eu2TMRZF7TOz8cg8mqdn5/hiESkPkj40KGaZmaNgEeB\nU4BVwDtmNsPdP4kp+qa7D6j1AOuYjVu2ZDoEEamHMnkl0Rv43N2Xuvse4CngrDjlKm1Ykf2pt5OI\npEMmk0RnYEXU9MpwXjQHjjezD83sFTPrWWvR1XGbtm7FIfLatHVrhiMSkbooY9VNQDL9MxcAXd19\nu5mdAbwIHB6v4OjRoyPvCwsLKSwsTEOIIiL1Q3FxMcXFxSmvl7H7JMzsWGC0u58eTo8ESt39vgrW\nWQL8wN03xsxv8PdJxNJ9EyJSkbpwn8S7wGFmVmBmjYFBwIzoAmbWwcwsfN+bIKlt3H9TEku9nUQk\nHTJW3eTue83sOmAW0AgY7+6fmNnV4fKxwHnAz81sL7AdGJypeOsa9XYSkXTQsBxZqE2LFuUamlvn\n55c76eeZsTeqfC6Qn59fbp08YE9UmdhtVLYPEanfkq1uUpLIQpW1J8RbDuy/TorbqE/HUEQqVhfa\nJEREJMspSYiISEJKElmosp5JuVHLLJyOXScvpkzsNtT7SUSSoTYJEZEGSG0SIiJSbUoSWahx1MB8\nZkZeJdPxBu/TAH8ikg6qbspCcbunVjYd8/nVxVVEKqLqJhERqTYlCRERSUhJIgvFdl+N1+W1su6r\n6uIqIumgNgkRkQZIbRJZIpleRqn2Zoo3nWoPKPV+EpFk6EqihiXTy6hKvZlipklmHQ3wJyIhXUmI\niEi1KUmIiEhCShI1LJleRqn2Zoo3nWoPKPV+EpFkqE1CRKQBUpuEiIhUm5JEBqTa/TS2fLzuriIi\nNUHVTRmQavfTpLrI1qHPLyKZp+omERGpNiUJERFJKKNJwsxON7PFZvaZmd2SoMzD4fIPzaxXbcdY\nE1LtfhpbPl53VxGRmpCxNgkzawR8CpwCrALeAYrc/ZOoMmcC17n7mWb2Y+B37n5snG3VqTYJEZFM\nqwttEr2Bz919qbvvAZ4CzoopMwCYBODu84FWZtahdsMUEWm4MpkkOgMroqZXhvMqK9OlhuMSEZFQ\nJquzk60fir0cirve6NGjI+8LCwspLCysUlAiIvVRcXExxcXFKa+XyTaJY4HR7n56OD0SKHX3+6LK\nPA4Uu/tT4fRioK+7r4vZltokRERSUBfaJN4FDjOzAjNrDAwCZsSUmQFcCpGk8nVsghARkZqTseom\nd99rZtcBs4BGwHh3/8TMrg6Xj3X3V8zsTDP7HPgGuDxT8YqINEQalkNEpAGqC9VNIiKS5ZQkREQk\nISUJERFJSElCREQSUpIQEZGElCRERCQhJQkREUlISUJERBJSkhARkYSUJEREJCElCRERSUhJQkRE\nElKSEBGRhJQkREQkISUJERFJSElCREQSUpIQEZGElCRERCQhJQkREUlISUJERBJSkhARkYSUJERE\nJKHcTOzUzNoATwOHAEuBC9z96zjllgJbgBJgj7v3rsUwRUQavExdSfwC+Iu7Hw7MDqfjcaDQ3XvV\n9QRRXFyc6RCSojjTS3Gml+KsfZlKEgOASeH7ScDZFZS1mg+n5tWVPxrFmV6KM70UZ+3LVJLo4O7r\nwvfrgA4Jyjnwupm9a2Y/q53QRESkTI21SZjZX4CD4yy6NXrC3d3MPMFm+rj7GjNrD/zFzBa7+7x0\nxyoiIvGZe6Lzcw3u1GwxQVvDWjPrCMxx9yMrWecOYJu7/ybOstr/ECIidZy7V1qdn5HeTcAM4DLg\nvvDni7EFzKw50Mjdt5rZAcCpwJ3xNpbMBxURkdRl6kqiDfAM0I2oLrBm1gn4g7v/xMy+BTwfrpIL\nPOHu99R6sCIiDVhGkoSIiNQNdfqOazM73cwWm9lnZnZLpuOJx8wmmNk6M1uY6VgqYmZdzWyOmX1s\nZv80s2GZjikeM2tqZvPN7AMzW2RmWXt1aWaNzOx9M3sp07EkYmZLzeyjMM5/ZDqeRMyslZk9a2af\nhL/3YzMdUywzOyI8jmWvzVn8fzQy/F9faGZTzaxJwrJ19UrCzBoBnwKnAKuAd4Aid/8ko4HFMLMT\ngW3AZHc/KtPxJGJmBwMHu/sHZnYg8B5wdrYdTwjaq9x9u5nlAn8FbnL3v2Y6rlhmNgL4AZDv7gMy\nHU88ZrYE+IG7b8x0LBUxs0nAm+4+Ify9H+DumzMdVyJmlkNwXurt7isyHU80MysA3gC+7e67zOxp\n4BV3nxSvfF2+kugNfO7uS919D/AUcFaGY9pP2GV3U6bjqIy7r3X3D8L324BPgE6ZjSo+d98evm0M\nNAKy7gRnZl2AM4H/I/tvCM3q+MysJXCiu08AcPe92ZwgQqcAX2RbgghtAfYAzcOE25wgocVVl5NE\nZyD6F7AynCfVFH7T6AXMz2wk8ZlZjpl9QHAj5hx3X5TpmOJ4EPhvoDTTgVSiLtyweiiwwcwmmtkC\nM/tD2Psxmw0GpmY6iHjCq8bfAMuB1cDX7v56ovJ1OUnUzXqyLBdWNT0LDA+vKLKOu5e6+/eBLsBJ\nZlaY4ZDKMbOfAuvd/X2y/Fs6wQ2rvYAzgGvD6tFskwscA/ze3Y8BviHxeG8ZZ2aNgf7AtEzHEo+Z\ndQeuBwoIagsONLOLEpWvy0liFdA1arorwdWEVJGZ5QHPAX9y9/3uXck2YZXDn4EfZjqWGMcDA8L6\n/ieBk81scoZjisvd14Q/NwAvEFTjZpuVwEp3fyecfpYgaWSrM4D3wmOajX4IvOXuX7n7XoJbDY5P\nVLguJ4l3gcPMrCDM3IMIbtKTKjAzA8YDi9z9oUzHk4iZtTOzVuH7ZsB/Au9nNqry3H2Uu3d190MJ\nqh3ecPdLMx1XLDNrbmb54fuyG1azrheeu68FVpjZ4eGsU4CPMxhSZYoIvhxkq8XAsWbWLPy/PwVI\nWGWbqTuuq83d95rZdcAsgsbL8VnaE+dJoC/Q1sxWAL9094kZDiuePsDFwEdmVnbSHenur2Ywpng6\nApPC3iM5wBR3n53hmCqTrVWjHYAXgvNE5IbV1zIbUkL/D3gi/EL4BXB5huOJK0y2pwDZ2r6Du38Y\nXtm+S9BmtgAYl6h8ne0CKyIiNa8uVzeJiEgNU5IQEZGElCRERCQhJQkREUlISUJERBJSkhARkYSU\nJKRBMbOScBjnhWb2THhDXqKyR5vZGUlsszDecOCJ5leXmZ1lZt+Omi42sx+kez8ioCQhDc92d+8V\nDtu+GxhaQdleBCO5ZptzgJ5R07rZSWqMkoQ0ZH8FeoTDU0wIH2a0wMwGhONY3QUMCq88LjCzH5nZ\nW2GZv0UNE1EpMzsgdh/h/CFm9ryZzTSzf5nZfVHrXGlmn4brjDOzR8zsOILB434dbudbYfHzw3Kf\nmtkJ6TtE0tDV2WE5RKojHEf/dGAmcBsw292vCMeFmg+8DtxO8ECeYeE6+QTPNSgxs1OAXwHnJbnL\nW2P3YWZlwzMfDXyf4MrmUzN7mODq4DaCq5ltBA+J+cDd3zazGcBL7v58GBdAI3f/cVg9dgfBmFYi\n1aYkIQ1Ns6ixqeYCE4C3gf5mdlM4vwnQjWCY7+ihvlsBk82sB8FJPC+F/Z6aYB9OkDy2ApjZIoIh\nnNsTPInt63D+NCD6yiV2CPLnw58LwvVF0kJJQhqaHeHzEyLCb+ID3f2zmPk/jln3boIT+jlmdghQ\nnOK+E+1jV9SsEoL/y9h2htikELu8bBtl64ukhdokRIKRhCMPrDezsiSyFciPKteC4ElekPoopIn2\nEe+hRE7wzPa+ZtYqrBo7l32JYWsYi0iNU5KQhiZeT6C7gTwz+8jM/gncGc6fA/Qsa7gG7gfuMbMF\nBMPTR28r3nY9an6ifXi8dd19NUGbxz8IGtiXAGXPdX4K+G8zey+q4bqyzyhSJRoqXCRLmdkB7v5N\neCXxPMEzU6ZnOi5pWHQlIZK9RoeN7AuBfytBSCboSkJERBLSlYSIiCSkJCEiIgkpSYiISEJKEiIi\nkpCShIiIJKQkISIiCf1/O4vMaWbetX0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ba1e710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure('sepal')\n",
    "colormarkers = [ ['red','s'], ['greenyellow','o'], ['blue','x']]\n",
    "for i in range(len(colormarkers)):\n",
    "    px = X_iris[:, 0][y_iris == i]\n",
    "    py = X_iris[:, 1][y_iris == i]\n",
    "    plt.scatter(px, py, c=colormarkers[i][0], marker=colormarkers[i][1])\n",
    "\n",
    "plt.title('Iris Dataset: Sepal width vs sepal length')\n",
    "plt.legend(iris.target_names)\n",
    "plt.xlabel('Sepal length')\n",
    "plt.ylabel('Sepal width')\n",
    "plt.figure('petal')\n",
    "\n",
    "for i in range(len(colormarkers)):\n",
    "    px = X_iris[:, 2][y_iris == i]\n",
    "    py = X_iris[:, 3][y_iris == i]\n",
    "    plt.scatter(px, py, c=colormarkers[i][0], marker=colormarkers[i][1])\n",
    "\n",
    "plt.title('Iris Dataset: petal width vs petal length')\n",
    "plt.legend(iris.target_names)\n",
    "plt.xlabel('Petal length')\n",
    "plt.ylabel('Petal width')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Supervised Learning: Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In 1936 Sir Ronald Fisher introduced the Iris dataset to the statistics world, using it to develop a _linear discriminant model_. What he did was to build a linear combination of the attributes that separates a species from the rest, that is, find a straight line similar to the one we suggested in the previous section.\n",
    "\n",
    "Our first task will be to predict the specie of an Iris flower given the four sepal and length measures. For the moment, we will start using only two attributes, its sepal width and length. We will do this to ease visualization, but later we will use the four attributes, and see if performance improves. This is an instance of a **classification problem**, where we want to assign a label taken from a discrete set to an item according to its features.\n",
    "\n",
    "The typical classification process roughly involves the following steps: \n",
    "- select your attributes, \n",
    "- build a model based on available data, and \n",
    "- evaluate your model’s performance on previously unseen data. \n",
    "\n",
    "To do this, before building our model we should separate training and testing data. Training data will be used to build the model, and testing data will be used to evaluate its performance.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Separate training and testing sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first step will be to separate the dataset into to separate sets, using 75% of the instances for training our classifier, and the remaining 25% for evaluating it (and, in this case, taking only two features, sepal width and length). We will also perform _feature scaling_: for each feature, calculate the average, subtract the mean value from the feature value, and divide the result by their standard deviation. After scaling, each feature will have a zero average, with a standard deviation of one. This standardization of values (which does not change their distribution, as you could verify by plotting the X values before and after scaling) is a common requirement of machine learning methods, to avoid that features with large values may weight too much on the final results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn import preprocessing\n",
    "\n",
    "# Create dataset with only the first two attributes\n",
    "X, y = X_iris[:, [0,1]], y_iris\n",
    "# Test set will be the 25% taken randomly\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=33)\n",
    "    \n",
    "# Standarize the features\n",
    "scaler = preprocessing.StandardScaler().fit(X_train)\n",
    "X_train = scaler.transform(X_train)\n",
    "X_test = scaler.transform(X_test)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that, after scaling, the mean is 0 and the standard deviation is 1 (this should be exact in the training set, but only approximated in the testing set, because we used the training set media and standard deviation):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set mean:0.00 and standard deviation:1.00\n",
      "Testing set mean:0.13 and standard deviation:0.71\n"
     ]
    }
   ],
   "source": [
    "print ('Training set mean:{:.2f} and standard deviation:{:.2f}'.format(np.average(X_train),np.std(X_train)))\n",
    "print ('Testing set mean:{:.2f} and standard deviation:{:.2f}'.format(np.average(X_test),np.std(X_test)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display the training data, after scaling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEZCAYAAAB/6SUgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VOX5//H3TQICkkAQRJYgCm64Vay4IsG2Sq0gCiKo\nuP5cav26oaIoNSrauvSrVdt+tcW9LrjUuoIWCVIXrOJCxQ0UCAZZhLJUCBDu3x/nECYhy4SZyZnJ\nfF7XNZczZ73Pidxz5j7PeR5zd0REJLs0izoAERFpfEr+IiJZSMlfRCQLKfmLiGQhJX8RkSyk5C8i\nkoWU/AUAM3vFzEYle9kGxtDPzD5P9nabEjNrZWYvmtl/zOypqOOJh5kVmVlpzOd/m9mRUcYkkBt1\nALLtzGwNsPlBje2BdUBF+Pk8d38i3m25+7GpWLYh3H06sGei2zGzecDZ7v5GwkGln2HAjkCBu7uZ\nPQSUuvu4aMOKn7vvE3UMouSf0dy9zeb3ZvYNcE5NCc/Mct19Y6MGFy0HLOogUmRn4EtP0tOZZpbj\n7hX1LylNjco+TVD4M3uhmV1lZouACWbWzsxeMrMlZrY8LB10jVmnxMzOCd+faWb/NLPbw2W/NrOB\n27jsLmb2ppmtMrPXzewPZvZoHXHHlgfmmdloM/s4LHM8aWbbhfM6hMezwsy+D/dh4ba7Ay+a2Woz\nuyJc/mkzWxRuZ5qZ9Y7Zz0NhXC+Fcb5rZrvGzN87jP17M/vOzK4Jpzczs6vNbI6ZLTOzp8ysIJzX\n0sweC6evMLP3zGzHOP9+NcZqZjcA44CTw2M7DzgFuCr8/PdwuS5m9mz4t/7azP4nZtvFZvaMmT1q\nZiuBM2rY/7Fm9ml4Lhaa2eiYeceb2UdmtjI87mPC6WeZ2exwnblhbLUd3zwzOyomnolm9nC47r/N\n7MCYZfuY2YfhvInhOb4pnvModVPyb7o6AQUEifB8gr/1hPBzd2AtcG/M8s6WEhJAX+BzYAfgtnDd\nbVn2ceBdoD1QDJxWbd26OHAScAywC7AfcGY4bzRQCnQgKINc44FRwALgOHfPc/c7wuVfBnoBHYGZ\nwF+r7evkML4CYA5wM4CZ5QH/AF4BOofbmBKu8z/AYODIcN4K4A/hvDOAfKBbeOznE5zzeNQYq7tf\nD9wCPBke2/3hvFvDz8ebWTPgReBDoAvwE+BSMzs6ZvuDgafdvS3B36e6CQRlw3xgb+CN8Fz0BR4G\nRofrHgnMC9dZDPwiXOcs4E4zO6CW46v+9x8EPAG0BV4g/P/SzFoAfwMeIPi7PAEMqWF92QZK/k3X\nJuB6d9/g7uvcfbm7/y18v4YgifSvY/357j4hLC88AnSu48q1xmXNrDvwY+DX7r7R3d8i+MfdkJLM\n3e7+nbuvIEhqPwqnrydIuD3cvSLcdq3c/SF3/6+7bwBuAPYPEzsEyeQ5d38/LIH8NWY/xwFl7n6n\nu6939zXu/l4473zgOncvi9nuMDPLCePbAdgt/FL60N1Xx3PA9cRqbH3+Yj8fBHRw9/HhOf8G+Asw\nImaZt939hXBf62oIYT2wt5nlu/tKd/8wnH4OMMHdp4Trlrn7F+H7V8J94e5vAq8B/eI5XmC6u08K\n//95DNg/nH4IkOPu94R/478B79W6FWkQJf+ma6m7r9/8wcxam9l94U/ulcA0oK2Z1ZaIv9v8xt1/\nCN+2aeCyXYDl1RJMKQ3zXcz7tTEx3E5whf5aWGYYU9sGwvLMb8MyxUrgm3BWh5jFFteyn0Lg61o2\n3QP4W1jWWQHMBjYS/BJ5FJgMPGlm35rZrWZW7z22OGOty85Al80xhXFdE8a02cJ6tjEUOBaYZ0GJ\n75Bwejdgbi1x/zwsl30f7vNYgi+/eMSe+x+AluEvmC7At9WWLaXp3s9pVEr+TVf1n8ajgd2BvuFP\n9v7UfBWZTIuA9mbWKmZa9wS2V3lM4RX4Fe7ek6CMcbmZDai+XOjUcJmfhMe+Szg9nmNfAOxax7yB\n7l4Q82rt7ovCq+4b3X1v4DCCXxCnx7G/hsZa/VgXAN9Uiynf3Y+LWb7Oskn4C2gIQdnpeWBiOKuU\noBxVhQX3YZ4lKPnt6O4FBGWyRP/fWgR0rTatOyr7JIWSf/ZoQ3BFu9LM2gPXp3qH7j4feB8oNrPm\nZnYoQRLc1n+8lcnEzI4zs17hL5dVBE1cN4WzFwM9Y9ZrA5QDy81se4KSV43brcHLBGWsS8xsOzPL\nC2vfAP8H3BKWtzCzjmY2OHxfZGb7hiWg1cCGMMbNNzmn1rK/+mKtbjFVv5zeA1ZbcLO/lZnlmNk+\nZvbjOI6V8O90qpm1DUtgq9nSfHgCcJaZHRX+QulqZnsALcLXMmCTmf0cOLrGHTTMO0CFmV1kZrlm\ndjxBWUuSQMm/6aqeYO8CWhH8A30beLWGZWLXrT5vW5c9FTgU+B64CXiKoKYcb9y17asX8DpBcnob\n+IO7Twvn/Qa4Lix7XE5wH2I+QQnh3wRJxWvZbpU4wjr9zwhuSi4CvgSKwmV+T3AP4zUzWxVud/MX\nw07A08BKgnJQCUEpCIJS0j9rOcaGxjoB6B0e63PuvongC/ZHBOWqpcD9BDefazvW6k4DvgnLTucR\n/A1x938R3swF/hMeU/fwHF1M8AthOTAS+Hu1bTb4/5+wbHkiwb2GFWEcL1H3/z8SJ4t6MJfwyuh9\nYKG7D4o0GEk5C55Kne3uN0QdS1TM7EPgqPAmtjSAmc0A/ujuD0cdS6ZLhyv/SwiujFTHa4LM7Mdm\n1jMsE/ycoJ79fNRxRcndD1Dij4+ZHWlmO4VlnzOAfYBJUcfVFESa/M2sG0GrgL+gO/hN1U7AVILy\nzJ3ABe7+cbQhSQbZA/iIoOxzGTDM3RfXvYrEI9Kyj5k9TXBDKx+4QmUfEZHGEdmVv5kdBywJHyDR\nVb+ISCOK7MrfzG4BRhE8FNOS4Or/WXc/PWYZ3QcQEdkG7l7nRXVkV/7uPtbdC919F4JHz9+ITfwx\ny+nlzvXXXx95DOny0rnQudC5qPsVj3Ro7bOZrvJFRBpJWvTn78HDOdPqXVBERJIina78pQ5FRUVR\nh5A2dC620LnYQueiYSJ/wrcuZubpHJ+ISDoyM7yeG75pUfYRkfRWe8/fErVtvUBW8heRuOhXePpJ\n5EtZNX8RkSyk5C8ikoWU/EVEspCSv4hIFlLyF5GsM2/ePJo1a8amTZvqX7iJUvIXkayVzS2YlPxF\nJKXa5+djZpWv9vn59a/UQLfeeivdunUjPz+fPffckzfeeAN357e//S29evWiQ4cOnHzyyaxYEQyg\nduSRRwLQrl078vLymDFjBu7O+PHj6dGjB506deKMM85g1apVAKxbt47TTjuNDh06UFBQQN++fVmy\nZAkADz74IL179yY/P5+ePXty//33J/34UiLq3ufq6ZnORSR6tf1bLMjL2zwAe+WrIC9v63VjXsn+\nd/355597YWGhL1q0yN3d58+f73PnzvW77rrLDz30UP/22299/fr1fv755/vIkSPd3X3evHluZl5R\nUVG5nQkTJnivXr38m2++8TVr1viJJ57oo0aNcnf3//u///NBgwb52rVrfdOmTT5z5kxftWqVu7u/\n/PLL/vXXX7u7+7Rp07x169Y+c+bMpB5jbWo7l+H0uvNrfQtE+VLyF0kPdSaZepJ7fck/ni+Qunz1\n1Ve+4447+j/+8Q9fv3595fS99trLp0yZUvm5rKzMmzdv7hUVFf7NN99slfyPOuoo/9Of/lT5+Ysv\nvvDmzZv7xo0b/YEHHvDDDjvMP/nkk3rjGTJkiP/+97+PO/5EJJL8VfYRkUitWL2a6tl/xerVca/f\nq1cv7rrrLoqLi+nUqRMjR46krKyMefPmccIJJ1BQUEBBQQG9e/cmNzeXxYtrHgJ40aJF7LzzzpWf\nu3fvzsaNG1myZAmjRo3imGOOYcSIEXTt2pUxY8awceNGAF599VUOOeQQdthhBwoKCnjllVf4/vvv\nt/2ENBIlfxFJqYK8PAwqXwV5eUnfx8iRI5k+fTrz58/HzBgzZgzdu3dn0qRJrFixovL1ww8/0Llz\n5xq7RejSpQvz5s2r/LxgwQJyc3Pp1KkTubm5/PrXv+bTTz/l7bff5qWXXuKRRx6hvLycoUOHctVV\nV7FkyRJWrFjBsccemxE3kpX8RWSbVU/sNSX35atWVSk3LA9voibLl19+yRtvvEF5eTnbbbcdLVu2\nJDc3lwsuuICxY8eyYMECAJYuXcoLL7wAQMeOHWnWrBlz586t3M7IkSO58847mTdvHmvWrGHs2LGM\nGDGCZs2aUVJSwqxZs6ioqCAvL4/mzZuTk5PD+vXrWb9+PR06dKBZs2a8+uqrvPbaa0k9vlRRx24i\nss2SkcgL8vKwamWehvw6KC8v55prruGzzz6jefPmHH744dx///106tQJd+foo4+mrKyMHXfckREj\nRjB48GBat27Ntddey+GHH86GDRuYPHkyZ599NmVlZRx55JGsW7eOgQMHcs899wDw3XffccEFF7Bw\n4ULatGnDiBEjGDVqFM2aNePuu+9m+PDhlJeXM2jQII4//viEz0ljUH/+IlKvsH/4qMOQamr7u8TT\nn7/KPiIiWUjJX0QkCyn5i4hkISV/EZEspOQvIpKFIkv+ZtbSzGaY2UdmNtvMfhNVLCIi2Saydv7u\nvs7MBrj7D2aWC/zTzI5w939GFZOISLaItOzj7j+Eb1sAOcDyCMMREckakSZ/M2tmZh8Bi4Gp7j47\nynhERGqTl5dXpe+fbdGjRw+mTJmSnIASFGn3Du6+CfiRmbUFJptZkbuXxC5TXFxc+b6oqIiioqLG\nDFFEBIDVDehptDabB7RJtpKSEkpKShoWS7o8sm1m44C17n5HzDR17yCSBhLp3qGsrIzHHnuM8vXl\nDD1xKL17905ydMlRUVFBTk5OSvexyy67MGHCBI466qgGrbdx40Zyc7e+Vs/I7h3MrIOZtQvftwJ+\nBnwYVTwism3eeecd7rjjDh577DE2bNhQZd6CBQvoc9B+vDH3f/lg5R85ov/BvP3220nd/6233spJ\nJ51UZdoll1zCJZdcwqpVqzjnnHPo0qUL3bp1Y9y4cZWDtj/00EMcfvjhXH755XTo0IEbbriBOXPm\n0L9/f9q1a0fHjh0ZMWJE5TabNWvG119/DcDatWsZPXo0PXr0oF27dvTr149169YB8MILL7D33ntT\nUFDAgAED+Pzzz2uMu7y8nEsvvZSuXbvStWtXLrvsMtavXw8EV/LdunXjtttuo3PnzpxzzjlJPWdA\ndCN5AfsCM4GPgE+AK2tYJv4hbaRJqD6qU0NGdJLUqe3f4l8m/Nk7dsnzQZd29v36d/QBPzvCN2zY\nUDn/4kt/5SdctZM/6/v7s76//88jhX7U0YdX2cann37qBx68r+e3a+0HH36Af/HFFw2Kbf78+d66\ndWtfvXq1u7tv3LjRO3fu7DNmzPAhQ4b4BRdc4D/88IMvWbLE+/bt6/fdd5+7uz/44IOem5vr9957\nr1dUVPjatWt9xIgRfsstt7i7e3l5ub/11luV+zEznzt3rru7X3jhhT5gwAAvKyvziooKf+edd7y8\nvNy/+OIL33777f0f//iHb9y40W+77Tbv1atX5Tnp0aNH5ehi48aN80MPPdSXLl3qS5cu9cMOO8zH\njRvn7u5Tp0713Nxcv/rqq339+vW+du3aBv1d0DCOkmlI8Xivsm1q+jts2rTJ2+S38t/P3sOf9f19\n4sb9fK+DO/jzzz9fuczpZ4/08/7UtTL5j5/e0/sc3Lty/urVq71LYUc//75Cf3Dp3n7O3d28+y6d\na012tTniiCP8kUcecXf31157zXv16uWLFy/27bbbrsq2Hn/8cR8wYIC7B8m/e/fuVbZz+umn+3nn\nnecLFy7cah+bk39FRYW3atWqxiEdb7zxRj/55JOrnKOuXbv6tGnT3L1q8u/Zs6e/+uqrlctOnjzZ\ne/To4e5B8m/RooWXl5fXedyJJH894Ssi22TDhg2s+6GczrttB0BOjtFl9xZVhjAcMugkXrp9DXPe\n/4Gyr8p54ur/MGTQlhLNp59+SpuOcPR57cnvkMux/7MDOa3W8+WXXzYollNOOYUnnngCgMcff5xT\nTjmF+fPns2HDBjp37lw5lOMFF1zA0qVLK9crLCyssp3bbrsNd6dv377ss88+PPjgg1vta9myZaxb\nt46ePXtuNW/RokV079698rOZUVhYyLfffrvVsmVlZVsNG1lWVlb5uWPHjrRo0aIBZ6FhNJiLiGyT\nFi1acGi/g/jr1fMZOq4Dc9//gZmvruJP4/pVLnPCkBP4bvEibh0xng3rN3DqqWcy9upxlfMLCgr4\nvmwta9dU0KpNDv9dWcGKxWtp165dg2IZNmwYo0eP5ttvv+X555/n3XffJT8/n+22247vv/+eZs1q\nvs6t3vKmU6dO3H///QC89dZb/PSnP6V///7suuuulct06NCBli1bMmfOHPbbb78q63fp0oVZs2ZV\nfnZ3SktL6dq161b73jxs5F577QUE90e6dOlSa2zJpit/SSuNMd6rJM8zT77AD7P34IKuX/Hw+et4\n/NGn2W233aos88vzL2TenDK+XbCU235zR5UWNbvvvjtDBg/lhiPLeOzq7yju9y2nnXZ6lavneHTs\n2JGioiLOPPNMdt11V/bYYw86d+7M0UcfzeWXX87q1avZtGkTc+fO5c0336x1O08//TQLFy4EoF27\ndpjZVl8czZo14+yzz+byyy9n0aJFVFRU8M4777B+/XqGDx/Oyy+/zBtvvMGGDRv43e9+R8uWLTns\nsMO22tfIkSMZP348y5YtY9myZdx4442MGjWqQcedkPrqQlG+UL1XJC2k8t/ipk2b/Omnn/abbrrJ\nn3vuOd+0adM2befRRx91M/M77rijctrKlSv9l7/8pXfr1s3btm3rBxxwgD/11FPu7v7QQw95v379\nqmzjqquu8q5du3qbNm28Z8+e/uc//7lyXrNmzSpv+K5du9YvvfRS79q1q7dt29b79+9feW/hb3/7\nm/fu3dvbtm3rRUVFPnv27MptxNb8161b5xdffLF37tzZO3fu7JdccklljX/q1KleWFhY7zHX9nch\njpp/2rTzr4na+YukBw3jmJ4ysp2/iIhER8lfRCQLKfmLiGQhJX8RkSyk5C8ikoWU/EVEspCSv4hI\nFlLyl6Rrn59fOWiFmdE+Pz/qkESkGiV/SboVq1cT2y/ziiSMgCSyLY499lgeffTRbV6/IUM3JmOY\nx8akJ3wl6cyM2L+agZ4OzXB6wjc96QlfEUlLGzbA3LlbPi9eDCtWRBdPrIqKiqhDiJSSvySdeubM\nHgsXQmwOXbCg6vx334UjjoBPPgkS/4AB8MwzyY2hrmEcBwwYwIQJE4Cah21cvnw5gwYNom3btvTt\n25frrruOfv22dEkdO3TjmWeeya9+9SuOO+448vPzOeSQQyrnVV+2pmEey8vLATjppJPo3Lkz7dq1\no3///syePTu5JyROSv6SdMtXrarSe+DyVauiDklS5Oqr4ayzgi+AF1+Evn1h2bIt8/v1g7vugh/9\nCHbaCU4+Gc49d8v8igq47LLgiwGgrAxGj4ZwmN24jBw5kldeeYU1a9aE26zg6aef5tRTTwWq9ov/\n3nvv0bNnT5YsWcLYsWO58MILycvLY/HixTz88MM88sgjdfaj/9RTT1FcXMyKFSvo1asX1157bY3L\nXXHFFXz44Ye88847LF++nNtvv71yu7/4xS+YM2cOS5cupU+fPpVxNrr6uv2M8oW6dBZJC7X9W/zv\nf92POsq9XTv3HXZwf++9rZf57rstI3N+/PHW86+/3r13b/eZM9132839N79peHw1DePo7l5UVOQT\nJkxw962Hbdy4caM3b97cv/zyy8pp1113nR9xxBGVn2PH7T3zzDP93HPPrZz3yiuv+J577rnVsnUN\n81jdihUr3Mx81apVDT9o1zCO0kBqiinJ0ro1nHce/Oc/sOOO0KdP1fmbSz3FxfDkk3DMMUEJKFZx\ncVAa6tMHjjsu+DXRUDUN41iT2GEbly5dysaNG6tM69atW5376dSpU+X7Vq1aVf7aiFXXMI+bNm3i\n6quvplevXrRt25ZddtkFM2NZ7M+lRqLkn4XUFFOS5cUX4eKLoaQEOnfeUgLabNkyOPtsuP76oOTz\n+98HpZ1YZWUwdSq0bAmTJ28pATXEsGHDKCkpqRzGsbbkH1vS6dixI7m5uZSWllZOi32/rWKHeazu\nr3/9Ky+88AJTpkxh5cqVfPPNN7GVjkal5C8i2+zjj+Gll6B//+CLoE0biL3Fs/fecMUVWz4PHw4D\nB275XFERfD77bFi7Fk46CX7xi4bV/KHmYRzrk5OTw4knnkhxcTFr167l888/59FHH6215h9vgq5r\nmMc1a9aw3Xbb0b59e/773/8yduzYBh1nMin5i8g2u+46OOig4H3r1vDHP0JBQfzr5+TA889vKfUU\nF8PEiVDLeOt1OuWUU5gyZUqdV/3VE/u9997LypUr2WmnnTjjjDMYOXIkLVq0qLJOXetXn7/ZHXfc\nwb777stBBx3EDjvswDXXXIO7c/rpp7PzzjvTtWtX9tlnHw499NCUD9Rem8ge8jKzQuARYEeC6sP9\n7n53tWU8qviasvb5+VVKPQV5eWqRI3XKloe8xowZw5IlS3jwwQejDiUuiTzklZuyqOq3AbjM3T8y\nszbAB2b2urt/FmFMWUGJXiTwxRdfUF5ezr777su//vUvHnjggcrnApq6yJK/u38HfBe+X2NmnwFd\nACV/qZV+tUgyrV69mpEjR1JWVkanTp244oorGDx4cNRhNYq06NvHzHoA04C93X1NzHSVfaQK9RsU\njWwp+2SaTC37ABCWfJ4BLolN/JsVFxdXvi8qKqKoqKjRYhMRyQQlJSWUlJQ0aJ1Ir/zNrDnwEvCq\nu99Vw3xd+UsVuvKPhq7801NG9uppQfumCcDsmhK/SE3UaZxIckTZ1PMI4E3gE6i8mLvG3SfFLKMr\nf5E0EFVbdKnftl75p8UN39oo+YuINFxal30kc7WI6RTOzGihq0KRjBN5ax/JPBtgq5uuIpJZdOUv\nIpKFlPxFRLKQyj7SYM2pWuppHlUgIrLNlPylwdarBZZIxlPZR0SyUmkpTJ++5fP06cG0xlo/akr+\nTVTzas0xm6s5pkgV8+bBiScGQ1CWlATv581rvPWjpoe8mij1gSNSv5KSYIB5CMYRbmi/kYmunyp6\nyEtERGqk5C8iWWn69GDA+KlTg9dJJ1Wt4ad6/aip7NNENTdjY8znXGCDzqVIpdLSoEbfr1/wefp0\n6NEDCgsbZ/1UUsduIiJZSDV/ERGpkZJ/E9U+P79KU8/2+flJWzeRbYtIelDZp4lKpKlnfeuqGalI\nekvKAO5m1hIYCvSIWd7d/caEIxQRkUjE07fP34H/AB8A61IbjoiINIZ4kn9Xdz8m5ZFIUhXk5WGr\nV1f5nKx1E9m2iKSHemv+ZnY/cK+7f9I4IVXZt2r+IhksndvCN2UJNfU0s1lmNgs4AvjAzL7cPM3M\nGv2LIB1F2epFLW4kE2R652dNWa1X/mbWI3zrbD1Mq7v7/NSFVRlDWl/5R9nqRS1uJFOka+dnTVlC\nrX3cfV64kUfdfVS1DT8KjKppPRERSX/xPOS1T+wHM8sFDkzGzs3sATNbHJaXRKSJyfTOz5qyumr+\nY81sNbCvma3e/AKWAC8kaf8PAgOTtK1GV5CXh0HlqzFbvUS5b5F49egBzz0XlHqKioL3PXpEG5ME\n4mnt81t3vzplAQT3Fl50931rmJfWNX8RkXSUUM3fzPqEb5+OeV/J3WcmGJ+IiESkroe8/pegpU8r\nghr/5uad+wHvA4emNrRAcXFx5fuioiKK1FQAgBZmbIj53BxYH/MrqX1+PiuqPYi1fNWqRoktyn2L\nZKOSkhJKSkoatE48ZZ/ngOvdfVb4eR/gBncfuo1xVt9+D1T2abB07nxNzVBFopWs/vz33Jz4Adz9\n38BeiQYnIiLRiSf5f2JmfzGzIjMbYGZ/Bj5Oxs7N7AngbWB3Mys1s7OSsV0REalbPGWfVsAvgbB3\nDt4E/uTuKe/hU2Wf2qnmLyK10Ri+IiJZKNGO3Z4O//vvmA7d1LGbZLTJkyczeOhABg8dyOTJk+Ne\nr7S06pOp06cH00QyVV01/0vC/x4HDKr2GpziuDJCKsfJrU+LmHXNjBZW9Uu+ru2net/pavLkyZx6\nxjB2HjybnQfP5tQzhsX9BaDeKaWpiafm//+Aae7+VeOEVGXfaV32SeU4uYmuX9f8VO87XQ0eOpCd\nB89mwBntAZj68HLmv9CbF56dFNf66p1SMkVSxvAFugP3mdkuBA93vQlMd/ePkhCjiIhEoN7k7+6/\nhspWP+cBVwF3ATmpDU0kuX513mWcesawys9PjFnBXx++LK51Y3unhOD9c89tGaFKJNPEU/YZBxwG\ntAE+AqYD/3T3spQHl+Zln0SaNCbaHDKRpp6p3nc6mzx5Mn+4/04g+DI45pj4hqfWcISSSZLS1NPM\nPgQ2AC8TlHzedvfypEVZ977TOvmLiKSjpHTv4O4HAD8F3gN+BvzbzP6ZnBBFJFEzZsC99275fO+9\nwTSRutRb8zezfQme7j0S+DGwkOAXgESovtKNnrLNHv/6F1x8MWzcGHy+/HK4+244+OBo45L0Fk/Z\n5yWCOv904F/uvqHOFZJIZZ/apXOvntL47roLLgvvXd95J1x6abTxSLSS0tTT3Y9LXkgiIpIO4mnn\nLyJp7N57g1LPnUEjJi6/HHJz4aKLoo1L0puSf4YqyMvDqtX0GzJfmo6DDgpq/JuTfW5uME2kLurV\nU5qUbW3H3xjqii3dnyNI5/MqW0u0V88X63i9kPxwG1+iHZzVt36i209l7E1RIh23pVp9saVzx3Hp\nfF5l29V65W9mRXWt6O4lKYinegwpvfKPsnO1RKk1z9YS7bgtleKJLV07jkvn8yo1S6i1T2MkdxER\niUY8D3ntDtwC7A20DCe7u++aysBEGiqRjttSrb7Y0rnjuHQ+r7Lt4nnI6y3geuB/CQZyOQvIcfdx\nKQ8uxWWMiPWrAAARDUlEQVSfRJ+CjfIpWz3BW7N0vjGpG77SWJLVsdtMd+9jZrPcfd/YaUmMtbZ9\nq7WPiEgDJWswl3VmlgPMMbOLgDJg+2QEKCLRKy2FiRNnMO3t6wHof9gNDB9+cNr86pDUqLdXT+BS\noDVwMUHHbqcBZ6QyqEyRyjF8M3WcXMk8EyfO4More5KzWz45u+Vz5ZU9mThR3YI2dXE/5GVm+QDu\nnrTCspkNZMuoYH9x91urzU/rsk8qx/BVU05pLIOHDiRnt3yev3UiAEPGDKfiq1VqypnBktKfv5kd\nZGazgFnALDP72Mx+nITgcoB7gYFAb2Ckme2V6HZFRKR+8dT8HwAudPfpAGZ2RDhtvwT33ReY4+7z\nwu0+CRwPfJbgdkWkAfofdgNXXtmTIWOGA/D32/7I7bfPjTgqSbV4kv/GzYkfwN3/aWYbk7DvrkBp\nzOeFQEYNP5FI52n1rducoNQT+1kkFYYPPxiYwbS3g4ru7bfPDadJUxZP8p9mZvcBT4SfTw6n9QFw\n95nbuO+4CtjFxcWV74uKiihKl2feIaF29fWtmykDokvmKyyE0aMPZvRo1fgzVUlJCSUlJQ1byd3r\nfAElwNTaXvWtX8d2DwEmxXy+BhhTbRmX9DNp0iQfdOIxPujEY3zSpEkNXn/8+PFe2LOjF/bs6OPH\nj0+L2BYscH/zzS2f33wzmNZY6tt/fcdV1/xUHlt9247yvKZzbKkW5s66c3B9C6TqRfCrYy7QA2gB\nfATsVW2ZhE5AQV6eE/zCcMAL8vIS2l4y1RdbovNTZdKkSb5DpzZ+0UOFftFDhb5DpzYNSrLjx4/3\n1vnNKtdvnd8saV8AicT25pvuHTq4T50avDp0qJoYUq2u/dd3XPXNT+Wx1bftKM9rOseWaklJ/sBO\nwITNV+kELXPOqW+9eF7Az4EvgDnANTXMT/wExLzS6ZdEfbElOj9VBp14jF/0UKE/6/v7s76/X/RQ\noQ868Zi41y/s2XGr9Qt7dkyL2KZO3XJKp05NSkgNUtv+6zuueI47lcdW37ajPK/pHFsqxZP843nI\n6yHgNaBL+PkrICm9Orn7q+6+h7v3cvffJGObIiISh/q+HYD3w/9+GDPto/rWS8YLXfmn3ZW/yj6p\nobJP8qVzbKlGkso+JcAOm5M/wY3aafWtl4xXoglNNf/U0A3f5NMN3+RL59hSLZ7kH0+vngcC9xD0\n5/8p0BEY5u4fb/vvjfike/cOIiLpKCndO7j7B0B/4HDgPKB3YyR+abomT57M4KEDGTx0YKOPBZvI\nvutat7Q06IN/s+nTg2nJEuU5S6VUnzepXV0DuPc1s84A7r4BOJBgRK/fmVn7RoovpTQIeuOLcjDw\nRPYd5QDsTXkA9XQeuL6pq2sA9w+Bn7j7cjM7EngKuAg4ANjT3YfVuGIyg0vzAdyl4aIcDDyRfUc5\nAHtTH0A9XQeuz2SJDubSzN2Xh+9PBu5z92eBZ81MZR8RkQxWV/LPMbPmYcnnpwT1/njWE6lVlIOB\nJ7LvKAdgb8oDqKfzwPVNXV1ln2uBXwDLgELgQHffZGa7AQ+5++EpDy7NB3CXbRPlYOCJ7DvKAdib\n6gDq6T5wfaaKp+xTXzv7Q4ETgO1jpu0O9KmvDWkyXqTRQ1nSOOp6BiDV7bITeX6hvtjefdf9nnu2\nfL7nnmBaY0n02YyoRPmMQiYjnTt2i+el5J9d6nv6N5VPZCb65HJ9sd1zj7uZ+513Bi+zql8GqZTo\nsUUpyqeTM5mSv2SUeDp9S1VHXIl2ChdPbHfeuWX+nXcmJey4JOPYohRlp3SZKp7kH0/HbiIi0tTU\n9+0Q5Qtd+WcVlX1SQ2Wfxt921FDZRzKNbvimhm74Nu62oxZP8q+3Y7coqWM3EZGGS0rHbtL0ZGon\nYaWlcO65j9C9145077Uj5577SIM7AUvVsdfXQZk6MJN0o+SfZTK5k7Abb3yEv0z4OQcOK+LAYUX8\nZcLPufHGR+JeP5XHXl8HZerATNKNyj5ZJpM7Cevea0cOHFbE87dOBGDImOF88EwJC+YsiWv9VB97\nfR2UqQMzaSwq+4iISI3UQVuWyeROwo4ZcAd/ue3nDBkzHIDnb/sD/++cV+NeP5XHXl8HZerATNKN\nyj5ZKFM7CSstDer+k6deAQRfBr/+9ekN6gQsVcdeXwdl6sBMGlM8ZR8lfxGRJiZta/5mdpKZfWpm\nFWbWJ4oYJHVSNU5uotsWiZXtzW+juuE7i6Cr6Dcj2r+kSCrHyc3kZqqSfrK9+W2kZR8zmwqMdveZ\ntcxX2SfDpHKc3Exupirpqak2v03bso+IiEQrZU09zex1YKcaZo119xfj3U5xcXHl+6KiIoqayldz\nE5XKcXIzuZmqpJ+m1Py2pKSEkpKSBq2jso8kXarGyU102yKxmnLz27Rv6hkm/yvc/YNa5iv5i4g0\nUNrW/M3sBDMrBQ4BXjaz+B/TzALJaIIWZZPIbW2umclN70pL4Xe/m1F5XL/73YxGiz2Tz5tEJ5Lk\n7+5/c/dCd2/l7ju5+8+jiCNdJdoELcomkYk018zkpncTJ87gyit7krNbPjm75XPllT2ZOHFGo+w7\nk8+bREdP+KapRJqgRdkkMtHmmpna9G7w0IHk7JZfpcfRiq9WNVoz1Ew9b5IaaVv2ERGRiNU3zmOU\nL7J0DN9EB5aOcsDu+vZd1/xMHlD7jjvedbOlPmTMST5kzEluttTvuKNxBunN5PMmqYHG8M1MyWiC\nFmWTyG1trpnJTe9KS4O6/7S3rweg/2E3MHz4wY0SeyafN0mNtG/qWZ9sTf6Q2e3Zb775Zu578PcA\nnH/WJVx77bURRySSXeJJ/hrMJQ1tbhEz8tYCAE49Yxh/ffiZjPgCuPnmm7nltl9z9t1dAbjl4l8D\n6AtAJM3oyj8NZXIHZt177cjx41pWif3vN62Le5xdEUmcWvuIiEiNVPZJQ5ncgdn5Z11SWeoBeODi\nbxl71Y0RRiQiNVHZJ03phq+IbCu19hERyUKq+Tdh6TyWbTrHlkrZetySmZT869A+Px8zq3y1z8+P\nOiQgvceyTefYUilbj1syl8o+de+f2L0bkA7nK52bgqZzbKmUrcct6UllHxERqZGaemagdG4Kms6x\npVK2HrdkLpV96tA+P58Vq1dXfi7Iy2P5qlWRxRMrnZuCpnNsqZStxy3pR009RUSykGr+ItUk0hxT\nTTmlKVHyl6yRSHNMNeWUpkZlH8kaiTTHVFNOySQq+4iISI3U1FOyRiLNMdWUU5qaSMo+ZnY7cByw\nHpgLnOXuK2tYTmUfSapEmmOqKadkirRt6mlmPwOmuPsmM/stgLtfXcNySv4iIg2UtjV/d3/d3TeF\nH2cA3aKIQ0QkW6XDDd+zgVeiDkJEJJuk7Iavmb0O7FTDrLHu/mK4zLXAend/vLbtFBcXV74vKiqi\nqKgouYGKiGS4kpISSkpKGrROZO38zexM4FzgJ+6+rpZlVPMXEWmgeGr+kTT1NLOBwJVA/9oSv4iI\npE5UrX2+AloAy8NJ77j7hTUspyt/EZEGStumnvFS8q+d2pyLSG3StqmnJEadjIlIonTln4HUyZiI\n1EVX/iIiUiN17JaB1MmYiCRKZZ8MpRu+IlIbtfYREclCqvmLiEiNlPxFRLKQkr+ISBZS8hcRyUJK\n/iIiWUjJX0QkCyn5i4hkISV/EZEspOQvIpKFlPxFRLKQkr+ISBZS8hcRyUJK/iIiWUjJX0QkCyn5\ni4hkISV/EZEsFEnyN7ObzOxjM/vIzKaYWWEUcYiIZKuorvxvc/f93f1HwPPA9RHFkTFKSkqiDiFt\n6FxsoXOxhc5Fw0SS/N19dczHNsCyKOLIJPofewudiy10LrbQuWiY3Kh2bGY3A6OAH4BDoopDRCQb\npezK38xeN7NZNbwGAbj7te7eHXgIuDNVcYiIyNbM3aMNwKw78Iq771PDvGiDExHJUO5udc2PpOxj\nZru5+1fhx+OBD2tarr7gRURk20Ry5W9mzwB7ABXAXOCX7r6k0QMREclSkZd9RESk8aX9E75mdruZ\nfRY+FPacmbWNOqaomNlJZvapmVWYWZ+o42lsZjbQzD43s6/MbEzU8UTJzB4ws8VmNivqWKJmZoVm\nNjX8t/FvM7s46piiYGYtzWxG+PDsbDP7TV3Lp33yB14D9nb3/YEvgWsijidKs4ATgDejDqSxmVkO\ncC8wEOgNjDSzvaKNKlIPEpwLgQ3AZe6+N0Gz8V9l4/8b7r4OGBA+PLsfMMDMjqht+bRP/u7+urtv\nCj/OALpFGU+U3P1zd/8y6jgi0heY4+7z3H0D8CRBY4Gs5O7TgRVRx5EO3P07d/8ofL8G+AzoEm1U\n0XD3H8K3LYAcYHlty6Z98q/mbOCVqIOQSHQFSmM+LwyniVQysx7AAQQXilnHzJqZ2UfAYmCqu8+u\nbdnInvCNZWavAzvVMGusu78YLnMtsN7dH2/U4BpZPOciS6llgtTJzNoAzwCXhL8Ask5YJflReG90\nspkVuXtJTcumRfJ395/VNd/MzgSOBX7SKAFFqL5zkcW+BWJ7fy0kuPoXwcyaA88Cj7n781HHEzV3\nX2lmLwM/BkpqWibtyz5mNhC4Ejg+vKEhgWx7AO59YDcz62FmLYCTgRcijknSgJkZMAGY7e53RR1P\nVMysg5m1C9+3An5GLQ/QQgYkf+Aegp4/XzezD83sj1EHFBUzO8HMSglaNLxsZq9GHVNjcfeNwEXA\nZGA28JS7fxZtVNExsyeAt4HdzazUzM6KOqYIHQ6cRtC65cPwlY0toToDb4Q1/xnAi+4+pbaF9ZCX\niEgWyoQrfxERSTIlfxGRLKTkLyKShZT8RUSykJK/iEgWUvIXEclCSv6S0czs2rAb34/D9t19k7z9\nIjPbqluN2qYnYX/Hx/ZIaWYlZnZgsvcjkhbdO4hsCzM7FPgFcIC7bzCz9sB2EYeVqBOAFwl6pgT1\naSQpoit/yWQ7AcvCLp5x9+XuvgjAzA4Mr5rfN7NJZrZTOL3EzO4KfyXMMrODwul9zextM5tpZm+Z\n2e7xBmFm24eDq8wI1x8cTj8zHIDoVTP70sxujVnnHDP7IlznfjO7J/wyGwTcHm5n13Dxk8Llvqir\nf3aRhlDyl0z2GlAYJsU/mNmRUNnJ1z3AUHf/McHAJzeH6zjQyt0PAC4EHginfwb0c/c+wPXALQ2I\n41pgirsfDBxFkLxbh/P2B4YD+wInm1lXM+sCXAccTNA1wZ6Au/s7BP0VXeHufdz963AbOeG2Lw1j\nE0mYyj6Ssdz9v2E9vB8wAHjKzK4GPgD2Bv4R9PlFDlAWs+oT4frTzSzfzPKBtsAjZtaL4AuieQNC\nORoYZGZXhJ+3A7qH25ni7qsBzGw20APoCExz9/+E058GYn9pVO+077nwvzPD9UUSpuQvGS3sv3wa\nMC0cz/YMguT/qbsf1oBN3USQqE8ws52ppRvcOpzo7l/FTjCzg4HymEkVBP/mqtfxqyf76vM3b2Pz\n+iIJU9lHMpaZ7W5mu8VMOgCYB3wBdDSzQ8LlmptZ75jlTg6nHwH8x91XAfls+XXQ0B4yJwOVg4ab\n2QGb39awrAP/AvqbWTszywWGsiXhrw5jEUkpJX/JZG2Ah8zsUzP7mKB2XhzeAB4G3Bp2b/shcGjM\neuvMbCbwR+CccNptwG/C6TlUvfquqcWNx0y/CWhuZp+Y2b+BG2pYZsuK7mUE9xTeA/4JfAOsDGc/\nCVxpZh/E3PCtvl+RhKlLZ8kqZjYVGO3uMyOOY/vwnkUuQU1/grv/PcqYJLvoyl8kGsVm9iEwC/ha\niV8am678RUSykK78RUSykJK/iEgWUvIXEclCSv4iIllIyV9EJAsp+YuIZKH/D1HBM6nnCceFAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10747e5d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "colormarkers = [ ['red','s'], ['greenyellow','o'], ['blue','x']]\n",
    "plt.figure('Training Data')\n",
    "for i in range(len(colormarkers)):\n",
    "    xs = X_train[:, 0][y_train == i]\n",
    "    ys = X_train[:, 1][y_train == i]\n",
    "    plt.scatter(xs, ys, c=colormarkers[i][0], marker=colormarkers[i][1])\n",
    "\n",
    "plt.title('Training instances, after scaling')\n",
    "plt.legend(iris.target_names)\n",
    "plt.xlabel('Sepal length')\n",
    "plt.ylabel('Sepal width')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A linear, binary classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start, let's transform the problem to a binary classification task: we will only want to distinguish setosa flowers from the rest (it seems easy, according to the plot). To do this, we will just collapse all non-setosa targets into the same clas (later we will come back to the three-class original problem)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New training target classes:\n",
      "[1 0 1 1 1 0 0 1 0 1 0 0 1 1 0 1 1 1 1 1 0 0 1 0 0 1 1 1 1 1 1 1 0 0 1 1 0\n",
      " 1 1 1 1 0 1 0 1 0 1 1 0 1 1 0 0 1 0 0 0 1 1 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1\n",
      " 0 0 0 0 1 1 0 1 1 1 1 0 0 1 1 1 0 1 1 0 1 1 1 1 1 0 1 0 0 0 1 1 1 1 1 1 1\n",
      " 0]\n"
     ]
    }
   ],
   "source": [
    "import copy \n",
    "y_train_setosa = copy.copy(y_train) \n",
    "# Every 1 and 2 classes in the training set will became just 1\n",
    "y_train_setosa[y_train_setosa > 0]=1\n",
    "y_test_setosa = copy.copy(y_test)\n",
    "y_test_setosa[y_test_setosa > 0]=1\n",
    "\n",
    "print ('New training target classes:\\n{0}'.format(y_train_setosa))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first classifier will be a linear one. \n",
    "\n",
    "Linear classification models have been very well studied through many years, and the are a lot of different methods with actually very different approaches for building the separating hyperplane.  We will use the `SGDClassifier` from scikit-learn to implement a linear model, including regularization.  The classifier (actually, a family of classifiers, as we will see) receives its name from using Stochastic Gradient Descent, a very effective numerical procedure to find the local minimum of a function. \n",
    "\n",
    "Gradient Descent was introduced by Louis Augustin Cauchy in 1847, to solve a system of linear equations. The idea is based on the observation that a multivariable function decreases fastest in the direction of its negative gradient (you can think of the gradient as a generalization of the derivative for several dimensions). If we want to find its minimum (at least a local one) we could move in the direction of its negative gradient. This is exactly what gradient descent does.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every classifier in scikit-learn is created the same way: calling a method with the classifier's configurable hyperparameters to create an instance of the classifier. In this case, we will use `linear_model.SGDClassifier`, telling scikit-learn to use a _log_ loss function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,\n",
      "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
      "       learning_rate='optimal', loss='log', n_iter=5, n_jobs=1,\n",
      "       penalty='l2', power_t=0.5, random_state=42, shuffle=True, verbose=0,\n",
      "       warm_start=False)\n"
     ]
    }
   ],
   "source": [
    "from sklearn import linear_model \n",
    "clf = linear_model.SGDClassifier(loss='log', random_state=42)\n",
    "print (clf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the classifier includes several parameteres. Usually, scikit-learn specifies default values for every parameter. But be aware that it is not a good idea to keep it with their default values. Later (or in future notebooks, I do not know yet), we will talk about _model selection_, the process of selecting the best parameters.\n",
    "\n",
    "Now, we just call the `fit` method to train the classifier (i.e., build a model we will later use), based on the available training data. In our case, the trainig setosa set.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,\n",
       "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
       "       learning_rate='optimal', loss='log', n_iter=5, n_jobs=1,\n",
       "       penalty='l2', power_t=0.5, random_state=42, shuffle=True, verbose=0,\n",
       "       warm_start=False)"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(X_train, y_train_setosa)\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How does our model look? Well, since we are building a linear classifier, our model is a... line. We can show its coefficients:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 30.97129662 -17.82969037]] [ 17.34844577]\n"
     ]
    }
   ],
   "source": [
    "print (clf.coef_,clf.intercept_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and we can draw the decision boundary using pyplot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOwAAAEZCAYAAACU+8kKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYVNWVwH9HFmO0UZAkKqKdREXFJYK4JC5t1IkZTYxo\nx0TN55bFGMcFZASJ0gOOJiIDAk4wSoxbFCXELSI6SRV0ghBZFBDBtQPuoiAQlK3P/HFftdVFLa+r\n6tV7r+r8vu99vOW+e081deou555zRFUxDCMebBe2AIZh+McU1jBihCmsYcQIU1jDiBGmsIYRI0xh\nDSNGmMLWICKSFJGLA6p7LxFZJyLiXX9JRGaJyFoRuUVEhonIHQG0e66IzCh3vVGjc9gCxAUROQa4\nGTgQ2Aq8BFypqvN8vNsK7KOqrwcrpW/UO8pfseoKoC7t1k+B91W1W7naEJF64HWgs6q2eu3eD9xf\nrjaiiimsD0SkG/AE8DPgIWB74FhgY0eqCUC0OLA37sctCGrvb6qqdhQ4gMOB1QXKXAQsBT4CngL2\n8u7PAlqB9cA6oNG7/xPgFeBD4FFg97S6xgLvAR8Di4C+3v1TgYXe/RXAiAIynQ4875V/Ffg3734C\nuMg7/yrwV2AV8AFwH7BzWh3XAG8Ca4FlwDe9+0cA87y63wXGePfrvc/bCfg9sAn3w7YWOBFoAu5N\nq/8YYDaw2vtM5xf6rN51q/f3XAscBVwANKeV+TrwHLAG+AdwdNqzJDAS+Jv3/gxg17C/Z76+i2EL\nEIcDN8Rb5X0BTwG6Zzw/3VO+Prh1geHA39OetwJfSbv+pqccXwO6AuOBmd6zb3mK0M277gPs5p0f\nn6a8B3uKcnoOmY/wvqwnetd7AH2880yFPRHoAvQEZgJj09pekdb+XqnPATwLnOudfx440jtPKex2\n3vVdwMg0uUakFBbX+64FzvYUvAdwaKHP6r3X1oZ3r01hvXpWA+d6/x8/wP2QdveeJ73/r32Az3l/\nj5vC/p75OWzRyQequg7XEyhwB/C+iDwqIl/0ilyC+w9frm5OdRPwNRHpnaPKc4HJqvq8qm4ChgFH\ni8heuB6pDjhARLbz6nzXk2Omqr7onS8GHsR9sbNxsdfGX7zyb6vq8iyf7TVV/YuqblbVVbjePVXn\nVtzwv6+IdFHVFfrZPHwTsK+I9FTVDao6N8+fUHKcnwM8o6pTVHWrqn6kqi/4+KyFhsKnAstV9X5V\nbVXVB3Gjg++mPjZwl6q+qqqf4qY5XytQZyQwhfWJqi5T1QtVtTdwEK7HGuc93hu4VURWi8hq3DAX\noFeO6nYH/plW97+8d3qpagKYCNwGvCcit4tIHYCIHCkiCRF5X0TW4ObUu+ZoY0/gtUKfy1vFfVBE\n3hSRj4F7U3Wq6qvAlbhh7Hsi8oCI7O69ejGwH/CSiPxDRE4t1FYWeuMWj7LJ1ZHPmskeuJFBOv/0\n7qd4N+38E2Ann3WHiilsEXg91d04xQX35fipqnZPO3ZU1Tk5qngbN3QEQER2xH0Z3/Lqn6Cqh+NW\npPcDhnhF/wA8AuypqrsAk8j9f7gSN+QrxI24nvQgVd0Z+FF6nar6gKoei/tRUuDX3v1XVfUcVf2C\nd2+qiOzgo710VuCG5NnI91kLrXC/5cmbzt7e/VhjCusDEekjIoNEpJd33Rv4IW4eB+7LdK2IHOg9\n31lEGtOqeI/2X8wHgAtF5FAR2R6nNHNUdYWIHO71Ll2ADcCnOIUC1wusVtVNInIEbkiZ68s72Wvj\nmyKynYj0EpE+WcrtBPwLWOt9vtSPAyKyn/f+9riFozZZROQ8EfmCV/RjT47WbH++HPKBU8qTRKRR\nRDqLyK4icqiPz/qB11YuZZ8O7CciP/TqPRvYH7fS70euyGIK6491wJHAXBFZj1PURcBgAFV9BNfL\nPOgNKxfjFo9SNAF3e0Pms7x55XXAH3G97ZdxCyMA3YDf4hZJWnCLXaO9Z5cCI0Vkrff+lFwCq+pz\nwIW4Oeka3ELLXlmK/hfQD6d0j3sypRRje9x8/APgHdyi1DDv2beAJSKyzmvjB6qaMnOl/4hk2nzb\nrtXZbP8d93f8ELcqfEihz6qqG4D/Bv4uIh+JyJEZ9X4InObVuwq4GjhNVT/KkCOXjJFFvFUzwzBi\ngPWwhhEjTGENI0aYwhpGjDCFNYwYEenN/yJiK2JGTaKqWc1Oke9h8+2rHDFiROh7O02mYOVZv17Z\nc09l1qzoyBT0kY/IK6xR24weDd/4Bhx7bNiSRINID4mN2mbFCpgwARYuDFuS6BDrHrahoSFsEbbB\nZCqMX3muuQZ+8QvYK9v+rDITtb9RLiK900lENMryGcHx97/DD34Ay5bBjjuGLU1lERE0rotORu3R\n2gpXXAG/+lXtKWshTGGNyHH33dC1K5xzTtiSRI/QhsQi8jlcOJLtcWFSHlXVYRllbEhcY6xdC/vv\nD48+CgMGhC1NOOQbEoc6hxWRz6vqBhHpjAuIdbWq/i3tuSlsjTF0KLzzjutla5V8ChuqWUedXyO4\nHrYTzgfUqFFeew3uvBMWLQpbkugS6hzWi4TwPC4iQ0JVl4YpjxEuQ4bAoEGwxx6Fy9YqYfewrbjo\ngjsDM0SkQVWT6WWamprazhsaGmJjLzM6xl//6jZI/OEPYUtSeZLJJMlk0lfZyNhhReQ64BNVvSXt\nns1ha4AtW6BfPxgxAs48M2xpwieSdlgR6Skiu3jnOwAn42L6GDXGnXdCjx4wcGDYkkSfMIfEu+MC\nk22H++G4V72g10btsHq161lnzACJZRzDyhKZIXE2bEjcMXp068bqdevarrvX1fHR2rUhSlSYq66C\nDRvg9tvDliQ6RNYOWwhT2I4hIu1idQoU9K8Mk2XL4JhjYOlS+OIXC5evFSI5hzWMQYPg2mtNWTuC\n+cMaofDkk26jxCOPhC1JvLAetoroXleHQNvRva6uwBvhsHmz613HjHGb/A3/WA9bRUR9gSnFbbdB\nfT2cWky+uxrHFp2MivLBB3DggTBzpvvX2BZbJTYiw89/Dl26wPjxYUsSXSLrrWPUFosWwbRp8NJL\nYUsSX2zRyagIqnDllXD99W4bolEcprBGRXjkEXj/ffjZz8KWJN7YHNYInE8/hb593fbDk04KW5ro\nYzudjFAZNw4OOsiUtRxYD2sEyjvvwMEHw5w5sM8+YUsTD8ysY4TGRRdBz55w881hSxIfzKxjhMK8\neTB9OixfHrYk1YPNYWNCj27dEJG2o0e3bmGLlBdVF73/hhsg4qLGCuthY8Lqdeva+7qmOapHkQcf\nhE8+gQsuCFuS6sLmsDEhTs7pGza46P333295XYvBzDpGRRk9Go4+2pQ1CGxIHBO619W1GwZH1dd1\n5Uq3sX/BgrAlqU5sSGyUlXPOga9+FUaNCluS+GJ2WKMi1HIS5nJic9gIEjczTSFSSZhvusmUNUhs\nDhsScTPTFOKee5xjuiVhDhYbEodEnMw0hVi3Dvr0cS50RxwRtjTxx4bERqDceCOcfLIpayWwIXFI\nxMVMU4jXX4c77rAkzJXChsRGSQwcCP37w/DhYUtSPURySCwivUUkISIvisgSEbk8LFmM4kgkXBLm\nQYPClqR2CHNIvBm4SlWfF5GdgPki8oyqWky9GLBliwuqNno07LBD2NLUDqH1sKr6rqo+752vB14C\n9ghLnmokSFvvnXdC9+6WMb3SRGIOKyL1wEygr6e8qfs2hy2BoExHq1c7b5wZM+BrXyu5OiODSEec\n8IbDU4Er0pU1RVNTU9t5Q0MDDQ0NFZPNyM7IkXD66aas5SKZTJJMJn2VDbWHFZEuwBPAdFUdl+W5\n9bAlEEQPu2yZc5t78UXL6xoUUV0lFmAysDSbshqlE0T6ycGDYehQU9awCK2HFZFjgFnAImjrCIap\n6lNpZayHjRDTp7sN/kuWWF7XIDH3OqNkNm928YVvuQVOOy1saaqbSA6JjXhx222w996WhDlsrIeN\nKF1F2Jx23QXYFNLfYtUqOOAAS8JcKWxIHEOi5H536aXQubMlYa4UkbbDGtFm0SKYOtWZc4zwsTms\nkZNUEuYRIywJc1SwHjaidMENg9OvK40lYY4eNoc1srJxo1tgsiTMlcfMOkaHsSTM0cR6WGMbUkmY\nn30W9t03bGlqDzPrhEwXEbakXXcGNkf4c1kS5nAxs07IbIFtbKpRxZIwRxubwxptpMw4loQ5upjC\nGm1MmeJyu1oS5uhic9gKEIc5rCVhjg42hw2ZqClnNiwJczywHtZg5UoXn2nBAudCZ4SLbZwIGT/h\nRjPLdE07Dzod5TXXOI8cU9boYz1sBfDjKpe1TIF3ysHs2XD22ZaEOUpYD2tkxZIwxw9T2Brmnnuc\nY7olYY4PtkpcAfyklswsk+leV+50lOvWuYxzf/oTbGc/27HB5rA1yrBh8NZbrpc1ooVt/jfa8frr\nMGCAC//Sq1fY0hiZ2KKT0Y4hQ1xOV1PW+GEKm0E5UjQGmeaxVBIJmD/fkjDHFRsSb9tmyfbPKIUo\nTWfrVujXD667Ds46K2xpjFzYkNgAXBLmXXaxJMxxJux0k78DTgXeV9WDszy3HrZMrFkDffrAU0/B\nYYeFKopRgCj3sHcBp4QsQzvKkaIxiDSPpZJKwmzKGm9Cn8OKSD3weFR62Gpk2TI45hhYutTyusaB\nkvxhReRzwJlAfVp5VdWRZZPQCJTBg91GCVPW+ONna+KjwBpgPvBpsOIY5Wb6dHjlFbcF0Yg/fhS2\nl6p+K3BJctDU1NR23tDQQENDQ1ii+CYzVWRnaBcipntdHR+tXduhOnt068bqjP3IherYvNnZW8eM\nsYzpUSaZTJJMJn2VLTiHFZHfAhNVdVHpomWtv54qm8MG4dtazMrzrbfCk0+6lWGJcmxVox1F7SUW\nkcXeaSdgX+ANYKN3T1X1kDII9gBwPLAr8D5wvarelfbcFDZfnXnqsCTM8aVYha33TpVtY1+rqv6z\nXALmwhS2QJ156rj0UujUCSZM6FAzRgQoapVYVVu8l+9V1R9lVHgv8KNs7xnb+rJ2zrgu2rZbwKc2\nxeLFloS5WvGz6HRQ+oWIdAb6ByNOdbApgFGB30WqVPT+66+3JMzVSM6dTiJyrYisAw4WkXWpAzfX\nfKxiEhod4tFH4b334JJLwpbECAI/q8S/UtWhFZIns+1YzmHDIpWEedIkOPnkwuVXrlxJS0sLx3rR\nw5ubm6mvr6d3794BS2rko6i9xCLST0T6AQ+nztOPwKQNmWJiCGeWyYwp3CVLjOGO1tHVh11m3Djo\n29efsgK0tLQwcODANjvgwIEDaWlp8feyEQr5VomTuMXNHXBz1pQd9hBgnqoeHbhwEfXWKVTGzyox\nWe4VrCPP3+Ldd13G9I4mYU4mk5xwwgkAJBKJWGxMqXaK6mFVtUFVTwDeBvqpan9V7Q8c5t0zIsTw\n4XDhhZYxvdrxs0q8v6qmNlGgqktE5IAAZTI6yPz5bkdTR804zc3NNDY2kkgkAGhsbGTatGltc1oj\nevhR2EUicidwH25kdg7wQqBShUgxMYQzy/i1w3akji455FV10ftHjYKdd85RKAf19fXtFHTatGnU\n19d3rBKjovhZJd4B+DmQ+tmdBfxGVQP33LFV4sJMmQK/+hXMm+d2Nhnxx+ISVykbNrj9wvfeC8cd\nF7Y0RrkoamuiiDysqo0isoT2C5pQps3/1UqmK1wXaOduV4x7XTZuuQWOPNKUtZbIZ9bZQ1XfTnMC\naEdqr3GQxLWHrUTqyFQS5vnzwaad1UWxm/9TppuTgJmq+koQwhnFMXSo88gxZa0t/KwS7wXcLiJf\nBubhFp2aVfX5QCUzcjJ7tvNzvf32sCUxKo3vRSdvtfinwNXAHqoa+JpkXIfEQc5hW1vdvPXyy+FH\n5uBYlZQaNfE64OvATsDzwGDgb2WVsMoox4JSLu6915lvzj03sCaMCOPHDrsQ10H8GTccnq2qG/O+\nVCbi2sMGxbp1sP/+MG2a62WN6qSkyP+qehhu4ekfwMnAEhGxHjYEbroJTjyxeGVduXIlzc3NbdfN\nzc2sXLmyJJmCqDPMdiKPquY9gIOBS4EHgVeBJDCy0HvlOJx45aV7XZ3irCwKaPe6urzPu6SdZytf\nTBvF8Nprqj16qL75ZvF1zJo1S3v27KmJREITiYT27NlTZ82aVZJcQdQZZjtRwPveZ9eJXA/0M6V5\nArgGN4/tUqh8OY8gFBa3/bbtyGwj6/M85YtpoxgGDlQdNarkajSRSLT9kCQSidIrDKjOMNsJm3wK\nW3DRSVVPK7b3NspDMuk2SNx3X9iSGKGTS5OjcGA9rG7ZonrIIaoPPVR0FW3YkDgeUMqQOMwjCIWN\n2xx20iTV445TbW0tuoo2VqxY0e5LPmvWLF2xYkXk6gyznSiQT2HNWyfCrFnjzDjTp1te11qi2Mj/\nj+epU1X1u+UQLh+1rrCDB8PatXDHHWFLYlSSYnc6jQlIHsMHy5fD3XfDiy+GLUnlsfCrucnnrZOs\noByRwU9ax8wymekku1B69P/Bg51Hzpe+VFI1sSQVfvXhhx8GPos1ZQrrb2vifsCNQF/gc95tVdWv\nlNy4yCnAOFyGvDtV9dcZzys+JC46zGmBdzrCU0/Bf/yH611rNa9rLYdfLWlrInAXMAm3n7gBuBu4\nvwxCdQImAqcABwI/tGiMLgnzVVfB//xP7SqrkRs/CruDqv4frjf+p6o2AaeWoe0jgFdVtUVVN+O2\nPp5ehnpjzW9+A717w2k1vF0lPfxqIpGgsbGx3T7iWsaPA/unXm/4qohchgsivmMZ2u4FpO/efhMI\n3QelmDCnmWFMc4UkLcSqVXDDDZBI1HbGdAu/mhs/Cnsl8HngcmAU0A04vwxt+5rkNTU1tZ03NDQE\nPpfx48salL/riBFw9tkuP04t07t373YLTNUe2DyV28gPHYk40Q1AVcvybRWRo4AmVT3Fux4GtKYv\nPNWSHXbxYuc699JLsOuuYUtjhElJi04iMkBEFgOLgcUi8oKIHF4GueYB+4pIvYh0Bc6mRvPOqrqF\npuuvd8rqx/dz7ty5TJw4se164sSJzJ07N287YfiUZmtz7ty55ttaLLn2LOpn+3kXA8emXR8DLCr0\nnp8D+DawHOdnOyzL81K2ZAbil5qt3sz9xh3df/zII6oHHqi6ebO79rPRfcKECSoiOnbsWB07dqyK\niE6YMCFvO2FsoM/W5oQJE2pmI38xUKI/7MIs9xYUeq8cR6kKSwB+qTnrLdLD59NPVb/6VdWnn25/\n34/v59ixY9vKjB071pfsYfiUZmuzVnxbiyGfwvpZdJopIrcDD3jXZ3v3+nkatcBnZ25k4dZbXdZ0\nv0mYjRonlyanDlxImESuo9D7pRxUeQ/7zjuqu+6q+vLL7e/bkLi2oVb9YaM+h73oItWrr972vh/f\nzzlz5rRT0AkTJuicOXPyyh2GT2m2NufMmVMzvq3FkE9h/ewl3g34b6CXqp4iIgcCR6vq5NL79/xU\ns1lnwQI49VSXhLmjeV2N6qbUvcS/B54G9vCuXwGuKo9otYlq8UmYjdrGj8L2VNUpwFYAdft+t+R/\nxcjHQw/B+vVw4YXF11EOm2qhOirRhtFBco2V9bN5ZBLYFc+8AxyFy2YX+TlsNoKa1/plwwbVvfZS\nTSZLq6ccC0iF6qhEG8a2UKIdtj8wG/jY+/cV4NBC75XjCEJhCWjl2C8jR6o2NpanrnLYMgvVUYk2\njPbkU1g/cYnni8jxQB/v1nJ1w2Kjg7z5Jowb52IMG0ZR5NJknL/q7mnX5+P2+o4HeuR6r5wHVdbD\nnnuu6vDh5anLhsTVC8UMiYGFKcUEjgPeAc4EbgCm5nqvnEcQyhTWHHb2bNVevVTXrStPfeWwqRaq\noxJtGNuST2HzhTl9QVUP9c5vAz5QF22i3bMgqRY7bGsrHHWUi9NkSZiNQhRrh+0kIqngCSfhtiKm\n8LMH2fC47z4XQSLIJMzFuNtlEpYJxkw//smneA/gNvmvAjYAzQAisi+wpgKyVQXr18OwYfDHP8J2\nfqzeRfLcc89x+eWXs2WLM5EPGjSI8ePHc2QHksmGFV7Uwpp2gFxjZW8oejRwBrBj2r39gH753ivX\nQYVNLkFw7bWq551XmbaKcbfLJCwTjJl+PoNizTqq+myWey+X7+eiunnjDbj9dnjhhbAlMaqGXJoc\nhYOY97BnnlmeJMx+KMbdLpOwTDBm+mkPJTqwG0WQTMK8eXDvvZVpb8CAAYwfP57LLrsMgM6dOzNg\nwIAO1RFWeFELa+ofSzcZAFu3Qv/+MHw4NDaGLY0RN0p1rzM6yOTJ0K0bnHVW2JIY1YYpbJlZs8aF\nK7311tzR+ythd/RjlzX7Z/ywOWyZGTUKvvOd/BnTK2F39GOXNftnDMm1GhWFg5itEi9b5oKqvftu\n4bKVsDv6scua/TN6kGeV2IbEZaSWkzAbFSKXJkfhIEY97FNPqe6zj+rGjYXLVsLu6Mcua/bPaILZ\nYYMllYR5zBh/SZgrYXf0Y5c1+2f8MDtsGRg/Hp54AmbMqO28rkZ5yGeHNYUtkVWrXKqNRMLyuhrl\nIXIbJ0SkUUReFJGtqRw9cWXECPj+94NVVj/20swyU6dOZerUqXnfMeJHWHPYxTi3vdtDar8sLFkC\nDz/skjAHiR97aWaZH//4xwD07Nkz5ztGDMm1GlWJAxfFIqdvLRFeJW5tVT3xRNXx4yvTnh97aWYZ\ns7HGE8wOW34eewzefhsuuSRsSYxaIrAhsYg8A+yW5dG1qvq433qamprazhsaGmhoaChZtlLZuNFt\nkrjtNujSpXD5UmlubqaxsZFEwoXVSg1vU+aYbGW+973vAeR9x4gGyWSSZDLpq2yoq8QikgAGa46k\n0FFdJR49GmbNgsd9/+yUxsqVK2lpaWlTtubmZurr69vNRzPLpBaczvJchrK9Y0STyJp1PIW9WlWz\nxsKPosK+955bEZ49G/bbL2xpjGokcgorImfgMgj0xOXsWaiq385SLnIK++Mfwy67wC23hC2JUa1E\nzg6rqn9S1d6quoOq7pZNWaPIggVuR9N11/l/J6iUjXPnzrUYwrVIruXjKBxEyKzT2qp67LGqv/1t\nx94LKj/NhAkTLGBalUIp6SbDPKKksFOmqB56qOqWLR1/N6iUjRZDuDrJp7DmreODTz6BIUPgnnug\nU6ewpTFqmlyaHIWDiPSwI0eqnnVWce/akNjoKBSTvS4KRGGV+M034dBDXRLmYlxF/dhQi6mja9eu\nbNq0qaR6i6Ecn8fIT+TMOn6JgsKed55T1BtuCFUMo4aInFknLjz7rIvgP3Ro7jJRCVlq1Aa26JSD\n1la44gq48UbYaafc5aISstSoEXJNbqNwEOKi0913qx5xhOrWrYXLRiVkqVEdYGadjlGpJMyG0VFM\nYbNw001wwglw1FGFy/pxfSuViRMnMmjQIMaOHQu4IXHnzp3bIiIatYMpbAZvvAGTJvlPwhyVkKVG\nbWBmnQwaG+GQQzq2wd8wykk+s471sGnMnAnPPee2IBpGFLElFY+tW50Z5+abYYcdiq8nCLtslFza\noiRLTZJr+TgKBxUw63Svq/PMJT9RmKm77FRXUn1B7LWN0v7dKMlSrWDudbkBdA3d9Eu8o/M5TMvR\nZhB22Si5tEVJlmokn8LakBgYxXWcxhP0Y2HYohhGfnJpchQOKtLD7qu78oG+yxdVvV6jFGxIbJQK\n5l6Xm66dp7N5618BF1Wte10dH61dW3R9QbifRcmlLUqyVCvmXpeDGTPgsstcjpzttw+sGcPoEOZe\nl4VUEuZbbjFlNeJDzSrspEnQqxd897thSxIehWyqZnONHjWpsB9+CKNGwdixtZ0xPeXLm8rtMnDg\nQFpaWnw/NypPTc5hU04uaUEcapZkMskJJ5wAuMRZmcnGCj03yo/tJU5jyRJ46KHgkzAbRiDksvdE\n4aDMdtjWVtWTTlK99dayVhtbCtlUzeYaDpgd1vHYYy6g2gsvVCava9QpZFM1m2s4RM4OKyKjgdOA\nTcBrwIWq+nGWcmVT2I0b4aCD3Lz1W98qS5WGEQhRtMM+DfRV1UOBl4FhQTc4fjz06WPKasSb0IfE\nXq7YM1X1vCzPytLDWhJmI05EsYdN5yLgySAb+OUv4fzzTVmN+BOYWUdEngF2y/LoWlV93CszHNik\nqn/IVU9TU1PbeUNDQ4ftgAsXwuOPw7JlHXrNMCpGamOKH0IbEovIBcBPgBNV9dMcZUoaEqvC8ce7\n/Dg//WnR1RhGRYncxgkROQUYAhyfS1nLwcMPw9q1cPHFQbVgGJUlLLPOK0BX4CPv1rOqemmWckX3\nsJ98AgccAL//PdhuOiNORK6HVdV9g25jzBg4/HBTVqO6iMIqcdl56y3niTN6dOXbNpc0I0iqUmGH\nDoVLLoEvf7nybZtLmhEkoW+cyEcxc9g5c+DMM2H58vx5XYPEXNKMUoj6xomykUrCfNNN4SmrYQRJ\nVSns/fe7f8/bZpNj5UhPP5lIJGhsbGw3pzWMUqiaIfH69bD//s72evTRAQuWB3NJM0olcu51fumI\nwv7yl9DSAvfdF6xMhhE0Va+wLS3Qv79zTN9zz+DlMowgqfpFp//8T7jySlNWo/qJfRC2mTNh7ly4\n++6wJTGM4Il1D7t1q+tZS03CbBhxIdYK+7vfOXvr978ftiSGURliu+j08cfOjPPnP0O/fhUWzDAC\npCpXiYcMgY8+gsmTKyyUYQRM1SnsK6+4zRFLlsBu2YLQGEaMqTqzzuDBzpRjymrUGrEz6zz9NCxd\n6rYgGkatEasedssWl4R5zBhLwmzUJrFS2EmTYPfdP0vC7Dc0ZCUxmQoTNXkgmjJlIzYK++GHMHIk\njBv3WRLmKP6RTabCRE0eiKZM2YiNwjY1QWOjS2hlGLVKLBadXnwRpkxxi02GUctE3g4btgyGEQax\n3DhhGEZ7YjOHNQzDFNYwYkXsFVZERovISyLygohME5GdIyBTo4i8KCJbRSQ0XyIROUVElonIKyJy\nTVhypMnzOxF5T0QWhy1LChHpLSIJ7/9riYhcHrZM+Yi9wgJPA31V9VDgZWBYyPIALAbOAGaFJYCI\ndAImAqfj1C2/AAAET0lEQVQABwI/FJEDwpLH4y5PniixGbhKVfsCRwG/iMDfKSexV1hVfUZVW73L\nuUDokZ1UdZmqvhyyGEcAr6pqi6puBh4ETg9TIFVtBlaHKUMmqvquqj7vna8HXgL2CFeq3MReYTO4\nCHgybCEiQi8gPQvXm949IwciUg8chvvhjySx2DghIs8A2ZzprlXVx70yw4FNqvqHqMgUMmav6wAi\nshMwFbjC62kjSSwUVlVPzvdcRC4A/h04sSICUVimCPAWkJ5uoDeulzUyEJEuwB+B+1T1kbDlyUfs\nh8QicgowBDhdVT8NW54sZN2xUgHmAfuKSL2IdAXOBh4LSZbIIiICTAaWquq4sOUpROwVFpgA7AQ8\nIyILReR/wxZIRM4QkZW4Vcc/i8j0SsugqluAy4AZwFJgiqq+VGk50hGRB4DZwH4islJELgxTHo9v\nAOcBJ3jfn4VeJxBJbGuiYcSIauhhDaNmMIU1jBhhCmsYMcIU1jBihCmsYcQIU1jDiBGmsBFHRIZ7\nbl8veDbCI8pcf4OIbLOVMtf9MrR3ero3jIgkRaR/udupVmKxNbFWEZGjgVOBw1R1s4j0AOIeQv0M\n4HGcVwzYnucOYT1stNkNWOW5x6GqH6nqOwAi0t/rneaJyFMispt3Pyki47zeeLGIDPDuHyEis0Vk\ngYj8XUT28yuEiOzoOZ/P9d7/rnf/Ai9owHQReVlEfp32zsUistx757ciMsH7AfoOMNqr5yte8Uav\n3HIROaYcf7iqRVXtiOgB7AgsBJYDtwHHefe74Lb47epdnw1M9s4TwO3e+bHAYu+8DujknZ8ETPXO\nG4DHs7Tddh+4ETjXO9/Fk+fzwAXAa17d2wMtOBe+PYA3vLKdcY7847337wIGprWTAEZ7598Gngn7\n7x7lw4bEEUZV/+XN744FTgCmiMhQYD7QF/g/t3edTsDbaa8+4L3fLCLdRKQbsDNwj4jsgxuGdumA\nKP8GfEdErvautwf28ur5i6quAxCRpUA98AVgpqqu8e4/DKT36JkOEdO8fxd47xs5MIWNOOqiacwE\nZnqxkM7HKeyLqvr1DlQ1CqdcZ4jI3kCyg6IMVNVX0m+IyJHAxrRbW3Hfqcx5aaaCZj5P1ZF638iB\nzWEjjIjsJyL7pt06DDfsXA58QUSO8sp1EZED08qd7d0/BlijqmuBbnzWC3fUS2YG0BacTEQOS51m\nKavAc8DxIrKLiHQGzuQzJV3nyWIUgSlstNkJ+L0X0e8FYH+gSd0i1FnAr0Xkedw89+i09z4VkQXA\n/wIXe/duBm7y7neifS+XbaVW0+6PArqIyCIRWQL8V5Yyn72o+jZu3vsP4G+4+ezH3uMHgSEiMj9t\n0SmzXSMH5l5XZYhIAhisqgtClmNHbw7eGTdHnayqj4YpUzVgPawRFE0ishAX8vV1U9byYD2sYcQI\n62ENI0aYwhpGjDCFNYwYYQprGDHCFNYwYoQprGHEiP8HZQ6N0htG490AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x106eea7d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_min, x_max = X_train[:, 0].min() - .5, X_train[:, 0].max() + .5\n",
    "y_min, y_max = X_train[:, 1].min() - .5, X_train[:, 1].max() + .5\n",
    "xs = np.arange(x_min, x_max, 0.5)\n",
    "fig,axes = plt.subplots()\n",
    "axes.set_aspect('equal')\n",
    "axes.set_title('Setosa classification')\n",
    "axes.set_xlabel('Sepal length')\n",
    "axes.set_ylabel('Sepal width')\n",
    "axes.set_xlim(x_min, x_max)\n",
    "axes.set_ylim(y_min, y_max)\n",
    "plt.sca(axes)\n",
    "plt.scatter(X_train[:, 0][y_train == 0], X_train[:, 1][y_train == 0], c='red', marker='s')\n",
    "plt.scatter(X_train[:, 0][y_train == 1], X_train[:, 1][y_train == 1], c='black', marker='x')\n",
    "ys = (-clf.intercept_[0]- xs * clf.coef_[0, 0]) / clf.coef_[0, 1]\n",
    "plt.plot(xs, ys, hold=True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The blue line is our decision boundary. Every time $30.97 \\times sepal\\_length - 17.82 \\times sepal\\_width - 17.34$ is greater than zero we will have an iris setosa (class 0). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, the really useful part: when we have a new flower, we just have to get its petal width and length and call the `predict` method of the classifier on the new instance. _This works the same way no matter the classifier we are using or the method we used to build it_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "If the flower has 4.7 petal width and 3.1 petal length is a ['setosa']\n"
     ]
    }
   ],
   "source": [
    "print ('If the flower has 4.7 petal width and 3.1 petal length is a {}'.format(\n",
    "        iris.target_names[clf.predict(scaler.transform([[4.7, 3.1]]))]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we first scaled the new instance, then applyied the `predict` method, and used the result to lookup into the iris target names arrays. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Back to the original three-class problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, do the training using the three original classes. Using scikit-learn this is simple: we do exactly the same procedure, using the original three target classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n"
     ]
    }
   ],
   "source": [
    "clf2 = linear_model.SGDClassifier(loss='log', random_state=33)\n",
    "clf2.fit(X_train, y_train) \n",
    "print (len(clf2.coef_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have know _three_ decision curves... scikit-learn has simply converted the problem into three one-versus-all binary classifiers. Note that Class 0 is linearly separable, while Class 1 and Class 2 are not"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmQAAAD6CAYAAAAP4lumAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXl4FEX6xz+FiIIIGg7xiLKirrrueovuLhrUEHTXg2B0\ndz3QVdEVlEtFRQUh6KIccnkfICquHCrqz4xXIvEAdcUTD0RjosuiIt6AC3l/f3QnzEzm7u7pnpn3\n8zz9JD3dXfV2d32naqreesuICIqiKIqiKIp/tPLbAEVRFEVRlEJHG2SKoiiKoig+ow0yRVEURVEU\nn9EGmaIoiqIois9og0xRFEVRFMVntEGmKIqiKIriMznZIDPGjDHGzPHbDsUbjDGNxpjd/bYjKORj\neTfG7GqM+cEYYxymo2UljEJ8HkHShzHmXWPMkRlem7Im3NJPvmGMOdsYU+u3HZkS2AaZMeZvxpjX\n7UL3H2PM/xlj/mAfDnzwNGNMiTGmwW87go4xpsYYc67fdiTDGDPLGDPOw/Rzuryni4jUi8i2ooEQ\nM6aQtJMr+hCR/URkcYbXpqwJ1Q8YY7rbP0AC245pItUfSoG8EWPMcGAKUAl0BYqBmcAJTaf4ZFrB\nYIxpnaWssvaFksV7Sot8Le/GmC38tiEeXtqm2nE977zQR5D14DZZvtesvH8X7im5nSISqA3oCPwA\n9E9wzhhgTtj+PGAV8C3wArBv2LHjgfeA74HPgRH2552BJ4C1wBpgMWDi5DcFWA18B7wN/Mb+fCtg\nIvAZ8F/gVmBrYBtgHbDJvpfvgW72+TcDX9jbFKBNPHvC8r8C+NhO5z3g5Dh27gT8DGwf9tmBwFfA\nFvb+34HlwDdAFbBr2LmNwEXACmBlnHvf1/68Bjg37NqzgVr7fxPvmUXZOx7YaD+rH4BpYXZcAHxk\nP48ZUdfFvYeo87rbaf3dfkc1KTyDFnYDA4FfgA22nY/la3nHKsM3RX32GDAsrIwtAL4EPgEujrJz\nPjDHfn5/Bw4DXrf3/wtMino3rez9IuBeLF18AzwSlu75WGVyjW3LjlFldvewZ3mfbVsdMKrpHrHK\n50vAZOBrYKxqJ/jaIXf0MdT+vw44OoEefmWn/T3wDFbDck4cTdQAY4EX7fNDQKd09ANsb9/Xl/bn\njwM7x3mOI4F5UZ9NBaaGvYu7gf/Yz25cWP5nE6UvYA/7+X+LpaOHYtkerYl418Wwt95Op6mOPRwY\nANQCN9n3+wnQN6o8xbyHOOUq+v0legbRds+1P19s2/mjbWtF3LKcjjiysQF9gf/Fe0hxBHg2ViNo\nS6wvhWVhx1YBfwh7GQfa/9+AJa4t7O0PcfIqw6pQOtj7vwa6hX0BPQpsB7QHFgHX28eOAhqi0hoL\nvIwl/s52AR6bzB7glLA8T7VfbLc49j4HnBe2fxNwi/3/SVgVxq+xekdHAS+FnduIJfrtsBqPie69\nGvh71DuoTfbMYtgbkU6YHYuADli/hr8EylK5h6h0uttpzQLaYjWW416f5H7vJaoSz9Py3guoD9vf\nHquh0s1+Xv8GrgZaY1UuK4E+YXb+Apxo728NvAKcbu+3A3pGvZumL7Mngbm2za2BXvbnR2N9uR0A\ntAGmAS9ElZWmBtl9wCP2s9kN+LCpbNnP7H/AIPs+tlbtBF875JA+7P1PiWyQxdLDjVhl/A9YFf19\ncTRRYz/vPexrq4Eb0tRPEdDPvr498DBhP3ai7m1X4Cegvb2/BVbD4zB7/xH7GbUFugBLgYHx9GXb\nc6V9vA3w+1i2R5fleNfFsHe3GOmcbT/zc7F+3FwIfBF2PO49xClX0e8v0TOIazdh31MJy3smIvFy\nA04HViU5ZwxhAow6tp1989va+59h/UrrEHXedViNqR5J8uqN9cXeM+rFG6yG0e5hnx0BfGL/X0LL\nBtnHRLbW+wCfpmOPfe6ypkIS49i5wHNhNtYDf7T3nyKyImiFJcDisEJTkuzeowUUJoSmSuXoeNfF\nsLeasN6CMDvCC/O/gMtTuYeodLrbaXUP+yze9bsmud97gXEFUN6NnUbTF/r5wLP2/z2Bz6LOvxK4\nJ8zOmqjjL9ifd47zbloBO2L1JneMYc/dwD/D9rfB+pLcNays7I5VeWwA9g47dyBQHVY+P0ty76qd\ngGmHHNKHvR/dIKsJO7YrVqNl67DP5hC/h6wauCrs3H8AT6Wjnxj2HwB8k+B4LXCm/X8p8LH9/w7A\n+ijb/wo8H09fwGzgdqJ65KLvM1oT8a5LUEajG2Qrwvbb2ed0TXYPccpV+PtL9gzi2k2KDbIg+pCt\nATqn6qhnjNnCGPNPY8zHxpjvsAQhWD1QAP2xuqnrbCfYw+3Pb8JqID1tjFlpjBkZK30RqQZmYHUt\nrzbG3G6M2RarddwO+LcxZq0xZi3WF1bnWOnY7IQl5ibq7c8S2mOMOcsYsywsn/2ATnHyWAgcYYzp\nBhwJNIrIi/ax3YCpYemssT/fOez65okICe49ISLyfJrXSYzP/hv2/89Yv+5SvYdowidXxLt+p0zv\n1yFBK+8CPIT1RQPwN+AB+//dgJ2anp39/K7E+rJr4vOoJM8F9gLeN8a8aoz5U4xsi7Eqie9iHNuR\nMM2IyE9Yzyz6fXfG6hGJ1lfMsh0H1U5L/NZOLukjFuF62AmrnK8P+yxZmQx/l+vY/C7DiasfY0w7\n+13U2c/jBaBjgtmZDxJf+1sCq8Le/21Y9WC8e7kcqwH7qrFmn54T9y7dua6J5mcmIj/b/7ZP8R6i\nCX9/ya53ancgG2SvYP3S7Zfi+X8DTgSOEZGOWMMoxt4QkddF5GSsh/YoVpctIvKjiFwqIj3s64cb\nY46OlYGITBeRQ4B9sSqXy7CGUdZh+Sdsb2/biUiHpstiJPUfrFZ9E7van8Wzp7cxZjfgDqyu4CIR\n2R54lzgOgiKyFngaOM1+NnPDDtdjda9uH7ZtIyJLwpNI4d7B+mW8Tdip3VK8roXJcT6PRyr3kCiP\nhNcnsDtdO1MlcOUdq8ycYpe9w7B8xsB6dp9GPbsOIvJn+7jQsvx8LCJ/E5EuwARgvjGmbVR+DUCR\nMaZjDFsiNGOM2Qbrx8gXUed9jdX70D3ss12J/EJN+A5VO0nz8EM7uaSPWITf+yqsch5e/ndN8b4S\nkUg/I7DexWH28ziKsOcRg/lAiTFmZ+BkrAZaUx4bsHzYmt59RxH5bdi10eV/tYgMFJGdsfwab7Fn\nGv5kn9Iu7PRuKVwXTbrlKpV7iE4/PI+E16dhd1wC1yCzW/nXAjONMSfZLfwtjTHHGWMmxLikPdZD\n+sb+sr6+6YB93enGmI4i0uRgv8k+9mdjzB72L4Xv7c83RSdujDnEGNPTGLMl1q/N9cAm+5fSncDN\nxpgu9rk7G2P62JeuBjoZYzqEJTcXuNoY09kY09m+zzkJ7GnE+uIWrAqnld3q3i/JY3wQy7mxP5sF\nBVZr/ipjzL52nh2NMRXxEol37/bhN4FyY0xbY8weWD0hksJ10awGeiS5n/AvkLTuIQZxr09i92qs\noTFXCVp5t216E6u83QVUicj39qFXgR+MMZfb730LY8x+xphDmkyITssYc0aTPrD8ZQSrXIfntwqr\nd/kWY8x29n00xXKaC5xjjNnfGLOVfb9LRKQ+Ko1NWJXreGNMe7uyHAbcH+seE6DaiU/WtZNj+kh2\nL59h+dmNsW05AvgziRsWSWfmxdFPL/twe6yOg++MMUXA6CRpfYXluzYLy/3mw7A8ngYmG2O2Nca0\nMsb0MAlirhljKowxu9i739r32Wjn8QVwpv0d8nfCynG862Jk8ZX9eTINNN1buvcQ8eyTXZ/E7lS0\nGjwfsqYN65fOa1h+WquwZoccbh8bzWZHyG2wful8j9U9fSaWkHbH6l58Cmu2xXdYDnhNjoVD7fN/\nxGr5jopjx9HAW1ji/QqrAdXOPrYV1mynlXb6y4HBYdfejSXcb9g8y3Iq1q/+/2DNuGyTzB6s6d5r\n7PwnEcOZN8rmre3n8U6MY2dgzYD6DusX711hxzYR6ROX6N47YTkxf4/ldzAae2Zoouti2HM4lu/J\nN8DNEmO8nSin4ET3EJV2d/ueon1aYl6f5H73wPLdWwsszNfyHmbP1Xa6/aM+3xGrobLKzudlNvvM\nNNsZdv4crC+jH4B32OwgG/FusJyjZ2ENN3wDzA9L4wKs4aQ1WA7rO8Uqs1j+QnOwHNnr7XtommU5\ngLCZy6qd3NIOuaOPT0msh93ZPMvyWSyfo6ZnGPHMaelr2FyGY5wbUz9Yeq2238sHWP5zLd5rjHfc\niD0DNezzDsAt9vP5FngDODWevrB6xD+38/6YyAkzfbFmQK7FilTQfK+Jroth63VsnkHaM44d4d8R\nce8hRtqx3l+iZ5Dofi/AqvPXAqfEu5+mLytFURRFUbKIMeZfwHIRuc5vWxT/CdyQpaIoiqLkI/bQ\nbg97uOs4LH+1R/22SwkGgYxcriiKoih5SDes2bydsIa9LhSRt/w1SQkKOmSpKIqiKIriM4HuITPG\naGtRCRQi4uu6eaoJJWioJhQlkkw1EXgfsnizEUaPHu3qLLd0Nz/zD/q9b9woHHigcP/9+XX/QSFo\nzyUI+RfyvfuZf1AI2nMJQv6FfO/x8v/f/4QePYQXXvAuXycEvkGm5B633w7bbgt/+5vfliiKoiiK\nxdy5sMsucGTc6Gn+EughSyX3+OorGDMGnnsO4i7OoSiKoihZZNMmqKyEW2/125L45GwPWUlJScHm\nH+R7v+oqq2fst/EWo/A4/0LG7+eimijc/IOK389FNRGc/P/1L+jSBXr39seeVAj0LEtjjATZPiWS\nV1+Fk0+G99+HjrFWVctxjDFIAByYVRNKUFBNKLnApk1WJ8HUqVBa6m1eTjSRsz1kSrDYtAkGDYJ/\n/jM/G2OKoihKbrJggVUvHXus35YkxjcfMmPM1sALWOs7tgEeE5Er/bJHccbdd8NWW8GZZ/ptSe6i\nmlCUSFQTilMaG2HcOLjxxuD7NfvWIBOR9caY3iLyszGmNfCiMeaPIvKiXzYpmbFmDVxzDYRCwS/w\nQUY1oSiRqCYUpzzyCGy9NfTt67clyfF1lqWI/Gz/2wbYAmvFdiXHuPpqqKiAAw7w25LcRzWhKJGo\nJpRMaWyEsWOt2ZW50Fnga4PMGNMKeAPoAdwqIsv9tEdJn3//2/oF8v77fluSH6gmFCUS1YSSKY8/\nDltsAX/+s9+WpIbfPWSNwAHGmI5AyBhTIiI14eeMGTOm+f+SkhLfp9Iqm2lstBz5r78ett/eb2vc\np6amhpqamqzmqZpQgoxqQskVRKzesWuv9bZ3zE1NBCbshTHmGmCdiEwM+0ynMweYe+6BO+6Al1+G\nVgUwXzfbU/xVE0rQUU0oQeWJJ2DUKFi2LLv1U06GvTDGdDbGbGf/3xYoBZb5ZU9QCYVC9O/Th/59\n+hAKhfw2p5m1a60gsDNnFkZjLBuoJpLT0NBAbW1t835tbS0NDQ0+WqR4iWoiOaqJljT1jl1zTW7V\nT34OWe4IzLb9A1oBc0TkOR/tCRyhUIgB/foxYd06AAa8+CKzH3mEsrIyny2zCnq/fnDwwX5bkleo\nJpJQV1dHeXk58+bNA6CiooKFCxdSXFzss2WKR6gmkqCaaEkoBD//DOXlfluSHoEZsoxFoXdF9+/T\nhxOfeYYB9v5sYFFpKQueftpPs3jzTSgrg+XLoVMnX03JKhqVPBjU1NTQ217/pLq6Wv2FfEQ1EQxU\nE5sRgd//HoYOhdNOy37+OTlkqeQmIjB4sBVor5AaY4qiKErwefZZ+PZbOOUUvy1JH22QBZiBI0Yw\nsm1bZmP1jo1s25aBI0b4atOcObBhA5x7rq9mKAVKbW0tFRUVVFdXU11dTUVFRYT/jKIUGqqJzYjA\ndddZsTG32MJva9JHhywDTigU4o5JkwCrgean/9h338E++8Cjj8Jhh/lmhm/o8Iz/NDQ0UFdXR69e\nvQCrMurevXtB+8v4iWrCf1QTm6muhgsusNxpWvvkIe9EE9ogU1Jm6FD46Se4806/LfEHrXwUJRLV\nhBIkeveGc86Bs87yzwYnmvA1MKySO7zzDjz4oPXLQ1EURVGCxOLF0NAAf/ub35ZkjvqQKUlpcuQf\nMwY6d/bbGkVRFEWJZOxYKzamX0OVbqANMiUpc+fCDz9YY/OKoiiKEiReeglWroQzz/TbEmfkcFtS\nyQbffw+XXQbz5+fmrBVFURQlvxk3zuod23JLvy1xhjr1Kwm59FJYswbuvddvS/xHHZgVJRLVhOI3\nS5fCqafCihXQpo3f1qhTv+IRy5fD7Nnw3nt+W6IoiqIoLRk7Fq64IhiNMadog0yJSZMj/7XXQteu\nflujKIqiKJG8/jq89RYsXOi3Je6gTv0BIBQK0b9PH/r36UMoFPLbHADmzbOGKv/xD78tUQqNhoaG\niEjjtbW1NDQ0+GiRoviLaiI248bByJGw1VZ+W+IO2iDzmVAoxIB+/TjxmWeshcT79fO9UfbjjzBi\nBMyYkdtTiJXcpK6ujvLycmpqaqipqaG8vJy6ujq/zVIU31BNtGTZMquH7Pzz/bbEPdSp32f69+lj\nNcTs/dnAotJSFjz9tG82XXEFfPGFtW6lshl1YM4eNTU19O7dG4Dq6mpKSkr8NUiJiWoie6gmIikv\nhyOPtFaQCRLq1K+4xocfwl13WZH5FUVRFCVovP02vPwy3H+/35a4iw5Z+szAESMY2bYts7F6x0a2\nbcvAESN8sUUELr4YRo2CHXf0xQRFoba2loqKCqqrq6murqaioiLCf0ZRCg3VRCSVlVZIpnbt/LbE\nXXTIMgCEQiHumDQJsBpoZWVlvtixYAGMHm2Nzed6gD0v0OGZ7NDQ0EBdXR29evUCrMqoe/fuFBcX\n+2yZEo1qIjuoJjbz3ntw9NHwySewzTZ+W9MSJ5rQBplHBKWRlSo//QT77mvFHStw14S4aOWTOVqh\n5CeqicxRTWTGX/8KBxxgza4MIk40oUOWHhDEmZPJuOEG+P3vtTGmeIPOElOUSFQT6fPBB/Dcc3DR\nRX5b4g3aQ+YBQZw5mYgVK+CII6wAezvv7Lc1wUV7A5yhs8TyD9WEM1QT6XHmmbD33pafc1DRWZZK\nxojAkCFW9682xhRFUZQg8tFHUFUFM2f6bYl3aIPMAwaOGMGAF1+EdesAa+bkbJ9mTiZj0SL49FN4\n9FG/LVHymfBZYgAVFRUsXLiw2X9GUQoN1UR6XH+9FQWgQwe/LfEOHbL0iFxw6l+3znLkv+suOOYY\nv60JPjo8kznqwJyfqCYyRzWROitXQs+e8PHHsN12fluTmJycZWmMKQbuA7oCAtwhItOizslJoeUK\no0fD++/Dww/7bUlu4HXlo5pQcg3VhJINzjsPdtoJxo7125Lk5Oosy/8Bw0TkN8DhwCBjzD4+2pMT\nuLUQ+SefWGtV2p14SjBQTaSJLrqc96gm0iTfNFFXB488ErwlkrzAtwaZiPxXRN60//8ReB/YyS97\ncgE3w2kMHWpFOtbe8eCgmkgfDR2Q36gm0iffNHHDDXDhhVBU5Lcl3hMIHzJjTHfgBeA3tuiaPteu\n6DDcCqfx5JMwbJi1XuVWW7luZt6STX8Z1UTqaOgA/1BNBJN80UR9vRUE9qOPoHNnv61JjZwOe2GM\naQ/MB4aEi6yJMWPGNP9fUlKSswUrKKxfb4W5mDlTG2PJaPqFmW1UE0pQUU0o2WTCBDj//GA3xtzU\nhK89ZMaYLYEngKdE5OYYx/WXTxhNQ5YTwsNpPPJIWjM4x42z1qpcuNArK/OXbPQGqCbSo7a2lvLy\ncubNmwdo6IBso5oIHvmiic8/h9/9zorO37Wr39akTq7OsjRYo25rRGRYnHNUaFE4CadRVweHHAL/\n/jfstptHBuYxWZhRpppIEw0d4C+qieCRL5q45BJo0wYmTvTbkvTI1QbZH4HFwNtY05kBrhSRqrBz\nVGgu0q8fHHwwXH2135bkJlmofFQTSk6hmlC8YNUq+M1vYPly6NbNb2vSIyfDXojIiyLSSkQOEJED\n7a0q+ZW5z/jx49mjUyf26NSJ8ePHZyXPqirLif/SS7OSnZIBhaqJpUuXMmPGjOb9GTNmsHTpUh8t\nUoKCasKi0DRx001w1lm51xhziu9O/YXG+PHjufHqq2mKbHiJ3V01ysPVUjdssLp/p06Frbf2LBtF\nyYjXXnuNSy65hI0bNwIwfPhwpk2bRs+ePX22TFH8oZA1sXo1zJoF777rtyXZJxBhL+KRj13Re3Tq\nxDXffBMRumJcUREfr1njWZ433ACvvGKtW6lkji4T4x0333wzw4ZZLkJTpkxhaCFEgcwDVBPeUaia\nuOwyKxrA9Ol+W5IZOR32QvGW+norGv+rr/ptiaIoiqLE56uv4O674e23/bbEH7RBlmXOGT68eZgS\n4BLg8uHDPctvxAgYPBh2392zLBTFETNmzGD48OFMmTIFsIZnWrduzeDBg322TFH8oVA1MXkynHYa\n7LKL35b4gzbIskyTr9i4yZMBqzHmlf/Ys89aIS7uu8+T5BXFFQ499FCmTZvWXNm0bt2aQw891Ger\nFMU/ClETa9bAHXdYcTILFhEJ7GaZlxsMGDBAurZuLV1bt5YBAwb4bY5s2CDy61+LPPaY35Z4T319\nvSxevLh5f/HixVJfX5/0WLrY5VE1kQKPPfaYDBkypHl/yJAh8lghFMaAoJoIHqqJxIwaJXL++d7m\nUVVVJaWl5VJaWi5VVVUpH0sHJ5rwvdGV0LgcEdqAAQOkA8gse+sAvjfKbrxR5LjjRBobfTUjKyxe\nvFg6d+4s1dXVUl1dLZ07d26ucBIdSxetfFJnyJAhAsigQYNk0KBBAkRURoq3qCaCh2oiPt98I1JU\nJPLJJ97lUVVVJW3b7iAwS2CWtG27Q3PDK9GxdNEGmc90bd1aZoH1OO1GWdfWrX2z5/PPRTp1Elmx\nwjcTsk51dbVgBY6U6urqlI+lg1Y+6dFU6TRVQkp2UU0ED9VEbEaPFjnnHG/zKC0ttxtcTVX1LCkt\nLU96LF2caEJ9yPKQSy+FCy+EPfbw2xJFURRFic+338KMGVBAcW/jk2lLLhsbOfLLJ0hDls8/L7Lr\nriI//eRL9r6gwzPBQ4dn/EU1ETxUE7EZO1bkrLO8z0eHLAtEaCLBcOr/5ReRffcVWbDAl+x9Qx2Y\ng4c6MPuLaiJ4qCZa8t13Ip07i3zwQXbyC7pTv0bqzyMmT4ZQyFq30vgaOzs/0ajkihKJakJxwg03\nWEskPfCA35a4R04uLp5vhEIh+vfpQ/8+fQiFQgmPjx8/PuG5mbBqFVx/PUybpo0xxX8aGhqora1t\n3q+traWhoSHmsfnz5zN//vyY5ypKvqCaiOTHH2HKFAiLk65k2rWWjY0c6YquqqqSHdq2bfYh26Ft\n24guz/DjI2wfs3jnZsoZZ4iMHOk4GSUB6PBMyqTjw9SxY0fp2LGjKz5NSnZRTaSOaiKSCRNETjvN\nbyvcx4kmfG90JTQuR4RWXlraIuxFeWlpzOPl9vF452bC4sUiu+wi8sMPTu9ESYRWPumRTtgFt8Iw\nKNlFNZEeqgmLH38U2WEHkXfe8dsS93GiCR2yzHE2boRBg6wFxNu399saRVEURUnM7bfDH/8I++3n\ntyUBI9OWXDY2cuSXj59DllOnihx9dGFE5PcbtDcgZXR4pjBQTaSOasLip59EunUTefNNvy3xBiea\n0FmWLhEKhbhj0iQABo4YQVlZWdzjBx11FG+88ELcc1Nl9WrrF8YLL8C++zowXkkJnVGWOg0NDdTV\n1dGrVy/Ackru3r07xcXFLY41OS+fcsopLc5Vgo1qInVUExZTp0JNDTzyiN+WeIMTTWiDLIc55xzo\n1AkmTvTbksJAKx9FiUQ1oaTD+vXQowc8/jgcdJDf1niDhr1wQLJwFW5f5xavvAJPPw2jR2c9a1dI\nNAVc8Y9M34u+T+foMwwmqgn3uPtuqyGWTmMsFArRp09/+vTp70tdm1UyHevMxobHvgHJfL/cvs4t\nNm4UOfBAkQceyFqWruPm8i3ZggLwl8n0veTi+wwaufgMVRPuX5evrF9vRQN49dXUr3FzSaNs4UQT\nqRT2rYHTgVHAaHu7NtMM0zLOY6ElC1fh9nVuMXOmyFFH5b4jf65N624SWj5rQiTz95Jr7zOI5Noz\nVE14c10+cuutIscdl941paXldmOsqbqdJaWl5d4Y6BJOGmStU+hEewz4Fvg3sD6F81PGGHMP8Cfg\nSxH5rZtp5ytffQVjxsDzz2tEfh9RTShKJJ5oQvWQH/zyi7VM0kMP+W1JwEnWYgPezbS1l0LavYAD\ngXfiHHe15RpNLg5ZnnuuyNChWcnKU3KxO5/NvQF5qwkdnvGPXHyGXmsimR5ENZET3HmnSCaDSDpk\n2bKw3wH8LtMMUki/u1+Vj4j1wstLS6W8tDStF53pdU5YulRkxx1Fvv02K9l5Sn19fcSX0+LFi6W+\nvt5Hi5ITVvnkrSYyfS+5+D6DRi4+w2xoIpEeRDUReH75ReRXvxJ58cXMrq+qqpLS0nIpLS0PfGNM\nxFmDLG7YC2PMO/a/WwB7Ap8CGzZ3rMnvUuqCS4IxpjvwuMTojtbpzJvZtAkOPxwuvhjOOstvawoT\nY40Rv4tqQlGA7GgikR7s46qJAHPvvXD//fDcc35bkh28Cntxgr0dhyW0PmGfnZBJZvnE+PHj2aNT\nJ/bo1Inx48c7CoORyrV33w1bbQVnnumG9YoDVBMxWLp0KTNmzGjeHzduHLfddlvzfjpT/jVcQM6h\nmoiBasJa2m/8eLj2Wr8tyQ3iOvWLSB2AMWaOiEQ0A4wxc4CsNA3GjBnT/H9JSQklJSXZyDYh48eP\n58arr2aavX/J1VdjWrdm6saNAAx48UVmP/JIShH4Q6EQA/r1Y8K6dXGvXbMGrrnGijumjvzZo6am\nhpqamojPRKRONdGS1157jUsuuYSNtgZGjx5Nu3bt2HvvvQGoqKhg4cKFKUUar6uro7y8nHnz5qV9\nreItqonUUU3A3Lmw885w1FF+W+IdsTSRMcnGNIFlUfutgeWZjpHGSL87PvqQZUKPoqIWYS8OzzAM\nRiohNC7AQeH1AAAgAElEQVS4QGTwYC/uREkHNvvLqCZiMGXKlOYp/lOmTHE05V/DBeQG2dBEIj2I\naiKwbNwostdeIs8957cl2aVJE5lscYcsjTFXGWN+AH5rjPmhaQO+BBZl0viLkcdc4GVgL2NMgzHm\nHDfSzSdefx0efRTGjfPbEgVANaEokXipCdVD7vKvf0GXLtC7t9+W5BDJWmzAPzNt7TndCOgvn8rK\nSulg92bNAukA0rF164zCYCQKobFpk0jPniJ33+3l3SipwubeANVEFNOnTxdjjEyZMkWmTJkixhjZ\nZpttMpryr+ECcgfVRHwKWRMbN4rss49IKOS3JdkHBz1kcX3IjDFNq03NC/s/vCH3hhsNwlxk1KhR\nAIybPBmAy4cP55BDDuGOSZMAmD1iREr+YwBlZWXMfuSRmNfOmmWdc/bZ7tmuOMPWgmoiikMPPZRp\n06YxePBgAH744Qe6dOnS7MuzcOFCunfvnlJa3bt3Z+HChfTq1Svta5Xso5qITSFrYsEC6NABSkv9\ntiS3SDTLcjIwCZgJLAXutLel9md5QaIZjl7NnIyeoVlWVsaCp59mwdNPNzfG1q6Fq66CmTOhVQpL\nwCebhZOrs3QCSF5rIlE5cVKGVq9ezdKlS2NeGz0b7bHHHqNNmzbN+7169crYeTkUCnFi/76c2L9v\nCx2qJlxDNRHjWDL80EQq9jpdzLuxEcaOhdGjdRJa2iTrQgMWAr8N298PWJBpl1w6Gz5G6k90LNaQ\n5TZhQ5bbtWkjXdq0SfnaysrKFrYNGiRy4YWp30uyLu1c6/IOImwenslbTSQqJ4mOJRue6dixo3Ts\n2DHla6dPn+74XqqqqqTTDu1l8KxiGTyrWDrt0D7ClUA14RzVRG5pIlmZdyMy/vz5IocckvtrLWcK\nDoYsUynsLWbKxPrMi81roSWa4ZjoWKxZlruE7R9uf5bqtT2KiiLsWrZMpGtXkTVr0rufZLNwcm2W\nTtAIq3zyVhMiictJomPJZpSlc60bnFBeJoNnFcsC2V8WyP4yeFaxnFBelvL9KMlRTeSWJpLZ63Qx\n702bRH73O5FFi1wzN+dw0iBLZXHxt40xdwH3Awb4G/BWCtcpGdDYCIMGWbMqi4r8tkaJg2pCUSJR\nTSg8/jhssQX8+c9+W5KjJGuxAW2B4cAj9jYM2DrTFmA6GwU4ZDl7ttXdu3FjeveiQ5bew+begLzV\nRD4Nz+iQpfeoJnJLE14OWTY2ihx4oMgjjzg2M6fBQQ+Z52JxsnktNJHEi4QnOlZZWSk9ioqkR1GR\nVFZWtjg3nWub+PZba/HwpUvTv49kC9nqQrfOcSI0tzavNZGonCQ6tmTJkogKY+zYsXLrrbc278+b\nN0/mzZuX0rXTp0+XJUuWuHI/VVVVckJ5mZxQXtZCh6oJ56gmcksTqZT5TBfzfvxxa7hy0ybHZuY0\nTjSRaHHxeSJSYYx5F2u8OapjzZ2FlBNRaIvGDh0KP/0Ed97ptyVKLJoWjVVNKIqFakIBy9usZ0+4\n/HI45RS/rfEXrxYXH2L//TORi8WeAJyYSWb5RHRYCychMgDeeQcefBBuuMEDY10kejr2jBkzIqZu\nFwiqiSiip9PPnz+f+fPnN+/nc0gJ1QSgmmhBIWkiFIKff4by8s2fjR8/nk6d9qBTpz0YP368f8bl\nEsm60IDzgD0z7YJzspGFIctMiPYvS+QzlgqNjSK9eonccouHRruEV74NuQCb/WVUE1FE+6Yk8o/J\nN1QTqolYFIomGhtFDj9c5KGHNn9WWVkp0KHZFw06xAzvlI/gpQ8ZMBZ4HvgUmAdcDByQaYZpGRdQ\noUWHxEgU5iIV7r/fcoZM15HfL7yajh10wiof1UQM0pnSn2+oJlQTsSgETTz9tMjee0fWX0VFPVqE\nzygq6uGfkVnESYMsadgLEbkWwBjTFhgIXA7cDGyR7FolOd9/b427z59vTRdWgo9qQlEiUU0UJiJw\n3XVw9dVaf7lB0gaZMeYa4PdAe+BNYATwosd2BZqBI0Yw4MUXYd06AD5o04bLAH75BYCRbdsye8SI\nlNIaOxb69IEjjvDIWJeZMWMGw4cPZ8qUKQAMHz6c1q1bN6/XVgioJlpSW1tLRUUF1dXVAJx88skA\nzfsVFRURa/HlE6oJ1UQsCkETNTXw5Zdw2mmRnw8ffg5XX31J2CeXMHz45dk0LSdJJTBsOfA/4Elg\nMfCyiGzw1KqAE70g+EN24yvdxcWXL4fZs+G997yz1W2iF8xt3bo1hx56qM9WZR3VRBTRix/fdddd\nABktpJxrqCYA1UQLCkETY8fCqFHQOqolMWrUKAAmTx4HwPDhlzd/psQnbtiLiJOM6QD8AegFVACr\nReSPHtuW19OZReCYY6BfP7j4Yr+tUVIhfDqzakJRVBOFzOLF8Pe/wwcftGyQFTJehb1oSvy3wBnA\nAOBU4Ass582cIFk4ivDj48ePTzl0hdMwFw8/DGvWwD/+kfalzURPq850GrVX07Pdsi9o5LImkr2T\n8OMNDQ3MmDGj+Xii9xekdx0KhTixf19O7N83I23GSuf+++937f7csi9IqCbSTzcbuGlDdFrDh6+l\npOQVjj++P3369M+4LAfhOQWGZF7/wBPASCz/gC0znT2QyYbD2TOJlj+KdbwDyIgUQlckSzcZP/wg\nsssuIrW1jm7PtaVfvJqenW9L07B5RlnOaiKdJbamTp0qxhiZOnVq0vcXlHedbLmkTNPpsH1bVzTh\nln1BQTURbE24aUN4WtOm/VuMqZM2bXbMaJklr2wMAjiYZZk10WRknEOhRYenmBUVjiLm8RRCVyRL\nNxkjR4qccYajW2vGrWnUXk3Pzqdp3k6E5tbmVBMiyd9J+PHwcA7J3l8Q3vUJ5WUyeFaxLJD9ZYHs\nL4NnFcsJ5WWupHNEr0Md359b9gUF1UTLc9NJNxu4acPmtJ6SXXed2CK0RWlpue82+o0TTSQdslTc\n5cMP4e674cYb/bZEyVVE3WUURfGNw4B96NSpym9D8o9MW3LZ2MizIcvGRpHSUpHJkx3dVjM6ZJld\nCEhvwB57iFx3ncgnn6R/D/k8PCOiQ5bZJiiacEI+a8KLIcuePb+WoUM/lA4dOshWW3XRIcsonGjC\n90ZXQuNc6IquqqqS8tJSKS8tjVlYwo9XVlYmPDeddGMxf77Ib34j8ssvGd1KC+rr61t8cdTX1ztO\nZ968eTJv3jzX0800naAQlMrn1VdFBg8W6dzZWnLrzjtFvv02tXtI9k7Cj9fX18v06dObjyd6f0F6\n11VVVXJCeZmcUF7mqLETns6cOXNcuz+37AsCQdGEE/JZE27aUF9fL3fc8YbsvLPI+vVWWnPmzJHS\n0nIpLS3PuCwH4Tm5iRNNxA17YYx5PHHHmni+cGw2pjOHQqHm+GEDo+KHRR97/fXXuXfyZADOGT48\nrbgqP/0E++5rxR2zw9DkHUuXLuW1115rjsc0Y8YMDj30UHr27OmzZe5gjAHLeTkWWdfEL7/AU0/B\nfffBs8/CccfBWWdZgYYznYbe0NBAXV1dc+yk2tpaunfvTnFxcYtjt99+O19++SXXXHMNkH/v2w0a\nGhp4+OGHeeHlZwA46velnHrqqRQXF/tsmTsETRNeoJrYzEknwbHHOgvV1KSJUOhlAMrKfp93mpAM\nw14k+tVRkmjLtAWYzobHa5QlGnqMPtapdWtpZ/8/yx7eTGex1KuuEvnLX7y6k2CQ74ssYzmdBlIT\nX39tLU5/xBEiO+wgMmyYyBtvWMPk6ZBo+CD6WPv27ZudnPPxfbvBxIkTxRjk5JFd5OSRXcQYZOLE\niX6b5RpB1oRbqCYs3nhDZKedRNatc5bOxIkTJXrh8XzThGRaljO90I0N6At8AKwARsY47vaziiDR\nbMlYxw6P2u9RVJRSPh99JNKpk8jnn3t5N8EgnxdZdiK0VDc3NPHRRyLXXCPSvbvIfvuJ3HijyBdf\npH6fiWY8RR/L5/ftBieUl8nJI7s0P6OTR3bJ6VmV0eSKJpyimhDp10/EjdspLS13bXZmEHGiiVQC\nw+5ljJlvjHnfGPOpvX2S7LoU0t0CmGGLbV/gr8aYfZymGzREYMgQGDkSdt7Zb2sUNwi6Jvbc01rS\nZOVKmDnTmtm7335QVgYPPGANnyuKmwRdE4oz3n4bXnkFBg7025I8J1mLDXgJOBZ4G9gNGAOMy7QF\nGJbuEUBV2P4VwBVR53jThLXJxpDlo4+K7L23yIYNnt5KICiEIUvJUU38/LPI3Lkixx8vst12Imef\nLfL88yKbNkWep8Mz7lIIQ5aSo5pIFdWESEWFyE03uZOWDlkmKO9JT4A37L/vRH/mZANOAe4M2z8D\nmB51jgePK5JEsyWjj1VWVkqPoiLpUVSUUmPs55+tYaNnn/XK+mCxZMmSiC+f6dOny5IlS3y0yF3C\nKp+c1sSqVVbolQMOECkuFrnySpHly61jiWY8RR+77bbbZOzYsc37+fa+3aC+vl4mTpzYPKty4sSJ\nOT2DLJp80UQiCl0T774r0rWryI8/upNekyaaZmfmqyYy2ZIuLm6MeRlrsdj5wHPAf4AbROTXCS9M\ngjGmP9BXRM63988AeorIxWHnyOjRo5uvKSkpoSSHpiiOHg3vv2+tW6nkHjU1NdTU1DTvX3fddYiI\nySdNvP02zJljDWXusos1S/Mvf4HOnTNOUsljCkETSiR//SsccIDldqO0JJ4mMkosWYsNKyzvtkAx\nMAtYCByeaQswLN3DieyKvpIoh00y/OWTSYywVNJJJ2bZxx+LFBWJRDf8k8VcSaeXKZ1fbunEdkkU\ndyc6Rlk6McsyjeeTKJ10bXACm3sDckoTqZSFjRtFQiGR008X6dhR5MQTrbh569fHTyf8uTt5nyLJ\nY3NVVlZKcY8uUtyjS9Le6XhpuaUJEWvY5di+RzbnkWnMMtVEcDWRSTpuaSKRfW7VEU1UVVXFjSX2\n/vsiXbqIfP99bDubru3V6/iIYcdY5S+dmGXhNlVWVmYc6yz63hLdq1vg5ZClbC70HYAOmWYUI73W\nwEqgO9AGeBPYJ+qctB+G04W/46XTpU0b2a5NG5llR/PvEOZPFiuPE04Quf76lukmi0qcjh9WOr4N\n6UQ/ThSZOjqKfzpR/TONeJ3svt1aWSAZ0ULLFU2kWxa+/17k3ntFeve2ZgdfeKHIyy+LvPBC/Ofu\n5H0mi15fWVkp7Tq0aj7erkOruI2yRGm5pYkJEyaIMUj5lV1l8Kxi2Xa7rWW7Lm0ziuqvmsgNTaSa\njluaSGSfW3WEiKWXtm13kHjR9s84QyTe75/oa6GDTJgwIWb5Syeqf6x0YUTS65Kl06bNdtKmjfOV\nBZLhaYMMOBR4B/jM3t4CDsk0w6i0jwM+BD4GroxxPO2H4XTh70TpNIW9KLf34+XxxBMie+4Z2bsQ\nTqIp1CLphY5IZzp2OoRfG72YbnS66eSTKN1M7UvXhkwJ6w3IKU2IZF4WPvvM+mHx61+L7LGHyDnn\nfCLQPeZzz/R9Jltwu7hHlxbHi3t0ySgttzRx8sguzXnsdXg7RwuRqyZySxPJ0vHqfYbjVh2RKATF\nhx9aq4F8913sdGNdC+1jPoeDDy6Jm09q6ZYnvS55OoenbIMTnDTIUonnfQ9wkYjUAhhj/mh/9rsU\nrk2IiDwFPOU0nSCxfr0V5mLmTNhqK7+tUTyiYDSx665w5ZVwxRXw2mvwz39uCbwKvM+TT+7AUUd9\n6beJSjAoGE0UCtdfb0Xk79DBb0sKiGQtNmBZjM8cz55JZSMHhyzHjrUC6MVDhyxzf3gm1zTh9vDM\n00/XyLhxb0vr1o8JfCu9e6+WgQMfFWitQ5YFOmRZ6JqI9dxzecjy448td4W1a+PbqEOWscFBD1kq\nhf1m4HY2L4dxKzAFOAg4KNOMUzIuw65ov5z6P/3UKsR1dfHTVKf+vHBgzilNeOnAfM89j8qtt4oc\nfPB62XbbH+W8876XN96w/M3UqT95PqqJ/NNEPjj1n3uutdpHMtSpvyVOGmSphL2owRoHjtfD1jth\nAg7IxuLibtKvHxx8MFx9td+WKF7QtGisaiI2K1ZYITTmzIH27a0QGqefDjvt5LdlileoJvKPujqr\nHluxAoqK/LYm9/BkcfEgbGTYQxaOW71lyXjqKZEePZwvvJrpL7dk12X6S9ytX5JO7Q8COPjl49bm\nVBPZeM6bNom88ILIeeeJbL+9SJ8+Ivff7yywZLIetEyvazp+bN8jI37hJ3sumdqTDqqJ/NGEm7jZ\n0xurnjjhhP/IVVdZ+xMnTpRevY5P2qOUzWeYjV4uJzjRRCqFvRtwN3YsGKz1xM7NNMO0jHMoNLf8\nyZKxfr01q/KJJ5ynlalvQ7LrMvVVccvXwqn9QYDNwzM5q4lsP+effxZ56KHNSzYNGCDy3HMtl2xK\nRDIfs0yvCz9efmVXMYZmH5hEzyVTe9JFNZGfmnCKW76Qsa6dN2+pGPONPProizJhwgQJX+Iokc9V\ntp5hsjAdQcDrBlkVcBrwtr2/JfBuphmmZZxDobkVAiMZ119vxR1zi+oMp2Mnuy78eDrTsTO1J12y\nlU+mhFU+OasJEf+ec6IlmxKRLJRFptdFHz95ZJeUnkum9mSCaiK/NZEpTuxNdO1FF4n85S+f2cfb\npxUmIhvPMFGYjqDgpEHWKvGAJgCdReRfwCa75P8P2JjCdQVBfT1MmgQ33+y3JUoWUU1kQLduMGwY\nLFsGTz4JGzfCMcfAYYfBjBnw9dd+W6g4QDWR43zxBcydC6ee2uC3KYVLshYbUAN0wp7WjLWUxQuZ\ntgDT2ciBIctTThEZPdq99HTIMrjDBmzuDchZTQTtOaeyZJMOWQbjXcVCNZF9vBqyvPhikdNOa4gI\n76JDlumDgx6yVAr7wcDLwHf23xXA/plmmJZxAXfqf+YZkV/9yvKTcQt16vc2HyeEVT45q4kgP+d4\nSzY1NqpTfyo2+YFqIvt44dT/n/9YE3Bef/3zFuFd1Kk/PTxtkFnpsyWwn71tmWlmaRvnQoPMKzZs\nsJaTeewx7/KIjjczduxYue2225r3M21IRR+79dZbZezYsc37ieLapJOPE9JJN1tfBuFCU014S/SS\nTdddJ/LJJ9ax8Jhkl156aVrvPtUYZU40kSgfJ6gmClsT8YgVl+zmm29OqdHSVE6GDRMZMqRlOYlu\n/FRWVkpRUQ8pKuqRNB5grHyacLM8ptNAC3ocskSF/DBgx7D9AcAiYBpQlGmGaRkXYKHdeKM1c6yx\n0bs8oiMyA9K+ffuUuoUTdSFHH9tmm21SjvycTj5OSCfdbHWXYzmsqiaySGOjyKuvigwebK2rt9tu\nn0mbrc+X82/ZVwbPKpat2hlp165dSu8+nSj+TjTh1ZCmakI1EYtY9cSWW3ZKeaixqGgf2XbbX2Te\nvJciykn08GDr1ttEDGFCh5QbZV6Vx3SGMLM13OlVg2xZk6CAI4FVQH+gEpifaYZpGRdQoX3+uTWk\nsmKF93lFLyKbzkyWROdGH0tnsdp08nGCW/fqFnb6qgmf2LBBpHO3M6THIU9Ku47fyx9Oe0r+POws\n6bxjkSuzI93ShJezMFUTqolYhJfVvfY6OK2ZiKed9pnAtBblpOWMxl1apFtU1CNlG70oj+nMuszW\nDE0nDbJEsyxbicg39v+nAbeLyAIRuRrYM8F1ec+ll8KFF8Iee/htiZJtVBP+0aYNtN0mxHGDL+SW\nT/7Eb0pe5/XHL2bN6uXAZOAArPpZySaqidzlq6/g//5vR2CC36YokLCH7F1sPwDgQ+CosGPvZdoC\nTGcjgL98nn9eZNddRX76yfu8dMgykMMzqgkfiV5ofKt2Rrbe+rdy5pmfSrduP8sWW7wn//jHSvni\ni5bX6pClaiIfcTJkefrp9bL11vfELCc6ZJkZeDRkOQprtswirOHLVvbnewIvZZphWsYFTGi//CKy\n774iCxZkJz916g+eA7Nqwn/iOfVv2iQybdqb8pe//NC8ZNOcOZFLNqlTv7uoJvwnU6f+r78W2W67\njfLww0ubP1Onfuc4aZAlXFzcGHME1pIYT4vIT/ZnewHtReSNFDvhMiZoi8ZOngyhEFRVgcls6VAl\nhzHWS/89qonAs24dLFoE990HL78MJ51kLXZeUgKtUgmHraSEaiJ3ufpqWL0a7rzTb0vyCyeLiyf8\nahKRV0TkkSaR2Z99lA2RBY1Vq+D662HatOSNsYaGBmpra5v3a2traWhwHv140aJFDB06tHl/6NCh\nLFq0KO08o8+dP38+8+fPd93efEQ1kTmhUIgT+/flxP59CYVCrqTZ0NDA+eefz657dGXXPbpy/vnn\n09DQQNu2cNppcMklIQ77/V947Y3bGTjwe7p3h6uugvffb5mOaiIzVBOZ4VU9sXTpUgYNGkSfPv3p\n06c/gwYNYunSpRF5rl0Lt94Kxx77asI8Q6FQczqhUKjFvuIymXatZWMjQF3RZ5whMnJkaud6NV4+\nZMgQAWTQoEEyaNAgAWTIkCFp5xl9bseOHaVjx46e+5vkOjjoinZrC5Im0sErv6rzzjtPMMjJI7tY\na1Ea5Lzzzoub5623viiXXSay444ihxwiMm2ayJdfqiYyRTWROV7VExdddFELX6+LLrooIs8BAz6V\nvn3/kzRMTLjPVZs220mbNl0898HKdZxowvdGV0LjAiK0xYtFdtlF5IcfUr/GqynnTQ2xpoZZpnlG\nn5uNKfK5jlY+meNVKIjiHl0iFgU/eaTlW5Ysz1hLNo0Z845AG9VEGqgmnOFHKIjHH18s8JVAj4R5\ntkzn8KyEjch1nGhCvSmSsHEjDBpkLSDevr3f1iiK4gZbbAF9+sD990NDA/TrB489tjPwBXAL773X\nAVG3JCUPWbhwF+BJYKXfpijRZNqSy8ZGAH75TJ0qcvTR6UXk1yHL/ATtDciYoAxZJlsc+aGHXpbz\nzlsprVp9IK1afSxnn/2JPPjgK6qJOKgmMsePIcunnnpJjPlKZs9ekjRPHbLMDCeaSDjL0m/8nj2z\nejXstx8sXgz77JP6dQ0NDdTV1dGrVy/Actbs3r07xcXFjuxZtGgRzz//PDfffDNgOfUfffTRnHji\niWnlGX1uk/PyKaec4qq9+YaT2TMu2uCrJpwQCoWYeccUAAYNHEZZWZnjNBsaGhg7diyh6scAKOt9\nEtdee21z2U01z2hNzJs3n5Urt+eLL47hoYdgp52+4/TTN3HBBUV07OjY7LxBNZE5XtUTS5cu5b77\n7mPFiv8CsOee3TjrrLPo2bMnI0d+yxtv/MIzz3RNKc9QKMSkSXcAMGLEQICIfTc0nG840YQ2yBJw\n9tnQpQvcdJNvJigBQiufwuSXX+Cpp2DOHHj2WSgrs0Jo9OkDW27pt3X+oprIHX78EXr0gJqa9DoY\nlPTwLOyFVxhjKowx7xljNhljDvLDhmS8/DI88wxce623+SSa+qxT8VPDq+nj2SQXNJFNEoXIiD7m\nRTiNcNq0seKYzZ8Pn3wCvXvD+PFQXAzDhsGyZQTO30w1kV+4UU/ccotVdgu5MRb4sB2ZjnU62YC9\ngb2AauCgBOc5GcrNmI0bRQ44QOSBB7zPK50ljtTXKzbZXCZGClQT2SSR71f0se26tJVtt9vadd+0\nVPjoI5FrrhHp3l1kv/1EbrxRYi7Z5AeqifzCaT0RCr0oO+wg8s47Pt+Ij+T00knZ2IIqtJkzRY46\nKj1Hfickmvqs4SlSIxvPxcvKRyTYmsgmicJVxDq21+HtXA+nkQ6bNom88ILIueeKbL+9SGlpyyWb\n/EA1kV84qScmTRLp398fu4NCsnAgbuFEExr2IoqvvoIxY2DGDF0eSVGU5LRqBUceCXfdBV98AX//\nO8ydCzvvbPmhPv88NDb6baVSqGzY0IqbboJrrvHbEiUZrb1K2BjzDNb6ZtFcJSKPp5rOmDFjmv8v\nKSmhpKTEsW2JuPJKOP10a3ZlNqitraWiooLq6moAKioqWLhwIb169Wpx7OSTTwaIeW4hk+gZOqGm\npoaamhoXLLTIVU1km0EDh3H6gFOa9+eOXMsDs4fFPHb/ZV+z6X9C9exvWpzrB23bwl/+Ym3//a/V\nMBsxAtasgTPOgDPPzI4Pj2qixLFtQcJJPfHnP1dx0EH7sv/+nf0xPiCMGDGQF18cwLp11n7btiMZ\nMWK243Rd1USmXWtubASsK3rJEmtJlW+/zV6e9fX1Eb4dixcvlvr6+pjH5s2bJ/PmzYt5biGT6Bm6\nCTo8kzWqqqrkhPIyOaG8rIWfR/SxROcGhbfeErn00pZLNnmFaiK/yLSeWLdOpHPn9fLkk6uya3BA\nqaqqktLSciktLffsu8KJJoIwZBmIgcFNm6yI/P/8JzHjDKUzy8XJjKbVq1c3LwQL8NZbbzWntcMO\nO9CzZ8/mY7169cooZs3SpUuZMWNG8/6MGTMi8kxEEGdvFRcXR/zyz/S5BIhAaCIZycqCm7Mfm9Iq\nPe4o3n33XRYtqGLRgiratWvHvvvu27yfaVykhoYGJk2a1GzvpEmTUi7XqWjid7+zwufU10NlJSxZ\nAnvuac3eXLAANmzIyOy4qCb8I95MPre+OxsaGnjrrbea9+vq6vjss8+a89x2222bY0refTccfvhW\nHH98rE7I5PlMmjSpOd10NAHBnNFYVlbG008v4OmnFwQzhlqmLTknG9APaADWAf8Fnopznuut13jc\nfrvIH/4Q35E/nVku6cxoSjRDZurUqWKMkalTp7o6U2r69OlijJEpU6bIlClTxBgj06dPz8jeQprp\nibczygKniWQkKgtOIvNHX7vtdlvLdl3ayuBZxVJ+ZVcxBpkwYYKr5W/ixIliwiL+G4NMnDgxpWsz\n1cR334nce69I794iRUUiF1wg8tJL2ZtM5AaqiUgSzeRzq56Irhc6dOggW23VMoL++vXWGsxLl2Z2\nLxMnTmwR8T9VTWRrRmMQcaIJXxpkKRuXJaF9/bVI164ib76Z+Lx0ZrmkQ6IZMlOmTPFkplR4ulOm\nTDOzP/UAABJCSURBVHFkb6HgZeWT6hakykckfllwsph49LV7Hd4uYj98MXG3yt8J5WUtFilPZ7am\nU03U1YmMHy/y61+L9Oghct11IitXpp1M1lFNRJJsJp9b9UT49/fBB5fEzPO220T69vXuXry6Ntdx\nookgDFn6zqhRcOqpsP/+fluiKEohsttucNVV8P771kSAr76Cnj03z9787ju/LVRyicbG1txwg/eB\nzRWXybQll42NLPzyee01kW7dRNauTXyeDlnGtleHLPNPE6miQ5YWXmliwwaRRx8VKS8X6dBB5NRT\nRZ54QuSXXxwn7RqqiUiCMmQ5ZMg7Ulrq7F50yDIznGjC90ZXQuM8FtqmTSI9e4rcc0/yc9OZ5ZLO\njKZEM2Tq6+tl+vTpzWm5NVNqyZIlEQ2w6dOny5IlSzKyt5BmemrlE0mysuBk9mO8mZTH9j0yolJw\nq/zV19fLxIkTm/OcOHFixhr2QhNffy1yyy0ihx9uuVcMHSryxhv++5upJloSbyafW/VErHphzpw5\nzXk+8URIfvUrkdpaZ/fRpImmdNPRhEh2ZjQGESeaKOjFxe++G+6801q3spUO3ipJ0IWUlSDw0Udw\n//3WYuft21uxzU4/3QpEm21UE8Fj1iy47z4rILGSfXJucfEgsHat5Ts2c2ZqjTGvwj04CUGhKH7j\n1cLe48ePZ9c9urLrHl0ZP368a+nmA3vtBWPHwsqV1vfXRx9Zgaz79LEaaj/95LeFhYvf9cTGjVZY\nldGjHWep+EGmXWvZ2PCwK3rQIJELL0z9fK/8RJz4cynZBR2eicCJn1giKisrpV2HVs3ptuvQSior\nK12wOH/5+WeRuXNFjj9epGNHkQEDRJ57znLL8BLVRCR+1xP33Sdy5JGOs1Mc4EQTvje6EhrnkdCW\nLbP8MNasSe86r8I9OAlBoWQPrXwicRLaIhHFPbq0SLe4RxcXLC4MVq0SmTxZ5IADRIqLRa64QmT5\ncm/yUk20xK96YuNGkb32Enn2WdeyVDLAiSYKbsiysdGKyF9ZCUVFflujKIriLt26wbBhsGwZPPGE\nNYx1zDFw6KEwfboVUkPJPx5+GDp3hqOP9tsSJWMybcllY8ODXz6zZokcemj6Xfl+d0Ur/oP2BkSg\nQ5a5w8aNIqGQyOmnW0OaJ54oMn++yPr1ztJVTUTiVz2xcaPIPvtY71jxFyea8L3RldA4l4W2dq0V\ncyyTpSS8mtruJASFkl208mmJVwt7V1ZWSnGPLlLco4s2xlzm++/dW7JJNRGJX/XEv/5lhXDyOwyK\n4kwTBRX2YuhQawbSnXe6lqRSQOgUfyXf+OwzeOABK0zCxo1w1llwxhmw++6pXa+a8J/GRmuVmQkT\n4Pjj/bZG0bAXKfDOO/Dgg1Ba+jz9+/Shf58+GU/Td3Nqs1fTpIOWpxJM3CwLbobA8CqcRtDy9Jt4\nSzb16mX9cP32W78tzD5uaSJb9cSjj8LWW8Nxx2WUdEZ5Kh6RaddaNjZc6opubBTp1Utk8OD3ZIe2\nbWUWyCyQHdq2zWiYxU0/AT+WIirk5Y+cQB4Oz7hVFtz0J/PKNy1oeQaVWEs2xZu5p5rwPp1EaW3a\nJLL//iKLFmWUbEZ5KolxognfG10JjXNJaPffL3LQQSL9ji2TWZuXn5dZIOUZLvjl5tRmr6ZJBy3P\nXCcfKx8Rd8qCmyEwvAqnEbQ8c4E1a0RuvVVkxozYx1UT2UknXlqPPmqFN/HKd0zrifRxoonWXva+\nBYHvv4fLL4f582Hi6Ea/zVEURckZiorgwgv9tkKJhYi1YsO114Lx1YtPcY1MW3LZ2HDhl8+IESJn\nn239X1VVpUOWPuWZD5CHvQE6ZOlfnvmAasL7dOKlNWHCO/Lb33q3GoPWE5nhRBO+N7oSGudQaO+9\nJ9K5s8jq1Zs/q6qqkvLSUikvLc34C9fNqc1eTZMOWp75QD5WPm6WBTdDYHgVTiNoeeY6qgnv04mV\n1gsvLJb9918vDz+cUXIZ5an1RGo40UTehr0QsaJT9+sHF1/ssmFKQaJT/BUlEtWEP1RVwYgRVvSA\nVgUTKyE30LAXMXj4YVizBv7xD78tyQ7JpijrFGalEEkWyqIQQ10ouY0IXHcdXHNN+o0xrSeCTV42\nyH78ES69FGbOhNZ5P23Boq6ujvLycmpqaqipqaG8vJy6urqUjytKvhEKhTh9wCnsduJydjtxOacP\nOCWi0ZXsuKIEkeees+LDVVSkf63WE8EmL4csR46EVaus6NOFRE1NDb179wagurqakpKStI4ridHh\nmdzixP592e3E5fQeUARA9exv+GzRvixaUJXScSU5qonsIgJHHmnNfD399MzS0HrCW5xoIu/6jz74\nAO6+2xpbVxRFUZR8oaYGVq+G007z2xLFEzKdDeBkA24C3gfeAhYCHeOcl9bshsZGkdJSkcmT07os\nL0g2RVmnMDsHD2eUeaWJQiZZKAsNdeEc1UR2KSkRmTUr8+u1nvAeJ5rwZcjSGFMKPCcijcaYf9qK\nuiLGeZKOfQsWwOjRsGwZbLmle/bmAg0NDdTV1dGrVy/Acsbs3r07xcXFKR1XkuPl8IxXmih0QqEQ\nM++YAsCggcMoKytL67iSGNVE9li8GM45xxoFyrR+03rCe5xowncfMmNMP6C/iJwR41jKQvvpJ9h3\nX8tv7Kij3LZSUbLnL+OWJhTFa1QT2ePYY+Gvf4Vzz/XbEiURuR724u/A/zlN5Prr4Q9/0MaYkhe4\noglFySMKWhMvvQQffwxnneW3JYqXeObUb4x5BugW49BVIvK4fc4o4BcRedBJXitWwO23w9tvO0lF\nUbwlm5pQlFxANZEa48bBVVcVnitOoeFZg0xEShMdN8acDRwPHJPovDFjxjT/X1JS0mIKrghccglc\ncQXstFOGxipKDJpi8bhFtjShKF6hmsg+S5fC8uXw2GN+W6LEwk1N+OXU3xeYBBwlIl8nOC+pb8Bj\nj1mNsbfegjZtXDZUUcLw2IHZNU0oSrZQTXjPn/5kbRdd5LclSirknFO/MWYF0Ab4xv7oFRFpUdyS\nCW3dOsuR/667rHUrFcVLPK58XNGEomQT1YS3vP46nHwyrFwJW23ltzVKKuRcgyxVkglt9Gh4/31r\n3UpF8RqNSq4okagmvOWkk6zZlRdf7LclSqrk+izLjFi50lqrctKk7OetC7AqSkt0oW5F2YzTemLZ\nMnjtNTjvPC+sU4JIzjbIhg61FhD3I16dLsCqKJHoQt2KEonTemLcOLjsMmjb1jsblWCRk0OWTzwB\nI0ZYYS78GlfXBVgLDx2eiY8u1F2YqCYSk2k98fbbUFZmjQS1a+ehgYrrFNTi4uvXW71jM2eqk6Oi\nKIqSf1RWWp0O2hgrLHKuQXbTTfC731m/HvyitraWiooKqqurAaioqGDhwoXN638pSqExaOAwTh9w\nSvP+3JFreWD2MB8tUhR/ybSeeO89eOEFuOeebFipBImcGrKsq4NDDoF//xt2280/u3QB1sJEh2cS\nowt1Fx6qifhkWk/89a+w//5WfE0l9yiYsBf9+sHBB8PVV/tolFKwaOWjKJGoJtzlgw/gyCMt37Ft\nt/XbGiUTCsKHrKoK3nkH5s712xJFURRFcZ/x42HIEG2MFSo50UO2YQP89rdw881w/PF+W6UUKtob\noCiRqCbcY8UKOOIIq3esY0e/rVEyJe8Dw06eDHvvrY0xRVEUJT8ZP96KyK+NscIl8D1kn30mHHQQ\nvPoq7L673xYphYz2BihKJKoJd1i5Enr2hI8/hu2289saxQl53UM2YgQMHqyNMUVRFCU/ueEGuOgi\nbYwVOoHvIfvVr4T33tPlIxT/0d4ARYlENeGcujoresCKFVBU5Lc1ilPyuofs5pu1MaYoiqLkJzfc\nABdcoI0xJQd6yBobBePr7y9FsdDeAEWJRDXhjPp6OOAA+Ogj6NzZb2sUN8jrHrJ4jbGampqs2hGk\n/Av53oOQf1Dx+7moJgo3/6Di93NJlv+ECXD++d40xoJ+7/mefyYEvkEWD78ftlY+hZt/UPH7uagm\nCjf/oOL3c0mU/xdfWIHOR4zIft7ZoNDzz4ScbZApiqIoSq5y441wzjnQtavflihBIWeWTlIURVGU\nfGDVKpgzB5Yv99sSJUgE3qnfbxsUJZwgODD7mb+iRKOaUJRIMtVEoBtkiqIoiqIohYD6kCmKoiiK\noviMNsgURVEURVF8RhtkiqIoiqIoPpPTDTJjzE3GmPeNMW8ZYxYaYzpmMe8KY8x7xphNxpiDsphv\nX2PMB8aYFcaYkdnK1877HmPMamPMO9nM18672BhTbT/zd40xl2TbhlxANaGaUCJRTagmcoWcbpAB\nTwO/EZH9gY+AK7OY9ztAP2BxtjI0xmwBzAD6AvsCfzXG7JOt/IF77bz94H/AMBH5DXA4MCjL954r\nqCZUE0okqgnVRE6Q0w0yEXlGRBrt3aXALlnM+wMR+Shb+dkcBnwsInUi8j/gIeCkbGUuIrXA2mzl\nF5X3f0XkTfv/H4H3gZ38sCXIqCZUE37YEmRUE6oJP2zJhJxukEXxd+D//DbCY3YGGsL2P7c/KyiM\nMd2BA7G+XJX4qCYKBNVEyqgmCoRc1ETgI/UbY54BusU4dJWIPG6fMwr4RUQezHbeWabgg8YZY9oD\n84Eh9i+ggkM1EYFqQjWhmohENZGjmgh8g0xEShMdN8acDRwPHJPtvH3gC6A4bL8Y69dPQWCM2RJY\nANwvIo/6bY9fqCYiUE2oJlQTkagmclQTOT1kaYzpC1wGnCQi6/00JUv5vA7saYzpboxpA5wGLMpS\n3r5ijDHA3cByEbnZb3uCimpCNaFEoppQTeQKOd0gA6YD7YFnjDHLjDG3ZCtjY0w/Y0wD1kyOJ40x\nT3mdp4hsBAYDIWA58C8Red/rfJswxswFXgb2MsY0GGPOyVbewB+AM4De9rteZn/RKpGoJlQTSiSq\nCdVETqBrWSqKoiiKovhMrveQKYqiKIqi5DzaIFMURVEURfEZbZApiqIoiqL4jDbIFEVRFEVRfEYb\nZIqiKIqiKD6jDTJFURRFURSf0QZZFjHGjDLGvGuMecuOj3KYy+mXGGNaLNUR73MX8jvJGLNP2H6N\nMeZgt/NR8hfVhKJEopooXAK/dFK+YIw5AvgTcKCI/M8YUwRs5bNZTukHPA40BR3UoHZKyqgmFCUS\n1URhoz1k2aMb8LWI/A9ARL4RkVUAxpiD7V8Nrxtjqowx3ezPa4wxN9u/kt4xxhxqf36YMeZlY8wb\nxpiXjDF7pWqEMWYbY8w9xpil9vUn2p+fbYxZaIx5yhjzkTFmQtg15xpjPrSvucMYM93+4jgBuMlO\nZ3f79Ar7vA+NMX9048EpeYtqQlEiUU0UMiKiWxY2YBtgGfAhMBM40v58S6xlJjr9f3v3DhpFEMdx\n/PsjOQI+Ypo0CmqhIlql8oFBBREsLIxCCgsVexsVLCMBRe0sAhZBsVFRBLESlRi1UnI+zggiPqrU\n0SBooX+LnZjd4wIewm3wfh8YMjc3szsc+cHs7hyXXg8Co6k+BlxK9X6glupLgY5U3wXcSvUdwN0G\n5/7TDpwBDqZ6T5rPIuAw8CEduwv4DKwAlgOfUt9O4DFwMY2/DAzkzjMGXEj1PcD9sj93l4VbnAkX\nl2JxJtq7+JFli0TEt/TcvB/YCdyQdAqYADYCDyQBdABTuaHX0vgnkroldQPLgKuS1pDd/q00MZXd\nwF5JJ9LrLmBlOs7DiJgBkPQWWA30AuMRMZ3abwL5K636H8y9nf5W03izhpwJsyJnor15QdZCEfEL\nGAfGJdWAQ2RBm4yIrU0capgsFPskrQIeNTmVgYh4n2+QtAn4kWv6Sfb/Uf+8vz5Y9e/PHmN2vNm8\nnAmzImeifXkPWYtIWidpba6pj+x27zugV9Lm1K8iaUOu32Bq3wZMR8RXoJu5q6MjTU7lHnAsN6++\n2WqDvgE8B7ZL6pHUCexnLlwzaS5mTXMmzIqcifbmBVnrLAGuSJqU9ApYDwxFtnnzAHBO0kuy/QNb\ncuO+S6oCI8DR1HYeOJvaOyhefTT6Bkvk2oeBiqTXkt4Apxv0mRsYMUW2n+AZ8JRsn8CX9PZ14KSk\nidxmzfrzms3HmTArcibamNLGOluAJI0BxyOiWvI8Fqe9DZ1kz/5HI+JOmXOy9uRMmBU5E/8P3yGz\nvzEk6QVQAz46ZGbOhFkdZ+If+Q6ZmZmZWcl8h8zMzMysZF6QmZmZmZXMCzIzMzOzknlBZmZmZlYy\nL8jMzMzMSvYbsc/m82vHVTwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x107410550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_min, x_max = X_train[:, 0].min() - .5, X_train[:, 0].max() + .5\n",
    "y_min, y_max = X_train[:, 1].min() - .5, X_train[:, 1].max() + .5\n",
    "xs = np.arange(x_min,x_max,0.5)\n",
    "fig, axes = plt.subplots(1,3)\n",
    "fig.set_size_inches(10,6)\n",
    "for i in [0,1,2]:\n",
    "    axes[i].set_aspect('equal')\n",
    "    axes[i].set_title('Class '+ iris.target_names[i] + ' versus the rest')\n",
    "    axes[i].set_xlabel('Sepal length')\n",
    "    axes[i].set_ylabel('Sepal width')\n",
    "    axes[i].set_xlim(x_min, x_max)\n",
    "    axes[i].set_ylim(y_min, y_max)\n",
    "    plt.sca(axes[i])\n",
    "    ys=(-clf2.intercept_[i]-xs*clf2.coef_[i,0])/clf2.coef_[i,1]\n",
    "    plt.plot(xs,ys,hold=True)    \n",
    "    for j in [0,1,2]:\n",
    "        px = X_train[:, 0][y_train == j]\n",
    "        py = X_train[:, 1][y_train == j]\n",
    "        color = colormarkers[j][0] if j==i else 'black'\n",
    "        marker = 'o' if j==i else 'x'\n",
    "        plt.scatter(px, py, c=color, marker=marker)     \n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us evaluate on the previous instance to find the three-class prediction. Scikit-learn tries the three classifiers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 15.45793755  -1.60852842 -37.65225636]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([0])"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaler.transform([[4.7, 3.1]])\n",
    "print(clf2.decision_function(scaler.transform([[4.7, 3.1]])))\n",
    "clf2.predict(scaler.transform([[4.7, 3.1]]))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `decision_function` method tell us the classifier scores (in our case, the left side of the decision boundary inequality). In our example, the first classifier says the flower is a setosa (we have a score greater than zero), and it is not a versicolor nor a virginica. Easy. What if we had two positive values? In our case, the greatest score will be the point which is further away from the decision line. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluating the classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The performance of an estimator is a measure of its effectiveness. The most obvious performance measure is called _accuracy_: given a classifier and a set of instances, it simply measures the proportion of instances correctly classified by the classifier. We can, for example, use the instances in the training set and calculate the accuracy of our classifier when predicting their target classes. Scikit-learn includes a `metrics` module that implements this (and many others) performance metric."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on the training set:0.83\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "y_train_pred = clf2.predict(X_train)\n",
    "print ('Accuracy on the training set:{:.2f}'.format(metrics.accuracy_score(y_train, y_train_pred)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This means that our classifier correctly predicts 83\\% of the instances in the training set. But this is actually a bad idea.  The problem with the evaluating on the training set is that you have built your model using this data, and it is possible that your model adjusts actually very well to them, but performs poorly in previously unseen data (which is its ultimate purpose). This phenomenon is called overfitting, and you will see it once and again while you read this book. If you measure on your training data, you will never detect overfitting. So, _never ever_ measure on your training data. \n",
    "\n",
    "Remember we separated a portion of the training set? Now it is time to use it: since it was not used for training, we expect it to give us and idead of how well our classifier performs on previously unseen data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on the training set:0.68\n"
     ]
    }
   ],
   "source": [
    "y_pred = clf2.predict(X_test)\n",
    "print ('Accuracy on the training set:{:.2f}'.format(metrics.accuracy_score(y_test, y_pred)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generally, accuracy on the testing set is lower than the accuracy on the training set, since the model is actually modeling the training set, not the testing set.\n",
    "\n",
    "One of the problems with accuracy is that does not reflect well how our model performs on each different target class. For example, we know that our classifier works very well identifying setosa species, but will probably fail when separating the other two species. If we could measure this, we could get hints for improving performance, changing the method or the features. \n",
    "\n",
    "A very useful tool when facing multi-class problems is the confusion matrix. This matrix includes, in row i and column _j_ the number of instances of class _i_ that were predicted to be in class _j_. A good classifier will accumulate the values on the confusion matrix diagonal, where correctly classified instances belong. Having the original and predicted classes, we can easily print the confusion matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 8  0  0]\n",
      " [ 0  3  8]\n",
      " [ 0  4 15]]\n"
     ]
    }
   ],
   "source": [
    "print (metrics.confusion_matrix(y_test, y_pred))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To read the classification matrix, just remember the definition: the “8” on row 2, column 3, means that eight instances if class 1 where predicted to be in class 2. Our classifier is never wrong in our evaluation set when it classifies class zero (setosa) flowers.  However, when it faces classes one and two (versicolor and virginica), it confuses them. The confusion matrix gives us useful information to know what kind of errors the classifier is making."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Accuracy on the test set is a good performance measure when the number of instances of each class is similar, i.e., we have a uniform distribution of classes. However, consider that 99 percent of your instances belong to just one class (you have a skewed): a classifier that always predicts this  majority class will have an excellent performance in terms of accuracy, despite the fact that it is an extremely naive method (and that it will surely fail in the “difficult” 1% cases).\n",
    "\n",
    "Within scikit-learn, there are several evaluation functions; we will show three popular ones: precision, recall, and F1-score (or f-measure)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "     setosa       1.00      1.00      1.00         8\n",
      " versicolor       0.43      0.27      0.33        11\n",
      "  virginica       0.65      0.79      0.71        19\n",
      "\n",
      "avg / total       0.66      0.68      0.66        38\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print (metrics.classification_report(y_test, y_pred, target_names=iris.target_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Precision computes the proportion of instances predicted as positives that were correctly evaluated (it measures how right is our classifier when it says that an instance is positive).\n",
    "- Recall counts the proportion of positive instances that were correctly evaluated (measuring how right our classifier is when faced with a positive instance).\n",
    "- F1-score is the harmonic mean of precision and recall, and tries to combine both in a single number."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the four flower attributes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To end with this classification section, we will repeat the whole process, this time using the four original attributes, and check if performance improves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "     setosa       1.00      1.00      1.00         8\n",
      " versicolor       0.78      0.64      0.70        11\n",
      "  virginica       0.81      0.89      0.85        19\n",
      "\n",
      "avg / total       0.84      0.84      0.84        38\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Test set will be the 25% taken randomly\n",
    "X_train4, X_test4, y_train4, y_test4 = train_test_split(X_iris, y_iris, test_size=0.25, random_state=33)\n",
    "    \n",
    "# Standarize the features\n",
    "scaler = preprocessing.StandardScaler().fit(X_train4)\n",
    "X_train4 = scaler.transform(X_train4)\n",
    "X_test4 = scaler.transform(X_test4)\n",
    "\n",
    "# Build the classifier\n",
    "clf3 = linear_model.SGDClassifier(loss='log', random_state=33)\n",
    "clf3.fit(X_train4, y_train4) \n",
    "\n",
    "# Evaluate the classifier on the evaluation set\n",
    "y_pred4 = clf3.predict(X_test4)\n",
    "print (metrics.classification_report(y_test4, y_pred4, target_names=iris.target_names))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Unsupervised Learning: Clustering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, is possible to take an unlabeled training set and try to find a hidden structure or patterns in the data: there is no given target class to predict or to evaluate the resulting model. We call these class of machine learning tasks _unsupervised learning_. For instance, _clustering_ methods try to group instances into subsets (called clusters): an instance should be similar to another in the same subset and different from those belonging to another subset.\n",
    "\n",
    "In this section, we will perform clustering of the Iris data set, to see if we could group instances using their petal and sepal width and length. The trainig set is the same we used we used for our last example on supervised classification.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "K-means is probably the most popular clustering algorithm, because it is very simple and easy to implement, and it has shown good performance on different tasks. It belongs to the class of partition algorithms that simultaneously partition data points into distinct groups, called clusters. We will apply k-means to the training data, using only sepal dimensions, building 3 clusters (note that we could have selected a different number of cluster to group data into)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KMeans(copy_x=True, init='k-means++', max_iter=300, n_clusters=3, n_init=10,\n",
       "    n_jobs=1, precompute_distances='auto', random_state=33, tol=0.0001,\n",
       "    verbose=0)"
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import cluster\n",
    "clf_sepal = cluster.KMeans(init='k-means++', n_clusters=3, random_state=33)\n",
    "clf_sepal.fit(X_train4[:,0:2])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can show the label assigned for each instance (note that this label is a cluster name, it has nothing to do with our original target classes... actually, when you are doing clustering you have no target class!)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 0 1 1 1 0 0 1 0 2 0 0 1 2 0 2 1 2 1 0 0 1 1 0 0 2 0 1 2 2 1 1 0 0 2 1 0\n",
      " 1 1 2 1 0 2 0 1 0 2 2 0 2 1 0 0 1 0 0 0 2 1 0 1 0 1 0 1 2 1 1 1 0 1 0 2 1\n",
      " 0 0 0 0 2 2 0 1 1 2 1 0 0 1 1 1 0 1 1 0 2 1 2 1 2 0 2 0 0 0 1 1 2 1 1 1 2\n",
      " 0]\n"
     ]
    }
   ],
   "source": [
    "print (clf_sepal.labels_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using NumPy's indexing capabilities, we can display the actual target classes for each cluster, just to compare the built clusters with our flower type classes..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "print (y_train4[clf_sepal.labels_==0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 1 1 1 1 1 2 1 0 2 1 2 2 1 1 2 2 1 2 2 2 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1\n",
      " 2 1 2 1 1 2 1]\n"
     ]
    }
   ],
   "source": [
    "print (y_train4[clf_sepal.labels_==1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2 2 1 2 2 2 2 1 1 2 2 1 2 2 1 1 2 2 2 2 2 2 1 2 2]\n"
     ]
    }
   ],
   "source": [
    "print (y_train4[clf_sepal.labels_==2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usually, is a good idea to display our instances and the clusters they belong to, to have a first approximation to how well our algorithm is behaving on our data: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "colormarkers = [ ['red','s'], ['greenyellow','o'], ['blue','x']]\n",
    "step = .01 \n",
    "margin = .1   \n",
    "sl_min, sl_max = X_train4[:, 0].min()-margin, X_train4[:, 0].max() + margin\n",
    "sw_min, sw_max = X_train4[:, 1].min()-margin, X_train4[:, 1].max() + margin\n",
    "sl, sw  = np.meshgrid(\n",
    "    np.arange(sl_min, sl_max, step),\n",
    "np.arange(sw_min, sw_max, step)\n",
    "    )\n",
    "Zs = clf_sepal.predict(np.c_[sl.ravel(), sw.ravel()]).reshape(sl.shape)\n",
    "centroids_s = clf_sepal.cluster_centers_\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display the data points and the calculated regions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEoCAYAAACZ5MzqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8VGW9x/HPjy0pipiipnjBTLC8BqikhlqZklII3i8h\nWcfCUjpqmpSKUChanjxeSI/HFDXJG1pekiixXXhQFO/XLgoqXsE7KMrv/PGs2aw9e2b22rNnZq2Z\n+b5fr/3ac1mz1m+tmVm/eZ5nPc9j7o6IiEg5eqQdgIiI1C8lERERKZuSiIiIlE1JREREyqYkIiIi\nZVMSERGRsimJVIiZrTSzLTMQx2lm9j9px9FVZjbRzK6u0bY2N7N3zMy6uZ69zGxRpeLKOjO7w8y+\nmXYcldLV98/M5pjZt6PbR5rZXdWLruvM7DEz26PW2614EjGz58zsK7H7h5nZEjMbVultNRozu9LM\nJndnHe5+trv/R6ViqoYiX96KdVjq7Di6+0J3X9tr2EnKzMaaWWs9b8fd93P3qiR6M5tgZv+Kkvsi\nM5tRje10k0d/uPu17r5vyvG04+7buftfa73dapRE2g60mR0NXATs5+5V/wI1OzNrSTuGbuhWqSBP\n22eww0bMVqvgdqQCovPEUcBX3H1tYCdgdrpRSWLuXtE/4N/AV4DvAq8Bg0ssuxIYBzwLvA1MAj4D\n3Au8CcwAesaWHwE8BCwF/g5sH3vux8A/ovU8DhwQe24s8DfgPGAJ8C9geN7z/4xe+y/giCLx9gAm\nxLYzH9gkti9bRrfnAN/OW39rdNuA/wJeAd4CHgG2BY4FPgQ+AN4Bbo2W7wfcBLwaxXZ8bL0TgRuB\nq6N1fTt67Oro+S2iuMYAz0fvx4TY63sBV0XH5AngFGBRifdrN+D+6L25D9g19tyc6P37W3Rs7gL6\nFljHWsAy4ONoP98GNgbOBH4XxfM28BgwJPa6osehwDZ+A0zOOwbHRMdgDtA/eqxHF9//XsCV0fF6\nHPhR/HhR5DMIfC7a54+ifV4SPb4/sCB67xYCZ8bWtQZwDfA64fN+H7Bh9Nw6wP8CLwEvAJMJn82C\n2ymwH88RTtjxz9HVJba7Qf7nms6/U58G/hodiz8BF+e2USCeC4H/KvF+FtzfWBx/j9bxJvAk8OXY\na79F+Gy/Hb3Hx8ae24vSn/evAk9F670w2v9j8r/TVTiXPQecBDwce+3q0XPrA7dFr3sD+Guh9xVY\nHfgV8GL091/AJ2L7/QJwIuE89BIwNrae/Qif37ej5U4qec7vLCl09Y+QRG4CXo4fmCLLrgRmAr2B\nbQgn0L8Qvvh9oh0ZEy07KNrhnQkn4jHRtnpGzx8EbBTdPgR4F/hU7A3/kHCSNeB7wIuxk9pbwIDo\n/qeAbYrE+yPCST+37A7AerF9ySWRu3MftvwPHLAvIfn0ie5vHYv7N8Ck2Ot6AA8APwVWI3wx/wns\nE/vyfwh8I3YCOJOOSeTS6EO1A7Ac2Dp6/pwo1nWATaJ9W1hk39cjfHCPjOI6jHDyWDd2gnkW2CqK\n427g7CLr2pO8L2+0L8uA4dF7NAW4N8lxKLD+QknkSkISWD32WI8uvv/nAPcAnwQ2JSS6hbHnS30G\njyZ20okdh22j29sTvjMjo/vfBX4fHUsjfP7Xjp6bCUyL9mcDYB7RybHQdop8R+Mn2jOB6Qm22/a5\npsR3Knr+XuDc6P3aPTrG04vEcyThhHgyoRTSkvd8qf0dC6wAxgMt0XF/k1Wfy/2AT0e39wDeAwbF\nTqYFkwjhZP02MDpa7w+j7ZRKIpU6l/0b+D9gI2BdQhL8bvTc2dGxaIn+di/0vhKS2NxoP9YnJKpJ\nsf1eQfjOtQBfi47LOtHzi3PrJZwbBpX8PHWWFLr6R8iGb0UH1DpZdiXtf83OB34Uu/8Lol8o0YGb\nlPf6p4A9iqx7AatOrmOBZ2PPrRlte0PCSWRp9GHp1Um8TwFfL7EvSZLIl4GngaFEv6Ziy7Wd/KL7\nQ4Hn85Y5Dbgiuj0RmJP3/EQ6JpF+sefnAYdEt/8JfDX23Lcp/qX6JvB/eY/NBY6O7XO8lDMOuLPI\nuvbK304U96zY/W2A95MchwLrL5REtog9n3ssl0SSvv/tEhfwH8WOV5HPYGcn918B50e3v0XeL9To\n8U8RfgisEXvscOAvXdhOfhKJf2YKbjf/c03p79TmhJNUPMarKVISiZ4/glBieZdQCjqlC/v7Yt66\n5gFHFdnOTOCEYp/D2HJjgLl5jy2idBLp7rlsWOz9OSL23FRgWnT7LOAW4DOl3ldCiTheMtwH+Hds\nv98ndv4hJLVdotvPE2pG+pT6HOX+qtUm8j3CL+zLcw+a2eNRo9k7ZrZ7bPlXYreX5d1fTviSQ6iC\nOMnMlub+CL8GN47WP8bMFsSe2w7oG1vXy20Bur8f3ezt7u8Bh0Yxv2Rmt5nZ1kX2bTPCiaRs7v4X\nQjvRxcArZnapma1dZPH+QL+8fT6N8EXNeSHBZl+O3X6f8GsJQhVRvIG71Lr6Eapc4p6PHi+0nWWx\n7SQVf+/fB9Ywsx4kOw6dKXgVThff//zj1e54JPgMkrf8UDO728xeNbM3CaWA3PJXE6oEZ5jZi2Y2\nNWrP6Q/0BBbHtvNrwi/0cnnsdrHtFlLwO0U4TkvcfXls2ZJXQbn7b939q4Rfvt8DJpvZPiTb3xfz\nVvc8q84LXzOz/zOzN6LX7keJ9ySmHx2/D51dydXdc1mS79J5hAQxy8z+aWanloj/+dj9hXnrf8Pd\nV8bux88LBxKO03PRFWlfKLINoHqX+L5CaBcZZmaXALj7th6uiFnb3f+ecD3xD/dC4Ofuvm7sr7e7\n/87M+gOXAd8nVC+tS6hqSNRY6+6z3H0fQvHxKaDYJbKLCNU1nXmPVR8YovXGt3ehu+9E+LU9kFBN\nBh0bgxcSfj3E97mPu4+ILZ//mvz7pSwmJMaczYotSPii9s97rD8dv8BJFIqxVNyLKH0cyt1meCL5\n+7+Y8Cs7p+12gs9goe3/lvCrclN3/yTh5Ngjiukjd5/k7tsS2qJGEH4dLyRUlfSNHYt13H37zvYz\nJv/zuXHuRontdsViYD0z6xV7bPNiC8e5+8fufiOr2go7218IVbFx/Qk/CFYnVK2fS2hPWhe4g2Tn\nhZeIfR+iy8FLfT86k+hc1tlr3f1ddz/Z3T8DfAM40cy+VCT+LWL3N48e6zxQ9/nufgAhUd8CXF9q\n+ar1E3H3xYREMtzMzu/CSy3vdu7+/wDfM7NdLFjLzPY3s96EL4QTisE9zOxbhF+BnW/MbEMzG2lm\naxGK4O8RGn0LuZzwC2mrKIYdzGy9Ass9BIw2s15mthWhmsij7e0U/QLtScj+y2PbewWI9zW5D3jH\nzE6J1tViZtuZ2U4FjhUlHivmeuA0M/ukmW0C/IDiJ6E7gIFmdriZrWZmhwKfJTTydXXbrwB9zaxP\nwtd2dhzyJT4GXXz/48drU+D42HOdfQZfATaN3vec3sBSd//QzHYhVOnkPid7mdn20RV370Sxfezu\nLwOzgPPNbG0z62Fmn7FV/QMKbSffQ8Bh0fu4E+GXZ8ntllhXB+7+PKE6Z6KZ9TSzXQnJqOBny8yO\nNrP9YvvzNUICmZdgfwE2NLMTom0dTPhc3gF8Ivp7HVgZrXefhLtxO7CtmY2KSmInkPdjMIFyzmUl\n12NmI3LnH0KbzceEqrR81wE/NbP1zWx94AxCKbN0wOEYHmlm67h77uKXku9/VTsbuvsiQhvAQWb2\n80KLdPJY2y9td3+AUAd9EaFB91miX0ju/gTwS0Jj3suEL+/fCq2nwHZ6AP9J+EX9BjCMUJ9fyPmE\nE8ksQrvP/xAaIPPj/i9Co+MrhPr5a2LP9SH8Yl1CaD96nVBEhXAFyjZREffmqLg5Avg84eqX16LX\n5k6+xfYr/xgWM4lQZP93tE83RHF34O5LolhOimI+GRgRPV5oW4Viy63rKcKH/F8W+hBtXGJfiD7M\npY5Dh00UiKXQMtC19/8sQhXBv4E/AtNjMXb2GfwzoXH1ZTN7NXrsOGCSmb0NnE64Oi1nI8L78Rah\nYXUOq04CYwgnxycIn6MbWHWCK7SdfKcTrhxaSmgPuTbhduM6KwUfCexKOKaTo30r+NkinAwnEI7t\nUsIFDN9z97nR86X2F0IbyADC52IycKC7L3X3dwgn/+uj1x0O3Foi5lUPur8BHBzF8jqhBqLUOaUS\n57Ji39X4trYitB29Q2iTvNjd7ynwmp8REvkj0d/86LFS8eYcBfzbzN4itI0cWWLZ0PAtAmBm4wiN\n7oWKxyJlM7PfAU+4+1kVXu9YwmXH6sycEg170sTMbCMz2z2qItiacN34zLTjkvoXVdt+JlY99Q1C\n/bo0GPXebW6fIDTmfppwbf11wCWpRiSNYiPgZsKVUIsI1VMPV2E7RatNpTZUnSUiImVTdZaIiJRN\nSUSqxkoMHW5mW1gYPr+pPoMWG068m+vp9tD51slw5tZkQ91LeZrqC9yIzOwIM5tvYSSAl6IT9+6d\nv7LT9Xb7JOVVHDq8jlWqDr/b6/C84cytSnPiWI2GwZd0KInUMTM7kdAn5WeEIUA2Iwyn8o0abNui\nDk+ZZRka9j06XPXwfcvce5ql91E6qocPtRRgZusQOr8d5+63uPuyaMiI29391GgZM7Mfm9k/zOx1\nM/udma0bPZerThpjZs+b2WtmNiF6bjhhbKpDoxLOgujxOWb2MzP7O6Fn96fNbDczu9/M3jSz+6Le\nycSWz80E12Jmv4i280/CMOjx/RlrYSygty1MTnREkf3exczujTpkvmRmF1qsd3a0T8eZ2bOEgS5z\nvXwfil7zdzPbvtC6Y68fZ2bPRrFMii5VvTfaxxm57VnouX6bhbGvlpjZHyz0/I/vf+54vUu4Ci6+\nrY3N7BEzOym6/wUzmxvF+ZCZ7Rlb9tNmdk8U0yzCyKzF9uEeMxsd3d492qf9ovtfib2fbSUEM8tN\nZvRw9J4fHFvfiWb2SnS8x5bYbof30Mw+S7gCcNdovUuiZfe3MM7YW2a20MzOjK0n99k8xsyeB2ab\n2epmdk30OV4afda6MnaaVEuSURr1l70/wpDpK8gbCThvmfGEXq39CIPY/Rr4bfTcFpQeJv5M8obu\nJvRefo4wb0UPwgirpYaHj4/6+j3CXA+bEIa3vpswnEJXh2MfDOwSva4/oRfz+NjzKwkDCH4y2q9i\nw25/osj6uzKk93rAKMKoBb0JPaNnljheq+WOCSGhPA18J1p2E0LP6OHR/b2j+32j+/cSRoLtSehV\n/3b++xPb7lnAf0e3c/PfnBPdn8Sq0WTH0nEk2i1j9/eixJDhedss+h7S9WHwt6D98P1rUGKIev2l\n+6eSSP3qC7zu7UfizPdd4Kfu/pK7ryCcXA6y9tUqZ7n7B+7+CGESnB2jx+Nj/eQ4cKW7Pxltdx/g\naQ916yvdfQZhAMNC1WmHEE5eL7r7UsJ8IfH1rwS2N7Ne7v6Kh2FEOnD3B939vmh7zxOGP9kzb7Gz\n3f1Nd/+AMGzDpe5+vwfTCYmh1Mik53oY6O4J4FHCkPbPufvbwJ2EExjuvsTdZ7r7cnd/N9qneCzt\njpe7fxQ9vi0hMZ3h7rmRro8C7nD3P0brnk0YqmJ/M9ucMM/G6e6+wsMsoX+g4/uTc08sjmGEOShy\n9/eMnk9qBWHY8o/d/U5CiarYKMfF3sMOcbr7Pe7+eHT7UcLES/nv40QPJezlhCFT+hKSlLv7Ag/D\nmkjKlETq1xvA+la6nn0LYKatGm76CcKsd5+KLVNsmPhi4lfrJBkePmdjigyj7l0Yjt3MBkbPL7Yw\nts/P6Ti0d3w7JacQKKLUkN5tw3Kb2ZoWhvJ/LorlHmAds3ZtRflXNxmh5PYCYYTZeJwH58W5O6HT\nXj/CQI3LYsvHh/nO93+EwTI3JIw3Nh3YzMz6EkpkXZmHu9SQ4W268h5Cp8Pg58SPXVeGqJcaUhKp\nX/cSflGPKrHMQkL1SHzI6TU9jLDcmVKDweV0ZXj4osOoQ5eGY59GSIZbufs6wE/o+DnuzrDbXXES\nYSj/XaJY9qRjCa7QIIVnEn4E/Db2I2AhYdKmeJxru/u5hGO3rpmtGVtP/wLrDhsIc3s8QJiN79Go\nFDo3ivcf3n7QzIop8R52aRj8+Cpj667EEPVSBUoidcrd3yIM73yxhaHM17QwjPPXzGxqtNivgSlR\ndQhmtoGZJb1y62Vgi7xf1dD+BJlkePic64ETzGwTC437P25bYdeGY+9NGMH0/ajRttiIuzldHXa7\nkPwhveOxLAPesjAlwJl0VKjKaQVhhNi1gOnRMb4G+LqZ7WPhIoQ1LPTT2MRXDa1+VvQef5FwEi3l\nHsLcJrmqqzmEof5LVWW9Qhjdt8s6eQ+7NAx+kfV3e4h6qQ4lkTrm7ucTBk38KfAq4dfscawaRPEC\nQmPkLAvDjd9LaJRuW0WJ1d8Q/X/DzOYXeo0nGx4+538I1REPE06IN8XW1ZXh2E8mnHDeJrSHzMjb\nj3b75CWmECgi8ZDehOlsexH2fS6hvaTU8OjxuFYQpuT9FGEKgBeBkYSG8Nx7eRKrvqNHEKYJXkL4\n8XBViX2AkCx6s6rq6q+EpBWvysrvszIRuCqqTjuowPOllHoPuzoMfi62uKRD1EuNaewsEREpm0oi\nIiJSNiUREREpm5KIiIiUTUlERETKlunOOmamVn8RkTK4e00G08x0EgFYcdvNaYeQiknXzuCMIw9L\nO4y6oePVdaWO2bI99q5xNNk3ZcoUJkyYkHYYifTp06dm28p8EhHZ8OAjWbps1Ygf6/bqxas3XJti\nRCKSoyQimbd02bJ2Pc9s2bKiy0r3qRQiXaGG9Yzac/vt0g6hruh4dV2hY6YEUtywYcPSDiGTMt1j\n3cy8WdtEZJWeI0a3L4nQvG1l1aQE0jj69OlTs4Z1lUQk89bt1attaFyL7otINqhNRDJPjegi2aWS\niIioKkvKpiQi0uSUQKQ7lERERKRsSiIiTUylEOkuJRGRJqUEIpWgJCIiImVTEhERkbIpiYg0IVVl\nSaUoiYg0GSUQqSQlEZEmogQilaYkIiIiZVMSERGRsmkARqkazUiYLarKkmpQEpGq0YyE2aEEItWi\n6iwRESmbkohIg1MpRKpJSUSqRjMSijQ+tYlI1agRPX0qhUi1qSQi0qCUQKQWlERERKRsSiIiDUil\nEKkVtYk0GXUAFJFKUhJpMuoA2PhUCpFaUnWWSANRApFaUxIREZGyKYk0GXUAbFwqhUga1CbSZNSI\n3piUQCQtqZVEzGwzM7vbzB43s8fM7IS0YhERkfKkWRJZAfynuz9kZr2BB8zsT+7+ZIoxiYhIF6RW\nEnH3l939oej2u8CTQL+04hGpV6rKkjRlok3EzLYABgHz0o1E6lWzdqJUApG0pZ5EoqqsG4HxUYlE\npMvUiVIkHakmETPrCdwEXOPutxRaZtK1M9pu77n9duy5w3Y1ik4k21QKkZzW1lZaW1tT2ba5e+dL\nVWPDZgZcBbzh7v9ZZBlfcdvNtQ1M6lLPEaPbl0SARv/sKIlIMX369MHdrRbbSrOz4e7AUcCXzGxB\n9Dc8xXikjjVbJ0olEMmK1EoiSagkItKREoh0pllKIiIiUueURETqiEohkjVKIiIiUrbU+4lIfVpz\nxGhWxO73BN5X+5VI01ESkbKsgA6X1Ep1qSpLskjVWSJ1QAlEskpJRCTjlEAky1SdJWXpSfsqrJ5p\nBSIiqVISkbKoEV1EQNVZIpmmqizJOiURkYxSApF6oCQikkFKII3jhReMuXNb2u7PndvCCy90/aL4\nSq2n0tQm0mB6jRjNR7H7qwHL1H4hkpqFC3tw5JFrMH36cgDGjFmDa69dzqabfpzKeipNSaTBfIQ6\nAdY7lUIay267fcz06cvZf/81Abj99vfZbbeun/grtZ5KU3WWiIiUTUlEJENUCmk8c+e2MGbMGtx+\n+/vcfvv7jBmzRru2jVqvp9JUndVgVqN9FZbeYJF0bb75Sq69dnlb1dO11y5n881XpraeStPMhiIZ\noVKIVIpmNhRpMkogUq+UREREpGyqMm8wGx58JEuXLWu7v26vXrx6w7UVX0cltiMi9U9JpMEsXbas\nfT+R2Im+kuuoxHYkUFWW1DNVZ4mkSAlE6p2SiEhKlECkESiJNJh1e/XCoO1v3V69qrKOSmxHROqf\n+omIpEQlEakW9RMRaXBKINIolEREakwJRBqJkoiI1JWsTs7UrNRPpISsdajLWjzSdSqFdF9WJ2dq\nVkoiJWStQ13W4pGuUQKpjKxOztSsVJ0lIiJlUxIRqQGVQionq5MzNatUq7PM7Apgf+BVd98+zVgK\nWbdXr3ZVRml3qMtaPCJpyOrkTM0q1c6GZjYMeBeYXiiJqLOhNAKVQqTWmqazobu3AkvTjEGkmpRA\npNGpTURERMqmJCIiImXLfD+RSdfOaLu95/bbsecO26UYTbrWHDGaFbH7PYH389qMstQhMUuxpEFV\nWVIrra2ttLa2prLtzCeRM448LO0QMmMFtO9sWGCZLHVIzFIstaYEIrU0bNgwhg0b1nb/nHPOqdm2\nO00iZrYGcCCwRWx5d/dJ3d24mV0H7An0NbNFwBnu/pvurrfZLP/wQ077zdVphyERJRBpJklKIrcC\nbwIPAMsruXF3P7yS62tGr731Fsdfchk3/f3eVY8BG6QXkog0kSRJZBN337fqkUinetK+CqsF2PSb\n32blyvYdrTaM3U6zQ6I6R4o0vk47G5rZZcBF7v5IbUJqt211Nizhx1dM55c331LwuZNGH8A5x4yp\ncUSiqizJgkx0NjSzR83sUeCLwANm9kzuMTOreUKRjo7/xv5lPSfVoQQizahUddbXo/9OxwuBsjsx\nexPZZP2+7Pq5rbn3yafbPb7r57Zmk/X7phSViDSToiURd3/O3Z8Dfpa7HX+sVgFKaUO3HpjoMenc\nrAcWMOrcyYw6dzKzHljQpdfmSiGadU+aTZI2kQXuPih2fzXgEXffpurBpdwmUqnOcpVaT6HOhu/c\negP3P/sPtt50EwA+c/R/8M4HHxTdVjVjye/4WE9mPbCAMRf+gsN/sT4A1538OtOPP5l9hgzq5JVB\nLonMndtScNY9TZoktVTLNpGi1VlmNgE4DehlZu/EnloBXFbtwLKgUp3lKrWeQp0NW1pa+MJnt257\n7J0PPii5rWrGUs+m/fk2Dv/F+nzp6PVWPXbFbYmSSLwtRLPuSbMpVZ01xd3XBn7h7mvH/tZz9x/X\nMEYREcmoUiWRwdHNG2K327j7g1WLSqTGxn1lBGNO/kXb/VCdNbbT1+VfkRWfdQ9UnSWNr9TVWecT\naix6AUOA3GW9OwDzgV2rG1r6KtVZrlLrye9s2LOMbdUylnqyz5BBTD/+ZKZdcRsA048f22lVVqFL\nejXrnjSbJA3rNwNnuvuj0f3tgLPc/cCqB6fOhpJR6hMiWZaJzoYxn80lEAB3fwz4XPVCEhGRepFk\n7KxHzOxy4BpCDcYRwMNVjUpEROpCkpLIt4AngPHACdHtb1UzKJEsa7SqrPvv78Fll61q1brssp7c\nf78mPZVkOi2JuPsyQiP7+dUPR0pJ0lGw2WcTrLZGSyAACxa08KMfrc6KqPfohAmrc955H7Dzzrog\nQDpX6hLfG9z9YDN7jI5jZbm771Dd0CRfko6CzTybYLU1YgIBOPbYFaxYAaedtgYAZ5+9nGOPXdHJ\nq0SCUiWR8dH/EbUIRERE6k/RJOLuL0U39wbucfdnaxOSiNTSZZf1ZMKE1Tn77DDe14QJq9OzJyqN\nSCJJrs7aHLjUzD5N6GT4V6DV3R+qamTSQZKOgppNsDoatSoLYNCgjznvvA/akkbPnuExkSQ67WzY\ntqBZL+BY4GSgn7u3dPKSblNnQ8mKRk4i0ngyMYpvjpmdDuwG9AYeAk4C/lbluEQyQwlEpLgk1Vmj\nCSN/306oyprr7h+UfolIY1ACESmt0x5F0YRUewP3AV8FHjMzlUSk7nRn5sI0zJ49m8OOGsVhR41i\n9uzZHZ6vx1kUO9snqT9JqrO2B4YBewA7AS8QSiSZVcsZCWvZuU8dCcuXP3PhmJN/0enMhWmWQmbP\nns13xn2TI6aGSbK+M+6bXD7tavbee1VMCxf2KDiL4qabZrNRPMk+Sf1JUp11NtAK/Ddwv7tn/rq/\nWs5IWMvOfepIWL7uzFyYhsuvvJgjpq7XLt7Lr7y43Qm33mZRTLJPUn+SDHuizobSdNQWIpJMkpKI\nSN1LMnNhy4Dlbbd7L76NdzdO7/fTd8Z+n++M+2bb/d+euoTLp32/3TL1Notikn2S+tOQSaSWMxLW\nsnOfOhKWr9jMhfHEkS/NRLL33ntz+bSrufzKiwG4fNr3O1T71Nssikn2SepP4s6GaVBnQ6m0Ukmj\nmDRLJCLlyERnQzP7Q4nXubt/owrxiFRUOUkjX+/FofSiZFJZy5cv54wzzgBg0qRJrLHGGilHJOUo\nVZ31y5pFIVIhlUgaUn2vv/46J554IrfccgsAr7zyCr/85S9Zf/31U45MukrVWVK30koYKpF09MIL\nxs03P8jc+yYBsNsuZzB69GA23bTj+eXOO+/k8MMPZ+XK9u03PXr04LrrruNrX/taTWJuZJmozsox\ns4HAFGBbIFfedHffspqBZUEtOy2uOWI08Q44PYH3lUA7yEJJI+0rt7Lo5psf5PTTt2HkKaEPyOmn\nbwM8yAkndOyHM3fu3A4JBGDlypXMnTtXSaTOJJlI+TfArwnjZ+0FXAU0RTfpXOe+3N/SbnZaLLWe\nFbHnPbovIWnE/7Ii104iwdz7JjHylOO4Zer13DL1ekaeclxbqSTfuHHjiq6n1HOSTUmSSC93n02o\n+nre3ScC+1di42Y23MyeMrNnzezUSqxT6ltWk0YhvRffpmRShn79+jF06NAOjw8dOpR+/fqlEJF0\nR5J+IsvNrAX4h5n9AHgJWKu7G47WeRFhcMcXgfvN7Pfu/mR31y31JevJojOq3gptIKefvg0HnHoI\nALeeewmTJz9RdPmdd96ZefPmdXhM6k+SJPJDYE3gBGAy0Ac4ugLb3gX4h7s/B2BmM4CRQGaSSC07\nLfYELO9/Y61vAAATQklEQVR+o6r3pFFIsyeS0aMHAw8y974lAEye/ET0WOELdyZPnswBBxzAgAED\nAHj22WcZMmRIjaKVSurKzIZ9ANz97Yps2OwgYF93/4/o/lHAUHc/PraMrs6qc42YMDrTzMlEsiFr\nV2ftDFxBKIFgZm8C33b3+d3cdqLsNenaGW2399x+O/bcYbtublaqqRmThkjaWltbaW1tTWXbnZZE\nzOxR4Dh3b43ufxG4xN136NaGzb4ATHT34dH904CV7j41toxKInVAiaMwlUgkLbUsiSS5OuujXAIB\ncPe/AR9VYNvzgQFmtoWZfQI4FPh9BdYrVdYyYDmzX57H6IvOYvRFZ3FX68Nlr2vKtJn033cc/fcd\nx5RpMysYZUd3tT7MyPFTGTl+alkxL1q8Jq3zN2i73zp/AxYtXrPo8tW+civpzIZJZhMstUwtZ1BM\nsq0szehYb/FWQ5Ikco+ZXWpme0V/06LHBpvZ4HI37O4fAT8A7gKeAH6X9MqsDQ8+kp4jRrf9bXjw\nkeWGURNJ4q3UMtWQf9ntXa0Pc9SEC+h/2BL6H7aEoyZcUNZJecq0mUy54iZGTuzDyIl9mHLFTVVL\nJJWI+bkX1+LAH3yROfM2ZM68DTnwB1/kuRdLX6hYzUSSm9mwtbWF1tYWjjxyDRYubP+Vzs0muOXo\nZ9hy9DN8Z9w3OySJzpZJsp1a7lMt42m0eKshSXXWHEq0X7j7lyocU3zbBauzeo4Y3X6GPyDL1V5J\n4q3UMpVSqopq5Pip9D9sSdsMdXdftYTnZ6zHrRd0ratP/33HMXJin3bruXXi2zx/17TyA69yzHPm\nbciXx3wFgL9M/zN7DX018WurUb3V2trSbmbDYcPazyVy2FGj2HL0M+32+183D2TGNTO7tExn26nl\nPtU6ns5kMd5MNay7+141iENSpDaN2mj2y4ClMSW5Omsj4OfAJu4+3My2AXZ19/+tenRSFd1NGscd\nNJyjTrmg7f51p7zGNVOO6PJ6vnvAPkw54aa2+1ec8CITjjmwW7EVU4mYW+dvwCHjd+cv0/8MwCHj\nd+emi/7GsJ1eS7yOSiaSJDMbJplNsLNlajmDYpJtZWlGx3qLtxqSVGf9kTB+1k/cfQcz6wkscPeq\nX2tbrDqrUgMj1kqSeCu1TCHVKGnc1fowl9z4RyCcoPcdtmNZ65kybSaX3jILCEllwrhRFYsxX3dj\nXrR4TZ57ca22pNE6fwO22OQ9Ntv4/bLi6W4yeeEFY+HCHm0no7lzW9h885UdRs6dPXt222yC3xlb\neDbBUssk3U4lJNlWLeOp13hrWZ2VJInMd/edzGyBuw+KHnvI3T9f9eB0iW9ZVD1VP1S9JdWQtUt8\n3zWzvrk7Uf+Ot6oXkpSjXgYtlPY0gKPUuyRjZ50E/AHY0szmAhsAB1U1KilJiaKxqMFd6lmSq7Me\nMLM9ga2jh552d013UWONmjgq1baSpVg6W0/htpW/sO7gL3cj+uKStIk0miy1mzS6otVZZraLmW0M\nECWNIYQZDn9pZuvVKL6y1FtnxELqaV6NclWq02KWYkmynmKdFqsxP0mSzoaNqNE7+GVJqZLIpcBX\nAMxsD+AcQg/zQcBlZLhKKzeTYI6VOSNhrTRqkujMJTf+kcPP3aCtkxvAJTP+mEpppFKxJFnPsJ1e\n4/oL/t6u02L8MuFKVm9dfuXFHDF1vXbxXH7lxQ1fGtltt4+ZPn15uw5+jXJJbdaUSiI93H1JdPtQ\n4FJ3vwm4yczS+bnYQJo1cUgyaieRelEqibSYWc+oKmtv4NiEr5MilDjaq1SnxSzFkmQ9STst5qq2\nupNMknQ2bESN3sEvS4r2EzGznxDmUn8d2AwY4u4rzWwAcKW771714MrsJ5KFzohKGMmoYT1Zp8Xu\nJBI1rDdfw3pmOhua2a7ARsAsd38vemwg0NvdH6x6cHXW2VCJQ6pJ1VuSVGYGYHT3ews89kz1wqkf\nShhSa2onkSxS20ZCShqSBZVoJxGpJF04XUKj99OoF53NftjVGQe7o7uzI0KyeOc9vB4XXTOg7f5F\n1wxg3sOrLtOt9XApSWZHrDe1mnFQMxs2ifzOfUoc2ZBk9sNyZhwsR6U6JCaJ9/5H+zL+Z0P41ZUD\n+dWVAxn/syHc/2jfdsvUKpE0aofFWnVIbPSOj52O4pumajesK1FkX9LZD7sz42BSlZodEZLF+6sr\nB3Li2UMAOP+0B/jh2OLNkdWs3koy82G9qtWMg009s2GjUeIQEamchk0iShaNIcnsh5WYcTCJSnVI\nTBLvRdcM4KRzBnP+aQ8AcNI5g1ltNecHRz1bcJ3VbHBv1A6LteqQ2OgdHxuqOkuJozF1NvthpWcc\nLKUSHRKTxDvv4fW4/9G+bUnjomsGsPP2bzB0xyUF1xlXjUTSiB0Wa9UhselnNkxTkiSixCHSkS4B\nbm5qEylCCUMkGfUnkVrJ/HVmuuRWpHyafleqLfNJRCqrEp3lsmbR4jU59qdPtnVIPPanT5bV2bBW\nxyZJZ8NKdqBUIpFqUhJpIlmaSbCSJl/8AJffeDxDDt2HIYfuw+U3Hs/kix/o0jpqeWySdDasdAfK\nasyaKAJ11iYi3ZOlmQQr6a77/5sDTnmKW6ZeD8ABpx7CXb+bBUwr/cKYWh6bzmY2TLqMSBaoJCLS\nRFQikUpTSaSJZGkmwUrad+cTuPzc4zng1EMAuOXci/nOQRd2aR21PDZJOhtWuwOlhpWXSsl8P5GV\nT/827TAaSpZmEuyu5R98yKnnXce77/fEfXv+/OAlQEgqp39/SJc7G9bq2CTpbFirDpRKJI1JnQ0j\nSiJSzGtL3ub7E6/gxrvuA+Dg4UO56MxvscF6fVKOrD4pmTQWJZGIkogU8oe/PMCo75/PypXtP7s9\nehgzLz6Rr395SEqR1TclksZRyySihnWpO63zn+6QQABWrnRa5z+dQkSNQQ3uUo5UkoiZHWxmj5vZ\nx2Y2OI0YpPoq1Xkvfz3jjx5edNmH/vmvhuj7khZdvVV5mtmwOh4FRgF/TWn7UmWV6rxXaD2PPbOI\n3QYN7LDsRlt9gs+OfadhOlFKY2j0mQ1TucTX3Z8CMGucbCztVarzXrH1DP38Vsxd0H6mv52/sU7b\nco3QiTJNGsCxcnbb7WOmT1/ebmbDRplLBNRPROrUuT86goOHD2XrT2/MEaddwNq7vsYhEzdKO6yG\no/4k0pmqJREz+xNQ6Fs9wd3/kHQ9Ey+8se32Xrtsw15Dt6lAdFJtleq8V2w9LS09+MLnBwDww8NH\ncNSEC9hoq9W7tS0pTImke2oxs2Frayutra0VW19XpHqJr5ndDZzk7g8WeV6X+NaxSnXeS7KeRupE\nmWVKJl2nmQ2rufGQRE5294JDriqJiGSPEkn2NXw/ETMbZWaLgC8At5vZnWnEISJdp0uAJU491kWk\nbCqVZJPmWG8ylRxsL0ttA5Voy6jVQIS1tGjxmlx/x2L++sivAdhjh+9xyH4bp7ZP3TnGanSXxunx\nUscqNYtdlmYuTBJLkmUqPcNfFlx/x2JOOe9AWnYcSMuOAznlvAO5/o7FqcXT3WOs6q3mpuqsjJgz\nb8N2s9jtNfTVLq9j5Pip9D9sSVuHu7uvWsLzM9bj1gtOrWislYolabyVODZZMnL8VFp2HNhuJsaP\nH34mlfcppxLHWCWS7Gj4hnURaTwqkTQntYlkQKVmscvSzIVJYkmyTLVn+EvDHjt8j1POPbBtJsZb\nz72Yc390U2rxVPIYa7iU5qPqrAxQw7oa1uu1Yb0UJZL0NE1nw840SxIRaVRKJOlQm4iINAS1kzQ+\nJRERqSpNdNXYlEQyolKzAGbJlGkz6b/vOPrvO44p02amHY6kTImkMSmJZECWOglWypRpM5lyxU2M\nnNiHkRP7MOWKm5RIRImkAekS3wyo1CyAWXLpLbM45r83abdPl06cxYRxo1KMSrJAlwE3FpVERCQV\nKpU0BpVEMiBLnQQr5bsH7MOUE1Z1oLvihBeZcMyBKUYkWaQBHOuf+olkRJY6CVbKlGkzufSWWUBI\nKqrKklKUTCpHnQ0jzZRERJqdkkjlqLOhiDQd9SepT0oiIpIpSiT1RQ3rdaTe2k3qLd5a0rEpTQ3u\n9aNpSyJ9Bx9Dj62PaPvrO/iYtEMqqd46JNZbvLWkY5OMqrfqQ9OWRJa+t5z4JQX23vLUYkmi3jok\n1lu8taRj0zUqlWRb05ZERKR+qESSXU1bEqk39dYhsd7irSUdm/JouJRsatp+In0HH8PSWBXWumut\nwRsPXlGVbVVKvTXG1lu8taRjUz4lkc6ps2FEnQ1FpBglk+LU2VBEpBNqJ8kGJRERqVtKJOlTw7pI\npFLtFGrvqC01uKdLJRERKtcBUB0J06NSSTpUEhGhch0A1ZEwXeqYWHsqiYhIQ9FwKbWlkogIlesA\nqI6E0mzUT0Qkoob1xtSM1VsN39nQzM4DRgAfAv8EvuXubxVYTklERLqt2RJJM3Q2nAVs6+47As8A\np6UUh4g0AbWRVE8qScTd/+TuK6O784BN04hDRJqHGtyrIwtXZx0D3JF2ECLSHJRIKqtqScTM/mRm\njxb4+3psmZ8AH7q7Gj5EpGaUSCqnapf4uvtXSz1vZmOB/YCvlFpu4oU3tt3ea5dt2GvoNpUIT0Sa\nXCN1TGxtbaW1tTWVbad1ddZw4JfAnu7+eonldHWWiFRdoySTnGa4OutCoDfwJzNbYGaXpBSHiIiq\nt7ohlR7r7j4gje2KiBTTSNVbtZSFq7NERDJBlwF3nZKIiEgeJZLklERERApQIklGSaSO3NX6MCPH\nT2Xk+Kma6EikBpRIOqckUic0Y55IOpRIStN8InVCM+aJpEfzuBenkkhGzZn3RNoh1BUdr67TMeua\nOfOeUKmkACWRjJpzX/sv+HEHDee6U17j7quWcPdVS7julNc47qDhKUWXPfnHSzqnY9Y1ueOlRNKe\nqrPqxL7DduSaKeO5ZEaYMe+aKUeoKkskJareWkVJpI7sO2xHJQ6RDFEv9zqYYz3tGERE6lFDz7Eu\nIiKNQQ3rIiJSNiUREREpm5JIhpnZwWb2uJl9bGaD044nq8xsuJk9ZWbPmtmpaceTZWZ2hZm9YmaP\nph1LvTCzzczs7ui7+JiZnZB2TFmiJJJtjwKjgL+mHUhWmVkLcBEwHNgGONzMPpduVJn2G8KxkuRW\nAP/p7tsCXwC+r8/YKkoiGebuT7n7M2nHkXG7AP9w9+fcfQUwAxiZckyZ5e6twNK046gn7v6yuz8U\n3X4XeBLol25U2aEkIvVuE2BR7P4L0WMiFWdmWwCDgHnpRpId6myYMjP7E7BRgacmuPsfah1PHdI1\n6lITZtYbuBEYH5VIBCWR1Ln7V9OOoc69CGwWu78ZoTQiUjFm1hO4CbjG3W9JO54sUXVW/ahJ79M6\nNB8YYGZbmNkngEOB36cckzQQMzPgf4En3P1XaceTNUoiGWZmo8xsEeGKkNvN7M60Y8oad/8I+AFw\nF/AE8Dt3fzLdqLLLzK4D5gIDzWyRmX0r7ZjqwO7AUcCXzGxB9Kcr3CIa9kRERMqmkoiIiJRNSURE\nRMqmJCIiImVTEhERkbIpiYiISNmUREREpGxKIlLXzOwn0fDcD0fX7+9S4fXvZWYdhp8p9ngFtjcy\nPkKsmc0xsyGV3o5IpWjYE6lbZrYrsD8wyN1XmNl6wOoph9Vdo4A/EEaKBY0NJhmnkojUs42A16Mh\n4HH3Je6+GMDMhkS/4ueb2R/NbKPo8Tlm9quo1PKome0cPb6Lmc01swfN7O9mNjBpEGa2VjTZ07zo\n9d+IHh9rZjeb2Z1m9oyZTY295ttm9nT0msvM7MIoKX4dOC9az5bR4gdHyz1tZl+sxIETqRQlEaln\ns4DNopPrxWa2B7QNlnchcKC770SYiOnn0Wsc6OXug4DjgCuix58Ehrn7YOBMYEoX4vgJ8Gd3Hwp8\nmZAE1oye2xE4BNgeONTMNjGzfsBPgaGEITU+C7i730sY9+tkdx/s7v+K1tESrfuHUWwimaHqLKlb\n7v5e1F4wDPgS8Dsz+zHwALAtMDuMnUcL8FLspddFr281sz5m1gdYB5huZlsREk3PLoSyD/B1Mzs5\nur86sHm0nj+7+zsAZvYEsAWwAXCPu78ZPX4DEC/55A+2eXP0/8Ho9SKZoSQidc3dVwL3APdE84Yf\nTUgij7v7bl1Y1WTCCX+UmfUH5nQxlNHu/mz8ATMbCnwQe+hjwncuv50jP2nkP59bR+71Ipmh6iyp\nW2Y20MwGxB4aBDwHPA1sYGZfiJbraWbbxJY7NHr8i8Cb7v420IdVpZWujmx7F3BCLK5BuZsFlnXg\nfmBPM/ukma0GHMiqxPFOFItIXVASkXrWG7jSzB43s4cJbQsTo4b2g4CpZvYQsADYNfa65Wb2IHAJ\n8O3osXOBs6PHW2hfGih0hZTHHp8M9DSzR8zsMeCsAsuseqH7S4Q2l/uAvwH/Bt6Knp4B/MjMHog1\nrOdvVyQzNBS8NBUzuxs4yd0fTDmOtaI2ndUIbR7/6+63phmTSDlUEhFJx0QzWwA8CvxLCUTqlUoi\nIiJSNpVERESkbEoiIiJSNiUREREpm5KIiIiUTUlERETKpiQiIiJl+38fDGaT/PtBMgAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10bc40a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(1)\n",
    "plt.clf()\n",
    "plt.imshow(Zs, interpolation='nearest', extent=(sl.min(), sl.max(), sw.min(), sw.max()), cmap= plt.cm.Pastel1, aspect='auto', origin='lower')\n",
    "for j in [0,1,2]:\n",
    "    px = X_train4[:, 0][y_train == j]\n",
    "    py = X_train4[:, 1][y_train == j]\n",
    "    plt.scatter(px, py, c=colormarkers[j][0], marker= colormarkers[j][1])\n",
    "plt.scatter(centroids_s[:, 0], centroids_s[:, 1],marker='*',linewidths=3, color='black', zorder=10)\n",
    "plt.title('K-means clustering on the Iris dataset using Sepal dimensions\\nCentroids are marked with stars')\n",
    "plt.xlim(sl_min, sl_max)\n",
    "plt.ylim(sw_min, sw_max)\n",
    "plt.xlabel(\"Sepal length\")\n",
    "plt.ylabel(\"Sepal width\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Repeat the experiment, using petal dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KMeans(copy_x=True, init='k-means++', max_iter=300, n_clusters=3, n_init=10,\n",
       "    n_jobs=1, precompute_distances='auto', random_state=33, tol=0.0001,\n",
       "    verbose=0)"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_petal = cluster.KMeans(init='k-means++', n_clusters=3, random_state=33)\n",
    "clf_petal.fit(X_train4[:,2:4])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "print (y_train4[clf_petal.labels_==0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1\n",
      " 1]\n"
     ]
    }
   ],
   "source": [
    "print (y_train4[clf_petal.labels_==1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 1 2 2 2 2]\n"
     ]
    }
   ],
   "source": [
    "print (y_train4[clf_petal.labels_==2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the clusters "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAEoCAYAAACKM4weAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe8XFW5//HPQ4gklNAJEEpEaiiKSEIxv+SKF0ORpig1\nInoFlAtXkJJYCKgJYLuiiFhQECQiEKRKiBI4Gi4IhCYEgpQUQgCDFEMgJM/vj7XmZJ/JzJyZM7Nn\n75n5vl+v8zoze3ZZu8x+Zq2111rm7oiIiKRhlawTICIi7UtBRkREUqMgIyIiqVGQERGR1CjIiIhI\nahRkREQkNQoyGTCz5Wa2VQ7SMc7Mfp51OmplZhPM7DdN2tYWZvaGmVmd6xltZnMbla68M7NbzezY\nrNNRDzN7zsz2qXLe48ysK/H+DTMbmlbaapXldz3TIFN8Es3sCDNbZGYjs0xXKzCzX5vZN+tZh7tP\ncvf/alSa0lDm5tywxl29HUd3n+Pua3kTG5QV37BacTvuvr+7N/yHQLwelseb+OtmNsvMjqtiuaFx\nuVrueU4fr7V4zTzXl2XTkOV3PeucTPdJNLPPAD8G9nf31L9gnc7M+mWdhjrUlasoUvZGYmarNnA7\n0jjz4018EHAW8HMz26HKZRt57Ug13D2zP+BZYB/gBOBl4IMV5l0OnATMBl4HzgPeB9wD/AuYDPRP\nzH8g8BDwKvBXYOfEZ2cDT8f1/B04JPHZccBfgO8Ai4BngDFFn/8jLvsMcFSZ9K4CjE9s535gSGJf\ntoqvpwOfK1p/V3xtwA+AhcBrwCPAjsAXgHeAt4E3gD/E+TcFrgNeimn778R6JwDXAr+J6/pcnPab\n+PnQmK6xwPPxfIxPLD8QuDwek8eBM4G5Fc7XXsDf4rm5D9gz8dn0eP7+Eo/N7cD6JdaxBvAWsCzu\n5+vAJsA5wO9iel4HHgN2SyxX9jiU2MavgG8WHYPj4zGYDmwZp61S4/kfCPw6Hq+/A2ckjxdlrkFg\nh7jP78Z9XhSnHwDMjOduDnBOYl0DgCuBVwjX+33ARvGztYFfAi8A84BvEq7NktspsR/PAfsUXUe/\nqbDdDYuva3r/Tr0XuDseizuAiwvbKJGe0RRdd/E8H0b4vhSO6yvxGlk3zjMnnsc34t8Iwv3jz3He\nl+O+rF10f/pImXSsD9wYz8e98bh2Fd2vCt/xXwM/AW6N2+4CNgZ+GI/bE8AHqrl+4/G/hvLX/lnx\nPL8OzCqkP3ne4vuDCNfdq8CdwPZF5/x04GFW3FtXi59tANwcl/tnPG9W8T7fl+DQqL94Eq8DXiQR\nBMrMuxyYAqwJDCPcYP9MuDEMigdsbJx3V8KNefd44Y2N2+ofP/8ksHF8/SngTWBw4gvxDuEmbMCJ\nhF9OEG56rwHbxPeDgWFl0nsGISgU5t0FWK/EBXgncHxiueNYEWQ+RghOg+L77RLp/hVwXmK5VYAH\ngK8BqxK+uP8A9k1cZO8AByVuEOewcpC5FFgtpncJsF38/PyY1rWBIXHf5pTZ9/XiRXh0TNcRhJtL\n4Qs/nfBjYeuYjjuBSWXWNYqVbyoTCDfIMfEcTQTuqeY4lFh/qSDza0KQWC0xbZUaz//5wF3AOsBm\nhJvBnMTnla7Bz5C4YSWOw47x9c6E78zB8f0JhBvegHg8dgXWip9NAS6J+7Mh4Yb4hXLbKfMd/Uji\n/TnAFVVst/u6psJ3Kn5+D3BhPF97x2N8RZn0jC5cD/GcHEq4F2wDnArMINyk+wM/BX4b5+3xYyFO\nex/hR25/ws3zLuAH5fa9KB2T499Awg+/ecDdRferZJB5OR6f1YA/EW7kx8Tj8U3gzzV8j8td+9sR\ngmnhutoikYbkd31bwvW2D9CPcK+aDaya2O//IwTCdQk/Kk+In00iXE/94t/evd3nsy4uM+CjhIvs\nsSrmv9Dd33T3x4FHgdvc/Tl3fx24jXASIfzSv9Td/+bBFYQLcU8Ad7/W3V+Mr68hHOARie087+6/\n9HBUrwA2MbON4mfLgZ3NbKC7L4xpKeVzwFfdfXbcziPuvqiKfUxaCqwF7GBmq7j7k4V0R8ms/+7A\nBu7+LXd/192fBX5BuMEXzHD3G2N6llC66OBcd3/b3R8h/JJ5f5x+ODDR3V9z9/mEX2Hlih4OAJ50\n96vcfbm7Tyb8qjoofu7Ar9z96ZiOa4APlFlXuW10ufsf4zm6MpHOao5Dbya4+1vu/naJz6o9/4cD\n33b3f7n7PIqOVy/X4Er77O53ufvf4+tHCTe4UfHjdwi/rLeJ1/tMd3/DzAYD+wFfjvvzMvC/iWPR\nl6IjSyxXcrtlliv5nTKzLYAPAd+I5+uvhMBVKW2bmtmrhBv314Fj4/fsROBr7v6Cuy8FzgU+Geth\nSh3Tf7j7n9x9qbu/Qig1GFU830oHIBQ1HxbT/FY8L5dXSLMD18fj8zYh8P/b3a+Mx+MaVty7qrl+\ny137ywhBbEcz6++hPvGZQrITy38auDnu+zLgu4RguVdinovc/UV3fxW4iRXfz3cIpQlD3X1ZPF8V\nZR1knHBhbEc4kACY2d9jxd4bZrZ3Yv6FiddvFb1fQvilCeFXy+lm9mrhj/BrcpO4/rFmNjPx2U6E\nL0tB943c3RfHl2u6+78JJ+hE4AUzu9nMtiuzb5sTfoH0mbv/mVBPdTGw0MwuNbO1ysy+JfHLl9iv\nccBGiXnmVbHZZBBbTMg5Qvh1mKyAr7SuTQm/qJKej9NLbeetxHaqlTz3i4EB8WZSzXHoTcmnwGo8\n/8XHq8fxqOIapGj+EWZ2p5m9ZGb/IuQiCvP/hlDkONnM5pvZBbE+aUvCr/QFie38lJCj6StPvC63\n3VJKfqcIx2lR/LFR0NtTeC+4+7ruvr67fzAGaQj7OyWxr48TigMHl1qJmQ02s8lmNs/MXov7U/Yc\nJGxIyGWUPb8lvJR4vaToffL6r+b6LXntu/vTwP8QcjsLzexqM9ukRFp6fD9jsJpLKKEoKPf9/A6h\nOHKqmf3DzM6qsM9A9kEGwgHbBxhpZj8BcPcdPVTsrVVNpIySF/8cwq/IdRN/a7r778xsS+BnwJcI\nxVfrEnJRVf2qc/ep7r4vISs5Cyj3WOBcQnFQb/7NiuBIXG9yez9y9w8Rigi3JWRtYeXK6jnAs0X7\nPMjdD0zMX7xM8ftKFhACZ8Hm5WYE5hO+LElbxum1KpXGSumeS+Xj0Ndthg+qP/8LCMUVBd2vq7gG\nS23/t8ANwGbuvg4hWKwS0/Suu5/n7jsSfo0eSCginkPIwa+fOBZru/vOve1nQvH12X3TqrDdWiwA\n1jOzgYlpW5SbuRdzCHU9yXO/ursvoPS+TiT8+t/J3dcGjqW6e+LLhOBV8vzWqbfrt+I5c/er3X0k\n4fvmwAUlZuvx/YyP529O+e9n9zY9lCR9xd3fRyiZOM3MPlIpTXkIMsSLYB9gjJl9v4ZFreh14f3P\ngRPNbLgFa5jZAWa2JuEL44TKvlXM7LOEX5G9byxk7w82szUIRVn/JlykpfwC+KaZbR3TsIuZrVdi\nvoeAw8xsoJltTShm87i9D8VfsP0Jv1iWJLa3EEi2tbkPeMPMzozr6mdmO5nZh0ocKypMK+caYJyZ\nrWNmQ4CTKX/B3wpsa2ZHmtmqZvZpYHtChWGt214IrG9mg6pctrfjUKzqY1Dj+U8er82A/0581ts1\nuBDYLJ73gjWBV939HTMbDhzFiutktJntHItx3ohpWxaL46YC3zeztcxsFTN7n5n9vwrbKfYQcEQ8\njx8CPtHbdiusayXu/jyh3nGCmfU3sz0JwaqWH0AFPwUmWiiCw8w2NLNCEe3LhKLO9yXmX5NwDl+P\n1/QZVCEWMV0f0zzQzIYR6rfKqeV71pfvcfjAbFsz+4iZrUb4cZG8XyT9HjggztufUMm/hFCfVTH9\nZnZg4Z5GeLhgWZltdMtFkAFw97nARwhlqN8uNUsv07p/qbv7A8B/EYqaFhHKu8fGzx4HvkeoB3qR\n8OX+S6n1lNjOKsCXCRH/n8BIwhNvpXyfcKOZSqjI/DmhgrQ43T8glHMuJFRCX5n4bBDhF+8iQkXh\nK4TsKoQnhobFLPX17r6c8OX8AOGJlJfjsoWbc7n9Kj6G5ZxHKCJ7Nu7T72O6V+Kh7ulAwsX7CvAV\n4EDvWSdV8tyVWNcs4GrgGQttqDapsC+FG0Cl47DSJkqkpdQ8UNv5P5dQRPgs8EdCPUQhjb1dg38i\nPMjyopkVilW+CJxnZq8T6iF+l5h/Y8L5eI1QRDSdUPQD4bp/T5y+KM5XyC2X2k6xrxNuzK8SimGu\nqnK7Sb3loo8m1Jf+k1AJ/jvKXFsllk36IaE+Z2o8TvcAw6G7iO7bwF/jdTSccI4+GNN/E+EhpGqD\n28mEIPUicFn8K3cdlbrG+nr9VjqWqxEq5l8m5BA3IBS19VjO3Z8kPHTwozjvAcDH3f3dMvua3ObW\nhCcA3yAEpYvd/a4yywHx0TORWpnZScCn3P0/sk6LtBcz+x3wuLufm3VapH65yclIvpnZxma2dyxy\n2Q44jfCUjEhdYrHw++K1tR+hrP+GrNMljaEWzVKt9xDKvN9LaKB1NaGBmUi9NibUcaxPqPg+0d0f\nzjZJ0igqLhMRkdSouExERFKjICOZsgpdwlvfes5teWY23cw+14D11D0kgpkdbWa3V/i8o4YwkNp1\n1Je3U5nZUWZ2v4UeFF6IN/a9e1+y1/XWfRPzlLqEb3FlH+nuw3rqW0HoGuhjhfeW0lhI1qThDaT5\nFGTanJmdRmiL8y1C1xSbE7qpOajScg3atsVGW7llOerOPx6uVvhO5u6c5uk8Sk+tcEFLH5nZ2oQG\nZ1909xtiZ37L3P0Wdz8rzmNmdraZPW1mr5jZ78xs3fhZobhqrJk9b2Yvm9n4+NkYQkOvT8cc0sw4\nfbqZfcvM/kpoTf1eM9vLzP5mZv8ys/tiq24S838uvu5nZt+N2/kHoZFYcn+Os9Bf0utm9oyZHVVm\nv4eb2T2xoeoLZvYjS7Rqj/v0RTObDTwZpx1oZg/FZf5qZjuXWndi+ZPMbHZMy3nxEdx74j5OLmzP\nQov/my30ObbIzG6y0Lo8uf+F4/Um4em95LY2MbNHzOz0+H4PM5sR0/mQmY1KzPteM7srpmkqoTFe\nuX24y8wOi6/3jvu0f3y/T+J8ducwzOzuuPjD8ZwfnljfaWa2MB7v4ypsd6VzaGbbE55c3DOud1Gc\n9wAL/bu9ZmZzzOycxHoK1+bxZvY8MM3MVjOzK+N1/Gq81mrps07S4L1006y/1v0jdAe+lET35iXm\nqdQ9+lAqd/9/DkVdshNafT9HGK9kFULnhJW6/b+TFV3Cn0gYW2MIoYvxOwldVtTazf4HCS29Cx1m\nPg6cmvh8OaFjx3XifpUbGuI9ZdZfy7AT6xG6ox8Q578GmFLheK1aOCaEgPMk8Pk47xBCDwpj4vuP\nxvfrx/f3EHrU7U/ojeD14vOT2O65hJ52YcW4R+fH9+cRu7wnMfREYt+3SrwfTbjGJhC6ft+P8ONi\n7RLbLHsOqX14g6H0HJZhABWGHtBfdn/KybS39YFXPHQ5U84JlO8evaBc9//J/uIKHPi1uz8Rt7sv\nlbv9T/oU4eY230MX4xOL1l9VN/vu/qC73xe39zyhW45RRbNN8tAN/9uUHxpij7JHrcphJ9x9kbtP\ncfcl7v5m3KdkWnocL1/RtceOhMD1DXcv9FB+DHCru/8xrnsaod+vA2xFl/lf99B1fRehq5RyRVt3\nJdIxktAdSeH9qPh5tZYSxjZa5u63EXJk5XqnLncOax3eoGCChxz6EmobekCaREGmvf0T2MAql/MP\npffu0ct1/19O8mmjarr9L9iEMt2new3d7FvoKPBmM1tgoQv3b7NyF+7J7WxJhaEhyqg07ER31+hm\ntrqFIRqei2m5C1jbrEddVfHTWUbI+c0j9KeVTOfhRencm9CYcVNCB5pvJeZ/vkL6/4/QielGhH6y\nrgA2N7P1CTm6uyssW+yfRT9kSl4jtZxD6HV4g4Lksatl6AFpEgWZ9nYP4Rf5oRXmqdQ9em/KPb2U\nnF5Lt/9lu8eHmrrZv4QQLLf20IX7V1n5Wk+msezQEGXWX4vTCUM0DI9pGcXKOcBSHR6eQ/iR8NvE\nj4Q5hNENk+lcy90vJBy7dc1s9cR6Ct29r8RDh5EPEMYfeTTmYmfE9D7ttQ+wV5UK57Cm4Q2Sq0ys\nuxFDD0iDKci0MXd/DfgGcLGFLupXt9Cd+n5mVhhnolL36L15ERha9Kscet5Aq+n2v+Aa4BQzG2Lh\n4YOzu1dYWzf7axJ6iV0cK5XL9ZRcUGloiGoVDzuRTMtbwGsWhno4h5WVKtJaShhdcw3giniMrwQ+\nbmb7WnhIYoCFdipDfEWX+efGc/xhwk22krsIY9oUisamE3oXrlRUtpCe3eVXrZdzWNPwBmXWX/fQ\nA9J4CjJtzt2/T+jM8muE0fjmELqNL3RuWbZ79MIqKqz+9/H/P83s/lLLeHXd/hf8nFDc8TDhhpns\ner2Wbva/QrghvU6oj5lctB899skrDA1RRtXDThCGOx5I2PcZhPqaSt3eJ9O1lDDM72DC0A7zgYMJ\nFfWFc3k6K77HRxGGcF5E+HFxeYV9gBBM1mRF0djdhKCWLCorbrMzAbg8Ftd9ssTnlVQ6h7UOb1BI\nW1K1Qw9IE6nvMhERSY1yMiIikhoFGRERSY2CjIiIpEZBRkREUtMWDZXMTE8viIj0gbun2uFpWwQZ\ngOVP/haACT+6lgn//cmMU9N3Wab/zU16a1bRu4kTJzJ+/PgGpCYbSn+2Wj390Fr7MGjQoNS3oeIy\n6bbmgptZc0GpNpIiIn2jICMrUbCRVjRvnjFjRr/u9zNm9GPevPpKgtJYZ6dpuyAzeviwrJNQlzyl\nvy+BZuTIkSmkpHmU/mzVk/45c1bh6KMH0NXVj66ufhx99ADmzKnvFteXdbb6OWi0tmjxb2ZeqJOR\n9DSizkYkTV1d/TjggNBH6C23LGbkyPq7LktjnXkxaNCg1Cv+2y4nIyIi+aEgI1VTXY3k2YwZ/Rg7\ndgC33LKYW25ZzNixA3rUp+RlnZ1GxWXSJyo6k7yZN8+YM2cV9torFGfNmNGPLbZYzmab9f0el8Y6\n86QZxWUKMlI3BRyR1qQ6GWkJKkYTkXIUZKRhFGhEpJiCjDRUIVejgNN51HBRSlGQkdQo0HSWNBpD\nSutTxb80hR4O6Azt3HCxHaniX9qGcjUinaltuvqX/EsGGuVs2k+y4SLA2LEDuOqqJd1tTKQzqbhM\nMqNA017aveFiO1JjzCopyLQ2BRuRbKhORjqCHnkWaV8KMpIbCjYi7UdBRkRyIY+NOfOYpnLymlYF\nGckd5Wg6Ux4bc+YxTeXkNa2q+Jfc04MBnSOPjTnzmKZyak2rKv5FUM5GpJUpyEjLULBpb3kchTKP\naSonr2nNtLjMzC4DDgBecvedS3w+GvgD8EycdJ27f6vEfCou60AqRmsveWzMmcc0ldOXtLZ9Y0wz\nGwm8CVxRIcic5u4H9bIeBZkOpmAj0jdtXyfj7l3Aq73Mlv0zeJJrKkITya+818k4sJeZPWxmt5rZ\nsKwTJPmkwdI6R17bg6ShHfY1770wPwhs7u6LzWw/4AZg21IzTvjRtd2vRw8fxugRikedas0FN6sI\nrY0V2oNcccUSYEVvz5ttlt9Hi/uq0fva1dVFV1dXI5PYq8zbyZjZUOCmUnUyJeZ9FtjN3RcVTVed\njJSkYNOeWqntSr3S3Ne2r5PpjZkNNjOLr4cTguKiXhYT6abiM5FsZVpcZmZXA6OADcxsLnAO0B/A\n3S8FPgmcZGbvAouBI7JKq7QuDZbWXjppcLR22NfMi8saQcVlUgsFmtbWSm1X6pX2vrZ9O5lGUZCR\nvlCwkU7X8XUyIiLS2vL+CLNIalRXI5I+5WRE0FNoeVCq4eHf/rZKVdPmzbOqGy7mtYFjXtNVLwUZ\nkUg9BmSr1KBbM2f2q2ranDmrVD1oV14H98pruuqlin+RClSM1lylGh5WO63c8tVuJw+anS5V/Itk\nTDkbkfqo4l+kF4VAo1xNuko1PBw37h0mTXpPr9OuumpF3169NVzMawPHvKarXiouE6mRgk06SjU8\n7N/fWbrUep22xRbLAapquJjXxpxZpEuNMaukICNZULCRVqc6GZEcU31NPixZsoQzzzyTM888kyVL\nlmSdHCmiOhmROqi+JluvvPIKp512GjfccAMACxcu5Hvf+x4bbLBBximTAuVkRBqgk9rYlGs02OzG\nhLfddhtbb711d4ABmDJlCltvvTW33XZbatutRb3HpB0aaCrIiDRQJwSbco0Gm92YcMaMGSxfvnyl\n6cuXL2fGjBmpbbcW9R6TdmigqYp/kZS0cxFavY0hG+GFF15g++23L/nZrFmz2HTTTVPbdi3qPSYa\nGVNESuqEXE2WNt10U0aMGLHS9BEjRuQmwIiCjEjq2i3QJBsN3nLLYsaOHcCMGf3KTk/T7rvvXtW0\nrNR7TLI4po2m4jKRJmqHIrRyjQahusaQjbRs2TIeeOABttlmGwBmz57NbrvtRr9++bgR19vAUiNj\n5oSCjLSidgg40tpUJyPSxtqtGE2kFAUZkQzp4QBpdwoyIjnQSoGm3saYrdTAsJXSmlcKMiI5UcjV\n5D3g1NsYs5UaGLZSWvMq04p/M7sMOAB4yd13LjPPRcB+wGLgOHefWWIeVfxL28nzgwHtOjJlKa2U\n1lp1QsX/r4Ax5T40s/2Brd19G+ALwCXNSphI1lohVyPSm0yDjLt3Aa9WmOUg4PI4773AOmY2uBlp\nE8mLvAWaehtjtlIDw1ZKa15l3k7GzIYCN5UqLjOzm4BJ7j4jvp8GnOXuDxTNp+Iy6RhZF6PV2xgz\nryNTltJKae2LjmiMWUWQOd/d/xrfTwPOdPcHi+ZTkJGOknWgkfbQjCCT90HL5gObJ95vFqetZMKP\nru1+PXr4MEaPGJZuykQypMHSpC+6urro6upq6jbznpPZHzjZ3fc3sz2A/3X3PUrMp5yMdLRGBZtp\n06bxi19fDMDnj/sSa6+9LzNn9uMLX1gKwM9+1p9dd13G7ruvPI5LGkoVV/Xv7yxdalUVYTWzuKsV\ni9ba/ukyM7samAFsZ2Zzzex4MzvBzE4AcPdbgWfM7GngUuCLGSZXJLca8XDAtGnT+PxJx7LVYU+x\n1WFP8fmTjuXqq2dzxhmrcfHF/bn44v6cccZqzJzZvIrvUu1UZs7sV3XblWa2c1GbmtIyz8k0gnIy\nIkE9OZojjjmUrQ57iv/4zHoA3Hn5Ip65fltG7nkz48YNAGDSpCV86UtLG5LWapVqp1JL25VmtnNp\ntTY1bZ+TEZHGUtsayZu8V/yLSB8kA021uZvPH/clPn/Ssd3vf3vWIg77+HcZP341Jk1aAsD48avR\nvz/ddTRpS7ZTARg7dgDjxr3DpEnv6THtqquWdNeF9LZ8uXnTSGta22olKi4T6QDVBhpV/Dc2rar4\nV5AR6Sh65FmSVCcjIg2lOhtpNgUZkQ6kYCPNoiAj0ibmLlidrvs37H7fdf+GzF2wesVl+hpopk2b\nxhHHHMoRxxzKtGnT+rQO6QwKMiJt4rn5a/CJkz/M9Hs3Yvq9G/GJkz/Mc/PX6HW5WgdLK9VoU4FG\nylHFv0gbmX7vRnxk7D4A/PmKPzF6xEs1r6O3hwPKNdqcfOWU2hMsmVLFv4g0neprpJHUGFOkTXTd\nvyGfOnVv/nzFnwD41Kl7c92P/8LID73cp/WtueDmkrmaUo02f3HJl/qWaGl7Ki4TaRNzF6zOc/PX\n6A4qXfdvyNAh/2bzTRY3ZP3JgFPcaPOjH/1oQ7YhzaXGmFUyM1968/X022ZJ1kkRaWtqzNleVCdT\no2WzB2SdBJG2proaqVVbBRlQoBFJmwKN1KLtggyEQKNgI53o9q6HOfjUCzj41Au4vevh1LZTrm1N\n3hppzptnzJixYpC1GTP6MW9eqqVDUqQtg0yBgo10ktu7HuaY8T9kyyMWseURizhm/A9TDTQFhWCT\nx0aaGq0yex3xCPOy2QP0UIC0vZ9c+0eOvHDD7kaSAD+Z/Ec+NvL9Tdn+r352LkddsF6P7f/i1xdn\n+uTZXnst44orlvQYrbLTx3dptl6DjJkNAD4BDE3M7+5+XorparhCjkbBRkSkearJyfwB+BfwANDy\nd2jlaqRdffGTYzjmzB92v7/6zJe5cuJRmW7/5z/9YYUl0qfRKrPXazsZM3vM3XdqUnr6pNBOphYK\nNNKObu96mJ9c+0cg3PSbVVTW2/azal/TiqNVNlMuGmOa2c+AH7v7I2kmpB4KMiKtQY058yXTIGNm\nj8aX/YBtgGeBt+M0d/dd0kxYLfoSZAoUbESaT8EmH5oRZCrVyXw8/negOBFtk9dUHY1I85XrfFPa\nT9kHxt39OXd/DvhW4XVyWiM2bmZjzGyWmc02s7NKfD7azF4zs5nx72uN2G4xtaeRPGhEQ8rdDhnH\noD2PZdCex7LbIeMakq6Jl0xhy4+dxJYfO4mJl5QfM6bWkTk1pEBnqKZVUo9KfzNbFdit3g2bWT/g\nx8AYYBhwpJntUGLWu9x91/jXkOBWjgKNZKURDSl3O2Qcs+bP5bjvb8px39+UWfPn1h1oJl4yhYmX\nXcfBEwZx8IRBTLzsurKBpt6ROaU9lS0uM7PxwDhgoJm9kfhoKfCzBmx7OPB0zBlhZpOBg4EnipPS\ngG1VTe1pJAuNaEg5e+E8PnfRkB7r+PVp8+pK16U3TOX4onVeOmEq4086dKV5R37oZa754V97jMxZ\ny1g2hUCjYrT2Uqm4bKK7rwV8193XSvyt5+5nN2DbQ4C5iffz4rQeyQD2MrOHzexWMxvWgO1WRbka\nkWwoV9NeKuVkPhhf/j7xupu7P1jntqt5eOBBYHN3X2xm+wE3ANuWmvG8qyZ3vx61806M2qX+pj3K\n1UizNKIh5TaDN+OXp6z43fbLU+az/ZDN60rXCYfsy8RTrut+f9kp8xl//CdKztvIkTmVq0lHV1cX\nXV1dTd1mpUeYpxMCwUBCHUyhncwuwP3uvmddGzbbA5jg7mPi+3HAcne/oMIyzwK7ufuioul9foS5\nWgo0krbaAdDKAAAUgklEQVRGNKTc7ZBxzF4Yisi2GbwZD9wwqe50TbxkCpfeMBUIQadUURmkNzKn\nAk168tIY83rgHHd/NL7fCTjX3Uv/nKl2w+EBgieBfYAXgPuAI939icQ8g4GX3N3NbDhwjbsPLbGu\n1IMMKNCIZEnBpvGybidTsH0hwAC4+2NlngKribu/a2YnA7cTGnz+0t2fMLMT4ueXAp8ETjKzd4HF\nwBH1brceKj4TyU6yrkYBp3VUk5OZDLwJXEl40usoYE13PzL95FWnWTmZJAUaKdbMfsPq3Va55bPu\n+6wWCjT1a0ZOppp2Mp8FHgdOBU6Jrz+bZqJagRpwSlIzBwyrd1vlls9q0LO+Uvua1tBrcZm7vwV8\nP/5JEXVLI9DcAcPq3Va55YFMBz3rK3VRk2+VHmH+vbsfbmaPsfLjxrnqIDNrqqsRyZbqa/KrUk7m\n1PhfZ6xKytV0rmYOGFbvtiotn+WgZ42inE2+VFPx/3lC/2Gzm5Ok2mVR8d8bBZvOo4r//FGwqSwv\n7WTOAz4MvBe4H7gb6HL3h9JMWC0UZESkHAWa8nIRZLpnNBsIfAH4CrCpu/dLM2G1yGOQKVCwEckP\nBZyecvEIs5l93cxuA6YCWwOnA/V1iCQikgE98tx81bT4P4zQvf8thKKyGe7+duVFpEBPnrWOtOoj\nSvX9VWrakrff4azvXA3ABWccyYDV3lM2XbXUqbRbPUu91Plmc1VVXGZmg4C9gZHA4cBCd/9wymmr\nWp6Ly5IUaPKr0BDxyAvDyI7hyapT674hFwb9Ov6iMIrFZafMZ68dhzHj74/3mHbqEQcy+/kXufb2\n+wA4fMwIfnzOZ3nw78+ulK4vH/VxfvDbm1ZKK1D1vJ0eaAo6PdDkou8yM9uZEFz+H/Ahwrgvd6eZ\nqHalXE1+pdWYstSgX5ef9kSPac/OfItJP7yxx3K//+O9XDf1Pnb7wJYrpevSCVOrbkxZbl4FmUC5\nmvRV063MJGAt4CJgB3cf7e7fSDdZ7U3d0UjSC0+VLn1evtxZ9Oq/m5yazqS6mvRU062MQnwKlKvJ\nl7QaU5Ya9GuvHYdx2SmPd097ouvNsstP+OLh/M+Zl/VI15eP+jg/OPOmkmkt3odK80pPytWko+pH\nmPOsVepkylGgyYcsK/5vvethZsx8qsdye+26LX+ZPEEV/xlq94CTq3YyedbqQQYUaDrd6edfyQ9+\ndWuPaV/+7P587+xjMkqRFLRzoFGQqVI7BJkCBZvOtGzZcv726D/Y7r2bAPDkswvYfef30a9fNdWm\n0gztGGwyDTJmdlPJDwJ394PSSVLt2inIgAKNSJ61U7DJ+hHm76W5YSlPDwW0pmZ2OplWB5nSOz0g\nUJuyeXF3n17pr4lp7Fh61Ll1NHO0ybRGxpTa6LHn6lTTGHNbYCKwI1C467m7b5VmwiRQrqY1NHO0\nybRGxlRupnYaLK131dQq/gr4KaH/stHA5cBVKaZJSlCuRiTflLMprZoOMge6+zQzM3d/HphgZg8C\nX085bVJEI2/mVzNHm0xzZEypj+prVlbNoGUzCH2XXQv8CXgBmOTu26WfvOq029NlvVGgySdV/EtS\nKwSaXLSTMbPdgVnAOsA3gUHAhe7+f2kmrBadFmQKFGxEWkNeA04uBi0D3uvub7j7XHc/zt0PA7Zo\nxMbNbIyZzTKz2WZ2Vpl5LoqfP2xmuzZiu+1C9TQiknfVBJlxVU6riZn1A34MjAGGAUea2Q5F8+wP\nbO3u2xCGfr6k3u22GwUakfxbc8HNHftgQNmKfzPbD9gfGGJmFwGFLNVahCfN6jUceNrdn4vbmwwc\nDDyRmOcgwtNsuPu9ZraOmQ1294UN2H7baJcHAlqpnkAdUUpfrLng5twWnaWlUk7mBeABYEn8/wBw\nP3Aj8LEGbHsIMDfxfl6c1ts8mzVg221n2ewBLZ2raaUGgqXSOvGSKS2TfslWIVfTKTmbsjkZd38Y\neNjMrgL6A1u4+6wGbrvanjmLK6VKLnfeVZO7X4/aeSdG7bJTH5PV2lq18WYrNRAslVaNQCl90exH\nnru6uujq6mrKtgqqaSezH/AdYDVgaKx8P7cBHWTOBzZPvN+ckFOpNM9mcdpKvnH0EXUmR0QkG80q\nRhs5ciQjR47sfn/++eenvs1qgswEYARwJ4C7zzSzRnQpcz+wjZkNJRTNfRo4smieG4GTgclmtgfw\nL9XHVKfVcjSt1ECwVFo1AqXUq127qKmmncy97j7CzGa6+65x2iPuvkvdGw8PF/wv0A/4pbtPMrMT\nANz90jhP4Qm0fwOfdfcHS6ynI9vJVKtVAk0rVZyr4l/S1oxAk5fGmJcRWvqfDRwGnAL0d/cT00xY\nLRRkqtMqwUZEVkgz2OSlMebJhB6Y3wauBl4H/ifNREk6WvnpM5FO1epPolVqJzMQOBHYGngE2NPd\nG9E+RjLUanU1tZh4yRQuvWEqACccsi/jTzq0IfOm0U+YitakVq3a+WalnMzlwG7Ao4QnzL7blBRJ\nU7RbrmbiJVOYeNl1HDxhEAdPGMTEy65j4iVT6p43jQHC1KZG6tFquZpKT5ft4O47A5jZL4C/NSdJ\n0iztlKu59IapHH/RkJXarpTKodQybxoDhKlNjdSrlXI1lXIy7xZeuPu7FeaTFtduuRqRTtEK9TWV\ncjK7mNkbifcDE+/d3QelmC5pslbv/+yEQ/Zl4inXdb+/7JT5jD/+E3XPm8YAYWpTI42W5z7Ren2E\nuRXoEebGaeVAo4p/6XS1BppctJNpBQoyjdfKwUZEqgs4eWknIx1I9TQirS0vdTUKMlJWqw8fINLp\n8vBgQDUdZEqHa6dHnUU6UZadbyonIyLSQZqds1FORqqmHI1Ie2hmoFFORkREUqMgIzXTAwEiUi0F\nGekzBRoR6Y2CjNRFuRoRqURBRhpCgUZESlGQkYZRrkZEiinISMMp0IhIgYKMpEK5GhEBBRkREUmR\ngoykSjkakc6WSZAxs/XM7A4ze8rMpprZOmXme87MHjGzmWZ2X7PTKY2jQCPSmbLKyZwN3OHu2wJ/\niu9LcWC0u+/q7sObljpJhXI1Ip0nqyBzEHB5fH05cEiFeVMdtU2aT8FGpHNkFWQGu/vC+HohMLjM\nfA5MM7P7zey/mpM0aRYFGpH2l1pX/2Z2B7BxiY++mnzj7m5mXmY1e7v7AjPbELjDzGa5e1ej0yrZ\n0fABIu0ttSDj7v9Z7jMzW2hmG7v7i2a2CfBSmXUsiP9fNrMpwHCgZJA576rJ3a9H7bwTo3bZqZ7k\nS5Mtmz1AgUYkZdPvfZzp9z3e1G2ae7lMRIobNbsQ+Ke7X2BmZwPruPvZRfOsDvRz9zfMbA1gKnCu\nu08tsT5fevP1TUm7pEuBRqR5VtnuKNw91XrvrOpkzgf+08yeAj4S32Nmm5rZLXGejYEuM3sIuBe4\nuVSAkfaiehqR9pJJTqbRlJNpP8rRiKSvnXMyIhUpRyPSHhRkJLfUnkak9SnISO4p0Ii0LgUZaQnK\n1Yi0JgUZaSkKNCKtJbXGmBJsdPjRvPrWW93v1x04kJd+f1WGKWp96iVApHUoJ5OyV996C4fuv2TA\nkfooVyOSfwoy0tIUaETyTUFGWp4eChDJLwWZlK07cCAG3X/rDhyYcYralwKNSP6o4j9lquRvLj0U\nIJIvyslIW1KuRiQfFGSkbamuRiR7CjLS9hRoRLKjOpmMqJFmc2nkTZFsKCeTETXSbD7laESaT0FG\nOorqaUSaS0FGOpICjUhzKMhkRI00s6dAI5I+VfxnRJX8+aAHAkTSpZyMdDzV04ikR0FGJFKwEWk8\nBRkREUmN6mR6UUujyYEHHsa7iferAmsNHFhy+WrXq0abzadONkUaJ5OcjJkdbmZ/N7NlZvbBCvON\nMbNZZjbbzM5qZhoLamk0+W5iPo/vyy1f7XrVaDM7KjoTqV9WxWWPAocCd5ebwcz6AT8GxgDDgCPN\nbIfmJE8kUD2NSH0yCTLuPsvdn+pltuHA0+7+nLsvBSYDB6efOpGVKdCI9E2eK/6HAHMT7+fFaU1V\nS6PJVRPzWXxfbvlq16tGm/mhXI1I7VKr+DezO4CNS3w03t1vqmIVXsv2zrtqcvfrUTvvxKhddqpl\n8bJqqWR/6+brG75eVfLnjxpwSquafu/jTL/v8aZu09xrupc3duNmdwKnu/uDJT7bA5jg7mPi+3HA\ncne/oMS8vrSGG7xIoyjYSCtbZbujcHdLdRtprrxK5XbwfmAbMxtqZu8BPg3c2LxkiYhIvbJ6hPlQ\nM5sL7AHcYma3xembmtktAO7+LnAycDvwOPA7d38ii/SKlKN6GpHKMi0ua5Q0i8tWP/Awlibe9wcW\n33x9yelrlml4KZ1BRWfSappRXKYW/71YSs8nEKzC9ELDye5pajjZUdRTgMjK8lAn09KWAKdmnQjJ\nFRWfiaygIFOHl4FjgYuKpomorkYkUHFZL/rT8/G3/vF/P2CjEvMnp6nhpKhNjXQ6BZleLC7zQMH/\nHHYI37v+hpKfnX7YIZx//Ng0kyUtRIFGOpmKy/rovw86oE+fSWdS0Zl0KgWZPhqywfrsucN2K03f\nc4ftGLLB+hmkSPJOgUY6kYJMHUZst21V00QK9ECAdJq2b4xZy8iSpRpYFgYfK1iV0BFm8SiY/eL/\nZUXzUmZ5jXgpqqeRrKkxZgPU0kCyXMPLUtPerWHeUtPUcFPUeFM6gYrLRDKm4jNpZwoyIjmguhpp\nV20fZGoZWbJ/Yj6L70uNdkmZ6dVOqzVd0jkUaKTdtH3Fv0irUl2NpK1TBi0TEZE2pSAjklOqp5F2\n0PaPMJeTRjuV4rYzhTYxIvVQ32fSyjo2yKTRTqVc2xmReqlNjbQqFZeJtBAVoUmrUZARaUEKNNIq\nOjbIpNFOpVybGJE0KFcjraDt2snc9chjjNplp4xT1HdKf7ZaNf2Fuprp9z7O6BHDMk5N37V6+qG1\n9kHtZPrgrkcfyzoJdVH6s9Xq6Z9+3+NZJ6EurZ5+aI99aKS2CzIinUjFZpJXCjIibWLZ7AH4P1UT\nKPnSNnUyWadBRKQVpV0n0xZBRkRE8knFZSIikhoFGRERSU3LBxkzO9zM/m5my8zsgxXme87MHjGz\nmWZ2XzPTWEkN6R9jZrPMbLaZndXMNFZiZuuZ2R1m9pSZTTWzdcrMl6vjX83xNLOL4ucPm9muzU5j\nJb2l38xGm9lr8XjPNLOvZZHOUszsMjNbaGaPVpgnt8ceet+HPB//pnP3lv4Dtge2Be4EPlhhvmeB\n9bJOb1/SD/QDngaGEgbsfAjYIeu0x7RdCJwZX58FnJ/341/N8QT2B26Nr0cA/5d1umtM/2jgxqzT\nWib9I4FdgUfLfJ7bY1/DPuT2+Df7r+VzMu4+y92fqnL23HWMXGX6hwNPu/tz7r4UmAwcnH7qqnIQ\ncHl8fTlwSIV583L8qzme3fvl7vcC65jZ4OYms6xqr4e8HO8e3L0LeLXCLHk+9kBV+wA5Pf7N1vJB\npgYOTDOz+83sv7JOTI2GAHMT7+fFaXkw2N0XxtcLgXI3gzwd/2qOZ6l5Nks5XdWqJv0O7BWLm241\ns9bo5yTI87GvVisf/4ZqiZZbZnYHsHGJj8a7+01VrmZvd19gZhsCd5jZrPhrJHUNSH+mz5lXSP9X\nk2/c3Su0Wcrs+JdQ7fEs/iWal+f9q0nHg8Dm7r7YzPYDbiAUy7aKvB77arX68W+Ylggy7v6fDVjH\ngvj/ZTObQihyaMpNrgHpnw9snni/OeHXXVNUSn+s/NzY3V80s02Al8qsI7PjX0I1x7N4ns3itDzo\nNf3u/kbi9W1m9hMzW8/dFzUpjfXI87GvSosf/4Zqt+KykmWgZra6ma0VX68B7AuUfbIlQ+XKcO8H\ntjGzoWb2HuDTwI3NS1ZFNwKfia8/Q/jF1kMOj381x/NGYCyAme0B/CtRLJi1XtNvZoPNzOLr4YSG\n161yg8vzsa9Kix//xsr6yYN6/4BDCeW3bwEvArfF6ZsCt8TXWxGewHkIeAwYl3W6a0l/fL8f8CTh\nqaI8pX89YBrwFDAVWKcVjn+p4wmcAJyQmOfH8fOHqfDkYh7TD3wpHuuHgBnAHlmnOZH2q4EXgHfi\ntX98Kx37avYhz8e/2X/qVkZERFLTbsVlIiKSIwoyIiKSGgUZERFJjYKMiIikRkFGRERSoyAjIiKp\nUZCRjhKHVJhpZo+a2TVmNrDCvO+PXYL0ts7RZrZS90DlptfLzA42sx0S76eb2W6N3o5IIyjISKdZ\n7O67uvvOhIZ0J1aYd1dCt/N5cyiQ7HBRjd0ktxRkpJP9Bdg6dntzmZnda2YPmtlBZtYfOA/4dMz5\nfMrMdjezGXGev5pZ1R0emtkaxduI048zs+vN7LY48NsFiWU+Z2ZPxmV+ZmY/MrM9gY8D34nr2SrO\nfnic70kz+3DjDpFIfVqig0yRRjOzVYExwG3A14A/ufvxFkb2vJfQVc7Xgd3c/ZS4zFrASHdfZmYf\nBSYCn6xyk18t3oaZTYufvR/4ACFn9aSZXUTInXyNkJt6E/gz8JC732NmNwI3ufv1MV0A/dx9RCze\nOweou1NZkUZQkJFOM9DMZsbXdwOXAfcAHzezr8TpqwFbEDosTXZaug5whZltTQgC/WvY7r5ltuGE\n4PMGgJk9ThjxckPgLnf/V5z+e3p2FV/cmer18f+DcXmRXFCQkU7zlrv3GDM+5gQOc/fZRdNHFC37\nTUJAONTMtgSm17jtctt4OzFpGeF7WVzP0tv4KoV1FJYXyQXVyYjA7cAphTdmVghCbwBrJeYbROh5\nF+CzDdpGqeEdHPgbMMrM1olFe59gRWB5I6ZFJPcUZKTTlHoS65tAfzN7xMweA86N0+8EhhUq/oEL\ngUlm9iDQr2hdpdbrienltuGllnX3Fwh1PvcRHlB4FngtfjwZOMPMHkhU/Pe2jyKZUFf/IjllZmu4\n+79jTuZ64Jfu/oes0yVSC+VkRPJrQnxI4VHgGQUYaUXKyYiISGqUkxERkdQoyIiISGoUZEREJDUK\nMiIikhoFGRERSY2CjIiIpOb/A8oQ9NcAb61tAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10873f110>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "colormarkers = [ ['red','s'], ['greenyellow','o'], ['blue','x']]\n",
    "step = .01 \n",
    "margin = .1\n",
    "sl_min, sl_max = X_train4[:, 2].min()-margin, X_train4[:, 2].max() + margin\n",
    "sw_min, sw_max = X_train4[:, 3].min()-margin, X_train4[:, 3].max() + margin\n",
    "sl, sw  = np.meshgrid(\n",
    "    np.arange(sl_min, sl_max, step),\n",
    "    np.arange(sw_min, sw_max, step), \n",
    "    )\n",
    "Zs = clf_petal.predict(np.c_[sl.ravel(), sw.ravel()]).reshape(sl.shape)\n",
    "centroids_s = clf_petal.cluster_centers_\n",
    "plt.figure(1)\n",
    "plt.clf()\n",
    "plt.imshow(Zs, interpolation='nearest', extent=(sl.min(), sl.max(), sw.min(), sw.max()), cmap= plt.cm.Pastel1, aspect='auto', origin='lower')\n",
    "for j in [0,1,2]:\n",
    "    px = X_train4[:, 2][y_train4 == j]\n",
    "    py = X_train4[:, 3][y_train4 == j]\n",
    "    plt.scatter(px, py, c=colormarkers[j][0], marker= colormarkers[j][1])\n",
    "plt.scatter(centroids_s[:, 0], centroids_s[:, 1],marker='*',linewidths=3, color='black', zorder=10)\n",
    "plt.title('K-means clustering on the Iris dataset using Petal dimensions\\nCentroids are marked with stars')\n",
    "plt.xlim(sl_min, sl_max)\n",
    "plt.ylim(sw_min, sw_max)\n",
    "plt.xlabel(\"Petal length\")\n",
    "plt.ylabel(\"Petal width\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, calculate the clusters, using the four attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KMeans(copy_x=True, init='k-means++', max_iter=300, n_clusters=3, n_init=10,\n",
       "    n_jobs=1, precompute_distances='auto', random_state=33, tol=0.0001,\n",
       "    verbose=0)"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = cluster.KMeans(init='k-means++', n_clusters=3, random_state=33)\n",
    "clf.fit(X_train4)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "print (y_train[clf.labels_==0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2 1 1 2 1]\n"
     ]
    }
   ],
   "source": [
    "print (y_train[clf.labels_==1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2 2 1 2 2 1 2 2 1 2 2 2 1 2 1 2 2 2 1 2 2 2 2 2 1 1 2 2 2 2 2 2 2 1 2 2]\n"
     ]
    }
   ],
   "source": [
    "print (y_train[clf.labels_==2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Measure precision & recall in the testing set, using all attributes, and using only petal measures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "     setosa       1.00      1.00      1.00         8\n",
      " versicolor       0.64      0.64      0.64        11\n",
      "  virginica       0.79      0.79      0.79        19\n",
      "\n",
      "avg / total       0.79      0.79      0.79        38\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_pred=clf.predict(X_test4)\n",
    "print (metrics.classification_report(y_test, y_pred, target_names=['setosa','versicolor','virginica']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "     setosa       1.00      1.00      1.00         8\n",
      " versicolor       0.85      1.00      0.92        11\n",
      "  virginica       1.00      0.89      0.94        19\n",
      "\n",
      "avg / total       0.96      0.95      0.95        38\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_pred_petal=clf_petal.predict(X_test4[:,2:4])\n",
    "print (metrics.classification_report(y_test, y_pred_petal, target_names=['setosa','versicolor','virginica']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wait, every performance measure is better using just two attributes. It is possible that less features give better results?  Although at a first glance this seems contradictory, we will see in future notebooks that selecting the right subset of features, a process called feature selection, could actually improve the performance of our algorithms."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Supervised Learning: Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In every example we have seen so far the output we aimed at predicting belonged to a discrete set. For classification, the set was the target class, while for the clustering algorithm the set included different calculated clusters. What if we want to predict a value extracted from the real line?. In this case, we are trying to solve a regression problem. \n",
    "\n",
    "To show how regression works in scikit-learn, we will apply to a (very) simple and well-know problem: trying to predict the price of a house given some of its. As the dataset, we will use the Boston house-prices dataset (find the dataset description and attributes [here](http://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Boston dataset shape:(506, 13)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "boston = load_boston()\n",
    "print ('Boston dataset shape:{}'.format(boston.data.shape))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['CRIM' 'ZN' 'INDUS' 'CHAS' 'NOX' 'RM' 'AGE' 'DIS' 'RAD' 'TAX' 'PTRATIO'\n",
      " 'B' 'LSTAT']\n"
     ]
    }
   ],
   "source": [
    "print (boston.feature_names)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create training and testing sets, and scale values, as usual"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_train_boston=boston.data\n",
    "y_train_boston=boston.target\n",
    "X_train_boston = preprocessing.StandardScaler().fit_transform(X_train_boston)\n",
    "y_train_boston = preprocessing.StandardScaler().fit_transform(y_train_boston)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Create a method for training and evaluating a model. This time, to evaluate our model we will a different approach: instead of separating the training set, we will use _cross-validation_. \n",
    "\n",
    "Cross-validation usually involves the following steps:\n",
    "1.\tPartition the dataset into k different subsets.\n",
    "2.\tCreate k different models by training on k-1 subsets and testing on the remaining one.\n",
    "3.\tMeasure the performance of each of the k models and use the average value as you performance value.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def train_and_evaluate(clf, X_train, y_train, folds):\n",
    "    clf.fit(X_train, y_train)\n",
    "    print ('Score on training set: {:.2f}'.format(clf.score(X_train, y_train)))\n",
    "    #create a k-fold cross validation iterator of k=5 folds\n",
    "    cv = sklearn.cross_validation.KFold(X_train.shape[0], folds, shuffle=True, random_state=33)\n",
    "    scores = sklearn.cross_validation.cross_val_score(clf, X_train, y_train, cv=cv)\n",
    "    print ('Average score using {}-fold crossvalidation:{:.2f}'.format(folds,np.mean(scores)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the main advantages of cross-validation is reducing the variance of the evaluation measures. Note that when you have a training and testing set, you result will depend on how you created them. Evaluation within machine learning generally assumes that the distribution of classes on your training and testing sets are similar. If this is not the case, you may get results that are not an accurate measure of the actual classifier’s performance. Cross-validation allows us to mitigate this risk: since we are averaging on k different models built on k different datasets, we are reducing variance and probably producing more realistic performance scores for our models. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scikit-learn includes a linear model method called `linear_model.SGDRegressor` , which tries  to minimize least squares loss using stochastic gradient descent. We will use it to find the model for Boston house values, and predict a new house value, given its 13 learning features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score on training set: 0.73\n",
      "Average score using 5-fold crossvalidation:0.70\n"
     ]
    }
   ],
   "source": [
    "from sklearn import linear_model\n",
    "clf_sgd = linear_model.SGDRegressor(loss='squared_loss', penalty=None, random_state=33)\n",
    "train_and_evaluate(clf_sgd, X_train_boston, y_train_boston,5)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you remember the classification, we used accuracy, the proportion of correctly classified instances in the evaluation set, to summarize in just one number our method’s performance. For regression, accuracy is not a good idea, since we are predicting real values, and it is almost impossible that we may exactly predict the final real value. For regression, the default score function in scikit-learn is the coefficient of determination (or $R^2$ score), which measures the proportion of outcome variation explained by the model.; $R^2$ score ranges from 0 to 1, and reaches its maximum value of 1 when the model perfectly predicts all the target values. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us show the the model  hyperplane coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.06777406  0.06767528 -0.04290825  0.08828856 -0.11797833  0.3394894\n",
      " -0.01969258 -0.23195707  0.09594823 -0.05271866 -0.19913907  0.10355794\n",
      " -0.36852382]\n"
     ]
    }
   ],
   "source": [
    "print(clf_sgd.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You probably noted the `penalty=None` parameter when we called the method. The penalization parameter for linear regression methods is introduced to avoid overfitting. It does this by penalizing those hyperplanes having some of their coefficients too large, seeking hyperplanes where each feature contributes more or less the same to the predicted value. This parameter is generally the L2 norm (the squared sums of the coefficients) or the L1 norm (the sum of the absolute value of the coefficients). Let us see how our model works if we introduce a L2 penalty:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score on training set: 0.73\n",
      "Average score using 5-fold crossvalidation:0.70\n"
     ]
    }
   ],
   "source": [
    "clf_sgd1 = linear_model.SGDRegressor(loss='squared_loss', penalty='l2', random_state=33)\n",
    "train_and_evaluate(clf_sgd1, X_train_boston, y_train_boston,folds=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To end with this introductory notebook, we will summarize the main common steps to apply a supervised learning method in scikit-learn:\n",
    "-\tGet your dataset. Select your learning features, and create a 2D data array, with one column for each feature, and one row for each learning instance. Every feature is represented by a real number. Your data probably does not look like that in its original format. In the real world, this the preprocessing stage usually takes a lot of time. In the next chapters we will show several examples of preprocessing different types of data. \n",
    "-\tCreate an instance of an estimator (a classifier or a regressor). In scikit-learn, this corresponds to an object that implements the methods `fit(X,y)` and `predict(T)`. This estimator takes as arguments the model’s parameters. You can set these parameters by hand, or using tools for model selection (we will address this later in this book).\n",
    "-\tSeparate you training and testing sets (or, alternatively, use cross-validation)\n",
    "-\tBuild the model using the `fit(X,y)` method, being X your training data, and y the corresponding target class. \n",
    "-\tEvaluate you estimator on the testing data, using the `predict(T)` method, being T your testing dataset.\n",
    "-\tCompare your results with the original target classes, using one of the several measures in the metrics module.\n",
    "\n",
    "The previous steps are only an overview: scikit-learn offers several additional methods implementing different machine learning operations (such as dimensionality reduction, clustering, or semi-supervised learning), and many dataset transformation methods. \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}