{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Applying logistic regression and SVM\n",
    "> In this chapter you will learn the basics of applying logistic regression and support vector machines (SVMs) to classification problems. You'll use the scikit-learn library to fit classification models to real data. This is the Summary of lecture \"Linear Classifiers in Python\", via datacamp.\n",
    "\n",
    "- toc: true \n",
    "- badges: true\n",
    "- comments: true\n",
    "- author: Chanseok Kang\n",
    "- categories: [Python, Datacamp, Machine_Learning]\n",
    "- image: images/plot_4_classifiers.png"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scikit-learn refresher"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### KNN classification\n",
    " this exercise you'll explore a subset of the [Large Movie Review Dataset](http://ai.stanford.edu/~amaas/data/sentiment/).The X variables contain features based on the words in the movie reviews, and the y variables contain labels for whether the review sentiment is positive (+1) or negative (-1)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Large Movie Review Dataset : This is a dataset for binary sentiment classification containing substantially more data than previous benchmark datasets. We provide a set of 25,000 highly polar movie reviews for training, and 25,000 for testing. There is additional unlabeled data for use as well. Raw text and already processed bag of words formats are provided. See the README file contained in the release for more details.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import load_svmlight_file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, y_train = load_svmlight_file('./dataset/aclImdb/train/labeledBow.feat')\n",
    "X_test, y_test = load_svmlight_file('./dataset/aclImdb/test/labeledBow.feat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train[y_train < 5] = -1.0\n",
    "y_train[y_train >= 5] = 1.0\n",
    "\n",
    "y_test[y_test < 5] = -1.0\n",
    "y_test[y_test >= 5] = 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction for test example 0: -1.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# Create and fit the model\n",
    "knn = KNeighborsClassifier()\n",
    "knn.fit(X_train[:, :89523], y_train)\n",
    "\n",
    "# Predict on the test features, print the results\n",
    "pred = knn.predict(X_test)[0]\n",
    "print(\"Prediction for test example 0:\", pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applying logistic regression and SVM\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running LogisticRegression and SVC\n",
    "In this exercise, you'll apply logistic regression and a support vector machine to classify images of handwritten digits.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "0.96\n",
      "0.9955456570155902\n",
      "0.9911111111111112\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\kcsgo\\anaconda3\\lib\\site-packages\\sklearn\\linear_model\\_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
     ]
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "digits = datasets.load_digits()\n",
    "X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target)\n",
    "\n",
    "# Apply logistic regression and print scores\n",
    "lr = LogisticRegression()\n",
    "lr.fit(X_train, y_train)\n",
    "print(lr.score(X_train, y_train))\n",
    "print(lr.score(X_test, y_test))\n",
    "\n",
    "# Apply SVM and print scores\n",
    "svm = SVC()\n",
    "svm.fit(X_train, y_train)\n",
    "print(svm.score(X_train, y_train))\n",
    "print(svm.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear classifiers\n",
    "- Classification: learning to predict categories\n",
    "- decision boundary: the surface separating different predicted classes\n",
    "- linear classifier: a classifier that learn linear decision boundaries\n",
    "    - e.g. logistic regression, linear SVM\n",
    "- linearly separable: a dataset can be perfectly explained by a linear classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing decision boundaries\n",
    "In this exercise, you'll visualize the decision boundaries of various classifier types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#hide\n",
    "X = np.array([[11.45,  2.4 ],\n",
    "       [13.62,  4.95],\n",
    "       [13.88,  1.89],\n",
    "       [12.42,  2.55],\n",
    "       [12.81,  2.31],\n",
    "       [12.58,  1.29],\n",
    "       [13.83,  1.57],\n",
    "       [13.07,  1.5 ],\n",
    "       [12.7 ,  3.55],\n",
    "       [13.77,  1.9 ],\n",
    "       [12.84,  2.96],\n",
    "       [12.37,  1.63],\n",
    "       [13.51,  1.8 ],\n",
    "       [13.87,  1.9 ],\n",
    "       [12.08,  1.39],\n",
    "       [13.58,  1.66],\n",
    "       [13.08,  3.9 ],\n",
    "       [11.79,  2.13],\n",
    "       [12.45,  3.03],\n",
    "       [13.68,  1.83],\n",
    "       [13.52,  3.17],\n",
    "       [13.5 ,  3.12],\n",
    "       [12.87,  4.61],\n",
    "       [14.02,  1.68],\n",
    "       [12.29,  3.17],\n",
    "       [12.08,  1.13],\n",
    "       [12.7 ,  3.87],\n",
    "       [11.03,  1.51],\n",
    "       [13.32,  3.24],\n",
    "       [14.13,  4.1 ],\n",
    "       [13.49,  1.66],\n",
    "       [11.84,  2.89],\n",
    "       [13.05,  2.05],\n",
    "       [12.72,  1.81],\n",
    "       [12.82,  3.37],\n",
    "       [13.4 ,  4.6 ],\n",
    "       [14.22,  3.99],\n",
    "       [13.72,  1.43],\n",
    "       [12.93,  2.81],\n",
    "       [11.64,  2.06],\n",
    "       [12.29,  1.61],\n",
    "       [11.65,  1.67],\n",
    "       [13.28,  1.64],\n",
    "       [12.93,  3.8 ],\n",
    "       [13.86,  1.35],\n",
    "       [11.82,  1.72],\n",
    "       [12.37,  1.17],\n",
    "       [12.42,  1.61],\n",
    "       [13.9 ,  1.68],\n",
    "       [14.16,  2.51]])\n",
    "\n",
    "y = np.array([ True,  True, False,  True,  True,  True, False, False,  True,\n",
    "       False,  True,  True, False, False,  True, False,  True,  True,\n",
    "        True, False,  True,  True,  True, False,  True,  True,  True,\n",
    "        True,  True,  True,  True,  True, False,  True,  True,  True,\n",
    "       False, False,  True,  True,  True,  True, False, False, False,\n",
    "        True,  True,  True, False,  True])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_meshgrid(x, y, h=.02, lims=None):\n",
    "    \"\"\"Create a mesh of points to plot in\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "        x: data to base x-axis meshgrid on\n",
    "        y: data to base y-axis meshgrid on\n",
    "        h: stepsize for meshgrid, optional\n",
    "        \n",
    "    Returns\n",
    "    -------\n",
    "        xx, yy : ndarray\n",
    "    \"\"\"\n",
    "    \n",
    "    if lims is None:\n",
    "        x_min, x_max = x.min() - 1, x.max() + 1\n",
    "        y_min, y_max = y.min() - 1, y.max() + 1\n",
    "    else:\n",
    "        x_min, x_max, y_min, y_max = lims\n",
    "    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n",
    "                         np.arange(y_min, y_max, h))\n",
    "    return xx, yy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_contours(ax, clf, xx, yy, proba=False, **params):\n",
    "    \"\"\"Plot the decision boundaries for a classifier.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "        ax: matplotlib axes object\n",
    "        clf: a classifier\n",
    "        xx: meshgrid ndarray\n",
    "        yy: meshgrid ndarray\n",
    "        params: dictionary of params to pass to contourf, optional\n",
    "    \"\"\"\n",
    "    if proba:\n",
    "        Z = clf.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:,-1]\n",
    "        Z = Z.reshape(xx.shape)\n",
    "        out = ax.imshow(Z,extent=(np.min(xx), np.max(xx), np.min(yy), np.max(yy)), \n",
    "                        origin='lower', vmin=0, vmax=1, **params)\n",
    "        ax.contour(xx, yy, Z, levels=[0.5])\n",
    "    else:\n",
    "        Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])\n",
    "        Z = Z.reshape(xx.shape)\n",
    "        out = ax.contourf(xx, yy, Z, **params)\n",
    "    return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_classifier(X, y, clf, ax=None, ticks=False, proba=False, lims=None): \n",
    "    # assumes classifier \"clf\" is already fit\n",
    "    X0, X1 = X[:, 0], X[:, 1]\n",
    "    xx, yy = make_meshgrid(X0, X1, lims=lims)\n",
    "    \n",
    "    if ax is None:\n",
    "        plt.figure()\n",
    "        ax = plt.gca()\n",
    "        show = True\n",
    "    else:\n",
    "        show = False\n",
    "        \n",
    "    # can abstract some of this into a higher-level function for learners to call\n",
    "    cs = plot_contours(ax, clf, xx, yy, cmap=plt.cm.coolwarm, alpha=0.8, proba=proba)\n",
    "    if proba:\n",
    "        cbar = plt.colorbar(cs)\n",
    "        cbar.ax.set_ylabel('probability of red $\\Delta$ class', fontsize=20, rotation=270, labelpad=30)\n",
    "        cbar.ax.tick_params(labelsize=14)\n",
    "        #ax.scatter(X0, X1, c=y, cmap=plt.cm.coolwarm, s=30, edgecolors=\\'k\\', linewidth=1)\n",
    "    labels = np.unique(y)\n",
    "    if len(labels) == 2:\n",
    "        ax.scatter(X0[y==labels[0]], X1[y==labels[0]], cmap=plt.cm.coolwarm, \n",
    "                   s=60, c='b', marker='o', edgecolors='k')\n",
    "        ax.scatter(X0[y==labels[1]], X1[y==labels[1]], cmap=plt.cm.coolwarm, \n",
    "                   s=60, c='r', marker='^', edgecolors='k')\n",
    "    else:\n",
    "        ax.scatter(X0, X1, c=y, cmap=plt.cm.coolwarm, s=50, edgecolors='k', linewidth=1)\n",
    "\n",
    "    ax.set_xlim(xx.min(), xx.max())\n",
    "    ax.set_ylim(yy.min(), yy.max())\n",
    "    #     ax.set_xlabel(data.feature_names[0])\n",
    "    #     ax.set_ylabel(data.feature_names[1])\n",
    "    if ticks:\n",
    "        ax.set_xticks(())\n",
    "        ax.set_yticks(())\n",
    "        #     ax.set_title(title)\n",
    "    if show:\n",
    "        plt.show()\n",
    "    else:\n",
    "        return ax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_4_classifiers(X, y, clfs):\n",
    "    # Set-up 2x2 grid for plotting.\n",
    "    fig, sub = plt.subplots(2, 2)\n",
    "    plt.subplots_adjust(wspace=0.2, hspace=0.2)\n",
    "    \n",
    "    for clf, ax, title in zip(clfs, sub.flatten(), (\"(1)\", \"(2)\", \"(3)\", \"(4)\")):\n",
    "        # clf.fit(X, y)\n",
    "        plot_classifier(X, y, clf, ax, ticks=True)\n",
    "        ax.set_title(title)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\kcsgo\\anaconda3\\lib\\site-packages\\sklearn\\svm\\_base.py:947: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD7CAYAAABUt054AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2de3xU5bnvv2uSSUCC4FaUFoRWoEKVoKDgBaRbrSKxaIutu145n723x2PPrrpPbeupglCUvT9Yr8djj5e9QfBSChahla22qIDcNMokBERQAYmEhHBJApNkZs17/lizhjWTteZ+n+f7+eSDM7Nm5k38zW+e932e93k1pRSCIAhC9nHlegCCIAilihiwIAhCjhADFgRByBFiwIIgCDlCDFgQBCFHiAELgiDkCDHgFNE0bZ6maffEee1jmqbdmekxCUI6EG1nHk3qgJNH07QBwBZguFLKq2laBfAKcAEwFPh7pdR7luu/AWwGhimlunMwZEGICxttXwT8FhgH6MB7wM+VUvuD14u2k0Ai4NSYAbyplPJa7lsH3AI0RV4cFOunwLSsjE4QkmcG4do+BXgO+BZGcNEO/Kd5sWg7OcSAU+Ma4H3zhlKqWyn1hFJqHUaUYMd7QE0WxiYIqRCp7VVKqT8qpdqUUseB/wNcGvGc9xBtJ4QYcGqMBnYk+JztwJgMjEUQ0kksbV8GNETcJ9pOkPJcD6DA6Y8xFUuE9uDzBCGfcdS2pmnVwEzguoiHRNsJIhFwahwG+ib4nL7AkQyMRRDSia22NU0bDqwC7lZKrY14WLSdIGLAqVEHfCfB54wCPBkYiyCkkx7a1jRtKPBX4LdKqUU2zxFtJ4gYcGq8CUy23qFpWqWmab2CNys0TeulaZpmuWQyRgQhCPlMmLY1TRsErAaeUUr93uE5ou0EkTrgFNA07TSMWskRZrmOpmm7Mcp0rHxbKbU7WCv5IXCW1EoK+UyktjVNmwU8BByzXqeUqgpeL9pOAjHgFNE07RGgWSn1RBzX/g74XCn1fzM/MkFIDdF25hEDFgRByBGyBiwIgpAjxIAFQRByhBiwIAhCjhADFgRByBEJbUU+pXcvNahvVabGIpQwyudj25G2g0qpAdl+b9G1kEkaWloddZ2QAQ/qW8XS6ZHbvwUhNfzNjQCMfv2tPbl4f9G1kElG/f4/HHUtSxBCTjHNd8H453I8EkHIPmLAQs4Q8xVKHTFgISfoLWK+giAGLGQdvaURpWDrPStyPRRByCnSkF3IOkrBkpplHF/f49g8QSgpJAIWsoq57nu8pTXHIxGE3CMGLGQNSboJQjhiwEJWEPMVhJ6IAQsZx6x4kKSbIIQjSTgho+itTZJ0EwQHJAIWMorSdQZfMVaSboJggxhwEdHW1cUdK1fR1tWV66EAxrrv4CvGMrf9zlwPRShw8k3b6UIMuIhY5NnK5sb9LPZszfVQQkk3MV8hHeSTttOJGHCR0NbVxSt1DSwHXq5ryGmkoLcaa71S8SCkg3zSdroRAy4SFnm2cq1STAFqlMpZpKC3NKJ0XSoehLSRL9rOBGLARYAZITyo6wDM1PWcRQpKgZr7HB9JxYOQBvJJ25lADDgHpDuhYEYIw4O3h5ObSMFMui2U4LdkKVZtZwox4ByQakLBKvLICMEk25GCJN0ESE3bkeadL9rOJGLAWSYdCQWryBd5tnJxIEAZ8KXlpwy4KBDISqQg24wFSF3bkeadD9rONLITLsvYJRTuGj8u7udbRX5zXQMXfmMg2yrcTHa43nXocDqG7Yg0VhdMUtF2pK5vGXMuuw8dpi6H2s4GYsBZxBTZJktCYUJQbCdXVsb1GpEi7z/gVJ6ouSqDo3bGjHyX1CwD2elW0qSqbTvznj/lygyPOvfIEkQWSTWhkE8ZYWutr2wzFlLRdj7pOtuIAWeJdCQU8ikjrHSdQffelfX3FfKPVLWdT7rONmLAWSLVhEI+ZYTNpYeHPzgva+8p5C+paDufdJ0LZA04S6SaUIgUuYlV5Ikk85JFKh6ESFLRdr7oOleIAWeJVBMKmc4It3V18Yu3V/PoVZc7Jk3MigdJuglWUtF2Niod4tF2rhADLhAynRG21mDaRRzmUfJLapZJ0k1IG9modIil7VwiBiyE1WD+g6eed3YfpsXrZWCfKm6vHkLNiLNQCgbdexfHPxDzFQqHtq4uXg5q+4aP61i8bR/frDo5pGuXpuV0fJKEyyNy1XTaWoN5ta7YeWg8rd4PaDg4n1lrDvEvK1bzjbvvlKSbkDS50vZLW+qZoutMAabipq3zxpCu735rAwGlsjqeSMSAM0wiwstF0+nILPTDKNysAgYA0/H6N7OhuS+3Pbs3a2MSCoNC0PYiz1bmBE32EbpxsxC4Aq9/Mx/sc/Pmri+yNh47xIAzjJ3wAkqx8rPPuWHpu0xcuJIblr7LkoYdoalSNstv7Gowf0CAMh4L3tMLr/4QtWuXZGU8QuFQCNqeGnDSdi+8/lksqMttYCEGnEHsmpMElOLnb61n1prDNBycH5rqz1v3WWiqlK0idKcazEfw4uYx4EjwntG0H23M+HiEwqFQtP1bwpcYwrU9mqaOjoyPJRqShMsgdvvbz+zfn/X7KvH6NwG9glcOAHWMOUGxJNMjItnxOdVgXoqf93gMnTlAPX37DcrYOITCozi0PYaBVVUZG0M8iAFnCKfmJKef/E28fmMKZFLOo0zDfitmJstmrDWY3XqAY74A0B8wMsNu6tHpxO2ex7hJt2VsHEJhUUjavthG1wButlBRvpwZ1UMyNoZ4EAPOEE7725e3HQSqLVceoZwneBhv2POzESlYazDN6eP6r9x49YeA0UA9bvdEho44k5HV0zIyBqHwKCRth3S9rwKvfxamrnuXz+bSwT6mDj8rI+8fL2LAGSAyQjCZqeu8jhfYAIwAjAhhIv6cb8V0aRqPnT+UVQP28/jXj9N+tJG+/QYxbtJtjKyehuaSdIFQeNp2aRpPXX0Jb+76ggV199HU0cHAqipmVA9h6vDc1wGLAWeAaOtPEzV4n1/QqX4C9KKcBjbSl2qqgCP0cbuoKDthdtlqOu1vbsSlabRM/zO3ZOUdhUKkELXt0jSuHTGMa0cMy8r7JYIYcAYw15/G+/woXacv0A5oZWVUlpdR7jtEL9c4OvU5dPLvnJgSDeDJqy/O+reyNFYX4sVJ28rlolspTqs8Tqt/PF7/rLzQdr4jBpwB5k+5krauLq5Z9Ac2YayR7QImaBpTzx7BS3UNfO9bHew7lvspUdiRQmK+QgyctD1WKXxKcf13hzP0lP55Od3PR8SAM4RdouL7gQCvbd1unHu172tW3XpjzrszmT0e+CCnwxAKiEhtnwYopULnua269ca8nO7nI5JZyQBOGxzOCASYGggkVZCeib30/uZG1NznpMeDEDd22n4S+BEkvdEiV30i8gEx4ASJRyx2JwR4gMXA3OA1iXb8d9pLn6x4zXXfhSsSeppQxCSjbQ/wFPBg8PFkTrJIt7YLCTHgBImnqUhog0OvytDP5WVlXAUpHVpot5f+hVoPmxr382KtJ+7fQY6SF+xIRtup6Bqctd3W1cU/LH2DTVlu4JNtxIATIJpYrN/U86dcydszbg79LP3pDaBp/Dbi9RI9tDByitfW1cVr9Q1owCv18UUdZmN1MV/Bip227SJQq7ZT1TU4a/uF2i00tndwI9lt4JNtxIATwEkssSKHdB5aaBX3C7WeUPJDUyquKFgp2HqPrDsI4dhpO5O6BmdtN7a189rW7bwBrMJIXhdrFCxVEHHitP/9upHfCUUONztsr7T2XPD6/HTqOr3KyujtNv78sQ4ttNv2+eLHHl6rbwglP67HiIL/cdwYx8oKf3Mjg+69iwUfNKXypxCKDCdtByyVDbF0DT21Hc9Bs3bavv+v755IVgP9AwEWZaGBT7oxZ5vRKGkDTuSwPiexPLD6/R6RQ+T2SnNfulk/+UfgZk1j6U9viPq+0bZ9jq03PiCh5AewPBgF33vJ+B6v5W9uZPAVY5krFQ9FT6KHUNpp+2pdZwfEpWvzPdOl7ermg/w+ePsB4GJORMH5dqZbJFbTHXzFWOa23wmvD3W8vqSXIOLt0u9UVjZT1/E0NXO3zfKA0/tZzfrFWk/ULG+0Kd6EgOJswpMfZhQc+XpmxcPc9juj/p5CcZDI6RNO2p6jFHuU4gjxremmU9uTgD8HrxuOEQUPDATyei3Y39xolHUGD65dMP65uD5vJWvA0SoLIklILA4ZYLv1rlfrG6J+UOyqKSb3qmR8WRkbgcgOvTMBItaCTfOVpFtpkIiuIcY6LkaJWazKhnRp+7LKSsYA24FPLdc+gFHCOU7X82ot2DRdc3a5YPxzLBj/XEKnhpfsEoRd0sFpehO51gVGMutoVxf9gVMt9zu12ou1e8hufcvpyO4fvbqUwUfbeBKbLlPA2j17w5YhxHxLh0R0DT21bdV1GXBy8P5oLSTTpe1nNtey82MPzwZvm9ouA84Hal0uKrPUwCcaZlADls9We3KvVZIG7JR0cFrkdxLLl1vqeToQAMLFEtlqz269y273ULzrW8NO6Y+nqyvsC8HKmFP6AyfWfZMVh1BYJKpr6KntSF1Dz8oGq07Tqe3dhw6zrVelo64vGXiGY1CSaaymC+kLakrSgJ0SaomYoBk5XKbgmK+bPu4KrL1GrBngyGleG8bUblPw8UQbVMcjQrPiQZJupUM6dT0ZIxqO1HZkZUM6tZ0rc42GbbSbRkrOgKNlX5MxwWc21/L8xx5uOWdk3EsYx3x+rtL1jB3TYoom0R4Pnd6j/Nfi/8GUW56lV+9+KY9DyB7p1jXkp7aTJZHKEKvpDrr3roz2Sik5A47WUDrRLv3WhIfTWhfYl+vY7R5KxzEtqSTdtqx5nn2fr8ez9nkmXPWLpMcgZJ906hryU9upYK0Msfs76K1NqOCXV6h8DDLeJbCoqiDiad7hVFkwuVcl9RVuvkxgkd9pZ1ys56SyeygaqZhvp/codeteYDmKunUv0uk9mvQ4hPQTS9vp1DXkn7ZTIVplSKh8TNfZes+KuMvH0kVRRcCxvuUgfetMySQ8wL6iwkqyx7Sk2mBny5rnmaaCu48CAYmC84xY2k7n+mm+aTtVIr9MXtqwgTu/OyL0eOgzsz77O0SLxoDjnTKli2QSHm1dXRz1+WLuEkoUvbXpRI+HJERkRr8L/EZkMMffybh1LzJm0j/LWnAeUMraThXbL5Odezj/dxvYXt+Z49EV0RJEMlOmZIm2My7WTrh4dyglgtJ11Nzn+CjJb3Az+g37wAWjYCH3lLK2U+WlDRu4NhCu7as1Nwt+Nzfa07JGURhwtI5hmSCZta5EdyjFi1nrm2xjdTP6fcgfPp45/k5ZC84DSlnbqeBvbuTQvt28snMPD1pqmiG/tF0UBhxtymRHqp32k0l4ZCKKMdd9U0kabFnzPJcGdNsP3MW6X6LgHFOq2k4GvSV8a/Avj53DBMryWtsFvwYcT/0jEFYDGE+yLhqJJjyiJTWqKir4y84vWFi3l6ZjHQzsU8Xt1UOoGRH9FNl09XhoO7CDzyqrmODw+MCmHSm9vpA8partRLCWj4HRCOd4Syu0Q9uBf857bRe8AcdT/6ggJMpbxpyb1YSGOUa7KGaRZyvbD3Wyfl8lXv98oJpWbx2z1szm7S8O8OTVF0cV6pKaZSkfJX/VbbmPAgR7SlnbsbBullBznzuxBGf5PBSCtgvegGOVvugtrXy4vykkyk6/nlCzklSJFsVc4Kmni1F06puAXsFHRuD11/DBvvG8ueuLHsd7m9/4au5zHF+RmvkK+U2paTsWtr12AQr4gJeCN+BYU6ZnNtdyxtf7Q6J8tb6BT4L/F7OxQydaFHOhHuBdRnFCoCa98PpnsaDuvh4iVbpubI+MIToVCPCp5w1q1/2RtiP7QPnp6uomoHdT7j6Jc8ZeweU/mI2rvOAlULSUmradsEa7/3nB7w1dP/JH2o68UvC6LoxRJond+tRyjHZ5kJ196rsPHWaLu5zqbh/9Ktxh064jnd24OYJu+8zRNHV0hN1j7fFgNdj2o4307TeIsZdOBzRq1/2R1gM70ANuUD8GTgEagd8A1fh9dXg2zWLXtu9zx6/eKRixCifIB23vbGnlA6WYVFnRYzkhUW1bCSjFitqPWLTrME1eL0POOJVvXHQnKgCr51xGd2cHMBT4O0BhdAwuTF3n/whTwG596nqMbk0zg/dlOlKYP+XKUFOTH50zKuzDcMPSd2k46FTBUM/AqqrQLWvSTQUCvLH4HvbuasTX/WugmuMdW/ivP94NnIZSs4BqoA6YCzQDO4GTgq82AqjhWPsFrP7zQ1x5fX7URArxkw/aHjHgVN77al8PXUNi2rbSfWAf92zcwcbmk/HqTwDVHNq7Bc9XdwOnotTjnND2LOBc4IcYBV2Fp+uiKEOzw1o/eQS4BozjVTBEuhb4HnCMzO5Tj1YjeXv1EHqXzwYid+R00rt8NjOqhwA9Kx4+9bwRNN+1wHQM4XWj1GCU2my5bzqwARgA3AdcjvFRvRz4V6Acz8Y3WPz0TWz/5E+oiHpJIT/JB23Hqv2NV9sQfrLEm1/tZ13rALy6Vcemtj8kXNsfAbuAf+GEtqcCg/Bs/DPPzp2c99ouWgO2rk/NBt4F5nCiu36NprEBuLysLKFmJWadZWNbe9i/8Z4DZ/0w1Iw4i0sGd9O7fDywDPgMWEbv8vFcOtjHlGHf5o3aD7lxdQMT3t4REtNHa5cEI99OyrkSeA74JfAreq65VQD9gTXAzzAO+v5Z8HZ/YDMHGu/lndcXsWLxvXkrVOEE+aDtWLW/8Wh7+UcfMv21t5n8l4+4clMHv3Jfw+Nf901Q21X01HYjxqzwv/Je20W7BGFmkCcpxaGubpZjfGcuqawAINDt4w3iO8HVilln+cDq9/mkqZkHV7/Px03NtmttsZqauDSNp66+hDd3fcGCuvto6uhgYFUVM6qHMGXYt7nn7Q2s/6oiNBU73lHHO6/PQw/sBs6lnPm4+BtlbMPYSlFtM+JXgQ7gQ6zZaOOow0lALXATPl8Ne3ZO5NO6FYw67/r4/9BC1sm1tuNp1uOk7Vu/VcXVg87k5yvfZeORAXi9hrZpLE1t53UEnMqunvlTruTtGTfzw3NG8eOyMqYA08vK+NE5o/jhOaOY7nIlvHPHOu3yNDXz/4AtTc2O07B4djG5NI2pw8/i9tFDGNiniqaODhZ49vLwuo1B8w1fUvD51qECA4EnKOd3vAG4aQO+hbEuFsmLwP3YZaPh18ALods+3/3Url0S199CSI1C1na8u/NMbd/6rSrOqKxkf1sbr7WVM2P/GcYyg9e6hFaa2s5rA061wYfdPvpFnq28XLc1qb311mnX9cBLGMv/dmKPt6lJQCl+/tZ6Zq05TMPB+bR6P6Dh4Hz+sO0QXn0WduJSajblLGIaPqYAPyBAGacB/0bPNbc92EcPAKOBvWG32482OlwrpJNC1XYizXq6D+zjf65YzeyPYduRJzjUtYEtO36DZ9Nfg8sMou28NeB0NPiw+6b+diDANXog5rd3ZIQSKbxfYKS3ZgevjxRgvE1N/rLzi+BuoU1YowFFFc7iGko5x3gYY03rEby4eQejNGcicDdwCXAmRnrGLnoAqAeGhN3u2y/ysHsh3RSytmPp+qUNG2Ik1KZjVOOItiGPDTjVBh9239RHgD1KMdvcThPE7ts7MkKJFPxKjAjBSezxNjVZWLcXr38mPaOBoTiJq5x5XIcW9t5GpDASQ9yrgf8V/PdmjPx4z2y0EVX8U+i22z2PcZN+YvueQvooZG076rrCTX15GbuPdoROlnhi/8kOka5o2yQvk3DJduS3YrdL5yngYoh5blZkA+zrRn4nbDxHgKeBjRHvaY7zupHfibs5ddOxDuyjgX8EHsFIKFgF3EQ5y5lL+AftEbys5FF0vsuJpEQAaAEOB3/zBzCmZvUYwh0AjAWW4XbPY+iIMxlZPS3qeIXUKHRtnz3g1JCurVuD4USZpNmX2pjyi7ajkZcGnO7jtU2OdfvwBwKcB5S7XPSpcIc9xzwyJTJCeXD1+2GCfwq4CGexm1nkeMY7sE8Vrd46jOmZlZ8Cz6NpF6DUbExxVbj+J5MUlKme730p3bzHMPSQqF8FPseolVwOPIOxLjYYTWunqp8P3X8NffsNYtyk2xhZPQ3NlbeToqKgkLV9oa7z3v4DYUf6hDXCiaBvv0Ec7xBtRyPvDDgTx2un8v4zdZ1xB1o4uaKCC3yGyM3v52rA3ICpYQj/JLebtgMtcXekur16CLPWzMbrj4wGuikvb+ecC8bS9NXjoe3GvRnAlsNdTAi9s0LXu/H7uggE9Ijtny9i1E+eBNwU/Ak+Sy3jpD6Pc8u/vJLU30lInELWdpmm4VeKS4E/7P4ablxuHFcVpSfJuIk/5p3X5+HzibadyDsDTvfx2sm8/7VKcRrGDqNXgetcLvqfMzLsfc3txXeMHdPj/jbPVqboelyRTc2Is1i1fTcbmy/Eqz+EGQ0Y06YhXDFtTtzf3IufvokDjdbtn3uJliXOx6xwMVNo2v6nkWeFIt2Xj7Xz1d9qWezv4iY/cR3aOnLMdeyof5e9Oyfi892PaLsneWfAuTxZ1RohPImxw+gpekYoTockJrO+F2j5micuOpv7K2uoXXsiGkhm2tQz4hiCkeyInAJCvmaFi5lC0fbLnq2Gtnfu4aSfreH4wUO89MgEPk7w0FbN5eK6W57g07oVom0H8s6A03m8dqKYEcoxDHEuB24BriM8QrHLYlvvj3d9T281khUvTXieUZDyLp2eEcdUjGY8kVNAMyt8W0rvJyRGPmvbXNdd2LCTaS6NKbpxeOWGP/07SuF4aGusKFhzuRh13vWibQfyzoBziRmhXO7zc5WuMwW4AmNPfR93Oa5Dhx2j3MhssolTFGxmkNXc59LWUDoy4mg7so+A7sfvuwhdf5DwKWB+ZoWFzOCobZeLPuVlaEc7+P3o+SxaOYGPfX7AiHTHrn0BpRQLbA5tjScKThfFqm0xYAvzp1xJW1cX1yz6A78N3vcw8FfLnvpnNtfaRrkPRGSTTZzW90zzTfY0YyciIw4VCKRlCigUNtG0Pe3+Wnr17sfGtx7tEelO1X1sUfa6Ng+2jBUFp4ti1LYYcATRlhHMM7fsotzzD7TQr8LNZIdzrqzre2Z7yXSbrx3pmgIKhc/C9Ru4NhBusFdrbjxrn2fMpH+mbt0LPSLduQE/1cCFvfvj0nqaWi4PtiwGbYsBW4hVJtTp9ztGuZdpGsNsGlNHkq7TjAUhHky9tXX7+MOeRjZGtGQ0lxL8vk4uDej22i6r4NjFt2Yt0i0lxIAtxCoTWr/nK9pSyGKL+QrZIPKo9gXjn2PjW49yoW+d41LC1u1/4/M8P8K9GBEDthCrTGjMKf2TzmSL+QqZxnp45dZ7VoS2BAO0HdjBZ9EM9vThBXGMe7EhBmwh02VCS2qWQYscJS+kD6vphiV1LeYLiLnmKWLAWcD8kBwX8xXShNV4Q7OqLCR1hfQiBpxhZOlBSBdO3ceEwkUMOIOI+QrpwHGZQSh4xIAzhJivkArWaHfwFWOZ2x5sRCPmW1SIAWcAs8eDmK+QCJHlY6FKhvYcDkrIKGLAacaMXLbes6JHJloQ7Ii3kkEoPsSA04zZ4+GjFfLhEZyRJQYBxIDTir+50fgwyYdIcMC2fEyWGEoWMeA0YX6wQpGMIASxmm5YtCuUPGLAaUBvkYoHoScS7QqxEANOA+lurC4UNrbGKwg2aMq6tSbWxZrWAuzJ3HCEEmeoUmpAtt9UdC1kGEddJ2TAgiAIQvoojHM7BEEQihAxYEEQhBwhBpwimqbN0zTtnjivfUzTNKlBEgqCBLX9uqZpUzI9pmJD1oBTQNO0AcAWYLhSyqtp2neBl4BhwUtqgZ8rpbYFr/8GsBkYppTqzsWYBSEeIrUd8dgs4CHg+0qpvwbvGw88q5SKfiiiEIZEwKkxA3jTItCvgRuAvwNOwyhMe828WCm1H/gUmJbdYQpCwswgXNsAaJo2DEPj+633K6U2AydrmnZB1kZYBIgBp8Y1wPvmDaXUEaXUbmVMKzRAh9Ap4CbvATVZG6EgJEeYti38H+BXgN0M7j1E2wkhGzFSYzTQ47hYTdOOAFUYX3AzIx7eDkzP/NAEISV6aFvTtB8D3UqpNzVNs3vOdmBiFsZWNIgBp0Z/bDaXKqX6a5rWB7idngX+7cHnCUI+E6ZtTdOqgEeAq6I8R7SdIGLAqXEY6Gv3gFLqmKZpvwdaNE0bpZRqDj7UFziSrQEKQpJEans2sEgp9WWU54i2E0TWgFOjDvhOlMddwEnAIMt9owBPJgclCGkgUttXAD/XNK1J07Qm4ExgiaZpv7JcI9pOEDHg1HgTmGze0DTt+5qmna9pWpmmaScDj2FEEtstz5kMrMruMAUhYcK0jWHA5wLnBX++Bv478IzlGtF2gsgSRGq8BGzRNK13sFynP/A0MBjwAh8CU5RSnRCqA/4usDxH4xWEeAnTtlKq1fqgpmk6cFgp1RG8fSFwLFiOJsSJbMRIEU3THgGalVJPxHHt74DPlVL/N/MjE4TUSFDby4AXlVJvZn5kxYMYsCAIQo6QNWBBEIQcIQYsCIKQI8SABUEQckRCVRCn9O6lBvWtytRYhBKnoaX1YC6OJBJdC5kkmq4TMuBBfatYOv269IxKECIY9fv/yMm5bKJrIZNE07UsQQiCIOQIMWBBEIQcIQYsCIKQI8SABUEQcoQYsCAIQo4QAxYEQcgRYsCCIAg5QgxYEAQhR4gBC4Ig5AgxYEEQhBwhBiwIgpAjxICLiLauLu5YuYq2rq5cD0UQ0kqxalsMuIhY5NnK5sb9LPZszfVQBCGtFKu2xYCLhLauLl6pa2A58HJdQ9FFCkLpUszaFgMuEhZ5tnKtUkwBapQqukhBKF2KWdtiwEWAGSE8qOsAzNT1oosUhNKk2LUtBlwEmBHC8ODt4RRfpCCUJsWubTHgHJBqRtf6/MgIwaTYIgWhMEhF25HPLQVtiwHngFQzutbnL/Js5eJAgDLgS8tPGXBRIFA0kYJQGKSi7cjnloK2EzoTToyby58AABeNSURBVEgda0b35roGbhlzLidXVib9/Au/MZBtFW4mO1zvOnQ4HcMWhJikom275+4+dJi6Ite2GHCWscvo3jV+XNLP7z/gVJ6ouSpj4xWEeElF23bPnT/lyoyONx+QJYgskmpGt9gzwkLhkoo2S1nXYsBZJNWMbrFnhIXCJRVtlrKuxYCzRKoZ3VLICAuFSSraLHVdiwFniVQzuqWQERYKk1S0Weq6liRclkg1o1sKGWGhMElFm6WuazHgLJFqRjfTGeG2ri5+8fZqHr3q8oTK4gQhFW1mo9Ihn7UtSxACULzt/gQhn7UtEbAQVgT/D5563tl9mBavl4F9qri9egg1I87CpWm5HqYgJExbVxcvB7V9w8d1LN62j29WnZw3upYIOI/IVdd/axH81bpi56HxtHo/oOHgfGatOcTdb20goFRWxyQUF7nS9ktb6pmi60wBpuKmrfPGvNK1GHCGSUR4uZgqRZYBPYzCzSpgADAdr38zH+xz8+auL7I2JqEwKARtL/JsZU7QZB+hGzcLgSvyRtdiwBnGTngBpVj52efcsPRdJi5cyQ1L32VJw47QVCmb9Y92RfA/IEAZjwXv6YXXP4sFdXuzMh6hcCgEbU8NOGk7P3QtBpxB7I5SCSjFz99az6w1h2k4OD801Z+37rPQVClbu4CciuAfwYubx4AjwXtG09TRkfHxCIVDoWj7t4QvMYRrO/e6liRcBrFrMHJm//6s31eJ178J6BW8cgCoY8wJimWmrjMhiU5pyYzPWgRvUgZcip/3eAydOUA9A6uqMjYOofAoDm2PybmuxYAzhPkNvMnSYGRCXQOnn/xNvH5jCmRSzqNMw34vfCKd0hLFWgTfrQc45gsA/QEjM+ymHp1OepfPZkb1kIyNQygsCknbF9voGsDNFirKl+dc12LAGcKpwcjytoNAteXKI5TzBA/jDXt+NiIFaxG8OX1cv68Cr38WMBqop3f5eC4d7GPq8LMyMgah8CgkbTvrenZe6FoMOANERggmM3Wd1/ECG4ARgBEhTMRvO1Uy98JnMlIwcWkaT119CW/u+oIFdffR1NHBwKoqZlQPYerw3NdLCvlBoWk733UtBpwBoq0/TdTgfX5Bp/oJ0ItyGthIX6qpAo7Qx+2iouxEbjSbe+Fdmsa1I4Zx7YhhWXtPobAoRG3ns67FgDOAuf403udH6Tp9gXZAKyujsryMct8hernG0anPoZN/58SUaABPXn1xzr+VBcEJJ20rl4tupTit8jit/vF4/bNE23EgBpwB5k+5krauLq5Z9Ac2YayR7QImaBpTzx7BS3UNfO9bHew7ln9TIkGIhpO2xyqFTymu/+5whp7SPy+n+/mIGHCGsEtUfD8Q4LWt242DB/d9zapbb0zo0MJ87egklBaR2j4NUEqFDtRcdeuNCU33S1nbshEjQeLZfum0weGMQICpgUBSBelOWzlztcdeKD6S1faTwI8g6Y0WpaxtMeAEiWdPu12Xfw+wGJgbvCaZQwvttnK+UOthU+N+Xqz1JP9LCQLJadsDPAU8GHw82YNmI7Xd1tXFPyx9g0152kYyXYgBJ0A0I7QS2uDQqzL0c3lZGVdBSocWRkYYbV1dvFbfgAa8Ul/852cJmSNZbaeia3DW9gu1W2hs7+DGGOMpdMSAEyCaEVqnSvOnXMnbM24O/Sz96Q2gafw24vWSObTQ+pwXaj2htTdNKYmChaSx07bdEoBV26noGpy13djWzmtbt/MGsAojd1KsUbAYcJxEM8JYU7dUDy2023X04sceXqtvCK29XY9EwUJyOGn7hVpPxnRtPt9O2/f/9d0TuRJgYCBQtFFwSRtwov1MnYww1tTNOm0bX1ZGNTC+rIzJvSqpr3DzpUNBerQju1+tbyCg1Im1NyQKFgwSTV45afvV+vh1HantaLo2x+ik7a3NB/ll8PYDwMsUbxRc0mVo1sg12pbIaNsvx9Zvo4bwDHDka5n70s36yT8CN2saS396Q9Sym2i7jiYEFIcIX3szo+B/HDem5Mp5hBPEq2uIru3lwEXE1rX5OunS9iTgz8BYgl8GQP9AgEVZ6KKWbUo2Ao436QDRp1oTAgHOCASA2GtfketsL9Z6okYqdsk8M9LYCAyKuH4mgETBJU0iuobo2p6EUeEQz5puOrR9WWUlY4DtwKeWax/AqCAap+tFFwWXbARsl3RwihasbRuteIPbMQcEb0drtWfXwm9sfQPdUd7b6cjuH726lMFH23gSmyYnwNo9e7n3kvEx/gJCMZKIrsFe217LNuN+xG4hmS5tP7O5lp0fe3g2eNvUdhlwPlDrclGZxd4oiaK3NkFAJ5Fj5krSgJ36mTpNb+zEErkd08TptWLtHkpkajXslP54urp6fCGYjDmlf1yvIxQXieoaemo7UV1D+rS9+9BhtvWqdNT1JQPPcAxKsoXe2oSKWK4xGXzFWAAe6/Ubjre0nnjg9aGOr1eSBuyUdEikPV60NazIVnt262x2u4fife9ci1DIT7Kta0ivtvNd1/7mRsAw2rntd/a8oN38t7XnYw6UnAFHSzok0iTanLpdpuCYr5s+7gqsvUasrfYiRd2Gsba2Kcn3FoRI0q3ryYCy0XZkC8li17ZpulYe6/WbhEw2GiVnwIl+wzthfls/s7mW5z/2cMs5I+NeQz7m83OVrmf9mJZYlHJTlEIn3bqG4te23tIYc71WzX2OhSsi7mxJj/lCCRqwU0LNJJEm0daMc7S1LrtyHbvdQ7mOFBIpXxLyi3TqGgpf25GR68KGnWxu3M9LGzZw53dHhO5fUrMsfL02kkjzTTNFZcDxRHDpXGdKNONsPicdkUq6ifcDJ+SGWNpO9/ppIWjbbnnAyoLxzwHQ6T3KopUTWA7c+EUT3LicXr37GRelMZpNhqIy4GxGcMlknNu6uli+7VNwu5ns0Js6m0cQWUnmAydkj1LUdjxLBDEjWGDLmueZpoJbmwMBPGufZ8JVv0hoLJmiaAw42xFcMhnnRZ6ttHR2ccfYMXllbsl84ITsUezajhbJLqlZFvpvW6ONYb6d3qPUrXuBBX5jQ8gcfyfj1r3ImEn/fCIKziFFsxPOqVNZJoi2j91px1CiO5SySbQPnJB7ilXb/ubGkPkuqVnGgvHP9fg53tIa+kkGM/oN03YwCs4HisKAo3Uqc7o+lU77yXSByuaHKBGS+cAJ2aMYtW0a76B77woz2nRjRr8P+cP/FnP8ndSte5FO79G0v2eiFIUBx4rgIkUZT+f/aDj1aHDqbpbohyibpNpSUMgsxartJTXLePiD85IaY7xsWfM8lwZ0W21frPvzIgou+DXgeArQraK8Zcy5Ka+nJZpxjvYhuvPCsfxl5xcsrNtL07EOBvap4vbqIdSMyM4psukuXxLSR7Foe8HHn9Pk9TKwd29uHX4KU8/8RkYi3kjaDuzgs8oqJjg8PrBpR8bHEIuCN+BYpS8v1npYuu3TkCg7/XpWs/2xPkSelmN8uP8kvP75QDWt3jpmrZnN218c4MmrL864Cef79s9SpqC17dnKJ42HqD3YH6/+BFDNoa46Znrm8eLxk5h2YQDNldkJ+FW35T7CjUXBG3CsCO7LPXvDRPlqfQOfBGtbspHtj/YhulDXeX9fG52qHugVfGQEXn8NH+wbz5u7vkjoeG8rAaVCkfX+jnYUfjq6NfwBncqyCq4/+zR+M/EiyjP8IRCSp6C1HQjwfqufTn0zVm37fDXs2TmRT+tWMOq86xN+TxUI8KnnDWrX/ZG2I/tA+enq6iagd1PuPolzxl7B5T+Yjau8MKytMEYZhWgRnLkzx7o+tRyjWxNkZ5tktA9RW5cP1Lc4IVCTXnj9s1hQd19CBmya7oK6vew6dBhfoBLFTzA+Ho3Ab4BqOvU6Xts2i799+RdW31ojJpynFJy2AwFc7nKOq950dbaB/0zstO3z3U/t2scTMmAVCLB9y3L+tuIxujs7gKHA3wEKo2NwNX5fHZ5Ns9i17fvc8at3CsKE83+EKWC3PnU9wSbTwfsyHSnMn3Kl4y6miQtXctS72OGZo2nq6HB8XWuE23SsgzNO6oOm6Xxx5JTQcgbUAXOBZmAncFLw2SOAGlq8FzDvg408OOmSNPymQjbJB20/+PeTwnTtb24MbYx4du5kOjuctd1+1Ln21xrlth9tpOrkb6Jpfg42daPrj3NC27OAc4EfYtQTGLo+1n4Bq//8EFdePze9v3AGKNrQx7G8CkOka4HvAcfIfLbfKTM9sE8VhpDsqGdgVZXtIwGl+Plb65m15jANB+fT6v2Aba3TaDh4Ml7/JmA6hhinAxuA04HlEa/SC5jN6zsOJv+LCTkhX7QdreKib79BRNO28XhPVCDAG4vv4Z0/LeZA470c73ib5q8v4EDjcXR9A+Ha/gj4HHjN8gqGrhtq/5r8L5ZFitaAretTHgxBejjRXb9G09gAXB7HAYKpEK1I/fbqIfQunw10Rjyrk97ls5lRPcT2Nf+y8wvW76vE63+Lcp4F3gWWArOxm/LB/8aIhC/HiJUuB+4CHqXT38UNS99l5WefE0iklb+QM/JB27E2X4yb+GPc7nnYadvtnse4ST+xfd1PPW+wd1cjvu6VEdp+CHtt/5Ke2l6G33ecZ+dOZvHTN7H9kz+hgseG5RtFa8DWesbLy8pCgpzcq5LtlRUEgDcAggcIxlsNYNZdNra1h/0b7zlw1mihZsRZXDK4m97l44FlwGfAMnqXj+fSwT6mDPs2Kz/7nBuWvsvEhStDRrnAswevfyblPI2Lv1HGQxj/K6ttRhAAFgUf/xmwKvjvGqAS+JCGg/OZteYQd7+1IWcmHKuxinCCfNB2rM0XI8dcx5ARg3G7J2LVtts9kaEjzuTsc69l+yd/YvHTN4UZ5Udrl+Dr/nWK2q4DzuN4x39xoPFe3nl9ESsW35uXJqypBD5w555+mlo6/boMDif9mMmKl/1+bi4vZ9WtN7LIs5U2z1YW6jq3lZXRf8y5cScqzB6p5w88nU+amhk78HQ+bmq23QMfOt7F72c4sAuYEByDuSYXUIo3dxmJs6aODgZWVTGjeghThn2be97eEIx0Z2Kue/Uun40/sBdf4B16MYk/0cUP6UMn5wP3YEzNrLwM/A5YT3gE0Ylx7OK9wE0YUfd45kz+u6QrL5LFNN/Rr79Vq5S6IKtvTmHqGnKnbSddr5xyGa9P/M/QdSoQ4NO6FdSuXUL70Ub69hvEuEk/4exzr2XFK/8ajHR/jaltt3seemA3AX1V2rXtdk/k+9NvS6ryIlV+9+uhjrrO6wg41W2VYH9aa7K70qzTLk9TM/8P2NLU7DgNi6fHgkvTmDr8LG4fPYSBfapo6uhggWcvD6/bGDTf8DVdr38z/sAgyvkZ0/AxBfgBAco4Dfg3ek75nseofrCbvv0aeCF026i82Bvz75BOTPM1WweWCoWsbSddv7Jrd9hrai4XI6unMW7ij+nbbxDtRxupXbuE1StnBc13LVZt+3zrUIGBGdG2UXmxJObfIdvktQGnuq3Sbpvkq/UNSTeesQr+euAljPyr3TQs3h4Ldgm1hoPz+cO2Q8HIt6e4FL+inA95GGNK9Qhe3LyDUZozEbgbuAQ4E/gE++kbwGhgb9jtaJUX6aZUzRcKV9vRdP3qzj38aN1/4/Zpxn12CbUDjffi2fTXYORro231y4xpO1rlRa7IWwNOR/cwp9Na42k8ExmhRArvFxj1BbMdXiPeHgsnEmrhka6iCidxlbORaRD2QTMihZEYpWargf8V/PcsomWjYUjYbafKi3Sjt5Su+RaytqPqOhgFaw/cwYzNd1gSauGRrqHR7GvbqfIil+StAafaPczum/pJYDLE1XgmMkKJFPxKjAjBKdqIt6nJwrq9DpHuUOzFdYRy/oOHI+41IoVHgQ7gQ04I/l8xssQ9s9HGtO6fQrejVV6kE721CaWM87ZKkULWdixd7+n0U366YXS733rUIdLNvrajVV7kkrzciJGOBuF22yQ/xvhevABwlZXR2x3+65uNZyIbYF838jth4zkCPA1sjHhPc5zXjfwOR30+lv70hpjjbTrWgX008I/AI0ANVgGX829Mott2++eldPMew9BD1weAN4EDwMUYO4ZGB/8KM4EBwFiMyovZXDrYx9ThZ0UdbzpQum5/2GEJUOjaPnvAqXHpuvz0QRzo+phEtT0xA9p2u+cxdMSZjKyeFnXMuSAvDTiZjvyRRNsC3AcYM/AMx/KcyAjlwdXvhwn+KeAicGyS8sDq9/mkqTmu8Q7sU0Wrtw7jG93KT4Hn0RiHYg6muCq1Z/lIczG54sT/um49QKdfR1cKN0c4ERe9ilGovgtjI8YzGOtig9E4yjeqmunSLwtVXkwdnvkObOa6bymaLxS2ti/Udd7bfyDuscbUtnYBSs3G1HaF9ns+dLmZUNkH0ACFrnfj93URCOjxa1trp6qfD91/TbDy4jZGVk/LePOfZMg7A46nBV88kUKyXb7M939H17kGeFjX+fsDLZxcUcEFPh/+QACzcK8aQyYE/y13uTjJ7abtQEvcLQFvrx7CrDWz8frDowHopldZKz8aGcDTfJ+lRO18R6O8Yem7NBy803LPi8CvMNbObgr+GCiWcUrv+1g6/e8T/RMlTSkn3aCwtV3mcuEPBLgUY906nrFG03Z5eTvnXDCWpq8et5SozXE0ysVP38SBxji1rZZxUp/HueVfXknkz5MT8s6Ac31qsBkhrMTYg/Nn4DqXi/7njAx7X7NmMrJG8pnNtbR5tjJF1+OKbGpGnMVbXzSxft94vP5ZmNGAsSTg5zcT429J2VPwe4mWJZaKh+xSyNo2db1Q17ktzojdSdvGksAQrpg2J+6odNzEH/PO6/Pw+eLTdj5WPNiRdwacywbh1gjhKoyJza3AW7rO9y3f+k6HJCazvufSNJ66+pLgZoz7wjZjJLok0FPwp2MkOyKngJDNigeTUjZfKFxtA0mtW1u1/Z+197DbX570ksDIMdexo/5d9u6ciM93P7G0nY8VD3YU/U64RHhmcy1fbqlnRCDAIWAhcBtGic8Ol4th543mrvHjwqMBy24j6/0mie5GShXrzro9Rw5yzHcmiloidwpla9eb3tqE0nUG3XtXzCNoou0YyiTFrmtITdsKUta1v7mRrfes4KP1TUn/DtaddYcP7sDXfQZKfUSktnO5682OaLrOuwg4l+w+dJgt7nJWd3WHimRmAmOAU9zluA4ddoxyI7PJJtk+4t2laVw7YhjXjhgW2uRhv7yRvYqHwVeMZW6Gz/8SopOKtgNK8WGOdQ3GzrpR513PqPOuD23yOBERW5c38rPiwQ4xYAvzp1wZigCGBwU3HJheVkb/c0aFIgS7LPYDEdlkk2yt79mRzuWNZDA3W8xtvzPGlUKmSVbbV+s6nyiVFl2f+8Q0PkrTMpTmcnHdLU8EI2JrIi9/Kx7sEAO2ECtLHS3KPf9AC/0q3Ex2MLVcHW5pjYizjVLG6bdk4QBGITrJanuOUowBJlVW2H5hx6PrTHW6s0bEhYoYsIVYWerImknr45dpGsOCkYRw4kOXjdNvhdikou3vuVxJaVtvacRMMZV6AtYJMWALsbLUZUfb2SdHuMdESs7yj1xpu1R3PMaLGLAFOaI9dUq5yU4+I9rOTwpjpVooGJSCQffelethCHmC9sAduR5CXiMGLKQNc+khVr2vUBqUDTA2Q5w04NQcjyR/EQMW0sqSmmW5HoKQR2ga/GtnZINJwUQMWEgLUvUg2OIqY9/fPs71KPIWMWAhZSTxJjhRdupAAGZslrVgO8SAhZSRxJsQDfOEDKEnYsBCSkjiTYiXCy4ZmOsh5B1iwELKbL1HKu0FIRnEgIWkMaPfVFoMCkIpIwYsJIXeapiuJN6EeNDKyjj3icJoEZlNxICFpDD7/ApCPJjVEEI4CZ2IoWlaC7Anc8MRSpyhSqkB2X5T0bWQYRx1nZABC4IgCOlDliAEQRByhBiwIAhCjhADFgRByBFiwIIgCDlCDFgQBCFHiAELgiDkCDFgQRCEHCEGLAiCkCPEgAVBEHLE/wf7sr0MtNfANgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.svm import LinearSVC, SVC\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "# Define the classifiers\n",
    "classifiers = [LogisticRegression(), LinearSVC(), SVC(), KNeighborsClassifier()]\n",
    "\n",
    "# Fit the classifiers\n",
    "for c in classifiers:\n",
    "    c.fit(X, y)\n",
    "    \n",
    "# Plot the classifiers\n",
    "plot_4_classifiers(X, y, classifiers)\n",
    "plt.savefig('../images/plot_4_classifiers.png')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}