{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "Linear Regression\n", "====================================\n", "\n", "*Important:* Please read the [installation page](http://gpeyre.github.io/numerical-tours/installation_python/) for details about how to install the toolboxes.\n", "$\\newcommand{\\dotp}[2]{\\langle #1, #2 \\rangle}$\n", "$\\newcommand{\\enscond}[2]{\\lbrace #1, #2 \\rbrace}$\n", "$\\newcommand{\\pd}[2]{ \\frac{ \\partial #1}{\\partial #2} }$\n", "$\\newcommand{\\umin}[1]{\\underset{#1}{\\min}\\;}$\n", "$\\newcommand{\\umax}[1]{\\underset{#1}{\\max}\\;}$\n", "$\\newcommand{\\umin}[1]{\\underset{#1}{\\min}\\;}$\n", "$\\newcommand{\\uargmin}[1]{\\underset{#1}{argmin}\\;}$\n", "$\\newcommand{\\norm}[1]{\\|#1\\|}$\n", "$\\newcommand{\\abs}[1]{\\left|#1\\right|}$\n", "$\\newcommand{\\choice}[1]{ \\left\\{ \\begin{array}{l} #1 \\end{array} \\right. }$\n", "$\\newcommand{\\pa}[1]{\\left(#1\\right)}$\n", "$\\newcommand{\\diag}[1]{{diag}\\left( #1 \\right)}$\n", "$\\newcommand{\\qandq}{\\quad\\text{and}\\quad}$\n", "$\\newcommand{\\qwhereq}{\\quad\\text{where}\\quad}$\n", "$\\newcommand{\\qifq}{ \\quad \\text{if} \\quad }$\n", "$\\newcommand{\\qarrq}{ \\quad \\Longrightarrow \\quad }$\n", "$\\newcommand{\\ZZ}{\\mathbb{Z}}$\n", "$\\newcommand{\\CC}{\\mathbb{C}}$\n", "$\\newcommand{\\RR}{\\mathbb{R}}$\n", "$\\newcommand{\\EE}{\\mathbb{E}}$\n", "$\\newcommand{\\Zz}{\\mathcal{Z}}$\n", "$\\newcommand{\\Ww}{\\mathcal{W}}$\n", "$\\newcommand{\\Vv}{\\mathcal{V}}$\n", "$\\newcommand{\\Nn}{\\mathcal{N}}$\n", "$\\newcommand{\\NN}{\\mathcal{N}}$\n", "$\\newcommand{\\Hh}{\\mathcal{H}}$\n", "$\\newcommand{\\Bb}{\\mathcal{B}}$\n", "$\\newcommand{\\Ee}{\\mathcal{E}}$\n", "$\\newcommand{\\Cc}{\\mathcal{C}}$\n", "$\\newcommand{\\Gg}{\\mathcal{G}}$\n", "$\\newcommand{\\Ss}{\\mathcal{S}}$\n", "$\\newcommand{\\Pp}{\\mathcal{P}}$\n", "$\\newcommand{\\Ff}{\\mathcal{F}}$\n", "$\\newcommand{\\Xx}{\\mathcal{X}}$\n", "$\\newcommand{\\Mm}{\\mathcal{M}}$\n", "$\\newcommand{\\Ii}{\\mathcal{I}}$\n", "$\\newcommand{\\Dd}{\\mathcal{D}}$\n", "$\\newcommand{\\Ll}{\\mathcal{L}}$\n", "$\\newcommand{\\Tt}{\\mathcal{T}}$\n", "$\\newcommand{\\si}{\\sigma}$\n", "$\\newcommand{\\al}{\\alpha}$\n", "$\\newcommand{\\la}{\\lambda}$\n", "$\\newcommand{\\ga}{\\gamma}$\n", "$\\newcommand{\\Ga}{\\Gamma}$\n", "$\\newcommand{\\La}{\\Lambda}$\n", "$\\newcommand{\\si}{\\sigma}$\n", "$\\newcommand{\\Si}{\\Sigma}$\n", "$\\newcommand{\\be}{\\beta}$\n", "$\\newcommand{\\de}{\\delta}$\n", "$\\newcommand{\\De}{\\Delta}$\n", "$\\newcommand{\\phi}{\\varphi}$\n", "$\\newcommand{\\th}{\\theta}$\n", "$\\newcommand{\\om}{\\omega}$\n", "$\\newcommand{\\Om}{\\Omega}$\n", "$\\newcommand{\\eqdef}{\\equiv}$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This tour studies linear regression method in conjunction with\n", "regularization.\n", "It contrasts ridge regression and the Lasso." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We recommend that after doing this Numerical Tours, you apply it to your\n", "own data, for instance using a dataset from .\n", "\n", "_Disclaimer:_ these machine learning tours are intended to be\n", "overly-simplistic implementations and applications of baseline machine learning methods.\n", "For more advanced uses and implementations, we recommend\n", "to use a state-of-the-art library, the most well known being\n", "" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Usefull functions to convert to a column/row vectors." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# convert to a column vector\n", "def MakeCol(y): return y.reshape(-1,1)\n", "# convert to a row vector\n", "def MakeRow(y): return y.reshape(1,-1)\n", "# find non zero/true elements\n", "def find(x): return np.nonzero(x)[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Dataset Loading\n", "---------------\n", "We test the method on the prostate dataset in $n=97$ samples with\n", "features $x_i \\in \\RR^p$ in dimension $p=8$. The goal is to predict the price value\n", "$y_i \\in \\RR$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Load the dataset." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from scipy import io\n", "name = 'prostate';\n", "U = io.loadmat('nt_toolbox/data/ml-' + name)\n", "A = U['A']\n", "class_names = U['class_names']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Randomly permute it." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "A = A[np.random.permutation(A.shape[0]),:]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Separate the features $X$ from the data $y$ to predict information." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "X = A[:,0:-2];\n", "y = MakeCol( A[:,-2] )\n", "c = MakeCol( A[:,-1] )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$n$ is the number of samples, $p$ is the dimensionality of the features," ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "97 8\n" ] } ], "source": [ "[n,p] = X.shape\n", "print(n,p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Split into training and testing." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "I0 = find(c==1) # train\n", "I1 = find(c==0) # test\n", "n0 = I0.size\n", "n1 = n-n0\n", "X0 = X[I0,:]\n", "y0 = y[I0]\n", "X1 = X[I1,:]\n", "y1 = y[I1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Normalize the features by the mean and std of the *training* set.\n", "This is optional." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "mX0 = X0.mean(axis=0)\n", "sX0 = X0.std(axis=0)\n", "X0 = (X0-mX0)/sX0\n", "X1 = (X1-mX0)/sX0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Remove the mean (computed from the *test* set) to avoid introducing a bias term and a constant regressor.\n", "This is optional." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "m0 = y0.mean()\n", "y0 = y0-m0\n", "y1 = y1-m0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Dimenionality Reduction and PCA\n", "-------------------------------\n", "In order to display in 2-D or 3-D the data, dimensionality is needed.\n", "The simplest method is the principal component analysis, which perform an\n", "orthogonal linear projection on the principal axsis (eigenvector) of the\n", "covariance matrix.\n", "\n", "\n", "Display the covariance matrix of the training set." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAAD8CAYAAABaQGkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADCNJREFUeJzt3VuIXfUVx/Hfz5MxlzFpqLXROqHRGgMi1UgQJCBUe9Eq2kILChUqBR+KErEotm998KWlogUrSNQKWqX1AuK1gtpWqJprrTFRkqhkjDFRK7mMNTOT1Yc5sWMydfbk7P0/Z1a/Hxgyc2az1zpJfvPfZ88+ezkiBCCno7rdAIDmEHAgMQIOJEbAgcQIOJAYAQcSI+BAYgQcSIyAA4nNaGKnX/piKxYt7Gti14fZ8N5xRepIkgpf9Nc3dKBYreEF5Z7crBkjxWoN7ZtZrJYkqVXm73Hkg39pdM8+T7ZdIwFftLBPLz+9sIldH+brv/lpkTqS5NFipSRJC1YNFav13vWfFKu1+NhdxWqtW3VKsVqSdKC/zH+SHTf9ttJ2HKIDiRFwIDECDiRGwIHECDiQGAEHEiPgQGIEHEisUsBtX2D7ddubbd/YdFMA6jFpwG23JN0m6UJJp0m63PZpTTcGoHNVVvCzJW2OiK0RsV/SA5IubbYtAHWoEvATJW0b9/Vg+zEAPa5KwCd6x8phb5mxfZXt1bZX7/qg8LsyAEyoSsAHJY1/a9iApO2HbhQRd0TEsohYdtyxrbr6A9CBKgFfJWmx7ZNsHy3pMkmPNtsWgDpM+n7wiBixfbWkpyW1JN0VERsa7wxAxyrd8CEinpD0RMO9AKgZV7IBiRFwIDECDiRGwIHECDiQGAEHEiPgQGIEHEjMEfWPWpl9/ML42hXX1b7fibzys98VqSNJp99aboqKJH18QrnRRQteLFZKRw2XG5M0d8ueYrUkaWRumVFJq9bept173pl0dBErOJAYAQcSI+BAYgQcSIyAA4kRcCAxAg4kRsCBxAg4kFiVySZ32d5p+9USDQGoT5UV/PeSLmi4DwANmDTgEfFXSR8W6AVAzXgNDiRWW8DHjy4aHdpX124BdKC2gI8fXdSa01/XbgF0gEN0ILEqvya7X9LfJS2xPWj7J823BaAOVWaTXV6iEQD14xAdSIyAA4kRcCAxAg4kRsCBxAg4kBgBBxIj4EBik17ockRC8mgjez5MyXFCr64oNyZJks74Vbnn1rd3pFit4f5y68qb359frJYkzfqgTJ2RTdX+DlnBgcQIOJAYAQcSI+BAYgQcSIyAA4kRcCAxAg4kRsCBxAg4kFiVmy4utP2c7Y22N9heUaIxAJ2rci36iKSfRcRa23MlrbH9TES81nBvADpUZTbZuxGxtv35HkkbJZ3YdGMAOjel1+C2F0laKumlCb7339FFHzO6COgFlQNu+xhJD0m6NiJ2H/r9z4wums3oIqAXVAq47T6Nhfu+iHi42ZYA1KXKWXRLulPSxoi4ufmWANSlygq+XNIVks6zvb798d2G+wJQgyqzyV6Q5AK9AKgZV7IBiRFwIDECDiRGwIHECDiQGAEHEiPgQGIEHEiskdlkfUMHtGDVUBO7PsyWH84qUkcqOytMkv5xQ7lZaN/5ypnFavXP/0KxWvM2DxSrJUmtD/cWqfP2R8OVtmMFBxIj4EBiBBxIjIADiRFwIDECDiRGwIHECDiQGAEHEqty08VZtl+2/Y/26KJflmgMQOeqXKr6iaTzImJv+/bJL9h+MiJebLg3AB2qctPFkHTwAtu+9kc02RSAelQdfNCyvV7STknPRMTnji4aHmZ0EdALKgU8IkYj4kxJA5LOtn36BNt8Orqor4/RRUAvmNJZ9Ij4SNLzki5opBsAtapyFv042/Pbn8+W9E1Jm5puDEDnqpxFP0HSPbZbGvuB8MeIeKzZtgDUocpZ9Fc0NhMcwDTDlWxAYgQcSIyAA4kRcCAxAg4kRsCBxAg4kBgBBxJrZHTR8ILQe9d/0sSuD7PggXKji/r2jhSrJZUdJ/T09vXFal38xoXFam17vNyYJEnae8qcInX+fVO16LKCA4kRcCAxAg4kRsCBxAg4kBgBBxIj4EBiBBxIjIADiVUOePve6Otscz82YJqYygq+QtLGphoBUL+qk00GJF0kaWWz7QCoU9UV/BZJN0g60GAvAGpWZfDBxZJ2RsSaSbb7dDbZ6O6h2hoEcOSqrODLJV1i+y1JD0g6z/a9h240fjZZa16Zt8wB+HyTBjwifh4RAxGxSNJlkp6NiB813hmAjvF7cCCxKd3RJSKe19h0UQDTACs4kBgBBxIj4EBiBBxIjIADiRFwIDECDiRGwIHEGhldNGvGiBYfu6uJXR9mx/C8InUkabi/7M/D/vnlxu6UHCf02KlPFqt10qaritWSpCWLtxep86+Zw5W2YwUHEiPgQGIEHEiMgAOJEXAgMQIOJEbAgcQIOJAYAQcSq3QlW/uOqnskjUoaiYhlTTYFoB5TuVT1GxHxfmOdAKgdh+hAYlUDHpL+bHuN7bJX7wM4YlUP0ZdHxHbbX5b0jO1NEfHX8Ru0g3+VJM1ecEzNbQI4EpVW8IjY3v5zp6RHJJ09wTafji6aOX92vV0COCJVhg/225578HNJ35b0atONAehclUP0BZIesX1w+z9ExFONdgWgFpMGPCK2SjqjQC8AasavyYDECDiQGAEHEiPgQGIEHEiMgAOJEXAgMQIOJNbI6KKhfTO1btUpTez6MKdu2V2kjiS9+f35xWpJ0rzNA8VqbXu83JikkuOE3rzkjmK1JOn6HUuL1Hmlxegi4P8eAQcSI+BAYgQcSIyAA4kRcCAxAg4kRsCBxAg4kFilgNueb/tB25tsb7R9TtONAehc1UtVb5X0VET8wPbRkuY02BOAmkwacNvzJJ0r6ceSFBH7Je1vti0AdahyiH6ypF2S7ra9zvbK9v3RAfS4KgGfIeksSbdHxFJJ+yTdeOhGtq+yvdr26tF9+2puE8CRqBLwQUmDEfFS++sHNRb4zxg/uqjVzwIP9IJJAx4ROyRts72k/dD5kl5rtCsAtah6Fv0aSfe1z6BvlXRlcy0BqEulgEfEeknLGu4FQM24kg1IjIADiRFwIDECDiRGwIHECDiQGAEHEiPgQGIEHEiskdlkaoUO9I82sutDjcydWaSOJM36oFgpSVLrw73Fau09pdw9PJYs3l6sVqlZYQf9+vh1Rer8pW+o0nas4EBiBBxIjIADiRFwIDECDiRGwIHECDiQGAEHEiPgQGKTBtz2Etvrx33stn1tieYAdGbSS1Uj4nVJZ0qS7ZakdyQ90nBfAGow1UP08yVtiYi3m2gGQL2mGvDLJN0/0Tc+M7poD6OLgF5QOeDtoQeXSPrTRN//zOiiuYwuAnrBVFbwCyWtjYj3mmoGQL2mEvDL9T8OzwH0pkoBtz1H0rckPdxsOwDqVHU22ZCkYxvuBUDNuJINSIyAA4kRcCAxAg4kRsCBxAg4kBgBBxIj4EBijoj6d2rvkjTVt5R+SdL7tTfTG7I+N55X93w1Io6bbKNGAn4kbK+OiGXd7qMJWZ8bz6v3cYgOJEbAgcR6KeB3dLuBBmV9bjyvHtczr8EB1K+XVnAANeuJgNu+wPbrtjfbvrHb/dTB9kLbz9neaHuD7RXd7qlOtlu219l+rNu91Mn2fNsP2t7U/rc7p9s9daLrh+jte62/obE7xgxKWiXp8oh4rauNdcj2CZJOiIi1tudKWiPpe9P9eR1k+zpJyyTNi4iLu91PXWzfI+lvEbGyfaPRORHxUbf7OlK9sIKfLWlzRGyNiP2SHpB0aZd76lhEvBsRa9uf75G0UdKJ3e2qHrYHJF0kaWW3e6mT7XmSzpV0pyRFxP7pHG6pNwJ+oqRt474eVJIgHGR7kaSlkl7qbie1uUXSDZIOdLuRmp0saZeku9svP1bantb3AO+FgHuCx9Kc2rd9jKSHJF0bEbu73U+nbF8saWdErOl2Lw2YIeksSbdHxFJJ+yRN63NCvRDwQUkLx309IGl7l3qple0+jYX7vojIckfa5ZIusf2Wxl5OnWf73u62VJtBSYMRcfBI60GNBX7a6oWAr5K02PZJ7ZMal0l6tMs9dcy2NfZabmNE3NztfuoSET+PiIGIWKSxf6tnI+JHXW6rFhGxQ9I220vaD50vaVqfFK102+QmRcSI7aslPS2pJemuiNjQ5bbqsFzSFZL+aXt9+7FfRMQTXewJk7tG0n3txWarpCu73E9Huv5rMgDN6YVDdAANIeBAYgQcSIyAA4kRcCAxAg4kRsCBxAg4kNh/AMlu8KMeZsRmAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "C = X0.transpose().dot(X0)\n", "plt.imshow(C);" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADkRJREFUeJzt3X2IZfV9x/H3p64SNRFjHe3W1Y6BxUYKPjBYU0GoG1NTg+4fMSitLGXL/pOKtoV00/8CoRgoefijBBY12VDrQzcRRcVGNkoaSDWzPsSHNZjYjW7cuJNGq4ZSq/n2jzmTDJsZ7507cz13f75fMNx7zj2X80Vn33Pm3HvupKqQJB3+fqvvASRJa8OgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNWLdO7mzE088saanp9/JXUrSYW/Pnj0/q6qpQdu9o0Gfnp5mdnb2ndylJB32kvx4mO085SJJjRgq6EmOT7IryTNJ9ib5UJITktyf5Nnu9v3jHlaStLxhj9C/BNxXVb8PnAXsBbYDu6tqI7C7W5Yk9WRg0JMcB1wI3AhQVW9U1SvA5cDObrOdwOZxDSlJGmyYI/QPAHPAV5I8muSGJMcCJ1fVAYDu9qQxzilJGmCYoK8DzgW+XFXnAL9gBadXkmxLMptkdm5ubsQxJUmDDBP0/cD+qnqoW97FfOBfSrIeoLs9uNSTq2pHVc1U1czU1MC3UUqSRjQw6FX1U+CFJGd0qzYBTwN3AVu6dVuAO8cyoSRpKMNeWHQNcHOSo4DngL9g/ofB7Um2As8DV4xnREnSMIYKelU9Bsws8dCmtR1nedPb73mndvUb9l1/aW/7lqRheaWoJDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSI9YNs1GSfcBrwFvAm1U1k+QE4DZgGtgHfKKqXh7PmJKkQVZyhP7HVXV2Vc10y9uB3VW1EdjdLUuSerKaUy6XAzu7+zuBzasfR5I0qmGDXsA3k+xJsq1bd3JVHQDobk8ax4CSpOEMdQ4duKCqXkxyEnB/kmeG3UH3A2AbwGmnnTbCiJKkYQx1hF5VL3a3B4E7gPOAl5KsB+huDy7z3B1VNVNVM1NTU2sztSTpNwwMepJjk7xv4T7wEeBJ4C5gS7fZFuDOcQ0pSRpsmFMuJwN3JFnY/l+q6r4k3wNuT7IVeB64YnxjSpIGGRj0qnoOOGuJ9f8FbBrHUJKklfNKUUlqhEGXpEYYdElqhEGXpEYYdElqhEGXpEYYdElqhEGXpEYYdElqhEGXpEYYdElqhEGXpEYYdElqhEGXpEYM+yfopHeV6e339Lbvfddf2tu+dXjzCF2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGjF00JMckeTRJHd3y6cneSjJs0luS3LU+MaUJA2ykiP0a4G9i5Y/B3yhqjYCLwNb13IwSdLKDBX0JBuAS4EbuuUAFwG7uk12ApvHMaAkaTjDHqF/EfgU8Mtu+beBV6rqzW55P3DKGs8mSVqBgUFP8jHgYFXtWbx6iU1rmedvSzKbZHZubm7EMSVJgwxzhH4BcFmSfcCtzJ9q+SJwfJKFP5CxAXhxqSdX1Y6qmqmqmampqTUYWZK0lIFBr6pPV9WGqpoGrgS+VVV/BjwAfLzbbAtw59imlCQNtJo/Qfd3wK1JPgs8Cty4NiNJejv+eTwtZ0VBr6oHgQe7+88B5639SIcf/4FJmgReKSpJjTDoktQIgy5JjTDoktQIgy5JjTDoktQIgy5JjTDoktQIgy5JjVjNpf+SdNh4N1zR7RG6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDViYNCTvCfJw0keT/JUks90609P8lCSZ5PcluSo8Y8rSVrOMEfo/wtcVFVnAWcDlyQ5H/gc8IWq2gi8DGwd35iSpEEGBr3mvd4tHtl9FXARsKtbvxPYPJYJJUlDGeocepIjkjwGHATuB34EvFJVb3ab7AdOWea525LMJpmdm5tbi5klSUsYKuhV9VZVnQ1sAM4DPrjUZss8d0dVzVTVzNTU1OiTSpLe1ore5VJVrwAPAucDxydZ1z20AXhxbUeTJK3EMO9ymUpyfHf/aODDwF7gAeDj3WZbgDvHNaQkabB1gzdhPbAzyRHM/wC4varuTvI0cGuSzwKPAjeOcU5J0gADg15V3wfOWWL9c8yfT5ckTQCvFJWkRhh0SWqEQZekRgzzoqgkDWV6+z297Xvf9Zf2tu9J4RG6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSI9YN2iDJqcDXgN8BfgnsqKovJTkBuA2YBvYBn6iql8c3qlozvf2e3va97/pLe9u3NC7DHKG/CfxtVX0QOB/4ZJIzge3A7qraCOzuliVJPRkY9Ko6UFWPdPdfA/YCpwCXAzu7zXYCm8c1pCRpsBWdQ08yDZwDPAScXFUHYD76wEnLPGdbktkks3Nzc6ubVpK0rKGDnuS9wNeB66rq1WGfV1U7qmqmqmampqZGmVGSNIShgp7kSOZjfnNVfaNb/VKS9d3j64GD4xlRkjSMgUFPEuBGYG9VfX7RQ3cBW7r7W4A71348SdKwBr5tEbgAuBp4Islj3bq/B64Hbk+yFXgeuGI8I0qShjEw6FX1HSDLPLxpbceRJI3KK0UlqRHDnHLRYcyrMaV3D4/QJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGmHQJakRBl2SGjEw6EluSnIwyZOL1p2Q5P4kz3a37x/vmJKkQYY5Qv8qcMkh67YDu6tqI7C7W5Yk9Whg0Kvq28DPD1l9ObCzu78T2LzGc0mSVmjUc+gnV9UBgO72pLUbSZI0irG/KJpkW5LZJLNzc3Pj3p0kvWuNGvSXkqwH6G4PLrdhVe2oqpmqmpmamhpxd5KkQUYN+l3Alu7+FuDOtRlHkjSqYd62eAvwXeCMJPuTbAWuBy5O8ixwcbcsSerRukEbVNVVyzy0aY1nkSStgleKSlIjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNcKgS1IjDLokNWJVQU9ySZIfJPlhku1rNZQkaeVGDnqSI4B/Aj4KnAlcleTMtRpMkrQyqzlCPw/4YVU9V1VvALcCl6/NWJKklVpN0E8BXli0vL9bJ0nqQapqtCcmVwB/UlV/2S1fDZxXVdccst02YFu3eAbwg9HHXZUTgZ/1tO9BnG00zjYaZxtNn7P9XlVNDdpo3Sp2sB84ddHyBuDFQzeqqh3AjlXsZ00kma2qmb7nWIqzjcbZRuNso5nk2Ras5pTL94CNSU5PchRwJXDX2owlSVqpkY/Qq+rNJH8F/BtwBHBTVT21ZpNJklZkNadcqKp7gXvXaJZx6/20z9twttE422icbTSTPBuwihdFJUmTxUv/JakRzQc9yU1JDiZ5su9ZFktyapIHkuxN8lSSa/ueaUGS9yR5OMnj3Wyf6XumQyU5IsmjSe7ue5bFkuxL8kSSx5LM9j3PYkmOT7IryTPd992H+p4JIMkZ3X+vha9Xk1zX91wLkvx19+/gySS3JHlP3zMtp/lTLkkuBF4HvlZVf9D3PAuSrAfWV9UjSd4H7AE2V9XTPY9GkgDHVtXrSY4EvgNcW1X/0fNov5Lkb4AZ4Liq+ljf8yxIsg+YqaqJey91kp3Av1fVDd07046pqlf6nmux7iNFfgL8YVX9eALmOYX57/8zq+p/ktwO3FtVX+13sqU1f4ReVd8Gft73HIeqqgNV9Uh3/zVgLxNypW3Ne71bPLL7mpif/Ek2AJcCN/Q9y+EiyXHAhcCNAFX1xqTFvLMJ+NEkxHyRdcDRSdYBx7DE9TaTovmgHw6STAPnAA/1O8mvdac0HgMOAvdX1cTMBnwR+BTwy74HWUIB30yyp7tKelJ8AJgDvtKdqrohybF9D7WEK4Fb+h5iQVX9BPhH4HngAPDfVfXNfqdankHvWZL3Al8HrquqV/ueZ0FVvVVVZzN/BfB5SSbidFWSjwEHq2pP37Ms44KqOpf5TyH9ZHfKbxKsA84FvlxV5wC/ACbqI6+700CXAf/a9ywLkryf+Q8dPB34XeDYJH/e71TLM+g96s5Pfx24uaq+0fc8S+l+LX8QuKTnURZcAFzWnau+FbgoyT/3O9KvVdWL3e1B4A7mP5V0EuwH9i/6TWsX84GfJB8FHqmql/oeZJEPA/9ZVXNV9X/AN4A/6nmmZRn0nnQvPN4I7K2qz/c9z2JJppIc390/mvlv6mf6nWpeVX26qjZU1TTzv55/q6om4ogpybHdC9x0pzM+AkzEu6uq6qfAC0nO6FZtAnp/Af4QVzFBp1s6zwPnJzmm+ze7ifnXuyZS80FPcgvwXeCMJPuTbO17ps4FwNXMH2EuvF3rT/seqrMeeCDJ95n/zJ77q2qi3h44oU4GvpPkceBh4J6quq/nmRa7Bri5+/96NvAPPc/zK0mOAS5m/gh4YnS/0ewCHgGeYL6ZE3vFaPNvW5Skd4vmj9Al6d3CoEtSIwy6JDXCoEtSIwy6JDXCoEtSIwy6JDXCoEtSI/4f3WSJYxNziwAAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "u = X0.transpose().dot(y0)\n", "plt.clf\n", "plt.bar(np.arange(1,p+1),u.flatten())\n", "plt.axis('tight');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compute PCA ortho-basis and\n", "the feature in the PCA basis." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "U, s, V = np.linalg.svd(X0)\n", "X0r = X0.dot( V.transpose() )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Plot sqrt of the eigenvalues." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHohJREFUeJzt3Xl0leW99vHvb2dnYMoACYQESJhBQIJEjQTUClZEBVpPB1qttSptHUrbt8vX1nN6Ttc5bU97PK2tUupYbUWtI1DnoVgBQU0wzCAhEiAJIYQhEIQM+37/SMqLSiBk2M9+dq7PWq6QzcbnWiy9uLn389w/c84hIiL+F/A6gIiIdAwVuohIlFChi4hECRW6iEiUUKGLiEQJFbqISJRQoYuIRAkVuohIlFChi4hEiWA4L5aamuqys7PDeUkREd8rLCzc65xLO937wlro2dnZFBQUhPOSIiK+Z2alrXmftlxERKKECl1EJEqo0EVEooQKXUQkSqjQRUSihApdRCRK+KLQC0v3M39pMYWl+72OIiISscJ6H3pbFJbuZ879q6hvDBEfG2DhjXlMzErxOpaISMSJ+BX6qpJq6htDOKCuIcSqkmqvI4mIRKSIL/S8IX2ICwY+8b2IiHxWxBf6xKwUHr8pj4tHpBFy8HFdo9eRREQiUsQXOjSV+h+vnUhmcjd+8dImQiHndSQRkYjji0IHSIiN4fbpI9lYUcOiojKv44iIRBzfFDrAVWdnMC4zibte3cLRem29iIicyFeFHggYP54xivKDR/nTiu1exxERiSi+KnSASUNTuWRUX/6wtJh9tXVexxERiRi+K3SAH18+itq6Bn7/5lavo4iIRAxfFvrwfr34yrkDeWxVKdv31nodR0QkIviy0AF+MG0EsTEB/ufVLV5HERGJCL4t9L6JCdx04RBeXFfB6h06tEtExLeFDvDtC4eQ2jOeX7y4Cef0sJGIdG2+LvQe8UF+cOlwCkr38+qGSq/jiIh4yteFDvCV3IEM69uTX7+ymfrGkNdxREQ84/tCD8YEuGP6KEr21vLkezu8jiMi4pnTFrqZPWxme8xs/Ul+7kdm5swstXPitc7U0X05b3Bv7n5jK4eO1nsZRUTEM61ZoT8CTP/0i2Y2ELgU8HxZbGbcOWM01bV13PePEq/jiIh44rSF7px7G9h3kp/6LXA7EBG3l4wfmMxV4zN4cHkJuw8e9TqOiEjYtWkP3cxmAmXOuTUdnKddbr9sJKEQ/OZ1PWwkIl3PGRe6mXUH7gR+2sr3zzWzAjMrqKqqOtPLnZGBvbvzjQuyeLpwF5t313TqtUREIk1bVuhDgcHAGjPbDgwAVptZ+sne7Jy73zmX65zLTUtLa3vSVrr1kmH0ig/yy5c2d/q1REQiyRkXunNunXOur3Mu2zmXDewCznHO7e7wdG2Q3D2OWy8Zxj8+rGL51r1exxERCZvW3Lb4BLASGGlmu8zshs6P1T7fuCBb80dFpMtpzV0uc5xz/Z1zsc65Ac65hz7189nOuYhaCmv+qIh0Rb5/UrQlmj8qIl1N1Ba65o+KSFcTtYUOmj8qIl1LVBc6aP6oiHQdUV/omj8qIl1F1Bc6NM0fjQtq/qiIRLcuUeh9ExO4aYrmj4pIdOsShQ4wV/NHRSTKdZlCP3H+6GsbNX9URKJPlyl0+P/zR3/1suaPikj06VKFrvmjIhLNulShQ9P80fM1f1REolCXK3Qz4yeaPyoiUajLFTpo/qiIRKcuWeig+aMiEn26bKFr/qiIRJsuW+ig+aMiEl26dKFr/qiIRJMuXeig+aMiEj26fKFr/qiIRIsuX+ig+aMiEh1U6Gj+qIhEBxV6M80fFRG/U6Gf4J/zR+/5u+aPioj/qNBPcOL80dJqzR8VEX85baGb2cNmtsfM1p/w2v+Y2WYzW2tmz5tZcufGDJ8fTBtBbEyAX7+iIwFExF9as0J/BJj+qddeB8Y6584GPgR+3MG5PKP5oyLiV6ctdOfc28C+T732mnOuofnbVcCATsjmGc0fFRE/6og99G8BL7f0k2Y218wKzKygqqqqAy7X+TR/VET8qF2FbmZ3Ag3Awpbe45y73zmX65zLTUtLa8/lwkrzR0XEb9pc6GZ2HXAl8HUXhfsSmj8qIn7TpkI3s+nA/wVmOueOdGykyKH5oyLiJ625bfEJYCUw0sx2mdkNwL1AL+B1Mysysz92ck5PaP6oiPhJ8HRvcM7NOcnLD3VCloh04vzRa/KySE9K8DqSiMhJ6UnRVtD8URHxAxV6K2j+qIj4gQq9lTR/VEQinQq9lTR/VEQinQr9DGj+qIhEMhX6GdD8URGJZCr0M6T5oyISqVToZ0jzR0UkUqnQ20DzR0UkEqnQ20jzR0Uk0qjQ20jzR0Uk0qjQ20HzR0UkkqjQ20HzR0UkkqjQ20nzR0UkUqjQ20nzR0UkUqjQO4Dmj4pIJFChdwDNHxWRSKBC7yCaPyoiXlOhd5AT54/+x5INzF9aTGGp7nwRkfA57UxRab3xA5PJH5bKs6vLCBjEBQMsvDGPiVkpXkcTkS5AK/QONiYjEYCQg7qGEKtKqj1OJCJdhQq9g102Jp24YNNva8ihe9NFJGy05dLBJmal8MRNeby5qZI3NlVy12sfUt/omDd1OIGAeR1PRKKYVuidYGJWCrdPH8XfbpvM1ecM4HdvbuWWx1dzpK7B62giEsVOW+hm9rCZ7TGz9Se81tvMXjezrc1f9anfScQHY7jrS2fzr1eM5tUNu7l6wUp27T/idSwRiVKtWaE/Akz/1Gt3AG8654YDbzZ/LydhZtw4ZQgPf/Ncdu0/wqx7V/D+9n1exxKRKHTaQnfOvQ18uoFmAY82//hRYHYH54o6F4/sy6Jb8knsFsvXHlilJ0pFpMO1dQ+9n3OuAqD5a9+OixS9hqb1ZNHN+eQN6cMdz63jP5ZsoEFnv4hIB+n0D0XNbK6ZFZhZQVVVVWdfLuIldY/lT988l2/lD+aRd7Zz/SPvc/CIjgoQkfZra6FXmll/gOave1p6o3PufudcrnMuNy0trY2Xiy7BmAA/veosfn312awqqWbW/OUU7znsdSwR8bm2FvoS4LrmH18HLO6YOF3Ll88dyOM35XHoaANfmL+CpVta/HNRROS0WnPb4hPASmCkme0ysxuA/wYuNbOtwKXN30sbnJvdmyW3TWZA7+7c8Mj7PPB2iZ4uFZE2sXCWR25urisoKAjb9fzkSF0D/+epNby8fjdXnzOAn39hLAmxMV7HEpEIYGaFzrnc071PT4pGiO5xQeZ/7Ry+P204z67exZwHVrGn5qjXsUTER1ToESQQML4/bQQLvn4OmysOMfPeFazbddDrWCLiEyr0CHT5uP48890LiAkYX7rvHf62ptzrSCLiAyr0CDUmI4nFt+YzNiOJ2574gLte3UIopA9LRaRlKvQIltoznoU3nc+Xcwdw79JivvNYIbXHdGKjiJycCj3CxQdj+NXVZ/PTK8/ijU2VXL3gHXbu04mNIvJZKnQfMDO+NXkwj1x/HuUHPmbW/BUabScin6FC95ELR6Sx6JZ8krvHcs2D77Lw3VKvI4lIBFGh+8yQtJ4suiWf/GGp3Pn8en66eD31OrFRRFCh+1JiQiwPf/NcbpoymD+vLOW6h99jf22d17FExGMqdJ+KCRh3XnEWd31pPAXb9zP7DyvYWnnI61gi4iEVus/9y8QBPDE3j9pjjXzhD+/w5qZKryOJiEdU6FFgYlYKS27NJzu1Ozf+uYAFb23TiY0iXZAKPUpkJHfj6W9PYsa4/vzqlc384K9FHK1v9DqWiIRR0OsA0nG6xcVw75wJjOrXi/99/UM+2lvL/d/IpV9igtfRRCQMtEKPMmbGbVOH88drJrJ1z2Fm3rucNTsPeB1LRMJAhR6lpo9N59nvTiIYCPCl+1ayuKjM60gi0slU6FFsdP9EltyaT87AZOY9WcSvXtmsExtFopgKPcr16RnPYzecz5zzBrHgrW3M/UsBh47Wex1LRDqBCr0LiAsG+MUXxvKzmWNYuqWKqxe8w45qndgoEm1U6F2EmXHdpGz+/K3zqKw5xsz5y3ln216vY4lIB1KhdzH5w1JZfEs+qT3jufah9/j5ixuZv7SYwtL9XkcTkXbSfehdUHZqD567eRLffPg9Hlj2EQAJwQALb8pjYlaKx+lEpK20Qu+iEhNiuWRUX6z5+6MNIV5cV+FpJhFpHxV6F3bB0FTiYwMEmlv9sZWlPFWwU+fAiPhUu7ZczOwHwI2AA9YB1zvnjnZEMOl8E7NSWHhjHqtKqhnRtxcPrSjh9mfWsqJ4Lz//wjh6xmtHTsRP2vx/rJllAt8DznLOfWxmTwFfBR7poGwSBhOzUo7vm18yui/zlxZz9xsfsmbnAe6Zcw7jBiR5nFBEWqu9Wy5BoJuZBYHuQHn7I4lXYgLG96YO58m5F3CsIcQXF6zgoeUfaQtGxCfaXOjOuTLgLmAHUAEcdM691lHBxDvnDe7NS9+bwkUj+vKfL2zkxkcL2KcRdyIRr82FbmYpwCxgMJAB9DCza07yvrlmVmBmBVVVVW1PKmGV0iOOB74xkX+/6iyWbd3LjN8tY1VJtdexROQU2rPlMg34yDlX5ZyrB54DJn36Tc65+51zuc653LS0tHZcTsLNzLg+fzDP3TyJbnExfO2BVdz9xoc06oAvkYjUnkLfAeSZWXczM2AqsKljYkkkGZuZxN9um8ysnEzufmMrX3tgFbsP6mYmkUjTnj30d4FngNU03bIYAO7voFwSYXrGB/ntV3K460vjWbvrIJf/7m3+vlkDqUUiiYXzDobc3FxXUFAQtutJ59hWdZhbH/+ATRU13DB5MLdPH0l8MMbrWCJRy8wKnXO5p3ufnhSVMzY0rSfP3zyJ6y7I4qHlH/EvC1ayfW+t17FEujwVurRJQmwMP5s1lvuunciOfUe44vfLNOZOxGMqdGmXy8ak89K8KYzun8i8J4u4/Zk1HKlr8DqWSJekQpd2y0zuxpNz87j1c8N4unAXV92znE0VNV7HEulyVOjSIYIxAX502Ugeu+F8ao42MGv+Cv6yqlTHBoiEkQpdOlT+sFRenjeFvCF9+LdF67l54WoOHtFQapFwUKFLh0vtGc8j3zyXH18+itc3VjLj98s04k4kDFTo0ikCAePbFw3l6e9cgBl8+b6V/OGtYkI6NkCk06jQpVNNGJTCS/OmMH1MOr9+ZQvX/ek99hzSsQEinUGFLp0uMSGWe782gV9+cRzvfbSPGb9bxtsf6uRNkY6mQpewMDPmnDeIJbdOJqV7HN94+D1+9cpm6htDXkcTiRoqdAmrkem9WHLrZOacN5AFb23jy/etZOe+I17HEokKKnQJu25xMfzyi2dzz5wJFFceZsbvl/HyugqvY4n4ngpdPHPV+Axe/N4UhqT24LsLV3Pn8+s4Wt/odSwR31Khi6cG9enO09+ZxNwLh7Dw3R3Mnr+C4j2HvI4l4ksqdPFcXDDAT2aM5k/Xn0vVoWNcdc8Knnp/p44NEDlDKnSJGJ8b2ZeX5k1hwqBkbn92LfOeLOLQUR0bINJaKnSJKP0SE/jLDefzo8+P4IW15Vx5z3LW7jrgdSwRX1ChS8SJCRi3XjKcv377AuoaQly94B0eXFaiLRiR01ChS8Q6N7s3L8+bwsUj+/JfL27ihkcLWLplD/OXFuuwL5GTCHodQORUkrvHcf+1E/nzylL+84WN/H3zHoymD1IfvymPiVkpXkcUiRhaoUvEMzOum5TN188fBIADjjWE+OFfi3iqYCc1+uBUBFChi4/MzMkkITZAwCAYMI41NHL7M2vJ/a83+O5jhbyyvkIPJkmXpi0X8Y2JWSksvDGPVSXV5A3pwzmDkinaeYDFReW8sLacl9fvpldCkBlj+zMrJ4Pzh/QhJmBexxYJGwvnnQO5ubmuoKAgbNeTrqOhMcSKbdUs/qCMVzfspraukX6J8cwcn8GsnEzGZCRipnIXfzKzQudc7mnf155CN7Nk4EFgLE1bm99yzq1s6f0qdAmHj+saeWNTJYuLynhrSxUNIcfQtB7MzslkZk4GWX16eB1R5IyEq9AfBZY55x40szigu3OuxadAVOgSbvtr63hpfQWLPyjnve37AJgwKJlZ4zO4cnwGqT3jPU4ocnqdXuhmlgisAYa4Vv5LVOjipbIDH7OkqJzFRWVs3n2ImICRPyyV2TkZfH5MOj3j9ZGSRKZwFHoOcD+wERgPFALznHO1n3rfXGAuwKBBgyaWlpa26XoiHWnL7kMsLipjcVE5ZQc+JiE2wLTR/Zidk8mFI9KIC+oGMIkc4Sj0XGAVkO+ce9fMfgfUOOf+raVfoxW6RJpQyFG4Yz+Li8p4cW0F+4/Uk9w9lhnj+jM7J5PcrBQCulNGPBaOQk8HVjnnspu/nwLc4Zy7oqVfo0KXSFbXEGLZ1ioWF5Xz2sbdHK0PkZncjavGZzB7Qgaj0hO9jihdVGsLvc2bhs653Wa208xGOue2AFNp2n4R8aW4YICpo/sxdXQ/ao818PrGShYVlfHAshL++I9tjErvxcycDGaOz2BASnev44p8Rnvvcsmh6bbFOKAEuN451+KpSVqhix9VHz7Gi+sqWPRBGat3NN3EdW52CrNyMrliXH9SesR5nFCiXVhuWzxTKnTxux3VR1iypoxFReUU7zlMMGBcNCKNWRMyuXR0P7rFxXgdUaKQCl2kEznn2FhRw+KicpYUlbO75ijd42K4bEw6s3IymDwslTW7Dh4/pkCnQkp7qNBFwqQx5Hjvo30sLirjpXUV1BxtIDEhSG1dI845YmN01K+0jwpdxAPHGhp5a0sVv3ntQ7ZUHjr+elK3IBcMSWVsZiJjMpIYk5FI38QED5OKn3T6XS4i8lnxwaZtl9Se8Xz9gVXUNYYImDEmI4ktlYd4ZcPu4+9N6xXPmIxExjYX/NjMJAakdNMhYtJmKnSRTjAxK4WFN+V9Zg/90NF6NlUcYn3ZQTaU17Ch/CDLtu6lMdT0N+XEhODxFfzYzKavQ9J66hhgaRVtuYh47Gh9I1t2H2JDeQ3ry5uKfnNFDccaQgB0i41hVP9eJ6zmkxiR3pP4oO6o6Sq0hy7iYw2NIbZV1R5fya8vP8im8hoOHWsAmiY2De/Xi7EZicdX86P7J9JDB4xFJRW6SJQJhRw79h35xEp+Q9lBqmvrADCDwak9GJOR1Fz0TVs2evDJ//ShqEiUCQSM7NQeZKf24Iqz+wNN98NX1hz7xEp+del+/ram/Pivy0zuxlmf+vC1X2K8PnyNQip0ER8zM9KTEkhPSmDaWf2Ov76/tu74h67rm7++samSf/6FvE+POMY0f+j6z6Lfe/gY7360Tw9C+ZgKXSQKpfSIY/LwVCYPTz3+Wu2xBjZV1Jywmq/hgbdLaAh9cts1GDB+8+XxXDU+Q6t4n9EeukgXdqyhkQ93H+bepVt5dUPlJ34uPTGB/GGpTBmeSv6wVNJ6aVyfV7SHLiKnFR+MYdyAJOZeOJR/fFhFfUOIYEyA6ydls2P/Ed7YVMmzq3cBMCq9F5OHNa36zx/cRweRRSCt0EUEgMLS/Z95EKox5I4//LSieC8F2/dT1xgiLibAOVnJTBmexuRhqYzNTNLDT51Ity2KSIf7uK6R97bvY0XxXpZt3cumihoAkrrFMmloHyYPT2XKsDQG9dEAkI6kLRcR6XDd4mK4aEQaF41IA6Dq0DHe2baX5Vv3srx4Ly+vbzqrZmDvbkwelsaU4alMGtqH5O66Fz4ctEIXkQ7hnGNbVe3x1fuqkmoOH2vADMZlJjXtvw9LZWJ2io4tOEPachERT9U3hli768Dx/fcPdhygIeRIiA1w3uA+TBnWdPfMqPReBLT/fkoqdBGJKIeO1vNuyT6WFzdtzxTvOQxAas848pvLfcrwVPondfM4aeTRHrqIRJReCbFMO6vf8SdaKw5+zPLm1fvy4moWFzUdVzA0rUfz7ZFp5A3pTa+EWC9j+4pW6CLiOeccm3cfOr7//u5H1RytDxETMCYMTD6+eh8/MJnYmIDXccNOWy4i4lvHGhopLN3ftHrfupe1ZQdxDnrGB8kb0pus3t0JBIzpY/t3iXNnVOgiEjUOHKlj5bZqlhXv5c2NlVQeOgY0HRl8/aRsvnvxsKg+mkCFLiJRaf7SYv73tS2ceKZYTMDIH5bK7JwMPj8mnZ5RNugjbB+KmlkMUACUOeeubO+/T0TkVPKG9CEuGKC+IURsMMB/f/Fstu45xOKicn741BoSYtcxbXQ/ZudkcuGINOKCXWfPvd0rdDP7IZALJJ6u0LVCF5GOcLJzZ5xzFJbuZ1FRGS+urWD/kXqSu8cyY1x/ZudkkpuV4tv73cOy5WJmA4BHgZ8DP1Shi0gkqG8MsWxrFYs+KOf1jZV8XN9IZnI3ZuZkMDsnk5HpvbyOeEbCteVyN3A74K/fHRGJarExAS4Z1Y9LRvWj9lgDr2+sZFFRGfe/XcKCt7YxKr0Xs3IymZmTQWZy9DzI1OYVupldCcxwzt1sZhcDPzrZCt3M5gJzAQYNGjSxtLS0HXFFRNpu7+FjvLi2gsVFZazecQCA8wb3ZlZOBleM6x+xh4h1+paLmf0SuBZoABKAROA559w1Lf0abbmISKTYUX2ExUVlLCoqY1tVLbExxkUj+jJ7QgZTR/WLqAEeYb1t8VQr9BOp0EUk0jjn2FBew+KiMpasKaey5hg94mK4bGw6s3MymTS0D0GPn07VWS4iIq1gZozNTGJsZhJ3XD6ad0uqWVRUxsvrd/Pc6jJSe8Zz1fj+zMrJZPyApIgenK0Hi0RETuJofSNvbdnDog/K+fvmPdQ1hhic2oOZ4zOYPSGTwak9wpZFT4qKiHSQgx/X8+r63SwqKmNlSTXOwdkDkpiVk8lV4/vTt1dCp15fhS4i0gl2HzzK39aUs6iojA3lNQQM8oelMnN8BtPHpnfKcb8qdBGRTlbcfOTA4qJyduw7QnwwwLTR/ZiZk8HFI9M6bNSeCl1EJEycc3yw8wCLPyjjhbUVVNfWkdQtlhnj0pmVk0kwYLz70b5PHFVwJlToIiIeqG8Msbx4L0uKynl1w26O1DUCEDCICwZYeGPeGZd6awu96xxDJiISBrExAT43si+//UoOBf86jSvP7g9AyEF9Q4hVJdWddm0VuohIJ+keF+T6/MEkxAaIMYgNBsgb0qfTrqcHi0REOtHErBQW3pj3meN+O4MKXUSkk03MSgnL7FNtuYiIRAkVuohIlFChi4hECRW6iEiUUKGLiEQJFbqISJQI66P/ZlYFtHWoaCqwtwPjdDY/5fVTVvBXXj9lBX/l9VNWaF/eLOdc2uneFNZCbw8zK2jNWQaRwk95/ZQV/JXXT1nBX3n9lBXCk1dbLiIiUUKFLiISJfxU6Pd7HeAM+Smvn7KCv/L6KSv4K6+fskIY8vpmD11ERE7NTyt0ERE5BV8UuplNN7MtZlZsZnd4nedUzOxhM9tjZuu9znI6ZjbQzJaa2SYz22Bm87zO1BIzSzCz98xsTXPWn3md6XTMLMbMPjCzF7zOcjpmtt3M1plZkZlF/FgxM0s2s2fMbHPzf78XeJ3pZMxsZPPv6T//qTGz73fa9SJ9y8XMYoAPgUuBXcD7wBzn3EZPg7XAzC4EDgN/ds6N9TrPqZhZf6C/c261mfUCCoHZkfh7a2YG9HDOHTazWGA5MM85t8rjaC0ysx8CuUCic+5Kr/OcipltB3Kdc764r9vMHgWWOeceNLM4oLtz7oDXuU6lucvKgPOdc219HueU/LBCPw8ods6VOOfqgCeBWR5napFz7m1gn9c5WsM5V+GcW93840PAJiDT21Qn55ocbv42tvmfiF2NmNkA4ArgQa+zRBszSwQuBB4CcM7VRXqZN5sKbOusMgd/FHomsPOE73cRoaXjZ2aWDUwA3vU2ScuatzCKgD3A6865iM0K3A3cDoS8DtJKDnjNzArNbK7XYU5jCFAF/Kl5S+tBM+vhdahW+CrwRGdewA+Fbid5LWJXZn5kZj2BZ4HvO+dqvM7TEudco3MuBxgAnGdmEbmlZWZXAnucc4VeZzkD+c65c4DLgVuatw4jVRA4B1jgnJsA1AKR/tlaHDATeLozr+OHQt8FDDzh+wFAuUdZok7zfvSzwELn3HNe52mN5r9evwVM9zhKS/KBmc370k8Cl5jZY95GOjXnXHnz1z3A8zRtdUaqXcCuE/6G9gxNBR/JLgdWO+cqO/Mifij094HhZja4+U+5rwJLPM4UFZo/aHwI2OSc+43XeU7FzNLMLLn5x92AacBmb1OdnHPux865Ac65bJr+e/27c+4aj2O1yMx6NH8oTvPWxeeBiL1Lyzm3G9hpZiObX5oKRNwH+Z8yh07ebgEfDIl2zjWY2a3Aq0AM8LBzboPHsVpkZk8AFwOpZrYL+Hfn3EPepmpRPnAtsK55bxrgJ865lzzM1JL+wKPNdwoEgKeccxF/O6BP9AOeb/rznSDwuHPuFW8jndZtwMLmRV4JcL3HeVpkZt1pukvv251+rUi/bVFERFrHD1suIiLSCip0EZEooUIXEYkSKnQRkSihQhcRiRIqdBGRKKFCFxGJEip0EZEo8f8AHx3kRW/NKlwAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.plot(s, '.-');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Display the features." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAADuCAYAAAAOR30qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsfXl8FOX9//uZ2d1kc5/kIBfhiELwICjghVqKpUVRPKhg1XoAv59HPdqvNz++aK1+rS1tpfXg21qtImhAqtUW8QCtnEkREiEcgZyQkGSTbLJJdnfm+f0xO7MzszOzO7ubgzbv18uXTHaO93N9ns/zeT7P50MopRjFKEYxilEMP5jhJjCKUYxiFKMQMCqQRzGKUYxihGBUII9iFKMYxQjBqEAexShGMYoRglGBPIpRjGIUIwSjAnkUoxjFKEYIRgXyKEYxilGMEIwK5FGMYhSjGCEYFcijGMUoRjFCYDFzc0ZGBi0qKhokKuZQUVHRRinN1PptMHi63Bxq23pAKUAIUJyRgDgbG9KzoXCN5P3RhBHXlLR02h+bNuwcRQx1HzADeXt6O1vA9XUTrfuGmmewfhatOh2K/hzN9h9svkZc5TAlkIuKirB3797wWYnvefRvAX878dwPTL2DEFKnul4KYCkAFBQURMSzos6BnbXtmFmcjrLCVADAms+P4sUtNeCpsKwom5iBB+ZMkn6PlKv8/SwB7plbgnuumBAxd62yhMs1few4ZN+yWvg7gHuuCo9jtGDENSe/CD9+4Z2Qyx1trPn8KH75jxpQACdff0Dxm7r9Xyn/xFQbRcrLqB8H66uhcpWXn8Hg9JVocRX5Go0/s+MoGFc9mBLIIxmU0lcBvAoA06dPNx2gQ6zw1DgbVn1YDbeXh83C4K27ZqKsMBWpcTYwhIBSCh7AP4+2Yc+JDun3SLnOLE6HzcLA4+VhtTCYWZxutgioqHNgydqdEvcV86dg5V+r4OYoWAI8fe1ULJ5REDbXvImlUr1SAM4+T1A+QyVoACXX2NyJ9MUtNYo2HEqkxtkgVZZKN5bznHzO+VTeZoPJtaLOgebOPlgYAi8n9OOvjrRhx7F2rFpQqtk3wuUqLz/vux5shMpVq19qjb9gMmEwMKgCWUsTHomQCzKGEHA8BQUw4OGxsbIRNaeceGpzFTieSmOLp4DHy2Nnbbtm48gbPRSUFabirbtmRiTAdta2w+3lwVOB+x+/qoWbE4YFR4En3z+AquYuXD8tT/H+ULn2DHiRIrveUduue696cjDTidV8wqkTSoU2chu00WDi85rWkO7rGfDC4mszo/4UDsR6dPZ5sKO2Hd+e7AbHU1hYBlPzkrC/sQsUgJeneOL9A1Hlqi7/5zWtppUBLby9qx4fV53EvNKcsLjq9Uv1+AOgkAleXhhHAx5lubWEu1yQO1xuU8rVv42GHAnkggygIEQY0BTAut310r+FX/0gDFHMpHIzgbzRic0eHwoPsWOEAq1vNnX2gWUIeE6YUI6e7lU8w1Ng3a56bKxsxFt3zQQAvLLtGD491ApKaVCuHK9ceHSpNGQ5J3mdigMCMBauFXUOvLztGD471Aqep7CyBCAEXs4/eOTvCAU8HRrtTI09xztCui8hxgKPr80YX38yCz2hsGTtTgx4eKiXi16OR+nYZBxo6oIY7JH6Jmyj9k+IsYAPcRW3r95heB0O3t5Vj8c3CRPHl0fawMSlZITKNTXOhic2HUBVU5eiX26sbAwQnmWFqVjz+VHpPl4WEZMCONLixJrPj2pqzgAU9c4QmJIBowIZyuUKwxB4OH8D8OreLAPlKWpOOQMaRS2MGFtcYjT5apkmRA6aO0dyzhA4lVc24r2KRri9vPSbOwhXThWqtVsmkLU42VQDwkhjrqhz4OZXd0gaPQBfO1CJ88bKRpRXNpqe6KKlnZlBd7+xOUdEv4cD7yuzhxP6k5lJWWxH+aRVVpgq9UGt7stTIDEmcOjzFGDtSbpSNs7G4oZZRfh79Sl8b0q2Ic+OXrfhdTj4uOqk4pqJjdclIOd6Xn4KVn5QrejrDAFYlsG7exvg5Sl4KliWYqxCHcplAg9A3vU372sGIVCspuVKh7zezcqAESOQo7HRFy7E5crGyka8s7s+5Od4KnQSuZlgY2UjFk7LUwgj3u1yRpOv2jSxfk+9dB1MIAPCLjKB0FHkYAgx5MoS5duT7FZNTh4vD4fLrVgCamnM8gG9s7ZdMRECkDQMAmHwVDV1SZqH2xN6Jz/Y3BXKbVEFwwA8F/y+rj4P5IX4uOpkwOQRqgYsr9eZxemwsIxCCEncAFSf7DZdpo5eN17eXgsAeHl7LQrS43UnOrUiY6TYhIp5pTn48kib/539vbpqt5zriXZXwO8XT8hAfloc3tldL3GTC9Z7rpgg9d9tNa3YfcL/KQrfCppSsIywryRfMdgsDNweQZAzBKZkwIgRyMONssJUlFc2gjPRcSiAKTlJ2HW8Q5oV393bgIXT8hTCaPozfb3B3mUG8sFGAVQ1d4EgNGEMAF5e0JCsFv+AZQmwakEplvxCnyvLKL+QLBPIWpsiahOMxbf6YDWW5jOL02FliUJDBoSBzBCA53l80+gXrDwAynPeUMrb6QpNW40m5k/Nxfv7moPeZ7cqXaum5CQprvVsnjtr2wPMESzrFwo1p5zwaghjAsBmZTCvNAd7TnQo3mFjCbi+bt2NgY5et2IPYf2eel2BnJFgw+ket+I6UojfEm3IS57vbNO7V81VDgsDPDBnEgDgvYpGeHzjiAEUglXsv6lxNoVAFkEB3HXJOCTarYrJUhz7cjNIqDLgP1YgV9Q58PzHB1Hf4cK1543Fd6dk472KxpCeJRAag0DQNC6flIlPvm0BhWBnFWfYwdpIKitMxQ1leVi3q973TfPvqD7ZjZVXT8H6PfUYkxSL5bPHo6wwFUsMnomxKM8RjcvwWwxC2pQkQs1xFFj1QTUWXVCAkuxE6Zl1S2ehvLIR1U1dSuFLlUtGwKc5M2xI/dcdTgVFiAvHpYckkBnVJJcom+Qq6hxYvfWwf1Ug04CdfZ4Ac8R5eclYvfUwpuQk4bUva6EutY0luHF6Phb6NnXFulcIjmf1BYeVVbb/mKRY3XL1ezjD63CxeEaBJJiN+qqaqxzTClKlcnO839b7nbOzMCYxJuB+vQ1acfyr3V/N7AWp8R8pkCvqHLjplR3SJtXL22tRWe8IWMJrwWZhAEolu9OXR9rAEoBlCShPw3ZZM4vrp+VhY2Wj5qaNFhiiXDamx8s2JFqcWD57fNB39Lo52GXXe0+EtnEFCCYJL8dLk9Y3jV34pvEArKxghxPtzmNT7EiKsSgEMqDaTIXQDqEuA/lhSFO2fk9opq/eAa/CZLHLN5mLmnG/x98nxQ3KijoH1n51XPEehkDS4uTLehHZSTG4/zuTFBqtWcGhnpDHZ+ib8HvcnOG1HIPhHqnmKsfuEw7sOeEAkY0JngKfHmwBAJT7Nr1FLq3d/YrnCQEIRcTur1r4jxTIO2vbAzwGDp4KPrZT4qz44fR8OAe8qJJpcRwFWAosurBAcikz6/ZmFqJG+sh73wR4U4SCD/eflDYk3J7Q3K3Ugs0lG2TB3NxEP271O0S7sdvL46nNVeB5GqA1qpHlEy5LQlwGhjDPRh1ZSbEAgtuuXW6lQN7jm+TKfZOtHAwAh8uNjZWNkhsWIAjj4swEHG3tkf4mruJEtHQPYNWH1SjJTgxbcJhxe1TPgXpzYiTukWa4BvDT4CRWqdur9AqaVZyuUBCWXVqM6pPd+OfRNsXe0ZAIZPXpl5EKMzxnFqdLrm0inP1eMES/4wCCLfLl7bVgCGBhGbAEks2ZUoqxKfaQ3N6iUacVdQ5srGzEifbQhLF6U0U+oI0c9+VcE3KKFb+Vjk2W/m20aVdR58CqD6vB8TRAU2cZAt73B072f7VAkUMULkb1assevhOEALBs9nh8VtMKL+fb/ZFByVO5MuF4ioo6B96raFSUX+xz3zR04rNDLdLfWYbg6QWlACC5hAHAssuK0T3gRXVTl+RvHI6fs7L9lVyNtFCWUZrS9CwIwTZ7zcCIqxnwVDj0JB/Dyy8ThPC80hwsnlGAijqH5t5RpEI5aHAhSumrlNLplNLpmZmZKCoqwtatWyP6aDg4ceIECCHwenX3cR4H8KjI0whlhalYdmlxwN9FQdHfUIWm15aB6ChqPBV8OcdlJsDdcACNv78NNpmpIpjbm7pOzUIU+G/tqg9b+xO9F8R/O1zabklyrqzFqvhN9EMuKioCaT6A7n++jfYPfxlgtpG7YFEKXFiUiksnZmD5ZcVgfST66vajcc1t0jOp8f5vESg3FEXhYlSvJqvDFN566y3MnTvX8J6ywlSsXzoLP7uqBB5Hc40eT6LaiiWECB4nsoa9sCgVU8cm4/hLt+OvH/1DanMCYNEF+ZJd9dnrpuLSiRl49rqpePT7Z+PZ66ZixdVTEGMVlIdwzGlyrrExwqTdXy+0VbKBf3dAuXS2nMXN4FD5VdQ5sObzo6io09hk0+Cqh7R4q4LRhDEJ/vEAwTYsH8OJdivevHOGJIx31rZj9qRM6Rmvb+8oUpg2WTQ6+rBk7S7Ytw4o/j5ULmqhIphdKtFuDdDCRO0tNr8UY+9+xVBb5ilwtLUHDAHirKzkFC46jIseB20frQbn6tTf/QijHKKACxeEANecm4u/V58ydVRbXR9yDpOyErFwWh72VffgD6plp9orZF9DJ9YtnSXYlX1mE4GY/90dvX7PCAqgrCAFKXE2fFbTCo6jYNnQbcjRDmm4ZMkSLFlitKUkQLTR3uvWN62ozTPxNlZ57BpAZb0DHA/FASXRZ/b6aXnSffINLzmHSE+AiuB4ZZ/r0pnEASAp1oIOmXdLUqy2qDHDz8yBKzVXADg7OxEOlxunnQNw9HoUBzfuuHgcVn1YLY0H0QtFPT5EDl213+DU248hadYipF72I/AU2P/ZZrBz/gt2u3+n5cMPP8Tll1+uWyY1/i1tyC43F9QuNbM4HSzjPxJJAMw5OwufHmoNsC+rIRfkHE8l7U19MMLhcuPvRzLx4X4uLK94PfvazOJ0TXusFvJSYpGeEKOwgVEK/L36lMQx1IGqPhjiVB1+yE2xw5UVaKMM9Aqh0gCUXOVYBvE2S4BJQ8SeEw5YfRuqUiFCxDCYkEMGrypsr5tDtcpvWj73ihuaN5TlBRyB10Mku/5y9Ht4yJ3yalr058O0hBiFQE5LCPReMMvPzIErNVcAONbWK7m4AcJEffEEf3AluceP1rXIYWDAjfatr8KWUyK9m4GwHzBr1ix89dVXQcuih7CVB8pz6NqxAU2v3IX6X9+IsrIyNDQ0AAB+8pOfID8/H/W/vhEnX/8J+huqAABeZzvqX1wIrs/fkO6WY2j47WJQzgtKeXR+/Q4a//BjNPxuCW699VZ0dZly6o8lhPR1ufrh9vLo+Oc7qPn5fHx+4AQA4Mknn8QDDwiRt0qz41BS9z6a/vBjNPzuFnRsWYOLCpOwYdksXBLfgsbf+5fPA6eOovlP96P+1zfi9Pu/QOvm5+HY/qZQgYTAZmHw8+f+B0d+dTPqf/cjdFT+Aw6XG9Yjn+Hvm98FgGxCSA8h5AMzhdlZ247WL9ej4eW7cPh/rsf82Rdg06ZNKCtMxcr5Z6Pz8/9Fw28Xo/HlO+Gs+AB1z88H9Z1G4Ad60fbRb7D3Fzfhy1U3wLH9Tek3wH94w4x7nnqvLdYaGJ6wv78fixYtQmJiIqZNm4ZvvvkGgOAV0vjyHejeuQGNa/8PHrt2Ol5a+RD+eMt5eGhuCZ74wWQwBOjatRENv1uCxpd+hJ79n0jvFc0UHA9JqEfrBOTzzz+PsWPHIjExESUlJXjrrbdgt9vR0eH3IvnXv/6FjIwMeDwevP7667jkkkui8ekA8JQG2M5ZIvxHCHDlWWPwlzsuQMLBD3DTldORmJioGHuEEPz2t79FcXExMjIy8LOf/Qy8hrYYCggh0wgh/yKEOAkh7w44Tkr9HgD6fJu6zc3NuP7665GZmYlx48YJ3/d5YAw01+Dkmw/jy6fmIycnB/feey/cbkE/oZTiwQcfxJgxY5CcnIxzzjkHVVWCrOjq6sKtt96KzMxMFBYW4plnnsGFRamwWRi4DmzFyb/8FzhXp50Q4iCEHCeEzJNz72s5jq4dG9C89v+gYfUitP9tNdwD/aAAuna9h8aXfoT6NbeiqG0nphel4ejRoygrTMUPz0nFyntvRVJSEpbfMBcnP30dP1lytfTemcXp6Nn7PuLGnY+YzHypbWxWBuMzE8KqZznCFsjde95H77fbMOaGlch/YANOnns7LvrlVyh69G94q9YKLPwf5P/kHcRPno3Tm58D9bphSUyHLfcsuGr+Kb2n99ttiCu5CIS1oPfAVvQe+BRZN/8CY5etRU9PD+69914ztPoB7CGefuG0TEM1rMljYGs7AgDYvn07Zs+eDQB45JFHwDpPYc27W1G4/DV4utvw8ONPAgAunZiJWAsrnMLhPDi96edImPod5N+/DvFnz4br8A4QIhx0+PHF49B+uhWt7Q4U3/cGMr9/P9q2vIzJ6SyWLl0qLm1PUUoTKKVXa9PWxszidNjTc5G75HlM/Nm7uP+nj+GWW27ByZMn0bP/H0jrqMZ/v/4h/vDuJ8jsUAaHafvbr0EYFr/f/E/c+vw76D9RiZ79WwAIWlY49sSkWKUNec7ZWQH3bN68GTfeeCM6OjqwePFiXHvttfB4PCgrTEVmQgxi63Zg41//hhPHa3H48GF89OYaKcxhV0cb4OlD/v/9M9Ln3Y+OT14G398DliGSjdHK+v8dqskiJ0lfO6upqcFLL72EPXv2wOl04h//+AdmzpyJWbNmoby8XLrv7bffxg033ACr1ar7rnAQo5rUJo5JQGlusuJvV5+bi4fmliAzIQZ3XDIO28r/hHXr1uGjjz5Cd3c3/vjHPyIuLk66f9OmTdi7dy8qKyuxefNm/PGPfzTNixBiA7AJwOsA0gCs4/t7FPfkptjB8zyuvvpqnHvuuWhqasKnn36K1atX4xycEDbyGBYZc+7G9v212LFjBz799FP8/ve/BwBs2bIF27dvx+HDh9HZ2Yn169cjPV3ok/fddx+6urpQW1uLbdu24Y033sD+z97HW3fNxFWl2XCfOgzwnBNABoD/AfC/hPh3fQgBer/9AmNuWoXcZWuB7mY4d6zHwPEKOPdsxo9//r/Ytmc/KnZ9DQCoahIUv3vuuQfx8fE4deoU/vznP+PPf/6zoswZ6EbciS+xYsUKzJ6Yieum5eGhuSV4666ZKMqIlybuSZMm4emnnzba89JE2AK5Z/8WpFz2I1jT80AIgW1MMVi7sEhImHIFWHsSCMMi6cKFgNcDT4dw6CJ+8mz0HtwOQJghew9uR/zkywEIFZh0wbWwpmSDsdmxK/0q/OXtdSj8r7/ikuc/AwCMf/yjYNS2uft68efbp8PS1YD7778fzYcq8XXNSezYtRvbu9Ox90QHXnvtNfz617+GxxoH2OxInnUTuqu34/mPD2LVB9Xo93JgCXBZsgPxVoLUCxaAtViQVnoJJpWeh5nj0rB+2Szkp8WBJyyaxn0fxGLFj2++HinJibC7WoLxlKC3UVFWmIoX/ms5Lju/BP/v6ql47N47MXHiROzevRsbNmzAIz99CPNmlMJtseP+h34KQBC2XK8DfbV7ccO9T+DccVn48EgvkqZfK9X7BUWpWCizPYaKgFgWA0Jnc3t5bN7XhObOPpSVlUmC66GHHkJ/fz927twJQFhqr3jkIfxg1lSkpaXhiSeewLp161BR58DP//YtwLBIu+RmzJ06FsklM0BssSBdzXh6QSkemluCdXfPxLqls6QBQA1ss3LkpcXp/sayLAYGBvDtt9/C4/GgqKgI48ePx+LFi7Fu3ToAQj995513sHjxYtN1Fgzp8crNp1tmFgVssH64/6Rk3gGAtWvX4plnnkFJSQkIITj33HMlQQYIykZaWhoKCgrwwAMPSOUwiZkQTJq/pZR6KKUbLTF2xQ0TsxKxZ88enD59GitWrIDNZkNxcTHuvvtubPlgIwghiMmeAHve2bBYLCgqKsKyZcuwbds2AIDVaoXT6cShQ4dAKcXZZ5+NnJwccByH9evX4xe/+AUSExNRVFSEhx9+GG+++SbKClPxnbOzUFRYCABtlFIOwJ8B5ACQNAQCgsRp82FJygRrT8TFNyyFvWEXirr34/vX34zzzpmKui4Oh3KEzdmHNuzD7to2lJeX47//+78RFxeHyZMn47bbblOU+f7778f//OLneOj75yI9wYac5FhplXnZZZehqqoKra2tKC8vx7p16/DCCy+YqvSwbchcdxssKdmav3Xv3gjnN1vA9XQAhIAOuMC5hLPz8SUXw7H1FXid7fA6mgEQxORNAQB4nR1gk8f4ySWPAXgOnP6RdS1s6+npeZK0H8eMsvNwx6IF+OEtt+EvjSkgSdnYdNCJzfv+AZfLhbKyMnA8hcvNCZ5JlMfuEw5B8FBh59Qy0IWS4kL8YflFkj3pl0cnYHxRGsoKU/HGxm4w9kRQwoLjeOFgQ0I8enp6gjOFsb27os6Bh5/9LTp2bsTbXa1YZmPR5+pFW1sbmpub0R+TKj2LLuEwQIyVQU9LK8BxeO+hH2A9T9Hv5QHKw5IoeHRU1ndir89tzozfp2uAg1zX/FedAxV1DpzuGRCOoDY14rx0v9bMMAzy8vLQ3Ow/sZafny/9u7CwEM3Nzb44FjyY2ETwYHBufgqWzR6PuWsT8MicIs2NKjM42qqvSE+YMAGrV6/GypUrUV1djauuugq/+tWvcMMNN+C+++5Dc3Mzjhw5AkIILr30UlPfDQXtPW7IS/PmjhN45rqpQsxin32Zp8od/IaGBowfr+/WpVXHYSAXQBOVu+wxSnHR2OFCXd0pNDc3IyXF7/XLcRyyJp0H73gKT0cTWj9bi9m/rwXDueH1elFWVgYAuPLKK3HvvffinnvuQX19Pa677jr88pe/RF9fH9xuNwoFoSuVo6mpSbrOzs7GsWPHAACUUpdPOZZsBhQUbJLfg6nHkoL21lMoKHJgV2sWqrbUCPHN44SJzMvx+PRfR+D1ehX1J//3Bx98AKfTiUWLFmlWWHGx33Nr6tSpWLFiBV544QU89thj+rWsQtgaMpuUAW/nqYC/9zdUoWtXOTKvfRT5D7yDggfWg4nxayhMbAJii86H69BX6P12G+InXwZxpWFJTAPX5T+m6O0+DTAsWP2gTlr4ur+/H5s2bcLs2bMxefJk1Nc3wHlkD2ILBH9N3pYAa0wsqqur4ezuwiPrdqDgwQ0oeOg9xYsYQjCrdAKampowrSBFmglFex0ATM5JEtyydNx2iJ7vnA89A17NMJUA8NHX+3Hqb79B6neXo+Anb+O5zRUoLS0FpRQ5OTl4b9s+9HuEZ/scQr29cccM3HP1TMTExMDR0Y4vq+sw4eENKHjwXeTe9XuwjHAyTut7waDeuXa43MLzVNiI43iKY8f9iRF4nkdjYyNyc3Olv8nrrr6+Hrm5ub44Fgwgq8OywlQkxlowKStyM3G/x9iGunjxYnz11Veoq6sDIQSPPPIIUlJSMHfuXGzYsAFvv/02br755qBtGQ7Ux7obHC6UFaZi1YJSWBgieQHI+1V+fr4kjLSgVcdh4CSAsXIzAM8pN3EdLjfy8/Mxbtw4dHZ2Sv85nU7c9KRglujYsgbWtDwsW/Mhuru78eyzzyrcsu+//35UVFSguroahw8fxgsvvICMjAxYrVbU1fn7Un19PcaOHWuqAFz3aenfbaeakJubC96egoGu00JYTZ6CdwqKjIVl8J3zJ8JisaCx0R9CQV6Xn376Kfbu3Yvs7GxkZ2dj/fr1WL16NRYsWKD5feJLaGEGYQvkhHPmovPLv8DT0QRKKdytx8H1dYO6+0AYFqw9GeA5dP5zHXh3n+LZ+Mmz0Vv9GVyHv0b85NnS3+POno3uve/D03kKvLsPndv+jPizLgVhQs9tRSl1xcXFYc2aNZK9ePqMGejZ9zFi8qcCAGxWC66/+VY8+OCDaG1tRVKMBV5nG/pqK/wvIkLgkL7UYnAgeOmll+D1erF582bs3r1buq0kOxFp8TZpGa3W3rKysgBA14iZEGPR9cOMIR4ABKw92edW87606THugjnY9cFb8DrbwPf3oGunfzJJTh+DGZdegYcffhgTU1m8ddcMzCskuCShFU8vKA3bL9Wi8u7PTooVnie+cIYMQcORamzcuBFerxerV69GTEwMZs6cKT2zZs0aNDY2oqOjA88++ywWLVqEssJUPPGDyYizsmGZUoIhL8Wu+1tNTQ0+++wzDAwMIDY2Fna7HSwr9LfFixfjjTfeQHl5+aCYKwAIMZ9lyPHFh1g8owDrl83Cwxr96q677sJTTz2FI0eOgFKK/fv3o73dP7G+8MILcDgcaGhowG9+8xtdjS4IdgDgANxLCLEQQhbwbuUR4uykWFx44YVISkrC888/j76+PnAch6qqKpzFtsLGEvDuPlhi43DzRZNw6NAh/OEPf5Ce37NnD3bt2gWPx4P4+HjExsaCZVmwLIubbroJTzzxBJxOJ+rq6vCrX/0Kt9xyi6kCOCv/Bm93G7g+J05/tR6LFi3C4h8uQs+BreDaG2CBGwX1HwMAfnXTebiwOAMLFy7EypUr4XK5cOjQIbzxxhvS+55++mkcPnwY+/btw759+3DNNdfg7rvvxp/+9CcAwMcff4yWFsFUeejQITz99NO6wloPYZsski64FpTzoGXDCvB93bCm5SHzuicQO24aYseVoem1pWCssUi8YAHYRGUcafuEGWj/++9gScyEbYxfzU8457vgejrQ8vajoF437OOmIfW7y0OllABgDSHEmZSUBLfbjQsvvBAAcO287+LTj/6KxQuuQkJKOhZOy8OUrCuwatUqzJw5E40nW0Dj0pB4/vdhLxaWU7EWFq/vOAG3l0fs936Gl/7wKh577DHMmzcP8+fPR0yMX8ZaWUY3X9idd96J559/3k4I6QTwBaX0WvVJvY06fpjxWUVIvvA6nPrLTwHC4MgPrsfFF18MABiYcDlix+3ByT/eBxITh7OuvBH/tLICAAAgAElEQVSHGqtw65/2CBHVzrsDmSc/wuTJk+F0OlFcXIxHHnkEP5xRoOnOowc5V7vqVFlnn3+z7vqyPAxkdaNjzAKsX78et912GyZMmICNGzcqNsIWL16MuXPnorm5GQsWLMCTTz4p/dbn4fDObmUAfTPQO6nnHNDfWBkYGMCjjz6KgwcPwmq14qKLLsKrr74KALjmmmtw1113oaCgAOeee65pPqHwjFXVqZyrnjvYQw89hIGBAcydOxdtbW0466yzsGnTJun3BQsWoKysDF1dXbj99ttx5513muZIKXUTQhZC2NhbDaCLiYkHkcVzOtXdD5Zl8cEHH+Dhhx/GuHHjMDAwgJKSEjzzzDNYt3QW/pK5Ept+txKXlxbg/PPPx6JFi/DZZ8J+UHd3Nx588EHU1tYiNjYWV111FX76U2Ev5He/+x3uu+8+FBcXIzY2FnfffTfuuOOOYLQXEUKuAwAwFsRPno3WDU/B29OB9MkX4cknn0RcXBwOf1uFN9Y+hX6rBQ+uXIGvPt6IsmLBVPrSSy/h9ttvR3Z2NkpKSnDzzTdL+TkTExORmOhfsdntdsTHxyMtLQ2AoEHffvvt6OnpQVZWFm655RY8/vjjpuqdmFGpp0+fTtvm/LepDwwGTjz3AxBCKvROZE2fPp2aSXIqz0QACFrLjdPzpVipLAEekiU9nDFjBpYvX44f//jHIb0/XK6iH7LonC7XlNScb8pqw9rnHkfKHa9qcg4VRlzzJpZSy/XPS9fLLyvGo98/O+R3FxUVYe3atZgzZ07Ab+okk6FwN+IakzOR5twmJGS9sCgVG5ZfFDLPaMOIZ1rhWTTp5hel60i5EkJw5MgRTJgQ3tFxI66WGDtN+c4yJJzz3ahwjRRGXFmLlWbcsBL2ovMA6HM9ePAgSktLMTAwAIslUD995JFHJI+LweKquC+YQJbP5gBKAIjHQDMA6MYjDYJoPFuol1abEHIaQJ3qfkMwcSkZrD0xg/Kch+t1nAIAa2ruJAAM7+7jvT0dx+Dp7waQDqAQwAEAoQTazQAQHyJX9XNtxGaPZ2xxibzb5VR5FRBii8tj4lNi+b6ebtrfnQqGddsyi5IhBKOC19FcE6onggyG9comZXoZW1wG73a1cd2nm7Tu0ysLgKkATgAI2GUjNnu8r74JAOrpaGqlnv5g79flytqTqCUlC6CUevTrIZx+GM4zhQCe8CXhVICwlk5bZlGSaHA04Brqt8sAVAEYCHKf3rulOiWEzIYw3tsgRLt83ZpRSInFGoxrJOPbzLuM+ipvSckBExuvxTUFQuQnBoJMGwAgGuVjIfTBPgBxACZCGJ+dEfLX5aoApTSs/wDsPROejeRb4vMQJqQWAL0A9gP4wWB/P9hzvs6yB4JwawXwJwBJ0Sr3YLQBBGE8ZyjabbC+MVjtGa13QDg3MyEa7w6330ez7SKob92+BuDvEARyBwAHgBzZbxcAOArA5XvHY/AprkPB/9/y6HS0QWXpxUcKKKUuCJ3njAGltGi4Ofy7g1IaNVeQkdjvQ4VRX6OUfk/8NyFkL6X0pOy3PQCGLVRgtOOujGIUoxjFKMJEJAI5kplzKJ+NdIYfrueHm3c03x0Ol6HQzIaSVzTKM5La9Ex814h/v6lNvfj4+LKzzjorLFYuN4fath5QKpwzL85IQJwt0L841PsqKiraqI6RPCMjgxYVFYXFczCg5hqtOtWCXv2FW68Kt7f4hDI+KTvoOwYL6jIMtBzvopxXOiIm58rYE8ssyVnDwlPN1dvZAq6vWxZnwbj9Q22raHIUv3PwwL6o9NWOXjeaOv3nD8am2JGmOiYeKYKNq4LiiVGpx2i0h5G8ksO025sZdzJ5LN+dte0huTWF6v4UTbc3PRQ9+jfNv5uN/TwUXEXo1Z/87wTCwYOfXzfVFNeCkqnUsvC5iNzqIoG8DAwB6l+6rcnrbNM8SSK6vTEEeHiIeaq5nnz9J3Tg1FHN1ej06dPpK+WfKPzCw3EBjJSj+J17r5xo2FfVXPXwo//dpcjtd+nEDLx554yo8g82rq564k942xfyNZJ6jEZ7hOr2FpVNPa0g6upYvivmTwlIE6/1rJh7DTCXMDTStEh6wvdMgyLGsK/+KuocaOrsE47+UqpIOQMg5EMiCTEWeFnh3fKU80OBijoHvmnoVCSlpDwXNJSWmBh0qCFvB+hno4LLzWHRK1/Dywvp6dcvu0izDfUQSYJQM98Rud786g54OAorS7Bu6Szdb84rzVEI5HmlOaa4RQqXm8P6PfX+2McMQVNnHyrqHGHVk2WI+n3EAlkviLo6mLTD5cZbd81EeWUjCICaU06UVzbivYpGeDm/0BZzr7EMwYr5U0KuPPmO8PTp080dIP83w8JpeWhzDiAzMQY1p5xY9WF1QHZqL0dRXtmIjZWNirYLCnFFZWJlFSkq6hxY9OoOIT+dDIw1Rj+Mmwyb/tUYEKBosCHPhHHfr5oP69132jmAeF84Cy8PvLztGF67dXpIWTQiTRBqNpuIw+VGjJiU1td/opUlOtpwuNyIkYUJ4XiKdbvqsWFPA1YtKDXfH4ao30cskPWSFM4sToeFIcIxXoZIs4ooANRZITxeHh9XnZTlXqO6ed5GoQ1xgIrClyFCgCQxu7QcDCNkOdMLbKSFngEvLL53iVk/zAzIcLW5V7YdCxDGZnAohIzig4FQUjj1eTjI8xB9e7Jb8awRopEg1Ew2ES9HFUFZ2pz6Z0/W76kPuB7KSVHNVZQ3Xp7iqc1VqGruCjnjijzVWDj93gwiFsh6y56aU05w1LdW8wWMkncgOcQklnYrK2V+FoV4JEuy/zTIk4kCQifUSvPEEGDVglKUZCeivLIx5CVrQowFXo1JNhSEqs1ptXdtW6A8s7AEXF93SKHqYg2yIw8mxLIY5X4LgK+95PUAaJuVzJocIh1LFlUgpIxE/cD/6ozURhmqBwNqruq0a+t21SvCzxrVjdl6joh3pC/QWvZU1DmwYnOVlJvO65u9Rfsw9dkxGQgVd3nJGHxx+DS2HmxRzGSfVJ+SAvzIl9SmO/kZgkgHjLgqcQfRJgkhKMkWgqRcPy0Prc4BjDEYXHJwvrZTB6sPxj8UbU5LaNeccqL2dGBsacZEKMw429Cff6qoc+CmV74GxwPW1NwSvftsKsExNsWuqAcLI+RuEs168olMHHuiGTAYH+mdrDInn1r464FV1XlSjH69qntHtBb64XJVg0Loh6LZ7t29DfDyFAwhkklD/q1wEsWGylWOqPRU9bKnvLJRkSiUEMDZ58GvPzks2YfvumQcEu1WpMbZ8HHVSUXyQUDQ7l7eXitde7w8NlY2otxn8vDFPfi3QaT2wIo6B8orG1GQHo+jrcbB8Sn124/l5o3yykbDiU5u7+R4wZTw6q3TQ+IfipYh59Pv4bH8zb043aNttuI44ySXcqiTsQ4Fnv/4IKRQxwaBlFlGqTmmxNmUkxdHIRqc3DoTmWgGLFclHHh7Vz0+rjqJKTlJqD7ZLdWt28tLGqK4bxNKJuc+Dwd5hVf7zCtaON7ea3gdDsxknVZzFWULgV9bJgRCYgXFqlIwaQBQ1MuK+VMUPIKtXsxwlSPqqkNFnQPvVTQGCNfXvqyFqLhxPMXWQ0JA9eNtPeBCyMHIMkTqTLJE0f820NMgjbRO8TdnnwevflmrMAXJl2hqEAj2P/lGXyiZfNX2TvmAlPPXEhx6G0hiGVLjbFi/t0HBWS2MWQKwLAOO403l1BuEuPJBUdWsL6zkUAf97/RlARf3XxgG0vjQ8hiR1/uAh8fDG/Zh6WVCSE8xGqDc20GEqCGK+zahtL9FleVWnX5KUY5et+G1HKGuDM1knVZzFU2hAKTBQUECFEFAkE/r99Qr+vOKzVXgKYWFZcDzQrJdhoGUnCHGqlRCzHBV8A7lJhEuN4c1nx81rLidte3wqiSssMRVXmtpcYmxFjj7tT2Zbpyej4XT8rDRZ/NE9FZBIwJ67mpG6Z3kG3hqGFUOT4HPDrUo7hETn4Yq5ACAyMwWqXE2hUualquZeiUlLx/xbT7qfgvA09dOVcRynv5MaJHsHK6h15C1TDpaGPAqx8px0V5OJD0OYmsygGKjW3RnFF2yKIAT7S48vukAJmQaK2Rie88rzcGu4x2SS5dR+/d5OMhD/RtNOqpiBVzLyxDqylA9RsxwTYq1orNP6Adi0/Aam90ixiTFwtbihMfXN8V0Wm5ZQeRizu1RKiFmuMphSiDXtvXgxS01msJBHCTygakGAwAEur/rCWOWIZiSmxyyK9GZCC0Ncs3nR3XtruWy5b1ZCLvF/msC4JKJGXhgziRjIacSMl5ZQzpcbkl0qAWHCHEJPa80B4tnFCg3eYMIsGWXFUu79Gbt6wZyftAQa2EwECR1FBC46drr9kpKDYUgNFifeqf235fbmbOSYnCq2+/10KljphEF8Q1leUiKsWD9nnq/lh6kDdSeLqed/Tp3Bq7Q9BYpZjxFygpTsWL+FKkPLTHoq2quWmYrVpa3EBBWYDwV4qEvnz0ey2ePl1agcvOpFhjZJrcoD1fMnwKHb8UTqvIQVCCrszCIFVde2SiRXfvVcfCUwmZhcP20PN3lckl2Ijw8xbHWnpAFCUOETrnqw2qUZCeG5Eo00qF3iEWtQerZXSvqHHhXtbw3C9a3mcT5nPwfmDNJcyDoZeEAgHbfMlTU1KwWvzlBbSOWB9T/8kgbvqhpxeUlY2CzMJqeN3Kcl5ccUiB8I65DjcQYC7r6tBUMOc8YVcYQG8tKBxEkjxlKUTo2GYsuKFAsicUJ2ctRnJOXglPf+jOdd+jY3r87OQvLZo9HzSmnIsEBICzV1UtrI65GrohWVrm5rE5VJcLsIRjRrrvnREeAXdaIK8sQcDI+E8YkIDc5Fttl5py7Ly1Got2qsACUFabippe/1uUECOaRVQtKJRNjJHtBQQWy/MBFbO5EKtpi3tldD0qVgtft5XHaOQCWIeApDRhkB8PwBxXfEa6f5WBA61SfmePU6kMsejY0tUYgH4weE365WhPkFSVjsO3waXCghkZWOdeYnImK13A8DdDUfnhhARZq7N5/XHVS8d4t37Zg+5HTUvm+OtKmO8Hsa+zC27uC+7EacR1q5KTY0diprUEqxpSKZ7+HQ1lhKi6flIktPgHLUWB/YxdqWvxKSWqczW//B3B5yRhcXjIGH1edhN3K4hOZcJYjIzEGZYWpWL01cIGptbSWc7WruPIGs6hX9Zv6WoSZwynB7LJGXAFhUhDHzdHWngCzafeAVzHxi/1Xy7wquOoCiy7w93ctjmZllimTRW6yHQRC5kO90fNFTavkSXHNuTl4f19oKcgnZMbj6Gl9pdes3+uZApebM7QTyzUCcTDOLE4XVg4hihz1bTaWYExijLQs5rjwJzt5B+R4itwUu6am8L0p2QHPiic4H5gzCXtOdEi2ZAIaYHP8uOrkkJ+2iwShHtdWt42b41FR58AXNa0B98kP73xcddJvIiKCieieKyZI7lrbj5yG28ODEqUlQpx61UebGQKsmD/F0AygNsCoM2bLwRCiMMcYuSmGejhFfdjMyC6rZubhaNDNXfnPyv0N5X3XnpeLiVmJmFmcjppTTqzeelgyw0Xqs2xKIHt5CtZACBRnxKO2rVc6aRcXY1HMSrokGOCOS4oF4ePhQRiCcRnxqD3dIwXDuXF6flS045EWs6JnwAuLzoyqN9uK/+054Qj5OwRAVlIMpualYPlsYTln5lCI3jv1OqDa6+LD/SelZ4h4+IdlkOpz81oxfwqqm7tAAZTmJuPzmlaFljfUsRAiRWaIft1qGcEy/pNhIhji08hYBt80dOI3Ww9L5gACwKZqP7nWmRpnw8oPqqX2EeOXLJ5RgC9qWiUtnEDb7m/E1UjAjU2xo67DpbiOCsTNziDSVetXuXlMvWpkGWBKbrLktCDvvyyEnHytzgF8b0q2pEWrzXCAUK/h+CyLMCWQheAyftuQ2FFEQ7goVMXGJzC2M4lYtWAqFquyIQNQJPgcjPTwIwEJMRbwOjOqkQ35m8YuU9+hAFq6B9B55DSWzx5vOo4BIKxS5Ei2W3XfI+cu96CQ+4HylGLlX6vg5QV3IlAKL0+llcIVviW4qH2ECx3z5aBi4bQ8vOvzcYVBSMVAV7KYgAM+DENwZckYfOGbpBSbZT7NVt1+cq1TL8v4stnjsf3I6ZAn5VirMuRkaW6y4f3RhnyzM5gfuporBXD7rCLp7IPD5YazzyO54xIAKz+oVsTVkY+9R+adHVDHajOcuIozcxxdDVMCOc7G4pWls6STQQun5aHmlFMxaNRCdf2eBl37kQhxZlYXJJKZ5kxBnI3FKzrlNLIhi66FBMKABaWS4NOrbfmyV65phwr14Veb7zis1nvUWpo4URPfUlYYVBScjJeao7gEjxTDYUwuK0zFurt9HkG/bq4J/oTy2Run50uhIylP0efhpHgKagTTbPXa2eykrJ6QE+1W3XvbewcMr8OBGVcyNVdA8JuXhwBd8/lR2XFqwSdc7H9iMDSjuhmMiHamD4bIG1du49xxTLBtiTOEiFULSqVj1AxDAnz/WALdmTmSmeZMgl45jWzI8o4puteIgk/tDscQQRPjeMF1KjXOFtSfXAvxqqOywTQqLS1NLpxZn1bM8VTx72jHCxifMTyn7EPxCNKr04XT8hQmpXmlOdhzQvAXJgwRNtR9nk2R1JWZMWZXaZ1TcpJ0751zdpZi/2jO2VnhEZRBPYEYuZKpuQJ+gSk/jCSOI9Z3RF3uJRSsbkRlIRqrOBERndST21l4SrFic5UkNERoac3PfXwQx073YnxmPB7VWAqciYjU80ILRjZkvdlbrGtnnwc7atuRlRSLZT6bsVwghuOWw6i0DrUwMYLeElrkZXQMNVJcOII3g/XqVKuN9eptqMaPGCFQPK5ipCH/aFYR/rqvGTyEldWPZhVFhUOoE4icKwDMnZwlbXiq47SLvsKA+TpdPKMgqpvNEQlkYbffv5vK64SmU1fiu8sviuSz/zEw2rE1WoYaOdYbHTYxi3BNAWqOev+OFs6kI51yrlr1NNh1ZYSEGAt4a2geBDtr26XNAkIw5C6raq5ypUQdp12e/WO4lcOIBHJZYapkkuB5CptV2UjDHTpzpHlUmEU4G2/B6jwSt5zUOBvcDKTsFteP4I1WUe8c6TzPpDo12u9QQy8e+lBBi2tFnQPNnX0K891Ic6U1dVJPKzWS2iQhty9HcmLl3xlm0k2ZsfGFUudmhbyca05+ERIYBoTnwTDDE2PYCOqTesKG58jmeSbVaUGBSQ+CEF3UogUjrurwo4suzA85QP1QImgPoJS+SimdTimdnpmpnTS1rDAV91wxQVE4LfvnKASEUqfhINQ612qvULha7AkBh0lGEuRcAZwRPM+kOjXTV9UuakNRLiOuigNMHI+xvgNMIw2msk4TQk4DqPNdZgAIjOsn3muzx/tiFhMA1ONoPkz9u82GzwaB+GyhXgpwACUAalT3h4toPB+vlwJcVacRfVde5xSA19FcQ83H/CjU5cpaOm2ZRYkQJnJe1aZGCKcOQ3lGlytrT6KW5DEUgX1vKHgF8ATwhO9orwKyOtUaJ9H4dqjQHFdyGPTVgHcRm73PYPyHw0sLIXM1kEeDWadACPWqAKU0rP8A7D0Tno3kW8P5/HDzHu42GEz+w8ErGuUZSW16Jr7rTHj/yDNajWIUoxjFfyhGBfIoRjGKUYwQmLIhZ2Rk0KKiosFjA6Cj143mzj4p51VxRgLibIGnbioqKtqojk1msHi63Bxq23pAqZKby82hZ8CLhBjLiOEaLkYSV636BiDV9cED+3S5pqSl06TMXN02GWzIuXs7W8D3O5dRDRtyRkYGHZObb9h/hoonIcBAy/EuynlTtO412/7BxsVQco02l2hylcOUH3JRURH27t0bNslgyQEr6hy46eWvke2bIxgC3DO3ROG4LYIQoru5EC5PLR9e+THLj6tOov9omxABysdtZnE6lqzdKURsszB4RcPVbDC4BrwnSicFI+Eajt+50TNPbDogxXNgCXDdhQUor2yU6prUXK+7GdMfk4bU656DlyV4ZemsId9RX/P5UfzyHzWgAE6+/gAG+roDhDEAjMnNh/fqn4PlaNhc5X1UPHUW6jvWfH4UL26pkfp03Uu36WbINdNXxbHMUGCAAG8uvyjiNgiXq+jyph6jFXUObKwU8n+G4wJn1HfNcJUjKklOQxmIaj9AUGVkr7LCVCFbtTx2K4meQ3kwjlo+vACkv8njIzEEklN5OAGpzfghjzTo1WM4fufBcgbKs6KwLKNIchsscST1/efmhAzbQy2QnX0e/6k7Azdch8uNGF+nD4erOrciQ2DK7z/c3G/B8NzHB/1JjalwHekJ3XC5qseomL3+3b0NUmjg9/Y2YJ2JydBslvVByalnlph88Krj44pwe3kpwHN1kzKkZFlBSlQGUijCQqvRqpq60K/Ki8YAmDo2GVPGCqEHxYp3e3mAEOxr6ERFnUN6p9aEQlUZQyIu4BDB5eZw86s74PGlfZJ34GBZp7VgNJnJYwITAJdPUlkmCELu5G3OyCONmYU8I7cRvByFPHKyyDWUVPMAsFGVW1GrLtW5DOUwE7BHzcuofY+pkk2or8N5pzr6oVEwffG95ZWNaHMOSAkdKODLI6g8pu7htMM+6HE06ruDmlMvGOS5vcTMq4DQUcSCW1mCOy4ep5nhgqdCcOcvj7QFKBKicItUKOtVnryS5TMawxCs21UfkHWAAGBYgoMnu3GgqQsb9jQIR8fnT8FTvoh2n3zbgs8PtYBhGCm2qjr315kKI20ulKzT6k4tr3Mx+Prjmw7g+ml5SI2z+bJMCLGSv6hpVeRo43iAiU0IapMDgL0nOiIsuXmoQzOGCorAbNzEl1cy2CoCUK7eAP0g6nKEevrO5eaw6JWvpWPe65fpmyFsqqBJ6msRZlZWwXLqqbne9PLXUIdjFwKhBd7PsvqrcS2ORrHKBzWnXjCoc3s5+zxYsnanQrN0cxQ7att1k5+KUP/m5ShWbz2sm4AzVGhVnlbFvXXXTGysbMQ7u7WFMYUwMDjqD7C+YnMVrjxrjCKFvZcHiCy2qtHS+kyGfIgFyzqt11HFOl+/t0HKXrFhTz0YhpFSgc2elIlPDwbmiGNi4kPqFB0u7QzMg4nFMwqw+3g7vjh8Gk39PbrH1NxeHnKp0tjh0s3GHWwVMWdyFs7LT1Fomuog6q9uPxYQkVGtjevhtHMA8b6B4eWBl7cdw2u3Tte81+XhDK9FmDH5BcupF8A1hLWnmL3GyN1M/d3yykaMTbErNGCtleKg59TTQnWz0syww0dIjRhLYJEZIvwnLh/UApuHMKt/fawdTy8oDTvMnVb8hsc3HZA0+wEPj5e3HcN5+Sk+QRv4DvmSUA6OpwHCwsIIMRTE2KrRsssNN9RJLXsH/FmVZxanI8YaOOmJdV4uW1qrQ4nurG1XZAT28gB8qek5nkppitTmI36gN6QcVkMUSkGBt3fVS/GAmdgEXUnn4ZVlanH2B0RRBISxoa5XeTxfq4WRMsHIOZzsUiZaPdHuws2v7cS6u4UJ8e1d9VK88hir8WpuwMspJo/jbfqrcHWd67WBmWBXZuyyaq56EKtYS/kTTT1TcpKkQEmEABv2NkjZ2u+4eNzw5dTTLJDqekxSLCynnAqhzDJAZUNnwL0UkAKQsAxBnI2Fsz8wdTrHUzz1/oGAmd0M1IFG3qtolPhQAJ9824JPD7aAEGI6XKMopwiAc/KSseLqKYpMKsFsXYMFvWh34cZp7u73Qp4ZbYss55160gOg2MQVs4QAQpM3d/ZJ5qiZxemwinZ4FSiATw+1oKwgFQNeHsl2K+o7XPjelGw89vzpplB42zWUgcHGrz8JLUlIQBYWhkFZYSrmn5ODzfuaQSFM8FNyk7HoAkEhETfxGIagrCAFE7ISA7wE5KYKNdy+PRIAkqkNEEyORlqnOt1UWpx+POQe1ThWX4swE+zKjL1bzZUAmF6UimOne9HR61+9sQwB5Sl4AF8dacOOY+1YtaAUABSmHoZAUtZExcTNUby8vVa6BxjinHouNxeQaeL6aXl4z7dbaWUJxmfEB2iMlEKxpJf/Xcy5x/FUUxiL4Ki5mKpGGwU7a9ullEFyqJeIoUB+t5UlWHH1FAAI2dZ1JoFX1Y3LrVyGyic9ddxl5XuAdbvrUV7ZKJku1t09E4+U79dMuc7xwG5fQlcrS/CObzPxsRB5uzyBbT3Y6OoLzUyitnE6B7wK7RoQ6utAUxdqWqqxcFqetNLgeIrdJxzY19AZELZz/Z56w+9SAK9sO6YYlyRIJmf1hNkgS2IarFxGqTXNRJAL9V41Vwqgsr4zQA5dfU4O9jV04kS7CxRCIucVm6swJVeZDUW+P6KHaOTUC6o6EEKWEkL2EkL21p7uwYtbarBk7U7Jk6CsMBXrls7CT68qwcprSrH2q+MBpMUsr5HCKLW6nGdzazuWrN0ZwFXEzOL06BCSfx/+zNhmbF0jHfJ6NfOcuHRjfZtMrKyniRsr8k1gADjeFtxV08NRSbuLFtfBQE6yfpZlOU+eKgWHm+MDhKlUX15eyD6t0v7cPg8BERV1Dnxr4OVhYwlKc5MDFKcrzxoDdQAgOVePamC3DIP3ihGMuAKBSiEB8ME3zahrdwXcNyYp1vT3o5FTz1T4TVFll4d2lGuiDpdbUxMGgmdtCGbn09oo0uNpsScYhqEsK0zF9Ag2CbWosgyRMmOrhdGZbEOmqpCWcoj1UFHnwJrPjyomPnF5+dDcEqy7eyYWXVAQUG88/JOsoK35f4uP0T9NpdeXjLgONbKSYnR/U/JU1gpP9YUBTwXTxaoFpVA7LaTG2aR22FjZqCl8LCzBkhkFWLd0Fhwud0Buy+W+rBr6XJUYBtO8Zl8TYbb9GQJpg17xHgBXlIzBs8HFFUgAACAASURBVNdNxaUTM5CXomwPMXO6zcLg2vNyUZQeh+WXFQ99Tj3i24QjhMDZ58ETmw7g3b0N0gGPFfOnIMbKwO3hA7wUguGsrETYLIyU3l7cNf7SRJpyEQkxFvBBDOuPzjsbN73ytUIIhIpz8pJROjZZKDsnJG9dtaBUWqaY9e08U6CevWMtjKGbj3rpVq7ymWWIf5Jt6VZuPvUOKM0h4uC3smREZ9UQIbdTmgGlgmD87GBLwDJfVEruuWICqpu78NYuQZNmiLC5LprJLAwREgr7XmBlCW6cHhiQXfSfZ4iy/+rBTNZp0e9Xfh0p1H3NyBSo5soSpdkkOykGBWlxkilMwR3+el48owDPfXRQshUDwILzcjExK1GRn/L1HSfw3SnZEbvomhLIucl2MITAy1MFQSAwdXZqnA1PbDoQ8gbZ4RYnnr52Kg6erJLs0ctnj8fy2eNNG8hDSTVTVpiKK8/KwiffBrpTBcOiC4TEhgun5el+IxI70oiFahlDGBKym4/o1L9+Tz2qT3aDqjJgzypOlyZjNViG4OkFpaaPBUs8Td0dHRRnJuCowWEIPRAIdbVh+UXSoYYvalolF0BxM1SdlZrCf4qR4ylKxyZjf2OX5KqpDsiuPmQRinan3l7xauzDiEiPt+F0j1txHSnMmALVXMXkJSLu/84kVDV3KQQy6+soaiUu0W5VJHedmJWIe66YENX8lCJMCWQvT2HR2PSSu+TIBdHnNa0BAo9AmLE5ShXaKaVAVXNXQNqXcAVbsOcq6hzYdvi0qXdmJ8Xg/u9Mkjrvv6XQNYBa60iNt4Xs5iN36rcwBDdeWCAsv2UZsJdfVoy/V5/CCZVN78qzxkS0HBwOt7dls8fj85pW4WiuQQQvNTWxjtVeQeWVjXivolGxGar2atkoE9CLLihATUu1bruoD1mE5sGkLAZnsAF+8YQMxcbkxRMygrw7OMwdR1Zy8/LA8suKUX2yW5qAKuocCoeElddoT/rysxYUfjNbpC5uWjAlkEVTgGiSYAhgYRncUJanGZxDvfQiAC6ZmIEH5kwCINgNPz3YAkoBm5UBAQLSvgyWwBNTzIi8CKBrZhEnkTVLyv6jBLAaYxJjFHV0z+UTQnZbUqTQ4SlyU+xwuNwKDSPRbsWLN52Hm1/dIZ3Ks/hWSpEgGtqZWZQVpuKdpbOws7Yd9/26WdcHLlm17P/+1MCNIXGj2MsptTF1Gi51O2jluhQRzgGGpFgl16umZOve264y2aivw4EZU6CaKyBoum/eOUPxvnW+NjLqu3qHnsy47IUKUwJZNAWIS6nMxBhMyU3W3WwrK0zF09dOFbJSU8HOLHe8fvXW6QGnhOTLsGhnhNU7Km21MLh9VhF21LYjxsJgX0MnvDwFQwjuumQcEu3WYcucPZKQFm/DJefl4ovDp3H5pExTKwU9bUL9N3GQlFc2ggBYGIVElA9+tySi58OFWC/3GqQuyk+LQ15eMqqau1Gam4TVPzxf875QtDF1Oxi1SzjaXahcgcCj49HwQABCX5Xmp8UhNTMeR3xmo1hraHWmBa1DT2b5hIqwDoZsrGyUlp7vkkYpZoP63HZFnQMOlxurDOx/6gJFe8aRc9E6Ki3au+VLZ/XSRRTkIt//VHT0uqVl6Pv7mnHhuPSQTQl62oT6b2Jdj8SMwIOBjl432ny2832NXXh7V71mnUZbGwvnfaFyBfzxMszYqKOJjl432mQ2/NtnFUV0qGyw5JIapgWyYqnDUQA04DgsEH6QjcGyy2ot0cQln9o4L+6wRlKOwYTeCbxIng/l9F5XnwfyXRTRET5UaLWt2lY6GHW9fo++4BhudPS6IY+QZMQ12mPD7PvMcAUEoTxc9a7muiPCrNdDtV9k+kypwseWJYLTPwncmQw1Jf1QQe0bLOdq9NtIK8dwQm3vjNYyVMRg1XVWGE7+QwUrqxyCo1yjgzOJqxymNWStmAVaqvxg7EACfjuw2ePIRssOo9/UISKbZDEY/tOQFm/DDT5PiO9NyY669hPNPiN6L1hYgmURbgqGi1D6amZiDNwsgZejQ8Y1nMwugHmu4X4nGginXoeTr4igAlkru4V6mamFwbC7yJe01tTcScF4anEyy0Msx0ZfhoF3dtdjoywGw7875PWak1+EP359Ah4vjz9+HR1HeDki7TNyrpl543DpxAzMK80ZlnYy01d/cU2pZGsdbK5mzULhch0OU18k9TpSTJNBBbJRdotghYi23UURJ1blwhlJFo5QyiHGno2mE3gwRGorjgbk9ZozYQoVg7aIEcOiXQeR9Bk519jcifSfR9tM+NhGF6H21cnnnE/N+wNHh1co/ThcrpHGBQ4HkdTrcPDVQkThN4e6EPIlLYKHxwgZoZRjsEwwIkaC8DWLqDXAIIBS7XRGQ4VQ+2rPgFdIvjkMY8hsPzbDdbDHSzS5AsPPV0REAnmoCyFf0t73q+bD0XpvqD6eQ+X6MlKRGmeDlyXSyaaRHFOCEGhu0g4VQu2rocRdGSxeZvuxGa7DPV7M1utw8xVBDE51Bt5MyGkAYpr4DABtxGaPZ2xxibzb5VSH7jNABgDzCceUzxZSSjO1btDiGeylBuWIhKv4fLycq9zWBaAEgNZJrmh8N5znDeuV2OxtjM2ezbv7Tg1ye4fyjD5X1tLJxqX0BOmXg8VLjUIAT/iW1EqeUp2GNIYi7RNGCGlchcg1mjyN3jXSuBq9X5erApTSsP4DsPdMeDaSbw3n88PNe7jbYDD5DwevaJRnJLXpmfiuM+H9Q5/bZhSjGMUoRqGJUYE8ilGMYhQjBKZsyBkZGbSoqCike11uDrVtPaC+SJq5yXY0d/UJ1wByU+xIi7fB5ebgcLnh6FVmMCAEKM5IQJxNO3NERUVFG9WxyZjhaRbqcmXEx6CtZ0DJ3fd/CuGegZbjXZTzpmi8btC4utwcega8SIixIM7GoqPXjabOPgXH3BRZm/jq++CBfcNSr+HAqA+wccnUkjwGAJBityI/LW5IuclRUVHRBh0bcmJKGnXb/RtOY1PsiLWy/j7m+7vYl9RjQt3O0eCqV6cpaem0PzZN0V/0vnmqqx+ne/wpnjITYpCdHN3TckG5xqRpureEwyXSejbiKocpL4uioiLs3RtaurLHNx3Aul31oBB2uy+akIGvjrT5Mw8zBOefNQbbDp9GrJeHOpAfS4B75pZIMSXUIITUaf5gkqdZrPn8KF7cUqPINK3mLh9ELAHqXrpNN1lctLlW1Dnw8rZj+OxQKyyUgrcweOWumVj1QTV4VQD4iydm4J9H28BTf33fe+XEoPUabiyMaMOoD1iSxyDnttUAALuVwd6n5w0ZLzUIIXVawhgA2KQxyFn8onQ9NS8Zc6dkS31M3ZfkY0L0n7d4eamdI/UOMKrTpMxcpFz3nHAfgHuu0h+f01ZtgcXlT/SaFmfF3hVzI+IWLlc1GAK8svyikOsqGvVsxFWOiNze9FBR58B7FY2S8GUZgnmlOfj6aJsUG5njqWbwejEqPyHEMKnpYKCizoGNlQJvdbQx8VjlkRanIjWN1gzMEIBlGXBcKIG0o4eKOociljAADHh4rN56GDaL0jrFAJiSk4Qdx9pBKR2W+h4qsMMRoR6hHZ1WpzaK8WVREfsYBcAyAM8LfSpYnBWtPhvMjUsdAlcPPO/fdKIAnAaZte02FpAJZHsUtHcgPK5qUAqs3noY80pzQspCo67n8srGqNarHIMikMWYxiIuLxmDkuxEMAwBZ5AP/LuTs1CcEe/LXE2x6sPqITtlVVHnwM2v7ZTSh7+3twHrls5CzSkn1u+px4GmLsMU4CIsvvx68uDgg5VTT2zw1DgbHC439jV0KoQxIAycr460gWWJIsj20suK8fqOE+B4KqWUX/VhtekYIWcCWHboBbLR0Wk5WEYpNpJ9bSlXTqgvESevMi+q/efFdFiiAAjlKLCZPHV9Hk4R7a/aILP1+MwENHX2K64jRSRcMxJs6Oh1S3X55ZE2fHmkTcrivWpBqW5sFkU8G4bgvQr9kMPhcJUj6gJZrR0DwFZfunG9jNQiMhNjkGi3gqfCEeUBz+Acz9XCztp2xSTi4SgeKd+Po63GqenFBtUKZB+t+B1as7F6AlGlDFOAAvCKGTh8nU/M1iFPTRMsT9mZij43F/ymKEN5dJrobp5zvDJPTZdPYxMDooMQadx4OaoYD/LDDOqY3tdPy5O+7/YKqyR5cggtnmbb32iaO9ziNLwOB5FwnZyThJ/MmYTVWw8rTKcUQmq6FZurdJU/eT03d/Zh3e56icNGHW05XK6mBLLLzUkzsN7SSJ4aSQRPgU++bQHDEBBKDQWHs8+jqKx39zZEJWuEFtQZRKy+LLzit42EMUOAa84Vss8O1skeoxgbYpIAEeo61RPQHE+lZZpNlY5rKM0rQwmj3G+DBbnZwUhyqVc0TV39ioBWu2rbFclS1SURY3+oY3pT+LNK81RYJe2qbceN0/MV48lMnjq11lnVrK8hp8bZcKp7QHEdKaLBNT8tDlYLA6+XV6Qj8/IUqz6oxpSxyZrJEcR6FvMbitEf393bAC9PA8anufx/fpiK9haTPQEvbqlRfFwtNFbMnwKbhVGkewcgLYs1vwEhtnJSjCUgmzXH05DO94cS7U0OLWG37u6ZeGXbMXzybYthnAZChGXk36tP4UdhZCIIlauRjTCYiGEZoCgjIWBSESc9tXYlCmm1ecVsvQ4n5Fxt2f7NpuGwIX9R06r7m3JMKcNCdslyz5VXNmLA418BGR1XVwuA66cJeS7lGqGbo3h7lz9JqihkjPLUGXFVa/dyuFSrEvV1OIiEq9vDSePdwhDcPKMAvQNeRSLWbxq78E1jl2SuDKYtN3X24R2Ztiwfn8G46iGoHzKl9FVK6XRK6XTBjqUMHq4WGg6XG2/dNROLZxSANXg7AVCUHofllxXjp1eVYN3SWZo2qVDP98t5ZmYGP6G4s7YdAx7fks7jr8xz81OCZymm/iV+OEHUQ+VqFDi/NDcZRqZRSoF4nY0UdT2XZCcGJMw0y3UkQM5V/nd1tuyhQEt3v+5vcp5E1dlEbV4cV6LN/5KJGVh5TSl21rZrhrwtK0zFivlTcNGEDKyYP0UStg/MmYQYXwJhQLvflhWmhtT+NouyP2UmxOiWsb13wPA6XITLlRAEJNmdmJWouXjxcNRwXIscSnOTwRACBtpyyoirHkyZLLQCtmgF5hE7w8JpeXhl2zFsPdii2BATl9P1HS68vuOENFurEyPOnZyFZbPHD4o5QJ7am0dgam+1hi/CxhLBrscNfjAYvYAnYgp3CsEmPP+cHMVMTwApFfzBU9UK0wYgZPoYKfFfhwIXFKUN+TdnFafjG5WboRb0Mjmrx9W80hyFjVgrf6VWuEmxD5VXNuK9isaI+m1GgjLr+B2XFOveOyEzAftk5Z8QhU09M1BzdXkEzZjjqaL8MVa/2U6ElSVB60esb55SMAyRJsFIYUogF2ck4J65JQEbV0aZOM7NT5E29QgET4qW7n5809glbTiI2ulQJkZU72TLM2dfPy0PFECbcwBbvvVzv2RiBh6YI2yYD1VUKK34wPJVCQHFxKxE/P/23j06jurOE//UQ91Sy623LPklC2FbgI0HbAXbkwyBScJAlgkBQkhgMslOCGEPmxM2u+eEXzLjw3Jm+ZHdyQ5nZthfTDIzmc3hGZ55mRAn5pGMbbAciCUc2bJsyXq/Ws+W+lF1f39U39Kt21XVVd1d3S2nPudwcLWqbn3v9977re/93u/jvlQlj6s21WBrU1g3Qzz059sRicYxv5RA98iczlfe5lis/K+FwJ4iZHsLV5St2PFt7Eub6kK4YmsD3j4/jWta6/RKzvy6yuR6ZWfeonPo9l0bc5q3dZUBfO3WKx2tzzBX7ou/9hp1lQHsuqJJX78gBNdd1oSlhGJIWM+a7V7vGcfY3DLu/EBLRv7wa5CVH7nAlUAOBSRTR3CrpOKd/REMzSxBZnxyr2tfi/2vnNTvUYnRn7G9OYxINI72Zm8P+1kNmaSuqdYYS6gQUod25Uz5b/6UuliVqPe21UMWtTSYkqj5D//jr84gnlQxPLOE6+IKXu8ZNxw2ANqioHwtlfyvhYCdv6xXMHhKgFgaW6NxBUfPTWtC9Ny0oTwYv67sXK/sXOCs2sumZJHTwqX1lQHb60Lgyx++FG+emdB59vrpCSQVFcfOTaNreFY/vKNnYXSH0TOW7m771LEBw4fIq/XjiR8yYDw0k0UBH7m8CQ3hILqHZ8E5YeB7vz6Hj6W2aoXaRkeicYgC9Gio7uFZRKJx3VRBiFbq/r5r29Lc2Upiuy+k9C9BQPfw7IqLk0JWtAJoroMH3jiLN89MpNFbCvlfC4EfnxzBgx+/vKDvdJoPORKNI+igCgvb3nsXZvSDZ7aCupULnNn89HoOv8PZufnrQsDKXS2eVPH0MWMpNrsdxlPHBvCNlzQlkppU79rTonvC5NOHx7PkQmwHkwrBr34/jmfeHsAPj19Ic9RXiWZEL2SF571t9ZBTp47Uva42FEg70OsemUszzBe7EjV1LSQAFGXFxcnsgIIA+OWpMf0Ak6U3m0OH1YhiaMjACn9d5I22Xdy7N9dib1s9Xu8ZX4mCZaL36Puoj7nd/PR6DvM8L/YY3LZro2GN8IebdgfoB7tGDG2y1y+cGMQzbw/g7u8dtawv6gaeCWS2g6Io6MEeikrw6Y5N+NgVTZBEAaKgCRPqC8wyhW678tFRHrs31+JTuzfqA5RUCA52jeATf7TecJ9ZqXu7wSsE+PffvmsjnrxnLz67p0X/nfUsUEkqnLuIFTSKics8Nn9ZobM/gscP99pGadWGAgikoigDDqqw0NqOgLaz+9TudJ9ZJ/OT3iPCmzQFPM+LNQYUVFu+a08LApKQxhv696/d0G5w6X38cC+2r6sytEVlghcftayqTjuBWRQR6yNJO8xvmd1su3Kl8/ZdG7UAi9Qp6296JxGQRdx3bZvhAMyub7lWR87Gt9fq/ezBTW0ogId+1IW4ogXiiKKAOzo2mTq9lzLcJDIy80OWRODrNxXWXAG4qzr94r37HM8lM39jHk7mJ3WT2/9KlyFNAY9s5+onr96It89HDNdeIxOtrPeXlROClVnSTCZ4YUfOqep0JrAdZHM7WB0ysOhi7KJuq+M6pZNO3McOndazniWSKsIVZfjBF/cY7uU/Hna0Z0IuPGVpt1po9Pfu4Vk8lcq4p6oEG2oqcj7U8Qr5KPJKuKrD9/9Ze9H6Rv3cU7qsYSfKj7+buWQmbM3G0UmbkWhc37laaXjZzlU7Lyav4JRWXvA6CX02kwlenMN4dqjHwwkTDAeBkmjqN+gFXQ98dBveOT+d9qVjk/e40dZLBbft2qiHedJ+rfY+rRawXjy2SR+ygJ0mZ3WAZ7bevPS0MfNiskKhFAP+PXa8c8qbXBQzM+QskN0y044J7FdJUVR85poWrK+p8HygePPK0b4p9IzO6wJLFIQ0TaJUhJcd//kvOLDixVLKfcoH+iYW8Hc/70GZLOLpLxX+Y8N68WQ6hs9FIDlJwWm13rLR8JzSyvZfFKw1ZC+9Pfj0l/x7MvluF8MLKSeBnA0z7ZjAf5W8SipkBvoeM4GFVDSOAG+1dbdwwn/2C84Gg5Rqn/IFKgftXMm8BDuXM/kh5yKQMmlymQS2Gw3PDa1ONcxM9GULnlY2+x19TyYa8639OkFOAjkbZtoxodi+sYaUiZzA2n/zdkfJrAsJt/zneV+KffIChc/15twPeSGW1CpRZCmQMq2ZfJol3NDqdC17ZTbhaaWuoXyKh1LzxXctkGn6ucl5LVkIjU9QUhF39O8CgHBQxqHfjwOE6HHvB7tGcE1rHU6PzSMoS3j4x9248wMtaG8O48UTgxifj2FwOoofHDmPvW31iMYVPZzxrj0tjqowUDp5exGl67bUyfSBN86ib2IBdZUBzC8n0T+9qOfcUAEERAGiKGDH+moMTC3iSN8UXu8ZR20ogIZw0NZbxE21AD7RPNsOHyHEgk1VqgI41jeF7//mHKZSGcOqymXUhAIYn49BFgX86WVrcduujZicj6EhHPQ8+X8+DurygUyuZF7hq0+fwNDMMuTqpkus7lkTlBFNDaJCoM+Xp44N4Nl3BtBUVW6Zz4WdG2YRtPTv17TWYWA6iqs21ejmuEg0jtpQAN3Dsxifj2FtOKivC7e0WuHOA/+OpArIItD7iLlnjFuhyPbZDa071ldjx/pqHOwaQX1lQK8YAkBfe/TfZoelPzhyHq+fnsB12xpxzSX1Og1mzgq5mKBcFTm9YufVJPnn/yMthysLSURaJJ4TOHnuvlSVi3hSxdC/flWNjfaapjO7YufVRL3lEUNKUOoCBgCyJICQzAnzMyEgCXjoEzvSDscAo73q9P+8/fdqLGrqe0VppSfy1C/7yXv2omd0Xo8QAoBHbr1SF8ps9FA2EKAlVuG3nYIgdPLZ0ig6OjqIm5p6Xglk+i47WoPrthJaU8+KPi/xoUd/icFUxYyRf3sAsZEzpkd7PJ0igL+99UrD2MqSgGe5dJD8+LNzw+zvFHaFDAKyiN7/ZT1Xy9dtJc0MrXaCdss3fgo2p5XdvU7B9+nCP/5Fv7IYaXVCK6Ct14RinY9dFjWbN5UZdF0DsKwWxD5jtvbp+rKbq4b2Mt3A+vbVb7gEYRthDGQnjJ0+92r3KFOFwXh2zdNZxWxXDnaNIMHQnczQB6dIpIJJzJzD7aoFmNGqa7pMO7wb0sGuEUMCplzARirlQ0suBW3YKh9yMcCWL+JhR6eK9LFNKuk5wc2ix1iBbDU/7GZ+prnK05q0WbP83+zudQq+T2J5pWHiZhp/O2HM0khlBiNrHD1jtvbdri9XGrIgyTOBxtYqCIKgOxjyoBUz7WD2rIPnlOjsqFhR1SQASM6OC8rSnOn9KTrDAAQCQJmfGpDD9S16+7TTmei0opWhOam1vSl1F0lENHthKhhAIAASE+fniJKssadVEJn0YGpiemhcKCuPyVUNm+m9ybnJfjU6MwkAYqimgf2bY7rp7/Q9keHTXGjvZrZcOTvJAbQD6AHQAGASzuH2fqfPGGhlIYWqiVy9VrsgQHyst7OAdKGssXWHIMlBAEjOjkNdmvuyWeVpqaKKyDVNKz/QOcWOLSEkERnuIfGlCvpufvzZuWH295W2YKwmzP4OQuI2c9WM1vjY2RMmtzYEmi5tMawv63szQec336f4RP8CScZNQwDTaE3RAAhCWr8F9u/av5LzUxdW1jX3TFqb2r/4tQ+AJKaHxklieQg2czWtvWz+A3B8NTyby7uK+Xyx6S72GHhJfzHoykd/SmlMV2Nbq6F9z3JZ+PDhw4cPd/AFsg8fPnyUCHIRyGn2sBJ9Npd3FfP5YtOdz7azocVL+nN5R7HGM19tFKLtUm2r5NvPeKjHHupUVlbuvuyyy7InLYVoXEHfxILhxHNDTQUWY0nMWORNFQCsr6nA8OwSCAGSM2NgD/VyodOMHicQALQ1rkGIKyYajSvom1wA0Y4QEBs7N8selJjRyj/T1pDebj5o31BTgbrKAKYX4xiaWTL2JwOtUkV4t1S9clBSJol6WkWe/vXVK2Nl1R/DM6nf2H6wv5m10dnZOUksDkoaGhpIa2urQ654C55OlqcVlWt2q1XN2Y17inc8BAFoqAxiYsF9YVH+oCzbddU1NJs2ljs2VLumh0U268rsWZYms3XDyxon4GVBJlot28kkkFlQP9Rc8fjhXvyvn/cYfvuTrQ04OTSLmah1Ius/2dqgZ2Ub+f5XSWy011TDd0vn44d78Xc/78lKIP+3P2tPc8p//HAvvv1aD1Si5SDu/6fPDyXnJ0297imt/DNfuyG93XzQ/idbG/CDL+7B5/75mKGgLJCZVt5nVgBwLuXfy9P/x1tWxsqqP+wzdouDwLyNTD7Tkx/976Y8KLRPsh2dLe1XEvm2R7Mad8o7HpKg1eo7PxV1TevQd788l5gaNJWcbtaVm7SpTpHNujJ7NhNo/Uw6f52AlwVuaGVRFBuyVq3D6Edy0451uG6btVeILGr30MTbyGNE7N62epTJ7llhVZ2WTxCuxqPzTmjIJum9He1mnjo0OomPdKKVqu1oLZOM79lQU25JPztWmZKk03v4bsii9rtTngiCcK8gCMcFQTg+MTFhe2+pYE1QznrcaYJ5YGWsxVQ7N6ZKormFuryYl2oQZmOZK7JZV/yzlAxR0NavGcokwTB/Re42UUgXnLwsyJbWomjIgBZeeOCNs4awaAB44Jnf4mcnRyAKApqqylEmi7ikoRL3pcJHaVjiV27caRlRlA2dnf0RvT7WjvXVONwzjvG5ZVzSUIlzk4uYiSYwH0uiKRzEprqQIXTaqj0aPtnRWpcx+o1/xo0zOaV9fD6mL0xK3w+OnMfPu0exJijjv3ysPS2a62DXCLavq9LrBmaitfxT38LQzDI21JTj1w9+xLLPViHldnwCoIflU/oB6wrfF4OG3NHRQQ688Iusx50NuedD7/nxpZXH6TWttNw9MoeKMgl/9cFLcPfezY7mqhPQaL18ROnxfXazrvhnWT71jM7rPJqPJUEA05QIbOj05/a16u10D88ansmGVhZFE8i5wkmIb6nAp9UbXCwCeTXx1Kc1OzgNnfbd3nz48OGjROALZB8+fPgoEfgC2YcPHz5KBBkF8mo5uV4tdAI+rX/oWE089WktLDIKZELIE4SQDkJIR2Nj5mRFxcJqoRPwaf1Dx2riqU9rYeGbLHz48OGjROALZB8+fPgoEfgC2YcPHz5KBL5A9uHDh48SgS+Qffjw4aNE4AtkHz58+CgR+ALZhw8fPkoEcqYb2KTPLS0tGe4uHlYLnYBP68UIs/y/gHkio9XEU5/WwsIPDCkCfFr/sLGaeOrTWlj4JgsfPnz4KBFkNFmwiMYVfPOlk3oi9MZwEIuxJH71+3EQAHdfPDQyswAAIABJREFU04KPbW/Gd944i3OTixibXcJCTIEoAptqQ4hE44jGFRBCoKhalYPayjKUSSK2NYUxu5RAPKkioRI9KT1gnaDcDmbJ0l84MQgBwG2pxOf/9bl3cWE6ivKAhIbKIKYXY1hOqkgoRMv0L4lIqCpkUURlQEJcUbGrpRbLCQW94wtoqirHxroQZqNxTC/GUVcZwJamcFpidbe0snj0Z6fwavcobtzejAc/vpKP/6ljA/j7X/RgJppAUBZRkarlNbeUQExZyXEdkAQEyySUiQJmlxMgBKgqL8MHLqnD9e1rbRNsm4HdmtPtuFnib7Nk3d86eAoD01FsawqDQKta0t4c1p99+beD6B1fQHN1OQgB+iYXQQDsvaQOywkFA9NRhIMyJhbjttVlVhvMePrAM7/Fz7tHUVku44rmKgxMR3HVphpsbQqjNhRA1/AsJue1mnlmyfwB6AUXorEk3r0wkzaHsimIYEZrrve6oYMm3ecr3mR6/7VbG/D+yBwWYwo21pQjoRJctakGwzNLGJiO4pNXbUBLfaXe9l17Wgx0/aJ7VF+H7H0ADM+47Q8PVwnqy9dtJc1MTTXTBpG/2kqSAEiSiKSiIiCLePKevXoH7RI+X7HzaqLe8gjiSe25/Tdvx0M/6kI8JahkSYCiEscFDN1CFgFRXKH79P+83bK6CU8r28dHf3YK33mzT7/3vmvb8ODHL8dTxwbwjZdO5pXmgCTg6Xv32VY24GvqAcAL/+mPcff3jiKWUEGglbfh+9HZH8GnD/w7FDW9TVkEVALHtctYDB24ZyoRGWkw+1uhE9Tb2ZDt5qoZTz951Xq8/O6wq/fLkgARQFIlkEUBBEBCSWcqnUOd/RHc/b2jafPOLa1WvHRaU8+KDjPw8/7CP/5Fv7IYaXVKqxvcd20bvn/kPOJJFYIA07nL45Fbr0R7c9g1X1m4Mlk4WTP5lHEKARJJFWrq/0f7phw9txBLIs48d7BrxDA5k4p3whgAkqqRbjEQClvdy9PK9vHV7lHDvfT6YNdI3mlOKMQxf1kc7ZtCPKnq427Wj6N9U5YTOqlmJ4wBQAiEcitjXKJ4/bT7TGVJhSChEI3/qX+bgc4hOm5u11a+4YYOft6L5ZXu1E8XeLV7VKfLiTAGNPpy5asrDVmqqCJyTaoMPIF5FU2r393eAwBG4kgiMnyaxJcqAEwC2GxVAl6Q5JlAY2sYgEAAKPNTA3K4vgWCIKy0KwjOaHBIqwXdBEBi4vycVQlwllas9HERQINU1RiUQtV6tUolOjuqzE0MiaGaBrmqYXNe6SaEJCLDPSS+1GDFVylUReTqppW2CZCIDP2+rHb9NkAQV7ZHRGX6AaGsfENZ3YYmnf/ce03HwgH9iamhmBqP6pVW2VN2AO0AekwfXEEDtLnkBtk8sxnANwkhT/B/0HlKQQA1tjAllq9Jt3XRWWW67kwWsgm/6RwSAhWV2ril5t300DhJLA/BZl2Z0Rof6+00ubUh0LRls4FOi3vT6GDmDW0LKX7z8z4+0b9AknFTZSeNVitYzDMlOjsqVVSt1f9qNnc5JOcm+0kytuSWr0Z6CMnqPwDHV8OzubyrmM8Xm+5ij4GX9BeDrnz0p5TGdDW2tRra970sfPjw4aNE4AtkHz58+CgR5CKQ0+xhJfpsLu8q5vPFpjufbWdDi5f05/KOYo1nvtooRNul2lbJt+/qUK+hoYG0tra6JcgxonEFfRMLBk+NqvIyyJKA2lAAoZSvLQB0dnZOEgsjeTZ0RuMKFmJJrAnKmFtKYGJB8/EUBKCtYQ1CAUmjb3IBhBh/z9TeqZPv5pVWvn2WhmhcQSQaBwDUhgJYTigYnlnSzi5MaOb7FBs7N2t1AFlTV0+Wg3X6OUhbo3X/zWi0otnueTt+282Bmrp6UtW43vG78o3pxTiGZpYAAMnZcSjRWdNDIa/XlFvka11F4wrOTizo15dmmCtO23Q6V/PBV14mzC4lEApIuj+/nQxwQysL17ksjh8/nk3fHOEbL53EU8cGTP+WlAQcuHcfAM1V5sSNOw0n3bnQSX0h5aSKhChAVgio27kA4P4/a8f912/BN1P0EWg+0vffoP1u154qixB6bndEq1OHcr79Aylfx87+CD773aMIJjU/nbgICBDQnPIrE5m+UDx+uBfffq0HKtH61P9Pn19g38XSGl7fhubPrfh2fuiKJjzxl+aulTyNX7t5Ox7+SXcazXbgaeP5LQhCvxWtweYtqLvtUcfvyjc+9Ogvoc4sAwBG/u0By/taW1tx/Phxx367XsOOp27W1U2PvYno6Lx+vbk5jIMPXJsTbW7majayil1/APT5GxMEyCoBdXupSP3fTgZkotUKGQUy0Vx1ngA0Z3snjWYLO7+ShELw4olBvHBiEPGkipRrSV7oZH0HVc5/UxCAvW316OyP4IfHL+jauySJllF4vC8i74dsRqsbB3kzX8fdm2txtG8KieSK06T2z5X+iKKQRvPetnoEZBGJpIoyWYQaj86zf2dprdrUbmBO34T1HONpPNg1YkqzHXjaMkU9srQG120lbt6Vb8wuJwr6Pq+Q7bo6P7Voe50N3MxVtzKAX3+379qoz1dwVgQBWgCU3ZzMRKsVXIVOe43bdm3EM+9cgGISKVAmadFHOpPcewdbgmWeKAoGp/p7/6QNuzfX4hsvndR/FwBc3mwZ65HVYFgJWSft00mxt60eZbKIeEooy6LmPplQCAQBuOdDl6S1uXtzLZ68Z6+uGXT87ZLlygnKxq3ZJY1rHPPgph3r8M756YzCld8lsLS5EaqCoGkmTgS5F7isuQrvnI8U/L2lgtb6SpxiNOTW+sqc23QzV+1gthN94cSgHm2aSAU60fkLQTDIpFuuWo+tTWHbOZktrUURyFZb857RecPXiMYZSKKAhz6xA+3NYbx4YpBqgXnT1lnm1YYCONwzjnOTi6gNlWE+lsSjPzuFZ99Z0Y4JgJNDs7j7e0dNNdlsBsONNmglqHZvrsXTX9qLA2+cxdjcMva11eO7v9ZCrwkB/vk353B2chFrw0HcxuSu2L251pGwC5fLYIOWrm9fa/h7JmFK81ZYTWRWSxEFAQ/fsgN37WnJSrutLi9DuKIM121rLLh2DAC7W2ovSoHs1Kz24W2NBoH8YZvcI7nkfnBDK81n83znoCEdAwA83zm4svsVBdy+ayNu37XRNJdFpnwg7G9m5gw7FFwgW23NO/sj2P9KF1iLgS4ACUEkGjcs8q/87+HT+aSLMpPSRj+IZotKADJqsk6FHHu/G23Qrv03z0wgnlTRNTxnCPtMKAS/eH8MAPDDzkE8/SV3ttX55SRYPef1nnFDQhWzcWXbz8QTg+mIEOx/pQvtzeGsFunMUgIV0QRefncYzVXlhkVUCBwpUiiyl4jGFcdmNb7/VvxwY6rj7xUCFZZqN0/r/tQZBtWCAWNoczK1UAQAd3RsMigr9P/8HDKjHYDj/pih4AKZXXSxhIr7n+zElRtr0BgOmpoqAEAUVmyfdFH/53h22xWntFlBEIAyUdvC5Hs77FaIU7BfZHbrxdu+WGRjW01wQf1jc8v6v52YXDJpQnvb6iEKAtQU3YpK8Nih03jgo9ty0pxe7R4tuEBeW1UOYLag7/QaC7GkdijrwKzG91+7TocbU12msxk7WukZBht9zq5fdndKs0GyYLMufmx7M472TWFoZsk0b4XbsxIWBRfIdGtOhcboXAyj749BEjU7cZw7VBMF4OFbdhRk27m3rR6ymE4DC0KAv/rgJQhXlOV9i5UN2K+0LIlQCdEnnSgYE/ewmfiy+ZgEZaPb+iUNKwpKJpOLE01o9+ZaPHzLDm2npGr9+E3vJN45P+1a02DRUhfK6rlccGlDJX5h8bfVWtliTVCG6tCsFksottcUbkx1bs5meFrZMwxJFHBHxyaD2c5ud8pmXfzOm3347lt9IABkSYRsopy5OYjmUXCBTLfmX/q/xzG9GNd/V1TgM3s2YWA6il+fmdR9XT97TYu+LS4EbXd0bMKTnOudLApIMpKte2QOP/jinoLQlAm81kAhAFgbDmJ0Lqb/tnNjNbZvqDbkhH78cK/jD0uS2zpMMeOXyeRipQnxWvNde1rQ3hzGwz/uxnuDs1lrGiw8dQ2yQPfInOXfCum5lE+EAhIOODSr/W5o1vaawo2pzs3ZjBmtZmcYTuy9fNZFqq8lkyquTK2p2x0K90woyqFez+i8QRgDmkcAFRLsabzZ9sFL3LZrI354/IJBS+YFkZPk2IVCbSiga8HaV1sASX2xP3nVBkM+5Ts/0JLR5muH6ooyw6Eezwc7k4uZJmRHwylGoNm5GDrB9nVVWT+byzvfOuM2KVzpw6lZrbUuhHejs4brXNvM9V7+2ukauHF7s2EdSYK2U1ahHe73jM3rBQLc0sijKAKZz2taUSbib27ermtMt+/a6KqKRT6xe3Mtnr53H144MYhfnRozaJjlZSK+sK/VVGP36qQ4EyLRuG6KEAF8umMTNtRU6HSw1Q3am8O6RsxqrHGHGmhdZQBfu/XKtAoJFHY8MNOEHj/ca+lPnWR8Gz+12702zyJcUebq/nzA8M5VowPnDzzPizEGmXC0b0o3ncYT1mvgwY9fjtG5Zbx+egLXbWvE5/a14rFDp/Gb3sm87OBYFEUg89rDUkLFQz/qwus943j99ITuknK7x9qxmVsMvX7k1ivxaFA2fBljCRXfP3IeH9venNWX1gvsbatHsEzUbWNC6ree0Xk8dug0btqxDj/44p40Gr+wr1XXrFWiadq5wKmNmPL58cO9qA0FLP2p2d93rK/Oib+59i0bGN6ZN4/54sOp4lFfGbC9LgQy0VobCqwUVgAwv5Qw/eh39kf0hPWvdo/ic/ta8cBHtznyq3eLgglktu7a94+cT/t7XCF4LeWSBXgfYcX7vN7zoUv0ki3UTeb7R87ra4mk/jOjy81Jcb5BNU/qX/n02wN47vgFPYjlrTOTGJhaRLiizEBj98icQbOm+S/sML0Y10vo0A8q1ZIz8YAd/4d/0m3gM1+Hj9emc+XvYcY9r1DoHr64PCwAd25vXcNzttdewwmt/BjRwzr+frP5d//1W/DkPXv1uoX5QkEE8lPHBvSTc0lccWuyAu+S4gXY7YpKCJ54S9OEzdxkRGjO4oSYu7q5DfHNB/iv/9G+KSQV8/DvJ97qw99+8sqsIudYzC4lwPoZHewa0QWdHQ/4j59KyMqHYXgW62sqwIO3w+XC33OTefeQzIiL0Urhxu1tifOq4K+9BktrLKHihRODabTyY0SXDd83u7lNUzm8eGIwLztjzwUyDfigNkEqlEEIVKTiwkUBggCQ1N94lxQvwG5XAE0Qy6IAAcRUYJlpchRugzpyhZl5wC78mxBNA3YbOcfD7lDPjgcG/25CtPEGgSSJ+OHxC0iqxFbjypW/daHC2y9v37URz9OdipuUiiUMN25v66vLMRRZMlwXEmuCMpIpF1YCLRKPP5Nix0hKFYk1iy+wmn9e7Iw9Fcid/RE8dui0IeBDEjXzQPfIHLavq9L9eXtG5y0Pi7wAexgGaML44Vt2oHt4FgRAe3PYMAgAdMdvM9tULierbmG1hdp/83adh2+fm8Ir7w6DAAiWmS8etzRnOtSzao/XMOjHbWhmCc+8PWA5odmS7zSEmtqf3WjJW5us8454BXo4fLRvCl/5++FMtf1WBUIBCV9j5pjd3NnWFDZEuW4r8BiEAhJ2ta/VzaDJpKoHGQHQ1y8dI3aNWyld/G9e7Ixdp980g1U8N1seXsCKMKa22iNnp/DwLTsAQLcrvnN+OqtwWbfO9nvb6lEmaZokDT5pbw7rdDz3zgU8fMsO3H/9FmPwhSgAgmCIhfeaVjPazVzIaGjov/dO6rsOSRCw/+bt6Bmdx/5XuqASe200E63tzWFEonG02yRX4mGlYXT2R/TcJJIkYmhmCZ39EezeXGso+U7t1XyJdT50lqU10LziU7oYSzqmNZ/wMqq0UGB5um5Tq+N1Gg7KtteFoHWuZ1z/mwotyOjYuWmAEH1X9oV9regemUNtKOA6b4oXO+Oc02/y22eq/bx3YQbLiZX48A9tbcBNO9bhYNeIwXa7/5Uu3PmBTTmr/lk52wuajiyKArqGZ9E9PJuWSwGAgeaEQkBj4ezcxfjcqjnTysDKhYzSqBBA0TPTEXQPz+LZdy7oZqOYjYuPHa1X7Lya3PnEESQVAlkS8Oy9+3LyH6X9ePHEIH54/AKeeXtAt8XxrpFPvHkWH9zSYHBTsktrGly3VefrmxehP7BT5JpnmeVpS/uVhPI/0xw69PvxtGuvw9d5WhPcWQobPEX7QL2o+ENqO/AKaD53xjl/tnh/VjbslYJAc3WjX1eD7TZ1b6EPxeghGBWyTx8bQJkkaNW+mVwKVKukh3uiSPMMW7uLuUmCki34ibC3rR6iKKTlAxFELW0pG9xCkJ0r2MR8DJWpSZ5UCA68cdYyQb1TsH7H7Af5ph3rDK6R/VNRDEYGDG5KRFUcqb5LRdKQSxXZCmlVXamKTKC5iVlhhvPa4a+9BksrhQhAlkWAEF1GsdZ99pDaCl67uOYskNnts5DKG8qrfAK0UFIquAXoyikCZWJaqrtC2GL5nBoEmgD+08ubcPj349qAiiseAaIAfHBLA1rqQnrVECt3MTdJUPKF3Ztr8aeXrdWzuVFsX1eF23dtxHOMhixY0J0JdsmFcoGZCYbOgSfePIv+qWhqfFaeEQVAECVn83eV+gGbCU4rFKK6yFJCMXjZ2IWH14QCmFyIG64LCZ5WAPjg1gaDDXl+KWGIM3ASgeu1i6srgRyNK/jmSycxPh+DAKAxHMRiLImAJKJMEvGRy9bi1e5R3VRBQb+m9GtEAFSWSQjIIq7d1miIJX/q2IAe0HDXnpasI+CscqGyeRx2bqjGqdF5LCwn9a3+m6cnIArAhroQQgEJv0/ldCUEGJlZQn1lAKKQcpERtBSUb/SMI5ZUsa+tHuGKsrSAh0wJ6u36yGaZ4rd8DzzzW/y8exRBWURNKJAWjg5oQvPrz7+HqnIZ01FNoxFFAe9emMFTxwb0Q0wnUZF1lQGDl8WdH1gJw3YyRqwvMuuxQk0Xf/PySZybXMR/fe5dNIaDiCzGsTYcxPDMkqE4AIF2HuG0CsOfbW92clveQfvrxQ6pGMgUOs/irz54iX4GQK/zATtTIAueVlEABqai+Py/HMOWxjUIV5Rh+7oq3HBFE8bmlnHnB1oMkax8oJiVC1xtKJB1BKkZXAnkvomFtMQ7LF5+dxjXbm0wtdm9O2h0wl6IK0BcwcvvDgNYCUtmbToDU4uGYA2n2wOzXKgP/ahLz0/x7PELIASm6T5jKXtE/3TU8DsB0DuxiN6JlfMZlRjzJb+X6mN5mdFNzi4Jip0DO59lCoAulB945rc675YSKmaWzLflWuh3zPCbomp5kVlt+vnjF/B0qmahFYZmlsAuwb9+6WTaQZvVGPGHvKJgdMD/wZHzeH9Ek6/np6I4P5Xi/8QiRCHdZzShEAhyMN2B2QQvvzuMxz5ztZNb8wZ2a8uXG/MCbrTpbMGP/zdeOmm5xX/kp++nXefqPeXGFMjTqpKVNU1l0VtnJiFA80ICYJo/2Sy/N1vMwuyeXMCbWWzh5PTJKqtTJiRSYYksaLgin280ExZiybR6bqyBP6kQy9zL+UAiqSISjeP+67dkHCCeVraPZvygeP30RH5pVohj/lKoMN/CmYHep9uAufvt+mM1VGJ5ZXFzn9qAy629So0m2WMhrtheZwMvTIE0+pav+WhWA5Ji9+Za3H/9FkSi8azkkx0ENz7rUkUVkWuaVnpiMs3U5YUpsXxNhmqU3LMEAIiqLM2NS6FqfX+pRGdHpYqqtam7SSIyfJrElyoATALYbFWuXJDkmUBjaxiAQAAo81MDcri+RTuxw4qjPr12Q6uT+0HUFK2LABoAVDqhlenjIoAGqaoxyPNDmZsYAgC5prnVls+u6SYkERnuIfGlBita9fGnbaeeSWmAPP0sGoRAxZJ2nyCuOICv8CmtPyz9Fn2JT/QvkGTcdFFKoWoiV6/Vn4+P9Xaa3NYAbS65gaNnhEBFJeVLcnZcUJfnv5zyAjC4ZwFoB+DUTzkbep2Ctm25rqRQFZGrm1Z+sOFr2dq2jYIo6kUYiaoqifG+d3Ogy8BTACQ+cX6eKMkaR7RaITUPk/NTF+Rw/SbaNn9tNq95eqzmPjLw1UgPIVn9B+D4ang2l3cV8/li013sMfCS/mLQlY/+lNKYrsa2VkP7rkwWPnz48OHDO/gC2YcPHz5KBK5syA0NDaS1tdXx/dG4gr7JBRCi+R23NaxBKCDZPjO9qOU4oNhQU4E6k1yqnZ2dk8TCJuOUTiv63P6eCfmg1W0fsu1rbOzcrJVdLlxTR+IVK2Zeq7HJFzL1zY6vrA3ZazrNwNKenBkDa0NmUVNXT5bL61zPqWzoMGvfzfjnOlfzDS/XlROY8RaA/htFJr6ycOX21traiuPHjzu+//HDvfj2az1QiVb25P4b2i3rVlF87p+PGSK0Pri1wbR+nSAI/bnSaUWf298zIR+0uu1Dtn3t/6fPL1i9K1DThPrP/J1+bTU2+UKmvtnxVa5ei3Wff6wgdJqBpX3k+18lsaW5NGEMAFWN61F326Ou51Q2dJi172b8c52r+YaX68oJzHgLQP+NIhNfWXia8SObbEh8yKyX9eus6HP7ezHhlCanfbILtnATGJAP5IvfxaiByNIOG49RNyktc6XDSS5vp8E2TpFrLo1ShhVvA7KIeEKFCs3f3g1fXZksOjo6iNuvTjaRdnzaRTMIgtBJCDFNpOCGTiv63P5uh3zR6rYPTu9jf+9orbOl9WuPv1jQNKl2fbPja9vlO8kH/9t3C0anGSjtX7lx5+/VWNQ0s05HRwc58MIvPE0bkGl+uBl/t3PVS4Hs9bpyAqtMl3xEqh1fWXgukL1CKQyGU/i0eoPVQutqoRPIP60Xu0B2CjtaWWT0shAE4V5BEI4LgnB8YiK/0WH5xGqhE/Bp9QqrhdbVQudqw8XA14wCmRDyBCGkgxDS0diYOdDECWjVh87+SOabHWK10Al4Q6tTuO2TG1q94pdTsLRWVtcVlRY7FHr8iz0uhUImvq4GPhSs6jSF1/lE84XVQqcbeNmnUuNX3+QCvv1aT0nQUkyU2rgUC6uFDwUPDHGajKbYWC10uoGXfSo1fhGSnsDoDxGlNi7FwmrhQ8E15FJ0HTPDaqHTDbzsU6nxS6sniJKgpZjIdVzM6j+uRle2UpufVii4QPaiMKAXWC10uoGXfSo1frU1rMH9N7SXBC3FRK7jQnKs/1gqKLX5aYWCC2TAffn5YmG10OkGXvaplPgVCkh5jXhbzSilcSkmVgMf/ORCPnz48FEiyIuGbBVZR+vBXbWpBtG4ghMDEcSTKkJlEiLLCSSSKmRRQGM4iD/fuR7hijK8fGIQZycXEZRFtNZXIqESXNJQifs+fKmhzlWmOmVmNfX4628dPIXe8QVcunYNbr16I17vGcf7w7OYiSawrCiorQhgQ00FeicWoKoEcYWgoTKAlvoQphfjuKRxDa5vX4vu4VlMzMfQEA7qtelYnjiFmyhAvv3/c/gMZpcTKBNFLMSSkEStgnYiqSKhEgQkARUBCZIo4tqtDdjaFMaZsXkc7ZvCproQHrzpckfaw5Zv/BRJFZBFoPcRo93QjucA8K2DpzAwHcW2pjAIVkKaD3aNYPu6KvRNLqJvYgF1lQFUhwIYnI4iEo3jk1dtQEt9JQ52jUAAMDAdxY0Z6uSdHJrVbZ3Fsm9SXgWaLt1ld9+O/a9iIa5gTUBC18M3AtDG919+cw4gBDs2VGNqMY7t66r0mo1dw7OYnNdKcxEAa8NBhIMyukfmcNOOdWhvDuv83ttWj61NYcwvJdA9Mqe3M7+UwKFTY1hKKNi+vhpf/vClnvOk0KAyqKUuhPdH57C4nER9ZQALcQU7N1RjYDqK4ZklXLGuCgFZxMB0FJ+8agMe/PjlhjX2r7/uw9nJRVzaUImPXN5kkGu0Jl8+IkJzjtR76tiAoZjhI7deibv2tBjqweUDsgg8fMuVeg2roX/9qhob7TVNi3XFzquJessjlvWx9t+8HX/zSpcnZZwCkoC/+uAlhr5f+Me/6FcWI61m91OeunHL4XmeD0gC8Nx9f2wb4lm+bitpTiXsAYxCmaef5bksiUgqqmUppmwx+P/9x9Hk7LjpFy+4bitZx9BaaKFMhTEAjPzbA4iNnDGt31Kxfhtp+su/16/XBCR84z9ckfP46kVZXECWBPT93adsw7yPHz/u6lCvmJF6n3r4B1nLIKvaoHagss8MeYvUy4SDXSOm13w9uFyRVGGocwWbAkVmNfX4a69q6iUUktZ3J7Xf3Ljl8DzPBxSCjK5APMeSTKYhnn6e516wWwyWbk29pJr5HgBQOYVoIa7kZXyzYXdSIXmpU1cqyEUGvX1+2vUz+Ri3nAUyvyWn15m2lG4hi1rbAVmElKqMZ3XvmqCs31cmi4bn6LUkelN3skwS0vquLi9mDA2ibjlOXLW8yF4mCcjoCsRzTGZmD08/z3Mv2K3GMvO1WJAdriyRK+u4JiDlZXyzYbcsCXnP9lZM5CKDrmmtc/1MPsYtow3ZzA+RBVXReRsyLVefbxtye3NYy6D1v4dP29H5IufiQp9jr720IVOb50071uHub81M2tEKuHPL4XkOeGdD5mmVRZjakM3oZ3kO5N+G/P98Syv4akZroHnFHloMG3LvI/9hxWzB2QV5nq4JSGk2ZABFsSF3PJJWpHPVgOcrK4P+YGzIxcLFkunJpzV7rBZaVwudgDNaV4sNebXwlYXv9ubDhw8fJQJXbm/RuIJvvnQS4/MxCAAaw0EsxpI42jeFlroQvn6TtkX4zhtncW5yEYmkiuloDFUVAdx/3Ra0N4fxwolBfasPbxjkAAAWhklEQVTVEA5ix/pqRKJx1IYC6B6eBQH0bT+QXUJ4K/Bt0S3J9nVVODEQQe/4glb/KqliV0stlhIKzk4soikcRLhcRiyp4s4PrJgL2C0k7QNNSO2GVrs+UredG7c361swigee+S1+3j2KNUEZ/+VjWvmY/3P4DCbmYxAEAQ1rArh553rMx5I4MzaPWFLFvrZ6hCvKsuanmdtbvsboqWMDePadAQRlEQRAJJpAW0MlrkuZhQigz5dM9u5ScHtzClaLpLSy22Xe3EaRa2GCix1mfC11uBLIfRMLePLYgOnfRudiuOM7/w4B2ok9i/nlJXzjpZOQREBxcPr8/PELePrefQCQtwxNvFvWF/a16i4xb5m4t7AuL9OLcf3f7w2uuCOxz1E3IwFAsMw5rdG4YtlH1nWQ/p8K5Qee+S1efncYALCUiJu4SREMziynuf28NzgLACh3QSNF19AsmlPjl1Q14fzsl/84L2Nk5crXO76A194fM/xGeZzJF52i9cGfluyCPDk0C/YoqPXBn+KRW6/UefHWmUntsE0lBv46dZNcLVnO8g0zvpbqHGDhymSRydqsknRhzMKJMAY017GjfVN5zdDEt5VvtzzC/N8NrbyLHvscTyN7/frp3BJwZ8NPM7e3fI2RG5chyuOLyUWLBc+LpELS+OuU76sly5kVWh/8adp/FzNcaciZXGlEAaYaMoWVhsw7sZdJgqFgYD4yNPHZnm7c3pzXwBXaBxHuMozZFbnkaWTdeK7b1qhryNnQmlXmL+5aFvNbiNRsp2IGyuOLyUWLBc8LWRJAVJJVwd3VkuXsYhe0TuHKy0KQ5Bk53BCHKJWBkACIughBEIWyijBRE3FlfmoQAKTK2mZBKiuHAAGCJIOoSWVxZpQkY0tSRVU9BLESghCHqiTURCwqiJJMVCUplgVDAKAszU2RuOZ+IwQqKsVAKKzGo/Op3xoATALYTAgxLbcgCMIEAFoinN6f1pYYqmkQyytrSTIeFcqCawSprBwAAQSRJGMLEARJkMoqoCoxQlRVgCAoS/OTgBbsQZLxKFRVIaqSpH0QRElmaKXvr7SjVQhUTHJ91OmWqho3iMHKWjW2GFHmjG5eck1zqxisrCVEVZSFyLDG+5p1gijJhBABRE2oscVpQRAlyGUVAgRBTSzPQ1UV7l0sbPkaaLq0nmjZLUl87OwJizHioY+BHcRQTYNUEW4gIAQEZYIoEaIkltVYdJbODTpfUu9qsKJVClURuboJIEB8rLfT4pWO6MrDM5sBfDOVOc0AjadbGugXndJK56a6vBghydiSGAiF1diiRBLL+hxwwHc395muK9aVDEA7gB4H/c2GR9m0lWGupvPVZfv5QEZ5ZQAhJKv/ABxfDc/m8q5iPl9suos9Bl7SXwy68tGfUhrT1djWamjfd3vz4cOHjxKBL5B9+PDho0Tgyobc0NBAWltbvaMGmhtY38SC7kLW1rgGoUB6UrfOzs5JYmGTyQed0biChVgSa4Ky/v5oXEHf5AJIyoja1mBOW6FpdQonfYqNnZslSrLGita16zelteElvXb8tuNrTV09qWpcXxA6zcDSnpwZg7o8/2ViYkMu1JqyGjO341+ouWoGXjbExkuXVh52c5WF61wWXocjfvOlkwZf51v3tOB/3HqlGV393HXe6KS+m3JShSqLOJDy3Xz8cC++/VoPVKIl47n/hnZHVSm8pNUpnPap/58+v2BF67pNrVBveSStDa+Qid92fA02b0HdbY8WhM5MtI98/6sktjSnC+NCjr/VuJvRmWn8CzVXrcDLhqHvfDFudW9rayvchnl7CX6uWiGjyYIQ8gQhpIMQ0tHYmPmQMFfw+rpT/T2fdFr5brrJyFYoWp3CaZ94VzKWVrliTUF9Wt3ym6WVoLhVp1nawU3jQo5/Jj9kN+NfqLlqhVVb0M8FPK+pR8M2nYYV375rI54/fgEJhUAUtFBZL2FW7YT33awNBfD44V7sbasvyUKJVqGxnf0RvHhiUA85NvNH5bO0dfyttfvUmqCMpCggoRBIouDqg+QmfJe9N1t+U59pwUFaUS+we3Mt9t+8HQe7RvDU/NQFu3vdhja7uX9vWz1kmzFzM/5uYVVJKFvcvmsjnntnQA/dV5bmVleUiwN4KpDpdimWULWgCQF6+CYA00m1e3MtHvrEDux/pQsqIXj4J91obw57IvzYcF3qiH/XnhbDJJ1fSmB/qroIDYkupeKZVqGxnf0RfPa72u+AVsnkoU/sMHwU2YXtuE9CyrGTy+ObDY30b3zZJ/7ebPhNtamkCvSMzhf849nZH8FDP+pCQiGQw/WW0sgudN6qXae81JFhzLwo/mm1tnKFKIoQVBWieHH6I3jaK7pdoouDbpteODGIu793FN9+rQd3f+8oOvuNecYj0ThUkh4umm9YVTsBtEm6t60e3/v1OSRVAgIgnii90FOrLenRvikkmLIVCYUgEo3j/uu3GASf1RiYYSGWRFLRxlNRnPPCikYzGrwI9fWiwkomvHBiEHFFmzcQrL9edqHzZnDDS3p/NmOWK559Z8D2OhvwfbkYQ+c9Fci1oQBEQdC3j2LKTiUA+qSKJ1U8dui0PoE6+yMYnlmCLAo522ozwaraCaXjsUOnkWRrDwlan0oJZnZWykNJWpEDAoy0ZyP41gRlvfKHG1OAlS3YjAb2XkkUMDyz5OhjYQcvKqxkgtP9A1/dhvKmsz+Cxw/3pvXdDS+driWrd+WCpqpy2+tskMnefTHAM5NFZ38ED/+kGyrRbFf3fOgSPe0jkNIgUhPo12cm8c756bTCmHdes8mQijPfsKp2QrWN5YQx8YZK4KkJJRvwNkBgJUOeJAp6jg0VwEM/6tJpzybHwXJCgcpke3NqCrCqhmJGA733hRODeL5zEE+/PYAXTgyuuixl2x2efYQCEg5wvLEzSzjlZW0ooLdht5b4dznNoOekX3bX2cBLe3epwDOBzH6xBRCEK8oMtsAn79mLrz//HnonFjVzQKowJrU3x5OqnjeZHqh5sSDv2pNeeuVbB0+lCWMKarYwm9j8omKFZL5h1f6BN87qtBO6ZU4hoRA8dug0HvjoNlcloyhmlxJg94gHu0Yc2wV3b65Fz+g8Hjt0Wv/4WdGwe3Otvj2lGt+LJwazPkx1Q2e+EIkyHlkZ3AN4G66Ztsufs5jZjVlevnBiUJ8HiaSKDTUVaXx76tgA/uGXpw335csM8O6FGdvrbGFn785Ubm41wDOBnEkD6xmdR+/EygdOJUB9ZcAwd197fwy/6hlPywXrJR792Sm8fd5666Yi3WzBaxmspp9PrcPsfbIoAIKApKJpxAkm1Z6ZCyHdjVBeuuEnX6l7finh+Fm7A1QzGtj5I0kifnj8ApJZzgNvytna4xhrAnJJgNPdi9UBaGd/xGCzJUgfK7P805KUPzPATDRue+0FUoE3TwBaCSfPX+gBPBPIZtoP+zU3O2iZWkwftGRKwJhpCvkEpe3ld4fS/tZaH0L/VFSPEDrYNWIwW/AazcGuEcN1vg8fDO9TCABNE1btklGnwOZrdsvLOJc79fx01PZ+u/HOpLWy82doZgnPvD1gqTFmwkAGOr3A74ZmHd/L766c7l6sNOmjfVNpaW67R+YM12br71O7N+L/tcka5wZzy0nbax/m8NTtjd9a8RU7DDlfRe3w5UjflC6EWbj1e3UDljb+PFwSBdx77aWaxptQoQL4Ta9Ry+Q1mpt2rMM756f163wfPhi0x5SGrCgqRE5D5vNM0zO+bA9KK8qMdsCdG6ztpJnG28lBG50/1J8625y+V20yja71FDs3VBuqzljByu3Nye7FSpPe21YPWRIM68jsAJvPP51Pn//6ygAmFuKGax+Z4XlgCMWLJwYN9uHukTncd20bjvRNYW1VOe778KV66fgDb5zF7wZnMDYX07XSOzo2GbTsfJgBaFvDM0u6piEBuKa1FgPTUb1OIF0Y//DL0xidi6VpJGYaDVsHzenhg1OHf7ODPBoAUhWUcaRvCgFZxIXpKEbnYvpzOzZU44btzVnb4yuDxumyx0Yw8tpbuKIMj9x6pV6LMBKNo7M/4qhGXDb2bhZbmwrvHbWnrX5FINtsXBZiSS2sOeVx5FT7p7zaf/P2tICr3Ztr8ey9+3DgjbOmJeo7+yOIROO4prVWN8+JAmf3zhGXravCBCPwL1tXlbe2L2YURCB39kfww+MXDP7IvC2TxR9tqsF17Wvx8E+69a//bbs2GrSustr123KlibXDypIIRdHexQpheu9DP+pCnNXcBcFgS+Y1Grf2Wbe1z/jdB/VaYW3Y/MEkvzDdQhYFsC3auQBaeVC0N4ctA1loEJEkCnj4lh0GWnMJXnBj684XDLyxsSHLogBqmleJM7dKJ3Nl9+ZaPPGXHWnPvXhiULfHy6KAgLwy7/O5A+W7XAw7/mpEXgRyJs3uaN+U0Z8XmtKwnFDx9Rd+h2/dvnMluuyJI0goBGUmkWWPH+7VtS7kOMasBpdUCT5zzSZ9i98zOm/QPh87dNoojKEdcOXTBS7TybodXmB2H7GEin/45Wk9Qo9CEoGu4VmDVuo2ZDepGhNo8xoV356ZVmvVT9YrIKkS/M0rXegans2L2yNvPy0Euoad2ZCTKoGElfJfLE+tQo9ZHsYSKr7zxllctanGNj0BHzULaHP4zms2YUNNRd69mE6Pzdte+zBHzgLZydeaakvxhGajpYlfAK2q8GeeOIJn7t2nRzcBQFwh6BqexSNMpjdW60KOuUZqQwGDZlIVlPEvvzmHRMpVTIBW2w+CYIh4Y5HPg8Zsa5919kfwfOegocgqNVOwNmRFBZ4+NoAXUz69AAwfv6fv3ZexHyonkFnN02oeWM0Ftp+0DywUleCpYwN47p0LadqyWxRDO+t1KIDWBGUkJc32LzO1JO1Cj2l+ChoJ+Iv3x3Do/THbqud81Cytq+iVn/9iLGl77cMcOUfqsV9ragOjoBFAALD/5u0QRQEEAJ+CmVaZ5hfO5HzMEEFEta6v3dCORGT4tBs6+WikSDS+EkEI4Ejf1EqoK1LeCApBgpnEFJIoaAVdObNFLmD75sati/rrmqGpKgiZGWFqv3/xxCAe/nG33t+4QvDCiUHTNlgsJRTDNat5Oon8Y+2ebD/t+kC15VyiyIqhncUsPuJmUIg2NuwmjPeCeOLNs4Z1cEfHJsN6Mcxbi6xucipQSJYEfHZPi6dupPxc4a99mMN1PmQevKZJBRRvfrijY5Oen4IHW2X6uVSmN0nUSt0fOjVmevr8nzn3HJ5OPnCC1972ttUjWLaiqWmhncZtppjS5vkPyJ/vXIef/G4Eikrw0I+60D08i9tcaBpWPM3GTko1TrNAlrG5GERR6wc7Rk+nXMgMNDmgtWp9m+Fv7Mm5drK/4jfMa/i8zf6Ojk1pfYgnVQjQ+HC8P6LTqKgEB944iz/aVGO7tWZpDTQzeZMt+uYl9rXV471Bc7MFS2d4fRvqUh2l/XziLzvSvCD6p6K4+3tH9XVw266N2o4y5fljV/WcnjHQIRcFwbFmnG2wRUCWkIwrhmsruDWd5YpSyZFshowCOZOzNdU0eRsYb34Yn49BlsQ026YA4L9/YgcAzVNATU1OQmCI1MpkGmDpvGLn1YQVwLft2pimvd1//ZY0T4VDp8YMgkol6XYRUQDOTS5CUVe0yyePDeBZF1vrfDqw795ciy/sa8V33uxLfw+Q5o8KIE0YS6JgGerL0lrTcpnhyXOTi/zNxv8zMOykFM0cwYZE7795u55V73dDs+jYXGsI0PnlqTEcOjUGSRRwxboq3PmBdD6ztAbXbdWJmDbxb/ca8zZbdJbO0PptBmYdO6dptnQePfHmWZxP+cCzUaKsjZ7ajs1syKa2Y8W5qS3buRpLKrbXFG4Psy925GxD5jVNKtx4rWR8bhlXbaw2jYLrHp5N8wpQiSYoJBDXJ8CsK1EipXWZ2S1fZLSG3Ztr8dHLm/Da+2N6O/zs07Z7IrpH5tL+llQJ9r+ykiuikKHTmQ6tWA3Z8DsAQRSgMgeUdiiTjBYuNmEMPbjVPgIkbcFTLZgKBrq1piHRQzNLUAnRf9/SFMa7F2b0vNj03EFVCN4bnMV7gychhmoabAlOwUG8TN7h9JUq9/Gia4C6pt24vVn/2KrQQpDpwayTHZWV7djrHNH8N9mqUlwuh9mFgJk2bYZ8adiuBHI0ruCbL53E+HwMAoDGcBCLsSRCAQkVa4K4/zptm/il/3scp7hTZuvtG9A1NIuYyZb7I5etRWM46Pr0bk1QhsoI4Nt2bdS3eAI0L4qHftyta+vPvD2A9qYwBiMrEV18UAUA1FYGsKWxEu9YhFYnVaLnanATOm23ZXv0Z6fwavcobtzejAc/frl+P5v8qEK2PwogRDML8UEjH72iCYdOjVnaHXmEy2WD29t17Wt1eoZmlgyug1aJ0GnSIEUxhkTLkqh/xFUC/LY/grXhIMrLJOzYUI0f/24kbVVLFWFHAlmWCm+0YAst0HrwZuA/cs1V5ejsj+AzKXOfyJF+6P0xvHVmwjanOAujKUnAnR2bXJnXeLACyk4IBSURS8xuOCiZz9FsD7PzDaeC12u4Esh9EwuGmlYGLCbw1y+fhAB3GolKzIV1mSTovsj0IMrpdsYqg9aLKV9dURAMbngqAU6NGg9+zLowvRjH2xm2v2+dmcSRs1OGfM52odN2Ccof/dkpXTui/3/w45fjaN+UYTexlLQ/QKIHlCwEaAL1zTMTjhfD0MwS2Hivv375pMGvWBYFfOaaFssFTzW623dtNA2JZg+m2PFgc54Y++WsQq9VoigvsXtzLZ6+dx+O9k3hK38/3GN1Hx+O3j8dxYE3zurjxe9s2J0F63tuuzZSbBKBnITxyaFZw/i3PvhTS6FcEZSxlIwbrs2Qa9DPxQZXAjnT7DfbFmeLOzo2IRKNZ72d4bdz7NbI4TrOGqpKIIoChJS5xS50mjevsH18tXvUcO+r3aN48OOX502LiETjOS0GlRj5qqgE602yivEwC4nOaliSiSWXTxQUVgfQmTA2t2z5N2pyoB4zmdZGJlOSV/j07o2Gc41P795oeW8uQT+lAisN260pQ7DZTaVBqqgick2TdkGdHlkQACDEskICfYZ/Nu2akERE0ypSEXkCAJKIDJ8m8aUKAJMANluV1RYEYQIArfLaAGBSCFRUsm2pscWIWL4ms2Qz66ddnzQeqMn5qQuCKMlqbFEiiWXZklZJngk0toa5Pi4CaJCqGoNSqLqZ3qtEZ0eVuYkhAJDrNmwRAxXWyQfs6E7RyLyLhSVf9fHX+6qNk8kY8W02QBuz9P4HKirFQCgMUZTYvjqgn8Qnzs9ZlYGXQtVErl6r3x8f6+00uc2SLhtk88xmAN9MHZBxdFYRubpp5QcCJOcn++Wqhs0rv2mLlAAgywuTtJacA76Dn/dW9zH9sh5/E1rt+CpVNW4Qg5W1amwxQudtFrDjt4FW1iMEQDsAy52Jw/bzgYx8NYAQktV/AI6vhmdzeVcxny823cUeAy/pLwZd+ehPKY3pamxrNbR/cVYK9OHDh49VCF8g+/Dhw0eJIBeBnGYPK9Fnc3lXMZ8vNt35bDsbWrykP5d3FGs889VGIdou1bZKvn1Xh3o+fPjw4cM7+CYLHz58+CgR+ALZhw8fPkoEvkD24cOHjxKBL5B9+PDho0TgC2QfPnz4KBH8/8CFgy9y5IFFAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "pmax = min(p,8)\n", "k = 0\n", "plt.clf\n", "for i in np.arange(0,pmax):\n", " for j in np.arange(0,pmax):\n", " k = k+1\n", " plt.subplot(pmax,pmax,k)\n", " if i==j:\n", " plt.hist(X0[:,i],6)\n", " plt.axis('tight')\n", " else:\n", " plt.plot(X0[:,j],X0[:,i], '.')\n", " plt.axis('tight')\n", " if i==1:\n", " plt.title(class_names[0][j][0])\n", " plt.tick_params(axis='x', labelbottom=False)\n", " plt.tick_params(axis='y', labelleft=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Display the points cloud of feature vectors in 2-D PCA space." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAEoxJREFUeJzt3X+IZeV9x/HPZ2d3k7QpZNndolVn1yVSEqw13cEOtH+kianbkJoaEBIkhAaZCrEk0NBUF2pSCRSsSUsipVuVpGWbNGBESQxVwWDzxxhnZK2a1bBdsnWNrWazkthA1sl8+8e9U2dmZ2fu3Pvc8zznOe8XLHjdu+d+74/zuc95zvc81xEhAEA9tuQuAACQFsEOAJUh2AGgMgQ7AFSGYAeAyhDsAFAZgh0AKkOwA0BlCHYAqMzWHA+6a9eu2Lt3b46HBoDWmp+f/1FE7N7oflmCfe/evZqbm8vx0ADQWrZPDHI/pmIAoDIEOwBUhmAHgMoQ7ABQGYIdACpDsANAZUYOdttvtP1d20/afsb2Z1IUBmCl+ROndccjxzR/4nTuUlC4FH3sP5f0roh41fY2Sd+x/a2ImE2wbQDqhfp1d87qzMKitm/dosPXT2v/nh25y0KhRh6xR8+r/Zvb+n/4IVUgodnjp3RmYVGLIb22sKjZ46dyl4SCJZljtz1h+4iklyQ9FBGPpdgugJ7pfTu1fesWTVjatnWLpvftzF0SCpZkSYGI+IWky22/RdK9ti+NiKeX38f2jKQZSZqcnEzxsEDrzJ84rdnjpzS9b+emplL279mhw9dPD/Vvcxn2uWJ0jkg7a2L7Fkn/GxF/c677TE1NBWvFoGu6NE/epefaJNvzETG10f1SdMXs7o/UZftNkq6U9Oyo2wVq06V58i491xKlmIo5X9KXbU+o90XxtYj4RoLtAlVZmid/bWGx+nnyLj3XEiWfihkEUzHoqi7NO3fpuTZl0KmYLOuxA121f8+OzoRcl55raVhSAAAqQ7ADQGUIdgCoDMEOAJUh2AGgMgQ7AFSGYAeAyhDsAFAZgh0AKkOwA0BlCHYAqAzBDgCVIdgBoDIEOwBUhmAHgMoQ7ABQGYIdACpDsANAZQh2AKgMwQ4AlSHYAaAyBDsAVIZgB4DKEOwAUBmCHQAqQ7CjKvMnTuuOR45p/sTp3KUMpG31oh225i4ASGX+xGldd+esziwsavvWLTp8/bT279mRu6xzalu9aA9G7KjG7PFTOrOwqMWQXltY1OzxU7lLWlfb6kV7jBzsti+y/Yjto7afsf3xFIUBmzW9b6e2b92iCUvbtm7R9L6duUtaV5P1MuXTLY6I0TZgny/p/Ih4wvavSJqX9EcR8b1z/ZupqamYm5sb6XGBtcyfOK3Z46c0vW9nK6Y1mqiXKZ962J6PiKmN7jfyHHtEvCjpxf5//9T2UUkXSDpnsAPjsn/PjlaFVhP1rjXlM+7HbNsXbG2Snjy1vVfSOyQ9lnK7AIa3NOXz2sJiI1NUHCHklyzYbb9Z0j2SPhERP1nj72ckzUjS5ORkqocFsIH9e3bo8PXTjY2gcxwhYKUkwW57m3qhfjgivr7WfSLikKRDUm+OPcXjAhhMk1NUTR8h4GwjB7ttS7pL0tGI+NzoJQFos6aPEHC2FCP235H0YUlP2T7S/383R8QDCbYNoIXadhK7Nim6Yr4jyQlqAQAkwJWnAFAZgh0AKkOwA0BlCHYA2bGWTVos2wsgK65UTY8RO4CsWL44PYIdQFZtW265DZiKAZDVeleqskrkcAh2ANmtdaUqc+/DYyoGQJGYex8ewQ6gSMy9D4+pGABFYpXI4RHsAIrFKpHDYSoGACpDsANAZQh2AKgMwQ4AlSHYAaAyBDvQUSyVWy/aHYHKDLK+Cpfr141gByoyaGCvdbk+wV4PpmKAigy6vgqX69eNETtQkaXAfm1hcd3A5nL9ujkiGn/QqampmJuba/xxgS5gDfN62Z6PiKmN7seIHagM66uAOXYA2dByOR6M2AFkQcvl+DBiB5AFv5A0PgQ7gCxouRyfJFMxtu+W9D5JL0XEpSm2CaButFyOT6o59i9J+qKkf0q0PQAdQAfPeCSZiomIRyX9OMW2AACjYY4dACrTWLDbnrE9Z3vu5Zdfbuph0QH0QgMrNdbHHhGHJB2SeksKNPW4qBu90MDZip2KYRSGQdAL3W7D7Odkw8ZStTt+RdI7Je2yfVLSLRFx17DbYxSGQQ26miHKM8x+TjYMJkmwR8SHUmxnCT8CgEHRC91ew+znZMNgilwrhlEYNoNe6HYaZj8nGwZT7HrsrCkN1G+Y/bzL2TDoeuzFBjsAYKVBg73YrhgAaQ3STULHydna+JoUOceOdujyIXHbDNJNQsfJ2dr6mjBix1CWPvC3P/icrrtztlWjmZqda3Q5SL8/1wScra2vCSN2DIW2s/KsN7ocpJuEjpOztfU1IdgxlLZ+4Gu23pftIP3+XBNwtra+JgQ7htLWD3zNNvqyHaTfP+c1AaWes2njdRK0OwIVKTUcN5LqJGVbn/+gBm13ZMQOVGSzo8tSgjDFOZu2drCMA8EOdFRJQZjinA0n9F9HsAOZ5Ro1lxSEKc7ZcEL/dQQ7kFHOUXNpQTjqSUpO6L+OYAcyyjlqrjEI29jBMg4EO5BR7lEzQVgngh1DK6Wjos1qHDUjP4IdQympo6LtGDUjNRYBw1DaujgS0AUEO4ayNDc8YW1qbriNa1sDbcNUDIYyzNxwF6ZvOO+AEhDsGNpm54ZLuiBmHLrwxVWbWr+ICfaOKOEDnLu1b9xq/+KqTc1fxAR7B5TyAa69ta/2L67a1PxFTLB3QEkf4Jpb+2r/4qpNzV/EBHsH1PwBLk3NX1y1qfmLmB/a6IgS5tgBjIYf2sAKjCSB7uACJQCoDMEOtAhX7mIQSaZibB+Q9HeSJiTdGRF/nWK7wLi08ZxDjrbVNr5OSBDstick3SHpPZJOSnrc9v0R8b1Rtw2MQyl9/ZvVdNtqW18npJmKuULSsYg4HhFnJH1V0vsTbBcYi7auTDnswmvDauvrhDRTMRdIen7Z7ZOSfnv1nWzPSJqRpMnJyQQPCwynrX39Tfddt/V1QoI+dtvXSroqIq7v3/6wpCsi4k/P9W/oY0duzB0PhtepLE32sZ+UdNGy2xdK+mGC7QJjQ1//YHid2inFHPvjki6xfbHt7ZI+KOn+BNvFEGiHAzDyiD0iFmzfKOnf1Gt3vDsinhm5MmwaXQwApER97BHxgKQHUmwLwytpFUcA+XDlaUWabocDUCYWAatIzcuQAhgcwV4ZuhgAMBUDoAp0hL2OETuA1qMjbCVG7ECH1TLKZV2blRixAx1V0yiXdW1WItiBDdS6XkpN1z3QEbYSwQ6sI9eotokvk9pGuXSEvY5gB9aRY1Tb1JcJo9x6EezAOnKMapv8MmGUWyeCHVhHjlFtbVMkaN7IP7QxDH5oA1hfrSdsMZomf2gDQGJMkWAUXKAEAJUh2IGC1XJlKJrFVAyQWKr58ZquDEWzCHYgoZRhXNOVoeM2rpPNbT2JTbBjbNq6U4wiZRi3te2x6fd9XEc2bT5iItgxFm3eKUaRMozbeGVojvd9XEc2bT5iItgxFm3eKUaROozb1vaY430f15FNW4+YJIIdY9LmnWJUbQvjlHK87+M6smnjEdMSrjzF2HRxjh287+PElafILvfIlYDJo+n3nff5bAQ7BtK2naerJ29rs9Hnjvd5bQQ7NtTGnaeLJ2/b9uW7kUE+d118nwfBkgLYUBt/KHjpJN6E1YmTt0shePuDz+m6O2erWIJgkM9d197nQTFix4ba2OHS5o6GYdQ4ch3kc9e193lQdMW0TK7D7doO82uzNGJfCsE2TJcNYpTPXY2f2UG7YkYKdtvXSvq0pLdJuiIiBkprgn04bZzrRnNyBFmp4VnrvtJUu+PTkj4g6R9G3A4GUOPhNtLJ0WZYanh2fV8Z6eRpRByNiOdSFYP1caIIJSn5pHrX95XGTp7anpE0I0mTk5NNPWxVmjpRVOrhNcoyykn1cX/Gur6vbDjHbvthSeet8VcHI+K+/n2+LemTzLG3X8mH1yjPMMFWy2csx/NINsceEVemKQlt0PW5SWzOMPP6tXzGzjUVVcIInj52rNDGnnW0Sy2fsdXPY8cvbS/mSGSkYLd9jaQvSNot6Zu2j0TEVUkqQxZc8IFxq+Uztvp5lHQkMlKwR8S9ku5NVAsKkXtVRtSvls/Y6udRypEIUzEAkEBJRyIEOwAkUsqRCKs7AkBlCHYAqAzBDgCVIdg7YP7Ead3xyLEqfnwBwMY4eVq5Wi7fxmhKXdME40GwV66kiyaQB1/u3cNUTOW6vnwpyl5eF+PBiL1yJV00gTxqWZsFg+M3T4EOYI69Dk39NB6AFijlikg0gzl2AKgMwQ6sgd5/tBlTMcAqtAei7Rixg9HpKrQHYlya2tcYsXcco9Oz0R6IcWhyXyPYO67EK1Nzt+bR+49xaHJfI9g7rrTRaSlHELQHIrUm9zWCveNKG52WeASxJPeRBNqtyX2NYEdRo9PSjiCWlHIkgXZral8j2FGU0o4glpR8JAGsRrCjOCUdQSwp9UgCWAvBDgyg1CMJYC0EOzCgEo8kgLVw5SkAVIZgB4DKEOwAUBmCHUAxWJAujZFOntq+TdIfSjoj6T8l/XFEvJKiMADdwkVg6Yw6Yn9I0qURcZmk70u6afSSAHQRyyWnM1KwR8SDEbHQvzkr6cLRSwLQRUsXgU1YXAQ2opR97B+V9K8JtwegQ7gILJ0Ng932w5LOW+OvDkbEff37HJS0IOnwOtuZkTQjSZOTk0MVC6BuXbsIbFwrhm4Y7BFx5Xp/b/sjkt4n6d0REets55CkQ5I0NTV1zvsBQBeM82TxSHPstg9I+pSkqyPiZ0kqAoAOGOfJ4lHn2L8o6Q2SHrItSbMRccPIVaFK/FAFum75PjDOFUNHCvaIeGuqQlA3epTRdWvtA+M6WcyVp2gEPcrounP9WMvHfu+tyQc5BDsaQY8yuq7JfcDrNLKMzdTUVMzNzTX+uMiLOXZ03aj7gO35iJja6H780AYa07UeZWC1pvYBpmIAoDIEOwBUhmAHgMoQ7ABQGYIdACpDsANAZbL0sdt+WdKJRJvbJelHibaVEnVtTql1SeXWRl2bU0NdeyJi90Z3yhLsKdmeG6Rhv2nUtTml1iWVWxt1bU6X6mIqBgAqQ7ADQGVqCPZDuQs4B+ranFLrksqtjbo2pzN1tX6OHQCwUg0jdgDAMq0Pdtuftv2C7SP9P+/NXdNqtj9pO2zvyl2LJNm+1fZ/9F+vB23/Wu6aJMn2bbaf7dd2r+235K5Jkmxfa/sZ24u2s3dV2D5g+znbx2z/Re56lti+2/ZLtp/OXctyti+y/Yjto/338eO5a5Ik22+0/V3bT/br+kyqbbc+2Ps+HxGX9/88kLuY5WxfJOk9kv4rdy3L3BYRl0XE5ZK+IekvcxfU95CkSyPiMknfl3RT5nqWPC3pA5IezV2I7QlJd0j6A0lvl/Qh22/PW9X/+5KkA7mLWMOCpD+LiLdJmpb0sUJes59LeldE/KakyyUdsD2dYsO1BHvJPi/pzyUVczIjIn6y7OYvq5DaIuLBiFjo35yVdGHOepZExNGIeC53HX1XSDoWEccj4oykr0p6f+aaJEkR8aikH+euY7WIeDEinuj/908lHZV0Qd6qpOh5tX9zW/9Pkn2xlmC/sX/4frftYn7JwfbVkl6IiCdz17Ka7c/afl7SdSpnxL7cRyV9K3cRBbpA0vPLbp9UASHVFrb3SnqHpMfyVtJje8L2EUkvSXooIpLU1YpfULL9sKTz1virg5L+XtKt6n3T3SrpdvVCoYTabpb0+03Vstx6dUXEfRFxUNJB2zdJulHSLSXU1b/PQfUOnw83UdOgdRXCa/y/Io64Smf7zZLukfSJVUet2UTELyRd3j+fdK/tSyNi5HMUrQj2iLhykPvZ/kf15owbc67abP+GpIslPWlb6k0rPGH7ioj471x1reFfJH1TDQX7RnXZ/oik90l6dzTYi7uJ1yu3k5IuWnb7Qkk/zFRLa9jepl6oH46Ir+euZ7WIeMX2t9U7RzFysLd+Ksb2+ctuXqMEL0oKEfFURPxqROyNiL3q7ZC/1USob8T2JctuXi3p2Vy1LGf7gKRPSbo6In6Wu55CPS7pEtsX294u6YOS7s9cU9HcG1ndJeloRHwudz1LbO9e6vyy/SZJVyrRvtj6C5Rs/7N6Z5RD0g8k/UlEvJi1qDXY/oGkqYjIvrqc7Xsk/bqkRfVW2bwhIl7IW5Vk+5ikN0g61f9fsxFxQ8aSJEm2r5H0BUm7Jb0i6UhEXJWxnvdK+ltJE5LujojP5qplOdtfkfRO9VYr/B9Jt0TEXVmLkmT7dyX9u6Sn1PvMS9LNuTvobF8m6cvqvY9bJH0tIv4qybbbHuwAgJVaPxUDAFiJYAeAyhDsAFAZgh0AKkOwA0BlCHYAqAzBDgCVIdgBoDL/B3B7T4rL3oO/AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.plot(X0r[:,0], X0r[:,1], '.')\n", "plt.axis('equal');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1D plot of the function to regress along the main eigenvector axes." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztnX+snVW55z/POW25ouTS0Gqx7Wll6iXSRrA9KSVMoigyhXQuGX6EqjFkUHs1mEh0Mv4g03j7z1zjjDPm0hvtgDEafnilrTQMFWrsjd4JxXY3KC0Fb6eXA8fWgcJBa0BOz9nP/LH3Lrub/ePd+13vfn99P0nT/e69zrue9f54nrWe51lrmbsjhBCifIykLYAQQoh0kAEQQoiSIgMghBAlRQZACCFKigyAEEKUFBkAIYQoKTIAQghRUmQAhBCipMgACCFESZmTtgDdWLBggS9fvjxtMYQQIjdUKpWT7r4wStlMG4Dly5dz4MCBtMUoHZWJKfYde5l1F13AmmXz0xZHCNEHZjYRtWymDYAYPpWJKT5x9z6mZ6rMmzPCvZ9eJyMgREFRDECcxb5jLzM9U6XqcHqmyr5jL6ctkhAiIWQAxFmsu+gC5s0ZYdRg7pwR1l10QdoiCSESIrYLyMyWAj8AFgFVYJu7f7ulzIeAh4B/rX+1w923xK1bhGfNsvnc++l1igEIUQJCxABmgC+5+0EzOw+omNked3+6pdwv3X1DgPpEwqxZNl+KX4gSENsF5O4n3P1g/fMp4AiwOO55hRBCJEvQGICZLQc+ADzR5ucrzOzXZrbbzFZ2OccmMztgZgdeeumlkOIJIYRoIpgBMLN3ANuBO9z9jy0/HwSWufulwN8DP+l0Hnff5u7j7j6+cGGkuQyigFQmpti69yiViam0RRGisASZB2Bmc6kp/3vdfUfr780Gwd0fMbN/MLMF7n4yRP2iWGgughDDIfYIwMwMuAc44u7f6lBmUb0cZra2Xq8SzEVbNBdBiOEQYgRwJfBJ4Ckze7L+3deAMQB3/w5wE/A5M5sBXgc2ursHqFtkkLhLSTTmIpyeqWoughAJYlnWw+Pj4661gPJFKPeN1iMSYjDMrOLu41HKai0gEZR27ptBFLjmIgiRPFoKQgRFS0n0RhlOIitoBFByQrtaui0lIbdO/jKcdM+KjQxAiUlKGbVz3+RN8SVFKBfZMNA9Kz5yAZWYYaZbKrWzRp5cZLpnxUcjgBIzSLrloC6BIqZ2DnIt8rTaatr3TO6n5FEaaMnp5yWL6xLo94UelgIYpJ5hu0fSUoZp1iv302AoDVREppO/vt1LH9d/3U9q57AUwKD1JOnLb73+nWQchnJOKx03T7GSPCMDIM6im0Js5xJISgklrQAach9/9fWB6knKPdLu+nfyxRe5h5y2+6ksyACIs+imeFv91xBOCbUakiQVQLOSnTNizBkdYXa2v3qS8uW3u/7trkXRe8h5ipXkGRkAcRa9FG+zS2Dr3qNBlFCnUUdSCqBZec5WnVvWLmXx+W/ru54k3CPtrn+na1H0HrJmgyePDIA4i34Ub6heeqfebFIKoFXuG1cvyYyi6XT9W69FHAOp7BrRQFlAIhYhlEljBNBQyFFdSXGyioDcKMF+M7W6lS1ydo0MWw1lAYmzSPLFCNFLH6Q3268ia1f+9qtWxJJ7GPTTzihlu8UOhqlAQ9dVZMOWJDIABScvL0a/hqTfIGheg6b9yB2lbCe33X1PPM/mhw4xW3XOmTvC5g0rmXptOhFjkMQzGfL+lmkkEWpLyPXAt4FR4G53/7uW388BfgCsobYT2C3u/lyIukV38qr4etFv/CGvaYX9yN0oO326ipkx/9x5bynTbrRVmZhi80OHmKnW3MFvnK6y+aFDVN0T6TRsPzjJG6erOOGeyVD3Ny8dplDENgBmNgpsBT4KTAL7zWyXuz/dVOxTwJS7rzCzjcA3gFvi1t2JMlnwXuRV8fWiX7dRXtMK+5F7zbL5bN6w8ozy3vLwYS5edF7PFVn3HXuZalMs0Ayq7pE6DYPEYR6sTNKobXTEgjyToe5vUTtMnQgxAlgLHHX3YwBm9gBwPdBsAK4Hvl7//CBwl5lZEttCls2C9yKvii8K/bqN8ppW2I/cU69Nd1Tend6NMyOHmSojZnz6376H7z/+XM9OwyDv2r5jLzMzWwXAgJvHlwa7JyHub1E7TJ0IYQAWAy80HU8Cl3cq4+4zZvYH4ALgZID6z6JsFjwKeVV8on+6KbBu6batnYSPrlzUs9MwyLvWKt8Nq5d0LJvGSL7IHaZ2hDAA1ua71p59lDK1gmabgE0AY2NjfQtTNgueJ8rimkuznd0UWLd3o908g36VedR37cbVS/D6/3EympKiTB2mEAZgEljadLwEON6hzKSZzQH+Enil3cncfRuwDWrzAPoVpmwWPC/k1TU3iI877XZ2UmCh341+z9d6bW7s0vvvtP6R3uuwhDAA+4H3mtl7gN8BG4GPt5TZBdwKPA7cBPw8Cf9/gzJZ8LyQR9fcoD7uLLcz9LvRz/n6uTato4tTr5/mlu8+HiQzqSwj0SjENgB1n/7ngUeppYF+z90Pm9kW4IC77wLuAX5oZkep9fw3xq1X5Is8uuZC+Ljz0M5h0c+1aR5dnHr9NNt+eYx6lirTCaw7VVaCzANw90eAR1q+29z0+c/AzSHqEvkkj665QZR51HaG7IVGPVfaPd9BUneBes//ze9HbPDU0ayP0IaNZgKLoZE311w/CqtVufa7LEUcd0aUc2Wl5zvIjO/mOQojBluuXzWw7CFGaGkb0pDIAAjRhSgKq1/lGrIXGvVcee35ts5R2HL9Kj5+eS07MMrCd+1WVY0zEs2KIQ2FDIAQMelXuYaME0Q91yB1ZqGn20lh91LE3X6PMxIdhiEd5nWXARAiJv0q15DxkKjnipuyOcx9iNvJ3lrXjh7rCSWlqJMO8g97hCEDIERMBlHoIeMhUc8VN2UT+tsCNCljUZmY4scHXnhzPaHRtyripBR10skMw3bVyQAIEYC8BbhbibIncz/KKcme7L5jL59ZudSAm9a8dUZxs6Kef+68MwYshAyNe12ZmGLr3qNBDcGw04hlAIQoOf3syRxVOSXZk223pWc7GvUlYYiSMnDDTpeWARCi5HRbJK51faBh7xfdjn7kSMoQJWnghjmalAEQouT0O0M3arwhyZ5sVDmSMkRFmfGtTeGFEJlI+UyKRtvmnzsv6DaXWb1m2hReCNEXeQ9idyOpWECoa5amIZEBEEIUnqzOhE57ZvHI0GoSQoiUaPjsR41M+ew7zbcYFhoBCCEKT1ZXo007mCwDIIQoNM0+9tuvWpG2OGeRtmGSARBCFJa0fexRSDMAHysGYGbfNLNnzOw3ZrbTzM7vUO45M3vKzJ40M+V1CiGGQto+9qwTNwi8B1jl7u8Hfgt8tUvZq9z9sqj5qUIIEZesBn+zQiwXkLs/1nS4j9qG70IIkQnS9rFnnZAxgNuAH3X4zYHHzMyB77r7toD1CiEKQhKTotLwsTe3A8isAeppAMzsZ8CiNj/d6e4P1cvcCcwA93Y4zZXuftzM3gnsMbNn3P0XHerbBGwCGBsbi9AEIUQRyEPANgrN7ZgzOgLuzFQ9k23qGQNw96vdfVWbfw3lfyuwAfiEd1hYyN2P1/9/EdgJrO1S3zZ3H3f38YULFw7SJiFEDilKwLa1HadnvWubGvsKVCamhi5rLBeQma0Hvgx80N1f61Dm7cCIu5+qf74G2BKnXiFE8Uh7UlQomtsxWh8BzFa9bZvSHvXEjQHcBZxDza0DsM/dP2tm7wbudvfrgHcBO+u/zwHuc/efxqxXCFEwihKwbW0HdI4BpL1GUdwsoLbT6uoun+vqn48Bl8apRwhRDoqyKmm7zXTakfaoRzOBhRAiJdIe9WR6QxgzewmYSFuOIbIAOJm2EDlA1ykauk7RKNp1WubukTJoMm0AyoaZHdBM6d7oOkVD1ykaZb5O2g9ACCFKigyAEEKUFBmAbKElMqKh6xQNXadolPY6KQYghBAlRSMAIYQoKTIAQghRUmQAMoqZ/SczczNbkLYsWSTqbnRlxczWm9mzZnbUzL6StjxZxMyWmtleMztiZofN7AtpyzRsZAAyiJktBT4KPJ+2LBmmn93oSoWZjQJbgWuBS4CPmdkl6UqVSWaAL7n7+4B1wO1lu04yANnkfwD/mdpGOqIN7v6Yu8/UD/cBS9KUJ2OsBY66+zF3nwYeAK5PWabM4e4n3P1g/fMp4AiwOF2phosMQMYws78Gfufuv05blhxxG7A7bSEyxGLghabjSUqm2PrFzJYDHwCeSFeS4aLF4FKg2y5rwNeo7ZlQegLtRldGrM13Gk12wMzeAWwH7nD3P6YtzzDJ9DyABQsW+PLly9MWQwghckOlUjkZdTG4TI8Ali9fzoEDB9IWQxScJDYiFyItzCzyCsqZNgBCJE3aW/IJkSYKAotSU5SNyIUYBBkAUWoaW/KNGrneiFyIQYjtAqpPWvoBtWyNKrDN3b/dUuZDwEPAv9a/2uHuW+LWLURc0t6ST4g0CREDaMymO2hm5wEVM9vj7k+3lPulu28IUJ8QQSnKRuRC9EtsF5Bm0wkhRD4JGgPoMZvuCjP7tZntNrOVXc6xycwOmNmBl156KaR4QgghmghmAHrMpjtIbaf6S4G/B37S6Tzuvs3dx919fOHCSHMZ3kJlYoqte49SmZga6O+FEKIMBJkHYGZzqSn/e919R+vvzQbB3R8xs38wswXufjJE/c0or1sIIaIRewRgZgbcAxxx9291KLOoXg4zW1uvN5GEa+V1CyFENEKMAK4EPgk8ZWZP1r/7GjAG4O7fAW4CPmdmM8DrwEZPaBGiRl736Zmq8rr7QMshCFE+Mr0Y3Pj4uA+yFpCUWX/IbSZEcTCziruPRylbyLWAlNfdH+3cZmW9fuo8iDJRSAMg+kNusxohRkIyICJPyACI3CyHkLRy7TUS6lW/XGkib8gACCD7brNhKNduI6Eo9cuVJvKGDMAQkFsgPsNQrt1GQlHqlytN5A0ZgARoVviA3AIBGJZy7TQSilJ/aFeaOg4iaWQAAtPqKrhx9ZLcuAWSVDhxzz3MOEU7WTvV31o2lCtN8QQxDGQAAtPqKnDou+eaRs8vSYUT6tzDiFN0k7W1/iSvmeIJ5SDtUZ4MQGBaXQU3rl7CjauXRL7JoZRKvw9WkgonT8qsH1mTbJfiCdFIW4HGIQujPBmAwHRyFUS9sSGUyiAPVhSFM+jLlidl1o+sSbYrL6m5aZIFBRqHLHSMZAASII6rIoRSGeTB6qVw4rxseVJm/ciadLuynpqbNllQoHHIQsdIBiAQoYaiIZTKoA9WN4UT92XLkzLrR9Y8tasdeXahZEGBxiELHSMZgACEHorGVSpJPFh5f9nySKtyDq2s8+5CAbhx9RK8/n/eZIf27/owjbIMQACyOBQN/WBlobeSNMPuDXerr1U5b96wki0PHw6qrJuf2+mZKv/zZ7/ljqv/Khf3tl26dR5pZ+SHaZRlAAKQh95xiAcr7+6Obgz7xetVX2unYvehE8E7GY3ntnHef/6Xk+x/7pVMjgRaFWUWO1390u4ZGHa7guwJbGbrzexZMztqZl9p8/s5Zvaj+u9P1DePLwyN3vEXr7k4ky8PlHOntH72hh729elVX0M5jxrMnTPCtasuPOs4RCej8dxeuWIBBjjZfDbue+J5bvnu4/z3x57lE3fvozIx9Zbrk8VOVy/aPQPDblfsEYCZjQJbgY8Ck8B+M9vl7k83FfsUMOXuK8xsI/AN4Ja4dWeJrPeO8zBKCUGjpzj/3Hl9uUyGdX2a5etW35pl89m8YSW7D53g2lUX8vHLx7h40XnBXVRrls3njqv/iv3PvZLJZ6MyMcXmhw4xU61tXDVdV5S3X7Ui9y7Jds/csF2tsXcEM7MrgK+7+7+rH38VwN3/a1OZR+tlHjezOcDvgYW9toUcdEcw0Z48Z3xEoXlIPWJG1Z2qw6jBF6+5mNuvWtHz75O8Pu38+lOvTQdPux1Utiw+G1v3HuW/PfosDUUxZ8T40d9ckSkZ45DEdR/2jmCLgReajieByzuVcfcZM/sDcAFwMkD9qZDVF6YbWR+lxKV5SI07IyOG4ZF7tUlfn9Yh/9Rr0x2NUlxfcL/PZ1afjXUXXcA5c0eYPl1lZMTYcv2qTMo5KGlf9xAGwNp819qzj1KmVtBsE7AJYGxsLJ5kERhEkRchfa5f8mDwWofU3XrYack3Z7Qm3+hocrOMi/R8liH7LE1CGIBJYGnT8RLgeIcyk3UX0F8Cr7Q7mbtvA7ZBzQUUQL6ODPqiFCEDoR/yolDa+c0zR8Pr2cP1GkfxFe35TLuXXGRCZAHtB95rZu8xs3nARmBXS5ldwK31zzcBP+/l/x8Gg2Z+5DkDoZ/MmAZ5ySCqTEyx5eHD/J+jJ9ny8OG+2piELK3Xed+xl5mpOg7MVr3ndVyzbD63X7Wib+WX5+dTDJfYI4C6T//zwKPAKPA9dz9sZluAA+6+C7gH+KGZHaXW898Yt94QxFkyIY/D0nY9eaBnO/KSQZSVnm+nEdMwN7XJ0vOZB/dhWQkyEczdHwEeafluc9PnPwM3h6grJHFelDwOS1sV5PaDk+w4ONnTtdPuOlUmpth+cBIDbsjINPysGKpOhmiYijkrz2de3IdlpfQzgbPyogyDVgVpELnH3Hyd7nvief7LT55itu7E+3Flkvs/E+3FTrI3mJWebzdDVKbnDbIzKhPtKb0BKBOtChJg+8HJM4pq/rnz2Lr3aFfl2ZiYM9sUwYn6Yg+jN5gFBZsVQzRs2hn3rIzKRHtKYwDkh6zRqiAbiirqzNl9x15mtnp2/D7qi93aG9xxcLKw9ySKISrSM9nJuJfVGOaFUhgA+SE703hJv7bzKd44XT1rPZh216h5Yo4ZfOR97+JvPvhvIl3P5t7g6OgIPz7wAjNVL+U9Kdoz2c3Vk4VRmWhPKQxAXvyQSfYIey09/GBl8szMvNER69ijjxs4b/zt7159nQd+9fxZAeky9RJbl2LO6jMZFbl68kkpDEAeHs4ke4RRlh6ema0CtSnbN48v7Vp3nB5d428rE1PsqMcfRkeMByuTzMwWozcchfnnzqPhSat67TjPyNWTT0phAPLwcCY5Sul17lYDecMQNtdovifHX32d+5tGA3nvDUdh6rXpM0swj9SP845cPfmjFAYAsv9wJjlK6XXutAxk82igORspiyO00DRiKWVqs8gesZeDTpKyLQedVgwgC2RdviQoY5tF8vSzHLQMQGD0Upcb3X+RNsPeD0DUKVpqn+gP3X+RN4LsCSxq5GXVTJEMuv8ib8gABETL8JYb3X+RNxQDCEwZfMBlaOOg6NqItFEMIEWynm4aF/m5u1P0+y+KhVxAoi/k5xaiOMQaAZjZN4F/D0wD/xf4j+7+aptyzwGngFlgJurwRGSPPCyrIYSIRlwX0B7gq/VtIb8BfBX4coeyV7n7yZj1iZTJw7IaQohoxDIA7v5Y0+E+ahu+i4IjP7cQxSBkEPg24EcdfnPgMTNz4Lvuvq3TScxsE7AJYGxsLKB4xaY5++TZ359i96ETXLvqQj5+ua6hEKI9PQ2Amf0MWNTmpzvd/aF6mTuBGeDeDqe50t2Pm9k7gT1m9oy7/6Jdwbpx2Aa1NNAIbSg9zZk5IyPGTH2/xl/+S83jJiMghGhHTwPg7ld3+93MbgU2AB/xDpMK3P14/f8XzWwnsBZoawBCULZc7ObMnOrs2bdg96ETMgBCiLbEzQJaTy3o+0F3f61DmbcDI+5+qv75GmBLnHq7UcY89ebMHGsaAQBcu+rCFCUTQmSZuDGAu4BzqLl1APa5+2fN7N3A3e5+HfAuYGf99znAfe7+05j1diQv2z+GpDUzRzEAIUQU4mYBrejw/XHguvrnY8Clcerph7LmqTdn5qxZNl+KXwjRk8ItBaE8dSGEiEamF4Mzs5eAiYROvwAoy8Q0tbV4lKWdoLb2yzJ3XxilYKYNQJKY2YGyLEmhthaPsrQT1NYk0WJwQghRUmQAhBCipJTZAHRcjqKAqK3FoyztBLU1MUobAxBCiLJT5hGAEEKUmlIbADP7ppk9Y2a/MbOdZnZ+2jIlhZndbGaHzaxqZoXLqDCz9Wb2rJkdNbOvpC1PUpjZ98zsRTM7lLYsSWNmS81sr5kdqT+7X0hbpqQws78ws1+Z2a/rbf3bYdRbagNAbUObVe7+fuC31Da0KSqHgBtIcBG+tDCzUWArcC1wCfAxM7skXakS4/vA+rSFGBIzwJfc/X3AOuD2At/XN4APu/ulwGXAejNbl3SlpTYA7v6Yu8/UD/cBS9KUJ0nc/Yi7P5u2HAmxFjjq7sfcfRp4ALg+ZZkSob6M+itpyzEM3P2Eux+sfz4FHAEWpytVMniNP9UP59b/JR6gLbUBaOE2YHfaQoiBWAy80HQ8SUEVRVkxs+XAB4An0pUkOcxs1MyeBF4E9rh74m0t3FpArQTa0CYXRGlrQbE23ym9rSCY2TuA7cAd7v7HtOVJCnefBS6rxyJ3mtkqd0801lN4AxBiQ5u80KutBWYSWNp0vAQ4npIsIiBmNpea8r/X3XekLc8wcPdXzeyfqMV6EjUAmZ4HsGDBAl++fHnaYgghRG6oVConoy4Gl+kRwPLlyzlw4EDaYogUKdv2nkLExcwir6CcaQMgyk0Zt/cUYpgoC0hklnbbewohwhHbAESZrWdmHzKzP5jZk/V/m+PWK4pPY3vPUaNU23sKMSxCuIAas/UOmtl5QMXM9rj70y3lfunuGwLUJ0qCtvcUIlliGwB3PwGcqH8+ZWaN2XqtBkCIvmne7F4IEZagMYAes/WuqC90tNvMVoasVwghRP8EywLqMVvvILWNiv9kZtcBPwHe2+E8m4BNAGNjY6HEE0II0UKQEUCv2Xru/sfGQkfu/ggw18wWtDuXu29z93F3H1+4MNJchrdQmZhi696jVCamBvp7IYQoA7FHAGZmwD3AEXf/Vocyi4D/5+5uZmupGZ5EcvqUOy6EENEI4QK6Evgk8FR9JTuArwFjAO7+HeAm4HNmNgO8DmxMat2ddrnjMgBCCPFWQmQB/TPtV2NsLnMXcFfcuqLQyB0/PVNV7viQ0bINQuSLwi0FodzxdJDrTYj8UTgDAModHxbNPX653oTIH4U0AHkgi+6SfmRq7fFv3rCy1K63LN5PIXohA5ACWXSX9CtTa49/6rXp0rresng/hYiCVgNNgSyuctmvTO0WaluzbD63X7WidMovi/dTiChoBJACWcxU6lemNcvms3nDSnYfOsG1qy7sW+kXyWWSxfspRBQyvSXk+Pi4F3VHsCwqwF4yNf8ODOz2yLrLZJB7k8X7KcqJmVXcfTxKWY0AUiKLmUrdZGpV2jesXjJw1k+3jKFhKdJO9QxqnLJ4P5uRgRLtkAEQkWhV2gYDuz1aXSbzz53H1r1HmX/uPLY8fDjxkUE3JR8nnbUyMcWOg5M4cOPqJZlRtFkfcYn0kAEoASF6f61K+4bVS7hh9ZKBztuYrLf94CQnT73B13cdYqbqjJhRdU98LkE3JT+oP78yMcXH/ldNyQI8eOAF7t90RSYUreZoiE7IABScUL2/TjOs4yiSHQcneeN0lTNRKHdGRgzDEw2mdlPyg84k33fsZU7XlT/A6VnPjKJVkFp0Qgag4ITs/YX0czfkaih/A+bNrU0om3ptOlFfdS8lP0g71110AXPnjJwZAcwdtcwoWi2PIjohA1Bw0uz9dXM9Ncs1OjrCTWuWJOY3bydH6KDtmmXzuf8z6zIZA4DhBqkVcM4PSgMtAWm8kFFcT90ycULJm0QANLR8RVKWCjinj9JAxVnE6f0NqqCiuJ7aydVujaE4LqHQAdBQCq4yMcX2g5P8+MALzMzWYh73fyb/yrLIAeeiGWuQARBd6Kbser0Mg7qemhXI9EyVzQ8douo+sLIN4QILvepp47r++fSbQePpmSo7Dk7mXrEUNeBc1JFNEANgZuuBbwOjwN3u/nctv58D/ABYQ20ryFvc/bkQdYvk6KTsorwMgwYemxWImTFbdZzBlW3cAGgSq542rmsrnZyxeep5xl0iJKsUdWQTYk/gUWAr8FFgEthvZrvc/emmYp8Cptx9hZltBL4B3BK3bpEsnXpzUV+GQVxPzQq7MTEsbm8yjgss1KqnzUq8cV2nT1epUsuAmjtq3Lh6yVv+ZvvBSR6sTDIzm4+eZ2Vi6sxkvv3PvcLFi87LtLxRKerIJsQIYC1w1N2PAZjZA8D1QLMBuB74ev3zg8BdZmZJ7QsswtCp95z0y9CssC9edF6qvd92be3XoLQbMTUbuXYxjsbfNM+TyEPPs6g95aKm0oYwAIuBF5qOJ4HLO5Vx9xkz+wNwAXAyQP0iQdopuxAvQ1S3Rtpr7IRoa2tcY9+xl3sum91unkQeep5F7SnD8J7FYbr8QhiAdhvCt/bso5SpFTTbBGwCGBsbiyeZ6MmgD1vczKI8BdTivvjzz51Htf60V7123Iuz5kmMGDePL+WGBOYWhFY27QxmnmIYaTPsdyOEAZgEljYdLwGOdygzaWZzgL8EXml3MnffBmyD2jyAAPKJDqSliIvqJujE1GvTGLUez0j9uBfDcDkkdf+bDWbejH3aDPvdCLEj2H7gvWb2HjObB2wEdrWU2QXcWv98E/Bz+f/TJ62drNrtJpYWlYkptu49SmViKrHz/+7V15lbb++8udHbu2ZZsjusDeP+a7e0/hj2uxF7BFD36X8eeJRaGuj33P2wmW0BDrj7LuAe4IdmdpRaz39j3HpFfOKsfBmnZ9qpdztsV0HSvdPm888ZMTauHUvEjTMoSfvrG8ZvzugIs7PFiwkkwbCDzUHmAbj7I8AjLd9tbvr8Z+DmEHWJcAzysIVcXbRd1sswXQVJD7ebzz9bdd59/ttSV/6tRjYpZZN145dlhpn4oJnAJaffhy0ppTkM32er8ku6Bxz1/MPcBa2dkU2iziwaP/FWZABEX7Su4vm7V1+nMjEV++Ved9EFzBl987xJuCPaKb/GxjTt0tTiEqWHPcyRzzADjEVOBy0SIYLAokQ0lNrGtWOQMksqAAAGJElEQVRUq1Xuf+J5Prbt8TBB1EZeQAL5AZ2Ckc/+/hT/uP8F7v/V83zi7n3Bg8G9ArnDDJLGCTD2GyxvPCdfvOZiZf5kGI0ARN+sWTaf7QcnaSxnMz3rbI+5kNm+Yy8zU1/3Z7Yafjetdj3SysQUmx+qbUcJb07SGqayCtlT7uVKGtTnP+goJe1JfKI3MgBiIFpdJnFdKEkrwnbKb+veo8xW3xxtjFjnXbyS8tPHUcqtk62iKOlBlHLZ5m2UCRkAMRA3rF7CjyuTZ20SH4dQGSndFGGr8lt30QWcM7e2KNvIiLHl+lWp+OlDrC2UpJKWP7+4yACIgVizrLYFYuhlBOKepx9FGNXoZK0H3E6eJJV0URdCEzIAIgZZ9PH2qwijtCFrPeBOK5QmqaSzeK9FfLQnsMgsg/rdk/DXZ21Bs6zJI7JDP3sCywCITBJnU3khyow2hRe5p5ffXatMChEfTQQTmaTXpCWtMilEfDQCEJmkV1Aza4FZIfKIDIDILN0yT9Ysm8/mDSvZfegE1666UO4fIQZABkDkksrEFFsePsz0TJX9z73CxYvOkxEQok9ixQDM7Jtm9oyZ/cbMdprZ+R3KPWdmT5nZk2amtB4RG8UAhIhP3CDwHmCVu78f+C3w1S5lr3L3y6KmJwnRjSxtKylEXonlAnL3x5oO91Hb71eIxNHyBELEJ2QM4DbgRx1+c+AxM3Pgu+6+LWC9oqTEXZ6g3YqaMij5RvewP3oaADP7GbCozU93uvtD9TJ3AjPAvR1Oc6W7HzezdwJ7zOwZd/9Fh/o2AZsAxsbGIjRBiP5pnUi2ecPKM0FlTSzLJ5oc2D89YwDufrW7r2rzr6H8bwU2AJ/wDutKuPvx+v8vAjuBtV3q2+bu4+4+vnDhwkHaJERPWoPIuw+dUFA55ygxoH/iZgGtB74M/LW7v9ahzNvN7LzGZ+Aa4FCceoWIQ2ViiuOvvs6cETsTRL521YUKKuccJQb0T9wYwF3AOdTcOgD73P2zZvZu4G53vw54F7Cz/vsc4D53/2nMeoUYiGY3wZzREW5Zu5QbVy9hzbL5XLzoPPmPc4wSA/onbhbQig7fHweuq38+Blwapx4hQtHsJpidrbL4/Ld13DFM5A/dw/7QYnCiVMhNIMSbaCkIUSrkJhDiTTK9IYyZvQRMpC1HABYAJ9MWIjBFbBMUs11FbBMUs10h2rTM3SOlUGbaABQFMztQtCUwitgmKGa7itgmKGa7ht0mxQCEEKKkyAAIIURJkQEYDkVc+6iIbYJitquIbYJitmuobVIMQAghSopGAEIIUVJkAIZE1N3T8oSZ3Wxmh82sama5zsYws/Vm9qyZHTWzr6QtTwjM7Htm9qKZFWbtLTNbamZ7zexI/dn7QtoyhcDM/sLMfmVmv66362+HUa8MwPDoZ/e0vHAIuAFou7R3XjCzUWArcC1wCfAxM7skXamC8H1gfdpCBGYG+JK7vw9YB9xekHv1BvBhd78UuAxYb2brkq5UBmBIuPtj7j5TP9wHLElTnhC4+xF3fzZtOQKwFjjq7sfcfRp4ALg+ZZliU99z45W05QiJu59w94P1z6eAI8DidKWKj9f4U/1wbv1f4gFaGYB0uA3YnbYQ4gyLgReajicpgFIpOma2HPgA8ES6koTBzEbN7EngRWCPuyfeLq0FFJBAu6dliihtKgDW5julx2UYM3sHsB24w93/mLY8IXD3WeCyenxwp5mtcvdE4zcyAAFx96u7/d60e9pHOu2eljV6takgTAJLm46XAMdTkkX0wMzmUlP+97r7jrTlCY27v2pm/0QtfpOoAZALaEhE2T1NpMZ+4L1m9h4zmwdsBHalLJNog9V2lroHOOLu30pbnlCY2cJGZqCZvQ24Gngm6XplAIbHXcB51HZPe9LMvpO2QHExs/9gZpPAFcD/NrNH05ZpEOrB+c8Dj1ILKv6jux9OV6r4mNn9wOPAxWY2aWafSlumAFwJfBL4cP09etLMrktbqABcCOw1s99Q65DscfeHk65UM4GFEKKkaAQghBAlRQZACCFKigyAEEKUFBkAIYQoKTIAQghRUmQAhBCipMgACCFESZEBEEKIkvL/Adfp1BIdXLYYAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.clf\n", "for i in np.arange(0,3):\n", " plt.subplot(3,1,i+1)\n", " plt.plot(X0r[:,i], y0, '.')\n", " plt.axis('tight')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Linear Regression\n", "-----------------\n", "We look for a linear relationship\n", " $ y_i = \\dotp{w}{x_i} $\n", "written in matrix format\n", " $ y= X w $\n", "where the rows of $X \\in \\RR^{n \\times p}$ stores the features $x_i \\in \\RR^p$.\n", "\n", "\n", "Since here $ n > p $, this is an over-determined system, which can\n", "solved in the least square sense\n", " $$ \\umin{ w } \\norm{Xw-y}^2 $$\n", "whose solution is given using the Moore-Penrose pseudo-inverse\n", " $$ w = (X^\\top X)^{-1} X^\\top y $$\n", "\n", "\n", "Least square solution." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "w = np.linalg.solve( X0.transpose().dot(X0), X0.transpose().dot(y0) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Prediction (along 1st eigenvector)." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsvXl8XHd19/++s0ka7RpttnbJluUlibfYTuLsibNCAqFZoGnyI0D7AIWWPi30gfahCy8KJX2gKS0EUpayhBAgJCGJnd1LvDtObHmRbe2Ste8ajTTL/f3xvXc0M5pVM7JG0vf9evklazRz51qeOXPuOZ/zOYqqqkgkEolk8WCY7xOQSCQSSWKRgV0ikUgWGTKwSyQSySJDBnaJRCJZZMjALpFIJIsMGdglEolkkSEDu0QikSwyZGCXSCSSRYYM7BKJRLLIMM3Hk+bn56uVlZXz8dQSiUSyYDl69GifqqoFke43L4G9srKSI0eOzMdTSyQSyYJFUZSWaO4nSzESiUSyyJCBXSKRSBYZMrBLJBLJImNeauzBcDqdtLe343A45vtU5ozU1FRKS0sxm83zfSoSiWQRkzSBvb29nczMTCorK1EUZb5PJ+Goqkp/fz/t7e1UVVXN9+lIJJJFTNKUYhwOBzabbVEGdQBFUbDZbIv6ikQikSQHSRPYgUUb1HUW+79PIpk32g7BnsfFV0nylGIkEolkVrQcgJ/cDaoHjBZ45Hko2zLfZzWvJFXGLpFIJDFz4tfgcYLqBvcUNO+Z7zOad2Rgl0gkC5vcSu0visjYK6+dz7NJCuIO7IqipCqKckhRlPcURalXFOUfEnFi0XC0ZZDvvnmeoy2DCTneiRMnuOaaa7zfHzt2jJtuuikhx5ZIJHNEZpH4WrldlmE0ElFjnwRuUlV1TFEUM7BXUZSXVVU9MNsD/sML9ZzqHAl7n1GHkzNdo3hUMChQV5xJZmpoffia5Vn83w+sDXvMtWvXcuHCBdxuN0ajkb/6q7/i8ccfn9W/QSKRXCImtMQut1IGdY24M3ZVMKZ9a9b+qPEeNxIjDhce7Vk8qvg+XgwGA2vXrqW+vp7f/OY3lJeXs3HjRhobG3nsscf4yEc+EvdzSCSSBDMxJL6Odc/veSQRCVHFKIpiBI4CK4Dvqqp6MJ7jRcqsQZRhPvbDAzhdHswmA995cAObKnLjeVoAtm3bxr59+/jP//xPXnnlFQCqq6t56qmnZGCXSJIRhwzsgSQksKuq6gbWK4qSA/xOUZR1qqqe9L2PoiifAj4FUF5eHvdzbqrI5eef2MaBxn62VdsSEtRBBPZHH32Uz3zmM5SUlCTkmBKJZA7xZuw983seSURCdeyqqg4pivIWcDtwMuBnTwJPAmzevDkhpZpNFbkJC+g6dXV1pKSk8MUvfjGhx5VIJHOEwyewe9xgMM7v+SQBiVDFFGiZOoqipAG3AGfiPe588Z3vfIevf/3rpKene2/r7+/nz/7sz3j33Xf5+te/Po9nJ5FIZqBn7Kob7APzey5JQiIy9mXAT7Q6uwF4RlXVFxNw3EvKhQsXuOuuu7jmmmt45JFH/H5ms9n43ve+N09nJpFIwjIxCIpRBPaxbsiIuDlu0RN3YFdV9X1gQwLOZV6pqanhzJkFe6EhkSxdHEOQVw3952CsC1g332c078jJU4lEsrCZGIKCVeLvsoEKyMAukUgWMq5JcE1Afq34XkoeARnYJRLJQkZvnGYtB0umzNg1ZGCXSCQLF13qmJYLGYUw2jW/55MkyMAukUgWLnrGnpoDGUUyY9eQgX2hIDfESCQz0Q3A0nJExi5r7IDcoLQwaDskNsS4nWBMkdakEomObykmsxguvDG/55MkyIw9gP/6r//i05/+tPf7r3zlKzz88MPzeEaIjTCuSbH6S26IkUim8SvFFMLkCEzZ5/eckgAZ2AN45JFHeOGFFxgaGuLFF1/kD3/4A08++eT8nlTltaBo/1VyQ4xEMo2esadmixo7wLiss8vAHoDVauWhhx7iy1/+Mp/73Od49tlnSUtLC3rfpqYmrr/+ekBsW1IUhf7+ftxuN+vWrcNuT1DmULYFijQr44d+KcswEonOxJCQORpNkFEsbpMN1AVeY287JMoSldcmNNh9/OMfZ/Xq1fz+97+npqYm5P1ycnIYHR0F4IknnmDbtm0MDg6yb98+br31VqxWa8LOCVUzxPTud5RIJDiGROMURCkGpOSRZA3sL38Juk6Ev8/kCHSfFHVnxQBF6yAlK/T9iy+DO/4lqqf/x3/8RwoKCnC5prcyNTY28rWvfY3h4WGeffZZALKzs7Hb7fT393Px4kWuueYaBgcHefLJJ/m3f/u3qJ4rahzD4uvEAFCV2GNLJAuViUFRX4fpUoxUxizgUoxjWAR1EF/1wBcnjz/+OA6Hg2eeeYbvfOc73tv1LUq+GAzi1/eDH/yAxx57jMzMTN5//33cbje1tbUJOR8v+r/PnpjF3RLJomDCJ2NPzxdJnizFJGnGHk1m3XYIfvJBoRIxWuC+H8ZdjnnjjTf40Y9+xP79+8nMzGRkZITjx4+zfv36kI8xGAw8//zz7N69m/b2dr71rW/xL/8S3ZVB1Hjc4goFtIxdIpEAohSTv1L83WCE9ALN4XFps3Az9rItQs9905cToutubW3lE5/4BL/+9a/JzMwE4POf/zzf/va3wz7OYrFwxx13YDKZyMrKYnx8nLvvvjuuc5mBHtRBLhKQSHyZGJouxYA2pCQz9uTM2KOlbEvCmqbl5eU0Njb63fboo4/y6KOPAmKL0pe//GXvFqW//du/BeD48ePe+z/22GM89thjCTkfP3zLTPb+xB9fIlmo+DZPQbMVkDX2hR3YLyHzukXJ4ZOxy1KMRCJwOsDlCMjYi6H71PydU5KwcEsxSwm/jF0GdokE8LETCCjFjPeAxzM/55QkyMC+ENADuylNZuwSiY7XACx3+raMIvC4pn+2RJGBfSGgB/bcSpmxSyQ6vj4xOplSyw4ysC8M9MCeVy0Du0SiE7QUowf2pS15TKrArupj84uUWf/7HMOAArkVshQjkegEy9i9gX1pSx6TJrCnpqbS39+/aIO7qqr09/eTmpoa+4Mdw8IuwWoDp12oASSSpY6vF7uOtBUAkkjuWFpaSnt7O729vfN9KnNGamoqpaWlsT/QMSxsSa154vuJATAvT+zJSSQLjQkfy16dlAwwp8OoDOxJgdlspqpKmlsFRQ/saVpgtw+IrewSyVJmYlBcyRqM/rfLFXnJU4qRhMGbsdvE93L6VCKZOXWqI6dPZWBfEAQrxUgkS51AnxidzCLZPJ3vE5BEQbBSjESy1AmbsUu5oyTZkRm7RDKTUBl7RqF4zyxh9ZgM7MmOxyNse1OzwZQiOv5y2YZEEiZj13afLuGl1jKwJzuTI4A6Lemy2mTGLpGoapiMXdOyL2HJowzsyY5uJ+AN7LlSFSOROCfAPek/nKSjL7VewsqYuAO7oihliqK8qSjKaUVR6hVF+XwiTkyiERjY0/Jk81QiCeYToyOnTxOSsbuAv1JVdTWwDfiMoihrEnBcCQTJ2PNkKUYiCeYTo5NeACgysMeDqqoXVVU9pv19FDgNlMR7XImGzNglkpmEy9iNJkjPl4E9USiKUglsAA4m8rhLmhkZu03c5nbN3zlJJPNNuIwdNC27VMXEjaIoGcBvgL9QVXUkyM8/pSjKEUVRjixmo6+E4w3sWeKrNQ9QpzMWiWQpEi5jhyVvK5CQwK4oihkR1H+uqupvg91HVdUnVVXdrKrq5oKCgkQ87dJAD+wpWmCX06cSyfTqu3AZu5Q7zh5FURTgKeC0qqr/Fv8pSfzQvdh1BzurJu+SDVTJUmZiCFD8LXt90R0eF+l+h0gkImO/BngYuElRlOPanzsTcFwJTNsJ6MiMXSIRpZjUIJa9OpnF4HEu2aXWcfuxq6q6F1AScC6SYAQGdt26V2bskqVMqKlTHd8hJd1jaQkhJ0+TnRmBXc/Y5fSpZAkTyidGZ4kPKcnAnuwEBnZLBhjMshQjWdpEzNiX9lJrGdiTncDArihy+lQikRl7WGRgT3YCAzvI6VOJZGIwuAGYTkommNJgdGku3JCBPZnx9WL3xWpbst1+iSSsZa+OomiSR1mKkSQbgV7sOtK6V7KUcdqFlDFcKQaE5FGWYiRJR6BPjI4sxUiWMpF8YnT0IaUliAzsyUyowK43T5foVJ1kiRPJJ0ZnCfvFyMCezITL2D0umBy99Ockkcw3UWfsxaIX5Zqc+3NKMmRgT2ZCZuxy+lSyhNGFA+FUMTA9fTq+9NxkZWBPZsKVYkA2UCVLk1hKMbAkXR5lYE9mwpViAOxS8ihZgsTSPIUlWWeXgT2ZCfRi19EzdlmKkSxFHJplb+D7IpDMYvFVBnZJUhHoxa4jrXslS5mJIXEVa4gQvtK1hT4ysEuSimB2AqDVFhWZsUuWJpF8YnSMZiE0kIFdklSECuwGo3hhy+apZCkSyU7Al4ziJWkrIAN7MhMqsIOcPpUsXSIZgPmSUbgkjcBkYE9mwgV2ad0rWapEW4oBbfpUZuySZEJm7BLJTGIpxWQWLcml1jKwJzMRM3apY5csMVQ19ozdPTktHV4iyMCerITyYtex2mTzVLL0mBoXPklRN0+X5iYlGdiTlVBe7DppucKX2um4pKclkcwrIewEjrYM8t03z3O0JeAqdolOn5rm+wQkIQhlJ6DjO31qXn5pzkkimW90OwEfVcxbZ3v4+I8PA2AxGfj5J7axqUL7eYY+fbq0GqgyY09WIgV2OX0qWaCEzK6jQe8r+ZRinj/eiUcFjwpOl4cDjT4lSj1jX2KSR5mxJyuxZOwSyQLhUFM/H/3BQTyqOjO7joYgpZjM1OkwZjYZ2FZtm75/ajYYU5ZcKUZm7MlKxMCuvXhlA1WygPjloTZcHjV4dh0NQZwdp9xCymg2Kvz041v8PygURZM8ylKMJBmQpRjJIsQ+5QJAIUh2HQ1BMvamvjEMCjjdKmZjkJC2BFfkycCerMhSjGSRoaoq9Z0jAKwsyoi9DAMiY1cMYMn03tTUN851tcLJ8WBTkPeDDOySpCGUF7uOKQXM6XLZhmTB0NQ3TvvgBAYFFJTYgzqIjN3Hsnd80kX3yCRXVuaxojAjeGkno1AGdkmSEMqL3RfpFyNZQOxuELtHd6wppql/HLdnFmP+AQZgTX3jAFTnp7O1Ko8jzYO43B7/x2QUi16U2xnbc7Udgj2Pi68LDBnYk5VwdgI61jzZPJUsGN5u6KXSZuXGugKmXB46BidiP0iAT4we2KsK0tlabWNs0sWpiyP+j/EOKcXQQD31e3hqB7zxz/CTDy644C4De7ISTWBPRiOwBZzlSOYOh9PN/sZ+rq8toLogA4ALfWOzONBQQONUBPZKWzrbqkTfaUY5Zja2AgefBFRQPeCeguY9sZ/rPJKQwK4oyn8ritKjKMrJRBxPQvQZezKVYtoOwY/vhtcXZpYjmTuONA/icHq4flUBNVpgb+wdj/1AQTL2kpw0Us1GCrNSqc5P52BjwHsiUw/sUWbsHjf0npn+3miBymtjP9d5JFEZ+4+B2xN0LAkszIy9eY9w0mNhZjnJyJ5zvfzrzjOzm9JMIt5u6MFiFPLGvHQLOVYzF3rjz9gb+8apyk/3fr+1Oo9DzQP+9Xtvxh7l9GnT22Dvg4LV4vv7noKyLbGf6zySkMCuqupuIIkizCIg2ozdMSwyjGSgcM30343mBZflJBtHWwZ55L8P8d03L/CxHx5Y0MH97YZerqzKxWoRU6LV+ek0xhrYVdUvY1dVlcbeMb/Avq3axqjDxWnfOnt6jDX2478Qz/Gh74nvxxfecJOsscdAXB4XsRJVYLcB6vQ03nwz0jn995v/fsFlOcnG7oYe9MRzVlOaSULn0AQN3WNcr2nNAWoKMmIvxUyNger2qmL6x6cYdbj8M/YqMfDk97syWcTVbTQ1dscwnH4BLvsILLsCssvh3KuxnWcScMkCu6Ion1IU5YiiKEd6e3sv1dMmjION/Tzw/f08vuvs3GdPkbzYdbzTp0nyhj/z4rSbnuoJf19JRIwGg9/fY57STBL2nBPv9+trC723VRdk0DM6yagjBgmibgCmlWJ8FTE6xdmpVNisHAiss0c7pFT/O3A5YP1HhR3Byluh8S1wTUZ/nknAJQvsqqo+qarqZlVVNxcUFER+QJLxgz2N8XlcxEIkL3Ydq6bnTYYG6sQQNO2Gy+8Xb6Ke0/N9Rguehu5RrBYxx/DQlrLZDfQkAW839FKclUptUYb3tmotGMeUtQf4xDT1TmvYfdlalcfh5gE8fnX2QhiNIrAf/wXkr4LlG8X3tbeBcxxa9kV/nkmALMVEwfiki8PN0xn6rDwuYiGSnYBOMvnFnNslNtus/gAUroaeU/N9Rgsah9PNm2d6uGf9cpZlpzJoj3G4JklwuT3sOdfHdbX5KIrivd2rjIlF8hjgE9PYN47ZqFCSk+Z3t23VNoYnnJzpGp2+MZqMve88tB2cztZB9IlMqdCwK/rzTAISJXf8JbAfWKUoSruiKI8l4rjJwn/vbWJ4wklxVgq56ebZeVzEQrSBPZn8Yk6/IMowJZtFE7XnjCgpSWbFOxf6GJ9ys2NtMetKsjnZsTB3dr7XPsSow+VXhgEoz7NiNChc6IkjY+8bozzPiinA+GtrdZA6u+7wGG6p9Xu/FD40lz8wfZvFKoL7uSUY2FVVfUhV1WWqqppVVS1VVfWpRBw3GRgYn+LJ3Y3sWFPEA1eWM2x3smZZCP+WRBF1YNete+c5sDsn4PxrUHen8PAoXA2uCRhqnt/zWsDsPNlNZoqJq2tsXF6STWPfOCOx1KOThLfP9mJQYPuKfL/bLSYD5XnWuDL2pr5xqvIzZtytJCeN0tw0Djb5LtwoEq/JyZEZ9weEsuy9X0LNzZC1DJgWS7TatsPABei/EP25zjOyFBOB/3zzPONTLv76tlWsXpaJR4VzPaORHxgP0QZ2SwYYzPPfPL3wpti/Wne3+F6XPco6+6xwe1ReO93NjXWFpJiMrCsVr4P6jhBBKYl5u6GXDeW5ZFvNM35WU5A+6xq726PS3G/31uoD2VZt41CTT509I8KQUtNuGOkQZRiEWOLBJ4VY4uPvaLr5BZS1y8Aeho6hCX66v4X7NpaysiiTumKRqZ+5mCSBXVGSY/r0zIuQkj2tWy9YJb7KwD4rjjQP0D8+xW1rhcLoshLxOlho5ZiB8Sne7xjmupXBxRLVBRk09sVgBjYxCIoRUjLpHJpgyuXxkzr6srUqj0G7kwY9CYtkK3D8F+L9tupOAH64twmnW4glmlwFDFgrZWBfLHz71QZQ4C9urQVEXTDNbOR01xxnTtEGdpj/6VO3C86+LNQDJou4LSUTcsplYJ8lr9R3YTEZuGGVCIj5GSksz07lxAIL7HvO9aKqcP2qEIE9P50pl4fOoSjNwPSpU0WZljqGCOy6uMFrLxAusOva9XX3gTkVt0f1+xBVAUfFzdC8FyZnMS07D8jAHoJz3aP85lg7f7Ktwtt1NxgUVhVnzn3GrtcBQ3mx+2LNm9b3ziE767v4t11nZ+r3W98RVwyr7/a/vWC1DOyzQFVVdtV3c93KfNJTpnd5rivJXnCB/e2GXnKtZu8VRyBeM7BoJ1B9pk69dr0hSjGluWmU5PjU2b1LrYME9vrnRP19/ccAePVUNxeHHXzh1lruvnwZHhX+p3+VsMlo2h3duc4zcpl1CL658yzpFhOfuXGF3+2rl2XyyskuVFX1k28llGi82HWseUKmNYc8f7yDzz19HIB/f+M8a5Zlsr48l5qCDG5rfYblxlSovhkDouF0oLGf+1KrKL7whvDANs6sr0qCU985QsfQBJ+/ZaXf7ZeVZLPrVDcjDidZqcn/+/R4VHY39HHtygKMhuDvkxotKF/oHeeGVVEc1McnpqlvnIwUEwUZKUHvqigKW6vyeLuhV7xX03KFmVewjP34LyC/Fko2AWJmpSwvjU/fUIPJaGD7ilb+7rcuPpdmJaXhFQx1d0ZxsvOLzNiDcLRlkFdPdfOn11eTm27x+9mqokwG7U56R+dwEi0aOwGdtLn3ZP/ZgRbv3xVgeMLJSycu8k8v1qOcfYnXp9ay+p/3cP033+SPvvcO39p5lsffM4LHuaCUBMnAzvouDArcsrrI7/aF1kA93TVC39ikd2VdMPLSLWSnmaP3jPHJ2HXzr3DJ1dbqPPrHpzjfMyb6URlBllr3X4C2A17t+tGWAY62DPLYNVVeGeWDW8r50t2X85ZrLcPvv4QncJFHEiIDewCqqvKNV86Qn5HCx7dXzfh5nSZ1PN01h+WYIIE9pE+N3jwNp8+NA6fbw9lusSzYqECK2cC/P7SR43+/g/c+WUiJ0k/Whnv5k6sqMBoVPKqoSZ52lYgDyEGlmHjlZBdbqvLIC0goFloD9W1tW9J1K/ND3kdRFKpjUcb4ZexjIevrOnqd/YC+BzWjcKbDY4B2/cndjWSnmfmjzWV+d3tsexWWutvIdfXy3Wd+jzpH77dEIQN7AG+d7eVQ0wCfv3mF14nOl7pisUT3TOCWlkQSENiPtgzy0R8cCO5Tk5YnJj4n5+aD5q2zvQxPOPni7XV8Yccqv+Gs7OZXQDGy9fY/5st3reFfP3IFJu2yu9VQiqoYZJ09Bhp7xzjXM8btmhrGl4XWQH37bC9rlmVRmJUa9n41BRmx1djTcpl0uWkfnIgY2MvzrBRnpU4PKgVm7B4PHP8l1NwEWctp6htn16luHt5W4dff0LnpA6IGbz/5Cv/y8pmkDu4ysPvg8YhsvcJm5cEt5UHvk2O1sCw71X9cOdHoC3s1DjT2M+nyBPepmePp018dbqMgM4XHtlfxmRtX+E/cnn4RKq72nsOmily+cKuoDf/dvRtR8mpkxh4DO+tF/XeHb2D32Ui1UBqoow4nR1sGQ6phfKkuSI/ODMzj0d4XObT221HV0I1THUVR2Fqdx8HGARGEA20FmnfDSDtc8RAAP9zTiNlg4E+urgh+vKzlqMWX80DOab6/u5En3pjb3lY8yMDuw/PvdXKma5Qv3FqL2Rj6V1NXnOnv95xoAjL2bdU2r3WFyRjgUzOHfjE9ow7ePNvDhzeWzBjbpu8c9J0V3jA+3LBK1IbTLEbNMyYBGfsSWbe3s76Ly0uzWa57n7Qdgh/fBa//E/zkg9ya2ULTAphA3X+hH5dHDalf96VamxzVVS4hmRoVjqFpOTRGkDr6sq3aRt/YpHhMRhGM9wmJLoimaUo21N1F/9gkzx5t58MbSyjMDH2VoazcQYX9JH98RTb/9moDP9zTGPEc5gMZ2DWmXB4ef/Usa5dn8YHLl4e9b92yLC70jjHlmqMmSkBg31SRS3aaUEJ88rpq/6x5Dm0FfnesA7dH5f6AeiMgdL8AdXf53VxhswLQ0m8XE6gDjcJyYLa0HYIf3ekNbos1uHcNOzjeNuQdSgLg/OtCYocK7ik2qfVA8jdQ327oJd1ijMpPaUWhroyJUI7xmTr17jmNIrBv9d2DmlEIqDDeC44ROPU8rPswmNP4nwMtTLo8fOLamX01P2pvQ1Hd/MPaLu68rJh//sNp/uWl05duT0OUyMCu8ctDrbQNTPA3t9dhCCHP0qkrzsTpVmPzuYgWj0e86HwC+8SUmyHN3W/IPuV//zkqxaiqyjNH2thcket14vPjzIuwfANkl/rdnJ5iIj8jhdZ+u8jYUaH37OxPpHmPUNdowW2xrtvbdUo09fwCu2+jz2DEtu4mILkbqKqq8nZDL1evyMdiihxeyvPSMRqUyA1UH5+Ypt5x8jNSopJ9VuWnU5CZIgaVMrXf7Vg3nPq9V7s+MeXmp/tbuLmukBWFmeEPWLIJ0vIwnn+Vbz+wgY3lOXxvdyPf2nkJ9jTEgAzsCFveJ944x7bqvLBdfJ3Vy+bQWmBqFFD9hpPaB+2AUGy92xqwLWmOSjHHWoe40DsePFsf6YSOo9PeMAFU2Kw0949Pe8b4LgaOlWVXTP9dURbtur2d9V3UFKSzolD7ELUPwMnfQtlWMFuh/Cqya7ezPDuV95M4sDf1jdM+OOG3LSkcFpOBsty0yIE9IGMP9GAPha5nP9jUj+q7Iu/4L8C2Eko385tj7QyMT/Gp66ojH9BghBW3wPlXsRjw/jtVkmvLlQzswA/3NNE3NsUXb6+LauioKj8di9EwN9YCQewEWgdEYL+q2saZrlEmpnx2nKblAErCM/ZnDrdhtRi58/JlM3945g/ia0B9XafCZhXnnFcthkLiaaC6tCsUczqk2YQt8CJjyD7FgcYB/2x97/8TSqe7vy2keO2HYcqe9Ba+uswx2sAOUSpjvBl77owF1pHYVm2je2SSdqeWLLXuFxPT6x/CrcJTe5u4ojSbLVrZJiIrd4jZkc5jbF9ZgEXrPxkMStJsuVrygf3NM938x5vn2FqVx4by6DzWzUYDKwoz5iZjDxPYP3jFcuFj0enzxjYYxX0TmLGPT7p48f1O7rpsGRlBZF+cfkFkOwXBxwUr8tK5OOzA4VHENpp4Gqgt+8CYAnd+E8a7oWXv7I+VpLx+uge3R50O7COdcOhJuOJBKFoDa+8V7pnnX+Xy0uykbqC+3dBLdX46ZXnWqB9TXZBOU9+4/8ajQDTbjFFDBn1jk37r8CKxrVoE7P3dWrg7/BSgwOUP8uqpbpr6xvnUdTXRT5KvuFlo38/tYlNFLr/45FZyrGaqCzKSZsvVkg7sR1sG+eRPj+J0qxxvG4qpPlZXnMnZuZA8BgnsbQMTpJmN3LRaXEoeDyzHWG0JnT596cRFxqfcPHBlkDKMfUCYIQV6w/hQmW/VztsOhXXxBfbmvVB6pTBoSskSl9CLjFfqu1iWncrl2nQpb39D+IPf8Lfi+4rt4v+4/jnWlSTvBKrD6eZAY3/YadNgVBdkMOny0BHODEwrxbSMi7p6LBl7TUEG+RkW9reOi8nVyWGouRGyS7z2AbetLYp8IB1rnnhNNuwEYHNlHp+9cQVnu0bnVi0XA0s6sB9oFLIsECu8YqmP1S3LpGvEweD4VOQ7x0KIjL08z0phZiqluWkcbwsM7Im17v31kXaq89ODZx8NO8Wm+LopK+zZAAAgAElEQVTgZRgQgyEAzXoDdbhNNIRjxTEMXe9D5XYwpwkFw6nfz9kwVkjmUG5pn3Kxu6GX29YWi4yx/wIc+x/Y/HHI1fTURhOs/iA0vMJlhSKwnegYCnPU+eFw8wAOpyeyfj3g91kTjRmYYwgMJi4MifdrtDV20OvsNg409qPqvauyrUHtA6Jm5a1w8bjXVOwjm0pJMRn87DfmkyUd2DeUifFkhSj3mPq8IL3e7InO2oME9vZBO2V5Qtu8viyHd1sDriwSaN3b2DvGoeYB/mhzWfBL0zMvQuZyoYgJQaVNvOla4m2gth4Q2uXKa8T36z8mShL1z8V+rNnS8g786A5445/nRG65u6GXSZeHHXrG+MY/ix2b1/1v/ztq5Rjbxd3aBGpyZIa+vH22F4vJwLaqMO+jhlfhv2/3+31Gtdha84lp7LOjKFBui77UA8I3ZtnI+2IgCWDv/+O1XS+QnWbm/mBXppFYeZv4ev41QAwu3n35cn73bkfkYatLwJIO7Lqs8cMbSyLvMW09KPTU2gtynSoC1ZlEN1ADAruqqrQO2L01y/VlOXQOO+gZcUw/JoHWvb8+2o7RoHDfxpKZP5yyC2113V1iBV4IcqxmMlNNmpZ9tbhxNg3U5r2i+Vp6pfi+9EpR27+U5ZgTzwrLBtUzJ3LLnfXd5FrNbKnMg4vvQf1v4apPT9vM6viUYy4rTc4G6tsNvWytyhPDacFwO+Gl/y2u+Hx+n7Z0C1mppvDyYc0npqlvnNLcNFJMUTif+rC1ysY2w2mvDYDqdqK07OPhbRVBrUMiUnwZZC6Dczu9Nz18VQX2KTfPvdsR+/ESzJIO7HpN/e/uXhO56fHeL4WeWntB5nYfwpZuSXwDVQ/s2iVj//gU9ik3ZbkisG8oF1cZ7/qWYxKUsbvcHn5ztJ0baguCe3xceF1of8PU10Fc+lba0mkZsEN2uVC0zKbO3rJP6IbNafqBhQtf6zuXzjVS1z6jiA+ZBMotp1weXjvdzS2ri0Qp4PV/hLRcuPrPZ97ZpxyzoTgl6RqonUMTnOsZC6+GeeVL/ntwtd+nMAPLCL/YWsvYQ+05jcTKwgxOpVyOS7GAYsSJiSOsDWkfEBFFEeWYC2+KDyzgitJs1pVk8T8HWubdR2ZJB/bDzQOsLMwgx2qJfOcJn0BqtKBUXUvdssy5ydgtmeKNjNaAZLpuvXZ5NiaD4l9nt+aBcxycjhmHi4Xd53rpGZ2c4Wzn5fSLovlUcU3EY5XbrLT2j2vLretiz9gnR6HzuKiv+3LFg0KR8N7TsR1vtpg0v++s5fDI81C2JWGHPtDYz6jDJdQwzXvFZf32vwxt2ayVY7Yr7wJxDCrNQc/gf/Y3A4T0R+fIj+DwD8WHVtX1InHx+X3WFGREzNjVtNyYNOy+GAwKqVVX8VnzVxnf/iUedn6Z6g03hrUPiMjKHWIpTusBQCQ0D2+roKF7jMPN8zuotGQDu8ejcqxlkM2VUciTPB5oOyj2LQJ8+Eko20JdcRZnu0ej39kYDQF2ArrUUa8pppqNrFme5a+MSdD06TOH28nPsHDz6sKZP3Q7oeFlWHVHVIszKvKstA9O4HJ7ZucZ03pQXLIHfohkLYfqG7UrqEvgiz3UJr46hqdLQgliZ30XVouR7Sts8No/iEv7LZ8K/QCtHLOy93VgloF9DiwajrYM8v3dwjPli799f6a6rOUdUYJZcQvc8g+ilDc5Iv69GtUF6XSPTDI26Qr+JBODTJoyGZt0xaSI8WVrdR67Rir4vwO3cdC1IrJ9QCSqbxDL5H12oX7giuVkpprmvYm6ZAP7uZ4xRhwuNlVEMZTQfhhGO+G6vxbfj4pR77riTBxOj2gSJoqAwN4+KCRgpblp3tvWl+XwfvvQ9AdKAqZP+8cmee10Nx/aUBLcAK15rzi3ENOmgVTa0nF5VDqHHKKBOt4rDJiipWUvGEzBM+T1HxVKm0thLzCsBfapsem/JwCPR2XXqW5uWFVAatOr0H4Irv/idNkpGFo5JqVxF9XZhtk1UE+/mHCLhgON/egvxRnTl0Ot8KuHIbcS7ntKzF2UbRU/azvovVuNt4EaImufGGJIFfeZbWDXxRHPHm1nc0VuZPuASKRkCndTn8ButZi4b2MpL5+8OLfLeCKwZAP7kRYRBDdHM1BQ/zsxJHPVZ8QKrbMvAT7WAolUxgRm7P128jNS/Bo868tyGJ9yc07fwJ6AjP1373bg8qihyzBHnhKBNpo9rExfYQhrAb2BGkPW3qzV1y1B3sR1dwlXvkvRRB1qmzZaS6C3/Lttg/SOTnLbmgJRW8+rgQ1/HPmBWjnm/pzTs8vYfdVJRnNCegabKkKoy6bG4emPig+Qh572LsmgaJ3ou/gFdlE3D6qM8XjAMUyfW3zozTawj/tcDbzfPpwYX5eVO8TvdHA6Q//jbRU43cJrab5YsoH9aPMg+RkWrxthSDweoZ1ecTOkZkHt7Vr2OsKKwgwMSoKXbgR4sQtFjH8Wt16TaXrLMXFm7Kqq8qvDbawvy6G2KEgW03JATJt6XPCL+6O6fPdKHgfs05LHaAPj1Dh0Hgtdy/fVtM9GHx8Lw22ihAAJ9ZbfWd+N2aiww71HHPemL0e3G1Yrx9zkeSf2BmrPGTj/KuRoDcM7/jUhPYNcq6ir33X5sml1marCc5+GrpMiU8/32eFqNEHpJm9tGkQiYFBCZOyTI4BK12QqFpNh2tY4Rg42Tb8/3J4E+brUarLHl7/ofV+sKMzgqmobvzjYOrNMe4ksqJdsYD/SMsjmirzIY8R6GWbNveL7VXeKS9kLr5NqNlJdkJHYNXkBGXvboN3bONWpyk8nO8083UD1WvfO7oX6Xvsw53rGght+AdT/ZvrvUV6+F2amkGIy0KL7YKflRh8Y2w6KD5HKME3a9R8TCp1Tv4/umLNhYkgElaJ1kFWSsIxdVVWeP95BZY4Zw1tfF9K5NR+K7sFaOaZmcB+pTMaWtb/xT2DJgA99T3xvjqNx6ENDt3j9f/oGn0Use74Fp56DW74KtTtmPqhsG3Sf9A6bpZiMlOVZuRAsY9d8YtomUqi0WUMux47EtmobqWYDRiXKuZVosA8Aiug/PbUD/uNKeOo2nnD+X742/lUGnroPfv0o/PZP4RcPztDwzxVLMrD3jDpoHbBH1zg99Zwow6y6Q3xftkVkyGdfAWBVoq0FfAK70+2hc2hiRmBXFIX1ZTk+gT2+UswzR9pINRu4+4oghl8w3TRWjFFL/gwGhQqbVWTsiiKy9mgDY/M+8VxaLTbovtfSzXOvaddr6jllWgM4MRn7797toGtkkquGXiRlrI1zl/1V2LmAGay9F6PLzg2G96IP7G2HxXDZ1Z8TJS7FAP2J2QB0rnsUg+Kz0ejMSyJ4XXY/XPP54A8q2yqkwx1HvTeFNAPTFGmNY+ZZl2FA7DX4+Se2zVjxGBctexFFKABVWEGYLOSlqhSaxnD0NIqrltZ3REIUoOGfK5ZkYD+qSZEi/sd6PGLKUS/DgGj+1N4mBhPcLlYXZ9I6YA/dzY+FAC/2i0MOPCpeDbsv68tyaOgeFc9rShE1S3vsNcOJKTcvHO/kzsuWhfa3HmkXWfdNX45J8leelz7dWC7QPGOi0fc27xWTrSmZHG0Z5MEn98/0u1YU2PCxudW0D2tTitnlIrD3Nkxv34mDZ4+2k4aDPzf9joOeOnZNrovtAFo55iOpR6JroKoqvPZVSC+Abf9LvF6yyxIW2Bu6x6i0pZNqNor/499+Uvz/ffDfIdQVcdmVgCLUTxrV+ek09wcxA9OG7y6MmWalYfdlU0XuzBWP8VB5rfh9KkYwpYmroUdewPDYLl65+mmuG/sabR/bDX9xAh7+nbhPDAnSbFlYgT1B9anDzYOkmAysXR5CL6wTWIbRqb1dvNjap60FEpK1617sWmDXpY7BnPLWl+fgUeH9dp+sfRYZ+yv1FxmddIUuw6iqkKvV3AzX/lVMNdlKzb5XVVURGCeHhXNhOKbsIovTyjAHGvtxulVUxECPX1308gc0Tfsvoz6nmBjyzdjXgntSbISKk/aBCb5i+hkFyjB/ULezrSbyDgA/tHLMdvUIDW3dke9//nWRWV73N5CiBUbbioR9IJ7rGRU+8vYB+OVDouH94C/CK3xSs8VVXNt0nb26IAOH00PncIAZmFaK6XdbZ6Vhn1PKtohkJ0jS89CWMgyKws8Ptka8b6JZOIHdq7/9x7jrU0dbBriiLCfyhpdTz4lP1lW3+9++4mZx+9mXqFsmmo0JGVQKsBOYDuwz3yDrS7UGql6OScudVfP0V4fbqLBZvSvEZtB7RtTuw9W7Q1Bhs+JweugZnYy+gdp+WPQwKsRgku95qcA23/PMWi42zB+fI037cKvwbUkviM8awYfG3jHKhg7yUdMbqMBXLT9jk+Fc7Adaey+pqoOqof3hG6geD7z+VdEw3fTo9O22GhHY45yQnHS5ae63c0PaefjBTaJ89cDPxf9NJMq3QvsRUb5gWvI4o86ulWKG1fSIC6znhbItQZOeZdlp3FxXyDNH2ph0ucPeN9EsnMDevEc01CCu+tTElJv6zpHIMkevGuaWmZOAKZliIvLsK5TkpJGZYkqMtUBAYG8btGMyKCzLnhnYc9MtVNqs08qYWVj3/uH9Tg40DnBNjS10E7lln/hacXVMxwYo15QxzX3j0QfGln0iCy/fBoBNm2QszU1DVREfEr6s/6goFTXvjvn8IjLUBtmlHG0d4nunTKgocTdQXz7ZxYeMe1EQlVmDxzm713LFdqZScrnLeCB8nb3+t9B1Am76Cph8JqxtK8QV4lhP7M/tQ1PfOFeoZ3nw1GdgsAnxr4ryw6Jsq2hOa7/Taq/kMaDOrmXsw6THVWOfDx6+qoKB8SlePtEV+c4JZOEE9sprp1+YcaxIO942hMujRm6cth+GkY6ZZRid2jug/xxK/4XEWQsEydhLc9NCqgA2lOdyvG1IlDpiLMUcbRnk808fB+A3xzpCa3qb9wk3x9zYp/Qq9cXWA3ZxfhnFkQNj816xDk/raehB6z8/tpHaogz+dedZnG6f7HzVHGrah9sYMhdx//f3883XW2hRixhsfi+uQ75ysoucdE2NEk+t1WjCU/dBbjK8y+nWEEHD7RRNzKJ1sO4j/j+z1YivA/GVY851j7HNcBpF1ZIu1RP9B5V3UEmUY/IzLGSmmmZq2SeGcCkmzKnp5KVHYf+RRFxTk0+lzXrJJ1ETEtgVRbldUZSziqKcVxTlS4k45gzKtsAjL4ox5LyaWV/KHNUGkzZG2pYUqgyjo9/e8DJ1xVmcuTgav/FPQGBv93F1DMb6shx6Rie5OOyI2Qhs3/m+yF70en294urQTbAwLM8RH0reBmokZYnTIS7NffTrJzuHMRsV6oqz+Jvb6mjsG/cf/DCnwmX3iY3zida0D7Xx/lgWbo+KR4WznrK4SjHtg3ZOdAyzKn0McqvjrrWmXvFh0pVJ1IZXg9/h2E9EFn3z389U3eRpgT3OBuq57lEOqXXadzEapeVWiqa81kBVFCW4MsYxxLiSQVVBZvRbjpIEg0Hhj7dVcKRl8JIu4Yg7sCuKYgS+C9wBrAEeUhRlTbzHDUrZFtj6p9B3Vowqz4IjLYPUFkUw/tLLMDU3hzZkyikXmdDZl6lblsnopCv8BphoCJKxRwrsoC24tuaJx3vcIe/vi/4RZAin6R1ohLGuWdXXQawQLM1NE/a9IOrsvWdD18M7jogGpY/x16nOEWqLMrGYDNy8upArK3P59mvnsE/5qFO8mvYE+rQ7HTDew2n79P//eaWMHEfbrM3WXjnZBagsHz8NVdfGX2ut2M6oIZvqntdm/mxqHN7+JpRfJaYjA8kpFz4ncTZQG7rHSMlZLgR/dXfF9kGlKOK+PhOo1QXpQTP2QTU9+RqnUTIfSzgSkbFvAc6rqtqoquoU8DRwTwKOG5w12qFPPR/zQz0elaMtg5H9YfQyzNoIQyOr7oDWA6zNFYEq7jq7T2AfdTgZtDuDSh11Vi/LwmIycLxtUJs+Vf1dKMOw91wvxVkpfOHW2tCaXm99fXaBHYQr5XRgXy0CsK91qy/N+wBFBCPEIM/JjmHWaeolRVH40h119I5O8tSepunHlWwSVg+JLMdoUsezjhxu13aRVq65EkX1iMRiFrxysotbCscwTg4JHX68GE20FN3CVtdhRkYD6uwHvwdj3WJAKFiWazBCXlX8GXvPKFdnaXX6az4f+wdV2TYYavH6L9UUZNA14vCTD7vtg/S7rQuuvq6jL+F47hIu4UhEYC8BfE0R2rXb5oa8aii+fFYThw09o4w6XJEbp5HKMDq1d4Dqpm5U1AjjrrP7eLG3DYjsP3A4yReLycDa5VlCGaNPn0ZRZz99cYTDzYM8tr2az960MrSmt3mfUITk18b0z/Cl0ia0yULyGEEZ07xHTGFqniKdww4G7U7WlUz702yqyGPHmiK+v7uR/jGtker1ad+fOE37sLgidGaW8MRHN1CeZ+WN/vzw5x+GnhEHR1sHub9Yq4eXbErIabpX30O6Mknn4Remb7QPwN7viNen1oQOSpySR10Rc5lF+zeFWG4eFv38NHsBPStv8snanWMDDKsLr3Hqy8NXVTB+CZdwJCKwByt6zSg2K4ryKUVRjiiKcqS3tze+Z1x7r3DD0wdIouSINpgUtnEaTRlGZ/kGyCgitXEnZXlp8VsL+Hixtwb4sIdiQ1kuJzqGcaVqBktRKGN+ur+FFJOBP9pcGv6OLftmXV/XqbBZGXW4GLI7p9/4werUrklxpeRThqnXGqdrS/z/H/7m9lXYp1z8x5s+2eblDyZU097aKLLyazdvxGw0cP/mUn7floJqsMyqzr6zvgtVhS2WRjHWX1AX+UFRULbhVvrVTMxnfBKdfd8WapOb/y78g201otw2S6loU984bo9KladVWC5Eer8Eo/hyISnV958WasoYH292j31wQSpifNGXcPxgdxPfffNcYgzIwpCIwN4O+E63lAIzplBUVX1SVdXNqqpuLiiIbYv5DHSlyukXwt8vgKMtg+RnpIQPlh1HoivDgGhI1d4G519nbVFa/GZgPnYC7YOhNey+rC/PweH00GzXFhxEaKAOTzh57t0O7lm/PHyfYahVaJLjKMMAVPiagaVkCD11sIy34xi4HAGN0xEMCqwu9neUXFGYyf2by/jZgRbvIhKylokP4yM/ht3finuIrf50PW4U7rxGZNYf2VSGRzHRm1o+q4z9lfouagrSyRl4XyQEhthWu4UiL9PKHtNVlPbuFsNdwx1w8PtieKtobYQH14iexkhsCZLOuW4RfAscTbP/oDJZYPlGrzKmQjMD89WyG6aGF3zGrigK163Mp3XQzuO7GvynqOeARAT2w8BKRVGqFEWxAA8CsRfAY8FWA0WXxbzU+EjLAJsrcsN31ut/F10ZRmfVnTA5wi1p52nqG8fhjK55GRSfwN46YCcz1UR2WnjHP30h94kBLVBEKMU8e7SdCaebP7mqMvy5NMdfXwe87pnTypgQnjG654aPXr6+Y5iagoygOzT/4pZaDIrC47t86t1lW8DeC298La4htrYBOxO9TYxZCsmwig/W4uxUblxVyLGJZagxZuyD41McaBzgrjW5wjckQWUYnabCHaSoDuHc+PY3RAP9xv8T+YG2FeLrLMsx57pHMSkeUobOT88pzIbyrWLf65SdFJOR0lzrtDLG48HiGsNtySI9ZRa7SZOIFJMRBfCoQXzrE0zcgV1VVRfwWWAncBp4RlXV+niPG5E194hP+Ugj6ho9Iw7aBiYSV4bRqboeTKlsmjyIR4XzPWHWe0UiILCX5VojyrtKc9OwpVs43K3dL0zG7vGo/OxACxvKc1hXEuHf17JPrMErjE/gpF8dTTdQ66CvAVxT/nds3isyTOt0Y/tk53DI8yzOTuXj26t47njn9ICOR29MxWey9ON3milV+kjLr/S7/f4ry3h/ajnKcHtM0spXT3Xj9qjcU9QvzjERjVMfUlZeS7+aiWvvv8O7P4MrH4PcKHZ52uKTPDZ0j7EtdwzF5YivtFS2VQwfdoqVfzW+ypjJYQyomDIS4MQ4z2xfWUBKot0lQ5AQHbuqqi+pqlqrqmqNqqpfS8QxI7I2tnLMkZYojL+8ZZgQQ0nBsFih+kZKe94C1Pi0qj5e7G0DM+16g6E7PR7smBTytTAZ+97zfTT1jfMnV0Xxptfr67G4DgYh1WykOCvVX/LocfkPxridIrv2uTroHZ2ke2SStctDL/b4s+tryE4z882dWta+4lbtJ7NfPD3icPKrw23UWAax2Px/TzfVFdKZUq2d4Jkgjw7OyycvUpqbRvWkdqVSktjAvq7UxivuLZg6j4g+Q81N0T0wcxmYrbPO2Bt8FTHxZOwBg0rVBRk09Y0JMzDNAMyaFaOfThIyJ+6SIVg4k6eB5K8UQSJKdcyRaIy/6nU1zB2xncuq2zGPtnGZuTO+bUpaxu7xqLQNTni3EEViQ3kOF/rseNJywzZPf7q/BVu6hTsvC2HPqzNyUTTVZmEjEIxym9V/SAn8G5Cd74LT7qeXr+/UGqdh/r+y08x89sYV7G7oZd/5PlGKWbY+rsXTvzrUxsTkJHnuPmH+5YPZaKD2crH3dKQlugnUEYeTvef7uGNdMUrHUdFkzIrw+4+Ry0qyOatqjXCPC555JLoylKJoDdTYA/uky01Lv53LzBfFDbNRxOhY84TyShtUqi5I95qBjQ6KdYpZuQs/sMMcuEuGYOEGdhBN1JZ3vBrYcBxpGWB9OOMvj0fIHGMpw+jUinr8/Zkn4pM8aoG9d2ySKZeHstzwjVOd9WXiRTJhyg5ZimkftPPGmW4e3FJGiilC4y4B+nVfKnVfdhAe6orRv87evHfG89V3it/jmjAZOwgZ2fLsVP7l5TMiwyu9UixvmMXiaZfbw4/2NXFbuYqiuoW1bQC3X30lY2oqzaePRHXMN8/04HSr3L6uWLhWJri+DsI3aHmqC4/u0xJLGSqvZlalGK8iRm0Vv6eUOPeHlm0VSjePx29NXnePeG/b8oMsWJeEZIEH9nsANWI5xj7lEsZf4errsynD6GQWw/KNXKce5fRsrQV8vNjD2fUG4/KybBQFhtRM76VrILp16Ee3RlmGsWQKKVoCqLCl0zs6KXZOmlNFlhgY2AtWQ/p0VlbfOUyFzRqxeZxqNvKFHas40THMSycvisxvciSqD/tAXjrZReewg/9vrdaky5kZ2KsLs+i0VOLqqo/q//nlE10UZaWwIc8Ng80Jr6/rDBdvYwpz7P4zthViX6c7tsGZBk0Rk29vTIx0s2yreO32n/M6ODb2jtHfJ0o9RUXF8T/HEmJhB/bCOvGiilCOOd42hNujsjncxOlsyzA6q+6gfOIUhvFeesdmsZ3cx4u9tT+2wJ6VahYTey5r0Izd4XTzq8Nt3LK6iJJo9kW2vCOUCsbEqBB0ZYz+geXnGeN2iZHyANuCkx0jYevrvnxoQwmrijL5151ncdm03Zp9DTGdo6qq/HBPI9X56WzK0Rrg2eVB72tetpZyVwuHmsIrkOxTLt5q6OG2tcUYLh4TN85Bxg6QseJqPjr5f9hT9qecue1n0ZehbCvEVp/B2Mbdz3ePYlbcpAxfiK++ruMzqFSQkUJmiokLveOMDIqZl6JCGdhjYUEF9qMtg3z7tQZ//eeae0SGGcZ+VN+YFNL4K54yjM6qO1BQudH47uysBXzsBNoG7SgK0QVhjfVlObRMpKIGaZ6+dOIiA+NTkSWOAON9ojGYoDIMQEWepmX3lTwONIFzQpO5jfkNJg3bnbQO2CMvQtEwGhS+eMcqWvrt/McJ8ZJubTge0zkebh7k/fZhPr69CoO+Ei87+AB1Se1G8pUR/rD//bDHfPtsLw6nR5Rh2rXG5rL1MZ1XtKSaDRxTa3nk3Hbufd4ZvUZ6lsqYhu4xrs4dQXFPJSaw21YIW4y2gyiKQnVhBo19Y4yPiJ6RJXPhq2IuJQsmsB9tGeSB7+/n26+d42M/8BH3r7lHWIWGKcfoxl/Z1hCX9fGUYXSK1uHJLOFWw9HZ1dl9AnvrgJ3irFSxaixKNpTn0OW0iuZpQIngp/tbqC5I55oVUbw5Wt4RXxMY2MttgZLH1YAqDMFagtTXL4rfRURJpg83ripkzbJMvn1wlFE1jT3v7I1pAOSHexrJtZq5b2OpGMyy2sQmoCBYlotVdi1njjA8EbqE8fLJLvLSLWypzBOvscI10xuMEsyIdh4xa6R1LXuMDdSGnlGuytQmyBNRilEUUY7RDMFq8oXk0Tk6gBNz+G1MkhksmMB+oLEft2YzO+n7wi1cIxpyIcoxHo/KsdYIxl/xlmEAFAVD3Z1cazzJ+Y6+2B/vE9jbBybCmn8FY31ZDoNqBorH5d38DmJ13vG2IR7eVhGd5WnLPrGXcfmGmJ4/HNlpZnKt5ukGqq9nTPNeURfPmG6OndIap9GWYkDIPsUVmcIFdTmVdEQd3Jr6xnn1dDd/vK1CDEMNtQVtnHrRzr/K08rz7wWfo5h0uXnjTA+3ri7CpDBnjVOd62oL0W37Y9JIW/PEvEIMGbtXEWNJgCLGl/Kt4jzG+6guSOfisAN1YhCHKfrXgUSwYAL7tmobKWaDdz+Ld/eEooisvXmPKCMEENH4azZDSaFYdTtpTJLasS/2xwZk7NHW171PXZTJuFF7A/iUY366vwWrxch9myL4wui07BOLhk2JXWhQbvNZbJ1bBcYU6D4pzJ8qAuvrwxRnpZKvbU+Klg9tLMVsVDivllCjdGI2Rudx86N9TZgNBh7W9f3DbUEbp17SC1CtNramd/Orw8Hto/ed72Ns0sXtlxWLbNgxPGeNUxAyuk9eKzT23/jw5bHJ6WwrYgrsuiKmwtMqLCJCXNnETJlWZ2876FXGZKhjuFPifF8uQRZMYNfF/X9560qq8618f3Ph6oIAABzTSURBVHcjF/Wlt2vvFeWYMy/OeFxE469jPxVeGcWXxX+SldcyabCyenif/5afaNAC+6Qpk64RR1TDSb6YjAay84rEN1oDdXB8ihfe6+TeDSVkpYZXlwBCldB10rtvNJFU2nzse40mKKiFk78RCpZK/+c72Tni5+gYLZsqcnn6U1eRV7GOYmWQJ146xuO7znqv9IIxZJ/i10fauWf9cgozU0UZa6gtZOMUAEVBKVzDprSLnOwYCbqa7uUTXWSmmrimJl/U1yHhg0mBPLZdbLlqj3UvgG0F9Ee/pNuriJloTEx9XWf5ejFk13bQuyYvm3EM1pzEPccSYcEEdhBv3M/dXMtTj27B6fLw+aePizdt0Tph5xvEO+ZI8wAFmSGMv1oOwB/+Uvz9nX+P2zgKUwp9Rdu50XCUpsAtMJHQAnvnpMiUI5l/BWPZMrFAeGpMXLmIJbqe6CZNQbNOVRM2mORLRZ6VzqEJplzaB17hGhjVLuV9Art9ykVj71jUjdNANlXkctN2cbxPrXHxxBvneeS/D01b/Abw84OtTDjdPHattvrP3i8848Nl7ACFqymwX8BiUvw3OgFOt4dXT3dzy+oiMTfRcURzdExQySLUKWWlsqE8h131MUo9bTUiuZmyR3X3892jpBjcpAwlSOqoY04Twb31IBU2K4oC2co4lowI+xMkM1hQgV2nKj+df7p3HYeaBnjijXNaOeZeaNo9Q+53pGUwuPGXqsLO/yMyfRA63ll6i/ii1N1BsTJI55kDsT1QC+wtY6JhGmvGDlBZJoLRxc4O3B6Vnx1sYUtlHnXFUWa/LftEr2EOSgYVtnQ86rRrpTcgpOX5bcM6fXEUjxpb43QG+SKA/vnlKt+47zIONQ9w9xN7Odbq30ydcnn4yTvNXLsyf/p3pJ9LuBo7QOFqFOc4H12l8Lt3O/zM3w42DjBkdwo1DIiMPYGOjuHYsaaY99qHp69mo8G7/zS6rL2he4yrcoZRPM7EZuwgGqid75KquCjIsJCtjDOuzE3DeTGzIAM7wIc3lvLhjSX8++vnRJNszT1Cj+tTjukecdA+ODGz3qiqsOsr0HkUDKb4lgoHkL/hA7hVqDz2jdiuADQv9rYhYY4Va40doLZaZOZdXZ283dBD28DEdN04Gpr3iXLBHCgQKgKVMQZNIz8x4OfEOG0lEEfDLLdSXNL3nuWBK8v57f+6GrPRwAPf38+P9zV5B4teeK+TntFJPqHVpgFRX4coMnZhiftA+RijDpe29k7wSv1F0sxGrltZINbodZ+c0/q6LzvWinLca6e6o39QjMoYoYjR5MWJzNhBBHb3JGfe3UPf2BTZjPOH84459y9fbCzYwA7wT/eso8KWzl88fZzBrNXiDe2jjpmurwdcyr39Ddj/H7DlU/DoS3EvFfbFMtyEoihUjBzB8+MPRB/cNTuB1gE7KSYDBTE2DgGKC5fhQWGwr5uf7m+hIDOF29ZGOdgxOSo05XNQhgEfX3a9geo7IeszAl/fMUJeuoVl2amzfzKjSWSh2pDSupJsXvjsdq6vLeCrL5zic08fZ2zSxQ/2NFJblMF1K318SIZ0DXukwC4C2ipDOxU2K09rTVSPR2VnfTc31hUIhU3X+8K/ZQ4VMb7UFGRQU5DOrlgCe572wRZFA9VfEaPEtV0rKNqgUv/p3RjwkKXYGfRY59TidjGyoAN7eoqJJx7aQP/4JH/9mxOoq++Bxre85ZgjLQOkmg3+2d87T8BbXxcLkG//hpBYxbtU2IeO47tAVYXvsmtKfB8NPoG9LM+KwRCdosMPg5EJQwZD/V283dDLQ1vKQ3vjBNJ2UFzxzHJxdSTyMyxYLcZpyeOqO8TmnICrpZOdw6xdnhX/Nvr8Wr/p02yrmScf3sxf37aKP7zfyXXfeJMzXaPcurrI/7mG20Q9PC2CqiQ1G7JKMfSe5v7NZRxoHKC5b5yjrYP0jk5y+zrN6OsSNU59uXVNMfsv9IfV2PuRkgkZxVE1UKcVMW0ikbLEfmUZloxCyK1ijes0+SZRThozZMypxe1iZEEHdhDZ2JfuWM1rp7v5g1vzdT77MiCGmq4ozcFs1P6ZR/5blGDW3AsffCJuS9pg7HevwYkoM3gwsN8dpZ+5FtjbBiaiNv8KhjMlB6tb6MAvi6UB2fKOCLKlifmAC0RRFCps6dOlmLIt8MgLfldLky43Dd2js26c+lGwSky3+vi+GwwKn7lxBX9/9xoG7OL2p/Y2+V/m6xr2aD5YNGuEj2wqxaCIZvXLJ7qwGA3cVKfp8juOzImjYzh2rC3C5VF562zoaewZ2KIzA/PziEl0fV2nfBu5/e/yw/uFPcRHr79izt0QFxsLPrADfPyaSm6uK+QLexWmMsvg1HMzjb/e+xW8+AVYeRt8+Adz1siq2nAjj7q/gkM1s8dzOVUbbozugY4h1NSsqH3YQ+G05JLLKKoKf/70sehrk837RINvjiYjQShjvKUYEMHd52rpXPcYTrc6K6njDPJrxRVIkLrx+JTbOwfhdAdMaQ63Rq6v6xSuht4GitJN3LiqkGePtrOzvotrV+aToW/7aT9yycowOutLcyjITGFXfSx19ugC+7nuUVINbizDCVbE+FK2BcZ7WYf4v6sqDW7tIAnNogjsiqLwr390BbnpFn7n2IR64U1OnG+dNv46/QI897+ErO7+nyR8+MaXTRW5/O9PPspB4ybWmtrYVB6lBtcxjNOcxeika1aNU50xQxa5isiqoh4tn7KLycg5qq/rVORbaRuYCKkr1xun6xKRseu13yBmYNuqbVhMITbZRJo69aVwjdgZOtDIA1eW0TM6ScfQBHXLNAvb8T4YarlkjVMdg0Hh1jVFvHW2J/pVjbYVYO+DiaGwdzvXPcbVOUNiwnmuMnZ9UKlhp/iaKnXssbIoAjtAXrqF//fAep4e34jicTJ8/PcoCmxxH4dnPw4lG+GhX14Sz4lNFbmYa2+iWO2l7UKUWwIdw4yoIqDHE9iz8grJVcZiW7/VcUSsbKtM/GCSLxV56Uy5PXSNOIL+/GTHCJkppriuWLzkay6PvTMDe8hNNpOjYotVLBk7QM8pcnx8iH64RyvvzEN9XWfHmiLGp9zsvxBl0zFPlzyGV8Y09Iyyba4UMToFdZCSLXa4QuR+h2QGiyawA1xdk8/262+jQ7VhaXiBm63nSfvtw6Le+rFfx78MIAaqt9wFQPOhmdOwM9C82Ac84kMnnsBmK1hGsdke2/qt5n2AMm2dOkd4JY9940F/frJzmNXLs2bXOA7Eki4y776zQX8cdJPNcLv4Gm3GXrBKODb2nOZw8yD6Wbv08k7HEdG3WD43jo7huKrGRkaKiV2nohxWimKxtd/WJMWQeEWMjsEgbC105VSazNhjZVEFdoDP31LLwdTtXK0e57uuf6TTlcF7N/zokn/qF1etpcdQgLlld+Q7a17svU4R2OPJ2LHmYnTZ+cz20ugbTi37hKVCvF45EfAG9oGZE45uj9gXm5AyjE6AMiYi0UoddcxpQirYc8rrZeR3pdSuOTomykslBlJMRm5YVeBdpB2R3EpACRvY/TxicqvE0pS5oswnyZClmJhZdIHdZDSQumw1FsWNBRf5DHPuzIlLfyKKQk/B1ax2vEvfSIRRbW3q9OKkhbx0y3TjbTZYtdJLmKXWfrgmof3wnJdhAJZlp2E2KjT3z8zYG3vHcDg9iWmc6hSsgr5z4oooGoa1qdNoSzGgKWNOzyzvlGVDxzEovbSNU192rC2mb2yK421RNNDNqeLfHaaBqitibHOpiNEp1xZcm1Ln9gNkkbLoAjvAFXlu3KqCooAJN1cZT0V+0ByQs/ZWshU77x58K/wdtcDeMWGJS+oIiBF9CLn7dAad74LLMeeNUxALMcryrN4NUb6c1Bun8VgJBJK/UizJHmmP7v5DbWJiNSOGbT2Fa0Rd2unwL+/0n4fJ4UuuiPHlhlUFmI1K9OoY24qwNfZz3aOkGVxYhpvmrr6uU7IJMIgJ5Xg9nJYgizKwl2zYgWJKwYMRg8lCyfod83MeG28DYKT+1fB31AJ707gpvjIMCH9tgEPfj+4NoS+SLp/7wA665DFIYO8YIcVkoDo/gWULzTMm6nLMcJvYmhTLfEPhauE3FFjL7zgqvs5D41QnK9XMVTX57Kzvim4Pb16NKMWEuK9QxAyKRd9znbF31wOq2K7lYzkhiY5FGdgp24Lh0Rcw3PxlDI++kLCp0lhRMgrpSlvJ8oGDYpFzKPTAPpoARciwtvjh2P/AT6KwNGjZpy2SvjSTfRWaL3tgoKnvHGb1sixMxgS+JHU3xSDKmKDEInXU8V0a4sslcnSMxI41RTT32znfE4XbqG2FsFEe7w3644aeUa7KSODWpHA07wG9He1jOSGJjsUZ2GHG8Mt84am6no3KWfadDr6QAfAG9gFPWvwZ+6B+Ka2KEsvvPysmbofaZt7X7YLWmYuk55IKm5XxKTf949MToR6PSn3H7DzYw5KeL0pTIZQxMxhug5wwPuzByKsRlgg9AeW+S+joGI5b1whTsKi8Y8IoY3RFzFpLp1D66HLSuaLyWjClJNSgbymxeAN7klB0xW2kKC4aj74e+k5aYB9R0+PP2FfcqnmwGMSbYmIIXvxL+PY6+O422PV3wt7YNQXHfwHOccgoiu85Y6Ay0AwMaBu0MzrpSoyVQCD5taKBGgnXFIx2xZ6xG02i5OObsTsnLqmjYziKslJZXxalR7sttBlYY69QxFS6W4USyBS7SV1MlG0RVhMJNOhbSsQhv5BEg7HqGlyYSGnbjcv9WPBSgxbYx0iLedfpDHQPluY9IsspvVLUmM/tgnOvwoH/EktFTFaxUAJgz+NQfcMlefP4LrbW99DWaztOEyp11CmohTMvRb7fSDugxqaI0SlcDa37p7+/qDs6zn9gB+Ed881XznJxeIJl2WGa89nlonkcpIF6Tivl5NkbYfnauTpVf8q2yIA+S2TGPtdY0hnJ38iVnvc51BxCqeIYZtL4/7d3r8F1XVcBx//r6uphPWw9ruS35Ui2XLee4Kah4yZ2cZy09AGEtqTTDnTKMFA+lLYMX9KBDxRmGDIMMAxfUgpkSGdS0g5taWY6BVxaaExxEyd1sGy5ku0oejmSLFtXb0uWFh/2OfK1ciVd6T7OPeeu34xGV1cP7z3naml77XXWroFYnJ31OSjtSk1Dibg870OfcyufJ1+HT3zN2/zy8tw5OmQkE3satiACvSkbqJ2DSeIxoWNHHvrUJA65W+XXqxLaaA17qpbDLo0z5/5AMejdcVoEK3Zwh29ABj3ay+Kunj3Nit1VxCxQMdHr9mRMUbPAXgC173iMI7Fezry2Sq53LsmM1LCrvupuJ8p8qayDt30YPvgUxLcUPIdZGS9j17Yt9KWkYjqHJujYXkdlPA/56DV6xtwj0wM20vE3UEcvu/cD52DrHqjbQNlkHh1oqaUt0x7tTQfS5ti7hyd5uP4WokvLvehN8bLAXgAVBx8FYLLrB+nLzuaSJLU6+zTMRgSYw2xtql5esasqFweT2Z2YtJZmL7CPrrOBOt4PiAvIG+WX/g17fYEGzwV6Y1I678+0R3tTuzsib8VNXT0jUykVMbZiL3YW2Ath51Hm43Ucnn2FS9cn3vr5uSS3Fqty0/xqIwKqHGptqqHPayswPHGbsen53N6YlGrbPreZnMmKvW7H5jp/1u9zpY0jXTA16s5NDfDGpHQy7tHe1O6qqSYGl5/yK2KOlA+6G4b86hlTtCywF0JZHPYf53hZZ9q7AJdmk4wt5qDUMSRam6q5OT3PxNwCnYP+Had5WrHHYtB0cP3APt63ufw6uH0M79CN5fx6kWyc+jLu0b5c8ng3z+5XxOxb6nflnXlse21yI6vALiJPiMhFEVkSkeJ6JReZio5H2SejXLhw/i2fuzMzzgQ1JRPY93uVMX1jM3QOJRGBwzvzFNjBpWPWS8Uk+zeXX/d5PWMYfCWwjo5rybhHe5r2vX5FTNP0Vcuvh0S2K/ZO4KNABi0MS1zbSQB2jJ2lf0V3Q7nterEXPBUTkH2Nfi37DBeHJmhL1FBdkcfK28QhtyJfmE3/+aUlSA7Ctk3k130tb3fVNz/7t8A6Oq4nox7tdTuhvPqeDdSe4UmqYwuUT7xh+fWQyCqwq2qXqmZ4W1+JazrAndqdPBy7wOnU6oSlJcrmJ5mgOvsGYCHht+/tHZvm4mAyf/l1X+IgoKvfqDQ17A4a2WwqBlI2UC8U3capL6Me7bHY3Z4xnu7hSY7X30RQW7GHhOXYC0WE+IFTnIh3cfri3Y0p5qeIscRcrJbGmtLIXdZUxknUVnK+f5yh5Fx+bkxK1bxOM7DlUscNthNI5Zc8QtHl130Z92hvarsnx94zMsWxWv/UJFuxh8G6gV1Evi8inWneHt/IPyQinxGRcyJybnQ0fZOhyGt7hK06yfQbP+WW3yvFu+u0vKYBkRycHBQSrU3V/He3ex3krdTR19juWiysFtjHvT4+2azYa1vccW6Q/9vts5BRj/amA3CrFxYXUipihtxdqU3tBRur2bx1A7uqPqaqR9K8fWcj/5CqfkVVH1TVB5ubmzc/4jBr+wUAHpJOfnDZWwF5gb2qrjGoUQWitama+TuuVjovPWJSlVdBfevqG6jZ3Jzk63/JOwkLeOHzRdtm9uShZspi8NT3LrtzWdNpOgC6CON9dytiFvtcSqusPP33mKJiqZhCqm1BW97OqYpLy3lOnXOnwtfWF6ZtbrFo9TZQ9zZuYVt1AYKFf5pSOuP97vi1bM7E7X1xuUNDMbeZ7RmeQhVe7r3Fx7/8v7xwfvCtX+RXxoxdoXvY/bFqnLma/1a9JmeyLXf8iIgMAO8Bvisi/56bYUWXtD3CO7nM2e5BZucXmbjlKhTqGxIBj6yw9ifcBuqW8rLVV465lOhweeOlNKV+2ZY6QmjazJ69drciZlGVzz9/ns8+9yoXvROsgHtq2a+MTFETm6d8oj//h2uYnMm2KubbqrpHVStVdbuq/mKuBhZZbScp13mOLHZx5soNbt50eeZEoiXYcRXYzLwLsD3DU/z6P5zNf3BvPgSLt13ueKXxfneHajZC0mb2WFsTFXF36HZlPMbHHtjNj7pH+fDfnuG3/ulldx2qG93B5mNX6R6e5ET9mKuIsRV7aFjb3kJrfQiNxTlVcYnTl95kh9wAoKWlcD3Ri8HIxBzgshcLd5Y4e23MnRWaL8vNwHru3QBUdSt2b/8jKyFoM+sfun322hjH2pp4V2sDydkFvvrjXp75n9f52NM/5qH2Jv6uZj+1Y1foGZ3iU7WjMIOt2EPEcuyFVlmL7Hk376vq4vtdI0wl3X+Nd20vrcB+/GAzVeVu5Vgej3GsLc97DMuBfcUG6uwtd65mNhUxIXPPodvAti3lfO7Rg5x58hR/9KHD9IxMcXqkjhtvXOL1G9Mkpq+wFKuAhvsCHrnJlK3Yg9B2kr19f87S3BgD199kmipqqoq3RC4f0q0c82pLvTspauX5p7moiImImso4v/PeNj71nlYuf/0/ab5yhgrmqZm4QndsB9MDk/m/TiYnbMUehPZHEJQT8S50NslsLA8HTITAypVj3iU63lrLns0BGxFVVV7G0Z9zd8+2xd6kIzZA99KeezZeTXGzwB6EXQ9ARR0fre9hq8wwRU1hKkNKXaLDpWJSe+Ln4q7TKPL2Ie6P97FHbnBV9uY/XWZyxgJ7EMricN8Jji6cZyvTjN6pKkxlSKlrPuRuCJtK6Uk+3u9Okqq2oHUPr5b9i/tdz5hffuyUpWFCxAJ7UNpO0nB7kLfF+khq9XJliMmjxEH3PjUdk+xzXR1LqJ1DRqq2Qu12Gq67G60OvOPnAx6Q2QgL7EFpOwlAo0wxSU1hKkNKXcJvBpZSGTOeg5uToqqxHRZm3AlUDfuDHo3ZAAvsQUl0uN7XQPve3Tz328fsv7r5tnWXO8IutTIm2W8bp6vx6/0TByGWh4PGTd5YYA+KyPKq/f6KId4VW6WPickdERek/FTM/AzMjNmKfTV+awGlaJuamfQssAfJXyn2noFnf8V+eQohcehuYE8OuPfZthOIKnXdNxnutNdnyFhgD5L6Dam0qDsCRkpzB0wMwu1Jt3EKtmJfzey498Ben2FjgT1IHR9wpXZF3hEwUlJ7xtjNSWs7/Ev2+gwpaykQJL8jYO+L7pemyBtIRUIi5Zi8ZL8LWt4mtlnBXp+hZYE9aCHoCBgpjfdBLO5OU0oOwNbd7oYxk569PkPJXtGmtJSVu/rsG91WEWMiy3LspvT4JY/jVsNuoskCuyk9zYfg5jWYHLIVu4kkS8WY0pM4BEt33GNbsZsIshW7KT1+MzCwFbuJJAvspvT4texgK3YTSRbYTemprIXqZvd48s1gx2JMHlhgN6Wn/yWY9XrfP/eE9UAxkWOB3ZSe3hddx0KwHigmkiywm9Kz/wTEK60HioksK3c0pcd6oJiIs8BuSpP1QDERZqkYY4yJGAvsxhgTMRbYjTEmYiywG2NMxFhgN8aYiLHAbowxESOquv5X5fofFRkF3tjktyeAGzkcTjGI2pyiNh+I3pyiNh+I3pzSzadVVZvX+8ZAAns2ROScqj4Y9DhyKWpzitp8IHpzitp8IHpzymY+looxxpiIscBujDERE8bA/pWgB5AHUZtT1OYD0ZtT1OYD0ZvTpucTuhy7McaYtYVxxW6MMWYNoQrsIvIBEfmZiFwRkS8GPZ5siUiviFwQkfMici7o8WyGiDwjIiMi0pnyXKOInBaRHu99Q5Bj3IhV5vMlERn0rtN5EflQkGPcKBHZKyI/FJEuEbkoIl/wng/ldVpjPqG9TiJSJSIvichr3pz+xHv+PhH5iXeNvi4iFRn9vLCkYkSkDOgG3gcMAC8Dn1TVS4EOLAsi0gs8qKqhrb0VkfcCU8BXVfWI99xfADdV9SnvD3CDqj4Z5Dgztcp8vgRMqepfBjm2zRKRncBOVX1VROqAV4BfBX6TEF6nNebzcUJ6nUREgBpVnRKRcuAM8AXgD4BvqerzIvJl4DVVfXq9nxemFfu7gSuqek1V54HngccDHlPJU9UfATdXPP048Kz3+FncL10orDKfUFPV66r6qvd4EugCdhPS67TGfEJLnSnvw3LvTYFTwL94z2d8jcIU2HcD/SkfDxDyi4m7cP8hIq+IyGeCHkwObVfV6+B+CYGWgMeTC78nIv/npWpCkbJIR0T2A+8EfkIErtOK+UCIr5OIlInIeWAEOA1cBcZV9Y73JRnHvDAFdknzXDjySKt7WFUfAD4IfNZLA5ji8zTQDhwFrgN/FexwNkdEaoFvAr+vqhNBjydbaeYT6uukqouqehTYg8tQHE73ZZn8rDAF9gFgb8rHe4ChgMaSE6o65L0fAb6Nu5hRMOzlQf186EjA48mKqg57v3RLwN8Twuvk5W2/CTynqt/yng7tdUo3nyhcJwBVHQf+CzgG1IuIf4RpxjEvTIH9ZeCgt0tcAXwCeCHgMW2aiNR4Gz+ISA3wfqBz7e8KjReAT3uPPw18J8CxZM0Pfp6PELLr5G3M/SPQpap/nfKpUF6n1eYT5uskIs0iUu893gI8hts7+CHwa96XZXyNQlMVA+CVL/0NUAY8o6p/FvCQNk1E2nCrdHCHin8tjPMRkX8GTuI60Q0Dfwz8K/ANYB/QBzyhqqHYkFxlPidx/71XoBf4XT83HQYichx4EbgALHlP/yEuLx2667TGfD5JSK+TiNyP2xwtwy24v6Gqf+rFieeBRuCnwG+o6u11f16YArsxxpj1hSkVY4wxJgMW2I0xJmIssBtjTMRYYDfGmIixwG6MMRFjgd0YYyLGArsxxkSMBXZjjImY/weTXUHea+6UiwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.clf\n", "plt.plot( X1.dot(w), '.-' )\n", "plt.plot( y1, '.-' )\n", "plt.axis('tight')\n", "plt.legend(('$y_1$', '$X_1 w$'));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Mean-square error on testing set." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Relative prediction error: 0.7023447693100116\n" ] } ], "source": [ "E = np.linalg.norm(X1.dot(w)-y1) / np.linalg.norm(y1)\n", "print(( 'Relative prediction error: ' + str(E) ) );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Although this is not an effective method to solve this problem (a more efficient approach is to use for instance conjugate gradient), one can do a gradient descent to minimize the function\n", "$$ \n", " \\min_w J(w) = \\frac{1}{2}\\norm{X w-y}^2.\n", "$$" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "def J(w): return 1/2*np.linalg.norm(X0.dot(w)-y0)**2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The gradient of $J$ is\n", "$$\n", " \\nabla J(w) = X^\\top (X w - y).\n", "$$" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "def GradJ(w): return X0.transpose().dot(X0.dot(w)-y0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The maxium step size allowable by the gradient descent is \n", "$$\n", " \\tau \\leq \\tau_\\max \\eqdef \\frac{2}{\\norm{XX^\\top}_{op}}\n", "$$\n", "where $\\norm{\\cdot}_{op}$ is the maximum singular eigenvalue." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "tau = 1/np.linalg.norm(X0,2)**2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Initialize the algorithm to $w=0$." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "w = np.zeros((p,1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One step of gradient descent reads:\n", "$$ w \\leftarrow w - \\tau \\nabla J(w). $$" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "w = w - tau*GradJ(w)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "tau_mult = [.1, .5, 1, 1.8]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Exercise 0__\n", "\n", "Display the evolution of the training error $J(w)$ as a function of the number of iterations.\n", "Test for diffetent values of $\\tau$." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsvXd8XcWd9/+eU26/uuqSbVmWK8a4YDDNQJZAgBQCIZtC2M2P3YQneVJ2gWwKpC3Jb8lutiXZbEvdtIVACCGBEAhgQg02NsU2GOMiy5aLrC5d3XbKPH+cc6+uZMmWLcmSr+b9eo2mnDkzc+7V/Xxn5syZI6SUKBQKhaL00aa6AQqFQqE4OSjBVygUihmCEnyFQqGYISjBVygUihmCEnyFQqGYISjBVygUihmCEnyFQqGYISjBVygUihmCEnyFQqGYIRhT3YBiqqurZVNT01Q3Q6FQKE4pNm3a1CGlrDlWvmkl+E1NTWzcuHGqm6FQKBSnFEKIlrHkU1M6CoVCMUMoGcF3XbUJnEKhUByNaTWlc6Lc+aXb6T+wmpXXZLng6vdNdXMUCoViWlISPfxAUGKbcVreODTVTVEoFIppS0kIfs3cCAA9bekpbolCoVBMX0pC8MsaqhCug9VfEpejUCgUk0JJKGS4voFQphNhhekayE11cxQKhWJaUhKCH6mdSyjTgS7KeGFP11Q3R6FQKKYlpSH4kRp0qxOpVbChWQm+QqFQjERJCH44EAW3A6lF2bSzY6qbo1AoFNOSkhD8kB7CxhP6g/uT9GWsKW6RQqFQTD9KQvB1TcfWOwEocwSbWrqnuEUKhUIx/SgJwQfImp7gV0pNzeMrFArFCJSM4OdCOQwryYJwUAm+QqFQjEDJCH42qhFOdzJL09jc2kM650x1kxQKhWJaUTKCb0dMwpkOwlkXy5FsbFG9fIVCoSimdAQ/GiCc7sAecAnqGk+83j7VTVIoFIppRckIvhsLEsp04LpwcUMFT2w/PNVNUigUimlFyQi+LIsQTnsrdS6oLaO5Y4DmjoEpbpVCoVBMH0pG8CmLE854D1+dFvO2S173uurlKxQKRZ6SEXw9kSCY7UbgYmQcFtXGeEIJvkKhUBSYMMEXQuhCiJeEEA/68flCiPVCiB1CiLuFEIGJqmskwuEy0gGXiEjR15Hh0qW1rG/uJJm1J7NahUKhOGWYyB7+TcC2ovjXgW9IKRcD3cCHJ7CuIwgHE/SHIeL00NeR5s2n1WI5kmd2qM3UFAqFAiZI8IUQDcA7gO/7cQFcCtzrZ/kx8K6JqGs08oIfyrbT15FmTVMF8ZDButfbJrNahUKhOGWYqB7+N4HPAq4frwJ6pJT5+ZRWYM4E1TUi4XA5/WFBKNVOut9CWi5vWlzDE9vbcV05mVUrFArFKcG4BV8IcRVwWEq5qTh5hKwjqq4Q4iNCiI1CiI3t7Sf+sFQ4UEYyDMHkIQB6D6d589Ja2vuzbD3Qe8LlKhQKRakwET38C4GrhRB7gJ/jTeV8EygXQhh+ngbgwEgnSym/K6VcI6VcU1NTc8KNCJsR+iNQ1rEToQlef/4gly6txdAEv91y8ITLVSgUilJh3IIvpbxNStkgpWwCrgPWSSn/DHgCeI+f7Qbg1+Ot62iEjbA3pdPfyWnn1PDq0wcIOXDx4moeePmAmtZRKBQznslch/854FNCiJ14c/o/mMS6fMH3wivPS+DaLi8/tpd3rZ7Dgd4MG9TLzRUKxQxnQgVfSvkHKeVVfni3lPJcKeUiKeV7pZTZiaxrOGEjTNIX/LiRZtGaOrY8uZ+L51USCej8+uX9k1m9QqFQTHtK5knbsBGmzxd8p6eHs986DzvrsOPpg1yxrI6Hthwia6s98hUKxcylpAQ/GfYWBzndPVTNibFgdQ2bn2jlqtPr6U1bPLldbZmsUChmLiUl+P1FPXyANW9rIpexcTd1URkJ8OuXR1wopFAoFDOCkhH8iBEZFPxu7wZtTWOcc94xnx0b2nhPZTmPbWujP2NNYSsVCoVi6igZwTd1E2nqOIbE6RzcJXPN25toWFpB+etJyrKSh9SafIVCMUMpGcEHCGsm2ZDE6Rycq9c0weUfOoNwxOQ9mRA/e3oPUqo1+QqFYuZRWoKvB8mGBqd08kTKAlxx4xnEbFi6O8uGXZ1T1EKFQqGYOkpL8I0gqfDgTdti5iyp4KLrl9Bk66z7n224jjtCCQqFQlG6lJjgR0iFJE5v/4jHV13cQHp5GWWdFg9+fytSbbegUChmECUn+ANhgdOXHDXP+69fxjMhi30vdfD4T7bhqJ6+QqGYIZSW4Adi9IfBGUgjnZGfqp1bGSG2upJNcZftzx/i4f/egpVTT+AqFIrSp+QEvycCSHD6+kbN95cXzmedniW6toY9Wzt54FsvkxlQ6/MVCkVpU1qCb0bpiXiXNNKN2zwXLKxixZwEPzjUzls+tIy2lj7u/fpGug4OnKymKhQKxUmntATfCNOdF/zu7lHzCSH41OVL2NeVZpOb45qbV5NL29z79Y00b1YvPVcoFKVJyQl+a4W3gVquec9R815yWg1nNZbz7XU7qJwX5723nUN5bYSH/mszGx7YrV6YolAoSo6SE/y95SAMQXbHjqPmFULw6StO42Bvhrs27CVeGeLdnz6L086t54Xf7uE333yJgZ5J3cJfoVAoTiolJfghI4SrC8xK/ZiCD7B2UTXnL6jkP57YRTrnYAR0LvuL07nshtNp29PHz/9ug5riUSgUJUNJCX7Y8LbL1CrcMQk+wN9ccRodySw/fLYZ8Hr+Sy+Yxfs+fw7R8iAP/edmHv/JNrJpe9LarVAoFCeDcQu+ECIkhNgghHhFCPGqEOIrfvp8IcR6IcQOIcTdQojA+Jt7dCJGBACZsLAPHz7qSp085zRVcsWyOv593U5au1OF9Ir6KO/93BrOfts8tj9/iJ9/dT17X1V78CgUilOXiejhZ4FLpZSrgDOBtwohzge+DnxDSrkY6AY+PAF1HZV8D98uS3sN27lzTOf97dVnAHD7b14bkq6bGudfs5A//ezZmEGdB779Cr//wauk+nIT2GqFQqE4OYxb8KVHfi8D03cSuBS410//MfCu8dZ1LPKCn014gjzWaZ055WFuuXwxj21r4/evHjrieF1TGe//wrmcc9V8dr10mDtvf56tT+1XK3kUCsUpxYTM4QshdCHEy8Bh4FFgF9AjpcxPfLcCcyairqORF/x0TKJFo2MWfPCevj2tLs7tv3mVgeyR8/W6qXHuVfO57ovnUt0Q48k7t/OLv3+BgzuPPW2kUCgU04EJEXwppSOlPBNoAM4FTh8p20jnCiE+IoTYKITY2N4+vpeMFwRfEwQXNJJ9Y+yCb+oad1y7nAO9Gf7pke2j5quoj3LNLau54sYzyCQt7vvnF/n997fS15EeV9sVCoVispnQVTpSyh7gD8D5QLkQwvAPNQAjvkFcSvldKeUaKeWampqacdU/KPgawcZ6sjt3HtfbrdY0VXLDBfP40XN7WPd626j5hBAsXlPH9befz5q3N7H7lQ7+9/bnefbeHWpPHoVCMW2ZiFU6NUKIcj8cBt4CbAOeAN7jZ7sB+PV46zoWYdMXfCEI1oZxenpwOo5vHf1tbz+dpfVxPv2LzRzqzRw1rxnUOe/qBfz5V89nyTl1vPz4Pn72pT+y8Xd7yGXUMk6FQjG9mIge/izgCSHEZuAF4FEp5YPA54BPCSF2AlXADyagrqOS7+GndJNg1NsI7Xjm8QFCps6/X38W6ZzDzXe/hDOGG7OxihCX3bCM93/hXGYtTLD+17v52Zf+yMuP7VVbLysUimnDRKzS2SylXC2lXCmlXC6l/KqfvltKea6UcpGU8r1Syknfp6AwpROvI6i3Ascv+ACLamN89ZozeH53F9967I0xn1fdEOMdn1jFn372bCpnx3j23p389It/5KXf78XKKuFXKBRTi3HsLKcOIT2EQJCO1WDsfQ29chaZExB8gPec3cD65i7+bd1O5lVF+dOzG8Z8bv2CBO+6ZTUHdnTzwm/38Nx9O3nx9y2sfHMDKy5pIBQ1T6hNCoVCMR5KSvCFEISMEOloJaQ6Cc4/74R6+PmyvnbtCvZ3p/ncLzczKxFi7aLq4ypj9uIKrrm5gkO7e9n4uz1seKCZF3+/lzMums3KSxsoqwqfUNsUCoXiRCipvXTAm9ZJh8oACNbHyO3YiXRP7L21AUPjvz94Ngtqonz0Z5vYfmjkl6Mfi/oFCa76xCqu+9K5LDyzhs1PtPKzLz3PI9/byqHm3hMqU6FQKI6X0hR8MwwIguUObiqFdeDgCZeXCJv8z1+eS9jU+fMfrGdH24mJPkDVnBhv+ctlfPDvLuDMt8xl72td/PLrm/jFP2xk+/pDOJZ6obpCoZg8SlLwM9KGqkUEg96SzOzr28ZV5pzyMD+78TwArvvu82w7OPr7csdCvDLE2ncv4oa/X8vF719CLm3z2P+8xo8//yx/vH+XeohLoVBMCiUn+BEjQtpOw6yVhLRdiEiEgeeeG3e5S+ri3P2R8wkYGh/43vNsbh3/lgqBkMHKNzdw/e3ncfVfn0nd/AQvPdLCT7/0Rx749svseukwjq16/QqFYmIoOcEPG2FP8OtXoiVbiZ5zFsknnzquJ25HY0FNjHs+egGxoMF1332eR0bYaO1EEEIwd1kl7/j4Sj54x1rOeXsTnfsHePg7W/nxbc/y7L076DyQPHZBCoVCcRRKV/BnrQIgtnwu1v795JqbJ6T8uZURfvmxtSyui/PRn27i24/vmBBjkideGeLcdy7g/7vjAt7xiZXMWljO5nWt/PyrG7jnay/wyrp9pPvV9swKheL4KallmTCC4DfqACSffIrgggUTUkddWYi7P3I+t923hX959A1eO9jHP7x7JYnIxK2v13SNphXVNK2oJtWXY8cLbbz+/EGeuWcHz967k8ZllSw+p475q6oJhErua1QoFJNAySlF2AyTttIQqYSyBky7mcCihQw8/RRVf/kXE1ZPyNT51/et4vRZcf7x4e28vO8p/uV9q1i78PjW6o+FSFmAVZfNZdVlc+ncn+SNDYd4Y0MbLVs7MUyNeSuqWHR2HfNWVGEG9AmvX6FQlAalJ/j5Hj7ArJVw8BVib3ofXT/9Ke7AAFo0OmF1CSH4yJsWct78Km65+2X+7Pvr+fCF87nl8iVEg5Pz0VbNiXHBtYs4/5qFHNzVw46Nh9n14mF2vdiOEdCYt7yKhWfVMm95ler5KxSKIZScIgwR/PqVsP13xN5yLl0//CEDzz9P/LLLJrzOVXPLefCvL+JrD23j+88089CWg3zxqmW8bXk9QogJrw9AaILZiyuYvbiCi9+3mP07etj1Yju7X25n14vt6IZGw+kVLFhVQ9PKaiJlk/5KYYVCMc0pScG3pY3lWJizVgKSyCwDLRIh+dTTkyL4AJGAwd+9awXXrp7DF+9/lY//74tctKiaW9+2lOVzEpNSZx5N15i7tJK5Syt503VLOLSrh90vd7D75XZatnSC8F7T2LSiinkrqqluiE2aIVIoFNOXkhR8gJSdIjH3fDBCiK13Er1wLcmnvOWZkyl2Z8+r5IFPXsjPnm/hm4/v4KpvP8Pbltdzy+VLWFIXn7R682hFPf8L37OIzv1J9mzuoHlzJ+t/08z63zQTTQRoPKOKxjOqmHt6BcEJvNmsUCimLyUr+Gk7TSJaD2fdABt/QPTsO+h/9DFyO3cSXLx4Uttg6Bp/ceF83n12Az94upkfPNPMw68e4i2n1/F/Ll7AOU0VJ6WHLYSguiFOdUOcNW+fz0Bvlr2vdtGytZNdL7Wz7bmDCAF188uYu6yKuUsrqJ1fhq6X3GpdhUJBiQs+ABf+NWz8ITFzMwhB7wMPUvupW05KW8pCJrdcvoQb1jbxo2eb+enzLTz6WhurGhL82fnzuGrlLCKBk/cVRBNBTl87i9PXzsJxXNqa+9j3Whf7tnXxwm+beeHBZsygzuzF5cw5rYI5S8qpnhtH09T0j0JRCoiJfGhovKxZs0Zu3LhxXGWs27uOm564ibuvuptlVcu8xAdugpfvpPXAtQy8uJnF6x6f0NU6YyWdc7j3xVZ+9Gwzu9oHiAUNrj5zNteunsPZjRVTKqyZAYv9b3TTuq2b1u3d9LSlAAhGDGYtTHjTREvKqZ4bUyMAhWKaIYTYJKVcc6x8JdfDjwe8efIDyQODgn/RLfDiT6lcCf1P9NJz36+o/OCfn/S2hQM6Hzx/Hn9+XiMbW7r5+YZ93PdiK3eu38vsRIirVs3myjPqOXNuOfpJFv9Q1GTh6loWrq4FINmdZf8b3Rx4o5sDO3vZs6UTACOgUTc/wayFnqubX6buASgUpwjj7uELIeYCPwHqARf4rpTyW0KISuBuoAnYA7xPStl9tLImoodvORbvvP+dlAfLuesddw3Old//cdh6H3s2X4Td1cvCRx5G6FP/kFIya/PYa2088MoBnnyjHduVVMcCXLq0lktOq2XtwirKI1O/pHKgN8uBHT0c3NXLwZ09dLYmkRIQUDkrSt38Muqayqibn6BydlRNAykUJ5Gx9vAnQvBnAbOklC8KIeLAJuBdwF8AXVLKfxBC3ApUSCk/d7SyJkLwAX6141d8+bkv829v/jfe3PhmL7FjJ/znefSlVrD//jbmfPOblL31ynHXNZH0pi2efKOdx15r44nth+nP2AgBK+ckuGBhNefNr+TspgrKQlPfo85lbNr29HFoVy+HdvfRtqeX7IANeKOAmsY4tfPKqJ0Xp6YxTnltBKGMgEIxKZw0wR+h4l8D/+67S6SUB32j8Acp5WlHO3eiBN9yLa65/xoiRoR73nkPmvDnnLfci/zFjex6dB56wxKa7rln2q5Htx2XV1p7eOqNDp7Z2cHm1h4sR6IJb6vm1Y0VrG4s58y55SyojmJM8by6lJLew2na9vRxeE8fh1v6aN+XLLzUxQzpVDfEqJkbp3pujOqGOJWzouimuh+gUIyXKRF8IUQT8BSwHNgrpSwvOtYtpawY4ZyPAB8BaGxsPLulpWVC2vLArgf4/DOf51/+5F+4oumKwQMbf0jXN75A26Zy5v3kR0TOPW9C6pts0jmHl/Z1s353Fy/t6+Glvd30Z7wedcjUWDarjDNmJ1g6K87S+jJOq48Tm6TtHcaK47h0HxzgcEs/7Xv76djXT0drEjvnGQFNE5TXR6huiFE1J0bl7ChVc2LEKoLT1hArFNORky74QogY8CRwh5TyPiFEz1gEv5iJ6uEDOK7Dtb+5Fg2NX179S3RtcL7effyf2PXp76FFw8y/73602vkTUufJxHUluzuSbNnfy5bWPrbu72XbwT76s3Yhz+xEiEV1cRbVxFhQE2VBdZT5NVHq4qEpm2N3XUnv4RQdrUk6W5N07Pf8ZHe2kCcQ0qmYFaVydpTKWVEq6qNUzIoQrwipaSGFYgROquALIUzgQeARKeW/+mnbmaIpnTwPNz/MZ576DB9d+VE+ufqTQ44lf/BF9v3TL6lcZlF3+9dgxXvhFO9VSinZ35Nm28F+3mjrZ+fhJDsO97Pr8ABpyynkCxoa86oiNFZGmVsZZm5FhLmVEeaUh5lTESYRPvn3CLIpi84DA3TtT9J1YICug55L91uFPIapkaiLUFEfobw2Qnmd72rDaqWQYkZzMm/aCuDHeDdoby5K/yegs+imbaWU8rNHK2uiBV9KyZef+zL377yfm866iRtX3Djk+KHPf5ru+35L4yUdRM8/Dy65FZoumrD6pwuuKznUl2F3+wDNHUlaOlO0dKXY25liX3eKVM4Zkj8eNKhPhJhVHmZ2IkRdmefqE0Fq4yFqy4JURYMnZeloOpmj+2CK7kMDdLel6Dnkhfs7MxT/64ZiJuW1YcpqwiSqwyRqwpTVRCirDhEpC6gpIkVJczIF/yLgaWAL3rJMgM8D64F7gEZgL/BeKWXX0cqaaMEHb2rn8898noeaH+LTaz7NDWfcUDjmptM0v/tPcXvbWfCOXnSrDeZdCGv/ChZdDnrJPaZwBFJKugZy7OtOc6Anzf7uNPt7vPChvgwHejJ0DmQZ/m+ia4KqaICaeJDqmO/iAaqjQapiASqjAaqiQSpjASojAcITvE+/Y7n0tqfpOZyi53CK3vY0vb6f7M5CUXuNgEa8KkxZdYiyqjDxyhDxKt9VhgjHTWUQFKc0U7ZKZzxMhuAD2K7NZ5/6LI+2PMr1S6/nr1b/FbFADID0lq3s+cAHiKw+k4YPn4u+6T+g/yBEa2HV+2HF+6B+xSk/3TMeLMelvT/Lob4Mh/uyHO73/Pb+LB3JLO3JLB39WTqSOXLOyC9dD5kalZEA5ZEAFVGT8kiA8rBJRSRAImySiJiUh00SYZMy30+ETSIB/bjF2LYc+jsz9Lan6evI0NeRpq8jTX9Xhr6ODLm0PSS/bmjEKoPEKkLEKoK+GwxHy4OEosooKKYvSvCHYbkW//zCP3PX63dRE67htvNu47LGyxBC0PvAAxy49TZCy8+g8b/+E/3wenj5f+GNh8G1oawBllwJiy+HxgsgXH7sCmcgUkr6MjZdAzm6BrJ0JnN0p3J0DuToSuboTln0pLy0nrRFjx93j/IvqGuCspBBPGRSFjaIB03iftzzDWJBg1je9110mB8ytYJgZ1MW/V0Z+jsz9HdlSXZl6O/OkOzKkuzOMNCbQw5rlGYIoomg7wJEyn2/LEgkESiEQzFTPXSmOOkowR+FLe1b+Mofv8L27u0sr1rOdUuv463z30p23VMc+NTfEFiwgMbvfw+jpgYGOjzR3/472LUOrBQgoH45NK6FOWfB7LOgahFoaj35ieC6kmTOpjdl0ZsedH2+35+x6csMhvszed8LJ7P2UQ1GHk1ANOAZgEhQJxowiAR0okGDcEAnGtCJBLxwRNcIuRC0wMy6iIyDSDvItIMzYGMlLbL9FnbWOaIeISAUDxCJm4TjAcLxAJF4gHCZFw9F/fSYSThuEggbauSgGDdK8I+C7drct+M+frbtZzT3NlMeLOfKpiu5vK2WxN/+F1o4TN2tn6Ps6qsHf4xWBvZvhD3PeG7/Jt8AAIEY1CyFumVQuwyqFkP1IkjMBW3qt28oZaSUpHIOyaxnBAay9pBwKmeTzDoMZG0GcrbvO6SyNgNZh5Rlk8o5pLIOqZwXtsdiQQBDQswVRKWgXNNICI0yNGJSEHYFIRdCNpi2RB95pgsEENTRQhp6SMcIG5hhg0DEIBg1CUYNwlGTSMwkEgsQjQWIxU0iUZOQaZz0PZcU0xMl+GNASsmGQxu4e/vdPN36NBknw2ldIT7xe5365l7cNSto+tJXiZ629MiTHRs6tsOBl+DgK3B4G7S9Cumi+9J6AMoboaIJyudB+VzPCCQaID4L4vVgBE/a9SrGRs52Secc0tagEchY+bgf9o9nLJe05ZDO2WQst5AvY7lkbS9f1naxsjYy66LlPKdbEtOWhKUgIgVh6RmIcD4swWB0MXeRZATkhCSnQU4DWxc4usDVwTU1pCGQpgBDQwQ0hKmjBzX0oI4R0DCCGqahE9A1TN8FDA1TF76fd2JIOKBrmP5xQ/PyGtpgHkMXmJqGaQgMzTtHjWImFyX4x0nGzrD+4Hqe2PcEmw6+wJIn93D9ky6RLOxcHKX1ypVE3/Qm5lcuZH5iPrOis4Y8zAWAlDDQDp07oWMHdO2C7j2+a4FMz5EVR6ogVg+xWojVQbQaojWeH6kadOEKCCXUiKGEkFKStV2yvnHI2p7ByPvplOW5pE1mwCKXsrEyNlbawcnYOFkXN+sgcw4yJxGWi7BcNFuijTaiKK4fiSUgJyCH9HwhyQGW8OKWf8wSXpoFw8L+MT9s++HhtsrQRMEQGLrA0DVMzfOHpxuaKBgQQxd+XEPXxeA5fnmG5oV1vwxdE5i6QNeK8wzG9aLzdE0U0orDhj4sf8HX0DQK5+ojnKsLMSX3cJTgj5OOdAcvb3+K1C/vp/7RV4j35OiNwCvzBS8tFGybbxKvb2BObA6zY7Opj9RTF62jLlJHdbia6nA1iWBicB8fgGw/9O6H3lboPwD9h6DvgGckkm3Q3+aF8y9vOQIBoTIIlXs3jkMJzwUTXnqwDIJx38UgkPej3rRTIApmxPOV4ShpXMcll3HIpe0i38bKOGTTnp/LesYjlx1Mz2UdrKx3jp11sHMuds4Zssx1TOgCdIHUBVITSB2kLnAFSE3gaOBq4AjfAbbwHRIbsJDYSCzphXNSYvnxrOtiSUnWdclJSdYPWxLkFA8mhABdiCOMwohpRenvXTOXD190Yk/9K8GfQKRt079uHZ2/+y3p555D9CYBGCgP0To7wK5qmz3xDG3lgvYE9ETBNgS60CkPllMRqqAiVEEikCARTFAWKCMWiBEPxImZMWJmjKgZJWpGCZthIi6EcynCuSRmug+R7oJ0d5HrgUyvN2LI9HqGJNMHuf6xX5QehEDEMwBmBMyQ5xshMMNDfSPkTT2ZYW+aKh83gl45RqDI98O66YcD3vMMegA00083B8NqqD/tkVLiWC5W1vFczsHOur7vx3N54+AOxi3XMxqWOxjPOTiW64UtFyefbrk4tnv8hmUEhADN0NB0UfCFLjxf88LCD+M7L0whLoVvODSQQuACUgOJd8zVvM/F9fO5CBz8OBTCjvScK6Tn4/mOH3ekxJESW8Ily+t479p5J3jNM/QFKJOBMAzKrriCsiuuQDoOma1bSb34EpnXXqPitdc47ZlmcIeOoe1YiGw8RDpikwp10B9sJ2k69OsWfXqWLt0lZ0DOBFsDywBbB0fz4o4OjiZA0zAME8MMYhiBgjP1AIZRiVFeh6GZGEYAXTMw8L5UHYkhJaZ00aWDJl0M6aC7NprroEsL3fHj0kZ3cohcFi2TRHcsNMdCc2w0J4fuenHh2t5vQkoE3m9DAJoEzf+laoCQXrpA+v7gCF8McRIhNNAMhGYg0BG6gRA6QjNA0xDCQGi6l0doflgv+KAjNK8MhOaVJ4rOEzoIzVtFJYQfHkzP50f4ZQqvZULofiO1YfmGu8E84JeP5u/5U5TmG7ah6RSFRVHdxecxeBwKeQq+nyYYnjd/bFhawb4WlTk8XFzPsLAQgiCCIIApPDf4SyngvD3sAAAgAElEQVQqj9ERAtB9FyjULaX3M3IscBxwHIlj+2HbC7uOF3eHHXNdcG05eMzx4q7rH3f8sh3XD/tpOf9cR3rnuSCLj+d9Rx7x8OFwNN8BnMhGH/VlIVh7AiceB0rwjxOh64RXrSK8alUhTVoW1sGD5Pbtw9q/H6ezE7u9A7urC7evF6e3D6e9Dzedxk3lkCn7KDUMx8GbFU1N9KWMkfwPMzSuUuQw/9g4DF67QjGUvLiezB2UJCCFhhR6wbmaPixNeOl+h0IKDVfk8+TzjRTWaHcH4Lp7JvUalOBPAMI0CTQ2EmhsHFN+6brIbBY3k0Fms0jLQuZynrMdpG2BbSMdB+l1dTzfdX1fgnSRjgvSBSn9sPQcni/zcQlSuriug+M6Xlg6uK6LlC6OdJGui4vrDVOlly6l9PL65bl+XS5eOlDID95NwMF075if6rWnkMv/66t/UQr5blQ+XDAQRd2rotSRw8OsihzFzAxJLw7mP8P8geK4HJ5+ZHjINOmQbmH+movLL04f3qYRzKQcFh/SziOvcGjaaOZ25M/tyMTiXu5YTfdR8o2pbcebZzzln0AdYzrVZXDXmdHLbzp7krv3KMGfEoSmIcJhtHB4qpuiUChmEOrxUIVCoZghKMFXKBSKGcK0WpYphGgHTvQdh9VAxwQ251RhJl73TLxmmJnXPROvGY7/uudJKWuOlWlaCf54EEJsHMs61FJjJl73TLxmmJnXPROvGSbvutWUjkKhUMwQlOArFArFDKGUBP+7U92AKeKUu24hxD4hxJnjKOK7fjl/L4S4ebRMxzp+PAghNgghzpiIsk6gbiGEaAF+dZznTVmbJ5BT7v97gpiU6y6ZOXzFqYEQogI4DMSklNlxlFMDvAwsklKmhRC1QBswS0p5aPjxCWj3+4D3Syn/dLxlTSbFnwPwJk6BNitOHqXUw1ecGqwA3hiP2Pv8BfBQkZivAtqllIdGOT5efgO8WQgxa4LKmyyKP4dTpc2Kk4QSfMXJZiWwBQpTFZ8TQrQIIXqEEPcIIRL+MU0I8WV/+ueAEOKdQoicP0IAeBvw5LByNxfFhx9HCJEUQsz1w/9XCCGFEHV+/DNCiO+P1mgpZQbYBFwxrqv36rpRCPGIEOK/hBDdQog3hBDLhBA3CSH2CiE6hBDvHpb//qJ2/1YI8R9+vgNCiMtH+hwmss2K0qAkBF8I8VYhxHYhxE4hxK1T3Z7JQAgxVwjxhBBimxDiVSHETX56pRDiUSHEDt+vOFZZU0yxMP//eMJ8PlAPBIEv+8duB97iH1sGfN5P/6nvrwJu9a/7bj9eLPgrgO3D6u4G4sJ7/dIngJ1AhR//KPDtY7R9m1/PeFkJrAHuxVtvvQX4nX9sId7n8rdCiHuFEK8D/wh0CiEqgS8AbwUuAJYA3wE+V1T2CoZ+DhPV5pOCEOIW//97qxDiLiFESAgxXwixPv9dCyECU93O8SKE+KEQ4rAQYmtR2oi/Zb9j9G++vm0WQpx1ovWe8oIvhNCB/8ATjmXAB4QQy6a2VZOCDfyNlPJ0PBH8hH+dtwKPSykXA4/78enMCmCz37P+K+B6KeVBvzd6L7DGn3+/BfiQlHK/lLIH6APai8qpAH7oX3c3cAlDha4cGP6CgB4ghtfj3Qls9fO9FdgvpXwFQAjxDSHESGug+/3842UV8PdSysellA7wGrBFSvktKaXlt2su8LCUcqkffwLvu3WA24B78IT+tWFlDx/pTFSbJx0hxBzgr4E1UsrleNu0Xgd8HfhG0Xf94alr5YTxI7z/u2JG+y2/DVjsu48A/3WilZ7ygg+cC+yUUu6WUuaAnwPXTHGbJhxfFF/0w/14Pbc5eNf6Yz/bj4F3TU0Lj43fk16O16O9GE/kDhRlqQYOApfhfac7/fMa8Iz5U0XlADzn+z/F+yxeKSqrG4gPa0I3nuDfDHwLz4hUAB8H/q0o3zLg1REuIY5nNIZf1x/86aGR3DMjlLMSeHBYfcXxs4AA8AM/vgL4I953XQk8wOB3vRxf9P3Oz+nDPocR2zyNMYCwEMIAInj/D5fidQZgmv+PjxUp5VNA17Dk0X7L1wA/kR7PA+Unel+mFAR/DrCvKN7qp5UsQogmYDWwHqiTUh4EzygAtVPXsmMyH3CllC1ADdA77Pg1wDN4wl9sCL7p+7t8vwrI4E1/gCeOGkN7u5vxpjyK6cGbSpklpfwDnuCvwhPN+4vyhf2VP58VQtwhROE9lcPFFAAp5SVSSjGKu6g4rxBint/eN4qSz8RbUZTnfDzj9D/+kD8KHMJbeaMD24u+69VF5y7BE8ziz2HENk9HpJT7gX8G9uIJfS/ePYgeKWX+JRKl/Pse7bc8YRpXCoI/0rt1SnatqRAiBvwSuFlK2TfV7TlOCjdsgReAC4QQC4UQMSHEV4E64Id4o5e1QohFQoj3AqcBDcBu/1yB90aYP/HjpwPZYSt/Hio6nqcbb6oo35vvA24CvuNPreSXNfYLIX4IHJBSfkFK6QohgsDZwKPj/AxW4Y1sXL++MmAeQ6dhTgNm4w3dP4+3p8qteL/Xwrk+qxkU9JUUrYCawDafFPw562vwOgaz8Qzd20bIWrK/71GYMI0rBcFvxZvvzNPA0N5hySCEMPHE/n+llPf5yW354Z3vH56q9o2Bwg1FKeVG4A68Hn0rnmhfKqVMSSkfB+4CXsK7KdmA9w/+Nbzh/TfxfgRvF0KEgXPwxLyYnxQdz9ON1wO+04/34c1vf29YG1cDzVLKnxWlXw38YdgU1ImwiqG9+ZV401cp8FYnAQvwjM16P/8GvGmeDP7IwP+uO/Bududv/K1gaG9+otp8sngL3ufe7t/LuA/vpX/l/hQPlPDvm9F/yxOncdJ/M9Kp6vB+wLvxegUBvH/4M6a6XZNwnQJPxL45LP2fgFv98K3AP051Wyfh2v8v3rz1JcCDftov8J48vRnYj2cEh5/3NbyR0PHUdTPwHuC3wKqi9PXA8pN4zU8Dp/nh2/3v+ajfNd6N3f8zVW2egGs+D+/eScT/f/8x3o39XwDX+Xn+G/j4VLd1gq63CdhaFB/x+wXegbeKS+BN92044Tqn+qIn6IN7O17PZxfwhaluzyRd40V4vdzNeD3El/3rrsK7o7/D9yunuq0TcK3n401zaMDleD2d84cJ/gK8nu9+IAvMnqC6f+CXXYO3jr96ij6DM4GN/vd9P97N5VG/a/9z6piq9k7gdX8FeB1v1PJTvKW6+e96py/+walu5wRc51149yksvB78h0f7fn2h/w9f37bgrWI6oXrV1gqKaYcQ4mPA3+G9o/oN4MtSyodGyLcFb5nizdK7CTsjUZ+DYqwowVcoFIoZQinctFUoFArFGFCCr1AoFDME49hZTh7V1dWyqalpqpuhUCgUpxSbNm3qkGN4p+2kC74Q4q14j7HrwPellP8wWt6mpiY2btw42U1SKBSKkkJ4L8g5JpM6pTODNjZTKBSKac9kz+GflI3Ndu9p5X8/dxsHOodvjqhQKBSKPJM9pTPSpj/nFWcQQnwEb8tPGhsbT6iSZ7/5n/RlLufXt65jayIMixIsXljOGXPKOWN2GbMSIQY3WFQoFIqZyWQL/jE3/ZFSfhf/hb1r1qw5oYcC3v6Vz/E/n/kIc7vPZ5m2HF4ewHo5ybPaPh7SJFZAEC0LUFkVYXZdhMbZcRY3JEhUBAlFTHRTLVZSKBSlz2QL/knZ2KwqkaDmr67gt/f8K7c8FKOr7lyci6+iI1BFb0cGa8BCO2QjDvXR92ofWzlU2G0KQOoCPaQTihjEygJEYwGCEYNg2CQQMQiGDYIRg0DYc8FwPqwTCBvoujIYCoVi+jOpT9r6O9y9gfdCi/14W+JeL6Uc6eUSrFmzRp7oKp0Ba4Ar7r2cy1oH+Mtfpch0GUTOO4/6L32R4KJFuI7LQF+O3fv72d3ax76DSQ53pujpyZJK5gi4EJKCkAtRTSOCIOCC5hy7bsPUCsbAMwg6gZCBGTYIhgxMPx4MG5ghz0gEQgaBQljHDBlompp2UigUx48QYpOUcqS3tA1hUnv4UkpbCPFJ4BG8ZZk/HE3sx0vUjPKBpdfzndx3+OANtdQ/vZvDmzex+13vouID11P98Y8Rr6hgVUWIVcuHLle1HJd9XSl2tQ+wqz1Jc/sAWzoHaO4YoKMvS1BCUAqCEipMg9nRILUhk0rTJGHqRIRGSAhMB4Ttkss4JLuz5NI2uayDlRmD1QCMoO4ZgSHGwDcIvmEIhIwhRmIwLX+eoaaoFArFiEyrvXTG08MH6M50c+Uvr+Tyxsu4IxfBfvQbtL8xi55tEi0eo/pjH6fiz65HC4z9HcgDWZu9XSlaOlO0dqfY15Vib1eK/T1p9nenGcgNFXNdE9TEgtSVBaktC1EbD1ITC1AdDFARMEiYOnFNJ6IJhI1nFDK256cdclnfz9hYGZtcxiGXtrGynj+Wr0vTRcEImMFiY5A3HIOGwiwyMGaRoTF946Op6SqFYtoz1h5+SQk+wNc3fJ27Xr+LOy66g7dpCbQHbyGzczeHt9Yw0CowZ9VRc8unKLvqKoQ2PjGTUtKXttnfk+ZQX5oDPRkO9WZo68vQ1p+lrTfD4f4M3SlrxPPDpk5lNEBlNEBFNEBlxKQ8EqAiEqAiapIIe648EiARNokHdSK6jrRcrIxDNl1kFDI2lu/n0o4/uvDCxXlyGS9u59wR2zQcI6ANG1UMjiTMotFFsWEpNiL5uBnU1UophWKSmLGC35Hu4GOPfYzXu15naeVSbjrzr7kwk0Gs/w7JZ57i8Ctxst0BgrPLqP3w+4m+6wZEtGqCrmBkcrZLRzJb5HJ0DeToTGbpTOboTuXoSll0DWTpGbDoz9pHLS8S0CkLmcRDhu8Gw7GgQSxoEgsZxII60aBBNOilRwMGUT8trOtojoudc/3RhTf9lEsXGY4RDIk1ZATi+a5z7P8hIfBGDWGjaCpqmIHIjyxGmLIqNi66oSnjoVAUMWMFH8CVLg81P8S/v/Tv7E/uZ1H5It6z5D28s+pM4rueoe/Xv6D9sb1YSZ1wdY6aN5UTPfc8mHUmzFoFdWdAMDYBV3RiWI5LT8qiN23Rm875vkVf2vZ9i/6MF05mbfozXjyZ9VwqN7Z7BpqASMAgEtCJBHTCRWHPGYQDOmHTi4fMwXC4KB7UBIYDppRoNmiORHMkMufi5NwiA+IbiSLjkh9x5I3LWN7UqWmicCN80CAMGhEzbBAIDt4DGXXaKqSrFVaKkmBGC34ey7F4cPeD3LP9HrZ2biWkh7i44WIubbyUiyvOxL3zR3T89FfYPSmis12ql3USqfanX8rnQe0yqF0K1Us8V7UIwuUT1r7JwnZcBrIOyZxN0jcEA1mbVM4mmXVI5WwGfD+VK44PpqVzDmnL8cM2KcsZ0/2D4Zi6IGToBE2dkKkRMnWChueHTM0/5vuGIOjfAA9KQUCCKQWmC4YDuivRbImwJcJ2wXJxLReZk7g5Bzvn4GTd45uyMrXjHmEEgsaRBieoI9QqK8UUoQR/GNs6t3Hfjvt4fO/jtKfbMTSD1bWrubBqDef9sYfAXb/F6eoietZSaq5cTDjSDodfg86d4BZNsUSqoHIhVC6AiiaonO/55Y0Qq4dx3heYrkgpydouGWvQEGQsz3lhl7TlkMk5ZGwvPZ1zC+GM5ZK18sdcsr6fLyNru4Xys7ZLzh6bYI+GkBDRBDFNI6p7K6nCQhD2/SD4RsUzLIbrO0eiO94oJW9YxBibogU0jKB3v8IM6v6yXM+AhMLecx35JbvDjUvBqIQNDFNNWSmODyX4o+BKly0dW3h87+M8t/85tndvB6BSRvjzbVWct+4gZn+G0IVrqfvkJ4msXA7dLdCx3RP/zl3QtRu6mqFvP0PmIPQAlM2B8rmQaIREAyTmeGmJBiibDcH4pF5fqeC6siD8eeOQcwaNRd5AZIsMRLGfsRxyzuD5Of/YYNpg/uL04vyWI8k5LpqEgL80NwAFI5EfhQT8JbsBBKbECw/Pg5dHG/Hh86FIwNHB0QWuLpCG5zA1MDWEKRCmhgjo6AEN3Tc0RmhwdZWZH42YOqauYRoapi68sO6FA37Y8MOGPjSPrkYspwwlI/iWZdHa2komk5mUOh3pkHNyZJ0sOSeH49iEcxDOgSbBMTVkJIwRimBoxtCel5Re7991fN8ejEtn6Mggj9BA00HoBT8kcjSEUphltd4oIVYLujkp16s4PqSUQ4xBznGxbEnO8QyN5cjCMcvxjYjjYvnx/LmWI8lZDpblYmVt7IyDk3Owsy5uzkFaEmm5kHPBcRGWN8LQHInujzq8UYjEdMGUIMZgPGwkOQE5Icnh+dl8XBTF/XzZQjpYmvQNjoY0BIah+YZBYGhDjYehDx4rpGnFxmQw/+C5wjc4RWFNFIzQ8HKGn2/oAtM/VhyeiQZrWjx4NRG0trYSj8dpamo6KcNc27VJ22nSuRT09BLpy6G7kDWhJyZxoyFChufCepigEUQTo0zjSBccy3c5z3eLwo6FdHJ0Dui07n6V+c9f558oIFoD8TrPAOT9WF1RuBbi9WCGJ/0zmckIIQgaOkFDn+qmDEG6EivnLc3NpCzSKZtUyiI9YJPNWGRTDpmiZbtWxnuWw/adm3Vxcg5uzoUxrLICcDVwDW/U4egCW5PYmoOtgaWB5RuKDJKMkGSRpIGMdEkjSbsuA9Il7Ursk9DRFIJBg1BkJAxt0EjljYquDTcYQw2NoYnCuUPTNEzfL+QpSisu54j8Q4yUoDIapDI69meEToRpL/iZTOakiT2AoRnEA3HigTjE6pCuS7arg0BHF3XdNnZ/hp5ohkMhiRReLytgBAjpviHwfUMzvN68EfTcKAgpqXIs2nNBuO4uSB6C/jboPwhJ3z+0BQYOewZkOMEyT/yjtZ4fq/P9fFrN4LGjtENxaiE0UXgeIl4RGldZruMOW4I7fDnu4IN/w1dVFeLpoy3RzXeIfKOZX6Ib0r2pqKCBEdTQAzp6UEML6GgBDVE0hSUNDXzf0QVSF7gGWIDtSs85LrYjsVxvFGa73sjKdlxsV2IVHbcd73jOzzcYdxnIOUPKsh3/3GF1eOdMnOH6v3+ykFvftnTCyhuJaS/4wJTewBKaRqi6FllVg9Pbi9bRQXVvhuoBAycRJRUzSMscA9YAvdnewnmGZhAyQgT1YMEQBPTAkaMBIRBGwJv/X/r20RviOpDqhP5DkDzsGYPkIUi2++E2aHsVdj8Bmd6RywgmhhqAWK03kojWDBqIaLUXn8JlqYqTi6ZrhKIaoej4pxEdyx32DMfQ5zmy/nMeVsYh6x/Pp2cGsoPPe2TG+FR5folu0CA87MG/weW4gcGtSYbcLC+6YR4+sSW6UuaNjWcEnGJjMMxgOG6xESrO52K5koU10RP4xI+PU0LwpwOPPPIIN910E47j8KEPfpC/ueEG6Owl3iUoLy/nuW37uOWzn2HL5i189yff5cp3XknGyTBgDZC/TyKEIKB7o4G8IQjqQUxtDD80TR8U6WNhZbwRQd4Y5MMDhz1jMdB+bONgRnzxzxuF6mHGwY9Hqr2VS7r6V1KAbmqEzQDhca5NkFJiW+7QBwHzI4zCaGPocx0Fw5G06O/MDBqX7NieS9ENzXtqvHiPqqMszx3p6fJI2MAMm9N2I0T1Kx0DjuPwiU98gkcffZSGhgbOOecc3vXe97J00SLszk6cnh7qHYfv/d0dfOsnPyFuxJgTnwN4q4JyTo6MkyFrZ8k4GVJWashoQBMaHekObn/udhZXLGZh+UIWlS+iKlR1YqMbM+QtEy0fwwtl7JxnAAYOw0CHF84bhXy4rxUOvuzFR7oRjYBwRZFhKDIOkaqhBiNS7eUt0eWriolBCIEZ0DEDOiTGV5Z0pW8QivamyuT3rcpPSw2bovLzDfTmyB1KFYyLbY3x+Y5hGyGOaESGGYuK+ggV9ZPby1eCPwY2bNjAokWLWLBgAQDXXXcdv/71r1l2220E5sxB1tWxqLYWu6sL0mmsgwex2trQKyrQAoHCTV6KptAd1yHrZAuGoJNOHt/7OL/c8ctCnvJgeUH8F5YvZGFiIQvLF1IVnsCtIIyAt3Q0MefYeaWETM/gaCFvIAbai8IdcHgbDDwF6e6RyxE6RCo98Y/6I4SCQfDDhWPVXl5tet00VZw6CE1477QIj1/uHGfwyfHhT40fa0uSvg7rqFuSnHVlIxdcu2jcbTwap5Tgf+WBV3ntQN+Elrlsdhl/+84zjppn//79zJ07+B6XhoYG1q9fX4gLw8CoqUGvrkaLxdACAez2duz2drRYDKOiAi0eH7JZm67pRLQIETMCQE+4h6fe/xSdmU529uxkZ/dOdvbsZFfPLh7a/RD91uD7eiuCFSwoX8DCxELP941Bdbh6cu93CL8nH66AmiXHzu9YkOqCVJExSHUONRCpTmjb6sUzPaNV7BuIKt8QVA0zCFV+WlG6ukGtmAR0XUOfoPsdtuUMMQqh2OQvxT6lBH+qGOlZhZGEVQiBME2MmhqCS5bgdHfj9PSQ27cPoevo5eXo5eWI0Mjv2BVCUB2upjpczfmzzh9S/+HUYXb17mJXj+d29+7md3t+R39u0BDEA/GCEViQWMD8xHwWJBYwOzZ79KWjk4luestI43Vjyz/EQOQNQlE81QEDndD+BqT+COmukVcuAQRiRaOFqsGRQiFeNTQ9VK6mmRQnFcPUMUydcHxyl2IOqfOk1TQBHKsnPlk0NDSwb9/gu9hbW1uZPXv2Uc/RAgG0ujqM2lrcZBKnuxu7qwu7sxMtGEIvT6AlEmPam18IQV20jrpoHWtnry2kSynpSHcUDEFzbzO7enbxh31/4L4d9xXyhfQQTYmmggHIu8ayRgL6yftnOybHayBc1xsV5EcOww1DPq3/kDfNlOoEKzVyWYVppqpjuKI8gag36lEoThFOKcGfKs455xx27NhBc3Mzc+bM4ec//zl33nnnmM4VQqDH4+jxONK2cfr6cLp7sNraoK0NLRJFL08g3ePfO0YIQU2khppIzZARAXgvg9ndu9tzPbtp7mvmlcOv8Lvm3xXy6EKnId7A/LL5zE8MdYngOO+UnQw0zRfgyrGfk0v5hsA3Bqkuf5qpoyi9Ezre8MNd3lPTI6EHhxmCEQxGuGKooTAjykgopoxpv7XCtm3bOP3006eoRYM89NBD3Hzzzd6yzA99iC984Qt8+ctfZs2aNVx99dW88MILXHvttXR3dxMKhaivr+fVV0d/m6Oby+H09OD09iKzWXa0tVF+513E33ol8be8BaOiYlKuI2WlaOlrKRiD5t5mmnubaelrwXIHX9RSEawoiH9TWVMhPDs223uobKbgupDtHWoY0sPDXUPT0z2Mus+zERo0AOHKocagEK8oild601PKSCiOQsnspTNdBH+ykFIis1lee+UVQl/8Eta+faDrRM87l/jllxO79DLMujGsvR8njutwIHmA5r5mdvfsZk/fnoIx6M4OrrYxNIPGeCNNZU00JZoKxqCprIny0PTfOvqk4Dqe6BePIvLGINU1eF+iOP1oRkIPDIp/YdRQOdQoDD+u7knMKJTgn2Js27aNpUuXkn39dfoeeYT+hx8ht2cPAOFVq4i95TLil11GYP78k/7kcW+2tyD+e/r2sKd3D819zezr34ddtC6/PFjOvLJ5Q4xBU1nT9LtXMB0pNhL5UcMR4e4j00ebbhKaJ/pHGIaKoQZiuK9WN52SKME/xRh+nVJKcrt20f/YY/Q/+hgZf3ooMG8esUsvJXbJJUTOWo0wp25XTdu12Z/cz57ePZ4h8I1BS18L7en2Qj5NaMyKzqKprIl5ZfOGGIX6aP3UrCAqBaT0npROd0Gqe+goIm8QCmndg2l2evQyzejohuGIND8eTKjRxBSjBP8U41jXaR06RP+6dSQfX0dqwwakZaGVlRG76CJif/ImohdfjFF5HDcvJ5lkLklLXwt7+jwDkDcKLX0tpOzBlTIBLUBjWeMQY5B3laFK9SKQySCX8kYLQ4xC1zDDMSz9aFNOxaOJIwxDxbC0ikGDoW5gTxhK8E8xjuc6neQAA398juQTfyD51FM4HR0gBKGVK4hd/CZib7qY0PLlQx70mi5IKWlPt9PS11IwBC39Xnj4FFHcjNNY1lgwAHnD0FjWSFmgbAqvYgaSXwKbn1Ya0Sh0Fx33fWtg9DL14DBDMMwojBZX005HoAT/FONEr1O6LplXXyP55JMkn36KzOYtICV6RQXRtWuJXnwRsQsvxKipmYRWTyy2a3MgeYCWvhb29u9lT+8e9vbvpaWvhQPJA8iiHmZlqJLGeGPBIBSMQbyx8PSyYhpgZ4cagSMMRfeRhiLd5b0zYjTMyKCBOKqRqBiabpTufSQl+KcYE3Wddnc3A888y8AzT5N85lmczk4AgkuXErvoQqIXXkj4rLPQgqdWLynrZGntby2MDPJub99eDqcPD8lbE64ZNATxQYMwNz6XsKFeGDPtkdJ7QG60kcMRRqIofcTN/XwCMd8AlA8agRENR7GhKD8lRhRK8CeYhx9+uLA98o033sitt9465PiPfvQjPvOZzzBnjrcJ2Sc/+UluvPHGMZc/GdcpXZfs66+TfOZZBp55htRLL4FlIUIhIuecQ/TCtUTXriW4ePEpPVeeslLs69/Hnr497O3bWxghtPS10JXp+n/tnXtwXNWd5z+nX+pudbdkSX63GsvyU7ZAsq1uO2Bn4wUTBhw2vDYEYhy7YJmBCrM11MZkKlVDTQ217CZxxgMLJCYmgeUVnoZlcIIN4yS21BhMePiRGPyQhB+ykayWWtbz7B/3qtVtyUhIarW6+/ep6pJv36t7z+GI3++e3++c7y/h2snuyQR8gQRHEPAazsBpG1khESHFaA3tkTiHEO8sGi/w/RdGqOrLHIU9N84pTOjvKC7kPMbQUYjBH0W6u7uZM2dOgjzyM888Q1lZWeyaJwLRjWsAABgySURBVJ54gj179vDQQw8N6xlj0c/ullai4TCtu3bRumsXHZ99BoB1YhG5y5aRu+xr5H5tGfbJQ5Q2SANaOlo4GjFmAseaj8UcwbHmYwn7CxSGfEVvmCgWLvJehN/rF2eQycQchZmc7ucYms6bXfRe98UgjsLdtyfCbc4W+jmLuI9v+lfbNR5HxtS0HQ9cUB45zuCnA1ZPLt6V38C78hsAdB4/bhj/P+2i9Q9/pHnrawA4SkuN+P+yZbiDVVg96Vv9yuPwsKBwAQsK++swNXc0U9tca4SHIkepba7lWOQY249uH9wZeAMU+4olTJQJKAVOn/H5KhvctYaOlv6zhfiVTfEO5PShPicyUI7iaz+AVf88at0aiPQy+P++wajvOppMKYer/ueXXjKYPHIvL774Ijt37mTOnDls3Lgx4XfGI/apU8m//nryr7/eCP8cPEjrrt207tpF029/S+OTT4LViqu8HPeypeQuXYarsmJIgm/pgM/hY0HRAhYUfbkzOBbpmx2c7wwAJrknxUJExd7ihDCRJJAzGKUgx2t8hlJsqBetobOt/4yioCR5bTVJL4OfIoYij7x69WpuvvlmcnJyePTRR7ntttvYsWPHWDVxxCiLBef8+Tjnz6dw/Tp6Ojpo2/sBrbt3Ed1dzZlf/JIzjzxqxP8XLYo5AGfZfJQ184qTDNUZ1EZqYw7h7dq3++UMJromJjoBXzEXeQ3H4HGk78xJGAFKgcNtfPL8Y/ro9DL4g7yJJ4uhyCMXFvZVobr99tv54Q9/OGbtSwYWh4PcUJDcUBD+HrojEaLvvkvr7mqi1btp+OnPaAAseXnkBqtwh5aSu2wpjpkz0zoBPBS+zBm0dLQYDiByLMEp/Kn+T7zS9krCtQXOAsMZmI6gN1QU8AXSQ61USDvSy+CniKHIIx8/fpypU6cCsHXr1pQnmkcbq9eLd+VKvCtXAtDV0EBrTZjW6t1Ed1cT+f1bANgmTsQdCpG7NIR76TIc/iGUTswgPA4PZYVllBX2z+/0riaqjfQ5gtpILeETYV777LWEa30OX6IjiAsTyQ5kYbiIwR8CNpuNhx56iCuvvDImj7xgwYIEeeRNmzaxdetWbDYbBQUFPPHEE6ludlKxTZxI3jVXk3fN1QB01NURra6mdXc1rdXVNL/+OgB2vx/30hC5oaW4Q0Hsk5Kv/DlecdvdzC2Yy9yCuf3Ones6R31LfYIjONp8lA8bPmTbkW30xFX2ctvcsX0Fvc6g2GskkCe5J4k2kXBBZFnmOCGT+tkr/Na6u5pouIbWmjA9zUYtYkdpKbmhEO6lIdxVVUnT/c8kOrs7qW+pT8gX9DqFupa6BDmKHGsOxd5i/F5/wmqigDfAlNwp2VXLIIuQZZlCylBKkTNrFjmzZlHwvVvR3d2c23+AaE0NrTXVNL3yCo1PPw1KkTNvnuEAQkHcVem9BDRZ2K12Q246b0a/c9093ZyInog5gdjPyDF2f76b9u722LU2ZWO6d3psNtAbIir2FeP3+EXCOguQN/xxQrb0E0B3dND28SdEa6ppra6hbe9edEcHWCw4Fy40HUAI96JKLG5Z1jhcenQPDdGGmAM43yG0xgmbKRRTcqcQ8AaM2UFczkCWl45/ZKdtmpEt/RyInvZ2YwloTTXRmjBtH34IXV1gt+O6+OKYA3BVXJJ2GkDjFa01je2NCeGhXqdQ21zbb69BkasoZvzPzx3IiqLUIwY/zciWfg6FntZWou/vNWYANWGj+EtPDyonB1dlJbmhoOEAystTWgAmk4l0RPqcQHNtn1MYQKzO5/DFnEDv7KD3uMhVJCuKxoCUx/CVUv8E3A70lj76kdb6jWQ9T8gcLLm5eJZfhmf5ZUDvHoA9Zg6ghoZ/3QSAcrtxL1pkLAENhXCWlWXkJrBU4HV4L7i8tK2rjfpIfd+MwHQEH53+iN8d/R3dcWUXXTYXfq+fYk9xwmqigC/AFPcUrBYZr7Ek2UnbjVrrnyT5GUKGY+wB6NMA6mpsJBp+N+YATv3kpwBYPB7cS5bE9gHkzJ07LovApDsum4tZE2Yxa8Ksfuc6ezr5vOXzBEdQF6njSPMR/lj/Rzp6+jRkbBYbfo/fcAiSRB4TZJXOEFi3bh2vv/46kyZN4uOPP051c7Ie24QJ+K5che/KVYC5CSwcJlpdQ2u4hpZ33gHAmpeHu3cXcCiIY9YsCS8kGbvFHqtQdj49uodT0VMJyePez95TewdMIifkDeJmCLn23LHsVsaQtBi+GdJZCzQDe4B/0Fo3DnDdHcAdAIFAYPHRo0cTzo+H2PbOnTvxeDysWbMmaQZ/PPQzU+g8ccJ8+w8Tramhs74eAGthIe5glZEEDoZwlMwQBzBO6E0ix88KeiUq6iJ1/TSKEmQpzFlB73F+Tn7WjeuYJG2VUm8BUwY49Y9ANXAao/LxPwNTtdbrvux+4zlpe+TIEa655hox+GlIR11dLPwTrQnTdfIkALZJk3AHg7hDQXJDIezFxVlnKNKFlo6WhBlB/Mqik60nE8pfeuyehJlB/AwhU3cij0nSVmt9+RAb80vg9ZE8C+DB8IMc+OLASG+TwLyCefwwmN5CZ8KX4/D7cfj9hgy01nQePRp7+4+XgbBNnUpuMBjLAdjPE8gTUofH4WF+4XzmF/Z/KWrvbqc+Up+4tDRSy8HGg+w4toMu3bcT2WFxxHIG54eKpnmmYbdk9qqvZK7Smaq1Pm4efhuQ4LeQcpRSOGbMwDFjBhP+602GDMRnn8Xe/lt27uTsq68CYC8ujr39u4Mh7JOzVwdoPJNjzWFm/kxm5s/sd66rp4sTrSdioaFY/qDFEK1r62qLXWtV1ljeIH7TWa+DyITNZ8lM2v4vpVQFRkjnCPDfRnpDeRMXRhulFDmlpeSUllLw3e8ahWD+esh4+w/XEPn9W5x94UUAHDNmGG//oSDuYBBbUVGKWy8Mhs1iw+81VgKdj9aaM+fOxJxA7+ygLlLHtqPbONt+NuH6IldR314Db+IS03TZfJY0g6+1/l6y7i0IyUJZLDjnzsE5dw4Fa76H7u42KoHVhIma4Z+m554DwDGrlNygKQMRFCG4dEMpRZGriCJXEZWTKvudP9t+tm9WEBcqqj5ezdZPtyZc63V4E5xA/Geie+K4yRvITtshcPPNN/POO+9w+vRpJk+ezP3338/69etH9RnjoZ/C4OiuLs7t2xcLAUXffx8djQKQM3duXwioqgqrz5fi1grJ4lzXudhKovMTyZ+3fJ6w+cxpdcZmGec7hameqaOSNxBphTQjW/qZaejOTto++tiUga6h7f296PZ2UArn/PnG238oiHvJElECzRI6ezo50XKiv0aRGS46130udq1VWZmaO5VibzGrS1ezunT1sJ6ZcmkFQcgGlN2Oe1El7kWVFN15Jz0dHZz7859jq4Aan3qKL7ZsAasV54IFZvzfVALNlc1DmYjdYjf2BfiK+53TWtPQ1tBv41ltpJam9qakt03e8McJ2dLPbKPn3DnaPvggFgKKKYHabLjKy2MhIFdlJRanM9XNFdIUecMXhHGAxekkd+lScpcuBaAnGjWVQI1VQGd+uZkzjz6GsttxXXJJLATkqqjA4hAtGWF0EYMvCGOIxe3Gc9mleC67FIDulhba3nsvFgI6/cgj8PDD/aWgFy5EiQMQRogYfEFIIVaPB8/Xv47n618HoLu5meiePTEtoJgUtMuFu7Iytg/AuXAhyib/+wpfDfmLEYRxhNXnw7tyJd6VKwFTCvrdd40loOEwDRs30oAxU3AtWUxuaKlRC2D+PKkFIAyKGPwhMJg88tmzZ7n11ls5duwYXV1d3HvvvXz/+99PQUuFTMM2YQK+VavwrTKloM+cIRoOx+SgT+383wBYvF6zFoCRBJZaAMJAiMEfAmvXruXuu+9mzZo1A55/+OGHKSsr47XXXqOhoYG5c+dyyy234JCYqzDK2AoL8V11Fb6rrgKg8+QpcwZg1gJ4+20grhZA0EgC58yeLUqgghj8obBixQqOHDlywfNKKSKRCFprWlpaKCgowCbxVWEMsE+eRN41V5N3zdVA/1oAkd+/BYC1oAB3MBjTAXLMnCkOIAtJK6t04oEHaN8/uvLIOfPnMeVHPxrRPe6++26+9a1vMW3aNCKRCM899xwWmU4LKcA+ZQp5115L3rXXAtBRV0+0psbcCRwm8uabAFgnFpFbFScFHQiIA8gC0srgj1e2bdtGRUUFO3bs4NNPP+WKK65g+fLl+ERLRUgxDv90HP7ryL/+OqMWQG1tnw5QTQ3Nb7wBgG3KlL5dwKEgDn9/dUkh/Ukrgz/SN/FksWXLFjZs2IBSilmzZlFSUsKBAwcIBoOpbpogxFBK4QgEcAQCTLjxRqMWwOEjMR2glj/8kbOvGiqQ9mnTYpvAckMh7FOnprj1wmiQVgZ/vBIIBNi+fTvLly/n5MmTHDx4kJkz+xdjEITxhFKKnJkl5MwsYcJ3vmM4gEOHYvH/lh07OPvyywDYA4FY/F+KwaQvYvCHQLw8st/v5/7776ezsxOAO++8kx//+MesXbuW8vJytNY8+OCDFElxDCHNUEqRM3s2ObNnU3DrLUYxmL/8JZYEbn5zG02/fQGQYjDpioinjROypZ9C+qK7uzm3/4CZBA4T3bOHntZWQIrBpBoRTxMEYVRRViuuhQtwLVxA4fp1/YrBNL3yCo1PPw2cVwxmyRKseelRAjDTEYMvCMKwUDYbrosvxnXxxXD77X3FYGqqaQ2HaXrueRp/86QUgxlHiMEXBGFUSCgG87d/O3gxmGCV4QQWLZJiMGOEGHxBEJKCxeHAXVWFu6oK7r4rVgwmGg7TWhPmzK9/w5nNjxvFYBYujCWBXZWVWFyuVDc/IxGDLwjCmBBfDGYiZjGYvXtjm8DObN7MmcceA7sd18UXxzaCuSorsOTkpLr5GYEYfEEQUoLF7cZz6aV4Lu0tBtNK2/vvxZLApx99DP7PIyiHA1dlZV85yPJyKQYzTMTgD4HB5JEbGxtZt24dn376KU6nk1/96lcsXLgQgI0bN7J582aUUpSXl7NlyxacUrtUEPph9eTiWbECz4oVQG8xmPdMJdAwp//tIU5v+jeU02nkCkwZCNfChSi7PcWtTw/E4A+BweSRH3jgASoqKnj55Zc5cOAAd911F9u3b6e+vp5Nmzaxb98+XC4XN910E88++yxr164d2w4IQhpiFIP5Bt6V3wCgu6mJ6J49sSRww89/DpjFYBYvjm0Cc5aVSTWwCyD/VYbAYPLI+/bt47777gNg3rx5HDlyhJMnTwLQ1dVFW1sbdrudaDTKtGnTxqLJgpBxWPPz8V5+Od7LLweg64sviIbfjSmBnvrJTwGweDy4Fy+OLQN1zpNqYL2klcH/w/N/4XRty6jes6jYw/Kb5ozoHpdccgkvvfQSl112GeFwmKNHj1JXV8fixYu59957CQQCuFwuVq1axSqzcpEgCCPDVlCA75tX4vvmlQB0NTQYlcB6tYD+4z8AsOTl4a5aYu4ENovBZKl8eVoZ/PHKhg0buOeee6ioqKC8vJzKykpsNhuNjY28+uqrHD58mPz8fG688Uaeeuopbr311lQ3WRAyDtvEieRdfTV5V5vFYE6eNJaAVlcTrQnT8tZ2AKwTJhjLRc0ksKO0NGtqAaSVwR/pm3iy8Pl8bNmyBQCtNSUlJZSUlLBt2zZKSkqYOHEiANdddx27du0Sgy8IY4B98mTyVq8mb/VqADrr62Px/9ZwmMjvfgeAtajI2ATWWwtgxoyMdQBpZfDHK01NTbjdbhwOB5s3b2bFihX4fD4CgQDV1dVEo1FcLhfbt29nyZJB9Y0EQUgC9unTyb/u2+Rf922jGExdXUI5yOY3/h0A26RJMRG43FAIe3FxxjgAMfhDYDB55P3797NmzRqsVitlZWU8/vjjAIRCIW644QYWLVqEzWajsrKSO+64I5VdEQQBsxhMcTGO4mLyb7jBqAVw5IgR/w+Had29m+bXXgPANm1qXznIUBD79Okpbv3wEXnkcUK29FMQ0gGtNR2ffZZQDrK7qQkAu9/fpwQaCmGfPDnFrRV5ZEEQhGGjlCKntJSc0lIKvvtdoxjMXw+Z8f8aIm9t5+yLLwHguOiivnKQwSA2M2c3HhGDLwiCMAjKYsE5dw7OuXMoWPM9dHc37QcPxsX/36Dp+ecBcJSWxuL/7mAQW0FBilvfhxh8QRCEr4iyWnGWleEsK6Pw+2uNYjD795tKoDU0v7qVpmeeBSBn9uy+GUBVFdb8/JS1Oy0MvtY6Y7LkAzGe8iiCIHx1lM2Gq7wcV3k5hevXozs7OffJJ7EZQNMLL9D41FOgFDnz5pEbNJLA7qolWL3esWvneDI2AyVtDx8+jNfrpbCwMCONvtaaM2fOEIlEKCkpSXVzBEFIArqjg7aPPoolgdv27kV3dIDFgrOsDHcoiO+KK3BVVAzr/mOStFVK3Qj8EzAfCGqt98Sduw9YD3QDP9BabxvOM/x+P3V1dTQ0NIykqeMap9OJ3+9PdTMEQUgSyuEw9H0WL4a/+zt62ttp++DPZgiomi9+8ySWHOewDf6Q2zGSN3yl1HygB3gMuLfX4CulyoBngCAwDXgLmKO17v6y+w30hi8IgpDp9LS1oTs6hl3sfahv+CNSENJa79daHxzg1LXAs1rrdq31YeAQhvEXBEEQzsPicg3b2H+l5yTpvtOB2rjjOvM7QRAEIUUMGsNXSr0FTBng1D9qrV+90K8N8N2AsSOl1B3AHQCBQGCw5giCIAjDZFCDr7W+fBj3rQOK4479wOcXuP8vgF8AKKUalFJHh/E8gCLg9DB/N53Jxn5nY58hO/udjX2Gr97vi4ZyUbLW4W8FnlZK/QwjaTsbCA/2S1rrYe9JVkrtGUrSItPIxn5nY58hO/udjX2G5PV7RDF8pdS3lVJ1wDLg/ymltgForT8Bngf2AW8Cdw22QkcQBEFILiN6w9davwy8fIFz/wL8y0juLwiCIIwemVTY8RepbkCKyMZ+Z2OfITv7nY19hiT1e1xJKwiCIAjJI5Pe8AVBEIQvISMMvlLqm0qpg0qpQ0qpDaluTzJQShUrpd5WSu1XSn2ilLrH/L5AKfV7pdRfzZ8TUt3WZKCUsiql9iqlXjePS5RSNWa/n1NKOVLdxtFEKZWvlHpBKXXAHPNl2TDWSqn/bv59f6yUekYp5czEsVZK/UopdUop9XHcdwOOrzLYZNq3D5VSi4b73LQ3+EopK/AwcBVQBtxsavlkGl3AP2it5wNLgbvMfm4AtmutZwPbzeNM5B5gf9zxg8BGs9+NGEJ9mcS/Am9qrecBl2D0PaPHWik1HfgBsERrvRCwAt8hM8f6CeCb5313ofG9CmNp+2yMTaqPDPehaW/wMTR6DmmtP9NadwDPYmj5ZBRa6+Na6/fNf0cwDMB0jL7+2rzs18B/SU0Lk4dSyg9cDWw2jxWwEnjBvCSj+q2U8gErgMcBtNYdWusmsmCsMVYOupRSNsANHCcDx1prvRP44ryvLzS+1wK/0QbVQL5SaupwnpsJBj/rdHuUUjOASqAGmKy1Pg6GUwAmpa5lSePnwP/AUGYFKASatNZd5nGmjflMoAHYYoaxNiulcsnwsdZa1wM/AY5hGPqzwHtk9ljHc6HxHTUblwkGf8i6PZmAUsoDvAj8vda6OdXtSTZKqWuAU1rr9+K/HuDSTBpzG7AIeERrXQm0kmHhm4EwY9bXAiUYO/RzMcIZ55NJYz0URu3vPRMM/pB1e9IdpZQdw9j/X631S+bXJ3und+bPU6lqX5K4FPiWUuoIRrhuJcYbf7457YfMG/M6oE5rXWMev4DhADJ9rC8HDmutG7TWncBLwNfI7LGO50LjO2o2LhMM/rvAbDOT78BI8mxNcZtGHTNu/TiwX2v9s7hTW4HbzH/fBlxIwTQt0Vrfp7X2a61nYIztDq31LcDbwA3mZRnVb631CaBWKTXX/Oo/Y8iUZPRYY4Ryliql3Obfe2+/M3asz+NC47sVWGOu1lkKnO0N/XxltNZp/wH+BvgL8CmGbHPK25SEPl6GMY37EPjA/PwNRjx7O/BX82dBqtuaxP8G/wl43fz3TAxBvkPAb4GcVLdvlPtaAewxx/sVYEI2jDVwP3AA+Bh4EsjJxLHGqAh4HOjEeINff6HxxQjpPGzat48wVjEN67my01YQBCFLyISQjiAIgjAExOALgiBkCWLwBUEQsgQx+IIgCFmCGHxBEIQsQQy+IAhCliAGXxAEIUsQgy8IgpAl/H9ZEW57ZUBfkAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "run -i nt_solutions/ml_2_regression/exo0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The optimal step size to minimize a quadratic function $\\dotp{C w}{w}$ is\n", "$$\n", "\\tau_{\\text{opt}} = \\frac{2}{\\sigma_\\min(C) + \\sigma_\\max(C)}\n", "$$" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Optimal tau = 1.9032630425437524 / |X_0 X_0^T|\n" ] } ], "source": [ "C = X0.transpose().dot(X0)\n", "tau_opt = 2 / ( np.linalg.norm(C,2) + np.linalg.norm(C,-2) )\n", "print(( 'Optimal tau = ' + str( tau_opt * np.linalg.norm(X0,2)**2 ) ) + ' / |X_0 X_0^T|' );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Stochastic Gradient Method\n", "=======\n", "\n", "We use SGD (which is *not* actually a descent algorithm) to minimize the quadratic risk\n", "$$ \\umin{w} J(w) \\eqdef \\frac{1}{n} \\sum_{i=1}^n f_i(w) = \\frac{1}{n} \\sum_{i=1}^n \\frac{1}{2} ( \\dotp{w}{x_i}-y_i )^2 $$\n", "where we used\n", "$$ f_i(w) \\eqdef \\frac{1}{2} ( \\dotp{w}{x_i}-y_i )^2$$\n", "The algorithm reads\n", "$$ w_{k+1} = w_k - \\tau_k \\nabla f_{i_k}(w_k)$$\n", "where at each iteration $i_k$ is drawn in $\\{1,\\ldots,n\\}$ uniformly at random.\n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "w = np.zeros((p,1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Draw $i_k$ are random. " ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "ik = int( np.floor(np.random.rand()*n0) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compute $\\nabla f_{i_k}(w_k)$." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "gk = (X0[ik,:].dot(w)-y0[ik]) * X0[ik,:].transpose()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Set the step size $w_k$ (for convergence is should converge to 0) and perform the update." ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "tauk = 1/np.linalg.norm(X0,2)**2\n", "w = w - tauk * gk" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Test different *fixed* step size $\\tau_k=\\tau$. " ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnWd4VNXWgN89M6mkF3oJ0qVD6CBdih27iHgt2O69YvnsV1GvDS/2iqKCYqUoXYogVSEESKiS0EtIgFTSZ/b340zNzCQTkkkZ9/s888yZffY5Z00yZ521115rbSGlRKFQKBS+i662BVAoFAqFd1GKXqFQKHwcpegVCoXCx1GKXqFQKHwcpegVCoXCx1GKXqFQKHwcpegVCoXCx1GKXqFQKHwcpegVCoXCxzHUtgAAMTExMi4urrbFUCgUinrF9u3bz0opYyvqVycUfVxcHAkJCbUthkKhUNQrhBBHPelXoetGCBEohNgqhNglhNgjhHjR3L5BCLHT/DolhPjZ3D5MCJFtt+/5qn0VhUKhUFQFTyz6ImCElDJPCOEHbBRCLJdSDrF0EELMB36xO2aDlPLKapZVoVAoFBdBhRa91Mgzf/Qzv6wlL4UQocAI4GevSKhQKBSKKuFR1I0QQi+E2AmkA6uklH/a7b4OWCOlzLFrG2B29SwXQnR2c84pQogEIURCRkbGRX8BhUKhUJSPR4peSmmUUvYAmgN9hRBd7HbfCnxn9zkRaCWl7A68jxtLX0o5U0oZL6WMj42tcNJYoVAoFBdJpeLopZRZwDpgLIAQIhroCyy165NjcfVIKZcBfkKImOoSWKFQKBSVw5Oom1ghRIR5OwgYBew3774RWCKlLLTr31gIIczbfc3XOFfdgisUCoXCMzyx6JsAa4UQScA2NB/9EvO+W3B02wDcAOwWQuwC3gNukV5ar/BUVgH/98syTmXnVdxZoVAo/qZUGF4ppUwCerrZN8xF2wfAB1WWzAM2nNjIiqwnyVt3Hx9f88+auKRCoVDUO+p1rZvuTVsAsGb/CbILSmpZGoVCoaib1GtFH3HmAACXcJaNB8/WsjQKhUJRN6nXil4f3QaAaJFFem5hBb0VCoXi70m9VvQGv2AAdMLIWyv/qmVpFAqFom5SrxW93hAIgMBIblEpJUZTLUukUCgUdY96regNek3Rx8Vo72nZyn2jUCgUZanfit4vCICwQAHAkXMXalMchUKhqJPUa0Wv1/sDYNBr+VgZuUW1KY5CoVDUSeq1otcJs/jCCEBWvoqlVygUirLUa0UvhMAgJaBNwn7zh0eraikUCsXfinqt6EGr4WCSRnQCcotKa1schUKhqHPUe0Wvl1BiKmVS/1Zk5BaRU6jcNwqFQmFP/Vf0gFEa6dgkDICFiSdrVyCFQqGoY9R7RW9AYJRGxndtAsALi/bUskQKhUJRt/ABRa9Z9OFBfkQG+wGw+2R27QqlUCgUdYh6r+j1CEpMWnjliqmXAXDl+xtrUySFQqGoU9R7RW9AYDSHVzYKC6xlaRQKhaLu4cmasYFCiK1CiF1CiD1CiBfN7V8JIQ4LIXaaXz3M7UII8Z4QIkUIkSSE6OXNL6BNxtqKmd0/tA3+eh1eWr1QoVAo6h0VLiUIFAEjpJR5Qgg/YKMQYrl53/9JKeeV6T8OaGd+9QM+Nr97BYPQUSqN1s9RDfwoNpq4UGwkJMCTr6dQKBS+TYUWvdSwrL7tZ36VZy5fA8wxH/cHECGEaFJ1UV2jR1BqZ9FHBGv1bzIvFHvrkgqFQlGv8MhHL4TQCyF2AunAKinln+Zdr5jdM28LIQLMbc2A43aHnzC3lT3nFCFEghAiISMj46K/gEEIjHZumuYRWkVLVclSoVAoNDxS9FJKo5SyB9Ac6CuE6AI8DXQE+gBRwJPm7sLVKVycc6aUMl5KGR8bG3tRwgPo0WO0c910MidOPfBN4kWfU6FQKHyJSkXdSCmzgHXAWCnlabN7pgj4Euhr7nYCaGF3WHPgVDXI6hKDTufguolsoLlu8lTdG4VCoQA8i7qJFUJEmLeDgFHAfovfXQghgGuB3eZDFgF3mKNv+gPZUsrTXpEeMAiDg6IH+PeItgAYTSryRqFQKDwJS2kCzBZC6NEeDD9KKZcIIX4TQsSiuWp2Aveb+y8DxgMpQD7wj+oX24Zepycfo0NbrDmePiO3iMbhKrZeoVD8valQ0Uspk4CeLtpHuOkvgYeqLppnhOoDKTtciIsOBuDw2QtK0SsUir899T4zVq8zYMIEdpE37RqGAvDfpXtrSyyFQqGoM/iIogeMtrj5xuGBNI8MYs+pHJUhq1Ao/vbUe0Wv0xkwIaAk36H9zoFxgFpHVqFQKHxA0fthEkBJgUO7pcBZRl5RLUilUCgUdQffUPTgpOijQ7R4+tX7ztS8UAqFQlGHqP+KXu+HEeGk6Hu2iATg2z+P1YZYCoVCUWeo/4pe54cUQP5Zh/Ygfz2jL23EicwCftuvrHqFQvH3pf4rer2fli415xqnfS9e3RmA15btr1mhFAqFog5R7xW9HqFF3bigqbmSpSqFoFAo/s7U+5U5RGCEFnWj83O5v09cJAZdvX+eKRQKxUVT7zWgPjBci7oJc722SVigH+cuqBBLhULx96XeK3qd0GmKPst1dI0Q8NeZPEzKfaNQKP6m+IaiFzoIa+5yf8+WWpjl3K0qzFKhUPw98RFFDwRHudz/4LA2APzn590u9ysUCoWv4xOK3ghgdF3TRlsXRWN5stfWP1EoFIo6S71X9HqhB0Aa3U+4Lv7nYADmbDlaIzIpFApFXcKTpQQDhRBbhRC7hBB7hBAvmtvnCiEOCCF2CyG+EEL4mduHCSGyhRA7za/nvfkFLBa70eh+jdiuzcPp2TKC9NxCb4qiUCgUdRJPLPoiYISUsjvQAxhrXgt2LtAR6AoEAffYHbNBStnD/HqpuoW2x2LRm7KPQTnKvm/rKI6dz6e41OS2j0KhUPgiFSp6qZFn/uhnfkkp5TLzPglsBVyHvXgZndC+glEIOLHVbb/OTcMpMUpS0vPc9lEoFApfxCMfvRBCL4TYCaQDq6SUf9rt8wMmASvsDhlgdvUsF0J0rlaJy2DQacm9JgC9v9t+bWIbAHDorFL0CoXi74VHil5KaZRS9kCz2vsKIbrY7f4IWC+l3GD+nAi0Mrt63gd+dnVOIcQUIUSCECIhIyPj4r+A2aIvFWjZUW5oExtCaICBJbtU5I1Cofh7UamoGyllFrAOGAsghHgBiAUeteuTY3H1SCmXAX5CiBgX55oppYyXUsbHxsZe/BcwK3oTAkzu/e+BfnpaRAWzYk/aRV9LoVAo6iOeRN3ECiEizNtBwChgvxDiHmAMcKuU0mTXv7Ewh8IIIfqar3HOG8IDGITmujECSGO5fWNCAwBIz1HRNwqF4u+DJxZ9E2CtECIJ2Ibmo18CfAI0AraUCaO8AdgthNgFvAfcYp6w9Qo6nd1krKl8RX/vkNYAvLpsn7fEUSgUijpHhWWKpZRJQE8X7S6PlVJ+AHxQddE8wxpeCWByH14JMKSd5iLan5brZakUCoWi7uAzmbGlggpdNwCXxDRgf1ouuYWuSyYoFAqFr1HvFb2nk7EWburTAoCu01aq0sUKheJvgc8oegmw5sUK+9/ap6V1Oz1XLUiiUCh8H59R9CYBpCVV2D882I+Zk3oD8KsKtVQoFH8D6r2itxQ1q4wTZmBbLaz/hUV7vCCRQqFQ1C3qvaLXmb+CqVlvj48JCTAwqlNDQMXUKxQK36f+K3qL66ZZvNZQ4pnifmh4WwA2pZ71ilwKhUJRV6j3it7qumkQrTWcTPDouM5Nw/HTCzYcVIpeoVD4NvVe0VtdNw3M5XS+usKj4/wNOkID/TijXDcKhcLHqf+K3hJeGRhR6WPjW0WSnqNCLBUKhW9T7xW9xXVjimhV6WMbhQWSkacUvUKh8G3qvaK3TsbqBHS5HqLbenxso7AAsvJLKCypuHSCQqFQ1FfqvaIXmCdjpQSdocLCZva0iQ0BVJEzhULh29R/RW9NmLIoes+t805NwgA4kJbjFdkUCoWiLlDvFb3VdSNNoNNXyqJvHhmEv17HnlNK0SsUCt+l/it67BW9AYyelx826HUMahvNnC1HybxQ7C0RFQqFolap94re6rq5CB89wPW9mwPwQ8LxapdNoVAo6gKerBkbKITYKoTYJYTYI4R40dzeWgjxpxDioBDiByGEv7k9wPw5xbw/zqtfwFqP3gR6fygtgkqsXHhlt6Z0bRbOvO0nvCWiQqFQ1CqeWPRFwAgpZXegBzBWCNEfeAN4W0rZDsgE7jb3vxvIlFK2Bd429/MaDj760MZQWgAXKlfW4Mb45qSk55F0IssbIioUCkWtUqGilxp55o9+5pcERgDzzO2zgWvN29eYP2PeP1JY/CtewCG80hCoNeZUzjrv11qrk7Nit6pPr1AofA+PfPRCCL0QYieQDqwCUoEsKaXFIX4CaGbebgYcBzDvzwaiq1Noexws+sg4rbESIZYAraKDAfhoXapaXlChUPgcHil6KaVRStkDaA70BTq56mZ+d2W9O2lPIcQUIUSCECIhIyPDU3mdsC0lKMEycCisnAsm0E/PmM6NANh65PxFy6JQKBR1kUpF3Ugps4B1QH8gQghhMO9qDpwyb58AWgCY94cDTtpTSjlTShkvpYyPjY29OOmxuW5M0gRBkVrjhrcrfZ63b+5BoJ+OlXvOXLQsCoVCURfxJOomVggRYd4OAkYB+4C1wA3mbpOBX8zbi8yfMe//TcpKhMFUEofwSssqU0c3grFyYZbB/gYuiQnh0Nm8ijsrFApFPcITi74JsFYIkQRsA1ZJKZcATwKPCiFS0Hzws8z9ZwHR5vZHgaeqX2wbRUat+uQ3+75x3HEmudLn6t4igo0Hz6oa9QqFwqfwJOomSUrZU0rZTUrZRUr5krn9kJSyr5SyrZTyRillkbm90Py5rXn/IW9+gfT8dAC2pm3VGm79QXsvqnyhsjsGtKLUJHl24e7qEk+hUChqnXqfGRvfSFsrdmjzoVqDZaUpD9eOtadTkzBu6dOCtQfSVelihULhM9R7RR8dFI1e6Gkf2V5rsMTS//byRZ1vSLtYjCbJ7pPZ1SShQqFQ1C71XtED+Ov9KTGZi5mFNtbe05Iu6lyD28YQ5Kdn4Y6T1SSdQqFQ1C4+oegNOoNN0VtcNxdJeLAf4UF+LEs+XQ2SKRQKRe3jE4reT+dHsdGuzHD7cdC460WfLy2nkMz8EhJU8pRCofABfELR++v9HRW9Tl/pMgj2fHBbTwCmLd5TVdEUCoWi1vEJRR/qH0p2kd3kqd6vUguQlOXKbk0JDTDQwN9QcWeFQqGo4/iEog/UB9p89AA6PzBdvKIHGHVpI05kFlRRMoVCoah9fELRG3QGSu1XlqrkIuGuaBUdzMmsAs7lFVVROoVCoahdfELR++n8HC36g79C9nEovPhFv/u2jgJgy6FzVRVPoVAoahWfUPROFn2THtp75pGLPmeXZuEALE9Wi5EoFIr6jc8oegeLfvAj2vvJhIs+Z1igH91bRPDrnjQuFFWuEqZCoVDUJXxC0a8/sZ595/fZGswVLVnySJXOe/fg1pSaJC8v2Vul8ygUCkVt4hOK3ommvarlNKM7aatOfb/tOMfO5VfLORUKhaKm8SlFb13fJDiqWs4X5K9n+cNDALjszbWk56o69QqFov7hE4q+e2x3AIzSLqQypFG1nLtTkzCahGsVMfu+sqZazqlQKBQ1iU8o+hEtRwA4Tsj2uE1LnKqGVQzXPDbUup2Rq+LqFQpF/cKTNWNbCCHWCiH2CSH2CCEeNrf/IITYaX4dEULsNLfHCSEK7PZ94u0vYTCvUe4QYhkQpmXHViHE0kKwv4H/3aiNGr7afLjK51MoFIqaxBOLvhR4TErZCegPPCSEuFRKebOUsoeUsgcwH1hgd0yqZZ+U8n4vyO2An94PKGPRH1ylvb/Xo1qucW2PpgB8uDaV7PyqlVdQKBSKmsSTNWNPSykTzdu5wD6gmWW/EEIANwHfeUvIijDoXFj0zcyRN7Edq+caeh33DG4NwMIdJ6rlnAqFQlETVMpHL4SIA3oCf9o1DwHOSCkP2rW1FkLsEEL8LoQYUmUpK+B0nrZIyNGco7bGkc9r7yXVFxb57BWduCS2AV9tPlJt51QoFApv47GiF0KEoLlopkop7YvI3IqjNX8aaCml7Ak8CnwrhAhzcb4pQogEIURCRkbGxUlvZlHqIod3AAwB2nvWsSqd2x4hBJP6t+LIuXz+OpNbbedVKBQKb+KRohdC+KEp+blSygV27QZgAvCDpU1KWSSlPGfe3g6kAu3LnlNKOVNKGS+ljI+Nja3Sl/DTaT56gXDcEdJYq2RZjQxoEw2gsmUVCkW9wZOoGwHMAvZJKd8qs3sUsF9KecKuf6wQQm/evgRoBxyqPpGdebzP4wCMbDnScUf7yyE4ulqv1bGxNjjZcPBstZ5XoVAovIUnFv0gYBIwwi5kcrx53y04T8JeBiQJIXYB84D7pZReXXw1JkhbEFwnynydwAjIO1MtsfT2XNGtCQAns9TCJAqFou5ToV9DSrkRyvpErPvudNE2H83NU2PotQEEJmly3LHre+39+FZo2a/arnfXoNYsTTrNgbQcmkUEVdt5FQqFwhv4RGasxZJ3KIEA0N8cwm+/cHg10DqmAQBT5mynoLhqK1kpFAqFt/EJRe/Wom/RX3sv+wCoIlEN/LmyWxNKTZJOz6+o1nMrFApFdeMTit6tRe+nFSOjtPrr07x9sy3jtrBEWfUKhaLu4hOK3q1FbzAr+pLqnzT10+t4/9aeAOw4llXt51coFIrqwicUvVuL3qLoU3/zynX7XxKNELD1sFeDihQKhaJK+ISid2vRN9DCLq0FzqqZ2NAA2saGsPd0tlfOr1AoFNVB9aaN1hI6ndmiN5Wx6APDIbwFxA322rUbhQWSqFw3CoWiDuPbFj1oNW+qObzSnibhgWTkFvGv73Z47RoKhUJRFXxC0bv10YO2ypTRe/Xjx5uzZBfvOsWafWe8dh2FQqG4WHxK0bu06HUGsK9TX80M79CQnx8aBMBH61K9dh2FQqG4WHxC0VtcNy4ter3BqxY9QI8WEdw5MI7tRzP549A5r15LoVAoKotPKPryLXo/be1YL3NTfAsAbpn5ByZT9RZRUygUiqrgE4q+3MlYvT8UV98qU+64tGkYnZpoJYyTT6pwS4VCUXfwCUVf7mRsWBPIS6sROWbf1QeATamqVr1Coag7+ISi1+vKsehDGmnLCeZ7P3u1YWggnZqE8fHaVGQ118BXKBSKi8UnFL3Voi+bMAVQZF7ednrrGpFlSLsYcotKWZJ0ukaup1AoFBXhE4q+XB99gNO65F7lnyPaEhPiz+vL92NUk7IKhaIO4MmasS2EEGuFEPuEEHuEEA+b26cJIU66WF4QIcTTQogUIcQBIcQYb34BqMBH7xfs7cs7EBboxxNjO3Iyq0CFWioUijqBJxZ9KfCYlLIT0B94SAhxqXnf21LKHubXMgDzvluAzsBY4CPLYuHeRC/0ri36Pndr7xGtvC2ClZEdGwIw8fM/1QpUCoWi1qlQ0UspT0spE83bucA+oFk5h1wDfC+lLJJSHgZSgL7VIWx56ITOtUUf2hjajoLAmnPhRIcEEOyvPdv+b96uGruuQqFQuKJSPnohRBzQE/jT3PRPIUSSEOILIUSkua0ZcNzusBO4eDAIIaYIIRKEEAkZGRmVFrwsbi16gJTVkJYMF2rOlbLj+dEALEk6zb7TOTV2XYVCoSiLx4peCBECzAemSilzgI+BNkAP4DQww9LVxeFOs5JSyplSyngpZXxsbGylBS+LW4ventxTVb6OpwQY9Cz9t1Yeedy7G7hQ5L16OwqFQlEeHil6IYQfmpKfK6VcACClPCOlNEopTcBn2NwzJ4AWdoc3B7yuYcu16Ee+oL1nHPC2GA50bhrOq9d1BeDVZftUbL1CoagVPIm6EcAsYJ+U8i279iZ23a4Ddpu3FwG3CCEChBCtgXbA1uoT2TU6nc51HD1Aw07a+/y7vS2GE7f1a8mAS6KZ++cx4v+7Wk3OKhSKGscTi34QMAkYUSaUcroQIlkIkQQMBx4BkFLuAX4E9gIrgIekrMinUnXKteh1ft6+fLm8eE1nAM5dKOaJ+Um1KotCofj7UeFSglLKjbj2uy8r55hXgFeqIFelKddH324U+IdCo0td7/cy7RuFsvelMVz6/K8s3nWK5pFBPDGmA9pgSaFQKLyLT2TGgqbo3Vr0AK0GQGlhzQlUhmB/AwsfHAjAx+tSaf30MvLUBK1CoagBfEbR64W+/KgbQwCUem/tWE/o2TKSP54eaf089Xu1zqxCofA+PqPoK7To9QFgLKo5gdzQODyQw69p1SJW70sn7qmlpGXX3khDoVD4Pj6j6Cu26AOhtPYVPYAQgqfGdbR+fn35vlqURqFQ+Do+o+grtOgN/nVG0QPcP7QNu56/HICfd54i7qmlfPPH0VqWSqFQ+CI+o+jLDa8EMARB8QWoQ0lL4cF+fD+lv/Xze2sO1qI0CoXCV/EZRV9uwhRARAsoLaiRlaYqQ/9Lojny+hU8Pa4j6blFJB7LrG2RFAqFj+Ezit5i0f+S8gsLDi5w7hDaWHs/lVizgnnI1T2aAjDho82Y1IIlCoWiGvEZRW9JmHpu03O8sPkF57oyQebimgVZNS+cBzQJD+Ky9lpxt+Ez1lFiLMcNpVAoFJXAZxR9WR/9T3/95Ngh3Fxn7fgfNShV5fjyzj4MahvN0XP53PjJltoWR6FQ+Ag+o+j9dH4Um2wJUS//8bJjB4uiL86vQakqh14nmHNXPwB2Hs/img83UVyqLHuFQlE1fEbRhwWEkVlYzkSmwR8i48BUt8sO6HWCHf8ZjRCw63gW7Z9bzrRFeygsUVUvFQrFxeEzij7cP5yUrJTyO9WR7NiKiGzgz6FXrWut89XmI0z9fmctSqRQKOozFVavrC8EGgIdPvdu1Nu5k8EfjCU1JFHVEELw69TLSM8t5Idtx1mxO4384lKC/X3mX6ZQKGoIn7Ho9ULv8NllTL2+bmXHVkSHxqEMaRfLVd2bUmqSXPr8r6Sk59a2WAqFop7hM4q+bFZsqStfvN4fUtfU+JKCVaVb83Dr9oSPNit/vUKhqBSeLCXYQgixVgixTwixRwjxsLn9TSHEfiFEkhBioRAiwtweJ4QosFuN6hNvfwmA7w987/DZZYGzAvNk7ceDakCi6qNJeBCz7+rLkHYx5BSWMuJ/62pbJIVCUY/wxKIvBR6TUnYC+gMPCSEuBVYBXaSU3YC/gKftjkmVUvYwv+6vdqldEOoXat02CAMlJhe++JzT2rurfXWcoe1jmTkpHoBT2YU88oOanFUoFJ5RoaKXUp6WUiaat3OBfUAzKeVKKaXFP/IH0Nx7YlbMY/GPWbdHthrp2qK3lEGAOlXczFOC/PUc+O9YABbuOElqRl4tS6RQKOoDlfLRCyHigJ7An2V23QUst/vcWgixQwjxuxBiSJUk9JBBzWzuGL3Qu/bRt+hr275wtgakqn4CDHo+naRFFL235iClqlSCQqGoAI8VvRAiBJgPTJVS5ti1P4vm3plrbjoNtJRS9gQeBb4VQoS5ON8UIUSCECIhIyOjKt8BgJigGOu2QWdwHXUz7g1ofZm2/cWYKl+zthjTuTEdG4fyy85TDHrjNzU5q1AoysUjRS+E8ENT8nOllAvs2icDVwITpbmKmJSySEp5zry9HUgF2pc9p5RyppQyXkoZHxsbW+UvYtAZHLZdWvT+DWDMq9r2+dQqX7M2mffAQPrGRXEmp4jvtx6rbXEUCkUdxpOoGwHMAvZJKd+yax8LPAlcLaXMt2uPFUILahdCXAK0Aw5Vt+DloRd6SqWbUgdBUbbt1LU1I5AXCAkw8MN9/QkNMDBt8V7+9+sB54qdCoVCgWcW/SBgEjDCLmRyPPABEAqsKhNGeRmQJITYBcwD7pdS1shqH7PHzmbxtYvLX23KUq4YIHFOTYjlNYQQzLqzD/56HR+sTeGjdfV7lKJQKLxDhfn0UsqNgHCxa5mb/vPR3Dw1Tq9GvYByfPQAfkHQZgSk/gYHlrvuU4/o2zqK358YxoDXfuPNXw8QF92AK7o1qW2xFApFHcJnMmPtKdd1IwRMWqhtlxZoCr+e0yQ8iO3PjaJlVDAPfZvIt38eI7ug/uUKKBQ1zpGNcHh9bUvhdXxS0et0OgpKCzzrvP5/3hXGDXnFeXSd3ZV5f82rlvNFhwQweWAcAM8sTCb+v6uQUpJdUEJRqYrKUShc8tUVMPuq2pbC6/ikop+zR/O9l1uf3sLRTV6WxjUZBVpI6Vd7vqq2c949uDXxrbQ5iBKjpPXTy+j+4ko6PLeC/Wk5FRytUPgeZwvO8lnSZ6532leyPX8Idi8AkwmSfqxXxQ89wSdr3lqyYlOzUolvHO+6U/ux8NcKCG1ag5LZMAjtT+8yDLQKzHtgIEfOXmBYmXo4Y9/ZwPQbunFTfItqvV51UlBaQGZhJk1Daud/ovA9Rv40EpM0EWgIZNKlkxx3nj1o236vp3njH9pbzikYPLVGZKwJfNKit5QslpQTbnjzN9p77qkakMiZcmWrInExDZh3/wA+ub03ydMu57M7tIfdE/OSyM6vu777x39/nDHzx6gwUUW1YYm+W3bIRexIQTnBgKtf8JJEtYNPKvr3RrwHUL7C0PtB+3E1JJEzlslibym1+LgoxnZpTGigH6MvbcScu7TyD91fWllnSyesP6FNim06VTvuNF8l7UIal8+7nIOZByvu7GP0adwHgN3ndnO2oEzZk4Xmeos9b3d9cD1ZpMgTfFLRh/lrFRdWHFlRfscIsxujFixIi8vm1IWaGVEMaRfDwDbRALy16i/aPru8zi5iciqvdkZZvsqqo6s4feE08w/WStRzrdLAr4F1+1zBOduOQ79D9nFte/izMMGFH7/Qd+a1fFLR++n9APjpr5/4cveX7jsGmEvwZB2tAakcqW7ffEUIIfj23v6kvjqeKZddAsCot9bz8Pc7KKkj1v2w5sMA741y/q5YyoPU9G+uruGyoi2kT90LAAAgAElEQVRAWFPodhM8fRKi2oBfsNae9EPNCedlfFLR++v8rdtvbX/LfccOZtdNLcTS/5X5V41fE0CvEzwzvhMzbuwOwC87T3HTp1s4eCaXlPRcOj+/gukr9teKbKH+2poC6QXptXJ9X6Wqij6nOIcZCTMoMta/SJQLJRes2w7Z8r/8U3u/c6mtLSAE/p0Idy7RPv/6NKTv84pc5wvPs+TQkhozanxS0fvp/Dzr2PBS7T03zXvCuCC/JJ//bPpPjV6zLNf3bs6hV8fTKCyAHceyGP32eka9tZ4LxUY+WpfKyj01+zcBOFeoDa2zCrNq/Nq+jCXCa/Wx1ZU+1iRNXPfLdXy15ytuXHxjtcqVWZjJgfPeXdbzWI6t4J/Dgy7b3N60l/NBsR1t27u+84pcn+z6hKc3PM2ujF1eOX9ZfFLRB+gDrNuDmw1239HfPET7/Q0vS+RIsbHYum2ZT6gNdDrB6xO6ObTdP7QNep3ggbmJTPhoEyNnrCMlvWYWONl8ajOA69XBFBfN/xK0pMDsouxKH/v8pudJz9dGWIezD1erXHcsv4MbFt9Qrecsi/13dvhdtRkBTXrYdIA9/g3gHvMof9O74K6cShU4naetdmcxbryNTyr6sIDaU56eYMI2hAzUB9aiJDC8Y0OOvH4Fqx8dyu4Xx/DUuI6sfnQoLSKDSDyWRWrGBUa99Ttfbarem7w8ik3FFXdSeExO8cVPKiamJ3rU71DWIbrO7srOdM+XuDyScwTAfQHCKlJQWkChsdA6Ieug6IvyICjC/cHNe9u2d3xT7bIFm+cBEtISqv3crvBJRR9sCCYuLA7AfXGzskwLv+jrZRdlV2oIav/D1uv0F33d6qRtwxBCArQhfuuYBvz22DA+ndSbR0ZpSwlMW7yXXi+vqpFFToqNxV67+RWVY0CTAQ6f7Uej9mxP3w7AgoMLXO4vD7eTpFXEYs2Pa63NxZXYh0ue2QOGoPJPENVGez9dRfdKcT582A8Ob7A2WRT9N/uq/yHiCp9U9EIIFl+3mF4Ne1XuR1Rwcb7h25fdzg2Lb/DY5WBvYXn8IKphdDrBmM6NeXhUO1Y/OhSA8xeK6fifFV6L0mkU3AjQwgG7z+nOxpMbvXIdb7EzfSfbz2yvbTHc0quhC390BWjLUdgoNBa67Bds0BTXwpSFlb6Gt6KBsoq0+zk2SFvYaMkh8yRrcT6UXIDMCkap/06EJt0hYZY2ArhYMg9Dxn5Y8oi1KSKgnNGEF/BJRW9Br9NXrOg7XmnbPnlxN6llCHo0u+IwzRJjCdf8fA1gXgnLXZXNOkTbhiHsfnEMl8RoQ+B2zy73SrRA2f+VxTpMSEuoF377ScsnceeKO2tbDCfaRrQFcCr0l5Gfwdj5Y0nOSPb4XCaT64e8JRv9YvCWsZNbrOWJ+Ou1KDxrXo3lPm/vwXKiFmv+u1u0OjgXg+VeKcqF3DRKjmzk8+TPARgXVzNJmz5Z68aCXugdh2uuGP8mXDIMlj2uFTZiZKWu8edp2zrpnlTM/HDnh9btBn4NvDZsrW5CAgyseWworZ/WUskt7xau6NqEQW1jSDqRxQ29m3NJbAhRDfxdncotZS27/JJ81p9Yz0NrHgIgeXIyecV55BTn+HY9nKJcLYY7/m6trHYVyC/JJyUrRdsutS4Ex+6zu7l16a0A/PjXj3SN7eryeFFmKQp3v1edsNmMUkqnkUB5eOseeH3r6wB0j+1ubSs1lWKYbTbuzOG85dL9Vi3y5sgG2DqT/N53EKwPBFMpRp2eElMJgYYK5tnWv6m956XBjA58GBkOEZqruKYMPd+26IUHFn1YU7j0Wm07fW+lr/H+jvet254o+lm7Z1m3C0sLyS7KJqswyyHet64ihGDn86MJMNh+NqFmv/7S5NM8szCZ77cd54ZPttDr5VVMW7THo/N2nd2VycsnOyn6Tac28XPKzw5tD6x+oM7Xw7mY6BYHVr8ISx+DlDVVlmXyisnW7aM5R9l/fj/rT6y3KnkofzLUXoGD9uBwhb1F/8T6Jyol4/f7v/e4b2Gpa9eRKyy5KvY64KOdH9k6DHiw4pNc+7F18+Ca5+j3bT+6ft2d1DeaMHX1A1z2w2WO/Vc9D5lHtO0lj8CGGbDX8TfcpsRmfNaU69aTNWNbCCHWCiH2CSH2CCEeNrdHCSFWCSEOmt8jze1CCPGeECJFCJEkhKi8Y7Ca8Mh1A9AgBvT+WlhVGc5cOMP0bdPd+hHtf3j2FpMryvqcLQkoQ34YQv9v+1csZx0gItifA/8dx5HXr+Dwa+NJfnEMW58ZyXf39ufKbk1oFR1MiyhtkuurzUd4a+UBikvdK5KMfK1cc2J6InkleVzf7nqH/auOrrJuJ2ckszNDi+qosjL1IoO/H1y1G9iiKKohn2D/ecfktxsX32gdIVko7x7JK3H0Ta86tspNTxsrjqzgj9N/lNvnZN5J6/YHOz+o8JwAB84foM/cPqw5WrkHYOuw1tbtRfaGg4v73Qkh+HHM03wcEcaE5raV265t3pR1p7dQUFqg/a9PJsKpnVo45rvdNSWf8AWsecnplNJulGTM2AfF3jfyPLHoS4HHpJSdgP7AQ0KIS4GngDVSynbAGvNngHFoC4K3A6YAHzufsmbQC71nN5wQENIIdi90/KOn/sarKx/k671fu/3htgi1lf1Nu+A+yUhKyQOrH7B+/lfPf9E8pHnFstVhLMPzhmGBDGgTzQe39eL3/xvOhidGsPTfWv7Ce7+lcM2Hm9iflsOhjDynRVDKVvEsb5LqtmW3WbdP5J2orq/hFf5M+7PiTu5IMSvTFU+V388DLEW9ysNkMmGSJvKKnSccF6UuAuDrcV8DtonNspR1QUzbPM3t9QpKCxg7f6xDW3n3joVv938LwNR1ZcoHZx6BeXdB1nE4kcA765+j62ybKyomKMa6faYgg2yd4L2uo9yOTuwpNhbz8l9z+SjS/e8y85UY+Gw4zBxqa0z4ghLgjagIsnQ6uHetdZfF7IkyGjGePwQbysneryYqVPRSytNSykTzdi6wD2gGXAPMNnebDZj9H1wDzJEafwARQohaWcR07fG1HMj0MOwx+zjknIBXbb7fyavv47ccbfj32ZJ/aMPpMrQOt1kLG05ucNpvwVKZ0UJkYCT/GeCYHVsfU8zd0blpOGseG8qoTg3ZdzqHse9sYMSM3+nw3AraP7fcmnlb1m2g1+nZdUfF4Wz2lubGkxt57c/XqvcLVJHNJzc7fJ6VPIs9Zz1zZVm5kKFVUExLvqiFMM4WnEXngS1XKkv5NOlTBnw3wG1WcscoLVv0VN4pTNJE19ldeWOrLdGwrEFlb7GXlenHAz86tT/2u/O9Bdoo7rpfriMjP8MxdPPUDijI1CZI3+0Ou+fDO13g85HMOvyLwzl022Y5fP4gMoLP8v7yqMjb7yd+r7DPWb3riej5oSF8Ex7GvXFtoVkvuOVbQFgV/Xm9no3BQXBorcvjq5NK+eiFEHFAT+BPoJGU8jRoDwOgoblbM+C43WEnzG21xv+2ebBcYKtBDh9LFk8lMdA2yZIYGAjbPodPL4NCm9ugyFhEkDked/2J9U6+45TMFD5P/tzJAh3WfJg1Nd1CZZJN6gNtYkP4fHIf3rm5B6DV2QkNMFBcamLK19sZMv03vtjheFPmFOWgEzprfRZ3WDILQfPbf7v/2zoVez9772xrWWApJe8kvsMtS2+p+MCy+Rwvx8Ang+HVyt9Cw38cbh1ZPN33abf9Vh1dxcojKwGtEKDlb7stbRugjYwDDYFEBUZx+sJpTl/Q9tvHgHs6oXrfqvusmbr2OFSWtOPdxHdJyUphy+ktjInTomTaR7SDmcPI+187+s7tw8Ygx8nQoDLRMWL5/zHvhO338n2YNgk7fdv0CufVLCOa8jjo7wehzrZsSu+JAATHdtIaOl4B07IwDn/GoV/msMrNaVwMHit6IUQIMB+YKqUsL9XO1XS708yZEGKKECJBCJGQkZHhqRgXxey9syuevLMvbrTs/zi/c47D7htyzCV9T++Ct23DwiJjkUPJha/3akPcEmMJXWd35bpF1/Fu4rsO57q1463EBsfSvWF3h3ZLOFh1k12UTamplHtX3ssHOz7AJE1sPrXZ4W+y8OBCus/pXs5ZLp5rezbjyOtXkPqq5tP/4+mRdGwcyonck8w9MNuh79LD2v+haYPyo2qe32xLzbew6aR369iXmkrLjeIq+3CasGgCUkoHt0a5Loo8u/ugvaNrA1MJ7PzWIzkPnD/g4LoAaBZS/oMiKjAKgPd2vMf1i7V5krt+vQuwKfHIgEjmH5zPmQtnrMf1+aYPe87tsc5hVZTp7a6Yn7tgBMuD6tmNz/LrkV+1vgXneCk6kmXBARSYinmgcUMO+9n+9l2LnJO6OpS4/r/NSp7lst3C5a0ud2q7vpHjfNozsTE82ns8vJAF07KhWTxc8xHNYjQF3yXG8X9RHKwt9znsguY6OhvTplwZqgOPFL0Qwg9Nyc+VUlrGT2csLhnzu+WuOwHYr1fXHHAqMC6lnCmljJdSxsfGuvb7VScVWnv24WBbZ3JO7/incXh62Z3rhwM/kFWUxX3d7gMgNSsFTCbOJzqWR7ZPkurZUFu2LEAfwGO9bUPWR9Y9Uqnomx3pO2xJIG4oMZYw+PvB9Py6J3+c/oNPkz5l0rJJ3LfqPqvPEzTFaZIm9pwr370gpWT6tumkZKZ4LGdZGocHsmLqZYS0nY7O4PhwO5c6ies/3kykX8Xhk89ufJY1+2xK58E1D3plQZXMwkySM5K5cfGN9J3b120/+9rnFrrN6cb+c7YJUbcuppPb4X9tbZ+v+xQGlfFF//yAFnppxz0r7+HjnY7TYK7qxzQMbujUZo/9ohzuDA7LOsf2kTyFxkJmJMywPgzcJVS5YvG1i63bluQmgI93fswve+a6jTo6WXSen8JCeTkmytp2RytNWZ7V6dga5Pphc09WNnopmdjRFnGUU5xDQWmB2wewpeS5PaExHZ3aVh1dRYnlgX7vGug50epeLBu5ZAnJvvbKmYD7h1x14knUjQBmAfuklPazBosAy398MvCLXfsd5uib/kC2xcVT03x2uW0xAaM0su74Oq5aeJX7TLzHbSvwnDY4Wmf5DWIoeOKwNrFSnAtfXQnZNnfM3c1HA7AgZSG8FIl++ZMOx1vCuiZ2mmhNyQaY3HkyS66zKWv76Ju0C2nl1im5Y/kdPL3B/ZAccF5VB0g6mwTAvL/mOT0Ab1licy9MXj7ZyTLMKMjg671fc92i68q9rjuO5xxn9VH3VRSL8mPYfjSTzdsGuO1jIfn0ae75dqVD26Dpv/Lasn2kZlxcJqOU0sFiBZi4bCK3LbuNlKwUt3HP2UXZbiOB7CeRfzvuoiT2noXw2QjHtqAIGP0i/CsRHrSb2H2tubZoRl46K/fP48/Tf/LRLu23dTrvtNXqtadzdGc6RXfim/HfMDZurNN+gEPZh1y2A/Rr0g9wXzNnW9o2lwltFRlXTUJcTN1JyUe7PuK5hNfhmwn0LvDswTGyqeZ6faJhjEN75+jO8OQRmJZNgxHPYxSCufttFSm/2/8dfef2ZfS80S5dp2VHcO8Mf8etYk7L0x4WSRlJjF8w3uoOyy3Opfuc7jy2TjPqtqZtBSA6UFsIqCayqT2x6AcBk4ARQoid5td44HVgtBDiIDDa/BlgGXAISAE+AzwIVvUO/Zv05589tLrTRmnkX7/9iyM5R5iZNNP1ASEN4flMALJ1jn+apfoi+v40lCGtmpOm11NyZAOlb3e27ve3X2gYkG7yRa685EqHz0IIWoW1ctl39LzRXL3warffzxPuWHGH230pWSl8sEMLbWsY5GzxWQpa5a150dpW1jqpDGuPrWX8wvE8su4Rjuccd9nn4MvX8+09/RgS14Xcfa+77GMhtySLkLbTHdrS88/y6fpDjJzxOwNeW8OkWX+yau8Zj9fK/emvnxg1b5SDi+F4rqOsXWd3dXIbPbLOlt7+5ZhyFrvBhQW39HHHz/fbuaCi22gve+ZczTcze/HYn7b/S5GxiMvnX87jv5c5FzCh3QRASxx6c+ibLmUqmxhl8c8DtAxt6e6rWFl4UCt9MGPoDGtbRSGwAfoAa9SaQJCen86viR859PE03zYopDHc8i2tmtsMhKjAKD4c+SEEaa6SimLwy45mpZROE7YjW460jt7LUmDU/P0zEmZwPPe4NWN+/sH5mKSJlUc1xW/5/1sSrd5JfMeTr1glPIm62SilFFLKblLKHubXMinlOSnlSCllO/P7eXN/KaV8SErZRkrZVUpZM+XZ3GCJ0bVfL/PjXdpQd1fGLmffvU4H07LZFH8r7hjdshnPxkZzc9PG1jb9etsNtDo4iJ0terg8tktMl3LlLTvEdlfG9GiOZ6tiVRS29lmyNuoZ0FS7QSx+ZntrrHjTO04uA62T57HiXWd35d9r/239fDjHsc7I2pvWsuqGVfjpdQxsG8Psu/qy4YnhPNntU65p+IrLc+r8bIrE4oOeO6Urz13RiVGdGhLkp2fDwbPcOyeB7i+tZPRbv5OWXUhekftsxJf/eBnQRh7ku188uux8gEWpRQVGEd843u1xAD8d+MmxId886up5O/znHDQu8xvR+2m+XzveiI50+Bz/jetrtg5vzWXNHZN6vhzzJTNHz+SLMV9wU/ubAMcw164xXa3+eYBn+z0LwNaJW11eo1NUJ2vE2OVxNp922bkpV9xx6R3W64/8aSSP7/7EtvP2+ZSWMZguj3X9PfNK8qDjFYQ2tt13v9/8O9FB0dbPkYGOf7OyRoslk9bC+zvet1rbDYMbEuqnTeI2atDIYV7OgiVcs2zFT8txYLtvGwY1tBZenHTpJJffqTrx6cxYgPu7awsA//SX4821+uhqbl92O93mdOPFLS86HWd5+rpjeUgD/gqwS/E/bRv2PdIolkeFs8vk0uhLK5Q3yBDEqqOr2HJqi9s+BzMPcuVC28hg6tqpbvt6imXoXWoqpdRU6jAxW6gTmsvg+DYKTtgsPV6KgtNJFBuLXbqILLhylVlGEgAzR88kJiiGxg0aO/RpERXM7T0H8t9xFY9qLPHdUneBe4ZcwueT+7DmsaG8dVN3bu2rWY0H0/Po/9oaurzwK++vOVjuBP3Bja/D9NZuq5qeyT+D0WRkR/oOVh9dbR0BLL5O8zu/O/xdpwVwpmRqynrGdpvVy8vm+amY9nDV+6AvJ9rohi+gx0T3+8uQPDmZRdcucvq7xjeOZ0DTAfRp3Idnu9zrfNxZW+2bJ/s8aa2wGuSm2uO+8/s4lH2IYS2GATbFVbZ+vat69recP8sd2a5dQsZLhlPcsLNDW2ZWKo+cz3Tq+3PKz5QYS9h7XstunznaedTet7Hj/EoDQ/kJUxYjCGDNjWvYfJstZPa3m35j3U3rHPonZSQ5uf0AcktsRpLlvr087nICDYEkT07miT51KOqmvjKhrTZs/TnlZ7rF2BbZsCxyAZqvuioLUlvCue7PLH+o6knFuqM5R3l03aNMWTXF5f5TeaeYsGiCQ9uaY+4zBYc0GwLAj1f+yM0dbnbbz97HWjbDN9NSSnnWKFIX/MPanhTgz82Lb6D3N70Z/uNw3tz2psswOVer6BzL1Vb4+eWaX6yjCSdy01yu/tUitAWROkeLypIvkZ11FEq0IboQggm9mvPahG5sf24UU0e1o3NTba2CGav+otu0lTw4dzuzNh7msR93EffUUoJ0mtX3YWka+eXUa8kvyafH1z24Y/kdDm6bkuIAOPYHw+dOJjE1le6FmqXbu6CQUfm2v+ucPXO0MF1L2d8r39FGk+XR5Xq49iOHELYHM13HvT/Z50mX7Q4YS9HN6FBul9svvd3tvqm9HA0Mi5X7SG/t71HWFeJybmbls5xzE4eeU5xDSZkH37aSTO7MzmV6z0ed+r+1/S3rRKer31TZRX7sFbCFElMJX+3+ysHt9Mpg5xFlmH8Y0UHRVtcwwJsJbzJq3iiX36Us5Rly3sDnFX2TkCZ0iupE5+jORAXZZunLWvj2flj7jLmHez3M+yPepzwWnNTmmh8a8Gy5/YrLSXr5V89/ud13vtDmQpi+bbrbfqBN/NhHxOh1ejpEdqBTdCfrELwsO9N3OpQaGPSdY07BLc0aY3kMZNlFI01s2pi9ATaFO2fvHH444LygcohfiFPbhZIL6IXeIbPYiRkdtFcZqzrEL4TgIkc/tyUeOmvF/8ErjZxOFR0SwNRR7Vn67yEc+O9YBraJJirEn2XJaby8ZC/zE4+jb7CfApPNWnykYQzulkD5co+zHz6wJJC9b4yAL8YgijQrdb+/ZtVvDwrE324EsWDbD/C65vuWQx6HOO1vvvN4FkuSTpF0Istt7f9Vt2uhv/1je9KpyPXcw6hWLhSOsVRLvMo6po0k3nKOHokVtlHqPV3vcdr/01U/8VTfp0ienMzdXe9mdKvR1n0WRW8ZyVjcOYtTFzPqp1G8t+M9a9/kybZRw9IQ15b1jIQZTgmP/8jKQQeMKzPXBba4fncjZ0sIKcA3413XgR/6/VBmbJ/Bl7tt/9+r27gfUd7X/T6S7khyu98dZV2X3sanq1dayCnO0TL1zmn+MldP8u1ntlujCyw/mCYNmnBP13schvg/XvkjL2550cFaaX7F+1qtnG43MjAn0WG0kHzFQrou1SJUEtLdz65P6TaFvo37Mmm5s79u6A9DmTt+Lt1iu5GalWpt33TrJqtSzi7KZtPJTfxw4AcS0xPZdccusouyWXd8nbW/EIK54+eiEzoiAyOtaeiurlmWd0f+m8cLBdOPl59N+PGuj3nQFArFedBfK/ngLsqpRVBDl+FrSKllPbogKiCSjyL7c+cZx5tremh3nsjdRb7FKj6XapvELM6H/HMQoT1UAgx6vr23P0iJadG/OdT1Ea7bcI3TtTYHB5Hi7yhfVH4k54Ndy9ZY5jJE7xhg9v6ZDKY0aUQDo2RxwVWA9htIFceYHhXBA5nZDDzxI8VvH8OYcR2lJttvLTTAwOWdG9OmYQNS0vM4mZVLquENivSan7dX00GYdju7GBNu+h2dfwRGk0QvgMQ5sPjfTv244Jy/8nTaSR5tpLmTyk44g5Yha8mSBbi27bVWI8F+UZJxrcex/PBytpzawjMbHROE3hlmN/kY1Ya3AlryaFEqZfkl1ZZM93Cvh/mHX1N035vnzhrEMLHTRNYdX8eCqxfQ79t+1r7xjVz78e1/a/YVLUFz/WUUZFh1Q2p2Kp2iOhEeUPGCREIIpvaa6nJStVNUJ/add15g/KORHzm1eROft+jBMR3blZIHTUFZJko6R2t+wef6Pwdo/8jE2xNZdcMqOkV3cj6450Topi2c/OnoT3mqr1ajZNOtmyC6Dd8FaMe8lF1+Grtl4Q1XTFym+WYtVtI3478hzD+MG9prMdPjF4znyQ1PWieCMgszWX54udN5usV2o0tMF5o1aMqnfZ8vV57PT9v8jevP7YLRL5Gr9+Ans+ifDnVaLG6h12UMS4+f4mZz8lnLs4e09P5pEZrVvuQRSPxaS2mf3trhlF+fSiPCaGR1o7HErH6Ro362m3bEhXzGJi3GICUXdGZ3y/u9NNdIWjJ8PkpLj58WDp8MgeR5sOz/4MUIdDvm0HZOT7df5eZmWgjgfZnZ+EnJzMy9BOtdW6Aldq4e0w1fwQtZDHgqnXeHfcgdbT+naLBj7Zqvw8MYGKc9fPyj/qB9p/X858pL+fC2XrxyXRfaNAxh5d40pq84wILEk+zM/8qq5AHe+L4pgSWO7ohFJ04R8EZr2j+7lKEvLWTx67e7VvIuWBo+iHC7rFKrq6O0GDa+Axec52Eua36ZdVLTWu8drIXHXLkgR7S0CyUtOM/oBq6jzuy5p+s96DuOR0zL1iamheCpvk+xbMIyp7mDsuVG7IlvFG+NmhnYdKC1fdG1jhmw646vY9/5fU7zLO4oWzwOYMl1Sxzu6ds72dxgg5oNcurvTf4Wir5DpM0PaRAGZl3uOhvOMlFinwlowU/vZ53UsrfmvxjzhdN5JnaaSPLkZO1GEYIut/xIcpNruO78Ga3K3VdXuoxiidHbFiq+wi+Wq1s5Loyw9NBSa/p5V3O2ncVNUzbGOT0/nde2ask5U7q58Pf/Pp2BPzgOzZ8852ip9issYmrnuwFtIm3LqS2MixtXbrhdtH3RskX/gsIc3t7+NgAN0/bQsrSUQPMIKcRk0tL7LV7nhC+0h0SWXURRvwfgmdP0uPYLNhw7iZ95IfdehZofvl+TfrxijEAAQSbJBftIitdbaudPt/MVpyXB/Lthq22y7myAo+L+R+PBTpmkQ6/8hMQjx+lQXMK61P1cn+scp18cHAmjX4b4u9F1vtaahDei1WU8eFl/nhzr7Cqx57hpKV3bZHBFtyZM7NeKnx8aRPK0Mex9aQxz7uqLf6Rj1EuAwcCL/u/zerqmgDsUFdO6RBs9HQ68nY38g6uKXCfUvSTup0/hh8QVfktRxkiKzg7lsj+uoktRMa2LS+harMNweghv/rqfAwmrYfULFPxwF7tPZjslpf18jVYR0t41OP0y9y5Ga636swe1kVuDhky+dLL1uLJRKOUtaqITOqfa9/aRNmX5cuyX/LOn5ld/LP4xq/wh/iEuy26Urd7pjomdHCfJ3x3+Lq3CWrHuxDprW69GtVbI9+/hurGsMAOalR3sF8yjvR9lyaElTinZRpPRmiRhf5w9A5sOtLpn3PVxopE5ZO6z4dr7a3aVKwf8E0b8B7/lT5B0+Bg5Oh3hpmPkH0xkUZzNh/3UBptFqHsxEnQGcrsNcXm5Fza/YN126f9f96pTU/OSMi4WQyB3x0/l69SfOVd4zmqdtQl3jOse4hfDhhJN2eTr7G66xDmQOIfE1tqDIdKsIE6Yk9GWhTTgjQzX4aMYArVFYbrcAP7B0OkqiOkAZzWf7Ts3r2L56c3c2vFWxF2kuN4AABF6SURBVOUC1v+PkMNzye1+BfR/Hl7zsDaMfyiJesfvvcuU75TK3yC2I1w/C+bfTZCU/OfseeaHanMPg5sNZuPJjYy4ZDz0L9963njLRl7Y+Bxr7BSAPXevvJuowCiahzTnk9FaqGF6fjqXtW8DdvN3w1sM573JWuJd9sGviFv3MK9mnAP/EM1tVobkiTsxNIjkyN4EhvxxN+diBpNxRHsozrtlGmfzilixO42RSZ/z7fFnaK07AifH8U7pBObICF7xg6Bjv3Pr+yuZ6L+ep3RzmKJ/kb8Cu9M8MpgRIXNI2mfgwME93BTfglZB/ZxkcKAgEz4wu1guvZrHGnXhoZ4PEWQIYlzrcdZSIgDfX+l5vXrw3C3SPrK9w1xBwsQEenztGBYdoA9g25Hz+Ol1FJYYaRYRRLOIIHQ6x4dL2bDo4S2GO13PWyVOPOFvoegtQ8s54+ZYF+X9R5d/kF+a76ToF6QssE4euhu2vTTwJevsekU1Way0Guh+35YPtBdaqQXL8DlYSrYfPkbv1m4saFMpg0Nbk5rt7N905Re0YrQptt+PnmBoK+2hE9P5BuKKj3Ak9yjzej8Hk7UonbIr6KRmpzKh3QRrNcGPblvLuuPrWHhwIb8d/43SO5dhSP0NNjgWr2pTUgoPJ3FTwUnWrLJLOnk8BXR67cY3GbU67IZA6FUm2eufWzXfe0QrIvUGbouIs+277HFichPJLM6BgDKTv0Megz73upx85Mq3KVz5sENT89DmzBg2g+E/2m7WIH2QFvWScxI2vYs+/xzPGsN4RZ/DM/2eoai0yKNVr8IDwnln5Ps8uPpBa7XTbjHdrNnKoE2+ny88z8DvBhIREEFWUZbThN/a47aKh+HtxrC4zV5Y95r2N0tdA4sf1lZOG/cmRF1CV3P0Sqemo2DUUd4Fyka5j+jYCG7oTsLOtrT+WSu5O9XguNh3cqBtFDjT+ALvZE9g6oUFFEk/OhR9BQi+3HQEgFAXXk6AopebEmC0TaZvyY6iS0QpoYE2F0zfxn2tGaT2cwKusJ9DW3fjBoL9grlQVMr5C8W0iAou50iN4lITG1MyWLMvnaJDz0LYFgJitAzmLSd2seo35wgZIeC6Hs0Y26UxRpMkNSOPrmGXc0nAaMZ3iGdZchqZ+cXcH/ctXx27j0JTLlF+LfnfoFlk5sHp7AK2Hj7PkbP5XNW9CZfEOgcsVCd/C0V/5SVXsitjl1P996vbXM2PB37k8laX8/0BzWp4actLPNxLu/FD3Sw11jC4IVddchUjWo4gNtjDOj0RLaFBrDYBNvYNWOFB+Bvg1kMY2RoyD/PIhi+YbfcguCSsFYfskqkub2i2mr6fCPvNw/jxZgXc736ixr3BF6f+oESa6Nx0AJ/np5N8NpkOdlEbZUvOhgeEM6bVGBYcXMB1bbWJ5mEthlkn5Xr+fj+/XreMpnlnYMfXNC8poXtRMeKxAxDaiIGRrXh72Nua+6mB3bzEE4c0Rf/1tTDUTS32slmidoQFhJFtqSwq9CCNWqEpy9D+/1K1OkUhNuvri91f8HasNtT/49YtHMj8i45RHQn2CyZ5cjL3rbqPzac2E2AI0M4z6GEY+G8ozOKWoEg8qEfpkteGvMZdv97FM/2eIbMwk0fWPcLwFsMdFDjYasCUrSHz6ahPHU+o08EIs+uk953a6yKJ79EDzj2mrY5UAZYHQYAoYdutBkrjhrJwyz4iTvzGxqg72HBeixBqUnIbF3KiGFS4ggDjMevxK429mfKlLQeleWQQOiFoHTuFWxtPJT2nkAfnbkcnBI3DAokK8aewxMS5vCKy8ktIzy1k25FM60Ol338307NlBNuOaG7IPnGRDOvQkJyCEvKKSmkWGcTJzAI2p54jK7+YID89Z/OKKTaPNiODo2gdfCPNozuw9tzHtNCPplmnRgxtH0NooB/5xUYOpueyIPEky3ensWCH/b0xgs0Y+WZ92bUIngUkd+6zj7G3lcLYduQ839xTwQioioi6sCRbfHy8TEjwXgKtlJJiU7HLbDaArMIspm2Z5hSPnjgp0ePJmEoz90Zo0Q9K8rVaJ+ftao0MfhSGPQ06A5xM4I1FE/kmxGZV/+t8FlMeOQ7fXA8pq9nr78fNzZrw8Pks7snOYc6Et3hzhxYB8N3JNLoUuwkSnJqsPYAq4OHfHrbWaOkQ2YFZY2YRHhDO4ezDNA9tbv0bTVg0wZqBPG3ANK5vfz2YjFz+0yj6Rnflv6Pec3uN6uCJ9U+w++xulk1YRnrWYT7b8xVP9H/OGm3xc8rPNG7QmP5NtHpCUkq6zbHlViTdkeTk780pzmF3xm4GNitnRFYNFJQWEKgPdJDHnif6PMH0bdO5us3VLuO6vcbeXyCkMTTpDn6B8GpzrdaTK/pO0SbTzaGuRQLi47TfV3KLm21rp9oxd+wumkYGsznlLL/uOUNUA38ig/1IOpHNuQva71YnoEl4EBm5RVaFbCGqgT8CKG6pxdV3KpzJgbRchIDBbWPYeyqHQ2cdQ3H99IKuzcKJi27A8cx88oqMXH5pI+4a1JrwYNv9XvT/7Z1/cFTVFcc/J9ksESKSALEMGEMAmaLDQAR/VAQpKOqArYUqqAO2to5Wxzo6IIydlvKP1mmVccqM2ql17FjKL0uRjlJ/VTsdjBAxgCAQSlqQHxEEojSRBE7/eDeb3ewuu2FJ3u7L+cy82fvOu5vc78vL2bv3xzmnviacF06a/7bx5Cm27j+OKgwtLeJQQxNHT5zk6P+a2bzvGDde9g0GXHAeWz47TmPzKQ4dbyKUL+yq/4ovvvK03T72Iq4e0pfCgrNLri4i1ap65q3YdBNHny7z3p8Xs1IlevzOTzYc3BDZkv5E/WGmFF9KwY/c5pPPPoLfTaQuFKKspYU8oEmEsW5sf0Pd3sjkZwzjHoHJv4i3J0BVeXXXqyxcv5DZI2Yzd+zchPW2HdnG7Wu94Z5HL3+UW4bewhNVT/BG3RtMHzadhd9a2CHdHWXR+kWs2LmCqjuqIsvtFl+3mEkXT4r0zKHt73rwxEGuX9m2Djwb/t6tGpIxpXwKv56QRn6FzkIV/rseSiq8lVU718Fdq+APNyWsvqqoF2OavubilnbzP9HftJJwsuU0+XlCvhsPV1Uam0/x6cEvCefnUd6vF0UuZ3Ht0Vr2NOyJWdff+p4jJ05S1CPE8cZmWk4r/YrC9AidnWPNNtJ19N1i6CZdnhr/VMIliX4THYxp6h1/g9KoDSEDK2HCfMrfexLKr4W6f8Y49sKJP/OWE5ZUwHS3pfvEYShOvaStFRFh+iXTKS4sjvSGEzGi7wg2z97MFa9cweHGw9z/1v1sO+JtSU+VSORc0LqjN3pN9Z6GPez9cm/M3oamliYKQ4U8Xd35Kdw6yoIrFzCwaCDThkzjtJ5m0fpFMZnL5o5J/CHbZYi0zTd9/6U2+zUPw7/cOvJbX4DhN8HrjzG9JiqGfrgI5tZCfo+UTh4gHIpdFCgi9AyHqCwrjqs7tHgoQ4uHxtlFhH5F3jf5s+01BwHr0bdjU/0mZr/uTQJmQw8PvJVAt665lXlj5zFu4Lj4CqreVvqQG5pqavACZZVUdG1DHROWTWBS2SQ+OPBBZMfx5LLJPDPxmU79vY0tjWeMF5+Mh0Y/xMj+IyMb5rKJ5tPNVP6xbVleouGlrEE13oHvftebcxk2Be6MTyFoZIb16M+S0aWjeWvGWzHbpf0mPy8/bkNHDCJtTh6gsLd3+ETPUE8aTjbEhJUYVjys039vsqBbqfjxyPjAXtlCQV4BW+Zs4bH3H6Pm85rsdfKQuJc+ZGJc5E2j6zFHn4ALeyXfoWqkZt9X+2Jy5C6furxLHD3AkklLqDpQxcvbXo679tp3X2Pa6mkxtrsvvbtL2pUpT1575tj8hnEmzNEbnU7CsBGdxPhB4xlVOooNBzfE7SUov6CcjXdtJJwXZufRncx4bQaVpf7tVuwIWd2TN7KebhECwehaZkXl5Ky6o/2a4s6nd7g3y6ctZ/V3Vkfi1K+cthLwdjqKCMNLhlN9VzUTy+J3MBpG0EjZoxeRF4GpQL2qXuZsy4DWADJ9gGOqOkpEyoHtQGts0Q9U9b5z3Wgju3lw9IMU5BXwwKgHIjuR/WBInyG8c1uCHK2OtMNXGEaOk87QzUvAb4HIoKeqRjJYiMhvgOjZlt2qmjiPntEt6B3unXStvWEYXU9KR6+q77ueehziDRzeBnw70XXDMAzDfzIdo78WOKSqu6Jsg0Vkk4i8JyKJQysahmEYXUamq25mAUujzg8AZap6REQuB1aLyKWqGpf9V0TuBe4FKCtLHW/FMAzDODvOukcvIiHge0AkSaiqfq2qR1y5GtgNXJLo/ar6gqqOUdUx/funGQHSMAzD6DCZDN1MBj5V1cjOGBHpL+KlgxGRCmAY8O8k7zcMwzC6gJSOXkSW4uW2GS4i+0TkHndpJrHDNgDjgc0iUgOsBO5T1S/OZYMNwzCMjpHOqptZSex3J7CtAlZl3izDMAzjXGE7Yw3DMAJOVoQpFpHPgf+krJicfsDhc9ScXKC76QXT3F0wzR3jYlVNuZolKxx9pojIxnRiMgeF7qYXTHN3wTR3DjZ0YxiGEXDM0RuGYQScoDj6F/xuQBfT3fSCae4umOZOIBBj9IZhGEZygtKjNwzDMJKQ045eRG4UkR0iUisi8/1uTyaIyIsiUi8iW6NsJSLypojscq/Fzi4i8qzTvVlEKqPeM8fV3yUic/zQki4icpGIvCsi20XkExH5qbMHUreIFIrIhyJS4/T+0tkHi0iVa/syEQk7ew93Xuuul0f9rAXOvkNEpvijKH1EJN9FtV3rzgOtWUTqRGSLiHwsIhudzb/nWlVz8gDy8YKmVQBhoAYY4Xe7MtAzHqgEtkbZngLmu/J84FeufDPwOiDAVUCVs5fgxRYqAYpdudhvbWfQPACodOXzgZ3AiKDqdu0ucuUCoMrpWA7MdPbngPtd+SfAc648E1jmyiPc894DGOz+D/L91pdC+yPAn4C17jzQmoE6oF87m2/Pte83JIMbeTWwLup8AbDA73ZlqKm8naPfAQxw5QHADld+HpjVvh5e2Ojno+wx9bL9AP4KXN8ddAM9gY+AK/E2y4ScPfJcA+uAq1055OpJ+2c9ul42HsAg4G28BEVrnYaga07k6H17rnN56GYgsDfqfJ+zBYkLVfUAgHstdfZk2nP2nriv6KPxermB1e2GMD4G6oE38Xqmx1S1xVWJbntEl7t+HOhLDul1LAbmAafdeV+Cr1mBv4tItcu9AT4+15kmHvETSWDrLkuIkmnPyXsiIkV4wfAeVtUGkUQyvKoJbDmlW1VPAaNEpA/wF+Cbiaq515zXKyJTgXpVrRaR61rNCaoGRrPjGlXdLyKlwJsi8ukZ6na65lzu0e8DLoo6HwTs96ktncUhERkA4F7rnT2Z9py7JyJSgOfkX1HVV5058LpV9RjwD7wx2T7iJfKB2LZHdLnrFwBfkFt6rwFuEZE64M94wzeLCbZmVHW/e63H+0C/Ah+f61x29BuAYW72Pow3cbPG5zada9YArTPtc/DGsFvts91s/VXAcfdVcB1wg4gUuxn9G5wtKxGv6/57YLuqPh11KZC6xUvM08eVz8NL3rMdeBeY4aq119t6H2YA76g3WLsGmOlWqAzGS/DzYdeo6BiqukBVB6lqOd7/6DuqeicB1iwivUTk/NYy3vO4FT+fa78nLTKc8LgZb6XGbuBxv9uToZaleDl3m/E+ye/BG5t8G9jlXktcXQGWON1bgDFRP+eHQK07fuC3rhSax+F9Fd0MfOyOm4OqGxgJbHJ6twI/d/YKPKdVC6wAejh7oTuvddcron7W4+4+7ABu8ltbmvqvo23VTWA1O2017vik1Tf5+VzbzljDMIyAk8tDN4ZhGEYamKM3DMMIOOboDcMwAo45esMwjIBjjt4wDCPgmKM3DMMIOOboDcMwAo45esMwjIDzf/dYkNp+kpZzAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "wopt = np.linalg.solve( X0.transpose().dot(X0), X0.transpose().dot(y0) ) # least square solution\n", "niter = 5000\n", "Jlist = np.zeros((niter,1))\n", "tau_mult = [.05, .3, .8];\n", "for itau in np.arange(0,len(tau_mult)):\n", " tauk = tau_mult[itau]/np.linalg.norm(X0,2)**2\n", " w = np.zeros((p,1))\n", " for i in np.arange(0,niter):\n", " ik = int( np.floor(np.random.rand()*n0) )\n", " gk = (X0[ik,:].dot(w)-y0[ik]) * X0[ik,:].transpose() # stochastic gradient\n", " # gk = X0.transpose().dot( X0.dot(w)-y0 ) # batch gradient\n", " w = w - tauk * gk\n", " Jlist[i] = J(w)\n", " plt.plot(Jlist/J(wopt)-1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Average on different runs to see the impact of the step size." ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd8HNW1wPHf2V2tui3ZkrtBbphiTDMdQk/oEEiBBEIS8ngkEEpIQksCJJDQAoRQElrgESAQMCWU0AwhphhsY+PeC7ZlS5Zt9bLlvj/urLWSVtJK2tXurM7389FnZmdnZ46k2TN3ztyZEWMMSiml3M+T6gCUUkolhiZ0pZTKEJrQlVIqQ2hCV0qpDKEJXSmlMoQmdKWUyhCa0JVSKkNoQldKqQyhCV0ppTKErz9XVlJSYsrKyvpzlUop5Xpz5szZaowp7W6+fk3oZWVlzJ49uz9XqZRSrici6+KZT0suSimVITShK6VUhtCErpRSGUITulJKZQhN6EoplSE0oSulVIbQhK6UUhnCFQl9xtItPPD+ylSHoZRSac0VCf3DlVXcN2Ml+vxTpZTqnCsS+pjiXBpaQlTVt6Q6FKWUSluuSOhlQ/MBWFdVn+JIlFIqfbkjoZfYhL5ma0OKI1FKqfTlioQ+pjgXr0dYu1Vb6Eop1RlXJPSsxdO5Of9Z1mjJRSmlOuWKhM76jzk1NENb6Eop1QV3JHTx4hPD2q312nVRKaU64Y6E7vHiJUx9S4jKuuZUR6OUUmnJHQldPHgJA7C6UssuSikVizsSuseLx0noWkdXSqnY3JHQxQsmhM8jrNumfdGVUioWdyR0jxcJhxhTnMv6Kk3oSikVizsSungBwy5D8li3TUsuSikVizsSuscLwLihOayratCui0opFUO3CV1ExorIeyKyREQWicjl7d7/uYgYESlJWpRiw9y1OIfapiA7GgJJW5VSSrlVPC30IHCVMWYP4BDgEhHZE2yyB04A1icvRHa20HctzgHQE6NKKRVDtwndGFNujJnrjNcCS4DRztt3A78EklsDEZvQy4ZmA7C6si6pq1NKKTfqUQ1dRMqA/YBZInI6sNEYMz8JcbUVaaEXZeP3eVi2uTbpq1RKKbfxxTujiBQALwBXYMsw1wNfjeNzFwEXAeyyyy69i9JpofvEML4kn5UV2kJXSqn24mqhi0gWNpk/ZYyZDkwAxgHzRWQtMAaYKyIj2n/WGPOQMWaaMWZaaWlpL6P0OgsLM2FYASu15KKUUh3E08tFgEeBJcaYuwCMMQuMMcOMMWXGmDJgA7C/MWZzUqJ0erkQDjGxtIAvtzXQFAglZVVKKeVW8bTQDwfOB44VkXnOz8lJjqutSEI3ISYOKyBsYI3e00UppdrotoZujJkJSDfzlCUqoJgiJZdwiInDBgOwsqKOPUYOSupqlVLKTdxxpai01tDHleQjgp4YVUqpdtyR0HeeFA2Rk+VldFGullyUUqoddyT0SAs9bO+JPr60QBO6Ukq1446EHtVCBxhfks+qyjqCoXAKg1JKqfTiroQeDgIwZfRgGlpCrNd7uiil1E7uSOjS2ssFYExxLgAbdzSmKiKllEo77kjo7UouOxP6dk3oSikV4Y6E3u6k6IhBOfi9Hj0xqpRSUdyR0D2tV4oC+Lwexpfms0L7oiul1E7uSOjtaugAuw0vZGl5TYoCUkqp9OOOhN6ulwvAnqMGsam6iWp9HJ1SSgFuSejS9qQowOQRhQAs26IPu1BKKXBLQvc49xCLKrlMHq4JXSmlorkkobfenCti5OAcBuX4WLZZ6+hKKQVuSehRD7jYOUmEySMK9fmiSinlcEdC93SsoYPT02VzLcaYFASllFLpxR0JXTr2cgGYNKyA2qYglXXNKQhKKaXSizsSuqdjP3SAcaUFAKyp1CtGlVLKHQldOp4UBdjd6bo4Z/32/o5IKaXSjjsSeict9OGDchhTnMvScj0xqpRS7kjo0vZeLtF2G17Icu2LrpRSLknoMS4sipg8opBVlXUE9OlFSqkBziUJPXa3RbBXjAZCRm+lq5Qa8LpN6CIyVkTeE5ElIrJIRC53pt8hIktF5AsReVFEipIWZSfdFqH1ni5L9QIjpdQAF08LPQhcZYzZAzgEuERE9gTeBqYYY6YCy4FrkxdlpIXe8QKi8aX5eD2itwBQSg143SZ0Y0y5MWauM14LLAFGG2PeMsZEmsyfAGOSFuXOk6Id6+TZPi97jCxkzjrtuqiUGth6VEMXkTJgP2BWu7d+CLzRyWcuEpHZIjK7srKyNzGCiB3GSOgAU0YNZpneAkApNcDFndBFpAB4AbjCGFMTNf16bFnmqVifM8Y8ZIyZZoyZVlpa2rsou2ihg62jb28I6C0AlFIDWlwJXUSysMn8KWPM9KjpFwCnAt81yWwex5HQAb3ASCk1oMXTy0WAR4Elxpi7oqafCFwNnG6MaUheiHSb0CdG7umiXReVUgOYL455DgfOBxaIyDxn2nXAvUA28LbN+XxijLk4KVHGuB96tNLCbPL8Xk3oSqkBrduEboyZCUiMt15PfDid6OTmXDvfdh52sbhcuy4qpQYud1wpurPk0nmZfq9Rg1i8qYZwWHu6KKUGJpcl9M7v17LPmCLqmoOs1rKLUmqAcklC77ofOsBeowYDsGhTdX9EpJRSacdFCV26TOgThxXg93lYsEETulJqYHJHQgdbdukioft9HvYdW8SHq6r6MSillEofGZPQAQ6bMJQl5TXUN3e8K6NSSmU6lyX02P3QIyYPt1eMrqqs64+IlFIqrbgsoXfdQp8y2p4Y/ULr6EqpAcg9Cd3j7bIfOsCY4lwG5fj0AiOl1IDknoQeRwtdRNh9pL3ASCmlBhoXJfSuuy1GHLBrMQs3VtPQoidGlVIDi4sSevctdICDyoYQDButoyulBpyMS+h7jhoEwFKtoyulBpiMS+jDCrMpysti6WZ92IVSamDJuIQuIhywSzGz1mzrh6CUUip9uCuhd/KAi/YOn1jCmq31bNie3AcpKaVUOnFRQu++H3rEEZNKAPhw5dZkRqSUUmnFRQk9vpILwKRhBZQWZjNzpd6oSyk1cLgoocfXD93OKhwxsYSPVm7VJxgppQYMFyX0+FvoYC8wqqpvobymKYlBKaVU+sjYhD6htACA1XrnRaXUAJHBCT0fgFUVmtCVUgNDxib00sJsSguzmbt+RxKDUkqp9NFtQheRsSLynogsEZFFInK5M32IiLwtIiucYXFSI43jARdtZndOjM5cuRUTZ3dHpZRys3ha6EHgKmPMHsAhwCUisidwDfCuMWYS8K7zOnk8XgjH30IHmFZWzLb6FjZsb0xSUEoplT66TejGmHJjzFxnvBZYAowGzgCecGZ7AjgzWUECzgMu4m+hA0wdXQTA519q2UUplfl6VEMXkTJgP2AWMNwYUw426QPDOvnMRSIyW0RmV1ZW9j5S8UK4Z/c432NkIfl+L5/pfV2UUgNA3AldRAqAF4ArjDFx35vWGPOQMWaaMWZaaWlpb2K0PN647+US4fN6OKBsCJ9qQldKDQBxJXQRycIm86eMMdOdyVtEZKTz/kigIjkhOjy+HrfQAQ4eN4RlW2rZXt+ShKCUUip9xNPLRYBHgSXGmLui3noFuMAZvwB4OfHhRQfi7VG3xYiDxg0B4LO12kpXSmW2eFrohwPnA8eKyDzn52TgVuAEEVkBnOC8Th5Pz2voAFPHDMbv8+j90ZVSGc/X3QzGmJmAdPL2cYkNpwu9qKEDZPu87De2SFvoSqmM554rRXtZQwd7o67Fm2pobOn5DkEppdzCPQldet4PPeLAcUMIho220pVSGc09Cb2XJReAg8qGkOUVfYKRUiqjDYiEnp/tY7+xxXy8Wp9gpJTKXC5K6L2voQMcOmEoCzdWU90QSGBQSimVPtyT0PtQQwc4bMJQwgY+WNGH2w8opVQac09C9/h6fLfFaAfsWkxhto+ZK7SOrpTKTC5K6J4+lVx8Xg8HjRuiPV2UUhnLRQm9bzV0gEPGD2X11no2V+uDo5VSmcdFCT0Lwn07oXnohKEAfLRKyy5KqczjnoTuzYJQ31roe44cREmBn3eXJPfGkEoplQruSegeX59b6B6PcNzuw/lgeSWhsD5nVCmVWdyT0L1ZEOp7H/KDxg2htjnI6sq6BASllFLpwz0J3ZNl+6GbvrWs9xk7GIC567cnIiqllEob7knoXudOv31spU8oLaCkwM+HK/U2AEqpzOKehO7JssM+1tFFbB19xtIKWoK9v1BJKaXSjXsSutdJ6Amoox8xqYS65iDzvtzR52UppVS6cE9C39lC71vXRYAjJpaQ7/fywpwNfV6WUkqlC/ck9ATV0AGK8/0cNbmUGcsqtPuiUipjuCehJ6iGHnHSlJFU1jbrVaNKqYzhnoSewBo6wAl7Dqcg28frC8oTsjyllEo19yR0j1NySUANHSAny8vRk0t5a9EWAiHt7aKUcr9uE7qIPCYiFSKyMGraviLyiYjME5HZInJQcsMEvH47DLUkbJFn7DuaqvoW/qsPvVBKZYB4WuiPAye2m3Y7cJMxZl/gN87r5EpwyQXgqN1KKc7LYvrcjQlbplJKpUq3Cd0Y8wHQ/qkQBhjkjA8GNiU4ro4S2G0xwu/zcNo+o3h78RZ2NCSu5a+UUqnQ2xr6FcAdIvIlcCdwbeJC6kQSWugAZ+43muZgmP/qo+mUUi7X24T+Y+BKY8xY4Erg0c5mFJGLnDr77MrKPtSqdyb0xLakp44eTL7fy8er9d4uSil3621CvwCY7oz/E+j0pKgx5iFjzDRjzLTS0tJero6klFzAPmv0yEmlvLGgXC8yUkq5Wm8T+ibgKGf8WGBFYsLpQpJKLgCn7zuK7Q0BZq7UsotSyr183c0gIs8ARwMlIrIBuAH4H+BPIuIDmoCLkhkkkLSSC8Cxuw+jMNvHq/M3cdRufTiKUEqpFOo2oRtjzu3krQMSHEvXklRyAXuR0Ql7DufNRZu55et74/e553orpZSKcE/mSuDNuWI5dZ+R1DQF9SIjpZRruSihJ/5K0WhHTCwl3+/lnSUVSVm+Ukolm3sSeoLvttie3+fh6N2H8daizQT13i5KKRdyT0LfeVI08TX0iJOnjKSqvoU56/QB0kop93FhQk/eJfpH7laCzyPMWKZlF6WU+7gnoSe55AIwKCeLg8cP4bUvyjFGLzJSSrmLexJ6P5RcAE7ZexQbtjeybEttUtejlFKJ5p6E7vGCeJLaQgc4fs9heARena9PMlJKuYt7EjrYsksSa+gAwwpzOHxiCa/M36RlF6WUq7groXuzkl5yATh9n1Gs39bAp2va3wZeKaXSl/sSepJLLgBfmzICgEuf+Tzp61JKqURxV0Lvh5IL2N4uB+xaTGVtMxu2NyR9fUoplQjuSuhef9Lu5dLevefuh88jPPzB6n5Zn1JK9ZW7ErrP3y8tdIDRRbmcOnUkT3y8jiXlNf2yTqWU6gt3JXSvH4JN/ba6q0/aHYAfPTG739aplFK95a6E7suBYP+00AFGDs7lnAPHsnFHI5+v1/u7KKXSmwsTev+10AF+deqeANzx5rJ+Xa9SSvWUyxJ6NgSb+3WVBdk+Tpoygo9WVdEUCPXrupVSqidcltD7v4UOcMa+owH4YLk+zUgplb5cltCzU5LQj9tjGMV5Wdz9zgq9HYBSKm25LKGnpoWe5fVw6bGTWFJew6Mz1/T7+pVSKh4uS+j9X0OPuODQXQG4+bUltAT1EXVKqfTjroSelQvNqblPuc/r4e5v7wPA24u3pCQGpZTqSrcJXUQeE5EKEVnYbvpPRWSZiCwSkduTF2KU7EJoqYcU1bFP32c0wwdl88LcDSlZv1JKdSWeFvrjwInRE0TkGOAMYKoxZi/gzsSHFoMvFzAQTk33Qa9H+Pa0sby3rIJVlXUpiUEppTrTbUI3xnwAtL8x+I+BW40xzc48/fNU5Z2PoUtNHR3g/EPLyPJ4+MPrS1IWg1JKxdLbGvpuwJEiMktE/iMiByYyqE75su0wRSdGAUoLszll6kjeWVLB5ur+73GjlFKd6W1C9wHFwCHAL4DnRERizSgiF4nIbBGZXVnZxwtzvH477Kdb6Hbm4qMmAPD0p+tTGodSSkXrbULfAEw31qdAGCiJNaMx5iFjzDRjzLTS0tLexmntTOipa6EDTB5RyPF7DOexmWuoqNFWulIqPfQ2ob8EHAsgIrsBfmBrooLq1M6SS//dcbEz15+yBw0tQf48Y2WqQ1FKKSC+bovPAB8Dk0Vkg4hcCDwGjHe6Mv4DuMD0xzXxO1voqU/o40ryOfegXXjyk3VU1qb2iEEppSC+Xi7nGmNGGmOyjDFjjDGPGmNajDHnGWOmGGP2N8bM6I9g06XkEvHDI8YBcP2LC1IciVJKue1KUZ+T0NOg5AIwobSAr+01nLcWb2H22vY9O5VSqn+5K6F7nRp6GpRcIu761r6MLsrlmukLaGzR+6UrpVLHZQk9vUouAPnZPm7++hRWVtTx8H9XpzocpdQA5q6EvrPkkj4JHeCYycM4ee8R3PvuCmatrkp1OEqpAcpdCT2nyA4bd6Q2jhhuO3sqwwfl8KuXFtLQEkx1OEqpAchdCT270A5b0u/GWIU5WfzuzL1YUVGnfdOVUinhroTuL7DDNEzoAMfuPpyz9x/Dg++vYvGmmlSHo5QaYNyV0H3Z4PFBc3omdICrT5oMwG9fXaTPH1VK9St3JXQRCAdh5l2pjqRTwwpzuOn0vfhk9TYu+NtnqQ5HKTWAuCuhu8R3Dt6F43YfxgfLK7njzaWpDkcpNUBoQk+CLK+HB87bH4CHPlhNeXVjiiNSSg0E7kvo036Y6gjiku3z8v7Pj8bn8XDVc/NpCYZTHZJSKsO5L6HPfswOt6Z/18Cyknx+e8ZefLSqigNveYdwWE+SKqWSx30JPaJxe6ojiMs3p41lj5GDqG4McM87y1MdjlIqg7kvoZ/1sB16vKmNowdev+wITpk6kntnrOQZfWydUipJ3JfQC4bZYaAhtXH0gIhw29lTyc3ycu30BazYUpvqkJRSGch9CT0r3w4D7uo5UpDt46n/ORiAU+6dyZqt9SmOSCmVadyX0P15dtjivoS4/y7FPHnhQQD89Jm5BEPa80UplTjuS+hZuXboopJLtCMnlXLzmVNYuLGGk+/9L+uq3LdjUkqlJxcm9EjJxZ0JHeCb08Zw61l7s66qga8/8JEmdaVUQrgvoe8subg3oYsI5xy0C69ddiRhYzjqjvd54P2VejMvpVSfuC+hZzkJ3cUt9IiJwwp44ceHMWpwDrf/exkP/mdVqkNSSrmY+xJ6pP/5+39IbRwJMqG0gA+vOZbT9hnF7f9expOfrEt1SEopl+o2oYvIYyJSISILY7z3cxExIlKSnPAGBhHh9rOnsvuIQn790kKenqUXHymlei6eFvrjwIntJ4rIWOAEQLNPAuT6vbx86eEcPbmU615cwMG/f4fK2vR6GLZSKr11m9CNMR8A22K8dTfwS6D/z+SJey7774lsn5cHv3sAx0wuZUtNM8fc+T6vzN+U6rCUUi7Rqxq6iJwObDTGzE9wPPHZ7zzIH5aSVSdbrt/L335wEO/87CsMK8zmsmc+57nZX6Y6LKWUC/Q4oYtIHnA98Js4579IRGaLyOzKysqeri62vCH2bosZ3M1v4rBC/v6jg5k4rIBfPv8F5z0yixc/35DqsJRSaaw3LfQJwDhgvoisBcYAc0VkRKyZjTEPGWOmGWOmlZaW9j7SaHlDIRyA5sy+ydWoolxeu+wIrjx+Nz5eXcWVz87nkqfn8uU293fZVEolXo8TujFmgTFmmDGmzBhTBmwA9jfGbE54dJ3JKbLDph39tspUyfZ5ufz4SXx63XEctVspr31RzpG3v8ezn63XB2Zkih3r4a9HwbY1/bveeU/Dqz/r33WqpIqn2+IzwMfAZBHZICIXJj+sbmQX2GFzXWrj6EdDC7J54ocH8cqlhzOuJJ+rX1jA+OteZ+aKrakOTfXVx/dD+Tx45af9u96XfgyzH4VAEzRsc81DY1Tn4unlcq4xZqQxJssYM8YY82i798uMMf2bVXIG22G6b4AbZsNbv07oIqeOKeKdnx3FNSftDsB5j87i8n98zntLK7TF3hvhsP1JpVl/scO1/03N+m8ZDrePg7v2Ss36VcK470pRgMFj7XDDp6mNozuPHAcf3Qs3DrY/TdWx51vxDmxZHPdivR7h4qMmsOimr3HK3iN5ed4mfvD4Z5z4pw94fUF5goIfIH5bDE99o/efDwVhex+u7o0+sS8p/joG6qHO6bgQCtht9ot/JnedxkDjDljzgf0eqD5xZ0IfOhEQqN3S+2V88c++1SznPQ3/vg4qlsCSf3V8f0eMroYLnu84beU78NTZ8OChPQ4hP9vH/d/dn/k3fJV/HriMZ2vO5/KnPuXchz7h0Zlr2FrnXJjU2Y5koKtxdn6r3u39A1PevQn+NLXn22KwxSbMl37SOs2EoXI53H8IVCewR1PDNnj9F/E9Q2DBc3a4/E07nP6jxMURy6y/wm27whOn2e9BvF69Mv6dTTjUu9hcyJ0JXQSKxkJjrOud4lC/1W6o9+4Li1/uvPtjOASLXrRf9vL5ULms9fD8pR/DJ/fDA4fAs+fB37/Rtqb/doxSS6zEuuaD1vFQsFe/zuDcLA5ccBPFpprpu77A+m0N3PLqQh78w89s0rh1F5rLl/Rq2Wkv2GJrwG9cY3/XniTCu3ZvHb9lBDz1ze4/88bV8PYNNoEbY4/AAOb9vWdxR3pozX+67fT7D4TKJXD3Xol5KteNg2055dOH4KP7Wqd/+Vns+d+8DjbObS0DJducv7V9vXFu95/ZvABmP2a/w12VyyqXwWePwG+H2PLnAODOhA6QO8S2PKI118HMe7rfI0e3zJ/7Hqx4u+M8K9+FvxwJ//y+/bL/9Stw/0Hw/A9iL3/l2/D4Ka2vF71ohyfeBpOd6bEOqT/8U+v4Ozd0fP/Jr9svZSzG2EPjYOstAvbe8jIfNn2d1Tnn8eus1iRzzf1P8r3HPuXjVVV2QlMNvPeHNp91pZtLbQ141oP29d19qAOveMseMXUmHLaJ7sN74I+7wd9Oan1v6Ws9W1f78z9Tz+k4z78u79kyo4XDUNXu7p2fPdw6Pv8ZO9z7m3BjNdwQ1WPs4WMSX89vrmstPd44GGY9ZKe3P0p++BjY9Hnr67pKu8Our7LbLEBN1NXTGzrZMYH9vr52lR1/7oK+/w7dWT8Llr+V/PV0wb0JvXAE1JbbVm0kwf5htE2KS1+FrStt63fGLXYD+vgBO88nD8LLP2m7rJqNdlj+hW05bVkEfz8LKhZ1XO/il+weP5byea1fIq/fDqf9EM55yo6/c4NtUQIsewP+dkrbz398n431tavsjgZg1Qw73LoSPn247c7kmXPgdyVwc/dXzd7tu5+r11/Mjx5+jx89NhNuHQv/uTWuz7rOx/fDxjl2vGKpPSJrr7OW/OKXW2vi21a3TjcGHjm27bzrP+5+eWBPjEcSWWRZL/5v23mmnNXxc18823WZJBSEe/a2Zb/2bhkOf96/7bR6pz5ev9X2bgE4/iY7FOl8PT0VCkBdRevrrSvtdzPaG7+AF38MoRgNioeOtsPmOrhzoj2SumO83WbLv4Cnv9U6b02cR2Q1G+DBw20pqZdHwp2KbF+PfRWejuMoL4l8KV17XxSXwfJ/w++GwrA94SdRX65IMoz25rUwdAL8+5qO7zVV243wr0fCoDHxbySx/Hl/KBgOISdx+/xt37+51LaInonRIov47BE7jD5Evu8AO3z957Dvd2HK2fb374G9WMWinAs73E5t7fz/ULbbPpBbFPuDwWZ7dOHN6n4lkfJVIhNEZzo7unjzuo7TLvsc/nMHnHYP+LJbex/lFLW9nmHu/9mfPc+wyT1i5D627NaZui22FZk/tON7kbJMxOr3YWO7EkDJbrGXO/8fsOCfdnv6nxlt31v2uu3D/sAhMP5o+Movoexw+15k+2tv4xx45ITW1wVRO/RvPwXPfrfjZ1rqwZ8fe3mBJrvzABgyvu1O8PQ/w77ntW67HX43p9w0bC/4+oP2KDjixsFwrPM/ij6a+euRbZcR64T0xw/AB3d0nL5lYevOoGQyXOp0qqirsI2/KWfb7daY+Lff166y39fjoi6cf/6HcOo9kDMovmUkkHtb6JvmtY5XLG5t+XYles8e7Z0bYMbNdjxWMt/tRDjpdvDlxv78ge1OHNXFOEF2zK9axzsrobT31vWxp897yh5BtHf9Zhg6qeP0iSd0nBal7MXT4bZduX/GCpZvqe345KSbh9kjgRsHwyuXwbJOdiRL/gU3FdkSVcT2ta0n2BKtvge3krh3P5tAbh5mv7D5zlXLVy6Ca9bDr9u14qOTOXSdzI+40g7vGA/rPmp7TiZyyB8RaGx7cnyf78DFH0LhyNZpN1bD2EOcz//MHglsnAMPH9d2Wc+d3zq++n14/GTbMPldF1dkT78IjHOUN3LftjvpPU6Fc5wke9q9cKZTR691rhls2Ga3gaWvwS2j7HgkmUPbZA62X/1vi9tOu+g/HWMq2sXuMG9sd45pxu86/z0i3r2p7ettq23jLXJ+7cirYK8Y35Wty2yDoLkO7pwEL1xoj8znPG634fblqs5EGl/v/rZ12sIXbKMgBaQ/H3s2bdo0M3t2gk5OvPs7+O+diVlWd6I3tEgy9uXAlYuhoQpKd4N3boKZd7X93PWbWx9q3dIAvx9JTP8zw96z8pFjYeq37aF2T5z3Akw8vvV1UzXcugscfDGcdJuddlOx7UXRhbsDZ/OnkO1psO/YIk7bZxQnjfMy6uG9O848an+77LEH2dc71tvD/4gfvGF/50jPhZPugIMvsslu41wYvX/bVtC8Z2DXw6B4147raq6zLcTo+T/6M7zl7CQP+D5Mu9B+GWc/2vHzsexxOmxdDpfMap0Wz472gB/YHXh+qf3ferNsgn7l0tZ5zrjf3kBu4xx4+NjOl3XxTBgR9Td78HDY/RQ45rrO44lsizXlbU/qdua6cnsEYsIdzy+0T6DtLXjeJjqAXQ6DUfvCJw90v87O/GabfUBNsMXVeJ9AAAAQEElEQVQeqUb8dK49egabSNuXimK5dmNrGefKRTB4jLMza/dohus3Q9VK+MsRHZcx+RQ46da22220IePh23+3txr58F449letj8CM6GybOel2OPh/Y7/XCyIyxxgzrdv5XJvQQwG4Y2LHy/8P+6n9sgOc9Qh8cDtc8C/44+SOy9j1CFg3s/N1XLuxYyLZttq29n5VYQ/d21v0ku2Kdf701mQe8dz32rb8frnG3misPWNsKyGi/aFsNPHCDTF6+9SU28NpT9SthkNBW6KKmHh8h5OAm4YeyirveG5pOYelm2s5zLOQp/2/j71uoOXQK/B/fE+n73cw6WuwIqrF/ptt8Oz5sMw5qRidZD64s7WVNvF4m9hH7A1f+Xnb/+dln9u/0ZbFtpQ1Ym97BFezAb73MvzfGbFjKd0DLvmk9fX6T2wPitd/3jrNk2XvGxTRWRJs/8W+7HO7nUSceGvHct+vt3Zdxnr81I4nJ7/7Akw6Pr6dz68q25b8oj9zzK/gqF90/fnazbG/N70xaDT8LMa1FqFAx79B+53VpbOhaNfWncDZj8Le34A7J0NdF3ccGX2A01gytlRz9LV2h7l5QewEH4/zpsNE50ippR5+P6rt+4deas+FQeu20rAN/AUdy689kPkJPWLDbHsBT8T1W1oPA6O/fO1bkKffZ2vR9ZW2xwLA/t+D4VPgjV/CN5+Avc5MbKyr3oMno5Z5w47Oa3WRL9+p98DUb9nW5JAJdgdjDNRusvXLkkk9q1dHlnvZPBgyzo5vXwt/2qfDrBvOehnPO79mVM0XcS++asKZDF31UvzxdIiv2v5+d0yEhjgvQP51FXjjOB0UCsJtZdASdVO3WAm6dkvrNhF5/4nTYJdDW1vPsZb9uxj1c7DXTfx0TtuE+rXfw6GXdB1vwzbb5bC9c56Gf3zHjp9xvy0HLXi+bTfe771s6+rRXr4EPnd6PsWT0KH7HUfBcHuk2lQNWTmtCe6q5XDfgdBcDUdd3fnfLZZwuLVU4y+E65wy6KIXbc38iCvs6/L5bevu0c552ibvWGK15Hvi+6/BmANtA+D/Tofc4tY6/w072jbGfrPNdqLY+1tw9sOxlxeHgZPQoe1Gd2M1/P1su6GdGXV4GGn1igdu2N7157evi33onwjG2JMwPr/dEDpTtcq29L/2+/iSVbwatkE42PZkGMBjJ7bttdHe2INh1H5d9k/+PDyRr7f8lrU53+l9fKfda1tz8V5k8tWb7VFZvKb/L3zxj9bXnbW413xgH0g+ptvvUFufPWrr3tEi6/jovtbzItdugOzC7pe39DUYvpe9OjpW76rIsiNHju2nR1vzX3jiVDv+gzdsias7G+bY0tb6jzq+d8pdcGC7WzsFW2xvr7EHQfVGu1Me2bGx0K3Id/LqdZ2frI+eL9qZf7HdMbv63rT/3I3Vdqc47ih7oVn7Xkhd+cZjMPNuu+Pa47TOd4Ldlbi6MLASenMd/HF3OP9FGHtgzz9/z1TYsc7WYU+9q/v5M1Hjdtt67Uz0xli7xXannP8MrLEnucyUb7DisNtZvrWZqvXLuGB265HIAcHHaPTk8YZcxq5SQbPJ4orAT3jQ/6f2a+lSOHcInPxHPC/8wE74+YqOO6buVC63F+8AXLHQXqCWSO3LZdD2b9eTHhTt/ecOeO/m1tcH/ghO+WPr6wcOs11tI6WGzuILh3reSAg02aPL7zxnuw86pb6WYJgsryAiGGMIhg3GgMHgFcHrsb+r9PR3jrdnVaSXCcDhl8PR19kjhe68fCl8/qQ9aXropbFLn0019qgncvSaVxL7qDH6HADYWxncFqNBePGHMGJK97HFMLASeiJsXmjLF7Hq4gNJ9OFuxPkvwYRjeracQKM9Ghjc2v84EAqzdMVyFtfmsbWmnhMXXMmQulUUByt4xX8yp7e83mYRZzXfyFyzG4I9mWvw4PUIP8qZwfrSo8kdMgavRygtzGZoQTaji3KYOqaIEYNy8Hhk583KdjQGqGsK8uX2BgqyfeRv+YyhVFO431n4vF139DLG0BQIU98SpLElRENLiG31LVTVN9PQHGJtVT352T7GleTj8whD8v0U5mRR+tFN5DZuwRxxJdlj96O8upH5X9rEPiTfT9hJfuGwwWAYPiiHcBgaAyEaAyGCoTAeEWqbgwh2HxA2MG77x+z9/g8B2LbPxazc92oaWoKICI07Kpg89yaeL7uJ+hb7N8vO8hAMGRoDIaobAtQ0BfCIUFHbTHMwxITSAsaX5JPl9TCmOJfy6iYWbqymMRDCI8LQAj/BsCHH58XnEbKzPJRXVFDVEGLF9jC1TUGG5PspKfCzZms9gVBrPvEI+H0eBGHisAImDS+gtCAbv8+DR4SWUJgsrwe/VyjI9lHbFGR7Q4CGliDVjQECIcOQ/CyaAmF8HqEgx0dOlpecLC95fi/ZPg8+jzCiwIvH5yfX76UlGGbjjka21bVQ0xSgujFAnt9Hts9Dc9DGu7KyjobmIJV1zXbnYwxNQefv5bPb2NjiPESgKRBiUn4jJYMLafLkctuitt+DHZ5ifln2PMV5forys3b+fmbth1ywrO31LpvOeZtRux8U19enPU3oqvdWzYDS3W0f7drytq2PZImxI5l34Tpysjws3FhDSzBMYyDE9voWKmubmbt+O42BEE2BENsbAoSi7jTp93kozPZRVd+C1yNt3mtvSL6fLK8QNuz8coeNwQCBYJiGQKjPD8byOMk4UfwEuC/rXi4LXEoTHRsgXo+Q5/diDLQEw3g9Qq7fS1FeFoU5WRhjGJrvx+f1sLqyjrVVDW3+RkV5WZQNzaemKUAgFMbn8dAUCBEKG5oCIUYV5TJycA4ji3IZXphDeXUjVfUtjC7KJc/vJT/bRzhsaAmFaWixn1tVWcfyLbVUNwZoDoYxhpj/m2yfhzy/l4IcH+EwhMKG7CwPobChrjlIUyBEUyC+u2Nm+zwU5WXR0BKiORgmx+fB7/MwcVgBuVlexhTn4RHbwcwjQk6Wl4aWIKGwYW1VPYL9O27c0ciWmiZEhMG5WeweXsV9dbar6u8H38D7HMD2hgDVDQFaQjY2v8/DhVlvcbV5jF8U3IrHBDnj6+dy2MTe1e41oSv3eekSe0+Usx6xfaLb9xLqRDhs2FrXzJqt9Xz+5Q6q6prZWtfC2OJcgmHD4Nws8rJ9lA3No7ElRNjAFxt2ODuEMMYYRGxZQLBfbo+A1+MhO8tDQbaPgmwfuX7bMizI9jFycC5ZXmGXIXnUN4f4cnsDYWOoqm+hvtleiVjdaFuI9c1BRhflUTY0j6I8P9sbWvCIOC1Vm+w37Wgk2+fZmTQaAyHy/F6CIcPQAj/G2B3D9oaA03oNEwobBuVmUZDtJWygMMdHUa6f4YOye1TiCIft0cL6bfUMyc9mSH7ve2PEwxhblvF4bJkmEDJsb2hhUE4Wuf7uHwAfDhuagiEaW0K0hMJU1tqWdnVjAANMGlZAidNSTprqjfa80yWzdnZlNMaww0nqpQXZeDyJu7hOE7pyn+qN9qTrsb/S0pdSUeJN6O699F9lnsGj4atxXB2olIrJvZf+K6WUakMTulJKZQhN6EoplSE0oSulVIbQhK6UUhlCE7pSSmUITehKKZUhNKErpVSG6NcrRUWkEojxEMC4lABx3iC7X2lcPaNx9YzG1XPpGltf4trVGNPFswWtfk3ofSEis+O59LW/aVw9o3H1jMbVc+kaW3/EpSUXpZTKEJrQlVIqQ7gpoT+U6gA6oXH1jMbVMxpXz6VrbEmPyzU1dKWUUl1zUwtdKaVUF1yR0EXkRBFZJiIrReSafljfYyJSISILo6YNEZG3RWSFMyx2pouI3OvE9oWI7B/1mQuc+VeIyAV9jGmsiLwnIktEZJGIXJ4OcTnLyxGRT0VkvhPbTc70cSIyy1nPsyLid6ZnO69XOu+XRS3rWmf6MhH5WgJi84rI5yLyarrE5CxzrYgsEJF5IjLbmZYO/8siEXleRJY629qhqY5LRCY7f6fIT42IXJHquJzlXels8wtF5Bnnu5C6bcw+Dip9fwAvsAoYD/iB+cCeSV7nV4D9gYVR024HrnHGrwFuc8ZPBt4ABDgEmOVMHwKsdobFznhxH2IaCezvjBcCy4E9Ux2Xs0wBCpzxLGCWs87ngHOc6X8BfuyM/wT4izN+DvCsM76n8//NBsY5/3dvH2P7GfA08KrzOuUxOctdC5S0m5YO/8sngB85436gKB3iiorPC2wGdk11XMBoYA2QG7VtfT+V21if/8DJ/gEOBd6Men0tcG0/rLeMtgl9GTDSGR8JLHPG/wqc234+4Fzgr1HT28yXgPheBk5Iw7jygLnAwdiLKHzt/4/Am8ChzrjPmU/a/2+j5+tlLGOAd4FjgVeddaQ0pqjlrKVjQk/p/xIYhE1Qkk5xtYvlq8CH6RAXNqF/id1B+Jxt7Gup3MbcUHKJ/NEiNjjT+ttwY0w5gDMc5kzvLL6kxe0cqu2HbQmnRVxOaWMeUAG8jW1l7DDGBGOsZ2cMzvvVwNAkxHYP8Esg8pj4oWkQU4QB3hKROSJykTMt1f/L8UAl8DenTPWIiOSnQVzRzgGeccZTGpcxZiNwJ7AeKMduM3NI4TbmhoQe69HZ6dQ1p7P4khK3iBQALwBXGGNq0iUuY0zIGLMvtlV8ELBHF+tJemwicipQYYyZEz05lTG1c7gxZn/gJOASEflKF/P2V2w+bKnxQWPMfkA9tpSR6rjsymwt+nTgn93N2h9xOTX7M7BlklFAPvb/2dk6kh6XGxL6BmBs1OsxwKYUxLFFREYCOMMKZ3pn8SU8bhHJwibzp4wx09MlrmjGmB3A+9jaZZGIRB5EHr2enTE47w8GtiU4tsOB00VkLfAPbNnlnhTHtJMxZpMzrABexO4EU/2/3ABsMMbMcl4/j03wqY4r4iRgrjFmi/M61XEdD6wxxlQaYwLAdOAwUriNuSGhfwZMcs4c+7GHXK+kII5XgMhZ8QuwNezI9O85Z9YPAaqdw783ga+KSLGzJ/+qM61XRESAR4Elxpi70iUuJ7ZSESlyxnOxG/oS4D3gG53EFon5G8AMY4uHrwDnOL0BxgGTgE97E5Mx5lpjzBhjTBl2m5lhjPluKmOKEJF8ESmMjGP/BwtJ8f/SGLMZ+FJEJjuTjgMWpzquKOfSWm6JrD+Vca0HDhGRPOf7Gfl7pW4bS8SJimT/YM9aL8fWZa/vh/U9g62JBbB7zwuxta53gRXOcIgzrwD3O7EtAKZFLeeHwErn5wd9jOkI7GHYF8A85+fkVMflLG8q8LkT20LgN8708c6GuRJ7mJztTM9xXq903h8ftazrnZiXAScl6P95NK29XFIekxPDfOdnUWSbTpP/5b7AbOd/+RK2N0g6xJUHVAGDo6alQ1w3AUud7f5JbE+VlG1jeqWoUkplCDeUXJRSSsVBE7pSSmUITehKKZUhNKErpVSG0ISulFIZQhO6UkplCE3oSimVITShK6VUhvh/NGaExYVE7J0AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "niter = 8000\n", "tau_mult = [.05, .8];\n", "nruns = 10 # number of runs to compute the average performance\n", "for itau in np.arange(0,len(tau_mult)):\n", " tauk = tau_mult[itau]/np.linalg.norm(X0,2)**2\n", " Jlist = np.zeros((niter,1))\n", " for iruns in np.arange(0,nruns):\n", " w = np.zeros((p,1))\n", " for i in np.arange(0,niter):\n", " ik = int( np.floor(np.random.rand()*n0) )\n", " gk = (X0[ik,:].dot(w)-y0[ik]) * X0[ik,:].transpose()\n", " w = w - tauk * gk\n", " Jlist[i] = Jlist[i] + J(w)\n", " plt.plot( (Jlist/nruns)/J(wopt)-1 )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use a decaying step size $\\tau_k=\\frac{\\tau_0}{1+k/k_0}$." ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[14.09087461]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHORJREFUeJzt3X2QHPV95/H3dx73Uatd7UpaiRUrhHkOh/DCETB+wIktMAZfLvHZzrlUMReVfSRlOHM2DjmHXOXKwdjOXYxzNjGUIUXAJuBgm/iCCmNkYhAWsh4RekASIGmRVg/7vDuzM/O7P7pHO1p2dnZXMzvTo8+ramp6enqmv+odfbr717/uNuccIiISfKFyFyAiIsWhQBcRqRIKdBGRKqFAFxGpEgp0EZEqoUAXEakSCnQRkSqhQBcRqRIKdBGRKhGZy5m1tra6zs7OuZyliEjgvfLKK0edc22FppvTQO/s7GTDhg1zOUsRkcAzszemM52aXEREqoQCXUSkSijQRUSqhAJdRKRKKNBFRKqEAl1EpEoo0EVEqkQgAv3ZHYf5u1/sKXcZIiIVLRCB/vyuHr77/N5ylyEiUtECEejhkJHRzaxFRKYUiEA3DOW5iMjUAhHoIQOnRBcRmVIwAj1kZJTnIiJTCkSgG6gNXUSkgGAEuhmKcxGRqQUk0NWGLiJSSCAC3TsoWu4qREQqWyAC3VA/dBGRQgIR6CFDvVxERAoIRKCbGaB2dBGRqQQk0L1n5bmISH6BCPRQdgu9zHWIiFSyQAS6v4GuA6MiIlMoGOhm1mFmz5nZDjPbbmafn/D+HWbmzKy1ZEWGsm3opZqDiEjwRaYxTQr4gnNuo5k1Aq+Y2Vrn3Ktm1gH8LvBmKYvMtqFrC11EJL+CW+jOuW7n3EZ/eADYASz13/4b4IuUuHnb0Ba6iEghM2pDN7NOYCWw3sxuAg465zaXoK5ThLK9XHRYVEQkr+k0uQBgZg3AE8BteM0wdwEfmsbn1gBrAJYtWzarIsebXGb1cRGRM8K0ttDNLIoX5o84554EVgDLgc1mth84C9hoZosnftY5d79zrss519XW1ja7InVikYhIQQW30M07TfMBYIdz7psAzrmtwMKcafYDXc65oyWqE9AWuojIVKazhX4N8GngOjPb5D9uKHFdpwidPFV0LucqIhIsBbfQnXMvMH5uT75pOotV0GRC6rYoIlJQIM4UTaQygAJdRGQqgQj0noEEMB7sIiLyToEI9HMXNgDj3RdFROSdAhHo6ocuIlJYQAJd/dBFRAoJRKCPn1hU5kJERCpYQALde1YvFxGR/AIR6GpDFxEpLBCBrmu5iIgUFohAzx4U1Ra6iEh+gQj0k9dD1xa6iEhegQj07B2LtIUuIpJfIAJddywSESksEIF+sg1dl3IREckrEIGufugiIoUFItBNZ4qKiBQUiEDXFrqISGEBCXR/C73MdYiIVLJABLppC11EpKBABLpO/RcRKSwQga6Lc4mIFBaIQNf10EVECgtEoKsNXUSksGAEevZaLmpzERHJKxCBPn4tFxERyScYgR5SG7qISCHBCHS1oYuIFFQw0M2sw8yeM7MdZrbdzD7vj7/XzF4zsy1m9iMzm1+6MrPXQ1egi4jkM50t9BTwBefchcBVwK1mdhGwFrjEOXcpsAv4csmKVBu6iEhBBQPdOdftnNvoDw8AO4ClzrlnnHMpf7KXgLNKVqTOFBURKWhGbehm1gmsBNZPeOszwM+KU9Jk8/WedYMLEZH8ph3oZtYAPAHc5pzrzxl/F16zzCN5PrfGzDaY2Yaenp7ZFamrLYqIFDStQDezKF6YP+KcezJn/GrgRuAPXZ72EOfc/c65LudcV1tb26yK1JmiIiKFRQpNYN7tgh4AdjjnvpkzfhXwJeB9zrnh0pU4fqao2tBFRPIrGOjANcCnga1mtskf92fA3wJxYK1/i7iXnHOfLUWRIX8/QnkuIpJfwUB3zr1AtiP4qf6l+OVMLtuGrku5iIjkF4gzRbNrE7Whi4jkF4xAVy8XEZGCAhHoJ88U1Ra6iEhegQh0M13LRUSkkEAE+vgWennrEBGpZAEJdPVyEREpJBCBnqUmFxGR/AIR6CFdP1dEpKBgBLqu5SIiUlBAAl1t6CIihQQi0HWmqIhIYcEIdJ0pKiJSUCACXWeKiogUFohAz26h7zkyWOZKREQqVyAC/dhgAoCHX3yjzJWIiFSuQAT6wnk1ADTEp3M/DhGRM1MgAr2pNkrIYPXVZ5e7FBGRihWIQAeIRUIkU5lylyEiUrGCE+hhBbqIyFQCE+jxaJhkWoEuIpJPYAI9Fg6R0Ba6iEhegQn0uNrQRUSmFJhA10FREZGpBSvQ1YYuIpJXcAJdvVxERKYUnEBXk4uIyJSCFehqchERyatgoJtZh5k9Z2Y7zGy7mX3eH99iZmvNbLf/3FzKQtXkIiIytelsoaeALzjnLgSuAm41s4uAO4FnnXPvAp71X5fMyFianYcHSjkLEZFAKxjozrlu59xGf3gA2AEsBW4GHvInewj4WKmKBPjl7qPo/hYiIvnNqA3dzDqBlcB6YJFzrhu80AcWFru4XBcsbizl14uIBN60A93MGoAngNucc/0z+NwaM9tgZht6enpmUyMAH754MQCZjDbTRUQmM61AN7MoXpg/4px70h992Mza/ffbgSOTfdY5d79zrss519XW1jbrQmMRr1T1dBERmdx0erkY8ACwwzn3zZy3fgys9odXA08Vv7xxcT/QdYEuEZHJTeeebtcAnwa2mtkmf9yfAX8N/NDMbgHeBP6gNCV6soGurosiIpMrGOjOuRcAy/P2B4tbTn5qchERmVqgzhQFbaGLiOQTmEAPh7xS0xkFuojIZAIT6NGQ1+ozlla3RRGRyQQm0MN+oKfVD11EZFKBCfRIWIEuIjKVwAR6tg09pUAXEZlUYAI9oiYXEZEpBSbQs23oKfVyERGZVGACPbuFnlIvFxGRSQUm0NXLRURkaoEJ9Ih/UPTt/tEyVyIiUpkCE+h7jw4C8OUnt5a5EhGRyhSYQF/Z4d2D+sMXLypzJSIilSkwgb64qQaAS8+aX+ZKREQqU2ACPeqfKaobXIiITC4wgW5mRMPGmK6HLiIyqcAEOnhXWtz19kC5yxARqUiBCnSAZ1+b9F7UIiJnvOncU7RiRMNGR3NducsQEalIgdpCf//5C4mGA1WyiMicCVQ6ttTF6B1JlrsMEZGKFKhAr4mGGB1TLxcRkckELNDDjI6ly12GiEhFClSgf3fdXhKpDEOJVLlLERGpOIEK9CuXtwDQ3acrLoqITBSoQP+TD5wLQO+wDoyKiEwUqEBvqY8BsOVAX5krERGpPIEK9OxJRev3HStzJSIiladgoJvZg2Z2xMy25Yy7zMxeMrNNZrbBzK4sbZmeproozXVRWhviczE7EZFAmc4W+veBVRPGfQ34S+fcZcBX/NdzorUhztHBxFzNTkQkMAoGunNuHXB84mhgnj/cBBwqcl15eYGug6IiIhPN9uJctwH/amZfx1spXJ1vQjNbA6wBWLZs2SxnN661Mc6WA72n/T0iItVmtgdFPwfc7pzrAG4HHsg3oXPufudcl3Ouq62tbZazG9faEOPogJpcREQmmm2grwae9IcfB+bkoChAW2OcoWSagdGxuZqliEggzDbQDwHv84evA3YXp5zCLmz3mu63H+qfq1mKiARCwTZ0M3sUeD/QamYHgL8A/hj4P2YWAUbx28jnwm8tbQLgsZff5KpzFszVbEVEKl7BQHfOfTLPW+8uci3Tku2DPpZx5Zi9iEjFCtSZolmXdcxn81vq6SIikiuQgR4yCIes3GWIiFSUQAb65cuaeePYMM6p2UVEJCuQgb5kfi0AB06MlLkSEZHKEchAX9biXXXx+JAuASAikhXIQF/eVg/Ay/smXmJGROTMFchAX9HWQOeCOjapp4uIyEmBDHSAjpY63joxXO4yREQqRqADfcuBPhKpdLlLERGpCIEN9HPbGgDY+fZAmSsREakMgQ30VZcsBuAnm+fs3hoiIhUtsIGe7Yv+97/cV+ZKREQqQ2ADHaAh7l1bTP3RRUQCHuh333QxAL/YeaTMlYiIlF+gA/3my5YA8L+e3lHmSkREyi/QgR4Ne+UfU5OLiEiwAx3g7o9eBMDDL+4vax0iIuUW+ED/nYsWAfCVp7YzOqaTjETkzBX4QD+rue7kjaP/cf2bZa5GRKR8Ah/oAE/deg0A//Onr/IPL+4vay0iIuVSFYEei4S471MrAfgfT23XnYxE5IxUFYEOcOOlS+hc4N344sZvvVDmakRE5l7VBDrA/7vtvQBsP9TPt57dXeZqRETmVlUFek00zNrbvVD/xtpduhKjiJxRqirQAd61qJE//8iFAPzV06+WuRoRkblTdYEO8F+uPYc/vnY5/7bnKDu6+8tdjojInKjKQAdY894VNMQj/OVP1OtFRM4MBQPdzB40syNmtm3C+D81s51mtt3Mvla6EmenrTHOnddfyEt7j/Oxv/uVQl1Eqt50ttC/D6zKHWFmHwBuBi51zl0MfL34pZ2+T1zRwZKmGja/1cvHv/siRwZGy12SiEjJFAx059w64PiE0Z8D/to5l/CnqcgLkodCxrovfoD/1NXBr/ef4Np7nuPAieFylyUiUhKzbUM/D7jWzNab2fNmdkW+Cc1sjZltMLMNPT09s5zd7EXCIe75/Uu59/cvJZHK8J57niOTUfOLiFSf2QZ6BGgGrgL+O/BDM7PJJnTO3e+c63LOdbW1tc1ydqfvD7o6uHzZfAD+9NHflK0OEZFSmW2gHwCedJ6XgQzQWryySuPxz17N1SsW8PTWbh761f5ylyMiUlSzDfR/Bq4DMLPzgBhwtFhFlUo4ZHz7U5fTVBvlL368nX965YB6v4hI1ZhOt8VHgReB883sgJndAjwInON3ZXwMWO0CkozN9TF+ccf7OXtBHXc8vpkvPbGFZCpT7rJERE6bzWUOd3V1uQ0bNszZ/KaSSmf46s9e44EX9gHw4pevo72ptsxViYi8k5m94pzrKjRd1Z4pWkgkHOLPP3Ihv3PhQgB++6s/576f6wqNIhJcZ2ygA5gZ31t9Bf986zWsaKvn68/sovPOp9lyoLfcpYmIzNgZHehZl3XM55nb38dKv1vjTff9G5fe/a9869ndnBhKlrk6EZHpOWPb0PM5MjDKf/7eenYdHgS829td0dnMtz91OfPrYmWuTkTORNNtQ1eg5zGSTLPr8ADfe2EfP9l8iPl1Ub606gI+3tVBODTpOVQiIiWhQC+in2w+xHfXvc62g/2saKtn9dWdXH9JO22N8XKXJiJnAAV6kTnn+OmWbr7xzE72HxsmEjL+6JpOVl3SzmUd87XVLiIlo0AvEeccm97q5Xu/3MfTW7sBaKmP8cELFvKZ9yznwvZ5Za5QRKqNAn0OHBtM8MKeo/x0Szcvvn6MwUSK31raxH9YuZRVlyymvamGPNcsExGZNgX6HDsxlOSJjQf40W8Osv2Qdx/TZS11nL2gjlves5x/v3wBtbFwmasUkSBSoJfRrsMDrNvVw9pXD7N+3/i9QS5qn8eqSxZzzbmtXLxkHjVRBbyIFKZArxCjY2l+/toRfr3/OOt29fB6zxAAkZBx8dImPnppO1eds4ALFjcSCes8LxF5JwV6BXLOsffoEDvfHmDbwT7W7e5h20GveSYWCXH+okYuXjKPKzpbOH9xI+cubNBWvIgo0IPirePDbHzzBNsO9vFqdz9bD/TRP5oCwAzOX9TIvztrPlcub2Fpcy1L59eyZH6tukmKnEEU6AGVSmfYe3SIHd39vH5kkC0H+3jljRMM+CEP3tb82S11LG+tZ+WyZpY217KkqYZ3LWqkqTZaxupFpBSmG+iRuShGpi8SDnHeokbOW9R4clwyleGtE8O83TfKm8eH2X90iH1Hh9jxdj/PvHr45HRmsLAxTmtDnM7Wela01tM2r4azW+o4p62e1oa4mnBEqpgCPQBikRAr2hpY0dbANRPe6x1Ocqh3lMP9o2w50MfB3mGODCTYdrCPf9naTe4OWMigo6WO9qYaljR5TTft88eHlzbXUh8Lq++8SEAp0ANufl2M+XUxLloyjw9csPCU91LpDMeHkuw6PMhbJ4bp7h1h3zHv+aW9xzg8kCCdObXJrS4WZun8Wtrn17J4XpxF82roaK7jrJZaOprraGvUVr5IpVKgV7FIOMTCeTUsnFcz6fupdIaewQSHekc51DvCvqNDnBhOcqh3hO6+UV7r7ufoYIIJmU9LfYz2phram2pZMn/8eUF9nKbaKG2NcVrqY8Qi6oYpMpcU6GewSDhEe1Mt7U21vPvs5kmnGUtn6O712u4P9Y5wZGCUQ32jdPeOcODEMC/vO3ayV04uM2ifV0Or36bf2hCjtSHOwsY4yxbUsaA+TmtjnEWNcfW/FykSBbpMKRoOsWxBHcsW1OWdZiiRortvhGODSfpGxugZTHC4b5SDvaMcHUxwuH+U7Yf6ODaYJDVhcz9k2WajKC1+81FzXZSW+vHh5voYzf5wdtqoVgIi76BAl9NWH49w7sJGzl049XSZjOPYUJI3jw9zYihJz2CC7t4Rjg8nOTE0xonhJAd7R9h2sI8Tw0kSqUze72qsiXghX++Hfl2MxpqIv+Ufo6k2SmNNlAX1MebVRJlXG6GxJqr++1LVFOgyZ0Iho60xPu0bg4wk037YJ+kdHuP4cJLenPD3HmMcG0zyes8gfcNjkzb/5GqIR5hXE2FebfRk0HvP0SnGe68b4hE1D0lFU6BLxaqNhVka886Ona5kyuvZ0zcyRv/oGL3DY/T7w/0jKf95/HV33yg7Dw+cfK/QeXaN8Qj18QiNNd5zQzxCXSzsPcfD1Mcj1Mci/rP3uqHGWzk05awk4pGQuodK0SnQparEIiEWN9WwuGnynj1TyWQcQ8kU/aMpL/RHxsaHR8foG/Eeg6MpBhMphpJphhIpjg4mGEqmGEqkGUykSE7RVJQVDRv18Qi10TA10TD1cW+lUBsNE4+EiUdDxCMh6mIR6v0VRUPuyiIepi7mTR+LGJFQiEjYiIVDRMLjw1pxnFkU6CK+UMhorPHa3meyVzDRWDrDcCLNYDLFUMJ7ZFcMfTl7B0OJFKNjaUZTGYYTKQZGU/QMJkimMiRSGUbH0gz7K42JXUenKxo2byURDXsBHw3RWBOlPhamLuatTOpi3sqhJhomHglREw1TE/Wesyuc7Ovs+w3+nkpdLKyVRgVRoIsUWTQcoqkuRFNdca6r45xjdCzj7wX4eweJNEPJFImxNMm0I5XOkEo7kumMN5zxhvtHvJVGIpUmkcqQGMswkEgxnEjROzx2cqUxnEwxMpZmLD3zNUc4ZP5KwQv/aDhELBwiGgkRD4eIRfyHPxwNh6iJhk7ZE4lF/NeR8T2TuliYWv97a6NhIuEQIfOWbzSc/Yw3rIPdnoKBbmYPAjcCR5xzl0x47w7gXqDNOXe0NCWKnNnMjFo/3FobpndAebYyGecFfyrN6FjmZNBn9xgSYxlGxtIMjqYYTnrNTsN+c9NwMkUilWEsnSGZypBMO5Ipb3zviD/Of4z6z4nU7FYiE0VCdnIPIh4JEc8OR8PU+HsV2RXLZCuZ3NfxU16H3zFtvMBny7m3Mp0t9O8D9wEP5440sw7gd4E3i1+WiJRDKDS+8pgrGX9vIjGWObkncXKvIenvQYylyWQc6YxjLO2tNBKpDMn0+Eoid0Xk7Y143zWSTNPrd4PNnT53eOL5Eacj6h+/mBj2X/29S7lyeUvR5jOZgoHunFtnZp2TvPU3wBeBp4pck4icQUIhoyYU9q8RVJ7LP2dXFIlJwt57nR5/L8+KIZHKTJgmfco09fHSryRn1YZuZjcBB51zm3UwRESCLhwywidXKsE140A3szrgLuBD05x+DbAGYNmyZTOdnYiITNNsTntbASwHNpvZfuAsYKOZLZ5sYufc/c65LudcV1tb2+wrFRGRKc14C905txU4edUOP9S71MtFRKS8Cm6hm9mjwIvA+WZ2wMxuKX1ZIiIyU9Pp5fLJAu93Fq0aERGZNV06TkSkSijQRUSqhAJdRKRKmCt0AehizsysB3hjlh9vBSqxJ43qmhnVNTOqa2YqtS44vdrOds4V7Pc9p4F+Osxsg3Ouq9x1TKS6ZkZ1zYzqmplKrQvmpjY1uYiIVAkFuohIlQhSoN9f7gLyUF0zo7pmRnXNTKXWBXNQW2Da0EVEZGpB2kIXEZEpBCLQzWyVme00sz1mdmeJ59VhZs+Z2Q4z225mn/fH321mB81sk/+4IeczX/Zr22lmHy5l3Wa238y2+jVs8Me1mNlaM9vtPzf7483M/taf/xYzuzzne1b70+82s9WnUc/5Octkk5n1m9lt5VpeZvagmR0xs20544q2fMzs3f7y3+N/dlo3BMhT171m9po/7x+Z2Xx/fKeZjeQsu+8Umn++f+Ms6yra387MlpvZer+uH5hZ7DTq+kFOTfvNbFMZlle+fCj7bwzwbkBbyQ8gDLwOnAPEgM3ARSWcXztwuT/cCOwCLgLuBu6YZPqL/JrieJcVft2vuSR1A/uB1gnjvgbc6Q/fCdzjD98A/Aww4CpgvT++BdjrPzf7w81F+lu9DZxdruUFvBe4HNhWiuUDvAz8tv+ZnwHXn0ZdHwIi/vA9OXV15k434XsmnX++f+Ms6yra3w74IfAJf/g7wOdmW9eE978BfKUMyytfPpT9N+acC8QW+pXAHufcXudcEngMuLlUM3POdTvnNvrDA8AOYOkUH7kZeMw5l3DO7QP2+DXPZd03Aw/5ww8BH8sZ/7DzvATMN7N24MPAWufccefcCWAtsKoIdXwQeN05N9XJYyVdXs65dcDxSeZ52svHf2+ec+5F5/3Pezjnu2Zcl3PuGedcyn/5Et69BfIqMP98/8YZ1zWFGf3t/C3L64B/KmZd/vd+HHh0qu8o0fLKlw9l/41BMJpclgJv5bw+wNQBWzTm3Ut1JbDeH/Un/m7Tgzm7aPnqK1XdDnjGzF4x725QAIucc93g/eAYv179XNf2CU79T1YJywuKt3yW+sOlqPEzeFtjWcvN7Ddm9ryZXZtTb7755/s3zlYx/nYLgN6clVaxlte1wGHn3O6ccXO+vCbkQ0X8xoIQ6JO1H5W8a46ZNQBPALc55/qB/4t3t6bLgG68Xb6p6itV3dc45y4HrgduNbP3TjHtnNXmt43eBDzuj6qU5TWVmdZSkhrN7C4gBTzij+oGljnnVgL/DfhHM5tXqvlPolh/u1LV+0lO3XCY8+U1ST7knTRPDSVZZkEI9ANAR87rs4BDpZyhmUXx/liPOOeeBHDOHXbOpZ1zGeDv8XYzp6qvJHU75w75z0eAH/l1HPZ31bK7mUfKUNv1wEbn3GG/vopYXr5iLZ8DnNoscto1+gfDbgT+0N/Fxm/SOOYPv4LXPn1egfnn+zfOWBH/dkfxmhgiE8bPmv9dvwf8IKfeOV1ek+XDFN83t7+x6Ta2l+uBdxOOvXgHYbIHXC4u4fwMr93qf08Y354zfDteWyLAxZx6oGgv3kGiotcN1AONOcO/wmv7vpdTD8h8zR/+CKcekHnZjR+Q2Yd3MKbZH245zdoeA/6oEpYXEw6SFXP5AL/2p80esLrhNOpaBbwKtE2Yrg0I+8PnAAcLzT/fv3GWdRXtb4e3x5Z7UPS/zraunGX2fLmWF/nzoTJ+Y6fzn3iuHnhHinfhrXnvKvG83oO3i7MF2OQ/bgD+Adjqj//xhB/9XX5tO8k5Il3suv0f62b/sT37nXhtlc8Cu/3n7A/DgG/789+Kd+/X7Hd9Bu+g1h5ygniWddUBx4CmnHFlWV54u+LdwBje1s4txVw+QBewzf/Mffgn582yrj147ajZ39l3/Gn/o//33QxsBD5aaP75/o2zrKtofzv/N/uy/299HIjPti5//PeBz06Ydi6XV758KPtvzDmnM0VFRKpFENrQRURkGhToIiJVQoEuIlIlFOgiIlVCgS4iUiUU6CIiVUKBLiJSJRToIiJV4v8DmwwA23HfJnEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "niter = 20000\n", "Jlist = np.zeros((niter,1))\n", "w = np.zeros((p,1))\n", "for i in np.arange(0,niter):\n", " tauk = 1/np.linalg.norm(X0,2)**2 * 1/(1+i/10)\n", " ik = int( np.floor(np.random.rand()*n0) )\n", " gk = (X0[ik,:].dot(w)-y0[ik]) * X0[ik,:].transpose()\n", " w = w - tauk * gk\n", " Jlist[i] = J(w)\n", "plt.plot(Jlist/J(wopt)-1 );\n", "print( Jlist[-1]/J(wopt)-1 )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use a decaying step size $\\tau_k=\\frac{\\tau_0}{1+\\sqrt{k/k_0}}$ and average the iteration $\\frac{1}{K}\\sum_{k" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "niter = 20000\n", "Jlist = np.zeros((niter,1))\n", "w = np.zeros((p,1))\n", "w1 = np.zeros((p,1))\n", "for i in np.arange(0,niter):\n", " tauk = 1/np.linalg.norm(X0,2)**2 * 1/(1+np.sqrt(i/10.0))\n", " ik = int( np.floor(np.random.rand()*n0) )\n", " gk = (X0[ik,:].dot(w)-y0[ik]) * X0[ik,:].transpose()\n", " w = w - tauk * gk\n", " w1 = 1/(i+1) * w + i/(i+1) * w1\n", " Jlist[i] = J(w1)\n", "plt.plot(Jlist/J(wopt)-1 );" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "Ridge Regularization\n", "=======\n", "\n", "Regularization is obtained by introducing a penalty. It is often called\n", "ridge regression, and is defined as\n", " $$ \\umin{ w } \\norm{Xw-y}^2 + \\lambda \\norm{w}^2 $$\n", "where $\\lambda>0$ is the regularization parameter.\n", "\n", "\n", "The solution is given using the following equivalent formula\n", " $$ w = (X^\\top X + \\lambda \\text{Id}_p )^{-1} X^\\top y, $$\n", " $$ w = X^\\top ( XX^\\top + \\lambda \\text{Id}_n)^{-1} y, $$\n", "When $p" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "run -i nt_solutions/ml_2_regression/exo1" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "## Insert your code here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Exercise 2__\n", "\n", "Display the regularization path, i.e. the evolution of $w$ as a function\n", "of $\\lambda$." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEQCAYAAABIqvhxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXl8VNXd/9/n3tm3bBMIhEDYd2RXVHBXVLQqKhU3ylNwqX2qP7G2tirVLm6t7dMW0UctblXcq9S2PlpSwA1BVJBFtgBhCdmTyexz7++POzOZCQkkkBXO29fxnPM95945Nwn3M2f7HqHrOhKJRCKRtAalsxsgkUgkku6HFA+JRCKRtBopHhKJRCJpNVI8JBKJRNJqpHhIJBKJpNVI8ZBIJBJJq5HiIZFIJJJWI8VDIpFIJK1GiodEIpFIWo0UD4lEIpG0GlNnN6C98Hq9emFhYWc3QyKRSLoVa9euLdd1PfdI9Y5b8SgsLGTNmjWd3QyJRCLpVgghdrWknhy2kkgkEkmrkeIhkUgkklYjxUMikUgkrea4nfOQSCTdm0gkQklJCcFgsLObclxis9no06cPZrP5qK6X4iGRSLokJSUluN1uCgsLEUJ0dnOOK3Rdp6KigpKSEvr3739U95DDVhKJpEsSDAbJycmRwtEOCCHIyck5pl6dFI9GRGMar68tQR7PK5F0PlI42o9j/dlK8WjEPzYcYMFrX/HIv7Z0dlMkEomkyyLFoxEzxvTi2pP78kTRdp5ZtbOzm9P2fPCBESQSyREpLi7GbrczduxYwNh8nLCPGjWqQ9pw5plnJjc8Jz4/EAgwduxYLBYL5eXlHdKOxsgJ80YIIXjgO6Oo8IV5cNlGvC4L3xmb39nNajt++UsjPvfczm2HRNJNGDhwIF9++WVnNyMNu93Ol19+SWe6YJLi0QSqIvj9d8dyw7OrWfDaV2Q7LUwdfERXLxKJpJ34xbvfsHFfbZvec0RvD/dfMrJV1+TmHvoeiMVi3H333fzrX/9CCMG8efP44Q9/yAMPPMC7775LIBDg1FNP5cknn2Tz5s3ceOONrF69GjB6MJdeeilff/01H374IQsWLCAajTJp0iSeeOIJrFbrET+/s5DDVs1gM6v87w0TGZjr4uYX1rK+pKazmySRSDqZzz///BDbU089xc6dO1m3bh1ff/011157LQC33XYbn3/+ORs2bCAQCLBs2TKGDx9OOBxmx44dACxdupSrr76aYDDInDlzWLp0KevXrycajfLEE0+06PM7C9nzOAwZdjPPzZ3MFYs+Zs5fVvP6LafS3+vs7GZJJCccre0hdCQffPABN998MyaT8TrNzs4GYPny5TzyyCP4/X4qKysZOXIkl1xyCVdffTWvvvoqP/nJT1i6dClLly5ly5Yt9O/fnyFDhgBw44038uc//5nbb7+9057rSMiexxHo6bHx/H9NRtN1bnj2Mw7Wyt2uEomkAV3XD1n2GgwGufXWW3n99ddZv3498+bNS+6pmDVrFq+++irffvstQggGDx7cLbcGSPFoAQNzXfzle5Op8IWZ9dSn7K0OdHaTjp4nnzSCRCJpE84//3wWL15MNBoFoLKyMikUXq8Xn8/H66+/nqw/cOBAVFXlwQcfZNasWQAMGzaM4uJitm3bBsALL7zAGWec0cFP0jqkeLSQsQWZvPBfkyn3hbh68ScUl9d3dpOOjqFDjSCRSNqE73//+/Tt25cxY8Zw0kkn8de//pXMzEzmzZvH6NGjueyyy5g0aVLaNbNmzeLFF1/k6quvBgw/U3/5y1+46qqrGD16NIqicPPNN3fG47QY0R27Sy1h4sSJenscBrVhbw3XP/MZJlXhxf86maF57jb/jHbl3XeN+JJLOrcdEskR2LRpE8OHD+/UNhQXFzNjxgw2bNjQqe1ojsShd16v96iub+pnLIRYq+v6xCNdK3serWRUfgav3jQFAcx66pPutwrrt781gkQiOSKqqlJTU5PcJNhVSGwSjEQiKErnvMaleBwFg3u6ee3mKTgtJmb/76d8XlzZ2U2SSCTtQEFBAXv27OmymwT37t2bXN3V0UjxOEr65Th57eYp5Lqt3PDMav7zbVlnN0kikUg6DCkex0DvTDtLb5pCodfJ3CWf8/wnxd1yyZ1EIpG0Fikex0iu28prN0/hrKG53Pe3b/j52xuIxLTObpZEIpG0K1I82gCX1cST10/k5jMG8tJnu7nhmdVU1Yc7u1lN88ILRpBIJEekvb3qnnrqqUesU1hY2KTn3KKiIj7++ONkfuHChSxZsgSAu+66i7y8PB577LFjbmNzSPckbYSqCH5y4TCG9HTxkzfWc9mij3jmxokM6tHFlvIWFHR2CySSbkV7etVNffm3lqKiIlwuV5MC9Oijj+J0tq8rJdnzaGOuGN+Hl+efQn0oyuV//pjlmw92dpPSWbrUCBKJpNU05dX2oosu4uuvvwZg3LhxPPDAAwDce++9PP3004DxMp80aRJjxozh/vvvT17rcrkA0DSNW2+9lZEjRzJjxgwuuuiitF3pf/zjHxk/fjyjR49m8+bNFBcXs3jxYh5//HHGjh3LypUrcblc2O32dnv2xnSJnocQYjrwB0AFntZ1/aEm6lwNLAR04Ctd12d3aCNbwYR+WfztttOZ99wavrfkc24+YyB3nj8Es9oFtDrhqTPuFkEi6Rb84ydwYH3b3jNvNFx4yKvmsDTl1XbatGmsXLmSwsJCTCYTH330EQCrVq3iuuuu4/3332fr1q2sXr0aXde59NJLWbFiBdOmTUve480336S4uJj169dz8OBBhg8fzty5c5PlXq+XL774gkWLFvHYY4/x9NNPc/PNN+NyuViwYAEAU6dOPZqfwlHT6W8zIYQK/Bm4EBgBXCOEGNGozmDgp8Bpuq6PBLquq8k4+Zl23rjlVK6ZXMDi/2zn6ic/YU+lv7ObJZFI2pipU6eyYsUKVq1axcUXX4zP58Pv91NcXMzQoUN5//33ef/99xk3bhzjx49n8+bNbN26Ne0eq1at4qqrrkJRFPLy8jjrrLPSyq+44goAJkyYQHFxcUc92mHpCj2PycA2Xdd3AAghXgG+A2xMqTMP+LOu61UAuq53sbGgprFbVH5zxRhOG+Tlp2+s56L/WcnDM8dw0ehend00iaR70coeQkcyadIk1qxZw4ABAzjvvPMoLy/nf//3f5kwYQJgeN396U9/yk033dTsPY60xD9xKJSqqkkHjJ1Np/c8gHxgT0q+JG5LZQgwRAjxkRDi0/gwV7dhxpjevPejqQzIdXHrS19wz1vrCUZind0siUTSBlgsFgoKCnj11Vc55ZRTmDp1Ko899lhyGOmCCy7g2WefxefzAbB3714OHkz//nv66afzxhtvoGkapaWlFBUVHfFz3W43dXV1bf48LaUriIdowtZYhk3AYOBM4BrgaSFE5iE3EmK+EGKNEGJNWVnX2vFdkO3g9ZuncNMZA/jrZ7u59E+rup9fLIlE0iRTp06lZ8+eOBwOpk6dSklJSVI8zj//fGbPns2UKVMYPXo0V1555SEv/ZkzZ9KnTx9GjRrFTTfdxMknn0xGRsZhP/OSSy7hrbfeSk6YdzSd7lVXCDEFWKjr+gXx/E8BdF3/TUqdxcCnuq4viec/BH6i63qzZzK2l1fdtuA/35bx49e/otwXZt7UAdx+7mBsZrVjPjyxXvwovXBKJB3FieZV1+fz4XK5qKioYPLkyXz00Ufk5eUd9f0WLlyYNqHeFN3dq+7nwGAhRH8hhAX4LvBOozpvA2cBCCG8GMNYOzq0lW3IGUNyef+OM7hyfB8W/2c7F/3PStZ0lHNFr1cKh0TSQjrSq+6MGTMYO3YsU6dO5d577z0m4bjrrrt48cUX23WvR6f3PACEEBcBv8dYqvusruu/EkI8AKzRdf0dYZzx+FtgOhADfqXr+iuHu2dX7nmksnJrGT95Yz37agLcOKWQH08fisPSjusY4jtQmTOn/T5DImkDukLP43jnWHoeXUI82oPuIh4A9aEoj/xzM899souCbDu/uHQkZw/r2T4fduaZRtyCCTmJpDOR4tH+dPdhqxMep9XEL74zildvmoJFVZi7ZA1zl3zefY+6lUgkxz1SPLoQk/tn848fTeOei4bx2Y4Kzn98BY/8czP1oa6xrlsikUgSSPHoYlhMCvOnDWT5gjOZMaYXi4q2c85v/8M7X+2TZ4VIJJIugxSPLkoPj43fzRrLG7dMIcdl4b9fXsfMJz7msx0Vnd00ieSEoTmX7G1JR7tSbyu6gnsSyWGY0C+bd247ndfW7OHxD75l1lOfcvawHvx4+lCG5Xlaf8P33mv7RkokxzHt6ZK9MR3hSr2tkD2PboCqCL47uS9FC87i7unDWFNcyYV/WMn/W/pl650tOhxGkEgkrSbhkt3n83HOOeck3aT/7W9/S9Z58MEHGTZsGOeddx7XXHNNshexfft2pk+fzoQJE5g6dSqbN28G6HBX6m2F7Hl0I+wWlVvOHMg1kwt4omg7Sz4uZtnX+7lmcgE3nzmQXhkt+ANctMiIb721fRsrkbQhD69+mM2Vm9v0nsOyh3H35LtbdU3CJbvNZuOtt97C4/FQXl7OKaecwqWXXsratWt54403WLduHdFolPHjxycdJM6fP5/FixczePBgPvvsM2699Vb+/e9/H3YHeFdGikc3JNNh4acXDWfOaYX84YOtvPTZbl5evYcrJ/bhljMGUpB9mJ7Fq68asRQPieSo0XWde+65hxUrVqAoCnv37qW0tJRVq1bxne98J9mTuOSSSwCjp/Lxxx9z1VVXJe8RCoU6pe1thRSPbkyvDDsPzRzDD84axOL/bOe1NSW8+vkeLh+Xzw/OGkSht3uMnUokR6K1PYT25qWXXqKsrIy1a9diNpspLCwkGAw2uyJS0zQyMzM7bO6kI5BzHscBBdkOfnX5aP7z4zO57pR+vPPVPs7+bRH//fI6NuyVnnslkrampqaGHj16YDabWb58Obt27QIM1+rvvvsuwWAQn8/H3//+dwA8Hg/9+/fntddeA4yey1dffdVp7W8LpHgcR/TKsLPw0pGsvPssvj91AB9uKmXGH1dxzVOf8u/NpWia3CcikbQF1157LWvWrGHixIm89NJLDBs2DDAOhrr00ks56aSTuOKKK5g4cWLStfpLL73EM888w0knncTIkSPTJtm7I3LY6jikh9vGPRcN5wdnDWLp57v5y0fFzF2yhoG5Tv5aFyLXZZHfGiSSY8Dr9fLJJ580WbZgwQIWLlyI3+9n2rRp3HnnnQD079+ff/7znx3ZzHZFvkOOYzLsZuZPG8iKH5/FH747FrtF5eTzfs6kc3/OI//cTEmVPFNdIjkcR+OSff78+YwdO5bx48czc+ZMxo8f3+JrO8KVelshveqeQOi6zqc7Knn2o518uKkUgLOH9eC6U/oxbXAuitLUoY4SSecgveq2P8fiVVcOW51ACCGY8tZfmALsu/sHvLzaWOL7wabP6Zfj4NqT+zJzfB9yXNbObqpEIuniyGGrE41ly2DZMnpn2rnz/KF8/JOz+eM14+jpsfHr9zZzym8+5JYX17J880GiMa2zWyuRSLoosudxgmMxKVxyUm8uOak335bW8dqaPbz5xV7+seEAPT1WrpzQh6smFMg9IxKJJA0pHpIkQ3q6+dnFI7jrgmH8e/NBXl2zhyeKtvPn5duZ2C+Ly8blc/HoXmQ5LZ3dVIlE0snIYSvJIVhMCtNH5fHsnEl88tNz+PH0odQEIvz87Q1M/vUHfP+5NSz7eh/BSKyzmyqRtCvNuWQvLi5m1KhRACxZsoTbbrutVfctLCykvLz8EPuSJUtYuHAhAI8//jh9+/Zt9b07CtnzONFopffOnh4bt545iFvOGMjG/bX87ct9/O3LvXywqRSX1cQFI/O4eEwepw/KxWKS30Ukxx8d6ZI9lTvuuIOsrCy66qpRKR4nGv/4x1FdJoRgZO8MRvbO4O7pxjG5b63byz+/OcAbX5TgsZk4b0QeM8b04rRBXikkkuOShEv2xuzZs4fp06ezc+dOZs+ezf33309xcTHTp0/n5JNPZt26dQwZMoTnn38eR/xIhD/+8Y+8++67RCIRXnvtNYYNG4bdbsflcnXkIx01UjwkrUZVBKcO8nLqIC+/unw0q7aV8fevD/D+xnQhuWBkT6YOzsVuUTu7yZJuzoFf/5rQprZ1yW4dPoy8e+5p1TUJl+yNWb16NRs2bMDhcDBp0iQuvvhivF4vW7Zs4ZlnnuG0005j7ty5LFq0KOmC3ev18sUXX7Bo0SIee+wxnn76aWbNmnXMz9VRyK+HJxoPPmiENsJiUjh7WE9+e/VJrPn5uTw7ZyLnjcjj/zYeYP4Laxn34PvMf34Nr68toao+3GafK5F0Jc477zxycnKw2+1cccUVrFq1CoCCggJOO+00AK677rqkHeCKK64AYMKECRQXF3d4m48V2fM40fjwQyO+9942v7XVpHL2sJ6cPawnkdhoVu+s5P1vDvD+xlLe31iKImBSYTbnDO/B2cN6MjDXiRByV7vkyLS2h9DRNP47TuSbswNYrcZmXFVViUaj7dzCtkeKh6RdMKsKpw3yctogLwsvHcmGvbX865sD/N/GUn793mZ+/d5m+mY7OHtYD84e1oOTB2RjNcnhLUn35P/+7/+orKzEbrfz9ttv8+yzzwKwe/duPvnkE6ZMmcLLL7/M6aef3sktbTukeEjaHSEEo/tkMLpPBgsuGEpJlZ/lW8pYvvkgL6/ezZKPi3FYVKYMyGHakFzOGJIrNyVKuhWnn346119/Pdu2bWP27NlMnDiR4uJihg8fznPPPcdNN93E4MGDueWWWzq7qW2GFA9Jh9Mny8H1p/Tj+lP6EQjH+GRHOf/efJAV35bz4eaDAPTNdjB1sJdpQ3I5dWAObpu5k1stkTRQWFjIhg0bAJgzZw5z5sxpsp6iKCxevPgQe+ocx8SJEykqKmqHVrYvUjxONHJyOrsFadgtDfMkAMXl9azYWsaKb8t4a91eXvpsN6oiGNMng9MGejl1UA7j+2ZhM8shLkn7k+qSvaP3ejz++OMsXryYmTNndujnthTpkl3SZQlHNdbuquKjbeV8tL2cr0tqiGk6VpPCpMJspgzM4ZQBOYzpk4FZlQsHjzekS/b2R7pklxyXWEwKUwbmMGVgDgsYSl0wwmc7Kvloezkfb6vg0X9tAcBuVplYmMXJ/bPjYpIpNylKJO2MFI8TjZ/+1Ih/85vObcdR4LaZOXdET84dYQxxVfhCrN5ZyWc7K/l0RwWPvf8tAFaTwtiCTCb3z2ZiYTbj+2bKOROJpI3pEuIhhJgO/AFQgad1XX+omXpXAq8Bk3Rdl2NSR0Mz5y53R3JcVi4c3YsLR/cCoLI+HBeTCtbuqmJR0XZi2jYUAcN7eZhUmM34flmM75tJfqZd7jGRSI6BThcPIYQK/Bk4DygBPhdCvKPr+sZG9dzAfwOfdXwrJd2BbKeF6aPymD4qDwBfKMqXu6v5vLiSNbsqWfr5HpZ8XAxAT4+V8X2zjNAvi5G9PXISXiJpBZ0uHsBkYJuu6zsAhBCvAN8BNjaq9yDwCLCgY5sn6a64rCZOH+zl9MFeAKIxjc0H6vhidxVf7Kpi7e4q/rHhAABmVTC8l4eT+mQytiCTkwoyGeB1ynPdJZJm6ArikQ/sScmXACenVhBCjAMKdF1fJoSQ4iE5Kkyqwqj8DEblZ3DDlEIADtYF+WJXNV+VVPPl7mre/KKEFz7dBYDbZmJMnwxG52dyUnyToxzuOrFIbPQbOnQoX375JYWFhUflh+rUU0/l448/pri4mDlz5lBUVMTKlSu56aabUBQluWekO9EVxKOpf4nJ9cNCCAV4HJhzxBsJMR+YD9C3b982at5xRp8+nd2CLkUPty1tqCum6Wwv8/Hlnmq+3FPN+pIanlm1g0jM+JPMdloYnZ/BmD6Ge/pR+R4pKMc5bXGex8cff3yIberUqbz33nvMmDHjmO7dWXQF8SgBClLyfYB9KXk3MAooiv8DzQPeEUJc2njSXNf1p4CnwNjn0Z6N7ra8+GJnt6BLoyqCIT3dDOnp5uqJxp9lKBpjy4E6viqpYX1JNV+X1LCoqJyYZvyJZdjNjMr3MKp3BiN6exjZ20N/rwtVDnm1GStf/ZbyPb42vae3wMXUq4e06prEeR779+9n1qxZ1NbWEo1GeeKJJ9iwYQM7d+7kkUceAYxTAdeuXcsf//hHXC4XPp8PVVXJzs5u0+foLLqCeHwODBZC9Af2At8FZicKdV2vAbyJvBCiCFggV1tJOgqrSWVMn0zG9MkE+gEQjMTYfKCODXtr+GZfLd/sq+EvHxUTjmkA2MwKQ/M8jOjlZkQvDyN6exjS0y2XDHdzEud5/PWvf+WCCy7gZz/7GbFYDL/fz7Bhw5gyZUpSPJYuXcrPfvaztOsLCgp48803O7zd7UGni4eu61EhxG3AvzCW6j6r6/o3QogHgDW6rr/TuS08zrj9diP+/e87tx3dHJtZZWyBMbmeIBLT2FrqY9P+Wjbur2XjvlreW3+Al1c3TOn1ybIzLM/DsDw3w3q5GZbnoTDHgUnukD8sre0htDeTJk1i7ty5RCIRLrvsMsaOHYvb7WbAgAF8+umnDB48mC1btiTP8jge6XTxANB1/T3gvUa2+5qpe2ZHtOm4pRPOYj5RMKsKI3obvYyENyJd19lfE2Tjvlq2lNaxaX8tmw/U8e/NpcRHvbCYFAbluhiaZwyXDc1zMaSnW86ldGGmTZvGihUr+Pvf/87111/PXXfdxQ033MCsWbN49dVXGTZsGJdffvlx/fvrEuIhkRyvCCHonWmnd6Y9uTMejGGvbQd9bD5Qx7eldWw5UJc8Fz6B06IyqIeLQT3cDO7pYnAPF4N7uOmTZZdLiDuZXbt2kZ+fz7x586ivr+eLL77ghhtu4IorruBXv/oV/fr14+GHH+7sZrYrUjwkkk7AZlaTy4ZTqQ1G2Fpax5YDPrYerGPbQR+rtpXxxhclyTpWk8KAXBeDergYmOuMxy76e51yo2MHUVRUxKOPPorZbMblcvH8888DkJWVxYgRI9i4cSOTJ0/u5Fa2L1I8JJIuhMdmZkK/bCb0S1+RUxOIsO2gj20H69ha6mNbmY8v91Sx7Ot9JBxjC2HMqQzwuhiQ62RAriEuA3Nd9HBbj+shlI7mxhtv5MYbb2yybNmyZYfYfL62XSnWFZDicaIxpGtNPEpaRobdzIR+WUzol5VmD4Rj7Cj3sb2snm0Hfewo87GjrJ7VOysJRGLJeg6LSmGOk/65TvrnOOnvdVLoNeIsh1kKSzO053keK1eu5NZbb8Xr9R65chdEnuchkRyHaJpOaV2QHWX1hqCU17OzvJ7i8nr2VAWSe1QAPDYThV4nhTlOCnMc9MtxUuh10Dfbiddl6TRhked5tD/yPA+JRJKGogh6ZdjplWHntEHp32zDUY09VX52ltVTXGGEXRV+1sWHwVJ0BadFpSDbQWGOk345DvrmOOibbYTemXZ5CNcJjBSPE4358434qac6tx2STsNiUhiYa0yyNyYhLLsr/ElR2V3pZ+vBOv69+WByEySAIqB3pj0pJgXZDvpk2emT5aAg206uS86zHM9I8TjR+Pbbzm6BpAtzOGGJaToHaoPsqTQEJRHvrvTzwaZSyn3htPo2s0KfrISg2FPSRpzj7LwhMcmxI8VDIpG0CFUR5Gfayc+0c8qAnEPK/eEoe6sC7Knys6cywJ5KP3uq/OytDvDlnmqq/ZG0+laTYtwvy568b++UfFeYj23Oq25xcTEzZsw4Km+4CxcupLCwkDlz5nDXXXfxwgsvsGDBAhYs6F4Ow6V4SCSSNsFhMTG4p5vBPd1NltcFI+ytDlBSGaAkLip7qwPsrQqwaX/tIT2Xpy/thdhfi1lVMKsCi0lJpo1YwaSIdu+9tIVX3eZ49NFHcTqd7XLv9kaKh0Qi6RDcNjPD8swMy/M0WR6MxNhbHWBfPLjValxWE5GYRjCiUReMojXqjQgh0sSkqXRbCkzCq24qsViMu+++m3/9618IIZg3bx4//OEPKSwsZNasWSxfvhwwnCkOGjQIl8uF3W5vk/Z0JlI8TjTGju3sFkgkTWIzq2nzLZs21VOQ7QBg+ZKnOFi8Ax3DX5imk0zrOmi6Hs9DynFAcQQCYxOlEAJFGIcIefv15/Tr52FWDJFpicuXhFfdVJ566il27tzJunXrMJlMVFZWJss8Hg+rV6/m+eef5/bbb2fZsmXdbniqOaR4NCISjlFbFiC7lxNxPPoPkt50Jd2V+EtfCIGSakwlRVQaC4xh0zEWjOnUBaPsKGvY+a0qDT2VRM+lsj6EpunUh6KYVYFJVVAa9WI++OADbr75Zkwm43Wael7HNddck4zvuOOONvxhdD5SPBqx79tqlv3pK2xOM70HZ9J7SCb5QzLJ6e06PsVEIukGnDVnfpvdKyEikZhOJKYRielEYxoRTScS1YhqOr5QlGhMp7Q2RDimsb2RyJSW+QjHNPZU+gmEo9QEIlT7w5hUBbMiki72U4fLjreVZVI8GpHb183ZNwxn39Yq9n5bzY4vywCwOkz0HpxJ3sAM8gZk0KOvG5OlGzqhu+46I5YnCkpOUIQQqEKgKhzWkaSu69hCLqwmw7VLVEsIjU6VqoAO9aEoE047kyeffJIBYyZjMpmoqaoiIyuLSEzjT08/xy0/WsCyN5YybuJkSmuDmFSBWTF6OCZVdIlVZUeDFI9GOFwqw0/pwfBTewFQVxlk37eGkOzdWs3Or8oBYwevt8BFzwEZ5A3w0LPQg8fbDc5fKCk5ch2JRBKfjFcQAjz29BMgIzUOLCaFYb083Hfnbdx1YDfXXjgNk9nE9XPm8r15N6MIgR6LcuWFZxGLxXjkz89QWhs85HMO1oWo18xsOVCXFBSTosRjoxdjUhrSiugavRgpHo3ZsRxenAkmG5gduC1OhpodDLU4YKAL/6BcSgP9OODL50CNl00rali/3OiiWm06ub1M5PZx0qN/NrmDeuDJdXaJX7REImk7CgsLk3s8zGYzv//944fUURXBPQtu53cP/TJp03Sj5xLVtHgH5w4rAAAgAElEQVSs47KasFtUbGaFqKYTjGhEtWia/7FUFGEIiZoQmaTgNIiOxaRgNbXvyIgUj8Zk9Ycz74FIPYTrIexPSdfjCG2lf2gN/aO1YKpD8+pURAs5GBnIwchAyvYM4qudfdFW1gLFWJV6cqz7yXFV4vX4yMkOkZ2rYHZngiMHHNnxkNMQzN1/GZ9EcjzQ1l51FSGwmASW+JT/XXfdxVtvvcWdd95Jv5z0/R6arhNLCI2WIjrJtDFXE4wY6dThrxynlfys9n2PSK+6x4KuQ8QPwVoI1kCwGgLVxOqrqdgXoOxAjLIyExXVdip8GURiFgAEGhnqfrJNe8g27U7GmaZ9qCIKZke6mDi9TeS9DXZbJigtdFB35plGXFTULj8SiaStkF51W05iEUBCVFRFtOhgMOlVt42prq4mMzPzyBWFAIvTCJ5eSbMK9BgPPVKq6ppObUWAipJ6ykvqqCjJpXLfAHZWhNC1xO10Mtwhspx1ZIYqyNL2kVm/i6yDX2EL7jF6QE22Q00XGWduisDE8wmxmTjWGJKTSCTHDamLAKwd9JlSPBqxZcsWli5dyvTp05k0aVKbzVcIRZCR6yAj18GAcQ27VGMRjapSP1X766ncX0/lvnqqSv3sKumBFh2WrGdzmsnsYSUjSyHTEyHD5SfDVk2muQxL9CDUl4O/wogPfG3EwepDG+IChAKPvp7ee0ntzaT1bLzGsJrSDVeWSSSSdkOKRyP69evHoEGDeO+999i/fz8XX3xxcvNPe6CaFbx9XHj7pHsx1WIatRVBqkv9VJf6qTrgp+agn707A2ypCmFsjsoCsrC7R+Lx2uPBhsdrJyPXjifLhNPiQwlUgL/cEJT68oZ0Ii79xkgHqppppUiZl4n3ZhyNRabRkJrJ0m4/M4lE0vlI8WiEzWbju9/9LsuXL2flypWUlZVx9dVX4/E07Y+nvVBUhcweDjJ7OGB0elliF3zNwQDVB/3UlAeoLQtQurOGbWsPoqes0lAUgSvbijvHgyenJ2OfvAOzRaX2yRdxZ9twZlpRTfH5klgE/JWNxKXi0Hz5Vqj/2Kh7iCuIOFZPXHC8zczZ5DT0apxeo75clSaRdBukeDSBoiicc8455OXl8fbbb/PUU08xa9YsCgoKOrtpAJgtKjn5LnLymzhzIabhqwxRWx4wQkWQuoogdRUBdn1TwdCdBwgAb/9unXGBAKfHgivbhjvbhivLiivLgysrF1eWDVcfK3aPpWm/P1oMAtXp4uKvaBCcxDBa7V44sN5Ix0JNP5RibrTqrNEKtITNnmK3OKXgSNqV5lyyH44zzzyTxx57jIkTjzjn3G4k2rl9+3ZmzpzJtm3b8Pl8R76wFUjxOAwjR47E6/Xy8ssvs2TJEi6++GLGjx/f2c06LKqqkJFrDFs1hf5pBrGIzqW3j6WuIoivMkhdVQhfZZCy3XXs/LqcWERLu0ZRBI4MC85MK65MK86UYOQLcOQNxGI7wp+TrkPYZ4hKUmRSejaBynjPpwIOboqXHaZ3o1pTRCZVWJpKZxmxLUMKjqRVtKdL9vYm0XaX69AvmseKFI8j0LNnT+bPn89rr73GO++8w86dO7ngggva5ZfREQghMFkEBcOymyzXdZ1gfQRfVQhfVYj6KkNc6quNULm/nt2bKokEY4dca7aqhqhkWHBkWA3B8RhxQ9qGNbMfIquwZQ3WYsYy6ISoNA5JwamMz91UxOdumhEcoYI9K0VUGolLsqxRXu69kdDgkl3TNG677Tb+85//0L9/fzRNY+7cuVx55ZVp9d9//33uv/9+QqEQAwcO5C9/+Qsul4sHHniAd999l0AgwKmnnsqTTz6JEIL/+Z//YfHixZhMJkaMGMErr7xCZWUlc+fOZceOHTgcDp566inGjBnDwoUL2b17Nzt27GD37t3cfvvt/Pd//3daO9sTKR4twOFwcN1117FixQpWrlzJ1q1bueCCCxg7duxxt3tcCIHdZcHuspBb0PShPgDhYDQpKPU14Xgcwl8Tpr4mROnOGvw1YaKNejEAiipweCw4PBbs8djhbkjb3Ym8GZvDjEhspGRQyx5C04yVZv7KBnFpLq7eA/u/MtLRQPP3NNnSBcWeGY+PEOTQWptQ/e52wvuaWap+lFh6O8m8ZGCrrkm4ZH/zzTcpLi5m/fr1HDx4kOHDhzN37ty0uuXl5fzyl7/kgw8+wOl08vDDD/O73/2O++67j9tuu4377rsPgOuvv55ly5ZxySWX8NBDD7Fz506sVivV1cZqyfvvv59x48bx9ttv8+9//5sbbrgh2RPavHkzy5cvp66ujqFDh3LLLbdgNpubdB3f1kjxaCGqqnLWWWcxatQo3n33Xf72t7/x1VdfMWPGDLxeb2c3r+Wcc06b3MZiM2HJM5GV1/wpaLquEwnG8NeG04QlUBfGX2uE+uoQZbvrCNRF0ib6EwhFYHeZsbvN2N2GsBh5i2FzWbAlyl0WrA4TQlEadu63hkjA6LWkiUxVSqg05ngCVVCxvUGgmpvHAVBMDUJiy2wQHVtmgwgl7I1j2dvpsqxatYqrrroKRVHIy8vjrLPOOqTOp59+ysaNGznttNMACIfDTJkyBYDly5fzyCOP4Pf7qaysZOTIkVxyySWMGTOGa6+9lssuu4zLLrss+VlvvPEGAGeffTYVFRXU1NQAcPHFF2O1WrFarfTo0YPS0lL69OnTET+ClouHEOIBjP1vXwJf6rq+td1a1YXJzc1lzpw5rFu3jvfff58nnniCadOmcdppp7Xrkt424957O+yjhBBY7CYsdhOZPR2HratrOkF/hEBtBH9dOCkwgdowAV+EQNx2sLiWQF2YcBPDZmCIjc1pwuYyRMYWD3ZnPHaZsToTomPUs9hUowdpthvB07t1D5oQnYTwBKtTBCclHawGXymUbTHsoZrD31e1Ni8utox4OiPFntFQZnUfVz2e1vYQ2puWeObQdZ3zzjuPl19+Oc0eDAa59dZbWbNmDQUFBSxcuJBg0HCY+Pe//50VK1bwzjvv8OCDD/LNN980+VmJEQ+rtWFLoKqqRKPRY3msVnHEt50QwqbrelDX9fuEED2BccBMIcRAXdfntX8Tux6KojBhwgSGDBnCP//5T5YvX866deuYNm0aJ510EqoqN9S1FqOHYQyXZXPkM52jkRhBX4RAXYSAL0ygLmLkfYbYhHwRAr4IVQf8BH1hgvXRJns2YCwIsDpN2JxmbE5DXGwuMzaHCZvLjNVhjpeZjDKnGavDhNl6jKKTmM9JCEuguiFO2II1Dfa6/VC2OW6vpdl5HeMHaix/TopKakixWT1NlHvA4m65y5sTkNNPP53nnnuOG2+8kbKyMoqKipg9e3ZanVNOOYUf/OAHbNu2jUGDBuH3+ykpKaFHD8P3hNfrxefz8frrr3PllVeiaRp79uzhrLPO4vTTT+evf/0rPp+PadOm8dJLL3HvvfdSVFSE1+vt8K0DTdGSr8qfCyHeB57QdX0b8M94OOFxu91cddVVjBs3jg8//JB33nmHlStXMm3aNMaMGdM1ReTCC434H//o3HYcIyaziitLxZXVMlcruqYTCkQJ+iIE61OCL3KIra4iSPmeOoK+SJNzNgkUVWB1mJJiYk3EDiO2ORL5BlsiNlkUhKIe3fAaGPM6odpDBSaYYkuEQDxfvq3B1pyrmyQiRVg8KUJzmLQ1XtfqMXo+x/F8z8yZM/nwww8ZNWoUQ4YM4eSTTyYjIyOtTm5uLkuWLOGaa64hFDKGNn/5y18yZMgQ5s2bx+jRoyksLGTSpEmAcRb6ddddR01NDbquc8cdd5CZmcnChQv53ve+x5gxY3A4HDz33HMd/rxNcUTHiEIIBbgYmA8owBPA3/U29KgohJgO/AFjWOxpXdcfalT+/4DvA1GgDJir6/quw92zQxwjpqDrOt9++y1FRUXs37+f7Oxspk2bxujRo7uWiEjHiK0iGo4RrI8S8hvCEqqPEvQbghPyJ+xGHPJHjTr+KOHA4YcPEsKTFBW7CUs8tjqMoT6rw2zYEzZbQ5nJohzbYo1YJF1gQrXp+WBtii0ehxqV6c0LK2CsbLO6GwmLO0VsPCnljfNuNu2rZ/iI4UYvqpMoLi5mxowZSffrqfh8PlwuFxUVFUyePJmPPvqIvLy8TmjlkXG5XE3u82hvx4gZwDfAL4AxwCPAn4DCFlx7RIQQKvBn4DygBKOn846u6xtTqq0DJuq67hdC3BJvw6y2+PzG+NesYd+P70b1ejF5vZhycjDlelFzcjDleDHlxu1eL4qzYXhFCMHQoUMZMmQIW7ZsoaioiLfffpuioiImTpzIuHHjcDqPPBwj6VqYLCoui4orq3Xu5jRNJxxoEJVQfZRQat7fkA4HjLLaimDSpsUO/91MURLzSSpWhxmLXTXEJS42FrshNha7aohPEzazPQfhPMrFHrpuHFOQFJ4UsUnLx+NQnZGu3QuhzQ127TAie8GrsD8MCMO3mlCNoTSRSKuGsKTFzaWVeGid4B7OJfuMGTOorq4mHA5z7733dknhSGwS7NmzZ5vfuyXiUQF8AnwE1AFPAbVt2IbJwDZd13cACCFeAb4DJMVD1/XlKfU/Ba5rw89PQ3E6sU+cQKy8gkhJCYGvviJWWWn8Y2mEsNsPFRhvLj29OcweOJCdffuydvduPvjgA5YvX86IESOYNGkSBQUFx90SX0k6iiKScyitRdd1ohGNcCAuLH5DXBLphNgky+NxbXkgno4RDkYPOyUCgIivmrOpcWExBMicEJi0MhNmW4PNnCiz9cCc0wtVPYrega5DNNggLKFUwakDkQXuXsbckB6Lx5qRjoUhmmI74sMmnrmxoKQIUqrIxNMFXjd7tn5j5COBtPpF3aD33p4bHFsiHhOBH2J4WHoaeEvXj9RfbRX5wJ6UfAlw8mHq/xfQbgP2tuHDyX/kkTSbHo0Sq6oiWl5OtLyCaEU5sUS6vJxoeTnh4mKia9YSq2pwLmgBpgAjPB52DBvK5lCI9evXkxWNMsJkYmh2Du4euQ29Gm8OppyctB6N5MRDCIHZomK2qDgzjs7Btq7pREKxBpEJxuJxQnRiDelgQ3mwPkJtRZBwMEokGCMSanpVW2NUkxIXlwZhMVsTQqNisRrikxAgszVe16pittmw2JyYHQWYsw2bogjYtAncLfg2r+vxkBAZLT1OS2sNYpRIRyPp9hajxIWkkQgJJT2dLFOa6Q2pDb2ibvSl8ojioev6F8D3hBDZwDxghRDiPV3Xf91GbWjqp9Xk1wghxHUYYnZGM+XzMeZm6Nu3bxs1D4TJhCk3F1MLdm3qkQjRyiqi5WXEKiqIlpWTW1FB/4pyAmXlfBsOsdlu5yOTiY/Ly+ix8Rv67t5NfslerOGw8Xl2O6bsbFRvDqbsHEzeHNRsQ1jUnGwjzo7HmZmI1sypzJhxtD8GSTdCKA3LpI8FLS5CkWCD4ESCsQbBSclHgjHCIaNeJBQl6AtTWx6/thVCBGAyK4y/JoOKvT6EEPF3a0qcSCuikV1FKCYjrSbKG+odkSaFSGvU82ksTlpDmRY9tF6LESk9n1RRaU6AGotPyrWKyQjtSEsmzIswToFwGE+HBoR0XW8TJ09CiCnAQl3XL4jnfwqg6/pvGtU7F/gjcIau6wePdN+OnjBvDbquU1payoavv+abDRuoqq1FEYK+LheDTCb6BIKYKyuIVVQSragwRKiyEmJN/CEKgZqVhZqdhSk7Li5Z2UacnY2alR0vy0bNzkbNyGid2EgkbYSu6UTCsQaxCcWSvZtwqKGnkxAaRz8/AwsHo2s6um4Ima7rxjtZjx+72splO4eKTYpNNBabRkIlRPz9nlKWkm76ofUUIdEOFRZdSxehpupo2qHXHAmHFzKP7Mi1vSfM5wDVQE1brrBK4XNgsBCiP7AX+C6QtmBaCDEOeBKY3hLh6OoIIcjLyyMvL49zzjuP/fv3880337BhwwY+iA979R40iEHTBzF48GAK8/MRQKymhlhlXFAq04UlVlFJtKqS0OYt+CsridU0swFNCNSMDENIsrMwZWU1CExWlmHPzELNysKUlYmalYWw2+UcjeSYEYqIz5mYcLbgvLtNmzbh8R5+l70hJnr8HR2PW2nTYjq6rjWIUjP7gQ7/cKli01hgGuwIY4n2IWXNiZNIsSf+DTYWpKZEx9T+5wm2ZNiquD0boOt6VAhxG/AvjKW6z+q6/k18R/saXdffAR7F6P28Fv8B7tZ1/dL2bFdHIYSgd+/e9O7dm3PPPZd9+/axbds2tm3bxsqVK1mxYgU2m42BAwcyYMAA+vbti3fAgCO+zPVIhGhVFbGqKkNoKiuJVVbheuAX6Af2U3byycQqKwnt3Ensi3XGXI3W9DcaYbEYvZusLNTMTNSsTENoMjObDYrbLQVH0u4Yw1Nt/3eWEJGdO4sZNXokQ4YM5fPP1jB46CC2bNya7PWkC1GiR5QiTJqOHjNspNRtirq6Wk4/dzIXXTCD3zzwGACXz7qY0rID2Kx2EPDaS2/TI7cHD//u1/Qt6Me1372O+x74Ga++/jI//MHt/Oi220EILDYVazufx9Yl/Gnouv4e8F4j230p6XM7vFGdgBCC/Px88vPzOeOMM/D7/ezYsSMpJt988w1gOGrs168fffv2pV+/fvTs2fOQvSTCbMbcowfmHj3SP+SZpwHo84ffp5l1TTN6NtXVxKqqiVXHhaeqimhllWGvriZWVUVo02b81dVG76a5zqiqGj2cjIwGUUmmM9LKlIwM1IxM1AwPisslRUfS6SRESTUpDBw4kK+++jJu56hW0KWSFB49XXgeeORnTDvjDCx2E54cO7quo5oV/vLMc4wbOz6+qMy4RlEEiSN2fnHvr7DbHcSiGqFANDmqZXUcWzuPRJcQD0nTOBwORo0axahRo9B1ncrKSnbt2pUMmzZtAsBsNpOXl0d+fj69e/cmPz+f7OzsVr2EhaJgyjKGrujfsmv0WIxYbW2DsFRXE6uuaUjXNOQjBw4Q3LyZWHU1euAw3mtVFdXjQfV4UDIzUD0ZRj4jAyXD05DPNGIlXlf1eBAOhxQeSbuScHVeVFTEfffdR05ODlu2bGHatGksWrQIRVF45plnePjhh+nduzeDBw/GarXypz/9ie3bt3PttdcSi8W48MIL+d3vfpfcuLd27VrKK8q48MLprFmzBpvLePErquG2x5WZ7knB2zOLnLzMpGNSh9uC02XF26d5T9htjRSPboIQgpycHHJycpIHUtXU1LB7925KSkrYu3cva9asSTpGs9ls9OrVi549e9KjRw969uxJbm4ubdmTFaraIDitQAuH0WpqGno6ybiWWK1h12pqDXtVFeFdu4z6dXXNDq0BYDKhut1poqJ43Khuj9GrcXtQPW6UeB3V7UbxGL0d1eNBWK1SfLoo//jHPzhw4ECb3jMvL48LE+56Wkiqq/PVq1ezceNG+vXrx/Tp03nzzTc59dRTefDBB/niiy9wu92cffbZnHTSSQD86Ec/4kc/+hHXXHMNixcvTt5H0zTuvPNOXnjhBT788MNDPvN73/seqqoyc+ZMfv7znyOEYMGCBUf51G2HFI9uTEZGBqNHj2b0aOOQ81gsRllZGXv37mXfvn3s378/TVCEEPzX3r1YzGbWf/ghXq83GVK9c7Y3isWC0sKlz6nomobm8xm9nZoatNpaQ3DqahvStXF7bR2xuloi+/YZ9WtrIRI57P2F2WwIS1xUVLcLxeVGcbtQ3Z547E6xGWmjngvF7UaxtPNAs6TLMHnyZAYMGADANddcw6pVqzCZTJxxxhlkZxv+yq666iq+/fZbAD755BPefvttAGbPnp0UgEWLFnHRRRc1ecz1Sy+9RH5+PnV1dcycOZMXXniBG264oSMe74hI8TiOUFU1uYprwoQJgPGtpqqqitLSUkpLSzmwaxd1dXWsWrUqbeLO7XaTk5NDdnY22dnZZGVlJWObrWXOB9sboSjJISpaeWaBruvowSCxujq0ujpitbXxuM4QnDofms/Ia3V1Rr3aWiKlB428z4fu9x+5jRaLISIuJ6rLHRcVF6ozLjAuV7wsnnY2yieCFKE0WttD6Aga91KFEC1y1d6YTz75hJUrV7Jo0SJ8Ph/hcBiXy8VDDz1Efn4+YPz7nD17NqtXr5biIekYFEVJDneNGDEC4ofWTI1Gqaqqory8PC1s3rwZf6OXpMPhIDMzk4yMjEPijIwM7N1gKa8QAmG3o9jt0HgRQQvRo1Gj5+PzNQhMIu3zNQhQMm3UjewpIVRXR6y+Hs3na3q/TmPMZlSnEyURXK54bOTTypyulHQiOFAc8bTDbhyQJWlTVq9ezc6dO+nXrx9Lly5l/vz5TJ48mTvuuIOqqircbjdvvPFGcmTglFNO4Y033mDWrFm88soryfu89NJLyfSSJUtYs2YNDz30ENFolOrqarxeL5FIhGXLlnHuuV1n7ZAUjxONuDCYHA5yc3ObPOs4GAxSVVVFVVUVlZWVVFVVUV1dzcGDB9m6deshB86YTCbcbjcejwePx5NMu1wu3G43LpcLl8vVoUNj7YEwmZIrx46WRA8oKUI+H1pcVGLJdD2ar85I19cT88XjqioiJSUN17SgJ2Q0XKDY7XEhcaTHzoa8cDjSyx1xAXI44mLUEITN1uW/MLQ3U6ZM4Sc/+Qnr169n2rRpXH755SiKwj333MPJJ59M7969GTFiRNJV++9//3uuu+46fvvb33LxxRcf4sK9MaFQiAsuuIBIJEIsFuPcc89l3ryuc4SSFI8TjYsuMuLDOHVLTLb36tXrkDJd1/H7/VRXV1NTU0NNTQ11dXXU1tZSW1vLnj17qKurI9bEt2uLxYLL5cLpdDYZHA5HWugWJzO2ktQeUGvnfBqjaxqaPxAXGV9SbBLCkpZPsxlxpOwg+q5AQ5nf3/zS60MfBMVuRzhTRMZuN9LxWDgS+UQd47mF3R4XJSNv2FLKu8nv3eFwsHTp0kPss2fPZv78+USjUS6//HLOP/98APLz8/n0008RQvDKK68wceKhm7jnzJnDnDlzAHA6naxdu7Zdn+FY6B6/JUmXQQiRfNknxmMbkxAYn8+XDHV1dcl0fX09lZWV7NmzB7/f3+w4scViSQqJ3W5PhtS8zWZLhkTebDafEN+KhaKgupyoLidwdENxqSR7RSliYghOaroeLWCkdX8gvV4gQKy2lmjpgbRr9LjPthY/l9mMcDiIPPQQQdWEUETcX5NiDL/FfTql2YUC8XzzdeJp0UI/VxzeJXtzLFy4kA8++IBgMMj555+fPIt87dq13Hbbbei6TmZmJs8++2yrfi7Ncdddd/HWW29x5513tsn9WsoRfVt1V7qyb6tOpYsdBqVpGn6/n/r6evx+f5MhEAgkg9/vT5733ByKomCz2bBarUlhSaStVmuzwWKxpMUmk+mEEKH2Ro9G0YLBuOAYIqMFAkavKeBHT6bT8+XnnMOQgj6gaeha3AeUphnuQ3Stwd5qRIOopApMXIAaxCeeTrWLFIESzQhT6vVd/O+nvX1bSSTthqIoyTmRlqJpGsFgkEAgQDAYPCSdGkKhEMFgkPr6+mQ+cSRoS9pmsViaDWazudl0Ip9IJ/Imkwmz2Yyqql3+xdJWCJMJ1eVCbcXvGKBq0yYsTSxfTUVPeMFNCElqOp5PE554ebPpWAw9Ejm07Gi/ZItGoiOaEKpDyhrlE2nRSPCauq4DBUuKh6TboShKcjjraNA0jUgkkhSSYDBIOBwmFAo1GTcOtbW1yXQkEiEcDrd6iaYQIk1YEqLSVDqRbyrdlE1V1UPKE/YudSRyC9B1/bAvw9SXanu+MtOEqSnRSRWpZDqlh9RUvZiWLlSJsmMdDYp7izA3MWeZ9kzH+DlSPE404pNxJzKKoiSHqtoCXdeJRqNJIYlEIk2mE3WaCqll4XCY+vr6pD1RFo1Gj/kfvBDiEEE5Utycran8kezNBUVRUJT0c9ltNhsVFRXk5OR0ei8tudRZVdtVpKBRbypFWJoUobR8Qqx0FMeRvRFXVFQc0x4uKR4nGlI82pzUXsTR9oZaSiwWSxOV1oRYLNasLTUOh8NN2lPT7TVXmiooVquV0aNH42rCWWZqPpFuyna4+sdSdrh6TeU7nOoq2LfvsFVsNht9WrnZNhUpHica5eVG7PV2bjskR0VXGXrSNO0QQWkqNFWmaRrRaDTtHk0FTdOoq6ujuro6mU8ta1y3uTqJuKNJ9KYSIbWHdThba8qbCwmnjO2JFI8TjSuvNOIustpK0j1JvKTM5vZ1+92WNBYTTdOatB2urLG9pfnW2hPieriQ6AE2tgNMmjRJiodEIpG0BQnBO97REnMj7YwUD4lEIjmO6CiBPP5lWCKRSCRtjhQPiUQikbQaOWx1onHLLZ3dAonkuEDXdWKRCNFImFgkQiwaIRo24lgkkrTFIhGiabYosWgELRo10im2NHs8aNFISjpKLBZNq6PFomjRmHFtLIYWizL67PM584b29cArxeNEY9aszm6BRNLmJF7kkXCIaChEJBQiGjaCkQ7H8+GUsnCaPRox8rF4HI2EiYUjRnmKSETD4aQotBWKqqKoJlSzCdVkRjGZUE0mVNUUT5tRTCqqyYTFZkc1mYxrTIarG0U16ismI50/dGSbta05pHicaOzZY8RH8BkkkbQHuqYRCQUJBwKEg0EiwQCRYJBwKEA4YKQjwQCRUIhwMEAkFIzbgkY6FGqwxUUgEgwSCYeOyq2HEAomiyUerJgsZkxmI61azFidThyZWZjMZkwWC6rZKFctFkxmM6opbjeZUS1mTCYzaiJvNvKKyRS/zpwUBpPZKE+IhKJ0/t6d1iLF40Tj+uuNWO7zkLQCXdMIBfyE/X5C/vp48BP21xMKBAj56wkH/PEQSKZD/oAhAgE/oYAhBi19yQtFwWK3Y7baMNvsmK1WzFYbNqcLd7YXs9WKyUX27PgAAB51SURBVGpN2k0WK2ZbPLZaMVksmC1GHUMYLHG7NSkYiio9Jx8tUjwkkhMETYsR8vsJ+uoI1tUZsa+OYL2PYL2PUL2PoK/eSPt9hOrjIlFfTyhw5IOihKJgdTix2B1Y7XYsDgcOj4eMnnlG3m7HbHNgsdux2BJ5OxabDbPNhsVmj4uBDbPFbngeJn4uuI7x+fFYb5Qn4fg21UYTNh0Igu7XiREipodSro8/X9o1TdgTaWhoGxiOEBvVM6KUOo3uoScqpZY3qq83vr6Z++nJ/4G5lxPbwKM/8bIlSPGQSI4Rw1kdDWdNxOJO6jQdPRaPD5ePX9/42rR6esJm1NNjMSLBEJFAkGhiWCcYJhoKEQuHiYYixMJhYpEosbAxCatFY4jEf0IgUOKx8Z9TteBRnaim3iiKCdViQrGrKLkmY4OdUFEUFUUYhy8JIYw4fj/0+Dd4TW94wWs61AG1KS98jRQBiIAeIabXEoCGl6LkmHCe0kuKh+TExXiZaugRDT2mx+N4PqpBVDfyMR2ihk2P24jpRj4Wv0eqPXFNIh1NvMgT5fGXdrSRGCSuSxWBeNzZmAGTbkZDBezGt1UR/9ZqFQgb8fMjjDMkhKokY8WkIFQ1fihSXAiEUd84j4K423OBEA1pBPFzJYifIxEvS+STZaTfN+X+yWvg0PsojfLQ8LmN75WwkWKD9DbQqG6qTWlog0i5Pu1eIvVezZQl7I2fh6avQ4hkMw69tqF9DXVSr210TWrS1P67MKR4SFqNrsVf5KEYWjiGngwaejiGFtEa8pGYUTccjyMptrgIpKWjKeIQa8OXsiIQqgBVQZhS0mr8pWhKTQvjZarGX4CqQKhKyj0SdqWhPDVO1Iu/rDU0Qn4fwYAxJBT01xGoryPgqyXgqyHgq8NfV0PI70NDQ9c1dHQ0XUNHw2yzY3M7sbo92D1ubG63EXs82D0ZODwebBkenBmZ2NweTN3I35Sk+yLF4wRDv/0OtHAM7aAfLRBFC0bR47EWiKGHomjBmCEMwUQ62iAUIUMUWoUCwqwizEpKUI0XtkVFcZoNm0k5JMYUzyeDSLEnXvqK8eJO1FHF/2/vzqPjqu4Ej39v7VUqlVTabS2WvILxgm1hE4IJYLOHNoeGhuEkgRCGGZIhYTKZPumB7iE06SbJyZmETiBhQg4EuhOWHpbuQBIWO5gG22BjvO+2rH0p7aoq1aI7f7zSYlkyKleVqsr6fc6559333n1Vv6snvZ/ebsyPTRveiKdCNBKmv9NHT0c7fR3t9Pk6jNLZQb/PR19nB4HentOWM1ss5HgLyfF6cc8uoDR/ETl5Xlz5XnJixZWXh8uTj8VmS0nsQiRCkkcW01qjAxGi/WGG+sNEB0IMDcTq/WGG/GGG/BFjWiDCkD+MDhUYC+/YPvGHmhUmhxllt2Cym1EOM2aPHWU3G+M28+l1q2mkrqwmTDYzyhabZo1t3LNUJBSit6ON3rZWetpb6Wk36r3tbfT62hno7jrtRLIj10NuQSG5hUWUzV+Au6AQd0Ehud5CcgoKcXsLcLhz5SofkdUkeWQoPaQZ6gsR6R4k2h0k2j1ItCdEtC9EtHd0SGTivQCTy4Ipx4rJZcWcb8c6KweTy4rFV4dymFHnnYfJYUE5LZgcFkxOi5E0rNl3vXmiAv19dLc00d3aQk9LM92tLXS3NtPT2kx/V+cpbU1mC56iYjzFxVQvX2nUi0rILSrGU1SMu7AIqy05bygUIpNJ8kgjHRki0hkk0hEYLb4Aka5Boj2Dpx3zV3YzZo8Nc64Ne1UuJo8Nc64dc64Vk9uKKceG2W0kDGWe5L/ay+82hjPsPo9waJDu5ia6mhvpig07Y/VgX+8pbXO8BeSXljFn2QrySsrIKynFU1xCXkkZOV5vVt7QJUSyZUTyUEpdC/wUMAO/0lo/Nm6+HfgNsArwAbdprU9Md5xnSw9pIr4A4eaB0dLmJ9oVPOXSRJPLgqXIia0yF8vSIsxeO+Z8B5Z8O2avHZM9I1ZXRgsF/Pga6vE1nMTXWE9nYz2+xnp62lpPObzkLijEO6uchWsuwVs2m/yy2eSXlpFXWobVfvbvdRZipkj71kgpZQZ+DlwFNAAfKaVe11rvG9Psa0CX1nq+Uup24AdARj6kSWtNtGuQwbpeQnW9hBr6iLT60eHY4SUTWIpd2CrcWFaUYClyYi1yYil0YHLJVTJTFY1E6GpqoKO+jo76OtpPnqDjZB297a0jbcwWC95Z5ZTWzGfx2isomF1BQXkl+WWzsDmcaYxeiOyX9uQBrAaOaK2PASilfgdsAMYmjw3Aw7H6y8DPlFJKT8frsj6DHtKEmwcYPNZDqK6Hwbo+hvpCgHGYyVbhJmfNLKyzcrCW5WAtdRlXAIkpCweDtNUdp+3EUdpPHKPtxDE66utGHkynTCYKZlcwa/5Cll55NUWVcyisqCSvpAxTBrzvW8xM0aEog9FBwkNhQtEQoaGQMR4dHQ9FQ6Pzx00LR8OEhkbnDw8jQ5HRNmPmDS8THgpzdfXV3LP0npT2LxOSRzlQP2a8AVgzWRutdUQp1QMUAh3TEuE4Q6Eog4e7CR7oJHCgcyRZmL12HPPysFV7sFV5sJblpOwS0XNVJBSive44LUcP0XLkEC1HD9PZ3DhyyMnhzqWkei4rrr2R4jk1FFXOoaC8Uu5tEKfRWhMZihCMBhmMDhKMGMNTSmS0HowGCUVDI+OhaIhgZHTayLwhIwGcMi2WFAaHBkcSQVRHk9YXq8mKzWzDarKeWjdbT5nmtrmxmqx4bJ6kffdkMiF5TLR1Hb9HMZU2KKXuBe4FqKqqSjyyMYb8Yfy72gns62TwWDdENMpuxrHQi+O8AhwL8jF7suAqm4ceSncEI7TW9LS10nRoP00H99Ny9BDtdScYikYAcOXlUzZvAYsuWUtJ9TxKauaSW1gsl7ieA8LRMP6In2AkSCASIBg1hoFIgGAkaJRJpg0ngfHjgUhgZEM+XB/Scd6TNIbFZMFutmM327GZbaNDkzF0WVzk2/NPmWcz2YzhcDGNzhvewA+3HxmPLWM1W0frY+ZZTJn58MZMSB4NwNjng1cATZO0aVBKWYA8oHNcG7TWTwFPAdTW1iZ8SEtrzeCxHvwfteDf0wERjaXQgfvi2TjOK8Be7cm+Q1Dr16ftq6ORCG3Hj9J4cB9NB/fTdGi/cZ8EYHM6KZu3gFVfvImyeQsom7eQ3MKijPyjmSm01oSGQvjDfgbCA/gjfvxhP/6In0A4cMr42GnDG3x/eLQ+tgQjQSI6Enc8DrMDp8WJ3WIfrZvt2C128u352C3Ghn5kutmOw+IYqY8dd5gd2Mw2HJbY0Ow4JUnYzXbMclXdGWVC8vgIWKCUqgEagduBO8a1eR24E/gQuAV4N5XnO6J9Ifw7Whn4qJVIRwDlMJNTW0bORWXYyt2p+trpsXOnMbzwwpR/VTQSofXYEer37aZh324aD+wzHskN5JWUUrX0QmYvPJ/ZC8+jqGqOXAKbBFprgtEgA+EB+kP99IeNMhAeOKX0h/pHEsJAeID+cP9IkhieHggH4trIOy3OU4rL6sJpcZLvyDfGLa7T2jgsDhwWIxE4zU6cVudIAnBanDjMjpENvvwjkVnSnjxi5zD+G/BHjEt1f6213quUegT4WGv9OvA08JxS6gjGHsftqYonsM+H7/n9MKSxVXvwXlGJc2kRJts5smF74AFjmIL7PLTWdNTXceLTHZzc86mRLIIBAAorqrjg8nVULl7K7EWLcXsLkv7954JQNERvqJfeUC99oT76Q/30hfroC/edMt4f7jfq4b6RJNEX6mMgPDClY+0WZSHHlkOOJQeX1UWONYdcWy5lOWW4LK6RaTnWnJEN/3A9x2osM9zOZXHhsDgwqSzbCxcJSXvyANBavwG8MW7a342pB4FbpyMW2xwP7ktnk1NbhrXENR1fmdX8vT3U7d5J3aefULdrx8gd2QXllSy+7EqqLlhKxflLcOWl9vHQmSQcDdMT6qF3sJeeUA89g0YZTgq9g72n1ftCffSGehmMDp7xs83KjNvmxm11k2vLxW11Mytn1inTcqw5uK1ucmyxYWzcbXXjsrpw29zYTDb5T14kJCOSRyYx51jJv35uusPIWFprfPV1HN2+jaMfb6X56CHQGoc7l6qlF1K9fAXVy1aSW1iU7lCTIhAJ0B3spmuwa3Q42G2UYDc9gz0j48N1f8Q/6ecpFG6bG4/NYxS7h3mueXhsHnJtueTack+rDyeFXFsuTotTNvoiI0jyEJ8pGonQeGAvRz/eytHtW427tYGy+Qu55NY7qF6+ktK587PinEUgEqAz2ElnoNMYBjvxBX10BjvpCnbRFewy6rFkEYwGJ/wchSLXlku+PZ98ez5FziLm588nz55Hvj2fPHueUWzGcDhRuK1uORErzgmSPMSEhqJRTu7dxcEPNnNk2wcEB/oxW63MWXohq2+6lbkrV2fMeYvB6CAdgY7R4u/AF/ThC/joCIzWfUEfgUhgws9wWpx47V68Di+FzkIWeBeMjHsd3pEkke/Ix2v34rF5JAmIGU2Sx0zzD/8w6ayhoSgN+/Zy8MP3OLz1AwJ9vdicTubVXsyC1Z+jetlKrI7pe+5TOBqmLdBGu7+dVn8r7f72kfF2fzvtAaP0hfpOW1ahyLfnU+gspNBRyLLiZRQ4CkbGh+sFjgK8Di9OizyuRIh4SPKYaS655LRJvoaT7Nn0Nvs3b2SguwuL3c68VWtY9LlLqbmwNiUvIxqMDtI60ErLQAst/hZaBlpoHWilzd9Gq7+VVn8rncHTbuXBYrJQ4iyh2FXM3Ly5rC5bTZGziGJXMUXOopFS4CjAYpJfbyFSRf66ZpoPPgAguHwZBz94jz2b3qblyCFMZjM1Ky7i/EsvZ+7K2oSeLKu1pmewh6aBJpr7m2kaaKKpv4nmgWaaB5ppGWiZMDF47V5KXCWUuEq4oOgCSlwllLpKKXGVUOwspsRVQr49X04YC5EBJHnMIFprQg98i/6uTp5fs5xIOERRVTWXf+Uezr/08rgup/WH/dT31dPY3zha+hpp6G+gqb/ptCuOnBYns3NmU+YuY3HhYspcZZTljJZSVykOizwKXYhsIcljBggHg+x/fxOf/PHfufLEMUxmM0uuvIoll19FSc28Sf+T7xns4WTvSU70nqChr4H6vvqR4gv6Tmnrsrgozy2nIreCNbPWUO4uZ3bObGa5ZzE7ZzZ59jzZYxDiHCLJ4xzW3dLMzj/9O3s2vs2gf4DiOTV4Z1fg8uRRfvd9gHHu4WTvSY73HOd4z3FO9hnJ4mTvSboHu0c+S6EozSmlMreSyyouozK3ksrcSipyKyh3l8vhJCFmGEke56DmIwfZ+spLHN2+FZPJxILVl7Bo3ZUMlFoJ/HkjXQONPPbONzjec5zG/sZTnjxa4iqh2lPN+jnrqfZUU5VbxRzPHMpzy7Gbs+CpwUKIaSHJ4xyhtaZ+724+fOW3NOzZjclpY2hNJcdqwvw+/BatO54H4Nc9xzEpE60DrSwuXMwNc2+gxlNDTV4NczxzcFnlkSxCiM8mySNLaa1pGWhhv28/+z7aTN97u7G1BvHbo+w9r5eDVX1Y7PXUmGu4qPAi5uXPY37+fCrnBSl2FfPyylXp7oIQIotJ8sgCQ3qIut469vv2c6DzAPs793PAdwBXY5BVB/Mp7LUTzdF0ri1h9sUr+VrxeSz0LqTCXXH6XdCVE3+HEELEQ5JHhhneo9jj28Oejj3s7djLXt9e+sP9gPE6yuXRGr64uwJLQx+OQi+X/Jc7WHbZVZgtU1idb79tDNP4UighRPaT5JFmwUiQvb697Gzbyaftn7KrfdfIZbAWk4WF3oVcX3M9S4qWUBUtpvHN9zm85X2cuXYuvutelq2/Lr73dz/6qDGU5CGESIAkj2nWEehge+t2Pmn7hJ1tOznYeXDkbW1zPHP4fPnnWVK0hCWFS1hUsAib2UZwoJ8PXvxn3nvrGUwWCxf/5e3UfvFm7C45uS2ESA9JHinW1N/E9tbtI+VE7wnAuON6SdES7lpyF8uLl488uG8srTX7N29k03NPE+jtZem6q/ncLXdkzNNshRAzlySPJOsMdrKteRtbmrewpXkLjf2NAORac1lZupKbF9zMqtJVnF94PlbT5IebOpsaeOfpJzi5Zxdl8xdy8998j9KaedPVDSGEOCNJHgkajA6yvWU7HzZ/yJbmLRzoPAAYyaK2rJYvL/4ytaW1zM+fP6X3P0RCIba++hIfvfYSFpud9fd8naXrrsmKFy0JIWYOSR5nobG/kc0Nm3m/8X22tWwjEAlgNVlZUbKC+1fcz8WzLmZx4eK4HwneeGAff3ji/9Dd2sz5a6/gC1+6m5x8b3KD/+Uvk/t5QogZSZLHFESHonza/invnnyXzY2bOdZzDIAKdwU3zb+JteVrqS2rPesXCkUjEbb862/Z+spLeEpKuPVvv0/VkuXJ7MKoRYtS87lCiBlFksckQtEQW5u38s7Jd9hYv5HOYCcWk4WLSi/iloW3cGn5pVR7qhN+GGBXcyNv/OzHtBw5xAWXr+fKu+7F5kzhVVT/9m/G8MYbU/cdQohzniSPcQ52HuTp3U/zXuN7DIQHcFlcrK1Yy7qqdawtX4vb5k7K92it2bPxLTY+8xRmi4Ub//t3WXjxpUn57DP68Y+NoSQPIUQCJHmME4qG2NqylWuqr2Fd1TrWzFqT9KfJBvr7+NMvHufIRx9StWQZ13792+QWFiX1O4QQIpUkeYyzpGgJ79767pSujDobHfV1vPajR+ntaOcLX7qbVTfchDKZUvJdQgiRKpI8xlFKYVapSRxHt2/jjX/6EVa7g9sefozZC89LyfcIIUSqSfKYBlprPnr9X9n822cprZnHhu88JIephBBZTZJHioVDg/zpF49z4D/+zKJLLuOa+76F1ZbGN/I991z6vlsIcc6Q5JFCfZ0dvPaj79N6/AiX3v4VVt90a/rf810pL/QQQiROkkeKdLc089KjDxLo62PDdx5ifu2adIdkeOEFY3jbbemNQwiR1SR5pICvsZ6X//5BIuEwt/3vf6R07vx0hzTqySeNoSQPIUQC0nqNqFKqQCn1llLqcGx42oOclFIXKqU+VErtVUrtUkpl9Favve44Lzz8XYaGhjIvcQghRJKk+waD7wLvaK0XAO/ExsfzA1/RWl8AXAv8RCmVP40xTlnL0cO8+Mj/wmyxcNvDj1FUVZ3ukIQQIiXSnTw2AM/G6s8CN41voLU+pLU+HKs3AW1A8bRFOEWNB/fz0t8/iM3p4vbv/YCC2RXpDkkIIVIm3ec8SrXWzQBa62alVMmZGiulVgM24Oh0BDdVJ/fs4tUfPoK7oJBb//b7cg+HEOKcl/LkoZR6GyibYNaDcX7OLOA54E6t9dAkbe4F7gWoqqqKM9Kz03rsCK/+8BE8xcaj1JP+/o1ke/nldEcghDgHpDx5aK3XTzZPKdWqlJoV2+uYhXFIaqJ2HuD3wENa6y1n+K6ngKcAamtrdWKRf7aetlZe+cH3cHo82ZE4AIpkr0gIkbh0n/N4HbgzVr8TeG18A6WUDXgF+I3W+qVpjO2Mgv39/L/HHiYSDnHzdx/OjsQB8MwzRhFCiASkO3k8BlyllDoMXBUbRylVq5T6VazNXwGXAXcppXbGyoXpCdcQCYd57ceP0tPazIbvPERhxfQcIksKSR5CiCRI6wlzrbUPWDfB9I+Be2L154Hnpzm0SemhIf745E9o2LeH6+//DpWLl6Y7JCGEmHbp3vPIOu+/8BwH/uPPXPqf7uT8Sy9PdzhCCJEWkjzi8Olbb7Lt1ZdYtv5aVm+4Jd3hCCFE2kjymKLGg/t559dPUrOilnV335f+p+MKIUQapfsmwawQHOjnjX/6EZ7iEm745l9jMqfmTYPT4o030h2BEOIcIHsen0FrzVtP/Yz+Th833P8/sbtc6Q4pMS6XUYQQIgGSPD7Dno1vcWjL+3z+ti8za8GidIeTuCeeMIoQQiRAkscZ+BrrefeZX1K1ZDkX3XhzusNJjhdfNIoQQiRAksckIqEQv//pD7Ha7Fz3jW+jTPKjEkKIYbJFnMTmf3mG9rrjXHPfA7gLCtMdjhBCZBRJHhM4un0bO958nRXX3ci8VavTHY4QQmQcSR7j9Hd18scnf0LxnBouu+Or6Q5HCCEyktznMU5nYz0mi4UbvvnXWGy2dIeTfJs2pTsCIcQ5QJLHOFVLlnPP4786NxOHEEIkiRy2moAkDiGEODNJHkIIIeImyUMIIUTcJHkIIYSImyQPIYQQcZPkIYQQIm6SPIQQQsRNkocQQoi4Ka11umNICaVUO1B3FosWAR1JDiddpC+ZSfqSmaQvhjla6+LPanTOJo+zpZT6WGtdm+44kkH6kpmkL5lJ+hIfOWwlhBAibpI8hBBCxE2Sx+meSncASSR9yUzSl8wkfYmDnPMQQggRN9nzEEIIEbcZlTyUUtcqpQ4qpY4opb47wXy7UuqF2PytSqnqMfP+Jjb9oFLqmumMeyJn2xelVLVSKqCU2hkrv5ju2MebQl8uU0rtUEpFlFK3jJt3p1LqcKzcOX1RTyzBvkTHrJfXpy/qiU2hL99WSu1TSu1SSr2jlJozZl7GrJcE+5Ft6+S/KqV2x+J9Xym1eMy85G7DtNYzogBm4CgwF7ABnwKLx7X5OvCLWP124IVYfXGsvR2oiX2OOUv7Ug3sSff6iLMv1cAy4DfALWOmFwDHYkNvrO7Nxr7E5vWne33E2ZcrAFesft+Y37GMWS+J9CNL14lnTP0vgD/E6knfhs2kPY/VwBGt9TGtdQj4HbBhXJsNwLOx+svAOqWUik3/ndZ6UGt9HDgS+7x0SaQvmeYz+6K1PqG13gUMjVv2GuAtrXWn1roLeAu4djqCnkQifck0U+nLRq21Pza6BaiI1TNpvSTSj0wzlb70jhnNAYZPaid9GzaTkkc5UD9mvCE2bcI2WusI0AMUTnHZ6ZRIXwBqlFKfKKX+rJRam+pgP0MiP9tsXC9n4lBKfayU2qKUuim5ocUt3r58DXjzLJdNpUT6AVm4TpRS31BKHQV+CHwznmXjMZPeYT7Rf93jLzWbrM1Ulp1OifSlGajSWvuUUquAV5VSF4z7j2U6JfKzzcb1ciZVWusmpdRc4F2l1G6t9dEkxRavKfdFKfUloBb4QrzLToNE+gFZuE601j8Hfq6UugN4CLhzqsvGYybteTQAlWPGK4CmydoopSxAHtA5xWWn01n3Jbbb6gPQWm/HOPa5MOURTy6Rn202rpdJaa2bYsNjwCZgRTKDi9OU+qKUWg88CPyF1nownmWnSSL9yMp1MsbvgOG9peSvk3SfBJqugrGXdQzjZNHwyaYLxrX5BqeeZH4xVr+AU082HSO9J8wT6UvxcOwYJ94agYJM7suYts9w+gnz4xgnZb2xerb2xQvYY/Ui4DDjToZmWl8wNqRHgQXjpmfMekmwH9m4ThaMqd8IfByrJ30blpYfQhp/+NcDh2K/KA/Gpj2C8d8GgAN4CeNk0jZg7phlH4wtdxC4Llv7AvwlsDf2i7QDuDEL+nIRxn9OA4AP2Dtm2btjfTwCfDVb+wJcAuyOrZfdwNeyoC9vA63Azlh5PRPXy9n2I0vXyU9jf987gY2MSS7J3obJHeZCCCHiNpPOeQghhEgSSR5CCCHiJslDCCFE3CR5CCGEiJskDyGEEHGT5CGEECJukjyEEELETZKHEBlKKXWTUur/KqVeU0pdne54hBhLkocQCVJKLVVK1Sml7ptg3i+VUp9XSm0a93Kx+5RST4wZf1Qp9dzYdlrrV7XW/xm4C7gtxd0QIi6SPIRIkNZ6N8bzw74ywew1GO+IGO9Z4EalVL5S6ovADcC9k3zFQ8DPkxGrEMkykx7JLkQqtWE8fG6EUup84JDWOjr+PVxaa79S6rfA94HrgKu01oGx7WIv73oMeFNrvSPF8QsRF0keQiTHY4BdKTVHa10Xm3Yd8IczLPNrYD+wQU/8joj7gfVAnlJqvtY67e+bF2KYJA8hEqSUuhbjlZ+/x9j7GE4e1wBfPcOifwe0M8nfodb6ceDx5EUqRPLIOQ8hEqCUcmC87vPrGI/tXhKb7gLydexlQhMs9z8wHpv/V8C3pidaIZJHkocQiXkI+I3W+gRjkgdwBcb7FE6jlLoSY4/kTq31JsCjlLow9aEKkTySPIQ4S0qpRcBVwE9ik8YmjwnPdyilqoBfAbdqrftik38KPJDaaIVILnkZlBApoJTaAazRWodj45uAu2J7KGdabkrthEg3OWEuRAporVemOwYhUkkOWwkxPZ4BupPYToi0ksNWQggh4iZ7HkIIIeImyUMIIUTcJHkIIYSImyQPIYQQcZPkIYQQIm6SPIQQQsRNkocQQoi4SfIQQggRt/8P1Z4DNyZCBXYAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "run -i nt_solutions/ml_2_regression/exo2" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "## Insert your code here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sparse Regularization\n", "---------------------\n", "In order to perform feature selection (i.e. select a subsect of the\n", "features which are the most predictive), one needs to replace the\n", "$\\ell^2$ regularization penalty by a sparsity inducing regularizer. The\n", "most well known is the $\\ell^1$ norm\n", "$$ \\norm{w}_1 \\eqdef \\sum_i \\abs{w_i} . $$\n", "\n", "\n", "The energy to minimize is\n", "$$ \\umin{w} J(w) \\eqdef \\frac{1}{2}\\norm{X w-y}^2 + \\lambda \\norm{w}_1. $$" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "def J(w,Lambda): return 1/2*np.linalg.norm(X0.dot(w)-y0)**2 + Lambda*np.linalg.norm(w,1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The simplest iterative algorithm to perform the minimization is the\n", "so-called iterative soft thresholding (ISTA), aka proximal gradient aka\n", "forward-backward.\n", "\n", "\n", "It performs first a gradient step (forward) of the smooth part $\\frac{1}{2}\\norm{X w-y}^2$ of the\n", "functional and then a proximal step (backward) step which account for the\n", "$\\ell^1$ penalty and induce sparsity. This proximal step is the soft-thresholding operator\n", "$$ \\Ss_s(x) \\eqdef \\max( \\abs{x}-\\lambda,0 ) \\text{sign}(x). $$" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "def Soft(x,s): return np.maximum( abs(x)-s, np.zeros(x.shape) ) * np.sign(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The ISTA algorithm reads\n", "$$ w_{k+1} \\eqdef \\Ss_{\\la\\tau}( w_k - \\tau X^\\top ( X w_k - y ) ), $$\n", "where, to ensure convergence, the step size should verify $ 0 < \\tau <\n", "2/\\norm{X}^2 $ where $\\norm{X}$ is the operator norm.\n", "\n", "\n", "Display the soft thresholding operator." ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGxdJREFUeJzt3Xl4VPXd/vH3h33fw74vsm8mJFpr3St1t9aFSoAQRG1ttWqtlrbWVtuqba1aH5EaAgRcqxR3xb1u2SDsO7KENeyBQEIy398f5Hl+tKVAmDNzZs7cr+vKdTFmrnPuCZc333znnM+Ycw4REQmOWn4HEBERb6nYRUQCRsUuIhIwKnYRkYBRsYuIBIyKXUQkYFTsIiIBo2IXEQkYFbuISMDU8eOkbdq0cd27d/fj1CIicauwsHCHcy7pRM/zpdi7d+9OQUGBH6cWEYlbZrb+ZJ6nrRgRkYBRsYuIBIyKXUQkYFTsIiIBo2IXEQmYsIvdzBqYWZ6ZLTCzJWb2gBfBRETk1HhxuWM5cL5zbr+Z1QU+M7O3nXNfeXBsERGpobBX7O6I/dUP61Z/6fP2RESOUlZRya9fW8LessMRP5cne+xmVtvMioDtwFznXO4xnjPRzArMrKCkpMSL04qIxIWKyhC3zpzHjC/XMW/j7oifz5Nid85VOeeGAZ2BVDMbdIznTHHOpTjnUpKSTnhHrIhIIIRCjrtfXsAnK0t46OrBnNe3bcTP6elVMc65PcDHwEgvjysiEo+cc/z69SW8tmAz94zsy6jUrlE5rxdXxSSZWYvqPzcELgSWh3tcEZF499j7q5jx5Xomfqsnt57TK2rn9eKqmA7AdDOrzZF/KF5yzr3hwXFFROJW9udf88QHq7g2uTP3facfZha1c4dd7M65hcBwD7KIiATC7PnFPPD6Ur49oB2//+7gqJY66M5TERFPfbh8G3e/vJAze7bmiVHDqVM7+jWrYhcR8Uje17u4deY8BnRoxpQxyTSoW9uXHCp2EREPLNm8l8xp+XRq2ZBpGSNo2qCub1lU7CIiYVq34wBjp+bTpEEdcjLTaN2kvq95VOwiImHYtu8Qo7NyqQqFyMlMpVOLhn5HUrGLiJyqPWUVpGflsvtABdMyUundtqnfkQCfPsxaRCTelVVUkjEtn3U7ysjOGMHQLi38jvR/tGIXEamhisoQN+cUsmDjHp4YNYyzerfxO9K/0IpdRKQGqkKOO18q4p+rdvDwNYMZOaiD35H+g1bsIiInyTnHr+Ys5o2FW7jvO/24fkR0hnrVlIpdROQk/XnuSmblbuDmc3pycxSHetWUil1E5CRkffY1T364mutTunDvyH5+xzkuFbuIyAm8UljMb99YysiB7Xno6kFRH+pVUyp2EZHjmLt0G/e8spCzerfm8VHDfBnqVVOxn1BExCdfrd3JD5+bx6COzXgmPYX6dfwZ6lVTKnYRkWNYvGkvN00voEvLhmRnpNKkfvxcHa5iFxH5N2tL9jN2ah5Nq4d6tWpcz+9INaJiFxE5ypa9B0nPysMBORPS6BgDQ71qSsUuIlJt94EKxmTlsffgYaZnpNIrqYnfkU5J/GwaiYhE0IHyI0O91u8qY3pGKoM7N/c70inTil1EEl55ZRW3zCxkYfEenhw1nDN7tfY7Uli0YheRhFYVctz54gL+uWoHj3xvCBcPbO93pLBpxS4iCcs5xy/+sZg3F21h0iX9uS6li9+RPKFiF5GE9ei7K3g+bwM/OLcXN32rp99xPBN2sZtZFzP7yMyWmdkSM7vdi2AiIpH0t0/X8j8fr2FUald+enFfv+N4yos99krgLufcPDNrChSa2Vzn3FIPji0i4rmXCjby0FvLuGRwex68KvaHetVU2Ct259wW59y86j+XAsuATuEeV0QkEt5dspV7X1nI2X3a8Nj1w6hdK1ilDh7vsZtZd2A4kHuM7000swIzKygpKfHytCIiJ+XLNTv50fPzGdy5BZNHJ8fNUK+a8qzYzawJ8Apwh3Nu379/3zk3xTmX4pxLSUpK8uq0IiInZVHxXm6aUUC3Vo2YNm4EjeNoqFdNeVLsZlaXI6U+yzn3qhfHFBHxypqS/YzNzqN5w7rkZKbRMs6GetWUF1fFGJAFLHPO/Tn8SCIi3tm85yDpz+ZiQE5mKu2bN/A7UsR5sWI/C0gHzjezouqvSzw4rohIWHYdqCA9K5fSQ5VMH59Kzzgd6lVTYW8yOec+A4L3trKIxLX95ZVkZOdRvPsg08enMqhT/A71qqngvnsgIgmrvLKKm3MKWLx5H5NHJ3NGz/ge6lVTGikgIoFSFXLc/nwRn6/eySPXDOGiAe38jhR1KnYRCQznHD9/dRHvLNnKLy8bwDXJnf2O5AsVu4gExsPvrODFgo3cdl5vMr/Zw+84vlGxi0ggPPPJGiZ/soYb07py17dP8zuOr1TsIhL3XszfwO/fXs5lQzrwmyuDN9SrplTsIhLX3lm8hfteXcTZfdrw5+uCOdSrplTsIhK3vli9gx8/X8TQLi14Jj2ZenVUaaBiF5E4tWDjHm6aUUD3No3IHjeCRvV0W87/UrGLSNxZvX0/47LzaNm4HjmZabRoFOyhXjWlYheRuLJpz0HSs3KpXcuYmZlGu2bBH+pVUyp2EYkbO/eXk56Vy/7qoV7d2zT2O1JM0qaUiMSF0kOHGZedz6bdB8nJTGNgx8QZ6lVTKnYRiXmHDlcxcUYhS7fsY0p6Mqk9WvkdKaZpK0ZEYlplVYgfPz+fL9fu5I/XDuGC/ok31KumVOwiErOcc9z36iLeW7qN+y8fwNXDE3OoV02p2EUkJjnn+P3by3m5sJgfX9CHjLMSd6hXTanYRSQmTf5kLVM+XcuYM7vxkwv7+B0nrqjYRSTmPJ+3gYffWc4VQzvy68sHJvxQr5pSsYtITHlr0RYmzV7EOacl8cdrh1JLQ71qTMUuIjHjs1U7uOOFIoZ3bcnk0Rrqdar0UxORmFC0cQ8TcwromdSYqWNH0LBebb8jxS0Vu4j4btW2UsZl59G6ST1mjE+leaO6fkeKa54Uu5lNNbPtZrbYi+OJSOIo3l1GelYedWvXYmZmGm011CtsXq3YpwEjPTqWiCSIHfvLSc/K40BFJTPGp9KttYZ6ecGTYnfOfQrs8uJYIpIYSg8dZuzUPLbsPUj2uBH079DM70iBoT12EYm6Q4ermDC9gBVbS3n6xmRSumuol5eiVuxmNtHMCsysoKSkJFqnFZEYU1kV4rbn5pO3bhd/um4o5/Vr63ekwIlasTvnpjjnUpxzKUlJSdE6rYjEkFDI8bNXFvH+sm38+vKBXDmsk9+RAklbMSISFc45HnprGa/MK+aOC/sw9hvd/Y4UWF5d7vg88CXQ18yKzSzTi+OKSHD8z8dryPrsa8Z9ozu3X6ChXpHkyScoOedGeXEcEQmmWbnrefTdFVw1rCO/umyAhnpFmLZiRCSi3li4mV/8YzHn92vLoxrqFRUqdhGJmE9XlvCTF4tI6daSp75/OnVrq3KiQT9lEYmIeRt2c3NOIb2SmvCshnpFlYpdRDy3clspGdn5tG1WnxmZqTRvqKFe0aRiFxFPbdxVRnpWLvXrVA/1aqqhXtGmYhcRz5SUlpOelcvBiipmZKbSpVUjvyMlJE8udxQR2Vc91GvbvnJmTkijX3sN9fKLVuwiErZDh6uYMK2AVdtLeXr06SR3a+l3pISmFbuIhOVwVYgfzppH/vpdPH7DcM7tq6FeftOKXUROWSjkuOfvC/lg+XZ+c+Ugrhja0e9IgopdRE6Rc47fvrmU2fM3cddFp5F+Rje/I0k1FbuInJK/fria7M/XkXFWd247v7ffceQoKnYRqbGcr9bzp7kr+e7wTvzyUg31ijUqdhGpkdcWbOZXcxZzYf+2PPy9IRrqFYNU7CJy0j5esZ07XyxiRLdW/FVDvWKW/lZE5KQUrt/FLTMLOa1dU54dl0KDuhrqFatU7CJyQsu37iMjO5/2zRowfXwqzRpoqFcsU7GLyHFt2FnGmKw8GtarTU5mGklN6/sdSU5Ad56KyH+1vfQQ6VNzKa8M8fItZ2qoV5zQil1EjmnvwcOMycqjpLSc7IwRnNauqd+R5CSp2EXkPxysqCJzWj5rSvYzeXQyp3fVUK94oq0YEfkXh6tC/GBWIYUbdvPkqOF867QkvyNJDWnFLiL/JxRy3P3yAj5aUcKDVw3isiEa6hWPVOwiAhwZ6vXA60uYU7SZn17clxvTNNQrXnlS7GY20sxWmNlqM7vXi2OKSHQ9/sEqpn+5ngnf7MEPzu3ldxwJQ9jFbma1gaeA7wADgFFmNiDc44pI9Ez/Yh1/eX8V30vuzKRL+2uoV5zzYsWeCqx2zq11zlUALwBXenBcEYmCOUWbuP+1JVw0oB1/+O5glXoAeHFVTCdg41GPi4E0D44rPireXcZjc1dRVlHpdxSJoJBzfLBsO2k9WvHkqOHU0VCvQPCi2I/1z7v7jyeZTQQmAnTt2tWD00qklJSWk56Vx7Z9h+jcsqHfcSTCzu/Xlj9dN1RDvQLEi2IvBroc9bgzsPnfn+ScmwJMAUhJSfmP4pfYsO/QYcZl57Fl70FmTUgjuVsrvyOJSA158XtXPtDHzHqYWT3gBuA1D44rUXbocBUTphewYmspk0cnq9RF4lTYK3bnXKWZ3Qa8C9QGpjrnloSdTKKqsirEbc/NI3/dLh6/YTjn9m3rdyQROUWejBRwzr0FvOXFsST6QiHHPa8s5P1l2/ntVYO4YqjuNhSJZ3oLPME553jwzWW8Om8Td150Guln6G5DkXinYk9wT320mqmff03GWd350fm9/Y4jIh5QsSewmV+t54/vreTq4Z345aUDdGOKSECo2BPU6ws288s5i7mgX1se+d4QatVSqYsEhYo9AX2ysoQ7XypiRLdWPHXj6dTV3YYigaL/oxNM4frd3JJTSO+2TXl2XIruNhQJIBV7AlmxtZTx0/Jp16w+M8an0qxBXb8jiUgEqNgTxMZdZaRn5dKgbi1yMtNIalrf70giEiH6zNMEUFJazuisXMorQ7x085l0adXI70giEkFasQfc3oOHGTM1j+37ysnOGEHf9k39jiQiEaZiD7CDFVVMmJ7P6u2lPJOezOldW/odSUSiQFsxAXW4KsQPn5tHwfrdPDlqON86LcnvSCISJVqxB1Ao5Ljn7wv5cPl2HrxqEJcN0VAvkUSiYg8Y5xy/eWMps+dv4qcX9+XGNA31Ekk0KvaAefLD1Uz7Yh2Z3+zBD87t5XccEfGBij1Acr5cx5/nruSa0zsz6ZL+GuolkqBU7AExp2gTv3ptCRf2b8fD1wzWUC+RBKZiD4CPVmznrpcWkNq9FX/9/nDqaKiXSEJTA8S5wvW7uHVmIX3bN+VvYzXUS0RU7HFt2ZZ9ZGTn06F5Q6ZrqJeIVFOxx6kNO8sYMzWPRvXqkJOZSpsmGuolIkeo2OPQ9n2HGJ2Vy+GqEDmZqXRuqaFeIvL/qdjjzN6yI0O9duwvJ3vcCPq001AvEflXKvY4crCiiszp+awp2c8z6ckM11AvETmGsIrdzK41syVmFjKzFK9CyX86XBXi1lmFFG7YzeM3DOfsPhrqJSLHFu6KfTHwXeBTD7LIfxEKOe5+eQEfryjhd1cP5pLBHfyOJCIxLKyxvc65ZYBuXY8g5xwPvL6EOUWbuWdkX0aldvU7kojEOO2xx7i/vL+K6V+u56aze3DrORrqJSIndsIVu5m9D7Q/xrcmOefmnOyJzGwiMBGga1etOk/GtM+/5vEPVnFtcmd+rqFeInKSTljszrkLvTiRc24KMAUgJSXFeXHMIPvH/E38+vWlfHtAO37/3cEqdRE5adqKiUEfLt/G3S8v4IyerXhilIZ6iUjNhHu549VmVgycCbxpZu96Eytx5a/bxa0z59G/QzP+NkZDvUSk5sK9KmY2MNujLAlv6eZ9jJ+WT6cWDZmWMYKmGuolIqdAv+PHiHU7DjBmah5N6tchZ0IarTXUS0ROkYo9BmyrHupVFToy1KtTi4Z+RxKROKZi99mesgrGZOWx+0AF0zJS6d1WQ71EJDxh7bFLeMoqKhk/LZ+vdxwgO2MEQ7u08DuSiASAVuw+qagMccvMeRRt3MMTo4ZxVu82fkcSkYDQit0HVSHHnS8V8enKEh6+ZjAjB2mol4h4Ryv2KHPOcf9ri3lj4Rbu/U4/rh+h8Qoi4i0Ve5Q9NnclM7/awM3n9OQWDfUSkQhQsUfR1M++5okPV3N9ShfuHdnP7zgiElAq9ih5dV4xv3ljKSMHtuehqwdpqJeIRIyKPQreX7qNn/59Id/o1Zq/3DBMQ71EJKLUMBGWu3YnP3xuHgM7NmOKhnqJSBSo2CNo8aa9TJheQOeWDZmWkUqT+rq6VEQiT8UeIV/vOMC47DyaNqhDTmYarRrX8zuSiCQIFXsEbN17iNHP5hJykDMhjY4a6iUiUaRi99iesgrSs3LZU1bB9IxUeiU18TuSiCQYbfp66EB5JeOy81m/q4xpGSMY3Lm535FEJAFpxe6R8soqbplZyMLiPTw5ajjf6KWhXiLiD63YPVAVctz54gL+uWoHj3xvCBcPbO93JBFJYFqxh8k5xy/nLObNRVuYdEl/rkvp4nckEUlwKvYw/fG9FTyXu4Fbz+3FTd/q6XccEREVezie/edanvpoDaNSu3DPxX39jiMiAqjYT9nfC4t58M1lXDK4PQ9eNVhDvUQkZqjYT8F7S7bys1cW8s3ebXjs+mHUrqVSF5HYEVaxm9mjZrbczBaa2WwzC/ynMX+5Zie3PT+fQZ2a80x6MvXraKiXiMSWcFfsc4FBzrkhwErgvvAjxa7Fm/Zy04wCurZqxLRxI2isoV4iEoPCKnbn3HvOucrqh18BncOPFJvWluxn7NQ8mjesS05mKi011EtEYpSXe+zjgbc9PF7M2LL3IOlZeQDkZKbSobmGeolI7DrhXoKZvQ8c61bKSc65OdXPmQRUArOOc5yJwESArl27nlJYP+w6UEF6Vh57Dx7mhYln0FNDvUQkxp2w2J1zFx7v+2Y2FrgMuMA5545znCnAFICUlJT/+rxYsr+8kozsPDbsKmPG+FQGddJQLxGJfWG9+2dmI4GfAec458q8iRQbyiuruDmngMWb9zF5dDJn9GztdyQRkZMS7h77X4GmwFwzKzKzyR5k8l1VyHHHC0V8vnonj1wzhIsGtPM7kojISQtrxe6c6+1VkFjhnGPS7EW8vXgrv7i0P9ckB/ZCHxEJKN15+m8eeXcFL+Rv5LbzejPhbA31EpH4o2I/ypRP1/D0x2v4flpX7vr2aX7HERE5JSr2ai/lb+R3by3n0iEd+O2VgzTUS0TiloodeGfxVu59dSFn92nDY9dpqJeIxLeEL/Yv1uzgx8/PZ2iXFjyTnky9Ogn/IxGROJfQLbaweA83TS+ge5tGZI8bQaN6GuolIvEvYYt99fb9jMvOp2XjeswYn0aLRhrqJSLBkJDFvmnPQcZk5VLLICczjfbNG/gdSUTEMwlX7Dv3l5OelUvpoUqmj0+lR5vGfkcSEfFUQm0q7y+vJGNaPpt2HyQnM42BHTXUS0SCJ2GK/dDhKibOKGDJ5n1MSU8mtUcrvyOJiEREQmzFVFaFuP2F+XyxZid/vHYIF/TXUC8RCa7AF7tzjp/PXsS7S7Zx/+UDuHq4hnqJSLAFvtj/8PZyXioo5sfn9ybjrB5+xxERibhAF/vkT9bwzKdrST+jGz+5SEO9RCQxBLbYX8jbwB/eXs7lQzvywBUDNdRLRBJGIIv97UVb+PnsRZxzWhJ/unYotTTUS0QSSOCK/bNVO7j9hSKGd23J06NP11AvEUk4gWq9oo17mJhTQI82jZk6VkO9RCQxBabYV28vJSM7j9ZN6pGTmUrzRnX9jiQi4otAFHvx7jJGP5tH7Vq1mJmZRttmGuolIokr7ot9x/5yxmTlcaCikpzMVLq11lAvEUlscV3spYcOMy47j817DzJ13Aj6d2jmdyQREd/FbbEfOlzFTTMKWL6llKdvTGZEdw31EhGBMIvdzH5rZgvNrMjM3jOzjl4FO57KqhA/en4+X63dxZ+uG8p5/dpG47QiInEh3BX7o865Ic65YcAbwK88yHRcoZDj3lcXMXfpNh64YiBXDusU6VOKiMSVsIrdObfvqIeNARdenBOej9+9tYy/FxZzx4V9GPuN7pE8nYhIXAr7Dh4zewgYA+wFzgs70XE8/ckanv3sa8ae2Y3bL+gTyVOJiMStE67Yzex9M1t8jK8rAZxzk5xzXYBZwG3HOc5EMysws4KSkpJTCtutVWOuTe7M/ZdrqJeIyH9jznmze2Jm3YA3nXODTvTclJQUV1BQ4Ml5RUQShZkVOudSTvS8cK+KOXo/5ApgeTjHExGR8IW7x/4HM+sLhID1wC3hRxIRkXCEVezOuWu8CiIiIt6I2ztPRUTk2FTsIiIBo2IXEQkYFbuISMCo2EVEAsazG5RqdFKzEo5cHhlv2gA7/A4RRYn2ekGvOVHE62vu5pxLOtGTfCn2eGVmBSdz11dQJNrrBb3mRBH016ytGBGRgFGxi4gEjIq9Zqb4HSDKEu31gl5zogj0a9Yeu4hIwGjFLiISMCr2U2Bmd5uZM7M2fmeJNDN71MyWV39o+Wwza+F3pkgxs5FmtsLMVpvZvX7niTQz62JmH5nZMjNbYma3+50pGsystpnNN7M3/M4SKSr2GjKzLsBFwAa/s0TJXGCQc24IsBK4z+c8EWFmtYGngO8AA4BRZjbA31QRVwnc5ZzrD5wB/DABXjPA7cAyv0NEkoq95h4D7iHCH9wdK5xz7znnKqsffgV09jNPBKUCq51za51zFcALwJU+Z4oo59wW59y86j+XcqTsOvmbKrLMrDNwKfCs31kiScVeA2Z2BbDJObfA7yw+GQ+87XeICOkEbDzqcTEBL7mjmVl3YDiQ62+SiPsLRxZmIb+DRFK4n6AUOGb2PtD+GN+aBPwc+HZ0E0Xe8V6zc25O9XMmceRX91nRzBZFx/p09IT4rczMmgCvAHc45/b5nSdSzOwyYLtzrtDMzvU7TySp2P+Nc+7CY/13MxsM9AAWmBkc2ZKYZ2apzrmtUYzouf/2mv+XmY0FLgMucMG9PrYY6HLU487AZp+yRI2Z1eVIqc9yzr3qd54IOwu4wswuARoAzcxspnNutM+5PKfr2E+Rma0DUpxz8ThI6KSZ2Ujgz8A5zrkSv/NEipnV4cibwxcAm4B84PvOuSW+BosgO7JCmQ7scs7d4XeeaKpesd/tnLvM7yyRoD12OZG/Ak2BuWZWZGaT/Q4UCdVvEN8GvMuRNxFfCnKpVzsLSAfOr/67LapezUqc04pdRCRgtGIXEQkYFbuISMCo2EVEAkbFLiISMCp2EZGAUbGLiASMil1EJGBU7CIiAfP/AJ21/y9tWcydAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "t = np.linspace(-5,5,201)\n", "plt.clf\n", "plt.plot(t,Soft(t,2)) \n", "plt.axis('tight');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Descent step size." ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "tau = 1.5/np.linalg.norm(X0,2)**2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Choose a regularization parameter $\\la$." ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "lmax = abs( X0.transpose().dot(y0) ).max()\n", "Lambda = lmax /10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Initialization $w_0$." ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [], "source": [ "w = np.zeros((p,1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A single ISTA step." ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [], "source": [ "C = X0.transpose().dot(X0)\n", "u = X0.transpose().dot(y0)\n", "def ISTA(w,Lambda,tau): return Soft( w-tau*( C.dot(w)-u ), Lambda*tau )\n", "w = ISTA(w,Lambda,tau)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Exercise 3__\n", "\n", "Implement the ISTA algorithm, display the convergence of the energy." ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8HWd97/HPT+do345ky5sWL4kT2/EiJ24aQtic0IYACaW0pKU00NCUlpbl9t4Cly7QW27L6/YCoVBe5JKUsENDCiGsSUhCQ4OD9zWJHTuWFG9yrM2LbC2/+8eMpGPlyIvOJs35vl8vvaQzZzTzjEbnN8/8nmeex9wdERGJvqJ8F0BERHJDAV9EpEAo4IuIFAgFfBGRAqGALyJSIBTwRUQKhAK+FBwzKzWznWY2J0Pbe8zM3jXBe/eb2Y2Z2I9IuhTwpSCMC8p3AD9390M52PU/AR/PwX5EzksBXwrRnwBfycWO3P0poMbM1uRifyLnooAvBcXMWoBLgHXh64Vm1m1mReHrL5rZkaT1v2pm77+I7c81s61m9t+TFj8GvD4jByCSBgV8KTQrgL3uPgjg7vuAXmB1+P4rgONmtjR8/Urg8QvZsJktCNf9rLv/c9Jbu4BVaZdcJE0K+FJoEkDfuGWPA69KasS9L3y9EKgBtlzAdpcR1OT/zt3vGvdeX7hfkbyK57sAIjnWBVSPW/Y4cDPQAfycIHC/HegH/tPdhy9gu28D9hBcLMarBronWV6RjFENXwrNVmCRmSVXdh4nSOW8Ovz5CeDlwKu4wHQO8FHgKPB1M4uNe28pF3aXIJJVCvhSUNy9A9gNXJ20bDdwCvgDgu6avcBh4Le58IA/APwOUAl8ZaQROPQq4Efpl14kPQr4UkhGJn/4AkHKJtnjwIvu3pb02oBNF7xx9zPAm4FZwD1mVmRmvwacCLtniuSVaQIUKQRmthH4e3f/rpmVEgTy6939YJb3+x3gbnf/YTb3I3IhFPAl8szsCmA9sMTd9+e7PCL5opSORJqZfQL4KfDBdIK9mR2f4OsVmSutSHaphi8iUiBUwxcRKRBT6sGrmTNn+oIFC/JdDBGRaWXDhg1H3b3hfOtNqYC/YMEC1q9fn+9iiIhMK2Z2Qe1TSumIiBQIBXwRkQIRiYD/0M7DrPmHh3n+6Il8F0VEZMqKRMAHOHr8NH39g/kuhojIlBWJgF9ZGgxOeOKMAr6IyESiEfBLgs5GJ04r4IuITCQaAb80DPhnhvJcEhGRqSsiAT9I6ZxUDV9EZEKRCPgVYUrnuAK+iMiEIhHwK0vCGr5SOiIiE4pEwI/HiiiNF6nRVkTkHCIR8AGqSuPqlikicg6RCfgVpTFOnFZKR0RkIpEJ+JUlcaV0RETOIToBXykdEZFzilbAV0pHRGRC0Qn4JTGldEREziE6Ab80rn74IiLnEJ2AXxLTk7YiIucQnYBfGuekGm1FRCYUqYA/MOScHlRaR0QklcgE/IqR8XTUU0dEJKXIBPyxMfGV1hERSSU6AX901ivV8EVEUolOwNe8tiIi5xShgK95bUVEziU6AV8pHRGRc4pOwB9J6aiGLyKSUoQCflDD18NXIiKpRSfgj05krpSOiEgqGQv4ZhYzs01m9mD4eqGZrTOz3Wb2LTMrydS+UikrLqLIVMMXEZlIJmv47wN2Jb3+BPApd18MdAG3Z3BfL2FmVJbENYCaiMgEMhLwzawJeD3wxfC1AWuB+8JV7gXelIl9nUtFaUxDK4iITCBTNfxPA38FDIevZwDd7j5S3e4AGlP9opndYWbrzWx9Z2dnWoWoLI1zXCkdEZGU0g74ZvYG4Ii7b0henGJVT/X77n6Xu69x9zUNDQ1plaWyJM5JpXRERFKKZ2AbLwduNrObgDKghqDGnzCzeFjLbwIOZGBf51RZGtODVyIiE0i7hu/uH3b3JndfANwK/Mzd3wY8CrwlXO024Hvp7ut8KkviGktHRGQC2eyH/0Hgv5nZHoKc/t1Z3BegeW1FRM4lEymdUe7+GPBY+PNe4OpMbv98Kks1r62IyEQi86QtqNFWRORcIhXwK0rjnDgzxPBwyg5BIiIFLVIBvyocMfPUgPL4IiLjRSrgV5RoEhQRkYlEKuCPTXOoGr6IyHjRCviq4YuITChaAV/z2oqITCiaAV9P24qIvES0An7JyLy2yuGLiIwXrYCvlI6IyISiFfBHGm3VS0dE5CUiFfArwm6ZGl5BROSlIhXwi2NFlMSLNOuViEgKkQr4AFWlcc1rKyKSQuQCfkVJTI22IiIpRC7gV5Vq1isRkVQiF/CDGr5SOiIi40Uu4Feqhi8iklL0An5JXDl8EZEUIhfwK0qV0hERSSVyAV+NtiIiqUUu4FeUqB++iEgqkQv4VaUxzgwNc2ZwON9FERGZUiIX8EfmtT2ptI6IyFkiF/CrwiGSj6unjojIWSIX8EdHzNQQySIiZ4lcwK9UDV9EJKW0A76ZlZnZU2a2xcx2mNnHwuULzWydme02s2+ZWUn6xT2/2dVlALzQdSoXuxMRmTYyUcM/Dax191VAK3CjmV0DfAL4lLsvBrqA2zOwr/O6ZFYlsSLjmUN9udidiMi0kXbA98Dx8GVx+OXAWuC+cPm9wJvS3deFKI3HWDSzkqcP9eZidyIi00ZGcvhmFjOzzcAR4CHgOaDb3UcS6R1A4wS/e4eZrTez9Z2dnZkoDkvm1vC0avgiImfJSMB39yF3bwWagKuBpalWm+B373L3Ne6+pqGhIRPFYcmcajq6TtHXP5CR7YmIREFGe+m4ezfwGHANkDCzePhWE3Agk/s6l8tnVwPw7GHV8kVERmSil06DmSXCn8uBG4BdwKPAW8LVbgO+l+6+LtSSuUHAV1pHRGRM/PyrnNdc4F4zixFcQL7t7g+a2U7gm2b2D8Am4O4M7OuCNCbKqS6Nq6eOiEiStAO+u28FVqdYvpcgn59zZsZlc6p5+qACvojIiMg9aTtiyZxqnj7Ui3vKtmIRkYIT6YDf2z/Iod7+fBdFRGRKiGzAv3xODYDSOiIioegG/NnqqSMikiyyAb+2oph5tWU8oyEWRESACAd8gMvnVKuGLyISinjAr+G5zuMMDGl+WxGRSAf8pXOrGRhy9naeyHdRRETyLtIBf0VjLQAPbs3ZMD4iIlNWpAP+ooYqbl41jy/8fC/tx07muzgiInkV6YAP8OGblhAz4x9+sDPfRRERyavIB/y5teX8+dpL+cmOw/zn7sxMsCIiMh1FPuAD3H7dQlrqK/jY93eqx46IFKyCCPhlxTH+5g3L2HPkOHd8eT2HNb6OiBSgggj4ADcsncVH37iMJ/e+yGs/+Tj/salDI2mKSEEpmIBvZrzj5Qv54XtfweLZ1XzgW1t4/Wee4MtPPk/PSc19KyLRZ1OplrtmzRpfv3591vczNOx8e307X1u3n+0v9FIaL2LJ3BoaE2XMqy2nrrKEypIYFaVxyotjlMaLKC2OURIrojhmFMeKiMeMeFERsSILvswwg1hR8L3IDCO40Jgx9nNYhmBZ+MrGymbJP5O8PPlV6nVEZPqKx4zSeGxSv2tmG9x9zXnXK8SAn2z7Cz3cv/EFdh/p40D3KV7oPkX/gBp2RSS33v2qS/jQ65ZM6ncvNOBnYk7baW15Yy3LwydyAdydM0PDnDw9xPHTg5weHKJ/YJjTg0OcHhxmcMgZGBpmYMgZdmdw2BkaHmZ4GIbcGR52HHCHYR/52XFntM1g5P2Rn5P3ncpE12Rn6lysRSQ9q5oSWd9HwQf88cyC26rSeIy6ypJ8F0dEJGMKptFWRKTQKeCLiBSIKdVoa2adwP5J/vpM4GgGizNdFOJxF+IxQ2EedyEeM1z8cc9394bzrTSlAn46zGz9hbRSR00hHnchHjMU5nEX4jFD9o5bKR0RkQKhgC8iUiCiFPDvyncB8qQQj7sQjxkK87gL8ZghS8cdmRy+TB9m1g680d03p7mdfwQOu/unJ/P+Re7rKeCd7r4j3W1NYt8GPA+sdffnLuL38lZmmZoU8CWnzKwOOAJUufvpNLbTAGwGLnX3U2Y2CzgMzHX3Q+Pfz0C5fxd4q7v/drrbyqbkvwPwSqZBmSV3opTSkelhBfBsOsE+9A7gh0nBfBXQ6e6HJng/XQ8ArzGzuRnaXrYk/x2mS5klRyIR8M3sRjN7xsz2mNmH8l2ebDCzZjN71Mx2mdkOM3tfuLzezB4ys93h97p8l/U8VgLbIEhVmNkHzWy/mXWb2bfNrDZ8r8jM/tbM2s3sgJndYmbDZvaTcDu/RRDMdpvZt4DVwNak/bwOeDx5x2Z23Myaw5/fbWZuZrPD1//DzL44UaHdvR/YAPxGun8AM3uXmf3EzD5vZl1m9qyZLTOz95lZm5kdNbM/MLP7zOxpMztoZo+H53qXmZ0I/y4vhn+b1yZtfuXI3yGTZc4VM/tA+P+93cy+YWZlZrbQzNaNnGszm/ZjnpjZPWZ2xMy2Jy1L+VkOPyefCePbVjO7crL7nfYB38xiwOcIPuDLgN8zs2X5LVVWDAJ/6e5LgWuA94TH+SHgEXdfDDwSvp7KRgMS8L8Izts1wBygFPjb8L2PAjeE7y0D7gROASOTF1wJ3BMedxfBBSA54K8Anhm37y6gOsyJvwfYA9SFr/8E+JfzlH0XQQ06XSuBNcB9BA/YbAN+FL53CcHf5U7gx+6+BPgO8CTBue0GTgM/Bb4IfAH4YNK2V3D23yFTZc46M2sE3guscfflQAy4FfgE8Kmkc317/kqZMV8Cbhy3bKLP8uuAxeHXHcDnJ7vTaR/wgauBPe6+193PAN8EbslzmTLO3Q+6+8bw5z6CD3IjwbHeG652L/Cm/JTwgq0AtoY1678Afj88tn6CALgmzL9/APgjd38BqAKGge0w2ohZCnwv3Oa9wBWcHegSQN+4fXeH2/oNgmC/PVzvRuAFd98Sbv9TZpbqoZe+cP10rQL+0d0fcfchYCewzd3vdPcBYC9QDdwdrr8CeIrgXBcB/wT8NcG53jlu28kX1EyWOVfiQLmZxYEK4CCwluB/A6bH//h5ufvPgWPjFk/0Wb4F+LIHfgkkJpumi0LAbwTak153hMsiy8wWEKQw1gGz3f0gBBcFYFb+SnZuYaBeTlCjfQVBkDuQtMpMgg/49QQX8T3h8k8TBLyRHiozgCGCgABwgCCQb0naVhdB0GTcsirg/QQ16F6gDvgz4DNJ6y0DUvVsqSa4aIw/rsfC9FCqrydSbGcl8OC4/SW/fjlBoP43M9tEcJfzLDAbWAJ8P+lcLycM+uHd7tJxf4eUZZ6Kwov7PwNtBP8HPQQpqW53HwxXi/Lne6LPcsZiXBQCfqpJnyLb9cjMqghu8d/v7r35Ls9FWggMu/t+oIHgA53sFuAJgsB/AMDM3gB0EqRA9obrGXAGuCx8vShcllzb3Zr0/ojucDtz3f0xgoC/iiBofjdpvfKw589fmdnHzWzkczI+mALg7q92d5vg67rkdc1sPlBCEMBHtBL0KBpxOUGt/PMEtTwD3krweY1zdqpqddLvXha+n/x3SFnmqSjMWd9C8H8yD6gkSGeMF9nP9wQyFuOiEPA7gOak102EwSJqzKyYINh/zd3vDxcfHrm9C78fyVf5LsBogy3wK+BlZnaJmVWZ2d8T1GDvIUhXXWtmlwKvAd5OkNt+N8Ht/acJUjyvDrf1cuDEuJ4/PwReNW7/XQSpopHafC/wPuALYWplpFtjn5ndAxxw94+4+7CZlQJXAQ+l+TdYRXBnMxzurwaYz9lpmAXAUXdfF67/DEGbRR/wdFiekXO9mrGAvpKkHlAZLHOu3ADsc/fOMLV1P3AtQQpjZO6OyH6+mfiznLEYF4WA/ytgcdiSX0LQyPNAnsuUcWE65G5gl7t/MumtB4Dbwp9vYyyvPRWNNii6+3rg4wQ1+g6Cmuhadz/p7o8A3wA2AW8kCNInCc7tz9z9bQSNWm82s3KCBttd4/b1ZeCm8P0RXQQ14K+Hr3sJatL/b1wZVxMEnq8mLb8ZeGxcCmoyVnF2bX4lQfrqJAS9kwhq+M+b2eXh+v0EtfZ9jDVa3wb8hKCxe6SnxwrOrs1nqsy50gZcY2YV4f/79QTH/SjwlnCdqf4/no6JPssPAH8Y9ta5BugZSf1ctGD6ven9BdxEcIv8HPCRfJcnS8d4HcFt3FaCgLE5PO4ZBMFvd/i9Pt9lzcKxvxv4PkGN/sFw2SKCWk4nQU3oT1P83v8mSH1dzL7eTxBcfgCsSlq+Dliew2NuBdaH5/u7BG0N5zzXBIHxj/NV5gwd98eApwkuYl8haJxfRNCGswf4d6A03+XMwHF+g6CdYoCgwnP7ROeXIKXzuTC+bSPoxTSp/epJW5lywlrMQYKGquuBrwE3e9BDYfy6ryX48Cxx97THTTezuwnuPPoIeob8dia2m22Z/jtINGlOW5mKVhPUsIsJ7tzeMUGw30bQW+ctmQpy7p7cx3t8G8CUlI2/g0STavgiIgUiCo22IiJyAaZUSmfmzJm+YMGCfBdDRGRa2bBhw1G/gDltsx7wzexGgqcaY8AX3f2fJlp3wYIFrF+/PttFEhGJFDPbfyHrZTWlU0ADm4mITHnZzuHnZGCzM4PDPLDlAGqAFhGZWLYD/nkH/TGzO8xsvZmt7+zsnNROvrOxg/d+YxN3fGUDx06cmXxpRUQiLNsB/7yD/rj7Xe6+xt3XNDSct80hpbeuaeZv3rCMx5/p5HV3/pz/2qOuyCIi42U74OdkYLOiIuP26xZy/59dS2VpnLfdvY6//u42ek4NnP+XRUQKRLYDfk4HNlveWMuDf3Ed77x2IV9f18YNn3yc7yu3LyICZDngezBpwZ8TjOq3C/i2u6eaWCJjKkri/O0bl/HAn1/HnJoy/uIbm3jT537BwzsPK/CLSEGbUkMrrFmzxjPZD39o2LlvQzuffXQP7cdOccW8Gt71ioXctGIupfFYxvYjIpJPZrbB3VNNy3n2elEO+CMGhob53uYD/Otje9jbeYIZlSXcenUz77h2IQ3VpRnfn4hILingpzA87PziuaN8+cn9PLLrMGXFMd513UL++JWLqC4rztp+RUSySQH/PPYdPcH//ekzPLj1IPWVJfz+1S28cdU8Lp8zft5rEZGpTQH/Am3t6OZTDz3L4892MuyweFYVa5fOYs38eq6aX0d9ZUlOyyMicrEU8C9SZ99pfrz9IN/fepBNbV0MDAV/l0UNlVzVUseV8+u4an4dlzZUUVSU6nkyEZH8UMBPQ//AEFs7eli//xgb93exsa17dMiG6rI4rc0JrmwJLgCtLQlqlP8XkTy60IA/pcbDnyrKimNcvbCeqxfWA8FE78+/eJKN+7vY0NbFxv1dfOZnu3EHsyANdGVLXfA1P8GimboLEJGpRzX8SerrH2BLew8b27rYsL+LTW1d9PYPAlBTFqe1pY4rW4I7Ad0FiEg2qYafZdVlxVy3eCbXLZ4JBF0+9x49wca2Lja1dbOprYs7Hxm7C7i0IbgLWN2S4Eq1BYhIHqiGn0XJdwEb27rY3N5N98lgQLfq0jitLQlWNydYPb+O1c0JEhXqESQiF081/Clg/F2Au7Pv6Ak2hncAG9u6+eyjexgOr7mLZlYGF4GW4AKwZE418ZjmmReRzFANP89OnB5ka0cPm9q72Li/m83tXRw9HvQIKi+OsaKpltUtCVY3B20Cs2rK8lxiEZlqVMOfJipL47zskhm87JIZQHAX0NF1ajQFtKmtm3ue2MfA0F4AGhPlY6mglgRXzKulrFgDwYnI+SngTzFmRnN9Bc31FdzSGswG2T8wxM6DvWxq6w4uBG3d/GDrQQCKY8bSuTWsbk6EF4I65s+owEwNwiJyNqV0pqkjvf1sbOtmc3uQBtra0cPJM0MA1FUU09octAW0NidY1ZygtlzdQkWiSimdiJtVU8aNy+dw4/I5AAwODbP7yHE2tQUXgM3t3Tz2bCcj1/NLGippbQ66hbaqQVikIKmGH2G9/QNsbe9hc3tXeCHo5sVwiIiy4iJWNgZpoNawPWBubXmeSywik6EavlCToltoR9cpNrUH3UI3t3fzpV88z5mhYQBm15SyqmnsIrCyKUFVqf5FRKJCn+YCktwgfPOqeQCcGRxm18He0QvA5vZufrrzMABFBotnVdPaPHYRuGx2NTE9ISwyLSngF7iSeBGrwobdEcdOnGFLezebwgvAj3cc4lvr2wGoKImxorF2tGtoa3Mdc2r1bIDIdKCALy9RX1nCa5bM4jVLZgFjTwiP3AFsbh95NiBo/5lTU3bWXcCKxloqlQoSmXL0qZTzMjMWNVSxqKGKN1/ZBIw9G7C5bewi8OMdh4AgFXTZ7DAVFN49KBUkkn8K+DIpZcWx0TkARrx4/DRbOrrZ3N7D5vZufrT9EN/81bhUUHgRaFWvIJGcU8CXjJlRVcraJbNZu2Q2MJYK2tLRPXoncM8vxlJBZ/UKakqwoqmWas0bIJI1CviSNcmpoN9aHaSCTg8OsfNAL5vbu9kyrlfQyOxhyV1DL5+tB8REMkUBX3KqNB4Lhn9OSgV1nTgTpoKCr4d3HebfN3QAwQNiy+fVjrYFtDYnaKor11hBIpOgJ21lynF32o+dYlN7F1vCJ4W3H+jlzGDwgNjMqhJWNY1dAFY1JaitUCpICpeetJVpy8xomVFBy4yxEUMHhoZ55lBf8GxAWzdbOrp55Okjo7+zaGbl2AWgOcHSudWUxjVstEgy1fBl2urtH2BbR89Z7QFH+k4DUBIrYum8GlqbakcvBAtmVGoeYYmkC63hK+BLZLg7h3r7x54Sbutm2wtjw0bXlMXPSgOtak7QUF2a51KLpE8pHSk4Zsbc2nLm1pZz4/K5AAwNO7uP9IV3AMHdwOeS5hFuTJSHaaBaWpvrWN5YQ0WJPhYSTfrPlkiLFRlL5tSwZE4Nb/21YNnJM4PsOBA+JdwRpIN+sC2YQWzkKeHVLWN3AXpKWKJCKR0RoLPvNFvD4L8pbBPo7R8EgqeElyc9JbyqOcG82jJ1DZUpIycpHTP7HeCjwFLgandfn/Teh4HbgSHgve7+k3T2JZJNDdWlXL90NtcvHXtK+PkXT7I57Bq6paObL/3X80ldQ0tpba4dvQtQ11CZDtJN6WwH3gx8IXmhmS0DbgWuAOYBD5vZZe4+lOb+RHLCzFg4s5KFMytHnxI+MzjM04d6zxo6+uFd6hoq00daAd/ddwGpbm1vAb7p7qeBfWa2B7gaeDKd/YnkU0m8iJVNwUxgb39ZsCx5GsktHT08seco/7HpBQCKY8ayuTWjdwCrmhMsmqmuoZI/2Wq0bQR+mfS6I1z2EmZ2B3AHQEtLS5aKI5IdqaaRPNjTz9bRUUO7+M6GDr785H4AqsviYfAP0kGtzQlm1WgCGcmN8wZ8M3sYmJPirY+4+/cm+rUUy1K2Drv7XcBdEDTanq88IlOZmTEvUc68xNldQ/d2Hh9tDN7S0c0XHt/LYNg3dG5t2WgaaFU4aqjmEpZsOO9/lbvfMIntdgDNSa+bgAOT2I7ItBcrMhbPrmbx7Gp+d03wsegfGGLHgd7RJ4S3dATzB8BLRw1d1ZTg8jnVFGvUUElTtqoRDwBfN7NPEjTaLgaeytK+RKadsuIYV82v46r5Z48aurmjm61hr6BHnj4yOmpoabyI5Y21o+mg1uYELfUV6hoqFyWtfvhm9lvAvwANQDew2d1/M3zvI8AfAYPA+939R+fbnvrhi4xxdzq6To0OG721Ixgqon8g6BqaqCgebQdobU6wsqmWGVUaKqIQaSwdkQgaHBrmmcN9wbMB4YVg95G+0aEimuvLRy8Cq5oTLJ9XS3mJuoZGnQK+SIE4cXqQ7S/0hHcBwfcXuk8BQfvB5bOrw+cDgpFDF8/SUBFRo4AvUsA6+06P9gjanGqoiHm1QdfQMB3UmNAsYtOZAr6IjBoZKiK5V9COCWYRC7qH1pKoKMlzqeVCaXhkERmVPFTEm1YHz0CODhXRMdYe8LNnjjBSB1wwo+Ksp4SvmFdDWbHaA6Yz1fBFZFRv/wDbO3rYFPYK2tLew6HefgDiRcaSudVn9Qy6pKFKQ0VMAUrpiEhGHOrpZ0vH2DSS2zp66DsdtAdUlcZZ0Vh7VqPwnBoNHZ1rSumISEbMqS1jTu0cfvOKYISV4WFn79ETo43BWzq6ufuJvQwMBZXHhupSVjUlRieRWdFUS225ho6eChTwReSiFBUZl86q4tJZVbzlqmDo6P6BIXYd7A0vAEGbwMO7Do/+zqKGyqAtoKmW1pY6DR2dJwr4IpK2suIYq1vqWN0yNlREz8kBtnSEbQEdPfwiaejoklgRS+fVsLKxlhVNtaxsqtXzATmgHL6I5MzBnlPBXMJhKmj7C70cP/3SqSRXNgXjBjXV6fmAC6EcvohMOXNry5m7opzXrQiGjh4edva9eIIt4VPC46eSrKsoZnljLSvCr1XNCeZqPuFJUw1fRKaUM4PDPHOobzQdtO2FXnYf7hudP6ChujQYK6gpuACsbNR8wqrhi8i0VBIvYkVTkNuH+cBYo/DIWEFb2rt5aOdYo/CCGRWsaEqworGG5fNqWTavRk8Kp6CALyJTXnKj8G3hsp5TA2wL00DbOnrYuL+L728Zm2epMVHOkjnBxDOXz6liZVMwp3Ahp4MU8EVkWqotP3s+YYAXj59m58FedhwIvp491MfPd3eOPiMws6qEX1tQz8qmBJfPqeKy2dXMqy0vmKeFFfBFJDJmVJXyisUNvGJxw+iygaFh9h09wcb9XTy17xjr9h0bnU4SoDhmzK4pY25tGfMS5bTUV9BcX8HKplqWzKnJx2FkjRptRaTg9Jwc4NkjfTx7uI/2Y6c41HOKgz39HOg5xYHufobCBuLW5gRvv2Y+V82vY2NbcMF4rvP46ABzdZUl/OVvXJb2haGvf4Cq0vik000aS0dEZBIGhoY50H2Knz19hK/+cj/PdZ4Yfa+mLM7SuTXEY0Fg3nWwj77+Af7s1ZfyntdcSkk8mGje3Xlq3zG+uq6Nn+06zK8vmsHbr5nPKy9reMnDZe7O8r/7CW+7Zj7/86alkyqzeumIiExCcayI+TMqeefLF/KOaxfw5N4Xef7oSVa3JLh8dvWpiqo9AAAHSUlEQVRZ+f5jJ87w99/fwZ2P7Ob+TR3MrSkHoPP4afYdPUFNWZzXLpvNE3te5J1f+hXN9eV89fZfZ/6MytFtdJ8c4MSZIWbXlGX92BTwRUQmYGZce8lMrr0k9fv1lSV8+tbV3Nw6jy/9134GwgfG5s+o4E9fdQlvXDWP8pIYZwaH+dH2g7zvm5v5yY5D3PHKsQ12dAXTUTbVlWf9eBTwRUTStHbJbNYumT3h+yXxIm5pbeTvHthB27GTZ73X0RW8zkXAL8r6HkREBIDmugrajp06a9lYDb8i6/tXwBcRyZGW+go6UtTwq8viOZkzQAFfRCRHmusr6Og6NdrtE4Iafi5q96CALyKSMy31FZwZGuZwOE8wjAT87OfvQQFfRCRnmuuDwD7ScOvudHSdVMAXEYmalvogdTMS8Ef64CulIyISMfMS5RQZow23ueyDDwr4IiI5UxwrYl6ifLSGn8s++KCALyKSU0Ff/PE1fKV0REQip6V+7OGrXPbBBwV8EZGcaplRwdHjpzl1ZiinffBBAV9EJKeaw5467V0nc9oHH9IM+Gb2f8zsaTPbamb/YWaJpPc+bGZ7zOwZM/vN9IsqIjL9jXTN3P/iyZz2wYf0a/gPAcvdfSXwLPBhADNbBtwKXAHcCPyrmcXS3JeIyLTXHAb4bR3dOe2DD2kGfHf/qbsPhi9/CTSFP98CfNPdT7v7PmAPcHU6+xIRiYL6yhIqS2I8ufdFIHddMiGzOfw/An4U/twItCe91xEuewkzu8PM1pvZ+s7OzgwWR0Rk6jEzmusr2NTWDUyxgG9mD5vZ9hRftySt8xFgEPjayKIUm0o5ea673+Xua9x9TUNDQ6pVREQipaW+gsFwxMxcpnTOO+OVu99wrvfN7DbgDcD1PjYjegfQnLRaE3BgsoUUEYmSkZ46ueyDD+n30rkR+CBws7snj+r/AHCrmZWa2UJgMfBUOvsSEYmKkZ46uazdQ/pz2n4WKAUeMjOAX7r7u919h5l9G9hJkOp5j7sPpbkvEZFIGAv4ucvfQ5oB390vPcd7Hwc+ns72RUSiqDlPAV9P2oqI5FhzfTmNiXKubKnL6X7TTemIiMhFKo3H+MWH1uZ8v6rhi4gUCAV8EZECoYAvIlIgbOxZqfwzs05g/yR/fSZwNIPFmS4K8bgL8ZihMI+7EI8ZLv6457v7eYcqmFIBPx1mtt7d1+S7HLlWiMddiMcMhXnchXjMkL3jVkpHRKRAKOCLiBSIKAX8u/JdgDwpxOMuxGOGwjzuQjxmyNJxRyaHLyIi5xalGr6IiJyDAr6ISIGIRMA3sxvN7Bkz22NmH8p3ebLBzJrN7FEz22VmO8zsfeHyejN7yMx2h99zOxpTjphZzMw2mdmD4euFZrYuPO5vmVlJvsuYSWaWMLP7zOzp8Jy/rBDOtZl9IPz/3m5m3zCzsiieazO7x8yOmNn2pGUpz68FPhPGt61mduVk9zvtA76ZxYDPAa8DlgG/Z2bL8luqrBgE/tLdlwLXAO8Jj/NDwCPuvhh4JHwdRe8DdiW9/gTwqfC4u4Db81Kq7LkT+LG7LwFWERx7pM+1mTUC7wXWuPtyIAbcSjTP9ZeAG8ctm+j8vo5gEqnFwB3A5ye702kf8IGrgT3uvtfdzwDfBG45z+9MO+5+0N03hj/3EQSARoJjvTdc7V7gTfkpYfaYWRPweuCL4WsD1gL3hatE6rjNrAZ4JXA3gLufcfduCuBcE4zgW25mcaACOEgEz7W7/xw4Nm7xROf3FuDLHvglkDCzuZPZbxQCfiPQnvS6I1wWWWa2AFgNrANmu/tBCC4KwKz8lSxrPg38FTAcvp4BdLv7YPg6aud8EdAJ/FuYxvqimVUS8XPt7i8A/wy0EQT6HmAD0T7XySY6vxmLcVEI+JZiWWT7mppZFfAd4P3u3pvv8mSbmb0BOOLuG5IXp1g1Suc8DlwJfN7dVwMniFj6JpUwZ30LsBCYB1QSpDPGi9K5vhAZ+3+PQsDvAJqTXjcBB/JUlqwys2KCYP81d78/XHx45PYu/H4kX+XLkpcDN5vZ8wTpurUENf5EeNsP0TvnHUCHu68LX99HcAGI+rm+Adjn7p3uPgDcD1xLtM91sonOb8ZiXBQC/q+AxWFLfglBI88DeS5TxoV567uBXe7+yaS3HgBuC3++DfhersuWTe7+YXdvcvcFBOf2Z+7+NuBR4C3hapE6bnc/BLSb2eXhouuBnUT8XBOkcq4xs4rw/33kuCN7rseZ6Pw+APxh2FvnGqBnJPVz0dx92n8BNwHPAs8BH8l3ebJ0jNcR3MZtBTaHXzcR5LMfAXaH3+vzXdYs/g1eDTwY/rwIeArYA/w7UJrv8mX4WFuB9eH5/i5QVwjnGvgY8DSwHfgKUBrFcw18g6CdYoCgBn/7ROeXIKXzuTC+bSPoxTSp/WpoBRGRAhGFlI6IiFwABXwRkQKhgC8iUiAU8EVECoQCvohIgVDAFxEpEAr4IiIF4v8DLIJ8cEF5NYsAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "run -i nt_solutions/ml_2_regression/exo3" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "## Insert your code here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Exercise 4__\n", "\n", "Compute the test error along the full regularization path. You can start by large $\\lambda$ and use a warm restart procedure\n", "to reduce the computation time. Compute the classification error.\n", "ind optimal lambda\n", "isplay error evolution." ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Lasso: 65.42184378732037%\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEQCAYAAABMXyhMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd4VGX6//H3TQq9JyAQSqjSW6SIunaxsq6KYKWJDdu6uz/dtX0t6+qua0UFlSKKgNhwRVkLWCiShB5qCCWhhtAJ6ffvjzNuxpjAhGTmZGbu13Wda+acec6cz7kCuXPa84iqYowxxviimtsBjDHGBA8rGsYYY3xmRcMYY4zPrGgYY4zxmRUNY4wxPrOiYYwxxmcBKRoiMlhENohIqog8VMrnrURkvogsF5FVInKZZ/lFIpIsIqs9r+cHIq8xxpjSib+f0xCRCGAjcBGQASQCw1V1rVebicByVX1DRLoAc1W1jYj0Bvao6k4R6QbMU9UWfg1sjDGmTIE40ugHpKpqmqrmATOAISXaKFDP874+sBNAVZer6k7P8hSghohUD0BmY4wxpYgMwDZaAOle8xlA/xJtngD+KyL3ALWBC0v5nmtwjkZy/RHSGGPMyQWiaEgpy0qeExsOTFHVF0RkIDBNRLqpahGAiHQFngMuLnUDImOBsQC1a9fue/rpp1daeGOMCQfJycn7VDX2ZO0CUTQygJZe83F4Tj95GQ0MBlDVxSJSA4gB9opIHPAJcIuqbi5tA6o6EZgIkJCQoElJSZW7B8YYE+JEZJsv7QJxTSMR6CAi8SISDQwD5pRosx24AEBEOgM1gEwRaQB8ATysqgsDkNUYY8wJ+L1oqGoBMA6YB6wDZqlqiog8KSJXeZo9CNwmIiuBD4AR6tzWNQ5oDzwqIis8UxN/ZzbGGFM6v99yG2h2esoYY8pPRJJVNeFk7eyJcGOMMT6zomGMMcZnVjSMMcb4zIqGMcaEgA27j7B8+wG/b8eKhjHGBLk1Ow4xbOJi/jJ7FYVF/r25yYqGKfbNN85kjAkay7cf4Ia3llArOpK3bkkgolppnXBUnkA8EW6CxdNPO68Xltb1lzGmqkncup+RkxNpVDua6bf1J65hLb9v04qGMcYEoYWp+xgzNYlmDWowfcwATqtfIyDbtdNTxhgTZOZv2MvIKYm0alSLmWMHBqxggB1pGGNMUJmXsptx05fRsWldpo3uT6Pa0QHdvhUNY4wJEp+v3Mn9M1fQvUV9po7qR/2aUQHPYEXDFJswwe0ExpgyzE7O4C+zV5LQuhGTRp5Bneru/Pq2omGKderkdgJjTCmm/7ydv36ymrPaxzDxlr7UinbvV7cVDVPs88+d1yuvdDeHMeZ/Ji/cwv99vpbzOsXyxk19qREV4WoeKxqm2AsvOK9WNIypEt5YsJnnvlrPJV2b8urwPkRHun/DqxUNY4ypYlSVl7/dxEvfbOLKns3599CeREW4XzAgQM9piMhgEdkgIqki8lApn7cSkfkislxEVonIZV6fPexZb4OIXBKIvMYY4xZV5Zkv1vHSN5u4tm8cL13fq8oUDAjAkYaIRADjgYuADCBRROao6lqvZo/gDAP7hoh0AeYCbTzvhwFdgebANyLSUVUL/Z3bGGMCrbBI+evHq5mZlM6IM9vw2BVdqObnvqTKKxDlqx+QqqppqpoHzACGlGijQD3P+/rATs/7IcAMVc1V1S1Aquf7jDEmpOQVFHHvjOXMTErn3vPb8/iVVa9gQGCuabQA0r3mM4D+Jdo8AfxXRO4BagO/9JjXAlhSYt0W/olpmDbN7QTGhKXjeYXc+X4yCzZk8rfLOnPbOW3djlSmQBxplFYqS3b4PhyYoqpxwGXANBGp5uO6iMhYEUkSkaTMzMwKBw5bLVs6kzEmYI7k5HPr5KV8vzGTZ//QvUoXDAhM0cgAvH8TxVF8+ukXo4FZAKq6GKgBxPi4Lqo6UVUTVDUhNja2EqOHmZkznckYExD7j+Vxw1s/s2zbAV4Z1pvh/Vq5HemkAlE0EoEOIhIvItE4F7bnlGizHbgAQEQ64xSNTE+7YSJSXUTigQ7A0gBkDk9vvOFMxhi/230oh6ETFrNxzxEm3tKXK3s2dzuST/x+TUNVC0RkHDAPiAAmqWqKiDwJJKnqHOBB4C0ReQDn9NMIVVUgRURmAWuBAuBuu3PKGBPstmdlc+M7SzhwLJ+po/oxoG1jtyP5LCAP96nqXJzbaL2XPeb1fi0wqIx1nwGe8WtAY4wJkI17jnDT2z+TV1jE9Nv60yOugduRysWeCDfGmABZmX6QWycvJTqiGrNuH0jHpnXdjlRuVjSMMSYAFm/OYszURBrVieb90QNo1dj/43n7gxUNU2z2bLcTGBOSvl67h3HTl9GqUS2mje4f0OFZK5sVDVMsJsbtBMaEnFlJ6Tz88Wq6Na/H5JH9Aj48a2WzomGKTZnivI4Y4WYKY0KCqjLhhzT+8eV6zu4Qw5s39aW2S6PtVabg3wNTeaxoGFMpioqUZ79cx1s/buHKns154bqeVWIsjMpgRcMYYypRfmER/2/2Kj5evqPK9lRbEVY0jDGmkhzPK+Su95OZvyGTP13ckbvPa49I6BQMsKJhjDGV4mB2HqOnJrF8+wH+fnV3buhf9fuROhVWNIwxpoJ2HTrOrZOWsnVfNq/f2IfB3Zq5HclvrGiYYnPnnryNMeZXNmce5ZZ3lnLoeD5TRp3Bme1C+9Z1KxqmWK3gfELVGLesTD/IyCmJVBOYMXYA3VrUdzuS31nRMMVef915vesud3MYEwR+3JTJ7dOSaVwnmmmj+tMmprbbkQIiNG4cNpVj1ixnMsac0GcrdjBqSiKtG9fmozvODJuCAXakYYwxPlNVJv6QxrNfrqdffCPeuiWB+jWj3I4VUFY0jDHGB4VFylP/WcuURVu5vEcz/j20J9UjI9yOFXABOT0lIoNFZIOIpIrIQ6V8/qKIrPBMG0XkoNdnz4tIioisE5FXJNSelDHGVHk5+YWMm76MKYu2MuaseF4d1jssCwYE4EhDRCKA8cBFQAaQKCJzPKP1AaCqD3i1vwfo7Xl/Js6Ifj08H/8E/A5Y4O/cxhgDzkN7Y6Ymkbz9AI9e0YXRZ8W7HclVgTg91Q9IVdU0ABGZAQzBGfe7NMOBxz3vFagBRAMCRAF7/Jo2nC1Y4HYCY6qUjAPZ3DppKen7j/Pa8D5c3iN0H9rzVSCKRgsg3Ws+A+hfWkMRaQ3EA98BqOpiEZkP7MIpGq+p6jr/xjXGGEjZeYgRkxPJzS9k2uh+9G/b2O1IVUIgrmmUdg1Cy2g7DJitqoUAItIe6AzE4RSf80XknN9sQGSsiCSJSFJmZmYlxQ5D//qXMxkT5n7clMnQNxcTVU2YfeeZVjC8BKJoZAAtvebjgJ1ltB0GfOA1fzWwRFWPqupR4EtgQMmVVHWiqiaoakJsbGwlxQ5D//mPMxkTxj5KzmDk5ERaNqrFx3cNomPTum5HqlICUTQSgQ4iEi8i0TiFYU7JRiLSCWgILPZavB34nYhEikgUzkVwOz1ljKl0qsr4+ak8+OFK+rdtxKw7Bgb1WN7+4vdrGqpaICLjgHlABDBJVVNE5EkgSVV/KSDDgRmq6n3qajZwPrAa55TWV6r6ub8zG2PCS2GR8vicNby3ZDu/79Wc568NnZH2KltAHu5T1bnA3BLLHisx/0Qp6xUCt/s1nDEmrB3PK+SeD5bzzbo93HluO/58caeQGmmvstkT4aZYzZpuJzAmoLKO5jLm3SRWpB/kySFduWVgG7cjVXlWNEyxL790O4ExAZO69yijpiSy53AOb9zYl8HdTnM7UlCwomGMCTuLNu/jjmnJREdWY8bYAfRu1dDtSEHDioYp9tRTzuujj7qbwxg/+jApnYc/Xk18TG0mjTiDlo1s8LHysNsDTLFvv3UmY0JQUZHyr3kb+PPsVQxo25jZd55pBeMU2JGGMSbk5eQX8ufZq/h85U6GndGSp37fjagI+5v5VFjRMMaEtKyjuYydlkzytgM8dOnp3H5OW2yEhVNnRcMYE7I2Zx5l5GTnDqnXb+zDZd2tl9qKsqJhijW2TtlM6Fi8OYs73ksmKkLsDqlKZEXDFPvoI7cTGFMpZidn8PDHq2jT2O6QqmxWNIwxIaOwSHl+3nomfJ/GWe1jGH9jH+rXjHI7VkixomGKPfyw8/rss+7mMOYUHMnJ574ZK/hu/V5uGdiaR6/oYndI+YEVDVNs8eKTtzGmCtqelc3oqYmk7TvGU7/vxs0DWrsdKWRZ0TDGBLXFm7O46/1kihSmjerHme1j3I4U0qxoGGOC1vSft/PYZ2toE1Obt29JoE1MbbcjhTwrGsaYoFNQWMTTX6xjyqKtnNsplleG96ZeDbvgHQgBKRoiMhh4GWfkvrdV9R8lPn8ROM8zWwtooqoNPJ+1At7GGWdcgctUdWsgcoeduDi3ExhzUoey87l7+jJ+St3HbWfH89ClnYmwQZMCxu9FQ0QigPHARUAGkCgic1R17S9tVPUBr/b3AL29vuJd4BlV/VpE6gBF/s4ctt57z+0ExpzQ5syjjJmaRMaBbJ6/tgdDE1q6HSnsBOJIox+QqqppACIyAxgCrC2j/XDgcU/bLkCkqn4NoKpH/R/XGFMVfb8xk3HTlxEdUY0PbhtAQptGbkcKS4G4ibkFkO41n+FZ9hsi0hqIB77zLOoIHBSRj0VkuYj803PkYvzh/vudyZgqRFV5fUEqIyYvpUWDmnw2bpAVDBcF4kijtJONWkbbYcBsVS30zEcCZ+OcrtoOzARGAO/8agMiY4GxAK1atap44nC1YoXbCYz5laO5Bfxl9krmrt7NlT2b89w13akVbffvuCkQRxoZOBexfxEH7Cyj7TDggxLrLlfVNFUtAD4F+pRcSVUnqmqCqibExsZWUmxjjJu27DvG1eMX8tWa3fztss68MqyXFYwqIBA/gUSgg4jEAztwCsMNJRuJSCegIbC4xLoNRSRWVTOB84Ek/0c2xrjpu/V7uG/GCiKrCdNG92eQPbBXZfi9aKhqgYiMA+bh3HI7SVVTRORJIElV53iaDgdmqKp6rVsoIn8CvhVn1JRk4C1/ZzbGuKOoSHltfiovfrORzqfVY8LNfa2H2iomIMd6qjoXmFti2WMl5p8oY92vgR5+C2eKdezodgITxo7k5PPHWSv5eu0eru7dgr9f3Z2a0XbfS1VjJwhNsYkT3U5gwlTq3qPcPi2JrVnZPH5lF0ac2caGZK2irGgYY1z135Td/HHWSqpHVuO90f0Z2M5GkKzKrGiYYmPHOq92xGECoLBIefHrjbw2P5UecfV586a+NG9Q0+1Y5iSsaJhiGze6ncCEicwjudw3YzmLNmdxfUJL/m9IV2pE2fWLYGBFwxgTUIlb93P3+8s4dDyff17bg+us/6igYkXDGBMQqsrbP27hH1+tp2XDmkwd1Y/Ozeq5HcuUkxUNY4zfHc7J588frmReyh4Gdz2N56/rYeNfBCkrGqZYr15uJzAhaO3Ow9z1fjIZB47zyOWdGX1WvN1OG8SsaJhiL73kdgITYmYlpfPop2toUCuKGWOtO/NQYEXDGFPpsvMKePyzFD5MzmBQ+8a8PKw3MXWqux3LVAIrGqbYTTc5rzaCn6mAjXuOcPf7y0jNPMq957fnvgs72nCsIcSKhimWkeF2AhPkPkxK59HP1lCneiTTRvXnrA7WO22osaJhjKmw3IJC/vrxGj5alsHAto15eVgvmtSr4XYs4wdWNIwxFZKdV8Dt05L5cdM+7r2gA/dd0MFOR4UwKxrGmFOWX1jE7dOSWZi6j+ev7cFQe7o75FnRMMUGDnQ7gQkiqsqjn67hx01WMMJJQIqGiAwGXsYZue9tVf1Hic9fBM7zzNYCmqhqA6/P6wHrgE9UdVwgMoelZ591O4EJIhN+SGNGYjrjzmtvBSOM+L1oiEgEMB64CMgAEkVkjqqu/aWNqj7g1f4eoHeJr3kK+N7fWY0xvpm7ehf/+HI9V/Roxh8vshEfw0m1AGyjH5CqqmmqmgfMAIacoP1w4INfZkSkL9AU+K9fUxq45hpnMuYElm8/wAMzV9C3dUP+dV1PqtlF77ASiKLRAkj3ms/wLPsNEWkNxAPfeearAS8Af/ZzRgOQleVMxpQhfX82t72bRNN6NZh4c18bAyMMBaJolPZniJbRdhgwW1ULPfN3AXNVNb2M9s4GRMaKSJKIJGVmZlYgqjGmLIeO5zNqSiJ5BUVMGnEGja1bkLAUiAvhGYD3VbI4YGcZbYcBd3vNDwTOFpG7gDpAtIgcVdWHvFdS1YnARICEhISyCpIx5hTlFxZx1/vJbM06xruj+tO+SR23IxmXBKJoJAIdRCQe2IFTGG4o2UhEOgENgcW/LFPVG70+HwEklCwYxhj/UlUe+WQNC1Oz+Nd1PRnYrrHbkYyL/F40VLVARMYB83BuuZ2kqiki8iSQpKpzPE2HAzNU1Y4U3HLBBW4nMFXQm9+nMTMpnXvOb8+1fePcjmNcJqH2OzohIUGTkpLcjmFMSPhi1S7unr6Mq3o25+VhvWzwpBAmIsmqmnCydoG4EG6MCULLth/ggVkrSGjdkOev7WEFwwA+FA0R6RqIIKYKuPRSZzJhb3tWNrdNTaJZ/RpMsFtrjRdfjjSm/fJGRMZ4fyAitSo9kXHP8ePOZMLa/mN53Dp5KQVFarfWmt/wpWh4H5PeVeKzHysxizHGZTn5hYyZmsiOg8d559YE2sXarbXm13wpGt5Xykue1LRrIsaEiMIi5b4Zy1mefpCXr+9FQptGbkcyVZAvt9ye5nlGYiW/LRqhdeuVMWFKVXny8xTmpezh8Su7cGn3Zm5HMlWUL0XjCSABGAnEiUgKsN4z2QDAoeSKK9xOYFwy8Yc0pi7exm1nxzNyULzbcUwVdtKi4emi439EJA7oAXQHfvBTLuOGP/3J7QTGBZ+t2MGznm7OH760s9txTBVX7ifCVTUDpz+puZUfxxgTSIs27+NPH66kf3wjXhhq3Zybk7ML2abYuec6kwkL63cf5vZpybRpXJuJNydQPdKexTAnZ0XDmDC069BxRk5OpFZ0BFNG9aN+rSi3I5kgEZAxwo0xVcfhnHxGTk7kSE4Bs24fSIsGNd2OZIKIHWkYE0byCoq4Y1oyqXuP8uZNfenSvJ7bkUyQsSMNY8JEUZHyl9krWbQ5i38P7clZHeyOeVN+VjRMsaFD3U5g/ERVeWbuOj5dsZM/X9KJP/SxcTHMqbGiYYrdVbJrMRMq3vw+jXd+2sKIM9tw17nt3I5jglhArmmIyGAR2SAiqSLym+FaReRFEVnhmTaKyEHP8l4islhEUkRklYhcH4i8YSs725lMSJmVmM5zX63nqp7NeeyKLjYuhqkQvx9piEgEMB64COehwEQRmaOqa39po6oPeLW/B+jtmc0GblHVTSLSHEgWkXmqetDfucPSZZc5rwsWuBrDVJ6v1+7hoY9XcXaHGP51nT28ZyouEEca/YBUVU1T1TxgBjDkBO2HAx8AqOpGVd3keb8T2AvE+jmvMSFh6Zb9jJu+jO5xDXjzpr5ER9rNkqbiAvGvqAWQ7jWf4Vn2GyLSGogHvivls35ANLDZDxmNCSnrdh1m9NREWjSsyeQRZ1C7ul2+NJUjEEWjtOPhsrpUHwbMVtXCX32BSDOcEQRHqmrRbzYgMlZEkkQkKTMzs8KBjQlm6fuzuWXSUmpHRzJtdH8a1Y52O5IJIYEoGhlAS6/5OGBnGW2H4Tk19QsRqQd8ATyiqktKW0lVJ6pqgqomxMba2SsTvvYdzeXmd34mr6CIaaP72dPeptIF4pg1EeggIvHADpzCcEPJRiLSCWgILPZaFg18Aryrqh8GIGt4GzHC7QSmAo7k5DNi8lJ2H87h/TED6NC0rtuRTAjye9FQ1QIRGQfMAyKASaqaIiJPAkmqOsfTdDgwQ1W9T10NBc4BGntGDwQYoaor/J07LFnRCFq5BYXcPi2ZdbuO8PYtCfRt3dDtSCZEya9/Rwe/hIQETUpKcjtGcNq3z3mNse4lgklhkXLPB8uYu3o3/x7a0572NqdERJJVNeFk7eyWClPs2mudV3tOI2ioKo9+toa5q3fzyOWdrWAYv7Mbt40JUqrKP75az/Sft3Pnue0Yc3ZbtyOZMGBFw5gg9fqCzUz4Po2bBrTiL5d0cjuOCRNWNIwJQlMXbeWf8zZwde8WPHlVN+tPygSMFQ1jgszs5Awen5PCRV2a8s9re1h/Uiag7EK4KXbnnW4nMCfx1Zpd/GX2Sga1b8yrw3sTGWF/95nAsqJhil1vPc9XZT9szOSeD5bTq2UDJt6cQI2oCLcjmTBkf6aYYunpzmSqnKSt+xk7LYn2TeoyeWQ/64DQuMb+5ZliN9/svNpzGlXKmh2HGDk5keb1azJtdD/q14xyO5IJY3akYUwVlrr3CLdMWkq9mlG8N6Y/MXWqux3JhDkrGsZUUduyjnHT20upJsJ7Y/rT3HqsNVWAFQ1jqqCMA9nc8NbP5BYU8t6YfsTH1HY7kjGAFQ1jqpxdh45zw1s/cyQnn2mj+3P6afXcjmTM/9iFcFPswQfdThD29h7O4ca3fmb/sTzeG9Ofbi3qux3JmF+xomGKXXml2wnCWtbRXG58+2d2H85h2uh+9GrZwO1IxvyGnZ4yxTZscCYTcAeO5XHj2z+TfiCbSSPOoG/rRm5HMqZUASkaIjJYRDaISKqIPFTK5y+KyArPtFFEDnp9dquIbPJMtwYib9i6/XZnMgF16Hg+N0/6mbR9x3jrlgQGtG3sdiRjyuT301MiEgGMBy4CMoBEEZmjqmt/aaOqD3i1vwfo7XnfCHgcSAAUSPase8DfuY0JhCM5+dw6aSkbdh9h4s0JnN0h1u1IxpxQII40+gGpqpqmqnnADGDICdoPBz7wvL8E+FpV93sKxdfAYH+EzDqayxNzUkjett8fX2/MbxzLLWDUlETW7DjEazf04bzTm7gdyZiTCkTRaAF4d2iU4Vn2GyLSGogHvivvuhVVIyqCaUu28d36vf74emN+5XheIWOmJpG87QAvD+vNJV1PczuSMT4JRNEorbN/LaPtMGC2qhaWZ10RGSsiSSKSlJmZeUoha1ePpFuL+izdYkcaxr+O5xUyemoiS7Zk8cLQnlzeo5nbkYzxWSCKRgbQ0ms+DthZRtthFJ+a8nldVZ2oqgmqmhAbe+rnhPvHN2Jl+iFy8gtP3jgUPfKIMxm/yc5zTkktScvi30N7cnXvOLcjGVMugSgaiUAHEYkXkWicwjCnZCMR6QQ0BBZ7LZ4HXCwiDUWkIXCxZ5lf9GvTiLzCIlakHzx541B04YXOZPzil4Lx85Ys/j20lxUME5T8XjRUtQAYh/PLfh0wS1VTRORJEbnKq+lwYIaqqte6+4GncApPIvCkZ5lfnNGmESKE7ymqFSucyVS67LwCRk5OZOmW/bx4fS9+39svl+aM8buAPBGuqnOBuSWWPVZi/oky1p0ETPJbOC/1a0XRqWnd8C0a99/vvNp4GpXqWG4BI6ckkrTVKRhDelnBMMHLnggvYUDbxiRvOxC+1zVMpTqW6xxh/HKXlBUME+ysaJTwu06xHM8vZElalttRTJA7mlvAiMlLSd5+gJeH9eLKns3djmRMhVnRKGFg28bUjIqw5zVMhRzNLWDEpKUs236QV4b15ooeVjBMaLCiUUKNqAgGtY/h23V78bomb4zPfukaZHm6UzDsOQwTSqxr9FJc2LkJ36zbw4Y9R8JrAJy//93tBEHvYHYet05aSsrOw7w6vDeXdbeCYUKLFY1SnO/pA+irNbvDq2iceabbCYLavqO53PzOUjbvPcobN/Xloi5N3Y5kTKWz01OlaFKvBme1j+HDpAwKi8LoFNWiRc5kym3P4Ryun7CYLfuO8vatCVYwTMiyolGGG/q3YsfB4/yw8dT6sgpKf/2rM5lyyTiQzdAJi9l9KIepI/txTkfr3tyELisaZbiwc1Ni6kQzfel2t6OYKmzLvmMMfXMxBzxjeve3AZRMiLOiUYboyGpc27cl363fy+5DOW7HMVXQpj1HGDphMTkFRUy/bQC9WzV0O5IxfmdF4wSG92tJYZEyKyn95I1NWFmz4xDXT1wCwMyxA+jWor7LiYwJDCsaJ9C6cW3Oah/DzMT08Logbk5o+fYD3PDWEmpEVmPW7QPp0LSu25GMCRgrGicxvF8YXRB/6SVnMmVavDmLm97+mQa1opl1x0DiY2q7HcmYgLKicRIXdWlKTJ3qTFuyze0o/terlzOZUv03ZTe3Tl5KswY1mXX7QOIa1nI7kjEBZ0XjJKIjq3Fj/1Z8t34vW/YdczuOf33zjTOZ35idnMGd7y+j82l1mXX7QE6rX8PtSMa4woqGD27s34qoCGHqoq1uR/Gvp592JvMrb/+Yxp8+XMmAto14/7YBNKod7XYkY1wTkKIhIoNFZIOIpIrIQ2W0GSoia0UkRUSmey1/3rNsnYi8IiISiMzemtSrweXdmzE7OYMjOfmB3rxxiaryz3nrefqLdVza7TQmjTiDOtWt5x0T3vxeNEQkAhgPXAp0AYaLSJcSbToADwODVLUrcL9n+ZnAIKAH0A04A/idvzOXZsSgeI7mFvBRcoYbmzcBVlik/O3TNYyfv5lhZ7TktRv6UD0ywu1YxrguEEca/YBUVU1T1TxgBjCkRJvbgPGqegBAVX8ZzEKBGkA0UB2IAvYEIPNv9GrZgN6tGjB18TaK7PbbkJZXUMS9M5Yz/eft3PG7djz7h+5EVAv4Aa4xVVIgikYLwPvpuAzPMm8dgY4islBElojIYABVXQzMB3Z5pnmqui4AmUs14sw2bNl3jO/D4fbbMHUst4DRUxP5YtUuHr70dB669HRcOCNqTJUViBO0pf2PK/mneiTQATgXiAN+FJFuQAzQ2bMM4GsROUdVf/jVBkTGAmMBWrVqVXnJS7i0WzP+Xm8db/2Yxnme7tNDyoQJbidw1cHsPEZOSWRl+kGev6YHQ89o6XYkY6qcQBxpZADe//vigJ2ltPlMVfNVdQuwAaeIXA0sUdWjqnoU+BIYUHIDqjpRVRNUNSE21n89jEZHVmPUoHgWbc5iVcZR4i8qAAAQlUlEQVRBv23HNZ06OVMYyjiQzbVvLiZlx2Fev7GvFQxjyhCIopEIdBCReBGJBoYBc0q0+RQ4D0BEYnBOV6UB24HfiUikiEThXAR37fQUOF2m160RyZvfb3Yzhn98/rkzhZm1Ow/zh9cXsedwDlNH9WNwt9PcjmRMleX3oqGqBcA4YB7OL/xZqpoiIk+KyFWeZvOALBFZi3MN48+qmgXMBjYDq4GVwEpVdfW3Wt0aUdw8oDVfrtlNWuZRN6NUvhdecKYwsjB1H0MnLCaimjD7jjMZ2M66NjfmREQ1tO4ESkhI0KSkJL9uI/NILoOe+45r+rTg2T/08Ou2Aurcc53XBQvcTBEwny7fwZ9nr6RtTB2mjDqDZvVruh3JGNeISLKqJpysnT0Rfgpi61bnur5xfJS8g72HbayNYKOqvLFgM/fPXEHf1g2ZdcdAKxjG+MiKxikae05bCoqKeOenLW5HMeVQWKQ8MSeF575az5U9mzN1VD/q14xyO5YxQcOKxilq3bg2V/VszruLt5F5JNftOMYHx/MKufv9ZUxdvI2x57Tl5et72VPexpSTFY0KuPeCDuQWFIbOnVTTpjlTCNpx8DjXTVjEvLW7eeyKLvz1ss5Us6e8jSk3KxoV0Da2Dn/oE8d7S7aFxjjiLVs6U4hJ3LqfIa/9xLZ92bxzawKjzop3O5IxQcuKRgXde34HCouU8fNT3Y5ScTNnOlOIUFUmL9zCDW8toW6NKD65+0zOP72p27GMCWpWNCqoVeNaXJcQx4zE7WQcyHY7TsW88YYzhYBD2fnc8V4y//f5Ws7pEMundw+ifRMby9uYirKiUQnGnd8BQXjl201uRzHAivSDXP7qj3y7bi+PXN6Zt29NsDukjKkkVjQqQYsGNblxQCtmJ2ewbtdht+OEraIi5a0f0rj2jUWowod3DGTM2W2tl1pjKpEVjUpy3wUdqFsjiqe/WEuoPWUfDNL3ZzP8rSU8M3cd55/ehLn3nk3vVg3djmVMyLGiUUka1Irm/gs7sDA1i2/X7T35CqZSqCozE7cz+KUfSNl5mOev6cGEm/tSv5adjjLGH2zA40p004DWTFuyjb/PXcc5HWOJjgyymjx7ttsJymXP4Rwe/ng1363fy8C2jfnndT2Ia1jL7VjGhLQg+61WtUVFVOORyzuTtu8Y7y3Z5nac8ouJcaYqrrBImbJwCxe88D0LU/fx+JVdeH9MfysYxgSAHWlUsvM6NeHsDjG8+PVGLuvejNPq13A7ku+mTHFeR4xwM8UJrc44xF8/Wc3qHYc4u0MMTw3pRpuY2m7HMiZs2JFGJRMRnhrSjbzCIh6fs8btOOUzZUpx4ahiDmbn8cScFIaM/4ndh3N4dXhv3h3VzwqGMQFmRxp+0CamNvdf2JHnvlrPV2t220hwFZBbUMi7i7bx6nebOJpbwE0DWvOnSzpRr4Zd6DbGDQE50hCRwSKyQURSReShMtoMFZG1IpIiItO9lrcSkf+KyDrP520CkbmixpwdT+dm9XjsszUczsl3O07QUVU+X7mTC//9Pc/MXUef1g358r5zeHJINysYxrjI70caIhIBjAcuAjKARBGZo6prvdp0AB4GBqnqARFp4vUV7wLPqOrXIlIHKPJ35soQFVGN567pzu/HL+TZuet59g/d3Y4UFIqKlK/X7eHV7zaxZsdhOjerx7TR3Tm7Q6zb0YwxBOb0VD8gVVXTAERkBjAEWOvV5jZgvKoeAFDVvZ62XYBIVf3aszyoBuXuEdeAMWe3ZeIPaZzXKZaLu9ppqrIUFilzV+9i/PxU1u8+QpvGtfjXdT25uncLIqwLc2OqjEAUjRZAutd8BtC/RJuOACKyEIgAnlDVrzzLD4rIx0A88A3wkKoW+j11JXnw4o4s2ryPv3y0iu5x9av2sKJz5wZ8k8dyC/hk+Q4mL9zC5sxjtG9Sh5eu78UVPZoRGWH3aRhT1QSiaJT2Z2LJfjYigQ7AuUAc8KOIdPMsPxvoDWwHZgIjgHd+tQGRscBYgFatWlVe8kpQPTKCV4f34fJXfuT+GSuYftuAMv9yzskv5ItVu1i0OYvIakKPlvW5smfzwJ3DrxW45xzSMo8ybck2ZidlcCS3gK7N6/HaDb25tFszO7IwpgoLRNHIALxH9okDdpbSZomq5gNbRGQDThHJAJZ7ndr6FBhAiaKhqhOBiQAJCQlVruOn+JjaPDWkGw9+uJJXvt3EAxd1/N9nqsr63Uf4eFkGHyZncDA7n5g60QDMTErnqf+s5dJuzRia0JL+8Y38O9rc6687r3fd5ZevP5pbwFdrdvPp8h38lLqPyGrCZd2bceuZbejTqoF1LGhMEAhE0UgEOohIPLADGAbcUKLNp8BwYIqIxOCclkoDDgINRSRWVTOB84GkAGSudNf0jWPh5n28/O0makRF0LtVA+Zv2MtXa3azLSubyGrCxV2bclP/1gxs1xiAVRmHmJWUzpwVO/lk+Q5aNarFpd1P45Kup9ErrkHlF5BZs5zXSiwauQWF/LRpH58s38E36/aQk19Ey0Y1eeDCjgzv15Im9YLo4UdjDBKIHllF5DLgJZzrFZNU9RkReRJIUtU54vyJ+QIwGCjEuVtqhmfdizyfCZAMjFXVvLK2lZCQoElJVbOu5BcWcf/MFXyxahcAkdWEM9vHMLjraVzctSkxdaqXut7xvELmpezmo2UZLN6cRUGR0rRedS7q0pQLTm/KGfGNqFO9Eur/uec6rwsWVOhrdhw8zoINe5m/PpNFm/eRnVdIw1pRXNGjOb/v3Zw+rRraUYUxVYyIJKtqwknbhVo33lW5aIBzl9Cny3dQr2YU/eIblXtwoEPH85m/3jlC+X5jJsfzC4moJnRrUZ8BbRsxIL4x3ePql1mATugUikZRkbI58yjLth9g2baDJG8/QOpe5ya3uIY1Of/0Jpx3ehPOah9DlF3YNqbKsqIRBnLyC0nedoAlaVksSctiRfpB8gudn2ds3ep0blaPzs3q0rFJXVo0rEmLBjVpWq9G2b3vnqBoZOcVsPdwLlv2HWNz5lE2Zzqv63cd5nBOAQD1a0bRu1UDzmofw7mdmtAutrYdURgTJHwtGtaNSBCrERXBoPYxDGrv9EybnVfAiu0HWbvrMGt3HWbdriMs3rzvf4UEQARi61SnQa0oalePpHZ0JLWiI6gRFcG4PUdRlOemJJJfWMSRnAKyjuWy70gex/N/fZdzg1pRtIutw+U9mtOnVQP6tG5I2xgrEsaEOjvSCHH5hUVs35/NroM57Dx0nJ0Hnenw8QKO5RVwLLeA7LxCcjynuaIiqhEZIURWq0bdGpHE1KlO49rRNK5TnZg60bSJqU272Do0qh3t9q4ZYyqRHWkYwOnOpF1sHdrF1nE7ijEmBNiVSWOMMT6zomGMMcZnVjSMMcb4zIqGMcYYn1nRMMYY4zMrGsYYY3xmRcMYY4zPrGgYY4zxWcg9ES4imcC2U1w9BthXiXHcZPtSNYXKvoTKfoDtyy9aq2rsyRqFXNGoCBFJ8uUx+mBg+1I1hcq+hMp+gO1LednpKWOMMT6zomGMMcZnVjR+baLbASqR7UvVFCr7Eir7AbYv5WLXNIwxxvjMjjSMMcb4LCyLhogMFpENIpIqIg+V8nl1EZnp+fxnEWkT+JS+8WFfzhGRZSJSICLXupHRFz7sxx9FZK2IrBKRb0WktRs5feHDvtwhIqtFZIWI/CQiXdzI6YuT7YtXu2tFREWkyt6F5MPPZYSIZHp+LitEZIwbOX3hy89FRIZ6/s+kiMj0Stu4qobVBEQAm4G2QDSwEuhSos1dwJue98OAmW7nrsC+tAF6AO8C17qduQL7cR5Qy/P+ziD/mdTzen8V8JXbuU91Xzzt6gI/AEuABLdzV+DnMgJ4ze2slbQvHYDlQEPPfJPK2n44Hmn0A1JVNU1V84AZwJASbYYAUz3vZwMXSNUc/Pqk+6KqW1V1FVDkRkAf+bIf81U12zO7BIgLcEZf+bIvh71mawNV9cKiL/9XAJ4CngdyAhmunHzdl2Dgy77cBoxX1QMAqrq3sjYejkWjBZDuNZ/hWVZqG1UtAA4BjQOSrnx82ZdgUN79GA186ddEp86nfRGRu0VkM84v23sDlK28TrovItIbaKmq/wlksFPg67+xazynQGeLSMvARCs3X/alI9BRRBaKyBIRGVxZGw/HolHaEUPJv/R8aVMVBEvOk/F5P0TkJiAB+KdfE506n/ZFVcerajvg/wGP+D3VqTnhvohINeBF4MGAJTp1vvxcPgfaqGoP4BuKzzZUNb7sSyTOKapzgeHA2yLSoDI2Ho5FIwPw/gsiDthZVhsRiQTqA/sDkq58fNmXYODTfojIhcDfgKtUNTdA2cqrvD+TGcDv/Zro1J1sX+oC3YAFIrIVGADMqaIXw0/6c1HVLK9/V28BfQOUrbx8/R32marmq+oWYANOEamwcCwaiUAHEYkXkWicC91zSrSZA9zqeX8t8J16riZVMb7sSzA46X54ToNMwCkYlXZ+1g982Rfv/7yXA5sCmK88TrgvqnpIVWNUtY2qtsG51nSVqia5E/eEfPm5NPOavQpYF8B85eHL//tPcW4eQURicE5XpVXK1t2+E8Cluw8uAzbi3IHwN8+yJ3H+wQPUAD4EUoGlQFu3M1dgX87A+avjGJAFpLid+RT34xtgD7DCM81xO3MF9uVlIMWzH/OBrm5nPtV9KdF2AVX07ikffy7Pen4uKz0/l9PdzlyBfRHg38BaYDUwrLK2bU+EG2OM8Vk4np4yxhhziqxoGGOM8ZkVDWOMMT6zomGMMcZnVjSMMcb4zIqGMcYYn1nRMMYY4zMrGsYEmIh08Yzd0FJE6rqdx5jysKJhzAmISHcR2SYid5by2QQRGSQiC7wH6hKRO0Xkda/5p0VkmteqUcA9wNXAUR8y/Or7jXGTFQ1jTkBVV+P07XNLKR/3x+lvqaSpwJUi0kBErsDpX2qs1+ctgck43dTYkYYJKlY0jDm5vUBX7wUi0hnYqKqFJRurM1jUB8AzwCs4IyYe9/r8P8BsVZ2rngGZPEc0C72+v4+IfOeXvTGmAiLdDmBMEPgHUF1EWqvqNs+yS4GvTrDOJJxeUoeo6uaSH6rq7hKLUoB2IhLhKUQvEBzjVJgwY0caxpyAZ8Sz2sAX/Ppo4xJOXDQeAzLx8Q8zVS3CKRxdReQaYLuqLjul0Mb4kRUNY8ogIjVwhmO9C6d76W6e5bWABqpa6uBKIvIgTvf6Q4H7yrHJJcAg4Angr6cc3Bg/stNTxpTtEeBdVd0qIqtxBuYBZ3Cb+aWtICLnAyOBgap6RETqiUgvVV3hw/aWAFOA8aq6o5TvFpwxE37GGbO+jqqOL+9OGVMRdqRhTClEpBNwEfCSZ9H/jjQo43qGiLQC3gauU9UjnsUvA/f7uNn1QC7wXBmf9wF+wBkjPRooqKxxn43xlQ3CZEw5icgyoL+q5nvmFwAjVHVrBb/3NSBRVaeWWL4AGAEcwhm7ehPQAudMwU2e6yHGBISdnjKmnFS1T2V+n4i0w7nQvrBkwSix3QM4Y9Yb4xorGsZU3BTg4Kmu7Lkl93R/fb8xlclOTxljjPGZXQg3xhjjMysaxhhjfGZFwxhjjM+saBhjjPGZFQ1jjDE+s6JhjDHGZ1Y0jDHG+MyKhjHGGJ/9fzu72YsE1XXBAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "run -i nt_solutions/ml_2_regression/exo4" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [], "source": [ "## Insert your code here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Exercise 5__\n", "\n", "Display the regularization path, i.e. the evolution of $w$ as a function\n", "of $\\lambda$.\n", "lot(lambda_list, W', 'LineWidth', 2);" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEQCAYAAABIqvhxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd4lFX+///nPSWTmcykk4SEFAJIDYQuVVAIiIAoIIoiyAIqur9dV/iq+1mVtexaWN1dFVFRcSmKvYBKBOmCdAUpUhIgAUIK6W3K+f1xJ5MKJJDJTJLzuK77GjJzzz0nKHnl3Oec91GEEEiSJElSfWjc3QBJkiSp6ZHhIUmSJNWbDA9JkiSp3mR4SJIkSfUmw0OSJEmqNxkekiRJUr3J8JAkSZLqTYaHJEmSVG8yPCRJkqR6k+EhSZIk1ZvO3Q1wleDgYBETE+PuZkiSJDUpe/bsyRBCtLrSec02PGJiYti9e7e7myFJktSkKIpyqi7nydtWkiRJUr3J8JAkSZLqTYaHJEmSVG/NdsxDkqSmzWq1kpKSQnFxsbub0ix5e3vTpk0b9Hr9Vb1fhockSR4pJSUFi8VCTEwMiqK4uznNihCCzMxMUlJSaNu27VVdQ962kiTJIxUXFxMUFCSDwwUURSEoKOiaenUyPKqx2R18ticFuT2vJLmfDA7Xuda/Wxke1Xx38DyPfvILCxOPurspkiRJHkuGRzVju7fmrn6RvLHhBMt31GmtTPOzbp16SFILl5ycjNFoJD4+HlAXH5c/361bt0Zpw7Bhw5wLnss/v6ioiPj4eLy8vMjIyGiUdlQnB8yrURSFZ2/txvmcYp766iBhvt6M6BLq7mY1rueeUx9HjHBvOyTJA7Rr1479+/e7uxlVGI1G9u/fjztLMMnwqIVOq+H1qb24650dPPzhXj64rx/9Y4Pc3SxJarH+/s1vHDqb26DX7BLuy9PjutbrPa1a1Sz5ZLfbeeyxx1i7di2KojB79mz++Mc/8swzz/DNN99QVFTEwIEDeeuttzhy5AjTp09n586dgNqDGT9+PL/++ivr169n3rx52Gw2+vbty5tvvonBYLji57uLvG11CT4GHe9O70uEv5Hp7+9k8+/p7m6SJElutmvXrhrPvf322yQlJbFv3z5+/fVX7r77bgAefvhhdu3axcGDBykqKmL16tV07tyZ0tJSTp48CcCqVau44447KC4uZsaMGaxatYoDBw5gs9l488036/T57iJ7HpfRymJg1f0DmPbuTmZ9sJs37u7FyJZ2C0uSPEB9ewiNad26dTzwwAPodOqP08DAQAA2bNjASy+9RGFhIVlZWXTt2pVx48Zxxx138PHHH/P444+zatUqVq1axdGjR2nbti3XXXcdANOnT+eNN97gz3/+s9u+ryuRPY8rCDYb+HB2fzq3tvDg8j1888tZdzdJkiQPIoSoMe21uLiYuXPn8umnn3LgwAFmz57tXFMxZcoUPv74Y37//XcURaFDhw5NcmmADI868Dd5sXxWf3pFBfCnj/bx6Z4UdzfJtd56Sz0kSbqihIQEFi9ejM1mAyArK8sZFMHBweTn5/Ppp586z2/Xrh1arZZnn32WKVOmANCpUyeSk5M5fvw4AMuWLeOGG25o5O+kfmR41JHFW8/SmX0Z1D6YeZ/8wrLmPI23Y0f1kCTpimbNmkVUVBTdu3enR48erFy5En9/f2bPnk1cXBwTJkygb9++Vd4zZcoUli9fzh133AGodabef/99Jk+eTFxcHBqNhgceeMAd306dKU2xu1QXffr0Ea7YDKrYaufhlXtZd/gC/zemM7OHxjb4Z7jdN9+oj+PGubcdUot2+PBhOnfu7NY2JCcnM3bsWA4ePOjWdlxK+aZ3wcHBV/X+2v6OFUXZI4Toc6X3yp5HPXnrtbx5T29uiWvN898e5r/rjzXJ+5WX9a9/qYcktXBarZacnBznIkFPUb5I0Gq1otG458e4nG11FfRaDf+5Mx6DXsMrP/xOYamdx0Z3lHV4JKmZiYyM5MyZM+5uRg3liwTdSYbHVdJpNSyc1AOjXsviTScottp5amwXNBoZIJIkNX8yPK6BRqPw3IRuGPValmxNoqjUzj9uj0MrA0SSpGZOhsc1UhSF/7ulMyYvLf/98TjFNjsLJ/dAr5XDSZIkNV/yJ1wDUBSFvyR05P+N7shX+8/y8Mq9lNjs7m7W1Vu2TD0kqYVzdVXdgQMHXvGcmJiYWivnbty4kZ9++sn59YIFC1i6dCkA8+fPJywsjIULF15zGy9F9jwa0Nxh7THqtfz9m0PM+d8e3prWG2+91t3Nqr/ISHe3QJI8hiur6lb+4V9fGzduxGw21xpAL7/8Mj4+PtfStCuSPY8Gdt+gtrxwexybj6Vz3/u7KCixubtJ9bdqlXpIklRFbVVtx4wZw6+//gpAz549eeaZZwB48sknWbJkCaD+MO/bty/du3fn6aefdr7XbDYD4HA4mDt3Ll27dmXs2LGMGTOmyqr01157jV69ehEXF8eRI0dITk5m8eLFvPrqq8THx7NlyxbMZjNGo9Fl33t1HtHzUBRlNPAfQAssEUK8UMs5dwALAAH8IoSY2qiNrIc7+0Xhrdfy6Ce/MO3dn3n/vn74GfXublbdlVfzLCudIElu993jcP5Aw14zLA5urvGj5rJqq2o7dOhQtmzZQkxMDDqdjm3btgGwdetW7rnnHhITEzl27Bg7d+5ECMH48ePZvHkzQ4cOdV7j888/Jzk5mQMHDnDhwgU6d+7MzJkzna8HBwezd+9eFi1axMKFC1myZAkPPPAAZrOZefPmATBkyJCr+Vu4am7veSiKogXeAG4GugB3KYrSpdo5HYAngEFCiK6A55aaLDOhZwRvTO3JgdQc7l6yg6yCUnc3SZIkFxgyZAibN29m69at3HLLLeTn51NYWEhycjIdO3YkMTGRxMREevbsSa9evThy5AjHjh2rco2tW7cyefJkNBoNYWFhDB8+vMrrt99+OwC9e/cmOTm5sb61y/KEnkc/4LgQ4iSAoigfAbcChyqdMxt4QwhxEUAIcaHRW3kVRndrzdvTtDywfA93vr2d5bP6E2LxdnezJKnpqWcPoTH17duX3bt3Exsby8iRI8nIyOCdd96hd+/egFp194knnuD++++/5DWuVKWifFMorVbrLMDobm7veQARQOUlnCllz1V2HXCdoijbFEXZUXabq0kY3imE92f0JeViEVPe2sHZ7CJ3N0mSpAbk5eVFZGQkH3/8Mddffz1Dhgxh4cKFzttIo0aN4r333iM/Px+A1NRULlyo+vvv4MGD+eyzz3A4HKSlpbFx48Yrfq7FYiEvL6/Bv5+68oTwqG1FXfUY1gEdgGHAXcASRVH8a1xIUeYoirJbUZTd6emes/PfwPbBLPtDPzLySpi8eDunMwvd3SRJkhrQkCFDCA0NxWQyMWTIEFJSUpzhkZCQwNSpUxkwYABxcXFMmjSpxg/9iRMn0qZNG7p168b9999P//798fPzu+xnjhs3ji+++MI5YN7Y3F5VV1GUAcACIcSosq+fABBC/LPSOYuBHUKIpWVfrwceF0Jcck9GV1XVvRYHUnKY9t7PGHQaVszqT/sQi7ubVLvyOeVXWalTkhpCS6uqm5+fj9lsJjMzk379+rFt2zbCwsKu+noLFiyoMqBem6ZeVXcX0EFRlLaKongBdwJfVzvnS2A4gKIowai3sU42aisbQFwbPz6acz12B9y+6Cc2HLngmRV5g4NlcEgSjVtVd+zYscTHxzNkyBCefPLJawqO+fPns3z5cpeu9XB7zwNAUZQxwL9Rp+q+J4R4XlGUZ4DdQoivFbVc7b+A0YAdeF4I8dHlrumJPY9yZ7IKmbNsD4fP5eLrraNdiJnYYDPtQnxo18pMu1Y+RAX64KVzU7aXrVJlxgz3fL4k4Rk9j+buWnoeHhEeruDJ4QFQVGrn070p/H4+jxPp+ZxIzyctt8T5ulajEGIxYPTSYvLSYtRrMXrpMOm1GL3Uo/Y/VzvH+V4tJi8dRr32yoUbhw1TH+swaCdJriLDw/WuJTw8Yapui2T00jLt+ugqz+UVW0nKKFDD5EIB53OLKbLaKSpVj5wiK+dzipzPFZbaKbLaqW/+e+k01UKlajjNu5CPRqOwcvUh14STJElNngwPD2Lx1tO9jT/d29SYSHZJQghKbA5nkBSV2tQ/l9oprBQ8hWWvFZU6KLTaqj1fNZzySmw4HIKVO0+7JJxc3nOSJMnlZHg0cYqi4K3XNmwBxq/U8Dr0zOirDqfisp5RodXu/HNOkZW0nOIa4dWY4WTy0hHq6010kIkQi0Hu/ihJV0mGh3RZLgmnSsrDqUYAldqqBE/VPzdMOHnrNUQH+hAVZCI60ER0kImoIB+iA01EBBjlniwSycnJdO7cmY4dO7J//35iYmIavDzIggULiImJYcaMGcyfP59ly5Yxb968y06x9QQyPKSavv220T6qcjgFuOD61cOpsMTG2ZxiTmcWcCqzkFNZhZzKLGDLsXSKrQ7n+7QahXB/b2e4xASZiAr0UQMm0ISPQf7TaSlcWZK9usYopd5Q5L8AqSaTyd0taDC1hVOHUAtQtbS2EIILeSVqoGQWcDqr0Bku3x04x8VCa5Xzg80Gost6LFFBZb2WsnAJ8vGSt8OaqfKS7Pn5+dx6661cvHgRq9XKc889x6233grAs88+y4oVK4iMjCQ4OJjevXszb948Tpw4wUMPPUR6ejomk4l33nmHTp06NXop9YYiw0OqadEi9XHuXPe2oxEpikKorzehvt70axtY4/WcIiunMws5laX2WMr/vONkJl/sT61ya8xs0BHlvA1mIrpSjyXc3ygH/K/Ciztf5EjWkQa9ZqfATjzW77F6vae8JLu3tzdffPEFvr6+ZGRkcP311zN+/Hj27NnDZ599xr59+7DZbPTq1ctZIHHOnDksXryYDh068PPPPzN37lx+/PFHj789dSkyPKSaPv5YfWxB4XElfkY9cW38iGtTs95QsdVOysUiTpcFS3nv5WhaHusPX6DUXnE7TK9VaBNgqgiXQBMxQWq4RAaamubOky2QEIK//vWvbN68GY1GQ2pqKmlpaWzdupVbb73V2ZMYN24coPZUfvrpJyZPnuy8RklJSa3XbipkeEjSNfLWa2kfYqZ9iLnGa3aH4HxusXorrOw2WHmvZe/pi+QVVy2vHebrXesAfnSQCX+TV2N9Sx6nvj0EV1uxYgXp6ens2bMHvV5PTEwMxcXFlyw35HA48Pf3b7Sxk8Ygw0OSXEirUYjwNxLhb2Rgu6qvCSG4WGitOsaSWcjprAI2/Z7Ohbyqv5n6euuILuulRJfdDisfbwm1eKORt8MaTU5ODiEhIej1ejZs2MCpU6cAtbT6/fffzxNPPIHNZmPNmjXMnj0bX19f2rZtyyeffMLkyZMRQvDrr7/So0cPN38nV0+GhyS5iaIoBPp4EejjRc+omnPNikrtZaFSPjNMfTyQmsP3B89jc1T8lmvQaYgMrDSAH2giOkgNl8gAk/vqpDVTd999N+PGjaNPnz7Ex8fTqVMnQN0Yavz48fTo0YPo6Gj69OnjLK2+YsUKHnzwQZ577jmsVit33nmnDA9Jkhqe0UtLxzALHcNqlu632R2czS6uGMDPKiQ5Q+3B/HQikyKr3XmuRoHWfkZnj6XylOPoIBMWb31jflvNQnBwMNu3b6/1tXnz5rFgwQIKCwsZOnQojz76KABt27bl+++/b8xmupQMD6kmWRDR4+m0GqLKZnMN6VD1NSEE6fkl6tiKc5ylgFNZhST+lkZmQWmV8wN9vMoG7quOsUQFmWhlbtmr8CuXZK/reMWcOXM4dOgQxcXFTJ8+nV69etX58+bPn88XX3zhDBxPJqvqSlILk1dsrTHGUv7nczlFVLobhslLS1RgpdlhlcIlwt+IzoWr8GVVXdeTVXWlhrVwofrYROefS5dn8dbTNdyPruE1px2X2hykXKyYFZZcNkvsZEYBG39Pp9RWMe1Yp1GICDA6gyU60Ieu4b70ig6QU45bABkeUk2rV6uPMjxaHC+dhthWZmJb1Zx27HAI0vKKqyySLB9v+Xr/WXLLph0bdBr6xgQyqH0wg9sH0yXcVy6MbIZkeEiSVCcajUJrPyOt/YxcHxtU4/WLBaXsO3ORrccy2XY8gxe/P8KLqAssB7YLYmBZmMQEmVr0OEpzIcNDkqQGEeDjxY2dQrmxUygAF/KK2X4ik63HMth2PIPvDp4HKFvzEsTgDsEMbBdMK4vBnc2WrpKc/C1JkkuEWLy5NT6Clyf3YNvjN7Jh3jCendCN7m38SDyUxp8+2k/f59cx6tXNPPPNIX48kkZ+ie3KF25EycnJGI1G4uPjAYiJiXE+361bNwCWLl3Kww8/XK/rxsTEkJGRUeP5pUuXsmDBAgBeffVVoqKi6n3txiJ7HlJNTbDCp+TZFEWhbbAPbYN9mHZ9NHaH4NDZXLYeV3slK34+xXvbktBpFOIj/RnYPpgbwxw4hEDj5ltcjVmSvbJHHnmEgIAAPHXWqOx5SDV99516SJKLaDUKcW38eHBYO5bP6s8vTyewclZ/5gyNxeoQvP7jMdLzSjh0NpekjALS80ooKrVfsnZUYykvyV7dmTNnGD16NB07duTvf/87oPZOOnXqxPTp0+nevTuTJk2isLDQ+Z7XXnuNXr16ERcXx5EjasVgo9GI2VxzsoInkj0PSZLczluvZWD7YAa2DwYgp9DKiWNHCfDxIr/YRtor/+DC8WMoioJWo6DVgFZRrqlXYujcibC//rVe7ykvyV7dzp07OXjwICaTib59+3LLLbcQHBzM0aNHeffddxk0aBAzZ85k0aJFzhLswcHB7N27l0WLFrFw4UKWLFnClClTrvr7aWyy5yHV9Oyz6iFJbuJn0mP00hLhb6RjmIVAkxcGvRatRsHuEJRYHeo2xKV2SmwObA6BO/skI0eOJCgoCKPRyO23387WrVsBiIyMZNCgQQDcc889zucBbr/9dgB69+7d4FvbNgbZ85BqWr9efXzySfe2Q5LKhP/t/5x/Lt9aOL/ERn6xjYISG8Vlt7OMei1mbx1mgw4fL12jVRquPvW4/OtLPQ9gMKizzLRaLTabZ00UqAsZHpIkNSmVtxYONhsQQlBYalfDpMRGRn4p6XklKIqCyUuL2aCGiclL67L1JT/88ANZWVkYjUa+/PJL3nvvPQBOnz7N9u3bGTBgAB9++CGDBw92yee7gwwPSZKaNEVR8DHo8DHoCEXdgKuw1ObsmaTlFpOGOkbiY9A5eyYGnabBwmTw4MFMmzaN48ePM3XqVPr06UNycjKdO3fmgw8+4P7776dDhw48+OCDDfJ5nkCGhyRJzYpWo2Dx1qul5v3U8vXlvZL8Ehu52VYA9FqNs1diNujQ12PPk5iYGA4ePAjAjBkzmDFjRq3naTQaFi9eXOP5ymMcffr0YWMTrGQtw0OqKahm6QlJaqp0Wg3+Ji/nNr4lNvUWV0GxjbxiGxcL1RL1Bl2l8RKDFp1Gc1Ul2RvKq6++yuLFi5k4cWKjfm5dyZLskiR5pMYoyS6EoNhqJ7+kLFBKbDiEQAGMXjrMBm3ZeEnjDb43JlmSXZIk6SooioLRS4fRS0criwFH+eB7sXqLKz2vlAt5JWgUBbNBh59Jj6+3XlYJRoaHVJsnnlAf//lP97ZDkhpZeUiYDeqPRrvDQUFZrySnyEpulhWNomDx1uFvVMdVmmOPpC48IjwURRkN/AfQAkuEEC9c4rxJwCdAXyGEvCflKpfYm1mSWhqtRoOvUYOvUU9rP28KS+1kF1nJKbSSU6QGiZ9Rj59Rj9lb5/Y6XI3J7eGhKIoWeAMYCaQAuxRF+VoIcajaeRbg/wN+bvxWSpLU0lWeEhzu501BiY3sQis5xVYuFpai1ahB4m/U42PQNfs9S9weHkA/4LgQ4iSAoigfAbcCh6qd9yzwEiC3t5Mkya0URcHsrcfsrSdcCPKLbWQXWckutJJVUKrO8CrrkbhycaI7eUJtqwjgTKWvU8qec1IUpScQKYRY3ZgNkySpZbvUfh6VaRQFX6OeqEATXVr7EhVowsdLS2ZBKSfS8zl6Po8+/a+nqNRGUlISw4YNA2DLli106dLFuS9IU+MJ4VFbJDvnDyuKogFeBR694oUUZY6iKLsVRdmdnp7egE1sYdq0UQ9Jkuq1n4dGo+Bv8iI6yIcurS1EBpgw6LW8/9lajl3I52RGAaV2B8VWO0OGDOHbb791cetdxxNuW6UAkZW+bgOcrfS1BegGbCzr+oUBXyuKMr76oLkQ4m3gbVDXebiy0c3a8uXuboEkVbHl49/JOJPfoNcMjjQz5I7r6vWe8v08zp07x5QpU8jNzcVms/Hmm29y8OBBkpKSeOmllwBY9r//sWfPHl577TXMZjOnzmeSm6HDaPbj97Q8jHotOfklbq0GfC08oeexC+igKEpbRVG8gDuBr8tfFELkCCGChRAxQogYYAdQIzgkSZJcrXw/j5UrVzJq1Cj279/PL7/8Qnx8PJMmTeLzzz93nrtq1aoq+3MEmQ0Mju/E96u/ItzPiKIopOeVUGK1c/xCPhn5JVjtjkb/nq6W23seQgiboigPA2tRp+q+J4T4TVGUZ4DdQoivL38FqcH9+c/q47//7d52SFKZ+vYQXK1v377MnDkTq9XKhAkTiI+Px2KxEBsby44dO+jQoQNHjx517uVRmV6rIdhiINhiwJHrg06rwSEEZ7OLOJtdpC5GLBts12k94ff72rk9PACEEN8C31Z77qlLnDusMdrUorlhv2ZJakqGDh3K5s2bWbNmDdOmTWP+/Pnce++9TJkyhY8//phOnTpx2223XXGWlZdOi06jcF2ohWKrnZyyGVup2UWczS7GXLYY0deoQ6vxrCDxiPCQJElqSk6dOkVERASzZ8+moKCAvXv3cu+993L77bfz/PPPEx0dzYsvvliva5bvURJiMVBsrViMeKbYiibb81a1y/CQJEmqp40bN/Lyyy+j1+sxm83873//AyAgIIAuXbpw6NAh+vXrd1XXrlxvK8xXXdVe3iMpX9XuW7YY0Z2r2mV4SJIk1dP06dOZPn16ra+tXl1zOVp+/tXNFKu8qr11+ar2ImtZmLh3VbsMD6mm6zxrcFKS3MWV+3ls2bKFuXPnEhwcXKfzq6xq91dXtedUXtWu0eBvarxV7TI8pJreftvdLZAkjxAZGcmZM2eufOJVGDJkCAcOHLiq95bfuvI16olwCPKKrWQXWcksKCUjv4RAkxdtAk0N3OKqZHhIkiQ1YRqNgp/JCz+TF3aHg9xiG/pGmOIrw0Oqac4c9VH2QCSpSdFqNASUbbfrajI8pJp+/93dLZAkycN51qoTSZIkD3KpqrrJyclXXQ13wYIFLF26FID58+cTFhbGwoULG6K5jUr2PCRJki6jPlV16+vll1/Gx8fHJdd2NdnzkCRJqqPyqrqV2e125s2bR1xcHN27d+e1114D1F7KY489Rr9+/ejXrx/Hjx8HwGw2YzQaG7XdriB7HlJNZV10SfIUG5a+zYVTJxv0miHRsQyfMade7ymvqlvZ22+/TVJSEvv27UOn05GVleV8zdfXl507d/K///2PP//5z6xevZp585rHZqgyPGphK7Wj89K6uxnuI6vpSlKdrVu3jgceeACdTv1xGhgY6Hztrrvucj4+8sgjbmmfq8jwqCYtKZc1i35h4MT2dOwf1iz3Hpakpqa+PYTGJIS45M+Jys83t58lcsyjGoNJh3+IifVLD/Ptol8pLrC6u0mN75571EOSpCtKSEhg8eLF2Gw2gCq3rVatWuV8HDBggFva5yoyPKrxDzVx26O9GDy5A6cPZfHJP3eRfjrP3c1qXCkp6iFJ0hXNmjWLqKgounfvTo8ePVi5cqXztZKSEvr3789//vMfXn31VTe2suHJ21bVpexBWfMIPXzbEDqkJ2t39+Szl3Zzw9SOdB4Y7u7WSZLkAWJiYjh48CAAOp2OV155hVdeeaXGeQ899BBPP/10YzevUcieR218QiDrJGHHXuIO03201uznx/8dYcNLH2JL3gUOu7tbKElSI6hcVdcV5s+fz/Lly5vkWg9FCOHuNrhEnz59xO7du6/tIrZSSDuA4/cf2bmpmD3nB9FKd5zRYW/j27kndBgJ7W4Cn6CGabSnGDZMfdy40Z2tkFq4w4cP07lzZ3c3o1mr7e9YUZQ9Qog+V3qvvG11OToviOiNJqI31w+H0J1JrFvu4OO0Zxlpe5PoA7MBBSJ6q0HSfiSE9wQP22u43prZwJ4kSQ1P9jyqSU1NZd26dUyaNKnWrmT2hUK+f+sgmWfz6TvEQN/WW1GOJ0LqHkCAKRja36QGSfubwBRY80MkSboi2fNwPdnzaEB6vZ6kpCR2797NDTfcUON1/xATEx/rzaaVR9m1+TxpXUczcuZf8CYXTvwIxxLh+Dr4dRUoGrVXUh4k4T1B04IXH0qS1GzI8KgmJCSE9u3bs3PnTgYNGuRcNVqZ3kvLTdM7Exbrx5aPf+fjf+zi5vvjaNV9MnSfrA6on90Hx36A4z/Axn/Cxn+Atz/EDlODpN2N4Nem0b+/Opk4UX387DP3tkOSJI/VxG/Ou8aAAQMoKCi47BaRiqLQbWgEtz/aG+EQfPbSHg5tO6u+qNFCmz4w/AmY/SPMPwET34VOt8CZn+HrP8KrXeH1fvDd42rIlBY00ndXB5mZ6iFJLdylSrJfzrBhw7jmyTrXqLydJ06cID4+HrPZ3OCfIXsetYiNjSU0NJQtW7YQFxdXa++jXGhbX+74v7788O5vbFh2hPMnchh653VVa2P5BEHcJPUQAtKPwPH16m2uPe/Dz2+C1guiBqg9kvY3QWg3aGblDCSpKXJlSXZXK2+7K8JD9jxqoSgKN910E1lZWezcufOK5xvNXoz9Yzx9xsRw+KdzfL5wL7kZRZe6OIR0hoEPw7TP4bFkmPYF9JsDBRmw7mlYPBgWXgerpsEPT8Hu99SgyToJ9hZYLkWSPER5SXaHw8HcuXPp2rUrY8eOZcyYMXz66ac1zk9MTGTAgAH06tWLyZMnk5+fD8AzzzxD37596dZUC+b5AAAgAElEQVStG3PmzKF84tJ///tfunTpQvfu3bnzzjsBtdzJhAkT6N69O9dffz2//voroG4qNXPmTIYNG0ZsbCz//e9/a7TTlWTP4xKuu+462rdvz6ZNm4iLi8NisVz2fI1Gof/4WEJjfFm39BAf/2MXI+7rQkxc8OU/SG9UexvtblS/zj0HJzeoPZNz++H378FeWnG+olXHSgJiaj+MAbLHIjU72d+coPRsw97a9Qr3wX9cu3q9p7wk++eff05ycjIHDhzgwoULdO7cmZkzZ1Y5NyMjg+eee45169bh4+PDiy++yCuvvMJTTz3Fww8/zFNPPQXAtGnTWL16NePGjeOFF14gKSkJg8FAdnY2AE8//TQ9e/bkyy+/5Mcff+Tee+919oSOHDnChg0byMvLo2PHjjz44IPo9fpaS8c3NBkelzFq1CjeeustPvroI2bMmIFer7/ie2K6BzP5ib5899YB1iz6lT5jYuh7S1s0mjr+QPdtDfFT1QPA4YC8c3AxueZx9DsouFD1/QY/CIiuPVj8ItW1K1dy0011a6sktVBbt25l8uTJaDQawsLCGD58eI1zduzYwaFDhxg0aBAApaWlzuKIGzZs4KWXXqKwsJCsrCy6du3KuHHj6N69O3fffTcTJkxgwoQJzs/6rGzyyo033khmZiY5OTkA3HLLLRgMBgwGAyEhIaSlpdGmTeNMxKlzeCiK8gygBfYD+4UQx1zWKg/RqlUrbr/9dj7++GO+/PJLJk6ciKYOCwD9WhmZ9P96s+nDo+xek8yFpFxGzuyKt/nK4VODRgN+EeoRM6jm66UFcPFUzWBJPwq/rwV7ScW5igZ829QSLm3VR1Og2mt58sn6t1OSXKi+PQRXq8v6OCEEI0eO5MMPP6zyfHFxMXPnzmX37t1ERkayYMECiouLAVizZg2bN2/m66+/5tlnn+W3336r9bPKy7sbDAbnc1qt1lnZtzFcMTwURfEWQhQLIZ5SFCUU6AlMVBSlnRBituub6F5dunRhxIgRrFu3jqCgIG688cY6vU/npeXGe9XpvJtXqdN5R9/fjZBo34ZtoJcPhHZRj+ocDshPq73XcixRfa3KtSxlYRJdM1j8I0FnQJIkGDx4MB988AHTp08nPT2djRs3MnXq1CrnXH/99Tz00EMcP36c9u3bU1hYSEpKCiEhIQAEBweTn5/Pp59+yqRJk3A4HJw5c4bhw4czePBgVq5cSX5+PkOHDmXFihU8+eSTbNy4keDgYHx9G/jnyFWoS89jl6IoicCbQojjwPdlR4sxaNAgMjMz2bx5M4GBgXUukqYoCl2HRBAcaeH7tw/w2ct7GDrlOroMDm+cjWE0GvU2mG9riK6l5EhpIWTX0mt5dg1Yi2Cqd+XvBnwjLj3W4hMsx1qkFmPixImsX7+ebt26cd1119G/f3/8/PyqnNOqVSuWLl3KXXfdRUmJegfgueee47rrrmP27NnExcURExND3759AXUv9HvuuYecnByEEDzyyCP4+/uzYMEC7rvvPrp3747JZOKDDz5o9O+3NlcsT6Ioiga4BZiDOjvrTWCNaMC6JoqijAb+g3pbbIkQ4oVqr/8FmAXYgHRgphDi1OWu2SCFESux2WwsX76c06dPc++999ZpvndlRfml/PDeIc4cyqLTwNbcUH06rycpL4z4zUe191ouJqvjMJXpfS4dLP5RoPdGkurDE8qTJCcnM3bsWGf59cry8/Mxm81kZmbSr18/tm3bRlhYmBtaeWVms9k506syV5cn8QN+A/4OdAdeAl4HYurw3itSFEULvAGMBFJQezpfCyEOVTptH9BHCFGoKMqDZW2Y0hCfX11pSippzz+P3623Yh4+DE3ZPUWdTseUKVNYsmQJq1atYtasWQQF1b2artHsxdiHe7BrTRK71ySTlpTLTdM7Exrj/u7nJVnC1CPq+pqvWYsg+3QtwZKkzhazFla7Vvilw8UcInstkkeqXJK9+lqPsWPHkp2dTWlpKU8++aRHBseJEyeYOHEioaGhDX7tuvQ8HMB2YBuQV3bkCiHea5AGKMoAYIEQYlTZ108ACCH+eYnzewKvCyFqGT2ucLU9j4Lt2zn7/x7Dlp6OxmLBd/Ro/CbcirFXLxRFITMzkyVLlmAymZg1axZGo7Hen3H6t0x+XHaEguwSzAEGgtuYCYowE9TGTHAbM34hprrPznKFay3JLgQUpF+615KbWvV8nfHyvRYv09W1Q2rSPKHn0dy5uufRB/gjEAcsAb4QQjiupqGXEAGcqfR1CtD/Muf/AfiuAT+/Cp8BA2i/cQMFO3aQ+/XX5KxeTfYnn6Bv0wa/8ePwGz+eO++8kw8++IBVq1Zxzz33XHYFem2iugZx19P9ObT1LBln8shIyef0b1k4HGqQa/UagsJ9aBVloXV7f8I7+GMJbEK3fRRF7U2YQyCyX83XrcWQc6b2YEnaDNZq8/nNYZfptYQ2/RL4ktQE1bkku6IogcBsYBzwrRDiHw3SAEWZDIwSQswq+3oa0E8I8cdazr0HeBi4QQhRUsvrc1DHZoiKiup96tRlh0XqxFFQQN66deR89RUF23eAEBh79CB1+DB+OH+enj17Mn78+GseALdbHWSdLyAzJZ+MsiP9VC6lxequhZYgb8I7qEES3t4fvxCj6wbdFy5UH+fNc831L0cIKMysCJOspFp6LZX+n9V5g/8l1rUERKuz0aQmSfY8XO9aeh51uW21ETADJkABHECJEKLX1Ta42vXrdNtKUZQRwGuowXGhxoWqaegBcwBrWhq5q1eT8+VXlBw7xoEe3TnUuTNDoqMZfvfdaLzqsACvHhwOQWZKPmePZXP2eDZnj2VTnK+WJzH5ejl7JeEd/AkK90Fx562uxmIrgewzFeMrzmA5pX5dWm1Q0CekZqgExkJQezlDzMPJ8HA9V4dHDJAN5DTkDKtK19cBvwM3AanALmCqEOK3Suf0BD4FRtd1caIrwqOcEIKSI0fI/vIrvj11itNhoQzeu4+uvXriN/5WjD3jXdIrEEKQnVaohknZkX9R7YAZTDo1TMoCJTjKjFbbwm7nCAGFWbUES7IaLrkpUPmOq8FX3WMldph6tO4h91vxIDI8XM+l4dEYFEUZA/wbdarue0KI58tWtO8WQnytKMo61DGX8vmhp4UQ4y93TVeGR2WlRUW8/+abXMjJ4aZNmwk4fx59VBR+48fjN34cXlFRLvtsIQR5mcWcPZ7NuWPZnD2eQ3aaOstJ56UhLNbP2TMJjfGt+9Tg5rqHua1UHWvJOgmZJyDzGJzaDhfKfk/x9oe2QyvCJDBW9kzcyBPCIzk5mc6dO9OxY0f2799PTEwMycnJLvms3NxcOnfuzG233cbrr78OqOXdz50755yYk5iYSEhICAsWLCAmJoYZM2Ywf/58li1bxrx585hXz1vNTX4nQSHEt8C31Z57qtKfRzR6o+rIy2jk7jlzWLJkCT/dMoYpkVE4vv+ejDfeIOP11zH26oXf+PH43jwabbVFRNdKURR8g434BhvpdH1rAApySjh3PMfZM9m5OgkEaHQKoTG+zltdrWP98DJ6xH/+xqPzgqB26tFhZMXzeWnqQP3Jjepx+Gv1eb8oiL1BDZK2N4DZ9ZVKJc/TWCXZn3zyyVp3L12xYgV9+lz6Z/nLL79c65bZrtbCfnq4htlsZurUqbz77rusvpjFzLcWo8nKIueb1eR89RXnFywg7fnnMQ8fjt+t4zEPGYLSwOMj5Xz8DLTvHUL73moJhOICK+dP5DjHTfYlnmbv96dQFAiOtDh7Jq3b+2E0u6ZNHs8Squ4A2X2yeusr66S6VqU8SPYtU88LjSsLk+Hqin05GN/ilJc637hxI0899RRBQUEcPXqUoUOHsmjRIjQaDe+++y4vvvgi4eHhdOjQAYPBwOuvv86JEye4++67sdvt3HzzzbzyyivOhXt79uwhLS2N0aNH12kjKbPZfFXLBBqSDI8GEhISwuTJk1mxYgWffvopd911F8FzZhM0exbFvx0i5+uvyF29hrzERLT+/viOGYPfhFvxjotzaakSbx89Md2DiemuloYvLbaRlpTL2WPZnDuezcHNqfyyXp0pHdDah/AO/vTLK8Vg1NEi7/4rSkXvpO8sdUvhc/sreiU734btr4NGr05Djh2mHuG9QCv/ObnKd999x/nz5xv0mmFhYdx88831ek/lUuc7d+7k0KFDREdHM3r0aD7//HMGDhzIs88+y969e7FYLNx444306NEDgD/96U/86U9/4q677mLx4sXO6zgcDh599FGWLVvG+vXra3zmfffdh1arZeLEifztb39DUZR6355yBfl/ewNq3749Y8aMYc2aNaxdu5abb74ZRVEwduuKsVtXQufPJ3/bNnK//prsTz/l4sqVGOPjCZozG/OwYSiNsF7By1tHZOdAIjsHAuoU4Quncp2zuX7feZ4O59Rxkx//9lOlnok/fq1cOD3YU2m0ENFbPYY8qtYDO7OjIkw2/AM2PK8OvscMrgiT4OvkeEkz169fP2JjYwG466672Lp1KzqdjhtuuIHAQPXf1+TJk/n9998B2L59O19++SUAU6dOdQbAokWLGDNmDJGRkTU+Y8WKFURERJCXl8fEiRNZtmwZ9957b2N8e1ckw6OB9e3bl8zMTHbs2EFQUBD9+lUsklP0eizDhmEZNgx7Xh45X31N1nvvkTL3IbzatyNo1iz8brkFpQ77hjQUrV5D6/ZqOPQeDQ67g4LgaeRcKCa4jYXkXzM5sl39jc/k5+VcZxLewZ/A1i1kenBlXqaqm3cVZEJypfGSo2VDd5bWFUHS9ga1OKV01erbQ2gM1X+RUhSlTqXaq9u+fTtbtmxh0aJF5OfnU1paitls5oUXXiAiIgIAi8XC1KlT2blzpwyP5iwhIYGsrCy+++47AgIC6NChQ41ztBYLgffcTcCUO8j9/nsy336Hc48/Qfp//kvQfTPwnzQJjanxy3JotBosf3sUC9AGEA7BxfOFzp7J2WPZHN+tLrMx+OgILwue8A7+tIo0o2lp04N9gqDrbeoB6qLGpE1qkBxLhF/K9nJo1akiTKIHgbcH1zST6mTnzp0kJSURHR3NqlWrmDNnDv369eORRx7h4sWLWCwWPvvsM+Li4gC1RPtnn33GlClT+Oijj5zXWbFihfPPS5cuZffu3bzwwgvYbDays7MJDg7GarWyevVqRozwnLlDMjxcQKPRMHHiRN5//30++eQT/vCHP1yyMJmi1+M3bhy+Y8eSv2kTme8sIe0f/yRj0Zv433EHvmNuxtCxY+PeLiosK2poMqFoFALDfQgM96Hb0IiK6cGV1pok/ZIBgN6gJaydn7NnEhJjQadvYSMngW3Vo/cMdT+VtIMVvZI9H8DPi9WthNv0qQiTiD512+FR8igDBgzg8ccf58CBAwwdOpTbbrsNjUbDX//6V/r37094eDhdunRxlmr/97//zT333MO//vUvbrnllhol3KsrKSlh1KhRWK1W7HY7I0aMYPZsz9lCySPWebhCY63zuJycnBzeeecdtFots2bNuuI+6OUK9+4l850l5G/aBA4H+ugofBNGYUlIwLtbV9cHST3XeRRkl1Raa5JNZqpam0qr0xASUzGjKyzWDy/vFvz7iq0EzuysCJOze9VFi3ofdZfItmXTgkO7yvESPGedR20l2Tdu3MjChQtZvXp1jfeUl2q32WzcdtttzJw5k9tuu43CwkKMRnXc8KOPPuLDDz/kq6++apB2LliwALPZ3PLWeTRXfn5+TJ06lffff79e+6CbevXC9OYibJmZ5K1bT97atWS+9x6Z77yDPjwcy6hRWBJGYuzRo1EG2a/Ex99Ahz6hdOij9q6K862cO1HRM9m79jR7vjuFolFoFWmuMgjv7dN44ztupzNA2yHqcdOTUJQNyVsrwuRYonqeT6uKIIkdpu7iKLnF5UqyX8qCBQtYt24dxcXFJCQkOPci37NnDw8//DBCCPz9/XnvvQYpTM78+fP54osvePTRRxvkenUlex6N4PDhw6xatYouXbowadKkOu2DXp3t4kXyf9xAbuJaCn7aDlYrutBQLAkJ+I5KwNizJ4q2gW4RNfAK89JiG2knK2Z0pSXlYrepZUICw32qFHz08W/BW93mpMDJTRVhUlBWwi2wXaXB9yFgDHBXCxuVJ/Q8mrsmX57EFTwpPAC2bdvGDz/8wJAhQ7jpppuu6Vr23FzyN2wgN/EHCrZsQZSWog0OxjJyBL6jRmHq0welnmXiq3BxeRKb1c6F5DznWpNzJ3KwlqjVg/1aGZ29kvAO/vgGe7e86cGgLla8cLgiSJK3qqXqFQ20jq8Ik8j+zXaXRhkerifDoxaeFh5CCL755hv27t3LhAkT6rwP+pXY8wvI37SRvMQfyN+0CVFcjDYgAMuIm7AkjMLn+v71n/rbyLWtHHYHGeXVg8vGTUoKbIB6S6xyzySgtallhondCql7KsIkZRc4bGpJ+qgBFWES1r3Z7G9y+PBhOnXq1DL/ezcCIQRHjhyR4VGdp4UHqBvcL1++nFOnTl3VPuhX4igsJH/LVvISE8nfsAFHYSEaPz8sw4djGZWAz6BBdSsbv3Sp+jhjRoO2r66EQ5B1roBzZbe5Uo9lU5hTCoC3WU/rdhUFH4PbtMDpwQAleXDqp4owuVC2a7MxoOp4SWBbd7XwmiUlJWGxWAgKCpIB0sCEEGRmZpKXl0fbtlX/H5Hh4YHhAVBUVMS7775LXl4e06ZNo02bNi75HEdJCQXbtpG3di15P27AkZeHxmzGPHw4vqMS8Bk8GI1307jdIYQgN6OoyvTg3IxiAPTeWmeYtG7vT2i0L1p9CwyTvPNVizuWb/XrH1V1saJPsLtaWG9Wq5WUlBSKi4vd3ZRmydvbmzZt2tSYxCPDw0PDA9QpvEuXLqWwsJD+/fsTHx/vLGfgCqK0VN1Wd+1a8tetx56Tg2IyYb5hKL6jRmEeMgRN5aqcGeq6DYI99wdN/sUSZ8/k7PFsss6WTQ/WawiN8a0yPVhvaGFrTYSAzOMVQZK0BUpy1NfC4irCJGqg3B9eqkGGhweHB0B2djarV6/mxIkTCCGIjo6mZ8+edOnSBS8XVdwFEFYrhbt2kbs2kbwffsCelYViMGAeOgRLwijMw4ehHTtWPbkJ7edRlF/qLEV/7ng26afzEAI0GoVW0epe8BEd/Am/zr/lrTWx28qKO25QZ3Od+RnspaD1UgfcyysFt46XxR0lGR6eHh7lcnJy+OWXX9i/fz9ZWVl4eXnRrVs34uPjiYyMdOm9XmG3U7hnD3lrE8lLTMSWno6i1xOTmYEuMBBl27YG34OksZQW2Th/sqIUfVpyLg6bQKNRCI31JaJjAGGxfoTG+LastSagFnc8vb2iZ3L+V/V5g586FTh2mHoEtZeLFVsgGR5NJDzKCSE4ffo0+/bt47fffsNqtTrrYkVFRREREYG/v7/LwkQ4HBTt30/e2kQsL76IKC3hdGw7fAYMwJIwEsuIEegCmu76ApvVzvkTOZw5cpGUw1lcOJ0HZf/r+4eaCG3rS2iML2GxfgRG+LSsLXwLMiqNl2yA7NPq874RVcdLLLWX2JGaFxkeTSw8KispKeHQoUMcPnyYkydPYrOp01ZNJhPh4eFEREQ4D1fsICaGDcNRUEDmlCnkrk3EeuYMaLWY+vXFNyFBDZJWTXtXvdIiGxdO5XI+KZe0pFzSknIoyrMCoNNraBVtITTGl9C2foS29cUcYGg5M36ykiqNl2yCoovq8yFdKmZyxQwCQ93K7UhNiwyPJhweldlsNi5cuEBqaiqpqamcPXuWCxcuOF/39/cnIiLCGSqtW7fGYLjGVdqV1nkIISg5fFgdI1m7ltLkZFAUTL17Y0lIwJIwEn1Y2LV9ngcoL/iYVhYm55NySD+Th8Om/vvw8fNyBkloW19Con1bxkC8w6He1ioPk9PbwVYMGp1a0DF2mHq06QPaFnb7r5mS4dFMwqM2JSUlnDt3zhkoqamp5OSos2kURaFVq1bOnkl4eDihoaFo61O6ZNUq9XHKlCpPCyEoOXasbIxkLSXHjgNgjI/HMmoUvgkj0ZftP9Ac2K3q4sW05JyyQMklN70IwFltOKwsTELb+hEQamr++5tYi9UBd2dxx32AAC+zWmo+dph6hHSW4yVNlAyPZhwetcnPz+fs2bNVAqWoSP1Bp9PpCAsLq3K7KzAw8Jpvw5ScPEleYiK5axMpOXwYAO9u3bCMSsA3IQGv6Ohr/r48TVFeKWnJFbe60pLzKC1Sbyt6GXWERFucA/GhbX0xWpp5qfWii+pU4PI9TDLVXyjwCakIktgbwM8165mkhifDo4WFR3VCCLKzs6uEyblz57Ba1fv63t7eNcZPnCXjz6h7mlPLtpiXUnr6tDNIig8cAMDQqRO+oxKwjBqFoWy7zuZGOAQX0worhUkumSn5lP+z8g32dt7uCmvrR3CkGa2uGQ/GZ5+pCJKTG6EgXX0+qENFmMQMBqO/u1ooXYEMjxYeHrWx2+2kp6dX6aGkpaU5t860WCxEREQw+sUXMRgMKBs34n0Vq9Ctqank/vADeWsTKdq3DwBDh/ZYRpYFyXUdmvXgs7XETvrpXM6fzFV7KSdzKCgrr6LRKbSKtDjDJLStL5agZlr8UQi1bIqzuOO2iuKO4b0qeiWR/dVy9ZJHkOEhw6NOSktLOX/+vHMwPjU1lfGvvALA0hkzCA4OrjIgHxYWhq4eFXutaWnk/bCOvLVrKdy9G4TAKybGuSeJd5cuzfMHZzX5F4srwiQph/RTedisall6o0VfZTA+NNoXL2MzXKxnK4XU3ZWKO+4GYQedEaIrFXcMjWs2xR2bIhkeMjyumn3oUEpLStj50kvOHkpBgVr+Q6PROMdPygMlODi4TnuU2DIy1M2tEtdS8PNOsNvRt2njHCPx7t69RQQJgN3uICu1QL3VVTYYn51Wtv2vAoGtfaqsPQlo7YOmuQ3GF+fCqW1lYbIJ0tVxM4yBZaveh6lHQIy7WtgiyfCQ4XH1qpVkF0KQm5tbZfzk7NmzlJaqt2K8vLyqjJ+Eh4fj5+d32SCwXbxI/vr15CYmUrB9h7q5VevW+CaMxDJqFMb4eI/YJbExFRdYuZCs9k7UXkqOszS93qAlJMZCaExFD8XHr5nd6sk9V3WxYt459fmAmErjJUPBJ8hdLWwRZHjI8Lh6ddjPw+FwkJmZWSVQzp8/j8Oh3orx8fGpMhgfHh6OyVR7ET57Tg55GzaQtzaRgm3bEKWl6Fq1wjJSDRJTn94Nt0tiEyKEIOdCkXPcJC05l4wz+TgcZWNUgd4Vt7ra+tEqyoxO30z+noSAjGOVxku2QEkuoEDr7pWKOw4AvdGNDW1+ZHjI8Lh633yjPo4bV6+32Ww2zp8/X2VAPqO8Qi8QEBBQJVDCwsJqFIG05+eTv3ETeWvXkr9li7q5VWAglhEjsCQk4NO/X/03t2pGbKV20s/kO293pSXlkpellizXaBWC25jVacJl04X9QozN41ag3aauKSkPkzM/g8MKWgNE9S9b+T4cwuNB00wC1E1keMjw8AjFxcWcPXu2SqDk5uYC6oLG0NDQKre8WrVq5VzQ6CgsJH/zZnVzq42bKja3uvFGdXOrgQPrtrlVM1eQU+IMkrRkde2JrWxbX28fPSExvoTFquMnIc2lEGRpAZzaXlEpOE2dHo63H8SUF3ccDkHt5GLFepLhIcPj6h09qj527OiSy+fl5VUZO0lNTXVu+KPX62ndunWVAfmAgABE+eZWiYk1NreyJIxU9yRpIptbuZrDIbh4roDzZbe60pJyyTpXUKUQZOWV8UERPk1/N8b89ErrSzZBTnlxxzZVFyuaQ9zWxKZChocMj6vXyHuYCyHIysqqMX5SXhDSaDRWGTuJCAmBAwfITUxUN7fKzkYxGjHfcAO+CSMx33BD1c2tJEqLbKSdyq3oodRWCLKtnzNUzAFNOIiFgIuVijue3ATF2eprIV0rwiR6IBjM7mqlx2pS4aEoymjgP4AWWCKEeKHa6wbgf0BvIBOYIoRIvtw1ZXhcg0YOj9rY7fYqBSFTU1NJT093Lmj08/NTw6R1a4JzczHt2kXxuvXYMzJQDAZ8Bg/Gd1QC5uHD0Vpk9dfqrlgI0t/gHIwPa+tLq6gmXAjSYa9a3PHUdrCXqMUd2/SrCJOIXrK4I00oPBRF0QK/AyOBFGAXcJcQ4lClc+YC3YUQDyiKcidwmxBiSq0XLCPD4xp4QHjUpqSkxLmgsfzIzlZ/o1QUheDgYMIMBvzPnsPn558xJyWh1WrxGXC9ut3ujTc26T1JXK1yIcjyBY2VC0EGRfioixnLxlD8Q5poIUhrUbXijvtRizta1NIpscPUo1XHFjle0pTCYwCwQAgxquzrJwCEEP+sdM7asnO2K4qiA84DrcRlGi/D4xp4aHjUpqCgoEZByMJCdbGdVqMhSEBASgr+p08TmJ1NWNcu+CYkYL7xRvQh8v73lVypEGT5QsbyXorR3AQnMBRmqVOBy8Mk66T6vDmsYqwkdhj4hrurhY2qruHhCTUQIoAzlb5OAfpf6hwhhE1RlBwgCMjAhUqSc8j5PhlRNnOlIdltVnIz0hFl6yKqE4Bd2HEIu3Ogs7G0O6EWszsx79PG/eBrEAAEYKArbSlUrFzUFnFRU8RFbSHHw9tgj1D/4euFBq/fTsBvJ1zcovr9xuqpv9/WaJe57EChGMhJh9/TUe8X1Ha+B1Dq3KpQoNINjRxgH7DvNOpd84Zsk2sFFCvMePkxl36GJ4RHbX+P1X9c1uUcFEWZA8wBiIqKuqZG5e88R/aXx9H6GdCHNfzg64VjyRQX5eFt8a3xmtVRSl5pHg4h0Gl0aBt5pfWpgbcAUKAtaNTPbUgBQkeA3QJ2CwJBvmIlV1NCjqYEh3CgCNHooXwpwlMacg0ECjiPur7nKksm/48AABAWSURBVDRCOjWH/x5abC7/DE8IjxSgcu3vNsDZS5yTUnbbyg/Iqn4hIcTbwNug3ra62gaVpuaT/eUJDO0DCJraCY13w/41Je3bTeK373HDtD8QN/Y25/NWu5VX9rzC8sPL6RTYiReGvEA7/3YN+tmSJEkNwRPCYxfQQVGUtkAqcCcwtdo5XwPTge3AJODHy413XAthdZC16igas56gOzs2eHAIIdi8cin+Ya3pOXqs8/kzuWeYt3kehzIPcVenu3i0z6MYtG6qXbR/v/oYH++ez5ckyeO5PTzKxjAeBtaiTtV9Twjxm6IozwC7hRBfA+8CyxRFOY7a47jTVe0pOpKJ7UIhwTO7oTE1/LS95P17yDidzOi5j6DVqdf/Luk7/r7972gUDf8e9m9uir6pwT+3Xv78Z/WxCQyYS5LkHm4PDwAhxLfAt9Wee6rSn4uByY3RFlNcK/SPmNCHumaR2a5vPsccGESnQUMpshXx4s4X+ezYZ8S3iufFoS8Sbm4ZMzokSWraPCI8PI2rguP8iWOc+e1Xht4zk5TCs/zpxz9xMucks+JmMTd+LnqNXKAkSVLTIMOjEe365nO8jCbscSHc/e3dKCgsHrGYgRED3d00SZKkemni1dCajuy08xzbsQ2fPh14cMsfCTAEsHLMShkckiQ1SbLn0Uj2rPkSoQje0q+hZ0gvXh32Kn4GP3c3q3b/+Ie7WyBJkoeT4dEIcrIz2bd+DSda5zGy61ieHvA0ek8uwDZQ9oYkSbo8GR4ull2czfOLHyDMJugxZhz3D/qL5+/s9tNP6qMMEUmSLkGGhwudzj3Nw2vn0v83O+aOsdw/4lF3N6lu/vpX9VGu85Ak6RLkgLmL7Luwj7u/vRu/40V4l2oZM+V+dzdJkiSpwcjwcIHvk75n1tpZ+Ov9GHw2itDYDrTpEufuZkmSJDUYGR4NSAjBO7++w/zN8+kW3I1nQ/9E/oV0+o6/3fPHOSRJkupBhkcDsTqsLNi+gP/u+y9j2o7h7ZFvc+j77/ELCaVDPznwLElS8yIHzBtAXmkef9n4F3ac28H93e/nofiHSD16iHPHjnLjffej0TaxvZ///W93t0CSJA8nw+Manc0/y0PrHyI5J5lnBz3LhPYTANj9zed4my10GzbSzS28CrIUuyRJVyDD4xr8lvEbD//4MCW2EhaPXEz/1uruuZmpZzix+2eun3gnem9vN7fyKqxbpz6OGOHedkiS5LFkeFylH0//yONbHifQO5AlCUuq7Pi3+5sv0Om96Dlq7GWu4MGee059lOEhSdIlyAHzq7D80HL+vOHPtPNrx/Ixy6sER/7FLA5v+ZGuw27C5OfvxlZKkiS5jux51IPdYeelXS+x8shKboq6iX8O+SfG/7+9Ow+SojzjOP59uGURQUFFQAgUihxeLBpjJIrigSAaiZoqS7BCjFdM1CpjBSumclSuwmi8orESDivxICisEI2gq/FAWQUheIAgRNS4IIggghxP/ugus4wzu92zM/PO7v4+VVM7xzvTv6d6d5/tY/tts89eYxY/XsXu3bsZdva5gVKKiBSfmkdC23Zu44Znb+CZdc8wYdAErh12La1b7X0W1eefbeO1J+cxYPgJdO3RM1BSEZHiU/NIoHZbLVcvuJq3Nr3F5OMnc9HA7FOoL3vqSXZ8+imVY79Z4oQiIqWl5tGAFZtWcNWCq9i8YzO3j7ydEb1GZB23e9cuXpn3KD0HDuKQwwaWOGWB3XNP6AQiUubUPOrxxJonuPmFm6loU8H0s6YzcP/cTWHFwufYsmE9Iy+9vIQJi+Tww0MnEJEyp+aRxabtm5hSM4XZq2YztNtQbjn5Fg6uODjneHdn0Zy/s/8hveh/7PASJi2Sqqro69ixYXOISNlS88jw8gcvc/0z17P1861MGjqJK4++krat6p/1b+2yJaxf+w6nf+8arFUzOPt5ypToq5qHiOSg5pGhT+c+DOk2hOuGXceArgMSvaemahYVXbpyxEmnFDmdiEh5UPPIcFDFQdx92t2Jx9euWc3apYv5+kWX0KZtGc9LLiJSQM1gH0tYNVWzaNu+A0eNGh06iohIyah5NMInG2p584VnGXrqGXTo1Cl0HBGRktFuq0Z4dd5sAIaNHhc4SYHNmBE6gYiUOTWPPG3fupWl859g4NdG0Ln7gaHjFFbv3qETiEiZ026rPL325Dx27tjePC9F8uCD0U1EJAdteeRh186dLH68ij5HHsOBffuFjlN4d8dnm114YdgcIlK2gm55mNn+Zvakma2Mv3bNMuZoM3vRzJab2VIzC/4b7Y1/Pc2nH29qnlsdIiIJhN5tdSOwwN0HAAvix5m2AZe4+2DgTOBWMws2y5Lv2UNN1Sy69+1Hn6Ga61tEWqbQzWMcMC2+Pw340gxK7r7C3VfG998HaoHuJUuYYfXiRWx8fx3Dx5yHmYWKISISVOjmcZC7fwAQf633tCUzOw5oB6wqQbasFs2Zxb4HdOewE04KFUFEJLiiHzA3s/lAtkvSTk75OT2AGcAEd9+TY8xlwGUAhx56aMqkDXt/xZu89+ZyTr5kEq3bNONzDWbODJ1ARMpc0X8DuvtpuV4zsw/NrIe7fxA3h9oc4zoDc4Gb3H1hPcu6F7gXoLKy0huX/MtqqmbRvqKCoSNPL/RHl5du3UInEJEyF3q31RxgQnx/AjA7c4CZtQMeAaa7+8MlzLaXTR+8x8pFL3LUqNG026djqBilMXVqdBMRySF08/g1MMrMVgKj4seYWaWZ3RePuQAYAUw0syXxreSnOb0y91Fat27NMWe2gDku1DxEpAFBd9y7+0fAqVmerwEmxffvB+4vcbS9bNv8McurF3DESSPp1HX/kFFERMpC6C2PJmHxE3PZtfNzKsecFzqKiEhZUPNowM4d21nyz7n0G3YcB/TSBQNFREDNo0H/rp7P9i2fMFyXIhER+UIz/meFxtuzezevPPYIPQYcTs+Bg0PHKZ1580InEJEypy2Peqx8+QU2137I8LHnt6xLkXTsGN1ERHJQ88jB3Vk0ZxZdDu5B/+HHh45TWnfdFd1ERHJQ88hh3evL+HD1SirHnEerVq1Dxymthx6KbiIiOah55LCoahb7dN6PQd/40r+hiIi0eGoeWWx4dy3vLK7hmDPG0LZd+9BxRETKjppHFjVVj9CmXXuOOn106CgiImVJzSPDlo0beOO5aoacMoqOnfcLHUdEpCzp/zwybFy3jg6dOlE55kuTGrYc1dWhE4hImVPzyNDnyKO57K6pzXuyJxGRRtJuqyzUOERE6qfmISIiqal5iIhIamoeIiKSmpqHiIikpuYhIiKpqXmIiEhqah4iIpKauXvoDEVhZuuBtXm+vRuwoYBxQlIt5ae51AGqpVw1ppY+7t69oUHNtnk0hpnVuHtl6ByFoFrKT3OpA1RLuSpFLdptJSIiqal5iIhIamoe2d0bOkABqZby01zqANVSropei455iIhIatryEBGR1Fp08zCzM83sLTN728xuzPJ6ezN7MH79JTPrW/qUDUtQxwgze9XMdpnZ+BAZk0pQy3Vm9rqZLTWzBWbWJ0TOJBLUcrmZLTOzJWb2nJkNCpEziYZqqTNuvJm5mZXtWUsJ1stEM1sfr5clZjYpRM4kkqwXM7sg/plZbmZ/LdjC3b1F3oDWwCqgH9AOeA0YlDHmSuCP8f2LgAdD586zjr7AkcB0YHzozI2s5RSgY3z/inJcJylq6Vzn/jnA46Fz51tLPG5f4FlgIVAZOncj1stE4I7QWQtUywBgMdA1fnxgoZbfkrc8jgPedvfV7v458AAwLmPMOGBafH8mcKqZWQkzJtFgHe6+xt2XAntCBEwhSS1Pu/u2+OFCoFeJMyaVpJZP6jysAMr1AGSSnxWAnwO/BbaXMlxKSWtpCpLU8l3gTnffBODutYVaeEtuHj2Bd+s8Xhc/l3WMu+8CNgMHlCRdcknqaCrS1vId4B9FTZS/RLWY2VVmtorol+41JcqWVoO1mNkxQG93f6yUwfKQ9Hvs/HjX6Ewz612aaKklqeUw4DAze97MFprZmYVaeEtuHtm2IDL/8ksyJrSmkDGpxLWY2cVAJfC7oibKX6Ja3P1Od+8P/Ai4qeip8lNvLWbWCvg9cH3JEuUvyXqpAvq6+5HAfP6/96HcJKmlDdGuq5OBbwP3mVmXQiy8JTePdUDdvyh6Ae/nGmNmbYD9gI0lSZdckjqaikS1mNlpwGTgHHffUaJsaaVdLw8A5xY1Uf4aqmVfYAhQbWZrgK8Cc8r0oHmD68XdP6rzffUnYFiJsqWV9HfYbHff6e7vAG8RNZNGa8nNYxEwwMy+YmbtiA6Iz8kYMweYEN8fDzzl8VGnMpKkjqaiwVri3SP3EDWOgu2/LYIktdT9IT4bWFnCfGnUW4u7b3b3bu7e1937Eh2LOsfda8LErVeS9dKjzsNzgDdKmC+NJD/7jxKdZIKZdSPajbW6IEsPfcZA4LMVRgMriM5YmBw/9zOib3yADsDDwNvAy0C/0JnzrGM40V8gnwIfActDZ25ELfOBD4El8W1O6MyNqOU2YHlcx9PA4NCZ860lY2w1ZXq2VcL18qt4vbwWr5eBoTM3ohYDbgFeB5YBFxVq2foPcxERSa0l77YSEZE8qXmIiEhqah4iIpKamoeIiKSm5iEiIqmpeYiISGpqHiIikpqah0ggZjYonjuit5ntGzqPSBpqHiIJmNlQM1trZldkee0eMzvRzKrrThhmZleY2V11Hv/CzGbUeWtb4PvAecDWBBn2+nyRkNQ8RBJw92VE1w66JMvLxxNdzynTNGCsmXUxszFE16+6rM7rvYG/EF3+Rlse0qSoeYgkVwsMrvuEmR0BrHD33ZmDPZq06m/AL4E/EM3i+Fmd1x8DZrr7PI8nhoq3cJ6v8/nHmtlTRalGpBHahA4g0oT8GmhvZn3cfW383FnA4/W8589EV2Ud5+6rMl909/9mPLUc6G9mreOGNIWmMU+GtDDa8hBJIJ6BrQKYy95bH2dQf/P4CbCehH+oufseogYy2MzOB/7j7q/mFVqkiNQ8RBpgZh2Ipom9kuiy1kPi5zsCXdw96yRPZnY90WX9LwB+kGKRC4ETgZ8CP847uEgRabeVSMNuAqa7+xozW0Y0QRBEk+w8ne0NZjYSuBQ4wd23mFlnMzva3ZckWN5CYCpwp7u/l+WzjWjOhpeAA4BO7n5n2qJEGkNbHiL1MLPDgVHArfFTX2x5kON4h5kdCtwHfMvdt8RP3wb8MOFi3wR2AL/J8fqxwLNEc7i3A3YVal5qkaQ0GZRInszsVeB4d98ZP64GJrr7mkZ+7h3AIneflvF8NTAR2Ew0t/ZKoCfRHoSL4+MlIiWh3VYieXL3Ywv5eWbWn+iA/POZjSNjuZuA8YVctkhaah4ihTMV+DjfN8en8g4s1ueLFJJ2W4mISGo6YC4iIqmpeYiISGpqHiIikpqah4iIpKbmISIiqal5iIhIamoeIiKSmpqHiIik9j9ay90yCLd+FQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "run -i nt_solutions/ml_2_regression/exo5" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [], "source": [ "## Insert your code here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Exercise 6__\n", "\n", "Compare the optimal weights for ridge and lasso." ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAFD5JREFUeJzt3X+QldWd5/H3N4gLBuJPksFuWZgaTYUCbEyDuhpCovgjmRKSjCPRyTqJkapMUCdOzcIkWyk2Vm1cw84kVqgkFM4GjYEkOpPBCbMm/hhxUhpBbBHFib9QGxhFxqBEXCB+94++MG17Ee37yNNw3q+qrn5+nPucb9P0p0+fe+5zIzORJJXlXXUXIEna/wx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEOqbuAvTnmmGNy9OjRdZchSQeU+++//4XMHLGvdpWEf0ScA3wLGAQsysyrm7T5Y2AekMCDmXnhm11z9OjRrFq1qoryJKkYEfH0W2nXcvhHxCBgATAN6AZWRsSyzHykV5vjgb8CTsvMFyPiva32K0nqvyrm/CcDj2fmk5m5A1gKTO/T5lJgQWa+CJCZz1fQrySpn6oI/zbg2V773Y1jvZ0AnBARv4yIexvTRJKkmlQx5x9NjvW9T/QhwPHAVKAduDsixmXmb153oYhZwCyAUaNGVVCapIPZzp076e7u5tVXX627lP1uyJAhtLe3M3jw4H49vorw7waO67XfDmxs0ubezNwJPBUR/0rPL4OVvRtl5kJgIUBnZ6dvNCDpTXV3dzN8+HBGjx5NRLNx6MEpM9myZQvd3d2MGTOmX9eoYtpnJXB8RIyJiEOBmcCyPm1+CnwEICKOoWca6MkK+pZUsFdffZWjjz66qOAHiAiOPvrolv7iaTn8M3MXMBu4FVgH/DgzH46Ir0XEeY1mtwJbIuIR4E7gLzNzS6t9S1Jpwb9bq193Jev8M3M5sLzPsa/22k7gysaHJKlmA/YVvq0aPfdntfW9/uqP19a3VLKqf+7fys/ysGHD2LZtW6X97g/e20eSCmT4S1LFbrnlFk4++WQmTpzImWeeyXPPPQfAXXfdRUdHBx0dHUycOJGXX36ZTZs2MWXKFDo6Ohg3bhx33303AEuWLGH8+PGMGzeOOXPmVF6j4S9JFTv99NO59957eeCBB5g5cybXXHMNAPPnz2fBggV0dXVx9913M3ToUH74wx9y9tln09XVxYMPPkhHRwcbN25kzpw53HHHHXR1dbFy5Up++tOfVlrjQTvnL0l16e7u5oILLmDTpk3s2LFjz1r80047jSuvvJKLLrqIT37yk7S3tzNp0iQ+97nPsXPnTmbMmEFHRwd33HEHU6dOZcSInptzXnTRRaxYsYIZM2ZUVqMjf0mq2GWXXcbs2bN56KGH+N73vrdnPf7cuXNZtGgR27dv55RTTuHRRx9lypQprFixgra2Nj7zmc9w/fXX07NA8p3lyF+SKrZ161ba2npucbZ48eI9x5944gnGjx/P+PHjueeee3j00UcZOnQobW1tXHrppfz2t79l9erVzJkzhyuuuIIXXniBI488kiVLlnDZZZdVWqPhL+mgUccy61deeYX29vY9+1deeSXz5s3j/PPPp62tjVNOOYWnnnoKgG9+85vceeedDBo0iLFjx3LuueeydOlSvvGNbzB48GCGDRvG9ddfz8iRI/n617/ORz7yETKTj33sY0yf3vdmya2J/fHnRX90dnZmK2/m4jp/6eC3bt06PvCBD9RdRm2aff0RcX9mdu7rsc75S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAK5zl/SwWPe4RVfb+s+mwwaNIjx48eza9cuxowZww033MARRxzBxo0bufzyy7npppve8JipU6cyf/58Ojv3uSLzHePIX5JaMHToULq6uli7di1HHXUUCxYsAODYY49tGvwDheEvSRU59dRT2bBhAwDr169n3LhxAGzfvp2ZM2cyYcIELrjgArZv377nMddddx0nnHACU6dO5dJLL2X27NkAbN68mU996lNMmjSJSZMm8ctf/rLSWp32kaQK/O53v+P222/nkksuecO573znOxx22GGsWbOGNWvWcNJJJwGwceNGrrrqKlavXs3w4cP56Ec/yoknngjAFVdcwZe+9CVOP/10nnnmGc4++2zWrVtXWb2GvyS1YPv27XR0dLB+/Xo++MEPMm3atDe0WbFiBZdffjkAEyZMYMKECQDcd999fPjDH+aoo44C4Pzzz+fXv/41ALfddhuPPPLInmu89NJLvPzyywwfPrySup32kaQW7J7zf/rpp9mxY8eeOf++IuINx97s3mqvvfYa99xzD11dXXR1dbFhw4bKgh8Mf0mqxOGHH861117L/Pnz2blz5+vOTZkyhRtvvBGAtWvXsmbNGgAmT57MXXfdxYsvvsiuXbu4+eab9zzmrLPO4tvf/vae/a6urkrrddpH0sHjLSzNfCdNnDiRE088kaVLl/KhD31oz/EvfOELfPazn2XChAl0dHQwefJkANra2vjyl7/MySefzLHHHsvYsWM5/PCe5arXXnstX/ziF5kwYQK7du1iypQpfPe7362sVsNfklqwbdu21+3fcsste7bXrl0L9EwNLV26tOnjL7zwQmbNmsWuXbv4xCc+wVlnnQXAMcccw49+9KN3qGqnfSSpVvPmzaOjo4Nx48YxZsyYSt+n98048pekGs2fP7+Wfh35SzqgDdR3I3yntfp1G/6SDlhDhgxhy5Ytxf0CyEy2bNnCkCFD+n2NSqZ9IuIc4FvAIGBRZl69l3Z/BPwEmJSZ/X+DXkkC2tvb6e7uZvPmzXWXst8NGTLkdW8c/3a1HP4RMQhYAEwDuoGVEbEsMx/p0244cDnwq1b7lCSAwYMHM2bMmLrLOCBVMe0zGXg8M5/MzB3AUmB6k3ZXAdcAr1bQpySpBVWEfxvwbK/97saxPSJiInBcZv7jm10oImZFxKqIWFXin3GStL9UEf5vvGEF7Hn2JSLeBfwN8Bf7ulBmLszMzszsHDFiRAWlSZKaqSL8u4Hjeu23Axt77Q8HxgH/HBHrgVOAZRFR31vYSFLhqgj/lcDxETEmIg4FZgLLdp/MzK2ZeUxmjs7M0cC9wHmu9pGk+rQc/pm5C5gN3AqsA36cmQ9HxNci4rxWry9Jql4l6/wzczmwvM+xr+6l7dQq+pQk9Z+v8JWkAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCnRIFReJiHOAbwGDgEWZeXWf81cCnwd2AZuBz2Xm01X0fSAaPfdntfW9/uqP19a3pIGj5ZF/RAwCFgDnAmOBT0fE2D7NHgA6M3MCcBNwTav9SpL6r4ppn8nA45n5ZGbuAJYC03s3yMw7M/OVxu69QHsF/UqS+qmK8G8Dnu213904tjeXAP9UQb+SpH6qYs4/mhzLpg0j/gToBD68l/OzgFkAo0aNqqA0SVIzVYz8u4Hjeu23Axv7NoqIM4GvAOdl5v9rdqHMXJiZnZnZOWLEiApKkyQ1U0X4rwSOj4gxEXEoMBNY1rtBREwEvkdP8D9fQZ+SpBa0HP6ZuQuYDdwKrAN+nJkPR8TXIuK8RrNvAMOAn0REV0Qs28vlJEn7QSXr/DNzObC8z7Gv9to+s4p+JEnV8BW+klQgw1+SClTJtI9UOm/ZoQONI39JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalA3ttHBwzvnyNVx5G/JBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpBLPSWpiYN9abEjf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klSgSsI/Is6JiH+NiMcjYm6T8/8pIn7UOP+riBhdRb+SpP5pOfwjYhCwADgXGAt8OiLG9ml2CfBiZv4B8DfA/2q1X0lS/1Ux8p8MPJ6ZT2bmDmApML1Pm+nA4sb2TcAZEREV9C1J6ocqwr8NeLbXfnfjWNM2mbkL2AocXUHfkqR+qOL2Ds1G8NmPNkTELGAWwKhRo1oqav2QC1t6fGu2vunZgVwb8w7fP2U07fvA/XcbyLUN5O/pQK5tQH9PK1DFyL8bOK7XfjuwcW9tIuIQ4HDg3/teKDMXZmZnZnaOGDGigtIkSc1UEf4rgeMjYkxEHArMBJb1abMMuLix/UfAHZn5hpG/JGn/aHnaJzN3RcRs4FZgEPC3mflwRHwNWJWZy4DrgBsi4nF6RvwzW+1XktR/ldzSOTOXA8v7HPtqr+1XgfOr6EuS1Drv5y+pPvt6QljvGG/vIEkFMvwlqUCGvyQVyPCXpAIZ/pJUIFf7SAc7V9SoCUf+klQgR/56PUeJUhEc+UtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQg7+cvSc0c5O9t4chfkgpk+EtSgQx/SSpQS+EfEUdFxC8i4rHG5yObtOmIiHsi4uGIWBMRF7TSpySpda0+4TsXuD0zr46IuY39OX3avAL818x8LCKOBe6PiFsz8zct9q3SHORPwEn7U6vTPtOBxY3txcCMvg0y89eZ+VhjeyPwPDCixX4lSS1oNfzfl5mbABqf3/tmjSNiMnAo8MRezs+KiFURsWrz5s0tliZJ2pt9TvtExG3A7zU59ZW301FEjARuAC7OzNeatcnMhcBCgM7Oznw715ckvXX7DP/MPHNv5yLiuYgYmZmbGuH+/F7avQf4GfDfM/PeflcrSapEq9M+y4CLG9sXA//Qt0FEHAr8PXB9Zv6kxf4kSRVoNfyvBqZFxGPAtMY+EdEZEYsabf4YmAL8aUR0NT46WuxXktSClpZ6ZuYW4Iwmx1cBn29s/wD4QSv9SJKq5St8JalAhr8kFcjwl6QCGf6SVCDDX5IK5Dt5SVXwpnM6wDjyl6QCOfKvg6NESTVz5C9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVqKXwj4ijIuIXEfFY4/ORb9L2PRGxISK+3UqfkqTWtTrynwvcnpnHA7c39vfmKuCuFvuTJFWg1fCfDixubC8GZjRrFBEfBN4H/LzF/iRJFWg1/N+XmZsAGp/f27dBRLwL+N/AX7bYlySpIofsq0FE3Ab8XpNTX3mLffwZsDwzn42IffU1C5gFMGrUqLd4eUnS27XP8M/MM/d2LiKei4iRmbkpIkYCzzdpdirwoYj4M2AYcGhEbMvMNzw/kJkLgYUAnZ2d+Va/CEnS27PP8N+HZcDFwNWNz//Qt0FmXrR7OyL+FOhsFvySpP2n1Tn/q4FpEfEYMK2xT0R0RsSiVouTJL0zWhr5Z+YW4Iwmx1cBn29y/PvA91vpU5LUOl/hK0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoFafSevgWve1rorkKQBy5G/JBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVKDKz7hqaiojNwNM1dX8M8EJNfe+LtfWPtfWPtfVPnbX958wcsa9GAzb86xQRqzKzs+46mrG2/rG2/rG2/hnIte3mtI8kFcjwl6QCGf7NLay7gDdhbf1jbf1jbf0zkGsDnPOXpCI58pekAhn+vUTE30bE8xGxtu5a+oqI4yLizohYFxEPR8QVdde0W0QMiYj7IuLBRm3/o+6a+oqIQRHxQET8Y9219BYR6yPioYjoiohVddfTW0QcERE3RcSjjf93p9ZdE0BEvL/x77X746WI+PO669otIr7U+DlYGxFLImJI3TU147RPLxExBdgGXJ+Z4+qup7eIGAmMzMzVETEcuB+YkZmP1FwaERHAuzNzW0QMBv4FuCIz7625tD0i4kqgE3hPZv5h3fXsFhHrgc7MHHDr1SNiMXB3Zi6KiEOBwzLzN3XX1VtEDAI2ACdnZl2vC+pdTxs9///HZub2iPgxsDwzv19vZW/kyL+XzFwB/HvddTSTmZsyc3Vj+2VgHdBWb1U9sse2xu7gxseAGVVERDvwcWBR3bUcKCLiPcAU4DqAzNwx0IK/4QzgiYEQ/L0cAgyNiEOAw4CNNdfTlOF/AIqI0cBE4Ff1VvIfGtMqXcDzwC8yc8DUBnwT+G/Aa3UX0kQCP4+I+yNiVt3F9PL7wGbg/zSmyxZFxLvrLqqJmcCSuovYLTM3APOBZ4BNwNbM/Hm9VTVn+B9gImIYcDPw55n5Ut317JaZv8vMDqAdmBwRA2LaLCL+EHg+M++vu5a9OC0zTwLOBb7YmHocCA4BTgK+k5kTgd8Cc+st6fUaU1HnAT+pu5bdIuJIYDowBjgWeHdE/Em9VTVn+B9AGvPpNwM3Zubf1V1PM42pgX8Gzqm5lN1OA85rzK0vBT4aET+ot6T/kJkbG5+fB/4emFxvRXt0A929/oK7iZ5fBgPJucDqzHyu7kJ6ORN4KjM3Z+ZO4O+A/1JzTU0Z/geIxpOq1wHrMvOv666nt4gYERFHNLaH0vMD8Gi9VfXIzL/KzPbMHE3PFMEdmTkgRmIR8e7Gk/c0plTOAgbESrPM/Dfg2Yh4f+PQGUDtiwv6+DQDaMqn4RnglIg4rPEzewY9z88NOIZ/LxGxBLgHeH9EdEfEJXXX1MtpwGfoGbnuXuL2sbqLahgJ3BkRa4CV9Mz5D6gllQPU+4B/iYgHgfuAn2Xm/625pt4uA25sfF87gP9Zcz17RMRhwDR6RtYDRuMvpZuA1cBD9GTsgHy1r0s9JalAjvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBfr/cbamflaaL5AAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "run -i nt_solutions/ml_2_regression/exo8" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "raw", "metadata": {}, "source": [ "" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.1" } }, "nbformat": 4, "nbformat_minor": 1 }