{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Artificial Intelligence, Part II: Learning\n", "\n", "Today we will continue the discussion on the learning part of the course. We will study how the linear model can be used to learn a mapping on data that was non linerly generated. We will illustrate regularization and classification.\n", "\n", "For those who are taking the ML course in parallel you can jump directly to exercise 4 below. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 0. Decision tree\n", "\n", "__Part I.__ The dataset below was generated from two classes (a True and False one) in a 9 attributes. Using the 100 examples from this dataset and computing the entropy gain to order the nodes, we would like to build a binary decision tree (the decision being stored in y). Since the attributes are continuous, we will proceed as follows. For each attribute, compute the minimum and maximum possible value for the attribute. Then define the bins for that attribute to be $[x_{\\max} - x_{\\min}]/5$ (that is to say we will consider 5 bins)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import make_classification\n", "import numpy as np\n", "\n", "X, y = make_classification(n_features=9, n_redundant=0, n_informative=9,\n", " n_clusters_per_class=1, n_classes=2)\n", "\n", "data = X[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__Part II.__ In this second part we will use the decision tree implementation of scikit learn to learn a diabetes model based on a series of 8 features given in the table 'col_names' below. Start by loading the csv file from [Kaggle](https://www.kaggle.com/kumargh/pimaindiansdiabetescsv)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Load libraries\n", "import pandas as pd\n", "from sklearn.tree import DecisionTreeClassifier # Import Decision Tree Classifier\n", "from sklearn.model_selection import train_test_split # Import train_test_split function\n", "from sklearn import metrics #Import scikit-learn metrics module for accuracy calculation" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "col_names = ['pregnant', 'glucose', 'bp', 'skin', 'insulin', 'bmi', 'pedigree', 'age', 'label']\n", "# load dataset\n", "pima = pd.read_csv(\"pima-indians-diabetes.csv\", header=None, names=col_names)\n", "\n", "# learn the decision tree\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can finally use the lines below to display the decision tree" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "\n", "from sklearn.tree import DecisionTreeClassifier, plot_tree\n", "\n", "tree.plot_tree(clf)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1. Linear regression on non linearly distributed data\n", "\n", "Recall that the general linear regression model in the univariate case reads as\n", "\n", "$$y(x) = \\beta_0 + \\beta_1 x$$\n", "\n", "\n", "In previous sessions we saw how to code gradient descent an used it to learn a linear regression model on linear data.\n", "\n", "Consider the dataset below which was generated by the function $f(x) = x^2 + x +1$. Although the function is non linear, linear regression can still be used to learn the distribution of the data. From the original set of examples $\\left\\{\\mathbf{x}^{(i)}, t^{(i)}\\right\\}$ where $\\mathbf{x}^{(i)}$ are the feature vectors and feedback the agent gets, one can consider instead of $\\mathbf{x}^{(i)}$, the extended vector $(\\mathbf{x}^{(i)}, (\\mathbf{x}^{(i)})^2)$ and learn a _multivariate_ regression model of the form \n", "\n", "$$y(x) = \\beta_0 + \\beta_1x_1 + \\beta_2 x_2$$\n", "\n", "on the examples $(x^{(i)}, (x^{(i)})^2)$. That is we keep a linear model but now the model is defined on an extended set of attributes/features including higher powers of the original attributes.\n", "\n", "We can do this because the model does not have to know that we hide non linearity in the new features. The agent sees the new features and learn a model on those features not knowing what is hidden in those features. \n", "\n", "Once we have learned the model $y(x) = \\beta_0 + \\beta_1 x + \\beta_2 x^2$, we can recover the quandratic function by taking any new sample $x^*$, squaring $x^*$ and getting y as $\\beta_0 + \\beta_1 x^* + \\beta_2 (x^*)^2$. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAATfElEQVR4nO3df4wc533f8fdHUoSU+WUpomTqB3kOQLhRg6YWDq7sBGkKS4XFNqFSwICMS8O2AQ4G4tYu+gNMCRQBAhVu0QZtANfBVXbBNIcYqWNXhMvUkZgUbhtY8Emx9SOMQtklaYaMdHESOy2Buq6+/WPnyuVp78fe7N4Nd94v4LCzzzyzz8PZuQ/nZp6ZSVUhSZp9N+11ByRJu8PAl6SeMPAlqScMfEnqCQNfknrilr3uwGbuuOOOmpub2+tuSNIN49lnn/3Dqto/al6nA39ubo6VlZW97oYk3TCSXNhonod0JKknDHxJ6gkDX5J6wsCXpJ4w8CWpJ2Yv8JeXYW4Obrpp8Lq8vNc9kqRO6PSwzLEtL8PiIly9Onh/4cLgPcDCwt71S5I6YLb28E+cuBb2a65eHZRLUs/NVuBfvDheuST1yGwF/sGD45VLUo/MVuA//jjs23d92b59g3JJ6rnZCvyFBVhagkOHIBm8Li15wlaSmLVROjAIdwNekt5gtvbwJUkbMvAlqSumfOHo7B3SkaQb0S5cOOoeviR1wS5cOGrgS1IX7MKFowa+JHXBLlw4OpHAT/LuJC8neSXJ8RHzk+Tnm/nPJ3lgEu1K0szYhQtHWwd+kpuBDwOPAPcD701y/7pqjwCHm59F4CNt25WkmbILF45OYpTO24FXqurLAEk+DhwFfmeozlHgF6uqgM8leVOSA1V1ZQLtS9JsmPKFo5M4pHMP8JWh95easnHrAJBkMclKkpXV1dUJdE+SBJMJ/Iwoqx3UGRRWLVXVfFXN79+/v3XnJEkDkwj8S8B9Q+/vBS7voI4kaYomEfifBw4neUuSW4HHgFPr6pwCfqIZrfMg8DWP30vS7mp90raqvpnk/cBngJuBj1XVS0ne18z/BeA0cAR4BbgK/K227UqSxjORe+lU1WkGoT5c9gtD0wX81CTakiTtjFfaSlJPGPiS1BMGviT1hIEvST1h4EtSTxj4ktQTBr4k9YSBL0k9YeBLUk8Y+JLUEwa+JPWEgS9JPWHgS1JPGPiS1BMGviT1hIEvST1h4EtSTxj4ktQTBr4k9YSBL0k9YeBLUk8Y+JLUEwa+JPWEgS9JPWHgS1JPtAr8JLcneSrJueb1thF17kvym0nOJnkpyQfatClJnbW8DHNzcNNNg9fl5b3u0XXa7uEfB85U1WHgTPN+vW8Cf7+qvhd4EPipJPe3bFeSumV5GRYX4cIFqBq8Li52KvTbBv5R4GQzfRJ4dH2FqrpSVc81038KnAXuadmuJHXLiRNw9er1ZVevDso7om3g31VVV2AQ7MCdm1VOMge8DXhmkzqLSVaSrKyurrbsniTtkosXxyvfA7dsVSHJ08CbR8wa67+tJN8O/Crwwar6+kb1qmoJWAKYn5+vcdqQpD1z8ODgMM6o8o7Ycg+/qh6qqu8b8fMk8GqSAwDN62ujPiPJtzAI++Wq+uQk/wET1/GTLpI66vHHYd++68v27RuUd0TbQzqngGPN9DHgyfUVkgT4KHC2qn6uZXvTdQOcdJHUUQsLsLQEhw5BMnhdWhqUd0Sqdn7UJMl3A78CHAQuAu+pqj9KcjfwRFUdSfKDwH8FXgBebxb9x1V1eqvPn5+fr5WVlR33b2xzc6P/JDt0CM6f371+SNIOJXm2quZHzdvyGP5mquqrwLtGlF8GjjTT/w1Im3Z2zQ1w0kWSdsorbYdtdHKlQyddJGmnDPxhN8BJF0naKQN/2A1w0kWSdqrVMfyZtLBgwEuaSe7hS1JPGPiS1BMGviT1hIEvST1h4EtSTxj4ktQTBr4k9YSBL0k9YeBLUk8Y+JLUEwa+JPWEgS9JPWHgS1JPGPiS1BMGviT1hIEvST1h4EtSTxj4ktQTBr4k9YSBL0k9YeBLUk+0Cvwktyd5Ksm55vW2TerenOS3k3y6TZuSpJ1pu4d/HDhTVYeBM837jXwAONuyve5bXoa5ObjppsHr8vJe90iSgPaBfxQ42UyfBB4dVSnJvcBfBZ5o2V63LS/D4iJcuABVg9fFRUNfUie0Dfy7quoKQPN65wb1/hXwj4DXt/rAJItJVpKsrK6utuzeLjtxAq5evb7s6tVBuSTtsS0DP8nTSV4c8XN0Ow0k+WvAa1X17HbqV9VSVc1X1fz+/fu3s0h3XLw4XrmkbpnxQ7K3bFWhqh7aaF6SV5McqKorSQ4Ar42o9gPAjyY5Anwr8J1JfqmqfnzHve6qgwcHh3FGlUvqtrVDsmt/pa8dkgVYWNi7fk1Q20M6p4BjzfQx4Mn1Farqp6vq3qqaAx4DfmMmwx7g8cdh377ry/btG5RL6rYeHJJtG/gfAh5Ocg54uHlPkruTnG7buRvOwgIsLcGhQ5AMXpeWZmbvQJppPTgkm6ra6z5saH5+vlZWVva6G5L6YG5u9CHZQ4fg/Pnd7s2OJXm2quZHzfNKW0mCXhySNfAlCXpxSHbLUTqS1BsLCzMV8Ou5hy9JPWHgS1JPGPiS1BMGviT1hIEvST1h4EtSTxj4ktQTBr4k9YSBL0k9YeBLUk8Y+JLUEwa+JPWEgS9pdsz4M2nb8m6ZkmZDD55J25Z7+JJmQw+eSduWgS9pNvTgmbRtGfiSZsPBg+OV95CBL2k29OCZtG0Z+JJmQw+eSduWo3QkzY4ZfyZtW+7hd43jiCVNiXv4XeI4YklT1GoPP8ntSZ5Kcq55vW2Dem9K8okkv5vkbJJ3tGl3ZjmOWNIUtT2kcxw4U1WHgTPN+1H+NfCfq+rPAt8PnG3Z7mxyHLGkKWob+EeBk830SeDR9RWSfCfwQ8BHAarqG1X1Jy3bnU2OI5Y0RW0D/66qugLQvN45os73AKvAv0vy20meSPJtG31gksUkK0lWVldXW3bvBuM4YklTtGXgJ3k6yYsjfo5us41bgAeAj1TV24D/xcaHfqiqpaqar6r5/fv3b7OJGTGJccSO8pG0gS1H6VTVQxvNS/JqkgNVdSXJAeC1EdUuAZeq6pnm/SfYJPB7r804Ykf5SNpE20M6p4BjzfQx4Mn1FarqD4CvJHlrU/Qu4HdatqtRHOUjaRNtA/9DwMNJzgEPN+9JcneS00P1/g6wnOR54C8A/7RluxrFUT6SNtHqwquq+iqDPfb15ZeBI0PvvwDMt2lL23Dw4OAwzqhySb3nrRVmiaN8JG3CwJ8l3i1Q0ia8l86s8W6BkjbgHr4k9YSBL0k9YeBLUk8Y+JLUEwa+JPWEgS9JPWHgS1JPGPiS1BMGviT1hIEvaXLaPoDHB/hMlbdWkDQZbR/A4wN8pi5Vtdd92ND8/HytrKzsdTckbcfc3Ojbcx86BOfPT395AZDk2aoaeTt6D+lImoy2D+DxAT5TZ+BLmoyNHrSz3QfwtF1eWzLwJU1G2wfw+ACfqTPwJU1G2wfw+ACfqfOkrSTNEE/aSpIMfEnqCwNfk+WVklJneaWtJscrJaVOcw9fk3PixLWwX3P16qBc0p5rFfhJbk/yVJJzzettG9T7e0leSvJikl9O8q1t2lVHeaWk1Glt9/CPA2eq6jBwpnl/nST3AH8XmK+q7wNuBh5r2a66yCslpU5rG/hHgZPN9Eng0Q3q3QL8mSS3APuAyy3bVRd5paTUaW0D/66qugLQvN65vkJV/T7wL4CLwBXga1X16y3bVRd5paTUaVuO0knyNPDmEbO2dSauOa5/FHgL8CfAf0jy41X1SxvUXwQWAQ56KODGs7BgwEsdtWXgV9VDG81L8mqSA1V1JckB4LUR1R4C/kdVrTbLfBJ4JzAy8KtqCViCwa0Vtv4nSJK2o+0hnVPAsWb6GPDkiDoXgQeT7EsS4F3A2ZbtSpLG1DbwPwQ8nOQc8HDzniR3JzkNUFXPAJ8AngNeaNpcatmuJGlM3i1TkmaId8uUJBn4ktQXBr4k9YSBL0k9YeBLUk8Y+JLUEwa+pGt8YtlMM/B1PX/h+2vtiWUXLkDVtSeWuQ3MDANf1/gL328+sWzmGfi6xl/4fvOJZTPPwNc1/sL3m08sm3kGvq7xF77ffGLZzDPwdY2/8P3mE8tmnoGva/yFb+9GH+W0sADnz8Prrw9e/e5nypZPvFLP+IjCnVsb5bR24nttlBO4TtUJ7uFLk+IoJ3WcgS9NiqOc1HEGvjQpXRjldKOfQ9BUGfjSpOz1KCevlNYWDHzNlr3cw93rUU6eQ9AWDHx1S5vA7sIe7l4Oa/QcgrZg4Ks72gZ23/dwu3AOQZ1m4Ks72gZ23/dw9/ocgjrPwFd3tA3svu/h7vU5BHWega/uaBvY7uF6awRtqlXgJ3lPkpeSvJ5kfpN6707ycpJXkhxv06ZmWNvAdg9X2lTbPfwXgb8OfHajCkluBj4MPALcD7w3yf0t29UsmkRgu4crbahV4FfV2ap6eYtqbwdeqaovV9U3gI8DR9u0qxm214HtlaqaYbtxt8x7gK8Mvb8E/MVdaFcaj3e71Izbcg8/ydNJXhzxs9299Iwoq03aW0yykmRldXV1m01IE9D3cfyaeVvu4VfVQy3buATcN/T+XuDyJu0tAUsA8/PzG/7HIE1c38fxa+btxrDMzwOHk7wlya3AY8CpXWhXGk/fx/Fr5rUdlvljSS4B7wD+U5LPNOV3JzkNUFXfBN4PfAY4C/xKVb3UrtvSFDiOXzOu1UnbqvoU8KkR5ZeBI0PvTwOn27QlTd3aidkTJwaHcQ4eHIS9J2w1I3ymrTTMZ/pqhnlrBUnqCQNfknrCwJeknjDwJaknDHypS7yXj6bIUTpSV3gvH02Ze/hSV3gvH02ZgS91hffy0ZQZ+FJXeC8fTZmBL3WF9/LRlBn4Ulf4TF5NmaN0pC7xXj6aIvfwJaknDHxJ6gkDX5J6wsCXpJ4w8CWpJ1JVe92HDSVZBS7scPE7gD+cYHcmxX6Nx36Nx36NZxb7daiq9o+a0enAbyPJSlXN73U/1rNf47Ff47Ff4+lbvzykI0k9YeBLUk/McuAv7XUHNmC/xmO/xmO/xtOrfs3sMXxJ0vVmeQ9fkjTEwJeknrihAz/Je5K8lOT1JBsOYUry7iQvJ3klyfGh8tuTPJXkXPN624T6teXnJnlrki8M/Xw9yQebeT+T5PeH5h3ZrX419c4neaFpe2Xc5afRryT3JfnNJGeb7/wDQ/Mmtr422laG5ifJzzfzn0/ywHaXbWMb/Vpo+vN8kt9K8v1D80Z+n7vYtx9O8rWh7+efbHfZKffrHw716cUk/zfJ7c28qayzJB9L8lqSFzeYP93tq6pu2B/ge4G3Av8FmN+gzs3Al4DvAW4Fvgjc38z758DxZvo48M8m1K+xPrfp4x8wuGAC4GeAfzCF9bWtfgHngTva/rsm2S/gAPBAM/0dwO8NfY8TWV+bbStDdY4AvwYEeBB4ZrvLTrlf7wRua6YfWevXZt/nLvbth4FP72TZafZrXf0fAX5j2usM+CHgAeDFDeZPdfu6offwq+psVb28RbW3A69U1Zer6hvAx4GjzbyjwMlm+iTw6IS6Nu7nvgv4UlXt9Kri7Wr7792z9VVVV6rquWb6T4GzwD0Tan/NZtvKcF9/sQY+B7wpyYFtLju1flXVb1XVHzdvPwfcO6G2W/dtSstO+rPfC/zyhNreUFV9FvijTapMdfu6oQN/m+4BvjL0/hLXguKuqroCg0AB7pxQm+N+7mO8cWN7f/Mn3ccmdehkjH4V8OtJnk2yuIPlp9UvAJLMAW8DnhkqnsT62mxb2arOdpbdqXE/+ycZ7CWu2ej73M2+vSPJF5P8WpI/N+ay0+wXSfYB7wZ+dah4mutsM1Pdvjr/xKskTwNvHjHrRFU9uZ2PGFHWeizqZv0a83NuBX4U+Omh4o8AP8ugnz8L/Evgb+9iv36gqi4nuRN4KsnvNnsmOzbB9fXtDH4xP1hVX2+Kd7y+1n/8iLL128pGdaaynW3R5hsrJn+ZQeD/4FDxxL/PMfv2HIPDlf+zOb/yH4HD21x2mv1a8yPAf6+q4T3vaa6zzUx1++p84FfVQy0/4hJw39D7e4HLzfSrSQ5U1ZXmz6bXJtGvJON87iPAc1X16tBn///pJP8W+PRu9quqLjevryX5FIM/Jz/LHq+vJN/CIOyXq+qTQ5+94/W1zmbbylZ1bt3Gsju1nX6R5M8DTwCPVNVX18o3+T53pW9D/zFTVaeT/Jskd2xn2Wn2a8gb/sKe8jrbzFS3rz4c0vk8cDjJW5q96ceAU828U8CxZvoYsJ2/GLZjnM99w7HDJvTW/Bgw8oz+NPqV5NuSfMfaNPBXhtrfs/WVJMBHgbNV9XPr5k1qfW22rQz39Sea0RQPAl9rDkNtZ9md2vKzkxwEPgn8jar6vaHyzb7P3erbm5vvjyRvZ5A7X93OstPsV9Of7wL+EkPb3C6ss81Md/ua9Fno3fxh8Mt9CfjfwKvAZ5ryu4HTQ/WOMBjV8SUGh4LWyr8bOAOca15vn1C/Rn7uiH7tY7Dhf9e65f898ALwfPOlHtitfjEYBfDF5uelrqwvBocoqlknX2h+jkx6fY3aVoD3Ae9rpgN8uJn/AkOjwzbazia0jrbq1xPAHw+tm5Wtvs9d7Nv7m7a/yOCE8ju7sM6a938T+Pi65aa2zhjs3F0B/g+D7PrJ3dy+vLWCJPVEHw7pSJIw8CWpNwx8SeoJA1+SesLAl6SeMPAlqScMfEnqif8Hp3Cyg6mIXpMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "\n", "np.random.seed(1)\n", "\n", "xi = np.linspace(-1,1,20)\n", "noise = np.random.normal(0,0.05,20)\n", "\n", "ti = (xi-1)*(xi+1) \n", "tinoisy = ti+noise\n", "\n", "plt.scatter(xi,tinoisy, c = 'r')\n", "#plt.plot(np.linspace(-1,1,100), (np.linspace(-1,1,100)-1)*(np.linspace(-1,1,100)+1) )\n", "plt.show()\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The linear regression model can be learned from scratch through gradient descent but it can also be learned via the scikit-learned library. The linear_regression model from scikit learn comes with the functions fit and predict. Fit is used to learn the weights of the model, predict is used to predict the value of the model at a new point. \n", "\n", "Once the model has been learned, it is possible to represent the corresponding plane in the space $(y, x_1, x_2)$. This shows that if it is not possible to learn a linear model in the original space, it is possible to lift the data in a higher dimensional space and learn a linear model in this space. \n", "\n", "Once the plane has been learned, we can use it to get the value at any new point $x^{(i)}$ by getting the pair $(x^{(i)},(x^{(i)})^2)$, obtain $y$ from the plane, and bring back this value back in the original space. \n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from sklearn.linear_model import LinearRegression\n", "\n", "'''Using the function linear_regression from scikit learn, fit the linear model \n", "to the extedned feature vector. Then plot the result on top of the data''' \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2 Closed form\n", "\n", "We will now use the matrix equations to solve for the weight vector. Stack all the feature/attribute vector as the rows of the matrix $X$ then solve for the weight vector using the expression $\\mathbf{\\beta}^*= (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{y}$" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3. Overfitting \n", "\n", "Consider the same dataset as above, scikit learn also provide a function 'PolynomialFeatures' which can be used to generate additional powers from a some original set of features. We will use this function to change the complexity of our model and plot the result. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsT0lEQVR4nO3deXhU5dnH8e+dDTIBEkJCCIEkLGHfNxEUZVXACtjaaqOiVUPd1/ZV8e1rtbTaWlu3qoCi1iBuKGhRRMAFUCAgSzBAAoQkhCUhECCBbPO8f8xoI2ZlZnImM/fnuuaamXPOnOfmzPDLmTPPeY4YY1BKKeX7AqwuQCmlVNPQwFdKKT+hga+UUn5CA18ppfyEBr5SSvmJIKsLqEtUVJRJTEy0ugyllGo2Nm3aVGiMia5pnlcHfmJiImlpaVaXoZRSzYaI7K9tnh7SUUopP6GBr5RSfkIDXyml/IQGvlJK+QkNfKWU8hO+F/ipqZCYCAEBjvvUVKsrUkopr+DV3TIbLTUVUlKgtNTxfP9+x3OA5GTr6lJKKS/gW3v4s2f/N+y/V1rqmK6UUn7OtwI/J6dx05VSyo/4VuDHxzduulJK+RHfCvw5c8Bm+/E0m80xXSml/JxvBX5yMsydCwkJIOK4nztXf7BVSil8rZcOOMJdA14ppX7Ct/bwlVJK1UoDXyml/IQGvlJK+QkNfKWU8hMa+Eop5Sc08JVSyk+4JfBF5FIR2SUiWSLyQA3zRUSecc7fJiJD3NGuUkqphnM58EUkEHgemAz0Aa4WkT5nLTYZSHLeUoAXXG23LpVVds5UVHmyCaWU8ojS8kqq7MYj63bHHv4IIMsYs9cYUw4sAqadtcw04HXj8A0QISKxbmj7J06cqeDiJz/n5TX7PLF6pZTynNRUnvnFfUycNZczXbu7/Xoe7gj8OCC32vM857TGLgOAiKSISJqIpBUUFDS6mDYtg+kW3YoFa/fpXr5SqvlITeXkbXeR2m00vY/speW+PY7rebgx9N0R+FLDtLO/jzRkGcdEY+YaY4YZY4ZFR0efU0GzLupK4alyFm8+cE6vV0qpJjd7Nm/2HMPJFmHM2rDYMc3N1/NwR+DnAZ2rPe8E5J/DMm5zftd2DOgUzryv9nrsWJhSSrlTeV4+Lw+bzqjsrQw4lPXfGW68noc7An8jkCQiXUQkBLgKWHrWMkuB65y9dUYCxcaYg25ou0Yiwqwx3dhXWMKK7w55qhmllHKbJRdeweHW7Zi14b0fz3Dj9TxcDnxjTCVwO7AcyADeNsbsEJHfishvnYstA/YCWcA84FZX263Ppf06EB9p48Uv9mKM7uUrpbyX3W6YN+bX9CrMZsy+zf+d4ebrebhleGRjzDIcoV592ovVHhvgNne01VCBAcLNY7ryvx+kszH7GCO6RDZl80op1WCf7z7C7rJA/jkoGklIcBzGiY93hL0bh3v36TNtrxzaiXZhIbzweVb9CyullEVe/HwvHcNbMvWWX0B2Ntjtjns3X9vDpwO/ZXAg149KZPWuAjIOnrC6HKWU+om07CI2ZBdx85iuBAd6NpJ9OvABrjs/kbCQQF78Yo/VpSil1E+88Pke2tqC+dXwzvUv7CKfD/xwWzDJIxP4cGs+OUdLrS5HKaV+sPPQCVbuPMINo7tgC/H8FWd9PvABbrygC0EBAcz9SvfylVLe46Uv9mILCeS68xOapD2/CPyYNi25Ykgcb6flUXCyzOpylFKK3KJSlm7N59cj4omwhTRJm34R+ACzLupGRZWdV9bqoGpKKevN+2ovAQI3Xdi1ydr0m8DvEhXGlP6xvPH1fopPV1hdjlLKjxWcLOOtjblcMbgTHcJbNlm7fhP4ALde3I2TZZW8vi7b6lKUUn5s/pq9VFTZueXibk3arl8Fft+O4Yzr1Z5X1u6jpKzS6nKUUn7oeGk5b3y9n8sGdCQxKqxJ2/arwAe4bWx3jpVW8OYG941Ap5RSDfXqumxKyqu4dWzT7t2DHwb+0IS2jOwaybyv9lJWqRdIUUo1nVNllSxYm82E3jH06tCmydv3u8AHuH1sEodPlPHupjyrS1FK+ZHUbxydRm4f192S9v0y8Ed3b8fAzhG88PkeKqrsVpejlPIDZyqqmPfVPi7oHsWgzhGW1OCXgS8i3DmuO3nHTvPBt3oZRKWU5725IYfCU2XcYdHePfhp4AOM69Wevh3b8PzqLCp1L18p5UFnKqp48Ys9jOgSyXld21lWh98Gvohwx7gkso+W8tE2j11tUSmleGdTHodPlHHX+CRL6/DbwAeY1CeGXh1a8+yqTL3YuVLKI8or7bywOouhCW0Z1c26vXvw88APCBBuH9edPQUlfJyue/lKKfdbvDmP/OIz3DGuOyJiaS1+HfgAk/vF0r19K55dmYVd9/KVUm5UUWXn+c+zGNgpnIt6RFtdjgZ+YIBwx7ju7Dp8ko/TD1ldjlLKh7y/+QC5Rae5c3yS5Xv3oIEPwGUDOtItOoynV+7WvXyllFtUVNl5dnUmAzo5xvDyBhr4OPby7xyfxO7Dp3QvXynlFos355FbdJq7J3jH3j1o4P/gsgEd6d6+le7lK6VcVlFl59lVjmP3Y3t6x949uBj4IhIpIitEJNN537aGZTqLyGoRyRCRHSJylyttekr1vfxl2mNHKeWCxZvzyDt2mrsn9PCavXtwfQ//AWClMSYJWOl8frZK4D5jTG9gJHCbiPRxsV2PmNo/lqT2rXj6M+2Xr5Q6N+WVzr37zhFc3NP6njnVuRr404DXnI9fA6afvYAx5qAxZrPz8UkgA4hzsV2PCAwQ7pqQROaRU3y0Ld/qcpRSzdA7m3Kde/fec+z+e64Gfowx5iA4gh2o82CViCQCg4H1dSyTIiJpIpJWUFDgYnmNN6VfLL06tOafn2XqGDtKqUY5U1HFsysdZ9Ve7AX97s9Wb+CLyGcikl7DbVpjGhKRVsB7wN3GmBO1LWeMmWuMGWaMGRYd3fQbLCBAuGdiD/YVlrBYR9JUSjXCmxtyOHTiDPdN9K5j998Lqm8BY8yE2uaJyGERiTXGHBSRWOBILcsF4wj7VGPM4nOutolM6hND/7hwnlmZyfRBcYQEaWcmpVTdTpdX8fzqPYzsGsmo7lFWl1MjV5NsKTDT+XgmsOTsBcTxZ+5lIMMY85SL7TUJEeHeST3IO3aadzblWl2OUqoZeP3rbApPlXHfpJ5Wl1IrVwP/cWCiiGQCE53PEZGOIrLMucxo4FpgnIhscd6muNiux13cI5oh8RE8tyqLMxV67VulVO1OnqngxS/2MKZHNMMTI60up1YuBb4x5qgxZrwxJsl5X+Scnm+MmeJ8vMYYI8aYAcaYQc7bsrrXbD0R4f5LenKw+Ayp63OsLkcp5cVeWZPNsdIK7pvYw+pS6qQHp+swqlsUF3SP4vnVWZwqq7S6HKWUFyoqKWfeV3u5tG8HBlp0rdqG0sCvx/2X9KSopJxX1uyzuhSllBd68Ys9lJRXct8k7967Bw38eg3qHMGkPjHM+3Ivx0rKrS5HKeVFDhWf4bV12cwYHEdSTGury6mXBn4D3H9JT06VV/Lil3usLkUp5UWeXZWJ3RjumeD9e/eggd8gPWJaM2NQHK+uzeZQ8Rmry1FKeYHswhLe2pjL1SPi6Rxps7qcBtHAb6B7JvbAbgxPr9xtdSlKKS/w5Ke7CA4M4PZx3a0upcE08Buoc6SN5PMSeDstj6wjp6wuRylloe15xXy07SA3XdiF9q1bWl1Og2ngN8Id47oTGhzIk8t3WV2KUspCT3yyk7a2YFLGdLW6lEbRwG+Edq1acPOFXflkxyE25xyzuhyllAW+yixgTVYht43tTuuWwVaX0yga+I1004VdiGoVwuMf78QYvUiKUv7Ebjc88clO4iJCufb8BKvLaTQN/EYKaxHEneOT2LCviNW7ahwcVCnloz7clk/6gRPcO7EHLYICrS6n0TTwz8HVI+LpEhXGX5bt1IukKOUnyiqr+NvyXfSObcP0wV550b56aeCfg+DAAP7n0p5kHjnFO5vyrC5HKdUEXl+3n7xjp3loSi8CA7zv4iYNoYF/ji7p24FhCW15asVuSnRgNaV82vHScp5dlcmYHtFcmOR9ly5sKA38cyQiPDilNwUny5j/lQ6sppQve351FifLKnlwci+rS3GJBr4Lhia0ZXK/Drz05R6OnNQhF5TyRblFpby2bj8/H9KJ3rFtrC7HJRr4LvqfS3tRUWXnqU91yAWlfNHjn+wkIIBmMfxxfTTwXZQYFcZ15yfyVlou3+WfsLocpZQbbdpfxH+2HWTWmG7EhodaXY7LNPDd4M5xSYSHBjNn2Xd6MpZSPsJuNzz6UQYxbVow66LmNYRCbTTw3SDcFszd45NYm3WUVTv1ZCylfMGH2/LZmnuc313SC1tIkNXluIUGvpskj0yga3QYc5ZlUKEnYynVrJ2pqOKJj3fSL64NVzTTk6xqooHvJsGBAcye0pu9BSX8++v9VpejlGqM1FRITISAAEhMZN5z75NffIaHp/YhoJmeZFUTDXw3GterPRcmRfHPz3ZTpNe/Vap5SE2FlBTYvx+M4WDRKf6VJ0xpU87Iru2srs6tNPDdSET4w2V9KCmv4qkVOma+Us3C7NlQWvrD0ycuup4qCeDB1McsLMozXAp8EYkUkRUikum8b1vHsoEi8q2IfORKm94uKaY1145MYOH6HO2mqVRzkJPzw8NNcb34oO9YZm1YTOeMLdbV5CGu7uE/AKw0xiQBK53Pa3MXkOFie83CPRN6EB4azKMf7dBumkp5u/h4AOwIj4yfRYeThdzyzTs/TPclrgb+NOA15+PXgOk1LSQinYCpwHwX22sWwm3B3DupJ9/sLeLj9ENWl6OUqsucOWCz8W7/CWyPTeKBz1/FFhzomO5jXO1cGmOMOQhgjDkoIu1rWe6fwO+B1i6212xcPbwzC9fn8KePvmNsz/aEhjS/iyUo5ReSkymuEp7YHMDQvO+YVrIP5s6F5GSrK3O7evfwReQzEUmv4TatIQ2IyGXAEWPMpgYunyIiaSKSVlBQ0JCXeKWgwAAendaX/OIz/OvzLKvLUUrV4R8RAzkW2ppH/5qCZGf7ZNhDAwLfGDPBGNOvhtsS4LCIxAI472s6zXQ0cLmIZAOLgHEi8kYd7c01xgwzxgyLjm6+406Tmsrwi4cwY8dqXlqRQfYrb1pdkVKqBhkHT/D619kkn5dA347hVpfjUa4ew18KzHQ+ngksOXsBY8yDxphOxphE4CpglTHmGhfb9W7V+vU+uPoVgisreGx5pmO6UsprGGP4v6U7CA8N9onRMOvjauA/DkwUkUxgovM5ItJRRJa5WlyzVa1fb/uSY9y99k1WdhnKyud1L18pb7J0az4b9hXx+0t7EWELsbocjxNv7jY4bNgwk5aWZnUZjRcQANW2a0VAIJNveJYzQS1Y8cxM/QFXKS9w4kwF4//+BbHhLXn/1tHN9jq1ZxORTcaYYTXN0zNtPeGs/rvB9ioe+/QF8iJi9AdcpbzEU5/upvBUGX+a3s9nwr4+Gvie4OzXW935R/cwI6Kcl77Yy56CUxYVppQCSD9QzOtfZ3PNeQkM6BRhdTlNRgPfE5KTHf14ExJAxHE/dy4P3jaZFsEB/N8SPQNXKavY7YaHP0gnMiyE+yf1tLqcJqWB7ynJyZCdDXa74z45mfatW/K7S3qyJquQD7cdtLpCpfzSoo25bMk9zkNTehNuC7a6nCalgd/Eks9LYECncB798DuKSyusLkcpv1JwsozHP87gvC6RzPChC5s0lAZ+EwsMEP48oz9FJWU8sXyn1eUo5Vce++g7zlTYmTOjPyL+8UNtdRr4FugXF85vRndh4foc0rKLrC5HKb/wxe4Clm7N55aLu9G9fSury7GEBr5F7pnYg7iIUB56fzvllXoNXKU86XR5FQ9/sJ2uUWHcOrab1eVYRgPfImEtgnh0Wl92Hz7F3C/3WF2OUj7tmVWZ5BadZs6M/rQI8t8THzXwLTS+dwxT+nfgmVVZ2jdfKQ9JP1DM3C/3cuXQTpzfzbeuUdtYGvgWe+TyvoQGB/LAe9uw27VvvlLuVFll54HF22hrC+HhqX2sLsdyGvgWa9+6JbOn9mZj9jEWbsip/wVKqQabv2Yf6QdO8Ni0vn7X574mGvhe4MqhnbigexSPf7yTg8WnrS5HKZ+wr7CEf6zYzSV9Y5jcP9bqcryCBr4XEHH0za+yG2a/n67DLijlIrvd8MB72wgJCuDRaf2sLsdraOB7ifh2Nu6/pCerdh7h/W8PWF2OUs3aG+v3s35fEf87tQ8xbVpaXY7X0MD3ItePSmRYQlseWbqDIyfOWF2OUs1SztFSHv94J2N6RHPlsE5Wl+NVNPC9SGCA8NdfDKCs0s5D72/XQztKNZLdbvj9e1sJEOHxK/xz+IS6aOB7ma7Rrbh/Uk8+yzjCB1v00I5SjZG6IYdv9hYxe2pvOkaEWl2O19HA90K/uaALQ+IjeGTpdxzWQztKNcj+oyX8ZVkGF3SP4qrhna0uxytp4HuhwADhySsHUlZZxe/f3aaHdpSqR5XdcP87W384LKqHcmqmge+luka34sHJvflidwFvbsi1uhylvNrLa/ayMfsYj/ysrx7KqYMGvhe7dmQCo7u340//+Y6co6VWl6OUV9p9+CRPLt/NpD4xXDHE/y5q0hga+F4sIED42y8GEijCfe9soUrH2lHqR8or7dzz1hZatwziz9orp14a+F6uY0Qof5zWl43Zx3jxCx1GWanq/vHZbnbkn+DPV/QnqlULq8vxehr4zcCMwXFcNiCWf6zYzba841aXo5RX+GbvUV78Yg9Xj+jMJX07WF1Os+BS4ItIpIisEJFM533bWpaLEJF3RWSniGSIyPmutOtvRIQ50/sT3boFdy/aQml5pdUlKWWp4tMV3Pf2VhIibTrscSO4uof/ALDSGJMErHQ+r8nTwCfGmF7AQCDDxXb9TrgtmL//ciD7jpbwp//o5lP+yxjDH5akc+jEGf551WDCWgRZXVKz4WrgTwNecz5+DZh+9gIi0gYYA7wMYIwpN8Ycd7FdvzSqWxQpF3Zl4focPkk/ZHU5Slli8eYDLNmSz13jkxjUOcLqcpoVVwM/xhhzEMB5376GZboCBcACEflWROaLSFhtKxSRFBFJE5G0goICF8vzPfdN6kn/uHD+571t5B/XsfOVf9lbcIr/XZLOiC6R3Da2u9XlNDv1Br6IfCYi6TXcpjWwjSBgCPCCMWYwUELth34wxsw1xgwzxgyLjo5uYBP+IyQogGeuHkxllZ27F2lXTeU/yivt3LnoW0KCAnj6qkEEBmgXzMaqN/CNMROMMf1quC0BDotILIDz/kgNq8gD8owx653P38XxB0Cdoy5RYTw2vR8bsot4dlWm1eUo1ST++slO0g+c4ImfDyA2XM+mPReuHtJZCsx0Pp4JLDl7AWPMISBXRHo6J40HvnOxXb93xZBOzBgcxzMrM1m3p9DqcpTyqJUZh5m/Zh/XjIzXLpgucDXwHwcmikgmMNH5HBHpKCLLqi13B5AqItuAQcCfXWxXAY9N70diuzDuWrSFgpNlVpejlEccOH6ae9/eSp/YNtoF00UuBb4x5qgxZrwxJsl5X+Scnm+MmVJtuS3O4/IDjDHTjTHHXC1cQasWQTyfPIQTpyu4+61v9Xi+8jnllXZuX7iZKrvh+eQhtAwOtLqkZk3PtG3mese24Y+X92Vt1lGeW5VldTlKudXflu/k25zjPP7z/nSJqrVzn2ogDXwf8KvhnZkxOI5/rtzNV5nalVX5hk/SDzHvq31cOzKBywZ0tLocn6CB7wNEhDkz+pHUvhV3vvktB7R/vmrm9hac4v53tjKwUzgPX9bb6nJ8hga+j7CFBPHiNUOpqDLc+sYmyiqrrC5JqXNSWl7JLW9sJjhQ+Nc1Q2kRpMft3UUD34d0jW7Fk1cOZGteMY9+qD1fVfNjjOHBxdvZfeQkz1w9mDi9epVbaeD7mEv7dWDWRV1JXZ/D2xv10oiqeVmwNpslW/K5b2IPLkzSM+3dTQPfB/1uUk8uTIri4Q/S2ZyjPWBV87A2q5A5yzK4pG8Mt16s4+R4gga+DwoKDODZqwfTIbwlv/33Jo6cOGN1SUrVKbeolNsXbqZrVBh//+UgAnScHI/QwPdREbYQ5l43lFNllczSH3GVFystryTl35uoshvmXTeMVjq+vcdo4PuwXh3a8PcrB/JtznEeXLwdY/RMXOVd7HbDvW9tZdehEzxz9WAS9eQqj9LA93GT+8dyz4QeLN58gJe+3Gt1OUr9yFMrdvPJjkM8NKU3F/es6XIayp008H1ZaiokJnLnpJ78bH8aT3ycwac79EpZyjt88O0BnludxdUjOnPjBV2sLscvaOD7qtRUSEmB/fsRY/jbu39mwOE93J2aRvqBYqurU35u0/4ifv/eNkZ2jeSPl/dDRH+kbQoa+L5q9mwoLf3hacvKcua980ciTh7jxtc26uURlWWyC0u46bU04iJCeSF5KCFvvQmJiRAQ4LhPTbW6RJ+lge+rcnJ+Mql9yTFeeesPlJZV8ZtXN3LyTIUFhSl/VlRSzvULNgCw4PrhtP3gnR++iWKM4z4lRUPfQzTwfVV8fI2Te9kML1wzlKwjp7g1dTMVVfYmLkz5qzMVVaS8nkZ+8Rnmzxzm6JFz1jdRwPF89mxrivRxGvi+as4csNl+PM1mgzlzuCApij/P6M9XmYU88J5211SeV2U33PPWFtL2H+MfvxzE0IRIx4wavonWOV25RAPfVyUnw9y5kJAAIo77uXMd04FfDu/M3ROSeG9zHn9dvsviYpUvM8bwyNIdfJx+iIen9mbqgNj/zqzlm2it05VL9JQ2X5ac/EPA1+Su8UkUnCzjhc/30L51C24YrV3jlPs9tyqLf3+zn1kXdeWmC7v+eOacOY5j9tUP6zi/iSr30z18PyYiPDqtH5f27cAfP/yOJVsOWF2S8jEL1+fw9xW7uWJIHA9c2uunC9TzTVS5l3jz8dthw4aZtLQ0q8vweWcqqrh+wQY2Zh/jpWuGMqFPjNUlKR+wdGs+dy36lot7RDP3umEEB+r+ZVMQkU3GmGE1zdN3QNEyOJD5M4fTr2Mbbl24mXVZhVaXpJq5VTsPc+9bWxieGMkL1wzVsPcS+i4oAFq1COLVG0aQ2M7GTa+n6Tj66pyt21PILW9spk/HNrw8cxgtg/UShd5CA1/9oG1YCG/ceB7RrVsw85UNbMs7bnVJqpnZsK+IG19NIz7Sxqs3jKB1y2CrS1LVuBT4IhIpIitEJNN537aW5e4RkR0iki4ib4pIS1faVZ7Tvk1LFt48kvDQYK59eYOOu6MabNP+Im5YsIHYiJak3nwekWEhVpekzuLqHv4DwEpjTBKw0vn8R0QkDrgTGGaM6QcEAle52K7yoLiIUN68eSStWgRx7cvryTh4wuqSlCc4R1N1xxg23+YcY+YrG2nfpiVv3jyS9q11n84buRr404DXnI9fA6bXslwQECoiQYANyHexXeVhnSNtLLz5PFoEBfLred/onr6vqTaaqqtj2GzaX8S1L28gMiyEhTefR0wbDXtv5WrgxxhjDgI4739yBQNjzAHgSSAHOAgUG2M+rW2FIpIiImkiklZQUOBiecoVCe3CeGvWSGwhQfx63jdsyT1udUnKXdw0hs36vUe59uUNRLduwVuzRhIbHurGIpW71Rv4IvKZ89j72bdpDWnAeVx/GtAF6AiEicg1tS1vjJlrjBlmjBkWHR3d0H+H8pDvQz/cFsw189eTll1kdUnKHdwwhs3arEJmLthAx4hQ3krRsG8O6g18Y8wEY0y/Gm5LgMMiEgvgvD9SwyomAPuMMQXGmApgMTDKnf8I5Vmd2tp4e9b5RLduwbUvb+DL3frNq9lzcQyb5TsOccOCjSS2C2NRykja62GcZsHVQzpLgZnOxzOBJTUskwOMFBGbOC5rMx7IcLFd1cRiw0N5e9b5JEaFceNrG1m2/aDVJSlX1DGaan3e3ZTHLW9som9cGxaljCSqVQsPFanczdXAfxyYKCKZwETnc0Sko4gsAzDGrAfeBTYD251tznWxXWWB6NYtWJQykoGdIrh94Wbe3KBD2DZb5ziGzctr9nH/O1sZ3T2KN248jwibdr1sTnQsHdVop8uruCV1E5/vKuCu8UncPSFJr0nq4+x2w5+XZTB/zT4m9+vAP68aRIsgPYPWG+lYOsqtQkMCmXfdMK4c2omnV2byP+9t0ytn+bCyyiruXPQt89fs4/pRiTz36yEa9s2UjoevzklwYAB//cUAYiNCeWZlJodOlPHcrwfTRk+l9ynHSsqZ9cYmNuwr4sHJvUgZ01W/zTVjuoevzpmIcO/EHjx+RX/WZRXy83+tI7eotP4XqmYh68gppv9rLVtyj/P0VYOYdVE3DftmTgNfueyqEfG8/psRHD5xhmnPr9W++j5gTWYhV/xrLSVllbx580imDYqzuiTlBhr4yi1GdY/ig9tGEx4azNXzvmHheu3B0xwZY5j/1V5mLthAbHgo7986mqEJNY6JqJohDXzlNl2jW/HBraMZ1S2Kh97fzoOLt1FWWWV1WaqBTpdXcfdbW/jTfzKY0Ls97906is6RtvpfqJoN/dFWuVW4LZhXrh/OUyt28fzqPWQcPMnzyUOIi9DT7r1ZdmEJt6RuZuehE/zukp7cclE3AgL0eL2v0T185XaBAcLvLunFi9cMIevIKaY+8xWrdh62uixVi/9sO8hlz67hYPFpXrl+OLeN7a5h76M08JXHXNovlo/uuICO4aH85tU0/rIsg/JK7a/vLc5UVPGHJenctnAzSTGt+M+dFzK2508GvFU+RANfeVRiVBiLbx1F8nnxvPTlXq54YS17Ck5ZXZbfyzh4gsufW8PrX+/npgu68FbK+XrYzQ9o4CuPaxkcyJwZ/XnxmqHkHTvN1Ge+4o1v9uPNw3r4Krvd0Qtn2nNrKSqp4NUbhvPwZX0ICdIo8Af6o61qMpf268Dg+Ajuf2crD3+QzvIdh3j85wN0z7KJZBeW8Pt3t7Ehu4gJvWN44uf9aacjXfoVHTxNNTm73ZC6IYe/LMsgQITZU3tz1fDOehanh1TZDa9/nc0Tn+wkODCAP1zWh18M7aTb20fVNXiaBr6yTG5RKb9/dxtf7z3KiMRI5szoR1JMa6vL8ik78ot56P10tuYeZ2zPaP5yxQA6hOvFSnyZBr7yWna74Z1Nufzl452UlFWSMqYrt43tji1Ejza64uSZCp7+LJMF67Jpawvm4al9mDaoo+7V+wEdHllZKzUVEhMhIMBxn5r6w6yAAOFXw+NZee9FXD4wjudX72Hck1+wZMsB9/+oW0cdvsJuN7y9MZexT37B/DX7+NXwzqy892KmD47TsFe6h688LDUVUlKgtNoomjZbrVdX2phdxKMffsf2A8UMiY/goSm9GZYY2eR1NEfr9hTyl2U7f9h2//ezvgzsHGF1WaqJ6SEdZZ3ERNi//6fTExIgO7vGl9jthnc35/G35bsoOFnGhN7t+d0lvejZwYXj++dQR3ORfqCYJz7ZyVeZhcSGt+SByb24fKAevvFXGvjKOgEBUNNnTATsdZ91W1peyYK12bz4+R5OlVcytX8sd4xLOrfgd6EOb5V+oJhnV2WyfMdhImzB3D62O9eMTKBlsF6Nyp/VFfj6y5jyrPj4mves4+PrfaktJIjbxnbn1yMcZ+n+++tsPtp2kEv6xvDbi7oxOL4Rw/a6UIc3McawMfsYL3yexepdBbRuGcSd45O46cIuerUxVS8NfOVZc+bUfOx8zpwGr6JtWAgPTO7FrDFdWbAumwVr97F8x2GGJrTlxgu6MKlPDEGB9fQ/cEMdVqqosrNs+0FeXrOPbXnFtLUFc/+kHlw3KlGDXjWYHtJRnpeaCrNnQ06OY496zhyXfig9VVbJO2m5LFibTU5RKR3atOSXwzvzq+Gd6z5r1811NIWco6Us2pjD22l5FJ4qo2t0GL8Z3YWfD+lEaIgeulE/pcfwlU+qshs+yzjMwvU5fJlZgACju0cxfVAck/rG0LqZ7vkWn67gk/SDfPBtPl/vPUqAwLheMSSfF89FPaJ16GJVJw185fNyi0p5Jy2X97ccILfoNC2CAhjbsz0T+8Qwrld72oaFWF1inY6eKmNlxhE+/e4wX+4uoLzKTpeoMGYMjuPKYZ2IDdfxhlTDaOArv2GMYXPOMZZsyWf5jkMcPlFGYIAwNL4to7tHcUFSOwZ0iiC4vmP+HlZeaWdL7nHWZBWyJrOALbnHsRvoGN6SS/p1YPqgOAZ0CteularRPBb4InIl8AjQGxhhjKkxnUXkUuBpIBCYb4x5vCHr18BXrrDbDdsPFLPiu8N8sbuA9PxijIHQ4EAGdApncHxbBnWOoFeH1sRH2jx2qKTKbth/tISdh06yNfc4m3OOsS2vmLJKOwECAzpFMKZHNJP6xNC3YxsNeeUSTwZ+b8AOvATcX1Pgi0ggsBuYCOQBG4GrjTHf1bd+DXzlTsdKylm35ygbs4v4NucYO/JPUGl3fP5tIYF0i25FfDsbCZE24tqGEt2qBe1ataBdWAi2FoHYQoIIDQ7k+78LdgOnK6ooLa+kpKyKopIyCk+VU3CyjLxjp8ktKmV/UQlZR05xpsLR1z8kMIC+cW0Y3LktI7pEcn63doSHNs/fGpR38lg/fGNMhrOBuhYbAWQZY/Y6l10ETAPqDXyl3KltWAhTB8QydUAs4LjE385DJ9l16AQZB0+yp+AUOw4Uszz90A9/CM5VcKDQqa2NzpE2ks9rR88OrenVoTU9YlrriVHKMk3RDz8OyK32PA84r7aFRSQFSAGIb2YnxajmpWVwIIM6RzDorPFmKqvsFJwqo/BkOYUlZRSdKqe0oorT5ZWcLrdjcPwxEITQkABCQ4IICwkkMiyEqFYtiGrVgujWLQjU3jTKy9Qb+CLyGdChhlmzjTFLGtBGTZ/6WnefjDFzgbngOKTTgPUr5VZBgQHEhodqzxjlc+oNfGPMBBfbyAM6V3veCch3cZ1KKaUaqSn6pm0EkkSki4iEAFcBS5ugXaWUUtW4FPgiMkNE8oDzgf+IyHLn9I4isgzAGFMJ3A4sBzKAt40xO1wrWymlVGO52kvnfeD9GqbnA1OqPV8GLHOlLaWUUq7RSxwqpZSf0MBXSik/oYGvlFJ+QgNfKaX8hAa+Ukr5CQ18pZTyExr4SinlJzTwlVLKT2jgK9XUUlMhMRECAhz3qalWV6T8RFMMj6yU+l5qKqSkQGmp4/n+/Y7nAMnJ1tWl/ILu4SvVlGbP/m/Yf6+01DFdKQ/TwFeqKeXkNG66Um6kga9UU6rtKm56dTfVBDTwlWpKc+aAzfbjaTabY7pSHqaBr1RTSk6GuXMhIQFEHPdz5+oPtqpJaC8dpZpacrIGvLKE7uErpZSf0MBXSik/oYGvlFJ+QgNfKaX8hAa+Ukr5CTHGWF1DrUSkANh/ji+PAgrdWI67aF2No3U1jtbVOL5YV4IxJrqmGV4d+K4QkTRjzDCr6zib1tU4WlfjaF2N42916SEdpZTyExr4SinlJ3w58OdaXUAttK7G0boaR+tqHL+qy2eP4SullPoxX97DV0opVY0GvlJK+YlmHfgicqWI7BARu4jU2oVJRC4VkV0ikiUiD1SbHikiK0Qk03nf1k111bteEekpIluq3U6IyN3OeY+IyIFq86Y0VV3O5bJFZLuz7bTGvt4TdYlIZxFZLSIZzvf8rmrz3La9avusVJsvIvKMc/42ERnS0Ne6ogF1JTvr2SYi60RkYLV5Nb6fTVjbxSJSXO39+UNDX+vhun5XraZ0EakSkUjnPI9sMxF5RUSOiEh6LfM9+/kyxjTbG9Ab6Al8DgyrZZlAYA/QFQgBtgJ9nPP+CjzgfPwA8ISb6mrUep01HsJxwgTAI8D9HtheDaoLyAaiXP13ubMuIBYY4nzcGthd7X10y/aq67NSbZkpwMeAACOB9Q19rYfrGgW0dT6e/H1ddb2fTVjbxcBH5/JaT9Z11vI/A1Z5epsBY4AhQHot8z36+WrWe/jGmAxjzK56FhsBZBlj9hpjyoFFwDTnvGnAa87HrwHT3VRaY9c7HthjjDnXs4obytV/r2Xbyxhz0Biz2fn4JJABxLmp/e/V9VmpXuvrxuEbIEJEYhv4Wo/VZYxZZ4w55nz6DdDJTW27XJuHXuvudV8NvOmmtmtljPkSKKpjEY9+vpp14DdQHJBb7Xke/w2KGGPMQXAECtDeTW02dr1X8dMP2+3Or3SvuOvQSSPqMsCnIrJJRFLO4fWeqgsAEUkEBgPrq012x/aq67NS3zINee25auy6b8Sxl/i92t7PpqztfBHZKiIfi0jfRr7Wk3UhIjbgUuC9apM9uc3q4tHPl9df8UpEPgM61DBrtjFmSUNWUcM0l/ui1lVXI9cTAlwOPFht8gvAYzjqfAz4O/CbJqxrtDEmX0TaAytEZKdzz+ScuXF7tcLxH/NuY8wJ5+Rz3l5nr76GaWd/VmpbxiOfs3ra/OmCImNxBP4F1Sa7/f1sZG2bcRyuPOX8feUDIKmBr/VkXd/7GbDWGFN9z9uT26wuHv18eX3gG2MmuLiKPKBzteedgHzn48MiEmuMOej82nTEHXWJSGPWOxnYbIw5XG3dPzwWkXnAR01ZlzEm33l/RETex/F18kss3l4iEowj7FONMYurrfuct9dZ6vqs1LdMSANee64aUhciMgCYD0w2xhz9fnod72eT1FbtDzPGmGUi8i8RiWrIaz1ZVzU/+Ybt4W1WF49+vvzhkM5GIElEujj3pq8CljrnLQVmOh/PBBryjaEhGrPenxw7dIbe92YANf6i74m6RCRMRFp//xiYVK19y7aXiAjwMpBhjHnqrHnu2l51fVaq13qdszfFSKDYeRiqIa89V/WuW0TigcXAtcaY3dWm1/V+NlVtHZzvHyIyAkfuHG3Iaz1Zl7OecOAiqn3mmmCb1cWzny93/wrdlDcc/7nzgDLgMLDcOb0jsKzaclNw9OrYg+NQ0PfT2wErgUznfaSb6qpxvTXUZcPxwQ8/6/X/BrYD25xvamxT1YWjF8BW522Ht2wvHIcojHObbHHeprh7e9X0WQF+C/zW+ViA553zt1Otd1htnzM3baP66poPHKu2bdLqez+bsLbbnW1vxfGD8ihv2GbO59cDi856nce2GY6du4NABY7surEpP186tIJSSvkJfziko5RSCg18pZTyGxr4SinlJzTwlVLKT2jgK6WUn9DAV0opP6GBr5RSfuL/AbqzfCFYFbBIAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.preprocessing import PolynomialFeatures\n", "\n", "\n", "np.random.seed(1)\n", "\n", "xi = np.linspace(-1,1,10)\n", "noise = np.random.normal(0,0.05,10)\n", "\n", "ti = (xi-1)*(xi+1) \n", "tinoisy = ti+noise\n", "\n", "plt.scatter(xi,tinoisy, c = 'r')\n", "plt.plot(np.linspace(-1,1,100), (np.linspace(-1,1,100)-1)*(np.linspace(-1,1,100)+1) )\n", "plt.show()\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4. Regularization.\n", "\n", "\n", "On top of the LinearRegression model, scikit learn also provides models for the sum of squares and sum of absolute values penalties on the weights. Those two regularized models are respectively known as Ridge Regression and LASSO. We will use those models with multivariate linear model so see how they can reduce the amount of overfitting " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from sklearn.linear_model import Ridge\n", "from sklearn.linear_model import Lasso\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 5. Classification.\n", "\n", "\n", "The linear model can be used to learn classification boundaries as well. When considering $K$ classes, we can just define our attribute/feature vectors as $K$ dimensional vectors whose $k^{th}$ attribute/feature is $1$ if the point belongs to class $\\mathcal{C}_k$ and $0$ otherwise. We can then minimize the sum of residual as in the regression case \n", "\n", "$$\\min_{\\mathbf{\\beta }} \\sum_{i=1}^N \\left(y^{(i)} - (\\beta_0 + \\beta_1 x_1^{(i)} + \\beta_2 x^{(i)}_2)\\right)^2$$\n", "\n", "For any plane, note that the equation of the plane $\\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 = 0$ gives the points that are lying on the plane. A point above the plane will return a positive value for $\\beta_0 + \\beta_1 x_1 + \\beta_2 x_2$ and a point below will return a negative value for $\\beta_0 + \\beta_1 x_1 + \\beta_2 x_2$. \n", "\n", "The minimization problem above will then find the plane that gives the larger values to the points with targets/labels $1$ and the smallest values (or even possible negative values) to the points with targets $0$.\n", "\n", "In particular our plane (i.e. the points satisfying the equation $\\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 = 0$) should thus located in between the two classes. \n", "\n", "\n", "Using this idea we will learn a classifier for the dataset below. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAtwklEQVR4nO3deXiU1fUH8O/NnslSkF0QQUEEUQSiolg3QCkiiIAVxapUqVWrP5cqKtalVuuKorgg7oJiVRQVwYUKsglBUEEWEUSRLSwKmWSyzfn9cUizzDvJJPO+887y/TzPPJJJ8s7NmJy5c++55xgRARERRa8ktwdARER1Y6AmIopyDNRERFGOgZqIKMoxUBMRRbkUJy7avHlz6dChgxOXJiKKS8uXL98lIi2sPudIoO7QoQPy8/OduDQRUVwyxmwO9jkufRARRTkGaiKiKMdATUQU5RioiYiiHAN1uESAVauAhQuB4mK3R0NEcYiBOhybNgHdugF9+gCDBgEtWwIvveT2qIgozjBQN5YIMGAAsH494PUC+/YBhYXA1VcDy5e7PToiiiMM1I21dCmwYwfg99e83+cDnnzSnTERUVxioG6sXbuAJIunz+8Htm2L/HiIKG4xUDdWnz5AaWng/R4PcM45kR8PEcUtBurGatYMuP12ICur6r7MTKB9e+Cyy9wbFxHFHUdqfSSM8eOBvDzgiSeAPXuA4cOBK6/UWTURkU1CCtTGmCYApgDoDkAAjBGRxQ6OK3YMHKg3IiKHhDqjfhzAbBEZYYxJA8ApIxFRhNQbqI0xuQBOAXApAIhIKQCLXTQiInJCKJuJhwEoAPCiMWaFMWaKMSarvm8iIiJ7hBKoUwD0AvC0iPQE4AUwrvYXGWPGGmPyjTH5BQUFNg+TiChxhRKotwDYIiJfHvj4LWjgrkFEJotInojktWhh2U2GiIgaod41ahHZboz52RjTRUTWAegH4Dvnh0YUJr/f+vQo1SQCLFmiZRHattUDW+npbo+Kqgn1t/hvAKYaY74BcCyA+xwbEVG43noL6NgRSE4GWrXS2isibo8qOpWWanrpgAHALbcAY8booa3vv3d7ZFRNSOl5IrISQJ6zQyGywQcfAJdcAhQV6cc7d2oAKi0FbrjB3bFFo4kTgS++qKqlXlKiVSDPPx9YscLdsdH/8H0hxZfbb68K0pWKioB77w2sdEjA888HNrwQAdauBbZudWdMFICBmuLLxo3W9xcW6o1qKi8P/rmyssiNg+rEQE3xpXNn6/tzc4Hs7MiOJRZceCGQkRF4/yGH6Fo1RQUGaoov99+vVQyr83iAe+5hBoiVm28Gjjyy6kXM49EXtddfB4xxd2z0P6yeR/HlrLOA//xHA9D69cDBBwN33qnZDBQoKwtYtgx4/31g0SKdSY8eDRx0kNsjo2qMOJC2lJeXJ/n5+bZfl4hq8fu1ofIzz2gbuAsvBK69lqV2Y5AxZrmIWGbXcUZNFMsuvhh47z1tsAzoEs9//qMHWFJT3R0b2YaLdkSxavVqYMaMqiANaKrd+vXAu++6NiyyHwM1UaxauNB6w6+wEPj008iPhxzDQE0Uq1q31mPytaWnM7UuzjBQE8WqgQN107D2rDolBbj0UleGRM5goCaKVWlpwLx5QJcuGrCzs7UI1fvvaxU8ihvM+iCKZV26AN99B2zYoAWVunXjwZ44xEBNFOuMCX50nuICX3qJiKIcAzURUZRjoCYiinJco44FixcDr72mtYNHjQJOPZWVzYgSCAN1tLv9duCxx6q6cEydqtXNnnnG1WERUeRw6SOarV8PPPqotpIS0ZvXC7z6qpamJKKEEFKgNsb8aIz51hiz0hjjXP3SoqK6WwMlmlmzrLtn+3x6qIGIEkJDZtSni8ixweqlhmXxYuDoo6vaJY0ZU7MiWKLKzLSu5ZCSwnrDRAnE/aWPDRuAAQOAVauAigo9XfX668Dw4W6PzH3nnWc9o05KAi64IPLjISJXhBqoBcDHxpjlxpixVl9gjBlrjMk3xuQXFBSEPoIJEzQ4V+fzAfPnaxBPZC1aANOm6ew5J0dvGRnAs88CHTq4PToiipBQsz76ishWY0xLAJ8YY9aKyPzqXyAikwFMBrQVV8gjWLXKel06LU0DdadOIV8qLp17LrBtGzB7tr7jGDgQaNrU7VERUQSFFKhFZOuB/+40xswAcDyA+XV/V4hOPFHbBpWW1ry/pAQ46ihbHiLm5eYC55/v9iiIyCX1Ln0YY7KMMTmV/wZwJoBVto2gshFn9QMcHg8wcqR2RI41O3YA//43MHasHlKpvaxDRNRAocyoWwGYYTSQpgCYJiKzbRvBwQcDX34J3HQTMHeuzh6vvhq45RbbHiJivvwS6N9fl3J8Pt0Uvfdevf93v3N7dLGrsFA3VXNydKns5ZeBfft0s3XAAJb1pLhnxCqrIEx5eXmSn+9cunVUEgGOOCJwAzQ9HbjuOuCBB9wZl5WvvgLy83VDsl8/6xTAaPDjj8All2j6JgC0awds3aovhBUVmsp55pnaddsqWO/cqUG+QwcGc4p6xpjlwdKf+dtrly1b9FZbSQkwfXrkx2OlpAT4wx+A3/8euP56YMQIfXHZutXtkQUqKQFOOglYsAAoK9Pbpk16f0WFfk1hITBnDvDhhzW/d/t2rYfSvr3m5x9yCPDxx5H/GYhswkBtl7Q065xnQGfV0eChh7R1U1GR3vbvBzZv1toh0ea99zQQ+/11f53XqzPqSiK6HLJokQb1oiJ9IRo2TI/kh6KwEBg3TmfwhxwC3HorD2CRqxio7dKqFXDssYHLCB6PbixGg+eeqyruVKmiAli4EPjtN3fGFMymTRpk65OUBKxbB7RsqSc2u3XT5afaKZ+lpcCkSfVfr6ICOOUULYT1yy/6Lumxx4DTTqv/RYPIIQzUdpo+XTdHc3I0QHs8ugZ87bVuj0zVToGsrqwscuMIRc+eeoS+PklJwNdfAwUFGmTXrtWN3NrKy4GNG+u/3pw5wPff18zW8fn0up98Evr4iWzEQG2nQw/VYDB9up64XLgQmDkTSE11e2Rq+HBdoqmtc2egefPIj6cu/fvr+nn1ZaPUVA3M2dl6S0/XpY5QUiArXzTrk59vvcxRVKSfI3IB61HbLSVFN+yi0d13a0W+nTs1GGVm6nhffdXtkQVKStL19Lvu0vGJaH2TW2/VDUavV18Yhw7VtfbajKnaM0hLA5o102Jf9Tn0UCArS9epq/N49HNELmB6XqIpLtYZ/4IFOmO97DKtKeK0H34A7rtP32UcfrgG3JNPDu+a+/bp3kDtpQ5jgGOO0UC9b58ew7/99tDeNXi9ms63e3dVoDdGv3fz5tCWYyqJsBMPhayu9DyIiO233r17C9H/rFkjkpMjkpxc2f5AxOMReeut8K997bV6rarWCvpxfn7jr7lunUhenkhamt6OP15k/frQv3/WLJEjjtCxpKaKNG8ucumlIj/80PgxUdwDkC9BYipn1OS8884D3n03MH2xTRvNqgjnMEpFhc7UJ0wAfv0V6N4dmDhRszTCtXu3/rdZs9C/Z948YNCgwIwVY3STedkyfSdDVAsPvJDy+YCHH9ZDID17arpaJDrqfPGFdY753r1aGyUcycnAHXcAe/Zo0P7mG3uCNKABuiFBGtCxWKUViuha+vjx9oyNEgo3ExNFRQVwxhnAypVVudQ33wx89JG29XJyLbVlS2DXrsD7ReytgRIN68Hr1gX/nIjWWSdqIM6oE8Xs2cC339Y88FJUBHz+ObB0qbOPPW5cYOuwjAxg1Kj4aynWrVvdn2/dOjLjoLjCQJ0oFiwITDkD9KDLwoXOPvbo0VoNMTNTqyOmpwNDhgBPPeXs47rhn/8Mnhni8eiLFlEDMVAnioMPtg4gaWm6qeckY4B//EPzt+fNA37+WVMEG5LqFitOPlkPOXXvXrUUk56uQXr8ePa6pEZh1kei2L0b6Ngx8HDIQQdp5kU8Bs1oUFCgRaE6d46/ZR6yFbM+SLMXPv5YS39mZWnQ6NxZ16gZpJ3TogXQo0fdQfqrr3QmnpamG6//+ldVKVciMFA756ef9A8wmlpx9emjxfjz87WQ0bp1mqoXq8rK9Ih5q1Za+2Po0NjrXL9+vdbOXrhQf56CAs0Lv/pq5x9bRJeiJk3SF3FWB4xaXPqwW0GBHvDIz68qxvTYY6HVmaCGGTlSmwZUZrIkJWm635o1GrxjwZgxwCuvBM6gMzJ0Ld+pYln792u65tq1+tgpKbqPsWBB9BXoShBc+oikc87RHok+n/4x7N8P/O1v+gdA9tm4Efjgg5rphn6/fhxL2STLl1svc6SnO/vu4JZbNF2zsFCfs/379Tm98krnHpMajYHaTt9/ryfjatd2Li4GHnnEnTG5bds24M479V3GAw9UHcsO16pV1p1zfD5gyRJ7HiMSjj7a+gi9zwccdphzjzttWuCyXFmZdtbh+njUCTlQG2OSjTErjDEfODmgmLZjh3XtaRHtFhLLGrNE9s03QJcuGqBnzNAyq0ceqd1bwtWpk3UjhNTU2Fp3HzdOlzmqy8wEzj9fNxadEqx0QGVpK4oqDZlRXwdgjVMDiQs9elh3SklPB846K/LjscN332ntjORkDYKZmZp3fdtt1p1UqrviCn1LXTlzKy7Wmhw33hj+uLp1A044IXBWnZ6uS02xont37SpzzDGad52VpRuJzz/v7OOec46uS1eXlKQbm7XvJ/cFK6tX/QagHYDPAJwB4IP6vj6hy5w++KBIVlZVyc20NJE2bUR27XJ7ZA33yy8iubk1S4hW3jIyRE47TcTvt/5en08kKcn6e7Oy7Bnf/v0il1wikp6uJVR79hRZurTq84sWiQweLHLkkSJjxohs2GDP4zqlvDz482m3rVtF2rat+l3NytJyrNH+HMUxhFvm1BjzFoD7AeQAuElEBlt8zVgAYwGgffv2vTdv3mzbi0nMmTVL16R37tSSlzfdFJni/HYbP16r7QVLMczKAj77TMt2Tpyom3utWwPXX68NYrOyrJcnmjWzLtLUWBUV+k6m+hLCu+8CF11UVckuOVlzmb/8Euja1b7HjmVFRcAbb2gaabduetQ/N9ftUSWssBoHABgM4KkD/z4NnFEnjkGDrGfElbfMTJFHHhFp315n2NUL9z/xhMjo0TrbrT0Tv+UWZ8ft94scfHDgeI0ROfdcZx+bqJFQx4w6lMWovgCGGGMGAcgAkGuMeU1ERtvwIkLRrFcvnTEHm1GnpGjZ1J07a65XFxVp+tf332uK2bff6vpnebmud999t7PjLiiwzi4R0drYRDGm3s1EEblVRNqJSAcAFwCYyyCdIK66KjAjoVJKimYlrFljvamYkqLZHYsXaw3m557T7iazZlmn1dkpJyf452JxCYoSHvOoKbg2bYBFi/QEW1JS1S01FejXT2enwSrvlZdXnXDr1Qv44x+Bo46KzLgzM4ELLwx8kfF4tFkCUYxpUB6OiHwO4HNHRkLRqVs3Xf6otG+fbsxlZenH11+vn6/efiolRTfsunSJ7FirmzRJO4rPnKnFjsrKdFP30kvdGxNRIzFhkhqmdlbA6acDDz6oM9WUFJ1Jd+2q7b3clJmpNa937tTDRp061b0kQhTFWJSJ7OH1akW+5s3ZZZuoEepKz+OMmuyRlQWcdJLbo4gdfr9moSQnuz0SigHcTCSKpB07tEBVerreBg4EEvlwGIWEM2qiSCkvB/r21cBcWRTp00+1ZsnGjWzVRUFxRk0UKbNn6+Zm9cp1FRVaE/rNN90bF0U9BmpKTLt3A3//u/aNzMvTLitOl/dct876cJDXq1UKiYLg0gclnv37gd69talBZdGoq67Sk5NPPOHc43bvruvStUvhZmdriVyiIDijpsTzwgtaD6R6ZT+vF5gyxdkGDwMGAB076gGcSikpwEEHAcOHO/e4FPMYqCnxfPJJzZOUldLSdFbtlKQkrXvypz/p4RuPRzu5LF0avKZKrCkr059x3jzrJhrUKFz6oMTToYPmL9fuDej3ayduJzVpogWqnnvO2cdxw9y5wIgRVc9rcjLw1ltaK4bCwhk1JZ5rrgms4JecDLRvDxx3nDtjinW7dwNDhgB792o9mH379N9DhtjX0DiBMVBT4jnySOA//9EyrVlZuuzQp48uiRjj9uhi05tvWmfNiGjNFQoLlz4oMQ0aBGzdqo0NcnKcX/KId3v3WjeY8Pn0cxQWzqgpcSUnaylWBunw9etn3RAiMxPo3z/y44kzDNQUPhFtGjthgjZLLS52e0QUaccfr+vRlXXKAf334MH6OQoLlz4oPGVlwLnnVqVjpafrZt28eZHr6ELuMwaYOhV4+23gxRf1xfuyyzQLhOv+YeOMmsLz7LPA55/rgZHSUj31t2eP/oGSvUpKtDFwu3ba+/GKK7R2SLRISgJGjtS+mB99pDniSQwxduCMmsIzeXLg4RERrRC3cSNw2GHujCseVb5zqVxaevll4OOPtU5I9SUHijv1vtwZYzKMMUuNMV8bY1YbY+6OxMAoRtQ+NFLJmJpV4ig8K1fqib/q6/9lZZqjPHWqa8OiyAjlfUkJgDNEpAeAYwEMNMb0cXRUFDv+9Cfd2a+tRQutTEf2+Oor67Ver1c7xVNcqzdQiyo88GHqgZvD9SApZlx7LXDMMVoBDtCgnZOjhxy4iWSfww6zfj4zMvQAD8W1kNaojTHJAJYD6ARgkoh86eioqPE2btTqcDt36qGOc85xti9fZiawcKFuHs2fDxxyCHDhhUCzZs49ZiI65RSgbVvghx9qLimlpgJjxrg3LoqIBnUhN8Y0ATADwN9EZFWtz40FMBYA2rdv33sz+8BF3owZwOjRunZZVqYbTL16abun6qU1Y8xXXwGrV+vEMS8vgSfqO3YAl14KfPaZfnzkkbqh2LOnq8OKiCVLgNde0xepP/4ROO20uPtFqKsLeYMC9YGL3QnAKyIPB/uavLw8yc/Pb9goKTw+n9au2L+/5v0eD/D448Dll7szrjB4vfqmID9fs7xEgKOP1kSHnJyGXWv5cr116KCH6GK6+XdlKmTTpm6PJDLuuAN49NGqjVSPB7joIk0NbSCR6I3vdQXqULI+WhyYScMYkwmgP4C1to6Qwrd0qfVvYFFRzGYF3HKLHngsKtK2gl4vsGIFcN11oV+jpAQ46yzg1FOB66/X9O4jjtAyHzErKytxgvSGDcDDD+svgYjevF6dXS9dGtIl/H7g/vuB5s31Bbp7d63IGktCyfpoA+C/xphvACwD8ImIfODssKjB0tP1N9JKjHa3fuWVwDo/JSXAtGmhtzd84AHgiy/0b7uoSN9wbN4MXHyxPWP0+3XFacQIYNQoYM4c51svJpRZs6zvLy4GZs4M6RLjxgH33quZjCK6jDZ4cMhxPirUu5koIt8ASIBFsBh33HFAbq5OPavLygL+8hd3xhQmq2JsgC6/h/oWdsqUwNIjFRXAggVaMjk3t/HjEwEuuEBjider973/vq4yPfZY469L1WRmWp9uTE0NaQLi9QJPPhn4O1BcDNx5p+6BxwKe74wXSUnABx9o/72cnKo6y2PGaOZHDOrfP/Bv1BhNgAj1ZHL1toi1hdspav78mkEa0H9PnqwNx+3y0UdA3756cnzECGDNGvuuHfWGDbO+PzlZ38LU45dfgu9HrF4dxrgijIE6nvTsqYuvr7yiG4irVgETJ0bv7kk9nnhCl2Irz9NkZgK/+x3wzDOhX+O883TyVVuXLuFnENYO0pVEdAnEDi++qMF50SINOjNmaDG6776z5/pRr3lzXevyeHQCkp2tE5CnntJGwfVo29b68KwxujEdK1jrI96kp2tNiDhw2GHA+vWaFp6fr69Df/6z/u2G6p//BGbP1rRyr1eDfWqqvpaFq0kTzXqsPWtPSdEXlHCVlwM33VSzlIrfrz/H+PHAO++E/xgxYehQYNs2/R9ZXg4MHKjvHEOQlaVnsp54oubzmJkJ3HWXM8N1QoPT80LB9LwY4vdrLu6kSRoBRo7U6BDO4m2UKS7Wg5KLFwOdOmkqcosW4V/3p580lbn2+md2ts5+w30Kf/5Zr2/VML1VK2D79vCunyj8fk0cefhhLezYvbvuIZx2mtsjq8nWPOpQMFDHkDFjtN9d5Xv4jAxNNl6xQv9NdXr7beCSS6rWQY3R5YnTTw//2l6vvnvw+QI/17u3vsug+BFWHjXFsQ0bgNdfr7nQ6vPpVO6NN9wbVwwZPlwPDE6bpv1yd+ywJ0gD+rZ99OjAmlcejy59UOJgoE5kS5bogmptXq925KaQZGUBZ58NnHmmddvAcDz5pKYAZmTo4+TmAg8+GDfbEBQibiYmsjZtrDNC0tJ0+YNcl56um6mPPw4UFGiKXgyXbaFGYqBOZKedprvnXm/NU40pKdrmiaJGTk7D65tQZBUV6cphiAkpDcKlj0SWnKz9Dnv00IXQrCxNJ3jvPc6oiUK0Z48uRTVtqm9Su3bVDCM7cUadiDZuBN56S08CDBumdUQ3b9YpQZcubEhKFCIR3Zv45puqk65r1+p9q1YBhx5qz+PwLzLRTJoEHHWUpg384x9ar/ree/U3qmtXBmmiBvjqKw3MtcsRlJYCTz9t3+PwrzKR/PSTHmbx+fQ3q7xcT2vcd19sFT4gihIbN1rXEikttbcmCwN1IglWFrKsTE9uUFz64gst6zp0qJZxDrcYFVXp2dO68FdmJnDyyfY9DgN1IjEmeIGmGC3cRHW7/34tjTF1qr5OX3klMGBAzbaL1HidOgFDhtSsuJqcrBk6djZVYqBOJEOHWle1T03VEm0UV3bsAO65p6o5CqCZmPn5esyd7DF1qm73tG+vFRkvukjXru1swsNAnUjatdOTExkZepIiPV3/feedupFIjbZvnxb9GTBAK/ytXOn2iDTz0qrEq9ebQJX3IiAlRdvGbd4M7NqlNc7atrX5Mey9HEW9sWOBP/xB16QrKjQB9PDD3R5VTNuzR5Nndu7UvdmkJC2V8tJLWozQLbm51itaycnh1+KmyGL1PKIw3X478Mgjga3DmjbV5QerWW0klJUBBx+ss7zqPB5tRNCjhzvjImusnkfkoHffte7vWF7ubieW1FTtNNOypW5u5eZqNsKECQzSsabepQ9jzCEAXgHQGoAfwGQRedzpgRHFimCbRmVl2gXGTb16aRODyk7sv/+9Pd1nKLJCmVGXA7hRRLoC6APgamNMN2eHRRRceblu3HXqpG/tr7pKK8u55f/+T8ukVJecDBxzjH1HiMORkqI1sgcPZpCOVfXOqEVkG4BtB/693xizBkBbAInSXpOizAUXaGfuyhZVU6YAH36ohyuzsyM/nuHDNR3r0Uc1kaaiQvuuMgWO7NKgzURjTAcA8wF0F5F9tT43FsBYAGjfvn3vzZs32zhMSgSFhVq477ffgP79gSOOCPyaNWu0DVXtPoUej86y//rXyIzVys6dmqPcurWeWHPjDNHXXwPPPqvvMIYOBc4/n/WrY4UtPRONMdkA5gH4l4jUmYXJrA9qqAULgEGD9GBGRYXe95e/6Cy1esB77TUNxoWFgdf44x8Tu4PYyy/rc1Naqs9hVpbW35o/357OM+Xlmi/epAlrdzkh7KwPY0wqgLcBTK0vSBM1VFmZHsPdv18DcHGx3p57TrMWNm0CJk7Uwn+114Irpadbz8AThdera/XFxVUvdF6vltp8+eXwru33A3ffrQXx27TRLJJnnw1/zLFuxQrgb3/Trvbvv1+z94btRKTOGwADzfp4rL6vrbz17t1biEL16aciubkiOp+ueTv6aJGMDL1lZoqkp4u0aSOSklLz67KzRX7+2e2fxD2ffBL8OezXL7xr33uviMdT85oej8i0afaMPRZNmKDPQVKSPh9ZWSJDhohUVDT+mgDyJUhMDWVG3RfAxQDOMMasPHAb5NxLByWauqq5rV6tVVl9Pp0tlpToScATT9S11/R0oHNn4OOP9YR8osrODj6jCyfTw+8HHnqoauO2UlERcNddjb9uLNu5E7j1Vn0OKp9zrxeYO1c3tZ0QStbHAuismsgRp5xS9Xa9utRU6/uTk7XwzcyZGsBbtYr+4n+LFml2iterG3znnmtdx7ixjj9e145rr91nZYW3wVpcrGO2smVL468byz77TH83fb6a9xcWauOkc86x/zG5JUCu83h0HTUzsypDITtb2zZaBeDKDccmTTTDItqD9H33abGml14C3nwTuOQSXZO3c00zKQmYPVtftCob4WZkAH//u2bQNJbHA7RoYf25o45q/HVjWfWSptUlJ+vpTyew1gdFjR9/BF59Fdi7VzNAmjUD+vYNTMXLyAC+/z42ljq2bgUOOyzwiHl2NvD663oIxU7l5Vo1b88efafSunX415w6VWt5VV/+yMwEZs3SRvaJprhYN1V/+63m/R6PngDt1atx160r6yOkzcGG3riZSHa57TbdRExOFklN1X9PnBiZxy4rEykvD+8ar7yiG51Wm3yXXWbPOEPl94u8/LJI164iBx0kcvbZIt9+G9r3vveeyDHH6IbliSeKzJvn7Fij3fz5+lzk5ork5Ogm96OPhndN1LGZyDKnFNX+9S89iThjhh6FHjlSNw+d9OOPOoOcO1eXVQYPBp55RpcVGionxzrnODk58nVA/v1v7WNcOTOeNQuYNw9Ytgw48si6v3fIEL2R+v3vge3bdRPb69XlpZYtnXs8Ln0QVeP16lLFrl1Va8ipqdq9Y926hm8A+ny6/FD7bXJmJvDll8DRR4c/5n37tOXW9Om6xn/FFcC119Ysr1pcDDRvHpi9kZSkL4RTp4Y/DgoPy5wShWj6dA3W1Tf6yso0JWv27IZfLyNDZ65NmuhGU+Um32OP2ROkS0s1VXHCBD0YtG4dcMcdwLBhNb9u0ybrFxm/H1iyJPxxkLO49EEJq6JCZ85NmlQdsV6zxjodraREg+DZZzf8cU46SRsIzJ2rM9ozzrBv2eOdd4Cffqq5WVlcDPz3v8Dy5VoXBdDNL6tu2QAb/MQCzqgpIb3wgi5JdOigR6NvvFEzJo491roCX1paeDPgtDTtBn7eefauTS9YYF33RARYurTq46ZNdYkjM7Pm13k8wPjx9o2HnMFATQln5kyt0bBrl64hFxXpZuEtt2jJ0mbNdOOyUlqarlv36+femIPp2DEw+AI6/trpi88+qznclb2N27TR/PVTTonMWKnxuJlIjhHRokA+n5b9TImShbZevbSgTm0ej+Yf//orcMMNWnI1KQkYNQp48MHoLLpfUKBLF/v3V92XlKRB+McfrZ9zn0+/vlkzVsGLJnVtJkbJnw7Fm1WrtB7yjh0aDFJTNbNg4EC3R6Zrulb8fj1s07p17GRBNG2qyzg336zpYn6/dpaZPj34C2NGht7C8fPP2iyhfXtdLor206Gxjq+nZLuSEm39tHGjbszt368z1eHDgwfJSAp2cqyu49LRaPp0ze2+9FI9AXnCCcDKlbo23bGjM4/p9wOXX64lZf/0J80nPu64wE7nZC8GarLdRx8F78r94ouRH09t998fWK/B49H77SyU5KQlS4AxY/QF0OvV53vRIuc73Dz9tB599/k0f9vrBb75Brj4YmcfN9ExUJPtCgqsq96VlgLbtkV+PLX17q0n8gYM0IyPHj20c8zYsW6PLHSPPBJYA6W0VA/RbNrk3OM++WTgoZmyMk09/PVX5x430XGNmmx3yim6kVhbdjZw5pmRH4+VvDw9/hurNm2yfo7T0oBffnFu6WPfPuv7k5J0dh3pY/GJgjNqsl2XLpopUb1tlsejZTFZL8Ie/fpZN60tKbHnxGMwgwdbb1K2agUcfLBzj5voGKjJEVOm6O2UU3Sz6d//1vKb0ZKiF+tuuEGPpFd/Pj0eYNw4Z9MI77pLa4ZU5m6npOjjvvACMz+cxDxqohj1yy/AP/+pDYBbtNAmASNHOv+4e/cCkyfrC2/nznp4yOmKhomgrjxqBmoioijA6nlERDGs3kBtjHnBGLPTGLMqEgMiiieFhcDChdo6jKixQplRvwQgCg7+EjlLBJg/X49j33uv1soIx4QJ2vVj0CDN1T7hBK1rTdRQ9QZqEZkPYE8ExkLkGr8fuPBCDaoPPQTccw/QrZuewmuMOXO0fGhxseYeFxdrbYxzz7V12JQgbFujNsaMNcbkG2PyCwoK7LosUUR8+CHw/vtVTQPKyjS4Xn55zcp0oXr00cATfOXlWovDyZODFJ9sC9QiMllE8kQkr0UsVbYhAjBtmnVnl5QUPR7dUNu3W9+fmgrs3t3w61FiY9YHEWo2gq2tMYd0Bg2qau9Vnd+vJzSJGoKBmgjAZZdZd0oRaVxnlxtv1ML81YO1x6PFlKweJ5H4fMC33wZ/10GBQknPex3AYgBdjDFbjDF/dn5YRJEjoqVZqzd/NUYD6owZjSuy37y5lv+8+WYtADV0qD5GLFXoc8Izz2gmTN++2q/y7LOB336r+3t8PuCNN4AHHgA+/bRmh/hEwZOJlPDefRcYPbrmGrUxeix67drErmGxcSPw1FP6PJx6qm6uNm3auGvNmaPNfatvsqan6zuWDz+0/p4fftAu7sXFesvI0KWjuXMDa4rHOp5MJKrDk08GbiSKaC2NNWvcGVM0+OILbes1caIG0jvvBLp21W4yjfHAA4GZMCUlwGefBV8GGT1au8fs369ZM4WFwNdfa5OHRMJATQkv2Fvv5GQNDIlIRNftvV5NVQR0Rrt7N3DHHY27ZrAAn5ZmfRBo927NPa+91OHzaff0RMJATQlv5Mjg69DHHhvRoUSNggJgy5bA+8vLgQ8+aNw1zzjDOrtGRHswWt0fTKKtUzNQU8K76irg8MOrGh0kJ+tG4pQp1sX5E0FmZvBAmZ3duGvedhuQkxNYQ/uBB6xfKJs3B7p3D9wjSE8HLrqocWOIVQzUlPCys4Fly4DHHweGDdMGscuWRaa2c7TKydGekrVnwB4PcM01jbtmu3a6vnzFFTqDPv104J139IUymNde083LyhfR7GztIDR+fOPGEKuY9UFElnbvBs46SzM+kpM1fXHECOCllyLbrb2wEJg+XYtkHXecpvTFSrf4hqgr64ONkYjIUrNm+s5ixQoNkj17Otc0ty7Z2cCfE/z0BgM1EQVlDNCrl97IPVyjJiKKcgzUlBAqKjS1jCgWMVBTXNu9Gzj/fE03y8jQTAO2xaJYwzVqilt+v9anWL++6nTdvHlAnz5aQ6JJE1eHRxQyzqgpbn3+ObB5c1WQBvQQh88HvPqqa8MiajAGaopb33+va9O1FRUBq1ZFfjzRZM0arVqXkqLpb9dcE1gwiaIHlz4obnXvDiRZTEWysoDeve1/vI0btYhRt27RfSBj+3bgxBO16a6Ijvn554F164BPPnF7dGSFM2qKWyedBBx9dM0uK8nJQG6udhy3y6ZNQI8e+sJw0klAmzbaJCBaPf20Lv9UP5Ts8wELFwKrV7s3LgqOgZriljE6Q7ziCt049HiA4cP1tF1jCwvV5vdrJsmqVVoGtLBQK8+NGKEbltHoq6+0DnRtqamJXX87mjFQU1zLzgaeeALYu1ff4k+fDrRta9/1P/8c2LMnsOxmWRnw7LP2PY6devWyrlZXVqaNASj6MFAThSFYZ5KyMs04iUZ//asuB1UvH5qRoX0M2SE9OoUUqI0xA40x64wxG4wx45weFFGsOPHEmul/lbKytPJcNGrdGli8WAv5JyfrWC+/HHjvPbdHRsHUm/VhjEkGMAnAAABbACwzxswUke+cHhxRtOvYEbjkEq2bXNl3MSMDaN8eGDXK3bHVpWtX7ehNsSGUGfXxADaIyEYRKQXwBoChzg6LKHY8/TTwzDPACSfo0sFttwFffqnH1onsEEoedVsAP1f7eAuAE5wZDlHsMUa7ZY8e7fZIKF6FMqM2FvcFtIUxxow1xuQbY/ILCgrCHxkREQEILVBvAXBItY/bAQho/C4ik0UkT0TyWrRoYdf4iMgBJSW6PPPhh9q9haJbKIF6GYDOxpiOxpg0ABcAmOnssIjICRUVwA036OGfPn2AwYO10ezw4daHYCg61BuoRaQcwDUA5gBYA+BNEeFBU6IYdMcdegCo+gGdsjLg/fcTr7N3LGEXcqIEUV6uR+kr0whr+93vgF9/jeSIqLq6upDzZCJRgti/HygtDf75wsLIjYUahoGaKEE0aQI0axb88yefHLGhUAMxUBMlCGOAhx6yLsjk8QATJ0Z+TBQaBmqiBDJ6NPDWW1o7OyNDZ9mjRml502OOcXt0FAwDNVGCOfts4NtvtX723r3AtGlam4QaZvt27XCfkaHvSEaPBnbtcuax2IqLiKiBSko0D/2XXzSbBgDefFObUnz3nf2t2DijJiJqoHfeAXbvrgrSgOajb9vmTBs2BmoiogZavdo6nbG42Jm+kwzUREQN1LWrdd/NzExn2pkxUBMRtm4F7rsPuPJK7Stp1bWGqgwfrhkz1deiU1KAli2BQYPsfzwGaqIEN3++Fma65x5tyHv55cBxxwU/ak6a6bFkiRa1SkkB0tKAYcOARYv0Y7sx64Mogfn9wIUX1gzKhYXAunXAhAks1FSXtm2Bd98FKsslGavK/TbhjJooga1fb12IyefT/GqqnzHOBmmAgZoooaWn1yx5Wp3VUXNyBwM1UQLr2BE4/PDAGaHHA/zlL+6MiQIxUBMluLffBlq1AnJyNEBnZuom2eWXuz0yqsTNRKIEd8QRwE8/AbNn68m6k07Sok0UPRioiQipqcA557g9CgqGSx9ERFGOgZqIKMoxUBMRRTkGaiKiKMdATUQU5YxUHlS386LGFADYbPuFndccgEPNdFzHny028WeLPY39uQ4VkRZWn3AkUMcqY0y+iOS5PQ4n8GeLTfzZYo8TPxeXPoiIohwDNRFRlGOgrmmy2wNwEH+22MSfLfbY/nNxjZqIKMpxRk1EFOUYqImIohwDdS3GmJHGmNXGGL8xJi5Sh4wxA40x64wxG4wx49wej12MMS8YY3YaY1a5PRY7GWMOMcb81xiz5sDv4nVuj8kuxpgMY8xSY8zXB362u90ek92MMcnGmBXGmA/suiYDdaBVAM4DMN/tgdjBGJMMYBKAPwDoBmCUMaabu6OyzUsABro9CAeUA7hRRLoC6APg6jj6f1YC4AwR6QHgWAADjTF93B2S7a4DsMbOCzJQ1yIia0RkndvjsNHxADaIyEYRKQXwBoChLo/JFiIyH8Aet8dhNxHZJiJfHfj3fugffVt3R2UPUYUHPkw9cIubjAZjTDsAZwOYYud1GajjX1sAP1f7eAvi5I8+ERhjOgDoCeBLl4dimwNLAysB7ATwiYjEzc8G4DEANwMI0jK4cRIyUBtjPjXGrLK4xcVMsxarRvZxM4OJZ8aYbABvA/g/Ednn9njsIiIVInIsgHYAjjfGxEXjL2PMYAA7RWS53ddOyFZcItLf7TFE0BYAh1T7uB2ArS6NhUJkjEmFBumpIvKO2+Nxgoj8aoz5HLrPEA8bwn0BDDHGDAKQASDXGPOaiIwO98IJOaNOMMsAdDbGdDTGpAG4AMBMl8dEdTDGGADPA1gjIo+6PR47GWNaGGOaHPh3JoD+ANa6OiibiMitItJORDpA/87m2hGkAQbqAMaYYcaYLQBOBPChMWaO22MKh4iUA7gGwBzoptSbIrLa3VHZwxjzOoDFALoYY7YYY/7s9phs0hfAxQDOMMasPHAb5PagbNIGwH+NMd9AJxGfiIhtaWzxikfIiYiiHGfURERRjoGaiCjKMVATEUU5BmoioijHQE1EFOUYqImIohwDNRFRlPt/0v6rAk1VJhEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.datasets import make_blobs\n", "from matplotlib.colors import ListedColormap\n", "cm = plt.cm.RdBu\n", "cm_bright = ListedColormap(['#FF0000', '#0000FF'])\n", "\n", "X, y = make_blobs(n_samples=70, centers=2, n_features=2, random_state=0)\n", "\n", "plt.scatter(X[:,0], X[:,1], c = y, cmap = cm_bright)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 5. Multiple labels \n", "\n", "The idea above can be extended to a multiclass classification problem by introducing a vector of targets. For a three classes problem like below, we can associate to each point a vector of size $3$ such that $t_k = 1$ if $x^{(i)}$ belongs to class $\\mathcal{C}_k$ and $0$ otherwise. A training point in class $\\mathcal{C}_1$ is thus given the target $(1,0,0)$. The LinearRegression model can be used with $d$ dimensional targets and we can thus use it to learn our multiclass classifier. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA5YElEQVR4nO3dd3xV9f3H8dfn7nuzIAxB9hAVEAe4FXGgKCou1Dpqa9Vql6v150arLVWrrbbWat3VuhcuRAVBFIEgewmydwiQeff9/v64YSS5gUBO7rk3+TwfDx/CSXLOOyH55Nzv+X4/XzHGoJRSKns57A6glFKqcbSQK6VUltNCrpRSWU4LuVJKZTkt5EopleVcdly0bdu2pnv37nZcWimlstaMGTM2G2Pa1T5uSyHv3r07RUVFdlxaKaWyloisTHVch1aUUirLaSFXSqksp4VcKaWynBZypZTKclrIleXWLFnP3K8XUllaaXcUpVoEW2atqOapdHMZ9454iKWzVuD2uIiGo1x+94VcdueFdkdTqlnTO3JlmQcufozFRT8SCUaoLK0iEory2uj3+PaD6XZHU6pZ00KuLLF5bQkLv/uBeDRe43ioMsxbj46xKZVSLYMWcmWJ8i0VON3OlG/bVlyW5jRKtSxayJUlOh+4Pw5H3W8nl8fF0WcdYUMipVoOLeTKEm6Pm9/842q8AQ8i1ce8bvIKc7nk/86zNZtSzZ3OWlGWOe2Kk9i/VwfefuwjNq3azMChA7jgpuEUtM23O5pSzZolhVxEWgHPAv0BA1xtjJlixblVdul77IHc+9aBdsdQqkWx6o78cWCsMeYiEfEAAYvOq5RSag8aXchFJB8YDPwMwBgTASKNPa9SSqmGseJhZ0+gGHhBRGaKyLMiklP7nUTkOhEpEpGi4uJiCy6rlFIKrCnkLuAI4CljzOFAJXB77XcyxjxjjBlkjBnUrl2dDS6UUkrtIysK+RpgjTFmavXf3yZZ2JVSSqVBo8fIjTEbRGS1iBxojFkMnAosaHw0pdR2wcoQ0z6ZSbgqzBFDB9B2/0K7I6XVsjkreeWBt1jy/XK6HtSJy+++kL7H6uyo7ayatfJb4NXqGSvLgJ9bdF6lWrzZE+dzz7l/QRASJkE8muDKUSP5ye3n2x0tLRZPX8qtJ99HJBTBJAwblm9i9lfzGfXO7zly2OF2x8sIlqzsNMbMqh7/HmCMOc8Ys9WK8yrV0oWDYe4d8RDB8hBV5UFCFWGi4SivPvg2i6YtsTteWvz71pcIV4UxCbPjWDgY4Z+/e97GVJlFl+grlcFmfD4n5fFIKMrY5yekOY09lsxYlvL4hmUbiYR0pjNoIVcqo0VD0eRa6VpMwhCuCqc/kA3y2+alPO4NeHF5tMsIaCFXKqMdfuohxKKxOsd9uT4GjzzWhkTpd/EfRuALeGsc8wY8nPurM1J23GyJ9KugVAbLb5PHDX/7GV6/B4cz+ePqy/Ux8LRDOHp4y5jlO+LXw7jgpuF4/R78eT48PjdDrzyJnz/4E7ujZQwxJsXrtiY2aNAgU1RUlPbrKpWtls9bxbiXviJYHuT4845i4OmHZu3d6NKZy1kxfzVdDupEn4E9ke19j/cgWBli08pi2nYqJKegzuLxFkFEZhhjBtU5roVcKZUOwcoQd589msXTf8ThEIwx9BzQjdFj7yaQ57c7Xlaor5Bn5690pVTWee6OV1n43RLCVWGCFSFClWGWfL+Mf930gt3Rsp4WcqVUWox76Sui4WiNY9FwjPH/m4wdIwPNiRZypVRaRMN1Z98AxKIxLeSNpIVcKZUWR5x2COKo+WBTRBhwUt+sfXCbKfSrp5RKi18/fjW5rXLw+D0AePwecgoC/O7Ja21Olv10WZRSKi3279WBF394grHPjeeHGcvodWg3zrzmVFq1K7A7WtbTQq6USpv8wjwu/sMIu2M0Ozq0opRSWU4LuVJKZTkt5EopleV0jFxlvMqyKia+OYWStVs46OjeWd1nRKmmoIVcZbQfZ6/g1iGjiMfihCrD+HN9dOvXhb+OH4XX793zCZRqAfS2RmW0P/3kb1SWVhGqTG6iEKwIsWzOCt7520c2J1Mqc1hSyEVkhYjMFZFZIqJtDdMgGokSqdW3ornZtKqYjSs31zkeCUYZ99JX6Q+kVIaycmjlZGNM3Z86ZanNa0t47Jp/8/2XczHGcMiJB3PrczfQscd+dkezngjU04NDRF9MKrWd/jRkkVg0xo3H382ML+YQj8VJxBPMnbSA3x17F6FmuH9j+y5t6dirA7X3HfD6PZzx85PtCaVUBrKqkBtgnIjMEJHrUr2DiFwnIkUiUlRcXGzRZRsYLrEVE/wIE/oMk6hK67Wt9N1HMyjfWkEinthxLJEwhKrCTHzzWxuTNZ27X7+Z3Na5+HN9OJwOfLk++gzqxQU3Dbc7mlIZw6qhleONMetEpD3wuYgsMsZM2vUdjDHPAM9Acocgi667R4mqN6DsQRAXIEACWj2JeI/f48caE4XwJIivBXdfcA9s8LZUTWHd0g3JXdVrCVWEWLNkvQ2Jml73fl3436p/8/U731VPPzyAQ4f0s/XfQalMY0khN8asq/7/JhF5DzgKmLT7j2p6JvYjlP0JCIPZOfRgtv0K2n2DOHLr/9j4OkzJpWDKwUSTvwhcfaHwBUTsmfbWY0A33F43sWi8xnF/ro/eh3a3JVM6+AJehl55kt0xlMpYjR5aEZEcEcnb/mfgdGBeY89rBRP8AEjVzF4gPH73H7vtD5DYBKYSiICpguhcTMVT+5bFGExoPIktV5PYPJJExXN7PcwzcOgAOvbcD5dn5+9fl9tJ6/0KOO68I/cpl1Iq+1kxRr4fMFlEZgPTgI+NMWMtOG/jmSognuK4AROs/8MS5RCdBSRqvSUMwXf3LUrF3zDbbobIZIjNhorHMVsuwZiGP6R0OBw8NumPnHXNqeS2ziGnIMBpVw7mH9+Nxu1x71MuZZ+yknJmjp/L6sVr7Y6ispzYscXSoEGDTFFR0083N+HvMNuury7ou/Ii7cYhzo6pPy6xDbPpeCDFPG1HWxzt9+7BoolvwhSfAkRqvcWP5N+NBEbu1fnU3jHGUDRuNpPenoLX72HoT4dw4KBetuZ5/s7/8e7jH1cPlcXodWh3HhhzO/lt8ur9uNWL1/JD0TLadWnDIScerM8JWiARmWGMGVT7ePNeou85GrynJIdRTBXJFyAeyL2+3iIOII5WGFdviC2s9RY3+M7a+xzRmSAeMLULeRATnqCFvAkZYxh9xeNMGVNEqDKMwyGMfX48V947kktuO8+WTOP/N5n3//kpkVCUSPXD6x9m/Mjoyx9n9Ni767x/PB7nL1c8wbcfTMfpdoKBNp0K+ev4+2jTsfUer1e6uYxEwtC6vW7g0Fw163nkIoIUPIq0+gf4LwT/pUjhf3Hk/mrPH1vwCEg+4K8+EgBnZyT3t3sfxNGa5AzNOm8ARzNcyJNBZo6ft6OIQ3K6Zrgqwsv3vcnmtSW2ZHr7bx/uyLNdLBJn9sT5bCsurfP+H/xzLFM+LCISihIsDxGsCLH+xw38+bK/7/Y6637cwG+OuYNLO/+Sy7tdz7UDbmX53JVWfioqQzTvO3KSxRzviYj3xL37OHcfaDceExwD8VWIewD4TkfEs/ch3INA8qpfFexa0D1I4Cd7fz7VYJPfm1qnaAI4nA6KPpvNsKtPSXum8i0VKY87XU4qS6vqbH324VOfEa6q+WouHkuwcMoPlJWUpxyOiYSj3HTCPWwrLsUkkt9zK+at4paTRvHK8ifJKcix6LNRmaBZ35E3ljjyceRcgSP/TsR/9r4VcZLLyaXwZXB2A/GD5ILkQMHo5C8M1WT8OcmFRLWJQ3ZsApxuRw47DKfLWee41++lQ4/2dY6n+kUEyc8hHKw9XJf03YdFhKpCO4r4drFIjAmvN8/FYy2ZFvI0EVd3pO1nSOGbSOtnkfZTcfh1dWJTG/rTk2pM19zOJAzHnD3QhkRwxT0jySvMxe1NzjQSh+ANeLjx39fhdNYt8CdccHTKz6HN/oW07VSY8hobV24mmqKpWqgqzIblGxv5GahMo4U8jUQEcR+IeI7Y57t7tXe69+vCLx+5ErfXjT/XRyDfjy/Hx/3v3UYgz7/nE+yl0s1lrFy4hmik/s6UbTq25tl5j3HxbSPoe2wfTvnJCfz96wc58YKjU77/FfdcRGHHVvgCyYVobo8LX46P2178db0zVw48sheuFFNS/bk+Djr6gH34zFQma9bTD5XabltxKTPGzcHjczNo2GH4c3yWnr+qPMhDP/0H08fOxOV2IQ7hukeuZPi1Qy05f7AiyBf/ncScSQvYv1cHhl93Gu27tqv3/Y0x3DpkFIunL90xM8btddG5z/48NePhlEM7tUXCUVbOX01B27zdXkulT33TD7WQK7UbxiQgMgViP4CzO3gHI1K3CN474iGKxs0iGt65ktgb8HLfu39g0OmHpjHxTpFQhNcfep9xL35FIp7glMtO4LK7LmzQK5GxL4znqZteBEmOq/c5sjej3r61zoNYlV5ayJXaSyZRjtlyGcTXJNcAiAccbZDC1xFn2x3vt3VTKZd3uyHlmPRhJ/fjkS/vS2Pqxps3eSG3D3uwxkwZl9vJAQN78cS3f7IxmaqvkOsYuVL1MOUPQ2xZdb+daPL/8XWYslE13m/bptKUDyMBNq22Z656Y7z16Id1pjvGonGWzV7B2qXNs8tmttNCrlR9Qh9Rt01DDMITMGZnD59OvTuk3MnI6XJw+Cn9mzZjE9i8JvUvH6fHxdYN29IbRjWIFvIWyiS2YGKrk2PAqh71fW0Muy7s8vg8/GL0ZXgDO9sbO10O/Hl+LrvzgqaN2ASOHHbYjqmRu4pHYvRsxu2Ss5kW8hbGxDeTKLkSs2kwZvNwTPFgTHiy3bEyk/c06i5+doDnGERqHh/x6zO5581bOOTEg+nYcz9O/9nJ/Pv7R7Jytsf5Nw4nrzC3xnCRL8fLlfdd3CRTNlXj6cPOFsQYgyk5F2I/UrNPux9p+x7i6mlXtIxk4psxJReCKa1urxAAhx8pfAtxdbY7XpPaVlzKm4+MYdon39N6vwIuuvVcjj7rCLtjtXg6a0VhonMxJVcCtdv6OsF/GY6Ce+yIldGMCUPoM0x0IeLqBb6zEEfA7lgZobKsii9emcTS75fTc0BXhv50CLmttIdLU2qZbWxVTfGNII4UjRjjEF9lR6KMJ+IF/7mI/9y0XTNYGWL8q1+zaPpSuh7UiTN+dvJu+5TbYdOqYn591B0EK0KEq8J4Ax5eeeBtnpjyZzr1rr9F9L4wxlC+tQJfwIvHpyuiU9FC3pK4+6XoiQ7gA88xTXLJH2b8yAt3v8bS75ezX/f2XDlqpL5E340tG7by66Nup2JrJaHKMF6/h1cffIe/T36Q7v262B1vhydvfIGy6j7nAOGqCJFQlCd+9SwPjbPuld2Mz2fz918+w+Z1WxARTrrkOH735DWWr8zNdvqwswURZ0fwX8DOHusAbnAUIIGLLb/e4ulLueWkURR9NpttxWUsnr6UBy5+lC9etX1f7oz17O2vsnVD6Y6Oh+FghKqyKv569b9sTlbT9LEzdxTx7UzCMGvCPBIJa2ZC/Th7BaPOf5gNKzYRi8SIhqNMfPNb/vSTv1ty/uZEC3kLI/n3Qf5d4OoDjv0hcCnS5n3EYf1L9//c/grhqpotWMNVEZ75/cuW/bBnAmMMi6Yt4fOXJ/LDjB8bda4pY4qIx2ruM2sMLJ25jGBlqFHntpLLnfrFvMPpsGwLujcfGUM0VHMefzQUZeYXc9i0qtiSazQXlg2tSLIBRRGw1hhztlXnVdYScSTvvpvgDry2pd8vT3m8fGslFVsrM27cd19UllVx+xkPsmLeKhABY+h9RA/+/Mld+/Ty3+Wup5mVCM4UfdXtcurlJ/LZixNq9JZxeVwMHnmsZYV89eK1de76AdxeN5tWbc7KqZ1NxcrvjBuB2ptcqhasvl7ZTpeTQH7zmI/81E0v8OOs5YQqw4QqQoQqwyye9iPP3v7KPp3vjJ+fjMdXczGO0+3kyDMOy6gHfdc+fCU9B3THl+PDG/Dgz/XRrW9nfvPE1ZZdo99xB6b8xRYNR+lyUCfLrtMcWFLIRaQzMBx41orzNZZJbMVEF2EStafZqXS64p6Laqx2BPAGPIz49Rn1vjTPJsYYxr/2TY27UkgWms9fnrhP57xy1EgOPqYPvhxvskDm+di/VwduefYGKyJbJpDn5x/f/ZnRY+/iV3/7OQ9+dAdPzXiYvNa5ll1j5O/PxRvwIo6dd/jegJfh1w2loG2+ZddpDiyZRy4ibwOjgTzg96mGVkTkOuA6gK5duw5cudL6TWCNiWBK74TQ2Opd62OQey2S8xvLXu6pvfPBk5/ywt2vE4vGwMDZvxzKtQ9f2aB+2JkmkUgw7ZOZTP14BrmFuZx+1RB+0femOtupQXKY4dPQa/Wey0QXQ/ir5NZ/vmGIc+cWb8YYFk9fyrI5q+jYsz2HDumHw5E5wyrptGbJep69/RVmfzWfvNY5XHjLOZx7wxkt9ue5yRYEicjZwFnGmF+JyBDqKeS7aqoFQYnSeyH4PrDrQyE/5N+LI3Ch5ddraUx8E6bi8eoClAOBK5HA5YjsvsjEY3G2btxGfpu8jBoe2B0T31z9eTrAezKJRD53nT2a+d8uJlQRwuly4nI72a9HO1YvXMeuP0fiEI4683Ae/PCOuuc1BlP+Z6h6g+Tq2upfaAWP4PAPS8vnprJXUy4IOh44V0TOAnxAvoi8Yoy5woJzN5gxEQi+B9TeqDYIlU+DFvJGMYlSTMl5kNhGsgAVQ/kjmNgCpGD0bj/W6XLStlObNKS0RqLqDSh7EMRZvXhqFBM+vZb53yzaMS0wHosTj8XZsLyYQIGfaDhGJBjBG/Dg9Xv49eP1jBVHi6DqTXbebFQPy5TehvGegDisG5pQLUejC7kx5g7gDoBd7sjTWsSTQSqpt1tdYktaozRHpup1SJRTs0dLCIIfYXJ/izj3tyuapUxsVbKIE66xAnbC/z4hVFl3lo3L7eTmp69nw/KN/DhrBQcM7Mmwq0+pd6zYBMdQ8xVjNXFCeBL4z7LmE1EtSvY/cdpOWoGjEBK1dwgXcB9uR6K9ZqLzMKEvQbyI7yzE1dXuSDtFplH31Q4gbojOh+ZSyEOfAPE6xz31zSQ00Kp9PieNPLahV9jXaErVy9InKMaYr+yaQy4ikHcvydGd7RwgfiTvD3ZEajBjDImyBzAll0HlU1DxD8zm4SSq3rY72k6u7qT+vR8HZzOaCmbCpHpld+blW/EF6j6gdXtd9D/+oAafPtmzJcVvBRMH74l7EVSpnZrVo3CHfyhS+CJ4BoOzG/jOQdq8h7j72B1t96LfQ9XbJF9yJ0juShOGsvsxGTIsJIErgdqbDbjA2Qtx97UjUpMQ32lA3QeyA0+q4pwbTsTjc+PL8RLI85FTEODBj+7Yuxk47iMhMJJkMXdWX8sLBQ81yepa1TJoG1sLJXfbkb2eGpUo/SMEX6Xuy+4AUjAK8Z9vVcRGMeHvMKV3QGIzkADPcUirhxFHa7ujWSpRtn1WSQgQwAO51+PI/RUbVmxi5pdzyW2Vw1FnHY7X793D2VIz0UUQngASqJ5+uJ+Vn4JqprSNbRMy0SXJDXmjMwAPxj8CybsDcTSwN7O4SBaMVL9UM2e+tXiPgXbjIbEJJNBs7yAd+XdifMOrx8udiP/sHa86OnRvz5m/OLXR1xD3QeBu+JBMpojH4pSVlJPbOge3p+52cMoeWsgbycSLMVsuBVNBshCHIfg+JrYCadOwZdriPyc5K6TObIYEeIfsWy4TwVS+CME3kwuj/GchOTc0uviKCLSAu0fxHIp4DrU7RkZ594mPeXnUm0TDURwuJxfePJyfjrq4xS5WyiT6L9BIpur16gdku95NRyA6J/nyuQHEfQjkXAN4SY6Z+pJ/LngEcezbUmSz9Qao+Gdyw4jEOqh8GVMyMjnfXqm99NlLE3j+zteoLK0iEooSqgjxzqMf8cZD79sdTaGFfK8ZE8JEF+98CBlbAKQojuKEeOruf6k48n6HtP0IyfsDkn8n0u4rHP4z9i1jdE5y4UmNO/wIxDdAaNw+nVO1bK8+8HadlsShqjBvPPIBdjxnUzXp0MpeSFT8Byr/CTjARDG+08B1IIQnU2eOtYmDs/denV9c3cB1VeODRueCSbU4qgoTmYH4tctwc7Jlw1b+ddOLTPmwCBFh8EXHcP1jV5FfaN0zjJJ1W1MeD5aHiIajWdN6obnSO/IGMsGPk0MVJli9ijQCoS8htgLES/Jh5XZe8AxE3AfYE9bRsfoBam0+cGbOdmGq8SLhKL899k4mvzuVSDBCuCrMhNcmc9MJ9xCP113YtK+690/9fdN2/0LcXn3oabesKeTGhDChzzBV72Li69N//cqngWCto2EIfwGFL4PnBMAFkpvcdaf1U2nPuIN3MEgedf55xYkEzrMjkWoik9+dSnlJRY1dhWLROJvXllD02WzLrnPdIz/F66951+0NePjloz9tsZ0IM0lWDK2YyEzM1l+QfKBooCyOybkGR96N6QuR2FzPGxyIoxApfC59WfZAxAWF/8OU3pJcPo+AsxPS6lHEkXqzh71lTATC4yG2BJw9wHc6IvryOt2Wz1tFsKJu75ZIMMrK+ast2+j60JP68ZfP7ub5u15jxfzVdOy5Hz/74yUcOSw72l80dxlfyI2JYrZeVz29bxeVz2O8xyKeo9ITxD0Qwp9TZ/m2+MGReVtOiasz0ubN5ENZE7V0wYlJbMGUjEw2IzOVyUUt5Q9DmzcRZwfLrqP2rOtBnfDn+uoUc4/fTecDre1/0/+Eg3ls4h8tPaeyRuYPrUSmkaqJEYQwVW+mLYbk3Zws2jW+ZH7Iv5vkdqWZSRyFlq8aNGV/hvj66mcFgKmCRDGm7D5Lr6P2bPBFxxDI9+PYZT9Pp8tJ6/YFlt2Nq8yX+YXcpOi4l3xD8sFjmoirJ9LmffCPAGdX8ByPtH4Gh/+ctGXIGKFx1GxnCxCH8MTqNgUqXbx+L09M+TNHDjsMh9OB0+XkuBGD+Ps3f8rKXZjUvsn4oRU8RyVXJtYmAcQ3PK1RxNUNKXgordfMSCLajTWDtO/Slgc/vINEIoHI3vf6Udkv4+/IxZEL+fexs1scyTFZ9yDw7duCGdVI3jOoew/gBO/Je9z2TTUdh8OhRbyFyvw7csARuADjGYAJvgeJUsR7KnhP0qJhE8m/AxOdBYni5PCW+EEKkPz77I7WpDauLOalUW8w4/M55LfJ5aJbzuH0q4Zo8VS20za2ap8YE0tuThxbAq6e4D0Fkea7MKRk/VauPeQWKkurSMSTzwF8OV7O/fUwrv1L+nc2VC1TfW1s9ZZW7RMRF+I7Dcm9AfGd0ayLOMDbj31IsCK0o4gDhCrDvP/EJ5RtKbcxmVJayJVqkDkTFxCL1H3o7va6WTl/jQ2JlNqp0YVcRHwiMk1EZovIfBG534pg2cSYhE67a+b2790BcdQdC4+Go7TtbM1qWaX2lRV35GHgFGPMocBhwDAROcaC82Y8k9hCYuvvMBv7Yzb2JbHl55jYartjqSYw8tZz8NRqDuX2uuh3/EF07NH8N9pQma3RhdwkbV8/767+r9nPMjYmkdz1PvwFycUxCYhMSW7ekKi0O16LFI/FWblgNZvXllh+7j4De3HX6zdT2LEVXr8Ht9fF0WcNZNQ7v7f8WkrtLUumH0pyjfoMoDfwpDFmaor3uQ64DqBr165WXNZekW8hsZGaKxwTYIKY4EdIziV2JWuRvv1gOo9e8xSRcJRELE7vI3py71u30qajdRtDH3vOII4efgSb124hJ99PTkED92RVzUo4GGb53FUUtMvPmFdjljzsNMbEjTGHAZ2Bo0Skf4r3ecYYM8gYM6hdu8xrMrXXYsvARFO8IZickqfSZvm8Vfz58r9TVlJOqCJEJBRl8bQl3H7Gg5bvXuNwOGjfpa0W8RZqzL/GclH7X/B/pz/ANf1v4aYT72ZbcandsaydtWKM2QZ8BQyz8rwZydWnns0bAoj74LTHacne/8cnRMM1Z5TEYwk2LN/I0pkN325Pqd35/su5PHPbK4Qqw1SVBYkEIyyetpT7zn9kjx9bXFXJ37/7lqs/eJdHvv2a9eXWTlm1YtZKOxFpVf1nP3Aa0LBdh7OYcQ8guVnyrhzgyAP/WXZEyngmNJ5EyWUkioeSKL0fE99gyXk3riiuMb97O6fLUe8WZUrtrbcf+7DOvqWxaJwlM5ezfvnGej9u+batDP3vCzw9YxpfrVzOczNncPorL7KgeJNl2ay4I+8ITBCROcB04HNjzEcWnDezbb1+ZxvXHVzQ+gWSv8+aF2OimPA3mNCEfXqYm6h8HrPt5uSm0PGVEHwDs3kEJt74b+Yjhh5aZ/cagEgoRp9BPRt9fqUAttRzU+ByO9m2qazej/vjxAmUh8OEq7fei8TjVEYj3DPhC8uyNfphpzFmDtCitgkx0QUQnQ1Ear3FAeFJ4N67TZcznYl8j9n6S3b0hTcxTP6DOALnNuzjE1VQ/jg1t8qLganAVD6L5N/ZqHzDrz2VD/75KVs3bCNavWjHl+Nl+C+HUtjBuoedquGK1q3lwx8WIcCIAw/m8I7WbnJhhyPPPIxVi9bUGcZLxBP0OKT+CRxT1qxKOY1v1ob1xBIJXI7G309nRdOsjBP7oZ5WriGIzbUjUZMxJojZek3dHZrK7sZ4BiCu7ns+SfxHEGeKr1c0OfunkXIKcnhqxsO88fAHfPvBNHJa5XDBjcM5+dLjG31utfcemDSB1+fNIRSLISK8tWAeVx16OLcdP9juaI1y4c1nM+6liZSXlO+4YfAGvPxi9OX4ArWHWXfyu9xEUmyE7XY6cVrUcE0L+b5w9iD1VHkvuA5Kd5qmFZpA6s81hgm+i+TdsudzONrWM8MHcHRsTLod8tvkce1DV3DtQ9rAyk4LizfxWnURBzDGEIzFeGHWTC44uB+9C9vYnHDftWpXwNOzHuHtxz5k+qezKOzYmpG3nsMRpw3Y7cdd2v8QXpz1/Y6hFQCP08l5Bx5sWedMLeT7wj0AnL0gthjYXqAExIMERtqZzHqmAlK2H4hBomHTrsTZEeMZCJHp7Px6AfiQnGusSKkyxPgVy4imuPtMmAQTVizL6kIOyWJ+zegruGZ0w28Ybjr6OJZsKeHbVatwOR3EEgkO69CRewafbFkuLeT7QESg8EVM2f0QGgvEwX04UvCAZbvUZwzPcaTcM1UCiO/UBp9GWj2RfNgZmQriBhyQdyfiPdqyqMp+XqcLp8NBvFYxdzoceJ0ts9x4XS6ePed8lm3dwpItJfRo1Zo+bdpaeo2W+ZW1gDjykVaPYswjQAJJOac8+4mrMybwM6j6LzseVoof3EeC54SGn8eRjxQ+h4lvhsQWcHVHpO5ME5XdzjygD49O+abOcWPgzN59bEiUOXq2LqRn66a50Wue1SeNkrsUNe9uwI7832O8x2GCb4EJIb5zwHfGPu3QJM624LT2bkRljk55+Yw+dSh3fDlux2yMeMLw8NBhtMvR1bBNRQu5ahDxHod4j7M7hsoC5x3UlyHde/DViuWICCd370G+12d3rGZNC7lSynKtfH7OO6iv3TFajOY9JqCUUi2A3pGrFmFbcSnj/zeZrRu2MWBIPwYOHYDDghV1SmUCLeSq2Zv79ULuPOvPJOJxIqEo7z85lgMH9WL02Ltwe5r3ptGqZdBbEtWsJRIJHrj4MUKVyT7lAKGKEIumLeWTZ7+0OZ1S1tBCrpq1ZbNXEqoM1Tkergrz+UsTbUiklPV0aCXNjDEQmweJSvAc2ixb3mYSp8uBqWcLWadb72NU86CFPI1MbClmy7VgtgIOMHFM/v04AufZHa3Z6t6/K/mFeYQqam4I4MvxcuYvTrMplVLW0luSNDEmjtnyM0isA1NV3RY2CGX3YqIL7Y7XbIkI9737B3IKAvhzfbg8LrwBL4POOIyhP83utqpWWl9ezm2fj+WoZ5/itP8+z6tzZ5OweL9T1XT0jjxdItOrdxSq/cMRwVS9jhTcb0eqFuGAI3ry2pqn+ea9aWzbVMohg/ty4KBedsfKGFuCVZzz+n8pDYWIG8Pmqir+/PVXLNpczAMn66uWbKCFPF1MfS1fE5DYnNYoLZE/x8dpV+gdeCovz55FZSRCfJc78GAsxlsL5vHbo46hfU6ujelUQ+jQSrq4B6beXEH8iLfh7WCVstq0tatrbHqwndfpZEFxsQ2J1N5qdCEXkS4iMkFEForIfBG50YpgzY0420LuL4FdZ6n4wdkT/GfbFUtlmHgiwdItJawvL0/bNbu3ao0jxU410USCTnn5acuh9p0VQysx4FZjzPcikgfMEJHPjTELLDh3s+LI/S3GfTim6lVIlIHvLCRwofblVgCMX76M2z4fSygeI55IcHDbdvxr+Ll0yM1r0uv+/LCBfLB4IcHYzk2F3Q4Hfdu254A22b2jT0vR6DtyY8x6Y8z31X8uBxYCnRp73uZKvCfgaP0Ujjav4si5HBFt76lg6ZYSfvPph2wJBamKRgnH48zZtJEr33s7ufagCR3Qpg3/PnsEHXPz8DqduB1OBnfrwXPnnt+k14XkugqdHdN4lj7sFJHuwOHA1BRvuw64DqBr165WXlaprPffObPq7HWZMIb1FeXM3riBwzpYs0l1fU7s2p3JP7+WjZUVBNwe8r317wpvhapolD99/RXvLVxAJBFnYMf9eeDk0yzfAq2lsOxhp4jkAu8ANxljymq/3RjzjDFmkDFmULt27ay6rLJAIjKXxLabSWy+iETpA5jYSrsjtThrykprzBrZziHCxsqKtGQQETrk5jV5EQe45sP3eHfhfELxGAljKFq3loveeo1NafpcmxtL7shFxE2yiL9qjHnXinOqpmeMwZTeAaH32DG/PTYHE3wDWv8T8Q6xM16z99WK5Tz23TesLi0l3+vB43QSqXVXHo3HOXS/DjYlbBqLSzYze8P6GjNlDMnP9ZU5s7nl2OPtC5elGl3IRUSA54CFxpjHGh9JpU14PITGkHKR0rbboP0URJx2JGv2PlmymD98PnbHA8bScAgBXA4HsUQCAL/Lxci+/Zv8YWe6/bhlC84UveDD8TgLijfZkCj7WXFHfjxwJTBXRGZVH7vTGPOJBedWTcgE3yI56SjVG0MQWwLug9KaqSUwxvCnryfWmCUCyV+nrX0+cj1e8rxefnbo4Yw48GB7QjahAwrb7PhltSuv08mAZvbqI10aXciNMZOBupNQVeYzdReB7JQACaQtSksSjsfqHfcuC0eYes0NaU6UXge0acNR+3dm6i4LkQTwulxcdsih9obLUrqyswUT/wignh1yXN0Rl84uagpep4scd+qve/ucnDSnsce/zz6Xyw45lFyPB5fDwQldu/PuxZfRNqA3D/tCe620ZL4zIfgxRL4Cdrk7l0Kk1b/sStXsiQjXDTySf02fWmN4xe9y8bujjrUxWfr4XG7uGXwy9ww+OeXb15eXM27ZEuIJw9CevelSUJDmhNlFC3kLJuKE1v+CyDRM8ONka13faYhvGCIt+8WaMQmITMZEvkec7cE3HHFYV0xuGHQ0sXiC/8wsIp5I4HW5uOno47iwbz/LrpGtXp83h/snjt/x90e+/ZpbjjmeawceaWOqzCZNvWoslUGDBpmioqK0X1ephjAmjNlyFcQWJXvH4wNxIoUvIe4Bll4rGo9TGg7TyufDlWImR0uzvrycU15+rk4TL5/TxUeXXUnP1oU2JcsMIjLDGDOo9nH9zlGqFlP5X4guqC7iACEwlZitN1q+XN7tdNI2ENAiXu3zZUuRFHMnYibBJ0t+sCFRdtDvHqVqC74P1N2wmUQJxFekOUzLkjAm5R6r2pNl97SQK1VbipauO+mPTFMa2rN3yuNup5NhvQ9Ic5rsod+VStXmvwhI0ZXS2QGcOiWzKXXKz+e24wfjdbpwOxw4RfC5XFw/8EhtqLUbOmtFqVokcBkmPAmiRWCiGDxEEw7+Mu98SmOfcGn/ARzduYvdMZutnx92BEO692Ds0h+IJwxn9DpA+6Lvgc5aUSoFYwxEZ5IIz+DfM1fw/MJCtoaTb/O7XPxy4JH87ujj7A2Zgb5bs5qXZ89kayjIGb0O4JJ+h+CvZ/GT2nv1zVrRO3KlUhAR8BzB56vyeGr+p1RFd+63GozFeKpoGpf0G8B+ubox8Xb/mTGdv0/9dscip9kbN/D6/Lm8d/Fl+FwuXp07myenT6UkWEXv1oXcNXgIx3fpZnPq5kHHyJXajS+WLa1RxLdzOhxMWbPKhkSZqTQU4rHvvqmxUjUUi7G6dBvvLpzP0zOmM3ryRDZWVhBLJFhUsplrP3yf6evW2Ji6+dBCrtRuFPh8OFPMYnEg5HmafgOGbDFzw3rczrotj4OxGB8tWcwT06bU6fYYisV49Ntv0hWxWdNCrtRuXNzvkJQFyukQTuiqwwLbFXi99S6Wmr5uLaFY6nbJS7aUNGWsFkMLuVK7cWCbttx/0in4nC5yPR5y3B5a+/y8dN5FeF36iGm7wzp0pNAfSNnPencLeXq0bt10oVoQ/U5Uag9G9juEYb37MG3dGvwuN0d16qxL6msREV4670J+9sE7bK6qwilCeSSy24/xuVzccoxu62YFnX6olLKMMYa5mzZSFg7zq0/GUFFPMe+cn8+ok07h1B690pwwu+n0Q6VUkxORHdu1Dey4P5NWrqjTOWX/3DwmXnVNcopnE0oYQ9G6tWysrODwDh3pnN98e5prIVdKNYnbTziJ6evWEo7FiBuzYzu3+08+tcmL+LryMi57901KqpIdLGOJBOcd1Jc/nTIURxNf2w6WDPSJyPMisklE5llxPqVUapurqpi3aSPl4XCTXysci/H1yhVMWLEs5Vz6PTmwTVvGXHoFIw48mJ6tW3NKj568ev7ItAyn/OrjMawtK6MyGqUyGiUcjzNm8ULeWTi/ya9tB6vuyF8E/gm8bNH5lFK7CMdi3PbFWD77cSkep5NoPMEvDj+CW489oUnubqesXsX1H49JTikUSCQMfx06jGEH9Nmr8/RsXchfTz/T8ny7s668jMUlm4nXev4XjMV4efZMRvbtn9Y86WDJHbkxZhKwxYpzKaXq+uOkCYz78Uci8TgVkQjheIwXZn3P6/PnWn6tsnCYaz96n/JImIpohIpIhKpYlFvGfcra8jLLr2e1YDSKs55ZRfvyyiIbpG0OlYhcJyJFIlJUXFycrssqlfXCsRjvLpxPOF5zUU0wFuOZGdMtv97ny5amPB43CcYsXmj59azWo3UhgRSNujwOJ2c2057maSvkxphnjDGDjDGD2rVrl67LKpX1grFovYtqtgSrUh5vjIpImHgiUed4NJGgLNT0Y/ON5RDh0aFn4ncle5oD+F1uOublcV0z3cBZZ60oleEKvD7a5eSwrry8xnEBBu3fyfLrndClW/XZawq43Qzp3sPy6zWFE7t155PLruJ/c2ezuryUE7p04/yD+jbblrpayJXKcCLCH4ecxm8+/ZBwLIYhedfpd7n4v+MHW369XoVtuLT/Ibw1fx5VseSYcsDl5qRu3TmqU2fLr9dUurVqxR0nnmR3jLSwZGWniLwGDAHaAhuBUcaY5+p7f13ZqdTem7VhPU8VTWX5tq0ctl9HfnXk0XRv1TS9SowxfL1qJW8tmEcsEWfEgX05vVfvZjkHe29MXLGcl+fMpDQUYljvPlx2yKEpx+ObSn0rO3WJvlJKNcATU6fw9IxpO9rx+lwuuuQX8P4ll6dtyKa+Qq6df5RSag9Kqqr4V9HUOhtnrCkr5d1FC2xMlqSFXCml9mDWhvV46tk448tlP9qQqCYt5EoptQet/L6UG2c4RGifk2NDopp01opSzdDsjRt4fmYRP5SU4HIIXQtac8HBfTm5e88W/8ByXxzeYX9a+/0EY7Eac/o9TidXDDjMvmDVtJAr1cx8uHgR//flZzW2V5tfXMxXK5ZxSo+ePDHs7CbvPrg7CWOYtHIFU9ason0gh3MPOph2AfvvanfHIcJ/zxvJ1WPeZWNlBQ4REsZw/5BT6d9+P7vj6awVpZqTaDzOkf95irJI6hWYfpeb/5xzHsd16ZrmZEnhWIyfvv8284s3URWN4nU6cTocPHfO+RzduYstmfaGMYZFJZspD4cZsN9++FzpXWCks1aUagGWbdtK3NRdXr9dMBatt5dKOrw+fy5zN23c0bwqHI9TFY3y208/StkWINOICAe3bcdRnTqnvYjvjhZypZqRAq+X2G4KosvhIMftSWOimt5btKDGkM92wViURZu1md6+0kKuVDPSITePAft1wCWpf7Rd4uCCg/umOdUu16+nvawx4EoxvU81jBZypbLMguJNPDBpAnd8OS65J2at51xPnnUufdu3rzHv2e1w4HW6uH/IKfRsXZjuyDtc0u8Q/K66cyxa+/30KWxjQ6LmQWetKJVFnp85g79OmUwkFieB4cMfFjGkWw/+cebOmShtAwHev+Rylm4pYU1ZKaWhEG6nixO6diXf67M1/wUH9WXiyuVMWL6MuDG4HU5cDuHps0fYOpMm22khVypLFFdV8si3XxOOx3ccq4pG+WrlciatXMFJtVrM9i5sQ+8Mu8t1Ohz888xzmL9pI1PXrqFtIMDQnr2bbXvZdNFCrlSWmLxyZXILs10KOSSL+dgfl9Qp5JmsX/v96NeA+ddzNm7g9XlzKA0nuw2e2btPvePsLZkWcqWyhNflwpFiwwcHktZWquny8uyZPPTNJMLxOAljmLhiBf+bO5uXz7sItz4YrUF/tSmVJYZ074Gh7gI+j8vJhQf3syFR0ykNhRg9eWKNJfFVsShzN23k06U/2Jwu82ghVypLBNxunj77PHLcbnLdHgJuN16nk1uPPYG+7drbHc9SU9euxu2oe9ddFY3yyRIt5LXp0IpSWeS4Ll2Zes0NfLViOeFYjBO6dcv4PiX7IlDPoiUB8r3e9IbJAlrIlcoyAbebsw7oY3eMJnVM5y54XE6I1jzudbm4tP8Ae0JlMB1aUUplHJfDwYsjLqTQ7yfX7SHH7cHrdHLLMcdzRMf97Y6XcSy5IxeRYcDjgBN41hjzFyvOq5Rqufq3348pV/+S79aspjwS4ZjOnSn0B+yOlZEaXchFxAk8CQwF1gDTRWSMMcb+jeyUUlnN7XRyYrfudsfIeFYMrRwFLDXGLDPGRIDXgREWnFcppVQDWFHIOwGrd/n7mupjNYjIdSJSJCJFxcXarlIppaxiRSFP1emmzqoFY8wzxphBxphB7dq1s+CySimlwJpCvgbYdY+mzsA6C86rlFKqAawo5NOBA0Skh4h4gEuBMRacVymlVAM0etaKMSYmIr8BPiM5/fB5Y8z8RidTSinVIJbMIzfGfAJ8YsW5lFIqEy0u2cyG8nL6tmtPu5zMaougS/SVUmo3tgSruHrMeywp2YzL4SAcj/OT/gO4d/DJGbOrkS7RV0qp3bj5s09ZWLyJYCxGeSRCJB7nzfnzeHth5owgayFXSql6bA0Gmbp2NdFEosbxYCzK8zNn2JSqLi3kSilVj4pIBEc9wydl4XCa09RPC7lSStWjU34+eZ66/c9dDgen9OhpQ6LUtJArpVQ9HCL85bTT8blcOKvvzL1OJ618Pn531LE2p9tJZ60opdRunNy9J+9fcjkvzPqeVaXbOLZzF64YcBitfH67o+2ghVwppfagT5u2jD71dLtj1EsLuVIq40Xicf4zYzqvz59LJB7j9F4HcPMxx+lGE9W0kCulMt71H33Ad2tWE4rHAHhj/lwmrlzOZ5f/DL/bbXM6++nDTqVURltQvInv1u4s4gCxRIItVUE+/GGRjckyhxZypVRGm7tpY8pND6piUYrWrU17nkykhVwpldE65eWnXJTjdTrp0bq1DYkyj46RK5Wl4okEE1YsY+KKFRQG/Fx0cH+6FBTYHctyx3XpSptAgFBZGXGzc/Mxl8PJRX3725gsc2ghVyoLReNxrnr/HeZs2kBVNIrb4eA/3xfx+BnDGdqrt93xLOUQ4Y0LL+WWcZ9UD6UI3Vu14q+nn0m7QGa1k7WLFnKlstD7ixcye+N6grHkA8BoIkE0keDWcZ8y/dob8Lqa14/2frm5vHrBxZSFw0TjcdoEdNrhrnSMXKks9P6iBTuKeA0CMzesT3+gNMn3erWIp6CFXKks5HWmvuM2xuB1OtOcRtlNC7lSWeiyQwYQcNVdCJPj9nBoh442JFJ2alQhF5GRIjJfRBIiMsiqUEqp3Tu1Ry8u6tsPr9OJ3+Uix+0h3+vlP+eeX2//bNV8NfaJyDzgAuBpC7IopRpIRLhvyKlcddgRTF2zmgKfj1O692x2DzlVwzTqX90YsxDImA1IlWpperRqTY9WuiimpUvbGLmIXCciRSJSVFxcnK7LKqVUs7fHO3IR+QLokOJNdxljPmjohYwxzwDPAAwaNMjs4d2VUko10B4LuTHmtHQEUUoptW90+qFSSmW5xk4/PF9E1gDHAh+LyGfWxFJKKdVQYkz6h6tFpBhY2YhTtAU2WxQnnbI1N2Rv9mzNDdmbXXM3nW7GmHa1D9pSyBtLRIqMMVm3AClbc0P2Zs/W3JC92TV3+ukYuVJKZTkt5EopleWytZA/Y3eAfZStuSF7s2drbsje7Jo7zbJyjFwppdRO2XpHrpRSqpoWcqWUynJZW8hF5AERmSMis0RknIjsb3emhhCRR0RkUXX290Skld2ZGirb+s+LyDARWSwiS0XkdrvzNISIPC8im0Rknt1Z9oaIdBGRCSKysPp75Ea7MzWUiPhEZJqIzK7Ofr/dmfZW1o6Ri0i+Maas+s+/A/oaY663OdYeicjpwHhjTExEHgIwxvyfzbEaREQOBhIk+8//3hhTZHOkeomIE/gBGAqsAaYDPzHGLLA12B6IyGCgAnjZGNPf7jwNJSIdgY7GmO9FJA+YAZyX6V9vAEn24c4xxlSIiBuYDNxojPnO5mgNlrV35NuLeLUcICt+Ixljxhljtu+a+x3Q2c48e8MYs9AYs9juHA10FLDUGLPMGBMBXgdG2Jxpj4wxk4AtdufYW8aY9caY76v/XA4sBDrZm6phTFJF9V/d1f9lRT3ZLmsLOYCI/ElEVgOXA/fanWcfXA18aneIZqoTsHqXv68hSwpLthOR7sDhwFSbozSYiDhFZBawCfjcGJM12SHDC7mIfCEi81L8NwLAGHOXMaYL8CrwG3vT7rSn3NXvcxcQI5k9YzQke5ZItW1VVt1lZSMRyQXeAW6q9ao5oxlj4saYw0i+Qj5KRLJmWAsav2dnk9qLXuj/Az4GRjVhnAbbU24RuQo4GzjVZNhDimbUf34N0GWXv3cG1tmUpUWoHl9+B3jVGPOu3Xn2hTFmm4h8BQwjuSdxVsjoO/LdEZEDdvnrucAiu7LsDREZBvwfcK4xpsruPM3YdOAAEekhIh7gUmCMzZmareoHhs8BC40xj9mdZ2+ISLvts8dExA+cRpbUk+2yedbKO8CBJGdRrASuN8astTfVnonIUsALlFQf+i4bZttAsv888A+gHbANmGWMOcPWULshImcBfwecwPPGmD/Zm2jPROQ1YAjJlqobgVHGmOdsDdUAInIC8DUwl+TPJMCdxphP7EvVMCIyAHiJ5PeJA3jTGPNHe1Ptnawt5EoppZKydmhFKaVUkhZypZTKclrIlVIqy2khV0qpLKeFXCmlspwWcqWUynJayJVSKsv9P8WRTm57MzH0AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.datasets import make_blobs\n", "from matplotlib.colors import ListedColormap\n", "\n", "from sklearn.linear_model import LinearRegression\n", "\n", "cm = plt.cm.RdBu\n", "\n", "X, y = make_blobs(n_samples=70, centers=3, n_features=2, random_state=0)\n", "\n", "plt.scatter(X[:,0], X[:,1], c = y)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 6. Hard threshold and Perceptron \n", "\n", "Instead of classifying our points by first learning a plane via a minimization of the $\\ell_2$ loss and then discriminating between classes by looking at the value of the prediction, one could add a threshold function on top of the linear model, defining our hypothesis as \n", "\n", "$$h_{\\mathbf{\\beta}}(\\mathbf{x}) = \\text{f}(\\mathbf{\\beta }^T\\mathbf{x})$$\n", "\n", "where $f(a)$ is defined as $f(a) = +1$ if $a\\geq 0$ and $-1$ otherwise. In such a framework, we can then define the error as \n", "\n", "$$E_P(\\mathbf{\\beta }) = -\\sum_{i\\in \\mathcal{M}}(\\tilde{\\mathbf{\\beta}}^T\\mathbf{x}^{(i)})t^{(i)}$$\n", "\n", "$\\mathcal{M}$ representing the misclassified points. We can then apply stochastic gradient descent on this cost, getting iterates \n", "\n", "$$\\mathbf{\\beta}\\leftarrow \\mathbf{\\beta} - \\eta \\nabla E_P(\\mathbf{\\beta}) = \\mathbf{\\beta}+\\eta \\mathbf{x}^{(i)}t^{(i)}$$\n", "\n", "Implement those steps below." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "from scipy.io import loadmat\n", "pointsClass1 = loadmat('Perceptron_Class1')['Perceptron_Class1']\n", "pointsClass2 = loadmat('Perceptron_Class2')['Perceptron_Class2']\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 7. Logistic regression \n", "\n", "\n", "#### 7.1. Scikit learn\n", "\n", "Another problem with the simple LS classifier is that it can return large values for points that are located far from the decision boundary. With the points given below we will compare the output of the simple least squares classifier and the logistic regression model below.\n", "\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "from scipy.io import loadmat\n", "pointsClass1 = loadmat('Xclass1LR')['Xclass1LR']\n", "pointsClass2 = loadmat('Xclass2LR')['Xclass2LR']\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 7.2. From scratch\n", "\n", "Using the data below, we can also get the separating plane through gradient descent. Implement this idea and plot the resulting boundary" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "from scipy.io import loadmat\n", "pointsClass1 = loadmat('Perceptron_Class1')['Perceptron_Class1']\n", "pointsClass2 = loadmat('Perceptron_Class2')['Perceptron_Class2']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 8. Complex data \n", "\n", "\n", "When no intuition is known on the data distribution, and/or the data is complex (such as images or sounds), an alternative to generating a lot of polynomial features and combining those features with a regularization is to learn the classification through a neural network. Scikit learn comes with a variety of datasets that can be used as benchmarks for classification problems. " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABDb0lEQVR4nO2dd5gT5fbHv+/WbLK71EV6E1DxKlJEBGkiTZqKKKgIogIC6rWgXL1X/YnYsCB28KKAiCAiRUAURRSwsFQporhSl963l5zfH2dzN5tMdlMmMynn8zx5YJPJO2cyyZl3znvO9ygigiAIghD5xJhtgCAIgmAM4vAFQRCiBHH4giAIUYI4fEEQhChBHL4gCEKUEGe2AeVRvXp1atiwodlmCIIghA0bN248QURpWq+FtMNv2LAh0tPTzTZDEAQhbFBK7fP0moR0BEEQogRx+IIgCFGCOHxBEIQoQRy+IAhClCAOX4g8iIDt24G1a4HcXLOtEYSQQRy+EFns3QtceinQrh3Qpw+QlgZ8+KHZVglCSCAOX4gciIDu3YHdu4HsbODcOf533DhgwwazrRME0xGHL0QOGzYAR44AdnvZ5/PygLfeMscmQQghxOELkcPJk0CMxlfabgcOHzbeHkEIMcThC5FD27ZAQYH781Yr0L+/8fYIQoghDl+IHKpVA/7zH3bwDpKSgHr1gLvuMs8uQQgRQlpLRxB85okngNatgalTOcQzcCAwejRgs5ltmSCYjjh8IfLo2ZMfgiCUQUI6giAIUYI4fEEQhChBHL4gCEKUIA5fEAQhShCHLwiCECWIwxcEQYgSxOELgiBECeLwBUEQogQpvBKCw88/AwsWAAkJwJAhwGWXmW2RIEQ94vAF/bn/fmDGDO42FRMDTJkCTJwIPPKI2ZYBxcUssJaUZLYlgmA4EtIR9OWXX9jZ5+RwQ5LiYnb8//43cPCgsbYQAXv2cBesggLgoYeA1FQgJQVo1gz45htj7REEk9HF4SulZiiljimltnt4XSmlpiql9iiltimlWumxXyEE+fxz7T6yMTHAl18aZ8cvvwCNGwMtWgDNmwM1agDvvccXouJi4M8/gRtuADZuNM4mQTAZvWb4HwHoVc7rvQE0LXmMBPCuTvsVQo3ERCA21v15pTiebwT79gFduvDMPieHL0Bnz3LnK2dyc4FJkzyPc/YsMGYMULUqULkyMGIEK3AKQpiii8Mnoh8AnCpnkwEAZhHzM4DKSqlaeuxbCDFuuw2Ij3d/3m4HBgwI/v5XrgSaNnV37loQAb//rv2a3Q506gT897/A6dPs/D/+GLjqKu0mK4IQBhgVw68D4IDT3wdLnnNDKTVSKZWulEo/fvy4IcYJOnLJJcALLwAWCzcisdl4gXT2bG5QEkzOnAFuugkoLPRu+9hYoE0b7de++QbIyCjr3AsLgaNHgUWLArVUEEzBKIevNJ4jrQ2JaBoRtSGiNmlpaUE2KwwoKOAZJml+XP6zbRswfz7w22/6jgsADz7Ii6VTpnDz8IMHuRFJsFm0SLunrSeSkoAnn9R+bds2ID/f/fmsLGDLFn+s048NG4Bbb+WL1fjx0q9X8Bqj0jIPAqjn9HddAJkG7Ts8yctjxzlrFocXatYE3nkH6NMnsHGzs4F+/XhRMzaWFzDbtQOWLi3bGjBQ6tQB7r1Xv/G8ITsbKCrSfi02ljNzzp8HTp3i0MyrrwIXXaS9fZMmfJfiereQnMzjmMXChcDQobz+QMQX7BkzgM2bgfr1zbMr2OTlAevWcbiwfXsgTjLK/YKIdHkAaAhgu4fX+gBYAZ7ptwPwqzdjtm7dmqKWwYOJkpKI+GfND6uV6JdfAht3zBiixMSy4yYmEo0dq4/dZrJnD5HFUvbYACKliB54gKigwPuxCgqI6tcnio0tHScmhqhGDaLs7OAdQ3kUFfH+XY8vNpZo+HBzbDKCxYuJUlKIUlP5Ub060bp1ZlsVsgBIJ09+2tMLvjwAzAVwGEAheDZ/N4DRAEaXvK4AvA3gLwC/AWjjzbhR6/CPHnV3yg7HNWBAYGPbbO7jAvx8JPDEE3xhVKr0uIYOJbLbfR/r0CGi3r2J4uLYqXbtSpSRob/N3vL333xsWuevTh3z7Aom+/e7T3wAdvznz5ttXUhSnsPX5b6IiIZU8DoBGKvHvqKCgwc5vdE1hkzE+eOB4Cl7xZuslnBg0iSgd29g5kxe/7jtNqBHD04L9ZXatYHly3kcIj4nZlK5MofgtKhe3VBTDGP2bA5pumK3A4sXA7ffbrxNYYwEwkKRJk20U/9iYzn2HAidOwOrV5ddBFYK6No1sHFDiWuu4YdeGFU/UBGVK/PFbMWKspMBmw149FHTzAoqp05pL54XFXEyg+ATIq0QiqSmAg8/XHYRVSn++4knAhv77beBSpV4QRLgfytV4mwab9i5E3jtNWD6dN2LkM6f56G7dQPuvJOTUQQXZs7k+gDHebNYgH/+M3Jnur168QXNFaWA664z3p4wR5He6X460qZNG0pPTzfbDHMgAqZNAyZPBk6cADp0AF5+Gbj00sDHPnqUx960CWjVChg1iqUHKrLnoYf4fXY7Z0kQsZRCL40i66IiDof89hsXQg0YUG5I5OxZNuXw4VLNNYsFePdddv6CC3v3AocO8fehcmWzrQkeREDfvsCaNZyFBfAFYOhQ/nIIbiilNhKRZoGJOHzBO779lp2240fnIDkZOHasrPrkqVN8gTp0iLe32Xg2+vPPnK6pwcSJwPPPuy8lpKTw8I4bEiEKKS4G5s3jeH5CAnDPPXwR8GddJgooz+FLSEfwjpkz3Z09wFPxVavKPvfYY1ylev483w2cP89T93Ly8hct0l43VgrYujUw04UwJzaWF99XrOCF2n79xNn7iTh8wTs8ZYdovfbZZ+6LzsXFLFfgUsi0cycwd67nOpqiIqBKFT/sFXwnOxt46ingwgu5uOz557UXTIWwRbJ0BO+4/XaeXbnO8ouK3BfPvAgTFhSw2sK337KzLyjgmwXnDDxHcayZha0RRXExh0WmT+fzNmwYh0cSEvi1Tp34Cuy41XruOb5If/edzKgjBJnhC97Ruzd7aJutVOo4KYlDPcnJZbe9+WZ3xczYWE6/KXl+0iR29rm5HPFxTCRjYjjcb7Oxo1+yxIBjc/DHH8DXX0euNs2QIcC4ccD69cCvv7IOT8+efJVdvpyP3zmulpvLqVJr15pns6ArMsM3moICXoBauhS44AJg5Ej9+r0ScTbDpk3c/KNPn1LHu24dp14eP86NP0aM8E07Ryngo49YH37ZMk4dvfVWoF49920nT+b9ZWay2FhyMm8/ffr/Nnn/ffc+KXY7mzt7Nq/ttmxp0MTy3DlekP7lF76Q5eUBw4ezdpEvYmyhzMaNfN5yckqfy8kB0tN5Fv/TT3yuXCko4M+lY0fjbBWChjh8I8nL4x/Orl0cGomNZb31adOAO+4IbOzsbJ5B79jBP9LERJ4qr1/PoZjHHy8V3PrpJ+7+9Ouvvjv9q66quPirWjW2Y+lSVp1s1oxli53SMj0V9hJxDZjrTUNQuece/pwKCkqvQrNnc8rj/fcbaEgQ+eEHbWG5rCwuxGvYkL8LzhcEgNOjtC7qQnjiSXMhFB4Rp6Xz9tvaWig2W+CCXI895i4cFhtL1LGjthaJ1Ur05pv6HJcfDBlSVpfM8WjVymBDsrO1dYsAosaNDTYmiHz8MVFysvsxWixEU6YQnTlDVKmSu3ZTjRpEublmWy/4AMrR0omQ+9UwYf589xkUwDP9X38NbOxZs9ynzcXFPHPVSoHJyWGpXWf+/JPjvLVrczxl3rzAbCqHl1/mGwFH+n5iIufcO0V9jEEr1dTB2bPG2RFsbrhB+3vgSHmsVInDgZdcwrP6xETuB/zjj1IEEUFISMdIKlXSft5uDzyG4SltUilt8SmlAOcGM3//zQ01srJ4+8OHOc6/ejVXNV51la4a5HXrArt3c0Tr55+5z/ioUXytMYrcXGDdluqIr3ojOhz+DHFw+gxjYnhBM1Kw2XiV/MYbuTDO8dy8eaXfgxYtOEvn4EE+fiNPhmAMnqb+ofCIuJDOihXa8sQNGvgn3+vMmDFECQnut+Rt2xJdeGGpXLBzSOeHH0rff/fd2jEWgEMB1aoRrVkTmI0hxMKFfFipqUSp1kKqjuO0LuYa+l9/gKpVWY440rDbibZuJdq0ifX1hYgD5YR0RFrBaP7zH85icSgwOmZezZsHNu7p09y5ypEZY7Pxbfm6dTwz79mTNQpiYnhx8oUXWHTLwUUXcVpeeSQnAwcOhL12y759HLlwzRJKjc9FZtsbYOtyJS/WXnCBOQYKQgCUJ60gIR2jmTiRUxt//BGoWpVTUmJjAx+3ShUWKlu0iFPtmjTheHxKCr++Zw/nVDsuDK7hpUaNKnb4RMCCBZzVEsbMmqUdASNLEpaMXYkh5XZ3EITwRRy+GdSqBdxyi/7jJiTwuFpjKwW0bev5vU88wRchrUVlB/n5uksim8HJk9rtBkRiXYh0JEtHYDp14hSZ6tU9yxgnJHCuf5hz/fWe18i7dzfWFkEwEnH4Qim33QYcOQL8/jtX6To3nrDZgP79OZMnzLnuOq5/cz28u+9m6X5BiFQkpCOUJTaWqy4XL+aUvY8+4nDQiBHAoEFmW6cLMTGs0TN/PvDxx3xDc889PPMXhEhGZviCNo6CnK+/BlauBG69FXbE4JVXOD3bYuH2uJs26bfLvXtZMv+ii7jv+OrV+o3tSlwcH97y5cAXX/ANjQhCegERC+ZdfjkXU9x9N+ftC2GBpGUKXvPgg8AHH5Rd17XZOCno4osDGzsjg1scZmeXSr5YrdLiMOR47DEWlXNUKMfFcZru9u2SxhoiSMcrIWBOn2aNN9cknrw87pMRKM88wzLJzvpeOTncRldL80swgZMngalTy8pRFBXxiXvjDfPsErxGHL7gFX/9VVor5kxxMSvvBsrq1doKEPn5XCgVyvz+O0shRXxzqG3btHV18vOB77833BzBd8ThC17RsKG2Q4uJYRXhQKlVS/v5oiIWWQtF9u3jVgatW3M6Z1oa8MknZlsVROrV0y5giInhtohCyCMOX/CK6tW5cNehbunAYuGarUCZMMFdmt9i4b4kgSg57NnD9WTnzrm/dvw4i4kePer7uESc3rlzJ4eezp3jyMa99wJbtvhvb0jTpAkX77ne6lkswMMPm2OT4BPi8AWvmTYNGDuWF2pjYliP5ssvgSuuCHzsm24Cnn2Wx05J4VTJ3r2BGTP8G+/kSaBDB04m6deP1xMdaw1FRZyGWb8+p2I2bMgLwy791cvl11+5ZME1DJWfz43FIpZFi4BevfgEJSUBNWsCn37KctpCyCNZOoLPELFz1IrpB0puLsvy16wJ1Kjh/zjduvHM3tmJW63AnDnA1q2sx++8AJ2UxHppL73k3fhLl3KTMq07h169gBUr/Lc9LDhzhvsF1KsXOW0gI4TysnTE4QsRR2Ymh5S12ih26MCLrFqSQCkp2g5ci2PHgAYN3PdhtfLFZOxY3+0WBD2QtEwhqjh92nOvluPHPTeycvR+8YYaNYDx48vKM1gsXIs0fLhP5pbL8ePAU0+xFMTw4Xx3Igj+ItIKQsTRrJm24nR8PIdb0tN5sdaVK67wLTrx7LPAlVdyavrp08DAgaVrHHpw6BCHxs+d47WB9euBzz5jxYu+ffXZhxBd6BLSUUr1AvAGgFgAHxDRiy6vdwGwGMDfJU8tJKJnKxo3akI6e/cCu3axp5L0Nl34+GNumZiby2sOiYmc7bNlCzvSzp05HFNczBeHxERg1Srg6qvNtryUUaN40dq18KxmTT4GCZ0LWgS1AYpSKhbA2wC6AzgIYINSagkR7XTZ9EciknmJMwUFvPK3dCl7nPx8Xm387DP3/EfBJ+64g6+dr73G+fI9erA0RFoaO8xNmzjWvnEjZ/JMmMBZR6HE8uXaVcbnzgH793N2kSD4gh4hnbYA9hBRBgAopT4FMACAq8MXXHnmGc5rzMsrXf379lsODkd0bp8xXH01Xzu1aNaMdYFCmapVtXXJioqA1FTj7RHCHz1uCusAOOD098GS51y5Wim1VSm1QinlsTZTKTVSKZWulEo/fvy4DuaFMO+9595YNS+P7+NDOHtKMIaHHnJfD0hI4IKvqlXNsUkIb/Rw+Fqisq7eahOABkTUAsCbABZ5GoyIphFRGyJqk5aWpoN5IYyzCJUzeXnep4sIEcuwYRzHT0zkFsRWK/efmT3bbMuEcEUPh38QQD2nv+sCyHTegIjOEVFWyf+XA4hXSlXXYd/hTceO2iLsV16pT2PzEKagQG5iKkIp4NVXgQMHuHf8xo3AunUyuxf8Rw+HvwFAU6VUI6VUAoDBAJY4b6CUqqkUezalVNuS/YZ/N+xAmTqVq30cJavx8dxs9Z13zLUriMydy8WZFgvr87z6qjj+ikhL4zBOoD0HBCFgh09ERQDGAVgJYBeA+US0Qyk1Wik1umSzmwFsV0ptBTAVwGAK5RJffygqAl55BWjUiH+hd93FJZ/l0bw5sGMH8MADQJcuwJgxLEHburUhJhvN4sWsYXPwIDv5U6e4qOiVV8y2TBCiA5FW0Ivbb2dhKYdAS1wc6/ru2gVUqWKqaUZy9CgnHhGVipY5uOwybozkSuXKLHUQCnnlW7dyL4+MDJ5VjxkjIRQhvBBphWDz11/AwoVl1biKijhhOgRy/4qL+cZh167ghk8++ohvcB58kB8NGwL//W/p63v3ar8vO5ulhc1myRKgfXtu2bpmDTBpEmv9+yOfLAihiDh8Pdi8WVs6MjeXJRtNZPVqoE4dFg1r04bzz3fs0H8/Bw4A993Hh5ydzde+vDxg3LjSjlWeYtCVKvFShpkUF3O4KSenNEEqL4/vPCZNMtc2QdALcfh60KgRewxXEhJMXWnLzOSwytGjLAyWk8M3I1276t+O7/PPtZ8n4gwTAHjhBfcCYqsVeO4588M5GRnu/XoBllf+8kvj7RGEYCAOXw9atWLHHh9f9vn4eA4Cm8TMme6l+UQ8c122TN99FRZqX/OKi0s16a+7jpc5Lr+cc8sbNwbef59zzc2mUiXPzdKjaAnGWIhY4+K773hGIgQdcfh6oBSwciVLMSYk8KNZM37ORMGTQ4e0Z/JFRfrHpfv396xQOWBA6d89evDCaF4e323ccYe+dvhLjRrANde4X7NtNuneFxT++ot/I507AzfeyCfg/ffNtiriEYevF9Wq8arfyZOcd7h7NwfOTaRbN07r16Jjx4rfv38/MGsWhzS0elc7c9FFwGOPuevQd+wYeqJknpg7lyWSrVbWqrFYeF3ittu0t9+8mathu3ThOP/p00ZaG8YQ8ZU/I4Nn9ufO8eLPww8DP/9stnURjejh601ysmcvazD9+nGq/2+/lUr22Gw8G//HPzy/jwh4/HHgzTfZgSvFNy2rVpXfvzYmhrd3Do2sXctSAEOH6nJIQSUtjXvV7thRqkXvSd1j4UI+JocKxi+/cL3c5s2BtWaMCn79lVuGucqH5OYCb78NtGsXvH0T8Zfy8895QemOOzgVK1ogopB9tG7dmkKW4mKir74ieuwxotdfJzp2zGyLNMnJIXrlFaKWLYnatSP68EM2vTyWLSOy2Yj411H6qFPH83uLi4lSUtzfAxA1bar7YZlKYSFRtWrux5mQQPTww2ZbFwYsX06Umqr9ZbnuuuDt124nGjGCv9xKEcXGEiUlEb3xRvD2aQIA0smDTzXdqZf3CFmHn59P1LUrUXIyf4QWC3+J1qwx2zJd6NdP+7eYkkL000/a78nKIoqL036f1Wqs/cFm587SU+/6uPDCwMefP58v0DVrEt18M9Hvvwc+Zkhx6hT/ZrS+KG+9Fbz9rl6tPZOxWIgOHw7efg2mPIcvMXx/mD6d7+EdmQV5eZx8fsst2qkqYYYnEU+l3NWcHVitZatqnbnsMn3sChUqV/ac0VOtWmBjv/oq967dvBk4coRDR1deCezZE9i4IUWVKsDEifylcZCUxOnNd90VvP0uWKCdexsbC3z1VfD2G0KIw/eHWbO0vzjZ2VzSGubcdpt2X1a73XN4VSnWxHH+DQP89+TJ+ttoJrVqAVddpX9GT24u8PTTZb9adjv/PXGi/+OGJI8+yu3I6tblFfKOHTk7wPULpCdJSdrqtI5FqihAHL4/uKaiOCDyT9b40CGe1qWl8Sxn8mRT7xSGDuUFS4fTj4vj3+EHH5TfeXHwYJ5EtWnD+jOdO/NCrzcZQeHG/PlcT2C1cg5/YiJXFd9yi/9jZmRoF6AVF7Mssj9s3szafCNGcO1FyLRZ+PxzvjpmZnKWzpo1fBU9fDh4+7zjDj5RrtjtodUV/vx54Pjx4OigeIr1hMLD7xh+VhZRQYF/7/WGGTO0Y4H16vHCkC+cPElUowYvIDnHMm+7LTi2e0lhIdFnnxENG0Y0frzxceTsbKKiIvfn9+0j+uc/idq3Jxo5kmj3bmPtcmXbNqKVK/VZsz9xgigxUXttoFs338ebMoXXJGNieAybjWjAgIoX7YNOURFR9eruBxkfTzR2bHD3PXkyx+ytVl6IsVqJvvwyuPv0lhMniPr04dX/hATOdli71udhEDWLtuvWEV16KTvPxESi4cPZ+etNURHRoEH8ZUlI4C9O5cpEGzf6PtYLL/CvUmshKSNDf9tDnBUriBo35lNosxE99FDptXvHDk7uiI/njygujreZP5/ft2+fubbrweDB7uuZVivRt9/6Ns7Ro9rrosnJIeDf/vhDe8IEEDVqpM8+zpwh2r9f++p28CDR9OlEs2fzdqGA3c4r9Y4vt+NhsxH9/bdPQ0WHw//zT/cvUWIiUY8ePn1YPpGeTvTqq0Qff+z/haVPH+0vvsVC1LMnp4ydPq2r2aHKL7+wc3N1diNG8Os9enA2netHpRRRpUr8kd16a3Bv7oJNTg7RnXfyV9dq5Ynw7Nm+jzNnjuc02eHD9bfbJ44d83wr06ZNYGOfOcO3MYmJPJGqVYtoyRJdzA4qGzZoXwQTEjj12weiw+GPGaOdF5iUxDOKUOWRR9yv6q4er0oVol27dNmd6bfz5dC3r7ZDt1g4k0/rRkjrdP/732YfSeCcP893LFphLW/44gvtVPeYGKJx43Q11T969NCezX7ySWDjdu3qfjGxWok2bdLH7mAxf77n2oT+/X0aqjyHHzmLttu3a+fKJSSwbkeoMnZs+RkCOTnAmTPA3Xf7vQsirgKtWZPXlBs25EXHUGP3bu11qoQEll/2RkI5Nxd49139bTOa5GSgfn3/Wxv37Kn9vMUS3MxHr/nkE843TUriVW+LhZsoDB7s/5h//cXSDK4CUnl5od9WrVUrbf2SpCRdsx4ix+FffbW248zPL19HwGwaNWKRtWbNPDt+Is7795QEXwFvvQWMH18qmLZvH//oFy/20+Yg0bq1dpZKVhZr1lxyCfuFihDhRfYTS5dyxmNKCl9ALBaWom7VymzrwAUL69ZxGtEXX3Cm2qRJ2mmT3nLggPZvyG4P7UkfAFx4IYvIOaelxsXxxfCee/Tbj6epfyg8fArpHDrEC6fOMQGrlWjoUJ9uh0zl+HHtmn1HBkNens9D2u2eh7z00iAcQwDs2OF5Lc+xJGO18r+VKnmO53fvbvaRhA45OUQLF/Iy09GjZlsTZE6c0F6pTkggevxxs62rmMJCopdf5oXrtDRevDp0yOdhEBUxfCKO1ffvz6kItWsTTZrEH2I48cgj7l/a+Hiifv3oxAmiTz/l+Gx2tnfDlSd5kJQU1CPxi40bibp0Ydu0HHpiIqdlfvMN0aJFfKodoeCEBA6D7txp9lGEJidOEK1fT5SZabYlQeTRR8uu/MfG8owngqQTKiJ6HH4kkJ3NSeY2G3u9lBSiZs1o2itnyGLhP1NS2NGtWlXxcHY7Txa0HP7llwf3UIqKiJYu5dTKyZN9+82tW+d5Dcv5a5GRQXT//USdOvFv/cAB/Y8j3Cku5oVai6U0m+mWW/y6YQx97HZWCPzHP3jSN3x4ZOTr+oA4/HDDbmeP9847RF9/Tbt2FGtmqNhsROfOVTzce++5pzsmJbEqZrDIyyO65hp3fbnVq717/x9/eM7K6dMneHbrxZkzRE89xX6nfXuiuXN9r8nTi1decT//SrEwpVk2hSWHDnEK2IABRM8/z7dMIYg4/DDniSe0wzIpKZxv7Q0ffkhUvz6Pc/HFwU9NnjpV22HXqOF9quGVV7oftz9FSEaTlcVFks7ZgY4iMjOoU0f7wgkQPfOMOTaFHZs28Q/OcVKTkrhIIgSLI8tz+JGTpRPBnD+vnXFaXOxZ2dKV4cM5O6ewENi1i5ujBJOZM7WTinJyvNeXW7KENX2SkjjbxGoFXnoJuPZafW3Vm1mz3NtLZmdzuuihQ8bbc+aM59deekmymrzi3nv5h+g4qbm5wKlTLAIXRkS3wz94kNtBObpshyg33OBZvbJXL8PN8YryMkxdVSY9UbMmN0fauhVYsYKbJI0bp5+NweKrr7TFVBMSzOng176959fi44E//jDOlrAkLw/YssX9ebudU6rDiOh0+MeOcTFD06bcd7ZGDW5oGqJ07cpifg6nHxPDs90nnwTq1TPXNk+MHKl9kUpL872jXNOm7LS0xgsGxcWs1PvYY8AbbwAnTvj2fk8FU0R8ETOa117zXMBVUADUqWOsPWFHXJznDzCYcs7BwFOsJxQeQYvht27tXtZttbKeRYhit/Mi67BhRKNGee48FSoUF7OujSNvPiWFqGpVoq1bzbasfHJzia66qnSxOSmJ/79+vfdj7NzpvkgaG0vUpIl5i6Tz5rmvh1gsRAMHmmNP2HH77Zz36/oBPvGE2Za5AVm0dWL7dvdfoyNtwWRJ4khk82bWf/v0Uy4CCnVefll7sbl+fd+c9aJFfIFLTubxWrY0PztwwQJum2ix8EV46FDv6zminjNnStOlU1P5pPbrx+1OQ4zyHL7i10OTNm3aUHp6ur6DfvstMHAgcPas+2sdOnBHeyFqufxyXtZxxWYD0tOBiy/2fqyiImDHDpY2aNxYPxsDwW7n1omVKhkXIosoNm8G/vyTvyi+fBkMRCm1kYjaaL3moXVTBNOypbu4EsCdcNq354Bt9erG2yWEBJ5CtUSeG515Ii4OaNEicJv0JCYGqF3bbCvCmJYt+RGm6LJoq5TqpZTarZTao5SaoPG6UkpNLXl9m1LKPPmmqlWBCRPKTm/i4ng6NnUq99i85hoWYhKijnvucV+HU4q/FhdeaI5NgqAXATt8pVQsgLcB9AbQHMAQpVRzl816A2ha8hgJwFwB26ef5mTpDh2AJk34F11czDP//HzOnevcOYQagFZMdjYwZQofUt++nBqoFzt2ALfeymGJnj2BH3/Ub+xQY9QooFs3dvoJCawyWbUqt2ANRMhREEICT8F9bx8Argaw0unvfwH4l8s27wMY4vT3bgC1KhrbkErbiRO1FfZSUlihKwzIyeESfufFRptNnyrKzZt5LEdfVEdC08KFgY8dyvz6K9Hrr4fPYrMgOECQK23rAHCOfxwsec7XbQAASqmRSql0pVT68ePHdTCvAvbs4cIKV+z2sAnrzJ4NZGSUrWzNzgZefJFLDgJh/Hgey/lmJycHeOABdv+RypVXAv/8J9/ZJCWZbY33FBYCa9ZwboLW11qIbvRw+Fo3uq6uwJtt+EmiaUTUhojapKWlBWxchXTu7LmMtY3mQnfIsXSp58rO9esDG/vXX7WfP3pUO9FJCB4FBdwrZOpUjjq6XnDXrgVq1QL69wduugm44AIuIBMEB3pk6RwE4FzvWRdAph/bmMPgwdxp58CB0hZjVivQowdw2WXm2uYlNWty9oXrkgNR4AlHNWoA5865Px8XJ2l9RvLXX5xLkJ3NX9O4OF6vWbqUL+znzgG9e7vr4txyC2cRSjWtAOgzw98AoKlSqpFSKgHAYABLXLZZAuDOkmyddgDOEtFhHfYdOElJwIYNLNJSrx7X8U+cCHz2mdmWec3YsZxV6oxSvNhYno6KN/zrX+5ZK0lJrCXlrSaO4Dt5ecDvv5feRQ0ezOE5h35XdjYvnk+Zwq8vWqQ9jt0OzJljhMVCOBCwwyeiIgDjAKwEsAvAfCLaoZQarZQaXbLZcgAZAPYAmA5gTKD71ZUqVYBXXwX272clqYcf9j3p2kSuuAJ47z3OKElN5Zl3kybAqlXaPWJ94a67OIvVauUCIosFuO220O8JHa7s388zeZuNbzBr1ACGDGGFUdc7uNxc4IMP+P+nT2trAObnAydPBt9uITyIvkrbCCY3l6tBU1O5EFDPNMKcHHZGtWpxlabgH46fm9a5+eMPvni7ykrHx7OzLy52f0/DhsDff3Pq7JVXur/XZuOwT9euelgvhAPlVdpGp1pmhJKUxCKgLVronzNutXIluTh7/8jI4GWh+Hg+T0OHuuvUP/aYdg+BwkJtZ5+YyHdbACuQDh1adl3FZuOagi5d9DoKIdyRGb4AAMjMBD76iGfx114L3HijxOj14uxZDrGdOlUalklIAJo3BzZtKr04V67sOfMpPp4dfHExXxSSk3l2v349h9oAvntYuhSYMYMXdu+8Exg0yLNchBCZiJaOUC4//ABcfz2rS+Tn8yLfCy9wmp9k4gTOrFkcEnOOwRcUcAnIjz8CnTrxc+U5/GrVOI7/ySfcuax9e2DAgLIXZaU4JbN//6AdihDmSEgnyrHbOSyQnV2qKZeVxRkijgwQfzlzhhuI3HUX8Oab0Zu3v22bdp2E3c6fs4OHHuJFcVdiYzn3Pi0NePBBbmhy881yByb4jjj8KGf3bu2ep3l5PJv0lz17OIzxxBMcKpowgTNe9+71f8xwpWVL7TulmBgO6zi4/35gxAgO98TH84y9WjUunho0yDh7hchFHH6Uk5joWSPONbffF8aM4VRBx8w2J4fTA++/3/8xw5U77mCH75wim5jIi+AdOpQ+FxMDvP021wB+8w1n35w4Ebp9iwPmyBHu03nddaxj8fffZlsU8ciirYB//APYubNsqb7VCrz+Ovem9RUqaVSulVmSkKDdjiDS2beP9Ye++oo/m9tvByZP5hTaqOTPP4G2bXkFOj+/dFV61SrgqqvMti6skUVboVwWLuSFw5wcdtJELLF8993+jxkXp+3wozXu3KABsHix2VaEEI88wos6jllGYSE/Ro0Ctmwx1bRIRhy+gGbNOIywYgVw+DBngAQiI6QUq0x++mmpPBHAE7jbbw/cXiEC+O47bbnV7dt51h9OEqVhhDh8AQDPvPVM55s6lX+7u3fz71opDh2FiyRDYSHw7rvAf//LaxzDhrHcklYWjeAHycmcGuZKXFz03gYagDh8IShUqsQyD+vXc+ph8+ZAu3bh0TXKEdJau7Z00fmppzgks2ZN4PpEAnhV/8UXy5YWJyaySlwY6ViFG7JoKwgurF3LmTGuE9DkZGDBAm7zKARIYSGXAi9axI6+oIBjiYsW8Qct+I0s2gqCD6xfr51JlJUFrFsnDl8X4uOBuXM5fWn7di7auOgis62KeOTmVBBcqFVLO1ZvtQK1axtjw6lTXHlbvz7n60+ZwtIXEUeDBkCfPuLsDUIcfiB8/z23SKxVi6d9GzaYbVHUc/o0awHNmcNO0x9uukl73bCgAOjePTD7vCEnh6WO33mHs6d27+b6JMlwEgJFHL6/LF7MM5MffuCKwa+/Zh3atWvNtixqmTuXW/mNHs2POnXKl4ew21ldcsgQDievXs3P22zA/PnuC8xEwMCBZbMJc3OBlSu5Xsg5BTUQ5szhnsHO4+XksK27dumzDyE6EYfvD0RcCu6qiJWTAzz6qCkmRTuZmVwolpvLsfasLNYDuvtu4NAh9+2JWPJgyBCuF5g9G+jXDxg/nl9fu9Z9ll9czL1lf/6Z/166lDtS3XILXwhq1OD08kD5/nvtjMXYWLmJFAJDHL4/5OezcLwWW7caa4sHjhzhhIf167XrW1zJyeH0Sa2G5eHAggXax0mk3Z547VpgyZKyjjU7G3jrLa7637VLe8auFEu+ZGZycVlWFn9m585x4Wj//tpidL7QpIm2jpFS3HY5ZMjK4tzbw6HRnlqoGHH4/pCY6Dl17IILjLXFBSLg8ce5OcawYby00LQpJ0N42v7ZZ1l6t21bNv+++8JvgTA3V1vKwdEwxJUvv9SeRQOsd9O+vXvzdoA/lyuu4PCRJ9G5zz/32mxN7r3XPRU9NpbvIDp3DmxsXSACnnuODerWDWjcmMX5PX2gQsggDt8flOIUClePYLWyHrCJfPEFKy7m5/OsMyuLZ6SeqminTwdefpln+OfPcxhk1izgX/8y1u5A6dtXu14nPp5DNa6kpmovzMbFcQep4cN5G+cxk5JY2LF5c14c1roDyM5mbSJPFwNvqFuX1wUaNeJ9JiYCV18dQkVfn37KHXJyc/lLlpfHa1iBiC+FOidPcqrUAw/w1V6vBRujIaKQfbRu3ZpClqIiokcfJUpKIrLZiJKTiZ5/nshuN9WsLl2IeApW9mG1Ev3xh/v2jRppb5+QwIcYTowfz8epFD+sVqKHH9be9u+/+dS5HrfNRnTmDG9z6BDR0KFEVasS1a5N9MwzRPn5/NqPP/K2Wp+dxUL0+OOBH4/dTrR/P9GxY4GPpSstWmgfeGIi0dmzZlunP5s3E6Wmln5hkpOJLrqo9IsSYgBIJw8+1XSnXt4jpB2+g5wc9h55eWZbQkREV1yh/VtMSSFKT3ffPjlZe3uAaOpUoj//JNq4kaigIDC7MjOJxo0jataMqFMnoiVLAhvPE+vX837GjiVat678bT/5hC8Kqan8SEkh+uYb7/ZjtxPdcgtRbKz2Z5eURJSdHfjxGEp+PtGJE0TFxeVvV6uW51nFvn3G2Gokl16qfXF75BGzLdNEHH4UMXEizzBdv5+VK5fOTp0Z0XoLPYLJdDemU2WcKvOe2Fh2XCkp/P7PPvPPpsOHidLSiOLjS8e22YgmTw7sWPXg3DmiL74g+vJLvnb7QnExz/61fF9yMl8sw4KCAqIHHuCTnZhIVLMm0dy5nrcfMoQoJsb9oNPS9L0tzM0l+uc/+QsYF0fUrRvRrl36je8NR47wZ6J1kuvWNdYWLxGHH0WcPcuzaKu11GlbrRrO2m4nuvtuKrJYKQ8JdB42Og8bdcF3Hmf8VivRb7/5btOjj3KISGsWfP68LoetKzk5RD/9RLR7d8Xb9uzp+bMKmxn+6NHu8S2r1fPtzp49RJUqsRN23n7ePH3t6t277OxFKd7v4cP67qc8TpzQ/vICRA0bGmeHD4jDjzKysojefpuoTx/+LW/bprHR4sWaQehTqETxyNf8fsfG8ni+4inMlJpK9PPPAR+urkybxh9Lair7sFatOJbviZ9/1vaVTzxhnM0Bcf689i0hQNSxo+f37d1LdN99RM2bE/XtS7R2rb527dqlvciSmEj0n//ou6+KaN/e/Y4mKYnoueeMtcNLxOEL7tx4o+aP/AxS6Vqs8jjL79PH91317as9lsXCfiNUWLu29M7I+SLXokX57/vuO6LLL+dta9TgUJXJa/fes2eP59VnM0MWn3/OV10tu66/3lhb/v6bP4uUFP7S2mwcXgqRdTtXynP4opYZrWglrZdQOcWOuFz3XHyrFejd2/ddjR/PFajOhckJCayP36CB7+MFiylT3HP2i4u5EGvHDuDSS7Xf17Wrf/V2W7eyXs6hQ6zSMWyYdu5/UKlbVzvXUymgjabCrjFcfDFLKLuSmAi0amWsLQ0bAhkZXKCxbx8LHbVtGx7NHVzxdCUIhYfM8IPIggWaMzt7SgpRbi6NH1/2ZYuFqGlTDhf5w4wZpZkwFgtR9+5EJ0/qe0iB0r699oQyNZVn8XoyZw7fTTiyfKxWoksuMWlN45VX3G9tbDYPsUADue4693BTamr5MTZBQjohjd1ecRpcMCgu5rxCm40XwywWjksuW/Y/sxYs4BTKFi2Inn028BTr/Hz2IZmZgZsfDCZN0g5nJyXpm16em8sXPq39vPSSfvvxiTlz+IpTpQqvRG/aZJIhTmRn8zpBUhLH0Dt18i9rIMoQhx+KZGURjRzJHiYmhqeXRs+o7HYOXD/1FNFrr4WuJ66AoiKil18mql+f/dWQIf6lg585Q9SgQVmnb7Xqnz7600+ew9NXXqnvviKGsFkUMZ/yHH5AMXylVFUA8wA0BLAXwC1EdFpju70AzgMoBlBEHtpvRRX9+3P7JEdrpfXrgQ4dWMHMqC4bSvE+O3QwZn9BYvhwljNwrBHMnw988w0LoFWv7v04lSoBmzcDb77Jwmo1arAoao8e+tqbmupZq6hKFX335RN2O7BiBbBsGVCtGn+wF15ookFOhGO8PBTxdCXw5gHgZQATSv4/AcBLHrbbC6C6r+NH7Az/t9/cY6aOlLN//9ts68KKvXu1wzAWC4ehQhG7naMnrpl+NhvR0qUmGVVYSNSjR2npdXw8h1L0zq0Xgg7KmeEHKsU0AMDMkv/PBHBDgOOFN4WFwMSJPENPTeXWSRkZ7tvt3q2t9JWfz1NMwWu2buWMH1fy8kK3F41SrNbZoAELtaWmckvFhx5iEThT+PRTvuPMyuK/Cws5ZWnECPe+D0LYEmha5gVEdBgAiOiwUqqGh+0IwNdKKQLwPhFN8zSgUmokgJEAUL9+/QDNCzIHDrAufvPmfC8+dCjHAhy5fYsXs8Thrl0cH3Bw6aXaKWcWi7mpcGFIo0ba4ZH4eOCSS4y3x1saN+ZmKj/9BBw/zmqYNTz9eoxgzhzPXVfWrtU/riWYQoUzfKXUKqXUdo3HAB/204GIWgHoDWCsUqqTpw2JaBoRtSGiNmlpaT7swkCysjhxulkz/rd2bWDUKHbwzoncdjv/iN5+u+z7L76Y2yE6d8pWiv++7z5DDiFSuOwy1qd3neUnJLCSbSijFOvuDxhgsrMHWIfZE1od3YWwpEKHT0TXEdE/NB6LARxVStUCgJJ/j3kYI7Pk32MAvgDQVr9DMIF77wW+/ZbjBmfP8r8zZ2ovLOXnl/bEc2bhQmDsWF4pTEjgGdQvvwS/gcrOnVwJNXIkL84FItweIixbxmvgCQk8s7/4Yq6RadzYbMvCiJEjuZmvK4mJfFUSIgLFMX4/36zUZAAniehFpdQEAFWJ6DGXbWwAYojofMn/vwHwLBF9VdH4bdq0ofT0dL/tCwrZ2ZzB4MiucUYpXn9zJj4eGDcOeO01Y+wrjxkz2JbCQo6DJCfzncaiRXzrHubk5vK119RMl3CFiPsxv/MOV97GxvK/X3/NVaVC2KCU2kgeMiEDdfjVAMwHUB/AfgCDiOiUUqo2gA+I6HqlVGPwrB7gNYNPiGiSN+OHpMM/fJinjnl57q/FxfEPxflikJwMbNvGwWYzOXOGQ0+u2gE2G7e4uummisfYuJHvZAoKgEGDgGuvlXS5SCMjg3UwKlfmcGV5oR4hJCnP4Qe0aEtEJwF003g+E8D1Jf/PANAikP2EFDVrAlWrchdrZ2JiuJeezcaJ4MXFvDg7bZr5zh4Avv+e7zZcHX52NjBvXvkOPzcXuO02YOlSDgERAR9/DAwcCHz0kTj9SKJxY4mFRTCh0CEzvFAKeP99VrlyOLr4eM6tmzwZmD2bF3XPnuWcwauuMtdeB54W3pQqX7ErO5tXRRct4ouY444wO5u7df/4o+f3EgEffgi0aAHUrw+MHs13SIIgmIKoZfpD377ADz9w9+8//wQ6deL4Z926/Hp8vHaHbDPp2lVbFTEpqfzm09Oncxd0LXJyuGt6Jw9JVw8/zO93pPv997+8/c6dvA4SiWzcyHd4SgGDB/PF0kB27OAMy/x8vgG7+uoovQHLzORQasOGvIovMJ4qskLhEbGVtmaxdm2pZKXNxuWozzxT/ns8SUg6xOKffFL7fUeOeC6B/b//0//YQoEJE7iCOiaGH0lJhpb7vv467zI2lvXwbDZuWBNVMjTFxaUaVZUq8QfSsWPINhwPBhDxNOF/5OSwDOaMGUQHDlS8fZ8+nh1+YiLRzp3a71u5kn9wWu/r3FnPIwoqhYV8nVyzRrsn8P/Ytk27Q5PFYkhz24MHta+vVmvFzdwjijfecJctSUggGjjQbMsMozyHLzH8aCMpie/177qrNARVHuPGaednKwW8/rrncta6dbWriWNjQ0eQqwLWrQNq1eKmL/36cXHUypUeNl68mLOXXCHi6usgs2KFdsQuN5eXWsKWZcs4LlW7NicW7NhR/vZTp7pLQRQUsJaFViVxlCEOXyifXr24UMtiYeGXxESOv//0U/lVwc2bcxms61pGYiJLUIY4586xoz9xAjh/nv8+e5Z9zpEjGm9ISND2uDEx2mI/OhMf73n3iYlB331wmDEDuOUWLlw8fJgTB9q1A377zfN7zp71/Jo4fHH4ghc8/TRrBs2eDaxaBRw75l320bJlQLdu7HGsVp4uz5/PF4IQZ+FC9xo6gLNS587VeMOgQdqCeETe1TgESP/+2l0rExI4ozbsKCriRAjn2ToRO+1//9vz+3r21C4irFMHCFWpFgMRhy94R1oai75cc432VFKLatU41nDoELB9O3DwIBfzhAGnTmlHaPLyWOzMjUaNuCmuxcIhMJuN/z9tmiH9DapUAT75hCN2ycl8fbVYgOeeA/7xj6DvXn+OHNEubiTSlipx8PzzXDTmuK2JjeUPY/r0KE1XKoukZQrBp1q1sEvD7NaNJ+yuTt9mK0c4cuRInmp/+SU7l/79DZ1V3nADX1uXLGG7r7+eJ7ZhSdWq2rdYQPlrT/Xrc9rvW29xjcjFFwMPPiipmSUEJK0QbEJSWkGIGoYPBxYsKA392mxczrBkiUwWDWH0aJb9cK4Ot1q5yvvGG82zK8QJmrSCIEQyH37INXYzZnBI+c47gSFDxNkbxtSpPMufNYtDM3FxwAsviLMPAJnhC4IQ2mRl8cJJ3bqhV8EegsgMXxCE8CU5mR9CwEiWjiAIQpQgDl8QBO8pKuKG5zffzA3Of/rJbIsEH5CQjiAI3lFUxDmpv/7KqUtKcS+FZ58FHnnEbOsEL5AZviBEGrm53LOhXz9g1ChgyxZ9xv3ii1JnD3AGTU4O8OSTHqrRhFBDZviC72RlsUTCwYMssdC9u/fVt0Jwyc7mc7J3L/8/Npbz1t9/H7jjjsDGXrhQW48mIQFYvZp1b4SQRhy+4Bvbt3PDk4IC/vEnJ3Mrx+++K79zlmAM773HfWkdxUrFxTwLHzOG4+6eOp95Q+XKfGG328s+rxQL6wkhj0zLBN8YPBg4fbp0ppeVxa0cX33VMBMKCrixVEaGYbsMHxYscO9bDLBTDrSm5Z57tC8YcXGsRSGEPOLwBe85dAj46y/35/PygJkzDTFhzhyWp+nalUXBrrySzRJKqFJF+/niYu67HAitW3PfZouFx0pJAapX5yYBBkhAC4EjDl/QBwP0BtLTWZ/s3DnWqM/N5fXIXr0862xFHVoNa5TiKlVfZalPnOC+sM4SxWPGcL/Yjz7iziqZmUAbzaJOIQQRhy94T506QNOm7s49KQkYNizou5861V0xt6iIe6xv3Rr03YcH11/POvKOWXhyMitILlvm/UU5Nxe49Va+SFxzDd9Svfhi6etVqrCeTffuInUQZojDF3zj00/5B5+czAt4yclAy5bsZILMwYPu64UAJ6IcPRr03YcPzzwD7NvHYbYVK3ixw5e2kqNHsyRofj7fSuXkABMncs59oBBxx6otW7RPphBURDxN8J2cHL6dP3CAUwCvvdaQkM7kydx8y3VN0mLhi0GYSe6HJllZHJfPz3d/7fLLA7uV2rSJ7wxOnuTvi83G6b2dOvk/puCGiKcJ+mK1AkOHGr7bUaOAd97h9qYOf2SzcctdcfY6ceaM55qKQG6jsrM5k+fMmdLnsrK4A1pGhrQfNAgJ6QhhQ2oqTxIff5wnm127clu/p58227IIolYtbWXKmBigY0f/x120iBdcXCkq4pMoGII4fCGsqFIF+L//48jCd99xF0FBR2JjgTfeKFtEFxvLF4FJk/wf9+hRz02CDx/2f1zBJ8ThC+5s2QIsXswxeiH6GDIEWL4c6NmTs7LuvJNvrZo183/MTp24QMuV5GS+VRMMQWL4QiknTnBS+++/86yuoIBj9e+9J1o50UbnzvzQizZt+Lu1cmVplbbVysVc3bvrtx+hXAL6FSulBimldiil7Eopj9UXSqleSqndSqk9SqkJgexTCCJDh3KhTXY2Vzfl5XFp67vvmm2ZEAnMn8/FFFdfzSXSL78MfP21TCYMJKC0TKXUJQDsAN4H8CgRueVQKqViAfwBoDuAgwA2ABhCRDsrGl/SMg3kzBngggu046wXXcSzfkEQQp7y0jIDurQS0S4i2l3BZm0B7CGiDCIqAPApgAGB7DeqKSoC3n4buOIKFpN56SVtsSxfyc72PNM6fz7w8QVBMB0j7qXqAHBe/TtY8pzgDwMHAo89xmkqO3ZwykqXLiyOFQi1awM1arg/HxcH9O0b2NjhyMmTwMMPA40bcw7o9OlSGSqEPRUu2iqlVgGoqfHSk0S02It9aJVgeowjKaVGAhgJAPXr1/di+CgiPR349tuyYla5ucDOnayVEkiOolLAjBnAgAFc1VRUxBo5lSpxqX40cf48LyYePlwa4nroIe72NH26ubYJ2hQVAUuXAt9/D9Srx+tRF1xgtlUhR4UOn4iuC3AfBwHUc/q7LoDMcvY3DcA0gGP4Ae47sli3Trt4JSsL+OGHwJPSu3UDNm/mhbU//+R0uZEjPUvuRioffcQt+5zXM7KzuXPUk08CDRuaZZmgRW4up33+/jv/FiwWvvNdsYLF34T/YURa5gYATZVSjQAcAjAYwG0G7DfyqFWLdcdddU6SkljJUg+aNgXefFOfscKV774rexflID6e77L0cvj5+SxItnw5K1OOHBlYrnu0MnUqhzcda1kOSdUhQ4D9+w3ReQoXAk3LvFEpdRDA1QCWKaVWljxfWym1HACIqAjAOAArAewCMJ+IdgRmdpTSvz+QmOj+BY6NDbxfqVBKo0basr9E+l1Ys7OBtm1ZX37ePK5ubdmSC94E35g9Wztx4fRpyS5zIdAsnS+IqC4RJRLRBUTUs+T5TCK63mm75UTUjIguJKIA6rOjHIsFWLOG0ySTklg5rH59zmUW8Sn9GDPG3eHHxXFsuF07ffbx7rscNnMUIRUV8V3F8OFAYaE++4gWPHXbstulE5cLUvEQbjRvDuzaxZriGzcCe/dyIYugH02asNhXnTpcDZqYyJ/xt9/qFx6YN097VlpczNIWgveMHFlW+wfg89SggW99AKIAkVYIV+SLHFy6d2ctoYwM1nvRO+NDS5ESYIfv2qJQKJ977wW++Qb46iue1cfH893wwoVmWxZyiMMXBE8oFbwL69ixwIYNpSEdx/7q1QMuuSQ4+4xUYmO5Ic/mzZzJVqsW144kJpptWcghDl8QzGDgQGDtWuD990vXC1JTOZdcskr8o2VLfggeEYcvCGagFDBlChd0rVvHVc5du/JsVRCChDh8QTCTBg34IQgGIFk6giAIUYI4fEEQhChBHL4gCEKUIA5fEAQhShCHLwiCECWIwxcEQYgSJC1TEIxkzx5g5kzg1CmgTx+gVy9p4i0Yhjh8QTCKefOAu+5iZczCQmDWLG7QsXQpq3EKQpCRqYUgGEF2NjBiBCtkOuSPs7KAH38EPvvMXNuEqEEcviAYwY8/as/is7OBTz4x3h4hKhGHLwhGUJ5yY1KScXYIUY04fEEwgmuu0W6baLMB99xjvD1CVCIOXxCMID4e+PJLoFIlICWFOzRZLMB993GzFUEwAEkNEASjaNcOyMwEli0Dzp4FunXjhumCYBDi8AXBSKxWYNAgs60QohQJ6QiCIEQJ4vAFQRCiBHH4giAIUYI4fEEQhChBHL4gCEKUoIjIbBs8opQ6DmCfiSZUB3DCxP0biRxr5BEtxwnIsTrTgIjStF4IaYdvNkqpdCJqY7YdRiDHGnlEy3ECcqzeIiEdQRCEKEEcviAIQpQgDr98ppltgIHIsUYe0XKcgByrV0gMXxAEIUqQGb4gCEKUIA5fEAQhShCH74RSapBSaodSyq6U8pj2pJTqpZTarZTao5SaYKSNeqGUqqqU+kYp9WfJv1U8bLdXKfWbUmqLUirdaDv9paJzpJipJa9vU0q1MsNOPfDiWLsopc6WnMMtSqmnzLAzUJRSM5RSx5RS2z28HknntKJj9e+cEpE8Sh4ALgFwEYDvAbTxsE0sgL8ANAaQAGArgOZm2+7Hsb4MYELJ/ycAeMnDdnsBVDfbXh+PrcJzBOB6ACsAKADtAPxitt1BPNYuAL4021YdjrUTgFYAtnt4PSLOqZfH6tc5lRm+E0S0i4h2V7BZWwB7iCiDiAoAfApgQPCt050BAGaW/H8mgBvMM0V3vDlHAwDMIuZnAJWVUrWMNlQHIuX7WCFE9AOAU+VsEinn1Jtj9Qtx+L5TB8ABp78PljwXblxARIcBoOTfGh62IwBfK6U2KqVGGmZdYHhzjiLlPHp7HFcrpbYqpVYopS41xjTDiZRz6i0+n9Oo63illFoFoKbGS08S0WJvhtB4LiRzW8s7Vh+G6UBEmUqpGgC+UUr9XjL7CGW8OUdhcx4rwJvj2ATWV8lSSl0PYBGApsE2zAQi5Zx6g1/nNOocPhFdF+AQBwHUc/q7LoDMAMcMCuUdq1LqqFKqFhEdLrntPeZhjMySf48ppb4AhxBC3eF7c47C5jxWQIXHQUTnnP6/XCn1jlKqOhFFmthYpJzTCvH3nEpIx3c2AGiqlGqklEoAMBjAEpNt8oclAIaV/H8YALe7G6WUTSmV4vg/gB4ANLMGQgxvztESAHeWZHa0A3DWEeIKMyo8VqVUTaWUKvl/W/Dv/qThlgafSDmnFeLvOY26GX55KKVuBPAmgDQAy5RSW4iop1KqNoAPiOh6IipSSo0DsBKcITGDiHaYaLa/vAhgvlLqbgD7AQwCAOdjBXABgC9KvldxAD4hoq9MstdrPJ0jpdToktffA7AcnNWxB0AOgLvMsjcQvDzWmwHcp5QqApALYDCVpHqEE0qpueDslOpKqYMAngYQD0TWOQW8Ola/zqlIKwiCIEQJEtIRBEGIEsThC4IgRAni8AVBEKIEcfiCIAhRgjh8QRCEKEEcviAIQpQgDl8QBCFK+H+8nxKdxhlu3QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.datasets import make_moons, make_circles, make_classification\n", "from matplotlib.colors import ListedColormap\n", "from sklearn.model_selection import train_test_split\n", "\n", "dataset = make_circles(n_samples = 200, noise=0.2, factor=0.5, random_state=1)\n", "X, y = dataset\n", "X_train, X_test, y_train, y_test = \\\n", " train_test_split(X, y, test_size=.4, random_state=42)\n", " \n", "cm = plt.cm.RdBu\n", "cm_bright = ListedColormap(['#FF0000', '#0000FF'])\n", "\n", "plt.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_bright)\n", "plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 2 }