{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# MATH60629A\n", "# Week \\#5 - Neural Networks - Exercices\n", "\n", "This tutorial explores neural networks." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--2021-10-01 15:19:56-- https://raw.githubusercontent.com/lcharlin/80-629/master/week5-NeuralNetworks/utils.py\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 13166 (13K) [text/plain]\n", "Saving to: ‘utilities.py’\n", "\n", "utilities.py 100%[===================>] 12.86K --.-KB/s in 0.002s \n", "\n", "2021-10-01 15:19:56 (7.38 MB/s) - ‘utilities.py’ saved [13166/13166]\n", "\n" ] } ], "source": [ "import numpy as np\n", "\n", "# Code to obtain utils.py\n", "!wget https://raw.githubusercontent.com/lcharlin/80-629/master/week5-NeuralNetworks/utils.py -O utilities.py" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## A tiny neural network classifier" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to classify the examples, we will use the following simple neural network:\n", "\n", "\n", "\n", "where $\\sigma$ is the sigmoid function defined as:\n", "\n", "$$\n", " \\sigma(x) = \\frac{1}{1+ e^{-x}}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 1\n", "\n", "Assume that the parameters of the neural network are as follows: \n", "\n", "\\begin{aligned}\n", "& w_1 = -5 & w_2 = 10 && w_3 = 5 \\\\\n", "& w_4 = -10 & w_5 = 20 && w_6 = 20 \\\\\n", "& b_1 = 25 & b_2 = 40 && b_3 = -30 \n", "\\end{aligned}\n", "\n", "What would be the predicted label for the following data points:\n", "\n", " | x1 | x2 | o | label |\n", " |-------|-------|-----|-------|\n", " | 4 | -4 | | |\n", " |-4 | 4 | | |\n", " | -4 | -4 | | |\n", " | 4 | 4 | | |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can use the following piece of code to evaluate the output of the network:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "def sigmoid(x): \n", " return 1 / (1 + np.exp(-x))\n", "\n", "def nn1(x1, x2, w1, w2, w3, w4, w5, w6, b1, b2, b3):\n", " h1 = sigmoid(w1*x1 + w3*x2 + b1)\n", " h2 = sigmoid(w2*x1 + w4*x2 + b2)\n", " o = sigmoid(w5*h1 + w6*h2 + b3)\n", " return o" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Finding good parameters for our network\n", "\n", "Let's move to a slightly more realistic example. Here we focus on the task of (binary) classification. As always, we first load the data that we want to classify:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "scrolled": true }, "outputs": [], "source": [ "from utilities import load_data, plot_boundaries, plot_data # we wrote some helper functions\n", "X_train, y_train, X_test, y_test = load_data() # to help with data loading" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can plot the data using the helper function `plot_data`:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAD4CAYAAACOqX/yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2de3gUZZ7vv78QLgYwjhBBbt243OTqDImsg+NMxJ0dHHHBdZHZDPJ4zj7BeGaGgAwjZIKKk1EiCM5z1khWZx8uOQM8DDADDq6XZL2dlU1QLqIwByFBuWhghgiCYszv/FFdSXenqrur6q2qt7rfz/P003Slu+rtTvj27/1diZmhUCgUis5k+b0AhUKhkBUlkAqFQmGCEkiFQqEwQQmkQqFQmKAEUqFQKEzI9uOiffv25XA47MelFYqMYM+ePWeYOc/vdQQdXwQyHA6joaHBj0srFBkBETX5vYZ0QG2xFQqFwgQlkAqFQmGCEkiFQqEwwRcfpEKh8J49e/Zck52d/RyAsVDGkU4bgPdaW1v/ZeLEiZ/G/1AJpEKRIWRnZz/Xv3//6/Py8v6alZWlmjAAaGtro+bm5tGnT59+DsCd8T8X8i1CRFcR0RYiOkREHxDRTSLOK5qaAzUIrw4j69EshFeHUXOgxu8lKRReMjYvL+8zJY4dZGVlcV5eXgs0q7oToizIpwG8yMx3E1E3ADmCzuuYmgM1KHu1DE0tTSAQGNrfRlNLE4p3FAMAisYVoe5YHepP1mPR5EV+LlehcJMsJY6diXwmhsaiYwuSiK4EcAuA5wGAmS8z8zmn5xVBzYEaFO8oRlOLlhKmi6POxa8uouzVMszdMRfTN01HwYCC9p/VHatD5VuVnq5XoVDIhYgt9nUAmgH8OxG9S0TPEVHP+CcRUTERNRBRQ3Nzs4DLJqfs1TJc/Opiwuc0tTRh48GNIFD7sbpjdZi5ZWaMYCoUCmecOXOmyxNPPGGruue73/3usDNnznQRvaZkiBDIbADfAlDFzN8E8DmAh+KfxMzVzJzPzPl5ed5UQB1vOZ7S8+ZNmodt92zDzC0zsbRuKWZumYnNd29G4dBCl1cYTCrfqkTdsbqYY8riTkeevRoYMA7ImqjdP3u1k7OdPXu2y/PPP3+N0c9aW1sTvva111470rdv36/tXPfGG28cefjw4W52XitCID8G8DEz74483gJNMH1nSO6QpM+ZPmo6qhqqAAAl+SV47PXHUJJfosQxAQUDCjBzy8x2kVQWdzry7NXA/BBwqhvA0O7nh5yI5IMPPjjoo48+6j5q1KjRc+fOHbRz587ekyZNGjFt2rShI0eOHAMAt91229+MGTPm+mHDho1ZsWJFX/21AwcOHHfq1Knsw4cPd7vuuuvGzJo1KzRs2LAxkydPHn7hwgUyv6ozHAskM58G8BERjYwcmgLgfafnFUHFlArkdI2NF+lb6SzKwpLvLMG2e7Zh892bMX3TdPxm929Qfks5qhqqOllIig7LsXBoITbfvRkzt8zEvdvuxR2/u0NZ3GnHsoHAF3H68EWWdtweK1eu/Hjw4MFfHjp06P01a9Z8DAD79+/v+eSTT5748MMPDwJATU1N48GDBz/Yu3fv+2vWrOl3+vTpTtvq48eP9/jZz3726ZEjRw7m5uZ+vW7dum/YXVMyREWxfwqgJhLBPgrgPkHndUTRuCIAmi/yeMtxDMkdgoopFTjx2QkUDCiI+Q9NINwz5h4sK1yGwnCh2mYboFuO+ucyddhUrN+/HrPHz1afU9px2mRLanbcHuPHj/981KhRl/XHy5cv7/fCCy9cBQCnT5/uevDgwR79+/f/PPo1AwcO/PLb3/72JQD45je/ebGxsbF7/HmffvrpPlVVVf0A4Pjx491vv/324V27duXBgwd/+fLLL3+Y6vqECCQz7wWQL+JcoikaV9QulGbUn6zHtnu2tf8n1y2k+pP16j9+FNGW49RhU7Fh/wbMHj8bu47sarcs5acSQAGA6LXWAagHIGuKVw2AMgDHAQwBUAEg8d+0c/pf1rbVRsfFkZOT06b/e+fOnb1fe+213g0NDYd69+7dduONN468dOlSp11ut27d2tNRunTpwkbPmTdv3tl58+adBTQf5Pr164+NHDnS8tpVuRGARZMXdfrPXTi00FFOZJACGVYS6KMtxx+P/zHWzVjXLprBcEsUAJgJTRQRuZ8ZOZ4qNQDC0P77hCOP3aIGQDGAJmi+wKbIY7eLHJaeAHq0xR7r0aYdt0dubu7Xn3/+uanmnDt3rktubu7XvXv3bnv33Xd77Nu3r1M2jNdkvEC6VV0TlEBGdK4og9sT6M0+h7pjddj6wdZOlqNucctPIYDN0ERxaeR+M2ItykR4LVhlAOJT1S5GjrvJ/X8BVjUB114GCNr9qibtuD369+//9cSJEy8MHz58zNy5cwfF//wf//EfW1pbW2nEiBGjlyxZMmDChAmfG53HS8iPudj5+fksQ8NcXRyicyVzuuagelp10m15KuiiWJJfgqqGqnYRifd/ltWW4Zn6Z9DyRUu7n9TO9fWqoWh/a7LzhFeH2xPpownlhtBY2mj4fnQfZPzjYLEUwGMAygEss/C6MDRRjCcEoNHxqjqTBcDo/yhB67NgDBHtYeYYt9e+ffsaJ0yYcEbs+tKDffv29Z0wYUI4/nhGW5BGieR6dY0ICocWdkodircsy2rL8Os3fo1zX5xLyYIzw6olqGOWK2p0vP5kfYwYBstyjKYOQBU0caxCx3Y7Fcxya1PLubWOWapa8hQ2hXMyWiCtiIMd6o7VoaqhCtNHTUfFGxWgRwn3/eE+FE8sbk9Kf+LNJzq9zo5I2xV7s1xRo+Nu+Gq9R/c5boZmOerb7VRF0mvBqkDn1gY5keMKt8logbQiDlbRt5/FE4vx0ocvoY217VBTSxNWv70aNw+5GY+9/lj78XisirRdsTfKFc3pmoOKKen6H7AesT5H3SeZqhXstWAVAaiGtoWnyH013I9iK4A0FEgrQRc3xUHfjtbsrzG07P54+I8ov6UcWWT8K7Aq0nbFvmhcEaqnVSOUGwKBEMoNCfPBekslOluBdZHj0SxC54BMIVJP8fFDsIqg+TfbIvdB+90El7RqmBsfdIlvaRaPWSK5CHHQt51mFlwbt2FZ4TJ8zV/j12/8OuZndkS6YkqFYcAplfOkkisqP3r6jm4dRm+lRVMEJVKZQVoJpJkf7id/+gkG9BoQ4z+L7v/opjgMyR1iGiUGgIpbNQFzGsV2U+yDQXT6Tgm04IuV9B1ZCGIie/qSVltsM2vt3BfnfMtJTGUbX3FrBf76i7+i7eE2NJY22ha1onFFqJhSgSG5Q3C85TjKXi3zpWu6f0nyhdDE8bHIfdDEERCTyC4nmdruTBrM/G2h3FB7tYfX7cy89PHZTfURjX9J8k7Sd3RS9WW6hdNEdnE8W//s1QNWDhiX9WjWxAErB4x7tj6Y7c6ckFYCmchaM8pJ9IqicUVoLG10bCEmw+28zlSJrtn27gvJafqOjgwWnP+W8LP1z149/6X5oVMXTnVjME5dONVt/kvzQ05EMiPbnclEImtNz0lM53Zmbud1WsH7LyQn6TvRtdX3QSsd9NOCE2EJO2PZ68sGftEa2+7si9Yvspa9rtqdBRqjiGx8SVy6tjMzCwiJyOu0SvwXUmG40OXP2iiAUYjkwqbXVuuWdxOA1QC+j45SRK/FMVqUC+GHSJ++YNzWzOy4XdxqdyaKtLIgzUifErnEyJL0Hf2FtKxwmeTdfsyaQfwR/lhwThPZxdC/l3FbM7PjdjFrd3b48OH3r7/++kuptDtrbW2N2WK3trZi1KhRo0eNGjW6tLR0gJP1pZ0FaYRRKVzhULctGu+RJdUn0ReSfJ+5mfuhDZovU5QFl2pPR7uWsFiW3rL0xPyX5oeit9k9snu0Lb1F/nZn2dnZOHTokJCpBhlhQaaKk9Znqaa2uJ0CIzogZOczCVbNtpn7IRS5F2HB+dXT0T73F9z/l1XfX9V0ba9rLxMI1/a69vKq769qur9AtTtzHVnanUXjtPVZqq3AgtQyzO12cHIQ74MEtNpqkeWDYXjbIk21O7OKaneWBKcpMqmmtviTAmMPWdKG3OeKqH/3gfjaaq9bpClEkXECabbFNYr+AtZSZFJNbfEzJ9MKMqUNuYNuPZ6NOnbJheuono5BJeME0qzKo1/PfobPt5Iik2quZVByMt1sBycHXo0zUD0dg0rGCaTZFnfl3690lCKTampLkFJgZEkbcg+vtr6qp2NQyTiBBIy3uE5rplPNtfQzJ9NqBD19ekWakcrWV1QdturpGEQyMoptNExLVj+gSIIUQfcGowg2ACyBtv2Nr2oJDiqKbQ3Xo9hE1IWI3iWinaLO6QZB2uKKJkgRdA23508bbX2XRI7520knHXHS7gwAli1bds358+c93fWKvNg8AB8IPJ8rZErZoRlBiaB7l1wdv/WtgN+ddOTgl/2AHb1jj+3orR23R6J2Z6mwZs2afhcuXEiqWQMHDhxn9xrxCBFIIhoE4IcAnhNxPjcJVpWHeMwi6E6qiNzBqwhzPH520vG7F2U0ky4C917XIZI7emuPJ8X/UlImvt0ZAJSXl/cbO3bs9SNGjBg9f/78AQDw2WefZX3ve98bNnLkyNHDhw8f82//9m/f+NWvfnXNp59+2vW73/3uiEmTJo0Q8Q5TQVQt9mpoRaS9zZ5ARMXQTAAMGZIuaSLBwqyrUfHEYqx+e3XKs3y8wY/kar876Xg5VycZ084D645qojinGVibpz2edt7uGVeuXPnxHXfccYVeJ71169Yrjxw50mP//v0fMDNuu+22Ybt27er1ySefZPfv3/+r//zP/zwCaJZnnz59vq6qqur32muv/fnaa69N3F1XII4tSCK6A8CnzLwn0fOYuZqZ85k5Py/PthsicMhkmZm5F56pf0bCihk/kqv97qQjTzdxjWnnNXF8+lrt3r44GvHiiy9e+frrr185evTo0WPGjBn94Ycf9jh06FCPb33rW5feeOONK0tKSga++OKLvfr06ZO0k/js2bOH6B18Pv300676v3/xi1/0d7JGERbkZAB3EtHtAHoAuJKINjDzj+2esOZAje8daURgdcqi25h1NWr5osXw+f5WzFTAuEbazRxMGTrpRHcT97oXZTw7emuW47xT2v2U8yJFkplRWlp66uc//3mnyPo777zz/u9///vcsrKyga+88spnK1asOJXoXOvXr2//Yx04cOA4abr5MPNiZh7EzGEAswDUOhVHGeaqiCAotcxyVsxkanK1/93ENXSf47qjwOqTHdvt+MBN6sS3O5s6depn69ev79vS0pIFAMeOHet64sSJ7MbGxq69e/due+CBB/5SWlr6yd69e3MAoGfPnl/rz/UK6RLFgyIqqRCUWma9Yubn3wa+F9aOdVTM+BUkAMQkV7udKiQSUXN1RLA7J9bnqPskd8fXTKZMfLuzu+6667N/+qd/+ktBQcGoESNGjJ4xY8bfnDt3rsuePXuuuOGGG64fNWrU6OXLl1+7dOnSUwAwZ86cM1OnTh3uZZBGukTxrEezwOi8JgKh7eE2g1fIS3h12HQmdmNpo/cLSkDNgRrsPPwgfjP1E/xsVz/cMXIlisYNgP9+MCd40cpMJOJmYqtEcWsEpt2ZnNs9ewSplrloXBF+d/dp5PWsxe/u/hpF4w4j2OII+JcqZJdF6PxZF8KqOCrEIZ1ABklUkhHMWmb/R46KQ4Y+jDLlNiqsIp1ABlNUzBExAsHbVCFZggQikKEPowxzthV2kXJol9Ho1kzF21QhvxOlReNHqlA80bmNJdC+dJJ9nqkO+FK4jXQWpCIWb6P6fidKm2E3Em2WKgSb57OLFbdF8AZ8pTPSCWTlW5WYu2NuTHedumN1mLtjrrDJf0HC21QhGYMETgUjPlUIDs9nBytui6AFltIb6QSyYEABNh3chOmbpqPuWB3qjtVhxqYZ2HhwIwoGpL/fJt7fePUVVxs+L1FU3+3Rst4iWjC8FiCruY0yBJbcIdPbnQmhcGghtt2zDQTCD//PD3HH7+4Ag7H9nu0St+ZKnUQBF6MqovOXz6NrVteYcySL6pvN3QnmF4xowfBagKy6LWQILAG/rP1lvx2HY6tmdhze0fuXtfK3OxOJdAIJaCL5s0k/w6XWS7j41UXMmzTPE3F0O1qcrIzSyN94+evLuLL7lZai+sFrjJsI0YLhtQBZdVvIMeBr0sBJF+/dfu91ukjuOLyj973b771u0kDV7sx36o7V4Te7f4Mrsq8AEaHyrUpc1eMqLLhpQcxz6k/WC+vj6EW0OFHApWhckalf8S+X/oIzi6wVQEQ3xi2/pTyg4giIj0TLENlOhP635m8Ue9rIaefXTV939N7t9143Z8Kc5rX71uatm77u6LSRqt2Zr+g+RwbjhX9+ATt/tBPZWdlY+NJCPPVfT7U/R/SW0Wq02Kqfr/KtyqSzt0VWEQVltGxynDatiE/ULgJQCuAqm+fzAjkGfE0bOe38nAlzmp/e/fS1cybMaXYijkaIbHfmFtIJZP3Jetwz5p52n2Ph0ELs+NEO/HD4D7G0bqlrW0ar0WKrfr6CAQXIIuOPWxdAUVVE6Td3x4lgGCVqVwPYavN8mcOOwzt6r923Nm/epHmn1u5bmxfvk3SK3u7s0KFD7x86dOj948ePvzd//vwz48eP//Kdd955f9y4cZfKysoGLly48NpE56mtre2p93+sqanJFblG6QRy0eRFWDNtTYz4FQ4txI5/3oEFNy1wbZaKFetNtxKj/XwzNs3AXaPuMl1X4dBCPHTzQ52ORwugqCqiTJ+7E4tsTWiDge5zXDd93dHVP1h9Ut9uOxFJt9qd3XrrrZ/rIltUVGTc3NQmUvogjYjfMhaGC4WKZMWUihgfJGBuvenW4+a7N7f7+a7IvgKzxs5KfI1bK/B+8/vYfmg7AK2rj94MONqn6tTnadYYN7h+SKfI1IQ2GOw+sTsn2ueo+yR3n9idY3erHd3u7NZbb21Zs2bNxwcPHuxRUFAwCgBycnLaampqjh06dKj74sWLB2VlZSE7O5ufeeaZJqCj3dk111zz1e7du/8s7t2aI127MyO8mudspZN53bE6TN80Ha1trWBmdOvSDdvu2ZZwPfq6pw6big37N2DF91dgwU0LXHo/4lpnBZtKaHbA4+go9VsMoBXp/DmodmfWMGt3FggLMtGWUaRAWq0Bb21rxcWvLmpR4sgALDORixfBG/rfgIUvLcTe03ux68guF9JwZBoAZRcRNcnZABYCWAFgAbTgjP5YoUhMIARSxi3jxvc2IjsrO2bLn0i040V+wU0LsPf0Xqzfv96lNBw7TRJkIr7ZrV4SCFgTyVZoYvg4gHPQPocVkeMKRWICIZCyMXfHXGw6uKkj0h4uxIxNM3DPmHuwZtoaw9fEi3zdsTrsOrKrXWA/ufAJZo2dFSOUznM9g+x7S1QSaEUg9c/uHDo+hwXmT09v2tra2igrK8t7v5rEtLW1EbSUhk5IF8UOCvFjIYzGRJhhlIaz8eBGzNg0Q3B5oBe9Hd2a+SKyJDCdelw64r3m5ubciCAooIljc3NzLoD3jH4eiCCNjOgCVpJfgqqGKks+xMq3KlEwoKCTtbjxvY3YemirrXMarBDmPkhRlqSbM1/C0LbV8YTQ0ZUnFbz4HOTDKEizZ8+ea7Kzs58DMBbKONJpA/Bea2vrv0ycOPHTTj9lZs9vEydO5HSgvLac8Qi4vLZcwnMuZ+bauGO1keOiCLHxrzgk4NwbmDkn7rw5keNW8OJzkA8ADezD/+10uymBtEnt0VruW9mXy2vLuW9lX649Gv+fUI5zugux8a+YBJ1/A2tiS5F7q+KYuSiBVALpG7qQ6QIW/1iOc3ohLiE2/hWHXLiWwgpKIMXclB/CBm6U8ok9p+i2/WaT+W6CDK25kqMmCyrs4ThIQ0SDAawD0B+aw7OamZ9O9Jp0CNLITRhiAhw6iQIdJ+F3a67kZF6gxihIo7COiDzIVgAPMvM7RNQbwB4iepmZ3xdwboUtRHfNTpZ0LpsgxhP0pHmFXzjeYjPzKWZ+J/Lv8wA+ADDQ6XmDjLdzrI1wo2u2lcl8MhL09Sv8QKgPkojCAL4JYLfBz4qJqIGIGpqbm0VeViqSjVXwBjfa9gc92Tro61f4gTCBJKJeAH4PoJSZP4v/OTNXM3M+M+fn5dkebCY99udYiwwkOO3CHY/VyXyyEfT1K/xCiEASUVdo4ljDzFtFnDOo2J9jbdT5embkuB1Etu23OplPNoK+foVfOBZIIiIAzwP4gJmfcr6kYGN/rozMna+tTubzklRqwWVev0JmRFiQkwHMBnArEe2N3G4XcF6hWB2yZRdnc2VUIMEaovM9vcatRh8KUYiIYr/JWrnGeGa+IXL7k4jFicTqkC27OJsrk6mBBLv+10Qt0WQn6OKeIfhRvuNXqaHctc61zNyXOxorxD9OZ+y+d7drwd0kxG6WaUKVGqpSQ6sUDi1sH7LlxmREZ0QHEnTLKTqQkMyiCnI5nV3/qxv5nl4hOplf4QYZJZB1x+rw1H89hdnjZ6OqoSpmuy3aF2md6ECCHtHWj6cS0RYZBfdDbO34X93I9/SKIIt7BuGH2erHFlvfXq/8vysN7+XabjN3bDPLOfWttp3XJDqPl9t9u2sPaks0Uf0ujYHaYgu5pbVAbti/gUOrQkyPEF/1xFW85NUlzNwhlrO3zuacihzH4rj8zeWdzlF7tJaXv+m0KWs5ax+Zlea5dl5jhCixtXKtzPK/bthfwqFVXZgeAYdWdeEN+0uEnVsJpJhb2m6xaw7U4L7t97WX/J374hxWv70aZbVlqD9Zj5L8Eqzfvx4P3vSgZV9kfK11y5ctSSLkdrasdiLaIqPgXqYcZV4it1aSuhZNLV9rMeyWr1G8Y217Sar/9fwKII1n0oRXh9HU0rnlVxZl4cm/exKPv/m4rdkveq11dDlhTtcclP5tKar3VJuc06zd1l0AZiFWfOoAbASw1eD5iQIXdlt6VULzU5qtQXW/cQOzv89QbggVUyoM/8ZSTxdT7c5EkbYCmfVolumkwb45fdsFLHrCYCoimegP+94J9+Kx1x9D+S3lWFa4LO4ZumBFCw5gTTjrYV79YSZ0iV4Tva7oNUyHVsO9DdbEVpEqZn+fBMKQ3CGGf2N9ruiDM4vOpHR+JZBiSNsttllp31U9rnLUudusprqppQlVDVXtc67jq3aMt6xm6S1rYK00zq446ueNX8MsdIhj9HPc3vJadUXIVIlibe2JSlLN/sbOXjqrttoek7YCaVTyBwAPFDzQyVIsHFqIRZNTq8s1+8POoqyYOdfRPkkNM/+gCF+f0xSf+DVYFWhRWHkfslWiWPsdJCpJTVS3n7wrlEIkaSuQReOKUPq3pcgi7S2GckNY8p0lqN5TbWDdpY7RH3bXrK546OaHElilidptiQisOG10IUuJo5X3IVuZobXfQaKS1ER1+8m7QimE4kfo3Ks0H7fSb6LTh0KrQrxhf7LcNbPZzMUsNr3FTopPKik2XucapvI+ZC0zFJNm1Wd5H8Yj6HQLrQql9HqoNB8ht7QWSPkROdTebt5isjW4kdBsJLj6OqLfRy5rXyJGhNj4zyvkYF1OEZc7umH/Bs6pyIkRx5yKnBS+jDWUQCqBVLTjZqJ1iMUKkZngLmHm7szckzuE8srIYyORNDoPmLkP+1NNI/53YH2n0oESSDG3tE3zySycRLGTkQUYpksRtG7lVgnDfCTtWAAvQGsvugvAYgCPQouqrzF4TQ2AeQDOxh3PgbMRE3Zw83dgHZXmIwYlkIokhJF4xrZVETATXADYAOA/AKwH8B1oAzKTBZuSrS8zUQIphrSNYitEYdQxp3vkuJ2OQYm61fxPANuhieMbAKYieSTeLKrbhM7R+DIA34AceZOKIKAEUpGE+AmJ/QB0A3AY9qprjARX50sA2dAsx9nQLMpkY47MBLcfYvMSywD8GsA5JM6bDHJfTYVolEAqUiB6QuJpAKWwn9iuC64Zf4UmuusArICWU5goL9OsJ+RKxOYlPmHwWqO8SdHTJRVBJi0FUnVCcUKy8j0RSeVF0CxSI/qhQ3QXANiB2BLH+PUB5jPAoyuEzAJK8Vt0madLKjzHj9C5m2k+TvPHMptkOY8iU1nspOlYzcmMzkvMMrgW2DxdyX7Ct5P0HFFApfkIuaWdBVn2allMmygAuPjVRVXDmhLJyvdE9m3Ut9p94o6fhXlNtZXywvjyzocMnmM2nsG+lay3w9P7kDa1NKF4R7HaxQSUtEvzSdRGqu1hO3l7mYTonMdUCCP1NB0r6zPKSywD8AyAFmjBnQp0zpW021dTI1E7vMbS+PfjHirNRwxpZ0EmaiMFAJVvVXZqViHH0C4Z8GOQlJXpflbWFz0ETacCWhCoDZr4GiWSO7OSzZpJqCYTwUSIQBLRD4joMBEdISKjvYxnJGojBQAFAwqSjEfIZPyYEmhF9LxYn5GwJm/1pgcGjXYvgPkXt0JuHAskEXUB8K/QsnpHA/gREY12el67JGojBXS0Ipu5ZSaW1i211E08/YnPeYyOCLuFFdHzY33JifY7GhH9Ba0IFo59kER0E4BHmPnvI48XAwAzP272GhlKDZfWLU0wHkHhLTXQ/IPHYe4blBczvyPQMWMm1VkyolA+SDGI2GIPBPBR1OOPI8diIKJiImogoobm5uakJ3XTV1h3rC7JeASFt0QnojciSOIImPsXCYTG0kYUjStSubkBRYRAksGxTmYpM1czcz4z5+fl5SU9qVu+wughXebjEYKOKpdzAzORSxYYVKk/wUWEQH4MYHDU40EATjo9qVu+wvqT9Y6GdgUDVS4nGjORe+CFB3Dh8oVOz4/2O6rc3OAiwgeZDeDPAKYAOAEtH+Kfmfmg2Wus+CCVr9AuRmNmVSDKLmZ+RgJ1ilz3uaIPnp76dLvf0Y/cXOWDFINjC5KZWwH8BFojvw8AbE4kjlbQfYXTR01HxRsVoEdJ+W9SRsS0xKDhnmvBzM9oJHy9uvWKCcok24Ir5EVIHiQz/4mZRzDz3zCzkHwG3VdYPLEYL334EtpY+w4vQe0AABVOSURBVKZV/hsgNSGQZVKhE6zOvXbPtWBFzOLFNFlurkJepK2k0X2FNftrlP+mE8mEINGYWZlIJIB25l6714nHSOTIMD7ZWUyT5eYq5EX6Wuzg1VZ7ldOXyMcoYj6KyPdhVhf9JICvoo5Fz5IJw/4ohaXQXAvl0L4gxFBzoAZlr5bheMtxDMkdgtuH3461+9bGfIHndM2RQvyUD1IQfrQQstLuLLQq5Gg+sLe4MSI1EWJmMHdG9PswapOWrP2Y3bnX4kavpoIMrc2MgGp3JuQmvQWpp1fI+C3dmTC8GyDlZpQ6DPHvI369Z0yep3fmsbMGZ5140gllQYpBWh+kTrD8N1Y60zjBbR+jG+8jPqpu1lFc99/ZaUwhsl+lQgH5t9jusoG1LR1F7p1uj0KceNsoiuXceetYGzkughCLfx/xW98lnHwbL/r3IxfL31zOtUdjf4+1R2t5+ZvOf49QW2whN+ktSPewEyVNhlftwuy15Eod0e/DyOKthjb8K74zD9AR2S6LXDOYNdrJUK33AoAfqiyHBRlid6y9dLF6RL6PVC1er4Nc/lN7tJb7Vvbl8tpy7lvZt5NFaRcoC1LITfogjXv4MV5AkZgwvAtyyYMb5bQqSCOGDN5i+zFeQJEYr4Jc8pBK6z01JsQ/Mlgg/RgvEDSMShrLAHwDqZf/WSGzvrRSbb2nfJU+4se+Xg4fJHP6+AvdIj7Bewl3/nWK9BHa9UEG8/doJYpt1VcJ5YMUcstwgVQkJzo9J1n1iwisil3mBHbKa8sZj4DLa5NXTimBFHPL4C22IjWiE7zNglcifYRWxy+UAbgYd+xi5Hj6oMaE+IMSSEUSotummf25+OkjTP/ATmaMCZETJZC+YbXXoR/oCd53QbMkjUae+x3YSv/ATmaMCZGTDM6D9BO9iid6axjd6ksGKgF8CGBW5PFMAIsB7ATw3wA+B9APwEr4u2ZvPsv4Vmd+jHK1gsqDFIOyIH3Bit/MrwmFBQC2Rv5dCE0cF0KbyXYFgFoAp+G/oBdBE8P4kkWx4qimEmYmSiB9wYrfzK8JhfHduR8H8GMA6yHfjBt352qrqYSZixJIX7DiN3NvjEByoiPYUwHsgnszbuSd5W02sMvsuCJ9UALpC1arePyaUKhHsGcD2ABtm+3WjBt5Z3mbDey6puc1MY9V+V/6oQTSF6z6zRJNKHQrGh7domwsgBXQttl1cKcRbXJL2a+aZKOBXd27dMfFry6q8r90x4/sdFVJYwWjeS76YzerSNxuymuG+ZwdvdxOL7OLf+wmRrNn3GpVJgKoShohNyWQ0pNIqEJs/BGHvFueUJIP3JJNlKyU/3mJEkgxN0dbbCJ6kogOEdF+ItpGRFeJsWvTAVFBh0Tdw9OpiiS1OTuFQwtRkl+Cx15/DCX5Je3J02bUHKhBeHUYWY9mIbw6LDQ1R5X/pT9OfZAvAxjLzOMB/BmaF18BIDboUAngKcQGHUREaNOpiiS1gVtWRMnN/EVV/pcZOBJIZn6JmVsjD9+GlkWsABAbdHgPWpL1YsSOI7Xr0NcDM03QgjzR+F36Z5fkc3asipKb+Yuq/C8zEFZqSEQ7AGxi5g0mPy+GVhOGIUOGTGxqMmqt75QaaNUox6FZURXwv9JjKbT0nNnQ8gidzrE2Kq0jaOMjQpDjPbtD5VuVKBhQELOtrjtWh/qT9Vg0ufPAsqxHs8AGYzUIhLaH03ushio1FENSgSSiVwD0N/hRGTP/IfKcMgD5AO7iFBTXnVpsGeubdUtRF8Wp0CpRyqH52ewQRibObbFDeHUYTS2dP6tQbgiNpY3eL8hDlECKIekWm5lvY+axBjddHOcAuANAUSri6B6y9QWMDzoshpZsPRvOKlHSKTDjLkb5izldc1AxJVguCDcDTYrEOI1i/wDALwDcyczx6uQxsglHdNChDlqS9QpoSddOKlHSKTDjLkXjilA9rRqh3BAIhFBuCNXTqqXuwhOPapThL458kER0BEB3AGcjh95m5vuTvc6dLXYY8m49K6EFZKJ9jnXQRLSz7ywxmeuDzETsugnUFlsMTqPYw5h5MDPfELklFUf3MKpv7o7YiK5fzQ+SR2hTJ7pMEegQR0D7giiGt8135W0y4Qeit8OqUYa/pFEtdnx9cz8A3QAMiPxcnuYHztHbe4WATlFar/2u8jaZSAWR9d1ubIfNGmWYHVeIJY0EEojtC3gawB/gT5swr5DB7+pnOzbniJw57UbeZboEmoJKmglkPNFtwm4GcB/Ed73xc4spS8DGr3ZsztETvGdumYmldUvbE9GTlTAa4cZ2OB0CTUEmzQVSbxM2HcB2aD46hlhfndEWcxqAbIO1iBZNq30l3SJROzb5sVrfbYZb2+GicUVoLG1E28NtaCxtVOLoIWkskNF5iO8a/FyUr85oi7kMHb0To9ci2i9npa+kF30j3Wqm6y6imk6o7XAa4kcLIW/anUW3CSOTpZDA68X3MUzeuss70rFvpBhE95g06hvpB1DtzoTcMmTsaxju5kjGlxTqQQq9DttJaaEIwpA3R9RfrNZ3BwWVBymGDBFIN+u0o7eY0Z16FkPbZjttTqHjJNk8C53TgQBtW57eTRsyFSWQYkhjH2Q0bs5ONupjuBia1SjSL+ck39BptFt8pF6W+mJZ1iH7mjKV+FBrGlMEd0rwjKy3VgA7Ydz81a4VGR0MsmqVVsDYgk41eKCLc7yVvDnF18eiJ1TrOYN6QjUATyO0sqxD9jVlMgHfYsvY/9Ft7Po1nX5WZn5W68jShkyWdUQjak1qiy2GAG6x9XQVgtY6zI3cRllxkm8YXWXUCOtfJGbJ4Na337LUF8uyjlSurWqv/SFgAqkHW/RvWL/rkL3E73xDM3G27huVpb5YlnWkcm1Ve+0PARNIo6a48bjxTetWkrUVUhtq5Q6JxNl6LbZRQnX3Lt1jEqrtNoywgoyJ3TKuKZMJmECmIn6iv2mjrVY/t/IiW6ZZJZk4W6vFjq8v7tezH7p16YYBvbTOS04aRlhBxjpnGdeUyQQsSBOGccKzjhszaMyuqZKsO3AewNFFsSS/BFUNVbYbRig0VJBGDAGzII2aM+hjT0XmNkYjQ0sxO3jVZUiMb1RUwwiFQiQBE0ijhO/10La+jXAnxUeWlmJW8aqRrRjfqKiGEQqFUPwoAPemWYUo3Gz04DYyNcwwR3TDCIVqViHqFjAL0g/cLFN0m2A0sq0/WR/jc9Sb2Naf9CJCnxxV+pe5BCxI4wcyV+skW5u46pdMJb70D9DSbmSPLKsgjRiUBZkQWVJ8jEi2NqvBExlyPeXDjTkziuCgBDIhRonpslTrJFubleCJzF8E/qJK/zIbJZAJ8SvFJ5UUnWRrs5JYbu+LQOTIVFlRpX+ZjRCBJKKFRMRE1FfE+eTBrxSfVFJ0RK7N3heByJGpsuJH6Z8KCkmE0zA4gMEA/gPavqxvKq8JTpqPnyk+yVJ0RK4txMa/qlDyVUZScspry9M2NcfLOTMb9m/gnIocxiNov+VU5Fi+JlSaj5CbCIHcAmACtExtjwTSy0FRG1gTCorce5n/GD8ILB5Ra7MmtvGCMX3jdMYj4PJas3UqUiW0KhQjjvottCpk6TxKIMXcHG2xiehOACeYeV8Kzy0mogYiamhubk7h7Imiql5ViQDO+ygC9sr+Uun9KGJt+nlSy/XU016aWprAYDS1NGH7oe2YPmq6qoARgAoKyUVSgSSiV4joPYPbP0Dz4i9N5ULMXM3M+cycn5eXl+TZyaKqhQDuAjADsS22APG1xiKwKuh+9H4sgpZHOQSa77EMRlFso7QXAHj31LvYfPfmGJ+k34gKInnpE1RBIblIKpDMfBszj42/ATgKYCiAfUTUCGAQgHeIqL/zZaUSVZ0F4DI6qkQA96xIp1jtmehH78fUUn0SWTiyVcCICCIZWczFO4pdE0nVD1IuhFXSREQyn5nPJHtu8kqaVMaU1kGzIC9HjmcD2A65K0VkmZNtRBiptHWTcY5LIpy2UbP7fp3M2645UIOyV8twvOU4huQOQcWUCstVO6qSRgyS5kEmS2HRt6DbACyEZl22erAuJziZJ+MFqaX6BM3CcdpGza5P0In1WjSuCI2ljWh7uA2NpY1SlzSmO8IEkpnDqViPqWHU9zF6TKm+BQU6RCcbwEYxlxeO3/NkUiG1vMqgdbx22kbNrk9QdzfM3DITS+uWYuaWmaoJcBDxI3SeWppPshQWPU+w1uSxTHiZlmSXILd1M0ZEGzWneYnlteW+pEBBpfkIuUkskMkIgugEDT9zPsWz/M3lncSw9mgtL3/T2t+I3URxP5PolUBmvEAq0hlR4uYXfjcBVgIp5iZpkEaRCSTKUwx6nbfsTYAVKeKHKisLUsGc3MrKhDpvt4CyIJUFqQg2ySK9atKhwm+UQCp8JZEIqkmHCr9RAikFXs2wlg8zEdR9jpvv3oxlhcukq/NWZAZKINu5HcBTcceeihx3Gy+7E8lDIhFMFOTIhE7mCknww/EpZ5BmJWv5fytNHovEKIdzJTP3ZNlnWIvEbiqP3yk0QQAqSCPkpgQyBl0Uv8PuiSOzeRXQbNY+ItV4Nhkqwp0YJZBibmqLHcMCADcDeCNyv8Cl6xi1P1sMYBfkbWYhFyrCrfACJZAxPAXgTQDfidzH+yRFUgitj+VjAKYCeBxyN7OQCxXhVniBEsh2noLWOm0FgNcj9wvhnkhGtz/7PTQL0ssGucElnSPcKgAlFwEWSNGpMa9AE0V9W70AwI+gdTE3movjhPj2ZzuhWZDR78dshrUincv4gl5imXb44fgUE6Rxu92ZUfsvityH2FmnG9WJSGGOiAAUVJBGyC3bb4G2T3SgowTadjXRnBerGM3F0cdA6PNaAHuTBI0sw0LIPS5C4RXRAajyW8pVAMpHArzFBmIDHTcDuA/itsPJxmzGDxHzDy+n7jklSGv1CxWAkocAW5BAR6BjOrSBXTpOLTxAGzVgNMQqGv9nFetT9/RRrPrUPQDSjUEI0lr9IjoAVTi0EIXhQjWuwUcCbEFGBzreNfi5UwvPaC5OPP7PKjaaU33xq4soe1WcdSvK6vNirX4i4nNK5wBUEBE29tUKyce+pkIltFrlQqQ2JtYONdBEtilyruhr5ACohn0LVQxZj2aBDd47gdD2sJP3rhFv9QHaFEM7g7rcXqsdRIxY1c8j6nMSgRr7KoYAW5CL0BHUSG0in3WKoM2EZgDroc2Ipsi9/+II2J+6lyoirT6312oVXdSaWprA4PYtvx3LL92t40wlwAIZTbIxsSLQxbItcu+/OALmc6pvH367kG2x3bnQ8dQcqMGFyxc6HfdzprZIURP1OSnkIk0EsgiaRWfFwkuPHoxGc6rnTJiDtfvWCrGMRFh9uqV29tLZmON9rujj60ztZKJmxacom3WsEINjgSSinxLRYSI6SEQ+qotVCy99ejAWjStCY2kj2h5uQ2NpI/70//4kzDIys1CtWH1GlhoA9OrWy9fodSJRs7r9TuVzUmWEwcORQBJRIYB/ADCemcdAq9ULCEYddUQmmvuHyO2ekYVq1eqTdfuZSNSsbr9T+ZxUGWHwcBTFJqLNAKqZ+RUrrxMTxRbFUmiJ5uXQ6qKDT3h1GE0tnXM4Q7khNJY2Zvx6ojGLYrsVcddFsSS/BFUNVa7lN6oothicbrFHAPgOEe0moteIyPSrkIiKiaiBiBqam5sdXlYU0R110qcHo4htcTqvJ5p494Ru8bnlU1R9LINFUoEkoleI6D2D2z9Aq8T5BoC/BfBzAJuJiIzOw8zVzJzPzPl5eXlC34Q94jvqpE8PRhHb4nReTyq4JeqqjDBgOOl0AeBFAN+LevwhgLxkr5Nj5ILqqKNIzIb9Gzi0KsT0CHFoVYg37HfSwcnbWTpQ3XyE3Jz6IO8HMICZlxLRCACvAhjCSU4qlw9SofCGyrcqUTCgoNPs7/qT9Vg0WWzvT+WDFINTgewG4LcAbgBwGcBCZq5N9jolkAqFuyiBFIOjbj7MfBnAjwWtRaFQKKQiTSppFAqFQjxKIBUKhcIEJZAKhUJhghJIhUKhMMGXhrlE1Izk8wz6AjjjwXLUGtQa0nENIWaWoSIj0PgikKlARA1+pymoNag1qDVkNmqLrVAoFCYogVQoFAoTZBbIar8XALUGHbUGDbWGDENaH6RCoVD4jcwWpEKhUPiKEkiFQqEwQXqBlGUoGBEtJCImor4+XPtJIjpERPuJaBsRXeXhtX8Q+fyPENFDXl036vqDiaiOiD6I/A3M83oNkXV0IaJ3iWinH9ePrOEqItoS+Vv4gIhu8mstmYLUAinLUDAiGgzg7wD4NWXqZQBjmXk8gD8DWOzFRYmoC4B/BTAVwGgAPyKi0V5cO4pWAA8y8/XQOtf/Lx/WAADzAHzgw3WjeRrAi8w8CsAECdaT9kgtkABKADzBzF8CADN/6tM6VgFYBBhMcfIAZn6JmVsjD98GMMijS98I4AgzH420ttsI7QvLM5j5FDO/E/n3eWiiMNDLNRDRIAA/BPCcl9eNW8OVAG4B8DygtRpk5nN+rSdTkF0gUx4K5hZEdCeAE8y8z+trm/A/AOzy6FoDAXwU9fhjeCxO0RBRGMA3Aez2+NKroX1B2h9n6JzrADQD+PfIVv85Iurp43oyAkcNc0VARK8A6G/wozLEDgUrgDYU7LpkIx0Er2EJgO+LvJ7VNTDzHyLPKYO25TSeXu/CsgyO+WJFE1EvAL8HUMrMn3l43TsAfMrMe4joe15d14BsAN8C8FNm3k1ETwN4CNpIToVL+C6QzHyb2c+IqATA1ogg/jcRtUEr1hc6N9ZsDUQ0DsBQAPsiwxoHAXiHiG5k5tNerCFqLXMA3AFgiugviAR8DGBw1ONBAE56dO12iKgrNHGsYeatHl9+MoA7ieh2AD0AXElEG5jZ6076HwP4mJl163kLNIFUuIjsW+ztAG4FgMhQsG7wsJsKMx9g5muYOczMYWh/pN8SLY7JIKIfAPgFgDuZ+aKHl64HMJyIhkbmD80C8EcPr4/IGOHnAXzAzE95eW0AYObFzDwo8vufBaDWB3FE5G/uIyIaGTk0BcD7Xq8j0/DdgkzCbwH8lojegzYUbI6H1pNM/G8A3QG8HLFk32bm+92+KDO3EtFPAPwHgC4AfsvMB92+bhyTAcwGcICI9kaOLWHmP3m8Dhn4KYCayJfVUQD3+byetEeVGioUCoUJsm+xFQqFwjeUQCoUCoUJSiAVCoXCBCWQCoVCYYISSIVCoTBBCaRCoVCYoARSoVAoTPj/A/jxh5OQaGwAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plot_data(X_train, y_train, X_test, y_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see, this data is not linearly separable. In other words, the positive and negative examples cannot be separated using a linear classifier. Our goal for the rest of this notebook-session is to learn the parameters of a neural-network model which can separate the positives from the negative examples.\n", "\n", "What do we mean by *learning the parameters*? Remember that our neural network has 9 parameters including three biases ($w_1, \\ldots, w_6, b_1, b_2, b_3$). Every different assignment of values to these parameters leads to a different classifier. We want to find the one which matches our data the best.\n", "\n", "Let's see how different choices of parameters changes the classifier. For a given set of parameters, the function `plot_boundaries` shows the regions of positive prediction (coloured blue) and negative prediction (coloured red):" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQAAAAD8CAYAAACYVXqwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAOm0lEQVR4nO3dX4xkZZnH8e9jI7vZcWaHBAgJPWQgi7gsYmBbds1k/wkaXMl4sxeYaIhedNasZEwkCEy836wblajJpgN44yTsBnExhlWHKCZ7wSzDyIgwaAgh0ioBEnEIiJOZefaiq01XdVVXddWpc07V+/1cTXW/deqBqfrNe55+zunITCSV6W1NFyCpOQaAVDADQCqYASAVzACQCmYASAWrJAAiYndEPBARz0bEiYh4XxXHlTRd51R0nLuB72bmP0XEucCfVHRcSVMUkw4CRcQu4DhwWTpVJM2UKnYAlwGvAF+PiPcATwAHMvONjYsiYhlYBtixY8dfvuvyy7uPsrDQ9fDMmc0v1Pu1UdacPbt5Te/X+sXWKGtGibuq1ozC+NW63/zmBd5449UYtq6KHcAS8BiwLzOPRMTdwMnM/Pyg5yxdc00effTRrq/lrj/tenzy5ObnvfYaQ9f89rfdj3/3u81r3nyz+/Fbb21ec+rU1o8BTp/uftwvbHrX9FvX73nDnjPumqbNQo3z4CtfWWJ19ejQAKiiCbgKrGbmkc7jB4BrKziupCmbOAAy8yXgxYi4ovOl64FnJj2upOmr6qcAtwKHOj8BeB74REXHlTRFlQRAZj4JLI38hDNnNp3Q956s7OrpCcyCfuf75/T5P9xvXa/ec+W39dmrjbOmabNQY0mcBJQKZgBIBTMApIJV1QTcnjNn+v8Qf4N+P8Cc177AOD0B2Hw+Pe6aps1CjfPKHYBUMANAKpgBIBXMAJAK1lwTsPeqnRHMw7AQbG76OSzUbRZqnBfuAKSCGQBSwQwAqWDN9ADOnu1/p45tcliom8NC2i53AFLBDACpYAaAVDADQCpYc03A3lvzVsRhoW4OC2kr7gCkghkAUsEMAKlgzfQAMvv/Op4pcFioW1XDQoPWNclhoe1zByAVzACQCmYASAUzAKSCNTcI1O/3bdfEYaFu4w4Ctb3pNguNy6ZVtgOIiIWI+HFEfKeqY0qaripPAQ4AJyo8nqQpqyQAImIR+DBwTxXHk1SPqnoAXwZuB3aOtDqz0R5AL4eFuo07CDQL59xt71vUbeIdQETcBLycmU8MWbccEUcj4ugrr78+6ctKqkAVpwD7gP0R8QJwP/D+iPhG76LMXMnMpcxcumDnaBsFSdM1cQBk5p2ZuZiZe4GbgR9k5scmrkzS1DkIJBWs0kGgzHwUeHSEhaN1oxrksFC3eb3l+Cw0LqfJHYBUMANAKpgBIBWsmYuBYPOJ1oz1BGA2+wJVDQuBdxyeB+4ApIIZAFLBDACpYAaAVLDmbgs+bEKl5U1BcFio17wOC8Fs1DgOdwBSwQwAqWAGgFQwA0AqWHsnAcftRDWopGnBQet6OS3Ybu4ApIIZAFLBDACpYM0NAg07YarysrUGldQXcFio+3Eba+zlDkAqmAEgFcwAkApmAEgFa24QqNcoHZM5GBYCryLs5bBQc9wBSAUzAKSCGQBSwdpzMdA4UxQOC7WKw0LDta1GdwBSwSYOgIjYExE/jIgTEfF0RByoojBJ01fFKcBp4LOZeSwidgJPRMThzHymgmNLmqKJdwCZ+evMPNb58+vACeDiSY8rafoqbQJGxF7gGuBIn+8tA8sAl5x33uYnVzXp4bBQqzgstLWma6ysCRgR7wC+CXwmM0/2fj8zVzJzKTOXLtixo6qXlTSBSgIgIt7O2of/UGY+WMUxJU1fFT8FCOBe4ERmfnHykiTVpYoewD7g48BTEfFk52t3ZebDA5/R745A05r0cFioVRwWGq7OGicOgMz8X/q/PyW1nJOAUsEMAKlgBoBUsPbeEWiakx4OC7WKw0Jbm2aN7gCkghkAUsEMAKlg7ekB9Kpz0sNhoVZp27DQoHVNqmpYyB2AVDADQCqYASAVzACQCtbeJmA/Dgttm8NC3cZ9e7T9KsJ+/x0jPa/aMiTNEgNAKpgBIBVstnoAvRwW2jaHhbqNOwg0C8NCo3AHIBXMAJAKZgBIBTMApILNdhOwH4eFts1hoW7zfMvxXu4ApIIZAFLBDACpYPPXA+jVtmGhQetaxGGhzeb2jsNNFyCpOVX9evAbI+JnEfFcRNxRxTElTV8Vvx58Afga8CHgSuCjEXHlpMeVNH1V7ACuA57LzOcz8xRwP/CRCo4racqqaAJeDLy44fEq8Fe9iyJiGVgGuGT37gpedgJNDguBVxE2yGGhblXsAPq9N3LTFzJXMnMpM5cu2LGjgpeVNKkqAmAV2LPh8SLwqwqOK2nKqgiAx4HLI+LSiDgXuBn4dgXHlTRlE/cAMvN0RHwa+B6wANyXmU9PXJmkqatkEjAzHwYeruJYjai7o+NVhK1R5fDmLE4LOgkoFcwAkApmAEgFa+5qwFFOhprksNC2zeuwEEzvluNNDwu17FMnqU4GgFQwA0AqmAEgFaw9twSbtaYgjNfRGbSul8NCrTKtqwibHhZq2adMUp0MAKlgBoBUsPb0AHqNej7dpHFP1qZ1y/EZ7AnAbPYF5mVYqGWfKEl1MgCkghkAUsEMAKlg7W0C9jMPw0L91jks1GUWm4LQrmGh6Ndt7aNlnyBJdTIApIIZAFLBZqsH0GsWh4VgepMeDgu1StPDQqNo2adFUp0MAKlgBoBUMANAKthsNwH7afuwEDR7y3GHhRpV57DQKCb6dETEFyLi2Yj4SUR8KyJ2T3I8SfWa9J/Hw8BVmXk18HPgzslLklSXiQIgM7+fmesblseAxclLklSXKnsAnwT+c9A3I2IZWAa4ZHeNZwoOC3VzWKhVpjksNIqhARARjwAX9fnWwcx8qLPmIHAaODToOJm5AqwALC0u5ljVSqrU0ADIzBu2+n5E3ALcBFyfmX6wpRky0SlARNwIfA74u8x8s5qSJNVl0pPhrwI7gcMR8WRE/EcFNUmqyUQ7gMz8s6oKqZXDQt0cFmqVKv46vCOQpKEMAKlgBoBUsPm7GGgcDgt1G2U6ZdC6FnFYaLiWvcsl1ckAkApmAEgFMwCkgtkEHMRhoW5zehXhLDYFYfiwkINAkoYyAKSCGQBSwQwAqWA2AUfltOBmc3AV4TxPC46iZe9gSXUyAKSCGQBSwewBTMJhoW5zOiwEs9cXcBBI0lAGgFQwA0AqmAEgFcwmYJVKGhYatK7XHAwLwexdRWgTUNJQBoBUMANAKpg9gGmbxb7AqOf707rl+Az2BKBdfYFR32KVvBMj4raIyIg4v4rjSarHxAEQEXuADwC/mLwcSXWqYgfwJeB2ICs4lqQaTRQAEbEf+GVmHq+oHkk1GtoEjIhHgIv6fOsgcBfwwVFeKCKWgWWAS3bv3kaJc6jtVxGOOwjksFCXJpuCow4CDQ2AzLyh/wvEu4FLgeOx9mqLwLGIuC4zX+pznBVgBWBpcdHTBakFxv4xYGY+BVy4/jgiXgCWMvPVCuqSVIOW7T0l1amyQaDM3FvVsYozi8NCML07DjssNLFaB4EkzSYDQCqYASAVzACQCubVgG3V9mEhaPaW4w4LbckmoKShDACpYAaAVDB7ALPCYaFuDgttaWFhtHUtewdJqpMBIBXMAJAKZgBIBbMJOMscFurmsNAf2ASUNJQBIBXMAJAKZg9gnjgs1K3gYSF7AJKGMgCkghkAUsEMAKlgNgHnncNC3QoZFrIJKGkoA0AqmAEgFcweQGkcFuo2yrDQoHUtsmlY6MyZkZ7Xsr95SXWaOAAi4taI+FlEPB0R/1ZFUZLqMdEpQET8A/AR4OrM/H1EXFhNWZLqMOkO4FPAv2bm7wEy8+XJS5JUl8jM8Z8c8STwEHAj8BZwW2Y+PmDtMrDceXgV8NOxX7h65wOvNl3EBm2rB9pXk/Vs7YrM3Dls0dBTgIh4BLioz7cOdp5/HvDXwHuB/4qIy7JPqmTmCrDSOebRzFwa9tp1sZ7h2laT9WwtIo6Osm5oAGTmDVu8yKeABzsf+P+LiLOsJeEroxYqqTmT9gD+G3g/QES8EziXdm2DJG1h0kGg+4D7IuKnwCngln7b/z5WJnzdqlnPcG2ryXq2NlI9EzUBJc02JwGlghkAUsEaDYA2jhFHxG0RkRFxfsN1fCEino2In0TEtyJid0N13Nj5O3ouIu5oooYNteyJiB9GxInOe+ZAk/Wsi4iFiPhxRHyn6VoAImJ3RDzQef+ciIj3DVrbWAD0jBH/BfDvTdWyLiL2AB8AftF0LcBh4KrMvBr4OXBn3QVExALwNeBDwJXARyPiyrrr2OA08NnM/HPWZk/+peF61h0ATjRdxAZ3A9/NzHcB72GL2prcAbRxjPhLwO1A453RzPx+Zq5fg/oYsNhAGdcBz2Xm85l5CriftdBuRGb+OjOPdf78Omtv7IubqgcgIhaBDwP3NFnHuojYBfwtcC9AZp7KzNcGrW8yAN4J/E1EHImIH0XEexushYjYD/wyM483WccAnwT+p4HXvRh4ccPjVRr+wK2LiL3ANcCRZivhy6z9ozHCzQdqcRlrg3hf75yW3BMROwYtnuoNQaoaI66pnruAD07rtbdbT2Y+1FlzkLWt76E6a+vo90tpGt8dRcQ7gG8Cn8nMkw3WcRPwcmY+ERF/31QdPc4BrgVuzcwjEXE3cAfw+UGLp6ZtY8SD6omIdwOXAscjAta228ci4rrMfKnuejbUdQtwE3D9NINxC6vAng2PF4FfNVDHH0TE21n78B/KzAebrAXYB+yPiH8E/hjYFRHfyMyPNVjTKrCames7owdYC4C+mjwFaM0YcWY+lZkXZubezNzL2v/Ea6f54R8mIm4EPgfsz8w3GyrjceDyiLg0Is4Fbga+3VAtxFo63wucyMwvNlXHusy8MzMXO++Zm4EfNPzhp/OefTEiruh86XrgmUHrm7wn4LhjxKX4KvBHwOHOruSxzPznOgvIzNMR8Wnge8ACcF9mPl1nDT32AR8Hnupcig5wV2Y+3GBNbXQrcKgT2s8Dnxi00FFgqWBOAkoFMwCkghkAUsEMAKlgBoBUMANAKpgBIBXs/wHwBxmqi9eiQgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "w1 = 1; w2 = 1; w3 = 1; w4 = 1; w5 = 1; w6 = 1\n", "b1 = 0; b2 = 0; b3 = -1\n", "plot_boundaries(w1, w2, w3, w4, w5, w6, b1, b2, b3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let's project the plot of data on these decision boundaries:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUkAAAD8CAYAAAD6+lbaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydeVxU59n3v2eGYVhkcQOUVUXFfUE0UYuexCSSaGKbZtLnpUlqFyJp3yxt49OU4NtHQ/s82vRN2qYYuqRJOn3TydI00WizjRI1UXCPBqNGQBBQVBYZGIaZ8/5xHGCYMyzDwAw438/HD8zhPmcOOPOb+76u6/5dgiRJ+PHjx48fZVTevgE/fvz48WX8IunHjx8/3eAXST9+/PjpBr9I+vHjx083+EXSjx8/frrBL5J+/Pjx0w0eEUlBECIFQXhDEIQSQRC+EAThZk9c148fP368TYCHrvM8sFOSpG8KghAIhHjoun78+PHjVYT+FpMLghAOHAUmSv7KdD9+/AwzPDGTnAhcAl4SBGEOcBB4TJKkps6DBEHIArIAQkNDU1MmT3a8ilrt8NBqdX6irsesVjhdfxyLrbXbG1QJagQERgbGcLW1mmjtRIJVYTTbGqk2f0W0ZiLB6jCHc2w2x2soyX9vPhI8NaY3+D+i/Ni5erWUpqZawdv3MRzwxExyAfAZsESSpP2CIDwPNEiSlOvqnAXz5knFu3Y5HJPCIxweNzQ4n1dXh9OYOW+pkOj5d8iMy+WhhI0cqTfyTImOlWOy2Vmbz5NJBiZrRKfxra3dPwZoa3N83FVYlcYojVM6r6dz3B3jbYbCPQ4Hfve7BVRUFPtF0gN4InFTAVRIkrT/+uM3gPkeuG6viAlO6HFMiDqCbdX5HKk3MjdCZOWYbAw1m1g5JptZYc4C6adndlk2c8ZqdDh2xmpkl2Wzl+7Ij5+Bod8iKUlSNXBeEISp1w/dCpzs73V7y6Mz8ghSu84TaQQt/3viC+RMNZB3SscbF37Dztp8dNG57KzN53ij0eW5fpyxi2OcKg29WccZq5EzViNvmB9Gb9YRp0rz9i368eNRPJXd/t+A/npm+ytgrYeu2yN3JWQC8NsTOVQ3lxMWMAoBqG+7zEhNNFlJz3LrWHnM/XFP8Ur5BnImvMusMJFZI0S2lOp4bLyBGaH+GWVvsItjptZAptbAK+Y1WGlDTQAPat8mWe3/O/oZXnhEJCVJOgIs6PUJVqtTgLFr8CS8S4yyO/5jZib/MVMWwq6xzPr6ju9VAW38z5x3mRYsv5EXhYg8HWTg8ytFpAb1/82tFH8MUPgLK43rStfYnUphzu/OmP6SrBbJ1BrQm3XcFJCNlTYsmEgPyO2VQA7GPfpR5uDBg1EBAQF/Ambi30hixwZ83tbW9v3U1NSLSgM8NZMcEmQmrgegubnj2NwIkSkKiRs/rklWi9wUkM1HbZvQEMytAbl81pbPJLU4JGeSCxdupro6jfLyjntPSDASE1PEgQPrvXhnfWPaND3LluUQHl7Oa69pnH4eEBDwp5iYmGljx469qlKp/LUQgM1mEy5dujS9urr6T8DdSmP8nyYeYkfjZk6ZHeObp8xG3m/y/UTGIYueXzYnsd6k4pfNSRyy6Lsdf8ZqZG/bb9EQjJpAJnWaXXZN5gwFqqvTuOceHQkJ8r0nJBi55x4d1dV9j69Om6Zn3bok1q9XsW5dEtOmdf+39BTTpunJyMgiIqIMQZBQqxXL4maOHTu2wS+QHahUKmns2LH1yLNr5TGDeD9e4/0qPd/cm0T6Ryq+uTeJjy55/oU7QZPGH+t07UJ5ymzkj3U6EjW+ncg4ZNHzpiWLOqkMkKiTynjTkuVSKM9YjejNOuao72etdjsPav+J3qwDIFNroMJWNIh37xnKy0X+9S8D99yjY+nSDdxzj45//cvgMLPsDV2FKiKijIyMrEERymXLctBoTD0NU/kF0pnrfxOXWuid5bbVqlwI2QmlAq++xCntbC/Xs+VUFi1W+QVU01LGc2ezCNTA7eMy+3w9V8wOFFmHga1XdCwPzWZXUz7rRhtIDnB+o3WNU7oTowTH+N7BVj3bzTnUSeVECglkBOaRqsl0igF2vc7OthwsOL65LJjY2ZbDfI3z36fCVkSm1uCwrLaL43LNereW2z3d42BQXi5y+HA2S5ZsYu/e3D4LJCgLlUZjYtmyHL74wnOvNSXCw8sH9Po3MsN+JvnbEzntAmnHbDNR8FWOx58rJUhkeWg22xo3sTw0m5QgkX9fU1iGtxr5c91/8HRtEo9cVPF/6pIoMrs/2zjYqsfQ4jgbfMOcxcEels0AdZLym8vVcSUhTFaLLNcMndidEgkJRubNy2fv3lzmzctvX3r3BVdCNRgC1tDQc72wt6mtrVX/93//91h3zl22bFlybW2tuueRnmfYi2R1s/IL9GKL51+4JS1GdjXlMy9oDdsb8/h+hcCHTb/hD3X3OCzDt9bfw9HWf3LFJovaVVsZrzVluS2U75mVZ4M7Wnv+IIgUlN9cro4PR+wxyH/9y8CePRvbl959FUpXQjUYArZ7dx4Wi6d9ZbaOgvGzQJUqf906qj9Xu3z5svrPf/5zlNLP2npYTu3evfvMmDFjFDYr98zChQunnjp1KtCdc+EGEElXO3Kigjz7wj3ZbGTrFR3pIVmcML+PhLxmbJRqsEqt/KHuHt5p3MAf63QEEkIbZofzLZjY1uze7PZqH2eDnVkZkIemi2mThhBWBuS5dS9DkZiYIocYpD1GGRPTt/iqklBZLCHs3j3wf8svvshkx44C6usTkSQBq9VtTbjO1lHwRCJUBYKE/PWJxP4I5U9+8pO48+fPa1NSUqY//PDDcdu2bQtbtGjRlNWrV0+YOnXqDIAVK1ZMmjFjxrTk5OQZv/71r8fYz42NjZ1VVVUVcOrUqcCJEyfO+Na3vpWYnJw8Y8mSJZOvXbs2oNsvh71IKu3I0apCyJro2RfuOXMR60YZ+KxZT6vkOKtrw4wgqXivaRPpIdk0SorlWFy1uTe7HdmP2eB8TSb3agqIFBIBgUghkXs1BYrxyOHKgQPrnWKQ5eVin8t/ugpVfX0iO3YUDHg8svPzb91ayubNNi5dmtXPq22MhZYu+tCiko+7x7PPPlsRHx9vLikpOfniiy9WABw7dix0y5YtlWfPnj0BoNfrS0+cOPHFkSNHTr744ovR1dXVTkvs8vLyoEcfffTimTNnTkRERFhfeeWVke7eU2/wXuKmc5V3LxEAfcnr5OzdSHljBfERCfzyljz+1yzXL8L/mJlJcDBsOZTDhaZyxocm8KNpee07dTzF16PW09oKV2qVha6FelaF5bKrKZ8wIYpGqcZpzChVQntSpy/JnDu1eRhashyW3BpCuEvr/EGgVMw9X5PpU6LoqYJzb9Q/fvFF5qCJ4sBS7WIq6uq4e8yePbspJSWlvV7pf/7nf6K3b98eCVBdXa05ceJEUExMjIOjWGxsrHnx4sXNAPPmzTOVlpZqu173+eefH52fnx8NUF5err3zzjsnazQaKT4+3vzBBx+c7cs9Dqlicn3J62R9+CimNrkavLy+jKxtWQDdCuWaSZmsmdTx8x4S6/1idEACl9vKnI6HqaJZE7GRFK3I7y7fQwBahyV3ICHcHere7DY1UP7d3jPncFUqZ6SQwJ3aPFIDM2/o3Sz2+kf7Urpz7NFPT8S0yktspeOeIyQkpP0Vum3btrDdu3eHFRcXl4SFhdkWLlw4tbm52ekjMzAwsL2MSa1WS0pjHnvsscuPPfbYZZBjkq+++uq5qVOnunXvQ2q5nbN3Y7tA2jFZTPxox48wnnMMsheeN/J88eAXct83Mo9AwXF5H4CW+yOeBeQM+COR/2Ku9uuMUslL3FGqRP5XWAELg92fgaQGZpIbVspvwm3khpW2C+eNjKfqH32RhQs3OyWWEhKMLFzoqdf8hkoI6vIRG2STj7tHRESEtampyaXm1NXVqSMiIqxhYWG2w4cPBx09ejTU3efyJENKJMsbKxSP17XUcf+bunahLDxvZO17OuZFD34h9+KwTL47poDRAdcFUJ3Id0b+mZtCO0Rrqlbk+yP/H7+MKmVrjI1nxpT2SyDtHGzVs6kxiR83qNjUmMTB1sHZ7eEKX7BT61z/ePhw9rAQSPDsLiFl1l2B/1sG41rlQNe4VvnxuivuXjEmJsaampp6bfLkyTMefvjhuK4/v/fee+vb2tqEKVOmTP/5z38+fs6cOU1K1xls+m266w4LpkyRil94wfFgcLDj4wjnwvGkf2dQ1uT8QZYYFs9Lt7+A7r21rFvwCFsP5vOPew2IE8Rem/d2pWvItLnZeYypywaHlhbnMUpmvV2P9caYt6cxxWY9/2h2jkvqggocZpWDad5r351jLz7v+rg/9PYe7eJx+HA28+bluz2T9MX93d39bkqmu0ePHi2dM2dOrVdu1sc5evTomDlz5iQp/WxIzSTz5j9JiNpRTEMCgslbsgExPp3s2d/lmU82sS41G3HC8Jgx9JbtLcq1ku+ZPV8031s6Owb9u3WDxwSyt3iq/hEGY+bWd4brLNnXGFIimTlpDQWLf0ViaCwCAolh8RSs+C2ZKfdhPF9I/rG/8PTXctl6MN8pRjnccVUr6er4YNHZMeimgOxBdQnqb/1jZ7OKO+9cy+HDWT4V3/TELiE/PTOkstsgC2XmpDXyg8hIAIznC9G9txbDnS+xfMZqxCSR+9/U8Y97DaSOvjE+XUcKCVyVnLPqrmooB4szViOfteV7xU5NaRlcXi72StjsZhX2vdgREWUsXPgcX311e7/2d3uKzrNk++/kC8I9HBlSM0lXFNUcxnDnS4jx6QCIE0T+ca+BogtDz5HGXe4KUt45c6dCreRg0TkGeUfgxiFlp+bKrGLy5Hd8YubmqV1CfnrGOzNJm80569EP1ic/JH9TV9fuHnTL6PncMno+UrjHnmZQ6Zqo6cnh3J4d397SUSt5V1CeU5H4YDqcd3UMsscoK2xF/Z5NDrTDuStTCkGwsWfPxgGZuXU2zW1oSGD37jyXhen9mSX76RtDbrntCV4v0bNxbw4VjeXEhSXwk7l5DsXmPaEv28y0sLT2NhAAR+qNfH6liG9Ee88NZ4E2kwVax9/Dm8XkSs5AyUPEvbyhIYGICOfwRUNDIuA4c/OEMCkt7zMy5I0Sw2MHz9BlWCy3+8Lfj+t59MMszjeWISFxvrGMp/Zl8fbZ3tcUTgtLY8PnOo7Uy8utI/VG8k7pmBzinOl85/JmTjQ5LstONhvZXuf7juUABy16nmnqvWv5cKE3ZhXu7O92RXdelMMFv1XaEOHnH+fQ3Ob4Ymy2mthyqPcvxvmjRDbOlFvUvly+gbxTOnKmGhR7eE8KSuO5C7p2oTzRZOSFGh0TtL7tWA6yQL5h7r1r+XBisM0qvOlF6YqtRVtHjX92/CzVf6lSxz87ftbWoqFpldZfvLPcliTlyusBoKuH0vl65RfdhaZye7K8V4gRImuuZvNy6SYeSsrl5hhRseA8FZH1gQa2lOpYOSabnbX5PB7b9xa2venE6AmH885jdrS6di1foPW9feGedDjvS3zQE7he3nunOmFr0dZRT7z/RGJLm+wEVHWtKvCJ959IBFiX5t6um85WacuWLWtYvXp1/aZNm8ZFRUVZTp48GXL27NkTK1asmFRVVRVoNptV69atq/npT39aC7JVWnFx8RcNDQ2qjIyMyQsXLrxWXFw8Ijo6uvXf//73mREjRgzYrphhOZPcfHwrxqp9DseM5wvZXPw8CWFOu6EAiAvr24vxwCUjb1fm81BSLm9X5nPoiutM56wwkZVjsjHUbGLlmOwh0+O7r67lwwVv9KrxphelEhsLN8baBdJOS1uLamPhjWeVNixFMm3MbHS7ftQulPY6yrToeeQt2UCIxvHFGBwQwoYlvX8xHrhk5Mn9OjbONPD9SRvZONPgEKPsyvFGIztr89FF57KzNt8pRumr3Kiu5d6ID3rbi7Ir1deULdFcHXcXJau0qVOnTk9NTZ1mt0rrek5vrNI8ybDMbovjFmNY/nt0u35Edt33yD/2F4c6SoKD+fnHOZyvLyc+IoHcm/O4L6X3L8bPrxaxZZGBqYHyjNAeozx+pYi5EY6zxOONRraU6ngySY5ZzhohsrlUx+Pj+77kHmwyAvN4w+y8HzwjcHi7lnsrPuhLXpQxI2Jaq645W6XFjPB9q7S2tjZmzpw5HWDlypV1zz333IX+3OOwFEmQhTI75dts2r+F3EVPdggksvdkZ//JvvpLfneKnNHsbIIxf5ToUBJk57SpqF0gQV56Pz7ewNmWIp8XydTrNZY7Wp27MA5nfC0+6A02pG+o7ByTBAgKCLJtSPd9q7SAgABKSkpOunufTtfz1IX6hM2mbI/jQYyXDpD/xavkznmU/CN/RoychzhuMeCczHGnVW1v+fYE5xKRVERS6b9A9rXg/MPmzcSr0pis6Xju0xYjZW1F3KLtuM/OCZA0bSZpLpI0vtAKtjvcLTjfvTvPoWYRoK1N6xAf9LYD0EBjT85sLNwYW32tOjBmREzrhvQNle4mbcDRKu2WW26pX716tYPX1r333ltfUFAwdsqUKdMnTZrU4itWaR4TSUEQ1EAxUClJ0ipPXdcdjJcOoNv/JIZFWxAnrUCMuQndrh9hWP77dqG8EUkISOOlazq+E2JgskbktMXIX006Hgz2fafulBQ96ekd2ebCwjxKSgZmRmtf8tqz201NUWg0JpqaxgPcMA7n69LWXemPKCrx7rvvnuv8eNWqVY3274ODg6XCwsLTSudVVlYeBxg3bhynT58+YT++ceNG5z4oHsaTM8nHgC8Ar28ELLr6uSyQYxcCHTHKotpjN7RITtGIfCfEwF9NOpYEZrO3NZ/vhBiY5OM7YFJS9Kxc6bgbZeVKeTfKQApl5/igp3wp/Qw9PJLdFgQhDrgL+JMnrtdf1k/5brtA2hHHLWb9rHVeuiPfYbJGZElgNu+bN7EkMNth6e2rpKcrZ5vT0wdvN4rfu/HGxVMzyeeA9UBYr0ZL0oDHJPuCUtPegYxTDhS9KTg/2Wxkb2s+dwTlssecz5RAeS91sVnvYI5hbyRmZzCNMbriyWyzu3HUrt6N3jKT8EWH9OFOv2eSgiCsAi5KknSwh3FZgiAUC4JQfKmxsbuhfgaIU61GXrqmY+0IA3eFbGTtCAMvXdOxzZTDP5qzrvtRSlyVyjC0ZHm9R44dV1nlwco2e9LhvL/4okP6cMcTy+0lwN2CIJQCrwG3CILwt66DJEkqkCRpgSRJC8aG9W7CORTZXq7njh1JzHlLxR07kni/yjeEBqDMUsTaEQamXF9iT9GIrB1hYG/rH3yu9UNnCguVd6MUFg5OvaYveTcO5w6Qvkq/l9uSJD0FPAUgCMJy4KeSJH27u3OOWypQVXyfBPUo8sK/QWboTf29DZ9ge7me/zqcRYtVFpyq5jI2l8gJhtvHeb+28PbQ9U5L8ikakWbqFcd7u/WDHXtyZrCy213xNe/GzvFRbzuk3wh4ZVtiq2RFAsqsV8iqewV902feuA2P89sTOe0CacdsM1HwlW/MyFzhqsWDt1s/dKakJJOCglJ+/WsbBQWlgyaQvshQ7W3TH6s0gI0bN0Y1NjYOumZ5tJhckqRdwK6+nGOSWsmpf8vnZpPuFJxXNyvPvC62lCt1yB0UelNwviokj9eanLcf3hWU157ocLeY3FcLzruru/SVe1Ri8HrbPB0Ni0ywulMC4d0w2B8Cz7hVm2i3SvvZz352yZ3zX3zxxegf/OAHV8LCwrr9H4qNjZ1lr6v0BD5hcFFu82i9qtcYH6o884oJ9p0ZmRJp2ky+FVrASFUiTy6G1ROiuT+4oN3lPC7OyIIFQ8MkuDfY6y47u/ysXJlFSorvxI9dMXjx0UUmeHCiLIwgf31wonzcPTpbpT388MNxALm5udEzZ86cNmXKlOlPPPHEeICGhgbV8uXLk6dOnTp98uTJM/74xz+OfOaZZ6IuXryoWbZs2ZRFixZN8cRv2Ft8Yu92gqpfXp4+w5Pz83hqXxbNnZbcQeoQHp3h+4YQ9u2HsY1GfvFNHTt2jKeiQhbIjAwd27cPn90l3dVd+voyfvDio6sb4ZWvZGF86BK8PFZ+vNrt0pRnn322YtWqVcH2fdVvvfVW+JkzZ4KOHTv2hSRJrFixInnHjh0jampqAmJiYiy7du06A/IMdPTo0db8/Pzo3bt3fzlu3LheOKd6Dq/PJEMIJG/EPd6+DY+wZlImv1pcQGxoIgICsaGJ/J95BdyV4NtvvM5UVors2GEgI0PHTTdtICNDx44dBioqhk9ywBddwH2T1Y2yQD4/Tv7qvkAqsXPnzvDCwsLw6dOnT58xY8b0s2fPBpWUlATNnz+/+ZNPPgnPzs6O3blz54jRo0f36Ej+wAMPJKSkpExPSUmZfvHiRY39+//8z/+M6e99emUmGSiosWAlQTWKvNC7yQxK652tthfpbcH5d1Iz+U5qhyjW1Q3gTblBbwrOL1wQ+fzzbBYt2sSBA7lcuCB6rJh8oLsc9oaeXH4G8x59uzj83TB5BvlYlfz11kZPCqUkSTz++ONVTz75ZG3Xnx06dOjkm2++GZGTkxP74YcfNvz617+u6u5ar776avsnXGxs7CxPugB5ZSY5KyAWW9QfKB3zDJnBC3s+4Qblo0t6vn0wiTv2qfjBiSR2XxmcmFlsrJFZs/I5cCCXWbPyiY0dGtnT3uLtusvO+G5xuD0G+cpX8NyFjqX3u24XOXe1SsvIyGh49dVXx9TX16sAzp07p6msrAwoLS3VhIWF2R555JErjz/+eM2RI0dCAEJDQ632sYOJT8Qk/Tjz0SU9z53NwmyTY2eXLGX84bxcc7ls1MAt38ePN3L77fISu7JSpKJCJCNDx3vvDZ8lt7frLjvTuTi8r+YZA9uHZ3+IYwzSHqPcH+LubLKrVdqLL75YceLEiaC0tLQUkA149Xr9uZKSEu1TTz0Vp1KpCAgIkP7whz+UATz00EO1GRkZk6Oioiz79+//0kO/aI8IkjRg/XNcMjUhWjr11C8AMLae4rWWYiYFRLF+xB0dg5RqVQIDu38MENTF7T0kxHlMcLDjY6X6nPAuZkYKXcKkLsttJfNepeV213H1CrXc9+5J4qLZeUk4VpPIH2eUAs691JS2w3c9prTc7ryUnDt3M1VVaVRWdrxJY2ONREUVcfDgepfnucJTYzozULZp3iwLWrp0Q3tx+J49G3sc37VPN8izYXvLh9/9bgEVFcUOUaKjR4+Wzpkzx2lp6weOHj06Zs6cOUlKP/PKTPJM2yWMracA+Hp9AZIk8fbIbG/cis9yyaycRKi1DGxy4cgR5105lZUi58/7xixyoGzTvGHHZscd84zu+vD4SguI4YJXRFIQ4K66PyAAAah5O+z7iJrJjh/XPp7IAc85nG8v1/PbEzlUN5cTE5zAozPyiApKoKZFYSapTVCcHNt55/JmJgWlObSGONls5Jy5iLsilRMBfXU4d8VgFJwPVPlOT9cdqKJ4d4vD/Rn6wcMriZsoIYxmLJiw8FjQMkTNoNaGDjqvl+iZ9eckRj6nYtafk3j7bEcC5u2z8n7vquYyJCSqmsv4r8NZ3DzqTrQqRzXUqkJYm9B9cmFSUBrPXdC1d2Q82WzkhRodE7TeTgR4hoESB2+JjrvF4d52RrqR8IpIXpQaCUZDCBqeb9mN0TJoMVgA9Jc+Iungt1Htu4Okg99GX/X+gD3X6yV6Hv0wi/ONsgiebyzjqX1Z7UK55ZDzfu8Wq4lPr7zH+pQCooPkmssobSKPTyrg1rHdz5ZmhMqNxp67oMNwaQMv1Oj4YbSB6QpNyoYiAyUO3hKdAwfWO80Yy8vFHst/fK1P93DGKyIpSbA97GG2hT2MIEBG41Z+0/Shwxij+RSbr/3b48+tv7KbrLPPUWa+iIREmfkiWSWbB0woN+7NobnNUQSbrSa2HJJNLy40ud7vffu4TN5YUkrhrTb+llrao0DamREqcltkNm9d3sQt4dnDRiBh4Mp3fKksqDf4Wp/u4YxXRDJZPQZRMwVRM4V/jvg+twVMZUPTtvZkjrH1FLq6P5KmSfL4c+dU/Q2TzexwzGQzk/NVgctzNn/5F4xV+xyOGav2sbn4edfn7N2M8ZyRikZlEbSLo6v93lFB7s9gTjQZ+aAun2+MzuXjhnxONg+fOseSkkx27nQUh507C/qdXBmo6w4kX3yRydatpWzebGPr1lK/QA4QXknchAlB7ZFvUZ2MOCIZo/UMuvo/ka1dSr55D4aIgUnmlFuUKyDKWy5Cc7Piz9I0E9AZf9jeXKy9G6P4gkONT+dEzsLIaejeuI+xoVFcbHI2TYkLSyAyEn7xtTwe/cBxv3ewOoTHZ+a55Rx0vNHI8xd0rL/e63tGiMhzF3Q8Pt7Q6z7fvdmV42pcVwZiV05JSWa/xGvhQrnMqXPGPj7eSHh4JQUFpb26hi/sHBpq1NbWqv/0pz+NctcFaOPGjVFPPPFEbU8uQJ7G63u37YiaKWRrl7Kp5d9ka5ciBk4dkOdJCBitfFyrbHO3uUyOHRoWbUG3/0k2nHyBr3/6ON8Yf0u3nRfF+HQMd75Es8VEgMpRYYIDQtiwRF7G3ZfivN/7V4vd3+992lTEk9cFEjpilGdbBt9F21epqkrj7rt1xMfLM+z4eCN3362jqmp4JLc8wdMfPx397inH3TXvnno37OmPn45295p2qzR3z3/xxRejr127Nuia5TMiabR8Sb55D7lBd5Bv3tO+9PY0eWO/RYigdTgWotKSl7BWcXxa2DR0n28AIHuijk0lL9Jqa+Nb8Rk9PpcYn85jix6nzdZGuDYCAYGEiER+u6KA+1I6RHDNpEz23lfKue/Y2HtfKWsmuT9L+kb0+naBtDMjVOTu0d7eB+w7nD8v8s47Bu6+W8eSJRu4+24d77xj8JlaUF9gUewi04NvPzjRLpTvnno37MG3H5y4KNZvleYVjJbT6Ex/xTBi7fVY5WR09X/GEPE9j88oMyOWgkZDTtXfKLfUkqAZQ17Sd8kce6vieHHUfAwzN7Lm08dpk9oIVgURqOrdn814vpCtB0oFPlEAACAASURBVPO5JelWiquK+PCBjxAniDQ0QOF5I4drinhsgV+8vMH58yJHjmSzePEm9u3L9QtkF1ZPXd34yppXvnrw7QcnPjTnoUsvH3157CtrXvlq9dQbzyrNOyIpSQ4BnKK2Mgwh30FUJ4PNhqhOxjDiOxSZzyGqJnWc1zUw5maMMjNkEZmTFnUcCAoCUzcfkK1m2mwWTDYzuUkPIY6cj+6zn2DgWaf+3naMVfvQ7foRhlV/BeDr736br/9jDW+v/htSyAi+u0PHP+41OO1+7C+JiZtpaEjj6tWON/2YMUYiI4s4c8Z9Qe5NnNJXCs57w223PUxKymvs25fL3Ln57WU448a5777jqy7s7rJ66urGh+Y8dOn5/c+Pe2zRY1X9EUglOlulAZhMJlVJSUnQrbfe2piTkxOfnZ0de88999SvXLnymieft6/4xHJ7vfYWOUnTCVEzhfXBK7x0R468VruLACGA3KSHyK98GwDDzI0UXf3c5TlFtccwLP89Ynw6Ynw6/1z9NwQg78Cz3P+mLJDiBM/PXhoa0pg9W8fIkXK8bcwYIwsW6KirG/rxtpQUPd//fhI//rGKrKwkt53E4+ONpKS8BgiUl8tL7zVr1rBmzdf9cclOvHvq3bCXj7489rFFj1W9fPTlsV1jlP3FbpVWUlJysqSk5GR5efnnTzzxRO3s2bPNhw4dOjlr1qzmnJyc2J/+9KfjurvOxx9/HGr3j9Tr9R5vlOITy21fxlh/hLcu7+Xt2b9EHDVfnkV+vgHDzI2sn/Jdl+etn7XO4bEYn86j8x5m0/4tPP213AERSICrV0WOHTMwe7aOiopsYmPzKS42UFs7tJeTKSl6br/dM3urx40r4u235Q+7u+/WceRINiBw6tT9/mX3dewxSPsS+9YJtzZ2fuzONZWs0n7xi1+Mz8rKuhIREWE7d+6cJjAwULJYLEJUVFTbI488ciUsLMz28ssvj4YOq7Rx4xw185ZbbmnypH9kV/wi2QNbKl/nqbj7EUfNB+QY5VOJD7Cl/O99Ejrj+ULyj/2F3EVPkn8wHzFJpOhCEWnj00gd3XEde6zyoWT3l8ZXr4pUVGQzceImTp3KHfICCbB0qef2bHdeTneOS+7d27P7zo3C/sr9IZ0F0R6j3F+5P8RdkfRbpfWBBfHxUvGPf+x4sGtAR6kQrWsQrDdjlOzUuh7raq8G7RZrxvoj6E7lYZi1EXHUfIxXDrXPJJ1EUinAGBmJ8XwhuvfWYrjzJcT4dD6+fIj739TxsyVP8d97f8VfMgykx4sUnjey9j0dL91pYHaXDLWSDVtXizV7mefIkUaXM8mu9mrgbKemZLnWncVacrKem27KYcSIchobE9i3L4/Tp5Utxty1Tvvxj1UIgvNrVZIEfv1r94J/9tKfI0eymTs33+MZbm/GJP1WaX3D56zSFOnNK8pTdjV9QNRMwZD4U3THN5Ads4r86m0YpuYgBk9TNoJUoKh0H4b03yGGzYa6Om4ZPR9Dxl8oqjmMIeMv6HboWJeazdaD+Ri+aWjPgLtDRISRlBQdJSUG6utFrlwRSUvTceyYwSGZ4wna2mDSJD3LlnUsg8PDy7j11izUajhzJtNjBefdtVzoru2tK+wCaRfG8nLR46VA/oLz4YFPJG58HTFsFtkxq9hUoSc7ZhVixNw+nb9+1jqnwnMxPp31Cx5DjE9nXWo2z3yyiXWp2f2OVY4YUdQukNARowwPH5hi8kWLlJfBN92U49Hn2bNHeW/1nj3u7a0eN67IQRDttZPjxvmL7v044jszSR/G2Hic/Opt5MZlkl+9DTFiTp+F0uW1r9dSPv21XLZej1X2RygrK51jmVevih6fRdoZMUJ5b7qr4+5ijzsuXdrhHL5nj/uO5EplPufP+4658ABhs9lsgkqlGvwYmw9js9kEwOUc3y+SPWBsPI6udAuGlKcRI+YiRsyRY5RTcxCDb+7fta/HKv/xzdcRJ4iISWJ7eVDnZI4vc+1aAmFhzsvga9c8bzHW3z3bfvj80qVL08eOHVvvF0oZm80mXLp0KQJwWc/nPZHsKRDlTowSPFZwbqeo4QsM4x+TjYFNpvYYZdGVz/s9m2yPVY6eDw31HbHKc5/0azYZEaEnJiYHjaYcszmB0tI8amsHRlyKi/NYutS518r+/XmoVEO74Hwg8OY9trW1fb+6uvpP1dXVM/GH2uzYgM/b2tq+72qA97Lbjz/ueFApyt0VH82At9O1wRgMapOxhgZZIOPislCpOkTLag3h9OkCamszFY2Oum42UsqAJydv5vLlNGpqOsR79GgjY8YU0dQUy4IFcnb72rUE9u/P4+xZ5ex2T43IlB4r4YkxrtyA+rPrpq8MlEgqZbf9uEe/P00EQYgXBMEoCMIXgiCcEAThMU/cmB/3iInJcRBIALXaRFJS/xIply+nkZ6uIzpa3skTHW1EFHXU1qbx1VeZGAylFBTY+PvfS9sF0tfxuwH56Q2eWG63AT+RJOmQIAhhwEFBED6QJGnAKuD9uEajUU6YaLX9S6TU1IgUFhpIT9fx5ZfZTJmSz8cfG6iqGhqxUyU6uwENVK2kn6FPv2eSkiRVSZJ06Pr3jcAXQGx/rzuUaO+Z81E6SXu/OaA9c3rCYlFOmJjN/U+k1NSIfPllNrNnb+LLL7OHtEDa6ewGdORItl8g/Tjh0cSNIAhJwDxgv8LPsoAsgISRI51PdsfCWolBLjjXX9lN1vk/YJLklhBlLTVklWwGS6tL+7Xe4G672vr6PEaPdoxJ2mwhXLzontN5Z6KijEyZks+JE7lMmZJPdbXoEKNUYqDa1XrC4RzkJfbcufkObkCDKZT+gnPfx2MZLkEQRgBvAo9LkuSUZpAkqUCSpAWSJC0YGxrqqaf1OjlVf2sXSDsmm5mc8pd6PnnMXyD0gOMx7T4I3+r2/ZhMmVy+XEBbm9yrpbU1kYqKAurr+xcnjIoycvPNOj791MDnn2/k008NDjHKoUjnXTd7925sX3rbY5R+/ICHRFIQBA2yQOolSXrLE9ccKrjsmWPuRRuP5pmQ8KQsjCB/HfsjMM/u1z2ZTJlUVpZSXm7j1KnSfgskwKhRRXz6qYGLF+VZ1sWLcoxy9Oihu0PFv+vGT2/o93JbEAQB+DPwhSRJv+n/LQ0tEjRjKLM4C6KrnjkONC2E8i2Q+CNo/DaE/Q0u/R7Mi0GhmsiblJQ4l8TU1PS83PZlbtBdN376iCdikkuAB4DjgiAcuX7s55IkvefyjC7O5MDAVQsPcMF53pj7yaouwCR1WOeECFryojO7dzu3Uz8VQu+D6N9CzcNwaSbgXBSpVPDW2zilLzHQDudTp+pZujSHsDDlrYv+gnM/fcUT2e09kiQJkiTNliRp7vV/rgVyANls/hij5bTDMaPlSzY3fzhgz5kZsZSCmCwSA8YgIJCoGUtB/CNkjlrWuwtEHILRBlkgRxucY5Q3GHPnbiY21jEmGBtrJDV1c4/nTp2q57bbsggPL0MQJCIiyrj99iy3Hcx9gZQUPevWJbF+vYp165KYNm3o/i5DlWG1NSlNHY/O9Nd2oTRaTqO79hJpAZ7fR9yZzIillCb/Htvcf1I64499E8iUDfKS++IP5a8JT97QQnnxYhoZGbp2oYyNNZKRoaOmpucCb1fGvEuXetaRaLBISdGzcmUWEREdop+RkeUXykFmWImkGDAZQ8h30Jn+yobm9xw6MPokI76Ako1ybBI6YpTBrnvnDHcuXBDZscNARoaORYs2kJGhY8cOAxUVPccJw8KUC+bDwz3rSDRYpKcri/6yZUNT9Icqw0okAUTNZLIDl7DJ/D7ZgUt8VyABKjOhXm4L0V4O1LQQaq/3zul1OdBmwHGJqlYbCQzseYnqi1RWihw/ns3ChZs4fjybysreJVIaG5VXDA0NA7uSGChciftQFf2hiu9YpXmoWtjYeorfmHfxQEAq+a17EQMmI2omY7ScpshazvqgW73icN4zEyDlJ/LMsn6+LJhjnpRnlk3KNuXtyRz1NAi5j/Dw1wERWTB1gEHJO6NHYmM3c+1aWrtxL8gtIcLDiygr87zxQ9c/dUKCkVmz8ikuzmXWrHyqqsT2lq/dsWdPHrfd5uxItHdvnkv3cl8u5u7Ojd3P4DGsZpLGttPoml9ho3YlO6wlPBV4KzrTX/lNixGd6a+kqX34xVU/XxbIlA2Q8Cc5Nlm+pWMp3h3WdDC9hCyMG7ALpCyYfefatTRSUnRERMiz04gIuWdOQ8PAGz+MH2/k9tt1vP++gaKijbz/voHbb9c5JXOUOHUqkw8+KKChQS6kb2hI5IMPCjh1amgYbnSlsFDZjb2w0D03dj/u4TszSTfRtx4kx/we5dJVIgjmEc1ifqxdzjx1LLrmV8gImEZuy3tsC81y6u3tDpsbd5CmmYAYlNJ+zNh8kiLzOdZHfb1/F6+fD1VrIOFlOdvdG4G0Y00HshGETUhSLu4KJEB9vUhJiYGUFB1VVdmMG5fP0aOe75GjxNixRbz/voELF+TnunBB5P33DURHF/Vq2X3qVOaQFcWulJRksuvyXl6/UkBFo5W4MDX3jXqImEvD4/cbKgzpmaS+9SBrW/4fZdJVJKCOZp6zFJLTvJ0i63myNYt51VLMT7SiRwQSIE0zAd2VrRhbSgBZIHU1L5CmndAxKOUdiDrheOKY45Dcw2akiEMw7m0of6jv5UDqQiD/ukDm0zVG2Vfq60WqqrJJSNhEVVX2oAgkwNGj69sF0s6FCyKHDg2Ov6Mvccii53dlL3O+0YoEnG+08ruylzlk8We3BxPfnUn2opg8x7wdS5fWFCYs/LflY7ZoV5Nv2Udu4G1ybFI9CTHAhVB2U+Gsbz5ATtM7lNuukKAaRd6IezBE/ADd5a1km5eT37QLw6h1iOrkjl6sNQmQ/hwU/hCuzpUFM/U5+PRxZUdbgJFHYGoeHM2Rz6mf7xij7I7r2xmFpr/KM0p1GoTcB6aXCA9f3e2pgYGb0WrTMJs7hCk01EhExGtERLxFTU0u48fnU18vOsQoB5KBLjjv6xhX4waanW05WHDMblswsbMth/kaeTZ5yKJnZ1sOdVI5kSRwR0Ae8wL8M01PMqRnkuVSneJxGxK/av0IQ/CDbAzKwBD8ILrmVzC2nVYc7wp98wGyGv9Ome0KElBmu0JWg54L1jqyQ9LZ1LiN7NDlDktvAGqmywKZ/gLMNMDN1wXy4gx5ljnmuOP4kUcg8XU4dl0goSNGOeKLnm9Ue0zezmhNlx/bY5Tqwz2earWmMXasDq1WnnlqtUYSE9cQGfkPyssNXLy4kfJyg0OM0s/gUCcpZ7Htxw9Z9LxpyaJOKgMk6ijjH20P8Lb5kUG8y+HPkBbJBEE5dRtJMIbgB9tnjmLAZAzBD1JkPd+n6+c0vYOJVodjJlr5SeMb5JsKyQ1bRX7TrvaltwM10+HLW2DGW3D2NlkgAa5MggVbOoRyzHGYnQdl93UIpJ36+XKZkCvCt8qzyIZ18n5vkJfdgc/LQtnas0m81Spy6ZKBsWN1RERsYOxYHXV136Ks7J80Nckzx6YmOUY5YoRvGz9MmaJn7dokHn1Uxdq1SUyd6jvL0oULNzu5C8XHG1m40HWZVqSgnGi0H1eaaYLEZ9JWTNKVft2vnw6GtEjmBWYQgsbp+COaxU5LazFgMuu1t/Tp+uU25RdajdSIIfIHbIxYg2HUOocYZTvRJ2HKx3DiGzDpg44Y5cUZUPykLJQpf5e/dp5B9gXzbNk1yO4ipC6EkLVgnde3y5hFGhuziYzcRGNjNhcuvNgukHbq60XFdrUDyZw5mxk/3lFYxo83Mneus7BMnqxnxYqOLYnh4WXcdluWzwilO60iVgbkocExu60hhJUBcnbb1UwTJBqkSo/ct58hLpKZmlQe13wN1fWKwUQhkp9rbqWg7bM+L62VSFCNUjweLYQhaqcCIAalYBi1jiLLuU4DTspL7cIfwuc6eal983MdQlk7C0pXwlSD/NUdgQR59njp97JQRvxGFkjTSx3L7l6i1RoJC8unri6XsLB8QkN9Y1l96VIaK1bo2oXSXh508aKzsCxe7NtbEju3iliyZEO7j2V3jkPzNZncqykgUkgEBCKFRO7VFLTHI13NNAGsXVZAftzHdxM3SihUAkcIwXwY9DBiQHL74RXqyRS1lSOqJvXL4Twv+C6yml7DhKX9cAiBPDviXvka17MHYkCy/Pz2xE3kafj4YahKhsBWqJgsC+boL+Xvo05A4o7rs8wdsmjWzurzn0NmJgjXXYTqHoX62UBHrLZnh3MjYWE6TCYDgiDS3CySmKjj0iWDQzLHG1y8KGI0GlixQsfJk9lMn57Phx8aqK4WnRI5rrYkhoWVO7wE3C0m94QzT+dWEfv25fbKkm2+JrNdFLuyMiCP1ywPAM4dT9WCQgdQP24xpGeSAOsDRQeBBFm01gf2/w2eqU2jIPRbJKpGIgCJqlEUhP0vMoN7qF88ngFVCsmck3fJs0x7Isc+y+wco+wroQc6XITC/tax9O41RZhMBqxW+e9lj1FqtY7xx4gIPVOnJjFzpoqpU5OIiBicZWxVlUhJSTapqZs4eTLbqTzIzrVryrMqV1sVvUHXVhH9dUCfr8nkJtU6un4Uaggh/MZqMzWgDK2ZpBfI1KaRqU1T3rroDqPPdWS6oSNGGXm677PJ0AOOO3OkZfLS227c2yvWY7U6HjGbRYdZZEiI3qFvTmBgGXFxWZhMUFvbv3KThAQ9s2blEBpaTlNTAocP56HVVlJbm0ZVlci4cUZSUvL58ssHmD37N1y4ICoK5Wef5bF8ufKWRF+gc6uI8+flbZa9WXL3xDeC/kCSZUlHGZCQwMqAPD4R/q8H7/7GRpAk56n6QLMgLk4qfswD7bmV1kqeGtP1mJJI9mZMoMKyp+uxoCDnMSGOAXuCFazKJ70pt4Cw78wJD5dnktpjcsYb6Lp5W1Iw6m3osjW8rktlVWxsEgEBznuIW1oSKS4ubX/c3Ox8i119hzuXiSYk6ElLyyIgoGNQW1sIx48/TkpKAZcu3UR09CccPryBOXN+xeHDT5GaupGzZ+9n9+4XnZ5rwgQ9ixfLhrstLaOQJAgOvkJjYwJ79+bx5ZeZvVome2pMZxYu3ExVVZqDIMbHGxk3rkjRIb2/PP/8AioqipW8mv30kaE9k/RUtbAPOJy7xdl7r39T3+ngzOv/ejDF6ERPDudqtete3v3pwDhnTo6DQAIEBJiYPFnPJ58YuOWWDAQB5s//Lz788O3rIyRUKuXPtXPnMjl3LpPkZL3DrDI8vIwVK7IQBJy2LLpbTN7XULe/VcTQZcjHJP0MPJKkHNez9/gODTUyZkzfbdmCg5XFNzS0jODgC5w8uR612oxK1ca4cUZEUcdHH73N3r3Os8jO3HSTcqZ78WLfyHT7GVr4RdJPj7S05GGzOS7/bTYt1dV5hIYaSUjQ0dzcd4eg5mZl8RUEWLz4e0yb9hzHjuUCEvPmbaKkJJuqqp5nXiNGuMp0lxEX55gsiYszkpHxH3zve0k8/riK733Pt4rQ/Xgfv0j66RGLxbGXt9UaDQSi1Z4iIUFHebnBqfi8N5w8mUdbW4jiz9RqM21tIVRXi9hsgVgswUyf/lvGjes5I+wq020yRZORoWsXyrg4I3fffQ/Jyf/02SJ0P97HL5J+ekXnXt4VFdU0NDxOdPQmLl/OdksgASorMzlypABXucOgoBrS03Xs2vVPPvhgO199dT+iqOtRKD/7TNmH8ZNPnm1vDXHTTXJriLa2EAICzA5jXRWhp6Upby1MSxuaDvB+eseQTdzoLYfIadtJuVRHghBJXsBKMjXzPeZw7lYyxycczvuOQzJH8zrh4c8A5UACkAc4JjvUaiMhIfIOnTFj8pEk0e3C86tXM2luziEkxDl7brFE8umnso9laytcvixy/vy3iI4uUlx22//UX32ViSDAokU5jBhRzrVrCXz2WR7nzmXS1gaff57NokWbOHAgl7S0ZxTvq2txukoFNTVprFqlY/t2A+XlIvHxRlat0rFtm6F9TGd8xeHcT/8YkiKptxwiy/Jm+06YMqmOLMubALJQ+nEPzesQ/CiCYK/lKUOSsq5/LwulLJDyDp36epGWFpGxY+UdOu4a/Z48mcfcuc6lQAcP/p6LFx2vWVMjUlPTcWziRD0LFnSI4f79eZw9m8nZs5mcPu1cwxkbK7eGOHBAbg1hMkURGlrjNE6pCL2iQmT7dgN33aXj6NFs5szJZ9s29+ocHSzOrtc2utpZ48e7DMnldk7bToetgiD7SOa07fTSHQ0TgjaC4FjsKAgmoGPpqVY77tAxm5V36PQF+7LbZEpEksBmU6NWm5g9O4eEBNexwYkT9SxdmkVYmBxPDAsrY9myLCZNUj5n/Hhje/fF/fs3smOHAY3GRFub1mGcxRLCnj3KRegVFSLHjmVz882bOHo0222BdLA4k8p405LlN9P1UYakSLrykXR13E8vESpc/KBj6dnaur5dIO2YzSINDf0riK6szOTkyTys1hBUKiuCIJcCpaVluRTKBQuUS30WLVIu9YmKKmLHDkN7G4jKSpF33/0XZ858vdd9ceLijMyenc+nn+YyZ457Wwu7M9P143sMyeV2ghBJmYIg2v0lN1t2kaaKk93CrweGjNYzFNkqWK9Z7t2Cc1fjfIHQ8RDgbLElSHGEhw/808+YoVxcPmdODuXlzqLlqtRnxIhyAgKc/8zHjq3HZnP8L1Ha5mizKf/3x8UZuesuOSZZUSFSUSG2xygrKsReh7p7MtP141sMyZlkXsBKJx/JEDTkBawEIE0Vh86sx2g9A8gCqTPrSVPFDfq9DimuPgm2LtsfpWBo2TAoTx8UpCwSrorOm5qUS31clQD1l5iYonZBhI4YZUxM30INPZnp+vEtPCKSgiCsFAThlCAIZwRB+JknrtkdmZr5FGjuJVGIlN15hEgKNPe2J21EdTIGbSY6s54Nrf9GZ9Zj0GbKM0s/rjGtgcu/Als8SIL8tfm3YLlvUJ6+pUVZJFwVnR8+7FxnabGEUFw8MKYWxcXr2wXSTkWFSHFxz6GGgxY9zzQlsd6kwixdQ43j/v3OZrp+fIt+L7cFQVADLwC3ARVAkSAI70iSdLK/1+6OTM38bjPZojqZ7ICb2NT2EbkBt/oFsreY1kDgd7zy1GfO5DF9ehZqtWOW++RJZfEoLZWX4PPmyS5C164lUFycx1df+VaW+KBFzxvmrPY4ZDOXUaEhhNGYuOLPbvs4nohJLgTOSJL0FYAgCK8B9wADKpI9YbSeIb/tM3IDbiW/7TO5W6JfKH2a6mpZJCZNyiE4uJzm5gROnsyjsps+P6Wlme1i2eqjZtw7Wp0TNTYsBAoj+EVwrZfuyk9v8YRIxgKdO2xVAIu6DhIEIQvIAogcE4TResZBtBwSK/3EHoO0L7FF9STHJbc3C87B++5BbuCOe5A71Ndnsnevoyh2dY3rD56q/+9Lrs+fqBnaeCImqfT+cdpoJklSgSRJCyRJWhClHTGgiZUiW4VDDNIeoyyyuSpx8eNn4OhtouaQRc8vm+W45S+bk/x1kz6CJ0SyAojv9DgOuNDdCWFoBzSxsl6z3OlaojrZI7NUnyf5LRd9vQ3euZ8biIOtejY1JvHjBhWbGpM4eF3kMgKVux5mBHbEWv0F5r6LJ0SyCJgsCMIEQRACgW8B7/R0UufESnbATf54oaeom+zYM2fkEbmvd8MU797XMOdgqx5DSxZXr4vcVamMN8xZvNn8SHtMUkANQKSQyDe1BaR2StT4C8x9l36LpCRJbcCPgH8DXwAGSZJO9HRe18SKfentp5/UznLs6z07z/2+3sOAWbM2O7kGjR9vZM4czzr3vGdWFrlPrVuvzw5Bwto+g0ztksn2xy19F4/suJEk6T3gvd6Ob8TcvsS+YGtADdxiLiCaUJ7VrB58k4rBbAMBA+8eVDEZIlfADAOc0kHlFOjyBvYEPber9T51dWnccouOwkIDNTViu8O50WhQ3JUD7v13XHUpZo7heQsmdrTmtIuk/SUUKSS0i2ln/AXm3scrO26apNZ2gcyyvEkNTQDU0ESW5U30lkPeuC3fZ9YOGFfieCz6JEzf7ngs6gRM+kDu65200/12tT5GTIyepUuTWLFCxW23JREb23O8rqZGpLDQQHq6jjlzNrQLZG8czvvCyD6ImdLs0FXc0l9g7n28IpIxQhiiOtnv5tNXaieAuLVDKKNPQvoLcHlCx5ioE459ve1Lbx8Wyq49vceMcRa/mBg906dnERwsO/6EhJQxd25Wr4Xyyy+zmT279y0g+spdQc4ip1z4oTw7TNVk8k1tAZFCIiAQKSRyr6bAX2DuA3jV4GJYuPlMPQhL34Owq9A4EvbcCadSB+a5qlLAuE4WypLlMG03FP4QaqZ3jBl11rGvtz1G2de+3hHbIea3oKkGSwzU/6e8G6efhIdvxmxOazfplXt6fw+VSnYHDwwsY/Jk2cOyc0/v5OQch504IJtfTJ+e022xOUB0tJEpU/I5diyXlJR8qqpEjwvlAm0mkiTHJq9K5YwUEpimvpOitpcdYpVds9qdSdVkkqrpXdtbP4OH90TSZiOBSMpw7ebjVXpTTD6lCG57AzTXZ8PhV+G266U2dqH0dMH5+WQ4mQ6p2+DwKvkxnbaaHFt5/ZtOxyomy//o1PS6O8a8D7GbQX29rUFgFYz+mdxYu/4ul6f1puC8qSmNiAhduydlWNhP2gXSjlptYuLEHCyWDvHrzvyiu2LzqCgjN9+s49NPDVy8KFJdLTrEKJVwt/4/LSiTtKCOe7bZYELrEgfhvFObR2qg45iu+B3OfQuvugDlBdzRrZuPz7N0R4dA2tFY5JnlQDH+FEwvhIN3Qsou5xilJ0gq6BBIO6oWeWbZT6xWEZPJQEiIDq12A4Lg7AoOoNE4iqLZrBzzc2WKLs2O/gAAIABJREFUYWfUqKJ2gYSOGOXo0e6bBPeF1MBMcsNK+U24jdywUgeB9DM08KpIZgbMoyDgGySi7Obj84S7CAuEXR2Y5xt/Clb8ET78ARTf3bH09rRQai8qH9dUe+TyVqtIa2s2QUGbAOWMuL2nt53SUtmQ1/E6IZw5031io6RkvWILiJMn+2cS7OfGweumu5kB88gMmKe8nPV1GiIhQkEoG0cOzPONLZUF8sJU+bE9RjnmnPy9pzBHQZDCDM8S45HLq9VGAgPzaWnJRat9DptN67DkttlCqK52FD97fDIpKQettpyWlgTOnMlrN8Xw42eg8LpIDjgphyB9pzzra4iEwpVQ4qGZ6p4MuP0NxyW3RSMnbwaCo3c4H6tK8axAApRmweTNjktuWxBUP9rvS3duJGa1irS1iYSE3IPVGolKdRGrNYELF/Kor3cWv9razHaxbG52+vGwxNi6mVghjWR1x2z4jNVIha2I5Rr/bHgw8B2R7E2ipK+kHIKVb3aIWESd/Bj6LpRK0fOS68mZpTs6RHhPBpTMA66PH4rtas9/DSytkPwSBF2ClrFQ9jDULgXq+3Sprsmc0PAiwEBoqP1NL9LU9C/U6iJaW+U3vSRBeLic+R45Mge1uhyLJYHqamXxHAwGyj2op5dHvCqNV1t0PBBkIDlA5EybEb1ZR6bWvxd/sPAdkRwI0ncqJ1bSd3puNlmS2iGWw4nqW+V/doKDXY/tE86zH6tVdGouJpcGZaFSyeUzgYFlxMXJpUHeEkpvkBwg8kCQgVdbdNysyeZTSz4PBBmYqPJ8racfZYZgILAPuEqsuDrux2cYOTKnXSDtqFQmYmJuPMOH5ACRmzXZfGjZxM2abJID/AI5mAxvkWxwUW/p6rif3jHmfVjwTZg5B6beIReeexi1Wrkusmtp0I3AmTYjn1ryWaHJ5VNLPmfalNvY7rJs5oy17y1u/XSP7y63e1Nl2xOFKx1jkiAnVgo9VIfprnv5QLWrHYgYZfJbsv2a6brZfMxHMPk3oL7+Nw2sgthfgMkEDT03DOutw7kkJSAIzoYPVmvCoLS37QlP/Xf09PI43WZ0iEkmq0WHx52vE6dK88crB4DhPZMsmQ8774X6SNmMpT5SfuypeOSNgN2fcuQR+fGUFzsE0o7aLBege5CWljwkybEu0mYL4erV3hk+jBnTYYaxdGkSMTFD07z2vLXIQRDtMcrzNudi+GS1SKbWgN6so0Hq1vfaTx/w3ZmkpyiZ7xfF/mDf+52WBxWrINBFPNdVAbqb2LckBgXlIAjlWK0JXL2ah8nUc9JmzBg9kyd3dF0MDi5j+nQ56eNrnRR74hbteqfZZnKASDLKcclktchNAdl8JG0ahLu7MRjeM0k/nqF2liyQE/XQ5mKjtDnK409rsWTS2FhKQ4ONysrSXgkkyAXnXc0w1GoTycnDP+lzxmrks7Z8woRx3r6VYYNfJP30zJjjELcNvsoEQQKr4357rFq5AN1H0GqVkzuuTDKGC2esHTWU4cJ4b9/OsGFoLbcHouDck3QXhU85qFB0nuo4pqdrdWUwCs6jTsDC/4FTmXD2Hnl3z8JfgjUQNE3QEgVn1kL11yC4b4XmdjztcG6xJBAY6Jz0MZsTPNqetiveKji3j6ls64hfvqdsZenHDXxMZYYpKQfl7YsRdbIiRNTJj1MOevvOembUWfj8fpj8pjyjrJ0Fp74FqjY4uBn2/M2x6NwHqK7Ow2ZzNsMoLR0Yl+/PmvSsr0piXbWKn19MYr/JO0kiMXC9v4ZyABhaM8mhiktLtR2ud+ss+Biq46/7QF4n7jREl8PBQRKllHfgyiTZwNc0Rc5yX5wP4z6F/U+DyTebi9l35ERFyWYYZnMCpaV5Dia+nuKzJj2v1GXRKskx0Cu2MvQNcuihs7ekn6GLfyY5GLiz86c6Hu56RRZGkL9m/BVqBrEx1JVJciuIqBPyDPLifIjfBVU3983l3AvU12dSXFzK3r02iotLB0QgAd5qyGkXSDutmPjXteGfJLpRGNozSU8UnA80NptrSzX7zh+l36NiMmx/UBbKY4th9j75ccUk2s0zuuLpgvOKyXJ7iPTnoGouxO2B0q9BzEEIPwgNA7NnvbcF577AFatyMuiKrVzxpTgQBeeuxvjxDD6mKMOUwpXyTp/OWDRy8qY7KibLAnnTB/LXzkvvwaJmOlTOggmfQOlS2P9DuYfOzc8NbHOxwOdBXdjloBHwbL/s/jI6QHlmP0rtbwU7XPCL5GCgtPPn/W/27B4Ud1qeQX52m/zVvvQeTKJPQkIxnPsajDsiL70vzpCFMnIA78c6D0LWdhJKI6AD0gbuOd3gvpF5BAqOSaJAIYRJgTdzyuy4j/qU2cj7Tb4l8n56Zmgvt4cSnXf+9CYkEHdaXmpvf1CeQZ5PdnwMMKUYlmzv6NS4bxWc9qCI2FvWGn8MV+c6tqu9OGPAltsAWNPB9JIslK3fBV4CDNBpp8nzxZuZF51GenzHsX1VRo7VFrFu1uAY0i4Ok2Odr1/N4XJbOaPUCXwjPI9I9Xi2Xtbxg0gDU7Uip8xG/lin43sR/n3VQw2/SPoqMecdBdEeo4w5L38/9SCsMDh2arz1Nfl7Twnl6HMdLWsD6ZhBjjrb0bJ2ILGmywIZtAWkXOiyFW9edBpr39Px0p0G0uNF9lUZ+dEuHb9fPrhCtDgsk8VhmbS2Oh7/QaSBP9bpSA/JptCUzw8iDUzW+Et0hhrDTyR9veAcenePB5bbB3eMaW8Ni9yRUamsaPE2x77f/alwPnLb9W+U2tO2Kp3hWbT7YMSfoe5RhPA/gClNFk7kRM6qGSKGEAP3v6ljXWo2+cX5/HWVwWFm6U1mILK8LZttjZtYFZbLjFBxUAvO/XiGfv1pBUHYIghCiSAIxwRB+Kcg+ELD7BsEVx0ZB6pT42Cj3QdjfwSXfg/1P+5YendJ5ogTRNalZvPMJ5v47uzsXgnk9nI9d+xIYs5bKr65N4n3qwam+LukxciupnxWheWyqymfkha/1+NQpL+fPx8AMyVJmg18CTzV/1sagizcBfFnHI/d9gbc/qbjsfgz8lhP4Koj40B1ahxstMdkgTQvlh/bY5Tqww7DjOeMbD2Yz9Nfy+Uvx/IpPN+9EG0v1/Nfh7Ooai5DQqKmpYzNJVkeF8qTzUa2XtGxbpSBNREbWTfKwNYrOqdkjh/fp18iKUnS+5Ik2RcHnwFx/b+lIUhVHNyt7xDK+DOQcgymHnU8drdeHusJ9typXFa09y7PXN/bNKzrEEg71nRofaz9ofGckfvf1PGPew1sFDfy0p0G1r6n61Yof3sihxarY/G32Wai4CvPFn+fMxexbpSBlCB5ZpsSJLJulIFSi7MPpB/fxpMxye8C/3D1Q0EQsoAsgITIQVyVD0bB+flkeCdTFsEjN8Hcz+DtB+WfdT72TqY81p17tNmcTTI+T4NJX3Rkt/fcCafm47LYHLzncO5hBKD43CcYMv6COHo+NNS3xyiLLhSxaobysru6Wbn4+2JLOREerFf/etR6p0TO7ECxXTTtDKTDuR/P0KNICoLwIaDUlT5HkqR/XR+TA7QBLtcskiQVAAUAC+LiJLfuFmDaYVj27w6h2H0HfDHP7ct5jPPJshgu/gj23dohhkrH3MFuktG5Pe7MIvhA55iouYFYv+Axp2PiBBFxguu45PjQBCqbnB2CYoL9xd9+lOlxSiVJ0gpJkmYq/LML5EPAqv/f3rlHR1Xl+f6z84KERwLIU1IEgYAPnAYBbXlZiBLUJV610z1djtO00xkYn7Nuy6AlrDU46XHBzB277zhwcxV7OV2unvJx1ekGUZtSXi2Gh4rIS4RUgqBCIIEkJKlk3z9O6plTSaXqVJ1Tyf6sVauok3NO/VIJ3+z927/9/QEOKWX84hcLV++HxW+Gu+ksflM7bjaFX2mjxV23as+FX+kfi5eoJhmbEou7j/Hk9HJyM8OLv/tn5vHYtclxCFKkPwlNt4UQJcA/APOljNjlnwzmb9EXivlbzB1N+vON/um0dwLc84r2tbceDB4LPaenRDPD6C2r2Sningla8fe6fU6+afAyKtfGY9eWc6ct/R179ra42NTs5Lz0ktme3f0FiphINCf570A/4H0hBMDHUsplCUcVDav20R5dEy5+1RPh8PUgRPixdxzaufGIZDSTDICHnu3IR/bNaXdPuWeCIyCW9fUmB2MQe1tcuC+X0Yo2VmmTKahj7SMkJJJSygSSbHHQnZtOrBhdiRso/A7h/fs7H6ueGLtARsa4Y3F4TtKPQNttc1vHLpNIobSKw3kKMNrh3Czi+XFsanYGBFJhLOlVp//Ros5lL75M7bgf23GY9VFq40oFh2/QTDH8JhmRmJ2fnLpZ2+sdysgvNeNeRYBdF138vbeIB7/O4O+9RXzcYEx95nnZu/v3mEl6ieShabD53qBQXBoAbVnQ0NGt3nYclrwKZ3ppuebhG+DFZ6J/3cz85NnxmhmGXyj95hi1E8yLKQE2Hl3LJ9+H11vuq/XgqorfxWdHnYuNZ8s456sCJOd8VbxyocwQoRwi1Op8ski/vduHpoUv0viFcf+NMG03vP1TbZGkNxMt7WDmbpvTUzoMel+AowugeKv2+nwKjDCSwHVDZvLk7lLW3ehm1nA7+2o9rP6ilDXXxW+e8fvvdVzMZSNv1ju5aUBiC0d39i/nv5rK1JQ7CaSfSEbinaAJ5OytcOQauOO1ntdQxlpwPutDbcdMaF6x8CttMUYvL2kkoTFuK4GSN8Lzk63ZsL2k+3xrPDlKiC0xVj0RDs2HaW/D/ru01znmLSAk4nC+cLCdF/LcPPJhKQ9MWc5/HlrPv96kCWa8nPNFcTFv85KTE/26WH4cs3IdiAz4Q5OT8+1eMoVa3TaK9Jpu62E7ro0gj1wDxV8mt4ZSb/vh3S4oONe5BtLIfdqRxGvim2xGH4YpH2oCOeVD7XUac/NoOw9MWc5vPnuW0quWJySQAFdk60+Jo7mb95SZ/Rz8Y8FJfjO0nTGZ1u5BlE6k90jSP9V++6faCDJy6GB0DaXe9sN3OqZJoTWQoXWTyeLwdDg6I/bzJ+/VFnbCtjAaKKpjjoD9/4JnmTb1Pj0F7Btg+yOaH2Uasuu0h98dXs9jf7GK/zy0npnD7QkJ5QOjy3mhuixsyp0j8vjREFXIbmXSWyRH1QRzkKmqoYy2/VBPPBPZhmgkk/dqJUKhBr3RSobiZfjJoECC9uxZBiNPpKVIhhr43jzazvWD7WE5yniYP9RBa6uWmzzn8zIsy8b9Q8oD7uYKa5LeIvnJ/OC/jaqh7I7IrYbeCcH6R6P2aRtNNIPeOZuME8nPFnVOlJ2eAueuN+b+Kebzs5UBgQSYNdzOuhvdfHG+MqHR5Jx8B3Pyg6IYaYKhsB7pLZKhfLRIy0FGLmaE1lD2BL0FEL3th6HTaj3xTCaxFMW3t3dv0GvUYk4vKjhfObdzj5yFg+0sxBqu590VnAu9VStFXKT/wo2fyBrKugLttZF7uvW2H77jgCmfBsVy56Lg1DsRQws/eoa+PV0UijaatoBBr+viLoq+eoSMw39J0VeP4KrbYXZICkUYvUckQRPEDSth7XPas9GmF5/c0nl0WD0R6q7QF8/RNYm/Z7QV9Z6Y9+5YHKXv9x2x32P6B3Dl0fBjY47AD96L/R4RuC7uouzsRqp8Z5FIqnxnKTtTgavWGjumXjvsYupLRQx5PoPZrxXx1vHktHmIl92NLp7+rohlZzJ45mwRnzRZK77eQu8SSbOIJp5G1E6GrqjP3hKfk1Dklsb6IT33ofzWBotfDgrllUfh9pfgu3E9+nZCcZ5/jcYII4ZG2YLz9O/ivqdRvHbYxWMflFF9UWvzcKqhiqd2lVlGKHc3unDVl1Hbru3eqW2v4tWLZUook4C1c5JWNdhNNUYsCh2+QXvEa+Zxqhg2L9WE8sAcmLoD3vsb+GZyfPcDvL5z+sdbz8Z9T6NYs9NJky9890pTWyPr9jkDDkJm8vYlJy0Ru2taaOSdBiezcs2PrzdhPZEMFUYIZtr9xeFgjlCmog1ENKKtqEcSaxuIeM4B7T0/nw03boHdt3ds/wxZQeihe5AtcyhVbbWdj2cNg8uXo15nJNF25dRc1N8d802Dl1R2H4lGbXuU3TvtXtVe1mCs9XFGOo9HKw7vS4SuqBu9KNRTxh6DqTs1gZy6Mzj1jjNfWT74XvJE+H68PJFD+fCfGBl1XBTm6++CGTvIGkYSQzP14xiaYY34ehPWEkk95/FIkmmwe/V+WPYcrFipPVuhLUS0FXUjFoV6wthjsPi3sPln8PEd2rM/R+nPV445op075khM+UrHgJuoKHiQcVnDEMC4rGFUXPFzHPlzkvzNdM+vFpSTlx3e5iE3K4/Vs62xO+beweXkiPD4cshjyUBrxNebsNZ0OxYBNLo43I9/FBvaaMvM6b0fvcWfnpj3GsVIryaMNZO01zWTtBzlSC/sW9iRr3wJDs6Fa7fDew9p+cpu/gw7BtyEY8i8wOu1F/6Ip+Eg9gFB9yBPw0Eq66pYMfLeJHxj+vx0qpbXe3qrk+o6L2MH2Vg9u5wfTbFGvs/vGvRmvZPaNi9DM2wsGVjOjXnWiK83YS2R7KpFASRWHN4d8fTPMdrhPBnEWnDe3TmV/iLqkHMDYt2uPR+YDbM2wyeLgvnKHhacz8wopPTU87hHPow99xo8TV9S+u0LuK98IuV5Sse4u3AsvQsAaUGX83k5DuYN0UQxcueOKiY3Dmv9r9ZzHpckrzg8FKv2z0kXxh7TVrw/WaQ9R+YoY8TefwrukQ9T+u0LrK59QxPIkQ+HjSwVilRirZGkXwDNKPtJ1d7vVNCV7+WeBca/XyBfuVQrFaqZpOUoNy+Ny9zCnnsNywcv4NkLb7OqYAn23PQzyFD0HqwlktDZeTxVGL3320z8u3RSZd3mz1d+U6y99tdUjvTGJZKepi9ZX7+VVQVLWF+/FXvu1dhzfmBszApFjFhPJM3CzFGs0UTzvayemJwEy95btefQe58q1h49/A3zXD5Mae3/CeQk7blXa1PurCcsOeV+7bCLNTud1Fz0MmaAjSenl1ui2FxhHEokQ0l0FGtmwXkk0XbpGFVwHu28SHrYH7Xy8nHcQ/8We+ZEaGnBnjkR99C/pbLhKPbsSd2/X5LQcw969YCLx/9URmOrtvPlVEMVT/25jAEDMH0VXC3cGIcSST+9bQtkrLt0LMaKgZ19Ke39p2DPsZ4v5dNbnQGB9NPka2TNTqfpIqkwDiWSYN0aya6Ysg/mvRsU9W0lWksH6Nr3Mg2EMl2ortPfGhhtS6MiPVEiCfHVSJrJlH3h3RLzL2ivQRPKrnbpRBPJKXthzuag6O5YbH5jMYtTmG/DW1fV6bhVti4qjEGJJCS3RjIZecp57+qL+rx3NZHs6S6d4j1we4To3v669m+/UEb5PtY2b2VmZiH2rEmBczy+Y1S2VbOin065US9yOP/VD5+h7IPHaPQ1BY7nZefxzwvLGTzYvNjAmvsa0hVDPkohxC+FEFIIcYUR90s50WohU1kj2RMHcqNFPZroztnc7aUzMwspbXoFj+8YoAlkadMrzMwsjC+WNMIx5UdULPwN4wYVIhDY8sdRcVdFYEujoneQsEgKIQqB24D0TcTo7fRJdY1kTxzIjRb1BETXnjUJd+6DlDa9wurLmyltegV37oPayLIP4JjyI04+dID2J85z8vGTKRPIj2pd/OJgEf/j0wx+cbCIHXXKbDdZGDGS/DdgBdrmQWOY9ZHWUzsU23HteDJIRX+c7uiJA/m2En1R31YS33snKLr2rEksz76ZZ1veZ3n2zX1GIM3iT9+7+I/qMr5v1VzTv2+touJMmRLKJJFQTlIIcTdwSkr5mTCyMOvMWFjyarCntu148HWyMGunTyixOpD7V7GjrW73lG0l4QtB0NEDZ3HUS1yte3G2vItXnmcEA2mklVU5t7G+dRf2rIlKKJPIy14nzTLClVw28vvvnWHtahXG0K1ICiE+AEbpfMkJPA3cHssbCSHKgDKA4rF5ml9jtJpE7wRNEJe8CvtvhGm7g4KZDsz6SBP60Hhtx2FUjdYrPJrrTk9qGw9Pj18U9e4FQdGVArI6cpLtUvt6yEqAq3UvZc2v04gmqt9yiX5kMjljRGDqHZhyJ6Hg3KrE2q42Ub5v1s9snfN56d+/IxZVTG4Y3U63pZQLpZTXRT6Ar4HxwGdCiJPAWGCfEEJPUJFSVkgpZ0gpZwwa0RR0H/fXJEYa3I6qga+LYfZWTSj9I8pkTbmNxD8S9qcM/CPhM110ODTbgfzwdG1E6cuGDBn82ZS8oZUcheBs2RwQSD/NtOFs3hTIUVa2Vacm7h6y9ts38Vw8EHbMU/cpa0+547qf6/hbFL02m4zfjqfopam4Dr9mRJhdMqK/fonRFdmq9CgZxJ2TlFIekFKOkFIWSSmLgBpgupTyTLcXi4j0pV5bhvYMuHY/fDFNG0nO2N690FiF0JHwnPfCUwfRsIIDeVelRSF4pf6CjleeB7QcpW75jwWYmTeJ0pPrAkLpqfuU0iPlzBxY3ON7uY6/Rdmup6hqOKW1xL1YTdkHjyVdKMuuKqdfRrgreT+RxwOjlSt5MrBONVXoSqrtOPzwQ9h6J1x1VBtRLvgj/PmW9JlyeydoI+DQkXBXJLMtbazEuMptE/oLOjYxxOiIDMc+aCruoicpPbmO1adfpfRIOe7JTuz5PXcZcu5bR2NbU9ixRl8Tzp1rYrr+13vWsuu0J+zYrtMeNh5d2+V1t4928MSECkb0G4dAMKLfOP6usIL5Q1U+MhkYVkzeMZqMn9CV1FE1wZFX/yZNaL6YBhkx5Lesgu24NgLeuUB79k6ILpRWMcboylMzJMbyrBLKWt8Im3LnkU15Tkn3Lufx5CjB0IJze/Yklhcs5Nlv3awaWYo9uxgaG7u/MAJvwzf6xy/WwIULUTsx+pkzfialr5fy8h1u5hXa2Vbt4dFtpfzvee5ui9Hvsjm4yxYUxaZwrVbF5AZizkcpI359ImsSP5kfzEH6heaqo+kx1Ybw1fgdtwen3pFlTVYjxtIiR/Z0KrLvY5wo0Bp4iQIq+t2PIzs9tjF6Gg6y/sL7rBp2L+vPvtspRxkrtlzd9Du2AWNiut4+3s7Ld7hZuqmU8j+vZukmTTBvHm3v/mJFyjBnW2JdAdTJrh13QoXGPwqLJbdnBUJHwhDMUY6qsXbsPSgtcmRPx9FvRooDTBxPw0FKv3ke9xjNn9JeMI3Sk+twFz2JfdDUHt2r/NrHKNv/jzS2BXvv5GXmUj79yZjvMa/Qzs+vX8663c/y5I2rmFdo54LqGGIpzBHJy3mw4dGuz0lXoQFtJBxJV9NtK2FkaZEFqbx8PCCQEMxRVjYe67FIOmx3AuA8+Bu8TWewDRhD+fQncUy4J+Z7bKv2sPHz9Tx54yo2fr6euWPtXD9IjSSthJDSuI0ysTJj7Fi559FuRLKvkw5JpVhijOccnWvWNm9lZvY47DmTA+d4mo9Q2XpS86CEznnLnJzO7xV5zF9YGEpeXudjubnhr/N1aiAjE4kFnRe4Qrsuek54OuUkl27ScpKhU+76+s5vVVcX/joyJ/nwwzM4enSPqpY0AOUCZFX6UrvaGBZzZmaMpbTuRdwDl2LPLsbTfpzSupdw5z8UvD7NCs73nNiO+3439vGaIN51rR13npsdJyq5oyCx0aQVf13SFSWSCksQZrnWgaf1GJVtXlb0vxV7djHugUspvfQyy/vNYX3zTtz5DwVHlmnIihmPd+rnbR9v54ZharptJdTfG4Ul6GS51nqM0sbfMjMzuIvEnl3M8n5zePbyFpbnzk1rgVSkD0okFZYgzHKtaROljb/FnfezsOZfntajrG/ewar+i1jftB1PyxETI1b0FdR0O12wSsF5VyTYiTFgudb8HqtybsOeOSHodt4xsvQLpz17UliOEkhbh/NIjDDGyMxM+BaKDiz2v6wPkWrPzDTA4zvG+tZdAcs1/9QboLLNGzay9OcoK33p6/WsSA/6nkje/7JmlhHKjO3a8VQSj1NQL8bf9sGd+yBr+i8OTL39Qqkt3oR7VNqzi1mRu9CMcBV9iL4nkicnamYZfqGcsV17fTKJrVb1Ro0AR67tmVNQL6ayrTqs7UNPLdfWNn3QKUfpaT7C2ktbolyhUMRG38tJ7pmrPS/4IxQfhLEnNbch//Fk0JXTeuNAzcBj54I+K5CArrWaPWtSzA7nM7NsgbpJe85kPC1HtNcFvzA6VEUfo++JJGiCWHwQCk9CdVFyBRKiO61D7E5BeqiC8wD2zIm4B/6M0roXO+ood+DO/xttih56bRou5sSzkKMWbozDgv+rUsCM7doIsrpIe47MUSaDSH9JSE+nIAsTVkfZb46qo1QYQt8TSX8Ocuud8Ooy7Tk0R5ksIv0lr/4suoGHIi7C6iibd6g6SoUh9D2RLPoqPAe5Z672uiiJvWT0/CWLD3Y+zztB30FI0S2e1mOUXnoZ98ClrMm7U9vCWPdSrxTKtTvXsq26s6P5hgNdO5or4sOaOcnuug0mwutLOx/bMxcaBnbdwTERUmX71gcKzqOdU+mr0uooMydCe3sgR1nZfAJ7Rshn3AsKzmcVXE3p5lL+6z7NHMNzQnM0f/kOd8B4SOUkjcOaVmmRhruRr43m6v1ax8bQJlj+j8VowUw1VhNJPVJoudZJJGM5BzpbrOnZsEXariXRcm3ruX38+I1Slt2wnA1717NxsWa35ueWW2awf7+ySjMCa44kU913e/6Wzl0C/b9e/pa3kL5Cqeh12MfbWXbDcv5p+7M8M3dVmEAqjMW6w4zI1eAB9dp0eMVK7TmyT3ciROsS6Eev5a1FcPn2U3T5OTIur6To8nO4fAZ+LknG1bqPoqZfkXHplxQ1/BOu1r1mh5Q2eE542LB3Pc/MXcUfXrcZAAAIY0lEQVSGves75SgVxmHNkSSErwbP2AmZPshq075m9OguWpfAULoTUhNw+fZT5nsz0LWwiguU+bTPxZFl7VGvq3VfWMfFKnmBsubXAdKmoZhZeKq38ePNPw/kJO1F9jCHc4WxWHMkGbka3JoVFEg/Ro7uPlrUuUtgJPX6vabNxOnbEtbWFaCRVpy+kM+lvT38ESeBUV/jCoqafoWrdV/c9wJw+t7Vj71ls/4FsXwfkefoPXy+8Ecs5/h80NLS/ePy5fBHY2PYw1WzmaId95Hxp3kU7bgPl/cPWh+GyEd9ffjjwoWwR+XJXbgXb2TBsOmI+joWDJuO+343X16oZPBgLaWpFm6Mw5ojycjV4AEN+ucZNbrzj0bnbwneMzTlHdny1iJ40f/+ox2PF91RX+sbgNY1MR68MkrsUY6nGlfjbpyX3sbbXostYyjl+ffiGHBT/Per/Yiy6v+gUTYDUNX8HWXHn4fsHByjb+/RvVZMXdZpMcc+3h5oA6EwFmuOJP19t/1EG8UZObo7NA02rIS1z8F//1hreyvRnjffa8lFGxv633+04/ESddTnezfue9pElNijHE8lrsbdlNW7qGqvRQJV7bWUXXgFV8PHcd/Tefp3AYH009jejPPrigSjVSQba4pkJHrT4WSO7kIFc8NKSwokQHnWIvII/1zyyKY8y9jPJRmjvvKskiixl8R9T6NwXnqbRlrCjjXKFpz1b8Z9T2/rWf3jl7+L+56K1GDN6XYkkdPhdK9dNAj/4ozTtwUvF7BREBDIosvPhR0LLOTEUXBuEwVU6QhivKM+V+u+wOg0E0EbknGigPKsEm36nqSCc110ism97bW6p3rbagPnuxo+xln/Jt62WmyZQykffC+OIfOivo0taxhVvs5Caes3vHM/2DjoVBDZ1qZ3miIO0kMkQRPEeEQxmbt3LIAja1rYSnYyVrzLs0rCcpIQ/6gvMr/ZhgzcK978ptHYMoZSpSOUtoyhQMd0/KKLRqmNNqvatOk4WVk4Bt2se8/y4T+h7ExF4BqAPNGPcpvODjCFpUh4ui2EeFQIcUQIcVAIYb3No33MATymFe8e4sieTkX2fYwTBQhgnCigIvu+uEQtGflNoykfcDd5hO+oySOH8oFLgI7puNSZjp9/Leo9HflzqBhVxrjs4QgE47KHU1H4dziG3xo4Z+0pN566T8Ou89TuY+3RjYl+S4oESGgkKYSwA0uA66WUzUKIEcaEZSCp3r1jMsla8XZkTzdkpGf1VW0AR+4sAJwN7wRXtwcuwZGnWdxFnY77znV93/w5OEZGbzcxc2AxpUfKceeswT50Op7afZR+sRr3Tf8a53eiMIJEp9vLgeek1JbtpJTWzEKH7t7p5Q7gNgqo0hFEo1e848Xo/GaycOTOCoglEJbvjDodzxqW0Hva83+Ae7KT0i9Ws/zKe1h/6i3c163BPnxW9xcrkkZCBhdCiE+Bt4ES4DLwSyllZZRzy4CyjpfXAV/E/cY9JD+fQePHc9W5c3w/bBjDT5zg67o6LoaccgWgv/xoDvHHk8dQ8hlHeCqlnTqqaER/CJTsmEIxLj7zfmZ634NEUs/JBD9jjcGMYQCjaeA09XwT510mSykHJRyLovuRpBDiA2CUzpecHdcPAW4CZgJuIcRVUkd5pZQVQEXHPfdIKWckEriRqHi6x2oxqXi6Rgixx+wYegvdiqSUMmoSRQixHHizQxQ/EUK0o/2F/964EBUKhcI8El3dfgtYACCEKAZysNa0VaFQKBIi0YWbjcBGIcQXQAvw13pTbR2sthdLxdM9VotJxdM1VosnbTHFmVyhUCjShfTYu61QKBQmoURSoVAousBUkbTilkYhxC+FEFIIcYXJcawTQhwWQnwuhPh/QphTbS2EKOn4GX0lhFhpRgwhsRQKITxCiEMdvzOPmxmPHyFEphBivxDiD2bHAiCEKBBCvN7x+3NICPFDs2NKZ0wTyYgtjdcC/2JWLH6EEIXAbYDX7FiA94HrpJTXA0eBp1IdgBAiE3gBWAxcA/ylEOKaVMcRgg/4n1LKq9Fqcx82OR4/jwOHzA4ihF8D70oppwB/gbViSzvMHElacUvjvwErCDaUNQ0p5XtSSr+P18eAGY4cs4CvpJRfSylbgN+j/WEzBSnlaSnlvo5/X0T7z3+lWfEACCHGAncCL5oZhx8hxGBgHvASgJSyRUoLbYxPQ8wUyWJgrhBitxDiIyHETBNjQQhxN3BKSvmZmXFE4edAlOYvSeVKoDrkdQ0mi5IfIUQRMA3YbW4kPI/2hzX+BkLGchXaZo6XO1IALwohBpgdVDqTVD9Jo7Y0piiep4GeNRtJYjxSyrc7znGiTTNdqYytA73m9qaPsoUQA4E3gCeklPUmxnEX8J2Ucq8Q4haz4oggC5gOPCql3C2E+DWwElhlbljpS1JF0mpbGqPFI4SYCowHPhNCgDa13SeEmCWlPJPqeELi+mvgLuDWZP7x6IIaoDDk9ViI23DBEIQQ2WgC6ZJSxt9PwRhmA3cLIe4A+gODhRC/k1I+YGJMNUCNlNI/wn4dTSQVcWLmdNsyWxqllAeklCOklEVSyiK0X7TpyRTI7hBClAD/ANwtpWw0KYxKYJIQYrwQIgf4CfCOSbEgtL9gLwGHpJT/y6w4/Egpn5JSju34nfkJsNVkgaTjd7ZaCDG549CtwJcmhpT2mNm+Id4tjX2Ffwf6Ae93jG4/llIuS2UAUkqfEOIRYAuQCWyUUh5MZQwRzAb+CjjQYdMH8LSUcpOJMVmRRwFXxx+2rwHVIyIB1LZEhUKh6AK140ahUCi6QImkQqFQdIESSYVCoegCJZIKhULRBUokFQqFoguUSCoUCkUXKJFUKBSKLvj/7gfbZz1htl4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plot_boundaries(w1, w2, w3, w4, w5, w6, b1, b2, b3)\n", "plot_data(X_train, y_train, X_test, y_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It appears that the classifier obtained using the above set of parameters does not match our data. (Of course, this is to be expected. This classifier with fixed weights a priori has a high bias and a low variance.)\n", "\n", "### Question 2\n", "Try the alternatives below and see which one is a better match for our data:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "w1 = -1; w2 = -1; w3 = -1; w4 = -1; w5 = 4; w6 = -3\n", "b1 = -4; b2 = 4; b3 = 1" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "w1 = 1; w2 = -1; w3 = -1; w4 = -1; w5 = -4; w6 = 3\n", "b1 = 4; b2 = -4; b3 = 2" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "w1 = -1; w2 = 2; w3 = 1; w4 = -2; w5 = 4; w6 = 4\n", "b1 = 5; b2 = 8; b3 = -6" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Obviously, we need a better way than trial and error to find the best parameters. The way that we do this is by *minimizing a loss function*. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Loss function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A *loss function* evaluates how much the predictions of our classifier are different from the actual labels. The loss function that we will use for our network is the *binary cross-entropy* loss. Let's represent our training data by the set $\\{(X_1, y_1), \\ldots, (X_n , y_n)\\}$ and our neural network function by $f$. Then the binary cross-entropy loss function will be defined as:\n", "\n", "\\begin{equation}\n", " \\ell = \\sum_{i=1}^n -y_i \\log f(X_i) - (1-y_i) log(1-f(X_i))\n", "\\end{equation}\n", "\n", "The binary cross-entropy relates to the Bernoulli distribution (maximizing the Bernoulli likelihood is equivalent to minimizing the binary cross-entropy). **It is the loss function that should be used for binary classification problems. It can be generalized to multiclass classification problems, see [cross entropy](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html).**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 3\n", "Let's see what this loss function means using a tiny example. Assume that our training data consists of only four examples, and the values of $X, f(X), y$ of those four examples are as follows:\n", "\n", "|X|f(X)|y|\n", "|:---|:---|:---|\n", "|(5.4, 1.6)|1|1|\n", "|(1.4, -0.5)|0.3679|1|\n", "|(3.5, -3)|0.8647|0|\n", "|(-3.5, 1.1)|0|0|\n", "\n", "Calculate the loss function using the equation above. You can calculate the *log* using this function:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "-0.6931471805599453" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.log(0.5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is important to remember that the loss function $l$ is a function of network parameters, since it is defined in terms of the network output. We can write the loss function as:\n", "\n", "\\begin{equation}\n", " \\ell(\\mathbf{w}, \\mathbf{b}) = \\sum_{i=1}^n -y_i \\log f(X_i, \\mathbf{w}, \\mathbf{b}) - (1-y_i) log(1-f(X_i, \\mathbf{w}, \\mathbf{b}))\n", "\\end{equation}\n", "\n", "In principle, we want to find the set of parameters $\\mathbf{w}, \\mathbf{b}$ for which $\\ell(\\mathbf{w}, \\mathbf{b})$ has the smallest value. We will use *gradient descent* to find these values. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Minimization by gradient descent" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The plot below shows the function $f(x_1, x_2) = x_1^2 + x_2^2$:\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 4\n", "Point A on the plot has coordinates $(1, 1, 3)$. The blue vector AB shows the direction $(-1, -1)$, and the green vector AC shows the direction $(0, -1)$. \n", "Assume that we are at initial point $(1, 1)$ and we want to move in a direction that minimizes the function $f$. Which of these two directions moves faster towards the minimum: $(-1, -1)$ or $(0, -1)$?\n", "\n", "### Question 5\n", "Calculate the gradient of function $f$ in the point $(1, 1)$. How is this gradient related to the fastest path to the minimum (i.e. the steepest descent)?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Training the neural network\n", "\n", "We now understand the theory of training neural networks. But how do we do this in practice? We will now develop our practical skills using the *scikit-learn* library to train our tiny network. Let's first define the network:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "from sklearn.neural_network import MLPClassifier\n", "clf = MLPClassifier(hidden_layer_sizes=(2,), \n", " activation='logistic', \n", " solver='lbfgs',\n", " random_state=0,\n", " max_iter=500,\n", " tol=1e-7)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The argument `hidden_layer_sizes=(2,)` states that we only have one hidden layer with two neurons, and the argument `activation='logistic'` shows that we use the sigmoid activation function (Let's ignore the other arguments for now). \n", "\n", "We will now train the network using our training data:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "MLPClassifier(activation='logistic', alpha=0.0001, batch_size='auto',\n", " beta_1=0.9, beta_2=0.999, early_stopping=False, epsilon=1e-08,\n", " hidden_layer_sizes=(2,), learning_rate='constant',\n", " learning_rate_init=0.001, max_fun=15000, max_iter=500,\n", " momentum=0.9, n_iter_no_change=10, nesterovs_momentum=True,\n", " power_t=0.5, random_state=0, shuffle=True, solver='lbfgs',\n", " tol=1e-07, validation_fraction=0.1, verbose=False,\n", " warm_start=False)" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "clf.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once the network is trained, use the helper function `tiny_net_parameters` to get the parameters of the trained network (`tiny_net_parameters` is a wrapper around `clf.coefs_` and `clf.intercepts_`):" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "from utils import tiny_net_parameters\n", "w1, w2, w3, w4, w5, w6, b1, b2, b3 = tiny_net_parameters(clf)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUkAAAD8CAYAAAD6+lbaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2de3xU1bn3f2sSAgRCUEK4BJMJKpdgQIVAlSoZsOcVKyL6Go2InvYtwfG0ElqCmphEQqGFiJXT1gBKe0BSNFbR4rW1maDH09LECyI3D5ohgkAAIUAChCTr/WNnJ7Nn9p7L3mvfJuv7+TyfYe/sy9rDzG+etZ5nPYtQSsHhcDgceRxmN4DD4XCsDBdJDofDCQIXSQ6HwwkCF0kOh8MJAhdJDofDCQIXSQ6HwwkCE5EkhAwkhPyZELKPELKXEHIDi+tyOByO2cQyus4aAO9SSv8vISQOQDyj63I4HI6pEK3J5ISQAQB2AhhJeWY6h8OJMlh4kiMBHAfwR0LIBAAfA1hIKW32PYgQkgcgDwD6ARPHMLgxAOxKBlpjgh8TQwFCgaHngKP9gZGngIRW4Gwc8PVl3ducKKNXr4BdX/cfL9k+dcqoxhiNF5SeIGa3Ihpg4UlOAvBPAFMppTsIIWsAnKGUFiudM4kQWqfprt04SgEaxkeheDtQ5gE8TiDnHsBdB1RMAqpeAVxeRo3hWIvhwwN25d58WLL90ktGNcZoJoHSOi6SDGARuDkE4BCldEfn9p8BXM/gumGR2hT6mMQLgiB6nIIguuuAZdOEVy6Q6lg1VXg/ffE4hf0cTjShWSQppUcBfEMIGd25awaAPVqvGy7L/w7EB+kq974E/P4twWPMuQd45nuCYBZv7xZOTviI4ph1WHg/PU7BFtwubGcdDnEBDsdmsIpu/wxAZWdk+2sAP2J03ZDM3SW8Fs0AGhKBy1uE7ZPxwJBmYPV73cc88SFQMh3Y9ifBg3TVC19s3uUOH1Ecq14R7M57gbYYILYdeP1l/j5yog/NY5JqYDkmGQmrpgpfct8vsscJ1KYASz4yoUE2xXdcd/UNQEtc95ivqUyaJNnMaK4NOGTvXqMaYzaBY5Iff/xxcmxs7AsArgGfSCLSAeCLtra2n0ycOLFR7gBWnqQtkBNCl5d7P5HiO67bt7V76MJVb8/3sqBgFWprs1BT4+ral53tQVZWLcrLl5jYssjIza3EihVFSE1twLXXBkb2Y2NjXxg6dOjYwYMHn3I4HDxdD0BHRwc5fvx4xtGjR18AcIfcMfzXhBF2DmRUZgLOfCFTwJkvbAfD4wT+c4ogkHEdgjiKY752HOOtrc1CVVUOsrMFVzg724OqqhzU1mZFfK3c3ErU1zvR3u5Afb0TubmVrJureN/nn8+D03kQDgdFXJzsQP01gwcPPsMFshuHw0EHDx7cBMG7lj/GwPaYRqQioAbfQAbQ3SW1eiCjMhPImwUcHCikUh0cKGwrvUfic937BfDWn4CtLwnbgCCUtSmGNZ0ZNTUu5ORUoaoqB0uXlqCqKgc5OVUSzzIc/IXK6TyI55/PM0QoV6woQr9+LaEOc3CBDKTzPVHUwqgfkxRFoCWue198K7B+W3dAhxVm5mBWZnYHr1KbhKh/OM/nzBeE0Z+004D32cD9po3rOvw+wwsWBBwy+JXnJNsnTkR2i6VLS1BSsgxlZcUoLS2LtIWor3fC6TwYsN/rTUN6ujfi60VCe7sDvvo3aRJQVyfNIN65c6d3woQJEb4rPYOdO3cmTZgwwSn3t6j3JItmSAUSELaLZrC/l1wOplI3PPdudt5tpN6gLw2Jke1f8lGg8Lu89g98ZWd74HZXoKysGG53RVfXOxJSUxsi2s+ShoZU3e+hlRMnTsT8+te/Hqzm3GnTpl114sSJEHPr9CHqRTJSEdCCxyl4kHfuBZbfBJBS4JkbgNm50m747Fxg6xh1oiaHlh8CpWT8cJL0owVxDDInpwqlpWVdXe9IhVJJqIwQsMLC5WhuZl1XZu3lwPBMwDFReF17uZarnTx5MmbDhg3Jcn9ra2sLeu727dsPJCUltau57+TJk0fv378/LvSR8kS9SBolAmJXO+9j4K9XAh0OAAQ41h9odQjCWOISjom/BFz0Cz5q8W61/BDIJePHtwr7ewpZWbWSMUhxjDIrKzCFKBhyQtXcHI/CwuXM2qrEli1zMX/+eni9aejoIGhtVa0Jnay9HFiUBhyJAyiE10VpWoTyF7/4xYhvvvmm95gxYzIWLFgw4s0330yYMmXKqFmzZqWPHj16HADccsstV44bN27sVVddNe7pp59OEs9NSUnJPHLkSOz+/fvjRo4cOe6+++5Lu+qqq8ZNnTr16nPnzuk6/TLqRdIoEahNEcYgK8cHenUXewEO2t0Nb+wnfw213q2WH4K5u4Tx2bTTQhGQtNP6jNdamfLyJQFBmpoaV8TpP/5C5fWmYf789diyZS7L5ga9f3q6FzExHdi1S2t0siwFuOCnDxccwn51rF69+tAVV1xxcd++fXvWrVt3CAA+//zzfuXl5Ye/+uqr3QBQWVnp3b17997PPvtsz7p164YcPXo0oIvd0NDQ59FHH208cODA7sTExPZNmzZdprZN4WC7PMlIAxT+M3IiCWpEgjgmpyR0Tb278wmTmwUP0x+13u3yv8sHp8L9IZi7y2KiGCvzsXz/fclm31unBRxy4YJ024z8xy1b5homivpyVMEVVdqvjvHjxzePGTOmy41ZuXLlkLfeemsgABw9erTX7t27+wwdOlRSUSwlJeXijTfeeB4Arrvuuhav19vb/7pr1qwZVFFRMQQAGhoaet92221X9+rVi15xxRUX//a3v30VSRttJZL+kWpxLA8ILZRGiUBqk3y0eEizMCPFVS90vXv7dbm1eLdG/RDYDTH/UexK+449ckIxtFXoYsvtZ0d8fHyH+O8333wzYfv27Ql1dXX7EhISOiZPnjz6/PnzAb3duLi4rjB+TEwMlTtm4cKFJxcuXHgSEMYkX3zxxfrRo0erarututtKAYqf3madRG657n3vS8IcckCIBL+xBZizj20Xd+4uIWWnY6nw2tMFEmCX/2hFCgpWBQSWsrM9KChYxegOJYeBPh3SfX06hP3qSExMbG9ublbUnNOnT8ckJia2JyQkdHz66ad9du7cqTAwZSy2EkmlruzpPtZJ5JYb49vwF6loubzAllfZi5oRSfORYIVZSDU1LlRUuFFSsgwVFe6oEEiA7SwheR7+DvjNQWBYK0AgvP7moLBfHUOHDm2fOHHiuauvvnrcggULRvj//e67725qa2sjo0aNyigsLBw+YcKEZrnrGI2tksmDJT7/8fWeXUzXyKT5cBF/rMT/C//tLvr0kZz3Re35gGtdd510O0TGSBeieFRUuOF2V6j2JK04vzv4swUWuODJ5MpETTJ5sEh1Ty+ma2TSfLi4vN1zusX0JyN/vFjlPwJGeG6RE61estWwlUgGS1cRE7l7ajFdI5PmI8HMHy+t+Y++xSr++McfYd26PEuNb7KYJcQJja2i24B8pNq/G9cTi+kqRdXNnjnj/+NlZDk1uW5wTY0rLGETi1WIRSOczoNYtOhZvPfev3XN7zZbIH2F2uNxWUK4oxFbeZJKiInc4pdP7ObZsSKNWqw4c8b3x6vMY69yanJVdfr1a8Hs2X+xhOfGapYQJzS2CtxwgqO2EpBeyFYMGp+A2uQ2LPmkb9e+x35yUnJeeXngtYz+mPpX1RHp6ABiYmiAJ8cC36K5DQ2pKCxcriExnQduIiFY4MZ23W0WaBUTqy4DYbWZM7KV4A/1gutQYNVsq9HQkCpb9qyhIQ2A1HNjIZJy3fvnn88DgCiZwWNfoqK7HQlayoqJRFJg1wq5glqwWu6lUYRTrELN/G4llLr3K1YUMbm+FeCl0mwCi1SZSFJb7FqxHGDzg2JXjC5WYWYtSiXW1q69fPjq4ZmOpY6Jw1cPz1xba89SaVrpcd1tVqkyvqktxduVI7a+gmq3RPdgPyhhd+tHjZJs3jhof8Ah/2A1k44hbMcHQ6PcvTenmO7a2rWXL/rrorQLbUIloCPnjsQt+uuiNAB4OEvdrBvfUmnTpk07M2vWrKZly5YNS05OvrRnz574r776avctt9xy5ZEjR+IuXrzoePjhh48tXrz4BCCUSqurq9t75swZx8yZM6+ePHnyubq6uv5Dhgxpfe+99w70799ft1HrqPQkg3VxWdWXjCQv066J7lbNvdQbM9aqMbMWpRxlH5SliAIpcqHtgqPsg55XKi0qRTJYF5dFqkykqS12TXTvqVXLzRgfNLsWpT9Hz8mXRFParxa5UmmjR4/OmDhx4lixVJr/OeGUSmNJVIpksDFDFkVmI8nLtHOuoBVzL43ArPFB36K56eleU6PaQ/vLl0RT2q8WpVJp+/fv3zN27Njz4ZRKa2trk6Q6tbW1YcyYMRljxozJyM/PH661jVE7JhlszFBrqoxsaotXvhsdTFCt3u3uqXUqrTY+aAYlN5cc9h2TBIA+sX06Sm62fqm02NhY7Nu3b4/adgZcj9WFrIaZ0+F8iURQ9UZNfmfQHxT/ZV7vuCPgkCvqtkq2D30ZdnNNo7BwuSRnEQDOn+8tGR80uwKQ3ojBmbIPylKOnjsaN7T/0NaSm0sOqw3aANJSadOnT2+aNWuWZODm7rvvblq/fv3gUaNGZVx55ZUXoq5UGiEkBkAdgMOU0tuDHav3jJuwS3T1MJi/L2pE8pCK+8D4aLPv/RobkxEf34LZs98IqHBu3XnSfMZNJBhVKm0hgL0Mr6caPpdbHrNLl6nFjGiz7/jgsGFHMXv2G5aqAMQxDiYiSQgZAeCHAF5gcT2tLPko8Ivv8po7ZdAq2DEdyQqzUXjtxp4LqzHJZwEsAZDA6HocnVAaqw05n92/aw0Av/udZDNhiTvgkHPntLfZCrNR/Gs3ejzhlVxjjRUrpEc7mj1JQsjtABoppR+HOC6PEFJHCKk7rvWmHFUopSMVTbf29EOlqLJR0WaWFc61YsUK6dEOi+72VAB3EEK8AF4CMJ0Qstn/IErpekrpJErpJFUz3G2ClQtCKI3VPpdlvaUffDF7NoqVajdG8wqQVkVzd5tS+gSAJwCAEJINYDGl9IFg5+xKBhzu6Mu7U7suuFEopSM1BcxpELDK9EMxim1kdNsXLRXO9cB3fNTsCuk9AVNm3LTGWLNbpxUrLsYVDnaYfmil2ShmY9e1bbSUSgOAsrKy5LNnzxquWUxvSCmtCZUj6Y8dRCRc7FoQQnH6YU0MEBcnWGtrgPXOd0vs3DkEmFXxXeSrvt6pazoRS4wbH31yCLDNLxC7LUHYr45gpdLCYd26dUPOnTsXUrNSUlKYul2WmLttdREJFzt4ZHL4zmdfciNw32Bg/dsxmLunswDLtA6AyKypYFPMyLtkhXHjo1NagAdHdgvltgRhe0pL8POU8S2VtmDBghEAUFxcPOSaa64ZO2rUqIxFixYNB4AzZ844srOzrxo9enTG1VdfPe7555+/7Je//GVyY2Njr2nTpo2aMmXKqOB3Yospa9yQ4YRiQfd22mnA+6zhzWCO/5gkIHhkkRbQMJVsAFUA5sYC2x2CQFa2AZf/DYB07Kt3vLSKVSvT0gf6UV/vlJ2b7fWmIT3da3yDdIHFjBtRGB86DmwcDGz6Gph1Vm2L9u/fH3f77bdf/b//+7+7AeC1114b8Morr1xWWVl5kFKKW2655aolS5YcPXbsWOy7776b+NJLLx0EBA900KBB7WJNyWHDhgWt0JuSkpJ5+PDhiL5xRs24UUU0VZVhUWHIdGogCGRlG1DSJrzOjYW/QNoZK+Rd2oNZZwWBXDNMeFUvkHK8++67Az744IMBGRkZGePGjcv46quv+uzbt6/P9ddff/7DDz8c4Ha7U959993+gwYNClmRfN68eali5Z/GxsZe4r8fe+yxoVrbaUqBi7h24BKNvug2YL3FuMJiwADJ5oZ7m3D9oRJcV7QMn35ajE/uLcN8mbW7TOiEMMFKVX6snRy+LUHwIBceEV5nnGUplJRS5OfnHykoKAjwbj/55JM9r776amJRUVHK+++/f+bpp58+EuxaL774YtcvXEpKSibLKkCmeJKZjUDHUqGLbTtBMRCzci6HDfNgzJgKfPppMcaMqcCwYfaInoaL2XmXvlg3OVzsam/6Gnj2W+HVd4wycvxLpc2cOfPMiy++mNTU1OQAgPr6+l6HDx+O9Xq9vRISEjoeeeSR7/Lz84999tln8QDQr1+/dvFYIzG9u82Rx7RFuG5qg8uVA4+nCp98UgaPpwoulzmzS/TCSlXAtSSH6xuh3xEvHYOcdVbY3hEf/DxlfEulLViwYMRdd9115p577vkuKytrzKhRozLmzJlz5enTp2M+/vjjvtdee+3YMWPGZKxcuXJYSUnJEQB46KGHTsycOfPqHhG4Gd2b0P2dg/weJ/DSNcCVp3gBCl+c+YIw+qNLkMu3u73wIt7+3js4cqT7SzpsmAfbtwd2/8zqbutVNs3ocmy+LF1a0pUcXlpaFvJ4/3W6AcEb7hZ7XiotEiwXuDlwuSCOHicw5z7gpXH2WGLVSEzLuVzTWyKQAHDkCLv1pbWiV/qOmWlBapLDrVAZqadgiicZO4zQuB8DBEBsO/D6y/Yo2aUXchV4imao8yRDVh8fMSLgnDmTvpFsv/66qscwBL3Sd8xKC/Iv4BtuQd/2dgccjsDvbkcHQUxMB7gnGRmW8ySTm4HzcUI+4cJ/Rb9ABgvAKI093rZf3SJcwVaKjAb0St8xKy1IbXK42ZWRehKmiGRjP6Bvq/ClXzPZ+JUDjYwahwrAKM33fnu0upxLu1YfDxe9xMEs0SkvXxLgMdbUhB7esFKEPtoxRSQpgLf+BLz5J4AQYOZc4JnvSY/xOIWuI2uMjhqHKnoRbOxx7i6hax1pupQdq4+Hi17iYDfRsVKEPtoxRSSv+q57xcCtLwE/+BoomW5MF1FNpZ5VUwO93VAiLp4TKgCjx3xv/+rjdljjO1z0Egc7ig6vjGQMpsy4SfAZaxPFUhRGd53wxdari6gmaiyO88mtMhjqnORm4Fj/wL+LIrj87/LzvdVO1fRfAdEVPw459+9H1c7RcJ1KxOgTgXlWX1o4UCPHli1zNQmC0iyXESMOR9Hcbetx4sSJmBdeeOHyxx9/XNXiBGVlZcmLFi06kZCQ0MG6bcGwTDK5UV3ESD030Vv0Heebcx9w197gbRTHBlt6CRF8X3xFkPV874Dq46cSUbVzNGoTLVy3zGCsO8vFOjxZ/eSQbfuls2u27d+W8GS19UulscYyImlUF1GxdqKC5yZ6hEC3iLc6gPu+CH0vlxfI/yfQFgMkXlAWQbVjj3LIrhR5KhFLvD18PV0f+BIIoZmSMqXlwdcfHCkK5bb92xIefP3BkVNSel6pNFO62/4EdBHr9YvKigIUdGVAH0SP8M57BbHr2wrEhensi8I/42ugdhjw943dzyPJXeQYDl8CITizRs86u+nOTV8/+PqDIx+a8NDxjTs3Dt5056avZ41WX+Bi9erVh26//fa+YvGJ1157bcCBAwf6fP7553vFUmnvvPNO/2PHjsUOHTr0Uk1NzQGgu1RaRUXFkO3bt38ZqlQaaywhkkoLVNWm6NPtVlOppy1GGDcs3h6eiPuPW865TxDa118WtkONaYaF/zKvP/85kP0v4JuhwFdC6srlG8rx/e97cN11tfjtb5fg1CmN94wS1q5dgHvvfUmyRCwAi1TfsQazRs86+9CEh46v2bFm2MIpC49oEUg5fEulAUBLS4tj3759fWbMmHG2qKjoCrfbnTJ79uymW2+91dSxIkt0t2W7iF7reFkvXSOMK4pDAUC3iCvhK/xiFJ8QYPnNOucufjMUeGAbcKWQBP3973vwhz/k4NNP7T/exqqgQ3a2B/fe+xIAAo9H6Hpv3Xontm6dw8clfdi2f1vCxp0bBy+csvDIxp0bB/uPUWpFLJW2b9++Pfv27dvT0NDwxaJFi06MHz/+4ieffLInMzPzfFFRUcrixYuHBbtOdXV1P7F+ZGVlJfOJu5bwJK2Mxwm8NrZ76qSvFxlMxP3/5vICj+4QxjSLt+uYu/hVKrB5liCU/5iAPzz2X/jxj6vw3/9t7+6kf0EHcW41gIgj3VlZtZgzRwjpV1XloKLCDYDg5Zfv5d3uTsQxSLGLPSN9xlnfbTXXlCuV9tRTTw3Py8v7LjExsaO+vr5XXFwcvXTpEklOTm575JFHvktISOjYuHHjIKC7VNqwYVLNnD59ejPL+pH+cJEMQfmNwBMfSocCnvhQ2B+J0PkHplz1grcZdJ61Wr5KBf4xAfjBP/HH8mLbCyQQvKBDpCLp2532HZcMp/pOT2HH4R3xvoIojlHuOLwjXq1I+pZKmz59etO6desO7d69u09WVtYYAIiPj++orKys37dvX+8nnnhihMPhQGxsLH3uuecOAt2l0pKTky/t2LHjS3ZPGxxTClxMIoTWGX5XdfgHlfy3tVzjiQ+BX92k4dpyY5KA0NXu9CRPXHdA4knqMSZpRImx0AUdIkdM/amocMPtroiyCDcvcBEJwQpccE8yBL5zodUmugcLTIV97ViZ/6p//Uuy2ffG63DzzR5sXpKDB+5+Cx984ML3vhdeVRm1sOwGB4P1kgv+1XY8HhdPBeLIYonAjdXRmugeLDDFOol+0qRaPPBAFT74QO8lRwWMqmvIem61cUuzcuwO726HgZ5TJsO+dpiepD8XLjBppiJ6dIOVMLNyuP2Q7W5/nZmZecoh9x/Wg+no6CC7du26bMKECSPl/s49yRD4jhOWebq7xyxmBOl5baMwssQYL+igmS+OHz+e2NHRQUIf2jPo6Oggx48fTwSgOIeOj0mGQM9E96DXbpR2Lf/xfnPA+TdPlm63dc5DMNLjKixcLrvWilVLjPVk2trafnL06NEXjh49eg24gyTSAeCLtra2nygeQSk13CYKa0hxC2bx8RL7n/+hARYbKzWA0tzczfTcuXjJW37uXDzNzd2sqTkFBStpdna1ZF92djUtKFhJc3M30/r6NNreTmh9fZrmexllwZ7J7LZpt4nUjO92NJr2CwBXAPAA2AtgN4CFXCQZmEqRrK9Pk33b6+vTNDUnO7uaNjYmdYmK/7YdLRqfqdu4SFpJJIcBuL7z3wkAvgSQwUVSo6kUyfZ2Ivu2t7cTzU0SRWTp0uKoEZNofCbBuEiyMs3jEpTSI5TSTzr/fbbTo+xRdbmMXDMnFHoGUnwr51RUuKMinzAan4nDGJaKC8AJoAHAAJm/5QGoA1CXav7PLDPbnAkaXwiKp7otvlDYH/Z1kpIC7Kc/pRIjJNDkLqXXmCQQnV5XtDwTHTBAYhMdDu5JMjJ2FwL6A/gYwF2hjo2m7nZavlQgRUvLD+P8AlCaDalAzh5A6VPxqkUSoLoEUqJx/C6anomLpH7GJA2AENILwKsAKimlr7G4pl1Qs2ZOF7UAqgBM7SyVPrUVeOEs8Km2zCw98gmjcYZKND4Thz2aZ9wQQgiAjQC+o5Tmh3OO3WbcBMOZLyxL60/aaWEphpBkA3iFAP/VB/j3C8BPEoCP4vCz+6RrJf3+94Gnavyv40QRdID0V3nSuXOoa2/nSeMMYOFJTgUwD8B0QshnnXYbg+vagkjXzAEAZGR0W2MGNrzxJLD4PDa88SSmHGnClJHH8bvfQWJynSxOIKwK81odmjxEYjhzRmodhi4oGNWwiG7/N6WUUErHU0qv7bS3WTQuUtSsj60VzasdTm7GXXdVYMOGYtx1VwUmTvTo11gbUFCwqmsVQ5HsbA8KClaFPFesSOR0HoTDQbsqEtlZKHuK6FsaMwZC9QrcVDtBkwqEV7lty1hGhmD/nkbpyRjqdlfTyZMpdbur6XffJVG3236BA1amJZiiVyK9WRYsU4EmJ0vN7+SJADU74BEtFnVVgPSs2MMMYd0j4McngC/6Ysoeb9efJk70YOzYWjz6aM9djEptMVwjKxIZQX29U7aGptebBueU89KdjY2SzUkA6ijlY5IMiLpJ7qzrM+rKH5KAf/UDADzwwCpMnOjBxx+7sHmzIJDhdjO1dFGtiNoEbyMrEhlBampDRPs5OhFN3W0KoWvdrxB03p2BXe+VU/W7LwUodTgCLScnwIYOpRLT2s1kne9nduEHtQneeibSm2FKwwf0m5iQJ/PuNjsz5aZ6j0mu/p78a7VTn/t2mQaR1CIOWs9VupYZSdZa723XikRKz+Iv+rSZUPpoUsiTuUhykeyyzZnC7BZSCjrwMdDC6cL+aqcgjPPmCNMEq51s7rdyauC1qp2dXqpGkQQoXbq0mFIKunRpccTN03Kuv5k1Xc9sL9ZqdkOem6aVg5KnQNPKQTcv7k+p0xnyRC6SXCQphSCQvZ4MnDddOF0QrWKXsK/YxeZ+FCEi6KIwLiGUTidSkVyaTenm8bbwJEVjKbrcVFjmZorCeOnn+0lCN7u4J8lFMkxTmjftKOnuYhe7tHW1fT3VtHxhu9rZee0ZRLj2lQ6hVtmmTYK99zil5xMofe9x2r8/pTNnVtPjx5PozJnVtH9/+VuZNSap5LmtXZsXFYUf7Gj0mmsoveYamlbQK2RdALnPJwUXSZZm6+i20vzoDtK9nrWWtWMqM4G8WcK0Q0qE17xZwLcJnRH0myjcnxC4Dvq9jccygA/+A7j59ygqKsGmTTl48MEqfPihC/n58pHoxYvLVc8j1jIHubY2C1VVOV1tys72YOvWO3HvvS8jJ6cKpaVlyMmpkhzDMYaG+Evy+zs/93Kfz3l3AY/MNLCRPQBbi2Rqk/z+gReU146JhKIZQEucdF9LHPCL/yPkYBZ/SFBxPYUnTSYH71gG8OV0PP74MrzwghsffigI2McfB4pSVVUOnn66ICDVpabGhfJy5XxJMfWnvHxJ17li6k+oc33vIYrg0qUlqKrKwcsv34c5c7barvCDlWenqEnTSm3pJb+/83Mv9/mkBFg7Gfiur6bmcnyxc3dbrpYjSruDN1qNlMp351Ha2X2PjaXVVzq6u9xid9uny/2rXxVLutr9+7MbP2QZhbbi+GMkQRyrp/9E8n8ldk1Eb30AAB7lSURBVLc3zxxB458kirVKFT+fT4HGJYN3txmZKTdlGd0unC6MQaJUGJMpnM4u3UdpzHPI4s5jOtdNqL7SQVfe5IhoTJKVKLEQXKsWno1EWOwwJTHc91kUSVEo5cYcg30+8RQohoGLJCMz5aYsRTJoSo7GaytWHb82htK4OHruHJVYXJxgTzyxkv7gB9U0Lq77cr4eEGtR0iK4Vi88G+57pefaPizN//+KZmcHWgSfTyVvknuSXCQNM0n0cFG3QAYTSV9T+tKzEiWtghtul9bMJO1wfgTs6klqEUkKUPfMQKGMLwRN7wsukozMlJvaSSS7TEYB1Ygky2Rpo7xAPcf75MTX9z0Sn2njxnn07Nl+is9m1zFJmj9Bk0hSyKcB8RQgLpLGGyORZGlGzU7Ry0tTErZlywppY2MS/ctfbqenTiXS/PzVtLExiebnr6anTiXStWvzFK8nCm5j4yDa2DjIMtMTlf6v6NqRmkVSzrhIsrOoK5XGjAEDJJtrygLzjRYtkm6b8FYagl4lyIKVAvvRj/6It98WEv4uXuyNOXNeBwBs3XonXn75Pjz88DrF64rFd/v1a+na19wcj/nz1zNZ70cLNPd+6Y4tW3S5Dy+Vxg5b50lyjCFUCTK1ZdmUSn6lpR3EsGHforx8Cfr2vYhevdrgcgn5pHPmvB5UIAFgxYoiiUACQL9+LVixoijiNnI4XCQ5ISksXI7m5njJvvPne6OwcHlXMnxtbVbE11USX0KADRv+H/Lzn0VZWTEAGlFtSeU6jAdlE7orK3Mtm4TOMR8ukpyQbNkyF/Pnr4fXm4aODoKjR4fg0qU4jBmzH1VVOWFXDvdHTnxF+va9iJaWeHg8Lly6FIfm5r549NH/DGtqpJL4NjYOCZjt9MYbszFnztaoWheHwxYukpyw8F3Le9iwo3j22fyIK4fLXXP+/PWKY7nJycc6u9hbcfvtb+Hll+8Naw65nPg2N8fj5z9fHTAFs6UlHn37XpQcq9Q1j7YK8JwwsWt0W6n6iSpLTQ2wmTOpxMyMjBpp4eRDsk6GV4qenzw5UHX0Pthz+OZdRpKEHmnKFX3yyUAz6D+SR7fZmSk31SqSijNh1AolF8kuYQmVa6hHbqaWHMdIk9z9Bf7IkSGyH1Ol9KZIfiCCiSTTH3kZ4yLJzmzZ3VaqzlM0w5z2RAvhRIW1lGVTQjrmCbS1xaBvX+G+wcYGI11nWwwy+ZaAi49vwfnzvSXHNTfHo7Bwuew11C5S5otSCb7KzIgvxTEAW+ZJOkqFD5c/hAIdS1VcMDVwoP+2cdL8vXfeUXFdm2H2kqyR5jcGy7NMT/cG7C8oWIXa2iyJsGVnezB//nrceOM/kJragIaGVBQWLlfMp4xkuVv6ZHHgzl/+Es58QRj9STsNeJ+VvVTE8DxJdthSJEN9yFZNBbIOS5eT9TiFepJLPgKQnS05b+RBT8C16us1NNCmRCo6Zt/faFH39URralwB2/RPfonh998vex3mP/IycJFkhy2728v/DsS3SvfFtwr7AUEgfSuRe5zCdtZhI1tpP5SiwkpdT9ZEus600etssxpqUCoWrbSfYy5MRJIQcishZD8h5AAh5HEW1wzG3F3A+m2C50io8Lp+m7Af6K5EnnMPUOISXn0rlXPk8c+H9HrTDJ3KF6noGS3qvhXgRcKtAF+ZKfSAHKXAuV5AXJv0774/8hxrEav1AoSQGAC/B/ADAIcA1BJC/kIp3aP12sGYu6tbFOVweTvXoZkGFG/nAhkuW7bMNW1+c2HhctkxSSXRE9u5YkVRWOOJZiEGasRg48l+QK82YFAz8F284EEu/3vwzzPHPDSLJIDJAA5QSr8GAELISwBmA9BVJEPhcXauQ7NdeHXVc6G0OmpEz0xRDxe5bIxLsUD/c8AJRmOQHP1gIZIpAL7x2T4EYIr/QYSQPAB5ADCwH+AZHCSwogWHAx4nRc7dFFWvErgeKIYL9ch56FVU0bvhQjou+0/pJ/P0aY335DDDDqIHAHTP3sCdGfKBGqVVPZX2c6wFizFJuQhaQMiRUrqeUjqJUjop+ZK+gZXa4RAE0is0zYV0VNG7UYtv2dyAw4mAcAM1vuOWznyeN2kVWIjkIQBX+GyPAIKrUUKrvoGVJf/TLZAiLqRjCaayuYGF4fOLzUNJ5EJlY4jn8gRza8JCJGsBXE0ISSeExAG4D8BfQp3kG1hx1/HxQlbU1sqv662mlBknfCoPvikrco/M7ByT7AXEtAOQycYA+CwyK6NZJCmlbQB+CuA9AHsBVFFKd4c6zz+wIna9OdoQc/d8K92oLWUWDRjlWRft+o2syK2d3DnxgQDtMUD8JflINh+3tC4sAjeglL4N4O1wjz8b193F/jYBiKHA9IeAIc3A6vciSIWIlWn+HmlQve/4qwMOuXAh3JbaE9/5xWVlxT1WIIFuz1pulkwktLT47YjPkGw2lEJ2dN5/zovoHfp/xlOb5GeR8QRz8zFlxk1zr26BzJsFHOsPgAivfBxGmXC9ouxsD9zuCpSVFcPtrgirUK0dyM2tjLiCuFGedSRiJucdhjNuyTEHU0RyaLMwBsnHYSIjnPFGuUo34RSqNZNwxC/Sij++sKjcEwo5kSMKZRHkBDXULDKOiZhRn02sJ+m/qLpopDTMunmxsYH25ZcS69OHBhjrGoyR1DPUaqHqGbJaZlav5/JvX27uZtrS0lvyEZGrJallWVsWRYJbWqQmd5B/jUj3TMZ1TyMwXk+SnZlyU1Ek0/LlRTItX+E/Pz5eYu++SwPMXzP1/CxqKRarxXwra9vtufyL9IZb9DaSCuLB7hdOkeAPPww0tQ+sd3FdJeMiGSUiGXGFcYuJpBbvRqvIsFo6wYzn8n2Gjg75j4m/+KltkxrPmqVImmVcJNmZKTf1Xb4hol9ai4mkWu9Gq7iwXDrBrOcSveFTpxJl7+UvfkZ67Vwkufma6fUk5+4SCuV2LBVe7TRQbdd6hqHQ+7l8o+8OR0dYyyeYXcaN04MxQ5lVLwSmwpPUM7Bi1pik3mbkmGR2djVtakqgR44MMSz4Fcqs5EmunApa7ZTuq3YK+4Odxz1JdmbKTcMSyeTkAPvJT6jECAk0o77svvcwMrptlBkV3QaUxwiNeG+fey7QTH/zfazaCZpU0C2U/ttKxkWSnVl3jZvk5IBd8+84JtnesCHwNN/HMXvNFo56Il0UTC3PPRe4z/2ItZaGEatkueuEKbzhFIPha9yww/QxST2JdM0UjnUIZ3nbngIvBmMuUS2SRgdWegpqpgdGCv+B6ybcYjCrpvJCMbpgmTHJ8eMldu21NMDUjKtFY2DFSFM7Q0ar6ZWrmZ8vNdPf4BBW7Qx/TNL3b3xMkp1FtSfJ00a04z9f/JlnfoG+fS9KjtGjG6x1JUQjvF0jqE2RjkGKK4HWpgQe67tK6LcJBjYyyrFO4Gb8eMnmdY6dAed99pl+beIoIxbNqKhwo7h4GYhMOKCjgyAmpoPpfXNzK1WthBgs6DNkiPT83zwbfbGNEhewbB9Av+WBGxZwkeSExdKlJSgpWYbTpxMxcGBgGRsrZQwEy2pYs8Yr2RdtIilGwnutBb49w0WSBVHd3eawQc0MGTPpqUEfUSCrXgGGnzW7NVGEKYGbQYMoffBBifnnjltgzJyp2TXpPDu7mp49G0/z81d3bTc1JdCTJy+j7e2w5LMoBX2OHUszv3E6mu/sHB644YEbW6GlYKzZZGXVorh4GQoLf4XsbA9qalwoLX0KcXGtmDGjGunpXssFwuSCPhcuxKOyUh9v1ypLwS75iOdQ6gL3JPU3NeksrIrnajHfNohzrjdunEfPnu2nS4k2lpabu5keOyZ47seOpdFnntlM58yhzG8Ucbk/g4x7kuyMyUJgnOCoGSNjtYCVFvzb8M47M/Hggy9i06Z5ll9cbMuWubhwQX8PN9gSJHaqaMUJghnKTMhE2rcvlZjZnoeepjYx2ogCu6HM14Nsbyd048Z5prUllN1wg9SMuKnmJUh0Mu5JsjM+JmkAahOjjVjAKhS+HuTmzQ/goYc2GbK4mFHrZWtFaZVEvhRs9MBF0gDUzvyxwtKw2dke3HXXq9i0aR5mznynK3ijR7FfX8JZGdIKKC0Fe8M3gfOoPU5hfjXHZvDutjUtnKUa9E4rMmq5iFD3VxxumLqSwtm974YbKM34YTVNzV1pWHebQn4JkmqnujqQrIx3t9mZKTflIhnaQkW3jSjeYYUIe9CVIZ3VFAVJXUKZ8cNqGluYRDN+WG2oSCpZtVMQxmKXsQJJwUXS9iIJTDT782t7M2OlRqMtmCc5dqxgqdOqaczjSTTpnmLDhSgcK3YJgZxil7H35SJpEZEEUA5gH4DPAWwFMJCLpDFm9EqNRluorr4okmPHUpp0T3FEQmTUWtjVTu5JRoNpDdz8DcA1lNLxAL4E8ITG69kSuUjs2rULsHbtAsk+ltHZaC8oHO7KkM3JHpy6sgJJu4uDFqQVqcwE8mYBBwcClAivebPYz5LxnUdd5ukuYcaL4toQVmoLYA6Ayp7oScp5PadODaCnTiXqFvTo6QWFfbvaqdOq6dixNKzgSFq+fF5jWj7bBqpd5ZCVcU+SnbG7ELANwANB/p4HoE6wVNO/ZKxNbvyMdTK4fzT7t79127JohhobMUJqFOqEyKrJ36yNi6SBIgngfQBfyNhsn2OKIIxJyg+UBVxTvSdp5Wo6cpHYoNHZCJ+7J3uOciKpxozyJM02LpIGimQYIvoQgH8AiA//HHUiaWWh0NuT7AnR7GDGSiStWpCCtXGRtIhIArgVwB4AgyM7T51IWlUojBiTjPZodihjJZIUxkW3jTbf54pLBhdJRqZp+QZCyAEAvQGc7Nz1T0rpw6HPm0SF4cnIaG93wOEIbK8e66tEQkHBKtTWZknmVouR7YcfXte1Lzvbg6ysWpSXL4n4HkpLElAKHDyYFvb6L3ZlxAjp9jeH+MoEvohR+66KROvA17hhhClr3KgVyWBrl1hlfRW9kFvcyhdxoatoEMqkpMB9x0/w73swnPlCOlMXXCSZYasCF3LVdM6f7y2ppmPFSjEs8C2SIfe7pseyrpFgl6o9ZqNXFfOGRDbX4QRiK5H0r6Zz9OgQXLoUhyNHhgOwbqUYVmzZMhfp6V5QBQfBzIWu7FK1J1xWTWVfxUfPRHZemk1HzBgIZZlMboXCtEab1QNYWv8vkpICzeiHqXayr+KjZ/pRQNR+GHjghpGZc1PGM27EXMRXX71T1xxKK1TFAaydCqUmL3TAAKmZ/hCdVu1kO/da70R2Ht3mIilrovfy6qt30o4O6a1YC4dS0YW1a/MMF08rJtWr9SStKpIUbKv4GJnIzvMkuUhKvpTZ2dWGdUGDJY2bVZw2HLNygV6rimS1k60naWQiOxdJLpIUkHZ/jUy2lutSWnls1OoFeq0oktVOfSqLG5XIzkWSi2SAmelJin9jNU/bru+NWrOiSJpdxUercZHkIhlgRnhLwbqUVvYkrTSl0X/Zjr59rSGK0WZcJLlIypre425KXcq1a/N0G5NkEVG3kifpL5C9rt8clfOozTYuklwkLWV6pgaxCAqx8LJZPaO/QKIo3rIVeaxeCCNY+7hIcpHsUcaiK6/Vy2YVwfcVSbIozbK1Ha1eUi1U+7hIsjNLT0vMza1Efb0T7e0O1Nc7kZtbaXaTTKGmxoWKCjdKSpahosItqTYULuKUxpiYDqSneyMuhCGuMVNVlYOlS0tQVZUjWYNGDXSA/DRKK8xDLprhU1Gnk5Y4Yb8VsHr7ognLiWS3MBJs3jwPTudBOBwUTudBPP98Xo8UyuxsD9zuCpSVFcPtrggoJGEUasQ6Lk5qLedJl6U1UdlzrDAPWUmorSDggPXbF01YSiTFcmCCMCKgdqTZlW7MQCwUkZNThdLSsi5vzgyhVBJrtRWAlv8diG+V7otvFfabjZJQW0HAAeu3L5qwlEiuWFGkWC9RRM9KN1bs3oe7tKreBBNrtRWA5u4C1m8D0k4DhAqv67cJ+83GygIOWL99UYWVAjdK+Xy+plfaipWLRljBQkW3lYJLcXFSC3UjKyVx8+g2t85PrHVEUimfTzQ9RctKuYR2tWXLhBlHy5YV09hYSmNjQ4uiv1U79ZkO2NOMi2SUiqScN9feTmh7O3SvdGOlWSl2NNGTXLZM8CRnzKhWJZIU7AtL9ETjIhmlIgmYVwIsmjxJo+teigIpCuOMGd3bai/KskRZTzQuklEskmZZNI1JGl26TRRlsYstCuVjj61UdcFqJ/cktRoXSS6SupgVC9mqNTMKbviKZE8Zk7RqcIeLJDuz1ZKynMhYurQEJSXLUFZWjNLSMubXj42Vbl9q076C6aqpQNZhwOXt3udxArUpwJKPNF+eKQFrXUNIw7FCGtMkAHVKK8ZxIsJSeZJmYsUcSS1YZZZOpCz5SCqQgLBtNYEE+NTAngIXSfjP9LHHFMhgom6lWTrRDJ8a2DPgIgn5mT5WngIZStTVzNKJNk/aCPjUwB4CD9zYL0eSdbpSOJF9QgKNwlozZIw2K5dT44EbdsbEkySELCaEUEJIEovrGU1DQ2pE+/UgkiIRSvPX1c5r1+JJZx0Gcu4RgiuA8Jpzj7A/2rHy3HMOQ7R7hbgCwHsADgJIsqMnaYUcyUhyG1l7kuF40kqeJAXPazTDQqUecU+SnWm/APBnABMAeFmJpNEzRgBr5EiGm9vIWtTDEd1gIknBZ8gYaeF087lIWkQkAdwBYE3nv5mJpNEzRqxk4S5Ly1LU1YxJIrN7Aa8hi0ETnuCepFGWlh+45IX/shdcJA0USQDvA/hCxmYD2AEgMRyRBJAHIYO87vLLB4X8glt5idZQptYTNvOZu0UX9NKlGNrRIXiS99+/OcBrlPNkehcJ+6udXCj1NlIqL5KktPsYLpIGimQQ0csE0Ngpjl4AbQAaAAwNde511zkku+S6igUFK+nGjfMopd1eld5dblamxhO2gves5FHef/9myYGhPJlqp3Wj26yj8WZMS+SepE1EUkY0w+5uT5wYuNs/6JCfv5q2txO6ceM82tiYRPPzV9vKo4zUKzRjHNbfgo1N+h4YjidjVat2spsbblYKEB+T7KEi6RtJFQVGFMaNG+fR9nZC8/NXm/0di8jCHV+0igWLcvseGI4nY2WrdrKJxmt9H7R4tTy6bZwxm3FDKXVSSk+oPd83J1GcMfLssz9HRYUbDz74IjZvfgC9erUxaasR2HHutFJeqKOBSrbtvr6Kywu464Bl04RX/7ni4aJ1WqKWHNO5uwDvs0DHUuGV52bqiBnKHM6YJGDf4I0VxhfVmNyYJD0HSnMDD7ZqibBwrNppDU+SZVv8jXuS7MyUm15+eXrY0W27CQ1gjfFFtZabu5nSelDaDuFVRiDtbNVO641J6pFjykXS5iIZzowbOwuN3c30BuhoVotuVzu5J2l140V3OQFQWK9Wq52K8YaLOAZZ9YrwXP7bWuBFd9nBS6VxQEEkZkWisZBGbYpUEF1eYbs2xcxWcfzhniTHEsIYjqcoCqO7DqiYxMbjila4J8kO7klyLEE4niKr1B0OJxK4SHIsgdjVzLkHKHHJj815nIIHWbxdeBUFlcPREy6SPQz/8UcrdLVFgnmKvkGNMk+3oHKh5OgNF0mTiKQSeU8hmKfIgxwc07BqnqRe9uabMwPmgOfnr6ZvvjnT0HaYlSxv2hsfwqqd7JK8ufE8SaZ61dNEUqwuJAql/7YeppQYv3ZtnuHTLs3+8iqZ1iTvnrwgmZxxkeQiqclEYdy+/SZDqgsF8xqNrhRk9pdXL6t2ck/U17hIcpHUbNu330QpBd2+/SZD7idXrMOIAh6mv9EGWrWTL0gmGhdJLpKazGhPUjRfr9GoMUmzv6xGG1+QTDAuklwkVZsZY5JAoCe5dm2eIQU8zP6yGmnVTu5JisZFkoukajMjum1m2Tezv6xGWbWz54xJhhOk4iIZ5SIZbetuG/U8Zn95zbSeFN2udob+QeAiGeUiabTnJVeRu72d0PZ2MBdMPc30BnAzzKqdwYcWuEhGuUgCxi7doLRKoGhKy0tYzUxvADdDLViQioskO7PstMSaGhcqKtwoKVmGigo3hg37FvX1TrS3O1Bf70RubiWze6WmNgT9e79+LVixoojZ/ZiSWQnkO4FSB5z5QGWm2Q0Kn8pMwJkPOEphu7abDS/2YSB28CSbmhJoS0tv3by7UJ4kpdIlby1jmZspCuMNX/eZhZm1ZnU0WLWTj0n2eE8yO9uDqqoc5ORUobS0DC0t8ejb96LkGJbeXWHhcjQ3xwc9Rmm5VTNJm/EAENci2dcSBxTNYH8v1l5f0Qyhrb7o1XarwOo95MU+jCXW7AbIIa67XVPjAgAkJzfKHheqmxwuW7bMBQCsWFGE1NSDAAgcDtr19+bmeBQWLmdyL5ZoXfc5XCozgbxZ3aJ2cKCwDahf79motqulMlMQ7IZEILVJWFNcy9rWLN9DuTV9XF5ehFgvLOlJlpcv6RJIQNmLY+ndbdkyF+npXsTEUDzwwIvwetPQ0UHg9aZh/vz1XUJqJVKbItuvFj28PqPargZR0A4OBCjpFjQt3nNP9JyjBUuKpD9y3WE9vbtuwexAerrXkgIJCN5NfKt0X3yrsJ8lenh9RrVdDXoImtU9Z44ythDJLVvmYv789bbw7vRCrqL43F3A+m1A2mmAUOF1/TbheJbjh6y9PrEr29ILiGkH4NN2LV1aVoQjaJGOL1rZc+YEJ+pXSywoWIXa2ixJ9z0724OsrFqUly8xpA0sCHeZBf+xL0Dw0LQIEMtr6tE+1jjzhS62P2mnAe+z6p7B6OfmqyWyQ7MnSQj5GSFkPyFkNyHEcmsP1NZmoaoqp2upBDFyXlubZXLL9EGPrqKSx6rmy22HsblQQwFqniGc93DV1MB8R49T2M8xD03RbUKIC8BsAOMppRcJIclsmsWOmhoXcnKqUFWVg4oKN9zuCknkPNrQa+xr7i42Ho8dxubE51SKbqt9hlDvobisrpje47v4Gcc8tKYAuQH8mlJ6EQAopfK5OibjO3unrKw4agUSEL7Qcl1Fq4x9Wb19IsEETa9n8F1W110nzKTxX1aXYzyaxiQJIZ8BeAPArQAuAFhMKa1VODYPQF7n5jUAvlB94whJTERCejpGnjyJ44MGYXB9Pb5uasJZn0OSAJwwqj1hoL498bgciUiDdCilA004iBZ8Z0qbfGHXPvP+z+SegYLiDLwa32OBARiOfhiGZhzBGXyr8iqjKaUJmtvCCe1JEkLeBzBU5k9FnedfBuB7ALIAVBFCRlIZ5aWUrgewvvOadZTSSVoazhLentBYrU28PcEhhBgTGe0BhBRJSuktSn8jhLgBvNYpiv8ihHRA+IU/zq6JHA6HYx5ao9uvA5gOAISQUQDiYK1uK4fD4WhCa+DmDwD+QAj5AkArgIfkutoyrNd4X9bw9oTGam3i7QmO1dpjW0xJJudwOBy7YItpiRwOh2MWXCQ5HA4nCKaKpBWnNBJCFhNCKCEkyeR2lBNC9hFCPieEbCWEyKQvG9KOWzv/jw4QQh43ow0+bbmCEOIhhOzt/MwsNLM9IoSQGELIp4SQN81uCwAQQgYSQv7c+fnZSwi5wew22RnTRNJvSuM4AE+b1RYRQsgVAH4AgE01X238DcA1lNLxAL4E8ITRDSCExAD4PYCZADIA5BJCMoxuhw9tAH5BKR0LITf3P0xuj8hCAHvNboQPawC8SykdA2ACrNU222GmJ2nFKY2/AbAEgOnRLErpXymlbZ2b/wQwwoRmTAZwgFL6NaW0FcBLEH7YTIFSeoRS+knnv89C+PKbumgBIWQEgB8CeMHMdogQQgYAuBnABgCglLZSSk+b2yp7Y6ZIjgJwEyFkByFkOyHE1LI8hJA7ABymlO40sx0K/BjAOybcNwXANz7bh2CyKIkQQpwArgOww9yW4FkIP6wdJrdDZCSEyRx/7BwCeIEQ0s/sRtkZXde4YTWl0aD2FAL4N73uHWl7KKVvdB5TBKGbyW4N3fCRq0doupdNCOkP4FUA+ZTSMya243YAjZTSjwkh2Wa1w49YANcD+BmldAchZA2AxwEUm9ss+6KrSFptSqNSewghmQDSAewkhABC1/YTQshkSulRo9vj066HANwOYIaePx5BOATgCp/tEYDqggtMIIT0giCQlZTS18xsC4CpAO4ghNwGoA+AAYSQzZTSB0xs0yEAhyiloof9ZwgiyVGJmd1ty0xppJTuopQmU0qdlFInhA/a9XoKZCgIIbcCeAzAHZTSllDH60QtgKsJIemEkDgA9wH4i0ltARF+wTYA2EspfcasdohQSp+glI7o/MzcB6DaZIFE52f2G0LI6M5dMwDsMbFJtsfMJWXVTmnsKfwOQG8Af+v0bv9JKX3YyAZQStsIIT8F8B6AGAB/oJTuNrINfkwFMA/Ars4yfQBQSCl928Q2WZGfAajs/GH7GsCPTG6PreHTEjkcDicIfMYNh8PhBIGLJIfD4QSBiySHw+EEgYskh8PhBIGLJIfD4QSBiySHw+EEgYskh8PhBOH/AyhuvNoHzzSkAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plot_boundaries(w1, w2, w3, w4, w5, w6, b1, b2, b3)\n", "plot_data(X_train, y_train, X_test, y_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The learned classifier does a good job at predicting labels both for the training examples and unseen examples (test data). \n", "\n", "--- \n", "\n", "In addition to the decision boundaries in the original data space, we can also visualize how the data are transformed through the neural networks. Since we use a hidden layer with two neurons, we can visualize its \"output\" in two dimensions. \n", "\n", "En plus des frontières de décisions dans l'espace original des données, nous pouvons aussi visualiser comment les données sont transformées à travers le réseau de neurones. Nous utilisons le fait que la couche cachée utilise deux neurones et donc nous pouvons visualiser sa sortie en deux dimensions. \n", "\n", "(for better visibility, we changed the color of the yellow class to blue.)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAApAAAADmCAYAAABiUbWlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydeZhcVbW3319CGAIhgSQqU7qZ1AsyKBEHUFD0Csh0+RDFJoYx4nAVxctguElAW5zuJaAoBgQxaWZFAVEZFBAUJEEgDCKD3SE3ATJAEiBChvX9sXclp6tPzVVd03qf5zx9zh7XOdVVtWrtvdaSmeE4juM4juM4xTKk3gI4juM4juM4zYUrkI7jOI7jOE5JuALpOI7jOI7jlIQrkI7jOI7jOE5JuALpOI7jOI7jlIQrkI7jOI7jOE5JuALpOI6TA0kfkPS0pFckHVJveZJI2klS0XHYJM2SNK2GIjmO00a4Auk4TkMRlbXMsVbSysR11yCL803gfDPbzMxuHuS564akeyQdV285HMdpXDaotwCO4zhJzGyzzLmkXuAkM7s9V3tJG5jZ6hqJ0wE8Vk7HGsvlOI5TV9wC6ThOUyHpm5KukXSVpBXAsZLeJ+k+SS9LWijpQknDYvsNJJmkz8bl6JckXZgY762S7pa0TNJiSVfG8l5gHPDbaP0cKmlbSTdLWirpKUknFJDrm5KujmWvSHpY0o6Szpa0SNI8SR9JjDFK0uXxHuZLOlfSkFg3VNL5kpZIegY4sMBz2kvSQ5JWSLoK2ChRN1rSLVGGlyTdJGmbWPcd4H3AxVHm6bH8h1Gm5ZIekPT+il5Ix3GaGlcgHcdpRv4DuBIYCVwDrAa+DIwB9iEoV5/N6nMwsBfwToJyl1HcuoHfAFsA2wIXAZhZJ7AAOCguYa+Jc/0T2Br4JPBdSfvlkQvgcOCnwCiCNfP2KO9WwHnAjxP9ZwErgR2B8cDHgeNj3eeAfwf2APYGjs71cCRtBPwauAzYMp4fkWgyBLiEoCB3AKuAC+J9nwH8BTgl3vepsc/9wO5xvOuB6+I8juO0Ia5AOo7TjNxjZjeZ2VozW2lmD5jZ/Wa22syeBWYA+2X1Oc/MlplZL3AnsGcsXwV0AluZ2b/M7N60CSVtT1DczoztHgQuBybkkiuW3Wlmt8fl7OsICth34/XVwE6SNosWwAOAr5jZa2b2PDAd+FQc52jCfsz5ZrYE+Hae57MPYMAPzGyVmV0N/C1TaWaLzOyG+OyWA99KeV79MLOZZrY0yv1dYHNgp3x9HMdpXVyBdBynGXkueSHp7ZJ+I+l5ScuBcwnWyCTPJ85fAzJ7LU8DhgGzJc2VNDHHnFsDi83s1URZH7BNLrkiLyTOVwKLzGxt4pooSwdhmfmFuBT/MsEa+ubE/Mnx+3LImWk738ySXtrr2kvaVNKlcQl9OfAHBj6vfkg6XdLfJS0DXgI2LdTHcZzWxRVIx3GakezwNT8BHgV2MrPNgSmAihrIbKGZnWRmWwFfAGZEa2M2C4AxkjZNlI0D/i+PXKXwHEGx3dLMRsVjczPbPdYvBLbLmjsXCwnL8UmS7U8Htgf2js/rw1lt+92HpA8BXwX+H2EpfgvgFYp8xo7jtB6uQDqO0wqMAJYBr0r6Nwbuf8yJpKMzDiTAywTlaU12OzP7JzAb+JakjSTtSdif2FOp8HH854C7gO9L2lzSkBjr8YOxybXAqZK2kTQaOCPPcPcAQyR9MToRfQJ4V6J+BEFZfSmONSWr/wvADlntVwOLCdbaaQQLpOM4bYorkI7jtAKnAROBFQRr5DX5m/fjPcADkl4Ffgl8wczm5Wj7SWBnwnL49cDXzeyPZUs9kGMJitnjhGXi64C3xLofA3cAc4EH4vypmNnrBIeek+M4RwK/SjT5X4KjzxLgz8Bvs4aYDhwTl9L/F7iF4PzzFNALLCdYOR3HaVPUf4uM4ziO4ziO4+THLZCO4ziO4zhOSbgC6TiO4ziO45SEK5CO4ziO4zhOSbgC6TiO4ziO45SEK5CO45SFpM6YY3qDesvSSChwecwx/dci+/xM0jerNP+dkk7KUTcuk9c7R/00SbPyjN2bzN1dLap5/47jDA6uQDqOU1BxiG1qojxUiqTjJN3TQOPtC3wU2NbM9q7B+GVjZvMSeb0dx3HKxhVIx3Gc6tIB9GalPHQaDLecO05luALpOG2CpAskPSdpuaQ5kj4Qyw8Evg58Mi5vPpzSdyYhFd5Nsc3piequmFN5saTJiT5DJJ0p6RlJSyRdK2nLPPKdLOlpSUsl3Shp61g+YKk8s0wbs85cDLwvyvVyrP+ZpIsl3SZphaS7JHWUO16KrFtHGZdGmU+O5ScClyb6n5PVL9/4W8R83isk3S9px0S/t8d7WSrpSUlH53qOkQ5J98axbpU0Ju3eJW0fn80KSbeRldta0gRJffH1m5xVl/P1TcwzMe1/Ix+StpB0s6RFcRvAzZK2jXWfkDQnq/1pkn4VzzeS9P045wvxf2CTWLe/pPmSzpD0PHB5MfI4jpOOK5CO0z48AOwJbAlcCVwnaWMz+x3wLeCauLy5R3ZHM5sAzAMOjW2+m6jeF3gbcAAwJSpJAF8CjgD2A7YmZES5KE0wSR8GzgOOBrYC+oCrC92QmT0BnAL8Jco1KlHdBXyDoBQ9RBEpBwuMl+QqYH68r6MI6Q0PMLOfZvWfWsL4xwDnEPJMPw10Ayjk3r6N8Jq9Kbb7kaRd89zKpwlpFt8EbAh8LUe7K4E5hGf0DUI2H+K8uxCy30yI9zma/vm1i3l9c/1v5GMIQbnrIPxoWQn8MNbdCGyfNc6xwMx4/h3grYT/852AbeifpvEthP//DmBSEbI4jpMDVyAdp4mRNF7SVyR9T9K5CnmdU618ZjbLzJaY2Woz+x9gI8KXe6WcY2Yrzexh4GEgo4B+FphsZvNjar1pwFFKXzrsAi4zswdj27MIVrrOCuT6jZndHcebHMfbroLxAIhj7AucYWb/MrOHCFbHCRUO/Usz+6uZrSYou3vG8kMIS+KXx9fuQeAXBMU1F5eb2T/MbCUhh/ae2Q0kjQPeDfy3mb1uZncDNyWaHAXcnHiG/w2sTdQX8/rm+t/ISfwf/YWZvWZmKwiK9H6x7nVCmspj4z3sCnQCN0sSIXXjV8xsaez7LeBTieHXAlPj/a4sJIvjOLlxBdJxmhAFR4wHCYrWJsCTwIsExeY2SVdEBSHZ5zRJT0haFpdOR5K1ZFkmzyfOXwM2i+cdwA0K+ZRfBp4A1gBvThlja4LVEQAze4WQp3mbCuR6Lmu8pXGeStkayCgoGfqoTFbI/xzfk3mO8Vl2sT5HdiljJdkaeClrr2ZfVn3yGb5KeE0yFPP6FiNHPyQNl/STuHS+HLgbGKX1nuNXAJ+OCuME4NqoWI4FhgNzEjL9LpZnWGRm/yokg+M4hfFNxI7TnGwK7JPLiiJpT2BnwrIzCvsdzyAsJT5mZmslvQQodrEi5iymTZLngBPM7N4i2i4gKCREeTclLJn+H5BRcIYDy+N5UnnKJdc6a6OkzQhLlwuAjAJR6nhJWbeUNCKhRI6LshZDOc/xLjP7aIn9CrGQsO9y04QSOY718i0E1i0VSxpOeE2ScqW+vhVajk8jWMbfY2bPx//lvxH/V83sPklvAB8gLNV/OvZbTFju3tXMcr0WpT57x3Fy4BZIx2lCzOyifEtwZvaQmd2RKBoBrAYWARtImgJsnqh/AeiUlO8z4QVghxLEvBjo1nrnlbGSDs/R9krgeEl7StqIsPR4v5n1mtkignJ2rKShkk4Adkz0fQHYVtKGWWMeLGnfWP6NON5zFYwHgJk9B/wZOE/SxpJ2B06kiD2WxYyfws3AW6NDy7B4vLvI/YQ5MbM+YDZwjqQNJe0LHJpocj1wSOIZnkv/74xSXt9SGEFQBF+O2zGmprT5OWFf5Gozuyfez1rgEuB8SW+KMm0j6WNVkMlxnCxcgXScJkXSxySdmG3tiQpRNr8Hfgv8g7BM+S8Sy5PAdfHvkrg0nsZ5wNlxeTCXU0aSCwhOD7dKWgHcB7wnrWFUdv+bsLdvIUGhS+5dOxn4L8IS6q4EBS7DH4DHgOclLU6UX0lQPpYCexGWfSsZL8kxhL13C4AbCPvqbsvRNptixl9HtHL+O+F5LCAsC3+HsIe1Uj5NeE2WEp7VzxPzPgZ8gfAcFxKcZOYn+hb9+pbIdMK2jMVxzN+ltJkJvIP1zjMZziA4IN0Xl79vpzr7fB3HyUJmbtF3nGZD0nnAPsCDBKvRdDP7Qax70MzeVU/56o2knwHzzezsesviVJ8YmudF4F1m9lS95XGcdsQtkI7TnBwCfNjMTiVY1w6SdH6sU+5ujtMSfA54wJVHx6kf7kTjOM3JBjHcC2b2sqRDgRmSriPE/XOclkRSL+FH0hF1FsVx2hpfwnacJkTSzcD3zOyurPJvAl83M19dcBzHcWqGK5CO04Rk0rOleWJL2iZPGBPHcRzHqRi3UjhOExKze6yEdaFK3i/pg5I+SP+QNE2NpKskDepSpUIe7W/mqX9FUmo4oxjg/Z48fe+UdFI15KwWkj4g6cky+/5SIZe64zhthiuQDYakr0u6tNptixjLJO1UZNtpkmZVY16nMiR9B7gXOJsQlua/yJ33uKmI8RX3AH4dr/MqZ4NFzGH9bL3lqBZm9iczWxfqRlKvpI8U2f3bxJzdjuO0F+5EU0MkHUfIqrAjIePFDcBZZvZyrj5m9q1ixy+lbb3wcCo15wjgbTGVW6vxWaDHfJ9Nw2Jmf5W0uaTxZja73vI4jjN4uAWyRkg6jRDs978IOYffS0jVdluuDBSSXKF3SuVZYFi9hagRBwF3AcSsKxcD74tLyC9L2j7+HRLbXCrpxUxnSbMknRrPt5Z0o6Slkp6WdHKBubeQ9BtJKyTdL2ndtoCktV7S6Djuckl/JWv7gKSPSvq7Qv7xH5IVYknSCQr5yV+S9PtMVpfEPKdIeirWXyQpNURT9rK7pP0lzU9c90r6mqRHoizXSNo4u62kmYR0hjfF53y6QradWZKWxOf9gKRkvus7gY8XeJ6O47QYrkDWAEmbA+cA/2lmvzOzVWbWCxxNUCKPje2mSbo+fjgvB47LXh6W9BlJffHD+7+Ty0vJtpI64xfOREnzJC2WNDkxzt6S/hK/ABZK+mEuRTblfraXdFf8Mr0NGJNVf52k5+MX092Sdo3lkwjZP06PX0Y3xfIzJT0Tx3tc0n+U96TbF0k/kHQh8BrwkKSfSLowc9RbvkpRyIW9PfAkgJk9AZwC/CUuIY8ys38SLPvvjN0+ALyi9Sn+PkhUQIGrCFlUtgaOAr4l6YA8IhxDeA9vQchskmuZ9iJCVp+tgBPikbmHMYTMOmcT3jPPEIK/Z+qPAL4OHAmMBf4U5UxyCPBuwlL+0UAlafmOBg4kPNfdgeOyG5jZBEL+9EPjc/4uMJHwI3g7Qi7sUwipBjM8EeVzHKeNcAWyNrwf2Bj4ZbLQzF4hpJP7aKL4cELO2VFk5dKVtAvwI4ISthXhQ3ybAnPvS0jddQAwJfFlugb4CuGL7H2x/vNF3s+VwJzY9xuEL5QkvwV2Bt5EyIzSA2BmM+L5d+OXUSbP7jOEL/uRhC/pWZK2KlIWJzCb8JrcSHhN/hyvM0ezMyr+XVGg3V3AfpLeEq+vj9fbE3J9PyxpO8L74gwz+5eZPQRcCkzIM+4vzeyvMdZmD7BndgNJQ4H/B0wxs1fN7FHgikSTg4HHzex6M1tFSNH3fKL+s8B5ZvZEnOdbwJ5JKyTwbTN72czmAX9Mk6MELjSzBWa2FLiphLFWERTHncxsjZnNMbPlifoVrH+9HMdpE3zJtDaMARZnAj1nsZCQOSTDX8zsV/F8ZdYK1VHATWZ2D4CkKcCXCsx9TvTOfVjSwwTLwBNmllQqeiX9BNiP8KWWE0njCBaQj8R9dndnLIkZzOyyRPtpwEuSRprZsrQxzey6xOU1ks4C9iY6SziFMbMrCrdqajL7hEcQLHy5uAs4jGBdvJuwnDoh9vmTma2VtDWwNOaUztAHjM8zblLRew3YLKXNWMJnaDKneF/ifOtknZmZpGTbDuACSf+TKBPhR2JmnGLkKJbssbYust9MgvXxakmjgFnA5KgUQ3iNcu7rdhynNXELZG1YDIxR+p7GrWJ9hudS2mTI/gJ6DVhSYO7ULxxJb5V0c1xqXk6wdoxJGyBFhpfM7NVE2bovSUlDJX07LkkvB3pjVc6x47L8Q3E5/WXgHUXK4kQk3STpUEkD9j9K2kHSuZJOSOvbDMT/t2eAtyaLU5reRbBm7x/P7yEsE+/H+uXrBcCWkkYk+o0DKo2VuQhYTVCukuNmWJisi/sXk22fAz4bl+MzxyZm9ucyZHkVGJ64fkuuhkXQ7znHLTjnmNkuhNWVQ4DPJJr8G/BwBfM5jtOEuAJZG/4CvE7Y27SOuK/rIOCORHE+D9OFwLaJ/psQlpLK4cfA34GdzWxzwt6rYnImLyQ4FGyaKEt+SX6asAz/EcKSdGdG3Pi33/3F5blLgC8Co81sFPBokbI46zmZoDj9PTo13CLpD5L+CfwEmJO0DDcptxAUwQwvANsm9+7GXMgrCfuK745Lqy8Qlpbvim2eIyzxnxcdQnYHTiRry0ipmNkawjaVaZKGxy0nye0dvwF2lXRk/DH5JfordhcDZyX2DI+U9IkyxXkIOFjSlnE5/9Qyx4Hw/NbFuZT0IUm7xSX75YQl7TWJ9vsRtrE4jtNGuAJZA+LS7TnADyQdKGmYpE7gOsJS28wih7oeOFQhSPSGccxyFa0RhA//VyS9HfhcMZ3MrI+w3+4cSRtK2hc4NNFkBEFZXkKwgGSHFur3ZQRsSlAqFwFIOp5ggXRKwMyeN7PTzWxH4BOEfZBfBXY1s4+aWStsB5gBdGn9vo4/AI8Bz0tKWvHvApbEfYKZawF/S7Q5hvDjZgEhnNZUM7utCjJ+kWDlfx74GXB5psLMFhNem28T3h87E2J2ZupvIERquDpa7x8l/MAsh5kEK2AvcCtwTZnjAJwHnB1XCL5GUHqvJ3x+PEF4vhnnvXcDr5rZXyuYz3GcJsRTGdYQSScSHFcycSB/BZxpZi/F+mmEjenHJvr0K1OIJXkuQfGaTvCA/JSZ/SnZNiqo/wSGZfZeSroTmGVmlypkKJlBsGj+jbAh/8Nmtm9sawTr5NMp97EDwTngnQTr6pPAqDjvZgRLzoeBpcB/x7Y7m9nTknYmKM6dwJ1mdoSkboICuxb4OWFP6Ewzq0pQdKd1kHQlcG1in7DTQEj6BfBTM7ul3rI4jjO4uALZRERl7WWCcvbPesvjOI7jOE574kvYDU50lBge9yB+H5jLekcVx3Ecx3GcQccVyMbncMK+rQWEPVSf8tRujuM4juPUE1/CdpwmRtI+wDRCTMENCM4jZmY75OvnOI7jOJXgCqTjNDGS/k5w1JpDIrSKmRWKF1oTxowZY52dnfWY2mli5syZs9jMxtZbDsdxiqclM9H4l5hTDk36JbbMzBomBl9nZyezZ8+utxhOkyGpr3Arx3EaiZZUIP1LzCmHJv0S+6Ok7xECWr+eKTSzB+snkuM4jtPqtKQC6ThtxHvi32ReZyPE5XQcx3GcmuBe2I7TxJjZh1KOgsqjpMskvSjp0Rz1knShpKclPSLpXZXKKg08AHp6oLMThgwJf3t68pcXotx+rU7P3B46p3cy5JwhdE7vpGduT1F1lcylc5R6OI7T/LgFsgA9c3uYfMdk+pb1MVRDWWNr6BjZQfcB3XTt1lVv8Zw2R9JIYCrwwVh0F3BuTKeZj58BPyRkAkrjIELYqJ0JVs4fs97aWYacucuHD4fXXgvXfX0waRLcey9cccXAcoCuPG+7np7QrtR+rU7P3B4m3TSJ11aFB9O3rI9JN01aV5+rrpzPuOy50tA5wqa6A6fjNDMt6YU9fvx4q8YeyHwfhMOHDWfGoTNciWwhJM0xs/GFWzYOMZXco4T0kQATgD3M7Mgi+nYCN5vZgFzkkn5CSD15Vbx+EtjfzBbmGzPXey+XApmLoUNhzZqB5R0d0Nubu19nZ1AaS+3X6nRO76Rv2cAH0zGyAyBnXe+pvVWbK5ukAtmM7z3HaXd8CTsPk++YnPNX9GurXmPyHZMHWSLHGcCOZjbVzJ6NxzlANWJAbgM8l7ieH8sGIGmSpNmSZi9atKgKU6crjwDz5uXvl6u+UL9WZ96y9Acwb9m8vHXVnMtxnNbCFcg8FPog9A9KpwFYKWnfzEUMLL6yCuOm2QxTlyvMbIaZjTez8WPHVicK0tCh6eXjxuXvl6u+UL9WZ9zI9AcwbuS4vHXVnMtxnNbCFcg8FPog9A9KpwE4BbhIUq+kXsK+xs9WYdz5wHaJ620J6TSrzvDhA68nTUov7+7OP1Z3d3n9Wp3uA7oZPqz/gxk+bDjdB3TnravWXI7jtB6uQOYh3wdhJR+wg0G1vSqblVb2yJU0BHibme0B7A7sbmbvNLNHqjD8jcBnojf2ewkBy/Puf8zHrFm5y2fMCHsUpfB3xgz40Y/Syws5wnR1ldev1enarYsZh86gY2QHQnSM7Fi3hztfXaVz5cIdaByn+XEnmgKkeWGP3mQ0AEtXLmXcyHEN55Gd5vzTbE4/PT0weXLYuzZuXLAglaoEZHvkQrBG5VIomnEjv6S7zeyDhVsO6HcVsD8wBniB4Mk9DMDMLpYkgjXzQOA14HgzK/imyvfe+/znw7NfsyYsUU+aFBRFx2nG957Tekh6M3Ad8E5ghpmdVmeR+hFXmU4ys9vrLQu4BbIgXbt10XtqLzbVWD1lNbOOnMXK1StZsnIJhq0LeVGtuGnFWAwLtU1z/mkmp5+M4tfXB2brQ7GUaj2cPLm/8gjhenJzPIZiuU3S1yRtJ2nLzFGok5kdY2ZbmdkwM9vWzH5qZheb2cWx3szsC2a2o5ntVozymI+enhCWJ+Mcs2ZNuG4li7DjNDtxK8xKSSskvSzpz5JOiasdxfTvlGSSyg4RGPvvVG7/CpkELAY2bzTlsRFxBbJEqqGcZSuAn//N55l00yT6lvUVVEoz1sV8bavtVTnYVEvxaxOP3BOALwB3A3Pi0XB5PNtEmXecVuBQMxsBdADfBs4AflpfkdZTiXJaBB3A41bG0myN5WpIXIHMQS4rX6XKWZoCePHsi4tWSotRYKvtVTnYVEvxawePXDPbPuWoRhifqtImyrzjtAxmtszMbgQ+CUyU9A4ASR+X9DdJyyU9J2laotvd8e/Lkl6R9D5JO0r6g6QlkhZL6pE0Km1OSZn+D8f+n5S0v6T5ks6Q9DxwuaQtJN0saZGkl+L5tolx7pT0DUn3RmvqrZLGxLqNJc2K8rws6QFJb5b0M2AicHqc+yOSNpI0XdKCeEyXtFEcJ02uTNnpMdPXQklHSDpY0j8kLZX09YScQySdKemZKM+1yRUkSRMk9cW6hvu5XXcFMprM50p6SNIAy0ncxF/VlGqFyGflq1Q5S1MALT06SqpSWowCW22vysGmWopfO3jkSvpM2lFvubJpB2XecVoRM/srISrDB2LRq8BngFHAx4HPSToi1mX2Y48ys83M7C+EkGDnAVsD/0aI7jAtx1yZ/nvE/tfE67cAWxIshJMIusvl8XocIXTZD7OG+zRwPPAmYEPga7F8IjAyyjGaEMlipZkdB/QA341z3w5MBt4L7AnsAewNnJ2YI1uuTNnGhLi5U4BLgGOBveIznCIp8yP/S8ARwH7x+bwEXAQgaRdCBrAJsW40IRpGw1B3BTLyITPbM8cm6mRKtUmEB1pT8ln5KlXOSllGTlNKcymqW26yfttbtb0qB5tqKX5t4pH77sTxAcIH82H1FCiNdlDmHaeFWUBQlDCzO81srpmtjREfriIoQKmY2dNmdpuZvW5mi4D/zdc+B2uBqXGMlWa2xMx+YWavmdkKoDtlzMvN7B9mthK4lqAEAqwiKGM7mdkaM5tjZstzzNtFSA37YpT9HIJClypXYvxuM1sFXE1wVLzAzFaY2WPAY4SoGRBCrk02s/lm9jrh8/uouBx+FCFT2N2x7r/jfA1DM6zZHw78PO5JuE/SKElbVRJSpBD5rHwZJWzyHZOZt2xeyV7Y40aOS03zJdTPEplLKe0+oJvjf3U8q9au6le+4o0V9MztWSdHJjxHM5JR8Cr1ws6M1WIKYz/M7D+T1wq5sWfWSZycdHWF/NZJL+yJE1v7tXGcFmIbYCmApPcQ9ka+g2DZ24jguZyKpDcBFxJ+4I4gGK5eKnH+RWb2r8SYw4HzCVEitojFIyQNNbNMHqvnE/1fAzaL5zMJ1ser41L6LIIS1/9LNbA1kPzC7otlqXJFliRkyCiVLyTqVyZk6QBukJRUDNcAb47zrMsGZmavSlqSImPdaAQLpAG3SpojaVJKfVEp1VTFdGqFlqkzntlrp66l99TekhS1XBbMU8afUpTFsGu3LjbfaPMB5W+seaNpvKyhcHzGrq6Qu3jt2vDXFY2ieY1grW8o3AvbcZoTSe8mfOfeE4uuJMSJ3c7MRgIXsz5zVdp+rPNi+e5mtjlhOTct01U+ssc9DXgb8J44Zmbpu+C4ZrbKzM4xs12A9wOHEJbk01hAUPIyjKN/QoVK4yA+BxxkZqMSx8Zm9n/AQhLJHKLSPLrC+apKI1gg9zGzBfFXym2S/m5mdyfqi0qpZmYzgBkQYtFVIlD3Ad2pcRSrsYewUgsmhPiTaTSLl3V2fMZMmB5wRbFUJN3E+vfDEGAXwnJNQ5HPC9tfc8dpPCRlFLMLgFlmNjdWjQCWmtm/JO1N2Gt4a6xbRFhm3QH4R6L9MoJjzTbAfxWY+oXY/+k8bUYQLHkvR6eTqSXc14cIoXoeB5YTlpzX5Gh+FXC2pAcIn7NTCBbLanEx0C1popn1SRoLvN/Mfg1cD9yvkKr2r8C5NIbRbx11F8bMFsS/LwI3EDapJhm0lGoZit1DWG62l0osmND8XtbVCOnSyhlmSuT7wP/E4zzgg2Z2Zn1FGoh7YTtO03CTpBUE69hkwp7F4xP1nwfOjW2mkPjBamavEfYj3hs9nN9L2Df4LoIS+RvglwXmnwZcEfsfnaPNdGATgiJ4H/C7Eu7vLfrs3x0AACAASURBVATlbDnwBHAXuZXCbxLCoj0CzAUejGXV4gKCNffW+DzvA94DEPdLfoFg8V1IWPafX8W5K6aumWgkbQoMMbMV8fw2wobV3yXafBz4InAw4cFeaGbZSmY/qpmJJhf1zPZSztyZjDrlWj2ryZAhIUB4NlJYsi5EqRlmikVNmg1DUgews5ndLmkTYIO4sXzQyfXe6+wMluZsOjrCFgWnvWnW957jtDP1tkC+GbhH0sMEE+1vzOx3CpHvT4ltbgGeJZizLyH8+qk79cz2UqqXdTHBxweTSkO6eFDq9Ug6mfBr+iexaFvgV/WTKB33wnYcx2kt6roH0syeJcRWyi6/OHFuBDNuQ1HvbC+leFnnU3brYYU8+GD4cUowpoMPLq6/L4f24wuEbR/3A5jZU3E/cUNRTc96x3Ecp/7U2wLZtDTTPsR6K7vZ3HJLaeXZeFDqfrxuZm9kLmL8sPrtS8mDe9Y7juO0Dq5AlkkzZXuphbJbiRNLpRZEXw7tx10xNdYmkj5KiMd2U51lchzHcVocVyDLpJmyvVRb2c04sfT1BWeYTBieYpXISi2IbZJhpljOJITOmEvIanAL/VNtOY7j1ARJnZIsrny0BZKOk3RP4ZatT129sGvFYHhhNwNJz+tMqsOlK5dW7IVdqUdtrbyoK8U9QSvH33tOOfh7r/5ImkZI73dsnja9wEkxTzSSOoF/AsPMbHXtpcwp1/6EWJU1zxUt6TjCM9i3xvP8DJhvZlUxCFR7PHAL5KBTbuzIcuZJel4vWbmElatXMvPImWXFnkxS6RK0WxCrh6R9JN0m6R+SnpX0T0nP1lsux3GcVqOdLK3F0HYK5GApcLnmHqxwOrUMM1QNJxZ3qKgaPyUE+t0XeDcwPv4tiKQDJT0p6WlJA4KPSxon6Y+S/ibpEUlF+sk7jtOISLpA0nOSlsf0wR+I5QcCXwc+KemVGFovu+9MQiq/m2Kb0xPVXZLmSVosaXKizxBJZ0p6RtISSdfGzDFpsu0vab6k0yS9KGmhpOMT9RtJ+n6c5wVJF0vaJMaQ/i2wdZTrFUlbS1opaUzse7ak1TG7DpK+KWl6PB8p6eeSFknqi22HxLrjJN0r6XxJSwlBzrPl/p6keySNTKnbSNJ0SQviMV3SRomx78lqb5J2Ukjr3AWcHu/npljfK+ksSY9LeknS5ZI2Lne8SmkrBbLe8RAHM3ZkLT2vS3Vi8awxNWWZmf3WzF40syWZo1AnSUOBi4CDCOkPj5G0S1azs4FrzeydwKeAH1VbeMdxBpUHgD2BLQkZTq6TtHFM3vEt4Boz28zM0sLrTQDmAYfGNt9NVO9LyE19ADBF0r/F8i8BRwD7AVsTsqlclEe+twAjCbm3TwQukrRFrPsO8NYo/06xzRQze5XwObYgyrVZzHD3QJwXQkrGPmCfxPVd8fwHcc4dYvvP0D/zznsIsajfRMiyA6xTji8Bdgf+3cyWpdzPZOC9UeY9CCHXCi4hx9TMPcB34/0cmqjuAj4G7BifR6XjlU1bKZD1DP4NgxtOp5ZhhkpZgq7U4cYpyB/jL+D3SXpX5iii397A02b2bAwDdDVweFYbAzaP5yOpcQpRx3Fqi5nNij8yV5vZ/wAbERS/SjnHzFaa2cPAw6yP7/xZYLKZzTez1wkWvKPyLAWvImSjW2VmtwCvAG+TJOBk4CtmtjRm2voW4YdtLu4C9otz7Q5cGK83JqzS/Cn+kP4kcJaZrTCzXkJa2AmJcRaY2Q/iM1sZy4YR8mRvSVCos1JbrKMr3s+LZraIkNZxQo62xfJDM3vOzJYSFNpjKhyvbNpqPb/e8RDHjRxH37KB3ie1iB3ZfUB3arrDaoUZ6uoqbtk5X9YYX7auCu+Jf5MOCAZ8uEC/bQi5bjPMT4yVYRohR+t/ApsCH0kbKC6PTAIY16bBOB2nGZB0GnASwRqY+YE4pgpDP584fw3YLJ53ADdISiapXUPIQvd/KeMsyXLGyYw1FhgOzAm6JAAChuaR6S7C9p53EaJU3EbY8vNewo/nxZLeDGxIsE5m6CN8PmZIfk5m2IloUUzG4U1h65Sxt87TvhiS8lRjvLJpKwtkvYN/lxpOp5L9mo0SZqheWWPaZdnczD6UchRSHiF8+A4YLuv6GOBn0bPxYGBmZm9QlgwzzGy8mY0fO3Zs6TfhOE7NifsdzwCOBrYws1HAMtZ/FhQTkqXUsC3PAQeZ2ajEsbGZpSmP+VgMrAR2TYwz0swyimqaXH8mWFf/A7jLzB4n7OH8OOuXrxcTrJ4diX7j6K/cpo39BGGZ+7eS8llwF6SMnVnJeZWgFAMg6S1ZfXM96+2qPF7ZtJUCWe/g36UoddXYr9m1Wxe9p/ayduraij2v0yhGSctlkDKrnWLny+ZFMZ/+H0TbMnCJ+kTgWgAz+wuwMdWxVjiOM/iMAFYT4sZuIGkK67eoALwAdKb9SMxqs0MJc14MdEvqAJA0VlL2VpmCmNla4BLgfMVUrZK2kfSxhFyjk44scVl5DiHda0Zh/DNhWf2u2GYN4TOuW9KIKOdXgVlFyHQVwfHodkk75mh2FXB2vO8xwJTE2A8Du0raMy6rT8vqm+tZf0HSttEZ6evANRWOVzZtpUA2glUuqdR1H9DN5Dsm97MwZqyOx/7y2Lru1yxEsUpamsNNhlopdvmWzZ11PADsLGl7SRsS9hLdmNVmHmFTPHFT/MaELx/HcZqP3xO8lf9BWPr8F/2XQ6+Lf5dIejDHGOcRFKKXJX2tiDkvIHyu3CppBXAfA7fKFMsZwNPAfZKWA7cT92+a2d8JytqzUbbMsu5dhP2Kf01cjwDuToz7nwTr3bPAPQTnosuKEcjMrgDOBf6gEBMzm28Cs4FHCMvoD8YyzOwfse/twFNx7iQ/BXaJ9/OrRPmVwK1R3merMF7ZeCDxOpGxMCaVxGFDhiGJN9bk3lIhxNqpa3PWDxalBBPv6QnKW1r7XH0qYciQoNRmI4WwQblQmwUzVgjLM52wj+gyM+uWdC4w28xujF7ZlxD2IBlwupndmm/MZnjvOY1Hu733HKcclBXIvd60lRNNtUlmeik1u0uaR/iqtasK9hus/ZqFKGVvY8bhJpdiV+39kOPGpSurrerfIen9QCeJ97OZ/bxQv+jleEtW2ZTE+eOsD3vhOI7jOOtoqyXsalLpHsVyPL8Hc79mIcoJJl6NAOTFUGqcymZGIbjv91kfSDwTTNxxnAZFhYP4HxcDWz8Uj5PqIafj5MMVyDKpNKZkqZbEenlR56IcJW2wFLs2S5U4HtjHzD5vZv8Zjy/VWyjHcdIpMog/hKDee8bj0kEV0mlIzKyzUZavwRXIsqk0pmSaR/iwIcPYcOiG/cqGDxvOrCNn1cSLuhLKUdIGU7Fro1SJjxKyNziO0xwUE8TfcRqeuimQkrZTyLP7hKTHJH05pc3+kpYlzPhT0saqB5XGlEzzCL/8iMu57PDL6h67sRgyjjHz5oUl6O7u4pS0NlLsBosxwOOSfi/pxsxRb6Ecx8lJWhD/bVLa/T+FHPTXS9oupd5x6ko9nWhWA6eZ2YOSRhAizN8WN+4n+ZOZHVIH+fJSjUwvXbt1pSqHjagwJsmE8MmEysmE4wFXCOvAtHoL4DhOSRQTxP8m4Coze13SKcAVpGSXSmaB2nTTTfd6+9vfXm1ZnTZgzpw5i82s5CwQdVMgzWwhsDCer5D0BOFXWLYC2ZBklLxyvbAbhXIsiZ6esHEws7ti8Nudzex2ScPJn97LcZz6UjCIv5ktSVxeAnwnbSAzmwHMAA+h5ZSPpBxB9vLTEGF8YgDOdwL3p1S/T9LDhDfY18zssRxjDHo+3lwWxGahXEtivdITOgORdDLh/35LYEfCj7CLiQHAHcdpONYF8SekzPsU8OlkA0lbRSMLwGGE1HmO01DU3YlG0mbAL4BTzWx5VvWDQIeZ7QH8AMgZPb0d8/FWmu+53IwtgxWOxymKLxBiNS4HMLOngDfVVSLHcXJiZquBLxIywzwBXGtmj0k6V9JhsdmXom/Aw8CXgOPqI63j5KauFkhJwwjKY4+Z/TK7PqlQmtktkn4kaYyZLR5MORuRauxDLNeS2N3df25o3TiLTcDrZvaGFLZVSdqAgfupHMdpIIoI4n8WcNZgy+U4pVBPL2wRcjM+YWb/m6PNW2I7JO1NkHdJWtt2oxr5nsu1JLZZnMVG5y5JXwc2kfRRQj7bm+osk+M4jtPi1HMJex9gAvDhRJiegyWdEr3OAI4CHo1m/AuBT1mTJu/umdtD5/ROhpwzhM7pnUVnrMlFNfYhVhLY28PxNAxnAouAucBnCVaNs+sqkeM4jtPy1NML+x7Swxkk2/wQ+OHgSFQ7MmkPMyF/MmkPofyQPdXI95xR+sqJ5+g0Bma2luCleUm9ZXEcx3Hah7o70bQDlaY9TKNaaQGrbUms1LHHKQ5Jc2OQ4dSj3vI5juM4rY0rkINApWkP02jEfYgZx56+PjBb79gzGEpkGyquhwCHAr+LR1c8bgGur6NcjuM4ThvgCuQgUGnaw1w02j7Eajj2lEM9Fdd6YWZ9ZtYH7GNmp5vZ3HicCXys3vI5juM4rY0rkINA9wHdDB/Wf7251LSHzUC9AozXS3FtEDaVtG/mQtL7gU3rKI/jOI7TBjREJppWp1XSHhaiGo495dDmmXFOBC6TNDJevwycUEd5HMdxnDbALZAFqFb4na7duug9tZe1U9fSe2pvyymPUD3HnlLZcr8eOLUTpg4Jf3cLr1E7ZMYxszkxU9PuwB5mtqeZPVhMX0kHSnpS0tOSzszR5mhJj8esGFdWU3bHcRyneXELZB7KDb/TM7en5a2NadQjLFDP3B5WfGgSWFzDHtUHh05iSOe9LB5/LTonxJ0fvcloLjjogpZ5HSR9NUc5ALmC8yfaDQUuAj4KzAcekHSjmT2eaLMzIRvGPmb2kiRPkeg4juMAboHMSznhdzJKZ9+yPgxbp3RWGji8GejpGfyYkpPvmMwblrUBcsPXWLvXj3nV1ictWrJyCSf8+oRWeh1GxGM88Dlgm3icAuxSRP+9gafN7FkzewO4Gjg8q83JwEVm9hKAmb1YJdkdx3GcJscVyDyUE36nFjEfm4F6eUKXEgrpjTVvtMzrYGbnmNk5wBjgXWZ2mpmdBuwFbFvEENsAzyWu58eyJG8F3irpXkn3STowbSBJkyTNljR70aJFpd+M4ziO03S4ApmHcsLv1CLmY7WoZazEenlClxoKqRFehyozDngjcf0G0FlEv7QsUNlpQjcAdgb2B44BLpU0akAnsxlmNt7Mxo8dO7YYmR3HcZwmxxXIPJQTfqdWMR8rpVwLYbFKZy6P5zSv7GqS9hopT4bMer8ONWAm8FdJ0yRNBe4Hfl5Ev/nAdonrbYEFKW1+bWarzOyfwJMEhdJxHMdpc1yBzEPXbl3MOHQGHSM7EKJjZAczDp2R1xGjUWM+lmMhTFM6J0yAz39+YNtcHs9SbZex016jU8afwrAhwwa03XDohnV/HaqNmXUDxwMvEUL4HG9m3yqi6wPAzpK2l7Qh8Cngxqw2vwI+BCBpDGFJ+9lqye44juM0MWbWcsdee+1l9WTWI7Os4/wO0zRZx/kdNuuRWXWVx8xMMgtqYP9Dyt2noyN3n1lZtzRrVu45OjrWt+noCO06OgaOUU1mPTLLRn9ntDENYxo2+jujC74OwGxrgP/fYg5g8/h3y7SjyDEOBv4BPANMjmXnAofFcwH/CzwOzAU+VWjMer/3nOakmd57jXr4e88pl3Lffwp9W4vx48fb7Nmz6y1GQ9HZmb6c3NER0iCmMWRIUAHTSOunHCvHEsycGayZSSvo8OH1z9+dRNIcMxtfbzmKQdLNZnaIpH/Sf++iADOzHeohl7/3nHJopvdeo+LvPadcyn3/+RJ2DqoVQLzeZPYw9vUNVPAKBfnOF4g7uecxM0e+cdo83WDVMbND4t/tzWyHxLF9vZRHx3Ecp31wBTKFVonlmNzDCMGamFEiOzoKW/+6u3NbFbfcMn2ObDJKajXSDbaKUl9tJG0j6f2SPpg56i2T4ziO09q4AplCq8RyTLP6ma1ffi60dNzVBaeckl63fPn6wOHZc2RIKqm5rJnFphtsFaW+2kj6DnAvcDbwX/H4Wl2FchzHcVqevAqkpM0l7ZhSvns1Ji+Ui1fSRpKuifX3S+qsxryFaORYjqVQDavfj34Eo0cPLF+1an3WmTSk/kpqpXmyv/zbL7eEUl8DjgDeZmYHm9mh8Tis3kI5jpObRv3ua1R0jgYcrYY08Gh0ciqQko4G/g78QtJjkt6dqP5ZpRMncvEeREi9doyk7BRsJwIvmdlOwPnAdyqdtxgaNZZjqVRq9cuwdGl6eSZlYTFzdHUFa2RHR3hjFLOEnqFnbg9LVi5JrWs2pb4GPAsMjFnkOE5D0sjffY1ILmWxlZTIfA6ojUw+C+TXgb3MbE9CnLmZko6MddW4rWJy8R4OXBHPrwcOkGr/SBs1lmOpVGr1y5BPSSxljq6uYJVcu7a4JfQM+ayMzabUVwtJP5B0IfAa8JCkn0i6MHPUWz7HcXLSsN99jlMK+RTIoWa2EMDM/koIKDxZ0pcYmPKsHIrJxbuujZmtBpYBKQuq1c3HW04A8UakEqtfknxKYrXmSJLtLNO3LHc6m75lfe3qUDMbmEMI/v0N4M/xOnM4jtOYVO27z/PQO/Vkgzx1KyTtaGbPAJjZQkn7E7JT7FqFuYvJxVtMm1BoNgOYASEeVmWiBSWy2RTGNLq6Ko+zmOmf2fOYsTxmyqsxR4aMs0xmv2M+5TFDxqEGaInXrBjM7IrCrRzHaUCq9t1X7e89xymFfBbIzwFDknszzGwFcCBwUhXmLjYX73YAkjYARgI5duSVhoeEKY3s5WcoLkd2qaR5wBeDO9Q4jtMk1PW7z3GqRU4F0sweNrOngGslnaHAJoTUZinZkEummFy8NwIT4/lRwB+sCqlz6hESJhNsu9oK12CQLfvnPz8wR/akSdW5p0qcYtyhxnGcJqBu333NiE1Nv+1c5c1Irle20V/xYuJAvofwS+jPhH/8BcA+lU4c93V8Efg98ARwrZk9JulcSZkwJD8FRkt6GvgqMCDcQTkMdpzHZLDtaitcxc5frvKaJvvFF9cuq0wlTjHt5lAjaaik79VbDsdxiqee333Nik21AUerYTbwaHTy7YHMsApYCWwCbAz808zWVmNyM7sFuCWrbEri/F/AJ6oxV5LBjvOYL41frfNAZxTAzPwZ5RWKmztXMPI0SokvmYvuA7r77YFMY/Qmo1m5emW/Ns3oJV8pZrZG0l6S1K7WCcdpRur13ec41aQYC+QDBAXy3cC+hJhV19dUqhpTSZzHcqx51QjoXS6V5qAuRcZS40umkfSAB1DWXvLhw4ZzwUEXtISXfJX4G/BrSRMkHZk56i2U4ziO09oUY4E80cxmx/PngcMlTaihTDUnzcpVjAWrXGveuHHpuaKroXAVolLlNZfs2ZQTXzIXSQ/4nrk9TL5jMvOWzWPcyHF0H9C9rq5NFcZstgSWAB9OlBnwy/qI4ziO47QDBRXIhPKYLJtZG3EGh4zikUsxyUW5S9Hd3f0VT6iuwpWPSpXXNNmzGT0aLrigNsvxrRJOqVaY2fH1lsFxHMdpP4pZwm5JunbrovfUXtZOXUvvqb1FKSnlWvNqEWy7WNKCgA8bBq+8UtwyfFL2XGy22eDcizMQSW+VdIekR+P17pLOrrdcjuM4TmvTtgpkOVSSW7rcNH6Vkq28jh4d/i5ZUrxHeEb2XIm0shXoZg5Z1IRcApxFcHbDzB4hhAUpiKQDJT0p6WlJOb08JR0lySSNr4rEjuM4TtPjCmQJVCu39GCTVF432wzeeKN/fbFONcUo0PUOWdSGDI+pRpOsLtRJ0lDgIuAgYBeCc9wuKe1GAF8C7q+CrI7jOE6L4ApkCdQk7/MgW+sqcaopRoGu1OvbKZnFknYkpjmTdBSwsIh+ewNPm9mzZvYGcDVweEq7bwDfBf5VJXkdx3GcFsAVyBKp5lJ0Pax1lS7DF1KgK/X69hSTJfMF4CfA2yX9H3AqIQ1pIbYBnktcz49l65D0TmA7M7s530CSJkmaLWn2okWLShLecRzHaU5cgawj9bDWVboMX0iBrkRBrUeKyWYnWhA/AowF3m5m+5pZbxFd03a0rgtGLmkIcD5wWhEyzDCz8WY2fuzYsUVK7jiO4zQzrkDWkXoEGK+1R3glCupgp5hsBSRtJOnTwJeBr0iaImlKoX4Ei+N2iettCWlKM4wA3gHcKakXeC9wozvSOI7jOOAKZF0px1pXjT2TtfQIr0RBHewUky3Crwl7F1cDryaOQjwA7Cxpe0kbEjy3b8xUmtkyMxtjZp1m1gncBxyWFhfWcRzHaT+KyUTj1IhSA4xXmtd6sOjqKk+ecSPH0bdsYNTzYlJMtjHbmtmBpXYys9WSvgj8HhgKXGZmj0k6F5htZjfmH8FxHMdpZ9wCWUdKtdY1q4dzsY4x3Qd0M3xY//XvYlJMtjl/lrRbOR3N7BYze6uZ7Whm3bFsSpryaGb7u/XRcRzHyeAWyDpTirWuHnsmKyXjGJPZ25hxjIGBuazLTTHZjkiaS3B62QA4XtKzwOsE5xgzs93rKZ/jOI7T2rgCmUJPT7DqzZsX9iN2d9d3iTgjj1l6fbF5retBPseYNMXQc18XzSH1FsBxHMdpX1yBzKLR9hlmy5NNo2fCcceY2mBmfQCSZprZhGSdpJnAhNSOjuM4jlMFfA9kFo22zzBNngxDh8LEiY3lQJNNLgeYwXKMaYPA5LsmL2KKwr3qJIvjOI7TJtRFgZT0PUl/l/SIpBskjcrRrlfSXEkPSRqUDfyNts8w37xr1sAVVzR2nulqOcaUowi2cmBySWdJWgHsLmm5pBXx+kVCaB/HcRzHqRn1skDeBrwjbvT/B3BWnrYfMrM9zWxQAhhXkkmlFhSat9G9sLt262LGoTPoGNmBEB0jO5hx6IyS9jmWqwi2cmByMzvPzEYA3zOzzc1sRDxGm1m+95PjOI7jVExdFEgzu9XMVsfL+whZMBqCtEwqG24Ir7xSWfDuasqTTSN7YUNQIntP7WXt1LX0ntpbspNMuYpgO+y/NLOzJB0m6fvxcOcax2lQJG0p6TZJT8W/W+RotyauvD0kyWOyOg1JI+yBPAH4bY46A26VNEfSpHyDSJokabak2YsWLSpbmOzYjKNHB+/nJUvC34xTzWApkUl5cjFuXHUy1DQq5SqC9d5/ORhIOo+QxvDxeHw5ljmO03icCdxhZjsDd8TrNFbGlbc9zeywwRPPcYqnZgqkpNslPZpyHJ5oM5mQgi2XurOPmb0LOAj4gqQP5prPzGaY2XgzGz927NiKZE+m+ttsM1i1qn/9YC8bZ+SZNSs9z/TBBweltq+vPkpurSlXEWyTwOQfBz5qZpeZ2WXAgbHMcZzG43Dginh+BXBEHWVxnIqomQJpZh8xs3ekHL8GkDSREMuuyyw9wqGZLYh/XwRuAPaulby5aCSnmlyZa265pbE8x6tNuYpgNfZfNglJJ7SRdZPCcZxCvNnMFgLEv2/K0W7juKJ2nyRXMp2GpC5xICUdCJwB7GdmqUFqJG0KDDGzFfH834FzB1FMICwP9w1Mz1w3p5q0zDUTckT8q4aS2zO3pyaZYUoZt9QMNbWSuUE5D/ibpD8SstB8kPxOaY7j1BBJtwNvSakq5Sf9ODNbIGkH4A+S5prZMylzTQIy27tel/Ro6RJXxBhgcQvPV48563GPbyunk3IY/2qKpKeBjYAlseg+MztF0tbApWZ2cHzj3BDrNwCuzOTrLcT48eNt9uzSo/6kZaCBgYG8hw/Pn7N6sOnsTFdyOzrC0ne5ZKchhGD5q9SKV6txKx1b0pzB8vavJpK2At5NUCDvN7Pn6yVLue89p71p1vdeqUh6EtjfzBbG9+2dZpb3y1vSz4Cbzez6Au1mD/YzHOw5/R4ba856eWHvZGbbJTYJnxLLF5jZwfH8WTPbIx67Fqs8lkNPD4wZA8ceO3AfIaQvGzeK8gjpntrVyFBTqzA4tQyv08qhe3JhZgvN7EYz+3U9lUfHcQpyIzAxnk8kJWarpC0kbRTPxwD7EBzkHKehaAQv7LqSSRW4ZMnAusw+wqRTTW9vbuWxXp7QufZGVqrk1ioMTi3D67RD6B7HcZqWbwMflfQU8NF4jaTxki6Nbf4NmC3pYeCPwLfNzBVIp+Fo+1zY+VIFQvH7COudQzttb2SljBs5jr5lA9fGKw2DU6txaz224zhOJZjZEuCAlPLZwEnx/M/AbmUMP6My6cpisOf0e2ygOdveAllIQSzWWabRcmhXg1qFwalleJ02Cd3TD0n7Sjo+no+VtH29ZXIcZ3Axs0FXPAZ7Tr/Hxpqz7RXIfApiKfsIGyncT7WoVRicWobXaaPQPQBImkqIaJDxvB4GzCqy74GSnpT0tKQBAY0lfVXS4zFn/R2S8oSzdxzHcdoKM2u5Y6+99rJimTXLbPhws+A6s/4YPTrUFcvo0QPHALOOjuLHcMxmPTLLOs7vME2TdZzfYbMeKeFFqBBgtjXA/28pB/AQwfv6b4myR4roNxR4BtgB2BB4GNglq82HgOHx/HPANYXGLeW95zgZmvG9V+8D+ATwGLAWGJ+n3YHAk8DTwJkVzrklcBvwVPy7RY52a+Jn00PAjWXMk1dmQhSXa2L9/UBnhfdVaL7jgEWJezqpwvkuA14EHs1RL+DCKM8jwLuq8P9SaM79gWWJe5xSaMy2t0CmOaDMmgWLFxe/p7CnB1asGFg+bFjlntDZ8xTrpNOMqQ0zIXj6lvVhGH3L+ph00yR65jaB8PXjDQvv/vCpE2KmFsPewNMWoh28AVxNyJKxDjP7o62P09pQOesdx+FR4Ejg7lwNJA0FLiJkc9sFOEbSLhXMWfNUjEXKfCLwcg3xvgAAFiFJREFUkpntBJwPfKeUOcqYD8IP6Mw9XZpSXwo/IyituTgI2Dkek4AfVzhfMXMC/ClxjwXjbre9AgnFe1nnYvJkeOONgeWbb149x5aMk04x6QpLadtItGMInipwraSfAKMknQzcDlxSRL9tgOcS1/NjWS5OJEfO+mrloXccp3jM7Akze7JAs4I/FEtkMFIxFiNzUo7rgQMkqYbzVRUzuxtYmqfJ4cDPLXAf4fN9qxrPWTKuQFaBXPscl1bxpSrFSadZHXqqFYKnZ24PndM7GXLOEDqnd7a0BdPMvk/4AP0FIZvAFDP7QRFd0z5sU7MKSDoWGA98L4cMVctD7zhOVSn1h2IhBiMVYzEyr2tjZqsJS6+jS5ynlPkA/l/cD369pO3KnKvaMlWb90l6WNJvJe1aqHHbh/GpBoOR7rAUJ51mdeipRgie7Ew0mWVwoGUdaczsNsJ+pFKYDyQ/BLcFFmQ3kvQRQgq2/czs9bKFdBynZPKlRTSzAUHI04ZIKcubfm4wUzHmEiGlLFvmku+rwvluAq4ys9clnUKwfn64zPmqJVO1eRDoMLNXJB0M/IqwhJ6TtrRAVnt/YK0ywSTJpYymledqO2RIuNdGtdBVIwRPuy2DSzpS0lOSlklaLmmFpOVFdH0A2FnS9pI2BD5FyJKRHPudwE+Aw8zsxepL7zhOPszsI2b2jpSjGOURivyhWMKcL2SWUuPf1M8FM1sQ/z4L3Am8s0h5i5V5XRtJGwAjKX95tuB8ZrYk8QP6EmCvMueqmkzVxsyWm9kr8fwWYFjMhJSTtlMga7E/sFaZYJKUoqSmtQVYswaOP7+HE25oTEeVrt26mLjHRIZqKABDNZSJe0wsyXLYhplovktQ8Eaa2eZmNsLMNi/UKS77fBH4PfAEcK2ZPSbpXEmZTe/fAzYDrpP0kKQbcwznOE5jUvCHYokMRirGYmROynEU8IfoTFgOxfyYTu4/PIzwmVlLbgQ+o8B7gWWZrQO1QtJbMvtIJe1N0A9TcvQl+pT/zBuX8ePH2+zZs1PrOjvTl5s7OoIDTSPT0xP2Mc6bF6yM3d350ypOnBiUxn6c2gmjBj6AjpEd9J7aW22RSyJ7+RmCBbKUOI6d0ztTl8GLuT9Jc2yQk9hXiqR7zWyfesuRId97z3Fy0YzvvXoj6T+AHwBjgZeBh8zsY5K2Bi41s4Nju4OB6YTQXZeZWdlrY5JGA9cC44B5wCfMbKmk8cApZnaSpPcTVi7WEpSQ6Wb20xLnGSCzpHMJ4Z5ulLQxMJNg2VwKfCpaO8u9r0LznUdQHFfH+T5nZn+vYL6rCGFzxgAvAFMJMXwxs4ujIvdDgtf0a8DxFrIVlU0Rc36REK5tNbAS+KqFrEi5x2w3BXLIkGB5zEYKXtitROq9Th0CGvgAhFg7tb4PoBLlL0MlSmgzfYlJOjKe7kfYr/QrYN0eRTP7ZT3kcgXSKYdmeu85jhNoOyeawXB4aRRS73XZuFQLZCPkiq7G8nNGSZx8x2TmLZvHuJHj6D6guxUdaA5NnL8G/Hvi2oC6KJCO4zhOe9B2CmR3d9jzmAxzU22Hl0Yh7V6H/akbHTaJN6y/ha4RckVXwwsbghLZggpjP8wsk/t6HzO7N1knqWGWtB3HcZzWpO2caAbD4aVRSLvXy7/SxWX/0Zi5oqvhhd2GpMV8LCYOpOM4juOUTdtZICEoVq2oMKaRfq+NaaFro+XnipH0PuD9wFhJX01UbU7YCO44juM4NaMuCqSkacDJhOTkAF+PcYey2x0IXED4QrzUzL79/9u7+2g56vqO4+9PcgMYGqIJoFCbBMWKKSCQAFrUI8QnOEAAUR4iEqRGG1GwVYuNCsTmVEALWHqiFwzhISCIgKGhoIYWsCKQIJJEgcOxN4hQCIkNgURikm//mLnJsJm9O3vv7p27u5/XOXt29zdP35nNw3d+M/P7DlqQVopOuPzcIDuQDLHTBYzKtL9IMqyFmZlZ05TZA3lJWoYtV6bA+ftJBtV8SNLCiKhnPCmzthQR9wD3SJofETmPhZmZmTXPUL4HctALnJu1GiePZmZWhjITyLPSwuTzJL0uZ3pdxcQlzUiLty9ZtWpVtdnMzMzMbICalkBK+qmk5TmvqcBc4M3AAcCzwLfyVpHTVnXU84jojojJETF5t912a8g+1KPR9bXN+iLpwvT9I2XHUtTMRTPpmt2FLhBds7uYuWhm2SGZmVk/NS2B7Ksge0Q8FxGbI2ILSWHyQ3JWMejFxPsrr772xz4Gu+7qRNKa5ihJI4Avlx1IETMXzWTukrlsjqS25ubYzNwlc51Empm1qFIuYVcUJj8eWJ4zW6OLwBfSn57EWbNePVh3r9Wrk8SykUnkgmULmHDpBIZdMIwJl05gwbLmZaiDuS2r253AC8D+kl6UtC77XnZwlbqXdtfVbmZmQ1tZ90BeJGmZpEeBw4HPA0jaU9IdABGxCTgLuAv4DXBTRKxoZlB5PYlFEsCn+qi0t359kmA2JL60zvPKtSsJgpVrVzLj9hmFE7t6EsKBbsuaKyK+GBGjgUURsUtEjMq+lx1fpd6ex6LtZmY2tJWSQEbEaRGxX0TsHxHHRsSzafszEXFUZr47IuIvI+LNEdH0ciR5PYlFEsBadbT7SjDrMWvxLNb/6dUBrv/TemYtrp2h1psQDmRbRdTbu+ne0HwRMVXS6yUdnb4G/wbgAoYrf2zzau1mZja0DeVhfAZdtUSvVgI4Z05ST7uaWglmUU+tzQ+kWntWvQnhQLZVS73JrHtDq0sfonkQ+AjwUeBBSUNuIPEZk2bU1W5mZkObE8iMaolerQSwt+b02LHbTxs5MkkwG2Hc6PxAgqjZK1dvQjjmNWPqiqEe1ZLZs//j7Nxexmb3hra4rwAHR8TpEfFxkgfSvlpyTGZm1uacQGbk9SQWTQCnTYMXXoDrroPx40FK3ru7G1d3e86UOYwckd/VWatXrlril9e+YNkC1m1ct137iGEjmDNl4NlwtaR19YbVub2MzewNbQPDIuL5zPfVFPx7LelDkh6X9KSkc3Om7yjpxnT6A5Im9DfIuUvm1tVuZmZDmxPIjN6exIEkgNOmQU8PbNmSvDcqeYSkTnT3Md2MHz0+d3pfvXJ5yefIESNzE8JZi2excfPG7dp32XGXhtSpLtqL2bs/9SS/HehOSXdJmi5pOrAI2K6ufKVMqdAjgYnAKZImVsx2JvCHiNgbuAS4sKGRm5lZy3ICWaGZCWAjTNtvGj3n9KDccdar98plk08hxo8eT/cx3bkJYbV1rNmwpv+BZ/TVk5oXSz3Jb6eJiC8C3wX2B94OdEfEPxRYtEip0KnA1ennm4EpkvL/4JmZWUfpKjsA659xo8excu32ZZD76pWbtt+0Qj2I/Vl3PXpjmLV4Fk+tfYpxo8fx0saXWL1hde428+afM2VOQ3pD20FE3ALcUudieaVCD602T0RskrQWGEsy/uRWkmYAMwDGNeqJMTMzG9LcA9mimtkrNxg9fr09qVvO20LPOT1cduRlfW6zcn4njwNWpFRooXKiZZcRNTOzwecEskXVc0l6KK17KG2zwxUpFbp1HkldwGigMfcxmJlZS1PEdh0KLW/y5MmxZMmSssOwFiNpaURMLjuOekg6OyIuq9WWs1wX8AQwBfg9SenQU7PVniR9BtgvIj4t6WTghIj4aF/r7evvni7YvkMzzmu/f3+sfq34d8+s0/keSLPWdjpQmSxOz2l7lfSext5SocOBeRGxQtJsYElELAS+B1wr6UmSnseTBxKok0Uzs/bhBNKsBUk6BTgV2EvSwsykUSRjQdYUEXdQMeRPRHwt8/mPJBVuzMzMXqUtE8ilS5e+IGn7x4j7Z1cqnjptcd6f6vIH2Byafg48S7L/38q0rwMeLSUiMzPrGG15D2QjSVrSTvfmeH+smSStAmqdvLXbSUw9vO/5xkeEH+E3ayFt2QNp1ikkrWPb0Do7ACOAlyNilzLiKZIEdHLS733vzH03a0dOIM1aWESMyn6XdBxJlRkzM7Om8TiQtXWXHUCDeX/aWETcBhxRdhxmZtbe3ANZQ0S0VYLi/Wkvkk7IfB0GTCanWswQ08m/mffdzNqCH6Ixa2GSrsp83QT0AFdExPPlRGRmZp3ACaSZmZmZ1cX3QBYg6XxJv5f0SPo6quyY6iXpQ5Iel/SkpHPLjqcRJPVIWpb+Jh1Zu1LSmyTdLmmVpOcl/UjSm8qOy8zM2psTyOIuiYgD0tcdtWcfOiQNB/4NOBKYCJwiaWK5UTXM4elv0qnDg1wP3ATsAewJ/AC4odSIUrVOWiTtKOnGdPoDkiYMfpTNUWDfp6dJf+9J6d+UEWejSZqXnsgsrzJdkr6dHpdHJR002DGaWWM4gewMhwBPRsRvI2Ij8H1gaskxWWMoIq6NiE3p6zqGwEM0BU9azgT+EBF7A5cAFw5ulM1RxwnbjZmT0isHNcjmmQ98qI/pRwJvSV8zgLmDEJOZNYETyOLOSs+Y50l6XdnB1OnPgd9lvj+dtrW6AH4saamkGWUHU5L/lHSupAmSxkv6ErBI0hhJY0qMq8hJy1Tg6vTzzcAUSRrEGJulY0/YIuJeYE0fs0wFronEL4DXStpjcKIzs0byMD4pST8F3pAzaRbJWfLXSRKWr5PUHv7E4EU3YHn/KZfeS9UAh0XEM5J2B34i6bH0P7BOclL6/qmK9k+Q/MZl3Q+Zd9JyaLV5ImKTpLXAWFq/1F+RfQf4sKT3AE8An4+I3+XM026qncw+W044ZtZfTiBTEfG+IvNJugL49yaH02hPA3+R+f5G4JmSYmmYiHgmfX9e0q0kPT+dlkC+LSL+mG2QtFNlWwmKnLS064lNkf26HbghIl6R9GmSnthOGAC+XX9zs47jS9gFVFxiOR7IvUF8CHsIeIukvSTtAJwMLCw5pgGRtLOkUb2fgQ/Qer9LI/y8YNtgK3LSsnUeSV3AaPq+/Nkqau57RKyOiFfSr1cAkwYptrK15cmsWSdyD2QxF0k6gORMuYftLxcOaenlwbOAu4DhwLyIWFFyWAP1euDW9Ja5LuD6iLiz3JAGj6Q3kFz6e42kA9nWs7MLMLK0wLbZetIC/J7kpOXUinkWAqcD9wMnAndHewxMW3PfJe0REb2XbY8FfjO4IZZmIcn95N8nuay/NnMczKyFeCBxsxYk6XRgOknpwuwYmOuA+RFxSxlxZaXjpV7KtpOWOZJmA0siYqGknYBrgQNJeh5Pjojflhdx4xTY938mSRw3kez730bEY+VF3BiSbgDeC+wKPAecB4wAiIjvpA9JXU7ypPZ64IyI6MgxXM1anRNIsxYm6cMR8cOy4zAzs87iS9hmrW1fSX9V2RgRs8sIxszMOoMTSLPW9lLm807A0XTO/XRmZlYSX8I2ayOSdgQWRsQHy47FzMzal4fxMWsvIylv8HAzM+sQvoTdYSTdCbwD+FlEHF12PDYwkpaxbSDm4cBugO9/NDOzpnIPZOe5GDit7CCsYY4GjklfHwD2jIjLywxI0nRJew5g+QmSKseMLLLcxZJWSLq4v9uuc3s9knbNaT9W0rlVlnmpSvt8SSc2OsahRtKlafnGvo7fPpLul/SKpC9k2neQdG866LyZlcwJZJuSdLCkRyXtlFZtWSFp34hYTDJWoLWBiFgJvJYkgTwemFhuREAyPmW/E0hgAtsPOl7Ep4CDIuKLA9j2gEXEwoj4RpkxNFojkjZJY4B3FKhXvwb4HPDNbGNEbAQWs63+u5mVyAlkm4qIh0iqPvwTcBFwXUR0Yqm/tibpbGABsHv6WiDpsw3ext9JWp6+zknbJkhanpnnC5LOT3vRJqdxPCLpNWlP04WSHkxfe6fLvKrXLdM79w3g3enyn6+IRWlP43JJyySdlLYvBHYGHuhtyywzRtJt6QnVLyTtn7afL2mepP+S9FtJn8ss87E01kckfVfS8CqH57OSHk5j2Sdddrqky9PPe6W9aQ9J+nrFflwu6deSFpH8dr3TJkm6R9JSSXcpLaWaxtl7HJ+Q9O6c3+q96Xw3S3pM0oJ08O5a652cft5VUk9mP34g6Xbgx30c+6rbrHAiUFktarvjFxHPp/9+/SlnHbcB06r8FmY2iJxAtrfZwPtJ/kO/qORYrDnOBA6NiK9FxNdI7m/9ZKNWLmkScAZJ2bl3AJ9UUjoxV0TcTFIZZ1pEHBARG9JJL0bEISRVSC6tsdlzgfvS5S+pmHYCcADwduB9wMVKygIeC2xIl7mxYpkLgF9GxP7APwLXZKbtA3wQOAQ4T9IISW8j6eU6LCIOADZTPWl5ISIOAuYCX8iZfhkwNyIOBv4303488FZgP5Lf668BJI0A/hU4MSImAfOAOZnlutLjeA5JlZc8B6bTJ5I8UHVYgfVW807g9Ig4girHvto2c9Z1GLC0oq3W8au0HDi4wHxm1mROINvbGODPgFEkYwRa+xFJgtNrM9vqYjfCu4BbI+LliHgJuAXYruergBsy7+8cYDw3RMTmiHgOuIfaCcW7SEomEhF3A2MljU6nLYqIVyLiBeB5khrrU4BJwEOSHkm/V3uyvbdk5FKSS++VDmPbvl+baX9PZj+eAe5O298K7Av8JN32V4A31rE9gAcj4umI2AI8ks5Xa73V/CQi1qSf+zr2edustAewqqKtyP5sFRGbgY2SRhWI3cyayDcjt7du4KvAXsCFwFnlhmNNcBXJZdtb0+/HAd9r4PqrJaObePUJaK0TlMj5vHUd6SXPHQYQT73L9MbwSqZtM8m/iQKujogvF1h37/K9y+apNthuXruAFRFRLckusr1q+1RtvdnfsvJ3fLkitmrytllpQ876i+xPpR2BPxac18yaxD2QbUrSx4FNEXE9yT1lB0s6QtJ9wA+AKZKeluQBp1tYRPwLySXmNcAfgDMiotYl4nrcCxwnaaSknUkuvd4HPAfsLmmsksHLs0NCrSPp9c46KfN+f/q5h6SnD2AqMKKP5bPxnCRpuKTdSHryHiywD9MguV+P5LLpi33Mvxg4UdLu6TJjJI2vsY1q/hs4Of2cvQx+L3Byuh97AIen7Y8Du0l6Z7rtEcopVdkPfa23h22/Q19Pgvfn2Gf9Bti7rqgrSBoLrIqIvPsjzWwQuQeyTUXENaT3eqWXfQ5NJ91ddSFrSRHxMPBws9YtaT7bEoUrI+KXAJJmAw8A/wM8lllsPvAdSRvYdrl6R0kPkJy0npK2XQH8SNKDJElbb2/Xo8AmSb8C5lfcB3lrus5fkfTgfSkisvcW5jkfuErSo8B64PQa+/xrSV8heXBkGMnDHJ8BVtbYTp6zgeuVPOz0w4r9OAJYBjxBcjmYiNio5MGib6eX2btI7hld0Y9tb1Vjvd8EbpJ0Gn3/+5B77HsffilgEcmT8lf2NZOkN5DcR7sLsEXJg1sT06T/cOCOgtszsyZyKUMza6r0qd7J6X2G1sEk/Qw4OiL+r5/L3wJ8OSIeb2xkZlYvX8I2M7PB8vfAuP4sKGkH4DYnj2ZDg3sgzczMzKwu7oE0MzMzs7o4gTQzMzOzujiBNDMzM7O6OIE0MzMzs7o4gTQzMzOzuvw/D3RyOoqKBIAAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from utilities import plot_data_transformations\n", "plot_data_transformations(X_train, y_train, w1, w2, w3, w4, w5, w6, b1, b2, b3, \\\n", " language='English')" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAApAAAADoCAYAAABYW9TVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydeZhcVdGH318WlkAIS6KsmUEWFQygjKCCgkb9ENn0Q1wGBFkiiwqKgjgYCDqg6KeAghgQxGRAFkEJBBWigKgsCQJhEUWcSWJYskASSIQs9f1xTid3evr23tPdM/U+z326+9yz1O2Z7q5bdapKZobjOI7jOI7jFMuQegvgOI7jOI7jNBeuQDqO4ziO4zgl4Qqk4ziO4ziOUxKuQDqO4ziO4zgl4Qqk4ziO4ziOUxKuQDqO4ziO4zgl4Qqk4zhOCpLeK+kZSa9IOqje8iSRtKOkovOwSZoq6dwaiuQ4ziDCFUjHcRqKqKxljjWSViRet/ezON8GfmhmG5vZbf28dt2QdJ+kY+oth+M4jcuwegvgOI6TxMw2zjyX1A0cb2Z3pfWXNMzMVtVInBbgiXIG1lgux3GcuuIWSMdxmgpJ35Z0vaTrJC0DjpT0bkn3S3pZ0nOSLpE0PPYfJskkfT66o1+SdElivp0l3StpiaSFkq6N7d3AWOCOaP0cKmlbSbdJWizpn5KOLSDXtyX9Mra9IulRSTtIOlvSAklzJH0wMcemkq6O1zBP0nmShsRzQyX9UNIiSf8CDijwPu0p6RFJyyRdB6yfOLeFpOlRhpckTZO0TTz3XeDdwOVR5oti+4+jTEslPSTpPRX9IR3HaWpcgXQcpxn5GHAtMAq4HlgFnAqMBvYhKFefzxpzILAn8HaCcpdR3DqB24HNgG2BSwHMrBWYD3wkurBXx7X+DWwNfBK4UNJ+eeQCOBT4GbApwZp5V5R3K+AC4CeJ8VOBFcAOQBvwUeBz8dxJwIeB3YG9gCPS3hxJ6wO/Aa4CNo/PD0t0GQJcQVCQW4CVwMXxus8E/gqcGK/7tDjmAWC3ON9NwI1xHcdxBiGuQDqO04zcZ2bTzGyNma0ws4fM7AEzW2VmzwKTgf2yxlxgZkvMrBu4G9gjtq8EWoGtzOy/ZvbnXAtK2p6guH099nsYuBo4Kk2u2Ha3md0V3dk3EhSwC+PrXwI7Sto4WgDHA182s+Vm9jxwEfCpOM8RhP2Y88xsEfCdPO/PPoABPzKzlWb2S+BvmZNmtsDMbonv3VLg/BzvVy/MbIqZLY5yXwhsAuyYb4zjOAMXVyAdx2lG5iZfSHqLpNslPS9pKXAewRqZ5PnE8+VAZq/l6cBwYKak2ZKOTllza2Chmb2aaOsBtkmTK/JC4vkKYIGZrUm8JsrSQnAzvxBd8S8TrKFvTKyfnL8nRc5M33lmlozSXttf0kaSrowu9KXAH+j7fvVC0hmS/i5pCfASsFGhMY7jDFxcgXQcpxnJTl/zU+BxYEcz2wSYCKioicyeM7PjzWwr4BRgcrQ2ZjMfGC1po0TbWOA/eeQqhbkExXZzM9s0HpuY2W7x/HPAdllrp/EcwR2fJNn/DGB7YK/4fn0gq2+v65D0fuArwP8SXPGbAa9Q5HvsOM7AwxVIx3EGAiOBJcCrkt5K3/2PqUg6IhNAArxMUJ5WZ/czs38DM4HzJa0vaQ/C/sSuSoWP888F7gG+L2kTSUNirsf3xS43AKdJ2kbSFsCZeaa7Dxgi6QsxiOgTwDsS50cSlNWX4lwTs8a/ALwpq/8qYCHBWnsuwQLpOM4gxRVIx3EGAqcDRwPLCNbI6/N378XewEOSXgVuBk4xszkpfT8J7ERwh98EfMPM/li21H05kqCYPUlwE98IbBnP/QSYAcwGHorr58TMXiME9JwQ5/k48OtElx8QAn0WAX8B7sia4iLg09GV/gNgOiH4559AN7CUYOV0HGeQot5bZBzHcRzHcRwnP26BdBzHcRzHcUrCFUjHcRzHcRynJFyBdBzHcRzHcUrCFUjHcRzHcRynJFyBdBynLCS1xhrTw+otSyOhwNWxxvSDRY75uaRvV2n9uyUdn3JubKaud8r5cyVNzTN3d7J2d7Wo5vU7jtM/uALpOE5BxSH2qYnyUCmSjpF0XwPNty/wIWBbM9urBvOXjZnNSdT1dhzHKRtXIB3HcapLC9CdVfLQaTDccu44leEKpOMMEiRdLGmupKWSZkl6b2w/APgG8Mno3nw0x9gphFJ402KfMxKn22NN5YWSOhJjhkj6uqR/SVok6QZJm+eR7wRJz0haLOlWSVvH9j6u8oybNladuRx4d5Tr5Xj+55Iul3SnpGWS7pHUUu58OWTdOsq4OMp8Qmw/DrgyMX5S1rh8828W63kvk/SApB0S494Sr2WxpKclHZH2PkZaJP05zvV7SaNzXbuk7eN7s0zSnWTVtpZ0lKSe+PfryDqX+vdNrHN0rv+NfEjaTNJtkhbEbQC3Sdo2nvuEpFlZ/U+X9Ov4fH1J349rvhD/BzaM5/aXNE/SmZKeB64uRh7HcXLjCqTjDB4eAvYANgeuBW6UtIGZ/RY4H7g+ujd3zx5oZkcBc4CDY58LE6f3Bd4MjAcmRiUJ4EvAYcB+wNaEiiiX5hJM0geAC4AjgK2AHuCXhS7IzJ4CTgT+GuXaNHG6HfgWQSl6hCJKDhaYL8l1wLx4XYcTyhuON7OfZY0/p4T5Pw1MItSZfgboBFCovX0n4W/2htjvMkm75rmUzxDKLL4BWA/4akq/a4FZhPfoW4RqPsR1dyFUvzkqXucW9K6vXczfN+1/Ix9DCMpdC+GmZQXw43juVmD7rHmOBKbE598Fdib8n+8IbEPvMo1bEv7/W4AJRcjiOE4KrkA6ziDBzKaa2SIzW2Vm/wesT/hxr5RJZrbCzB4FHgUyCujngQ4zmxdL650LHK7crsN24Cozezj2PYtgpWutQK7bzezeOF9HnG+7CuYDIM6xL3Cmmf3XzB4hWB2PqnDqm83sQTNbRVB294jtBxFc4lfHv93DwK8IimsaV5vZP8xsBaGG9h7ZHSSNBd4JfNPMXjOze4FpiS6HA7cl3sNvAmsS54v5+6b9b6QS/0d/ZWbLzWwZQZHeL557jVCm8sh4DbsCrcBtkkQo3fhlM1scx54PfCox/RrgnHi9KwrJ4jhOOr4HxHGaGEltwHsJFqAVwOPAXWa2OEff04HjY18DNiHLZVkmzyeeLwc2js9bgFskJZWO1cAbgf9kzbE18HDmhZm9ImkRwYKU3bdY5mbNtziu80KZ82XYGsgoKBl6gLYK5833Pu6d5e4exjqrWylzJdkaeClrr2YPsF3ifPI9fDX+TTLk+/uWIkcvJI0AfggcQLDGAoyUNDQG/1wDXCfpbILSfoOZvSbpDcAIYFbQJcN0QDLifIGZ/beQDI7jFMYtkI7ThChE8j5MsNRtCDwNvEiwjN0p6ZpoYcr0fy9wJsFFvFl0nS4h/MBCUCgLUUyfJHOBj5jZpoljAzPLpRDOJygkGXk3IrhM/wNkFJwRif5bFiHXWmujpI0Jrsv5FcyXlHVzSSMTbWMpXtEt5328J+t93NjMTipxnmyeI+y73CjRNjbrfPI9HEH4myTlKvbvWwqnEyzje5vZJsD7MiIAmNn9wOuEG6fPsE6RXki4ido1Ic8oM0sqraW+947jpOAKpOM0JxsB+5jZ/5rZ+WZ2pZn92My+ZGZ7Eiw4OyX6jwRWAQuAYZImEiyQGV4AWiXl+054AXhTCTJeDnQmglfGSDo0pe+1wOck7SFpfYLr8QEz6zazBQTl7EhJQyUdC+yQGPsCsK2k9bLmPFDSvrH9W3G+uRXMB4CZzQX+AlwgaQNJuwHHUcQey2Lmz8FtwM4xoGV4PN5Z5H7CVMysB5gJTJK0nqR9gYMTXW4CDkq8h+fR+zejlL9vKYwkKIIvx6Ccc3L0+QVhX+QqM7svXs8a4Argh9EaiaRtJP1PFWRyHCcLVyAdpwkxs0vz7eEys0fMbEai6XfAHcA/CG7K/5JwTwI3xsdF0bKZiwuAsyW9LCktKCPJxYSgh99LWgbcD+ydIu8Mwh67XxEsXzvQe+/aCcDXgEXArgQFLsMfgCeA5yUtTLRfS1A+FgN7EvZZVjJfkk8T9t7NB24h7Ku7M6VvNsXMv5boKv8w4f2YT3ALf5ewh7VSPkP4mywmvFe/SKz7BHAK4X18jhAkMy8xtui/b4lcRLCqL4xz/jZHnynA2+jrxj+TEIB0v6SlwF1UZ5+v4zhZyMwt+o7TjETLyrbADDPrTrQfa2ZX1U2wBkDSz4F5ZnZ2vWVxqk9MzfMi8A4z+2e95XGcwYhbIB2nCZF0ASGyeBwwQ9IXE6e/UB+pHKffOAl4yJVHx6kfHoXtOM3JQcDbzWyVpHOBayW9ycy+zLrAGMcZcEjqJvyPH1ZnURxnUOMubMdpQiQ9ZWZvTbweCkwmBMbsYmb5kkw7juM4TkW4C9txmpN/Sdov88LMVpvZcYR0PhVF5zqO4zhOIVyBdJzm5BPAg9mNMWik4morjYKk6yT1q6tSoY72t/Ocf0VSznRGMT/nfXnG3i3p+GrIWS0kvVfS02WOvVmhlrrjOIMMVyAbDEnfkHRltfsWMZdJ2rHIvudKmlqNdZ3yiOXhVsDaXHfvkfQ+Se+jd07DpiXmV9wd+E18nVc56y9iEu9n6y1HtTCzP5nZ2lQ3krolfbDI4d8h1ux2HGdw4UE0NUTSMYSqCjsASwn54s4ys5fTxpjZ+cXOX0rfeuHpVGqLpO8CnwSeJJSRg1Bt4966CVU9Pg90mW/UbljM7EFJm0hqM7OZ9ZbHcZz+wy2QNSLWHf4uIVnxKOBdhFJtd6ZVoJDkCr1TKocBbzazA83s4HgcUm+hqsRHgHsAYtWVy4F3Rxfyy5K2j49DYp8rJb2YGSxpqqTT4vOtJd0qabGkZySdUGDtzSTdLmmZpAckrbXqJq31kraI8y6V9CBZ1l9JH5L0d0lLJP2YrAh5ScdKekrSS5J+l6nqkljnREn/jOcvlZQzwj7b7S5pf0nzEq+7JX1V0mNRluslbZDdV9IUQjnDafF9PkOh2s5USYvi+/2QpGS967uBjxZ4Px3HGWC4AlkDJG0CTAK+aGa/NbOVMdHzEQQl8sjY71xJN8Uv56XAMdnuYUmfldQTv7y/mXQvJftKao0/OEdLmiNpoaSOxDx7Sfpr/AF4TtKP0xTZHNezvaR74o/pncDorPM3Sno+/jDdK2nX2D6BUP3jjPhjNC22f13Sv+J8T0r6WHnvtAM8CwyvtxDVRqE+8/aEoCDM7CngROCv0YW8qZn9m2DZf3sc9l7gFa0r8fc+ogIKXEeoorI1cDhwvqTxeUT4NOEzvBmhskmam/ZSQlWfrYBj45G5htGEyjpnEz4z/wL2SZw/DPgG8HFgDPCnKGeSg4B3Elz5RwCVlOU7AjiA8L7uBhyT3cHMjgLmAAfH9/lC4GjCTfB2hFrYJxJKDWZ4KsrnOM4gwhXI2vAeYAPg5mSjmb1CKCf3oUTzoYSas5uSVUtX0i7AZQQlbCvCl/g2Bdbel1C6azwwMfFjuhr4MuGH7N3x/MlFXs+1wKw49luEH5QkdxDqLr8BeDhzHWY2OT6/MP4YZers/ovwYz+K8CM9VdJWRcriAJJ+JOkSYDnwiKSfSrokc9RbviqwaXxcVqDfPcB+kraMr2+Kr7cnpDR6VNJ2hM/FmWb2XzN7BLgSOCrPvDeb2YNmtorwP7xHdgeF1En/C0w0s1fN7HHgmkSXA4EnzewmM1tJKNH3fOL854ELzOypuM75wB5JKyTwHTN72czmAH/MJUcJXGJm881sMTCthLlWEhTHHWO0/ywzW5o4v4x1fy/HcQYJrkDWhtHAwvijkM1z9Lbg/dXMfm1ma3LUNj4cmGZm95nZ68BEwv62fEyKARaPAo8SLQPxS/9+M1sVraE/BfbLMw8AksYSLCDfNLPXzOxewo/PWszsKjNbZmavAecCu0salTanmd0Yf8jWmNn1wD+BvQrJ4vRiJkGpv5Wg1P8lvs4czU5mn/DIAv3uAfYnWBvvJbhT94vHn8xsDcHquDjWlM7QQ/6bsaSitxzYOEefMYR95Mma4j2J51snz8W9nMm+LcDF0SvwMqEetbLkKkaOYil3rimEWuq/lDRf0oWSklbvkaz7ezmOM0jwPXe1YSEwWtKwHErkVvF8hrmkk/0DtFzSogJr5/yRkLQz8AOgDRhB+NsXo2hsDbxkZq8m2nqIqWKiFaaTkFZmDLAm9hkNLMk1oaTPAl8BWmPTxmS5xZ38mNk1hXs1L2b2qqR/ATsDCzLNObreA3yP4J6+B7iPsFfyv6xzX88HNpc0MqFEjgX+U6GYC4BVhM/C3xPzZniOREqluH8xmWJpLtBpZr08D2XyKuFznWHLtI5F0Ot9jtbTScAkSa3AdMLWgp/FLm8l3Kw6jjOIcAtkbfgr8Bphb9Na4r6ujwAzEs35LIrPAdsmxm9IcCWVw08IP3I7mdkmhL1XxZS8e44QULBRoi35I/kZghv+gwSXdGtG3PjY6/qie+4KQr3mLcxsU+DxImVxIpKmSTo4yxKUOfcmSedJOjbX2CZiOr2t5C8A2yb37sZayCsI+4rvja7VFwiu5Xtin7kEC+0FMSBkN+A4sraMlIqZrSZsUzlX0oi45SS5veN2YFdJH1cIkPsSvRW7y4GzEnuGR0n6RJniPAIcKGnz6M4/rcx5ILx/a/NcSnq/pHHxZnEpwaW9OtF/P8I2FsdxBhGuQNYAM1tCuGP/kaQDJA2Pd+43EiwlU4qc6ibgYIUcf+vFOctVtEYSvvxfkfQW4KRiBplZD8FdOknSepL2BQ5OdBlJUJYXESwg2amFev0YARsRlMoFAJI+B7yt5KtxTiDsI/17jIqdLukPkv5N2J4wy8yuqq+IFTMZaI+WO4A/AE8Az0tKWvHvARbFfYKZ1wL+lujzacLNzXxCOq1zzOzOKsj4BYIF/Xng58DVmRNmtpBgmf8O4fOxE/DnxPlbCJkafhmD6B4n3GCWwxSCFbAb+D1wfZnzAFwAnB1d618lKL03Eb4/niK8v5ngvXcCr5pZn6T2juMMbLwWdg2RdBwhcCWTB/LXwNfN7KV4/lzCxvQjE2N6tSnkkjyPoHhdRIiA/JSZ/SnZNyqo/waGZ9zmku4GpprZlQoJpicTLJp/I2zI/4CZ7Rv7GsE6+UyO63gTITjg7QTr6tPApnHdjQmWnA8Q9nB9M/bdycyekbQTQXFuBe42s8MkdRIU2DXAL4A9gSlmVpWk6ION+LffimCJ+4eZLa+rQFVE0rXADWb263rL4vRF0q+An5nZ9HrL4jhO/+IKZBMRlbWXCcrZv+stj+M4juM4gxN3YTc4cZ/biLgH8fvAbIKbynEcx3Ecpy64Atn4HErYtzWfsIfqU17azXEcx3GceuIubMdxHMdxHKckPA+k4zQxkvYhJG9vIXyeRchZ/aZ84xzHcRynEtwC6ThNjKS/EyL9Z5HIzWdmhRLO14TRo0dba2trPZZ2mphZs2YtNLMx9ZbDcZziGZAWSP8Rc8qhSX/ElphZwyRxbm1tZebMmfUWw2kyJPUU7uU4TiMxIBVI/xFzyqFJf8T+KOl7hIoor2Uazezh+onkOI7jDHQGpALpOIOIveNjW6LNCIndU5F0FXAQ8KKZ9akEFKu/XAwcSKipfowrpY7jOE4GT+PjOE2Mmb0/x5FXeYz8HDggz/mPENJG7QRMINRSrwip7zFsWHhsbYWuiipTDzy6ZnfRelErQyYNofWiVrpmN98blLkGTVKfw3Gc5sYVyBIYCF/ozsBC0ihJP5A0Mx7/J2lUoXFmdi+h9GQahwK/sMD9wKaStipfztztq2PYT08PTJjgSmSGrtldTJg2gZ4lPRhGz5IeJkyb0FTfOclryIUrkY7T3LgCWSS5vtCPuvkoNEmuTDr15CpgGXBEPJYCV1dh3m2AuYnX82JbzVi+HDo6arlC89Axo4PlK3uXNF++cjkdM5rnDcp1DY7jDBxcgSySXF+GRkiB1KzWAbemDgh2MLNzzOzZeEwCqpEDMpd5KGfOL0kTMhbQBQsWVLTonDkVDR8wzFmS+41Ia29EmklWx3FKxxXIIin0ZdhM1oGTbz+Zo24+qqndY85aVkjaN/MiJhZfUYV55wHbJV5vSyin2Qczm2xmbWbWNmZMZVmQxo6taPiAYeyo3G9EWnsj0kyyOo5TOq5AFkkxX4bNcMfdNbuLy2devtZ6mqGZFGCnFycCl0rqltQN/Bj4fBXmvRX4rALvIuSbfK4K86YyYgR0dtZyheahc3wnI4aP6NU2YvgIOsc3zxuU6xocxxk4uAJZJMV8GRZSMhvBbdwxo6OP8pih1gpwV1eIth0yxKNuq4GkIcCbzWx3YDdgNzN7u5k9VsTY64C/Am+WNE/ScZJOlHRi7DIdeBZ4BrgCOLkSWU86KU2O8NjSApMnQ3t7JasMHNrHtTP54Mm0jGpBiJZRLUw+eDLt45rnDUpeQy7sHK+C5jjNzIAsZdjW1ma1SCTeNbuLjhkd9CzpQaiXIjZi+Ii8X/CZIJzkPspCY2rBkElDUhXIllEtdJ/WXZN1u7pClO3yxDbSESMaS2mQNMvM2gr3bBwk3Wtm76u3HBnSPnutrSHSOpuWFujurrlYToPTjJ89Z+Ah6Y3AjcDbgclmdnqdRepF9DIdb2Z31VsWcAtkSbSPa6f7tG7sHGPKx6eUZB1Ii6o8+paj+9UimWYlFaqpe6yjo7fyCB51WyXulPRVSdtJ2jxz1FuobNKCYzxoxnEah7gVZoWkZZJelvSX6JkoSleQ1CrJJJVdpCSO37Hc8RUyAVgIbNJoymMj4pVoyqR9XHtJlsM09/BqC4nwMoEsmblrRef4zj6WUCFObDuxpuu6AlEzjo2PpyTajOpEYleNzTeHRYtytzuO01AcbGZ3xXyy+xEqUu0NfK6+YgUkDTOzVTWavgV40spwzdZYroak7hbIeMczW9Ijkvr4vuIm/kskPSPpMUnvqIeclVJMEE5/BLLk2ls15eNTuOyjl1VtjVx7HdOiaz3qtjLMbPscR0Mpj47jNB9mtsTMbgU+CRwt6W0Akj4q6W+SlkqaK+ncxLB74+PLkl6R9G5JO0j6g6RFkhZK6pK0aa41JWXGPxrHf1LS/nGf9pmSngeulrSZpNskLZD0Uny+bWKeuyV9S9KfozX195JGx3MbSJoa5XlZ0kOS3ijp58DRwBlx7Q9KWl/SRZLmx+MiSevHeXLJlWk7Q9KLkp6TdJikAyX9Q9JiSd9IyDlE0tcl/SvKc0PSgyTpKEk98VzD+evqrkBG3m9me6Tsgal6SbV6UGxEYn9Ecmdc8WvOWUP3ad1VtTxm9jr29IDZugojBx4Y9jwm8ajbypH02VxHveXKZnFKzZu0dsdxGgMze5CQ1uu9selV4LPApsBHgZMkHRbPZfZjb2pmG5vZXwk5ZS8AtgbeSkgPdm7KWpnxu8fx18fXWwKbEyyEEwi6y9Xx9VhC6rIfZ033GYLV9A3AesBXY/vRwKgoxxaETBYrzOwYoAu4MK59F9ABvAvYA9gd2As4O7FGtlyZtg0IhRcmEoIQjwT2jO/hREmZm/wvAYcRLL1bAy8BlwJI2oWg7xwVz21BSKfWMDSKApmPqpZUK4dqRE9nW/6GamjOfvXMnVaN60zb6zh9egiYaWkJkbe1jLodZNHe70wc7yV8MR9ST4Fy4RZox2lq5hMUJczsbjObbWZrYsaH6wgKUE7M7Bkzu9PMXjOzBcAP8vVPYQ1wTpxjhZktMrNfmdlyM1sGdOaY82oz+4eZrQBuICiBACsJytiOZrbazGaZ2dKUdduB88zsxSj7JIJCl1OuxPydZrYS+CUwGrjYzJaZ2RPAE4SsGRBSrnWY2Twze43w/X143EN6OHCbmd0bz30zrtcwNIICacDvJc2SNCHH+X4vqZakmjVpk5a/az52TUPleavWdebb69jeHiJu16wJj+3t1Vf20iygA1WJNLMvJo4TCNGD69Vbrmw6O90C7ThNzDbAYgBJe0v6Y3QfLyFY8EanDZT0Bkm/lPQfSUuBqfn6p7DAzP6bmHOEpJ9G9+5Sgut8U6mXZeb5xPPlwMbx+RTgd8Avo1v6QknDU9bdGkjmj+iJbTnliiwyi8EN64o6vJA4vyIhSwtwS3Slvww8BawG3hjXWav7mNmrQI6d5PWjERTIfczsHQRX9SmSslOSFFVSTVUsp5akVjVpGy3PW7WusxRLUy2UPY/2Zjlhu0dD0d7efxZox3Gqh6R3EhTI+2LTtYRCA9uZ2Sjgctb9TucKPrkgtu9mZpsQ3Lm5ftfzkT3v6cCbgb3jnBm9oeC8ZrbSzCaZ2S7Ae4CDCC75XMwnKHkZxtK7IleleRDnAh8xs00TxwZm9h/gORLVwCSNIFhOG4a6K5BmNj8+vgjcQthjkKSokmrVLKeWpJY1aWu5F7FUqnWdpViaaqHsDbZob0nTJN0aj9uAp4Hf1FuuXOSyQDuO05hI2kTSQQQ37FQzmx1PjQQWm9l/Je1F2GuYYQHBzZoM5BsJvEIIrNkG+FqBpV+gcBaJkQRL3ssx6OScYq4JQNL7JY2L1sqlBJfz6pTu1wFnSxoTg3AmEiyo1eJyoFNSS5RtjKRD47mbgIMk7StpPeA8GkBnS1JXYSRtJGlk5jnwYeDxrG79XlItSaU1aRuh+kwxVKv2bimWplooe4Nwr933gf+LxwXA+8zs6/UVyXGcJmaapGUE61gHYc9iMoXPycB5sc9Ewv5CAMxsOWE/4p+jW/ZdhH2D7wCWALcDNxdY/1zgmjj+iJQ+FwEbEnI23g/8toTr25KgnC0luIzvIV0p/DYwE3gMmA08HNuqxcUEHef38f28n5Ayibhf8hSCxfc5QoDNvCquXTF1rUQTI5FuiS+HAdeaWadiOTUzu1ySCNFVBxDcc58zs7xlZqpZiaaSCjKFxmYq28xZMoexo8bSOb6zblbIelTKqUV1kkoq3qhJq2HEu9edYu62DYFhcWN5v5Pvs9fVFazLc+YEhb6z062QTqBZP3uOM5ipayJxM3uWEBqf3X554rnRO0lyv5JRnvYXTi4AACAASURBVMpR9ArtK0wqbP2VSDyNSq6zXDo7cyt7lQRWZBSSwaKoSDqBkD5ic2AHwhaPy4Hx9ZQrm2zFPrPfFQbu38ZxHGcg47WwS6BUi2Fa3Wkhxo4aS8+Svua3WtajbkQaySrVjFYQSY8Q9g0/YGZvj22zzWxcPeTxWthOOTTjZ89xBjteyrBIsl28xVgM05TEsaPG1jQ4p5lob3cLVIW8Zmavh50eoZwWlUcGVp1cymO+dsdxHKexaaiInkamnDQ3uarPZHI9VitoJUmzBOwkGWRJv2vBPbE01oaSPgTcCEyrs0x9GJo7b35qu+M4jtPYuAJZJOVYDPPlesynXJZDNROe9xeDLel3jfg6IXXGbEJVg+n0LrXVEKxOSZKR1u44TuMjqVWSRc/HoEDSMZLuK9xz4OMKZJGUazFMy/VY7UTitUp4XktqkQdysFk0YzmxK8zsE2Z2eHzecC7slpbS2h3H6V8knSspb45DSd2SPthfMhWLpP0lNVSKm0qR9HNJVUsZVO35wPdAFk3n+M6caW4qKT3YPq69alHOzbinstp5IAdjpK+kfQh501oIn2cRkhcUSsTbr9Qi4t5xHKc/GUyW1mJwC2SRNFrpwWxqsaey1lQ76fcgLWP4M0Ki332BdwJt8bEgkg6Q9LSkZyT1ST4uaWysefs3SY9JOrBcIb2UoePUH0kXS5oraamkWZLeG9sPAL4BfFLSK5IezTF2CqGU37TY54zE6XZJcyQtlNSRGDNE0tcl/UvSIkk3xMoxuWTbX9I8SadLelHSc5I+lzi/vqTvx3VekHS5pA1jEZI7gK2jXK9I2lrSilg9BklnS1olaZP4+tuSLorPR0n6hUJt757Yd0g8d4ykP0v6oaTFhJv1bLm/J+k+SaNynFtf0kUKNbfnx+frJ+a+L6u/SdpR0gSgHTgjXs+0eL5b0lmSnpT0kqSrJW1Q7nyV4gpkCTRS6cFs0vZUHrh+Z8O6dEspe1gMg62MYWSJmd1hZi+a2aLMUWiQQhmvSwk16HcBPi1pl6xuZwM3xPRAnwIuq0RQL2XoOHXnIWAPQt7Ya4EbJW1gZr8FzgeuN7ONzSxXfuajgDnAwbHPhYnT+xJqU48HJkp6a2z/EnAYsB+wNaGayqV55NsSGEWovX0ccKmkzeK57wI7R/l3jH0mmtmrhO+x+VGujWOJ5IfiuhBqZfcA+yRe3xOf/yiu+abY/7P0rryzN/As8AZClR1grXJ8BbAb8GEzW5LjejqAd0WZdyekXCu4R93MJgNdwIXxeg5OnG4H/oeQ93fnKsxXNq5ADhByWUiP3mwy13y1vWGDVKptlRqEZQwB/hjvgN8t6R2Zo4hxewHPmNmzZvY6od7toVl9DNgkPh9Fjhr0juM0D2Y2Nd5krjKz/wPWJyh+lTLJzFaY2aPAo6wrEPJ5oMPM5pnZawQL3uF5XMErgfPMbKWZTSfU0H6zQp6yE4Avm9niWGnrfMKNbRr3APvFtXYDLomvNyB4af4Ub6Q/CZxlZsvMrJtQFvaoxDzzzexH8T1bEduGE+pkb05QqLN8X2tpj9fzopktIJR1PCqlb7H82MzmmtligkL76QrnKxv35w8gsvdUtramu3QbxfpTTh7ItOTjg3Sf3d7xMZmE2YAPFBi3DaHWbYZ5ibkynEuo0fpFYCOg4TbPO45TPJJOB44nWAMzN4ijqzD184nny4GN4/MW4BZJaxLnVwNvBP6TY55FZrYqx1xjgBHALMWct4T93vkSgd1D2N7zDkKWijsJW37eRbh5XijpjcB6BOtkhh7C92OG5Pdkhh2JFsV4A57G1jnm3jpP/2JIylON+crGFcgBzEB06RYTKNMolW36AzN7f5lDlaMtO3r708DPzez/JL0bmCLpbWaW/DEg7q+ZADB2gJt7HadZifsdzyS4mZ8wszWSXmLdd0Ex2RtKzfAwFzjWzP5c4rhsFgIrgF3NLJfimUuuvxCsqx8D7jGzJyWNBT7KOvf1QoLVswV4MraNpbdym2vupwiu+DskfcDMnk6Re36c+4nE3BlPzqsEpRgASVsWcU0A2yWeV2O+shl0LuyTbz+ZYecNQ5PEsPOGcfLtJ9dbpJoxEF26hQJlfJ9d0cyj9xfRtvR1UR8H3ABgZn8FNiCHtcLMJptZm5m1jRkzpkbiOo5TISOBVYS8scMkTWTdFhWAF4DWTABJCi8Q9goWy+VAp6QWAEljJGVvlSlIvGm9AvihpDfEubaR9D8JubZIBrJEt/Is4BTWKYx/IbjV74l9VhO+4zoljYxyfgXIm84ojr2OEHh0l6QdUrpdB5wdr3s0MDEx96PArpL2iG71c7PGpr3Xp0jaNgYjfQO4vsL5ymZQKZAn334yP5n5E1ZbyF682lbzk5k/aVolslDlmWoHqTQCA9GqWiceAnaStL2k9Qh7iW7N6jOHYK0gborfgPDj4zhO8/E7QrTyPwiuz//S2x16Y3xcJOnhlDkuIChEL0v6ahFrXkz4Xvm9pGXA/fTdKlMsZwLPAPdLWgrcRdy/aWZ/Jyhrz0bZMm7dewj7FR9MvB4J3JuY94sE692zwH2E4KKrihHIzK4BzgP+IKk1R5dvAzOBxwhu9IdjG2b2jzj2LuCfce0kPwN2idfz60T7tcDvo7zPVmG+slED5hyumLa2Nps5c2af9mHnDVurPCYZqqGsmriqT3sjk12bG0LUdXZqobT9gs1Ka2vu+sktLcHiWAmSZplZW+GeAwOFtDwXEfYRXWVmnZLOA2aa2a0xKvsKwh4kA84ws9/nmzPts+c4+Rhsnz3HKQdJ3cDxZnZXvWWBQaZAalKubV+BllEtdI7vbKjUPPlovaiVniV9NamWUS10n9bd/wL1E9l7ICFYVauRU7BZf8QkvQdoJbGn2cx+UQ9ZXIF0yqFZP3uO0580mgI5qFzYQ5UesFXt2tGF3MuFzheimSrPVLO8oCek7o1Cct/vsy6ReCaZuOM4DYoKJ/E/Jia2fiQex9dDTsfJx6CKwp6w5wR+MvMnqecztaMrtUKefPvJXD7zciwGPWWUUwipdrLdz9nni2HsqLE5LZCNVnmmFuUFy0n9M4BpA3ZpxPrXjuP0JZHE/0OEYLaHJN1qZk9mdb3ezL7Q7wI6DYuZtdZbhiR1s0BK2k6hTNpTkp6QdGqOPvtLWpK4C5tYyZqXffQyTmo7Ka8lslILXtfsrl7KY4aMcgrQMaOj197FzPlT7+jzFqTSOb6T9dQ7QmY9VVabuxYM0vKC/cnjhOoNjuM0B8Uk8XechqeeLuxVwOlm9lZCYs9TcpRSA/iTme0Rj/MqXfSyj17GqomraBnVkvN8pRa8jhkdfZTHDBnlNE1JXbRiUfGu7MfasVsnw8stYIKXW8LrxxrLNJcr4CVfu1Myo4EnJf1O0q2Zo95COY6TSq4k/tvk6Pe/CjXob5K0XY7zjlNX6ubCNrPngOfi82WSniJ8iLLN+DWhc3xnzijmSi14+SyYGeU0zf0McOTNR9Ixo6NgQE9HB6zsaYdZ6/qspLGqzAAMHQqr+wa+MzRf/QCnFM6ttwCO45REMUn8pwHXmdlrkk4EriFHdalkEv+NNtpoz7e85S3VltUZBMyaNWuhmZWcxLch9kDG/ElvBx7Icfrdkh4lJDn+qpk9kaNPyWSUs44ZHcxZMoexo8ZWJQp78w03Z9GKRTnPZZTTzvGdHHnzkalzFLMnslnyIeZSHvO1O6VhZvfE5Lc7mdldkkaQv7yX4zj1pWASfzNL/ohcAXw310RmNhmYDJ4BwSkfSWX5BOsehS1pY+BXwGlmtjTr9MNAi5ntDvwISE1+KWmCpJmSZi5YUFyu4/Zx7XSf1s2ac9bQfVp3TVP4bDR8o7Xzt49rZ4sNt8jbP7lnMhfNUmWmJfdOgdR2pzQknQDcBPw0Nm1Dns+J4zh1p2ASf0lbJV4eQiid5zgNRV0VSEnDCcpjl5ndnH3ezJaa2Svx+XRgeCwH1IdGKae2eMXinO3ZQTMXf+RiRgwfkbNvhnzu8GapMtMscjYxpwD7AEsBzOyfwBvqKpHjOKmY2SrgC4TKME8BN5jZE5LOk3RI7PalGFz6KPAl4Jj6SOs46dQzCluE0jpPmdkPUvpsGfshaS+CvLn9ww1CWhBOdnv7uHYmHzw5NZgn31xQfj7EauZkLIaMnFskDK4bbljbNQcZr8VITgAkDaPvfirHcRoIM5tuZjub2Q5m1hnbJprZrfH5WWa2q5ntbmbvj6X6HKehqKcFch/gKOADiTQ9B0o6MW4aBjgceDzehV0CfKrR8911ju/sY1lMC87JuNCnfnxq0WN6jW8P5fvWrAmPxSiPEyaECGizdTkZa61EAqxYse75okX9t+4g4B5J3wA2lPQhQj3baXWWyXEcxxngDKpShv1F1+yukoNzyhlTKrWsI92I65ZKM5ZTkzQEOA74MCG683fAlfW60ar3Z89pTprxs9do+GfPKZdyP38NEYU90Ggf116y8tc+rh0ea6fjohBJ3TEW6KxuSp56RW43S8R4M2JmawhRmlfUWxbHcRxn8OAKZINQi5J/2Ywdm9sSWOvI7XqtO5CRNJs8ex3NbLd+FMdxHMcZZNQ9jU8z0DW7i9aLWhkyaQitF7UWXy2mBPqj5F+9IqI9ErsmHAQcDPw2Hu3xmE5I6+M4juM4NcMVyAJ0ze5iwrQJ9CzpwbC1Sb6rrUT2h5u33MjtDOVGcFe6rtMXM+sxsx5gHzM7w8xmx+PrwP/UWz7HcRxnYOMKZAE6ZnT0yeG4fOVyTr3j1Kquk+bOHTKkr6JWqiKX7N/RESx/xUZuJ+eoJIK71Ihxp2g2krRv5oWk9wAb1VEex3EcZxDgCmQB0pJ5L1qxqKpWyFxuXggl/5KKWqmKXLVS9/SHi90pi+OASyV1S+oGLgOOLWagpAMkPS3pGUlfT+lzhKQnY1Lja6sntuM4jtPMuAJZgM033Dz1XL5Sgxly7Z/MZUHMuHmH5qhinFTUSlXkqqX4eSR1Y2Jms2Kpz92A3c1sDzN7uNA4SUOBS4GPALsAn5a0S1afnYCzCG7yXYHTqn4BjuM4TlPiUdh56JrdxdLXsstzryNfqcHM+AnTJqx1gfcs6eHYWyZgt8LKnuDD7emBY4+FU0+FxYuDlTDnWnN6P6adh6CQdnSEtkLzFYtHUjcWkr6S0g5AWnWnBHsBz5jZs3HcL4FDgScTfU4ALjWzl+KcL1YotuM4jjNAcAtkHjpmdLByzcrU8/lKDWbGZ++ffN2Ws/K9vc1/r78eqrPkS/2cUdQ2TzGIZs5nu6wLzVcsuVzsEhx4YGnzlEt/l2BsAkbGow04CdgmHicSLIqF2AaYm3g9L7Yl2RnYWdKfJd0v6YCKpXYcx3EGBK5A5iGfhbGYUoOp40eVZv7LpLzp6oKlOQyi6623LiVOLpd12nyl0N4ORx8dlMYMZnDNNbVX5upZgrFRMbNJZjYJGA28w8xON7PTgT2BbYuYQjnasm85hgE7AfsDnwaulLRpn4mkCZJmSpq5YMGCUi7DcRzHaVJcgcxDmoVxqIYy+eDJBavNpFoolxRn/stOedPRAStzGERHjlwX1ZzPNV1pCp3p0/taNfsjkMYDePIyFng98fp1oLWIcfOA7RKvtwXm5+jzGzNbaWb/Bp4mKJS9MLPJZtZmZm1jxowpRXbHcRynSXEFMg+d4zsZMby333bE8BFc87FriipVmGv8ehrB8D8VNv+1tPRNeZOmHC5evO55mms613ylkmsPZD65qoUH8ORlCvCgpHMlnQM8APyiiHEPATtJ2l7SesCngFuz+vwaeD+ApNEEl/azVZPccRzHaVpcgcxD+7h2Jh88mZZRLQjRMqqlKMtjvvFXfWwyV3+5fW1S7S22gOHDe49LczGnKYfJ9lpVfenq6u2+LkaualHMdQ9WzKwT+BzwEvAy8DkzO7+IcauALwC/A54CbjCzJySdJ+mQ2O13wCJJTwJ/BL5mZotqcR2O4zhOk2FmA+7Yc889rZmYOtWspcVMCo9Tp6b3GzHCLDiSwzFiRN/++eYrdq1sWlp6r5s8Tjqp8mvLR7HXXSnATGuA/99iDmCT+Lh5rqNecjXbZ89pDJrps9eoh3/2nHIp9/NX93/6WhwD+YNUiUJWiSImWaoC2dJS2XrFXFM1FNFCNNOPGHBbfPw3wa2cOf4NPFsvuQbyZ8+pHc302WvUwz97TrmU+/lTGDuwaGtrs5kzZ9ZbjIajtTX3PsaWlrA3spyxEFzba9aUt14mwjoZJDNiRH1qZUuaZWZt/bvqwCLfZy+Zn3Ts2LCtwktaOuCfvWrgv3tOuZT7+fM9kIOISoJROjtL3wNZzHoeYV05kraR9B5J78sc9ZYpm66ukDA/mYrp2GMHdyomx3GcZqauCmShWryS1pd0fTz/gKTW/pcykKskYbNRSTBKezuceGJfJTJfgE4x63mEdWVI+i7wZ+Bs4Gvx+GpdhcrBqaeGhPlJXn89tDuO4zjNR14FUtImknbI0b5bpQsXU4sXOA54ycx2BH4IfLfSdcshU5KwZ0kPhtGzpIcJ0yY0nRJZaYT2ZZfBlCmsjSAvlFOymPU8wrpiDgPebGYHmtnB8Tik4Kh+ZlFK7HZau+MMZJrJeNIIaJLCcW48zhKjP/tF3nj2XhxzwxeZu2Ru4UlqwNy58MUvwu67w267wR57hNdzyxBH6ns0OqkKpKQjgL8Dv5L0hKR3Jk7/vAprr63Fa2avA5lavEkOBa6Jz28Cxkv9/7bmKkm4fOVyOmY0l5+1vT0ofMUqgGlzdHcXl1OymPVqlXZoEPEsMLxgL8dxGoJmMp40ApoUf/KNdfWz1odFb/oxLw5/iGse/ynjLtu935XIuXOD4nj55fDYYzB7Njz6aHi9++6lKZFpWk2jK5H5LJDfAPY0sz0IeeamSPp4PFeNyyqmFu/aPhby1i0Btqh04VLd0WklCfOVOmxUSlEA+2O9aii1gxFJP5J0CbAceETSTyVdkjnqLV82W6R8atPaHWcA0zTGk4YhqTxmPw5dybLXXuHCP1/YryJdeCG88gqsWtW7fdWq0H5h/4pTF/IpkEPN7DkAM3uQUJGiQ9KX6FsztxyKqcVbTJ/Qsch6vOW4o9NKEqaWKuwnurpCpPOQIeGxWQMS+lupHSDMBGYRqsd8C/hLfJ05GoqLL+6bMH/48NDuOIOMqhlPBk0d+mxNIOv1Gq3kwf882G/iADzwQO7SwhDaH+xfcepCPgVyWXL/Y1Qm9yfcGe1ahbWLrcW7HYCkYcAoYDE5sCLr8Zbjjk4radg5vn5+1kz6m2RU64QJzatEOqVhZtfkO+otXzbt7XD11b0tzVdf7TcLzqCkasaTYn/3mp7sK896PcSGs9c2e/WbOAB77933pjjD8OGwV/+KUxfyKZAnAUOSezPMbBlwAHB8FdYuphbvrcDR8fnhwB+swsSV5bijKy1pWAsaPf3NQLGOOtXDLc2OA1TZeDIoEOuUxuzH1cMZuf7GnLHPGf0q0hlnwMYbw7BhvduHDw/tZ/SvOHUhVYE0s0fN7J/ADZLOVGBD4AfAyZUubMXV4v0ZsIWkZ4CvAH2i1UqlXHd0+7h2uk/rZs05a+g+rbvflcdshSwtqXcjpL9x66jjOE4qdTGeNCt2TrzspBL5Gmzx7Bd4w8q9OPptn2f2yY+y3ajt0qaoCdttF4JmTjwxBM2MGxceP//50L5dCeKk/WUb/S8+rHAX9iZEgP0FGAl0AftUY3Ezmw5Mz2qbmHj+X+AT1VgrQ+f4TiZMm9DLjV1vd3Qhsqu19PQEN2Cuf65qpb+ppGpIPuuoW52qR4zm/I6Zfa3esjiOUxxmtkpSxngyFLgqYzwhlJS7lWA8mRKNJ4sJSuagZa0S2WBstx386EfVmavRlcVcFKNArgRWABsCGwD/NrMcheuag4zlsGNGB3OWzGHsqLF0ju+sqzu6ELkUMrO+SmS10t/kUlgnTAjPi1EAPTl4/2BmqyXtKUmD1TrhOM1IPYwnjlNtilEgHwJ+A7yTEAX2U0mHm9nhNZWshrSPa29ohTGbNMXLLAQjVLu2cKUWxLFjc7vYPTl4Tfgb8BtJNwKvZhrN7Ob6ieQ4juMMdIopZXicmU00s5Vm9ryZHUpQKJ1+Ik3xammpTVBCpRbE/k4OPhDKTFbA5sAi4APAwfE4qK4SOY7jOAOeghZIM5uZo21KbcRxctHZ2dulDLVVyCq1IGYU2XL3UJZCJq9nZk9rJq8n0FRW5nIxs8/VWwbHcRxn8FGMBdKpM9Wo1lJKWp1qWBD7K2VLobyeA906KWlnSTMkPR5f7ybp7CLH5q3Hm+h3uCST1FYtuR3HcZzmxhXIJqEShazUtDrNVF4wX17PcqoONSFXAGcRgt0ws8coImKzyHq8SBoJfAl4oIoyO47jOE2OK5CDgHKSjjdL0ud8eT3LqTrUhIyIpUaTrMrZszfF1OOFUCbxQuC/lYnpOI7jDCRcgRwEpAW/pCUjbybylZksp+pQE7Iwlhw1CO5m4LkixhWsxyvp7cB2ZnZblWR1HMdxBgiuQA4C0oJfpOavDpOvzGS5VYeajFOAnwJvkfQf4DRCGdJC5K21K2kI8EPg9IITSRMkzZQ0c8GCBcVJ7TiO4zQ1rkA2MNWqJ33ggbnbzRqndnYlpJWZzGedHChEF/QHgTHAW8xsXzPrLmJooXq8I4G3AXdL6gbeBdyaK5DGzCabWZuZtY0ZM6bMK3Ecx3GaCVcgG5Rq1pOePj39XCnVYaql0PYX+ayTAwVJ60v6DHAq8GVJEyVNLDSOAvV4zWyJmY02s1YzawXuBw7JldbLcRzHGXwUU4nGqQPVrCedT0ksNrdjvvKG7NZVk9KQXbMrn7fZqg6VwW+AJcAs4LViBxVZj9dxHMdxcuIKZINSzXrSaYnBpeJzO6YptKde2cWKD1U/kfdgTxBeAtua2QHlDCxUjzerff9y1nAcx3EGJu7C7kdKcQGnWQbLqSedKzG4BCeeWLw1M01xXbRHbVLlDJIUPNXgL5LG1VsIx3EcZ3DhCmQ/kWtP45FHwujRuRXJataTzpUYfMoUuOyy4udIVVxH1SZVziBJwVM2kmZLegzYF3g4VpR5LNHecDTbHlrHcRwnHXdh9xO5XMAAixat20uYtAZWu550e3tlycDT6nFvOHwsi1b19Y9Xmipn7Kix9Cyp/rwDiIPqLUAp5NtD26hJ6h3HcZx03ALZT+Tbu5hWFaaRqsGklTe8+JDapMoZDCl4KsHMesysB/h25nmyrd7yZVNONSTHcRyncamLBVLS94CDgdeBfwGfM7OXc/TrBpYBq4FVZtYnB12zkBbIkqGc4Jj+JrcVMzRUOwo7M74W0d0DjF2TL2KN6z3rJEsq1QwKcxzHcepPvSyQdwJvM7PdgH8AZ+Xp+34z26MeymM192zl2tOYpJzgmGpQjWtMS+RdKbWadyAg6SxJy4DdJC2VtCy+fpGQ2qehqGZQmOM4jlN/6qJAmtnvzWxVfHk/oQpGQ1HNRN6wzgW8xRZ9z5UbHFMp1b7Gktae3UXrRa0MmTSE1ota6ZrtERWlYGYXmNlI4HtmtomZjYzHFmaW74asLlQzKMxxHMepP42wB/JY4I6Ucwb8XtIsSRNS+tSEauzZyrbuASxcCFOn9t1LWI/9jfXal5bJ8dizpAfD1uZ4dCWydMzsLEmHSPp+PBoyuCZtD60H0DiDCUmbS7pT0j/j42Yp/VZLeiQentTfaUhqpkBKukvS4zmOQxN9OoBVQJrmsI+ZvQP4CHCKpPflWW+CpJmSZi5YsKBi+Svds5XLunfUUeHHs6MjWF7qHRxTr31pnuOxeki6gFDG8Ml4nBrbHMdpPL4OzDCznYAZ8XUuVsStW3uY2SH9J57jFE/NgmjM7IP5zks6mpCKZLyZWcoc8+Pji5JuAfYC7k3pOxmYDNDW1pZzvlJIC3opds9WLute5iobJYVJpddYLp7jsap8FNjDzNYASLoG+Bv59xX3O57Gx3EAOBTYPz6/BrgbOLNewjhOJdTFhS3pAMKH5hAzy5EdESRtJGlk5jnwYeDx/pKx0j1bhax4jZDCpF770tJyOXqOx7LZNPF8VN2kyIOn8XEcAN5oZs8BxMc3pPTbIHrU7pd0WP+J5zjFU69E4j8G1gfulARwv5mdKGlr4EozOxB4I3BLPD8MuNbMfttfAlaayLtQ2h4ozlXc1VW9ZOK55po8uXrzF0vn+M5eda7BczxWwAXA3yT9ERDwPhrM+giexscZPEi6C9gyx6lSbpfGmtl8SW8C/iBptpn9K8daE4BMfMBrkvrNyBIZDSwcwOvVY816XOObyxplZgPu2HPPPa3eTJ1qNmKEWXBc5z5aWkqfY8SI0F4NecqdqxpMfWyqtfywxXSurOWHLTb1sToJkgCYaQ3w/1vqAWwFHEJwj21ZT1nSPnstLeV9BpzBQbN+9ko9gKeBrWzd5/bpIsb8HDi8iH79/h7295p+jY21ZiNEYQ9IklGnEIJnkhTjKq6m2y9trqOPrk9tYs/xWD3M7Dkzu9XMfmNmz9dbnlx4Gh/HAeBW4Oj4/Ghy5GyVtJmk9ePz0cA+hAA5x2koXIGsIZlShGYwZUrpKUyq6fZLG7N6dXE5ID1vo1MJnsbHcQD4DvAhSf8EPhRfI6lN0pWxz1uBmZIeBf4IfMfMXIF0Go567YEcdOQuA5ifakZJF7MnM2PdzJYzk7cxs2cxk7cRcMuhUzTlfAYcZyBhZouA8TnaZwLHx+d/AcaVMf3kyqQri/5e06+xgdZ0C2QDU8jtV0oZwkKlFDPkslR63sbGRtK+kj4Xn4+RtH29ZXIcp3+xkMpuQK/p19hYa7oC2cDkc/uVWoYwe66hQ3P3y2Xd9LyNjYukcwgpsTKR18OBqUWOPUDS05KekdQnobGkr0h6UtJjkmZIaqme5I7jOE4z4wpkg5PZNhhYlQAAFolJREFUR5ldtaacAJvkXNdcU3xQg+dtbGg+RojAfhXWJt8fWWiQpKHApYQqT7sAn5a0S1a3vwFtZrYbcBNwYRXldhynAiR9QtITktZIasvTL++NYolr9kspxiJubteXdH08/4Ck1tKvpqT1jpG0IHFNx1e43lWSXkxLu6TAJVGexyS9o5L1ilxzf0lLEtc4sdCcrkA2KZUG2JQS1NA5vpMRw3trm563sWF43UIeBoO1SfeLYS/gGTN71sxeB35JSAO0FjP7o61L9H8/sG2VZHYcp3IeBz5OSnU2KPpGsRRqXoqxSJmPA14ysx2BHwLfLWWNMtYDuD5xTVfmOF8KPwcOyHP+I8BO8ZgA/KTC9YpZE+BPiWs8r9CErkA2KWmBNCUF2OzWBae1wjlDwuNuuf3f7ePamXzwZFpGtSBEy6gWJh882QNoGoMbJP0U2FTSCcBdwBVFjNsGmJt4PS+2pXEccEfZUjqOU1XM7Ckze7pAt4I3iiVyKKEEI/GxFlVyipE5KcdNwHgpO1leVderKmZ2L7A4T5dDgV9Y4H7C9/tWNV6zZFyBbDCKDYypNK9eJrK6Z0kPhq2NrE5Lz+N5GxsTM/s+4Qv0V4RqAhPN7EdFDM31ZZuzhrykI4E24Hsp5yfEsmszFyxYUJzgjuP0B6XeKBaiP0oxFiPz2j5mtgpYAmxR4jqlrAfwv9GdfJOk7cpcq9oyVZt3S3pU0h2Sdi3U2RVISotmrrUcxQbGVJpXzyOrBw5mdqeZfc3MvmpmdxY5bB6Q/BLcFpif3UnSBwkl2A4xs9dS1p9sZm1m1jZmzJhSxXccJwVJd0l6PMdRrIWs6BvFKq4JoRRjG/AZ4CJJO5QwthiZS76uCtebBrTG/eB3sc76WSuqeX3F8jDQYma7Az8Cfl1owKBXIHMpbUcdBSefXNs1cymspQbGpAXYFINHVg8MJH08bmhfImmppGWSlhYx9CFgJ0nbS1oP+BShSkZy7rcDPyUojy9WX3rHcfJhZh80s7flOPpUsEmhqBvFEtZ8IeNKjY85vxdiMB9m9ixwN/D2IuUtVua1fSQNA0ZRvnu24HpmtihxA30FsGeZa1VNpmpjZkvN7JX4fDowXKESUiqDXoHMpbSZweWX18YSmc/KWM3KM4Xor8hqr2BTcy4kKHijzGwTMxtpZpsUGhTdPl8Afgc8BdxgZk9IOk9SZtP794CNgRvLjaZ0HKeuFLxRLJH+KMVYjMxJOQ4H/hCDCcuhmJvp5P7DQwjfmbXkVuCzMRr7XcCSzNaBWiFpy8w+Ukl7EfTDRfnGDHoFMk05Myuv5nQh8lkZqxIYUyT9EVld6j7LYud0hbQXL5hZWV9mZjbdzHY2sx3MrDO2TTSzW+PzD5rZG8uNpnQcp3ZI+pikecC7gdsl/S62by1pOqTfKFawbM1LMRZ5c/szYAtJzwBfIT0avFrrfUkhZdKjwJeAY8pdD0DSdcBfgTdLmifpOEknSjoxdpkOPAs8Q7B4VuwTLWLNw4HH4zVeAnyqkFKu8pX2xqWtrc1mzpxZVN/W1vQSf1JwD1eTIUOCcpprrSlTgjUyqWCOGFG7msFds7vomNHBnCVzGDtqLJ3jO6saHNN6USs9S/q+uS2jWug+rbvk+bJLKkJQeqsVES5pVty30/BI+nh8uh+wJWG/yto9imZ2cz3kyvfZO/n2k5k8azKrbTVDNZQJe07gso9e1s8SOo1IM332HMcJDPpa2J2dYc9jLqWuFpa/fPWtk0nC/7+9O4+aq67vOP7+JCRsDWhCEKiVYPGIFJAlbKIeFkWlrBoL+FRDoCKlKGDBQkHW5lRAC1gKyBICGCKCgKHhgBRawIpAgsiiwOHQRFkKYSn7lvDtH/dOnsvkzsyd55mZO8vndc5znju/ucv3zpwn+d3f9v3DH5KymTPblz94aNOhts6mbvU4y3oTfwZwVvgeme3XgV0zrwMopQJZy6HzD+W8BcNLmS2LZctfuxJpZtZ7Br4Le2gIDjkkaQHMamZJnGY0Wn5nNBNjOqlIV3Krx1l64s+wiJgRETOAiyrbmbKLy46v2gUL81Ot1io3M7PuNvAVSIBzz026j0e6JE4zRrv8ThmqK4uHzj+00NjGVo+zdErFXHlrPhZZB7KjlsWypsrNzKy7DXwXdsXQUOcqcZ281mhVjztc/NJizl9wPlG1JFVeV3Jlu1XjLGfuMjN3DOQgplSUtD3wCWCypG9n3loDGFtOVLWN1djcyuJYdV2oZmZWQCktkJJOkvRkJmn3bjX2a1kS+G7XLYuZV8sbd1hdeazI60puZQYbp1R8j/EkS+ysBEzI/LxMMpuuq+w4Zcemys3MrLuV2QJ5ZpqGLVcmwflnSRbVvEfSvGaWA2inOXPg8MPh+XSVpEmT4Oyzm2tZnDMnmTCzeHHSnV2ZyFNZGxLKb6lsZnzhGI1hzMlj2jKju6LdE396RUTcBtwmaXZE1FhHoHs89sJjTZWbmVl36+YxkB1PcF7UnDlw4IHDlUdItmfMKN5ymF1QHFacBV4vA00n1RpfqJxMS8tiWcvWe7RieqHyCJ4AZWbWb8qsQB6WJiafJen9Oe+XlUy8oeOOg7ffXrH8nXeKV/ryFhSv1o4MNM2qNRHmkKmHLO9KzhvH5rzaluUJUGZm/aVtFcgGCdnPA/4c2Bx4GvhB3ilyymquei7pYEkLJC1YsmRJS+6hlnoVu6KVviL7tWMdymbVGnd47l+eu3xs47uRv9p6Wa1Lg5CtRtJp6e8vlx1LEZ3IfGRmZp3TtgpkvYTsEfFMRCyLiHdJ0vRsk3OKppKJR8QFETE1IqZOnjy56XibmcRSr2JXtNLXaL92rUM5Eo0mwnRT61I70id2qd0kjQOOLTuQIjwBysysv5Q1CzubmHwf4MGc3VqdBL6m7HjEiOFJLLUqkTNnwvjxK5aPG1e80pe3oHhlMfNeWBsyq5tal+plq+kzNwLPAZtJelnSK9nfZQdnZmb9rawxkKdLekDS/cBOwJHQ9iTwNeWNR6w3iWVoCGbNSmZeV0yaBJdcUrzSl7eg+OWXJxXYbs5Ak6ebWpcGZbJGRBwdEWsC8yNijYiYkP1ddnzVBqhl2MxsICjykkD3uKlTp8aCBQsK7z9mTH4ubClJKWi9Y8pZU1j80ooTk9dfc30WHbGo7rGSFkbE1DaF1jaSPgBsnb68KyLaOwi4jlp/e6P5Xqz/9erfntkg6+ZlfDqm1njEbpjEYs3ppu70Tkgn0dwNfBn4K+BuSV23kPigtAybmQ0KVyDJH49YaxJLt2aMsUQ3dad3yPHA1hExPSK+RjIh7bslx7SCiatObKrczMy6myuQ5I9HzJvE0uxkG2uvWsv1tDJ9Yg8YExHPZl4/T8G/60apQiWtLOnK9P27JE1pTchmZtbrykxl2FWGhhpPXKk32aaXJr30g8qkjMqM68qkDKDfK4zVbpR0EzA3fb0vcEOjgwqmCj0IeDEiNpS0H3Baev6mPf/G802Vm5lZd3MLZBNqLf7dDRljBs0ALddTV0QcDfwI2Az4OHBBRPxDgUOLpArdC7g03b4a2EVS3gL/ZmY2YNwC2YQPfWg4d3V1uXWWJ2UMi4hrgGuaPCwvVei2tfaJiKWSXgImkaw/aWZmA8wtkE1oZrKNtVc3Zb/pUUVShRZKJ9rJNKJmZtYdXIFsQtHJNtZ+g7ZcTxsUSRW6fB9JKwFrAi9Un2i0aUTNzKz3uALZpKGhJFPMu+/2XsaYfjKAy/XkknR4kbIcRVKFzgOmp9vTgFujHzMPmJlZ0zwG0nrW0KZDA1dhzDEdOLuq7ICcsvdIxzRWUoWOBWZFxEOSTgEWRMQ84GLgckmPkbQ87jfSIOPEQCev2CMeJ7o+ambWi1yBNOtBkvYHvgJsICnbcjiBZC3IhiLiBqqW/ImIEzLbb5JkuGkJVxbNzPqHK5BmvelXwNPAWsAPMuWvAPeXEpGZmQ2MvqxALly48DlJOQvuNG0t+m/Jkn67p1bez/otOk/bRcRiYDGwfdmxmJnZ4JHHxNcmaUFETC07jlbqt3vqt/tplqRXGF5aZzwwDngtItYoKZ4lJBXbevrtIaYZg3zvUPv+148IT+E36yF92QJpNigiYkL2taS9SbLMlKJIJWCQK/2DfO/g+zfrJ17Gx6yPRMR1wM5lx2FmZv3NLZD1XVB2AG3Qb/fUb/fTFElfzLwcA0wlJ1uMmZlZK7kCWUdE9F3lpN/uqd/uZwT2yGwvBRYBe5UTSmGD/J0N8r2D79+sb3gSjZmZmZk1xWMgG5B0kqQnJd2X/uxWdkwjIenzkh6R9JikY8qOZ7QkLZL0QPqdLCg7nrJI+rCk6yUtkfSspJ9L+nDZcZmZWX9zBbKYMyNi8/Tnhsa7dxdJY4F/A74AbAzsL2njcqNqiZ3S72SQZ3VeAfwUWBdYD7gKmFtqRKlGDy2SVpZ0Zfr+XZKmdD7K9ihw7weklf7Kg+nflBFnO0ialT7MPFjjfUn6YfrZ3C9py07HaGaj5wrkYNgGeCwiHo+It4Gf0P3j5KwYRcTlEbE0/fkxXTCJpuBDy0HAixGxIXAmcFpno2yPJh7Yrsw8mF7U0SDbazbw+TrvfwH4SPpzMHBeB2IysxZzBbKYw9In5VmS3l92MCPwp8AfM6+fSMt6WQC/kLRQ0sFlB1Oi/5R0jKQpktaX9B1gvqSJkiaWGFeRh5a9gEvT7auBXSSpgzG2y0A/sEXE7cALdXbZC7gsEr8G3idp3c5EZ2at4lnYgKT/ANbJees4kqfjU0kqLKeS5B0+sHPRtUTef8qlt1KN0g4R8ZSktYGbJT2c/sc1aPZNf3+jqvxAku+4rPGQeQ8t29baJyKWSnoJmETvZ2opcu8AX5L0aeBR4MiI+GPOPv2o1gPt0+WEY2Yj4QokEBGfKbKfpAuBf29zOO3wBPBnmdcfBJ4qKZaWiIin0t/PSrqWpNVnECuQH4uIN7MFklapLitBkYeWfnywgWL3dT0wNyLeknQISUvsoCwA36/fu9lAcRd2A1VdK/sAuQPDu9w9wEckbSBpPLAfMK/kmEZM0uqSJlS2gV3pze+lFX5VsKzTijy0LN9H0krAmtTv+uwVDe89Ip6PiLfSlxcCW3Uotm7Qdw+0ZoPILZCNnS5pc5In5EWs2FXY9dLuwcOAm4CxwKyIeKjksEbjA8C16XC5lYArIuLGckPqLEnrkHT7rSppC4ZbddYAVistsGHLH1qAJ0keWr5Stc88YDpwJzANuDX6Y2Hahvcuad2IqHTZ7gn8vrMhlmoeybjyn5B07b+U+SzMrEd4IXGzHiRpOnAASerC7DqYrwCzI+KaMuLKStdMPYvhh5aZkk4BFkTEPEmrAJcDW5C0PO4XEY+XF3HrFLj3fyapOC4lufe/jYiHy4u4dSTNBXYE1gKeAU4ExgFExPnpRKlzSGZqvw7MiIiBXcvVrFe5AmnWwyR9KSJ+VnYcZmY2WNyFbdbbNpH0F9WFEXFKGcGYmdlgcAXSrLe9mtleBdidwRpPZ2ZmJXAXtlkfkbQyMC8iPld2LGZm1r+8jI9Zf1mN8hYPNzOzAeEu7AEj6UZgO+CXEbF72fHY6Eh6gOFFmMcCkwGPfzQzs7ZyC+TgOQP4atlBWMvsDuyR/uwKrBcR55QZkKQDJK03iuOnSKpeM7LIcWdIekjSGSO9dpPXWyRprZzyPSUdU+OYV2uUz5Y0rdUxdhtJZ6XpG+t9fhtJulPSW5KOypSPl3R7uui8mZXMFcg+JWlrSfdLWiXN3PKQpE0i4haStQKtD0TEYuB9JBXIfYCNy40ISNanHHEFEpjCiouOF/ENYMuIOHoU1x61iJgXEd8rM4ZWa0WlTdJEYLsCOetfAL4FfD9bGBFvA7cwnP/dzErkCmSfioh7SDI+/BNwOvDjiBjUdH99S9LhwBxg7fRnjqRvtvga35b0YPpzRFo2RdKDmX2OknRS2oo2NY3jPkmrpi1Np0m6O/3ZMD3mPa1umda57wGfSo8/sioWpS2ND0p6QNK+afk8YHXgrkpZ5piJkq5LH6h+LWmztPwkSbMk/ZekxyV9K3PMX6ex3ifpR5LG1vh4vinp3jSWjdJjD5B0Trq9Qdqado+kU6vu4xxJv5M0n+S7q7y3laTbJC2UdJPSdKppnJXP8VFJn8r5rnZM97ta0sOS5qQLdzc679R0ey1JizL3cZWk64Ff1Pnsa16zyjSgOmPUCp9fRDyb/vv1Ts45rgOGanwXZtZBrkD2t1OAz5L8h356ybFYexwEbBsRJ0TECSTjW7/eqpNL2gqYQZJybjvg60pSJ+aKiKtJMuMMRcTmEfFG+tbLEbENSQaSsxpc9hjgjvT4M6ve+yKwOfBx4DPAGUrSAu4JvJEec2XVMScDv4mIzYB/BC7LvLcR8DlgG+BESeMkfYyklWuHiNgcWEbtSstzEbElcB5wVM77ZwPnRcTWwP9myvcBPgpsSvJ9fQJA0jjgX4FpEbEVMAuYmTlupfRzPIIkw0ueLdL3NyaZULVDgfPWsj0wPSJ2psZnX+uaOefaAVhYVdbo86v2ILB1gf3MrM1cgexvE4E/ASaQrBFo/UckFZyKZQznxW6FTwLXRsRrEfEqcA2wQstXAXMzv7cfZTxzI2JZRDwD3EbjCsUnSVImEhG3ApMkrZm+Nz8i3oqI54BnSfKs7wJsBdwj6b70da2Z7ZWUkQtJut6r7cDwvV+eKf905j6eAm5Nyz8KbALcnF77eOCDTVwP4O6IeCIi3gXuS/drdN5abo6IF9Ltep993jWrrQssqSorcj/LRcQy4G1JEwrEbmZt5MHI/e0C4LvABsBpwGHlhmNtcAlJt+216eu9gYtbeP5aldGlvPcBtNEDSuRsLz9H2uU5fhTxNHtMJYa3MmXLSP5NFHBpRBxb4NyV4yvH5qm12G5euYCHIqJWJbvI9WrdU63zZr/L6u/xtarYasm7ZrU3cs5f5H6qrQy8WXBfM2sTt0D2KUlfA5ZGxBUkY8q2lrSzpDuAq4BdJD0hyQtO97CI+BeSLuYXgBeBGRHRqIu4GbcDe0taTdLqJF2vdwDPAGtLmqRk8fLsklCvkLR6Z+2b+X1nur2IpKUPYC9gXJ3js/HsK2mspMkkLXl3F7iHIUjG65F0m75cZ/9bgGmS1k6PmShp/QbXqOW/gf3S7Ww3+O3Aful9rAvslJY/AkyWtH167XHKSVU5AvXOu4jh76HeTPCRfPZZvwc2bCrqKpImAUsiIm98pJl1kFsg+1REXEY61ivt9tk2fevWmgdZT4qIe4F723VuSbMZrihcFBG/AZB0CnAX8D/Aw5nDZgPnS3qD4e7qlSXdRfLQun9adiHwc0l3k1TaKq1d9wNLJf0WmF01DvLa9Jy/JWnB+05EZMcW5jkJuETS/cDrwPQG9/w7SceTTBwZQzKZ4++AxQ2uk+dw4Aolk51+VnUfOwMPAI+SdAcTEW8rmVj0w7SbfSWSMaMPjeDayzU47/eBn0r6KvX/fcj97CuTXwqYTzJT/qJ6O0lah2Qc7RrAu0ombm2cVvp3Am4oeD0zayOnMjSztkpn9U5NxxnaAJP0S2D3iPi/ER5/DXBsRDzS2sjMrFnuwjYzs075e+BDIzlQ0njgOlcezbqDWyDNzMzMrClugTQzMzOzprgCaWZmZmZNcQXSzMzMzJriCqSZmZmZNcUVSDMzMzNriiuQZmZmZtaU/weVAlVj1B7T2AAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from utilities import plot_data_transformations\n", "plot_data_transformations(X_test, y_test, w1, w2, w3, w4, w5, w6, b1, b2, b3, \\\n", " language='English')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tensorflow Playground questions\n", "\n", "We will now investigate a few properties of neural networks using [tensorflow playground](https://playground.tensorflow.org/). Take a few minutes to familiarize yourself with the playground:\n", "\n", "- Change the number of hidden layers to one\n", "- Change the data distribution to *exclusive OR*\n", "- Push the *run* button and see how the network is trained\n", "- Stop training after epoch 500 (each epoch involves doing gradient descent using the complete dataset)\n", "- Hover over the neurons in the hidden layer and see the vizualization of their outputs. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Learning rate\n", "\n", "Open [this](https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=gauss®Dataset=reg-plane&learningRate=3®ularizationRate=0&noise=35&networkShape=1&seed=0.68448&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false) example on tensorflow playground. \n", "\n", "- Push the *run* button and see the learning process for 500 epochs. What do you observe?\n", "- Stop training and press the *restart* button. Change the learning rate from 3 to 0.1, and press the *run* button again. What is different from the previous run?\n", "- Try these steps using three learning rates: 0.3, 0.03, and 0.003:\n", " + Press the *reset* button\n", " + Change the learning rate\n", " + Press the *step* button (located at the right of *run* button) a few times, and observe how the training/test loss changes in each step. \n", " \n", "Which of those three rates would you use? " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Regularization\n", "\n", "Open [this](https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=gauss®Dataset=reg-plane&learningRate=0.03®ularizationRate=0&noise=50&networkShape=4,4&seed=0.64895&showTestData=false&discretize=false&percTrainData=10&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false) example on playground. \n", "\n", "Let's first observe a few things about this example. Check the box titled *Show test data*. Uncheck the box again. As you can see, the data is noisy and the number of training examples is small. This is a situation prone to overfitting. \n", "- Press the *run* button and let the training proceed for 500 epochs, then pause the training. \n", "- What do you think about the decision boundary of the classifier? \n", "- What causes the difference between the training error and test error? (Check the *Show test data* box again)\n", "- Write down the test error\n", "\n", "We will now see how we can avoid overfitting using $L_2$ regularization. \n", "- Press the *restart* button\n", "- Change *regularization* from *None* to *L2*\n", "- Change *Regularization rate* from 0 to 0.3\n", "- Press the *run* button and run the model for 500 epochs\n", "- What is different from the previous setting? \n", "- Write down the test error\n", "\n", "Just like learning rate, different regularization rates will affect the classifier performance. Try these steps with regularization rates 0.03 and 0.003:\n", "- Press the *restart* button\n", "- Change *Regularization rate*\n", "- Press the *run* button and run the model for 500 epochs\n", "- Write down the test error\n", "\n", "Which of these regularization rates would you use?" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 4 }