{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# MATH60629A\n",
"# Week \\#5 - Neural Networks - Exercices\n",
"\n",
"This tutorial explores neural networks."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2021-10-01 15:19:56-- https://raw.githubusercontent.com/lcharlin/80-629/master/week5-NeuralNetworks/utils.py\n",
"Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ...\n",
"Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 13166 (13K) [text/plain]\n",
"Saving to: ‘utilities.py’\n",
"\n",
"utilities.py 100%[===================>] 12.86K --.-KB/s in 0.002s \n",
"\n",
"2021-10-01 15:19:56 (7.38 MB/s) - ‘utilities.py’ saved [13166/13166]\n",
"\n"
]
}
],
"source": [
"import numpy as np\n",
"\n",
"# Code to obtain utils.py\n",
"!wget https://raw.githubusercontent.com/lcharlin/80-629/master/week5-NeuralNetworks/utils.py -O utilities.py"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## A tiny neural network classifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to classify the examples, we will use the following simple neural network:\n",
"\n",
"\n",
"\n",
"where $\\sigma$ is the sigmoid function defined as:\n",
"\n",
"$$\n",
" \\sigma(x) = \\frac{1}{1+ e^{-x}}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Question 1\n",
"\n",
"Assume that the parameters of the neural network are as follows: \n",
"\n",
"\\begin{aligned}\n",
"& w_1 = -5 & w_2 = 10 && w_3 = 5 \\\\\n",
"& w_4 = -10 & w_5 = 20 && w_6 = 20 \\\\\n",
"& b_1 = 25 & b_2 = 40 && b_3 = -30 \n",
"\\end{aligned}\n",
"\n",
"What would be the predicted label for the following data points:\n",
"\n",
" | x1 | x2 | o | label |\n",
" |-------|-------|-----|-------|\n",
" | 4 | -4 | | |\n",
" |-4 | 4 | | |\n",
" | -4 | -4 | | |\n",
" | 4 | 4 | | |"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use the following piece of code to evaluate the output of the network:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def sigmoid(x): \n",
" return 1 / (1 + np.exp(-x))\n",
"\n",
"def nn1(x1, x2, w1, w2, w3, w4, w5, w6, b1, b2, b3):\n",
" h1 = sigmoid(w1*x1 + w3*x2 + b1)\n",
" h2 = sigmoid(w2*x1 + w4*x2 + b2)\n",
" o = sigmoid(w5*h1 + w6*h2 + b3)\n",
" return o"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Finding good parameters for our network\n",
"\n",
"Let's move to a slightly more realistic example. Here we focus on the task of (binary) classification. As always, we first load the data that we want to classify:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"from utilities import load_data, plot_boundaries, plot_data # we wrote some helper functions\n",
"X_train, y_train, X_test, y_test = load_data() # to help with data loading"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can plot the data using the helper function `plot_data`:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAD4CAYAAACOqX/yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO2de3gUZZ7vv78QLgYwjhBBbt243OTqDImsg+NMxJ0dHHHBdZHZDPJ4zj7BeGaGgAwjZIKKk1EiCM5z1khWZx8uOQM8DDADDq6XZL2dlU1QLqIwByFBuWhghgiCYszv/FFdSXenqrur6q2qt7rfz/P003Slu+rtTvj27/1diZmhUCgUis5k+b0AhUKhkBUlkAqFQmGCEkiFQqEwQQmkQqFQmKAEUqFQKEzI9uOiffv25XA47MelFYqMYM+ePWeYOc/vdQQdXwQyHA6joaHBj0srFBkBETX5vYZ0QG2xFQqFwgQlkAqFQmGCEkiFQqEwwRcfpEKh8J49e/Zck52d/RyAsVDGkU4bgPdaW1v/ZeLEiZ/G/1AJpEKRIWRnZz/Xv3//6/Py8v6alZWlmjAAaGtro+bm5tGnT59+DsCd8T8X8i1CRFcR0RYiOkREHxDRTSLOK5qaAzUIrw4j69EshFeHUXOgxu8lKRReMjYvL+8zJY4dZGVlcV5eXgs0q7oToizIpwG8yMx3E1E3ADmCzuuYmgM1KHu1DE0tTSAQGNrfRlNLE4p3FAMAisYVoe5YHepP1mPR5EV+LlehcJMsJY6diXwmhsaiYwuSiK4EcAuA5wGAmS8z8zmn5xVBzYEaFO8oRlOLlhKmi6POxa8uouzVMszdMRfTN01HwYCC9p/VHatD5VuVnq5XoVDIhYgt9nUAmgH8OxG9S0TPEVHP+CcRUTERNRBRQ3Nzs4DLJqfs1TJc/Opiwuc0tTRh48GNIFD7sbpjdZi5ZWaMYCoUCmecOXOmyxNPPGGruue73/3usDNnznQRvaZkiBDIbADfAlDFzN8E8DmAh+KfxMzVzJzPzPl5ed5UQB1vOZ7S8+ZNmodt92zDzC0zsbRuKWZumYnNd29G4dBCl1cYTCrfqkTdsbqYY8riTkeevRoYMA7ImqjdP3u1k7OdPXu2y/PPP3+N0c9aW1sTvva111470rdv36/tXPfGG28cefjw4W52XitCID8G8DEz74483gJNMH1nSO6QpM+ZPmo6qhqqAAAl+SV47PXHUJJfosQxAQUDCjBzy8x2kVQWdzry7NXA/BBwqhvA0O7nh5yI5IMPPjjoo48+6j5q1KjRc+fOHbRz587ekyZNGjFt2rShI0eOHAMAt91229+MGTPm+mHDho1ZsWJFX/21AwcOHHfq1Knsw4cPd7vuuuvGzJo1KzRs2LAxkydPHn7hwgUyv6ozHAskM58G8BERjYwcmgLgfafnFUHFlArkdI2NF+lb6SzKwpLvLMG2e7Zh892bMX3TdPxm929Qfks5qhqqOllIig7LsXBoITbfvRkzt8zEvdvuxR2/u0NZ3GnHsoHAF3H68EWWdtweK1eu/Hjw4MFfHjp06P01a9Z8DAD79+/v+eSTT5748MMPDwJATU1N48GDBz/Yu3fv+2vWrOl3+vTpTtvq48eP9/jZz3726ZEjRw7m5uZ+vW7dum/YXVMyREWxfwqgJhLBPgrgPkHndUTRuCIAmi/yeMtxDMkdgoopFTjx2QkUDCiI+Q9NINwz5h4sK1yGwnCh2mYboFuO+ucyddhUrN+/HrPHz1afU9px2mRLanbcHuPHj/981KhRl/XHy5cv7/fCCy9cBQCnT5/uevDgwR79+/f/PPo1AwcO/PLb3/72JQD45je/ebGxsbF7/HmffvrpPlVVVf0A4Pjx491vv/324V27duXBgwd/+fLLL3+Y6vqECCQz7wWQL+JcoikaV9QulGbUn6zHtnu2tf8n1y2k+pP16j9+FNGW49RhU7Fh/wbMHj8bu47sarcs5acSQAGA6LXWAagHIGuKVw2AMgDHAQwBUAEg8d+0c/pf1rbVRsfFkZOT06b/e+fOnb1fe+213g0NDYd69+7dduONN468dOlSp11ut27d2tNRunTpwkbPmTdv3tl58+adBTQf5Pr164+NHDnS8tpVuRGARZMXdfrPXTi00FFOZJACGVYS6KMtxx+P/zHWzVjXLprBcEsUAJgJTRQRuZ8ZOZ4qNQDC0P77hCOP3aIGQDGAJmi+wKbIY7eLHJaeAHq0xR7r0aYdt0dubu7Xn3/+uanmnDt3rktubu7XvXv3bnv33Xd77Nu3r1M2jNdkvEC6VV0TlEBGdK4og9sT6M0+h7pjddj6wdZOlqNucctPIYDN0ERxaeR+M2ItykR4LVhlAOJT1S5GjrvJ/X8BVjUB114GCNr9qibtuD369+//9cSJEy8MHz58zNy5cwfF//wf//EfW1pbW2nEiBGjlyxZMmDChAmfG53HS8iPudj5+fksQ8NcXRyicyVzuuagelp10m15KuiiWJJfgqqGqnYRifd/ltWW4Zn6Z9DyRUu7n9TO9fWqoWh/a7LzhFeH2xPpownlhtBY2mj4fnQfZPzjYLEUwGMAygEss/C6MDRRjCcEoNHxqjqTBcDo/yhB67NgDBHtYeYYt9e+ffsaJ0yYcEbs+tKDffv29Z0wYUI4/nhGW5BGieR6dY0ICocWdkodircsy2rL8Os3fo1zX5xLyYIzw6olqGOWK2p0vP5kfYwYBstyjKYOQBU0caxCx3Y7Fcxya1PLubWOWapa8hQ2hXMyWiCtiIMd6o7VoaqhCtNHTUfFGxWgRwn3/eE+FE8sbk9Kf+LNJzq9zo5I2xV7s1xRo+Nu+Gq9R/c5boZmOerb7VRF0mvBqkDn1gY5keMKt8logbQiDlbRt5/FE4vx0ocvoY217VBTSxNWv70aNw+5GY+9/lj78XisirRdsTfKFc3pmoOKKen6H7AesT5H3SeZqhXstWAVAaiGtoWnyH013I9iK4A0FEgrQRc3xUHfjtbsrzG07P54+I8ov6UcWWT8K7Aq0nbFvmhcEaqnVSOUGwKBEMoNCfPBekslOluBdZHj0SxC54BMIVJP8fFDsIqg+TfbIvdB+90El7RqmBsfdIlvaRaPWSK5CHHQt51mFlwbt2FZ4TJ8zV/j12/8OuZndkS6YkqFYcAplfOkkisqP3r6jm4dRm+lRVMEJVKZQVoJpJkf7id/+gkG9BoQ4z+L7v/opjgMyR1iGiUGgIpbNQFzGsV2U+yDQXT6Tgm04IuV9B1ZCGIie/qSVltsM2vt3BfnfMtJTGUbX3FrBf76i7+i7eE2NJY22ha1onFFqJhSgSG5Q3C85TjKXi3zpWu6f0nyhdDE8bHIfdDEERCTyC4nmdruTBrM/G2h3FB7tYfX7cy89PHZTfURjX9J8k7Sd3RS9WW6hdNEdnE8W//s1QNWDhiX9WjWxAErB4x7tj6Y7c6ckFYCmchaM8pJ9IqicUVoLG10bCEmw+28zlSJrtn27gvJafqOjgwWnP+W8LP1z149/6X5oVMXTnVjME5dONVt/kvzQ05EMiPbnclEImtNz0lM53Zmbud1WsH7LyQn6TvRtdX3QSsd9NOCE2EJO2PZ68sGftEa2+7si9Yvspa9rtqdBRqjiGx8SVy6tjMzCwiJyOu0SvwXUmG40OXP2iiAUYjkwqbXVuuWdxOA1QC+j45SRK/FMVqUC+GHSJ++YNzWzOy4XdxqdyaKtLIgzUifErnEyJL0Hf2FtKxwmeTdfsyaQfwR/lhwThPZxdC/l3FbM7PjdjFrd3b48OH3r7/++kuptDtrbW2N2WK3trZi1KhRo0eNGjW6tLR0gJP1pZ0FaYRRKVzhULctGu+RJdUn0ReSfJ+5mfuhDZovU5QFl2pPR7uWsFiW3rL0xPyX5oeit9k9snu0Lb1F/nZn2dnZOHTokJCpBhlhQaaKk9Znqaa2uJ0CIzogZOczCVbNtpn7IRS5F2HB+dXT0T73F9z/l1XfX9V0ba9rLxMI1/a69vKq769qur9AtTtzHVnanUXjtPVZqq3AgtQyzO12cHIQ74MEtNpqkeWDYXjbIk21O7OKaneWBKcpMqmmtviTAmMPWdKG3OeKqH/3gfjaaq9bpClEkXECabbFNYr+AtZSZFJNbfEzJ9MKMqUNuYNuPZ6NOnbJheuono5BJeME0qzKo1/PfobPt5Iik2quZVByMt1sBycHXo0zUD0dg0rGCaTZFnfl3690lCKTampLkFJgZEkbcg+vtr6qp2NQyTiBBIy3uE5rplPNtfQzJ9NqBD19ekWakcrWV1QdturpGEQyMoptNExLVj+gSIIUQfcGowg2ACyBtv2Nr2oJDiqKbQ3Xo9hE1IWI3iWinaLO6QZB2uKKJkgRdA23508bbX2XRI7520knHXHS7gwAli1bds358+c93fWKvNg8AB8IPJ8rZErZoRlBiaB7l1wdv/WtgN+ddOTgl/2AHb1jj+3orR23R6J2Z6mwZs2afhcuXEiqWQMHDhxn9xrxCBFIIhoE4IcAnhNxPjcJVpWHeMwi6E6qiNzBqwhzPH520vG7F2U0ky4C917XIZI7emuPJ8X/UlImvt0ZAJSXl/cbO3bs9SNGjBg9f/78AQDw2WefZX3ve98bNnLkyNHDhw8f82//9m/f+NWvfnXNp59+2vW73/3uiEmTJo0Q8Q5TQVQt9mpoRaS9zZ5ARMXQTAAMGZIuaSLBwqyrUfHEYqx+e3XKs3y8wY/kar876Xg5VycZ084D645qojinGVibpz2edt7uGVeuXPnxHXfccYVeJ71169Yrjxw50mP//v0fMDNuu+22Ybt27er1ySefZPfv3/+r//zP/zwCaJZnnz59vq6qqur32muv/fnaa69N3F1XII4tSCK6A8CnzLwn0fOYuZqZ85k5Py/PthsicMhkmZm5F56pf0bCihk/kqv97qQjTzdxjWnnNXF8+lrt3r44GvHiiy9e+frrr185evTo0WPGjBn94Ycf9jh06FCPb33rW5feeOONK0tKSga++OKLvfr06ZO0k/js2bOH6B18Pv300676v3/xi1/0d7JGERbkZAB3EtHtAHoAuJKINjDzj+2esOZAje8daURgdcqi25h1NWr5osXw+f5WzFTAuEbazRxMGTrpRHcT97oXZTw7emuW47xT2v2U8yJFkplRWlp66uc//3mnyPo777zz/u9///vcsrKyga+88spnK1asOJXoXOvXr2//Yx04cOA4abr5MPNiZh7EzGEAswDUOhVHGeaqiCAotcxyVsxkanK1/93ENXSf47qjwOqTHdvt+MBN6sS3O5s6depn69ev79vS0pIFAMeOHet64sSJ7MbGxq69e/due+CBB/5SWlr6yd69e3MAoGfPnl/rz/UK6RLFgyIqqRCUWma9Yubn3wa+F9aOdVTM+BUkAMQkV7udKiQSUXN1RLA7J9bnqPskd8fXTKZMfLuzu+6667N/+qd/+ktBQcGoESNGjJ4xY8bfnDt3rsuePXuuuOGGG64fNWrU6OXLl1+7dOnSUwAwZ86cM1OnTh3uZZBGukTxrEezwOi8JgKh7eE2g1fIS3h12HQmdmNpo/cLSkDNgRrsPPwgfjP1E/xsVz/cMXIlisYNgP9+MCd40cpMJOJmYqtEcWsEpt2ZnNs9ewSplrloXBF+d/dp5PWsxe/u/hpF4w4j2OII+JcqZJdF6PxZF8KqOCrEIZ1ABklUkhHMWmb/R46KQ4Y+jDLlNiqsIp1ABlNUzBExAsHbVCFZggQikKEPowxzthV2kXJol9Ho1kzF21QhvxOlReNHqlA80bmNJdC+dJJ9nqkO+FK4jXQWpCIWb6P6fidKm2E3Em2WKgSb57OLFbdF8AZ8pTPSCWTlW5WYu2NuTHedumN1mLtjrrDJf0HC21QhGYMETgUjPlUIDs9nBytui6AFltIb6QSyYEABNh3chOmbpqPuWB3qjtVhxqYZ2HhwIwoGpL/fJt7fePUVVxs+L1FU3+3Rst4iWjC8FiCruY0yBJbcIdPbnQmhcGghtt2zDQTCD//PD3HH7+4Ag7H9nu0St+ZKnUQBF6MqovOXz6NrVteYcySL6pvN3QnmF4xowfBagKy6LWQILAG/rP1lvx2HY6tmdhze0fuXtfK3OxOJdAIJaCL5s0k/w6XWS7j41UXMmzTPE3F0O1qcrIzSyN94+evLuLL7lZai+sFrjJsI0YLhtQBZdVvIMeBr0sBJF+/dfu91ukjuOLyj973b771u0kDV7sx36o7V4Te7f4Mrsq8AEaHyrUpc1eMqLLhpQcxz6k/WC+vj6EW0OFHApWhckalf8S+X/oIzi6wVQEQ3xi2/pTyg4giIj0TLENlOhP635m8Ue9rIaefXTV939N7t9143Z8Kc5rX71uatm77u6LSRqt2Zr+g+RwbjhX9+ATt/tBPZWdlY+NJCPPVfT7U/R/SW0Wq02Kqfr/KtyqSzt0VWEQVltGxynDatiE/ULgJQCuAqm+fzAjkGfE0bOe38nAlzmp/e/fS1cybMaXYijkaIbHfmFtIJZP3Jetwz5p52n2Ph0ELs+NEO/HD4D7G0bqlrW0ar0WKrfr6CAQXIIuOPWxdAUVVE6Td3x4lgGCVqVwPYavN8mcOOwzt6r923Nm/epHmn1u5bmxfvk3SK3u7s0KFD7x86dOj948ePvzd//vwz48eP//Kdd955f9y4cZfKysoGLly48NpE56mtre2p93+sqanJFblG6QRy0eRFWDNtTYz4FQ4txI5/3oEFNy1wbZaKFetNtxKj/XwzNs3AXaPuMl1X4dBCPHTzQ52ORwugqCqiTJ+7E4tsTWiDge5zXDd93dHVP1h9Ut9uOxFJt9qd3XrrrZ/rIltUVGTc3NQmUvogjYjfMhaGC4WKZMWUihgfJGBuvenW4+a7N7f7+a7IvgKzxs5KfI1bK/B+8/vYfmg7AK2rj94MONqn6tTnadYYN7h+SKfI1IQ2GOw+sTsn2ueo+yR3n9idY3erHd3u7NZbb21Zs2bNxwcPHuxRUFAwCgBycnLaampqjh06dKj74sWLB2VlZSE7O5ufeeaZJqCj3dk111zz1e7du/8s7t2aI127MyO8mudspZN53bE6TN80Ha1trWBmdOvSDdvu2ZZwPfq6pw6big37N2DF91dgwU0LXHo/4lpnBZtKaHbA4+go9VsMoBXp/DmodmfWMGt3FggLMtGWUaRAWq0Bb21rxcWvLmpR4sgALDORixfBG/rfgIUvLcTe03ux68guF9JwZBoAZRcRNcnZABYCWAFgAbTgjP5YoUhMIARSxi3jxvc2IjsrO2bLn0i040V+wU0LsPf0Xqzfv96lNBw7TRJkIr7ZrV4SCFgTyVZoYvg4gHPQPocVkeMKRWICIZCyMXfHXGw6uKkj0h4uxIxNM3DPmHuwZtoaw9fEi3zdsTrsOrKrXWA/ufAJZo2dFSOUznM9g+x7S1QSaEUg9c/uHDo+hwXmT09v2tra2igrK8t7v5rEtLW1EbSUhk5IF8UOCvFjIYzGRJhhlIaz8eBGzNg0Q3B5oBe9Hd2a+SKyJDCdelw64r3m5ubciCAooIljc3NzLoD3jH4eiCCNjOgCVpJfgqqGKks+xMq3KlEwoKCTtbjxvY3YemirrXMarBDmPkhRlqSbM1/C0LbV8YTQ0ZUnFbz4HOTDKEizZ8+ea7Kzs58DMBbKONJpA/Bea2vrv0ycOPHTTj9lZs9vEydO5HSgvLac8Qi4vLZcwnMuZ+bauGO1keOiCLHxrzgk4NwbmDkn7rw5keNW8OJzkA8ADezD/+10uymBtEnt0VruW9mXy2vLuW9lX649Gv+fUI5zugux8a+YBJ1/A2tiS5F7q+KYuSiBVALpG7qQ6QIW/1iOc3ohLiE2/hWHXLiWwgpKIMXclB/CBm6U8ok9p+i2/WaT+W6CDK25kqMmCyrs4ThIQ0SDAawD0B+aw7OamZ9O9Jp0CNLITRhiAhw6iQIdJ+F3a67kZF6gxihIo7COiDzIVgAPMvM7RNQbwB4iepmZ3xdwboUtRHfNTpZ0LpsgxhP0pHmFXzjeYjPzKWZ+J/Lv8wA+ADDQ6XmDjLdzrI1wo2u2lcl8MhL09Sv8QKgPkojCAL4JYLfBz4qJqIGIGpqbm0VeViqSjVXwBjfa9gc92Tro61f4gTCBJKJeAH4PoJSZP4v/OTNXM3M+M+fn5dkebCY99udYiwwkOO3CHY/VyXyyEfT1K/xCiEASUVdo4ljDzFtFnDOo2J9jbdT5embkuB1Etu23OplPNoK+foVfOBZIIiIAzwP4gJmfcr6kYGN/rozMna+tTubzklRqwWVev0JmRFiQkwHMBnArEe2N3G4XcF6hWB2yZRdnc2VUIMEaovM9vcatRh8KUYiIYr/JWrnGeGa+IXL7k4jFicTqkC27OJsrk6mBBLv+10Qt0WQn6OKeIfhRvuNXqaHctc61zNyXOxorxD9OZ+y+d7drwd0kxG6WaUKVGqpSQ6sUDi1sH7LlxmREZ0QHEnTLKTqQkMyiCnI5nV3/qxv5nl4hOplf4QYZJZB1x+rw1H89hdnjZ6OqoSpmuy3aF2md6ECCHtHWj6cS0RYZBfdDbO34X93I9/SKIIt7BuGH2erHFlvfXq/8vysN7+XabjN3bDPLOfWttp3XJDqPl9t9u2sPaks0Uf0ujYHaYgu5pbVAbti/gUOrQkyPEF/1xFW85NUlzNwhlrO3zuacihzH4rj8zeWdzlF7tJaXv+m0KWs5ax+Zlea5dl5jhCixtXKtzPK/bthfwqFVXZgeAYdWdeEN+0uEnVsJpJhb2m6xaw7U4L7t97WX/J374hxWv70aZbVlqD9Zj5L8Eqzfvx4P3vSgZV9kfK11y5ctSSLkdrasdiLaIqPgXqYcZV4it1aSuhZNLV9rMeyWr1G8Y217Sar/9fwKII1n0oRXh9HU0rnlVxZl4cm/exKPv/m4rdkveq11dDlhTtcclP5tKar3VJuc06zd1l0AZiFWfOoAbASw1eD5iQIXdlt6VULzU5qtQXW/cQOzv89QbggVUyoM/8ZSTxdT7c5EkbYCmfVolumkwb45fdsFLHrCYCoimegP+94J9+Kx1x9D+S3lWFa4LO4ZumBFCw5gTTjrYV79YSZ0iV4Tva7oNUyHVsO9DdbEVpEqZn+fBMKQ3CGGf2N9ruiDM4vOpHR+JZBiSNsttllp31U9rnLUudusprqppQlVDVXtc67jq3aMt6xm6S1rYK00zq446ueNX8MsdIhj9HPc3vJadUXIVIlibe2JSlLN/sbOXjqrttoek7YCaVTyBwAPFDzQyVIsHFqIRZNTq8s1+8POoqyYOdfRPkkNM/+gCF+f0xSf+DVYFWhRWHkfslWiWPsdJCpJTVS3n7wrlEIkaSuQReOKUPq3pcgi7S2GckNY8p0lqN5TbWDdpY7RH3bXrK546OaHElilidptiQisOG10IUuJo5X3IVuZobXfQaKS1ER1+8m7QimE4kfo3Ks0H7fSb6LTh0KrQrxhf7LcNbPZzMUsNr3FTopPKik2XucapvI+ZC0zFJNm1Wd5H8Yj6HQLrQql9HqoNB8ht7QWSPkROdTebt5isjW4kdBsJLj6OqLfRy5rXyJGhNj4zyvkYF1OEZc7umH/Bs6pyIkRx5yKnBS+jDWUQCqBVLTjZqJ1iMUKkZngLmHm7szckzuE8srIYyORNDoPmLkP+1NNI/53YH2n0oESSDG3tE3zySycRLGTkQUYpksRtG7lVgnDfCTtWAAvQGsvugvAYgCPQouqrzF4TQ2AeQDOxh3PgbMRE3Zw83dgHZXmIwYlkIokhJF4xrZVETATXADYAOA/AKwH8B1oAzKTBZuSrS8zUQIphrSNYitEYdQxp3vkuJ2OQYm61fxPANuhieMbAKYieSTeLKrbhM7R+DIA34AceZOKIKAEUpGE+AmJ/QB0A3AY9qprjARX50sA2dAsx9nQLMpkY47MBLcfYvMSywD8GsA5JM6bDHJfTYVolEAqUiB6QuJpAKWwn9iuC64Zf4UmuusArICWU5goL9OsJ+RKxOYlPmHwWqO8SdHTJRVBJi0FUnVCcUKy8j0RSeVF0CxSI/qhQ3QXANiB2BLH+PUB5jPAoyuEzAJK8Vt0madLKjzHj9C5m2k+TvPHMptkOY8iU1nspOlYzcmMzkvMMrgW2DxdyX7Ct5P0HFFApfkIuaWdBVn2allMmygAuPjVRVXDmhLJyvdE9m3Ut9p94o6fhXlNtZXywvjyzocMnmM2nsG+lay3w9P7kDa1NKF4R7HaxQSUtEvzSdRGqu1hO3l7mYTonMdUCCP1NB0r6zPKSywD8AyAFmjBnQp0zpW021dTI1E7vMbS+PfjHirNRwxpZ0EmaiMFAJVvVXZqViHH0C4Z8GOQlJXpflbWFz0ETacCWhCoDZr4GiWSO7OSzZpJqCYTwUSIQBLRD4joMBEdISKjvYxnJGojBQAFAwqSjEfIZPyYEmhF9LxYn5GwJm/1pgcGjXYvgPkXt0JuHAskEXUB8K/QsnpHA/gREY12el67JGojBXS0Ipu5ZSaW1i211E08/YnPeYyOCLuFFdHzY33JifY7GhH9Ba0IFo59kER0E4BHmPnvI48XAwAzP272GhlKDZfWLU0wHkHhLTXQ/IPHYe4blBczvyPQMWMm1VkyolA+SDGI2GIPBPBR1OOPI8diIKJiImogoobm5uakJ3XTV1h3rC7JeASFt0QnojciSOIImPsXCYTG0kYUjStSubkBRYRAksGxTmYpM1czcz4z5+fl5SU9qVu+wughXebjEYKOKpdzAzORSxYYVKk/wUWEQH4MYHDU40EATjo9qVu+wvqT9Y6GdgUDVS4nGjORe+CFB3Dh8oVOz4/2O6rc3OAiwgeZDeDPAKYAOAEtH+Kfmfmg2Wus+CCVr9AuRmNmVSDKLmZ+RgJ1ilz3uaIPnp76dLvf0Y/cXOWDFINjC5KZWwH8BFojvw8AbE4kjlbQfYXTR01HxRsVoEdJ+W9SRsS0xKDhnmvBzM9oJHy9uvWKCcok24Ir5EVIHiQz/4mZRzDz3zCzkHwG3VdYPLEYL334EtpY+w4vQe0AABVOSURBVKZV/hsgNSGQZVKhE6zOvXbPtWBFzOLFNFlurkJepK2k0X2FNftrlP+mE8mEINGYWZlIJIB25l6714nHSOTIMD7ZWUyT5eYq5EX6Wuzg1VZ7ldOXyMcoYj6KyPdhVhf9JICvoo5Fz5IJw/4ohaXQXAvl0L4gxFBzoAZlr5bheMtxDMkdgtuH3461+9bGfIHndM2RQvyUD1IQfrQQstLuLLQq5Gg+sLe4MSI1EWJmMHdG9PswapOWrP2Y3bnX4kavpoIMrc2MgGp3JuQmvQWpp1fI+C3dmTC8GyDlZpQ6DPHvI369Z0yep3fmsbMGZ5140gllQYpBWh+kTrD8N1Y60zjBbR+jG+8jPqpu1lFc99/ZaUwhsl+lQgH5t9jusoG1LR1F7p1uj0KceNsoiuXceetYGzkughCLfx/xW98lnHwbL/r3IxfL31zOtUdjf4+1R2t5+ZvOf49QW2whN+ktSPewEyVNhlftwuy15Eod0e/DyOKthjb8K74zD9AR2S6LXDOYNdrJUK33AoAfqiyHBRlid6y9dLF6RL6PVC1er4Nc/lN7tJb7Vvbl8tpy7lvZt5NFaRcoC1LITfogjXv4MV5AkZgwvAtyyYMb5bQqSCOGDN5i+zFeQJEYr4Jc8pBK6z01JsQ/Mlgg/RgvEDSMShrLAHwDqZf/WSGzvrRSbb2nfJU+4se+Xg4fJHP6+AvdIj7Bewl3/nWK9BHa9UEG8/doJYpt1VcJ5YMUcstwgVQkJzo9J1n1iwisil3mBHbKa8sZj4DLa5NXTimBFHPL4C22IjWiE7zNglcifYRWxy+UAbgYd+xi5Hj6oMaE+IMSSEUSotummf25+OkjTP/ATmaMCZETJZC+YbXXoR/oCd53QbMkjUae+x3YSv/ATmaMCZGTDM6D9BO9iid6axjd6ksGKgF8CGBW5PFMAIsB7ATw3wA+B9APwEr4u2ZvPsv4Vmd+jHK1gsqDFIOyIH3Bit/MrwmFBQC2Rv5dCE0cF0KbyXYFgFoAp+G/oBdBE8P4kkWx4qimEmYmSiB9wYrfzK8JhfHduR8H8GMA6yHfjBt352qrqYSZixJIX7DiN3NvjEByoiPYUwHsgnszbuSd5W02sMvsuCJ9UALpC1arePyaUKhHsGcD2ABtm+3WjBt5Z3mbDey6puc1MY9V+V/6oQTSF6z6zRJNKHQrGh7domwsgBXQttl1cKcRbXJL2a+aZKOBXd27dMfFry6q8r90x4/sdFVJYwWjeS76YzerSNxuymuG+ZwdvdxOL7OLf+wmRrNn3GpVJgKoShohNyWQ0pNIqEJs/BGHvFueUJIP3JJNlKyU/3mJEkgxN0dbbCJ6kogOEdF+ItpGRFeJsWvTAVFBh0Tdw9OpiiS1OTuFQwtRkl+Cx15/DCX5Je3J02bUHKhBeHUYWY9mIbw6LDQ1R5X/pT9OfZAvAxjLzOMB/BmaF18BIDboUAngKcQGHUREaNOpiiS1gVtWRMnN/EVV/pcZOBJIZn6JmVsjD9+GlkWsABAbdHgPWpL1YsSOI7Xr0NcDM03QgjzR+F36Z5fkc3asipKb+Yuq/C8zEFZqSEQ7AGxi5g0mPy+GVhOGIUOGTGxqMmqt75QaaNUox6FZURXwv9JjKbT0nNnQ8gidzrE2Kq0jaOMjQpDjPbtD5VuVKBhQELOtrjtWh/qT9Vg0ufPAsqxHs8AGYzUIhLaH03ushio1FENSgSSiVwD0N/hRGTP/IfKcMgD5AO7iFBTXnVpsGeubdUtRF8Wp0CpRyqH52ewQRibObbFDeHUYTS2dP6tQbgiNpY3eL8hDlECKIekWm5lvY+axBjddHOcAuANAUSri6B6y9QWMDzoshpZsPRvOKlHSKTDjLkb5izldc1AxJVguCDcDTYrEOI1i/wDALwDcyczx6uQxsglHdNChDlqS9QpoSddOKlHSKTDjLkXjilA9rRqh3BAIhFBuCNXTqqXuwhOPapThL458kER0BEB3AGcjh95m5vuTvc6dLXYY8m49K6EFZKJ9jnXQRLSz7ywxmeuDzETsugnUFlsMTqPYw5h5MDPfELklFUf3MKpv7o7YiK5fzQ+SR2hTJ7pMEegQR0D7giiGt8135W0y4Qeit8OqUYa/pFEtdnx9cz8A3QAMiPxcnuYHztHbe4WATlFar/2u8jaZSAWR9d1ubIfNGmWYHVeIJY0EEojtC3gawB/gT5swr5DB7+pnOzbniJw57UbeZboEmoJKmglkPNFtwm4GcB/Ed73xc4spS8DGr3ZsztETvGdumYmldUvbE9GTlTAa4cZ2OB0CTUEmzQVSbxM2HcB2aD46hlhfndEWcxqAbIO1iBZNq30l3SJROzb5sVrfbYZb2+GicUVoLG1E28NtaCxtVOLoIWkskNF5iO8a/FyUr85oi7kMHb0To9ci2i9npa+kF30j3Wqm6y6imk6o7XAa4kcLIW/anUW3CSOTpZDA68X3MUzeuss70rFvpBhE95g06hvpB1DtzoTcMmTsaxju5kjGlxTqQQq9DttJaaEIwpA3R9RfrNZ3BwWVBymGDBFIN+u0o7eY0Z16FkPbZjttTqHjJNk8C53TgQBtW57eTRsyFSWQYkhjH2Q0bs5ONupjuBia1SjSL+ck39BptFt8pF6W+mJZ1iH7mjKV+FBrGlMEd0rwjKy3VgA7Ydz81a4VGR0MsmqVVsDYgk41eKCLc7yVvDnF18eiJ1TrOYN6QjUATyO0sqxD9jVlMgHfYsvY/9Ft7Po1nX5WZn5W68jShkyWdUQjak1qiy2GAG6x9XQVgtY6zI3cRllxkm8YXWXUCOtfJGbJ4Na337LUF8uyjlSurWqv/SFgAqkHW/RvWL/rkL3E73xDM3G27huVpb5YlnWkcm1Ve+0PARNIo6a48bjxTetWkrUVUhtq5Q6JxNl6LbZRQnX3Lt1jEqrtNoywgoyJ3TKuKZMJmECmIn6iv2mjrVY/t/IiW6ZZJZk4W6vFjq8v7tezH7p16YYBvbTOS04aRlhBxjpnGdeUyQQsSBOGccKzjhszaMyuqZKsO3AewNFFsSS/BFUNVbYbRig0VJBGDAGzII2aM+hjT0XmNkYjQ0sxO3jVZUiMb1RUwwiFQiQBE0ijhO/10La+jXAnxUeWlmJW8aqRrRjfqKiGEQqFUPwoAPemWYUo3Gz04DYyNcwwR3TDCIVqViHqFjAL0g/cLFN0m2A0sq0/WR/jc9Sb2Naf9CJCnxxV+pe5BCxI4wcyV+skW5u46pdMJb70D9DSbmSPLKsgjRiUBZkQWVJ8jEi2NqvBExlyPeXDjTkziuCgBDIhRonpslTrJFubleCJzF8E/qJK/zIbJZAJ8SvFJ5UUnWRrs5JYbu+LQOTIVFlRpX+ZjRCBJKKFRMRE1FfE+eTBrxSfVFJ0RK7N3heByJGpsuJH6Z8KCkmE0zA4gMEA/gPavqxvKq8JTpqPnyk+yVJ0RK4txMa/qlDyVUZScspry9M2NcfLOTMb9m/gnIocxiNov+VU5Fi+JlSaj5CbCIHcAmACtExtjwTSy0FRG1gTCorce5n/GD8ILB5Ra7MmtvGCMX3jdMYj4PJas3UqUiW0KhQjjvottCpk6TxKIMXcHG2xiehOACeYeV8Kzy0mogYiamhubk7h7Imiql5ViQDO+ygC9sr+Uun9KGJt+nlSy/XU016aWprAYDS1NGH7oe2YPmq6qoARgAoKyUVSgSSiV4joPYPbP0Dz4i9N5ULMXM3M+cycn5eXl+TZyaKqhQDuAjADsS22APG1xiKwKuh+9H4sgpZHOQSa77EMRlFso7QXAHj31LvYfPfmGJ+k34gKInnpE1RBIblIKpDMfBszj42/ATgKYCiAfUTUCGAQgHeIqL/zZaUSVZ0F4DI6qkQA96xIp1jtmehH78fUUn0SWTiyVcCICCIZWczFO4pdE0nVD1IuhFXSREQyn5nPJHtu8kqaVMaU1kGzIC9HjmcD2A65K0VkmZNtRBiptHWTcY5LIpy2UbP7fp3M2645UIOyV8twvOU4huQOQcWUCstVO6qSRgyS5kEmS2HRt6DbACyEZl22erAuJziZJ+MFqaX6BM3CcdpGza5P0In1WjSuCI2ljWh7uA2NpY1SlzSmO8IEkpnDqViPqWHU9zF6TKm+BQU6RCcbwEYxlxeO3/NkUiG1vMqgdbx22kbNrk9QdzfM3DITS+uWYuaWmaoJcBDxI3SeWppPshQWPU+w1uSxTHiZlmSXILd1M0ZEGzWneYnlteW+pEBBpfkIuUkskMkIgugEDT9zPsWz/M3lncSw9mgtL3/T2t+I3URxP5PolUBmvEAq0hlR4uYXfjcBVgIp5iZpkEaRCSTKUwx6nbfsTYAVKeKHKisLUsGc3MrKhDpvt4CyIJUFqQg2ySK9atKhwm+UQCp8JZEIqkmHCr9RAikFXs2wlg8zEdR9jpvv3oxlhcukq/NWZAZKINu5HcBTcceeihx3Gy+7E8lDIhFMFOTIhE7mCknww/EpZ5BmJWv5fytNHovEKIdzJTP3ZNlnWIvEbiqP3yk0QQAqSCPkpgQyBl0Uv8PuiSOzeRXQbNY+ItV4Nhkqwp0YJZBibmqLHcMCADcDeCNyv8Cl6xi1P1sMYBfkbWYhFyrCrfACJZAxPAXgTQDfidzH+yRFUgitj+VjAKYCeBxyN7OQCxXhVniBEsh2noLWOm0FgNcj9wvhnkhGtz/7PTQL0ssGucElnSPcKgAlFwEWSNGpMa9AE0V9W70AwI+gdTE3movjhPj2ZzuhWZDR78dshrUincv4gl5imXb44fgUE6Rxu92ZUfsvityH2FmnG9WJSGGOiAAUVJBGyC3bb4G2T3SgowTadjXRnBerGM3F0cdA6PNaAHuTBI0sw0LIPS5C4RXRAajyW8pVAMpHArzFBmIDHTcDuA/itsPJxmzGDxHzDy+n7jklSGv1CxWAkocAW5BAR6BjOrSBXTpOLTxAGzVgNMQqGv9nFetT9/RRrPrUPQDSjUEI0lr9IjoAVTi0EIXhQjWuwUcCbEFGBzreNfi5UwvPaC5OPP7PKjaaU33xq4soe1WcdSvK6vNirX4i4nNK5wBUEBE29tUKyce+pkIltFrlQqQ2JtYONdBEtilyruhr5ACohn0LVQxZj2aBDd47gdD2sJP3rhFv9QHaFEM7g7rcXqsdRIxY1c8j6nMSgRr7KoYAW5CL0BHUSG0in3WKoM2EZgDroc2Ipsi9/+II2J+6lyoirT6312oVXdSaWprA4PYtvx3LL92t40wlwAIZTbIxsSLQxbItcu+/OALmc6pvH367kG2x3bnQ8dQcqMGFyxc6HfdzprZIURP1OSnkIk0EsgiaRWfFwkuPHoxGc6rnTJiDtfvWCrGMRFh9uqV29tLZmON9rujj60ztZKJmxacom3WsEINjgSSinxLRYSI6SEQ+qotVCy99ejAWjStCY2kj2h5uQ2NpI/70//4kzDIys1CtWH1GlhoA9OrWy9fodSJRs7r9TuVzUmWEwcORQBJRIYB/ADCemcdAq9ULCEYddUQmmvuHyO2ekYVq1eqTdfuZSNSsbr9T+ZxUGWHwcBTFJqLNAKqZ+RUrrxMTxRbFUmiJ5uXQ6qKDT3h1GE0tnXM4Q7khNJY2Zvx6ojGLYrsVcddFsSS/BFUNVa7lN6oothicbrFHAPgOEe0moteIyPSrkIiKiaiBiBqam5sdXlYU0R110qcHo4htcTqvJ5p494Ru8bnlU1R9LINFUoEkoleI6D2D2z9Aq8T5BoC/BfBzAJuJiIzOw8zVzJzPzPl5eXlC34Q94jvqpE8PRhHb4nReTyq4JeqqjDBgOOl0AeBFAN+LevwhgLxkr5Nj5ILqqKNIzIb9Gzi0KsT0CHFoVYg37HfSwcnbWTpQ3XyE3Jz6IO8HMICZlxLRCACvAhjCSU4qlw9SofCGyrcqUTCgoNPs7/qT9Vg0WWzvT+WDFINTgewG4LcAbgBwGcBCZq5N9jolkAqFuyiBFIOjbj7MfBnAjwWtRaFQKKQiTSppFAqFQjxKIBUKhcIEJZAKhUJhghJIhUKhMMGXhrlE1Izk8wz6AjjjwXLUGtQa0nENIWaWoSIj0PgikKlARA1+pymoNag1qDVkNmqLrVAoFCYogVQoFAoTZBbIar8XALUGHbUGDbWGDENaH6RCoVD4jcwWpEKhUPiKEkiFQqEwQXqBlGUoGBEtJCImor4+XPtJIjpERPuJaBsRXeXhtX8Q+fyPENFDXl036vqDiaiOiD6I/A3M83oNkXV0IaJ3iWinH9ePrOEqItoS+Vv4gIhu8mstmYLUAinLUDAiGgzg7wD4NWXqZQBjmXk8gD8DWOzFRYmoC4B/BTAVwGgAPyKi0V5cO4pWAA8y8/XQOtf/Lx/WAADzAHzgw3WjeRrAi8w8CsAECdaT9kgtkABKADzBzF8CADN/6tM6VgFYBBhMcfIAZn6JmVsjD98GMMijS98I4AgzH420ttsI7QvLM5j5FDO/E/n3eWiiMNDLNRDRIAA/BPCcl9eNW8OVAG4B8DygtRpk5nN+rSdTkF0gUx4K5hZEdCeAE8y8z+trm/A/AOzy6FoDAXwU9fhjeCxO0RBRGMA3Aez2+NKroX1B2h9n6JzrADQD+PfIVv85Iurp43oyAkcNc0VARK8A6G/wozLEDgUrgDYU7LpkIx0Er2EJgO+LvJ7VNTDzHyLPKYO25TSeXu/CsgyO+WJFE1EvAL8HUMrMn3l43TsAfMrMe4joe15d14BsAN8C8FNm3k1ETwN4CNpIToVL+C6QzHyb2c+IqATA1ogg/jcRtUEr1hc6N9ZsDUQ0DsBQAPsiwxoHAXiHiG5k5tNerCFqLXMA3AFgiugviAR8DGBw1ONBAE56dO12iKgrNHGsYeatHl9+MoA7ieh2AD0AXElEG5jZ6076HwP4mJl163kLNIFUuIjsW+ztAG4FgMhQsG7wsJsKMx9g5muYOczMYWh/pN8SLY7JIKIfAPgFgDuZ+aKHl64HMJyIhkbmD80C8EcPr4/IGOHnAXzAzE95eW0AYObFzDwo8vufBaDWB3FE5G/uIyIaGTk0BcD7Xq8j0/DdgkzCbwH8lojegzYUbI6H1pNM/G8A3QG8HLFk32bm+92+KDO3EtFPAPwHgC4AfsvMB92+bhyTAcwGcICI9kaOLWHmP3m8Dhn4KYCayJfVUQD3+byetEeVGioUCoUJsm+xFQqFwjeUQCoUCoUJSiAVCoXCBCWQCoVCYYISSIVCoTBBCaRCoVCYoARSoVAoTPj/A/jxh5OQaGwAAAAASUVORK5CYII=\n",
"text/plain": [
"