{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Machine Learning and Statistics for Physicists" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Material for a [UC Irvine](https://uci.edu/) course offered by the [Department of Physics and Astronomy](https://www.physics.uci.edu/).\n", "\n", "Content is maintained on [github](github.com/dkirkby/MachineLearningStatistics) and distributed under a [BSD3 license](https://opensource.org/licenses/BSD-3-Clause).\n", "\n", "##### ► [View table of contents](Contents.ipynb)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns; sns.set()\n", "import numpy as np\n", "import pandas as pd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These are the PyTorch imports you will normally need:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import torch.nn\n", "import torch.optim\n", "import torch.utils.data" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import mls.torch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Best Practices for Neural Networks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the [previous notebook](https://nbviewer.jupyter.org/github/dkirkby/MachineLearningStatistics/blob/master/notebooks/NeuralNetworks.ipynb) we focused on the static building blocks used to create a neural network model. We now turn to the \"dynamics\" of learning to explain our data with the model.\n", "\n", "The techniques described below are the current \"best practices\", but keep in mind that most do not have a solid theoretical basis and are justified by empirical studies on benchmark problems that might be quite different from your problem.\n", "\n", "A good source for a deeper dive into these topics is [Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization](https://www.coursera.org/learn/deep-neural-network/home/welcome) on Coursera, which you can audit for free." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Test Problem" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We start by setting up a test problem that we will use throughout this notebook.\n", "\n", "Our problem is to learn the function $y(x) = \\sin(x)$ with samples of $x$ drawn from a uniform distribution on [-3,+3]. Create 1000 TRAIN samples and 100 TEST samples:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1xUdf4/8NfcQLl8UWkGaAzRKKiERaVV2b6ouyoiqMW6pbTR1bTcbH18pUh9eGnXr6vhJTe1XddqM/ObWyrrpuRPd3W3sLyUCuWiaGYOyHBbUkOZy/n9gUwMIDKHGc6Zc17Px6NHnNvweYPMez53jSAIAoiISLW0UheAiIikxURARKRyTARERCrHREBEpHJMBEREKsdEQESkckwEREQqp5e6AGLV1V2B0+nZFIjw8BDU1Fz2UYm6F2ORH6XEATAWuRIbi1arQe/ewTe87reJwOkUPE4Ezc8pBWORH6XEATAWufJFLGwaIiJSOSYCIiKV89umodYEQUBdXRUaG68CaL/qZLVq4XQ6u7dgPnLjWDQICOiB3r2N0Gg03V4uIvI/XU4Ely9fxpQpU/D666+jb9++btdOnjyJefPm4cqVK0hOTsbixYuh1+tRXl6O3Nxc1NTUoH///sjPz0dw8I07MjpXjnpoNBpERPSFRtN+RUev18JuV0YiuFEsguDEf/5TjcuX6xEa2kuCkhGRv+lS09Dx48cxdepUnDt3rt3rubm5WLBgAT766CMIgoCtW7cCABYvXozs7GwUFhZi4MCBWLduXVeKAQBoaLiM0NBeN0wCaqHRaBEa2hsNDcoYJUFEvteld82tW7di4cKFMJlMba5ZLBZcvXoVSUlJAICsrCwUFhbCZrPh8OHDSEtLczvfVU6nAzqdYlq6ukSn08PpdEhdDKI2yiz1+PDgOew/ZsGHB8+hzFLvdr75+EbnyDe69M65ZMmSG16zWq0wGo2uY6PRiMrKStTV1SEkJAR6vd7tvDewTbwJfw4kJ2WWepSer0NwTwO27D0N2/UmTQ2amjinjr4DW/aeht3hhF6nRe7UQQCAV7Z84To3dfQduNJgQ1x0b8SawySMRpl89hHa6XS6vSEJggCNRuP6f0ti3rjCw0Pcjq1WLfT6m1dwOnOPN7zyylKcOHEcNpsNFy58i/79BwAAHnpoKtasWYWIiEi3+198cR4GDkzA3/++F3/+8xtwOBwQBAHp6Rn45S8fxaefFmHt2jUAgAsXvkV4eDh69gzCrbeasWzZijbfX6vVwmgM9X2gXuJPZe2IUuIAvBPLv8/VIv//voDd3vR+4GyxD5YAwOFwovhsLRwOJwSh6fhCzffA9WuCANjtTmzecwqCIECv12LJjJ8AAIrPVCPh9lsQH9OnW2KRC1/E4rNEEBkZiaqqKtdxdXU1TCYT+vTpg0uXLsHhcECn06GqqqrdpqWbqam57Daxwul03rQjuDs7i2fPfhEAUFFRjueem44333wXALBr10785CepmDdvUZtnKiou4tVXV+KNN95BWFgvfP/99/jVr55G377RuO++EXjzzWEAgF/96mlMmzYDP/rRYABoNyan04mqqks+is67jMZQvylrR5QSB+C9WD49YYHN3vSGrhEEaLVNyUAAoNEAOp0WCQP6oORsDeBwQqfTom94EICma3A4AY0GTmfTM3a7Ex/+6ww+KbkIu8MJnVaD+xKikJIQdcOaAn8vTTOLW394bslnicBsNiMwMBBHjx7FkCFDUFBQgNTUVBgMBiQnJ2PXrl2YMGECduzYgdTUVF8V46aaq61yqHL+5z//gd1ux9WrVxEWBgQFBWH+/EUICAiUtFxEndX67ykuujf0Oi0c19/km5t4gnsa3Jp6+hpD2vwd5k4d5Nak1PwaAGBvri04BOw/Vo4Dx8uRFHsL0of1k/zv2B95PRFMmzYNs2bNQkJCAvLz8zF//nxcvnwZ99xzD3JycgAACxcuRF5eHtavX4+oqCisXLnS28XolDJLvVs7ZO7UQd3yj+jjj/+Jxx7Ldh0bDAZs2PBn3HHHnfjv/x6BBx+chDvvjMOgQckYM2Yc+va9zedlIuqqG/09Nb+hd/RhK9Yc1uZay3MtEwUAfFJy0dXXAACCAHxxuhrHy6rxy7Q4jEwy+yhKZfJKIvj73//u+nrDhg2ur+Pj4/H++++3ud9sNmPTpk3e+NZdUnq+zvXJwuFwovR8Xbckgvvua79pCADmzHkJjz76JA4d+hSHDh3E9OmPY+HC32DEiJ/6vFxEXXGjv6f23uQ91fo1cqcOQlFxBQ4cL0eLbgc4BeCdPafQ1xjCmoEHVD3ovrnaqr3eVtn8aUMqRUUfY9++PTAaTcjImIjFi5fi17+eg7/9rUDSchG1p/Xwzu78e4o1hyFnXDweSYuDttVYE6dTQMHHZzns1AOqHnjf2Wprd+nRowdWrXoFd989EFFRt0IQBJw+fQp33BEnabmIWiqz1KOouAIfF1fA4RQ8bgbyppFJZvQ1hmD3p9/gWFm1q3bw5dd1KD3/ORIGhCPylhAMig2X/O9bzlSdCID22yZ9rXUfAQA89FA20tMz8cQT0/DCC7+G3W4HAAwdOhyPPfZUt5aP6Eb2H7Ng855TcLQYseftZiBPxZrD8NzPE1FmqUfBx2fx5dd1AJo6kr84XQ2crsb/O/QNXsgezGRwAxpBEPxyoe7Ww0cvXvwGkZH9OnxGDWsNNevMz0MulDK8TylxAO3HUmapx+82f95mPXyDvvsGWtxMc4e1rZ2/jZ+PGICM4THdXygv8rvho0SkLKXn69ySgFYDpP7o1g7H8He35uapouIK/PNEOZoX6NXrNJL3AcoZEwERdUpcdG8YrtdENVoNfjn2TlkO02xunkpJiEJRcQV69gxgH8FNMBEQUafIbXDFzTQnBCU12fmKohJBe+sYqZGfdvuQH5CiM5h8TzHzCPT6AFy58p3q3wQFQcCVK99Brw+Quijkx/59rpZLQKuIYmoEvXsbUVdXhcuX/3PDe7Ra5WxV2VEsen0Aevc2tnuN6GZaDhGV04gg8h3FJAKdTo9bbonq8B4ltRUqKRaSjzJLvds8Abu9+5Zekcr+YxYcLbViSJxJlp3f3UExiYCIuqZ5QlbLyWIarbKHXe4/ZsHbhaUA4JqIpsZkoJg+AiISr3ki1lfX3wwBQHd9iKiSawNHS60dHqsFEwER/bByKJo2jBl0pxEvPjxY8Z+Oh8S5b4oVbQpVZSc5m4aIVKzlfsItN5DJTotHeLBB6uL5XHOiO1pqRbQpFHuPXuj2/UnkgImASKXKLPVY/u7nsDsE6HUaZI+507VrWHxMH9UMRhiZZMbIJDM+PHhOkv1J5KBLiWDnzp1Yv3497HY7Hn30UTz88MOuaydPnkReXp7ruLa2FmFhYfjb3/6G7du3Y8WKFQgPDwcAjBw5ErNnz+5KUYjIQ0XFFbA7ro8Ocgg4f/EScsbFS1wq6bTcVlOj1aCm/irKLPWqSAaiE0FlZSVWrVqFbdu2ISAgAFOmTMHQoUMRGxsLALjrrrtQUNC0oUpDQwN+8YtfYNGiRQCAkpIS5OXlITMzs+sREJHHyiz1+KZSHZ/4O6vlgnUfX9/97JOSi6poIhLdWVxUVIRhw4ahV69eCAoKQlpaGgoLC9u99w9/+APuvfdeJCcnAwCKi4uxfft2TJgwAXPmzEF9vbo6Zoik1DxC6FzFD4lAp9MgJaHjeThqEGsOQ3hYDzicglsTkdKJrhFYrVYYjT/MXjWZTDhx4kSb+y5duoStW7di586drnNGoxFPPPEEBg8ejJUrV+Lll1/GihUrPPr+Ha2t3RGjMVTUc3LEWOTHH+LYf6ICjhYjhJLuMCI7LR7xMX3c7vOHWDrLk1iGJZqxs+gc7HYn9HothiWaZfWz8EVZRCcCp9PptsDbjRZ8++tf/4rRo0e7+gMAYO3ata6vn3rqKYwZM8bj7996Y5rOUNJsXMYiP/4SR9/wIOh0WuD6CKH0odEIDza4ld1fYukMT2MJDzZgzpQfVllt/bORkq82phHdNBQZGYmqqirXcVVVFUwmU5v79u7di/Hjx7uOL126hLfeest1LAgCdDqd2GIQkYea28IfSB2givZvMWLNYcgYHqOan43oRJCSkoKDBw+itrYWDQ0N2LNnD1JTU93uEQQBX375JQYNGuQ6FxQUhD/96U84fvw4AOCdd94RVSMgIvHU9kZHHRPdNBQREYHZs2cjJycHNpsNkydPRmJiIqZNm4ZZs2YhISEBtbW1MBgMCAwMdD2n0+mwevVqLFq0CFevXkVMTAyWL1/ulWCIiHypeQKeP2zM4wnFbF7fGWpu95QzpcSilDgAxtKeMks9lr37ORwOATqdBi9mD+72ZCC7PgIiIjUpKq6A4/oEPIdDQFFxhcQl8h4mAiIilWMiICLqhJSEKOh1TUPk9QqbgMdF54iIOiHWHIYXsgcrsrOYiYBIYZQ6skUOYs1hivyZMhEQKUjzOkJqXFNfCkrZ75iJgEhBXDuNqXBN/e7Wer/jqroG/GJUrMSlEoedxUQK0rymvlYD6HRaRW88L7XW+xsXHjrvt1tcskZApCDN6wixj8D3hsSZ8OXXPyxRLQjw2xoYEwGRArTuIPbHNyN/MzLJjKq6BhQeOg9BAAx6/62BMREQ+Tl2EEvnF6NiMehOo9/XwJgIiPxcUXEFbHYnAHYQS0EJNTB2FhP5sTJLPf7VYs0brVbjt80TJB0mAiI/Vnq+zrUKrwbAfQlRfv/plLofEwGRH2s5XFSv1ypq/RvqPuwjIPJjHC5K3tClGsHOnTsxfvx4jB07Fps3b25z/bXXXsOoUaMwadIkTJo0yXXPyZMnkZWVhbS0NMybNw92u70rxSBSNW47SV0lukZQWVmJVatWYdu2bQgICMCUKVMwdOhQxMb+MMW6pKQEK1eudNuzGAByc3Px29/+FklJSZg7dy62bt2K7Oxs8VEQEcmQvywAKLpGUFRUhGHDhqFXr14ICgpCWloaCgsL3e4pKSnBH/7wB0yYMAEvv/wyrl27BovFgqtXryIpKQkAkJWV1eY5IiJ/1zy/Y9s/z+KVLV/IevkJ0TUCq9UKo9HoOjaZTDhx4oTr+MqVK7jrrruQm5uLfv36IS8vD+vWrcPIkSPdnjMajaisrPT4+3e0/2ZHjMZQUc/JEWORH6XEATCWrtp/ogKOFgsAXqj5HsOT+nb5dX0Ri+hE4HQ6odFoXMeCILgdBwcHY8OGDa7jJ554AnPnzkVqamqHz3UWN69nLHKjlDgAxuINfcODoNNpAYcTOp0WfcODulwO2W1eHxkZiaqqKtdxVVUVTCaT67i8vBzvv/++61gQBOj1+jbPVVdXuz1HRKQEzSO6HkgdIPtlP0QngpSUFBw8eBC1tbVoaGjAnj17kJqa6rreo0cPvPLKK/j2228hCAI2b96MMWPGwGw2IzAwEEePHgUAFBQUuD1HRKQU/jKiS3TTUEREBGbPno2cnBzYbDZMnjwZiYmJmDZtGmbNmoWEhAS8/PLLeOaZZ2Cz2TB48GA8/vjjAID8/HzMnz8fly9fxj333IOcnByvBURERJ7RCILgWUO7TLCPgLHIjVLiABiLXMmuj4CIiJSBiYBIxsos9fjw4DlZj0En/8e1hohkihvOUHdhjYBIpkrP18HeYkJS6fm6mz9EJAITAZFMtVxiWqfz3/1wSf7YNEQkU1ximroLEwGRjClhP1ySPzYNEckIRwmRFFgjIJIJjhIiqbBGQCQTHCWkXlLXBFkjIJKJ5lFCjuvLFnOUkDrIoSbIREAkIz8ZGAkASEmIYrOQSrRXE2QiIFKh/ccs2LznFJxOAXq9FikJUVIXibqJHGqCTAREEiuz1OOd60kAAOwSfSokachhvggTAZHESs/XuS2prtVo2D+gMlLPF+nSqKGdO3di/PjxGDt2LDZv3tzm+t69ezFp0iRMnDgRzz77LOrrm3rEt2/fjvvuuw+TJk3CpEmTsGrVqq4Ug8ivxUX3hkGvhQZN68Y/PPZO1gZUTIoRRKJrBJWVlVi1ahW2bduGgIAATJkyBUOHDkVsbCwA4PLly1i0aBE++OADRERE4NVXX8Xvf/97zJ8/HyUlJcjLy0NmZqbXAiHyV3JoGiB5kGoEkegaQVFREYYNG4ZevXohKCgIaWlpKCwsdF232WxYuHAhIiIiAABxcXGoqKgAABQXF2P79u2YMGEC5syZ46opEKmVv+xtS74l1VwS0YnAarXCaDS6jk0mEyorK13HvXv3xpgxYwAAV69exR//+EeMHj0aAGA0GvHss8/ir3/9K6KiovDyyy+LLQYRkWJIteKs6KYhp9MJjUbjOhYEwe242aVLlzBz5kzEx8fjgQceAACsXbvWdf2pp55yJQxPdLT/ZkeMxlBRz8kRY5EfpcQBMBYpGI2h+N9eQSg+U42E229BfEyfdu/xNtGJIDIyEkeOHHEdV1VVwWQyud1jtVrx5JNPYtiwYZg7dy6ApsTwwQcf4LHHHgPQlEB0Op3H35+b1zMWuVFKHABjkVJ4sAEjE5vmkbQut+w2r09JScHBgwdRW1uLhoYG7NmzB6mpqa7rDocDM2bMQHp6OubNm+eqLQQFBeFPf/oTjh8/DgB45513RNUIiIjIO0TXCCIiIjB79mzk5OTAZrNh8uTJSExMxLRp0zBr1ixcvHgRX331FRwOBz766CMAwMCBA7FkyRKsXr0aixYtwtWrVxETE4Ply5d7LSAiIvKMRhAEz9pXZIJNQ4xFbpQSB8BY5GT/MQuOlloxJM6EX4yJ90nTEGcWExHJ1P5jFrxdWAoA+PLrOoSG9MCQ2HCvfx/uR0BEJFNHS61ux0Unyn3yfZgIiIhkakic+0jMAT6acMhEQEQkUyOTzEgfGo3mKVo7DpzB/mMWr38fJgIiIhkL6qFH85Aeh1PA5j2nvL4gHRMBEZGMxUX3hlb7w6oNTkHw+hpETAREPiT1puTk/2LNYfjl2Duh02qg0QB6H6xBxOGjRD4ih03JSRlGJpnR1xiCCzXfo294kNf/HTEREPmIHDYlJ+WINYdheFJfn0yOY9MQkY9ItaQwkadYIyDyEe48Rv6CiYDIB8os9a4EkDE8RuriEHWIiYDIy9hJTP6GfQREXibVvrNEYjEREHkZO4nJ37BpiMjL2ElM/qZLNYKdO3di/PjxGDt2LDZv3tzm+smTJ5GVlYW0tDTMmzcPdrsdAFBeXo6HH34Y48aNwzPPPIMrV650pRhEshNrDkPG8BgmAfILohNBZWUlVq1ahXfffRc7duzAe++9h7KyMrd7cnNzsWDBAnz00UcQBAFbt24FACxevBjZ2dkoLCzEwIEDsW7duq5FQSQDhQfPYcV7X/hkdUgiXxKdCIqKijBs2DD06tULQUFBSEtLQ2Fhoeu6xWLB1atXkZSUBADIyspCYWEhbDYbDh8+jLS0NLfzRP5s/zEL1r5/HF9+XYe3C0uZDMiviE4EVqsVRqPRdWwymVBZWXnD60ajEZWVlairq0NISAj0er3beSJ/1nonqdbHRHImurPY6XRCo/lhaVRBENyOb3S99X0A2hx3RkcbMXfEaAwV9ZwcMRb5GDkkGl9+Xed27O8x+Xv5W2IsHROdCCIjI3HkyBHXcVVVFUwmk9v1qqoq13F1dTVMJhP69OmDS5cuweFwQKfTtXmus2pqLsPpFDx6xmgM9cmCTVJgLPIyJDYcMyf/CPuPnseQOBOGxIb7dUxK+J00YyyAVqvp8MOz6KahlJQUHDx4ELW1tWhoaMCePXuQmprqum42mxEYGIijR48CAAoKCpCamgqDwYDk5GTs2rULALBjxw6354j81bjhMfifhwZhZJJZ6qIQeUR0IoiIiMDs2bORk5OD+++/H5mZmUhMTMS0adNQXFwMAMjPz8fSpUsxbtw4fP/998jJyQEALFy4EFu3bsX48eNx5MgR/PrXv/ZONERE5DGNIAieta/IBJuGGIvcKCUOgLHIleyahojUjFtQkpJwiQkiD3F1UVIa1giIPFR6vg52e9PqonauLkoKwERA5KHgngY0904JQtMxkT9jIiDy0JUGm+trTatjIn/EREDkobjo3jDom/Yb0Ou53wD5P3YWE3mI+w2Q0jAREIkQaw5jAiDFYNMQUSdx7gAplapqBP8+V4tPT1hYnSePce4AKZlqEkGZpR75//cFbHb+IZPnSs/Xwe5omjvguD53gP9+SClU0zTkNgnI7kRRcYXURSI/EhfdG3pd00ghnY4jhUhZVFMjiIvuDa1WA6dDgADg4+IKpCRE8VMddQpHCpGSqaZGEGsOw5gf93MdO50ClwYgj8Saw5AxPIZJgBRHNYkAAH6afJtrIhCr90RETVTTNAQA8TF9WL0nImpFdCIoLy9Hbm4uampq0L9/f+Tn5yM4ONjtHqvVipdeegnV1dXQarV44YUXMHz4cNhsNgwdOhS33Xab695t27ZBp9OJj6STOBGIiMid6KahxYsXIzs7G4WFhRg4cCDWrVvX5p7ly5fjpz/9KQoKCrBixQrMmTMHDocDpaWlGDRoEAoKClz/dUcSICKitkQlApvNhsOHDyMtLQ0AkJWVhcLCwjb3jRkzBpmZmQCAfv364dq1a/j+++9RXFyM2tpaZGVl4cEHH8ShQ4e6EAIREXWFqKahuro6hISEQK9vetxoNKKysrLNfc2JAgA2btyIu+66C6GhodBoNPjZz36G6dOn4/Tp05g2bRp27tyJPn36dLoMHe2/2RGjMVTUc3LEWORHKXEAjEWufBHLTRPB7t27sXTpUrdz/fr1g0ajcTvX+rilt956C++99x7eeecdAMCUKVNc1+6++24kJibi888/x+jRoztdcG5ez1jkRilxAIxFrny1ef1NE0F6ejrS09PdzjV39jocDuh0OlRVVcFkMrX7/PLly3HgwAFs3rwZkZGRAIAdO3Zg8ODBiI6OBgAIggCDQbpdnsos9RxJRESqJaqPwGAwIDk5Gbt27QLQ9Maempra5r633noLn332GbZs2eJKAgBQWlqKN954AwBw9uxZnDx5EkOGDBFTlC5rXkxs2z/P4pUtX3BlSSJSHdHDRxcuXIi8vDysX78eUVFRWLlyJQBgy5YtsFqtmDVrFtauXYuQkBA88sgjruf++Mc/YubMmZg7dy4yMzOh0WiwbNkyhISIa/PvKi4mRkRqJzoRmM1mbNq0qc35qVOnur4+fPjwDZ9fs2aN2G/tVc2LiTkcTs42JiJVUtXM4vZwMTFiHxGpneoTAcDZxmrGDWeIVLboXGdxS0L1aK+PiEhtWCNohZ8Q1YV9RERMBG1wFJG6sI+IiImgDX5CVI+WncQZw2OkLg6RZJgIWuEnRHXYf8yCd/acgtMpwKBnEyCpGxNBOziKSNnKLPXYfD0JAIDdziZAUjeOGiLVKSqugKPFgoUarYZNgKRqTASkKmWWevyruMJ1rNUAvxx7J2sDpGpMBKQqpefrXE1CGgCpP7oVI5PM0haKSGJMBKQqzaPCtBpAr9ciJSFK6iIRSY6dxaQqHBVG1BYTAakOR4URuWPTEBGRyomuEZSXlyM3Nxc1NTXo378/8vPzERwc7HaPxWJBZmama0vKW265BRs3bkRjYyPmzZuHkpIS9OjRA/n5+bj99tu7Fkk34ZLFRKQ0omsEixcvRnZ2NgoLCzFw4ECsW7euzT0lJSWYMGECCgoKUFBQgI0bNwIANm3ahJ49e2L37t2YO3cuXnrpJfERdCNua0lESiQqEdhsNhw+fBhpaWkAgKysLBQWFra5r7i4GKdOncKkSZOQk5OD0tJSAMD+/fsxceJEAMC9996L2tpalJeXi42h23DJYiJSIlGJoK6uDiEhIdDrm1qWjEYjKisr29wXGBiIiRMnYvv27XjyyScxc+ZMNDY2wmq1wmg0uu4zGo24ePGiyBC6T8uhh1yQjoiU4qZ9BLt378bSpUvdzvXr1w8ajcbtXOtjAHjuuedcX48YMQIrVqzA2bNnIQiC2/2CIECr9SwnhYeL2+zeaAwV9Vzzs//bKwjFZ6qRcPstiI/pI/q1vKErsciNUmJRShwAY5ErX8Ry00SQnp6O9PR0t3M2mw1Dhw6Fw+GATqdDVVUVTCZTm2c3bdqEzMxM9O7d9MlZEATo9XpERETAarW6OpGrq6vbfb4jNTWXXTNEO8toDEVV1SWPnmktPNiAkYlNk5C6+lpd4Y1Y5EIpsSglDoCxyJXYWLRaTYcfnkU1DRkMBiQnJ2PXrl0AgB07diA1NbXNfYcPH8b7778PADh06BCcTicGDBiAESNGoKCgAABw5MgRBAYG4tZbbxVTFCIi6iKNIAiefay+zmKxIC8vDzU1NYiKisLKlSsRFhaGLVu2wGq14vnnn0dlZSXy8vJQVVWFwMBALFmyBPHx8bh27RoWLFiAkpISBAQE4Le//S3uuecej76/VDUCuWAs8qOUOADGIle+qhGITgRSk0sikGpeAf9xy49S4gAYi1z5KhFwiYku4Eb3RKQEXGKiC1rOK7A7nCj4+CwnmUmozFKPDw+e4++AyEOsEXRB87yC5mTw5dd1OPXtF6wZSIC1MyLxWCPoguYlje+O+WFiGWccS4OzvonEYyLoolhzGCbdNwAGPWccS4mzvonEY9OQF7Te7AQAPjx4jiuUdiNuOEMkHhOBlzRvdsK26u7Vevguf9ZEnmMi8LL22qr55uQbTLpE3sE+Ai9jW3X3KLPUo+Djs7DZ2UFM1FWsEXgZ26p9z1UTsDsBABow6RJ1BROBD7Ct2rdczW8ANBrg7pjemHTfAP7MiURi0xD5nZbNb3qdlkmAqItYIyC/w+Y3Iu9iIugmUq1SqlRsfiPyHiaCbsBhjkQkZ+wj6AZcB6druKookW+JrhGUl5cjNzcXNTU16N+/P/Lz8xEcHOx2z4wZM1BRUQEAcDqdOHXqFN5//33Ex8dj6NChuO2221z3btu2DTqdTmxxZK25c9PhcHKYo4fKLPVY/u7nsDsE6HUavJA9mLUpIi8TnQgWL16M7OxsZGRkYO3atVi3bh1yc3Pd7nn99dddX7/66qtISkpCQkICSkpKMGjQIGzcuFF8yf0IOzfFKyqugN82xrQAAAwmSURBVN3RtBOd3SGgqLiCPz8iLxPVNGSz2XD48GGkpaUBALKyslBYWHjD+8+ePYsdO3bgxRdfBAAUFxejtrYWWVlZePDBB3Ho0CExxfArseYwZAyP4ZuYB8os9fimUhlbDBLJmagaQV1dHUJCQqDXNz1uNBpRWVl5w/vXrVuHJ598EiEhTXtmajQa/OxnP8P06dNx+vRpTJs2DTt37kSfPn3EFIcUaP8xCzbvOQVHi32pdToNUhKiJCwVkTLddPP63bt3Y+nSpW7n+vXrh/Pnz+PAgQMAALvdjkGDBqG4uLjN8/X19Rg3bhz279+PwMDAdr/HM888g5///OcYPXq02DhIQf59rhZ5az92SwKD7jQiOy0e8TH8sEDkbTetEaSnpyM9Pd3tnM1mw9ChQ+FwOKDT6VBVVQWTydTu8wcOHEBqaqpbEtixYwcGDx6M6OhoAIAgCDAYDB4VvKbmMpzODnNYG0ZjKKqq5NXUIHZ+gRxjEat1LJ+esLjXBLQapA+NRniwQdYxK/l34s8YC6DVahAeHnLj62IKYzAYkJycjF27dgFoemNPTU1t995jx44hOTnZ7VxpaSneeOMNAE39BydPnsSQIUPEFMWvNc8v2HbgLJZt/hz7j1mkLpKkmoeJBvc0wKDXQoOmf8APj72TfStEPiR61NDChQuRl5eH9evXIyoqCitXrgQAbNmyBVarFc8//zwA4Ntvv8XIkSPdnp05cybmzp2LzMxMaDQaLFu2zNV/oCal5+tgtzctnuZwCtj0USkAYGSSWdqCSaD1pLupo+/AlQYbR1kRdQPRicBsNmPTpk1tzk+dOtXteMOGDW3uCQkJwZo1a8R+a8WIi+4NrVbjagYRBGDznlPoawxR3Ztfy6RodzhxpcGGjOExUheLSBU4s1hCseYwPDz2Tmg0P5xzOJvGyqvJv8/V4mz5d2juFRAEILinZ31GRCQeE4HERiaZ8UhaHLQtksG/iitUs5xCmaUe817/BF+crnad0wC40mCTrlBEKsNEIAMjk8xI/dGtrmPheq1ADevrNDcJNdMA0Ou5DAdRd+LqozKRkhCFT0ouwuFwQqvV4J8nyuF0Nu3A9UhanGI7kOOie0Ov18Jub4r7voQopCREqa6PhEhKTAQy0XI9orPl37maSgQB2FSo3NFEseYwLJnxE3x6wsIRQkQSYSKQkebNVt4u/LfbeQHKGU20/5gFR0utGBJnciW2+Jg+CA9m5zCRVNhHIEMpCVHQtfrNOAXB7/cx2H/MgrcLS/Hl13V4u7BU9RPoiOSCiUCGYs1hePHhIRh0xy3Qapr6CfQK2MfgaKm1w2MikgabhmQq1hyG536eqIi9jpubg0J7BridHxLX/vpURNS9mAhkzp83aS+z1GP3p9+4zREYdncELjU0uvUREJG0mAjIJ1puMdnSpYZG/M9DgyQqFRG1h30E5HVllnps2XuqTRIA2BxEJEesESiEXPoSmlcRtbWYLQwAvUMDMOEn/dkcRCRDTAQK0HoJ59ypgyRLBqXn62B3uCcBnU6DZ+5P8Nu+DiKlYyJQgOY3X0EAHA4nSs/XSfamGxfdG3qd1rVUBpeMIJI/JgIFaPnmq5N4vkHLpTKkbqYios7pciJYvXo1dDodnnvuuTbXGhsbMW/ePJSUlKBHjx7Iz8/H7bffDkEQsHz5cvzjH/+AVqvFb37zG1VuVekt3fnm294SEe2VhwmAyH+ITgSXLl3C0qVL8eGHH+Kpp55q955NmzahZ8+e2L17Nw4fPoyXXnoJW7duxUcffYQzZ85g165d+OabbzB9+nTs2rULej0rKGJ58uZ7s47lMku9a3Oc6MhQnL/YtFl2z0A9dn92HgDw5ddNy12w85fI/4l+5923bx9iYmLw+OOP3/Ce/fv3u/Yuvvfee1FbW4vy8nIcOHAA48ePh1arRf/+/REVFYUvvvgC9957r9jiUCfdrGO5zFKPZe9+Dkc7Qz81rY6PllqZCIgUQPQ8gvvvvx9PP/00dDrdDe+xWq0wGo2uY6PRiIsXL8JqtcJkMrU5T77XXsdy6+vtJQEAaH2WcwKIlOGmNYLdu3dj6dKlbucGDBiAt95666YvLggCNC025BUEAVqtFk6ns93znggPD/Ho/mZGY6io5+RITCzDEs3YWXQOdrsTer0WwxLNbq8zLNGMgo+/bncymEGvxcT/HoCzlnqkJN6KcV7cXF4pvxelxAEwFrnyRSw3TQTp6elIT08X9eIRERGwWq2Ijo4GAFRXV8NkMiEyMhJW6w8rTzaf90RNzWU4ne1/cr0RozEUVVWXPHpGrsTGEh5swJwpP3Qshwcb3F4nPNiAF7IHt9tH0HoYqLd+lkr5vSglDoCxyJXYWLRaTYcfnn3aOztixAgUFBQgOTkZR44cQWBgIG699Vakpqbigw8+QGZmJi5cuIBz584hISHBl0WhFm7WscxRP0Tq4vVEsGXLFlitVjz//PN45JFHsGDBAmRkZCAgIADLly8HAIwbNw4nTpzAxIkTAQBLlixBjx49vF0UIiLqBI0gCJ61r8gEm4YYi9woJQ6AsciVr5qGuPooEZHKMREQEakcEwERkcr57ZoOWm3rea6+fU6OGIv8KCUOgLHIlZhYbvaM33YWExGRd7BpiIhI5ZgIiIhUjomAiEjlmAiIiFSOiYCISOWYCIiIVI6JgIhI5ZgIiIhUjomAiEjlVJUIjhw5gqysLEyYMAEzZsxAfX291EUS7ejRo5g8eTImTZqERx99FBaLReoiddnq1avx+9//XupiiLJz506MHz8eY8eOxebNm6UuTpddvnzZtXGUP3vttdeQkZGBjIwM134o/urVV1/F+PHjkZGRgTfffNO7Ly6oyOjRo4XTp08LgiAIr7zyirBixQqJSyTeqFGjhJMnTwqCIAh/+ctfhBkzZkhcIvG+++474aWXXhISExOFNWvWSF0cj128eFEYNWqUUFdXJ1y5ckWYMGGC69+ZPzp27JiQmZkp3HPPPcK3334rdXFE++STT4SHHnpIuHbtmtDY2Cjk5OQIe/bskbpYonz22WfClClTBJvNJjQ0NAijRo0Szpw547XXV1WNYNeuXYiNjYXNZkNlZSX+67/+S+oiidLY2Ijnn38e8fHxAIC4uDhUVFRIXCrx9u3bh5iYGDz++ONSF0WUoqIiDBs2DL169UJQUBDS0tJQWFgodbFE27p1KxYuXOjxPuJyYzQakZeXh4CAABgMBtx+++0oLy+Xulii/PjHP8bbb78NvV6PmpoaOBwOBAUFee31VZUIDAYDSktLMWLECHz22WfIyMiQukiiBAQEYNKkSQAAp9OJ1157DaNHj5a4VOLdf//9ePrpp6HT6aQuiihWqxVGo9F1bDKZUFlZKWGJumbJkiVITk6WuhhddscddyApKQkAcO7cOezevRsjRoyQuFTiGQwGrFmzBhkZGRg+fDgiIiK89tqKTAS7d+9Gamqq23+PPfYYgKZPz0VFRXj22Wcxe/ZsaQvaCR3F0tjYiDlz5sBut2P69OnSFrQTOorFnzmdTmg0PyzzKwiC2zFJ6/Tp03jiiSfwwgsvICYmRuridMmsWbNw8OBBVFRUYOvWrV57Xb/dj6Aj6enpSE9Pdzt37do17N271/XJeeLEiVi2bJkUxfNIe7EAwJUrV/DMM8+gV69eWL9+PQwGgwSl88yNYvF3kZGROHLkiOu4qqrK75tVlOLo0aOYNWsW5s6d67ctAABw5swZNDY24q677kLPnj0xduxYlJaWeu31FVkjaI9er8fixYtRUlICoOnT6eDBgyUulXi5ubno168fVq9ejYCAAKmLo2opKSk4ePAgamtr0dDQgD179iA1NVXqYqleRUUFZs6cifz8fL9OAgBw4cIFzJ8/H42NjWhsbMS+ffswZMgQr72+ImsE7dHpdFi1ahUWLFgAh8OBiIgILFmyROpiifLVV19h3759iI2NxQMPPACgqV16w4YNEpdMnSIiIjB79mzk5OTAZrNh8uTJSExMlLpYqrdx40Zcu3YNv/vd71znpkyZgqlTp0pYKnFGjBiBEydO4P7774dOp8PYsWO9mty4QxkRkcqppmmIiIjax0RARKRyTARERCrHREBEpHJMBEREKsdEQESkckwEREQqx0RARKRy/x/ziK+kBGd/KAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "x_train = 6 * torch.rand((1000, 1)) - 3\n", "y_train = torch.sin(x_train)\n", "x_test = 6 * torch.rand((100, 1)) - 3\n", "y_test = torch.sin(x_test)\n", "plt.plot(x_test.numpy(), y_test.numpy(), '.', label='TEST')\n", "plt.legend();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Build a simple fully connected network with 1 input for $x$, 1 output for $y(x)$, and 2 hidden layers with 20 and 25 nodes:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "torch.manual_seed(123)\n", "net = torch.nn.Sequential(\n", " torch.nn.Linear(1, 20),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(20, 25),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(25, 1),\n", ")" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden", "solution2_first": true }, "source": [ "**EXERCISE:** How many parameters does this model have? Compare with the size of our training dataset. What are the model's hyperparameters?" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "**ANSWER:** Each layer has $(n_\\text{in} + 1) n_\\text{out}$ parameters, so the total number is:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "solution2": "hidden" }, "outputs": [ { "data": { "text/plain": [ "591" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "2 * 20 + 21 * 25 + 26 * 1" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "We can query the model directly for details about its parameters:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "solution2": "hidden" }, "outputs": [ { "data": { "text/plain": [ "{'0.weight': torch.Size([20, 1]),\n", " '0.bias': torch.Size([20]),\n", " '2.weight': torch.Size([25, 20]),\n", " '2.bias': torch.Size([25]),\n", " '4.weight': torch.Size([1, 25]),\n", " '4.bias': torch.Size([1])}" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "{n: theta.shape for n, theta in net.named_parameters()}" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "solution2": "hidden" }, "outputs": [ { "data": { "text/plain": [ "591" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.sum([np.prod(theta.shape) for theta in net.parameters()])" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "Since there are almost as many parameters (591) as training data values (1000), there is a real danger of \"overfitting\" (memorizing the training data). Note that a polynomial model with only 4 (carefully chosen) parameters does an excellent job of \"learning\" $y(x) = \\sin(x)$:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "solution2": "hidden" }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deViU57n48e8s4ALGBYdFjKKhQhKxihiX5IfaqIh7qG3UpDSb0eipqaeSuOS4NdbG4BKbmNU0J2ptPIlKaBA9pkfPSTFRcANjUDSGOCCDgLgEZZh5f38gIwOIMsww2/25rlzx3Waem4H3nmd5n0elKIqCEEIIr6V2dgGEEEI4lyQCIYTwcpIIhBDCy0kiEEIILyeJQAghvJwkAiGE8HKSCIQQwstpnV0AW5WVXcNsbtojEAEB/pSUXHVQiVqWxOJ6PCUOkFhcla2xqNUqOnb0u+1xt00EZrPS5ERQc52nkFhcj6fEARKLq3JELNI0JIQQXk4SgRBCeDm3bRqqS1EUysqKqay8DjRcdTIY1JjN5pYtmIN4XiwKvr6t6dhRh0qlcnaRhPAqzU4EV69eZcqUKbzzzjt07drV6tjJkydZtGgR165dIyYmhmXLlqHVaikoKCApKYmSkhJ69OhBcnIyfn6378i4u3KUo1KpCArqikrVcEVHq1VTVeUZN09Pi8VorOLSpYtcvVpOu3YdnF0kIbxKs5qGjh07xtSpUzl37lyDx5OSkli8eDG7d+9GURS2bdsGwLJly5g2bRrp6en07t2bDRs2NKcYAFRUXKVduw63TQLCtalUatq160hFhWeM7hDCnTTrrrlt2zaWLFlCYGBgvWN6vZ7r16/Tt29fABISEkhPT8doNHLo0CHi4uKs9jeX2WxCo/GYli6vpNFoMZtNzi6GcKA8fTlfHDjHvqN6vjhwjjx9udX+mu3b7ROO0aw754oVK257zGAwoNPpLNs6nY6ioiLKysrw9/dHq9Va7bcHaVt2b/L5eaY8fTm5+WX4tfFh697TGG82aaqobhacOuJnbN17miqTGa1GTdLUfgC8vvWIZd/UET/jWoWRiG4dCQ9t78RoPJPDvkKbzWarP2xFUVCpVJb/12bLDSAgwN9q22BQo9XeuYJzN+fY0/r1a7l06RKLFy+z+2u3dCx3a9KksWzY8D5dunRp8HhWViYffPAub7/9vmVfTSxqtRqdrl2LlNMR3Lnsddkjlu/OlZL89yNUVVXfD8y11sFSAJPJTPbZUkwmM4pSvX2+5Ce4eUxRoKrKzJY9p1AUBa1WzYqZDwOQfeYiUfd1JjKsU4vE4iocEYvDEkFwcDDFxcWW7YsXLxIYGEinTp24cuUKJpMJjUZDcXFxg01Ld1JSctXqwQqz2XzHztOW7mDNzDxIWloqgwc/Yvf3dfXOYpPp9p9H9R+4YjleOxaz2Uxx8ZUWK6c96XTt3Lbsddkrlq+P6zFWVd/QVYqCWl2dDBRApQKNRk1Uz07knC0BkxmNRk3XgLZA9TFMZlCpMJurr6mqMvPF/53hXzkXrGoQjdUS5HOpfrK47pfn2hyWCEJDQ2nVqhVZWVn079+flJQUYmNj8fHxISYmhrS0NMaPH8/OnTuJjY11VDHuqKbaau8q5+XL5bz33gZ+85unycs73eA5hw9n8p//uRGNRkthoZ4HHniQl1/+D3x9fXn33bfIyjrE5cuX6dy5M8uXr6RTpwDGjRtBRMQDlJZe5P33P2b16j9z9uwZSktLCQ8PZ+nSFZSWlrJgwTy6d+/O99+fpVevSHr37sOuXf/gypXL/OlPyYSF9bAqy+TJ4xkxIo5Dh75Bo9Hw1FPP8fe/b+b8+R+ZPfv3PProSEpLS/jzn/9IUdEFNBoNzz8/m0GDhnD5cjnLl/8HBkMRYWE9qaysBMBkMrFhwxscOZKFyWRmzJhxPP74E3b7GQvXU/fvKaJbR7QaNaabN/maJh6/Nj5WTT1ddf71/g6TpvazalKqeQ2AqpragsnM1r2n6ODfivZ+vgyJCpGmIxvYPRFMnz6dOXPmEBUVRXJyMq+88gpXr17lwQcfJDExEYAlS5Ywf/583n77bUJCQlizZo29i3FX8vTlVu2Qd/pm0RSrVv2J55+fhcHQeP9HdvZxPvpoC/fe253/+I/5bN++jUceGUp+/jneeedD1Go1f/zjYnbv3sXUqU9y6dIlnngikYceeojMzEy0Wh/effevmM1m5syZyYED/yIi4n7OnDnNwoWLCQ/vxdSpCXTurOPdd//Khx++x+efb2fOnD/UK0unTgFs3LiJP/1pGZs3f8T69e+QnX2M9etX8+ijI1m79nWio2OYMuVJ9PrzzJr1HH/96xY++ugDevWKJDl5PUePHuaf//xvAFJTdwDw4YdbqKys5N///d+IjHzALj9f4Xpu9/dUc0Nv7MtWeGj7esdq76udKABLjUBR4PvCK0D1t+T/PVbAk3ERDOsb6rhAPZBdEsE///lPy7/ff/9Wu29kZCSffvppvfNDQ0PZtGmTPd66WXLzyyy/TCaTmdz8MrskgtTUnQQFBRET8xBpaamNntu3bz+6dQsDYPToMXz++Q6mTHmSf/u3uaSm7iQ//wdOnMgmNPTWMxoPPtj75rXR3HNPez77bBv5+ec4f/5HKioqgOqbeq9ekQDodIH07z8AgODgEI4cKWiwLIMGDQEgKCiYzp11aLVagoNDuHKl+o/s8OFDvPzyKwCEhnblgQd68+23ORw5ksXSpX+ylKlLl+o/wszMg5w+fYqsrEwAKip+4syZvHq1EeEZbvf31NBNvqnqvkbS1H6kfHWWE9+XWZ1nVmDznlN01flLzaAJvHq8Zd1qa823jeb68ss9lJRc5KmnpnH5cjkVFRWsX7+a6OgYPvjgXQAeeSSW6OgYNBqN5TqzWUGj0fDddydZunQRU6ZMY/jwR9Fo1Ci1OtlatWoNwFdf7eeDD97lV7+awpgxE7h06ZLlPB8fH6sy1X6f26l9TUPn15/sSsFkMlkGAdS91mQyM2vWHIYO/QUAly5dok2bNpw4kX3HsgjXd6dmIHv9PTUkPLQ9Ex/pyXf5hzGZrH8vzWaFlK/OMvGRnpIM7pJXJ4K7rbY21bp1tx6QS0tL5ciRLEtTzCOPDLUcO3w4k+PHj1JcbCAgoDPp6V8wcOAQjh7Nol+//kyaNJny8ktkZHxluZnWlpl5kF/8YgRjx05Arz/PkSNZDBjwkF1iaEj//jH84x87LU1D2dnH+MMfFhAT8xC7d6cRHv4iJ0+eQK8/bzn/88938vDDsVRWVjJr1rPMm7fAYeUTLSNPX05GdiFfZRdiMitNbgayl/DQ9rw8LZqM7EIKLl7jtL6cmu8jJ74vIzf/MFE9Awju7E+/8ABJCo3w6kQADbdNtqTOnXW8+uoSiosNDBgwkPHjJ1FaWsLChUkkJj4OQETE/RQW1m/OGT/+MZYtW8TevbvRan2IiupDQUEB/fs7pqy//30Sq1atIC0tFZVKxcsvv0Lnzp159tkZrFixjCef/DXdu3e3NA1NmjSZ8+d/5Omnp2EymRgzZjzR0TEcPpzpmAIKh9t3VM+WPacw1aod2rsZqClqv1+evtyquajKpHDk9EU4fZH/PvgDL02LlmRwGyqldp3ejdQdPnrhwg8EB3dv9BpXG3J5+HAmH374Hm+++V6Tr3W1WJqjdix38zm6Kk8fppinL+fPWw7XayL00dp3oEVz1HRYGxv42/jl0J6MHRzW8oWyI7cbPiqE8Cy5+WVWSUCtgtifd3GpIZs1zVMZ2YX87/ECaibo1WpUDu2zcHeSCJwoOjqG6OgYZxdDiLsS0a0jPjdrbyq1iidH9XLJYZo1zUVDokLIyC6kTRtf6SO4A0kEQoi70tKdwc1VkxA8qcnOUTwqETQ0j5FwH27aXeVVnD24QjiGa85aZgOt1pdr1y7LzcRNKYrCtWuX0Wp9nV0UQfVkcTIFtPfwmBpBx446ysqKuXr10m3PUas9Z3lHT4xFq/WlY0fdnS8QDlV7iKgrjQgSjuMxiUCj0dK5c0ij53hSW6HEIhwhT19u9ZxAVZX9pl5xVY6aeNKdeEwiEEI0T80DWbUfFlOpPXvYZZ6+nNf+Vj1NhUaj4mUvfehMEoEQ4tbMobUexNKoVTwxqpdH3xgzsgstcxWZTAoZ2YUeHe/tSCIQQtyaOZTqBWP6/kxH/MBuXnlT9EYeM2pICNF0NQvE+7XxQatRo1aBVqNmWlykVySBIVEhaDXVQ861GhXdgtt55WgpqREI4aXy9OWs+tthqkwKWo2KaSN7WVYNiwzr5BUd+OGh7XlpWrTVSmiOWKjK1TUrEaSmpvL2229TVVXFb3/7W5544tYyhCdPnmT+/PmW7dLSUtq3b88//vEPduzYwerVqwkICABg2LBhzJ07tzlFEUI0UUZ2IVU328erTAr5F66QODrSyaVqeTUPyX1x4JxDFqpyBzYngqKiItauXcv27dvx9fVlypQpDBw4kPDwcADuv/9+UlJSAKioqOBXv/oVS5cuBSAnJ4f58+czbty45kcghGiyPH05PxR5/jf+pqi9sI5KraKk/Dp5+nKvSAY29xFkZGQwaNAgOnToQNu2bYmLiyM9Pb3Bc999910GDBhATEz1BGvZ2dns2LGD8ePHM2/ePMrLvas9TghnqhkhdK7wViLQaFQMiWr8ORxPVzOXUuzPu6AC9h8r4PWtR7yiv8DmGoHBYECnu/UUaGBgIMePH6933pUrV9i2bRupqbfW7tXpdDzzzDNER0ezZs0ali9fzurVq5v0/o3Nrd0Yna6dTde5IonF9bhDHPuOF2KqM0JoWlwkkWGdrM5zh1ju1t3GotO143zJT5iPFViaiM6X/MTgvl3vfHELccTnYnMiMJvNVhO83W7Ct88//5wRI0ZY+gMA3nrrLcu/n3vuOUaOHNnk96+7MM3d8KQnWCUW1+MucXQNaItGo4abawvHD+xGgJ+PVdndJZa70dRY6v58uga0dZmfhcstTBMcHExm5q0lB4uLiwkMDKx33t69e5kxY4Zl+8qVK3z22Wc89dRTQHUCuZuF1YUQ9uFu00m3NG/8+djcRzBkyBAOHDhAaWkpFRUV7Nmzh9jYWKtzFEXhxIkT9OvXz7Kvbdu2fPDBBxw7dgyAzZs321QjEELYLjy0PWMHh3nFTc4W3vbzsblGEBQUxNy5c0lMTMRoNDJ58mT69OnD9OnTmTNnDlFRUZSWluLj40OrVq0s12k0GtatW8fSpUu5fv06YWFhrFq1yi7BCCGEaDqPWbz+bnhzu6cr85RYPCUOkFhux9kzlbpcH4EQQngTT56pVOYaEkKIu9DQTKWeQhKBEEJ4OUkEQghxF+rOVOpJT2JLH4EQHsbZHZqeqvZMpZ72s5VEIIQHsaw05oVTKbeEmplKPY00DQnhQSwrjdWaSlk4zr6jelZ/coR9R/XOLkqzSI1ACA9SeypljUbt0QvPO9u+o3o+Ts8F4MT3ZRSXVfCr4eFOLpVtJBEI4UG8cZ4cZ8nKNVhtpx/Mp18vnVv+zCURCOEB6nYQu+PNyN30jwjkxPe3mt4UBbdd1UwSgRBuTjqInWNY31CKyypIP5iPooCP1n2b4iQRCOHmMrILMVaZAe9ba9fZfjU8nH69dG7fFCeJQAg3lqcv5/9qTXWgVqvc9lupu/KEpjgZPiqEG8vNL7PMwqsCHokKcfubkmh5kgiEcGM1w0XVKtBq1R417YFoOdI0JIQbk+Giwh6aVSNITU1lzJgxjBo1ii1bttQ7/uabbzJ8+HAmTpzIxIkTLeecPHmShIQE4uLiWLRoEVVVVc0phhBezduWVRT2Z3ONoKioiLVr17J9+3Z8fX2ZMmUKAwcOJDz81pN1OTk5rFmzxmrNYoCkpCReffVV+vbty8KFC9m2bRvTpk2zPQohhBA2s7lGkJGRwaBBg+jQoQNt27YlLi6O9PR0q3NycnJ49913GT9+PMuXL+fGjRvo9XquX79O3759AUhISKh3nRBCeII8fTlfHDhHnr7c2UVplM01AoPBgE6ns2wHBgZy/Phxy/a1a9e4//77SUpKonv37syfP58NGzYwbNgwq+t0Oh1FRUVNfv/G1t9sjE7XzqbrXJHE4no8JQ6QWJrru3OlJP/9CFVVZrRaNStmPkxkWKdmv64jYrE5EZjNZlQqlWVbURSrbT8/P95//33L9jPPPMPChQuJjY1t9Lq7JYvXSyyuxlPiAInFHr4+rsdYVT0TbFWVma+P6wnw82nWazpq8Xqbm4aCg4MpLi62bBcXFxMYGGjZLigo4NNPP7VsK4qCVqutd93FixetrhNCCE9Qe2ivq88Ea3MiGDJkCAcOHKC0tJSKigr27NlDbGys5Xjr1q15/fXX+fHHH1EUhS1btjBy5EhCQ0Np1aoVWVlZAKSkpFhdJ4QQnqBmaO9jsT1dfv4nm5uGgoKCmDt3LomJiRiNRiZPnkyfPn2YPn06c+bMISoqiuXLl/PCCy9gNBqJjo7m6aefBiA5OZlXXnmFq1ev8uCDD5KYmGi3gIQQwlW4y/QTKkVRmtbQ7iKkj0BicTWeEgdILK7K5foIhBBCeAZJBEK4MHcZhy7cm8w1JISLkgVnREuRGoEQLio3v4wqU/U49JoFZ4RwBEkEQrgodxqHLtybNA0J4aJkimnRUiQRCOHC3GUcunBv0jQkhAuRUULCGaRGIISLkFFCwlmkRiCEi5BRQsJZJBEI4SJklJBwFmkaEsKFPNw7GIAhUSHSLCRajCQCIVzAvqN6tuw5hdmsoNWqGRIV4uwiiRaUpy936jBhSQRCOFmevpzNN5MAQNXN/gGpEXgHVxgkIH0EQjhZbn6Z1ZTqapVK+ge8iCsMEmhWIkhNTWXMmDGMGjWKLVu21Du+d+9eJk6cyIQJE5g1axbl5dVjo3fs2MEjjzzCxIkTmThxImvXrm1OMYRwaxHdOuKjVaOiet74J0b1ktqAF3GFQQI2Nw0VFRWxdu1atm/fjq+vL1OmTGHgwIGEh4cDcPXqVZYuXcpnn31GUFAQb7zxBn/5y1945ZVXyMnJYf78+YwbN85ugQjhrmQqCe/mCp+/zTWCjIwMBg0aRIcOHWjbti1xcXGkp6dbjhuNRpYsWUJQUBAAERERFBYWApCdnc2OHTsYP3488+bNs9QUhPBW4aHtGTs4TJKAl3L2529zIjAYDOh0Ost2YGAgRUVFlu2OHTsycuRIAK5fv857773HiBEjANDpdMyaNYvPP/+ckJAQli9fbmsxhBBCNJPNTUNmsxmVSmXZVhTFarvGlStXmD17NpGRkTz22GMAvPXWW5bjzz33nCVhNEVj6282RqdrZ9N1rkhicT2eEgdILM7y3blSss9cJOq+zkSGdap33BGx2JwIgoODyczMtGwXFxcTGBhodY7BYODZZ59l0KBBLFy4EKhODJ999hlPPfUUUJ1ANBpNk99fFq+XWFyNp8QBEouz3GkoqcstXj9kyBAOHDhAaWkpFRUV7Nmzh9jYWMtxk8nEzJkziY+PZ9GiRZbaQtu2bfnggw84duwYAJs3b7apRiCEEJ7GWUNJba4RBAUFMXfuXBITEzEajUyePJk+ffowffp05syZw4ULF/j2228xmUzs3r0bgN69e7NixQrWrVvH0qVLuX79OmFhYaxatcpuAQkhhLuqGUpqMplbdCipSlGUprWvuAhpGpJYXI2nxAESizM1Nt2Eo5qGZIoJIYRwIc5YlU6mmBBCCC8niUAIIbycJAIhhHBh+47qWf3JEfYd1TvsPaSPQAghXNS+o3o+Ts8F4MT3ZbTzb03/8AC7v4/UCIQQwkVl5RqstjOOFzjkfSQRCOFAefpyvjhwjjy9TKwomq5/hPVsDT0dNJpImoaEcBBXWHlKuLdhfUMpLqsg/WA+igKpX50loqv9h5dKjUAIB3GFlaeE+2vb+tb3daPRTEZ2od3fQxKBEA7iCitPCfcX0a0janX1XG0K8FV2od2bGqVpSAgHcYWVp4T7Cw9tz/+LCmHf0eqOYrNZITe/zK6/T1IjEMIBajqJAVl5TDTbkKgQfLSOq11KjUAIO5NOYmFvNbXL8yU/0TWgrd1/nyQRCGFnDXUSSyIQzRUe2p7Bfbs6ZCZVaRoSws6kk1i4G6kRCGFn0kks3E2zagSpqamMGTOGUaNGsWXLlnrHT548SUJCAnFxcSxatIiqqioACgoKeOKJJxg9ejQvvPAC165da04xhHA54aHtpZNYuA2bE0FRURFr167lb3/7Gzt37uSTTz4hLy/P6pykpCQWL17M7t27URSFbdu2AbBs2TKmTZtGeno6vXv3ZsOGDc2LQggXkH7gnMNniRTCEWxOBBkZGQwaNIgOHTrQtm1b4uLiSE9PtxzX6/Vcv36dvn37ApCQkEB6ejpGo5FDhw4RFxdntV8Id7bvqJ63Pj3Gie/L+Dg9V5KBcCs2JwKDwYBOp7NsBwYGUlRUdNvjOp2OoqIiysrK8Pf3R6vVWu0Xwp3VnSWy7rYQrszmzmKz2YxKpbJsK4pitX2743XPA+pt343GFmJujE7XzqbrXJHE4jqG9e/Gie/LrLbdPSZ3L39tEkvjbE4EwcHBZGZmWraLi4sJDAy0Ol5cXGzZvnjxIoGBgXTq1IkrV65gMpnQaDT1rrtbJSVXMZuVJl2j07VzyBhcZ5BYXEv/8ABmT/45+7Ly6R8RSP/wALeOyRM+kxoSC6jVqka/PNvcNDRkyBAOHDhAaWkpFRUV7Nmzh9jYWMvx0NBQWrVqRVZWFgApKSnExsbi4+NDTEwMaWlpAOzcudPqOiHc1ejBYfzh8X4M6xvq7KII0SQ2J4KgoCDmzp1LYmIikyZNYty4cfTp04fp06eTnZ0NQHJyMitXrmT06NH89NNPJCYmArBkyRK2bdvGmDFjyMzM5Pe//719ohFCCNFkKkVRmta+4iKkaUhicTWeEgdILK7K5ZqGhPBmsgSl8CQyxYQQTSSziwpPIzUCIZooN7+Mqqrq2UWrZAlK4QEkEQjRRH5tfKjpnVKU6m0h3JlXJYLvzpVKu65otmsVRsu/VXW2hXBHXtNHkKcvJ/nvRzBWSbuuaJ6Ibh3x0aoxmcyy3oDwCF6TCGq368qqUaI5ZL0B4Wm8JhFEdOuIVqvGaDSDSiXtuqJZwkPbSwIQHsNr+gjCQ9szfWIUarUKxaywde9p6SsQTSLPDghP5TU1AoArP1ViVhQUpHlINI08OyA8mdfUCACi7ussi4oLm+Tml1Flsu5jEsJTeFWNIDKsk3TyCZtEdOuIViMjhYRn8qpEANLJJ2wjI4WEJ/O6RCCEreRLhPBUXtVHIIQQoj5JBEII4eVsbhoqKCggKSmJkpISevToQXJyMn5+flbnGAwGFixYwMWLF1Gr1bz00ksMHjwYo9HIwIEDuffeey3nbt++HY1GY3skQgghbGJzjWDZsmVMmzaN9PR0evfuzYYNG+qds2rVKn7xi1+QkpLC6tWrmTdvHiaTidzcXPr160dKSorlP0kCQgjhHDYlAqPRyKFDh4iLiwMgISGB9PT0eueNHDmScePGAdC9e3du3LjBTz/9RHZ2NqWlpSQkJPDrX/+agwcPNiOE5pMnRoUQ3sympqGysjL8/f3Raqsv1+l0FBUV1TuvJlEAbNy4kfvvv5927dqhUql49NFHmTFjBqdPn2b69OmkpqbSqVOnuy5DY+tvNkana2e1/d25UpL/foSqKjNarZoVMx8mMuzuy+FMdWNxZ54Si6fEARKLq3JELHdMBLt27WLlypVW+7p3745KpbLaV3e7to8++ohPPvmEzZs3AzBlyhTLsQceeIA+ffpw+PBhRowYcdcFt9fi9V8f12OsWW2qyszXx/UE+Ln+hHSyILfr8ZQ4QGJxVY5avP6OiSA+Pp74+HirfTWdvSaTCY1GQ3FxMYGBgQ1ev2rVKvbv38+WLVsIDg4GYOfOnURHR9OtWzcAFEXBx8c5N195YlQI4e1s6iPw8fEhJiaGtLQ0oPrGHhsbW++8jz76iG+++YatW7dakgBAbm4uH374IQBnz57l5MmT9O/f35aiNFvNE6OPxfaUicSEEF5JpShK09pXbtLr9cyfP5+SkhJCQkJYs2YN7du3Z+vWrRgMBubMmcNDDz2Ev78/99xzj+W69957Dz8/PxYuXMjZs2dRqVQsWrSIQYMGNen97dU05K4kFtfjKXGAxOKqHNU0ZHMicDZJBBKLq/GUOEBicVWOSgTyZHEDZDipd5HPW3g7mXSuDlmAxLvI5y2E1AjqkQVIvIt83kJIIqinZjiprGLmHeTzFkKahuqRBUi8i3zeQkgiaJAsQOId8vTllgQwdnCYs4sjhNNIIhBead9RPZv3nMJsVvDRSiex8G7SRyC8Tp6+nC03kwBUzzElncTCm0kiEF4nI7sQU62HEVVqlXQSC68miUB4lTx9Of+XXWjZVqvgyVG9pFlIeDVJBMKr5OaXWZqEVEDsz7swrG+ocwslhJNJIhBepfZzA1qtmiFRIc4ukhBOJ6OGhFeR5waEqE8SQRPVHnsuNxH3JM+JCGFNEkETyARlQghPZHMiKCgoICkpiZKSEnr06EFycjJ+fn5W5+j1esaNG2dZkrJz585s3LiRyspKFi1aRE5ODq1btyY5OZn77ruveZG0gIYmKJNEIIRwdzZ3Fi9btoxp06aRnp5O79692bBhQ71zcnJyGD9+PCkpKaSkpLBx40YANm3aRJs2bdi1axcLFy5kwYIFtkfQgmSCMiGEJ7IpERiNRg4dOkRcXBwACQkJpKen1zsvOzubU6dOMXHiRBITE8nNzQVg3759TJgwAYABAwZQWlpKQUGBrTG0GFnfWAjhiWxKBGVlZfj7+6PVVrcs6XQ6ioqK6p3XqlUrJkyYwI4dO3j22WeZPXs2lZWVGAwGdDqd5TydTseFCxdsDKFlhYe2Z+zgMEkCQgiPccc+gl27drFy5Uqrfd27d0elUlntq7sN8Lvf/c7y76FDh7J69WrOnj2LoihW5yuKglrdtJzU2PqbjdHp2tl0nSuSWFyPp8QBEourckQsd0wE8fHxxMfHW+0zGo0MHDgQk8mERqOhuLiYwMDAep41IvoAAA7mSURBVNdu2rSJcePG0bFjdVu6oihotVqCgoIwGAyWTuSLFy82eH1jZPF6icXVeEocILG4KpdavN7Hx4eYmBjS0tIA2LlzJ7GxsfXOO3ToEJ9++ikABw8exGw207NnT4YOHUpKSgoAmZmZtGrVii5duthSFKeThc+FEO5OpShK075W36TX65k/fz4lJSWEhISwZs0a2rdvz9atWzEYDLz44osUFRUxf/58iouLadWqFStWrCAyMpIbN26wePFicnJy8PX15dVXX+XBBx9s0vu7Qo3Amc8VyLcc1+MpcYDE4qocVSOwORE4myskgi8OnGP7/55FUapnsXwstmeLrXQlv9yux1PiAInFVblU05CoVvu5ArVaRUn5dWkiEkK4HUkEzVDzXEHsz7ugAPuPFfD61iOSDJxE+muEsI3MNdRM4aHtLXPcy9QTziPzQAlhO6kR2IFMPeF8Dc0DJYS4O1IjsIO6c9xDdUeyTFXdcmqSsclklmQsRBNJIrCTmjnupYnCOWTBGSFsJ4nAzmSq6pZVd6Eg+VkL0XSSCOxMmihajtS+hLAPSQR2Jk0ULSNPX07KV2cxVpkBqX0J0RySCBxAmigcy1ITuJkEVMhoLSGaQxKBcDuWfhhApYIHwjoy8ZGeknyFsJE8RyDcTu3nNrQatSQBIZpJagQtpO7oFmE76YcRwr4kEbQAGd1if9IPI4T9SNNQC5DpD4QQrkwSQQuQuYiaR2YVFcKxbG4aKigoICkpiZKSEnr06EFycjJ+fn5W58ycOZPCwkIAzGYzp06d4tNPPyUyMpKBAwdy7733Ws7dvn07Go3G1uK4NGnTtl2evpxVfztMlUlBq1Hx0rRo+fkJYWc2J4Jly5Yxbdo0xo4dy1tvvcWGDRtISkqyOuedd96x/PuNN96gb9++REVFkZOTQ79+/di4caPtJXcz0qZtm4zsQqpM1SvRVZkUMrIL5ecohJ3Z1DRkNBo5dOgQcXFxACQkJJCenn7b88+ePcvOnTt5+eWXAcjOzqa0tJSEhAR+/etfc/DgQVuKIYQQwg5sqhGUlZXh7++PVlt9uU6no6io6Lbnb9iwgWeffRZ//+o1M1UqFY8++igzZszg9OnTTJ8+ndTUVDp16mRLcYQHytOXk5FdSPm1SjRqMJtBrVExJCrE2UUTwuPccfH6Xbt2sXLlSqt93bt3Jz8/n/379wNQVVVFv379yM7Ornd9eXk5o0ePZt++fbRq1arB93jhhRf45S9/yYgRI2yNQ3iQ786VsmDDV5YmIY1GxaiHuvOLmHuJDJMvC0LY2x1rBPHx8cTHx1vtMxqNDBw4EJPJhEajobi4mMDAwAav379/P7GxsVZJYOfOnURHR9OtWzcAFEXBx8enSQUvKbmK2dxoDqtHp2tHcfGVJl3jaLY+aOaKsdiqbixfH9dbkgCAyaTQ1ldNgJ+PS8fsyZ+JO5NYQK1WERDgf/vjthTGx8eHmJgY0tLSgOobe2xsbIPnHj16lJiYGKt9ubm5fPjhh0B1/8HJkyfp37+/LUVxazUPmm3ff5bXthxm31G9s4vkVDXDRP3a+KDRqCz7tRqVDLkVwoFsHjW0ZMkS5s+fz9tvv01ISAhr1qwBYOvWrRgMBl588UUAfvzxR4YNG2Z17ezZs1m4cCHjxo1DpVLx2muvWfoPvElufhlVVdWTp5nMCpt25wIwrG+ocwvmBHWfvn5iZC/yL1R/8xkSFSIjhYRwIJsTQWhoKJs2baq3f+rUqVbb77//fr1z/P39Wb9+va1v7TEiunVErVZhutnEpSh4bTKonRSrTGauVRhJHB3p7GIJ4RXkyWInCg9tzxOjeqG61QqCosDmPae86ina786VcrbgMjW9AooCfm2a1mckhLCdTDrnZDXf/DftzqVm/JZiVrxmta08fTnJfz9CpdFs2acCrlUYnVcoIbyM1AhcwLC+ofwmLgK1WoUK0GrV+LXx8Yr5dWqahGrUxC+dw0K0HKkRuIhhfUPpqvMnN78MvzY+bPnvU5hMCioV/CYuwmP7DCK6dUSrVVNVZUatVvFIVIh0DgvRwiQRuJCa+Yg+Tv8Ok6lWB3K653Ygh4e2Z8XMh/n6uF4m5BPCSSQRuAEF2LLnFF11/m5/o9x3VE9WroH+EYGWxBYZ1okAP+kcFsJZpI/ABQ2JCkFT55MxK4rbL2iz76iej9NzOfF9GR+n53r9A3RCuApJBC4oPLQ9Lz/Rn34/64xaBaqbi7S7ewdqVq6h0W0hhHNI05CLCg9tz+9+2ccjFr2vaQ5q18bXan//iIbnpxJCtCxJBC7OnRe0ydOXs+vrHzhy+qJl36AHgrhSUWnVRyCEcC5JBMIhai8xWduVikr+8Hg/J5VKCNEQ6SMQdpenL2fr3lP1kgBIc5AQrkhqBB7CVfoSamYRNdZ6WhigYztfxj/cQ5qDhHBBkgg8QN0pnJOm9nNaMsjNL6PKZJ0ENBoVL0yKctu+DiE8nSQCD1Bz81UUMJnMTp2wLqJbR7QaNSaTTBkhhLuQROABat98NU5+3iA8tD1JU/u5RDOVEOLuNDsRrFu3Do1Gw+9+97t6xyorK1m0aBE5OTm0bt2a5ORk7rvvPhRFYdWqVfzP//wParWaP/7xj165VKW9tOTNt6EpIhoqjyQAIdyHzYngypUrrFy5ki+++ILnnnuuwXM2bdpEmzZt2LVrF4cOHWLBggVs27aN3bt3c+bMGdLS0vjhhx+YMWMGaWlpaLVSQbFVU26+d+pYztOXk5FdCEC34HaWJSPbtNKy65t8AE58Xz3dhXT+CuH+bL7zfvnll4SFhfH000/f9px9+/ZZ1i4eMGAApaWlFBQUsH//fsaMGYNaraZHjx6EhIRw5MgRBgwYYGtxxF26U8dynr6c1/522DL7aW2qOttZuQZJBEJ4AJufI5g0aRLPP/88Go3mtucYDAZ0Op1lW6fTceHCBQwGA4GBgfX2C8drqGO57vGGkgBA3b3yTIAQnuGONYJdu3axcuVKq309e/bko48+uuOLK4qCqtaCvIqioFarMZvNDe5vioAA/yadX0Ona2fTda7IllgG9QklNeMcVVVmtFo1g/qEWr3OoD6hpHz1fYMPg/lo1Uz4fz05qy9nSJ8ujB4c1pziW/GUz8VT4gCJxVU5IpY7JoL4+Hji4+NtevGgoCAMBgPdunUD4OLFiwQGBhIcHIzBcGvmyZr9TVFSchWzueFvrrej07WjuPhKk65xVbbGEuDnw7wptzqWA/x8rF4nwM+Hl6ZFN9hHUHcYqL1+lp7yuXhKHCCxuCpbY1GrVY1+eXZo7+zQoUNJSUkhJiaGzMxMWrVqRZcuXYiNjeWzzz5j3LhxnD9/nnPnzhEVFeXIooha7tSxLKN+hPAudk8EW7duxWAw8OKLL/Kb3/yGxYsXM3bsWHx9fVm1ahUAo0eP5vjx40yYMAGAFStW0Lp1a3sXRQghxF1QKYrStPYVFyFNQxKLq/GUOEBicVWOahqS2UeFEMLLSSIQQggvJ4lACCG8nNvO6aBW133O1bHXuSKJxfV4ShwgsbgqW2K50zVu21kshBDCPqRpSAghvJwkAiGE8HKSCIQQwstJIhBCCC8niUAIIbycJAIhhPBykgiEEMLLSSIQQggvJ4lACCG8nFclgszMTBISEhg/fjwzZ86kvLzc2UWyWVZWFpMnT2bixIn89re/Ra/XO7tIzbZu3Tr+8pe/OLsYNklNTWXMmDGMGjWKLVu2OLs4zXb16lXLwlHu7M0332Ts2LGMHTvWsh6Ku3rjjTcYM2YMY8eO5a9//at9X1zxIiNGjFBOnz6tKIqivP7668rq1audXCLbDR8+XDl58qSiKIryX//1X8rMmTOdXCLbXb58WVmwYIHSp08fZf369c4uTpNduHBBGT58uFJWVqZcu3ZNGT9+vOX3zB0dPXpUGTdunPLggw8qP/74o7OLY7N//etfyuOPP67cuHFDqaysVBITE5U9e/Y4u1g2+eabb5QpU6YoRqNRqaioUIYPH66cOXPGbq/vVTWCtLQ0wsPDMRqNFBUVcc899zi7SDaprKzkxRdfJDIyEoCIiAgKCwudXCrbffnll4SFhfH00087uyg2ycjIYNCgQXTo0IG2bdsSFxdHenq6s4tls23btrFkyZImryPuanQ6HfPnz8fX1xcfHx/uu+8+CgoKnF0smzz00EN8/PHHaLVaSkpKMJlMtG3b1m6v71WJwMfHh9zcXIYOHco333zD2LFjnV0km/j6+jJx4kQAzGYzb775JiNGjHByqWw3adIknn/+eTQajbOLYhODwYBOp7NsBwYGUlRU5MQSNc+KFSuIiYlxdjGa7Wc/+xl9+/YF4Ny5c+zatYuhQ4c6uVS28/HxYf369YwdO5bBgwcTFBRkt9f2yESwa9cuYmNjrf576qmngOpvzxkZGcyaNYu5c+c6t6B3obFYKisrmTdvHlVVVcyYMcO5Bb0LjcXizsxmMyrVrWl+FUWx2hbOdfr0aZ555hleeuklwsLCnF2cZpkzZw4HDhygsLCQbdu22e113XY9gsbEx8cTHx9vte/GjRvs3bvX8s15woQJvPbaa84oXpM0FAvAtWvXeOGFF+jQoQNvv/02Pj4+Tihd09wuFncXHBxMZmamZbu4uNjtm1U8RVZWFnPmzGHhwoVu2wIAcObMGSorK7n//vtp06YNo0aNIjc3126v75E1goZotVqWLVtGTk4OUP3tNDo62smlsl1SUhLdu3dn3bp1+Pr6Ors4Xm3IkCEcOHCA0tJSKioq2LNnD7Gxsc4ultcrLCxk9uzZJCcnu3USADh//jyvvPIKlZWVVFZW8uWXX9K/f3+7vb5H1ggaotFoWLt2LYsXL8ZkMhEUFMSKFSucXSybfPvtt3z55ZeEh4fz2GOPAdXt0u+//76TS+adgoKCmDt3LomJiRiNRiZPnkyfPn2cXSyvt3HjRm7cuMGf//xny74pU6YwdepUJ5bKNkOHDuX48eNMmjQJjUbDqFGj7JrcZIUyIYTwcl7TNCSEEKJhkgiEEMLLSSIQQggvJ4lACCG8nCQCIYTwcpIIhBDCy0kiEEIILyeJQAghvNz/B0NpU4iXVeEyAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "simple_model = lambda x: x - x ** 3 / 6 + x ** 5 / 120 - x ** 7 / 5040\n", "y_pred = simple_model(x_test)\n", "plt.plot(x_test.numpy(), y_pred.numpy(), '.', label='4-param model')\n", "plt.legend()" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "This model has no explicit numerical hyperparameters, but its entire architecture is an arbitrary choice that we do not learn from the data. Therefore, we should really consider the number of hidden layers, nodes per hidden layer, and choice of activation function as \"hyperparameters\".\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our choice of ReLU activation is fairly arbitrary here (since other activations would also work). We do not use any activation on the final outputs since this is a regression problem (although a Tanh activation could work in this example since our output range is [-1, +1]).\n", "\n", "Save the initial state of the network so we can restore it before each of our learning experiments below (details [here](https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-model-for-inference)):" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "torch.save(net.state_dict(), 'net.pth')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, define the [loss function](https://pytorch.org/docs/stable/nn.html#mseloss) we will use throughout:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "loss_fn = torch.nn.MSELoss()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since our network is initialized with random weights, its initial prediction is completely wrong:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1jUdd74/+ecIBSRZAdQMTPZW13PRGl2L1hhhqK5aOUhtUyzbnft9lox1DY112VzPZSl3qVudlv51TsUokXyZ3dSK94qaSiuUmDt5iEGwROKMIfP749pRgaQw3CY0+txXXutn9PM+wU0r3mfVYqiKAghhPBZalcXQAghhGtJIhBCCB8niUAIIXycJAIhhPBxkgiEEMLHSSIQQggfJ4lACCF8nNbVBXDWpUvXsViaNgUiJCSQ0tLyVipR25JY3I+3xAESi7tyNha1WsWdd7a/7XWPTQQWi9LkRGB7zltILO7HW+IAicVdtUYs0jQkhBA+ThKBEEL4OEkEQgjh45qdCMrLy0lISODs2bO1rp06dYrExERGjhzJ4sWLMZlMAJw/f54pU6bw2GOP8eKLL3L9+vXmFkMIIYSTmpUI8vLymDRpEj/88EOd15OSknj11Vf57LPPUBSFnTt3ArBs2TImT55MVlYW/fr1Y8OGDc0phhBCiGZoViLYuXMnS5YsITQ0tNa1c+fOcfPmTQYNGgRAYmIiWVlZGI1Gjhw5wsiRIx3OCyG8X0VRIWWZn3I5ez9lmZ9SUVTocN52fLtzonU0a/joihUrbnvNYDCg1+vtx3q9nuLiYi5dukRgYCBardbhvBDCO1UUFVJRcBp1+0BK/t9HKMYq6wWVCpVWh37iZOt5kxGVVkfE/AUAnF210n5OP3EyluvlBPTqTUDPSBdG451abR6BxWJBpVLZjxVFQaVS2f+/uprHjRESEuhUufT6Dk49544kFvfjLXFAy8Ry9XQBhatXYjGZrP/9Wyy3LioKitlE1YljKGaT/Vh99nvrZds5k5GSj7ahKApqrZa+y5daXzv/JEH9+hLUu1ebxOIuWiOWVksE4eHhlJSU2I8vXrxIaGgonTp14tq1a5jNZjQaDSUlJXU2LTWktLS8yRMr9PoOlJRca/J7uSOJxf14SxzQcrGUHTqKxWi0fqCrVKBWg8UCimKtEWi0+PUfjOrkKRRMqDRaLBE9AFBptCiYgJ8TiKJgMZn4V+b/x9WcA9bagkZD0IO/JmjYg7etKcjvxTqzuL4vz62WCLp27Yq/vz9ff/019957L+np6cTExKDT6YiOjiYzM5MxY8aQlpZGTExMaxVDCNGGbM1AtiacgF69UWl1KGbrh7ytiUfdPtChqcc/IsLhOYCI+Qscm5R+fg0AxfRzcjGZuJL9BVe+zKb9wEF0ih8lTUdOaPFEMGvWLObOnUv//v1ZtWoVr7zyCuXl5fTt25dp06YBsGTJEpKTk9m4cSOdO3dmzZo1LV0MIUQbqygqdGjXj5i/gICekfYP9Pra9wN6Rta6Vv1c9UQBWGsEtr4GAMXC9W+Ocv34N4ROmUZw7PBWidFbqTx183ppGpJY3I23xAHOxVKW+SkXd6dam33Uan4xLpFOoxJapXwVRYVczTnAlS+zQbE4XlRr6PbyQnsS8fXfCzTcNCQzi4UQTqk5vNPWDIRajUqjtX97bw0BPSMJmzqd0KenWfsdqrOYKf0kTYadNoHHrj4qhHAN27fxqwe+QjGbm9wM1JKCY4fjHxFB2Z5Mrud9Y68d3DiZT0XBadr1G8CV8F/gN/h+6TuohyQCIUSjXc7ej+GjbWA2288pZpP1w//nNv22/sAN6BlJ19/OpaKokNJP0rhxMt9aLpPJ2m8AqPb9LxFJyZIMbkOahoQQjVJRVIjhQ8ckALR6M1BjBfSMJGTsOFQ6v1rXFJM1WYm6SY1ACNEoFQWnwVItCajVdPx1bL1j+NuarXnqas4Brnz1pb28Kq17JCt3JYlACNEoAb16o9L5Wcfwq9SETpnqlsM0bc1TQcMe5GrOAe4I0EkfQQMkEQghGsUVncHNYUsI3jR8tLVIIhBCNJorOoNF65POYiFELVdPF8gS0D5EagRCCAeXs/dbV/s0m1Hp/OxzBIT3khqBEMKuoqgQw89JAKyLu3n7sMvL2fs5u3YVl7P3u7ooLiM1AiEEgH1ClsM8AZXaq4ddXs7ej2HbVgD7RDR3HAnV2qRGIISwrxx64x8nb53UaAidMtWrm4XKj+bWe+wrJBEIIagoOG1f4x+ViuBBA+m2YKHXfzsOjIp2OPbvdpdPdpJL05AQPqz6fsLVN5DpNukpKkO6uLp4rc6W6MqP5uLf7S4uf76v1n4KvkASgRA+qqKokLN/+TOKyYRKq0U/6Wn7rmFBvXv5zCSs4NjhBMcOpyzz01s7n1VbSM8XNCsRZGRksHHjRkwmE9OnT2fKlCn2a6dOnSI5Odl+XFZWRseOHfn000/ZvXs3q1evJiQkBIDhw4czb9685hRFCNFE1n1/TYB1UbbKf/2TsKnTXVwq16m+rSYqNcbSUiqKCn0iGTidCIqLi1m7di27du3Cz8+PiRMnMmTIECIjrT+0Pn36kJ6eDkBFRQVPPPEES5cuBSA/P5/k5GQSElpn9yIhRP0qigq5+a9/uroYbqX6gnVXD3zFlS/3czXngE80ETndWZyTk8PQoUMJDg6mXbt2jBw5kqysrDrvfeedd7jvvvuIjrZ2zJw4cYLdu3czZswY5s+fz5UrV5wthhCiiWwjhCp/+P7WSY2WoGEPuq5QbiKgZyS6kBDrPIpqTUTezukagcFgQK/X249DQ0M5fvx4rfuuXbvGzp07ycjIsJ/T6/XMmDGDqKgo1qxZw2uvvcbq1aub9P717b9ZH72+g1PPuSOJxf14Qhxns7+3Nn/YRggNHEC3SU8R1LuXw32eEEtjNSUW/yFRlH36CRaTCbVWS+chUQS50c+iNX4vTicCi8WCSqWyHyuK4nBs88knnxAXF2fvDwBYv369/d8zZ85kxIgRTX5/2bxeYnE3nhKHJaIHKo0WBesIoQ7xY6gM6eJQdk+JpTGaHEtIF7r+/tYqqzV/Nq7UWpvXO50IwsPDyc29NfmipKSE0NDQWvft27eP2bNn24+vXbtGamoqzzzzDGBNIBqNxtliCCGayNOWk3YFX1tl1ek+gmHDhnHw4EHKysqoqKhg7969xMTEONyjKAonT55k8ODB9nPt2rVj8+bN5OXlAfDBBx84VSMQQjgvoGcknUYl+NSHnbg9p2sEYWFhzJs3j2nTpmE0GpkwYQIDBgxg1qxZzJ07l/79+1NWVoZOp8Pf39/+nEaj4Y033mDp0qXcvHmTu+++m5UrV7ZIMEII0ZpsE/C8rSalUhSlaQ3tbkL6CCQWd+MtcYDEUpeKokJ+XPlnMJtAo6XbguQ2Twat1Ucgaw0JIUQjXM05YE0CAGaT9dhLSCIQQggfJ4lACCEaIWjYg6i01m5Vlda7JuD51KJzubm57Nu3n+jo+xk4cHDDDwghxM8CekYSkZTslZ3FPpMI8vKOMXv2s1RWVqJWa1i06A+MH/+Uq4slRIvz1pEt7sBb5xf4TCLIzT1MVVUViqJgNptISVlOZOS/Sc1AeBXbOkK+uKa+K1zO3k/50VwCo6I9ehMfn+kjiI6+H7X6VrgWi4Xc3MOAtbawZcs75OUdc1XxhGgR1Xca85UF01zFtt/xjZP5GLZtpeTjna4uktN8JhEMHDiYFStWoNVqUavV6HR+REffT17eMZ5//lnWr3+T559/VpKB8Gi2NfVRq1FptF698byr1dzf+NJnezx2i0ufaRoCePrppwkPv4vc3MP2DuMtW97BaKzCYrFgMhnttYTq9wjhKWQdobYTGBXNjZP5t04oisfuauZTiQCsNYPqH+7R0fej0/lhMhnRanUEBwfz/PPPYjRWodP58e6770kyEG6vZgexJ34YeZrg2OEYSwxc+mwPKAoqnZ/H1sB8LhHUNHDgYN599z17DSA393CtGoIkAuHOpIPYdfQTniRwcJTH18B8PhFA7VpC9RpCdPT9Dvfm5R2TZiPhVq7mHEAxVgH43Kbr7sAbamCSCGqoWUOo/mFv61iWZiPhLiqKCrny96/sxyq12mObJ4TrSCKoQ80agk1dzUa281JDEK5QUXAaLGbrgUpF0IO/9vhvp6LtSSJogvo6lrVaLWPH/oYxY8ZJQhBtxjZcVDFbt530pvVvRNuRRNAE9XUsV1VVkZq6k4yMdBYsWMjly5elliBanQwXFS2hWYkgIyODjRs3YjKZmD59OlOmTHG4/vbbb5OamkpQUBAATz75JFOmTOHUqVMsXryY69evEx0dzbJly9BqPSMn1dWxXFVViaIoKIqC0VhFSspyLBaL9COINuENnZXCtZyeWVxcXMzatWv56KOPSEtLY8eOHRQWOs6qy8/PZ82aNaSnp5Oenm5PFElJSbz66qt89tlnKIrCzp2eOTXbVkMYP/5JdDo/NBoNKpUai8VSqx9BlrEQwvdUFBVSlvmp2884dvpreE5ODkOHDiU4OBiAkSNHkpWVxW9/+1v7Pfn5+bzzzjucO3eO++67j5dffpmLFy9y8+ZNBg0aBEBiYiLr1q1j8uTJzQzFNWw1hDFjxpGbe5jg4GBWrkxxGH4qo42E8D2eNL/D6URgMBjQ6/X249DQUI4fP24/vn79On369CEpKYnu3buTnJzMhg0bGD58uMNzer2e4uLiJr9/fftv1kev7+DUcw2Ji4shLi4GgOjoQRw8eJAHHniA6Oho3nrrLUwmo72WcOpUnv3e5mitWFzBW2LxljhAYmmus9nfo5hN9gUA1We/Rz+0+V8AWyMWpxOBxWJBpVLZjxVFcThu3749mzZtsh/PmDGDRYsWERMTU+9zjeXOm9d3796L7t17AVBSco0+fQai1eoA0Gp19Okz0F6OvLxjZGSkATRpxJFsLu5+vCUOkFhagiWiByqNFgXriC5LRI9ml6O1Nq93OhGEh4eTm3tr9b2SkhJCQ0Ptx+fPnycnJ4cJEyYA1g98rVZLeHg4JSUl9vsuXrzo8Jw3ut0ktby8Y8yaNZ2qKuus0NTU/2H48Id45pmZ0nQkhIfzpBFdTncWDxs2jIMHD1JWVkZFRQV79+4lJuZWc8cdd9zBX/7yF3788UcUReHDDz9kxIgRdO3aFX9/f77++msA0tPTHZ7zVgMHDua552Y7fMBbh58a7ceKYuGLLz5n5szpvPnmKl588TlSU3e4orhCiBYQ0DOSTqMS3DoJQDNqBGFhYcybN49p06ZhNBqZMGECAwYMYNasWcydO5f+/fvz2muv8eKLL2I0GomKiuLZZ58FYNWqVbzyyiuUl5fTt29fpk2b1mIBeRLrBDWdvUZgYzRW8d57mwE4ePAAmZkZzJ37e6klCCFahUpRlKY1tLsJd+4jaIq8vGNs3bqZ7OwvsFgsAKhUKmr+Wvz8/Ni06X17MnDHWJzlLbF4Sxwgsbgrt+sjEC1j4MDBrF273qHTuEOHDvYagY3RKEtiCyFahyQCN1HXQndbt26x1wx0utpLYgvvV3PDGSFagyQCN/XSS/MZPvyRRg8tlX0SvI8nTUgSnk0SgRu73XLYNcnMZe9UUXAaxWS0T0iSDWdEa3F6+KhwH7fbJ0F4NtsS06jVqDRa2XBGtBqpEXiBmvskSF+Cd/CkCUnCs0ki8AL1ba8pPJssMS3agiQCL9HY/gSQjmV3JqOEhCtIIvAx0rHsvmSUkHAV6Sz2MdKx7L7qGiUkfIOrN7CRGoGPkY5l91VzI3oZJeQb3KEmKInAxzSmY9nZPRJE8wUNe9D+/9Is5BvcYb6IJAIfVF/Hcl7eMWbOnI7RaF0RNS0tlXHjxktCaGWXs/dj+GgbWCyotDp7QhDezx1qgtJHIBzk5h7GZLq1R4LJZOLjj3cwY8ZU2RuhlVQUFWL4cBuYzdZvhSaj9A/4ENt8kV+MS3TZAAGpEQgH0dH3o9Xq7DUCG7PZRErKcs6e/ZHTp08RF/co48c/5aJSepeKgtNgMd86oVZL/4CPcfV8kWYlgoyMDDZu3IjJZGL69OlMmTLF4fq+fft46623UBSFiIgIUlJS6NixI7t372b16tWEhIQAMHz4cObNm9ecoogWMnDgYDZvfp+MjDRKSy+Snb0fy88fUmaz2WHDHECSQQsI6NUblc7P2k6sUhM6ear0D/gwV8wlcToRFBcXs3btWnbt2oWfnx8TJ05kyJAhREZaC15eXs7SpUtJTU0lLCyMN998k7feeotXXnmF/Px8kpOTSUhIaLFARMup3oeQmrqDlJTl9k1zqm+Ys2/fXkkELUCWkhA2rhpB5HQfQU5ODkOHDiU4OJh27doxcuRIsrKy7NeNRiNLliwhLCwMgF69enHhwgUATpw4we7duxkzZgzz58/nypUrzQxDtJbx459iy5ZtzJnzEtOnz3C4Fhf3qItK5X08ZW9b0bpcNZfE6RqBwWBAr9fbj0NDQzl+/Lj9+M4772TEiBEA3Lx5k3fffZepU6cCoNfrmTFjBlFRUaxZs4bXXnuN1atXO1sU0cqq1xAiIrqxb99e6SMQohW4agSR03sWb9y4kcrKSv7zP/8TgJ07d5Kfn89rr73mcN+1a9eYM2cOERER/OlPf6r1OleuXGHEiBEcPiwzXL1Fbm4uBw8e5IEHHiA6OtrVxRHCo1w9XcDV/JME9etLUO9ebfKeTtcIwsPDyc3NtR+XlJQQGhrqcI/BYOC5555j6NChLFq0CLAmhtTUVJ555hnA2uas0Wia/P7esnm9s9w1luprGanVanr37sNvfjOh3tqDu8bSVN4SB0gsLhXSBf/YLlRCrXK31ub1TvcRDBs2jIMHD1JWVkZFRQV79+4lJibGft1sNvPCCy8QHx/P4sWLUalUALRr147NmzeTl5cHwAcffGBvQhKez3EtIxP5+SdYvnwJiYmjZR6CEG7K6RpBWFgY8+bNY9q0aRiNRiZMmMCAAQOYNWsWc+fO5aeffuIf//gHZrOZzz77DIB+/fqxYsUK3njjDZYuXcrNmze5++67WblyZYsFJFzLtpZRZeVNh/NnzhSxfPkSQIacCuFunO4jcDVpGnLfWPLyjrF162a++OLzWtfuuacnYWHhDp3N7hxLU3hLHCCxuJPL2fspP5pLYFQ0v5wwplWahmRmsWhxAwcOZu3a9aSm7uDDD9/nzJkz9mtnzhRx5kyRTEgTohEuZ+/HsG0rADdO5hPY4Q60UUNb/H1krSHRasaPf4pduzL5wx+W8cADD3LPPfc4XN+3by9gHWW0Zcs75OUdc0UxhXBb5UdzHY7Lcg62yvtIjUC0uvHjn2L8+KdITd1h7ycA64S0vLxjzJ79LFVVjR9lJISvCIyK5sbJfPtxu3t6tMr7SCIQbcb24V59QtqWLe9gNBqxWCxYLBby80+Qn3+Cs2d/JDCwg+yrLHxacOxwjCUGLn22BxSFC+kZ6NsHExw7vEXfRxKBaFO22oGNdZSRDrPZ7HDf1q1/BRTUag3Tpj0jSUH4LE27dvDzmB7FbMbw0Tb8IyJadDkS6SMQLjVw4GB27NjBQw894nBeUSwoioLZbOK99zbz9ttvMHPmdOlHED4noFdvUFebdGuxtPgaRJqlS5cubdFXbCMVFVU0deBr+/b+3LhR1fCNHsCbYvnlL3vw61/HodfrMRqNjBgxkhMnjqMoFof7LBYzWq2WmJiHyMs7xt/+9gkajYbw8M4uKrmjun4nFUWFXPu/HFCr0XXq5KKSNZ03/X15eiy6Tp3QBHXk+skTAKi0OkLGPt6kvyeVSkW7dn63vS5NQ8JtVG82iojoRkrKckwmU637qi9jodVqGTv2N265laY7bEouvENw7HD8IyJQn/0eS0SPFv87kkQg3NL48U8RGflvZGSkkZaWitlsRqfTMWbMOIdlLKqqqkhN3UlGRjoLFizk8uXLbtOX4A6bkgvvEdAzEv3Qwa0yOU4SgXBbtuWvbR/+1T/gdTo/qqoqURQFRVEwGqv405+WoygWdDo/3n33PZcnA3fYlFyIxpBEINxe9f0QbMfvvvseGRlppKfvxmIxo1KpMJvNKIqCyWQkN/fwbRNBXt4x+1acISG/aLVmJdl5THgKSQTCI9WsLQQHB7NyZQomkxGtVkd09P11PpeXd4yZM6djNN7qPExLS2XcuPENJoS8vGO1aia3U33f2U6jZEtW4d4kEQiPVr22EBn5bw1+UOfmHsZkMjqcM5lM9n6G2zUpVe+g1un8mDz5aU6fPuWweN4HH3xAWtonjL33Pu7JzQWTCUWt5vivfkWvhx5pMMnUVUtpSvIRwlmSCITXqNmEVJfo6PvRanUONQKgwSYlxw7qSt57bzOAffE8wL58Rtj333N3125oVCosZjOnPtvD6u0f1JtkatZS0tN3k5y8mJUrU+zJp77nMzLSANxy9JRwf5IIhE8ZOHAwmze/b//2DfD3v3/18xyF2zcp2fZZMJmM9g5qG9vieTYnr13FZLGAWo3JYiH/6pUGk0ztWoqRffv2Vtvkp+7nayaRTz7ZzcsvL3ar0VPC/UkiED6nZs2hMc0vtg7q3NzDlJdfs9cIwLp4HtyqHXx3vZwVhd/SJzCQ/KtXKKq4gU7nV2+SqVlL0Wp1xMU9ytGjX9fb71EziVRVGd1u9JRwf81KBBkZGWzcuBGTycT06dOZMmWKw/VTp06xePFirl+/TnR0NMuWLUOr1XL+/HmSkpIoLS2lR48erFq1ivbt2zcrECGc1ZgmpZr3RUR0c1g8DyAw8A7S0j4hLu5Re39Fv+DgBr+d16ylVO8jaKjfo2YS0WjUWCwNj56yJb/y8mv2vo7q7xUXF1PrGeG9nN6hrLi4mEmTJrFr1y78/PyYOHEia9asITLy1hC5hIQE/vjHPzJo0CAWLVpEv379mDx5MrNnz2bs2LGMHj2a9evXc+PGDZKSkpr0/rJDmcTibkxH/4+fsv9OYFR0i68OWZ/qfQR9+vzKYfRUXTUCW8e3bR6GjVarxWKxoNVqefjhhwkM7Firz6Hme9VMcu7Yue0tf1/QepvXO10jyMnJYejQoQQHBwMwcuRIsrKy+O1vfwvAuXPnuHnzJoMGDQIgMTGRdevW8cQTT3DkyBHWr19vP//00083OREI4U5q7iQFtFkyqFmjaagWYev4rvkd0LacR1VVFVlZWYC103rz5vftI5hqdmqr1Wp7ExTgMLLKloSqJwfb+wfXqCnVlUDqeq7m/cHBwZw69Q97f09IyC/sCcp2LSDAj7i4UbdNTKmpO+y1O9vPrmb56rrXtsfG7t0fo9eH8swzM+tMuu6WGOvidCIwGAzo9Xr7cWhoKMePH7/tdb1eT3FxMZcuXSIwMBCtVutwXghPVnMnqfKjuW1aK6iuoaYuW8d3XTUC26Q8m+rNS3V1alfvyAZqdW7DreSg0WhRqcBotHa425LIggULa42Oqus5k8lU636LxXFhQhuVSuUQx44dO9i06f1aP5fqmyUdPHgAjUZrb1qrnuQGDhxc696jR3P5298y7K/11VfZbNmyzSGR1ZUY61Jf0quefF54YWadzzeX04nAYrGgUqnsx4qiOBzf7nrN+4Bax41RXzWnPnp9B6eec0cSi/swxf47Z6rtJBUe++9uG1NcXAw7d+7g4MGDXL16lZMnTzJq1Ch69+7Nxx9/zPbt2+21A51OR1zccPT6DsTFDeedd9ZTVVWzRmC9B2DTpo0YjUb7uYMHD2IyWTceUhRrErF9QNsSxpdf/q/9HpPJyKlTeQB1Plfz/tupWdsxGq2vW7Pv48sv/9fh2Gy+tchh9fLExcXUujcn5+8OxyaTyeE9Tp3KqxVXXX0vubm5zJ79LEajEY1G83M5rGtrzZgxgw0bNgDW5LN3byaLFi0iOjr6trE7w+lEEB4eTm7urW9BJSUlhIaGOlwvKSmxH1+8eJHQ0FA6derEtWvXMJvNaDSaWs81lvQRSCzuRBs1lHv+A3sfgTZqqFvH1L17L7p371Xr/O9/v5i4uFHs25dJRUUVY8aMo3v3XpSUXKN7915s2vR+nX0Ettd655337N9mu3fvxeXLN9BqdQCo1Rr7N3uLxYJarUar1RET8zD/93+HAOtoqT59Btr/Xf05s9nscL+iKI2uEeh01tet+TuJiXmY7Oxs+7FGo0VRLA7lsz1X895hw/7doUag1Wod3qNPn4H2GKq/Tk379u2nqqrKvksf3Epkn376N4d7Dx06xBNPPFFn7aY+rdZHMGzYMN566y3KysoICAhg7969LF++3H69a9eu+Pv78/XXX3PvvfeSnp5OTEwMOp2O6OhoMjMzGTNmDGlpacTEyAgF4fk6jxyBNmqoq4vRbAMHDiYuLqbOD62Gmp1uty5UQ30EdfVr1PVczfub20dQc/vU+voI6tpqNSoq+rZ9BDVjv93PrfocFY1Gg6Jgn9fyyCMjHIYqg7V2U99aWs5wetQQWIePvvOOdc/ZCRMmMGvWLGbNmsXcuXPp378/p0+f5pVXXqG8vJy+ffuSkpKCn58f586dIzk5mdLSUjp37syaNWvo2LFjk95bagQSi7vxljhAYmlr9fURvPnmKt57bwtg/bzz8/Nr8RpBsxKBK0kikFjcjbfEARKLu7EN221oBNTttFrTkBC+rPrqorK8tGhttia31kpqkgiEaCLZglJ4G7WrCyCEp3HYgtJkpKLgtKuLJESzSCIQoonU7QPB1rWmKNZjITyYJAIhmshyvfzWgUrleCyEB5JEIEQTBfTqjUrnB2o1Kq1ONqUXHk86i4VoItmUXngbSQRCOCGgZ6QkAOE1pGlIiEaqKCqkLPNTKooKXV0UIVqUT9UIrp4uoOzQUanOiyaTuQPCm/lMIqgoKqRw9UosRvkPWTSdw9wBs8naPyB/P8JL+EzTUEXBaSwmk30S0NWcA64ukvAgAb16o9LqrCOFNFoZKSS8is/UCAJ69UalVqNYLKAoXD3wFUHDHpRvdaJRZKSQ8GY+UyMI6BlJaNzD9mPFYpGlAUSTBPSMpNOoBEkCwuv4TCIACH1o+K2JQFK9F0IIwIeahgCCeveS6r0QQtTgdCI4f8fJ1rgAABQjSURBVP48SUlJlJaW0qNHD1atWkX79u0d7jEYDCxcuJCLFy+iVqtZsGABDzzwAEajkSFDhtCtWzf7vbt27bJv3NyaZCKQEEI4crppaNmyZUyePJmsrCz69evHhg0bat2zcuVKHn74YdLT01m9ejXz58/HbDZTUFDA4MGDSU9Pt/+vLZKAEEKI2pxKBEajkSNHjjBy5EgAEhMTycrKqnXfiBEjSEhIAKB79+5UVlZy48YNTpw4QVlZGYmJiTz55JMcPny4GSEIIYRoDqeahi5dukRgYCBarfVxvV5PcXFxrftsiQJgy5Yt9OnThw4dOqBSqXjkkUeYPXs23333HbNmzSIjI4NOnTo1ugz17b9ZH72+g1PPuSOJxf14Sxwgsbir1oilwUSwZ88eUlJSHM51794dlUrlcK7mcXVbt25lx44dfPDBBwBMnDjRfu1Xv/oVAwYM4OjRo8TFxTW64LJ5vcTibrwlDpBY3JWzsTR78/r4+Hji4+Mdztk6e81mMxqNhpKSEkJDQ+t8fuXKlWRnZ/Phhx8SHh4OQFpaGlFRUdx1110AKIqCTqdrdFAtTTYiF0L4Mqf6CHQ6HdHR0WRmZgLWD/aYmJha923dupVDhw6xfft2exIAKCgo4K9//SsAZ86c4dSpU9x7773OFKXZbIuJXdydytlVK2VlSSGEz3F6+OiSJUtITk5m48aNdO7cmTVr1gCwfft2DAYDc+fOZf369QQGBjJ16lT7c++++y5z5sxh0aJFJCQkoFKpeP311wkMdM2+r7KYmBDC1zmdCLp27cq2bdtqnZ80aZL930eOHLnt8+vWrXP2rVuUbTExxWyS2cZCCJ/kUzOL6yKLiQnpIxK+zucTAchsY18mG84I4WOLzjWWbEnoO+rqIxLC10iNoAb5huhbpI9ICEkEtcgoIt8ifURCSCKoRb4h+o7qncSdRiW4ujhCuIwkghrkG6JvuJy9H8OH28BiRqXzkyZA4dMkEdRBRhF5t4qiQgwfWZMAgGIyShOg8Gkyakj4nKs5B8BsvnVCpZYmQOHTJBEIn1JRVMiVv39164RaTeiUqVIbED5NEoHwKRUFp+1NQqhUdPx1LMGxw11aJiFcTRKB8Cm2UWGo1ai0OoKGPejqIgnhctJZLHyKjAoTojZJBMLnyKgwIRxJ05AQQvg4p2sE58+fJykpidLSUnr06MGqVato3769wz3nzp0jISHBviXlL37xC7Zs2UJVVRWLFy8mPz+fO+64g1WrVtGzZ8/mRdJGZMliIYS3cbpGsGzZMiZPnkxWVhb9+vVjw4YNte7Jz89nzJgxpKenk56ezpYtWwDYtm0bAQEB7Nmzh0WLFrFw4ULnI2hDsq2lEMIbOZUIjEYjR44cYeTIkQAkJiaSlZVV674TJ07w7bff8vjjjzNt2jQKCgoA2L9/P2PHjgXgvvvuo6ysjPPnzzsbQ5uRJYuFEN7IqURw6dIlAgMD0WqtLUt6vZ7i4uJa9/n7+zN27Fh2797Nc889x5w5c6iqqsJgMKDX6+336fV6fvrpJydDaDsOQw9lQTohhJdosI9gz549pKSkOJzr3r07KpXK4VzNY4Df/e539n/HxsayevVqzpw5g6IoDvcrioJa3bScFBLi3Gb3en0Hp56zPjyY4D8u5Wr+SYL69SWody/nX6sFNCsWN+MtsXhLHCCxuKvWiKXBRBAfH098fLzDOaPRyJAhQzCbzWg0GkpKSggNDa317LZt20hISODOO+8ErB/4Wq2WsLAwDAaDvRP54sWLdT5fn9LSciwWpUnP6PUdKCm51qRnagnpgn9sFyqh+a/VDC0Si5vwlli8JQ6QWNyVs7Go1ap6vzw71TSk0+mIjo4mMzMTgLS0NGJiYmrdd+TIET7++GMADh8+jMVi4Z577iE2Npb09HQAcnNz8ff3p0uXLs4URQghRDOpFEVp2tfqn507d47k5GRKS0vp3Lkza9asoWPHjmzfvh2DwcBLL71EcXExycnJlJSU4O/vz4oVK+jduzeVlZW8+uqr5Ofn4+fnxx//+Ef69u3bpPd3WY3ATUgs7sdb4gCJxV21Vo3A6UTgau6SCFw1r0D+uN2Pt8QBEou7aq1EIEtMNINsdC+E8AayxEQzOMwrMBkp/SRNJpm5UEVRIWWZn8rvQIgmkhpBM9g3uv85Gdw4mU/Ft99KzcAFpHYmhPOkRtAMtiWN2/3qVke3zDh2DZn1LYTzJBE0U0DPSELGjkOl85MZxy4ks76FcJ40DbWAmpudAJRlfiorlLYh2XBGCOdJImghts1OpK26bdUcvis/ayGaThJBC6urrVo+nFqHJF0hWob0EbQwaatuGxVFhZR+koZirJIOYiGaSWoELUzaqltf9ZoAACqVJF0hmkESQSuQturWVb35DZWKdr/qS8jYcfIzF8JJ0jQkPI5D85tWJ0lAiGaSGoHwONL8JkTLkkTQRly1Sqm3kuY3IVqOJII2IMMchRDuTPoI2oCsg9M8sqqoEK3L6RrB+fPnSUpKorS0lB49erBq1Srat2/vcM8LL7zAhQsXALBYLHz77bd8/PHH9O7dmyFDhtCtWzf7vbt27UKj0ThbHLdmX6XUbJJhjk1UUVTI2b/8GcVkQqXVEpGULLUpIVqY04lg2bJlTJ48mdGjR7N+/Xo2bNhAUlKSwz3/9V//Zf/3m2++yaBBg+jfvz/5+fkMHjyYLVu2OF9yDyKdm867mnMAxWQCQDGZuJpzQH5+QrQwp5qGjEYjR44cYeTIkQAkJiaSlZV12/vPnDlDWloaL7/8MgAnTpygrKyMxMREnnzySQ4fPuxMMTxKQM9IOo1KkA+xJqgoKuTmv/7p6mII4fWcqhFcunSJwMBAtFrr43q9nuLi4tvev2HDBp577jkCA617ZqpUKh555BFmz57Nd999x6xZs8jIyKBTp07OFEd4ocvZ+zF8tA3M5lsnNVqChj3oukIJ4aUa3Lx+z549pKSkOJzr3r07//rXv8jOzgbAZDIxePBgTpw4Uev5K1eu8Nhjj7F//378/f3rfI8XX3yR8ePHExcX52wcwotcPV1A/qI/oFRLAsGDBtJt0lME9e7lwpIJ4Z0arBHEx8cTHx/vcM5oNDJkyBDMZjMajYaSkhJCQ0PrfD47O5uYmBiHJJCWlkZUVBR33XUXAIqioNPpmlTw0tJyLJZ6c1gten0HSkquNemZ1ubs/AJ3jMVZNWMpO3TUIQmg0dAhfgyVIV3cOmZv/p14MokF1GoVISGBt7/uTGF0Oh3R0dFkZmYC1g/2mJiYOu/95ptviI6OdjhXUFDAX//6V8Daf3Dq1CnuvfdeZ4ri0WzzCy7uTuXHlSlczt7v6iK5lG2YqLp9oHXHN5UK1BpCJ0+VvhUhWpHTo4aWLFlCcnIyGzdupHPnzqxZswaA7du3YzAYeOmllwD48ccfGT58uMOzc+bMYdGiRSQkJKBSqXj99dft/Qe+xGHxNLMZwwfvAxAcO9y1BXOBmpPu9BMnY7leLqOshGgDTieCrl27sm3btlrnJ02a5HC8adOmWvcEBgaybt06Z9/aawT06g1q9a0OUUXB8NE2/CMifO7Dz2HSncmI5Xo5nUYluLpYQvgEmVnsQgE9IwmdPNXaBGJjNnM154DrCuUCV08XUHHmjLVmBKAoqNv7Xg1RCFeRROBiwbHDCX16urVm8LMrf//KZ5ZTqCgq5OQflnL9m6O3TqpUWK6Xu65QQvgYSQRuIDh2OB1/HXvrhGLhas4Bn1hfp6LgNJafZw4D1t3GtDpZhkOINiSrj7qJoGEPWpdTMJtQqdVc+epLsJhBpSb06Wle24Ec0Ks3aq0Wi8kad9CDvyZo2IM+10cihCtJInAT1dcjqjhz5lZTiWLx6tFEAT0j6bt8KRcOHZURQkK4iCQCN2LbbKV42/uOF7xoNNHl7P2UH80lMCrantiCeveiMqSLawsmhA+TPgI3FDTsQai5JLfF4vH7GFzO3o9h21ZunMzHsG2rz0+gE8JdSCJwQwE9I+m2YCHtB0VZRxN5SQdq+dHceo+FEK4hTUNuKqBnJF1/O9cr9jq2NQdpAjs4nA+Mir7NE0KItiSJwM158ibtFUWFlO3JdJgj0GHIA5jLrzn0EQghXEsSgWgV1beYrM5cfo2IefNdVCohRF2kj0C0uIqiQgz/76NaSQCkOUgIdyQ1Ai/hLn0J9lVEjVUO5zV33klIwuPSHCSEG5JE4AVqLuEcMX+By5KBfRXR6jRaurwwx2P7OoTwdpIIvIDDEs5mk7Vm4KIP3YBevVFpdfalMmTJCCHcnyQCL+Dw4avRunS+QfWlMlzdTCWEaJxmJ4I33ngDjUbD7373u1rXqqqqWLx4Mfn5+dxxxx2sWrWKnj17oigKK1eu5IsvvkCtVrN8+XKf3KqypbTlh29dS0TUVR5JAEJ4DqcTwbVr10hJSeFvf/sbM2fOrPOebdu2ERAQwJ49ezhy5AgLFy5k586dfPbZZxQVFZGZmck///lPZs+eTWZmJlqtVFCc1ZQP34Y6liuKCu2b4/jf1Z3Kf/0TAHVAAJeyrPtU3ziZD3jnQnhC+BqnP3k///xz7r77bp599tnb3rN//3773sX33XcfZWVlnD9/nuzsbEaNGoVaraZHjx507tyZY8eOcd999zlbHNFIDXUsVxQV8uPKP4O59tBPh53UsC4RIYlACM/n9DyCcePG8fzzz6OpuThaNQaDAb1ebz/W6/X89NNPGAwGQkNDa50Xra+ujuWa1+tMAnBrK8mfyZwAIbxDgzWCPXv2kJKS4nDunnvuYevWrQ2+uKIoqKp9i1QUBbVajcViqfN8U4SEOLenrV7foeGbPIQzsfgPiaLs00+wmEyotVo6D4kiqNrr+A+JovSTtDong6l0OjqPGc2NM9/TadgDdB45olnlr85bfi/eEgdILO6qNWJpMBHEx8cTHx/v1IuHhYVhMBi46667ALh48SKhoaGEh4djMBjs99nON0VpaTkWi9LwjdXo9R0oKbnWpGfcldOxhHSh6+9vdSxXhnRxfJ2QLkQkJdfZR2AbBmpLwS31s/SW34u3xAESi7tyNha1WlXvl+dW7Z2NjY0lPT2d6OhocnNz8ff3p0uXLsTExJCamkpCQgJnz57lhx9+oH///q1ZFFFNQx3LMupHCN/S4olg+/btGAwGXnrpJaZOncqrr77K6NGj8fPzY+XKlQA89thjHD9+nLFjxwKwYsUK7rjjjpYuihBCiEZQKYrStPYVNyFNQxKLu/GWOEBicVet1TQkq48KIYSPk0QghBA+ThKBEEL4OI9d00GtVjV8Uws+544kFvfjLXGAxOKunImloWc8trNYCCFEy5CmISGE8HGSCIQQwsdJIhBCCB8niUAIIXycJAIhhPBxkgiEEMLHSSIQQggfJ4lACCF8nCQCIYTwcT6VCHJzc0lMTGTMmDG88MILXLlyxdVFctrXX3/NhAkTePzxx5k+fTrnzp1zdZGa7Y033uCtt95ydTGckpGRwahRo3j00Uf58MMPXV2cZisvL7dvHOXJ3n77bUaPHs3o0aPt+6F4qjfffJNRo0YxevRo3nvvvZZ9ccWHxMXFKd99952iKIryl7/8RVm9erWLS+S8hx56SDl16pSiKIryP//zP8oLL7zg4hI57+rVq8rChQuVAQMGKOvWrXN1cZrsp59+Uh566CHl0qVLyvXr15UxY8bY/8480TfffKMkJCQoffv2VX788UdXF8dpBw4cUJ566imlsrJSqaqqUqZNm6bs3bvX1cVyyqFDh5SJEycqRqNRqaioUB566CGlqKioxV7fp2oEmZmZREZGYjQaKS4uJigoyNVFckpVVRUvvfQSvXv3BqBXr15cuHDBxaVy3ueff87dd9/Ns88+6+qiOCUnJ4ehQ4cSHBxMu3btGDlyJFlZWa4ultN27tzJkiVLmryPuLvR6/UkJyfj5+eHTqejZ8+enD9/3tXFcsr999/Pf//3f6PVaiktLcVsNtOuXbsWe32fSgQ6nY6CggJiY2M5dOgQo0ePdnWRnOLn58fjjz8OgMVi4e233yYuLs7FpXLeuHHjeP7559FoNK4uilMMBgN6vd5+HBoaSnFxsQtL1DwrVqwgOjra1cVotl/+8pcMGjQIgB9++IE9e/YQGxvr4lI5T6fTsW7dOkaPHs0DDzxAWFhYi722VyaCPXv2EBMT4/C/Z555BrB+e87JyeE//uM/mDdvnmsL2gj1xVJVVcX8+fMxmUzMnj3btQVthPpi8WQWiwWV6tYyv4qiOBwL1/ruu++YMWMGCxYs4O6773Z1cZpl7ty5HDx4kAsXLrBz584We12P3Y+gPvHx8cTHxzucq6ysZN++ffZvzmPHjuX11193RfGapK5YAK5fv86LL75IcHAwGzduRKfTuaB0TXO7WDxdeHg4ubm59uOSkhKPb1bxFl9//TVz585l0aJFHtsCAFBUVERVVRV9+vQhICCARx99lIKCghZ7fa+sEdRFq9WybNky8vPzAeu306ioKBeXynlJSUl0796dN954Az8/P1cXx6cNGzaMgwcPUlZWRkVFBXv37iUmJsbVxfJ5Fy5cYM6cOaxatcqjkwDA2bNneeWVV6iqqqKqqorPP/+ce++9t8Ve3ytrBHXRaDSsXbuWV199FbPZTFhYGCtWrHB1sZzyj3/8g88//5zIyEh+85vfANZ26U2bNrm4ZL4pLCyMefPmMW3aNIxGIxMmTGDAgAGuLpbP27JlC5WVlfz5z3+2n5s4cSKTJk1yYamcExsby/Hjxxk3bhwajYZHH320RZOb7FAmhBA+zmeahoQQQtRNEoEQQvg4SQRCCOHjJBEIIYSPk0QghBA+ThKBEEL4OEkEQgjh4yQRCCGEj/v/AfDpDZoNKyNtAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "y_pred = net(x_test).detach()\n", "plt.plot(x_test.numpy(), y_pred.numpy(), 'k.');\n", "plt.plot(x_test.numpy(), y_test.numpy(), 'r.');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, our prediction is still a smooth (and differentiable) function of $x$, by construction!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Training Loop" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Train the network for 200 \"epochs\" (one full pass through the training data) using [stochastic gradient descent (SGD)](https://pytorch.org/docs/stable/optim.html#torch.optim.SGD) to update the parameters after each epoch. The parameter update rule is:\n", "$$\n", "\\theta_k \\rightarrow \\theta_k - \\eta \\dot{\\theta}_k\n", "$$\n", "but with a new [momentum hyperparameter](http://ruder.io/optimizing-gradient-descent/) $\\rho$ which smooths out the estimated gradient over many cycles:\n", "$$\n", "\\dot{\\theta}_k \\rightarrow \\frac{\\partial\\ell}{\\partial\\theta_k} + \\rho \\dot{\\theta}_k \\; .\n", "$$\n", "With $\\rho = 0$, this reduces to the simpler SGD rule we used earlier." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "optimizer = torch.optim.SGD(net.parameters(), lr=0.1, momentum=0.9)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is the standard PyTorch training and test loop that we will build on below:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAELCAYAAAAoUKpTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfVzV9f3/8cfncw4gCIkoFyIqKl601LQsr7VLtZLsQgvtJzrntu/KvptrWSt3c13Ymrba1oX6XWuu5ipTy+zCWJdeUWmiZgsRFVHBI4gS15zz+bx/fxzOEZAjoJwL4HX/Jz4HOOflhxNP3teaUkohhBBCnIfu7wKEEEIEPgkLIYQQjZKwEEII0SgJCyGEEI2SsBBCCNEoCQshhBCNkrAQQgjRKKu/CzifoqIilixZQlhYGBMmTOCGG27wd0lCCNEuBXTL4rXXXmP27Nk88cQTrFmzxt/lCCFEuxXQLYvCwkLi4uIu+PtPny7DNJu/QL1Ll3BOnSq94Nf1Fqmr+QK1NqmreaSu5mtubbqu0blzR4+fD+iwiIuLo6CggJiYmAv6ftNUFxQWru8NRFJX8wVqbVJX80hdzdeStQV0WEyfPp2lS5cSFBRESkqKv8sRQoh2yy9hUVpaSkpKCitWrCAhIQGAjRs3snz5chwOB7Nnz+aee+4hJiaGZ555xh8lCiGEqEXz9a6ze/bsYdGiRRw+fJhNmzaRkJCAzWZjxowZrF+/nuDgYFJSUnj22WdJSkryZWlCiDbCNE2OHj1KWVkZsq92XZoGHTt2pEePHuh60+c4+bxlsWbNGhYvXszChQvdj23fvp2RI0cSGRkJwKRJk9i0aRPz58+/qNc6dar0gvrsoqMjKCgouajX9gapq/kCtTapq3maW1dJyRkcDoOuXbujad6b9Gm16jgcptee/2J4qk0pkzNnCsnJOU5ERKT7cV3X6NIl3OPz+Xzq7JIlSxg+fHidx06ePEl0dLT7OiYmBpvN5uvShBBtREVFKRERkV4NitZK03QiIjpTUdG8WVwBcSdN00TTNPe1UqrOtS9lHy/mrU+yyD5e7JfXF0JcPNM0sFgCev6OX1ksVkzTaNb3BMTdjIuLY+fOne7ri5kuezGyjxez7PUMDMPEYtF5cMYwkrp38nkdQoiL568/OFuDC7k3AdGyGD16NOnp6RQVFVFRUUFaWhrjx4/3eR37c0/jMExMBYZhsj/3tM9rEEKIQBQQLYvY2FgWLFhAamoqdrudadOmMWTIEJ/XMaBnZ6wW3d2yGNCzs89rEEK0PX/60x/59ts9OBx2jh07SmJiHwCmT0/hhRf+TGysc6cKw3Bgt9u5995fMn78Ne7vX7v2DV544c+sW/ceXbp0dT8+bVoyzz+/km7d4hk7djgPPbSI5OTb3J+fP/9nzJ37M664ou448YXwW1h8+umnda6Tk5NJTk72UzVOSd078eCMYRw7VU5ClzDpghKiHco+Xsz+3NMM6Nm5xX4HPPDAQwDk5+dx//0/Z9WqfwPwwQcbGTt2PI8++nv3127e/DnLlj1VJyw++GAj48Zdw/vvv0tq6lyPr7Ny5QtcffVId/i0pIDohgokSd07Mf36/hIUQrRDrnHL9ZsPsez1DL9MdDlxIp9LLrnkbE3ZB/jhhx+4557ZbNz4Dqbpearu9Okz+OMfn/RKXRIWQghRwzVuqXw4brl162bmzJnJ9OlTufXWSezf/z1/+MOf3J9///13ue66Gxg48FIsFgtffZXu8bnuuWc2xcXFbNz4TovXKWEhhBA1XOOWuobPxi3Hjh3PqlX/ZtWq1SQk9KBHj5707NkLAIfDwX/+8yE33DAJgGuvvYENG9Z5fC6r1cqjjy5m5coXsNlOtGidATHALYQQgcA1btnSYxZN0bFjOIsWPUZq6t0MH341gwYNYdu2zZSUlPDIIw8CzvA4fbqIkydtxMTENvg8ffokMX36DJ566vEWrU9aFkIIUUtS907cMirRL+OW8fHdufPOu/nLX55BKcX772/kpz/9BWvXbmTt2o28886HDB58eaPdTK7uqO+++7bFapOwEEKIADJr1o/Jz89j06b3ycjYyZQpt9X5fErK/+O99zZgGJ5XYFutVn73u8datC6f7zrrS7KRoG8Eal0QuLVJXc3T3LpOnDhCXFwvL1bk1Bo3EnSpf48CbiNBIYQQrY+EhRBCiEZJWAghhGiUhIUQQohGyTqLegxbNllffce3JVHEXzqE3tYCHHmZWOMHYomVY16FEO2ThEUthi2bso1/RDPsXIaFDd9dTUzHnWjKQZVmZXdiKvGXDpF9o4QQ7Y6ERS2OvEwwHVg0QJkMsuaA6QAUpung+He7WL3bkEORhBDN0tQtyl0efPARLrtsEJ999jGvvbYKwzBQymTy5FuYOTOVr75KZ/ny5wE4fvwoUVFdCA0No1u3eP7wh2e88m+QsKjFGj+QKt2KYTgw0NnnSGRgSCGm6bw+YI9zby4mYSFE22TYslu867k5W5S7FBSc5IUX/swrr/yLTp0iKS8vZ/78n9GzZy/Gjp3AiBGjgJY9s+J8JCxqscQm0TH5IX7I+Y7vSqIYf+kQOlqvx5a5m1W7THKNrnIokhBtmGHLpvy9pWDaqdaDCJuy0G9jlWfOnMHhcFBZWUmnThAWFsaiRb8nODjEL/VIWNRjiU2i/6BhdC4oIft4MZsOWRmQdCN3JeGXzcWEEL7j7Iq2g1JgOnDkZXo9LFxblLsEBQXxt7/9k379+jNu3ATuumsq/fsPYNiw4dx442QSEnp4tR5PJCw8cB2C4jBMrBadB2cM45ZRif4uSwjhRdb4gVTrQc6xSt2KNX6g11/TUzcUwG9+81tmz/4JX3/9JV9/nc7Pf/5jFi9+ggkTrvN6XfVJWDSg8th+ynd+QgLBHFbRMk4hRDthiU0ibMrCgJguv337Vioqyrn++onccsut3HLLrbz77tu8994Gv4SFLMqrx7Blk7/69yTaPuXeiDR6WwtknEKIdsQSm0TIsCl+X1fVoUMHVqx4kfz8PACUUhw4kEW/fgP8Uo+0LOpx5GWiDAcaiiBNMbVfNWHDG54q641ZE0KI9qX+mAXA3XfP5KabpjB37k9ZuPBXOBwOAEaMGMWcOfP8UaZsUV6fYcum4v2lKMPZZ+lpNkTtWRP4aNZEW9k+2pcCtTapq3lki/Lma+ktyqVlUY8lNolu9/yewv/uOm+LofasCdOwY8vcTby0LoQQbZSERQM6JAwgJCTe4+ezjxeTd6oTl2tWVM2CvVW7TO5KKpZBcCFEmyRh0Uy1p9QmWm8gyXKCA444co2uMmNKiACilELTNH+XEZAuZPRBwqIZDFs25TvT3VNqc+zR5BoxKKVkxpQQAUTXLRiGA6s1yN+lBCTDcKDrlmZ9j4RFE7kGtBMNO/dG6LxUMpFjxDLjhn6UVdhlZbcQASQ0NJySkjNERnZB02SFQG1KmZSUnCY01PNgdkMkLJrINaDtnFJrkhqfBUMG0GtQd3+XJoSoJzy8E6dPF2CzHQO8N+FT13VMMzBnQ3muTSM4uAPh4c3741bCoonc2wAYzsCIKj2E+vJFNp+QMy6ECDSaphEVFeP11wnUqcbQ8rVJ+6yJXNsAWLpfBmiAwjScZ1wsez2D7OPF/i5RCCG8RloW51F7hTY4u6KsfYZjnMjCNOznnHHhOoL1RFAC+0oiZRxDCNFmSFh4UGdfe61m1oAyQA8iZPRMCk8W1jnjYlDEGcrfewll2AlXOrtKJvLutlg5VU8I0SZIWHhQZ197ZeAaJDMNO4UnC4mfMI27kordZ1zEndxCdc0AuAWTvtYTHKmKlrUXQog2QcLCA9eAtjIcmGhoGijTPGe1tisIDOtAqjQrSjlXdB90xMnaCyFEmxHwYfHf//6XpUuXsmrVKp++riU2iaIR9/Llp5vJqo4FjfOu1j5WUEpmRW+Ugp2OvvQadDkzBneTVoUQok0I6LA4evQon3/+ORZL81YatpR9JZGkVQxCKef8p4ZWaxu2bKqzthGZuZkRwQYGFo4ZUQyt2kFv6yhAwkII0foFVFi8/PLLbN261X39yiuvcO+99/Lzn//cL/UM6NkZq0XHMEwsFv2c1druQXCjGg3QNUAZTAv7Ct0G5e9tIWzKQg47ouX8biFEqxZQYTFv3jzmzfPPwR4NSereiQdnDPP4i949CF5DAWg6OgoN59blR7an8c1RB1nVsTI7SgjRagVUWASi2oPY9dUeBHco+LoqiTzVhTs77sSsGRiPPLmTSSEmN4RYeKlkosyOEkK0Sj5ZwV1aWsqUKVM4duyY+7GNGzdy8803M3HiRFavXn3e71+5cqW3S7wgrlXdObHX8mLJJNaUj2R7ZT/eDb2dDyuH8lVVEjomFg0smPQPtsnsKCFEq+T1lsWePXtYtGgROTk57sdsNhvPPfcc69evJzg4mJSUFEaMGEFSUsueNHe+IwIbEx0d0cQvHMaZ8F4cz96GrplYrTo/GjGCv20Io7uycXXIQXRMNIuVG2+dyIChCRdcU7Pq8rFArQsCtzapq3mkruZrydq8HhZr1qxh8eLFLFy40P3Y9u3bGTlyJJGRkQBMmjSJTZs2MX/+/BZ97Qs5gxuavwFXl45B/Cal7thGp5rr0ogBxNmPuY9ovZiNvQJ107JArQsCtzapq3mkruZrbm1+P4N7yZIl5zx28uRJoqOj3dcxMTHs3bvX26V4Vf2xjbrXQ/1TlBBCtBC/7Dprmmad4w7l+EMhhAhsfgmLuLg4CgoK3NcFBQXExHh/73lfMmzZVGW8h2HL9ncpQghx0fwSFqNHjyY9PZ2ioiIqKipIS0tj/Pjx/ijFK1yL9ap3rqNs4x/Z/PEWOe9CCNGq+WWdRWxsLAsWLCA1NRW73c60adMYMmSIP0rxito71pqm84Ck1bsNWZAnhGi1fBYWn376aZ3r5ORkkpOTffXyPuVarOc6IKnUDOGaoD3kfW8hqfs4wNn6sGXuJtseK8eyCiECnqzg9gLXYj1b5m427T7FHWFfY8FAO7yXPf/+huKgOC4v/oQww85lWFj57STuumuSBIYQImBJWHiJJTaJ+NgkJrOWoP2m89Ru0yCxZDc1V1icD9Jbz5dtQIQQAc0vA9ztSezAoWiWIMyaa10DDYVCw1AaBjqHzW6yDYgQIqBJy8LLXF1SBRmfEXwkHV0pDHTeqbiKMf0v4WRwD+6SMQshRICTsPABS2wScZOTOLLvKvK/y6CwQ08mDB9OUvdODPR3cUII0QQSFj7Ua9BQeg2SrT+EEK2PjFkIIYRolISFH8mWIEKI1kK6ofzEfX63aadKs7I7MVUW5wkhApa0LPykzpYghnNLkGWvZ8geUkKIgCRh4SfW+IGgB2HiXGtxwB6HYZjszz3t79KEEOIc0g3lJ7W3BFm1yyTX6IrFosviPCFEQJKw8CPXliB3JRWT9/1ekoJsxFoLABm3EEIEFgkLHzNs2TjyMt1ncgP0/CGD2MOvgTIpy/qQ/MhhdLh0rKzJEEIEDAkLH6o9A6paDyJsykIAqra+hlIGGoBpEFe0E8e2DHacnsdV40a5A6byR1dASLxf/w1CiPZJwsKHas+AUoaD775Mp2tkKOE1QaGU8+t0DSzK5OCuHcR0DiXqq5fAtJOf8S6htyx0t0iEEMJXZDaUD7lmQCl07Epjw4FgVu0yMTQrhgIDMNAxFCigxAwm+5uvUcbZgHHkZfr7nyGEaIekZeFDrhlQ332ZzoYDwRx2RKMbsC1mBmVHviOrOpZultNMC/sKHcUdYTt4u+gqhobpBGkmSrNwIiiBXv7+hwgh2h0JCx+zxCYRNjyaYwcy0DUTi0Vn4PBhMHw44bmnGVa9E30/aDi7osK0Kl4qmUi/IBsH7LFY/pPFnFPZxA4cKt1RQgifkbDwg6TunXhwxjD2555mQM/O7i0+krp3wrA5KM9OQxkODDQOOuI4YsaAHYYHH2RESDaW/YqSrE0Uj5ovM6aEED4hYeEnSd07NbgPlKurypGXyemgBK4oiWSyI58+/03DinMgXAMM0+DLTzdj79xb9pMSQnidhEUAssQmYYlNohfQC6jK2EeV5jq5G0zlHAjPqo4lXM7uFkL4QLNmQ1VXV5OXl+etWoQH1viBaJYg0HSUZiG9uj8vlUzkGLGyPYgQwicaDYv//Oc/PPHEE5SWljJ58mSmTp3KP//5T1/UJmq4u6aGTOXF0kmsKRvJETOGGTf0k1aFEMInGg2LlStXctddd5GWlsbQoUP57LPP2LBhgy9qE7VYYpPYE3o1B6u7Oh9QirIKu3+LEkK0G42GhVKKAQMGsH37dsaPH094eDjKtdRY+NTgvl2xWnTnCm/ZoVYI4UONDnDrus4HH3zA1q1beeihh/jiiy/QNM0XtYl6BiZG1Zly29taQFXGljqbEgohhDc0GhYPPfQQL7zwAr/+9a+Jjo5m+fLlLFq0yBe1iQa4ptw2tCmhBIYQwlsaDYvhw4ezatUqwDkb6tlnnyU+XnY+9be6x7LasWXuJl7CQgjhJTIbKoAZtmyqMt7DsGWf8zlr/ECUZsVQGg6ls2qXyee7j/N+eo6c4y2EaHEyGypAubqZqneuo/y9pecEhiU2id2JqXxYOZQXSyZiGIoTX6xj17avWPZ6hgSGEKJFyWyoANXQ2ReZOUV1vib+0iF8br8cDbg3Io3JHXZxb0QaCdjYn3vaP4ULIdqkRsPCNRtqy5YtjBkzRmZD+UhDZ188umIb2ceL3d1Tva0FPDhjGFP7VROkmVg0sGDSP9gm02qFEC2qybOhHnjgAZkN5UMNnn2hmeR9v5fYI6+5Z0H1HD0TI8rEUWBBmSboOiOvG08vWdkthGhBTZ4Ndfz4cY4cOcIbb7zhi7rYsWMHb731Fkoprr76aqZPn+6T1w0k9c++sFp1koJs7u4pDDtVW18DTBQ6+Z2voMOlY2XbciFEi2s0LHJycrjvvvs4efIkpmnSuXNnVq5cSd++fb1a2A8//MDjjz9OcHAw9957b7sMC6h79sXIId2JLD1CeXYamA5AA2UCClOZ7Moz+PzIGR7sXCx7RgkhWlSjYfHEE08wb948br/9dgDWrVvHY489xquvvtqihbz88sts3brVff3KK6+glOKZZ54hNTW1RV+rtXEtxIuOjqCgIMh93oXWIZyq7f/GNOwY6Bywx2EYpntwu/7hSkIIcaEaDYtTp065gwLgzjvvdC/Sa0nz5s1j3rx57usffviBP/zhD8ycOZPBgwe3+Ou1Zq7zLrKPF5PXy0JM9VHW77eSa3TFYtGJceTzzbp3yaqO5d1tsTw4Y5gEhhDiojQaFoZhcObMGSIjIwEoKipq5DtaxpNPPsmJEyf45z//Sbdu3XjggQd88rqtRfbxYpa9noHDMLFa4plxQz8ur7AzKOIMndJfoF+Ig4khGp9X/ojynUcxrKNkOxAhxAXTVCOLJt58801eeeUVbrrpJjRN44MPPmD27NnMnDmzSS9QWlpKSkoKK1asICEhAYCNGzeyfPlyHA4Hs2fP5p577rn4f0k789YnWfzrw+8xFega/L+bLmX69f05vW09RZ+vdp6qV/OTVRoozYp5428YcNVVfq1bCNE6NdqyuPvuu+nZsydbt27FNE0WL17M6NGjm/Tke/bsYdGiReTk5Lgfs9lsPPfcc6xfv57g4GBSUlIYMWIESUkt/1fvqVOlmGbzFxA6xwZKWryei1W7roQuYVgsOhgmFotOQpcwCgpKMDr1RtMsKGWgac7A0HGe2Z32bhpFQd1avEsqUO8XBG5tUlfzSF3N19zadF2jS5dwz59vypOMGjWKBx98kIceeojRo0fzpz/9qUkvvmbNGhYvXkxMTIz7se3btzNy5EgiIyMJCwtj0qRJbNq0qUnPJ85yzZK6fXyfOmMSltgkQsbOcgZGzdfWPrNbVnYLIS5Eoy2LhqxevbpJYwhLliw557GTJ08SHR3tvo6JiWHv3r0XUka755olVV/wpddgiUrAlrmbDzIKCaOKbEccx1QMM2VltxDiAlxQWFzM3lCmadbZLkQpJduHtBDDlo0jL5MTQQnsK4lkQNKNXJME27/NJxGYObjlu6CEEO3DBYXFxfxyj4uLY+fOne7rgoKCOt1U4sK4dqlVhp1wpbOrZKJ72mzq5IH+Lk8I0cp5DIu0tLQGH1dKYZrmBb/g6NGjef755ykqKiI0NJS0tDSeeOKJC34+4eTapVZDYcGkr/UER6qi2Z97WloTQoiL5jEsXnvtNY/fdDGL5GJjY1mwYAGpqanY7XamTZvGkCFDLvj5hJM1fiDVehDKcGCgcdARh8Wiy+6zQogW0eg6i9asLU+dbcg5Yxb1tvowbNlUZ21DA05Zozmaa6OwQ08GDh9+Ua2PQL1fELi1SV3NI3U1X0tPnb2gMQsRmFzbgPQCetX7nGHLpnzj02A6UECEgksBBxZWvFnI3XdPlu4qIYRHTVpnIVo/55iGAwDX9AS95rCk3voJWX8hhDgvCYt2who/EKVZUOrsNiCuxXqHzTgZ2xBCnJfHbqi8vDzi4+Mb/NzmzZsZP36814oSLcs1lpHV7SYKDmeBguNGFAmXgKNrP+6+yDELIUTb57Flcd9997k/vv/+++t87rnnnvNeRaJFudZfVO9cR/8TH5HhSGJdxUh2GANJmjyDKVOvl6AQQjTKY8ui9iSpo0ePevycCGyu9RcohYbBnCt0MoL7yKFIQohm8diyqL1Ku/6Kbdmeo/Wwxg8EPQg0HaVZyLbHegwKw5ZNVcZ7GLZsP1QqhAhkTWpZiNbLEptE2JSF2DJ3s2qXycFv7Fh3Z9TZqda1/sKxfwuYDqo1CyFjZxF86TX+LV4IETA8hoVpmhQXF6OUwjAM98fgPD1PtB6W2CQyDlk5WH0IpXCf093bWlAnJNyUQdXW17BEJcjpekII4DxhkZWVxciRI90BMWLECPfnpBuq9RnQszNWi45Rc1jSoIgzlL/3EhjVdb5OQc0peybffZlO2PBoGdsQQngOi8zMTF/WIbzMdVjS/tzTDOjZmbiTW6g27XW+RqFjKoWGwoHOhgPBHDtQt8tKCNE+nXe7D1cXlNVqpbS0lO3btzNgwAB69aq/mYRoDWoflmRYnRsPYjpA0wkaMI6PixLYdaCQJOsJsh1x5Dii0TVTdq4VQngOi+zsbH72s5/xu9/9jlGjRjF9+nQASktLefrppxkzZozPihQtzzXw7dp48ItjHdiyPw/DjCbH4TzJUNOQnWuFEMB5wmLp0qX86le/4tprr2XdunUAvP/++9hsNhYsWCBh0QZYYpM47Ihm2esZ2B1nzyjRgKH9utIn/hJZjyGEAM4TFvn5+dx6660AfPXVV1x//fXouk63bt0oLS31WYHCu/bnnsZh1D3MymrVuWlkL3pbC3DkbeHI6Ya3PBdCtB8ew0LXz67Xy8jIYNGiRe7rqqoq71YlfKb2LClN1xg3uBujB3ejt7XAfUxrhNLQq5J4fXtfxl0/lmuGdvd32UIIH/MYFp06dSIzM5PS0lIKCgq46qqrANi1axexsbE+K1B4V/1ZUq6WQ+WWd8CoRgOsKEaHZHF1yEGWfwIJ0c6zL1wbFFb+6AoIaXjTSSFE2+AxLH79618zZ84cSktL+c1vfkNYWBh///vfWbFiBS+++KIvaxReVnuWFDhXdNv3b6H2Gn5dA4tynu2d9/1euh865FzMpwzyM94l9JaFsoBPiDbMY1gMHTqUzZs3U1lZySWXXALAsGHDeOutt0hMTPRVfcIPHHmZKNNAw3nmhUKDmrMvKujA0JxXcdRao6EMB468TAkLIdowj2Fx5syZcz7u06eP+zoyMtLLpQl/scYPpEKzgGlgoPN2+VUMig+huksSk4NsaFmOOl+vWazODQuFEG2Wx7AYOXKke1uP+psKaprG999/793KhN9YYpMoHjWfLz/dTFZ1LMeI5Zqxw9zjFOXZae7FfOUJV5MbMYTOjmikXSFE2+UxLG677TYyMjK47rrruPPOO0lKkl8F7UmvQUOxd+5NeO5pZvTsTG9rAVUZW7DGD3Qv5ttb2oX/S6/EVOVY02VbECHaMo9h8fTTT1NRUUFaWhpLliyhvLycW2+9leTkZPcYhmjbXAPfrtP2MO1UoZMfOYwf4q5kZXolpulsdToM2RZEiLbsvHtDhYaGMnXqVKZOncqJEyfYsGEDqampJCYm8uc//9lXNQo/q33aHsogrmgnXYsy6KlPJMd0bg2ia5psCyJEG3besKitqKiIoqIiTp8+TZcuXbxZkwgw1njnpoNmzboL1zTa/kE2NKBfkI2+V1wlrQoh2rDzhkV+fj7vvvsuGzZswGKxcOutt7JmzRpZlNfOuDYdLMj4jOAj6ehKYaBz2cAeTM59H10ZaFn7MPpHy/RZIdooj2Exa9YsDh8+zM0338wzzzzDj370I1/WJQKMJTaJuMlJHNl3FcUHv6VT38H0tx+jOtcAFJiy1kKItsxjWOzYsYOQkBDeeust1q5d635cKYWmaezatcsnBYrA0mvQULI792Zf7mmIgCjXmRi6rLUQoi3zGBaffPKJL+sQrUT28WKWvZ6BwzB516LzyOR76RtcQFWn3u5WhWHLxpa5m2x7LPGXDpGxDCHaAI9h0b277CwqzuXa0lwpMAyTfSWRDL91HOm7j7E/PYdBEWfo/OWLhBl2LsPCym8ncdddkyQwhGjlmjwbSgiou6W56xS9zJwid2ujNHQfkzs4sGiAMumt58v6CyHaAAkL0SwNbWn++d58ErAxPPQgl2gVGEpD4dx48JARx92y/kKIVk/CQjRb/S3Nh1xSzJCIj7DiPHHPQCO9qh87q/vSa9Dl7q91nX9hjR8os6aEaGUkLMRFi6k+SpFmotVc60pxRnXkGLHMGNwNw5ZNddY29/kX1XoQYVPk/AshWpOADYsDBw7w/PPPExYWRnJyMmPGjPF3ScKD0F6XgWZBmQbg7H6KSBzEg8OHuY9nxag++w2yJkOIVidgw6K8vJxHHnkEi8XCs88+K2ERwDokDGB37zmUfb8FFOys7suEvsH0OLmFqoJDdYMCZE2GEK1QwITFyy+/zNatW93Xr7zyCrm5ueXMGTwAAB26SURBVDz88MOkpqb6sTLRFPGXDmHZbgPDMOkTXMjQnFepPmSvczQrmoXggeMJ6j9GWhVCtDIBExbz5s1j3rx57ut9+/aRmJjIG2+8wdy5c7n55pv9WJ1oTO1ZUsOqi86epqdA05zHs6ZX9eVk2QhGO6LpLYPdQrQqARMW9VVVVfHoo48SHh7OhAkT/F2OaIKz5184KM9Oc+9SaypwYOHrqr7k7M4j9797uC/iP1iUQbVFBruFaA00Vf/M1BZWWlpKSkoKK1asICEhAYCNGzeyfPlyHA4Hs2fP5p577vFmCcIPKo/tJ3f3Dtan5xNKFdmOOHIczrMvbujwLTeHZmDRQKGR12Mi0eOnMTAxys9VCyE88WrLYs+ePSxatIicnBz3Yzabjeeee47169cTHBxMSkoKI0aM8MqxradOlbpPcmuO6OgICgpKWryei9Wq6gqJp/OIqUxIKGb7t/l0Lqvm2KFTGIaiTIUAGqZSGGjkHDjM+sy3vbItSKu6ZwFA6mqeQK0Lml+brmt06RLu+fMtUZQna9asYfHixcTExLgf2759OyNHjiQyMpKwsDAmTZrEpk2bvFmG8KOk7p1InTyQ++8cwsKZV3D7pYo7wnYAChPQgJEhWfwi7EPOZPzHz9UKITzxastiyZIl5zx28uRJoqOj3dcxMTHs3bvXm2WIAJHUvRM9TlZQZXMu4FOAUia65tz6vl/+B1R/3xVVWSoD30IEGJ8PcJumiaZp7mvX+RiifbDGD6Ta4jwDQ9N0lGnUvAdAmSaVW19FQ8kqbyECjFe7oRoSFxdHQUGB+7qgoKBON5Vo21xHtAYPv4Ow5Ic5EH8LJhqmAhMNpUxQCmU4+O7LdLKPF/u7ZCEEfgiL0aNHk56eTlFRERUVFaSlpTF+/HhflyH8yBKbRMiwKVhik4gcdiMvld/EBxXDWFs+AoeyYCgNh4KS3CwOvv0iR/bt9nfJQrR7Pu+Gio2NZcGCBaSmpmK325k2bRpDhgzxdRnCz7KPF7u3Ob/rrkls2HqI/x4+Tb7RmauCDzIy5ACDg48CoNIPYkT/VrqkhPAjn4TFp59+Wuc6OTmZ5ORkX7y0CEC1j2a1WnQenDGMqWP7kHU0g1wjmv7Y0HGOYwCgDNl4UAg/83k3lBD1j2Z1naT3yORI/nfgccZdlYSmW5yzpQBNNh4Uwu8CdrsP0XY1dDSrYcsm6quXiDLtUBhEyNhZFB05QHFJFT/EXUnFt/kkZe4mduDQOi0MOVBJCN+QsBA+19DRrFUZW8C0g1JgOig8Wchj+/phd5gkHsnlvog0LBiUHfiIjskPYYlNwrBlO8/KMO0y1VYIL5OwEH5R/2hWa/xAqnXn+gt0K9n2WByG3fm11hNYMLBoYJoOsjdvIiImjii9rE7AyLiGEN4jYSECgmv9hatLKd4RjXV3Bg6HSbYjDgMLKBMTjeiiXehFJtWafnZBp4xrCOFVEhYiYFhikzjsiGb/odMM6Im7q6pj6AC+K4wn+FQ2RsEhBgUddW4RglkzAm4hZPRMaVUI4UUSFiJgNDSl9pZRic5BbFVMYcdYQs9sce4rpXDuQgiAQlWW+q9wIdoBCQsRMBqaUtvbWuAexL4EHaUp9yaEaDqmUmiaBa1DOFUZ78msKCG8RMJCBIyGptQ68mrNksJE03ScZ7VaWFs2nFAqqaADd25bjWY6qNKt7tlSQoiWI2EhAkZDU2oNa91ZUiGjZ1J4spBPjoWxrbADCrixw7cow46ugWE4sGXuJr6BsJA1GUJcOAkLEVDqT6l1zZKyZe4m2x6LWdWH13eA3WECoGmQbbhmSxkoIK8E4us9b501GZqFoAHjCOo/RkJDiCaSsBAB77AjmmU7uuIw7GhaFqrmqFwNSIyLIDK8K+8cPc2doV+ho+h/4iMM21AATmcdxujUG0de5tnuLOXA/v1n2LO2yUI+IZpIwkIEvNoD3yiFrmvOA5N0jaMnS8k5UcKNodVoWs1mZ6aDym3/xizKpVyZoFsJGnQDzs8aZ59YFvIJ0WQSFiLg1R/4nnFDP8oq7JwqruSLPXkoBQeqY5kc6uyKAoVZeOjsExjV2PduAmUCOugamM4TwLUOng+oF0KcJWEhAl5DA9/gXJexbd8JDMPkGLEUj5pP9JGPcRz/Dg3lnF7ropznfqOBpcflGLl7QJlUbf83lqgEaV0I0QgJC9Eq1B/4dj1WO0TswKcHhzFKfY8F5wC461wMpUBpzu3O9bBOGJiA7CklRFNJWIhWzRUirtXfdofGLutEkqwnKFMh3BG2A0vNnlIFna8gafxkAOxZ29zTcT0t6JOptkKcJWEh2gTXIDhAjiOaHEc0APlGZ5KsJzhsxnH3lMlYYp3BktdrFklBNqI6mFRtfQ2USbUliJDRM1GVpajqcux7P3I/LrOmRHsnYSHahNqD4JquMW5wN3rGRVBQ3IuKCjtj4iLYn3uaYwWlvP7xARyGSd9gk/nhm9CUM2QwqmuCw6j75IZduqpEuydhIdoET4Pg0dERpO8+5t6gUNM0lOkc/O6t5aNcA98AaOcGRc3jqvQUhi1bAkO0WxIWos1oaBAc6q7TUEq5w+GAIw6HsmDVDDQ0OLtFYa0AcU61tX//GfbMzQQNmYQWHCbjG6LdkbAQbd6Anp3RdQ3DqJlMW5MJOY5oXixxDoZHWcoYGZzlPI1PQWXcYDr1HIAqPYX9+8+c36cM7Hs+AHBuGVITHDK+IdoD3d8FCOFtSd07MW5wt7MPKLDozrZEjiOajysHs6OqLwYWDKXhwEJmx6sJGTaFoP5jQLOc+6Q1wVG9Y60zQGoWA2JUU7XzHQxbtq/+eUL4hISFaBdGD+5GkFVH18Bq1blnYn8mDI3HaqkJDSOa5aUT+bByKCvLJ2F27cP76TkcdkQTMnZWw4HhgXH8O8rfW+oxMAxbNqe3rZdAEa2KdEOJdsHTAHjPuAhWp2VhmoojZgydEi4lAvj3f7JwGM59qCZd1YPY3nPcU22dXU4ND4Q7xzwUGHaqs7YRWm9cozprG479W9x7VkmXlWgtJCxEu9HQAHhZhR1TOWdHmYZi78FTGObZjUJMU/HhV7logNXalQdnDKN34hU48jLROoSjKkvr/Ldq27+ci/1QODI3U15ejB7WCYJDzw2ZBgJFiEAlYSHatdrrM9A0TFM1+HWKs0e90jOa/ZVWBsR0JunSuuFjFh6pMyBuHNlFQ20Q17M6MjdTCXK2hgh4EhaiXavdPdUxNIjXPz5QExw4t46qnR2axvGCMt7ZchjTVOgW5+K/0YO7uVssQf3HOLcSMexAw8FTe4ouynBPyw0ZO4vgS68BznZZaYDetReqslSm5Qq/krAQ7V7t7qmE6HD3uAY412iUVzpI23EUw1R8+V+b+/sMQ/H57jy2fJvvDg2IJq/XLPo7Mul49MtzxzZqptw6vvsY5agVKMqgasurOHL3Op87d0+979Wo1uWEP+E/EhZC1FJ/XCOpeyfeT8/BVJ5aCWdD44s9eeiahqkUVks/Hpl8FTGn9zTYOogYOpaTX/8HR+YXNedsAJgYR3Z5eBXnDrmeFgfKokDhbRIWQjTCNa7hcJjujiVNwxkM5tlzM5QCoyZUHA6Td/fr9ImfUGf2lUuHhAGEhsRT3bVXw/tRnU+9xYGWvldjHPzauShQt2DpMQQ9rJO0QESLkrAQohH1xzXKKuzubqrt3+azeU8e9cfFFZBxoJCMA4Xu6bdhHaznBEfwpddgiUpwTqnN3Hw2NDQLlp6Xo4d1Qu/aC6PwSL1WiOuFDIzs9LPXpsM9qG7P3HzOc7haOa6PJVBEU0lYCNEEnvadSure6exajZpWRf0eK9f0W8AdHOgaFRX2msHxJEJjkzD6j3EPajf0S7zZrZBGZ2PVDZSg/mMgeljTnlu0OwEdFoZhMGfOHBYuXMjgwYP9XY4QDbpmaHf3wLhrRlXtLqvaagcHwJa9eQzp25VOHYPpGRdBbtkIAEY7oqn/976rFeLIy6y1H1XtKNBrJlrVa32cT61AsWdupuDo9VR3jG+wFSKzstq3gA6LFStWEBMT4+8yhGhUQzOqas+i8sQwnd1V9dUOkdpTcy2xSe5f1EH1Fgda4wcCUJ21DVVe3MCMqkYog5JdaY1+mWsTRVVd4W4FATLA3sYFTFi8/PLLbN261X09Y8YM+vXrh2k2468kIQJA7eAY1j+6ycFRW+0Q2bI3j3FD4p0tjxMlgHObkrIKKwN6jjune8y1Irz+Wo36rYULChSoM8AOYP/+i5oWjaozwC5jI22LptR55gT60a9//WvCw8PZt28fffv2ZdmyZc1+jlOnSj2uyD2f6OgICgpKmv193iZ1NV8g1ZZ9vNjdVVVQXMmJwjL2HizEuIi/h1xjIBVVDoA6rZCmcAWKKi/GOLoXzJrdc1tavQF71/YoTQ2TQPo51haodUHza9N1jS5dwj1+PmDDwuX555/nmmuuuaAxCwkL3wjUuiBwa3PVlX28mO3f5gO4Ww7FZdUXHCIWnbpjIDUtkaaEiGHLJqT4MOVGUINjFg3ub9VSGpj9Vfv1Q0ODMHpcHXCtk0B9f0ErDIvS0lJSUlJYsWIFCQkJAGzcuJHly5fjcDiYPXs299xzj1deW8LCNwK1Lgjc2hqryxUixWXVfHvoFIahLurvfU8h4uzOsrun9DZWl2vxn6tVULdFojV/gL05PASKP7u4AvX9Ba0sLPbs2cOiRYs4fPgwmzZtIiEhAZvNxowZM1i/fj3BwcGkpKTw7LPPkpQUWH8xCBEoMnOK+PZgIRFhwRw6fgbQCOtg5Z0vDjZ5DKQxFl3jtgl96RgaVOd1rhveg4GJUef93spj+6k48h2hvS4DoGTv56BBSGwfqk4cwig7Q3n2NzWB4gW6hbCkK7GER7pf0/X6RkUJltAIjIoSQntdRoeEAd6poR3walg8+uij3H777SxcuJBXX32VhIQE3n77bXbs2MFTTz0FwIsvvohSivnz57f460vLwjcCtS4I3Npaoq7aYyAX233lyflaJM3t4qq/MWKd1smFDLQ3V71ZXC0xLThQ31/Q8i0Lr86GWrJkyTmPnTx5kujoaPd1TEwMe/fu9WYZQrRJDS0UbOkxEE9Te2s7Z61Ig11cSec9t6OxmVvBRhnlB765uECpN4urIfWnBcs6k7N8PnXWNE00TXNfK6XqXAshLpynleYNhQhAaIi1WVN6G9JYoNSfseUKEVeLyP2YteF9tMD5V/KJfRnnBEqLt0qaGSgFoUFUd4w/Z3ZXWwwWn4dFXFwcO3fudF8XFBTIwjshvMxTiMDZtSAdQ4NA18E0W7Rbq/6q9fNpKFhyT5QQGhpMdKfQsyvc+3QjycPZH/V/YTe82v0i1AqUxjp5GmuptKZw8XlYjB49mueff56ioiJCQ0NJS0vjiSee8HUZQogatYOkfj+3pxZJS3RxNaSpweJpm5SeERHklsW5Py6rdLZg9N7dSAqy0TWmq2+nBTehpVJbU7rBmrM+pSX5PCxiY2NZsGABqamp2O12pk2bxpAhQ3xdhhCiCc7XInHxZheXJ00ZS6lP17sw6aoEKqrOhkntYCmN6ULXylw6RkZiFh4BDbr27k+Uo8D760xcmhkunnYW9samkAG/KO9iyGwo3wjUuiBwa2svddWfsQUNj1l4M1guRv3ZYKVHMusEitVqwYxMcP/F7wqZ2JgoOh7+wvszvDzRrcTPepySkPimf4s/Z0MJIdq3prRMXGqPndTv7nKOWYR4bYqwJ+e2YDSgV83HtXtEOtV5zHIMru85nUQtv05LRe/Sq8GPQ8MjiD2xDTDRal7lopgOKo58B/2bHhaNkbAQQgSE8wVLc8dSXB+XVdibvYljSzBMSMsJ5pxgyfXwMZBoDSfJeoJyFcLlUeUEB+nnhEvtFkyIWUa3ioNoNJCaupXQXpc1OgDfHBIWQohWpzktFjh/q8XTx75swQDkOKLJcTjXoG3Pr3nwnHCp24JJtCZxVfBBAI4ZUSRYigDY5ejL/zii6RLScvVJWAgh2rzmhotLYy0YV/dY7TEYX4ZM7YCp79uDhVwzpFuLvZaEhRBCeNBYyHiaENDUbjJvtWasFo3Bfbte3JPUf84WfTYhhBAX3JKB8wdNQyvfPe3XNTAxqkVntklYCCFEALmYoPEm3d8FCCGECHwSFkIIIRolYSGEEKJREhZCCCEaJWEhhBCiUW16NpSuX/gOKxfzvd4kdTVfoNYmdTWP1NV8zamtsa9t07vOCiGEaBnSDSWEEKJREhZCCCEaJWEhhBCiURIWQgghGiVhIYQQolESFkIIIRolYSGEEKJREhZCCCEaJWEhhBCiURIWQgghGtWm94Zqro0bN7J8+XIcDgezZ8/mnnvu8VstL7zwAh9++CEAEyZMYOHChfz2t7/lm2++ITQ0FID58+dz4403+ry2WbNmUVRUhNXqfPs8/vjj5Obm+vXevfXWW/zrX/9yXx87doypU6dSUVHht3tWWlpKSkoKK1asICEhge3bt/OHP/yBqqoqbrrpJhYsWADA999/z6OPPkpZWRnDhw/nsccec99bX9T15ptv8tprr6FpGoMGDeKxxx4jODiYF154gXXr1nHJJZcAcNddd3n151q/Lk/vd0/30Ztq13bw4EGeffZZ9+dsNhuXX345K1eu9Ok9a+h3hFffY0oopZQ6ceKEuvbaa9Xp06dVWVmZSk5OVgcOHPBLLdu2bVN33323qqqqUtXV1So1NVWlpaWpKVOmKJvN5peaXEzTVGPHjlV2u939WCDdO6WUysrKUjfeeKM6deqU3+7Z7t271ZQpU9Rll12mjh49qioqKtSECRNUbm6ustvtau7cuerzzz9XSil1yy23qIyMDKWUUr/97W/V6tWrfVbXoUOH1I033qhKSkqUaZpq4cKF6h//+IdSSqmf//znateuXV6r5Xx1KaUa/Nmd7z76sjaXkydPquuvv14dPnxYKeW7e9bQ74iNGzd69T0m3VA1tm/fzsiRI4mMjCQsLIxJkyaxadMmv9QSHR3Nww8/THBwMEFBQfTt25e8vDzy8vJ45JFHSE5O5q9//Sumafq8tkOHDgEwd+5cbr31Vv71r38F1L0D+P3vf8+CBQsIDQ312z1bs2YNixcvJiYmBoC9e/fSq1cvevTogdVqJTk5mU2bNnH8+HEqKysZOnQoAHfccYdX7139uoKDg1m8eDHh4eFomkb//v3Jy8sDYN++faxcuZLk5GQef/xxqqqqfFZXRUVFgz87T/fRm+rXVtvSpUtJSUkhMTER8N09a+h3RE5OjlffYxIWNU6ePEl0dLT7OiYmBpvN5pda+vXr5/7B5uTk8OGHHzJu3DhGjhzJU089xZo1a9i5cydr1671eW0//PADo0aN4sUXX2TVqlW88cYb5OXlBcy92759O5WVldx0000UFhb67Z4tWbKE4cOHu689vb/qPx4dHe3Ve1e/ru7duzNmzBgAioqKWL16Nddffz1lZWVceumlPPjgg7z99tv88MMPvPTSSz6ry9PPzh//n9avzSUnJ4evv/6a1NRUAJ/es4Z+R2ia5tX3mIRFDdM00bSz+7krpepc+8OBAweYO3cuCxcupE+fPrz44ovExMQQGhrKrFmz+OKLL3xe07Bhw1i6dCkRERFERUUxbdo0/vrXvwbMvXvjjTf48Y9/DECPHj0C4p6B5/dXoLzvbDYbs2fP5s4772TEiBF07NiRv/3tb/Tt2xer1crcuXN9eu88/ewC5X4BvPnmm8ycOZPg4GAAv9yz2r8jevTo4dX3mIRFjbi4OAoKCtzXBQUFDTY7feWbb75hzpw5PPDAA9x+++3s37+fjz76yP15pZRXB0E92blzJ+np6XXq6N69e0Dcu+rqanbs2MF1110HEDD3DDy/v+o/XlhY6PN7d/DgQVJSUrj99tu57777AMjLy6vTCvP1vfP0swuk/08/+eQTbr75Zve1r+9Z/d8R3n6PSVjUGD16NOnp6RQVFVFRUUFaWhrjx4/3Sy35+fncd999PPPMM9xyyy2A84331FNPUVxcjN1u58033/TLTKiSkhKWLl1KVVUVpaWlvP322yxbtiwg7t3+/ftJTEwkLCwMCJx7BnD55Zdz+PBhjhw5gmEYvPfee4wfP57u3bsTEhLCN998A8CGDRt8eu9KS0v5yU9+wi9/+Uvmzp3rfrxDhw4sW7aMo0ePopRi9erVPr13nn52nu6jrxUVFVFZWUmPHj3cj/nynjX0O8Lb7zGZOlsjNjaWBQsWkJqait1uZ9q0aQwZMsQvtfz973+nqqqKp59+2v1YSkoKP/vZz5gxYwYOh4OJEycyZcoUn9d27bXXsmfPHm677TZM02TmzJlceeWVAXHvjh49SlxcnPt64MCBAXHPAEJCQnj66ae5//77qaqqYsKECUyePBmAZ555hkWLFlFaWspll13m7gP3hbVr11JYWMg//vEP/vGPfwBw3XXX8ctf/pLHH3+cX/ziF9jtdq644gp3954vnO9n5+k++tKxY8fqvNcAoqKifHbPPP2O8OZ7TI5VFUII0SjphhJCCNEoCQshhBCNkrAQQgjRKAkLIYQQjZKwEEII0SiZOivanCeffJIdO3YAzgVn3bt3p0OHDoBz1a3r48Z88sknpKens2jRIo9fY7PZ+OUvf8kbb7xx8YUDDz/8MNu2bSMqKqrO43feeWeLT6l9+OGH6devHz/5yU9a9HlF2yRTZ0Wbdt111/GXv/yFwYMH+7uUJvHlL3AJC9Ec0rIQ7c6gQYO4/vrryczM5JlnnmH//v28+eab2O12iouL+elPf8rMmTNZv349H330EStXrmTWrFkMHTqUXbt2kZ+fz6hRo3jiiSfIy8sjOTmZjIwMnn/+eY4fP05BQQHHjx8nNjaWZcuWERMTw969e/n973+P3W6nZ8+e5OXl8fDDDzNixIhm1X7ddddxyy23sG3bNkpKSvjxj3/MzJkzAdznUui6TteuXfnd735H7969KSsr48knn2TXrl1YLBZuuOEG9zkHGRkZpKSkUFhYSL9+/fjTn/7kXgEvRG0SFqLdsdvtXHvttfzlL39x/yL9v//7Pzp37szu3bvr/AKuLTc3l9dee43y8nJuuukmvv76axISEup8zc6dO3nnnXcIDw/nf/7nf3jjjTe49957uf/++3n88ceZMGECX375JXPmzPFY36pVq3j33XfrPLZ06VIGDBgAQHFxMevWrcNms3Hbbbdx5ZVXUlRUxMsvv8ybb75JVFQU69ev57777uP999/nr3/9K1VVVXzwwQcYhsHcuXP5+uuvAWc32quvvkpwcDDTp08nLS2N22677SLvsGiLJCxEu+Tacrpjx46sWLGCL774gpycHDIzMykvL2/we6699lp0XSc8PJxevXpRXFx8TlhcffXVhIeHA/CjH/2I4uJisrKyAOdpZgAjR46kX79+HmubM2fOebuGZs6ciaZpxMXFMW7cOLZt20ZhYSE333yze6zjjjvuYMmSJRw7dozt27fz29/+FovFgsVicZ8o+Pbbb3PDDTe4T6Lr168fRUVFjd470T7JbCjRLrm6Wk6cOMFtt93G8ePHufLKK/nVr37l8XtqD4xrmkZDw30NfY3FYjnnay0WywXXXnsnU9M00XW9wUOdlFI4HA6sVmudLanz8/M5ffr0Oc/l6d8kBEhYiHZu3759REVFce+99zJ27Fg+++wzAAzDaLHX6Nu3L8HBwWzevBlwnpqXlZV1wecwvPPOO4BzS+xt27Yxfvx4xo0bxwcffOBuGaxbt47IyEh69erFqFGjePvttzFNk+rqav73f//XPVtMiKaSbijRro0ZM4a1a9cyefJkNE3j6quvJioqiiNHjrTYa1itVp5//nkWL17Ms88+S2JiIl27dvU4hbehMYvLL7+cxx9/HHDueHrHHXdQWVnJokWL6NOnD3369GHOnDnMnj0b0zSJiopi5cqV6LrO/PnzWbJkCVOnTsUwDG6++WYmTpzIp59+2mL/RtH2ydRZIXzgj3/8Iz/5yU/o2rUr+fn5TJ06lY8//phLLrmkWc/T2qYCi7ZDWhZC+ED37t2ZM2cOVqsVpRRPPvlks4NCCH+SloUQQohGyQC3EEKIRklYCCGEaJSEhRBCiEZJWAghhGiUhIUQQohG/X+pg7DZwaVM5QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "net.load_state_dict(torch.load('net.pth'))\n", "losses_train, losses_test = [], []\n", "for epoch in range(200):\n", " # Calculate the loss on the training sample.\n", " net.train() # configure the model for training\n", " y_pred = net(x_train)\n", " loss = loss_fn(y_pred, y_train)\n", " # Update the parameters.\n", " optimizer.zero_grad()\n", " loss.backward()\n", " optimizer.step()\n", " # Save the training loss after each epoch.\n", " losses_train.append(loss.data)\n", " # Calculate and save the test loss after each epoch.\n", " net.eval() # configure the model for evaluation (testing)\n", " y_pred = net(x_test)\n", " losses_test.append(loss_fn(y_pred, y_test).data)\n", "plt.plot(losses_train, '.', label='TRAIN')\n", "plt.plot(losses_test, '.', label='TEST')\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The plot above compares the TRAIN and TEST losses (on a log scale) after each epoch of training. They track each other well which indicates that our learned model has not simply memorized the training data. The change in (log) slope after ~100 epochs indicates that there is not much benefit in going much beyond 200 epochs with the current training loop.\n", "\n", "Plot the final predictions for the TEST data, which look great:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfVjUdb74/+fMwIAIcufcAN4AkgIKgVLetEGmZoh3cdzWzbO61bbVerQ6Zz1ft7oqO9uvs53uLWvPbtueU7uddStRE8jNXGq9xzsgEW8Ab7iZgRlEEIabmc/vD2SSNJUBnAFej+vqqs/MZ2ZeL+2a17zvVYqiKAghhBi01O4OQAghhHtJIRBCiEFOCoEQQgxyUgiEEGKQk0IghBCDnBQCIYQY5KQQCCHEIOfl7gBcVVd3AYeje0sgQkP9sVga+yiiG0ty8TwDJQ+QXDyVq7mo1SqCg4d+7/P9thA4HEq3C0Hn6wYKycXzDJQ8QHLxVH2Ri3QNCSHEICeFQAghBjkpBEIIMcj1uBA0NjYyd+5czp49e9lzxcXFZGZmMnv2bJ566ina29sBqKysZMmSJdx99908+uijXLhwoadhCCGEcFGPCsHhw4f58Y9/THl5+RWfX7VqFc888wyff/45iqKwfv16ANasWcN9991Hbm4uEyZMYN26dT0JQwjh4aw52TQdLXb+G6DpaDFnX38V69ZcrDnZznur/+ePmP73j11e3/la0Td6VAjWr1/Ps88+i16vv+y5iooKbDYbSUlJAGRmZpKbm0tbWxv79u1j9uzZXR4XQgwsl37p+0ZFUfnu21SVnuH066+S9+6fKF+7lnrvodSu/z8q65rZtKOMgm27aczfw/m9uynYtpt/vPshhX/6mDNvv8WBC74cOlFLwzffUPThX9m0o4xDJ2oH1Iwgd+nR9NEXXnjhe58zm83odDrntU6nw2QyUVdXh7+/P15eXl0eF0IMLJ1f/k3z/pkyHz0XQhOZdHAnZb5hROX/jSL/aMYUHGCX4VZS8rZiGXaKUedL+DoxAwWFievfxzwkAl1jKV8OTyH/BKTv+m80DWVsGnEnpWfL0HpruNW3nvlRGkLnZLg75X6rz9YROBwOVCqV81pRFFQqlfPfl/ru9fUIDfV3KS6dLsCl13kiycXzDJQ8oPu52B0K+4+aKK2oJzoikKSpKXzy1d0krH8f67BxJNeXUOQfTUJjKWd89SQ0lrIjOJGdAbFoWm3cVlfAjuBEdjcPA0A1bBy31RVQ5B/NVGshvvZWxp4vQ1FBzLlS2vw7WgLJx/L4TDOP23YdYESblZH/dE+Pc/FkfZFLnxUCo9FITU2N87q2tha9Xk9ISAgNDQ3Y7XY0Gg01NTVX7Fq6FoulsdtNQp0ugJqahm5/lieSXDzPQMkDup+LJXsLm8rs7LUF0tpmR+utYYLKgm9tBQcv+UIf01RBkX804xtLKfSPJrm+hGa1luT6EnYEJ5JcX8KpIUaALo+d9ItwFopTQ4xkVm8n8fwJFFT8PXQi5srz3HxgM5+OnMT00jfxMRgISZ/jUi6ezNVc1GrVVX8899n00YiICHx8fNi/fz8AGzduJDU1FW9vb1JSUsjO7hj4ycrKIjU1ta/CEEL0AYdD4dCJWmc//VltCMkFWzDUV6AAhvoKbiv5nBa7ytkSGN9YSrH/aKKbKtgWmsKYpgqK/Uczw5LPzuAEvg5NIsuYRmb1djKrt5NlTOPr0CR2Bicw4ZLCAbA/MA4NChoc3GnZz6LKbR33nd5H7a5dOFQqGVzuhl5vETz00EOsXLmShIQEXn75ZZ5++mkaGxsZP348S5cuBeDZZ59l9erVvPPOO4SFhfHqq6/2dhhCiD7gcCgcOlnL/31xnPrGVtrtDrTeGgL8vPEzpLGwOo+DgR3dQDuDE5h2rpAsYxphLbVs80kh1XqQr0KSyQ+Ox+wTwtT6b/jacCtahx0VYAqMoIqxcPG/R9VXMO1cIV8OT0GtODgafBP/VPElCrAjOJHJdUV44cAbO3dYDtCu1vB1UCI/yNpI9MqVbv2z6k9U/fXweukaklw8zUDJA66cS3u7g+f/Zx8jju2h0mc4p/2MzueiW0wYmmvxsrc5u3DsXt606CIoUkJpbbPj7aUmQW0lccgFasffhkatYpQxgAmRIRSVWzljamCkIYDE6FAACkotXPgiF/+YMThGxXC2ppHRNhN+Wf9D7YhYtjbpWFi5HW+lHQ0d3wUm7yAC7M1sDEsjccYUZg234W2pwuf2GTfuD68P9VXXkBSCfkpy8TwDJQ+4PBeHQ2H9r3/L0faOgdyF1XlkGdMAiGssY1zjaQrCk0msOsjBYeNIPl/CwcQMfvLw3Mu+5NXq7k8O6WTNycY3KgqHQ6Fs7Zt8PWwCqZaDaHCgRkEFWLwCOD3ESJ1PIFPPFTF02c8xBvrSuG8vWr3eOXbQH/VVIei3u48KIW6cI3/+mHNN7Sy0dhSALGMai6q+RKPYaVN7sXt4EmmWQlruvZ/hvgbUNhO3b/6Q1hM3kRQbR1LM8F6Jo/NL3JqTTfiCBdy5aSNtajXbg5NJtR5Eo9gJaW8gqKEBdQNsC02h9vOjZFZux9fHi/Dl0l10JVIIhBDXVO0byhTrF+wMTmBhdR4n/SLwVtpRAXuD49EN0xLxz8vxj48nEYAomiICsZWV4Rcb1+vxhKTPwZqTTfCUKWj0Bn6QtZFPwmegKHC3eRch7Q3YUZFmPQgK2NVqPo+axaNjY3s9loFANp0TQlzT8KREtkTcwbS6QizaQBIaS7GjZp8+iSlNJ5g1/zb84+O7vMYvNq5Pu2FC0udgWPpT1IpC9MqVJM6YgloNvo5WivyjAfBW7HhjZ39gHIfagykotfRZPP2ZFAIhxDUlRoeiHRtLuf8IRtrM2FHhUGuYv2QWI5f/C6b/fse5ncSNFpI+B//4eGYNt3GP6Ws+i7iDgmEx2FGjAO2omVRfTERjFWdMA2MMp7dJIRBCXJNareKB4Grizp+kOcSIWutDWGYm5t+9C0DYI7/AVlbm1hhbT5UTtWIF86ZF8k/V23Go1WwLTaEwYAwo8E/V2xltk+1srkTGCIQQ19R0tBjrpg3o7l1MyF1303S0mKp31xE8JwNbWRkh6XP6ZCygOzq7oUaeyqZl2lQ+tQZxqD0Yu0PhRHA0Ke1nSWmRrqErkUIghHCy5mSjHR1JmW40hcfMjDIEENNq5tzWXCJWPuH8sveLjXO2AjxtOmZnPI86FApKLVgbWwnxT+zx1NWBTAqBEALoKAKKSkXZ2rVsDk/jpLeeyeeL0dYeZNTj/3rZL36/2Di3twKuRq1WkRQzfECt7+grUgiEEEDHttFn3n6LnUHjmXNmO6V+EYxvLCVPfwu3avUkuTvAXnb29VcZEhvHmZhbOG1qYJQhgJEn9tF8tJgRj/+ru8O7oaQQCDHIda7W9R0bS/bI6Uw/8TcaNUOY0FhKkX80u4fFEWZq6LVFYZ5iSGwctR//hT26k+wJjGdy/RGG1OQzfNGP3B3aDSeFQIhBzjcqiqp313Fh3hKOqoYTpw0h2laF2TuI6KYKolpMjDQkujvMXndqTAp7hp/kzpp8YhpOM8Jm5svhKUwek0Kou4O7waQQCDHI+cXGYfj5o5S/tZZ7NEFE2aoo9Q3D2GplZ3AC86vyGNN6MzCwWgT5xWb2BcUztvE0I21mTN5B7AuKp+qrUtQqFTGtZlpPlXvcYHhfkHUEQgxCl54n7HAovHO4hQqvIKKbqygbEsb6EbPIMqYx7Vwh6h/MpPVUuXsD7iMpdUcYcbEI6NvOMcu8m7M1F9jy5y8oW7sW7ehId4d4Q0iLQIhBqNVsxpq9maaFy9h9IYBhBTuIbKrC4hWAscXKqKZqKgPCOWjIYH6IZkD+Kp56vhitJZ9toSnkB8czy7ybieePEdLagKHVyqYR08kYgIPkVyKFQIhByP+WW6nduRP+8h6hfiOZ0FhKGxo+108FOraZPha5kEU/nTdg594H155ib8ztHNaMAbuDv+mnENVURZStih3BiZzU6tl7xDQouoh6VAg2b97MO++8Q3t7O8uWLWPJkiXO54qLi1m9erXz2mq1EhgYyGeffcaGDRt45ZVXCA3tGJK54447eOKJJ3oSihDiOnTOEDrupWOT8Q4WVWy7uIEcfBw+w3nYTPbI6czXNg7YIgAw4vF/I9yhEF1qYW+xCdOBAvzsNtpUGiadK+bUECPmgybKcr4iasUKd4fbp1wuBCaTiddee41PP/0UrVbL4sWLmTx5MjExMQDExcWxceNGAJqbm/nhD3/Ic889B0BRURGrV69m7ty5Pc9ACHHdfKOiqHz3bQ7EzEJRQHXxZC/lkuFCH28NQ2+KJ/6eBHeFecN0LjqLaTVTlv0VWSPuxG5XyKzezg8rt2FXq9k0YsaA7yJyebB4586dTJkyhaCgIPz8/Jg9eza5ublXvPe3v/0tt9xyCykpKQAUFhayYcMG5s2bxy9/+Uvq6+tdDUMI0Q2+Y2P5+qa7uKXgM+6t/AINDor8o2lXacis3s4k7TkeXjCeNT+fNqBbA9/VuWGdITmR035G9gfG4Y2dap9QSrX6Ab9rqcstArPZjE6nc17r9XoKCgouu6+hoYH169ezefNm52M6nY4HHniAiRMn8uqrr/L888/zyiuvdOvzr3bs2tXodAEuvc4TSS6ex9Pz2Hukmn0tgUR7BRDWaqXQP5otxh8wqqmazOrt3DXUwrSpUYDn59Id18pFt7RjEdksXTW1hwtJri9hR3AiyfUljGkzM2HsZI/58+iLOFwuBA6HA5Xq218MiqJ0ue60adMmZs6c6RwPAHj77bed//2zn/2MWbNmdfvz5cxiycXT9Ic8Co+ZMZyrILTtPIX+0YxpqmBUUzVnhxrJv3kuC0Z6UVPT0C9yuV7dyUVXc4p5lXlsGjGdUq2eyoBw5lfmoatJpKZmaB9Heh3xedqZxUajkfz8fOd1TU0Ner3+svu++OILHn74Yed1Q0MDn3zyCT/96U+BjgKi0WhcDUMI0Q2RNhOjTHl8HHYnp/2MjGqqZqEpD9OMe5m1aP6g6g66ks4uooyL3UEjDYlEtSZ2rKP4zglsA4nLYwTTpk1j165dWK1Wmpub2bp1K6mpqV3uURSFb775huTkZOdjfn5+/P73v+fw4cMAfPjhhy61CIQQ3RfRYuFgYgamwAhUgCkwgoOJGaT42wZ9EYBvTztLihnOvNuiSIoZ7jyCs/HIEQ6dqGXTjjIOnail8cgRrDnZbo64d7jcIjAYDDzxxBMsXbqUtrY2Fi1aRGJiIg899BArV64kISEBq9WKt7c3Pj4+ztdpNBpef/11nnvuOWw2G5GRkbz00ku9kowQ4upC52SwzKGQXGq5+Is3QPbpvw4tZhOVWRvYHH4nZT4GolpM3FP5JaHTprk7tF6hUhSlex3tHkLGCCQXTzNQ8gDJ5bsOf7EL1fo/gAL7g+KYdK4YVKDc+wA3z5zaS5FeW1+NEcheQ0IIcQ17mobxqXE6GhzcVleABgefGqezp2mYu0PrFVIIhBDieinf+fcA6VGTvYaEEOIaJg85j6p6O3a1mr2B45lUX0xm9XYU3yh3h9YrpEUgxABx6dbSnZqOFg+YmS3uZKg8ikatYlPEnXwdmsSmiDvRqFUYKo+6O7ReIS0CIQaIzn2Emub9M+W+BiJtJvw2f0j4I8vdHVq/p9UbGL3yceZeXF8wJm8vIdOmodV9u3bKujWXpiNH+uV5x1IIhBggOvcRSl7/PpZh4xh1voSvEzNYNjbW3aH1e51bUCcBSTHDsV5Ionb9/2EZO4mvzx1gcn0xYfl/Y9gd07HmZPe7Laula0iIAaKg1MJeWyAHh43jtroCDg4bx15bIAWlFneHNuAMu/MuDgfHEnJsP9O++l+M+X/jUNBYGvP34RvV/8YNpBAIMUCcNjVgrK/osmGasb5iwO+c6Q6bd5aTE3orNRePuDyv8WPs+VNsCkvDtx+2wKQQCNGPWXOynVsf2EqOssCUx87gBFrVXmQZ01hgymO0zeTuMAec4xXnSKk7gq7tHPUaPwLtTVRrQ/iG0H7ZApNCIEQ/ph0dSdnatWz58xfUlRxnV3AC084VUuUz3LmPUERL//ti8nST64uZYcnnwLCxeCt2Sn3DiLJVMdO0u1+2wGSwWIh+ypqTzRmvYDYbU5l7djsHA8cxqb6YEv8owlNuZmG8QfYR6iPRzdXsChpL3PlTZBnTOO1nZJZ5NzefP0G7zQT0r3ECaREI0U9pR0fCJ/9Du13hYGDHALHG4eCIfyRhoX4kxQyXItBHRj7xb0y/I5Evx8zkzNCOc57zwqeyJ3lhv2yBSYtAiH7qhFbP5rA0Ms9+iQYHbWiwq9VoNCpGGjzjNK2BTDd3Lo/PUSgYADu5SiEQop86bWrAblfQ4MBbsbMjOJFTQ4zcU5XHmNabgeHuDnHAU6tVJMUMJymmf/9ZS9eQEP3UKEMAE5rKsaN2ThfVaFSwaFnHiVpCXCdpEQjRT8W0mtFcOMOmkTOc5+surMojKvxm/ONv3B75ov/rUSHYvHkz77zzDu3t7SxbtowlS5Z0ef6tt97ik08+Ydiwjj277733XpYsWUJxcTFPPfUUFy5cICUlhTVr1uDlJTVJiO4YrOfr9gfWnGx8o6Lwi41zPtZ0tBhbWZlHbj/h8revyWTitdde49NPP0Wr1bJ48WImT55MTEyM856ioiJeffXVLmcWA6xatYpf//rXJCUl8eSTT7J+/Xruu+8+17MQYhD67v43HYZLEfAA/W0DQJfHCHbu3MmUKVMICgrCz8+P2bNnk5ub2+WeoqIifvvb3zJv3jyef/55WlpaqKiowGazkZSUBEBmZuZlrxNCiP6scwNAx/r3sWRtwLH+fb6+6S6P3X7C5RaB2WxGp9M5r/V6PQUFBc7rCxcuEBcXx6pVqxg9ejSrV69m3bp13HHHHV1ep9PpMJm6vwT+audvXo1ON3Cm1Ukunmeg5AGSS0/sPVLNvpZAuLgB4I7gRPa1BHKHpYlb4409eu++yMXlQuBwOFCpvp0vqyhKl+uhQ4fyu9/9znn9wAMP8OSTT5KamnrV110vObxecvE0AyUPkFx6qvCYGcO5rhsAnh5ipOjYKKJ0Q11+X487vN5oNFJTU+O8rqmpQa//9pCGyspKPv74Y+e1oih4eXld9rra2tourxNCiP4u0mZigSmPLGMaX4cmefwGgC4XgmnTprFr1y6sVivNzc1s3bqV1NRU5/O+vr7813/9F2fOnEFRFP70pz8xa9YsIiIi8PHxYf/+/QBs3Lixy+uEEKK/i2ixcDAxA1NgBCrw+A0AXe4aMhgMPPHEEyxdupS2tjYWLVpEYmIiDz30ECtXriQhIYHnn3+eRx99lLa2NiZOnMj9998PwMsvv8zTTz9NY2Mj48ePZ+nSpb2WkBBCuFvonAyWORSS+8n2EypFUbrX0e4hZIxAcvE0AyUPkFw8VV+NEcgqLiE8kDUnG+3oSE5o9Zw2NTDKEEBMq5nWU+UeuSBJ9G9SCITwQM4DZ8LSKNXqiW41M78qj6gVK9wdmhiAZNM5ITzQCa2eTWFpZJzdzg8sh8g4u51NYWmc0MoMO9H7pBAI4YFOmxoo1eqdB84cDBxH6cU9hYTobVIIhPBAowwBRLeauyxIim41y4Ezok/IGIEQHijm4pjAphHTnVtMz6/KI6o1ETlwRvQ2KQRCeIDvzhKKLvuGsAXzWVDbSFlUlGwxLfqUFAIhPMDls4TszN+xiagVK5gQH3XxLtliWvQNKQRCeIATWj1bLs4SOhg4juT6EjaNmE6GVk+Su4MTfcZT1otIIRDCA3x3ltCO4ETnLKH+fjC6+H6esl5ECoEQbmbNyWa0dzCRLSbnLKFJ54oJUGyMNCS6OzzRhzylJSjTR4VwI2tONg6VCs1f32fB2S/JMqbRrNbihYPYhjJiWs3uDlH0IU9ZLyKFQAg30o6OpDJrI6d8dKCChPMnmGHJ5+vhydgXPdAxS0gMWJ6yXkS6hoRwoxNaPZuMqcyvzKNGG0xCYymF/tHsCYwn3M/IzbdNdXeIog95ynoRKQRCuNFpUwPlvkZK/SKY0FjKGV89Y5oqiLRVy/jAINB6qpyoFSvI0Oq58EUu/mPG4FAt5dBXB/DX6m/YDKIeFYLNmzfzzjvv0N7ezrJly1iyZEmX57/44gvWrl2LoiiMGDGCF198kcDAQDZs2MArr7xCaGgoAHfccQdPPPFET0IRol8aZQhgSv0RxjeWUuQfTXRTBTuDE1hY/RXRrUnIKuKBrfMLPglobJ1I2dq1bApLo1QbTvSfv7hhM4hcLgQmk4nXXnuNTz/9FK1Wy+LFi5k8eTIxMTEANDY28txzz/HJJ59gMBh44403WLt2LU8//TRFRUWsXr2auXPn9loiQvRHMa1mtJZD5OlS2B0YT6StmoXVXxG+cIGsIh5k3DmDyOXB4p07dzJlyhSCgoLw8/Nj9uzZ5ObmOp9va2vj2WefxWAwADBu3DiqqqoAKCwsZMOGDcybN49f/vKX1NfX9zANIfqn1lPljHr8X7n1wcXcc3sU8//5LqJXrkStKHIAzSDjzhlELhcCs9mMTqdzXuv1ekwmk/M6ODiYWbNmAWCz2fjv//5vZs6cCYBOp+MXv/gFmzZtIiwsjOeff97VMITo10LS5+AfH09SzHDm3RZFUsxw/OPjpQgMQu6cQeRy15DD4UCl+vYgZkVRulx3amhoYPny5cTGxnLPPfcA8Pbbbzuf/9nPfuYsGN1xtfM3r0anGzjb+Eounmeg5AGSy402qbIcr6o8No2czklvPVXDwllQmcfNflMI0UU57+uLXFwuBEajkfz8fOd1TU0Nen3X05PMZjMPPvggU6ZM4cknnwQ6CsMnn3zCT3/6U6CjgGg0mm5/vhxeL7l4moGSB0gu7mA9fITIFSuYc7E7aKQhkZHNCWzP+prdeVZuigjipwsSqK+70O33vtbh9S53DU2bNo1du3ZhtVppbm5m69atpKamOp+32+088sgjpKen89RTTzlbC35+fvz+97/n8OHDAHz44YcutQiEEGIg+W434YTIEJ76wsIfz0dw9NQ5Nu8sZ9lzubS3O3r9s11uERgMBp544gmWLl1KW1sbixYtIjExkYceeoiVK1dSXV3NkSNHsNvtfP755wBMmDCBF154gddff53nnnsOm81GZGQkL730Uq8lJIQQA8Hh5/8/4luCyA/+duZYbOVhDj+fx6Tnn+7Vz1IpitK9/hUPIV1DkounGSh5gOTiCT75z98z4cQ/2BaaQn5wPCl1Rzq2HzHcygP/8Shq9eVjst/nWl1DsrJYCCE8kHranWyrszHDks+4C6cZYTOzLTSF/cNimXiylok36a79Jtf7Wb32TkIIIXrNvGmRaFUKZu8gRtrMnPXVkx8cz8gL1Rz58K/d7hG5GikEQgjhgby81EyO8Ebfdg6rZigjbGZmmXezsDqPC60Ojvz54177LCkEQgjhgaxbc/E5vJtTw0YRbL9AtXcIE88f44yvjsmWQqp9Q3vts6QQCCGEB2o6coTh9y5G98gKjgwbQ1iblXMaf2KaKtkyYjrDk3pvd1oZLBaiD3jKoeSi/xrx+L8CoD1yBK/mSs4OMTCi2cSRYWPQ3jSOxOjeaxFIIRCiD3jKoeSif2s6Wozpv98hfMF8tFs+48K4JOJKDnF7cHW3po9eixQCIfqApxxKLvo3W1kZwXMyqMvewsjl/8Lo22+l5E9/xZK1gSGRkfjFxvXK58gYgRB9wFMOJRf9W0j6HLA7CHvkF84v/ZC77iZi5ePYysp67XOkRSBEH7jSlsKVAeFy/KTotiuNKfnFxvVaawCkEAjRqzoHiVWV9cyt/DubI+7AblcIUGxuOZRciOshhUCIXtQ5SHx86EiK9KlogIXmr+CflhEVESjHTwqPJIVAiF506SBxg8qX5PoSPhsxnYyom/CPGS5FQHgkGSwWohfJILHoj6QQCNGL3HnurBCukq4hIXpRzMWFY5tGTKdUq6cyIFwGiYXH61Eh2Lx5M++88w7t7e0sW7aMJUuWdHm+uLiYp556igsXLpCSksKaNWvw8vKisrKSVatWYbFYiIqK4uWXX2bo0KE9SkQIdzn7+qv4xsbxuS6BC3/bwdA7FjG3rQ778Z2of/IIUa2JMkgsPJrLXUMmk4nXXnuNP//5z2RlZfGXv/yFEydOdLln1apVPPPMM3z++ecoisL69esBWLNmDffddx+5ublMmDCBdevW9SwLIdzINzYOy8d/4eT6LD5jDCcPFOP7988YNXUSSTHD8Y+Pl/2FhEdzuRDs3LmTKVOmEBQUhJ+fH7NnzyY3N9f5fEVFBTabjaSkjgX1mZmZ5Obm0tbWxr59+5g9e3aXx4Xor772G8e20BRmWPJZcjaXGZZ8toWm8LXfOHeHJsR1cbkQmM1mdLpvj0rT6/WYTKbvfV6n02Eymairq8Pf3x8vL68ujwvRXx2vOEd+cDxnffVdTpI6UXHO3aEJcV1cHiNwOByoVN/ufqcoSpfr73v+u/cBl11fj6sdxHw1Ot3Amb0huXiGm8fq8T+0kxE2M2d89YywmUmpO8KYWQv7dV79OfbvklyuzuVCYDQayc/Pd17X1NSg1+u7PF9TU+O8rq2tRa/XExISQkNDA3a7HY1Gc9nrrpfF0tjtMzt1ugBqagbGfG7JxXNMqikk8mJ3UH5wPCl1R5hhySe0Zgw1NSPcHZ5L+vvfyaUkF1CrVVf98exy19C0adPYtWsXVquV5uZmtm7dSmpqqvP5iIgIfHx82L9/PwAbN24kNTUVb29vUlJSyM7OBiArK6vL64Tob2xHiwld9CPG3LuQuNFBhGXMIXTRj7AdLXZ3aEJcF5WiKN37WX2JzZs389vf/pa2tjYWLVrEQw89xEMPPcTKlStJSEjg6NGjPP300zQ2NjJ+/HhefPFFtFotFRUVrF69GovFQlhYGK+++iqBgYHd+mxpEUgunmag5AGSi6fqqxZBjwqBO0khkFzc4WpHUI5b+qN+k8e19Ke/k2uRXK5dCGRlsRDdIEdQioFI9hoSohvKD3zDruDxZJzdzuthUhYAACAASURBVA8sh8g4u51dweMpP/CNu0MTwmVSCITohiqfUG6tLeSkXwS31RVw0i+CW2sLqfYJdXdoQrhMCoEQ3dBoiGRncAITGks546tnQmMpO4MTaDBEujs0IVwmhUCIbvA3lTGtrpBv/KMZaTPzjX800+oKCTD13kHiQtxoUgiE6IawFit7QxOIbqpgR3Ai0U0V7A1NwNhidXdoQrhMZg0J0Q2RE8ej7PjysvMGIife6e7QhHCZFAIhruHStQONXx0gIHMpc1XQeOIkQ2fe/e15A0x2d6hCuGRQFAJrTja+UVGgu9X5WNPRYmxlZbJPvLimrmsHwonedYr5VXncvGJFx4H0yKH0on8bFGMEvlFRVL27jnMFhUBHEah6d11HcRDiGk5o9WwKS+uydmBTWBontN3fLFEITzQoCoFfbBx+yRP55oX/ZOfaP3Dm7bcw/PxRoKO1IMTVnDY1UKrVczBwHLfVFXAwcBylWj1nTANj2wIhBkUhcDgUchtDaWtpY/jhr9gzNIa/bj9B5btvS6tAXNMoQwDRrWaS60vYEZxIcn0J0a1mRhoGzh73YnAbFGMEBaUWKi1N2FGDCibVFmG3HuHCvQ/gFxvn7vCEh4u5uJ/Qd2cKRbUmAsPdHZ4QPTYoCkHtoQIyzm7n07DpjG6u5ra6AnDA3w5UkHCnglrd/RPSxODReqqcqBUryLjYHTTSkPjtTCEZJBYDwKAoBEabhQ0Rd4BdcTbvJ9UXE1F9jIJSC0kx8qtOfL/OmWVJcMn/KzJTSAwcg2KMIP6+RfgP8WZhdR5ZxjS+Dk3iU+N0bmo8Re2hAneHJ4QQbuVyi6CyspJVq1ZhsViIiori5ZdfZujQoV3uMZvN/OpXv6K2tha1Ws2///u/M3XqVNra2pg8eTIjR4503vvpp5+i0Whcz+Qq1GoVs8IcZDXfwWkfAwCn/YxsGTGdBTZLn3ymEEL0Fy63CNasWcN9991Hbm4uEyZMYN26dZfd89JLL3HnnXeyceNGXnnlFX75y19it9spKSkhOTmZjRs3Ov/pqyLQKX7JDwmYMB4fbw0qwMdbg/amccTft6hPP1cIITydS4Wgra2Nffv2MXv2bAAyMzPJzc297L5Zs2Yxd+5cAEaPHk1LSwtNTU0UFhZitVrJzMzk3nvvZe/evT1I4fqo1SrW/HwaDy8Yz8Lbo3h4wXj+7UdJMlAshBj0XOoaqqurw9/fHy+vjpfrdDpMJtNl93UWCoD33nuPuLg4AgICUKlUzJgxg4cffpjjx4/z0EMPsXnzZkJCQq47hqudv3k1s6YOnHUDOt3Amcc+UHIZKHmA5OKp+iKXaxaCnJwcXnzxxS6PjR49GpWq6y/p715f6o9//CN/+ctf+PDDDwFYvHix87n4+HgSExM5cOAAM2fOvO7Ae+Pw+qsdRO7pexDJgdyeZ6DkAZKLp3Lb4fXp6emkp6d3eaxzsNdut6PRaKipqUGvv/K+Ky+99BJ5eXn86U9/wmg0ApCVlcXEiRMZNWoUAIqi4O3tfd1J9RY5iFwIIVwcI/D29iYlJYXs7I59erKyskhNTb3svj/+8Y/s2bOHjz76yFkEAEpKSvjDH/4AQGlpKcXFxUyaNMmVUHpENhMTQogezBp69tlnWb9+PXPmzCE/P5/HH38cgI8++og33ngDRVF4++23sVqt/OQnP2HBggUsWLAAk8nE8uXLsVqtzJ07l8cee4zf/OY3+Pu71uffE7KZmBBC9GAdQUREBB988MFlj//4xz92/ve+ffu+9/Vvvvmmqx/da660mVhlQDgjDYnuDk3cAJ3nVFy635ScUyEGo0GxxcT3kc3EBjffqCgq332bpnn/TLmvgUibCb/NHxL+yHJ3hybEDTWoC8F3NxOLKrNgvHU+5Qe+obRuSL+aRSS6z3dsLF/fdBfJ69/HMmwco86X8HViBsvGxro7NCFuqEFdCL67mVhjcDNla9eyKSyN0rNlMotogCsotbDXFgjDOsaIdgQnstcWSLJsRCgGmUGx6dz1kllEg8tpUwPG+oouY0TG+gqZLCAGnUHdIviu784i2hGc6JxFJL8QB47OhYQ+FVUsMHXsSAvg197MAlMeals0MHBWoAtxLVIILiGziAY+a042DpWK0jff5LzfKHYZ0tC3WEm1HmTjiJk0xSQwv0V2pBWDixSCS8gsooFPOzqS0jff5B+BE5hWV4jG0c74xlK2D08hfvpkFvwgSjYiFIOOFIJLyJGEA5s1J5szXsFsNKQyvyoPizaQCY2lVGuD2RsUzz0alRQBMShJIbiEHEk4sGlHR8Kbb+LQp3LSL4KExlLsqAhqbyTSVi1dgGLQkkIgBo0TWj2bw9JYdOYLvLHTjpp2tYZ/BN/MwuqviG5NQroAxWAk00fFoHHa1IDdrqDBgQrYEzyBT43Tue1cEeELF3R0AQoxCEkhEIPGKEMAE5rKaVN5OWeFaTQq+OEy1Ioiq8fFoCVdQ2LQiGk1o7lwhk0jZzhnhS2syiMq/Gb846e6Ozwh3EYKQTc4HAoFpRbnaWaJ0aEyy6QfkVlhQlyZFILrZMnewqYyO3ttgbS22dF6a7jVt575URpC52S4OzxxHWRWmBBX5vIYQWVlJUuWLOHuu+/m0Ucf5cKFC5fdU1FRQXJysvNQmgcffBCA1tZWVq1aRXp6Ovfccw8nT550PYMbpMInlOSCLRjqK1AAQ30FyQVbqPAJdXdoQgjRIy4XgjVr1nDfffeRm5vLhAkTWLdu3WX3FBUVMW/ePDZu3MjGjRt57733APjggw8YMmQIOTk5PPnkk/zqV79yPYMbpNzXwEZDGgur87jdcoiF1XlsNKRxytfg7tCEEKJHXCoEbW1t7Nu3j9mzZwOQmZlJbm7uZfcVFhZy7NgxFixYwNKlSykpKQHg73//O/PnzwfglltuwWq1UllZ6WoON8QoQwDVgRFdjrWsDoxgpCHA3aEJIUSPuFQI6urq8Pf3x8urY4hBp9NhMpkuu8/Hx4f58+ezYcMGHnzwQZYvX05raytmsxmdTue8T6fTUV1d7WIKN0ZidCi3+taTfP7ihnTnS7jVt57EaOkaEkL0b9ccLM7JyeHFF1/s8tjo0aNRqbrOlvnuNcCKSw50SUtL45VXXqG0tBRFUbrcrygKanX3alJoqGuH3et0rv2CP1dQSNrJL1Duf4TRvgYCbCbS1r+PjymRoMQEl96zp1zNxRMNlFwGSh4guXiqvsjlmoUgPT2d9PT0Lo+1tbUxefJk7HY7Go2Gmpoa9PrLD2/54IMPmDt3LsHBwUDHF76XlxcGgwGz2cyoUaMAqK2tveLrr8ZiacThULr1Gp0ugJoa1w4dsR46gvHnj+IXG8cYAMJpCvLFdOgIbWGRLr1nT/QkF08zUHIZKHmA5OKpXM1FrVZd9cezS11D3t7epKSkkJ2dDUBWVhapqamX3bdv3z4+/vhjAPbu3YvD4SA6Opq0tDQ2btwIQH5+Pj4+PoSHh7sSyg0Tkj4Hv9i4Lo/5jo3l9E23smlHGYdO1Ha7MAkhhCdQKYri0rdXRUUFq1evxmKxEBYWxquvvkpgYCAfffQRZrOZxx57DJPJxOrVq6mpqcHHx4cXXniB2NhYWlpaeOaZZygqKkKr1fLrX/+a8ePHd+vzb3SL4LLPd/O6AvmV43kGSh4guXiqvmoRuFwI3M3dhaBg224c698ny5DGaT8jo5qqWWjKQ33v/STOmNIrn3E18j93V56w6lv+TjyT5HLtQiAri11U7mvgoCGNRVVfUjJ0FGOaKsgyphHeFED0kSO0niqXTcxuEHe3zoTo72T3URd1risoGTqKhMZSTvpFcNrPiOlAAWVr13YcgiJuCFn1LUTPSIvARZ3rCsY0V1DoH82ExlJU1RDdVMGmEdPJ0OpJcneQg0Rn62xhdR4HA8eRXF/CRmMaE30NyJljQlybtAhcZDt2lNuPb+XY5IVsMf6Aby4Wg1K/CEov7m4pbgxZ9S1Ez0ghcJGtrIzwR5YzNvUWxrSaiW6qoFobwrjGciJbTFRZmjh0opbGI0ew5mS7O9wByeFQOHSillPVDUxQWWTVtxAukq4hF3UOBMccOcL8qjw2jZhOW7uDzOrtLKj4kg12hS2H1MyvyiPqkhXWondcOkBsrK9ggSmPIyMmcdPwIajHTuP2zR9iO3bTZWs/hBCXkxZBDzkPO7lvJuEpN7Mp4k4AptYVkHF2O5vC0jih7d6qaXF1DodCfqOvc4DY2FLLzqAE4s/uZ9jYm0icMYXwR5ZjKytzd6hC9AvSIuihSw87OW1qYI+Pgf2BcdxWV8CO4ETneMG3B6GInuhsCey8EEDExQHiUr8Ixl44xSdhM5wDxH6xcdIaEOI6SYugF40yBBDdaia5vsR5OHp0q1kGLXtR51TRiMYqTvsZKfWLYEJjKceGjpYBYiFcJC2CXhTTanaOF3Qejj6/Ko+o1kRAWgS94dKpoif9IhjfWEqRfzTRzRUyQCyEi6QQ9CI5HL3vjTIEkBMYwcnzESQ0llLoH01u+O0sinRw+84NMkAshAukEPQiORy973Uu5Iu9cJoi/2jGNFcwbWgDsxbNw5YYjq2sTAqBEN0khaCPWXOy0Y6O5IRW79wQLabVLHsRuahzIV/T4p/h7WtAbTN1mSoqRUCI7pNC0Me0oyMpW7uWLWFplGr1RF8cR5C1Ba7pXMjnFxt3cfuIKJoiAqUlIEQPSCHoYye0eraEpZFxdrtzHxzZi+j6XLE1NTrysi99aQkI0TNSCPrYaVMDpVq9cx8cWVtw/VrNJqo2ZvFZ2HRKfQxEt5i4p2o7oVOnujs0IQYUlwtBZWUlq1atwmKxEBUVxcsvv8zQoUO73PPII49QVVUFgMPh4NixY3z88cfExsYyefJkRo4c6bz3008/RaPRuBqOx7rS2oLKgHBGGmRfzGsxhceicuxkfsWX7A+MY1J9MfaLjxvdHZwQA4jLhWDNmjXcd999ZGRk8Pbbb7Nu3TpWrVrV5Z53333X+d9vvPEGSUlJJCQkUFRURHJyMu+9957rkfcTsrag+6w52fhGRbGneRiVxun8sHIbt9UVYEfFX8JnEW4bxs3uDlKIAcSllcVtbW3s27eP2bNnA5CZmUlubu733l9aWkpWVhb/7//9PwAKCwuxWq1kZmZy7733snfvXlfC6Bcu3Yto4e1RZNw3k6gVKzrWFojLWHOyQaOm8t230Zw+AYCajiNJFS4ePdkvD1cVwnO51CKoq6vD398fL6+Ol+t0Okwm0/fev27dOh588EH8/TvOzFSpVMyYMYOHH36Y48eP89BDD7F582ZCQkJcCcejydqC7vGNiqLy3bfZG5rILYc/Y5piR42D0iFhhNtqyazejuIX5e4whRhQrlkIcnJyePHFF7s8Nnr0aFSqrgeDf/e6U319PTt27OCFF15wPrZ48WLnf8fHx5OYmMiBAweYOXPmdQd+tYOYr0an84y9aM5+moXfmDGc8NZRWlFPdEQgMW01NJ08yYjMhdf1Hp6SS2/ozMUeegsffD6Tyd/k0K7S4Ku0UeobxvqIWUTaqsms+jsjLCfR6e5yc8RXNhD/TgYCyeXqrlkI0tPTSU9P7/JYW1sbkydPxm63o9FoqKmpQa+/8lbLeXl5pKam4uPj43wsKyuLiRMnMmrUKAAURcHb27tbgVssjTgc3esj0OkCqKnxjJPD2kLDKHzu1/xjeBK7h8UTaatmYfVXhC9cQMn//uWai808KZeeavl6G3ZdOH6xcRw6UUt+SyCR2hCibFWYvYMwtloZ1VSNMeVmImOSaD1V7pG5D6S/E8nFM7mai1qtuuqPZ5fGCLy9vUlJSSE7u+PkraysLFJTU69476FDh0hJSenyWElJCX/4wx+AjvGD4uJiJk2a5Eoo/dYJrZ5/DE8izZzPvRV/Y35lHv8InEBF1kYcKtWgOtXMP2YMle++TcG23Wzdd5o7q3cRaauizDcMf3szO4MTWGjKY8rQBvzj42VFthC9zOVtqJ999lnWr1/PnDlzyM/P5/HHHwfgo48+4o033nDed+bMGQwGQ5fXLl++HKvVyty5c3nsscf4zW9+4xw/GCxOmxrYPSyesiFhRDdX0agZwrS6QnZcLAbeo0a7O8QbpuH4CQqMyTjWv8+YPZ8x8fwxjvmNoHxoGFnGNKbVFXJkxCQiWizuDlWIAUmlKEq/nIPR37uGDp2oZdOHW5lfmYcDFf4OG2bvIPztzWQZ09AF+bI41ofQORlXfL0n5eKqzpXD+cdrCNryZ076dewoavEKYIijlSxjGhX+YSR51fGjcT4Mz7jyn4WnGAh/J50kF8/UV11DsrLYTWJazSys/op/BCfwA+th7IC+7RylvmEA3FbyORU3389A3l2/cx+mrwy3Mzw4gRmWfOyoCGlvYFtoCo7RY1ieNobE6FDU6itPRhBC9JwUAjdpPVVO+MIF3Ja1kX+E3MwP6g6Do50oWxUjK038NXwmTcccTPUuZWZoM22nTw24vvHOfZgWntmGBgd21HjhoMg/mml1hRxTjSEpZrK7wxRiwJOjKt0kJH0OakUhYuECbqsv4lPjdHYHJ6ACvHAwrrGc2nobh7/YTdkbr1N1sABL9hZ3h92rOvdhqvYNxVuxo6BiR3Ai0U0V7ApJIE47MJrzQng6KQRu1FkMolesQBfkS/L5Egr9owGYeP4YPzr7NzKrtqNWHNjLT5JXUD2gikHnPkzGFgttaLCr1ZwaYmRTeBo/qC8iauIEd4coxKAgXUNuFpI+h6ajxcw+/SVN997PnmMOTOUHmGHJJ8pWhR1oU3vzj+CbmVa2l4pb+ue4wdnXX2VIbBxnYm5xbilt2PkZmWe+4pvAGL7xi0SjUXFPVR4sWkZUeJIc8SnEDSKFwANcetjKVO9SDleFOPvLNUCtZijT6grJMqYxyddAf9y3dEhsHLUf/4U9upPsCYxncv0R7qjJJyj1Dial/xCfY2ZGGgIY03ozrafK8Y+fKkVAiBtECoEHuHQQeGZoM2Oqt+NQqWhDg0axY2g7R+mQMKoCwhlp6J9L5U9Hp7B7+EnurMknpuE0I2xmvhyewpQpGdwVbyRK17mFuezDJMSNJmMEHuZC/j58vVQoag15Icm0qr2xoyKquYrZ9pMkRvefjqGzr7+KJTeHQydq2fB1KfuC4qnTDGWkzcxZXz37guLZV2J2d5hCDHrSIvAwWr2ewCnT8NIbmLHlMw5PyQRgQvVhEk7vwXZsmscfy9h5nsCl3UFq7xAeNO8m2H4Bq2YoI2xmUuqOgGK49hsKIfqUFAIP09lNZM3JZuTyf2Gc80s/naajxf3ikPZWsxlr9mb23TyXutAUZtTkowAq4JjfCDaE30lK3RFmWPJpbTACU9wcsRCDmxQCD3WlxWOefkh7Z0tgaMot1OzYwfi9WZzwG4lCRx9kg9qXDeF3AnBo+AQMwb5Mrjnl1piFEFII+r3OL99LC0Rny+FGr0TuPFQmf8Icjhqnc2/lF0xoLEUBTN5B6NvOkVJ3hOqxt5CZNobE6DTZOkIIDyCFoJ/r/PJtmvfPlPsaiLSZ8Nv8IeGPLL/xsYyN5eub7iJ530a8h0SgwQGAAxXbdLeib7F2dAf5GkmQrSOE8Bgya6if6/zydax/H0vWBhzr3+frm+7Cd2zsDY+loNTCXlsgJ4d07CKqAGW+YbSpvMis3k6tbwhFMT8gWLqDhPAo0iLo5zq/fBk2jtvqCtgRnMheWyDJpZZLzkjuHe3tDjbvLOd4xTluighi3rRIvLy+/S1x2tSAsb6C2AunaEdNu1rDrpAEADKrt5MZeo74f3lUuoOE8DBSCPq5zi/f5PoSdgQnklxfwukhRs6YRrlUCBwOhUMna8kvNqOgMHyYL7X1NhL2fEKREsruwI7FXkdPncPyeS6zhzcz8ol/AyDSZmKUKY+igGiK/TsOmF9oyuPoLQuI+tHjtJ0+JUVACA/U40Lw+uuvo9FoWLFixWXPtba28tRTT1FUVISvry8vv/wyY8aMQVEUXnrpJbZv345areY//uM/Bt1Rlb2l88s3y5jGaT8jp4YYWWjKQ22LBqK63Nt5EMwJrd65309Mq5nWU+WEpM/Bkr2F/N1H2K2KwGirJai1AZM2kNC2euraW0lryiemvpzjAaOwo+Z2Sz6lI2cx8uL7R7RY2JSYwV5bIK1tdrTeGg4aMpgfrhAwfjyMH3/D/3yEENfmciFoaGjgxRdfZMuWLfzsZz+74j0ffPABQ4YMIScnh3379vGrX/2K9evX8/nnn3Py5Emys7M5deoUDz/8MNnZ2Xh5SQOluzq/fE22QFRtdkyBER1fvlc41rHzIJgtYWmUavVEt5qZX5VH1MUiXuETSlj1Me7hGP8IvpnxjaV4K3baVF58FZJEZFMVEa21aM+3o2s7x7bQFC4ExpF28f1D52SwzKGQXGrhjKmBkYYAOVRGiH7A5W/ebdu2ERkZyf333/+99/z973/nscceA+CWW27BarVSWVlJXl4ec+bMQa1WExUVRVhYGAcPHuSWW25xNZxBqztfvp0HwWSc3c7BwHEk15ewacR0MrR6koByXwMHjNPJrNpOmvUgqounmKoVB2mWg9jVGuo0w9C3neOMr5784HjmRQR1+Qy1WkVSzPBeH58QQvQdl2cNLVy4kJ///OdoNJrvvcdsNqPT6ZzXOp2O6upqzGYzer3+sseFazq/fOfdFkVSzPDv/QXeeRDMwcCOgeWDgeMo1eo5Y+o4AGaUIYDKgHD2B8XhrdjxwsFZXz1eOPDGTqXPcHQXi8AIm5lpDUeZNy3yBmYqhOgL12wR5OTk8OKLL3Z5LDo6mj/+8Y/XfHNFUVCpVF2u1Wo1Dofjio93x9UOYr4ana5/7t55Jd3NJWGsnqIv93QZWK4aFs6EsZPR6QK4M9Sfgm27STpeTJtKg0pRGGEz044aFQpRzVUU6+Ion3Q3fuePkrorF8eeOHQL59/wXDzVQMkDJBdP1Re5XLMQpKenk56e7tKbGwwGzGYzo0aNAqC2tha9Xo/RaMRs/nbXyc7Hu8NiacThULr1Gp0ugJqagXH8oSu56GpOMa8yj00jplOq1VMZEM78yjx0NYnU1Ayl6WgxUws/w+Gt4WTMbUQW7wCHAzQamn39GWo7z4TGcu6aoMUv9kdYRwRi3ncQ7W3Tb3gunmig5AGSi6dyNRe1WnXVH899OjqblpbGxo0bSUlJIT8/Hx8fH8LDw0lNTeWTTz5h7ty5nD17lvLychISEvoyFAG0nionasUKMi52B400JBLVmug8CcxWVsawW6cQcOtkdGVltIXehrfRSGtVNVq9Ht+oKM7v2ePc+C7krrsJuetud6clhOihXi8EH330EWazmccee4yf/OQnPPPMM2RkZKDVannppZcAuPvuuykoKGD+/I4uhRdeeAFfX9/eDkV8R+feQ0lwyWDutwfBXLo30fdtbufJm94JIVyjUhSle/0rHkK6hiQXTzNQ8gDJxVP1VdeQ7DUkhBCDnBQCIYQY5KQQCCHEINdv93RwdduCgbTdgeTieQZKHiC5eCpXcrnWa/rtYLEQQojeIV1DQggxyEkhEEKIQU4KgRBCDHJSCIQQYpCTQiCEEIOcFAIhhBjkpBAIIcQgJ4VACCEGOSkEQggxyA2qQpCfn09mZibz5s3jkUceob6+3t0huWz//v0sWrSIBQsWsGzZMioqKtwdUo+9/vrrrF271t1huGTz5s3MmTOHu+66iz/96U/uDqfHGhsbnQdH9WdvvfUWGRkZZGRkOM9D6a/eeOMN5syZQ0ZGBu+//37vvrkyiMycOVM5fvy4oiiK8l//9V/KK6+84uaIXDd9+nSluLhYURRF+etf/6o88sgjbo7IdefPn1d+9atfKYmJicqbb77p7nC6rbq6Wpk+fbpSV1enXLhwQZk3b57z/7P+6NChQ8rcuXOV8ePHK2fOnHF3OC7bsWOH8qMf/UhpaWlRWltblaVLlypbt251d1gu2bNnj7J48WKlra1NaW5uVqZPn66cPHmy195/ULUIsrOziYmJoa2tDZPJxLBhw9wdkktaW1t57LHHiI2NBWDcuHFUVVW5OSrXbdu2jcjISO6//353h+KSnTt3MmXKFIKCgvDz82P27Nnk5ua6OyyXrV+/nmeffbbb54h7Gp1Ox+rVq9FqtXh7ezNmzBgqKyvdHZZLbr31Vv73f/8XLy8vLBYLdrsdPz+/Xnv/QVUIvL29KSkpIS0tjT179pCRkeHukFyi1WpZsGABAA6Hg7feeouZM2e6OSrXLVy4kJ///OdoNBp3h+ISs9mMTqdzXuv1ekwmkxsj6pkXXniBlJQUd4fRYzfddBNJSUkAlJeXk5OTQ1pampujcp23tzdvvvkmGRkZTJ06FYPB0GvvPSALQU5ODqmpqV3++elPfwp0/HreuXMnv/jFL3jiiSfcG+h1uFoura2t/PKXv6S9vZ2HH37YvYFeh6vl0p85HA5Uqm+3+VUUpcu1cK/jx4/zwAMP8O///u9ERka6O5weWblyJbt27aKqqor169f32vv22/MIriY9PZ309PQuj7W0tPDFF184fznPnz+f3/zmN+4Ir1uulAvAhQsXePTRRwkKCuKdd97B29vbDdF1z/fl0t8ZjUby8/Od1zU1Nf2+W2Wg2L9/PytXruTJJ5/stz0AACdPnqS1tZW4uDiGDBnCXXfdRUlJSa+9/4BsEVyJl5cXa9asoaioCOj4dTpx4kQ3R+W6VatWMXr0aF5//XW0Wq27wxnUpk2bxq5du7BarTQ3N7N161ZSU1PdHdagV1VVxfLly3n55Zf7dREAOHv2LE8//TStra20traybds2Jk2a1GvvPyBbBFei0Wh47bXXeOaZZ7Db7RgMBl544QV3h+WSI0eOsG3bNmJiYrjnnnuAjn7p3/3ud26ObHAyGAw88cQTLF26lLa2NhYtGK4axwAAAHlJREFUWkRiYqK7wxr03nvvPVpaWvjP//xP52OLFy/mxz/+sRujck1aWhoFBQUsXLgQjUbDXXfd1avFTU4oE0KIQW7QdA0JIYS4MikEQggxyEkhEEKIQU4KgRBCDHJSCIQQYpCTQiCEEIOcFAIhhBjkpBAIIcQg9/8D+9tzho3eshMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "net.eval()\n", "y_pred = net(x_test).detach()\n", "plt.plot(x_test.numpy(), y_pred.numpy(), 'b.', ms=10);\n", "plt.plot(x_test.numpy(), y_test.numpy(), 'rx');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The typical error in $y$ is ~0.01, which we could have predicted from the final loss ~1e-4 shown above:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "std(dy) = 0.008.\n", "MSE = mean(dy ** 2) = 0.00007.\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD7CAYAAACYLnSTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASVklEQVR4nO3de2wUZfvG8astbcG3NWjdiqJi4jExoDFirRiQxIClLMWCiagoYoMQD4AHAnjoH4qgCZZXCUZjowkCCgoWEIiHKhFrUIiBKGjQiBHBsn1RaTl0e3h+f/ijyqGyOzO7y979fhKSzu4+M/fcO3Oxme48zXDOOQEA0lpmqgsAAPhHmAOAAYQ5ABhAmAOAAYQ5ABhAmAOAAYQ5ABjQLZUb/9//mlK5+VNOQUEePTkB+nI8enI86z3JzMzQGWf8p9PnUxrm7e3cr3QsenJi9OV49OR4XbknXGYBAAMIcwAwgDAHAAMIcwAwgDAHAAMIcwAwgDAHAANS+j1z2JZ/eg91z/V2iB1ublXj/kMBVwTYRZgjYbrndlP4kRpPY1fNLVNjwPUAlnGZBQAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwICYwrypqUnDhw/Xrl27jnr8zTff1NixYxNSGAAgdicN8y1btmjMmDHauXPnUY//8MMPevXVVxNVFwAgDicN86VLl6qyslKFhYUdj0WjUT311FN66KGHElocACA2J52bZdasWcc9NnfuXI0aNUrnnXdeQooCAMQn7om2Pv/8c+3Zs0czZszQxo0bfW08FMr3Nd4ievKXaEvbUb2Ipy/RljblZGcloqxTCsfK8bpyT+IO89WrV2vHjh0qKyvTwYMH1dDQoClTpmjevHlxbzwSYV68fwqF8k31xM+JlZOd5WvGRUt9PBFrx0oQrPckMzNDBQV5nT4fd5jPnj274+eNGzdq/vz5noIcABAcvmcOAAbE/Mm8trb2uMeKiopUVFQUaEEAgPjxyRwADCDMAcAAwhwADCDMAcAAwhwADCDMAcAAwhwADCDMAcAAwhwADCDMAcAAwhwADCDMAcAAwhwADCDMAcAAwhwADCDMAcAAwhwADCDMAcAAwhwADIg5zJuamjR8+HDt2rVLkvT2229r+PDhCofDmjFjhqLRaMKKBAD8u5jCfMuWLRozZox27twpSfrpp59UXV2tt956SytXrlR7e7sWL16cyDoBAP8ipjBfunSpKisrVVhYKEnKyclRZWWl8vLylJGRoUsvvVS7d+9OaKEAgM51i+VFs2bNOmq5d+/e6t27tyRp3759WrRokWbPnh18dQCAmMQU5p2pr69XRUWFRo0apaKiorjHh0L5fjZvEj0JRlfoY1fYx3h15Z54DvMff/xRFRUVGjt2rMaPH+9pHZFIo9fNmxQK5ZvqSSpPLEt9PBFrx0oQrPckMzNDBQV5nT7vKcybmpp07733asqUKRo5cqTn4gAAwfD0PfN33nlHDQ0Nev3111VWVqaysjL997//Dbo2AECM4vpkXltbK0kaN26cxo0bl4h6AAAecAcoABhAmAOAAYQ5ABhAmAOAAYQ5ABhAmAOAAYQ5ABhAmAOAAYQ5ABhAmAOAAYQ5ABhAmAOAAYQ5ABhAmAOAAYQ5ABhAmAOAAYQ5ABhAmAOAAYQ5ABgQc5g3NTVp+PDh2rVrlySprq5O4XBYQ4YMUVVVVcIKBACcXExhvmXLFo0ZM0Y7d+6UJB0+fFgzZ87UggULtGbNGn3zzTdav359IusEAPyLmMJ86dKlqqysVGFhoSRp69at6tOnj84//3x169ZN4XBY69atS2ihAIDOdYvlRbNmzTpqee/evQqFQh3LhYWFqq+vj3vjoVB+3GOsO1FPoi1tysnO8rQ+P2ODGJ8q6Xhsxdvrf+5jur5PQUvH9z0oMYX5sdrb25WRkdGx7Jw7ajlWkUijl82bFQrln7AnoVC+wo/UeFrnqrllvvrsd9upko7HVirfZws6O3+syMzMUEFBXufPe1lpr169FIlEOpYjkUjHJRgAQPJ5CvMrr7xSP/30k37++We1tbVp9erVGjhwYNC1AQBi5OkyS25urubMmaMHH3xQzc3NGjRokG6++eagawMAxCiuMK+tre34ubi4WCtXrgy8IABA/LgDFAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwABfYV5TU6PS0lKVlpbqueeeC6omAECcPIf5oUOHNGvWLC1cuFA1NTXatGmT6urqgqwNABAjz2He1tam9vZ2HTp0SK2trWptbVVubm6QtQEAYtTN68C8vDxNnjxZJSUl6tGjh/r376+rr746yNoAADHyHObfffed3n33XX3yySfKz8/Xo48+qurqalVUVMS8jlAo3+vmzUpET7pin9nnrqkr98BzmG/YsEHFxcUqKCiQJJWXl2vx4sVxhXkk0uh18yaFQvkn7InfA9RPn9P15EjHYyuV77MFnZ0/VmRmZqigIK/z572u+PLLL1ddXZ0OHjwo55xqa2vVt29fr6sDAPjg+ZP5DTfcoG3btqm8vFzZ2dnq27evJkyYEGRtAIAYeQ5zSZowYQIBDgCnAO4ABQADCHMAMIAwBwADCHMAMIAwBwADCHMAMIAwBwADCHMAMIAwBwADfN0BilNftKUtbSfLSoX803uoe6730+Jwc6sa9x8KsCIgNoS5cTnZWQo/UuN5/Kq5ZQFWc+rrntvNd7/sztuHUxmXWQDAAMIcAAwgzAHAAMIcAAwgzAHAAMIcAAwgzAHAAMIcAAzwFea1tbUqLy9XSUmJnnnmmaBqAgDEyXOY//LLL6qsrNSCBQu0cuVKbdu2TevXrw+yNgBAjDzfzv/hhx9q2LBh6tWrlySpqqpKubm5gRUGAIid5zD/+eeflZ2drYkTJ2rPnj268cYbNWXKlLjW0ZUmgIq2tCknO+ukr+tKPUmkVPUxlRObcex07R54DvO2tjZt2rRJCxcu1GmnnaZJkyZpxYoVKi8vj3kdkUjXmZIoFMr3PIFTV5vsKghejy2/YeBnYjO/73NXOp9OJBTKN92DzMwMFRTkdf681xWfddZZKi4u1plnnqnu3bvrpptu0tatW72uDgDgg+cwHzx4sDZs2KD9+/erra1Nn332ma644oogawMAxMjzZZYrr7xSFRUVuv3229XS0qIBAwZo1KhRQdYGAIiRrz9OMXr0aI0ePTqoWgAAHnEHKAAYQJgDgAGEOQAYQJgDgAGEOQAYQJgDgAGEOQAYQJgDgAG+bhoCTkWpnLkQSBXCHOakcuZCIFW4zAIABhDmAGAAYQ4ABhDmAGAAYQ4ABhDmAGAAYQ4ABhDmAGBAIGH+3HPPafr06UGsCgDgge8w/+KLL7RixYogagEAeOQrzP/44w9VVVVp4sSJQdUDAPDAV5g/9dRTmjp1qk4//fSg6gEAeOB5oq1ly5bpnHPOUXFxsZYvX+5pHcxsBwQnHc+naEubcrKzAhubjj0IiucwX7NmjSKRiMrKyvTnn3/q4MGDevbZZzVz5syY1xGJNHrdfNrpygcZkiMdz6dQKN/XDJf/3OdQKD8texCrzMwMFRTkdfq85zB//fXXO35evny5vvzyy7iCHAAQHL5nDgAGBPLHKcrLy1VeXh7EqgAAHvDJHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAMIMwBwADCHAAM8PU3QOfPn6+1a9dKkgYNGqRp06YFUhQAID6eP5nX1dVpw4YNWrFihd577z19++23+vDDD4OsDQAQI8+fzEOhkKZPn66cnBxJ0kUXXaTdu3cHVhgAIHaew/ySSy7p+Hnnzp1au3atlixZEkhRAID4+LpmLkk7duzQfffdp2nTpunCCy+Ma2wolO938wD+X1c8n47d567YgyN8hfnmzZv10EMPaebMmSotLY17fCTS6GfzaaUrH2RIjnQ8n/yeF//c51AoPy17EKvMzAwVFOR1+rznMN+zZ4/uv/9+VVVVqbi42OtqAAAB8Bzm1dXVam5u1pw5czoeu+222zRmzJhACgMAxM5zmD/xxBN64okngqwFAOARd4ACgAGEOQAYQJgDgAGEOQAYQJgDgAGEOQAYQJgDgAGEOQAY4HuirWTLP72Huud6L7s52qbcnCxPYw83t6px/yHP2wZORak8p1LFzz773d9E5UjahXn33G4KP1LjefyquWWex6+aWya70/igq0r1OZUKfvbZz/4eGZ+IHOEyCwAYQJgDgAGEOQAYQJgDgAGEOQAYQJgDgAGEOQAYQJgDgAGEOQAY4CvMV61apWHDhmnIkCFatGhRUDUBAOLk+Xb++vp6VVVVafny5crJydFtt92moqIiXXzxxUHWBwCIgecwr6ur03XXXaeePXtKkoYOHap169bpgQceiHkdmZkZnrZdeEYPT+OCGO+1Zr/bTdXYVG47HcemctupOjb9jg9yn+PpQSqPES/v1cnGZDjnnJdiXnnlFR08eFBTp06VJC1btkxbt27V008/7WV1AAAfPF8zb29vV0bG3/9TOOeOWgYAJI/nMO/Vq5cikUjHciQSUWFhYSBFAQDi4znMr7/+en3xxRfat2+fDh06pA8++EADBw4MsjYAQIw8/wL07LPP1tSpU3XXXXeppaVFo0ePVr9+/YKsDQAQI8+/AAUAnDq4AxQADCDMAcAAwhwADCDMAcCAhIb57t27dccdd+jmm2/WpEmTdODAgeNeE41G9dhjj6mkpES33HKLfvzxR0nSgQMHNHnyZIXDYYXDYb3//vuJLDVp/PTkiO+//16lpaXJKjmhTjZZ2/bt21VeXq6hQ4fq8ccfV2trq6TY+piuvPbkiHnz5umll15KVrlJ4bUnmzdv1ujRo1VWVqa7775bv/76a7JLTx6XQBMmTHCrV692zjk3f/589/zzzx/3mtdee809+eSTzjnnvvzyS3frrbc655x74YUX3Jw5c5xzzjU0NLgBAwa4SCSSyHKTwk9PnHNuxYoV7oYbbnCDBw9OTsEJ9Ntvv7nBgwe733//3R04cMCFw2G3Y8eOo15TWlrqvv76a+ecczNmzHCLFi1yzsXWx3Tkpyf79+93M2bMcP369XMvvvhi0mtPFD89GTx4sNu+fbtzzrlly5a5iRMnJrf4JErYJ/OWlhZ99dVXGjp0qCSpvLxc69atO+51n376qUaMGCFJ6t+/v/bt26fdu3fr2muv1dixYyVJBQUF6tmzpxoaGhJVblL47UljY6M+/vhjvfDCC0mtO1H+OVnbaaed1jFZ2xG//vqrDh8+rKuuukrS3/2KtY/pyGtPJOnjjz/WhRdeqHvuuScltSeK155Eo1FNnjxZl19+uSTpsssu0549e1KyD8mQsDD//ffflZeXp27d/rovKRQKqb6+/rjX7d27V6FQqGM5FArpt99+04ABA3TuuedKktasWaNoNJr20+v67Ul+fr5eeuklnXPOOUmrOZGO3c/CwsKj+nGiPtTX18fcx3TktSeSNHLkSE2YMEFZWVnJKzgJvPYkJydHZWVlkv6aS2r+/Pm66aabkld4knm+A/Sf1q5dq9mzZx/1WJ8+fY6beOtEE3G5Yybocs4pM/Pv/2PWrl2rZ599Vq+99lrHyZsOEtkTK042WVtnzx/7OunEfUxHXntimd+eRKNRTZ8+Xa2trbrvvvuSU3QKBJKOJSUlKikpOeqxlpYWFRUVqa2tTVlZWZ1OxHX22Wdr7969uuCCCyRJDQ0NHa9buHChqqurVV1drcsuuyyIUpMmUT2xpFevXtq0aVPH8rH9OHYytyN9OPPMM9XY2HjSPqYjrz2xzE9PDhw4oEmTJqlnz556+eWXlZ2dnbzCkyxhH/eys7N1zTXXaM2aNZKk995774QTcQ0aNEg1NTWSpE2bNik3N1fnnnuuPvroI73xxhtasmRJ2gV5Z/z2xJqTTdbWu3dv5ebmavPmzZKkmpoaDRw4MOY+piOvPbHMT08ee+wx9enTR/PmzVNOTk5K6k+aRP52ddeuXe7OO+90JSUlbvz48e6PP/5wzjm3ePFiN2/ePOecc4cPH3bTpk1zw4YNcyNHjnTffPONc865cDjsBgwY4EaMGNHxb+vWrYksNyn89OSIX375xcS3WZxzbuXKla60tNQNGTLEvfrqq8455yoqKjre6+3bt7tRo0a5oUOHuocfftg1Nzc75zrvowVee3LEiy++aOrbLM5568m3337rLr30Ujds2LCODKmoqEjlbiQUE20BgAH2fqsGAF0QYQ4ABhDmAGAAYQ4ABhDmAGAAYQ4ABhDmAGAAYQ4ABvwfNw9LwemUpbsAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "dy = (y_pred - y_test).reshape(-1).numpy()\n", "plt.hist(dy, bins=20)\n", "print(f'std(dy) = {np.std(dy):.3f}.')\n", "print(f'MSE = mean(dy ** 2) = {np.mean(dy ** 2):.5f}.')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Preprocessing" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our input data is uniformly distributed on [-3,+3], which works well for this learning problem:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAARa0lEQVR4nO3df2xT9b/H8Ve3dhMuEHS2YIRAvkCcIcIiiE7NJgkORplgJXH4x/wRww8RCPkCwbGMECX8EDNEghpDSCT+MxDcXMZAwSzCCGRLlCBICGEEcJYyFRhs3dae+wff23u5IqynZ3T78HwkJrTrOed9tvnc2el66rIsyxIAwEgpyR4AANB9iDwAGIzIA4DBiDwAGIzIA4DBiDwAGIzIA4DB3Mke4P/788/rikbj/9P9jIx+am5u6YaJ7j32pecxZT8k9qWnsrsvKSkuPfjgf/3jx3tc5KNRy1bk/2dZU7AvPY8p+yGxLz1Vd+wLp2sAwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGA97u/ke6P+A/rogXRnP5Veb/+7PqYt3KlrV1sd3S4AsxB5BzyQ7lbBvyvu+Xa//Wi6rt3zrQLoTThdAwAGI/IAYDAiDwAGI/IAYDAiDwAGI/IAYDAiDwAGI/IAYDAiDwAGI/IAYDAiDwAGI/IAYDAiDwAG4yqUAPAf3XHZ8K5q74h0y3qJPAD8R7IuGy7dvHR4d+B0DQAYjMgDgMGMOV3T3hHp0lvmAXYk8/uLt3lEIroU+ZaWFhUWFuqzzz7TkCFDVFdXpzVr1igcDis/P1+LFy+WJJ08eVIrVqzQ9evXNX78eK1atUpu9735OZLmSTXuXBp6jmR/f/E2j7Drrqdrfv75Z82aNUuNjY2SpLa2NhUXF2vLli2qrq7W8ePHVVtbK0launSpSktLtXfvXlmWpfLy8m4dHgBwZ3eNfHl5uVauXCmfzydJOnbsmIYNG6ahQ4fK7XaroKBANTU1unjxotra2pSVlSVJCgQCqqmp6d7pAQB3dNdzKatXr77l9qVLl+T1emO3fT6fgsHg3+73er0KBoMOjgoAiFfcJ8yj0ahcLlfstmVZcrlc/3h/vDIy+sW9zP2sNzzZ3Btm7Omc/hx2ZX3tHRGleVId3W5XhTsiSu/itk36/uqOfYk78oMHD1YoFIrdDoVC8vl8f7v/8uXLsVM88WhublE0asW9nElf6HiEQj37KTmvt3+Pn7Erkv395eTnsKtfE6+3f1KfbE7GtpP9RxR2vs4pKa47HhzHHfmxY8fq7NmzOnfunIYMGaKqqiq98sorevTRR5Wenq6GhgaNGzdOFRUVysnJiXtg4E6S+bJzoDeK+/+W9PR0rV27VgsWLFA4HFZubq6mTJkiSdqwYYNKSkrU0tKi0aNHq6ioyPGBcX9L1svOk32EB9jV5cgfOHAg9u/s7GxVVlb+7TGZmZnauXOnM5MBABLGZQ0AwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMxqtKejGucQ7gboh8L8Y1zgHcDadrAMBgRB4ADMbpGtgSz/MByb6CI3A/I/KwJVnPB3ChMCA+RB7o4brjr6j47er+QeSBHo7fmpAInngFAIMReQAwGJEHAIMReQAwGJEHAIMReQAwGJEHAIMReQAwGJEHAIMReQAwGJEHAIMReQAwGJEHAIMReQAwWEKRr6iokN/vl9/v17p16yRJJ0+eVCAQ0OTJk7VixQp1dnY6MigAIH62I9/a2qrVq1dr+/btqqioUH19verq6rR06VKVlpZq7969sixL5eXlTs4LAIiD7chHIhFFo1G1traqs7NTnZ2dcrvdamtrU1ZWliQpEAiopqbGsWEBAPGx/c5Q/fr106JFi5Sfn68+ffroqaeeksfjkdfrjT3G6/UqGAw6MigAIH62I//rr7/q66+/1g8//KD+/ftryZIlOnTokFwuV+wxlmXdcrsrMjL62R0JAHq17njvXduRP3jwoLKzs5WRkSHp5qmZrVu3KhQKxR5z+fJl+Xy+uNbb3NyiaNSKex7emBhAbxcKXYt7mZQU1x0Pjm2fk8/MzFRdXZ1u3Lghy7J04MABTZgwQenp6WpoaJB0869vcnJy7G4CAJAg20fyzz//vE6cOKFAICCPx6MnnnhCs2fP1osvvqiSkhK1tLRo9OjRKioqcnJeAEAcbEdekmbPnq3Zs2ffcl9mZqZ27tyZ0FAAAGfwilcAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDJRT5AwcOKBAIKD8/Xx988IEkqa6uTgUFBcrLy1NZWZkjQwIA7LEd+fPnz2vlypXasmWLKisrdeLECdXW1qq4uFhbtmxRdXW1jh8/rtraWifnBQDEwXbkv/vuO02dOlWDBw+Wx+NRWVmZ+vTpo2HDhmno0KFyu90qKChQTU2Nk/MCAOLgtrvguXPn5PF4NHfuXDU1NemFF17QqFGj5PV6Y4/x+XwKBoNxrTcjo5/dkQCgV/N6+zu+TtuRj0Qiqq+v1/bt29W3b1/NmzdPDzzwgFwuV+wxlmXdcrsrmptbFI1acc/THZ8cALiXQqFrcS+TkuK648Gx7cg//PDDys7O1kMPPSRJmjRpkmpqapSamhp7TCgUks/ns7sJAECCbJ+Tnzhxog4ePKirV68qEonoxx9/1JQpU3T27FmdO3dOkUhEVVVVysnJcXJeAEAcbB/Jjx07Vm+//bZee+01dXR06LnnntOsWbP0r3/9SwsWLFA4HFZubq6mTJni5LwAgDjYjrwkzZw5UzNnzrzlvuzsbFVWViY0FADAGbziFQAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMRuQBwGBEHgAMlnDk161bp+XLl0uSTp48qUAgoMmTJ2vFihXq7OxMeEAAgH0JRf7w4cPavXt37PbSpUtVWlqqvXv3yrIslZeXJzwgAMA+25H/66+/VFZWprlz50qSLl68qLa2NmVlZUmSAoGAampqnJkSAGCL7ciXlpZq8eLFGjBggCTp0qVL8nq9sY97vV4Fg8HEJwQA2Oa2s9COHTv0yCOPKDs7W7t27ZIkRaNRuVyu2GMsy7rldldlZPSzMxIA9Hpeb3/H12kr8tXV1QqFQpo+fbquXLmiGzduyOVyKRQKxR5z+fJl+Xy+uNfd3NyiaNSKe7nu+OQAwL0UCl2Le5mUFNcdD45tRX7btm2xf+/atUtHjx7VmjVrNG3aNDU0NGjcuHGqqKhQTk6OndUDABxiK/L/ZMOGDSopKVFLS4tGjx6toqIiJ1cPAIhTwpEPBAIKBAKSpMzMTO3cuTPhoQAAzuAVrwBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAYj8gBgMCIPAAZLKPKbN2+W3++X3+/X+vXrJUl1dXUqKChQXl6eysrKHBkSAGCP7cjX1dXp4MGD2r17t7755hv98ssvqqqqUnFxsbZs2aLq6modP35ctbW1Ts4LAIiD7ch7vV4tX75caWlp8ng8GjFihBobGzVs2DANHTpUbrdbBQUFqqmpcXJeAEAcbEd+1KhRysrKkiQ1NjZqz549crlc8nq9scf4fD4Fg8HEpwQA2OJOdAWnT5/WnDlztGzZMqWmpqqxsTH2Mcuy5HK54lpfRka/REcCgF7J6+3v+DoTinxDQ4MWLlyo4uJi+f1+HT16VKFQKPbxUCgkn88X1zqbm1sUjVpxz9IdnxwAuJdCoWtxL5OS4rrjwbHt0zVNTU2aP3++NmzYIL/fL0kaO3aszp49q3PnzikSiaiqqko5OTl2NwEASJDtI/mtW7cqHA5r7dq1sfsKCwu1du1aLViwQOFwWLm5uZoyZYojgwIA4mc78iUlJSopKbntxyorK20PBABwDq94BQCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMBiRBwCDEXkAMFi3RP7bb7/V1KlTlZeXp6+++qo7NgEA6AK30ysMBoMqKyvTrl27lJaWpsLCQj399NMaOXKk05sCANyF45Gvq6vTM888o4EDB0qSJk+erJqaGr377rtdWj4lxWV7274H+9heNlHJ2jb7bP52k7lt9vnestO/uy3jsizLsjvQ7Xz++ee6ceOGFi9eLEnasWOHjh07pvfff9/JzQAAusDxc/LRaFQu1//+ZLEs65bbAIB7x/HIDx48WKFQKHY7FArJ5/M5vRkAQBc4Hvlnn31Whw8f1h9//KHW1lbt27dPOTk5Tm8GANAFjj/xOmjQIC1evFhFRUXq6OjQzJkzNWbMGKc3AwDoAsefeAUA9By84hUADEbkAcBgRB4ADEbkAcBgxkS+vr5egUBABQUFmjt3rq5cuZLskWxraGjQzJkzNX36dL3++uu6ePFiskdK2MaNG/XJJ58kewxbTLrgXktLi6ZNm6YLFy4ke5SEbN68WX6/X36/X+vXr0/2OAn5+OOPNXXqVPn9fm3bts35DViGmDRpknX69GnLsizrww8/tD766KMkT2TfxIkTrZMnT1qWZVk7duyw5s6dm+SJ7Lt69ar13nvvWWPGjLE2bdqU7HHi9vvvv1sTJ060/vzzT+v69etWQUFB7Pust/npp5+sadOmWaNHj7bOnz+f7HFsO3TokPXqq69a4XDYam9vt4qKiqx9+/Yleyxbjhw5YhUWFlodHR1Wa2urNXHiROvMmTOObsOYI/nq6mqNHDlSHR0dCgaDGjBgQLJHsqW9vV2LFi1SZmamJOmxxx5TU1NTkqeyb//+/Ro+fLjefPPNZI9iy/+94F7fvn1jF9zrjcrLy7Vy5cpe/wp0r9er5cuXKy0tTR6PRyNGjNBvv/2W7LFsmTBhgr788ku53W41NzcrEomob9++jm7DmMh7PB6dOnVKubm5OnLkiPx+f7JHsiUtLU3Tp0+XdPM6QJs3b9akSZOSPJV9M2bM0OzZs5WamprsUWy5dOmSvF5v7LbP51MwGEziRPatXr1a48ePT/YYCRs1apSysrIkSY2NjdqzZ49yc3OTPJV9Ho9HmzZtkt/vV3Z2tgYNGuTo+ntd5Pfs2aOcnJxb/nvjjTck3Tzqraur0zvvvBO7CmZPdqd9aW9v15IlS9TZ2ak5c+Ykd9AuuNO+9GZccK/nOn36tN566y0tW7ZMw4cPT/Y4CVm4cKEOHz6spqYmlZeXO7puxy9r0N3y8/OVn59/y33hcFjff/997Ij3pZde0rp165IxXlxuty+SdP36dc2bN08DBw7Up59+Ko/Hk4Tp4vNP+9LbDR48WPX19bHbXHCvZ2hoaNDChQtVXFzca39rl6QzZ86ovb1djz/+uPr06aO8vDydOnXK0W30uiP523G73Vq1apWOHz8u6eZR5ZNPPpnkqexbunSphg0bpo0bNyotLS3Z49zXuOBez9PU1KT58+drw4YNvTrwknThwgWVlJSovb1d7e3t2r9/v8aNG+foNnrdkfztpKamqqysTKWlpYpEIho0aJBWr16d7LFsOXHihPbv36+RI0fq5ZdflnTzPPAXX3yR5MnuT1xwr+fZunWrwuGw1q5dG7uvsLBQs2bNSuJU9uTm5urYsWOaMWOGUlNTlZeX5/gPLi5QBgAGM+J0DQDg9og8ABiMyAOAwYg8ABiMyAOAwYg8ABiMyAOAwYg8ABjsvwGWOk2JN/m8CAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(x_train.reshape(-1));" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, the ability to learn a model can be quite sensitive to the input distribution. To demonstrate this, compare learning with different linear transformations applied to $x$:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "variations = {'x': x_train, '2x': 2 * x_train, 'x+3': x_train + 3}" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEJCAYAAABlmAtYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2deXxU9bn/3+fMZE8gCWSBBIIhLCogiJVNQHFrFRQt1oAVrHJtr2ItVnGjP62WatHqrVjFXqu21usOKlqRugthFWRRQhJCWJIQEhIC2WfmnN8fkxlmJjOZTDJbkuf9evkys53z5JvhfM73WRVd13UEQRAEoR3UUBsgCIIghD8iFoIgCIJXRCwEQRAEr4hYCIIgCF4RsRAEQRC8ImIhCIIgeEXEQhAEQfCKMdQGBJKamno0zfcykn794jl+vC4AFnUNsct3wtU2scs3xC7f8dU2VVVISorz+HqPFgtN0zslFrbPhiNil++Eq21il2+IXb7jT9vEDSUIgiB4RcRCEARB8EqPdkMJgiD4iq7r1NRU0tLSBLTvxjl2TEXTtOAY5iOebVOIjIwmKSkFRVE6fDwRC0EQBAfq6mpRFIW0tEwUpX3ni9GoYjaHp1h4sk3XNU6cqKKurpaEhMQOH0/cUIIgCA40NtaRkJDoVSi6K4qikpCQRGOjb1lcPXM1hKBjqSiieceHWCqKQm2KIHQJTbNgMPRsp4vBYETTLD59pmeviBAUmo7so+HD5aCZaFEjiJ25BENaTqjNEoRO44svvzvSmd9PdhZCl2k8+D1oJtB10MyYy/JDbZIgCH5GxMINTUf2iUvFB2KyzgY1AhQVVCPGgSNDbZIgCH5GxMIFS0UR5a89TMu2d2n4cLkIRgeIzhxB7MwlRJ53rbighF5NUWktH20soai01m/HfPvtN1i06FZ0XWfnzu/Izb2GhoYGvx2/o0jMwgVzWT66xezkUpGLn3cMaTmyTkKvpqi0lide34HZomE0qNwzdxw5GX27fNw5c67n66+/YPXqd3jnnTd44IGHiI2N9YPFviE7CxeMA0eiGIziUhEEwSf2HarBbNHQdbBYNPYdqvHLcRVF4f77/x/PPfcXpkyZxpgxY/1yXF+RnYULhrQcBtzwMFU/bMc4cKTcLQuC0CFGDE7CaFCxWDQMBpURg5P8duyjR8uJjY2joCAfXddDkq0lOws3RGeOIGrcTBEKQRA6TE5GX+6ZO45rpmX7zQUF0NDQwPLly/jTn54iKiqK1avf8ctxfUXEQhAEwU/kZPTlyklD/CYUAM899wyTJl3AmWeezV133csrr7xIWVmp347fUcQNJQiCEMbcffd99p/T0wfwwQefhMQO2VkIgiAIXhGxEARBELwiYiEIgiB4RcQiyEh3VkEQuiMS4G4HS0UR5rJ8v9VbWCqKpDurIAjdEhELFywVRdQUHKDFEkFz3v/59cJuLstv051VxEIQhO5AWItFdXU1y5YtIzY2lunTp3PJJZcE9Hy2O/8GzQwooGuA/y7sxoEjaVEjQDNLKxFBELoVYS0Wr776KgsWLGDMmDHceuutARcLpzt/FGt/KHS/XdgNaTnEzlziV9eWIAg9n5de+huff/4pAJMnT+G22+4Mug1hLRZVVVWkp6cH7Xyud/5Rk+ehN9X59cIeyO6s/o6xCILgG4H4N7h162a2bt3Eyy+/hqIo/Pa3d/DVV18wffpFfjl+RwlrsUhPT6eyspLU1NSgnM925x9Ve4Dmvmd0qwtuy94vaV7/KugaLQYJngtCsAlUAku/fv25/fbFREREAJCVNYSKiqPk5l7LP/7xOpWVx7jjjl/yt7+9QkpK4K6VYS0W1113HcuXLyciIoLc3NygnNOQlkPSqHFUVp4Kyvn8gaWiqFUoWgewW0wSPBeEIBOoBJbs7KH2nw8fPsTnn3/K88//nYKCfF599WW2b9/G7bffGVChgBDVWdTV1TFz5kyOHDlif27NmjVcccUVXHbZZbz22msApKam8uSTT/LYY48xY8aMUJjaLTCX5Z8WCgBFleC5IAQZ48CRAR0vXFy8n8WLb+f22+9k0KDB3Hnn3axZ8x7Jyf245JLL/XoudwR9Z7Fz506WLl1KSUmJ/bmKigqefvppVq1aRWRkJLm5uUyYMIGcnK6pcr9+8Z3+bEpKQpfOHSjc2dV01rmU7/gA3WwCVaX/5f9Fn1HjQm5XuBCutoldvhEsu44dUzEaO34fbXuvMWM4hqvvxVSaT0TGSIzpw/xm086d33H//fewePHdXHqpVRhOnqzBYDBw6FAJFouJqKgoj7a5Q1VVn9ZU0XVd9930zvPggw9yzTXXsGTJEv75z3+SmZnJ6tWr2bp1K3/84x8B+Otf/4qu6yxatKhL5zp+vA5N8/3XS0lJCLobqiOBsfbscvw84PFYgQjAhWK9Okq42iZ2+UYw7Tp69CDp6Vkdeq/RqGI2awG1p6LiKLfc8nN+//vHGD/+RwBYLBZuu20h8+bdyLffbiU6OrpNhpQ321x/T1VV2r3BDvrOYtmyZW2eO3bsGCkpKfbHqamp7Nq1K5hm2SkqreXLXeVk9ov1a0/69vBHYMyWZdXesZxeUwxEjJhKxPApXRINWxGjpZslBAhCd+H11/9Fc3MLK1Y8bX9u+vSLSE5OZvr0GZx33vnceOP1TJs2g1GjRgfMjrAIcGua5jQmMFRjA20D121jEf057coVxzt8fwbG2juWqWADWFqsb9TNmPZ+galgQ6ezNpyKGFWjZGAJQgD4zW/u5je/udvj63Fx8axa9VHA7QiLRoK2FFkbwUyXdcQ2cF3z88B1V2wX2ZZt79Lw4XKU6Hi/BcY8BdksFUWY9n3T9gOtgtIZTguT5nQcaZYoCD2PsNhZTJ48mRUrVlBdXU1MTAzr1q3j0UcfDbodgRy47ojr3b/eVOe3ym5PVeLuMqZslepKdOcSAdy1L+mIS81SUURLwQYU6LIbTBCE4BAWYpGWlsbixYuZP38+JpOJOXPmMGbMmKDbYRu4fuR4Q0BjFu4usv6s7HZ3LNdzRoy6BNOuT0DXaN7wL7Sqg6j9s3yqWHdXxNi848PTQmgx0VKwgRiHY1kqimhY87jVDsC07xtiZ90ngiEIYU7Qs6GCSThnQ3UmK8kXu9xlRynR8XYxMJfl07Lt3dbdhQuKgagLbiTyzAt9tstVDFAMRIycRsTwKQA0b3sPS+kep88bMkYRdd7sgAiGZPf4htgVftlQnaXbZ0OFO8W1B3mvZBeNTSYmDBhPVpMpIP2WAt0jyjHrCbC6oFzcQi1qBFhMgItg6Baa17+KITmzU1lZESOmYtr7hf1Ypr1fYNr7FagKaJY2n7GU7qGhbG8bgZJeV4IQPohYOFBce5D/2b4SS6tvf1P5Fv6rtJasxqawHVbk7oLqFBPRLdjFwCE7yuZCainYgHnfN60XcQfR0C00b3uvU3f8EcOntGZeOQqRBvabHAW1/xmggFZZfPp8DgLl1OtK9U+aryAInUfEwoHCmv12oQAw6xrFUQpZDf7p9eJ6Ye/qnbOnYLJTfAIF0K3C4ZJpZUjLISYtB8vwKZjL8tFbGlrjGNY1sJR+T8PRAp9F0kmI8r9qnQvi+IYIoqfMA6Dhg8dOB951C00b/g8lLhHLoe9Of07repqvIPQGLBYLf/7z4+zevRNVVbnhhpu47LIf++XYIhYODEsaikEx2AXDqKhkN+t+SWl1vbBHTZ7ncRJfR0XEU02F212DYiBq8jy3x3N0iUUMObc1rvA9XRn8ZBOilv5Z9h0CbnYIURfc6NQEUasqhioPB5XpgoLQLuvWfUx9fT2vvvoWdXW1/Oxn13LBBVOJjY3r8rFFLBzI7pvFb879FbtqHGIWZ/knZtHmwl68ze2F3pdq7vYm7xnSclCd0mV19KY6r3Ya0nKIOm82DUcLWt1InU+tBYg880IMyZke19D2urvAtxXVGutwszMShHCjuPYghTX7GZY0lOy+HQuSe+Ptt9/gq68+Z8WKF9i1ayePPfZ7XnrpNWJjY1m06FYefPBhBgwYCMBPfjKTSy+17iQqKyuJiDBiMPjnMi9i4UJ23ywm5Iw6nXnRF7/cyba5sGefh+VoQZsLvS/V3N4m73V2jKshLce682ndETTn/V+ngt2Ox2vvs3aBKs/3mEVlq8voLMW1B9lcvg1QmDBgvN/+IQuCjeLagzyz42+YNTNG1civx93ql+/ZnDnX8/XXX7B69Tu8884bPPDAQ8TGxnp8v9Fo5PHHH2Xt2o+48cZfuG0w2BlELNrB8S6hq1lR7i7s7u64rXfxKqB16ALf3oW4K2NcrbsQ/84gbw9DWg6xs+5zW6xnqSjCXLABNFOn4hZtExe2cee5vxTBEPxKYc1+zJoZHR2LZqawZr9fvmOKonD//f+P+fOv55prrmPMmLHcddciqqurKS09zD333InRGME99zzA2WePAuC++37HHXfcya9+tZDRo8/h/PMndtkOEQsP2O4STJoJFYWrK+uYcLKhS1lRrhd218eWiiJrHENvP8bQlXN2FKddiaKi1x3HUlEUcMGIcXP8rvbOapu44L9/yIJgY1jSUIyqEYtmxqAaGZY01PuHOsjRo+XExsZRUJCPrus89dSzAG3cUPn5e4mLi2PQoMH07ZvIxIlTKCoq9ItYhEVvqHAjv6SaNTu3YdJMAGjovN8/loNRhi71UvKG/aIIgI6l6mDIeizZdiXGkdMBMOV/ScOHy51ssfWAajqyL6C2dHWojC1xwX48xb//kAUBrC7sX4+7lZnZl/vNBQXQ0NDA8uXL+NOfniIqKorVq9/x+N4fftjDc889g6Zp1NfXs2XLRsaMOccvdsjOwoWi0lqefGMH5iiIHKmgqNY6AR3YHxPJ4BZTwIKsrnfz5n3fgG6x72ZICe5AI6cgeTuB+PIdHxBzZeBSWj250zqaNWZLXJCYhRBosvtm+f279dxzzzBp0gWceebZ3HXXvdx6601MnDiZgQMzePbZvzm99+qrr2X//kLmz8/FYFC55prrGDXKP62TRCxc2HeoBrNZQzMlYT54JpFn5GOtU1A4emIIz9YN4WfmFAJxWXS8KOp1xzHlf2m/SLcUbKCm1nluRDAqnO0C5pIZ5ega0i3BiWnYRKp5x4co0fEeU4/dEYh/xIIQDO6++z77z+npA/jgg088vtdgMHDPPQ8A/m9FImLhwojBSfZFVmqG8LPpP2JHWT67d8HmuiRUxSoo/mwy6HrRt10UTQUbnHYZNfmafW4E0OWBSR3BU2aU4y5IMQQnpdVxNwNqa9Ge+4aFgiD4FxELF3Iy+rLsV1PYtKuUEYOTyMnoy8CYTH7YsgMU/7cu91RX0d4uwx4z8dPAJG+4y4yKGjfTbl//s87lVNTAgJzbEadANxr26nR0zPu+wSLtQAQhYIhYuGHkkGT6xUVQVFrLRxtLGDE4iXvmjmPfoRq7gPiL9jJ93O4yHAK8namh6Aye6jVs9kWnJHAqCB1BXe0wDBqFpWS79UVdk+puQQggIhYesI1YNVs0jK0jVq+cNMTpPf6IGXSkcM7d3AjAbwOTvNGVeo1A2GGrxVD7Z2E5vCcogin0LkI12jlYdGYyhYiFB2wjVvXWEaubDuylsOYw2Y0mcgafD/gnZtCRC7FNlBLPOhezg7snkG3O3dkZzMB6e9gK9GjtseXLwCZB8IbRGEl9/Uni4vr0SMHQdZ36+pMYjZE+fU7EwgNOI1b71LK15T9YKswYdVj4n2/IyZrot5hBexf9YKaodgSn1uGG4Kf0uhtJGzVuptN7nAY/BTndWOj+JCWlUFNTSV3dCa/vVVUVTQvP4Uft2WY0RpKUlOLT8UQsPGAbsbrvUA21cT+QV2VBVxRM6GyPMzIUrIViDimlgbjjDnaKantYKoqcOsRisbZAYVTwLsje3HauCQNNiQ9DEILvQs/BYDDSv/+ADr03XCcLgv9tE7Foh5yMvuRk9KW4VmFT1df2lhHbEqKJb05kwlmzSdjzDugaTetftW5ZdQstiv+G9YQiRdUTZqcutoCiBt0eb247151H48HvYbiIhSB0FRGLDpDdN4tJA89nfdkmACzA2tJjVDWfIK1vNEObWhjUaOZgjJEDMTFkN7aQ5adhPY4Xx2ClqHrCOHAkLYZI625KUYm64MY2va2CFXD3FNvR6o6DYsDWiDEm62zc3VsFopW0IPRkRCw6yIQB49l89FtMmhldU9BNEew8owpFieMLPY6fVNbxcUo8ZgWMehwLy2rIavaP2yjYKart2eHprt6XORyBwKlgTzEQMXI6EcOnEJ05os2aBaqVtCD0ZKSRYAexNQmb0n86lqIJqBEmdEW3xjEUhY/69MekKq2P4dOkeA7GRPndTWNrdxGK5oLQWtE9bqZX90+gmi16wnnuuIYS369tR9/WdXPXSloQhPaRnYUPZPfNIntMFlPU79h3oIIPFRULGihgiWk+/UZFoSgukpL4GH4dHUG2n87fdGSf9e7Z0kKLYiDqghuJPPNCPx29a3R20FIwzu+66xk6Y0HAWkkLQk9FxMJHLBVFJG9+jkmaifL+CWzpE427EW46YNE1Cmv2d3lwko3Gg9+DpaX1BBaa17/apQl2/iTUhXvtnd911zPoRA2/HnerxCwEwQdELHzE8cIz7mQj2xJisKCjKKCgoCoqCgqabsGgGhmqR/nNlx+TdTY1iuF0RlKYtbgIZpGgK+0F193tOqQLrSD4hoiFjxgHjqQwJpr9UQqDGyz0LRxOZZzG2CEDyR4cw7CkoRyprGNHWT7jBo5k0IkDFEboFMfEkN1oJuLQFkoaD3XqjjY6cwRRF9xoL4rDECEtLvAeXA/1rkcQegIiFj5QVFrLpgNVbB2YiAULJCk05yeiHuvHFecmkl5zhKN1Nby29gSWaNh7cCM1I/vyxcBEzAooOnBqJ/qpnZ3Owok880K3s7t7Mx0Zu9peuxJ3uxJJrRUEZ8JeLH744QeWL1/OK6+8EpLzF5XWsu9QDXExEbz+aSF6aiHGDIvV7aTCqDEwOzmR5M3P0Wwx0QcDqXGTOJ5TCKrGulMKqNaghq6ANZoB5i4MdA+luycc8RTctlQUUVPQdmCU4y4kavI8+xClwphoDo+dQULSYN4p/EBSawXBgbAWi8OHD/Pll19iMBi8vzkAOHaeVRQFXdNRTibDABUMOkbVyKxzziO9eDfNFhMKOopuIaFPOdWq1hr49tzdseTkYV7PX+U05lPuaH3HnZvJJgoNrQJic0212YUUbwPNxMEoAy+mx2M+thWl8lt0XXdKrZW/hdDbCSuxePHFF1m/fr398UsvvcRtt93GL3/5y5DY49h5Fl1HVRX0+iQsRROYPiWKiVlnk903C8tAE42KATQLFlSi6uKJ0Gsx2Q7U2rlSRUVvHdqjoLCr6nsANpZtZdLAHzEoIYN3Cj/ApJlQFZWLB00jxhhNXEQc9aZ6zldGk4Rvzb96C667LU+uKdcxsWr/wViOFlAcE4FZaZV2XUNRVBQdSa0VhFbCSiwWLlzIwoULQ22GHafOswaVuZcMo77R1GYAkiEth8KRt7B/+1YKTWkMj6xgRtkJdiRE8W1CDBYFFIxclHYpcXE61U0n7K1DACxYWF+2CQUFvXUnouka/zn0pf09CgprD37GHWPFJdIR2hvY5DgmtqjwCw6NOZ8+Fh1jYzEWXcOgGpkz7CrqTfWywxOEVsJKLMINx86z7U3IKyqt5cXNzZjMo1BVhR+PG0zWvt0MrqxnXJ2Z/1PPpLImk0+2K9wz91yGJZ1gY/lWe2NCG3o7LisdHZPFzH8OfsGQPoPlIuYFm2vKcGgLTU0mp9dsY2IPRre6nk4VYFQjmDN8tgiEIHggKGJRV1dHbm4uK1euJDMzE4A1a9bw/PPPYzabWbBgATfccIPHz7/wwgvBMNMtts6z7WFzVwGg6+ytT+I/py7lDLWc+vooxijNFJrNHLJo7DtUw5WThvCbc3/F5vJtnGypY0/VXjRO95133GE4oqOzq+oHdlX9YHdTNZobAYVBCRkcPnUEUJxiIL2dut1foltMTk0dbbsOR9eTRTNTb6rn8iEzQm2yIIQlAReLnTt3snTpUkpKSuzPVVRU8PTTT7Nq1SoiIyPJzc1lwoQJ5OT4N8OnX7/4Tn82JSWhw++dOCaDNXklmM0aRqNKTEwkxS39MRt0bk9YhwELl2Hgbw2XM3HMVFJSEkhJGcWEnFEAfLr/G/7+7Rtouk6EwchN467jVHM9CVFxnGqup6i6hK2lO53O6eqmcmRj+VZmZE9m+pCJDO/vr2Yj7ePLegWLmoID7DVCcYK1xmVs7QGSRo2jQOnLzguvoH9tDREnCzHrGkbVwPlnjCalv/vfo6CqmO+PFXB26nC/rWk4rhmIXb4SrnaBf20LuFi89dZbPPTQQyxZssT+XF5eHhMnTiQxMRGAyy+/nLVr17Jo0SK/nvv48To0zfdZs74ODekXF8HduafdVQCfbj3EsIijGLBgUEBBY8G5Kv3iItoc+5w+Y/nNuUmns6D6OO8KBkZm8l35D/bmd96w6Bb+s/8bPivewPXDr2FgfHpAM6zCdQBMUUQsLw7s29oJGBZFxKIW7XHqODtn2NV211OSnmL/PRxrLw5GR/i9S224rpnY5Rvhahf4bpuqKu3eYAdcLJYtW9bmuWPHjpGScjqrJzU1lV27dgXalIDi6q66Z+44yvYaUEv2gG5BVY2kjRzr8fPttZ+wdbwtazlCVW0tnx3+Gk33PspR0zVe3/duaxaWjqqonN1vJH0iExiUkNHj/fP7lWYsqvV3tygK+5VmcOk46+p6slQU0VKwAfO+b6yDrNQINp07CZNmjXtIKq3QWwlJgFvTNKdB6Lqu97jB6FbxmIqlYoBfqq2z+2YxIWUUlZWnGJNyNoU1+4mLiLPHKWwxC3cxENvPFt1iT9e14S5Ft6cIyLCkoRgNRswW5+6ynjrO2gv2bM0agYMRsKnmdLt1VTEQFxHHJyWf95h1EoSOEBKxSE9PZ9u2bfbHlZWVpKamhsKUgBOIamvnXcgEh1esP68v3cybBas7vPtwjX2oisr1w6/hgowJ7j/UTcjum8X/u/A3bDmw2+nC7qnjrL02w4Hi2Ci77CrAWf2GS3W30CsJiVhMnjyZFStWUF1dTUxMDOvWrePRRx8NhSkBx9YupL3UW39zQcYEBsans7l8G5vKv8Wim7EVAjruODyh6Rpv7FvF98fz6ROZ0K2zq4b3zyZJdy5kdHX52armhyYmkWarzVBUIkZM5azMHD4v+RCzbsGoGADFrUtKKu+Fnk5IxCItLY3Fixczf/58TCYTc+bMYcyYMaEwJaA4tgsxGlTumTsuaIJhuyBOGHCe/SIGsLl8G6AQY4xuN/ZhTdM9XWF+dv+R3V443OE6YnXRjAUMOlFjdxueUVHEwtITFEcpxGoKH+h77Z9VFQPDkobKmFahVxA0sfj888+dHs+aNYtZs2YF6/QhwbFdiMWikbe73GmXEYxdh+tdtOPPjrGPelM9jeYmtwJiwdJjhWPf4S2YNZO91mK/0kzOuJn2181l+WQ1NpHVoPNFUpzTzmxC4jCy+2bxScnnbca0dvd1EQRXpII7gDi2C1FUhfW7yzFbrD2mLv/RID799khIdh023GVgjUk5m83l29hYts3aht0Fd8KR1jeZMYljut0F0lJRxKDvPseYHo9ZAYPaNnjt2Esqu9GMEcVal6HDmO+3YBk41RpIlzGtQg/H8PDDDz8caiMCRWNji7UJoI/ExUXR0NDi/Y1eSO4TzZlZSaQkxpDSN4bicmvOs67D/rJaNK21akLXSUmMYfigxKDY1R5J0YmM7n8WI5KHYVBUEqMSqWqo9lhRXtFQSXHNIbaUb+dE80niImJJim7/9wgmjmtmqSjCVJiHoqqo8cmYCvNIOLSboY3NJJt1zul3JquObSa/upCtFTsYnpRDv37ZKDEJWA7vJtFsZmiDiWSzmYtr6kHX2VZ3kH6x/Tk/cRhJTU38uM8IBlVX2c/REbvCCbHLN8LVLvDdNkVRiI2N9Pi6TzuLlpYWqqqqGDhwoC8f69XY6i+KSmv5ele5vUhQ18GgKui6jsGg2ov5wgXHXUdx7UF7a5Lvq/I97jjWl20ir3yLPRU3nIK97qbp2XYNWc1mskwmvjEoTi4pmzvJ1ksKdLKaWshqVjkYpfLiwETMSh2f7n+PheWnmNbYxMFoI5/ERJL9/UeceelvZe6I0GPwKhb/+c9/2LRpE4sXL+aqq67i1KlTLFq0iAULFgTDvh5DTkZffn7ZcF5bV4Cm6xjb6WIbbvgiHI6puOGUguuuZXnUuJn2ORhKdDyDv33TySVlr8tw6WAbNXkeJYe+wqzUoSsKJnQ+TYxiVAR82D8BkwKqDj898AUXilgIPQSvYvHCCy+wbNky1q1bx9ixY3nkkUeYP3++iEUnuHBsBpkp8UFPpfUn7oSjWWni29LdbdJybSm4h0+VhjwY3l7LckNaDs07PiSrsYmFZSaKY6IYMWSK3V53w5VGRkfw6f73MLW654piI9kfE4mmAIqChs67DUVoXz5NIxYS0kbQGBMXVrstQfAFr2Kh6zojRozgf//3f5k2bRrx8fHonQkECEDHuth2F2zCkZKSwOrvPnVbCKijh4V7yt0F3xFXl1Ts4POdaydciitzzpjCIuCjw19RaD6BjnVsrkLrACVFQdM1VlvKrBJaVoGiKBjVCH497lZSUkYF7XcXBH/gVSxUVeXf//4369ev59577+Wrr77qca05hK5jKwQsrNnvNgXX5p5SUFAVlUkDfhT03UZ71fSuYnIwOoJntq+0F+P9+txftbE154wpzErOdHrfhYOm8tmhr6wtbPTWueuKAro1RcCsmdhcvs3ecVgQugtexeLee+/l2Wef5a677iIlJYXnn3+epUuXBsO2XkMoqrwDgaOLypaCm1e21ck9paNj0U8Hw8MlpgHOYrJvz9vW2glFwayZ2Xd4i3WErkM3WkNaDllNJnvRXnazzvD0WHLKTlqL+CyaNYbhkEmmAxvLtqB+eITxqeeRc8aUEP22guAbXsXivPPO45VXXgGs2VBPPfWUZEP5kVBWeWmOEgMAACAASURBVAcSm3AMSsj02KcqnGIarmQ3mjDqYEbHqFsfu8uocizaQ1ExF287/RiFgTFpHOibwOG4WHafKgHAomt8XXeEjaeOcG1TNaVKCzK0Sgh3JBsqxLhWee87VNMjxMKGo3vK1iXXcbfhGNMIp11GzuDzWfifbyiOUslu1sgZfb7bjKo2gfPs87AcLbA/HjHxRs5Ky6HowAbyTx3EZIv3tWZRvVX+jX3fsal8G3ee+0sRDCEs8VqUd++99/LrX/+aTZs20dLSwj/+8Q8ef/xxrr/++iCZ2HlCXZTXEVRVYdP3FdBab3HuiBS+P1CNqiok94kOmV2+4M2upOhEchLPYHCfTEb3P4s+kQn8UL3PqdBPR+f74/nUNp/ya2FfZ9dMjU+mX9pIsiOSSBt7FYa0HBRVxVS4CdCtKbTjr7aOac04EyUhhajxVxMx5Fz747IzJ/KtuRpqykj77GWG1jdiQOdoZMTp39wh/qehcaLpBP1j+9OntsqpgDBYdNfvWKgIV7sgBEV5kg0VWHIy+nLPXOuUvbiYCF7/tLDHuaRcceyKG867DNeAuKeMKnfvOxgdwbO25oIoLIyErCYzWU11AGzuE2tNnQIc72j21hSy79sirq6sY8LJBru7S4r7hFDT4Wyob775RrKhAoRNEN5fX4zZrFmzZszWxoM9USyg/ZhGOMcyOjqfpNBxIh9QHBttF4pzT5nYnqBjxlq8N6KhmZNGlSNREQBous77/WNBt9BgMBC/dw2N+6MYFjeALC2iy4O0BKEzdDgb6re//a1kQ/kRxwwogCde34HJ7Jg1BN/sKgNg8ugBPVY0ussuw1dcmwueNX4uxiNFKEAOsPDABopjjGQ3tpDVZOZgtJEXBiahobfWaMB7KQlWd5WlDKURPmk4wHmnmjj3+48YPv56LFUHUYCI4VNEPISAo+gd9CmVlpZiNpvJygqfuzxvHD9eZ+/F5AuBHsLumgE1ZVQ6X+0s8xhfiTBaW4OgqmT2iw074fDXenma8KegMGXghE7tMgL9t2wPTwORLBVFNH60HN1sAoe4zeY+0byfkuBcB99ao2H/P9bdyNVVp0hvMVMcE0l2k8bw83LRm+qcdh2uab4dIZTr1R5il+/4apuqKvTrF+/xda9iUVJSwu23386xY8fQNI2kpCReeOEFhg4N/zbM4SoWH20sYdXXxeg6qApMO2cgG/YctbcyVwCL5XT4V8H6h7Q1HQy3WIY/18vWQsS1PgN87zVVXHuQspYjDIzMDCtXFkBCcxnHtvwH875vQNdAUUHXORil8GlSHIWxkU4CYaf1OQUw6GBp7UN13qkmzj3VRFaLjmGQdZCY5fCu1ql/BiLGXI7e0mj9LvXPsu9K1P5ZdpEBiKo9QHPfM8JupxKuF+VwtQtCIBa33HILM2fO5JprrgHg3Xff5f333+ef//xnh40IFeEqFradhcWi2S/+gJNbKm93Oet3t3apVRT01nbmNnHp1zc6bIr4ArFeXd1l2KbX2dxA4Ta9zrZmjnf/AC0FGzioNbDSchCTbkHV4YLaJkzJA9ms1djdVDik4Np+jtBhYVkNWU3mTlikWr9curVexDBoDGps3zbC4uvPrrudrq5XuBGudkEIxGL27Nm89957Ts/NmjWLNWvWdNiIUBGuYgEdq9q2vceWJeW089D0sMmYCtR6dWWX8UnJ56wp/gQdHRWFmdmXc/mQGX63sbN4W7Pi2oPWqvFGEzmDz8eQlsPXe9fwdtnXDithT6eyi8ZZ9c0MajbbYyFhgZudjd5UhxId32HxiYmJoCVuYKfEypfz+Pqzza5An8fX3ydi+BTSR43zq1h4DXBbLBZOnDhBYqI17726urrDJxc805GGgo7vyUyJ58jxBg6V1drjGz2xiM8RbxlTbxasZmB8utsdgz3ArFswKIZuN73O3RTDaWfOYmB0slPzQsDag6r1nu+HuCh+iItC1eO44EQDMTpkNzaHVjh0C6ad/+7SIcLz3j187TLt+4amxEcgyn/dNrwW5RkMBh588EGOHz/O5s2befzxx/n5z3/O6NGj/WZEoOgORXkdJblPND8aNZDSY6fYU2wVbKNB5aoLzmhTvBdsAr1eg/tkMrJ1ct+RU+X2Yj4d3V7E5lrElxSdyPCkHLL6DeCSzIvCygUFnV+z5KTBpCYOZmvFd6DrGNUIzkoYzLGWWuvuovU/XVE4GBPJ/thItveJJWnIBAYlDsGQcgYRZ16IEpeEIeUMDBlnoR0rth5cNRI77DxM1UdxDLwL3RBdw5iUjtav4zdJ3oryOpQNtXHjRtavX4+maUydOpXJkyd32IBQEs5uqM5wvN7EA89vwGTWUFWFn182nAvHZoTarKCul7tYRnsuqXD9W3bVLsdMK4BndvzNPuXPHe3FehzjJumjxnF0zw5aCjZ02W1CZAymXZ+A3nayohBgVCMDb3yEUz7sLLocs3DHn//8Z37729/6+rGg09PE4std5bz68V57FtU107K5ctKQUJsV9PUqrj3Iv4vXsbem0P6cqqgsPve/21wIw/Vv6W+7bOLhrj28I6qicvGgaTSaG3HXvNDfdtmEyNXPLjGLHhizcMdrr73WLcSipzF6aH+MBtWeRRVuc7uDRXbfLK7Ivox92/fbL4qarvHv4nVckX0ZgP2uu7cMGXJtD+9troiNjWVbmTQwcLNFOlrx7o3eIvrhTKfEQnpDhYaRQ5LtfaTCJW02VGT3zeL64dc4uaT21hSS/20RqqKi6RpG1chDib8hiZQQWxtcOjJXxIYF59ki16RcEmxzhW5Cp8RCekOFDtcsqp4yOKkz2FqFOLqkbMOVACyame+PFXBBSu8SC0c6MlcETvfjqjIfo78xlXpTvcwLF5zwKBbr1q1z+7yu62ia+y+cEFxc24bMvWQY9Y2mXiUc7lxSNgyqkbNTh0tiD+7nipxsqWNP1V6nflz/2f+N/TO+VswLPRuPYvHqq696/FB3SJvtDTgOTjKbNV5bV4Cmh0+xXrBwdEnpuu4043t4/+xe41P2hnPthlUAPFXKw+ndxvfH8+kTmRB2HYCF4NIpsRCCi83VNHFMBv3iIuzPjxicZA94oyhorS1Benqxnjsc75zFfdJxHLv+bir/FotuaTOUalfV94A1GH52/5EiHL2UTsUsgsHWrVt5++230XWd888/n+uuuy7UJoUER1fTmrwS7s49vWNwNzipN2dKuat6FrxjW7cJA86jrOUIVbW1btNvLVichCOQWVRC+BG2YnHy5EkeeeQRIiMjue2223qtWLi6mlx3DLaAd1FpLVNGpQM9e/6FEDiy+2YxIWUUlZWn7FlUG8u2YaFtUZ1jFtXFg6YRY4yWHV0PJ2zE4sUXX2T9+vX2xy+99BK6rvPkk08yf/78EFoWWhxdTUaj+x2Da6B78ugBIbBU6Ek47jY2l2/jZEsd31fltxEOx7oNCYj3bDyKRVlZGQMHui8V//rrr5k2bZpfDVm4cCELFy60Pz558iSPPfYY8+bN69UBdUdXk2vMwhbLOF7bZN992OIVQK9NqRX8h6Nrz9YF2DWLyoYExHs2HsXi9ttvZ/Xq1QDccccdrFixwv7a008/7XexcOUPf/gDR48e5R//+AcDBgzo1RXjNleTY7Wo425CVRUMqjXAbTCoxMVEOO00elNmlBA4HIXDUxaVa0Bc4ho9B49i4VilffjwYY+veaOuro7c3FxWrlxJZmYmAGvWrOH555/HbDazYMECbrjhhjafW758eYfP0RtxjGXoms5Uh4FIrnGO1z8tICstQWIZgt9wzD7z1I/KtTpc3FPdG49i4Vil7Vqx3dEK7p07d7J06VJKSkrsz1VUVPD000+zatUqIiMjyc3NZcKECeTk+H+MY3tNsbyRkpLgR0v8h82uiWMyWJNXgtlsjWVcOXUoI4ckA5CUGMuavBJMJg0dOFB+igPlp9iw5yh//O8p9vcFwq5wJFxt6+52paSMYgLW3lvTh/2Irw5s4vMDefYKehuarvHmvlUU1RWRGN2H6UMmMrx/dsDsCjbhahf417YO7Sw6y1tvvcVDDz3EkiVL7M/l5eUxceJE+zClyy+/nLVr17Jo0aIun8+VntZ11tGufnER3J17uk9Uv7iINq+9v76Y7w/U2D9vMmts2lXqFPfwt13hRrja1tPsSiKF2UNmMSZpjNu4hobO1tKdAHy+P89n91RPW69g4O+xqh7FQtM0amtr0XUdi8Vi/xms0/M6wrJly9o8d+zYMVIcevWkpqaya9euDh1PcKa9aXs5GX25+oJs8g9tx2Kx/t2MBqVX1l8IwaMjcQ2be2pj+VbO7idFft0Fj2JRUFDAxIkT7QIxYcJpf2NXGglqmub0eV3XpTGhn3FsLnjvvHPJ210OSP2FEFwcq8Pd1WtYdCny6054FIv8/PyAnDA9PZ1t27bZH1dWVpKamhqQc/VGXGsu7pk7jvk/Hhlqs4Reirt6jR+O78Oim536O0owPPxptyjP5oIyGo3U1dWRl5fHiBEjyMrqvPJPnjyZFStWUF1dTUxMDOvWrePRRx/t9PEEZxwzoXpjjyghPHFXr+FutyG1GuGLR7EoKiri1ltv5Xe/+x2TJk2yt9uoq6vj8ccfZ8qUKZ06YVpaGosXL2b+/PmYTCbmzJnDmDFjOme90AbHim9vPaJ68ywMIXS42224tkp3dU/9WJna64ZYhRseZ3DfeuutzJw5k6uuuop3332XF198kY8++oiKigoWL17MG2+8EWxbfaYnZ0O1R0dE4MvvSq0tzTUdo7FrhXvhul4QvraJXc601yodwKCozAjDHlTh+neEIGZDlZeXc9VVVwGwefNmLr74YlRVZcCAAdTV1flgshBs2suSKiqtJW93OV/vLMOmo2ZxVwkhxnswXHpQhRqPYqGqqv3nHTt2sHTpUvvj5ubmwFolBARb8Ntkdr57UxVJqRVCjzf3lA2Ja4QGj2LRt29f8vPzqauro7Kykh/96EcAbN++nbS0tKAZKPgPW/DbEVVVuOGy4bKrEMIG6UEVnngUi7vuuoubbrqJuro67r77bmJjY/n73//OypUr+etf/xpMGwU/4Rj8VlSFqaMHSO2FENbY3FPehjJJ2m3g8SgWY8eO5euvv6apqYk+ffoAMG7cON5++22GDBkSLPsEPyMDkoTuRkeHMol7KrB4FIsTJ060+Tk7O9v+2NbbSegeyIAkoSfQmbRbEQ3/4FEsJk6caG/D4ZpdqygKe/fuDaxlgl+RYj2hJyE9qIKPR7GYPXs2O3bsYMaMGfz0pz8NSAtxIXh0tFhPCvWE7ob0oAoOHovyABobG1m3bh3vvfceDQ0NXHXVVcyaNcsewwh3emtRnie8CYHr9L2OBsDDdb0gfG0Tu3yjo3Y5jn5114PKhr9qNcJ1vcD/RXntioUjR48e5f333+fjjz9myJAh/M///E+HjQgVIhYdwyYixWUn2VFY5fRaRAequ8N1vSB8bRO7fKMzdrXXgwpAQWF0/7O65J4K1/WCIFZwu1JdXU11dTU1NTX069evwwYI4Y2nQj0bEt8QuisSDPcv7YpFeXk5H3zwAe+//z4Gg4GrrrqKt956S4ryehDuCvUA1NYRI96aEQpCuONLMDyvfAsXh2EPqnDAo1jceOONHDhwgCuuuIInn3ySs846K5h2CUHCFvg2m63zuhUFjAaVuZcMo77RZBeKf661zjeR+gyhO+MtGK5JDyqPeIxZjBw5kqioKFRVdTvZbvv27UEzsrNIzKJj2GIWcTERdoGwCUJRaS1/+j/n0axL5p3rJBjhul4QvraJXb4RCLscg+HuelCBNa4xZeAEj+6pcF0vCGLM4rPPPvPNMqHb4qlLbVFpLe+vL7YLBYDZoksMQ+gRdLQHlbinrHgUi4yMjGDaIYQZngLfRoN0qBV6Hjb3VGHNfhrNTW16UIl7yodsKKF34Rj4VoAhAxLISkuQmIXQY3Hcadh6UOWVbfXYIv3wqdJeNcFPxEJwi2vF99xL3LcxLyqt5ctd5WT2ixUREXoMNuEYlJDZrntqY/mWsJzgFwhELAS35GT05Z6549pUfDtWgR+prLOOZtV1jIaujWYVhHDEm3vKNsFPQUFVVCYN6Lm1GiIWgkdcA9+u7UB0TZfRrEKPpyPuKR0di96zazVELIQO49i5VrPoTj13ZDSr0Bvw5p6CnhsMF7EQOozrpD0FsFh0Gc0q9DocJ/jRYuTwqSPtBsN7wkAmEQuhw9jiGHm7ywEYnJ5AfaOJiWMy6BcXEWLrBCG4OE7wgwntBsN7Qg8qEQvBZzbsOWqduLfHGtQeOSQ5bKtYBSFYeAuGQ/fuQSViIfiEu4l7k8ZmtpmVIUOUhN6Iu2B4T+lBJWIh+IS7iXv5JdVO870vGZ/Juq2H0TQdYwfmYQhCT8Rbi3Qb3SWuIWIh+IS7+osvd5Xbdxtms8baLYfQJaVWEICO96ByjGuc3T/8ZoaLWAg+41p/MXpof/tuA0Vx6vQrKbWCcJqOxjXCMSAetmJRWFjIihUriI2NZdasWUyZMiXUJgkeGDkk2Z4lVVvfwu7i41gsOoqk1ApCGzoa14DwCoiHrVg0NDTwwAMPYDAYeOqpp0QsugG2LClVVZg+dqA0HRQEL7iLa3xfle8xIG5rK3J2v+C7qcJGLF588UXWr19vf/zSSy9x6NAh7rvvPubPnx9Cy4SO4JglpWs6/fpGu+0nJeIhCG1x3G20N5TJ1lYkFPGNsBGLhQsXsnDhQvvjPXv2MGTIEN544w1uvvlmrrjiihBaJ3jDXZYUwJfflVqbDUpmlCB0iI4ExG24xjcchSMlZZRf7QobsXClubmZBx98kPj4eKZPnx5qcwQvuMuSKiqt5V+tQgFgMmu8v76Y8SNS24xvFQShLY4B8biIOA6fOtJufMMmHJvKt/Fw4mK/ztoIuFjU1dWRm5vLypUryczMBGDNmjU8//zzmM1mFixYwA033NDmc+PHj2f8+PGBNk/wI65ZUvsO1bSZgf79gRq+P1CDAhiNKnMvGSbCIQjt4LjTgAle4xsAZt3M98cKuCDFf2Kh6Lque39b59i5cydLly7lwIEDrF27lszMTCoqKpg7dy6rVq0iMjKS3NxcnnrqKXJycgJlhhAi8kuqeXDlBkwmzTpuT8epU62CdUi8rltdVMt+NYWRQ5JDZK0gdD8Kqor56sAmTjSfZHvZHiy6VTgiVCMPXbSY4f2z/XaugO4s3nrrLR566CGWLFlify4vL4+JEyeSmJgIwOWXX87atWtZtGiR389//HhdmzvbjpCSkhCWvY66m1394iK4O9fqmoqLieD1TwsxmzV0QFFAaa3J0LEW823aVer3hoTdbc1CjdjlG6G2K4kUZg+ZBcC0dGtgHBQmDBjP8P7ZPtmmqgr9+sV7fD2gYrFs2bI2zx07dowUh61Ramoqu3btCqQZQghxdE1lpsTbhaO+0eQkICgKcTHSuVYQOouzu8r/BD3ArWkaiqLYH+u67vRY6Lm4xjRs2LKlXv+0kMyUeIldCEIYogb7hOnp6VRWVtofV1ZWkpqaGmwzhDChvtGEpp92Rb2/vpii0lqKSmv5aGMJRaW1oTZREARCsLOYPHkyK1asoLq6mpiYGNatW8ejjz4abDOEEGMr1IuLicBoUO2xjB8O1JB/aLt1Cp+mYzRIbYYghANBF4u0tDQWL17M/PnzMZlMzJkzhzFjxgTbDCGEFJXWOrU0n3vJML7dd4wfDtSg4zzf2yJdawUhLAiKWHz++edOj2fNmsWsWbOCcWohDHEdoFTfaOLqC7IpOLzDab63pulO1eCCIISOsK3gFnou7lqDuFaAA/ZZ30cq66S3lCCEGBELIei4aw1ie96x+eCGPUcxma09caTiWxBCi4iFEBI8pdHasLmqbNiypV5bV4CmS+BbEIJN0FNnBaEj2FxVtgocRQFFba341k8HvgVBCA6ysxDCCsfZFzZXlWvFt2sbdEEQAo+IhRA2uKbU3jN3HFdOGuL0HlvLEJtQfLSxROIXghAERCyEsME1pdZdfYUt1uGuVkMC34IQOEQshLDB07Q9V4pKa3l/fbG96lsC34IQeEQshLDBU0otOLcHef3TwtMptS6tzqXiWxACg4iFEFa4S6l1dDkpioLeOqNEAYakJ5AYH8Xu4uNS8S0IAUTEQgh7HGMZ6Lp9up6iKhw+VkfJ0VOoqsK0cwYyefQA2VUIQgAQsRDCHtdYhi2Yfby2ia92lqHroGs6/fpGtxGK/JJqNu0qlcC3IHQREQsh7PEUy7C1BLGJSFxMBB9tLLHXZTQ0mVm39TCaZp3xLYFvQeg8IhZCt8BdLMNRRFwD366YJfAtCF1C2n0I3ZqcjL5cOWkI9Y0mp15SrqiKYg98yxQ+QfAd2VkIPQJbXMNWe+GIqipc9qNB7DtUw5HKOl7/tNCpSlx2G4LgHRELoUfg6pKy9ZJCVUHT7C4qRcGaVYW1mC9vd7mIhSB0ABELocfgLq6RkpLAKx/stscydIdthw58s6sMgMmjBwDIkCVB8ICIhdDjGTE4CbW1vbkrFg2+/K6Mb3aXowAWTVqGCII7JMAt9HhyMvry88uGY1AVFAUMBgWjQbHPygCwWHTMltOzMvJ2l0sQXBAckJ2F0Cu4cGxGm/bmebvL+WZXGa5JVFb3VDkWzVotfvmPBhEbbRT3lNCrEbEQeg2uMQ3bz19+V2Z/zhYAt7QGNzRN5+PNh+wzwMU9JfRWRCyEXs3k0QPsVeA4NCl0xdYK/eNNB+kbF2n/rAiH0FsQsRB6Ne6qwK3CAeiguWRP7Sissj/+ZlcZU8dI80KhdyBiIfR6HN1TrnGNfYdqKC476SQSNhwzqaaOHiDpt0KPRsRCEBxwF9coKq1lz4Fqt9XhYM2k+vK7Mr7aWYaqKBIYF3okIhaC4AVXV9Who6eorW9hd/FxLBbdLiDuAuOACIfQIwhrsbBYLNx0000sWbKE0aNHh9ocoRfjaYJf3u5yvt5Zhoe4OOBeOFAVGhtNEu8Qug1hLRYrV64kNTU11GYIgltsAjI4PYHX1hWg6TqKm8C4I47CAdYg+Zih/ekbF8ng9AQOHT0FSKaVEH6EjVi8+OKLrF+/3v547ty5DBs2DE3z3HZaEMIBdwV/+w7V2IcvWdrZdlg03AbPXUXE1hixvtEkriwhJISNWCxcuJCFCxfaH991113Ex8ezZ88eDh06xBNPPBFC6wShfTwV/I0bntJh4XDEk4jAaVdWY7MZkF2IEBzCRixceeqppwBYsWIFF154YWiNEYRO4igiNuGIi4mgsraJo1X17Npf1abdiDfElSWEgoCLRV1dHbm5uaxcuZLMzEwA1qxZw/PPP4/ZbGbBggXccMMNHj9/xx13BNpEQQgKjsKRkpJAZeUpe5AcsF/oa+tbfBKRjrqybCJic2uJO0vwBUXX9Y7tizvBzp07Wbp0KQcOHGDt2rVkZmZSUVHB3LlzWbVqFZGRkeTm5vLUU0+Rk5MTKDMEoduRX1LN59sOAQrZGX051dBCfaOJ977a32FXljcMqsLs6UNpaDI5nSchNpLi0hOAwozzBjFySLJfzid0bwK6s3jrrbd46KGHWLJkif25vLw8Jk6cSGJiIgCXX345a9euZdGiRX4///HjdW5nGHjDdtcXbohdvhOutnmzq19cBNdNH9rm+RGZfdvUe3TGlQXW2R3vflHU7nvWbS7xuDsJpruru/4dQ4mvtqmqQr9+8R5fD6hYLFu2rM1zx44dIyUlxf44NTWVXbt2BdIMQegxtFfvAZ13ZXmivUC7jfbcXRI/6TkEPcCtaRqKcnrsjK7rTo8FQfANdwIC7kUEICbK6FNmljc6Iyi2VGB3wiIxlfAk6GKRnp7Otm3b7I8rKyul8E4QAoAnEQHnzCzHizSqCprm190JdExQXHFMEY6JiSSlb5SISwgJulhMnjyZFStWUF1dTUxMDOvWrePRRx8NthmC0KvxJCSufm5PuxN/u7vc4Zoi7A3X+pOO7GDETdZxgi4WaWlpLF68mPnz52MymZgzZw5jxowJthmCIHSA9nYnNkIpKI74Ki6OdCTu4u5n246nN+xsApo6G2okGyo4hKtdEL629Ta7XAWlvTt+f8dUgoECqAaFMdn9fBYc28/+FpxulQ0lCIIAHduhOOIaU2kvZhEO4qJjnWvia1zGFXeuNFdB6ahbLSUloUu2uCJiIQhC2OEqLt7ukj0F7DtycQ2Wm6wjdMWV5sj63eU8lhhLv7gIP1hlRcRCEIRuj687F1e8xV08/ey44/lmV1lYCA6A2aKze38VF44Z4LdjilgIgtDr6azYOO54Jo8e0CnBAf+70owGhdFD+/vlWPZj+vVogiAIvZSu7m48udI6G7MYOSTZr8kKIhaCIAhhQFfFJtCooTZAEARBCH9ELARBEASviFgIgiAIXhGxEARBELwiYiEIgiB4pUdnQ6lq5+dkdOWzgUTs8p1wtU3s8g2xy3d8sc3be3t0I0FBEATBP4gbShAEQfCKiIUgCILgFRELQRAEwSsiFoIgCIJXRCwEQRAEr4hYCIIgCF4RsRAEQRC8ImIhCIIgeEXEQhAEQfBKj2734Str1qzh+eefx2w2s2DBAm644YaQ2fLss8/y8ccfAzB9+nSWLFnC/fffz7fffktMTAwAixYt4tJLLw26bTfeeCPV1dUYjdavzyOPPMKhQ4dCunZvv/02//rXv+yPjxw5wtVXX01jY2PI1qyuro7c3FxWrlxJZmYmeXl5PPbYYzQ3N/OTn/yExYsXA7B3714efPBB6uvrOe+88/j9739vX9tg2PXmm2/y6quvoigKo0aN4ve//z2RkZE8++yzvPvuu/Tp0weAn/3sZwH9u7ra5en77mkdA4mjbfv37+epp56yv1ZRUcE555zDCy+8ENQ1c3eNCOh3TBd0Xdf1o0eP6hdddJFeU1OjQU40JAAACkFJREFU19fX67NmzdILCwtDYsuGDRv066+/Xm9ubtZbWlr0+fPn6+vWrdNnzpypV1RUhMQmG5qm6RdccIFuMpnsz4XT2um6rhcUFOiXXnqpfvz48ZCt2XfffafPnDlTP/vss/XDhw/rjY2N+vTp0/VDhw7pJpNJv/nmm/Uvv/xS13Vdv/LKK/UdO3bouq7r999/v/7aa68Fza7i4mL90ksv1U+dOqVrmqYvWbJEf/nll3Vd1/Vf/vKX+vbt2wNmS3t26bru9m/X3joG0zYbx44d0y+++GL9wIEDuq4Hb83cXSPWrFkT0O+YuKFaycvLY+LEiSQmJhIbG8vll1/O2rVrQ2JLSkoK9913H5GRkURERDB06FDKysooKyvjgQceYNasWTzzzDNomhZ024qLiwG4+eabueqqq/jXv/4VVmsH8PDDD7N48WJiYmJCtmZvvfUWDz30EKmpqQDs2rWLrKwsBg0ahNFoZNasWaxdu5bS0lKampoYO3YsANdee21A187VrsjISB566CHi4+NRFIXhw4dTVlYGwJ49e3jhhReYNWsWjzzyCM3NzUGzq7Gx0e3fztM6BhJX2xxZvnw5ubm5DBkyBAjemrm7RpSUlAT0OyZi0cqxY8dISUmxP05NTaWioiIktgwbNsz+hy0pKeHjjz9m6tSpTJw4kT/+8Y+89dZbbNu2jXfeeSfotp08eZJJkybx17/+lVdeeYU33niDsrKysFm7vLw8mpqa+MlPfkJVVVXI1mzZsmWcd9559seevl+uz6ekpAR07VztysjIYMqUKQBUV1fz2muvcfHFF1NfX8+ZZ57JPffcw+rVqzl58iTPPfdc0Ozy9LcLxb9TV9tslJSUsGXLFubPnw8Q1DVzd41QFCWg3zERi1Y0TUNRTrfo1XXd6XEoKCws5Oabb2bJkiVkZ2fz17/+ldTUVGJiYrjxxhv56quvgm7TuHHjWL58OQkJCSQnJzNnzhyeeeaZsFm7N954g1/84hcADBo0KCzWDDx/v8Lle1dRUcGCBQv46U9/yoQJE4iLi+N///d/GTp0KEajkZtvvjmoa+fpbxcu6wXw5ptvMm/ePCIjIwFCsmaO14hBgwYF9DsmYtFKeno6lZWV9seVlZVut53B4ttvv+Wmm27it7/9Lddccw379u3jk08+sb+u63pAg6Ce2LZtGxs3bnSyIyMjIyzWrqWlha1btzJjxgyAsFkz8Pz9cn2+qqoq6Gu3f/9+cnNzueaaa7j99tsBKCsrc9qFBXvtPP3twunf6WeffcYVV1xhfxzsNXO9RgT6OyZi0crkyZPZuHEj1dXVNDY2sm7dOqZNmxYSW8rLy7n99tt58sknufLKKwHrF++Pf/wjtbW1mEwm3nzzzZBkQp06dYrly5fT3NxMXV0dq1ev5oknngiLtdu3bx9DhgwhNjYWCJ81AzjnnHM4cOAABw8exGKx8OGHHzJt2jQyMjKIiori22+/BeD9998P6trV1dVxyy23cOedd3LzzTfbn4+OjuaJJ57g8OHD6LrOa6+9FtS18/S387SOwaa6upqmpiYGDRpkfy6Ya+buGhHo75ikzraSlpbG4sWLmT9/PiaTiTlz5jBmzJiQ2PL3v/+d5uZmHn/8cftzubm53HrrrcydOxez2cxll13GzJkzg27bRRddxM6dO5k9ezaapjFv3jzGjx8fFmt3+PBh0tPT7Y9HjhwZFmsGEBUVxeOPP84dd9xBc3Mz06dP58c//jEATz75JEuXLqWuro6zzz7b7gMPBu+88w5VVVW8/PLLvPzyywDMmDGDO++8k0ceeYT//u//xmQyce6559rde8Ggvb+dp3UMJkeOHHH6rgEkJycHbc08XSMC+R2TSXmCIAiCV8QNJQiCIHhFxEIQBEHwioiFIAiC4BURC0EQBMErIhaCIAiCVyR1Vuhx/OEPf2Dr1q2AteAsIyOD6OhowFp1a/vZG5999hkbN25k6dKlHt9TUVHBnXfeyRtvvNF1w4H77ruPDRs2kJyc7PT8T3/6U7+n1N53330MGzaMW265xa/HFXomkjor9GhmzJjBX/7yF0aPHh1qUzpEMC/gIhaCL8jOQuh1jBo1iosvvpj8/HyefPJJ9u3bx5tvvonJZKK2tpb/+q//Yt68eaxatYpPPvmEF154gRtvvJGxY8eyfft2ysvLmTRpEo8++ihlZWXMmjWLHTt2sGLFCkpLS6msrKS0tJS0tDSeeOIJUlNT2bVrFw8//DAmk4nBgwdTVlbGfffdx4QJE3yyfcaMGVx55ZVs2LCBU6dO8Ytf/IJ58+YB2OdSqKpK//79+d3vfscZZ5xBfX09f/jDH9i+fTsGg4FLLrnEPudgx44d5ObmUlVVxbBhw/jzn/9sr4AXBEdELIReh8lk4qKLLuIvf/mL/UL6t7/9jaSkJL777junC7Ajhw4d4tVXX6WhoYGf/OQnbNmyhczMTKf3bNu2jffee4/4+Hh+9atf8cYbb3Dbbbdxxx138MgjjzB9+nQ2bdrETTfd5NG+V155hQ8++MDpueXLlzNixAgAamtreffdd6moqGD27NmMHz+e6upqXnzxRd58802Sk5NZtWoVt99+Ox999BHPPPMMzc3N/Pvf/8ZisXDzzTezZcsWwOpG++c//0lkZCTXXXcd69atY/bs2V1cYaEnImIh9EpsLafj4uJYuXIlX331FSUlJeTn59PQ0OD2MxdddBGqqhIfH09WVha1tbVtxOL8888nPj4egLPOOova2loKCgoA6zQzgIkTJzJs2DCPtt10003tuobmzZuHoiikp6czdepUNmzYQFVVFVdccYU91nHttdeybNkyjhw5Ql5eHvfffz8GgwGDwWCfKLh69WouueQS+yS6YcOGUV1d7XXthN6JZEMJvRKbq+Xo0aPMnj2b0tJSxo8fz29+8xuPn3EMjCuKgrtwn7v3GAyGNu81GAydtt2xk6mmaaiq6naok67rmM1mjEajU0vq8vJyampq2hzL0+8kCCBiIfRy9uzZQ3JyMrfddhsXXHABX3zxBQAWi8Vv5xg6dCiRkZF8/fXXgHVqXkFBQafnMLz33nuAtSX2hg0bmDZtGlOnTuXf//63fWfw7rvvkpiYSFZWFpMmTWL16tVomkZLSwu//vWv7dligtBRxA0l9GqmTJnCO++8w49//GMUReH8888nOTmZgwcP+u0cRqORFStW8NBDD/HUU08xZMgQ+vfv7zGF113M4pxzzuGRRx4BrB1Pr732Wpqamli6dCnZ2dlkZ2dz0003sWDBAjRNIzk5mRdeeAFVVVm0aBHLli3j6quvxmKxcMUVV3DZZZfx+eef++13FHo+kjorCEHgT3/6E7fccgv9+/envLycq6++mk8//ZQ+ffr4dJzulgos9BxkZyEIQSAjI4ObbroJo9GIruv84Q9/8FkoBCGUyM5CEARB8IoEuAVBEASviFgIgiAIXhGxEARBELwiYiEIgiB4RcRCEARB8IqIhSAIguCV/w+F9cYTV2pnvwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "net.train()\n", "for label, x in variations.items():\n", " optimizer.load_state_dict(torch.load('opt.pth'))\n", " net.load_state_dict(torch.load('net.pth'))\n", " losses = []\n", " for epoch in range(200):\n", " y_pred = net(x)\n", " optimizer.zero_grad()\n", " loss = loss_fn(y_pred, y_train)\n", " losses.append(loss.data)\n", " loss.backward()\n", " optimizer.step()\n", " plt.plot(losses, '.', label=label)\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note how seemingly trivial changes to the input data drastically impact our ability to learn the model!\n", "\n", "The reason for this behavior is that individual modules have a range that their input distribution needs to cover for efficient learning. This is clearest for activation modules that saturate outside a narrow range centered on zero: they need inputs roughly centered on zero with enough samples probing their non-linearity but not too many samples saturating. A nice feature of ReLU is that it is scale invariant, although it still expects a distribution centered near zero (since it is not translation invariant).\n", "\n", "The case of linear modules is more subtle since they are mathematically degenerate with a linear transform of their inputs. However, they still set an effective scale through the distribution of their initial random weights and biases.\n", "\n", "The histograms below follow the input distribution through the first layer to show how the network responds differently in each case:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAAEUCAYAAAAspncYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfXxU9d3n//cEyBJIMBgmBC/cy1ray8tWaCuSSH1oxRYqkmJrtAEeJtS23BnUyBpSjcUKKRrRtBsV0J8u6gWXDYqP38YuMW5zsV0bWo17XXBZ6rXsXk2BKsmYpJIRkEzm7B80Q2YyIXNz5pwzM6/nP+05OTPnnUPz6fmcm+/XZRiGIQAAAABA1DLsDgAAAAAAyYqGCgAAAABiREMFAAAAADGioQIAAACAGNFQAQAAAECMaKgAAAAAIEY0VAAAAAAQo7F2BwjV2/uJ/P7Rp8bKy8tWd7fXgkSRI1NknJbJaXmk5M6UkeHS5MkTLUgUP+qNeZyWRyJTpJI5U7LUG2qNucgUGadlcloeybxa47iGyu83Iio6g9s6DZki47RMTssjkckK1BtzOS2PRKZIkSmxqDXmI1NknJbJaXkkczIl5JG/gYEB3X777frXf/3XRHw9AAAAADhCQhqqbdu2KT8/PxFfDQBBuIADAADsZPojf2+88YY+97nPye/3m/3VgO0GBnzq7fXI5zuT0P10dWU47m8oNNPYsZmaPNmtMWPsfXKYCzhIRVbVGol6A6Q7zm2CM8VSbxLSUGVnZ+u9997TkSNH9Nhjj5m9C8A2vb0ejR8/QRMnFsjlciVsP2PHZsjnc1bRGZrJMAx98skJ9fZ6NGXKNNsycQEHqcqqWiNRb4B0x7nNuUyx1hvTG6onnnhCktTQ0KCvfe1rZn89YCuf74wlJzhO53K5NHHiJHm9f7E1BxdwkKqoNec4pd4AqYp6c06s9Sbid6i8Xq8WLVqkY8eOBdY1NTVp4cKFmj9/vnbu3Bm0/dq1a3XFFVdEFQZIBhScs5xwHJ544gk9/PDDuv7661VWVmZ3HMBUTvgbcwqOBZBY/I2dE8uxiOgO1YEDB1RTU6OOjo7Aus7OTtXX12vPnj3KzMxUaWmpCgsLNWPGjKhDDJWXlx3xtm53Tlz7SgQyRcZpmSLN09WVobFjrZkP26r9RCM0U0ZGRkL+Lb1er0pLS7Vt2zZNnz5d0tkLOFu3bpXP51N5ebmWLVsW2H7t2rWmZwAAAIhERA1VY2OjNmzYoKqqqsC6trY2FRUVKTc3V5K0YMECNTc3q6KiIq5A3d3eiMaDd7tz5PH0xbUvs5EpMk7LFE0ev99vyfO/yfCcsXT2eIQeu4wMV1QXRkJxASdyTsvktDxS8may8uKNdO5iyd69v9TOnS/K5XJp/PjxuvfeKv39319uWY5wmQYl6gIOAGd69dVf6LXXXpXLJf3N30zX+vU1mjz5QrtjhRVRQ1VbWztsXVdXl9xud2A5Pz9fBw8eNC8ZUsqDbZvVc7o3aN2F4ydr49wf2ZQITsUFnMg4LZOT8oTWGyfVmkiPk1UXb6RzF0uOHOnQk0/+TM89t1NTpkzR/v1vaf36ddqz55eW5AiXaahEXMCJ1sDAgJYvX66qqipea4AkZ9ebZPb++3/QP/7jP2jHjn9Udna2nnzyZ3r22a2qqnrA7mhhxXz5y+/3Bz1jaBgGz19iRD2ne/XUvDo9Na9Ojd/dqqfm1Q1rsHB+e/e+rttuW6yTJ0/q1KlTWrasRHv3vm53LNPV1tZq9uzZQevCXcDp7Oy0OhqSxGC9odZEZ9y4TK1f/6CmTJkiSbrsssvV09Otn/70J3rwwWpJ0r//+//VokXfUEfHH+2MahumaUAo6k1k9u59Xbfc8q0Rz2GuuSb4//cvu+zv9fLLryk7O1uffvqpPJ4uXXBBrtWxIxbzKH8FBQVqb28PLHs8HooMkEA33rhIb7/9Wz399H9Wf/8ZzZz5Zd144yK7Y1mCCzhA4k2bdpGmTbtI0tm/sYaGel1zzbWqrKzSHXcs0969r2vXrhd111336pJLPmNzWusxTQMQuxtvXKR33vldVOcwY8eO1a9/vU+PPrpR48Zl6gc/WGVR2ujFfIdq7ty52r9/v3p6enTq1Cm1tLTo2muvNTMbgBD33fcj/e53+/X737+ne+75T3bHsUxBQYE8Hk9gmQs4QOKcOnVKDz5YrWPHjmr9+geVlZWln/zkp3r00U26/PIvav78G+2OaIs33nhDb731lv7pn/5JL774ot1xgKSzfv39Qecwv/vdfi1fvlTLly+VpMB//93v9gc+c+21X9Mvf/kr3XHHCt1771rHXtCI+Q7V1KlTVVlZqbKyMvX396ukpEQzZ840MxuAED09PTpz5lP195/RRx959Dd/M93uSJaYO3euGhoa1NPTo6ysLLW0tGjjxo12xwJSzvHjx7V+faUuueQSNTRs03/4D+MlSUeO/EkXXHCBDh/+N/X392vcuHE2J7Ue82wC8enp6Q46hyksvFqFhVdLOvvI344duwLbHjt2VN3d3Zo160uSpJtu+pa2bNmsvr4Tjnz0L6o7VK2trYEhjCWpuLhYr7/+ut544w398Ic/ND0cgHN8Pp8eeugB/eAHq/S97/1QDz10v3w+n92xLDH0As7NN9+sRYsWcQEHMNnJk59o7dqVuu666/WTn2wONFMffviBfv7zx1Vf/5T+43+8RFu3/mebk5qLeTaBxPP5fPrxj++P+Bymu/sjPfTQ/frLX85OsNvSslef+cxnHdlMSXHcoQJgre3bn9KFF16o4uKbJUn/83/u0zPPPKU1a+62OVlitLa2Bi0XFxeruLjYpjRA6nv11UZ1dn6oX/96n379632SpIEBn/x+v5YsuV2XXjpD9967XuXlpZo9u1Bz515jb2ATWDVNA1M0mM9JmQazhP6nEzhlmoatW8+ew3z729+RJL311v/Qs88+rbVr75Ek/fa3/yto+yuvvFLLl39fd921UmPGjNGUKW499tgTCckZ7jujnaaBhgpIEnfeGdw4bdmSWleJAdjr9tu/p9tv/955t5k0aZJee+2/WZQo8ayapoEpGszltEweT19QJqdkc9I0DatX3x00HcJjj/1cks6738WLb9HixbcErTM750jzfoZO0zDaFA00VAAAIC0xzyYAM1g3DTsAAIDDMU0DgGjRUAEAAPwV0zQAiBYNFQAAwF8xzyaAaPEOFQAAwF8xzyaAaNFQAQCAtMY0DQDiQUMFxOG+p9vUfeK06d875YLxqls91/TvBZC8qDcArECtiR4NFRCH7hOn9Xz1PNO/945HWkffSNI118zWW2+168MPP9Ctt35L9fVP6qqrigI/LykpVkPDdk2bdpFaWpr14ovPyefz6dZbl+iWW24zPTeAxLGz3lBrgPTBuU30GJQCSBFjx47Vo4/W6uTJT4b9zOPp0rPPPq2nn/7/9F/+yy791//6mv74x3+3ISWAZEetAWCVZKk3NFRAipgyxa2rripUQ8PPhv2svf1tfeUrszVp0gXKysrS9dffoH37fmVDSgDJjloDwCrJUm9oqIAUUlFxj95+e7/eeee3Qes/+sijvLwpgeW8vCnq6uqyOh6AFEGtAWCVZKg3NFRACpk4MVvr19cMuz3u9/vlcrkCy4ZhKCPDFe4rAGBU1BoAVkmGekNDBaSYOXOKht0ez8+fqu7ujwLLPT3dmjLFbUc8ACmCWgPAKk6vNzRUQAoavD0+WGhmz56jd999R729vTp9+rT27WtVYeHVNqcEkOyoNQCs4uR6w7DpQBzyJo2PeBjQaEy5YHxcnx+8PX7vvRWSJLc7Xz/84RrddddK9ff7VFy8WJdf/kUzogKwiBPrDbUGSD1OrDWSs+sNDRUQh8fWJGaCurFjM+Tz+Ufd7q232iVJ06ZdpFdeaQr62Zw5RYGfS9L8+d/U/PnfNDcoAMvYWW+oNUD64NwmejzyBwAAAAAxoqECAAAAgBjRUAEAAABAjGioAAAAACBGNFQAAAAAECMaKgAAAACIEcOmA3Hw7lonw9tt+vdm5EzRxCVbRt3ummtm66232vXhhx/o1lu/pfr6J3XVVUWBn5eUFKuhYbumTbtIkvTJJ16tWnWH6up+FlgHIDnYWW+oNUD64NwmejRUQBwMb7dyVuww/Xv7nlke9WfGjh2rRx+t1YsvvqwJEyYO+/nvf/+e6uo26ejRIyYkBGA1p9Qbag2Q2pxSa6TkqTc88gekiClT3LrqqkI1NPws7M+bml7Tvfeu15QpbouTAUgl1BoAVkmWekNDBaSQiop79Pbb+/XOO78d9rPq6gc1a9aXbUgFINVQawBYJRnqDQ0VkEImTszW+vU1evTRWp08+YndcQCkKGoNAKskQ72hoQJSzJw5Ree9PQ4AZqDWALCK0+sNDRWQggZvj3d3f2R3FAApjFoDwCpOrjc0VEAKGrw93t/fb3cUACmMWgPAKk6uNwybDsTBlZ0X0zCgo8nImRLRdm+91S5JmjbtIr3ySlPQz+bMKQr8fKjQ7QAkBzvrDbUGSB+c20SPhgqIQ/bSxxPyvWPHZsjn8yfkuwEkJ+oNACtQa6LHI38AAAAAECMaKgAAAACIEQ0VAAAAAMSIhgoAAAAAYsSgFLDN5AFDd7ZWBS1v+sZjNiYCAAAAokNDBdus/6NHOSt2BJaHNlfJ4sG2zeo53Wv69+aNn6yH5/7I9O8F0lHoxZvBdcl2AYd6A8AK1Jro0VABceg53aun5tWZ/r2RNpcnT57U1q0Nevvt/Ro/PksTJ07UHXes0OzZc1Rb+5C+/OUrtXBhcWD7557bLkn64hdnauvWBknSn/98VBdemKesrAmaNu0ibd68RZLk8/m0bt1alZd/X1/5ymyTf0PAOqEXb6TkvIBDvQFgBbtrjWRdvZkzZ44pvxsNFZCkDMPQ+vWV+tznPq9/+IfdGjdunP73/35f9913jzZs2HTezxYWXq3CwqslSRUVK3THHSuCTmKOHOnQ5s0P69/+7d8S+jsASA7UGwBWScZ6w6AUQJL6539+V8ePH9fatfdq3LhxkqTPf/4ylZd/Xy+88Fxc3/366/+/liwp0+WXf8GMqAlz+PBh3XXXXaqurtZvfvMbu+MAKYt6A8AqyVhvaKiAJPX++4d02WV/L5fLFbT+S1/6sv7wh0NxffeaNXfr2mu/Ftd3WOHkyZO6//77tW7dOr3++ut2xwFSFvUGgFWSsd7wyB+QtFwaGBgYtra/3yeXS8MK0aCMjNS5jjJr1ix1dHSourpaZWVldseBg/Q9s1x9f/3vruw8W7OkBuoNAKskX72h0gFJ6gtf+KLef/+QfD5f0Pr33juoyy67XDk5k+T19gX9rLe3Rzk5OVbGTKj33ntPU6ZM0csvv6xXXnnF7jhwkJwVO3TpA68qZ8UOZS993O44SY96A8AqyVhvaKiAJDVr1pf1mc98Vj//+eOBovP++3/Qiy8+p/Ly72v27Kv03/97i06dOiVJ6u7+SG1tb+krX7nKztim+vTTT/XAAw/ogQce0HXXXWd3HCBlUW8AWCUZ6w2P/MEyfc8sP/uff11OhcdwLhw/OSHDL+eNnxzRdj/9aZ2eeeZp3X77bRozZoxycibpwQc3Bka0+T//57BWrChXRkaGXK4MrV69Vpde+lnT85rN6/WqtLRU27Zt0/Tp0yVJTU1N2rp1q3w+n8rLy7Vs2TJdeeWVuvLKK21OC1iDegPACnbXGin56o3LMAzDtr2H0d3tld8/eiS3O0ceT9+o21mJTCO7s7UqMKfBSJmGbmOlaI7R8eN/UkHB3yY4kTR2bIZ8Pn/C9xONcJnCHY+MDJfy8rJj3s+BAwdUU1OjP/7xj2pubtb06dPV2dmpJUuWaM+ePcrMzFRpaameeOIJzZgxI+b9IHXd9ovVavzu1ri3sdPvf39IF12U+FqTTD744E/6whcutztGTDi3MZeTMg2euwxmsutcJpxIjxPnNsMzhR6T0c5tuEMFwFEaGxu1YcMGVVWduzrW1tamoqIi5ebmSpIWLFig5uZmVVRUxLUvTnLM47Q8Hk/fqJnsyBvpcfL7/ZadeCTLSY7f7x927OK9gAMAZqChAuAotbW1w9Z1dXXJ7XYHlvPz83Xw4EErYwEAAIRFQ4WE8O5aJ8PbfW7FjHz7wpjMMIwRh+xMJ1Y+Lez3+4OOOf8G6WFYHdHZdy/NGLVv8oAR9I7A5AFDm77xWNzfayb+d36Ow95OAFIO9eacWOoNDRUSwvB2K2fFjnMrEvByox0yMsZoYMCnsWPH2R3FdgMDPmVkjLFkXwUFBWpvbw8sezwe5eenTpOejiJplobVkb9+bnCAm3CfiVRo81Tz5n3DXsK2s8mi1gSzst4A6YZ6EyyWekNDBUQhKytbfX1/UW5unlyu9J11wDD86uvrVVaWNe8uzJ07Vw0NDerp6VFWVpZaWlq0ceNGS/aNxIi0WQoV2jwN3T4e4RqnRIxyFSlqzTlW1xsg3VBvzom13tBQAVHIzr5Avb0edXYek5S4R1AyMjLk9zvrJfHgTC5lZo5XdvYFlux76tSpqqysVFlZmfr7+1VSUqKZM2dasm9YJ5Y7Ta7svOFNVQo8YmxVrZGoN4CZQh8nHlzntEeKh+LcZmim2OoNDRXiNtKjO9FKhiLkcrl04YWJP1lz2ohpkvWZWltbg5aLi4tVXFxs2f6RHMI2YSbdWbLzPSurao1EvQHMtP6PnmF33+282x0Jzm3iz0RDhbiFe3QnFk575AZAegutSdQjAEA46f2gJAAAAADEgTtUcLRkGNoYAAAA6cv0hurw4cNqaGjQhAkTVFxcrK9+9atm7wJpxOlDGwNIH+He84z0c9QoAEhdpjdUJ0+e1P33368xY8boiSeeoKGCqXjPCoBdYm2KqFEAkNpMb6hmzZqljo4OVVdXq6yszOyvB4bhsUDA+UJHA41lJNBkFUmNqnnzPvWOcZ13GwCAM5neUL333nu65JJL9PLLL+uOO+7QwoULzd4FbOa0EyNG4gKcZaSpFMwYDTQZDa1RbneObvvF6mHb9I5x6al5dYFl6hgAJA/TG6pPP/1UDzzwgLKzs3XdddeZ/fVwALOGSU+UZJjPCkhlTq8RdhupRo22Tbjvoa4BgP0ibqi8Xq9KS0u1bds2TZ8+XZLU1NSkrVu3yufzqby8XMuWLdOVV16pK6+8MmGBgdHwnhUAJ4ukCYpkG+oaADhDRA3VgQMHVFNTo46OjsC6zs5O1dfXa8+ePcrMzFRpaakKCws1Y8aMuALl5WVHvK3bnRPXvhIhHTL1xfidQz9jx3E63z7T4d/NDE7MBAAAYKeIGqrGxkZt2LBBVVXnroa1tbWpqKhIubm5kqQFCxaoublZFRUVcQXq7vbK7zdG3c7tzpHH0xfXvsyWTpli+c7Bz9h1nEbaZzr9u8Uj0kwZGa6oLowAAGC1cO+DZy993MZESGYRNVS1tbXD1nV1dcntdgeW8/PzdfDgQfOSAQAAOATzbKaW0Hc9+55ZblsWJL+YB6Xw+/1yuc4N8WoYRtAy4CQMrQ4AiAfzbAIYScwNVUFBgdrb2wPLHo9H+fn5poSCc4w0/HGyYWh1IHG8u9apz0FTKQCJwDybyStVzmXgXDE3VHPnzlVDQ4N6enqUlZWllpYWbdy40cxscACGPwYwGsPbrUsfeNVx7/0BZmKezeQVybmMKztv2GN/vFeFSMXcUE2dOlWVlZUqKytTf3+/SkpKNHPmTDOzAQAAOALzbKa2cI0T71UhUlE1VK2trUHLxcXFKi4uNjUQAACAVZhnE0C8Yr5DBQBIP7yLgFRi1TybzLFpvmgyxTp/Zujnjjy5Sr6PPcEbzcgPbON254y4L7uOodP+7ZyWRzInEw0VACBivFfpHKGjlw6uYwTTyFk1zyZzbJorlkyx/A6u7Dz9e+0tQcvD6l9rlTyevqBMofuaPGDotl+sDlq24u/Uaf92TssjmTfHJg0VgoSb6A5Aegh39ykUNcE5wp2QMYJpdJhnE+dj1oAUjDSc+mioEISrz0D64u8fYJ5NANGjoQKANMUdaWA45tkEEC0aKgBIU9yRAoZjnk0A0aKhAgAA+Cvm2QQQLRoqAEgDDHcOjIx5NgHEg4YKaYnhhpFueLwPQLqw+v3QvmeWa3Dg7Uj2xTlI6qGhQloaWrQG5yBgGFMAyS7ciVq4bThxQyqz+gJSzoodUc2xxJQHqYeGCgBSECP4padIGiVO3ADAXDRUAJCCeMQPAABr0FClMV5SBwAAAOJDQ5XGuIINpAYujgBIV9Q/OAENFQAkGe+udeoLeT+KiyMA0hEXh+EENFQAkGQMb7cufeDViEeUAoZiyGYks3AXlAC70VABAJBGQhsntztHt/1itU1pgOhwQQlOREOVRhhGGQAAADAXDVUa4TljAEAkat68T71jXEHreCwQAMKjoQIAAEF6x7j01Ly6oHVMCAw4Cxc+nIOGCgAAALBRuMFiwm0ztFniwodz0FABAAAANorkrlLNm/cFNUyTB4xERkIUaKgAAAAAh+NRPueioQIABwsdnVNihE4A6YH6h2RBQ5WiBovQ0FkaKEJA8mF0Tlgh9P0NHiWCE1D/kCxoqFLUYBFyu3OY/A4p65133tHu3btlGIbmzJmjW2+91e5IQFLiUSI4AfNlIlnRUAFIWidOnNDDDz+szMxMrVmzhoYKAJIYd6SQrGioACStG264QQMDA9qyZYvKysrsjgMAiBDvRyGV0FABSFonTpzQ5s2btXTpUl1xxRV2xwEARIi7UUglGXYHAIBYbdq0SX/+85/1wgsv6PHHH7c7DgAASEPcoUoRvMgZv3CjXPGitj28Xq9KS0u1bds2TZ8+XZLU1NSkrVu3yufzqby8XMuWLVNdXd0o3wQAQPrgXMYeNFQpglvn8QstOEMLEqxz4MAB1dTUqKOjI7Cus7NT9fX12rNnjzIzM1VaWqrCwkLNmDEjrn3l5WVHvK3bnRPXvsI58uQq+T72nHebsRe4R9x3IjLFw0l5BrM4KdOgZM5kZXYnHifA6Yaey7jdObrtF6ttTJM+aKgAOEpjY6M2bNigqqpzDW1bW5uKioqUm5srSVqwYIGam5tVUVER1766u73y+0efbydR0w/4PvZEdCEk3L6dNiWC0/J4PH2OyyQ57zhJ0WWyKnukmTIyXFFdGIF9eJIGqYyGCoCj1NbWDlvX1dUlt9sdWM7Pz9fBgwetjAWkPR4lQjx4kgapjIYKgOP5/X65XK7AsmEYQcsAEo/HooHUUPPmfeodc+7/Q7k4Ej8aKgCOV1BQoPb29sCyx+NRfn6+jYkAAHC+0DvLkjRZ0lPzzg3qxMWR+NFQAXC8uXPnqqGhQT09PcrKylJLS4s2btxodywAAByNO0/WoKEC4HhTp05VZWWlysrK1N/fr5KSEs2cOdPuWACAMEIHoJAYhAKpjYYKgCO1trYGLRcXF6u4uNimNACASDEABdJNht0BAAAAACBZcYcKAAAASFNhB65g5L+o0FABI6DAAACAVBfuvIaR/6JDQwWMgAIDAACA0dBQAQCAhAidQFTiTj+A1ENDBQAAEqJ3jCtoAlGJO/3J5siTq+T72BNYdmXnKXvp4zYmApyHhgoALBI6NwvzsiDVhN6Rmjxg2JgGZvB97AkaAt27a536nll+3s9Q25BuaKgAwCLMzYJUF+6OFFILd6eA4WioACABQu9GSVy1BQAgFdFQAUACcDcKAID0QEMFAACQhsLdSQ819gK3RWngJKFzcTI65/nRUAEAAKShSO6ku9058nj6rAkExwhtnhid8/xoqAAAQNRCr2APrgOAdENDlYR42R0AYDce/wGAs2iokhAvuwPOwxxTAIBUNdIdaS6snEVDBQAm4EIHACBVhWuceK/qnAy7AwAAAABAsuIOFRAFhhEFACQj3r8GEoeGKgnwboZzMIwoACAZ8VgykDg0VEmAIggAAAA4k+kN1TvvvKPdu3fLMAzNmTNHt956q9m7AICUcN/Tbeo+cTpoXd6k8XpszVybEgEAgGiZ3lCdOHFCDz/8sDIzM7VmzRoaKgAYQfeJ03q+el7QujseabUpTWoZehxpUgEAiWR6Q3XDDTdoYGBAW7ZsUVlZmdlfDwCwUOhdtGRpTp6vnie3O0ceTx9NKgAgoRJyh2rz5s1aunSprrjiCrO/HgBgodC7aHY3J7E0eHmTxg/LnSyNoZPxyCoAnGV6Q7Vp0yYdP35cL7zwgqZNm6Z169aZvQsAsFW4E8mfX2hTmDQTS4MX7gTf7sYwFfDIKgCcFXFD5fV6VVpaqm3btmn69OmSpKamJm3dulU+n0/l5eVatmyZ6urqEhYWcJrQeakG1zE3VWr5/qYWdfWeCiznTRo/7ESy75kXrY4FRCRZH9sEgGQRUUN14MAB1dTUqKOjI7Cus7NT9fX12rNnjzIzM1VaWqrCwkLNmDEjrkB5edkRb+t258S1r0RIRKa+OL/XKcdpaA6nZBoUa57tS7cNW3fbL1ab8vs57RhJzsxkha7eU8MaKDsl4wlyMmZOFU57bBMAUk1EDVVjY6M2bNigqqpzV+Lb2tpUVFSk3NxcSdKCBQvU3NysioqKuAJ1d3vl9xujbjf4srGTJDJTrN/rpOM0mMNJmaTE5In3+5x2jKTIM2VkuKK6MILodZ84rabHFwf+PZx2gjzSuzWc1Ecu1veTRvocAJgt9CmddH5CJ6KGqra2dti6rq4uud3uwHJ+fr4OHjxoXjIAQFIK925NOgvX5IQKbZZifT8pGY99uOOTPzlLj6y82qZEACIR2jyFvgKRTmIelMLv98vlcgWWDcMIWgYAwKlCR/5L5COIQ5ucke70JsMdu3CPbYaK5bgmy+AW77zzjnbv3i3DMDRnzhzm2QQQEHNDVVBQoPb29sCyx+NRfn6+KaEAIBmZcYIebojvcNuM9plYHg9Lxveawv3uWXNG/1zo72n3CXy4RsRpIrn75bTjaqYTJ07o4YcfVmZmptasWUNDBSAg5oZq7ty5amhoUE9Pj7KystTS0qKNGzeamS0teXetk+HtDlrnys6zKQ2AaJjxjlAsDU2sw4KnwmAF4X73O1ubbUgSn2RrZNkTDm4AABJtSURBVNPRDTfcoIGBAW3ZskVlZWV2xwkSeu7gys5T9tLHbUwEpJeYG6qpU6eqsrJSZWVl6u/vV0lJiWbOnGlmtrRkeLuVs2KH3TEAACFSZSJbJ0x07PS7ceGcOHFCmzdv1tKlS3XFFVfYHSdI6LmDd9c69T2zPGgbLs4CiRNVQ9XaGlyAi4uLVVxcbGogAEgFTjhpHYrR36IX7pFIO9/1ieVx0HBC/zfodudo+U/eMOW7w30m3COZyTZwhiRt2rRJx48f1wsvvKBp06Zp3bp1dkcaEXenAGvFfIcKADCyWB/DS5RkHP3NbpEcMysHt0hkM25lZqc9kun1elVaWqpt27Zp+vTpkqSmpiZt3bpVPp9P5eXlWrZsmerq6mxOCsCpaKgAAIhRKg/CkA4OHDigmpoadXR0BNZ1dnaqvr5ee/bsUWZmpkpLS1VYWKgZM2bEvJ9o5uaLZQL1vhg/FyknTurupEyDWZyUaZDVmUbbX6oeIxoqAEhBsYwaN9JjiqOJZDhtwIkaGxu1YcMGVVWdmz+nra1NRUVFys3NlSQtWLBAzc3NqqioiHk/3d1e+f3GqNvFM6l7oiaDT+aJ5q3i8fQ5LpNkz3E63/6S+RhlZLjOe2GEhgoALGLl0NhmjRYYaqSmq+nxxVH9H2Uk7wON9Dkni7UphT1qa2uHrevq6pLb7Q4s5+fn6+DBg1bGApBkaKgAwCLJNhpcOGb9DqlwLMJJ1d8rnfj9frlcrsCyYRhBywAQKsPuAAAAAE5RUFAgj8cTWPZ4PMrPz7cxEQCno6ECAAD4q7lz52r//v3q6enRqVOn1NLSomuvvdbuWAAcjEf+ACS9Q4cOqa6uTjt27LA7CpIU7z5h0NSpU1VZWamysjL19/erpKREM2fOtDsWAAejoQKQ1I4ePap9+/ZpzJgxdkdBEuPdp/TW2hrcTBcXF6u4uNimNACSDQ0VgKR28cUXa82aNVq5cqXdUQAg4by71snwdgetc2Xn2ZQGOGfygKE7W6uGrdv0jcdsSmQdGioAAIAkYXi7lbNih90xgGHCNU6hDVaqoqGCZcK9n8BjNgAAAEhmjPIHyzxfPU/PV89T0+OL9Xz1PHWfOG13JDiY1+vVokWLdOzYscC6pqYmLVy4UPPnz9fOnTuDtt++fbvVEQEAALhDBcB5Dhw4oJqaGnV0dATWdXZ2qr6+Xnv27FFmZqZKS0tVWFioGTNmxLyfvLzsiLd1u3PO+/O+CLYxm9X7G42T8gxmcVKmQWSKbH9OPE4AEA4NVYq47+m2oDs+PE6HZNbY2KgNGzaoqurcs9dtbW0qKipSbm6uJGnBggVqbm5WRUVFzPvp7vbK7zci2tbj6TNlG7O43TmW7m80Tsvj8fQ5LpPkvOMk2ZPJrL+njAxXVBdGACARaKhSRPeJ03q+el5gOfR9pXRHw5lcamtrh63r6uqS2+0OLOfn5+vgwYNWxgIAABiGhgppgYYz+fn9frlcrsCyYRhBywAAAHagoULSS9Tdp9DvNfO7Eb2CggK1t7cHlj0ej/Lz821MBAAA4lXz5n3qHXPuAmkyzl1FQ4WECG1GsuYkbl+JuvsU+r1mfjeiN3fuXDU0NKinp0dZWVlqaWnRxo0bbcnCxJoAAJijd4xLT82rCywn49xVNFQpKm/S+JjmfTLrbk9oM3Jna3PU3wEMNXXqVFVWVqqsrEz9/f0qKSnRzJkzbcnCxJoAAIxu8oAR1CAl492nSNBQpajBJmjo6E2R3F1J53eNwjWTsFdra/D//oqLi1VcXGxTGgAAEI2hzZPbnaOVu1YNuwM1eSCy0XadjIYKUUvVEfPCPeIHAAAAc6Ti3SmJhsqRYrkrZOXdFDvvYo00UAQAAEAyYNCr1END5UCJuksS+l6V3X+8seQx6y7SSO+YRfs5u48hAABILskw6FWqPo2UKDRUaST0D8HuP1478wzd99D3zKL5nGT/MQQAADBbaNN339NtMQ12ZiU7m0AaqjQW612aRH1POAwUAQAAkoXTHuczK0+47Z12UdnOV1JoqNJYJH9MkTRLZhWJkfYV7SN+iWzwAAAARuK0x/mclidV0VDhvKy8omLWvpx0+xkAAGA0sbyj7bS7YYnk9He6aKgAAAAAG8XyjrZZd5+sHEE5kiYwdDJgSZr0Wb+eX7AlsOy0u2w0VEAcwj1emDXHpjAAAABRsnIeznD7Gj7gxY3DPzin2ZT9hzZ0+ZOz9MjKq+P+XhoqIA7hbjff2WrOHz0AAECqi+TRPbPOrRI1cAUNFQAAAICUYuUgZTRUAAAAgMMl41Qydma2ctAKGioAAADAQcyaSsZKIw04MZjZ7c6Rx9Nnyr6cNkUODRUAAADgIE4aEjxSVg5u4bTjk2F3AAAAAABIVtyhAgAAABIk3KS9SC00VAAAAECCOO3xNJiPhgoAAABIAVbeDePO2zk0VAAAAEAKGHo3bKRR9cxqhLjzdg4NFQAAAJAmaITMxyh/AAAAABAjGioAAAAAiBENFQAAAADEiIYKAAAAAGLEoBQAAABAjBg6HDRUAAAAQIyer5434hDlSA888gcAAAAAMaKhAgAAAIAY0VABAAAAQIxoqAAAAAAgRjRUAAAAABAjRvmzmXfXOhne7sBy98BE5diYBwAAAEDkaKhsZni7lbNiR2D57kda9bx9cQAAAABEgUf+AAAAACBGNFQAAAAAECMaKgAAAACIEQ0VAAAAAMSIhgoAAAAAYpSwhurQoUNavnx5or4eAAAAAGyXkIbq6NGj2rdvn8aMGZOIrwcAAAAAR0hIQ3XxxRdrzZo1GjuWaa4AAAAApC46HgBJq6enR7W1tZowYYKuu+46ff3rX7c7EgAASDMMSgEgab300ksqLy/Xxo0b1djYaHccAACQhqJqqLxerxYtWqRjx44F1jU1NWnhwoWaP3++du7cGbT99u3bzUkJAGF89NFHKigosDsGAABIYxE/8nfgwAHV1NSoo6MjsK6zs1P19fXas2ePMjMzVVpaqsLCQs2YMSPmQHl52RFv63bnxLyfRIk2U1+Yz5j9eznlOA3N4ZRMg5x4zJ12jCTnZSooKJDH41F+fr7dUQAAQJqKuKFqbGzUhg0bVFVVFVjX1tamoqIi5ebmSpIWLFig5uZmVVRUxByou9srv98YdTu3O0ceT1/M+0mEWDOFfsbM38tJx2kwh5MySYnJE+/3Oe0YSZFnyshwRXVhJB633nqr6urqNG7cOJWWllqyTwAAgKEibqhqa2uHrevq6pLb7Q4s5+fn6+DBg+YkA5C2vF6vSktLtW3bNk2fPl3S2ceLt27dKp/Pp/Lyci1btkz5+fnasmWLzWkBAEA6i2uUP7/fL5fLFVg2DCNoGUhHk/r9urP13J3cyQOGNn3jMRsTJRerHi+WYn/EONyjunZwQoahnJRnMIuTMg1K90yTB4xhNXL70m22ZorGoUOHVFdXpx07dliyP++udTK83YFlV3aeJfsFELm4GqqCggK1t7cHlnmXAZA6/3mhnq+eF1geeuKA0Vn1eLEU+SPGUmIfzY2F0x4LdVoej6fPcZkk5x0nyfpMoReY7mytCrt/pz1iLElHjx7Vvn37NGbMGMv2aXi7lbNih2X7AxC9uIZNnzt3rvbv36+enh6dOnVKLS0tuvbaa83KBiAN1dbWavbs2UHrwj1e3NnZaXU0AGnu4osv1po1azR2LNN4AjgnroowdepUVVZWqqysTP39/SopKdHMmTPNygYAkni8GAAAOFfUDVVra2vQcnFxsYqLi00LBACheLwYAAA4VVyP/AGAFXi8GEAieb1eLVq0SMeOHQusa2pq0sKFCzV//nzt3LkzaPvt27dbHRGAg/EQMADH4/FiAIlixciisY4oKjljVFG79x+OkzIxqmjk4smTqN/FjO+loQLgSDxeDMAKVowsGumIoiONuGjnyJCMTDk6RhWNTLx5EvW7mDGiKA0VAABIW7W1tcPWhRtZ9ODBg1bGApBEeIcKAABgCEYWBRANx92hysiIvGBFs61Vos009gJ30GfyJ2eZ/ns54Ti5J1wYlMMJmYYyM0/ov2Ho725HJrNEksmJuUcSadbQf9PQv1u7OCHDUE7JM/RvzimZhiJTsHA1MtL/L0xUbrNHFo3n3MYJ9cbu/YfjlEzUm+jEmifWc6nRmFVrXIZhjP5QLwAAQAqbN2+eXnzxRU2fPl2dnZ1asmSJXnnlFWVlZam0tFQbN25kMBwAYTnuDhUAAICdGFkUQDS4QwUAAAAAMWJQCgAAAACIEQ0VAAAAAMSIhgoAAAAAYkRDBQAAAAAxoqECAAAAgBjRUAEAAABAjGioAAAAACBGSdtQdXV1acWKFbr55ptVWlqqY8eO2R0p4NChQ/riF79odwxJ0rvvvquSkhItXrxY5eXl+vOf/2xblqamJi1cuFDz58/Xzp07bcsx1JNPPqmbbrpJN910k+rq6uyOE/Doo4+qurra7hiSpNbWVn3nO9/RjTfeqE2bNtkdxxbUm8hQb0bm1FojUW+chnozOmrN+VFvImNqvTGSVHl5ubFr1y7DMAxj165dxt13321zorNOnjxplJaWGp///OftjmIYhmFcf/31xh/+8AfDMAxj9+7dxqpVq2zJcfz4ceP66683ent7jU8++cQoLi42Dh8+bEuWQb/5zW+M7373u8ann35qnDlzxigrKzNaWlpszWQYhtHW1mYUFhYa69evtzuKceTIEeOaa64xPvzwQ+PMmTPGkiVLjH379tkdy3LUm8hQb8Jzaq0xDOqNE1FvRketGRn1JjJm15ukvEPV09Oj999/X6WlpZKkW265Rffcc4/Nqc565JFHVF5ebncMSdKZM2d0991367LLLpMk/d3f/Z0+/PBDW7K0tbWpqKhIubm5mjBhghYsWKDm5mZbsgxyu92qrq5WZmamxo0bp89+9rP64IMPbM30l7/8RfX19Vq1apWtOQa9+eabWrhwoQoKCjRu3DjV19dr1qxZdseyFPUmMtSbkTmx1kjUGyei3oyOWnN+1JvImF1vkrKhOnr0qC666CI98sgjuuWWW3TXXXdp3LhxdsfSr371K50+fVrf/OY37Y4iScrMzNTixYslSX6/X08++aS+/vWv25Klq6tLbrc7sJyfn6/Ozk5bsgz63Oc+py996UuSpI6ODu3du1fXXXedrZl+/OMfq7KyUpMmTbI1x6A//elPGhgY0KpVq7R48WLt2rVLF1xwgd2xLEW9iQz1ZmROrDUS9caJqDejo9acH/UmMmbXm7EmZkuIvXv3avPmzUHr/vZv/1aHDh3S2rVr9aMf/Ui7d+9WdXW1XnrpJdsyXXrppfJ6vdqxY4clGSLNtGPHDp05c0bV1dXy+XxauXKlLfn8fr9cLldg2TCMoGU7HT58WCtXrlRVVZUuueQS23Ls3r1b06ZN09VXX609e/bYlmOogYEBtbe366WXXtKECRO0evVqvfbaa/rOd75jd7SEoN7El4l6c35OqTUS9cYJqDex56HWjI56c35m1xuXYRiGyRkT7siRI/r2t7+td999V5J06tQpFRUV6cCBA7Zl2r17t7Zv366JEydKkt5//31ddtll2rlzp7Kzs23L9cknn2j16tXKzc3Vli1blJmZaUuO1157Te3t7aqtrZUkPfXUUzIMQxUVFbbkGfTuu+/qrrvu0v3336+bbrrJ1izf+9735PF4NGbMGH388cc6efKkbr75Zt1///22ZfrZz34mr9ermpoaSdLOnTt1+PBhPfTQQ7Zlshr1JnLUm5E5qdZI1Bunot5EhlpzftSb0Zleb0x4r8sWCxcuDLw89stf/tJYsmSJzYmCOeGlTcMwjNWrVxs1NTXGwMCArTkGX9zs7u42Tp48aXzrW98yDhw4YGumDz74wCgsLDTa2tpszRHOq6++6oiXNv/lX/7FWLBggfHxxx8bPp/PWLlypdHY2Gh3LMtRbyJDvQnPybXGMKg3TkO9GR21ZmTUm8iYXW8c/8jfSBoaGrRhwwY99thjys7O1iOPPGJ3JMc5dOiQfvWrX2nGjBn69re/Lens873PPvus5VmmTp2qyspKlZWVqb+/XyUlJZo5c6blOYZ67rnn9Omnnwb9b6e0tFRLliyxMZWzzJo1Sz/4wQ+0dOlS9ff366tf/apuueUWu2NZjnozOurNyKg1kaHenEW9OT9qzflRbyJjdr1Jykf+AAAAAMAJknKUPwAAAABwAhoqAAAAAIgRDRUAAAAAxIiGCgAAAABiREMFAAAAADGioQIAAACAGNFQAQAAAECMaKgAAAAAIEb/D5AtPofjXp+DAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "net.load_state_dict(torch.load('net.pth'))\n", "mls.torch.trace(net, True)\n", "bins = np.linspace(-6, 6, 50)\n", "fig, axes = plt.subplots(1, 3, figsize=(12, 4))\n", "for i, (label, x) in enumerate(variations.items()):\n", " y_pred = net(x)\n", " ax = axes[i]\n", " ax.hist(net[0].input.reshape(-1), bins=bins, histtype='step', label='IN0')\n", " ax.hist(net[1].input.reshape(-1), bins=bins, histtype='step', label='IN1')\n", " ax.hist(net[1].output.reshape(-1), bins=bins, histtype='step', label='OUT1')\n", " ax.legend(title=label)\n", " ax.set_yscale('log')\n", "plt.tight_layout()\n", "mls.torch.trace(net, False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The solution to this problem is to carefully preprocess your network inputs to ensure that they flow \"smoothly\" (i.e., without saturation) through your initial layer. PyTorch does not provide utilities for this, but [sklearn.preprocessing](https://scikit-learn.org/stable/modules/preprocessing.html) is a good starting point.\n", "\n", "An alternative approach would be carefully tune the distribution of initial random Linear parameters, but preprocessing your inputs is the standard and simpler approach.\n", "\n", "We will see below that it can also be useful to standardize the distribution of hidden node values (aka \"BatchNorm\")." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Minibatch Training" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Recall that **stochastic** gradient descent involves performing parameter updates on random subsets of your data, known as \"minibatches\", rather than the whole data (as we did above). We now have the tools to study the tradeoffs when choosing the batch size hyperparameter value.\n", "\n", "PyTorch provides [convenient utilities](https://pytorch.org/docs/stable/data.html) for wrapping your training data tensors into a \"minibatch factory\":" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "xy_train = torch.utils.data.TensorDataset(x_train, y_train)\n", "loader = torch.utils.data.DataLoader(xy_train, batch_size=5, shuffle=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We use the standard python iterator syntax to perform a random pass through the full data (one \"epoch\"):" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "BATCH 0:\n", " x_batch[:,0] = tensor([ 0.2595, 2.6137, 1.3224, 2.1310, -1.8430])\n", " y_batch[:,0] = tensor([ 0.2566, 0.5037, 0.9693, 0.8471, -0.9632]).\n", "BATCH 1:\n", " x_batch[:,0] = tensor([2.6128, 2.4146, 1.7817, 2.1530, 1.1425])\n", " y_batch[:,0] = tensor([0.5045, 0.6647, 0.9778, 0.8353, 0.9097]).\n", "\n", "...\n", "\n", "BATCH 198:\n", " x_batch[:,0] = tensor([-2.3623, 0.5899, 0.9353, -2.8777, 1.7667])\n", " y_batch[:,0] = tensor([-0.7028, 0.5562, 0.8048, -0.2608, 0.9809]).\n", "BATCH 199:\n", " x_batch[:,0] = tensor([-2.8350, 2.9135, 2.8679, 0.9135, -1.2125])\n", " y_batch[:,0] = tensor([-0.3018, 0.2261, 0.2703, 0.7916, -0.9365]).\n" ] } ], "source": [ "for batch_num, (x_batch, y_batch) in enumerate(loader):\n", " if batch_num < 2 or batch_num >= len(loader) - 2:\n", " print(f'BATCH {batch_num}:\\n x_batch[:,0] = {x_batch[:,0]}\\n y_batch[:,0] = {y_batch[:,0]}.')\n", " elif batch_num == 2:\n", " print('\\n...\\n')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Update our training loop to use a `loader` and try different batch size values. Note that the learning rate should generally be adjusted when changing the batch size: with less data used for an update, we are less confident in how much to change the parameters and so want a correspondingly smaller step size (learning rate)." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAELCAYAAAAoUKpTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9eXxU9b3//zxnJglJCCEkk0ASCISQhLWgKKu4i1qCWlFZLPi95Xax2BZbqbe1F6+W1mu92l+1dalarba2aLWKWkRFlL1Ewk4IISQQsi9EkkAyc875/TFzTs5MZiYzWSfh83w89MGcOct7Jsl5nffnvUmapmkIBAKBQOAHua8NEAgEAkHoI8RCIBAIBB0ixEIgEAgEHSLEQiAQCAQdIsRCIBAIBB0ixEIgEAgEHSLEQiAQCAQdYu1rA/xRV1fHunXriIqK4sorr+S6667ra5MEAoHgoiSkPYvXXnuNFStW8Oijj7J+/fq+NkcgEAguWkLas6ipqWH48OGdPr6+vglVDb5APT5+MLW1jZ2+bk8h7AqeULVN2BUcwq7gCdY2WZaIi4v2+X5Ii8Xw4cOprq4mMTGxU8erqtYpsdCPDUWEXcETqrYJu4JD2BU83WlbSIvFHXfcweOPP05YWBiLFy/ua3MEAoHgoqVPxKKxsZHFixfz3HPPkZqaCsCGDRt49tlncTgcrFixgmXLlpGYmMgTTzzRFyYKBAKBwESvi8X+/ft56KGHKC4uNrZVVlby1FNP8fbbbxMeHs7ixYuZMWMGGRkZvW2eQCDwg6Zp1NdX09p6Aei95ZeqKhlVVXvteoESqnaBP9skwsMHERdnQ5KkgM/X62Kxfv161q5dy5o1a4xtO3bsYObMmQwdOhSA+fPns3HjRlatWtWla8XHD+70sTZbTJeu3VMIu4InVG3rj3ZVVVVhtcrYbKOQpJBOphT4QNNU6upqgAvYbIHHg3tdLNatW9duW1VVFTabzXidmJjIgQMHunyt2trGTgV4bLYYqqvPdfn63Y2wK3hC1bb+ald1dS3DhiWhKAC990Rttco4HKH3BB+qdoF/26KjY6murkSSIo1tsiz5fcAOiQC3qqpu7pCmaUG5R91JUUMJ26pLSQ5PJT02rU9sEAhCFVVVsFhC4rYh6AIWixVVVYI6JiR+6sOHDyc3N9d43ZV02a5Q1FDC7/JeQFEdWGQrP5j2bSEYAoEHffUgJ+g+OvMzDIlFx9mzZ7Nz507q6uo4f/48mzZtYt68eb1ux/H6EzhUByoaiurgeP2JXrdBIBAEzt69uVx//RXcc89SVqxYwrJli1i//o2Ajj1xopC7777Tbdsbb7zO0qW3s3jxN/j8883tjikvL2PRopygbNQ0jWee+S1Ll97O3XffwYED+7zut2nTRu6++w4WL76Nf/zDvWOFw+Hghz/8Hnv35no9tjcICc8iKSmJ1atXs3z5cux2O4sWLWLKlCm9bse4uLFYZSuKpmCRLIyLG9vrNggEguDIyhrPM8+8AEBzcxN3330nl102gzFj0n0e869/vc/zz/8eq7XtFnj06GE2bfqQP/3przQ3N/Gd7/w/pk27lCFDYrtk35Ytn1JScpLXX3+T0tLTrFnzI15//U23a1dXV/HHP/6Bl156jbCwcL773f/gkkumM2ZMOqdOFfPrXz/CsWPHumRHV+kzsdi82V21c3JyyMkJTrG7m/TYNH4w7duUtYqYhUDQnRSeaeDYqXqyRsWRkdK1m68/WlpakGWZwYMHc889S9u9/8gjv2LYsAS2bfuchx9exy9/udZ4b+fO7Vx55TVEREQQERHBtGmXsn37Vm66aYHXa61b9zBffdVAaelpvvvdVbz00gter7dz53auvfYGZFlm1Kg0kpKGc+jQAaZOvcTYLzf331xyyXRDmK6++lq2bPmUMWPSef/9d1myZDnr1/+1q19PlwgJzyKUSI9NY4ZtUkhmqggE/ZHCMw385o08HIqK1SLzwJJp3SoYx44d5Z57lqJpKqWlp7nmmutJSLDxyiu+b67r1v2G8vIyt201NdWMHz/ReB0fn0B1dZXfa8fGDuV///cpAK644iqv+9TUVBMfn+B23qqqqg73OXLkMAD33vtDACEWAoFgYHPsVD0ORUXTQFFUjp2q71axMC9DNTU18uMf/4A///llPvvs03b7PvLIrxg1arTX8zizMD1f+w/rTpw4CQBFUfjWt77p9Xresj1l2T3AHMg+fY0QC4FA0KNkjYrDapFRFBWLRSZrVFyPXSs6ejDXXHM9ubm7/XoW3rDZEqmpqTFe19XVMmqU/6XoiIgIACwWi8/rJSYmUVvrft6EBFu7ffbvz/O7T18TEtlQAoFg4JKREssDS6Zx27z0bl+C8kRRFPLyviQzMzvoY2fOnM3nn2/mwoUL1NfX8+WXe5g+/fIu2zRz5hw2bdqIoiiUlp7m9OlTjB8/wW2f6dMv58sv91BfX8+FCxfYsmUzM2bM6vK1uxPhWQgEgh4nIyW2x0RCj1lIkjPFNCMjk2XLVgR9ngkTJnHDDTezcuVyFMXBypXfNdph3HPPUp544v/rlH1XX30tR44cYsWKJQA8+OAviIgYRE1NNT/5yQ955ZW/YrMl8p//eS8/+MF3sNsd5OTcwoQJkzp1vZ5C0jQtdJuxdxHR7qN3CFW7IHRt6692VVSUMHx472cJ9nVbjd/97v/41re+Q3S0ezuMvrbLHx3Z5vmz7Kjdh1iGEggEAj9omkZ29oR2QnGxIcRCIBAI/CBJEjfccFNfm9HnCLHwQkFNER8Vb6aooaSvTREIBIKQQAS4PShqKOHpfS9gVxxYTc0EixpKOF5/gnFxY0Vlt0AguOgQYuHB8foTOBQHmkczwd/lvYBDdRcQgUAguFgQy1AejIsbi9ViRUbCIlsZFzfW6EbrKSACgUBwsSA8Cw/SY9P476t+xL9PHnRbcrLKVmPOhehGKxAILjaEWHghMyGdOM1GUUMJHxVvZlzcWH4w7dsiZiEQhBh79+by05+uJiVlJJqm4XDYueWW27nzziUdHnviRCFr1/6M119vmx3xxhuvs2HDO6iqxve+t4orr7wGcM6a+POfX8LhcHDHHUu4/fY7251v7tzpbNsW3LwJX9czs2fPbp555ilaWlq45prr+fa37wXg888/4+WXn0dRVMaPn8CaNT8nLCwsqOsHgxALH+hT88xxivmj2/8gBQJBxyiVhTjK8rEmZ2NJyujWc/f0PIuWlhafsya6QiDzM1paLvDrXz/CM8+8QGJiEmvW/IidO7czdeolPPXU47z88usMGxbP2rX/xYcfbuCWW77RJZv8IcTCB97iFMKjEAiCR6kspPn9x0G10yqHEbVgTbcLhk5PzLMAfM6a8MZLLz3P4cOHqKqq4Pbb72Lr1i3U1dW57fODH9zPvn17O5yfceTIYUaOHEVycgoAN9xwE5999gmzZs3hrbc2YLVajV5WMTFDuvbldYAQCx8YU/NEnEIg6BKOsnxQ7aBpoDpwlOV3q1j09DwLSZJ8zprwRWtrC6+//iYAt922yOs+n3zyUYfzM7zNudD3sVqt7Ny5nUcf/W9sNhuXXz7Tr01dRYiFD/SpeR3FKUT9hUDgH2tyNq1yGKgOkK1Yk4PvCOuPnp5noapK0LMmzE0A779/lVfPIpD5GZ77gPs+s2bN4cMPP+X553/PE0/8mocfXufXrq4gxMIP6bFpXgVAF4josGjeOv6eqL8QCPxgScogasGaHotZmOmpeRbBzprQ51wAPPnkM1732bdvb4fzM5w21Rqva2trSUhI4KuvGsjPP2p4E9dffyNr1/5XRx+xS4g6iyDRA98bij7i7wXviPoLgSAALEkZRExb0KNCAT0zz6KnZk0EMj9jwoRJnD5dQmnpaRRF4eOPP2LmzDlomsYjj/yCiooKAD777BOmTJnaZZv8ITyLIDEHvnG5jZKGiGsIBH1Eb8yz8DVrYvXq+/jWt75DdvYEf6cO+nr6/IyEBBs/+9lafv7zNbS2tjBr1hyuvvpaJElizZqf89Of/giQGD16DA888LOgbQgGMc/CC/56+uuehR74XjRuIU32pl6JWfTXGQh9Saja1l/tuljnWfhi/fq/Mn36DNLTQ+9BsbvnWQjPIkjMge/osOheEwqBQBB6xMXFdbneor8gxMIDpbKQ+oKTKLFjfK6v6sIgmgsKBBc3N9309ZD0eHoCIRYm9OKhZleKn7/iIX9FeyKdViAQDDSEWJjoqHjILAK+iva8tQkRgiEQCPo7IS8WR44c4fHHH+eVV17p8Wt5Kx7yV1PhWbRX1FDCh0WbRJsQgUAw4AhpsTh9+jRbtmzBYrH0yvX04qGIhpO0xI6hZFCY4SVIkoSmaW4iMH/0NW5LT7/LewG7agdAMs3DEAgEgv5OSBXlvfjii9xzzz3GfykpKdx7771unSF7GktSBnFzvoElKcMtLqFpKpIkuQ1FMqPvCyAB2XEZYglKIOhh9u7N5frrr+Cee5ayYsUSli1bxPr1bwR07IkThdx9t3ur8TfeeJ2lS29n8eJv8Pnnm43tmzZt5O6772Dx4tv4xz/aWpr/+9+7WbFiMYsX38YLL/zB63UWLcpp14eqI3xdzxuPPvrffPjhBuN1RUUF3//+f3LXXd/gwQfvp7m5Oahr+yKkPIuVK1eycuXKPrWh8EwDWw6Ukxof1S4u4a+mIjos2tk/RnMOSro5/QYR8BYIeoG+alGenJzMunX/w9NPP+/WPnzWrDld+jzV1VUBtUSvqanm8cd/xZdf/ptLL73M2P7kk49x222LuPHGm3jxxRd45ZUXuffeH3TJJggxsehrCs808Js38lAUFYtF5oEl0wJuJvjW8fdQNRVZklk0biEAHxVvFv2jBAJ674GpN1uUT516CSNHjvTaPtwbH364gX/9630aGs4yZ848amtrKCg45rbP0qXfRFGUgFqib9r0L6644kpiY9vmXzgcDvbty+NXv3oCgJtuWsCqVd/pP2LR2NjI4sWLee6550hNTQVgw4YNPPvsszgcDlasWMGyZct8Hv/888/3hpkcO1WPQ1HRNEBROXaqnq/PGt3hL7d5CQpN4/S5UkMgvMU6hFgILiZ6OkOwr1qU+2sf7ovq6ipef/1Nv0vrr732p4Baoi9duhyAAwf2GdvOnj1LdHS0cX6nTZV+bQqUHheL/fv389BDD1FcXGxsq6ys5KmnnuLtt98mPDycxYsXM2PGDDIyurfJmL/SdW/MnJLChh3FOBwqVqvMzCkp2GwxHR53uTSZjSWf4lAVrLKFyMhwFKN/FMiShIaEVbZw+ZjJ2BI6PqcvArGnLwhVuyB0beuPdlVVyVitwYU6TzQUuWUInmgoIjN+TNB2ebuuxSKTnT2BZ5/9I+BsUf6jH63i9df/xObNn7Tb/5e/fIy0tNHGsZ7ntVjaPp8kgdVqQVEUL9tlJAkkSTK2y7Lktp+nnbIskZWVzaBB4QA8+ujadp7FsmXLAc3r9Xx975IkIctOO2TZ3SbnNu/HyrIc1O9gj4vF+vXrWbt2LWvWrDG27dixg5kzZzJ06FAA5s+fz8aNG1m1alW3XjvY3lDx0WH8ZPE0SmubSY2PIj46LKD+PXHYuG9q23IVwBZ5F3iJdcRptk73BOqv/YT6klC1rb/apapq0BXLY2PT3WJ/Y2PTgz6Hrz5HiqK6Zm8734uIiOLqq50tyv/0J++ehb6voqhurxMSbFRVVRmva2pqSE0dBThblJu3DxuWQHy8jZqaGmN7dXU1w4Yl+LRTVTXCwyOM9//rv9a22w+c8RRv1/P1nWmahqo6v4MhQ4bS2NhIS4udiIgwKiuriI+3eT1WVVW3n3VHvaF6PBtq3bp1TJ8+3W1bVVUVNltbP/jExEQqK7vHVeoqGSmx3HFtJhkpsSiVhbTkvY9SWdjhcemxaUYqrd4/akH6fH4w7dvMTZnhlmYrEFxMeP499OTfQW+2KJ8wYRKnTrVvH95VutIS3Wq18rWvTeXTTz8GYOPGD5g5c3aXbYI+CnCrqtpu8pQk+Z881dt0dW6wr8FJAsHFSE/+PfRli/Jf/OLhdu3DAR577FHmzp3H3LlXBm2HzZbo83o/+ckPWLnyu35bov/4xw/yy1+u5c9/fonExOHdNj2v11qUX3PNNfz5z38mNTWVd955h9zcXNatc36I3//+92ia1ufLUDo2Wwylm96gNfcfztYfkkz49G8QMW1Bxwd70J1ZIP116aIvCVXb+qtdokW5O77s+vzzzYSFhTN79tw+sMpJd7co75OivNmzZ7Nz507q6uo4f/48mzZtYt68eX1hilculB5DbawFyQKS3Om5weaper/Le4GihpIesFYgEIQaDofSbupdf6dPlqGSkpJYvXo1y5cvx263s2jRIqZMmdIXprRDqSyk/IPH0RQ7SBbCsq8kLHNOp8ZBenam3V2e266XlCjWEwgGHtdee31fm9Dt9JpYbN682e11Tk4OOTk5vXX5gHGU5aMpDufyEyrS4PhOzw02V4BLksyu8i9xaA5kSebakfPYUrpdFOsJBIJ+QUj1hgoFrMnZSBZrl5afdMxZILNGXIZDcxbuqZrKJ6c+b9edViDoDwzgScwXDZ35GYp2Hx5YkjIYsexhao7sxZqc3WmvQkfPAilqKGFH+b9RNWfASUNDlmQkDdGdVtBvsFrDaWr6iujoISGXwSgIDE3TaGr6Cqs1PKjjhFh4YVBqFhERyd16zvTYNO7KvI2/F7yDpmlYO2hMKBCEInFxNurrq2lsPNur15VlGVUNvWyoULUL/NtmtYYTF2fz+p4vhFj0InNTZpA8eLgIagv6LRaLlYSEEb1+3f6aatyXdLdtQix6GVGsJxAI+iMiwB0CFDWU8FHxZlGHIRAIQhbhWfQxPd2+WSAQCLoD4Vn0MZ6FeyKFViAQhCJCLPoYvXDP12xvgUAgCAXEMlQfoxfumWdhfFS8WWRLCQSCkEKIRQigi8Lu8ly3liB3Zd7G3JQZfWydQCAQCLHoM8xNBAF+l/cCdtVuvK9qKn879janz51hxohLhZchEAj6FCEWHaBUFuIoy++W1h86nhlQM4ZfikN1tNtPQ2Nb2S52V3zJonELodpBcniqEA6BQNDrCLHwQ1en5fnCMwMKNKM7LTj77ai0lek7VDt/L3gHNA2LSK8VCAR9gBALPzjK8kG1O9uVqw4cZfndIhbm1uUW2cqMEdOZMWK627KUHr9QNQUkCU3T0HDa4TkXQyAQCHoaIRZ+sCZn0yqHgerocrtyM54ZUPoN33zjT49NMwQkOiyat46/h6IpSEjsKv8SRVNEEZ9AIOg1hFj4wZKUQdSCNd0es4DAekSZ90kePJyy1lJO11ayvWy3WxGfEAuBQNDTiKK8DrAkZRAxbQEALXnvo1QW9okd6bFp3DbhRkbGpCBJEpIo4hMIBL2I8CwCoKcC3cFSUFPEW8ffQ9VUZElm0biFwqsQCAS9gvAsAsBboLsvOFxV0JZiq2k02Zv6xA6BQHDxIcQiAKzJ2SCHdctc7q4wMTFT9JESCAR9gliGCoCeDHQHQ2ZCutcsKoFAIOhphFgEiCUpo89EwoxnFpVn2xAhJAKBoCcQYtGPMbcNkSUZCUnUXwgEgh5BxCz6AfrY1YKaIrft5rYhqqbg0MQQJYFA0DOErGexZ88e3nzzTTRN4/LLL+eOO+7oa5N6pKlgR5i9h40ln3Lf1DaPwdw2RHJ5FqqmiOC3QCDodkJWLL766iseeeQRwsPDuffee/tcLPqq1sLsPThUxa1i29w2JDosmtPnSgFJtDQXCATdTsiIxYsvvsi2bduM1y+//DKapvHEE0+wfPnyXrUlv7iOXQfOkDUqjoyUWKDnmgp2hNl7sMqWdh6DLgpuLc9HXNrjdgkEgouLkBGLlStXsnLlSuP1V199xa9//WuWLl3K5MmTe82OwjMNPPG3POwOFatF5oEl08hIie2xpoIdYfYeLh8zmTjNZrynZ0LVXTjr1vJcj1eIzCiBQNBdBCUWra2t1NTUkJyc3FP2GPzyl7+koqKCV199lREjRvDjH/+4x68JcOxUPQ6HiqaBoqgcO1VPRkpsn9Za6OmytoQYqqvPAe0zoSySxYhXRIdFu3kaIjNKIBB0lQ7F4uOPP2bXrl2sXr2ahQsXcu7cOVatWsWKFSsCukBjYyOLFy/mueeeIzU1FYANGzbw7LPP4nA4WLFiBcuWLWt33OOPPx7kR+keskbFYbXKOBwqFotM1qg4471QqbUA91iGpqnMSr6cYYPiGBc31iPOYeetgncZGZMqYhkCgaDTSJqmaf52WLRoEevWrePw4cNs27aNRx55hOXLl/P22293ePL9+/fz0EMPcfLkSTZu3EhqaiqVlZUsWbKEt99+m/DwcBYvXsyTTz5JRkZo3ITBGbM4eKKGyWMTyB49rN37F0qPcb7kMJFpExmUmtUHFjqbCj6y5bc4VAWrbOG/r/oRmQnpbu/ZFadg6ITJVtZevdrYTyAQCAKlQ89C0zSysrL44x//yLx58xg8eDAd6IvB+vXrWbt2LWvWrDG27dixg5kzZzJ06FAA5s+fz8aNG1m1alUnP4JvamsbUdXAbDWTPXoY8dFhAMayj445K6q+lzvQ2mxty1Bx2LhvalvrjzjN1u69D4s2cbT+uHG8XXXw75MH3eIe3W1XqBGqtgm7gkPYFTzB2ibLEvHxg32/3/EJZD788EO2bdvGnDlz+Pzzz5EkKaCLr1u3junTp7ttq6qqwmZru1klJiZSWVkZ0PlCgVDpQAvOWMb80de0W1rSA99TE6dgkSzGdqsk6i8EAkHn6NCz+OlPf8ozzzzD/fffj81m49lnn+Whhx7q9AVVVXUTG03TAhafUKCvsqICxRz4tspW7sy8VdRfCASCLtOhWEyfPp1XXnkFcGZDPfnkk13Khho+fDi5ubnG6+rqahITEzt9vt4mVDrQ+sIc3FZUB032JpZk397XZgkEgn5Oh8tQH3/8MY8++iiNjY3ceOON3HLLLbz66qudvuDs2bPZuXMndXV1nD9/nk2bNjFv3rxOn68v0EethppQQFsRn5h5IRAIupMOxeL555/nzjvvZNOmTUydOpXPPvuMd999t9MXTEpKYvXq1Sxfvpxbb72VBQsWMGXKlE6fT+COXsS3IH2+3/qKooYS3sj/B2/kv01RQ0kvWykQCPobPZoNpbN582a31zk5OeTk5ARnqSBgPGdeeFLUUMJv9z6HoikA7CrP5YeXfEfEMwQCgU8CzobaunVr0NlQgtCjqKGED4s2GUIB4NBES3OBQOCfgLOhfvzjH3dLNtRAQqkspLVgOxIQljknJGMYZvRMKbtqd9suUmoFAkFHBJwNdebMGUpKSvjb3/7WG3aFPEplIc0bHnOm0AL2Y1uJynkwpAVDz5QCkIBRMamiDYhAIAiIDsWiuLiY73//+1RVVaGqKnFxcTz//POMHXtxP4k6i/McbRt6sW15Z4kOi3YuIWpgla0syrxFiIRAIAiIDmMWjz76KCtXrmTPnj18+eWXfO973+N//ud/esO2kMaanA2ySWtDsEDPTFFDCW8dfw9VcxZFLhq3UAiFQCAImA7Fora2lttuu814ffvtt1NfX9+jRvUHLEkZROU8iHX81YSNv5qonAcBaMl7H6WysI+ta495CQpNo8ne1LcGCQSCfkWHy1CKonD27Fmj8V9dXV2PG9VfsCRlEOladmo9uoWWba+BptJq6d0Gg4Fgnrjnr1hP7ysVHRZNk71JDE8SCARAAGJx9913c9ddd3HTTTchSRIffvhhwLMsLhaUykKXULjSURV7yMUvzBP3fAnAtjO7+XvBO6iaCoCEJIYnCQQCIACxuOuuuxg1ahTbtm1DVVXWrl3L7Nmze8O2foOjLL9NKACQ0BprUSoLQ04wvN30ixpK2F2ey46yPaioxnbzmFYhFgLBxU1AY1VnzZrFrFmzjNf/93//12tjTvsD1uRsWi3hoNgBCWQJe/4W7AXbQ245yhNftRfg9CxEfymBQABBzuDW+ctf/nJRiEXhmQZ2HCwHYPbkEWSkxHrdz9yJVmusxZ6/xW3eRSiLhVvg24UsyVw7ch6R1kGGUHxUvFnELwSCi5hOiUWwvaH6I4VnGvjfv+5FUZyfddvBctYsvcSvYFiSMlAqC7EXbA/ZeReemAPfkiQza8RlbkV6Zs9DlmTuyryNuSkz+thqgUDQ23RKLC6G3lDHTtUbQgHgUDSOnar3KRY6oT7vwhN/gW+9j5S+RKVqKn8veIfkwcOFhyEQXGT4FItNmzZ53a5pGqqqen1vIJE1Kg6LRTIEw2qRyBoVF9CxupfRX/AW+PYVy9A0TQS8BYKLEJ9i8dprr/k8aPLkyT1iTCiRkRLLT5deElDMYiDiGcuQcHqTVhHwFgguSjolFhcLGSmxF5VAmPEs4ls0bqHXIr2ihhK2VZeSHJ4qvA2BYADTqZjFxUjhmQaOnaona1TcRSEgvmIZRQ0lRmZUWWMFfy94B03TRPGeQDDAEWIRAIVnGvjNG3k4FBWrReaBJdMuGsHw9CJ+l/cCDtWBLMnO+JWriM8hivcEggFNh40EBc7MKIeiommgKCrHTnXcSFGpLAzZpoKdRY9jaGiomuJW7S1JkohlCAQDGJ9iUVZW5vOgL774okeMCVWyRsVhtcjIElgscodZUUplIc3vP05r7j9ofv/xASMYehxDRkKWLFglKxJgcdVfCK9CIBi4+FyG+v73v88777wDwH333cfTTz9tvPfUU08xb968nrcuRMhIieWBJdMCjlk4ByPZ+00Vd6DocYzd5bmAxMiYFJrsTVw+ZjJxmq2vzRMIBD2IT7EwV2mfPn3a53sXC8FkRlmTs2mVw/pNFXew7K7Yi0N1GEHtzIR0qqvP9bVZAoGgB/EpFuYqbc+K7Yuhgrsr9Lcq7mAwxy30jrQzmGTMwdAzpzxfCwSC/k1AnoUgePpbFXegeBuiVFBTZGRJWWUrV6XO4dPTX4iUWoFgAOFTLFRVpaGhAU3TUBTF+Dc4p+cJAkOpLBxQHoa3+ottVdsNb8Oh2vnk1OdoOH9XREqtQDAw8CkWBQUFzJw50xCIGTPaOpHWgyMAACAASURBVI32xjLU8ePHefrpp4mKiiInJ4c5c+b0+DW7Gz0rCtVOqxx6o1Y7i2f9xcTETKyyFYfqnOehCwWIlFqBYKDgUyzy8/N70452NDc387Of/QyLxcKTTz7ZL8XCMyuqtWA78gDyMnQyE9JZNG6h20hWwGhpLrwKgaD/47eCW1+CslqtNDY2smPHDrKyskhL6/4//hdffJFt27YZr19++WVOnTrFgw8+yPLly7v9er2BW1aUJOM4thU0ZUB5GTpN9ibDC5WA7Lhx3Jx+gxAKgWCA4FMsCgsL+fa3v80vfvELZs2axR133AFAY2Mjjz32WLc/6a9cuZKVK1carw8dOsTo0aP529/+xn/8x39w8803d+v1eoP+PEEvWDwD37pQ6PO9QXIbqiQQCPoXPsXi8ccf50c/+hFXX301//jHPwD44IMPqKysZPXq1T2+LNTS0sLPf/5zBg8ezJVXXtmj1+pJ+usEvWDxFvguaijht3ufQ9GcCRE7y/YwK/kyo5hPpNUKBP0Hn2JRXl7OwoULAdi9ezfXXnstsiwzYsQIGhsbA75AY2Mjixcv5rnnniM1NRWADRs28Oyzz+JwOFixYgXLli1rd9yll17KpZdeGuznCUn0jKiI2UtRakoYqFUqnoHv4/UnDKEAUFDYVrYLcM7HEGm1AkH/wadYyHJb26i8vDweeugh43VLS0tAJ9+/fz8PPfQQxcXFxrbKykqeeuop3n77bcLDw1m8eDEzZswgI6P7l2Ti4wd3+libLcbne/nFdRw8UcPksQlkjx7m9zwXSo9R/sHjaIoDZBmQQFVwHN/OiGUPMyg1q9vs6ku82XW5NJkPiz/GobZPtdbQUDSFA/UHKGstZWJiJpkJ6b1mWygg7AoOYVfwdKdtPsUiNjaW/Px8Ghsbqa6u5rLLLgNg7969JCUlBXTy9evXs3btWtasWWNs27FjBzNnzmTo0KEAzJ8/n40bN7Jq1aqufA6v1NY2oqrBFxfabDE+21cE26685cheNMWVEaVo4Eor1RQHNUf2EhGR3C129SW+7IrDxg+nfZfd5bl81drIkdpjKJoDDadnISHx2cmdKJrSY15Gf/vO+hphV3CEql0QvG2yLPl9wPYpFvfffz/33HMPjY2N/OQnPyEqKoqXXnqJ5557jt///vcBXXzdunXttlVVVWGztTWdS0xM5MCBAwGdLxTw1q7cn1h4ZkQBoKk+4xYDsYjPPDjpeP0JosOiabI3UXfhLNvLdhvFfLvLc8WSlEAQovgUi6lTp/LFF19w4cIFhgwZAsC0adN48803GT16dKcvqKqqW1Gfpmn9ptdU4ZkGahsuYJElVFULqF25Z58owKcYDNQiPh1vw5R2lu9B0RQ0YFf5l8wYMV0IhkAQgvgUi7Nnz7b7d3p6uvFaX0YKluHDh5Obm2u8rq6uJjExsVPn6k3My0+yLDHva8nMnjwioE60nn2i9Oyo81tfRQLCMudgScoYsK3NfZEem8asEZcZQW9Fc/Bh0SZuTr8BQDQiFAhCCJ9iMXPmTOOJ37OpoCRJHD16tFMXnD17Nk8//TR1dXVERkayadMmHn300U6dqzcxLz9pqkZ87KBOj1ZVKgtp3vCYc2kKsB/bSlTOgwO+tbkZfUlqZEwKYXIYDtWOBuTXF1KwtwgJqUdjGQKBIDh8isWtt95KXl4e11xzDbfffnu3ZSslJSWxevVqli9fjt1uZ9GiRUyZMqVbzt2T6NPyFEV1W34qPNMQ8FAkHacH4Wjb4PIiIqYtGLCtzc2YZ3lbZSuLxi1kX9UB8usLjZGt+uOJIhoRCgQhgU+xeOyxxzh//jybNm1i3bp1NDc3s3DhQnJycowYRqBs3rzZ7XVOTg45OTmds7iP8DYtL9jMKB2nB2FtEwzJgtZYi1JZOGBbm5vxnInRZG/i5vQbKMwrRlEdSJKMhISqKUYbdIFA0Lf47Q0VGRnJLbfcwi233EJFRQXvvvsuy5cvZ/To0fz2t7/tLRtDBs9pecFmRulYkjKIynmQ1oLtaM0NKKcPYM/fgr1g+4ALanvD20wMzwpwwGgTUtZYIeIXAkEf41cszNTV1VFXV0d9fT3x8fE9aVO/wdfSVCBYkjKITMqgJe99lFN5F01QG7y3BtG3m9Nsd1fsxa7agbaK70XjFopWIQJBH+BXLMrLy3nvvfd49913sVgsLFy4kPXr1wdclDfQ8bY0FSwXU1DbjGcarSf6UpWOXovx94J3xAQ+gaAP8CkW3/zmNzl58iQ333wzTzzxBBMmTOhNu/oNnktTwaLXYbQWbPfbM0qpLKS+4CRK7JgB7XmYC/f0gUpGxbckoWma2/xvIRYCQe/gUyz27NlDREQEb775Jm+99ZaxXS+i27t3b68YeLHgKNgOqt1r3EIv1mt2eR8DNa7hLUuqyd5kVHxHh0Xz1vH33GIdAoGgd/ApFp9++mlv2nFR01Ex3sVSrOctS2r+6Gvc9kkePNwtCP5R8Wbj3yIILhD0HD7FIiUlpTftuKjxjFtIgwbTkve+UWtxscQ1vGVJeaILwe7yXHaVf4miKciuVFtRxCcQ9BwBZ0MJeg5z/yhp0GBadvy1XX+oqAVriGg4ScsAjln4ypIC91jGW8ffM7KkAFHEJxD0AkIsQgS9GK8l7/12S07gXIoaOuESHEG0NO+PeMuSMscy9CC3OxIWSUbTVBHLEAh6CCEWIYa3JSm9E2153ntEfn1gBrf9YY5loGnOCm8NcOWPaS6/YnbyDDHnWyDoIYRYhBieLc3NwW1Ncfc0zP2jBtocDDOesQw9S8o8D0PTVIYNGtpu6eqfxQc4f8EuREQg6CJCLEIQz/5QuqchWdw9DT2mAQz4ORjeYhnOKu8vDRGJDovmo+LNRIdFc/pcKTvLclFwjnTdVZ7LDy/5jhAMgaCTCLHoYTrTldaM2dNImHAJNUf2eo1pDPTUWm+xDLOIeAt8m3FoIvAtEHQFIRbdhDdR6GxXWh3z0lLEtAUMssVgPdvsNY3WV2qtv+WpgbB0pYvIR8Wb3dqDeGKV2gLfemaVqMkQCAJHiEU34EsUOtuVFryPWMU2DQBr5hy3CXuA1zkY/sa0DrQRrnpcw9weRJZkLkmeRIQWyciYFI7Xn6CssYK3jr9nVImLmgyBIDCEWHQDvkShK11pvVVtXxgaZdzgkcMIy5zTzvvwd47Wgu3IXgLnA2HpynNJSu9MOyNjErsLD/G7vBewq3YkJCN7yqHa+bjkM0YPGSWqwAWCDhBi0Q3oouBwqCBJREeGAcF1pfVcxvJWtX2+5HC7m7/eU8qbd+B2DknGcWwraAqtchgRs5fCAKsK99XJ9nj9CSOWodFWo6EBB2qOcLDmqKgCFwg6QIhFN5CREsuS68bxl00FqKrGG58cJ9U22OhI29HSk/dlrAzqZtxLw4mDxI6dTExSBpFDo6g312CAT89Bz6jSl6e0xlrs+VuMfbULjRfFCFdwLlHJkoyqqV7fF6NcBYKOEWLRTTSdt6NqzufWQOITZk/C2zIWwG82nsWhpGA9fpYH4hqYNTXL7Qav1JUCMqC28xx0L0P/T6ksxF6w3c2T8DbCdSAEvT1Jj03jrszbjFkYbV6Ew+RnOKvAVU0BSeK844JoUigQmBBi0U2Y4xOSLFHbcIHCMw1eBcPTk1hy3bh2sQ1vAjJraqrbzb9lx19BU0CyYBk1BaXE98Q9z2I/b0Iw0ILeZuamzGjXsfZ4/QnOOy7w6ekvnHMyNA0Jpwfy8aktQFugXNVUsTwluKgRYtFN6PGJHQfL2XawnM/3l7H9UIXXdFlPIWg6b/ca2/AXHDcC1ABoSJGxbTEISUZrrEWpLDSExVcQ3Os5B0jQ2xPPmIaecqsPVHKPaDjR0FA0Z2GfvjwFwtMQXHwIsehG9HRZRdX8pst6y5LyjG10FBz3DICHZ84hPHOOM+h9bCv2/C3YC7YTMXup1y62ZgEBjI63wQS99XNcmHAJ9NMGh+ZWIpLX5SkzEuVNlbx/cpOxnDVrxGWilYjgokCIRTcTSLpsoFlS/oLjvpaV5LJ859KUpoFix57/hdeKb2O5SbI4T6gp4MqS0i40tluq8oxlmJes+nODQ89WItB+eUpVNZAkFFT2VOYZxyqawrayXews32OIhn688DoEAw0hFt1MdwhBoHgLUFuTs50CoDkADbWmBCQLoKJJFnbXxpLx1T5idAHRFNCfo11ZUp5LVZ6xjIjZS3EU5YJiB9oaHJpt6Wqbk97E2/IUwBTbRDbsz+XImTNYEkuRfAxJ10Vje9luZEk2BjJdO3IekdZBQjgEAwIhFj1AdwhBZ7EkZRCWdQX2o5+5tmhYs+ZRr0bzyl6VE1/aGRuusirGioQCkuzaTW0X69Bxi2Uodlq2veYSGQAJyeK+ZNXVNiehQnpsGl8fO5QjhzdDQhmarBqC0Rb4bku5Ncc3zEFyIRyCgUBIi4WiKNxzzz2sWbOGyZMn97U5/YawzDluabLhmXPIK7JyorUITYOi1gT2jV7OjPiGtt5SHrEOcyaUW3wEySksAEhYUiaSdN1SzkUkG0tVZbWxXWpzEmjqbm+k+WakxPLAwmvYdXIE8pA6RsXHG9Xh4BzvuqNsDyreazjAu3BQrIjW6YJ+RUiLxXPPPUdiYmJfmxGy+Frq8RbPyHI0uMVSksdPIcJ0jFusw0uBX/uxr04hiph+K4NSszh7KM9YqpoqWdkWfj1FrQlBtTkJJnW3N9N8nZ7iTK/vpcemMTIm1ajhkFwDmXyJh1k4AHaW7WFiQjZDwmMYGZPC6XOlgCRERBByhIxYvPjii2zbts14vWTJEsaNG4eq+n5iu5jxVqvRdN5uCIc5nqGLiuc+Zry2BlEdtEoWIuZ+k/DxVxmBbW+NDM1LVRIK/zG+gaqWFmLHTibVWk1L3tYOO9/aC7aD0up8w5S6682DCKU0345qOHxVjgMoKByoOdxuu6eINNmb3HpeCSER9DYhIxYrV65k5cqVxuv777+fwYMHc+jQIU6dOsVvfvObPrQueHo6wGuu1XA4VGerEU1rFyPwFJWf3TiU4VWHUKzuN+52rUH0mIemcGHrq5w8dIjBqekMOfJPt0aGOp5iE126mzGaglb5GY1oSECrxU/nW8nivPHrSLKzSt2HB+Gtd1Zf4i9IHqhwmPElItC2lHXecR6QDDEJRRHpT4kOAv+EjFh48uSTTwLw9NNPc9VVV/WtMT7w9YfQGwFec4oukoSqem81YhaVVCqJ3fk6rShel27cWoPkf4GmKc5FFU1jeF0uat2XaBJItH+a99mHSnMKhSSBptjdjnHzDsxZWUhYRk4xzuXNgwikIj0UMIuILhzRYdHU2KuoqD7FkYZiFD3Lyle6lQeeS1k63kSkL5e1Bkqig8BJj4tFY2Mjixcv5rnnniM1NRWADRs28Oyzz+JwOFixYgXLli3zefx9993X0yZ2Cn9/CL5alnfnU5Y5RTc6Mow3PjnutbbDLCqZ4ZXI+k3Zz9KNJSmDrybdTvSBN5FcXoEs4ap0lpEkyevTvH4Tbz26BZDRXCNNJcmlBxJojbW0Ht2CdqHRvQjQIytLOX0ApWQvIIMsO232uKa31OFACTY43h3BdLNw2GwxlG56g+Nl+9gbEwFIjBoxhfKhw/iqtZHDNfnGSNhA8SUi4Ds20pNeSVfmuQhCjx4Vi/379/PQQw9RXFxsbKusrOSpp57i7bffJjw8nMWLFzNjxgwyMrr/yTA+fnCnj7XZYvy+v+VAOYrpD6G0tplZU51iOHNKCht2FONwqFitMjOnpFDbZOeJv+UZ29Z9dw7Zo4d1yq784joOnqhh8tgEZg6N4uCJGr5962TONbcyeWyC23ltthh+5dpnypBk5M2H0BTnPO+ECZcwyMfn3BI9ja3nzjE9/AQzIgqRNQ0FGfnypcRHqUSmTWRQala77+tC6THKd/7VlTEloSAhaSoaYJFl7Ee3gD6eyBpGwg3/gXL+HJFpEwE4X3IYx1fVnNv7sevMKmgyMdOuI2byVW7X7IgLpceo336YGA9bL5Qeo/yDx9EUB3aLlRHLHvZ73s7sf77kcLvvyHOfCMdXpNkhraYZyWJlxM1fN/YvqCni85O7QIIxcSM519JEs72Z9499ihLgUpYZf8taABZJZkHWtURVRxETEc3JutPGtfV/Xzl6JpkJ6QFf09vfQUd/V/7oyrE9SajaBd1rW4+Kxfr161m7di1r1qwxtu3YsYOZM2cydOhQAObPn8/GjRtZtWpVt1+/trbRWX0bJDZbDNXV5/zukxofhcUig+tpPjU+yjgmPjqMnyxuK8yLjw7jg53F2B1tMYZdB84QHx0WtF0795UaHo0sO3NvFLUtVhEfHdbO9vjoMK6aMgIYgfL1tqWbcxHJnPPxOVPjoygliZLzNr50jOW6lGZGTJxG2qSpOIBzYBxr/r5ajuxFcxXrSZJMy6hZVLUMIiniPFGnd4K5KsFh52zxMSKvWIFhRaYzBZe8T9tqOTSNFssQ8GOvJ22xDgf1stU5aRCMpS1NcS5taYqDmiN7ifDTrsT4TAHsb46x1PvI0lIqCzn/wePOc0oWwrKvJCxzjtvPIw4bt47OaXf+jOhMYylL9w4irYOCiod4tVtTeTf/Y7/7bD6xw6d34stT8fw76OjvyheB/E32BaFqFwRvmyxLfh+we1Qs1q1b125bVVUVNpvNeJ2YmMiBAwd60oweoaNKbc/CPF8DkjzpaKnK7NqrSlvju0Dd/ECXbtw/36U+z6tUFlJfcBIldozXwLNt2tUM19uDnGmr+nai4Ti2FcWUVaXbGDH3m67iPxUsYVSEpXJoZ7HfJTx9qUgaNNitwtxzUBSSxahq91WIqFQW0lqw3bkEl5Dms0mj57U9YyyV+fvIK7K62e0oy0dTHK6Avoo0OD7gpS33QPoMY/sw0sgry2d0pErDmW2gqSS3qlRlTuecRXZf1nJLJAgsTgIdeydm3OIncRIVpLAn311QzILXX1KFL+aAfa8HuFVVda55u9A0ze11fyKYSm1/A5J0AgkIerZClwBV1YIe29oRgfxR6E/RzS5h0J+ivQWezduV6iKU4r3Ok2hqu5oOwJmqOywVR1k+FWGp/GrjWRxKXbvvxSwQLTv+2pZ6q+OKr7gNikLFmn0lAPZjW2k9uoXWY9uIzvlpW9+rDY+5ihBx1pPMuRulpqRdk0Y99qI3azQLkSZZnFXzrUVudluTs7FbrE7BkK0BCWFHP6u//LMahxLH8MhD3DroK2cSgiQTjo2IKQsoaihhd3kuSs0phpceoyzCCkhEJ45jS2tZl7wSb/iLn3jDX0zl9LlSIkvCSbAmhlzAPpSXoLqbXheL4cOHk5uba7yurq4e8IV3+o23tuGC1wFJ5vc7Cgh6ejRAtz/pBJrF0q4NSO4/iZh+q0/vxZxt1Xz6kNdxr+ZlG33/QzuLcSh17b4X85KPubLcGREBDYnI0VOQpziXc4yqdklGAhqaWohQHFgkUBSnF5AEtOT+s00owOiZJQ+ONwoXVaWVC9v+7LwpI7uu3SZE8uB4dtfGcuJLezu7LUkZjFj2MDVH9voVwkAxe5sFrUnMj7RgQXWK5KDBtOS9T1pyNunZtzu/s2NH4VyzU+Bn38y0QWHG0hbhDmi1trtZdzbo7g1Na0t60J8Tg/FaoGNx8eXBmJfJ0i7Yg0pa8MwsbM59jwsJ1/bbjsvB0utiMXv2bJ5++mnq6uqIjIxk06ZNPProo71tRq9hvvHKsoRFltw8gY7eN59ny4FyUuOjvLYz704CzWIxlpxcyz3KmcM0VxT4rKZu81ZsjHF5GWcry4g8tdNrOq6Or06+bmLlWtrSXP9TAQcyjnE3McxUTKg1N6CcPoA9fwsRmoyKDK7gfdk5iHn/8fbeiSkLqwUZjJRiFZfL4srm0oz2KpakDJLPNGDdl2fYPSnmLC1572NNzmbQpGlERCRzaGcxqRxjbEQFJxzDO5UxlDUqjrHhNYyRyzmpjqBh1iqG20vdPB6zEHt6funglqVVcSiPy85Xm26izuUu3TvxdlMONn5iXgnrDMGKiycyEnPrm2mRNSj5mNFZ13FGasXfZ4sYEkfY6MMM5jwLLpxgdGUr5X/Z2m87LgdLr4tFUlISq1evZvny5djtdhYtWsSUKVN624xew3zj1VSNK76WTHzsIMMT+GBnsd/3oU1w9JtOR0+fga6r+tqvozbrnjd99cAGLpw8gL+UXG/eCok21m/5iO9EyVhQkWWL1+K6jJRYfnbjUGMeeZo+HMpDrHD9/6B9JKccCZxwDOfGM2eJPP2q4b2YvQAZjd2tGdSp0ZxUR/Ata51JKCTkhDFYbGlulerlQ6cxvC7XlUrs8mIs3lu7m73ASTFnGbb7D7S6btwXhj4MEclMijnLzJhNWFBQkbE3NKBUXhNUX6wxVlgV87ErTnSIaNslWJIW0JL3vs8aFXCKbWl1I4fODTV+By6UHvPZRsWz8FAXEaWykPTK0xTaE0keP4Upl0xsF4TXb7oNTXbyDrSCpQWUCKZNCUMKa+02ryVQVDS+GDrIeL27fGtAx1lscF6Dl7UhZDe3EKNopB7dgP3817x+3o7+HUjacqiMOu4Vsdi8ebPb65ycHHJy2md6DEQ8b7yzJ4/we2P2fB/cBYcOAtmBLiH52q+j1iDejps27y7KTh31W03ta874idYEfq/cwLiwClImXsI8H+Neh+3+A8NUO9RspdXSdmOOWrDGaIKIpiJJFr6wT6aoNYH08Bqy8l/CodhNZ2vzAiTZSvbVN1NScY6rHflEnd7dtptsYdCcpe3+OAeNn4tjex4WTUVFwj5qJgnT2m7uSmWh4T1YkjIML7Al731aTTfu8yWHITOZ4fZSWiTVGUhHxXpqB02le4z4iTc8q9rDMucguVrSoymGKPiqcteP1xQ7gzWZvedu4L3tSTywZBrZTYeDaqOiVBbStOF/iVLsTMTC8wfnc+ed85k/+hrXHjNMezv/XTi8/UNKR15LZGRbzKI7l8SCCfC3HeP0XI9ERwCwWymDovJOm+CvmDJFC6P42CfOhIWCf2GfeBUxcaO6JD6dJWQruAcKgWRNdTT/IpCBSjqBLiH5unl3JDTeZ4NP7rCa2tdnsFpkShw2TqmJ3O0jh989NtJKy7ZXQWtrHxJ5xQqUzDnG9e902Dh2qp5prXVIBR43FA8vIAoYtvsPHktPEtasK9wEQD932qSplHBfm5czaapxlL/mhu16b31V7VwaS86m1RJmpOZKpvhJs+tzeP5eePbF0sBntpa5AaQ++Eo/XkLDgspYawUlLTbKjh4gI7a6LVPMFPPw9XN1nssZ90FTGSOXd7iU5i0xxJfXov+7LQ008CWxkTEpnKs/RdjhLZSFSyBZGDzmcj6rO+hcJuuMUHjLJOvimlqHyQAx4W3/rtrj/M8PsiRzV+Zt3Ga7rkt2eSLEohfoKGsqkPcfWDKN0tpmI2bhjcIzDdQ2XPAZ9zATHRmGLEmoaEiyRG3DBXYcLO9QaHzd9P2l5PrzVjwzxIB2+7RbbtL/OE3tQ8zXz3B9Z0qlg/MnNjmzjiSZsKwr3JaUgLalGjOWMMJdfa+8CUDapKlgEgkdf80N9Ru37gWdy/sE9m8hasEaohasofCLjdjq9hrFj3vPxrHBh3AHOlLXLFaew6uQw9AUBwoSJxzDSQ+vYWrxx5zTnDNOwrKvRE5I8xrz0L+XtlG8VhTFgYLMSXUEd3ZjVp4v1MahDKm/1PR70t6DAWipqqb1q2ZXNF0mPC2WqZd8z+cymb9/jzhby6ny/ZyzyByLiqBtkgltWRUmOpmh3J4gRUnVVP5e8A4TU9OJw9bxAQEixKIPCSZnOyMllllTU30W2XgGyud9Ldnrkpa+7xufHEdRnU+yaPD5/jK/AXazHYFMAvRml7flLnOGmGdDxDZxccZGWnL/iXLmUNvJXc0GfWHOOvL1ZGy+8WpIlA+dxqDxc4nx0r9KVexU5u8j2YcodtTc0JKU0a4VvKMsn4hpC7DM/ibPrU9kmrUQ2XVv8CXcAY3UNYlVO0/kQqNxfH1YKpecG8q01lykAodxQ5IGx6NdaPQqfm5ZaHIYg+Yso6aqhkJ7EneOn9JhHUxX1969/U6NsVZ7Pbe3n4naOBRHeTrDR8UxN7st7nJpdb6zm3JcvLHdSds+zYf3gOqgJNLO/owJ2FtbKTg5iBopHMkVg4mNjuBsVQT7yk44n22ah5CeAWlJQzqVDGAQhOJomsbhqgLm2oRY9Hu6u8maZyA9PnaQz/Pp+wKggeK6QfgKsHsSTH1JR8tdZoEyN0T01kl3zPRbaa4ocHoYkkzE3G92eNMZlJrltzpbv/FW5u9z1kTUJWAtOcsDcQ1GTUSLZEVVnU/Or+xVuTOjwe3zm2+C/pbjCs80UFYby1TJNaXQJCgZKbEsuz6T2J2bnP27qk9ywGMmiOfN1rMF/aSYVIaZboz6EpJbDy7XNfXjUysLGW4/hDQ0gZZC92MV00hes62eAlpTVUPylYtIRo/ZtG9H3x3zR3wN1yo7eoCkkte8ntvs0UlAaXUjv9lY2k5ozHU19mNbicp5sJ19ZpEen5zNpKQMthwoZ2vFUTTN2T9tZGs6X790NIUxDez7d7ThgS/KMP99twmRubmk2Zs5VXUEtekso+Iz3LK0vKUCe4qPpmlYZSsTEzOhaytkbgix6COCabKm3wxmTknx2SLE1/KQN+/FX2GfL2+ks3iza8fBcuwOV02ESaDMDRG9dtKd1TOdZi1JGW6TBD1rIvaNXs6Zw3s5bh/OKSXB7Wfl7SaozzA339xLqxv5cvMXFLQmsc1yPd+bHcGQ0RPdPsNweymtOBs9aqrC1xPLOG9tYsTEaaRaq73ebAvPNLB+/UeMkcv5izqCZdff2y5tFtfSk1JT4rZS4ukhRMxeSpTFTrMS5lZkqLcjMcdfvAnohmbd0gAAIABJREFUGB82QmDzR/x5HuYsramSlcJB0xkh1yJLkOVIcDu3tyJPvXo/lq2kcj0nNZvxcx45KL9dXY2vWSqey62TxyZ4/bsL1AP3VpGvVBbytb07XT+Xo8zxKqxtgqNUxrtszGTS6FiOle8na8TXyExI79ZWJEIs+ohAg9ZmD2TDjmJ+sti7B+Ltl9OX99JThX3ehMnbtbYebMsckWXJTaBSbYP9dtINttNsfnEduw6c6fBz+vt5JI+fwl/2KV7f83UT9JzVMVTTmB+hcF2EhT+cu4H9kZdzVdIINxv0JRNNcaBoGmmN+51df3ftwJ41160AsrVgO5FJGZQdPcB3oj7C4qw84HDFCNKua582q9SUGDdMPZ7hzUP42qL/R9OmN9yq3TX9c5q+f28COnJQPpriDJxrirsgeAb49SD8SVcQX08t9uV5nC9py9KSNDuLInehPzZLp0+4tW/R06RbJCv7Ri8nI6ySGNexMs7uyyWKzfhZWq3ZtMpWt4p9b7NU6mbc65ZmDJA9ephPUQjGAzcTjLC61dJIFpKAJE2Bo/u4kDymWwsGhVj0EYE+eXgOOfLngXj+cvrzXrqjsM/s8Rw8XmUEqmWLxBWTRxgiYL7WBzuLjeaOEjDXw5PR9y0808CcScMBOu3tFJ5p4Im/5WF3eG+66Hldf3/0vt7zFadwn9XhMNq8ozlvVjFR4Xyw073Fh77MUfjFRpLqvkTGGVPSVDtKdQltbd817Me2Ep45h4ywSiwoRjZSRlil1+Uut3YnrhuQNw/hO9PrGOptaqKm0CpZjCQBbwJaUZ/KYM1ZM6MgUR+WSqrppuZZFNl6bBvrz13PidYEGiMPcdOgNqHx7KkVmTaReskCmsP1u9O2vqJpCmHZVyMPjnebpaKqDs4c3ss2dQSrYpzfhSRbmXtZBuOrzphqdmKJynnQWKrSvSiz4GqKg12bv2DT+Untfn/031lfS3D+8OZNdRT78tm5wDV+wPmzbkvP7i6EWPQhgTx5mJ94rdbg+j8Fk3IbLGav5d1tJ1FVDb3Br6JobNlXxvZDFe1uyt7qSsznNHsVukdk3icYjp2qx+EIvOmiv5+Hr/d8BZutydmu6X/6zc31ZyxbSJ92GX989yB2R/t41UmHjXo1miRdKFxZNmrNSaM63Zleqzjbk2RPpen4R6iqA9lixR6fYfxctoVfzz2XyCRlOzO39HYnmmTh41OR1FU6GGZbzLniQ4aHcPBEDVdNyaBuxr00nDjY1i3YJXr2o59hz/+CUXO/2U5APzg1lL3nbmCs1VmNfmXFOY+0ZMlZ4+IqitQ0B2Pkcgq1BApak7guok1o/vSlQpG9rafWrKlZhGVdYUxw1Iz/gYLE+aYWEjKzjc+pKnYUZONz7Ru9nBnxDUiDBjNkx18Z4qrZUWxrDG8p0uMGb75pq8gUtCb5XDYONCZjFgfA6zG+Up69LumZilEVzVlqKqMhyxYi0ybSnf1whViEOOanWn8xi46O7e4ume5ei+Y1jubtj8qXTWbxkSQJzRSv2HGwvFOfwbnE4Oz025NNF70tjVmSMtxubiBhTZlIxPRbqSqy4nC0j4/o30EqUYyLsTifhCXJ9RStuf4vOZMSkCm0J5GclEF0zk+NG9DWIqvRR6uoNYG88HS+njQawAjkv/ylQlGNBJRhkSUkaYrxvUwem+C0Y+NZHEoKY8NrnE/letEfOJd4tr3GmIWpZMxq+9xZo+J4b3sSOCAzvJIsR4NHWrKGPtzK+UkkhlmaGGOtpkRN5A8uoWnWIhgjV6BY4JRic9XypBKWOccQPFWTONyajAZMCD9D5KmdNJ/JNVKR9YSFU4ozQSB5/BQiXMWRgRQd6jd1vSanIiyV0o1nkSXvD16BLh15FlP6S7WG9inPbkPDFDuay5PQgMP2VM6pkUhA1IQrGJuaFXBL/0AQYhGCeK796/91pnd+Z9dNO8LsIciuXE9F0UACWZLQNP/pt36r1DUNWXaeQ5Ilth0s97p81FHqcUZKLOu+O6ddzCI6MszIyuqJ70bHfHNDthpNFrMcDYaImb8j/Ts4qdn4w7kbuGVcK9mZI11r0g4kycI/mqYTyQVOqiNYNjzGKJjTg+pZjgaf3qQeyC9qLTK2KSp8Y7zKKMqIHTuZ7NHDeOW9M8bPoqjV+VR+WUQRjvzPTUseKod37SRqus1YghlZlc9/z5GJOvQxsqYgnZadsQRjZK5kFEXqHXxnhRcwM7yQU2k5PHPQKTT3utqfKFh4vnk+WaMuNezXn7grwlL588azXBW2n0mcNpZe9FTk5KQM7sxo//sRyOx2z8B/1II1pCVl8EBc2/nAuaSqP8AFsnTUkvtPt7b5bsWUXo5p16hz22uAiiZZKRg+n2H2KoZW5SKjoSIxIewMMioKFhqH39yZX1m/CLEIMfrL3GLdQ9hxsJzIyHBssRFGMR0EHzD3XJ7SayxqGy7w+f4yn0/hHX1P2aOHtfPGeuv79bVE5Sli3rLUSkkiavo0wlNijVbt1uRs5rkCwss8ek3pSxgdeZNZo+KwWCSnsANjw6uZV/2J03Oo2cqFcYntfhbJ46cQmXIFrQlptGx7DU1TsWsy/y5qIqbkVSIvy2DIkX+CaicGCU1VkSVQVAVlaBrRyenICWluvbOcT/jOWI6EyuhT7/OzG++j4cRpwipVY/s9l8gkmz6D7sWlAQ/ENVB21IJcfMgpSB43XF8PStbMOW6xCU98eQnmeFr7pBP3FF2lrtRtHoq5fb6GhIpMPpmcT0snI6ySpOyp7WxxEyAjNuGMxRw/cYZPW6aQJscz1lrBMLmJWYOOI+OMbA23lwbxmxoYQixCjL6cW9yZwS7bD1V4bXAYrM3+lqfM1/B8Cg/2e+rt79dX9pY3EfP1HXirTvfsNWVewugo9vLTpZeww5WRdl1kXVsxnisompF5vVc79Bkjh3ft5N9FTdwWtQcLCtLBfWAslTkXRhRNc964Gk5hbywnyuPG3BbTcbZj0TQFDmwgfsylSDXOJRZZkhk2SOX81lepjgxDGXm52zmcn/MKlMoRAaVTe3oMYa4qfU/89dTyrPPwTDpxFGz3MlNFNoodNSQK7CP48PzXKN5+HgmwWhN4IMOGbrm3uh0968kci9FUjVMkUtJiIz28htnySTRVQZVkKsJSSfX5TXQOIRYhRk8Gpf3R0ZO6NyEJpsFhIHi7yfm6gXb2ewq8o27vTELzteTYkU2BLKf4wnwNpdJKc+Em4zx6UNRvQH+6jZiSV40sLM01aAk0Y6lskrWYLGs5so9OxJaktmmImqaABkPPncBxoJimsVcSffJzZwbW/g8BsAPs28wXtiXURaS4ZciZxdRXrUbhmQaac3cy2pXa6y9e4c0jNAvNVMnKNlfBpDnpxPBIPNFUV2qvhorMvtZRZFgrACh2tNV7jLFWm5piKu3qdizDUqnM38fGfbWMC6vAapGYffUcl0d/KfX1mexy1fKUbjzLr8bUBT262R+Whx9++OFuO1uIcf58a6d6fEVHR9Dc3Nrxjj3AsCGDGJ8Wh21oJAvnjnH7g+0OuwrPNLDzcAWyLDFsSFuL5p2HKzhcXGfEDGxDI8kcOdQ45jdv5HG4uI5dhysZnxbHsCGDkGWJXYcrwRWfWDh3jNs5u4thQwaROXKo27n9fU/mz7rrcAUOh2oc6+848+fccaiCs+daiI4M8/mZ9O+yrLaJwyfr2n2n/oiOjuDA8Wqv36vnNbx+94OHYU0ZjxRjI+LSWzpM1fT1c/c8z9CMSR3+jg0bMoiY6AgGncl1xgpkKwUjbiQieRxxcxYRmz2Dem0II746ZLwfcektyIOHuZ3HYhuNNXUiVWfOMKi13mhz0uSQibTXOV+4sr+cmcEqh6otbC6NYufhCsaPHub2WfQbunLmEPbju7CmjEcePMz4DsvrLnBpeBGyJCFZnDZpTXXYj+9AkmU3++TBw7COyDS22Y/vcLaa0ZwB5QmTMhmaPpEVX59IanwU/397Zx4XxZXt8V8vtIAYtcNmAEEEMRFXfCouOK5RkcFtMoijEkwm45Jx/MzEp4kzJkQdo04y0TjRmeSZT6IzYlRMzMonJjpPMB8V95cR3FqQRgTh0wqy9HLfH0033U1Vb3RVN3i+f1HV1dWnTl3uufeec88BAIlUCu21H81+ndabyY3ZBkL7QqN8BgNrvkOCnxrDu9zCdV0v1Em7Ye4AQP7DX2G4d8Pi+wySbiGQ9+pnlikwQIH4W5+in1yNEf4qxCWNREL/WCif8EfBjUZ8ccMPtYauAGMIf7IrYsKcr+QnkUgQGKjg/ZxmFj6IUE5pe7MHeyNuvqUb06jfUYJDobDVk+UIHABvDRC+71lXKuQP/zV9Z+u/zpt3ohuXE1zzgTizJGa7z+azkzeRPjYWcRHdjRvaGuVI0PUEn6m4Xq5B4eUK3iABoO1SmTOzq+jEIdCHrGlNk3Jfabx3XIhLy0OysDhgUBp0BTsgYwboIYW8TxLwUxkM+majkTCHyEpxXWfce6PTszb64vM1mDIG3EJr4MCAUckAuENXbfXHlUolrP8QpIbFWAWd2Ia8mnbMW/prgtVXrVLSp8c3I3D4UITdPAKtnaJbls/IlY4eaPs/PLBvMKfO3YWMRSfF0bIR1yY9PseoPUMSF2E/waEYzwUAxy+UmzcFyuVSjEkMd2qJjKtSod6JPRlW+bWANqVyncGZpTTTNTqdcSf1T7dqUVJ2HvMnx1vtRXHGoAFGg1N4uYJXxquqGqcDAOylSTF9bm/G0/ou+6BujHXad31sP/OSSy/pfUgBnGnuC5XOmBhPLpNY6YtzI6J/ENQnDqL0Jz3Qkn21jIUicPgwyMK6Q33iIILsLEtZto3PZVK8Os2YSsWe8TMb8NCeiHv6Z23CZbuMzoRE5tcS3SZFP6UB0gfn0VRsUXxJIoNf/xROB7y95Ufb/+H+MUpK90HYh28G4ahz4pvRCLlfwxXsZbDd22IoAJg7cWd8GlyVDAHg5OUKu3sybDtxiQQu+5ic0avpms9O3sRPt2rNRqmo+J7TsxJLWMuz8e2Kv3yj2qUAAHd9R1zvclB6a9p3WVgcngqLQ0qcxhxxN7Z7F/S+a+z8LOW3HCic7GLciBgcGoymwn8iSK/F0q5S7DRMxW1diDljwPVyDQ6cM9it0mjSX7SsCvF+d3H77jBET57Z5llMfpK7fpEt+1NanynqHnfGX8t08q3VGwFAAnn/FPiPW8ypN74IOxNCrUoAZCw6JY6Wjdzp9IVshM7C91zFpbVmQwEY93mMbkk34miJjK9S4eiBvZzqxE37NriqCjqDM3qNi+iO9LGxKClrXVZLSghFSZnGqVmJXm80aKblHIOh7RIOYOzAq2ofWaWp7xrg1yYtCZ8eXHl+V6LS+CLuTDJbDhRutGxEnNx4BZYFnuL97qIcYeZsAMWlta1VGuV30a1PImbadLwJvY21zV8KNO77kKquQF/Zy6qDtkxw2B2yNkkK+8S25vsywBilFG2bqp6jhrvl89mLjhMTMhadEEfLRt7u9N2F77kSeveEn8VO7QVT+5mf0dESGV9n52wnbu8aT0ZWcclpSrrorEHjSsxoKavJxyNpqYfSO7wb/vXdNXNurV9N7YefDYlwWQ9cODsjcRRxxzVQaE0OaFyukUpliBgwDK9Y1NpI6N0TUqkEKl0IVLoQyEsk6F9uTD1vmin0eao/soZJ4Vds9DHY+ggA6wSHlkkKTQXFbul6wW/kMqsopVd6atDHImmkQSJFfeIcBPsbrGYLrpRIFmPWT8aiE+Iry0aexl7H3p7nFcKACrG50lJOZzsIy+/YMy6WnbK0pR5KfYPW7O8wGBj25ZcgMiTIrefgChHme2eW1zoyKvwDhe5WyzUpFh2w6d7jBvbC8Qtq8/OZwlctfQzBozOt6nzYLlUFRA9AbYtRkkjlGDUxBU13/HHycgVOXDQGSIxJDMeJhsSWmhet6fZrLI1IQaCxjYRZG0JHsy8xN/GSseikdOQZhD3s+VWE6PCdNUC21wq5+c/dDsKejvg6ZampMBUAA+NevnJXXi55uK61F3Fnz+hwRXlZ3nv+5Hj4yVufuWuAH/7vx/+12othWVWQy0fgH5lg9Xm3sDg8+VAFvYGZ3z3A7T+78rAH8m2NiJOBJSbE3GRKxqKDIfamsccVVzpkrmuF3FwpRAdh6nRtO+VfTe1nVbHQnedwRV6ua1OTY+wuJzo7ULC9d32Dts0yXSQUWNZNCj8Jg8SmqiAftp9z+cG4fGDOBJzY1oKx9R+JuYmXjEUHwpWwRqJ98HVwzoYkpybHCLYUKFQHwRUG/bMhEQ59I56UV8jOj+veJkPz5SmVVRLHKVGP0PxkHJ7ShfDuYeHD3nKpM9cB1oPC1OQYpwuZkc+CAOB6WGNnRYzZFVfH4mpIsjs+BmcQ2yfl7hKf5TM7K6+Qz2bv3pbvsJSF4n9KAb1KC/mF824NypzVGdd1tnuG5k+ON4ZKt4RpOypkJhRkLDoQfPV+HyfEcuhxdSym0aerIclcMoeEOJ+GgU8+Xx4ocD1zanKMU99t77PZM8z2fF6md8iX6bg9v+uK7JahwFqdAfvyS6A3VZds2c/jKKRZCHzWWNy8eRN/+MMfEBsbi8TERGRlZXlbJK9jr97v44KYDj3bjsXdkGQumZOHeDonaFu86d8S8z1Z0p7BhOkdHr9QDuklCQxwvlCWpwYxtqHAEgmsyhA/E9MTSQmhDnfvC4HPGouioiKEh4fD398fQ4cO9bY4PoOvjyiFRkyHni3uLpF4Q2Zv10Xx1ntqr5G6Xq4x7kcxGAtwzZ8c75bz3N2osfuaRshb6o1IpBI8+19R+K7ojlmP6WNjvWaIfcZYfPDBBzh58qT5+E9/+hMmTZqEoKAgLF26FB9++KEXpSN8BWciRIT+fXfWr21l/vRYiaDJF73VoZjgMqze8jW5glWKFMZQ38CRcrwdv8unA9scZeOHPGXOJjC0X0ib73jDEPuMsXjhhRfwwgsvmI+PHDmC5ORkKBQKyOU+IybhA5g6bG+Pnl3BVma+9BWewpszMBO2Dn5v+ZqcwdSJdw3wc0tvzkQ2mcJzuXRgm6Psye7+vA5ssQMcTPhsLxwbG4vNmzcjKCgIzz33nLfFIXwQb4+e3cFR+or24k4Ekhh409fkCK7Neu7k+uL6Xct7SyQSMAPjjGhy1bh7YzlacGNRV1eHjIwM7Nq1C5GRRqfe0aNH8f7770On02Hx4sVYsGBBm+8NGjQI77zzjtDiER0YXxg9u4qQMrcnAklofPldcW3W85TerAYHzOgHYayt49xbswVXENRYXLx4EevWrYNKpTKfq6ysxDvvvIPDhw9DoVAgIyMDI0eORFyc57MoPvlkkNvfbW9oo1CQXNa/ualHIC7fqMbAvsHoH6Pkvc5XcFZmdzh+qcKYYbal07tz/5FbUVd8+rqqqnFbbk88tzPv0R0ZRw2KwNFCFXQ6A+RyKUYNinCpzdi71vbeL6YPxMNHzZzyhYR083iUnCfbvoQxdwqPOsdrr72G2bNnY/Xq1fj4448RGRmJvLw8nDlzBps2bQIA7Ny5E4wxrFixwuO/f/9+nVUYmrNYVr/yJUgu1xFbNlNlOgC8NSOEkssT/hA+uSwLKdnLQCsUzuirPX4Rd53vzsrl6N5COP9dbWNSqcTuAFvQmcXGjRvbnLt37x5CQkLMx6Ghobh06ZKQYhAehHJT8XO9XIO3/nkOer1xgHLycgVWZw4TTU9CLmUUl9Z6LAOtULTHLyKkD8DRvTtKoIboDm6DwejoMcEYszomfJeO0qi9RXFprdlQANx1ooVGqE7PVP+hvRlohcSX/SL26CiBGlKxfzA8PBxVVVXm46qqKoSGhootBuEGXI26o3C9XIMvT6lwvVzDeewJEnr3hEzWOvCxrRPdkYmL6I5fTe0HmVQCiQRuZ6AVEtPManZKrEcHMnxt5Xq5Bp8eK2l3GzIZOakbpXnFRPSZxejRo7Fjxw7U1NQgICAA+fn5ePPNN8UWg3CDjjpy4wqNFCJdQlxEd/x35jCnfBYdEU9koBUaT8+s7NV999R+mY4QCQV4wViEhYVh1apVWLRoEbRaLebNm4dBgwaJLQbhBh2lUdtiOyMqKr4n2LS/s6dj6ezPZwvfEpGn98t0BL2KYiy+//57q+O0tDSkpaWJ8dOEh+kIjdoW2xlRUkIoSso0HW6GRIgP32y6o86y24PP7uAmCE/BNSPy9eUUwjfgm02bzvOVe+2MkLEgHgu48us8Dv/gRPvhaytxEW0rC3ZmRI+GIgiCIDoeZCwIgiAIh5CxIAiCIBxCxoIgCIJwCBkLgiAIwiGdOhpKKnU/51R7viskJJfr+KpsJJdrkFyu44psjq4VNEU5QRAE0TmgZSiCIAjCIWQsCIIgCIeQsSAIgiAcQsaCIAiCcAgZC4IgCMIhZCwIgiAIh5CxIAiCIBxCxoIgCIJwCBkLgiAIwiGdOt2Hqxw9ehTvv/8+dDodFi9ejAULFnhNlvfeew9ff/01AGD8+PFYvXo11q5di6KiIgQEBAAAVqxYgSlTpogu28KFC1FTUwO53Nh8cnJyUFpa6lXdffrpp9i7d6/5+M6dO0hPT0dDQ4PXdFZXV4eMjAzs2rULkZGRKCwsxJ///Gc0NTVh+vTpWLVqFQDgP//5D1577TXU19dj+PDheOONN8y6FUOu3NxcfPLJJ5BIJEhMTMQbb7wBhUKB9957D4cOHcITTzwBAHjuuecEfa+2cvG1dz49ComlbDdu3MDbb79t/qyyshKDBw/G7t27RdUZVx8haBtjBGOMsbt377IJEyaw2tpaVl9fz9LS0ti1a9e8IktBQQH75S9/yZqamlhzczNbtGgRy8/PZzNnzmSVlZVekcmEwWBgY8eOZVqt1nzOl3THGGMlJSVsypQp7P79+17T2YULF9jMmTPZgAEDWFlZGWtoaGDjx49npaWlTKvVsuzsbHb8+HHGGGOpqans/PnzjDHG1q5dy/bt2yeaXDdv3mRTpkxhDx8+ZAaDga1evZrt2bOHMcbYSy+9xM6dOyeYLPbkYoxxvjt7ehRTNhP37t1jkyZNYrdu3WKMiaczrj7i6NGjgrYxWoZqobCwEKNGjUKPHj0QGBiIZ599Ft98841XZAkJCcGaNWugUCjg5+eHvn37Qq1WQ61W49VXX0VaWhq2b98Og8Egumw3b94EAGRnZ+PnP/859u7d61O6A4DXX38dq1atQkBAgNd0duDAAaxfvx6hoaEAgEuXLiE6OhpRUVGQy+VIS0vDN998g/LycjQ2NmLIkCEAgDlz5giqO1u5FAoF1q9fj6CgIEgkEvTr1w9qtRoAcOXKFezevRtpaWnIyclBU1OTaHI1NDRwvjs+PQqJrWyWbNmyBRkZGYiJiQEgns64+giVSiVoGyNj0cK9e/cQEhJiPg4NDUVlZaVXZImPjze/WJVKha+//hrjxo3DqFGjsGnTJhw4cABnz57FwYMHRZftwYMHSE5Oxs6dO/HRRx9h//79UKvVPqO7wsJCNDY2Yvr06aiurvaazjZu3Ijhw4ebj/nal+35kJAQQXVnK1dERATGjBkDAKipqcG+ffswadIk1NfX4+mnn8Yrr7yCvLw8PHjwAH/7299Ek4vv3Xnj/9RWNhMqlQqnT5/GokWLAEBUnXH1ERKJRNA2RsaiBYPBAImkNUUvY8zq2Btcu3YN2dnZWL16NWJjY7Fz506EhoYiICAACxcuxIkTJ0SXaejQodiyZQu6desGpVKJefPmYfv27T6ju/379+P5558HAERFRfmEzgD+9uUr7a6yshKLFy/G3LlzMXLkSHTt2hX/+Mc/0LdvX8jlcmRnZ4uqO7535yv6AoDc3FxkZmZCoVAAgFd0ZtlHREVFCdrGyFi0EB4ejqqqKvNxVVUV57RTLIqKipCVlYXf//73mD17NoqLi/Htt9+aP2eMCeoE5ePs2bM4deqUlRwRERE+obvm5macOXMGEydOBACf0RnA375sz1dXV4uuuxs3biAjIwOzZ8/G8uXLAQBqtdpqFia27vjenS/9nx47dgwzZswwH4utM9s+Qug2RsaihdGjR+PUqVOoqalBQ0MD8vPzkZKS4hVZKioqsHz5cmzbtg2pqakAjA1v06ZN0Gg00Gq1yM3N9Uok1MOHD7FlyxY0NTWhrq4OeXl52Lp1q0/orri4GDExMQgMDATgOzoDgMGDB+PWrVu4ffs29Ho9vvjiC6SkpCAiIgJdunRBUVERAOCzzz4TVXd1dXVYsmQJVq5ciezsbPN5f39/bN26FWVlZWCMYd++faLqju/d8elRbGpqatDY2IioqCjzOTF1xtVHCN3GKHS2hbCwMKxatQqLFi2CVqvFvHnzMGjQIK/I8uGHH6KpqQmbN282n8vIyMCvf/1rzJ8/HzqdDlOnTsXMmTNFl23ChAm4ePEiZs2aBYPBgMzMTCQlJfmE7srKyhAeHm4+7t+/v0/oDAC6dOmCzZs34+WXX0ZTUxPGjx+PadOmAQC2bduGdevWoa6uDgMGDDCvgYvBwYMHUV1djT179mDPnj0AgIkTJ2LlypXIycnB0qVLodVqMWzYMPPynhjYe3d8ehSTO3fuWLU1AFAqlaLpjK+PELKNUaU8giAIwiG0DEUQBEE4hIwFQRAE4RAyFgRBEIRDyFgQBEEQDiFjQRAEQTiEQmeJTseGDRtw5swZAMYNZxEREfD39wdg3HVr+tsRx44dw6lTp7Bu3TreayorK7Fy5Urs37+//YIDWLNmDQoKCqBUKq3Oz5071+MhtWvWrEF8fDyWLFni0fsSnRMKnSU6NRMnTsS7776LgQMHelsUpxCzAydjQbgCzSyIx47ExERMmjQJV69exbZt21BcXIzc3FxotVpoNBq8+OKLyMzMxOHDh/Htt99i9+7dWLhwIYYMGYJz586hoqL61GOtAAAD3klEQVQCycnJePPNN6FWq5GWlobz589jx44dKC8vR1VVFcrLyxEWFoatW7ciNDQUly5dwuuvvw6tVovevXtDrVZjzZo1GDlypEuyT5w4EampqSgoKMDDhw/x/PPPIzMzEwDMdSmkUimCg4Pxxz/+EX369EF9fT02bNiAc+fOQSaTYfLkyeY6B+fPn0dGRgaqq6sRHx+Pv/zlL+Yd8ARhCRkL4rFDq9ViwoQJePfdd80d6d///nf07NkTFy5csOqALSktLcUnn3yCR48eYfr06Th9+jQiIyOtrjl79iyOHDmCoKAg/OY3v8H+/fuxbNkyvPzyy8jJycH48ePx448/Iisri1e+jz76CJ9//rnVuS1btiAhIQEAoNFocOjQIVRWVmLWrFlISkpCTU0NPvjgA+Tm5kKpVOLw4cNYvnw5vvzyS2zfvh1NTU346quvoNfrkZ2djdOnTwMwLqN9/PHHUCgU+MUvfoH8/HzMmjWrnRomOiNkLIjHElPK6a5du2LXrl04ceIEVCoVrl69ikePHnF+Z8KECZBKpQgKCkJ0dDQ0Gk0bYzFixAgEBQUBAJ555hloNBqUlJQAMFYzA4BRo0YhPj6eV7asrCy7S0OZmZmQSCQIDw/HuHHjUFBQgOrqasyYMcPs65gzZw42btyIO3fuoLCwEGvXroVMJoNMJjNXFMzLy8PkyZPNleji4+NRU1PjUHfE4wlFQxGPJaallrt372LWrFkoLy9HUlISfve73/F+x9IxLpFIwOXu47pGJpO1uVYmk7ktu2UmU4PBAKlUylnUiTEGnU4HuVxulZK6oqICtbW1be7F90wEAZCxIB5zrly5AqVSiWXLlmHs2LH44YcfAAB6vd5jv9G3b18oFAr8+9//BmCsmldSUuJ2HYYjR44AMKbELigoQEpKCsaNG4evvvrKPDM4dOgQevTogejoaCQnJyMvLw8GgwHNzc347W9/a44WIwhnoWUo4rFmzJgxOHjwIKZNmwaJRIIRI0ZAqVTi9u3bHvsNuVyOHTt2YP369Xj77bcRExOD4OBg3hBeLp/F4MGDkZOTA8CY8XTOnDlobGzEunXrEBsbi9jYWGRlZWHx4sUwGAxQKpXYvXs3pFIpVqxYgY0bNyI9PR16vR4zZszA1KlT8f3333vsGYnOD4XOEoQIvPXWW1iyZAmCg4NRUVGB9PR0fPfdd3jiiSdcuk9HCwUmOg80syAIEYiIiEBWVhbkcjkYY9iwYYPLhoIgvAnNLAiCIAiHkIObIAiCcAgZC4IgCMIhZCwIgiAIh5CxIAiCIBxCxoIgCIJwCBkLgiAIwiH/D+IHSewJj/DIAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "net.train()\n", "for batch_size, learning_rate in zip((10, 100, 1000), (0.01, 0.03, 0.1)):\n", " optimizer = torch.optim.SGD(net.parameters(), lr=learning_rate, momentum=0.9)\n", " net.load_state_dict(torch.load('net.pth'))\n", " loader = torch.utils.data.DataLoader(xy_train, batch_size=batch_size, shuffle=True)\n", " losses = []\n", " for epoch in range(200):\n", " for x_batch, y_batch in loader:\n", " y_pred = net(x_batch)\n", " optimizer.zero_grad()\n", " loss = loss_fn(y_pred, y_batch)\n", " loss.backward()\n", " optimizer.step()\n", " losses.append(loss.data)\n", " plt.plot(losses, '.', label=f'B={batch_size},lr={learning_rate:.2f}')\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the smallest batch size learns fastest but is also very noisy. At the other extreme, putting all the data into a single big batch has a relatively smooth loss curve, but takes much longer to learn the data.\n", "\n", "In general, you should break your data into minibatches and the batchsize is yet another hyperparameter that you will need to choose carefully." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Learning Rate Optimization" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We discussed above how learning rate should be adjusted when tuning the batch size, but how do you find the \"best\" learning rate for a fixed batch size?\n", " \n", "We will start by running a study of learning progress (measured by the loss curve) for a fixed batch size of 200 (which might not be optimal, but gives less noisy loss curves) and varying learning rate:" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEQCAYAAABBQVgLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOyde3wU5b3/3zO7m01CIFxy4RIBIRC8gIAoNwVrFVsLVqs9BVTsOYdj+7O2v9r2UE9rf3qqaA/11L6qrbZaT9WKPdp6A62iUuRmFSoQRJMQbiEQcieQ22bn8vtjdmZnZnezSdhNNuR5/5Ps7OzMs7O7z2e+z/cm6bquIxAIBAJBJ8h9PQCBQCAQpD5CLAQCgUAQFyEWAoFAIIiLEAuBQCAQxEWIhUAgEAjiIsRCIBAIBHERYiEQCASCuHj7egCd0dDQwOrVq8nMzGThwoVcddVVfT0kgUAgGJCktGXx3HPPcdttt3H//ffz4osv9vVwBAKBYMCS0mJRV1fHyJEj+3oYAoFAMOBJabEYOXIktbW1fT0MgUAgGPBIqVwbqqamhjVr1uDz+bj66qu58sor+3pIAoFAMCDpE7Fobm5m6dKlPPHEExQUFACwbt06Hn/8cRRF4bbbbuPmm2/u7WEJBAKBIAa9vgy1Z88eli1bxuHDh61t1dXVPPLII6xdu5ZXX32V//3f/6W8vLy3hyYQCASCGPS6WLz44ovce++95OXlWdu2b9/OnDlzGDp0KJmZmVxzzTW89dZbvT00gUAgEMSg1/MsVq9eHbGtpqaG3Nxc63FeXh7FxcVnfK76+mY0rfurbLm5g6mtPX3G5080YlzdJ1XHJsbVPcS4uk93xybLEiNGZMV+PhGDOlM0TUOSJOuxruuOxwKBQCDoW1JCLNwhsrW1tY5lKoFAIBD0LSkhFvPmzeODDz6goaGBtrY2NmzYwIIFC/p6WAKBQCAIkRK1ofLz87nrrrtYsWIFwWCQm266iWnTpvX1sAQCgUAQos/EYuPGjY7HS5YsYcmSJX00GoFAIBB0RkosQ6UqbQfKaXhzPW0HRM6HQCAY2KTEMlQq0XagnMr3D9GGj9o/rUVXgkheHwU/WEXGxMK+Hp5AIBD0CUIsbLQdKKfy4TXoqgJIoGug6+hKkFPbtwmxEAgEAxaxDGWjrbQEXQmCphlCYeZ66Dqntm0Ry1ECgWDAIsTCRkbRFPB40SUJPF4GTZtuPadrGm2lJX04OoFAIOg7xDKUjWPpuawdczVjmqs4ljWKm2dNRtr3CbqqIHm8hpgIBALBAESIhY3SikaOpuVQMSwHWYIyfShX/mAVbaUlZBRNET4LgUAwYBFiYaNo7DC8HhlV1fB4ZIrGDiNjTLYQCYFAMOARYmGjcEw2/75sBpX1rRSMyKRwTHZfD0kgEAhSAiEWLgrHZDN3ekHKlh0WCASCvkBEQwkEAoEgLkIsBAKBQBAXsQzl4mDTEbbWVjI6rYAJ2eP6ejgCgUCQEgixsHGw6Qi/+vgJFF3FK3n4zsxvCsEQCAQCxDKUg9KjH6FoCjqgaAqlRz/q6yEJBAJBSiDEwsaEtiBeHSRdx6vDiLrmvh6SQCAQpARiGcpG4dhL+dcNmznolxnbpvLXhkEMn9Qk8i0EAsGAR1gWNjz5hTSOupnWqiLW1V7JwY4cSisa+3pYAoFA0OcIy8LF6POmsXaPimIr+SEQCAQDHSEWLgrHZLP6m/P5e/ExisYOs5ag2g6Ui4KCAoFgwCLEIgrjvbVkpX+C1zsFyA530BMtVgUCwQBFiIULtbqcqjfWoKtBOmQfmYtXhTvo6Tq6qhgWhhALgUAwgBAObhfK8RKjB7eug6agHDeWniSvD2RZNEESCAQDEmFZuPCOnkLQ4zUEQ/biHT0FT34hBaIJkkAgGMAIsXDhyS9k1M33Uffpx5ZQAGRMLBQiIRAIBixCLKKQXlCE3z+6r4chEAgEKYPwWQgEAoEgLkIsBAKBQBAXIRYCgUAgiIsQC4FAIBDEJeXF4tNPP+XrX/96n5xbrS7n+Pt/ZvO7W9i0+xhvfHCY8mNNfTIWgUAg6EtSOhrq6NGjbNq0CY/H0+vnVqvLaV73X2SqQS7Aw693L+KIkovXK/Pvy2aIsuUCgWBAkVJi8dRTT7F161br8dNPP80dd9zBN77xjV4fS3XJbjLVIB4JJF3lkrQDHFZyUVWN0opGIRYCgWBAkVJisXLlSlauXNnXw6Cs7iCvB08xJz2NCe0dSMBsfzk7OyZSSb4oWy4QCAYcKSUWqcDBpiM8uvt3BGWFfWOGcvuxBsYHFDzoXD+ilvR0D2PaawFhWQgEgoFDrzi4m5ubWbx4MZWVlda2devWce2117Jo0SKef/75Tl//29/+NtlDtNjfeICcE61cvK+Z/PogBzPT0ZFRWj2kf7QbfeMbVD68xuhvcaCchjfX03agvNfGJxAIBH1B0i2LPXv2cM8993D48GFrW3V1NY888ggvv/wyaWlpLF26lNmzZ1NYmNjaSyNGZHX7NTMODKZg40k8qo7qaSX7377C8IvSOVlSi77vHatMeXDXR9Rs3ISmKMheLxfcfx9DphQldPzRyM0dnPRz9IRUHRek7tjEuLqHGFf3SeTYki4WL774Ivfeey+rVq2ytm3fvp05c+YwdOhQAK655hreeust7rzzzoSeu76+GU3Tu/Uab2k1Ps34X9KgsvQgHYuvR23IJch7yGhousSJumYIGj0utKBC1Ycfs699EKUVjY4Oe4kkN3cwtbWnE37cMyVVxwWpOzYxru4hxtV9ujs2WZY6vcFOulisXr06YltNTQ25ubnW47y8PIqLi5M9lC6RUTQF2edDCwZRZHjXe5i6Xb9jlncJpWMWcU7rCdpkP5MqmxiPhARoSHzQnMXrL+xCUTW8HhFeKxAIzi76xMGtaRqSJFmPdV13PO5LMiYWcsH99/H+Wy/xrvcwVbk+ZE1BHtJATVY+6LD0+AY8uoqGzJ4hk/hk8ESqDmqYNowIrxUIBGcbfZLBPXLkSGpra63HtbW15OXl9cVQojJkShF5i6+nLj8TGQmP7OXSwdnce0kdV/gr8egqMiChc8o7iOMZudgXuyQJDh4/xbNvlYiMb4FAcFbQJ5bFvHnzePTRR2loaCAjI4MNGzZw//3398VQYjIhexzfmXE7+xsPMFH3k7/xGdCCXJQhUyPJ6LqOJslUZo6MeK2mwa79dQBs3VvFquUzhZVxFtF2oLxHXRPtryN3RhJHKBAknj4Ri/z8fO666y5WrFhBMBjkpptuYtq0aX0xlE6ZkD2OCdnjCOxaT4dmOLP9WTqDr5nL0bp0cqdPZcU557J9bxVb91YZznRJQtN0RrfVMrbtBBUZI3lt60G+fNkEIRhJpqeTeHfPUfnwGnQliOT1UfCDVV0618n3N1Gz9jnQNCSvj6EP3AcjRIMtQf+h18Ri48aNjsdLlixhyZIlvXX6M8I7egodsg80oy/36Ms/xzmhdqttB8pZrJUz9/ICyvShDMrwsfeldVxV/SESOqrk4QXgZ0dOcsuiyVwxfUzfvpmzlJ5O4t0+T2kJuhK0QqjbSkvinqftQDk1zz8HmgqArgQ59ck+/AtHR+w3UPu8D+T33l8QGdxdwJNfSObiVSjHSxx9ud0T1JU/WAUojK79CB0dCUBXGdt2guMZuTy/oQyAlrZg0sJre8LZ8EPtySTeEzKKpiB5feiqguTxGktKXRibKRQAyDJDLryAgH2fXhK7VKS/vPe2A+Wc2r4NgCHz5gNQ+f4htIJzU3K8iUaIRRfx5BdaIqFWl6McL6Flf314glKC1L/+Kr6cXGOpIfQ6HYmKDMOvoWo6z71dCjoR1WvLjzUlLEejO5N/V3+ovbXE09NzxJrET5WU0vDhxwkbd8bEQgp+sKpb48womoLkSzO+K5JM3vJbGTKlyBED312xOxsE3qSnQt+b16DtQDlH1/wMVAWApq2bkSQJXdOQPN6ov5uz6TMCIRZxOdh0hP2NB5g0bCITssehVpfTun4NaEGkFg+S14OuqKDrtO77BCQZPB7Dyy3JtFw0j6kHD3Dh6QN8MngixzOM/JKgolm+DICf23I0ll01yWF9mHc0TRk+0mZcGncS6c5dWqwfqsMZC2d052ceSx6UhdbSHPXH051xu8dm/m9O4vKgLNpKSwhUVlL3v2vRgsYxc5cujzh/T37QGRMLuzWRd0VgOrNY3Mfr7Fp15f109z0natKLdZweWWu2a4DsIfuyyxkyb37ybmRKSyyhAEBVrQhIHeN3A1iWh5yRQeM7b1s+qqQti7qsnWSKkhCLTjjYdIRf7fodiqbglb18Z8btjDleAiFnd9ogjfzrFnKqpNYQCgBdA10me8EV+MeOg7V/ZIaqoAPTTu2nOJSXcTwjl08PNVJ2dBcXnjucoGKkjeedrqb8T8UcSc/n1UF5fHG0xoWb14Kq0gRI724kd9ktaC3NUSffrt6l2Sdw9w/V+iEGO0D2MGjaRTGP2XagvFNT3HLsqqFlGEmK+uPpzrjtk4Qkga6q1oThHzuO2j+tte7ijc/DsPzsDubcpcsJVBzh1LYt6KoafUw9mCTdE7ldoIZfuzjimpnv3S52bjFzH6/5450Oi/bU9m0xRcR+/HhCE+19ByorHdetp5NeZ+ftibVm/76gKjS9/zdObd/WpfF11/I2fyd4vGHBkENZB7oOkkzbwYPUvfZK+HtuQ1eCXfZtdVfE7dbOqW1bKPj3u5MmGEIsOmF/4wEUTUFHR9UUSo9+xKjmVpA8gAayl0Ez5+ObDK2ffRZel9Z1fCNGoLU0Wx+kBHjQmXGqjKmnD/DC6EUcz8glqGjs3l/H6LZaLjx1gGnN5ci6xnwk3s6ZTV1tAE1VrYQYXVE6nXytuzQlCEjGXbbtCx+oOILS1ETrJ8XWJGmf0ADqX3/VEAoATaWleDeSx2OZ3OZ+9ggf+90dGHdYSlMTLcW7jedNbEt2I667PjxhRBl3NNyThJXfEpowkD2WQICGJBthziAZ47ALh+2HbQqUeQ55UJYlOvZrZBdo+77mNofouQTKnLwrH16DriogyZbYWZ9h6Bgm9a+/Gnk8+4Sk6zRt3Rx+aBPchr++ScvePY7zu0X51PZtjgnqVEmpQ4zRNevzM4UpmlUXzbKxPx/vZsB+sxOorHTcBJni2oYvvN3+fdF1x/ji3RzZP1fzdxPNH3Fq+zbjZkJRQPYwbNE1aG1t4d+PooREQ6dl98dRv68AyHJca8kdLRd3ReBAOfWvv+qwdnQleb46EGLRKZOGTcQre1E1BY8kc87ujSht7SB58E1ZiJwzDuV4CWmjp5B3862OD9v6ctjuRkw/hldXmdp8AICxbSdIUzuY3fQpUsgpLgE6OtfUfcjbObPRkJEwfrAaEp4ok6/5RW8rLWHo56+yTOCatX80JiTFZkLb0FUFraWZ4dcudloUjp10hsy/HN+IEciDsji1fZsxEdmFIDRZN23dAuhR77Dsx2v9dB9tZWUOocpduty6hjVr/0ig4kiEaS0PyjJ+oLGOr2vG86E7vqGzZqKlDwpbHKqCJRw2JI8XtbWVo2seCj0nGceCyElaksJWjXldQ6Kdu3S5Zak5BMomRtYEJ+khITO2Nfz1TVr3feK0mmzHR5Ijxm1ce9UQSo/XEnUk2fH56MEO6l9/layZs8Ljk+QIyypQeSg8Pk21JmKTpi3vG9viWXW25yOuiyQTrK+3qjU7JnHzuxe6ntZ32XVzlLt0OUPmzTduSPbuMZ7XdU5t2xLxnXHc1Ng/V9tnEtUfYf/NaCqN77zNOav+g7bSElr27Ap93/Ton4mJ7CFv+a1RLVbzJkMelBUZLRcSZfMGD8A/dpy1v+NahZC8XuRBWTS8uT4puTySruvdq7TXj+hJIUFwFuAyfRbja08wevc71iTknbIQpWybsSQl+8hcvIqO5uh3WtZdtvmlxpj0DXTrP0Mkwv9rwOHMUZQOGkd+ewNIUOMfzqKGncjuH7LLLA7fXcdB9pB3860MXXgFDW+up+6Vv4RfZ5ZgsU0GNWv/6Fy77SqhO7PA0QpaP90XPocsW5bJoGkXGT9Cx/syxucvKHDe6UlS1PcvebzOJSabAxKMz0dtbbXEFNnDoKlGjk+EFWTiPlesbbJM9uULww+jrFtD2P8DUuiD1sLvQddDn6Hk+Bwyz7+ArJmzwoJns0rs+w26aAbe7GzaK44QOHQwYsz2Jbj2iiMEDh8KTf7G2DMyfFS/u9E4LhLIUniSRe/8O+Ww6qJfF7tViySHjxntO2t+/6KdM3StJK+PzAsuDN/ZSxL+8eeSPnYc/rHjCFQcMQQuyucqeb3kLruF5o93hpeRO0OSyLnhRjKKpljWoRT63pkCPWjqNLzZ2dbE7l5SirYsa9wEqI7zIEXe0ISvSVjw3O/XbjVd+MB9BLqRy9PnhQT7O2ZinppRTmvxJivXQgLLd4Gm0FG2DTlrBNkXh0NrwekMrX7uGeMOEJDRCS2OOP4aP0vJCr09t7WKc9pqrGWrczpqkWfMJntQmiFAoR+JHvpimeG6lpM91o/bEheN2j+txV9QEL5rj7Ku37R5U+SX2n2saF9uj4fsyxZYd3ttB8ppLbE5C83X2Je77BOgplLzx2ei/3g8HmPCCAmE/cfZ8OZ66zg6xnKLaRnVr3vd+LHKHoZdvYiT770baU11ht2ysI9VN+5KTRGSJKzz5C5dbn0Pcpcup3btc8ZnpttqooUmbcxlM0m3JkRzyc5fUOBY5jm1fZtxztA4WvbuibwrNoUnZIW27C02LBj7ew6Nvck6tylWhv8NsL67MdHU8ARvP7fNgnGIgm5fTovxnY31/dXC1oE3OzscbabrBA4djBRKN5KE/9wJkct6EHnjRfhzML9f5pLeqNkzOXmytUtBBae2b4sUrtByqfXezZkglrWi64RvKUHyeMgLfbeqn3vG+kx1VYmay3MmCLHoIu5cC4Bg2TZDPCQZpXQL6CodISvDLhgmQ+bN59T2bdaX2i4UAJIs817ebGq8Q5nfuIdzW6tA15F1jbFtJ5CApSfegaMqp7w+9C9cT3DPHjy6alu+Mo6Tt/xW447JvIu33XWaNG3e5Fi7PrV9m2NyM62N8ISohS0BsIRg7LVXU/HmO+HJxHa3414SyJhYSPZll0efeELLXYDzR2XebduIFd1knccWYRN1sjIOTOBoRegu335w2bijNidO213coItmkDFhgmNN3rJUVBXMeUdTCRvtuuG/Mp9qaTae0/WQM8spepZVpDiFJpoD1PxrXU9NC5/XZpFYFqGuGxaU+w5e18Jjt1s1uoZvxAgyiqaEv7vmZ0KU/R3X0fgeAGELxv4dsr/GYyzXaC3NzutpHsrjwT9hIu37y5yn8HgZMm8+Q+bNp/71Vzu3EOyfqyTTfqDcMSn7z52AN3uozR8RtmqjXfeMiYUMyR1MoPZ01yIUY92QSDLDrr4GT2Ymwfr6+KJsvU5iyPzLre+GsQRsHjIyl+dMEWLRDey5FoAlHnpzPcGSTZaVoRwviSoW5h2JtZyiqkiSzLBFxhclo2gKX0jPpbSikQlSETzzG3RFQUOiMnMk4wPV1vKTrijUVdXz1zGLuKDJcIxLunFnUjf3WiYvvML4kpeVWZFOw794bcTymPkcYJs0w5ObO6zRnMwgHKo3JHcwQ062Oo6XZ7uTdmOJprmub7tzM49pLHlFv+vLvnxh3DBB81rLlYc4WXHcEkZrsgpNzlkzZ1nXCEl2OOkda+lRrqF5noY310feCZqWhysowLymsteLpiidW0W2z6KzaCLH9TSXp0LnNS2SQMWR8CTktmB029jtVpNt7Pa7afc1cSwDgSV+SDIdRyui+nXskWhm7snQhVcAOK9nSPAmrljOyZOtjuAAd7jsiOuud1qttvfj/lwjJuXQHbrDHxG69vFCpeNhOfftyB4GX3Ipp3d+BJrGyffeNT5TCItyaG7Q2toArGU1+/Kq/T1ZVn9IRNy5PGeKEIszwBSEjrJtjggp0/KIhnVHMm9+VNO1EKykvLZc48dZP7yA2fpQRnfUo/xxj9GACQl9XCE1zQGOp+fyyZCJjGs/wbGsUdyy6GrrXNx2B3W795I7fWrEJGcPVwQck725rathjd0Jf4x2bvfrhobEzi6sSLLlX+kKGRMLyZ0zA+3vuxzvzT05R7tztK4fxHze2s9lxbgnpmjHveD++6iKkSwYLe+gs2iirlxPu6DYr4F94keSyb/6StJmXBr1GPZJ070c1rrvk4hjB+vrDZEGa9K3R8DF+g243/+I665nyJQiArWnO/2OZUws5JxVd1vBHrH8BvboJ3uipLlPd3M+4hHr+9FWWsLpj5wBEMOvXRz3dxTturmvmfn9SyTCwR2FrnaYsifoIXnwFV2Ob/L8qFZFIthUXMV7L2/mnNYTHM0YSd6088gelAbA2JGDqThhjHne1FEUjsmm/FiTI9kvXkOmniZfJbtb2JkkhZljS3Y2bXePH++axUzEC00GPcl3iDVG+/axc2b06LOMduwzGbP7eMn4jnXlesQbb1fGFStxLhGfaWfvJ9Gd8oRYRKGrFzmwaz0dO/9imdxps76Cf8bingy1S9S3BLn7N1tRVZuDC6N0yLKrJvHCu/sdwlBa0cjLmw8aqw4S3LBgAl+aOz7h4zqbWkv2Fj0ZV2+Uj0j09UrUmPvr5xgvCTKZn2m/a6t6NuOuRtvZ8lMimDJ+OJdPHcWm3cetbTpGZ75/lNagqJqRq6ZqbN9bBYBHNkqmezwyRWOHJXV8guRypmvnfUF/HHMi6UoyYn+5PkIszoBoEVKBXesdlWkTzbypo9j2yQkUxWjjKkng8chcXJRH2dEmVFUDCTbvOR7KnZJYcNFoa2lKIBD0Hj2pe5WqCLE4Q8wIKbv/orPw2TOlcEy2tcQ0KMPnKDhYkJvF9r1VbN5zHHP1zVyGK61otI6RqOq2AoGgc3pS9ypVEWKRIBRbgUHUIIGdr+KfdX3SBCPaRF84JpvSikbsbhoJo7WrqulGvlQovN9eIj2R5dEFAoGT/rTU1BlCLBKE5b9Qg4COemwfrSfKkmZhxKJo7DB8XhlF0ZBkiYsmjmB3eV1EXpuiapa10Z2IKYFAMDARYpEgTP9FYOerqMf2YaTsx07QSxb2ZSrTof3JoQbLx2EiSxJFY4dRWtHocIyXVjQKsRAIBBEIsUggnvxC/LOup/VEWcjCkJDSY4eiJQv3MtW/L5vB9r1VxnKUqiPJEosuOcfye3g9MqpqWCL1Te2UH2sCsCKqhHNcIBAIsUgwnvxC/POWE9j6HOgage1r8Qwv6FXrwo0pHvOmjrIEwp6TseyqSVScOM3WvVW8v+c4W/YaNanUUMWFLcXHuXyaiKgSCAYycvxdBN1Fb2/GqB8bXopKBQrHZPOlueNpaQs6lp5a2oKMyE5H1fTQtrBQAKgabNp9nJ+/sMuyOgQCwcBCiEUS8I6eArLPqHIpyejN9ajV5X09LIuiscPwemSj2q0kMSjDR9HYYciy1OnrVJtTXCAQDCyEWCQB09ntnWI0fAmWbKJ1/ZqUEYzCMdksu2oSsiyhazovvLsfgMunjrL2kYBzRw1mxqQcvB4JOZT8Z88CLz/WxBsfHKbkcEMvvwOBQNDbCJ9FkvDkFyIfLzGautgbJIWyvfvShwHQ0hZE03WrXEhpRaOVHa6qGh6PzLKrJlt5GKaz28RepHDd9sP8YKkIuRUIzmaEWCQRR+6FDkrpZtC1pGZ4dxVzKcoUBjMhzx52a5/8t31yAkXV2LK3imkTRnCyOWCF4ypK7JBbkfAnEJwdCLHoImYv7knDJjIhe1yXXuOMjFINnzf0Sf6Fm1jCEC073JmLobNrf53jeSnk97BjWiNm9rhI+BMI+jdCLLrAwaYj/GrX71A0Ba/s5Tszbu+yYIQjo2z0QoXarhCrbIgb0wpxJ/ZBqDW2rvP8O2XsPVBP9qA0MvxeNuw4imqrOyIS/gSC/o0Qiy6wv/EAiqago6NoQT6s2tllsXCUMZfkpDdISgamFbJ9bxVbio9bYbVmO2PD7xFpcTiQJFrbFd744HCnS1Ji2UogSE1SVix27NjBSy+9hK7rXHrppXz1q1/ts7FMGjYRWZJRdRUd+HvVP5g9alaXBMNdxrw/iYQde2Kf6eweO3KwkdwXxeIwkQhZH5rOXz+sQMIom3751FERSX7d7ewnEAh6j26JRUdHB3V1dYwePTpZ47E4deoUP/3pT0lLS+OOO+7oU7GYkD2OuaMuYevxvwOg6Sr7Gw90y3fRX0XCjXvpqiA3i13ldWz48IgjkQ+MzltmIUMT0wrZtPs42z45wbKrJlll1mPVqRLWhkDQ98QVi3feeYe///3v3HXXXVx33XWcPn2aO++8k9tuuy2hA3nqqafYunWr9fjpp59G13UefvhhVqxYkdBz9YTZoy7mwxP/QNUUPLKXScMm9vWQUoLCMdnMnV7AjMIch8VhCgBEL2QIRhTV8xvK0HTDAX7VxQXIkoRGuLNfPGtDCIlA0DvE7cF90003sXr1avbt28fWrVv56U9/yooVK3j55ZeTOrBTp07x0EMPsXz5cqZOndqjYyS6B7cZETXIN4iWYEu3IqNM1OpylOMlSOlZ6O3N3Vqa6q99iM0JfVCGz6pBpao6SM6y6bIEmm5YJNdccg6Z6V7qm9p5P9T1TwIWTh/Nii9MsY4bb9mqv16zvkKMq3uk6rigD3pw67pOUVERTz75JAsWLCArK4s4+pIQHnjgAU6cOMEzzzzDqFGj+P73v5/0c8bDFIaeRkZZ3fTUDmtbh+TBf9mtpJ13RTKGnBK4l67GjhzM8xvKHNFSgNW0Sdd0Nuw4iqbryLKRPa6GHOlb91ZZvg77spWiaLy29SBfvmyCsDAEgiQQt9yHLMu8+eabbN26lfnz5/P+++8jSZ3XELLT3NzM4sWLqaystLatW7eOa6+9lkWLFvH8889Hfd2aNWt49tlnefjhh1NCKEzskVGqprC/8UCXX2t107OjqwS2PpcypUB6AzN7PBaSLKGFihrqms7Y/MHWc6qq89rWg5Qfa3LUuNKBTw81Ooodlh9r4qX3ykTxQ4EgAcS1LH74w60RCd8AACAASURBVB/y2GOP8b3vfY/c3Fwef/xx7rnnni4dfM+ePdxzzz0cPnzY2lZdXc0jjzzCyy+/TFpaGkuXLmX27NkUFibeAdyZSRWP3NzBUbdfKk3lrSPvoWgqXtnDpedOJTcn+r5u2s+fSdWu19EVo5teGA1/0yGGXTijx+Pqa7ozrjnTxrBu+2EURUOWJSRJQlWN/6++dBwTxmTz5Gt7URQNr1fm2vkTePK1vQSDht/j08ON7K/cxepvzufB/zOftW+XsHt/reUYr6xvZdjQTB7+0y7rGKu/OR+AvQfqmDoxhynjhyfpSnSds+Gz7E3EuLpPIscW12dhp7vRUD/+8Y+54YYbWLVqFc8++ywFBQW88sor7NixgwcffBCAX//61+i6zp133tmzd9AJifZZmPQkm9vE9FnoHa0Ei98GXQPZ06X8i1RdH+3JuOyOaSDCSe12XJcfa+K1rQf59FAjOoZ/Y8FFoxmRnR7uzxFqJXvLosm0tAV5efNBdN3Y96LCHIoP1KNpuqP/eF9xNn2WvYEYV/fpdZ/FmURDrV69OmJbTU0Nubm51uO8vDyKi4vjHiuVmJA9rtsiYWIPo/WNn0lH2TaU0i0ESzYRLNvW5zWjegu3H8M9cZuPzZLohWOy+fJlEyg7usuKrNpSXOWIpNqw4yiaZmSTT5swAk9oOUuSJfaU11k+EUWE5QoE3Sauz+K3v/0t//RP/8SGDRuYPn06f/vb33jttdd6fEJN0xw+D13Xu+UDOZvw5BciZ42IqEwb2LV+QPkwomFGOr28+aDlh3CUVtexNWvSqKg5bauia2ST68ClF4xkbF4WmiPqyqhl9exbJaxZ+7HjHAKBIDpxxcKMhtq+fXtCoqFGjhxJbW2t9bi2tpa8vLweH6+/YzVKQrIq03bs/EtK9b/oC6Il6EF057jHI3NxUV64oVMITdXZ8Wk1h6rCprgc6j/+wrv72bT7OIqqR5zDjdm3Q4iJYCATdxnKjIbasmULP/zhD7sdDeVm3rx5PProozQ0NJCRkcGGDRu4//77e3y8/k7MyrRqkI6ybWQMgCWpaEQroe7eLsnOsiEFuVlWpVtN00GSLGGRgPPPHcaXL5vA9r1VBBVnunm0xk7b91bR1NLB3oP1onKuYMDT5Wio73//+92OhopGfn4+d911FytWrCAYDHLTTTcxbdq0Hh/vbCBqZVp0lNItqP2s6GCi6KyEeqyeG/b6VWYS4J/eMxzfHo/Mly+bAMAWWyMnjwzTJuaQPSiNytpmSyCKD9RFlC9xV84V/g7BQKLL0VDHjh1DURTGjeuZY7cvSFY0VKKxkvU0xdigh2YpSSZt1lfwz1jcJ+PqKqk6LoD6liB/Lz5mTehvfHDYipKSgOmTcvjkUEOEpRENORRpZbdgVM1IHIxWGLEzUvWaiXF1j1QdF/RBNNThw4f51re+RU1NDZqmMWzYMH77298ycaKojZQo7JVppfQsAtvXGsJh63uhVpfTWHYINfvcAWlp9JQp44czYlC4MZN7eSt7UBqK24RwIYdWXXVN548bSkHH4TC3F0YUy1SCs5W4YnH//fezcuVKbrjhBgD+8pe/8J//+Z88++yzSR/cQMIeUusZXuAoaW5aHq0hARko4bXJwL2MBaGWsa5Ch/blKcCoT0XY6IuGWKYSnM3EFYv6+npLKABuvPFG/vCHPyRzTAMed0lzq0xIKLy2r1uy9nfcOR6meJiFDgHHklL5saaogmIm++09aCT72Z3k0YocAtZ5kGUKRmQKERH0G+KKhaqqnDx5kqFDhwLQ0NCQ9EEJDOzZ3iCDpKVMS9aziXjtZe2dAs2KuWam+BXTx8TMNjfFRVE0/vy3cg4cP2UVT5QkIkREWCCCVCauWNxyyy187Wtf44tf/CKSJPHmm28mvJdFf+VMyn7EI1qFWmQP/nnLhVXRB7gjrdwRWmBM+LvKaiP6j+tAWaUzR8OslGuKiFmGxN4MSgiHIJWIKxZf+9rXGDt2LFu3bkXTNO69917mzZvXG2NLaQ42HelxqfKuEL1CrR4KsxX0FdGsEHPJqSsRVXbcIhJ0NYMSznJBKtGltqpz585l7ty51uP//u//Tqmy4X1BtFLliRQL7+gpdMg+UM0KtRKSx1iCMpen+nNP77MJM9vcjSwZ4Yiqqlu+DlmWKBw9JMLSgHCvcnPp6oV3yxiXP7hbIbkCQbLoVg9uk+eff37Ai8WkYRPxyt6ktVl1h9Pq7c3knD+TkydbQzkZQTpkn4iMSgHMcFy7A9yek2E6te2tZh/+0y6r5Lq5/zWXnMO7/6i0jnOo6jSHqozOgquWzwRwRHGZbWzdznjh/xAkgx6JRW90ykt1JmSP4zszbk+azwIio6LScwcT/OixsB9DREalBPZwXLsouH0adlZ/cz5/Lz4WISJtAYUj1acd9awUVWf73iojIivU9wNdtzLMtxQfZ9rEHAD2HqxHUXVHa1ohHIJE0COxGKhVYt2cSanyntBeWUqwdItti4TeXG8VHBRLU31HvIgqN+5kQXuorSxLeGQsMfB6jN+bWVhRsy1rgbHfrv11juNrms5fP6wAwlbOFdPH9OzNCQR0IhYbNmyIul3XdTSte468gUAyI6NM2o7sM4oNWugEP/sbwc/eNxbINXVA9PQ+G7FX2dU1nQUXhRuMzZs6CjCSB80CinbLIh6apvP8hjIKcrOEhSHoMTHF4rnnnov5oqlTpyZlMP2VZEdGmWSMu4BG2ReqISXZhEML1yEM9fT2DC8QFkY/wl2GJJpT28z1ABg7cjAVJ05HFD00S5O4S6KpmtG7/MuXTYgazWX6Ocw2nNF8H8IfMrDpkVgInCQ7MsokvaDIWUPKLGvuRteEL6Of0Vk1XXOSHpThs/wW3k/CobVmOXUIWyGlFY20tiuOnI99hxr57MjHDl8G4Mg0f3BoJnv31xghvLYWtO79RFjvwKNHPguBk2RHRtlxO70NwdBAkgHdyPby+ESWdz+ksxwORTU6TOqa2Q0wXIcq2uvMxzMm5/La1oPsO2Q0djJ9GRLg9crMv3Cko8nUxp0VvP1hhVWtOahovLb1ILnZGdZ+iqKxfW9VTLGI119d0D8RYpEAeiMyKhpp513hKDoIkU5ue05GtOcFqY3dl4Guh1rK6hHNmmJh9i7/7MjHjnL9puA0tXQgSxIaxjFBiijr/+mhRmTPSWQJVN147da9VVGXyjbtPmZZJbJHQgLROOosQYhFgujtyCgTt6VhVqkN7FpvK3cepEPyGDvoqsjP6Ee4fRk9KQdSOCabWxZN5vkNZY7aVJIsWV0AZVniqosLAB2vx0gkxPCjh6rt6owbOdgK6dU0I5zXnfexec9xy1+iqmHRcVfkFfQ/YorF8ePHGT16dNTnNm/ezIIFC5I2KEHPCTdSCgJyqKa2HvJthH68Ij+j39CZL6M7XDF9TESCYH1TO+/vOQ4YYrBhx1F03SiSuHD6aMaOHMwL7+63hOryi0ZTWbvfisiyN3+SMPJB7JgZ7O6KvIL+SUyx+Na3vsUrr7wCwLe//W0effRR67lHHnlEiEWK4ihnjs2XIcnGDrpRuVZKzyKwa71YkuoHdDeHoyvHMZ3intBkjiRZpUZkTWdEdrpDYEyhMh+bQhMt7wMiM9iFz6L/E1Ms7FnaR48ejfmcILWwakqFGiX55y1HrTuCBMg549Dbm53LU2JJasDhTgBccFGkFWFaAW6hMh+bPT5MK0PCWJqSorSYFSJxdhBTLOxZ2u6MbZHBnbrYa0qZTm1TGJB9+OctRzm4M1ygUFPoKNuGLBzfAwZ3AqDdiqisb+1yU6b5F44EwjkfkLg6VSKnI/XokmUhiI09cxvo9YioaNid3oFd68PLUmowMjdDB6V0M+iasDIGCG6nud2KmDu9gNra052+3m2ZSCHfhTeUTOjeJ1aTp1iCEO21nSULdrZdkDhiioWmaTQ1NaHrOqqqWv+D0T1P4MzcliUZCQlVV5Oaxd1dHMtSSM4m0lLosblJOL4HBN11mrsnYrtlYvdX2COe7PuoquYohOgNRXW98O7+qGJS39TueG1pRSNzpxdEiJS53AXOhEHRQCo5xBSLsrIy5syZYwnE7NmzrefEMpSBPXNb09XwjyaJWdzdxV3q3FiSiiIcIFq2DiC66jSPdpdvt0zs/gq7leK2XppaOqzmUKqq8Y/SGocg/PXvRyg+UG/lZ5iOd0mWOHj8FL/58x7a2jpsr9HZtPs4m4uruGjiCEfCoGgglRxiikVJSUlvjqNfYs/clkKWhaarSc/i7gruBkmmtWAm8TmEQ5LxFV2Ob/J8YVUIHLgthNKKRr40d7zDMjH3c5dlt5dtf/6dMuuYkgRpXg8eW2Oo3eV1mCvfqqpz7qjBDM3yU3ygzqiou78Oj4zjNWCI1J7yuqhRXUoo+zxaPSxB9+k0Kc9cgvJ6vTQ3N7N9+3aKiooYN67v75hTAXfmNuD4/+3DG/vEf2HPtXD7IaIJh3BsC2LRmX/DHSXlxtznjQ8OO7PHdUMcJCnUHTCKe/Rw1WkkuRl7gWtVg3NHZTE0y8+e8jrrdTpw2dRRjMhOd9TD0jGyz0sqPmbahBFkD0pz1M4Sy1TdI6ZYlJeXc/vtt/OTn/yEuXPn8tWvfhWA5uZmfvaznzF//vxeG2Qq487cnpA9rteq0MbCkWvRiR/Cnf0NkRaJYGCTiKRAu+Bgq2+FTkR+hmTLGkfTjcr7tp0OV53G623hmkvHsmHHUWu5ye67UDXdcRxV1a1+H1uKjyNJkihB0gNiisWaNWv47ne/y+c+9zn+8pe/APDGG29QXV3NXXfdJcSiE3qrCm0s3LkWXfVDdGaRCAYuZ5oU6F6SMvM5ouVnuPM9ll01iYoTpzne0Mr+ipNWTavMdC8/vHmmdUy7YxwAHWtpyt0oypSoZC5TnY3RWTHFoqqqiuuuuw6ADz/8kM9//vPIssyoUaNobm7utQH2R3qzCm003LkWXZ3wY1kkwtoQnCl2wbFndUPkklC0rO/6liA/enybYznMfM7ZYTBcXsQUmi3Fx6M2ijKXqcqO7nJEY7lb43Zn4jcz47fawonPFuslpljIsmz9v2vXLu655x7rcSAQSO6o+jl9VYXWTrQlpnhEs0iEtSFINPH8HdEsmSnjh0ddDnMnGF5+0WhGZKc79pk3dRTb91ZxpPo0h6tOOywN01IxQ3vNiC2zhPtVFxcYy12hKC13drodM3LMPAaEgwLMsfZnSyOmWGRnZ1NSUkJzczO1tbVccsklAHz88cfk5+cnfWD79+/n0UcfJTMzkyVLlvS7Za++qkJ7pngnz7dKgyjHS9Cb6x1JfR1l28gQYiHoA6KJSFc6DNpLlPz8hV1RQ36B8BIWWNFUb31U4YjSMsN1zQZSc6aNofFka+QymIkk0dquxGwc1Z+Wq2KKxfe+9z2+/vWv09zczA9+8AMyMzP5/e9/zxNPPMGvf/3rpA+stbWVH/3oR3g8Hn7xi1/0O7Ew6Y3e3InAUa1W8kDpFiPTW/IAMmBUrVVKt6CKEFtBitAdB7x7Xwjf7VfWNiMXS6i6rYS7FNnbA5wNpF7begjAqr4bDuHFcOBrOm/vOOpoWmWWdjf9N4nsPhitRW6iiCkW06dPZ/PmzbS3tzNkyBAAZsyYwUsvvcT48eMTOgiAp556iq1bt1qPn376aSoqKrj77rtZsWJFws/XG/R1VFR3cPgr7OXM0ZBzxqPVHjQeipatghSjOw74aPvafQyyLHHNJefQFlBoaukw+n2EensA2KsgmZFW1i9F1Rk/ajDj8o1J+v09x8NRXaGmVfbS7u7Oh/b+ID0RDXcC5YNDMxkxyNft48QiplicPHky4v8JEyZYj4cOHZqwQQCsXLmSlStXWo8/+eQTxo8fz5/+9Cf+5V/+hWuvvTah5+sN+joqqjs4/BWucua+KQsINFRaz+nN9ajV5YAhMu3nzwR/9N4nAkGqEs3HgK7TFlCs0iRyqLfHvKmjqKxtdjSQAsMCkW2JgIerTlNZ28Kyqybh/SSyadXB46esMF5750Mk2Bwq+W72Pe+uYLgTKPceqOOKaaMScKUMYorFnDlzrLIe7qKCkiTx2WefJWwQ0QgEAvz4xz8mKyuLhQsXJvVcyaKvo6K6Q7RqtY4M8OEFdJRtQyndQrBkE8HSLcYLdZWqXa+T8SXh+Bb0L8zJ1Y7df2GvymtaJGaklpn8p+lGTsf4UYMt57mqarS0BSOWx8qPNfFqaNkKQJZh2sQRAI4kQyVGV8F4/g23/2bqxJwEXq1OxOL6669n165dXHnlldx4440UFvZsImhubmbp0qU88cQTFBQUALBu3Toef/xxFEXhtttu4+abb4543cUXX8zFF1/co3OmCqkQFdUdorVotf8vHy8xlqhcS1W6EiSw+008eRNEeK2g3+CucWUvTGj26nB3+HNkpeu6JSjj8gdTWdsSEdprn9RLKxpjZLJLjsRDWZIiugpGq9EVzUluF6gp44fHrSDcHSS9k1rkbW1tbNiwgVdffZXW1lauu+46lixZYvkw4rFnzx7uueceDh06xFtvvUVBQQHV1dUsW7aMl19+mbS0NJYuXcovfvGLHotRKlJWd5B9NWVckDeZyTkT+no4Paa9spS2I/vIGHcB6QVFtFeWUvX8fehK0LC/wVWMUELy+hh1832kFxT1yZgFgu5QcriBvQfqmDoxhynjh8fdbn/+x09sQ1E0vF6Z1d80AnA6O9bgzDSefG0viqKFBMIQG4lw+1lZlvjmV6YxftQQ6zWnWzuobWzj7b8fxvSdTzpnKBMLhpKZ7uXV9w+g6Tq+0DiijTcRdCoWdk6cOMFrr73GX//6V8aPH88vf/nLuK/58Y9/zA033MCqVat49tlnKSgo4JVXXmHHjh08+OCDAPz6179G13XuvPPOM3snUaivb44azRCP3NzBPVbkZDq1z2Rc3cURHWXLr+j4bFOoJ4YWLnFuR5JJm/UV/DMW98o449Gb16w7iHF1j1QcV/mxprjNotwWgem7sGey230aZmSW3TciAbLHCPVVo7SwNZEk+MqCCXxp7nig+9dMliVGjMiK+XynhQTtNDQ00NDQQGNjIyNGjOjSa1avXh2xraamhtzcXOtxXl4excXFXR1GytOfnNqdESubW29vxmiAESq8I3lCgqEDkiOZz6xuq7c3i+UpwVlHV5pFuZ3OLW1BazKPlqlefqyJP24ocy5XYURaTZ+UQ4eisu9QY9RzRVu+SiSdikVVVRWvv/46r732Gh6Ph+uuu44XX3zxjJLyNE1z9MPQdf2s6o/Rn5zanRGrvlS0Ht+ZniCtqs8SBcCwStSO0NEkOjwi+1sw8IhVtReih/G6/RomOrD3YD3Lr55MScVJI5zXhixLLLrkHCtbPBkJfjHF4tZbb+XQoUNce+21PPzww5x//vkJOeHIkSPZuXOn9bi2tpa8vLyEHDsV6G9O7VjEqi8Vbfuw3MEotacta8LK+rbovPqtQHC20t2qvUVjh+HzyoZfQ5YoHD2EssomwEgGbGkL8sPlM9m+twow+p/bl7WitaJNFDHFYseOHfj9fl566SX+/Oc/W9tNS+Djjz/u0QnnzZvHo48+SkNDAxkZGWzYsIH777+/R8dKVfprqQ83sepLxSptbvk4kLFSWC0kpPQsx/6iOKFgINDdpEF3lrlZoiRWlBXAGx8cjtqKNpHEFIv33nsvoScyyc/P56677mLFihUEg0Fuuukmpk2blpRzCXoPh4+DUI92yYNn4qWoBz4CXSOwfS2e4cYX2CpOKHmsLn3mcYSACAYybjHoimXS2XJXoogpFmPGjEnYSTZu3Oh4vGTJEpYsWZKw4wv6HsuXoQYJWxQ6tJ3GcoiHlqL05vqwP0NXCH72N0eSn6huKxCE6YplkogmVfGQ4+8iEMTH9GV4z7sCZK9RMkT24p0wC2Sf8ViSUWsOEizZHHkATTWc5rboK4FA0HUKx2Tzpbnjk1a9tsuhs4Ke018qz54pnvxCMvILUSfPj1kqRD3i8nVJcmjpSjLqH+h6RHc/4d8QCPoeIRZJpj9Vnk0U0cqGWKVCHDum4bvwKoLFb4dyNWR8UxbgC5VAV6vLLZERy1MCQd8ixCLJnC1JemeKu6qt6dQ2lpvMJD/NaDpzvAS1oZLA9rW2XA3iht8KC0QgSB5CLJLM2ZKkd6Z01hfcLiKmFQFRSol0sjwFiPavAkESEWKRZOxJeoN8g9jfeMDaPtCIlp9hFxG9uZ5gyaaQD8OWoyF5HMtTQERvcN/k+VHLkwgEgsQgxKIXMIVhoPkuuoopImp1OcGyba7wWwnvlAWkX34bQGSWeEgcdDCirlzlSQQCQWIQYtFLuH0XH1btHBARUt3BtDLCTm2jU19aKGEvok+45LGWrDw540hzRWEJBILEIcSilzB9F4oWRAc+qNqJpmvCynARK/wW3FniGp6x01Er9ljZ4ZmLV0WURm+vLCXw6cdCQASCM2RAiYWqKjQ21qIoHZ3uV1Mjo2lap/t0l0zgu4W30Rpsw16RXgKk0+2caDsS9xjJGFdX8XrTGDYsF4+nd74y0fwb7oq3cmY2qhlJpQYJ7HwV/6zrHX6NqjfWoKtOp7eImhIIus+AEovGxlrS0zMZNGhkp2XRvaGqj4mmKXAKOXDKWV5PksjPyMHv9cd9fbLGFQ9d12lpOUVjYy05OYlrAN9dovUJt/s41GP7aD1Rhn/ecvT2ZvTmenQ1nBXeUbYNRN6GQNAjBpRYKEpHXKFIJukeP02ShBRqTpjhTUeWPH0ylu4gSRKDBg2huflkn43Bbg3Yl5oyF68isPNV1GP7sCyMrc9h5G6Y1WwkQEIped8ZjhslakpYHQJBdAaUWAB92mjJ7/WTn5FDuxpAlmQaA02g67QorV22LvqKvrxu7jBZ03IwJ3T/rOtpPVFmLE9Z+Rm26rfIgBbyddiIkrchcjUEgugMOLHoa/xeP36vn6bAKT7dU8xfnv8TP/mv1bSrgbhiEQwGeeCB+ygp+Qy/38+9965m3LjxEfu98MIfWbfuFTRN5//8nztZuPBKADZseItnn/09iqLw1a8u48Yb/wmABx/8T4qLd5OengHAP//zv7Fw4ecS+bbPCIdj22Y52Cd0c3lKSs8KZX67qt86hELGd95CR95GxHlcVoewOAQDHSEWfUS6x290WAeQJONxHF588QXS0zN4/vk/s3v3x6xefR+/+90fHPt89tk+Nmx4k//5n7W0trbwjW/8MzNmXEwgEODJJ3/D73//HD5fGt/85r8wc+Yszj13AiUln/LYY0+Sk5OThHd65jgc23bLwTah2x3i9sKF6JpRrBCMyraSjP+yW/EML7DKiphWituBLqVnEdi1PixArv4bQjQEAwkhFl2g/FhTwuvE+71+hqVl45W9/Nfd95GdPZRDhw5w990/4ZFHfh6x/29+8yTbtm3lX//1GwBMnz6TkycbOXHiBCNHjrT2++CDbSxceCV+vx+/38+MGRezbZvRK2LmzFkMGWKM/3Of+zybNr3HqFGjqa4+wUMP/ZS6uhoWLPgc//zP/4Ysp071+qiWQyfJd/bwW3/TIQLZ5wJElgax1Z3qkDz4L7s1nE3e0WpYMKbYmAJl9t8o2+ZYphKWh+BsR4hFHMqPNfHzF3Y5etsmSjB8Hh+ekIN74sRCHnzQEIk//GFt1P3r6moZMSJ89z9iRA61tdUOsairq+W88y5w7VODJEkRr/30033U19cxc+YlfP/7d5OVlcWqVd9l/frXuO66GxLyHhOF23LoysTsyS9k2IUzqK09bT0GCOxa7+oRDugqga3PkXndf+AdPYXW1x8KV8nV1VACoOkLwWHVJMPXIcRHkGoIsYhDaUVjRG/bZDQXOf/8CwGorj7BD3/4vYjnf/ObJ9E0zeVo1pEkpwVg9Eh3P5bRNNXxWl3XkWWJMWMKeOihh63tN974Nd56642UEws7sXqDd5XoXf0wBGPnq8hDcl3l1GU8Yy8CQD1abCxn2XqKd+br6AnC0S5IRYRYxKE3etvquo4iqwSUAPn5I2NaFnl5+dTV1TFmjNHHur6+npycXMc+ubl51NXVWY8bGuoZO9bIDt+zZ5dje05OLgcOlHP06BGuuOLz5mjwes/ur4WjeGFHa6ifhiEO6rF9qFUeo9tfSBSQJdSKXSD78E1dZPXfMHuKu8uv6831qNXlXZ7g3VZEosVHIEgEqbMwnaKYvW1vWDAhoUtQJrqu06EFaQm2UN1WR0AJxNx33rz5vPXWGwDs2bObtDS/YwkKYM6cebz//kba29tpbGzkH//YwaxZlzJr1qX84x87aGxspL29nU2bNjJ79lx0XedXv/oFp06dQlEUXnvtFRYsuCKh7zEV8eQX4p+xmPTZ/0Tmdf+BZ8yFGPkYRl8Nb9HlpF1yI77zFhrLT6GJW6urAFfWOBj5Ht4pCwEIlmyidf0a1Opy63xqdTltW56hfcszEdtb16+hY+dfrNd4R08Jt6IVRREFKcLZfQuZILrSML2naKGJRwfQdZqDrbSrAdI9/ohQ2q9+dSkPPfQAt9zyT6Sl+fjJT34KQEnJpzz11BM8/PCvOP/8C1m06FpWrlyBqiqsXPlNcnPzAPi3f7uD73znGwSDCkuWfNla+rrllq9zxx3/iqIoLFx4JVdf/YWkvNdUJSJXI1S80FEJ13SoT5iFeqIsIms8c/Eq5KwRhoXi6iPeUbYNpWSzZb0ESzZTe/TzqOdcGspADznaQ6/xz1gcs/eHQNBXSLruzlQ6e6ivb0bTwm/vxIkjjBwZv2Bfb5bVCCgBqtvqIhPGopQB6atyHyaxrl9u7mDLiZxqdGdssZzK7u1qdbkza1ySSZv1FcMxvn6NISSSjG/aNQQ/edfZ7c+O7DU+d9M/InvJXHJ3n4pDqn6WYlzdp7tjk2WJESOyYj4vLIs+xp7VrWgqLcEWdEAKZ6RQJwAAIABJREFUWRmpnNV9thHLcR6tp7jbErGyyectt0Jug8VvRXb7s6OpOPp2FF0urAhByiLEIgUws7oDSoCWYAtgTCEtwRayfJkOwQgogZjLVILeI1abWL29mXBPcWe3v2gRVciysZ+tb4eJCJ8VpBJCLFIIv9fPIN8gmkOCATjKgLTblqyaulGtVpAcOi2jHqXbX0ao21/HZ5sIbHsONKPYobtlLIjwWUHqIaKhUoSAEqApcIo0jw9JkpAgogxIWzAAetgZ3q7GjpwS9B5qdTmBXeutcNnMxavwnneF4ZOQZPD4HFaD3t4c7jOua0hZIyKEIFr4rPtcscYQb4wCQU8QlkUK4HBySxLD/NlouoYsyZYg+L1+Mnx+GttDJc67WE9KkFxiWQCxuv2BYX0EPV6j14YtNFatLqejbBsSIOeMi+gpHutc8ayQzp4XS12CriLEIgVoV8MWg6TraLpGuscfXnICBvkGkZ2RZTnDhc8iNegsga4zh/mom++jztbuVa0up3Xdz0LFEgHZi3/+LY5S7FaZEte5Iqry2joGmpFb1rJYnDIl5M4QAiKIihCLFMDRFClkMdgFBKA52EKr0kpeRg7Z/iF9Ol5BGHel2q4m0KUXFOH3j7YeGxO+Et5BU9Dbmx2NnmJlirv9JGbuh+/CqxzZ6YZDPTzGaELXPjSz094hdkxRkdKzYu4Tbf9Y+wmRSm1SWixUVeXrX/86q1atYurUqX09nITz8cc7efrp3/HYY7+LajGYAmIKRrCjgwf++14O7i/vUT+Lp5/+HRs3vgsY2eB33PF/Adix40Mee+wRAoEAV155NbfffkfS3/vZQqyoqO5iTPheh2VhX54yj5+5eJVVfj342d8Ilmy2quW6OwYG9/wVe+0rz5gLHD3Kowld25F9cXuHmGNyVu6V6PDEdsSfyVKZIDVIabF44oknyMvL6+th9Apm+Kz9cX5GDs3BViuc9q3X3yAjI4Onn3menbs+4oEH/h9PPvms4zix+lmUlpawY8ff+Z//eR5Jkvj+97/N++//jTlz5vLQQz/lscd+R15ePqtWfZcPPtjG3LnOME5BbM60sKF5jMwld1s+C58tg9yaREO9NCQIi4quEtjyLL7zFrqyy8FRJFHy4J0wy3KUm2M2xUcC1IZKlOZao8IuGrF6hwDOzHPzXJ3UsYpX70rUw0p9UkYsnnrqKbZu3Wo9XrZsGZMmTULT+i5j2STZ5vGdd97OkCHZMftZaLrKj//rAXbv2MlNtyyjurWWc4rOpb6xgYpjRxg7JpxVHaufRVHReXzrW3fh8/kAGDduPNXVJ/j0032cc85YRo8eA8CiRV/kb397V4hFH2A6xu04JtFQLw0ryspK+NOM7Z60KEtPgOTBN+2acAMn1527Ypv4A0gge/BNWYicM87WdVAyenyEmkEFS7e4Ri9FXYYzfzt6RytWe9so+3VlOU8sU/UtKSMWK1euZOXKldbj733ve2RlZfHJJ59QUVHBz38e2RCoN+gt87izfhZNgVM0BU7RWN/A0OHDrfvFocOHcaz6mEMsYvWz+OIXw2vfR49WsHHjuzz++O8pKfk0So+MmoS/P0HPiJq3oWt4xk5HrdjjFAVHoUMAyVp6iuUEt7ZbhMN50867AiCUka4S3PMmILmESsIzbgaevAnWBB/YtT5mkykkD/55yyN+Q/GW89orS61jmY2qzPG56Y6oCAHqOikjFm5+8YtfAPDoo49yxRVX9Nk4ess87qyfha5r/GjN/WihXhW2rhQR4bOx+lmYHDx4gFWrvsu3vvV/OeecsXz22T7H/tF6ZAj6DvtSkdUmVvbin34tTL82Yru1FBW6Q7f7KKI5wf3zlhshuqYYSU4LIZyRbqLbWtXq4bGAbSwqHbIP3+T5kU2m0EPHNHBP1rF+W21H9oVFJ9SoyjO8IKrTvas3d8JP0j2SLhbNzc0sXbqUJ554goICow/DunXrePzxx1EUhdtuu42bb7455uu//e1vJ3uIndLTaJfu4vcbk36sfhYBJUB+Xj5Si05+Zi7taoDTJ08xKn+MY7/O+lkUF+/mnnt+yHe+8z2uuuoa2/711v5Gj4zU7MU9UOksbyPa9midBE3RcTvB1bojjpa1mZ4ggexzI53gdssmVCRRSsuMbUGoQbTWJqcQuZaqupP/kTHuAholj617oea4cbOWu5rroyYzRrMehJ+keyRVLPbs2cM999zD4cOHrW3V1dU88sgjvPzyy6SlpbF06VJmz55NYWHiPyR3BcWaGhmvt2t3zeZ+3jGT8Xz5hwSPleAbMwXvyEkJG5/HI1vd6zyezsfm9Waw4LKF/O29d5hz6Rz27y4h3Z9OQUE4/LJdCTBt1gwee+QX3HLLrbS1tfOPf+zgG9+4g/r6Gn70ox/wwAM/Y9asS63XTJs2jaNHj1BVVcno0WN49923WbLky1HHIssyubmDo44v1vZUIFXH1u1x5c6AC2fE397Jfu1DMzn+x/8HqgLoKGVbybv0atIvXBbznO1D76PtyD609haaPloHmoay711G3Xwf6QVFNG57mVZ72C8AOurRPWQWXownayj+/AmobafJGHcB6QVFANTu/MhRnt3fdIhhF86gvbKUqjfWoKsKQY/XOk/OF/6NurefNJbJPD5yzp9Jeu5gx/7IcrhxlSzjbaqg9ePXQFMdxwJoP38mVbteR1cVJI/XOl53SdXvFyR2bEkVixdffJF7772XVatWWdu2b9/OnDlzGDp0KADXXHMNb731FnfeeWfCz+8uUa5pWpdKfEeUAs+ZiC9nIkBCS4SrqoZZIV5V44/N7GexdOlNpKX5WPUf91DfcpIj+w/yh/95iu/e9x/kjR/F7IWXcdvXb0ZTNVau/CbDh+fwy18+TCDQwS9/+QvreNdf/xWuv/4mfvSje7n77n+noyPA3LnzWbDgyqhj0TQtasnjs6lMc2/RZ+Pyj8Y3+XLDIQ6gqdR9+rGV8xF1XP7RMHm0kRQYio7SVcV6nZp9bmiCNiO0NOvYrWUfgScNdfGleM6Zw2ngdO1pw6rYvTF8DkkmkH0utbWnCXz6Mbpq3PGb5ykoKCJwzhwyl+RYVsJp/2hO156m/aN30BVTdLCaUCmlW4zzhzCP5T3Zah0j40urIo7XVdTqcvxNhxyWWCrRr0qUr169OmJbTU0NubnhVqB5eXkUFxcncxgpy8yZs5g5c1aX9/f7/dxzz38C4RIhTYFTDB2Xy48fuN8qb/6lr3yZ5ctudSTvffe7P+C73/1B1OPOmnUpzzzzwhm9F0H/wTd5vrOhUxeXVmMtydqd01J6li2CKnSj5lrisbLKtXCyoL08u9HbPHrklNuvoVaXOyOzJJm00LKcs4+6cR69ozVi6cue+NhVzCW01tC1iJY3kkzHuf34EH2ZLdH0uoNb0zRr6QVMB6zUySsE0XCXCAFAEnWjBPHpSiJhtMmus9fZJ3HP8IJIx7vbT2FL5rMXWlSryw2x0VUrcgqgcdvLqLY7eIePQo8uOpaw2drlOnqMdOKniDfZR+twGJGweAaO887O7869AaygAnvGPblRliLPgF4Xi5EjR7Jz507rcW1t7YBJvEsk7hIhWb5MsnyZtKsBZEmmOdhKc7A1oh9GsjjYdIT9jQeYNGwiE7LjdyMU9C2dRR51Ntl1JQGxM4e8M1RXisgqdz6vo1QUE9j6HK26BqEMcbA71GXDT4GzJ4hd2PTmeoIlm4jsMSLHLOJo5aSEEiHtSZJWm9wYx4lVi8t8Pp4V0FnByAgnvm5roOXKuG8fep+xhJggel0s5s2bx6OPPkpDQwMZGRls2LCB+++/v7eH0e+xZ3gDdGhBq1JtY/tJKxejJdhCfmZuUgWjrO4gv9r1OxRNwSt7+c6M24Vg9GMSFSXUab8PV2ivvc4UtvpXasXusCWgBq3oprBlooEm4ztvoRGqSzjPwzy/1Ufd3WMkZIVEFHGUZFsJeSMRMli2zeiCuH2tK3PddRy31dTF6K941x+wXovkCWfZm2Hu5jUyrSxNMcKNJ/djscjPz+euu+5ixYoVBINBbrrpJqZNm9bbwzhraFFaDSd5MJx/Ye/mreNsoJQM9tWUoWgKOjqqprC/8YAQi35MMsPFoy1l2SdRbEspenN92BEPjjv4DnsYLTpS1giAqJNxrFwV0wqJKOKoa0a+if2HpCkESzZH9FOXvD48OeMI7FofvuM3noltNUUJ67UXY3Rffyk9y2mtoOGdshA5a0T4ekSxdjLGXUAiQyh6RSw2btzoeLxkyRKWLFnSG6c+qzH9FiahSHZzhRZC/yfbf3FB3mS8shdVU/DIXiYNm5jU8wmSS6KKI3Z2/M7yHcxquw6LQJbxz7/Vep3/slutXud4fHhHT4lbLj7W0lhEEUcI/a5kkKVQnxkZre5I+HnJg2/KArLHF1G34enIO35XQmT4PE4RcOendIRKs3gnz3cuibmslTRXZ0X5eAn2zH1v0eWkFxR1K7orHimbwS2Ij91vYQqF2TypI1RMLsuXCRglQ5LVA2NyzgS+M+N24bM4i0hEccSu0pUoq5zzZ3Latv6edt4VUZMP41lE9vflqOYbKuKo1R1Bqz2Ekc0O3qIFyFkjbH4PMNvkpl9+G2rZOw7/gZxzLp7ccY4lMSk9C7XuCBLgn7fc+l85WhxhqTjKqnh8eNqaHELhtlZiXUN3P/dEIMSiD7GXKO8KwWCQBx64j5KSz6wS5SPHjLKc2mbTJL/XT21tDStX3sofX/oLje0nAXhz02bW/e/LaKrKDTd+lS9ct5h0j5/iXbvjligPKAHalHYONh2JKgYTsselvEgIJ3xq0pUoq/TcwRF3yW5B645FFM1/kHH5bbbtzkm3o2ybw2owt2eMu4BGW5a6VncIrbEyikUQQvIYS1wRSYxuQhn2h3eHN8ke/LOuB/5/e2ceV1WZN/DvufeCELiAG4qkYpglZi6Ju6mpZe5bZJMRtpeZ75SvNs5Yjpajjr3aqmW7n1HHrc3UGZ1qRMXArTJERVBkUQQRCPAu5/3jcg7nXO7lXhQuN32+/uM9nOU5zznn93t+y/N79HGZmt771SKUxe+IDRv+QUBAIGvXbuTw4YMsWvQKq1d/XMVa2LdvDytWLOfixYtqsDs/7yIbPvmcJW+tJMA/gP994QUibmtPi7CWbkuUF10ppqDsEqXmUlYf+vR3GcBOK8wQQXgfprYsGU/P48pl5Sh0QR9Y9us0SM2MAvsiVlXKqNgsWNKSnNTFwh5nkR03ShjCorDlntIvVOVQsNF06wBde64mU+1aEMrCA+p6ROquRDnAO++8T0LCHqZPfxKAO+/szqVLBeTk5BAWFqbb9+uvv+C115YwbVqs+l7+fPgIt3ftgn9QADagV78+JO7Zy+1dOtOydSuCmjbEipUh9wxj565v6X5XTxqYGlBuKVcVjgxYqglgu+onXxjRnyg4JYLwApXqgvhaoatbyhZ7NV5nFXMb9BzHb5oCjvq1RfTritgtCyXl1e5uCoiZAuiD3eoER42VU5/1rISycIO3RqTVlShXyMu74KSceG4VZbFoUaWykSR7gO7SxXyahISq25uEhnIqNZX8i/k0CmlsL4MOSEEmcnKzyS3NU1fv0w6EJElSA9iKEugldaGg8De1nwySgT6t7iKmVQ8AnxjRR4V0EEH4CnxBedc3nrptPM0Mc3Y+JaaijVn4aTKwnC1HW0URXUVcpq4QysIN3hqRVleiHOyWhePsd0/KiSsCv4GxAeVSeZVjldpUikKQZRnJYM8zV5Z5VRSOhMQDHccT2bitToluz9hFr5Y9KvtJtrInaz+JOcnEhPWolf67VgEX2bitCMIj3HFaPJ1g6GkswFkMxdX+nloD7uIyUDV+UVcIZeEGb41I3ZUoB2jRoiV5eXmEh9tLvdvLiTd3uq963orlWiNaRXDwUBLBfkEAlBeVEdo0lKbNmnKpoEDd/1JBASGhdgvEYrOCsVLhSA3Kub11V0CvRC0VJrXJYMJiM6uKx2qzqNsd+68mwr+2BJwIwl/74OdGtEq8mRnmCdrJht5cj0MoCzf40oi0b99+bN/+DV273smRI4fx929QxQXlip49e/Hhh6sxlEFgYCD7/ruHWX98idbtInh32Qou5V6kcbMm7P3uv9w9fCgAxeYSSswlhAQ0oXGDRpQaKpWKokQtNjMSENGwDTGtepKYncT+7GSssgUkSd2u7b+aCv/E7GTMFcFCi83MtrSdjIwcft0JK2+M+q9l8COsEt/C2/ELsSSaB0Q2bsuIdkPq/cOYPDkWs/kKf/jDFFasWMqf/7wAgJSUY7z44vPVHtu8eQsef/wZnn/+SeLipjJs2Ai6dulG84bNmPenV/n7wtd56akZhEeEE9O/b6VbCigou0S5xe7CSivM4B8pm0jMTubuNv2QJAM2WWbjiS8BeLDTRCZ3HKu6uJTtI9oNAWBH+m4Ss5OrjG5dkVaYwb7sH9XfMpBScJKVh1aTVpjh8rjfI85G/bWNMvgZFTmixsLeG+0TeI6pdSd7aRTJ4JX4hbAs6pFrKVGupVOn21m2bGWV7Xv2JOl+Dx9+L8OH31tlv549e/HpJ+sot5Srcza09aXAPlvcbLPwfwc/wFqR3mfAgFzxT+vSKDGXIMtyFaGiDYAbJSM22ep2dHui4BQ2Wb+2Rl3Fj+rLxaJcN8gvqEaj/pq2V7u/orxrgkgSuHrq4t3yxtwKLUJZCFSU+IaCMplPKXmeb7OoigLAhr1woYSEUTKqwkMrVCTJQH7ZJZ01YZOt3NywTYWLqofLjyetMIP8sgJVsUgV13KnZLQfZvPm0VW2uUr71bpYJkWNocRcUieKI60wgz0XMmntb489Xc11q3MJObvX2nAh+ZJL9vdEXbrvvDrT3itXEfzuaOgfjL/BT82IamBqgMlgwigZVYVhkkxM7jgW/C209m+jfgCKUFHiFwlZiao1YZUtyMCZonNkleSq6bUK2lH2xhNfqpZI39Yx6r6eCn2DZGDIpb40M7VQz+VMIKcVZrAtbaeqzMw2M+uPb0GuCM57IsA9HTkq7VNG547ZYiXmEo9G/a4C1a4EU21l9f0ekgR8jdrOqKwvC1goC4FLHC0NP4OJF7o/RWJ2EiCpVoGz5RsVAWWVrXZHlWyjT+teXCzNJ6XgZBUXlaOCkCRJdWXJso3QgCY6ZeQK3YcpW/nXqf9iqIif2DO3zKxP3YIsVyqCjSe+VAPoCraKomyO+z/f7Qn1OlcTsNe2D5uFy1cuV6QmUyVbzLGftbhyCWmTAbSCqTZcSJ4IqeslW6o27+Na+17bFqi/eUtCWQhqRE1Glo4fSUwre3zm5KF0dVuppYw3Dr6rrpioCHVk+zwQyUGIOqK1RM4WZXL5SrHOggGQZZt6LjTXsNos7M1KrKIo9EhqzMRqs/CvjP/w88UUnfLwZOToGJew7w+/XDyurkNyd5t+nCg4RVZxDhtSt6oW3P7sJGZ2f1J3TmcuIcdkAAm7C1Cp53UtLiRPFKI3XHneUEa17TZy1feeKl9tW2pr3tLVIJSFoM5w9ZEo24L8glifukUVxrIs22MgMkiSgc5NO9HIv2GVkbXykZVayth19ocqAXAjRro068yxi8exylYkSWJoxEACTQEa68U+H+RsUZbuOEmyx0SU1UFsVJ5bBn7K+9WuzKgsfeJu5Oj4wd/dph+7z/6AVbapCkGWbew6+4OqNLX3ZJGdCwVHxe2YDCAjk5CVSGJOsirwahoIV/b3RCHq595UtchcKVBPy8N4K3W3Ohff1Zazcex7T+/FsS2u5i15A6EsBHWKMwGlbNuRvlsn3CQkopveBsCxi8f5Ke+YfTSliWsoH1l11oAVK+0aRdC5aSc2pG7BJst8l5mg+yAVJaUIfgno07qnOickv+wSCVmJurbJqq1Ssa2i9IlWKQb5BZGYnURidrKq5ByFaGrBSXXmfOW5DHqrSnM9k1S9UFBcVopVZZOtVSwoT+t5Kasemm1mDJKBBzqOp394jEeuFO0+7q6vfY7a6zgTogB7LmRy9mKuR6NqVwF+rSvH2f+VfZ3dq6v2VrdKZHVKxNM4RpBfkOqmtFdrkKq12Jwld9QWQlkI6o2okA74Gfyw2MyAhEEy8FPeMZ07yvFDUj6y6lCE64mCU07Po6T26o4x+BHTqqeqyNIKM0jMSdYJPi0GycDQiIGq26jEXEKQX5BT91FUSAcMkqEifgOZRdkYDUaw2TO8+rS6i4iG4Ww88aUqoCZFjeFsUSauYhZat5b2mkaM9G0dU+V8zoS7M8GcdSVTVcQ22cb61C20Dg5z6fbS/nZUmooFhyQRVFE5QOFEwSndddYd38wvF1MoLL+sE6KJ2Ukk5hxUM+vcpVw7vafiHNXKUbL3rLIVqcJ6VBIZtNaX473uSN/ttF+yrmTqBgIbU78gomEbAk0BqqXozHJwVEhBfkHsSN+tU2BKH9pkm9rWhKxEj92A85u8QAjVV3ioCUJZ1CO1sZ5F27btdPtYrVbeeGMJR44cQpZhzJhxTJkyFYAZM56koKAAk8n+2F966WU6d45m587tfPrpGiwWC5MnP8jEiVNq8S5do/0olZG8u3iFdua4Ir4VwV1qKcVRuJqMJixWvcB0TO1Vih66igkoH612f0UYKwJEQnLpPhrRbgh9Wt3Fnqz9AMjYuLtdfwIJ0o0OWweHOYxEYwC7EFAEiWNA3fGaVqyEBjShf3gMrYPD1CC5MxxHt4nZSQQG+GPAoLrfbLJNN2PeXSquoyWpCGllcqYyIlYUqOqCROZo3i/qcRISRoMJkNQ2SpqU64iG4WpyhPZ6jgH+xOwk9mb/WBl30qR+ay1Fs6YygNI3jtaGrr2ybC+i2b6L7n3MKMokoyhT189mJ1UHtBmDl68U88/UL7DKVr0y0w1SZLXtyn25cxX+cj6V/s2FsrghcbWehZZt276isLCQTz5ZR3l5OY8/Po2uXbvTseOtnD17ho0bv1KVBcCFC+d5//13WLPmM/z8/HnqqXi6d+9J+/aRXrknZyN5ZWTtzNR2FOLVBVAjG7flL3e/wIHTP+n28TTYqxV8joJ8R/punYXjzH1kxEj65bP8I2UzEQ3D8TP4qfc3qH1vQuTmLq+n4JgK3LlpJ93o253LKjHnIBabRRe3UM7rOIdlf3ayGq/RKoyUgpOcPJSuO94TN4p2cqazGMYDHcfrYlZaOoXcogruxJzKOTpnis6RWZzN/uwkrLJV56pKzE5iX1blRFSDZESboOCOlIKTpB5MU4W1ycHCGxoxULUWDBXzh8Aeg9uWtpNfC05Ue+6Th9J177XyfLQuVVuF9QnoBk3aOUbKs9Lev7OMt84tOjpZO+PqEcrCA0pPnaT0eAqBt3YisEPtT4CpzfUs2rfvQOfO0RgMBgIDA2ndOpzz53MICAgA4H/+5zkKCwsZM2YcEyc+QFLSAbp370mjRo0BGDx4KN99t8trykKhJtk6NcnI6tgssopQruk5nO3vaOFISDrhcvlKMb/kpaijZWVOiiIoOjaLrJJu7AzHVGDH0bejQNNaSI6xEu3IWauA+ra2WzCKZWcA+rpIc1bO7ehLV5Sidi6Mdka6sxjGiHZDVOtnX1YSVirn72hH4c93e4J/Ze7mp9wUdVKnIgO1riqt0JWA3q16ENOqR4Wysbs6tdaEvQKBTd3ueG6Lzcy645t1iviBjuM5W5TJ/uxk9mTtZ1/2AaZ0HM/IyOGkHkzTWS7KM1LOrShMJfstuultVVyqBsmom3jqqFwcrXDH5xIT1l19Dzx9xzxFKAs3lJ46SeayJcgWM5LJjzYvzq4ThVFb61lER3dR///TT0c4duwX5s1bwJkz6fTocRezZs3GYrEwY8YT3HxzW6fnPHasUiB5k5oK8Pqkegsnhh3pu3WC3SJ7PuFOizO3m4Iy+ta6rKo7VhndatMvlTksUSEd7JadbMUoGXVpzo5xh7TCDJ0vXZYrXUj7sn5EkvQj8xJziZq55jifRHnmShFKR4WnxEV6R3Tj1wsnVVegVqAqriot2hiUM3eiVhA7uhmVc+PErVhiLiE0IASLbL+etSJ+Mav707o5SBENw6ucW3s+m2zjp7xjOstOyf5TjnU2aHJmhQf5BfGPlE06a8NxsmttIJSFG0qPpyBb7JUdZavFbmHUgbKo7fUsDh8+yPz5c5k//680atSI6Og7iI6+Q/37qFFj2bcvgcaNm+jOKcsyBoNzH7dAT3XKLSqkQ5XZ7leT5qj1bVc3+q7u2G1pO3UWgmP6ZZBfECcKTjEpakyV2fiTosbo4g6KO65SOMvYNGrMSuWyocqM9KiQDqw8tFodUU+KGuM2DRj0Lji/DOej7Ep3TnKVGBSgxnoUJV01LoS63Zmg1yYPaJ+hs/iFq2KjyjUdU8XBbv2EBoToJqQqStZZTEb7XLUKUGtVKdZGDCIbyqsE3toJyeSHbLUgGU0E3lo3lR1rcz2L77/fzbJli3n11dfUQoVHjhzGbL5Cz569APsLbjKZaNGiJUeOHFKPzc93v0aGwD2Rjds6ne1+teeqbvTt7tiRkcN1EyFjWlWmCDsKqfmDX9C57ZwVhXRMEEBGVWLauSqKBaFTLrJMibnEbbsdS7BYbFanlplyD1r3i2MSQHUBeC1KfEe7v1aJaPtcibcoMZjqBgKugv6mimehjYF5OkdFm37uaFXV1fwLoSzcENjhFtq8OLtOYxae4sl6Fr/++gvLli3mjTfe5pZbotTtxcVFrFnzHu+++yFWq4Vvv/2Gl16aS8uWYXz44WoKCgoIDAzku+92M3v2y96+teuS2narXe35XMWDHIWUswwaZ3MOHM8H6AQqVM0mqmk1Xe1cGntsxqg7zvk+Jt31azLT2dX+rvpcyTbLupKps8TcoRxX3dwOT+fIgD52ZHCR2VdbCGXhAYEdbqlXJaEweXIsr7++kD/8YQr+/n669Sw4+Ce9AAAOH0lEQVQ++OA9li1bySefrMFqtbJw4Xz1uMcee5L+/Qdx7NjPxMc/hNVqY8KEyapbSlnnwmy2MHr0WNUlJrh+cCX03GXQVKdoHLPUHK/n6hyALhXYEa0lImGPzTzUfZzO4nGcb+MoWGtaj+lq6jdFNm5LTPPoGgeRnT2L6mIr1ZW6UWJHBsnAlI7j6B9eNXZVW0iy42yj64iLF4ux2SpvLycng7Aw9xrXZDJgsXiWbudN6rtdrvrPWSFBX8FX2+ZL7dJOrou5pebCr6bX8rTGlCIsn+/2RJV2VZ5Hn43m6QxqV22rad2punqOnrRlR/puvkrbUZHBJjEqcoTOTVfTthkMEk2bBrv8u7AsBIIbHG9moXniHvIkjdqT+TbXmh5dn3jSltqoJFwThLIQCARew1MB54mw9CXhXh/UZG5SbSCUhUAg8BreFnDXO95UmD6rLNLS0njxxReJjIwkOjqauLi4WjmvUgJaUDOu49CWwMvc6BbB7xXnM7p8gOTkZMLCwggICKBbt261ck6TyZ+SkstC8NUQWZYpKbmMyeRf300RCAT1hM9YFh988AF79uxRf//lL39h6NChBAcH8/TTT7NmzZprvkZISHMKCi5QXHyp2v0MBgM2m+9lQ9Vnu0wmf0JCxGQ9geBGxWeUxWOPPcZjjz2m/t66dSt9+vTB399fVyX1WjAaTTRr1srtfr6U1qjFV9slEAiuf3xGWTgSGRnJ4sWLCQ4OZsoU76yvIBAIBALn1LmyKC4uJjY2lvfee482bew1jb766iveffddLBYLjzzyCA899FCV4+644w7eeOONum6eQCAQCDygTmdwHzlyhHnz5nH69Gm2b99OmzZtyM3N5cEHH2Tz5s34+/sTGxvL8uXLueWW+i+nIRAIBALn1Gk21IYNG5g/fz4tWrRQt+3du5fevXvTpEkTbrrpJkaMGMH27dvrshkCgUAguEbq1A21aNGiKtvOnz9Pc01VyxYtWnD06NG6bIZAIBAIrhGvz7NwXMBHTJITCAQC38fryiIsLIwLFy6ovy9cuKBzUwkEAoHA9/C6sujbty/79u0jPz+f0tJSdu7cycCBA73dDIFAIBDUAK/Ps2jZsiWzZs1i2rRpmM1mJk2axB133OH+QIFAIBDUG9f14kcCgUAgqB18tpCgQCAQCHwHoSwEAoFA4BahLAQCgUDgFp8tJFgfeFKzylu89dZbfPvttwAMGjSI2bNnM3fuXJKTkwkMDATgueeeY9iwYV5v28MPP0x+fr5aDXjBggWcOXOmXvvun//8J59//rn6OzMzk7Fjx1JaWlpvfeZYF23v3r28/vrrlJeXc9999zFr1iwAfv31V/70pz9RUlJCz549efXVV2ut0rIn7Vq/fj2fffYZkiQRHR3Nq6++ir+/P2+99RabNm2iUaNGAEyZMqVOn6tju1y97676sS7Rtu3UqVMsX75c/Vtubi5du3Zl1apVXu0zZzKiTt8xWSDLsizn5OTIgwcPlgsKCuSSkhJ59OjR8okTJ+qlLQkJCfIDDzwgl5eXy1euXJGnTZsm79y5Ux41apScm5tbL21SsNlscv/+/WWz2axu86W+k2VZTk1NlYcNGyZfvHix3vrs8OHD8qhRo+TOnTvLZ8+elUtLS+VBgwbJZ86ckc1msxwfHy9/9913sizL8v333y8fOnRIlmVZnjt3rrx27VqvtSstLU0eNmyYXFRUJNtsNnn27NnyRx99JMuyLD/55JPywYMH66wt1bVLlmWnz666fvRm2xTOnz8vDx06VD59+rQsy97rM2cy4quvvqrTd0y4oSrwpZpVzZs3Z86cOfj7++Pn50eHDh3IysoiKyuLl19+mdGjR7Ny5cp6WQgpLS0NgPj4eMaMGcPnn3/uU30H8MorrzBr1iwCAwPrrc8c66IdPXqUtm3bEhERgclkYvTo0Wzfvp1z585RVlbGnXfeCcCECRPqtO8c2+Xv78/8+fMJDg5GkiQ6duxIVlYWAD///DOrVq1i9OjRLFiwgPLycq+1q7S01Omzc9WPdYmzGncKS5YsITY2lnbt2gHe6zNnMiI9Pb1O3zGhLCpwVrMqNze3XtoSFRWlPtj09HS+/fZbBgwYQO/evXnttdfYsGEDSUlJbNy40ettu3z5Mn369OHtt9/m448/Zt26dWRlZflM3+3du5eysjLuu+8+8vLy6q3PFi1aRM+ePdXfrt4vx+3Nmzev075zbFd4eDj9+vUDID8/n7Vr1zJ06FBKSkq47bbbeOmll9iyZQuXL1/mnXfe8Vq7XD27+vhOHdumkJ6ezoEDB5g2bRqAV/vMmYyQJKlO3zGhLCrwxZpVJ06cID4+ntmzZxMZGcnbb79NixYtCAwM5OGHH+b777/3epu6devGkiVLaNiwIaGhoUyaNImVK1f6TN+tW7eORx99FICIiAif6DNw/X75ynuXm5vLI488wsSJE4mJiSEoKIj333+fDh06YDKZiI+P92rfuXp2vtJfAOvXr2fq1Kn4+9vXpq+PPtPKiIiIiDp9x4SyqMDXalYlJycTFxfHH//4R8aPH8/x48fZsWOH+ndZlus0COqKpKQk9u3bp2tHeHi4T/TdlStX+PHHHxkyZAiAz/QZuH6/HLfn5eV5ve9OnTpFbGws48eP59lnnwUgKytLZ4V5u+9cPTtf+k537drFyJEj1d/e7jNHGVHX75hQFhX4Us2q7Oxsnn32WZYtW8b9998P2F+81157jcLCQsxmM+vXr6+XTKiioiKWLFlCeXk5xcXFbNmyhaVLl/pE3x0/fpx27dpx0003Ab7TZwBdu3bl9OnTZGRkYLVa+frrrxk4cCDh4eE0aNCA5ORkAL744guv9l1xcTHTp09n5syZxMfHq9sDAgJYunQpZ8+eRZZl1q5d69W+c/XsXPWjt8nPz6esrIyIiAh1mzf7zJmMqOt3TKTOVuBLNavWrFlDeXk5ixcvVrfFxsbyxBNP8OCDD2KxWBg+fDijRo3yetsGDx7MkSNHGDduHDabjalTp9KjRw+f6LuzZ88SFham/u7UqZNP9BlAgwYNWLx4MTNmzKC8vJxBgwZx7733ArBs2TLmzZtHcXExnTt3Vn3g3mDjxo3k5eXx0Ucf8dFHHwEwZMgQZs6cyYIFC3j66acxm810795dde95g+qenat+9CaZmZm6dw0gNDTUa33mSkbU5TsmakMJBAKBwC3CDSUQCAQCtwhlIRAIBAK3CGUhEAgEArcIZSEQCAQCtwhlIRAIBAK3iNRZwXXHwoUL+fHHHwH7hLPw8HACAgIA+6xb5f/u2LVrF/v27WPevHku98nNzWXmzJmsW7fu2hsOzJkzh4SEBEJDQ3XbJ06cWOsptXPmzCEqKorp06fX6nkF1ycidVZwXTNkyBBWrFhBly5d6rspHuFNAS6UhaAmCMtCcMMRHR3N0KFDSUlJYdmyZRw/fpz169djNpspLCzk8ccfZ+rUqWzevJkdO3awatUqHn74Ye68804OHjxIdnY2ffr04a9//StZWVmMHj2aQ4cO8eabb3Lu3DkuXLjAuXPnaNmyJUuXLqVFixYcPXqUV155BbPZzM0330xWVhZz5swhJiamRm0fMmQI999/PwkJCRQVFfHoo48ydepUAHVdCoPBQLNmzfjzn/9M+/btKSkpYeHChRw8eBCj0cg999yjrnNw6NAhYmNjycvLIyoqir///e/qDHiBQItQFoIbDrPZzODBg1mxYoUqSFevXk1ISAiHDx/WCWAtZ86c4bPPPuO3337jvvvu48CBA7Rp00a3T1JSElu3biU4OJinnnqKdevW8cwzzzBjxgwWLFjAoEGD2L9/P3FxcS7b9/HHH/Pll1/qti1ZsoRbb70VgMLCQjZt2kRubi7jxo2jR48e5Ofn88EHH7B+/XpCQ0PZvHkzzz77LN988w0rV66kvLycbdu2YbVaiY+P58CBA4Ddjfbpp5/i7+/P5MmT2blzJ+PGjbvGHhZcjwhlIbghUUpOBwUF8d577/H999+Tnp5OSkoKv/32m9NjBg8ejMFgIDg4mLZt21JYWFhFWfTq1Yvg4GAAbr/9dgoLC0lNTQXsq5kB9O7dm6ioKJdti4uLq9Y1NHXqVCRJIiwsjAEDBpCQkEBeXh4jR45UYx0TJkxg0aJFZGZmsnfvXubOnYvRaMRoNKorCm7ZsoV77rlHXYkuKiqK/Px8t30nuDER2VCCGxLF1ZKTk8O4ceM4d+4cPXr04IUXXnB5jDYwLkkSzsJ9zvYxGo1V9jUajVfddm0lU5vNhsFgcLqokyzLWCwWTCaTriR1dnY2BQUFVc7l6p4EAhDKQnCD8/PPPxMaGsozzzxD//79+c9//gOA1WqttWt06NABf39/fvjhB8C+al5qaupVr8OwdetWwF4SOyEhgYEDBzJgwAC2bdumWgabNm2iSZMmtG3blj59+rBlyxZsNhtXrlzh+eefV7PFBAJPEW4owQ1Nv3792LhxI/feey+SJNGrVy9CQ0PJyMiotWuYTCbefPNN5s+fz/Lly2nXrh3NmjVzmcLrLGbRtWtXFixYANgrnk6YMIGysjLmzZtHZGQkkZGRxMXF8cgjj2Cz2QgNDWXVqlUYDAaee+45Fi1axNixY7FarYwcOZLhw4eze/fuWrtHwfWPSJ0VCLzA3/72N6ZPn06zZs3Izs5m7Nix/Pvf/6ZRo0Y1Os/vLRVYcP0gLAuBwAuEh4cTFxeHyWRClmUWLlxYY0UhENQnwrIQCAQCgVtEgFsgEAgEbhHKQiAQCARuEcpCIBAIBG4RykIgEAgEbhHKQiAQCARuEcpCIBAIBG75f8i3ibexhcM4AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "net.train()\n", "loader = torch.utils.data.DataLoader(xy_train, batch_size=200, shuffle=True)\n", "for learning_rate in (0.005, 0.02, 0.1, 0.25):\n", " optimizer = torch.optim.SGD(net.parameters(), lr=learning_rate, momentum=0.9)\n", " net.load_state_dict(torch.load('net.pth'))\n", " losses = []\n", " for epoch in range(200):\n", " for x_batch, y_batch in loader:\n", " y_pred = net(x_batch)\n", " optimizer.zero_grad()\n", " loss = loss_fn(y_pred, y_batch)\n", " loss.backward()\n", " optimizer.step()\n", " losses.append(loss.data)\n", " plt.plot(losses, '.', label=f'lr={learning_rate:.3f}')\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note how the learning progress depends very non-linearly on the learning rate: the best (0.10) and worst (0.25) learning rates are relatively similar! You should always study your loss curve and look for the features demonstrated in this example:\n", " - $\\eta = 0.005$: Learning is slow and steady and appears ~linear on this log scale (after some initial rapid progress). This type of curve generally means you can safely increase your learning rate by a factor of ~10.\n", " - $\\eta = 0.02$: Learning is proceeding well but slowing down after 200 epochs. If this is the best loss curve you can achieve after 200 epochs, just keep going for more epochs.\n", " - $\\eta = 0.1$: Most of the learning happened in the first ~50 epochs, then essentially no more progress. If the plateau level gives acceptable performance (on both the TRAIN and TEST samples), then you only need to train for 50 epochs in future.\n", " - $\\eta = 0.25$: After some initial rapid progress, the network gets \"stuck\" in a bad state where it can no longer learn with additional training epochs. This is a clear sign that your learning rate is too large.\n", " \n", "Note that we are focusing here on the TRAIN loss curve since learning the training data is a necessary (but not sufficient) condition. However, remember to check that your final chosen learning rate has a similar TEST loss curve, to ensure good generalization.\n", "\n", "---\n", "\n", "There have been some important new developments in optimizing learning rates. A good starting point is these papers:\n", " - [Smith 2018 \"A disciplined approach to neural network hyper-parameters\"](https://arxiv.org/pdf/1803.09820.pdf)\n", " - [Smith 2015 \"Cyclical Learning Rates for Training Neural Networks\"](https://arxiv.org/abs/1506.01186)\n", "\n", "The first idea is that you do not need to repeat a full training loop to study each possible learning rate. Instead, you can often get away with increasing your learning rate on the fly, after each minibatch update to your parameters, using logarithmic steps.\n", "\n", "This technique is not (yet) implemented in PyTorch, but is implemented in the useful [fastai library](https://docs.fast.ai/) and also as `mls.torch.lr_scan`. To run a study, we provide a data loader, a model, a loss function, and the optimizer to use:" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Recommended lr=0.0588.\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEQCAYAAACugzM1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcHklEQVR4nO3dfXBU5aHH8d9uAkJIJIgJ0aClGhWkUGk7I6AX8KXXOpIiCm2QW0IplSnDWCmFySAILaC8pDCOZQTHYdoChaIFR2DKMEijvIl0agX0ouBtAGkhCYHwkmiyu+f+gbvdbHZPdje7e87Z/X7+Yhc2++Rhz/7O8+4yDMMQAAARuK0uAADA3ggKAIApggIAYIqgAACYIigAAKYICgCAKYICAGAq2+oCJMOFC1fl88W3PKRnz1ydP38lwSVKX9RXbKiv2FBfsYm3vtxul3r06Bbx79MyKHw+I+6g8L8e0aO+YkN9xYb6ik0y6ouuJwCAKYICAGCKoAAAmCIoAACmCAoAgCmCAgBgiqAAAJgiKAAApggKAIApggIAYIqgAACYIigAAKYICgCAKYICAGCKoAAAmCIoAACmCAoAgCmCAgBgiqAAAJgiKAAApggKAIApggIAYIqgAACYIigAAKYICgCAKYICAGCKoAAAmCIoAACmCAoAgCmCAgBgiqAAAJgiKAAApggKAIApggIAYIqgAACYIigAAKYICgCAKYICAGCKoAAAmCIoAACmCAoAgCmCAgBgiqAAAJgiKAAApggKAIApggIAUuzEmQZtP1CtE2carC5KVLKtLgAAZJITZxq0bMMH8nh9ys5ya+a4QSop7m51sUzRogCAFPrk1AV5vD4ZhuT1+vTJqQtWF6ldBAUApNBdt/ZQdpZbbpeUleXWXbf2sLpI7aLrCQBSqKS4u2aOG6RPTl3QXbf2sH23k0RQAEDKlRR3d0RA+NH1BABJ4LSZTWZoUQBAgjlxZpMZWhQAkGBOnNlkhqAAgARz4swmM3Q9AUCCOXFmkxmCAgCSwGkzm8zQ9QQAMEVQAABMERQAAFMEBQDAlG0Hs+vr67Vo0SLl5ORo+PDhevjhh60uEgBkJNu2KNauXavy8nItWLBAmzZtsro4AJCxbBsUdXV1KioqsroYAJDxbBsURUVFqq2ttboYAJDxbDtGMXbsWC1dulSdOnVSWVmZ1cUBgIyV8qC4cuWKysrKtGrVKvXu3VuStHXrVr3yyivyeDwqLy/X+PHjVVhYqMrKylQXDwAQIqVB8eGHH2rOnDmqrq4OPHfu3DmtWLFCmzdvVufOnVVWVqZ7771XJSUlcb9Pz565HSpnQUFeh16faaiv2FBfsaG+YpOM+kppUGzatEnz5s3TrFmzAs/t379fgwcPVn5+viTpkUce0Y4dOzRt2rS43+f8+Svy+Yy4XltQkKfa2stxv3emob5iQ33FhvqKTbz15Xa7TG+wUxoUixYtavNcTU2NCgoKAo8LCwt1+PDhVBYLAGDC8llPPp9PLpcr8NgwjFaPAQDWsjwoQqfB1tbWqrCw0JKynDjToNff/jQtzrgFgESxPCiGDh2qAwcOqL6+Xk1NTdq5c6eGDRuW8nL4z7hd95f/1bINH0QVFul0eDoARGL5OopevXpp+vTpmjBhglpaWjRmzBgNHDgw5eUIPuNWXp/2H/l34HQq/98Hn1Rldnj6iTMNpq8FACexJCh2797d6nFpaalKS0utKEqA/4xbr9cnl9ulvUf+La/PkNvtkkuS12coO8utcQ/foatNLTrf8EWrw9P9wdKtaydt2HVcHq+vzWsJE8C5gq/ZTLtOLW9R2IX/jNvPzzfq1L8a9M6H/5JhSD6vIf9EW4/Hp/U7P5XPuBYgWW6XfD6jVbC4XC4ZvmuvCX6t1+vTJ6cuqKS4e6vWiFmYALAHsx6ETEBQBCkp7q4h9/TWgX98rn1HzwZaFy7p2roM17VgMCQZPkP/9c2b1bN7F51v+CIQLPoqRAzDaPXarCy3unXtpO0Hqlu1RkLDJLjLK5M+iICdBXdNB9/0ZQqCIgx/6yK0a8jfreT1+pSV5dbQATcFWgj+YMkK6p4K91p/KyK4NeIPk+CWif+uxf96ggNIrFi6koK7prOy3IFrO9XlsApBEUFJcfdW/2n+P/cuyG3znxoaLKH/2SXF3bX9QHXgjiS4NRIcJsEtE3/rYt/Rs4HmbnAA2fUDBThBrF1J7V3jiSqHXa9xgiJGoQHS3vN+oXck/tZI8OtDWyaSAuESPD6SiX2kQCLF05XU3jXe0XLY+RonKFIkmjuScF1e/uAIHh/JxD5SIJGS2ZUUbznsfI27DMOIb/c8G0unTQH9/Zeh4yPRNFFT0fdpt/qyO+orNsmsL7uMDUS6xiNNpw8ua+jzabEpIGIX3Nz1j48ED4xH6tfM9Ol8QHuS0ZXU0XKEGwONdC2Hez5ZW7ITFA7i/0AFD4xH6tcM7YNl2i1gf+HCK9J4Srjnh9zTOynlIigcyKxfM3iFeLiV5rQuAGeJNJ6SynEWxihCOKUPOVy/pstkuxH/tFu3Sxo97DY9NqRPQsrhlPqyC+orNtTXNZG2/An+M2MUaCNcv2boOoyrTS16bEifNtNu/SvE6YYCnMF/vYcbl0jUTZ8ZgiINBH+IggPBf8cRPO02dCCcbijAnsLNdLJqKxGCIo2YrdUINxBut7naAK6JNNPJqvUfBEWaiXWFuFULjQBEFqnlkKytRNpDUGQYsw+aXRYgAXZh1TVhdkNnxfoPgiIDBX/QQmdPMXYBXGPlolWrWg6REBQZLPhCCD5wiQV6gPVnUNhl5bhEUGS0VueEhxy4FLxAz65bHwPJxHjefxAUGSz0Qgi3QM/OWx8DiRJuLMJu3T9WIigyWKQLIXg9hp23PgZiFS4QzMYi7NT9YyV3NP+orq5Ob7/9tiRp2bJlKi8v17Fjx5JaMKRGSXF3PTakT5s1FzPHDdLoYbfpf/77TmVnu+V2qdWq7hNnGqwrNBAHfyBsfvf/tGzDB4HPcLixCLQWVYuioqJC999/vw4cOKA9e/Zo4sSJWrhwodatW5fs8sEi0WxvnsxtjYFEizQ4zVhE+6JqUVy8eFETJ07Uu+++q5EjR+qJJ55QU1NTsssGm/C3Oq42tbS50I5V19PCgCP4A8HfOg7d4mb0sNs6PAZ34kxDWl4PUbUoWlpa1NLSoj179mjx4sVqampSY2NjsssGmwm98+rWtZOeW7VPLR7WXsD+otnipiPS+bCwqILioYce0pAhQ9SvXz994xvf0MiRIzVy5Mhklw02E3qhfXLqgjweDkeCcyRzcNrqdRfJFFVQPPPMM/rBD36gXr16SZIqKyvVt2/fpBYM9hR6oWVnu+XxcDgSYDbW4fTtcbLmz58/v71/VFdXpyNHjui2227TsmXLtHnzZvXr10833nhjCooYu6amZsV7HFO3btepsbE5sQVKUzdc30VDvlmsvC7ZKujeVf88ezmweK8gv6vuvCVf0rWL5MBHZ+V2u3TD9V2sLbTF+HzFxkn1dcP1XdTvaz1UkN9V37//622m335UXa/3Pjqnfl/rkbTrIN76crlcysnpHPHvmfWEDunb5wb17NYp4uFI7CGFTBLLmddOElVQ+Gc9LVmyJDDraf369ckuGxwk0uFIoXtIOfEiAToiHabfMusJCRPucKTgPaScepEAHZEOW4Ew6wkJF2kPKadeJEBHOX0rEJdhRDfse/bsWRUVFUmSjh07ZutZT+fPX5HPF99odkFBnmprLye4ROkrUn05fZZHsvD5ig31FZt468vtdqlnz9yIfx9Vi8Ln82nr1q1699135fF4dN9996mkpETZ2ewpiPCcfgcF4D+i2sLjN7/5jd577z2Vl5frxz/+sT744AMtXbo02WVDGgre4iBdtzsA0k1UTYI9e/boz3/+szp16iRJGjFihL7//e9r9uzZSS0c0kvwFgdut0suiQV6gANE1aIwDCMQEpLUuXPnVo+BaATPJ/d5DXm8Bls7Aw4QVVD07dtXL7zwgk6dOqXTp0/rxRdf1J133pnssiHNBO/e6c5yKTvLxTkXgANE1fU0b948LVy4UGVlZTIMQ/fdd5/GjRuX7LIhzYTOJ5cU8ZwLuqGQKMzA67iogiI3N1eLFy9u9dy3vvUt/f3vf09KoZC+QmdDhS7QYxdaJFI6b/2dSnHPb41y+QXQruAFeqG70LJYDx2Rin2WMqHFEndQuFyuRJYDGSy4S+p8wxd658N/yTAkj8en9Ts/lc8gNBCfePZZiuWLP1NaLKyYgy34u6SCd6GVyyXfVxsKhoZGul6QSKxY91mK9Ys/HXaGjYZpUAwaNChsy8EwDH3xxRdJKxQyV7hdaENDI50vSCReLLsExPrFnw47w0bDNCi2bduWqnIAAcEXdu+C3Dahkc4XJKwV6xd/OuwMG42oNwV0EjYFTJ1U1ldw37EkR16cfL5iY0V9OXlw2tJNAQE7CB7HyIQBRFiDDS3bimplNmAn4fqRASQPQQHHCd4KhPEKIPnoeoLjhNsKZPuBakf2KQNOQFDAkSKNV7AoD0g8ggKOFjxewaI8IDkYo4CjBY9XuNxfLcpjkBtIKFoUcLRIK7lTNcidDms7gPYQFHC8cCu5U/FlbXa0K2MlSCcEBdJKKhdLhR7t6t8LgLESpBvGKJC2TpxpSOrxqpGOdmWsBOmGFgXSUjK3+QgelzA72pUNDJEuCAqkpUScExBuc7hwAfTYkD6B11gxVgIkG0GBtBRuu+hEnFwWbQCxsRzSCUGBtBRum49EnFzW0YNqjlXX673DZ2hpwFEICqSt4Lv67QeqE3JyWUcOqjlxpkGVGz9Qi4ft0eEstg6Kjz/+WEuXLtXvfvc7q4sCh+voyWVS640HYw2IT05d0PmGL+TxxD5u4uSDdJAebBsUp0+fVlVVlbKysqwuCtKAWUsg0hdxIg5KCl2Ul5XllmLotuKQJtiBbYLitdde0969ewOP16xZo6lTp2rKlCkWlgrpJLgl4A8H/1RWsy/ijsygCn6t4TP08OBbldM5K+rt0RMxewvoKNsExeTJkzV58mSri4EMEHyX7nK5ZPiuraqO9EXckQHs0Nc++J1b1bNbp7AtBantXlEdHTyn2wqJYJugAFIl+C5dhiG32yXDMCJ+EXdkADv0tX373KDa2sttWgr7j/xb+46ebdOy6ejgOd1WSISkB8WVK1dUVlamVatWqXfv3pKkrVu36pVXXpHH41F5ebnGjx8f8fWrV69OdhGRYULv0qPZwK8j6yLCvTa0DJIidjHF+950WyFRkhoUH374oebMmaPq6urAc+fOndOKFSu0efNmde7cWWVlZbr33ntVUlKSsPft2TO3Q68vKMhLUEkyg9Pqq6AgTy/k5+jIZ3UacPuN6tvnhpS/f2gZJGn/R2fl8fiUne3W4IHFHa7XwQOLtXV/deBnFhXkqerwvy35nTvCaZ8vqyWjvlyGYRjt/7P4PPfccxo9erRmzZqlP/zhD+rdu7e2bNmiQ4cO6YUXXpAkrVy5UoZhaNq0aQl73/Pnr8jni+/XKijIU23t5YSVJd1RX7Exq69EjSeEOyMjdNDeKdug8/mKTbz15Xa7TG+wk9qiWLRoUZvnampqVFBQEHhcWFiow4cPJ7MYgCMkYtuPSHtRBS84ZBv0jsu0SQIpH8z2+a7NNPEzDKPVYwCxa7Wor52tR+T6aht0MXYRj0ycJJDyoCgqKtLf/va3wOPa2loVFhamuhhAWHa5U4x3A0O326Wsr87DiLT1CNugd0wmThJIeVAMHTpUL7/8surr69W1a1ft3LlTCxYsSHUxgDbscqcYbTnCtSIMn6H/+ubN6tm9S8SV5hLboHdER9e2OFHKg6JXr16aPn26JkyYoJaWFo0ZM0YDBw5MdTGANuxypxhNOcxaEUMH3NRuucOtUic0otORtS1OlZKg2L17d6vHpaWlKi0tTcVbA1Gzy51iNOUI3RokUiuiPXZpRTlNpp03wsps4Ct2uVOMZgPDbl07tQqTaFoR4Zi1XmhpwI+gAILY5U4xXDlC7/4TsRYiUuuFlgaCERSAQ4Te/V9taml1Xnc8IrVe7DJeEy9aQ4lFUAAOkawxlEjbr8fzXnb4gqY1lHgEBeAQyR5DibdrK5azPVLB6a0hOyIoAAdJ5hhKPF1bsZ7tkQp2mb2WTggKAJLi+4KN9WyPVLDL7LV0QlAAkBTfF2w8Z3ukgl1mr6ULggJAgNkXbLiBau7eMwNBAaBdZjOJuHtPf26rCwDA/sLNJLKTE2catP1AtU6cabC6KGmJFgWAdsUz0J2qNRWsm0g+ggJAu2Idi0jGdiORsG4i+QgKAFGJZWvy4C/vZB+9yrqJ5CMoAMQkmq6eaI9eTUT3FDOvko+gABCTaLp6ojl6NZFjC8y8Si6CAkBMou3qae/o1WjOwhg8sFg9u3VKzS+GiAgKADGJp6sn3B1/NGdhbN1frV+WMYvJagQFgJgloqsnmrMwPB5mMdkBQQHAMu21NLKzmcVkBwQFAFsJbmkwRmEPBAUA2/G3NAoK8lRbe9nq4mQ89noCAJgiKADYAhv72RddTwAsZ7b4LlWbCyIyggJASoX74o+0+O5YdT07w9oAQQEgZSK1HCItvjvyWR07w9oAQQEgZSK1HCItvhtw+43sDGsDBAWAlDHbJyrc4ru+fW6IuF0IYxepQ1AASJlE7RPFqXapRVAASKlE7BPFqXapxToKAI7j78Jyu8TYRQrQogDgOJxql1oEBQBH4lS71KHrCQBgiqAAAJgiKAAApggKAIApggKAY7AVuTWY9QTAEViNbR1aFAAcIdxqbKQGQQHAEViNbR26ngA4AquxrUNQAHAMVmNbg64nAIApggIAYIqgAACYIigAAKYICgCAKYICAGCKoAAAmCIoAACmCAoAgCmCAgBgiqAAAJgiKAAApggKAIApggIAYIqgAACYIigAAKYICgCAKYICAGAqLY9Cdbtdlr4+01BfsaG+YkN9xSae+mrvNS7DMIx4CwQASH90PQEATBEUAABTBAUAwBRBAQAwRVAAAEwRFAAAUwQFAMAUQQEAMEVQAABMERQAAFMERZS8Xq9+9KMf6ciRI1YXxfaOHz+uZ555RhUVFdq3b5/VxbG9Q4cOadasWZo5c6Zef/11q4vjGB9//LEmTpxodTFsrb6+XjNmzNDcuXO1a9euuH8OQRGlVatWqbCw0OpiOEJjY6Nmz56tGTNmaNu2bVYXx/YuXbqkX//611qyZInefvttq4vjCKdPn1ZVVZWysrKsLoqtrV27VuXl5VqwYIE2bdoU989Jy91jO+q1117T3r17A4/HjRunO+64Qz6fz8JS2Vdofa1Zs0anTp1SRUWFJkyYYGHJ7ClcfRmGocrKSuorgnB1NnXqVE2ZMsXCUtlfXV2dioqKOvxz2D02Cr/4xS+Um5uro0eP6vbbb9eyZcusLpKtHT16VH369FFubq4mTZqkNWvWWF0kW7t06ZJefPFFPfXUUxowYIDVxXGUKVOmaPXq1VYXw7ZWrlypESNGqH///nr66af16quvxvVzaFFEYfny5ZKkl19+WSNGjLC2MA7w5Zdf6rnnnlNubq6GDx9udXFsb+HChTp79qx+//vf66abbtKMGTOsLhLSxNixY7V06VJ16tRJZWVl8f8gI4NcvnzZeOyxx4zTp08HnnvrrbeMRx991Pjud79rrFu3zsLS2Q/1FRvqK3bUWXxSXW8ZExT/+Mc/jJEjRxr9+/cPVO7Zs2eNBx54wLhw4YJx9epVo7S01Dh+/LjFJbUH6is21FfsqLP4WFFvGTPradOmTZo3b16rmUv79+/X4MGDlZ+fr5ycHD3yyCPasWOHhaW0D+orNtRX7Kiz+FhRbxkzRrFo0aI2z9XU1KigoCDwuLCwUIcPH05lsWyL+ooN9RU76iw+VtRbxrQowvH5fHK5/nOouGEYrR6jNeorNtRX7Kiz+CS73jI6KIqKilRbWxt4XFtby6I6E9RXbKiv2FFn8Ul2vWV0UAwdOlQHDhxQfX29mpqatHPnTg0bNszqYtkW9RUb6it21Fl8kl1vGTNGEU6vXr00ffp0TZgwQS0tLRozZowGDhxodbFsi/qKDfUVO+osPsmuN1ZmAwBMZXTXEwCgfQQFAMAUQQEAMEVQAABMERQAAFMEBQDAFEGBtPf5559r0KBBlrz3Sy+9pDfffDPl71tVVaWXXnop5e+L9JTRC+6AZPv5z39uyfseOXJEDQ0Nlrw30g9BgYzW3NysyspKHTp0SF6vV3fffbfmzJmj3Nxc/fWvf9Xq1avV3Nys+vp6Pf7443r22Wd18OBBLVq0SDk5Obp69apmzZqllStX6pZbbtHx48fl8Xj0q1/9St/+9rdVUVGhO+64Qz/5yU80YMAAPf3009q3b59qamo0efJkPfXUU/J6vVq6dKl2796tvLw8DRw4UJ999pnWrl3bqqybN2/WG2+8oaamJuXm5mr16tWaP3++Tp48qYsXL6pbt26qrKzU5cuXtXHjRnm9XuXl5Wn69Ol6/fXXtWHDBvl8PuXn52vu3Lm6/fbbLap1OA1BgYz26quvKisrS5s3b5bL5dLy5ctVWVmpefPmac2aNVq8eLH69Omjc+fO6YEHHtCECRMkScePH9euXbtUXFysgwcP6vDhw5o3b5769eunNWvWaMWKFVq3bl2r92publaPHj20ceNGHT16VOPGjdOTTz6pLVu26KOPPtK2bdvkcrn0s5/9LGJ5T5w4od27dys3N1c7duzQ9ddfrz/96U+SpOeff17r16/X3LlzVVZWpgsXLmj69Ol6//339eabb2r9+vXq2rWr9u7dq2nTpukvf/lL8ioWaYWgQEarqqrS5cuXtX//fklSS0uLevbsKZfLpVWrVqmqqkrbtm3TZ599JsMw1NTUJEm66aabVFxcHPg5N998s/r16ydJuvvuu7Vly5aw7/fQQw9Jkvr376/m5mY1NjbqnXfe0ahRo3TddddJkn74wx+2aU343XXXXcrNzZUkfe9739Mtt9yitWvX6uTJk3r//ffDjsVUVVXp5MmTrc5MvnTpki5evKj8/PyY6guZiaBARvP5fJo9e7aGDx8uSbp69aq+/PJLNTY2avTo0Xr44Yf1ne98R08++aR27dol/9ZoOTk5rX5Oly5dAn92uVyKtIWaPwz8ZwUYhqHs7NaXodsdeY5J8Pv+8Y9/1KZNmzR+/HiVlpYqPz9fn3/+edjfcdSoUZo5c2bgcU1Njbp37x7xfYBgzHpCRrv//vu1fv16NTc3y+fzae7cuVq+fLlOnjypK1eu6Nlnn9WDDz6ogwcPBv5Nog0fPlxvvfWWmpub5fF4IrZGQu3du1ejR4/W2LFj9fWvf127d++W1+uVJGVlZcnj8QR+x+3bt6umpkaStGHDBpWXlyf890D6okWBjNDY2NimW2bjxo2aOnWqlixZotGjR8vr9apfv36qqKhQTk6ORowYoUcffVSdO3fWnXfeqZKSEp08eVKdO3dOaNmeeOIJ/fOf/9Tjjz+unJwc9e7dW127dm33dZMmTdLzzz+vN954Q5J0zz336NNPP5UkDR48WL/85S+1YMECzZ07Vz/96U81adIkuVwu5ebm6re//S0nxyFqbDMOWGzv3r06f/68Ro0aJUlauHChrrvuukBXEWA1ggKw2Llz51RRUaG6ujr5fD717dtX8+fPV15entVFAyQRFACAdjCYDQAwRVAAAEwRFAAAUwQFAMAUQQEAMEVQAABM/T9V/VRrbL5JswAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "loader = torch.utils.data.DataLoader(xy_train, batch_size=200, shuffle=True)\n", "net.load_state_dict(torch.load('net.pth'))\n", "optimizer = torch.optim.SGD(net.parameters(), lr=0.1, momentum=0.9)\n", "mls.torch.lr_scan(loader, net, loss_fn, optimizer, lr_start=1e-4, lr_stop=1.)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The scan results are noisy, but clearly show an optimum learning rate somewhere near 0.1. Note that approach allows us to scan a wide range of learning rates with much less computation than the brute force study above.\n", "\n", "---\n", "\n", "The next technique is adjust the learning rate each epoch according to some predetermined \"schedule\". Cyclic learning rates have been associated with \"superconvergence\", i.e., must faster learning than a fixed learning rate, but your mileage may vary.\n", "\n", "Compare some different schedules that we can easily implement using [torch.optim.lr_scheduler](https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate), including a custom [1-cycle policy](https://medium.com/@nachiket.tanksale/finding-good-learning-rate-and-the-one-cycle-policy-7159fe1db5d6) schedule:" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEJCAYAAACKWmBmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydd5xU1fn/3/fe6dsrTQTpUUEErBCxE0AsxIqKKLYYY2+JDdtXjRIsicbYSzTWWCIi6jdGUSygxhJAAUFgly0zW2an7Mwtvz/u3NlZtszcO418f/N5vXgBc8+558yZc87Tn0fQNE2jgAIKKKCAAvqAmO8JFFBAAQUUsHOjQCgKKKCAAgroFwVCUUABBRRQQL8oEIoCCiiggAL6RYFQFFBAAQUU0C8KhKKAAgoooIB+USAUBRRQQAEF9AtbvieQDbS0BFBVa+EhVVXFeL0dGZ5R+ijMyxx21nnBzju3wrzM4f/SvERRoKKiqM/n/ycJhapqlgmF0X9nRGFe5rCzzgt23rkV5mUO/7/Mq6B6KqCAAgoooF8UCEUBBRRQQAH94v+k6qmAnR+aptHS0kQkEgYyL743Noqoqprx92YCO+vcMj8vAYfDRUVFDYIgZPC9BeQaWSUUb7zxBg8++CCyLHPGGWdw6qmn9truqquuYv/992fu3LkA1NXVceWVV+L1etltt924++67KSrq29BSwH8fOjraEASBAQN2QRAyL9jabCKyvPNdxrDzzi3T89I0ldbWZjo62igpKc/YewvIPbKmempoaGDJkiU8++yzvPrqqzz//POsX7++R5vzzz+ft99+u9vnN910E/PmzWPZsmXsueeePPDAA9maZgF5QijUQUlJeVaIRAE7BwRBpKSkglBo5/MMKsAcsnZKP/74Y/bff3/Ky8vxeDzMmDGDZcuWdWvzxhtvcNhhhzFz5sz4Z9FolM8//5wZM2YAMHfu3B79sg1V03a+P+pOMIcd/qS1xqqCJBU0n//XIUk2VFXJ9zQKSBNZO6mNjY3U1NTE/19bW8vXX3/drc3ZZ58NwOrVq+OftbS0UFxcjM2mT62mpoaGhoZsTTMOJRDgp1sW8f5hx/PUv1tAdmR9zFRx0rZ3+L54KF+Wjcv3VOJwVDYxZHwdkl3jsCHT2Wfg3ogmpYNs6a2VYJBIawv2QYN3Gt24oiq0RdoJRkN4HG5K7SXYxJ2HUHbW16GVlyG4M6vitbr+mqbxZdM3LNv0HpIkcvgu05lUu9dO83u2f/oJ2/73HQZfc91OM6eQHEbKkoSetZ2qqmq3BdQ0LaUF7a2d2R+iqqrYVHuAYGcb0eYmNv30Pu4JfmZVLcApeky/J+PQVIY++Ay1QwfwsyN2DkKxpvVr1mqrCSsVlDuLeGrN80RtYY7b/Rcpv6OxUcRmy86mVqIR1EgESdAQbVJKferq6pg79yiOPXYu11xzXfzz779fx/z5p3DddYs46qije+177LGzeeCBhwGNxx9/hGuvvZE1a/7DK6+8xLXX3kB7ezs3334jmzf+iCCIlFdVcNYF57Pv7pPZXl8f75MvaKqG1tmJGu7EWVKS8feLokhNjbn3vrP+Qx799lmGlA5E01Qe++5ZcCr8YvTBGZ+fFfjrfyKwYQOVRRK2ncB+GoyGWPT2vUwatCcLB55ser2TIWuEYuDAgaxatSr+/6amJmpra5P2q6ysxO/3oygKkiSl3C8RXm+H6YCTSEsIAJvSDrYo252fsHDP00y9IxtQAgE2aCpDigX23ntwvqdDe8TPWytXoraVMXfsmRw5dSR3/vPPvPjtPxjpGcWgogEpvUdV1awZdFVZBkDujCKmqF1VFJWysjJWrvyYzs4okqQTmOXL36a8vAJV1fqdr6Ko1NfXsWXLFmRZZfTocVx99XXIssqS+xYzeNeh3HjjbbhtLt5+503u+Z87uffPf2br1m3xPvmCsV6aomRlHqqq0tTkT7m9L9zC01+9zLiK0fx64kJqqku4+b37eObff2eYczeq3ZUZn6NZdDT5AGjYtB2HyfspG3hu3Ss0B3yML9sTwNR6gx6Z3R+DnTUbxYEHHsjKlSvx+XyEQiGWL1/OQQcdlLSf3W5nypQpLF26FIBXX301pX5pIya1OBSdYHzR+DXfeddmf9wkUPz6Dx5ta8/zTHT8ff2bRNUIkY3j0TQRURA5aexxOG1Onlv7Sr6nB+gXHoBm0tXT7fYwZsxY/v3vL+OfffbZJ0yZsi8A06ZNiX++dOkb3Hbbom797733btatW8PixXfyxReruPDCc4koEZq8jQgKOEVdnTlzxmxOX3AmHdEgS+65K94H4Omnn+Css07ljDNO4YEH7kXTNOrr6zjjjJO5/vprOO20E7niiotob28zvS59wlgvZeewJbz4/euoaMwb90tEQUQURU4ZNxcRgRe/fzXf0wO6zqXiz/+5/LFtMyu2fcIhQ6exW9mwrIyRNYliwIABXHrppcyfP59oNMrxxx/PhAkTOOecc7jooosYP358n31vvPFGrrnmGh588EEGDRrEH/7wh2xNswuiod7SqIrIyKLAPze+yx5V+VX3KB36hpR3gg3ZHvGzquErplTuw7/CxXGDdomjmJnDD+elH15ni38bQ0uGmH73R9/Us+Lr+ozMU41EQFUR7M0IksS0CYOYOn5QSn0POeQI/vnP95g0aQpr1nzHqFGjSbWs/MUXX8Fjj/2Fyy+/mi++0KVpfyTAsaecyL233Mnb//gHkybtw3777c+hhx2JN9rCWRecx8vP/I3LL7+aTz75mHXr1vDww08hCAK33HIDy5e/xYQJE9mwYT0XX3wFkyZN4f77l/DYY3/hkkuutLxGidDUnYdQNId8fNP8H34x/DCqEiSHSlcFh+86nX/8uJzGYDO1nuo8zhKUjo5uf+cT72/9CLfNxVEjZmRtjKxa0+bMmcOcOXO6ffbwww/3aHfHHXd0+/+QIUN4+umnszm1HojbQVQQNIF9QvCe9BPNHdupLh6Y07kkwtiI0XZzomQ28En9KlRNZXL1FP7Fhm7qvf0GTuK1DUv5qO4zTh57XB5nCV0BfOY9s6ZNO4iHH34QVVV57713OPTQI3jvveWWZxKUg+z5s/G8+OLrfPPNv1m16jOeffZp/v73l7nj3iV0yp3xtqtWfcZ//vMtCxeeDkBnZ5gBAwYyYcJEhg7dlUmTdIlm5syjuOmmay3PqQcUXfLSlPzHdqys+wyAqYP37fHsgMH7sHTTu3xc9xnHjpqV66l1g8HAGX/nCx3RAF81fsPUIfvjlLLngLPzuF3kG3FvAQ0QmDrheN7b+DIfrnqa4w7ODOdmBcZGVMNh1EgE0ZEfbyxVU/mo7jNGl4+gxlMDbOjmIuuxe5hUuxefb/+C40bNNr1pp45PnetPhvCWn0BRkMrKsVdUmOrr8XgYNWo0X3/9FV988Tnnn39hN0JhOFvIMb1+f1A0BUVVefT+B7jk4ivZe+/J7L33ZM4551yOP/5Y6n/cihZrB7rL8IknnsLJJ+u2Mb/fjyRJtLW1dnMl1jQ1o67FcYlCVVJ2OskGFFVhZf3n7FE1jgpXzwC9cmcZ46t+xsr6zzlqxJF58xrTNK1L9ZRnieKz+tXImtIrYc0kCtFOBhIOh4BA7W77M04q47PIdiKbvsjbtIwNCfnlXja0/khzyMuBg/dFiq2VsoPDwNTB+xFWOvmq8Zt8TBHQD7HBIWPRf//QQw/nz3/+I2PH7h530wYoLy/nxx83oGkaK1Z80KOfJNlQEtQ3iqbikOz8tHkzzz33dDw9Rl1dHYqiMGzX4bjsLiLRKACTJu3D228vJRgMIssyv/3t5bz//nsAbNmymR9+WAfAm2++wf77H2jpu/WGuCShaZDH1CL/8a2jLeLv99KbOmQ/OqIBvs2j/VCLRNBiv1ni+cwHVtavYrfSXRlSnBkmqy8UCEUMQsxGISRwyVNGH0m7TeLHlU+gBlvzMq9E4pBP7uUb7xpsgsSE6j0QY2u1o+p+RNkwyhylfONdk4cZxqCqGConq6qUqVMP4ocf1nHYYUd0+/z88y/kqqsu5bzzzmTXXXsaDYcPH05Hh59bbrkeVVPRNJUiu4ebbrqdH3/cwAknHM1pp53AzTffwI033kppaRljRowhEOjgppuuY9q0g5g+/VDOPXcB8+efxOjRY5k58ygASkpKefTRhzjttBNpafFxxhkLLX23XpFAUM06AGQS3zSvwSU52b1qbJ9txlWMxm1z8W1z/vZY9zOZP0LhDfmoC2xn0oC9sj5WQfVkIFGiiF2Ae9TsDutgrV1j6PuP4J55Wc5TTij+joR/529Tfuddx6jyEbhsTsKirlffUaIQBIE9qsbxZdPXKKqCJKYWw5BJdLvoTEgUgwYN5qWX3gB09dN7730Uf3bttYvi/z7qqGN79DX6ATz99AsABKIBrrvzNlw2F2XVpdx00+3xNok5lWqrBnDng/dRGVO1LFhwNgsWnN1jDJfLxR13ZMepoxtBVRXAnpVx+p2DpvGddy3jKkf3q1KSRImfVY7hO+/avKnJup3JPDJv33l1CTMXDjcFicKAQQA0MLZeiaOYYSVD+X7AIJSt3xL99p2cT0vp8CPE7BL54l68IR/bAw3sUa1vSDF2OHuLVdmjehwhOczGtk25nGIXDNWPIObVOGtEyTrE/i9du2jDLtoIyaEczawXqEqcUcrXmtUFttPa2ZbSpbdH1TjaI362dGzLwcx6QunQPRBFpzPPzNsaqt1V1Lqz7wFWIBQxxDkTTSORR9mjaiybo210DptA56cvojRvyum8lI4OHAN1/WMiJ5NLGPEkxiE2VE+9EYqxFaOQBCnO7eQahkQhOuxxI23O56BphOQwLpsrJY7XbXMRVjpRtd4v6URpJxvQFAXBrhO0fK3Zd836HutP7WTAaPNdc372mHEO3bsMyZtEEVGirGvZwB5V43IiVRUIhYEdjNkG9qgeh4bGxt0PQHCXEnr3QbRI7rg/xe/HXjsARDFvEsV33rXdOBdJ7N2YDfqlN7J8t7wFKxqxAKLdDqqacgxEJtGpRFA1DbfNlVJ7l82FpkE4wVU2l9BUNU4oyJNE8a13LUOLB1PuLEvattRRwrCSoXnbY8Y5dA8ZkjeJ4ofWjUTVKHvmKM6rQCgMiF0SRaL7/a4lu+C2uVgfqMN12Plo/ibCHz6RswtI6fBjKy3FVlycl02pairrWzcxrmJUnHMxVE99rcG4ilHUBbbTEQ3kbJ5xxDhi0eHImxdPp6Jf+C7JmVJ7p+REELr65RKGl5hgs4Eg5CXoLqpE2dz+E2MrR6fcZ2zlKDb7txBRIlmcWe9Q/H4QRVwDB6AGA3lZs/WtG5EEiVHlI3IyXoFQGIjZKIQEGwWAKIjsVjqMjW2bsA0cg2PKXOQNnxJd+6+sT0mTZdRQCKmkBHtpSV4kivpAA2ElzIiy4fHPxH4kCiDe9se2zdmeXg9oigqCoF985MeLp1OJYBdtKRvzRUHAITrozMOlZ3iJCZKEIEmWXYrTwWb/VmRN6bbHkmFk2XBUTWVT+5bsTawPKB0dSEXFOMp16UcJ5J4h2tC6iaElQ3BIuXE8KBCKGPrT840oG059oIFgNIhj4iykIXvQ+fFfUXzZ3aRKQNd/SsUl2EtL86IP3dC6CYCR5cPjn4mxXdNXTYphpUORBCneN6dQla5LD7qM2zmCpml0Kp2mAw6dkpOIEunTTpEtGIRUEPU1ywdhNRwfRpjIU2TkNMqH04TS4UcqKcFWWhb/fy4hqzI/+beYWq90USAUBoSuXE8C3YnGyHJjU25GEERch5yL4PAQfvcBtGg4a1MyVE1SSTG2PBGKjW2bKHWUUOXqyrvTn9cTgEOyM7RkSF4OsaaoIIpxQpHriy+qyqiahjNFtZMBp+RAQzdS5hQGIZVEBFHMixplY9smaj3VlDhSLw9QZPcwsGgAG/JCKDqQiouxl+qpvHOtEt7i30ZUlRlpQgJLFwVCYSBuo4AdZYthpbsiCiIbY6oU0VOG69DzUFu3E/4oezmpDMKgSxQleclUubFtMyPKhneTuARBQBD6Vj2Bzh1u9m8lqiZPdZFJaKoS4471rZ3qxVdfX8fBB+/PggXzuv15+eUXTI0fidkZzEsUevtU7BSPPvpQPMPtHXfcwtq1/+mzbX19HccfP6fP54aXkyFR5Nqmo2mavsdKh5vuO6J0GD+2/ZRzKUzx+2Pq4FL9/zmWKAziuFsOCUUh4C4GoQ8bBeiHeJfiwd04ZNuQ3XFMOprIF68RHfwz7GOmZXxOxgaUSkogJlHkMsiorbMdb9jHwbv0TBchiUK/5VBHlg3nf7d8yBb/tpyKyCgqOG1dqicTF191dQ1PPPFsWsOHlQiSIJrOQySJEnbRlpKd4ssvV7P33pMBuOaa6y3N00A8bkLSpbBcSxSNwSYC0SAjys3vkRHlw/m4/jO2BxoZnMPEnUqHH6m4GJshUeRY0t/YtplqVyVlzswXmeoLBUJhIEH11FvS0ZFlw1lR92m3iGPHpGNQ6tcRXvEUYvVuSJXm02v3B8NfWyouRigtBVVFDQWRPLmpqGUYo3vjXERR6Lc41IiYTWNj26acEgpNVRBFCUEUM+LFs27dWq688mKefPJvSJLImWeeyh13LOaDD96noWE7mzb9SFtbK8ccM5d58+YTjoZ56qFH+M+/v0UQYMaMWZx22gK++GIVTz/9OC6Xi82bNzFixEhuvPE27HY7b731D1588TmicpRho0Zw3VWLcLlcHHPMDA4++DC+/vorJMnGzTffzr///SXr1q3hzjtv5X/+526WLPk9Z511LhMmTGTx4jvYuHEDPp+PUaNGsWjRbcm/YIJEQUyiyCUzsrH9JwBThmwDI2P76se2zTkjFJqq6qqnRIkih6onTdP4sW0zYytS9xDLBAqEwoDQpXoSe8gUsGvpLkS3rqAh2BTflIIo4jr0PIKv3Ej4nfvxHHcjgsOdsSkZqiapqBipLLYp2/05IxQ/+bchCiK7lPSsrCcKQr/MeqmjhHJnGVv8qUXPRr//iOi6non2zEDTQOsMo9hsyHY7SjiMLIqw52HYx0xN2r+5uYkFC+Z1++z662/mmGPm8sAD9yLLMsceezyjR4/lgw/eZ926NTz44GOoqsrChaex96QpfPTFClqavTz55HNEo1F+85tzGTFiFC6Xi2+//Zq//vUlBg4cwMKF8/n005UMHjyEN954lQcffIyIEOWBB+/n2eee4qwzz8Xr9TJ58r5ceulV3H//El5++QV+85tLefPN1znrrHMZOXJUfJ7ffvs1Npudhx56HFVVueii81m58iPGjv1Z/2um6FHZgijqhAJ0u4UtN1fDFv9WnJKDAZ4a031r3NW4bS5+6thG8l83M1ADAdA0pOISRLsd0e3OKaFoi7TTHvEzrHSXnI0JBULRBaFnUsBEGMV4tvi3deNexKIKXIddQOjN3xP+4DFch12QMW5M9rcjFhcj2GzYymIeFv52GJgb7mmLfxuDigZg70WNIiWRKEBfs63+umxNrxfE5hNff6Fn5sJ+0Jfqadiw4SxceDpOp5Prr785/vnhh8/A49Hrqk+bdhCfr/qU7775hqN+MQdJkpAkiSOOmMnq1Z8xdepB7LbbSGprByCKIsOG7Ybf386XX9azdesWzjvvTDRNJRQJMy7hct9vvwMAGDFiZLfKezti4sRJlJaW8fLLL/DTT5vYunULoVDywFBNUeNqukQHgFxlUNri38YuxYMRLeRQEwSBXYoHp8yMZAJGATEpJk1IJaU5tR0a39VKcbB0UCAUMQiC0EUsejkmAzw12EU7Wzq2sR+Tuz2zDR6Hc9/j6fz0BaIDluMYn5lKU0p7O7YSfUPaY4RCbs/NptQ0jS3+bX3m3hGE/m0UAEOLB/Nt8xo6lUhS4659zNSUuP7+oHZ2Eqmvw15bi7O0hOC2OjRFxT44vVrjHR0dBINBgsEg7e3tlJfrCfyMutoQ8wAT9XWTuhFWLZ563JFQS0QQBL2ugaJy6KGHc8klV+rBjY0b8Ng88XZOp7Nb+76wYsW/eOSRhzjhhJOZNetoWltbUwsKVZW4JGHWASBdqJrK1o56Dhi0j+V3DC0ZwofbVuYsCaUSO3/GuZRKSnJ2JgG2+usQENgly2nFd0TB6ykRcULRE6IgMqR4UJ8csn3CTGzDJ9H5yQvI27/PyHQM7wroIhS54l7aIu34ox29qp1Alyj683oC/RBraGzryEyJ02RI9OCJ/52BALLFi+/gl788geOOO57Fi7uqMX7wwftEIhHa29v56KMPGD9pInvuNYHlb7+FoiiEw2GWL1/G3ntP6fPde+89mQ8+eJ+WFh8CAo//6S+88mL/nlY71r0AvTreoYcezuzZR1NcXMyXX65GTeG7a4rStV45jj1pDDYTUSJpccdDS4YQVWUagk0ZnFnfiLusGxJFaWlOVU9b/Nuo8VThSjE9TKZQkCgSIfSvqhhaMoTPt3+Jqqk9RGVBEHAdfDaBV24i/O4DeOYuQvT0rNJlBnJ7G85dhgJgL8ut4cwgiH0dYjGJ11Ni31x5PsU5YePCi3nxpGqc7c1GIctRHA4nixbdhqZpnH32fN57T88i7HQ6+fWvzyYQCHD66WcycJdBzBo8hxeanmHBglOQZZkjj5zJ9OmHxGto74jRo8dw5pnncNFF56NpGkN3G8acE+f2O8/99juAu+++neuuuyn+2Zw5x3HTTdfy7rtvY7PZGT9+AnV1dUye3M+L0NfMqJooSEY0e24IxVZDjVJsXeLbJdZ3R5VwthBXPcUkCltJKeH167M+roEtHXXsVrprzsYzUCAUCRAEIRZH0fulMrR4MB8qK/GGWqjxVPXs7/DgPuJCgq/eQvi9B3HPvirOrVmB0u6Pb0jRZkMsKsqZmLvFv61fEVc3ZvdPKMqdZRTbi+IXQtYRc/XspnPXNP1PEkIxaNBg3n//k6RDGDaMRx99iAkTJrJw4XlATI3ir6PUUcIll/QsnTtp0pR4zWvoXuNizpxjmTNHr3PR3umnpbMNRVVYsaKLuMyaNYdZs/R4iHnzTmfePL2u9h//+Jd4m6eeer7XOfeVeVYzcmElqp5ymO/pp45t2ASJQUUDLL+jP5VwNqC0t4MgIBXrwYFSaSlKh1+364jZVdB0RAP4wi0cNOSArI7TGwqqp0SIur+T0Mf9F+eQ+8mDL1UNxfXzM1Dq1xH5/GXLU9FkGTUYwBYTcUHnXpT2NsvvNIMtHXX9irii2HdktgFBEBhaMoQtHbkxaBsePHEVomGczcHFF1VlNEg7947RP6LmIEJbVfW0+gm2FkQxZ6qnrf46BhcPSsu2IIkSu/SjEs40lPZ2pOKSOFGQSktB03ISS5FMys8mCoQiEYZE0Qf3Oah4IKIgJt2U9jFTsf/sECL/Xkp04+eWphIPtksgFLnUh26NeaP0BVEUk6qeQFcN1HVsR8mFOkOJ5Xkyfr+YcTYbF9/ChefFpQkgnsXUbjIie0fYRUe392UTBgFNlHr1oLvsRzprmsZWf12/eyxV7FIyhC3+upxkdJb97d3OpGHUzoXtcGuM4crEmplFgVAkQBBiSqc+9ptdtFHrrqYusD3pu5wHzkOsHUn4/UcsJQ80VEyG6kn/d0nc6yKbCMudeMMtDC7q27MiFWM2wODigSiaQmOoOZNT7BWaqkDipSdKXZ9nGVFVRhQEbEJ6njeSKGITJaI5kCjiebCkHQhFDtarPeInIAczYlcYXDSAsBKmtTP70rbS3h53MAHi/87Fuazr2E6po4RiR27iqBJRIBSJSGKjAF2qqA80JH+VZMd9xIUIDjeht+9DC5sTTXd0wwOQSsvixrRsYntQ/36Di/vWHYtCctUTENc/p7Jm6UKLSRRxxFVP2eeQo2oUu2jLSAyNXbTlJkeWIVF0Uz1JOVE9GfthcFH6hGJQ7B11Odhjit8fzxoL+pkEcnIu6wMNGVkvKygQikQY+Z76aTKoaADekC8l1YBYVIH7iAvRAi2E3nvQFKcWj8ou7eJebKWlqIEAmpzdS6S+Qz9w/RkZk6XwMDDAU4uAQH1HciksbShqPBYAcuvuGVWi2JPUx04VdtFOVI1mXZXSpXrqvma5sOkYhGJQP8xIqjDeUZ+CpJ8uFH97jzMJuuNJNqFqKtsDDWkZ/tNBgVAkQhRikdn9SBRFA9DQ2B5sTOmV0oBROKedjrLtOzo/ezHlqcRVT4nciyHmZjlbZX2gAbtoo9rd07PLgCgIpEAncEh2atxVWef2NE3rqXoSBMhB6mxFVVA0NaOEQtP0ugPZRJxx2UH1hKZlPT17fWA7RXYPJfbUU4v3hWJ7ESWO4qxLrWo0Eisk1iXlix6PXqY4y04mvnArETVaIBQ7A4R+IrMNDDZUKR2pb0rHuOnYdz+U6NfLiK5fmVIfpb0dwWZDdHV5HRkbNNsusnWB7Qzw1PabVkFP4ZHaZTKoaED2VU+9efBgjkMOBDpYvPhOTj/9RBYsmMdvfnMe69Ylr8ts2BPsMY+lZKm/k0HU4Nennsk999xl+R0pQdEJq7Hv6+rqOOlMPY7kkYcfZMWK7FVxrOvQueNMpbsZVDTQ1Jm0AkNqSFQHC6KoR2dnWfVkSEuZkMCsoEAoEiEIvaYZT0SNuxqbIJm++JwHzEMaOIbwvx5Dad6UtL0S865IPEjxilpZ9nyqDzTE9b59QUjRmA06oWgKNWdV7x6Pyt6RUKQYna2qKldccTGlpaU8/vizPPHEs5x55jlcccVFtLW19tvX+F5GTqxrrrmeceN2t/I1AFj12eeMGDuaf/3zn4TD2SuM1cOmQxeztPCMhUybNj0742paxvXtg4sGUB9syGptCmWHPE8GbDnwRkxFHZxNFALuEiGIxPxj+2wiiRK1nhrT+lBBsuE64kKCrywitFzPNCu6S/tsLycE28XHLs2+h0VIDtHa2RaXnPqClKLqCfTNrWoqjcEmhmQpR01XXYUdvI4kCS2a3J70xReraGjYzsKF5yHGdPaTJk3hd7+7AVVVeeqpx1i+/C1EUWSfffbnggsuIhwOsWjRtTQ2N6JqKucu/BU///nBXHjhuZx11rkA8dTimzb9yMiRo7jxxtuw2TUfkekAACAASURBVJzx1OKqqjF27Dguu+zqeF6nt5b+g/0OPAARgXfffZujjjoGgNtuW0RRUTHr1q2JRZGfzezZR/Poow/R3NzEli0/0dCwnaOOOoYzzliIoig88MC9fPnlahRFZdasozjppFORZZnFi+9gw9o1tLS1MmrsuB4pyf/nzluZtM9+7L33ZH73uysYMWIk33+/jsrKKm655Q5KS8t47713ePTRP+NyuRkzZiyKonQLJOwLrZ1thJVwRi+9QUUDiCgRWsKtVLkrk3ewgC518A7nsqQ0615PdYEGyp1luG2Zy05tBgVCkQBB1CUKUUsSxVs0gE2xPPpmILpLcR95EcHXbyO8/H7cR12F0EeAluJv70kocuCzXR/QbS/JRFxRFIjKKaqeYi6Q9R3b+yQUn9avZmW9tZgT0LljLRpFaHYgiGI8G4sajbJv0VimDek/LfP3369j9OgxcSJh4IADprFy5UesWPEBjzzyNDabjeuuu4pXX30Zj8fDwIGDuWLRtWzcsIEvPvyMn//84G79jdTi1dU1nHfeAj79dCW77jo0nlrc6XTy5z//keeee5oFC86mpaWFVas+5ZxLfo1NsvHaa6/ECQVAY2MDDzzwCBs3buA3vzmP2bOPBmD9+h944IFH6Ojwc+KJxzJ37om8997bADz22F+JRCJcdtmFjBu3O5qmYbPZuf/WO9FsNq645QZWrvyI3Xffo2s9Ewzp69f/wG9/ewNjxozj2muvZPnytzjssBncd99iHnnkKaqqqrnuuqspKkrNbTNuyM4ooRgYf3e2CIVx7my9nMtoU2o2S6vYHtieN2kCCoSiO2LusckwqGggqxv/TVjuxGUzVxtZqhmO6+BzCL/3AOEPnsB18Nm96mmV9nacQ7pHYIpuN4LNllUbRVwXmmRTppLryUCtpwZRELNr0DbmsuNaCgJoyYvxiKKAw9H7b7l69eccfvgMXDF70ezZR/PWW2/yq1/9hoce+hNbtv/EfvsfwIIFC3v0NVKLA/HU4qtXfx5PLQ56PqkxY/QsvcuXL2Xy5H2oKKtk4n5TeOjeP/L992vjz/fddz8EQWDEiJG0JxhQJ02agt1up6KiktLSUgKBDlat+owffvie1av1VCChUJANG9Yzd+4JlJaW8epb/2BrU2P3lOTxuixdv21FRWV8/BEjRtHe3s7XX3/JnnuOp6amFoCZM2fzwQfv97m+iegiFJlTPRn7tS6wnT2r+6/BYRVKXxJFaWlWz6SqqWwPNvLzipFZGyMZCoQiEYYxO4mBbUCRXmSlMdTEriXmC4jYR+6L2lpHZPWrRCoG45w4u9tzTdN6lSgEQdCjs7PoYdEQaMIm2qh0VfTbLtWAO9B191Wuin4zfO43aDL7DbKeqyfa2oLS2opzmF7f22YTkWUVub0d2edNWoxn3Ljd+fvfX+pBUB566E+sXv0ZM2d21Z3WNFAUmaFDd+XpZ57nrX+9xbervuKNl//OM89092zrLbW4qnalFgcIBoPxbLBLl/4Dr7eJhaeeiqopiKLAa6+9wpVX/i72vq6048nGURSVCy64iOnTDwWgtbUVt9sdS0n+Z4495AhmHjGT9nC4uyuuKHYjFInv1r+/pkfmp6p73AENwUaK7J6MBo557G5KHMVZzSKrtLcjOByIzu4Mha20FK2zE7Wzs8ezTMAXbiWqygz01Gb83amiYMxOgCCIMWN2EkIRq8bVGLC+KR2TjsE2cj8in71EdNPqbs/UYBBNluPFihIhlZZllXtpCDZR665OWkhGFAQ0ExfFAE8NjdlMBb2DB4+BVPM97bXX3lRUVPLYY3+JX9qffrqSpUtf58QT5/Huu2/T2RlGlmWWLn2dSZOm8PLLz/PIow+x38+ncvGlV9DS0kIgEEg61UmTpsRTi2uaxuLFt/PCC8+ydu0aGhsbePnlf/Dc8y9z7xMPc8vtv2f58mUEg8nfuyMmT57C66+/iizLBINBLrhgId999w2rVn3GIQcfxi+mH0JxaWmPlOTxZIr9YM8992Lt2v/Q3NyMpmm8++7ylD2YGoJNliraJUO295jc3tbHmSyNP88GDOJXm4U1SxUFiSIR/dSjSESNuxoBIS3uRRAEXNMXEmxvIvy/DyEefS1StZ6KW27TN5xU1jNNua2sDNnnszxuMjQGmxicgsFZFAUUEwFhAzy1rG1Z32uK9kygNw8eSKza1j+hEASBO+74A/ffv5j580/CZrNRVlbOXXfdy5gx42KG7vkoisy+++7PL395Ep2dYa674Rqu/tVFuOxOfv3riygpSV7wfsfU4qNGjeG00xbwpz/dw6xZc3A6XfHcWHtMGM/QobuyfPlbptfk2GOPZ+vWLZx55jwURWHWrDlMmjSFsrJyFi36He8sexOHyx1PSd5tzZJ4D1VUVHDJJVdw6aUX4HA4GTRoEA5H384ZiWgINvVZECsdDPDU8FXTtxl/rwG5ra2PM6l/prS1QU3muX6D+BmajHygQCgSIQopSRQOyU6lqzxtMVewOXDPuIjg328m9Pa9eI67AdFTjhJzx+yNe7GVlRH+cWNa4/YFWZVpDvvYu3ZC0rappvAwMMBTg6zK+MKtVGfB2NgXoYin8ZCTu8iWl5dz/fW39PpswYKzWbDg7G6f2WzF3HDbbfgjfoaWDIlz1Impv1NJLW7gssuu7pq2KCEJIlE1ymOPPQPoF38ijDTkickJoXta8d5Sno8cOYonHnqcaGMjjkGD4+oSm03kpZfeINLUyFW/ughnzAEg8X3GWG1traxf/wNPPvk3RFHknnvuYpdY7ZT+EJJD+CMdWZEoaj01BKJBOqIBiu2Zz4ektLXh6KVaonFO5SRu1FbREGzCbXNlJDjRKrKqenrjjTeYNWsWRx55JH/96197PF+zZg1z585lxowZXHvttcix1BRbt27l1FNP5ZhjjuH0009n27Yc1TNIrLWcBLUZEnNFTznuGRejdQb0nFByJC7CJkZlG5BKy1D8/qxEzjaHfKiamtIhTjWFhwFDbM6aDllRerrGkhBXkaVEd7IaxZahHE87wibashad3aPIUwJSCVIsLS3D7/czf/5JnHHGyQQCAebMOS7puMbvny3VE5A19ZPc1tbnmYSYRJEFNASbqPXUZGWPpYqsEYqGhgaWLFnCs88+y6uvvsrzzz/P+h0qQV155ZXccMMNvP3222iaxgsv6CUg7733XmbPns1rr73GkUceyZIlS7I1zW5I1UYB+qZsCDVnJB+PVD0M16Hnojb9SPiff0FujUkU5b1JFOV6/vssuMia0YWaMWZDggNAFg6xpml9q55EUU/jkYJEYQVRVY4H2mUa2UwOqPWWEDAGQZJAVftlRgRB4JJLruCZZ17kySf/xrXXLorHgfSHhkD2CUVDGrbDvqBGI3p9mN5sFCUlIIpxlXGm0Zglm44ZZI1QfPzxx+y///6Ul5fj8XiYMWMGy5Ytiz/ftm0b4XCYiRMnAjB37tz4c1VV6YgVAgmFQnG3xKwj5nyfKqGIKJGMpTa2D5+Mc/+TkH9cRefaT/X0HW5Pj3ZSXMzN/KZsNMHtiTHPmlRRYi/GbXNlR6LoI32HgWwlutM0DVmVsWUox9OOsIk2FE3NTrRxTALrlUvNYsGnxmAToiD2m0fMKqpclUiClJU9Fs/m3Auh0NN4lGblTIblTlo72/JOKLJmo2hsbKSmpuvL1dbW8vXXX/f5vKamhoYG3b/64osv5uSTT+bpp58mGo3y/PO9l3jMOGL1KFIR8BJVKRWu9GpjG7CPn4Hqb0J+/T0kT1Gvh9jYqNkQcxuCTZTYi/HYk0d/iiYlCkEQqPXUZOUQx7ljW9+EIhsZZOVYVbvsSRQ6AYqqUZxSZt0u+7Tp0FU7G0UBe2aJYEOwiSpXBbYsrJkkSlS7q7IitXY5mPQkFKCfy2y4rTeG8u/xBFkkFKqqdrvodvRP7+/51Vdfzc0338zhhx/O22+/zYUXXsjrr7+eso6uqsqa0WebwwYRkCSJmpr+vVfEot3gKwiK/qRtzUA7+jya3vocobMdT8taisbsE39WU1NCWB3MFsCtdmZ0XADf1z52KR+Y0ns9HkfcppPqPIZVDObbxnXU1JTQ2Chis2VGoFUiOsGyOexICe803i/bbCidnRkbz0AkpppxORym351Ke5fggBCoqJmfu6Ii2mw93muziYgOO1FARMvIuKIoxveIN+JjaPkg03s31fZDKwZR72/I+NnwbujU5zFsMMUJ7zbGaaqpItLamvFx1wV1zcrPhgynpjz1d2d6HlkjFAMHDmTVqq7i8E1NTdTW1nZ73tTURfmbm5upra3F5/OxceNGDj/8cABmzJjBjTfeSEtLC5WVqXnLeL0dloKBZEVDQPfmaWrqP8mXpkk4JAcbGrfQVJ7ZhGCKUIxUJNPwyh/wzLkGqXYENTUlNDX5UWX9J2vZuh0xyRzNYlvbdiZU75H0uwNEIjJyLIVHKu0ByqRyfKFWttQ3o6pqvH+6UCJ69lZFE9Bi7zQC7gC0mI0iU+MZCMdySAmqaOrdiXPrD4Km13DvjEZwS5nN8aMpMoLN1W0exry0mEwtR6KQgTVTVVXfu5pKvb+BUaUjUt4zQHzvp4IKqYIv/d/S0NiWUTfs1i16xgK/ZicUm0vivBR3EeENG019r1SwvmELAgJS2J3yu82slwFRFPplsLNmozjwwANZuXIlPp+PUCjE8uXLOeigg+LPhwwZgtPpZPVqPdjstdde46CDDqKiogKn0xknMqtXr6aoqChlIpEWhNTcY/WmAjXuKppD3oxPQ/a34xi9N4KnjNDb96C2dxFU0eFAdLszrnoKyWE6ogFqPKnpjkXBnNcTQE1ML+0NZzYOpD/DbPxzrX/jrBXIml7+NBtxIRCLxBclZC2zBm3D+N+bxxOgR2YjoCmZHbc94ieqytS4qzP63kTUuKtQNIWWcGbPh9zWBoKAVNw7p24rK0dpb8/4HmsOeSl3luHoIydcrpA1QjFgwAAuvfRS5s+fz7HHHstRRx3FhAkTOOecc/jmm28AuPvuu7n99tv5xS9+QTAYZP78+QiCwB//+EfuvPNO5syZw1133cX999+frWl2h0DKhAKg2l1FcyjDl54so3Z0YK+qxT3zUjRVIfTWYpRQF4cglZVlPArU+B6pGhklkwF3ie/ONHHVFEVXg4l9bGcTxtlAoIPTTz+R+no9AC0ajfKXvzzA/PknsWDBPM49dwGff/4pQMyQnZprbH19HccfP6ffNkuXvsHMmYeyYME8FiyYxymnzOXhe/9EZ7Sz337TpumxGq+++hKvvvpS0rnEjf992XQEASQx43adpqD+u9dkwZBtIFt7TGlrQyouRugjDYxUVgaqitJhruRxMjSHvFmJOzKLrAbczZkzhzlzuh+Ohx9+OP7vcePG8dJLPTf2hAkTePHF1KvBZQpairmeDFS7K/nOuzaj0caJMRRS+WDcR15E6M272P7CHdiPvBTB5tS5lwxLFMbBSvUQpyNRNIW8DHBk0K4TM8z29bulapz97rtv+f3vb2XLlq7MwLfdtgiHw8HDDz+F0+lkw4b1XHrpr7n33gdx1njixYoyhWnTDooH5imKwtnnncE7by1jwUlnJe27Y0BeX0gmgRnPMu31ZOyxbHg8GUgkFGMZlbH3yu29R2UbsCXEUthKU4tQTwVNIS/jq7KT5NAMCrmeukEvhZqqRFHjrkJWZdo6MxfTYBAAw7vJNmgsrkPPpXPrOkLvPoCmKnoajywRilQPsdmAOwCP3YPH5s64FNZXsJ2BVPM9vfHG37nssquprtY9TLZu3cIHH/yTSy+9Kh4jMHLkKBYtug2n08mzTzzJc48/Ge9/222LeO+95WzfXs9FF53P6aefyDnnzGf9+h+6jeP1evntby/nrLNO4+yz58cllB0hSRLjx0/gp02bUDWVN998ndNPP5H580/ittsWEQwGu7V/9NGHePTRhwBYvnwZp512AqeddiK33baISCTC8cfP4aefNqMpCqFwmONPO5HOzt6lFUGyZYVQiIJIZYa8BHtDhasMSZBoyrBEIbe29uoaa8BI45HJ6Oyw3Ik/0pFVwpoqCik8EmGonlKWKLq4l0y5yMo7EAoA+4h9KZoRxfv2w4Q/eEJPDJhhQtEU8lJsL8JtSy1mxQqhAENd54Udzlz7xx/RtuID0+8DUCMREATEBGlBSIjzKD1wGs5ddkl68V1zzfXd/v/DD+sYOnRX3O7uhuRJk6YgqzLTjziMO65dxG9+dSmdnZ2sXv05V1zxW6677iqmTz+UX/7yRFauXMGTTz7KBRdcFO+/ZMldzJ59NNOmTae5uZkLLljIE08822M+bW2tfLl6FTN/eQzrfljHU089xl/+8gRlZeUsXnwnjz/+ML/+9cU9+jU1NXL//X/g0UefprZ2ALfccj2ffPIxM2cexfLlb3Hmyafx4eefcuABU/sMkhMkCTXSv8rLLJpCXiqd5Uhi30Q9XYiCSJW7IvOqp/Y2nIN6pu8wkI34JsOWVyAUOxvicRSpCVpdqhQfozOUK76vhIBlU36Bv6mRyBevAePQOsOo4XC3mtrpQNeFpr4hxVjpDivqp83+rSZnlwSahiD2Q9wF/Vc1a5wVRbFHim0DUVWmdtBABg4cxFdffUFDw3YOPHAaTqeTr776Il4x7oADpnHAAdPiNg+Azz//jE2bNvHIIzr3L8sy27bpa7JixQcsWDBPNzhrKj//+cEcePBBfPz2+0yd+nPKYvvi6KOP4/bbb+p1bt9++zXjx+8Vr4Nh5K8aPXoMl1xyAQtOOIV3Pnyf8y66rO8vH4s9SVbHwwyaQ76cXHrVGXYy0TQtlhCwP4nCUD1lTqJoMqkOziYKhCIBGuaM2RXOckRBzOimNIJ2etNzOiYfixZqI/jh+4Au5jpcmSn+0hzyslvZsJTbS7GLOdXiRQaq3JV82fRNjyzWpQdOpfTAqabeBfoh7ty8CVt5BbbyLuK6owtqeMtPpo2zY8fuzqZNm+jsDON0dhHkF154FndpERMOnMzs2UfzzjvLaGhoiJc/laSuY6VpGps2/dgtu4CqKtx334OUxvTazc3NVFRU8MMP33ezUYBetGaLvy6e+jzhm/fymfHdbd3qN7W0tAAwaNBgBg4cxL8+/Bct7W3sscf4Pr973H6RpI6HGTSHvOw9IHnCyXRR465iY+vmjBE5NRAARYnbIXqD6HQiulwZlSi61MH5N2YXbBSJiKmeUoUkSlS6Mivmyq0tSCUlvXpXCIKAc+p8HENHAxD+7qPMjBnL6mqGcxENQmFBolAzmJYiLiX0Y6MAazr3gQMHcuCBU1my5K64Lv/779fy178+ydBhwxCAQw85nNWrP8fna2aPPfYEYOLEvXn33eUArFr1Kb//ffd61JMn78Mrr+jOGj/+uJH58/WU5b1BFEQkQWT3CXuyYsUH8ap2r7/+KnvvPaXXPj/72R589923eL3NANx//x9YseJfgF6d7/6H/8ThPz+k30s0VbtOqghGQwTkYE6442p3FWElTCAaTN44BcitOqG1VSQp5lVenlEbRVPIi8fmxmPvmcon1yhIFAkwzNhmuJAad1VGDWdyS0s3znhHCKKIe/qp8K9FhD/9B57Ro7ENTY9L84Vb0NDMqZ5ihMJMGg/o0rdmLH+RnNyDB/T0HlrUfFzAb397Iw8+eB9nnjkPu92By+Xi+utvYciwoUTVKC6Xmz33HM+IEV0eNpdeehV33nkrf//7S7hcLq6++rpu77z88qu5/fZbOOOMk9E0jeuvvxmPp++02DbRxi7Dh3H66Wdy4YXnIssyY8f+jCuv/G2v7aura7j44su57LLfoKoKe+45gVmzdO/D6dMP4c47buHIgw/t93sbrrOZIhS58HgykOhdl4kqenFC0c+51J9XIMekt0ygOWhOHZxNFAhFIgRiSQFTF7Sq3VVsbt+SsSnIra3YyvvnXOzVeoS7aisjtPyPuGddjm3QWMtjNpmMoQDdPRbMEwrjECtaZi6grjxP/W9lQZJQw71z7Tsisf6Cy+Xi0kuv6tGmPtCAJEgEAh2sW7eOCy7oMioPGDCQP/zhj32+t6amht///p4ez2fNmhO/0BNhEyU6lUivNSyg97oUhxxyOIcccni3dpqm8cUXq5g0fi+qa5MU2ImpzzIVdJdLfXuik8luZbum/T7j8k92Lm3l5YR++D7t8Qw0h7wMK01e4yMXKKieEqAhmPJ6Al1/GJRDBDMo5iYTcUW3G8HpRBg0EbG4ktCyJShNP1oe04ou1KrqqcxZGs+KmgkYF1lSiUKyJU2dnfKYsayxP/6wnuOPP5qjjz6OqqrsRRvbRBuKqpi2B+2I++77A0uW3MV5p5zeFVvSBzKtejL2WFWSWuyZQJWrstuY6cJI+y+lIlG0tmak9ICiKvg6W3caiaJAKBIRj8w2J1EAGVE/abKM4vcn5VwEQcBWXoHSEcQ9+yoEVzHBpXej+Kx5EzWHvNhFO2UplrKELmO2YvLiFQWRKldl5mwUcpKobAMZVKXoNhaN3Xffk7fe+l9OPPGUtN/ZH2yCDQ1Q0qxNcfHFl/P8c6+wy8BByQmrIOh2nwzV8WgO+SixF+NK0f06HTgkfS9nKl5Hbm1BKi7p5n7dG2zlFaAoKB3p53tq6WxF1dQCodgZoQlYslFAZrgXua0NNC0p5wK6mCu3tiAWV+KZfRWCZCf05u9R27abHrcplibAzPc2VE9WYilq3JWoGVM9yf1GZRsQ4qqU9Mc1qs7ZhNxobo2U3JnI+ZQsJXsidAeAzKiezLpfp4vqDNoO5ZaW1M5kRXm8fbroUtXl3+MJCoSiB8y4x0KXmNuUAe6ly2iWXDzXxVy9vVhai3v2VaBpBP/xe1R/s6lxrRxiq8Zs0A+xrCmomUigpihJ1SiQ6O6Z/sUnx4hcNmoq9IY4ochAOdeu9B0prJkt/TQehhqmKceEIpMJO1OxG0LXuTVUVekgl8b/VFAgFAmwYqNw2ZyUOIozI1HELn57EhsF6K56SoI+VKoYjHvWFWjREME370INprZZNU2jOeQzbWRMR6KodlfRGPbREWhLW5+rKUpcrdQf4jr3DKhS4hJFFiOMEyEJIqIgZKQsqian5k4MsXxPaayXpmkEAu1INjutnW055Y6r3VW0RdqJKNG036XbDVOR8g1CkRmJwibaKHNmLm9UOih4PSVAVz1pCCYT/GWKe5FbUjOaga56MjLNSiV6gj2pehiemZcTfPMuQm/ehWfObxFc/Rdxaou0E1WjpjkXKcGYbfa6rHZX8tTGvzGyfDjBgPU8WZoGcqsX0eVCUkLdnomi2E1i0TSQAy2ISggplB7H1xEJEFVlGoPWLtEd55YKgp3thIVWIo70ArqUYAg1FMTWLPRgiHacV7xtvWo5cM1mc6A4RdPu1+nCIErNIS+Di60HpaZqN4RYdLYgZET11BzyUeWqzFoKe7MoEIodYVL1BDr38n3LhrSHlltbEGy2PnPeJyKRezEIBYA0YBTuGRcTWvYHgkvvxnPUVQiOvgN2zKYXN2DsX0XVsJvcyzXuKoJKmO20su/ASeY6J0AJBtlw/Y1Un3ASlTNmdh+jl+ItP97/AK5hwxh43gWWxwT4w+oHEASBSyf9ylJ/K4VlXvv6SRpDzVy/3+WWxjTQ8MxT+Fd9xtB7errv7jivthUf0vDEo+x2+13Ya6yX4vy2eY3+/hRrnWQC1Z4u22E6hEJu1+2GqRAKwWZDKinJSBqP5pB3p7FPQEH11A3xFB4muadqdxVtne1E0xRz5VbdaJbK+IYLbbQX7sU2ZHfcR1yI6t1CcOlitEjfrrtWjWZSzMvIiuqpylWJgJC2sTHViFkDugNAZg5xrnXH1e5KvCFv2t5icmtLPNNpMhgBZumqUqwyI+kgU3Up4jEUKaieIDNBd5qm5dymkwwpE4r29syl0t5ZYcXrCXQOWUNLu3JbqkYzSK4Pte06EdcRF6A2beqXWDSHvAgIVJr0b7cacAdgl+yUOUszd4hTXbOK9A9xRInQFvHnPFFbjbuKqCrTHknP9VJuSR6nY8Bol+6aNYe8OCQHJXZrteytoMjmwW1zpe1kYsbBRG9XnjZh9Uc7iCiR/y5CsXHjRmbNmsXs2bNpaGhg5syZbNiQvpplp0QsjkI0KWhlKpYiWfqORMSzVfbDIduHT05KLJpDXipd5aY9eIywBSsSBWTGrmP1EKdjQM8Hd5w4nlElzirk1paMMSOpoinkpcZdlbEstKlAEISMZJE17IbmmJH0pFazRcRygaQ34q233sq1115LVVUVAwYM4LTTTuOGG27Ixdxyji6vJ/PGbCDtAB8zEoWuDy1NeoiTEQurIq7VgDsDmfBzN9RIKRPX8grdASAQsDxmvlI/Z0KVoikKSnt7yuslejwIdnva6rp8qOogM+nG5dYWkCSk4tSkIT0Q1o8ata6GNpiB/yqJorW1lalTu9I/n3rqqXRkuC7szgTBQmriYnsRTsmR1sWnhEJoneGU1QKgcy9RX3Jurz9i4bVYI6DLPdZ0V0A/BP5IB2HZenEcubUFsagIsY+aETuiS5VinaB78+TfXuWqSDulvRHQmeoeMzIApLNeqqbiDfvykiq7xl2FN9ySll1Hl8DKEZJF/sdgEOH+JP1kaA77EBCo+m8zZnd2dsYvz6ampswESu2E0ATNVOEiA4aY60vDRiH79AvAXpH65rBVVqZ8iHsjFmE5TEc0QLXL/IYU05Yo9DF9YetqDdnrxV5pYr1iaxv1Wf+dmsM+XJKLohynfpZEiQpnGd501iu2x2wm91g669Ue8RNVZUt7LF1UuypRNCWtUsWyz4e9MnWmwBZrG02LGfFR5izFnqOAzlSQ9EY85ZRTWLhwIV6vl8WLF3PSSSdxyinZzW2TLxheT1Z8lytdFXhD6RxifWPZTF58xuFPBTsSi2Z/PQBVmcCvewAAIABJREFUbvOJ2tIxZkNXcri0CEWLz+Slpx9iOY2LzxtqsbRemUClqyJNQqF/b1PE1QQz0huM3zcf3HFl7HdKd81M7bFYWzPnckd4w76cJE80g6Q34gknnMDFF1/MnDlzkGWZW265hXnz5uVibjmHJqDHUVgwulW5KvS6DhYNpVELhMJeWYUaCqGGQ8kbG30SiEX9R48DUJmGRGHVmG2Mmc4hjvp88cs/FdjKykCS0r74qvLAHYPuVpwOYTW4XDNrZq+s0jOiWpQcDeYpHxefMabXou1QU1WiLT6TZ9IgFOkxI1bOZDaRlFDcc8897LPPPlx55ZVcffXVTJ06lVtvvTUXc8sLrLjHgr4pw0onQTn1SzsRss8Lopiyjzt0ERWzqgGDWHhjRrNKof+smL1BSiPXE0Cpoxi7aLPsUqx2dqIGAqa4Y0EUsZWXE7XI7Wma7gKdN4nCXaHH61hM5SH7fHoUuyd1tZmtshJU1bJB22AE8nHxVaQptSr+dlAUU3tMdLkQPUWWVU+KqtDa2UZ1nvZYX+hTCXbffffR3t7O0qVLuxmvo9EoK1as4Lrrruur638tugLuLKie3AaH7LOkv5Z9Pt1olkIOHgNd3IsX5+AhpsazD5+M3/c1dt9/kJbdhzr7KsSi1DenaLFmtgFB0GM3fBbVdXF9u4lDDDEO2SK3F5CDdCqRPEoUFWhotIRbqfWYr38R9XlNr1eiKsXMhWnAF/ZR4ijGIZlnRtKFXbRR5ii1LLVGveYlML19JbLXGjPS0tmGhvbfI1HstddelJeXI4oi5eXl8T8DBw7k7rvvzuUccwZN0L2eRIsSBWD54oua1LdDguHM4sXXatdrfmuBFoKv/w9qe1PKfeM2CsV6TEI6OvcuVZ2FQ2xxvQwVhtngxEwhXbuObFJVB2CvSs+u4w3lT1UHuv3N6h6TW6wyI9btOoZDzM5mo+hTopg+fTrTp0/noIMOYsKE9Goy/7dAw1A9mZco4vpQq5vS68U1fLipPrbycj0JWRoXX1XJADyzTyb41mKCb/wP7tlXIpUPTto3XYkC9DXb4t9mqa8VLzHQOeRoy+doqpqyy6MB47fN1yHusutY+71lnw/XsGGm+sQ9xSxefN6wj11LdrHUNxOodFWwqe0nS327jP9mmZEqQhvWWxqz2bDp/LeongyUlpZy6623EgwG0TQNVVXZvHkzf/vb33Ixv5xCA0SLqie3zY1LclkiFJqm6R48kyab6idIkh5tbJVQhFsYXjYMqXYEnqOuIbT0LkKv34571hVI1f1fKHFCkYZEUeWqpCMaICx34rI5TfWN+nwgCKbiTiCmrlMUFH+7KXsQJHrw5OcQlztLEQXRktSqRiP6dzZJWCWPB9HlsrTHVE3FF25lYs14030zhSpXJV80fo2qqaa9GWWfD8HhQCwqMtXPXlmJGgigdnYiOs3ta18shqLCaW5vZhtJV+7yyy8nGo3y5ZdfMmTIENavX8+YMWNyMbeco8s91rzqSRAEqtwVlmIpFL8fTZZNi7igcy9WjLMhOURQDsW5Y6lqKJ45vwPJTvAfd6I09M8RxegEShoSheG+aEWVIrf4kErLEGzmfM3j6jqv+d/JG2rBY3PjtrlN980E0omlkGOBmWZVT0YfK3usPeJH0ZS8csdVrgpUTbUUS2HYdMw6t9gqrbvIesMtlDvLkHJU6yRVJCUUgUCAm266iWnTpnHQQQfx+OOP89VXX+VibjlHl3ustR/JaixFXI1ihVBUWNOH+sK6F0uif7tYPhDP0XoNi+CbdyHXremzf1c9CuvBl+no3GWvz9p6GYe4xfwh9u0E/u1W7TrGHrG6ZlYkCuMs5NMwm04shdziw15hjbCCNdthPuN0+kNSQlEeC0kfNmwYP/zwA6WlpTlN7pVb6JHZViQKsB5LETfMWtiU9tghNjumYZjd8eITS2rwzPktYkkVoaWLiW78vNf+mTFmW4+liLaY9+CBLn2zlYuvOdwS927LF6zGUlgJ6DRgt0oodgLDbDqxFHqcjoX1MjzFLDFw+TX+94WkhGLYsGHcdtttTJo0iWeeeYann34aWc5MwfWdDXpSQM2yRGE1liJ+iKusqZ60aBTVZP6tLv/2nodYLKrAM+d3iDXDCb/7AJHv3uvZJgPGbKuxFJqmWfLgARCLihAcDtMXn6Zp+EI7gURhMZYiaiF9hwFbZRWKvx01GjHVz9fPHssVrMZSaLKM0tZmTR1cUWHJyURWZVo72/K6Xn0hKaFYtGgRU6ZMYffdd+eEE07gk08+4eabb87F3HIODc2yjQK6x1KYgezzplzZbkd0Bd2ZU6V4wz4cop1ie++GOsFVjGf2lUi7TqDzo6fpXPVKN6kl3YA7sB5LoQYCaJGIaY8nY0w9f5G59eqIBoio0bwf4sRYCjOQfT6kkpKUEygmoiuWwtzv5A35KHWU5CWGwoDVWAq5tQU0zZKqTrDZkErLTO+xlrAeQ5FvZqQ3JCUU559/PjNmzABg3rx5/OlPf2LyZHPeOf8t6CpcZF2iAPOxFFFvM7aqaksqPcPPPWoywMcX0tUo/Y0p2Jy4j7wI+9ifE/nidTo/fAJN1etEC2mm8DBgRece9TYDYKuylsHVXlVtfr3y7BprwKpdx9hjVtAVS2GWGWnJ+3qB1T0Wk8DSWDOzQXdxVd1/o43C7/cTDPZdSvP/EtJJCgjWYymiXi/2aqsbUu8nNzeb6ucNt1CdwiEWRAnnQWfhmHgU0bX/IvzOH9HkSEYkCuiy65hBNPZd01kz2WtuvYxaI/lO/Ww1liLqbY5f+GZhrHO0OfWATND3WL4lMNAvXp9JG0V8j1klFNXV8Xekii5mZOezUST1LXS73RxyyCGMHTsWT0KOmD//+c9ZnVg+oMUq3BncsllYjaWQm5tx7WouEMqAWFSE6HLFuexU4Q23MKJseEptBUHAue/xCJ4yOj9+ltDSuxEO/jWQvkRhJZbCuOTTOcSK32/Kz31n0LeDtVgKTdOQvV6K95poaUxbRSWIoqk9pmoqLeFWJtXmP1i30lVhOpZC9jbrcToWVE+gSyLR1atMBXZ6wy2Igki5s8zSmNlEUkJx/PHHW375G2+8wYMPPogsy5xxxhmceuqp3Z6vWbOGa6+9lkAgwJQpU7jpppuw2Ww0NjZy3XXX0djYiMvl4u6772aXXXIR3RnzejJZj8KAlVgKNRxG6fBb5o4FQcBWXWPqEAejIUJyyLSI69jzCAR3GeF/PoSw7PeUCfunLVEkxlIMLh6YUp9oczOi2206EMqAzeCQvc0p58jyhlvidZjzCSuxFEp7G1o0ir26xtKYgiTpdh0THHJbZzuKpuSdsEL3WIoKV2qBbNHmZmzlFYh2a/YVe3UNKApya2vKdg5vaOeMoYAUCMVxxx1n6cUNDQ0sWbKEV155BYfDwcknn8x+++3HqFGj4m2uvPJKbr31ViZOnMjvfvc7XnjhBebNm8dVV13FjBkzOOWUU3juuee4++67ueeeeyzNwwyMiIB03H/1WIrUCUVcF2qRUIB5MTcd7tg+cl8EVzGh5fdxWdlStvkHAald8L0hUeeeMqFIw6YDXZJItNkMofDFiVq+YVbnbuwNq/p2iNl1TOyxfKc7SURVght26oSiyTLzBt3VdakSip0hTqcvWGOdU8DHH3/M/vvvT3l5OR6PhxkzZrBs2bL4823bthEOh5k4UReH586dy7Jly/D5fKxdu5aTTz4ZgF/+8pdccskl2ZpmNxi8sWTRRgHmYymiXl3va1WNYvSVvc0pj5muf7ttyO645/wO7f+x9+ZhcpTXvf+nqrp679l6enZtoxVtLAJJLBYIbMA2RFgGnMSxsB0Tcq+vnRCb2Ak2wYEYJ+Ya+9r3+sbcm/iSgM1mI4NjlmDAGCQWsWjfNaPZp2d6tt6Xqt8fNdXTs/Re3erhp+/z+DGaqq737erzvue855zvOarAqoM/Id5ZOAGzEC5FbGjIkEWcT5zCF6qMwCzkz6UoNqYD+cd1hiskpgMppLu8DLihgpMlICV2mM87q5CYzlwomaIYHBzE45k66jY0NDAwMJD2usfjYWBggK6uLlpaWvjOd77DJz/5Sb785S8jF3j8yxfqpIFaDJ0wXy5F3KBFrIRCKDkmHSStvSIWseRewPfGP0bIWk/o+R8Q3f9CQc/Jl0uh+duHilKsUlU1giznbCFrfSgqhwiVL5ei2JgOaPIZHx1FicVyuj95aq2AmkX6HHJVrmoiQXxkpKg1qXOicpWxmBJnLDJeEYp1LpSsKauiKNNcA6qqTvt3uuvxeJyDBw/ypS99ib/5m7/h8ccf5+tf/zr/9m//lvPYbrezoDmLkjYfp8uGx5M/pwFgSaQVjoNijeCpa8x6vz84jmg207S0LasrJd2cxPYFeAFnIojTk919E+ryYzVZWNzcWJSbzS842LvsT/lw+FmCrz+MJTaK+8O3IOTpY/U43AQUf07vPDY+gRIOU7OoNev9ma53N3gQJ0ZzGnM0PE5MibHI01ywXOQzt2xY7G9GPaUi2mN4XNkt0LHAGHJ1FY1t2Te+dPNSl7QxrKpUCRFsnuybWeCUn1prNS1Nxmx8xb73Wms1AXKTsfDAICgKdYvbipKxztpapMBYTmP2TwyiorLYIBkzSk51ZFUUn/nMZ6ZtJoIgYLPZWL58ObfddhtO59ybclNTE2+//Xby316vl4aGhmnXvd6pdLuhoSEaGhrweDw4HA62bt0KwHXXXZd3R73hYX9B2TiJybpFIX8Er3ci788DmKJasPNEXzeuRPZFPN7Vi1RXx9BQZma1x+NKO6ewWQvqDh7rJFSVPWDZPTJInaU265jZIAoCcUFG3PJfkC11jL/1a4KDvViv/HMEOfeqmdVyNb1jgzm983BHBwARa/r3AZnfF4BQU4e/tz+nMU+NdQFgjtsKlot85pYN5phWlPBYbzdSXfYChRPdfYi17qxjZppXyKyt84GjnThM2Q2xntFBasw1FfG+AGrMNfSM5iZjwaMdAIQtxcmYVOdmorsvpzGP+boB7bct9rsW8r5EUchoYGd1PS1btgxZlvnMZz7DLbfcgsvlwm63Ew6Hufvuu9N+7pJLLmHXrl34fD5CoRDPP/88W7ZsSV5vbW3FYrGwZ88eAHbu3MmWLVtYuHAhTU1NvPLKKwC89NJLrFmzJtfvWxR01VIojwLy51LEhopzo0D+/tDhsM8QX6goajwKQRSxXvxHWC79E+Kn3yP49LdRArn70PPhUiRjOkW4BfTP58o9mWomUxlugXy5FMUGZiF/LoUvdOZaxs6FfLgURgT/YTKuk6OM6b9lpXW205F1R9y7dy///M//zEc+8hGuuuoqvv/979Pd3c2dd97J4cOH036usbGR22+/nR07dnDDDTdw3XXXsX79em699Vb27dsHwP333899993HtddeSzAYZMeOHQD88Ic/5P/8n//Dddddx0MPPcS3v/1tg75uZqiTqkIsIkqRL5ciNlxcYBZAtNsRbbac/aG+sDEVKiVRmHZyM6/5MLZr/gJltJ/gU/eQGOrI6TmpXIpsKJYIpUN215Pwa26sbJiqi3Xm/e2QH5dCVRTiw8NFvy9TTS2IYk4bn6Iq+CKjFRWYrbPW4ouMoqjZqx3HJjkUhZTvSIVcX09sxIeaSGS91xfSORRVRY1ZKmR1PU1MTEzLplEUJcnUFrMQSa6//nquv/76aX978MEHk/+9atUqnnjiiVmfa29vzysmYRT0YHahtZ4gPy6FEg6j+P1FL2JBEDSCTw7WnsahCBtzohCEWTwK08LzsP/B3xJ67gcEd34b6xV/irx0U8bn5MOliA9Pcijs+fclnzbPVC5Fa2aOznB4BIdsx3qGORQ68uFSJMbHUOPxoo2RfLgUY5FxFFWpmCwxyI9LEZ/kUOTb62QmTO76SS7FSNY1PhweobZCORSQw4li69atfP7zn+fJJ5/kiSee4Atf+AJXXHEFv/rVr6gvUvgqDfqWV0iHu1Tk2pci5tXdKIURoVKRK5di2MAyAeKME4UOqX4R9u13I3kWE37xx0TefAI1gyWXT/2imFdzoxRb6l6ub0g+LxsqKTVWR65cCmNlzJOTMWKkjBkFdx5p2Ea46gDkyazOXGSsklNjIQdF8bWvfY2Pf/zjvPjii/zud79j27ZtfOUrX8HtdnPfffeVY45lQ1JRFJUgmzuXIuodBEBuyJ4dlQ3mhkZiQ17ULI2EjGzeLoqzTxTJa7YqbB//a+RVW4i+9wyh5/4HanTulOF8uBRR76BB70tXFINZ79UWceVsepA7lyI6qMtYQ5Y7s8Pc0JDb+5qMBVQKQRHy41JEBw2SMU/uMlapfSh0ZD1biaLI9u3b+ehHP5rc+MbGxrj00ktLPrlyw4gYBUznUjjk9C6S2KDGK5EbDLD2PA2o0SjxsTHkDH2kk/52AxaxKMx9otAhSCYsH/ocYt1CIrseIbjzHmxX/wVi9fRFmCuXQlUUYl4vzvMuKH7uDgeizZbcSNOOqar4wj7WulcVPaaRqLPVMtavcSlkMf0yjnkHNH97ke5N0GQsMTFBIhRCsqXPtqokDoWOXLkUSiRCYmw0eRooBqa6OpCkrDKmcygqSbHORNYTxc9+9jMuvPBCNm/ezMUXX5z8/w8ilEn9UAwzG3LvSxHzDiI6HEj2wmoWpULO0UL2hUawSGYcpuJ8/KAFsxNZTjCCIGBe+2FsH/sqSnCMwFN/T7z7wKx7culLER/xQSKB7CneOhYEAdmT3UKeiPmJKfGKW8S59qWIDXoxud1F+9uB5HvP9s6GwyNUm13IZ7APxUzIkky12ZX11BrzGncCEyRJK32S5X2NhEcrtg+Fjqw74v/9v/+Xn/3sZxw6dIhDhw5x+PBhDh1K30v5g4BCiwLqyLUvRWzQa8imBymLOIv1ojOMjWhnq50ocrvX1Loaxyf+DtFeQ+g39xPd++w011wuPnf9u5kNWMSgbQZZN71Q5dQsSkWucZ2YdzDpAikWSWMkBxmrNFcdaC7OXBWFYe/M48n6viql10kmZN0R6+vry8ZjONPQXU/FxijqcuRSxLyDmA3whYLW1xhRzMHaM4ZDAXqMIkdNAYhVDdi3fQPTovOJ7P454Rd/nIxb1OXApYgaaO2BHtcZypi+WGkcCh1TMpb51KrFdAw2RrKeWiuLQ6EjFy7FVEzHoHXZ0EjMO5gxXlnpHArIQVFcdtllPPLIIwwMDDA6Opr83wcRU0UBi1MUdpMNq2TJqCjUeJyYb9iQ+ARo7Rdltzv7IjaIQwGZg9npIJhtWD/y3zBvvIn4qbc0vsVIL25rbVYuRWxwECSpoL7Pc0H2TJaCztDbuNI4FDpqLNVZuRSJYFBLvzbIOpZsNiSXK6OMVSKHQkcuXIqY14todyAVWMJ+JsyeBq0OWyCQ9p5K51BADsHsn/zkJ0Sj0Wl9sgVB+EC6n1Rh8kRRYOMiHUmfewZrLzY8DIpi2CIGzeLLFDgzkkMB2YPZ6SAIIpbzPo7kWUL4xR8TfOrvqd7wESAzlyLmHUSu9+TcCCYb9Hcf9Q6mDV5WGodChyRK1GThUiT97WWUsUrkUOjIhUsRM/AEBlOn3+jgILY05Y4qnUMBOSiKvXv3lmMeFQHdzhCL68UDTB5zMwQaY97JjCcjF3FDI+E330h73ej8dqmAE0UqTK2rsW//FqH//J843/sNtNXhCw2nVxSDgwa/r1RXytzu1UrkUOjIVvrE6JgOaPIaOnY07fVK5FDoyKUvRWxwAOviJYaNOeWuG8DW3j7nPZXOoYAMimLnzp1s27aNf/3Xf53z+uc+97mSTepMIVlmvMisJ9D8jcdGTs2qmqtjahEb4wsFzZWiBAMkAoE5j85GcihAq/Wk5NgDI+0znHXYr/8bGl5/CGKH6X/7MVZffjuiffpCVlWVmHcQ2/IVRY2XCp19q6cpz4Xh8AjNDuN+IyNRZ63lyMjxtNdLcqJoaGDizd0osdic3d8qkUOhQ3cfasp1tjJQ43Fiw8O4Nm42bMwk6S7DKcwXHmFV7XLDxiwF0iqKzs5OAI4eTW89fFBRrOsJdC5FmFA8hH0OLkXU60Uwm5GqjeuPm0oikxyzF4KRHAqYZGYnij9+CZKJ+ss+i+mlv2U4NELwF3djvfI2TC3nJO/R6zIZ6RYQRFFLkR2cmzlbqRwKHW6r1pcirsQxzcGliA4OIlVVIVqNc5uZGxpgsieIual51vVK5FDoqNUTANLEdWI+n+HuYNFsxlRbmzauMx84FJBBUXz5y18G+MCxrzNBj1GISvEnCt1qHwr7WDiHoogN9CN7GgxJU9WhZ2pEB+Y+PhvJoYDJGEWRJ4qpZ4nU2esYr26H4ycJ/fqfMF+wDfP5f4AgisQGdHKicYtYf140zYmiUjkUOupsdaio+MKjNNhnE+pigwOGZe/oSJWxuRRFJXIodJglmSqzK22mWGywHyiBjHkaiA7MLWPzgUMBOcQo3n33Xb73ve8xNjY2LcXr6aefLunEzgSmemYX/6xkobvQCAtds4vORQcGsLRlLkaXL+SGBhAEov19c143kkMBxccoZsJtrcMXC+LYfjfh3z9EdM9TJHoPY73yNqID2iI2N87enIqBuamJ4IH9qIoyK0heqRwKHalcirkURXSgH8e69YaOaW7U4kexyd9jJiqVQ6HDnYGvE+3XNvO5FGAxMDc1439nz5zX5gOHAnJQFHfddRfbt29n9erVhlq/lQlt0yuWmQ1Qn6F+kRqPExvy4tpwYdHjpEKUzZjc7qT1PRNGcihAy+4qJOspHdzWWromehBkK7atf0asdTXh3z9E8Mm7iCRWgSQZUqwtFXJjE2o8Ttw3PKtwXqVyKHS4M/SlSIRCJMbGkhu7UZCcTkSnk2j/3IrCF/KxuHqhoWMaCbetjo6x03Neiw70IU6mABsJubGRhH+ChN+PNCPzaT5wKCAHRWEymT6Qgeu5oCSD2cU/K1NfitiQFxIJwy0X0Cy+dCcKX3iEZTXGZXRIokA0kTvhLhtS+1JYTRbkFZchetoJv/i/CL3xFianAwTjFBNMWY/R/v5ZiqJSORQ6MvWliE3KgLnJWEUB6WVM51BcYD3X8DGNQp21lncG96KoyqwGZbH+fsxNzYYbxEkZG+jH5lw27dp84FBADopi+fLlHDlyhJUrV5ZjPmcUU9Vjiz9RZOpLoVtjcikWcVMzY8ePz8q2MppDAVowOxiIc3qg+HaXAMpkG9kDPd14rLqf2AWX3E789W9iEgOMPnkvgQs/i2J3Z3zWRFRhZCQ9yUmHKk62+DzeicmzeNq1Tt8gNsnG4HAMiOX7dWahodaG1Wxcm/pMfSl0V51ssKsONBkL7N836++VzKHQkYlLER3ox7bS+MQFXVlH+/uxLZ2uKOYDhwJyUBRdXV188pOfpKWlBYtlqg/yBzFGoRp4ogC9L8UciiLpby+FtdeIGgmTGBvVupJNohT57VazRPegn7v/9S1Dnic4RrGugR//x1soY1MBRUFV+MpEmGO1bVww3IP5uX/g8eAm9kTnzkvPC6rKX4oyB17dxwuHpwf5zSs6EWTZsO937lI3f3GTsdZ2uhpZ0f5+rWqsAVVQZ8Lc2Mj4a6/OqiJbyRwKHem4FEokQtznK8malN31IElzxnXmA4cCclAUX/ziFzGbzeWYyxnHVJlxY9wbbmstR0dmW/exgX4kl8uwMgGpkFNcKamKwmgOBcAfXbWcD29azPj43H0m8kUw4ednfbu5YlMtq53rkn8XRn2YTii0bLyY/uVLaTr0M3aIv+cTDX4Gl29HkWeXvK6qsuU8L/mhlzjXqrBy+7ppf3+i/01q5UaumvH3QvCr104xHiz+VDITbmsdh3yzU9hjA5orbS6uQ7HQZSw2MIC0eHHy75XModCR2pci1Q2rc2lK4aoTTCZkjydpIKZiPnAoIAdF8d3vfpedO3eWYy5nHMkThUEpn25rLZFElEA8iFOeUgrR/n7kElguMHVKifb3YV81xUMwmkMBUO20sGxJPV6vMa4nVa3niQETdleMC5ZPWcKB/X30AMvWL8O+YiXquruIvvdrXHueoipwGuvWW6dxLgA8HlfO8+pbtIDQsaOsXjE1pqqqPNQ7zob6NdPmUih+v7eP4fHs/bnzRZ2tlrHo7L4UZZGxgT6sKYqikjkUOvS5zUwA0N3BpThR6M+dmQAwXzgUkENRQJvNRn+aDIcPGlSDk7rck30pZgYbo/19JbFcAEy1tQhm8yyhNJpDUQpoNbJml4JOLuJJS1YQJSwX/AH2bd8Ak0zomX8ivPtR1ERhFru5qYm4bxglMlWQUOdQGOVGMZlE4gYG/nXo2XUjKe9MVRSiA/0lk7GpNOzpMlbJHAodWl+KqjlkTAvOG8070WFuaiI20D+tA+V84VBADieKUCjEVVddRVNTE/aUhvYfxBiForucDEr5TG3xubBK40wkgkES4+OG8wF0CKKIubFxlj/UaA5FqeCeo5hidKAf0W6flbYoNbTj2P73RHb/jNje35Do2Y91658j1bXmNWaSGzA4iGXBAiCFQ2GQtSdLArG48YoitaR9g107+cRHR1Gj0ZIpClGWtR7tc8hYpad5gl5ufIaiGOjHVOdGTInDGglzY/OsNOz5wqGAHBTFnXfeWY55VBTUDGWI84E7uYinNr4p67h09YPkxmYiHaem/c03T4JmdbZaOie6pv0t2teHubFpTiUnyBasH/ospoXnEn7lXwj+8u+wXHQT6tZP5Dymnn0W7etNKoqkG8Wgd2aSSnOicKcQO3Xo1nGp3CigyVi0r3fa3yqdQ6GjzlrLqRlcimh/f2nfV1LG+pKKYr5wKCAH19PGjRtZsWIFCxYsoK2tjebmZmIx44NylQBFz9E3KEZhl23YTNZpFT6jfT0AmFvys3rzgaWlhdjw0DRXyrCBfShKCbe1lkAsSDg+5c/cRyvnAAAgAElEQVSP9vVmfV+mRedjv/FepNY1RHb/jL5//zuUsfTF/lJhbmoGQSCSsvEZrihMInED6mLNRLVZ41KkulKivdr3KLWMRfunXCmV3IdiJtzWOkYioyQUrWGVqiiajLW2lGxMy+Rvkapc5wuHAnI4UfzgBz/gJz/5CQCSJBGLxVi2bNkH0vWk6harQYoCZqfIRnt7JrMgjK0nkwpzSwuoKtH+PqyLFk9yKELzZBFPuVJanc0k/H6NYdySfRGL9mps1/wl8WOvEd31CMoT38Sy8UbktR/OWBFYNJuR6z1Ee3uSfxsOj+Aw2bEZ1IdClsSSuJ40LkXN9FNrb4/Wi72qdBuQuaUFNRYj5vVibmycFxwKHTqXYjQyjttWS9w3jBqJYG4unWKVnE4kVxWRGTI2HzgUkMOJYufOnbz00ktcc801PP/889x3330sW7Ys28fmJfT0WCMVhXtGcDba24u5udmw5jtzQbckdcvSNw/y23Xox3B9zroFZsnROhYEAXnFZbT92feRWlYR2fUIoWf+EWU8c+c/c2vrNGtvOOwzNBulVK4nmKxflOp66uvF0tJa0niUeYaFPB84FDqSddgmlWukNz8ZKxTm1tbkmoT5w6GAHBRFXV0dDQ0NtLe3c/jwYW644YYPbOlxvYSHmkcf6Gxw2zRClF5QMdLXW1LLBSZ7XEjSHIu48oWyfjJTTN/4Ikk3Sn5uAVOVG9u1t2O9/E9JDJ0m8MQ3iO5/IW38ydzcQnRgADUeB4xvWGSStAKKRlXbTYXbVpfc9FRVJdLTk/f7yhfmZu35+ilsPnAodLhn1GHT14m5uTQJJjrMzS1E+3qTe4FvMsFkPiCrojCZTJw+fZr29nbefvtt4vE4kUj6vsbzG6U5UUQTUQKxIEo4THxoqOSLWDCZMDc0Jo+5vhJwKEoFp+xAFuWkKyXa24NgsRTUJ1sQBOSVH8Jx0z8gNa8k8vrD2ulibHa6t6WlBRIJooMDqKpquLUnm7SlFi+B+8ltrWUsOkEsESMxPoYSDJTcGJFsNky1dbNlrII5FDpqrTUICFOKoqcHqbp6VsE+o2FpaUEJhYiPjMwrDgXkoChuu+02vvnNb3LFFVfwwgsvcMUVV7B5s3EdoCoJSfVgcIwCNFdGMhulxEdcbYyW5DF3OOyreA6FDkEQprX41Fx1LUW56kRnHbZr/wrrls8nTxeRd59BVeLJe1Lddf5YgJgSM9Tak6VJRVEC95MuY77IaPI3t7SWW8Yqn0OhQxZNVFuqkpliuquu1Eh1180nDgXkEMzeunUrW7duBeCpp56is7PzA1sgMOl6Mig9FqYHZ2t7NaawpcQnCtCE0v/OHpRodNKNUvkcCh11tqn6RZG+Hhyr5+5nnQ8EQUBetQVpwToir/070beeIH7iDaxbPofU0J7MfIr29TKxQktdNjJLzDR5ooiVIPMpldhpTrpRyiNjY6+8hKoo84ZDoUOrkeXTXHW9vVRfelnJx9Q9CdHeHnzNGl9jviiKrGZaIBDgW9/6FrfccguRSIRHHnmEYDBYjrmVHXqHO6MId5Byogj5iJQh40mHpVnLfIoN9M+roBlMJgCEfCSCARKjo4a6UURHLbarv4T16i+hRvwEn7qH8OsPI4gqsrueaG9PMj5i5DszSaV1PYHWTTHa26uREw1ssZsOluYW1GiU+PAwvpBvXqRf69AbGMVHfKiRcMndwQAmVxWS00W0r3decSggB0Vx7733UlVVxfDwMBaLBb/fz1133VWOuZUdpUiP1bgUNnzhEaK9PciNTQhS6dPhzJOuh0hP97zhUOhwW2sJxkNMdGmkwVLkt8uLN+C46dvIq7cS2/+fBB6/E7nWSaSnx3AOBZTW9VRtqUISpKSMmUuc8aRDl7FwT9e84VDocFtrGY2MEe7WyJ3lcAdr47RoMjaPOBSQg6I4dOgQt99+OyaTCZvNxv3338+hQ4fKMbeyQ0+PVQ3OTKmftF4i3V1Y2hYY+ux0MDc2gSThP90xbzgUOnRXyuipYwAle2eC2Yb1sh3Y/+BvEWQLgv+UZu2N9BjKoYAU11MJThSiIFJrrWE4OEyk63TZZEyPg4x1HJ83HAodblud1peiQ5cxY9sSp4OlbQGR7m6GQ755w6GAHBSFOCOImEgkZv3tg4IpHoWxi7nOVsf42BBxn69si1gwmbC0tBA83QHMj/x2HfqGE+zqQLTbC8p4ygdS03Ls27+Fbe1FoKqEDuymVpWmBbuLhUnSLPxYCbkUoaEBlFCobJueaLUhezwEuzon5zB/ZKwuKWOdmOrcSHbjS/7PBUvbAtRImJC3f14Zb1l3/Isuuojvfve7hMNhXn31Vb70pS+xcePGcsyt7FBK4HoCbRGLfRrhy7KgPIsYwNy2gERPT3IO8wX6Akr09GFpW1AWN4ogyTi3aPWhpKBI9ZiX4JN3E+87YsjzS+l6Au33Ffq9QOlOYHMhVcbmS6onTCm1eE9v2RQraO8LQOzzzivFmlVRfPWrX8Vut+NyuXjggQdYuXIlX/va18oxt/KjBMFs0Da+Gp9Wu8jSVr6iaZa2BQgTAWxhZV4tYqfswCyYMA2W7wQGWolpQZaxjifwtK5HjYUIPX0foZceRAmOFffsEvIoQAuK2if7b5Rz47O0LUAcHsMUV+cFh0JHrbUaUwIk7wiWBWVck62tIAjYvPOHQwE5KApZlvniF7/I448/zi9+8Qtuv/12Tp8+ne1jgFaK/GMf+xhXX301Dz/88Kzrhw4dYvv27VxzzTXceeedxOPTj/oHDx5k7dq1OX6V4qEvYaNjFG5rLfWjcXCUJxtFh77JNo0zLzgUOgRBYEHchRSNl1VRCJKEqbmZupEInoZzcNz0bcznXUf8xG4Cj32d6P7/LJi1r2c9lSI9FrRU3vrROIK7DtE6u+NfqWBpW4CgqiwMmucFh0KHSTSxIGRFUNWyyphosSB66qkfjc+rU35BwYZPfepTWe8ZGBjggQce4JFHHuGpp57i0Ucf5fjx49PuueOOO7jrrrt47rnnUFWVxx57LHktFApxzz33lLVSbbJxkcExCretjvqROPEmd1m5DLqltNAvzxsOhY6FE1qQTy/7XS7Em9zUj8RxW2sQZAuWjTdiv/EeJM8SIq//O8Ff/B3x3vyTOUwldz3VUT8aJ9FUX5Lnp4MuYwsm5o+S0LFwQqORlV3GGnUZ+4Arilws7tdff53NmzdTU1OD3W7nmmuu4dlnn01e7+npIRwOc9555wGwffv2ade/853vcMsttxQyvYKhJBWFwScKSy3usTj++tKWCJgJU1UVIZuJxrHSWLGlRMOY1kZKLgNxLBUBjwt7RKUuPtXARqppwfaxO7B++L+iRoOEnvlHQs//MGuhwVToWU8lUxSSi5qJBIEyy5hcX0/MJNAwD2WscVwlLgkl62qXDoF6FzX+BG6xvL9VMShIUeRinQ4ODuLxTPUabmhoYGBgIO11j8eTvP7iiy8SDoe59tprC5lewdAJd6rBMQpxeAQ5Ab6a8lpdqqoyVCMl4yPzCdXDIUZdEkHRuMyjXDBSawbAOaPftiAIyO0bcdx8H+YLtxPv3kfg8b8l8uYTqNFQ1ufKetZTiWIUFu8oogrDteWVsTgKQ9UmanzZ30GloXo4xHC1REwtr4z56rTfyOItLu5VTmQt4VEoFEWZplBUVZ3273TXvV4vP/7xj/npT39a8Nhud2GaWnc9uZwWPB5X5pvzgPfQewAMuIWinpvvZ0dCYwzUSLQdHcddY0WUS7OJGPmudNgHR+mqNRG3hPDUF1bVs5B5+RpkWgHryFD6zzd/mvjF1+J7+WH87z1D4vhr1G39NM51l6fteyHI2lKz2swFzy0T+vdopxtvvVRWGesd78dba6Kpe5R6t6Nk5fONfl+qqmIbHOVkswnFFsFT7S7bvIbqJRYBFt8gHs+GgsbNBqPfV1pFcf755895clBVlXA4u4Xa1NTE22+/nfy31+uloaFh2nWv15v899DQEA0NDbz88suMjo7y6U9/Onlt27ZtPPzwwzhzrO44POxHKeBUoH9iYjyIOMOiLAbe/YdRJJFjcgBvgc/1eFx5f/b4aCeDdSaEhELP+4exLlpc0NhGzysbEn4/qm+MwYUOjvV1Uat6sn/IoHl1hoZpr7IwfPAIloyfNyNc/Dns7R8ivOsRvE//iOHdv8ay6VOYWlbNuns8GAVgZFQrf2P0Oxs6cISYxcRJxssqY0eGTmsydjxE78GTmBuNd+OUQsZiQ16EYJjBOhdHe05jjea/sRY6r1PKGOusMkMHj2DaaHyNqULmJYpCRgM7raJ45pln8hpoJi655BJ++MMf4vP5sNlsPP/889xzzz3J662trVgsFvbs2cOGDRvYuXMnW7Zs4aabbuKmm25K3rdy5Up27txZ1FxyxVRRQGNdT+HOTqINtYzGtFLQ5coOGQoNM1hnmpxDR0kURSkQ7uwAYLBOxhsaLuvYQ6Fhwk21RDo7c7pfalyGfds3iB/bReStJwk98x2khedi2XgzUt1UWYgpHkVpfPnhzg4iTXUMh30oqoKYoaOfkRgK+ZIyFunsKImiKAWmZMzEUBllTFVVvGEfkeZaIpNzmA9IqyhaiyxT3NjYyO23386OHTuIxWLceOONrF+/nltvvZUvf/nLrFu3jvvvv59vfOMb+P1+1qxZw44dO4oas1ioJQhmq6pKpLMDYd0yVAYYDvtocpRnMQ2Fhhl3mhBttnkllPpcI011ZV3E0USUsegEtLUSO/ouCb8/px4FgiAir7gUU/tFRPe/QPS9Zwg++Q3kFZdhvnA7oqM2yaMoBTNbjceJ9nTDpjXElD7GoxPUWMqThj0UGmaizo5gmiDc2YFr46ayjFssIp2dIEkE6p14U1oVlxoTMT/RRBRaW4js2ocSiyLK5rKNXyhKFqMAuP7667n++uun/e3BBx9M/veqVat44oknMj7jyBFjmLG5IFnCw8BgdmxwECUUwrZoCTCANzRcNkXhDQ1TZ6vFsshNOEcLuRIQ7uxA9niormkoq6IYmtwwrIuWAO8SPt2ZV4lzwWTGct7HMa+6nMi7TxM78CKx429gXnc18rkfBUpDuIv0dKPG49gXLQH68AaHy6YovKFh3M56zG3mpJU+HxDu7MDS0kqt011mGdPGsi5eDL9/j0hXN7b29rKNXyg+mEWbCsRUCQ/jFrNuHdctOweY2ozKgaGQj3qbG+uiRUS7u5JtPisdkc5OLIsWU29zl9X1pI9V275ych4dBT1HsDqxXvxHOD51H6YlG7QTxs+/xlbbYRJx43lB+gZds1SXsfJufLqMRU53Gu62LQVUVdUUxaSMlfN9eYPaWHWTv9V8OemfVRQp0EXcSGEPd3YgmEzULFyGRTKXdePTF7Fl0WLUeDzZtrKSkfD7iQ15sU4u4omon3C8PK139Q2jvr4Nud5TtIUsujzYrrwN+/a7EesXcoPtTS7r+DET+142tC97pLMD0Wajvm0poiCWbeNTVIWhsI96ax2WRYtRgkFiKQkqlYq4bxjF79dkzFrHcHgExWCSbToMhYYREHC3tCM6nfPmFHZWUaRAEY3vmR3u7MDc2oYoy2W1XkLxMP5YAI/NnQxihztOlWXsYqAvHMvCRXgmy43rTV5KjaHQMDaTFYfJjmXRIiIdHYY8V6pfjO1jd/DT6DXERAveX/2Q4BN3EjvxhiHdFMMdmnVskkzUWmrKZoyMRcaJK3HtRLFwMQCR+SBjk3O0LFqEx+YmoSYYCY+WZWxvyEeNpRqzJGNduIhIZ+W/LzirKKYhGcw2KEahKgrhU6ewTvogPTZ32VxPSevY5kZuaES0OwifOlmWsYuBPkfrkiXU27Tc9nIpV+/kCUwQBKxL2okNeYlPjBvybEEQ6BAW8nz9Dho/eQcIIuEXf0zwyb8j1rGn4FOsEo0S6e7CuuTMyZjH5sbS1oYgy4Tmg4ydPKmV4V+wMClj5VKu2ilfM4Cs7e1EurtRIuU5MReDs4oiBVONi4w5hkb7elEjYWztSwFt09bTF0sNb4qiEAQBa/tSwifnwyI+gbm5BcnuwHNGFrE2pnXyNzPyncmSSFwBx6rN2D95D9Yr/xw1ESP8/A8J/vJbxE+/n7fCiJzuhEQiRcbKlymmZwvV29zaxrtoMeGTJ8oydjEInzqJZeHC5CkfymeMDIWGk3JtbV8Kk/GSSsdZRZECo/tRhE9oi8aaoijiSpyxiDFWaiZMWXua9WJbupRobw+JUOWWWlBVlfDJk8n3ZZft2E22sljICSXBcHhkahEvWgyiaOjGZzKJyawnQRSRl23GcdM/YL38T1EjAULPPkDwV/9AvPtAzgpDn59+aq23uQnEgwRjpf+dh0LDiIJInVUrL25rX0qks6OikybURIJwx6mkjNVaq5EEqSwyFo6HmYj5k8rJtmTSGDlR+cr1rKJIgdE8itDJE4gOR7LoWDkt5KHQME7ZgXWynad1STuoakX7kGNeLwn/RHIRA2WL64xExlBUJekWEC0WLG0LDD9RzORRCKKEvPJDOG6+D8uHPovq9xH6j+8Sevo+4l17syqM0MkTmNxuTNXaZu0po4U8FBqmzlKTbOdpbW/Xkia6cmtDcCYQ6elGjUaxTm7SoiDittWWaU1OncAAJJcL2dMwL05hZxVFCpJL0qgTxckT2NqXJkuhlPOY6w35kpsGkPRhhypYKMMntTL0thRF4SlTimyqv12HtX0p4VMnDMtQMpmEtNVjBcmE+ZwrcHzqO1gu+ROUiSFCv/kewV9+i9ipPWndobqM6Sinz92b4qqDqZNzRcvYpPU+852VS7HCbBkLnTxe8WnFZxVFCpTJt2HExpAIhYj29U6zjmst1YiCWLaNL3URSw4H5qbmirZewidPIFgsmFOqAtTb3PjCIySUREnHTo3p6LC1L0UJh4n29RkyhiyJWQl3gsmMee2HcfzhP2HZ8jnUaJDwCz8k+MQ3iR3fhZryHuKjI8R9vhknMO1EVK6Nr94+9b5MtXVINTUVL2OSqwpT/VTfDs+koij1Zj2XjFmXLiUxNkbcVz5+VSE4qyhSYKTrKXzqJKjqtEUsiRJ11tqSL+K4EmckPJrcNHRoAe0TFWu9hE6exLp4ybQKpPW2OhRVYSRS2vTFodAwJkGaxmhOBrRPHE/3sbxgksScO9wJkgnzqstx3Hwf1itvA1TCv/1nAo/9LbHDv0NNxAnNiIEBWE1WXLKz5DIWjAUJxkPTrGNBELC1L61on3vo5Ams7e3TCp7W29yEExH8sUBJxx4KDeMw2bHLUx0IbQbLWKlwVlGkQME4HkXo2FGYzDZKhacMx9zh8Agq6jTLBcC2bDmJiQliA/0lHb8QKOEwkdOd2JYvn/b3crlShkLDuG1104rpyY2NSC4XoeNHDRnDJIl5Ny4SRAl52cXYb7wX60e+hGC2Ev7dvxD4+V8T2P2fWo/vGcUey8Fon8s6Bk3GYkNe4qMjJR2/EMTHxogN9GNbtmLa38sV19ErJaTC0rYAwWw2TMZKhbOKIgWqqFkZRljcoWNHsSxYiGSb3r9YW8SlPWbODJrpsK3QFkjoaOUJZejEcVAUbMtXTvv7mVzEgiBgW7ZCU/oGwGTKX1FMzUVEXrIB+yfuxnbtXyE63YSOHEa2J4i+8xRKcOrEVV8GLoX+fM9MGVtewTI2uRnr60DHVOyw1O9seNYpXzCZsC1dZpiMlQpnFUUK1Mn/FVvrSY3HCZ84PksgQXOlhOIhgrFgUWNkwlxBMwC5sQnJVUXwWPkKLeaK0LEjIAjYlk4/gVVbqjCJppIuYlVVZ8V0dNiWryDm9RIbKd5CliWh6A53giBgWrge6zV/RSwkYG1pIPrerwk88lXCr/wLiZFePLY6RiNjxJTSpanqMua2Tt/4LAsXIVgsBCtw4wsdO4pgNs86genfoZTGSEJJ4IuMzlqToMlYpLubRLC0rq9icFZRzIAqUDQzO9zZgRqLJa2rVJQjRXYoNIxZlKkyT2/GIggCtuXLK9J6CR09imXhIkTr9BOYKIi4raUlkfljAcKJyNyLeIV2wgkZoFxNc6THForQiROgqriu/RMcn/oO8qotxI7vIvj431J1/C1UVIZLqFyHQsO4zE6sJsu0vwuSVLEWcujoUaztSxFM04tmmyWZGkt1SdekLzw6mX6dRsZUldDxyo1TnFUUKVDRFEWxrif92D3TFwrl8bl7Q0NJRvZM2JavID40RKyCsiyUWIzwqZNzKlbQSIOlfV+6v71u1jXLggUIFqshG59syp71lCtCx46AKGJbugyxuhHrZTtw/PF/x7zhE9T6tLao3b/9IbHju1FLcLLwpjCMZ8K2fAXRnm4SgcqxkBOhEJGu02llrL7kMjY0Oc7sd2Zd0g6SVJHKVcdZRZECo3gUoWNHkBubMFXP7gngsdUjIDAQLF2VzYGglwZ7/ZzX9BiAERayUYhkOIEBNNg9DAaHSlb6RP8tGuyzW64KkoRt2TJDfO5aMNsgMmfyBGZN/k20VWHZsI1F2+4GYFCNEP7t/ybwyFeJvPMrlJBxFQEGgl4abHO3qLUtXzFpIR8zbLxiET5xDFQ1vYzZPAyWeE3C3DImWixYFy0mdLRy1uRMnFUUKVAFUAWhqBiFmkgQOnYU+8qVc143SzK11pqSCWVCSTAU8s0pkKBZyKLNRujI4ZKMXwiChw8Bs4OMOhrsHmJKjJHwWEnGHwx6EQWReuvsE4U2r5VEe7qLLhAoFxHMToUSiRA+eQL7irllzGWrxWGyM7piI7Zrb0esayP69i8IPPxXhF56kIS3OHZ+KB5mPDpBYxoZ0907lSVjh2HSLTYXGh0e/LEAgRLFDgeDXqySlSrz3B0TbStWEu44hRIOl2T8YnFWUaRABSgyRhHu7EAJhbCvWp32nka7p2QniqHQMIqqpF3EgiRhW7mK4KGDJRm/EAQPH8KyYAEmV9Wc1/XvUirlOhD04rG5k6UoZsJ+jvZbhg4dKmockySSUFSUImNgoePHtI5256SXMf0UZlp4LvaPfRXHzfchn3M58Y49BH/5LQI779XcUon83VL679DgmFvGRLMZ67LlBA8dyPvZpULw0EFs7UsRLZY5r+syVqp1ORD00mj3zOkOhkkZmzQyKxFnFcU0aPVji4lR6Buw7Zxz0t6jLWJvSYhvuqCnUxQA9lWrtUyeoTPfZEaJRgkfP5ZVsUJpF3G6ExhoBQJFm43g4eKUq0nSNoliA9rBQwc16ziNGwVmGyNiTTPWSz+D89MPYLnk06ihCc0t9bOvEtnzFEog96yu3GTsHCJdXYaVaS8GCb+fyOnOrIoVzpyM2ZYtRzCZKsqAS8VZRZECVUA7URSpKMxt6a1j0BZYJBFlLGr8IsppEU8umEoQSt06tmVYxFVmF1bJUpJFrKgKQ8GhjO/LqFOYLGnLrdgU2eDhQ5p1nBKfmIlGu4fx6ASh+HRXhmC2YV77ERyfuk/jY7gXEN3zFIFHvkL/498hfnpv1hI2A0EvAsKcgVkdyVPY4TPvfgoeOQyqmlFR1Fs1smUpTq2RRJTRyFhGGRMtFqxLl1XEmpwLZxVFCjQeReExCt06dmQQSEixkAPGC+Vg0ItTdmCX7WnvMbe0IFVXEyzSlWIEdOs4nb8dtLRe/RRmNIZDI8TVRMZFDMacwkymSUURK7xuVSIQINLZgW1V+hMrTLmF0r0zQRAxLVyP/aNfwfGH/4R5/bWEu48QevZ7BB79ayLvPj2NxJeKgaAXt60OWTTNeR3AuniJdgqrgI0vePgggsWSLIw5FyRRwmNzl8QY0X+DxjSuOh32c1YT6TpNYmLC8DkUi7OKIgV6jKJQl1D4xPGs1jGU1pWi+0IzQRAE7KtWEzx00NDezYUg6TvOYB1D6eI6A0EtlTSTWwBSTmEHC9/4TAacKHKxjiE/GROrGrBsuplFX/4J1qv+K6LLQ/StJwk8/BVCL/yIePf+adVrB3ORMUnCtmJlRcQpgocOYlu+chZ/YiYaSiZj2U/5kCJjh8+8ATcTZxVFCnQeRaHB7MD+vQgmU0brGDS2sVmUS2Ih56IoABxr1pKYGD+jvQPiE+NEOjuwr16T9d5Gu4eRyCiRRNTQOQzmuIjNLS2YamsJHNhX8FhJ11MRMYrg/n2IVuu0Mtlzod7mzjsNW5Bk5KUbsV/3NRw3fwd53UdI9B4m9B/3E/j514i88yviE0MMZnHV6bCvWUvM6yV6BmuLxbxeYv39OHKUsaESpGHrrjqPbe6UdR3WxUsQ7faiZKxUOKsoUqAKk0GKAk8UgX17sS1fmdU6FgWxJNZLMBbEHwtktY5BW8SgzflMIbh/P6gqjnXrs96rf6fB4JChcxgIerGbbDhlR8b7BEHAvmYdwYMHCu7gJpuKO1Goqkpg317s56zJah3Logm3ra5gGRNrmrBu/kMcf/IA1iv/HNFVT/TtX9D7+NeIKTHc/gnUeGal7Vir/a6BfWdu4wvs1+TbsT67jDXaPcTVBMMhYwsaDga91FprMEtyxvsEScK+ei2BfdkbVpUbZxVFCqaY2fkv5NjwENHe3pw2PdBdKYN5j5MJuR5xAUzV1VgWLzmjiiKwby+SqwrLwkVZ721yNACU5J1lSltMhWPdepRQSCtgWACmXE+FxSiiPd3ER3w5y1iT3cNAoLj3JUgy8rLN2injD7/L6OrLAKjZ/xL+f/9Lwr9/iMTgyTk3NnNDA3JjE4F97xc1h2IQ2LcX2eNBbmzKem+jvbQylgsc69aRGBuruC6BZxVFCpI8igK0ub7h5mK5ADQ7GhkOjxjqSukLDEw+O/uiAG3jC588QcLvN2wOuUJVFAIH9uFYu25a/4l0aJhktOvf0Sj0BQZodjTmdK999RqQpIKVq8k0mR5b4IlCt8ztuSoKRyODQa9hTZ/EKg9DLZrLa/GW/4Jp4bnEjrxK8Km/J/jEN4ju/UVkLZQAACAASURBVM2sALhj3XpCRw6jRCKGzCEfKLEowcOHcKxbn5Mh0DxpjBgpY4qq0B8YzFnGHGvXAWf2pD8XMp9f/38GrXGRUFCMIrBvL3J9bpYLkBSc/sAAi6oW5D3eXOgLDCCLMm5bbU73O9auw/f0TgIH9lO1abMhc8gV4VMnUQIB7OvW5XS/LMl47G5DF/FE1I8/Fsh5EUs2G7Zlywns24vnxpvzHk+PURw65WPAkdkNMRdMb+2BhmaOjCgwkr0uUSLoIK4meP3YCWrl9KmsOqqHQ4yNZWYmHxg6jVW0051YBIsXIbR9HEf/ezj73kLZ/Sjh3Y8RrltGoOkCgg3roX4xcjzOoVfeRF2WOQA/E1aziaWt6dPMsyF05AhqNIp9bW6K1S7bqTa7DJWx4dAIMSWWs4yZqmuwLFxEYN9e3B+/3rB5FIuziiIFyWB2nicKJRIheOggVZd+KCfLBaYURZ/BiqLJ0TCt+U4mWJe0IzldBN5/r+yKIvD+eyCKOFavzfkzLY4megPGBUbzPYEBONafy9DjjxIb8iLX5+ZO0OG0mwH46a/zz5yyJiJ8+fRJ3qhZwyuP5ebKERxjWNfA/3v5bZSR3L9jJlhWd6IqVr63O3UOduByGsQxLrScZIP3FPW+R4keeIID4VbqBZEDz77Csw2xvMe767MX0tBQmLLwv/8egtmMfeWqnD/T7GgyVFH0TcprrooCNBnz/fpp4hPjGflY5cRZRZECzfUk5B2jCB48gBqN4rpgQ86fqbe5MYkmQze+Xn8/q+qWZ79xEoIo4jjvPPx73kaNx7MGSI2E/913sK1YieScu/bNXGh2NPK+9wCxRAw5S2AwF+jvvtmZ+yJ2nnc+Q48/iv+9d6n98NV5jdda7+CeL2zCajMzMppfTSH1/bdQT6lcctPVXNqWPaYDEFOi/OjYLq7YXMXm+uyyWVtjzzgvVVX5X8d/y+qq9Wy9KN3zriShqvhGO7D27OHcvncYq1G4wH+cLed6iLReSKyuHbIYM71DAX76m8OMBwpzzaqqSuC9d7CvWZu2bMdcaHY28lrPGyiqkrPBlQm9k0qnKQ9F4Tz/AnzP/IrA++9TfdmHip6DETirKFJQaPVY/7t7EO2OjCUVZkISJRrtHsOsl2AsxFh0PC/LBcB53gWM//5VzZe7Njc3ULGI9vcR7evFs/XKvD7X7GhERWUg6KXN1VL0PPoCA9hMNqrNuVtt5sYmzC2t+N/Zk7eiAE1ZeDwuvN78FF3vU4cI1dTQftHanGI6Ouq76ohIoyxrnV3JeCayzcsXHiF6NMqqxgXZn9d2Hqw9D1W5BeU3T+L95W+Qj76Fo2sXgtONvHQTpvaNiPWL5jyF2y3a1hSKFBZfiXScIj4ygvuGT+b1uWZHI1Elhi88kpF5niv6Av3UWmqwmTJnQqbCsnARpjo3/nf3VIyiOBvMToFWPZa8FIWaSOB//z0c556bt0Xe7Gikz2+Mophyo+SnKOxr1iBYLPjf3WPIPHKB/x1tLOd55+f1Od1FZNQprM+vBbJzdRfqcF5wAaFjR8tWx0iJRAgc2Ifz/AvyUhKgWchGGSOFuOoE0UTV1utAklCarsR65W2Ita1E9z5L8Jd3E/j5XxN547FZmVM2XVFEC0tF9r/7DogiznPPy+tz+ncz8p3lc2IFLRXbef4FBA/sr5hqsmcVRQqSrqc8gtmho0dQAgGc5+fudtLR7GhiJDI6qx5PISjEFwogymYca9fhf+/dsrG0/e++g2XxEuS6/Cy2Bns9oiAasohVVaU/j4ynVDjP3wCqqsVZyoDgwf2o0WjBMjYQ9BI3oHlRocaIZHdgX3UO/vffx7R0M/aP/hXOz/wPrFs+j1jTRHTvcwSf+nsCP7+D8O6fkxg8gUXWtqZQpEBF8c6evF2bkJL5ZIABl1ASDAS9BcrYBajxOIH9lUG+O6soUqAKQt49syfefgvBbMaxJvegrI6WlMynYtEbGMAimam11uT9WecFF5IYGytLo5nYkJfwqZN5xXN0mEQTDQa568ajEwTiQVrysI51WBYuwlRfz8TbbxU9j1ww8fZbiA5HVsb/XGh2NKKoiiFExV5/P9VmF44MdcTSwXnBBmKDA0l+gGB1Iq/agv2jX8H5mR9gvfxPEWtaiO1/geBT96D84uvcYHsb82hn3uSzSE830f6+gmTMZrJRa6kx5NQ6FBomrsQLkjHb8hVILlfZZCwbziqKSSSFUcidma3G40zseQvneefnFTDTYaQrpc/fT5OjsaAAnPPc8xDMZibefKPoeWTDxFtvAuC6aFNBn29xNNLnL/599RZ4AgPNNeC6cCPBgwdK7n5SIhH8772La8OFBSUbTLlSDJCxQH9ebqdUuC64ECRpThkTrE7klR+aOmlc8QXEujY+ZD3Mxs5/4fQP/4zw7x8i3rUvp/4ZE2/sBkHAueGiguba7Gg0ZE32FngCA42l7dxwEYG971WE++msopiEqoey8ygKGDx8EMXvx7WxsNRSt60Wq2She6KvoM/rUFWVbn8vbc7CAryi1Yrz3PPwv/1WweUpcsXEm29gbV+K7MkvtVRHm7OFobCPUDxU1Dy6J3oBaHU1F/T5qk2bQVHw73m7qHlkQ2Dv+6iRSMEy1uRoQBIkuv3FyVhCSdAbGCj4fUkuF/Zz1jDx5hsZ15dgcSCvuAz7tbdzf+JPeKP241halhM7+ntCv/nv+B/6b4T+838RO74LNTK7J7eqqky89Qb2c1bP2Yo4F7S5WugPDBIr0l3X7e9FFMTClevGTajRKP733y1qHkagpIri6aef5mMf+xhXX301Dz/88Kzrhw4dYvv27VxzzTXceeedxCc3qT179nDjjTeybds2brnlFnp6eko5TQCUSf+8mketp4k33kC025N1k/KFKIi0Olvomiju+/nCIwTjIRYUkQnk2riJhH+ipJUro329RLpO49pY2GkCoM3VCkxt9IWia6KHWktN1hpP6WBuW4C5qbnkp7DxN3cjVddgK8DtBFrNp2ZHY9Ey1h8cJK7EWeBsLfgZVZs2EfcNE86xBIpodnBYWkXTjX+Nc8ePsF3zl8hLN5LoO0z4t/+M/6EvE/z1PxHd/wKKXyMghk+dIub1FqxYQTNGEmqi6FNY10QPTfaGrDWe0sG2bDmm2jrthHSGUTJFMTAwwAMPPMAjjzzCU089xaOPPsrx49MF5I477uCuu+7iueeeQ1VVHnvsseTf7733Xnbu3Mn111/PvffeW6ppJqHo1oMA5BDUVSIR/O/uwXn+BkS58Jz+ha5Wevy9RVWs1DeBBa7CF7F97XpEm43xN3YV/IxsGH9jF0y6bQqFrgyL3fi6/b0sLOJ9CYKAa9NmQseOEhvOzpIuBAm/n+C+vbguuijvbKdULHC10jXRU1ShOSNkzHHeBQiynLOM2SymZDBbMJkxLToP65bP4/iT72Pf9g3M669BDYwQef1hAo98hcCTdzH6q38DScKRZ0ZdKhYYZIx0T/QU9b4EUcS1cSOBA/vPeI+KkimK119/nc2bN1NTU4Pdbueaa67h2WefTV7v6ekhHA5z3nla+tr27dt59tlniUaj/MVf/AWrVmlsypUrV9LXV9yxORckSXY5xij877yNEg5TdcmlRY3b5mohqsSKKjneNXnEbXEU5hYAEGUZ10Wb8O95m0SoOLfOXFAVhfHXX8O+eg2mmvwD7jqqzC6qzVV0+QtfxOF4hMHgUNFcjKrNl4CqMr7rtaKekw4Tb72BGo9TdcllRT2nzdWCPxZgNDJW8DO6JnowizIN9sylsjNBstlwnr+BiTfeQIllJ9JZLRLhOdJjBUFEalyGZdPNOG6+D8fN38Gy6WaQLPgPn8JanSD05NcIvfi/iR17HSWUXxyp3laHVbIUZYyMRSYYi04UL2MXXwqJBONn+FRRMsLd4OAgnhQ/dENDA3v37k173ePxMDAwgNlsZtu2bYDmDvrRj37Ehz/84bzGdrvzS4kDCOrxIkFAliU8HlfG+wfe3IW1qZGFl16Ydx5+Ks6VV/Bvh2AUH+s8yzLem25OA4cGaK1qorWpruB5AFivu4a9v3sZDu/Fc3Xu7zzbuwIYfe994j4f7Z//bE73Z8JS90L6An1Zn5Pu+mHvICoqa9uWFzcXj4vhtWsI7H6dlZ/947zkIJdxe994HceSxSzYUJhrU8c6lvP4URgXR1jhyVwuJq2M7Rtgce0CGhsK8/vrkD9+NQfe3I108jD1l2U2smpcVkYmohnnpV10wfLlDO9ahPrMd2n+g09gkkYInXyX8IndgIClZRm2pedjX3oBlpalCFmSPpbULaQ/PFCwjHX3dQKwrm1ZkTK2mqGlSwm+8Tor/mh77h8rco3NRMkUhaIo0xaOqqrT/p3tejQa5etf/zrxeJzbbrstr7GHh/0oeRb2C4Smql5GIzG83vRHvZjXy9i+/bi3fYKhoeIqr5oVB7Jo4mDvCVY50re31Fizc8/p5HAnq+pWZJxzLlBrmzA3NdPz7AtI5+cWR8g0r1T0/cfziHY7ytJVRc+z0dLIu30H6OkfxiyZ857Xvi4tDbhKqS16LvaNlzD+Lw9y+vU9Oaev5vLOIj3d+I+fwPOHf1z0HB3xagQE9vccZ5F5Sd7zUlSFk77TbGq6sHgZa16Mqa6Ort+8gLoyc7E+ERV/UFMUuYzb8x/PI9XUYLrkegRRxLZ5B8pQJ/GuvcRO7yXy6uOMvvoYgtWF1LYWU9tapNbViI7ZRTSbrFopj4HBsbSZhJl+x/3dmpvdmagp/vfbdDGDj/w7XXv2Y82hJH+uazIVoihkNLBL5npqamrC651yp3i9XhoaGtJeHxoaSl4PBAJ84QtfIB6P8+Mf/xi5iBhArtBjBKqQvXrs2GuvgiAU7RIArZRHi6O54GOuUUdc0PzuVZd9iPCJ40R6i/PPpiLh9+N/Zw+uTZsR5bk39nzQ5mpFRaWnwDTZLn8PLtmZV+mOdHBuuBDRamXs1VeKflYqxl79HUgSVZsuLvpZVpOFBnt9wT73odAwkUS0qGQJHYIoUnXJpQQP7M8a27FZTDkzs2MjIwT276Pq4kuT8RxBEJE8S7BcsA3HDd/EseN/YL3yNqS2tSS69xN++UECD99O4PE7Cb/+MPHT76HGNNfCAmdrUS7hrole6m1ubCZbQZ9PhWvjZgSTSZOJM4SSKYpLLrmEXbt24fP5CIVCPP/882zZsiV5vbW1FYvFwp49WjmHnTt3Jq/fcccdLFq0iO9///uYzcVvLLlASUzFKDIVBVTjccZefQXHuvXI7uJrwYAWoO3y9xQU0O6a6NaeUWBq7ExUXXqZJpQv/9aQ54GmWNVYjJotWw15np55c3ryu+eLroke2lwtRbkMdYgWC66LL8H/1puGBRyVSITx117FteFCJJcxLoQFrtaC39fpSSPGCGMEoPpDlwMw9ruXM95nM5uIxhQSObSOHfvdy6CqVG+5PO09otWFvOxibFfehuMzP8C+/VtYNt2MYK8hduhlQs9+H/9Pv0jwV9+msfsIAJ1jhTUQ6proNmxNSk4nzgsvYmLXayhh4+OHuaBkiqKxsZHbb7+dHTt2cMMNN3Ddddexfv16br31VvZNNmC5//77ue+++7j22msJBoPs2LGDgwcP8uKLL/LOO+/wiU98gm3btnHrrbeWappJ6Omx2RoX+d99h8TYGDVbrzJs7MXViwjFw/QX0I3s5FgnoiCy0KBS5SZXFc4LL2J812uGEH1URWHs5ZewLV+BZYExc6yz1lBldnFyrCPvz4biIXr9/Sypzq0Cay6oueIqzYB47VVDnjfx5m6UUMhYGatayGhkDF84/zafJ8c6MIsyrUUkS6RCdtfjOPc8xn73SkbejjVZGDDzqUKNxxn73Ss41q7D7GnIeK8OQRCR6hdhPvdj2D9+B85b/ie2j/+1lkkVj1D77gtYEgpH3nqY0HM/ILrvORJDnTmVuRmNjDEcHjFWxrZehRIOM767dFmJmVDS6rHXX389118/vfnGgw8+mPzvVatW8cQTT0y7vnr1ao4cOVLKac0JRdXTYzNnPY2+9CJyvadg7sRcWDopUCfHOmhx5kfOOTnWQZuzBUsaX30hqNl6FRO7dzH+xi5qLi/uFBA8uJ+YdxD3DbkH4rJBEATaqxdzcqwz78+eGjuNisrS6sWGzcfS2optxUrGXn6J2quvLSqVVVVVRl/6LebWNqzLci8Znw369z052kFdU26NrXScHO1gcdVCJFEybD41V1xJz3vvMrHn7bS9UGwWbbxAOJ7RovW/9w6JsVGqt3624PkIJjOm1tWYWldjAZTwBIvf+QmnpVESgz3EOydJb2Y7puaVSM0riazegCq6Z/3euly2Gyhj1valWBYuYvSl31J9+VZDTsP54CwzexKJpKWQXlGEOzsIHT1C9dYri9oMZsJjq8cpO/Le+BJKgo7xLkM3PUgRyv98oehCgSMvPI9UVYWzgLo7mbC0ehG+8EjeKZ8nxzoREFhs0AlMR82VVxEb8hIokkUbOnaUyOlOaq68ytDNoNXZjFkyc3I8PxkLxyN0+/tor1ls2FxAaysrNzYy8sJzafkdNrNmxwbDmRsejbzwPHK9B0eOnexygWh1sax5Pf1EkW78Fo4//u9Yt/4ZcvuFJEZ6iez+OT3/cgf+h75I8NkHiO79DQlvB6qicHKsA1k0GRLT0SEIAjVbryTa002ohKTYdDirKCahJCbr3gtC2s1x5LlnEa3WpI/VKAiCwNLqxZzI05XS5e8hpsQMX8SCIFB79TVE+3qL6t0b6eoieGA/NVd9pChS4lzQv3O+ylU7gTVjzaM/QC5wnr8BU309vmd/U9RzRp79DySnS8ufNxCSKLG4aiEnRzvy+lzHuHYCM9I6Bi2oXfuRa4h0nCJ07Oic9+ilxoPh9K6n0PFjhE8cp+YjVxtqvIF2IlBROTV2GtHpRl5+CdYtn8f5h/+I49MP0LDtL5HbN6KM9RPZ/SjBX96N//99keOdb7BAdED/cdS4cb3CXZsvRqqqwvdccTJWCM4qikkkA8lpYhSx4SEm3n6T6i1XINnzr56ZDe01ixkKDTMezT0gOnXENc4XqsN14UZMdXWMFCGUI88/i2CxFO2+mgsLnK3IopzXxpdQEpz6/9q797Aoy7yB499nzgxnkIMCHkARwyzzLIqKoZKimaakq6b2tpW7tdqVndy3dLVc6tpr38xar7Y1N81DHjItDx1cyxMqeQgQJFBBUA7DcRBmBuZ5/0BIEEZQZnDt/vw3J+bHPfc8v7mf+35+d1lWmydWqC3i5hk9lqqMX267Cq8pN4eKs2fwiBqNwg6LOELcu3DZeIWqVpS1zyy9iIREsHvnNo/HbegwlC6uFO/9usnHdddPPdmaoyjatweF3hn3YZHNPud2dXXrjITU5FyYwtkTl97D0UXOxWV6beLQRT2DHDyAHMlC5/xcKnevxLj2OSp2LKPq6EYsmSewXiu5+Y1aSKHW4BH1MNeSfsZ0OfsO/rPbeG+Hvttd7NfJ7KZPPRXv2wOShMfD0XZ5/7pfbK35hZxZchEvnSce2ju7CKopkkqF58NjqDyfRmULa/PcyGIopOz4MdyHRbZ6T4CWqP2FHNSqUVhOxRXMNWaC3do+sQK4RwxHoXemaM9Xt/X64r1fI2k0bTqJfaO6X8gXy1p+kMksvURHZ782WebZmEKjwSNqNBVnzzR54Kvb5a6isulTT6bcXCpOn8JjVNRtVW++FZ1KS6BLRzJa8J1UOHui7j6Yq31GYJUgbOh8nMb+Cc0D45CUKiwp31P17Woq1v8J48aXqPx+DeaU76kxZLfq9K7HyCgkrZaiPU0nV3sRieK6GusNp54aJYrqkmJKfziI25CIVm+201KdXQPQKNSkFbXsoGyVraSXZNLdo/kLqO6Ue+RIlC6uGL78otWvLfp6d+0prLHj7BBZre4e3cguz6HC0rL9p88XZwAQYqc2U+h0eEaPoeLMaaouXWzVa815eZQdO4r7iFFttiS2sW7uXVBICtKKW9bHLNZqMkou0N0j2C7xAHhEPYxCp8Ow+8ubHtPVzVE0M6Io2v1lbWK10483gO4ewVwovYi5xvY8SZ3zxb8gIRHiE4aqy4NoBz6OfuJruDz5IfpH/4x2cBxK787U5CRjOvRvrm37M8Z1C7j2VTym41uxXPzJ5qhD6eKCx8goyo8fw3zV/qWN6ohEcZ0s/5ooGhcFLNrzNbIs4zV+gt3eX6VQ0dOrB8mG1BYVb7tUlo3RUkG4d5jdYlLodHiOjeFaclKrRhUWQyGlh37EbVik3RIrQLh3GDIyqUVNn+NuLLkwlQCXjre1uVNLeYyORqHXY9i1s1WvK/pqF5JSide4GDtFBk4qHSHuXUk2pLbo+b+UZGK2Wgj3vr3KtS2hdHHBY3Q0xpMnMOU0vM6jbtVTZROT2eYruZSfSMBj1GhUrnd+4WRzwr3DsFirSS/JaNHzkw2pBLt3Qd9ocydJqULpG4KmzzicxvwR59/9H85x8ehG/g/q7oORTRWYz+yhav97taOODYuo/OZ9TKe/pjr3HLL51+snPMfGIKnVTSZXexGJ4jprTdMbF1mKDJQePIDbkKEtXqN9u8K9wzBUFZHXgqtBkw2pSEj08gq1a0weo6JqRxVf7Gjxa+o6sNcj4+0VFgBd3IJwVutJNtx6OXVldRW/lF6wa2IFUOr1eEaPpeL0KaouZLboNearVyg7dgT3ESNRudsviUFtH8sxXmnRarFkQypqhYpQzxC7xuQZPbZ2VNGoj2nVSiSp6clsw5dfIGk0dh2xQu2oVaNQtyi5lprKySrPaVEfkyQJhZsv6tAIdMPn4PzYUlzmfoh+4utohzyB0j+UGkMW5uNbqNz9V4yfPEfF50uoOvgx1suJuA3qT3nCMUy59t+CAUSiqGeVmz71ZPhiOwDesZPsHkPdL7eWdMpkQyrd3Lvc1raUraHQ6fAaP4Fr55KpSE665fNNOTmUHfoRj1FRdh1NQO1+Hvd59STZkHrLq9rTitKxyla7JwoAj4fHoHR1peDzzS0aHRZu24qk1uAVY78Ra526/7+lfayHZ0iz9bTaitLFBc+xMRhPJTYYuUqShJNGRUWjEUVlZiblJ47jGT3WrqMJALVSTU+v7iQX3nqkn1JU+4PldvuYpNKg9O+B5v6xOI1+Bpe4eJxnr8Jp3CI0/SYhuXhRffEUpkPr0BqPIClk8t5fTtUPazGnHKCm4AJy3Sn0NiYSxXU3zlHUjShM2VmUHT2Cx+ho1N63X165pbx0nnRy9r/ll7g1v1zagvvIKNQdfCjcuvmWE2+F27ag0OnwnmD/xAq1X0qjpeKWtbKSDak4qXR0c2v71TuNKZ2c8I6dROX5NCp+PmPzuZXp6RhPJeI1Lua2d2RrjY7OfnhqPW45Csu/Vkj+tUKH9THP6LEo3d1vSq5OWmWDEYUsyxRu3YzS1dWup+luFO4dRmFV0S3rPiUbUnHXuBHg0jZXsEPt9Ryqzn3Q9nsUfcwinGevwjkuHudxz+H+UChV+ZUYTyVgOrSOazuWYj6xrc3eu0Ecdvmr/4W8VG4MLbmGk6QF2Yosy+Rv3ojCSY/XI/b/pVcn3DuM9JJMys3NV6U9U5BU/1xHUKjVeD82BVN2NqWHmi9MVpGcRMXZM3g9MsEuK52a0ss7FIWk4FT+z80+p9pazdnCFHp5hbbp1cW2uEeORO3nR8GWTc2WqZCtVvI3f4bS3QPPMfY9hVJHkiR6d+jFOUOazWWydX2st6P6mE6H98TJVP2SjvH6vupQW8bjxuWxxp8SqTyfhvfER1Ho2n4lVlPqvmenCpofUZtqzJwzpBHuHWbXq6brT1mFDKTDky+i8vSiotQX/bS/oov+A+r72n4pOti5hMd/EwUSEwuNXFF4g7WG8hMJVKaew/d3s1E63952mbdjUMd+fJP1HxKuJvJw55sv7JNlmcO5CQS4dCSwDX+53IrrgEGUHviewu1bcX2o/02JwGqxkP/ZetS+fng8PMZhcbmonent3YtjV04yIXgMKsXNXfpsYQpGSwWDO/Z3WFySSoVv3Exy/u9vFH+zD6+Ym+drSn/8AdPFC/g/9bRdlnc2Z5B/P37MOcrJvNMMC7i5fEZdHwtx70YHJ/uePryR+/BISg8eIH/LRpz79EGhc8JJq+JUWj5/vlqGqsbCxNObMOm9WZ+hR8607za0N1L5+/BV2g8c+k6PRG0iUCkVVF8vWGhyuUiVj4mfE5348xHHxdWlQz8i07/hs9W7Oe/fmzEDLDxmh0VqIlFcJzv78G1lOGFaBTXF2RRs2YS2cxfcI0c6NI6Ozn4Eu3flcE4Co4Mib/p1klV+mcvGXKaHPurQei+SJOE7cxaXlr1B4fbP8Zs9t8Hjxfv3Ysm7SsCfFrX5Vdi3EtFpIGcLk/m58Bx9fe+/6fHDOQl4aj3sPvHfmPP9fXDu+xCGXTtxHTiowenLmvJyCrd/jlNoT1zboJR4a3R1CyLApSOHcxOaTBTpJRkUVBp4pJv9lp02RVIo8J05i+y3l2P4cic+0+J4uF8gZzKLMJuq6Z76A85mIykPjcfPyzEj1jpO1p7kqA/h6lOKS3VtaQ6NVoX5+mjngsslNDXuBOqDkPSO+16aPO7HUJTGQ9nHqQoOx93ZPvNJIlFcZ1Uo2VXZjzBdEjUlJSBJdHru+TYvC9ASwzoN4t/nNpNekkGoZ8Nd7w7nJqBWqBngf/t7At8ubWAQng+PoXj/XlwHDELf6z6gdqli0a6duPTr36b1dlrqPu+eeGjdOZRz7KZEUVhpILU4nQndxjS7AY09+cbN4OL/vk7evz8h4E8v1if3/I3rsVZV4TtzlsMLvEmSRESnQWw5/wVZZZfp7BbY4PFDOQnoVU486HNz0rU3p5DuuEeOoPibfbj0H8jAXsGMj+xO9omfydqbiFvEcGbOlLoCOQAAEBRJREFUdczcxI0s1l68fvgnvLrl8z/3jwV+3SDocnkub58oZEqPWKKCHN//zUM7cGnp/zK+5CSdQpousHinxBzFdfUTaNcPJp5jxuIUbL8LjWzp69sHF7UzOzP2NljNk2u8SsKVRAb697XLlbIt4T1pMmpfP/LWrcVaVYVcU8PVT/6FpNXiO2NWu8SkkBQMDxhManE65xpdU/FFxh5UkpIhnQa0S2xq7w74THmca8lJlF0vQ244lkD58QS8J0xEGxB4i79gHwP8+qJTavki4+sGk8eXyrL5Kf8sQzoOQKN07MiwToep01F5eJL3ycdYLRasFgtXP/kYpasbPtPi2iUmtULF0I4DOVOQxIUbrtSWZZkvMr5Go9Qw0P+hdolN07ET3hMfxZh4kvLEE3Z5D5Eorqv/rnj7oA0KwntS25XFbi2NUs3UHhO5WJbF99m1B5caaw3rUz9Hp9IRG+yYic+mKLRa/J6ch8VQSMGWjVzetoOqjF/wfWKmQ1btNCcqKBJffQc+S91WP0l7Kv9nTuWf5ZFu0XYpc9JS7iOjcArtScGmz6j8JZ2MD/6BNqhzk/MWjqJXO/Fo90dIK/6FI7m1k8eWGgufntuCu9aNmG72KSPSEkq9Hr/ZT2LOzcGwfStZn23CfDkbv1lzHDpf2Ni4rlF4aN359NznWK5fqX3syknOFZ1nUkgMLur2i81zbAxOYb0wZd/eRku3onzzzTfftMtfbkeVlWZbW0o0qfyame9/yiFiYiQhseNQqNr3rFwnZ3+yjbn8mHOMoqpivv7lezJLLjGz1+N0s0OBttZQe3dArq6m5NtvKE1OwXXAILwnOnbOpDGlQkmQSwAHsg9xpjCF3PKr7Dy/h04uHZnVa1q7nHaqI0kS+rD7KD14gNKDB8AqE7DwRbtfXHcrQa4BZJRc4FDuMUpMpXx5fj/ZZbnMDZ/Rpks8b4fGz4/q8nJKvvuG8tQ03EeMxGus40853UilUOHv7MuBy4dIMqSSVZbD7oxv6ObWhek927f/SwoFbkMi0If1wtlZy7Xre423+PWShF7f/PyGSBTXWWX49mQ24yO64aJr/6kbSZK4zyuUquoqjl05iVW2Mrn7eAb592vXDlnHqUco186loNKo6bjgebtUO20tL50HAS6dSC1K52xe7cT278Iev6mcQntQ6vWoff0w/nSS4N8/hTr0vvYOCUmS6q9DOXr1JEqFksd7TKKvr+PPszdFH9aLirOn0Xq44//0c0jt/OMNwEffAT+9DymGVFIK0xng35cnek7BqY3L1t8OSZKQJMkuiUKSW3Lp6H8Zg8GI1dr6f6vSVE3nQE8KCtpm7+O2UlldRSdfT4qL2me/3ObI1dV4e+kpKmtdp7Q3q2zFxUPNtVL7XKV6J2quXcO/i99d2Mcq6eTnTbGhZQUWHcVqseDTwQVDadvt69AWaqw1uHlqqSi1vU1re6ibZG8NhULC27v5lWRijuIGdRul3G2cVDpUyrsvNkmlQunA9f8tpZAUOGvafxTRFHvsZdIWnFROqBx0MWJrKNTqu2K02phSoUSvaZ8FJe1BJApBEATBJpEoBEEQBJtEohAEQRBsEolCEARBsEkkCkEQBMEmkSgEQRAEm+6+NZdtQKG4swvS7vT19iLiap27NS64e2MTcbXOvRLXrZ5/T15wJwiCILQdcepJEARBsEkkCkEQBMEmkSgEQRAEm0SiEARBEGwSiUIQBEGwSSQKQRAEwSaRKARBEASbRKIQBEEQbBKJQhAEQbDpnizhcTt27drFhx9+SHV1NXPmzGHmzJntFsv777/Pnj17ABgxYgSLFy/m1VdfJTExESen2l21/vCHPxAdHe3QuGbNmkVRURGq63sXL1u2jKysrHZvt88//5z169fX3758+TKTJk2isrKy3drMaDQSFxfHP/7xDwIDAzly5Ahvv/02JpOJmJgYFi5cCMC5c+d4/fXXqaiooH///ixdurS+fR0R1+bNm/n000+RJInevXuzdOlSNBoN77//Ptu2bcPNzQ2AadOm2fWzbRxXc/29uXZ0RFwZGRn87W9/q38sLy+PBx54gDVr1ji8vZo6Rti1j8mCfPXqVXnUqFFycXGxXFFRIcfGxsrp6entEsvhw4fl6dOnyyaTSTabzfLs2bPl/fv3yxMmTJDz8vLaJSZZlmWr1SoPGzZMtlgs9ffdTe1W5/z583J0dLRsMBjarc1Onz4tT5gwQQ4PD5ezs7PlyspKecSIEXJWVpZssVjkefPmyf/5z39kWZbl8ePHy6dOnZJlWZZfffVVecOGDQ6LKzMzU46OjpbLy8tlq9UqL168WF67dq0sy7L8+9//Xv7pp5/sFoutuGRZbvKzs9WOjoqrTn5+vjx69Gj5woULsiw7tr2aOkbs2rXLrn1MnHoCjhw5wuDBg/Hw8ECv1zN27Fj27t3bLrH4+PjwyiuvoNFoUKvVhISEkJubS25uLq+99hqxsbG89957WK1Wh8aVmZkJwLx585g4cSLr16+/q9qtzptvvsnChQtxcnJqtzbbsmULb7zxBr6+vgCcPXuWLl26EBQUhEqlIjY2lr1795KTk0NVVRUPPvggAI899phd269xXBqNhjfeeAMXFxckSSI0NJTc3FwAkpKSWLNmDbGxsSxbtgyTyeSwuCorK5v87JprR0fFdaP4+Hji4uLo2rUr4Nj2auoYcfHiRbv2MZEogPz8fHx8fOpv+/r6kpeX1y6x9OjRo/5DvXjxInv27GH48OEMHjyYt956iy1btnDy5Em2bt3q0LjKysoYMmQIq1ev5pNPPmHTpk3k5ubeNe0GtQm/qqqKmJgYCgsL263NVqxYQf/+/etvN9e/Gt/v4+Nj1/ZrHFdAQAAREREAFBUVsWHDBkaPHk1FRQW9evXipZdeYseOHZSVlfHBBx84LK7mPjtHf08bx1Xn4sWLHD9+nNmzZwM4vL2aOkZIkmTXPiYSBWC1WpGkX8vsyrLc4HZ7SE9PZ968eSxevJjg4GBWr16Nr68vTk5OzJo1i4MHDzo0nr59+xIfH4+rqyteXl5MnTqV9957765qt02bNjF37lwAgoKC2r3N6jTXv+6WfpeXl8ecOXOYMmUKgwYNwtnZmY8++oiQkBBUKhXz5s1zaNs199ndLe21efNmZsyYgUajAWi39rrxGBEUFGTXPiYSBeDv709BQUH97YKCgiaHm46SmJjIk08+yYsvvsjkyZNJS0tj37599Y/LsmzXCc+mnDx5kqNHjzaIISAg4K5pN7PZzIkTJ4iKigK4K9qsTnP9q/H9hYWFDm+/jIwM4uLimDx5MgsWLAAgNze3wejL0W3X3Gd3t3xPv/vuOx555JH62+3RXo2PEfbuYyJRAEOHDuXo0aMUFRVRWVnJ/v37iYyMbJdYrly5woIFC3j33XcZP348UNvx3nrrLUpLS7FYLGzevNnhK57Ky8uJj4/HZDJhNBrZsWMH77zzzl3TbmlpaXTt2hW9Xg/cHW1W54EHHuDChQtcunSJmpoadu/eTWRkJAEBAWi1WhITEwHYuXOnQ9vPaDQyf/58XnjhBebNm1d/v06n45133iE7OxtZltmwYYND2665z665dnSkoqIiqqqqCAoKqr/P0e3V1DHC3n1MLI8F/Pz8WLhwIbNnz8ZisTB16lT69OnTLrF8/PHHmEwmVq5cWX9fXFwcTz/9NE888QTV1dWMGTOGCRMmODSuUaNGcebMGR599FGsViszZsygX79+d027ZWdn4+/vX387LCys3dusjlarZeXKlfzxj3/EZDIxYsQIxo0bB8C7777LkiVLMBqNhIeH15/3doStW7dSWFjI2rVrWbt2LQBRUVG88MILLFu2jGeffRaLxcJDDz1Uf0rPEWx9ds21o6Ncvny5QT8D8PLycmh7NXeMsGcfEzvcCYIgCDaJU0+CIAiCTSJRCIIgCDaJRCEIgiDYJBKFIAiCYJNIFIIgCIJNYnmscE/q2bMnoaGhKBQNfwutXr2awMDANn+vo0eP4uXl1aZ/tzVeeeUVevTowfz589stBuHeJRKFcM9at25dux68BeFeIRKF8JuTkJDAu+++S6dOncjMzESn07Fy5UpCQkIoLy9n6dKlpKamIkkSw4cPZ9GiRahUKs6cOcPy5cuprKxErVazePFihgwZAsCqVas4c+YMJSUlzJ8/v8m9CO6//36efvppDh8+TH5+Pk899RQzZsxg+/bt7Nu3jzVr1gA0uP3KK6+g0+k4f/48BoOBqKgoPDw8OHDgAAUFBSxfvrw+hsTERPbt24fRaCQiIoKXX34ZlUpFRkYGK1asoKSkhJqaGmbNmsXUqVNJSEhgxYoV6PV6Kioq2LZtW339IkG4kUgUwj1rzpw5DU49BQYGsnr1aqC2LPTLL79M//792bhxIy+99BLbt29n+fLleHh4sGvXLiwWC88++yz/+te/mDt3LgsWLGD58uWMHDmSpKQkXn31VXbu3AnUFrJ74403SElJYfr06UybNg21Wt0gHrPZjKenJ5s2bSIpKYknnniCKVOm3PL/SElJYcOGDZSUlDBs2DCWLFnCpk2bWLduHR999FF9orh69Srr169HpVIxf/58tmzZwrRp03j++eeJj48nPDyc8vJypk+fTvfu3YHawnLffvstAQEBbdLmwr1JJArhnmXr1FNYWFh9CekpU6awbNkyiouL+eGHH9i4cSOSJKHRaIiLi2PdunVERESgUCgYOXIkAL1792bXrl31f6+uxESvXr0wm80YjUY8PT1vet/Ro0cDEB4ejtls5tq1a7f8P0aNGoVarcbHxwe9Xs/w4cMB6Ny5MyUlJfXPmzRpUn2tq4kTJ3Lw4EEGDhxIVlYWr732Wv3zqqqqSElJISQkhI4dO4okIdySSBTCb5JSqWzyvsZlma1WK9XV1SiVypvKM58/f57g4GCA+mqhdc9prjKOVqu96XmSJDV4vsViafCaxqeDmqtMeuP/VFfBtKamBldX1/qRD9RWEHV1deX06dP1iUUQbBHLY4XfpNTUVFJTU4Ha/QX69u2Lm5sbw4YNY/369ciyjNlsZsuWLQwdOpTg4GAkSeLw4cMAJCcnM2fOnDbZNc/Ly4v09HRMJhMWi6VBie3W+OqrrzCbzZhMJnbs2EFkZCTdunVDp9PVJ4orV64wYcIEkpKS7jhu4bdDjCiEe1bjOQqARYsWodPp6NChA3//+9/JycnBy8uL+Ph4AJYsWcLy5cuJjY3FYrEwfPhwnnnmGTQaDatWreKtt94iPj4etVrNqlWr2mTyNyIiggEDBhATE4OPjw+DBg0iLS2t1X8nMDCQGTNmUFFRQXR0NJMnT0aSJD744ANWrFjBP//5T6qrq3nhhRfo168fCQkJdxy78NsgqscKvzkJCQn85S9/Yffu3e0diiD8VxCnngRBEASbxIhCEARBsEmMKARBEASbRKIQBEEQbBKJQhAEQbBJJApBEATBJpEoBEEQBJtEohAEQRBs+n8KW5+Kv7mIfwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def OneCyclePolicy(epoch, period=50, frac=0.1):\n", " phase = (epoch % period) / period\n", " gamma = frac ** (-2 / period)\n", " #print(phase, gamma)\n", " if phase < 0.5:\n", " return frac * gamma ** (period * phase)\n", " else:\n", " return frac * gamma ** (period * (1 - phase))\n", "\n", "optimizer = torch.optim.SGD(net.parameters(), lr=0.1, momentum=0.9)\n", "schedulers = {\n", " 'MultiStep': torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones=[50, 100, 150], gamma=0.5),\n", " 'Exponential': torch.optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.99),\n", " 'CosineAnnealing': torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=25, eta_min=0.01),\n", " '1CyclePolicy': torch.optim.lr_scheduler.LambdaLR(optimizer, OneCyclePolicy),\n", "}\n", "for name, scheduler in schedulers.items():\n", " rate = []\n", " for epoch in range(200):\n", " rate.append(optimizer.get_lr())\n", " scheduler.step()\n", " plt.plot(rate, '-', label=name)\n", "plt.legend()\n", "plt.xlabel('Epoch number')\n", "plt.ylabel('Learning rate');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Data Augmentation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To motivate the need for our next technique, imagine that we have a very small training set (compared with the number of parameters we wish to learn) of only 25 samples:" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "x_sparse = x_train[:25]\n", "y_sparse = y_train[:25]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Not surprisingly, the network learns to memorize the 25 samples with its 591 parameters, as we can demonstrate by comparing the TRAIN and TEST loss curves. Note that we train for 1000 epochs here since there are only 25 samples per epoch:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEJCAYAAABlmAtYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deVzUdeI/8NdnBlAQRVEOhdRyTMs0TU3QUjHXGzXTRC1krbTDDmszK1r7pdZmHlvU12Pd1nT9fsvyIC+iTS0VvK+s1GhFFJFDEMHhmJnP5/fHOOMMzDAHc8/r+XjsY/l85nq/+eC8ep8fQZIkCURERA2QubsARETk+RgWRERkEcOCiIgsYlgQEZFFDAsiIrKIYUFERBYxLIiIyKIAdxfAmcrKbkIUbV9G0rp1KK5dq3RCiTwX6+wfWGffZ299ZTIBrVo1M/u4T4eFKEp2hYXutf6GdfYPrLPvc0Z92Q1FREQWeXTLorS0FIsWLUJISAgGDRqEoUOHurtIRER+yaPDYv369Zg+fTp69OiBmTNnMiyIyCqSJKGsrBi1tdUoKhIgiqK7i+QyRUUyC/UVEBTUFK1aRUAQBKvf16PDoqSkBNHR0e4uBhF5mcrKcgiCgKioWAQGBkCt9p+wCAiQNVhfSRJx/XoJKivL0bx5S6vf16PHLKKjo1FcXOzuYhCRl6mqqkTz5i0hCB79FecWgiBD8+atUFVl24wpj25ZTJo0CYsXL0ZgYCCSkpJc8pk5+eXYe7oAsa1DoIgJc8lnEpFjiaIGcrlHf725lVweAFHU2PQat/w2KysrkZSUhJUrVyI2NhYAsG3bNqxYsQJqtRrTp0/HtGnTEBkZiSVLlrisXDn55fjo/05AoxEhl8vw+pReDAwiL2VLf7y/sed34/KwOHXqFFJTU5Gbm6s/V1hYiOXLl2Pz5s0ICgpCUlIS+vXrB4VC0ajPat061Kbn7z1dAI1GhCgB0Ii4fE2J+J6xjSqDN4mIaO7uIrgc6+ybiopkCAi43QVl+LM7fPTRBzh9+hRUKhUuX76EO++8CwAwefIUfPLJckRFacdmNRoNamtr8eKLr2DQoAT96zdu/BKffLIM6ek70bp1G/358eNH43/+5x9o164d4uIewJtvvoNx4x7V1/e5557B00/PQu/efeqVSSaT2fS34PKw2LhxI+bPn4+5c+fqz2VlZSEuLg4tW2oHW4YPH46MjAzMnj27UZ917VqlTYtTYluHQC6XAbdaFrGtQ1BcXNGoMniLiIjmflNXHdbZd4miqB/ktTTg6wpz5rwBACgouIIXX5yFf/3rfwEAO3duw4ABA/H22+/qn/vTT3vxt78twoABg/Tntm9Px8MPD0Z6+lYkJ88wem+N5nZdV6xIQ1xcPFq3jgSgnRVm+LghURSN/hZkMqHB/8B2edwuWrQIffoYp1xRUREiIiL0x5GRkSgsLHR10aCICcPrU3rhiZH3sAuKyE/l5JdjR3YucvLL3fL5V68WoEWLFrfLk/M7bty4gWnTpmPbtq0NToudNGkK3n//PaeUyyNGgERRNOpDkyTJbf2NipgwxPeM9Yv/+iIiY7pxS7VGRICLxi337/8JKSlTcfPmTdTUVKNv33744IOl+sd37PgWQ4YMRdeu90Aul+PQoWzExw8w+V7Tpk3Hvn17sW3bViQmjndoOT1iXlndKbLFxcWIjIx0Y4mIyB+dyyuDWiNCkrTdO+fyypz+mQ89NBBr1/4v1q7dgNjYO3DHHe3Rvn0HAIBarcb33+/C0KHDAQAJCUORnr7J7HsFBATgnXf+H1at+hSFhVcdWk6PaFn0798faWlpKC0tRXBwMDIzM7FgwQJ3F4uI/EyX9q0QIJfpZ0R2ad/KZZ/drFkoUlP/H5KTJ6NPnwdx3309cODAT6ioqMBbb70OQBseZWWlKCoqRGRklMn36dRJgUmTpuDDDxc6tHweERZRUVGYM2cOkpOToVKpMHHiRPTo0cPdxSIiP6MbtzyXV4Yu7Vu5fNyyXbsYPPbYZHz88RKsXv0FduzYhmeeeQ5PPJGif87s2TOxbdtWPPXULLPvM23adPz001789785Diub27qhdu/erV9jAQCJiYnYvn07vvvuOzzzzDPuKhYR+TlFTBhGx3d02wSXJ5/8MwoKriAjYwdOnDiKMWOMxx6Skp7A9u3p0GjML6oLCAjA22/Pd2i5BEmSfHajd1unzgLaAa7L15R+t4LbX6ZUGmKdfdfVqxcRHa3t9/eEqbOuZG19DX9HgOWpsx7RDeUpuIKbiMg0j5gN5SnO5ZUhFoUY0uRnxKLQJTMhiIi8AVsWBu5rfh1xzTMhhwYayFHZvIu7i0RE5BHYsjAQrbqMQEEDuQAECiKiVZfdXSQiIo/AsDAgNA2Fbt24AAlCU9s2IiQi8lUMCwNSdSVwKy4kCCgpKnFvgYiIPATDwkBAu66QZHKIEqCWBKw9LrptMzEiIk/CAe46NKKkT1CNRsK5vDJOnyWiRlm69EP8/PMpqNXa+1l07Ki9n8WkSUn49NO/6+9nofP662+hW7f7sGfPf7B+/VpoNBpIkogRI0Zj6tRkHDqUjRUr0gAA+fmXEB7eGsHBIWjbth0++miZU+rAsDBQePYkQiQNZAIASUTfJn9A0X6Eu4tFRC6kKcyB+spZBLTrCnlU427ApvPaa8b3s1i79vb9LB56yPh+FjrFxUX49NO/4/PP/42wsJZQKpWYPXsm2rfvgIceGoR+/eIBaLf/mDFjJh54oP4NjhyJYWEgRxWF+yCDTBIhAIhr8geaBxQDYMuCyB9oCnOg3L4YEFWolQUiZMxchwWGra5fvw61Wo3q6mqEhQEhISFITX0XQUFN3FIehoWBdvf0wOHfFIgPOq9vXRSePYl2bvpjISLXUl85C4gqQJIAUQ31lbNODwvd/Sx0AgMD8Y9/fIHOne/Gww8PwuOPj8Pdd3dBr1598Kc/jUBs7B1OLY85DAsDipgwnG3fD+qrf0AuidBAhhxVFNq5u2BE5BIB7bqiVhYIiGpAFoCAdl2d/pnmuqEA4C9/eRPTpz+Fw4cP4vDhbMya9WfMn78AgwYNcXq56mJY1NG1Tx+s3ngNPQNyIBOArtG+f3N7ItKSRykQMmauw8cs7JGVtR9VVUo88sgwjB49FqNHj8W3327B9u3pbgkLTp2tQxEThlmP9kB80/8iPuh3hB/6H2gKHbcnPBF5NnmUAk16jXFrUABA06ZNsXLlZygouAJAe7vp338/j86d3bMNEVsWJkTWXkIZNABc129JRP6p7pgFAEyePBUjR47BjBnPYO7cV6BWqwEA/frFIyXlaXcUk/ezMKV5zRXkr58PSVRDkAWgWeIbPh8W/nKfA0Oss+/i/Swcfz8LdkOZkKuOwO6qLijRhGJ3VRdcUEe4u0hERG7FsDDhStYODAk6gwhZBYYEncH1E9+7u0hERG7FsDDhjqrzAADh1ha0d9782Y2lISJ7+HAPe6PZ87thWJggte+t/f9bv8/AG5c5I4rIi8hkcmg0ancXw2NpNGrIZHKbXsOwMOGU7F78rNKukhQEAJJGu7KTiLxCcHAoKiquQ5L8Z2DbWpIkoqKiDMHBtt2vh1NnTejeqQ32/RgG4JK2dSEAUq3S3cUiIiuFhoahrKwYhYWXIZMJEEX/CQ2ZTGahvgKCgpoiNNS2Pe8YFiZ07RgOKUYNlGhvhSQBEEvy3F0sIrKSIAgID48E4D/ThXWcVV92Q5lR07YHIN0at5CAsvB73V0kIiK3YcvCjB/K70KNMg73B17EKVUHNCm/C8nuLhQRkZswLEyovnwO7Uv24ydNK2TX3A0A6HWz1s2lIiJyH3ZD1aEpzEHBhnfRS3kALzTPRMeAYgDA9coa3o+biPwWw6IO9ZWzkDRqyCBBDhGKgKsAgAsFFfjo/04wMIjIL3l8WPz6669ISUlx2ecFtOsKSRAgAoAgQ7BQi2dDv0d8k/NQa0ScyytzWVmIiDyFR49ZXLp0CXv37oVcbttKw8a4XFyJ5hoJcgASNHik6S8AgK6BBWgjq0CXW6u7iYj8iUeFxZo1a7B//3798eeff47nn38es2bNclkZyv/4GWEQb92DW3tOELRTaB8J/gUBv3wBTcAon9+ynIjIkEeFxdNPP42nn3bPjT10wjp1h+bqXkDS6PvoJOn2poKa3ONQ5p1GSOI8BgYR+Q2PH7NwtQ739YQw/HVcDeoACbdDwmiPxlt3zyMi8hcuCYvKykqMGTMGly9f1p/btm0bRo0ahWHDhmHDhg0Nvn7VqlXOLqIRKaITNl/vBjXk0N1oT4C2hSEBgCBDQLuuLi0TEZE7Ob0b6tSpU0hNTUVubq7+XGFhIZYvX47NmzcjKCgISUlJ6NevHxQKx3brNHSLwIbsPX0e/61tg880w5AUkoVouXa6rG7sQoKEkNoStIjo5cjiul1ERHN3F8HlWGf/4G91dkZ9nR4WGzduxPz58zF37lz9uaysLMTFxaFly5YAgOHDhyMjIwOzZ8926Gfbew/u7p3aQC6XQQDQRq7dkEsyHOyGhJKMf0AZ1MZnxi38bbM1gHX2F/5WZ3vr6/Z7cC9atAh9+vQxOldUVISIiNv3tY6MjERhYaGzi2K1rh3D8fqUXhjXuRYBgqQftwBuDXYDvMcFEfkVtwxwi6IIweAbWJIko2NPoIgJQ7e4eIiCwbiFoGtZ3Dpual83FxGRt3FLWERHR6O4uFh/XFxcjMjISHcUpUH7CoLxSfmfkKduoz+na1lIECBVV7qvcERELuSWsOjfvz+ys7NRWlqKqqoqZGZmYuDAge4oSoOOnStCrjoCW6r6Qg2ZvoUhSoBGkHNGFBH5DbcsyouKisKcOXOQnJwMlUqFiRMnokePHu4oSoN6d4nELxfKkKuOwKcVw6EIuIqbUhM0E2rQ4s77MMZHBreJiCxxWVjs3r3b6DgxMRGJiYmu+ni7DO4ZA0Dbwmgf2R6ZRy+hvVCEzkGF6N6ptZtLR0TkOh613YcnGtwzRh8aHQOKofjte8ihgXDoDDQRc31m6iwRUUO43YeV9p7Mxx/Hj0CQ1BAgQdLUoubkTncXi4jIJRgWVsjJL8e/M8/jvCpKOwvq1kC35uJxKHe7disSIiJ3YFhY4VxeGURRQq46AsWaFgB002cBTU42an/b687iERE5HcPCCs2CA/U/F4kt9D/rlhHWnNju4hIREbkWw8IKN6tU+mDYXX0fRNzeKwoAUFmC6kMb3VAyIiLXYFhYoUv7VggI0P6qctUROKO6o95zVKd2QVOY4+qiERG5BMPCCoqYMLw+pRcG92wHmWCmdQEJtecPuKmERETOxbCwkiImDMkjumLg/e2Qq47A18o4fWDoMkN99ke2LojIJzEsbNS/e1vIZUB2zd24dGuDQf1+uZLIbcuJyCcxLBrhYK3CICm0uG05EfkihoWNsn4ugEbU/pxdczd+rr0Dt4cuuG05EfkmhkUj/VB9HzSQ6481xf/luAUR+RyGhY36d2+Lejf1k8RbrQsJmtzjUH77AQODiHwKw8JGipgw9FTcvnOeIuAqZJCMhy4kDafREpFPYVjYIaxZkP7nHHU0gLprLuqNexMReTWGhR36d28LuVwbB7nqCPyhrn//cHX+r+yKIiKfwbCwgyImDG9MfQBtWjQBAGyv6q1doGfwHOlGIZTpixgYROQTGBZ2ulxciZIbNQC0rYtvlHGQ6j1L4g2SiMgnMCzsdOxckdFxVs3dOFZzZ73AkJTXXVcoIiInYVjYqXeX+uMU/775MK7C+LwQFuWqIhEROY1NYVFbW4srV644qyxeZXDPGPTq3Kbe+aPKGKPWhSYnm7deJSKvZzEsvv/+eyxYsACVlZUYMWIExo0bhy+++MIVZfN4I+M61Fugl6OOhlhn4iwDg4i8ncWwWLVqFR5//HFkZmaiZ8+e2LNnD9LT011RNo+niAnDk8O7GJ3LVUfgqqxtvecyMIjIm1kMC0mS0KVLF2RlZWHgwIEIDQ2FVHcFmh8b3DMGd7ZtbnTugjLUxMwoBgYReS+LYSGTybBz507s378fAwYMwI8//gih3uZI/u3h+9sZHR+p7WS0uaAhTU42114QkdexGBZvvPEGNm7ciFdffRURERFYsWIFUlNTXVE2rzG4Z4x+gR6g7YpKuzEMVVHdTT5fmZnGwCAiryJINvQp1dbWoqSkBO3atbP8ZA9w7VolRNH2LrOIiOYoLq6w6TV7T+ZjXcY5o3N3x4bhlc5/QHXK9MK8kHGpkEcpbC6fM9hTZ2/HOvsHf6uzvfWVyQS0bm3+5m2cDeUgg3vGoLVB6wIAzl8ux+XY4ZB3fMDka5S7V7uiaEREjeaxs6GOHDmCuXPn4vXXX8fXX3/t9M9zhNH9O9Y7t+vgRTS5fxRM7kNbUYSbO5Y4vVxERI3lsbOhbty4gffeew8ffvghfvjhB6d/niMM7hmDsJBAo3N/5JdDHqVAyLi3gaBm9V4j5p9B9aGNrioiEZFdrJ4NtW/fPqfOhlqzZg1SUlL0/0tISEBgYCCWLFmC5ORkh3+es9wVE2Z0fEOpwtd7crSBMXKOydeoTu3klFoi8mgBlp7wxhtv4NNPP8Vrr73m1NlQTz/9NJ5++mn98Y0bN/DBBx9g6tSp6N7d9KwiTzQyrgNO/F5idG7XoTz0ujsCihgFQsalQrlrOVB70+g5mpxsKAGEDJnlwtISEVnHYsuiT58+WLt2LQYMGICLFy/iyy+/xAMPmB6wdaSFCxciPz8fX3zxBZYuXer0z3MURUwY7o4Nq3d+18GLANBgC0OTk80uKSLySBZbFrm5uXjhhRdQVFQEURTRqlUrrFq1Cp06dbLqAyorK5GUlISVK1ciNjYWALBt2zasWLECarUa06dPx7Rp0+q9bvHixTZWxXNMTFDg/fXHjM6d+L0Ee0/mY3DPGMijFAi8f5TJKbW6c037Pe6SshIRWcPiOounnnoKY8aMwaOPPgoA2LRpE9LT07Fu3TqLb37q1CmkpqbiwoULyMjIQGxsLAoLCzFlyhRs3rwZQUFBSEpKwrJly6BQeMZ6A0dZ9K9DOHjmar3zH734MLp2DAcAXPthPcoPbjX5+jYjn0WLB/7k1DISEVnLYsvi2rVr+qAAgMceewxr16616s03btyI+fPnY+7cufpzWVlZiIuLQ8uWLQEAw4cPR0ZGBmbPnm1j0S1z5aK8uob0ijEZFv/YchrznuitPegxHoFVtSZbGCW7VqKishpB9wxuVDms5W8LlwDW2V/4W53dtihPo9Hg+vXbd3srLS21+sMXLVqEPn36GJ0rKipCRESE/jgyMhKFhYVWv6e3UMSEYWS/9vXOn79cjq/33N7qo2m/xxF4/yiT71Gzby3HMIjII1hsWTzxxBOYPHkyRo4cCUEQsHPnTkyfPt3uDxRF0WjqrSRJPrsx4aQEBc5cKMWlokqj87sO5ekfB7SBId4sgyYnu957qE7thHizjLOkiMitLLYsJk+ejHfffRcqlQrV1dWYP38+pk6davcHRkdHo7i4WH9cXFyMyMj6tyj1FXXvd6Gz61AecvLL9cchQ2ZBFnOfyedya3MicjerbqsaHx+P119/HW+88Qb69+/fqKms/fv3R3Z2NkpLS1FVVYXMzEwMHDjQ7vfzdOa6owBgxZafjY6bjf4L5Ip4k89lYBCRO9l0D26dDRs22P2BUVFRmDNnDpKTkzF+/HiMGTMGPXr0sPv9vMGkBIXJwCirrMXSL08YnQsZMsvsGAYDg4jcxeKYhSm27g21e/duo+PExEQkJiba89Fea1KCAmUVNTj4q/Fg/i+5ZVj65Qm8ltRLf063xsLULClNTjaqm7XiOgwicim7Wha+OiDtbDPHdsOd0c3rndcFhqGGZkmpTu3kLCkicimzLYvMzEyT5yVJgiiKTiuQr3snpS/mrcxC0fVqo/O/5Jbh6z05+hlSgDYwZC0iUbNvbb334SwpInIls2Gxfv16sy/ypo39PNHTid3qbQcCaGdIRbQKxuCeMfpzQfcMhnijyGyX1M2qCjQb/RenlpeIyK6woMZRxIThrSd746P/OwGV2riVprs1q2FgNLQOQ8w/g5s7ljAwiMip7BqzoMZTxIRhytDOJh9bl3HOaA0G0PAsKV1gEBE5C8PCjQb3jEHcvVEmH6u7BgPQtjCaPJxi8vli/hlUfPWWI4tHRKTHsHCzmWO7mQyMsspaLFh7pN75oHsGmw0MlF9hYBCRU5gNiytXrph90U8//eSUwvirmWO7oVvHVvXOX7haYVdgVG5+z8ElJCJ/ZzYsXnjhBf3PL774otFjy5cvd16J/NRrSb1MrsFoKDBCxqUCTeq/Rir5Lyr+PQeawpx6jxER2cNsWBiu0r506ZLZx8hx3knpazYw6i7aA7S3aG0+PQ0Ia1f/zZRlUKYvRO1ve51QUiLyN2bDwnCVdt0V21zB7TzmAuOX3DKs/vYXk69pPvl9CG3uMvlYzb61DAwiajSrWhbkWu+k9EWr0KB65w/+Wmg2MEIn/LXBwOD2IETUGGbDQhRFlJeX4/r169BoNPqfdcfkXM89anqV/MFfC43utGeoocBQndrJtRhEZDezK7jPnz+PuLg4fQujX79++sfYDeV8ipgwJI/ool/RbcjUtiA6oRP+ips7lkDMP1PvMa72JiJ7mQ2Ls2fPurIcZIIuDEwFhqltQXSajf4LlLtXcXsQInKYBhflSZIEtVoNAKisrERmZiYuXrzokoKR1uCeMWbvtLcu4xz2nsw3+Zil7UEqvniRU2uJyGpmwyInJwePPPII9u3bh+rqakyaNAnLly/HE088gQMHDriyjH5vUoLC7LYgpvaR0mloexDUVECZvpAD30RkFbNhsXjxYrzyyitISEjAjh07AAA7duzAxo0bkZaW5rICkpa5bUEA0/tI6TS42hsc+CYi65gNi4KCAowdOxYAcOjQITzyyCOQyWRo27YtKisrXVZAus3WfaR0GlrtDWi7pfJWvuywchKR7zEbFjLZ7YdOnDiBvn376o9ramqcWyoyy9Z9pHR0q73NTa1VX7uMin/O5AI+IjLJbFiEhYXh7NmzOHr0KIqLi/Vhcfz4cURFme4OIddoaB8pU9uCGAqd8FezA9/Q1KJm31p2SxFRPWbD4tVXX0VKSgpSUlLwyiuvICQkBP/85z8xa9YsvPTSS64sI5lgz7YgOk37PW6xW4obERKRIUFqYF+P2tpaVFdXo0WLFgC0rYrw8HB07NjRVeVrlGvXKiGKtm9bEhHRHMXFFU4okeO99ul+lFXW1jsfd28UZo7tZvH1FV+9BZSb345erohHyJBZjSqjp/Km6+worLPvs7e+MpmA1q1DzT9u7oHr169DqVRCFEX9Nh933XUXWrZsievXr9tcEHKOhrYFsdTCALSbEMoV8WYf1+RkcyyDiMy3LLp27arf1qPuUwRBwG+//eb80jWSP7QsAGDvyXyTq7wBYGS/9piUoLD4Hk0uHUTJrpUWnhSKJg9ORNA9g+0opefxtuvsCKyz73NWy8Lsdh/jx4/HiRMnMGTIEDz22GNQKCx/4ZB7NLQtSEP7SBlq8cCfoAxqA+Xu1UBFkekn1VSiZt9a1Bz+xqdCg4gsa3DMoqqqCpmZmdi6dSuUSiXGjh2LxMRE/RiGp/OXloVOQy2M5BFdGgwMwzrX/rYXNVn/C2jqj4UYEeSQd3rQa8c0vPU6Nwbr7Puc1bJoMCwMXb16Fenp6di1axc6duyIv//97zYXxtX8LSwA4Os9Odh1KM/kYw0Fhqk6m9u9tj4ZZNEKNO33OORR3tMC9ebrbC/W2fe5fIC7rtLSUpSWlqKsrAwVFf7zi/c29u4jZUqz0X/RTrFtHmnhmSLEq+ehTF+IivUvczCcyAc12LIoKCjAt99+i/T0dMjlcowdOxZjx451yaK833//HWlpaQgJCUFiYiIGDBhg83v4Y8tCZ/W3v+Dgr4X1zrcKDcLS2Q/VO2+pzrW/7UXNkS1AtZVh4wVdVL5wnW3FOvs+lw9wP/nkk7hw4QJGjRqFJUuW4N5777X5wxtDqVTirbfeglwux7Jly+wKC3+mW2NRNzB0+0i9k9LX1MvMCrpnMILuGQxNYQ6qD22EePV3AA0EsaTRTrvNOQghtDWCeo3hgDiRF2tw6myTJk0gk8mM7ownSRIEQcDx48cdWpA1a9Zg//79+uPPP/8ceXl5mDdvHpKTkzFqlJktKhrgzy0LnaVfnsAvuWX1zt8Z3dwoMOypc/WhjVCd/g6QrLzNriCH0KyVxwSHL11na7HOvs/lA9z5+aZvqqMTE9PwVMzGOnPmDDp27IjQ0FDMmDEDn3/+uc3vwbDQWrD2CC5crV+fbh1b4bWkXgAaV+fqQxuhOvMDoLFlg0kZhNBwtwaHr11na7DOvs/ts6Fc7dixY1i3bh1CQ0Nx9913Y/r06Ta/B8PiNnOBoVu054g6W91FVY8ABLdAkz6PujQ4fPE6W8I6+z6vDYvKykokJSVh5cqViI2NBQBs27YNK1asgFqtxvTp0zFt2jSnfDbDwpi5faSSR3TBpD91dWidqw9thOrn7wFRZfuL5YGQRdzp9Km4vnqdG8I6+z6vDItTp04hNTUVFy5cQEZGBmJjY1FYWIgpU6Zg8+bNCAoKQlJSEpYtW8YV4i5wNrcUr6ftM/nYYwkKpIyxvPGgrW4c/x5lB76BprIMEK0c2zAiAPJANGmnQOshT6BpbBeHl5GILHNqWLz99tt49NFHMXfuXKxbtw6xsbHYsmULjhw5gvfffx8A8Nlnn0GSJMyePdvhn8+WRX2O2EfKXprCHFTt+wJSaT4A0c53kQEyAQhogsB7BqNpv8ftLo8vX2dzWGff5/Kps46waNGieueKiooQERGhP46MjMTp06edWQwyMLhnDIrLqkyu8rZ2Hyl7yaMUCJ24AIDB+EbRHza2OERtztQqoTq1E6pTOwFBBkgSEBCEwG6PNCpAiMg0p4aFKaIompyKS64zKUGBsooak4v2dK0OZwWGjjxKgWZj3wJgEBzFFwCNHWMc0q1WirrmdoBAAAQBkMkhBId5zHRdIm/l8rCIjo7G0aNH9cfFxcWIjLS0nQQ5mrlFe6buiCAAABXZSURBVIA2MGIjQqGICXNJWQyDA7i1Wvz4dqDK3nEOAJC0rQ2NCKmyRLtb7r51gADteXkAatt1hvyBCV61nxWRu7g8LPr374+0tDSUlpYiODgYmZmZWLBggauLQWg4MFZs+dnktiCuoFstrqPfaqTmVj+sZO94h3h7Rq9GhZpLvwKXfoV+HESSAEGA0DIGwQ8nM0SIDLg8LKKiojBnzhwkJydDpVJh4sSJ6NGjh6uLQbc4elsQZ6gbHgCg3L0Kmv8euR0cdgcIoB8HAbQNktI8KNMXao8FmbY7i0FCfs5jF+U5AmdDWS9t82mcOF9S73zdbUE8WfWhjVD9uhdQV2u/3G1aGGirW+NsuvG2W11brlgf0hj++Lftb3X2ynUW7sawsF5ERHO89NFuk6u824aHYNHMODeUqvH04x/V5drWhyQ1shViA+HWHQButUq0A+4BkLXp4LZA8de/bX+qM8PCDgwL6+nqbG5bEG8ODFNqf9sL1bEtEKtu1VUQGjGYbi+D1omuq0sXLPJAp4SKP/9t+wuGhR0YFtYzrLO5wPCmLilrmLrOxjOxxNtf5C4PEgN1WyiGP+tCBgCaNre4v5a//237A4aFHRgW1qtb57dXH0RBqbLe83yphWHrdTbq0hI1db68XdS1ZRUT4yn6YIG2rIaPyWQQQjxn63hH87d/zwwLOzAsrGeqzr4eGI6+ziZbJcDtL2uPCpQGmBprMewia+ixwKaN3obF0fzt3zPDwg4MC+uZq7O5LqlWoUF47tHuLlu45wzuuM76GVuaGuMvWcMvXafO4nIhmdx8HS2FTt3HGrEXmL/9e2ZY2IFhYb2G6myuhQFotzd39tYgzuLJ19koVICGv0hFET4TMNbQ7QXW2ABq6DHAIZtVugPDwg4MC+tZqrMvBoYvXWf9/lolF7X7a5n9gkSdMQsv6RpzK4NwcWQgOalrj2FhB4aF9ayps7kuKcA7A4PXWct4I0e1/V9s/tS6cRdLXXuCACGwKQK6DrK5RcSwYFhYxdo6L/3yBH7JLTP5mLPvh+FovM6OZ7QNi4kvMrv+S5utH7sE3j/KpsBgWDAsrGJLnRsKjLh7o/T7TXk6XmfvUX8vMCd2AwE+EVBCiyiEJn1o9fPdevMj8k2vJfUyGxi6DQm9JTDIO4QMmQUMmWXXa+0JSKMxIFHt/HEJc481omsv4M7edr/W5Ps59N3Ib7yW1Aurv/3F5PbmB38tRIWyFq8l9XJDyYgar+49VtzJpq69RoxZWMJuKBO8taneGPbW2VxgAJ6/eI/X2T/4W52dNRtK1phCEc0c2w0j+7U3+VhBqRIvf7wPOfnlLi4VETkaw4IabVKCAskjuph8rKJKhffXH8Pek/kuLhURORLDghxicM8YvPVkbzQPNj0Mti7jHAODyIsxLMhhFDFh+PjlgWgbHmLy8XUZ5/D1nhwXl4qIHIFhQQ63aGYcunVsZfKxXYfysPTLEy4uERE1FsOCnOK1pF5mA+OX3DK8vfqgi0tERI3BsCCnaSgwOFOKyLswLMipXkvqZXZqrW6mFMcxiDwfw4KcblKCosGZUrsO5WHB2iMuLhUR2YJhQS5haabUhasV7JYi8mAMC3KphmZK6bqlVn/7i4tLRUSWMCzI5RoaxwC0GxGyW4rIszAsyC0sjWNcuFqB55f9yFXfRB6CYUFuoxvHMNctVV2rwbqMc2xlEHkAhgW5naVuqQtXK/DM4j2cYkvkRh4dFhqNBk8++SR+/vlndxeFnEzXLdUqNMjk4xpRwq5DeZwxReQmHh0WK1euRGRkpLuLQS6iiAnD0tkPIe7eKLPP0c2Ymv/5YYYGkQt5zG1V16xZg/379+uPp0yZgs6dO0MUvf/G6WSbmWO7YUjvWKR9cwoVVWqTz7lUVIn31x9Dt46tePtWIhfw2NuqvvrqqwgNDcWZM2fQqVMnfPTRRza/B2+raj1PrfPXe3Lw3eE8NHQZZYK2VTIxQQFFTJjV7+2pdXYm1tn3Oeu2qh4bFjppaWkYPHgwunfvbvNrGRbW8/Q6L1h7BBeuWi7fHZGheHJ4F6tCw9Pr7Ayss+/z2ntwV1ZWYsyYMbh8+bL+3LZt2zBq1CgMGzYMGzZsaPD1L774ol1BQb7lnZS+SB7RBSFN5A0+T9c99dzSHzl7isiBnNqyOHXqFFJTU3HhwgVkZGQgNjYWhYWFmDJlCjZv3oygoCAkJSVh2bJlUCgUzioG+ZiM7FysSf8ZNSrL41lyGdC1YzhSRndD147hzi8ckY9yali8/fbbePTRRzF37lysW7cOsbGx2LJlC44cOYL3338fAPDZZ59BkiTMnj3b4Z/PbijreWOdv96Tg8wjedBYOQcirFkQxj18Jwb3jAHgnXVuLNbZ9zmrG8qps6EWLVpU71xRUREiIiL0x5GRkTh9+rQzi0E+alKCApMSFPh6Tw7+c/QSVJqG/8Og/GYt1mWcw/rvziE2IhQvTe6F1s0CXVRaIu/m8qmzoihCEAT9sSRJRsdEttKFxt6T+dj6039xQ6lq8PmSpB3beD1tH+QyAX27RmLm2G4uKi2Rd3J5WERHR+Po0aP64+LiYi68I4cY3DMGg3vGICe/HN/syUFOfnmDU24B7crwg78W4uCvhQhpIsegnjGYlMDxM6K6XB4W/fv3R1paGkpLSxEcHIzMzEwsWLDA1cUgH6aICcO8J3oD0I5r7D5+2arBcGWNBrsO5WHXoTwGB1EdLg+LqKgozJkzB8nJyVCpVJg4cSJ69Ojh6mKQn9B1UdnS2gCMg6NJoAwdoprbvOiPyJd4/KK8xuBsKOv5U533nszHjqxclN6oga1/HTIZEB7aBKP6d9TPqvIm/nSddfytzn67grsxGBbW89c6/8/GE/jxZD6UNRq73iNQLsOdbb2n1eGv19mf6uyVU2eJPJ2umwoAVn/7C46eK4LawhRcQyqNiPOXy/H++mMQBCAogF1W5JsYFkS3zBzbDTOhnUJrT3BIElCjMg6PAJmAsGZBXtttRaTDbigT/K3ZCrDODdGt36hQqmwe4zAkCIAAQC4T0LuLe9Z28Dr7PnZDEbmJbv0GAP2sqgsFNyyuGK9LkgAJgKi5vbZDgPYfaatQtj7Is7FlYYK//ZcIwDrbS9fqqKxSWTUl1xoyAWga5Jx1HrzOvo8tCyIPZNjqABwTHqJkvM5DfutGAk0CuVCQ3IdhQeRApsJjR1YubtyshShJVu+Qa0j3GsMAEXBrDERgiJBrMCyInKhueADaLUh+PJmP6lqN3a0PCdoxEJhohUiS9n+CAARyKi85CMcsTPC3Pk6AdXYnw0FztUZq1IyrhshutUREUfv/gDZUggLlGPKA77ZMPOU6uwpXcNuBYWE91tmz6NZ56P5+HTV4bomuewu43ToRhNs/t2tj/T3OPYUnX2dnYFjYgWFhPdbZs+laHxcLK6BSi/ovb1eFiCkygyDRlQcwDhmZ4P5Fid50nR2BYWEHhoX1WGfvZdgKMfzy9rR/2LpFieaCpaHQacwgvq9cZ2sxLOzAsLAe6+ybdIPpNarbGyUajlm4s2ViL5lgfcjoQlN3L05f6VprCMPCDgwL67HO/sFUnQ1nZ5n70rVnyq83MtW1ZlUgWXgswIW7EzMs7MCwsB7r7B8aU2fdmpHrlTX61klDX5ae2BXmCbRjOY0PIHPHTYMCMPD+djZ32XEFNxE5hKk1I5bUXZRozxekN3aVNUSSAMNtxaS6P9vymInjm9Vq7DqUBwAOnQ7NsCAip7EnYEypO5XYEWMWvt61dux8EcOCiPyL4b1GbNVQ15s1XWuNHbNwV8uo992RDn0/hgUR+S1HtXwsMdxgEnDMoLmjxywsYVgQETmZq0IJcN7EDZnD35GIiHwOw4KIiCxiWBARkUUMCyIisohhQUREFvn0bCiZTLD8JCe81luxzv6BdfZ99tTX0mt8em8oIiJyDHZDERGRRQwLIiKyiGFBREQWMSyIiMgihgUREVnEsCAiIosYFkREZBHDgoiILGJYEBGRRQwLA9u2bcOoUaMwbNgwbNiwwd3FcahPP/0Uo0ePxujRo7F48WIAQFZWFhITEzFs2DAsX75c/9zffvsNEyZMwPDhw/H2229DrVa7q9gO8eGHH2LevHkAzNftypUrmDZtGkaMGIHnnnsON2/edGeR7bZ7925MmDABI0eOxMKFCwH4/nVOT0/X/21/+OGHAHzzOldWVmLMmDG4fPkyANuva6PrLpEkSZJ09epVKSEhQSorK5Nu3rwpJSYmSr///ru7i+UQBw4ckCZPnizV1NRItbW1UnJysrRt2zZp0KBBUl5enqRSqaQZM2ZIe/fulSRJkkaPHi2dOHFCkiRJevPNN6UNGza4s/iNkpWVJfXr10964403JEkyX7eZM2dK27dvlyRJkj799FNp8eLF7ilwI+Tl5UkPPfSQVFBQINXW1kpTpkyR9u7d69PXWalUSn379pWuXbsmqVQqaeLEidKBAwd87jqfPHlSGjNmjNStWzfp0qVLUlVVlc3XtbF1Z8vilqysLMTFxaFly5YICQnB8OHDkZGR4e5iOURERATmzZuHoKAgBAYGolOnTsjNzUWHDh1wxx13ICAgAImJicjIyEB+fj6qq6vRs2dPAMCECRO89vdw/fp1LF++HM8++ywAmK2bSqXCkSNHMHz4cKPz3ub777/HqFGjEB0djcDAQCxfvhzBwcE+fZ01Gg1EUURVVRXUajXUajUCAgJ87jpv3LgR8+fPR2RkJADg9OnTNl1XR9Tdp3edtUVRUREiIiL0x5GRkTh9+rQbS+Q4nTt31v+cm5uLXbt24YknnqhX38LCwnq/h4iICBQWFrq0vI7y17/+FXPmzEFBQQGA+tdYV7eysjKEhoYiICDA6Ly3uXjxIgIDA/Hss8+ioKAAgwcPRufOnX36OoeGhuLll1/GyJEjERwcjL59+yIwMNDnrvOiRYuMjk19XzV0XR1Rd7YsbhFFEYJwe4teSZKMjn3B77//jhkzZmDu3Lm44447TNbXV34PX3/9Ndq2bYv4+Hj9OXN1M1VHb6yzRqNBdnY23n//fXz11Vc4ffo0Ll265NPX+ezZs9i0aRP27NmDffv2QSaT4cCBAz59nQHzf8vO/Btny+KW6OhoHD16VH9cXFysb/L5gmPHjuGll17CW2+9hdGjR+Pw4cMoLi7WP66rb3R0tNH5kpISr/w97Ny5E8XFxRg3bhzKy8uhVCohCILJuoWHh6OiogIajQZyudxrr32bNm0QHx+P8PBwAMDQoUORkZEBuVyuf46vXef9+/cjPj4erVu3BqDtXvnnP//p09cZQL3rZ+m6OqLubFnc0r9/f2RnZ6O0tBRVVVXIzMzEwIED3V0shygoKMALL7yAJUuWYPTo0QCA+++/HxcuXMDFixeh0Wiwfft2DBw4EDExMWjSpAmOHTsGQDvTxBt/D//617+wfft2pKen46WXXsKQIUPwwQcfmKxbYGAg+vTpg507dwIAtm7d6pV1TkhIwP79+3Hjxg1oNBrs27cPI0aM8Onr3LVrV2RlZUGpVEKSJOzevRsPPvigT19nwPZ/v46oO29+ZGDbtm1YtWoVVCoVJk6ciGeeecbdRXKIhQsXYtOmTWjfvr3+XFJSEjp27IgPPvgANTU1GDRoEN58800IgoCzZ88iNTUVlZWV6NatGz744AMEBQW5sQaNs3nzZhw+fBh/+9vfzNYtPz8f8+bNw7Vr19C2bVssW7YMYWFh7i66zb755husXbsWKpUKAwYMQGpqKg4dOuTT13n16tXYvHkzAgMD0b17d8yfPx8XLlzwyes8ZMgQrFu3DrGxscjOzrbpuja27gwLIiKyiN1QRERkEcOCiIgsYlgQEZFFDAsiIrKIYUFERBZxUR75nIULF+LIkSMAgD/++AMxMTFo2rQpAOCrr77S/2zJDz/8gOzsbKSmppp9TmFhIV5++WV8+eWXjS84gHnz5uHAgQP6hXU6jz32GJKTkx3yGYaf1blzZzz11FMOfV/yTZw6Sz5tyJAh+Pjjj9G9e3d3F8UqrvwCZ1iQLdiyIL9z33334ZFHHsHZs2exZMkSnDt3Dl999RVUKhXKy8vxzDPPYOrUqdi8eTO+++47rFq1Ck8++SR69uyJ48ePo6CgAPHx8ViwYAGuXLmCxMREnDhxAmlpacjPz0dxcTHy8/MRFRWFjz76SL8p5bvvvguVSoX27dvjypUrmDdvHvr162dT2YcMGYLRo0fjwIEDqKiowJ///GdMnToVgLbVtH79eshkMrRp0wbvvPMO7rzzTty8eRMLFy7E8ePHIZfLMXToUMyZMwcAcOLECSQlJaGkpASdO3fG0qVLERIS4vDfOXk/hgX5HZVKhYSEBHz88cf6L9LVq1ejVatWOHnypNEXsKG8vDysX78eSqUSI0eOxOHDhxEbG2v0nKNHj2Lr1q0IDQ3Fs88+iy+//BLPP/88XnzxRbz33nsYNGgQDh48iJSUFLPlW7t2Lb799lujc4sXL0aXLl0AAOXl5di0aRMKCwsxfvx49O7dG6WlpVizZg2++uorhIeHY/PmzXjhhRewY8cOfPLJJ6ipqcHOnTuh0WgwY8YMHD58GIC2G23dunUICgrCpEmTkJmZifHjxzfyN0y+iGFBfqlPnz4AgGbNmmHlypX48ccfkZubi7Nnz0KpVJp8TUJCAmQyGUJDQ9GhQweUl5fXC4sHH3wQoaGhAIB7770X5eXlOH/+PABg0KBBAIC4uDijbePrSklJabBraOrUqRAEAdHR0Xj44Ydx4MABlJSUYNSoUfqxjgkTJmDRokW4fPkysrKy8Oabb0Iul0Mul+Pf//43AGDLli0YOnQogoODAWi3si8tLbX4uyP/xNlQ5Jd0XS1Xr17F+PHjkZ+fj969e+OVV14x+xrDgXHdts/WPEcul9d7ruFOsLbS3ZMA0G5VLZPJIIpivedJkqS/GZDhdtQFBQUoKyur917m6kQEMCzIz505cwbh4eF4/vnn8dBDD2HPnj0AtPeGcJROnTohKCgIP/30EwDtXc7Onz9v970Utm7dCkB7T+UDBw5g4MCBePjhh7Fz5059y2DTpk1o2bIlOnTogPj4eGzZsgWiKKK2thYvvfSSfrYYkbXYDUV+bcCAAfjmm28wYsQICIKABx98EOHh4bh48aLDPiMgIABpaWmYP38+li1bho4dO6JNmzZmp/CaGrO4//778d577wEALl++jAkTJqC6uhqpqam46667cNdddyElJQXTp0+HKIoIDw/HqlWrIJPJMHv2bCxatAjjxo2DRqPBqFGjMGzYMOzevdthdSTfx6mzRC7w4Ycf4qmnnkKbNm1QUFCAcePG4T//+Q9atGhh0/t421Rg8h1sWRC5QExMDFJSUhAQEABJkrBw4UKbg4LIndiyICIiizjATUREFjEsiIjIIoYFERFZxLAgIiKLGBZERGQRw4KIiCz6/4LiHHsjkVhXAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "optimizer = torch.optim.SGD(net.parameters(), lr=0.03, momentum=0.9)\n", "net.load_state_dict(torch.load('net.pth'))\n", "losses_train, losses_test = [], []\n", "for epoch in range(1000):\n", " net.train()\n", " y_pred = net(x_sparse)\n", " loss = loss_fn(y_pred, y_sparse)\n", " optimizer.zero_grad()\n", " loss.backward()\n", " optimizer.step()\n", " losses_train.append(loss.data)\n", " net.eval()\n", " y_pred = net(x_test)\n", " losses_test.append(loss_fn(y_pred, y_test).data)\n", "plt.plot(losses_train, '.', label='TRAIN')\n", "plt.plot(losses_test, '.', label='TEST')\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In cases like this your best option is to include more data, but this is not always possible. Your next-best option is to reduce the complexity (number of parameters) of your model, with fewer layers and/or hidden nodes.\n", "\n", "However, there is a third option that can help prevent memorization, called \"data augmentation\". The idea is to replace each of your input samples with a group of similar (but not identical) augmented samples. There are two general strategies for augmentation:\n", " - Exploit (approximate) symmetries in your data. For example, if we know the function we are trying to learn is even, $f(-x) = -f(x)$, we can augment each input sample $(x_i, y_i)$ with its mirror partner $(-x_i, -y_i)$. Perhaps it would be better to use a model with the appropriate symmetries built in but this is often not straightforward (especially with neural networks), so we instead symmetrize the input data to (approximately) symmetrize the model's predictions.\n", " - Perturb or distort each input sample to produce some equally plausible input samples. Selecting suitable perturbations requires some assumptions about the (unknown) distribution that the input data is drawn from, but small perturbations are generally safe.\n", "\n", "In our example of learning $x\\rightarrow y = \\sin(x)$, we augment each sample using even symmetry:" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "# Add (-x, -y) for each (x, y).\n", "x_augmented = torch.cat((x_sparse, -x_sparse))\n", "y_augmented = torch.cat((y_sparse, -y_sparse))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this plot, the original samples are drawn as circles, and the augmented samples are red dots:" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD7CAYAAABpJS8eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xU953v/9eMRqPeNRo1QEISklBBgETvYEAg043BjonjvU7izY2z3rve38bOI3k4G98kjr0pdtp6fe11ibPYMcY0gQGbJsCADaihBogipFHvZcr5/UE0ZpAEQoWRZj7Px8MPM+ecmfP5SvDW0fd8z/erUhRFQQghhMNT27sAIYQQ94cEvhBCOAkJfCGEcBIS+EII4SQk8IUQwklI4AshhJOQwBdCCCehsXcBd1Jf34rFcufHBIKCvKmtbblPFQ0/R2qPI7UFHKs9jtQWcKz2DKYtarWKgACvPveP6MC3WJS7Bn73cY7EkdrjSG0Bx2qPI7UFHKs9w9UW6dIRQggnIYEvhBBOQgJfCCGchAS+EEI4CQl8IYRwEhL4QogBURQFo9GI2Wy2dymin0b0sEwhxMjT0FBPQUEeRUUXMJlMWCwW/Pz8SUpKYcKEeNzc3OxdoujDoAO/paWFTZs28ac//YnIyEibfYWFhTz//PO0traSnp7OCy+8gEYjP2OEGK1yc89z+vQXJCZOZP36jfj6+qEoCjduVJCXl8vZs2dYvnwlOl2IvUsVvRhUl865c+fYvHkzly9f7nX/s88+y49//GP27t2Loihs3bp1MKcTQthRYWE+58+fZf36h5gxYxa+vn4AqFQqwsMjWLp0ObNmzWX37h00NNTbuVrRm0EF/tatW/nJT35CSEjPn+bXr1+no6ODtLQ0ANatW0d2dvZgTieEsBOjsYsTJ46TmZllDfrexMTEkpqaxqlTJ+9jdaK/BtW/8uKLL/a5z2AwoNPprK91Oh1VVVX39PlBQd79Ok6n87mnzx3pHKk9jtQWcKz23EtbcnNzmTAhmvj4cXc9dt68Gbzxxhu4mlpoOX4S/ZJFuPr6DqbUfnHW7829GLYOdYvFgkqlsr5WFMXmdX/U1rbcdU4Jnc6H6urmAdU4EjlSexypLeBY7bnXtnz5ZS7x8Yn9fk9goJ7cv3yA+vAhWls7CVy+YqCl2jAajZSVlVBQkE9DQz2KouDr68fMmemEhIxFq9UOyXnsaTB/z9Rq1R0vlIct8ENDQ6murra+rqmp6bXrRwgxMnR1ddHc3ITZbMbb2xtPz69nXezs7MTDw6Pne27cwLD1fUI2bkYbFmbd7unpSUfcBMaG6PGbPXfANZmbm2k8dgS/2XOpbW9j797dBAUFM3VqOnp9GCqVipqaaq5cKWH//kM88MAyIiIi7/7BTmrYAj8iIgI3NzfOnDnD1KlT2b59O/PmzRuu0wkhBqiy8gb79u3h3LmvUKtdcHFxQat1JTl5EjNmzGLs2HFotVq6ujp7vNew9X3acs9jACJ/8M/W7R0dnfjp9QTOnD2o2hqPHaHmw620tbWxv6mRefMWMH58jM0xERGRpKUlcvZsIZ9+updlyzIJCwsf1Hkd1ZAH/pNPPsnTTz9NSkoKL7/8Mj/60Y9oaWkhKSmJLVu2DPXphBCDsH//Pj766ANSUyfx7W9/j5AQPUZjF2VlpRw7dpj8/PMsWbKUiIhIyspKiYoab/P+kI2bMfz9/92MRiPl5ZeZOjV90PV1/3ZwoqWZ9PRpjNOFUJe9G7/Zc3Hxse3njoiIZN68BRw+/DkbN26+5y5kZzAkgX/w4EHrn19//XXrnxMSEvjwww+H4hRCiCF25MjnfPLJNv75n58lNnaCzb6QED0ZGdPZty+bw9nZZI4Zy3UVtEyfhbf3133E2rAwmyt7gNLSYkJC9Pj5+Q+6RhcfH1TTZ1L9yUcsTUikcf8+aj68Oby7t/sC0dHj+eKL49y4UUF4eMSgz+9oZGoFIZxQW1sbH3/8Ed/85hM9wr6bRqNh+fIVzPbzx/v0KVJVarKzd9HR0dHn51ZW3uDkyRNkZEwbslovXiwlLm4CGo0Gv9lzCd6wsc/7AiqVivj4RMrKSofs/I5EAl8IJ3T06FG8vLxIS5tyx+PUajWxa9bxlZ8vqtRJhIdH8Le/bSU/P5euri7rcU1NjRw/fow9e3axaNESQkL0Q1Zre3s7Xl43f6tw8fEhcPkKXHx8MDc3U5e9G3Oz7YgWb29vmpubqKqqorLyBi0tjrH04VCQeQ6EcEInT55k6tQMXFxcbLbfOiqmu488Im4CVeOioPwyDz/8COPGRZGXd54TJ3Lw8vLCbLbQ1dVFfHwC69c/dMcHswbC1dUVo9HYY3v3DV2A0PEPA1BbW8Px48coKiqira0NtVpNY2MjwcHBJCenEhUV7dR9+xL4QjihlpYW/P0DbLaZm5up+M8/0l5YgKWzk+DVa4Gb3SQBAQG0tbUCN2+ORkRE0tHRQVtbGy4uary8vIdtnqzw8AhOnMhh6tQMm7Du7tbp/n9paTFHjx6htbWFjRs3M2nSzaf8TSYTly6VcfLkca5evcLcufOdNvSlS0cIJ2Aymbhxo4IrV8qpqqrE1dW1R1dH/cH9tBcW9Pr+trY23N1tx+G7u7sTGBiIn5//sE6KGBk5BqOxi8rKGzbbb+3euXr1KseOHWH+/AV4eXmTlJRsPU6j0RAXF8+6dQ9RW1vD6dPOO+2DBL4QDqytrZXjx4/x7rv/zbFjRzh//iyHD39ObW0t+/fvpampqcd7PBInErBoifV1e3s7JSXFpKSk3s/SrVQqFRkZ0zl4cH+f/fE5OTlkZEzn9OlTTJ48pccPIJPJRHt7G9Onz+Srr768441nRyZdOkI4qLq6Onbt+oTo6PGsXbveZpjkzJnp/PSnP+NPf3qNp576Pj4+PgQsWoLaza3HGPcTJ3Lw8HAnIWGiPZoBQFxcPK2trWzb9gFTp2YQFzcBV9eb0yhUVVVy4cIFKioMxMTEkZqaZn1fU1MjeXm5FBVdQKvVolaruHTpIu+88yYrV65yuqGbEvhCOKCOjg527fqEjIzpJCQk9tiflpbGvHkLOXXqJG+99V889dT30fy9i+RWxcVFbNv2AVu2PGH3tSzS0qag04WQm3uOkyePExAQCEBhYQF+fl7MnTufceOircdfLsyn+MMP8J0z1+Zm8uTJ6Rw69BkHD+4nLm4C06bNcJo+fQl8IRxQYWEBERERvYY93OwmWb/+ITo6Ojh27DB//vPvWbVqLeHhEahUKgyGKj79dC8nT+awfv3DpKcP3bj6wei+Ydza2kJjYyOKohAUFExwsK9N2FdVVVH8tw+JKr9McMY0m5FDPj4+hIaGsnTpcnbu/ASt1o3Jk+88PNVRSOAL4WAURaGgII8lS5bd8TgPD08ee+xxAgMD2bNnFxUV1zEajVgsZjo7O0lMnMj/+T//RnT0+Dt+jj14eXlbx+Y3NjbQ0mK74MqpUyeIXbGSYIOhx0NaLS0tuLt74OHhybJlK/jb3/6HpKRkh5hp824k8IVwMC0tLZjNJvR624efbh1jz9/nW3dzc2PNmvUYDFUsW7aCzs5ONBoXQkPDbaZQGMmioqLZseMMqanTcHNzo6GhnpqaGpYvX2nthrq17UVFhUycmASAr68vERFjKCq6YLeb0veTjNIRwsGYTKZe+9u7H1RqPHbEZrtarcbV1ZXw8JtdQLGxE0ZN2AN4enoRFXXzYTCAq1evEh0dbfM16G57+Z9+T7Ohiujor2fcjImJ5erV8vtetz1I4AvhYNzd3Wlvb8dkMtls72semo6ODiwWC66urvezzCE1Z84c8vNzKS4uwmjsQqt1s9nvN3surgmJmIouMN3Ty+YJY3d3916f5HVE0qUjxCinKAoVFdcpLMyntrb2768rOHz4M+bPX2QNN5deRuEAFBUVMn58DGr16L3+8/f3JytrNbt376StrRVfXz/S0zNQqdTU19dRUJDHFV8fpi9YSPSDa2ze29bWipubWx+f7Fgk8IUYxVpaWti7dzdGo5GkpBQmT56KSqVCrw/l4MF9XLlyhczMlej1ob2+v6uri7y8XBbd8qDVaBUYGMTDDz9CXt553n33LRob6wEV3t4+JCQksv4b37RZxatbcXExMTGx979gO5DAF2KUam9vY/v2j5g4MYm0tCk2Y8kXLFhEV1cX1dVV7Ny5naysNT1u4nZ2dpKdvYsxY8YQGhp2+8ePSq6urkyePBWDoYqwsHCbh7B6U1dXi8FQxdKly+9ThfY1en+HE8LJHT+eQ3T0eOtV/a1UKhWLFz9AWFg4zc3N/Pd/v0FDQz0dHR3U19dz9OhR/vrXdwkKCmbOHMebTCwjYwZffnmGy5cv9nlMU1Mje/bsYubM2aP6/sW9kCt8IUah9vZ2Ll++yObNj/V5jEajYfHipaSkTOI///MPvPvuW3h6eqHVujFp0kRWrVpHQEBAn+8fzQIDA8nMzGLv3l2UlpaQlJRCaOjNRc+bmhopKMinsLCAjOQUQi5fwhwR2WPJREckgS/EKHTpUhljxozFw8PjrseGhOhZvXodNTU1LFy4GACdzofq6ua7vHN00+v1bNz4CMXFF/j88wM0NzejUqnRaFyIj09g7dr1WE4cv+OSiY5GAl+IUai1tbXXNWN7W8AEwNfXjytXnGOs+a3c3d1JTU0jNTUNk8mEoljQaFytXVjm2+bUd3QS+EKMQhqNhvb29h7bb10F6tYrVpPJ1GN1K2fT28NofQ1VdVQS+EKMQiEheo4ePYyiKHdcBarb1atXhnSdWTE6ySgdIUah8PAIFMVCRcV1m+23rgLVraOjg4sXS0lMtN989mJkkMAXYhRSqVRMmZLB4cOf0d7e1udxZrOZgwc/JS4uHg8Pz/tY4ehlbm6mLns35mbHu6ktgS/EKDVhQjxxcfFs2/YhFy+WYbFYrPsURaGy8gY7d25HrVYza9YcO1Y6uvQ1yZwjkD58IUax9PRpBAUFc+7cVxw7dhi9Pgy1WkVdXR1ms4mkpBSSk1NH9Tw591tf90EcgQS+EKPQrcMvo6PHEx09nrq6WurqarFYFCZOTCYsLNzhnqC9Hxx55I4EvhCjUG/DLwMDgwgMDLJnWWKEk8AXYhRy5G4HMXwk8IUYhRy520EMH7mTI4QQTkICXwghnIR06QgxgrS2tlBYWMDFi2V0dLSj0WgID48kKSkZnS7E3uWJUU4CX4gRQFEUzp37ii+/PE1sbBzz5y/Ey8ubrq4uLl++SHb2bkJC9CxatMRpFusQQ08CX4gR4OzZrygqKuShhzbjc9tCHIGBgUyaNJnPPtvP3r27yczMcvqZL8XASB++EHbW1NTI2bNnyMpa3SPsu+d1oa2NRYsewGw2U1RUaKdKBYzuuXYk8IWws4KCfOLjE/D29u6x79Z5XdRqNVOmpJOfn4eiKHaoVMDX35Mb/+/1URf6EvhC2FlpaQkJCb1PXew3ey7BGzZaH7CKjBxjXYhc2Iff7Ll4pqTSlnt+1E2wNqjA37FjBytWrGDp0qW89957Pfa/9tprLFy4kNWrV7N69epejxHC2bW3t+Hj49tje2/LFapUKnx8fO44JbIYXi4+PoQ98aTND+LRYsA3bauqqvj1r3/NRx99hFarZdOmTUyfPp3Y2FjrMXl5efzHf/wHkydPHpJihXBELi4umEzGHqNv7rRcYW/L9Yn7Z7Q+6TzgK/ycnBxmzJiBv78/np6eLFu2jOzsbJtj8vLy+POf/8yDDz7IT3/6Uzo7OwddsBCOJjQ0jPLyyz22396dA9Dc3ExzcxOBgYH3sULhKAYc+AaDAZ1OZ30dEhJCVVWV9XVrayuJiYk8++yzbNu2jaamJv7whz8MrlohHFBSUgp5ebk9bsT2tlxhYWE+cXHxuLpq73eZwgEM+PdCi8ViM9f27Yspe3l58frrr1tfP/HEEzz33HM888wz/T5HUFDPUQu90el87n7QKOJI7XGktsDwtCcoKImiovMUF59n9uzZfc5hX15ezpUrpWzatImAgMHXId+bkWu42jLgwA8NDeX06dPW19XV1YSEfP3od0VFBTk5OWzYsAG4+QPhXvsda2tbsFjuPPxMp/Ohunp0DY26E0dqjyO1BYa3PXPmLGbnzk+4erWSyZOnEhQUbN3X3t5GQUE+ubnneeCBZZhMmkHXId+bkWswbVGrVXe8UB5w4M+aNYtXX32Vuro6PDw82LdvH//+7/9u3e/u7s6vfvUrpk+fTmRkJO+99x4PPPDAQE8nhEPz9PRizZr1nDv3Fbt2fYKnpxfe3jenVqipqWb8+FhWr15HQECAvUsVo9iAA1+v1/PMM8+wZcsWjEYjGzZsIDU1lSeffJKnn36alJQUfvrTn/LUU09hNBqZMmUK3/rWt4aydiEcilarJSNjOlOmpFNVVUlHRwcajYaQED3u7u72Lk84AJUygh/Zky6d0c2R2gKO1R5Hags4VnuGs0tHnrQVQggnIYEvhBBOQgJfCCGchAS+EEI4CQl8IYRwEhL4QgjhJCTwhRDCSUjgCyGEk5DAF0IIJyGBL4QQTkICXwghnIQEvhBCOAlZGFMIIe4Ti8XC1avlFBTk09jYgKIo+Pr6k5g4kaioaNTq4b0Gl8AXQoj7oL6+nr17d6HRuJKcnEJISCgqlYrqagPnzn3F8ePHWL58xbCu3CWBL4QQw6ypqZEdO7aRkTGdxMQkm30BAQFMmBBPaWkxO3ZsR6d7DBieNYulD18IIYbZkSOHSUlJ6xH2t4qNncC0adPZt2/fsNUhgS+EEMOosbEBg6GK5OSUux6bkDCRpqYmamqqh6UWCXwhhBhGpaUlxMVNwNXV9a7HqtVqkpOTKSkpHpZaJPCFEGIYtbW14efn12O7ubmZuuzdmJttlzP08/Ojvb1tWGqRwBdCiGHk4uKCyWTusb3x2BFqPtxK47EjNtvNZjNqtcuw1CKjdIQQYhiFhOjJz89l8uQpNtv9Zs+1+X+3S5cuodeHDkstcoUvhBDDKDp6PI2NDdTV1dpsd/HxIXD5Clx8vh5339TUxPXr14mNnTAstUjgCyHEMHJxcWHy5KkcOPApnZ2dfR5nNHZx4MA+pkyZ0q8bvAMhgS+EEMMsOTmViIgItm37kMuXL2KxWKz7FEXh6tUrbN/+EQEBgcyYMWPY6pA+fCGEGGYqlYqZM+eg0xVz5sxpjh49gk4XgkqloqbGgKurlpSUVOLjE1GpVMNWhwS+EAPQ1NRIU1MTAH5+/vj4DN/8J8IxqFQq4uLiiYuLp6ammoaGegDS0iZbw3+4SeALcQ8uX77I+fPnqKurIzAwEIDa2lpCQvSkpk5izJixdq5QjAbBwTqCg3X3/bwS+EL0g6IoHD58mLNn80hPn0Z0dAwazc1/PkajkbKyUg4d+oyJE5OYMiXdztUK0TsJfCH6ITf3HNevX2bt2odwd3e32efq6kpCQiJjxoxh+/aP8PT0IiEh0U6VCtE3GaUjxF2YTCa+/PIMWVlZPcL+Vl5e3ixevIzTp7+wGYUhxEghgS/EXVy8WIZOp7P22d+JXq/Hw8ODq1fL70NlQtwbCXwh7qKi4jrjxkX32N7X5FfR0eOpqKi4X+UJ0W8S+ELchdls6vXJx74mv9JoNJhMpvtVnhD9JjdthbgLd3cPWlqae2zva/KrlpbmO/b1C2EvcoUvxF3ExsZRVFSIoig223ub/MpkMlFSUjxsk18JMRgS+ELcRUiIHq1WS1FR0V2PvXChkICAQAICAu5DZULcGwl8Ie5CpVIxd+5CPvvsM65evdLncRcvlnH69BfMmTPvPlYnRP9JH74Q/aDX61m1ahXvv/8BISF6kpJS0OluPhpfVVVJfn4edXW1rFiRRWBgkJ2rFaJ3gwr8HTt28Mc//hGTycQ3v/lNHn30UZv9hYWFPP/887S2tpKens4LL7xgfRxdiNEmIiKCRx55jJKSYk6dOklTUyOgIiAggMTEJGJiYuXvtxjRBvy3s6qqil//+td89NFHaLVaNm3axPTp04mNjbUe8+yzz/Kzn/2MtLQ0nnvuObZu3cojjzwyJIULYQ+urlomTkxm4sRke5cixD0bcB9+Tk4OM2bMwN/fH09PT5YtW0Z2drZ1//Xr1+no6CAtLQ2AdevW2ewXQghxfw048A0Gg7UPEyAkJISqqqo+9+t0Opv9Qggh7q8Bd+lYLBabCfsVRbF5fbf9/REU5N2v43Q6x1p8wpHa40htAcdqjyO1BRyrPcPVlgEHfmhoKKdPn7a+rq6uJiQkxGZ/dXW19XVNTY3N/v6orW3BYlHueIxO50N1dc+nIEcrR2qPI7UFHKs9jtQWcKz2DKYtarXqjhfKA+7SmTVrFsePH6euro729nb27dvHvHlfjz+OiIjAzc2NM2fOALB9+3ab/UIIIe6vAQe+Xq/nmWeeYcuWLaxZs4asrCxSU1N58sknyc3NBeDll1/m5z//OcuXL6etrY0tW7YMWeFCCCHujUq5fYKQEUS6dEY3R2oLOFZ7HKkt4FjtGZFdOkIIIUYXCXwhhHASEvhCCOEkJPCFuE1fSxcKMdpJ4Atxm76WLhRitJOp/YS4TV9LFwox2kngC3Gb7qULhXA00qUjhBBOQgJfCCGchAS+EEI4CQl8IYRwEhL4QgjhJCTwhRDCSUjgCyGEk5DAF0IIJyGBL4QQTkICXwghnIRDBr7MdiiEED05ZODLbIdCCNGTQ06eJrMdCiFETw4Z+DLboRBC9OSQXTpCCCF6ksAXQggnIYEvhBBOQgJfCCGchAS+EEI4CQl8IYRwEhL4QgjhJCTwhRDCSUjgCyGEk5DAF0IIJyGBL4QQTkICXwghnIQEvhBCOAkJfCGEcBIS+EII4SQk8IUQwklI4AshhJMY8IpXFRUVPPvss9TW1hIdHc3LL7+Ml5eXzTHXr18nKyuLsWPHAhAcHMwbb7wxuIqFEEIMyICv8F944QUeeeQRsrOzSU5O5g9/+EOPY/Ly8njwwQfZvn0727dvl7AXQgg7GtAVvtFo5NSpU/z+978HYN26dXzjG9/g2WeftTkuNzeX4uJiVq9ejZ+fH88//zzx8fGDr3qYKIrCjRsV5OfnUl1toKurCx8fX5KSUoiJicXV1dXeJTotg6GKK1fK6erqws3NjXHjoggO1tm7LCFGlQEFfn19Pd7e3mg0N9+u0+moqqrqcZybmxurVq1i06ZNHDlyhO9973vs3r0brVY7uKqHQVtbK9nZuzEYqtBoNLS2tqLVajEYDJw7dxYvLy8eemgT48fH2LtUp3L9+jVOnMihvb2dmJhYPDw8aG9vZ8+enXh7+zBz5mxCQ8PsXaYQo4JKURTlTgfs2bOHn//85zbbxo0bx5UrVzh06BAAJpOJyZMnk5ube8eTrVq1ipdeeomEhIRBlj202tvb+ctf/kJLSwteXl5MnTqVxMRE3N3dURSFy5cvc/DgQY4dO8YzzzxDSkqKvUt2CiUlJezfv5/FixcTGxuLWv11D6TFYqGoqIjPP/+c5cuXEx0dbcdKhRgd7nqFn5mZSWZmps02o9HI9OnTMZvNuLi4UF1dTUhISI/3vvPOO2RlZREQEADc7DLp/q2gP2prW7BY7vjzCJ3Oh+rq5n5/Zm+OHDnE9etVREREkpm5EldXLc3NRpqbjQB4ewezatVGPD39+OUvf8XPfvZSjxvUQ2Uo2jNSDKYtjY0NbNu2k6ysVQQEhFBb29rjmODgSObMWcwHH3zMxo2b8PLyHmzJdyTfm5HLkdozmLao1SqCgvr+dzCgm7aurq6kp6eze/duAD7++GPmzZvX47hTp07x4YcfAvDFF19gsVgYP378QE45bLq6ujh//iwuLhqWL78Z9n1ZsmQZen0Y+/fvvY8VOqf8/DwSEhLR6XpeSJibm6nL3o25uZnQ0DBiYmIpKMi3Q5VCjC4DHqXzk5/8hK1bt7JixQpOnz7NP/3TPwHw/vvv89vf/haA559/npycHLKysvjlL3/JK6+8YvNr+UhQXn4Jo7GLtLTJ/bq3sGjREk6ePI7ZbL4P1Tkns9lMUdEFkpKSe93feOwINR9upfHYEQCSk1MoLMznLr2TQji9AY/Dj4iI4J133umxffPmzdY/6/V63nzzzYGe4r5oa2ujubmFuDjb0UPm5mYajx3Bb/ZcXHx8rNsjI8egUqmorjbIzcJh0tbWhkbjgq+vn8327u+J96TJAPjNngtAYGAQRqPROoJHCNG7AQe+o3Bx0dDZ2YG7u7vN9u6ryLaiC4Q98aQ19M1mM+7u7nR2dtqjXKfQ15V69/cEIHD5Cpt9KpUKi8Uy7LUJMZqNrP4VO9Dr9bS0tNDe3m6z3W/2XDxTUmnLPW/tOgC4cqUcd3ePETm01FF4eHhgNBppa7O9Ues3ey7BGzZar+y7NTU1oVKp5OpeiLtw+sDX6UIIDQ0jJ+eIzXYXHx/CnnjSJmCMxi7OnfsKX1/fXm8misExGo1cuFDAoUMHaWio5y9/eYfy8kvWK3cXHx8Cl6+w6WIDKCjIIz4+YcTdHxJipJF/IcDy5Zl8+mk2tbW1NttvDRiLxcLBgwdQFIVJkybf0/BScWeKonD+/Fneffe/uXixjMjIscyfv4ja2hqOHj1iDf7e1NfXU1hYwMSJvd/gFUJ8TVILSEubyuTJZ3nllV/wjW88zoQJ8dZAVxSFqqpKTp06yY0bFXh7+5CaOsnOFTuWU6dOcvnyJdauXY+/f4B1u1ar5fz5c0yalMbnnx9kzpz5xMTEWvdXVVWyb98eZs6cZfM+IUTvJPD/7rHHHufDD/+H999/Bx8fX5KTU9BoXKmvr6O+vhaNxhUPD0+yslbj6elFS0sLTU2NAPj5+Q37Qz+O6tq1qxQXF7F+/UN4eHja7EtJmYRW68bJkzm4uLjwwQfvk5W1BoCSkiJaWpqZOXMOsbFx9ihdiFFHAv/vXFxc2LhxMzNmzOTUqZPk5eWi1bqiVrsQHKwjOTmF+PhEqqoqyck5isFQZX2CuL6+jrCwcFJSJhEREWnnlowueXnnmTJlao+w7xYfn0BsbByXLl1kz56dHDnyOQkJE5k0aTJRUdHSby/EPZDAv4VKpWLcuGjGjYums2wMuk0AABUwSURBVLOTzs4ONBqNNYxOnz5JcXExU6dmsHTpcuvsmUZjF6WlJRw8uJ/k5FQmT55iz2aMGu3tbVRUXGfx4gfueJyLiwuxsXFs2vQoe/fuYeHCxfepQiEciwR+H9zc3GyG+eXl5XLx4kXWrdvQ42rU1VVLYmISY8eOY/v2j/D09CQ+fmRNEDcSNTc34+vr22M6i74eegsKCqa1tQVFUVCpVPe7XCFGPfl9uB9MJhNnznzBAw8s67PrAcDLy5vFi5dy6tRJeQioH9Rqda+T490+dUI3mTpBiMGRK/x+uHSpjICAQAIDg+56rF4firu7O1evljNunEzZeyc+Pr60tDTT3t5m84O0+7mH2x+wqqy8QUBAoFzdCzFAcoXfDzduVBAd3XPhk1tnbbzV+PExXL9+/X6VN2q5ubkRHR3DhQuFNtv7esAqLy9XxtsLMQhyhd8PRqOx1+UNu7sezGYzlWPHceFCAU1NjVRVVaJWuxAYGCRLI95FSkoqO3d+QlTUeOuop96Ul1+isrJCbtgKMQgS+P3g7u5Ba2tLj+1+s+fS2NjI3orrhLi6MmvWHIKDdZw4kUNTUyNlZaWcOnWC5ctXylQMfQgO1jFz5mx27NjG/PkLGTs2yqbLxmQyceFCIadPf0Fm5kqZw0iIQZDA74eYmFgOHvyUKVPSbcLI0NLMofY2HliRZR1/bzabKS+/RFbWagIDg7h4sYxdu3awatVaAgMD7dWEES0+PgF3d3dOnTrBsWNHiI6Owc3NjdbWVsrKStHpdKxataZf91CEEH2TwO8HvT4UV1ctZWWl1qc6FUXh8OHPmT9/oc3DVsXFF/D19bOG0/jxMbS2tpKTc4SsrNV2qX80GDcuirFjx2EwVHHt2lW6urrw9vZh7dr1+Pn527s8IRyCBH4/qFQq5s1bwO7dO/Hw8CAiIpKKiusoikJU1NcjccrLL3Hy5AmyslbZvD8xcSJnzpyioaFe5ny5ze1j7vX6UPT6UHuXJYRDksDvJ70+lKVLl/Ppp3sJCwunra2V8eNj6Orqoqqqkvz8XKqrDWRmriQ4WGfzXo1GQ2xsHBcvljFlSrqdWjAy3WlREyHE0JLAvwcREZE88shjFBcX8dFHWykpKSI39xz+/gFMnJjEkiVLUXd0Upe9++YYck9PLl++RHn5JUpKijCbLfj6+hIdHYOLi4u9mzMi9DXmXggx9CTw75FWqyU5OQWDoYqwsDASE5Ns9tcd2E/Nh1upra3huNmEr68fcXETaGlpobOzg4KCfI4dO8KMGbNl+gW+HnMvhBh+EvgDFB4eQVlZaY/A95s9l6qqSs52drBs5Wr0ej2KopCfn8fMmbMZM2YsNTXV7N27h66uLlJSUu3UAiGEs5EnbQcoJiYWg6GKxsYGm+1NJhMnFYUV6zei1+uBm0/qms0mIiPHADfHnj/44Bq+/PIUtbU19712IYRzksAfIFdXV6ZMmcq+fdl0dHRYt+fn5zJxYpJ1KGFLSwsHD+4nI2O6zRh+X19fkpJSyM/Pve+1CyGckwT+IKSmphEZOYZt2z6guLiIrq4uSkqKSExMwmjsIj8/l23bPiAlNo7A0tIec+4kJk6ktLQEk8lkpxYIIZyJ9OEPgkqlYubM2YSHh5Obe55Dhw5QVFREQMA+6ypYixY9gEfueWo+3Epb0QXCnnjSOimYl5c3Go2G9vZ2fG6bKMxR9DW3vRDi/pPAHwLdq2QZDAbef/9tpk2bYbPOrdnXj7aiC7Tlnqfx2BGbUSmKoqBWj77pfi0WC3l5uRw+/Bl1dbWoVCr0ej1z5swnLi7eOmGcjLMXYuSQwB9COp2OgIBA3N3dbRY1d/HxIeyJJ61Xut0aGupRlJuTs40m165d4U9/+j3t7e1MmzaD9PRpmM1miooKefvtN/Hz82fLlm+h0yXKOHshRhCVMoKXEaqtbel1RaRb6XQ+VFc33/GY++nUqRN0dHQyd+78Po/p7uYo0rqi8vRixoxZ1n0jrT23u3r1Ki+//H9ZsGAxDz64Bo3G9pqhrq6Ov/3tr1y7do3nnvv/cHPzs1OlQ2+kf2/uhSO1BRyrPYNpi1qtIijIu+/9Ay1K9C4xMZnS0hIqK2/0eUx3N0fzsWOjakEPi8XCf/3XH5k3byFr127oEfYAgYGBbNnyD+j1obz99tuyLKEQI4gE/hDz9vZm8eIHyM7eTUlJUY+1bS0WC7XhEVyOiiZu/QZ8fX3tVGn/KYpCZeUNDhzYR1VVJfPmLbzj8W5ubqxYkUVFRQXXr1+7T1UKIe5G+vCHwdix41i+fAUnTuRw4kQOMTGxuLt70NHRQVlZKd7e3kx/4n8RHh7R52d0D/Gsrq7GYjHj5eXNhAkJd1wVaqhZLBby83PJyzuPSqXmzJkvCAwMYufO7YSE6ElLm2wzNfStxo6NIigoiDNnTlkfOBNC2JcE/jAJDQ1jzZr11NbWUF5+mc7OTtzc3Fi+fMUdV7+yWCycPHmcgoI8wsMjiYyMxMXFhYaGBj755CMCA4OYP38hvr7D2zduMpnYu3cPZrOJBQsWExoaxuXLF5k+fRZz5syjrKyEAwf2MXVqBnGBwRi2vk/Ixs1ow8IAUKvVREREUFtbO6x1CiH6TwJ/mAUFBRMUFNyvYxVFYdeuXVRXN/DQQ5vx9v765ktHRwfu7u6cPfslr7zyCxYvXsqUKRk2xwylI0cOodFoyMxciVp9s+dPrVZjMplwdXUlIWEi4eERZP/lHVwLCqCtDQMQ+YN/tn6GxWKRWUGFGEEk8EeQ3NxztLW1kZmZZb0hajKZyMk5SmlpMWPHjiMjYzrBwcEcPXqY0tISIiLGMG/eAjw8Bj6002KxUF5+mYKCPGpqqmlvb6eo6AJbtjyO0WjEzc0NgPDwSAoL81myZCkqlQpfXz/S6hugrQ21jw8hGzdbP1NRFEpKSpgxQ4ZjCjFSyE3bEUJRFHJzzzNv3jxU7e3UZe+ms6Ge3bt30NHRwebN32DJkmUkJaWwcuVqxo+PZdGiB/Dx8WH79r/R3t52T+fqHj3T1NTI1q3vc/bsl8TFxbNhw8MkJiaxcOFiamtr+ctf3ubSpYsAzJo1hxs3KqiouG79rDFbHqc+IACf7zxl7c4BKC4uoqGhgenTZyGEGBnkCn+EuH79Gm5uWkJDQyl+dys1H26lvPwyblFRNxdWUX/9s1mlUpGUlEJpaTFLlixDrVbz2WcHWLHiwT4/v6uri6KiCxQW5lNfX4eiKGi1WiorK3nggaVkZMywHtvQUM/UqRlERo7BYKhiz56dqNVqxgQFM0Ptwsfvv8vjT/1vvLy88YiIpG1FFrUWC/q/v7+trZV3332L+fPnD+o3DyHE0JIr/BGisbEBnS4ElUqF3+y5BKxdT54Kpqek0bAvu8fEazqdjsbGRgDS06dhMBh6TNXcraqqir/+9V1u3Khg9uy5PPnkU3z72/9IUFAQOp2OvLxcSktLrMebzWZrl1JIiJ6lSzM5dOggjceOklhVhd5QxR/+8CoXLhRgMpnQaDSYzWYURaGgII8XX3wBnU7Ho48+OkxfLSHEQMgV/gjk4uNDbUwswWoVlvNn+5yLpnu6ZY1GQ3x8AoWFBTZP7QLU1dWyZ89OFixYSFTUeOv29vZ2amtr+cY3Hqe1tYXdu3fg6qph3LhoPDw8aGpqIjT0ZhdNWFg4vr5+NIwZQ/CGjSxISqZxXzZvvfUGnp5eNDU1EhwczF//+h5tbW3Mm7eA1avXyQ1bIUaYQQf+b37zG1xcXPj+97/fY19XVxfPP/88eXl5uLu78/LLLxMTEzPYUzokf/8A8vPzbPrWdboQ/OITga/noumelsEQEIC/v7/1/TpdCGVlpT0+9/jxY6SnT7MJe4Dy8ktERo7Fw8MDDw8PlixZxoEDn/Loo+OIi5tAfn4eEybEW49PSJhIWfllYv/+Q+cf/uHb1NXVkpNzjP37sxk3LoqoqPFMnZqBVqsd2i+OEGJIDLhLp7m5meeee44333yzz2PeeecdPDw82LNnD8899xw//OEPB3o6hxceHoHZbKKiosJme/ear91TC3dPy1Dz2YEeyyverrGxAYPBQEJCYo997e3tNkM6w8Mj8PLy4vJXp3HbuZPO227O+vj49LgxHBgYhK+vL+vXP8zmzY8xc+ZsCXshRrABB/6BAweIioriW9/6Vp/HfP7556xatQqAjIwM6urqegSauEmlUpGSksbnn3+O0WjEz88fg6Gqx3F+s+dinD2HpnFR1i4XAIOhyuaKH+DSpYvExMT2OueNRqPpsfBKfHwCzds/pj3vPFMaGtm3L9s6J5DJZLROeQw3R/qcOnWSioprZGRMH1TbhRD3x4C7dNasWQPAq6++2ucxBoMBnU5nfa3T6aisrCQ8PHygp3VoSUnJdHU1s2vXJ8ydO5+cnKM0NTVan6o1Grs4V1RIoUbD6pWrrH34RqORoqILrFu3webzOjs78fLy6nEec3Mz/sXF5DXWY5kzzzoCyNPTi9LkZBKDggnZuJmFXZ3s2bOLsLAwmpqaCAsLo76+nmvXrpKfn4tWq2XVqrW4u7sP81dGCDEU7hr4e/bs4ec//7nNtvHjx/PWW2/d9cMVRbFZx/XmYh/9/6XiTtN83kqnc5yVlJYuXcoXX3zBZ59lYzS28fHH/8P8+fNpbGzkwoULREZG8uSTj9uskHXo0CHi48cTG2s7Z41O50dHR0ePr8+1Iwdo37OL8ZNSaGoyEBcXB4DBoMF3XASTv/cdACKA1NR4zp8/z5///GcgjYaGanQ6HWvWrCQyMtLm+9sbR/regGO1x5HaAo7VnuFqy10DPzMzk8zMzAF9uF6vx2AwMHbsWABqamoICel7Hpnbjcb58AdLp/Nh/PiJjB07gdLSYvbs2cm2bTvIyJhOZuZafHx86OiAjo5mOjs7OXXqJNeuXWX16nU9vg5eXkEcP76XxMTJNsGsSZtGcGsnblFRfPLJblatWou/fwBnzpwnKira5nOMRiPnzxeSlbWWWbNsn5qtqWm5a1sc7XvjKO1xpLaAY7VnOOfDH9ZhmfPnz2f79u2kp6dz+vRp3NzcpDunnzQaDQkJE4mNncDJk8cpKiqktbWVsLAwQEVtbQ0XL5Yyduw41qxZ32u3il6vx83NjfLySzajdLpvBAcC04Dt2z8iPj6Ra9eusWTJMuDmdAtXr5Zz+vQp/Pz8mDFj9v1puBBi2Ax54L///vsYDAZ+8IMf8Nhjj/HjH/+YlStXotVqeemll4b6dA5Po9Ewe/Zc0tOnUVJSbH1K1s/Pn02bHsXTs2cf/a2mT5/JwYOf4uvrT2BgYI/9CQkTcXd35623/h9arSu7d+/AxcWF+vp6PD09SU5OZcKE+Lt23QghRj5Z4nCEGY72FBcXkZNzlNTUSSQmTsTDwxO4eRO4pKSYr776kri4CUycmExDQz0WiwVvb59ef0DcC/nejFyO1BZwrPaM2i4dMTJMmBBPUFAQubnnef/9d/H29kGlUtHc3ER4eAQLFiyyLmQyXNMtCyHsTwLfSQQFBbNgwSJmzZpDU1MjiqLg7e1tvdoXQjg+CXwno9VqCQ7W3f1AIYTDkdkyhRDCSUjgCyGEk5DAF0IIJyGBL4QQTkICXwghnIQEvhBCOIkRPSxTre7f4/z9PW60cKT2OFJbwLHa40htAcdqz0Dbcrf3jeipFYQQQgwd6dIRQggnIYEvhBBOQgJfCCGchAS+EEI4CQl8IYRwEhL4QgjhJCTwhRDCSUjgCyGEk5DAF0IIJ+EQgX/69GnWrVvHgw8+yHe/+10aGxvtXdKAnTlzhg0bNrB69Wq++c1vcv36dXuXNCR+85vf8Oqrr9q7jAHZsWMHK1asYOnSpbz33nv2LmfQWlpayMrK4tq1a/YuZdBee+01Vq5cycqVK3nppZfsXc6g/fa3v2XFihWsXLmSN998c+hPoDiAJUuWKCUlJYqiKMqvfvUr5ZVXXrFzRQO3cOFCpbCwUFEURfnggw+U7373u3auaHCampqUH/7wh0pqaqryu9/9zt7l3LPKykpl4cKFSn19vdLa2qo8+OCD1r9ro9HZs2eVrKwsJSkpSbl69aq9yxmUY8eOKQ8//LDS2dmpdHV1KVu2bFH27dtn77IG7OTJk8qmTZsUo9GotLe3KwsXLlTKysqG9BwOcYW/e/duYmNjMRqNVFVV4evra++SBqSrq4sf/OAHJCQkABAfH8+NGzfsXNXgHDhwgKioKL71rW/Zu5QBycnJYcaMGfj7++Pp6cmyZcvIzs62d1kDtnXrVn7yk58QEhJi71IGTafT8W//9m9otVpcXV2JiYmhoqLC3mUN2LRp03j77bfRaDTU1tZiNpvx9PQc0nM4ROC7urpSVFTE/PnzOXnyJCtXrrR3SQOi1WpZvXo1ABaLhddee40lS5bYuarBWbNmDd/+9rdxcXGxdykDYjAY0Om+XvQ9JCSEqqoqO1Y0OC+++CLp6en2LmNIxMXFkZaWBsDly5fZs2cP8+fPt3NVg+Pq6srvfvc7Vq5cycyZM9Hr9UP6+aMq8Pfs2cO8efNs/nv88ceBm1fDOTk5/OM//iPPPPOMfQvthzu1pauri3/5l3/BZDLxne98x76F9tOd2jOaWSwWVKqvp5xVFMXmtbC/kpISnnjiCf71X/+VqKgoe5czaE8//TTHjx/nxo0bbN26dUg/e0TPh3+7zMxMMjMzbbZ1dnayf/9+65XwqlWr+OUvf2mP8u5Jb20BaG1t5amnnsLf358//vGPuLq62qG6e9dXe0a70NBQTp8+bX1dXV3tEN0hjuLMmTM8/fTTPPfcc6P2N/tuZWVldHV1kZiYiIeHB0uXLqWoqGhIzzGqrvB7o9FoeOGFF8jLywNuXmlOmTLFzlUN3LPPPsu4ceP4zW9+g1artXc5Tm/WrFkcP36curo62tvb2bdvH/PmzbN3WQK4ceMG3/ve93j55ZdHfdgDXLt2jR/96Ed0dXXR1dXFgQMHmDp16pCeY1Rd4ffGxcWFX//61/z4xz/GbDaj1+t58cUX7V3WgBQUFHDgwAFiY2NZu3YtcLPP+PXXX7dzZc5Lr9fzzDPPsGXLFoxGIxs2bCA1NdXeZQngjTfeoLOzk1/84hfWbZs2bWLz5s12rGrg5s+fz/nz51mzZg0uLi4sXbp0yH+QyYpXQgjhJEZ9l44QQoj+kcAXQggnIYEvhBBOQgJfCCGchAS+EEI4CQl8IYRwEhL4QgjhJCTwhRDCSfz/yxpZiTg0bYcAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(x_augmented.numpy(), y_augmented.numpy(), 'r.', ms=2);\n", "plt.scatter(x_sparse.numpy(), y_sparse.numpy(), s=150, facecolors='none', edgecolors='k', alpha=0.5);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Train with this augmented data and compare the TRAIN and TEST performance:" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEPCAYAAACzwehFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9e3xU9Z3//zpnZhJyIwkhNxJChFxACQZEuQiIioAiYBWViILrl227Xbq/pa1otzwWt/6oFVvtVvuV/uq6VEVdVFZENMUbQkiwIpAEhISQhEDuISHkPjPnfH5/nDkn55w5c0smmVzez8ejxZk5l89nknxe5/O+cowxBoIgCIJwAx/oARAEQRBDHxILgiAIwiMkFgRBEIRHSCwIgiAIj5BYEARBEB4hsSAIgiA8QmJBEARBeMQc6AG4o7m5Gdu3b0doaChuu+02LFmyJNBDIgiCGJUM6Z3Fm2++iQ0bNuDZZ5/Fnj17Aj0cgiCIUcuQFoumpiYkJCQEehgEQRCjniEtFgkJCWhsbAz0MAiCIEY93FCuDdXQ0IAdO3bAYrHgrrvuwh133BHoIREEQYxKAiIW7e3tWLt2LXbu3Ink5GQAwP79+/Hqq6/Cbrdjw4YNWLdu3WAPiyAIgnDBoJuhCgsLkZOTg8rKSuW9+vp6vPTSS3j77bfx4Ycf4n/+539QVlY22EMjCIIgXDDoobN79uzBtm3bsGXLFuW9/Px8zJ07F1FRUQCAZcuWITc3F5s2berXva5caYco+r5xio2NQGNjW7/uPdygOY8OaM6jg77Mmec5xMSEu/x80MVi+/btTu81NDQgNjZWeR0XF4eioqLBHBZBEAThhiERDSWKIjiOU14zxjSvCYIgiMAyJDK4ExIScPz4ceV1Y2Mj4uLiAjgigiCGM4wxtLQ0wmrtRkMDB1EUAz2kQaWhgXczZw5BQWMQHR3r00P5kBCL+fPn4+WXX0ZzczNCQkJw8OBBPPvss4EeFkEQw5T29lZwHIf4+GRYLGbY7aNLLMxm3uWcGRNx9WoT2ttbERER5f01/TW4/hAfH4/Nmzdj/fr1sNlsWLNmDWbMmBHoYREEMUzp6mrHuHHx4LghYWkfUnAcj4iIaDQ31w8Psfjyyy81r1euXImVK1cGaDS9lFW34lBRLZJjQpGWFBno4RAE0QdEUYDJNCSehYckJpMZoij4dA59myrKqlvxwjsnIQgiTCYeT+bMJMEgiGEKBcm4pi/fDYmFipKqFtgFEYwBEESUVLWQWBAE0W9+//vnUVxcCLvdhsuXLyE1dTIA4MEH1+KVV/6A+HipYKog2GGz2fCTn/w/WLRosXL++++/i1de+QM++OBjxMSMV95fs2YlXn75z0hMnIAFC2bjqae2YuXK+5TPN236IZ544oeYNWt2v+dAYqEiMyUaZhOv7CwyU6IDPSSCIEYAP//5UwCA2toa/PSnP8KuXW8DAD75ZD8WLFiEX/3qGeXYw4cP4YUXfqMRi08+2Y+FCxfjwIGPsH79Ey7v8+c/v4JbbpmLpKQJfp8DeX9UpCVF4smcmXj07mlkgiKIUUpZdSsOFFSirLo1IPevq6vF2LFje8dTdh7Xrl3DunUbsH//h27DgB98MAfPP///Dsi4aGehIy0pEvOyk0ddeQCCIHr9lnZBhHmQ/JZ5eYfx+OOPoKOjAz093bj55jl47rnfK58fOPAR7rhjCaZOnQaTyYRvvinAvHm3Gl5r3boNOHz4EPbt+1+sWLHar+OknQVBEIQDtd9ScPgtB5oFCxZh1663sWvXbiQnT8TEiSlISZkEALDb7fjss0+xZMkyAMDtty/Bvn0fuLyW2WzGr361Da+++jLq6+v8Ok7aWRAEQTgIpN8yLCwcW7f+B9avfxizZ9+C6dNn4OjRw2hra8O//duTACTxaGlpRkNDPeLi4g2vM3lyGh5++BG/m6NoZ0EQBOFA9lv+YNHkgPgtJ0xIwgMPPIz//M/fgTGGAwf24x//8Z/w/vv78f77+/Hhh58iK+tG7N//odvrPPbY42htbcWZM8V+GxuJBUEQhIq0pEismJcasACXxx77B9TW1iA39wBOnjyOe++9T/P52rWP4uOP90EQXCfVyeYofzKk26r2F+pn4T0059HBaJlzXd1FJCRIdn93dZJGKt7MWf0dAZ77WdDOQkdZdSve+6I0YGFzBEEQQxFycKsoq27Fnj1/w3V8LfLFRDz00DLKtSAIggCJhYaas0X4UejfYIIAAUU4czYRaUkLAz0sgiCIgENioSLNUg8TBJg4AExEmqU+0EMiCIIYEpDPQkX81GzwJgsYOPA8j3F8B4T6skAPiyAIIuCQWKgwxachbOVTGDvzLnAcB9u5Q+j8eAcJBkEQox4SCx2m+DSYI2MBJgCMAaId9ppzgR4WQRBEQCGfhQEhk25AM2cGY3ZwnAnmCVMDPSSCIIYx3vazkHnyyX/DDTdMx1dffY4339wFQRDAmIjly1fgkUfW45tvCvDqqy8DAKqrL2HcuBiEhIQiMXECnnvudwMyBxILHUJ9Gaoqz+B/O2YjBN2oEBPxkD0WaYEeGEEQg4JQXwZ7zTmYJ0yFKd4/f/m+9LOQaWxswCuv/AGvv/4WIiOj0NnZiU2bfoiUlElYsOA2zJkzD4B/Gxy5g8RChVBfhs6Pd8Ak2LB6DI8/tS1FlTCeOuYRxChBXgMg2mDlLQi9d4vfBMNXrl69Crvdju7ubkRGAqGhodi69RkEBQUHZDwkFirsNecA0QYODCaISLfUoRrx1DGPIEYJ8hqg9lcOtFjI/SxkLBYL/vKXvyI9PQMLF96Ghx5ajYyMTMycORt33bUcyckTB3Q8riCxUGGeMBVW3gKIdvC8CUk3zMKT02bQroIgRgnqNQC8eVD8la7MUADwi1/8Ehs2/B/8/e/H8Pe/F+BHP/oHbNv2LG677Y4BH5ceEgsVpvg0hN67BcGtFeiJvA6LArT9JAgiMMhrgL99Fn0hPz8PXV2duPPOpVixYhVWrFiFjz76X3z88T4Si6GAKT4N0dNnjorKnARBOGOKTwuoSMiMGTMGL730Aq6/fjoSEyeAMYbz50uRnp4ZkPGQWLhAjoiosyTjdFsUMlOiyRxFEITf0fssAODhhx/B3Xffiyee+Eds2fKvsNvtAIA5c+bh8cc3BmKY1M/CiIieGtS89QyYYION8fi/bUtxGfEB6Zw1WIyWPgdqaM4jF+pnQf0sBoWui2c0UVFTzHWD1rydIAhiKEJiYUDIpBvAODMExkEAjzJ7AnieoxBagiBGLeSzcEFt5I2oqL2Gv1un4KI9FrdlJ45YExRBjEQYY+A4LtDDGJL0xftAYqFDqC9D7YEdSLTbMD6Yx7fWKTCbeczPSgz00AiC8BKeN0EQ7DCbLYEeypBEEKRcMl8gM5QOe805iPZef0WapR45S9JxnbkRPSc/pnLlBDEMCAkJR1vbVTA2uhzb3sCYiLa2FoSEuHZmGzHkdxbff/89duzYgV27dg3K/cwTpqKbN0MU7BDAo0MMQuL5/0Xn8VMAEwJeL4YgCM+Eh0eipaUR9fWXwfMcRHF0iQbP827mzCEoaAzCw30zqw9psbh06RIOHToEk8m37VJ/MMWnQVjyc3z20UHEci14MPQYOHUQ1CDViyEIou9wHIdx4+IAjJ5wYTUDMechJRavvfYa8vLylNevv/46fvKTn+BHP/rRoI4j8+ab0VVbjojiPQAAjYtskOrFEARBDCWGlFhs3LgRGzcGJjtRT3BtEcAAjpMKUIIDgqbdDkvGrbSrIAhi1EEObhdUBkmCIEeYXQ2dREJBEMSoZVDEor29Hffeey8uX76svLd//37cc889WLp0KXbv3u32/D//+c8DPUQnombehfe75+GiEAMBPKI6q9C5/7foPvJXiogiCGLUMeC1oQoLC7F161ZUVFQgNzcXycnJqK+vR05ODvbu3YugoCCsXbsWL774ItLShtZT+7nKZjQefh8TLh0Eh96viTMHIXHdMxiTHJjqjwRBEIPNgPss9uzZg23btmHLli3Ke/n5+Zg7dy6ioqIAAMuWLUNubi42bdrk13v3tZCgHEkQE2ZBRGoiei5pr8EEO5q+P4Hg4An+GmrAoYiR0QHNeXTQlzl7KiQ44GKxfft2p/caGhoQGxurvI6Li0NRUdFAD8VrzlU241hRNTJTojGxux0O/7byL0VEEQQx2ghINJQoipqaLUOphktZdSt+9+5J2OwizCYeG+fEYDIzwQQRDIAYNRERM+4gRzdBEKOKgERDJSQkoLGxUXnd2NiIuLi4QAzFiZKqFtjtIhgDBEFEbrkZf2pbioKedDBwMLdeQk/+2+TkJghiVBEQsZg/fz4KCgrQ3NyMrq4uHDx4EIsWLQrEUJzITImG2cyD5wCTiUdUeDAq7bFoEcPAQwQPpmRxEwRBjBYCYoaKj4/H5s2bsX79ethsNqxZswYzZswIxFCcSEuKxPYf36r4LACgqPwKLtgTIMAEHgwc+SwIghhlUFtVA/SRBGXVrSipasH0iKtIsF2GecLUEeezoIiR0QHNeXQwLKOhRgJpSZGqxkfZAR0LQRBEIKByHz5SVt2KAwWVKKtuDfRQCIIgBg3aWfhAWXUr/ud/cjHLfAH133UhcmIUIsaNp5pRBEGMeEgsfODc8eP4SVguTHLpj/pLsNYDtpIjCF35NAkGQRAjFjJD+cDE9mKYwMBxUulyDo7MbgqlJQhihENi4QMJMaHKfzOm+h9nolBagiBGNCQWPjB+5h2AyazUn2WO/3VddxuZoAiCGNGQWPiAKT4NYSufhjlpOsABvMMcFVbxNZX/IAhiREMObi+RE/MyU2Jx3ez7INScBWOC5LNgouKzsNecG5FJewRBjG5ILLygrLoVL7xzEnZBqkT7ZM5MhEx/AGFF74EDgx082rp5jP14ByDaYOUtCL13CwkGQRAjBhILLyipaoFdkCrR2u0i8otrERN5PU60LcfsoAsYy3Uh9fxRQLABqkKDJBYEQYwUyGfhBZkp0eB5qd8GA5BXXIuwEAtMJg5zgsuQFXQJEd3VUNojUaFBgiBGGCQWXpCWFImFWYnKa1Fk6Oiy4fFZPMycqORcAIAp6QYEz38E9ppz5PQmCGLEQGLhJfOzEmEycZIocEB5zTWcuBoNp68wJAI9+W/DevwDdH68gwSDIIgRAYmFD8h9uEUROHm+CXvPcijoSYO6CLpQdkzyXTBqkkQQxMiBxMJLSqpaIBj0xvimZwpEqPuHy8eQ74IgiJEDiYWXZKZEw2ziNbIAAJX2WJyxJkMrIwzgeFimLyHfBUEQIwIKnfWStKRIPJkzEyVVLQgLseBIYQ0qaqVOVF92T8cNQTUwQeg9gQmwFf0NgEh5FwRBDHtoZ+EDaUmRWDEvFYuzk5CzJAMWMw+eAy4jHq3zfwrTpFnaE5hAvguCIEYEtLPoI/qdxuk2G2aOmYAInAR0RilwPPkuCIIY1pBY9AO5L7dcCuSkRcA/R5hgYnaNb4MbGxeYARIEQfgJMkP1E3UpkHJrLN5vnw2RafcW7GoNOvf/lhzdBEEMW0gs+om6FAgAhHE9AOAUNQXRDmvp0cEbGEEQhB8hsegnlxvbwVT5F2X2BAgwQYST5wL2c4dpd0EQxLCEfBb9oKy6FW8dLIU6V6/SHos/tS1FmrkOXQjGD2IrYemokz5kAqylRxFCIbQEQQwzSCz6QUlVC0SDrO5Keywq7bEAgKldbchCnWKWEi4Vw3r2EFh3OzVJIghi2EBi0Q8yU6JhMfOw2UWXx3xxNQXXR5yByVGZlrU3oefILgAcrCZK1iMIYnhAPot+IOdaxEWP0bwfZOn9WivtsbhsH+/swAAl6xEEMXzwSSysVitqamoGaizDkrSkSCyfM0nznl230zhmlXYOTnrBcZSsRxDEsMCjWHz22Wd49tln0d7ejuXLl2P16tX461//OhhjGzYszk7C+uWZuOG6aMxMH68RBQ5AQU8G9nTOhcAHK58xAFxEPJmgCIIYFngUiz//+c946KGHcPDgQWRnZ+Orr77Cvn37Bnxg3377LbZs2YInn3wS77333oDfr78szk7Czx+eiawpMeA5DhwHWMw8ls9JgYnnUNCTgQ/aZwJMKhcFBoitNeg8+EcKpyUIYsjjUSwYY8jMzER+fj4WLVqE8PBwMOYcAeRvrl27hl//+td4/vnn8cUXXwz4/fxBWXUr3vn8PASRgeM45CxJR+gYM0TGwAAUdGegyDYRAMA5wqOEyhPo3Lcd1rOHAjZugiAIT3iMhuJ5Hp988gny8vLw1FNP4euvvwbHOeUn95vXXnsNeXl5yuvXX38djDH87ne/w/r16/1+v4FALv0BAGBSn265D4YgiOB4Dl9bp+MGy2WYGEPv18gcEVJA0LTFARg5QRCEezyKxVNPPYVXXnkFP/vZzxAbG4tXX30VW7du9ftANm7ciI0bNyqvr127hueeew6PPPIIsrKy/H6/gUAtDCYTj8yUaCViKr+4FgCQkpCBC5dNSK/5GIC2LEjPkV0QrzVgzJyHAjB6giAI13DMB5uS1WpFU1MTJkyYMJBjAgBs2bIFdXV1iIuLQ2JiIn7+85/7fI0rV9oNk+Y8ERsbgcbGNp/PAyRTVElViyIU8nsvvHMSNrsInuew7OaJaCv6Ag+MKQAHQL9Rs9x4z6ALRn/mPFyhOY8OaM7ewfMcYmLCXX7ucWfx2Wef4dixY9i8eTNWrVqFtrY2bNq0CRs2bPBqAO3t7Vi7di127tyJ5ORkAMD+/fvx6quvwm63Y8OGDVi3bp3TeTt27PDq+kONtKRIRSRk8otrlcQ9UWTI/aYKDOkQRIYHQ4+Bh3aHYSv8BEJ9GcbMeYiipQiCGBJ43FmsWbMG27dvx5kzZ5CXl4df//rXWL9+Pfbu3evx4oWFhdi6dSsqKiqQm5uL5ORk1NfXIycnB3v37kVQUBDWrl2LF198EWlpI3NRPFfZjF/+3zzYBeOvOS24Cf806TzMTecNPuUQMesuRGQtxpjkzIEdKEEQhBs87izkaKi//OUvPkdD7dmzB9u2bcOWLVuU9/Lz8zF37lxERUUBAJYtW4bc3Fxs2rSpj1NwTSDMUHqOFVVDcDOGiJSpiL7/IdR9/gZCy78EoN5lMLSdOIi2wkMDXhaEtuqjA5rz6GAgzFAeQ2flaKgjR47g1ltv9Skaavv27Zg9e7bmvYaGBsTGxiqv4+LiUF9f79X1hiOy05vnAJOJQ2JMqObzq+09eO+rMmz9Lhmfd93Qm4ehRrCi59QngzdogiAIHV5HQ/385z/3SzSUKIoasWGMDUgo7lBB3as7MyUaAPD82ycgOMxSFbVtqKiVngA+7roJAHDnmDMAtH4M4eIJdH+zhyKlCIIICB7FYvbs2di1axeqq6tx8eJFvPvuu/26YUJCAo4fP668bmxsRFzcyO5RrXd6P/XILOzLK8f3FS1O9aI+7roJLSwCD4Z+Awbm5PgGAC4olMqbEwQxqHg0Q1VWVmLFihW47777cP/992PJkiW4cOFCn284f/58FBQUoLm5GV1dXTh48CAWLVrU5+sNR9KSIrF6wWSYzbxGDDgOmJk+HlMWr8Sp6GVSL2+dWcpW+Ams376Pzo93UJkQgiAGDY87i2effRYbN27ED37wAwDABx98gP/4j//AG2+80acbxsfHY/PmzVi/fj1sNhvWrFmDGTNm9OlawxF1HoZsngoLsSjZ3gAcORlxSDXfjXvHfIcplgYAur7eghU9xz+EefJsaqREEMSA4zF09r777sOHH36oeW/lypXYv3//gA7MHwQ6GkqfoCcn59kFEWYTjydzZjrlZBwoqMTew+Wa3cT/ifgKWZZLcOvZMQX1K2KKIkZGBzTn0UFAoqEEQcDVq1eV183NzT4NYLQiC8Pew+V44Z2TinDYBRGMAYIgoqSqxek8ffTU4uwJSL7tAXCeflSCjRopEQQxYHg0Qz366KN4+OGHcffdd4PjOHzyySdeZ2+PZoyEwah2lH73oY+eutzYjvfPdGHRtH/EDZ3fQqg/D3QbPDFwPDVSIghiwPAoFg8//DBSUlKQl5cHURSxbds2zJ8/fzDGNqzRC0NYiAUlVS3IWZLu5J9wZZb68rvLOPa9lINypgJYv/wBzA39HLazXzndzzTlFlhLj4IrPQp+/CTyYxAE4Vc8igUAzJs3D/PmzVNe//73v+9TYb/RhHqHEBZiwTufn3cShQMFlYZmKbnooJ7vShpw3Q03Igpfg4OozcMoK9AdzcFqsgx45jdBEKMDn3pwy+zevdvf4xiRpCVFYsW8VHR02QxFQeufkMxSmp4YOlLiIvCb3Kv447VlKOjJgDUyxc3dGSDayY9BEIRf8GpnoWcwOuWNJIx8FYBzdrdsgjKbeNjtopKwx3HA8ltSEDrGDLsgooLFotIeC9sYhkXXdgNMu8tQzmMMQmM5hPoy2l0QBNEv+iQWI7k8x0DgShTkz/SvjfIv0pIicehUNThIYsAA7CvhUB90Cx4IkcqcKx86YByT2rZWFSF05dMkGARB9BmXYnHw4EHD9xljEEVjMwnhGqM+F94eW1bdijdyzyGvuBbqtBFBBI52Z6DaHo2bgy/g+jGNiGYtTs2UINopgY8giH7hUizefPNNlycNlzanIwF1lz1XXLTHohrxGJ8djIjv/wIzk45Vi4ZQfRpC9WkAgNUUhOD5j5BwEAThNX0SC2Lg0OddGDm8OU5XxpwDcpak4+bsJFyMDoFQnIvxbSWAU5lCB4IVPXlvAhBh5aWIKcTOHKgpEQQxAuiTz4IYGIzKgaid4xzPYWFWIlISIvDWwVJNKZOOLhsAYNL0bGB6NoT6MnQffRtiU7nxzZgg/Stnfk8nsSAIwjUkFkOEsupW7MsrV6Kg5BDbFfNSXTrHdx8shcgYzIbZ4GkwxU5yLRYKDMzaOaBzIwhi+ENiMQTQ+yU4Vd4FYOwcT44Nx8IZiQCA+VnSv3JTJZOJw1OPzMJ1GbfCdu6wsotQQmp197cVforLjaXgpixA0LTFAzJHgiCGNy7FoqamBhMmTDD87PDhw6OuB8VAod5RANJCfn1qNFYvmOwyekptruJ5aelv7bAq3fcEgeGdz0uRsyQD1636JaylR3GtowcF56/h9iDJya2NmGKw1pQBNVJ/DBIMgiD0uMzg/ud//mflv3/6059qPnvppZcGbkSjCHnRP+PomMdxgNnMY/WCyQCkcuVl1a1O5+UX18JmlzPCGQ6dqsGpsibNMRW1bXjhnZOosMciZOEGHI9civ2ds1Bsneh2TLazh/01PYIgRhAudxbqLO1Lly65/IzoO+pIJ/WOAjAuMFhW3Yr84locKapxvhgDeA7aPAxVaZErrd0w8Ry+6pmO64OqYQYDBw6ANtJKbCpH58E/IvjGeyikliAIBZdioc7S1mdsUwa3fwgLsYDnOIiQnNSy6cmowODlxnbsPlgKQdfMyWGFgsnEI2dJOqrq2qTkPZHBZOLR2W3H87tPQBAZOA6ISpuGtsxMJNguQ2goh3DxhNO4hMoT6Lx4EsELNpBJiiAIAF7uLAj/U1bdinc+Pw9BZOB5DjlL0hUfhVF5c32oLABYzLym5Ll8/vysRKVciPo8xoCiC1dw99xZCE7KRveRv8IRQOvs/GYMPUf+CoB8GARBuBELURTR2toKxhgEQVD+G5C65xH9Q5Nsx5iSJwE415IqqWrRCAXPAYtunID5WYmGTnA5eupAQaWTwIiMoaSqBWlJkbBk3ApryREwwQ4G9NaXUmDoObIL4rUGjJnzkL+mThDEMMSlWJSWlmLu3LmKQMyZM0f5jMxQ/cdVJVp1rsSKeanK8RazVImW4zk8ujQDi7OTvLqHxcz3huRCqmgbFmLBgYJKhIWE4FLI/TA3nkc7C8aa0G9gAjMIrf0E9gt/R9DMe2mXQRCjFI6NYHvTlSvtTk/W3jBYDd71pT2MMrjlnYP+WHfXMfpMrmArN2LS15riOGBKUBOemNaK8OYSsHZtdJWMKXUWgm+8BwBgrzk3rGtLDdbPeShBcx4d9GXOPM8hJibc5eduk/JkE5TZbEZ7ezvy8/ORmZmJSZMm+TQIwhh9sp1R3275c1dVa90JjNF5BwoqDYsSZqfHImtyJr7rsmF68s0Yl/9H6COlAIfzu/IkwPMA660tNVwFgyAI73CZZ1FWVoY777wTR44cQXd3Nx588EG89NJLePTRR3H06NHBHOOowahznieMBMbTPeREPhkTz2H+jAl45/Pz2Hu4HL/JvYrm+f8CPiHDxVUYIAqSx1ywwlpKvw8EMdJxubPYsWMH/vVf/xW33347PvjgAwDAgQMHUF9fj82bN+PWW28dtEGOFtw1SXKFK9+Hu3s8ujRDCcPleQ7rlmagrdOqEZ3TbVFYserfpIKE3+yBWFfq8pr2s4fQ0XQRlqmLyKdBECMUl2JRW1uLVatWAQC++eYb3HnnneB5HomJiWhvbx+0AY423JmbXHXa81VgFmcnITk2XHPOlQ6boeiY4tNgnjgDVjdiATCIjeXoaSyHvbYEpuikYe3LIAjCGZdiwfO9FqqTJ09i69atyuuenp6BHRWhwVe/hDfoz5maOs5QdMqqW1FzJRLZvAWcaHN1OQWhrAACACvHI3jBetppEMQIwaVYREZG4ty5c2hvb0djYyNuvvlmAMCJEycQHx8/aAMk3Du+/YlRO1dZpPKC7sIT01oRdulYby8MdzCRkvoIYgThUix+9rOf4fHHH0d7ezt+8YtfIDQ0FP/1X/+FnTt34k9/+tNgjnHU46tfQm2yAuCTiUqNWqTKrePxXeQtWD7zdlhLj0Jsugix0XOvjJ4ju2CvLQFnGQMOgCXjVjJPEcQwxKVYZGdn4/Dhw+ju7sbYsWMBADNnzsR7772H1NTUwRofAd/8EureGBwHqfaUo0GSUd6GnH8xd0YSYsIsmmvJtasExgCOQ1iIBab4VITEp0GoL0Pn/t8Cot3j+IWyAuW/bWe/hmXabSQaBDHMcJmUd/XqVbcnRkVFDciA/MlQT8rzJ/Lif6W1G4dOOVel5TngB4smY8W8VOdmSwAsFh6/WKsVE/UxAGAycZgxOQaRYUGYn26LX8IAACAASURBVJWI68yNsJYeBetshXCpSAqnBQdEjAfaGtwPmDfDkrkwoKIxHH/O/YXmPDoY1KS8uXPnKmU99HrCcRzOnj3r00CIgUPfDElfqhzQ5m1o6lJBKiJot2t9IfpjAKl3xsnzUmb3kaIaLJwxAfOz7kNaUiSE+jIlo9tecw7Wb993P2jRDtvZr2ArPUpJfQQxDHApFvfddx9OnjyJO+64Aw888ADS0gb3j/n8+fN4+eWXERoaipUrV1JehxvUvgUmMtyYNh5FF65AZFIexcKsRE3RQdkHIvf7lpsuqX0h+mP0CCJw6FQNDhfVOmpVpWkWfKspCBCsngcvWNFz6hOY4iZTuC1BDGHc1obq6urCwYMH8eGHH6KzsxOrVq3CypUrFR/GQFJYWIj4+HiYTCa8+OKLeO6553y+xmgxQ8k7C9kB/mTOTABw6+T2xmfR22yp1qmPhhoTz+GpdbM0vhShvgzW0qOwlxxxmKcAGMqOCs6E4AWP+T16Sr3rUYvRcPs5+wOa8+hg0GtDhYSEYPXq1Vi9ejXq6uqwb98+rF+/HqmpqfjDH/7g00A88dprryEvL095/frrr6OqqgpPP/001q9f79d7jTSMHOByO9bLje145/PziolqYVYiUhIinHpgGP1ypSVFSuXRHc8THIDUxAhEhQejsKxJMXWpy57LmOLTJEd4xq3KQm0rPQrb2a9cT4QJ6DmyC7bz+TBFJ4EfPwmsux3mCVMB9K1woVBfhs6PdwCijepYEUQ/cCsWapqbm9Hc3IyWlhbExMT4fSAbN27Exo0bldenT59Gamoq3n33XTzxxBO45557/H7PkYQ6R0Ltw+A4DkxkYOjt1w04ypWrmicZ7SwA57DdnCUZSEuKxKFT1dh9sFQxdV1p7UZZdSsuN7bju5IG3JQZh8XZSTDFa81TtpIjHiOoxLpSTXkRKzipcKEowMqZYJmxDFxQqFfCYa85B4g2qY6VaIe19Ch4h+ggdqaX3y5BEG7NULW1tfjoo4+wb98+mEwmrFq1CqtWrRqUpLzvvvsOb7zxBsLDw5GRkYENGzb4fI3RYobSc6CgEnsPl4MxSRR4noPoEAw1ymeMwWLWRkOpcVVqRDZT5RXXKm1bRZVPfP3yTKe+G7J5yrs8DQ94yBJ3NoVxUlgYEwHeggmPPoO24An9G8MwY7j/bvcFmrN3eDJDuRSLxx57DBUVFbjnnntw33334frrr/dttA7a29uxdu1a7Ny5E8nJyQCA/fv349VXX4XdbseGDRuwbt26Pl2bMOZcZTN+tfMo7HYRZjOPf1ydhfLqq/j820sap7acg8GYtIY+evc0PHincaXZc5XNKL7QhKwp4zE1dZzy/ntflOKtT886RV8BwMyMWPz6R/MNr9d9uQQ1b/07IHjO03ALx2P88h9C6GpDyKQbMCY5EwBw5Ys30frNPslNwvNweP8150XfloPoW+/v0227L5eg6+IZzT0JYiTjUiymTp2K4OBg8Dyv6YzHGAPHcThx4oTHixcWFmLr1q2oqKhAbm4ukpOTUV9fj5ycHOzduxdBQUFYu3YtXnzxxQGJthpNOwujRkqenNpyIyRBkERl7Z3O/bzl89w1ZZKd69DtLGamj8fdcye5TCJUdhkt1RDry7SLuU9wkFSBA8JjwAWFgDVfcn+KKajPOwu1HwTDzA8yHH+3+wvN2Tv67OD+4osvfLqREXv27MG2bduwZcsW5b38/HzMnTtXSepbtmwZcnNzsWnTpn7fb7TiajHXL9JG78nVZxNiI/D/fVhsKAiemjKpneuXG9txpLAGVfXtOFXWhNMVzYpfRC9CshMc6I1YYtZO2ApzYdR4yTWs99/2Jk8xV+BjJ2PM/EcwJjkTbX1YRPR+EHvNuWEjFgTRV1yKRVKS5x7Pnti+fbvTew0NDYiNjVVex8XFoaioqN/3Gs30p9CgLCDvfV2uZGvrr+GpNpVahNKSItHRZUNlXRsYA2x2EW/9rcRhDXLO+ZBRO8ItqbN6M8OrCr0rXOgFDIAIEzoiUsEd/xDXrE3AxLk+X8c8YSqsnEkaF8cr0VoEMZLxOhrKX4iiaGjWIvqOL4UGXZmnPvv7ReUYnpfqQB0oqFSOU+8eAGg+MxoPz3MQBOkZX7YEytFYR0/XOZVZV6PfcTiVFOF48PFT3DZk0sMA1Nkj0SEGY8qFL2HngKbq0zCl/l3pKW4tPepjscMR276eIJwYdLFISEjA8ePHldeNjY2Ii4sb7GGMKLwtNOjKXFVS1aL4djgAWZNjlNwMvVnLU28NeTwLsxINa1QBvu1+jExV8pN850fP+bTriDO1gjNBmSfg6CleVeRwgEvXspUcRujKXyqCoU/qs9ec670vE8kMRYwKBl0s5s+fj5dffhnNzc0ICQnBwYMH8eyzzw72MEYc3jRAcmWuykyJhtkslfYwmXhEhgW5NGt5a/Kan5WIo6frFMc3E7XP4WEhzjkdrujdDcUibWbvohy66pfKbkBO4OPGhKPn6FtSLgdnklTBETbLQwrvZUxxh0vo8z5EAd3f7IF54gzpevlva5L6uDHhkNrXS9eVXhPEyGbQxSI+Ph6bN2/G+vXrYbPZsGbNGsyYMWOwhzEq0JucXJmr0pIisf3Ht+JYUbXynrzQ681a3pq81LudK63d+LqwRlELkQHvfH4eybHhHgXO3U5GvetQYxqXrNmB2GvOQWgsh73SEcHHqYTCBWJdqdRKluN7o7QEKzq/2Al0NPe+xwT05L8N07hk2l0QIxq3SXnDndEUOqvH1SLrKsFOnrOcaNfaYVVKkesXdFfX8DgWVVFCDsD110Vj9YLJbq+hTjBUl1l3d6/84loA0Ixd23+Dh2/RVp7gYEq6AcGz7xvygjESfrd9hebsHZ5CZ03PPPPMM/0c15Clq8uKvkhhWFgwOju9qJg6hCk4U4czlc3S/BlDbFQIMiZGYdzYMcq/asLCglF0vhHPv30CFbVtqGvuxOXGdtyaleh0rKtruGLc2DGYNikaJp7D5cZ2xQzUeLUbecW1sNoE3HDdOMNza6504HR5MwDAbOIxKzMWZyqawfOc0/3LqluV8VfWtaHgTB2mpY7DuLFjwIePgznpenBj48BHxkFsumh0uz7D2hpgK80DFxoJU2yqX6/tT0bC77av0Jy9g+M4hIYGufx80M1QxODgaytWQPJHyBFMAGAXnAsE9hXZpzI/KxH78spxpqIFgOQ/+PSbKsRGh2hKg+hLifA8hyU3JRs63r0dvxyeK9SXwVZ6VNplSGqqGikHd1FOTHUEP24iWMdVoMfxBMdE9OS9Id27qgis8yosUxdpypHIznJuTLjHIolG1XJdVdAliIGGxGKE4ksrVpnMlGiYTL0hr2YT51O/b29FJTYyBBxaNEvydyUNWJydhLLqVnx67KKmqi0AgDGcq2pxmQtiNH4Tbzx+U3waQu/dguDWCvREXgeh+TLs5cdhnjwbYtNFl5VxmeP/GCTXNlpqwOmjsZiIniO7lJc9jeWwnvkCfISUWySF//Y61K3gHeojwsrxsMxYDmbt6g0VZoLiWAfgUwVdEhbCn5DPwoDRbON0ZfOXUYsDAE2HPlcJd+pz5eMBaEyE65dL9ZXe+luJYZ0pk0kKY5Kb95l4ODr1JWp8MZ3ddvzt71UQmXTOursyUFXXZjgfo5+zUspDsAEqDwufkA573XnwYBAZUGUfjxRLE3iP36w/kHwi/NhY2M4dcnxxHMzTFiNkoXGBzd55WDV9Qkbz7/ZoYlALCY4ESCy8x5s5K2XJRQazmcet0xPwdWGNZtG3mI1zLwBnZ/WNaeNhtQu4KTMOybHh+O3uE4Y/r5np4xEZFqS5F89JS7nZJJVZl81T6pLscsSTfEWzicMjd2UopUfmZScbztmVqejaR78FJwoQwGNv581YE/YtzPBPdrl36ExkHA9TSjb40EhYMqROkq5b23LgohIREpcMTFs6qnYa9PfsHf1qfkSMDsqqW3GoqBbJMaFudwVvOYQCgLI70LdedZd7ofejqIsMHiiodCnscn0p82npXIberHC7XcSRwhol9wOO/hqMSYKhFjK7wJQeHGYTj99EhRr28ND34JC5Nm8TCr48jFJrPC6zOCydPxdxLYWSKAWFwFb4ieH4/Yfu+2EihIsnIACS6YyTFNRqsiB4/iPasF8wsKs16LxaA5w/DlNKNgAYCs1oEhLCe0gsRjlGLVmNFnp1ljcgPeO2dlgx3RHFVFx+BaLIvM690Ps4MlOiYXEkBoIDEsaFou5Kp6Npk4iOLhuezJnpaPNaA9mPzQBU1LaB4xy9xE29DZ3Ka67h5PkmzZhFpRGUiOILTVg8I9Frv8uk6dmwRV+HiKoWPJISjUlJkQCylc/5sXGwFh0EOMA8KRu24oMeGz35FVkZBSt6Tn7suoqvQ2QAyfdiO3dY+vJEO6xuWtuSD2R0Q2IxylFnZMPDrkC9mANQFmKzicNdsyfiXFULosKD3d7PVaa5Uf0ptYjJC7nU5tX5urJpK2dJuhJVVVbdiuLyK7ALDByA9ORIlNdeU0Qta8p4lFW3YsfbJ2AXGMwmDlsemeVWMNxlygdNW6xZZC2psxRzltB00Q+l2H2gvcnzMTJMUBXuFZSILtbdDmbthFB9DjCbITZckLoV8mYE3/qokjEvm+pIQEY2JBajHFchtvqnbX1Gtrruk11gilMZaENx+RWPi64R+oVY3kkYjVdt+pIRGdDRZdNcb8sjs5QQ3PPVreB5DotulBzjU1PH4fdvXoBdkE1rDPnFtX4JFQaMzVl6f4j6X6HpolK6pFdczmttaeDAhceA+SIGvqKL6HJCtOs+52A1Da++HoTvkFiMcmQRuHylU/FZuOuPIX9+pLhWCVHlOWie9tX5DX0JrVVz9HQd7IKoqVQri1Z1YweOfV+vHKsO9T10qlrpBR4TOQaCyBzN8hhiIqVkvve+KEVrh+fEpf7OQY0rf4grXDnbe7PRhwIMEKzoOvJXcLwFXHgUgm+8h4RjhEFiQSAtKVITGeSpWGBaUiSecjyxA0BKQgR2f1bqlJ/hTYVad7gah3yNj45WApDM7dlpvV35Dp2qxhu5JQCAMxUtuHtOimb3FBZiUUxcHM/BxEsd/ngTh/lZiZox9HcO/cWVuISufFpTRFEpnqjGKG9Dc4z7BERfYc2XpKs1AZ2VJ2GaNBN8aKRS5JFMVcMbEgvCCW+yv/Umo+TYcKf8jAMFlX1uyuRpHLKQANKSN3nCWOXa35U0aK5T1dCm8YeoRYgXGRbdOAExkWMQFmJBSVWLMj/1fYzm4M8dh6/oiyiaxiU7VeA1yggPbq1Ap2BRPrdVnlBFcfmzZhZTIrVkrI58FS5YCs/0RkiMdlaDLTjk2JcgsSCc6Ev2t5HjV+1fAMf5VJbc0zjcCclNmXFKORH5tXrxDwuxaPwerR1WpCREGJYScefTCeSOQ4+rCrz6Y6Knz4RdFX9vik9THPHmCVMhNF9Gz5E34N9CizJM07BKKyQAN26i1D/d4cMBoAsIkHwjlulLIFSfAxNtgCiAj0qAeeIM5TxXpVb6gvXsIfTkvSll2I9yvwwl5RlASTy+4e4JW5/I589F1dN9ZZ+FXEZEvbgvuSlZ5ZR3JPk5Snnoq9sa3cfXargDOVdf8ObnbOSEVzvf1e+xzlaIbY1gzZcxFDsH8gkZCJuQCmvYBGUOCAqB2FQF8+TZAAB7+XHw41PArF1Kp0RA6pxoVzLmJczTbneZNT+UoKQ8Ysjh6Qm7o8sGkfXmNvirMCHgPox1cXaSpjCh3pxU1dCmCTISmVRLijGm+DXUrWNd7Zp8KdTYV/y9i/EkPL464YFegWHWTtgKczEwOxPfEetK0eai/a5Qfdrwv6XaYMb+HHvJEVgd0WruTH4jERILwiXePM16coYP9qLqarz6cdyUGYeSqqtK2KyJ57BuqVRHqrXDirc/K1Wq3co1r+T5GvUld+XH8MeOwNvuhN5wrrJ5QMxnaoGxpM5S+qYDAOtpNwgBHuq4GKtT2LCEFRz4pBvAATBPnq004NL7WuSe8upe7/6oLjwYfhUSC8IQb59mjcRAv0D66v8YiPEajSMifAxe/aBQMkU5Eg2Pnq5TKtsCgCAwHDpVg8NFUqc/xqSoKVlA1KYn/RjUNau8WZhdCUt/fT9qii80+U14XGHkPzEybcm+BQSFQlQ92XtNcERvefiAw5Q5CAZzsYIDN/46sKZy5T3b2a+AoDDA2gmAwQoAEXHgTGaw1jpHJWITLFMXKaYztbkMQSGK74a1VAOMKX4VxM70+wxJLAgN8oJ1pbXbq0XFVea1UY6Gp3t6EhN3x3nz9K0fR1untTdxWWT4rqRBibDSI6relgVEnfuhH4PdLuKzby+5Lamun5srsUtLikTOknTF9+NtS1ojsqaMH7Sdnhoj05ZRnw91fw9ZWNRZ5NLBdsV5LT+ps85Wxw5mkDLkfYZphELB2qF93dag3dMwQVMy30iINAhWdH31F1yz/gCYOLfPozWCxIJQUC9YPM/BxHMe6z0B2kVYHS5rt4seM6K93cF4Os4Xc5csOgmxEU6mqdJLrRAEqaSJ6GHN0c8vMyUaPAcIDkd57ZVO5VjeRW8NGU9i5y/fz9TUcYOy0/MVvZh4a0rR72LUOxjZAR8cbEZPT29+CR8aqTyVw2yGWH9BKnniCs4EPn6KJpJrKMOu1aPp050IXtjd72gwNSQWhIJ6wWIiw0JH/oEvi0pmSjR4XmpAxADkFde67XFRompoJC++Rv008otr3T6le2vuUouOxdxbdDAsxIKOLpvyWl/SxAj9/C43thsKDAdgga7vhi+hwN587i3nKpuHnFD4E6MdjKfIILUfQR/tpfYtWM8egu3sYcBsVnJFAAzZaDB7+XESC2Jg0C9I7hZ5V6QlRWJhVqKy0Iqi+9asavs7AzQ9K/KKa7HlkVkAgCOqGlGuntI9mbsAZ1OR3NtCv2uR76lu0wo4XBtcr69Wnh8AvHWw1Gm5kCvhzs9K9Ghqcid2Rp+7M8sZNbEqq27F7949CZt9aOSGDBW8yVEBnAtFqlFHg+lNZggKhVhzRvml4RMyYIpPg73yFJhgBR8uVW5m3e1ggh1oa4Q/hEcODfYXJBaEgr+c0fOzEnH0dJ3Lp2D1Iqcu/Ac495+QF2I5X0b9lN4X1IJoNvNOGd3yrmXFvFSlpElrhxWRYUFISYhQdiHvfH5ecTh3dtvxzuel2hLuHLD8lhSEjjEr36UrE536+3CX26EWQ3fVcsuqW/H82ycUoZNFt6SqRUpEHEDn9mjFU7ixYbTSnIfcHsusnRCbqoCQCKCrTXFuy1Fm6gx4Zu2ErehvDnMah8i5qyH6cVcBkFgQOrx5OvfmGq5ER7/IPXJXhtSzx+BBSl0YUL/j8cfY5s5IUpofGZl4PH0Xuw9K4bWfflPl9Nn1k6JR1dCmyR43MtGp62rJAlPX3Kn0IOd5Do8uzdCUXS+pakF5zTWX1XLzdTsi+XMAMJl4YJCd24RvuSt9yXMBoMnEj5k+0++JxSQWxIDgaqHNL67VLHJVdW3IThuvaVKUGBOKzIlRGjOYP52y8vnFF5qUSru+Xl92OLviTKW0I5LLjizOTjI00R0prFEWdsbgJDyiKHX3S46VbOTqHuZGyBWB1Zh4SZgEkcGkKtE+kIl9xODTV5HxFhILIuDcPXcSisqvQBQYeBOHf7hnmsew1/7gqjtgX4ocGvXV0Of+flfSoBRabO2wwmzqjTKTmkV5KL+h8osojarQ22NcXS1X39HwusQIRIUHK2Ks3nGoM9T1ePKHDKW6WMTgQGJBDCrzsxJxpLhWEYaUhAiUVLVg3V0ZirPZl4XH06Jm9Jm77oC+PDHfOj0BgFSiXc78jgwLQkiwWbNDSImL0PgQTDxwY9p4xQ9SdKEJbjYLAKRAgOTYcMWMBQA8Dyycod0lhIVYFLEymTgsvHECdn/WG/LJABwurFF2GkaLvVFyofpn48+McmL4QGJBDCppSVIvDLn6qy8ZznrcPeG6+6y/lWT1+SgLASezTmx0iFLIsKPLpnmiF0Sg6MIViIzBfJrHuqWZitjIvgo1HCeZvfRmLMaAmMgxmjnv/qy093wGVNW1ORXTFBkgOsYjCCI+PXYRVrugFF1Ui4HNLuLNv5UAqsz1lARtfoq+jhYxMiGxIAYMV0/pssmnv/0u3D3h6j/LL651KkGi7g7o6Xqu72ucza0uZFhW3QqTSbUj4CRfhJxg19Flw/rlUubyG7nnnPI7zCpB00eahYVY8EbuOeVYrSgx5Xx1CRM1jPX2Uj9T0YLiC1eQNSVGY2KTzV7yXPX5Kf0RfGL4QGJBDAieTBlA/xPN3J2v/ozjOUOzi7o7ICCZcHiOgwj3WetG/gpPJVHUYbgAUFx+RVrYdbWe1GLAqYoY6vMx5GvtPliimLBMvPQ/9WsAyFmSjqq6NhwpqnEyd+n9LSfPN+F0RTNylqTju5IGTV8QGbtdxHclDVi9YDKZpEYRpmeeeeaZQA9ioOjqsvap0GVYWDA6Oz33Zh5J+HvOBWfqcKayWckGP13ejO8vtuDYmXpMmxSNcWPHYNzYMZg2KRqxUSFYteA6nxcZd+erP4uNDEFFnaMkOWOIjQpBxsQozZzLqlvx54/OwC5IlWbX3ZWBWRmxbu9r4jlcbmwHIIWkrlpwHcaNHePynLAQCz48UoGaK1I9IE4aDr6vbDH8TlYvmIzbZyVj3NgxKKtuRcGZOvC85NZ+98syVDd1aH6/GZPay05NiUJ0RDAarnahoq4N31e2YO2d6VgwYwLCw4JQWXsNygAMYCJDdHgw7ro5Bfln6gz/hhqvduPo6TqMDQ1SypqYPXwHgcLT77b6ux1qY+8rffl75jgOoaFBLj8f0jsLQRDw+OOPY8uWLcjKygr0cAgfUD/Zg+M0Zhf106e3UUieTFpGyJ+VVbe6TRIEtG1awZhTsqCra8/PSvTaIa5xrDO43JXo56Tfpd06PcFl+GxhWRMeXZaJji4bTpU1OSUazstOxsy08UquhjpkWUZdxkTtIwGA6IggtLRJi5AoMhz7vh6A5FfJWZLep11FIMNwKbLLe4a0WOzcuRNxcXGBHgbRB9S5C7Jdu6/mJm/+oN0tON7kUejNVldau1FW3epx4fBG7OSxye1c5XtwgBJC685JrDf1ANBcJy4qRHm6F5mULLhuaYZLE518/Sut3TDxUsFE3sRhxuQYRTzkMiZy9JrsC7nWYZV8LrrdBmNA8YUrLiPaXP18+vOz9fV9IzyZ0SifpJchIxavvfYa8vLylNc5OTlIT0+H6Kn0JzFkUS+kybHhff6j8+YP2tOC42lRV/sC8oprHT0sajXZ033Bne9Gnps7J/GhU9U4UdoInuPAHL6U+Q4/hros/G93n1CinkTHzkiej7sx8TyH27J7mzudrmjWCIxRBNaNaeNRdOGK4kCXKSxrwsnzTYZZ5y+8I9Wk0n/W15+tL+/Hxka4/Pm4i4z79NhFFDp2Z/5uCTwcGTJisXHjRmzcuFF5/bOf/Qzh4eE4ffo0qqqq8MILLwRwdER/6UtSndETudHOxF9O1t4cAseiq8qe7usioR9bR5dN0zDJXVTYoVPVeCO3RDl2Zvp43D13ksZcJfPo0gyp3wVjmuipo6frYBdEJVorNjbCqbqwOvzWaAemj8C6e+4k3D13EvKLa3GkSAocUO82RJHhrb+VoKquTRE1ORpL/526Cipw1VdFjmpz1W/F6HdhXnay25+5UYHG53d/pwkGsBtE1I02hoxY6HnxxRcBAC+//DIWL14c2MEQg4430VQy/mzdKtdvUj+l9yfCx5uxuTrmu5IGzXFWu+ByHIuzk5x2b/rChfvyyhEdFep2TEai7sqMp/bZhIVY8NbBUtX3BiWkeMlNyZr6XyJjSil6OUoNAK5LkHYArvqqyFFtdkcdLfVuSzbjeXqwkK+v3pXp55VfXOsUNcYBijDqCzeOFjjGBrYxbnt7O9auXYudO3ciOVlS+P379+PVV1+F3W7Hhg0bsG7dugG595Ur7U4JSd7gqf79SGSozflAQSX2Hi6X2phywA8WTdY8kevxxbYsH6suJKjm0KlqzVN6X80P6p2Rp+x0o/Hrdxbrl2f6ZBJTFl1HiC8HwGLh8Yu1Ugl2fz8ly9+b3jylFgoOkn8EDE7HAVLxyCyV74QDkBATijFBJk3ZEkD6vbgxbTwARyiyIzRa/2ARGxuBglOXnUx+vMNvpD9P7/jnAKQnR6L0cqvy3uLsCUpuTKBw9zvfl79nnucQExPu8vMBFYvCwkJs3boVFRUVyM3NRXJyMurr65GTk4O9e/ciKCgIa9euxYsvvoi0tIErgEUMP85VNuNXO4/CbpdKiW//8a2Ymjpu0K57rrIZxReakDVlfJ/uq74Pz3O465ZJuGP2ROVa3l4/t6AS+UU1mD9jApYbiKWn65yrbMbbfzuHU+cbFeF99O5pePDODLdj92ZsRsedq2zGl8er8Pm3lwzrZs3MiEVwkAnHTte5vK4rOE5auNUaI+865LeM5pdbUImde4sgMiaZvFTHK9eGtFgyJu1aRCYJHAdgjqOsi3rMd8+bhJ+syfZ5Dv5ioP4+3DGgZqg9e/Zg27Zt2LJli/Jefn4+5s6di6ioKADAsmXLkJubi02bNvn9/rSz8J6hNueYMAt+sbbX9BETZvHL+I4VVcNm7zXNHCuqNtxdxIRZsHiG5PTty33V9xEFhk8LKvH5t1VKYyVNuRBd4p2am9JicFNajGYc6h2Lp+zpmDAL7p6TgtPlV5QeHskxoS7n1N82tzFhFjx42xTMTBuPfXnlmqQ+E8/h7jkphk53b5Afa3lpYwLOYOE3maT5qXcS6t2OYPBszDnyTZSdDut9n+c4fOPIwVBHjs1MGz/gfy/udg6efo8HYmcxoGKxfft2p/caGhoQG9ub7BQXo0KKUQAAFXdJREFUF4eioqKBHAYxTOmLU9wTRs2PBgJ3Wd4AVE5Y43IhrlAv0hzHgbnIX1Gj9jm4Mr3J9K3kiXGb29ULJqP0kmQG43gO65ZmKMeoi0muuysDVXVtuFjfhopaDwscByxytPtVh2Srs90BaL4jTw+MqQkRqKpvg6A/jPWKCxOZcl/14u2N+bMv4beeRNvbagP+ZNAd3KIo/QBlpG2fi1RSgjCgP7Hvviyc/UEfiivnU6ibOXlbLkSNNrGPKaYTb5y5K+alenziVIucvhSJ0XHuHMnuHONyMUn9wiv7WNSta4HedrZyi1p3Idlqx75sVnJlbJdLoqgj9B1J8k65MOr7ym1r3VXv1czJi12kGn37X3Vzq7LqVkkkRWlu068bWPOTzKCLRUJCAo4fP668bmxspMQ7wmv8kXEr71gG2vTmLsvbnZDoUS/4+kXaXZSYrzkH8phzlqRLDn6R4Z3PzxuGDuuTLuUdk7f5LZ6irsJCLHj7s1JNR0X1PPU9xoHe/hz6XZ2RUHAckJ4UifLaa8puRu6XbpQLo7+vHBkl40rs84trlbBh/S5S/hwAQoLNSmfFxdlJTl0VjxTVKHNVX1MuBHm6onnA80AGXSzmz5+Pl19+Gc3NzQgJCcHBgwfx7LPPDvYwiGHKUCxc52mn42phdCUk+mvrF3yjp/Wy6lanDHBfcw5k5C6AevOWfp7yfVwlx/Vl9ydft6y6Vdl1MUAjWPoe44dO1Uh+B0fyXM6SdNw6PQEll64qme1qOA54zFES5Xy1FOHEAbg+NRqrF0x2aXa73NiOdz4/b1jB11WveX3XQkAq+/7+V2Uoq251yoRXd1ZUJ0MKIhyJojVgBnnKg/G3MOhiER8fj82bN2P9+vWw2WxYs2YNZsyYMdjDIIYp/syp8Af93el48ssYLfgr5qV6leXc1+/K6DxX9zAylwBwmbHtLeqOf0xkGsHal1euKcUO9O4ebHbRMHxXj7xzUM9z9YLJAKCUfPfW/3FdYgRylmQ4/Rz1XQvVqMNw9ew/WoHk2HAlGVJtrnRV0ILnpX71h05V47uSBiy+KUUJjPAXgyIWX375peb1ypUrsXLlysG4NTHCcGUHDxTe7nT6+qTtzYLvagx9/a6MzlP7AWx2qWHSTx+YYWguuVjf5jJjuz/zVpcNcYWcN6F/Lz05Ehdqrmky3PXzBKDZseQV12JBVqLGR6T3f/AcDIWirLoV5TXXFF+LN452mZY2K57f/R0WzpiAnCXpKL5wxbDgoxrGGE6WNiodGs9UtPicl+MJKlFuAJUoH9qMGzsGGROj+l1O2h9z5nkOx87UAw4ns1GJbnmRO1PZrCnR7g36MuwAnMppy2Ngjizn2VPjkJowVjlf/V15O2f9eTzP4WhxrfL3VNfcicjwIMzKiMXVth5U1kl2f8aAq+3O1zdxUjl39bjdlQY3mve+vHLUN3cBkAQgMSZUUx2YgyMBUHUdjpMSETeuvAGpiRGwCyKW3pKilJ9Xz7PgTJ0m1FdkUqRUdVMHwBjMZh5Lb56IitprUs4Kz+HRZZm41mnFB19fgNUuIjVhrGImq2nqBIMkKMtuSUF5bavX6xFjQKWjvHx6ciQu1ml9a/qYIMaA2isdGiG1CyLmT0/07oYY5iXKCWKo483Te3/9LGo7vpEpyFuntB5fdjtpSZGYFB+hCW39rqQBi7OTnCrTysgLGm/QfAqAx3Iu+nnLC6HsiP6He6YB6HUSt3ZYNZnf2enjMXnCWMydkYSWq51KTkrppVbD7yczJVrT0dBs4pyKNqYlRWpa5ja2dGme5gE4tdEVGRA6xoxFMyY4dUHkeSmXQ96Z6VGqDJt5TYhwSkIEdn9WqrlPR7ddc+5Nmf4NHCKxIIh+4snv4C8/izvRceWUdoW3pcHVEUcLb5yAitre8iPyYpSWFOnU94LjgOW3pCB0jBlXWrvxdWGNZtwAFFu87GdwVV5F3WvEyBEtC8rzb59QzjGZOGRNiVF2Ht6WIpdzPuQ5y9dWHyeLzrmqq0ovc5ncby5i+ZxJ4HmtfyEsxILMlGjDLojy+MJCLEov9uLyK5qQXaNAiOTYcHx67CJOnW/SCE1c9Bg8eGfm8PRZEMRgMFR7D/jLz+JtG1lvBMmbxVNvv9/yyCysX56pPFWr7eGyM1Yd0nnw+CXlKdh8Wju2y43tmgVOybA2GIuRI9qdM5kDkDU5RlnU9+dXYu2d6YY+EE+Z8K4aT8kJlXoaWrqx+7NS3DhlfO8uh5PE3N3vgZFQG+WnqElLisTkCWM1/gwTz2HjvTc4tQz2ByQWxIhgqHc880c2uqfFxhdB8iQuJVUtmsXQLjAlEsvIaSrff19eOb6vaHHscKS8Aou518wUFmJBfnEtLtZrFzKTKrlQ3whKneDo7Xwiw4I0UVpyfw+1M9ubTHi9qALOCZV65O/NYnb+fr39PXB3nD7vxmLmDbPk/Q2JBTEiGIr5FwOBu0XEF0FyJy5l1a1KFz25VLfZxHncraQlaUt8qLPTZX+EercCaBPhquraUNPUgbf+VmLYcEjfm8NILGVBUe9m5LIu6u9HHd3lLhNeL0KySUgtikZcbe9xmzDZV7zNuxkISCyIEcFQy78YDqgXTyOTDM9zmJkeg8iwIK9KVMjX1Geny21q8w2c4LL/AYCTw9buoamR0XhkQTGf7t3NGJV18TYT3pWoyqKo7jGvprK2DZcbzyvOfFctc41wZ071Ju9moCCxIEYEQy3/whuGio/FVXFCJjJMnjDWbR8RI9TZ6bJofF1YA57nNB31zCZO8T8cKKh0EhKe693NqBd3Vz3SjboSZqZEo/hCE5JjQvtstjPasanPVwRWtZuSTVr5xbW9AqaKBFP3OVH3O7nc2K5Etck7K/n4zJTogBQQlCGxIEYM/vALDBZDycfia3FCb5F3BILIlFLtSjgtBzxyV699XR+2ynPQ2N/1O5avC50r9ep3C2EhFrzwzknltZHpqj/fufr85NhwR82oGsV0xzsqEurbwqoDAWTkplBMZIqY2uwi3vm8VKqIK6rCbEXpZ5SzJH1Qf2dILAgiAAwlH4svxQn7c204di0y6oS6tCSpEq06VNfoaV4tPvrvTb9b0IhgP79jb+p/SWPrfW+Bw7+h7l8OQAkDVsMgianeB6LOaxFFQNm7MKb5/gYDEguCCABDyceidw77WprDm2urzTWu5uzNk76n701/DVfH+tqG15tdoL6ce2uHlMmuj8LS13sCHI2W5DavAnMq0S4jl08PxO8MiQVBBICh6GNxF23UH/Tmmv7M2Vdfw5M5M3H5SqfGZ+GrCdDbXWBHlw2O4rcAtKXD1X4ftXjqfRby/Yz8IEal2gcTEguCCBBDyccyWGYxf+Wb+BIirE9QczVXV7sNb3eBmSnRMJs9N7XyNH69sKrFxN156vF76lvSF0gsCIIYUmaxgcaXEuyA97sZo7Dh/nyXelEx6lmi/kw9/t9Ehfq9CySJBUEQQ9IsNlAYzVWdpNeX3YD+OE9NrXzFk+lM25FPRPGFJiye4X3FWW8gsSAIAsDQMosNNPq5+ntn5e/v0p2ZUN+Rj+c5ZE0Z77d7y5BYEAQx6hnqOyt3YqYvorggKxFTU8dRIUGCIIiBYCjtrIz6nbsSM6P6VQMBiQVBEMQA44+8DldiNli7IhILgiCIAWSg8jrUDMauiB/QqxMEQYxyjBZ/d8hmJZ4LTKa2K2hnQRAEMYD4Gmk1VJ3tJBYEQRADSF8W/6HkbJchsSAIghhghuLi7yvksyAIgiA8QmJBEARBeITEgiAIgvAIiQVBEAThERILgiAIwiMjOhpKbpg+2OcOV2jOowOa8+jA1zl7Op5jzKjTK0EQBEH0QmYogiAIwiMkFgRBEIRHSCwIgiAIj5BYEARBEB4hsSAIgiA8QmJBEARBeITEgiAIgvAIiQVBEAThERILgiAIwiMkFir279+Pe+65B0uXLsXu3bsDPRy/097ejnvvvReXL18GAOTn52PlypVYunQpXnrpJeW4s2fP4v7778eyZcvwq1/9Cna7PVBD7hevvPIKVqxYgRUrVmDHjh0ARv6c//M//xP33HMPVqxYgf/+7/8GMPLnLPP888/j6aefBuB6bjU1NVi3bh2WL1+Of/qnf0JHR0cgh9xnHnvsMaxYsQKrV6/G6tWrUVhY6HL9cvXz9xlGMMYYq6urY7fffjtraWlhHR0dbOXKlez8+fOBHpbfOHXqFLv33nvZDTfcwC5dusS6urrYbbfdxqqqqpjNZmNPPPEEO3ToEGOMsRUrVrCTJ08yxhj75S9/yXbv3h3IofeJo0ePsocffpj19PQwq9XK1q9fz/bv3z+i5/zNN9+wtWvXMpvNxrq6utjtt9/Ozp49O6LnLJOfn8/mzJnDnnrqKcaY67n98Ic/ZB9//DFjjLFXXnmF7dixIzAD7geiKLIFCxYwm82mvOdq/XL3d+4rtLNwkJ+fj7lz5yIqKgqhoaFYtmwZcnNzAz0sv7Fnzx5s27YNcXFxAICioiJMmjQJEydOhNlsxsqVK5Gbm4vq6mp0d3cjOzsbAHD//fcPy+8hNjYWTz/9NIKCgmCxWDBlyhRUVlaO6DnfcssteOONN2A2m3HlyhUIgoBr166N6DkDwNWrV/HSSy/hxz/+MQC4nJvNZsO3336LZcuWad4fbpSXlwMAnnjiCaxatQpvvfWWy/XL1d95XyCxcNDQ0IDY2FjldVxcHOrr6wM4Iv+yfft2zJ49W3ntar7692NjY4fl95Cenq4sFpWVlfj000/BcdyInjMAWCwW/PGPf8SKFSswb968Ef9zBoB///d/x+bNmzF27FgAzr/b8txaWloQHh4Os9mseX+4ce3aNcybNw9/+tOfsGvXLrz77ruoqanx6ufcn3WNxMKBKIrguN4SvYwxzeuRhqv5jrTv4fz583jiiSewZcsWTJw4cVTM+V/+5V9QUFCA2tpaVFZWjug5v/fee0hMTMT/397dhUTZtAEc/7trmmVQN5KGkWVI9GlhWFYqmogp4WIGspBZEoSm5ZmFYZiSqQnqQRlFkR0oZGqYlGRRsBnmR0kHJkQZ6iYrxqJZ67b6HPi01Ju+m0+WuF6/I3eY3Z1rvL0vZu5xJiAgwFo2WWwTxTgbY96yZQt5eXksWrQIRVGIjY2luLj4j/+e7fo8i6nw8PCgubnZ+tpgMFinbOyRh4cHBoPB+vpbvP9b3t/fP2v7oaWlhdTUVE6dOkVUVBRNTU12HfObN28YGRlh7dq1uLi4EB4ezr1791Cr1dY69hZzXV0dBoOB6OhojEYjw8PDODg4TBiboigMDg5isVhQq9Wz9m+8ubkZs9lsTZBjY2N4enr+0rX9OzHLyOJfO3bsoLGxkYGBAT5//kx9fT1BQUEz3aw/xtfXl7dv39LV1YXFYqG2tpagoCA8PT1xdnampaUFgJqamlnZD3q9nuTkZAoKCoiKigLsP+bu7m4yMjIYGRlhZGSEhoYG4uLi7Drma9euUVtbS01NDampqYSGhnLu3LkJY5s3bx5bt26lrq4OgOrq6lkZ8+DgIHl5eZhMJoaGhqiqqiI/P3/C+9dk1/x/ISOLf7m7u5OWlkZ8fDxms5nY2Fg2bdo00836Y5ydncnNzSUlJQWTyURwcDAREREAFBQUkJGRwdDQEOvXryc+Pn6GWzt1V69exWQykZubay2Li4uz65iDg4Npb29Ho9GgVqsJDw8nKioKRVHsNubJTBZbZmYm6enpXLx4kWXLllFYWDjDLZ26kJAQXr58iUajYXR0FK1Wi5+f36T3r8mu+amSk/KEEELYJNNQQgghbJJkIYQQwiZJFkIIIWySZCGEEMImSRZCCCFskqWzwu5kZ2fz/PlzYPwf1Tw9PZk/fz4AFRUV1p9taWhooLGxkYyMjEnr9PX1cfz4ccrLy3+/4UB6ejo6nQ5FUX4o37dv37QvbU1PT8fHx4fExMRp/Vxhn2TprLBroaGhFBUVsXHjxpluyi/5mzdwSRZiKmRkIeacDRs2sHv3bjo6OigoKOD169dUVFRgNpsxGo0cOXIErVbL7du3uX//PqWlpRw4cIDNmzfT2tqKXq8nICCAs2fP0tvby969e2lra6OkpISenh4MBgM9PT24u7uTn5/P0qVLaW9v58yZM5jNZlasWEFvby/p6els27ZtSm0PDQ0lKioKnU7H4OAghw4dQqvVAuOjprKyMlQqFW5ubpw+fZpVq1bx6dMnsrOzaW1tRa1WExYWRlpaGgBtbW3ExcXR39+Pj48PFy5cYMGCBdPe52L2k2Qh5hyz2UxISAhFRUXWG+nly5dZsmQJL168+OEG/L33799TVlbG8PAwe/bsoampieXLl/9Qp7m5merqalxdXTl69Cjl5eUkJSWRkpJCVlYWwcHBPHv2jISEhEnbd/36de7cufNDWV5eHmvWrAHAaDRSWVlJX18fGo0GPz8/BgYGuHLlChUVFSiKwu3bt0lOTubu3bsUFxdjMpmoq6vDYrFw+PBhmpqagPFptBs3buDk5MT+/fupr69Ho9H8Zg8LeyTJQsxJ37ZrX7hwIZcuXeLx48e8e/eOjo4OhoeHJ3xPSEgIKpUKV1dXvLy8MBqNPyULf39/XF1dAVi3bh1Go5HOzk5gfDsOgO3bt+Pj4zNp2xISEv7v1JBWq8XBwQEPDw8CAwPR6XT09/cTGRlpfdYRExNDTk4O3d3dPH36lJMnT6JWq1Gr1dy8eROAqqoqwsLCcHFxAca3dR8YGLDZd2JuktVQYk76NtXy4cMHNBoNPT09+Pn5ceLEiUnf8/2D8W9bXv9KHbVa/VPd73eCnapv5zHA+HbcKpWK0dHRn+qNjY3x9etXHB0df9iWWq/X8/Hjx58+a7KYhABJFmKOe/XqFYqikJSUxK5du3j06BEAFotl2r5j9erVODk58eTJE2D8lMLOzs7/fK5AdXU1MH6etE6nIygoiMDAQOrq6qwjg8rKShYvXoyXlxcBAQFUVVUxOjrKyMgIqamp1tViQvwqmYYSc9rOnTu5desWERERODg44O/vj6IodHV1Tdt3ODo6UlJSQmZmJoWFhaxcuRI3N7dJl/BO9MzC19eXrKwsYHwr8piYGL58+UJGRgbe3t54e3uTkJDAwYMHGR0dRVEUSktLUalUHDt2jJycHKKjo7FYLERGRhIeHs7Dhw+nLUZh/2TprBB/wfnz50lMTMTNzQ29Xk90dDQPHjywHgX6q2bbUmBhP2RkIcRf4OnpSUJCAo6OjoyNjZGdnT3lRCHETJKRhRBCCJvkAbcQQgibJFkIIYSwSZKFEEIImyRZCCGEsEmShRBCCJskWQghhLDpHzdF3R9M1N/bAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "xy_augmented = torch.utils.data.TensorDataset(x_augmented, y_augmented)\n", "loader = torch.utils.data.DataLoader(xy_augmented, batch_size=25, shuffle=True)\n", "torch.manual_seed(123)\n", "optimizer = torch.optim.SGD(net.parameters(), lr=0.03, momentum=0.9)\n", "net.load_state_dict(torch.load('net.pth'))\n", "losses_train, losses_test = [], []\n", "for epoch in range(500):\n", " net.train()\n", " for x_batch, y_batch in loader:\n", " y_pred = net(x_batch)\n", " loss = loss_fn(y_pred, y_batch)\n", " optimizer.zero_grad()\n", " loss.backward()\n", " optimizer.step()\n", " losses_train.append(loss.data)\n", " net.eval()\n", " y_pred = net(x_test)\n", " losses_test.append(loss_fn(y_pred, y_test).data) \n", "plt.plot(losses_train, '.', label='TRAIN')\n", "plt.plot(losses_test, '.', label='TEST')\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Augmentations can also be applied by mixing pairs of input samples or applying augmentations to inner layers rather than input features. See [this paper](https://arxiv.org/abs/2002.11102) for some recent developments.\n", "\n", "**EXERCISE:** Identify some potentially useful data augmentations for (grayscale) image data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Dropout" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Another approach for improving generalization with a small training dataset is called \"dropout\", which is implemented (in PyTorch) as a [DropOut module](https://pytorch.org/docs/stable/nn.html#dropout) that randomly zeros a preset fraction of its inputs, effectively turning them off for a single minibatch, and forcing the network to try and compensate with the remaining input values.\n", "\n", "Using dropout in training is similar to training an ensemble of networks with slightly different architectures and building a consensus output, similar to the design of a random forest of decision trees. Dropout can also be used to estimate the Bayesian uncertainty of a network prediction (see [this 2015 paper](https://arxiv.org/abs/1506.02142) for details).\n", "\n", "To introduce dropout, we rebuild our network with two Dropout modules that each zero just 1% of their inputs. Dropout is [usually](https://sebastianraschka.com/faq/docs/dropout-activation.html) applied right after an activation module:" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "torch.manual_seed(123)\n", "dropout_net = torch.nn.Sequential(\n", " torch.nn.Linear(1, 20),\n", " torch.nn.ReLU(),\n", " torch.nn.Dropout(p=0.01),\n", " torch.nn.Linear(20, 25),\n", " torch.nn.ReLU(),\n", " torch.nn.Dropout(p=0.01),\n", " torch.nn.Linear(25, 1),\n", ")\n", "torch.save(dropout_net.state_dict(), 'dropout_net.pth')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Repeat our earlier learning loop on the sparse data using the new model with dropout:" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEJCAYAAABlmAtYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9eXgU15mo/1ZVd2tHEkIbEpJAQgKbHWw2g3fw7mS84SXYN+PYN7Fzf+MsztyM8zgZjycTJxnnJvHYzmQyThzHGZw4IXjBeMMYCzDGgMBGEgIJgXYJSWhXd9X5/VFdpepWtyRAG+i8z2Oj7q7lq+qu851vPYoQQiCRSCQSyQCoYy2ARCKRSMY/UllIJBKJZFCkspBIJBLJoEhlIZFIJJJBkcpCIpFIJIMilYVEIpFIBkUqC4lEIpEMimusBRhJmps7MIzTLyNJSoqlqal9BCQav8hrnhjIaz7/OdPrVVWFxMSYsJ+f18rCMMQZKQtr34mGvOaJgbzm85+RuF7phpJIJBLJoIxry+LkyZM8+eSTREdHc+mll3LVVVeNtUgSiUQyIRnXyuLFF1/k3nvvZd68eTzwwANSWUgkkiEhhKC5uYHe3m7q6xUMwxhrkUaN+np1kOtV8HgiSUxMRlGUIR93XCuLxsZG0tLSxloMiURyjtHe3oqiKKSmZuJ2u/D5Jo6ycLnUAa9XCIOWlkba21uJi0sY8nHHdcwiLS2NhoaGsRZDIpGcY3R1tRMXl4CijOshbkxQFJW4uES6uk4vY2pcWxa33XYbTz31FG63m3Xr1o3KOcuqWtlaVENmUjR5GfGjck6JRDK8GIaOpo3r4W1M0TQXhqGf1j5jcjfb29tZt24dzz33HJmZmQBs2rSJZ599Fp/Px7333svdd99NSkoKP/nJT0ZNrrKqVn788l503UDTVL5950KpMCSSc5TT8cdPNM7k3oy6sti/fz+PPfYYFRUV9nt1dXU8/fTTvPrqq3g8HtatW8fSpUvJy8s7q3MlJcWe1vZbi2rQdQNDALrBiaZOli/IPCsZziWSk+PGWoRRR17z+Ul9vYrL1eeCcv49Fvz4xz+kqGg/Xq+XEyeOM336DADuuONOfv7zp0lNNWOzuq7T29vL17/+D1x66eX2/hs2/JGf//zf2bjxDZKSptjvf+EL1/Mf//GfTJ06lWXLFvF//+/3uPnmL9rX+9WvfoX773+QxYuX9JNJVdXT+i2MurLYsGEDjz/+OI8++qj9XmFhIcuWLSMhwQy2rF27ls2bN/Pwww+f1bmamtpPqzglMykaTVPBb1lkJkXT0NB2VjKcKyQnx02Ya7WQ13z+YhiGHeQdLOA7GjzyyHcAqKmp5utff5D//u8/APDGG5tYuXI1//RP37e33bZtK//2b0+ycuWl9nuvvbaRVasuY+PGv7J+/ZcDjq3rfdf67LO/YNmy5SQlpQBmVpjzcyeGYQT8FlRVGXCCPerq9sknn2TJkkAtV19fT3Jysv06JSWFurq60RaNvIx4vn3nQu65drZ0QUkkE5SyqlZe31FBWVXrmJy/traGSZMm9clTdphTp05x9933smnTXwdMi73ttjv513/95xGRa1xEgAzDCPChCSHGzN+YlxHP8gWZE2L2JZFIArHilj7dwDVKccvt27dx33130dHRQU9PNxddtJQf/vCn9uevv/43rrjiKmbNmo2maezatYPly1eGPNbdd9/Lhx9uZdOmv3LjjV8YVjnHRV5ZcIpsQ0MDKSkpYyiRRCKZiJRUNuPTDYQw3Tsllc0jfs5LLlnNCy/8gRdeeInMzGlMm5ZFVlY2AD6fj7fffpOrrloLwOWXX8XGjX8OeyyXy8X3vvcDnn/+l9TV1Q6rnOPCslixYgW/+MUvOHnyJFFRUWzZsoUnnnhirMWSSCQTjIKsRFyaamdEFmQljtq5Y2JieeyxH7B+/R0sWXIxc+bM46OPttHW1sZ3v/ttwFQezc0nqa+vIyUlNeRxcnPzuO22O/nRj/5lWOUbF8oiNTWVRx55hPXr1+P1ern11luZN2/eWIslkUgmGFbcsqSymYKsxFGPW06dmsEtt9zB//t/P+FXv/otr7++ia985avcc8999jYPP/wAmzb9lb//+wfDHufuu+9l27atHD1aNmyyjZkb6r333rNrLABuvPFGXnvtNd566y2+8pWvjJVYEolkgpOXEc/1y3PGLMHlS1/6X9TUVLN58+vs3fsJN9wQGHtYt+4eXnttI7oevqjO5XLxT//0+LDKpQghzttG76ebOmsxUdILnchrnhhMlGuurT1GWprp9x8PqbOjyVCv13mPYBymzkokEonk3EMqC4lEIpEMilQWEolEIhkUqSyCKKtq5ZV3S8eselMikUjGI+MidXa8ILvOSiQSSWikZeHAqt40RrF6UyKRSM4FpGXhoCArkVxPI9PVGsqNdAqyFo+1SBKJRDIukMrCwXRXAw/HvgVCB6WIGNciQLqhJBLJ2fHTn/6IAwf24/OZ61nk5JjrWdx22zp++cuf2etZWHz729/lwgvn8P777/Diiy+g6zpCGFxzzfXcddd6du3awbPP/gKAqqrjTJ6cRFRUNOnpU/nxj/99RK5BKgsHvaUfoQh/VaTQ6S39iKjUs1uASSKRnFvodWX4qotxTZ2FNkzP/ze/GbiexQsv9K1nccklgetZWDQ01PPLX/6M3/zm98THJ9DZ2cnDDz9AVlY2l1xyKUuXLgfM9h9f/vIDLFrUf4Gj4UQqCwfOpugCONXRQ9RYCSORSEYdva6MzteeAsNLr+om+oZHh01hnC4tLS34fD66u7uJj4fo6Ggee+z7eDwRYyKPVBYO3Pkr6S35EKH70FH5r0MJ3D63VWZESSQTBF91MRheEAIMH77q4hFXFtZ6FhZut5v//M/fMnNmPqtWXcrtt99Mfn4BCxcu4eqrryEzc9qIyhMOqSwcaKl5lKZdg3FsD/t7sznaO4WSymapLCSSCYJr6ix6VTcYPlBduKbOGvFzhnNDAXzrW/+Xe+/9ez7+eCcff7yDBx/8Xzz++BNceukVIy5XMFJZONDrysivfQvD5SXXVU8Dk2VGlEQygdBS84i+4dFhj1mcCYWF2+nq6uTKK9dw/fU3cf31N/G3v/2F117bOCbKQtZZODBNUB+aAi7F4L5FqrQqJJIJhpaaR8TCG8ZUUQBERkby3HPPUFNTDZjLTR8+XMrMmQVjIo+0LBzUujOJFSoaOgIQnuixFkkikZznBMcsAO644y6uvfYGvvzlr/Doo/+Az+cDYOnS5dx33/1jIaZcz8LJ6zsqqNm5mVujd6EgMFQXk276xzGfYYwGE2WdAyfyms9f5HoWcj2LEaUgK5FYrRcQqAoohk5d8b6xFksikUjGHKksHORlxBObfSE6GrpQ0FEp84ZeFF0ikUgmEjJmEcSsJUv41YYmcvz9oW6fPW+sRZJIJGeAEAJFUQbfcAJyJtEHqSyCyMuI54EHvshnO3ZyqbuOVFcDsj+URHJuoaoauu7D5XKPtSjjEl33oaraae0j3VAhyHE1sPDYi8SVvk7na0+h15WNtUgSieQ0iIqKpa2tBSEmTmB7qAhh0NbWTFRU+GB2KKRlEYKuY5+Nesm/RCIZPmJj42lubqCu7gSqqmAYE0dpqKo6yPUqeDyRxMaensdEKosQRGVfSPMol/xLJJLhQ1EUJk9OASZOurDFSF2vVBYhiMwsGDcl/xKJRDIekDGLMJxoaKekspkTDe1jLYpEIpGMOdKyCEHJ7t3EffRzEjDQa9/nGP+H7DkLxlosiUQiGTOkZRGC5n3v4cJAVcCFQfeh7WMtkkQikYwpUlmEYPKkyIDX8XFjszKVRCKRjBeksgiBmLECHyqGAAMFPT5jrEWSSCSSMUUqixAUnYrn1c6LEYCCIObAn2VhnkQimdBIZRGCublTmOY6iQqoCqjo9JZ+NNZiSSQSyZghs6FCMCtnMkZWBNSZrxVAdLWOqUwSiUQylkjLIgyxk6fYfwtAiZLNBCUSycRFKosw1CfOxyfMILdPqNQnzh9rkSQSiWTMkG6oMBxsS2BP21ryXLUc8aWxqC2B7MF3k0gkkvMSqSzCEBPlpsKXTIUvGYBVUbIvvkQimbhIN1QYKmvbBnwtkUgkEwmpLELQfaKE/NYd5LgaxloUiUQiGRdIZRGEXldGzUvf54LWbTwUt4UcVwOa/y6VVcn0WYlEMjGRyiIIX3UxQvehIHArgjXTOlEUhQ/2V/Pjl/dKhSGRSCYkUlkE4Zo6C0VzgaKiaC5cMXFc7ikiW2tA1w1KKpvHWkSJRCIZdca9svj888+57777Ru18WmoeSVd/GW3qBWjTF5Nf8wbXRe7lobgtzPA0UpCVOGqySCQSyXhhXKfOHj9+nK1bt6Jp2qidU68ro2HLf4Hutd9TFXCjc98ilakZspJbIpFMPMaVsvj1r3/N9u19Cw395je/4Wtf+xoPPvjgqMlQV7yPaJ8XTQEhQPH/iwIJPdXodWVyTW6JRDLhGFfK4v777+f+++8fUxnKvKlciAZCRyVQYfgqPkU/tp+IS76EZ/ZlYyqnRCKRjCbjPmYx2kydPY9fda7lza6FvNN9IQZ+ywKz+yxCp2f7i3J9C4lEMqEYFWXR3t7ODTfcwIkTJ+z3Nm3axHXXXceaNWt46aWXBtz/+eefH2kRbfIy4nnggS/inbWW0qQr+Kx3GuBwRwEIA1918ajJJJFIJGPNiLuh9u/fz2OPPUZFRYX9Xl1dHU8//TSvvvoqHo+HdevWsXTpUvLyhjcWkJQUe0b7NXWcpPCzWnw+gwVRUQGfCUDRNKZcsIjI5LhhkHL8kHyeXc9QkNc8MZho1zwS1zviymLDhg08/vjjPProo/Z7hYWFLFu2jISEBADWrl3L5s2befjhh4f13E1N7RiGGHzDIA4cacTrMxACOoXZQFA4DiMMQUtLJ1rE+dMvKjk5joaG8+d6hoK85onBRLvmM71eVVUGnGCPuLJ48skn+71XX19PcnKy/TolJYWioqKRFmXIzM2dgktTyaSOyyM/B0w3lIVh6NQV72OqzIqSSCQThDEJcBuGgeIYfYUQAa/Hmlk5k/n2nQtZkdyKggiIVwgBAoUyb+rYCimRSCSjyJgoi7S0NBoa+jq6NjQ0kJKSMhaihCUvI568xRejo6H7FYUhzJhFixFNbszEMWslEolkTJTFihUr2LFjBydPnqSrq4stW7awevXqsRBlQLLnLKB95dfZF3MJu3unA2b67GStg0kHNtB7aOuYyieRSCSjxZgU5aWmpvLII4+wfv16vF4vt956K/PmzRsLUQYle84CDhxpYmXHRygExi58Rz+RxXkSiWRCMGrK4r333gt4feONN3LjjTeO1unPmK37qmirOIgSKQJafyiAa8aSsRZPIpFIRgVZwT0AZVWt/H5LKYe9aXbswlBATZhKxKr7pFUhkUgmDOOqN9R4o6SyGcMQVBjJPNO2hnx3HasuzmNKpIE2OXOsxZNIJJJRQyqLASjISsTtUvH5DI7pyUxLjiH64Kv0CJ1ezU30DY/KDrQSiWRCIN1QA5CXEc+371zIpQumoioQ0XwExTCXXBW6T/aHkkgkEwapLAYhLyOepPhIdAPKfFbsQkFHxTV11liLJ5FIJKOCVBZDoCArEUWBCp8ZuzjYm8kJbwK1R0vHWjSJRCIZFaSyGAJ5GfF8aW0BCpCuNTPPc5xsVyNxsjBPIpFMEGSAe4hctiADgKTCNwD/+hZAzyd/kSm0EonkvOe0LIve3l6qq6tHSpZxTVlVKx1dXiInTe57U4DoaqXzvdFbnEkikUjGgkGVxdtvv80TTzxBe3s711xzDTfffDO//e1vR0O2cUNZVSs/fnkvr247yh/rchH0rc0N4CvbwbZX/khZVeuYyimRSCQjxaDK4vnnn+f2229ny5YtLFiwgPfff5+NGzeOhmzjhpLKZny6uRhShTeZYm864FcYAALmnXyL//mfzVJhSCSS85JBlYUQgoKCAgoLC1m9ejWxsbEIcfqrz53LFGQl4tJUVAVQ4IgvzVxe1W9ZKAqoCKartZRUNo+lqBKJRDIiDKosVFXljTfeYPv27axcuZIPPvhgXC1UNBpYxXnz86YghFlvIVCCFkSCLiIoyEocU1klEolkJBhUWXznO99hw4YNfOMb3yA5OZlnn32Wxx57bDRkG1fkZcTT69MBs97ilc6lGDi60CpwW8xuprsaBjyORCKRnIsMmjq7ZMkSXnjhBcDMhvr3f/93pk6dOtJyjUsWF6TwWbnpZtrRk89sVxXzPMex7Syh01v6EVGpeZRVtVJS2UxBViJ5GfFjJrNEIpEMBzIb6jS4bEEG1y7Nsl+3iaiAz/1GRkD21I9f3iuD3hKJ5JxHZkOdJtGRfcbY7t5cdH/swopbqFOyzewpn5k95dMNGfSWSCTnPDIb6jRxBrArfMn8yR+7ANOq6Pnwt2Sd2ot1h4SAmCj3aIspkUgkw8qQs6E+/PDDCZsN5SQvI578zL4YRIzSg+IPcJt3RZB9bCM5/kC3AnR0ecdCVIlEIhk2hpwN9c1vfnNCZ0M5ufXyPDT/nTuqp6EAwbbWlZEHAdA0RabTSiSSc54hZ0NVVVVx7Ngx/vjHP46GXOOavIx4vnP3Yn+202K8hSW4Tx4Fh8GVrJ4C+isRiUQiORcZVFlUVFTw0EMPUV9fj2EYJCYm8vzzz5Obmzsa8o1brHTYwgM19LYv5BaOojk0Q7J2ihxXA8f0ZEoqm2X6rEQiOacZ1A31xBNPcP/997N792727NnDV7/6VX7wgx+MhmzjmrKqVn70h0/Zuq+awroYftF2LU16DGDGLzQEF3mOoCrSDSWRSM59BlUWTU1NfPGLX7Rf33LLLTQ3y1TQkspmdL3PlKjwJVOlTw7YJk1t4e41+dKqkEgk5zyDKgtd12lpabFfnzx5ckQFOlcoyEpE0wKzwgKK9BTIjWhkVXrXgMcpq2rl9R0Vp124d6b7SSQSyZkwaMzinnvu4Y477uDaa69FURTeeOMN7r333tGQbVyTlxHPd+5axH+/cYiapk7ALNJbHnEYFWFmSAkDX3UxWmpeyGNYld4+3cClqXz7zoVDskLOdL/TQbYrkUgkTgZVFnfccQdZWVls374dwzB4/PHHWbFixWjIdk5QMC2B+uZOdAOO6clUJC5jRvMOOwuqsVslI8y+znUydH+l91AG5jPdz8lAymA0lJFEIjm3GNIa3MuXL2f58uX265/+9Kd885vfHDGhzgWcA6qqKiycmQRAR/MewL9Gt4ATxYfoyloZckBuau1GUxUMQ6Bp6pAD4db6GrpunNZ+oWS3lEFycpz9+XAoI4lEcn4xJGURzEsvvTThlYVzQDV0wb7DjQhgYcwp8PRtl9p1lD9seIvbb19r7xcT5ebldw7bimb1/KmsmJsedkAOtgKs9TUKD9QMSdbg/UMpg+ULMu3tz1YZSSSS848zUhYTuTeUhXNAtdbkhv6daCdrHXw1+g0++iSJTYc1fLqBoigIQ5j7GYKk+MgBFUWwFQBmfcf2AzXohuCjg7VhXUWh9h9MGVjK6FyIWcjYikQyOpyRspjIvaEs8jLiufOqmewpqcfrNSg9YWYlmUHuUlT6ll3VBFzQ/D5/0a/wd6gVdi+pwWbuwVZA4YEaPjpYi9dn2Nv4fAYbtx/l5ktm9BswQ1kR1y/PGVQZWBbMWDBUBSBjKxLJ6BFWWWzZsiXk+0IIDMMI+dlEoqyqNcCVpCpgCLPeol6PJ80VmNI6pfcEKyMP81HXTNsSUVWFO6+aOeAAF2wFtHb0BigKMFuKfF7eTOnxvf0GTGt/n88ARbE74DqVQVlVK1uLashMih7zwfZ0FICMrUgko0dYZfHiiy+G3Wnu3LkjIsy5hHOgEoYgdXK0nUL7Qc9s7nDttF1TimIO6LdG7SAxNoKutlbKfGlU6sl2R9pws2mnSygmys1Lb5fan2kqZKXGUVHThiD0gGlZQC9tKcUwBC+/c5jM5NgARfHjl/ei6waKqrBqbvqA8ZOR5nQUgIytSCSjxxkpC0n/gSrNoSx29OQzLTmWi9rfw62YVoDVmfZK31ZEFBgoPNN+DQVZiwedTVtWwOs7KjAMYR9v1TwzMP7jl/f2sxycdHR5MYQIqVCcgzO6YOu+6n4xkNGMC5yOAjiXYiung4zDSMYjZxSzkPQfqAAOHG3CpwtcmkLknMv5y/vt3B5tWhjOMI+qgCIED8ZsQT/Uw56Y1f1m00C/ASN4ILUsgIEsh1D7OQdgp5vKSlvw+czYSF5G/KjHBU5XAYxlbGUkkHEYyXhFKouzIHigevSuRfYgV1LZTGFPPnPdx5jtrrEVhjOPLELR4eh7XOz5lLrI2RR2z0TTVGKi3CEHjHApswNZDpac4QZg67O9ZY1s2XUM3TBl3H6ghhVz08ckLnC+KYDTQcZhJOMVqSyGEadrJybKjaoqPN9+Nd+d9BdStDZ7u4BYhoCInhZujdpBXnwvxoK/o6PLO+CA8dHBWny6YbuLhuK6GWgAzsuIZ/mCTLq6vGzdVw2AYQhbuWiqgk8XqKrp5np9R4V0kYThbF1IMg4jGa+EVRbV1dVMnTo15Gfbtm1j9erVIybUuYrlQvD6zAypi2elsLu4nsO+dFtZWO4oM4UW+z0hYH7PHvSde6mbdAGqshjdb4444xBnmgo7FFbMTeejg7X9BirbPaULfr+lFCGEdJGEYDhcSOdrHEZy7hO26+xDDz1k//31r3894LOnn3565CQ6hympbLbTWg1DsLu4njUXTWOPNxedPgXhjGE4/1UAl2KQ0XaQ+6O32Md5+Z3DdndZa+apKoE1GnkZ8Vy/POesBhdroPri6hn2QBfcit0wRL/YisQklCI/E4bju5RIhpuwloWzSvv48eNhP5P0UZCViOrv9QRgCEFlfRvlvmR+0XYtN0TuYbqrAU2xTIpA68Jpdcx21/Bw3GZe61pMpWO1vYFmnuFcIKfjGgl2VxVkJdqWj5OzcZE45YH+gfxzFelCkpzPhFUWzirt4IptWcEdmryMeO5Zk8/v3yrBEKCpCosLUig93soxXzK/bL+G5RGl3OHPkEIBr6HgVkXAYGwNznmuer4e9xbPdpopts7zDLVT7GCuEWvgXjYvg6SY/mm3eRnxXHNxFm/uqrTfWzhzCtcuyw6rkE40tLOnpJ7FBSlctiCw5+7WfVV25paqKSiAbpwfbi3pQpKczwzJspCcGQLITI61B5Cj1afYcdj8bL7nGN0p83npWBpLtGKuiihistrZzz2lYbAi+ZSdAWUNQL2HtuI7+gmuGUuonLSQjduP2umvzqD4QNk1TkWyqbCCb60LPVjfdnkeyYlRAQrAWnwpJspNZW2b3adKUcAq8P+s3HTDWAqjrKqV3/sVBRDg3hooZfhsGc26hYmcyTVekHUqI0NYZWEYBq2trQgh0HXd/hvM1fMk/bEHQ/8YaBiCwgM1JMVHEhPlJj7Gg6bCzp58PvYV8J0Vi7hqSgObPzbY0ZPPiohSbo/ZaR9P+P83s203k1pK6T5qUJs/j0l6C/qxTwHwVR1kW1cJn3XNBPzrfztcIE7XiKIqNLV2U1bV2k+R+HwDp2letiAjYNC3AvnBBM8x9pTU2/uVVDbbigLMehPV0aI9XMrw2SDrFiYWg7Xfl5w5YZVFaWkpy5YtsxXE0qVL7c9Gww21e/duXnnlFYQQXHzxxdx2220jfs6zJXgwVDDrFXx6fytNAU40tPPW7uP2AFvYk09+opcFvXtQ6Kv6jlN7iFPrzY2Ovkewqp7jqmA7M1GAC3ISWVyQQuGBGgr9tRJWbcb2AzV8sL86ZMqty9WnYAabmVlKJhSq2mdZACwuSLH/LshKxO0yCwAVVeGeNflkJscG1KaEK06MiXLT0eUNK1M4mc/VugU5Oz4zBmu/LzlzwiqL4uLi0ZSjH6dOneKf//mf8Xg8fO1rXzsnlEXwYDg/N4m9hxtDbqsbgj0l9QHKBeC3tReixdYz13PcVBhBelk4/rU+OujLsbOjFhek8NLbpbaLZ/uBGh69axGArbSsh6ggK5GVc9IAuH5VLkkx7iHNxENVfSsKLMgzYxnhYhahfPrONcSDA8SWpWFZMArgcoWOu4STeaCgc3HFSXYWVY27AXmkrKHx1DBypJBJBiPHgEV5lgvK5XLR3t5OYWEhBQUFZGdnD7sgv/71r9m+fbv9+je/+Q1CCH7yk5+wfv36YT/fSBCqBcj+I039FIJFVkocpcdbA9w5Ani3ew4XuI+jASh9SsH/0v5XAFpaPtfEJ5HndTN19rx+qa4+XfDmzmPsL+tTWlZxnXNAun5VLkA/11So1ufO6+zs9rFl93F0Q7D/SBNzc5Nsl5UV13AOxsHdboMHRWeFemVtW4AFE65CPVQbd6dCChV0Lqtq5Sd/NBXRUAbk0Zzpn601FEpWZ8NI7Tx2x8kkg5EjrLIoKyvjgQce4Hvf+x7Lly+3Z/bt7e3827/9GytXrhxWQe6//37uv/9++/WpU6f44Q9/yF133XVOdbkNDnDesyafF98q6efLFwLe3FVJYqyH5vbegM8q/Km290Z/QKKr07Yi7JYh9nQejNrDxNWWslB1cTLtYZpaI9FUsMZYTYV9ZY0B55+WEmsPxNaAdOBII5fNS+9nNYRrfe5sbqhbqcKG4KUtpWQmxwIELDsbqpttqEGxICvRrlBXVQVNVdB1s5WJMx7jHBCD4zIfFtX4M6wUHr1rUcigc0lls3mNYQbk4PTewa5lODmb2XG4xbKcCRCcQ+64M2E0kgys38dg7tHzibDK4qmnnuIf/uEfuPzyy/nzn/8MwOuvv05dXR2PPPLIsCuLYP7lX/6F2tpafvvb35Kenn7OLuN62YIMGpq72PxxZf/BHvopirhoF53dOsd8yfzg1K3cGLWHKyI/66cwwJxpC4T5vuGj+P03+KBzmb0meHyMh9aO3n6usPKaNirr2wPW/46L9vC7zabr0VrU6fPy5rCzeYtQtSVWrKFPEYTuZhtqUAxu/b5q/lQ7QcB6KE80tNvpt5ZbyplxZl2vTxd2Q8RQcrv8LsPgATl4wF05J23QaxlOzqSWxsBz45wAACAASURBVGKwxbKCEyDON0bDAgxO8AjnHj3fCKssampquOmmmwDYtWsXV155Jaqqkp6eTnt7+5BP0N7ezrp163juuefIzDQDTZs2beLZZ5/F5/Nx7733cvfdd/fb76mnnjrdaxmXlFW18vYnZhBbVWDtxVm24ghFW6cPTVPISTfXqdjUtZiD3izy3LWkqi0s9pQD9AXAHcELw18druuCXp/Oirnp/ZoO5rgayHPVcsSXRvac+fZA/PxfishU6rnIc4Tuw5CRvIgSLcJWJuEGF6u25KUtpRj+NiDWtsFxjWClE2pQdDeX0x51kNLeVE6Q2m8GH5x+63O0PMnLiLcVnsXknip69pZR687kYFtCwDrmT/7vlSFjFsED7lCuZbg5nVoaJ8EKGLBdeQqwYGYy1y7NGleD2nAN8KOV+Rac4DHYhOp8IayyUNW+TiB79+7lscces1/39PQM6eD79+/nscceo6Kiwn6vrq6Op59+mldffRWPx8O6detYunQpeXl5ZyD+wCQlxZ7xvsOVbvfKB0ftwLIhoLmjl7+7LI8/v18Wdh9DF6QmxVDV2IHPZ1ClpHLd9VdxtKqFz1s+5cK6NwLqYIQAQ9HY3Ztrv/dZeTPFxz7lC5fmommmKyfH1cBDcVvQ0NHRKO1RmaV6KOtIZZpSz0Nxb+HCfAh8rWUc0daSf/FFXLFkGrNyJgfIWFxxkgNHGpmbO4W5M1NY09oNKAHb/mtCNH9+/zAff15n9pNyqSyblxFwb5OT4+xsle4TJSR9/B9cG+nlmmgX+lXfRCREs7Wohrm5U5iVM5mtRTWIgPRbJeCY16/KNWs+dMEMTwOrGt6hu85HrFD5tG0NmwpTefJ/r2RWzmSSk+l3XQDL5mWwqbACn8/MErt+VS7Xr8rlvU8qeWf3cTt7LPhaRpqtRTXmmu9+JXaiqbNfpk9ychz/mhBtfzcAhZ/V2tdy19pZIa/5dHB+98NxrJ/8ca8tn/XdnAkD3Z/h/J6s34fXa9ju0bH4PQzESMgRVlnEx8dTXFxMe3s7DQ0NXHTRRQB8+umnpKamDungGzZs4PHHH+fRRx+13yssLGTZsmUkJCQAsHbtWjZv3szDDz98NtcRkqam9rDB5YFITo6joaFt8A2HQFdXoJvp489q+cd7FrP+mgLe3n2cXq9O0iRzdl90pNFuEf7JoTruujrfdr0AvLP7OD49ieWepdwavdNu7GUAdZEzUE4Fnls3BH/94Ah3r8mnsraN/NZjuFsN0yoROrNr34BaQb6ikh2j4sJwrBtuMEOrpaurl51FVTS3dPYLlnp95vaqothWxcK8JGoPHsNXXUy7O5M9xS1mtbaqsO7KmSTFuMPe2+qdhcT6vCgIFKHTcewgP/hTbcBMMTMp2nYfKarC3WvyA46ZFOO2W8XnNtcijnvRFLO4MddVy7GeZHYWVZEU4w74noNnt99a12fxWJXtt12ay8K8KQHvD9fvZChkJkWb1oLfashMimbHvhP9ZuVJMW4um5du7+e8llk5k89K5uGeve8sqsLr60uosL6bMyHU/WloaBvW5xnM+2vdU6d7dCi/h+GwogY7xpler6oqA06wwyqLb3zjG9x33320t7fzrW99i+joaP7rv/6L5557jmeeeWZIJ3/yySf7vVdfX09ycrL9OiUlhaKioiEd71xkxdx0PthfbbudBH0Vyo2t3fh0g1OdXr5950LiYzwBLcI7urxcvzwHgNd3VNiukRilJyCtVgXSuw/zUNwRPu7J4+PeXCp85j02hHmc9dfMQq9z0fG37QhD97uxhH+NDYNI1bQo7F5VQJbWyNbP9/OBNzlgYKg+VMSlrv1EunvJ1E6y35vNjp58dN2g+lARqcdeROheJqGRydWUk4wwBJW1fT/g4B98WVUrGz41eDBaRcNAVTXKvKn4dG+A/z0pPpI7r5ppH8sKpjuxXDjb3jlOMhoIAx2VMl9aSJfaQAOgFX8Jlc11tgz20Ad/HirbbigD93DKPNx1K8OZ6jqamVBnck+Du1Lfsya/XzucoR5jLIpMwyqLBQsWsG3bNrq7u5k0aRIACxcu5JVXXiEnJ+eMT2gYRkBRnxDivO41lZcRz5fWFvTz6Yd66LLS4sygsxAoqsLR6lP8bnMxK+amB6wrUeZLA0Wxg9v4g94uDFZElrIs8gg7e3LZ3ZPLCVLtB1BLzaM8Zi45bftQ/YF2QwQ2MbQQwFzPcWa5q3mmbY3dzHC6q4H55S8wP0q303hnuc24yCf6LPLcdQjdtA5U4WOJ5wjlvuSABZWg/yBXUtnMkd4pPKOvYaa7lowLFzF19jxc+/rWB7daiqhqX0+pgQLNU2fP4/kDa5mu1nBETyNnznzuCpHFFOq7CBVEH86Hcig9u8ItgGVt55xAhEoZHgmGu45hqAP8UGfkZ6sYRzJAHtyV2socPJ2U7YEKV63vYqRqacIqi5aWln5/z5gxw35tuZFOl7S0ND755BP7dUNDAykpKQPsce5z2YKMgEpl60sMLkB7+Z3Ddn8lYQg7q2f7gRruujofwz/tL/cl8373BWaWVFCGlQJo6KyIKGVZ5FFalz9MtsN99LeadB6KLUIRRt8Ofpyr+Vlvu9CZ6a6lyq90Tu78NRGGbp/T+ndNwjEuX3Mnqa4GTpW8iSZMZbI0oozdfktHN0JlSvWlzLo0FQXTrZWdFke2YyBpau22LTTDn0oLA89u8zLiuf32tZRUNrNugIc/VDFgqCA6DF/fqsFm6EOZwQenDFvKdLgL+QaybobjHIMN8KM1mx7p84TLHBxIQQbLE65w1UrrVsBOShlu+cMqi2XLltkz/uCmgoqicOjQoTM64YoVK/jFL37ByZMniYqKYsuWLTzxxBNndKxzieAHIvihs/L+wb/mhWNfn25Ve/e912l4EGBbCHZqLX2ZUorQcTeV0b2rFF/5Hro9eZR7c9nVk8eKiFLUIEXhxD6OAnOnRXPR0oVMdzXQXncg5PXFRRjEntpLXX0jdUoWM0S52ftJCPJctVT4klEVJSBTKnh2euNMnZX1b6MJH8qO/fRqXyJv9mW2m8pamEkJeigGmt1OdzUwLbIYl2sWEPrBCfVdBMe6Ort9ZzSQbN1XFbKafbAZeqhBIVSBYyhlOhzuobKqVrtFjFMBwei3lB+tli2Dned0rI5Q2w6UOThUeYIXOnNu45xEjUQtTVhl8YUvfIG9e/dyxRVXcMsttwxbtlJqaiqPPPII69evx+v1cuuttzJv3rxhOfa5hlOBnGhoD1AQqoLdkNClma3Oiytb7OrsI740dDSE0FHxWwJKn8JAgI5CcdFnLIkw022zqOOemGp6hBuB4ncBmucQ1n/CYWz4j5fduAP3UQ9eCIiVQJ+ScbXX0f3hC8QIyEZFRzU7APhjBao/GG1dbyjf+2XugyiRPvP4Qqdn+4vU6Al8cCISMOs/nAH/geoQYqLctB8rZmX9y2hCp1dzE33Do2ip/X/HoR5st0u1XQZCwFu7j/ezNAZ7ELfuq+J3m0uA/h14B5uhOz+3rM5Qisr6DTmVqVP5DGWACxU/Cm4U6azZOJuZ95m4eUarhcdA5wlX7BjuNxhuYhHOyxCKmCi3mTxC4KQoeOLptC6HOok6ExQxQC/yrq4utmzZwl//+lc6Ozu56aabuPHGG+0YxnhnPGRDDZXXd1Tw5w+O2q8XzpxiF9XFx3hsX3/hgRqqGzsoPdFKjquBddEfkeY6FdASxBr0dVTajUgS/K3P+/pKKRiAZsU8/Pu0EE+8aLUVgtMdRdDxwVHnEbS9IeBA7zR6Jk1j0oy51LvSAx4Mva4MX3UxSmQsorudXU3xfLCvmiWeIyyPOIyq+AsNUXijeyFbOucABFRkhyJ4kLsq8gDXRe1FU8yMsY6CG5h66a329k0dXl7/8EhI901ZVSsbtx+1CxODCV7TI9Qg+NP/2WsrCYALpyfyzTsWhpU93ADy+o4KXt121K7V+eLqGXbig3Pf4GriUIPW8gWZAb/tUNuUVDbb57Nwu8ziRMt6URVY7S+YHEh5h/p+zkTZnMms3rof4dZqGWif4PMEfwer508Nqzh/t7nYTlQJ9X0N5doGCoaHUu7OideJps4zilmccTYUQFRUFDfffDM333wztbW1bNy4kfXr15OTk8PPfvaz0xJEMjAFWYl2PQTAgaNNXL1kmt26YltRDfesyWf9NbN44re7AbMtSI/whDye6Z4yaNRjSVA77eI9KwtKpc+asNxW7pg4aG/tF+wOxqlggntXWZbJhZ4qTi2+hew5C+zPjh3cR/eh7aS37EURun20+YrK3DgDFYcZjYKhaJT29qVp+/RAH2+44J9Fh4gwr9Mv0+Z9TcyIr6Kjy0tMlJs/vnuYXm/g7Nk6fl5GPDdfMoPS42aAHUUJmHjsPdzI/iNNdufcUIPg4oKUAGXh7MDrxLkgVKhg+unOeJ2ZXIN1YA3XcsU5W7XamwABrsBQCQcDKYGzcScNNXAdqrp6U2EF666caX/vwcpgKMkGTa3dAR0PoH/czRq4P3QUwqqqMuAMP9y5A37L/ozGUNs7rW1L5uAJwXAxoLJwcvLkSU6ePElzczNJSUnDLshEJy8jnlVz0+0ZiU8XAZXeVvYEwLG6vh/Czt48st39O9ta8YwuIvikdzo5rkYSo8DV22Z/bg/6/sqiNxun8cXIEyhWhlQYWZ0KxolC35saBmneE4CpLKp2vEF80QYSHBubxxEgdDSC+14pdE9fTf7ndYCpGF1a34O3dV+VvSKhqsIXCgR57jpyPS6O9ExBYKYYW/fBEBAlum1/saIo/WJxwQNxSFeQo4rb+k7m5SbZg5Nz4MhMjmXhzCm0tPewav7UkGmS4SrSB4pvOWeTTf7061AD8FDcN6FiIyWVzSEHISBMjKTvPg7WGmak3Umhqqu9XoOXtpTaPcwUQNX6lOBAGUZWo0xrZcfV86f2U5zOa3HGuxTgkkF6iIVToOHulXN7n6/vus40Ffd0GFBZ1NTU8Le//Y2NGzeiaRo33XQTGzZsGHJRnuT0yEoLrLoMdhDqhmDzrmMIR6C7O2sFnszJePe/Ye5DoBKY6zmOgYZvSh6uxhJ7P6cLyUDhDf0SPuqajm4IbnMU/BF0PMUfFBEDKBNzJ5UDVd0ku/eR0ryf2EPv97m3+vxhdgWsvZt9MoOYo+9xXSRcE6nwvnYpiYuvtgdKZ3PGLLWBFbVvoWHwUKzG/rn3UaelU7ynAR21r85CT8Mw/NaLMB8wgQiYPQ8U0MxMjqXwQA3b9lfb8STdECE7+v5uc3GAeytUPQiEWhAq9EzUObN2zi6tZouh/NRDyVzKy4i3e4FlpcSFjY0Ey2HNoPXgtVoU8/qdOO/jSNdBWIOspdTtolHHPRYE9vi686qZA7bGtzAMQVJ8ZMi4WzgFbSmWweQNVgrhvjvn9ihKyCaeI1VFHlZZfOlLX6K8vJzrrruOn/zkJ1xwwQUjIoCkj44ub1+AGnMNbz0o5lLf3A346yo0lWuXZROZMQ93ziK63n0O0R5oZViptJpDUTgRwCudS9nRMx2AHT351IlE7prehKg5RLLW1uducmReHZ+ykunUYDRVYGovhcbJ8+htaSDdqAbDIK/qNUSVilcxbGvFPq8w4wjO4LxTJuulgplRdYVvK0cLD3HMdSclbQkBivQizxG7+lwIncWdH1LWFk16RJNpuQAqBuvSjvBKvcLR3ilomsoDX5hLbUNbyIFroDqHrLQ4tr+7nVxXLWW+NCq8ybasc2ck8fI7h0MGh0MNkgVZgWugOJMAwuGcXTqbLYYLlDvf6z5RQtfHb6MAdYnz+eBEpK3UDh1rQfiV6WBuomBLGLBTvl9+57BdPxDqPg7kvz9bgq3Bji4vaclx/OqvBwKsQgtdN+jo8obMMAomWJGHco2dTmqxpUSdRabBxwrlcnVen9MqtVJxR2qxp7DKYvfu3URERPDKK6/wpz/9yX7fKqL79NNPR0SgiUxBltkJ1ZplXLU4014rIriW4oKcxIB1Jsp9yVSn3My89t+g+jd0ZjpZ+zkfFuuYMUpgr69KPYXtUfP5oG0W2VoDX4reRpLWYY/gR70pvNW5iKlTYrg0v5s07wmOHa+j5/jnTHM12cdWBZh2iyO4LvrOG+AGcxJksljWTK5Whyj8Gfkx+Ux35VLuSyYUem0p0wXgCbzO1I5SHo45TOnM60lYeDULp3TQ2HYwZFrtQP71VeldLIp/BwwfQlHZ2ZPLx925HDPMmETwQGP596cp9bTvrsN9xWqy5ywIGCxOp811qNlrqP2Cg7Zz4lpI3PkLMMx4UZz4gIq2tfh0/320rC1/UahzCd5Qx81Ki8Pt6pvlOhWNpRyD3WThlKaVqgucVft3va6MafXFTJ8xCy01BzATVuKjXPa9sNeM14VtCYXKMHIqF2c232DB9qHEV4KtQ4RppVqLlYXLrgrVIn+oqbhnS1hl8e67747YSSWhCZnv7+gTovkfZE1TAxRFWVUrT/3hU3y6YIb7Gm6J2kmG1mwPzLbLyT/oduoakZph+vJVjXIjLWBAt2YqLk2lUk/m912reTh2M6oQGChs6l5MxYlWSk+0sv2AwuOXqKTVfmj/miwl5bRELIKVXnBxnxNnplWfewpS20t5KK6MX7atRQHilC7ro75aE6XvGM627iAoqH6NiNwpVG/+Peg+elUX0Tf+o51WGyqg6XwIfdXFKMIH/h5Wyz2lXOwu5YRvCruO56GpBRhGn3trck8VWuUulnrKUDEQhUXsbv57fr2rJyBQafnKh5LiOtjsNTjQO93VQFb0bhI1va//F4ZdAwOmUltz0TS6enz2ErwnDhVx3yKV1FkL0FLz+g1c82aYrfCz0kwXVqjgt3UfwxUOllW18qM/fNpvdcfTVRh6XRmdrz0FhpdeRcNdsAp3/kqKO7L73austDg7qcBpCVn33zl7VxuPkueuIzW9a0gZXcHZSaG+p8DJSN8PP1Rb/eBtnS3yTycV92wJqywyMkYuUCIJz0A51NaDGTzzKnSs833Um8zvJ93Bra6t5PYW90ttBdjUcxG1eiI3z+wlPncu005Ekuwz+ORQnT04rnAE/zq7s/jFHsiz3C6OGb1PF1D2Ud85RGDbdDtbynH+YHeUErwtfQO+fbygfTVhcFHEEZZHHEFFN1ueEHhcgl47z9t7YAvoPvOF4aNn3xt8nHoLH+6vprKuHcM/y3YGNK2iuOlTZ9GrukHvtcRGA7JdjWS7GsnPSKAtcxkFWYlkndpL9/aXEB7DVo6G8FH26W4ySSU3wmwXv/3dOnJdtWzYmc7tt68NOQBZE4KVkaXcnNXC1bOX4cnIIRSFB2psRWF2G+7rKGzdGx2VI740VH9qnDAE7+w5wco5aeiGIFtr4MHoLbhLDDrLthB9w6OUVLoCBq69hxtxu1R7rffg4LfTTRZcOOi0PoIHzDMpKPNVF4Pu9V+MD++h9+kt/oDt3RfQabgD7m1lbZvt4g3lcrP+rj5UxIKK34Hho610M8Upd+LTlQEL94IrqkNliTmtQ0H/36vzeNbERQ/TuWComWJny5CzoSSjR/AM0qqk3VfWiGsIQbOapk6eUZfy/8WYg5c9a1fgvd457OrNR9NUTmbP5JdvluLTBW6XGtDl1prxNbV2c6yujQpfcoCSsNBUBc+kyShdVUDo+oyAB8FSGiLQYrANKKfCcFgc9iH6DC2Wug+jWos/ObZXQhzH2sd+Lyi24zu2j20HUgOuUfgDmoA989U0hceuTSQ5cw563WHo7vM1W7LO6d3L5OXrzJnu9hdRhGHfF8v9lqo2syZuPxq6HVVRMNAp4rND6eRlrAqQr/BADZlKPVfEHGSe5zg0Qs+HxTS0dLKPC/qlglZ+vp+rImuJVHqZ765Ec3QUNgR0GW7K065m0byLybYGcQLX75jprkXDbNti6F7qivdRkHd12LU9rFhE4YGaAKvMmtwEV+EHWh99qzs6s95CEc4NpETGAkGjrjC43HMQAfj89xbmhUxvDbYINmx4i6s8exEur9ntwNBpP/YZmjo3bOFbSWUzmdTZkwDLVer1GQEWQ7D18tLbpRi6QFH7rhECV2hcMHMKB442jVjR3WBIZTHOCFWMkxQfac8qQqVWrpib3i8zRRiCXb15ZLsabUXhmX8dczLX4vY/EE6LxOszqKxtY/01s2w5nK6BcMzLTeJQzMUs4DN/qV+QoiBQIXS74vEYXaj+WXlfRhZ82jud3CQV1/TF1H+6lVxXvXm8oNgLmIFxq3jPeS6cCsOxU2BRIuDrswrM9wyuiDjIb3yX28fTNJU5cS3UbH+faUo0FSQzTalnUuHv0dEJwGEduTvqOHZwH61HDpAj9AAZLYUyz12JW9EdcvoHc2GQ567rd59z2vZxU9zmgMC/AE7ufY9XT0UG1GdUHyriqzHmuiWqY2On+y9a9XJBw5vErlhEuS89IA10xdx0Ls3spvuQhtKsYggDHYUXPjVYEd/OyjlptHb0BgxcwRlgTqss1AAZ3OsrJz2OhNiIkJazk4HcQI31jcQS+Puz7pOqmNZonruOvSHSWyFwYF46pZ0Ho9/qU5bCtMQO96Zyydz0sAkFc+JaWOZYM+aZtjX2BOTDohqy0uICJmTW/lamneX+++hgbcAKjcIQzJg6iWuX9XepjRZSWYwzSir7d6Zcc9G0gABxcGpiXkY837lrkf1jsx7gvMtvpPTEFHJ6y5g0exme2ZeR5zhPdWNHwHFaO/rW3igMlRbpQFHMGdmBo03sMwTbPddw/4zjRNUdJHh257QsvLPXkjQjn46//RBD6KDAZ94s3u26kBOk8u1LF3Kwspk9XW18PW4zWlCw3lIGoY5f7E03O+D6laMCGIo5Yw+QyGmxWPEVYJ7nOMt9pezoySc9KZovzBJM3vUfJOheZsapPNO2hjxXren26juUvb/1rwDY9isicYHWZ+XYnyng8bvObHebX5mpqhkfcNJ7aCsLmjfb98G2tARkak1kuxo45u8KnJcRT567zsyAcyQSBMeNLOl7Sz8ib9W9ATGQrFN76Sn8HYK+QL2CYKFWxofvCMp9yaiqwtqLphEd6SImys3297YzQ60lU0mjQiTbVlmogLDTyvD6LZTymjbcro5Bq7rDJR70tbjX/FaUYhemQt9g703KoyAxkVxPI9PVGsqNdFbMXUz1oSIuc+/nMKabNbr1KFqUeQ91AU1GHO93m7/ROx3KrKyqlZa9b9vPWJq3nR7F73J0xISsFSq3v1tPuS+5n6KzCvF0Q9jXBv17qI2WyykU2ve///3vj8mZR4Gurt6wvsCBiImJoLOzd/ANRwBVVfjoYG2A3B63SkOLP2UW84eVPy2BsqpWdnxWi6oq5GXEMz9vCrNzJpOcEMVNl0xnUX4yU/MvIHr2JWjJOZRVtfJaYQUvv3uYzyuaOdkWmAU1KyuBmCg3rxVW8GFRdYAMmmqmDgph/r16/lSyU+Mor21DCGgxopl0wUryC3LQK4uwhicFFT0qgQ4lhkMJlxI15wpajBgO+9KJS8mgs+A69roWkpCSxrorZwJQWtnCwTpBSW860WoPKa5TAW6r4PiD9XeS1sErnctIi4XYSA33BVcQe/N3aez1UFddxySl097e6bKy/kOBaZ5WPuy5gKTeauY2v0O80uZYZEohRTtFquZfZUoEnt/p5opSvMSqPX3ns4QN2sf6Um0ZVHPFQ29EApMnRaLXldH99jNYa484seTXUfncl8mSWSn4dMGxug7SWv3fQYh9LDnNa1bRJmcwJX0qEa0VePe+RlTJm0Bg+3sFyHI1sTiinMO+dJr1aMprTrF4Vgplez9lvfYaBe4aLoo4SqkvnXY1jpsumc7Jth5+/PJePqs4yc7P6pidncjkSZFMnhRJXLSborImWzYhBCkJUeRPC9/RWlUVdn5WB/5Ej5sumc7kSZHs+KyWj470cNiXTpMRy/7ebFKnptMakc6bTVlU+JLZ0j2flshMtJPl3Nj1F/Jd1VwcWYEREUNm6R+ZqVWzxH99J41YLoo4alqvCsSoXmZH1JKz4CIq21yoqsLJth62/emPrOp6D62zEV/lPqqMJCLaToD/e3m7Zz7xaicPxW2hwF3NEs9R4pVO2nUPnvikgGsNvrZ1V84kOy0Or26w5uIsFuWHzv5zUlbVyk7/yoiTJ0UOun3gb0MhOjp0RwiQlsW4Iy+jf2fKxQUplB5vDZhhDGW9AyehmsNB31iiaQpZaXEht5meHkd2ahwf7Ddz6oXANsODq1g9GTlokzPpLf3IbCGSv5JyX7IZnK0XaOV7UPzFRKqahEIzPt10W0RFuHhnzwnbFZAwfTZHY+aT0Pg3sjo+7xvggwZn631VCLJcJzGu/jaxjnuwjwt4tS2Sxyb9mSTVtKbCBdnjaePOqG0s8pTbSkL4p/HLIg6j+c0Be8B1Ki+npeD4LDjby/n3Id9UctMnEdFYAggMXUc9tIWu0o1UTkkmJT0NIfQBrAPIddWRrTXw4lumQs9S65kTJ3ApoFh7OmIm1r1DAb3hKG0bf0hrQj6TmktIsFyJQUrGek8TfbNl3W/5/l3kAVwRphtNEQbr4vcQsfwusjPi+6254XShVta2BV4TDOqHH6hYTVUV2+XzUNwWtHodRVXR1KW81z0XRVU4fqAGt6eIeZFmHEIYOjWfbifP5UNTTGsg31PHYX+bmT7FKlCFwdG9u9nSNQeXZvbKmuOqCPhOW44f4X+61pDrX+c+PSmaS/gEt1f31wwZrIgo5eKII7THFQx4bYBdJFl63IxhDJRibT3j1vM4ai3KJWNHqHS44NcDPYShcLq3nKgqrJo3letX5bKzqKpfjYDbpXLnVflA//YG4dtQuCiY8QVbnsLNxXZsxDy8+bezpbJhmO1NrAHX0AVFR5owhOBD9SLujOxiiae8f5aTPUU2/12c5mNK0H2wMk/e7Z7L7dE779ibWQAAIABJREFU+7mkwDGgA0s85ebhHEpJ9f/ldIcFHEaAjpkVJYKPKcL8Dcz21NI5aRZuf9GkQJgBbIDmRnpaDiGE4v8EavVEpmrNfe44BdK0Vv5P3Jvs6Z1OBD6yXE2mKwYwcMR1hJWNZd8u/wCmk9h8yL5my81lX6oVVxKmdTVZ7WC6q4FjRkrIRp1poh5lx8/p1dZTkLVwyC0+rsruZVr9h/SeMhtMuqbOCtklOFwx3D1r8nlxcwl5rlqHG87gluiPyb3wAva3xrP3cCOHvWmsjexzT/YYGgYqCIFQNUjJJ6+uBNV81y4gFQq06Z4AN9FBXw6zXDX2PUpQOpjjrqRbeEjTmvlirxm/cP5GVcCNCGiHE+ranM+41drDmkCGUgROF92otiiXjC3BD0Tw69PtsxMc57CwrIRZOZNpbukM2UBuoPYG1r8llebqclb1stOn7YyFAPY6GopqFnIZQYOr9VxZqY3CEJyYeQdVTUeZ0rSPOKWLNhHFCX0y18ceJFa0m8cDokL8ovuUWg5tZBFZuYv6plNoGDQYk5gW00OCt94+RkAmlf/fcGm81usi7zTajCh7nZCADKwBXEFCGEQeec8+n+rYxt4eQZF3Goe8Gdwavat/saViFkBe5FdyFtYgpwsNj6oHWjf2+ftfS6Cw0BGTiZKaj15TQlx3LSsiS1keWUZl9o388kAin/TmsjSizM64sq6re/vv+CSumynxCaRNjg7o0gtmYsZ2f5LFDHcD15zaQs9u3ZYvuP4FwndotWsiVCjzpRHQgl/ozK36M00tGSRFeugQEY57YDDHcxwdlZ29M9ndk0t5k4ccVxprIs0lea3vSsXgi9G7qdETOWakkJUWR+bcdewrjCetYQdpWqv5n6vVb8UpZraeYn4P2pQZGCePgWFaPEpkLD17X7O7Lwcrx+DWHlarmnCTw3AtzYcLqSzOUawBsNCRAhiM88EK1UrEKvAbrB+N8zjBrRqc7jBwzEINwZu7Ks2HTDNTIw0DFBXm506xs16s5UttxSAc7h3/OVTVdJFVMoNXqyJxGj8KmNaCf5Byz1od9n6Z15PD61zAqxVH7XN9cZrg0roXA47pxFkH4tWicOtdAZbFychp/Lb1Cqap9ayIKA2pIJwDsXPWDv6WJ843nArJ/zJFbSXe0xXQVj7geEGxHOt8GqApfcH0UATvG1DQCBw66WZR5zZU4bNlUjDIObaJ72YVcKhOZ2v3bK6M/MwWWsFcQtmoLaGmey41TZ3MzU3qNwF69K5FFH/yCdlVu1GFQ6EpIAwfx977E8bqr4ZtHQLYrlN7XxVqfAlkupvte631tHBlZItptaI42uBbkxdBsxFDuSMY/WrnRcQoPUxWO1gWUWr+vhSDme468ELdtgNMv2I1q29bR+OrFdDY2ndMLFuw795qGbMwTlaaA75hoH/4Iip92XLByjE4vdYqegxWBHpdGXXF+yj81EA3pqCpCndeNXPYA+FSWZzjWD31g9eiDtXK2NlKJFyLiWALZrCK1ZLK5pA9dywEpmWwev5UgH71IpbL7eV3Simv6euI65wBO/stKQrkZ8bj1Q0qatoo7DFdZFenNpC+cBWe2ZcNes8s/7aVjvy3UpXlmVPxtFX329Y5+xYKuH1mtbitzBSFyow1GLVeKnzJnNRjSHKZcZEABaFAbeRM4qdOI6Z2L3Q29x3feb3+Kb9tNfj/l+Y6FVbh2P86Bv1QiQC28nX8HRxfCTig/88lnnKE4bCuLOsBg8SWQ6zwmDNnJeg4CtizeIA9JfX9uqJOdzUwpe4PKFpQKrJf4KRTJTyz4S3uvjqfziMHyMRDuUjG5zPYuP0oyfFRAYtU3RC1hysjPwsIzjvvs2n1CQQqujDdTAZmJ4N2I4LboneyNMJfaa+6KJv193T1+FArykHoGKi0Gx6+5k+PZUcRvdo9RMdEoztKdwK+Tz++8j0Ioy92EXD/AQwf3tKPAqyL4PRaS3FY1f7TXQ10bPoRMYaPB6NVntHXUKkn2y3NhxOpLM5hBupfFNzKuLK27Yw6fg50jrKqVo5WnwqrKMA/o/IrhuDUQGcFanZqnK0soM9VpWkq8TGegAHhSPUp7l6Tz4kGc6a1y1eAZ/JlrJiUTn8Pd3/yMgKb4BmGoNurEJwHEmwBBA/KKOCZfy1TM+fh2mcGFt/3zuNW1w77npwyIqnQk3m/ew4nTqXy7SsXkhAh8B56P+AeWecLOLn1j3+kc7q3wsqoBCoCCFQCiqqBYaAEfWvBg1uw+y1AaVouHmt7xeoD1t+6Wew+yg6/Qve4tH69pnpLPzJn10p/GRT/wL5QK2NS4VskovNQnMozbWsp9yXzeXkzqtaCAmS7Grgi4qAd7wm4dqfCFKayfKdrNlGKlzili3YRRXT6DP7OeBuXf6Zvnlsn4mQZiUtuImbud/BVF1PrziTuvW2OmIi5oiPW+iyuCPD1IvwWoHV+BRCnzPoZK43XFMgIuOfHatuo2VfVrzjWem4LshIDJm+PXFBNmu5FU8z2LTPdtVQrqSNSsCeVxTlMqLhFoP+2b/a8/UANK+amn3bHz3CxkXDZVWBmVq2am96vAAlCr70Npv96W1G16apSYH7elIAVArcV1QR017Q6hQYXMg01A8Tyl+u62bOImSsRBzbYs1HbFRI8Q3TM2AWgeKKD3HeLiTxVgO/oJzRPvoD/qUy3V9tTFVNBuhPnk8BWMx3WIVOwe6lVjyRe67ati4DzotI1dSGR1XtCxhqClYYAXNmLSL3sVho//5TqI4dJOrm/77zBiiUEVnC80j0DulqYpjWhij6FESw/QJ6nnscm/433OmaxowwOlp8M+I4Ux3UHxnLM9wzM3l+a30XlUgzWJJ3gV3XJttW6aPIp1oktuPz1L8FK0jq+ZREZwkxtvjjiiF0854ubgru5r+ml8B+g7Xgpn1T8kdgClYQYD5n5s1h2xWrEjiIMoaMoSp+iAITPTJdWFVP27ogpRPY0BshRqU/hL53/f3tnHl5VeSf+z7lLFpIQEnJDNsIWNiUBZZFF1qJsMjpUW8AOWrs8TuvY8fdMFVtnnKfFWm1Hn479TWsf2+lPa9W2Uq2KYC1igaAssqrsCSELIQkhhKz33nN+f5x7zj3n3HOXJDeEhPfzD9ybs7zve859v+/7XacDMCPxFLOTTqEoMj5F4rUzHspPHUMCXC5VA2BMG28M1PP7ZfY0DmEFDiRFRkYiLSODJ8c3MdhVR7ia891FCIt+jJ2rnXHVMSZ3MMcrVZc7We5evp1wdgy7NM5Op31diGjX0tCMc4qCKeeQnTuxsTqe3W7FipbVVCtTW5iThiRJ+gqwrXAOjTVVDK/bqU8ylxOHkdZRa3LP1YUIIDkTcOVN0Pum3zd/AQkTFzAIuL2wSa+2JzkkTldf4o1TFxjuWMrK5H2McZ8PTCISpOdCk5p2w4eDNhJJp93UDyWgM7pUfCc/2DmIfLKZnnCKNKmNQVIHGY4WhjpbLCowCcnpJnHKcpIKxrPrc4XqylMsT0ZdHesHqv9IkhqIBqip5TGooIDh3nJw+lEC/kR6mnljGwlO/llc1L3QPu4cpz+jzs+34a87E/KsTPE9QKbzsv5ZArKGJOOok5ADNrcvFLTiPOvXd15KoC8yoODAgYKMQ7XTyT5kJHKcF3GhnuNAJqn1HH6HE9nvM9nMit1nKXafhQrwAt6jH5I0fCYftk8gz9HAZTlJr3GvCT7NsQCnmwPNmcx015vUUTn5udx+uYZEuYUMj4fm9CkcPV5JaV267vqroL7P+46dNwkHMC+4JhRmwGfa2MjMa9+GdFCh9cg7YWvOdxchLPo5Ia52AfuB1yfr+WUgennHWO+hEa70ZjRhZLyWMS01oLvXBj8HJ/9w2TVj8QqzS13iMEySmiBNGb2cN8oGM8Z1jjI5l7vnjEP6+H+Q/Z0hq19X/iQSp90R1lMHgtlG1yweqyco3H9CXWWWyx6ea17KqokKtxS26Z4wWn3yRncBl8pPklP9NmCwKwDOrFEc4Dp8/tOUKx7O+DyMzE3j7Hm15vzMhOOUuM9w0DtCTxh5/cxZelv3HTtPiy8HP04IrNi1tCAKmprEycbW6RQ4LzAz8ThOw6PR1EZ+JRi4Jxt2FnbqOwW4LfkTnA6J8YVT6fx8Gx3bfxvyrBTAqzhJkIKqqXxno/43gA/OOPS0/WsWjyXZNQSl8m8oBhfVC3Iq77dPItPRzGR3BQe9hVyQ07hz0Mc4UNRUMro9Q0GuP42CRLU/g1xno65WM6rhABRFJrmilEUJxh1I6G5GAc44hnPWm8l0t7ryB/AjkXj+M0YHdiNKxQn8ikRnx1gUw05AU99aY6yMCT7HF2Yw/Px2OgLGekndaqmN9XvxVR8VwkJgT0qy26SJNvrT32xI5haP3DLRdgka4e5nncDVCPFgmyF+xWaOVTSGpC6RFdUjTA5sGVrbffxlZzlen4dyfzZLpg/nSLOL7HFfJa1sK3ltJ4IrXofTVlDYZRvV/m8VhBplPg/vtw9mvC+DosDnY+0uxmdnMG3SFDo/z6Ljk7eRWoKqDPfEeaR0BN0kVQeGcXpfU5LH82uD58ygaTdQ5oNju8qZWZLP1PHZvFjWqKcvKZ5YyMjKd0H2IUkO6obeyCunh1Lm8zAr4TjOQFs1oaIbiw3fay+eN3M07QXTSD38B7292n9SpE7uTN5FxccOznfWMhizUAF1wmuWk9T6KWBakUuoK/ZBqKoeRYHDpxp4pewi65LyKHaf1Y/PdF7miym7cQYm6cXOT2mQU3RXVqMACF5fUVP7G/ql2YyMgZfa9yYXacPj1VyZR/lOM3xQOWe8WaQ4OjgvD+aynMzspOOmPjlR9EC9/9t8K2d8Hsbmp5OXlUKBJzWiy7rfNYE2yYkkm3dEkuTQd73xQgiLAYTVA8LoHju7ODemXPyxEovQiXQ/6wTul+GGsVkcOtWgT7KxVI2D8BXFjLsPp1MK2VkMTU+i7mIbiqywZc9ZFENch1Z0CuCWpFRyDCob94R56qS+q9ykDtPUBaZgQ8P/rTgdqLm1At5hVv30d9fcQNFEVaXV+fk2fKf34ho9jYrBN/D733+irq6BSaMyQ8bBuAuDoHvpG9vLuPvWcSy7qZB9x5Nwj5tK8cIi/LXF+KqP4sqbwJhhRazWxvH4dmg3q+CMGL2uFAXk5nryZy3n8tkdKBcNVfQC/yoKDK/byd/ar2dxUnDHZKTKn0mGsyVoD9HGMqDeKXTVM9JVB8D153dzY2ILeY4L+nFaM52GaHRFQY/eN8XAGPuCatfQA/EC9zfIQrPtyvBZUaDBn8IQQ7slVBvLmITzoECWs5kD3pH6vY0vhp7oMBAdf7yyiROVTbodLpytsczn4Wj2GlwVHzMj4SQORcYhSbhLlsR1VwEiN5QtfZkbqic4HBIffabmlnEFUo6PyU/nH24eRVF+Ors+Pcen5RfUMVEUPIY8PNY+G/NOWXPMaELAmu/Hyq5Pz3GkTP0RK7KC0yFRWXcZh0OivdPPQUNeIKdD4t7lE7m5JJfsjGSmTcimpc1re/9whGtX5uAkrhuZidMhkZGWyOBBbuovddDS7tPPVRT1BytJ6MGCGjIOpiWWqStGZwL1RSt56o1y032y0pP1vD5qXIk6i1j/P39yHvOm5FGUn45nSDJlNc0B11kFn1+mtrEt5PmcrGri3ePwmWMcgwvHUnq4htMGz7FzF1r56DPzc8gcnMS44UP0vEn6cwisxstqLnG5zUf5uWa1/bl5uHLH4UjNNJ3f1tZKYu2n+r2MhnOTi23g//KQfAZdPx8cTvwVB0zPx6hKO+fPwIHMYEd70OML1VbzWutsUqQOhjkvmYI0JdSJfJjzEtMTT3JT4ikKpHqGOS8xyOENuReSA0kKbq81l2Lt/8ZuGKcJRVG9lcr9HjId5mSbxv5q50mBi9T4MxjqaNE/azYMbcXvkBTynI2m71RVlhSw/0g0yGm0KolclFPUtsgKl9u81De1hfwWtPf9szpId7QyMrmVJEV1iJDrz+DKn6g/z1gQuaGuIaKpZGKN+rZLk270j4/kTqudrxZNMkzGqCmaNQP1nEk5pnvOLTHbO7qzA7K6C7+547ReUdCY7fSFtz8Ne415k/P0qm+a/eeM38PzrUv0inF/P+3C579g6v+KWSNNAVQV55pNxnStxrLRrrPtQJU+SSkKFGanmfTTWtrv7Yeq9UDEHYdrKB49NKTd1gysxndAiysJepOpOx4IHScr+bOWUwUkfvY2Cf7WQGMDf7RZ9bdM/AeygISJC5Avncd76F2TBV0OGKD1PFsBzqUXM2LceI5dHsrZXe1s7ZjE9QlncZpvo0/yzsDS3Lor0dpyzJvLlvYp5DgbmZV4kkKXuX6JjKTuHED3StPUN+0jZnLWl0mBqxGpoh4FOVgv3rqzIjjh5wxJgBZze83HmsWStpCt9g+h0Z/K9YnVzDKoo8p9qsfX/hP1HDhRb0pDD8H3fWbCcb6U/BHGrCLIPmGzEEQmUgrjWO0MdmnSjWUno7nsaqoUyfLL0tQ6vsC13S5zHWnj/buS90pDa5c2yX9W1sjxs/tN5TvDuftC0F6iGdON3lOzi6eSp/Xf12QrdLU2Gu8hoe4oNBvG9sM1esVDIxLQ1uHThagmsKxt9fkV0lMSTGo1zRgaKcHkkunD2by7wjZOwzpOGkGhM4ec0eO4/OYToIRO0ArByfnG5iGMCHyfdNOXcAzOpuOTt6Gl3rTiBnPOqmFNR3Dl3c750y5k5TTlPg+fdQ6nOPFscIyMtgPj/bXlfaBvMg62tE+hzOfRiw9pwkIBqnwZNPhTuS6hCmP6etXdVSZNamd87Xso/k595+CQtMgU+xoq+zpHcqkjhUWJlXay1Hh0EEm10RQ4G8lzNurCSMLPMk8V79aZq1Naa5hr7/vkhDP6/fSdjsMlbBaCnmEnTE5WNbHtUA0FQweFWYma3W4juexKkqrCUX9Eim43QVLTfYD6QhfmpJm8OnqS98rInEk5amW/gHrHKGys7r65WSmkJbl092KAM7XNuheZFh3vMth8rDWwjdG0dvdQ22CwzQRKkYJqs3A5Jdv61ECIazKoxxs9YlKS3aZYFrsEkwDv76sMBtdZrqmNU+nhGt07Td9dGYROcvFdpB4KxqJo5ypIbGmfQiXDmJvsDpafddWZA9bAdqJVDccKdfs/YHzxl3A5HXh9MpeU5ODspwTbbVRbaf/XBJECVKSUMCRrIgTGebL7jKm/+c5GcpwXcaDgkFQ34YDmT/XyqjiAosgmN1jJ6SJ5zt10fvImSkujqT8KcGNCOR91jNXVmVqku7XPpr4THEeHEtxpSMC4jkOMTTsMAbff3R1FtOGm4NT7VPoy2LcnkZmL5vHdNTdQXXoOLgSTGcrpuSRPjr/NQgiLa5xwaY3t4hqMhMuOiaIapzXDupZWpKGpnQ8PVOt665Y2b9hdUKw7ILt+aN5ITqcUUn7SKoT+z5obabzYqte2BiivaeYnr+wPCX4qPVxjEh7fXXNDSDSt9p217Gg4/DIsmJJrW58awtdfB7qkajQKMGubjKt1o7rLGLMQVLUt5/ClDoaXv6m7liJJXC65ixu5jrmGXaXL6eDx6fWkKTZpPCz312hq7mB8fjprFo/lpS3H2NM5htmJx4MeSNq/RFD1SBKTFt9Gis/DodMNyH6Fw74RTEgITqaqt1IwwtoBNMgpur1BK5ikHX/EOxz/mFuZN1EtdWt0+dXeZ0kJVLLEiSuQUt5o1DcFHAZOtO6UdNuM/lkJtFUOelApMN6l/o7kXYc4POoeLiV48CPpJYalS+foKP09zswCoYYSxI9waY3DxTXYYZ2gNAFhXPVCaIrzSERSp0Xrh1amc8SwtLBlPccXZjBhZCZ1dW4eXnsjb+44rUda2wU/ASH2EE96csgq3mi7OF19Sd9FSMDI3DQqaptNtabt6lPb+dOHy/llVR1ZE0xW1l2moaldr4mtCXItqaOEurKWDStbCLoWW5NNFi/5R84cGcXFg9tAgiElCxgxaQr5ELKrOekdxg22exn0MdHu6cdB0sSbAXUhoShQ7vOwt3MU0w2p6Y0eWaadSeCzY3AuzmFFFAGPrL2RYxWNzCyZS/1fLpJ16ajBq0ttl9YGzZCt2TE0I7isQKXsYerEEgA995jv9F5ITsN36mNkWcaHWrRqT+cYpieeYmaCOTZFu6622/AHoq41zysw9MMgXIy7Da3Z2u4F2UfG8TfIQMLhVHQBL6EIm4Ug/kRS+cQ6YdvtBOwmte7kpupqP7QVfXlNM5V1LSZbSLg+FeWnM3V8NkfPXNSjgq2TNQTUUgZ7iMN5UZ+ErbYLbQyOlF0wCFE1FkKbyCMJMqs/vca7H53R7Rh2ubo01ZRWshSCdpN5k9W6JY0XW0OEmZ3X4K2BFPPW5+XNGMX/1FxUn231Rb6b0aSrL43vUt7EEhLz76Fjx/+zvYH2VX1iAc5pdzJi0hT9Wbpd6rN8pW0eQ4flUNC0n4SAp4+Gblw2XLqp8GZSLc/a40ljb/FSvDtP4AzsdOS0XFwt5/WIbU0d1eBPI8vZrCdblCS4eXoR+Yb+JwTcmQH+Ll9H5ZFPdLsCQMrwcWTlLaTz4CYGS22c8A2jXUkgSeqkwHmBQ74R1PgyWJJ0gPHumpCU9po6DYLCwbQrMexSRjgb9MGQ0QSOJGwWgvijTVKVDa26zaK71zGea2ekXjFrZNhUHD0VIlo/rDsETWcf6fonq5rU9M+BqGC7uAUg5PqKrDB3cp5eNTDWgMFIDgiRxqe13adP7qCqXDQBte1Alao2DNg/jK6/Wlu1uiV1dWptkz9vPx12PCUJBiW5bP37wzkg2PY3fwHOzAJ81Uepb3eQcviPOAyz+97OUbSV3MOKScH7GK+TkuzmF+9L5DOEB9K24NLqggekhgyc9mbjlmQ+7iwil+sw57VV8WaM4tPMW5jcuAUJBWdzNUhOOnKKcdd+BoofBegITIlGVVdWkr1DBEDexBJe2u/T7VIup6R7lp0cfZ2eu8xviLdxSGrusy1lMMZ9HqfiDwqnwDHW9DK6isqdBN72YBu1cTBa1SWJxNlrhc1CEH+K8tOZNaWAurrm6AfHSFfddHsSKGgUNrffPFrPxaS5n0a7vkmnr6iuitaEd6COk/X60VKcdFWdZu2T0bvMSuGwVH0H87uAoABVMBijqo3eUhqlh2t0hwM77OxUGl3djTqHFVHm8/CTV/aTz1IWJR4h3dHGR51F7PaO5xFDckrjrmvFrJG8s6scr0+mHA8/b17C9IRTjHTW4UnqINlTwLGMhfxqV3uwgpxNm4+WX+Anr+xngbuWyUnGaod+vENG8HljApM79uFA9UwyqbkcbtMK3drGovx0Hll7Y9jdYlF+OrOLc3Whoe1Cl80cQeWYofzibzDGdY7Rrlomus0p8o3qKH0s867Df/YQyL7gDsSgqpIAFBl/vdmoHw+EsBCEEM+Vfixuut1xkzW2NZK6K5br2xmlwx3bHeO7tb3hzg0nIMIFls4N1Ag5VtEYUt5UUTBVK4zWVmP1QjvXXiPdGQPtOZQrHn7jW6jeMxDDowk8Y/qXHYdreHjtjaYKj+U+D+U+D06HxCN33EhWfjrTgYzRkd/Xw6fq8fllTpCDP1BOFVSLRdmRg5S4KvUgOeP4nfFn8VbHDL7k81BEaIoarY3hPAyNbdKEhrWdhddPxstkhhbnknRpPx3bX0SxRJ4HtxUOXIUlqrAwfG01+CvApZYOkqM+la4hhIXAhF3RpHC59aNNErGsqnviJguxqbuiXd9oFDaqDE5XXwqpv2DtV1fGI9Iuyvg3k/uxAZdT4pZpw6k438zU8dl6oKRRx6/lMQJAUcKqkgpz0kyfl8woZFCSyySotFQT2jhbVWmxCsqTVU00NLXrdgFQJ7cl04frfbCmf/H5FV2FaESSCEkDE60txWOycDkdVPg9bGybwZ3JH+nBeJNcZ4PR1BZDc6Uvk9OdWXpsQ0NTu20b7QSFNajV6ixiPaYwJ42iKarKruPAJnxnPjG7BaN6adXt+StpgQJKVvdbJeCk4MPBC5+l8+Xi0He3JwhhITBhjYI2us/a5i7q4cvY05V6NGET6/W1CSc50cW7H1dAQB11+HSDvnq0YrQTaNG1YG/AhtiLVWnux37DbkFCTQZ518IiXUBpgsyq4w9XftOIscyu0T5h9Wiycxk2LhqscR5WrGV3NRTgvT1nuWGcRzeOGwMNXc6gPUYThJJNNoFIiSq172dNKWDN4rHsO3aeIq8LWoKuwbqLsBKs9gfqhLu3c4wp/sXhUMsDG73Z7MbXGtT6uy3HcDgkfH5FFx4tbd4wga9FOAal27sGK5DWVqWeA0EhZzjkkHc4W9snUe7zdKskQSSEsBCYME6+SJKpSLw1t368Xsbu6vW1c6MJg0jXt6YV37LnrOnvkVaPRjuBLzCxbg/sTMCspoDIgs36t8VTC9iyu0I3XDoDbrbhdifGPkZzedZW+rHEomh9swb5WaPUrakoNIxC0LqC98uKKdWIne7/ZFUTcybl6JH0BZ5U/drhxsL6/TfbfPoip82dRnGKmprd6PGkALUZ00iaeDM53koa3QXc2DyEIQaPMb9fYVRuGkNS1VKxdqo64y5Ke3bG9CqaYLj71nFhA1/ttI7GXYTu3ksw+E9OyuD1C+MpbVc97sIJsp4ghIXARKRVqjW3frxfxu7SXWGjGT6NVcisuv9Iq0fjsY6ALiCSmiKSYLP+7ViFIVU2wRTzdhHa4dRkJ6ua9Ghq7Rhjnx0O1Z02kgsvhMbH2Eep2+cIM8Z4OJ0Opo7zqMkuA3xqk2rkWEUjlXWXdbWgFjApBdqiHRtup2b9vvRQtf653OthV85aCqkmO0UhpexDUGQkp5uieUsDHkRT8FY10VBZw8GT5pxSZTXNSFKzmh5eUUzt2Xagit9v44FcAAAaQElEQVRtOaZnq9UEhrXWhV9WqDjXHDbwNWHcHHzHtqtGbGyC+LT3IvD5on8QZSX/RsK5Zm5o6SQnK5UbiobGdVcBQlgIbIi0So01UK+3iFc9DggaPo1R00bd/5SiLJbNHGF7H6OdQAqkUy/wpJp2FnaCJpJgs/7NuMLX4kV66mVm7LPmTgtqQJ1RpWS0c9gJOKNDgJ23lTWiXiuTC6EpR8LtWKxYhZJxLCSHRENTOyermkLGaHZJHkdONwSeq0TqiPGUTPmCeu/a2Xpqds3VNFoOMUVRCz9Z2/7SlmOm4MGxBemcqr6kLioseqXth2p4pDiXR+6+MWRsy3weqkfew3WXdpJYfwy1nmNwRyEZVGcAn3SOYpNB6KxdOpGhKW7ijRAWgohYJ7CeqIx6SjQ3Wzu3xkhohs9oUdN2hNsl3H3LOLYfrMbtdJCXlRLSfruI7Fh2GtF2J9brhFt1W/usuRZHUinZ7VSMu087m4U1ov7QqQbkgC3G6ZRMuwVtoi89XGPrImxCknQPKatjgrEOu7F9za2dLJ5awHt7ziLLCq+8f0JPjOkcVhQSj1B6uCasoLBi3G1Zvda8fhk5kAZEsvzNH6jOqDljaGNrdDAYkzCaB9JOgezVx8owDCgKnPRl807HNN0hwu+XOXyqngUl5mDUeCCEhaDfEMlAHMmt0YjV8NmVoDm7axhX4FpwnzbRnKhsYvvhGuYW59om5oPIqdjDCWbr93ZC1OgObJxgJ4zMtHUt1lCwT1tud49wBXms6iej7Uv2K/qK26/nwIAPD1YHDMiSOYDNAfNK8khOdNlO9gB1TW36OUaPOG18NfubcUINZ287WaU+Mw2nA0rGqLsivQ0BoWctJWw00DsdEnMn51FZdyJwf0AO7qiMu85wnnCnO7M4MHIdN9a/jXypNjRxitPF5TEr+UrWaJO6uHhMlu1z6SlCWAj6DZFUMOFcL40TgtV76UdDBnV5pxTJtdhel6+w7YA6ERonq9LDNdQ1temqnFijze0wroSNk+WaxWN5+b3j+GWF372nJqK765YJtuouY4yJQmja8ljjYezUTwCHTtXjD0yWJ6uaDO69QZWOFhEPqprGHyh+UZiTRkubV1+lhzW0W9RhJuO6JcFlOPWd0RYlAXNL8li3NBiUd8M4T9idoJ2BXkt1v+NwDb5AYkCretPOE05rZ97EEhIu+fXkhRLgyBmHMyMf97g5zAvsiozqYTVKP34BthpCWAjiSjxtClYiGYgjuV5q7bJ6L3Vnu278YXt9Mi9tOQaKqrpZs3hs2IyziiGJn+aOqatiApNcLNHmVqwrYYcj2O+WNq/ufqt54RSPzTbps60ODfuOnQ9Jl2K1D4SbbE9WNfHmjtN6/43qJ9NYKJjGQgq0T3IEFS16uxV076FI2XQl4LqRGabdULgEl5HezRB7R4TcYnaBd3bOBscqGvX+SMDovMEAukovajvzFwDoZXW1vFTh2tVbCGEh6BFW19N41fgORyTVTLi0C2DvvdSd7bo12lubB70+mYpzzSHV8owpHkzp2g+qqR2Mk5zd6l1re7gJzroSvtnQ7/GFoXVJ7ASk1aHh+Nn9IeorQC/MZGcPshqF1RKm5lgRYxZb44QImOwODodkcT1VaGnzhjW0a5OstdqfJgi7kvcs0oLErr9WtWK4hYydnagrSTaNyQv7iqtWWJw4cYLnnnuOQYMGsXLlSubMmdPXTRJYsP5grDUg4h0UFK4N1tWd9h1gmjiNnk4lY0JLk8aCNpm8ueM0n5aZI4x3HK7RcxpphEszbnRHnTo+m2MVaqJAhyQho8S804i0Ei7KD61LEk1AFuWn6+orzT4AmOwt1tU2YLvKL8xOUwMcA4TLYqud75cV3UtrclGWvivR3ErtnC1iibGxy3sWaQccyyrdGrz6yvvHOXv+sl68yvqsjII2lqwDsTpD9OZO3spVKyxaW1v53ve+h9Pp5JlnnhHC4irE+tJD9NQa8STc6i5c0NqaxWPZfrCaitrLHDhZz6e/3Mm/re767qcoP5hQ0Og1I8uhdpJwqgnNi6eppZPf//W4rpICVUWjrbxjiamINGFa65LEos+22gdiCca0Ci1tp2QXJW6H9fxlM0ewbOaImCPvu0I8kldad5hlNcExtdZDN95rdnFuVJVerO+13Xe9KTCuGmHxwgsvsGPHDv3zb37zGyoqKli/fj3r1q3rw5YJwmG3qo3V9TQehFPb2E1sVk8lUFeE3d39WN02NZ275usfyzWNNSdMBNQuscZURJswuzqhWu+rBWPaqaaM97BVE7liWzxEchUOh3VVHesqO1ZjfSS0xcfmj89wvrHd9LdwRna7AlnhdlmxvNd2310TwuLrX/86X//61/XPR44cYeTIkbz66qvcd999LF++vA9bJ7AjHj/wWAh3TrjJ1O47q6cSqBNZT3Y/2iRsTEFt9PWPZcKyYvToiVV/Hm/C3deomjK6rhrP66qaKNL5kbDzSos1b1msQjjSvUOiyyVs3Wnt7hUt8/Dp6kt6tb5o7/WV3MlfNcLCSkdHB9///vdJTU1l/vz5fd0cQRh68gOP1dsn3DnhJiO776zRvnOLc1kxd4zuGRRL6vBIenGjzj2WVZ61PSWjh6rJ5fwycyfnmfp4pYSEEet97VxXYzUY94Z+3br67kresp4IYbvobjtPrHD3gvCqI2uskEOCNYvH6rsmOwcD67Xf2VXOzJL8/hnBffnyZVavXs0vf/lLCgoKAHjrrbf4xS9+gc/n45577uHuu+8OOW/q1KlMnTq1t5snuIJ0dftvdcW0OyecTSAW3b7Hk0ZdXXPMqcMjCbiurlaN7UlJdvPyX4/rk0RF7TEqzjXHFIV+pehK/8IZYrujX491Vxkub9m2A1UcPn2B4tGZpmy13RXC4XaodoLC7l6RcntZY4VkRRXSdnYP67WNx7xVWt4tW1w0elVYHDx4kMcee4zy8nL9u9raWp599lk2btxIQkICq1ev5qabbqKoKL4lAAGGDk2NflAYPJ606AcNMHq7zzNL8nmrtByfT8blcjCzJD/sPY+WX+Cnr+7H6w3mH4p2TjQ8njRmTSkI+W7boRr8hh9wZUOrflykv1mv86Mhgzh8qp7iMVlMGJkZc3v++LfjelZSUFNgbztQTemn53ji/jkxXcvI0fILEdvRnfGLtX/ac9Oesdb+WMcxlmsBNLR4WTx9OCCxaNpwJozMpHhstql9m3eV8+LmYwDsP15HWmoSS8MY2GPF+A47HBK3zBjBomnDAfVdifbsI/0GZpbk8+aOMl215Q78/fCp+qhjZxxfny+28e0qvSos/vCHP/D444/z8MMP69+VlpYyc+ZMhgwZAsCSJUvYvHkzDzzwQNzv39BwOSSLaCxoK85riSvR56Epbv5tdXB1PzTFHfaeHx2qwqslqiO4zdfOiYdKQ+tzwdBBajruwKq0YOggvV2R/qZhbIsWwxBtLI3nFAwdhMMQUKjh88l8dKiqSyqFaCv4njznoSnuqP17Z/spOr1ySPtjGUcr+juggNcr8872UwxNcYf08YaiodTVuUPat21fhel62/ZVMLWoey7TxjEwvsNF+em2dU3CvZORfgNDU9w8bIkVinXsjMe4XLGNrxWHQ4q4wO5VYfHEE0+EfHf+/Hk8Ho/+OTs7m0OHDvVmMwRXEbFu/+1cMXuq0ojUplhTh9vZM+Jhh9ECCptaOjl8uiGkzkSsxMPTp7tEiibvjp1ACyrU8j5pcSyx9nHq+GxTLMzU8dk97yShdhhrZoBoYx7pNxCrCtXuPO2YfmuzsCLLarIsDUVRTJ8FAoj8A+mNCbGrP+CetCVaUFZPdk099fSJRLR2RYomB/sEiEbjrJ1X3dziXLYdUKPdtTiWWPuo2SjsbBba/buSpdgOu8wAveGVFMsiSzumt7QEV1xY5OTksHfvXv1zXV0d2dnxkfiCgUW4H0hvToixYiwr2tW2RGu/0eBp/BwLveVuG80JwG4s7CK97a7nCOSHsot+nl2cG1J8qSg/XS+TOnV8dsQ+LpiSz123TLBVHcaSpTgaxswAWl2Tq8UpId5ccWExe/ZsnnvuOS5cuEBycjLvvfceP/zhD690MwT9mL6KP9CIlHk2VnfSeKu2jOf2xriE20F1dyyM15MNKcmtuzO7sdICLH1+meNnm2xjPmLpT7QsxbHQ1+/ileSKC4thw4bx0EMPsW7dOrxeL3feeSclJSVXuhmCfk5fxR9A6MTZ0uYNm8YiHPFWbUH8bTlGwu2GujsW1jgTLfNsuJ1WvNWQ4wsjZynuCn35Ll5Jroiw2Lp1q+nzypUrWbly5ZW4tUAQd3pbDWadSGNNIdKbxu1wK+jujoVdsFqsq/N4jH9RfuQsxVcTVzJZYCQkRbEWAxw4CNfZ2BF97hq9/QM2ppQIl8nU7hytMpwzzPG98ZzDBeH15kTclfHvz+92d3aL3e1vn7rOCgQDld5WPRTlR08hYpeeva9ySVnbFQ/jcVfu2dv01eq+L12hrQhhIbjm0H74veWPHi8iqVvCrTivBv25nfG49HBN3CbbeE/c0a7Xm7agaG2wvgMpyW69wt6Vfs5CWAiuKa5EDp14caVjTeKF1XjsdNAldVok4j1xx3K93h7rWJNlpiS7Y86s2xs4rtidBIKrAOMPX6tncTVTlJ8eUkUNgitOhyWN9dWAZjxeMCWPBVPymFuSF6JO6y7haj305vV6e6yjtUF7B+yKYV1JxM5CcE1h3Nb3tJ5FX3K1+/dbU2JYA+u6S7w90WK5Xm+Pdax96muVlPCGsqE/e090l2upz/3FZtEb9NVzjqedoavXitbnq8E1NdY2GKPlw6mkessbSggLG66liVND9PnK0xeTVF/3OZ7EOn4Dqc8a7+wqZ+PfT6MoapGkf5w3Wg+GFK6zAsEA4kp42AxkrvXx64v8aEJYCAR9wNXszdQf6I/jF8+dZF/YrISwEAj6gKshc25/pr+NX28kh7zSMTVCWAgEfcDV7s10tdPfxu9qTA7ZVYSwEAj6iKsh2ro/05/Gr7s7oatJ3SaEhUAg6NdcDa6v0ejuTiickOmLPgthIRAI+i1Xk5omGt3ZCYUr/tQXfRbCQiAQ9FuuJjVNb9EbxZ+6g8gNJRAI+i1Xc46s3qKv+ix2FgKBoN/S37yi4kFf9VkIC4FAEHeupAG2P3lFxYu+6LMQFgKBIK70J6OzIHaEzUIgEMSVeNecEFwdCGEhEAjiyrVodL4WEGoogUAQV65Fo/O1gBAWAoEg7lyLRueBjlBDCQQCgSAqQlgIBAKBICpCWAgEAoEgKkJYCAQCgSAqQlgIBAKBICoD2hvK4ZD65Nz+iujztYHo88CnO/2Ndo6kKIrS3QYJBAKB4NpAqKEEAoFAEBUhLAQCgUAQFSEsBAKBQBAVISwEAoFAEBUhLAQCgUAQFSEsBAKBQBAVISwEAoFAEBUhLAQCgUAQFSEsBAKBQBAVISwMvPXWWyxfvpxbb72Vl19+ua+bE1d+/vOfs2LFClasWMHTTz8NQGlpKStXruTWW2/l2Wef1Y/9/PPPWbVqFUuWLOH73/8+Pp+vr5odF5566inWr18PhO9bdXU1d999N0uXLuWf//mfaWlp6csmd5utW7eyatUqli1bxoYNG4CB/5zffPNN/d1+6qmngIH5nC9fvsxtt91GZWUl0PXn2uO+KwJFURTl3LlzysKFC5XGxkalpaVFWblypXLixIm+blZc2Llzp/LlL39Z6ejoUDo7O5V169Ypb731ljJ//nyloqJC8Xq9yn333ads27ZNURRFWbFihbJ//35FURTl0UcfVV5++eW+bH6PKC0tVW666SblkUceURQlfN+++c1vKm+//baiKIry85//XHn66af7psE9oKKiQrn55puVmpoapbOzU1mzZo2ybdu2Af2cW1tblenTpysNDQ2K1+tV7rzzTmXnzp0D7jkfOHBAue2225Trr79eOXv2rNLW1tbl59rTvoudRYDS0lJmzpzJkCFDGDRoEEuWLGHz5s193ay44PF4WL9+PQkJCbjdbsaMGUN5eTkjRoxg+PDhuFwuVq5cyebNm6mqqqK9vZ0pU6YAsGrVqn47DhcvXuTZZ5/l/vvvBwjbN6/Xy549e1iyZInp+/7GX//6V5YvX05OTg5ut5tnn32W5OTkAf2c/X4/sizT1taGz+fD5/PhcrkG3HP+wx/+wOOPP052djYAhw4d6tJzjUffB3TW2a5w/vx5PB6P/jk7O5tDhw71YYvix9ixY/X/l5eX8+677/KVr3wlpL+1tbUh4+DxeKitrb2i7Y0X//Ef/8FDDz1ETU0NEPqMtb41NjaSmpqKy+Uyfd/fOHPmDG63m/vvv5+amhoWLFjA2LFjB/RzTk1N5Tvf+Q7Lli0jOTmZ6dOn43a7B9xzfuKJJ0yf7earSM81Hn0XO4sAsiwjScEUvYqimD4PBE6cOMF9993Hww8/zPDhw237O1DG4Y9//CO5ubnMmjVL/y5c3+z62B/77Pf72bVrFz/60Y947bXXOHToEGfPnh3Qz/no0aO8/vrrfPDBB2zfvh2Hw8HOnTsH9HOG8O9yb77jYmcRICcnh7179+qf6+rq9C3fQGDfvn08+OCDfO9732PFihXs3r2buro6/e9af3Nyckzf19fX98tx2LRpE3V1ddx+++00NTXR2tqKJEm2fcvMzKS5uRm/34/T6ey3zz4rK4tZs2aRmZkJwOLFi9m8eTNOp1M/ZqA95x07djBr1iyGDh0KqOqVX//61wP6OQMhzy/ac41H38XOIsDs2bPZtWsXFy5coK2tjffee4958+b1dbPiQk1NDd/+9rf56U9/yooVKwCYPHkyZWVlnDlzBr/fz9tvv828efPIz88nMTGRffv2AaqnSX8ch//93//l7bff5s033+TBBx9k0aJFPPnkk7Z9c7vdTJs2jU2bNgHwxhtv9Ms+L1y4kB07dnDp0iX8fj/bt29n6dKlA/o5T5gwgdLSUlpbW1EUha1btzJjxowB/Zyh67/fePRdFD8y8NZbb/H888/j9Xq58847+cY3vtHXTYoLGzZs4PXXX6ewsFD/bvXq1YwcOZInn3ySjo4O5s+fz6OPPookSRw9epTHHnuMy5cvc/311/Pkk0+SkJDQhz3oGRs3bmT37t38+Mc/Dtu3qqoq1q9fT0NDA7m5uTzzzDOkp6f3ddO7zJ/+9Cd++9vf4vV6mTNnDo899hgff/zxgH7Ov/rVr9i4cSNut5vi4mIef/xxysrKBuRzXrRoES+++CIFBQXs2rWrS8+1p30XwkIgEAgEURFqKIFAIBBERQgLgUAgEERFCAuBQCAQREUIC4FAIBBERQgLgUAgEERFBOUJBhwbNmxgz549AJw6dYr8/HySkpIAeO211/T/R+Nvf/sbu3bt4rHHHgt7TG1tLd/5znd49dVXe95wYP369ezcuVMPrNP44he/yLp16+JyD+O9xo4dy9e+9rW4XlcwMBGus4IBzaJFi/jZz35GcXFxXzclJq7kBC6EhaAriJ2F4Jpj0qRJfOELX+Do0aP89Kc/5dixY7z22mt4vV6ampr4xje+wdq1a9m4cSNbtmzh+eef55/+6Z+YMmUKn3zyCTU1NcyaNYsf/vCHVFdXs3LlSvbv389zzz1HVVUVdXV1VFVVMWzYMH7yk5/oSSn/8z//E6/XS2FhIdXV1axfv56bbrqpS21ftGgRK1asYOfOnTQ3N/PVr36VtWvXAuqu6aWXXsLhcJCVlcW///u/M2rUKFpaWtiwYQOffPIJTqeTxYsX89BDDwGwf/9+Vq9eTX19PWPHjuW//uu/GDRoUNzHXND/EcJCcM3h9XpZuHAhP/vZz/SJ9Fe/+hUZGRkcOHDANAEbqaio4KWXXqK1tZVly5axe/duCgoKTMfs3buXN954g9TUVO6//35effVVvvWtb/Ev//Iv/OAHP2D+/Pl89NFH3HvvvWHb99vf/pa//OUvpu+efvppxo8fD0BTUxOvv/46tbW13HHHHUydOpULFy7wwgsv8Nprr5GZmcnGjRv59re/zTvvvMN///d/09HRwaZNm/D7/dx3333s3r0bUNVoL774IgkJCdx1112899573HHHHT0cYcFARAgLwTXJtGnTAEhJSeGXv/wlH374IeXl5Rw9epTW1lbbcxYuXIjD4SA1NZURI0bQ1NQUIixmzJhBamoqANdddx1NTU0cP34cgPnz5wMwc+ZMU9p4K/fee29E1dDatWuRJImcnBzmzp3Lzp07qa+vZ/ny5bqtY9WqVTzxxBNUVlZSWlrKo48+itPpxOl08rvf/Q6AP//5zyxevJjk5GRATWV/4cKFqGMnuDYR3lCCaxJN1XLu3DnuuOMOqqqqmDp1Kv/6r/8a9hyjYVxL+xzLMU6nM+RYYybYrqLVJAA1VbXD4UCW5ZDjFEXRiwEZ01HX1NTQ2NgYcq1wfRIIQAgLwTXOkSNHyMzM5Fvf+hY333wzH3zwAaDWhogXY8aMISEhgb///e+AWuXs+PHj3a6l8MYbbwBqTeWdO3cyb9485s6dy6ZNm/Sdweuvv86QIUMYMWIEs2bN4s9//jOyLNPZ2cmDDz6oe4sJBLEi1FCCa5o5c+bwpz/9iaVLlyJJEjNmzCAzM5MzZ87E7R4ul4vnnnuOxx9/nGeeeYaRI0eSlZUV1oXXzmYxefJkfvCDHwBQWVnJqlWraG9v57HHHmP06NGMHj2ae++9l3vuuQdZlsnMzOT555/H4XDwwAMP8MQTT3D77bfj9/tZvnw5t956K1u3bo1bHwUDH+E6KxBcAZ566im+9rWvkZWVRU1NDbfffjvvv/8+gwcP7tJ1+psrsGDgIHYWAsEVID8/n3vvvReXy4WiKGzYsKHLgkIg6EvEzkIgEAgEUREGboFAIBBERQgLgUAgEERFCAuBQCAQREUIC4FAIBBERQgLgUAgEERFCAuBQCAQROX/A51eIqrsvDP6AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "optimizer = torch.optim.SGD(dropout_net.parameters(), lr=0.03, momentum=0.9)\n", "dropout_net.load_state_dict(torch.load('dropout_net.pth'))\n", "losses_train, losses_test = [], []\n", "for epoch in range(1000):\n", " dropout_net.train()\n", " y_pred = dropout_net(x_sparse)\n", " loss = loss_fn(y_pred, y_sparse)\n", " optimizer.zero_grad()\n", " loss.backward()\n", " optimizer.step()\n", " losses_train.append(loss.data)\n", " dropout_net.eval()\n", " y_pred = dropout_net(x_test)\n", " losses_test.append(loss_fn(y_pred, y_test).data)\n", "plt.plot(losses_train, '.', label='TRAIN')\n", "plt.plot(losses_test, '.', label='TEST')\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note how dropout has a comparable effect on the loss curves to our data augmentation above. Since these methods achieve their results with different approaches, they can be combined.\n", "\n", "Also note that the TEST curve is much smoother: this is because we disable the dropout when testing using the model's [eval method](https://pytorch.org/docs/stable/nn.html#torch.nn.Module.eval). (There is a corresponding [train method](https://pytorch.org/docs/stable/nn.html#torch.nn.Module.train) to allow you to switch between TRAIN/TEST modes.)\n", "\n", "The fixed dropout rates used by each module are yet more hyperparameters that needed to be chosen carefully. To see the impact of a poor choice in this problem, try 50% (which is often recommended as a good default):" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEJCAYAAACKWmBmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9eXwc5ZXu/63qlqzVsiS3JEuyJNuyJIhlyxiM97AYA2FJmIQBA4Hc3MwaMneyAPfOcD/JTH5kIclk7oTchF9m7iVhnSQkEDAxhiQG27JZvciLJEu2LGuXJVlotbqr6v5R/VZXVVe3JCNbxnqfP8Dq7qp6azvnfc95znMUwzAMJCQkJCQkYkCd7gFISEhISFzYkI5CQkJCQiIupKOQkJCQkIgL6SgkJCQkJOJCOgoJCQkJibiQjkJCQkJCIi6ko5CQkJCQiAv/dA/gXKCvbwhdP7vykOzsNHp6Bqd4RBc25Dlf/Jhp5wvynCcDVVXIzEyN+f0F6yh6e3t55JFHSElJ4eMf/zgbN26c8La6bpy1oxDbzzTIc774MdPOF+Q5TxUu2NDTk08+yX333cc3v/lNfvnLX073cCQkJCRmLC5YR3Hq1Cny8vKmexgSEhISMx4XbOgpLy+P7u5ucnJypnsoEhISHyEYhkFfXzfd3SfRNH26h3Ne0dWlouvxzlkhMTGJzMwAiqJMeL8XrKO4/fbbefTRR0lISODOO++c7uFISEh8RDA42I+iKOTlzUfTpns05xd+v0ooFNtRGIbO6dOnGBzsJz19zoT3e95DT4ODg9x88820tLRYn7300kt84hOfYNOmTTz99NMA5OTk8P3vf59vf/vbXHPNNed7mBISEh9RjIwMkp4+B0W5YCPr0wZFUUlPz2RkZHLMqPO6oti/fz8PP/wwTU1N1mednZ388Ic/5De/+Q2JiYnceeedXHnllZSWlp71cbKz085qu9qmXrYfqKdy0VwqSrLO+vgfRQQC6dM9hPOOmXbOM+V8u7oMZs1KBMwZ9kzDeOfs8yUCxqSeh/PqKH75y1/y9a9/nQcffND6rLq6mlWrVjFnjrkMuv7669m6dSv333//WR+np2dw0hSxhtZ+vvfsXjRNx+dTeWDzckoLMs56DB8lBALpdHcPTPcwzitm2jnPpPPVdR1NM/D7lbhhmIsR44WeBHRddzwPqqrEnWCfV0fxyCOPRH3W1dVFIBCw/s7JyeHAgQPnc1gA1DX3EdJ0DAPQdOqa+2aMo5CQkDh3+MEPvktNzX5CoSAtLScpKVkIwO2338ljj/0rubkmu1PTQgSDQf72b/8bGzZcZW3/618/x2OP/SvPP/8y2dlzrc8/85lb+NGPHmfevHzWrbuchx56mNtu+zPr+/vv/0s+//m/5LLLLv/Q5zDtyWxd1x3Zd8MwJpWNnyqUF2Xi96nWiqK8KPO8j0FCQuLiw1e/+hAA7e1tfOlLf8UTTzwDwCuvvMS6dRv4x3/8hvXbN9/czve+9y2Ho3jllZdYv/4qtmz5Hffe+/mYx3n88cdYs2YN2dlTzxSd9gCeoMEKTBcltrQggwc2L+eeGy+ZUWEnCQkJEw2t/WzZ3URDa/+0jaGjo53Zs2dHxtRwlA8++IC7776Pl156IS719fbbN/Otb/3zORnXtDuKNWvWsHv3bnp7exkZGWHbtm1s2LBhWsZSWpDB7deWSSchITHDIHKUv3nzGN97du95cxY7d77J5z53F7ff/kluvfV66uqO8O1v/8D6fsuW33HNNRupqLgEn8/HW2/tjrmvu+++j/7+fl566YUpH+e0O4rc3Fy+/OUvc++99/KpT32Km2++maVLl073sCQkJGYQ7DlKLZyjPB9Yt24DTzzxDE888TSFhfOZP7+IoqJiAEKhEK+99ns2brwegKuv3siLLz4fc19+v5//+T//iccff4zOzo4pHee05Cj++Mc/Ov6+5ZZbuOWWW6ZjKBISEhLTnqNMTU3j4Yf/iXvvvYPLL1/JkiVL2bXrTQYGBviHf3gAMB1HX18vXV2d5OTkeu5n0aJSbr99M9/97v83peOb9hWFhISExHRD5Chv27Bw2nKU+fkFfPrTd/C//tf3MQyDLVte4i/+4m/49a9f4te/fokXXvg9lZXLxg0tiRDUoUM1UzY26SgkJCQkMJ3FTatLpjVH+dnP/hfa29vYunULe/e+y803f8rx/Z133sPLL7+IFkebxO/384//+PUpHZdiGMZFJ9h+NgV3AjOpMElAnvPFj5l0vh0dJ8jLK55w8dnFhImes7hGAuMV3MkVhYSEhIREXEhHISEhISERF9JRSEhISEjEhXQUEhISEhJxIR2FhISEhERcSEchISEhIREX0lFISEhISMTFtMuMS0hISFzMmGg/CoEHHvgHPvaxJfzpT6/z5JNPoGkahqFzww03cddd9/LWW7v5yU9+BEBr60mysrJJTk5h3rx8vve9fzkn5yAdhYSEhASgdTYQaqvFn1+BL/fsWzG7MZl+FALd3V089ti/8n/+z1NkZMxheHiY++//S4qKilm37uNceeVqYGqbE8WDdBQSEhIzHlpnA8MvPwp6kDE1gZSbH5xSZzFZnD59mlAoxOjoKBkZkJKSwsMPf4PExFnTMh7pKCQkJGY8Qm21oAfBMEAPEWqrPS+OQvSjEEhISOBnP/s5ixeXsX79x/nzP/8kZWXlLF9+OddddwOFhfPP+Zi8IB2FDQ2t/Ww/0E5hdopsXiQhMYPgz69gTE0APQSqH39+xXk5bqzQE8DXvvY/uO++/8rbb+/h7bd381d/9V/4+te/ycc/fs15GZsd0lGE0dDaz6PPvE9IM/D7FB686zLpLCQkZgh8uaWk3PzgOclRnA2qq3cyMjLMtddu4qabbuWmm27ld7/7LS+//OK0OApJjw2juqadkGYqzoY0g+qa9mkekYSExPmEL7eUWctvnnYnAZCUlMRPf/pj2tvbADAMg6NH61m8uHxaxiNXFBISEhLTBHeOAuCOO+7ixhtv5vOf/wsefPDvCYVCAFx55Wo+97kvTMcwZT8KgYbWfr77zPvomoHqU3hoBoWeZlKvAoGZds4z6XxlP4qp70chVxRhlBZk8NBdl9HSMyyT2RISEhI2SEdhQ2lBBqurCmfMzEtCQkJiIpDJbAkJiYsOF2FEfcpwNtdGOgoXapt62bK7iYbW/ukeioSExFlAVX1oWmi6h3HBQtNCqKpvUtvI0JMNDa39fP+5vQRDOn6fygObl8tchYTERwzJyWkMDJxm1qzAdA/lgoNh6AwM9JGcHDtx7QXpKGxoO3KAj/v3c9TIo1kLUNfcJx2FhMRHDGlpGfT1ddPRcRJNm1msJ1VV0fV456yQmJhEWtrk7Jp0FGFonQ1UNf2CpUlBNiX5eHz4esqLVkz3sCQkJCYJRVHIysqZUZRggXN1ztJRhBFqq0UxQvgUUND5s/KQXE1ISEhIIJPZFvz5FRiKH81QCBkqv6nzy4S2hISEBNJRWPDllrKv5F5+P1rFjwc20XhmLi/uPCadhYSExIyHDD3ZkH/JUp7ZrxEM6RjA4eN91J/cywOblwNQ19xHeVGmDElJSEjMKEhHYUNpQQaP/PVannj5IIeP92EAmqZTXdPOroMdhDRJm5WQkJh5kKEnFypKsvjkuoX4/SqqAj6feYlCmo5hmI6jrrlvmkcpISEhcf4gVxQeKC3I4IHNy61QE8Cugx1omo7Pp1qfSUhISMwESEcRA6UFGY7wkt1xyLCThITETIJ0FBOE23FISEhIzBTIHMU4aGjtlyKBEhISMxpyReGBhtZ+6pr7SE1O4NnXj0q2k4SExIyGdBQu1Db18r1n9xLSdBRFwdANiyYrRQIlJCRmImToyQats4HuN39NIZ0YBhi6gaoqFk02NTlBhqEkJCRmHOSKIgyts4Hhlx8lXwvyt+kq/3tgEy3ksnnjYoZGgjIMJSEhMWMhHUUYobZa0IMoGCQoBp9cPEbK5RFnsGV3U1TRXTxHIfIckk4rISHxUYd0FGH48ysYUxNAD4Hi43RqCSm278uLMvH7VDRNR1EVevpHaWjt93QCDa39Vp5Drj4kJCQ+6pCOIgxfbikpNz/IB02H+En1GRrfC+Lf5xQE3LxxMc0dA+ysaeeN/W3sOtjhKRhY19w3qdWHhISExIUM6Shs8OWWsr8zlcaxI5aRdwsCrl2Sh6YbMb9/YPNyx+pDSn5ISEh81CEdhQuVi+Y6jDw4BQGBuN/XNfdx0+oSKfkhISFx0UA6ChcqSrLiCgIW5aVbv11TOS/qe7GNlPyQkJC4WCAdhQdiCQK6KbJrKudFKc1K5yAhIXGxQTqKCUA4jlgUWbtj+bC0WEmrlZCQuNAgHYULoy11nDn8Pv78Cny5pY7vxktSf1harKTVSkhIXIiQjsKGsSPbadv1JOg6Y74EUm5+0OEsvBoabdndZP37xZ3HCIX7bZ8NLXY8Wq1cbUhISEwHpKMIQ+ts4MzOJzEMDQXQtSCdtfuob0/mvbouVpTnUBhIczgJMftXVQUFCGkGAEpYG6q8KHNSxj3eikWuNiQkJKYL0lGEEWqrtZyEYQAYvPL+KarP1AFw6HgfPlVBCwsFLluUbc3+dc1UmAVQgEtLMvnkuoWA05msr5xnJcC9EC8xLov4JCQkpgtSPTYMf34FuuJHMyKf3ZbyDiX+butvTTe/1HWDfUdPoSoKAKoKfp+pMuv3q3xy3cJIhXZIGHeD7fva+N6ze+Oqz5YWZHDT6pIoJyBWG6pttSIhISFxPiBXFGH4ckvpX30/XW/8kjJ/m2mQDZ1SfwdNoUDU7w1MhwGAonDXdWUMjQQdK4HU5AQM13ZnsxoQ4SuhZCtzFBISEucT0lHYULykivS0JPTXvg+Ghurzs3jFStrbklhRnkNNYw97j56yfi+cgKEbDI0EuWl1iWN/QyPBqGNMdjUgcxMSEhLTDekoXCi/4go6kv87obZa/PkVZIYCVPj7KAyk0d034vitT1XMVYWikJqcEL2vokwS/GZyGgWKc9NZvyx/SplQEhISEuca0lG4UNvUy55jfsqL1tPSPsjT295H1w1Un9kWVUABli7K5kBjD7pu8OzrRwGiQkNrl+TRPzTGgcYejrcPcLKrnsJA2oSNvRQYlJCQmG5IR2FDQ2s/339uL8GQyVIydAPhGzTNmW1QVYWM1ER0w2Q8BUM6T75aB4aZ0N68cTHPvn6UYEh3bBfSDH6/5wQL82dPKNcgJUIkJCSmG9JR2FDX3EeB0cmiWR00hvI4bgQo8XdT6u+gIZTHST0HXTdQVIW7N5VRGEgzJcbDRXZG2JeENJ336rqinITA/oZT7Gs4NeGcgxQYlJCQmE5IR2HDkvTTrErfhg8NDR+/Hb6C21Lesf7+ydAmii5d5qiFeGDzcl7ceYxDx/us/SjAivIcjpw4HWFG2RBZpciWqhISEhc+ZB2FDXnBFhIUHZ8CPnSWJpzAh2b9vcjXQXZGksNglxZksKI8B1Vx7qswkMY9m8qiPhdQGJ8BJRhPv3nz2Lj1FxISEhLnCnJFYYM/v4KgPwEtFERDZX+wmNKEThRDR0fhuD6PPw8bdjHTF9Lj9oWDbkB1TTv33lABwJOv1llhKQsKbN64OO4qIRbjyX5sWVchISFX3uca0lHY4MstZd7d3+Don7aw72i34ztVUbj7ujKKw4Za1DYoipMNJdDY2s83f/4OA0Nj0U4CwIius3A/7F6MJ3Fskf9QMJPnF0p9hXxhJc43pqPWaKY959JReCCl5W1WJQZZlaigYqAooKOTF2wBqhwzfcMwv3eXYJ/sHop7DAMYHg051Ge9HnY340n0xLDvJxTSeXHnMUs65Fwi3gtyoRQHzrSXeKbjfNcaXSjP+fmEdBQujJw4BHoIn2I6ATDZTIai0pFQSDGmNIfdMXiuGCaArW83W3TatUvyoh52IMrgiVWGYFqBOZTDx/uoP7n3nD60470gF0Jx4Ex8iWc6znet0YXwnJ9vSEfhQnLxx0D1o2tBFEzJcN2A3WcW8cLW02wOtfL0a/VRGk7xoCqwrHQuBxpPYVsMOOi0gONhT01O8DR49lVGanIC79V1cfh431n3wJgMxntBLoTiwJn4Es90nO9aowvhOT/fmJSjGBsb49SpU+Tn55+r8Uw7kgrLSb3lIU5Ub2NO17uohoGGyjtnFqGF6yPcxXfxkJ6SwG0bFnJVVQENrf1U17RTd/I07T3D1m9URWFNWIJcPOzxDJ69rqIwkEb9yb3n5aEd7wW5EIoDZ+JLLHF+a43iNTC7WCclvm984xvfiPeD1157jWeeeYYVK1Zw00038fOf/5xZs2ZRVVV1noY4eYyMxEggTwCpqbMYUdKoPh3gdw1J9OhpvDq6jKZQAL9fZdPKIg4e743av6LA8sVz2bRyPoeb+qwEdzCoc7ipj0uKzYcoNTmBbe+ctLZXFbjn+nIuKwuQNTuJsvlz6B04Q33zaVq6BwGTRnvrugVkzU6KGm/W7CQuKc4kMCeZW9ctGLcmY/ehDlRVcewrNXUWw8Nj416biRxLnIPXWMcbx2R/c7ZjhImf88WCqTjfs70n8bb/sPuMh3N5j+3v6vee3cuhpl72HOrkkuLMKT+PyeBsz1lRFFJSEmN+P+6K4vHHH+eRRx5h27ZtVFVV8c///M/ce++93HfffZMezEcJ5UWZ/G5XLifOBFBUhauqIk2HCgNpVNe0A1CUlx5FUS0MpPHizmOeIaG65j6rCE8BNizL56qqAuu49hi7qipsWJYft9kRjD+bEiuZnTXtaLrxoWL3H3bmNpEcwni/GS9ZLSvZpx7noh88eBM4PkqYKaHOcR2FYRiUl5fzs5/9jA0bNpCWlmYleS9mxAujCEMUy2CVFmTwyXULPUNC9mS0oioU5aU7jutgVOlGVIHfZOGm08L0PtATebHi/eZCSlZPJ7vqfB97ogYx1ri8tgc+8kZ2poQ6x3UUqqryyiuvsHPnTh566CHeeOMNFCVGufFFgNGWOs4cfh9/fgWlBaUs8HcTatuB5q/Al1tq/c5tsP7hhjnkBVvw55u/i+VoSgsy2LxxMU9vq7dUZ+1qslPx4IkVBED/0FiU5tR0PtATOT/3b1KTE6wY8HgG63wZ0HPtsC40GrJ9ghNLVj/euGLd94+6kb0Q8nLnA+M6ioceeojHHnuMr3zlKwQCAX7yk5/w8MMPn4+xnXdonQ20b3kUQwsypiaQsGQjwQOvgqEz5jP/1k814194OXWnSyyDVUgnc3Y/yZihMaaozFp3L4mXXBUVAtE6G+je+yeS2j+gSC3kuB4gFNJ59vV6inPTrRDTh0mUNbT2891n3vdMuPtUWL90/FDWucR4L5a7m5+ofBfGZ/PGxTGNy/k0oOOtej6M4bgQacjjTXDGG1es+/5RNrL2++xuWnaxYVxHcfnll/PEE08AJuvpX/7lXy5a1lOorRZDC5m8VS1IcP/vsQomtDGC+18x/9l6kKrKP+d3vhQ0TWdlUiOKoZm/M3TO7PwFvqxCxwpE62xg6HffJkXXuBQoSz/EYwPX0xQKcLx9gOPtA+ysaefBuy5zhLbsBsOrFer2fa28V9fFivIcrqoqoK65z9NJKJhOQsiKTCdi5RC8DKTb+AyNBGMal1jhjXNhiGLNkKfCWV2oNOShkaAlq3824/K67x/VfNKFFAI9HxjXUbz22mvs2bOHL3/5y9x6660MDAxw//33X5TJbH9+BUGf33QWKCCMvwfmtL/LA5v/G3XNfSzrb0Rpro98aeh01u5j7zG/ZaBCbbVgaIionc/QqUo/RVNfpB93SDMcek4v7jxmFdaFQro5mzMMVFVhfeU8Rsc09hzuBLDUa8uLMvH5FIezUBRzib+mct6UXatzAS8D6WV8YhkXr5DVuXqZY82Qp2K2P16YZ7rCHR8FevT5wkxJYguMS4996KGH+Lu/+zv27NnD2NgYP//5z/nOd77DHXfccZ6GOHmcLT1WTcsi+9IVnEmYg3/hFWith8HQQfGhZM2HEVO91QC6fbkYxVew6tI8ktMzCNbtRKw+DNXP/39iIdXHzliUueyMFIL1u6wqOw2V4KU3s7c1ZI3V71P45LqFFuWuo9dsvaoooIQbKYm+F00dA7S4ZEJCms7Nq0u4tCSLwZEgybN8rP5YHksXZTuoom5K4nRQRb1okaqqsOdQJxhmz49ARjLzc9NZWzkPVVEoykljfm66J/1QhAEuKw+wqCCDW9ctoL1niENNYSqzYRCYk0zZ/DnA2dMI7eMuLciIogLbzyEerTkesmYnkZ6SwMFjvRi6YdGr7fuZCA3Zjqm4x1NFjz5fOJfP9VTc53OBaaPHzjTWU1JhObNmmaE1X1ah1TsbYPilb2PoGpqh8lTrApqffp//uribosH9pGOGdwzgXSoJaQafSd6DArQd8cElS2lb8DlKeqvxjX4Ai9eSWXQp7HkfwnpS110+n7rmPnr6Rx16TiV5Zq/tp7bVYXj3QgLMHhgCB4/3EtJ0WrqHHDNpryVzIJBufSeS4Ocyj+GmAK+vnOfIzwgq7xv729h1sIPNGxebDaI0nV0HOzzpsrFWDpMJ0XjlFuyfwcTonGuX5AEf7hqOF+aZLnxUQ0VTjZm0eoJJsJ527Ngx41hPvtxSR54h5Zb/QcObWznW/gEAVybU8bGuPZGNw57iMm0vy9MVfGKFcayBH9fcgKYZfCy9ngRFRzn8Ao1D6ZHGRgZse+ekFVqy6wye7BqkMJBGbmaKo6LbOqwCN6wssuox4i2Lvb5bXVUYlQQX+RKxzURehokmcZ1jMNi+r83hAOqa+9B0A8MwQ2479rdZzK1QKNpoxjvf8Yy2Wy7ei+cvWuMuW5Q9LuPK7kg+TKhvptAuP8qYSU5zwqynr371qzOO9ZRy84OWozhxcB+jR3aSd3ovubM0Vs5qpC1kvryKEhYODP9bMUAJrxJM6Cz3N9CnpOJDM52AFiKxpwFVKUYLb6uFnYbuSkZrupm7yMtyOor5gVRWXpobZZjjGZlY37mT4CHNoLqm3ZrJjxfjjzWrdzuPhtZ+evpH8almHkUc0W54y4syUW3fn+gYsI5jQFTMPp4cezyj7SUXb5/BA5aD0nWD/Q2n8KkKum54Gm+7wzobRV/3tZrOGetU0Iyliu/5gf06i+jAVGPCrKfW1lZOnDjBc889d04GciHAwXrSQ4TaavHllnLi4D7Sdv2IOWEjryqAodOvJwNO9VjdMPMPKqbBEM6izNfOn0Ifw3QhEDQUtp1McYgEgrkoUcLGSEBVFMqLMikvyuRAYw+abuBTFT57Q0XMymQRwon1nZhF1zX3kTknJSoJ7veZA59ozUIsxpGbtSVm7aqqULV4LjXHeqIMb2lBBusr57F9X1vU9VWU6D4eXpRiOxFgIoV9hFdyhuEci2q7FwawrnIe2RlJnsZPOKxg+LiHjvdR2/w+D4WZbPEQy9FOh4GdCkbPxcQKupAdnvs6f2tOCtmp0TUuHxbjOoqmpia++MUv0tXVha7rZGZm8vjjj7No0aIpH8x0w8F6Uv1WbqK/sYYMNFTbykFVVQJ5uQwEfaQPNiEWD1pqgIOzVpAz2kj+yFHANP4B/wC3+99CwUAzFH4zfAVNoYDnOFZW5PBObReabhqvuzeVWbPx9UvNmbFXKMUdTxergR017SxdmE1GaqJjO/GAvVTdxNfuXM5Dd13myFGIfXgVvgFWqMrnM5s6uWf1bufxXl2Xo+p8Yf5sblxV7PkSrqmcZx2b8CrNwMw5eIVh3JRie5GhqirWKsM+83KvRLzox/dsKrPYZmJlEq/Ab/PGxfxia531vRZemY1nYGKFz6bDSE0Fo+diYQXFkh65UByH+zrXNJ7iqqVTz24c11F885vf5Atf+AK33XYbAM8//zz/9E//xC9+8YspH8x0Q3S4O2XLUQBkLKpE69gO4ZaoWu7HSOo+Qv7p9wAVFBWRZfYPn6Jq9A80Z62EsKOA8EqBiMZTqnLGcwwG8NaRTgzDNHD3bCqzlGfjhVLc3zv7WxjsPXoKgB0H2njo7hVRYZK65j5uWl0S9eCLmfrwaMgqtvL7VZYsyLJWH5pm0Nwx4BkqsRviFeU51J/snxDV1Z3YDoWd5kTbx7qvaUv3oCMH8a05KRMK71xVVUBhIM3zN15GxL3amSgmEj6bzKz8wziYiVRhn835fBThNsSTCcdOBB92IuC+zpWL5p71WOJhXEfR09NjOQmAT3/601YB3sUIO+sJzLxFXrCF+pKb6eroZvbCSiqpJ9gZEr9w7cFA14IUdlejY4apFMe3ZmiqMZSHqiqsrMjhrcOdjv4WItRiGIZleOwPbCGdDL/7OzT/asuZuR9oIKrBEYCmw6//1ED+3FRURUEzDFDg/fpuUpMTHAKFgPXwfufp960QTEjTae4ccPyuf2gsyuh7GWK7oOJ4sCe2xYUZzxDbjZw4b10z2LG/zXPm5R6z14sby5nFqvuwh/B8qmI59XhGwetaiW6Gk52Vx2O2TQQTqcKeiDDjhcwKmqiBdhtimDp9qngTgYmOz32dK0qy6O4eiPn7s8W4jkLTNE6fPs2cOSb/vLe3d8oHcaFC62xg+GUzuZ1vqLwwsAlf70nKU7cTi/dlACpgYKAqoON0FL7AQvoW38plA3PYHH4IrllRyO/3nGDf0VMOoy5yExB5YAvp5G/Tt5HQqTH8u+3MWvdZEi+5KuqBFmKDbaeGqG/pd4yxvqXf8ZmuE64ON0MmbmdR19znyJkoQO+Ac0WUkRqbg+2Gm+oqjhEv7j/Rmal9JbLjQBuabt6T5s4BK4GOopDuwRmPR9v1QqxiQHsIrygvnbrmvqgVTaxk/0RJCfEQi9k2GcSj5050pWMPB05EhmY84zhR4yl+t2ppgWe8fjIrNa/8lz0c+2FWSvHCjZNZSZ6PXNa4juKee+7hjjvu4MYbb0RRFF555ZWLsirbDq2zgVBbLVr3MdDGUAAfOov8HSxPbCKqQbYNCqZzMFDQDDOXAYZVuJe05i6Kc0sptm1TWpDBwvzZVngIzKStyE0IrF2SR1n/cRL6zaQ6hsaZnU/iyyoEAixZkMXpwTNUFGXy7OtHo8QAJ4L36rqiHEV5USYJ/ldOuFUAACAASURBVIji7bJF2Y6xqgrjsoqE4QUcIa9nX6/nZNdgTPnzs5mZ2l8ce0J86aJsDjSayfOfvVhDxp3OY1XXtFvXzIu262XUY+ly3XtDhSNfIphx5r69k/1Tce4wOQcTy/jG28dk8g8TNXoTkZaf7H5E7i0eiWEiqwK3IZ6IVtlE7lc8BuKFkq8SGNdR3HHHHRQVFbFz5050XefrX/86a9asOR9jmxaMttQx/PKjoEWqGw1ARyFZGaPA1zfuPhRV5WT2arKyM5mbM5czu54KOwoFrbfFKuITYSNBGfWHQxaKqnD9FfMZGgmyfV8rNY097G84ZRrX5DEuTbYdLCwX8t23s61wx4mOAc/KdLuxigV70R5EtKQ2rigkJcmUJNlb3+34zfUriyYQmjENr8+nOKixx9sjy2QvgUSY+IzJncwHk70lWFUZqYnWLNlejyEKDXccaIvaZzwGl0h837S6JKZOlXA84rorCp7J/lBI90x6xzv3eDL34zmYWD1KILK6i7WPyTiiWGy4yUqf2K+lVy2N1/GCwWiKsp2eHYvmPB5i3ZOzWQl4XeOpzldNBSbUCnX16tWsXr3a+vsHP/gBX/3qV8/ZoKYTIycOge6MgysojBWv5rLeDhh0hpJIzrCkPcwfKyiGQXHfO6SsfdDSeAJAN1cAoFt1GsdDAces++NV+RTlpcdcESQz6vxAUWkI5qJpkTHrBtaLIGo77MYtNTnB8X9UldqmHlaU51AYSLNmxS3dgxaD59DxPu69oZyW7kG2vt3svGZnQo7QgjBC/UNjkTxIGIZusH5ZPt39I1ZjJ+s78BRIdMOrNsNu9FRVAYMwjRir+RNEwgZ+v/MFdF9rVWhyxTDqdiaUl3ihGJ/qojpfWpLpMFz2epGdNe0TruYez3CM52C8epR4JWrFuYt9iv9PdKVjN3qKqnCs7QNe3Hk8agU5nvOxJ9S9amncxwvZKMp1ze9bxaP2d21Z6dxJhUzHw2RXKpNx9O58VXVN+3ldXUyqZ7bA008/fdE6iuTij9GnJoAWJFxCB74EAsuvRutt4cyORucGZ4ZA9YdZT0r4/5E6DH9+BWNqAujh5LflNMzv60b91gMgCu2GRoJRzB2BxlAeiP0pKrPWfZb82Uvx7Y1UVasKbLpiPilJfssZ2I2q+wELBNLp7h6IMj6FgVTHsXfsb+NE56BjVaIqOGalmzcu5unX6r0VbMOzaWG0a5u95dDBKZAo4DULFrUZTqMX2ae4jGI/4gUU8WvxAtqR4Df322wr9LMbMpSIE3YksVWFkGZYdNzSgowoeq3dSZQWOOtFJkqlhbOjn4p775aIAdNhmdc9NsPHTR+eaBjQzl6zhyztq6jxnM/QSNBSK7DX0sQKBz77er21WhXFo9kZSY4V7r6GU2DgKQvjvmYfJpTkhck6erfDnYpulZPBWTmKi13rKeVmcyWgJKVhjA5a9RTaqRMOKiwAho6/4uOoadkoSWmcqX7GNOLhOgxfbikpNz/IWP0uQrVvRrZTVDoSCunpHEVVsKqzd9a0c1e4JsHNWALIWbyEpMW5BI+8iZI2B19WIaW5Gdx9XRmvvXOSjt5hdANefeekRa0VsMt0KAp89vpyx/fOGL3OnLRZQMRYJvhUZ1JbgWWlc9knwmIhna1vnYgpc+6eTduNpBt+nxJVt+GeBQt5j1hOVaB/KBJGFC+gcI7uF1AksAEr+SwMib1Q8dnXj0YZBHHW9rMvDKSxdFE2pwfPsH5ZftQLvaZyHjtq2uOuKryM1WQT3bVNvY7ZtLsy3sBMvPsPejN8vFZRk4nPd/ePRD0X7vO1h6HEvRIoL8rE759YOKa0IIPi3HRHWNN+zcR7JcxYKIajnapQkhfOJk8i9t3TP8ob+9vOa43KWTmKi1nrCbByBw5BQFfeIgIF39xiEi+5ytzWJiQo9uPLLUWp32WTLVcYKrySb209HWXkdN1w9FxITU5gz8EOi6XUUX+Qkc5tqIYGp2C4+QC9q+7n2ddPO4yorhs8va3eojU2tPbz7OuRmb5hwFOv1tHdN0JH3wh5mcnssNFWVVXhxlXFzErwWVLmje0fOPIo92wqozCQZgoQhl++rj5XaIxI6MsdKwas/Rm231aVzqVyUbaDJSTqQuwQbCbhaMFc4ZQWZDhYXbHCC2KFsmRBVlQx4pbdTRGJdy26zsRdW/Gj5w9Erm1YcgXgu0+/Z61qTnbVR9FMvVYV9rh6PGM4mUR3TeMpR7GjO/xnhJ+7zRsXW/1NCgNp5ooiZBY9CpqyV+gjnoSL3cEr4f9EEvuR84XYyf2JhGPsBnNN5TyL9aaqOEQnX9x5zJLlN58ZxdPRns2qbaIrrViOfjwKtbimU8W8mihiOopt27Z5fm4YBro+eTbNRwmCFose7nRXtjYqb4HiQ7CZzlQ/E9WoyL2/YN2OyAeqj3p/BSEtSLiMAV9YPkJRFXr6TWMruma9V9dlbVrq73D0yTD0EP2NNQRDTqYSgG5EDJZXHF434PdvNUdtp2BKVQC8XRs5tq4bfHxZfpSEhXj5hNFRgJJ56RTnplOUl27lQ6pr2qmuaXfkYBQFqhabjsEe1nAbAYgU79lng7puHss+e8yfm8qx9g8IaQZ+n+LJyKpt6nWIILp/l5qcEJltG+bfsaisv/pTg4uxZhqe6pp2h0SLVzgNIlXo9rj6kRPvc8+mMisMGatr3ERnkpWL5joMkzhXe193uzBibfNp1lfOY+OKQra9czJSy4LpzN2hj1hG1Z6ENjc2BSxff6/FIXVSf3Kvo0jUSysrXjjGy2Cqqoqm66i2iW1pQaSfvWDxudmF8fZ/thTe8Rhz8ZytF6ZCoXgyiOkonnzyyZgbVVZWnpPBXCgItdWajiGs+WSAIy+QUL4egOCRP5kbaEFzG3A4GCEqaE9o68BI4ZXkX7IU/77ISypi4nZ5bfGgrCjPsWZADaE8DHwY4f2FUDmTVYqqjjrCQhCRu/CqVoaILLr1d7g4UBgSd/2EqiieD6b95YucT+TlcyvTqorppMC8xAcae7hxVbGnERDVwUV56daYhkdDloMzgIqiTFq6hxxGcE3lvLiFfTWNp6JEEO1G2F7YpwDNHQMx6yDcyf2iXHPV4D6+quA5+/Oa6eq6wVOv1nHP9eWO0FhP/ygNrf2TNg4VJVnjtiL1Yqmp4T4odujhVQlE8gxrKud5Gu2opLMBKUl+Nm9czJNb66znL2SbDAiHeTjsQNy5EbvRjUVPtodRdd2IcrATWY25fwfx6czjraq8VoX27SeygplKheLJ4KwcxcUORwJa9ZNYtpbEsrWOkNLYke22LQyUpDSng9GCnHn3BWZd/in8+RWcUfzoeggNlX8/PJsi2qNegC27myx5bU3TaTtygPld/SxIKGT54rnhOHc5B07lM3RkBxjw7tgiLvPP455NCTz1ah26YS61N7h6Y9uNjdB9snfIA3OmJyiwYjtRP4Fi1iHEQryXr67ZqUyrG06qrlj52B1LXXOfNZsV1cGbNy4GTJaVPbGZkuT3nJ159bAQ+84LpEeJINqNuKgdiVeRC6b4oDtlNydtFg2t/Y78g6LAPdeXxzVKn1y3kMNN79muC5Y0irtHx9kkML1WIO7P3LkxI8wiE8l7cFKs7XkGr/vvrqQXif7qmnbnJAUsB29fnbpzI3ZhSWF03fRkVVUcgxTHHO9axLpmgIMEEI/CG2tVNZEQ1kRWMGcTDpsKnFWO4mKHSEDbcxTuvIN26oRjG+3UCRLL1poOJsyY0loPMdxRT8rND1Kfdz36iffYP1bMsbEAx/a1keB3zkrsD8rCxFNUNb3GmWMh0gyV0wObaCGXBf5uEhI6eSJUyrGxuSiqQnH/KKnJCeYLrRlmN7hwRTB4G3HxYkGkn8XtV0fqOsTMzG6k9jWc4uDx3nErcd0waaJmmEjgyktyeae2y+q/caztA36xtdYKS7mlv4MhnaderQsLMir4bPURXiycWPx9YUwS/Cp3X1dmMZvc18trNiniwnaaZ8iVoFVVxXGdHrrrssnRGF1O50TnAGvCirX2SUQsA/Fhi7LWLsmjsbXf6p7oDzPARC2Pa3EBRGbsXlph7mLNuzeVWedlx7LSuda29tWpm2FmF5aMZYx1W84LID0lgT++18KLO49ZveUnAi/atbv+wn69U5MTUBUFHWd9xkSJB+5nrqV70KGvJujK06GhJR1FDIimRe58hQgnudP5ChEHc+bdF9BaDyFosmP1uyhrfxP8Gov9nbRrmTSFAp4xZ/GgLB/rhboQCgZ+NBb5OyAEGbufwofG/WkqdYkfY2t3AW/sM6xWqWCGDbwYKl6GVIw9Jcl8FLyWyRM1UmJ7+4MuEqMbluZbCVtFgYJAKtesuMwqdBMxfrtCL4bhWnmY/xfJWHeuxH5s9wuVmpwQ1YN8aCToqKCOFxpoaO1n7ZI8+ofGqDnW48xJAJcuyCSQkexgowhKZqwxuq9hXXOf20/Q1D7A9541wy/jJT+9DMtEnUWsepKNKwopDKTx7OtHPZ2EvYAwFuzxdIjOl/l8JnFCwP4euBlmRTnp1J44jY7hCMW52Wuijgagb2Asqrf8eM7C63q4n7uW7kHHKl5VFMuh2MUrvSYdsSRNxDO3fV8rT75aF8XMuml1ybRoaElHEQNCxsMY7HHkK0SPioSytWaCOhyeSihbC5jOYtbln2K4o976Tu9rRTE0syQDnWuTDvJ/h672jDmLB6V192EIVxErQLIyRllip8l2CksLlo0eYGH6IX48sImmUMBKiHvx/L1meuLFEsVnEFvoLpaaaCxDZWe2HDrex41XFjlCOfaltD19IooFRV+IJQuyHEYZIjkUu/G1azv5fAoP3XVZlLGxmDeKOfMX1368qt9YUhwCfr9qsXbsqw6vqud4MW777Bubw9Q03cGE80p+ilyC7jIsXjHuWOFBrzzWnsMd1Db3mWNyoawwg/y5zlobd3W8O57unqBcuiAzig0nthdkDsEws5yGHm4KZuBolyscUlFeOk9tqyMWvGRq3PC6HvbnrqG132HIdR2sgJ2HeKWdsTSeEGBqcgJPbat31StFwmeTITFMFWI6ira2NvLz8z2/e/PNN9mwYcM5G9R0Q+tsYPil74ST174ww0l39Kjw5ZaScst/jw5JhR3MrDV3WTUYo7uesfatAJdkBvlC4AR/aE3hjf2GZ8y5p6uHVMAXNhhXJx9hqPIzKIcPYmim41IV8Bk6pf4OTuo53B1mycTi+dthn+WsWlpA3+lhtuxuIjU5IapwDGDJgixTRsSmJgrENFTuqXFtc58nU6O8KLph0l3XlVm5GzD7f7vrHOwvV3SVsVlgdW+4sZO9qE7B7EHe0j1kGZmNKyKCeQYwPOqsNLc7ErsUh5dwYDyuO8RXHvWaSduds3uFY18hucMthM/jF1tNksVN6xfRd3o4puihmAy4VxR9A2P0DXjRws1EtT0P5M4fOKXunZMO8Wy6nUS8lZ2jONLAqvh35zH6h8aIR8x0y9TYYTfW7voae3iyrrkvasJgJ4PEWmHFConaQ1xKeKJn328sZtb5QkxH8cUvfpHf/va3AHzpS1/iRz/6kfXdD3/4w4vaUQTrdzkqqX1Fl6GkZESFm9w9te1hKmxhqoRLNnBmxzHrdwkj3Vyqd7I4VeXHA5vI9/WR8MYOxpavt+oxMhZVYnT80aTMKqBiMDdJx3/zgzS8uZVA7/uohmFJlt/tKq6L1UMBomdtPUNBz0Sgphvsre+2qIwCgroYyEi2dZHziEvY0NQxQFP7AH6/k6lRWpDh2TBJvEAQmwpoN5Zu2Ivs7LFjv08lwWYQNU2nuWvAwQDb+nYzGDjiwuNJcdjPR8wcd9a0Rznc8eLL7tmil9S3m0UmnJYCVs7EcFGfdx3siOpR4hY9FNLimleMCVgwL53mzgHTCCvQ1PGB4zq68weAZ7V6PNJDPEfqDi8pEF69OlfQHb3OvvKZ6YmUz89kYGQsKkfhtQISPdKvD6sbeK2ONq4odLD3MC8JxXnpnoWVXufg86kMj4b47tPvO6+5YVi5EFGv5DVmt+rCuURMR2Gvvj558mTM7y5GeJ1dqH4X6EGC9bscvbQdv3HRakWYShj/0LF3ISER7cReFAx8aNyZsos83wcwiCUPknjJVRQvqaKj42pSjv3JDD/5EqyVS/vi2/jP1wOU+jtoCOWRV7aEoZGgZwjLje37WqPi2C09w56JQMOIGE339Tl8vA9FjY6pQ9hwheO14u94VbDumbLdaSlghW/sDiZWTF2g5liPVdQnwhWqqrCiLOBgeimKYjVUilWxe9PqkrhSHLHCOe7Fld1IDo+GPJOr9n3FkvquDjOpBEry0tm80UwS/99Xjjj6qgsEQzr9Q2NRrCb7fsXxINxoy2YI/T7FOsbv95xg79FT1kpDzKLdjamK8tIxwhMA+3MSj/QQy5GK62KXVrHX6IgVtKIqdNochU9V+JtPVcZM/LubfdkptdveOclDd1/mWM2IGo9t75yMytnohqlV1tId3b/Dfu6CINI/NMarbzdH7UcQCLycgFcBo/33saTVPyxiOgp79bW7Evtir8xOLFtLqG4H6BqoPvNuiKpsmwNww6LVakFzadx9DK2zwXIWiZdcZa46Th4ELYiqGOT5PwAiQoOhY+9av0s9sROhN5WwZKN1zKGRICfCbVRL/R001sNbRwKeejx2NLT285TIIRAxhKuWFjhmavYQkmFEEsw+FYpy02lqHzBffA8bLaRBwIwFJ/p9LknyaGmOWPQ/u9NyzzDdMeQF89KZkzbLOpZg4gDWikPXDepORlYqYNY8iC528Sp2xW/ctRFejlech9VrWzcclcypyQk8/4a5wjx0vI+acB0JRCvUuhPyW3Y3OVZLAMW5Zu+R6pp2OjychEDNsR7uCjO9dta0W+yd4dEQP/jPvRTlpDuOt3FFIbXNfcxJm+WocxkLOZt1zUlPtIyxfSXrvgbi/omcEjhXibFWGzEnDwcj11vcm7qTpx2OcumibFq6Bz2dsnsFA84e6XbatrWq1Mz8m7tmyY6J0FZ3HezwFKIUApaxtnU/924KcSxp9Q+LCa0oZhrs+QclKS2s+BqGolp5Cq/tZq25izM7fgHoaE3vM9x8gJRb/rtDziPl5gcZrX4GvftY1D7UuUVAeHViSYYYBA+8SkKJqYC5fGwf7UmnuC35HXxoaPishLagkHolS70K6FKTE/jju80OGQs3m8PsfWE+lAk+NbJCUMwmTe53prljwIpdq6rikP24oiLHMqyqz7licNP/7OEF9wxTvLhiZn2ya5D1y/KtfIb4fUv3oGM2e9oVb1+/zMzDlRaYdQxCqFDEhQGHU3PH5L0cb2lBhiPc5U5su8UW9x416bTumL49gW1vRasKqXbdrDwvykvn0Wfej6LquqFphsX08ipeFKQDISYZyTcMUbko2zLg9gJQgA+GIolb92rBro4rqtvtYTO3SrDXaiPW5MEtze5lfIdGgg4FZIgwntwrmDXhPIR95Wh/5kR40qeAIhyKK58A3jkKe7jovbquqHCp6hFi8oKdWGIQzovYQm/xJNg/DGI6Cl3X6e/vxzAMNE2z/g1m17uLHSL/MLLj5w7JDN/8pTGlOgCM0UEcU+0YKxDdVYdhYWwECK9OFF/k2IZO994/kXTyLdIMjc+kKCiGbsphhxPaTaGAZbTd2v+pyQlRPS82XTHfofQqZCwKA2nmqtEwHElBXcehoWQYoPoUyubNpqG133JQEEnaCkohmHmDt4902sYYecHEeN30PzF+9wqptMCpkSQ0suxaRSJM47g/wPLFc0FRqFyYFfViipWdL6ymKoyaz2cmNO36T+/VdUV1/hPcenu4a+nCbIdw4uhY9Pujaboly27n4YtztreidUupVIdzIePBbqzFNX1xp3Oy0tw1wFfvWB4VanHTrZcvnmut3gzDW+n393siApGGAc+8Vs+6ynlxK+K9EE19NazOhW/ub6MoL53mjoEoJ5HgVwm6mEt2xpM9DCQQq0e6yc6LnMvSRdmMhTSKctJ5/b2WmElvO9PJPT47IUKE0byq7t2hTTvhobljwKJsi1XtuaitiOko6uvrWbVqleUcrrzyStsJXtyhJzvcZ6qmxPfUxphr6e+xAnH0qHBvH/6/L7eUWes+a65mDB1D9bO/oZsrE0JhaqBhddHTUGkI5Tn2I0IV7nim6jN7XogZpfulFS9NrIRm1Hh1g8pF2Xzm6lLHw2svTEqe5Y/SC4qCjXbrnlXGMiJFeelWQlFVFccsuP6k6dB2uByFTzVFAgULyM5ushtcXTf7bIvro2kGbaeGHPmb9OREB2NLvBb28ICobbHPAjt6hqNotkLKxIuH77UStIcnvKRKxL0uykmzdLAUJVqKZOOKQsfqQDCC7MYZW75JrJpuXFUctXoTcK8aBMS1jVcRb0csqY7qmvbIBMEwxS3tJsmnwvql+dy0fhE1R7usFr8AiX5flDEWpIMdB9pYH1Y0EKEzwMEKEw6h5ph5r+pP9jvCvRBNgY5FPbbnluJRZmMxwcSERLzfZYUZ/MVtS89vjqK2tnbKDzZZHD58mEcffZQnnnhi2sYQq14iFvRTTt0fNbskajURtVoQUFR8c4utP+1J8MNGCW/Xalye0AiGjobKb4evYEGWyuyFlbS+dQZVMfn3xbkm88Ld18LANFzZGUkA4U5fOOoYdta0U7kwtlSHY7i2YivxYNtjyRuW5Yc57fVxY7oivPTUtnrAnNWNV2G8fV+rFR6DiGGPsLDMGb/bWOkGvLGvjZ1h4yoYOddfMT9KPdcts+6enb5T28VSW1tYw4isfuwhF5EbeK+uKyKc6EEQiMxYDUe/BfdK0E2VtEuFWNdUVbj7OlPd93vPRjS4zHOOhLdSkvzce0O5tQpzz7bd4SnDMGm3dc2mxEpz14C1ehNwT0AExIrVrsPlnn0L2J2NqIsRdRUAbx5ot+USsGZYCqaTuPeGCgKBdMto7tjfRnPnYJS6gH1yoOlm69wdNe0Wg8weEopFfRYJ9Rd3HiPR74tJCXavKIpz06MS5bFycbG+s++zvqWfpvYPyC6d2Ps7GcQtuBNhJ7/fz+DgINXV1ZSXl1NcXBxvsynByZMn2b59Oz6f75wfKx5i1UvEgn/h5WitB62/Ey6JphH7cktJqNgQERUUMAyHEq3W2RDubxGkTKljm+86/vfAJhb5O2gM5eHzKXyiRCW3LMADCwPUvvsugycOcbQ7l2dfHwpT+CId5oRhFysNYdDn56ZysnMQMA1uRmpi3Lap87JTuC7cqtW9PLeHnLIzkkwmjWM2bMo1HGg8ZTko8a2QRgc8BfgErKS8bXyCcWI/zxXlORw5cdpx/AijKfKZrhsOdpdQz11TOY+aYz2mwVBM8cETnYOOZGdGaqJnIaFXWMyuWiriy17X2K5Way/0q1o8N0o8UTjUu12OSNcNtr51ghVlOaxdkkdyciLLS83Ernogkjs51vYBGamJcam+W3Y3OT63rw4VoLb5NM0dA46aDMdKC1hsK86zz4hFbsXvUxy5CnvYyt3QqbTA2RBKdeWy3EJ5V1UVMDQSpCncItiuTOsmBojj2Z+NJ1+ts/Yjxm2X+R4eDVnkBAgXjLrCh5s3LnZMbHxqhAoej+013nfuZ+g/X6sjI/ljU56j8H3jG9/4htcXDQ0N3H777ZSUlDBv3jw+/elPs2fPHp577jnKy8spKiqa0oH8+7//O//2b//GCy+8wAsvvMBnP/tZVq5cySuvvMItt9wyqX2NjIzFNHLjITV1FsPDzodHTcvCP68MNS1r3O19gRKUlDmghUis+oS1KnBDHz6N1rzf+jucG8YwDNTZARRV5cy7L2D0dwCGWcW6pAxlwUqyF1eyYHaIW0d+y6yeOoJH95CdE2DekWco9bWyclYj/VoSb55MsEIZd1aqXJvVwnUri2kdSuRQU68lcb6mch4nOgZM/rZP5c5rF+NTFJpsHd7CqggAjI6FuHFVMasuzSNrdlLkN6rCnkOd1n5uXbeAuRnJVB/qIFwwzmevL+eOaxZzeuCMY/92hDSdzr4R8x4aBoE5yZTNn2N9v/tQB4eb+jy3FdW+n7/pUi4rCzA7NZGDx3qjngdVCRtq+70LGxy/37wGpQUZpNu2P9bxAaX5s+kbOGNpTq1fls+NVxYTmJPMresWWC9oanKCScO1XYvSggzSUxKoaeyJ9N8IH9fNM1YVhcNNvXT0jlifdZ8eYd3SedY1376vlZ+8cJDDTX0cbupj08oiDjdFwlRDoyEaWvtp6higufMDinKFjlaksrm9Z5imjgGqD7ZzeuAMqckJjntq3dfwudilYgQMw6yTqT7UwemBM8zPTWdd5Tx8qkJJXjobqvLZcaCd4+0fsOdQJ5cUZ5I1O4mXq5s4FnbuoiJ/WelcGlr7+fV2ZyfJkrx0lpXOZfu+Vp5/o5H8ual8YlUxOXOS+eS6haytnBd9D8LvckNrP/XNp01ig2Fe6lOnR6k+1EFn7/CEbMXBY71cWpJF1uwksmYnkZ6SQFDT2bSyiNrmPrpPR/qw5GWlUJSbxqaVRVxWZrITDx3v5ciJPuueb1iWz9WXmYWeWbOTuKQ4M2r8E/luLKhZNHCA4TMhdh/q4JLwWCcKRVFISYndFjbmiuLRRx/l7//+77n66qt5/vnnAdiyZQudnZ18+ctfZu3a+CGYyeILX/gCX/jCF6Z0n9MFQYWNBWulYLcO4Qc4iMLAqMpsR6MkBVQ/uRVV3JRbAsCZvXsZOx4ycxVaiPa9O8g0QuEZhsFnUt6ifSCT43qAYrWLy1u24UND6foTVUs+w+98KdYs5ZrLi1heOjcq1GOfNdmlNNyyzeDNcycyesA0BKKi292DAdtv3Hx8dwzbLnXhfsd9PsUxO7bTWncKJVdV4W/+bCkDg6PWLM9dES62HxoJ2hr2GNS39Ju6rIUfPgAAIABJREFUPkRmm1Wl0TN9L6pnQ2u/mQB33nbWLzVnlkKCBIiSLQFzBSNotkLmQTiFYMgMtW3euJitb52IaiAVCkUK4sSB7YEQEXbxUglwJ1BFxbj72ruL+O69wczNxQutuCGKKJ3yFebzsn1fa1QPd3s4arxaCUUxqbx9A2NR1ewK5optLKQ5cjb2a2+n99pzYSvCzkCgs2+Ejt5h6k/2W/UUXgwrO+yrcve5xKo7Abj96lI6eoedLWYnQBCYLGI6ivb2dm699VYA3nrrLa699lpUVWXevHkMDg5O2QBmIoL1uxzd8sTD2hDKYcvICq5r7uRSLWgZWGV2DonLbnSEvUTNhqGFCBoK73cncW0SGOGZsopBWWInTaEAlyc24jO0sBPRmX3w1/zDDX/HwYE5lBdlUlGSRXd3QlxDl9B3nJzWg9SP5dJCrsXuETFsIQdup7yKamBhaO0ORuzf3fBoXZh1JQT43N3p3F3pACteLLaPFUIRCfzyokxWVxWye1+LyaIJGwyvIqmofgo4VXANwzTqBxpPWYlQL6pnLOaLvWHSic6BqPadEEmSqzaarVvmAUT/hn42rih0VGWLVZK9sDAWtdmre504F8CaDIi4vJeqrNsZeIVPxCzYF1YVVsM0X/c1ssuzuxladgaTV7VyIJDuDIcaRIoEbdXsogI6IzWRorz0qHCleZ9i66ENjDgjELo1sdCjnncxRrdDmGzbVTtuXFXMgWM9EyIInC1iOgpVVa1/7927l4cfftj6+8yZMxM+wODgIHfeeSc//elPKSw0l1ovvfQSP/nJTwiFQtx3333cfffdMbd//PHHJ3ysjwKiut0RTuYCdaECmo0c/tDaxeJUFR+aGfL5oIvRXU87uuiJmo32vTt4rTNAinIGHfBZ+1RZuPwKAocNOOM8FoZOXrCF4tVVUePz6sSldTYwvOcxbkwKcX2Sj/41XyKId9c8N+UVYstWiNoFe8Oj5Fn+KEmDN/a3ccPKIgKZyY44r1gF2Ps8r6mc51nQ5ZZq+NUf6mlu6/csCLPDLe4mrmHUbDrOjByi1XpFB8BYiqoCfp/CdZfPp7lrgES/z6LZ2mUe3AKC9gR1UU46KUl+R8WuqMdQfArLw8QFQa+MJWZo1yKyf25XlRUOzes+242kW7r741XzogQDBeyrUHf9hmBoxapW/taclJiJ5MCcJG64sthabe44YDKp/D6FlRU5vHWkEwzTgbn1vNyOz74CFrU/mmZECWiK7b3yMuPJl4zXIlXI4Ihc1Hmro8jIyKC2tpbBwUG6u7u54oorAHj//ffJzc2d0M7379/Pww8/TFNTk/VZZ2cnP/zhD/nNb35DYmIid955J1deeSWlpfGTxJNBdnbah9o+EEifopFEo6/+OMOGvTWkGv6fwsriBOYmqfzn/gA/HtjEDUn7KE9oR1UMdC3IwI6nyL3yBrSRAXzJ6QzufobMUJDbUpr47fAVGJhJNAVzZfH6uyfpPjOXlllmbkUUySmKytxLLyMpfJ61Tb3UNJ4iPSWRn71YQyhkKso+8tdrqSjJomP7NtDNZkE+NIpHDrNdz/XUWFLDiWShSnvT+kUmTbHxFJWL5lJR4szzBALpfGtOinX8n/7mQBSNVugWubV1NM0AVeVbf7PW2r/S3ci2322jaSyXplCAXQc7+MtPVVrnpaoKiqKgWcWAkbGuWlpAIJBuXY/KRXNZtbTA0XdCVRVWXprLnoMdnvc3FNLZ23CKzPA5iXNetbSAl6qbrDFUlGRzzeXzqSjJ4ld/qLecqhvz5qZaPH33eP/ik5UMDI9F3bdVSwuoKMni9usqrHMR1xpVxQjLghi6QeXiALdfW2b9rrtvhFf3NFkG6497W3nncKfjnmiaTku4+tk+blHFv6Iih7qWfn7/VjNrluZzw+oSAoF0Muek8I8/3UUw6GyMVJSfweqqQjLnpPBSdRNjQWfv95aeYVZXFXL7dRWkpyVRfaCNhQUZoKr0DAVp6Rl2jiM8rprGU9x+bRl/ORLiJ8/vdzw73X2jPPeHozzy12tJTk600X8Ns/2vYd7rv75tKTfYwlvuZ1bc38rFOdbfTe0f8NPfHEA3DJ77w1EqF+dYz/2v3jhmPUshzWBvQw+rqwqjno/hMZ2eoSAVJVnUNvXy/ef2Rr2X9ud0dVUhq6sKOVeI6Si+8pWv8LnPfY7BwUG+9rWvkZKSwn/8x3/w05/+lB//+McT2vkvf/lLvv71r/Pggw9an1VXV7Nq1SrmzDGTk9dffz1bt27l/vvv/5CnEkFPz2BcOmY8BALpdHd7J1mnAlrGAlD9Ft121pq70E6dIFS3g5zudwgoe3kn8TqOjQV49UwVpQmdKIZuau/0NnHq9z/FtPYqGDoKBgmKwfrCEL4+I1L3YehU+Ro4ylzSlDMY4VCDYBKePj2Mb9YADa39fP85wayJNAoKhXT2HGhlzuAJho++6ziH0dEghfkpUYlNESYArNnsngOtlBdlclU4Du91bbNTE7hq6Tyrw18suL/y+RQKs1PITk2gMDuFQ7v3UHnsCW6cpbFplspjA9fTFAqw/b1msz+zAYZmYEWmdYMNy8xx9Q+NsWVHI398+4QVRhOSHA96iBa+V9tlhXAWF2TQ2NZvFYFte+sEr73d7Jh9lxZk8LU7I02gtu5p4vV3mq2cjhr2gvYZqQEWGw3MsedlJ5OU6GP9snwykv10dA9QmJ3C1+6MhAmzUxPo7h5whDMS/Cpfu3M56OZ9VsKsnMLsFLq7B6xr2NzW7xjL24c6PK67uZ34t2HLV+gGDie6t76bgcFRrqoqYM+BVqtPtntfYgzXXuYKmymK9T3AitJsz9ySfRyC9Va5aC6797Ww/b3mqIS1gZnX2XOglZEYoSPDMOjoHrCupVcRqLjW4hkG2NLQHXE84WOI1Zz7WCMjY9b2Xs+HWImJ5zcY1Hni5YOsKM/xZAaerf1SVSXuBDumo6iqquLNN99kdHSU2bNnA7B8+XJ+9atfUVJSMqGDP/LII1GfdXV1EQhEkj85OTkcOHBgQvu7GCAkPMbqd5kz9KxCs5rb0ExWCRqfu0xlb+JCUpPLeWdnI6sT6h0FRWbmOyzhCSiqSnFeOsE+50/SlREUBUaUpDCjKiz0put01u4jP9cskguFH0LCVEPRC6K8KJNQ247wsSJQ5xY7KIoihHDPpghvPxjSOXS8zwoDTCTmak9SK6rCJUVzOGRjN9lDLfYEsjCIn5pVjS/RzMUohs4ViY20GDmeoQFBpRS9C7xkqe2igOBMNLoT1b/YWmujw2I5I3eFPGA1gbJXPIu6E+GI7Lkb2y21dIyaO+vMlZHNGd3kmvnawxlC7mLXwQ7Poj63ntKGcDW9OCfw1iKyd0DUtGipczBrGK6qKogqWnOHdMCsDLdD9B8XcPeBCGlm/Yw9b+IufoslHCm0vFq6B63VqpAKN8CRTxHhIp+K47p79fN2F3naq+GL8swWvLpm5vNEqNRyPB5NwtyyHYeP93HkxGlrUnc+WqLGdBSnT5+O+vfChQutv8WKYLLQwzMaAVNGe+ZUegMEm94nVLsdDINg/S5mrbkLbD267eym1pEqqKm3kr0mFFOsECNslQy0vlYIh57E2/qxxBauyhrl2sJsqIvw9g0UGoK55BOOt9qMs+inLV7gsQ+iZxmnuk6x97RZ0fzQ3c5Wn46eATCpB1nwze3FX9v3tVp/u6UVRMtW0cs4XYlQSVFg4RyDB681Y8D2bQFaeoYpzE6huqY9bu8Cez2DV3WsgGBxeTkjd91KxOFFKp51zaw7Eft0tgM1fbXdCJuXODJrtdcZCNgNs6oqnOgciIQLDYPmjgGrMt2rBqa8KNNipnnJXYt7JogCwmG45UTMwsX4fdUF3HkIIf8iUNcc3QdCKLa6JyPbD7RH5YVOdg06Chchkmexqx7bHekvttY6ivLs190tbeKuotcNePq1ekcBXzz5cq9OhuK6Pft6PcctQc7oSd25RExHsWrVKsuAuwUCFUXhyJEjZ3XAvLw83n03Esro7u4mJyd2I5GLDWNHthPc/0rkAy2IMTro6NFtZzdlhboJufaREKbeBmu3mx8YGnpHveM3igI+w+C67FYCFVczdPRVNC2IgcJvR1excO5Cy0g88tdr2bKj0eqLbZf0NrWrIjAUlSfe12kcO+Y5k3WL9YF3c3sv2JVY60/2s8DfzWqjhfVXVeDLjVQMQ/QM2KcqDBjJkfMHihYUkuzBQALInJPCngOtngVX1rmGX/KinDRHxbeX03MbQYiEJ9yidlWL57Iwf7az4plohpXow2FemzbHd+Ei6yjNI7shF2MSBtxekOhOWsdqtRqrF4gbbofx5v42S1TS3eY0Fo1VJLuXL57L6cEzrF+WH1Wl7y7mE/ASw6tcNNeaiSvhmhf3ZCNWMySvLnUCYtXh1UnSPfuH6AK+WPLlXp0MAauhWHNX5F30hUNubir6uUJMR/GpT32KvXv3cs011/DpT396ypLNa9as4Uc/+hG9vb0kJyezbds2vvnNb07Jvj8KCB1zxvtR8FSjtVqxjvRHfaekZePPr4ii2YZ3h6FE/piTmogvt5TUWx6ie++f6B84Q2XeUv7dFt/81t+sjdkX26ThhnMqikr9vE/QeCgrbpc2e1VyLMqqG24J9Ct8tcypfpsxDMZ8CfRe+bcWnddiidiaJi1fPJfUjPVwvNEM47nkUMQxhDF67g9HrQY1QolVVeD6lUU0dw1YYR9NM6Iqvr2cnhcrRfy/pXvQCmcYmAyjG1cVU9fcZ32uKDhkO9yzTHuLVBF2c2sePb2t3kHxFWPqHxpzzPJL8kzGlaAVh0I6zR0DUQbK3cZ0IvCiIo9378UEwV7t7ferFAbSPFdzd19X5mC/gbejrSjJspoxieZPXhOb8dSK11TO480DbdbKU1VMmZCivPSoTpLCOYueHV5wy5fbj9/TP0pLt+kQWroHrTyEyB+K61O5MNuSixdU9Ac2Lz9nRJyYjuI73/kOIyMjbNu2jUceeYTh4WFuvfVWbrnlFitncTbIzc3ly1/+Mvfeey/BYJDPfOYzLF269Kz391FDlMTH0hsBrM54Y6goGbkYH3Sa8QbFZyWuAasda0TS/ImoY1h+QvGRULYWrbOBsfpdpLa8RaqhkXt6H4Vcx3EjYLFD4kkF2Pc839/LokSdY2NzY/7OHobxKi6yQ+tsYPStX5JxqoNPzJrPyyMrKPF385mUt1DCczJDC7Lnj2+ybWSJZSxSkxMcoZgDjT3c+olM/NoytOZ9UXIozsIrc8kuwiwbbEqsIqRll9oQ1zRWV7vx+iDbKaQQkaRYUzkPvz/6mrupkmKW6dZHKspLj9k/wT4md6hG0HJ32PIKOw60WffOa6Y72Rh4vCIx97Vza4HZZ+hAVJ4lOyMpKhciHK3dYQcC6TGbP9nH+f/aO/fwqqo773/33ueShIRASEggIYEQQkDuoBIiFykDCCI+jtNaqtaxT/vam50+b2eKHWfs09FaHadOL/M8tTPzjrWXp1q1XhiKWhXBgCI3I5YkBEJCQnJICATCSXLO2Xu9f+y99l577cs5CQmXsD5/KDk5Z++99s5Zv7V+l++PV2Nl7wN9z7I5E02jTAgwLjvNoTQLWJL0pRNH4+CRTud1wl6TAcCsGfrk2GnbAosmixDjpNTVREUJE1wqel3zmWHLfPLVekpPT8fGjRuxceNGtLe349VXX8W9996LyZMn49///d9TPsk777xj+3nDhg0DluUYKbBCf4HSRQjNWIH+A1vMzngEKshZ648F0BCoWG7+HCyvMl1TvFvIgSRB7WpBf/VvrNauAGRoKA9F0KTmmdkh40YFHZLLgLMtbEbzLnwjK4iDk+/FxBlzXCcDL180v+pWIw2IvvooACAMYFXap5AApMt6Uydq8AgI6mP5tkmLZ5J8Ctm7fwuVMI46Q+K9MZFn6y9tfulgaQPxk8fnV00zheSoH5g3EnQ8NEbiJ+rGQqALE/bFVM8+4m5Gm+23QFfda66fZHRbs/dPYI0NC9UY4nd+rCAelb1OvnCw34dUdg88vE8fcO7cWIHFnTVt+MLqckdPhgAXCwoo9jqKZO1nAf8dFL/4ocehBtFtFxhwUQ+YOcVabLCfATg1YeiLGNlwcylM4JyKErIMd5zC11CwdHV1oaurC2fOnMG4cUOvTngtwUt8UDVZQhIAYYLOEiDJAYQMxdrESbuib2BiBWKu5V8GmorYvldtRgIAJCWAxSuXIdNWma2vpNjGPN9bOwY5XHEgAEhExY3juhF2mRT6PnwBicZ9KJqyEGWVnzVfd1t1TzxkxWqoy2xJdhtIwQxIzfXM7yQoigRZtX8hWF91eSgC2SHdLqGzT7YXYxmTyldun432jvOukxsr0SDLEpbNcXYdc4uRsMHr/919HLOyzmJh90cIjOrAnr6pOK7mmZMBAcyWrEGXPuK8ofVKEshICzgSCgBngxt6j5fOsfo5u8moUBmOoE87Tv5epVpR7GZQ+Ew3NtBL3zOn1FLoVTVic5Oxu4Dm9vM2g/3J0U6smDMhaQAdcFdppa/zfSDcjuO1C+RdUKzSrpcx55lblmuOj/6dUKPFqkVflqwnQJfxeO211/Dqq69CURTcdttteOGFF1IuuBOkhpJfBmXSHCSa9tvUIM9mlKDwr+4BwLim5KDZs1vtaoGrkZAVQ2eCgETtq29pzESkL78fkUQecN7+O/6PvfvoJ8hhJ1+jOJC6v3j6PnzBDNTT/6fd+FnXY9c1n0FBt71oTQKQPWMxgpMXIHpit+luk0CYlOGg+WX9LlPfUFmUA+nDQ4ZRNIwn0TDq0Mu6mw15NvdR5bwiz3xztwwg3kiwOxTanGlcdpqphVSECCqz3kBA0lAZAm4MHcW20X+Dt46HXDSS/PuIA+6dzVi/OJsFxk5sNJDtpqzKv4dNb6WTHZ9y63ev+HHwtQfUYLPS3ew1AMD88jzHhMfLuLD3p6G1G799sw7GnKmnkTN1FG730g1+58HvTtwy3djxsR0N2efCuqDYOJTbM3VDI7C5sGhHQPa5HW+3+nRf8hjFPffcg8bGRqxbtw5PPfUUZs6cOSwXIDBgMoSpsQhMmg0lv8zmmmI75jkC48E0hBffBSWnCL1v/wKkxxlMUyZMR2Miz7FFHzcq6PiyZE+dDXS8BxCjOLDqbpC+Hle59YbWbmQd/gDpzFDiR3abhoI/9qyssyDdEe4eSAhOXqDLsM9Za8sOy0nTML1orOPLS4XnAEDN0zPH1I5jUI/vBwDIRLW52dxiDDx+7gqHXIQxKfG+/anhdijQrHgRUUEidYA022bbqVshmduA96XzK32/FN4lsyeY6cB0cuVlWtj0VrfWswO9V/z1VM0qMO8ZlZNnA+/Urbajps2RhkvjKWztAeVPHzTZ5OoJgZnayu6Uk8HvGFKR1HDr483Xp0wvHguZ1k0w2X+sZhlgl1ChHfwAp1wMFfxbXzkZdc1nHAkolzxG8dFHHyEcDuMPf/gDXnzxRfN1Wvewf//+YbmgaxUpPZvxyQOQZeRX6FpMVAAQahyABClNr23gA+PhxXchNGMFYoe3uxoJQEaovAp1x/QvQYnSgevDR3H2rb9gTOVqlBWW2b4sRYEOsP36WK0pFvql+Vw4G4tCXbrLDAB6u6FGGqDklzm+iAWndiIGvoBBstrGxnptv4nXvIGT5yf4fnnptcX2/pE5IsHymWOQOarUFqzeXtNmTpw87LXOw18w9uB/InZOjyfxmk18gJtOnEcTBVAhQzLGqEHCkXiBTaU0Kz2E870xR+MfL/xWxrs+aTMnYjdhP7qD8jMoA81WYu8VH9viJ1oAnoF3WnkM2I1IS0ePWUPz3U1O9xoAnO1x0Z3zSW0FrEmaik5SI8/fXzchQ6+UZ3MyN87NZtiZmW3M+dkOgLxyMX2egK4YwLquZMlqt6s3HpMGZNgHi6ehePvtt4ftpAInofIqJIxOehIAaWyhHkiGJQBI26LSbB63wDj92Y3g3LUAgPmxvYikncbtaR8iAA04BURf34+MDZtRVlhmfllOvvcWMjXVFBKM1VdDNmo9AN3VRM53oC99JoowGnNDzfbCQEJs/cLZL6IaYI2f5UWX0jJdhRNBNJQFI5DlcdCMgql5+AsuvPwcpMwxCM9dBwDo3/sK+M6Boxrfw9rblkDJ143Ejhd/j1mB49iRmAzceZcZWGfrWMoKs1F87gD6d74AFTAN8vTi+bYJhN+hsEYmdqEfwWPvghgKXHT3QOsK6ITNN/5JBq+UOio9aKsGliQ4BPyoS8LPF++2U0kVmkG185M2fHfTAsdOY4kRIGcL1OjENio9aHO5aoTYfPtukuKUpXMn2lqdJtuhubVp3VlzEt/9wkLP55isMM6vwJJV+VWN3QD9NyWhEtPNR58tK2bJNs+iEjl8FX2qfzuDxdNQFBYWev1KMAwo+WUIV92N/p3PAdBAuk4g3nUC8bqdyNiw2chw0uMOrPvJrfcFv9OgaGfbEd3yJLK0OO7MkAytKPrLhG1Sb2jtxgv7NfyfDBkKNMiSpBsyouptXJkAefGFatyakW+p3Zq/Iebux228tMiQxKKI17xhGsFgeZVjsoesID6uDBL0+obFwTpkffKBvl7vBKJNB/UYisaXJwIgmjm2swfewp1puwEAFYE21B8YBzUwD9HXHjfHlnHbQ1DyyxA/vMN2mPjhHSi7Y0XS4GhZYTamBDoQfe09/R4ACEgEG6fFkLFovkv6KfFVnmVh5SQAexol/bk4P8vs5sa7JLx88Xw21UBkrncxbVhVVZ/kSyeOdgTDywqzXavrf/dWvWkkaPoov1PY9mGTQwa+obUbF3rjuOVGvfaFKuX6GTq2cx5F1fTXv/nX9jR9dmHDPi/a24Pvle2222CLlWmtR1FepmvfcK/dHqs1Ru+fXwxtOEg560kw/FjGgMGYwE33kyHz4RZMTnr86Fkz1iHx4TNZsR2zrvkMjsZy8R/qakwLtmPBRAUTz+4z0rG4LCoAk0f1Q4opIERlwy2+KbxKfpkVg2GMoO5643cbQFP7ebMo64ZgAzc4zaFJ5Ta20u6P9Gs2VrCl3XvR+95+yzARFX0fvoBRt30PUuYYgPHgSZm6bE0qwdHEyVqbsZMg4brFlVDyvbOS/ILB9LU/fdBky58ngEPOYenciWjpOOIaY/HyxbPHo9cxJdBhapKxadnJOHikEweOdJq9rv2C9GzPakCX2fj8qnK0dPTYdgqnzvThid/tN/WhAKScbUXZfrDVswjuYEMnth9sdZ34ywqzHc+L9v5gz+vmtgKzo6DB7LLCbFsiRiq1K2wM54aK8Y7AOYCkLtWLQRiKKwiaJmtbTRuTHLsCT9a729X1JAcQrFiG/l0tVmYQe55AGGpXC5T8MjQdOohxzQdQGszQm8pIEjKLSoFzNfpn9dnJdvhQbhG0tlpHEpbXjsIxbsYIhsqrECqvQv/eV6C2fgqawTQ+dgKyNAEaCM4xch3svaLZXrbjT19q9iAPXojYKteDF9odGSdaez3USAMCk+aYQXEAkLMLkCpsRTsxrqilowclRsKgV1aSVzCY9n842OCc6AIuqaz8yp0ek77GunHcsqlmZZ1F9PWfmzs0urN1+7tbMnuCqfHEBl/5Xtf8NbhNZiX5WTZj8np1o9lsiN15Vc0qGHBB4L66U56/IwT4zRt1IIAtOM0aIV5vKRU5F5oF51V/weKWGEAz7NgYDk2rZgPn9O+FfnYgO8JUEIbiCkLJL4NSPBdqkzU5KZPmprySo/CuJ2XyAoTnrtOPn1OExMlaSGmZRszDMBaxKPp3Pouu1mZkHd2OMdBQMUrvXSGBQPrLIYSXbALp64F6phVqw277SWNRQHPWMSQtCgQcRhDQV+SB0kVQ2+v1yVZS8HJdwMwsGT1tAaSmE/azjZkISQ5C62wEayyolAetQ2HshCex+mrImfZ6oXjNG5BHj3fN/Iod3o7Esb2Qc4shhTIQmFiB4PSliB1+11Du1dCy/SXEx06xrUC9Ash8LGHXJ23o6O615dxLAJbP06Uk+NgCPxHVHu+y+bVZ5Va3bKqCUzsR05zFi25/i9Q9Utd8BsdOnnNdtdMAMq8vBdi73LEZTUV5mTgXdQalkzXE8oIXHJyUNwonOi5YQ6QxEi5NmDUGrN6Sn4YZm77LFlUClnYTrzpb13zG4c7yU79lg/a2moxBVNInQxiKKww5Ixsq9zOgS1241VK44RXkpmg9pyH1nNalQs7aKzzDjTv1tE4zuGjEMbSEOUGyWUXmMTuOOV6DJPvuKNgAsvlaV4veT1yLA3LQNE4fns7G0X3GpEEIQudaHccjXSf0L7gk28pc1c4m/XJS2N2Y4znTahgYZo1MVN24QrM9g9jh7aaUCjXQMSOdWIMM2YgFzQ6eQP2Bt4DCO23nSra6pAJ+rIuGl3V3MwAsnxzt9I2J8O/Xkw0CVsyHcXd66VrxwdcAI6PNT3i8xDrtcsdeB1+1LRuWnQbHB5qhRVNuWTXiJ36330iO0HfONEXVTfOJvZ5UNMz4XSHVhuLjQasWFjl6oLCuKE+YDnqpSfAMHmEorjCC5VV6xo/hhgmyVdkutRReuAW51UgDoq//yD3ga6AQawUnsYFpOaDvQlyyilyRJFuGFn+tNsMnGQ1ciQqA6lrp41Q7myBnjkNJQRYCylmoqobSUCcmnD3gfW6iwegIDQBI1O2EWl6V0u6GorXXo//UMbAZWfqYjLGrcfMZxD550+UA+rVro/Ig90RMwzvlwicA7nS+n4Nd6bOSDRLsMhCpBsVZJVWv1TKLkl+GjA2bHTGKZLpWdc1nbKmeXhOeZKR10oJFt4AsP/m5VYq7XTu9Drb9K2XFvEJbjQabdgvA9d986nMqGmb08/R+J1Q9AM7HgxIJDdv2WI2V4gkNr75/DBtvKrUV8fE90vW+MgS/ebNE+wexAAAgAElEQVTeHNfff36+rV5mKBGG4gqDfkH5WIQtfiHJgwpm68bG20gAlpQGTXOVAMh5pUZ843cOtVoKfb+ZHkuM/6gx9L77nwjNvcVmuGyGj6iwJmTNqAAngCSbmVY5chDfW6sryM6PdUGq91lpAVAmz7PiC0bW00B2FPrHEtaYMsYA0bPsb5MeTwKQlleIxIWI+UL62NTlb9hVOqsz5Fa3kcwAVEzO8Y2JeCEbSsX079Arvdatp7bbBEsbFrkpr7qNPxX5DRbWkL2+6zi+c5fTkPm56Lz+PZjrYYUrCQGy0kN6vwtiBbd5AwDogfLa5v22Ir41108yW+KyAXK+eNFPceBiEIbiCoRmA7njVeyfnKTaUAYS91Pakk3WxO4FsY4aG1OM0PmTplEi5yKma4YaC1sAm0qDEM1sD0v6ekB6Tus9N4xdVO6xrVge7YY8vhSqpBjZV9bOwSQzF+G56xA9cciWJcZrZdlGOWosEM4C6bL6Q7BjItEzjpgG3aGEZq92VfENlldB7WoxDZYEIDBp4ErJfhNUsqA4f5xUi+rMHZ8aQ0xSEL7pHoRmrEiaXgs4DZXX9Xspr7LX1NLRg1ojfpKKoWANGdujYiCaVJRUDIsffNHfR7WnbBN/RlrADHZTCXli/M3xRXwZaQFbLIlV3GWLF4cLYSiuEmzplkxdwMBxMRKyAqKxqZwMkoR4fTXk3BIjZdW5oyDWW0EI0BdTEXJpGxeredM0FJ4BbGb1qkYaLPVaArM5k3ouYl2li90Lz7/VNUtM7WrRZduJZrrGzDFcOANc6DYNBP2foajgDHwzu7rQjBVItNXZA/ySDLWrhctASy2474bfBDUQA8C+n9eGYkmcrLWetRGbUXKKwFfvu6njuhkqt+tnr8NNB2r7wVY8t01PkaVBaL7DHg9ryAIBb+l2r4mVLWZ060mdKrRyOqBIZjdDc+dgTPw084waTDZDyi1Owt9Dt+LF4UIYiqsEy/WUuDjXE4ecV4r++Z9F09b/h6nKKa43NwCiIX74XUAJIThrFeIfb4O1gtdnacn4J/WzhtPCQK/TUJBzbaakB2DfOamRBsf7aUV64theqJEjQIItwrK+dDBTiiUEGRcXf/z+6t+YrrvctV9B9/E6fWwmln6TRvSRqYTAcIR5ZkmpkQYoYwtBCsqtToOE2LPKgKTB/cHilfaa7DN+K2xHqjazOOEnrGR9sJPhJeHBp7PuqzuV1FCwuxc2RpFKsNfRs2SQPanZTo2yoldO+7na2PvJ77K8DD/fC0MU3AmGDFu3OkAX4atYho/OZWMcsQfZHJOiloB6shasm0cpmQ+1+WOAqEa2EdA3cSEygwTqGf4AAC/pQeEzuoKzVkHrbIacW4z4oT97xkX0i7Ayo/zqS/o/3sr01dDQHzmmJw7U7rAmQ0nRe3ioKlTI2NFfgQVZXRibiEDii/mMsQCWsi8kBZADxk7FvmNJFtxPBk2/5bPYBivzzWtDsROhGmlApPYgusYtRvHp3ZAIAZSg6+LEza3kt1NxY3rxWFcdKD6ddeH01Fom04k3Ly/L9NenEl+wpZgSexEjlY5PNia+EZNmBOpTndj9YiZ+7/UL4A8FwlBcJQyF64kGyvsPboXadEBf9Vb/BnNWfQevJKahJNCpL9Bdl84StI7jtlcCxXMACVCP7zcNy+hsvfuhV14U1XKK1VeDRLvN9F8zsK3GTNVYNxkS27FGjUVowUZHdhdP7PB2W+EcAJyNnELN2TbMkyRIhpULzlkDefR4nDv8AdoxDp/p2gMp7hGXMQohbUF5pskUiXZDPVHDGY3UMtZ4WPl2ek/omFN1qfB1FGz6sCxLth4aYz/4D2SocYSh4MW+G7F23jjkV8zzvGZ+whpoLKCsMBt3ry53uFLo52g6a7LdRDKSxRe8sqwG4oZypvRatRYDiW8AqTeE8gvgDxXCUFwlDIWEB6AbCykjG6brRksgp2M/lt15F9rf70FB9yeun3OruSB9PZDS7X+QBLrAYfzwduscDGpnk80lowLGSp66OfjSbvukZjvXhTO+K3RqkBI295J+hg8bo+hS92NOWgKKkaYV/3gbIMsIExUlkOEpCQJAHlcMwOkSVHJLrDoQSUGwYjnk3BKjy+DAM9bUSINNbh3QNaeooUg1f56to2ADpRL0/st0IuxJP4S19J4QDenow4HQIqzPn+x6XH4yS9Vw8XituNl01ovppJcKZYV6Z0PeMA2kLez0Ynsjpi+sLk9ph8e/x8vgun3GK4A/lAhDcZUwEAmPZJDebtvP6oWz+h/W5/6v6eLQertBuqzKZyktE4STF5HSMhGaWGGo3qqArCBk5NuHl37RFDi0nSty1FmHQVQEKm6G1tnkKNxTpi6G2vCh4zgmHit0NmvHjT19U0EArE2TDG1X6Ocwg/D2NF15fCm0SINpPLSORkS3PInwkk32y+lssu0wpMxxUHJoj4CBZ6xRBWEWqjkFpJ6ySeso3AKlUwIdyA0ewhEUoD6WjzVpktlnulGbgM/6KLHyk9nFFH75rbgHs1MZKGxnw/oT3SjK0+NJA5HzTvV5JBsPX4NB05DdPuMVwB9KhKG4ivBPm00dfhegjLImHlqo17fzV4gzhkIZWwhlbCET/NUzeLzqPkIzVkDJKTL0miwXEk0/tSErUHJLkKi1q7UqZZVQxhZC9TISBm4rdL903mjpSrR+nI9EQsMf+xbjzowP9YldpoV/9jRdKS1T3yXYdhhEL7o7ttfmEiSAnh3Gp+UmcRt6rS7dTAufYpvMpdHQ2o2W01FXtVPd1fRzkLQE1DQZO2IzIUNP1VRAcP/sPow9tRNqwLk4cds9rK+cPODah1QY7E7lYs6x65M2szXwQOS8U3ExJRsPX4NBOzu6fcYrgD+UCENxDWL1vtB3AVlzVoAv0fGqEDfTVRn3l5cBU/LLEF50O6InD/tWcyuT5uppo9x7lLGFILGox6cM0rNdz201e7LvKJSyShSsvBePVcbxQU0rphcvxKjAZ2xpujR+onU2IVheZU8VtUEg5xabelSsoCFvONmaEdJz2pb95be6DJVX6QaUuTcDCYj7icWVFWbrCwItoRd/QcPy0ScgMclloxrfRawRrrIxXruHVAzXQA3JcEtUuJ0DgEPOG3Cv97jYc/HjYWswJOPnS3EPvFC+//3vf/+Sne0S0dsbS9qw3ItRo8KIRn2ybEYAcmYOAoUzIY0ej/Ci2zGmbJZjzPx7lPwy47UZkLLyEF64MaWJSs7MgZSRDbW5Bl5JpnSC1iJH7J/NmwytsxnkvLfqZ3DmSgSKrvMY4wwkIg1An2UGA/nTECiZh5LCMSjMyUDO6DT9vRPKIWfmgFzoQv+u34GcbYXWeRzxI7sg5xY7rk1HQqB4LgJTFgKJOEJz1iA4eYHteOy1EEmGdroZWscxxOurIWVkQ8mbjN2ftuPT411mtk1psBMF3Z9AkvXJKl5f7djNSFl5CEwo97wvFP7YeWPSUT7J2kHGmz+G1nncGA2gZI8Hes85D0Q0aD1dkLPHQ87MQezwdow6vAU3Th+DsSXlWDA9D22nL0CWJeSMTvO8Hmq4Pj3ehQ8+jWBGyVjH+xtau7H703bbsXJGp2FGyVjkjUnHbTdN8XXp0M9OKhg9oO8yf45J+Vn44NMIYGQ+LZieh2de+9T32pNBry83Ox1VsydAliQUj8/EpPws27FkWdJVYomu/0THnJURRFzVsPqGYiwoz3Pc0+37W1AxiOuSJAkZGc7e5BSxo7hGScWN5faewbi/lJwiRjiK2OU+JBlBQ1KcNyNaZ7NDCTdodLJLNO5DYMpCsye3F+SsvUUnu35wW9k6ZE7MtGAeCVCCILGoLpJINKjt9WY8wi2WRM51MCm6VhHb9OI8c6VYGurEvMY3EDumN1EKVixz7sZSSGaggouzsorwms8qlN+xkLNtCM5dx6Un631B1NZPEW2vN+pp9AD76NZDmDf7s/jBnzNSzgpKpRd1ESLo+SiC4MplKJmltwROZafi1gt+IPDnGEgf7WTw1/f5VdNM1xavz+WVdszHUPgEAhHMFly1xOqrrYkIhpEwpArOz74TWfllOJMzE1kth6yJXIKjvauthiCJgQBoIJg1DRJChguNTRdlJzdHrYms6DpPDHJBOQKT5hhS7c9Zq301rmdZ1VfbpC+UnCL3wDpR0X9wKyYmYvjnqpk4iJm44dQBSBGriZJ6ptWKeUCCnFuCYMUyX2MdO7zdbJubo+gaWY1avqtYnJJfBqVkrk0XSwplIGP9dwAAwckLmDiTnr6caNxnO0aicR8SalXKWUGsUZwf64IaSdi0pIoQwdey3oQCFdhdAzXPvQ8GDz+Rf3K0Eyvm+Av3JcOruNCvaM+rYI6/PioS6HXf+HN7GSoRzBaMCEjUnmVF3YL7YlPQi5koBHAQM9EWXYxl4b8AkHCueCkWGUbBTQk3FTTuvHLBNHPCsctuW186Jb8MwdmrrZRUoteLqCdqzJhOcNoSkL4eJMw6CQNJwrkL/UhXY4Ywor5rCFYs8wys094jo1sPYUVBObTuNu4NCWTc+g+6AarbCa2zEf27WnxTgu3GK4aCeAsWrV7qKhanRhoMtyAdgz19V8kv0+MwbE1LaJTtGOHxJQicTM13TlfKJw/XYN7xtyDVxRGt/5NdS2rfUQRoW90B1AzxPvzZU3OTfmYgJMtoYncMbs2P+OtbOH086k90pxxz8IsJiWC24KqH77FBSyNGy/0oNv7YpxePxWvVFfjgfLkedJ0/f8jPq4y1CrbYdFH2S6dGGhCv2WZ9iKjQOpvMzC4zA8oluN07/jr89+Ex+GqGBAVEL1xkM6G41q48pvwHe81G1hQ1PMlk5p3xDP8+HPaWrZLZDZDF4XqLXbD9mB0mA8p0KivMxqRT3Ygdo/1FLDfclAAwPnwUVJBRklOvO+En8orJOUOupOrn/mJX/G7Nj9wywgYiw+FnqNyq0YcSYSgEw06wvMpMqzU1/CRgwoKbUOjjkx2S87pkbgGW7LZrjIKbaAm4/t4etRnNSgmOxnLwrjwTn0n7VB+vEuRau/pXm7MoZZV6jIDKg1ClXJ+iPTczRPp60NdSh/6/7HfETazssLhZMAjAVj2vdTTajheYshDxT9403XOJup2YUl6Fskr/VT/bqMphvIydA6BnX5n3YNIc390Ee0w3HapLCS+n7tb8yE2ig5fhGIjEx6VCGArBsKPkl0Epq4TasNuSIs+djsLKdbb3DfWXwKvGw+98jglMks24BuDR11x/IyZmAVNDnVieVmsG6sNLNlk9RUoXQU2SKsyinTpm7UKIamiruO9IaKEk0rPsv5AVkFgUrc/9k641JQcwasN3bcKM4SWbzJhG/67fAYBRSe7sXSJl5iI4eQEQ67VqalJwDzn0vJh7ahzZSk82JeQB9USNLZXY75h+XR8vBfxiB/AW9eO5FAWFF4MwFIJLQsbK/4N2LR3RIx/h41gxth27Hn/f2j3sX4aBZmnpMuDWhByoWG77vJJfBmXqDc6e4UoQ+RXzcB8OIlinmdlbVFZcjTQYRXupGQkAIOci1g80AwBwTMysFhSPPH4qYjXbAE1vb6upcURqD2Iio6qrFw0yWlTH9no2uCI9nYhueRLBWassyXaPLCxWyFDrbLJ2YlrCET+iN0zJL0Nw+tKUjJBe32IYUjWGWH010i/SUNCdFNvVbyAMRNSP5VIUFF4MwlAILhn7Ri3Dy+eK9CJo6cr7MgDGjkEJ2groHPQ6fcB055APINrwpqMo0btoL0UIMXqBw+Z6ctOCYqGyIzQuRAA0xPMxEW4yJ5J+zaWLoLb5dENU44jXvAFT2n3WKseE6ugjTptTGefh40esQQiWV7kWdvLouz9rh5Wo2wF1EJO7OSyuVXC8bicyNqSWcXWxXM5iulQQhkJwybjSvwxAappafG0HoIsdsp+nq1LzMxMrEJP8hQaTolfNAVoC0bd/oRcIJmuEZKjXEmM3QiCjpEB3T9llTiQohdchULoIpK8H4aq7oXY26Sq4hpQ8fZ9usOjPBPGaNxCcvMB2r+wNmwCH/EkoHbbuhJJiq/RPRdfMMXZNte+0WurQu+ctc3dAx+x2zIbWbkT37sZko/2tfryBK/0OluGI0Q0lwlAILhlX+peBksxdFZqxAvEju2xZSny9ecIIotNVKYDBaAJyMAfo6YTW05nSp5SCaVDb6/XKawAF8RYA85yKxKWLLOVbxudPA8ZSWqalfcU2ZXJzD/GxErYVIdH0HQnsqcW2a07BZeiIJ8mKbad1csuPANXYHVAdMaIiJskI33SvmXJtFfmF8PUsGQHJcB0mKWz06hEyWC5nID4ZwlAILilX8pdhICj5ZfZ01lC6+c8YdZsAgJbQf9Z/4I4i2Zo/QVIQnLMGWmczkJ4F9eiH5o5AKVsMteEDDNzaSJDCmZBkRY9TKAHXlbuUlqnvAqjPn1lNNybyUNcXwPTxY1E2w3p2NADONzVSIw36tbP3yxynocrraAQ18B4r/I5CHj/VzJzSXX2M64yNDREN/e//yqxFofGBRpKH/zi/Bp8t6UBJQZZvjMLhWgOGxFhcqQhDIRAMAq2z2fNnXsad9HZDTncxjkoQ4XnrgHnr7L29qa+cTqaSjPB1n0FvZ5OjJ4g/EiArVrGgRzxB6zmNxL5XmJiEZK6mvbJxqDqwmyvHUcshybZxWrUoTF0JF3dJRU6fr6LX2usRaz+CmKJ3SfSFEMTrq6Hkl3EuUQlZ4wsQLLcaNblKvXCutcSxvcJQCAQCO3ycIlC6yPw3L+MupWfbazoMDSd2xWrz7/N6U0T3vYdmrzZXsX7IBeVm9TjpOc1IwxPEP/4T5NHjEZqxwqNnhx6roEKQdbutpj1FiCC69zXEzk0yW8+G59/qOD+/55GyC6B2tTja1cYP74B2+rjNqAw05VUeV8z1MNHl33XD7Z1ODFiV+37V4s2j57tLvfg8f5bhbrZ0qRCGQiAYBJ4aVHDKuNNmTn41HSyBiRWIsYFeGE2ijHPEat4E6fbaWegtaxVDC6v/IJ8RRcwqaNeeHUrQNBKAlYBA9ZeCERX9RtZuTA7YsoLYIj0wtSbk7EnDwEmIGT3OHdXtTMGd1RY37pny6t+YihiKv/Ugqq6R5ZaWbLbhhXe1+Mkpimvaqt/zp1zptREDQRgKgWCQeGlQeRmFgdV02FfCmpFVRc+pRhrQ9+ELVpxEkm1ZUVQbyjXF1ZiUbcFsSUZw+lKHX56utqN7X0MwotqD9lrCdN/wqaWADGnUWJALZ+xjUuN6YJk3UEzg2Cq4I0jU7XRNeeUztqTR45m6EwlSKAMTvvB9dP5lP7erss7HF/05iimJhrJgBAEl11XqRe1sgjw6j+lgaOdKr40YCMJQCATDwMV0I9RX1nZDwTtQlPwyjLrte7aMJD1jSc9gItFu7zoIEJBYNOU01LLCbKiBSkRf2+5YmSfa6sxrsJ9P44yEdW6ts8mQJNFcDVQqBXd8xlZo7i1WNbmR/ZRWNB2Bs1H0ffiC7bPSmIkIzV7tKqEfvukeW5A+v2Ie/r4sz+Y+SrXe4mpIB08VYSgEgisMh9S5pLgX/sFukNjgcr9PER4AxD/easYqUjFo5iTKxUjI2ZOIbnkSypSFnp+VRudDyshmssQIAtOXQc4c52qgUim44zO21M4mq3LdoK+ljtvlWNfs1SVQySlCoGKZrTK7DPYKa7eeJW7GbCjSwVMN7A83wlAIBFcY1HU1UCkJ1mjwAXU3Esf22owL4F2QBtAGVC46V1oCWptbcycdcu4USM9pQA6Ykh8hZkz8ZJjqToe+7ohVaCpi9dU4nx703lW5TO5sIB2uelQ6zp4l3vUWF5MOzvYViSmXV8tKGAqB4ApEyS+7KN2iUHkVEoe3wy/rhwBWhpGkGC+qntlGukvMnvZK/08ucNpN0HcSetxAj5vIuaVQ8krMCbj/wBbGZWbPckrVdecakAdBonYHzju77gKyou88XCZ381hJpNwHa8gHgl6n8SuYz0+NXbIqcTeEoRAIRiBKfhmUyfOtznXQlV8JU82tsTIkRIU5KXlMknxcILxkk5WCW7vdbpOUkC5HzrjAtM5j0M60QM4tsSrAQYvv/CdnN9RIA7Se01a8A7BSbYnqtJFyAOGqux1pul7j86vKvlhD7ofDSBj49RUZboShEAhGKOG56xBtrtEnPiWI0Pxb7dIbJnphHgDTNSSlZaL/wBZH1pabS0iNNFgxBaNdqzKxwlGUqL85bqjTGit3I6ANWKv8VPzyNjeRpEAp1vtq23WpOIhmGgmaissrA6fi8uKvY7AxhNjh7Ygf3gEpcwzCc9eZ2WP97/8abjvBpLpew4gwFALBCIVN082duQDnwxMRP7bXvpMAgHAmMtZ+CwC4DCpn0ZubS4j2tIgf3gGtqwlaxzGuCI4jELJ2AXIAwVmroHU2m0VrqRTc2dxERNUNBDTH/CqNmQhy7pTNAPodn/7bzZDwXEw/DFYCBJ1AtPljZGx4iOs4yI1F7CgEAsFwQCf2tLwsnO84D3K+w/mm/vNQu1rMegCtsyklXz3F7LWRooy62nzAqE5fbnNDqe31evwihXPb3EQeBXUAIGcXANkFINGzCFYs01flPscfyOSfakzD9bO8uq6hfOtnDMSOQiAQXBL4uAGlf+ezRlaSLk7IrviT9ax2DygDlqYuu8w3mmIQVf8fN3Gb/cWTxAlYNxGJRT17cqjNB824Rf/pZoSr7vY9fqqTvxkfIYZMCCGuk7yXwqxDqt7QujKr03kkKelzGE6EoRAIriHSDGkPq/EQg5lKqiFQsRyAXT7dzR/vCCgbBXRyboke6OYncckKXifqduoTN025NdrOhsqrXP3+nvGAWK/3gFlxQi0B0tfjG4dIFtCmMiXOqneC/upf22oz2M6DvMJsaMYKvViRdko0Msi8Wu0qxfNFHYVAILh0pN34Wcijx9uL5yTZmsTlABTGJRSvr7bpM8UkBeGb7oGSU2QFlCFDzp2MYMUyAJYGUnj+rZBHj7e1RGWrrmnDJ3bX4RYH4V1C5vXQzCkvWElzpl+FH4HyKte0V399KdgaJ+mdB/9k+zWrMKtLse9xfDYwsULvzcFuwuSArr57GRGGQiC4BqETFpt1o3a1mBO6w5d/bK81QRqCecGKZUz2kgqt4xj6O5vM1TC7ijYFDQ9vt/Xalozj6cfVEKuvhuxSAGhrJavGLL0o49xeyPll0NoboKfPSlC7WjwD9ckK7qwe3R5IspktRnpOg4+sswqzjqA163piugnSroOpBNeHE2EoBIJrFHYCNwPSRlA5vGST3ZdfugjqycO2Sd2MJ7B9JTiXSbx2h/McRhwkvGQTlJwiPbVWjQMESNS+pxsMSTFW1noBIC8RonU0Gim9NNPJvcWsFmlgrk2zp+ZyMYhk8Qm+R7cyeQEQCDMNpSRdb8qthkOSbOKBem/2kD5uSdZ3aMa5vLoODjSzaigRhkIgENgnSTUOtbPJ1ZfPCuaFyqug5JaYabF6qqqhYGsgZYxxngMw3U6hGSt0NxKt76AfZf+tJaCd4tNtCeScYih5JXrmFBUE5CHEXqdRughqe71rDCJZfILPOpLSsyFnjoNK214QzaE3xV4HK5lOg/G0upsaEb6W42Iyq4YSYSgEAgETRLXkvUPlVbbGRLxgHgArTmA0Y5JzS6xJX1JsvnWrz4ZqnkMtr9InYLf0VkmC2W3PJVtL62yEdqYFGeVVpqSG2QeEWhlZca3GThzbC6RnoX/vK2ZGUrKCO17jyQzGM1LteoW5uyuMlUy3BcWJinh9tad8SarV4mqkAWfqG6FmTxlyYyIMhUAggJJf5ivv7ea/t8UNSALauQ4Ey6uQcdtDPtXKzIo7Se2AnD8NgUlz7F3xjuyGJCuGFIm++0mcrEV4/q1Izy+DWl6lT8CH34PuliK2TCS3mg82luKnMaXkl0EpnmPJohiV3uElm5A4thdybjHin7zpfZOZ5kyOoLgxDsAuzJhqtTh9PlHDoAy1i8onXUAgEFxLBMur9FoKwNbDGnD33/MTvNp6SJ8AAYTn3+ouKmjrpa2v0q3MJzvy2ELzOGqkAfFDfwZ6zzqaIbHXYZ3T0nyK1Vc7x8ERP7zD9Rps44s0QG2uYa5fNqvY1ZOf6inHbu4vmlFGpVH2vuISFNd7hES3PInY3pcQ3fIk1EhD0mui19VHjR/RbEZnqBA7CoFAkBQ3/70+GXF9qZNVU9s6yBGoXS1wE3lle3CokQZmcjX0oRgcsQP+UG7j4FJctdPHoUYaXNNybfECxkUmjyu2V7F7ZF8pxfOgjC/VjcT7z9mNJXOVWmezwxgD/pImzs6CAG88hwJhKAQCAQAuZZNzPXm5QGIKN+kmqaaWc0tsOlCJY3sRXnQ74pwkulI8FwAQfeOnUJsO2H7HpxSRWNT2c7C8CnGmZzmb5qrklyE4a5Wzkpto6N/7itkv3Ioh7ND7WxjKs2yWl9bRCO10M2DGXdwJFM8B6etB7NO37UYiPRvov2BN8ulZjqrxZMFsZ2dBYzhDLPchDIVAIADA7RokGaTntG2Vzfvv3Sbd4KxVvr7xYMUy9DOGIlC6SDcgBdOYDngA6e9B9NUfwivtlYVXqaViiOHuRvS7BHZVD7eM2noI0ZOHEZyzRndz2ZohJZBorkGgvApaZ5OengsCEA1y7mRPEUQpZ5IV8OcyoiRJRmD2auP+EagNuxGcuw5SKMNujH2C2VaCAHOfUiwsHAjCUAgEAgD2lM1E3U7Ea7fbsnHc4Cddr0mYQmsqeP0jZWyhzVBoPV1IxUgA9kI2dixjZ81HR8d5x+/YlF0HRHVUVFPU5oMAiF4wKFtFg8GKZeg/3exc2UsyAvlliJ9pcU2bDUyrdBg5rbMZGeu/YxuHl7R7rL5a743O3Sdl0lyR9SQQCIYPJb8MMnVBpZC7z0+6vv1baBYAAA/5SURBVJOwAVvoR7HcRfrKWQqEfHrzMddbVuk4VjLC89Yh6te3ghoD2lAJgK5QS2tEdC0stud37NN3QLqsSV8aMxHpy+8HAH1cxG5EpJxJSLvxs4gd3m4TB/QyesnjEhZyxuBar/ohsp4EAoGNwMQK3VduZOr4uTHC89YZgoBw1E0MBOouCl1/p/7/2atT+pwcTBvcuW57CIEZN1tZXjYkvVK6ZL5Ry2H/HSQZEmAZicPbbUYC0OXN/Vb1JNaLvp2/Mt6s2P7ff2CLLeNJjTTYXvOKS5jnzi3x/N1gETsKgUBgQrN8aJvTZJ3b6KQ72C5v/LHYeAgA9O/fAlzo9PxMKrsOr3Ol55chlluC/j0vAf2si4pAO3fK6JZnloozv9Zsbrl4rTO1VjJW9Z6Tek+nXrMiydY5NM3obqeZGU6AM+vJS2HWOPOw9K0QhkIgEABwFtWlWrTlV6R2MVAXFeuPt7U6ZVJoB4MaafCU/kg07vN2TdHMJVpP4uJuo9eVNE2VaMaOTNeK4vuH6+exZz2F59+KYMUyqzgSMBVnpUBwWPpWCEMhEAgAXFzHtuFEyS+DYijbBuesAYn1usqADxQ/F05gykJn5pMNyXTLBSZWMDEPCeGlXzSvK+nq3ki7VTubdEN4osYMktMJP2am5Eqm4ZFzS6w+HoChaaVg3F/dj/5heGbCUAgEAgDJRfEuF2x/abX1EMJL7xtwANsNt9W+nFeKYMUyhGas0Ht2eLR4lcZMQGDCdAD+7jfX9FXaI0OSEa66G0pOEaeZtdxmBMNLNqF/568AoqJ/57N6wyPXHQ+B2uvM8hoKhKEQCAQAvFMxLzd8f2m2AdDF4Lbap0YiWR9wcvYk4mfbHGJ+PEp+GeQ8rs6Cuq4I0XcSbO8PaJAyx9mL6pprwMZIzK54PJKE9JLrMBymQmQ9CQQCEyW/zFWn6XLCp4y6pZAO6rgTK8ALflCjpPfIsFecO7FECf2gXf8srF7iibqd+s5GUozXJbPQ0TxL9Gwqw4GUlY+0oukpvXegCEMhEAiuaEIzViC89D4ohbOGzO0EGJXlc2+xvRYoXaQLENbttF40pMpdjQUnnugGe/3BuevsKbeaqosi0hoNoiJ+eLtNFNBpaNyRxxak9L7BIFxPAoHgisetSG8ooP3D2Urx/gNb7EFuTZcqd/S84DrTpXL9/Qe2cAq6sl5dbYs32JMJzDayNW/qZRxpmbYqdgC6MZs7fH21haEQCATXNLwRctYpEEfPi8HGcdxaoGqdTU5JQW6nwl5j9I2f2t4q55UibcmmYXUXCkMhEAgEDEp+GZSpNzBBY2fPi8FOym4JA2qkAXGjVzglMH2p6znUSIOeQkuRlGE3EoAwFAKBQGBDjTRAPbrH9tpQVju7qfAqU2+0ZTNJoXTXz9p7YkgIVCy7JIkHIpgtEAgEDHyDIkhDL9vNQ7ojtp/dVHjVSAO0ntNGb24A8sVVpg8EsaMQCAQCBrc4wnCv2pOp8JryKp6V4sOLMBQCgUDAcDkKD23S5y4qvImTtU4joamXTGZFGAqBQCDgGC6hQ7/z+anwuosLDn1vbC+EoRAIBIIrAD/j5BVMHw5JcTdEMFsgEAiucAITK6wGUZRh6I3tef5LchaBQCAQDBrqmuo/uBVadzvkMQUIz113ydxjwlAIBALBVYCSX4aMNQ9elnML15NAIBAIfBGGQiAQCAS+CEMhEAgEAl+EoRAIBAKBL8JQCAQCgcCXEZn1JMtS8jcN4+evRsSYRz7X2ngBMeah+oxECCG+7xAIBALBNY1wPQkEAoHAF2EoBAKBQOCLMBQCgUAg8EUYCoFAIBD4IgyFQCAQCHwRhkIgEAgEvghDIRAIBAJfhKEQCAQCgS/CUAgEAoHAF2EoDF5//XWsW7cOq1evxm9/+9vLfTlDys9//nOsX78e69evx5NPPgkA2LVrFzZs2IDVq1fj6aefNt97+PBh3HHHHVizZg3+8R//EYlE4nJd9kXzxBNPYPPmzQC8x3Xy5El84QtfwNq1a/HVr34VFy5cuJyXfFG88847uOOOO3DLLbfg0UcfBTDyn/Orr75q/m0/8cQTAEbms+7p6cGtt96KlpYWAAN/rhc9diIg7e3t5OabbyZnzpwhFy5cIBs2bCBHjhy53Jc1JFRXV5PPfe5zpL+/n8RiMXLvvfeS119/nSxfvpw0NzeTeDxO7r//frJ9+3ZCCCHr168nBw4cIIQQ8tBDD5Hf/va3l/PyB82uXbvIjTfeSL773e8SQrzH9ZWvfIVs2bKFEELIz3/+c/Lkk09engu+SJqbm8lNN91E2traSCwWI5///OfJ9u3bR/Rzjkaj5PrrryenT58m8Xic3HnnnaS6unrEPeuDBw+SW2+9lVx33XXkxIkTpLe3d8DP9WLHLnYU0K3z4sWLMWbMGGRkZGDNmjXYtm3b5b6sISEvLw+bN29GKBRCMBjE1KlTcfz4cZSUlGDSpEkIBALYsGEDtm3bhtbWVvT19WHevHkAgDvuuOOqvA9nz57F008/jQceeAAAPMcVj8fx0UcfYc2aNbbXr0beeustrFu3DgUFBQgGg3j66aeRnp4+op+zqqrQNA29vb1IJBJIJBIIBAIj7lm/8MILeOSRRzB+/HgAQE1NzYCe61CMfUSqxw6UU6dOIS8vz/x5/PjxqKmpuYxXNHRMmzbN/Pfx48fxpz/9CXfffbdjvJFIxHEf8vLyEIlELun1DgX//M//jG9/+9toa2sD4Hy+dFxnzpxBZmYmAoGA7fWrkaamJgSDQTzwwANoa2vDihUrMG3atBH9nDMzM/Gtb30Lt9xyC9LT03H99dcjGAyOuGf92GOP2X52m6/8nutQjF3sKABomgZJsmR2CSG2n0cCR44cwf33349/+Id/wKRJk1zHOxLuwx/+8AdMmDABlZWV5mte43Ib39U2Xoqqqti9ezd++MMf4vnnn0dNTQ1OnDgxYp8zANTW1uKll17Cu+++i507d0KWZVRXV4/4Z+31/Ibz71zsKAAUFBRg79695s8dHR3mNm8ksG/fPjz44IP43ve+h/Xr12PPnj3o6Ogwf0/HW1BQYHu9s7PzqrsPW7duRUdHBzZu3Iju7m5Eo1FIkuQ6rpycHJw/fx6qqkJRlKv6uefm5qKyshI5OTkAgFWrVmHbtm1QFMV8z0h6zgDw/vvvo7KyEuPGjQOgu1T++7//e8Q/a/75JXuuQzF2saMAsGTJEuzevRtdXV3o7e3Fm2++iWXLll3uyxoS2tra8PWvfx1PPfUU1q9fDwCYO3cuGhsb0dTUBFVVsWXLFixbtgyFhYUIh8PYt28fAD2j5Gq7D//zP/+DLVu24NVXX8WDDz6IlStX4vHHH3cdVzAYxKJFi7B161YAwCuvvHLVjZdy88034/3338e5c+egqip27tyJtWvXjtjnDAAVFRXYtWsXotEoCCF45513cMMNN4z4Zz3Q7+9QjF00LjJ4/fXX8cwzzyAej+POO+/El7/85ct9SUPCo48+ipdeegnFxcXma3fddRcmT56Mxx9/HP39/Vi+fDkeeughSJKE2tpaPPzww+jp6cF1112Hxx9/HKFQ6DKOYPC8/PLL2LNnD370ox95jqu1tRWbN2/G6dOnMWHCBPz4xz9Gdnb25b70QfHiiy/i2WefRTweR1VVFR5++GF8+OGHI/o5//KXv8TLL7+MYDCI2bNn45FHHkFjY+OIfNYrV67Ec889h6KiIuzevXtAz/Vixy4MhUAgEAh8Ea4ngUAgEPgiDIVAIBAIfBGGQiAQCAS+CEMhEAgEAl+EoRAIBAKBL6LgTjCiePTRR/HRRx8BAI4ePYrCwkKkpaUBAJ5//nnz38l4++23sXv3bjz88MOe74lEIvjWt76F3//+9xd/4QA2b96M6upqs2iO8td//de49957h+Qc7LmmTZuGL33pS0N6XMHIRKTHCkYsK1euxE9+8hPMnj37cl9KSlzKyVsYCsFAEDsKwTXFrFmz8JnPfAa1tbV46qmnUFdXh+effx7xeBzd3d348pe/jE2bNuHll1/GG2+8gWeeeQb33HMP5s2bh/3796OtrQ2VlZX4l3/5F5w8eRIbNmzAgQMH8LOf/Qytra3o6OhAa2sr8vPz8a//+q+mwOT3v/99xONxFBcX4+TJk9i8eTNuvPHGAV37ypUrsX79elRXV+P8+fP427/9W2zatAmAvlv69a9/DVmWkZubi3/6p3/ClClTcOHCBTz66KPYv38/FEXBqlWr8O1vfxsAcODAAdx1113o7OzEtGnT8G//9m/IyMgY8nsuuPoRhkJwTRGPx3HzzTfjJz/5iTmJ/vKXv8TYsWNx8OBB2+TL0tzcjF//+teIRqO45ZZbsGfPHhQVFdnes3fvXrzyyivIzMzEAw88gN///vf42te+hm9+85v4wQ9+gOXLl+ODDz7Afffd53l9zz77LF577TXba08++SSmT58OAOju7sZLL72ESCSC22+/HQsXLkRXVxf+67/+C88//zxycnLw8ssv4+tf/zr+93//Fz/96U/R39+PrVu3QlVV3H///dizZw8A3XX23HPPIRQK4W/+5m/w5ptv4vbbb7/IOywYiQhDIbjmWLRoEQBg1KhR+MUvfoH33nsPx48fR21tLaLRqOtnbr75ZsiyjMzMTJSUlKC7u9thKG644QZkZmYCAGbOnInu7m7U19cDAJYvXw4AWLx4sU36nee+++7zdQdt2rQJkiShoKAAS5cuRXV1NTo7O7Fu3ToztnHHHXfgscceQ0tLC3bt2oWHHnoIiqJAURT85je/AQD88Y9/xKpVq5Ceng5Al6Pv6upKeu8E1yYi60lwzUHdK+3t7bj99tvR2tqKhQsX4u/+7u88P8MGwal0cyrvURTF8V5W0XWg0J4CgC43LcsyNE1zvI8QYjbyYSWl29racObMGcexvMYkEADCUAiuYQ4dOoScnBx87Wtfw0033YR3330XgN7bYaiYOnUqQqEQduzYAUDvTlZfXz/oXgivvPIKAL0HcnV1NZYtW4alS5di69at5o7gpZdewpgxY1BSUoLKykr88Y9/hKZpiMViePDBB82sMIEgVYTrSXDNUlVVhRdffBFr166FJEm44YYbkJOTg6ampiE7RyAQwM9+9jM88sgj+PGPf4zJkycjNzfXM03XLUYxd+5c/OAHPwAAtLS04I477kBfXx8efvhhlJaWorS0FPfddx+++MUvQtM05OTk4JlnnoEsy/jGN76Bxx57DBs3boSqqli3bh1Wr16Nd955Z8jGKBj5iPRYgWCYeeKJJ/ClL30Jubm5aGtrw8aNG/HnP/8Zo0ePHtBxrrZ0X8HIQewoBIJhprCwEPfddx8CgQAIIXj00UcHbCQEgsuJ2FEIBAKBwBcRzBYIBAKBL8JQCAQCgcAXYSgEAoFA4IswFAKBQCDwRRgKgUAgEPgiDIVAIBAIfPn/GrPT8u8x8T8AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "dropout_net = torch.nn.Sequential(\n", " torch.nn.Linear(1, 20),\n", " torch.nn.ReLU(),\n", " torch.nn.Dropout(p=0.5),\n", " torch.nn.Linear(20, 25),\n", " torch.nn.ReLU(),\n", " torch.nn.Dropout(p=0.5),\n", " torch.nn.Linear(25, 1),\n", ")\n", "torch.manual_seed(123)\n", "optimizer = torch.optim.SGD(dropout_net.parameters(), lr=0.03, momentum=0.9)\n", "dropout_net.load_state_dict(torch.load('dropout_net.pth'))\n", "losses_train, losses_test = [], []\n", "for epoch in range(1000):\n", " dropout_net.train()\n", " y_pred = dropout_net(x_sparse)\n", " loss = loss_fn(y_pred, y_sparse)\n", " optimizer.zero_grad()\n", " loss.backward()\n", " optimizer.step()\n", " losses_train.append(loss.data)\n", " dropout_net.eval()\n", " y_pred = dropout_net(x_test)\n", " losses_test.append(loss_fn(y_pred, y_test).data)\n", "plt.plot(losses_train, '.', label='TRAIN')\n", "plt.plot(losses_test, '.', label='TEST')\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### BatchNorm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our last technique is (mini)batch norm(alization) or just \"BatchNorm\". The idea is to get the benefits of preprocessing but for the inputs to hidden layers, and not just the input layer. See the [original 2015 paper](https://arxiv.org/abs/1502.03167) for details.\n", "\n", "Normalizing in this case means applying a linear transformation to each input node so that it has a mean of zero and variance of one,\n", "$$\n", "x' = \\frac{x - \\mu_x}{\\sqrt{\\sigma_x^2 + \\epsilon}} \\; ,\n", "$$\n", "where $\\mu_x$ and $\\sigma_x$ are measured on each minibatch and $\\epsilon \\simeq 10^{-5}$ is added for numerical stability when $\\sigma_x \\simeq 0$. Note that calculating $\\mu_x$ and $\\sigma_x$ on minibatches, rather than the whole sample, introduces some noise (more for smaller batch sizes) that can improve generalization: this is a recurring theme that we have already met in SGD, data augmentation and dropout.\n", "\n", "A clever refinement to the basic BatchNorm is to take the standardized $x'$ and shift it to a new mean $\\beta$ and variance $\\gamma$ (for each input),\n", "$$\n", "x'' = \\gamma x' + \\beta \\; ,\n", "$$\n", "where now $\\gamma$ and $\\beta$ are learnable (per-input) parameters of the model. BatchNorm is usually applied **before** an activation module (unlike dropout). Since our network has a `Linear` module before each activation, the linear bias is now degenerate with the $\\beta$ parameter, so should be omitted using `bias=False`.\n", "\n", "Just as with dropout, we need to rebuild our network to include BatchNorm:" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "torch.manual_seed(123)\n", "batchnorm_net = torch.nn.Sequential(\n", " torch.nn.Linear(1, 20, bias=False),\n", " torch.nn.BatchNorm1d(20),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(20, 25, bias=False),\n", " torch.nn.BatchNorm1d(25),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(25, 1),\n", ")\n", "torch.save(batchnorm_net.state_dict(), 'batchnorm_net.pth')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this example, we find that adding BatchNorm significantly degrades the learning performance with a \"good\" batch size of 200 and learning rate of 0.1, presumably because the relatively small tensors in this problem have large fluctuations in the mean and variance estimate on each minibatch:" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEJCAYAAABlmAtYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2de3zUVPr/P8lML5Rya2mLtFyEQusFBEG5g3iBFSmg4lpQCiJ7+a24u6yKuLJfEAVdYHFXdIUVr8iKqHgBEVl1vdDiKlCuUmqFUmhLaWktlEI7M8nvj2mmmUwyk8wkmUz7vF8vXzLTSfLk5OR8znme55zD8DzPgyAIgiD8wIbbAIIgCML6kFgQBEEQASGxIAiCIAJCYkEQBEEEhMSCIAiCCAiJBUEQBBEQEguCIAgiIPZwG2AkNTUXwHHap5EkJsbj7Nk6AywKDbJLO1a1jezSBtmlHa22sSyDTp3aKv69RYsFx/FBiYVwrBUhu7RjVdvILm2QXdrR0zZyQxEEQRABIbEgCIIgAtKi3VAEQVgbnudRU1OJxsZLAJRdJmfOsOA4zjzDVGJVuwB/tjGIjo5Fp05JYBhG9flILAiCCBt1dbVgGAYpKWlgGGVHh93Owum0XqNsVbsAZdt4nsPPP1ehrq4W7dp1VH0+ckMRBBE2Ll6sQ7t2Hf0KBaEvDMOiXbtOuHhRWxYXPaEAuCqK0JC/Fa6KonCbQhAtDo5zwWYjB4fZ2Gx2cJxL0zH0lPzgqihC/dblAOdAIxuFuInzYUtJD7dZBNGi0OI3J/QhmDKnkYUfnGUFAOcAeB7gnO7PBEG0WPbu3Y1bbhmFWbOmY+bMabjnnqnYtOktv8d89NH7+M9/tiv+vby8DFOnZsn+berULPzrX//0+m7p0sXYtm2LduMNhkYWfrB3zUQjGwVwToC1w941M9wmEQRhMBkZV+D55/8FAKivv4B77/0lrrtuCC6/vJfs7w8e3I+BAwcFfb1Nm/6N0aPHIjPziqDPYQYkFn6wpaQjbuJ8OMsKYO+aSS4ogrAIRaW1OFpSg4zunZCe2sGw6zQ0NIBlWcTHx+OLLz7Dxo1voqGhAQ5HIx577P/gcDRi586vsWfP90hM7IwePXpi2bInUFNTjdjYWDz66F/Qtm1bNDQ0YNGix3Ds2E9o1649nn56JTp0cGcizZhxH5YtW4yXX34TUVFRXtfPzf0GL730InieQ9euqXjkkT8jISERU6dm4corr8aPPx7FX/6yBMuXL0OPHj1w/Pgx9O2biauv7o/t27fi3LlzWLZsJXr2vDzksiA3VABsKemIGTiRhIIgLEJRaS1WvJWPzV8fw4q38lFUWqvr+Y8ePdLkhsrG1KlZGDhwEBISEvHhh+9h+fK/4/XX38L06TlYv/41XH/9EIwcORpz5vwWQ4YMw9/+9gzGjLkR69dvwuzZv8brr78MAPj55xrcffc9WL9+ExISEvDZZzs81xs37lakpqbh1Vdf8rKjpqYaK1Ysw9NPr8Trr29Ev37XYNWq5Z6/Dx06HG+9tRmdOiXgp59+xD33zMRrr72Fgwf34/Tpcqxb9zpuvnk8Pvposy7lQiMLgiAiiqMlNXC6OPA84HJxOFpSo+voQuyGunChDg899Hv8+99vYNmyFcjN/QYlJSeQn78HLOvb1963by8WL14KABg2bCSGDRuJ8vIydO6chCuvvBoAcPnlvVFb+7PXcQ8//BhmzZqO0aPHer774YfDuOKKq3DZZV0BAJMm3YH161/z/F04HwAkJCSib1+3mzwpKRmDBl0HAOjS5TLk55eFWiQAaGRBEESEkdG9E+w2FiwD2GwsMrp3MuxabdvG48Ybb8Hu3d/jV7+aibKyUlxzzUBMnXo3eN53xrk4DZjneRw/fqzpe5vX76THJiZ2xoMPzsOyZYvhcDiafsP5HONyNae7xsTEeP4tdV9Jr6cHJBYEQUQU6akd8Mi0gbh9dC88Mm2goTELl8uF/Pw9iI2NAcMwyMmZjWuvHYyvvvqvZykNm83macQHDBjocTHt3v0/LF++VPW1BHfUV199AcA9cvjhh4MoL3ePDD76aDOuvTb4QHqokBuKIIiIIz21g2EiIcQsGAZwOp1IT++L+fMfxzPPPInp06eCZRlcf/0wHDiwDwAwePD1WLv2n4iPj8e8efPx178+hffff7cpwL1Q07UffvgxzJhxNwC3a+mRRx7Hn//8MBwOJ7p06YIFC/5P9/tVC8PLjaVaCGfP1gW1nntSUjtUVp43wKLQILu0Y1XbyC43p0+fQJcuPQL+zqprMFnVLiCwbdKyZ1kGiYnxir8nNxRBEAQREBILgiAIIiAUs5DgqihCTeFxuDpcTnMrCIIgmiCxECEsHFjftLwHLRxIEAThhtxQIpxlBeBdDoDnwLto4UCCIAgBEgsRp6PS4OBZuHgGDp7B6ai0cJtEEARhCcgNJeLQ+Y7Ye34cettP4ydnF1x7viMCJ/URBEG0fGhkISKjeyecQgq+aOiHU0gxdBkBgiCsR3l5GUaOHIzvv//W6/upU7M8M6mNZuTIwaZcRys0shAhLCNw6mw90hLjDF1GgCCI4HFVFBm2dYDdbsdf/7oUb7yxEXFxbXU9dyRDYiEhPbUDhg1Is+TsWoIgjN/uuHPnJFx33RCsXv13PPro415/e+ONV7BjxydgWRbXXTcUv//9H1FeXo6HHnoQHTp0RExMDMaNuxV5eTtx7lwtqqoqMXnyHTh9+jT27v0e7dt3wMqVzyEmJgZr176APXu+x7lz59C5c2csWfI0EhISdbsPvSE3lB+KSmvx8a5i3dfLJwgieMzY7nju3D/iu+92ebmjvv02Dzt3fo1169bjlVc2oLT0JN5//10AQEnJCfzf/z2Jv//dvUXqkSOH8fTTK7Fq1fNYvfpZDB06HK+/vhEA8N13u3Dq1EmUlBRjzZpXsHHjZqSkdMGnn36i+33oCY0sZCgorsbH3/yEnQfL4eJ42G2s4atbEgShDjO2O27bNh6PPrrQ444CgD17vsPNN49HbGwsAOC22yZh+/aPMWTICHTqlODZdwIA+vW7Bm3bxqNtW/daS+L9Jc6fP4+0tG6YO3cetmz5ACUlJ3D48EGkplo7+5LEQkJRaS1WbsxHo6N5AS4jNlghCCI4zNru+Prrh3rcUQB8FiV1b77kXppcvLcE4Lu/hN3u3dQWFBzB4sWPIzt7OsaOvQk2Gyu7P4aVIDeUhKMlNT4rNRq9wQpBENowa7tjwR119mwVBg0ajM8++xQNDZfgdDqxbdtHGDQouMylffv2YODAQZgyZSq6deuOvLydnv0xrIrlxeKHH37ArFmzTLteRvdOsNuFXbgY3DCgK7mgCKKVIrijHA4Hhg8fheHDR+L++3MwY8YvkZLSBXfdlR3UeW+6aRyKigqRk3M3HnzwN8jIuMK01NxgsfR+FidPnsSWLVuwZ88evPzyy5qPD3Y/i7MXHPj2QCkyuneylEjQHgjasaptZJcb2s/COPTez8JSMYt169Zh586dns+vvPIKfve73+E3v/mNqXZk9kxAYtuowD8kCIJoJVhKLObMmYM5c+aE2wwPRk78IQiCiCQsJRZWwuiJPy2NotJaHC2psZzrjrA+PM+DYZhwm9GqCCb6YEqAu66uDhMnTsSpU6c8323ZsgUTJkzAuHHjsGHDBr/Hr1271mgTfZBO/GkszEVD/la4KopMt8XqFBRXY8Vb+dj89TGseCufJjFaFCtOMmVZG1wuZ7jNaHW4XE6wrE3TMYaPLPbv34+FCxeiuLjY811FRQWeffZZbN68GdHR0cjOzsaQIUOQnq5vz91fsCYQna+8FuX5H4F3OQGWhbNwJ8C54LDZcdk9ixGblqGjpepJSmoXluv648sDhXC5uKa8cw6nztZj2ADrTDCyYpkB5tpVUFyNlRvz4XRysNtZLP3tCGT2TAi7XTyfiPPna9GpU2cwjP++q91uzeRNq9oFyNvG8xxqa2uRlJSo6VkbLhabNm3CokWLMH/+fM93eXl5GDp0KDp27AgAGD9+PLZv3465c+fqeu1gs6GSktrhfExXtLnNPfGHrzsLR8GXAM+DdzlR9cNexMR0DXgevbFqBk2/3p1hs7GAi4PNxiItMc4ydlq1zMy269sDpXA43YLudHL49kCpbBKH+eUVC6fzPEpLSwAov6ssy1pyHoJV7QL82cYgOjoWQKzXsw57NtTSpUt9vjtz5gySkpI8n5OTk3HgwAGjTVHNpVNH0fDDXti7ZiJm4ES4KorgKMw1dHmBSCazZwIemTaQYhYWJqN7J9htLFxNgm6VSaYMwyAhITng70j0taO3bWEJcHMc5xXQslKAy1VRhPKPl4N3eQe2zVheIJJJT+1AImFhhOX3SdCJYAmLWHTp0gW7d+/2fK6srERycuDehRm49+F2eq1oaUtJ9/xHEJEKCToRCmGJzAwfPhy7du1CdXU1Ll68iB07dmD06NHhMMUHe9dMMDY7wLDkciIIgmgiLCOLlJQUzJs3Dzk5OXA4HJg6dSr69+8fDlN8sKWk47J7FqOqKWYhHk3QJD2CIForponFF1984fU5KysLWVlZZl1eE7FpGT7ZTjRJjyCI1ox1E4Qthhm7cxEEQVgVEguV2LtmAmwUxTIIgmiV0NpQKqH0WYIgWjMkFhqg9FmCIFor5IYiCIIgAkJiQRAEQQSExIIgCIIICIkFoRtW3C+BIAh9oAB3EETSTG6zdrArKq3Firfy4XRxsNtYPDJtIK1D1MKg3RBbNyQWGigqrUXZkQMYUPwGGN5p+ZncZjbgR0tq4BRtgHS0pIYaFAnixtaqGzIpQZ0BgsRCJcLLckPUfvSPdcDGwGtVWitiZgNu1f0SzEBNj1va2C7rGCe7+ZBVoc4AQWKhEuFl+RFdMC7WBgYcWIvP5DazAW+t+yWo7XFLG9uDP1Xhhv6XhcHi4GjNnQHCDYmFSoSXpcSVhLX14zHrWhYpmQMsO6oAzG/AW+N+CWp73NLGtl/vzmGwNnhaa2eAaIbEQiXeL8sgdI2Ql8XoBryotBZfHihHWmJcq2xA1Pa4pY1tZs8Ew7fj1Dsg3Ro7A0QzJBYaoJfFG8EFIzSUWoKeLSWzRkuP28z6QwHp8NBS6rUcJBYqacmVIFjELhhoCHq2tIbMbBFQUw/9ucciuS5b2faWVq+lkFiowOxKoPcLYdQLFmzQsyVm1pjRiGmph0rPJpIbNKvb3hLrtRgSiwAUldbiw53H4HRy4GF8JRC/ECzLYFS/yzC832VBX8/IF0xwwZw6W68pZhHpmTVSYTCrEdPSGCm5x6TnyDtYbtmeuhSrN8aRXq8DQWLhB6ERcDg5AADDwLBKIDRAZ2sviV4IHl/uK0PuodN4ZNrAoCZyGf2Cpad2wLABaT7BWn897UjOrJETBrMaMa2NkZx7THwOhmWw82A5XBzvuRczJwtqHY1ZvTGO5HqtBhILPwiNAAAwAK7s2QmTR/bSvRJIRxM2loHLxYNv+rvQAA0bkKb53OF4wdT0tCM1WUBOGMwqYz0aI/E5ztZewlf7y7zuJZg6FgzBjMYioTGO1HqtBhILP0gbASOEAvBugHiOx6hrugIAdh4sB8fxITVA4XjBrO4uCAU5YTCzjPVojIRzFJXWIvfQaV1FTo8AvBrbCfMhsfBDoEagqLQWeQfLASCkuIK0ARLONbzfZbo0QIFeML2Ds1Z3F4SCUp2IxEZMb5HTIwBvFGqXZLHyqCXckFgEQKkRKCqtxV//vRcul9tZtPNgOeZPvzboSjbi6i4AvEVH7wZI7mUwIjgbjtGMmS96JAqDElrvxV856xGANwI1ddzqmVZWgMQiSI6W1HiEAgCcLj4od4u0kg7vJ79ekNaZ0mozdoxyGbWECWgtuacZzL0FKudQAvBGlrWaOt6SXad6QWIRJBndO8FmYzyCYbcxQQ2l1VRSrTOltWTstASXkREveqT1NLU0toHuTelcgcpZjdtW7m9y9gjX00M81NRxPd6Dlty5AEgsgiY9tQMenX5tyDELNZVU60xpLRk74XYZCfaGcm0jBC8cPc1gY2Bahc3fvRUUVyueS1rObdtE4eNdxapiN/5slJv7kXvotNdvQ0npVVPH1fzGnxhEWuciGEgsQkA6jJa+OGrPEaiSSnPjz9ZeQlFprabgob/rhMtlxLIMGMArzz/UdFC9ApjhCMAGGwPTurSHv3s7+FOV4rnE5dy2TRTe+uxHXQRKag8An9+GmtIrV8elZePvPQgkBq3BjUVioQOh9ioCNdbCS5pfVIX/fFeCr/Y3T9TT0kuyQmBW/FJxMnNJhN8EI7paXXNqytyMJTyEOQ/BxsC0Lu3h79769e7sVySF4z/eVaypcfQnUFJ7AOie0itFz9FYoPtrKZBY6IBcRRK+13N56FNn6+HieNXZJuEWBjmkoyQG7pnqYBjUX3IaNpQPpudndBn6TsYEmuaAaoqBKTX+/u5Z6d4yeyaoEslggtn+ziu1x2ih1lofAt1vONy5ZkNioQNyvlwjGr1AvT4zCDWIJ32pTlXWYcOOQnAcj0+/Pwme4w1Zg8tKPT8hs62krNZrMubopsmYgPYYmFzjH+w9qxHJYBpHLeJrtFDrLXbCb1qiSAiQWOiAtCIZ5b9U2+tTgxGpk2oRv1RHS2rA8U3uKI4HyzLged9Z63qLlJZz6JnlIs5sY5qWdhFm6WsViEB2Gd3bjeTG0Wixa4mQWISAq6IIzrIC2LtmIj013asiGdWL1aPCBtvohyqCaoKt027ugwsXHV6/+XJfqWf0YbfrI1JabNZzlCguQ7ZpaZfEDrFBCZgauyKxgTMrBVXcYRF/DgeRkHZLYhEkrooi1G9dDnAONLJRiJs437Mft9X9l8E2+qG4coIJtgrHvdkkFIA7S8bMTBO9R4lKS7uE2y6rYGYKqr9rmdl4R0raLYlFkDjLCgDOAXdajxPOsgKPWADW7tGF4ssOVgSDCbYKxwlCAQAso23yY0FxNb49UBr0S693rEMoQ617gBhpl7hhNHOJcjnMFEGla2ltvEMVlkgRfsuKxffff4933nkHPM/j+uuvx1133RVuk7ywd81EIxsFcE6AtcPeNTOs9mipsKE0+sGKYLCNW0b3Toiys3A63T7+e8b1Vby+tAz0cF/pNUqU2ia3B4hedoUym3tZxzgkto0K2i5/11Fjk9LEv6H9U3W3S6lOamm8/U1iDNUOq2FbvHjxYrU/bmxsREVFBdq1M773UVBQgJkzZ+IXv/gFnn/+eUycOFHzOS5ebHTPetZI27YxqK9vVPx7UWkt/lfciHa9+qFDl1TEDJoMW0o6ikprsevwabAsg4T2sV6/33X4NMrOXsDh49U+fw/VLuGFP1xcjW8PV+CKHp0Uzy/Y0rlDGwy9sktQdqi1S0xC+1hc0aMTkjq2waSRl6t+oYTjkju1wZRRvXBt3yTZ30nLoF1clJf7iud5JHdsg77dOipeS+n5JbSPRd9uHYMuK7nn061L+4BlFgg5u7TUBQDYdfg0DhdXezKyKn++iMT2sbrUi2BsEteTazOS8NZnP+JwcTW+3HsKmQHuRStKdZJlGXx7uAJoSrSYNPJyxet+V3AG+wor3e0MzyOpqY4p1SUtdkjRck5A3XsphmEYxMVFK/494MjiP//5D7799lvMmzcPkyZNwvnz5zF37lzMnDlTtRFqWLduHXbu3On5/Morr4DneaxcuRI5OTm6XisUxD2xj2wsHpk2CukpykNXn932gJCCtHKo7QmF2zca7KhEzXHSMthz9Iwm95WRZSP3fIzaZCjY+QPCtsH7fqzEoWNndXW9aLVJeN7iiX9OpzHuGbm6pWU0KZfOHuzGToHKO9zxlYBisXbtWixduhQ7duzAgAEDsGTJEuTk5OguFnPmzMGcOXM8n8+dO4enn34a06dPR79+/XS9VigoVfxA3wsIcwj03PtY7TA2UnyjwSAtg0EZySg8WavKfQVoE1ytz81MN0Ow8wc+3HkMPxyvUVU3tDaGobgghePsdnPdM2o7NnLp7Fpnt6tBbXxFnE2od/wpoFjwPI+MjAy89NJLGD16NOLj48EH49vRyFNPPYXTp0/j9ddfx2WXXYaHHnrI8GuqQaniB/pe6LkxDGT3Pg6lMqntCUWKbzQY5MogLSledSBZTdkEO/owOztObm+UQPZNHtkLhSfzVdWNYEYKwdy/+Di9YhZG9MKlwmLEe6YmvuJ0cu4YHc8bEn9i+AAt/1133YX77rsPS5YswZYtW/DDDz/gH//4BzZv3qzqAnV1dcjOzsaaNWuQluYeem/ZsgUvvvginE4nZs6ciXvuuSf0O5Hh7Nk6L1eEWpKS2vkNPvpbatnf923bROHCRYfX3scsA9w+uhduG9Yz4PX0eGGMeFkClVc40WJboLL5eFcxNn99zO9zU1u+auzS+qxCdaUVldaqEletS+brgb/yUltORrgalewy4j3zt3mZy8UBDONZAYFlgHtvvQI39JffH0cOlmWQmBiv+PeAI4tHH30Uzz//PP70pz8hKSkJL774IhYuXKjq4vv378fChQtRXFzs+a6iogLPPvssNm/ejOjoaGRnZ2PIkCFIT09XPlGQ+LvxQPgbwiUltZP1Oav9vqC4GnmHT8PpdA+vh/ZPVbxeQXE1Vm7Mh9PJYUteMZb+dgQyeyYEcUf+bVS69sGfqtCvd+eA1wx3yqU/1NoWqGyG9k/FlrxixecmflZ2OxvwWfmzK9C55J7NlwfK4RL1+E+drdcUG9FSTss6xqmuG3L3FsyxcvZpKfNQy0eLXVreMy3XkZ5T/CzaxUXjpQ8PesqiX+/Our6XAcVi8ODBeO211wC4s6FWrVqFrl27+j+oiU2bNmHRokWYP3++57u8vDwMHToUHTu6s1LGjx+P7du3Y+7cuUGY7x+jRhZixLO4xfMs/JHYNgoPZw/07F1Q83M9KivlRwzfHiiFw9k8zPz2QClqfq73WqFTzx6MeBSkdgnqljKyCITw3ITyTmwb5XVuuWelNBIMZJe/cyn1kNMS49xLfDf1+NMS4zTfu5xdcj3axLZRnl6rlmsE27tXKi8tZa5H+ai1SwkjRhziZ9FBVD8zeyZosi3kkUUo2VBLly71+e7MmTNISmpOf0xOTsaBAwcCnsuK+JvFrQZhgxd/y41Lg3ziRQr12g9CQPwiM6IhbahBOqOyNbTOJ1AjsGrWWzIjJuTvXEoxAyNiI0Yud2LEjHh/Za7X3JRg0bMslewNNuNQDaZnQ3GcuyES4Hne63MkEWgWtz/UvjTSIN+3B0r97geh14sMXnlRPy0YuT+22vOq3XBJj31J9Gqs/Z3LXwOpd2MRzsZdDVrLXK58zEop16ssw5UCb3o2VJcuXbB7927P58rKSiQnJwd9vnASyixurT2i9NQOSEpqh5qf6332gxBWLdX64kl7J/4W9QMQ1E6ARqXrajmv+Lf+BFYPW/VsrJXOZWZ2Vbgbd7XnDGWUYFZKuV5lGa4U+IBiwbIstm3bhm+++QaPPvoovvrqq5BGAsOHD8fq1atRXV2NNm3aYMeOHXjyySeDPl84saWkI27ifM0xC0CfdMJQYhZKvRM5m0LpyRiVrqvlvOLf+hPYSEotNtLdIL2O0Y27ngRTV8167nqVZbjqqepsqIceekhzNpQcKSkpmDdvHnJycuBwODB16lT0798/6POFG1tKuiaREBPsSyMcF4qf1Z/fW3quUHoyRvWCtZxXTmCF5AIzbJUj1AUOzcQsYdKKXP1XU1fl9t4OtPJxsB0y6XFay1LpHOFY1Vp1NlRpaSlOnDiBjRs3ar7IF1984fU5KysLWVlZms9DNBOq3zLYnnkwPRmjGhst5xX/tqi0VjG5wIyGsai0Fis3upeAsfKS1FrQI0AcygKIQhkGqqv+lsqXu2aw75kecQV/5wiHgAcUi+LiYjzwwAM4c+YMOI5Dp06dsHbtWvTu3dsM+wgFQvVbhtIzj/SGLdzLnhwtqXHP6G8hy674W3JC7X1pbVyDzQjT+uyDrSvBjHD0urZRBBSLJ598EnPmzMHtt98OAHjvvffwxBNP4I033jDcuJaAUSl5evgtg+2ZRzrhjk1kdO8Ee9Oy61aPjajB35ITRu3CGGxGmNZnH2xdCXaEo8e1jSKgWJw9e9YjFABw5513eibpEf4xMsXNyr39cG8RGWgzn3CXXXpqByz97YiIiVkEQtyogWHABTE/R2vDqFeCSKDjjLqOnDgK36uNpZhNQLFwuVz4+eefPTOuq6urDTeqpWD0MNKKvf1QBFIvv7eazXzCXXaZPRMM2WQoHIgbNWHmv3TzIiNcnaEmiBj1ezXHScVRPNlWTSwlHAQUi3vvvRd33303br31VjAMg23btum+PHlLxWrDSDMIViD1GoVJr3/wpypNi6kR2pGKfFpSvOYlYwB9BNzoUa1e55eKo9XiE3IEFIu7774b3bt3x86dO8FxHBYtWoThw4ebYVvEY7VhpBkEK5B6vSzS6/fr3VnzOayA3o2ekUuuKPWIjdjXIRhbjDy/moX61C7NEWrH0mihVLUH97BhwzBs2DDP57/97W+W2V/C6oRjGBnOmEGwAqnXKEx6fa2LqVkBvRs9s3cBFM5t9sja6N653PkDrSyrtuyl9RbQtmJCsEKmBVViIWXDhg0kFhbFjHVjQllsTwk9R2FW8vMGg1KjF2wnwMhGNFBWkpkja6PFKZjzayl7od4G8w4HI2RaCUoszNgpL9JQu1S50b1+o3tXRvZggmnkzZ4MZsZ55RolKy65AgQWBDOF22hxCub8RgtMKNfRSlBiEamrxBqF2qXKzej1G11ptPZgjBRHPcrzy32l7nkBHA+7PTyr4kqRc0l8uPOYZ2teqyy5Ij6/3DnD4Q41WpyCyaR6ZNpA2eVllAjmHTZjFKcoFjt27JD9nud5cBynuyGRjNqlys3IeDC60mipyEaLY6jlWVRaizebhAIAnDo+Ez1m2ItdEg6n+51jGFhqyRUlwrWMtlVRs3eNQLDvsNHPWFEs1q9fr3hQv379DDEmUlG7VLlZAT8jK42Wimy0OIZankdLarx2UmQZRpdnUlRai7O1l2BjmaCXjxfb6HQ1CS9EDywAACAASURBVAWAK3t2wuSRvQxveKX7xlspTuLPXitmHQZTFv7e4XDda1BiQXgjXar8uDMJR3f5ZjK0lFRatWJktDiGWp4Z3TshqmnZDYZlcM+4vrrk+Ys3Whp9TVcM73dZ0OeVlqFZQuE1mgE0u+jMzISy+ihGz7LQYx2uYAkqZkH4IixVHqjiRnqmjhbMEMdQytMI+8S9SJ7jkdghNuTMLrM7GOLRDABLxknE6D2K0bvnrmdZ6LEOV7CQWOhMJMzE1EooL4/VxVFv+4zoUZtdhsI9CAF1K8ZJxHXSyJ67Xg2wuCzU7gcvhx7rcAULiYXOhFpxQ2mY9ewRiX3WWpZsaO20BFej+B6CjVkYiVyDbkTP3eXikHewXPeZ9Gr2g1dC+mzE63AZPelRUSzKysrQtWtX2b99/fXXGD16tGFGRTKhNBahLsKnV49IfC6GYcCb2HtpCVh9NKUGK9+D3Oj9tmE9dbFX3NljWAY7D5bLNuZCZ2po/1RNC0KKbfe3H7w/xM9GWIfLDDFnlf7wwAMPeP794IMPev3t2WefNc6iCMZVUYSG/K243F4ZVOVVWrbY6GP9nYvneLAsAzZIV0SwFJXW4uNdxSgqrTXlela7PqGM0KAbUSeFzt7to3thVL/L4OJ4n3dK6Ext/voYHl+Tq6mOiG1nbQzsttDerfTUDroJZSAURxbiWdonT55U/BvhRu3EPH+E4sLS028rPZeZGReA/1GSGWmDVs+uae2YNclQ2H5X+k5Jg8yhBP6F81nJzaeEoliIZ2lLZ2zTDG5f1E7M80coL4GeL1C4/e7+1kYSUjpZlsG94/rihgGppl2fCC/SjoLRz0TpPRB3puz20AP/kVK3VI0siMConZgXiFBTQfWqeOH0WSuNko6W1Hhy/zmOx4YdhUhLitfdTrWjNKtMBLOKHUZSUFwdltGe3HsgFhGtMYtIRlEsOI5DbW0teJ6Hy+Xy/Btw755HeCOdmKd1VEE0469HxzbNigYAjufDtmSKVVxVVrHDaA7+VGWp0Z4gIklJ7SJuCfxgURSLwsJCDB061CMQQ4YM8fyN3FDyCBPziNBR6tHdO66v10SkcC2ZYhVXlVXsCBa1o6J+vTu3ul0nrYaiWBQUFJhpB0Go4oYBqaamCyph5nIWkWBHMGgZFWX2TIj4+SuRjt9JeYILym63o66uDnl5ecjIyECPHj3Mso8gfLDCHIBwJwFYzY5g0DoqssJzb80oikVRURF+/etf4y9/+QuGDRuGu+66CwBQV1eHZ555BiNGjDDNSIIIN3LuEqs0XlaxQytmjYpaQwKAGSiKxfLly/HHP/4RY8eOxXvvvQcA+Pjjj1FRUYF58+aRWPhB7a55VoNeKnmMDiK31nI3Y1TUWhIAzEBRLMrLyzFp0iQAwP/+9z/cdNNNYFkWl112Gerq6kwzMNLQY3JeOKCXShkjg8itvdyNXmwwlB0GrYJVOhOKy32wbPOf8vPzcd1113k+NzQ0GGtVBCM3OS8S0HO5kJZGKMtLBFo2hMrdGAQRPny8JqSVc8ONeGmRFW/lh3X5GcWRRYcOHVBQUIC6ujpUVlZ6xGLv3r1ISUkxzcBIQ6/JeWYTyVk1RhOsu0TNqIHK3RjCtcOg3lgpNVpRLP70pz9h1qxZqKurw8MPP4y4uDi8/PLLWLNmDV544QUzbYwoInVyXiRn1ZhBMO4SNS86lbsxhGOHQSOwUmdCUSwGDBiAr7/+GpcuXUL79u0BAAMHDsQ777yDnj17mmVfRBKpk/OsllVjFV9tsKh90a1W7i2BliLCVroPRbH4+eefff7dq1cvz+eOHTsabFpkE6kZUVahJQR+rfSit0Zaighb5T4UxWLo0KGeZT2kiwoyDIMjR44Ya1kEE6kZUVbCSr7aULDKi04QoaIoFlOmTEF+fj5uvPFG3HnnnUhPN7ex+/HHH7F69WrExcUhKysrouZ16LFceSQQ7G5harCSr5YIL0WltfjyQDnSEuNIeMOIolg888wzuHjxInbs2IGlS5eivr4ekyZNQlZWlieGYST19fX485//DJvNhlWrVkWUWERqRpQWxG6iLXnFeDhbXzcRuXAIoLmeCZ2GSHRHthT8rg3Vpk0bTJ48GZMnT8bp06fx4YcfIicnBz179sTf//53XQ1Zt24ddu7c6fn8yiuvoKSkBAsWLEBOTo6u1zKaSM2I0kIou4WphVw4hLieIYLdkS0Bv2Ihprq6GtXV1aipqUFiYqLuhsyZMwdz5szxfD506BB69uyJjRs3Yvbs2ZgwYYLu1zSSSM2IUkuou4URhBrIHWkd/IpFeXk5PvroI3z44Yew2WyYNGkSNm3aZMqkvIaGBjz++OOIj4/HmDFjDL8eoY3WulsYYS5CPTt1tr7VxCysmjKuKBYzZszA8ePHMWHCBKxcuRJXXnllUBeoq6tDdnY21qxZg7S0NADAli1b8OKLL8LpdGLmzJm45557fI4bNGgQBg0aFNQ1CXNojbuFEeaTntoBwwaktYo6ZuWUcYZX2Gw7MzMTMTExYFnWa2c8nufBMAz27t0b8OT79+/HwoULcfz4cWzfvh1paWmoqKjAtGnTsHnzZkRHRyM7OxurVq0yPdvKLC6dOoqLJw6jTY+rEJuWEW5zCIKwMO98Xog3PzkCjgdYBrj31itw1019w20WAD8ji88//zzkk2/atAmLFi3C/PnzPd/l5eVh6NChnkl948ePx/bt2zF37tyQryfl7Nk6z37NWtCrpyyeb1Gjw3wLq/bgrWoXYF3byC5tWNGuotJa3d1jaYlxsNlYoClGk5YYF/R9ay0zlmWQmBiv+HdFsUhNTdVmmQxLly71+e7MmTNISkryfE5OTsaBAwdCvpYVaS3zLQiitWFUSq+VU8ZVZ0PpBcdxsm6tlohnvoXLAYABE6us2gRBRA5GpvRaNWVccT8Lo+jSpQsqKys9nysrK5GcnGy2GaZgS0lHzPDpAMMCPIeGvH/DVVEUbrMIggiRUPY4iVRMF4vhw4dj165dqK6u9swQHz16tNlmmAZ/qQ4AByCyNkMiCEIZwV10761XWCpjyUhMd0OlpKRg3rx5yMnJgcPhwNSpU9G/f3+zzTCN1rD0B0G0RlpTSi9gklh88cUXXp+zsrKQlZVlxqXDTmtY+oMgiJaP6SOL1khLX/qDIIiWj+kxi9aMq6IIDflbKchNEETEQSMLk6ANkQiCiGRoZGESchP0CIIgIgUSC5Owd80E2Cj3nAvKiiIIIsIgN5RJUFYUQRCRDImFiQgCIbigSDAIgogUSCxMhILcBEFEKhSzMBEKchMEEamQWJgIBbkJgohUyA1lIhTkJggiUiGxMBla+oMgiEiE3FAEQRBEQEgsCIIgiICQGypMuCqK0FiYCwZAVN8R5JoiCMLSkFiEAVdFEeq3POPeEAmA4+g3iMtaQIJBEIRlITdUGHDPt3A2f0FzLgiCsDgkFmHAPd9CNKijORcEQVgcckOFAVtKOuKyFlDMgiCIiIHEIkzYUtLRhgSCIIgIgdxQBEEQREBILAiCIIiAkFhYBFdFERryt8JVURRuUwiCIHygmIUF8NrngrHB1q0/2LgOFPgmCMIykFhYAEdhLuBqdH/gnXCd2AsXvCfruSqKUFN4HK4Ol5OAEARhOiQWYcZVUQTH0W/k/9g0Wc9VfQoNO9ejnucAG+2wRxCE+VDMIsw4ywoA3iX6hmn+J2sHExuPhp3rm37DAy4HzfYmCMJ0aGQRZuxdM9HIRrmX/2DtiBk+Ha6qE2AAsJ17wHlst4+Y8HVn4aoootEFQRCmQWIRZpR2z/MEvYVYBuDejpVh4Cj4Eo7CXC93lKuiiHbgIwjCMEgsLIDc7nnuxQYdTZ8Y2FKvQlxKV5zP/w/A8554hhD89mRTsRTTIAhCfyhmYVHciw1GuUcTtijEDJ6Cdv1uaP5OtPigR1hEIkIQBKEnNLKwKHLuqdikdl7fAUBD/lYwsfFuEWmKe9AKtgRB6A2JhYWRc08JnxsLc+E8+o07+M1GIWb4dPCX6ihmQRCEIZBYRBiygW/O6REKwQVFgkEQhJ5YOmbhcrkwY8YMHDx4MNymWAbvwLcAA76xHvVbl6Nx93uo37qc1pgiCEJXLC0Wa9asQXJycrjNsBRegW/G5v4/z8Fx4FPA5T/ITYsVEgQRLJZxQ61btw47d+70fJ42bRr69OkDjuPCaJX1EAe++bqzcBR8CYAHeM4tHOBlg9yUXksQRChYRizmzJmDOXPmeD7/6U9/Qnx8PA4dOoSSkhKsWLEijNZZCyHw7aooci9CKJr9LQ1yC5P1+LqzPum1JBYEQajFMmIhZdWqVQCA1atX44YbbgivMRZFafa3gHg04XZZ2ZqWDmHc6bY6QjPICaJlY3jMoq6uDhMnTsSpU6c8323ZsgUTJkzAuHHjsGHDBr/HP/jgg+jXr5/RZkYstpR0xAycKNtAe03W4znYuvX3xDga8v6tS+zCVVGEym1rUb/lGQquE0QLxtCRxf79+7Fw4UIUFxd7vquoqMCzzz6LzZs3Izo6GtnZ2RgyZAjS0/XvjSYmBt97Tkpqp6Ml+qHFrktXXovy/I/Au5xgbHbEJXbG+RIegNsVFVN7HJ2uHqjuXKeO4uKJw2jT4yrEpmV4viv/eDl4p8N9TkDzec2gJTxLMyG7tGFVuwB9bTNULDZt2oRFixZh/vz5nu/y8vIwdOhQdOzYEQAwfvx4bN++HXPnztX9+mfP1oHjeM3HJSW1Q2Xled3tCRXNdsV0RZvbmt1ULgDY/6UnxtHQ4XJV5xO7s2pEwfGGH/aCd4mEAtB0XjNoMc/SJMgubVjVLkC7bSzL+O1gGyoWS5cu9fnuzJkzSEpK8nxOTk7GgQMHjDSjVSO4pwTBEGIcTGy8V3qtXLwhUHDca3l1hkVUxqgWvxUsxWaI1orpAW6O48AwzRv88Dzv9ZkIHXGDBsAnZdbeNdMzC7wRLMAyAM+hUbRsCBMbj4a8f3sHx8F5peUKAfaY2uNoaOHbvboqiryWWNEz/ZgEiIgETBeLLl26YPfu3Z7PlZWVNPFOR6TzKaL6jpBfkdazXAgHCFNZXA73rnzgALgD4W4XEwd75hiw8Yk+DZotJR2drh5o2aG4HigtsaJH+jHNfyEiBdNncA8fPhy7du1CdXU1Ll68iB07dmD06NFmm9FikS5XzgM+y5rbu2Y2jRQkMIw7tZYXTfJrOi667wifrCthRvilU0dNu79wILvEik6r+zoKc90iRMvLExbH9JFFSkoK5s2bh5ycHDgcDkydOhX9+/c324wWiyeO4HIAYGDr3APRfUf4uDliRs5o2tubA1gbbN36w1Wyv/lErA0xI+5VXMlW3CMuz/8IbW5Tv2ufmW6XS6eOouGHvX6vFcgeI2Izl04dxcXv/gNnwdfNXzIsLS9PWBaG53nt6UIRQmvNhmo88mWzENiUXRviRtJZVoDG3e+5e7hgYL/iBrQZNVPxGM9SIzwPMCyiB9/hblQlS6dLr+3l0mFsiBk5A9FX3OBjj16xgIsfL3dnbDE22UY+mLLSw/V08ePl4J0it5afMjca8b11uXogTh/Kt1wMJdLfyXAQUdlQRHjgL9XBHXfwv7SHdL+MRtEGSrbOPdCQv9VrNz5P0NvVCIAFWPdaVIzNDiY2XpVf31lW0Pwb3oWGnethS0gD4B2IV7s/h79G3FGY29wg8044jvzXa+9yV0VRk1C4mk7mQGNhLliZ88ntLRIszrIC8C6n95e2KET3HaHL+bUgjZmca5yN+h2vyMZQ1AgmBetbLiQWLRAvt4lK37p46RBxJlSjENvgXXAHvZsaVnAAxyLqijFIuv4WVP2wV9GvL25A7F0z3ecUzsNzzX56IdYiCrT7C/pKGzqxwACA4+g3vjfKOT2CwNedFd0PADCqsp2E+2Fi4zVtOCU+jrHZ3YKhwq1lZAMsjXHVFeySTZNWE4inYH3LhsSiBRJozSh/x9lS0tGQv1W0TIgLzZPuOACM6DMPJj4RsWkZsP9cL+vXB3xTd73iJbYoT+PuOR5McyaWqHGX3otXQycRmKi+I7yFgGE9//e4yRgbwNoBzgUwLGw9roHrRL7fxRZ9M6MYNPpxX/kcxzkANgqdx83GuaqqgL10o9J1BdxrhLEQ0qLjM4fhUskRn46G3D7vUjvU/CbSMFKoI20URmLRQgnFbSIN6AJoCoTbEXX1ze69MyQNvVSgAHfj4ao81tywuhxo2P0BYgZPQdykx3xeFN+RjdOrcZc2ll52igXG5YCr8oTX3BBh1OEVa5GkBLuqT8F1Yr/7GIYFX3fWs86VOLbjPYJS1zBKG1LXxfOIGThR8ffNsRSR4GlsgNUkGjTk/dsjnFFX3+y2S8YFKB2tMrHxHjel0m/0Cta7KopQU3gcLhVzefSOLWkdKam9fqijsHAIDYkF4YNSwy9UzKie18pWVPHS6T7xCwAAD1fpYdSfLnSPMAZO9KTfCucSzmdLSENjYS64qhPgKo9DEIHGwlxEozmGYu87AgwAtnOPpniKe/kRruo4wNrQbuDNcHW73mNXY2Gul4hEN7l/vBpOuGMxjoIvm11ZTWIVM3y6OxXZs8wJo6phlDakbXpcBbnQo2c0UfBVk/iJUHEdsavL40pUaIy8hK9pA62apv1QYoZP99qiV9FNKTq38JvGwlwwAFzVp2TrkPjfahrf+q3LUd9Ubv5cg3qPwnxGrvu2wZbcy6/4qnHVhbplQLjcfSQWhCzSkYmWYK/81q8C3vMJ/FV6pzAHQXSs88hXkoaUcWcx9R2BuInz0bD7A7hKD0PYEMrePgmQ+NzB2BCVOcYrTuBtMw8Im26J3XBNe52LG025/UOExlJ8fqkAx6Zl4LwkU0V2NAEAcMeGAsU1vDLRxJMqFRojfyMzuZiRcHzD7g+axVIuicHruTFoZG2e8vNaMUCSoSbXW5aOyORcklonTcpdR+675viaEwAP14m9cJ3IR6OtOT526cprgZiusrZKry+7ZYBkVQQ1tjrE5Wuiu4/EgtAd77keQu9bCJQ3vxz+Xi5lweG81i0UN1gxAyciZvAU1J8u9OnBe10LHJj4RK8XTNb1xrmabGfdxzXZLRVLV0URLn7zOvj6WvdclabG3nH0G8RlLfASDKFRrMnd7OVW8cnMEhClF4tHYUIZCe4z32O9d06Uuo2ERsh3eRflmJF3NhwgjKrE55Z300niXoLOizLUxO5NcQxIKmjujgLv9Rt/+9JLXWVyvXJAORMvKmMUHEf+630/IjEVzzEK5IaT1kF75hihFH1QGikJ9ar5Nr3n5ngt9ZOk78rPJBaE7khdFuIMJWkvSenlUm68pfNmvN1ASj34QC+y9LjmBti99ElU5mjZnr2rogj1W55paswk+OldSt0qzrICSWPvPZoQ956lvXPwnK/LimER1X88mOg4H7dRzPDpzet+iUcOCWmIqT2OeleUbMyoebTiLne28+Vg2nZEQ+6bAdx0CjZ6CsUBx/5Pmp+tq9ET2xKei63kO5zf91nzOURpzkxsvPuagtDBnZjh2L8N0gQEuZEKf66y2V7JqCpm+HTAFu19P0zzqI13eY925Bbq9InpNE2YZaLbwHHoM4BzwFGY6yvcMiMlAKJ6wsCeMUpRCC91XOwZ9egBiQVhCEquKqk7S+zjBrx7RtK4iaen1ZS9JDSGauZEqMkQEx/nfjGbete870hEwN34yAgFADA2nyC53xV8hUaJYb0mK3quI7eeFy+9NtO0bAsHx6HPPPcs9r07Cr6WtcGW4l7ny1l5HraENE+Cgqt4b/N1hdEKw4KrLgGqjjVfWsZN56o64X620W2aEyOEFQNOHmhqdBmfEZWr9BDqy454ysFee9xHbIRRhnA+Nq4DADTtSy/gvudLuf+GLakH2M49vITFx6XJeLvv5NyOXqMw1jcBw7NQp2T04iwr8BpBecoDvFsgd77epEeMz70Ku1vaEtK8Oj3iuTlSIbx44jDQl8SCaEE4C3PdvStJMFkIggu0SUmHS2bpErWIxUDTEh9+fMru39mbBYOxwdb9Gvc1Th7wCZIHWsFXySaf+SlyMLbm9F+Re07qe+eqTsjaIC0rAGjc80Hzl6IlYDxZZWIkbjpP0kDTCCZm5AyvGI/gauHra93CwQn3xnvKq+GbN8BVnUCHnhmiHj4DMHxzg8o53fEEW7TMyAZN93wMXNUx7+dTss+rUbalXgV7r8EiIZB3OwLNCRhs7Uk0lv0Ez6hk9wdg2yd5NdoN+7Y1uSc5bzHy+Szct3j0zDYNljg05L6JqIxRPtlq4qQGsIGTKIKFxIIIK169IUkwWS5wpzRi0ZJKqCabRM1IxPO7rAU+Qe2G/K1wlTTN2fDq/bt91e1Suvos6+4vccCWku49P8Wr98nAlnoVYgZPAQDUnzzk09h5+9552DNGy64iLMbbNeZ2eYiXZnEU5npcVeKevdfx4kUtL9X5pAsLHQUwNkRdcYM7q80r/sLBceS/OPtjrnf6s1ccQfip70jAeWw3XKWHmn/Du+A6sbdpP3pR752xedxewqgqUF1yB/K9RclVehiuclGHgGG9RUnoMIizzqQ2NhsFNqmnO7OvSfwdR/4L2KK9ViEQz98RC4lcEkUokFgQYcXfnA61efpaUwnVTh4LlPUl/l0bxZGAWCgYj+sgmGXdo6+4wdOQebtC7J6GDoCsyEX1HeHVuDNNNvq7P+noSuzykEuldTX53mWD0yoCvkx8okeMpAF73tUsNh6hEkYZLKOYgGBLSEN9uYyrkOeaBaPJ7SdORPDYB8iWUXNQ3R3HYNongz93BsKIQZi/4yNsTQkL4pGBLSHNnZQhcz9RmaPRkHfKW5REdVaaGSUnyHpBYkGEFWkPHlCfgy+gdeawUZPHxPj25pt7/6GkOUobQn/zXQDfGJAQ93EUfOnVsCtdy9/oSriO14x/SQwkoGtN5jkIoticDcSBsSm77ADlOiMe+XncXbz3RE05d47SHBUll0/0Nbd6ibd4/o5H2GRiUWruR1oW4mV0/GVG6Q2JBRF2pD14rY2p1sZfrYspVLx685Levx4EGvnIjbjY+ER49ixRIaxqRlf+yj+Qa03pOQijNSFG1fnKa3FelNmjpc6IR35K7kovd47CHBU5l0+czeFxJ0rFW5qeHMidqXQ/0rIQztOQv1UxM8oISCyIiCeYxl+ti8lsu/REbsRlxKgqlPsM9ByEv8cmtdPF/650PalLTDxHRdzbl8ZgOo2b5nEnSkd0YmHRY5a11HbpsxSvFG1EXSOxIFoEZjT+wRBOu+SEwSgBs2r5q0VaVmrWx/IntFpdo8EgFzfyStelSXkEQahBSRgivWE3AjUiqkVozYiLCTYpxY1wNYkFQRAqIWFQj5qy0pIhZ6YL0gxxIrEgCIIwADOF2gxxIrEgCIJoARgtTqxhZyYIgiBaDCQWBEEQREBILAiCIIiAkFgQBEEQASGxIAiCIALSorOhWFZuw0LjjzUSsks7VrWN7NIG2aUdLbYF+i3D8147bRAEQRCED+SGIgiCIAJCYkEQBEEEhMSCIAiCCAiJBUEQBBEQEguCIAgiICQWBEEQREBILAiCIIiAkFgQBEEQASGxIAiCIALSopf70MqWLVvw4osvwul0YubMmbjnnnvCZsvzzz+PTz75BAAwZswYzJ8/H4899hj27NmDNm3aAADmzp2LW265xXTbZsyYgerqatjt7uqzZMkSlJSUhLXs3nnnHbz55puez6dOncLkyZNx8eLFsJVZXV0dsrOzsWbNGqSlpSEvLw9PP/00GhoacOutt2LevHkAgCNHjuDxxx/HhQsXMHjwYDzxxBOesjXDrrfffhvr168HwzC4+uqr8cQTTyA6OhrPP/883nvvPbRv3x4A8Mtf/tLQ5yq1S6m+K5WjkYht++mnn7Bq1SrP3yoqKnDNNddg7dq1ppaZXBthaB3jCZ7nef706dP82LFj+ZqaGv7ChQt8VlYW/+OPP4bFltzcXP7uu+/mGxoa+MbGRj4nJ4ffsWMHP3HiRL6ioiIsNglwHMePHDmSdzgcnu+sVHY8z/OFhYX8Lbfcwp89ezZsZbZv3z5+4sSJ/FVXXcWfPHmSv3jxIj9mzBi+pKSEdzgc/OzZs/kvv/yS53mev+222/j8/Hye53n+scce4zds2GCaXceOHeNvueUW/vz58zzHcfz8+fP5V199led5nv/Nb37D79271zBb/NnF87zss/NXjmbaJnDmzBn+pptu4o8fP87zvHllJtdGbNmyxdA6Rm6oJvLy8jB06FB07NgRcXFxGD9+PLZv3x4WW5KSkrBgwQJER0cjKioKvXv3RllZGcrKyvDnP/8ZWVlZeO6558BxnOm2HTt2DAAwe/ZsTJo0CW+++aalyg4AFi9ejHnz5qFNmzZhK7NNmzZh0aJFSE5OBgAcOHAAPXr0QLdu3WC325GVlYXt27ejtLQUly5dwoABAwAAd9xxh6FlJ7UrOjoaixYtQnx8PBiGQd++fVFWVgYAOHToENauXYusrCwsWbIEDQ0Nptl18eJF2WenVI5GIrVNzPLly5GdnY2ePXsCMK/M5NqI4uJiQ+sYiUUTZ86cQVJSkudzcnIyKioqwmJLnz59PA+2uLgYn3zyCUaNGoWhQ4di2bJl2LRpE3bv3o13333XdNvOnTuHYcOG4YUXXsBrr72GjRs3oqyszDJll5eXh0uXLuHWW29FVVVV2Mps6dKlGDx4sOezUv2Sfp+UlGRo2UntSk1NxYgRIwAA1dXV2LBhA2666SZcuHABV1xxBR555BG8//77OHfuHP75z3+aZpfSswvHeyq1TaC4uBjfffcdcnJyAMDUMpNrIxiGMbSOkVg0wXEcGKZ5iV6e570+h4Mff/wRs2fPxvz589GrVy+88MILSE5ORps2bTBjxgx89dVXpts0cOBALF++HO3atUNCQgKmtxUupwAAB31JREFUTp2K5557zjJlt3HjRtx3330AgG7dulmizADl+mWVeldRUYGZM2fizjvvxJAhQ9C2bVu89NJL6N27N+x2O2bPnm1q2Sk9O6uUFwC8/fbbmD59OqKjowEgLGUmbiO6detmaB0jsWiiS5cuqKys9HyurKyUHXaaxZ49ezBr1iw89NBDuP3223H06FF8+umnnr/zPG9oEFSJ3bt3Y9euXV52pKamWqLsGhsb8f333+PGG28EAMuUGaBcv6TfV1VVmV52P/30E7Kzs3H77bfjgQceAACUlZV5jcLMLjulZ2el9/Tzzz/HhAkTPJ/NLjNpG2F0HSOxaGL48OHYtWsXqqurcfHiRezYsQOjR48Oiy3l5eV44IEHsHLlStx2220A3BVv2bJlqK2thcPhwNtvvx2WTKjz589j+fLlaGhoQF1dHd5//32sWLHCEmV39OhR9OzZE3FxcQCsU2YAcM011+D48eM4ceIEXC4Xtm7ditGjRyM1NRUxMTHYs2cPAODDDz80tezq6upw//334w9/+ANmz57t+T42NhYrVqzAyZMnwfM8NmzYYGrZKT07pXI0m+rqaly6dAndunXzfGdmmcm1EUbXMUqdbSIlJQXz5s1DTk4OHA4Hpk6div79+4fFlpdffhkNDQ145plnPN9lZ2fj17/+NaZNmwan04lx48Zh4sSJpts2duxY7N+/H1OmTAHHcZg+fToGDRpkibI7efIkunTp4vmcmZlpiTIDgJiYGDzzzDN48MEH0dDQgDFjxuAXv/gFAGDlypVYuHAh6urqcNVVV3l84Gbw7rvvoqqqCq+++ipeffVVAMCNN96IP/zhD1iyZAn+3//7f3A4HLj22ms97j0z8PfslMrRTE6dOuVV1wAgISHBtDJTaiOMrGO0Ux5BEAQREHJDEQRBEAEhsSAIgiACQmJBEARBBITEgiAIgggIiQVBEAQREEqdJVocTz31FL7//nsA7glnqampiI2NBeCedSv8OxCff/45du3ahYULFyr+pqKiAn/4wx+wcePG0A0HsGDBAuTm5iIhIcHr+zvvvFP3lNoFCxagT58+uP/++3U9L9EyodRZokVz44034h//+Af69esXblNUYWYDTmJBaIFGFkSr4+qrr8ZNN92EgoICrFy5EkePHsXbb78Nh8OB2tpa/OpXv8L06dOxefNmfPrpp1i7di1mzJiBAQMGYO/evSgvL8ewYcPw5JNPoqysDFlZWcjPz8fq1atRWlqKyspKlJaWIiUlBStWrEBycjIOHDiAxYsXw+FwoHv37igrK8OCBQswZMgQTbbfeOONuO2225Cbm4vz58/jvvvuw/Tp0wHAsy8Fy7Lo3Lkz/vKXv+Dyyy/HhQsX8NRTT2Hv3r2w2Wy4+eabPfsc5OfnIzs7G1VVVejTpw/+9re/eWbAE4QYEgui1eFwODB27Fj84x//8DSk//rXv9CpUyfs27fPqwEWU1JSgvXr16O+vh633norvvvuO6SlpXn9Zvfu3fjggw8QHx+P3/72t9i4cSN+97vf4cEHH8SSJUswZswYfPvtt5g1a5aifa+99ho++ugjr++WL1+OjIwMAEBtbS3ee+89VFRUYMqUKRg0aBCqq6uxbt06vP3220hISMDmzZvxwAMP4OOPP8Zzzz2HhoYGbNu2DS6XC7Nnz8Z3330HwO1Ge+ONNxAdHY277roLO3bswJQpU0IsYaIlQmJBtEqEJafbtm2LNWvW4KuvvkJxcTEKCgpQX18ve8zYsWPBsizi4+PRo0cP1NbW+ojF9ddfj/j4eADAlVdeidraWhQWFgJw72YGAEOHDkWfPn0UbZs1a5Zf19D06dPBMAy6dOmCUaNGITc3F1VVVZgwYYIn1nHHHXdg6dKlOHXqFPLy8vDYY4/BZrPBZrN5dhR8//33cfPNN3t2ouvTpw+qq6sDlh3ROqFsKKJVIrhaTp8+jSlTpqC0tBSDBg3CH//4R8VjxIFxhmEgF+6T+43NZvP5rc1mC9p28UqmHMeBZVnZTZ14nofT6YTdbvdakrq8vBw1NTU+51K6J4IASCyIVs6hQ4eQkJCA3/3udxg5ciT++9//AgBcLpdu1+jduzeio6Px9ddfA3DvmldYWBj0PgwffPABAPeS2Lm5uRg9ejRGjRqFbdu2eUYG7733Hjp27IgePXpg2LBheP/998FxHBobG/H73//eky1GEGohNxTRqhkxYgTeffdd/OIXvwDDMLj++uuRkJCAEydO6HYNu92O1atXY9GiRVi1ahV69uyJzp07K6bwysUsrrnmGixZsgSAe8XTO+64A5cuXcLChQvRq1cv9OrVC7NmzcLMmTPBcRwSEhKwdu1asCyLuXPnYunSpZg8eTJcLhcmTJiAcePG4YsvvtDtHomWD6XOEoQJ/PWvf8X999+Pzp07o7y8HJMnT8Znn32G9u3bazpPpKUCEy0HGlkQhAmkpqZi1qxZsNvt4HkeTz31lGahIIhwQiMLgiAIIiAU4CYIgiACQmJBEARBBITEgiAIgggIiQVBEAQREBILgiAIIiAkFgRBEERA/j8F+l/jUlxTxgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "for name, model_name in (('BatchNorm', 'batchnorm_net'), ('Normal', 'net')):\n", " model = eval(model_name)\n", " model.load_state_dict(torch.load(f'{model_name}.pth'))\n", " optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)\n", " model.train()\n", " losses = []\n", " loader = torch.utils.data.DataLoader(xy_train, batch_size=200, shuffle=True)\n", " for epoch in range(200):\n", " for x_batch, y_batch in loader:\n", " y_pred = model(x_batch)\n", " optimizer.zero_grad()\n", " loss = loss_fn(y_pred, y_batch)\n", " loss.backward()\n", " optimizer.step()\n", " losses.append(loss.data)\n", " plt.plot(losses, '.', label=name)\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can at least verify that BatchNorm does not do too much harm with a larger batch size (1000 instead of 200):" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAELCAYAAAAoUKpTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXyU1fX48c8zM9mDCdkhgSAEggoIisgiIG64EEHFNmBZivTb/hTbogWX0mJB1CKlC1qhtW5oRVRcUItxqbIEFSRsVgJBIJCEEEgMCdlmnuf5/TGZYWaYySQhsyQ579fL14vZMidP4pzce+49V9F1XUcIIYRogiHQAQghhAh+kiyEEEJ4JclCCCGEV5IshBBCeCXJQgghhFeSLIQQQnglyUIIIYRXpkAH0JTy8nKWLFlCZGQkY8eO5brrrgt0SEII0SkF9chi9erVzJgxg8WLF7N27dpAhyOEEJ1WUI8sTp48SUpKSqtfX1FxBk1r+Qb1+PhoTp2qbvX7+orE1XLBGpvE1TISV8u1NDaDQaFr1yiPjwd1skhJSaGsrIykpKRWvV7T9FYlC9trg5HE1XLBGpvE1TISV8u1ZWxBnSzuvPNOli5dSkhICNnZ2YEORwghOq2AJIvq6mqys7NZuXIlaWlpAKxfv55nn30Wi8XCjBkzuOuuu0hKSmLZsmWBCFEIIYQDxd9dZ3ft2sWCBQs4dOgQGzZsIC0tjdLSUqZMmcK6desIDQ0lOzub5cuXk5GR4c/QhBB+pmkaR48e5cyZM0j/a/9QFIiKiqJHjx4YDM1f4+T3kcXatWtZuHAh8+fPt9+Xm5vL8OHDiY2NBWD8+PFs2LCBOXPmnNd7nTpV3ao5u8TELpSVVZ3Xe/uCxNVywRqbxGVVVfUDFotKQkIqiuL5g8tkMmCxaH6Lq7mCNS7wHJuua/zww0kOHy6iS5dY+/0Gg0J8fLTHr+f3pbNLlixh6NChTvedOHGCxMRE++2kpCRKS0v9HZoQws9qa6vp0iW2yUQh2paiGOjSpSu1tS1bxRUUBW5N01AUxX5b13Wn2/50ZO9O9h39jogeF5E+YHBAYhCis9A0FaMxKD6GOhWj0YSmqS16TVCkc9sSWZvzWS57Po7s3Un0lhWkFH5E9JYVHNm70+8xCNHZBOoPw86sNdc8KJLFyJEj2bp1K+Xl5dTW1pKTk8OYMWP8HkflwT0YUTEqYESj8uAev8cghBDBKCiSRXJyMnPnzmX69OlMmjSJCRMmMGjQIL/HEdNnICpGVF1BxUBMn4F+j0EIETg7dmzn+utHM3PmVGbMmMJdd01m7drXmnzNe++9zccfb/D4eElJMZMnZ7l9bPLkLP7xj7873bdkyaN8+OH6lgfvYwGbLPzss8+cbmdlZZGV5f6C+kv6gMEc4T5qpWYhRFArKKokv7CCzJ5dyUiNadOvnZl5EU8//Q8AamrO8JOf/IgrrriSCy/s7fb5e/bsYsiQy1v9fmvX/psxY8bRv/9Frf4a/iCVJRfpAwaTOG50UC5rFEJYE8VTr+VhUTVMRgPzpgxp84RhU19fj8FgIDo6ms8++4Q1a16hvr4es7mBhx/+PWZzA5s3b+Sbb7YRH59AenovHn/8D1RUlBMeHs6DD/6OqKgo6uvrWbjwYb7//iBdulzAE08sIybGumx12rSf8vjjj/Kvf71CSEiI0/tv2bKJf/7zWXRdo3v3VObNe4S4uHgmT87i4osHcOBAPr/73SKWLn2c9PR0Dh36nn79+jNgwCA2bHif06dP8/jjy+jV68LzvhZBMQ0VrAqKKvlg62EKiioDHYoQolF+YQUWVUPXQVU18gsr2vbr53/XOA2VzeTJWQwZcjlxcfG8++5bLF36F1566TWmTp3O6tUvMmzYlVx11Rhmz/4FV145gj/96UnGjr2G1avXMmvW//HSS/8C4IcfKvjxj+9i9eq1xMXF8cknOfb3u+GGm0hNTeOFF/7pFEdFRTlPPfU4TzyxjJdeWsPAgZeyfPlS++PDh4/ktdfW0bVrHAcPHuCuu2bw4ouvsWfPLo4fL+G5517iuuvG895769rkusjIwgN//vUihGi+zJ5dMRkNqKqG0Wggs2fXtv36DtNQZ85U88ADv+Tf/36Zxx9/ii1bNlFYeIS8vG/c7n7euXMHjz66BIARI65ixIirKCkpJiEhkYsvHgDAhRf2obLyB6fX/eY3DzNz5lTGjBlnv+9///uWiy66hG7dugNw6623s3r1i/bHbV8PIC4unn79+gOQmJjE5ZdfAUBKSjfy8orP95IAMrLwyNd/vQghWicjNYZ5U4Zw25jePv8jLioqmmuuuZ7t27fxs5/NoLi4iEsvHcLkyT/GXackxz0juq5z6ND3jfcbnZ7n+tr4+ATuu28ujz/+KGazufE52jmvUdWzeyPCwsLs/3advnJ9v7YgycID218vBgWf/PUihGi9jNQYbhnRy+ejfVVVycv7hvDwMBRFYfr0WVx22VC++OK/aJr1w9xoNNo/xAcPHmKfYtq+/SuWLl3S7PeyTUd98YV18c/FFw/gf//bQ0mJdWTw3nvruOyy1hfSz5dMQ3lg++vFVysuhBDByVazUBSwWCxkZPRj/vzf8uSTi5k6dTIGg8KwYSPYvdu6aXfo0GGsWvV3oqOjmTt3Pn/842O8/fabjQXuBS1679/85mGmTfsxYJ1amjfvtzzyyG8wmy2kpKTw0EO/b/Pvt7n83nXWn6SRoH8Ea1wQvLFJXFbHjx8hJSXd6/OCtWFfsMYF3mNzvfZB10hQCCFE+yPJQgghhFeSLIQQQnglyUIIIYRXkizcqDuWT33e+6ilBYEORQghgoIsnXWhlhZQ8sFSdNVMgyGEyAnzMSZnoJYWYCneh6l7f4zJcja4EKJzkZGFC0vxPnTVAroOmgVL8T7U0gJq3l9Kw/a3qHl/qYw4hOigSkqKueqqoWzb9qXT/ZMnZ9k3x/naVVcN9f6kAJBk4cLUvT+K0QSKAQwmTN37YyneB5rZKYEIIQJHLS3w2VSxyWTij39cQk3NmTb/2u2ZTEO5MCZn0O2uRzn5vx1OU04NhhDQLPYEIoQIDNtIH815qritJCQkcsUVV7JixV948MHfOj328svPk5PzHwwGA1dcMZxf/vLXlJSU8MAD9xETE0tYWBg33HATubmbOX26kpMny5g48XaOHz/Ojh3buOCCGJYt+xthYWGsWvUM33yzjdOnT5OQkMCiRU8QFxffZt9HW5ORhRvhaZmEDZkAQH3e+wBETphPVb+byUufxiFLYiDDE6JT88dIf86cX/P111udpqO+/DKXzZs38txzq3n++VcpKjrK22+/CUBh4RF+//vF/OUv1lPvvvvuW554YhnLlz/NihV/Zvjwkbz00hoAvv56K8eOHaWw8DArVz7PmjXrSE5O4aOP/tPm30dbkpGFG/sOl/Pt1i8ZfPhlFN1CgyGE8ivv4fFtCVhUM6adedKyXIgAMXXv7/ORflRUNA8+uIA//nEJL79s/ZD/5puvue668YSHhwNwyy23smHDB1x55Si6do2ztxIHGDjwUqKioomKsrbPcGwZXlVVRVpaD+bMmcv69e9QWHiEb7/dQ2pqWpt/H21JRhYuCooq+e3KLRR9uwNNPfvXS+XBPdKyXIggYEzOIHLCfEKH3t7mU1COhg0bbp+OAs7pM2f9LLB2m3VsFw7ntgw3mZz/Lt+37zvmzp2DrmuMG3ctY8Zc7bbleTCRZOEiv7ACi0XjgDkFFSMaChhMxPQZKC3LhQgSxuQMwoZM8Pkydtt01KlTJ7n88qF88slH1NfXYbFY+PDD97j88tatXNq58xuGDLmcSZMm06NHT3JzN9tbngcrmYZykdmzKyaTgUJLIqtqxjPzMgPJ/QfTJTmDeV19d0i8ECL42Kaj7r9/DiNHjqaqqoq7756OqloYNmw4d96ZTUnJ8RZ/3WuvvYFHHpnH9OnWduSZmRf5bWlua0mLcnevO2Pmy91FQZcUpK11ywVrbBKXlbQo9522blEuIws3+veKIz4qxPsTQXZ2CyE6BUkWzVRQdHYK6kJTGZbifSjh0dTn/ttn672FECJYSLJohoKiSp56LQ+LqtEn9CRzunyMolsAA+gacHa9tyQLIVpG13UURQl0GJ1Ka6oPshqqGfILK+zLZi80lFjXd+u6NVEoBqfWIEKI5jMYjKiqJdBhdDqqasFgMLboNTKyaIbMnl0xGQ2oqsYhrRsY9oKugsFE2Mip6HXVUrMQohUiIqKpqvqB2Nh4FEX+dvUHXdeoqqogIsJzMdsdSRZe2GoVU67ry5laM5k9LyfKdJkUtYVoA9HRMVRUlFFaegzwPDViMBiCch9CsMYFTcWmEBoaTnR0y1Z6SrJogmOtwmQ0OLT4iLGfcVGf974kDSFaSVEU4uKSvD5Plhq3XFvHJsmiCY61CluLD9u+C193vhRCiGAik4RNsNUq3LX4aKrzpS977QshRCDIyKIJGakxzJsyxG2LD0+dL2XEIYToiCRZeJGRGuM09eRY2I6cMJ/SfTspMCejlURw5vvDDGnYSReXEYckCyFEeyfJwgtbgnC3W/uQJZGntiVgtpiBfC40lWEKO8iICCMKmuy9EEJ0GEGfLP73v/+xdOlSXnzxRb+/t+OUkrvd2vl1JiyqdWlaL1MZ93TJwYiKphkJv2gsIf1GyahCCNEhBHWyOHr0KJ9//jlGY8t2GrYVpyI2jbu10e0jhkyLtQBusWhkmI5jRMWogI6OEh0viUII0WEEVbJ47rnn2Lx5s/32888/zz333MPPf/7zgMTjWsR23a2dAfYCeJIlAvbtQdc1FKPz9JN0phVCtHdBlSxmz57N7NmzAx2Gna2I3dQHva0ArpZaaGA0ChDSbxQA9XnvS2daIUSHEFTJIhgZkzPO+XB3bFduTRQOtQ1DCIaEdHuCkM60QoiOwC+b8qqrq5kwYQLHjh2z37d+/XpuvvlmbrjhBl599dUmX79q1Spfh9hsthYg6zZ+z1Ov5VFQVHnuBr3vt5+9LZ1phRAdgM9HFrt27WLBggUcPnzYfl9paSl//vOfWbduHaGhoWRnZ3PllVeSkdG2f3E3dUSgN4mJXdze//nuElSHFiDHTtUw5OLLKMl7D121oBhNdB10FadK99tvx18/C7W2ioj0SwhPy2x1TE3FFWjBGhcEb2wSV8tIXC3XlrH5PFmsXbuWhQsXMn/+fPt9ubm5DB8+nNjYWADGjx/Phg0bmDNnTpu+d2vP4G6qAVdafCRGowFUDaPRQFp8JFVhMUTccra2UZ+cQcQtCU63AaqAqvNo7BWsTcuCNS4I3tgkrpaRuFqupbEF/AzuJUuWnHPfiRMnSExMtN9OSkpi9+7dvg6lTXhqAeJa23BX6xBCiPYqIAVuTdOcjlFsb8cqOrYAEUKIziAgXWdTUlIoKyuz3y4rKyMpyXtPeyGEEIERkGQxcuRItm7dSnl5ObW1teTk5DBmzJhAhOJz0q5cCNERBGQaKjk5mblz5zJ9+nTMZjOTJ09m0KBBgQjFp6RduRCio/Bbsvjss8+cbmdlZZGVleWvtw8IdwckOSYLaQMihGgvZAf3eXDdye3K1L0/9YoJXbegKEaU8GhqN72EAk67vGXUIYQIdpIsWsm2k9uiapiMBuZNGeKUMAqKKsndY+Ho6evobTxOLeFM3rwaRVetT1AMjd1spQ2IECL4SbJopfzCCiwOO7nzCyvs90dFhPDaJwcwWzQgkYMkcn3EHrAlCmhsA2LEseW5EEIEK0kWrZTZ03qWhdq4kzsqIsQ+0oDGQYODQ1o3UHafTRgGE2GjfuLU8lwIIYKVJItWct3J7TjScGQ0wKA+CcREdacirR9JFbvsbcwlQQgh2gtJFufBdSe37dQ8W75QsCaKvYfKsagaW/YamDdlkuz+FkK0OwHZlNcR2UYaYwd3x2RUMChgMhmIiQp1W9sQQoj2REYWbcg20hg5sJt9egpgy97jqKqGYlA4VVlHQVGljC6EEO2KjCx8ICM1hltG9LInj3lThjDm0u4owBe7iu2HJtlISxAhRLCTZOEHGakxxMeEo2r6OdNRtpYgDdvfoub9pZIwhBBBSZKFn9iW2hoUMBoN9ikqS/E+dNXaEkRXrZvzhBAi2EjNwk9cl9oCfLD1MEmWeHrrBoxoqChUhKSRHthQhRDiHJIs/MhWw7C1CjFbNBQF0o03kGE6zkFLCpdVxUqyEEIEHZmGCoD8worGViDWnd6HLYl8Wj+QYyQzoMsPUuwWQgQdGVkEQGbPrhgMCpp2drv31Wl1XB9fRNSXX9GgWWhQjIRdNY3Qi64OXKBCCNFIkkUAZKTG8PMR4RzcsY0D5mSMRoWJtR+jFJrRse781nWV+s2rMcalSVsQIUTASbIIALW0gP77XyAzwowWaaS+xzAotFgf1LEmDKUxYWx/h7ChkyRhCCECSpKFnxUUVVKzfSu9VDMKOkY0dBTMjSuibCMLRddRFFCLvqXm+H7CRk6lYr8ZNeZCSRxCCL+TZOFHtlVQaYRyTxcDIYqOYjCx39SfL6ouoI/pOAWWFABujthFv5ASQAfVTP3m1dQ3nn0hp+oJIfxNkoUf2dqYH9IT+XvVDUzs28Alw0fQ3ZLIsZ0qR+oTUQwKowd2IzEtE+Wrv4NmARTrYUmNp+o17N+CQc7uFkL4kSQLP3I8MOkYyUQOHYIxOYYMOGfD3t7CCgZceQ8p5mMo4dGN53VbQDFgyd8Euipndwsh/EaShR+57uJ27DzrumHPomq8ZzQwb8poMlJjMMalEVZ5iOrSYsz7Prdu0JCzu4UQftKiTXkNDQ0UFxf7KpZOwbEjrTuezvY2JmfQddTtGBLSsf7YFDm7WwjhN16Txccff8zixYuprq7mxhtvZOLEibz00kv+iK1T8tRwEKDuWL51OkpXQTEQNnKqjCqEEH7hNVmsWrWKH/3oR+Tk5DB48GD++9//8u677/ojtk7JNlV125jeTLmuL/mFFfazL2qPfAuaufGZOnpddeACFUJ0Kl5rFrquk5mZyT//+U/GjBlDdHQ0uq57e5k4D7YpKlvtwmQ0MG/KEIakX0KFIcRa6JYpKCGEH3kdWRgMBj788EM2b97MqFGj+OKLL1AUxR+xdWruahfhaZlETphP6NDbZRWUEMKvvI4sHnzwQZ5++mnuv/9+EhMTefbZZ1mwYIE/YuvUHJfZOtYujMkZkiSEEH6n6C2YU2poaODkyZN0797dlzG1mVOnqp06uzZXYmIXysqqfBBRyxQUVTrtvTh2qoa0+EinlVRqaQGWAG/QC5br5U6wxiZxtYzE1XItjc1gUIiPj/b4uNeRxccff8yXX37J3LlzufXWW6mqqmLOnDnMmDGj2UGI1nHde6GqGopBYVDveGKiQhmbVkfcV38HzUyDIYSwkVPR66plZ7cQos3Jaqh2wFa/0HRQVZ28Ayf5fGcxWz/baD+/29Y/qmH7W9S8v1QOTxJCtCmvycK2Gio3N1dWQwXIgC4/cEPEXi40lTndv78hGU0xgmKw/qdrTju7hRCirTR7NdSmTZtkNVQAqKUFxH31d24Kz+O+mI/pHXI2YRzTk6gcMYfQobcTdtU0MIZYk4YsqxVCtLFmr4Z64IEHZDVUAFiK96E3nn1hQGPWUCOf1FoXGIwc2A0z8ElhLJlJXblwQlrAi91CiI7Ja7IYOnQoL774IkVFRRw5coQ1a9b4Iy62bdvGG2+8ga7rDBs2jDvvvNMv7xtsjoekEd14MJKqK5jjM5g+wDpqcGw62Cf0JDMvM5Dcf7AkCiFEm/OaLA4fPsy9997LiRMn0DSNrl27smrVKvr06ePTwE6fPs2iRYsIDQ3lnnvu6bTJYm9VLDuqbqCP6TgHLSlcVhVLeuNjtsJ3urGMn0fmEJKvUVOQIxv2hBBtzmuyWLx4MbNnz+a2224D4K233uIPf/gDL7/8cpsG8txzz7F582b77eeffx5d11m2bBnTp09v0/dqTzJ7duW9LckcqU/EZDIwxaGxYFRECAZFoW/IcYyoKICuni1uy5SUEKKteE0Wp06dsicKgDvuuIMXX3yxzQOZPXs2s2fPtt8+ffo0TzzxBFOnTmXgwIFt/n7theMZGMMHpRIfFQJYp6By/7uFcaElnNHCUDGCrqGiUFVn4IL3l9r3X8hIQwhxvrwmC1VV+eGHH4iNjQWgvLzc50EBPPbYYxw/fpyXXnqJbt268cADD/jlfYORbXOe447M4u928/PIjzCiomJkXc0VRCn1HLSkMPHEKS7QzHJAkhCizXht9/H666/z/PPPc9NNN6EoCh9++CEzZsxg6tSpzXqD6upqsrOzWblyJWlpaQCsX7+eZ599FovFwowZM7jrrrvO/zvpZPa//wrKzrcxKqDqCh/VDyGnZgAGg8Kvr72A9F2r0FULGAxE9rkMY3QsXQZeTXhaZqBDF0K0Q83qDbV161Y2b96MpmmMHj2akSNHNuuL79q1iwULFnDo0CE2bNhAWloapaWlTJkyhXXr1hEaGkp2djbLly8nI6Pt//Jt772hXDnGpZYWcGb9H9E1C4rBRH7mLP6xtQ5N0zGZDDxyYyxJFbuw7NtoPSwJwGAiMuuhNh9lBOv1guCNTeJqGYmr5dq6N1SzjlUdMWIE8+bN48EHH2TkyJH86U9/atabr127loULF5KUlGS/Lzc3l+HDhxMbG0tkZCTjx49nw4YNzfp64ixjcgZRWQ8SfsUdRGU9yAlTNzRdR8fa0vyLY+EcKDeg2xIFyM5uIUSrtegMbptXX321Wc9bsmQJQ4cOdbrvxIkTJCYm2m8nJSVRWlramjA6PWNyBmFDJmBMznA6jlUxKGzeU8K7B0Kx6AZ0QAfZ2S2EaDWvBW53zqc3lKZpTu1CdF2X9iFtwHHVlH7iIFWH93LAnMLTVeO5IvQgigLxg8dxhRS6hRCt0KpkcT4f7ikpKWzfvt1+u6yszGmaSrReRmoMF5rKOPPtGrRwMzeEG3mm6gbeqBkOgHFrHV17VzqdhyGEEM3hMVnk5OS4vV/XdTRNa/Ubjhw5khUrVlBeXk5ERAQ5OTksXry41V9POLMU70PRLRgVUNDoG3KcwxbrtJ+m6+QXVkiyEEK0mMdksXr1ao8vOp9NcsnJycydO5fp06djNpuZPHkygwYNavXXE85M3ftTr5jQdesqqYzLhvHZ1jo0XcdkNBAVEcIHWw+T2bOrJA0hRLO1Klm01GeffeZ0Oysri6ysrDb7+uKsQ5ZE1lZdz4WGEg5p3fhR74t5sLe1j1RURAivfXLA3nhw1kWVxEaFEtJvlGzaE0I0qVU1CxG88gsrONiQQIGegEGx3r5lRC8yUmN4ecM+zBaNXqYyfhH5EaZCDTNgzt/kk/0XQoiOo1VLZ0XwclxCazQayGxsPFhQVMmmPSX0MpVxY/hOjGjYlynI/gshhBcysuhgHJfQOtYl8gsr6Kmc4J4uOWc71NpWQBtl/4UQomkeRxbFxcUeX7Rx40afBCPaRkZqjH3qySazZ1f6hZZiRMWogAYcscSTW9+Pnb1mYEzOoKCokg+2HqagqDJgsQshgpPHZHHvvffa/33fffc5PfbnP//ZdxEJn8hIjWH4NWPAYEJHQcXIO7XDeLthJN0vGmQ/dW/dxu956rU8SRhCCCcep6Ecd2kfPXrU42Oi/UgfMBg18SEsxfuoCEkj/Vj4Oafu6TpYLBq5e0pkaa0Qws7jyMJxl7brjm1pz9E+qaUF9tPzzF0vZMve43yxq5inXsuznrpnsP5cdWDznhIZXQgh7Jo1shDtn1paQI3D6XnF6dOcRhLf5J9gUO948g6ctD5f1Xl38/dMvKq3jDCEEJ6ThaZpVFZWous6qqra/w3W0/NE+2Ip3gcOp+dlhJRiMiZgsWjowP8OVWAwKpiMCqqq2+/bfzSPKdf15UytWXZ9C9GJeUwW+/fvZ/jw4fYEceWVV9ofk2mo9sfUvT8NhhDQLGAwkdx/MPMyEnl38/f871CFtY25pjOxv07EDwc5VK4RqdRTYEnhlY+sCcVkNDBvyhBJGEJ0Qh6Txb59skmrIzEmZxA5Yb69ZmFMziADmHhVb/YfzUNVNXqHnmRM2cegmbki0lq7sGDtXHvYkoiqatKIUIhOqslNebYpKJPJRHV1Nbm5uWRmZpKent7Uy0SQMiZnnNPSw3ET39DKApRCMwC6Yl39YNQ1MkzWzrWOO8KFEJ2Lx9VQBQUFXHvttWzatIm6ujruvPNO/vznP/OTn/yELVu2+DNG4WMXmsq4xvIFkUe/crpf00HFwPdaCkP6JjBqQIrT47ZNfPsOl/sxWiFEIHgcWSxdupRf//rXjBs3jrfeeguADz74gNLSUubOncuoUaP8FqTwHfsqKbXB4V4FU/oQfgjvToE5masSetu71W7Ze5wp1/Wl8HgVm/eUoGo663MP85tsqWUI0ZF5TBYlJSXceuutAHz11Vdce+21GAwGunXrRnV1td8CFL5lXyXlyBhC2OCb6Z6cQXdg4yebuDpkFwdI4bAlkdUf5eO4stpikVqGEB2dx2kog+HsQ3l5eVxxxRX22/X19b6NSviNqXt/MISAYgCDiZCLxhE5Yb69tqGWFjD48MvcFJ7HfV02MCJsP65bcBRFISoiJADRCyH8xePIIiYmhn379lFdXU1ZWZk9WezYsYPk5GS/BSh8y90qKUeW4n0omhmjAjo6d0Z+RYnalcOWRBRAUazHtb768X4Kj1cxcmA3GWEI0QF5TBb3338/M2fOpLq6mt/85jdERkbyr3/9i5UrV/LMM8/4M0bhY+5WSYF1VKFVn7KOOnTr+RcKOn1NxynUkri0Tzw7C06i69Yd35/vLGbj7hLGX9GDyHCTbOITogPxmCwGDx7Mxo0bqaur44ILLgBgyJAhvPHGG/Tq1ctf8YkAcWwPAobGhKGjGE2kDriMhy6ynpu+91C5fRc4gKbp/OerQhTAZJJNfEJ0FB6TxQ8//HDOv3v37m2/HRsb6+PQRCA5tQdRdEz9x2KIjsfUvT9jHEYh86YMIa/gJB99eQTNoZahg2ziE6ID8Zgshg8fbm/r4dpUUFEUvvvuO99GJgLKtT1IaL9RbpdF99MAABz8SURBVKeqLjSV0b/bIXqNjOYfW+tQHTOGolBTZ+HlDdZuAFLPEKL98pgsJk2aRF5eHtdccw133HEHGRnnflCIjstb4Rug4bvPqd+8mhpdo78xhAU338Peqlhq6izkbDuK2jglZbNpdzGjB3WXpCFEO+QxWTz55JPU1taSk5PDkiVLqKmp4dZbbyUrK8tewxAdm7vCt+1MDCU8mvrNq0Fv7ECsmkkxHyN9xGA+2HoYzU2Le1WDz3cWs2XvcallCNHONNkbKiIigokTJzJx4kSOHz/Ou+++y/Tp0+nVqxd/+ctf/BWjCBLnFL11h1b1isG6ZwPred8mo8Gp8O30daSWIUS702SycFReXk55eTkVFRXEx8f7MiYRpJyK3migGEHXwGAgbNQ0+yjEsTlhVEQIhcerqDzTwJ7vT6FpOopB4fvi07y8YR89U7rIWRlCtANNJouSkhLee+893n33XYxGI7feeitr166VTXmdlGvRO2zkVPS6ahIuvoyqsO5Oz81IjTnnw7+gqJLcPSVs2l1sP5HPxmBQ+MkN/bh6cKrPvw8hRMt5TBbTpk3j0KFD3HzzzSxbtoyLL77Yn3GJIOSp6B2e2IWqsiqvr89IjSG/sAJVO/cxTdN55aN89hw8RUxUqBTBhQgyHpPFtm3bCAsL44033uDNN9+036/rOoqisGPHDr8EKIKLLUFYivc53W6uzJ5dMTYe3epK07GPOGTllBDBxWOy+PTTT/0Zh2gnHIvcDYYQIifMh8Qhbp/nbtltRmoMD069jNw9JQBEhJnsy2ydXt+4cmrj7hKZnhIiCHhMFqmp8j+nOJdTkVuzWG8PGOKUHIBzEoprwnAcLQzpl2ivZbhOUdmmp6RJoRCB1ezVUELAuUVuU/f+1B3Ld0oOxh4Dzh6m1JhQmpqusiWPkQO7kbunhMozDewqOGlvH6LpSJNCIQJMkoVoEXdF7tr9H58dbahm1MM7z77AYf+FN44jjs93FvFqzn6n6SnNYUe4rJ4Swr8kWYgWc93ZHZF+CRW20QaKde8FAAqmzNEtLoIDXD04lbTEaHL3lLBxVzEuJQ1ZPSWEn0myEOctPC2TyAnzadi/Bb2mEvXo7sbNetYGhK1lG2n0TOlyzigDZPWUEP4UtMniwIEDrFixgsjISLKyshg1qvUfOsI/LPu3WKejFCMh/ccS4qFTbUvZRhn5hRVOTQodOa6estU1oiJCwGAgLT5SEogQ5ylok0VNTQ2PPPIIRqOR5cuXS7IIcq6tQJTo+DZJFDaO9Qxvq6ccO92Ctb4hhXEhzk/QJIvnnnuOzZs3228///zzFBYW8tBDDzF9+vQARiaaw90qKbDut2jYvwUF2myk4W31lCvXwrgkDiFaTtFdTzYKEnv37qVXr15ER0cza9Ysnn/++RZ/jVOnqtE8fYI0ITGxC2XNaF/hb8Eel+tGPLW0gJr1TzYWvmmcnhrTZknDkbvVU97YEkdtvTU+f9Y7gv1nGWwkrpZraWwGg0J8fLTHx4NmZOGqvr6e3/72t0RHRzN27NhAhyOawXWVlHVqynL2CbqK+bv/Ys7fREjm6DZNGo51jaiIEM7Umj3WN2xcp6w27S5mUJ8EWV0lhBs+H1lUV1eTnZ3NypUrSUtLA2D9+vU8++yzWCwWZsyYwV133eXLEESA1B3Lp/iV34Nqcfu4Ygol/vpZqLVVRKRfQnhaZpvHsO9wOXsOnuRMrZl3vjjY7JGH0ahwxUXJdO0STu/UGKpqGhjYJ4H+veLaPEYh2gOfJotdu3axYMECDh06xIYNG0hLS6O0tJQpU6awbt06QkNDyc7OZvny5T45tlWmofyjqbhsNQv7klpNBfuRSAooBkADN21B2jq2gqLKJldUedOW01bt8WcZSBJXy7Wraai1a9eycOFC5s+fb78vNzeX4cOHExsbC8D48ePZsGEDc+bM8WUoIkCMyRlENCYAW+Kw5G9q3Lhn28CnN6styPlyXVHlejjT7oMn3bZPt3E3bTV6UHd6pnSh8Lj1f0o5zEl0VD5NFkuWLDnnvhMnTpCYmGi/nZSUxO7du30ZhggStsSh9ht19hzv3H+DagYUlHDPf9W0taYOZ2pO4oCzezvcCWTxXAhf8HuBW9M0FEWx37adjyE6D9dCeP3m1aBr1Of+G2Ncmk9HF01xTCC2xAGe26g3paniec+ULrJZULQ7fk8WKSkpbN++3X67rKyMpKQkf4chgoReVw00TkWpZuq3v0PY0EkAbs/D8Bd3bdTdnSmuqjrNSSGqhtujZB1HHzKFJYKZ35PFyJEjWbFiBeXl5URERJCTk8PixYv9HYYIEvbNfKoZ0FGLvqWmxHoKH7rq9jyMQPA0beWYQKBloxB3u81BkogITn5PFsnJycydO5fp06djNpuZPHkygwYN8ncYIsAcN/BFTphP/fZ3UIu+xVrsdlgx5YfCd2u5SyDQuuK5I3dJRAEMRoVBvePtU1m2BCX1EOEPfkkWn332mdPtrKwssrKy/PHWIgi5O5o1bOgkao7vt27iUwzWJzZ2rm3ueRjBoqniObSuBqIDqqqfM5UF59ZDJIkIXwjaHdyi43J3NGvYkAlOhyrZnheomkVbc00g1wxL58vdRa2ewnLkrh4CkkRE25JkIfzOU9NB11VSYE0Yavkx9LrqDpM4APr3iiM+KuSc+12nsAD7B727LrtNaWkSkVVaoimSLITfuTua1ZV9qsp2ljcKDcbgKHb7kqc6CGDvsgtnE0hL6yHgOYkAKIq1wO6uNiKF9s5NkoUICHejCEf2qSo765RVw/4tGDrQ9FRLeEokjvWQ80ki0HiMuofaiI271VoyxdXxSbIQQcl1Sa2tj5S1VUjwLKkNBv5IIo48LfmFpqe4JKG0b5IsRFBxXVJrawui11WjV5/CvO9zp8K4JAvPWpJEbP8uq6wj56sjrU4mTU1x2XjqqSWJJbhJshBBw+2S2iETnB43799yTmFctExTdZHExC4MyUhwm0xau1rLVVM9tVzZRiopCdEkxoRJcgkgSRYiaLhbUus4cnAtjAPU573fKesXvtRUMmlqtVZbTHG5so9UvIxWbFynwc7Umt3GCpJYWkqShQganpbUOrIVxl1HIWEjp3a45bXBqKlEAk1PcbWmp1ZLNWcazKY59RVZAXaWJAsRNJqzpBas01H12985W/xWzdbOtWhS+A4wb8kE3PfU8pRY2nqk4qglicXTCrCIiFASY8I6xQhGkoUIKp6W1NoK32fPwDi7/wLF4NdDlMT5aU5CsbGNVGwfyoFKLk2tAPOmuSOY5vw7kKMcSRYi6DlOOYEtMQAoGBIuRImKtR7Z2k57SQnPbImluUeEuk6DefqL39ejFkctGcF401RDSdfRTGJilzZ5TxtJFiLoORW+0RobDeqgGNDKC+HUIVCMhPQfS0i/UTKq6MRaM2oBz3/Nt9UKsLbSVENJR5v3lPBEbKTbljKtJclCBD3XwretmO207wINJTpeEoVotuYmlqZWgHmrWfhzBOPIoursOXiSqwd1a7OvKclCBD1PhW9P+y4c6xtl24qpqzPLiEO0mrd9Kd6mx5ozgmnJv5vTUNJkVBjYJ6F532AzSbIQ7YK7wre7fRe1m16ytgTRrKtW6hufa87fRGTWQ5IwhN+1ZGqsOdw1lHT998iB3ejfK65ZdZ7mkmQh2rVz9l3YV0m5kFVSooNo6+TTXAa/v6MQPnBul1oXLtNU9Xnvo5YW+Ck6Ido/GVmIDsGpCK4YCMkcjSEhndBqa83CkJBuP0ipPvffTv2nZLQhhHeSLESH4KkInpjYheN789zs07Du/G7Yv4UISRZCeCXJQnQYnnZ/O+/TULFubQLQsezbSB0Q0m+U/bnSX0qIc0myEB2eqXt/GhQj6JZzH9RVzN/9F3P+JvttmZ4S4lxS4BYdnjE5g5DM0c53KkbOjjAATbXWOxzaowshzpJkITqFkH6jwBhqbRViDCHsqmmYLroaDCasSUMBg9H6uMFkbVgoK6aEsJNpKNEpeCqANySkW9ub6xpgwNjzUgDqt7wCmoUGxUjIoPEooZFSyxCdmiQL0Wm4K4DrddWArb25ilq4C3TV4Qkq5l0fAgoNRqlliM5LkoXo1Jz2Z6A4tD935VzLkFVTorORZCE6NcfpKfvBSrbEge6SPBTUiiIatr8NukaDwUhI5mhpUig6BUkWotNznJ4yxqU5NSa0FO9Db6jBvPsj0FXUgq1nX6hZ7MtubUnD9hoZdYiORpKFEA5c6xrG5Azq897HWtfwwJY0vvsCDErjqEPqG6JjkaWzQnhh6t4fDCE47cvAYF1q63SfZt2voVtbidTv/FCW34oOQ0YWQnjhWtfQ66rt01QN+7dg2feFm8K4jnpkB+qRHbL8VnQIkiyEaAZPfacikjOc92ooblZU2ZffYk8cZQaLnOAn2pWgThaqqjJz5kzmz5/PwIEDAx2OEG6FXnS1vTBuX1GlmgH93Cc3Jg7byRvmfRsx9rwUQ2QMhoR01JNHUECSiAg6QZ0sVq5cSVJSUqDDEMIrdyuqHFdReaSr1ukql7sdk4gkDhEMgiZZPPfcc2zevNl+e8qUKfTt2xdN83IyuRBBxjFxhPS6rPmJw5FDEjHv20hI/zFOIw9DQrq9diKJRPhD0CSL2bNnM3v2bPvt+++/n+joaPbu3UthYSFPPfVUAKMTonXcJQ4lPJrQ6mJqyk+e217EncY26u7YaiB6Q61MXwmfCppk4Wr58uUArFixgquvvjqwwQjRBhwTR2JiF8rKqlBLC2jYv8U+WlBPHkGvqWxeEgGn4jlIDUT4js+TRXV1NdnZ2axcuZK0tDQA1q9fz7PPPovFYmHGjBncddddHl9/3333+TpEIQLGmJzh9lhXWxLRaypRj+627t9wVzB31YwaiExnidZQdF1vxm9g6+zatYsFCxZw6NAhNmzYQFpaGqWlpUyZMoV169YRGhpKdnY2y5cvJyNDflGFcKfuWD61R77FGNGF+uPfW4/eCI2k8uv1jUmkDRiMxAzLwhAe5fQ+XQZeTXhaZtu8h2jXfDqyWLt2LQsXLmT+/Pn2+3Jzcxk+fDixsbEAjB8/ng0bNjBnzpw2f/9Tp6rRtJbnQtsUQbCRuFouWGNrUVxh3aFfdywAPYYD1uYjkckD7DWQFk9fudJUKr9855y7q/I+9Tgi8ecUV4f4OfpZS2MzGBTi46M9Pu7TZLFkyZJz7jtx4gSJiYn220lJSezevduXYQjRIbnbKHjeNRBXHqa1HMkUV+fg9wK3pmkoytl+OrquO90WQrSetxqI66iA0IiWLel1x0tCcV2xZUsithGRJJb2we/JIiUlhe3bt9tvl5WVycY7IXzMUxIB5yW9kUYzNWrI+Y9IHLms2GqKu8SinjxCWUQIDVHdZXVXAPk9WYwcOZIVK1ZQXl5OREQEOTk5LF682N9hCCEaOU5ndU3sgsVhntvTiKRNprjc8ZBYXGfem5r6ch2tODZ/lATTen5PFsnJycydO5fp06djNpuZPHkygwYN8ncYQohmaGpEYuPTKS5PmlFLceVp1NLUv2UEc5ZPl84GmqyG8o9gjQuCN7bOEpdaWuC0YstTzcKnieV8KMYmRzARjdNjrt9PMCSZdrUaSgjRuXlq7e6OY+3E04eyT6a+muJlBOPpo7i502TtaTQjyUIIERSaSiyufyV7q6U4jmBa3MSxLbRimszG21Jkd8nUXcIhcUibfkuSLIQQ7U5zaimOmhq1+K1431znkWhszPmbqItdZN3Q2UYkWQghOryWTIc58jaCcVezCFiScaRZqD3yLfSTZCGEED7nbQTjqYjc3Gkyn41mDCYi0i/xWFNpDUkWQgjRxlo6TeaoqUTTkppFeFomVW24sk2ShRBCBJHzSTS+ZAh0AEIIIYKfJAshhBBeSbIQQgjhlSQLIYQQXkmyEEII4VWHXg1lMLT+UKXzea0vSVwtF6yxSVwtI3G1XEti8/bcDt11VgghRNuQaSghhBBeSbIQQgjhlSQLIYQQXkmyEEII4ZUkCyGEEF5JshBCCOGVJAshhBBeSbIQQgjhlSQLIYQQXkmyEEII4VWH7g3VUuvXr+fZZ5/FYrEwY8YM7rrrroDF8vTTT/Of//wHgLFjxzJ//nwefvhhvvnmGyIiIgCYM2cO119/vd9jmzZtGuXl5ZhM1l+fRYsWUVhYGNBr98Ybb/DKK6/Ybx87doyJEydSW1sbsGtWXV1NdnY2K1euJC0tjdzcXJ544gnq6+u56aabmDt3LgDfffcdv/3tbzlz5gxDhw7lD3/4g/3a+iOu119/ndWrV6MoCgMGDOAPf/gDoaGhPP3007z11ltccMEFAPzoRz/y6c/VNS5Pv++erqMvOcZ28OBBli9fbn+stLSUSy+9lFWrVvn1mrn7jPDp75gudF3X9ePHj+vjxo3TKyoq9DNnzuhZWVn6gQMHAhLLli1b9B//+Md6fX293tDQoE+fPl3PycnRJ0yYoJeWlgYkJhtN0/SrrrpKN5vN9vuC6drpuq7v379fv/766/VTp04F7Jrt3LlTnzBhgn7JJZfoR48e1Wtra/WxY8fqhYWFutls1mfNmqV//vnnuq7r+i233KLn5eXpuq7rDz/8sP7qq6/6La7vv/9ev/766/Wqqipd0zR9/vz5+gsvvKDruq7//Oc/13fs2OGzWJqKS9d1tz+7pq6jP2OzOXHihH7ttdfqhw4d0nXdf9fM3WfE+vXrffo7JtNQjXJzcxk+fDixsbFERkYyfvx4NmzYEJBYEhMTeeihhwgNDSUkJIQ+ffpQXFxMcXExjzzyCFlZWfztb39D0zS/x/b9998DMGvWLG699VZeeeWVoLp2AI8++ihz584lIiIiYNds7dq1LFy4kKSkJAB2795Neno6PXr0wGQykZWVxYYNGygqKqKuro7BgwcDcPvtt/v02rnGFRoaysKFC4mOjkZRFPr160dxcTEAe/fuZdWqVWRlZbFo0SLq6+v9Fldtba3bn52n6+hLrrE5Wrp0KdnZ2fTq1Qvw3zVz9xlx+PBhn/6OSbJodOLECRITE+23k5KSKC0tDUgsffv2tf9gDx8+zH/+8x9Gjx7N8OHDefzxx1m7di3bt2/nzTff9Htsp0+fZsSIETzzzDO8+OKLrFmzhuLi4qC5drm5udTV1XHTTTdx8uTJgF2zJUuWMHToUPttT79frvcnJib69Nq5xpWamsqoUaMAKC8v59VXX+Xaa6/lzJkzXHTRRcybN4+3336b06dP8/e//91vcXn62QXi/1PX2GwOHz7M119/zfTp0wH8es3cfUYoiuLT3zFJFo00TUNRzvZz13Xd6XYgHDhwgFmzZjF//nx69+7NM888Q1JSEhEREUybNo0vvvjC7zENGTKEpUuX0qVLF+Li4pg8eTJ/+9vfgubarVmzhp/+9KcA9OjRIyiuGXj+/QqW37vS0lJmzJjBHXfcwZVXXklUVBT//Oc/6dOnDyaTiVmzZvn12nn62QXL9QJ4/fXXmTp1KqGhoQABuWaOnxE9evTw6e+YJItGKSkplJWV2W+XlZW5HXb6yzfffMPMmTN54IEHuO2228jPz+ejjz6yP67ruk+LoJ5s376drVu3OsWRmpoaFNeuoaGBbdu2cc011wAEzTUDz79frvefPHnS79fu4MGDZGdnc9ttt3HvvfcCUFxc7DQK8/e18/SzC6b/Tz/99FNuvvlm+21/XzPXzwhf/45Jsmg0cuRItm7dSnl5ObW1teTk5DBmzJiAxFJSUsK9997LsmXLuOWWWwDrL97jjz9OZWUlZrOZ119/PSAroaqqqli6dCn19fVUV1fz9ttv89RTTwXFtcvPz6dXr15ERkYCwXPNAC699FIOHTrEkSNHUFWV999/nzFjxpCamkpYWBjffPMNAO+++65fr111dTV33303v/rVr5g1a5b9/vDwcJ566imOHj2Kruu8+uqrfr12nn52nq6jv5WXl1NXV0ePHj3s9/nzmrn7jPD175gsnW2UnJzM3LlzmT59OmazmcmTJzNo0KCAxPKvf/2L+vp6nnzySft92dnZ/N///R9TpkzBYrFwww03MGHCBL/HNm7cOHbt2sWkSZPQNI2pU6dy+eWXB8W1O3r0KCkpKfbb/fv3D4prBhAWFsaTTz7JfffdR319PWPHjuXGG28EYNmyZSxYsIDq6mouueQS+xy4P7z55pucPHmSF154gRdeeAGAa665hl/96lcsWrSI//f//h9ms5nLLrvMPr3nD0397DxdR386duyY0+8aQFxcnN+umafPCF/+jsmxqkIIIbySaSghhBBeSbIQQgjhlSQLIYQQXkmyEEII4ZUkCyGEEF7J0lnR4Tz22GNs27YNsG44S01NJTw8HLDuurX925tPP/2UrVu3smDBAo/PKS0t5Ve/+hVr1qw5/8CBhx56iC1bthAXF+d0/x133NHmS2ofeugh+vbty913392mX1d0TLJ0VnRo11xzDX/9618ZOHBgoENpFn9+gEuyEC0hIwvR6QwYMIBrr72Wffv2sWzZMvLz83n99dcxm81UVlbys5/9jKlTp7Ju3To++ugjVq1axbRp0xg8eDA7duygpKSEESNGsHjxYoqLi8nKyiIvL48VK1ZQVFREWVkZRUVFJCcn89RTT5GUlMTu3bt59NFHMZvN9OzZk+LiYh566CGuvPLKFsV+zTXXcMstt7Blyxaqqqr46U9/ytSpUwHs51IYDAYSEhL43e9+x4UXXsiZM2d47LHH2LFjB0ajkeuuu85+zkFeXh7Z2dmcPHmSvn378qc//cm+A14IR5IsRKdjNpsZN24cf/3rX+0fpP/4xz/o2rUrO3fudPoAdlRYWMjq1aupqanhpptu4uuvvyYtLc3pOdu3b+edd94hOjqaX/ziF6xZs4Z77rmH++67j0WLFjF27Fi+/PJLZs6c6TG+F198kffee8/pvqVLl5KZmQlAZWUlb731FqWlpUyaNInLL7+c8vJynnvuOV5//XXi4uJYt24d9957Lx988AF/+9vfqK+v58MPP0RVVWbNmsXXX38NWKfRXn75ZUJDQ7nzzjvJyclh0qRJ53mFRUckyUJ0SraW01FRUaxcuZIvvviCw4cPs2/fPmpqaty+Zty4cRgMBqKjo0lPT6eysvKcZDFs2DCio6MBuPjii6msrGT//v2A9TQzgOHDh9O3b1+Psc2cObPJqaGpU6eiKAopKSmMHj2aLVu2cPLkSW6++WZ7reP2229nyZIlHDt2jNzcXB5++GGMRiNGo9F+ouDbb7/NddddZz+Jrm/fvpSXl3u9dqJzktVQolOyTbUcP36cSZMmUVRUxOWXX86vf/1rj69xLIwrioK7cp+75xiNxnOeazQaWx27YydTTdMwGAxuD3XSdR2LxYLJZHJqSV1SUkJFRcU5X8vT9yQESLIQndzevXuJi4vjnnvu4aqrruK///0vAKqqttl79OnTh9DQUDZu3AhYT83bv39/q89heOeddwBrS+wtW7YwZswYRo8ezYcffmgfGbz11lvExsaSnp7OiBEjePvtt9E0jYaGBn75y1/aV4sJ0VwyDSU6tVGjRvHmm29y4403oigKw4YNIy4ujiNHjrTZe5hMJlasWMHChQtZvnw5vXr1IiEhweMSXnc1i0svvZRFixYB1o6nt99+O3V1dSxYsIDevXvTu3dvZs6cyYwZM9A0jbi4OFatWoXBYGDOnDksWbKEiRMnoqoqN998MzfccAOfffZZm32PouOTpbNC+MEf//hH7r77bhISEigpKWHixIl88sknXHDBBS36Ou1tKbDoOGRkIYQfpKamMnPmTEwmE7qu89hjj7U4UQgRSDKyEEII4ZUUuIUQQnglyUIIIYRXkiyEEEJ4JclCCCGEV5IshBBCePX/Adx2BiDTBkmUAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "torch.manual_seed(123)\n", "for name, model_name in (('BatchNorm', 'batchnorm_net'), ('Normal', 'net')):\n", " model = eval(model_name)\n", " model.load_state_dict(torch.load(f'{model_name}.pth'))\n", " optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)\n", " model.train()\n", " losses = []\n", " loader = torch.utils.data.DataLoader(xy_train, batch_size=1000, shuffle=True)\n", " for epoch in range(200):\n", " for x_batch, y_batch in loader:\n", " y_pred = model(x_batch)\n", " optimizer.zero_grad()\n", " loss = loss_fn(y_pred, y_batch)\n", " loss.backward()\n", " optimizer.step()\n", " losses.append(loss.data)\n", " plt.plot(losses, '.', label=name)\n", "plt.legend()\n", "plt.xlabel('Training Epoch')\n", "plt.ylabel('MSE Loss')\n", "plt.yscale('log');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "BatchNorm is considered essential for many of the cutting edge architectures, but these are much deeper and wider than our simple problem. The lesson is that you should not assume that cutting edge techniques will benefit your (likely simpler) problem!" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 2 }