{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Data Analysis and Machine Learning Applications for Physicists" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Material for a* [*University of Illinois*](http://illinois.edu) *course offered by the* [*Physics Department*](https://physics.illinois.edu). *This content is maintained on* [*GitHub*](https://github.com/illinois-mla) *and is distributed under a* [*BSD3 license*](https://opensource.org/licenses/BSD-3-Clause).\n", "\n", "[Table of contents](Contents.ipynb)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns; sns.set()\n", "import numpy as np\n", "import pandas as pd\n", "import warnings\n", "warnings.filterwarnings('ignore')" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from mls import nn_unit_draw2d, nn_graph_draw2d" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import torch.nn" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "import mls.torch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Neural Networks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From a user's perspective, a neural network (NN) is a class of models \n", "\n", "$$ \\Large\n", "X_\\text{out} = N(X_\\text{in}; \\Theta)\n", "$$\n", "that are:\n", " - **Generic:** they are not tailored to any particular application.\n", " - **Flexible:** they can accurately represent a wide range of non-linear $X_\\text{in}\\rightarrow X_\\text{out}$ mappings with a suitable choice of parameters $\\Theta$.\n", " - **Trainable:** a robust optimization algorithm (backpropagation) can learn parameters $\\Theta$ given enough training data $D = (X_\\text{in},Y_\\text{tgt})$.\n", " - **Modular:** it is straightforward to scale the model complexity (and number of parameters) to match the available training data.\n", " - **Efficient:** most of the internal computations are linear and amenable to parallel computation and hardware acceleration.\n", "\n", "The \"neural\" aspect of a NN is tenuous. Their design mimics some aspects of biological neurons, but also differs in fundamental ways.\n", "\n", "In this notebook, we will explore NNs from several different perspectives:\n", " - **Mathematical:** What equations describe a network?\n", " - **Visual:** What does the network graph look like? How is the input space mapped through the network?\n", " - **Data Flow:** What are the tensors that parameterize and flow (forwards and backwards) through a network?\n", " - **Statistical:** What are typical distributions of tensor values?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Mathematical Perspective" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Building Block\n", "\n", "The internal structure of a NN is naturally described by a computation graph that connects simple building blocks. The basic building-block unit is a function of $D$ input features $x_i$," ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$ \\Large\n", "f(\\mathbf{x}) = \\phi\\left(\\mathbf{x}\\cdot\\mathbf{w} + b\\right)\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "with $D+1$ parameters consisting of $D$ **weights** $w_i$ and a single **bias** $b$. The corresponding [graph](http://alexlenail.me/NN-SVG/index.html) (with $D=8$) is:\n", "\n", "![NN unit](img/NeuralNetworks/nn_unit.png)\n", "\n", "where the left nodes correspond to the elements of the input $\\mathbf{x}$, the edges correspond to the elements of $\\mathbf{w}$ (thickness ~ strength, red/blue are pos/neg values), and the right node is the output value $f(\\mathbf{x})$. The recipe for obtaining the output value is then:\n", " - propagate each input value $x_i$ with a strength $w_i$,\n", " - sum the values $x_i w_i$,\n", " - apply the activation $\\phi$.\n", "\n", "Note that this building block is mostly linear, except for the **activation function** $\\phi(s)$. This is an application of the kernel trick that we met [earlier](Nonlinear.ipynb), and allows us to implicitly work in a higher dimensional space where non-linear structure in data is easier to model.\n", "\n", "The building-block equation is straightfoward to implement as code:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def nn_unit(x, w, b, phi):\n", " return phi(np.dot(x, w) + b)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For example, with a 3D input $\\mathbf{x}$, the weight vector $\\mathbf{w}$ should also be 3D:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "-0.9640275800758169" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_unit(x=[0, 1, -1], w=[1, 2, 3], b=-1, phi=np.tanh)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Activation Functions\n", "\n", "The activation function $\\phi$ argument $s$ is always a scalar and, by convention, activation functions are always defined in a standard form, without any parameters (since $\\mathbf{w}$ and $b$ already provide enough learning flexibility).\n", "\n", "Some popular activations are defined below (using [lambda functions](https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions)). For the full list supported in PyTorch see [here](https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity)." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "relu = lambda s: np.maximum(0, s)\n", "elu = lambda s: np.maximum(0, s) + np.minimum(0, np.expm1(s)) # expm1(s) = exp(s) - 1\n", "softplus = lambda s: np.log(1 + np.exp(s))\n", "sigmoid = lambda s: 1 / (1 + np.exp(-s)) # also known as the \"logistic function\"\n", "tanh = lambda s: np.tanh(s)\n", "softsign = lambda s: s / (np.abs(s) + 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These activations divide naturally into two categories depending on their asymptotic behavior as $s\\rightarrow +\\infty$:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1gAAAFgCAYAAACmKdhBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzs3XeUJHd56P1vVXV17sl5Z3d2NtXmXWWtgs01xj4YYbAINhhhccHYfg34GM7hYmxz5ZdrGxuwBa/AF2OEBUJISEJIQhIoIRR2V2FzrE0zO2knh57Ooer9oyfVpJ3ZndAz+3x0Vt31VOhf1cz0r5/+hVJs20YIIYQQQgghxOVTF7sAQgghhBBCCLFcSIIlhBBCCCGEEHNEEiwhhBBCCCGEmCOSYAkhhBBCCCHEHJEESwghhBBCCCHmiCRYQgghhBBCCDFHXItdAMMwdOBeYDXgAf6PaZpPjFn/buBLQAa41zTN7y5GOYUQQly5pK4SQggxU/nQgvURoMc0zVuBdwL3DK8YqtD+Hfgd4DeBTxqGUbUopRRCCHElk7pKCCHEjORDgvUw8PdjljNjnm8Czpim2WeaZgp4Fbh1IQsnhBBCIHWVEEKIGVr0LoKmaUYADMMIAY8AfzdmdQEwMGZ5ECi82DEzmaztcmlzWUwhRJ7JxOJceOpp7Mzo59yQsYEXs3EOtp4aifndXj73jvfh1vXFKKa4dMpiF2AsqauEEEJMYtK6atETLADDMFYCjwHfNk3zgTGrwkBozHII6L/Y8fr6YnNbwDlQXh6iq2twsYuR9+Q6zYxcJ4gc2EdiIDqyrLo9pArK2X/oObJWFgBdd7GxpJ6B/gSQWKSS5r98/H0qLw9dfKMFJnWVGCbXaWbkOs2MXKeZycfrNFVdtegJlmEYlcCzwKdM03xh3OoTwHrDMEqACPAbwNcWuIhCiDyT6e8j0dTkiPk3buSl1rMjyRWAT/dw3ar1C108sQxJXSWEEGKmFj3BAr4IFAN/bxjGcP/27wIB0zT/0zCMzwK/JDde7F7TNFsXqZxCiDxg2zbRI4cdMVdBAanKSsy3Djji19VtwiNdA8XckLpKCCHEjCx6gmWa5l8BfzXN+ieBJxeuREKIfJZqbSHd2+uI+bds49fnTTLW6Hgsr8vDrcYWBvula6C4fFJXCSGEmKl8mEVQCCFmxM5kiB0/5oi5q6pJFRRwvKvBEd9WuR6vtF4JIYQQYoFJgiWEWDLip0+RjcdHlhVVJbBlK3saTzpar9yam+tXy9grIYQQQiw8SbCEEEtCNhYlfvaMI+Zdu46k28WJznOO+NbKdXh190IWTwghhBACkARLCLFExI4dw86OzhCoer341m9gT4NJ2kqPxN2amxtXG4tRRCGEEEIISbCEEPkv3dVFss05KZt/02aSts2xjrOO+NbKtfjc0nolhBBCiMUhCZYQIq/ZlkX06Lhp2YuL8axcxd7Gk5O0Xm1c6CIKIYQQQoyQBEsIkdcSDefIhMOOWGDrdhLpNEfHtV5trlgjrVdCCCGEWFSSYAkh8paVTBI3Tzpi3lWr0EtK2NtoksqmRuK6qrOrXsZeCSGEEGJxSYIlhMhbsRPHsdKjXQBVXce/aQuJdIqjHc4ZBTdXrMHv9i50EYUQQgghHCTBEkLkpXRfH8mm846Yz9iI6vXyRuNpR+uVS3XJ2CshhBBC5AVJsIQQece2bWJHDmPb9kjMFQrhrV9DMp3mcPspx/aby+sJeqX1SgghhBCLTxIsIUTeSTadJ93X64j5t25HUVVeP3+K5PjWq/pNC11EIYQQQohJSYIlhMgrVipF7PhxR8xTXYO7ooJkOs2R9tOOdUZZPSGvbyGLKIQQQggxJUmwhBB5JXbyOFYqObKsaBr+rdsAeLPpNInM6DqX6uLmNdJ6JYQQQoj8IQmWECJvZPr7STY2OmK+DQaa308qk+bQhfGtV6ul9UoIIYQQeUUSLCFEXrBtm+jhQ46JLbRAEN/adQC8ef4MiUxidJ2qsateZg4UQgghRH6RBEsIkReSzU0TJrYIbNuGomlDrVfOmQM3lNZR6AssZBGFEEIIIS5KEiwhxKKzkklix445Yu7qatyVVQDsazpLfGzrlaKxa/XmBS2jEEIIIcRMSIIlhFh0sRMTJ7YIDE1skclmOdjmbL1aV7qK4oC0XgkhhBAi/0iCJYRYVOneXhLnGx2x3MQWuQRqX9NZYpn4yDpVUbm5XlqvhBBCCJGfJMESQiwa27KIHj7oiGmhEL5164Fc69WBNtOxfn3pKooDwQUroxBCCCHEbEiCJYRYNInGBjIDA45YcNsOFDX31rS/6SzRdGxknaqo7Fq9ZUHLKIQQQggxG5JgCSEWhZWIEz95whHz1Nail5cDkLGy7B839mpNyUpKg9J6JYQQQoj8JQmWEGJRRI8ewUqnR5ZVXSewZevI8sHmc0TT0ZFlRVG5RcZeCSGEECLPSYIlhFhwqY52kq2tjphv4yZUrw/ItV7ta3WOvVpTXEtpsGDByiiEEEIIcSkkwRJCLCg7kyF6+JAj5ioqxru6fmT5cEsjkZSz9ermNTL2SgghhBD5TxIsIcSCip0yycZGJ65QFIXgjp0jE1tkrCxvtZx07LOmaAXl0nolhBBCiCVAEiwhxILJhMMkzp5xxLxr1uAqKhpZPtJ6nsFUZGRZUVRuqpfWKyGEEEIsDZJgCSEWhG3bRA8dwLaskZjm8+EzNo0sW5Y1ofWqrrCGioLCBSunEEIIIcTlkARLCLEgko0NpHt7HTH/1u2ouj6yfLitiXBycGRZURRuWbMVIYQQQoilwrXYBRhmGMYNwL+Ypvm2cfHPAh8HuoZCf2aapokQYsnIxuPEThx3xNxV1birq0eWLcvirWbnNnWFNVRK65XII1JXCSGEuJi8SLAMw/g8cAcQnWT11cBHTdPct7ClEkLMldiRw457XikuF4Ft21EUZSR27EIzA+Nar3bJ2CuRR6SuEkIIMRN5kWABZ4HbgR9Osu4a4G8Mw6gCnjJN858vdrDiYj8ulzbHRbx85eWhxS7CkiDXaWaWynWKNbcQD/fgCnhGYsVXX0WornJk2bIsDh04g66PviWtLa1lx7pVl/36S+U6LTa5TjMidZUYIddpZuQ6zYxcp5lZKtcpLxIs0zQfNQxj9RSrHwS+BYSBxwzDuM00zZ9Pd7y+vth0qxdFeXmIrq7Bi294hZPrNDNL5TpZ6RT9v96NlUiOxPTiEuJFlSTGlP9o63k6Brod+15Vtf6yz3GpXKfFlo/XKR8rUamrxDC5TjMj12lm5DrNTD5ep6nqqrye5MIwDAW42zTNbtM0U8BTwFWLXCwhxAzFjh/HSiRGlhVVJTDmnleQa716o/mEY7+VBdXUFJUsWDmFuBxSVwkhhBgrL1qwplEAHDUMYxO5Pu+/Bdy7uEUSQsxEuruLRGODI+Zbtx5XoXPSipMdrfQlBhyxm2TmQLG0SF0lhBBiRF4mWIZhfBgImqb5n4ZhfBH4FZAEXjBN8+nFLZ0Q4mLsTIbIwYOOmBYM4lu/wRGzLIvXm5ytV7UFVdRK65VYAqSuEkIIMZm8SbBM02wEbhx6/sCY+A+ZfECxECJPxcyTZKMRRyy48yoUl/Mt51RnG73xPkds12qZOVDkL6mrhBBCXExej8ESQiw9mf4+EmfPOGLe+nr00jJHzLIs9p533veqJlTJqhLndkIIIYQQS4kkWEKIOWNbFpEDB7BteySm+fz4N01slTrddYGeca1XN0nrlRBCCCGWOEmwhBBzJn7KJBN2TlgR2LEDVdcnbDu+9ao6WEFdafm8lk8IIYQQYr5JgiWEmBOZgX7ip085Yp6Vq3BXVk3Y9nRHG92xXkds1+rN81o+IYQQQoiFIAmWEOKy5boG7se2rJGY6vUS2Lpt0u33Njlbr6qC5dSXVc5rGYUQQgghFoIkWEKIyxY/ZZIZcHYNDG7fgep2T9j2TOcFOqM9jpiMvRJCCCHEciEJlhDiskzaNbB2Je7qmkm33zNu7FVFoExar4QQQgixbEiCJYS4ZHY2S2T/JF0Dt22fdPuG7g46o92O2K46GXslhBBCiOVDEiwhxCWbbNbAqboGAuxuPOZYLveXsq6iet7KJ4QQQgix0CTBEkJcknRvz6y6Bp7v6aI90uWISeuVEEIIIZYbSbCEELNmZzK5WQMdNxT2Tdk1EGB3w1HHcpm/hLXlE6dwF0IIIYRYyiTBEkLMWvT4MbKRiCMW2HHVlF0Dz/d00RbpdMRuWLUJVZW3ICGEEEIsL/LpRggxK6nOThIN5xwxb3097sqpZwIcP/aq1FfMhorJuxIKIYQQQixlkmAJIWbMSqWIHNjniGmBIIHNW6fcp6m3m7bBDkfshjppvRJCCCHE8iSfcIQQM2LbNtFDB7ESiZGYoigEr7oaxeWacr8941qvir1FGBUr5q2cQgghhBCLSRIsIcSMJJubSLa1OmK+9RvQS0un3Kelv5eWcLsjJmOvhBBCCLGcyaccIcRFZaNRokcOO2KuomJ8xsZp99t9zjlzYLG3kE1VtXNePiGEEEKIfCEJlhBiWrZlEdn/FnYmMxJTNI3Q1degTNMS1dbfS3P4giN2/UppvRJCCCHE8iafdIQQ04qfMkn39jpigS1b0UKhaffb3eAce1XoLWBz9co5L58QQgghRD6RBEsIMaV0TzfxU6Yj5q6sxLO6ftr92gb6aBrfelUrrVdCCCGEWP7k044QYlJWKkVk31vYtj0SUz0egjuvRlGUaffd03DMsV+hJ8TWmlXzVlYhhBBCiHwhCZYQYgLbtokePEA2HnfEg1ddg+r1TrtvR3iA8wNtjti1tRul9UoIIYQQVwT5xCOEmCB5vpHkBWeS5Fu3Hndl5UX3fW1c61XIHWT7itVzXUQhhBBCiLwkCZYQwiEz0E/06BFHzFVUjH/T5ovu2xkeoLHfea+s62TmQCGEEEJcQeRTjxBihJVOM/jWm9jZ7EhMcbkIXXPttFOyD9vdcAzbtkaWQ+4g21bUzUtZhRBCCCHykSRYQghgaNzV4YNkIxFHPLh9J1oweNH9uyJhzo1rvbqmdiMuVZvTcgohhBBC5DNJsIQQACTPnyfZ0uKIeetW41k5s3tX7T533NF6FXQH2FG7ei6LKIQQQgiR9yTBEkIMjbs67Ii5CgoJbN02o/17IhHO9TU7YlfXGNJ6JYQQQogrjiRYQlzhrHSKwTcnGXd17XUoLteMjrG74RjWmNargB7gqlVr5rysQgghhBD5bmafnhaAYRg3AP9imubbxsXfDXwJyAD3mqb53UUonhDLkm3bRPbvJxudZNxVKDSjY/RFI5zpbXLErq7ZIK1XYlmSukoIIWZn+NYtNvaUMXt0hXN5zFIykyKZTU167LHbTXj96Us37drpKKjA5J+V8iLBMgzj88AdQHRcXAf+HbhuaN1rhmE8aZpm+8KXUojlJ376FKn2C46Yd3X9jMddQe6+V2Nbr/y6n6tXrZ2zMgqRL6SuEkLkG9u2sWyLtJUhY2fIWlkyVoaMnSVrZcnaFpadJetYzv3L2ha2bWFhj8SGj2cz+mjbNtbQoz3ucbgM9phUKRceE7MvPYkZy9/tIRZNTrwGw2WwxzyOjw0VY/S5PVKsseuxh0o+dAzs4XMYPebwayrAF6rvmLSs2l133TUnJ3057rnnnirgm8Dtn/70p/9rTHwrcI1pmv/16U9/OnvPPfdsBtRPf/rTx6c73ne+8+27OjousHbtBgDOnj3NU089ht8foLS0DIDnn3+G1177NVu37kBVVeLxOA8+eB8DA33U1eW6Nh0/fphf/OJJSkpKKSwsAuDnP/8p+/e/ydatOwDo6+vlkUd+RDKZYMWK3IfSffve4IUXnmHFilr8/gAADzzwA06ePM7GjVsAuHChlZ/97CcoClRWVgPw2mu/5te/fp61a9fjdnsAuP/+73HhQivr1hkANDSc4ec//yk+n4/S0nIAXnjhF7z22kts2bINVdVIJBL8+Mf/TV9fL6tX587lxImj/OIXT1BcXDJyLk899Rj79r3Btm07Aejv7+Phh39EIhGntnYVAAcOvMnzzz9DdfUKAoHcTHKPPvpjTPM4mzZtBaC9vY3HHnsIgKqq3Lns3v0yL730HGvWrMPjyZ3Lj350L62tzaxfnzuXxsZzPPnko3i9XsrKyod+Lr/gpZdeYPPmrWiaRiqV5IEH/pu+vh5Wr859aD958hjPPPMERUXFFBUVA/D004/z5pt72LbtKgAGBvp5+OH7icWirFyZmyb84MG3eO65p6mqqiYYzH3j8NOfPsiJE0dHzqWj4wKPPfYQtm1RVVUDwJ49r/DSS8+xevUavF7v0M/z+zQ3N7F+/UYAzp9v4MknH8Xj8VBWVgHASy89xyuv/IpNm7agaS7S6RQPPPB9enq6qK9fB4BpHueZZx6nsLBo5FyeeeYJ3njjNbZvz53L4GCYn/zkh0SjkZFzOXLkAE899QSVlVUj5/LYYw9x/PgRNm/OjZvq7Ozgpz99EMvKUl29AoDXX3+NX/3qWerq6lHDg0QPHuAXHS1cSMSo8wdxFRfTX1nBE08+itvtprx8+Fye55VXXsQwNuNyuchk0vzoR9+nrb2N0+lebGy0gRju5h7WVtWxYVU9AL/85ZPs3fsq27dfDUAkMshDD/2QSGSQVatWA3D48H6effYpKioqCYUKAPjZz37C0aMH2bx5OwDd3Z08+uiPyWQy1NTkzuWNN3bz4ou/ZNWq1fh8PgAefPAHNDaeZcOGTUM/z1Z+8pMfo+suystzN0l++eUXePnlF9mwYRO6rpPNZvnRj+6lq6uDNWvWA3D6tMnTT/+MYDBESUkpAM8++xS7d7/Mjh25c4lGIzz00A8YHAyPnMuRIwd59tmfU15eMXIuTzzxCIcP72fLlty59PR08eijPyadTlFTUwvAm2/u4cUXf8nKlXX4fH4AHnrohzQ0nMEwcvcfa21t4vHHH0bTNCoqqgB45ZUXefnlF1i/3kDX3ViWxf33f4/OzrHvf6d46qnHCAaDI+fy3HO5c9m2bSeKomDbab7//f8iHO6nri73szt69BC//OXPKSsro6CgEIAnn3yUgwf3jbz/9fb28MgjD5BKJUfe/956ay8vvPALamtXjrz/Pfzw/Zw9e2rkXNraWvjZz36CqqpUVubO5dVXX+LXv36edes24Ha7CQQ8/0CekbpK6iqpqxa+rvJ6c+/vP/7xfTQ1NYy8vzc1NfLEE4/MqK7q7u5kzZrcuZw6dYKnn36cgoICiotLgPyrq9JWhl+99Cwvv/wCoRWl9GcGaBts58mHf8Lp5lOEQwnO9jey//hbvPr8CzQmmmnJtnM+3My+V/Zw5K39RMrStEc7aO1t5eCze2jvbScWTNOX6Kf51DlOv36UhDdNXEsSSUU4t/so7Wea0ap9xDMJwn39NL5ynHgqjlqok86m6TrVSvvBRlwlXnApZO0sza+aRNr78dcUYNsW8e5BLrx+DlQFd2HufHuOt9F7rA13eQFZBVLpLO2vnCbWG8Uq9BNPZQi39NN7sImYZRNFYTCWpu9oC/1mOwN+LwPRFP19UXpfP0dfT5RuFHrCCfoaugkfa6U7maUnkaF3MEn0aCuDjT106y76I0nCvVESR1oIDyboR2EwliJ+vofkmU4GFJVI2iIaT5M53kqiM8yg100smSHZGyV7qp14KkNMU0mkMmSbesie7yHqdZPIWvz2lqsnravyogXLNM1HDcNYPcmqAmBgzPIgUHix46mqgsfjorw898fc3e1DVRUKCrwjMa9XR1UVystDaJpGLKahqgperz6yTTDoRVUVior8IzG324XLpY4sK0oSVVXw+90jsUDAjaoqFBcHRmLD+w4vx+N+VFUhGBwtk9+f26+0NEhBQWjMuYyWqadn+Fx8E86lrCyEruvE4y5UVcHnG92vpeXi56KqqUnOxTN0LqP76bqGqo7ul0wGhs7FM+FcSkoCFBXlYpqmOn4ufX25cwmFvI7rNHwubrebZNI94efS2po7l8LC0Wvg8bjQtNEyuVyZac4l4DgXYGQ5lQqjqgqBwMRzKS0NUlw8+bn09088l7E/F4/HQyqVmnAubW0XPxddz044l7NnmfRcLEsZWU6nB6c8l0KvSurIkZFlzaUSKimg6nd/i/Pt7RPOxecbPpcgPp+PdDqNqir0JgZRg25UXCiahqooXL12zZhzGf07yy3bE85lqr8zsEaWs9no0Lm4J/0dKysb/rmrjr+zwcGuCX9noz+XIH6/n2w2O+Hn0tEx+c9l7Ll4vUz4OwsGPRPORdc1slltZNmyYhN+LsO/myUlgTG/w85ziUT8k/xcRn83g8EglmVNeM/o6pr4njH256KqKpFIZOhcRq9vKDR8DZw/l1Rq9HcTElOey9jfTZdLQ9dHr0EsNvz+55nwO1ZaGiQ0w+6pC03qKqmrpK5a2LqqpCRAaenk74nh8GTviZPXVWPPpb19+vfE3PL811WaSyHrjRNORmjsaSBtpTkfa6K3a5C0laEr2UXaTnMufg6X5cK2LCws0kqapBYHDTRdQVEU3F4df8Azcuy0qowsp1UFVVXQdW0klhiqz7xj9lNVFVR7dDmTq0/cugt/wINt2wy6VBQFUFQyNmQtG8u2sS2b/liabNYmFU2RtWz6oin6uqJkLQtiaZSsRWtPDHQNbBvNssmks8TCuRYpJZlBtWziqSx2MjcmXLVsFBuyWRsUhUzaQgNQlFx5AUVVUcjVM7aeS2sURUFRQB9aJmuhDJ3jcEzRRvdjzH4oyuh+rgwKoGka6lBMHXo9XR/dbzKKPUfNdpdrqNJ60DTNG8fEtgNfMU3z94aW/x14zTTNR6Y7VlfXYH6c1Bjl5SG6ugYXuxh5T67TzFzOdbIzGQZee4VMf/9ITFEUQjfehLuiYsbHGYhH+f4bz5C1RyfHuGnVDnat2XhJ5ZoP8vs0M/l4ncrLQ8pil2EyUlcJkOs0U3KdIGNlGEgOEk6N/oumo2St0brTH5i869t8s8klL5msNfTPJmuNeZ61yVgW2axN1hrqP3dZlKH/T3x7nxibuI3H7SKVyky7zczWXWztzCiofPkDH5n0UHnRgjWNE8B6wzBKgAjwG8DXFrdIQixdtm0TOXTAkVwB+DZumlVyBbD73AlHcuVzeblGxl6JK5PUVUIIABKZJL2JPnoTffQlBwinBudsDNJUVEXFpbqG/mloSu4RWyWTgUzGJpOGdMYmnc79Sw09t+1copBLcBRcKOhDz4f/AwVFHY0Bo/GhJRibJDnjMNQ6dJkCXg/R7OSJqDbUUqeqCqoy/hHHsqKQe67kWgBVNVe+XAwUNTd9xXBLmDKy7Whs+FhTycsEyzCMDwNB0zT/0zCMzwK/JDel/L2mabYubumEWLoSZ85MuJmwu7oa3/oNszpOOB7H7Gl0xLZXr8ft0i+3iEIsGVJXCSEs26Iv0U9XvIeueDfh5OW32KmKitflxat58Lg8eDQ3Hs2DW9Nxq250Vcet6bgUF7alEk9kiSYyRONpotEM0USa/kSGVDo76fEVwDP0b06aci5CURRcmoJLU4f+5Z5rmoKuqWhDMU0dfdRUBU1Thh5VNEWhoiJEX18U11AipY1JqOYigZtLeZNgmabZCNw49PyBMfEngScXqVhCLBupjg5iJ445Yq5QiOBV18z6jWl3w3FH9wavy8v1devnpJxC5DOpq4QQlm3Rm+jjQrSD9mgnqXFTh8+ErukE9SBB3c+K8jKSERu/7sPn8uFWdUe9bNs28WSGcCxNJJZmMJZiMB4nEk9PmUTNB92l4tY1PLqWe+7ScOsquqai67ll3TW07MolU7nHuUmASgq8ZJPpOTiT+Zc3CZYQYv5kBweJ7HvTcb8IVdcJXX8jqj67VqfBRByzu9ER214lrVdCCCGWt2g6RtNgCy2DbbNKqgJ6gEJPAQXuEIWeECF3CI/mHllfXh6ii1zLVyZr0TeYZCCaYiCaIhxNMRhLkc5YUx3+suguFa/bhdet4XVreNwaXt2Fx63h0VU8ujaSVE3XJU44SYIlxDJnJZOEX9+LlR791kdRFILXXIcWDM76eLsbTpCxRgeZel0eab0SQgixLNm2TUesk/PhFrrjPRffQVEochdQ4iumxFNEsbcI95hkaqysZTEQSdEXz9DQ3Ed/JMlgLM1cTUCnqgo+jwv/8D+vC6/bhc/rwufW8HlcuDR1Tl5LOEmCJcQyZlsWg2+9QTYaccT9m7bgrqyc9fEiiQQnuxocsW1V6/HMshVMCCGEyGeWbdEaaefsQAPRVHTabXVNp8JfToWvjDJfyZQJVTyZoTecoHcwSd9gkv5IEsuyCQQ8RC9xFkFNUwl6XQR8OgGfTtCr4/e6CHh1vJ7c7VPEwpMES4hlyrZtoocPke7udsS9q+rwrlt3Scfc03Dc0Xrl0dzSeiWEEGLZsGyLlkgbZ/obiKfjU26nqzqVgQqqAxWU+UpRlYktQbFEmq7+BD3hBD0DCaKJSx8/pLtUQn43BX43Qb9OyK8T8rnxebS8m+BBSIIlxLKVOHOGxPlGR0wvLSOwY+clvRlHEgmOT2i9WodXn/ybOiGEEGIp6Y73cqLXnHYmwCJvIatCK6kJVKKpmmNdMpWlqz+e+zeQIHaJCVXAp1MYcOf+BT0U+CWRWmokwRJiGUq2thA9ftQR0wIBQtddj6JeWn/rvY0nHa1Xbs3NDauNyyqnEEIIsdhi6TjHe006op2TrlcUhZpgNfUFdRR6QiNxy7bpDSfo6IvT1RenPzL7bn5+j4uVlSE0O0BR0ENR0I3u0i6+o5gXtm1DNoudSWOl09jpNHY6g50Z9zyTwbYtyt9+66THkQRLiGUm3dNNZP8+R0zVdUI33Ijq8VzSMWOpBMc7zzliWyvXSuuVEEKIJcu2bZoGWzjRe5rsmC8Qh6mKxspQDWsK6/DrfgBS6SztvTE6++J09MVmNbufqigUhTyUFHgoCXkpDnnweVy5WQS7Lv/+WSJnOEmyUinsdAo7lcZKp3IJ0tjnQ4+jidTQP2s2MzZKgiXEspcZDDP4xuuONwdFVQledz2uUMElH3dvg0naGu3q4Nbc3Lh642WVVQjjTWFnAAAgAElEQVQhhFgs8Uycw13Hp5wZsCZYzcaSdfhcPmKJDGe7BmjvidEzkMCa4Sx/qqpQEvJSVuiltDCXUMmsfbNj23autSiVIqmmSHX0YiWT2Kmh5GjouZVO5R5TucfZJUlzTxIsIZYJKxFncO8erJTz3hzBnVfhLq+45OPGUymOjWu92lKxBp9bWq+EEEIsPe3RDg51HXN0ex9W5C1kc4mBhwCt7THaevroDSdmfOyikIfyIh/lRT5KJKGalG1ZuWQomcBKprCTCaxkcihZSo4+T6awUknsbO5myqnLmG1xoUmCJcQyYKVThPfsIRuLOeL+TZvxrFx1Wcfe22g6bqioqzo31svYKyGEEEuLZVuc6jvL2f6GCes01cXa0BqURBFHzBi94d4ZHdOja1QU+6gs9lNe5MPjvnLHT9mWlUuOEgnsRBwrkRj9lxx+TGInk3N2r6/5oGgaiq6juFwouo6q6ygu3RFTXC7Uab5olgRLiCXOzmQYfH0vmfCAI+6tW41v/YbLOnYineJoxxlHbHPFGvxu72UdVwghhFhIqWyKA51HJnQJtGwbLRPEk6zheIuNbV88sSoIuKkq8VNV4qco5Lki7jVlW1YuQYrHseIxrHgCKxHHisfJDj3mU+I0nCSpbjeK7kZxDyVKuhvF7UZx6ajuoaRJdw+tG/qnXX6SLAmWEEuYbVkM7nuTdI+zwnBXVhLYvuOyp3R9fVzrlUt1ydgrIYQQS0okHeWN9v0j97Wyyd30NxxN405UErTKSSnTJwYlBV5qSgNUlfoJ+vQFKPXCsi0rlyzFolixGFYsRjYWyyVUsRhWMrFo45oUTUN1u3EXF5IOWChuN6rbM/ToRnF7cgnUUDKlut2gLe609pJgCbFE2bZN5OABUu3tjrheWkro2kufjn1YMp3mSPu41qvyeoJeab0SQgixNPQnB3iz/QCpbIp01iIcTTEQSZHNaFSpa/EqQZjic3hJgZcVZQGqSwP4vUv/I7OdyZCNRslGo1jRKNloZCiRiuZaoBYwgVLdbhSPB9XjQfV4UT2eMcseFPfwozvXLU9RltRsi0v/t0WIK5Bt20SPHCLZ3OSIuwoKCV1/I4rr8v+0Xz9/iuT41qv6TZd9XCGEEGIhdMd7eav9AAPxBP2DKaLxNDY2XiVEtboGlzJxDE1hwM2K8iC15QH83qXXUmXbNlY8RnYwQjYySDYaITsYwYpGyMbj8/76qseD6vUO/fONPvd4ULy+0QRqDrrh5TNJsIRYYmzbpv/AQRINzkG6mt9Pwa5d0w66nKlc69VpR2xjeT0hr++yjy2EEELMt6aBC7zcuI/ewQTpTHYkHlLKKFfqUJTRXh5et4uVFUFqK4IUBpbGDLkjiVR4kOxgmMzgINnBQbKRQezMxNkR54Lq8aD6/Kg+L5rPn0ugfN5cbCiRWu6J00xJgiXEEhM/eYJEa6Mjpnq9FOy6GXWOEqA3m06TyIxOhepSXdwkrVdCCCHyXH8kyYHz5znYfQjLdnZ5K1KqKFFqURQFVVWoLg2wqjJIeZEvryeqsDMZMuEw2fAAmfAA2XCYbDiMlU5ffOdZUL1DiVPAj+YP5BInvx/N50P1+eakd8yVQq6UEEtI7JRJ7JRJIOAZialuDwU33YwWDM7Ja6QyaQ5dcLZeGWWrpfVKCCFEXrJsmws9Mc61DtAy0MkF6zQ2zuSqVKmlSK2mIOCmrjJEbUUQj55/rS12JkNmYIBMfx+Z/n6yA/1kI5E5mZ1PURRUnw8tGEQNBNACQTS/H3XoURKouSNXUoglImaeJHbyhCOm6joFN92EK1QwZ6/z5vkzJDKjN1XUVI2b6jfP2fGFEEKIuZDOZDnfEeFcW5hYIk3CjtBunZmQXFVq9WyqWM3qqgJKCjyLOrvcWLZtk40MEhnsInK2hUxfH9nB8GUnU6quo4VCaMHhfwHUYBDNH5AufAtEEiwh8pxt28TNk8TMk464quuEdt2Mq7Bozl4r13p1yhHbUFpHgU9ar4QQQuSHeDLD2bYBzrcPks7kkqmUHeeCdRqL0fFWukvjqqrNXF9n5EVrlZ3NkunvI93TQ6avl0xPD1Y6TTrgIRFNXvwA46i6jlZQgBYqQAuFcA09Kp78SSKvVLNOsAzD2ADUAnHgqGmaS2O+RCGWINu2iZ88QeyU6YgrLhehq65BLy6e09fb13SW+NjWK0Xj5jXSeiWWHqmrhFh+wtEUp1sGaO2KYI1p5cna6aHkKje5g9+rUxzycG3NZtYV1y9WcbEti0xfL+nubtLd3WT6erGz2YvvOAktEMRVWIhWWIiroBCtoCA3LkoSqbw0owTLMIwQ8Dng40AS6AC8wBrDMPYCXzVN88V5K6UQVyDbtokdP0b8jHM8lKrrVLztNwnjmWLPS5PJZjnY5my9Wl9aR6EvMKevI8R8kbpKiOWpZyDB6ZZ+2ntjE9bZtkW7fZaskqIo4KE45MGja6wtql/w5Mq2baxIhFRnB+nuLtLd3Zc0o58WDOIqKsJVVIyrqCiXTOlLY3ZDkTPTFqwXgR8C15im2TkcNAxDBW4B/swwjHWmaf7nPJRRiCuObVlEDx8icb7REVd1ndCNN+EpL4M5vtneW01niGVG75GhKTL2Siw5UlcJsUzYtk1nX5xTzf30hBNTbtOvtRD0Z6gNFuLScq05qwpqMYrXLUw5MxnSPd2kOjpId3aQjUZntb+q63irq7BdflzFJbiKiubkditicc00wbrZNM3U+KBpmhbwMvCyYRjy2yDEHLAti8j+t0i2tjriw2Ou5rpbIORarw6Ma71aV7qS4oC0XoklReoqIZY4e2hGwFPN/fRHph6XFPTp+EsGcdlJVMU7Ei/1lbCldOO8dp2zUinSHe2kLlwg1dkxq25/ms+Hq7QMvbQUV0kpWihERUUBXXP8palYXDNKsIYrLMMw/gB4zjTNyFTbCCEunZ3JMPjWG6Q6Ohxx1eOhYNdNczqhxVj7m84SS492vVAVlZvrt8zLawkxX6SuEmLpsmybtq4op1r6CUen/jMtDnlYX1uE7k/wRsdJVEYTKb/u5+qK7ahjbiI8Z+VLpUi1XyDV2kq6uwvbsi6+E7l7S+ll5ejl5eilZah+v4ybugLMdpKLfwYeNwzDAxwG/MD7TNN8Y85LJsQVxkomCe/dQ6a/zxHXfH4Kdt2EFgrNy+tmrCz725yTaKwrWUVxYG7uqyXEIpC6SoglwrJtWjojnG4ZYDA2dWJVUexjQ20RpYVeUlaaV1rfgjETXbhUF9dW7sStzV0jtZ3NkupoJ9ncTLqzY0ZJlaJp6KVl6BUV6OUVuVn9JKG64sw2wYqapmkZhvFOYDfwFeA/gd+c85IJcQXJRiKE9+4hG3V+4a4FgxTsuhnN75+31z7QdI7ouNarm6T1SixtUlcJkeeGEyuzuZ9oPD3ldjVlAdbXFlEcyk3sZNs2h7qOksyM6T6oKOys2EbIfflfDNq2Taa/n2TTeVKtLVjpqcs2TPP50CurcFdWoZeVyQ17xawTrKRhGKXAHwLfM03TNAxDvuYW4jKke3sZfH0vVsrZ19xVWEjBjTeher1T7Hn5Jmu9WlO8ktKg/FmLJU3qKiHy1EwSK0VRqC3PJVYFAWeLVEO4ia5YtyO2tnA1lf7yyytXOkWqpYXE+UYyAwMX3d4VCuGurkGvqsZVVCStVMJhtgnWXeS6W3QBvzIMQwGk0hLiEiVbW4gc2D9hgKy7ooLgtdej6vq8vv6hlkYiqdEZjxRF5Ra575VY+u5C6ioh8spMEitVUaitCLJhZRFB38T6rz85gNnrvHVJsbeIDcVrL7lc2cFB4g1nSTY1XXSyCi0YxLOiFnfNClwFBZf8mmL5m1WCZZrms8CK4WXDMAzgpTkukxDLnm3bxM2TxMyTE9Z5Vq4iuPMqFHXuB+mOlbGy7Gtxvv6aohWUBqXSEEub1FVC5I/hxOpUcz+RaRKrlZW5xCrgnfyLxYyV4UDnESx7dByUrursLN8260ktbNsm3d1F4uyZCZNKTSib24OnthbPypVohdJSJWZmpjcavsY0zX3j46ZpmuTuK+IB6k1zkk+LQggHO5MhcnD/hGnYAfwbDHwbNy3IG/iR1vMMpkbHfCky9koscVJXCZE/LNum8UKYPQdbpk2sVlWG2LCyEP8UidUws8852y3AtvLN+HXfjMtk2zbpjnZip0wyfX1TbqcoCnplJd661egVlfP+hadYfmbagvVFwzD8wAPA60AH4AUM4HeBdwGfAy6p0hq6CeS3gR1AEviEaZpnxqz/JnAzMHyTgPeYpnnxDrJC5JlsNMrgm69P6N+tqCqBHTvxrqpbkHJYlsWbzSccsdVFK6goKFyQ1xdinsxbXSX1lBAzY9s2rd1RzKZ+LEWZtDvgbBIrgN5EH43hJkdsVUEt1YHKGZcpdaGNuGmSCU/9Z6l6vXjrVuNZVTevk0uJ5W+m98F6n2EY1wOfBP43UAvEyfVxfwy41TTNy7lD2nsBr2mauwzDuBH4OvCeMeuvBn7XNM3uSfcWYglIdXQQ2f8WVso5Da3q9hC6/gb00tIFK8vh1sZxrVeK3PdKLHnzXFdJPSXENGzbpq0nhtnUN3Ifq0DA49hmuCugsbJoRokVQNbKcrjruGNKdr/uZ1PJhhntn+rqJHb8+IRboIzlKi7Gt2Yt7poV0lol5oRij/mFXSyGYfwb8IZpmg8OLbeaprli6LkKXABeAyrJzQh173THy2SytsulzXOphZgZ27YJHz/BwJGjjgoCQC8spPw3bsG1gLP2WZbF3c//jL5YeCS2vmwlH73l7QtWBiFmIK8GOsx1PQVSV4nlwR4aY3XkbDf9g8lJt1EUhfqaArauKSXon919qg61H8fsPueIvW31jVQEy6bdL9XXR/+BQySmGWPlW1FDwaZNuMtKZWyVuFST/uLMapILwzBWAZ8HioC3gPtM05z6K4GZKwDGttlmDcNwmaaZAQLA/wf8G6CRmxHqLdM0D091sL6+2FSrFk15eYiurstp5LsyLLfrZCWTRPbvI9U58Q3eU7MCz86r6IvbEJ/dOV/OdTrc0kjnQO/IsqIoXFW1flld92HL7fdpvuTjdSovv/Qba89TXTWn9RRIXbWUyXXKJVbtvTFONvUzEJk8sQoGPBQH3BircrMCxqNJ4tHJt51MX6KfAxdOOr6cXFVQixL30DVFvWmlUsROHifZ2MhkjQiKouCuWYFv/QYoLCQM0B2ZsN1Ckt+nmcnH6zRVXTWjdlDDMIZHED4ERIHngfXAG4ZhzMWczmFgbAnVoUoLIAZ8wzTN2FDXjhfJ9YEXIq+lu7vof+nFCcmVoigENm8leO118z4N+3iWZfFGi3Ps1aqCaqoKixe0HELMh3muq6SeEoJcYnWhJ8qvD7bx+vGOSZMrRVFYWRHk926u5xqjfNIp1y/Gsi2OdDu7BvpcXjZO0TXQtm0SjY30v/A8iYaGSZMrT80Kiv7H2wldex2uQhlzLObPTFuwWgzDaARWAd8FDgA/BLYD/05u8PDleA14N/CTob7tR8as2wA8aBjG1eQSwluA+y7z9YSYN7ZlET9lEj9lTniDV90egtdei7u8YlHKdvxCMwOJ0a6BiqLIzIFiOZnPukrqKXFFG26xMpv66Z+ixUpRFFaUBdiwqogCv5vCoIeueGrSbS/mfLjZMVYYYGvZZnR14kfXbCRC5OAB0j2TD4HUy8rxb96CXixfJoqFMdMEqwzYAjwFXAP8OblvBU1grWEYHyXXN/1Sp759DHiHYRi7yfVl/JhhGJ8Fzpim+YRhGD8C9gJp4AemaR67xNcRYl5lBwcZ3L9v0sG0emkpwWuuQ/PNfErZuWRZFm+MmzlwZUE1NUUli1IeIebBfNZVUk+JK1KuxSrGqeapEyuAFeW5ySsKArMbYzWZRCbBqb6zzuOHaqjwO8dd2ZZFouEcsRPHJ71JsBYIEti6DXdV1WWXSYjZmOksgjZw1DCMV4FXTdP8S8MwXMA24GngVuCvgasupRCmaVrkKsKxTo5Z/6/Av17KsYVYCLZtk2hoIHb86IQ3eUVR8K3fgM/YuKizE51ob6Ev4Zyedpe0XollZD7rKqmnxJXGtm3auqOcau5nIDp1K1RNWYCNq4rnJLEadrL3NBkrM7LsUl1sLF7v2CYbixHZ/xbpnp4J+yuahm+DgW/tOhRNJpIRC29Wk1wAfwX81DCMvwb2A5uAvaZp/umcl0yIJSIbiRA5dJB0d9eEdarXS/Cqa3BXLE6XwGGWZfF6k7P1qragilppvRLLk9RVQlwiy7Jp6YpwumWAwdj0iZWxqpjCOUysAHrifbRGLjhiG4rX4XWNTvmear9A5MD+Cbc9AXBXVhLYvlPuYyUW1awSrKH7e/yGYRg3A9cCe4CH56NgQuS7i3VN8FTXENixE9XjmWTvhWV2ttKX6HfEbl4trVdieZK6SojZy1oWTR0RzrQMEE1MvDkw5HpkVJf65yWxgtzEFsd6nF8Ghtwh6gpqgVy9Gzt+jPjZMxP2VXUd/9ZteFaukinXxaKbbQsWAKZpvkZuwK8QV6RMfz+RwwfJ9E0ca6XqOoFt23HXrsyLN3nLsnj9vLPCqglVUlsy/T1EhFjqpK4S4uLSGYvG9jBnW8MkUplJtxmZvGKOxlhN5Xy4ZZKJLTaiKipWIsHgm6+T7u2dsJ+7spLAjqsWbYyzEONdUoIlxJXKSqeInThBsnHyKWDdFRUEduxE8wcWoXSTO911gZ64MxG8SVqvhBDiipZIZTjXFqbhQph0xpp0G1VRWFEeZMPKQkKzvEHwbKWyaU73Oye2qA3VUOItJhMOM/j6HrIx573jFFXFv2kz3rXr8uILTSGGLd6IeyGWENu2STY30f/iCyQazk2cfl3XCV51NaEbb8qr5MqyLPY2HnfEaoIV1JWWL1KJxHx7+uknede73r7YxRBC5KnBWIoDp7t49s1mTjX3T5pcqarC6uoC3n5tLdcY5fOeXAGc7W8gnR3tmqipLozi9aQ62gm/+vKE5Erz+Sm4+VZ869ZLcpXH3v/+d/Poow8tymt/73vf4eMfv2PK9ffcczef+tQn5+W1Z9WCZRjGo6Zpvm9c7AXTNKU2F8tWuqeb6NGjk069DrkbFwa2bUf1ehe4ZBd3pqud7rizO8VN9VsXqTRCLAypq4Rwsm2b7oEEZ1sHaO+NTbmdS1NZXRVi7YpCfJ6F6+QUS8doDDc5YuuK6qGljcHDhyZ8qemuqCB4zXWo7vlP/MTl+e53f4BvkbpufuhDd/D+9//horz2jP56DMP4KbATWGEYxrkxq3SgafK9hFjaspEIsePHSF5om3S9FggS2LYNd2X+3l9j73ln61VVsFxar8SyJXWVEE5Zy6KlM8q5toFpp1r36BpragpYXV2AR1/4ac1P9p3Bskdb0rwuL9XdaSLHJt5OzltfT2Dr9kW97YmYueJFvLmz3+8HFmc2yZl+PXEnUAJ8A/jMmHgG6JjjMgmxqLLxOHHzJMnmJmxrYtcJRdPwrVuPb/2GvL6/xpnOC3TFnPcHkbFXS8+FC2184AO/zyc+8ef85Cc/ZvPmLXzsY5/knnv+DdM8SUVFJe9+93v50IfuQB33gWN43x/84EHWrFkH5LoQfutbd/PUUy8sxunMtzuRukoIYokMje1hzrcPkkxPnOV2WMCrs3ZFIasqg7i0xUlY+hL9XIi0O2LrejXi553JlaIo+Lduw1u/RroE5qHHH/8pDzzwAzo7O6iuruGOOz7GO995G+9//7v50Ic+wvve94fYts33vvcdHn/8p6RSSd71rt/n9OlTvPOdt/F7v/du/vEf7yIQCBCLxXjxxecIhQr41Kf+GpdL41vf+gb9/X3s2nULf/d3/4Cu6wA8++wv+NGP/pvm5mYqKyv56Ef/J+98521Arovg7t2v8r3v/RCAvXt38+1vf4OWlhZuuGEXRUVF83Y9Znqj4TAQHvp28Dcn2eQHc1oqIRaBlUgQP3OaRGPDpNOuA3hWrMC/aQtaIH/GWU1lz7jWq4pAGfVllYtUmvzS1R/n8Nmeae/xMl9Cfjfb15ZSXjS7LhO7d7/K//2/99LT083nPvcp/uiPPsIXv3gXLS3NfO1r/4xl2dxxx53zU+glQuoqcSWzbZuugQSNF8K098SwJpmIaVhRyMP6FYVUlwVQFzFZsW2bE72nxgYovRAh0BN1bKdoGqFrr8ddlb89RubDYtVVs62nTPMkd9/9Vb785a+wbt0Gdu9+lX/6p39g69btju3uv/8+fvazR/jCF75ETU0N3/72Nzl4cP9IQgS5RO3OOz/Bffc9yA9+cC9f+cqXWbNmLV/+8r+QyUT5zGc+wy9/+TS33fYenn32Gf75n/9fPvOZz3Httdfzxht7+Jd/+T8UFhZx0023OF67qek8X/jCZ7njjo/xjnf8Li+//BLf+c632LFj1vedn5HZdrB925jnOnAr8DJSaYklLBuPkzhzmsT5xikTK1dxMYEt29BLSxe4dJfmbFc7ndFuR0xar0YdOtNNJD75fV7m22AsxaEz3fz2tStntd8HP/ghVq2q47nnfsHGjZu5885PALBy5Sr+7M8+xTe+8dUrPsEa421jnktdJZa1ZDpLc0eExvbwtO9rw/ewWltTSEmBJy9agTpinY57NHqbu1jRo4Jr9IO9ommEbrgRd3nFYhRxUS1WXTXbeqq9/QKKolJZWU1VVTW33/4BVq5cSVGRs3vgo48+xJ/8ySe45ZbfAOBLX/oyt9/+Lsc2K1as5E/+5OMAvOc9t/PUU0/wiU/8OYaxkfLyEJs2baGhIdcD/KGHHuDd734vf/AH7wdy9WFDwzl++MN7JyRYP//546xbt4GPf/zPAPjIR+5k//59pFLJWV6dmZntjYY/NnbZMIwSYHGmBhHiMmUGwyTOnp2yKyCAFgzi37gJd82KvKiMZmpPo7NrRbm/lLXlV9Y3f8vNihW5G202NJzlwIF9vOMdt46ssyyLZDLJwED/VLtfUaSuEsudbdt09cc53xHhQk8Uy5q6tUp3qdRVhqivKSDg1RewlNOzbAuzb3Radk97LxWtg/iCo3WVquuEbrgRvVTu25jPbrxxF9u27eBjH/sw9fVruOmmW3nXu95NKBQa2aa/v5/u7i42bdo8EisoKGTlylWOY9XW1o489w5NHlZTs2Ik5nZ7SKdzLXoNDef4wz/8sGP/7dt38vzzv5xQxoaGs2zYYDhimzdv4eDB/bM93Rm53CliIsDqOSiHEAvCtm0yPd3Ez54h1d4+5Xaaz4fP2Ji7I/wSG0jb0N1Bx7jWq111m6fY+sq0Y13ZoncRnC2PxwNANpvlbW/7Lf70T/+fCdsEAkHH8mRfCmSnaKVd5qSuEstCJJ6muTNCc2eEWGL6lo2CgJv66gJWVize+KrptEXaiQzdVFjvCRM400ZpcPSDtKrrhHbdjL6IkyQstsWqq2ZbT3k8Xu6++1scOXKI1157hd27X+Hhhx/kX//130e2cblyKYc1xRfa47cba/z44tHX9UyI2bY95RcO43vNTvZac2W207T/ChgungrUA0/NdaGEmGt2JkOypZlEQwOZ8MCU22k+P9516/HW1eX1BBbT2TPuvldl/hLWV9YsUmnyU3mRj7dfU3vxDfNQXV09e/fuprZ2tOvGq6++zK9+9Tx/+7d3ObZ1uXLfVkcikZFYW1vrgpRzMUldJZaTVDpLW0+U5o4IPeHEtNuqqkJNaYDV1SFKC7x52/PCsi1ODd1U2BWOETzZRIEewK3lpl3PdQvcdUUnV7B06qr9+9/i8OGD3HnnJ9i+fSd/8Ref5i/+4n/y0ksvjmwTDAapqKjENE+yfftOIFc3tbQ0X/Lrrl69mqNHD/M7v/POkdiRI4epq6ubsO3atevZu3e3I3bqlHnJr30xs03d7hrz3Aa6TdM8PsW2Qiy6zMAAyabzJJubsNJTf9unBQL41m3As2rptViNdb6niwuRTkfsRmm9WlZuv/0DPPLIg9x991e5/fYP0NHRwVe/+k/ceutvTviWr6SkhIqKSu67714++9nPc+bMaZ5++olFKvmCumvMc6mrxJKTyVq098Zo6YrQ2ReftgsgQMCnU1cZoq4yhMed/18ONg22EE/HURMpQscaUS2bEm8umVIUheDV1y6ZMc8i15Xv+9//LiUlpVx//Y2cP99IY2Mjt932XvbufW1kuw9+8EPcd9/3qKlZQU3NCr773f8gHo9f8hcBf/zHd/L3f/+/qK9fy7XXXs++fW/w1FOP8zd/86UJ277nPbfzyCMP8u1vf4Pbbnsvb765l927X5kwEcdcmW2CtRv4S+C3yE17+4xhGCdM05z+L1+IBWSlU6RaWkg0NU15c+BhruJifGvX466uXtKJ1bDd48ZelflKWF9evUilEfOhoqKSr3/9Hv7jP77JnXd+mFAoxNvf/jv8+Z9/esK2qqryxS/+b+6++2t85CMfYPPmrXzyk3/Jt7/9jUUo+YKSukosOZmsRUdvjNbuKB19cbLZ6btSaapCTVmAusoQpYX521o1XsbKcKa/ASyL4IkmlHSGQk8hupprcQ9s34GnRnpdLCWbN2/lC1/4e+6//7+5++6vUlRUzB/90R/zrnf9Pt///ndHtvvgBz9Mb28P//iPd5HNZnjPe26nqqr6krvq3XLLb/C5z/0v7r//Pr75za+zYsVKPv/5v3W0aA2rrq7h61+/h29842s88shDbNmyjfe+9/2cOXNqkiNfPmX83bGnYxjGfYAP+CG5bhcfBVpM0/yreSndJerqGsy7SrS8PERX1+BiFyPvXep1srNZUh3tpFpaSHW0TzlpBeS+HXNXVeNduw5XScmSqZTGmuw6NfV28/Bh572N3rXxZjZW5X/3gvkif3czk4/Xqbw8dMl/mFJXXbp8/F3IR3N1nZLpLB29MS70xOjsv3hSBVBa4GVlZZAVZT09BAUAACAASURBVAF0V363Vk12nc70N2D2nsZ/phVvWw+qolJXsAqXquHfYODfdOX1urhS/u727HmVDRs2Ujo0aUkmk+G2236br3zl39i58+qL7p+P12mqumq2KeMNpmluHF4wDONJ4OjlFEyIS2Vns6S7Okm2tZFuvzBtF0AA1ePBW7f6/2fvzuPkqOv8j7+6+r7mnmRyX0AlHOEG5fBaRfl5/JZVfqvo7sLPXXFXxBsVdcVdwRX3t+CKu4tEXR0RQW4PPFbkCCAQSEICSeWGnJO5Z3qmz6r6/dE9PdOZSTKT9EzP8X7m0Y/q+tYx36p0z2c+Vd/6fgkuWow3UpmRvcfT0ztL717VhWs5aZauAsqMpFglk5LruvQms7R09NPSkaS9J8VoLnTHwn7mN8aYPytGLDx5egIcq6ydZUf3LgIHOwntawegJliNz/ASaJpDePmKCtdQxtMvf/kQicRPufbaTxMIBLjnnruIxeKcfPKpla5a2Y01wdppmuYJlmVtK8zPBqb/E9MyaTiZDNmDLWQOHMjfqcrljri+x+PB39hIcOEiAk1zpmzHFUezp6ONvb2lvSKev3DFYXveEZnmFKtk0sjmHNq6kxzsTNLSmTxq738DwkEf8xqizGuMURMLTMnWFofa1fMaTk8vVVvzX0evx0tNqAZvNErszLOmxTHK4X3qU5/nllu+ycc+9nfYts1pp53Ov/3bbQQCgUpXrezGmmD5gfWmaT5Bvl37RcB+0zQfBbAs6y1lrp/McK7rYvd0kz3YSubgAXIdHUds/jfAG40RnD+f4MKFeCPRCahpZT19SM+BtaEals+ed5i1RaY9xSqpGNtx6OxJ09adorUrSWdvGmeUj2NEgj7mNkSZ2xClNj45BgMul6ydZWfnrvxzV4WmkDWhanw+P/FzzsOYhn9kS6mGhgZuvPFbla7GhBhrgvXPh8z/a7kqIjLA7u8j29aWf7UexEkduVvaAUYwSHDefALz5uOrrZ1WgelI9nV1sLtnf0mZ7l7JDKdYJRMmZzt09qZp707R3pOioyeFfZRe/4aqjgZoqo8ypz5CdXR63Kkayc6eV/Ht2ou3Px/TvYaXmmA10dNW4qupqXDtRMprTAmWZVmPj1dFZGZyXRcnkSDb2UGuvZ1supfulo5Rb2+EQgTmzCU4Zy6++vpp0RPgWB367FVNqJoVM7hjCxHFKhkvruuSTOd4dX8P217toLM3RXciM+o7VJDv/a+hOszsugiz68JEQ1P3marRythZXttrEd3dWiyrCdYQXriY0KLFlauYyDgZ60DD91mW9d5Dyv5gWdaflbdaMl05mQy5ri5yXZ3kOjvIdXTiZNLF5d7o8FG5D+WrqsI/u4lAUxO+2qnZC2C57Ovu5LVD7l6dt0B3r2RmU6yScnBdl1TGpiuRpiuRKUzTpDM20WiQvr700XdSEI8EaKwJM6s2TEN1CJ93Zv2O3tm5i+DmXVBIRL2Gl7rqWURPPa2yFRMZJ6NKsEzTvB84A5hnmuaOIYv8wGvjUTGZ2lzXxUmlsHu6yXV3Y3d3k+vuwu7rG/O+PD4f/oZG/I2zCMyejTc6/Z+pGq1ndr5c0gNVdaiKU+YsqGCNRCpHsUqOVc52SCSz9PRl6OnP0J3I0NOXIZ21j2l/4aCPhuowjTUhGqrDRELHNs7PdJCxM+x7ZQ2hRLJYVhOsoer0M/XclUxbo/3GXwnUAd8Grh1SngNaylwnmUJc18VNp8j19mL3JrATPdg9vdi9PTiZzDHt02MY+Grr8Dc04G9szN+l0h2ZYVp6unm1e19J2TnzdfdKZrQrUaySI0hnbfqSWRLJLL39WXqTGXr7s/SncqPqLv1womE/DVUh6qpC1FeHiIZ8M7p1xVA79m0iuGuwpYXX8DJryQoCczSMiExfo0qwLMvqAXoKVwffOMIqPy5rrWRSGbgb5fT3Yff14fTlp3ZfAjuROGpX6Udj+P34amvx1dUz68SF9DgBPMc4qvdMsnrHxpI/CKqCcVbOXVjBGolUlmKVOK5LKm3Tn84nTX3JLH2pHP3pHIlklswx3pEayus1mFUbwVcXpi4epDYeJBRQzBpJOpehbd3zeIf0/lsXb6Rq5RkVrJXI+Bvrb4Q3DXnvBy4GnkBBa8pyXRc3l80nUMkUTrIfJ5ksvuz+fpxk/6i6Rh8Nj8eDN16VT6hqavHV1uKNx4t3qEKNcXon2Sjdk9H+rs4R7l4t190rkbw3DXmvWDVNOK5LOmOTytikMjlSGZv+dI5UOleY2iTTuTF1OHE0Xq9BdTRAdTRAbTxITSxILOJn9qwqWhWrjsqy1uBt7yrOew0vc86+ECMUqmCtRMbfWHsRvGrovGmadcDdZa2RHDfXdXEzGdxMBieTwUmncTNpnHT+5abTOOkUTiqNk0ri2sd/RW8kHp8PX7wKb1UVvpoavFXV+KqqdHeqDP7wynpcdzDpjQdinDZvUQVrJJWwf/8+Lr/8Pfz4xz9j6dITKl2dSUOxampwXRfbcclkbdJZh0zOJp2xyWQd0lk7/8rYpArTdNY+rmZ8RxMN+YlH/VRFAlQVkqpo2I+hpn7HJJ3LsPf5Z0vK6uYsJrJgcWUqJONq27at9PR0c9ZZ5xz3vi666By++c1buPDCi8tQs8o43r90E8DiMtRDhnAdB9fO4WZzg9NsFjeXw81l8++zWZzswPsMbiaLU5i62cy4BqFDeXw+vLE43ngcbyyGLx7HW1WNEYmoDfo4aE30sLX11ZKyc+Yvx2d4K1QjkUlPsWocOK6LbTvkbJdcYZq1HXI5h2zOGXxvO2SyDlnbJpsdnM/kbJwxjBdVDl7DQyzsJxr2Ewv7iUcChal/xvXsN95e3boGuhPFea/hZf5ZF+rvgmnqi1/8DO9//wfLkmBNB2Ptpv2PwMBvQwNYAvyq3JWaTFzXBcfJN5FzHHAdXNsplNm4jguOXSgrnbqODbaNa9t07Q2SaO8tzru2nX92ybbzSVRh3s3lxu2O0vEw/H6MaAxvNIIRieKNRPHGY3ijMTzB6TXa/GT31I6XS5rAxAJRVs5fXLkKiUwyMzFWOa6bf17WAcdxcdz8y3ZcXCc/HSgfeG8PzNuDZXs6knR09hXLco5TfF9Mppz81LbL03S83IIBL5Ggj0jITyTkIxryEQ35iYZ8hIPqfGIipLNp2jesZegIX7ULTyRUP6tidZLxNZEX9qeCsd7BumHIexdosyzrleOthGmaBvAfwOlAGvhby7K2DVn+d8DV5HuC+rplWb880v5an34GHDffhMp1S16u64AzpMxxiklU6XJncFkZZMIB+pPH1qveePN4vXiCQYxQGE8ojBEK5afhMJ5wJD/1D/6adMn/R+QGZlLH18nFUMG+DIlktmz7m266kwl2dO7B5xu80nr2PFN3r8aoLdnBxvZN9GXGPmzA8YoGopxav4KGcN2ot2lvb+PWW/+VP/3paSKRMK9//UVcc82nhq33vve9mw984EO8971/CczoJoQ3DHlfllhV7jgF8Nymlnwowi2EpPzUOWRaLHcKSVShzHHdQhlle+5orOM7TbSA30s44CUU8BEK5qeRoI9w0EsomH+vO1GV9+orz0J/Egqdf3i9PhaedVGFazX1VCpWjTVOXXPNRzhwYD+33PIt/vjHP/Cxj32C//qv23jllY3Yts2yZSdw7bWf4bTTTgfyTQCvv/6r3Hvv3bz66k6WLj2BT3ziM5x66sriPjdvfoX//u872L59G/PmzefTn/48l1zypvE43HEx1gTrCeCjwJ8Vtv2jaZqbLcs63stYfw6ELMt6vWmarwP+H/C/AUzTbCLf3e45QAhYbZrm7y3LOmwE2PLCpuOsTvn5/T6y2fIlIkfjeH24Xj+Ob+AVwB147w/g+AL5qT+Ia3hh4IpejnxjmgRAFuguvCbGZA/uldbqvFp49ir/B0TUH+WMBUsrW6kpaGPbJvqyE59cAfRl+tjYtok3Lbhw1Nt86UvX0dDQwO23/4BMJsN3v/ttvvrV6/nsZ78wjjWd0sYjVpU1TgHsa6vMZ3Ay8RoeAn5v/uUzCBbeB/0GoYCPQGEa9HsJBgy86shn0suk+ul4eX1JWe2yFQSraipUo6mrUrFqrHHqppu+xZVXXsFll72Pt73tUj784Q/yjne8i+uu+xL9/X3cdtutfOtbN/HjHw8+CnvHHf/JddddT0PDLG655WZuvvnGkuUPPngf11//j8yfv5DbbruFr33ty7ztbU+W/VjHy1gTrJuBE4EfAB7gKmAp8InjrMdFwG8ALMv6k2maQxtwngc8VQhUadM0twErgeeP82dOWi4eXK8X1yi8vL7CfH6aT57yL6c49eP6ClOvDxSEpp2sm6bXbSspO0t3r6a9F19cw/btW/n3f/8vAoVBOb/61a/z539+Kclkf4VrN2mNR6xSnBqBz2sUXh58PgN/cd7A7yu8hr73GQR8Bn6fl4Df0N2maWjXhqdxM4PXFgy/n4VnjP6Ckkw9VVXVGIZBJBIhGAxwxRV/zV/+5QfxFTo1u+yy9/GP//jFkm3+4i/+D69/ff6u5hVX/BVf+MJnyGQyxTj3wQ/+9ZDlf8PHPva3dHR0AFNjcOqxJliXAGcOXAU0TfNXwIYy1KOK0tsktmmaPsuyciMs6wWqj7Qzn89b3jbWhgfXY4DHwDWM/N0ej4FreIaUGfk7QYaRX9cYUu71YhemA+u5hgEDSZTXC4Vpfh9D7igdhqfwmo6hKRoNVroKk9L+7B58OYOB//W6WBWXnHU6fq8SrCNpbIwPK3tj+Bxe3L+RnnRihC3GV1UwxllzTqUxNrxeI2lt3UsqleJd73rrsGUvv7wWgNraKI2Ncbxeg1gsVDzmdDpasvxIjrZ8ihmPWFXWOAUQDvvLOrSC1/Dkh8IwPBiFl3fo1OPB6/VgGAbeQpnX8OD1Fua9HrxGPlkaKPMNSZgGkqjie68xo59nmmbfmbJIJXro27kZ/5BxwRacfQ7zF82pYK2mhskUq8Yap4Bi/DnppEXMmvUhHnroITZv3syuXbvYtGkTjuOUHOOpp5rF+blzGwGorQ0TiUQAWLHixOLyhQtnA5BKpZg3r74sxzjexppg+ciPKZIeMl+OHhl6gKH/i0YhaI20LA50cQSpZafg4ikkQvmX6ymkJCVlRunygcSpMB267vGKxUIkEqmjrueBwefDZqDRnqeZJu0k6c4dxFP4WMRiQS6YdzJdHbqDcSSNjfERx6rxEOTsmrMrUKOCJLQmRzeGTnd3H01Nc7jllu8OW5ZO538Vd3b20drai+O4JBKp4jG3tvaULD+cw52nSjrOP17HI1aVNU4BnLqoFo/HUwgzg1MDMArJ0tBlhseDMbCOMeT9kPWO14ifBdeBnEMuBzm14AYm53dmMtj61G9J9Q/G8GBVlIYlZ+lcHcWkjFVjiFMAtu2QSKTYtGknH/7wh1i8eAnnn38BF1/8Z3R2dvJP//TlkmPs788V57u68n/LtLb2Eonkf1UnEpni8s7OfDNJ13Un3WfpcLFqrAnWneTbst9VmP8A8NPjqNeAp4B3A/cU2rYPvdL4HHCjaZohIAisADYeaWdvvGTydRGpX8ajo/M0shcPvgSJquJ8U109J1YvqGCNZKIsWrSEtrZWIpEIdXX5K3cHDhzg1ltv5uMf/3TJuj6fj0Ri8Ernvn17J7Suk8h4xKqyximAeY2x46ySyOSR6u6kY3vpM/BNZ5xNIBiuUI1kIg1c4Hn88Ufx+/3ceut/FMvuvPNHQD5Bmil3vcc60PBNpmm+SP7BYQO40bKscnR9+wDwNtM0n6bQXt40zU8D2yzLetg0zX8Hniz8zC9ZlqVbHDJj9GR62Z84UFJ2yqyTMDLTsYGoHOrcc89nyZKlfPWr1/Oxj30Sr9fLrbd+i56e4Z3PrFhxCg89dD+vf/2FZLM57rjjP2ZMMBtqnGKV4pTIEex+cTWOM+RGcTTC8jMvoqd7cvagLOUVDofZtWsXJ598Ch0dHTz99GqWLl3G2rUv8KMf/QCATCZDMDgzHgMZ80DDlmX9hsKDvuVSaCf/0UOKNw9ZfgdwRzl/pshUsaVjW8l8VTDO/Ko5tLVN/PNDMvEMw+Bf/uXf+Pa3/5Vrr/0ohuHhzDPP4Stf+edCj5KDPvKRf+Ab3/hnPvrR/8vs2U1ce+2n+cIXPlOhmldWuWOV4pTI4aVaW+jcXRqr6k87k2AgCCjBmgkuv/wDfOc7t7Bx43re854/56abbiCbzbFkyVI+//kvccMNX8KyNrFy5RmVruqE8EzHgcFaW3sn3UGp6dvo6DyV6kx18fS+50rKzmk6k1MXLdV5GgV9nkZnMp6nxsb4tL/1plg1dek8DXJdl22/e4C2/TuKZU51nHPefRXz59TrPI2CPk+jMxnP0+FildoYiUxSruuyuWNrSVlNqJpZ4YYK1UhERKRU/87tdLa8WlLWsPJsgr6p0Z22yHgYUxNB0zR9wNuBOgod3gFYlvXjMtdLZMY70H+QjlRnSZlZe8KMfKZGZCwUq0Qmht3fz761z2APefYq11DLksUrK1grkcob6zNYPwUWAZuAgaYNLqCgJVJGtmOzqWNLSVljpIGG8NQY/0GkwhSrRMaZ67r0rH+Rzr62wTKvl8YzzyPg1d0rmdnGmmCttCxr+bjURESKdva8RjKbLM57PB5OrjMrWCORKUWxSmScZfbsofW1rSV3r9LL5rG0SV89kbE+g7XJNE0Nxy0yjlK5FNu6dpaULapaQCwQrVCNRKYcxSqRceSkUvRsWEdXenA87Vx1lLnmmQS8/grWTGRyGOsdrAhgmaa5EUiRb9vuWpb1lrLXTGSGsjq3YTu54rzf6+fEmmUVrJHIlKNYJTJOXNclsX4tnb2txbtXrmGQMhexpGZxZSsnMkmMNcG6aVxqISJAvlv2Pb37SspOql2mK4IiY6NYJTJOklu3kNq/j67U4GDnyUWzWTTHVKwSKRhrgvUE+YEW/6yw7R+B75S7UiIzkeM6bGjbVFIWC8RYGJ9foRqJTFmKVSLjINvaSnLzJrrS3dhu/u5VLh7BXjiHJVULK1w7kcljrAnWzcCJwA/IN7m4ClgCfLLM9RKZcXZ0v0pvpnQAvZPrTAyPhqsTGSPFKpEys5NJel94npyTozOVf/bK9ftIrFjICTWL8OvulUjRWP9yuwT4C8uyHrYs6yHgfeTHGhGR49CX7Wdr546SsnmxOTRG1C27HLtdu3byN3/zft785tdz++3fpbOzk9/97jej3v6aaz7CbbfdOo41HDeKVSJl5DoOiReex0mn6Ux14bgOeDwkzAV4o1EW6+6VHKPjiVM33ngDX/7ydeNcw2Mz1jtYPsAPpIfM24dfXUSOxnVdNrZtwnEHv0p+r58V9eqWXY7P3Xfficdj8JOf/Jx4PM5tt91Kf38fl1zyjkpXbbwpVomUieu6JNatJdveTs7J0ZXuASC5cBbZujjLq5fo7pUcs+OJU5/4xGdxXfeo61XCWBOsO4E/mqZ5V2H+CuCuI6wvIkexJ7GftmR7SdmKOpOgBmqU49Tb28sJJ5zIvHn55/gmayAaB4pVImXSv+kV0rtfA/IdMbmuQ7Y2TnLhLIK+IIurFlS4hjKVHU+cisVi41Wt4zamBMuyrJtM03wReAv55oX/bFnWr8elZiIzQCqXYlOHVVLWEK5nfkxD+IynbGsriQ3rsXt7j75ymXnjcWKnnY6/sXHU2zz00P389Kc/5uDBFubMmctf/dVVXHrpu0gkEtxxx3/w+ON/pLe3h9NPP4tPfvIzLFy4mGuu+Qjr1r0IwG9+8ysuvfRdPPLILwG46KJzWL16Dddc8xFWrjyDHTu28Nxzz7NgwQKuueaTnHvu64bV4fvfv52nn17N97/fXCy75pqPsHz5yVxzzSfp7+/jW9/6Bs8++wyZTJqVK8/kU5/6HAsWTHzTIcUqkfJIbt9GcusWALJOlu5MD04oQGL5AvB4OLFmKV7DW+FaTl+VilWTKU6tX7+W2267he3bt1NdXcXb3vYOrr76GrxeLzfeeAPJZD9f//rNAPzhD79n1ar/pKWlhXPOOZe5c+fR19fHl750A7/+9S+47757eMtb3so99/yUXC7H+edfwOc+dz3hcLjs53BUz2CZpvm9wvSPwHXAucDZwOdM03y07LUSmQFc12V968tk7WyxzPB4ObVhBR6Pp4I1m/4SL62rSHIFYPf2knhp3ajXt6zN3Hrrt/j4xz/FXXfdz/ve935uuulr7N79Gl/+8nW8+OIabrjhRr73vR8RDAb41KeuIZVKcdNN3+KCCy7mLW95Gw899Bs+8YnP8pa3vI0LLriYhx4abN9+113NnHnmmfzwh3fyutddyHXXfYrdhavVY/G97/0n+/bt5TvfuZ0f/OAnGIaHb3zjn8a8n+OhWCVSPum9e+h/eWNxviPZie330nPaEly/j4g/woL4vArWcPqrVKyaLHHKtm2++MXPcsEFF3PnnT/n5ptv5uGHH+SRR34xrA4bNqznn/7py1x22eX893/fybJlJ3LfffeUrLN9+1bWr1/HLbf8B9dd92Uef/xRHnzwvuM+XyMZ7R2s2wvTG8alFiIz0K6e3cOaBp5Uu4yoP1KhGslkdODAfjweg9mz59DUNIe/+IvLWbBgAR0dHaxZ8xyrVv2Y5ctPBuAf//HrvO997+J3v3uE97znMgIBP8FgkPr6BgCCwSCOYxfnAVauPINrrrmG1tZe/v7vP86zzz7DL37xIP/wD9eOsZ77iEQizJkzl0gkwhe+8BX279939A3LS7FKpAzSe/eSePGFYnOttJ2h2+mn97QlOOEgkI9X6uVWYPziVE9PNz093dTXNzBnzlwaG01uueU2amvrhtXhvvvu4cIL38D/+T8fAODqqz/GCy88X7JOLpfj85//EvX1DSxduozzz78Ay9o0bF/lMKoEy7KsFwrTx8elFiIzTG8mweaOrSVldaFallSrJ6aJEFt5Bn0b1pOrwJVBXzxO9LTTR73+6173ek477XSuuuoKlixZygUXXMw73/lutm7dgt/vxzRXFNcNh8OceKLJzp07jrDHUitXnlEyv2LFKezcuX3U2w/40Ieu4rrrPsm73vU2zjjjLN7whjfy9re/c8z7OR6KVSLHL737NRJrXyx5FqYt1UFi+ULseP4CYDwQY050dqWqOGNUKlZNljhVVVXNX/3VVdx884384Aff481vfhMXXfQWVqw4Zdi627dv5ZJLLi0pO/XU0+gdcu4ikWjJBcZoNEoymRz1cY7FmJ7BMk3zPsuy3ntI2R8sy/qz8lZLZPpyXId1rRtLeg30GT5ObzxVVwMniL+xkZq3vLXS1RiVYDDErbd+lw0b1vPUU0/y9NNP8vOf/4yvfe3GEdd3XXBdZ9T793pLn59wXQfDGP45HKnZqm0PfoZPPfU07r33YVavfpJnn32K22//Dx544N5Ck5DgqOtTDopVIscmtWsXfS+tK0mu+nNJDi6tI1sXL5adVHuC4tUEmCqxajzj1NVXf4xLL30Xq1c/zpo1f+K++z7G3/7tR/nrv/6/Jev5fD4c58j79PtHSnvGp/On0T6Ddb9pmjuAd5qmuWPIazcQGpeaiUxTVuc2egrd3A44pX45EX/5H7KUqe/FF9fwox99n5Urz+Dv//7jNDffg2maPPjg/WSz2ZLmDalUiu3bt7Bw4eIR9zVSkrRlS2knK5s2vcIJJ5w0bD2/309fX6I477puSRPA5uYf8vLLG7nkknfwla/8M7ff/kO2b9/G9u1bh+1rvChWiRwb13VJbt1KYv3aYb247VlURWZWbXG+LlTL7MjoOz+Q6W+84tSBAwf413/9Bk1Nc7jiir+mubmZD37wb/jd7x4Ztt3SpcuwrM0lZZs2vVKeAzwGo72DdSVQB3wbGNowPwe0lLlOItPWgb6D7OjaVVI2J9bEPPUaKIcRCoX44Q/voK6unvPOex2vvrqLXbt2cc01nyQUCnLTTV/jM5/5AvF4nB/84A4Mw8tb33rJiPsKh8Ps2LGdAwf209SU/8ytXv04P/nJT1ix4gx++csH2bdvD+9+92XDtl2+/GRuv/27PPDAvZx33uu499676ekZvFBw8OBBfvvbX/P5z3+Z+voGHnnkl0SjURYsWDQ+J2ZkV6JYJTImruPQ99J6Uq/uKin3GAaJ5Qvp9LaWlK+oP0kdMUmJ8YpTtbW1PPbYHwB4//s/xMGDLs8//2zxea6hLr/8A1x99VX8/Oc/4/zzX8+jj/6eDRvWV6QnWxjlHSzLsnosy9oF/CVwFvAG4I3AJcA/jlvtRKaRvmw/61s3lpSFfCFOrVevgXJ4J598Kl/4wle4++47ueKK9/LNb36d97//g7zzne/hi1/8KitWnMLnP/9prr76KtLpFLfd9j2qqqpH3Nell76bjo52PvjB99He3gbAW9/6dv7nf/6HK6+8grVrX+SWW75LU1PTsG3POec8PvShK7njjv/kwx/+EIbhKQmQ//AP13Laaadz/fWf40MfupwXXnieb33r28Tj8WH7Gi+KVSJj42Qy9Dzz1IjJVfjss9kaLG1tMTc2h5rgyL9fZOYarziVSCT45jdvZfv2bVx11RVcddVVLF26jE9+8nPDtlu+/GSuv/6r3HPPT7nyyg+wefMrXHzxG/H5xjrkb3l4xjKgl2ma9wO1wAnAk8CbgdWWZV0+PtU7Nq2tvZNuNM3GxjitrZXpFnoqma7nyXZsntr3HL2ZwWPzeDy8bs451IVqj7DlyKbreSo3nacjGxjH6mtf+8qkO0+NjfFjvuqgWHXs9J0ZnelwnnLd3fSueQ47kSgpN/x+Yueexy5fL1s7Bzu8MTxe3jj/gjE1Z58O52ki6DyNzpHO08svbyQcDrN06bJi2ec+9wlOPvlUrrrq78azTiPGqrE+obiS/MCNDwA3AxcCi4+rZiLTnOu67BreCAAAIABJREFUbGjfVJJcAayoO+mYkisROSrFKpHDcF2X1Ku76H7y8WHJlTcaperiN2DXVrGje1fJsiXVC/WssExamzZt5LOfvZZ1617kwIH9PPTQ/bzwwhre+Ma3VKQ+Y02wDlqW5QKbgZWWZe0AAuWvlsj0savnNfb2lo4HNCfWxOIqdckuMk4Uq0RG4GSzJF5cQ2LdWtwhvYAC+Ovrqb74jfjiVWzq2ILtDC4PeAMsq148wbUVGb3LLrucN7/5rXz1q9dzxRXv46GH7uPrX/9myR2tiTTWhokbTdP8DvCfwJ2mac4F9PCIyGG09B3klY4tJWWxQIyVDSfruSupuNtu+16lqzBeFKtEDpFtayWxbi12X9+wZaGFi4iuPB2P10tbsp39iQMly0+qXYbf65+oqoqMmdfr5eMf/xQf//inKl0VYOx3sG4H7rYs6xXgq8Ac4Iqy10pkGuhO97C2dWN+wIcCn+HjrFkr8RmVeehSZIZQrBIpcHM5Ei+to/up1cOSK4/PR/yss4mdeRYerxfHddjYXtrVdXWwigXxeRNZZZEpb6x/5d0OBE3T/AnwU8uyHh6HOolMeclciudb1mI7uWKZx+PhzFkriQdiFayZyIygWCUCZFoO0PfSeuz+/mHLfFXVxM85F++Qnj53dr9GX2ZIEubxcGrDCg0qLDJGY/rGWJZ1DnAZ+bbsvzZN84+maf7fo2wmMqNk7SxrWtaSzqVLyk+pX86sSEOFaiUycyhWyUxn9/XR8+yf6PnTMyMmV+Gly6i++A0lyVUyl2Rr146S9RbE5qpbdpFjMOZLEpZlbQP+DfgGUAV8sdyVEpmqck6O51vW0pMu7TFwSfUiFlUtqFCtRGYexSqZiZxslv7Nm+j64x/IHNg/bLk3GqP6oouJnrYSzyHjA73SvqWk1YXf62d53YnjXmeR6WhMTQRN07yMfDv21wG/AD5uWdbT41ExkanGdmxeaFlPZ6qrpHx2pFFBSmQCKVbJTOPaNqlXd5HcYuGk08OWezweQkuXEll+8rDECmB/XwsH+lpKyszaEwh41fmmyLEY6zNYHwKagSssy8qWowKmaYaBnwCzgF7gbyzLaj1knYeBeiALJC3LurQcP1ukXBzXYW3rBtqS7SXltaFazpy1Uu3XRSZWWWOV4pRMVq7jkN6zm6S1ecSmgAD+ujqip52Or6ZmxOUZO8PGtk0lZTXBanVsIXIcxpRgWZb13nGow98DGyzLusE0zfcDXwY+ccg6JwCnFMY1EZlUHNdhXetGWvoOlpRXBas4d/YZeA1vhWomMjONQ6xSnJJJxbVt0rtfI7l1y2ETKyMUInryKQTmLzjisCAvt1tk7Mzgdh6DlY2n6MKgyHEYVYJlmub3LMv6iGmafwSGBQ/Lso5nmOSLgJsL7x8BvnLIz54N1AC/ME2zBvgXy7J+eaQd1tZG8Pkm3x+1jY3xo68kU+o82Y7NM7tfoNvtJBINFsurgjHevOQCgr7xa14xlc5TJek8jc50OE/jGKvKHqdAsWqqq8R5stNp+rbvoHfLVtxkkpAHGBJ7oND1unkSVSuWY/iPPHbVvp4DdLkdJfHr1FknsXTWnLLVWZ+n0dF5Gp2pcp5Gewfr9sL0hhGWjfpqnWmaHwYOHQGsBeguvO8FDu2uJgD8P+DbQB3wlGmaz1mWdZDD6Owc+WpOJTU2xmlt7T36ijPcVDpPOSfHCy3rhzULjPgjnFJ/Cj2daWB4W/hymErnqZJ0nkZnMp6nYwyixx2rJipOgWLVVDbR58nu7SW5czvp117Dte0R1/EYBsFFi4mcZJINhWjvSgGpw+4zY2d5Yu+akh5vq4Jx6pldtmPT52l0dJ5GZzKep8PFqlElWJZlvVB4e+2hTS9M0/wD8Gej3M/3ge8fsv39wEDt4kDXIZsdAP7LsqwccNA0zbWACRwxcImMp6yd5fmWdXSmOkvKI/4I5zedTcgXqlDNRGaucsQqxSmZLFzbJnNgP6ldu8i2tR52PY9hEFy4iPAJJ+KNRke3b9dlY/umkuTK4/GwskFNA0XKYbRNBO8HzgDmmaY5dJAEH7D7OOvwFPC/gOeAS4EnD1n+VuAa4J2macaAU4FNiFRIfzbJ8y1rSWQSJeWxQIzzm85SciVSIeMYqxSnZMLkurtI795NevdunMzhW0F4vF5CixcTWnYi3nB4TD9jd2Iv+xMHSsqWVS+hOlh1THUWkVKjbSJ4JflmD98GbiTfXCJKvsekZ46zDv8J/Mg0zdVAhnzXupimeTNwr2VZj5im+XbTNP8EOMD1lmW1HefPFDkmXelu1rSsGzaIcFUwznlNZxNUl7YilXQl4xOrFKdkXNnJJJm9e0jv3k2up/uI63rDYUJLlhJctBgjMPaY05tJ8Eq7VVJWFYxzQs2SMe9LREY22iaCPUBPoYnFf1mWdZZpmouA+4BbgO8dawUsy+oHLh+h/Loh7z95rPsXKZcDfQdZ17oB2ylt/14bquGc2WcS8B75YWIRGV/jFasUp2Q8OKkk6X37yOzbS7a9/ajr++vrCS1ZSmDOXDzGsTXjsx2btQdfKoljXsPHmY0r1eOtSBmNdRysjwDnA1iW9appmmcDz3IcCZbIZOe6Llu7trO1c8ewZXNiTZzecIoCk8jkolglk5KdSJDZv5/MgX3kOjtx3SP3vWL4/QQXLCS4aDG+quNvvvdKh0XvIc3bT61fTiwwume3RGR0xppg+SntFi3DGHoRFJlqMnaWda0baO0f3tpnac1ilteeeMTxRUSkIhSrZFJwbZtsezvZgy1kDrZg9x69BzSPx4N/1iyCCxYSmN2ExzfWP9VGtrt3L6/17Ckpmxebw/z43LLsX0QGjfVb+yDwqGma95APVu8DHip7rUQmge50Dy8efIn+bGlXyh6Ph1Pql7OoakGFaiYiR6FYJRXhui52by/Z1oP5V3s7bi43qm19tbUE580nOG8eRmhsnVYcTUeqk41tpf2uRPwRTmlYUdafIyJ5Y0qwLMv6vGmalwNvALLArainJJlmXNdlR/erbOnchuM6JcsC3gBnzVpJfbiuQrUTkaNRrJKJ4rouTl8f2bY2su2tZNvacFKHH3vqUL6aWoJz5xGYO3fUXayPVX82yQst60vimdfwctaslfiN8twdE5FSY/5mWZb1c9M0HwDeC3wSOBeIlbtiIpWQzCVZ3/oy7cmOYctqQtWcNet0wuqGXWTSU6yS8eA6Drnubnra99G7Yw/ZjvYxJVQew8Df0ECgaQ7+pjlj7l59rHJOjjUt68jYmZLylQ2nqEt2kXE0pgTLNM0l5B8evgqoJd8N7rCelUSmGtd12ZPYz6Z2i6yTHbZ8UdUCVtSdpM4sRKYAxSopB9d1cZL95Dq7yHV1kuvsINfVhWvb5KJB0n2HH6NqKCMUIjC7Cf+s2fgbGzH8E9PjrOM6rDu4gd5M6XNfJ9YuZW6saULqIDJTjXag4cuAq4GzgQeAvwLusCzrn8axbiIToj/bz4a2TbQlh3eT6/f6Oa3hZOZEZ1egZiIyFopVcqxc18Xp7yfX3YXd3U2uq4tcV9cRB/o9HI/Ph7++Hn/jLPyNs/DG4xPeGZLjOqxvfZmW/taS8qbobE6sWTahdRGZiUZ7B+s+4B7g9ZZlbQMwTdM58iYik5vjOuzsfo2tXduHjW0F0BCuZ2XjKWoSKDJ1KFbJUbm5HLmeHuyeHnI93dg9Pdg93TjZ4a0XRsPj8+Gvq8NX34C/oQFfTe0xj1NVDq7r8nL7ZvYl9peUVwXjnN54inq+FZkAo02wVpJvarHaNM1dwF1j2FZk0mnpb2VT+xb6sn3DlhkeL2bdCSypWqhAJDK1KFZJkZPNYvf2Yid6B6c9Pdj9/Uff+AiMUIjIgnngi+Cvr8dbVV3RhGoo13XZ3LF1WHfsEX+Ec2efiU+dWohMiFF90yzL2gh8xjTNzwPvAq4EZpum+Svgu5Zl/Xr8qihSPr2ZBJs6tow4rhVAfbiO0xpOJuqPTHDNROR4KVbNPK7j4PT1Yff1YfclsBOJfCKVSIyp84nD8fh8+Kpr8NXU4Kurw1dbhzccpqExTmvr0ce0mkiu67K5cys7uneVlId8Ic5vOpuQWmOITJixdtOeIz++yIOmaTYCfw18A1DQkkmtP5tkS9d29ib2gzt8vFG/4WdF/UnMj83VXSuRKU6xanpxshmcvn7s/v58MtXfNzjt78cd4Xf6sTD8frzV1fiqq/FV1+CtqcEbjU2au1NH4rgOG9peYU/vvpLygDfA+U1nE/GPb2+FIlLqmO8VW5bVCvy/wktkUkrmkmzv2sXu3r3DxrQCwONhYXweJ9WeQNAbmPgKisi4Uqya3FzXxc1kcJL92P1JnGR//tWfxE724/T342QyR9/RGHg8HoxYDF+8Cm9VFb6qKrxV1RiRyJS8wGY7NmsPvjSsQwu/4ee8prOIBcZnfC0ROTw1xpVpKZHtY3vXLvYl9o+cWAF1oVpOrl9OdTA+wbUTEZn+XMfBzaRxUimcZDI/TaWwk8n8fDKJk0ri2sM7GSoHj2FgRKP4YnG88TjeWBxvVX7q8U6PITfSdoYXW9bTkeosKQ94A5zXdJbGuhKpECVYMm24rktHqotdPa9xoP/giE0BAWKBGGbtMmZHZk3Jq5UiIpXk5nI4mTROKo2bTuGk08XkaeD9QLnrjH8njt5wGCMaxRuN5V/x/NSIRqdE875j1ZXu5oWW9aRypc+ahf3h/J0rv+5ciVSKEiyZ8mzHZl/fAXb1vEZP+vAPHYf9YU6qWcbcWBOGZ/oGXRGRscjfacrgpNOknH7S+zvyyVGhzE2nBxOqTPqYuzM/VgN3orzhCEY0gjcSxRuNYhSmHt/M+1Pmtd49vNy2eVgLjXggxrlNZ2l4EZEKm3m/lWTa6E73sLt3L/sSB8g6hw/4EX+EZTWLmR+bq8RKRKYt13VxczncbAY3k8UpTN1sBieTwc1kBt+nB8pKE6ZsNEhf39gH1z0eHp8PbySCEQ5jhPNTbySCEYnk50MhtTYoyNpZXu6w2HtIZxYAdeE6zp51OgGvvwI1E5GhlGDJlJLKpdjX18LexH560j1HXDceiHNCzRKaorOUWInIlOA6Dm42m3/lsjgD74e8nGxmcD6TxcllC8lTdkKa5I2FEQhihIIYoXA+gQqFCu9DhWQqhMfnVwI1Cgf7W9nQtmlYk0CAJdWLWF53omKdyCShBEsmvbSdoaXvIPv6DtCe6jzss1UAeDzMjjSyuGoh9aFaBW0RmbR6nn4qnxxls4U7T9lx6/ChnDyGgScQyCdLwSBGMJS/yxQMFZKp0GDZNOlMopIydpbNHVvY3bt32DKv4WVlwynMjTVVoGYicjhKsGRS6sv209LfSkvfQTrSXUdOqsj3mDQ/NpdFVfOJaJBgEZkCMq0HK10FIN9tuccfwBMMEKyvJlfl4AkGMYJBPIFgIYkKFN97AgFdvJoAjuvwas9utnbtIGsPbwYfD8Q4Y9ZpVAXUE67IZKMESyYF27HpSHWxd/9rbN2/m75s39E38nhoDNezID6P2ZFGNY0QkRnP4/ViBAL5hCngx/AH8glRIFB4788vDwQL0wAev7/Y215jY5zW1sN3FiTjz3VdWvpb2dy5lb7MCLHQ42FZ9WJOrFmK19AdQpHJSAmWVITt2HRnemhPdtCW6qQr1YXjOkSiQfqzR37AuipYxbzYHOZEZ6unJBGZVvJ3k/x4fL588uPz4fH7Mfz+fLk/X2YMJEb+/NQIFN6rSd6U5bgOB/oOsq1rJ72ZkZPcWCDG6Y2nUBOsnuDaichYKMGSCZHKpelOd9OZ7qYj1UV3ugfHHf2zBlXBKpoijcyJNmlUehGZFqpedwEev6+QUOWTKLxeNb+bYbJ2lr19+9nVffjWGz7Dx7KaJSypWqi7ViJTgBIsKbu0naE73UNPppeeTC9dqS6SI/R6dCSGx6A2VMOsSCNNkUY9VyUi005g9uxKV0EqxHVdutLdvNa7l/19B7Cdw1xw9HhYGJ/HiTXLCPmCE1tJETlmSrDkmGWdHH3ZPnozfSQyCXqzCXoyvaRzxzaGSsQfYWntPIKxKPXhOvyGPp4iIjI9uK5Ld6aH/X0t7O9rIZlNHn5lj4c50dmcULNEnViITEH6C1aOyHZs+nNJ+rP99A1Ms/0ksn0jjsUxFmFfiNpQLfXhWhpC9UT8YT1gLSIi00bGztKWbKct2U5rsv2ocdPj8TAvNpdlNYuJ+dUcXmSqUoI1g7muS8bJksql8i87TTKXKryS9OeSx3w36lCGx6AqEKc6WEVtqIa6UA1hX7gs+xYREak013VJ2Sk6U910pDrpTHfTk+k96jAjkL/guCA+j/nxeeq8SWQaUII1zTiuQ87JkbGzZJwMGTtDuvDK2BlSdpq0nSadS5O2MziuU/Y6GB6DWCBGdSGhqgrEqQrE9WCuiIhMC1k7S1e6m0SmL988Pp1/5jhjZ0a9D8PjZVaknvnxeTSG6zXUiMg0ogRrknFcB9uxybk2OSdXeNlknSw5t/DezubnnRwZJ/8+a+fIOlkyTnZUV8vKwePxEPFFiPkjxAIxqgJx4oEYUX9EgUJERKasnJMjlUuTstOkhrTq6M/mp0aLS3/f2Ft4eA0vjeF65kSbmBVpwKdnjUWmJX2zj8BxHVzXxXEdHNzCvIPtOjiui+PahamD49qF8vzLLs7np7uzQTq6EtiujePky3JuDtvJr5tzcvltDteTUKV4PIR9IcK+MDF/hIgvQtQ/+FIiJSIik9HQC5b5aa54MXLo+4ydIetkS1p75JzcEfcdCY6+R794IEZjuIHGSD21wRq15hCZASZNgmWa5mXA5ZZlXTHCsr8DrgZywNcty/rlkfa1pmUdruvi4uYTpMLUZSBhGlzm4haTovz7gW2cst4JimSDx3S1a7z5DB8hX4iQN0jIFyokU/lXxBcm5AspiRIRobxxCqA7PdoOfY4ci9yjLB9xmxHim4uLpy9LRyoxJPy5h9SgMO8O/dlD1nHd4pw75OcMxtzC3EAcxsV1KYnJbuEi5kB8HozTbvEip1u48Fm8kOkMvbg5MD/xFyy9ho+qQJzaUDV1wRpqQzUEvIEJr4eIVNakSLBM0/w28HZg3QjLmoBrgXOAELDaNM3fW5Z12Gylpe/geFV1SvAbfvxePwGvn6ARIODNv4LeAEFvMP/yBQh5g2qeICIyCuWOUwCr9z4zHlU9LpHuyXkxcLIxPB6i/mhJE/mqQJyIP6yLkiLCZPkt8DTw94dZdh7wlGVZacuyuoFtwMoj7WzPYxat63YX5/sPdLPnMYu+/d3FsraXdrPnMQvXyXfyYGdy7HnMov2VfcV1end3sOcxi2RboljWsmYX+1ZvLc5nE2n2PGbRtbWlWNa9vZU9j1lkegbHuNj56CYOPLezOJ/q7GPPYxY9u9qLZZ2b9+frlHYI+oJEA1H2P7GV7pcO0BSdxfz4XGKJIC1PbqemJ8zpjadyzuwz8GxL0bZ6F2+ceyGXLnkrb2h6PXse20x2Sy/nNJ3JysZTcFvSrH749+Q6UtSHa4n5o/z2kV9w110/Kv78rq5OmptX8eyzTxXL1q59nubmVRw8OHh89913Fw88cHdx/sCBfTQ3r2L9+heLZU8//QTNzavo7e0plt155w945JGHi/O7du2guXkVlvVKseyhhx6iuXkV2Wz+QeFMJk1z8yoee+z3xXU2b36Z5uZVvPrq4Pn89a8f4qc//eHg/0F3F83Nq3jmmSeLZevWraG5eRUtLfuLZfff/zPuv/9nxfmWlv00N69i3bo1xbJnnnmS5uZVdHd3Fct++tMf8utfP1Scf/XVnTQ3r2Lz5peLZY899nuam1eRyeT/WMlmMzQ3r+LRR39bXMeyXqG5eRW7du0olj3yyMPceecPivO9vT00N6/i6aefKJb96U9/orl5FQcODH5eH3jgbu67767i/MGDLTQ3r2Lt2ueLZc8++xTNzavo6uoslt1114/41a8eKM6/9toumptXsWnTxiHH8j80N68ilcp3MZzLZWluXsUf/vCb4jpbtmyiuXkVO3duK5b99re/4Cc/+X5xPpHopbl5FU899Xix7KWXXqS5eRX79+8tlj344D3ce++dxfm2toM0N6/ihReeK5Y999zTNDevorOzo1j2s5/9mF/+8v7i/I4d+c/YK6+8VCx74ok/0Ny8imQy//20bZvm5lX8z/88Ulxn61aL5uZVbN8++F3/3e9+RXPzquJ8X1+C5uZVrF79WLFsw4Z1NDevYt++PcWyhx++l5///CfF+fb2VpqbV7FmzZ+KZc8//wzNzavo6Bj8fXD33c384hf3Fef37n2N5uZVbNy4vlj25JOP0ty8iv7+PgAcx6G5eRW///2viuts376lcCxbimW//33+WJzC779EIn8sTz75aHGdjRvX09y8ir17XyuW/eIX93H33c3F+Y6OdpqbV/H884OJwpo1+c9me3trseznP/8JDz98b3F+3749NDevYsOGwVxl9erHaG5eRV/f4O/bSaascQqmfqzKpbLFsr2PW7SuHfys9Lf05I9l3+DvzbaX9rDnMQvHLhxL1s4fy8uD3/3Ens7CsQze3Tv4wqulx9KXP5bOLUOOZUf+WNLdg8ey/5ntHHh28Hdruqu/cCxtg8diHcgfS3Kwc4q9T2yhde1rhP1hakM1hHq8tDyxnZqeCOc2nckb51+If1uO3Y9t4vT6U1hedyINgVoeuPsunnj8D8X9KFYpVoFiFczMWDWhty9M0/ww8KlDiq+yLOtu0zTfdJjNqoDuIfO9QPWRfo7H8ODzGUSi+TbSuZAfw/AQDPqKZV6fF8PwEIkG8RgGucK83+ctrpMJ+jAMD9FIiOqqOIbHoMvvJ5NzmVvfgOHxkgokaffupi5WzQlzFuM1DHa2OaT397J8zlLqGuvxGV5+/+J+/H4/bzFfh2EYtO1v5dFNbaycdxLnnHEuPsPLY12PsblzM+894+1UVVUB8O0nLObWzOIdp14MwGZjM3s2bGNZ0wJOW2oCsG3dBjq9XuY31eP3+0kmkxiGh3DYT2NjfoDCPXtCGIaHmppIsSwQ8OHzGcV5w8jkz0kkUCyLRoMYhofa2sHt/H4vhjG4XTodxTA8xGLBYlkkEsAwPNTVRampyZd5vQbBoK+4TmdnGMPwEI+HimX5enhoaIgTCARIp/P7CYUGj2Xv3vyxVFeHi2XBoA+vd7BOPl/uCMcSLTkWoDifyfTk/8+jw4+lvj5Gbe3Ix9LVNfxYQoXPXUNDnGAwSCaTGXYs+/Yd/Vj8fnvYsWzfzojH4jie4nw223vYY6mri1JfP3CuDAKBwWPp6YkMO5ZweOBYYoTDYbLZ7LBjOXAgfw6qqoYeS367wXl32LHEYiN/NsEpztt2X+FYAiMeS0PDyMfS29ta+GyO9P8SIxKJYNv2sGNpaRn5/2XosYRCDPuexWLBYcfi93uxbW9x3nH6h/2/DHw26+qiQz7DpceSSIz0/zL42YzFYjiOU/hdN1in1tYj/78YhkEikSgcy+D5jccHzkHp/0smM/jZhNRhj2XoZ9Pn8+L3D56D/v7IsN8ZA5+x+voY8XjlBladqDgF5YtV6cJnMxTyD27nNXCNwX2n7fx3zx8Y3Hd/IL+vUDhAKDr4TJHX6ymuQzL/XQ8MqVOv31f8vPgj+eZvHo8H75BjsQvHEggO1snnHzwWw2tgZ/K/p30jxN1gKFByLLZnsE4ZZ+BYBrdLBgrnIOwnPGQ7j2FQX12Nz/CSdIN0evfRWFXHsnnLCHgDbD5gs6ctyYXLzqK+roGQN8B/P7uX2bWz+cuz/hcAW7Zs4cDLu1gyay6nLFoKwEtrFKsUqxSrRjoWxao8z0jtsCuhELg+alnW+w8pfw/wDsuy/qEw/wBwo2VZa4bvJW/Dru2ux+Nh4J/h8VCc93jwYOTLCvMD7w2PgcdjYBTLy3eDTwPojo7O0+joPI2OztPoTMbz1NgY91S6DocqZ5wCuG/t7w4bgD1HPfrDr3C0TT1D1/CUlsfjIXp708MXFyrkGbLu4CJPSZmnsL4H8BTiqGdg6UA5h8blwelgfDaK8/n47MHAwPAMvDwYHi9ej1Gceg0Dr8eL1+MtbjMeJuN3ZjLSeRodnafRmYzn6XCxaio8gPMccKNpmiEgCKwANh5pg6borImol4iICBxDnAJ4w/zXj3e9xmwy/gEjIjLVTNoEyzTNTwPbLMt62DTNfweeJP/M2Jcsy0pVtnYiIjLTKU6JiMhIJk0TwXJqbe2ddAelq4Kjo/M0OjpPo6PzNDqT8TxNxiaC5aZYNXXpPI2OztPo6DyNzmQ8T4eLVZOlF0EREREREZEpTwmWiIiIiIhImSjBEhERERERKRMlWCIiIiIiImWiBEtERERERKRMlGCJiIiIiIiUiRIsERERERGRMlGCJSIiIiIiUiZKsERERERERMpECZaIiIiIiEiZKMESEREREREpEyVYIiIiIiIiZaIES0REREREpEyUYImIiIiIiJSJEiwREREREZEyUYIlIiIiIiJSJkqwREREREREykQJloiIiIiISJkowRIRERERESkTJVgiIiIiIiJlogRLRERERESkTJRgiYiIiIiIlIkSLBERERERkTJRgiUiIiIiIlImSrBERERERETKRAmWiIiIiIhImSjBEhERERERKRMlWCIiIiIiImWiBEtERERERKRMlGCJiIiIiIiUia/SFRhgmuZlwOWWZV0xwrJ/By4EegtF/9uyrO6JrJ+IiMxsilMiIjIakyLBMk3z28DbgXWHWeUs4O2WZbVNXK1ERETyFKdERGS0PK7rVroOmKb5l8BB4GrLst5/yDID2A88BcwGvm9Z1g+OtL9cznZ9Pu94VVdERMafp9IVGKrccQoUq0REpoERY9WE3sEyTfPDwKcOKb7Ksqy7TdN802E2iwLfAf7k95ojAAAKUklEQVQN8AJ/NE1zjWVZLx3u53R29pejumXV2BintbX36CvOcDpPo6PzNDo6T6MzGc9TY2O8Ij93ouIUKFZNZTpPo6PzNDo6T6MzGc/T4WLVhCZYlmV9H/j+GDfrB75tWVY/gGmajwKnA0cMXCIiImOlOCUiIsdrKvQieBKw2jRNr2mafuAi4MUK10lERGSA4pSIiBRNik4uRmKa5qeBbZZlPWya5p3An4As8GPLsl6ubO1ERGSmU5wSEZGRTIpOLsqttbV30h3UZGw3OhnpPI2OztPo6DyNzmQ8T42N8UnVycV4UKyaunSeRkfnaXR0nkZnMp6nw8WqqdBEUEREREREZEpQgiUiIiIiIlImSrBERERERETKRAmWiIiIiIhImSjBEhERERERKRMlWCIiIiIiImWiBEtERERERKRMlGCJiIiIiIiUiRIsERERERGRMlGCJSIiIiIiUiZKsERERERERMpECZaIiIiIiEiZKMESEREREREpEyVYIiIiIiIiZaIES0REREREpEyUYImIiIiIiJSJEiwREREREZEyUYIlIiIiIiJSJkqwREREREREykQJloiIiIiISJkowRIRERERESkTJVgiIiIiIiJlogRLRERERESkTJRgiYiIiIiIlIkSLBERERERkTJRgiUiIiIiIlImSrBERERERETKRAmWiIiIiIhImSjBEhER+f/t3X+MHHUZx/H3YQtFbCuYAomoaCBPIaAQCVJ+CIqA1djKj0gwaFpBIYEgFGKkSqIJiH+oBIVGtPyIFIFWxCAExFSxpIAUSPglPKQmQCrBKAlFghRbzz9mWve2e3fT3tzOHPd+JZfszE6/+8zkbj99Zr47K0lSTaY0XUBEzASWATOAHYFFmflg1zZfBc4CNgKXZuadfS9UkjQpmVOSpG3RhitYi4CVmXk0sAC4uvPJiNgTOA84AjgBuDwidup3kZKkScuckiRV1vgVLOAKYEP5eArwZtfzhwKrM3MDsCEi1gIfBtb0r0RJ0iRmTkmSKutrgxURZwAXdK1emJlryjOAy4Dzu56fAazvWP4XMHOk15k1a/rAWGsdD7NmTW+6hAnB41SNx6kaj1M1HqdCv3IKzKqJzuNUjcepGo9TNRPlOPW1wcrMa4Fru9dHxIHALcBFmfmnrqdfAzqP5nTg1XErUpI0aZlTkqSxanyKYETsD6wATs3Mx3ts8jBwWURMA3YC9gOe6mOJkqRJzJySJG2Lxhss4HJgGnBlRACsz8z5EbEIWJuZd0TEj4H7KW7K8a3M7J7/LknSeDGnJEmVDQwODjZdgyRJkiS9LbThNu2SJEmS9LZggyVJkiRJNbHBkiRJkqSatOEmF5NKRMwG/gzs4YegtxYRMym+Z2YGsCOwKDMfbLaq9oiIHYAlwEcovvj0zMxc22xV7RIRU4HrgL0p7uh2aWbe0WhRLRYRuwOPAsdl5rNN16PmmVMjM6dGZk5VY1Ztm4mWVV7B6qOImAH8kOINR70tAlZm5tHAAuDqZstpnc8D0zJzDvBNit8nDXU68EpmHgXMBa5quJ7WKgP+GuDfTdeidjCnKjGnRmZOVWNWVTQRs8oGq08iYgD4GbAYeKPhctrsCoo/IiiusHr2dKgjgXsAMvMh4JBmy2mlFcAlHcsbmypkAvgB8FPgpaYLUfPMqcrMqZGZU9WYVdVNuKxyiuA4iIgzgAu6Vr8A3JKZj5ffozLpDXOcFmbmmojYk2IKxvn9r6zVZgDrO5Y3RcSUzPSNuZSZrwNExHTgV8C3m62onSJiAfCPzPxdRFzcdD3qL3OqGnNqu5hTFZhV1UzUrPJ7sPokItYC68rFw4CHM/PjDZbUWhFxIHALcFFm3t10PW0SET8CHsrM5eXyuszcq+GyWici3gfcDizJzOuarqeNImIVMFj+HAQ8B8zLzJcbLUyNMaeqM6eGZ05VZ1aNbqJmlVew+iQz99n8OCKeB45vrJgWi4j9KS6bn5qZjzddTwutBj4HLI+Iw4AnG66ndSJiD+Be4NzMXNl0PW3V+R/niLgPOLvtgaXxZU5VY06NypyqwKyqZqJmlQ2W2uZyYBpwZTlFZX1mzm+2pFa5HTguIh4ABoCFDdfTRouBXYFLImLz/Pa5mTlhPhwrqdXMqZGZU9WYVW9jThGUJEmSpJp4F0FJkiRJqokNliRJkiTVxAZLkiRJkmpigyVJkiRJNbHBkiRJkqSa2GBJkiRJUk1ssCRJkiSpJjZYkiRJklQTGyyph4g4ICIGI+LkUbabGRG3l48PiYil2/FaW8YYyzhdY455jB5jDqlTktQss6rnmGaVGjel6QKklvoKsAI4C7hthO12BQ4GyMxHgDO347W2jDHGcbaoY4wehtQpSWqcWbU1s0qNGxgcHGy6BqlVImIqsA44CngA+Fhm/jUiBoDvAycCG4FrgGOBTwN3AVcC38nMYyLi18BNmXlbOeajwNnA14ADgD2AJ4DTgFs3j5GZJ0bEMR3jLAZOBzYB9wLfKOtaDLwB7Ac8CXwxM9/q2IfOMY7ptT1wOHAJ8B/gg8DDFEE3Z/O/Lce6AbgPOKmzzq5jdjzwXWAasAtw8eZ9lyTVz6wyq9ReThGUtvZZ4IXMfA74DUXQAJwCHAEcCBwKLAS+B7zU/SYO3EgRSETEvhRv5jsDb2XmHGAf4N3AZ4Dzeo0REXOBecAhFGfj9qEIPigC51yKEHo/cMIo+zTc9ocDXwdmlzWeM8IYw9U5ANwAzMvMg4H9gd+PUo8kaWzMqt7MKjXOBkva2kLg5vLxrcDCiNgROBpYnpkbMvP1zDwIeHmYMe4C5kTEdIrwWpaZq4AlEXEOxRnEfYF3jVDHscDNmflGZm4ErivXATyVmesy87/AM8Buo+zTcNuvysIgRdB+cpRxhvNP4KqIOBGYkpmvbec4kqRqzKptZ1apL2ywpA4RsTswF7gwIp4HllLM5z6JYnrCYMe2e1NMMdhKOQXitxRn9b4A/DIi5gE3UUx/uB5YBQyMUE733+cA///c5Jsd6wdHGWek7Td2vd7GHuNNHWngMvA+ClwLzAeeKM8USpLGgVllVqndbLCkob4ErMzMvTJz78z8AHAZxXSHVcDJETE1It4J3AO8h+FvFnMjcCHwSma+AHyK4qzi9cCrwCeAd1AERa8x/gCcFhE7R8QUirOVf6xrR0tHRsR7I2IH4MvA3RRn+D4UEdMiYjeKefQMV2dEzAY2Zea9FPP+dymDTJI0Pswqs0otZoMlDbUAWNK17mqKeezPAKuBx4A1FFMnHgRejIitwiQzVwMzgWXlqp9ThNCTFHd9Wk3xgd2/9xojM+8E7gQeAZ4GXgR+MuY9HOol4BfAX4C/AUsz82mKaSNPl3XeX27bs05gEfBsRDxW7uMpNdcoSRpqAWaVWaXW8i6C0iTVefemhkuRJKkns0oTkVewJEmSJKkmXsGSJEmSpJp4BUuSJEmSamKDJUmSJEk1scGSJEmSpJrYYEmSJElSTWywJEmSJKkmNliSJEmSVBMbLEmSJEmqyf8AG5EqOwje/osAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plot_activations(ax, names, s_range=5, y_range=2):\n", " s = np.linspace(-s_range, +s_range, 101)\n", " for name in names.split(','):\n", " phi = eval(name)\n", " ax.plot(s, phi(s), lw=4, alpha=0.5, label=name)\n", " ax.legend(fontsize='x-large')\n", " ax.set_xlabel('Activation input $s$')\n", " ax.set_ylabel('Activation output $\\phi(s)$')\n", " ax.set_xlim(-s_range, +s_range)\n", " ax.set_ylim(-y_range, +y_range)\n", " ax.axhline(-1, c='gray', ls=':')\n", " ax.axhline(+1, c='gray', ls=':')\n", " \n", "_, ax = plt.subplots(1, 2, figsize=(12, 5))\n", "plot_activations(ax[0], 'relu,elu,softplus')\n", "plot_activations(ax[1], 'sigmoid,tanh,softsign')\n", "plt.tight_layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that all activations saturate (at -1 or 0) for $s\\rightarrow -\\infty$, but differ in their behavior when $s\\rightarrow +\\infty$ (linear vs saturate at +1)." ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden", "solution2_first": true }, "source": [ "**DISCUSS:**\n", " - Which activation would you expect to be the fastest to compute?\n", " - Which activations are better suited for a binary classification problem?" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "The `relu` activation is the fastest to compute since it does not involve any transcendental function calls (exp, log, ...).\n", "\n", "The activations that are bounded on both sides only have a narrow range near $s=0$ where they distinguish between different input values, and otherwise are essentially saturated at one of two values. This is desirable for classification, where the aim is to place $s=0$ close to the \"decision boundary\" (by learning a suitable bias).\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Network Layer\n", "\n", "What happens if we replace the vectors $\\mathbf{x}$ and $\\mathbf{w}$ above with matrices?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "F(X) = \\phi\\left( X W + \\mathbf{b}\\right)\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If $X$ has shape $(N, D)$ and holds $N$ samples of $D$ features, then $W$ must have shape $(D, M)$ so $F(X)$ converts the $D$ input features into $M$ output features for each sample. We say that $F$ represents a linear network **layer** with $D$ input nodes and $M$ output nodes. Note that the bias is now a vector of $M$ bias values, one for each output value.\n", "\n", "We cannot really add a vector $\\mathbf{b}$ to the matrix $X W$ but we are using the \"broadcasting\" convention that this means add the same vector to each row (sample) of $X W$. We also cannot apply $\\phi(s)$ to a matrix, but we are using the \"elementwise\" convention that this means apply $\\phi$ separately to each element of the matrix.\n", "\n", "To connect this matrix version with our earlier vector version, notice that $F(X)$ transforms a single input sample $\\mathbf{x}_i$ (row of $X$) into $M$ different outputs, $f_m(\\mathbf{x}_i)$ each with their own weight vector and bias value:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "f_m(\\mathbf{x}_i) = \\phi\\left(\\mathbf{x}_i\\cdot \\mathbf{w}_m + b_m\\right) \\; ,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $\\mathbf{w}_m$ is the $m$-th column of $W$ and $b_m$ is the $m$-th element of $\\mathbf{b}$.\n", "\n", "The corresponding graph (with $D=8$ and $M=4$) is:\n", "\n", "![NN layer](img/NeuralNetworks/nn_layer.png)\n", "\n", "The `nn_unit` function we defined above already implements a layer if we pass it matrices $X$ and $W$ and a vector $\\mathbf{b}$. For example:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.73105858, 0.5 , 0.81757448],\n", " [0.5 , 0.88079708, 0.5 ]])" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_unit(x=[[1., 0.5], [-1, 1]], w=[[1, -1, 1], [2, 0, 1]], b=[-1, 1, 0], phi=sigmoid)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A layer with $n_{in}$ inputs and $n_{out}$ outputs has a total of $(n_{in} + 1) n_{out}$ parameters. These can add up quickly when building useful networks!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Network Graph\n", "\n", "Finally, we can build a simple **fully connected graph** by stacking layers horizontally, which corresponds to nested calls of each layer's function. For example, with 3 layers computed by $F$, $G$, $H$ stacked (left to right), the overall graph computation is:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "N(X) = H\\left(G\\left(F(X)\\right)\\right) \\; ,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "with a corresponding graph:\n", "\n", "![NN graph](img/NeuralNetworks/nn_graph.png)\n", "\n", "Nodes between the input (leftmost) and output (rightmost) nodes are known as **hidden nodes**.\n", "\n", "The corresponding code for arbitrary layers is:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "def nn_graph(X, *layers):\n", " for W, b, phi in layers:\n", " X = nn_unit(X, W, b, phi)\n", " return X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For example, here is a three-layer network with the same architecture as the graph above. Note how the output dimension of one layer must match the input dimension of the next layer." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1., 1.])" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_graph([1, 2, 3, 4, 5, 6, 7, 8],\n", " ([\n", " [11, 12, 13, 14],\n", " [21, 22, 23, 24],\n", " [31, 32, 33, 34],\n", " [41, 42, 43, 44],\n", " [51, 52, 53, 54],\n", " [61, 62, 63, 64],\n", " [71, 72, 73, 74],\n", " [81, 82, 83, 84],\n", " ], [1, 2, 3, 4], tanh), # LYR1: n_in=8, n_out=4\n", " ([\n", " [11, 12, 13],\n", " [21, 22, 23],\n", " [31, 32, 33],\n", " [41, 42, 43],\n", " ], [1, 2, 3], relu), # LYR2: n_in=4, n_out=3\n", " ([\n", " [11, 12],\n", " [21, 22],\n", " [31, 32],\n", " ], [1, 2], sigmoid) # LYR3: n_in=3, n_out=2\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The weight and bias values are chosen to make the tensors easier to read, but would not make sense for a real network. As a result, the final output of `[1., 1.]` is not surprising given how the sigmoid activation saturates for input outside a narrow range." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Visual Perspective" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Activation maps](img/NeuralNetworks/activation_maps.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**EXERCISE:** Identify which activation function was used to make each plot above, which shows the building block " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "f(\\mathbf{x}) = \\phi\\left(\\mathbf{x}\\cdot\\mathbf{w} + b\\right)\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden", "solution2_first": true }, "source": [ "for a 2D $\\mathbf{x}$ with the same $\\mathbf{w}$ and $b$ used in each plot. Red and blue indicate positive and negative values, respectively, with zero displayed as white. For calibration, (a) shows a \"linear\" activation which passes its input straight through." ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "- (a) linear\n", "- (b) tanh\n", "- (c) relu\n", "- (d) softsign\n", "- (e) sigmoid\n", "- (f) elu\n", "\n", "To distinguish between (b) and (d), note that both go asymptotically to constant negative and positive values (so sigmoid is ruled out), but the white transition region is narrower for (d).\n", "\n", "To distinguish between (c) and (f), note that (c) goes asymptotically to zero (white) in the top-left corner, while (e) goes asymptotically to a constant negative value (blue).\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden", "solution2_first": true }, "source": [ "**EXERCISE:** Experiment with the following function to determine how the displayed arrow relates to the three model parameters $w_0, w_1, b$:\n", "```\n", "nn_unit_draw2d(w=[0, 2], b=-1, phi=tanh)\n", "```" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "The arrow has the direction and magnitude of the 2D vector $\\mathbf{w}$, with its origin at $\\mathbf{x} = -b \\mathbf{w}\\, / \\, |\\mathbf{w}|^2$ where $s = 0$. The line $s=0$ is perpendicular to the arrow." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP0AAAD7CAYAAAChbJLhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAHlVJREFUeJztnVuMLVldxr+1VlX3OTMMExiug8pFHmYwIkFAIDFiBBMTnowPCBohJiYaH3jQxBAffPT+oDHGRI0xARTREPACBkEwkQcTNV6iEAGHGeQ6wAwDp3vvWmv5UHvVXlW11u5dtatWVe36fknn7N69T3efy1ffv9f//9VfWGtBCFkPcupvgBCSFoqekJVB0ROyMih6QlYGRU/IyqDoCVkZFD0hK4OiJ2RlUPSErIws1Rd6zY98zAohAABCSggpIGR5zZFCVM9leVY+FgIqU5BKQikJmSnkFxmyXEEpiSxXyHKF/EIhyyTuvjvH5aVEngtcXghcXgCXOXCZW9x7V4HLTOOWKnCptriQG1zKa9yz/Sry4g6y7TehNncgN3cgN1cQ19+Effwx2OsrmKsr2OsN9NU19PUG5nqDzde/Cb0poDcFTKH3v24Ntne21WNrLKw20Jvysd4YGGMAAHZrYQobfOxwzzWf9zGR58k6eX3xcXHTa5I6fXPk1+4E4GOMrZ43phSLsbYUkLHQ2sDY8nVm9741FtpYFEX5XKEtCo3dm0ChJQojUdjyTVsFbTNoWb4ZmcMq96ZgsxxQCsK9ZeWb9H6VmYTM3MWrvGhJJSDV7vHueQAQavcaJSDl/q9cZof/ffyPizz8Whl5npAYk5X31niOZkuhW0/w1gm9ErypPTbavZWCN9pCawutsX8zuzcLaCNQGAltFLRVKKyCFqXgjcphVAarMliVAyqHyHIgyyHy8gIgs2x/EVBqV5mUbzJTnvj3AgcAocqLQRNfxCIXlcBj4iZkKJKV99YYCClLt3ePjQVQPjbWQhoDKwV0oaEyAFJCF7r6HFKXF4Qi8Pk3mX/9co9LAV1twte26+x29PtVl5dw8pMADAC1e99cb6K/T2b16kVvDQAJdVFe6Mr3JIwxpdixL+NlJmAKC5GLYDkfe17mgmU+OZq05b1XzsceO0Jlvt2V9KEy3z0XLvNFpMzfl/itMn9X4ofKfFGV9+0yXyr/10aZ757vUOY3P8Yyn5zK5Kf37TLflfShMn9X4gfKfG3K8j5W5hdatMr8wioUyINlPlQOZFm8zJfyQJkvg2V+CCdi9yvLfDI2yUXvhOse+883abq9uyiE3L460Ovg9s0Dvbrbq6jby4NuL6NuL/znG25/CLo9GZLJnR6ou311oBdye3PI7W3c7U3E7e0ht8+jbl87zGu4vX+a33T70IEeQLcnaZnu9L6H2/vtu25uj1m4PYCo23dp3wF0e9KfZKI35vDpsnP7Wvuu6fbN9t2Jbu+375pubw+4fa1913D7evuu7vax9p1P0+0JGZpkLTugFL6Uota+E8J731gIiap9B0hAomrh+e07nwKAihyUAUCeC6BqwAm4a91V1vjje58iu7hde7r2GS6vqvZdE70JNRR3v89r3+mNqVqBEhImN1X7zrXmZCZgsJ/Gcy29m2ALjxxiwuGcvYPX3g+U+MC+UmhO6bkyPziltyvzw1N6oteUHqJTevsyvzml58r50JSeK/MdQ0zpEXKI5KK/qcyvXhco86NTejY8pbcv89tTeoUWwSm9qswPTOm5Mj84pXdMmd9xSi/0mJBTmfT0vqvbx2byjTkwk69Nwpl8GZ3Jrx5H3J4z+SQVs2jZ+URn8t2FoeNMvu4xk18gj87kI8viM/m11t0wbs/2HRmayUXfdPvm8z5M4CH4cbo96UIy0fsiPrZ9B4ybwAuN5s4tgQdwWIcMS9KWnWvNAfv2nf986gTedS4Qu+7NJoEH7Nt3TOCRAZi8vHcwgceZfJKGSQI3ju5l/sAJvNhMPhN45IyZjdMDUyTw5jGTfyiBx5l8MjSTi76L268xgeeg25OhmFz0IVIl8IKjuUzgkTNnEtE3BcwEXjyBFxrHZQKPnELSlp2P374Dpkngbbf1G2jOMYEHg9oNNJnAI6cyz/I+UQIvPqU3nwTe0FN6hMwicONIncCLzeQzgUfOmVk6PZAogZd0Jr9fAo8z+WRo0s3e27CLx0Qcfu2wCbw+W3GYwCNLZ7ZOD4yfwIuP5s4ngdd1OUbz43R70mTSBZbV81Ml8Doux2ACj5wDyVt2rhXXen6CBF5RNJtr80vgCdm4gSYTeOREZl3eO8ZK4PXZipM6gcetOGRoZiX69Am8HjP5iRN43IpDhmZWoj/EWhN4AGfyybBMM3sfOdBrwgRev604dHtyiMU4vWPQBF5gNHduCbzYcgy6PenLdIGbI07xgfhJvgvjlCfcohXEgTEoz701tAz/hy6K+GDQduuHcAD/LP46y/cvlICfsHFhHD/KY93LNvHTe3WRIXSHP7MVQK4AaLjv1n0nofBNKHjDMA7xmXb2/sgpvZOjtx1n8g9uxbGIz+TL+Ew+snQz+TITLPFJlOWV9wMm8PptxUmbwEu5FYesg8lFP5TbV6/rksCLzOQzgUfOmclF34ehEnjz2YoTT+BxKw4ZmlmIngm8Qwm87jP5TOCRQ8xC9H0YIoEXW44xpwRedDkGE3ikJ5O17Jp0DeL4H3NhHNe+s8bASFm18KzYt+8kFDTKMI5R+wtEgQL+X0ex9S8e/rVR4GrTjNLsuVS3au+7zygByMtbVfuu+bvNZhv4Wymg8sbX2OqqhWe1qIdxclO17fQRQRyAYZw1MiunHy16O+RWnJuit5GZfMutOGQmzEr0fTglgddvJj91Aq/7Vpyb4JTeulmM6EdJ4A09kz9CAq/PVhy6PTnEYkR/iL4JvD5bcZjAI0tndqJPmsAbeCvOGAm8PltxfOj2pMnsRH8KXd0+thxjTgm82HIMJvBIX2bTsvNJlcCTxlbtuyZzSeCpC5e8mzaBR86H2Tp9igRen604TOCRpTNb0fchyVac5Am87ltxxkjgscQ/H2Yt+rETeO5rzDqBN/BWnNBjsi5mLfo+dHH7Pltx0ifwum/FYQKPHGL2oh81gddnK07qBN7AW3GYwCOzF30fzimBF1uOwQQe6cssW3ZNxkrgWSlqN9CcYwJPRldhMYFH+rEYpx8ngddnKw4TeGTZLEb0fTiHBF6fmXwm8MghzkL0fd0+6Vacngm8PltxACbwSJyzEP0hmMDjTD6psyjRD53Aiy7HmFECr/NoLhN45AYWJfpTCLl9iNkl8DouxxgrgRd9HS8Ui2MRLTufIRN4QP0GmnNM4ImLq0DzryRlAu9Qa48si0U6/aRbcRIn8FJuxWECbx0sUvR9WGoCr89WHCbwyCEWK/oh3H7orThjJPD6bcVhAo/EWazo+zDETD4TeHT7pbNo0a8hgRcdzWUCj/Rk0aLvQ13M3UZzp0jgxZZjpE7g+dDtl83iWnZNTkngucdzTuCJW1e1G2hOlcBrtvSYwFsuZ+H055zA67MVhwk8coizEH0f+s7kp0/gDTuTzwQeOXvRLz+B130rDhN45BBnL/qbYAKPCby1cTaiP9cEXmw5RuoEHjkfFn9679M1jBMK4ggpYY2FkGitw9KFhsrKX0MUAFTkoAwA8lxgH8QR8K+516oRxtnhgjju6dpnuIyHcfSmHcJxiG35Z1IXgN6Y1km+BGqn934wBzh+FRZP8ufJ2Th9X7rO5LtKIRa9jc3ka3NgJl8PvRXncPQ2NpMvPPcfYkqPzJOzE/3ZJfD6bMUZIYHXfLwEtLWtN3Jm5X1fwmW+av1YUHu9UjDGQAgBIy2EsTCidHUhZc3tlRRVAk+p0uWVFlASUNJL4EkJZSVUw+2lzCFVAasKWLUtf6YvikrocC6vFaxWkMZAZhLWyN2fwbm9hTXuQM+WJbvG/lQfgNUSZneBvKmMPyZvP0WJv7EGv2Q+i3/HndbHno8L/JZ6XtLvZ26cndMDI8/kJ07g9dqKM0ICb0ntu//CVVDwAPBpbHDHhM9k1sJZir4PTOCdTwLvyVbhqZEjzmchx20ZO/5cB2db3veZyW+/1lai8E/yrTCAVLvtOYCVO8GLptsDRgoYKaDV7ud6aaG1gJQ7t1ftBF4mFLQwKKyCFhmUzGFUAWO2EDu3F0pDZDms1hC5htUaMstgC+05vq65vfvz1IZ1jC3dHgZa1y94zZL9mK04U/KI3eBP7aP4qP06QjXdk6Hw6/Jbk39fc+NsRQ90F35zFZZr3wG7n9N3wrdSVO07SFlr4Uld/ncLNcw2mV9Y+XfDwy6M0+Y6ux18HgDU5WXgTnwlJhrGAWRWl4T2WnjW2H37zpiytN99bnvEOqwpfra/SexA+ffzNvls3CvO+r/8UfBvoEEzgRd67DDGQqJ+qGd3B3oicKjnSn+pROtQz7XvlJBQ0laHelWJbzPI5qHe7kDPL/PdoZ7IygM9q9XuPGJ/qCeVgDXu18ahnmwk8I44B5kqgXdI7C/FXXiBuMS77VcBAG8S9+E7xF2Dfe0lc/aiH6PMF8ZWd85tl/llJRAq87WxENpChcr83YFepmytzC+sghI5lChaZX55oLeF0O0yvyrvA2W+kMXufdsq8xHwSpELoDmsM2GZf5PYf1TehwfEbWhrcT8ucJeQeDWelPz7nCtnL/o+WGMAKWtTeu75m9zeXRRCbu+375puX2hRte98t68d6LXcXlXtu6bby2zfvmu7/b5913R7v33nu32X9h0wjtsfK3aHEgKvE/f2+lrnDEW/o4vbV/fND7m9O9ALub220Cri9ibs9q59F3J7d6AXcvvaYV7D7evtu7rbSyVaB3o+U7h9V7GTw1D0B2jO5AM7wat6y8d3e2tsbVjneLevD+tUbt8c1vHcvmzfbYNu3xzW8d2+OaxzjNtr6ORuT7GPwypE32crTvDz7Ny+GcTx3b5y96Dbm6p913R7v33nu702+/Zd0+2t175run2tfddw+3r7ru72x7Tvmm4/NBT7uKxC9MByE3hXWeOfaKIEHgz27buREngUexpWI/o+xIUfnsk3xkIqBGfyhWgn8Pwy391k4+iZfFtO6YVm8qHiM/lWq91cfnsm3yXwUszk+1DsaVmV6Icq86vXBcr84JReVea3p/SqMl+jNaXnz+T7U3quzDcqb03puTJf5Hl7Sq8q80NTersyv8OUXuzxsVDs07Aq0fdh+gRefVhnqgSeUKI2rHOK2z9iNviT4lF8xDxOsU/A6kR/rNsf/ByRmXz383bU7SMz+aJK4B0/k6+ES+C1Z/KR7cQfmsmvte5Cbt9tJj/0c34Min0erE70fWi6ffN5n9BornN7N4YbdPtdAq/p9qHRXN/tW6O5O7cPjeZWbp+1R3Od2wdHc5WANKI1mnus2z9irvHO4lF8pGiL/bvl3XgDnkqxJ2SVol9SAi80mjtFAs9s63/+YxJ4D+trvOPOl6Nif2N2Hx6Qt3kfvcSsUvTAchJ413n9Bpo+KRN4QjZuoHkggfeI2OAdT3wJf785LPbqa/IGmklZrej7MEUCLzaTnzqBF5vJ93m4uMbbH/sSPnz9WFvs6m686fJpeEDdns1ttdYKRR9gVgm8yEx+6gRebCZf5AIP37nC2x/7Ij70jbbYX5bdjTdelGIn84Ci70j6BF5kJn8GCbxHrjf44y99Hn/3+NfaYr+4G2+6fDoe3P0Iwo2382HVoo/9XN9kygRebCY/dQLPb9995uoKf/T5z+GDX/tqS+wvv3wSfuyep1c/s8/xtlprZ9WiP4VUCbxC29Zo7hQJPCEN/veJO/iDh/8PH/zKV9piv/0k/Pi9z8CDsu3sAPfbz4nVi37+Cbz2aG7qBN5Dd+7g9z7xEP72S19uif0Vd9+Dn3jaM/Bgdlcl6jETeOR0Vi96YN4JvO22fgPNlAm8T3/9G/id//wU/uaRL7TE/sonPxlvfuaz8aJbd0FvdfV1uANv/lD0PUmVwOu1FefEBN4nv/o4fvtfPo6/fOhzMA3tveop9+In778fL7rrbli9q1RGTOCR4aHod8w1gRebyR8jgffJx57Ab3z0X/He/3m4JfZX3/cU/NTzvqUUu7GwetwEHhkPiv4EkiTwIjP5QybwPvn5x/FrH/wn/MV/fAqmseHne591H37mgefjRbfvgt4amK0eNYFXvYYl/mhQ9B5zTOD12YpzbALv45/9In7lff+Ad//zf7fE/n3PeQZ+9sUvxIvvvQem0Nje2Y6awCPpoOhPZOwEXmw5xikJvE984VH88rs+gHd97N9aYv/+5z0bb33ZA/iup94LvSmgN25CL7Ic48QEHkC3Tw1F36BPAq8ZxikDKmW5b6SsJfDK2IqGhIKGH8YpP3eBAv4/S7H1z839i4jYrcLyozR7LtWt2vsZgE889Fn86h++E3/2oX+EafzQ/toHnouff81L8ZKn3AN9vYXe+LG6AipvfI2tBnIFQMPqxlac3FSn9/rI22VT+Omg6AOMFb21UkRm8m29j3/iVhwtDArs+/af+Mwn8Zu/+/v48w98uBWSed13vhC/8EOvxsuf83Toq2tsH39isK04djufrThkD0U/EGMl8PpsxfETeO97/4fx02/9uZbYf/ClL8LbfvgH8IrnPhPm6gp2s7khgdd9K85NcEpvGij6jiRP4PXYiuMn8N7zV++vCfC1r3wpfvGNr8f3fPv9wPU17PXVUQm8Pltx5rYDj5RQ9AMyxwTem9/yFjz00EP4tvufibe++Q141YMvQPbEo7Cbq04JvD5bceayA4/UoegjzCeB130rjp/Ae8nLXoW/fu97cKGvkBd3YLbf7JXA67MVx4duPx+Oaz6TTthdG8x6ZbUN/IxrdsIuP25hdmL376Tjt+/cc25gp9B297M9qv32hZbleK5t5O13b6ZK4CnY3Q00Wwk897N9VopdZvXyXipRtfBc6Q+Ubl89rwTkrrqR2eGLZ/PjIg+/XkaeJ92g0x+gz2huaErPtfBCo7muhacj1UJRxA/Etls/hAP4d8O7zvL9CyVqXT0XxvGjPNa9bBO+f566cEGc9h3+zFZU7Tv33brvJBS+CZX8TOWlg05/A9aG/yM2nbvZ926/vvy42c3ZV206z+n9sn7fwtuV+NqUh3qmnMc32ovd6vqhnrYoS3yjqjK/QA4tst2kXlZFb8vbamUQWV7dOrs60HPbcBpbceqHesIbxy2rgCa+c7vHMhNRRyfjQtGPQCXkRpkfKvGB/QXD7C4GfpmvtWkl8FyZXxR2twMvUObvSvxQmb8v8SNlfhYq82W0zC+34ohgmS+9iqhLmc8Sfzwo+iMYyu2r1wXcvnL5ptubuNvrQ27vDvRCbq/ibi/yiNs3tuIM4fbNxyQNFP1IdHV793zT7e0ht98d6IXdXvZyexF1exV1e3egF3J7QbefHRT9kRzr9gc/h1cJNN3efY0ubm8OuL2fwGu6fflzfdjtEXP7huPX3V70cnsncLp9Wij6EWm6ffN5H79957u9OeD2fvtubLf323dNt/fbd023b7bvALr91LBl14EpEnhG7S8QYybw3GeQl7eq9p3/u2V0FRYTeEuDTt+RY8r8Y9t3gFfm++27Wpnvt+/qZX69fdcu8137rlXm19p3jTLfa9+1yvxG+65e5stgmR/CubffvvPfJ+NC0ScgNpl3TJkfnNLblfmhKb16md+e0iv8HXihMr/DlN6+zG9P6UnvAhCa0rsJTumNB0U/IKe5vQ27fWtYx3f75rCOjQ7r+G7fHtbZu31zWMe5fWxYp53Aq7t96EAPoNtPCUWfCCdc99h/vslRM/k3un2amfyb3B7gTP7coOh7EPu5vkkXtw+O5tpA+24Itw+O5jq3b4/mijzvPprrJ/Aibu+g26eFok9MV7cPjeZOksALjuamT+BFX8cLxdGwZdeTVAk8aWztBpo+KRN44iK+CitlAo85/NOh059AigRedEovcQIvOpPPBN7ioOgTM9RMfuoEXnwmnwm8pUHRn8jYCTz3NaZO4EVn8pnAWxwU/QR0cfvYTD4TeHT7vlD0AzBqAi84mps+gRcdzWUCb3FQ9BOxtAReaFhnigSeD92+H2zZDcRYCTy3CmvqBJ64dVW7geZUCTzeJ/906PQDMk4CLzKTnziBF5vJZwJveVD0E7OcBN6wM/lM4E0HRZ+Ivm4fnclPnsALz+Qzgbc8KPoZwAQeE3gpoegHZugEXnQ5RuIEXmw5RuoEHjkdnt6PQJ8wjv/7qlN9YyEkWmEcXWiorPw1RAFARQ7KACDPBfZBHAH/2n+tGmGcHS6I456ufYbLeBhHb9ohHIfYln8mdQHojWmd5EugdnrvB3OA41dh8SS/Dp1+Jgy5Fad8PzyTr82BmXw99Facw9Hb2Ey+8Nx/iCk9UoeiH4lJt+KMkcDrsxVnhARe8zHpDkU/I5jAYwIvBRT9iAzh9kNvxembwOu1FWeEBB7bd6dD0c+MIWbymcCj2x+Coh+Zc0ngRZdjJE7gARzWORW27BLQNYzTDOK49h1QPnYtPCtF1b6DlLUWntTlBSHUMNtk/rXevxsedmGcNtfZ7eDzAKAuLwN34isx0TAOILP6hU97LTxr7L59Z0xZ2u8+tz1iHRaDOHHo9DPllJl8V97HoreptuKIg1N6IjqTD4Az+SNC0Sdi6Qm8PltxmMCbJxT9jOk7kz9GAi/lTP6hBB5n8k+Hop8Bi0jg9diKM0YCz0G37w9FvwCYwKPbDwlFn5AlJ/DiyzHSJvBiyzHI8bBll5iuCbzQKqwpEnhXWeO/ykQJPBjs23dM4PWCTr8AZpHA6zGlN0YCb+gpvTVC0U/AIhN4PbbiMIE3Tyj6hTB9Am/omfx+CTzO5J8ORT8Ro87kj5DA67MVhwm8eULRLwgm8PptxaHb16HoJ2Sc0dxxEnjR5RhM4C0OtuwmZikJvOu8fgNNn5QJPCEbN9BkAq8zdPoFsuYEHrfinA5FPwMWkcDrtRVn+AQet+KcDkW/UNIn8DiTfy5Q9AuCCbx+W3Ho9nUo+oWTKoEXXY6R2O1jyzHo9sdD0c+EoUZzzz2BB3QbzWUCrw1bdjNizgm87bZ+A82pEnimkLUbaDKB1x06/cJJlcAbeitO3wReyq045wpFPzPmmsDrsxWHCbx5QtGfAUkSeEln8uMJPG7FOR2KfobMMYHXZyvOOAm87jP5TODVoejPhLETePHR3LQJvOhyDCbwjoainymzS+B1XI4xXgIvvByDCbzjYctuxswpgVcUzebaNAk8E2g3MoHXDTr9mTFWAq/PVpxxEnjdt+LcxNqm9Cj6hZI+gddjJn+EBF6frThM4NWh6M8QJvA4k38Iin7mDL0Vp3y83ARen604PnR7iv6sGTSBFxjNnSKBF1uOwQTe8fD0fgF0DeL4H/PDOOUJt2gFcWAMynNvDS3D/6GLIj4YtN36IRzAP4u/zvL9CyXgJ2xcGMeP8lj3sk349F5duL5CO5BjtgLIFQAN99267yQUvgkFb9YQxqHTL4Rk0duOM/kHt+JYxGfyZXwmH1m6mXyZidWV+BT9GTNkAq/fVpwxEnjdt+IwgVeHol8QkybwIjP5yRN4A2/FCT0+dyj6M2eoBN6St+IwgVeHol8Yq0/gDbwVZ40JPIp+BQyRwIstx0idwIstx2AC73jYslsgXYM4/sdcGMe176wxMFJWLTwr9u07CQWNMuBi1P4CUaCA/1+n2PoXD99HBK42zSjNnkt1q/a++4wSgLy8VbXv/N8to6uwCqi88TW2umrhWS3qYZzcVG07fUQQBzifMA6dfqGMFr0dcivOTdHbyEy+HXgrjojc7HOtM/kU/Yo4JYHXbyZ/+ARen5l8JvDqUPRnyCgJvKFn8nsm8PpsxQGYwPOh6FdG3wRen604TODNE4p+wSRN4A28FadvAq/PVhwm8OpQ9Culq9vHlmMkT+B1XI7BBF4btuwWTqoEnjS2at81SZnAExfxVVhzSeDNHTr9GZAigddnK84YCbyUW3HONYFH0a+UJFtxRkjg9dmKkzqBN/cSn6I/E8ZO4LmvMXUCr99WHCbwfCj6FdPF7ftsxWECb55Q9GfEqAm8PltxRkjg9duKwwSeD0W/cpaWwIstx2AC73jYsjszxkrgWSlqN9CcKoEnbl3VbqDJBF536PRnyDgJvD5bcYZP4PXZisMEXh2KngBYUgJv2Jn8NSbwKPqV0dftk27FOZjA674Vhwm8OhQ9qWACbx0z+RT9mTJ0Ai+6HCNxAi+2HIMJvOOh6EmLkNuHmCKB1305RvoEXvR1M7lQsGV3xgyZwAPqN9CcKoGX57drN9B0v3tOCbxDrb05QKc/cybdijNGAq/PVhwm8GpQ9KQFE3jnncATxx74EELOAzo9ISuDoidkZVD0hKwMip6QlUHRE7IyKHpCVgZFT8jKoOgJWRkUPSErg6InZGVQ9ISsDIqekJVB0ROyMih6QlYGRU/IyqDoCVkZFD0hK4OiJ2RlUPSErAyKnpCVQdETsjIoekJWxv8D3fMBblWvhhIAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nn_unit_draw2d(w=[2, 1], b=+1, phi=tanh)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Study the plots below which show the hidden (left) and output (right) node values for a network with 2 + 2 + 1 nodes. Each graph shows the node value as a function of the 2D input value.\n", "\n", "Note how the hidden nodes divide the input space into two halves, with a dividing line determined by their $\\mathbf{w}$ and $b$ values. The output layer then mixes these halves and can therefore \"select\" any of the four quadrants with an appropriate choice of its $\\mathbf{w}$ and $b$." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfMAAAHRCAYAAAB+a1B2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzsvXmcHFd5LvycU9XdMyON9n23JEuWLFtekW3AjsFgNuOYxYQlJEDgS8hNIPcm8OUjgSS/hMQJgUtuCAFCbtgCGEgcIIAxmMXGuy1LtoRteZG177tmprurzvn+OHWqTlWd6q7eZrp73uf3619319bVPVP1nOc97/u8TEoJAoFAIBAIvQs+0SdAIBAIBAKhNRCZEwgEAoHQ4yAyJxAIBAKhx0FkTiAQCARCj4PInEAgEAiEHgeROYFAIBAIPQ4icwKBQCAQehxE5gQCgUAg9DiIzAkEAoFA6HG44/VBL7zhZ2Q116NgjNmX8/hYkHOWWscYi15z9ZoHy7jD1TOP3rsFF47DwV0HjsPhFNTzwGARbsFBoejAdTmKRY5CkWN4ioNSkaFUBEoFYKAoUSwIDLg+ppfGUHKqKPEKiqyMohhD0RvFwOgxOJVROJURsPIoWHkMqIxBnDwBMVaGGBuDX65AlCsQVQ+V0yPwK174EJ4Pv+KhOlqFXxWQQgTPEn7Fh6gKCF9C+hJCCMiq+tf3RwUAQHjRpaDXmcvM5UkIy/LXeE8yAPjNjxyga6zHwbj9Wktvl7j2WHI9C69bfV2qa5Gp640xMA44DgdjDJwBhaIDx2FwHAbuMBRcBtdlGChxFAtAwQWKLlAsSJQKAiVXYLhURsnxUOIVlFgZs6cNoHz8AAZHj8OtjsCpjIJVx8AqY2DlMcizZyArZchKBbLqQVarEJ6nrjXPh/R9SM+HFALCE+q9UNeWfgYA4fkAEL43X0shrL+ZuW0WsrZZ/aX/zveHmUCQMie0BTYiN5G8SbHE9owxcM5DwmeMgTuK+B2Hw3F5eKMJH1zdbBwHcB3AdSQ4l3C5hMsFHC7gMB8OPDjSgyOq4MID96vgfhXM8wDfB/wqpFeF9Dx18wgewvchvPhD3WR8CF8qwhYC0pfqZuMHZB0QuYkkWQPZhE0g1ILt+oqvT/OOOSDXRM412TNEJB8QPWcB4TMGhwOcw3iWcBjAmYTLBFzmw2U+HOaj4DA4wgMXVTDfA4QP5vvqOhPBs+8DUgJSQEoJCDUg1s8maZtErmEj3HpEnYfIex1E5oSayKvKbcu1Ktc3F1OV63WO66gbhn52nEChK0XuFhz1cHlclRc4SoFiGCgCpYJEsSAxUBAYcD2U3EgtFFFG0R9FwRuDWx2FUxkFL48ClTGw8ghkeQxybEypg0CV++UKRMVTrxOK3K+KiNx9Cd8TkH6kxjWRa1VuErl+bRJ5K6qc0D/Iq8qTqLVbfJCdHlDr61sRtyZ2wHGCBzeeuRowq4ewDpiZFPEBs18F/CqYVwV8H9L3IvWtlbgIBsNSAiGJS0gZV9hJQraTul2V50GvEz6ROSETjRJ5rX01kcfehyQfPccUgqkUgnCgDgG6LgtuLgifXUcpcpdHisFhfqDKtTJP32BQVQpdhfzUTQZCQpg3mtjDCPfpZ03kIk7kGklSzwIR+eREs+H1rOOoSFc8vA4guLbi4XXucGt43Q2uM1t4vej41gGzI7xwwGyG11GtqNB6IrweEnoyvC4FYLneku/1slpoJbzeSyAyJ7QMbijvJNJqIK7Uw/A6ixN3GF63hdi5elY3HFt43VeKIRFeZ75nD6/7XizErsPrMiO8LoWMkbcOrwOK1E3UC6/nIXgCAWg9vM6NEDpghtfj63R4nfFIkafC6xkDZiZ9cFFNhddljfC6tITXEZI2hdfzgsicYEU7VHmU+JZQ5aZqMJ45s6vycO4uocqj0F8jqtyzq3LPb0iVSyEbUuV5QKp8cqLZ8DpgD7Gno2L1wusqxJ4Kr3MWvk6H1zMGzFJYw+thiD0RXocMrpuc4fVaYfRmQ+z9QvhE5oSWUEuVJ1FXlQcZ7XlVuVNLlYdEHlfl3K+2R5VrUk+ocjPxjVQ5YbyQV5Wz4DJNqXJuV+W8AVXuiCqYFGC+Z1flvm9V5chU5bVD6aTK4yAyJ6SQpcob2beWKjdL1fQzZwlFbqhyPaeXVOWuk63K3QxVzjJUuaxWG1LlGklVDsCqyusRN6nyyYlOqfJk5Mu2janKOWtClVsGzEz4wYDZosp9uyqXsjFV3u7Et34BkTkhN2rVlQM2Io/m55J15dzh4MGyWF25zmB3nVhdeaHIUSpyFAssVleua8oHXF/VlJt15d4oCt4onCAhR9eVozIGOToKWSnH68qrHjydxZ58VH0IT8Tqyv2KHxK5WVcOKPJOEjmpckKzqBX5qjcgMFU5D5LekqqcZahyp4Yqtw2YIUVUgpZS5bIhVV6bvEmVJ0FkTmgKjdSVm+F1/T4ZXo8UeN7wulbklvB6MI+XDK/rxDdbeF1awuthiD0ZXvft6jx8X4eoicgJQGsZ7PVUuc0gxox+qeuKgycTSl2g6DIUCiqDveACJVeimHPAzHwPvKKy2GWlAlQqyiTGUxnsYcWIzlMJs9hlyiAGSBvB5FHljRB5nm2rPaL6x80BbrLg4R+/CTPnXYmVF7w/tlyIKrb8/N3wKidx8bVfhFsYjq2XwsOWu/8f+NWzuOiaf8WRfXfima0fSx2fO4MYnLoUC895A+Ytebn1HPY+83Uc3vtDXHT15+ue71Ob/xJH9v4YG676B0ybdUEsxF4tn8ADP7wR5178Icxf9grju3jY/9x/4dDu2zFyeicYczE0bQUWnvNaLDznNeF2jHM8/eg/4blt5nkwuIUhDM9ajRXn34xVF/5arBztq3+3ESOn9mee77v++FZc8ZI3RGF2LgC/gt968zV4zY1vwDvf+U5F6Jbwug79yUR4PSTxnOF1nfiWInUj8S1veN3cNgkKsdvxzU9ejiVrXoYrXvnR2HLfr+A7n30ZyiPHcNPv3o3iwPTYeiE8fPdzr0C1fAqv/e2f4LnHb8O93/3D1PHd4hRMm70S6ze9G6sufGO43KuOYMvPPoGdv/wOxs4cxrTZK3HBi34fK9bfUPN87/rP38XSWcew6RV/iZnz16mFwWVWGTuFO7/2m7jwxe/Dafeq6Fz9Kp586At4Zss3ceLIDnDuYsbcNVhz6duw5uI3IZz8BvDQHR/Foz/9RPSBjKFQnIKZ88/D+ZvejvVXvB0MkQL/5z8+F2dO7Ms833d/6FZsuvYNRohdAH4Z73nrNXj1jW/Eb77jXdkDZikAPyO87lnC6/pakmnCbIbIG0FddS8l7jlwFGeqHtY1/SnjByLzcQLnBay68A/x+D2/h+ef+BesuuAPYuv3PfdNjJx6Buuv+Hs47lC4fP2mj8FxB8P3lbEj2PfsN/D0ox9FoTgDM+e9IHacI/t+il1PfBaDwysaODuJZ7Z+DBuv/hc4TrHmlr43iu33fwCnj/8Si1a+HivOfw+k8HH80P146uFbcGTvT3HBi24BQ3AcxuC4Q7j85Z8BAtL2KidxcNdPsPnOD+H0sSdx2cv/Mkx8YwBWX/Q6XHLte+G6yl6yUOAYHHJQdBkWr1gTK0eT/hg++ifvxJPbt+C1v/r6WHatTZXDz1DlfoYqD25Ajbi9WX/hBsid0Bwcp4irXvMxfP//3ohH7vxrXPGqv4mt337f53D84Da8/G23olCcEi5/2du+Hr6XUmL0zAFsv++zuPu238fA0BwsXn0tAODe7/4R9uy4Exdf+wFMm70Ku574Pn72zfeAvfFfsHzdq+ucncS2ez+NK1/793B4oeaW1cpZ/Ogrb8ORvY9i3aZ34pLr/hhC+Ni7407c850/xPPbv4uXvvlf4bilUN0XilPwynd+C5wDkBLl0RPYuf17+Mk3fh9H9j+Gl7zx7wFEin7d5W/Eput/D04QbncchmKJoeAwLF6+JlaO5ntl/N2fvhNPbX8UN9z4+ijxLVklInxAiIDE7W5v6fC6CLPZbW5v1l+yBgm3K7wupcRP9x7GwdEx3HjO4rYcs9MgMh9HTJu1AQuW34gDz38b85a+EsMzzgMAlEcPYvdT/4b5y27AjDmXxvaZOmNtSsVPn3MZHvzhjTi85/aQzL3qWeze8QXsf/YbcAtTGzovxx3C6Jld2Pv0V7Bs7TtS6021/tzj/4jTx7Zj49WfwvDMdWEIcM6iF2L2whdi693/E88+9hmce8n7ovly5mDmgotiHuxL1rwcg1Pm4IkHP40V62/AwlUvDG9MU4bnY/GqKzI92LUq3/X0I/jcLe/FnuefBgBwCOMm41lVeagYEqocVkUefwAJkxhS5V2HeUsvx9rLfgNPPfxFrN74a5iz+CIAwJmTe7DlZx/DmkvehoUrXxzbZ86ijSkVv3DlNfj6352PZ7Z+A4tXX4uR0wfx7GP/gRfe+Ems3ngzAGDRyqtx+thz2HbvP9clc7cwiDMn9+K5rf+B1Re9qea2D97+ZziydzNe+Y7/xNyl0f1g6ZrrsGztdfjhl96Ch398C15w/YeDNQyMO1i44nL1Lginr7zglRganouHf/xxrL30dVh67ouDrYGp0+dj2bmbMj3YtSrX19jeXcE1xmT2gNlXc+bMqwaD5LTbm1WV15gTnyhVvn9kDEfGyrhp5WKUHKfpzxlP0Jz5OGPZee9GcWAWnn3s42Fo6dnHPwm3MA3L1/12rmNwXgDj8XHYwV3fwdG9d2LNJR/GjHmbGjqn0tBCzF/2Gux5+isYOb0zc7vK2FEc3PXfWHjOTTEi15i98ErMX/Zy7NlxK7zqWQCAmRif9GBft+m9cNwSnnnsa2E5GsIGEPU92D/5kXdg6vAw/u+td6jfBaKrPdhJlY8PLnnp/4fBqfNx7/c+EF5j93//QygNzsClL/twnb0VHKcI7kQK2quOYM2lb8eilVfHtps2ZzXOnNhV93iDwwuwdO3L8Ozj/1Fz+5HTh7Bj81ex9rLfiBG5xpJzX4KVF9yE7fd/HtXgGjOR9GC/7GV/AKcwgG33fDFMfIOuK8/hwf4Pf/YOTB2ehs9/7ccAjAGz9Owe7MJIgOuAB3unIaXE/rOjWDRlEK9ftaRniBwgMh93uIUpWLnh/Th78ikc2v09HD90P44fvAerLvxfcAtTUttLKSCFByk8CL+CsbN78czWj0H4Y5iz+Lpwu9kLXoxLXvIVzFl0bVPntWL9b6NQnI6nt/xdWCqSxMkjmyGlj1kLrsj0YJ+39FoIv4wTBx9S2zAGMFg92EtD0zFr4UYc2fNw6MHOENS2MgHXEeBcwHUFHO4pC0nDg/1Pbvk3fOpfv4Vzli0EADjwutqDPfW3JVXeERRLw9j0qo/i2P7HsGPzV7Fnx4+x56kf4opX/y2KpeHU9kL4EMKDEB58r4xTx3binu/+IbzqCFZueB0AYNqsc3Dlq2/B0PACYz8P+57+CabPXp3rvNZc8usolIax7d5/ztzmwM5fQAoPS859SWqdDpGvOP818Kuj2P/MXdZjmIl1xYFpmL/0Yux77n61LiR7CSE9AMEjeM2kHytH++Bffwmf+JfbsGTZUnUOWQNmHf0Kw+yd8WCv1xktD7IGCVJK3LX/CO7efwRCylhJbS+AwuwTgFkLXoRZC67Gric/D8edgrlLXoGZGWr6wR/emFo2OLwCay75CGbNvzJcNjCl+XkdBga3MBUrN7wPTzz0pzjw/G1YuOKmlPIeG1GJaQND0Q0tWZM+OLxEbXt2f7De+ByLB/vglDk4fnBb5ErFgEd+8mk88pNPp85z5pzF+My3d4V15eeuXQ+XjcBhqh0ihyAPdgIAYNnaV2D5uldj809uQbE0jFUb34Ql577Uuu3XP3Z+atmMuWtxzes/gyVrrrPsobD5zltw6tizuPz6v8h1ToXiFKzf9FvY/JO/xa4nfoBl570itc2ZE7sBAFNnLc88zvDMZQCA0yd2h3XiQLYH+9DwXBze+1jo1wAAD9zxKTxwx6dSx545ZzE+/V+7Qg/2NWvPQ8kZgyMqAOwD5pgHu/B71oP9/oPHsOfMCF6/aknPETlAZD5hWLnhfdj807cD4Djn/N/N3O78Kz4Bxx2E553Fnqe+iPLoQay5+MOYMm1lQ58nZRDy0mAcjCXD5Fdj1oIX4flffg6z5r8QjjsQWx/eNJx0Ak/U4MEJN46Vo2V4sANqDk+XowHA+hfcjKte9X7V5KGgasunDAJDA4W021tQigYAXPrkwU4I8YJX/BVu+6erUWUcL7j+zzO3u/7t34JbHEJ17DS2/PzjOHtqL65+3aejzHMLtt71v/H4Pf+IDVf9j3CQIIQPIPo7M8s1Nn/5FZi/bBOeeuTLmLf08lgoH0B4jXKeHd7lbu0EuqQHu75udV05AFxw5a/hxTe8X5WkOYDrMgyWgIGBoqWu3M8cMKc82ANl3m4P9npoNSRfFQInKhX8ag/NkSdBZD5BKA7MxtDwOSgUp6cS3ExMmb46XD88Yz223PUebL//j7Dx6s+hWJqV+/Mev+d9OH388fD9vKWvwuqNHwBDfAS6csP7sfmnv4FnH/8kzr3og7F1WpGXRw5icMoiq9tbOVDkA1MWhHNxap3dg33k9AEMTVsY23bKtHlYsvqymEnMtCnAQDFyoTLNK3hwo2FSkAc7IcTQ8HzMnLcWpaFZqQQ3E7MWnB+un7PkUnz3cy/HHV95M254zx0YnDo3tq2UAg/c/mE88cDncd4L3oVLr/tQuO72L9yEQ7sfDN+fe/FbcNUNfx/trDPJr3g37r7t97H9/s9hw1XvjR1/6gwVzj57Yi+GZ0bq3KwrP3M8UO8zlsT2zfJgP3NiP4ZnLIqp+OEZc7F8zWVhZ7RiAZgyoDqjuY6Iu73VGjAno19SdsSDPa8/ey1kEf7zp89i3uAAXhFM1/UqiMx7CI47iFUX/i9su/f9eO7x/4O1l34k976rN34AvjcSvneLM6zblQbnYvm6d+PZx/43juyLyt44Z5g5bxMYc3Fk388wY+7F1v0P7fkpuFPCnEXBtEFoZJH2YPfKp3D84Dace9EbQ5MYPWee24MdHrhUNxomffJgJ7SEQnEIV776b3H7F1+PB37wJ7jmDZ8J1wnh467//B/Yue02XPjiP8DF134gtu9VN3wc1cqZ8P3A0GzrZwwMzcKaS96G7fd9FgeCbHuNRat/Bdwp4Plffg8LzrkqtS/jDDu3fw+OO4DFq1QyXi0P9sroCRzasxXnX/Frkdub2qshD/asAXPKgz1U5u3zYK+HVlT53jOjuH3XAdy0cgkG3d5U5BqUANdjmD77IsxeeC2O7v8JTh7dknu/wanLMHXGeeFjYGhBpgf7wnNeh+FZG7Bz+2diywul6Vi08ibse/Y/ceroVgBxVX7i4EPY/+x3sXTNG1EoDafm3JMe7Nvu+wx8r4x1L/j1KOyu0mxze7DrzFoA4EKQBzuhZSxYcRVWrH8tdm7/Ng48f2+4/MHbP4yd227D5df/RYrIAWD6nNWYs+ii8KFVtg3LznsFZs47D089/OXY8oGhWVh7+W/iiQf/DQefV0lrpuDe9+zd2LH5a1h/xTtQHBhOWSqbYIzh4R//A3yvjI0vekfowQ7dIa3BzmiAbcAcL/mEFG33YG+HKrfhRLmC7+/aj+uXLcDcwVLbjjtRIGXeAZw9/Qz2PfuN1PI5i16C4oB9tN4Ilq97D44dvBs7t/0jLnzxZ1Lzcq2CMYZzN34Am3/2rvC9xjkbfgcjp5/Hlrveh8Wr3ohZCzeBMeDYgfux56lbMWvhJqy59PejYwGAFDh+8FGwwBimWj6OAzvvxI5Hv4oLXvQ7WLRqU1iGBgBnTx7EnmfuRzGwlywVgKEBgaIrMX/ubKxYsSTW6IGLqvos6SfK0fKr8vBhs24lVd51OH5gG7bf99nU8nM2/CoGp85r+fiXvPRD2PXk7Xjw9g/jNe++HUf2bcETD/4rFq9+CeYuvgSH9zwcbsu4gzmLLqpxNDvOv+p3cM934g50jHNcdt2HcPLwDtz+xZuxbtO7sGT1NYCU2PvMT7H9vs9j8aprcPnL/yS2n5QCh3Y/pKbNmED57DHs3P4DPHbvF3DZdb+HpedeEWt9eubEATy/4z4UHHWNFRxgsCRQcCTmzZuDFcuXxQbM3DJgtnqwa/Jukwd7PbSiyo+MVbBp/iwsH05XEfUiiMw7gNPHHsPpY4+llg/PXN8WMh8YWoiFK16Hfc9+HYd2/wDzl72q5WMmMTTtHCw5963Y/eS/hcsY53D4AC588cex/7nv4MDO72L/c7cBAKbOWIV1m/4Yi8+9EZw7altXPXvVs/jZN2/WR8HAlDmYOW8Nrnvz57H20tcFjVRUQxXGgMfvuxWP33er9bxuvPmd+NBffAwlXkGJlVH0R8G8MQAA96uxcjRdiibL5Vg5WrIULSxHC0rR9LMuR9ME7ldFrBwtWX6WJO7k+7xz6oT6OLT7ARza/UBq+dwll7aFzIdnLsO6F7wT2+79NJ5+9Os4ffx5AMDep+/E3qfvjG1bKE3DWz74ZI2j2dXz1BlLsfKCm/D0o/H/dbc4hJf9+lex45GvYMcjX8VTD30RADBz/nm46oZbsPbSt4QNi/Thq+Uz+NY/vCx4zzA0dS5mL1iL1/7Wl7D+Ba8PVbke92/5xa3Y8gv7NfaaN7wLH/zzT9QfMNs6o0mRrcrD0rT6PcnbrcpjUTYpsfvMCFZPb8xcq9vBsmqK240X3vAzupN1EbJC7LU6o2nTF3O7cA6cRXPh+kbDOIdbcNOd0QoOSgOFVGe0YpFj6lQ31RltoCgxbbBqbfQwtXLc2hmNlccgTx5Pd0YrV1A5M5LqjCY8H5Wz5VRnNCkkvFEvltlu1pXbiDyL3DsVYn+N9yQDgN/8yAG6xroMOilt7fLaGegA8NRuP/Y+GUE3lXXY4tTSUMVx1HXqJpoVabe3gRIPk94KrmqoUixITBsMvBxcDyXHCwfMU/0TYSmaWx2Jl6OdORWUopUhKxUUNl6D8r0/gD9WjpoXGeVoftVLqXJzegvIX1febEOVX+w/goMjY7hp5eLc7Z5Xf+m/u75WjebMCZlotTNa1FIx3hlNu72lOqM59s5ovInOaOTBTugl1GpxqtZn9yVPdkYD1EAg2eI0dHtLtDjViW/WFqcZHuwsVOMJtzfItnmw10KzRL7jxGk8deI0Xrl8YW4i7xUQmU9C5FXlmfub7m+BKtfHNcN/oTmMqR4S/ZN5oCK0cjCT3qLkt4ybjMzujEYe7IReRr0Wp+GyjHI0vb0ZXnec4GEkvcUS3/IOmBPXWcztTci2ebC3285VSokdJ8/g1csX9nzmug00Z06wohFVnnyf9GDPrcozPNhdLq03mZqd0ciDndAjaEWV69dJVR4bPOfwYHcYGlblNg92dMCDvR0KvioEKr7Aq5b3di15LZAyn2RoRZVr5W3uU0uVA7Crcm5X5bocLa3KRYYqr9EZrepZVXkz1q0aSZMYgFQ5IRs2Es6DWru1rsphV+WOXZVbO6NlqXLPD0Ls7fNgbxd+uvcQHjp0rK3H7DYQmU8iNJv0ltzXTHrT25hJb9zh4PrZSHpzHK4aqrjxpLdCkRtub1HSW7EgMOD6UeJbkPRWFGMoeqMq8S2R9IbKGOToaJD4FiW9iaoHT2eypxqq+Mon2kh68yu+odBljMh14pst6S0JInJCLbSqyrUHe1qVw6rKWYYq5yxrGsvSGS1ZjmZatwbz5TZVbnqwazSjyvMSvd7uieOncGBkDFcumJNrv14FkTkhE/nC61HiWyy8nvBgD8PrthC74faWDK8nLSXNjk3kwU7oVuRR5Y0QOWNRxrpeZ3qwM66uI8b1dcXj11Xg2VB0GYquymJXzVQkijkGzLw6Bl5Rg2VWGYOsVIBKRV1n1SpE0FRFTVNFiaahSvf9KCkOCJer1/ld4Bol8qoQuPfAUVy/dAGKTn/THc2ZTxK0Gl43tzVVuV6fDK8nPdiTiW+28Hrc8S3D7U14+TqjkQc7oQdRK/EtfF83vB70K0+G1+u5vdUaMNuSS60e7Aiz1oHWPNhbhZQSBc7xljXLerZ5SiMgMifEwHmcuGuhrirXofacqryeB3tHVTl5sBPahHar8nBZDQ92M7yuB9DJ8HoeD3bbgDkzvG7xYEfM+S3t9qaRh7SbJXq93QOHjqHIOS6eOzPXfr2O/o47EAB0XpWbBjL62fRgt5Wj2VR5Hg/2VHYtebAT+gR5ytFqkbypyjmrrcrVs92DPbcqt3mwaytXtObB3ioOjoxh65GTfefyVgtE5oQQ9VR5TZMYiyrX83n5VTnsqjxWU+6hlc5o5MFO6CSazWBv5BhJkxibKmcZqtwJVLlrMYnJGjBnq3KLB7uUmaq8EQ/2VlR5VQjcsfsArlk8F8PF+s57/QIKsxMAxDPYNUxVHi8149ENI/Bgd1xHkbfL4TgOuM5cNx+GB3uhwFEqcZSKylZyoAiUChIDBYEB10tZShb90ZSlJC+PgpVHesaDnVQ5IQtNq3JuV+XhfHlClUdRMNHAgDmnKk9mrTfpwd4qOBg2zZ+Nc2cMt+2YvQAi8z5HsyH2ZHg99TpRV67W53d7S4bXwzB7IryeZV6hQ3+28HqoxnOG13XiWzK83qzbG2FyIq8qT5WC5iFyi3Wr9mDX01ZJD3Y3uM6SHuwDBd/qwW4bMLPqmCr5rFZiHuxSZ68HhB56sGuVLkVDHuwmWkmKOzgyhooQk47IAQqzE9DbHuy28Dp5sBPGG80SeZ7jTLgHuyW8rhPfkh7soUK3hNfzoBW3Ny8Ir496fv2N+xBE5n2MdqvyrvRgz1Dl5MFO6AXkUeVA+zzYHY5cA2Ym/Hj0y+b21oIHu22dDY2o8gcPHMOMUhHnTqKkNxNE5pMcPe/BnqHKyYOd0GsYDw92l+cfMGd2RkuWo5lub8ElkSxFU8tql6a1osqFlNh7dhTXLJrbd93Q8oLmzPsU49kZDeigB7uo48GeocrJg53QbcgzVx6uq6HKgWQ5Wj4Pdp34Vm/AbKpy5lWDQXJalds82FWGu2nXmiZ1E+1IfJNSwvMFXr9qScvH6mWQMu9DtOrBbtYUCyeXAAAgAElEQVSNd7sHuxgjD3ZCH8ByzWZ5sKsprcY92MNytBxNi2p1RstU5YlsdhOdVOXbjp3Cj3YfbHr/fgGROSFneL1LPdgt4XXyYCeMJ5pxfIvtYrFGruXBrq+58fJgl5VyyoNdZ6+nPNhj2erxqpFGXeDyqPaRqod79h3BZfNm1d2230Fk3mdoh9ubuc+EebCnzCsyPNgbDK+TBzthvJF7aquOB7vehjO7BztjOd3eMgbMyfA6jDlzGV5jIl5XnuHB3mp4PW/4/Rf7j2DtzGHMHSzl2r6fQXPmkxS13N4aVuXkwU6YpGjZ8S1PeD0Im8cG0BYPdsexe7Bnur1ZmhbVa3Ea635mCa/HzGNQO4s92qb5EPv5s6dj1iRyeasFUuZ9hMnowd6oKtcgD3bCeMB27dnL0YJ1dTzYk6pce7DbVLnD2+TBnqHK83qwJ9EOVS6kxKOHj2NeqTQpOqLlAZH5JESzndEA8mAnVU7Q6IQqN2EzicnyYE+q8rDNaVs82POpclWalq3K25n4tv3YKTx98kzNioDJBiLzSYY8HuxalZMHO3mwE9qLZlW53leTd/ieZ6nytAc7Z9kDZibUYJkZJC59T5F4lirP8GCvHVJvXZWPeT7u3X8E1yyevDXlNhCZ9wna6fZmC693qwd7I+F18mAntAvtUuUhkRt144ClM5rDo2mrwHQp9GB30x7sBVei6IrQg73IPZXFbhkw82oZvDICVqmokk8zg71cUdGvqhf3YA9yUpIe7CHZd1CV7zo9gtUzpmJOiZLeTFAC3CRCX3qwW8Lr5MFO6AY0YhIT2y5B5KExU3A4xhBObalBsxoka4XOeTwfxWHB9VZjwMxCR0UjvK5D7JbwOhAPo9ci60bal9bdRkqsmTmMc2dMBeiSjIGUeR9gvDqjdaMHe7YqJw92QncjS5Wnt2PhQIDzaH5ckboKrzNuhtcTHuzB9VVANb8HexBe1zXk6fC6iJWmAQGp1/j3l4ltm8H3du7H7tMjFF63gMh8kqAdqrwbPdizVDl5sBMmEs2qcpahyqMBc6TKo+ssGihHqlzaVbk5YA6I3OrBbia+1VDlNuRpc2rbpx52nR7B4bEyFk4ZyHXMyQYi8x5H33iwWywl83iwZyty8mAndAYdyWKvo8q1YVNSlYemMQlVzrmMEt6YD9fmwW5eZxmqHEZb06QqlzLKaO90m1MhJe7adxgvXDgHbs5722QD/So9jE55sHOndzzYUxns5MFO6DLUzGA3k03DQXJalfOQwBOq3ImbxESqXIQPxzZgTqhyM8xuK0fLo8ob9WNvBL6UWDdrGlZP0vameUBkPomQ1+3NfE0e7ETShMZQMypmurvp12a0i0U5KWH9eJCP4gbXmenB7jos8F+Pe7AXgwFzMSj5LInRaMBcHQOvjEYe7NWKymD3qnEPdvM684PKEe3DHmaxx6NgkGmV3upceVUInCpXccncmTEB03KEpM9AZN6j6KgHuyW8Th7spMoJ9ZG3oUrNZUiG16OuaWZ4XdeVF5x4eF1PY4XhdSQ82AMfdghfKfPgWouy1qVS4p4ftDSNwusAEuVoph97+5S4ia1HTuC+A0fbdrx+BZF5n4M82MntjdBdUDasSd8GxN7HnkPr1nh43SxFM4k8LP20ebDLoBTNqyoST7q9aVI3Et/M8Lq9BM0gcZkdam8GY56Phw8dx5UL51jXkzqPQGTeg+i0B3uWKicP9jRIlU8uNE0eOVS5zmIHYJSimdataVWuk970XLkTXGMxVa7L0YQPnqnKPbsqT5q/JE1iArRC2rX23XzkOFZNn4pZA8XMbYjQFcg0po/RrAd7lirvRg/2TFVOHuyEcUa+wXRyH7u9cnwKSzdUqa/KnWTiW6DKufSjzmg2VS6lVZVDmHPjtVV5J3DJ3JnI8xH6d+z0+XQzSJn3KVrxYGeM9YwHu8peJw92Qu+CMwRJcIF1q0WVcydS5pyZqhxWVR53VawGg2U1cA5VecrlTQbkHS9Hy6vK085wyZB7Y8lvjx4+jlHPx6CbvyuajhpORhCZ9xjGw4PdFl4nD3YCoTFoTonmvvVyFmWwayJniE1lKQXOAxfFyICp4DIUCtp/XQ2Yi65AyfXVYNmpGh7sYyh4oyhUR+FWR1UGe3VMlXxWK5DVMuBVISoVyGpVPQIf9tCD3by2EkQeT35rvie5DSfKFTxw8FjT7U1TU4I5H70MCrP3IVp1e7OG17vVg90WXicPdsI4I69JU+1jxKtDzPA65+p1FFZPGsVY6sqFacRUBRNBm1Oh5shD1zch1GsZDY6hyVkPmDM6o6XC2rpCrUVV/sDBY9g4Z0ZDqrwd6GVCJzLvIYyXB7ta30EP9gxVzrwqZMMe7HZVPhk92EUtY+w+BM+4HroBNVU5jAx2Q5UDMiTseL/yyJM93uJUxkrROBPgEGHiWyy8LvwwwVT3LIcxWA4T3xLKu5V2ps2iKgSOjVVwzeK5HTl+v4LIvM8wIZ3R+MR5sDeqysmDvX9Qb/DStWRfo7ZcP2tVzgx3t6Qq19eaur6i+XLORFyVpzzYE6o8JHLVUAVAWFuuXsfnytOmMO0NsbuM4U3nLqVmKg2C5sx7BOPpwZ6pyrldlU+sB3uGKicP9kkPIWXsMdHIGlxwzdxA/JpjaqAcRsPCsrRIlTsseASuii7z4UBdZ2ZXNFWO5oEJL1TlECIaNDfYUKVTme2HR8fw7ef2EZE3ASLzPkIjqjz5PldnNMeuynX4z27dmlblXGSocm0nabFubbQzGoCGVDmh/zFexJ6cdrURkxliBwAWLLOp8jC0boTYuVlfzoSRxR7kpEgfTAZ15fo6C6+16PoKVbkf1ZUnHd9sGezthHkfuv/AMSwbHurI5/Q7iMx7AI2qctt8uFlDHpajBc+6DC18NkrRwmYqwSPWUMUoR9OlaMVkOZouRUO8HE2XoqEyFitHM0vRbOVoUTMVEZF7UIom/Xg5GhCpcrMcLVmaBlA52mRDs6TeMqHZrmWLSVPo0W5aJfO0SQy3qXJjwFxLlUPKsKEKMlR5rUS2Rrul1cOhkTEcGBnDhtnT23bMyQSaM+9yTIQHe1Z4vSUPdtEZD3YgHV4nD3ZCXpiE3u45dn3dJI/Ngng5Y0GHQiCsI081KgobqqhytFJBouBKlFyBoqsGzUXuRR0I/VEU/DEUqqPg1bIqRauoB3RDlWp2Q5U8bU47EWJnnMEHcPWiuShQi9OmQGTeJ2irB7stvN6tHuwZndHI7Y3QKDSxdyxxjhkeDoY/e9okxt5QRYfXVdJb4MOuk96CeXJHenD8apjJDhFlrochdt2vPGESA0QZ7PXIud0h95Gqh/lDA1g0dbAjx58MoCFQF2OiPNizVHk3erBnqXKAPNgJzaGVOXVbmXLSjdF8y8Jns7Y83VCFsWie3GFQ4XUWv86UZat6jvmvh7XlFtc3o7bczGAHoix2E82SbL171o93H8T2YyeN7XvfxGW8QWTeB2i3B3vjqnwCPdhJlRPGGW1TjaEkN9ucxhuqxK1b421O60a/9FRWksSNhirxOXL7XHmnVfLhYK78vFnTUuv60amtU6Awex8ijyqP1qWtW7NUuXrOsm5Nq3I3Q5WzDFWu5vLa0xlNmIk6pMoJDUJI2VK4PUU4RogdQXIbEGStwwy9G3XlFsc3PWB2w7I0Yc9JSapyIUI1Hmaw665olu5oSbQj0Y1xbj3OAweP4dJ5sxqaKydCT4PIvEuRN8SeDOGliTy6cWjVrRPfuKOsW3mgwLnrhGpcNVJJZ7CXihzFAkOpyFAqAqUCMFCUGHB9DLh+5A2tE3Kqoyh4o3CCDHanMgIWZLHL0VHIShlirBw1U6l68HQWe6qhig/hCdU4Rahnv+KHRK5J3cxgr0Xk/abKJ8s843jdyFsldBtsIXb1RjnAJRNNkw1VeNBQhRsNVTh0iD1oc6rd3hKEHqstNzLYze5oGqZRTKexcc4MzBsa6Pjn9DuIzHsYnfJgzwyvd6EHuwZ5sE8etGPQkndAkJUUJ4XMdQxuZLSHnx0Ulatach7ZITtmFruZiwIUXYmCoxqqFB0fRUdlsRdZNWyo4npjcLwx8OoYuFcGq5bDqSxUvSiL3bRwTXRHy2p1qr+z+VwLjDPrdsn71LMnz2DJlAHKYG8D6BfsQvSNB7vM9mC3dUZrxoO9XZ3RKMQ+uZB0D6yHVo1mTC92IFDl3MhmZ1ECXDKD3TSJ4SxtEqMT30KTGFsGu+9F7UyNufJYU5UWv1+eZSaOj1Vwx/P7IUMvDJ7b0ZKQBinzHgV5sJMHO6E9MAm9FgE1GnbPY0mqL92k41tE6kYWezBPzpkuSRNwoMLrOsQOM4vdzGCXMhZKz5orDzukxRLg4mSfpbqtv0GNbR86eBQb585MtTlthtDHYzqg20Fk3mXoCg/2DFXekAe7GCcP9gxVHi4jVU5oAPr/KU8IPW+oPQmufdYRmcdE3usslsGuCF1lsTssPZUVqnLhR17sIakrhzdIo5ZcO76Z30MaBB/7fo0RZNZvYS7XnzHqeXj25Bn8xvqVDX1G9meToicy70F03IM9Q5V3owd7o6q8nzFZEuDyopVEuWaJuiYML/ZkiD1YbZSkxXuW81hjFRGpc53BLoM2p7pnudkdLcxcj0rSUiH2Gv87Ogs9lyJntdW4/k0HXRdvX38OBsa5X3k/g4YzXYSu8WB3ec94sGsiJw92QhL1Ijx59k+i0bnzyM7VWKiJ3AixQ5N3kJeiS9PM7mguixzfwp7lUqtyz1DlgQ+7NoExQ+yJVqfNwrzPNIqzVQ/3HziKoWKBasjbCFLmXYJu8mDPCq+TB3t3Ixk+7TWMZ9vLWoRuCwuby8258zwK3lwfTnvpEDuPWp2GteRBJnvBBQquRNFRmewF7qPIPRSYB5d5cEUVrl8B9ytgXvCoVlR3NM8DdMWIEQELk+Cg58ezy89sdeGm6tav80QHTWw+dAzVnL8/IT9ImfcYyIM97vZmgtzeehtSyo49GjqPOqVZQFqh55lf1mMVhkh56wXarpWF8+TKslUtj2rLORORD7ueyhI+mBRKlUsRqHKRVuVAGFZvfD7czMVh1tfqO7KaAmTM8/H40ZO4dMEsqyonld48SJl3AbrNgz1Lldf1YE+p8s57sGepcnJ7I5jIIvSsa288EuHCufFYEhwLlsWz2LUXO0fQWEX44FKEYXZN5GHim05+MxLfGipB4yw1j24q9aRC17+DuW0SUgjsPTOC1TOGMb1UzPcbEaHnBpF5D4E82MmDvRZEjdBlPyHpetgKkiSfJHcbYdnC7RpC2puthLJcrzSrTIwyNK3QbU1VdDkaR0DiUinyeDmakcFu+X6NRinUvUYYv4Od0MPfKLi/ZBnGrJ41HatmTqs5pUJlZs2ByLyH0SkP9ixV3o0e7KTKJx9aHbTUGgxosqtF6vUIve7nh+F1FobYGYtC7JzFQ+ycxUPsLCRzFVpnUihSDx/6+kk0UMlJ5IxxVWtuqPO4Eo8Tuvn7hMew/MZbDhyDkBIXzZ9V+/OpzKwp0K82wehGD/ZCMx7s3sR4sAOYdB7shNYggmkb85FEvfl2m/JsRFEmuS5ZX246vpkhds4EuBRgkKHrmwqjqwczCD2cL8843ySy5qmTgkAvS86jhx3fLMfwhcSDB49iwfBgrrnyvA9CBFLmPYBx92C3hNe71YM9i6gno9tbr2ezTxQYY5lqn3MW/10DvjaVquAAF2a2O8CFTEklngi16/lyzhELtWsid4P5cn3NucyHA99IfItanZpmTLrVaa4a8iCuLyEA4ztAqJOXPFDg0IQeH7Qw7sQHCmGJbPxzfnnoGGYMFLFo2pTMc2kGROgRiMwnEN3qwW4Pr9fwYLeE13XimyQP9nEDmcZko9ZN3zYI0teY/v/T15KUqrNZVtKbMlfJvn5jEXbDRCZuGCMj97fQIEZG3dFCL3YBptW4+vCofrzO/0I0F2685zI0hwkhjJB7cFzz+yW316JBrYvOoSIkrlgyj0LoHQSReZdjQjzYM1V593mw2zAZVTmhNuoNdJLEnJw7N0k9SehSyJQ6bwSm61v0Xkbz5gA4JBiTiryhw+kyTHQLn1E73K8H+tL3EXwhsMAhDpyBIe72xjgLVToQKfXweMnwgz6+QfKeELhk0ZyGfxdCYyAynyB0swd7tirvPg92UuURKAs4H+xlU/YELhup2wg9C1JKZRCTdS7GKhbUnrPYQ0aPgMRDQtcx/3rTK5yBSQYIhnDLMDlWQCIgdL1tSNDx/ycpJFgifp4kd1VEz2Ik/83tz+GFS+dj2fSptc+T0BKIzLsYE+XBnqXKyYO9u9Fqm85+QJ7M8lqOZ9E28ZIrk9SzCF1ICcVtAoJzNW9uWI8LKeHAlN8IjxlOnelsdkAluQXPsUeQva6JXZ2wEW4PfwyursXEMnWf0OrZIHTdcAXxefIkqccg4vcl9V14SPJ7Tp3F2aqHJTOmxkcuhLaDyHwC0O0e7MUCesaDHYhInTzYCY0OaEzyT1uXBklgZnlWQOCNKnRIWZfMuKHGgYQyN4gcSHxHmT5vqc+FK5KWQgJcKgXOOVhyvhwCENE+QJrUk8lvVugpvWAU8/D+o7h88Tw4nBqqdBpE5uOMXvBgt4bXu9SDncLrEYRHYfYk6s1h+wlijG3v+wmyD4gSirh9XwbRL4TiVc+da3UuhAzJGUjRcCKkHi3T9eUhiTMVVg8D5br0TJ00wBgkU9ezes/Bgj7hElHinQq5q4EF5zxqvGLOdSMIqes+40adOjNCDXrb2PdxOHjRVVNpUuKc2dNxwYLZcBxKfOs0iMy7FBPpwW4Pr5MHe7djspSmNdKQJW+Gv92SlMWUPmcsDE2HyW9SQghYCV1PL0spEQjjYIPsczIv7ZDEGcCSwwCT/aMTTky8c4AJFTnQ8Xuh7gtSJ74FKp1xB8LzFYFLGVPhUitz/TnaMMaJq20dpVCZ8cCxkTFcsmx+Q38vgHI/mgWR+TiiVzzYG1flE+fBXg+TRZVPJrRz0KKvK1sCnI3YNaErcGM+PTpONDAQkJIBpn4VEnBYoKrbNIesiZsbhB2E2pnk6vykVIobPpjrqPNgLAz/SynBAzYIv3d4zzGuOyGARMg8nIYI9mGOg9NjFfz7ozvwnis2pHuW1yHr5CCBkA9E5l2Iifdgz1Dl5MHe1RCkaHKD63nhrIFBxk+p870iUlcLBJOB+Uu8VE0KCcH0nHWCvCPjtpqQEpB6sIBocCAZhwzD60qVx0LtASnqT9XDB4lgXp4zK6nriAITloGORZCYZM+4cph8ZP8RbFg8F0ODpfT3EW0axBBiIDIfJ7TSq3m8PdizVDl5sHc3yDSmNszpqVoDH855jOQ1QZsISV2H3S3hdjhJ8lbhdiaZlcCljEfgJVig7PV7I9uecfVez5c7jmp9qkPtjgN4HGAqoz48ir5vcAkpWIrUGYwBjrFtuDscq7I2kwTLQmLrgaN4x6YNVpVdT3lTmL05EJlPMLrRg71YENke7NXu8mAnVR5hssyZNwJzEJ3Xn9wk+iSxa8vTeDlaoNBVPplS6FDrFHnLgMjN9LHgnGT8ObkciEhdggVqnEMwB5wxCO6Ac1e1QOUu4EhIqRLTmOOomfZgQMAASIOwlZVrQOoOojI2/Ts5irhjBK+XAzFSZ47+/ThKA0W84dLzMGN4qO7vbUP6VyLkAZF5F6FbPNit4fUu9WDPu34ywPf8+hv1KJpte5rnvyJWX+7HFbkvfGM7dQ5cBgNrHYbmmn4iQgcEmGAQzAfgwBeqWYpkgGSK5P2gvE0IGahyFvRGUeTtC6Yy4iVH0PgUPnPBubJyFU4BkBLSLUIAYEx7SjiQflURugi82l01YNbvmZCAFFFGetCURQZz6zH1bRB5pmrWtsuOg2dOjuC8RfNz/PIJJOvkCQ2ByHwc0HMe7LbwOnmwEyYQnerVzjmzEhTLCLWbyl0PntUcuV2hAwYBBupcK3VIVRcvAxLXDq1Cqki99nhT65kideaoB3cghKOImzuAE7+VMwDSlWrAEcyJM99XTVVUqrryWdcEGpC7lcRNBY5aeQYCh8+OYsu+w1i3bEG+PwChbSAy7xJ0lwe7XZWTB3t3g+YaGwML6sBt4MkMOH1NGl3ThBBRIl2Y9BYn9ND1TZenCQnJWUT2RhKcUusw1Ll69hmDLxkEHAj4BqG7gUJ3072sGQeTQiWbcVfNp3MHEL5KlpMiNsgIyV19meg4+veRcUJXixJ2r1Jiz/EzuHLdSrBi0VyR8RcgtBNE5h1Gb3qwZ6ly8mDvZtCceT5E5WjZgx+RoMeQ3BOkLrRLWnDMJKFLIcOwOpcImVtCBmH1SJmLgMh9oebbfcHgB+1JheTwJYcThNoZE+DcB1CA4xeCcwy/IOBXASnAhAsID1I6QWtUB4yrFqnMbNRihN1DyDSRRz+QTBXWPXfwKJyZDlYsnNtSwi+hORCZdwG6zYM9S5V3owc7qfIIlM2eDfOaqTfoYSweejcVfBapSy4TpjGK0KWQ8BGQJONBQpz6P3e4NObJFeGb8+a+ZBCSwRMcLuPwpaOIHBKcqdfgUHPnAbg6YUjG1WsRELjvK2UuBaTjh+SdDLMz87cx1yVhIfiVSxZAnrMYfMd+ihJNAIjMO4ie9WB3eseDPYnJqsoBCrPXinblzWQHYJkrN3wPAv1rkrryag9C7glCl1KRvBCqfpvLQK0jmFcXMpw3DwJc8IV6CKGT4Bi8IAnOk07oDIegDM7nbizTnQnV71xAJcAxKSCDpLlw7lwKReym+raF1GsRurHt3qMncOzUWVy8oQSvUMy2w6HoUcdAZN4h9LYHezq8Th7s3Y9J3zUt2SGsDpId1kyxGTZZMfLhGWeQvh9kuQfbCF9dczKyeOWSgwmmHNiqPrjDIX0BH1AXGwRE8J5zDt8T4MyB50lwBlQ90zUuMKWRDC4rouBwCMbhMQcF5sB3XKXCpbpWmfTh+FVw6YO5Hrj0ACnBhRFaDwhefUEjCU5n4CEicRlsk/lDBbj7wV/inIXzIEsDEMMzVLY8YVxBZD7B6EoPdkt4nTzYux+TXZnXgu36yhr8cJbOcE/6sasDBFND4EqpB8pcIFLpUghIHpjESBmG2/VculmeJoSaL/f8oHtp+FrlwniSgwkH5nS+BAfjEo70IKEGEpI5YNKHyyqQ0lXJd8JXHuxSBGo9UuHma5O4TcK3/qbB+iMnTmHP0ZO48aUvhuQuZGEgXhJIJWfjAiLzDoA82MmDfSLQzXPm9bqXdRp5Bjr6mkuSfNKPXW1rtj8VmYQupAQLQumOULavjCM0k2G+KmvzhYTvM3Cunr3g+uQsIHNwVH3j/sERdjjn8CEZg3A4uBTg0geXPoJCMjApwIUf9ULXtfNSF78hDMGHr/VvYptDT2DHgZ24eMN6OINTIR0XojBQ97cmtB9E5hOIrvVgz1Dl5MFOaBbdONBIDjCy+pnHOqclSF0/MahpI5PQo2x2RZRSslCdS1+oOW4nMI8REr4fuLAaz2qQrkjdEzxmjS6ZMqxhkHDhg8MHZ0I9pCph41KF1Jkj1eBBCjDph6/VuRskblPmQLY6h8Cll14MKSU8xuA6LrziYI5f3/IZhJZAZN5mdFqVj4cHe5YqJw/27gaF2fOXfALZAwyz61nyuGbntFjoPXhvEroQqv2o4DJogxqpcyElmASYL+AwtZ8vAN9XfdKrHjP6nKvzKTi6OYwyknE4gwiIWkUGHDjKJw48qEVnEOBB1rpW5kq1I3hvqnE7kbMaYfb7Hn4Uc2fPwqoVy9Tvwwvw3GwyT7VyJbQNROYThHr2lDVNYiyqnDHWoCpHZmc0LqrY/NCDWDF/FpbOGk6Uo+VX5eHDZt1KqpzQATQ6oLGRf7L1qXlcFoTOgYRK51A+5qY1rJCQPBrQMmaqcxmqc18iGAgDPpfw/CDznSlC5wGh6zC7Cq+rCnfJVLMUziQcJuBLlUzHIeDCM3qi66C8VGod8ZC6Jvrwe2cQu4lKpYz7HnsCb7359agWFIEL7oava/7ukz1ZswMgMp8A2OrKTVUeekAb5WlmOZouReMuj5WjxR5GOZouRSsVWawcLSxFS5Sjffxvb8GnPvevcByON7/qOnzwbTdi1ZzpqplKUIomy+VYOVqyFC0sRwtK0fSzLkfTBO5XRawcLVl+liTu5Pu8c+qTAWQakx0Zy0IW+YfZ7AlilwZhx/ubR4Qemz8PjqHMYQx1rufMA3XOwOH7Uvm9+BycA17QyZQHCr3iMQDKZlY6KsNdBN/XYRJ+EGJ3AvL2mSpjU0VtMiRoDl+VtgHRs7Feva+vzB/avh1LVqzC0Kz5qATLfF5AxckfZq8FCsE3BiLzNqJfPNh37doFAPB9gS9/54f46vd+hLde90L8v294GVYMul3lwZ6FyRZiB7onzN5IqLvdaMeAJllbHkGTsx8OuEXwXWOub74PKRgYl2H0TFVqqCp1HxzMUw1YAP0sUKmoBHnXFfB9BlFkKuxeUFntUjJUPQbXkXA4D3Neio6jlDlXCp0x1dSFM/V5oSoPn6HI3SBy8xmISD58bwmPHx1hWHPxNTgtp4XLBmUp9p4wfiAyH2f0ggf7X37w/Th+7CjufnAzAEXqX7z9Lnzljl/grS+6CH90/RVYPm0KebATrOiWQYUNeQYaSdOYcLmhypNz5Xo9EL9mtWmMhAws0HW4PbgmGA/D7cKXEEwlxDEBeJ55TioRTgTz5X7wWk2jczhMwhOAwxVhK2LnKgSPiNABpeIBpMmcmcrc/A3Sv5EQApde9XIAQMUYlPvSQUUU0zsQOg4i8zahnzzYF8+Zge/889/i3vsexN/8y1fw883bAAC+EPjizx/BV+7ejDdfth7/85qLsGzKIHmwE3oG9cLqqe11gpjh524jdBFcs8mQu+oV7ijihgMZEDmEDMvUmJSAL1WYnUt41eAYCfivnVkAACAASURBVHguC5uyOFw5xPGgRI5zReCeYOBA+D5U4mY4nUXErT8li8iT6/Rv8l+3fgHXXv+rmDljdmydLznKfgGE8QeR+TiiVzzYdSna1RvX4lc+9se466Et+OiXb8PPHtsBQCmHLz+wDV99aDvedOFqvG/T+VjoOOTBTuhZZJWlhesNUjeT4TShA9EceurYwb6mOjdNZKRidEgp4fsCvq+uUcEAP0iCAySqVQbpKDL3Da8IgEX16EwROReqZl2rcnXuCMLwwXdEtByICD/8DTIyz3fvfBICLgamzsOYH/++nuAY81ujleTggZAPROZtQKvlaI2ocgB2Vd6MdWvOzmhXr1+Jq//svfj5I9vwV9/4IX7+VDCnLiT+/dEd+PqWp/H6tcvw3g2rsXiglMu6VSNpEgOQKm8Wner53Q+oVz2ShEnYtbbRhA6IMNM93gZVJaBJCaXUA3WeNJGBUImhjDnK6pVDtSoNCtalVCQtJOBKNThXHdbUuTCmrvMo+x1wOAsy2fV9Iwi3a/I2votJ+KnfziBXKSU2P3Qf1l34IlQM0tb7+ZKj4ju5f2dC+0Bk3iLyEnmtDHa1jMWJnPOQyENDGP3sqkYqjtlQxXXizVSKHKUiR7HAUCoCpQIwUJQoFgQGXB8Dro+SU0WJV1BkZRTFGIreKAqeaqbiVEbAyqNg5TGVxT46ihetWozv/Pbr8PNfPodb7ngAd+/cDwDwpcStTzyPbz25C7+6YhHes2Y5FhWLEJ7qFuVXFan7Fd9Q6DJG5MKTKQ/2LMImIic0isy+5XVIPknq2vXNXG/OmdcK1zPGQnWeNJEB45EyFxwsSJgLu6mAw/OCdqlCqiYsXJnLAIGpDENA6Eqh+6FqN9R4sE69TucG2H4Oczvf9zF38RrMW7oeZc8yFeAzjFW5dVBA6CyIzMcZ/eDBftXyBfiPX38F7n56Lz5216O4Z88hAIrUv/XcXty2cx9eu3g+3rliCRY6BfJgJ3Qt8pJ8LLRumUfXiW/h/LlugcpUqRqEgGROqM5FcD1wcPhBGZhX9cGFhOMErVMdVa7muupZSH3NByF1rp4dR6vwQH0b5WxqmY70IfYcN6RKfP8MMh45cwbLz7sWZ8fsKr7iAWdGSZlPBIjMW0D/erBndEZLeLBfsXgObn3dr+AXO/fjE/dvx30HjwJQpP6few7g23sP4FXz5uIdSxZhAS+QB3uH0c1Z5M1gIkvcRNAApRaSKj21PmiyohuymOpcv4+7wqlkON+PK3JAQEp1rTtGFjsPHlIy+EwaRM4MIlfLABiKPP6sYVfl0esTR/fjgZ98Fde97v2qkYwFnq8InTD+IDIfB/SaB3ujndFeMG8WvvjSy3Hv3sP41LZn8MCREwAAXwLfOXgY3zt4GK+cMwdvn78gJHUNUuWELLRrcNLsoECrdpPUk6F0k9BrJcMJqWhZSqj2J1KCJbp+h6VqgXNcRO5qvRMMhh2HQQrAcVS3NSkD4uYAE9oRLiJ2+HFSjpR58ndK/wbmNk9u/QWWrbkCvuDwM/40QgCVql21EzoLIvMm0c8e7FmqvJ4H++WzZ+DzV16EBw4ew6d3PI+Hjp8EoGwxvnvkCL5/5AiunzkLvz57PubL6F+PPNgJnUS9TPV6SKr0WnPjNdcLAegyNc6sFq8+BHjIlEqVOw5XY2tfeb1LCXDHUPommbOISB2HAZAp8k6SdnjvCQYNtlvb2dPHcHDvszjv0htQqUbLk9v6Qt0yCOMPIvMOo54qn0gP9nZ1Rkt6sF8yYzo+c/H5eOjoCXz2ud14+NRpAOpe8b3jx3D78WN4+fBMvHl4LhYV4gYTpMoJnYatgUo91Au7JxuvZG6XTKCLWbxKcIHI+jWcz9eErohZzdlzSCOEzrgiUj1XrsLvwTrjdLiyoEuIEZmppPVXcdwhXHDlzQArwqtxXQrBUO3g4NoWPRgfdH+ogci8g7Bd/N3kwV70R1HwxuBWR+FWR+BURsHLo2DlkbZ4sF80dRj/Z+1aPHLiFD6/bx82nz0DQJH6908fx+2nj+PlQzPwlqlzMR9xo4m8HuykygmtIE8JWtZ+WRnuGrqXuWqFKnRReFBrDkgWhNpllKUOJ8hWZ1E2O2MMPkQwkFfbCiFUJE5G4X3OVFtU9X3Sc+Rc5eEF55o0hsn+rtXyaYyePYEZc5bBq+PYKATqbtMS/M4dutdBZN4E+sWD3dbiFL49vN6KB/tFU6bikyvPxeZTp/Fvh/dj88hZtQ2AH4ycwA9HTuBlAzPw5ilzsADkHkUYf+RR63mS4szjJY+j583DbYLQOrgifp3hHnZqk+m5+HCqjTEIH1F1CzfKy3Q0kMUjf+ap2+5hWar36cd+AQAoTV1sXW9OEXo+Q7lMA+yJAJF5h9ALHuzJ8LpOfLOF19vhwX7RlKn4+OAqPHrqNL5w9BAeLUekfvvYCdwxdgLXFafjTYXZWMjJ35kwMag3J97odjHoeXMoVR4mwmmXuGBALDjAA4XOg8R24UsVWtfd2jgDkwjVuubUWOmcHw1ABEwFbpkOSCS1cQZUyyPYv3MLLnvJe1IlpdFupgkUg+f1V1VFr4DIvEG0W5VPpAe7TZUzrwqZocrb5cG+cXAq/m72ELaUz+JLpw5hS3VErQfww8pJ/KhyEtcVpuNNRUXqFGIn9AKaIncDIshU18cSnINJCSF0iFwTO4Pgwfw6gjo1P+iYxlmY/a7nzn2DhGP3L792pEEA2PPsw5i14Dy4xeHYcbIghcy1HaH9IDLvAHrNgz1VjpahypMJb616sG8sTcEFM1dga+UsvnzmMLZ4BqlXT+JHVUXqN7NZpNQJfYPQq10AzAnC6JrEg4Q4rkzhojC8UE5sAggJ3UdgBesLReIS0GZtnAHgCIkdSM+VJ4nehoUrNsH3q7kJWsj6xyR0BkTmDWA8O6MB4+/BHpajZajyevPlQHMe7BcWp+CW4SFsrZ7Fl0ePYKufIHWcxEud6XiTE5E6qXJCJ9Gqym7484y2q1EinHaTC5LognaqgQWNul8YKh1AOCmfInYgNKvR9xffkkxm3uMO7dmKwSmzMTxzcW6CJmU+cSAyz4lWPdjNuvFu92AXY+Uoe71cgah68HQmu5nBXvHgV/22ebBfWJiCv2FDeMwbwVcqcVK/wz+JH/uK1G+WM7GAkVIndBeaIf+Y6YyU0CVQWp0DEaGb8+cIntIq3TCrMYgdMO5hCbI1A4Wa8H2vguce/xHOv+KtYWKe/TsnjG8QH5gQxg9E5h1CT3uwW8LrMiO8Hia+JcLrQPMe7Be4Q/gbdxm2ls/i372j2CoSpI6TeCmbhpvZLCJ1A5PlJlrLQrVb0Aixm25wIjCSARAkwQGa0JUbnAytntU1x6JIntSJbjJMflPnkv7MmDud5Zz2P/cwpsxYguLgbPi1uvEl1gkhKQFugkBkngPtcHsz9+l6D/YGw+uqi1M8vN4OD/YL+BD+ujiEx8RImtTlKfxYniJSn4SwDVo6QfDjHWqvBW31yjgLDGXU8qRS14idtkj/NqGSz8CxQ89g2XkvgZAypeSTiPVwl/GkV8L4gci8jajl9tawKu9CD/ZMVa5JPaHK2+XBHiP1yhFsxajaB0TqBIVkJ7NuAM8SAZblWQQYJcMxVace1Jzr8HyY4Q7E2DyT2GMHz/6t1l76a8oqNofI9mOlaekEuIlzbZtcIDKvg8nowd6oKtdIqnIAVlXerAf7+f4g/spZisfkCL4mjhKpE1IYT1JvtFJFZZjX8nSPJ7rZjqe3SxM6i313fQ0yzqxtXs3tTQjfw5MPfw1rLnkjHLeUea7ZkLHj5h0QEFoHkXmb0GxnNKB3Pdhj5WgdUuU2XMCGcIEzRKROyES99qR50K4QuzlobwapZDhmJ3QAmaQeO58MggeAg7sfBXOKYLyYuY1GHje8yZLH0Q0gMm8D8niwa1Xejx7svidCAverIiRyIURI1FmqvBUPdiJ1Qi20g9DzIEnW7ZxntzVuERIhoQMIs9azSN08T31MG4TvYf+z92LVxptiyj4LNrKX0r6c0HkQmddAO93ebOH1bvVgbyS8bnqw29zegPzh9WZBpE7IQqfC7vVC7LxFNQ5Y+qMH6hwARHBsG6kDCLPeE0dUm2UQdKVyFrMXbsCUaQsjY5sGa8ZVxI7i6hMBIvMW0Zce7JbweiMe7PXQKXInUid0G8KBezBfrq53gIGBBc1Rmhlo6Ex7rdLVsriazhrI2JSz8D1wXsLClS9uqKEMoXtAZJ6B8eqM1o0e7NmqvDEPdqAx4m6XBzuROqFV5HZ1tJSeNvd5CfMV00zGUOjx12lC19uYx8wKt5s4sm8Lzp7cj3M2vAZA86HyeB92wniCyLwFtEOVd6MHe5Yqb9aD3USnQ+4mGiV1X0o8gLOYAQfr2GDHz4/QW6ilVpMh9vFGktCBNKmHyxPkLvwq9u+8D6svfF1XlvgR8oHI3ILJ7sGerchb82CvhU52RstL6vNRwJflUQDAH7AFeAmf1vJnE/oPyfuAfSDPUyRa/7j5trepcyAyY8si9azPOLTnEUydvghD0xZE+zSZhU52rhMHIvMEOuXBzp3e8WBPZbC32YM9ifFqcVqP1M2//D/Jg1gtS1jGmqm1JUwG5Amxh4N3c768yVC8kJHvehahq+3Us21sYMtkH56xHDPmrmnqnAjdAyLzFpHX7c18TR7sEztyzyJ186zKkPhLsQ//wJdjgCysJj2SPhK23BgAqfLTLGQlvyXfh/XkwXMtQk+ej23qOknwp47txNDwfLiFwfbYsJKd64SB7lIGOurBbgmvd8yDXfaWB7sN49Hi9AI2hL9yluLP2WJMh5Navx9V/KnY0/HzIPQ+kte7ho7EaVVu37f5+ekkcWpSz4KQ0aNSHsHObd+BXx1r+vMJ3QNS5k2APNjHz+1tPOAxiZPS0twZwBMYgy8lHEoImjSoNaVm3752Imzy2Ayd/V+yqXQbDu66H9PnrkVhcKZVxQP2UD2hO0FkHqDTHuxZqpw82NMYD1VuYh0GsRgF7EU1tW4xCkTkhBiyQuzhesPhsd4xst7Xghlqr4VaXd98r4xjB7bjvMt/o85n5T4t9Zmy8X0I7QGReYNo1oM9S5V3owd7piofZw/28cIwc/BpvgLHEVfnrlTrCL2NRogyT6JrfPuoIiXreFkh9ppz6g2cs83yNVpnV+mOW8K6Te+CWxjI/TmE7gaReQNoxYOdMdYzHuwqe717PNjHA4wxzEpeDiTICQnUMoWqV45mC7EnGy41i1qErtZHg/Dy6DEc3f84Fq/+laY/j9B9oAQ4jI8Huy28Th7sBEJn0QlVnmxbnFpvXvdGFC7aNn5etbPe7euEpZY7OR2Whb1P/wyuOxhc49kPQm+BlHlOtOr2Zg2vd6sHuy283kUe7ARCtyBO0rXnyrUfu7n9eOPMid0YPXMYK9bfUHfbZghdSjmhA4F2dqzrNUx6Mh8vD3a1vlc82O2qvNs82AmEdqHZuXLAPl8elqTVmJpLLEx9brOoFXIvjxzHopUvBnf689Y/mSMK/fkXbTMmpDPaBHqwN6rKJ9qDnUCwoe1tT1kyXJ5W5TbwGuVotn3acd42Qve9MmYvurDlYxO6E5M3JoHWytGS29XzYM9U5e3wYJft9mDPUOVd7sFOIDSDmiVkDapy07I5bSRlP0azsM2bm4iXk1bxxINfwNjZIy1/LqE7ManJPA8aUeXJ97k6ozl2Va5ep1W5mi9Pq3IuMlS5V7WqcpGhymt1RgPQkConEPoFtXzYbcStwZNNWTocYk9CE/rhPQ9jaHgeBqbMaftnELoDk5bMG1Xltvlws4Y8LEcLnnUZWvhslKI5iZK0WEMVoxxNl6IVk+VouhQN8XI0XYqGylisHM0sRbOVo0XNVERE7kEpmvTj5WhApMrNcrRkaRrQ/eVohP5F3lC17XrP6/BYS5UnzyWmyo0IXu1zaw+5V8bO4NCuh7Bo5TVtOR6hOzEpyXwiPNizwuvd6MEOpMPrvezBTphcaGXO2UbktRweY/4SiZrx5Dy62VglXK5VeQddBh2niOXrX4PiwIzc5WuE3gMlwNVAWz3YbeH1bvVgz+iM1g9ubwRCI6hl3WrLVmfGNR8tS5D6OJakjZw+COFXMDxzeWy5JvSJKI8jdAaTTplPlAd7lipv2oM9pcrb58GepcqB3vZgJ/Q/2qHKax7f0h3NvO7jx8sxX548PovfP1qBlBJ7dtyBytip7G0SCa+E3gUp8wy024O9cVU+gR7spMoJkwCNeknUKkENl1lUeY0PAJAvxN4MuR878BgAhpnz1+fa3kbopNx7B0TmOZFHlScv+jyd0dRzlnVrWpW7GaqcZahyWa22rTOaMAwZSJUT+gl5M8ltqtyeRMetKrumd3sb582llDh2cDuWnHtdy57vtUBk3z2YVGSeN8SevLBtI3QzAYY7PEyA4Y6ybuWBAueuE6px1UglncFeKnIUCwylIkOpCJQKwEBRYsD1MeD6KDlV9eAVFFkZxeooCt4onCCD3amMgAVZ7HJ0FLJShhgrR81Uqh48ncWeaqjiQ3hCNU4R6tmv+CGRa1I3M9hrETmpcsJEoRUf9uS6eqq81j71zo0xVlOVt4sgV298U9uNc5JIkb2sPwAgdAaTbs68HjrlwZ4ZXu9CD3YN8mAn9COabaiirvdoWbKhil4evweYA/9sl7hanNsouZ89tQ/PbP1Gx4mc0F2YNGTeNx7sMtuD3dYZrRkP9nZ1RqMQO6FfkKehiiZy+/6WBLkOkK0QPnY/dQdmL7ig7ccmdDcmDZnnAXmwkwc7oTfRrElMI21OazVUSZ6L3i9XclsbOf3w7gdRKE7BjHnnte+ghJ7ApJgz7woP9gxV3pAHuxgnD/YMVR4uI1VO6GPUtm61h8ptqtwMz8eOy1gmydeyh86DKdMXY+b8dRRin4QgZR6g4x7sGaq8Gz3YG1XlBMJEohXr1lrrkqo8Naivo8rHE1JKHN2/FUPTFqE4MH3cP58w8eh7Mu8aD3aX94wHuyZy8mAnTDbUUuW29XpZI6o8vm9Lpxvi6L4tOLr/MVLkkxh9Tebd5MGeFV4nD3YCoXm0s6GKmezKDNXNbYN6bg7qzWm39Hx5qNTrEHmjIXZ9/MrYSRzY+QssW3s9GOvrWzqhBugvD/JgT7q9mSC3N0K3ot0NVazbNRBeT+5X65iWhfHjNRCqP3nkGcxdehm1N53k6NsEuG7zYM9S5XU92MX4e7BnqXJyeyP0K2zXfGy9JfFND+B5uE96vjyvKm8WvlfG3CWXQEq6xiY7+pbM84I82MmDvV9w8Yb6iU+bHz85DmfSebTi9lbPutUMk9v2zaPmbZ9rrkssrHm8rM8YO3sEz2z9Js57wTvhOMWGj0FQOG9Ff/x2FGZPoFMe7FmqvBs92EmVEyYTarU5VcuzVXlsmwxSbnQ6IE+IXQgfu574PuYvv4KInACgT8m8Gz3YC814sHsT48EOgDzYCV2LdqnyWjkyNlVu2rlGx4t7sjesypvEoV33wy1NxeyFG9t2TEJvY9KG2cfdg90SXu9WD/Ysoia3N0IvIQ+RJzPYo9d2D3a9j+nBrrdLHdtQ9PHonnlSzYXY5yy+KHb+BELfKfNu9WC3h9fJg51AmGhkXvM5wut6OyBB2HlI2rJNvf2qlbPYue3bcNwBuIWh+p9BmDToOzLPgwnxYM9U5d3nwW4DqXJCN6DdHuzRsmwPdls5WjK8XuvYsc9uQUhLKbD7l/+N0tBsqicnpNBX/xHd7MGercotHuxyYj3YSZUTJgOSnc/yerDb3ttUec2BRxPh9QM7fwHGGBasuLLhfQn9j0k3Zz5RHuxZqpw82AmEfOgWD/akKk9at5poRJXXC7EPTp2PeUsuI1VOsKJvyLzbPdiLBfSMBzsQkTp5sBP6Ec16sMf3yVbu7UxMGxs5irMn92LuIspcJ2SjL4Z4veDBbg2vd6kHO4XXCd2GbvFgN6tY1DYsvZ91IGCeUP2BgobvjWHntv/K89UJkxx9QeZ5MZEe7PbwOnmwEwjtQh4iz0IzHux1G6HkVOlZ66UUeH77f2N4xnLMXnhhzWMQCD0fZu8VD/a6qryLPNjrgVQ5YTzRzpB1Kx7s5jYaqc5o9dBA4pvwqxgcno8FK67KvQ9h8mLSKPOJ92DPUOXkwU4gdATt9mDnMeVuT6QNTWJaVOXHD/4Svl/BwnNeBMZ4W93jCP2JnibzVkbs4+3BnqXKyYOdQMhGJ1V5an2Tqjw3cqryE4eexP7nfg5IUX9jAiFAT5N5FrrRg71YED3jwU6qnNBrGE8P9ryqvO45WwYDZ07sxt6nf4xzNtyE4sD08BwJhHroSzI30S0e7Nbwepd6sOddTyB0Eu10e2u3BztPRO5sRB47/wxBkYTvlbFs3asxOHVeru9OIGj0bAJcz3mw28LrYeKb3YNdkgc7gdA2dMKDPd/n1t/+zKnDOLL3WcxZfHFsOalyQl70tTLvLg92uyonD3YCoXl0woM9frzaHuyx42Wp8jo4e/oIHr33VjhuKfc+BEISPanMe9ODPUuVkwc7gZBEuxPfGlLliYG93g5ovyofOXMcm+/5OlatvwYn/TWxdaTKCY2gb5V5t3mwZ6nybvRgJ1VO6AW0qsptJjHx49X2YDcd3xpR5Xq9lBKlgalYd9ErsXDphpr7EP7/9u40OJKzvuP4r3sunatzdeyhYw/vrvfe9W1jYyAGbIwdB2JCpahQxJAqDuNgnOMFpFIpkhcGAqRSFW5TQIUYQsUOLmIO2/jCNsZ4vWuv9z61q11Ju7o1R3fnhVajac1IGh3Tmkf6fqq2rJ3pme5xW/7N/+nn+TemY1yYG9uDPWROD/aJqMoRpEIsR0v/fZoe7FaOYfW8G8LMQG9Ph15+5gey7JDqGtdkPU9VjpkyKszN7sGefYtTerADfvN5Z7RC9WAf+3uu482nKu/qPKw9L/5EbZdd62tEM4Ygx2wYec18OkXZgz3H8Do92IGZm2sP9onvlWtd+FQ92CcG+WTd3nJV9In4oN7c8wttveouVdeunPJYgZkwpjJfND3Yc3R7owc7UJjh9Vw92NOV+oSRuXnt9jaB53nqOXdU0Vi5rrn5IwQ55t2iq8yLtgf7JFU5PdiBuZnud36qmei5qvKperDPpip3nIRee+lnSiVHVLO8VaFwZPoPBcyQEZV5oavyIHqwT1aV04MdKGxV7nsu4Ko8Ee/XoVd+qGisVDuvu1u2HZpye66XY7YWVWWe712SRn+e0CQmR1VuWdYMq3LN+M5oo9fL86/K039ytW6lKsciN919F7K3n7wqz33tfe492Mdel0oOKxwpU1Pb9dq0g6VnKKxFE+b59mPOnMmauRxtbCmaHbZ9y9F8fzKWo40tRYtFLd9ytPRStInL0TKWooWTQ77laGNL0bx43LccbeJStPRytEtL0cb+ObYcbSzAnaTrW442cfnZxOCe+Pd8r6kD86HQVXnmqFw+VXmmfCvlTe3R9M+Om9LBvb9Wcuiidl53t6x1+QX55e1Td4DbfyyR1/tgaSr6Yfal0IN94vD6QvdgnwxD7FhI0zWJyXU3xMmCPL0cLSPI/SNz/jsppt8vR1We+f+o/r5OvfSbh+Q4SW298s68v6gwvI65WhSVuQk92Ce7Mxo92IHpFXsPdsdJSdborPX2DderceWmaT8TMJ+KujJfND3YvSmqcnqwYwmbzyF2aYqRuAL2YB/sPa2XnnpInR1vaFlN84yDnKoc88H4ytyUHuyTVuX0YAemFHRVnm8Pdkk6dfBx9fcc1Q1vfaeWN/tvlAIEqWjDfK7L0WZSlUvKXZXPpnXrTO+MlkzlrMpn07p1zMQmMRJVOYpPcHdGy9VvffY92F03paG+DlXWtKqypk3N7TepYcWynMc0HapyzJeiDPN8g3yqGeyjj1n+ILft8YkvYw1hxv4ZHr2RSijzhirhkP9mKlFbsaitaMRSLCrFIlJJ1FM04qok7Kgk7CgWSipmJxS14oq6I4qmhhVJjd5MJZQYkhUflhUfGZ3FPjwsLxEfn70eT8hNppQam8medUMVR27Klee6ozPWXU9Owsmo0D1fkLspL6sH+2SBTZCjGOXTg93/WOGqcs9zdeHcAXUef06lFQ2qqG5RVT3VOIpDUYb5TNGDnR7sMEs+VWu+PdgnrgvP/gI/cTTO/3s/9tqJ18sn/j+k58zv1dt9SKsvu0UV1S0Zx5lj4hxVOQJWdGG+aHqwu3neGY0e7FhiTOnB7rqOerveVHfHy2rZ8B7Vrdip+pVX5LzTWa5jmgpBjvlWdGGeL9N6sM9bVU4PdszBzi1VUz7/yt7egI5k5mZ7Z7TM1/ofm7wH+8jQOZ144xHFymrU1HajSspr01/uMxXiXueYXxvbotNvtAgUVZgvmh7sWVX55HdGowc7lpJirco9z1N88KwudO5VTeNmlVY2qmXTe1VW2aRNa2IZE2gnHMTEy3kFrLjzCSW6xC1dRRXm+cr3Lklj2xRDD/aZVuX0YMdSFmQP9mSiVyfffFTyPNU2bVWsrF6hcExllU2TvuelJ6b9HEBQjAvzXL/UJvRgt+JDxvRgpypHIRTLndFSiQEN9h5Vf88h1TRtU1X9eq1c83aVVq6QHZr6rmb5HNN0uF6OQiiaMF/sPdhzDa8Xaw92LF3TXVOXCn9dfbY92Cd+gR/fXooPdykSq5AzMqyTB36qiuo21a3crat3b7h0f/EK3zK0sfF0/6W6zIOa3fB6oYN8qQ3FL5Xr4fkomjDPh8k92HMNr9ODHfCbS7e3iZKJXi0L79PF7pMKhSNq33qLqupWa/f2T8q2Q7LsHMvaMsI23yA3zXQBuJjCfikpijCf76p8wXqwT3FnNG+Sqpwe7DBNPtX7RBN/x/Op7ndsqfLl5vjvueULX8u2FB/pV++FkxroO6O+7jPasPXdisSWqbe7SS3rSt5YGwAAEiFJREFUrlVZeXXGclNblp37uArJlOH1uVS79dUhbWyL5vWFgKp6fhVFmOfD+B7sk1Tl9GDHUpTzC8HEqjxnkEvDQxc1ONCpocFuDQ506fId79XgwHn1XTytqpoVWtGyS+XLlitkWyor2y6lr5fbsuSfDDv63hOqcstatFV5UAjq4C14mAd5ZzRpAXuwT1KV04Mdi91sql/PcyXLVk/XUQ0NdGlk5KJk2Vq/6Rb1dB3W8GC3yiuXa1XbbklS3fK1Wt60XtKEYM7Zg31OHycLTWJQDBY0zOfagz3XBJixbYqtB7s7Qg92QBpd0+2k4kokBpWID6isvF52OKJjB55WfLhP8ZE+1S1fp/b1b1F/7xmFwhFV17WprLxOkrSq7Qp/G9assLazCuiZVOW+15HBMMSCV+YzYXQP9hzD6/Rgx2KSiA8omRyRk4pLlqVlVSvU03VY/b0dSiaGlUwOaePW29XTdUTHDj2tWKxCkWi5VrVfpbJYvaprWhVrrlSsZJlisTJJUtv6GzL24A9e36x1y9botfSMIE9PelU6yH2T3qzcQZ5vgxjfa6dAVY4gLFiYz0e3t8zXFH0P9hkOr9ODHYXgeaPnOpUakec6ct2UbDuiaKxcA/3nlEoOy3FSkkaHrvsudqj34km5TlKOk9TKlivkOEkdPfiknFRcKSeh5pXb1bxqhw7se0ySFAqXqGJZg6qqV8q2wyotq9Gy6pUKR0olWapbvl71DRl3G7v0O1/fOP5Y9v8esn+fx7e1s4fOM4I8vV3GNvMx8S3IyXPAdAIL82cfvcng//ItSbb8/7pmPqMXKKQTR59X38XT8jxXnuto6+4PqPv8QZ048qxc15HnOlq36Z0qr2zQ3t//SLYdlmWHVLd8vVa1Xqmuzv2Kxwdk22FFo2WqW75WkqeQHVYkUqpQKCo7FFY4HFPrmusVCkcUCpcoHB6d7LRl159mHVNNXduMP8dYTue7miX99xzD6xPfJ+sWyXOoyvNBVY6gWGPf1AEAgJnmeV4nAAAIWmDD7Nff/hRDAEABmH0JC8B8oDIHAMBwhDkAAIYjzAEAMBxhDgCA4QhzAAAMR5gDAGA4whwAAMMR5gAAGI4wBwDAcEbdAtUEL//qbtU0XKs1Wz/te9x1k3r1N/colejVzpu/p3Ck0ve856b06jMfk5Mc1I6bvq2ujl/r8J4Hs97fDpWqtGK1mtvfp4ZVt+Q8htOHf6Tzpx/Xjhu/Ne3xHnjln9R1+lfact1Xtax2q++5ZOKiXnr8Tq3f8fdanrEv102p8/gjOn/qcQ0NHJdlh1RW0aaGltvUsOqdl25HOerE/m/q1KHvZ7yrpVC4VKUVrWpsuU0Nq2/z3fzid798nxIjXZMe72W7/kH1K94qSRrsO6IT+7+u/otvSLK0rGaL2jZ/XCVlzdN+bgBYTAjzgNh2RGu33a+9z31Sx/d/U2u33ud7vuPojzXUd1iXX/NFhcJl6ccvv/pBhcKl6b8nRrrUceRhHfrDFxSJVqum4Srf+3R1PKkT+7+u0sq2GRydp8N7HtT2G78p245MuaWTGtYbL/6tBi7uV1PbH6tl40fkuY4unH9RR/Z8UT1nntKG3f8oOxQd/+yhUm2+ZvSLied5SiX71HP2GR3e86AGew9lffGpW3GzVrS/L+f+S8pXX/r30K19v71PpRUtWrf9b+S5KZ08+JD2Pvcp7bjpOwpHKmbw+QHAbIR5gJbVblFT6x06e/wRNax+tyqrN0qS4sOdOnngu2psuV3V9bt9r6mo3pBVxVfVX6GXHr9D50/9XzrMU8lBnTz4kM4ceXjGQRYKl2l44IROHfy+WjZ8eMptj73+7xq4+IY2X/fV9PFLUk3jNappuFpvvPh3OnHgO2rb9LH0c5Zlq7Jms+99ahuvUyRWo9OHfqC6FTerqm57+rlorDZr+4nOnfq5PDelTVf+c/rzVtZcrt/98v3q6nhCTa235/35AcB0XDMPWMvGexQtqdWR174kz3MlSUf2fkXhyDK1bvqrvN7DtiOybP/3sM4Tj6r79K912a7Pqbrh6hkdU6ysWY0t79Hpwz/UUP+xSbdLjHTr3MnH1Nh6hy/Ix9Q0XK36lW/T2aP/LSc1NO1+V679oGw7qnMnfjaj45WkWGmTVqy92/fFJVpSr1C4TPGhMzN+PwAwGWEesHCkXGu2fFqDvQd07uRjunDuBV3ofE5rt31G4Uh51vae58pzU/LclFwnoZHB0zq850G5zojqV74jvV1d01u0620/UP2Km2d1XK2bPqZIpEqH9zyoye5x39v9B3meo5opvizUNd0k142rt+v30+4zHClXefUG9V/Y63/CU/oz+/54TnqT5SvfrtXrP5R1fE5qUKUVLdPuGwAWE4bZF0Bt0w2qbbpRJ978lkLhci1f9a5JA/Klx+/Ieqy0sk2X7fq8ahuvTT9WUr5yTscUjlSofcun9ObLn9PZ4/+j5rY7s7aJD5+VNFoVT6akrOnStp157TcSrdFg7yHfY2eO/URnjv0ka9toSb2ueMePc75PKtmvI699SbHSJtXN8gsNAJiKMF8ga7bcq1ee/JAkW+2bPz7pdpuv+bJC4VKlUoM6deB7ig936rKdn1P5sjUz2p/nOVJmxW3ZvlnnklTXfKNqG2/Qif3fUG3jdb5JbJfe5NJLQ5PvyJriuTzVr3i7Vqx5f/ZbTzI5L5Xo1+svPqD48HltvvbLCoVicz4GADAJYb5AoiV1KqtsVyRalTXBLVN51br085XVl+vVpz+q11/4rLbf+A1FY7V572/vc/f6hrMbVt+qddsfyNqufcu9+sNTf6Eje7+idds/63sullF1T7b8a6x6j5Y05HVciZHzipXU+x6LxKpVkeOafO79der1Fx5QfPicNl35hZzX8gFgsSPMDRIKl2rtts9o3/Of1tG9X9OG3Z/P+7Xrtj/gm5QWjlbn3C5WulwtG+/R0b3/qu4O/7K36uVXyrLC6jn7tKrqduR8fc/ZZ2TbUVXV75r2mFLJfg32HdLylX+U9+fINDxwUvt++9dynbg2X/tlghzAksUEOMNU1e1QXfPN6j7zhHq7X837daUVLaqo3pj+M3ZtO5em1jtUWbNFx/d/3fd4JFqVXlrX1/Na1ut6u17RuVM/V2PbnTkn803UceS/5LlJNbbclvfnGJNM9On1F+6XPE9brv8aQQ5gSaMyL4DB/sPqOPJw1uP1K96maEndnN+/ddNH1dP5jI7t+zdte8t/ZF37nivLsrR22/169el7sp5r2XSPhgdP6PXf3q+m9rvS6+Ivnn9RZ479VNX1V6h1w1/6XuN5rvov7Lv0s6dUolc9nc/p3MnH1Nz+/qw15Yl4T3r7iSKxWpWUNevkm99WfLhT7Zs/JSc54Ns+Equb8ssKACw2hHkB9Pe8pv4clWtlzeXzEuYlZc1qbrtLHUd+pHMnf67Gllvn/J4TlVW2aeXaD+rUwYd8j4dCJdp01b+o88RjOnfyMXUef+TS9u1as+U+Nax+V9aXC9cZ1mvPjk3ysxSJVau0olWX7fp8ujVrpu6OJ9Td8UTO42psuV1rt31GPZ3PSJKO7vtq1jbN7X+i9s2fnOEnBgBzWZOtKZ5v19/+VDA7ApaYZx+9yZp+KwCLGdfMAQAwHGEOAIDhCHMAAAxHmAMAYDjCHAAAwxHmAAAYjjAHAMBwhDkAAIYjzAEAMBxhDgCA4QhzAAAMR5gDAGA4whwAAMMR5gAAGI4wBwDAcIQ5AACGI8wBADAcYQ4AgOEIcwAADEeYAwBgOMIcAADDEeYAABiOMAcAwHCEOQAAhiPMAQAwHGEOAIDhCHMAAAxHmAMAYDjCHAAAwxHmAAAYjjAHAMBwhDkAAIYjzAEAMBxhDgCA4QhzAAAMFw5qR5ZlBbUrAACWFCpzAAAMR5gDAGA4whwAAMMR5gAAGI4wBwDAcIQ5AACGC25pms33BgAACoGEBQDAcIQ5AACGC2yY3bbpAAcAQCFQmQMAYDjCHAAAwzGbHQAAw3HXNAAADEe5DACA4RhmBwDAcCQsAACGC7Ay55o5AACFwDA7AACGC64DHLPZAQAoCCpzAAAMxzVzAAAMR9MYAAAMF1iYh8KhoHYFAMCSEtwEuBBhDgBAIVCZAwBguAArc2azAwBQCMFV5oQ5AAAFEVxlzjA7AAAFQWUOAIDhAgvzaEkkqF0BALCkUJkDAGC4wMI8HOGaOQAAhUCYAwBguMDCPBIlzAEAKITgKvMw18wBACiEwMK8vJzZ7AAAFEJgYR6LUZkDAFAIwV0zj3A/cwAACiG4yjxKmAMAUAgBhnlQewIAYGkJLsyZ/wYAQEEEFuZV5U5QuwKWmMB+jQEUqeAq87Ab1K4AAFhSAgxzKnMAAAohsDAvCaWC2hUAAEtKcJV5KBnUrgAAWFICC/OonQhqVwAALCnBVeZ2PKhdYQ4GBwcVjUYVibCWEABMEViYVyYvBLUrzNILL7+iP//oJ1RZUa5f/PQ/VVdbu9CHhLy0LvQBAFhgwfVmTw0HtSvM0iOP/q8Gh4Y0ODSkJ594Qh+449aFPiQAQB6Cu595ciioXWGWnMT4Fy5nZJBzBgCGCCzMQwkq82JnOeO9AOxUknMGAIYILMxtgqHoWe54LwDLSXDOAMAQAYb5SFC7wmxlVOZWKsk5AwBDBBbmVh+z2YudlRzvBWCNDHHOAMAQgYW5F6fKK3aeMz7M7iWTnDMAMARhjnEZw+xKEeYAYIrAwtwdIRiKnZcR5m4yxTkDAEMEWJnTm73ouRn3nE+lOGcAYIjAwtwZoTd7sfNS/sqccwYAZggszJN9A0HtCrPkJsdvU+uMxDlnAGCI4CrzRGr6jbCgXNcb/znlcM4AwBDBhTnXX4uebwJcKsU5AwBDUJkjzXPGK3Mv5XLOAMAQhDnSvIzZ7K5DmAOAKYJbZ54xUxrFycu8Zu64nDMAMERws9mHWOZU7NyMa+ZOIsU5AwBDBDfMnnSn3wgLKrMy9xyXcwYAhmCYHWm+YXbX45wBgCGozJHmZZwiKnMAMAeVOdL8s9mpzAHAFFTmSPO8jJ+pzAHAGMFV5kmqvGKXVZlzzgDACAE2jaHKK3aek/mzyzkDAEMEdz/zjJnSKFIZ4+yexzkDAFMEN8zuEAzFznfN3PU4ZwBgiOAqc4Kh6HmZlbnLOQMAUwRXmbtcfy16mZW553LOAMAQwVXmSaq8Yudv58o5AwBTBNg0hmAodv515h7nDAAMYS/0AQAAgLlhmB3jMpeiOR7nDAAMQWUOAIDhuGaOtMy7prku5wwATMEwO8ZlrkRjmB0AjMEwOwAAhiPMAQAwHNfMkeZr5+pwzgDAFFTmAAAYjjAHAMBwhDkAAIZjaRrGZSxN81iaBgDGoDIHAMBwhDkAAIYjzAEAMBxhDgCA4QhzAAAMF1wHOGZGFz3PzewA58n1OGcAYAIqcwAADEeYAwBgOMIcAADDEeYAABiOMAcAwHCEOQAAhiPMAQAwHGEOAIDhCHMAAAxHmAMAYDjCHDrmxfVh54ie9wbSj33X69J9znH1e84CHhkAIB+EOfS8N6AupdQvN/3YgFwdUlwHNLKARwYAyAdhDm22SnM+HpWltYoFfDQAgJkizKGtKtVmZQf6rVa1qq3AbqwHAJglwhyyLEt/Ztf5HgtLusuqWZgDAgDMCGEOSdI2laou4/b2O1WuGqpyADACYQ5Jo9X5vVajSmSpSiF9wmpY6EMCAOSJ0gtpO+1yPaz1C30YAIAZojIHAMBwhDkAAIYjzAEAMBxhDgCA4QhzAAAMR5gDAGA4whwAAMMR5gAAGI4wBwDAcIQ5AACGI8wBADAcYQ4AgOEIcwAADEeYAwBgOMIcAADDEeYAABiOMAcAwHCEOQAAhiPMAQAwHGEOAIDhCHMAAAxHmAMAYDjCHAAAwxHmAAAYjjAHAMBwhDkAAIYjzAEAMBxhDgCA4QhzAAAMR5gDAGA4whwAAMMR5gAAGI4wBwDAcIQ5AACGI8wBADAcYQ4AgOEIcwAADEeYAwBgOMIcAADDWZ7nLfQxAACAOaAyBwDAcIQ5AACGI8wBADAcYQ4AgOEIcwAADEeYAwBgOMIcAADDEeYAABiOMAcAwHCEOQAAhiPMAQAwHGEOAIDhCHMAAAxHmAMAYDjCHAAAwxHmAAAYjjAHAMBwhDkAAIYjzAEAMBxhDgCA4QhzAAAMR5gDAGC4/wdvccxGS4vY7wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nn_graph_draw2d(\n", " ([[2, 0],[-1, -2]], [0, 0], tanh), # LYR1\n", " ([[1], [-1]], [-1], tanh) # LYR2\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The histogram on the second layer plot shows the distribution of" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "s = X W + b\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "feeding its activation function (shown as the dashed curve). Note how the central histogram peak is higher because both the lower-right and upper-left quadrants of $(x_1, x_2)$ have $Y W \\simeq 0$. The vertical white line shows how our choice of bias $b = -0.5$ places these quadrants in the \"rejected\" (blue) category with $s < 0$.\n", "\n", "Generalizing this example, a layer with $n$ inputs can \"select\" a different $n$-sided (soft-edged) polygon with each of its outputs. To see this in action, try [this demo](https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Data Flow Perspective" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The diagram below show the tensors flowing forward (left to right) in a typical fully connected graph. The main flow consists of $N$ input samples flowing from $X_0$ to $X_4$ with a number of features that varies between layers. \n", "\n", "The computation of each layer's output is parameterized by the weight and bias tensors shown: note how their shapes are determined by the number of input and output features for each layer. The parameter tensors are usually randomly initialized (more on this soon) so only the input $X_0$ and target $Y$ are needed to drive the calculation (and so must be copied to GPU memory when using hardware acceleration).\n", "\n", "The final output $X_4$ is compared with the target values $Y$ to calculate a \"loss\" $\\ell(X_4, Y)$ that decreases as $X_4$ becomes more similar to $Y$ (more on this soon).\n", "\n", "![forward flow](img/NeuralNetworks/forward_flow.png)\n", "\n", "The diagram below shows the gradient (partial derivative) tensors flowing backwards (\"backpropagation\") through the same graph using the chain rule:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "\\frac{\\partial \\ell}{\\partial X_n} = \\frac{\\partial \\ell}{\\partial X_{n+1}} \\frac{\\partial X_{n+1}}{\\partial X_n}\n", "\\quad, \\quad\n", "\\frac{\\partial \\ell}{\\partial W_{n+1}} = \\frac{\\partial \\ell}{\\partial X_{n+1}} \\frac{\\partial X_{n+1}}{\\partial W_{n+1}} \\; .\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![backward flow](img/NeuralNetworks/backward_flow.png)\n", "\n", "Note that these gradient tensors are just numbers, not functions. All of these tensors occupy the (limited) GPU memory when using hardware acceleration but, in most applications, only the final output and the parameter gradients are stored (with 32-bit floating point precision).\n", "\n", "When working with large datasets, the $N$ input samples are usually broken up into fixed-size randomly subsampled \"minibatches\". Optimiztion with the resulting parameter gradients leads to the \"stochastic gradient descent\" (SGD) algorithm." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PyTorch Primer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A fully connected network can be created with a few lines in PyTorch (for a similar high-level API in Tensorflow checkout [Keras](https://www.tensorflow.org/guide/keras)):" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "torch.manual_seed(123)\n", "\n", "net = torch.nn.Sequential(\n", " torch.nn.Linear(8, 4), #0\n", " torch.nn.ReLU(), #1\n", " torch.nn.Linear(4, 3), #2\n", " torch.nn.ReLU(), #3\n", " torch.nn.Linear(3, 2) #4\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As each `Linear` layer is created, its weight and bias tensors are automatically initialized with random values, so we initially set the torch random seed for reproducible results.\n", "\n", "This construction breaks each layer into separate linear and activation \"modules\". Each module can be accessed via its index (0-4 in this example):" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sequential(\n", " (0): Linear(in_features=8, out_features=4, bias=True)\n", " (1): ReLU()\n", " (2): Linear(in_features=4, out_features=3, bias=True)\n", " (3): ReLU()\n", " (4): Linear(in_features=3, out_features=2, bias=True)\n", ")\n" ] } ], "source": [ "print(net)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Parameter containing:\n", "tensor([[-0.4228, -0.1435, -0.3521, 0.0331],\n", " [-0.0934, -0.2682, -0.0455, 0.4737],\n", " [-0.0394, 0.0159, -0.0780, 0.0786]], requires_grad=True)" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net[2].weight" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Parameter containing:\n", "tensor([0.4928, 0.0345], requires_grad=True)" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net[4].bias" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To run our network in the forward direction, we need some data with the expected number of features ($D=8$ in this example):" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "N = 100\n", "D = net[0].in_features\n", "Xin = torch.randn(N, D)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "Xout = net(Xin)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The intermediate tensors ($X_1$, $\\partial\\ell/\\partial X_1$, ...) shown in the data flow diagrams above are usually not preserved, but can be useful to help understand how a network is performing and diagnose problems. To cache these intermediate tensors, use:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "mls.torch.trace(net)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "Xout = net(Xin)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each submodule now has `input` and `output` attributes:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.equal(Xin, net[0].input)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.equal(net[0].output, net[1].input)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use the `verbose` option to watch the flow of tensors through the network:" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0: IN (100, 8) OUT (100, 4)\n", "1: IN (100, 4) OUT (100, 4)\n", "2: IN (100, 4) OUT (100, 3)\n", "3: IN (100, 3) OUT (100, 3)\n", "4: IN (100, 3) OUT (100, 2)\n" ] } ], "source": [ "mls.torch.trace(net, verbose=True)\n", "Xout = net(Xin)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To complete the computational graph we need to calculate a (scalar) loss, for example:" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor(0.1996, grad_fn=)\n" ] } ], "source": [ "loss = torch.mean(Xout ** 2)\n", "print(loss)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now back propagate gradients of this loss through the network:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "4: GRAD (100, 2)\n", "3: GRAD (100, 3)\n", "2: GRAD (100, 3)\n", "1: GRAD (100, 4)\n", "0: GRAD (100, 4)\n" ] } ], "source": [ "loss.backward()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The gradients of each layer's parameters are now computed and stored, ready to \"learn\" better parameters through (stochastic) gradient descent (or one of its variants):" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([-0.0285, -0.0276, -0.0335, 0.0646])" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net[0].bias.grad" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using `mls.torch.trace` we have also captured the gradients of the loss with respect to each module's outputs $\\partial\\ell /\\partial X_n$:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(torch.Size([100, 4]), torch.Size([100, 4]))" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net[0].output.size(), net[0].grad.size()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These gradients can be useful to study since learning of all upstream parameters effectively stops when they become vanishly small (since they multiply those parameter gradients via the chain rule)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Statistical Perspective" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The tensors behind a practical network contain so many values that it is usually not practical to examine them individually. However, we can still gain useful insights if we study their probability distributions.\n", "\n", "Build a network to process a large dataset so we have some distributions to study:" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sequential(\n", " (0): Linear(in_features=100, out_features=200, bias=True)\n", " (1): Tanh()\n", " (2): Linear(in_features=200, out_features=100, bias=True)\n", " (3): ReLU()\n", " (4): Linear(in_features=100, out_features=10, bias=True)\n", ")\n" ] } ], "source": [ "torch.manual_seed(123)\n", "\n", "N, D = 500, 100\n", "Xin = torch.randn(N, D)\n", "\n", "net = torch.nn.Sequential(\n", " torch.nn.Linear(D, 2 * D),\n", " torch.nn.Tanh(),\n", " torch.nn.Linear(2 * D, D),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(D, 10)\n", ")\n", "print(net)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that our network ends with a `Linear` module instead of an activation, which is typical for regression problems.\n", "\n", "Perform forward and backward passes to capture some values:" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "mls.torch.trace(net, verbose=True)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0: IN (500, 100) OUT (500, 200)\n", "1: IN (500, 200) OUT (500, 200)\n", "2: IN (500, 200) OUT (500, 100)\n", "3: IN (500, 100) OUT (500, 100)\n", "4: IN (500, 100) OUT (500, 10)\n", "4: GRAD (500, 10)\n", "3: GRAD (500, 100)\n", "2: GRAD (500, 100)\n", "1: GRAD (500, 200)\n", "0: GRAD (500, 200)\n" ] } ], "source": [ "Xout = net(Xin)\n", "loss = torch.mean(Xout ** 2)\n", "loss.backward()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First check that the input to the first module has the expected (unit normal) distribution:" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD3CAYAAAAT+Z8iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAEBtJREFUeJzt3X+s3fVdx/Hnvb38sMktXpMjagaSiXkbE4OWYbsJthkC6XBWp9HFQBxzITPVUMEMxtq0JiS6BapTYEywYqbLGMWqxHQ0mcJqV+jEuoyI7w4UCDGYS3NLL1Qhba9/nG+3w+Xcc849Pff8+NznI2nyPZ/zOff7+X7u6et87ud8v5/v2NzcHJKkco0PugGSpKVl0EtS4Qx6SSqcQS9JhTPoJalwE4NuwHzT07N9Pw1oamolMzPH+73bkWRfdcZ+6oz91Ll2fVWrTY4t9JwjemBiYsWgmzAy7KvO2E+dsZ86dyZ9ZdBLUuEMekkqnEEvSYUz6CWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1Lhhm4JBKkkH/3Df2xavvO29/e5JVrOHNFLUuEMekkqnFM3Ug8sNEUjDQNH9JJUOINekgpn0EtS4Qx6SSqcQS9JhfOsG2kRenV2jRdSqZ8c0UtS4Qx6SSqcQS9JhTPoJalwBr0kFc6gl6TCGfSSVLi259FHxArgfiCAk8ANwBjwIDAHPANsysxTEbENuBY4AWzOzIMRcXGzur0/FElSM51cMPVBgMz8mYhYD+ygHvRbMvPxiLgP2BgRLwLrgDXABcAjwGVV/bfVBXb3/EikHnLZYZWk7dRNZv4tcGP18IeB/wEuBZ6oyvYAPwdcDuzNzLnMfAmYiIjaAnUlSX3S0RIImXkiIv4S+CXgV4Cfz8y56ulZ4DxgFXCk4WWny8ea1F3Q1NRKJiZWdH4EPVKrTfZ9n6PKvlo6y7Fvl+Mxd6vbvup4rZvM/I2IuBV4CviehqcmgaPAsWp7fvmpJmULmpk53mmTeqZWm2R6erbv+x1F9tXSWm596/upc+36qtWHQNupm4i4PiI+WT08Tj24/6WarwfYAOwD9gPXRMR4RFwIjGfmq8ChJnUlSX3SyYj+b4C/iIivAWcBm4Fngfsj4uxqe1dmnoyIfcAB6h8gm6rX3zK/bo+PQZLUQtugz8w3gF9t8tS6JnW3A9vnlR1uVleS1B9eMCVJhTPoJalw3mFKGiLeeUpLwRG9JBXOEb2WNZc60HLgiF6SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFc5lilU8lyLWcmfQSyPAO0/pTDh1I0mFM+glqXAtp24i4ixgJ3ARcA5wB/Ay8Cjw7ara5zLzoYjYBlwLnAA2Z+bBiLgYeBCYA54BNmXmqSU4DknSAtqN6K8DjmTmFcAG4G5gNbAjM9dX/x6KiNXAOmAN8GHgnur1O4At1evHgI1LcRCSpIW1+zL2YWBXw+MTwKVARMRG6qP6zcDlwN7MnANeioiJiKhVdZ+oXrsHuBrY3cP2S5LaaBn0mfk6QERMUg/8LdSncB7IzKcj4lPANuAocKThpbPAecBYFf6NZS1NTa1kYmLFYo/jjNVqk33f56iyr4ZHCb+LEo6hX7rtq7anV0bEBdRH4fdm5hcj4nsz82j19G7gT4G/AxpbMEk9/E81KWtpZuZ4h03vnVptkunp2b7vdxTZV8Nl1H8Xvp86166vWn0ItJyjj4jzgb3ArZm5syp+LCJ+utq+Enga2A9cExHjEXEhMJ6ZrwKHImJ9VXcDsK/94UiSeqndiP52YArYGhFbq7KbgT+OiLeAV4AbM/NYROwDDlD/8NhU1b0FuD8izgae5e3z/ZKkPmg3R38TcFOTp97XpO52YPu8ssPUz8aRJA2IF0xJUuEMekkqnEEvSYVz9UpphLmqpTrhiF6SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOC+YUjEWunhIWu4c0UtS4Qx6SSqcQS9JhTPoJalwBr0kFc6gl6TCGfSSVDjPo5cK5A1J1MgRvSQVzqCXpMIZ9JJUuJZz9BFxFrATuAg4B7gD+HfgQWAOeAbYlJmnImIbcC1wAticmQcj4uJmdZfkSCRJTbUb0V8HHMnMK4ANwN3ADmBLVTYGbIyI1cA6YA3wYeCe6vXvqNv7Q5AktdLurJuHgV0Nj08AlwJPVI/3AFcDCezNzDngpYiYiIjaAnV3t9rh1NRKJiZWLOogeqFWm+z7PkeVfTW6hvF3N4xtGlbd9lXLoM/M1wEiYpJ64G8B7qwCHWAWOA9YBRxpeOnp8rEmdVuamTm+mPb3RK02yfT0bN/3O4rsq9E2bL8730+da9dXrT4E2p5HHxEXUB+F35uZX4yIzzQ8PQkcBY5V2/PLTzUpk86I685Li9Nyjj4izgf2Ardm5s6q+FBErK+2NwD7gP3ANRExHhEXAuOZ+eoCdSVJfdRuRH87MAVsjYitVdlNwJ9ExNnAs8CuzDwZEfuAA9Q/PDZVdW8B7m+s2+sDkCS11m6O/ibqwT7fuiZ1twPb55UdblZXktQ/XjAlSYUz6CWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXBtV6+UVI5WK3/uvO39fWyJ+skRvSQVzqCXpMIZ9JJUOOfoNbS8k5TUG47oJalwBr0kFc6gl6TCGfSSVDiDXpIKZ9BLUuEMekkqnEEvSYXr6IKpiFgDfDoz10fEauBR4NvV05/LzIciYhtwLXAC2JyZByPiYuBBYA54BtiUmad6fRCSpIW1DfqI+ARwPfBGVbQa2JGZdzXUWQ2sA9YAFwCPAJcBO4Atmfl4RNwHbAR29/QIJEktdTKifx74EPCF6vGlQETERuqj+s3A5cDezJwDXoqIiYioVXWfqF63B7iaNkE/NbWSiYkViz6QM1WrTfZ9n6PKvirToH6vvp86121ftQ36zHwkIi5qKDoIPJCZT0fEp4BtwFHgSEOdWeA8YKwK/8aylmZmjnfY9N6p1SaZnp7t+35HkX1VrkH8Xn0/da5dX7X6EOjmy9jdmfn06W3gp4BjQONeJqmH/6kmZZKkPupm9crHIuJ3MvMgcCXwNLAf+ExE3Am8CxjPzFcj4lBErM/Mx4ENwD/1quGSemuh1UK989To6ybofwu4OyLeAl4BbszMYxGxDzhA/a+ETVXdW4D7I+Js4FlgVw/aLElahI6CPjNfANZW2/8KvK9Jne3A9nllh6mfjSNJGhAvmJKkwhn0klQ4g16SCuc9YzVw3htWWlqO6CWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOBc1k9SStxgcfY7oJalwBr0kFc6gl6TCGfSSVDiDXpIKZ9BLUuE6Or0yItYAn87M9RFxMfAgMAc8A2zKzFMRsQ24FjgBbM7MgwvV7f1hSJIW0jboI+ITwPXAG1XRDmBLZj4eEfcBGyPiRWAdsAa4AHgEuKxZXWB37w9Do8CbgEuD0cmI/nngQ8AXqseXAk9U23uAq4EE9mbmHPBSRExERG2Bui2DfmpqJRMTKxZ1EL1Qq032fZ+jyr4S9O594Pupc932Vdugz8xHIuKihqKxKtABZoHzgFXAkYY6p8ub1W1pZuZ4B83urVptkunp2b7vdxTZVzqtF+8D30+da9dXrT4EuvkytnGOfRI4ChyrtueXN6srSeqjboL+UESsr7Y3APuA/cA1ETEeERcC45n56gJ1JUl91M2iZrcA90fE2cCzwK7MPBkR+4AD1D88Ni1UtwdtliQtQkdBn5kvAGur7cPUz7CZX2c7sH1eWdO6kqT+cZliSV1x+eLR4ZWxklQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXCeXqmec5VKabg4opekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzgumJPWU69QPH4NeXfMKWGk0OHUjSYUz6CWpcAa9JBXOoJekwhn0klS4rs+6iYhDwGvVw/8CPg98FjgB7M3M34+IceBe4BLgTeBjmfncmTVZkrQYXQV9RJwLkJnrG8r+Dfhl4D+Bf4iI1cBFwLmZ+d6IWAvcBWw8wzZLkhah2xH9JcDKiNhb/YztwDmZ+TxARDwGXAn8IPAVgMx8MiLec8YtliQtSrdBfxy4E3gA+FFgD3C04flZ4N3AKr47vQNwMiImMvPEQj94amolExMrumxW92q1yb7vc1TZV+rGQhfYPXqXf+R3qtv/e90G/WHgucycAw5HxGvA9zU8P0k9+FdW26eNtwp5gJmZ4102qXu12iTT07N93+8osq/Ua76fOtPu/16rD4Fuz7r5KPX5diLih6gH+hsR8SMRMQZcA+wD9gMfqOqtBb7V5f4kSV3qdkT/58CDEfHPwBz14D8F/DWwgvpZN09FxDeAqyLi68AYcEMP2ixJWoSugj4z3wJ+vclTa+fVOwV8vJt9aHi4eJk02rxgSpIKZ9BLUuEMekkqnEEvSYXzDlP6Dr90lcpk0EsaKO8xu/ScupGkwhn0klQ4g16SCmfQS1Lh/DJW0lDyS9recUQvSYVzRL8Meb68tLw4opekwhn0klQ4g16SCuccfcGci1eJPBtn8RzRS1LhHNFLKoIj/YU5opekwhn0klQ4p25GnF+4Sq05pWPQjwwDXVK3ljzoI2IcuBe4BHgT+FhmPrfU+x1VBrrUH8tppN+PEf0vAudm5nsjYi1wF7CxD/sdaga6NJy6+b857B8O/Qj6y4GvAGTmkxHxnj7s84wt9tPe4JaWr2H/62Bsbm5uSXcQEQ8Aj2TmnurxS8C7M/PEku5YkgT05/TKY8Bk4z4NeUnqn34E/X7gAwDVHP23+rBPSVKlH3P0u4GrIuLrwBhwQx/2KUmqLPkcvSRpsFwCQZIKZ9BLUuEMekkqnGvdNIiIHwOeAs7PzP8bdHuGTUScB/wVsAo4G7g5Mw8MtlXDxSU/OhMRZwE7gYuAc4A7MvPvB9qoIRYR3w88DVyVmf+x2Nc7oq9ExCrqyzO8Oei2DLGbga9m5jrgI8A9g23OUPrOkh/AbdTfU3qn64AjmXkFsAG4e8DtGVrVh+Lngf/t9mcY9EBEjAF/BtwOHB9wc4bZH1F/w0H9r0H/6nmnty35AYzEkh8D8DCwteGxF1Eu7E7gPuC/u/0By27qJiJ+E/jdecUvAl/KzG9GxABaNXwW6KcbMvMbEfED1KdwNve/ZUNvFfBaw+OTETHh1eBvl5mvA0TEJLAL2DLYFg2niPgIMJ2Zj0XEJ7v9OZ5HD0TEc8DL1cO1wMHM/NkBNmloRcRPAF8Cfu/0+kX6rojYATyZmV+uHr+cme8acLOGUkRcQP2Cynszc+eg2zOMIuJrwFz17yeBw8AvZOYri/k5y25E30xmXnx6OyJeAK4eWGOGWET8OPU/uX8tM7856PYMqf3AB4Evu+THwiLifGAv8NuZ+dVBt2dYNQ44I+Jx4OOLDXkw6LU4fwCcC3y2muJ6LTOX/b0F5nHJj87cDkwBWyPi9Fz9hszs+gtHLcypG0kqnGfdSFLhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUuP8Hem1xzjp9X9UAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[0].input.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How does torch initialize the parameters (weights and biases) for each layer?" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD3CAYAAAANMK+RAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAEL9JREFUeJzt3H+Q3PVdx/HnXQ4K2AsGe4AzFvsH7XucqYpBS7VAYgvEGJhY7WimFitUwBqrUWYsv0pQGYsIFCi0KBABFatNCS2tkaj8CimYAXEEjW9FZ2RGLT0wkZNAaZLzj/0e2R67d3u7370fn3s+ZjKz+93P5/t97zeffe1nP7f7HRgfH0eStPANznUBkqR6GOiSVAgDXZIKYaBLUiEMdEkqxNBcHnx0dKzrr9gsW3YEu3fvrbOcWljXzFjXzFjXzJRa18jI8ECr7Qt2hj40tGSuS2jJumbGumbGumZmsdW1YANdkvStDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIeb0p/+SVKdzr3qg5fZNF713liuZG87QJakQBrokFcJAl6RCGOiSVAgDXZIKUdy3XBb7X7klLV7O0CWpEAa6JBWiuCWXmWq3RNOOSzcHTXXuPE/Tc3lQdVv0gS5Jky3UN1sDXQvSQn3BSf1koEtSj+bLBMNAlxaZ+RI+qp+BrmnN9A/HkuaGgT6HnCmVr843w8U4Xhbjc+6Fga7XOROfXskBU/JzWywWTaAbVpJK11GgR8TRwJPA6cA+4A5gHHgGWJ+ZByJiI7CmenxDZu7sS8WLmDMoLWT+EK3/pg30iDgE+H3glWrTdcBlmflQRNwCrI2I/wBWACcBbwW+APxQf0rWQuabUvnm46fhxTLuOpmhXwPcAlxc3T8ReLi6vRU4A0hgW2aOA89FxFBEjGTmaN0FS/pWiyWsNL0pAz0ifh4Yzcz7I2Ii0Aeq4AYYA44ElgIvNnWd2D5loC9bdgRDQ0u6qRuAkZHhrvv285i91jXT/p22n83z1a/nMFv7mem+5uq4c7mfsy78Yi37n+oYdbVvZ6afJuqssx+vx+lm6OcC4xFxGnACcBdwdNPjw8Ae4KXq9uTtU9q9e++Mim02MjLM6OhY1/27Nd0x66hrpv07aT/b56sfz6GX/cx0FjvT81Xnue33uZir/dR5jLl47Xdz3Hbte309tnszmDLQM/PUidsR8RDwi8DvRcTKzHwIWA08CDwLXB0R1wDfBQxm5gtdVzuP+YcdLTbzcU18rsz0XLRrf9+1a+so5w26+drihcCtEXEosAvYnJn7I2I78BiNa6yvr7FGSVIHOg70zFzZdHdFi8evAK7ouSKpB3M1g3IWq/lg0fywqGR+y0ESGOjzkrO9gzwXasVx0ZqBPgscfJJmw4IN9Dq/AyvNJ04A1K0FG+gqiyEm9c5Ar5GhJHXH1049DPRFyBePVKbBuS5AklQPZ+jqCz8FSLPPQC+YoSotLi65SFIhnKFLhfIT2uJjoEst+MM1LUQuuUhSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSrE0HQNImIJcCsQwH7gHGAAuAMYB54B1mfmgYjYCKwB9gEbMnNnn+qWJE3SyQz9LIDMfA9wOXBd9e+yzDyFRrivjYjlwArgJGAdcHNfKpYktTTtDD0z742IL1d3vxt4nsYs/OFq21bgDCCBbZk5DjwXEUMRMZKZo+32vWzZEQwNLenpCUjSQjQyMlz7PqcNdIDM3BcRdwLvBz4AnFkFN8AYcCSwFHixqdvE9raBvnv33m5qlqQFb3R0rOu+7d4MOv6jaGZ+GHgHjfX0w5seGgb2AC9VtydvlyTNgmkDPSLOjoiLq7t7gQPAExGxstq2GtgO7ABWRcRgRBwHDGbmC32oWZLUQidLLvcAfxgRjwCHABuAXcCtEXFodXtzZu6PiO3AYzTeKNb3qWZJUgud/FH0ZeCnWzy0okXbK4Areq5KkjRj/rBIkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBViaKoHI+IQYBPwNuBNwJXAPwF3AOPAM8D6zDwQERuBNcA+YENm7uxf2ZKkyaaboX8IeDEzTwFWAzcB1wGXVdsGgLURsRxYAZwErANu7l/JkqRWpgv0zwOfaLq/DzgReLi6vxU4DTgZ2JaZ45n5HDAUESN1FytJam/KJZfM/D+AiBgGNgOXAddk5njVZAw4ElgKvNjUdWL76FT7X7bsCIaGlnRXuSQtYCMjw7Xvc8pAB4iItwJbgM9k5t0RcXXTw8PAHuCl6vbk7VPavXvvzKqVpEKMjo513bfdm8GUSy4RcQywDfh4Zm6qNj8VESur26uB7cAOYFVEDEbEccBgZr7QdbWSpBmbboZ+CbAM+ERETKyl/ypwY0QcCuwCNmfm/ojYDjxG401ifb8KliS1NjA+Pj59qz4ZHR3r+uDnXvVAnaVI0qy579q1vS65DLTa7g+LJKkQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKsRQJ40i4iTgdzNzZUQcD9wBjAPPAOsz80BEbATWAPuADZm5s081S5JamHaGHhG/AdwGHFZtug64LDNPAQaAtRGxHFgBnASsA27uT7mSpHY6maH/G/CTwB9V908EHq5ubwXOABLYlpnjwHMRMRQRI5k5OtWOly07gqGhJd1VLkkL2MjIcO37nDbQM/MLEfG2pk0DVXADjAFHAkuBF5vaTGyfMtB37947o2IlqRSjo2Nd9233ZtDNH0UPNN0eBvYAL1W3J2+XJM2SbgL9qYhYWd1eDWwHdgCrImIwIo4DBjPzhZpqlCR1oKNvuUxyIXBrRBwK7AI2Z+b+iNgOPEbjTWJ9jTVKkjowMD4+Pn2rPhkdHev64Ode9UCdpUjSrLnv2rW9rqEPtNruD4skqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiGG6txZRAwCnwG+H/gG8AuZ+Wydx5AktVb3DP0ngMMy84eBi4Bra96/JKmNugP9ZOAvATLzceAHa96/JKmNWpdcgKXA/zbd3x8RQ5m5r1XjkZHhgW4PdN+1a7vtKklzbmRkuPZ91j1DfwlornKwXZhLkupVd6DvAH4cICLeDTxd8/4lSW3UveSyBTg9Ir4KDADn1Lx/SVIbA+Pj43NdgySpBv6wSJIKYaBLUiEMdEkqRN1/FO1ZRBwO/DFwNDAGfDgzR1u0Ox64NzPfWd1/C3A3cDjwX8A5mbk3Is4DLgD2AVdm5pf7VVdEbATWVMfakJk7I+JzwLFVk7cBj2fmuoj4EvAdwDeBVzJz9SzXtRy4D/jXqtlnM/PPWrWd5bpOAD4N7Kdx+Yify8znI+JG4D3VvgDWZmbzbx6mq2fKy1K0Gif9HlM91HUcsInG63cAOD8zMyJ+HfgIMHGeL8jMnMW6jgL+BXimarYlM2+YB+freuCEqsmxwJ7MfHevY2omdVVtRoCvAt+bma+2e41ExFnA5dVz2JSZt3Zax3ycoX8UeDozTwHuAi6b3CAizgY+B7ylafPlwN1Vv6eACyLiWOBXaPynrQI+GRFv6kddVUCuAE4C1gE3A2TmusxcCbwf2AP8WtXleODkzFzZbZj3UhewHLiuOv7KKszbtZ3Num4APlads3uAjzfVu6qp3pm+8NpelmKKcdLvMdVtXb8N3FSdo98BPll1WU7jDXDiHHUV5j3UtRz406bj3zAfzldmbqjO1ek0fvh4XtWl1zHVUV1VbauAbcAxTZvf8BqJiEOATwFn0Hh9nF89r47Mx0B//fIBwFbgtBZtdtN4stP1exewIzO/Uf1nPQt8X5/qOhnYlpnjmfkcMFS9I0/4TeDTmfnfEXEM8O3AfRHxaESc2WVNvdR1IrAmIh6JiNsjYriD5zAbda3LzL+v2gwBr1azn7cDfxAROyLi3F7qaXFZinbjpN9jqtu6LgS+UrUZAl6tbp8IXFyNqYt7qKnbuk4ElkfEwxHx+Yj4zinazmZdEz5GY8w9XdOY6rQugAM0xs//tOrDwfH1PcCzmbk7M18DHgVO6bSIOV1yiYiPcHDGOuF5Dl4+YAw4cnK/iY9sEdG8ufmyAxP9Jl+KoOX+aqprKfBii2ONRsTRwPua9nkojXfwG4CjgB0RsTMzvz6Lde0EbsvMJyPiUmAjjU8QLZ/DbNU18TE1In4E+GXgVODbaCzDXAcsAR6MiCcy8x+mqqvF8dpdlqLdOKltTNVZV2a+ABCNF8A1NGaH0PjUejONX2xviYgze1je6OZ8/TPwZGb+dUT8LI3/s3vbtO1WN3UREYfSWIp5V/VYHWOq07rIzL+q6mjXp5bxNaeBnpm3A7c3b4uIezh4+YBhGiHTiYnLDrzS1G/ypQg62l+XdU11rA/Q+Oi+v7r/NeCW6j/76xHxFBDAlIFec11bMnOi7RYag/uLUzyH2aqLiPgZ4FJgTbWmuAS4ITP3Vo8/QGOtciYvvqkuS9GultrGVM11ERE/SmPN9uxq/XwAuH5i2SAivgL8ANBtoHdT198Ce6ttW4DforGUMOfni8bs95GmZZW99D6mOq2rkz61jK/5uOTy+uUDgNXA9h767QROiYjDIuJIGh9nnmnTv9e6dgCrImKw+qPV4MRMisZg2trU9jTgzwEi4s3AO4Fds1zX/RExMVt5H/DkNM9hVuqKiA/RmJmvzMx/r9q+A3g0IpZUa4wnA3/XbT3xxstStBsn/R5TXdVVhfkNwI9l5hNV26XVY2+uwv29NP5PZ60u4Dbgp6o2E2Nqzs9X9djk12AdY6rTuqbtw8HxtQt4e0QcVX2qOBV4rNMi5t23XIDPAndGxKPAa8AHASLiamBztv/WxZVVv/OAF4APZubL0fhL9nYab16XZuarbfr3XFdEbKdx8geB9U19A5gIJzJza0SsiojHaaytXdJDcHZb10eBmyLiNRqfGM7PzJemeA59r6uaid8IPAfcU308fTgzN0bEnwCP0/hW0F2Z+Y8zrOcNl6WIxrdCns3ML7UaJxHR7zHVbV3X01i2u7M6R5mZF0TEJcCDNL5l8TeZ+RezXNdFwKaI+CXgZRrf9PjaXJ+vql/Q+LQAQGbuqmFMdVxXmz5veI1k5jerfvdXz2FTZv5np0X4039JKsR8XHKRJHXBQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmF+H/AHfP+95iukAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[0].weight.data.reshape(-1), bins=50);" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWkAAAD3CAYAAADfYKXJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAEaFJREFUeJzt3X+QJGddx/H35TYhHGxgMQOBKiRa4BcVJAYKEJJwCjEEcp4ohSkkQg6TiICAVwIJvwpESWkOCJCAkJyEUkSBHJcDYyIcP5JgSBFjGSzyjQEtLOXHEBZYsjnCHecf03NZ9nZ2enqmZx+O9+uv27mnuz/d/cxne3tndtbt378fSVKZDlvrAJKkwSxpSSqYJS1JBbOkJalglrQkFWxm0ivsdhcav1xkbm4D8/OLk4wzEeYajblGU2ouKDfboZir05ldt9LjRV1Jz8ysX+sIKzLXaMw1mlJzQbnZfpJyFVXSkqQfZUlLUsEsaUkqmCUtSQWzpCWpYJa0JBVs6OukI+Jw4DLgWGAfcFZm3tJyLkkS9a6knwbMZOYTgDcAf9ZuJElSX52SvhWYiYjDgKOAH7QbSZLUt27YH/2PiAcDO4F7A0cDp2XmZweN37t33/61fjfQpq07V3x817bNU04iadp+jJ//K74tvM7f7ngZcFVmnlsV9u6IeGRm7llp8Djvp+90Zul2FxovP0zTdbedqylzjcZcoys1W5Nc09iPcY5XpzO74uN1Snqeu29xfAs4HCjzjfOSdIipU9JvAbZHxDXAEcB5mXlHu7EkSVCjpDPze8CzppBFkrSMb2aRpIJZ0pJUMEtakgpmSUtSwSxpSSqYJS1JBbOkJalglrQkFcySlqSCWdKSVDBLWpIKZklLUsEsaUkqmCUtSQWzpCWpYJa0JBXMkpakglnSklSwoR+fFRHPA55XfXkkcBxwTGZ+u71YkiSo9xmH7wXeCxARFwHbLWhJmo7atzsi4jHAL2bmu1vMI0laYuiV9BLnAa8fNmhubgMzM+sbB+p0Zhsv2+a628w1jqa5Nm3dueLju7ZtHifOAdM4Xk32Ya3O47Cspc4vKDfbqLkGjR91Hk37XNYq6Yi4L/DwzPzksLHz84uNw3Q6s3S7C42XH6bputvO1VQbuSaxvrU+XoO2vda5VtLtLhSZq6/UbE1yTWP8OMdrULnXvd1xEvDxRluWJDVWt6QD+HKbQSRJB6t1uyMz/7LtIJKkg/lmFkkqmCUtSQWzpCWpYJa0JBXMkpakglnSklQwS1qSCmZJS1LBLGlJKpglLUkFs6QlqWCWtCQVzJKWpIJZ0pJUMEtakgpmSUtSwSxpSSqYJS1JBav7aeHnAr8BHAFcnJmXtppKkgTUuJKOiI3AE4AnAk8CHtxyJklSpc6V9CnAzcAO4CjgT1YbPDe3gZmZ9Y0DdTqzBz22aevOFcfu2rZ57HWvtv6l21i+7KQyjWvQPq31+lZbT9vHbrVtj7J/0zjH/TyTPo+T1ORcQvvPhVGP2bTGT/pc1inpo4GHAKcBPwNcEREPz8z9Kw2en19sHKbTmaXbXag9fpSxTcb3lxklV5NtNDXq8apjEutrmmtS+zJoPZM6XpM85qPOr2kbJ1ub+9Qk1zTGj3O8BpV7nZK+HbglM+8CMiL2AB3gG42SSJJqq/PqjmuBp0bEuoh4EHAvesUtSWrZ0JLOzI8CNwE3ALuAF2bmvraDSZJqvgQvM1/edhBJ0sF8M4skFcySlqSCWdKSVDBLWpIKZklLUsEsaUkqmCUtSQWzpCWpYJa0JBXMkpakglnSklQwS1qSCmZJS1LBLGlJKpglLUkFs6QlqWCWtCQVzJKWpILV+visiLgJ+E715X9l5pntRZIk9Q0t6Yg4EiAzN7aeRpL0I+pcST8K2BARV1fjz8vM6wcNnpvbwMzM+saBOp3ZVsYCbDl/96hxDmyj7rZGzbRp684VH9+1bXMr25vW+pqsp+1tDzrWk1r/OOtavs5x58UkNd3fQc+3Se3DqLmmNX7Sz8k6Jb0IXABcAjwMuDIiIjP3rjR4fn6xcZhOZ5Zud6H2+FHGNtXtLoyUa1KZ6qxn1OM1qe0O0zTXNI9dKetfq/lV16E0x6YxfpzjNajc65T0rcBtmbkfuDUibgceCPxPoySSpNrqvLpjC7ANICIeBBwFfLXNUJKknjpX0pcC742Ia4H9wJZBtzokSZM1tKQz8y7g2VPIIklaxjezSFLBLGlJKpglLUkFs6QlqWCWtCQVzJKWpIJZ0pJUMEtakgpmSUtSwSxpSSqYJS1JBbOkJalglrQkFcySlqSCWdKSVDBLWpIKZklLUsEsaUkqWJ3POCQi7g/cCJycmbe0G0mS1Df0SjoiDgf+Criz/TiSpKXqXElfALwLOLfOCufmNjAzs75RmE1bd440vtOZbbSdJtuou60t5+9e8fFd2zaPtN1B6xlk1PUPMqlj2mQ9a7ntUax2bto+z9OY8yttc9Tn5rD1rcV6pjV+0udo1ZKOiOcB3cy8KiJqlfT8/OIkctXS7S5MZRudzuzY22o766TWP4n1ND1eJe1Dqdue9r5NYu4vt1ZzbBrjxzleg8p92O2OLcDJEfEp4DjgfRFxTKMEkqSRrXolnZkn9f9dFfUfZObX2g4lSerxJXiSVLBaL8EDyMyNLeaQJK3AK2lJKpglLUkFs6QlqWCWtCQVzJKWpIJZ0pJUMEtakgpmSUtSwSxpSSqYJS1JBbOkJalglrQkFcySlqSCWdKSVDBLWpIKZklLUsEsaUkq2NBPZomI9cB7gAD2AWdm5pfaDiZJqnclvQkgM58IvBZ4c6uJJEkHDC3pzPwIcHb15UOAr7eaSJJ0QK0Pos3MvRFxGfAM4JmrjZ2b28DMzPpJZBuq05ltfRtbzt89kfW0nXXQ+jdt3TnSelbb313bNo+UZ1LbHmW7q61nGto+z6Meo0HnYNRzOUlrdX5G3e6o+90fP+njNcqnhT83Il4BfC4ifiEz71hp3Pz84sTCDdPtLkxtW+NqO+s0jkXdbXQ6sxPN43me/HbX6lz+OGlyTMc5XoPKfejtjog4IyLOrb5cBH5I7xeIkqSW1bmSvhz464j4DHA48NLM3NNuLEkS1Cjp6rbGs6aQRZK0jG9mkaSCWdKSVDBLWpIKZklLUsEsaUkqmCUtSQWzpCWpYJa0JBXMkpakglnSklQwS1qSCmZJS1LBLGlJKpglLUkFs6QlqWCWtCQVzJKWpIJZ0pJUsFU/PisiDge2A8cC9wDemJlXTCGXJInhV9LPAW7PzBOBU4F3tB9JktQ37INoPwh8aMnXe1vMIklaZtWSzszvAUTELL2yfvWwFc7NbWBmZv1k0g2x5fzdU9nOJHQ6s62ufxrHYtA+bNq6s9Xt/jid57XKOup2R5mPbc/dUo263/3xkz5ew66kiYgHAzuAizPz/cPGz88vTiLXIafbXVjrCGM7FPZBPXXPZacz+xN73kfd7253YazjNajch/3i8AHA1cCLMvMTjbYsSWps2JX0ecAc8JqIeE312KmZeWe7sSRJMPye9EuAl0wpiyRpGd/MIkkFs6QlqWCWtCQVzJKWpIJZ0pJUMEtakgpmSUtSwSxpSSqYJS1JBbOkJalglrQkFcySlqSCWdKSVDBLWpIKZklLUsEsaUkqmCUtSQWzpCWpYLVKOiIeFxGfajmLJGmZYR9ES0S8HDgDuKP9OJKkpepcSX8J+K22g0iSDjb0SjozPxwRx9Zd4dzcBmZm1o8V6lC05fzdax1hbJ3O7FpH0IQcCvOxbaMeo/7zY9LPk6ElPar5+cVJr1KF6HYX1jqCVKxud4FOZ7bx82RQufvqDkkqmCUtSQWrdbsjM/8beHy7USRJy3klLUkFs6QlqWCWtCQVzJKWpIJZ0pJUMEtakgpmSUtSwSxpSSqYJS1JBbOkJalglrQkFcySlqSCWdKSVDBLWpIKZklLUsEsaUkqmCUtSQWzpCWpYEM/PisiDgMuBh4FfB/4/cy8re1gkqR6V9K/CRyZmb8CvBLY1m4kSVJfnZI+AfgngMy8HnhMq4kkSQes279//6oDIuIS4MOZeWX19VeAn83MvVPIJ0k/0epcSX8XmF26jAUtSdNRp6SvA54GEBGPB25uNZEk6YChr+4AdgAnR8RngXXAme1GkiT1Db0nLUlaO76ZRZIKZklLUsEsaUkqWJ1fHI4tIu4J/A1wf2ABeG5mdlcY91DgI5n5iOrro4H3A/cE/g84MzMXI+Is4BxgL/DGzPxoW7ki4nXA06ttvTQzb4iIDwDHVEOOBa7PzNMj4grgp4AfAHdm5qlTznU8sAv4z2rYOzPz71caO+VcxwFvB/bR+9MCv5eZX4+ItwFPrNYFsDkzvzNCnlX/ZMFK86TtOTVGrp8GttN7Tq4Dzs7MjIg/Bp4P9I/zOZmZU852P+BW4AvVsB2ZeWEBx+ytwHHVkGOAb2fm48edV6PkqsZ0gM8Cj8zMPYOeJxGxCXhttQ/bM/M9dTJM60r6BcDNmXki8D7g1csHRMQZwAeAo5c8/Frg/dVyNwHnRMQxwB/ROwmnAG+KiHu0kasqvScBjwNOBy4CyMzTM3Mj8Azg28DLqkUeCpyQmRubFvQ4uYDjgTdX299YFfSgsdPMdSHw4uqYXQ68YkneU5bkHfWJNPBPFqwyT9qeU01z/SnwjuoY/TnwpmqR4+l9U+sfo8YFPUa244G/W5LhwhKOWWa+tDpeJwPfAc6qFhl3XtXKVWU7BbgaeMCShw96nkTE4cBbgF+n9xw5u9qvoaZV0gfeWg5cCTxlhTHz9MIPW+6xwHWZ+f3q4N8G/FJLuU4Ars7M/Zn5FWCm+q7Z93rg7Zn51Yh4AHBfYFdEXBsRpzXMNE6uRwNPj4jPRMSlETFbYx+mkev0zPy3aswMsKe6QnkY8O6IuC4itoyTZ4U/WTBonrQ9p5rm2gp8rBozA+yp/v1o4NxqTp07RqZxsj0aOD4iPh0RH4yIB64ydpq5+l5Mb97dPKF5VTcXwA/pzaFvrbQMd8+xnwduy8z5zLwLuBY4sU6Aid/uiIjnc/eVZd/X6X2ng97l/32WL9f/USkilj581ArLLX1s4PomlOso4PYVttWNiPsDT16yziPofZe9ELgfcF1E3JCZ35hirhuASzLzxoh4FfA6elf6K+7DtHL1fzyMiCcALwJOAu5F7xbIm4H1wCcj4vOZ+e+r5Vphe0vnwr6ImKneETtonkxsTk0yV2Z+EyB6T4AL6F3BQe+ny4vovfN3R0ScNs5thSbZgFuAGzPz4xHxu/TO20cGjJ1mLiLiCHq3QR5b/d8k5lXdXGTmP1c5Bi0z9hybeEln5qXApUsfi4jLufut5bP0iqOO/lvS71yy3PK3qddaX8Ncq23rmfR+bN5Xff014F3VyftGRNwEBLBqSU84147M7I/dQW+y7lxlH6aVi4j4HeBVwNOr+3PrgQszc7H6/9307vuN8mRa7U8WDMoysTk14VxExK/Su/95RnU/eh3w1v6P6xHxMeCXgXFKukm2zwGL1WM7gDfQ+zF+zY8ZvavUzyy5pbHI+POqbq46y4w9x6Z1u+PAW8uBU4FrxljuBuDEiDgyIu5D78eILwxYftxc1wGnRMRh1S92Dutf8dCbHFcuGfsU4B8AIuLewCOAL04511UR0b+ieDJw45B9mEquiHgOvSvojZn55WrszwHXRsT66n7dCcC/Ns0TB//JgkHzpO051ShXVdAXAk/NzM9XY4+q/u/eVWH/Gr1zOo4mx+wS4LerMf15tebHrPq/5c/DScyrurmGLsPdc+yLwMMi4n7V1f9JwL/UCTCVV3cA7wQui4hrgbuAZwNExF8AH8rBrzZ4Y7XcWcA3gWdn5h3R++3tNfS+ybwqM/cMWH7sXBFxDb2DeRjwwiXLBtAvHDLzyog4JSKup3ef6rwxyrBprhcA74iIu+hd2Z+dmd9dZR9az1VdMb8N+ApwefVj4acz83UR8bfA9fReDfO+zPyPEfMc9CcLovdqiNsy84qV5klEtD2nmuZ6K71bZpdVxygz85yIOA/4JL1XFnwiM/9xjFxNs70S2B4RfwjcQe8VDl9b62NWLRf0ruoByMwvTmBe1c41YJmDnieZ+YNquauqfdiemf9bJ4BvC5ekgvlmFkkqmCUtSQWzpCWpYJa0JBXMkpakglnSklQwS1qSCvb/wJeynwLGA9gAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[0].bias.data, bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These initial parameter values are sampled from uniform distributions centered on zero with a spread that depends on the number of inputs to the layer:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "\\left|W_{ij}\\right|, \\left|b_j\\right| \\le n_{in}^{-1/2} \\; .\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This default choice is based on [empirical studies](https://arxiv.org/abs/1502.01852) of image classification problems where the input features (RGB pixel values) were preprocessed to have zero mean and unit variance.\n", "\n", "With this choice of weights, the first `Linear` module mixes up its input values ($X_0$) but generally preserves Gaussian shape while slightly reducing its variance (which helps prevent the subsequent activation module from saturating):" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD3CAYAAAAT+Z8iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAEM9JREFUeJzt3W2MnFd5h/Fr1xvHdTVOt+pAi0pIIe1dtaqCnEgOJcErSHBNoAbaSlFE2yTNBypLxSVVCMhpjITEi5JUqUgEChi3KogK07RAZbAKNDVOQigNUqKmd5SAlG/VxlrHSw1Etqcf5lm6XeZtzewzM2eun2Rp5sw9nnNmdv9z9sw855lptVpIkso1O+oOSJI2lkEvSYUz6CWpcAa9JBXOoJekws2NugNrLS4uT9zXgObnt7K0dHrU3ajFtIx1WsYJjrUUzWZjptttzuiHYG5u06i7UJtpGeu0jBMc6zQw6CWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXB9t0CIiBuBG6urW4BXAwvAvcAZ4Ghmvj8iZoH7gcuAHwG3ZOYzEXHl2tohj0Eaurfc+k8d2w/e/vqaeyL99PrO6DPzUGYuZOYC8G3gz4CPATcAVwE7ImI78FZgS2a+BrgduLv6LzrVSpJqMvDSTURcAfwm8Fngwsx8NjNbwFeAN9AO8i8DZOajwBURsa1LrSSpJuvZvfJ9wPuBbcCpVe3LwCur9hdWtZ/tUdvV/PzWidx4qNlsjLoLtSlprN2WaLopaeyrlTquTqZprCsGCvqI+Dng1zPz69UsffUz1QBOAlvXtM/SDvlOtV1N4haizWaDxcXlUXejFtM01k5KHPs0vaYlj7XXG9igSzevA/4FIDNPAS9GxKsiYgbYBRwDjgNvAqg+gH2iR60kqSaDLt0E8N1V198JfBrYRPubNN+MiG8B10bEw8AMcFO32qH0XJI0kJlWa7xO6DSJZ5gq+c/BtSZxrDd/6GtD+79K/HrlJL6m56vksXqGKUmaYga9JBVu7E4OLo2zbstAJS7pqBzO6CWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mF84ApaQg8kErjzBm9JBXOGb2KMcxdKqWSOKOXpMIZ9JJUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwA32PPiLeC/wusBm4H3gIOAS0gCeBvZl5LiLuBK4DzgD7MvOxiLi0U+2QxyFJ6qLvjD4iFoDfBl4L7AReDtwD7M/Mq4EZYE9EbK9u3wFcD9xX/Rc/UTvkMUiSehhk6WYX8ATwIPBF4EvA5bRn9QBHgGuAq4CjmdnKzOeAuYhodqmVJNVkkKWbXwBeAbwZ+BXgC8BsZraq25eBi4BtwIlV91tpn+lQ29X8/Fbm5jYNPIBx0Ww2Rt2F2kzTWH9ak/JcTUo/h2GaxrpikKA/AfxXZr4IZET8kPbyzYoGcBI4VV1e236uQ1tXS0unB+jSeGk2GywuLo+6G7WYprEOwyQ8V9P0mpY81l5vYIMs3XwD+J2ImImIlwE/C3y1WrsH2A0cA44DuyJiNiIupj3rfx54vEOtJKkmfWf0mfmliHgd8BjtN4a9wPeAByJiM/AUcDgzz0bEMeCRVXUAt66tHf4wpPHkPvUaBwN9vTIzb+vQvLND3QHgwJq2pzvVSpLq4QFTklQ4g16SCmfQS1LhPJWgJo6nDJTWxxm9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOPe6kUbAE5KoTs7oJalwBr0kFc6gl6TCGfSSVDiDXpIKN9C3biLiceCF6ur3gI8D9wJngKOZ+f6ImAXuBy4DfgTckpnPRMSVa2uHPAYVyjNJScPRN+gjYgtAZi6savsO8HvAd4F/jojtwCXAlsx8TRXudwN7gI+trc3M/xjyOCRJXQwyo78M2BoRR6v6A8CFmfksQER8BXgD8EvAlwEy89GIuCIitnWpNeglqSaDBP1p4C7gE8CvAkeAk6tuXwZeCWzj/5Z3AM5Wbac61HY1P7+VublNA3RrvDSbjVF3oTbTNNa6jeq5nabXdJrGumKQoH8aeCYzW8DTEfEC8POrbm/QDv6t1eUVs7RDvtGhtqulpdMDdGm8NJsNFheXR92NWkzTWEdhFM/tNL2mJY+11xvYIN+6uZn2ejsR8TLagf4/EfGqiJgBdgHHgOPAm6q6K4EnMvMU8GKHWklSTQaZ0X8SOBQR3wBatIP/HPBpYBPtb9J8MyK+BVwbEQ8DM8BN1f3fubZ2yGOQJPXQN+gz80Xghg43Xbmm7hztUF97/0fX1kqS6uMBU5JUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFc6Tg0tjxJOGayM4o5ekwhn0klQ4g16SCmfQS1Lh/DBWI+e5YaWN5Yxekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFc6gl6TCGfSSVDiDXpIKN9CRsRHxEuDbwLXAGeAQ0AKeBPZm5rmIuBO4rrp9X2Y+FhGXdqod9iAkSd31ndFHxAXAx4EfVE33APsz82pgBtgTEduBncAO4Hrgvm61w+2+JKmfQWb0dwEfA95bXb8ceKi6fAR4I5DA0cxsAc9FxFxENLvUPtjrwebntzI3t2ldgxgHzWZj1F2ozTSNdVxs9HM+Ta/pNI11Rc+gj4gbgcXM/EpErAT9TBXoAMvARcA24MSqu660d6rtaWnp9OC9HxPNZoPFxeVRd6MW0zTWcbKRz/k0vaYlj7XXG1i/Gf3NQCsirgFeDfwt8JJVtzeAk8Cp6vLa9nMd2iRJNeq5Rp+Zr8vMnZm5AHwH+CPgSEQsVCW7gWPAcWBXRMxGxMXAbGY+DzzeoVaSVKPz2Y/+VuCBiNgMPAUczsyzEXEMeIT2m8febrVD6LMkaR0GDvpqVr9iZ4fbDwAH1rQ93alWklQfD5iSpMIZ9JJUOINekgpn0EtS4c7nWzeSanbzh77Wsf3g7a+vuSeaRM7oJalwBr0kFc6gl6TCGfSSVDg/jFVtun2gKGljOaOXpMIZ9JJUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFc6gl6TCGfSSVDiDXpIK13evm4jYBDwABHAWuAmYAQ4BLeBJYG9mnouIO4HrgDPAvsx8LCIu7VQ7/KFIkjoZZEb/FoDMfC3wl8A91b/9mXk17dDfExHbgZ3ADuB64L7q/j9RO9QRSJJ66jujz8x/jIgvVVdfAfw37Vn7Q1XbEeCNQAJHM7MFPBcRcxHRBC7vUPtgt8ebn9/K3Nym8xnLSDWbjVF3oTbTNNZx121H0C/evb751DS9ptM01hUDbVOcmWci4m+AtwG/D7y5CnSAZeAiYBtwYtXdVtpnOtR2tbR0evDej4lms8Hi4vKou1GLaRrrJFvPazRNr2nJY+31Bjbwh7GZ+cfAr9Fer/+ZVTc1gJPAqery2vZzHdokSTXpG/QR8YcR8d7q6mnawf3vEbFQte0GjgHHgV0RMRsRFwOzmfk88HiHWklSTQZZuvkH4FMR8W/ABcA+4CnggYjYXF0+nJlnI+IY8AjtN5C91f1vXVs75DFIknqYabVa/atqtLi4PF4dGkDJ635rDTJWTxk4egdvf/3Atf78lqHZbMx0u80DpiSpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgo30BmmJE2WbjuIrmdXS5XDGb0kFc6gl6TCGfSSVDiDXpIK54exOm+eMlCaDM7oJalwBr0kFa7n0k1EXAAcBC4BLgQ+APwncAhoAU8CezPzXETcCVwHnAH2ZeZjEXFpp9oNGYkkqaN+M/p3ACcy82pgN/BR4B5gf9U2A+yJiO3ATmAHcD1wX3X/n6gd/hAkSb30C/rPAXesun4GuBx4qLp+BLgGuAo4mpmtzHwOmIuIZpdaSVKNei7dZOb3ASKiARwG9gN3ZWarKlkGLgK2ASdW3XWlfaZDbU/z81uZm9u0njGMhWazMeou1Gaaxlqabq/dNL2m0zTWFX2/XhkRLwceBO7PzM9ExEdW3dwATgKnqstr2891aOtpaen0AN0eL81mg8XF5VF3oxbTNNYSdXrtpuk1LXmsvd7Aei7dRMRLgaPAezLzYNX8eEQsVJd3A8eA48CuiJiNiIuB2cx8vkutJKlG/Wb07wPmgTsiYmWt/l3AX0fEZuAp4HBmno2IY8AjtN889la1twIPrK4d9gAkSb31W6N/F+1gX2tnh9oDwIE1bU93qpUk1cctEKQp0mvbCveqL5dHxkpS4Qx6SSqcSzfqy10qpcnmjF6SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalw7l4pCei+S6knJJl8zuglqXDO6PVj7jsvlckZvSQVbqAZfUTsAD6cmQsRcSlwCGgBTwJ7M/NcRNwJXAecAfZl5mPdaoc/DElSN31n9BFxG/AJYEvVdA+wPzOvBmaAPRGxHdgJ7ACuB+7rVjvc7kuS+hlk6eZZ4O2rrl8OPFRdPgJcA1wFHM3MVmY+B8xFRLNLrSSpRn2XbjLz8xFxyaqmmcxsVZeXgYuAbcCJVTUr7Z1qe5qf38rc3KYBuj5ems3GqLsgbYjSfrZLG88gzudbN6vX2BvASeBUdXlte6fanpaWTp9Hl0ar2WywuLg86m5IG6Kkn+2Sf1d7vYGdT9A/HhELmfmvwG7g68AzwEci4i7gl4HZzHw+IjrVSpogHkg1+c4n6G8FHoiIzcBTwOHMPBsRx4BHaK/77+1WO4Q+S5LWYabVavWvqtHi4vJ4dWgApfw56AFTWo9JnNGX8rvaSbPZmOl2m0fGTiEDXZouHhkrSYUz6CWpcAa9JBXOoJekwhn0klQ4v3Uj6bx4INXkMOgL5tcoJYFLN5JUPINekgpn0EtS4VyjlzRUfkg7fpzRS1LhnNFPOL9ZI6kfZ/SSVDiDXpIK59LNhHCJRpPOD2lHxxm9JBXOoJekwrl0M2ZcotG0cUln4zmjl6TCbfiMPiJmgfuBy4AfAbdk5jMb/bjjzpm71Jsz/eGpY+nmrcCWzHxNRFwJ3A3sqeFxx4KBLg2XbwDrV0fQXwV8GSAzH42IK2p4zA1jcEvjaVi/myW+YdQR9NuAF1ZdPxsRc5l5plNxs9mYqaFP5+2Ld0/NHyOSClHHh7GngMbqx+wW8pKk4asj6I8DbwKo1uifqOExJUmVOpZuHgSujYiHgRngphoeU5JUmWm1WqPugyRpA3nAlCQVzqCXpMIZ9JJUODc1G4KIuAj4O9rHDGwG3p2Zj4y2VxsrIt4G/EFm3jDqvgzTtG3ZERE7gA9n5sKo+7JRIuIC4CBwCXAh8IHM/MJIO1UzZ/TD8W7gq5m5E7gRuG+03dlYEXEv8EHK/Pn58ZYdwO20t+woUkTcBnwC2DLqvmywdwAnMvNqYDfw0RH3p3Yl/qKOwl8BH68uzwE/HGFf6vAw8Kej7sQG+X9bdgATvWVHH88Cbx91J2rwOeCOVden7oBNl27WKSL+BPjzNc03Zea3IuIXaS/h7Ku/Z8PXY6x/HxELI+hSHda1Zccky8zPR8Qlo+7HRsvM7wNERAM4DOwfbY/qZ9CvU2Z+Evjk2vaI+C3gs8BfZOZDtXdsA3Qba+HcsqNAEfFy2gdv3p+Znxl1f+rm0s0QRMRv0P7z8IbMPDLq/uin4pYdhYmIlwJHgfdk5sFR92cUnNEPxwdpf6B1b0QAvJCZbnM5mdyyozzvA+aBOyJiZa1+d2b+YIR9qpVbIEhS4Vy6kaTCGfSSVDiDXpIKZ9BLUuEMekkqnEEvSYUz6CWpcP8L0mXfgx/EiM8AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[0].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A scatter plot of the the first `Tanh` activation function's input and output values just traces out function since it is applied element wise. Note how most of input values do not saturate, which is generally desirable for efficient learning." ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD3CAYAAADxJYRbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAGm1JREFUeJzt3X2UVPWd5/F3dxdN09i0rSlh447rujHfJGPi+Cxro44PYyDD4GYn5xjcORkCxpiZmAgzDFHJZBZXE1jC+BCcROFojsGdhMiiWWE2IybDQ8AM4C7O6lfJZBc1PrROA203TdPQ+0dVtdVFVTV96+FW3ft5nZPT9/7uvX2/P8p86te/evg1DA0NISIi8dAYdgEiIlI9Cn0RkRhR6IuIxIhCX0QkRhT6IiIxkgi7gGK6unrq7q1FHR2tdHf3hV1GVaiv0ROXfkK0+5pMtjUUOqaRfpklEk1hl1A16mv0xKWfEK++ZlPoi4jEiEJfRCRGFPoiIjGi0BcRiRGFvohIjJT0lk0zuwT4lrtfmdM+E/g6MAisdveHzGwC8BhwGtADfM7du0q5v4iIjE3gkb6ZLQQeBlpy2scBK4DfA64AvmBmU4BbgD3uPg34PnBn0HuLiEgwpUzv/Ar4dJ72jwJ73b3b3QeALcA0oBPYmD5nA3BNCfcWEYmsg70DHOwdqMjvDjy94+4/NrMz8xyaBBzI2u8B2nPaM21FdXS01uUHKJLJtrBLqBr1NXri0k+oTF/39xzm5Lbxx/0sdm5u2+L7NgOwcuHVBa8NqhJfw3AQyP6XbAP257Rn2oqqx49IJ5NtdHX1hF1GVaiv0ROXfkLxvr7e9R6nJ0/C93Vz0oRxnJ48abgNGB6F9/QNDLdl2hev2sHCz57H0sd3D/9cMvcSJk1sHnGPzLn5ji2ZewkAR/oH6Oof+4i/2JNZJUL/ReBsMzsFeA+4HPivwL8BZgDPAdOBzRW4t4hIQQd7B3jj3V56+4/Q/kYPg0cG2fOrdzjr9HYmd7TyVncfkztaWbzqOT4/4yOsfvolAP700+fwwBMvsGTuxbS1NnPnwzs4euwohw4fY8nci4eDf9LE5uEQz/2Za7RjlVK20Dez2cBJ7v49M5sP/B2p1wxWu/vrZvYg8KiZbQEGgNnlureISIbv6wbg5Ve7aWlO8LEzT+Fnu19j/LgmNu16nf4jx/Jc9erw1p9++pzhIE+ePGF4pJ8d7nfNS43Ec0f68H5g5/7Mp5LhXkhDLa+RW4/fsqk/j6MpLn2tt35+d/0emhONnP1bHTz/ShdTTmnl6R2vFr3m8zM+QmtLgva2CXlH+ud/+LQqVV85xb5ls6a/WllEJGPJIzs4enSInr4jTDm1lde63qPn0FEANu95K33WuwBM+/hkPnDyhBEj/UkTmzn/w6cNj8wzT3B2RsfwPXJH7VGk0BeRmrRszU72vnaA1pYEB/oGRxzr7n3/DYIf/7cdXPTRyTz/She/c3aS7p5+Zl521ojzb/y9j1Sl5nqg0BeRmvDTX+7j8Wf2HteeHfjJ9vEMDh5jyqmtJJoamTH1zOGReucnPli1WuuZQl9EQnPbff9w3Cg+V3trgsaGBr4w65wRUzESjEJfRKpq7jc3Mdo7NCY0w4U2mTmf+u2q1BQnCn0Rqbhla3by4r4DRc9pm9DEvV+5okoVxZdCX0Qq5vPf3FT0eHtrghW3Xl6lagQU+iJSZmuffaXoe+Wbm+Bv/vyqKlYk2RT6IlIW31i1nX1dhb8va/UiBX0tUOiLSEmKvTA7paOFu2/+91WtR4pT6ItIIMXm6zWqr10KfREZkyWP7ODXb/bmPaawr30KfRE5YYVG9wr7+qHQF5FRKeyjQ6EvIgXd/t1tvNndf1y7wr5+KfRFJK98o/szkq18Y+6lIVQj5aLQF5ERvvzXP6O3//jVpTS6j4bAoW9mjcBK4FzgMDDP3femj/0O8NdZp18KXE9qfdyXgRfS7evc/d6gNYhIec1csP64NoV9tJQy0r8eaHH3qWZ2KbAcmAXg7s8DVwKY2WeA37j7RjO7Bnjc3b9cWtkiUm75pnMU+NFTSuh3AhsB3H27mV2Ye4KZTQT+Csh8o9IFwPlm9nPgbeBWd3+j0A06OlpJJJpKKDEcyWRb2CVUjfpa//KN7p9aPiuESqovqo9pMaWE/iQg+7tSj5pZwt2zV0SYC/zI3d9J778E7HT3vzezG4H7gT8sdIPu7sLf41Gr6m1h6VKor/Wv0Og+in3NFdXHFIo/mTWW8HsPAtm/uTEn8AFuBB7O2t8EPJveXgecV8L9RaQEuYHfMi4+I/w4K2WkvxWYCfwwPae/J/ugmbUD4909+ztWHwZ+DPwQuBrYWcL9RSSg3MDX3H18lBL664BrzWwb0ADMMbP5wF53fxL4MPB/c65ZBKw2sy8BvcC8Eu4vImOkF2ulYWhotNUqw9PV1VO7xRUQ5XnCXOprfTmRwI9CP09UlPuaTLY1FDqmD2eJxEBu4OuTtfGl0BeJOM3fS7ZS3r0jIjVOgS+5FPoiEaXAl3wU+iIRpMCXQhT6IhGjwJdiFPoiEaLAl9Eo9EUiQoEvJ0KhLxIBCnw5UQp9kTqnwJexUOiL1DEFvoyVQl+kTinwJQiFvkgdWrZm5LeSK/DlRCn0RerQi/veX7ROgS9jodAXqTP5viJZ5EQp9EXqiObxpVQKfZE6ocCXcgj8ffpm1gisBM4FDgPz3H1v1vH7gMuAzNI0s4BxwBpgAvAbYI679wWtQSQuFPhSLqWM9K8HWtx9Kqm1b5fnHD8fuM7dr0z/7wDwdWCNu08DdgM3l3B/kVjY9fLbI/YV+FKKUlbO6gQ2Arj7djO7MHMg/VfA2cD3zGwysMrdV6evuTt92ob09opCN+joaCWRaCqhxHAkk21hl1A16mvlPZA1yn9q+ayK30+PabSVEvqTgANZ+0fNLOHug8BE4H7g20AT8KyZ/WPONT1Ae7EbdHfX38xPlBdbzqW+Vl7utE6la9BjGg3FnsxKCf2DQPZvbkwHPkAfcG9mvt7MNpGa+89ccyj9c38J9xeJNM3jSyWUMqe/FZgBYGaXAnuyjn0Y2GJmTWY2jtS0zq7sa4DpwOYS7i8SWQp8qZRSRvrrgGvNbBvQAMwxs/nAXnd/0sx+AGwHjgDfd/d/MrO7gEfN7CbgHWB2ifWLRJ4CX8opcOi7+zHgiznNL2UdXwoszbnmLeCTQe8pEgf6xK1Ukj6cJVJDNK0jlabQF6kRCnypBoW+SA1S4EulKPRFaoDm8aVaFPoiIdO0jlSTQl+khijwpdIU+iIh0rSOVJtCXyQkmtaRMCj0RWqAAl+qRaEvEgJN60hYFPoiVaZpHQmTQl8kRAp8qTaFvkgVaVpHwqbQF6kSTetILVDoi4RAgS9hUeiLVIGmdaRWKPRFKkzTOlJLAq2cZWaNwEpSi50fBua5+96s47cBN6R3n3b3vzKzBuA14JV0+y/c/WuBKxepQwp8CVvQ5RKvB1rcfWp6UfTlwCwAMzsLuBG4BBgCNpvZOqAP2OXuM0svW6Q+aFpHak3Q0O8ENgK4+3YzuzDr2KvAJ939KICZjQP6gQuA083sWeAQcJu7e7GbdHS0kkg0BSwxPMlkW9glVI36WtjMBetH7D+1fFY5y6kYPabRFjT0JwEHsvaPmlnC3Qfd/QjwTno6Zxmw291fNrMpwD3u/iMz6wQeAy4qdpPu7r6A5YUnmWyjq6sn7DKqQn09casXXVUX/1Z6TKOh2JNZ0BdyDwLZv7XR3QczO2bWAvwgfc6X0s3/CKwHcPctpEb9DQHvL1LTNK0jtSpo6G8FZgCk5/T3ZA6kg3w98L/c/ebMNA/wl8BX0+ecC+xz96GghYvUC714K7Uk6PTOOuBaM9sGNABzzGw+sBdoAq4AxpvZ9PT5XwO+CTxmZp8CBoE/LqVwkVqlUb7UskCh7+7HgC/mNL+Utd1S4NJPBbmfSL3Qe/Kl1unDWSIVosCXWqTQFykTTetIPVDoi1SARvlSqxT6ImWQPcpvGRdiISKjUOiLlCh3WmflAo3ypXYp9EXKSNM6UusU+iIl0Iu3Um8U+iJlolG+1AOFvkhA2aP89tagH24XqS6FvkgAudM6K269PKRKRMZGoS9SIk3rSD1R6IuMkV68lXqm0BcpgUb5Um8U+iJjoFG+1DuFvsgJ0tcmSxQo9EUCUOBLvVLoi5yAmQvWh12CSFkE/kSJmTUCK4FzgcPAPHffm3X8JuBmUksj3uXuPzGzDwBrgAnAb4A57t5XQv0iVadRvtSzUkb61wMt7j4VWAQszxwwsynArcBlwHXAPWY2Hvg6sMbdpwG7ST0piNQ0vXgrUVLKZ8c7gY0A7r7dzC7MOnYxsNXdDwOHzWwv8In0NXenz9mQ3l5R6AYdHa0kEk0llBiOZLIt7BKqJup9zZ3WeWr5rJAqqZ6oP6bZ4tTXjFJCfxJwIGv/qJkl3H0wz7EeoD2nPdNWUHd3/c38JJNtdHX1hF1GVcSpr5Ca1ol6f+P0mEa5r8WezEqZ3jkIZP/mxnTg5zvWBuzPac+0idQkTetIFJUS+luBGQBmdimwJ+vYc8A0M2sxs3bgo8AL2dcA04HNJdxfpGriMK0j8VDK9M464Foz2wY0AHPMbD6w192fNLP7SIV6I3CHu/eb2V3Ao+l39rwDzC6xfpGK0ChfoqphaGgo7BoK6urqqd3iCojyPGGuqPZ1wf2b6e49Mry/etFVke1rrrj0E6Ld12SyraHQMX04SyRHbuCLRIlCXySLpnUk6hT6IgVolC9RpNAXSdMoX+JAoS+Sh0b5ElUKfRFGjvLbW0t5J7NIbVPoS+zlTuusuPXykCoRqTyFvkgWTetI1Cn0Jdb04q3EjUJfJE2jfIkDhb7EVvYov2VciIWIVJFCX2Ipd1pn5QKN8iUeFPoSe5rWkThR6Evs6MVbiTOFvsTKT3+5b8S+RvkSNwp9iZXHn9k7vK3AlzhS6EtsaFpHJOByiWY2AXgMOA3oAT7n7l055ywDOtP3+J67P2RmpwAvk1ovF2Cdu98btHiRoDTKl7gK+s1StwB73P0bZnYDcCfwlcxBM/td4EPuPtXMxgP/ZGZrgfOBx939y6UWLjIWGuWLpAQN/U5gaXp7A7A45/gvgOfT20NAE3AEuAA438x+DrwN3OrubxS6SUdHK4lEU8ASw5NMtoVdQtXUQ19nLlg/Yv+p5bMC/Z566Gs5xKWfEK++Zowa+mY2F7gtp/kt4EB6uwdozz7o7v1Av5mNAx4lNb3znpm9BOx09783sxuB+4E/LHTv7u6+E+5IrYjyYsu56rGvqxddFajmeuxrEHHpJ0S7r8WezEYNfXdfBazKbjOzJ4DMb20D9udeZ2YdwFrgZ+5+T7p5E5BJ8nXAfx7t/iKl0LSOyEhB372zFZiR3p4ObM4+mH6h9xlgtbsvyTr0MPAf09tXAzsD3l9kVGuffWXEvl68FQk+p/8g8KiZbQEGgNkAZraU1Oj+MuAs4CYzuyl9zRxgEbDazL4E9ALzSqhdpKind7w6vK3AF0kJFPru3gd8Jk/7wvTmc8CKApf/bpB7ioyFpnVE8tOHsyTyNMoXeZ9CXyJHo3yRwhT6Eim5ga9RvshICn2JLAW+yPEU+hIZmtYRGZ1CXyJB0zoiJ0ahL5GjwBcpTKEvdU/TOiInTqEvdU3TOiJjo9CXunXbff8wYl+BLzI6hb7UrQN9g8PbCnyRE6PQl7qkeXyRYBT6Unc0jy8SnEJf6poCX2RsFPpSVzStI1Iahb7UDU3riJQu0CIq6eUQHwNOI7Uw+ufcvSvnnCeBU4EjwCF3n25mHwIeAYaAF4A/cfdjwcuXuFDgi5RH0JH+LcAed58GfB+4M885HwI63f1Kd5+ebvs2cGf6ugZgVsD7S4wp8EWCCxr6ncDG9PYG4Jrsg2Y2GTgZeMrMtpjZ76cPXQD8vNB1IvloHl+kfEad3jGzucBtOc1vAQfS2z1Ae87xZmA5cC9wCrDVzJ4DGtx9qMh1I3R0tJJINI1WYs1JJtvCLqFqKt3XmQvWj9h/anl4fxzG5XGNSz8hXn3NGDX03X0VsCq7zcyeADL/Wm3A/pzL3gT+xt0HgbfNbDdgQPb8fb7rRuju7hutvJqTTLbR1dUTdhlVUem+5pvHD+vfNi6Pa1z6CdHua7Ens6DTO1uBGent6cDmnOPXAD8EMLOTgHOAF4HdZnZlketEAPjiMr1wK1IJgd69AzwIPGpmW4ABYDaAmS0F1rr7BjO7zsy2kxrd3+7u75jZAuAhM2sm9SSwtvQuSBQNHH1/W4EvUj6BQt/d+4DP5GlfmLX91TzHXwauCHJPiQ+9cCtSOfpwltQUvR9fpLIU+lIzFPgilafQl5qgwBepDoW+hE6BL1I9Cn0J1ZJHdozYV+CLVJZCX0L16zd7h7cV+CKVp9CX0GRP69Tfl22I1CeFvoQidx7/IY3yRapCoS9VpxduRcKj0JeqUuCLhEuhL1WjwBcJn0JfqkKBL1Ibgn7LpsgJWbZmJy/uOzCiTYEvEh6FvlRMvm/LVOCLhEvTO1IRCnyR2qSRvpSd5u9FapdG+lJWCnyR2hZopG9mE4DHgNOAHuBz7t6VdfyTwKL0bgPQSWqd3AnAU8Ar6WMPuvvfBitdao0CX6T2BZ3euQXY4+7fMLMbgDuBr2QOuvtGYCOAmf05sNXdXzSzecC33X15iXVLDdH8vUj9CBr6ncDS9PYGYHG+k8zsXwN/BFyUbrog1WyzSI32v+ruPQFrkBqgwBepL6OGvpnNBW7LaX4LyLz5ugdoL3D5fGCFux9O7z8HPOzuO83sDuAvgT8rdO+OjlYSifr7/sVksi3sEqpi5oL1I/b/1SktfO+O60KqpvLi8rjGpZ8Qr75mjBr67r4KWJXdZmZPAJl/rTZgf+51ZtYI/D5wR1bzOnfPnLsOuL/Yvbu7+0Yrr+Ykk210dUX7j5dio/uo9j0OjyvEp58Q7b4WezIL+u6drcCM9PZ0YHOec84BXnL3Q1ltf2dmF6e3rwZ2Bry/hETTOSL1Leic/oPAo2a2BRgAZgOY2VJgrbs/BxjwzznX3QI8YGYDwJvAFwLeX6osX9jfc8tlTG4fH0I1IhJUw9DQUNg1FNTV1VO7xRUQtT8ZFz/0C15/99Bx7asXXRW5vhYTl77GpZ8Q7b4mk20NhY7pE7lSUL7RfXtrghW3Xh5CNSJSDgp9OU6+sAfN3YtEgUJfhinsRaJPoS8Fw/7UtmaW/UlnlasRkUpS6MdYobAHje5FokqhH0MKe5H4UujHxNpnX+HpHa8WPK6wF4kHhX7EFRvVg8JeJG4U+hH0xWWbGDha+HiyfTzfuuWy6hUkIjVDoR8RX1q+if4jxc/RqF5EFPp1atfLb/PAEy+Met5nr/4Q1150RhUqEpF6oNCvI7d/dxvvHuhn8BgU+1IiTd+ISCEK/Rr1etd7/GTbr3nt7fd49+ChUaduzj69ja/90UXFTxKR2FPo1wDf183q//F/uOgjp/Hmv/Txzv5D7OvKv4DMuPQKCKd1TGDJTVOrWKWIRIFCv4pe73qPXS+/zcHeAQAmTWxmy/9+g64DqdUkc99H3zFxHK0tCd49eIiGBvjOfL0QKyKlUeiX2f974yBbd7/Kqe0tAPzz6wcY39zE4YGjRT8cNbGlkSvOPZ03/6WPnr4BTpnUws2zPl6tskUkJhT6J+hg7wCTJjaP+Anwxru9dO0/RPLkCex7q4fHn9lb9Pdcff4Hh7cnTWxm50tvc81FZ9D5iQ8WuUpEpDxKCn0z+w/AZ9x9dp5jNwE3A4PAXe7+EzP7ALAGmAD8Bpjj7qGsfn6wd4CevgFOT540Yrolt72nb4C21mYWr9rBws+ex9LHd7Pws+fxzTW7OHr0GP0Dx0b83i/8wW/T03v4uJF+S3OCj515CqcnTxpx/szLzqpOh0VEKGG5RDO7F7gOeN7db8g5NgX4KXAh0AJsSW8vA3a5+yNmtgg47O4rCt2j1OUSM6PyfNu3P7Sdvv5B/mL2eXxn3QsMMcSi2edzzw92Dbc/8MQeevsHWTL3Ytpam0cd6ff2H+G6y/5dZJdgyxXl5eZyxaWvceknRLuvlVoucRvw30mN5nNdDGx198PAYTPbC3wC6ATuTp+zIb1dMPRLcbB3gMWrdrBk7iUAw9uTJqbC++6bLh0e0d81L3VObvt/ydrOyDxxZP+0Mzoq0QURkbIbNfTNbC5wW07zHHf/WzO7ssBlk4ADWfs9QHtOe6atoI6OVhKJptFKzCuZhJULr+bktvHAyO3M8bFsj+3ebcEurEPqa/TEpZ8Qr75mjBr67r4KWDXG33sQyP7XbAP2Z7UfymorqLu79On+rv6BvNuVEuU/GXOpr9ETl35CtPta7MmssUL3fA6YZmYtZtYOfBR4AdgKzEifMx3YXKH7i4hIHmUNfTObb2Z/4O5vAveRCvVNwB3u3g/cBdxgZluBqcAD5by/iIgUF/jdO9VQ6rt3whDlPxlzqa/RE5d+QrT7WuzdO5Wa3hERkRqk0BcRiRGFvohIjCj0RURipKZfyBURkfLSSF9EJEYU+iIiMaLQFxGJEYW+iEiMKPRFRGJEoS8iEiMKfRGRGNHC6GWW/irpx0gtGNMMzHf3X4RbVWUVWyu5nplZI7ASOBc4DMxz973hVlVZZnYJ8C13vzLsWirBzMYBq4EzgfGk1u9+MtSiqkwj/fKbDzzj7lcAfwx8J9xyKiu9VvI9RPO/peuBFnefCiwClodcT0WZ2ULgYVLrWkfVfwLedfdppNb0iN3Xu0fx/6hhWwF8N72dAPpDrKUatgG3hF1EhXQCGwHcfTtwYbjlVNyvgE+HXUSF/QhYnLU/GFYhYdH0TgmKrB/8SzObQmqa56vVr6z8Aq6VXO9y13o+amYJd49kULj7j83szLDrqCR3fw/AzNqAtcCd4VZUfQr9EhRaP9jMPg78N+DP3P3nVS+sAgKulVzvctd6boxq4MeJmf0WsA5Y6e5rwq6n2jS9U2Zm9jFSf0LOdvcNYdcjJRle09nMLgX2hFuOlMrMJgP/E/gLd18ddj1h0Ei//O4h9ULYvWYGcMDdZ4VbkgS0DrjWzLYBDcCckOuR0t0OdACLzSwztz/d3Q+FWFNV6auVRURiRNM7IiIxotAXEYkRhb6ISIwo9EVEYkShLyISIwp9EZEYUeiLiMTI/wccdc3D+1EGEQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(net[1].input.reshape(-1), net[1].output.reshape(-1), s=1);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The non-linear activation distorts and clips the output so it no longer resembles a Gaussian:" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD3CAYAAAAT+Z8iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAEq1JREFUeJzt3X+Q3HV9x/HnJQfE6IGH3VprYRjFvqczjnSCAtpQUlFiRCettjOOP6YFFauZEZSOooQGLTP4K1FEwQpk8BcdSjCDsQKZVqUhQlMxdqSNbwTbQduxvTAJnEbAkOsf3++Ny3G7d7vZ29v73PPx1+7n+/ne9/393ndf+9nP7n53aGJiAklSuZbMdwGSpLll0EtS4Qx6SSqcQS9JhTPoJalww/NdwFRjY+MdfwxodHQ5+/YdmItyDtug1jaodcHg1jaodcHg1jaodcHg1tZtXY3GyFCrZUWM6IeHl853CS0Nam2DWhcMbm2DWhcMbm2DWhcMbm1zUVcRQS9Jas2gl6TCGfSSVDiDXpIKZ9BLUuEMekkqnEEvSYUz6CWpcAa9JBVu4C6BIC1E537km9O2b77o5X2uRHoqR/SSVDhH9FpwWo2eW2k3qu7l35IGlSN6SSqcI3qpA52+ApAGgUGvebcY38hcjPus+WPQSwuYTxiaDYNeWgCcMtLh8M1YSSqcQS9JhXPqRn0zX9MPTntosTPopUWk3ZOeb+CWy6CX5pCvJjQIDHoNrMUYkotxnzX3fDNWkgpn0EtS4dpO3UTEEcBm4ATgKOAy4KfANuBHdberM/PGiNgAnA0cBC7IzF0RcSJwPTAB3Ausy8xDc7AfGiBOP8w//wdqNtMc/ZuBhzLzLRHxLGA38GFgU2ZunOwUESuAM4BTgeOAm4GXAJuA9Zn57Yj4HLAW2Nr73ZA0V7zMwsI3U9DfBGxpun8QOBmIiFhLNaq/AFgJbM/MCeDBiBiOiEbd94563VuBs5gh6EdHlzM8vLTjHWk0Rjpep18GtbbDreu1F97So0o0CDo9H2bbf1DPfxjc2npdV9ugz8yfA0TECFXgr6eawrk2M++JiIuBDcB+4KGmVceBY4ChOvyb29rat+9Ap/tAozHC2Nh4x+v1w6DWNqh1af50ej7Mpv8gn2eDWlu3dbV7cpjx45URcRzVKPyqzLwhIp6ZmfvrxVuBK4FbgOatjFCF/6Fp2iQNIOf1y9X2UzcR8WxgO/D+zNxcN98eEafUt88E7gF2AqsjYklEHA8sycy9wO6IWFX3XQPs6PUOSJLam2lE/0FgFLgkIi6p294LfCoiHgd+BpyXmY9ExA7gLqonj3V13wuBayLiSGAPT57v14DxTTepTDPN0Z8PnD/NopdN0/dS4NIpbfdRfRpHkjRPvASCZuTcrabjK8CFw2/GSlLhDHpJKpxBL0mFc45+EXLOXVpcHNFLUuEc0RfMkbvmg5/GGTyO6CWpcAa9JBXOoJekwjlHL6kvnLufP47oJalwBr0kFc6gl6TCGfSSVDiDXpIKZ9BLUuEMekkqnEEvSYUz6CWpcAa9JBXOoJekwhn0klQ4L2q2wPnjIpJm4ohekgrniF7SvPLyxXPPEb0kFc6gl6TCGfSSVLi2c/QRcQSwGTgBOAq4DPgP4HpgArgXWJeZhyJiA3A2cBC4IDN3RcSJ0/Wdkz0pnJ+ukdStmUb0bwYeyszTgTXAZ4BNwPq6bQhYGxErgDOAU4E3AJ+t139K397vgiSpnZmC/ibgkqb7B4GTgTvq+7cCrwBWAtszcyIzHwSGI6LRoq8kqY/aTt1k5s8BImIE2AKsBz6RmRN1l3HgGOBo4KGmVSfbh6bp29bo6HKGh5d2sg8ANBojHa/TL4NcmzSoWk1XbtvYu4mBQX1s9rquGT9HHxHHAVuBqzLzhoj4WNPiEWA/8Eh9e2r7oWna2tq378Asyn6yRmOEsbHxjtfrh0GuTVqIevV4GtTHZrd1tXtyaDt1ExHPBrYD78/MzXXz7ohYVd9eA+wAdgKrI2JJRBwPLMnMvS36SpL6aKYR/QeBUeCSiJicqz8f+HREHAnsAbZk5hMRsQO4i+rJY13d90Lgmua+vd4BSYuL36Tt3Exz9OdTBftUZ0zT91Lg0ilt903XV5LUP17rZsD4eXlJveY3YyWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mF8wtT88QvRknqF0f0klQ4g16SCufUjaQieFXL1hzRS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgrnF6bm2GsvvGW+S5C0yDmil6TCGfSSVDiDXpIK5xy9pKK1utjZto1r+1zJ/HFEL0mFM+glqXAGvSQVzqCXpMLN6s3YiDgV+GhmroqIFcA24Ef14qsz88aI2ACcDRwELsjMXRFxInA9MAHcC6zLzEO93glJUmszBn1EvA94C/CLumkFsCkzNzb1WQGcAZwKHAfcDLwE2ASsz8xvR8TngLXA1p7ugSSprdmM6B8AXgd8qb5/MhARsZZqVH8BsBLYnpkTwIMRMRwRjbrvHfV6twJnYdBLUl/NGPSZeXNEnNDUtAu4NjPviYiLgQ3AfuChpj7jwDHAUB3+zW1tjY4uZ3h46SzL/7VGY6TjdXrJa9pIC89850Yrva6rmy9Mbc3M/ZO3gSuBW4Dmykaowv/QNG1t7dt3oOOCGo0RxsbGO15P0uI2iLnRbZ61e3Lo5lM3t0fEKfXtM4F7gJ3A6ohYEhHHA0sycy+wOyJW1X3XADu62J4k6TB0M6J/J/CZiHgc+BlwXmY+EhE7gLuonjzW1X0vBK6JiCOBPcCWHtQsSerA0MTExMy9+mhsbLzjggZh6qbV9TQkDaZtG9fOe25M5zCmboZaLfMLU5JUOINekgrnZYolLUqtPhK9+aKX97mSueeIXpIKZ9BLUuEMekkqnEEvSYUz6CWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOH94RJKatPv954X6oyQGfYf8EXBJC41TN5JUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFc6gl6TCGfSSVLhZfTM2Ik4FPpqZqyLiROB6YAK4F1iXmYciYgNwNnAQuCAzd7Xq2/vdkCS1MuOIPiLeB1wLLKubNgHrM/N0YAhYGxErgDOAU4E3AJ9t1be35UuSZjKbEf0DwOuAL9X3TwbuqG/fCpwFJLA9MyeAByNiOCIaLfpubbex0dHlDA8v7WgnABqNkY7XkaRO9Ctner2dGYM+M2+OiBOamobqQAcYB44BjgYeauoz2T5d37b27Tswi7KfrNEYYWxsvOP12vHiZZKm6nXOTKfbPGv35NDNm7HNc+wjwH7gkfr21Pbp+kqS+qiboN8dEavq22uAHcBOYHVELImI44Elmbm3RV9JUh91cz36C4FrIuJIYA+wJTOfiIgdwF1UTx7rWvXtQc2SpA4MTUxMzNyrj8bGxjsuyDl6Sf3Qj1+YOow5+qFWy/zClCQVzqCXpMIZ9JJUOH8cXJJmqdV7d/2Yuz8cjuglqXAGvSQVzqCXpMIZ9JJUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFc6gl6TCGfSSVDiDXpIKZ9BLUuEMekkqnEEvSYVb9D884o+ASyrdog96STpcg/7LU07dSFLhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUuK4/Rx8Ru4GH67v/CfwtcAVwENiemR+KiCXAVcBJwGPA2zLz/sMrWZLUia6CPiKWAWTmqqa27wOvB34M/ENErABOAJZl5ksj4jRgI7D2MGuWJHWg2xH9ScDyiNhe/41LgaMy8wGAiLgdOBN4DnAbQGbeHREvPuyKJUkd6TboDwCfAK4FXgDcCuxvWj4OPA84ml9P7wA8ERHDmXmw1R8eHV3O8PDSjgtqNEY6XkeS5lK3udTrPOs26O8D7s/MCeC+iHgYOLZp+QhV8C+vb09a0i7kAfbtO9BxMY3GCGNj4x2vJ0lzqZtc6jbP2j05dPupm3Op5tuJiN+mCvRfRMTzI2IIWA3sAHYCr677nQb8oMvtSZK61O2I/jrg+oi4E5igCv5DwFeApVSfuvmXiPhX4JUR8R1gCDinBzVLkjrQVdBn5uPAG6dZdNqUfoeAv+xmG5Kk3vALU5JUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwXV+9cqFp9SvtklQ6R/SSVLhFM6KXpH5rNZOw+aKX97UOR/SSVDiDXpIKZ9BLUuEMekkqnEEvSYUz6CWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVrrjr0ftLUpL0ZI7oJalwxY3oJWnQ9fuXpxzRS1LhDHpJKpxBL0mFm/M5+ohYAlwFnAQ8BrwtM++f6+1Kkir9GNH/MbAsM18KXARs7MM2JUm1fgT9SuA2gMy8G3hxH7YpSar14+OVRwMPN91/IiKGM/PgdJ0bjZGhbjbSaIwAsG3j2m5Wl6SBMZlnvdKPEf0jQHPVS1qFvCSp9/oR9DuBVwNExGnAD/qwTUlSrR9TN1uBV0bEd4Ah4Jw+bFOSVBuamJiY7xokSXPIL0xJUuEMekkqnEEvSYVbcJcpjog/Af4sM984zbK3A+8ADgKXZebXI+I3gBuApwH/A5yTmQd6XNPTgC8DvwmMA3+emWNNy19F9a1gqN6QXgm8sK5pG/CjetnVmXljv+qq+3wNeBbwK+CXmbkmIk4ErgcmgHuBdZl5qFd1dVDbx6mO1TDw+cy8JiKOBe6r6wLYmplX9KCetpfqmK9za5a1vQd4Q333G5n5oYgYAn7Kr8+tuzLzA32u69PAH1D9fwHWAkcwz8csIn4f+FRT99OovsG/izk4t1rUdyrw0cxcNaX9tcBfU51nm+tzfsbHykwW1Ig+Iq4ALmeauiPit4B3U51Yq4HLI+IoqoN2Q2aeDuymerD22juBH9Tb+CKwvnlhZt6Wmavqf+rXqf7Be4AVwKbJZb0M+dnUVTsRWFlvf03dtglYX683RPUA7bW2tUXEHwEn1pfOWAm8PyJGqY7Z3zUds149EFteqmOez62Zanse8CbgZcBLgbMi4kXA84HvNR2nnob8THXVVgCrm2p4mAE4Zpn5/abH42eBr2bmbczdufUkEfE+4Fpg2ZT2I4BPAmcBZwDn1efebB7HbS2ooAe+Q7XT0zkF2JmZj9Un1P3Ai2i6BANwK/CKOahrVtuIiN8B3gJ8qG46GTg7Iv45Iq6LiN5+HW6GuiLi2cAzgW0RcWdEvKaprjtardeP2oC7gHPr2xPAUqpXHScDKyLijoi4KSKe0+t6prlUx3yeWzPV9hPgVZn5RP2q6wjgUarj9NyI+FZEfCMiop911SPqFwCfj4idEXHu1HWYv2M2WePTqR6L766b5urcmuoB4HXTtP8ecH9m7svMx4E7gdPpwTEbyKmbiHgr8J4pzedk5o0RsarFalMvtTAOHDOlfbKt17X97yy38V7gk5n5WH1/F3BtZt4TERcDG4C/6mNdR1KNdK4AjgV2RsQuYCgzJ9qsN+e1ZeajwKP1KOcLVFM3P4+IHwL3ZOY/RsSbgCuBPz2c+mrtLtXRl3Orm9oy81fA3nqq5uPA7sy8rx4JXp6ZN0XESqqX/i/pV13A06n+N5uonqS/FRHfZQCOWVPbW4GbMnNvfX+uzq0nycybI+KEWdTcs/NsIIM+M68DrutwtamXWhgB9je1/7Kprae1RcRXm7Y97TbqEc5rgIubmrdm5mTfrVQnVj/r+hnwufrk/7+I2A0E0DwfP5/HbBTYAnw7My+vm78JTM7pbgU+fDi1NWl3qY6+nFtd1kZELAM2U4XAu+rm71LN85KZd0bEcyOi+Ql8rus6AFwxOf8eEd+kmi8fiGNWexNPDvK5Ordma6bzrLmtIwtt6qadXcDpEbEsIo6hehl0L02XYADWADvmYNuz2cYLgR9m5i+b2m6PiFPq22cC9/S5rlcAfw8QEc+oa9wD7G565TQvx6x+A+qfqN6Q+pumRdcCr69v9/KYtbtUx3yeW21rq0fytwD/lpnvyMwn6kUbgAvqPicBD/Y45NvWBfwucGdELK1fla0EvscAHLO67RjgqMz8SVPzXJ1bs7UHeEFEHBsRRwJ/SDWFedjHbCBH9J2IiPdSzWt9rX6XfwfVE9jFmfloRFwGfKH+1MRe4Cmf1umBq+tt3Ak8PrmNiPgYsCUzd1GNlH88Zb13Ap+JiMepRtfn9bmuWyNidUTcTTWK/2Bm7o2IC4Fr6pNtD9Woutfa1kb1xufzgLfX/zuoLp9xEbA5It4F/AJ4W4/qecqlOgbk3GpbG9W0yBnAUREx+Wb6B4CPAF+OiLOpRvZ/0c+66mP2FeBuqvdWvpiZ/z4Ixywzv0b1RPRfU9aZq3OrrYh4I/CMzPx8XePtVOfZ5sz874iY9rHSCS+BIEmFK2nqRpI0DYNekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFe7/AavZf0jzFGVXAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[1].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, the next `Linear` module restores the Gaussian distribution! How does this happen when neither its inputs nor its parameters have a Gaussian distribution? (Answer: the [central limit theorem](https://en.wikipedia.org/wiki/Central_limit_theorem) which we briefly covered [earlier](https://nbviewer.jupyter.org/github/dkirkby/MachineLearningStatistics/blob/master/notebooks/Statistics.ipynb))." ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD6CAYAAACvZ4z8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAEqZJREFUeJzt3X2MXFd5x/HvrjdOcNmYpZrSN0LUmj6qVBXJoTiAXa+SEMuEyIXSKopAKClNi9w2LpYgEAe7EhIvIm5D3igOJtCClOLEBFOZWGogOMbBJbgqaa0nJC1KixTYWOt4wS2p4+0fc8MOm3lZb2Znxme/H8nSzLnn7px7fPd3z56Ze2ZoenoaSVK5hvvdAEnSwjLoJalwBr0kFc6gl6TCGfSSVDiDXpIKN9KpQkQsAXYAATwLXAUsB/YA362q3Z6Zd0XEVuAy4CSwKTMPRcQK4E5gGngE2JiZp7p9IJKk5uYyor8cIDNfD3wA2A6sBLZn5nj1766IWAmsBVYBVwC3VvtvB7Zk5hpgCNjQ5WOQJLXRcUSfmV+MiC9XT18B/AC4AIiI2EB9VL8JWA3sy8xp4ImIGImIWlX3gWr/vcClwO5WrzcxMTUQd3CNjS1jcvJEv5sxEOyLGfbFDPtixiD0Ra02OtRqW8egB8jMkxHxGeDNwFuBXwHuyMyHI+J6YCtwDDjasNsU9SmeoSr8G8taGhtbxsjIkrk0a8HVaqP9bsLAsC9m2Bcz7IsZg9wXcwp6gMx8R0S8F/gm8LrM/H61aTdwM3Av0Hiko9TD/1STspb6fVV8Tq02ysTEVL+bMRDsixn2xQz7YsYg9EW7C03HOfqIeHtEvK96eoJ6cN8TEa+pyi4GHgYOAOsiYjgizgOGM/Mp4HBEjFd11wP753UUkqR5mcuI/h7g0xHxdeAs6vPx/wXcEhHPAE8C12Tm8YjYDxykfgHZWO2/GdgREUuBI8CuLh+DJKmNoUFbvXJQ3owdhD/FBoV9McO+mGFfzBiEvmj3Zqw3TElS4Qx6SSqcQS9JhTPoJalwBr0kFW7ON0xJi8nVH76/afmeG12qSWceg146DZdvvrdp+c7rLupxS6S5c+pGkgrniF6LWqspmm79HEf6GgSO6CWpcAa9JBXOoJekwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOBc1kxaQi51pEDiil6TCdRzRR8QSYAcQwLPAVcAQcCcwDTwCbMzMUxGxFbgMOAlsysxDEbGiWd3uH4okqZm5jOgvB8jM1wMfALZX/7Zk5hrqob8hIlYCa4FVwBXArdX+z6vb1SOQJLXVMegz84vANdXTVwA/AC4AHqjK9gKXAKuBfZk5nZlPACMRUWtRV5LUI3N6MzYzT0bEZ4A3A28F3pSZ09XmKWA5cC5wtGG358qHmtRtaWxsGSMjS+Z+BAuoVhvtdxMGhn3RXaX0ZynH0Q2D3Bdz/tRNZr4jIt4LfBN4UcOmUeAYcLx6PLv8VJOyliYnT8y1SQuqVhtlYmKq380YCPZF95XQn54XMwahL9pdaDpO3UTE2yPifdXTE9SD+1sRMV6VrQf2AweAdRExHBHnAcOZ+RRwuEldSVKPzGVEfw/w6Yj4OnAWsAk4AuyIiKXV412Z+WxE7AcOUr+AbKz23zy7bpePQZLURsegz8wfA3/YZNPaJnW3AdtmlT3arK4kqTe8YUqSCmfQS1LhXOtGxWu13oy0WDiil6TCGfSSVDiDXpIKZ9BLUuEMekkqnJ+6UTHOpE/X+M1T6iVH9JJUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFc6gl6TCGfSSVDiDXpIKZ9BLUuEMekkqnEEvSYUz6CWpcG2XKY6Is4CdwPnA2cAHgf8G9gDfrardnpl3RcRW4DLgJLApMw9FxArgTmAaeATYmJmnFuA4JEktdBrRvw04mplrgPXALcBKYHtmjlf/7oqIlcBaYBVwBXBrtf92YEu1/xCwYSEOQpLUWqcvHvkCsKvh+UngAiAiYgP1Uf0mYDWwLzOngSciYiQialXdB6p99wKXAru72H5JUgdtgz4zfwQQEaPUA38L9SmcOzLz4Yi4HtgKHAOONuw6BSwHhqrwbyxra2xsGSMjS073OBZErTba7yYMDPuiN1p989SeGwfzj2HPixmD3Bcdv0owIl5OfRR+W2Z+PiJekpnHqs27gZuBe4HGoxylHv6nmpS1NTl5Yo5NX1i12igTE1P9bsZAsC/6bxD73/NixiD0RbsLTac3Y18G7AP+LDP/qSq+LyL+PDMPARcDDwMHgI9GxMeAXwWGM/OpiDgcEeOZ+TXqc/xffcFHo0XvTPpuWGkQdBrRvx8YA26IiBuqsncDfxMRzwBPAtdk5vGI2A8cpP4G78aq7mZgR0QsBY7ws/P9kqQe6DRHfy1wbZNNr2tSdxuwbVbZo9Q/jSNJ6hNvmJKkwhn0klQ4g16SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgpn0EtS4Qx6SSpcxy8ekfrFdedntOqLnddd1OOW6EzkiF6SCmfQS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgrX9s7YiDgL2AmcD5wNfBD4d+BOYBp4BNiYmaciYitwGXAS2JSZhyJiRbO6C3IkkqSmOo3o3wYczcw1wHrgFmA7sKUqGwI2RMRKYC2wCrgCuLXa/3l1u38IkqR2OgX9F4AbGp6fBC4AHqie7wUuAVYD+zJzOjOfAEYiotairiSph9pO3WTmjwAiYhTYBWwBPpaZ01WVKWA5cC5wtGHX58qHmtRta2xsGSMjS07nGBZMrTba7yYMDPtiMPX7/6Xfrz9IBrkvOq5eGREvB3YDt2Xm5yPiow2bR4FjwPHq8ezyU03K2pqcPDGHZi+8Wm2UiYmpfjdjINgXg6uf/y+eFzMGoS/aXWjaTt1ExMuAfcB7M3NnVXw4Isarx+uB/cABYF1EDEfEecBwZj7Voq4kqYc6jejfD4wBN0TEc3P11wIfj4ilwBFgV2Y+GxH7gYPULx4bq7qbgR2Ndbt9AJKk9jrN0V9LPdhnW9uk7jZg26yyR5vVlST1jjdMSVLhDHpJKpxBL0mFM+glqXAGvSQVruMNU5IG19Ufvr9p+c7rLupxSzTIHNFLUuEc0avvWo1KJXWHI3pJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwrnUjFchVLdXIEb0kFc6gl6TCGfSSVLg5zdFHxCrgI5k5HhErgT3Ad6vNt2fmXRGxFbgMOAlsysxDEbECuBOYBh4BNmbmqW4fhCSptY5BHxHvAd4O/LgqWglsz8wbG+qsBNYCq4CXA3cDvwNsB7Zk5tci4hPABmB3V49AktTWXEb0jwNvAf6uen4BEBGxgfqofhOwGtiXmdPAExExEhG1qu4D1X57gUsx6CWppzoGfWbeHRHnNxQdAu7IzIcj4npgK3AMONpQZwpYDgxV4d9Y1tbY2DJGRpbMsfkLq1Yb7XcTBkY3+uLyzfd2oSV6Ibp9Tvs7MmOQ+2I+n6PfnZnHnnsM3AzcCzQe5Sj18D/VpKytyckT82hS99Vqo0xMTPW7GQPBvihHN/8fPS9mDEJftLvQzOdTN/dFxGuqxxcDDwMHgHURMRwR5wHDmfkUcDgixqu664H983g9SdILMJ8R/buAWyLiGeBJ4JrMPB4R+4GD1C8eG6u6m4EdEbEUOALs6kKbJUmnYU5Bn5nfAy6sHn8beF2TOtuAbbPKHqX+aRxJUp94w5QkFc6gl6TCGfSSVDiDXpIKZ9BLUuEMekkqnEEvSYUz6CWpcAa9JBXOoJekwhn0klQ4g16SCjef1Sultq7+8P39boKkBga9tIi0uwjvvO6iHrZEveTUjSQVzqCXpMIZ9JJUOINekgpn0EtS4Qx6SSqcQS9JhTPoJalwBr0kFW5Od8ZGxCrgI5k5HhErgDuBaeARYGNmnoqIrcBlwElgU2YealW3+4chSWql44g+It4D3AGcUxVtB7Zk5hpgCNgQESuBtcAq4Arg1lZ1u9t8SVInc5m6eRx4S8PzC4AHqsd7gUuA1cC+zJzOzCeAkYiotagrSeqhjlM3mXl3RJzfUDSUmdPV4ylgOXAucLShznPlzeq2NTa2jJGRJXNo+sKr1Ub73YSBYV+Ubz7/x54XMwa5L+azemXjHPsocAw4Xj2eXd6sbluTkyfm0aTuq9VGmZiY6nczBkKrvnA54rKc7vnu78iMQeiLdhea+Xzq5nBEjFeP1wP7gQPAuogYjojzgOHMfKpFXUlSD81nRL8Z2BERS4EjwK7MfDYi9gMHqV88Nraq24U2S5JOw5yCPjO/B1xYPX6U+idsZtfZBmybVda0riSpd7xhSpIK51cJSgJav7nuVwye+RzRS1LhDHpJKpxBL0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgpn0EtS4bwzVlJbre6Y3XOjXxh3pjDo1dHlm+/tdxMkvQBO3UhS4Qx6SSqcQS9JhTPoJalwBr0kFc6gl6TCGfSSVDiDXpIKZ9BLUuEMekkq3LyXQIiIw8DT1dP/BP4WuAk4CezLzL+KiGHgNuBVwE+Ad2bmYy+syZIGQaulMXZed1GPW6JO5hX0EXEOQGaON5T9C/D7wH8A/xgRK4HzgXMy87URcSFwI+BKSAOq1eJVks5s8x3RvwpYFhH7qp+xDTg7Mx8HiIj7gIuBXwK+ApCZD0XEq19wiyVJp2W+QX8C+BhwB/BKYC9wrGH7FPBrwLnMTO8APBsRI5l5stUPHhtbxsjIknk2q7tqtdF+N0E64yzW35tBPu75Bv2jwGOZOQ08GhFPAy9t2D5KPfiXVY+fM9wu5AEmJ0/Ms0ndVauNMjEx1e9mSGecxfh7Mwh50e5CM99P3VxNfb6diPhl6oH+44j49YgYAtYB+4EDwBurehcC35nn60mS5mm+I/pPAXdGxIPANPXgPwV8DlhC/VM334yIfwbeEBHfAIaAq7rQZknSaZhX0GfmM8CVTTZdOKveKeBP5/MakqTu8IYpSSqcQS9JhfPLwSV1Vasb77xjtn8c0UtS4RzRL0IudSAtLo7oJalwBr0kFc6pG0k94Zu0/eOIXpIK54i+YL7pKgkc0UtS8Qx6SSqcQS9JhTPoJalwvhl7hvMNV53p/NjlwnNEL0mFM+glqXAGvSQVzqCXpML5ZuwZwjddtdj4Jm33OKKXpMI5opd0RnGkf/oM+gHjFI00P14AWlvwoI+IYeA24FXAT4B3ZuZjC/26kqS6Xozofw84JzNfGxEXAjcCG3rwuj3lSFwaTI70exP0q4GvAGTmQxHx6h685oIx0KUy9OJ3eVAuJr0I+nOBpxuePxsRI5l5slnlWm10qAdtmpNabfR5ZXtuLO6PEUmF68XHK48DjYk53CrkJUnd14ugPwC8EaCao/9OD15TklTpxdTNbuANEfENYAi4qgevKUmqDE1PT/e7DZKkBeQSCJJUOINekgpn0EtS4VzrpkFEvBn4g8y8ssm2Pwb+BDgJfDAzv9zr9vVKRLwI+HvgF4Ap4B2ZOTGrzpeAnwf+D/ifzFzf84YuoE5Ldyyy86FTX3wceD31cwVgQ2Y+/bwfVJCIWAV8JDPHZ5VfDnyA+nmxMzN39KF5z+OIvhIRNwEfokmfRMQvAn9B/WReB3woIs7ubQt76l3AdzJzDfBZYEuTOiuA1Zk5XlrIV366dAdwHfWlO4BFeT607IvKSmBddS6ML4KQfw9wB3DOrPKzgL8GLgXWAtdU50rfGfQzvkE94Jp5DXAgM39SncSPAb/ds5b13k+XrQD2Apc0boyIlwEvAfZExIMR8aYet68XfmbpDqBx6Y5Fez7M7otqtP9K4JMRcSAiru5PE3vqceAtTcp/E3gsMycz8xngQWBNT1vWwqKbuomIPwL+clbxVZl5V0SMt9ht9jIOU8DyBWhez7Xojx8wc7zNjnUp9VHdTcBLgQMRcSgzf7iQbe2xdkt3FHs+tNCuL34OuBnYDiwBvhoR38rMf+1DO3siM++OiPObbBrY82LRBX1mfgr41GnuNnsZh1HgWNca1UfN+iMi7mHmeJsd65PAJ6pf9B9GxGEggJKCvt3SHcWeDy2064sTwE2ZeQIgIu6nPpdfbNC3MbDnhVM3c3MIWBMR50TEcup/oj3S5zYtpJ8uWwGsB/bP2n4J8A8AEfFi4LeAIz1rXW+0W7pj0Z4PTfriN4AHI2JJNUe9Gvh275s4EI4Ar4yIl0bEUuB3gYN9bhOwCEf0pyMi3k19zu1L1ScL9lO/OF6fmf/b39YtqNuBz0TEg8AzwJUAEfFRYFdm7o2IdRHxEHAKeH9mPtW/5i6I5y3dsYjPh0598TngIeqfwPpsZv5bH9vacxFxJfDizPxk1S/3UT8vdmbm9/vbujqXQJCkwjl1I0mFM+glqXAGvSQVzqCXpMIZ9JJUOINekgpn0EtS4f4f8YmSmvF49UMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[2].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The next activation is `ReLU`, which effectively piles up all negative values from the previous `Linear` module into the zero bin:" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD3CAYAAAAALt/WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAFnhJREFUeJzt3X+U3HV97/HnJpNkSbIJKw4CsYgW+MgRoUQqIEFpEZS0UaTtvZd477X8qIC1VqHyo17UHn8VOakVaLRoaO0P2lIRgVtE76nakoUQa1INCp8QFcINVzPikt3sJrvZZO8f+2Nmh9nd2d2Z+X6/M8/HOZzz/c5nduY9Hyav+Zz3zPf7bRseHkaSlF3zki5AkjQ3BrkkZZxBLkkZZ5BLUsYZ5JKUcblGP2Gh0JuKn8l0di6mu7s/6TJSwbkoci6KnIuiNMxFPt/RNtlYy67Ic7n5SZeQGs5FkXNR5FwUpX0uWjbIJalZGOSSlHEGuSRlnEEuSRlnkEtSxhnkkpRxBrkkZZxBLkkZZ5BLUgPsKuyt22Mb5JJUZ7sKe7lpw+a6hblBLkl1tiK/lI9d/npW5JfW5fENcklqgHqFOBjkkpR5BrkkZZxBLkkZZ5BLUo319A029PkMckmqoZ6+QW7a8FhDw9wgl6QaWrZkIR+7/AyWLVnYsOc0yCWpxhoZ4mCQS1LmGeSSlHEGuSTNQaN/oVKJQS5Js5TEL1QqMcglaZaS+IVKJQa5JM3C2Clpkw5xMMglacbqfX7xmTLIJWkGdhX21v384jNlkEtSlUpX4mkJcTDIJalqe/cdSLqEiqoK8hDCGSGEb1e4fU0I4TshhEdDCL9X8+okKSV6+gZZ/9XHuX7taalajUMVQR5CuA74ItBedvsC4DPABcCbgHeHEI6qR5GSlLSxnxqGYzuTLuVFclXc50fAxcDflt1+ErAjxtgNEELYCJwD/PNUD9bZuZhcbv4sSq29fL4j6RJSw7koci6KnIuifL6DfD7pKiqbNshjjPeEEI6rMLQM2FOy3wssn+7xurv7qy6unvL5DgqF3qTLSAXnosi5KHIuitIwF1N9qM7ly84eoPSRO4AX5vB4kpQqSR96X625BPkTwAkhhJeEEBYCbwQerU1ZkpSskZ8aJn8elWpU0yOfIISwFlgaY7wjhHAN8HVGPhDujDHuqnWBktRoPX2DfPoftnLdJael4hD86bQNDw839AkLhd7GPuEk0tDzSgvnosi5KGr1uejpGxwP8TTMRT7f0TbZmAcESVIFWViJjzHIJWlUWk6CNVMGuSQBcWd3qs5oOBMz/rJTkprNlu27+dJDMZWH31fDFbmklhZ3dnP7Vx7nXW8NqTz8vhoGuaSWtvSwBSxeNJ/jVxyedCmzZmtFUssaa6nc+N9fl6lfqZRzRS6pJZW2VLLYFy/lilxSyxk77P76tadlti9eyhW5pJbS0zfI9Z/v4ua7tiZdSs0Y5JJaSm//IAMHhrls9aubYjUOBrmkFrKrsJe9+w6wuH0+p/zyS5Mup2bskUtqCSOnpd0MjPTGs/wrlXIGuaSWsbg9xx9c/NqmaamMMcglNb24s5v1X32cG9+5MvM/NazEHrmkphZ3dnPzXVt5z0UnN2WIg0EuqYnFnd3ces/3OWzRfI4+YknS5dSNQS6pKe0q7OXmu7ayb+Ag7/utU5rqy81y9sglNa0l7Tne24RfbpYzyCU1nV2FvXz6H7ZyQ5N+uVnO1oqkptLTN8in/u67XHdJNi8SMRsGuaSm0dM3yP97vo/+gYPs3Xcg6XIaxiCX1BR6+gb50Bc2cfQRS5rmrIbVMsglNYVvbH6Gvv1D9PYPtlSIg0EuqQk80PVjHnzsWVaf8Ust0xcvZZBLyrQt23fztceeZWGujQte/4qky0nEtD8/DCHMA9YDpwIDwBUxxh0l438EXAIcAj4ZY7y3TrVK0gRbtu/m9q88DsBlq1/d1Af9TKWaFflFQHuM8SzgBmDd2EAI4XDgfcBZwAXAn9ejSEkq19M3yOe/OhLi57z2Zaw65ZiEK0pONUG+CngIIMa4CTi9ZKwPeAZYMvrfoVoXKEmVfGPzMwwdgvNWHsOlv/GapMtJVDVHdi4D9pTsHwwh5GKMQ6P7zwI/BOYDn5ruwTo7F5PLzZ9xofWQz3ckXUJqOBdFzkVRWufiSw88zoOPPcuiBfP43bedwuEdi+r+nGmdC6guyHuA0lcwryTELwSOBl45uv/1EEJXjHHzZA/W3d0/q0JrLZ/voFDoTbqMVHAuipyLorTOxZbtu/nyt38EwPt/51QO7B+ksH+wrs+ZhrmY6oOkmtZKF7AaIIRwJrCtZKwb2AcMxBj3Ay8Ah8+6UkmaQk/f4PiXmycdu7zlfi8+mWpW5PcC54cQHgHagEtDCNcAO2KM94cQ3gxsCiEcAjYC/6d+5UpqZX/xle8BsHxxjg+ufV3C1aTHtEEeYzwEXFV285Ml4x8BPlLjuiRpgge6fsxTu3o5YUUHN/6PX026nFTxgCBJqbdl+27uffhpAN5yRmse9DMVg1xSqpX2xc9beQwrTzwy4YrSxyCXlGp/fEcXACuOOIx3XvDqhKtJJ4NcUmpd/7ku+geGAfxycwoGuaRU+sv7tlHYMwDAJecd37LnUamGQS4pdbZs381jTxSAkd+Ln/+rxyZcUboZ5JJS54mnfwHAK49aYkulCtUcECRJDfP333iSf93yHOetPMYvN6vkilxSajzQ9WP+dctzAJx72ssTriY7DHJJqRB3dvOfT430xd9xznEtecm22bK1IilxcWc3N9+1FRjpi685+1UJV5QtrsglJe62e/4TGDkZ1k2/e0bC1WSPQS4pUdfe9vD4QT9/cvmZCVeTTQa5pMT81b/8gO6+A8DIeVQ86Gd2DHJJidhV2MvD234GwBEdC/2p4RwY5JIScdOGkStCti+AW35/VcLVZJtBLqnhrvz0N8e311/76wlW0hwMckkNdfs93+PAoZHt9158crLFNAmDXFLDbNm+my1PPQ+MnAzLi0TUhkEuqSFKr/QDnl+8lgxySQ3x/ts2jm/feYN98VoyyCXV3VW3FL/cvOS84xOspDkZ5JLq6mN//RiDB0e288sXeZGIOjDIJdXNxu8/x09+2je+f/PVZydYTfMyyCXVzZ0PPlncti9eNwa5pLq47E+LffHLVnv4fT0Z5JJq7g/+/Nvj28fmF7PqlGOSK6YFTHthiRDCPGA9cCowAFwRY9xRMn4h8JHR3S3A78cYh+tQq6QM+PK3nqJv/6Hx/Y96atq6q2ZFfhHQHmM8C7gBWDc2EELoAG4BfjPGeCbwNPDSOtQpKSMefOzZ8W374o1RTZCvAh4CiDFuAk4vGXsDsA1YF0J4GPhZjLFQ8yolZUJpX9wQb5xqrtm5DNhTsn8whJCLMQ4xsvr+NeBXgL3AwyGER2OM2yd7sM7OxeRy8+dSc83k8x1Jl5AazkWRc1E0k7lYc+1949tvOvXoppvHNL+eaoK8Byh9BfNGQxzgeeA7McafAoQQ/p2RUJ80yLu7+2dZam3l8x0UCr1Jl5EKzkWRc1E0k7m4/nNdE/bfdeFJTTWPaXhfTPVBUk1rpQtYDRBCOJORVsqY7wInhxBeGkLIAWcCP5x9qZKy5svfeorCnoHxfVsqjVfNivxe4PwQwiNAG3BpCOEaYEeM8f4Qwo3A10fve3eM8fHJHkhS8/HLzeRNG+QxxkPAVWU3P1ky/o/AP9a4LkkZUPrl5jvOOS65QlqcBwRJmpU//Oy/jW+3AWvOflVyxbQ4g1zSjMWd3fTuOzi+v8GWSqIMckkzdvNdW8e37YsnzyCXNCOlfXEvEpEOBrmkqpWGOOBFIlLCIJdUlc/809YJ+7ZU0sMglzStnr5Btv2ke3zfEE8Xg1zStN5/28bxbUM8fQxySVMq7Ysf1dmeYCWajEEuaVLlX25+8so3JFSJpmKQS6ro+lv/bcK+LZX0MsglVfTDZ14Y3zbE080gl/QipS2Vc177sgQrUTUMckkTlPfFL/2N1yRUiaplkEsaV36lH1sq2WCQSxpXeqWfB9a9PcFKNBMGuSRgYkvlvRefnGAlmimDXNKL+uIrTzwyoUo0Gwa51OLKQ9y+ePYY5FIL21XYO2HfEM8mg1xqYTdt2Dy+bYhnl0EutajSlsryxbkEK9FcGeRSCyrvi3/mfW9MqBLVgkEutRiv9NN8DHKpxXiln+ZjkEstpLSlctjCBAtRTRnkUoso74v/xTWuxpvFtF9VhxDmAeuBU4EB4IoY444K9/kX4L4Y4+frUaik2bv2tocn7NtSaS7VrMgvAtpjjGcBNwDrKtzn48BLalmYpNrp7jswvm2IN59qgnwV8BBAjHETcHrpYAjht4FDwNdqXp2kOSttqaw+45cSrET1Us1RAMuAPSX7B0MIuRjjUAjhZGAt8NvAh6t5ws7OxeRy82deaR3k8x1Jl5AazkVRM83Fmmvvm7B/9X9ZOaO/b6a5mKs0z0U1Qd4DlL6CeTHGodHt/wmsAL4JHAcMhhCejjE+NNmDdXf3z7LU2srnOygUepMuIxWci6JmmotKJ8OayWtrprmYqzTMxVQfJNUEeRewBrg7hHAmsG1sIMZ43dh2COGjwE+nCnFJjeHJsFpLNUF+L3B+COERoA24NIRwDbAjxnh/XauTNCueDKu1TBvkMcZDwFVlNz9Z4X4frVFNkuagtKXSlmAdahwPCJKaSHlffIOr8ZZgkEtN4qYvPDph35ZK6zDIpSax6/l949uGeGsxyKUmUNpSydkYbzkGuZRx5X3xO653Nd5qDHIpwyod9KPWY5BLTcIQb10GuZRRpavx176yM8FKlDSDXMqg8pbKB/7raQlVojQwyKWMsS+ucga5lCG33PXdCfuGuMAglzLliZ3FSwMY4hpjkEsZUd5SkcYY5FIG2BfXVAxyKeUuN8Q1DYNcSrnhkm1DXJUY5FKKlbZU2hckWIhSzSCXUqq8L77+Wlfjqswgl1LILzc1Ewa5lHKGuKZjkEspU7oaX3nCEQlWoqwwyKUUKW+pvPe3Tk2oEmWJQS6lhH1xzZZBLqXA9Z/rmrBviGsmDHIpBQp7Bsa3DXHNlEEuJcyTYWmuDHIpQfbFVQu56e4QQpgHrAdOBQaAK2KMO0rGPwD8t9HdB2OMf1KPQqVmY4irVqpZkV8EtMcYzwJuANaNDYQQXgW8E3gDcBZwQQjhlHoUKjUzQ1xzUU2QrwIeAogxbgJOLxl7FnhrjPFgjPEQsADYX/MqpSZjX1y1NG1rBVgG7CnZPxhCyMUYh2KMB4CfhxDagFuArTHG7VM9WGfnYnK5+bOvuIby+Y6kS0gN56Ko3nOx5tr7Juw/sO7tdX2+ufB9UZTmuagmyHuA0lcwL8Y4NLYTQmgH7gR6gfdM92Dd3f0zrbEu8vkOCoXepMtIBeeiqN5zUakvnta5931RlIa5mOqDpJrWShewGiCEcCawbWxgdCV+H/C9GOOVMcaDcytVah32xVUr1azI7wXODyE8ArQBl4YQrgF2APOBNwGLQggXjt7/xhjjo3WpVsqw0tX4UZ3tCVaiZjNtkI9+iXlV2c1Plmz7jpSmUd5S+eSVb0ioEjUjDwiS6szfi6veDHKpjj5w679P2DfEVQ8GuVRHe/rHf+BliKtuDHKpTjzoR41ikEt1YF9cjWSQSzVmiKvRDHKpjgxxNYJBLtWQfXElwSCXasSWipJikEs1YIgrSQa5VGOGuBrNIJfmqHQ1ftKxyxOsRK3KIJfmoLyl8sG1r0uoErUyg1yaJfviSguDXJqFd99siCs9DHJpFoaGi9uGuJJmkEsz5EE/ShuDXJoB++JKI4NcqpIhrrQyyKVZMMSVJga5VAX74kozg1yahi0VpZ1BLk3BEFcWGORSlQxxpZVBLk2idDV+3spjEqxEmppBLlVQ3lJ55wWvTqgSaXoGuVTGvriyJjfdHUII84D1wKnAAHBFjHFHyfjvAVcCQ8DHY4z/u061SnW35tr7Juwb4sqCalbkFwHtMcazgBuAdWMDIYSjgPcBZwNvAT4VQlhUj0KlRjPElRXTrsiBVcBDADHGTSGE00vGXg90xRgHgIEQwg7gFOA7Na8U6OkbZNmShfT0DdLbP8iK/NKK45P97Zjy+1Qa21XYy4r80vHn+uHTv+CI5e0A9O8fIn/4YSw9bAE/6+5n5YlHsquwd/w+uwp76ekb5BVHLSN/+GEUXtjH4vYcP961h4EDB+nuHQBgQW7e+OMd//LlPPlMN/sHh/hFzwC/vGI5z/28j+f37Gd4GA4Ow7LFOfYPDjEwBAvmwfDwxLPwSWpN1QT5MmBPyf7BEEIuxjhUYawXmPJaV52di8nl5s+40Bd6B/jIbRv5xNVn86EvbqJv3xC3/9Gv8Yqjl00Yv/2Dv87hHYte9Lc33fowh4aHmdfWxvrrzgMgn++oOLZn7wAf3rCZT159Nh//68fo2zc0ZW3vfttruOP+H1QYea7q17ftJ90T9rc89fyL7rOnv1jHgUNVP7Rm4YF1b0+6hFTI5zuSLiE10jwXbcPDUy/pQgh/BmyKMd49uv9/Y4wvH91+G/DWGON7RvfvBT4RY/yPyR6vUOid9RqylivyfL6DQqG34hi4Im9ltlRGlP4baXVpmIt8vqNtsrFqVuRdwBrg7hDCmcC2krHNwCdCCO3AIuAk4PE51DqlsZBdtmRhxcCeLMRnMzb2ITH2XOUfGuX3W5FfOr698sQjJ9wnHNtZ8fZya86ecrhu0vAmTQvnQllUTZDfC5wfQngEaAMuDSFcA+yIMd4fQrgVeJiRL04/FGPcX79yJUnlpg3yGOMh4Kqym58sGf8C8IUa1yVJqpIHBElSxhnkkpRxBrkkZZxBLkkZZ5BLUsZNe0CQJCndXJFLUsYZ5JKUcQa5JGWcQS5JGWeQS1LGGeSSlHEGuSRlXDWnsW0aIYR3AL8TY1xbYaxlLiIdQjgM+DvgSEau6vSuGGOh7D73A0cAB4B9McYLG15onXhB8aIq5uJWRq7JO3aS9rfHGPe86IGaSAjhDODmGOO5ZbevAT7MyPviztEzv6ZCy6zIQwifBT5FhdfcgheRvhrYFmM8B/gb4H9VuM/xwKoY47nNFOKjvKB40aRzMWol8JbR98G5LRDi1wFfBNrLbl8AfAa4AHgT8O7R90oqtEyQA48wEmCVjF9EevSNOnYR6WY1fkFt4GvAm0sHQwgvAw4HHgghbAwh/GaD66u3CRcUBypeULzV3gvlczG6Wj8BuCOE0BVCuCyZEhvqR8DFFW4/iZGL6XTHGAeBjcA5Da1sCk3XWgkhXA58oOzmS2OM/xRCOHeSP5vxRaSzYpL5+BnF11vptS5kZGX2WeAlQFcIYXOMcXc9a22gml5QPOOmmoslwG3AnwHzgW+FEP4jxvj9BOpsiBjjPSGE4yoMpfp90XRBHmPcAGyY4Z/1AKWXyO4AXqhZUQmqNB8hhK9QfL2VXutPgc+P/mPeHULYCgSgWYK8/P/3vNHXWmmsad4Lk5hqLvqBz8YY+wFCCN9kpJfetEE+hVS/L1qptTKVzcA5IYT2EMJy6nwR6RToAlaPbl/IyDVXS70ZuBsghLAUOBl4omHV1d/465/kguIt+V6oMBcnAhtDCPNHe8SrgC2NLzEVngBOCCG8JISwEHgj8GjCNY1ruhX5TLTwRaQ/B3wphLARGATWAoQQPg18Ocb4tRDCW0IIm4BDwB/HGH+eXLk15wXFi6abi78HNjHy66W/iTH+IMFaGy6EsBZYGmO8Y3Revs7I++LOGOOuZKsr8jS2kpRxtlYkKeMMcknKOINckjLOIJekjDPIJSnjDHJJyjiDXJIy7v8D2cRb5blede8AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(net[3].input.reshape(-1), net[3].output.reshape(-1), s=1);" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD3CAYAAAD10FRmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAEBhJREFUeJzt3X+M5Hddx/Hn7m3by+lcWWSCaNpULL5jQkSv0GuxtWeBlqOSI+gfiGCkQVAv0ZOaUqD1xDRalJ6K/Ki2XpoQSJA2TUU8eiaVehytJeU0NDRvaFWqJpjpZa+3cNJyt+Mf8z0dNjOzO7PTmZvv5/lImsx85j1z73dv+/p857sz3861220kSWWZn3YDkqTJM/wlqUCGvyQVyPCXpAIZ/pJUoIVpN7BerdbyyB9LWlzcwtLSiXG2c0ZwrtlSx7nqOBPUa65mszHXa72II/+FhU3TbuE54VyzpY5z1XEmqO9c3YoIf0nS9zL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQWamcs7bMTrr7u35/r+G66ccCeSdGbwyF+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQZe3iEizgL2AxcA5wA3A/8JfAb4elX2scz8VETsBa4BTgJ7MvPhiLgQuBNoA48CuzNzpVftuAeTJPW31rV93gIczcy3RsQPAEeA3wf2Zeatp4siYhtwBbAdOA+4G3gFsA+4MTM/HxG3Absi4ht9aiVJE7JW+H8auKvr/kngIiAiYhedo/89wGXAwcxsA09GxEJENKvaB6rnHgCuArJXbWa2BjWyuLiFhYVNQ443WLPZGOvrTUMdZujFuWZHHWeC+s512sDwz8xvAUREg84mcCOd0z93ZOYjEfE+YC9wDDja9dRl4Fxgrgr57rWtfWoHhv/S0ol1jrR+rdby2F9zkprNxszP0ItzzY46zgT1mqvfJrbmL3wj4jzgH4CPZ+YngXsy85Hq4XuAnwKOA91/QoPOhrDSY61frSRpQgaGf0S8EDgIvDsz91fL90XExdXtVwGPAIeBqyNiPiLOB+Yz8yngSETsqGp3AocG1EqSJmStc/7vBRaBmyLipmrtXcCfRsSzwDeBd2Tm8Yg4BDxIZ0PZXdVeB9weEWcDjwF3ZeapPrWSpAmZa7fba1edAVqt5ZEbvfaW+3uuz/r/yatO5yW7OdfsqONMUK+5ms3GXK91v+QlSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCrQw6MGIOAvYD1wAnAPcDHwVuBNoA48CuzNzJSL2AtcAJ4E9mflwRFy43trxjyZJ6metI/+3AEcz83JgJ/BhYB9wY7U2B+yKiG3AFcB24E3AR6rnD1MrSZqQtcL/08BNXfdPAhcBD1T3DwCvBi4DDmZmOzOfBBYiojlkrSRpQgae9snMbwFERAO4C7gR+GBmtquSZeBcYCtwtOupp9fnhqhtDeplcXELCwub1jHS+jWbjbG+3jTUYYZenGt21HEmqO9cpw0Mf4CIOA+4B/hoZn4yIv6o6+EGcAw4Xt1evb4yRO1AS0sn1ioZWqu1PPbXnKRmszHzM/TiXLOjjjNBvebqt4kNPO0TES8EDgLvzsz91fKRiNhR3d4JHAIOA1dHxHxEnA/MZ+ZTQ9ZKkiZkrSP/9wKLwE0Rcfrc/28BH4qIs4HHgLsy81REHAIepLOh7K5qrwNuX2etJGlC5trt9tpVZ4BWa3nkRq+95f6e6/tvuHLkfs4EdXpr2s25ZkcdZ4J6zdVsNuZ6rfslL0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCrSwnqKI2A58IDN3RMQ24DPA16uHP5aZn4qIvcA1wElgT2Y+HBEXAncCbeBRYHdmrvSqHetUkqSB1gz/iLgeeCvw7WppG7AvM2/tqtkGXAFsB84D7gZeAewDbszMz0fEbcCuiPhGn1pJ0oSs58j/CeCNwMer+xcBERG76Bz97wEuAw5mZht4MiIWIqJZ1T5QPe8AcBWQvWozszW2qSRJA60Z/pl5d0Rc0LX0MHBHZj4SEe8D9gLHgKNdNcvAucBcFfLda1v71A4M/8XFLSwsbFqr3aE0m42xvt401GGGXpxrdtRxJqjvXKet65z/Kvdk5rHTt4E/B+4Fuv9NNehsCCs91o73qR1oaenECK0O1motj/01J6nZbMz8DL041+yo40xQr7n6bWKjfNrnvoi4uLr9KuAR4DBwdUTMR8T5wHxmPgUciYgdVe1O4NCAWknShIxy5P/rwIcj4lngm8A7MvN4RBwCHqSzoeyuaq8Dbo+Is4HHgLsy81SfWknShMy12+21q84ArdbyyI1ee8v9Pdf333DlyP2cCer01rSbc82OOs4E9Zqr2WzM9Vr3S16SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBVoYT1FEbEd+EBm7oiIC4E7gTbwKLA7M1ciYi9wDXAS2JOZDw9TO+a5JEkDrHnkHxHXA3cAm6ulfcCNmXk5MAfsiohtwBXAduBNwEdGqJUkTch6jvyfAN4IfLy6fxHwQHX7AHAVkMDBzGwDT0bEQkQ0h6nNzNagJhYXt7CwsGmI0dbWbDbG+nrTUIcZenGu2VHHmaC+c522Zvhn5t0RcUHX0lwV3ADLwLnAVuBoV83p9WFqB4b/0tKJtVodWqu1PPbXnKRmszHzM/TiXLOjjjNBvebqt4mN8gvfla7bDeAYcLy6vXp9mFpJ0oSMEv5HImJHdXsncAg4DFwdEfMRcT4wn5lPDVkrSZqQdX3aZ5XrgNsj4mzgMeCuzDwVEYeAB+lsKLtHqJUkTchcu91eu+oM0Gotj9zotbfc33N9/w1XjtzPmaBO5yW7OdfsqONMUK+5ms3GXK91v+QlSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgpk+EtSgQx/SSqQ4S9JBTL8JalAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCrQw6hMj4gjwdHX334C/AP4MOAkczMz3R8Q88FHgZcAzwNsz8/GIuGR17QZmkCQNaaTwj4jNAJm5o2vtn4GfB/4V+GxEbAMuADZn5qVV4N8K7AJuW12bmV/ewBySpCGMeuT/MmBLRBysXuP3gHMy8wmAiLgPeBXwIuBzAJn5UES8PCK29qk1/CVpQkYN/xPAB4E7gJcAB4BjXY8vAy8GtvL/p4YATlVrx3vUDrS4uIWFhU0jtttbs9kY6+tNQx1m6MW5ZkcdZ4L6znXaqOH/NeDxzGwDX4uIp4Hndz3eoLMZbKlunzZPJ/gbPWoHWlo6MWKr/bVay2N/zUlqNhszP0MvzjU76jgT1GuufpvYqJ/2uZbO+Xsi4ofohPy3I+JHI2IOuBo4BBwGXlfVXQJ8JTOPA8/2qJUkTcioR/5/BdwZEV8A2nQ2gxXgE8AmOp/g+aeI+BLwmoj4IjAHvK16/q+trt3ADJKkIY0U/pn5LPDmHg9dsqpuhU7Qr37+Q6trJUmT45e8JKlAhr8kFcjwl6QCGf6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQIa/JBXI8JekAhn+klQgw1+SCmT4S1KBDH9JKpDhL0kFMvwlqUCGvyQVyPCXpAIZ/pJUIMNfkgq0MO0GpunaW+7vub7/hivHUi9JZ6qiw7+ffiE/rnpww5A0XYb/lAy7YbhZSBonw39GeMpJ0jgZ/jPOdxCSRmH4F8Z3EJLA8FfFdxBSWQx/jcTNQppthr8mYtBm4cYgTZ7hr6kb5XsSvbiJSOs3tfCPiHngo8DLgGeAt2fm49PqR7PPTURav2ke+b8B2JyZl0bEJcCtwK4p9iMBbiIqwzTD/zLgcwCZ+VBEvHyKvUhjN65NZJy8bpVOm2u321P5gyPiDuDuzDxQ3X8SeHFmnpxKQ5JUkGle0vk40Oi6P2/wS9JkTDP8DwOvA6jO+X9lir1IUlGmec7/HuA1EfFFYA542xR7kaSiTO2cvyRpevzfOEpSgQx/SSqQ4S9JBarVtX3WumRERPwq8E7gJHBzZv7tVBodwjpm+m3gTdXdv8vM90++y+Gt5/IeVc1ngXsz87bJdzm8dfx97QT2Vne/DOzOzDP+F2/rmOt3gF8EVoA/yMx7ptLoiCJiO/CBzNyxav31wO/SyYz9mXn7FNp7TtTtyP//LhkB3EDnkhEARMQPAr8J/DRwNfCHEXHOVLoczqCZXgz8EvBK4FLgqoj4ial0Oby+c3W5GXj+RLvauEF/Xw3gj4Gfy8xLgH8HXjCNJkcwaK7n0flv61LgKuBPp9LhiCLieuAOYPOq9bOAP6Ez0xXAO6ocqYW6hf/3XDIC6L5kxMXA4cx8JjOfBh4HZiEoB830H8BrM/NUZq4AZwHfmXyLIxk0FxHxC3SOIg9MvrUNGTTXK+l8n+XWiDgE/Hdmtibf4kgGzfVt4BvA91X/rEy8u415Anhjj/UfBx7PzKXMfBb4AnD5RDt7DtUt/LcCT3fdPxURC30eWwbOnVRjG9B3psz8bmY+FRFzEfFB4Ehmfm0qXQ6v71wR8VLgzXTebs+aQT+DLwB+Fng3sBPYExE/NuH+RjVoLugciHyVzqmsD02ysY3KzLuB7/Z4aFYzY13qFv6DLhmx+rEGcGxSjW3AwMtgRMRm4BNVzW9MuLeNGDTXLwM/DNwP/Arwroh47WTbG9mguY4CX8rMb2bmt4B/BH5y0g2OaNBcO4EXAT8CnA+8ISIunnB/z4VZzYx1qVv4D7pkxMPA5RGxOSLOpfOW7tHJtzi0vjNFxBxwL/AvmfnOzDw1nRZH0neuzLw+M7dXv3y7E9iXmZ+bRpMjGPQz+Ajw0oh4QXXUfAmdo+VZMGiuJeB/gGcy8zt0AvJ5E+9w/B4DXhIRz4+Is4GfAR6cck9jU6tP+9DjkhER8S465+3+JiI+BByis+m9r/pBPdP1nQnYROcXUedUnyIBeE9mzsIP6MC/q+m2tiFr/Qy+B7ivqv3rzJyFAxBYe65XAw9FxAqdc+N/P8VeNyQi3gx8f2b+ZTXjfXQyY39m/td0uxsfL+8gSQWq22kfSdI6GP6SVCDDX5IKZPhLUoEMf0kqkOEvSQUy/CWpQP8Lg39cWAdJexgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[3].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The final linear layer's output is again roughly Gaussian, thanks to the central limit theorem:" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD3CAYAAAANMK+RAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAEBNJREFUeJzt3W+MXNV5x/Gv7QUTS4u7qaap2kIQTfS0UtVIpmD+yi6kuNRFblNUIRoaQRqi1FJxkgqcYGqnokpIMGqTENICLjRN1DR2KKWpg6USiOO4sUpIFVT0EGio2xdEa2uNFyxCbG9fzJhMzPzb2fmze+b7eXXn3jszz5mZ/d2zZ+49s2hmZgZJ0sK3eNgFSJJ6w0CXpEIY6JJUCANdkgphoEtSIcaG+eSTk9NFnWIzMbGMqakjwy6jr2xjGWzjwlapjC9qtN4eeg+NjS0Zdgl9ZxvLYBvLZKBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhhnrpvzQo13/s0Ybrt228dMCVSP1jD12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgrR9krRiFgC3AMEcAy4DlgE3A/MAE8B6zPzeERsBtYCR4ENmbmvT3VLkk7SyaX/VwJk5kURsRq4k2qgb8rMxyLis8C6iPgfYBWwEjgD2AGc25eqpR5xSgCVpO2QS2b+E3BD7eabgR8A5wCP19btBN4OXAzsysyZzNwPjEVEpfclS5Ia6Whyrsw8GhEPAL8LXAX8dmbO1DZPA8uB04GDdXc7sX6y2eNOTCxjbGxJN3XPW5XK+LBL6DvbWAbbWJ6OZ1vMzHdFxM3At4A31G0aBw4Bh2vLJ69vamrqSOeVLgCVyjiTk9PDLqOvRqGNQPFtHIX3seQ2NjtQtR1yiYhrI+JDtZtHgOPAf9TG0wGuAHYDe4A1EbE4Is4EFmfmgbkWLknqTCc99C8DfxsRXwdOATYATwP3RMSpteXtmXksInYDe6keKNb3qWZJUgNtAz0zXwZ+v8GmVQ323QJsmXNVkqRZ88IiSSqEP0GnBanZ+ePSKLOHLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgphoEtSITwPXfOa55tLnbOHLkmFMNAlqRAGuiQVwjF0qQF/a1QLkT10SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYVoeel/RJwCbAPOApYCtwH/BzwMfK+2292Z+cWI2AysBY4CGzJzX7+KliS9Xru5XN4JHMzMayPip4EngT8H7szMrSd2iogVwCpgJXAGsAM4tz8lS5IaaRfoXwK2190+CpwDRESso9pL3wBcDOzKzBlgf0SMRUQlMyf7UbQk6fVaBnpmvgQQEeNUg30T1aGXezPziYi4BdgMHAIO1t11GlgOtAz0iYlljI0t6b76eahSGR92CX03Cm1spqS2l9SWZkahjfXaTp8bEWcADwKfycwvRMRPZeah2uYHgU8BDwH1r9w41ZBvaWrqyOwrnscqlXEmJ6eHXUZfjUIbWyml7aPwPpbcxmYHqpZnuUTEm4BdwM2Zua22+pGIOK+2fBnwBLAHWBMRiyPiTGBxZh7oSeWSpI6066F/GJgAbo2IW2vrPgD8ZUS8CrwA3JCZhyNiN7CX6kFifb8KliQ11m4M/UbgxgabLmyw7xZgS0+qkiTNmhcWSVIhDHRJKoSBLkmFaHvaojQI13/s0WGX0JHZ1rlt46V9qkR6PXvoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCK8U1UAtlCtCpYXIHrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklSIlnO5RMQpwDbgLGApcBvwX8D9wAzwFLA+M49HxGZgLXAU2JCZ+/pXtrTwNZvXZtvGSwdciUrRrof+TuBgZl4CXAF8GrgT2FRbtwhYFxErgFXASuBq4K7+lSxJaqTdbItfArbX3T4KnAM8Xru9E7gcSGBXZs4A+yNiLCIqmTnZ6sEnJpYxNraku8rnqUplfNgl9N0otLFXupldclCv7yi8j6PQxnotAz0zXwKIiHGqwb4JuKMW3ADTwHLgdOBg3V1PrG8Z6FNTR7qrep6qVMaZnJwedhl9NQptHLZBvL6j8D6W3MZmB6q2X4pGxBnA14DPZeYXgON1m8eBQ8Dh2vLJ6yVJA9Iy0CPiTcAu4ObM3FZb/WRErK4tXwHsBvYAayJicUScCSzOzAN9qlmS1EC7MfQPAxPArRFxa23djcAnI+JU4Glge2Yei4jdwF6qB4n1/SpYktRYuzH0G6kG+MlWNdh3C7ClJ1VJkmbNC4skqRAGuiQVwkCXpEK0+1JU6ko3F9RImht76JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQnraornlqojS/2EOXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqREezLUbESuD2zFwdESuAh4Hv1TbfnZlfjIjNwFrgKLAhM/f1pWJJUkNtAz0ibgKuBV6urVoB3JmZW+v2WQGsAlYCZwA7gHN7Xq00wppNV7xt46UDrkTzVSdDLs8B76i7fQ6wNiK+HhH3RcQ4cDGwKzNnMnM/MBYRlT7UK0lqom0PPTN3RMRZdav2Afdm5hMRcQuwGTgEHKzbZxpYDky2euyJiWWMjS2ZddHzWaUyPuwS+m4U2jhMs319u30/RuF9HIU21uvmF4sezMxDJ5aBTwEPAfWv3DjVkG9paupIF08/f1Uq40xOTg+7jL4ahTYO22xf327ej1F4H0tuY7MDVTdnuTwSEefVli8DngD2AGsiYnFEnAkszswDXVUqSepKNz309wGfjohXgReAGzLzcETsBvZSPUis72GNkqQOdBTomfk8cH5t+dvAhQ322QJs6V1pkqTZ8MIiSSqEgS5JhehmDF1SHzW7gEhqxx66JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY6JJUCANdkgrh5Fxqy8mipIXBHrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhKctSgtcs9NKt228dMCVaNjsoUtSITrqoUfESuD2zFwdEW8B7gdmgKeA9Zl5PCI2A2uBo8CGzNzXp5olSQ207aFHxE3AvcBptVV3Apsy8xJgEbAuIlYAq4CVwNXAXf0pV5LUTCc99OeAdwCfq90+B3i8trwTuBxIYFdmzgD7I2IsIiqZOdnrgjV3jrlKZWob6Jm5IyLOqlu1qBbcANPAcuB04GDdPifWtwz0iYlljI0tmVXB812lMj7sErrmnC1lafdZXMif1U6NQhvrdXOWy/G65XHgEHC4tnzy+pampo508fTzV6UyzuTk9LDLkABafhZH4bNachubHai6CfQnI2J1Zj4GXAF8DXgW+HhE3AH8ArA4Mw90WaukPnLIrVzdBPoHgXsi4lTgaWB7Zh6LiN3AXqpftK7vYY2SuuAQ2ujpKNAz83ng/NryM1TPaDl5ny3Alt6VJkmaDS8skqRCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEP7ARcG8sEQaLfbQJakQBrokFcJAl6RCGOiSVAi/FJUEOK1uCeyhS1IhDHRJKoSBLkmFcAxdUkuOrS8c9tAlqRAGuiQVwkCXpEIY6JJUCL8ULYCzKkoCA11Slzz7Zf5xyEWSCmGgS1Ihuh5yiYgngRdrN78P/DXwV8BRYFdmfmTu5UmSOtVVoEfEaQCZubpu3XeA3wP+G/hKRKzIzG/3okhJUnvd9tDfBiyLiF21x9gCLM3M5wAi4hHgMqBloE9MLGNsbEmXJcxPlcr4sEuQhqrZl6UPb1034EpG7++x20A/AtwB3Au8FdgJHKrbPg2c3e5BpqaOdPn081OlMs7k5PSwy5DmpUH/bZT899jsQNVtoD8DPJuZM8AzEfEi8Ma67eP8ZMBLkvqs27Ncrge2AkTEzwHLgJcj4hcjYhGwBtjdmxIlSZ3otod+H3B/RHwDmKEa8MeBzwNLqJ7l8q3elChJ6kRXgZ6ZrwLXNNh0/tzKkSR1ywuLJKkQBrokFcJAl6RCGOiSVAgDXZIK4XzoC4g/ZKGFrNXn1znUe8MeuiQVwh66pKHz1496wx66JBXCQJekQjjkMkT+mympl+yhS1Ih7KHPQ56eKKkbBrqkecthydlxyEWSCmGgS1IhHHKRtOA4FNOYPXRJKoQ9dEnFGPWeuz10SSqEPfQeuvKDDzVcPyq9A0nDZQ9dkgphD11S8WZ79fVC/a/aQB8AL+WXNAg9DfSIWAx8Bngb8EPgjzLz2V4+hyT120I9W6bXPfTfAU7LzAsi4nxgK7Cux88xVPa2Jc1XvQ70i4GvAmTmv0fEr/X48V/TqzExA1rSXM2XMfpFMzMzPXuwiLgX2JGZO2u39wNnZ+bRnj2JJKmhXp+2eBgYr398w1ySBqPXgb4H+C2A2hj6d3v8+JKkJno9hv4g8BsR8U1gEXBdjx9fktRET8fQJUnD46X/klQIA12SCmGgS1IhnMtlDiLiDcDfAz8DTAPvyszJBvstA74JbMzMrw62yrnppI0R8QmqF5WNAX+TmfcMvNAutJuqIiLeA7wXOArclpn/MpRC56CDNr4fuLp2818z8yODr3JuOplypLbPV4CHMvOzg69yMOyhz837gO9m5iXA3wGbmux3F7BQv31u2caI+HXgLZl5AdVQvzkiJgZfZldem6oC2Eh1qgoAIuJngT8BLgLWAB+NiKVDqXJuWrXxbOAPgAuBC4DLI+JXh1Ll3DRtY53bgDcOtKohMNDn5rWpDoCdwNtP3iEi/pRq7/w/B1hXL7Vr417g+tryDLAE+NFgSpuzn5iqAqifquI8YE9m/jAzXwSeBRZi2LVq4/8Cv5mZxzLzOHAK8MrgS5yzVm0kIq4CjlP9/BbNIZcORcS7gfeftPoHwIu15Wlg+Un3uQx4a2a+NyIu6n+Vc9NNGzPzFeCViDgFeIDqkMtL/a61R07nx20DOBYRY7Wrm0/e9rq2LxBN25iZPwIORMQi4BPAk5n5zFCqnJumbYyIXwGuAa4C/mwo1Q2Qgd6hzLwPuK9+XUR8mR9PdTAOHDrpbu8G3hwRjwG/BKyIiBcy8zt9LrcrXbaR2hDLduCxzPxov+vsoVZTVZy8rWHbF4CW03FExGnANqoHrD8ecG290qqNfwj8PPAocBbwakQ8v9C+y+qUgT43J6Y62AdcAeyu35iZ15xYjoj7gX+Yr2HeQss21r40/Tdga2Z+fvDlzcke4ErgHxtMVbEP+Ita4C0Ffhl4avAlzlnTNtZ65g8Bj2bm7UOqrxeatjEzbzqxHBFbgBdKDXMw0OfqbuCBiPgG8CrVf+2IiI8D2zNz3zCL65GWbaT6peHZwHtqZ4UAXJeZ3x9GsbP0uqkqIuIDwLOZ+c8R8UmqB7DFwC214aWFpmkbqX7fsQpYGhFX1Pb/UGbuHU6pXWv5Pg63tMHy0n9JKoRnuUhSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVIj/B64/aCaFQAWcAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[4].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So far we have only looked at distributions of the tensors involved in the forward pass, but there is also a lot to learn from the backwards gradient tensors that we do not have time to delve in to. For example, this scatter plot offers some insight into a suitable learning rate for the second `Linear` module's weight parameters:" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD3CAYAAAAQYlNPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzsvXl8VeWd+P8+d82+kI3skAROAllIgAQlsUoUJXUB6gZorajFUWynnflV0drftNqKdtSZKXXq2mprmOIUrFpcEZVFErKQBZLDIms2Etaw5ebee75/nHsON+EGSEhAJs/79eLFzVme5/Nsn8/nWc7zSKqqIhAIBILhi+lSCyAQCASCS4swBAKBQDDMEYZAIBAIhjnCEAgEAsEwRxgCgUAgGOZYLrUAA6G9vfNbv9QpPDyAQ4dOXGoxzomQc3ARcg4+l4usl4OcUVHBkq/rokcwRFgs5kstwnkh5BxchJyDz+Ui6+Uipy+EIRAIBIJhjjAEAoFAMMwRhkAgEAiGOcIQCAQCwTBHGAKBQCAY5ghDIBAIBMMcYQgEAoFgmCMMgUAgEAxzhCEQCASCYY4wBAKBQDDMEYZAIBAIhjnCEAgEAsEwRxgCgUAgGOYIQyAQCATDHGEIBAKBYJgjDIFAIBAMc4QhEAgEgmGOMAQCgUAwzBGGQCAQCIY5whAIBALBMMcykJdkWTYBLwE5QBdwv6Io273uPwAsAJzA04qifCDLciRQCvgDzcC9iqKckGX5YeAHgAr8SlGUDy4gPQKBQCDoJwPtEcwE/BRFuQJ4DHhevyHL8kjgR8BU4HrgGVmW7cAvgFJFUYqAamCBxzg8BFwJFAP/LcuyNNDECAQCgaD/DNQQFAIfASiKsgGY5HUvH1inKEqXoihHgO1Atvc7wIfAtYqidAA5iqJ0AyOBw4qiqAOUSSAQCAQDYEBDQ0AIcMTrb5csyxZFUZw+7nUCob2u69dQFMUpy/JC4JfAf51P5OHhAVgs5gGKfvGIigq+1CKcF0LOwUXIOfhcLrJeLnL2ZqCG4CjgnWKTxwj4uhcMHPa6ftLrGgCKoiyRZfkV4ENZlq9RFGX12SI/dOjEAMW+eERFBdPe3nmpxTgnQs7BRcg5+Fwusl4OcvZlqAY6NLQOKAGQZXkKUOd1rxwokmXZT5blUCADqPd+B5gBrJE1lnvmBbrRJp7dA5RJIBAIBANgoIZgBXBKluX1wIvAT2RZ/qksyzcritKKNsSzBvgceEJRlFPA08CdsiyvA64AliiKogA1wNfAemCDoihfXliSBAKBQNAfJFW9/OZm29s7v/VCXw7dRBByDjZCzsHncpH1cpAzKirY56pM8UGZQCAQDHOEIRAIBIJhjjAEAoFAMMwRhkAgEAiGOcIQCAQCwTBHGAKBQCAY5ghDIBAIBMMcYQgEAoFgmCMMgUAgEAxzhCEQCASCYY4wBAKBQDDMEYZAIBAIhjnCEAgEAsEwRxgCgUAgGOYIQyAQCATDHGEIBAKBYJgjDIFAIBAMc4QhEAgEgmGOMAQCgUAwzLEM5CVZlk3AS0AO0AXcryjKdq/7DwALACfwtKIoH8iyHAmUAv5AM3CvoignZFn+CXCn59WViqL8csCpEQgEAkG/GWiPYCbgpyjKFcBjwPP6DVmWRwI/AqYC1wPPyLJsB34BlCqKUgRUAwtkWU4B5gFXAlcA02VZzh5oYgQCgUDQfwZqCAqBjwAURdkATPK6lw+sUxSlS1GUI8B2INv7HeBD4FpgL3CDoiguRVHcgBU4NUCZBAKBQDAABjQ0BIQAR7z+dsmybFEUxenjXicQ2ut6JxCqKEo30CHLsgT8FqhWFGXruSIPDw/AYjEPUPSLR1RU8KUW4bwQcg4uQs7B53KR9XKRszcDNQRHAe8UmzxGwNe9YOCw1/WTXteQZdkPeAPNODx0PpEfOnRigGJfPKKigmlv77zUYpwTIefgIuQcfC4XWS8HOfsyVAMdGloHlADIsjwFqPO6Vw4UybLsJ8tyKJAB1Hu/A8wA1nh6An8HahRFWaAoimuA8ggEAoFggAy0R7ACuE6W5fWABNwry/JPge2Korwny/J/AWvQDM0TiqKckmX5aeBNz4qiDmAu2qTzdwC7LMszPGEvUhTl6wtIk0AgEAj6wYAMgWdi98Felxu97r8KvNrrnTbghl7vrAD8BiKDQCAQCAYH8UGZQCAQDHOEIRAIBIJhjjAEAoFAMMwRhkAgEAiGOcIQCAQCwTBHGAKBQCAY5ghDIBAIBMMcYQgEAoFgmCMMgUAgEAxzhCEQCASCYY4wBAKBQDDMEYZAIBAIhjnCEAgEAsEwRxgCgUAgGOYIQyAQCATDHGEIBAKB4CwcPe641CIMOcIQCAQCQR8cPe7gydfL/s8bA2EIBAKBoA9CAm08dV8BIYG2IQn/22JghCEQCM6Db0uDFVx8+mME+lNPvHsbl7p+DcgQyLJskmX5D7Isfy3L8heyLKf1uv+ALMsVsixvkGX5Rs+1SFmWP5FleY0sy3+VZTnA6/koWZa3ybIszi/+P8ylruwD5WIPDwwknsGQ7XItn7642Ok53NnVr3qi9zY6T1z64aeB9ghmAn6KolwBPAY8r9+QZXkk8CNgKnA98Iwsy3bgF0CpoihFQDWwwPP89cAnQMxAEyH49nM5j7UO9fCANwPJp8HI20tRPkMZ16VIT1iwfUD15Lml1fxsTu5FqV99MVBDUAh8BKAoygZgkte9fGCdoihdiqIcAbYD2d7vAB8C13p+uz2/Dw5QFsFlwMVUpkPBxZJ7IPk0GHl7scvn6HEHT7y2YcgUtZ6ei01/8y8k0MbP5uQSHxU0RBKdH5YBvhcCHPH62yXLskVRFKePe51AaK/r+jUURfkUQJbl8448PDwAi8U8QNEvHlFRwZdahPOiP3Ie7uwiLNg+wHgG9JrX+//38tP3+xfrneBef/c/jIFi9evCJElERASdUZ981bGB5KnVr4uFv/2cJf/ftHPW2f7U67M92185D3d28e//s+m8ZBxKBmoIjgLeKTZ5jICve8HAYa/rJ72uDYhDh04M9NWLRlRUMO3tnZdajHPSW07dQ/Pl2ejd7Uvh2V+u+flt5WxyHj3u6LN8z3avvzx1XwHdpxy0nzrdK/BVxy4kT385P/+MOHqjx3k+nvnZ2sD5ytk7D88l42DmeV+GaqBDQ+uAEgBZlqcAdV73yoEiWZb9ZFkOBTKAeu93gBnAmgHGLRgijh538PPXyvrssl/uwzvDnfNZnXK2sfXBHnf3VY/Ot471Z0L2fJ752Zxcnltafc5wL7QN+MpD77B6x3+x5jokVVX7/ZIsyybgJbSxfwm4F03Jb1cU5T1Zlh8AfohmaH6jKMrfZFmOAd5E6w10AHMVRTnuFeYuIF1RlFPnir+9vbP/Ql9khsozHEzvAPrXI7hYMvmir/y8GHH3h29rj0A38ioqv75/ChERQXQPwAMdrPzuKxxf133V0aHomV5o2gbaI/C+7itdg9wjkHxdH5AhuNQMV0MwFA3gQuW8WI3Sl5yXcqiqL76thgB6epv//xvl/OudEwZlkrI/Sl2/3pfC83XdO0/1MH2FPZROwfmEfa6y76sX4CueoUpLX4ZAfFB2GfFtHJrpS6YL6co2tR87r+7wtzE/dM43/RdzeWNIoM3Iq1//09TzGgo5F30NXZztel/ldq7y9A7TV337+Wun4+traGsgDMbwjLLnEE+8toEnXt1wxtBrbwOhx9fUfuyi1Q9hCC4ig9Hovo1Kz1ejHGjDOXrc0a911d/G/Djf9F+qtftPvl5GaNDZ17yfryL1Vt69FVrv8L0NfO/nvd/ri7MZis4TDlRUQ05vo+Cd7qb2Y32G78to+TJc/f3Go6n9GM+WVnPvjHR+/cAUfn3/FCMs7zzxju9nc3J5trR6SJfYeiMMwUXiQhv95fRB1tka7Pl6+f0Zsvi25cn59pIu9odqepw/m5NLWLD9vDzvs13T8fZifQ1/6ApucWmVYeDPNSndF33VqeeWVrNwVpZx3626z3jvbBPCveXRjcnjr57pvfd+Tmd3y1GfYQYH2Hh0bi5vfqT0SENT+zHD6QGMhRpN7ceIjwri6fsLehiNoUQYgovEhTb6vjyvoeBCwj/bZPPZGpE3/Z2o7o+B1JVSX/cGi/PpJQ3mxOvZrnnHrSvNw51dfb7jq676uuadl33Vb2XPIZ58vYzOEw7wmtnry8s+etz3h2bnMg4PzczkpXfrOXrcQcuB45gkkxan17vxUUF9tkFf8j86NxdJ4owtILzboX69qf0Yj/z76h7DOd5hxkYEnpFe3QjoTs/T9xewcFaWYay8h/KGGmEILiIXWqjn8qQGg3OFf7Z4z/Vub2M2GOnoT+9D9/L6UjSDna+9vWPv4a4Lyefez/XlvSt7Dhlxe+fRU/cVEBZsP6un68vh6G0Eeudl73eUPYd4trSah2ZmAqCi8mxpdY/n4cw5IYme85nee/j0ZfReereen83JpfOEg2dLq/nBDJnnllafEbavpZq9w/X25H99/xTDgAA8+XpZj7j1fI2PCuJ3/3oNwQG2M8qjqf0YP3+trEec3j1fPb7OE6fTcbGHPMWqIYZm7L0/q0f6G7/3GOuFyu1LTr1r6ivec63S8ZbpXPL1R/7+rsbpS9Zz9VgGKz/1bn9vw+ftAfYVn7fsfcl6Lrmrtu5nyfJ6Hp2bi5wUfka4ERFBPPTcqj7l8SVD77zz9bf3O0++XsZDMzOJjQg00h4cYPNpUB6d23e+REUFs2PXASNMb5n0uupdZ3tf8yVn54nTHvkzb1cCEiZJMuTw1cZ6GyFfH77pcuqyLS6twu1WkSRYNG9ij15AbwOq/995wnFGPg0WYtVQHwy1J3guzrVCxpcXdCFjrH09o/+tKzBfYZyP9+3daHxN2PUOrz/0J1/PtjLlfCYqe08q9ifuo8c1r9Tbs9OHL7zz1peB8pYd8FnGumx9hXP0uIM/ftiIv83E71fU93hOlyks2H7GmLmvoSBdhqb2Yzzx2oYeZeo9dKErst49DzkpvIf3690L0FFRCQ7oGbc33sNY3jJVbd3Pk6+Xo+w51CMdupJdXFplXPMewvn5a2UsLq3ioZmZBAfYMEkmHpmdxaNzc3sMy3i3Tf23PkzkLYu3nE+8tsHID33ewmSSWDRvIsEBp+cpmtqP8cSrG3j81a+N4SzQh6HKWfTK+iHr9fti2BuCwZqw8+5mPvl6mc8xWF/vnG2FzNmGMs42Znsuw+b9zOHOLiMe78mr/njMfcWpovYYp/V+prcyO5dBO58tfs/X0Jwrj7RGX24MrfQ1/HI23KrbUG66Iumr2+8rfF3J+l55U+5Rgj1XwejhAPz6/ik8s+BKHp6V2UNZeytH7yEP0HoRvfFW4gtnZfH0/QVGWN5l2Xs1kK+hkN5paGo/RkigrceEaO88bmo/xkPPrWLRK18bShbgZ3O0ydeFszORk8KNdOjvtxw4bsxJeBu1Z0ureXhWJo/NzeP3K+rpPOHg0bm5/H5FPcEBth7hPFtazT03aHugeXvy3mXiXW772jp7DGvphvDX908xntWNQXCATZvQVmF702GeLa3m0417OHayG3+7GZN0cVXzsDcE4Hs8tD94N2S90p3PBlLejUwPp/f9s60c6G0EvD2VvrxNbxkBFv72czpPOAzF1deKnb6Upy9PUJftsbl5PTwsPU5vT+7nr5UZE4rexkH33PT49C1++6J3z+pc5Xk24x8fFcSjc3ONycfe6fPVi9NlBs2r0ycrdaWor5jxHhLwzitfafNWrL1le/MjpUcPo7ecIYE2Wg4c59nSakOulgPHkZAM71PnydfLWFvbzJLl9aytbTbi9nZA9CWQLQeOG6tpnnhtg+GN6ytf9KEN3QnovTTy6HGHR5nmExxgOyMP9eeVPYd44tUNLC6t4ke3TcBsMvHY3DxDXl2hvvmRYoShGwt9bqL3yiG9nv9+RT2gGetnS6uBnk7LolfW03nCgdPlYsnyeloOHD+j7PT/9XJraj/Gov9ex8OzMuk84ejRMwDNkDw0M9OYw+g84UBCQgViwgOwWUwsXbWdZ0ur+dH3svn1Az2XmA415n/7t38b8kgGmxMnHP82WGEdPe6gq9vFk6+XMTUzFrvtzF1Njx53nPW63WZmamZsj7G+wEA7JzyKwNe7+vu9x1d7y9DV7QLoMwz9XYBpeQmEBNp6POsdble3iyde3UBhVqzx3E3fScPldLG6utl435fMdpuZ7JQIosL8fYbd2/DYbWZCAm093unqdrHo5a9ZX9/GwtmZJMeE8GnFHsob21k4K4v/+N9aslMieOrNCtbUNqMCxRMTsNvMBAbaOXT4pBFfx5GThlzePZmoMH9DrphwfwL9rD7zY1peAl3dLuOed5qb2o8xOi60R7o6jpw08ubptyp6xNXV7eLxVzfwyca9pMSG8MJfN/HwrExefKeWyenRzChIpjgvwXj+56+VMTWrZzl3HDnJU29WGNd1RZyTGsHTb1WQnRJhPKfLFh8VxNTMWIAzyuHocQfPvF1Ft9PNtLx4Ok84eOqtSq6dGM+fPlJIiQ1hRLCdrm4X0/ISGJMYRkSInWWrdzA1Swvz88p95GdE89SbFUxIi6R6WwfFeQnMKEgmeWQQN+Qn8eI7tTw8KxOb1cyv3tzIqsp9rK1r4ZHZWUSE+jMtLwGAx1/9mlWV+1hd3UxKXAgRof488doGVlXsZVVVk1Enp2bGGrJazPDI7GxeWlFneNEAk9OjiQrzN57XDdsnG/dx9/Sx5I2NZpIcRVF2HHar2Sif9fWtPDYvj0JP+nLHRHFL4WiiwvxJjQvhP/63lshQPzY2tjN+dDg1Ow6w4OZx/OHvm1lb10JOaiS/enMjn1fuY1WlJnd+RjS/enMjhVmxlDe2kZ8Rw/N/3YTbrVIwLoZf/qmc4omJZCSH87vldayra0FFJXdMFOWNbUiSRO6YSL6qaWF+STqzr0rBZjVjt5qNuv3k6+VMkqMGZc4gMND+S1/Xh7Uh0BtbcV6CoQR19G5rXwq693W98erXRoQF0OrxHLNTIvocDtDveRsT72cef/VrVlc3k516Zhjeafi8cp+hNPXrvY1Ux5GTfLxxLzlpEUSG+tPUfoywUH+sJpiaddqQPfGaZiy8w+o4cpLnllb3yIe+ZP75a2Vkp2qK6+m3Kox3Oo6c5JOKfcwvSScjeQQAq6uaUFWVmUUpTPMoy6lZsRRPTGTcqHDDCxsRFsChwyeZlpdgjKN+VdtMTmrEGRNwIYE2YsL9WbK8ni827dMUgqd8QFMkdqvZSKfuCGSnRNBy4DhPvVXJJDkKu1WT2ftaVJg/UzNjexicyenRlExJJictgpdW1PFPngnSzyr3sa6+hWIvAwuwqmofxXkJhky7Wo/y4ju1uFU3xRMTsdvM7Go9SnnDfmZMSWZyejSLS6tYVbGXTzbuM+TQy8BXOdhtZnJSI8jPiGZ0XCghgTZMksr76/dgs5r4oqqJxOhAnv/rJqPehAXZDdlaDhynrKGNouw41tS2sLFxP/NL0nnp3XqSYoJYsrye8aPDqdrWTtmWNtbWtfLI7CyK8xJYU9tMWUMbq6uaKJ6oldea2hZA5c5pY1iyvJ6ctAim5SWwrq4VUI10d3W7cHS7mJYXT1F2HDarmbkzxuFyunj81a/5rHIfa2pbjDLd1XqUZ0urGTnCn6tz43j5vS1MkqMIDrAZSjs/I5q1dS3cW5JOkL+VZ96u5JON+1hX18rUrJHA6SGjN1Y2YjbB7deMIT89miB/K59s3IfFbKJkyijGJYeTnxFDWUMbkoSRPyVTkrkyO57nSis42eXG6VIZPzqcdXVtpCWE8Or7DbhVN4vumkhRdhz/8b+1LJyVxczCFIIDbOSkRZA3NtqrbjeRkxpJfFQQ6UlhjI4LPV+1dlaEIfBBV7eL1VVNhtfi7RE++Xo56UlhPbyu3p56RvJpRdVx5KShJABMZjMnTzqYnB59hgLVw8pOiehxT1cM+nNd3S5WVTVxr6cB+uqx6A1+rUfheHuTujLX3wkJtJGeFMZL79aTEhvCU29V8mnZbsMIgOZxrqtrJSM5nMhQf8MYratr5eFZWWcMG/U2jgCfVe5lfX0b0/ISDO/t6HEHjm4XG7a0sHnXIT6v0gzXuFHhVCjtPcpANxpPvVVJYnQgL75Tw6SMGJ545Wsmp0cTHGBjTW0zi+blER8VRHZKBMEBth49u1GxISRGB1K746Dh/f/8tTI+q9zLurpW8jOiWVPbQvFETUmnxIbw73/dxEZlPwtuHkdsRCBPvLqBjzfupUrZz7/cMcFojF3dLjqOnMTR7WJqVizPLa1mcrqmcK+amMjiP1cwLS+B/IxoJqRFGnXkidc2kJ8eTWGWZkia2o/xyz+W88WmZn548zhuu3oMIYE2Y2jDYoLxo0cwOi6UQo9xzEmLICLUn65uF7tajxIZ6m+Um25s9N7SL/9UzsaG/RRmaT2ol97djN1q4q7rxtKw+wBVSgcnHS5y0iKwec73KM5LYHvTYV5cVovZJCEnhXFDfhKFWbHGHIecFE5idCBp8WHkZ8Swtq4VVVUpa2jzeLraXIPL7SJzdAQvLKtBRcXthoY9h1hw8zheeX8LhVmxTBgTyczCFCNf9Z7V+NHhLFlex8cb95KfEYPVJPFZ5T5QVU463EzNGklIoI0uh4t1dU3U7DhI5ugRTM0aiZwUTle3i/z0aNbUtZA7JpKyLa2sr2+jrKGVRfMmkp8RzQ35SUYv4+PyXaTGh7Fl1yEW3TURu6eHM6MgmalZI8kdE0XboRO8uKyWqq37kZBYdJc2Afx5lWZsFv+lErcKJknF5YbUuBDqdx6kdnsHj3wvm/LGdoqyYwkOsDE5PZoX3qkhP13rUWzY3EpRdhwHj55iXV0rd0+X+cN79aTGhfbZ9geCMAQ+sNvMZKdGYLeae3jnusL83fJaCrPiAIz7oFXYx15ez5ebWkiMDuS3/1NteGqObhe//FM5K9ft4tOKPRRlx3FDQbIRnzd611ZvwN6KTFc0X27aR+03B/nBjHRGhPj1GM4ATflGhfmTkxphKFzQPO2M5HBslp4GJjJUM1bBATa+qm3i2YVFBPlZjPh/9eZG7p2RzovLaklPCvNU9Cbcqkp5QxuF2XF0dbt6/PPuDWkKMIbCrFgc3S4Wl1aRk6oN96yta2HRvIkUZceytr6F/PRoXnynlh/MkBkR4seTr5eREhtCZKg/LQeOU721nU07DvDo3DzSUyJJHRlkKN0ZU5INZfpsabXhyXoblEA/q9HT6+p2MTk92lBUE9IiqWhsZ3J6NNubDvOHv2/mZJeLu6eP5S+fbiU/PZr4qECqt3XgdKnckJ9klP2iV9bzycZ9rK5uYlpePDlpkbywrIbCrFjCwwIYnxSGo9vFb/5SyRebmo18/HTjHtbVtbCurpWc1AgWv10FEiy4eTx//LCRYo/BeubtSrqdKiaTxFc1Wh0bEeJH5wkHL75Ty6cb9/BZ5R6+3NRCelIYe9o6eba0mvSkMBa/XcXHG/cSFxnA5p2HUD3etqPbxYbNLSCpbGzswOkCp1vFbpUob9jP51X7WFXVxLhR4by4rBa7VcLtVtmwZT8btrQY9TgqzB9lzyFefb+BVVX7KMyKpSg7ltwxUXyxqZnqbe3ML8kgPSmcCqWDSmU/kgQ35CdRv/MQd08fS0byCD6v3MeXm/bxZU0LcZEB/OffaiiemEjyyCDKG/azeedBbp46im37DrGmppnkkUHU7DjA/O9mkJ8RjZwUbizPvP/GcdRsb6dC6aC8Yb/hPEzNimVtXQtlW1qZUzwWZe9hHpmdTUSoP4tLqyjMiuWXfyonyN/Kpu0Hqd7WwV3Tx5IUE0zLgeNsbNQclINHT/FsaTXlDfvxs5n48a05lG1poygnjr37O6ne1k7JlFHcdl06o2OCqFA6kHBTs+MgxXlxbGs6ipwYRv3Og6ypaebzqn3kjomirGE/JVOSCfKzUL39AP52bZ7AJMHmXQc42eVm07YOHp2b12NI9kIQhsAHR487jDE83XPXjcGBI6f4orqZqVkjiQrzJzslgmfermJV1T6SRwaxvr4NP6vEph0HkICf3jEBh9PNv/91EyZJ4p/vyOXzyiY2bGmhIEMbK0yNCzU8OJ3eStR76GNdXRMOJ9z6nRTeWNnIVzVNrKpqIif19HjxL/9UTk5qpKFwn35L80YzksN5trSaNbXNpMaF8Ju/VBjdab33kpMaSXpKpDGENTk9mnV1rdx6dRpxkQG8+ZHC5PRoxiSGUpQdxxebmslJi2BxaRWfVezl0417tGseI5QSG8Kxk928sKyGNbXNrK1rQUKiMDuWCWmR3FKYYlToccnhRIT682nFHtbXtzE1ayRpCaG8uKxWa8jLarn7epmbrhwFgJ+fjUC71otaXFpFcV4Cu1qP8vyyGh6bl8cMj7HtPKGN4afEhvDc0mqmeZ575u0q1tY1U96gDXG89kEDt1+Tyn+/W8/6+jYsZhPfv17mfz7fhtvlZm1tKxsb2/G3mfn+9TJJMcE8+XoZSTFBVG1t57arU/mm5QhF2XH8dmm14Vn/4rUNfFXT7BkK0bzD8ob9jIoNZn19G2azBMYY8X4emZ1NckwIq6uaSIoJYkSIH1Fh/lRv68Dl1pa9lDfs58uaJtbXt/GDGTLr69t48JZM6r7pIDYikDdWNrJwdiYjgv0M41W9rYMFnl5G5wltvgDg/hvHUb/zAPeWjKNSaeeeG9LZ2NiOxSwhSTCzMIWpWSPJGxvNV7Wt2MxQMiWZtz7eyrS8BKOn9pPbcwgLsvHmRwpr61sYP3oEdTs6uCE/ibc+3oqy9xALbh5P3tgokqKDeeeLbzBLWo/gWs+w35c1Wh5Vb+ug26kSHmzjnS928JPbchgdG8KfP9mKyw3dTjflDfu5bmI8b3+23VD2S1bU4XKpVG1t54GbxnPTlaMYPzrcmEiPCPVn3KhwvtjUQvW2Dm68IonSVdtIHhlMzfYDTEiL5IvqZnY0H0FCxWKGLbsP8VH5Hr6qaeG7VyThdLl55f0tmCQVm8XMj2/NYU9bJ9XbD7C2rpkNm/cjAZkpETicKr9mTwpvAAAgAElEQVRdWsX8knQ2bTuAWVLZ1qR9+9K45yA35CdSt/MQ3S6Vqq37MUkSUWF+/PmTrRRlxfDxxiZUt5tul8ptV6dSv/Mgd08fS/hZtgPpL8IQ+KDjyEk+2bhP6xpbzeSkRfLb/6kmNiKAV99vwCSpXJUTb3zg8VnlHlRVpXpbBw/eMp7bp401usaZo0fw4rJanC6VBTePY0L6SD76eicSEmMTw1hX18a6ulbSk8KwWcw9PGnA8GQB7FYzq6v3YjaZefCW8WSnRpKTFkHBuBjKtrSypraFTzfu4auaZk453ORnRLOxsZ3CrFij97Fl10Hqdx7k1u+k8KePGjnlcJOTFsGBI6d46q1KIkLsvPz+ZhIig4gdEWAMr0xOj+bg0VMsWV7PXdeN5aV361hf34acGMbWvYfJz4hhWl4Cq6v24XDC3dPH8sr7W4gZ4c+Ly2pZV9eK2QSqCj+6VWvQf3ivni+qm0lLCCHQz8qil9fzxSbNE9THifMzovnvFfU43SrX5MZTvqWVyq0dbNjcwicV+/jo628oyo7D0e1iXV0rkaF+LFleb0yG2q1mHn9Vm4h+eFYmv19Rz+3XpBLgZ/FMPErMLBxN455D1Gw/wO3XpPHGykYkVKxmkCSJ+p0HQYVT3W5MkorVYmJabjzLvvjGMFRLltfjdKnU7zzIT2/XjP+XNS2UFCQyYUwUq6uacLmcSEjcUJDE5l2HcbpUarZriv3BW8ZTs/0AG7a08sjsbP7zf2sYP3qEZyikntXVe6lQOrCYwK3CnOI0rs6NY0ZBMmMSQokJD/AMwUSSNzaKN1Y2Mqc4jdGxITz5ejlbdh3E5VaZU5zGX1dvZ1xyOP/+101oH46qbNqmedVvfaRw63dSmChHExcZwLzrZMaNGkFwgI2oMH8OHDmFnBhGxdYOGvccxuVyk52qtZPyhjbCgmysWLMLSXUz99qxvPzeFpwulcY9h5lTnMbsq1I5fqqbJcvrqd95EBPa4eROl0pOWgR72jqp33kQiwR2u5npkxJYsWYXP7xpHCOC/Xj5/c2YJbjtGk0hlhQksrJsLzaLCYsZanYc5Japo9i0/QBOl2ZsdSNwzw0y//1uPZ9X7yMlNoTqbR1YzbB512G6ndqzd00fS1p8GKur9zAtNwFl71EsZompmSPZ3tRJQUYUn1U2GwZod1snLrfK+roWar/Rjld3exYludywrq6VVRV76Xa6qd66ny6nitkk4fI8o7pVtuw+jM0CN08dxX3fHU/yyCBefm8LZgl2tR3HYgKn5/mrc+PIz4jmrY8a+aRi35BPFg/rL4v1sVgz4AKsZvAs0mFOcRpLV23Hz2bmlMPF/JJ0/mfVNpwuNw6nSoCfhUdmZwHwX3+rwSSZcLlduN1gNpv4xfwpOB3d/OYvlVojVFXmXiezdNU2QAVV4ke3ZvO75bVISDw2L4/f/KUSgPu+m8GS5doSt4WzM3nt/c1IJsnzhaKE6nbT5TkYdOHsTGLCAwD4zV8que+7Gbz+wRZOOtzYzGCxWuh2OOl2a8/q9wDsVhNd3W7ml6SzbPUO3J66oKpuTna58bebcbtdgITVYuH2a1J5Y2Uj80vSeWNlI4CWL59vw+1ScatuHM7Teafn56yiUaxYs8t4/o2VjcY9/W/vZ7x/zylO45TDyYo1u1g4O5M3P1K47erUHvGPjg2h7dAJ/riykcfm5XHsZDcvLtuEw6kyvySdtz9VKJmSzIo1u7BbTZhMEm63m+K8BE25mLUyU1H58a0TDAVms5hwON1YTPD9G9Ip/UzhlEM1lNKjc3MJ8rfyyz+W43RrsoQE+fEfyzb1qGdmCcwmcLi0/WtAm5zUwwEoKUjkk/K93DYtjb99uQOHU4tn9aYmTna5sVuhqxv8bCbmXjvWMABaHZWYe61s5AmAv10bPnS5XDicWllLqJzqVnsoHP23Lou/3cQN+UmsWLMLm1mT2Rt/u5n7vpvBH1c2cvyUEz+bybMMUmVabrxHWUtafrrduNwqqno6PgCLGZxe4RbnxbGqqhmLBA/OyjSU+Wvvb8atgsOp4meTmJarlZe/zYyKyimH26dseh03mzAUsZ7OOcVpHDnWxcqyvUb++ULPD4sETrWnzEVZMaypazPK1uXRRj+8eTwNOzuMezYLOJw9w/XO7zV1reSnR7GqqhmrCSRJM/56Xnm3o98+VDgohkAcTNOLo8cdPPHqBo6f0koqa3Q4jbsP0e0phAC7mW6nC0mScDi16PSCtZolbr061ahEflaJH982gbodHUbDBk3xvvGPBk50aTVIV366grGawWwyIZngR9/LMdY026wmHB5BrCbodoPVItHtVDGbtB7D1RPi+LRiL7derVUWvTEBhvFaODuTE6ecvLGykeK8OJJHhniMmaYcTjd+Mz/6XjbHT3Vz4pSTAD8Lb/yjgVsKR7N01Xbml6TTeuA4X9Q0M3FMJGMSww2l42eVmJoVy6qqZqNRzC9J5y+fNOJwao28vLGdayfGs2LNLvztZmYWjsbfbjHk0uUuKUgkJT6UV9/fTFe32uOe1Qwmk8QDN43nD+/WG43FbsEwijYL3DU9vYdC9G6MVjNYLWYjXd7oz/nbzcwpHsMbKxt9KkLQGujyr3Yw7zqZ0k+3cqq7p0KymLR47rsxg2+ajvSoE3arxD/fNsEoa6tFwuVU6RnCafQ6oysFm0XiyXsms2XXQfztFko/24bb5cLhwnBo9LzUFhs0n5bLS5ldMS6ar7ec+QGZt5HwhcUEJhMUZMRQsbWDK8fHaIrMI6dep7ydKotZwulSDaUKWnur23nI+N+b4rw4vqppNt6H023Cz2piWl48oUH2M8vQasJsknA6teGV3hRkRFG9rcMwsp9X7eNU99lVyZziNADeXbOTLocLN9qRjCr0MDTQM//NgMkrDwwZLRJOT3l7G5S+0HtS3luFXAjCEPhgbW0zb6xsJCMplIY9R3rc81VI3ooJNIX35oeNuNSeDUh/Tld63hXWV0Pzs5nISY2grKHdiLc4T1uWphuh86k0vWW3W7UNvCanRxvvmoBbvDzuW69O5f1136Cqag/vxWY5bQD7wtsb6o2fVWKax+P29viQtPRr6WvF0SszdCPZZ5y9Gp+ucLxl8ZZdb7T6dadT5Y7i0163YWDRGrG3Ee5Nf8rAbpGYN12m9cDxHoYANIVU1tDe49qY+GBjPFknb0wEVdu0vWu86563wjlfziW7L4V8vujKSo9nw5Y2zCaJbpfao6zO9t7okYHsbD1+xjPe5QeaE+ZLyYOWL9D/vDkjnLPU67PdG0qeui9/UE6TE4agF/oS0YFgNku4XKrRuHw1MptFe+Z8Ko02luj7QW8jJQFxEf40HTg5ILnPhbfi6U3vBumNd4P2/t0XutKzmPAMpWkeY+/wB6qcziarztnyfDAYKoWRFBXAnvYTA37/QhT+YMmg48v4nQuTZ/hkuDGnOI3rJiddcDiDbgi8DrDPAbqA+xVF2e51/wFgAeAEnlYU5QNZliOBUsAfaAbuVRTlhK9nzxb3YPUIlvytxlB83h6J3pU9X85H+QmGlpHhfrQeOtWvd+KH0KgOZdiCbxfn43hcKENtCC5kr6GZgJ+iKFcAjwHP6zdkWR4J/AiYClwPPCPLsh34BVCqKEoRUA0sOMuzQ0pT+7Ee3q93t7Q/RgCEEbgQfNbKAdBfIwAMqaIWRmD48H+hg3IhhqAQ+AhAUZQNwCSve/nAOkVRuhRFOQJsB7K93wE+BK49y7NDSnxUECUFiUMdzTkJDbCc13NRoUNuGy8J/xcakeDyYRA+zr0kJMUED2n456eFfBMCeM+wumRZtiiK4vRxrxMI7XXd1zXv630SHh6AxXJhJXq4s4tVVXvP/eAQc+SE89wPAe1Hzr2ttUAgODu+VoFdDnQ5VaKihs4YXIghOAp4S2byGAFf94KBw17XT/q41vvZPjl06MInqo4ed/BtmSfXl4gKBAKBrzkHu0Xq1wl9fdGXMbmQoaF1QAmALMtTgDqve+VAkSzLfrIshwIZQL33O8AMYM1Znh1yvi2GQBgBgUCg40st1e3oGNI4L8QQrABOybK8HngR+Iksyz+VZflmRVFagf9CU/SfA08oinIKeBq4U5bldcAVwJKzPDvkXOjw0oUQ6CfOBLpUmE2DNUUtuJSMie/p3Q6kOcdHnHszN/MlrC56GrNSI4c0nmH7HQHApxv39PmJ+aViqNe3ezMYa8r7w8VYZnchRATbONB5fifVnS0tfX0c1V/0D8cG+5sEP6uJaycn8sH63YMXaD/pTz3vvTy7r/zQv6g33uvjmwNfZef9kZ8eX3/yX39HD8d7+XDv7yW8h4KD/c10nvQ9caF/12OS4IWFQ7vFxLB2S8eNGnHGNXMfOZI1+vTn3UO5gmcojEBvz0lP4uZdvo2ARdK+ZD0bFk916s8qDIv5tCwZSafXA/TO8nPFDVoj98ZmkTChlZPFRxlmJIWeUbZ6ELrH13myGzxh2Cxnthdvz9Bs8my34HW/ICOK6yYlnGEELCbt63HQvrw10uCR2xclBYlYrNo7LvXcy2zNXumR0JSNjs2TIVZPAlTgk3LNCPSO3lfe6bKCtg+ULvOc4jSsXuWvh++dRjizrAoyorBbTT5X7c0pTjsjT7yNgNUM5j7q3Kyi0T3f82pK3u1XRctf76Ru2n7AkOeO4jTml6Tzb/flY7dIxs4BvdOrYzadlrGsoR2rqefy4W1NnT3Kz2SWjHA6T7qMNmE1a1/k6zTuPUJxXhxuFeMYzaFiWBsCX/T1SXzdzkNGo9FX8JytceoFLQ1Ct1KvoLOKRmEx9WxoZ2t0egXr/fWm3pD0hjKraFSPtDhVWFff2qOhZo0O76FcTmsd7YfV1FNR2iyakssbE2Fc63adlsV7S4/eWb7as5WCxaQZBYtZ2/vJG+9GbjVrG5O50cqp9w4Vkic+l7unkbF4CvSeGenGHj7FeXHU7TzEd69I7hFGb6/Q6da+inZ74jdLkBIXylc1++iNSdK2f7ZbJRKiTxtlN/SIx2qW8LOZmF+SzsqyvXR1u5lVNAqruacHazZpZQbaJCLA9QWJxjMqcN3kROOe6tkm02TSnujqdnv2zAKLp6LazFpPwWoxGWHreQtgtWq/PyzbbWzfccrhpNul7Z+0cHYmt16dCkBZw+mv7K0WySgrs6QZrLKGdk50uUiJD8VuNWHzWrJyyuHE4VQNxW01S9g99/U9nEwmsyGXd63425c7tDyxmjSlajMZz+k9X90B+WTj3h71rqvbzefVTdit2pkAS1dt49jJbro8aXW6Yemq7XS7tHzQ86ikILGHzvCzSiyYmQmc3qfIYgI/u5k5xWk8OjcXs9lkNB+rSdvCGrRe0o9vm2DkxyOzs7hpagqPzs0dlO0lzsawNgTHPB4gcM5vCqxmMHvcJV2ZzPRUBptFMpSkXoElSbugqqeVs1nSGpyuvH1lfm9v2GqGLzY1M6c4jRVrduF0w5jEcOaXpAOwpq4Nq0lrjEVZMT0U5LamTmyehu3t6S24JdOopAB/X7Pr9H48nkquqiory/Zit0rMKhql7aPuqfAm4EFPZdeHFk1mEy71dD7eNT2du6bLPT7am1OchsWMT69Kx89mxubxnqdPTtT2KDKZ+OEtWT7zp6Qg0bPHvya7zSIxpzitR/r0tFlMMCk9BrunsMwmTYGVfroVh1PbI35VVTMlBYl8WLYHP5tkNPh7ZqQTYDczvyTd6AW50fYm6nZpRmLpqu10dZ/2jHUcLnjlPW0jvXfXfkNBRhSgbZg3NlFXeNDtUpEkGB0bYpTvhxv2GBuX6WmXwLOTKvzz7ROYX5LOFzXN2CwS80vSsZhgZdleHrh5PHariW7Prps//35+DyX/0ztyuf/GDE/5aecRnHS4CQ/2M55xe+JWVS2dTo9iLM6LM/arkiR44x8N/H3tTuxWiSfvyTfqQbdTZeHsTE1hqvouv1r+vPbBFv75thzump5uxLdygxZm3c5DWM2w4JbxdDm1+m21mpn/3QxOOVxYzGbsFi08vbx1AzXvurH84gf5/PjWHI+h0jYTnF+STsMezcv+lztzDUMJWp1fvOBKfv79SfjbTDx+10SC/K3YrRjGWC8nSZJYsWYXc4rTjL20QNt91GQ2EehnJcDPQkSolo+a0+Bm6artBPlbuXPaGByenXd/emeukY8/vm0CQf5WHE5tw0qAJ17bwO+W1xnb1Q8Vw/o8gshQf0yStod6ZsoIGvccxmyWsFskvn9DOiOCbexs0TzYWwq105JuvyaNjFEjqFL2801LJybJzZP35DMxPZqNDW20HDxJSUEiezuOI6luXKpWeXRv0qXCnv3a0IGK5kFYzCZMJpV7bkjnw7I9TJ+UwLamo4A2nHKqW2X7vsM43SqzikZx3eQkwoLsfFXThCSpdLu0nTl3th4DNEWtKz+TpLJ512HDQNgtEiVTRvHq+5u14Q2zGadb88A6T3ZjkkzU7TyEy7OtcvHERJZ+vp2rskcaefHw7EwOHDlF/c6DRrj/NHM8tTs62NXWidOlndkw+6oUxo8OJz8jmszRIzh20sGu1k7MJhN3Xy/TsPsgEip3TEsjPyOaxt2HeOLuiSRGB7FlZweNe7U8MJlUNm3XDhb5srYF1e3GrZ7e6dJsMuFya/s6mU3aXvXbmw5jNUvcdGUyO5q09NutJsoa2nC7tSG4718vk5E8glVV+3C6VHa3dfL962WWffGNtlum2cQd08aQnxHNax9sRjKZuOnKUayrbzG+Pv+nmeOp+6aD718v07D7EPO/O46slBHGHvg2i4nbrknl1qvTjJOyPvh6D7OKRnFH8VhsVjNf12ubC9rM8MBN43np3Xomp0dT3rCfB2eOp3aHtuf+/JIMRscGU7m1g/kl6WzZdYiyhjY2NrZjQusVNe4+iMOpGmcUrKltwmwysWn7AdbVNbF512FtHyw3FGbF8oe/b8bpUnnwlkzyxkZR3rCfht0HePCWTDJHj2DWVSmsr2/mu1ck07jnMHardoDPjVeOJt1zjGVidDB1Ow8CKk/cPZm2Qyd4+zNt7k0/n/pPHymeIzIP8ZPbc6hU9nPK4aaisY0KpYOFszPJz4imZsdBfnp7DoVZsVRu0w66KW9s477vjqNmxwEKs2IpzIqlYFwMFYp2+M6y1Tto3KMdhJOfEc3L722hrEE7I6S8YT//emcuM4tSCPCzsLq6iduvSeOld+t55HvZbGxo5fZr0pgxZZRx8t/qTU0UZsXym79Ucsqh6Qe9/qCqODy9oB/MGEdcZADLv9rJ/JJ0UhNH8NWmJioa21k4O4uslEgSowO5JjeeWVelGueevPzeZhbcPI6CcSOJDPUnMTqQ+p0HKc5LIDjAxtSskQT5W3nqrUoW3DyOmu0HmJaXII6q7M1gnlD2h3frcLphV+tRbroymS27DmM2Q/03Bw1lbAIa9hzG5Yb6nQdZV9eKxYynch4kd0wkS5bX4XJpCmpb01Eenp1N1bZ2z7a9ZlBV3KrWlVXRPKqdLZ1Mn5zIzpZOTGYTNdva6epW2dZ0FD+riQdnjueqnHgKs2Ipb9yPzWLi9mvGGIfdp8aHUrX1AN/7Tgo1Ow5is8DVE+L4pqWTOcVp1O88iNUicfs1aWSljDD+lpPCWV/fhknCUBqrqpqxWkzcd2MGtTu0g3fe/nQbZQ2tuN3QfOAEN16RxO62o1Q0tBnx3X6NFs81ufHMKkolOsyfqVkjtcNEpiTT1e3ixWW1VG/TDji5bmICu9qOkZ8RTfW2Drq6tUNe6r/poMupMjo2mDdWNuJya56YZpy1ve3rdx7muonxKPuOMqc4jY837sNikkBS+Zc7cskdE0ndN4e4a/pY6nceZM61Y1mxZqfhKX7/epnaHdo2xLOKRvHB17uZlpfAqJHBlDfsx2rWdkzd5Dnda0ZBMs8trWbcqHDW1+/nbs8xhgUZMUzLi2daXjyxEYF8XtXETVeOYm1tM2Vb9lO9rQO7Vdti/FS3mx3NRyiZMor8jBhe/WALd103lvfX7zZOMjOZJG79Tio1Ow5Ss72dO4vH8NbHCk6XSmpcCA27D2C1mMgbG0VMeADT8uJJiglmdbV2bvPo2GAa9hxmwc3juP2aMaQlhPDGyka+3tzMKYeKzWLCapYwmUw8eMt4SqaM4nvFYwm0mSnIiGHDllZuv2YMI0L8+GLTXiRM1Ow4wE1XjiI4wMaqyn3U7zyE3SJhNpu56cpRxrGZ8ZGBxoHvkmSiKDuOl96t53tXpbCj+QjzrpMBmJwezcvvaceB3pCfxFU5cWzY0gqA1ay998r7W5CAWwpTiI8KIic1gvioIK6elMTzS6v5gecI1crGNjYq7YDKbVePISM5nI0N+9m0vYPCrFjW1bUax39azBIzi1KMI2gnyVGMjgs1zhb/rLKJb1qOMnKEP7ERgXQcOckMz5GcRdlxpCWEUL/zAAtuHs9VOXF8VavJ/K93audKvPrBFn5yWw4Ai/9cyU9vz2FaXjwvvFNDapx2aH15Yxv56dE8v6yGNbVNnOhycU1uPDaLdlrg71fUc29JOr9fUc9nlXspyo4jPiqISXIUclI42Z4TAAcDYQh80HHkJGvrWrnpymTumDaGUw4X1ds6uHnqKOp3HsLimRB0qponff3kRPa0HcXl1oxD9fYOUKG8sQ23W8ViMRuec5XSCqrmqbtVt9G9t1okLCbJGCvf1nQUp1tl+qQEGvcepaQgkX0dx7n/xnHEhAfw1FuV5I6JZJPnAOwXltUYR1U+t7Sam65M5r11u+h2ujFJsL25Ez+bievzk6hU9huK9urcOENZj4oNISLEzuZdh7BZTUyfnMS6ulYevGU8r/+jgVMONzddOcpoEBsb2/np7Tkkjwwhd0wUX9W2Mqc4jVuvTuPYyW6qt3UQHeaH0+Xm5fe2ULejg8c9B4AvLq1iwc3juCY3nqToIPLGRrOmVjt28V/umEBhVixyYhiVWzXlWTwxkXV1rfjZTNx/43jiIgOoUNqxW82YTLC77Rjfv15mohzNZ5V7ue/GDNbX7yd3TCRvfqTw8KxM/riykRNdLuO84fkl6UzNGsmfP1aQ0DYUnHfdWOM0t+eXbeLu6WPZvOsgGxv289i8POSkcONM6YOdpyhv2E/j7oOsrm5ibX0LMwqScXS7sFvNfF61j0LPOcxmE9gsZu6/cTwzCpIp29KKJEnkZ0QbJ8C98v4WHp6VSVlDK5JkYtG8iUwYE0VEiJ0KpcM4AS9z9AjjuEazBF/WtBjnJEeF+TPVcy70C8tqWHDzOOPc4xEhfuSkRTBl3EjW1bXyL3dMoGDcSDYq+7khP4nFpVVMGR/Lr/5Yzg0FyRRkxBAV5o/dZmZCWhTrNrdy74x0liyvo3hiAtdOSiQnLYJZV6UyblQ4zy+rYW1dC27VTcPuw/zw5nGUTBnFuvpWCrNiyUmL5C+fbmPRvDzjPPAbCpLJz4hmw5ZWSqaMIirMn7T4UCoa21FRtV7BrCym5SUYZ28//VYFUzNjGZ0YTurIIOSkcBKjA1m/eT8/vT2H4rwE7FYzz3mMxG1XpxEfFUR6UhhZKZFaz8GTNh39fHBvwzA2UTt5LiLEzgvLapiaNdJwtmIjAinMimOU5yxtvRcUEerP029V8LM5uTicbl5cVssT90xmTII2B/F55T7KPHVpRkGy59TBJhbNm8j40eG8/N4WPqvcy9paLR9rdxzk4VmZlDfsZ11dC4VZsWfkw1D2CIbtHMHR4w4Wv13FyS4XK9bs4td/rtBOzjLBxxv3UVKQqE06Slr31mSWWFm2l5IrRhlhnHK4UdG67CaTiTunaZ51UVYMDqf2WbjL7cZkMmO3akMZDqeKyWTCbpXw8xTsrKJRrKlrZX5JOl/VthonlB072c3C2Zpi08d6jp9ysvjtKjpPOLjNczjOHdPS8LNpY7h+VhP33ziOJSvqeOLuSSycnWmcAhbkb6XzhIOm9mOUfqpwyuHiJ3fkEeRvxd9u9px0puJvNxMcYGN702HjRLIgf6uxbfdT9+UzbtQI7Yzn1EjjRLFAP6s2vnr3pNPzLyq8/o8t/Of/1hjvPH7XJON0r5ferWd0bAgLZ2ditViJjQjkqfvyWbzgSkICbWSnRuJvM/HcI1fxwE3jOdnl4q+fb6flwHFOdrkI9LPiZzPxxj8a+Nkc7fCOXz8whUfn5hLgZ+bRublkp0byuueAoPtuzODRubkEB9iM5Xhut8pfV2/nkdnZIEFwwOllep0nHCxZXq+N2UvaSXK/vn+Kca50y4HjIGnzTYvmTcRqsTDfq/wW3TWRRfPyeObtSn7+WpknSzzzKpKJRfPyjPiyUyN56r58Hp2byxsrG3v8/fN7JhPoZ+kxcRgSaCM+Koin7ssnJjwACYnOEw4ef/Vrfr+intiIQB6dm0tsRCAvvVvPwllZWlwq/ObNcn42R/Nqn1tabYxBx0cFsXBWFm94TiDrPKEpzdiIQACWrKhDVVUemZ1llGPeWM3IPTY3j8WlVVrv2DO5EBJo46n7CgAtX02eCbOjxx289G49j83L45kfXsmjc3NZsqKOZ0urDUX91H0FhATaONzZZcgYEx6Av91MkL+V55Zqh/vcc4PMkuX1dJ5wGOHq6fFOmx7vk6+X9biWNzaaR+fmUpgd12Pff/0ZvZ7oYet156n7CggOsPHSu/U8OjeXKdlxhuyPzctD8tQl3eg8fb/2/J8+VFBVtzZhLMHjd03i6fsLkJPCWTQvr8cqFO98GEqGbY/AbjNTmBVLWkIIdd908ONbc6hS2rWzRiXY0XREWzZmlpgyfiQzCpLZsKWVnS3HuGv6WBr2HMYsaev6qrcdAFQadh/me99J4R8btD2MbBYJi8XCnOIxVCgdbGs6ir/NzI9uzaZiazuP36V5ByvW7OShmdqRk2tqW7jt6jRy0iJYsryOqq3tqIDb7WLWVdJ5NNQAAB0YSURBVGlMzRpJYXYsi0urqNuhebzTJycyqyiVtIQQNu3oYNO2A7hVlbGJYSxZXk/D7oOc7HLx9eZWPq3Yx4bNrZzyHFE5OTOORX9Yj6qqFGXHsa6ulUV3TcTR7eLZ0mrml6Tzzhc7yEmLpGrrfu1s5OxYnltazUMzMwkOsDFqZAhf1TYzszCFgnExHDvZzbOl1aQlhFCYFcvXm9vo6tbOVr556mjsVjPPllaztq6Fh2dl8cI7NdRsP8Cjc3OJCvM3vDHwnCtdsY+inHiW/K2WH940zvD8JslRRIT681nlXk46tLOLdYIDbORnxDA6LpSubhdfVDfzw5vH8fo/tlDesJ/Pq5vISY0wPPpF8yYyOi6UHB/d8K9qm7kmN54vNjUb5yNHhfkbwwypcSE8W1rNtLx45pWMJybUj/SkMJasqGNdXSsT0iKpaGzn4VmZRhwvvlOLipbnT79VQXZKBE+/VWH0Uj7ZuJepWSMJDtCUfUigjcKsWOKjgjh63NHDO+w84eC5pdWGgVtd3cyjc3MNb3laXgKT06N54Z0a8tOjmTElmduuSyfQbsZuM5OdcjrNR487WFxaxaJ5eUzLizfie/L1MqblJZCfHs3auhY2NuznK0/PLiUuhMWlVcwoSKY4L4H8jGjW1rawrr7VyGP9/WKvs7l177+r20VUmD+FWbFMy0swlJ6exojwACakaEu9f/XmRiQkCrNjmZoVi91q5sV3arQD7+NCsdvMTM2MNeqQ/tu73evpVfYc4qm3Ko1yhJ5K/8nXy3p44np4+m/939RMrVy63fAvv1vDVI8337sudXW7tF5mVizXTkyk2PNP743p8fd+bzB6AjpiaMgHXZ5uNUgUZMQwsyiF8GAbyp7DxrIxCdiwZT/5GdHMLEphWl4CMSMCyM+IZl19K6ja0M8Td0+mMCuW11c2YDVL/OyuSRTnxVOUHcvL72uTQzddOYqSK7RxV71b+/yyTTw8K4slK+pYW9sCqBRPTNTGZqv2YpIk7vQYkpy0CEbHhRpKoWBcDPkZ0cRGBHo81zoWzZtIYbY2TlqptOF0aWPj0ycnUqFoXeqZRSlMzRpJZkoEEeEBBNu1MeEJaZGUN+6nMEur7Kurm5g+OZHJ6dGaRzk7m/LG/cwoSGZyejQvvlPLqqp95KdHUzJFU2BPv1XBqJHBNOw+SPW2dsoatEnB+SXppMWHnW6gWbEU5yX8v/bOPciK4t7jn32wKLA8XA8GVFDA80MFEhYFEzAS8cZHkktMVSpKYlIJakoxVnJN0ASM5VWjchNjckVTAVJiJSSVEJKbm0RjBR8FKPhgDWwenatJygsCWbjArqzsLuy5f8z02DvMeezsnGWP8/tUbdXZ6e6Zb/fM/H7dv+6Z4dTMMCaOHc78OWfmjYNu2PYGCy47m3PHjeThXzb3mDjbte8QL/7ln3z+X89lTMNQlqzczPqXd/DU1h1sbN4V3FSzp45hUG01T764g9raar7wsWk8uHYb508ezXPNe4I6h4fhtsNgHU/9kLrAQFi9J4/wnMKpmWE0jBpCe3snJ4/wDNv5k0fz4NptLLpyCg//sjkoN21iQxALttus0bIhC3usaRMagnZrPdTJkpWbmTN1TPC/DVGcmhkWtK01sK5hXf/yDp5r3s35k0dzxmkjafd70NYRWZ56eQeXXzA+qJ9rPL1rz/uw+nPNe1h05ZTgu9vzZrx9rKeb3uCmK6fy4NptXNx4WqCjo+soS1duCa4bIGhPt46u8Rs6dDBHu44G52Lm2aODjsQ8f9/u8kq3rPu79VAnHV1Hg/o+uHZb4EDCRDkRazNcB2FHAAA1tTX8etPfmNd4Gh1dR7lr9Uues/Lzhetp/1ySDgWF0Y/X56H1UCe79h0Ker8/+O1fuMlfFfOLDa95y0Bz8LVrZgRD+CUrNwfhgbsfe4mOru7gm6I7W96kfkgdtYMHcetDG7yh7w9fpsoPK4A3XL1r4Sza2r3w1D3XefuqH1JHW3tnj6Gp3f7V7z/Pvde/N4hxgncDLb56Ovet2Up3d463Oo4GQ9vWQ528uvMAD61rDr5JPKZh6DEXdvuRHDd98+ngw/CfuUxY/YThxo9O4bs/30ZVVRVVwG2fbOTUzDB2trzZQ9+ufYd4+JfNwfB161//GYRSfvr0a8HH2q1Rs/nsDWS/FFfoU3ythzqZeEYDr/3DW4pqy9tvTp9YV01NTQ13XzurRznbU7ZGEgjOj9uOr+48wOonDIuvnt4jLdxWdltUmmXQCXV0He4sWG5ny5vcv6aJ7lw337juvQWH/Ttb3gyuF8vSlVu4+9q3wwXuObHHA3q0t93e1t7J/WuaeOTWeYFOqydHLgjv3HPtBYHmtvbOHtvCx1m6ckswGnEp1I7uuXHDdNZgurozmXpe+dOuyDqGj5Hv3Lj7Laat2Db3fNp9trV38p5zxvDaP/a9fX36dqKYtiit5QoF5XuyONUjAvA8/76Dh5kzdQyN2dFMHjeSk+pP4Ls/384tn5jOrHNOYd6M07h/TVPQi9nwhzc454yT6DzSzbN/2MXnrphMY3Z04M3PnzyaO1Ztof3wEX+CbBdVVLFx+y6ea94TDNvvWv0SOXLMPPsUlv24ifMnez2d2VPH0NF1NLhZzp88mgunjWXwoJqgR3Jx42mcPX5UEJqwvXx7s3R0HeVbP32FmirI5bp59g+7efekBk4e8XYIYHBdDWNPGc45p4/wVidMaGD5L5qD3utNH5vKvMbTgslR25uyvZWOrqPct2Yrty5oDCa27ET0nGljmeav+gCCiVd7kyxduSWY7LQToDYt3BMaXFdDVzd8+aENXOz3toYPrWPi2OFs/WsLX7vmPC6fNf6YntbwoXVMm9DAsh83BZrdsFNH11GWrNjMpu27uf4j5wSjjXCvDzxj+e+rX2TO1GN7iZbWQ50sXr6R9015V2Sv1O3BL7pyCi/9pSU4Xoff4w1j2w0Izrvbyw/3IG3bhvO5OtZv3cGH50zgwMG3gjaxIzQborHn4msrnvdHql6PP1yvwXU1TJvYQP2QOpau3MLvX/5fnm56Ixi1RdUpfG7cEV5UT3xvawdfWb6J8yTTI2wUHiFFnTf3mG5v3D0f9v+ocFC+EJG7TztfNHvaWIadUBukha+VUnv45RgJWDQ0lAfz+n7uX9PEy+afwVC9rb2TLX/ezayzTwlivzb+2dl1lCdf3MGm7bt55X/2cs0Hs8yZ5j1QYofigwfV8MwrO4NlZc++sovPzz+Xj889q0dc9PzJo7nigjMYPKgmKGedzZ2PvshzzbtZdOUUvv2zbWzY9kZwg13ceBpt7Z3c9djLnD56KN/+6bbACdjhrzWUL5p/UlVVTW0NvPSXFuZMHcPeg28FxuOkkUM45Jexx58/Z0IQ0rjMr3dUzLWj6yhPb90ZOInbV21hwpjhPLSumcnjRvLg2m1Mm9BwjAH1VmvtCsJjrnbrINwbsaPrKGNPqec9E04KRmHTJjTwwM+8sN4VF4zvYRjdG8l1QGHszTp76rsCRxgVUmk91Ok57RzHGMPw/j5y0aSeT2BH5LEx5dl+OGrJys08vXVnj3qHy0QZMnd/7jlZv3VHcM7y1XnIkDq+/NCGwMC5Rts6pY6uozy1dScAX/3UjMjQnR2x2Oty3gwvlBiVt9Rz4557t7MSHjHaUcHTW3cG5yvfuXb365Z3jXy+OYVi+7ShvMkTTqbdeRVEOCwVx8DHLZcPdQR5sA90vPLqXubNOD3o9d62oJGGEScGRsrtZZ4nGS5uPJX3nHUyD61r5jzxnhS9fdWW4Ca4aMY4htZ5k3XXf+QcHn3cBBer7cVs2u7Fa+9+7KUgbY4/AfZU0w5uW9DImWNHMG1iA9PPyjB/zpmBIbA6ZNwoJo8byZljRwSGdP3WHbx7ohcDvW1BI9PPynDJjNPZ2LyLc8aP4sG121h8tTcxaye4rPGwBsT22NyJLOjZi7LxaNdJ2Fi6Xas9eFBNcKO6PeJbFzTS6bf1ZbPGB4bH6nB7t+u37uCS88fz1ls94+Fzpo7pEZeO6r1ZzfmwbenGyu35cHt+0yY29HA4+bBzBIVw92sNc9jxRMXKCzkg97frdPONsOwEbFQYxM5L2AnPTc27vSWQeeLZ9lqy5zAqxt3bc+PmP2nkEMK53AnsqAnmUshn+KPyucfNN3IbOnRw5LnPV/dihOeDkkDnCPLgxjltjNm+4MnGZgvFhd34rO0d3fjRKXzvv/7InZ+bGeRz44Xu0rV8MU43nnzfj7Zy6PAR7lo4s0c81eZ3Y5/h/dph610LZwZ1cmPmmUx9j9i72ybhWG34eKXEV6O22zi9neMIx37Dx7Px15aWtqLx077EV/PNARSqc5hMpr5PHxCxx1p89fQe119f9hW1j0I63XmJYnMi4SWWtnzUfE9vz43Nn09rOWLpxepa6DqwOkudeyhFS3g+qK/kmyNItSMIT/bUD6ljyYrNdOe6qamuCdZsF7sA3JO8s+VN7luzlfsXXcgQ510mrvEvZEjDE6rWKMCxE67hfUcZVqvJneB1Df7EMxp63GThNgEih+Rh7WHjUUqbA0UvcJv/4cXzjpmEjcrb2xuwlAngUvZjcY1WqY6x0GRkXw1Avn0Uc1i9mdiMumaTXPveV+daKqVoL9QutlOVZP2Tdnb6Guo8WIO07MdNvLrzADlyVFdVH/PGv8VXTw8udGt47W/3AZX6IXVUUcWIYce+qvr2VVuCY0ad3OFDez48Yv8/NTMsWEseVdaGcmwv0nU60NOQh8MoB9o6etTHbZN7f7SV21e9wM6WNwu2YeuhzmAkVOyidetQygVu84+sL/zq7/B5yLctqoy3cil/PqujN+Q7dnh7VD73/Ns8cYlrREp1Avmu2XwOcCBTSLubB/LXpZR99FZTf5DaEUHY+9tJY/skpnvCl67c0mNpXRVV3Lpgeo9lfeELJNzTtmnhEFAxjXEMUL4yUcsMGxqGceOy9Sy+enqwhPCeay8I8rjLWW2ZqB6PeX1/j2WkSeDWxfa2ettr7+2IoK89sCRGBOG8Sfewwzrj0FtdfalHf40ISiVfXQaazig0NBRBVDw+XwgF3o67u7/zxZLD697Dx80X4rH/J21Y863Xdw1sKeGrsE63Pjd+dAoyblQiRjXcRoNOqOPGZesTaY9CBrovRjcq1JYE+WLufSEJoxU35t9bBqKBjarLQNQZRkNDEbgxzdZDPR/ksiEDm88d+oaH7u7+3PejLF25hSUrN0cOI/NNwrYe6gwe8HFDLX0dVtt30kQZlHDdig1vo+q9+OrpPPzL5iDMUkq4pRBhDSPrByfmBPLpKlbvQnVxQ21JYsNuhcJbx4tSz0UpGvuzHkkcq79CNv1FqkcElqgebPiJzjjD2agRQaHRgJs3PDqJWj2QxERSKb2Y3oY4kpzs7I3OUokbcitlIrEcI4J8ekudoI+ivydgC62CKta2SZ/7coTaIIUjAhE5UUR+LiIbROS3Iv5C+p557hCRF0TkORGZ6W+bJCIb/XKPiEi1k3+SiDTH0dMXXKPl9hS9HrTnBOL2bKMmRKN6nfbYLuGeu31jZbhMuXtSpU56wrGTnEn03stRJo6u3kwkJk0+J2yXAidxXHcRRJLYdjs1Myyv1qQnWEvR807r0feVuKGhG4DtxpgLgceApW6iiDQCFwGzgKuA5X7SA8BSv1wVMN/Pfw3wE+DkmHpi4RqzqAvEDZOEy/WFUpxDOD38npf+uKDztUt/HL83jq7QCpwk6S/jUaozswY2ieMVCmP2FdvRKhTi6k/DrE4gglwu1+u/bDa7LpvNXuD/HpHNZv8YSr85m83e5vzflM1mM9lsdmc2m63yt83PZrPL/d8fzmazQ7PZ7O5Sjt/VdSSXFPtbD5eU55O3/za3v/Vwbn/r4dzVS39TUrkkNZSDQsd163y86O25KbXMQOZ4tbu9tst9DOW4E2lTa4s5ChFZCHwptHkPcND/3QaE3+M6HNjn/G/zVBljcuFyxphf+8cqyXnt399eUr5SaSnyoBLAnZ+bSddhb/icy8G+fW8WfMCp1HhhEjHLvsS8Cz2oZetcSvuUC3vsQu0Z1nk89SYRJy5Hu4evkXw6y912cfZfCbF3qAydmUx95PaioSFjzCpjzBT3D88J2D3WAwdCxVqddDdPd8S2isCNeyf5yHdfQyxxwyH2uIUe1KqUIXSl6CyVpOvTX/NJSuUSd45gE3CF//tyYENE+qUiUi0i44BqY8xeoElE5hYoVxEkfaP2dX9xHUm+SUjlnYVOkCrFiOsIHgHOFZGNwPXAnQAiskxEZhpjXsYz8s8DPwcW+eVuAe4UkeeBOmBtX8S/kyl1kjS82qivx9Se4zsTdQJKIfQ5gjLRl3hhb+YNknwlQhL7KxeVEH8F1VkOKkVrJejUJ4sriN4M5QdamEpRlMpDHcEARQ2yoij9hToCRVGUlKOOQFEUJeWoI1AURUk56giUXqFLSxXlnYc6AqVk9DkDRXlnoo5AKRl9QlVR3pmoI1B6hToBRXnnoY5AURQl5agjUBRFSTnqCBRFUVKOOgJFUZSUo45AURQl5agjUBRFSTnqCBRFUVKOOgJFUZSUo45AURQl5agjUBRFSTm1cQqJyInAD4HRQBvwGWNMSyjPHcCHgCPAF40xL4jIJOBRIAc0A4uMMd0i8h/AHF/P940xK2LWR1EUReklcUcENwDbjTEXAo8BS91EEWkELgJmAVcBy/2kB4ClfrkqYL6IfACYZIx5L54zuFVERsXUpSiKovSSWCMCPIO9zP/9OHB7RPqTxpgc8LqI1IpIBpgBPOuU+yBwC/CKvy0H1ABdhQ4+atQQamtrYkrvPzKZ+uMtoSRUZ7KozuSpFK2VojNMUUcgIguBL4U27wEO+r/bgBGh9OHAPud/m6fKdw7BNmPMYeCwiAwCVuOFht4spGn//vZiso87mUw9LS1tx1tGUVRnsqjO5KkUrZWgM5+jKuoIjDGrgFXuNhFZB9g91gMHQsVanXQ3T3fENvxQ0FrgGWPMvcU0KYqiKMkRd45gE3CF//tyYENE+qUiUi0i44BqY8xeoElE5rrl/Inn9cAPjDF3xdSjKIqixCTuHMEjwGoR2Qh0AgsARGQZsNZfIbQBeB7P2Szyy90CrBCROuDPeKOAm4EJwHUicp2f77PGmL/H1KYoiqL0gqpcLlc81wCjpaVtwIuuhHghqM6kUZ3JUylaK0FnJlNfFbVdHyhTFEVJOeoIFEVRUo46AkVRlJSjjkBRFCXlqCNQFEVJOeoIFEVRUo46AkVRlJSjjkBRFCXlqCNQFEVJOeoIFEVRUo46AkVRlJSjjkBRFCXlqCNQFEVJOeoIFEVRUo46AkVRlJSjjkBRFCXlqCNQFEVJOeoIFEVRUo46AkVRlJQT6+P1InIi8ENgNNAGfMYY0xLKcwfwIeAI8EX/g/aTgEeBHNAMLDLGdIvIPcAl/vabjTEvxKyPoiiK0kvijghuALYbYy4EHgOWuoki0ghcBMwCrgKW+0kPAEv9clXAfBGZDlzg/10FrIipSVEURYlBXEcwB3jC//04Xm8+nP6kMSZnjHkdqBWRDDADeNYtZ4xpAi41xuSA8cCemJoURVGUGBQNDYnIQuBLoc17gIP+7zZgRCh9OLDP+d/mqfINfo9yxpgjfnjoZuALxTSNGjWE2tqaYtmOO5lM/fGWUBKqM1lUZ/JUitZK0RmmqCMwxqwCVrnbRGQdYGtcDxwIFWt10t083RHb7HGWiMh9wGYR2WCMeS2fpv3724vJPu5kMvW0tLQdbxlFUZ3JojqTp1K0VoLOfI4qbmhoE3CF//tyYENE+qUiUi0i44BqY8xeoElE5rrlRORiEbFzCIeBLno6DEVRFKWMxFo1BDwCrBaRjUAnsABARJYBa/0VQhuA5/GczSK/3C3AChGpA/4MrPW3f1xENgE1wHJjzN9j6lIURVF6SVUulyuea4DR0tI24EVXwjARVGfSqM7kqRStlaAzk6mvitquD5QpiqKkHHUEiqIoKUcdgaIoSspRR6AoipJy1BEoiqKkHHUEiqIoKUcdgaIoSspRR6AoipJy1BEoiqKkHHUEiqIoKUcdgaIoSspRR6AoipJy1BEoiqKkHHUEiqIoKUcdgaIoSspRR6AoipJy1BEoiqKkHHUEiqIoKUcdgaIoSsqJ9fF6ETkR+CEwGmgDPmOMaQnluQP4EHAE+KL/QftJwKNADmgGFhljuv38Q4DngNuMMU/Eq46iKIrSW+KOCG4AthtjLgQeA5a6iSLSCFwEzAKuApb7SQ8AS/1yVcB8p9hyPAehKIqi9CNxHcEcwPbaHwcuiUh/0hiTM8a8DtSKSAaYATwbLiciX8YbDfwhph5FURQlJkVDQyKyEPhSaPMe4KD/uw0YEUofDuxz/rd5qowxOXebiMwDzjLGfF5EZpcietSoIdTW1pSS9biSydQfbwkloTqTRXUmT6VorRSdYYo6AmPMKmCVu01E1gG2xvXAgVCxVifdzdMdsW0hMF5EngEmA40istsY80o+Tfv3txeTfdzJZOppaWk73jKKojqTRXUmT6VorQSd+RxV3NDQJuAK//flwIaI9EtFpFpExgHVxpi9QJOIzHXLGWMWGGNmG2Pm4oWbFhdyAoqiKEqyxFo1BDwCrBaRjUAnsABARJYBa/0VQhuA5/GczSK/3C3AChGpA/4MrI1z8Eymviqm7n6lUoaJqjNZVGfyVIrWStEZpiqX04U6iqIoaUYfKFMURUk56ggURVFSjjoCRVGUlKOOQFEUJeWoI1AURUk56ggURVFSTtznCBT69BbW0cAKYBRQA3zaGPPaQNPppC0AvmCMeW+5NPZFp4i8B/hP4CjQgdeee8qgrxp4GHi3f5xrjTGvOunXAZ/3td1tjPm1iJwMrAFOBN4APmuMKeuj8TF1jgN+gGcTqoDrjTFmoOl00t4P/MgYc3o5NcbVKSJD8Z63OhOow7t/Xjhm5wMEHRH0jbhvYV2GdxG/3y8zeYDqxDeyC/GMQ7mJq/M7eDfaXGAdcGuZ9H0UOMF3iLcB33K0vQu4GZgNXArcKyKDga8Da/w6NeEZjHITR+ddwEN+G34DuHeA6kRETsd7OHVQP2iMq/MrQLN/3q8DpJ+0xkIdQd+I+xbW2cBpIvJ74JPAMwNRp4g0APcBXyyzvj7pBK5yXktSCxwutz5jzGbgPCdtJrDJGNNhjDkIvApMo3idBorOW4Df+HnK2YZ90ikiJwDfA27sB32xdeI5hU4R+R1wO/C7ftTbazQ0VCIJv4X1DGC/MeYSEfk6Xg/26wNM50nA/f6+3kpCW5l0jrDDdBF5H3AT8P6k9TrHP+j8f1REao0xRyLSrH53e1SdBoRO/11giIgA38TrBQ84ncBDwDeNMTs9qf1CHJ0nA6OMMZeKyKfx2vTT/SW4t6gjKJGE38K6D/iVv+2/gXsGoM4RwFl4cc4TgHNE5EFjTCKjg4TbExH5BLAE+FB4XiFBwsev9o1BIW12+1tE12mg6EREPoAXC7+m3PMDMXV2AhcCk/y5opNE5CfGmKsGmM6oe/y2MmvsExoa6htx38K60Sn3fuCPA1DnC8aYc/2Y8VXAn5JyAgnr3Csin8IbCcw1xvytP/SJyAXAdiftBeBCETlBREYAZ+N9jrVYnQaETt8JfAe4zBjzUj9ojKPzBWOMGGPm+tfl//WDE4ijs5n+v8f7hI4I+kZf3sK6UkRuwBtWLhigOvubXusUkRrgu8DrwDo/XPCsMeaOMuj7BfAvIvIc3uT5Z0Xk34BXjTG/EpHv4hn6amCJMeawiNzt1+k6YC/lP9dxdT6It7pltd+GxhhT7ontXusss57EdIrIN/Du8eeBLgZwWAj07aOKoiipR0NDiqIoKUcdgaIoSspRR6AoipJy1BEoiqKkHHUEiqIoKUcdgaIoSspRR6AoipJy/h+0A8tN/Ud/pwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(net[2].weight.data.reshape(-1), net[2].weight.grad.reshape(-1), s=1);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the normalization of the loss function feeds directly into these gradients, so needs to be considered when setting the learning rate:" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0: IN (500, 100) OUT (500, 200)\n", "1: IN (500, 200) OUT (500, 200)\n", "2: IN (500, 200) OUT (500, 100)\n", "3: IN (500, 100) OUT (500, 100)\n", "4: IN (500, 100) OUT (500, 10)\n", "4: GRAD (500, 10)\n", "3: GRAD (500, 100)\n", "2: GRAD (500, 100)\n", "1: GRAD (500, 200)\n", "0: GRAD (500, 200)\n" ] } ], "source": [ "Xout = net(Xin)\n", "loss = 100 * torch.mean(Xout ** 2)\n", "loss.backward()" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD3CAYAAADxJYRbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzsvXl8VPW9//88M5PJRkJCSCBhCYSQSTALBAlYQhWircTrAipewF5brG2vor32cW8R0N+3Fje8t9qr1F4VF7SEFhSttdRaNmURQvaEJANhEbInbNkzk5nz++PM53BmmMkOhDKvx4MHk5lzPp/3Zznvz/v9er8/nyPJsowXXnjhhRfXB3RXWwAvvPDCCy+uHLxK3wsvvPDiOoJX6XvhhRdeXEfwKn0vvPDCi+sIXqXvhRdeeHEdwXC1BegODQ3NQz61KDQ0gHPn2q62GD3iWpETrh1ZvXIOLrxyDh7Cw4MkT795Lf0BwmDQX20ReoVrRU64dmT1yjm48Mp5ZeBV+l544YUX1xG8St8LL7zw4jqCV+l74YUXXlxH8Cp9L7zwwovrCF6l74UXXnhxHcGr9L3wwgsvriN4lb4XXnjhxXUEr9L3wgsvvLiO4FX6XnjhhRfXEbxK3wsvvPACaGq1XG0Rrgi8St8LL7y47tHUauGZdw5eF4rfq/S98MKL6x7BgUbWPDyT4EDj1RblssOr9L3wwgsv4LpQ+OBV+l544YUX1xW8St8LL7zw4gpgqMQLvErfCy+88OIyYygFir1K3wsvvPDiMmMoBYq9St8LL7zwoo/oj8U+FBQ+eJW+F1544UWfcL65c8hQNf2BV+l7Mei4Vh8GuDZkH6iM10IbhzJCgnwHRNVc7f73Kn0vBhVDKWDVV1wLsg9UxmuhjdcC+qPwm1otQ6L/JVmWr1rlPaGhoXnoCudAeHgQDQ3NV1uMHnEl5WxqtQyIv7yafdoX2QcqZ3/7qa/3uco50PG5XOhvf17p9vRHTqHs1zw8E7j8/H54eJDk6Tevpe/FoGMoKpTe4krJPhCLb6AyXsvj44qhYDn3Btrsnavd/16l74UXVwFDKYXvWsa11I9DRUZDf24ymUw64A0gBegEfmw2mytcrgkH9gNJZrO5w2Qy+QN/ACKAZuAhs9ncMBDhvfDiWsZQUQKXG5ebfrle+nGw0F9L/x7Az2w23wQ8BfxG+6PJZPo+8CUwSvP1vwPFZrN5DvAB8HQ/6/bCCy+uEVwr9MuVwlDoh/4q/XTgCwCz2XwAuNHldztwK3DW3T3A3xy/e+GFF//EEPTLlcBQUKjdYagsgP2id4Bg4ILmb5vJZDKYzeYuALPZ/A8Ak8nk6Z5mYHhPlYSGBmAw6Psp4pVDeHjQ1RahV7hW5IRrR1avnD3Dx6+T5f+9k3X/NY+QIN9ur+2vnOebO/l/r+/tVR2Dgf7IGR4Ob/wy44rI1x36q/SbAG2rdULh9/KeIOB8T5WcO9fWP+muIP5ZUzavZlrfP2ufXi0MBTmfXZaGtcNCQ8dFK9d1jg1UTnd1XA4MVM7LLR90vyj1l97ZB2QCmEymWUBxX+4B5gN7+lm3F5cZQ8UN9eKfB64GxOWYY701UvpS5z/jM9Bfpf8J0GEymfYDrwJPmkymX5hMpru6uef3wA0mk2kv8BPg2X7W7cUgQOwOdIdrKQ3Oi2sTV2uO9WWx+Wc1frw7cgeIoeA69wZaOZtaLTy9/iAyMs//eNaQU+7XYp8OZXiSc6jtzL1S/dmXdru7trdy9qd/B2tMvDtyvXBCcKCR5348c0gqfC8GjsGwYv/ZrFstggONvW7fQA5V66uXcKU8C6/Sv04xFLaDezH46K3i6I5eGWq0xmDLcSXa1x/66kpRXl6l74UX/0ToS168J+UylGI6lyvgeyXa15/yr0Sfe5W+F178E2KgivJyK5++0CuXQ0EPlQXtasCr9L3oNYaKu+9F93BVlOebO6+yRM7oq/U+FBT0YONqUmhepT9EMdQU7NXmeYdafwx1CEXZ1Gph+X/vHFL9N5Too6uFq9kHXqU/BHG1Faw7XM1JOhT741pBcKCRdf8176op2O72ggxFDOYc624vDFy9PvAq/SGIoWoJXS15hmp/XI9wp8S6S/u8lhbrwZS3qqGFp9cfZPX6A0Ou/V6lP0RxvSi465nXvRIYTHrHnVLsTlEOdLHuTubLoUgHy7hoarXw8qZ8ViyZ1qu9MFd6UfAqfS8GhIFM2KZWy5C0hP6ZMJj0jjul2JOivBybm3pjkfd3Tg2Gwhd9MiZ8WK8U/pX2hrxK34t+YzAmrITH3eL/VLgaC5uoczCP8nWnxC6HF9bdYhIcaOSXi6c5Bau1uFq0ktaI6W2faNtypeT1Kn0v+o2BusPiOIhrnbrp6WG9Gkqot3VeTpkGWraneSHoExEodW1nf+blYPVDd0aMKy0m/n95Uz5VDS1XbI54lb4XA8JAFfY/g8Lv6WG9GoHo3tQ5kMWoLwvdYGfEuLbNXTv7qvDd9UN/5PZkxGjr0Cp4LRV0peaIV+lfJgyU6/bi2kBvFfrVWNx6I1N35+94Qm8Oa9MeBzFYFqy2XkGHPPPOwW6v7+1vrv3Q1wVRK4une0R/vLwp34mecv3/csOr9C8DBmpBXUtpbl5c295Kfw5c6+1hbeJQv8GyYF3L6u6cod4Ggj0tHK519fQ8drfIaevQWvZXC97z9AeIy3FW+UDu9YRr5ex38J7/Ptjoj5wD6euqhpZ+KbX+yvnMOwfdLizdtUH7W09tda2jJzldyxMLwJWcu97z9K8CBjLAQ0mx9RfuglYDLe9a8YCutoyD0ff9mYPCchaByd5c3x/0NmjbXRtc0061gVV312rr+LamqVv53NFEnnA15opX6Q9xXG0F0h+4c58HI5OjPzTB1UjbG6ygoKfye1u/+Ox64FpfZenN9eZT51Tl9svF09Tsmt7I2VdZtPd5sqr7U2Z3GTSijqqGFh7/n11UNbR4pI5c7xtq7y3wKv0hjMs9KS5XsFk70Qeb0+2LfFfjoXLX3sGSozcLiru+1+bp9zdA2V0WTlVDC2uz8nn0nkSCA42MCR/mFKh0h574+J7uCw40Oinp/o53dxk07soZEz6M1/9zLkEBxl7vTu6uH67G8SJepT+EcTlT/S53sNnVfb6S0LrUnrjeywl36YODMY69XVC66/u+ytKbLJygACNrHk7DND5UlcmTpe/6XV+OdRC/CzpGZMGIcqD3StR1kXDNoOlOjujIYLf92Ns0WW35VwPeQO4AcS0H84ZasFmgr33qThZP8nUX+OuvnH2p/3Kgp8Bpb/qzN/J6olK0/alVyq7lVTW08PKm/EusafFZ259w6YLlOnZ9CcZ6KsddPT31iaf+1I6DpznhSf7BhjeQ64Vb9OYh7++9Vwp9dakH23tyV39fvKjBoHzWZl1qVWv/7uklKr2V11OAUqvEXC1n7fWu+emuZWrhaUxdUzZ7Kkdbv/jnSoH11N7ejJFCNWVT1dDiMTbQnfxXEl5Lf4C4li397tAXi3iwLZbBsPSvBAZq6Q+G19HUqpz38vyPZwE4KV9hyf6/d7N5dllaj7SDq1Lvz7h3d19PZWrHva9j6un6qoYWggKMPL3+IHbZDki88EjPJ19qy3UdI9f5KeoWGUticQsK6H5RuZzwWvoOXOko+bWM3lrEVysDQYur7XV4ShfsjeU8UK8jONDIU0tSaW5ztrK1lmxvTtl0TVsU2Sw93dOb73rzm6dre+stuZuDwvpubrPw3I9nsnLpdHRS3w7462mMtHUHBRidFL5rlpGn+680rhulPxSUk5DjSt7XXTl9cee7u8ZdcPF6QXcPc18ok772mVaZvJSVx9os91v7QTlls6fyXamZ3qRd9kVO18+9vdc1O8ddOa5zUPyuZOOkqUccjwkfpp6N0x0d5lpPT56La7aUqE9Le7k7Qryp1cLqt6/80eLXjdK/HJkwA5nEV+I+7f2u5VyuRXCoLK5a9Ebh9Qfnmzt5ev3AXyDS1z5zVdDP/3gWz/3Y89b+882dfT4UrrsDwHqryJtaLWr/9LaN2t+1mUPiLVQ9nUbpWo82sOrpGq1S1no53dXjemiaVmZRpvZ77embwoNqbrPQ2tFFc5tX6V82DLbCd7fxpaf6+7PwDGTB0srpapX0lEvdXZmu5WutoqH0asOelM1AFykZz2Gn3vaBpz7zpFzdBQS7U84hQb79OhSuu+yn3ipy0T+9mRfaxUk7n4IDlSO4n//xLLeLkesiqLWwtb8LRQ2XpnYKpay11j15sOebO90Gpd3JI8oUHoY24BsUYCTQz0BQwJV9VryB3AGgqdXCpAlhvQo69iYw1dczS3ob7HInp7sAVW/KdXdff4N+nuDjZ8TaMXjWT09195Rq5wnh4UEcO3nG44Pfl+/dXffMOwd59J5EIsMC+xzw1Y6Tuznal3a6Xqv9u6c529dxP3n6rMqH92WOuZPxmXcO8svF0xgTPkyV01NbtN+7q8u1Pz2Nuzt5XBfFmjOt6p6G/p5T1BMuSyDXZDLpTCbT/5lMpm9MJtNuk8kU6/L7IyaTKcdkMh0wmUz/4vhuhMlkanRcv9tkMv28v/UPBfT1AezOItJaAINVZndydmdd9vWExcEM9ja1Dt47XQV6eji7eyFHb8p25YA9ldPXMXv0nkTWZuXT3GbxuHvVtXzt/Z7G15W66Imi0QZ1XZVZT7x/bxe4plYLj768g7VZ+QCXKHzXwLJWdk/1i7iE2B/Q3XiIcfQ0Rn31YLWyP73+IKve/obVbx9g1dsHeH1rEXlH6nvVf5cD/bb0TSbTQuAus9n8Q5PJNAtYaTab73b8Nhr4B3Aj4AfsdXyeA9xtNpsf700dQ9XS10783qYXXk1LH/pmPfel3J7uc1VEvSm/J1n7K19vyvPUBnf1CUtfWJTajUcDtfQFtBZqd9Z+dxuftKmlT68/iIzM8gVJmMaHevTcBAQlIeiM7jZX9WdctPWHhQ3jzJkWtV5Xj0LUDahe0LpPikEGJNSXkGvL1JbV3GbxKL9r+d21o61LZuUbez1e6+o9iLqDAow0t1l44cNc2i02li9MJHZMyGWhQi9XymY68AWA2Ww+gKLUBdKAfWazudNsNl8AKoBkYDqQajKZvjKZTFtMJlPkAOq/KugvB9ybge2twtdyhb29XljP/fUMelOHu01Kq9cf4On1B91e54mz7u6drv3t/+6gtdb74vlo4ySCA9aW6akuV3TXFjEnurM0hcWo5ZjdBSIFt7x8QRJvfFpyCQcu7hNWqQhuirN03J1N42qNexpTd20U37m2y10gVVu3iEe98WkJTy1J5flHZqkKX8Dd5itPm8O0i5qn+IhW5tW/3+d0/INrO93JLzyYMeHDePGnN7F8YSIbvjB77JPLClmW+/UvLi5ufVxc3HzN36fi4uIMjs8PxsXFrdX89kFcXNytcXFxd8fFxd3q+G5pXFzcR93VYbV2yYONc00dQ6KMgdS99JltHmXo7vtzTR3y4qf/etnkdy1X1Cn+ab87WX1BXvrMNvlk9YUe2+SuzP7K5OmavvapkF/7m7tyeqpfe89AxsW1zsVP/1UuPtrgsT7R79rvxX0PrP7c6XdP7fLUVlGW+N1TXy1++q/y4qf/eskc0JbT2zYLuKvLnXwCxUcb1P7Qyu5pPFy/d+1Hd/1wsvqC2+fO9e/uZO8HPOrVgdA7rwAHzGbzZsfflWazeazj813A7Waz+VHH358AzwNmoM1sNttMJlMAUGw2myd5qmOw6Z2eXOT+oL87cgdCUXRHHXg620NQEU+vP9jvl5H3ReaqhhZeyspTd4pqKRDx/6P3JPLGpyWX7F70FCD1NH6eaLG+jHdf6BhxquRLj6UTYJA8Uhzd1e8u0NcbWT3JKbJBxLivevsAOklixZJpTJ0SeUkQX1uf62fwTFloP7vrd+HdiXEXZbvboSpoj7VZ+Tz3Y4Xe6Q0F2ZNXJrwST/eKttacaWVtVj7LFyaSGheh0l9i565OknjuxzNpbrM4UWwvPnpx3AVd9tSSVKc6tTuBtb/3NFe7k70vuFz0zj4gE8DB6RdrfssG5phMJj+TyTQcSABKgPXAvY5rMoDcAdTfJ7hzZfty72DL4skVHgj9om1fVUPLJTnkwr0fLPqmu2vXZuUj253T9ZTTGC9SIabxoeoCoIWnvHJ34ycC4OZT5y6RQ3t9d/SCuLa7NmvHZ21WPnbZzvBhvpcEGV2pA08KXzs2glLojcLvbufpqre/Uef54wuTeGyBEgh2TSt2rU/QJaJuT3EZbaBTGxx1lUdCornN+XWJgNv5GBRgVNM6XY+A1v4v2in6zt1mJ+08c5Vf29eCmvndJyUqzSL6bcWSabzwyE288MgsVeGL+SXuXf37fer1gi57KStPrcN1J7BINXVN1mhqtTjNmyv1GsWBKP1PgA6TybQfeBV40mQy/cJkMt1lNptrgdeAPcBOYLXZbO4AngL+3WQy7QZ+BlyR7B3tw9Ibi8/TvYMF1wfcEw/oTrae5BAK7qWsPOxuvLi+xAG6k9ndNdoHa8WSaUi6i8aGOCbAVQ53+dCueeVCobsbvzHhw1i+MJHffVLitm+0gT1X5e1uY5WrUnI9Vlg86CuXTickyFdVBNqH3rV+VzS3WZzy+3sbo/G0MIidpy88cpO64K/Nyue1j4uwy3YutHR2K5uICbibf8J4WPX2N2psxjWG4brorVgy7ZIFQSyU7mRw5ePdPQ+uivSpJake56J2rKsaWtS4kvju5U1KRpRdlokdE+JkKInfxII1JnwYK5ZMU2MgY8KHse6/5ql11ZxpvRhMdhkPbTzG9XvhjWkXgcsR0HWH6yZPv7cK352L3Z3rL3Kgeyq/t79357quXn8ACcnJWu9ONm05fTlauTe0iDsqyfUe86lzThkivXVdtbKaT51TXfANX5jdjo1wyYXSc9c+d5SGoCG013iiv9zRNm/8MgNrh6XX9wFOQUPx8LveK6gB6Nsirb3/xY15rFyaCigBTLvsTEFo55v2f222kOgvsVdAyKOdW4Kicc1cEnClerprj3bcRR8IakV819JuZZi/zyV96JqTHxRgVJ+XxxYo8rvOBe34i3ERlKOYI1pqR7QtLGwYj768g4duN7FuawkrlkxT+8e1j7p7jkTfXek8ff2vfvWrQa9wsNDWZvnVYJXVabXha9R3e42vUc/sxMhLBsjdfWKiZcwYz/kL7TzzzkFmJ0a6rUc82LOTIj3K4GvUO02U2YnO1/oa9aQnRTIvdewlysf1WnG9KLPTamNESABtmu3e3d3babU51eNJ3uSYMHx99Gpd2r6ramhhzQe5xI8PYUz4MGYnRhIe4u+xPK1cWllHDvcnfnwISTEjPY7N7KRIMqaPc+qXTqvNqX2uMvoa9aRMCnOSqfFCO3uKqsmYPs5tf4qygwONzE6MJGpUEG1tFlUGTxyzkMFV4Yu+3p5zWq1TWLRfFVayO7+62znjrp5Oq43nPshxUvD33xZPYnQIL2/KV6955p2DJMeE8dwHOSTHKP3Q1GpR/37ugxzmpY5lRnwEv/2oSJ0PYi7vyKskZVIYazbkMH9mtPp7VUOLei/Arrwq9bee2hEY6Etbm0X1VLbnVrKvpIZ0Rx8IDv5AaS3LFybx24+K1PFMjgnj5U35JMeE8dLGPNISIpg/M9pJflG/mAvpjjET7f7l4mlMjBrO7MRImtssPPt+Nl8equRGUzjhIf5q2+6YHUN0eCBv/6WMB78XR2pcBI0X2nn2/Wx25lWRMuli/3X3HPka9ZcYKT31UW8RGOj7rKffrgtLvzvLtTceQHflai19cL+yCyvd1Y31JCP0vKGoJ0tfW6+ExO9XZHDmTMslVqe7QFxvA4ruPA8tzKfOse6T4kusaU+cMeBkQbtrb0/fiTJcNzO5Xuua1y6sY8Dj0buufdPdjlxXOT15O0LJa+mAwbD0hcX78qZ8Xnw0na5Oq1N54nrzqXOqZevO8nct21W+1esPqAuMaONDt5vUwGh3bXClnESevjbw6yrzqre+YeWD0y/ZXSvKEhx8gJ9BHUd3bXFtkztPTnD/rt7EhHEj+NlL27HZ7XRY7KxYMo11nxQrMSxJObrZ1YJ358m7fjeYSSbXvaXvyYIXA5kcE9ZtRwvOTVjOWotBWCfurF1t/eluLEF3MkLPCt/Vgu2uzPSkSGbERxAy3J//XLdHvU9r1YlrqxpaCA/xVy2/7uDO83CVMyjAqFp6ovznPshx8i60lmOGwzIKCDBy/kK7KqerR9Ldd1rrqtNqY/XbB9iZX6lai+JaYdkJC3fNhhxkWWblg9M9tt11fK12+M91e9zOH1eeVnhGrotscKCRG03hTt8Lq7g771TMQ/G/uFY7rmuz8nlsQSIvfZij9q8or9Nqo9Nq48WNeTy1JBVfH6UMIa+2bq3H2Gm1sWZDjtrHKZMuWti+PnoSokN5dXMR8eMVz2JGfIRba1Yo8B15lezMq2JH3mn+tv+kOn7hIf6X9IGvUU96cpQ6Pu7GPjzEnxtN4WTOir6kLeK6mMhg1evR9of287zUsfj66J3qmp0Yia+PnoAAI5/vO86qB6czL3UME6OGk54USVrCKDJnRQM4zXMx75Njwvj1hkPszK1kUlQwL2Xlqd7SjPgIbp8ZPSgKH7q39K8LpQ/OE0Q7iYRb6KpAxGdhie0pqlYHKHnSxYc8MNCX2oYWpzK7q98TuqN2XMtxXVi6cwuFu5+eMoZbkiOdHoQZ8RGszcpnR54yCQUd89uPinpcCIUs7urVuvjCzRdyCEXr1J6kSNLiIwgP8afTauM/1+1hR16lulBq26ulV7TyuVI3os6d+ZU8tSRVrVPcnxwTRlCAUW1D8qQw9hTVkDlrglObmlotNF5od1LeAmGhAUwaPcyJNtFSNHuKqtXFxnzq3CX9KhSpr4/eqQ7xm6AEhbJ1VXJCeSXHhPHs+4ec+svXR8/23NPckx7D/bfFMyEiUG2vuD8hOpTd+dWkJUTw4sY8duSdJj0p6hJqTHh1u/KqmBEfwd7iGtIcyjw40EhMZDC/2VzArvxq5qWOIS0hAtP4UKdnS7RBlNvcZuHLQ5X8YtFU5qWOJXPWBO6/LR4f3aXKXNu32gVIfBZtFuOjnQPaMsS4/8+f8lm+MNkpviGuabzQznMfKEr41xuUPhV1CfnvmB3DTQnKfBVj1nihnZc35auLoajPldIbPcKf/KMNHCqrRwbmJEeyp6ia/SV1ThTUQOFV+g64sxBdJ4zrNcISmz8r2vEgOfOMjU2drHxzv5Ml2Rd5tA9DTxyggKtS6m6hEAvbc+8fYu60MU5tvN3BxWakjmVM+DBuNIUzMWq424XQnezuYheuyt21bVrlq1XOWm75rpsnc1NCxCWKVqsI3fWROx4+PSlSjTtoleWrW4rUhcXXqFeV0Oyk0ZfQCYLX1X4vvDw9kBwThsXRBqFw9xRV89TSVJULXvNBLj+5cwpvfFqiKrHV6w+wI7eS7Tmn+TLHuY5Oq40deZWkxUfwq/cOqvx+p9UGoHLVgoL48tBpVXZxzY68KtISImjp6OLZ95RFaFJUsBpjCQowMjtpNGHD/dmZVwkyamxEKC2hVIVX5+ujJ81hLAh5XtyYi4TE8oVJvLKlkAOldcxxWORCwQtrdnbSxdjO10XVZKSOVZWl1S7jo5PU/tV6v1qlvGZDDv/IOcXU2JFqnze3WdTPrt6B9vmuqDrP/pI68o80kJ4cdUk9v95wiBVLlAD4vuJaZsRHqOVqYzk2Rx8Lj27NhhxWLJmGxfG9dl5rA8xrs/L5xaKp3J0eQ1pCBGPCh5EyaSTzZ0a7ncP9hVfp49lCBNTVXTuw4H4AMqYrSlKs4r9+LxubXeZWRyCup2CM1r1dvf4AO3MryZg+1inY1V0Zrr9pLVxP9wUHGrnz5lh8dM73iN+0C6D4310/aWVwtcDMp86pD69WubvWJRaG1esPqArX1ePSPlRaCEWopSk8tVl87zq2o0L9Wf95GSuWTCM9yTm4/HVRNZmzop0Wo+25p/Ex6FQPQLvw6PR6zjuU0N7iGlY4PIpOq40pE0IJG+6vljU7aTSm8aFO3ohCCUSwt6SWXyxKYWLUcLV/ADJSx2Kx2vjyUCVPLkqhpd3KS1l57MqrUoOnFqtNXbC13POM+Aj2Fdewp7iarwuqMOgllt2RwKubi7jRFI6vj15d+AVNs6+kVu1bVwMoONB4sS+TFOs0LWEUvj56duUrC9zEqOGMHuHP/pI6pwXo2fcPYZdl9hRVs6+4Vg2gp0wKUxegomONPL8hR6WFhIL39dGr81FkCf1wvslhGY9RAt0+elW5nm3q4KWsPLcWenJMGK9vLUavk1n1gxtVak+0t9NqY1deFVNjR/Lbj4pYsURZUFNiR6pegZbSFR5dSmwYB0priRsXwtqsfL4urGKnY8GdnRSpjouvj15dZEVbYiKDeWVLIWmaxWUwFH93Sv+6OE9fTF5PwT+RqqX9TZvXu3r9AfUsEpHDK+5b9VAaOkc+urae3sjx1JJUkJTgk3iRgjY32bWc7soX93k6pdPdeTaeTlp0/c61vuBA5/x1kVYp+tDdYiGue+adg0pbNSH6plZLty/t0NbrGhR212ZRj+uZKuZT51i3tYRH70kkKMCovlC8qkEJcGsDfwJ6nZ5VD053+t4u22luU84zApw24DS1KvnXa7PyWfX2N+q4uDsz/WJbZYb5+2A+de6SnHhlQ1saw/x9lNTVBUlq4FwELV0D8iKP/vlHZvH4wmT0OolVD04ndkwI/r56dWesNng7JnyYU2qldpy18/1igFVSTwAVL3BparWw4QszyxcmOsmjkyQWZ0wGJOyyzEtZeU6buyqqzvPutnIn+UXbRB8KJfnLxdNIjYtgzcNp6usJxT0t7VZlP4BdvuT1kb9cPI2Wdis6SWL1D2YQFHDpBsbgQKOak//oPYmOeZvN2qx8zKfOXbIBUOTdD/P3ob3Txjt/LWXFkmk8fm8yMjIv/CGXl7Ly1HFdvf4ALe1WVr99gBc35jodGCc2Lg4Wp98drgtLX1iSWqtOu8Inx4Sp6V9ixRcWJSgWV8b0sU5ureAHk02jiB0dpFp4PaUBgWDYAAAgAElEQVRoaS1oYWG98IccvjxUybiIQHXCiTPFtcEg4S5rrXqxKO0rruWxBYlO7dBCWCeurrM2s0RY7k+vP8j23NPsyq8m2RHHmJ10qdss+m5i1HBuNIU7Wbbu+l/QEUEBRnbmK9SF1i0XsoiUTa0Vr5XbtVwtFSWolCfvT1GpqjHhw9QslSfvT1FzqnfkVTIlOlSNZQQFGJ145+BAI8mTwpy+nxEfwf6SOubPjOa+W020t1tUK1iMaXpSJLOTRivegY/eKd1RWJyAGhSVZZmvC6r5qrCGJxelcE96jBoLWb3+APNnRqtW4sSo4U59sLugkjtumuBkmYvgaafVpmbv+DgMk135VSofr71HS1GJuIOYc1ovWQRYUyaFkZ4U6RSs9TXqGRXqz7vbypkUFcxIx3wYNcKfNz8rxaCH1T+4kfkzo9W5DPDqlkJ+cucUbkuL5rWPCsmYrniK8eNDmJkwile2FLK3SPGkRFxC9LUYJ9HWJ+9P4Z45MU4JCVUNLbzwh1x2F1Tzk7umMMzfh19vOMTeohpkZDKmj3OKqQjq7PaZ0cxOGk1aQgSvfVzE7oJq4seHMD5qOLUNLWo8xmK1cbC0Fp2kIy1hFK9vLWJZZgKFx86oh9s9tiCRealjeXWLsmFOJ+mYlzqW+TOjSUuIUD2awcJ1T++I7AyR8+yatSMUuFgUOq020uKVgXh6/UEnRS4Wg8YL7azNymdGwih+/V62GpkX7rEn2sHdd7vzlcm44QuzujlEy+8LN1LkvLu63SJDRyg5wWFrIQLO7jJ/hPK0WG3Kw+jIexcc7vbc00yZMOISnl9YayIo6m6vgugHbYBRLHYvZeWpi5VWsbrufRC0Qkykoki0gTx3VNSeomruTo9ROdVRof68urmIJ+9PYZi/jxpkFrGMcRGBvPWXUna4UG1VDS08/2EOO/Or1O+1Srylo4vVb31DTKQSBP+qsJI5yVEAl8QztIvkS1l5al0z4iOYNjmcrwprWL4wkaSYkZfQDdqgoradh0+euYRKmeFimMxOjCQ0JID/XLeHeamK4aIdR1WmjXnsKa7mR/PjeePTErVN8eMVyiJZs6dBBKVvd+TBi/IOnzzDuq0lWLvs7CuuJX58CEaDXlXqRcfOMV+ToSLqj4kMJnpUMC9+mEtrRxexY4MJ9PPhpaw8ssvqWb4wiYNlSpzg1+8fUvPgtTSsMGKEF2M+dY5XthSSMinMccKlzE/vuoH3/2bm68IqVi6dztTJI1UKTTxfL2/KV2NdwYFG9YUnU2NHcsPEUJJiRqpxvO05p9mRe5p9xbWsfFAp77WPi2jvtFF8vJGVS6erz+QrWwpVSu7W6eOYMiGU32wuYMqEEby6pYjtuUoQ/Upw+tcFvQPOr7UT7p5wL13PEVm9/oDqurre9/yPZ6mHRInzV4T1KiFRc6a1R5pHi+BAZVu/2A5uGh/qdCaKKCPAV8/vPilRqQitK9jcZlFd5hc35qkvW3aliFzvA9T2v7gxT6UKRN2CQpCQWPdJsUrfNLVaVA7/odtNTi42OG+Df3q9M/0ifgsKUPrysQWJvL61yGmLv/Z4A1B2lN5/yyTVzXbdVq9tT3ObhecdNI0oY8MXZnXXpCuVV9XQwvt/M7N8QRI6SXKi7174MJcOix273a6WLeRsarWw8o296m5VP6MEsjL+QjZRv3Ycx4QPU+uqqDrPy5vyiQwLVA/9cjc3ggKMyDIqHQWQd6SedVtLWJYZT1DARS8CcDp+Q7RTjLs7Gk2cf2Ozybz3t3J+uVjpq0A/A8P8fbDZbE5U2NosZdxFLro4vGzdVuUsmzUPp7F8oWK8iLpHhQZcsutY/C+etd+vyGD5wkTWbS2houq8Sn8O8/dBJ+moO9dGa0fXJXNN+87ZZ945yN6iauX5tMkEBSh9+MIjN5EaF8FjCxJp77RTd66NtVn5vPCHHFrarfj76hnm76POueBAI3lH6lmblc8/Dp3ipY15vLutjKqGFla+sZcH5sYiSQASMjJ159p4fWsRAMsXJqLX6VVKT+gGIWNzm4XXPi6ircPG61uLeWxBIjpJd8XelXtdWPoiaKblgrWZBYIrFK6ryFQID/G/xLoSqVmPLUgkI3UsIcP9WfXmfualjiUhOtRhZY1WA0y9WblP1japQVBh4Yl/q98+wJ7ialY+OF11p7UWjvnUOX6zuQBZhmmTw8kuqwMk0hIiWLMhR81Q0en12LtsTl5IVUMLz75/iL3FNaxcmsoNE0OJDAt0Sof71XsHefzeZNISRvHGpyVKeuB72YqrfKfinWj7TkuliR2u+4prSU9SZJ6XOpaaM628lJXHlOhQZVOLDCuXpqpegNUO5x3WvejXDV+YefL+FEzjQ9XxAZy8i8YL7TzzTrZq+YpxFhSUsIRf3VJE8qQwR9urscsy98yJcbJaAXYVVPHTu6awaO5kAEdA0s6t08cBsD33NDnlDSREh3KwtJ4Oi408cwNICmWgzUrSUmQiGLluq0I3tbRb1QCrGFfhzQjqaE5ypJrhIbzMxRmxfPz1cXbmK97njPgIggKM7Mg7TVrCKPXaVW/uJyE6lJHD/VXKRkAsTqNHBPDN4Tp+etcUTONDVQoH4MucSn5y1xR8DDrVM3p3W5ma1WSx2nh1SxE6ncyiuXFYrDZe+7iIpxxjKvh5MS5CmcaPD1GpwTHhw9Ab9HRZurhhYijrtpYwL3UM6Y5AaPKkMH73SQl6PWpmkJbeE97KormxvLutnGWZ8Rw+eY6M6WNpbrOoz6LRoCclNozoUcF8XViFhMTBslqQYU9xDXuLa9iVX01MVDBv/aWUf7lpPFt2H0cnKZRM3LgQ8o40kF3WwC8WTeWeOTGKtb65CL0EErBo7mRmOFKQVWosKVLth4OltYDMz+6+gdvTxivegMMj6c0O7N7guqd3AJWzq2poUXPtfX30/HrDIR5bkMTEqOHqtULxiuwP7WKxZkOOap3uLa7lzvQYZjk4zZHDlY0hQQFGVeGmOHL63fHTgBMHLdK6xL1TokM5WFqHDGTOmuDEg4rUw7VZ+Rh0OpAgu7yOZZkJ3H9LLOEh/sREBXN3eoxiob71DUkxI9R7Bc0gyzKSBHHjQli3tYSvCqvU9MCaM63sLqjhUFktuUcanVLZDI6Hb+60MbS0WzEaLmZ8iP59eZMSeDxYVk9agkJRRI8axqubizDoJA6Z61m+IIns8nrmJEeqClIsovNSlQdWxALEGAllKFx6ERcQHKzoR0EJifz4NRtySE+KZG+xknK7v0TZzp9jbmBGfISaSaLNrpkQGaxm0+zMqwRwylp5bEEir31cRIfFhq+PhE4v8bgjB1zIJygycd+81LGMCPYjJTaMyLBA1mbl8+SiFIyOLJSYyGBe2pjHzvyL+ycOlNZyg6DYHFk/H3xpBhlWLp2Or49eTYvcV1zLnuJqduZVsaeoGrvdzu6CGsZFKHWJxUWkP85OisTHoONAaa26sJysbVIpnJTYMN78rJQvD50mfnwI6z8vY/mCJG5PG69Y0h8oz0SuuZEZ8RE8/2EOHRY7N0wM5ZXNhUyNHUmuuZ7vpkSpXrKvD+QeOaMaR51WG0++9hVfHqrkzu9MYF6qMq9EjMpitZGeFMm0yeFO+x0ENQvw90OnMZ86h0EvMS91LIfK69WYzddFVYweEcD//KmA7LI60hJGMSc5ymEo1aOY7TIP3zGF2UmRvL61GJvNxvGaFnSSjE7SAzL7S+r49SPf4ZaUSMKGK4tnUICR2LHBFFScod1iJ3ZsML/7pITkSWH8+n1nWu7romqW3ZFAYcUZ8o82qAaRmCPujgDpD657pa/lxF/dUoTNbmdPUTURIf4cPnGW7PJ61aoRAbln3jlIkL8Pr2wuVPOnRYB3XupY9hbXIgH/kh6Dj855o41Y2adEh6oTVHCQ2hRCcX1KbJh6MJnI901PiuSVzYXIyI4A0cVgmVAmY8KHET8+hHvmxDAzYRR7imrYX1JHSmwYnRYbaz7IJXZsMOs+KeaJ+6fy35vyVR55TPgw0pMimemY/Os+KeYXi5QgmLqj1WJjYmQQOeZGfrEohbDh/jz3QQ4rlqQybXI4a7Py2V+iBCC/KqwkdsxwWtqtZJfXkZ4Uye0OBbK3qJppk8M5WFZPYcUZnlyUwswpo7g7PUa53pHXffvMaCxWG/tKalWFL7wwcD4xU3DSwupdseTipi+hsB663cT6z8t49J5EjD569hTVMCc5UrUeZycpHHBCtKKcUiZdjIdUNbSoymj12weUQH5CBHuLa9lXomxO2l+i5HFnl9cjYeeZh9KYkxzlpJQ6rTaa2yzsLaphT1E1u/OrVY7+UHmDutlpXupYXsrK44fz43nrL6XK7mAHJzwuIpD8ow3kmBt5bEEir24pIi0hgmzHBp/MWRNobrOwp6iazFkTSEuIYE5yFOnJkewtqkGn1/HTu6aQGhdB/PgQwob7OzzIGuyyzA0TRvDbj4r4UWY867YW8+Whb5Wg8v0pBAUoxx+nJURwsKyOjNSxzE6K5JUthaqymhEfQdhwf+aljuV0fTP5RxvpssnMnTaGA6W1HCitpcNi50BpLZmzJpASG8aC78Y6eWsJ0aHkmhv4yZ2KpyHO2Xny/hTHNdl8c7iGQ+WKoSD6uLnNwosbc9VyM6aPZU9xDYfM9Ty1JFX1JKZNDufVzUXoUF4ksre4lr3FtWofPnFvMtll9ewrqSPPXI/dbqfTKvOD78VRevIckg6euDeF/KON3DxtLIG+BlavP8A/Dp1iZ34VBUcbWXZHAkUVDRRUnMEu25k2OZxd+dXMThqtzt/RI/wddKPyDGWX15MWH4HFamNvcS0Z0wdng9Z1r/SDA43Ejw8B4O70idwwYQS7C2rIP9rIT++aQnpSJL/ZXMD23Ep1g05K7EiVo4wMC1SVrcjtFjnWkl5H/ZlW1mzIUTNehCv324+KuP+WSSTGhKkBVu0mjl9vOMSO3Eqyy5RFRxl4xQoVGS4rl04nPTnykgwV4ToKWig8xJ85yVGkxIYpG4CSIjlwuIbCijPY7HayS+to6+giJfZiphKg5l3vK65lXuoY1W02nzrHmg9yWfjdGDWAJdxpoQhmJ43mjpsmkhIbRmJMGK9uLmJfcS0/+F4cb/2llBnxEby4MZe2Tht5RxpY9eB01ZIUC9Krm4uw2mQOHK5h5pRRaraJ3WbnuQ9yePSeRCxddtZ8kKvml2t3+a7ZkIPNblcDqKve/kYJjN95MTD+2sdFqsW7r6RO3VwHSnbMgcN1tHXa+OZwNbsLahg1wp+1Wfl8VVjFDRNGqA+uspEmTF00UuNH88qf8nlgbiw55kbSEhTlN37UMJUKW7Mhh68Lq1BOmZf4UWY8kWGBTIkOZV7qGIICjGowcU9RDQVHG1m+MIlD5npVxlc2F4Jj81NkWCDbcyvJLqtTM0REVpksy4SH+PPWX0rZU1TNnOQo9hbX8JO7knhvWxlB/j5s+MLsoKPqePzeZLLL68gpb+CxBYm89VkpNoeyW5wRS3z0CPWIDJFlsjYrn/SkSPYUVrO3qIYduZXsK65lR14l4SF+vPlZKf/2fRMLvxuD0UevGkc/+L6J0m/PMSU6VE1UEEZMTKRiGf980VT+788lpExS5uhP7pxCUsxImtss7MqvwkevQ5IkdeEROfHtnYp1/dZnpcpCV1zLyqXOLy3ptNjYlV+FzQ42Oxh0Eqt+MJ1pk0eSY27g7vSJpCWM4kBpLUgSkiRh0EPpyfPqomAaH8roEf68+GEuKbFh3DBxBPtLFEos70g9+Uca6LDK6PU6dDodGaljyS6rI93hxXZ12fjg70f4yZ1TAHjj0xJ+OD+e1z5WnhtZvrjfZ6C47pV+U6uF5z7I5qvCWqJGBpAaF8G4iECKjzWqVq4sw8N3JJDtoCJACZbNnxntlDmi3fDx7PvZ/Pnr42SX1fHUUmXlTkuIUF3ef7kpmg+/PKKmYgqrbvLY4Y40riQyUseyr7iGr4uq2F9Sx/IFSfxmcyFpCRGq8tfSOoK3Flyq68YRo0GvZpnszK9iWWY8RcfO8vSPZnKgpIa702OcHjiRhifoGOGBiPQ3o4/eiRIQC9XOvErmO84ZCQow8uqWQp68P4X0pEje+1s5sqxsMc+cNYGokQGUfntOtQzF+Sim8aHEjw9h2uSRTptt4mNGcu58OzPiI/ifP+WTXd6AQS8xJzmKZ9/PVjOJ5qWOZfyoYeSa6/jmcD0z4hUL94aJocSOCWFe6lha2q3sLqjGoINOq8xP75rC7Wnj1eya6FHDKKg4g14HOknHA/Ni2fiPo0iSjF6n4+70GFJiwwgb7q96FXuLa5RU2w9zeGxBkmpBD/P34VfvZbO/pE61WGOigskxN7IsM568o43sL6ljZ94pvi5ULOCdeZVMmTCCV7YU8vhChQrLSB2rxm8EzZWWcPG0SOFxFFac4YG5sYpRMSmMrwurOVTewL03x2A+fYGx4YEUHz/L3qIabk0dw+bdx5mTNJqPvjpGe6dd9fTuTp8IwN6SGp64N5lpk0fy7rZyUmLDFGXo4KcB/pFzyrGY1CIjI8uKQjxYWqtYxLKdstPnSXco5eULk8guq6P02/MsX5DE61uLeGBuLKNGBKhxq7VZ+XTZbEo8Sob5s6IZP2oYG74wMzsxkrNNHewrruVnd9+AaVwIb39eqmYOTZscjmlcCBv+Vk67xaZ4JA6PQjwrwrv5qrASSQIfvUSnVeaGiaG8+Vkpyxcmqc/ZnOQo1ftNjYsgLWEUB8vqyC6vZ1JUMP/3aQldNpk8s6Lku2wyaQkRFFScARkMOkCSeeLeFPVMnrNNHUwYHcSmHRUAFB1r4KuCGu6YNZ7NuyrosNj56V1TKDp29pKzivqL617pN15o5x85VWTOHMfm3cfVQBRc5JUPmeu5PW08Bw7XcLCsjt0FNfjodWTeNIHxDh5au1uwuc3CtMnhFFQ08pTj3PI1H+Syr7hW9RQ2bj/KT+6cQuyYEJVSmRSlWLcP3hbHe38rJz0pkqmTR6que2uHlX3FtRw4XMue4hp25VepXoA4xjYlNoyqxlZe2VzIvuJalX5qarWoVATAjrzTFFac5bEFidyYGElsZJBTPrAIMomAZ0xksBMXLTAvdQxjwofReKGd9KRI5iRH8nVRNTMTRqnnlMxOUjhOQRulJShB0YRoJSj307umsG5rsRrnEKmnL27M4/5bYpmdNJqWdquy4E0Zzeq3viEldiS7C2q47+YYFn5Xeavml4cqyT/awANzYwnwM7A2Kx+bHR78Xhy//3MJXxcq55goO0aVoK1eDz+/byqmcSF8+OUR9pfUEhMVzAt/yGF/SR2/WJTCrdPHsadIUZoPfi+O8lMXeGppKharjVe2FLIzt5I9xdUsX5jEDRNHMMzfh79nn6bgaKO6OP/qvYPoJB333RzD5l3HiIkKdlhzJt7/m5mVS1NVugzg/lsmUXriDDnmRmRZJmP6WPYW17C3uFqlu8TiHByonHEDqHTLiCBfPvzyiKrUbpg4glxzPUcqL9Bls5NjbuBnd9/AbWnRvPvXMgx6OFbdjF6Ch26P552/lrK7oIbYscG8vrWYZZkKtXR72ngmRgbx3rZyNbUwZdJIdXdwWkIE96THcMOEEXxVUMO0ySMprDjDXbMnUHjsLL9wxCe+LqwiY/o49pXUsnxBEsP8fRzj18iu/Ep25VcRPVqh2Q6V1fPkA6nc+Z0J1Jxp5dXNiqXvY9CpVv/6z0uV8bktDrss85vNhezOr1boJLuMjx5uSoxkd0ENKbFhnGvppPj4GQ6W1TIzYRRhwX4UHjvL92eM42hVE4UVDXRY7OSU17O7oIpJUcFqPO7rwmp2F1STa65HJ0nYZTuHyuppt9gdXstZAP7t+yayth91xK7qsNnsWLoUg3Fq7EhO1zfz6uYipk0eSf7RRhbMmcAD8+Jo67CwPbeaLpuMn1HHnOQo5qWOGbRg7nV/tDIoucW/+6QEa1cXer0OZHj83mR1o05zm4WWdivrthbz1NJU5fMnxSxfkKT+r71WOb5VzzM/moW/QUnHqqg6z3vbylm+ULnnKcdmktXrD4CMmkoojhy225Ugqk7S8cP5Jt75axntnTaMBuU7nV7Hssx41n9eit1mx+Bj4PGFSfzvR0rgcHFGLFMmjFBzk0XsQvtSB/Fmn7WPzWHF7/YgIanby1/KyuOBubH8aVcFP5ofr6YAhof409phZd1WJeVuxZJp6t/+vnruSZ/Iph0VrFgyjWH+Pry0MU+NPaxYcvHoYO2LMABe3JiH3WZDp9fxwiM3qbytKOeZd7IBWP3QDPwMEnXn2nj7sxI6u8DfV9kZ+8IfclmcMZl3t5Wrcq3//DBLbjXx7rZy/HwU13zVD25UF5bmNgsv/CGX9k4bfkaJn983Vd0J+aPMePUY4OY2Cy/+IRedTqcuwO/9rZynlqSqc6Orq4vOLliWGU/wMD/CgoxOr8JbMGcCn+w5iZ9RwqA3OMa1FLtd5j/un8q6rcW0dnShA8R7pBZnxPLnfSd44ZGblJTTD3PQ6ZSxT42LUF8eouSbK+m7MjLtnXb8fCR+fv9UXv+4CBlY9eB09doFcyZw5+wYfPyMlBypA+C3WwrQ6XSOAL6yU3bTjqOOvlHoE1mW6bAo0hkNYOkCP6Oyk7Wl3crrW4tZuVSZ2zVnWh3PlZVOq9KWmVNGs/LN/bRb7CxfmEign4+66fC3WwrotCqPtUEHXfaLdQDqy3Ieul15HnC0KSjAyFNv7qPDIuNn1NNhsZE5cxzbDp7GoJfosillrnk4jbpzbeqztDgjlk07KjAaQK/T0W6x42fUMX/meL48VIm1qwtLF+q4iWcDlBTZAF89j9+bzO8+UV6WUneujXVbSzDqwWIDox5+cncisWNC+OUbe7HYwEcHBoMeSQJJJ/Gj+fEE+vlwqq6ZTTsqnMZeL8FD8+PV+ex6FHl/cd0frQxgNCgBp30lChead6SRg2VKRsRLG/P4qrCSrwpq1KwUo4/C37+ypRC7XVb5ZkGdRI0MYO60sbzyp3x25FayI6+SgopGHpinpIxJQHqyQp1MigrmoCNjAFAt/plTRnFPupIquG5rMUjwg++ZOHzyLB0WO/fdHEPWdjMdFhmbDLdNH8OmHUfotCpT5nhNE3OSI/n1hkOq9RwW7Ms7fy1lV14VUyaE8vrWYmRZ5nszo5mbEkVCdCj/86cC9hbXYLfDofIG9DqJgopGkGVyjzSyzxHgAuUh/uDLI+wrrlWooooGCo6dxWiQOFRepwajVi6drtJWIsAmNiHNSVZc/QfmxXLI3IhOkpgYGcTbfykD7OSaG5k/K5rYscEUHzvDzrxK9hVV8U1pPQYdLJoby8LvTlLSEXNPs/Q2k5r58t9/zAN0HCpvYHFGLEcqLwASmbMmqFk+zW0W9pXUct/NMVRUNZOWEKEeeLXuk2LlmNuNeaQnR6oU3f9+VMD+knr0koxpfChv/aWUu2ZPoODYWXz0kHukkQOHa/mmpIY5KUoK4biIQD74+xGWZcYzc8povp82nte3FtPeacNmh/wjDSy7I4FJUcEUnzhL5sxxnKpv5vtp0Rxy0Iot7Va+KqhBQuabw/UE+Op57eNi8o7U87O7b2DutDFkTB9L3LgQ0hIiiI8ewTufl9JusdNlk9WjfsOCfdm0o4KwYF9e/kMuB8tqOVRWzxP3KUH0rwpqMOig5OQ5JOC+WyZxtPICy+5IINdcj6+Pnh9830SOuRG9BFYbqsW8O7+aA4dr2FWgHJudnhTJvpI67rs5hk07KvD31VF84hyZM8excXsFeeYGfpQZz1uflSLpJG6bPpajVU04XqGMXgcGPfgYdBRWnOGxBUmMCPLjy0OVdNkUGsbHoGPkcH9KTpzl375vouzbs5SdukDmzHGcbmhFttuxy3DDxFDe+1s5d8+eyJHTZ5mTEkXixBFKO3QSNrtMl02m/NR5fnrXFKablID4ydoL/OzuRN78rJTd+dWYxoVQfuosD98xhRFBfmp8pbXDSnZZPTZZkbvLDtll9YQGGSk8plj/Bj1IkpK/uSwzgbaOLtZtLeHI6bMsmhtL8Ymz6ACdBDYZSk+eweijJyYqmCW3xl12eue6sPSbWi2sfPMbnrgvmdc+LkIC2jqVNDx/Xx2gWDc6STmU6t1t5bR1dKmW9Isb85AkWL4gCUDddefvq+cX/5pKeLByrk1F1XnVCnjygWm8vrWYxxcmKYrXLjveGSuzLDPByYqODAtk1dvf8PjCZN74tER9DRuAr4/E0ttMfFvbxN7iGjXI9ud9J1mWGc+o0ABe3KikXj5xb7JqDfr6gF5vUCw6QK/X8eg9SqppW4eNZZnxJE8aSUXVeWLHhFBRdZ71nx9GtgOSRKfVjo8efI0+3PmdaDbtqGBZpmKRLM6I5c97T9DWaSPQz8BTS1NVC2jBnAl8eagSHP0lrMIXN+Yiy9DeefEgNWFd+Rt1PHFfCq9vLeJf503GYNDz1meH1d8BAvwMLMtUvJE1D6fR0m5VvQ8/o44lt8bxx50V/Os8xXN5YG4sm3YcYdWDN7I2K19tgyhTlPH61mK6uuxYuuz4+ujotNrx99Whk3TcNXsCW78+RqdVxmjQYelSFtvFGbH4+xpUOdc8nAYotEvRsUY2/sNMp1VmWWY8AX4G1v+lFBmZR+68QfWmPvx7OQaDHllWYgddti4kSYdep8Pa1YVO0rHktjj+uPMobR02fA0oey3sNux2RQm7IiM1ihvjRxEZFsjq9Qd4wJGz7u+rY3FGHFnbjyBJEk/cm8xrHxUhyzJICi+vtFFRVmKOjR8VpM4nUKzwto4u/rjjKDJKDEwbLMfhfQCqNSv6VHgRwksEMEjQ5XjC/Y06kCRsNjt6vcSqB2+kpd1Kw/l2p3N5hByBfj78dku+6l0o1rzEk4um8urmfNVzAMUr++OOo9wyNQ2QIhUAACAASURBVIptB0+rY3jbjPHquVH+Rj1P3JfMMH+fi9a8QcLiENDXINHZdVEdLc6I5dM9J2i32NQ5pcqhRz2Pq8PqrMKWZcaTtd2M3SarXoHeIDHDFMGe4jpVroGiO0v/ulD6Yvei0SAhSfAf90/ltY+LkGU7P79vKoB6XsZ7fyvnR/PjeeuzEiwONz550kjV7e6w2J2U3x93VPBLBz3x8qZ85iSNZtvB0+oE8PfV09Vlw8eguIn/u6UQGTt6nZ670yfy530nWbk0lbVZ+Tz3Y+WgKUGJlJ48y6YdFZdMOOHCrttaQqCfQfUuViyZxmsfF/GwY+F63JHtUXOmlZDhATz/fjY/nG+iraNLvf53n5SoG4UA/I16Ft86WX3QhAstFhpZtvPEvcr5NYISE9SMmPxGAzy56CL18/wjykFeYlHMnDmOmDHDnepEcl4QRL+veusblt2RAMC728pYlpnAmQsdquJYnBHLp3uPIyGpC7mQWZQj6Asf/UVlKWTNSI1iR161qoCMBnjmoTRO1DTxxx1H1TIB9VoBf189D9+RQKCfjyP3XFKpC60MgEopiEUMFIWo10l8P22c+r2YW3CRVlv/l8N0WGV8dMqLW7R99OHfy7HacKIMlmXG88edFax00JSvf1xMW2eXpl6FAuuwKAud8llpp5ZqWbFkGv+7pZAOR6U+Bglrl0zmzHHsyq9CclBQ720rxy7befiOKbR1dFF7ppVtB0/jo4ef3q0cXLb+81LsdrB02dXFQ7RTGFcvfJhLu0MOf6MeSQc2m5JNZNQri978meMIDfJj046jAMqz5aPH1mWns+si9aMdKz+jpNJ/or7P9p/kR/OVuEZ7px1fH+i0QqCfgeULk9wuNuLeoEBfpkSHsuL/9tJpRb1Xa6R4HnNFF4t5op2TAtq3qPUX3Sn96+IYhtgxIfjowdIl02mVae2w8vAdCegkHa9/XMRrHxditytKpavLzrvbyrj35lgANu04Ss2ZVurOtdFhseNv1DExMpgAXz1/3nuCVQ/N4PWtRerW9F0F1fj6QNhwPwDmTo3CaoMum43WDisdVjudVrhlahSbdlTQ1tHFiZomViyZpu5UXfnWN7S0W1XFptNJZKRGYTQoVg4osvoaJB6YF8sfdx7F16C0VSdJBPr5YLfbef3jIiqqzrM2K59n39mPtUtxM7O2H8HPqAy9XbY74hCJ+BsVrnjTjqP4GiR8dKgPr6LwZRZnxLE2K5+KqvPqVvYTNU34++qVILCPhKUL/vejQl74Qy5dti6a2yzUnGllwxdmFmfEsu3gado6uvDzkfAz6ln1g+k8cW8yAb56jAaJ/1g0lT/uVPq9y2Zj/eeHWbe1hLYOG2/+uUTtlwVzJvCX/d8iSTo1AwUUmQ2OmZ21/QgP35GAr4+E1aYoboBP9iie0v7DtQDcmT4BUBTeiZom3t1WTleXjeULE/Hz0TkpEYPmqVn/+WFe+7gIX4OETqdjcYYyb3bkVQGKYl+WGY9sl9VFZllmPACdVjtdNptqJfo6zr4O8NWzLDNe9WRkxxZ+q11R6AvmKLK2d3ZhtSmKQyh8ox427TiC7DjNsrXDCtJFg2FO0ig6rTJLbo0jc+Y4Oq12fn5fMssXJhLgp1fn/bLMeEzjQ1lyW5x6r9VheGw7eJp2ix2bzc47fy2ly2ZHtsO6rSW8u62c7bmV6j3rtpbw1mcldFgUb2rBnAnEjglhYmQwyxcm4mtAHc//7+FZ+Bn1+BpAxs7jC5PVAP6clCg6LEpfvbutnPZOG7enjcNqQ/UOl2XGIyHjb9STMGEERoOy+HZYZNo7u/D1kVTqzWazs25rCXabjL+vnoXfVdr9wLxY1mbl8+HfFYVvNCjj7euI223ZVcFbnx3mq4JKHG+gZOlt8eqcAqVOf6MOu92uzjO9QwV3WmWneSIU/swEJTvO9YTSy4HrgtPvtNrYmVtJl12xUrbuOc7+kjr0+osn/HZY7Nw2fSzlpy5gt8lUVF9Ar9Ox5NbJrNtaQnZZPcsy41l6m4nwEH9GhwWwr6SO704dQ3ZpHYvmTuKDv5tp77Tx7/ck8s5fy7j/lkl8kX2aO78TzfHqJhbNnUzUyACKjzVirmxSKBidjhxzg7rJ6b6bY8g/egbTuBA14i/JMhXVzdjsUFhRz96iWuw2xbLJP9qIhKKscsvrQZLZV1yDpUvGapMpOXGG1Mkj+bauFaNBx/duHMuJ2mZkWeaQuZ5/nTeZwyfPkp4UycHSWiRJUjamdMnYZUWxHq9p4uE7EiioaKT05HnuvTmGd7eVk11Wz+KMWD788gh6Cb45XM9Dt8eTf7QRH71ElyOTYX9xFV8V1jInaTT3fHcSOknm073H6bDIasrbax8XsThjMjnmRvKP1NJhkTlYUqtYtjLYZUVhV1Q346OXsMtQfuq8stErYRTvf2GmyyajlxQL646boik/dZ7v3TiWT/eeRJIk7rtlEp/t/1ZRkjLEjw/BfOo8kgQnapqx2mQ1kJ1dWovVBiODfSk7dYETNc346ODu9AmUfnseP6Oef7s9noOl9eh10Nml1F1+6jxddhmbXcagVxRl2bdnVcvuRE0zh0+exWZX6sqYPo78Iw1MmxxOjrmB/KON6HRw+OQ59pfU4eejQ5bt2BwBz4zp49i4/SgGnYT51Dm6bDIGgw6bgyD38dGrAdpJUcG8u62cu9NjKD15Dh+9xInaFow+ypw7WtUEwLTJIxkVGsDY8GF8+OURFsyZwOfffMuoUH82/uMod9w0nvJT5zHoFKVYcuIsBh1YbQolaumyo9PJyHbleTLoFXmMPjru/E40h0+eZ/nCRBInjuAv+7/li4Mn2Z5bRWFFPZ1divU7LiKI339azD3pEyk4dpYuG2SX1qo8+Yma5kue6+PV57HZIXKEPyfrWpRsHKuMhMyB0npsdtQ2lpw4i80O5afOccOEERw4XIfNLmOzw53fiWbL7uP4G/XEjhlOyYmzarzhRlM4pxva1LbZZeWohbJT55mZEE5VYxsTRg+j/NR5Nb5Qduq8o1+UQuLHh1B0XGmH0QALvzuJT/ecQEJW66lqbANg7rQxjBze/atKe4PrPmWz02pjR+5pumwylY2tyLKyOePn908lp7yBe9InUnLiLEermtADNuBfboqmouoCh0+eRScpAaCyk2e4YWIYRoOe/96Uh06CXHM9d34nWlV8i+YqK/ih8gZKTpylyyZzvPqCssJLMp/uPaGu7ovmxlJ47Ay+jof5aFUTx2ua+JebxrNpRwWHT5zBZlcCXBKKErnjpgkUnziH3hEEypw5jlMNLUgoGQO3TI3CfLqJjNQoTtQ0Myshgv2H6x22ooz5dBNdNmWyS0COuZH7bo7hwy+P0NZp49++b6L4+FlViZyoaaLDYqf42BkeviOB2UmRfPjlEX561xTSEiKYGBnM3uJqJEkpc3bSaIqPn2HxrXHkHlFSE/UO7uFYTTNnL7Tx90NV6HUSekkJUEeE+FF8/BzTJo9UFPHp83TZ5IsZDjow6CVO1zc7+uysqvjHRwzj/S/K6bDYyUiN4lhNM9hlSr89DzgeetmOTqdT+/OH8+MpPXmWgoozyLKM1Qa33agEF4uPNbKnqFZ9GI9WNZE5cxzf1jVhtSlKHUC2yeSYleDxD+dPITTISMGxs9w5W1lsZiaEc7peeZBvTol0Ulo+etDplDF/fWsxy+5IYGz4MHSSzImaZm6ZGsX30sZTfPwM82eO5/BJpU6bXVGEnVaZ++dOciweqApWLDQgqfMPoPTkObVemx0eut1E2bdn+N6NypzLLqtnV34V+UcbMRp0HD55jgdvi+PNz0qRZTvHqi/Q9f+z9+bRUZxX/venqrpbCyAQ2tC+INGS0IIESGCQF+RgI2MMGOOAcRbs2JnEdpzMeQfbSWbejLPZv8kyE8bvJF5ixwkktgOOF5zYls0OAqF9aySBkNAuIXZJvVT9/niqqqslQZJxMue8x/OcYwPdXVVPPct97v3e773Xhz7e08yxKC2Mo637IjZZ+AK8KhRmRLA4M5qWzvNImkajPg9z48JY6IwmcmYwla4hfUyvCNaNBlWtQ3i8qtlnEM8ryY2hc+AKIGAJDfFZz9AVFEXG69No7b7IkqwoTvVeNq8DYfWsWZ5CZ/8lQPhl0DSONvUGMIiMtbI0O5p3jwoos71HKAPGs4McCisXJdDRdxHfBEFtHIiusxfRVJGi0dg/NglOnh3hnpvTaeseweuFulNCLhiKjHVtFGREmsy7T9L+15F7xc0TPz+Mpqk8fk8B//FGHaNuYbq/9G4zV8eFM6ZwXrRJdzMw2UfW5/DCO40sy4k1zXsrdnf/bU5e/ZMrAFM12qbSdHbtb0NCwuvVsMDy2HUarseHKcCNe797uEM4eRRYkh3D0eYBPF4NuyLh0X8YbJfx+VQ8KpMcScb9PT6BjY67fagEUuRK8gLxadHXU4JCKsvctjjBfEcrzmtgpsafBnVNkUBRJBRFNrNSjlucWIZDT2CawqEtzNwMdnxwEq9XNcdnWrBNOFH3tZu+DGOMgh0y+XMjqGgeNN/HaEafAHJTw6k/PaI/WzjDDYw2yA5ofj/JxPuU5MZwoL4/YC6tGL0M2CxjYsXhHTbQ1EDs3fqMdSUp/LGii1G3L8BRCMK0/683G8y+iFGa7B+Y6r5BdglNg29/frEJTykS3JDjf5dgh8KKgjg+ru5h1O0j2C4FOBoNv4aBsX/n5WN4fZjrLitpJl2DV/nCKifHm/upaB4059/wbxl3sysCarT6ouyKIBSsKIhnT0UX60pS2HOkA8PdYOyhwowIqlqHA95TUSR8vkBxYMzxVHsP/GvOpHZa5ln0D+5bmcmv328JcPyuK0lhXmI4//H7WtMxDcKH89jdefzot9V4daVJRiiJf0mb+F5T4fkGzfaTtk89ZVMEZ53FpwrNsLnzHPfcnM7vPmrj0bvzON7cT2PHeeanhuPxqhxrHjDNwvjIUBpOn+d07yUcNkmnuZ1nXUkKrV3naT17AQnVXAQgJu507wVOnr2Iz6vi9gka121FieZ9ZUksQLsigQayDjm0dJ7Hp5uQPk1oGqqqLy59zW8qTWd5XixHmwZwKNCo92fP0U4+szDepMMpgGKTkSVQZIFpC0GDCVfcXpRIR/8latvPmXQ2I09L+YlOVA1uW5xISX4czWfOoeomsaxr9p9flUl92xA+TWiRmio0Z5+5uYRG/g9r5+uRi0OoOuyDpjEvcZbIVQIm7HLzgjj+cKgDmyKZ95HFMLFwXiQVzYMUZ0UxeGGMvLTZ9J4bNX8jI+4xcH4MRRLjblMUctIiaDw9hE2RdbaM6tcIbbJ4H01gq1V67hib3ne7Aq6ui9h1WqFXHwOjNXUMmf8WMIyMomjYbYIPr2l+7bOj96JJufVNkFT17cMmLGUIfICuwcvIEqhTSDZVjwLVNCFAZk23mwe/hl9TBUyt2OC022wy99w8l4bT54RlMig014bT5zhU3yvgG8u6G7owjs+nUtE8YGq5OiGN+tN++OLzt2fScGoITRPX2iTxLI9PM/sAYq3LEnxlnYB+qlqFZdh7bpR1JSmmVYVYKmQlzWTowrj52cD5MZKiQjl/VYDrqXOmcf6yB5sE95amk5UcblrwxlhNHLdK15CelkG8i7EHq04OcntRktmH4qwo+s5dZUn2HA7U9fn7NXlKApqiz42kv5expoJsEqo6+bBq6TwfUCv5v9s+9fn0Z4Q6zBfdWd7GmFtjZ3kbV8a8dPZfMrWtX/yhwWSUgFgIhrYbZJN4aM18JFnc6Q8HOvBp4PF42bIyU2h4+nUi+ZrG0uxoPKrQHN0++KDS7+BSFIlguzBPvRp4fWJzGE1/DApCiBiLwybDrv3t/FJPH2uzKWY/r4x52VPRJZy+itBAFs2LZNyr4fZqBNnFARBklyjJjcGjCqecx6KN2WT45Xst1LUP6U5CiT0VXby0pwVN08yxumNpCiAcpV59ERdnRWFVxoTAFU5okwmhCc0QxKH3+71t2BWJYIfM/bdlYlckc5zKliQTbBcHrXHfiuZB888xt2pqTqWFcbi9GjcuEI5ah01m44p0PD7weFVe2iO0OQmNmxfE4VXFRg4JUli1JMnsU0XzoKlRGsJxw83pbCoV9/J6A7epXYEv3ZljOuYAVFXlS3fmsPnWeZME+5hHvaagyJs7m3ePdgJ+BcKuwKriJNxeARtkJfmzwZbkxpjjaHTr3SMdAfdU5ECFz657FO02Ia3fOtRBSW6MOa5GM+ipE4WSImM6JUFYNCpC6waR3vvXf2ph3GuZZw1y9O+N5rDJrCtJweODU90XTDaO4WgPnxFskhOM1tx5IeDfkgSd+kEFcLrvivm8neVt5uFnvPe6khThZA1SCLLLAfPgVUUglfFuHq+PPx7rMvtT2SLW28kuf8nEIJtEWXEiwQ6FqQJoCzMiMIbfsNhuyhd8/wUZkXh1JcNuuXZrWebf3ZH7qYB3DMqmTYaVi/2msiILbSvYIbPYGUXlySF8Xp+5WK1m/qbSdN45coZ7bp4bQOUKdsiMuVWC7TLLcuewr6YHr+qn95UWxpE8J8xi/stsWTmPHR+6WFWcHEDngqlN+SC7xPob57Jrf7sueIVGJcuicMeYRzV9Edb7fHii29y899/mJC4iNIB3PfEdDfPXChM9sj6H4QtjhOg78KU9LdgVob14VaFdGZtt4nsYMIK1FWdFBQgYh13G7VED2DFWuOt6zS4L7Xcq83oq03mqfkw13lYt27yfTUL1adfs10Q6JUwNE4EQBvWnhq/Zv4lwk3GvQ/X9k4SwwwZ33yTmK8guoeqWlhUKBHFYNHdeCBhbh01G9akBsONUzYBPpurX9a6RFaHMGBr0xLa1LJMzfRcDYMaJ7VpjaLTUOdPoGrhyzX4ZfgDr463Qn9GM8YHA+IFNpen8fl87bt1Rrwfz4/FZ4FIFkCZDpuD/jXXsjPVl08fH2kKDFb7/paV/14jcT4XQv3jFzf/z3EFsisiJPepWyYifQWu3cKAYAsCu6ELEqq3KYqJlWQRv/Pi1GlMzLs4SuXes2PVUG8O4r6IIDPd6m2fb5gIGz4/y8p4WAXlYhIk1IMQQaobQctgwnW3WFmSXWX9jGm8ePMXa5WmmMDcOPOP6EIfC2pJUdh9o58HV83n+7UYkSWz3MfdfPw12GSQL7BAQeq5HRkKgvwD8G0LQMTtMTNnr0yYJYpsMNy0I3GiGULM+z6HDRsaYW78LCVJwu33XFObXw9OnasK3gUh1rQuriYeYdU7LihN5/3hXwHqYGDtwveawibXlVf3CzHpo2RW4q2Qub+xtN/99vcMQ/NizIgmL06CFyrLAyNeVpHDxitscd7sCq29ImaTAgBh7Q0heq10Lk5/YtpZNxt/Bv2as4ywhoDj3NTaacZhMtRfXlaQwdH7UnD/DbzFVm0pBsH5mHBRfv7eAn75WE+DjAD8059PEuvjGZwtwJoXzSdvfhafvdDplp9P5X06n84jT6dzrdDrTJ3z/JafTWel0Oo86nc7V+meRTqfzfafTecDpdP7O6XSG/nef/9e0S1fdumnuQ0NAG4bAB0yNz6YoAYsGdAeSCqNuH5Ut/QFQSEXzYEAwDghLYmLz+MSk3rE0GfAvsrLixADON4gw71f/5PJvAsvU7Sxvw6cGBhl9eELwwe9YmhIg8A1e/7hH5Y29bYyOq+wsbxOHmP5OioQp0LKSZ4n7+zS272pg3CPyAlkTrxnNMbHTerPpGQwVSQg1t9ffT+u+8lk6euvCROsrmptl9wEh8MuKE/FMIfBlxDhO0qwUIbQcFnvbp4LDrlCSG2MKGEUWAvP2IvF8m+yHS6xtqhJ2xuvn60VpwD9NPg28XgKsJ8NHA6JvigUf+aDy7CShU17VQ0luTMA7K9fYwm6vEBqAqb1qCFgQxPi/fei0+fs/J/AVCZrPnGdTabop8I3rDF/E7gMdAeNeOC/KFPj2CTCHql3/mSW5MSjK9d/RgI5Gx70YI7muJMX83Bgn68GqwSSBb4XgjjSK+ZnqTNh9oCPAurD6UqzwmvEcacJ31nnz6XBmZ/+lAIGv6GvIqwp/B4g98+9v1P1FZVY/SfskmP5aINjlci0FngB+ZHzhdDrnAI8By4DbgB84nc4g4J+BHS6XqwSoBh7+BM//i1t81HRKC+PwqDDm9k1pLubOjeT24skCuygz2lwY5VU95oa3K35BD37hadUKrWs4K2kmF6+4zcAa47eyIgcI/uffbsLj08xrDS3ZaG4vepUf49+ic0PnRwP6XV7VI2AgxW+KluTG4FUxMWvrJjFZBZbPbi9KDNSiJzzT2koL41B0Z531vptK001c1GjFWVFmZOKeiq4pNXjjGqvQNYSyVbAosmTi1MVZUYx7xbsYLJ6M+BnC9+JVOWCBR3z6obT7QIfwqahMuS4MbdDaDGpi7Sk/vdD6DuqE31q/r2odxqdq5rtYIZhcC+59oL5f5HdRhJJyPbjL+E5RJHNsFAVLQNBfiMno9xp1+3hjb9skYT1xHIxxN+A6mxwo4O2KZPpO/PsmULIfqO83r/FpYlytv5HxH2Y7y9twe0U2zfio6dSfHgkYM+utpxJsrs4R896CxSb+7rBJOGyYQXOFGaKgkrG/JR2YtyvSJL8C+Od24ndWGWH1L+SmhgfIBkPpCLKJyOi/d63cTyL0lwN/BHC5XEeBRZbvioBDLpdr3OVyXQDagDzrNcB7wK2f4Pl/cXN1jlBe1WMuiqk0uh//rjrAPDUmcuKmNzaxx8cks9+4vzGh1n3a3HmB8qoedh/oCLif26OaOLqIyFSxK5IZnQmTTV9DaCdF+Q2lA/X9ZlSftRlaylRCzTaFamWVD3uOdgR8J03Q4qzjeLihH88UwmVnedskbbyieZCiTFGzIMgmTeqHVYO39rlCDwYDLMJbMwVnRfOgzlP3N8Oi80yQmrmp1zahJx5SSJM3ykRKofUN/lxSXI9PvJcsBQqq0GDbpN/6fBoH6vsDrMJr2e0+nyYIAbLQLq8HWdlt17qLaKtvSAF0mE7/c+JZP3FMvWrgOFnnxb9vpu6UoQxVtQ4jy0LwBtn0/D22wHc2DglFEgeCIcglyX8wTXXMVbUOBzzfICUYfrKRS2Pm7wDer+wy+yxZ+m5E+iqWl51qND0+UUfZaMa6qD89EjCW414hA0oXJvyPROR+EqEfBliPNp/T6bRd47tLwMwJnxufXbOFh4cSFTXjE/+XMy+GkCCFzSudwGThd1N+7CStJsgmAnYAtq6eP6lvEpNhDmM9XUspM5gUihJ4nSKJwKpxPUWDx6exelkqQXbZDP+OnR1s/t6mBytZmQtAgIPUbpMJtov0zOZnij/3B/jZKcEOhWCHPMk0d3sC33ei0+l4y4D599JFCX8RLms0Yw4K50WZ/fhzze0VrKuJc7Xh5rnm39UJ32Unz5ryXi1nRrDpEM/Ee008pLw+vxAx2BcTDyrrGziCZDbcPNcU6FYYCPwCQtUCBXNF8yBWso11lVhxf+uzFs2LnNQPq0BRptjhExlbU7XdBzr4zKIEPKr/nhPXx1TtemtAAh5aM3kvAbxzuIP7bxP7MzdV+BSM9aaqk/fUz//Q4N9vOlxoZTH9NU2WxYu9OcEnYTzfwOgNy8DtFWNoZWZdazSt/gfr0rSuHwOe21PRJdKNeLVPLPOu+77X/fb67SJgvbvscrm81/huBnB+wufGZ9dsIyNXGRy89In/6+g6x+i4j9c+avVDBJY331fbO+nZy/PiTOrgpSvjk76fCjO0NsNEtDYDy9Y0cZ1xZvg0ePVPLhQwNfw39rYz7lFR9RXRe27MvI/bxzUFpSIJrUmWNL2gg7+PApcV11knXtNELhbDYWeT/At9KqsFhClslC0EeP9YJ3aZKa2N67UjTQPX/f4a7oOAtntfu/l3u10KwF2NaEtrK86KwqMLCK9lU5YVJ5r491TzB5iBZ94prC2jjY6rvLG33RRKVhgIxFiWFsYFWBSGC0LV/Nq/in+errXUKvWo52s1I/La2kyLVP9zqiG22/zU2Yz4GWiAfH3j4LrWE4j3/sVbjf5nWASfxwe/K3cB/ncyxm8qdMo4+HNTw/HqjnvDyrMpYh0bQnoq3xkIpU34vXxm/6bqm/H5HUuTzftMxPatzaoPGPsR/OOsSGL9OOzi+dZnBdklvOOeTyzzrtc+idA/BJQBOJ3OJUC95btjQInT6Qx2Op0zgSygwXoNsAo48Ame/1e3cY9qapgeNVALsisCYnHoicas2p7BfAiySQGY/PWalT9uzKlNFsLyoTU5AawB4VSU8AGqrj4E2SUU+c873kTfJfMw82nw7pEzjHsmOz+tzbqPxj0qOz5s1Wl+gjGiIRa2hh/GsW6cl/e0mPBWSW6MyPWjMonv/UmaBOTN9Qvfax0APs1/WGnaZGx1Yqs/dU4UupADx+FPx7pM/LumbXjKa431kxEvdBertTVR6DlsU28wRRLrq7yqx/ze7fMLKet5bvUzTQVLToSiDPGxqTQ94MC29sPUkPU/jTEQnHNx1Yab5poHuAGRadcElkSbSIO0NuNwFMIYM2bEOOzKihOR5cnWprU59FiQqZ6p4tfMvT7BNBL+HvigsmvSoVlWnMiWlfPw6dekzglMfWCFgRRJQDO7D3SY92nuvCACuqaYYOv8+TR4+1CH2Ufjs7LiRL79uUU8tCbHfJZNhm99bvHfJJ/+9dp/m7LpdDpl4DkEVi8BX0QI9DaXy/WW0+n8EvAQYr193+Vy/d7pdMYAryC0/CFgs8vlmkzy1tvfunLWv+2sFs6xa/DADc44YNLVNFUIQSsN61o0vpLcGI61DARQOCdSEqdqDhtssYSDby3LZHTcy87yNsqKE5kTMY1X3mv5s9z1ifQxGfEOfy7FgPUAmpg+GMRhJ8mw+VanSSU1Wm5qOCe7zjPuSxIoEwAAIABJREFU1cTBpWpImp9qaNL/9L5J+tgbAux6HOyJbeK8GfeWgWXX4XNPvG7iv/8aWuaf441b+fHW3xrjOjH97sS2qTSd3QfaTZrsVPx/a5uq78bh+JdAHQbN0+jXxP4Z6TXcXm1SGoGp6IrXaxNjNKZqE+fmL7lmqmaw1MY9k3v41/Z7YrPbZDyWwZ1qTfwl/bbJAtYds/BBt23++1M2PxU8ffAHaFlbSW4MR5v6r6lN2y0LJwCzvY4g/3MBLAY/3m6TUL1aAM4X7JBZVyI49bIksyQ7mvKqnkn59MG/cKe6D8Cc8GD6RsYCPjME0lTBKUbfFVmwFfLnRlJ1cgiPT8NuEzRMI0/LtYJtJnLAJwYITdX+kt/A5IAaWZ6clsD4jd0mISGZ0NbE+fprhP7E51rvJSNYMl6fX4g4bKBp134nmwRIgbx6a3MocMcEzvvWskx2fOCaVJDjv9usQs8osGLkwDfy1ThsMl6vaqbHsM5rYUYENa3DAQnxJkYeA3oiNv+1RiGaqYKSQPjQAqLWJ8yTIVytwYM2SaTFfvtgx6SYBiP/v2Fp+zRx+B5pGmBu7IzrWiZTxQ7IgN0Oqnrt+S3OiqK6ddBcI4okDon8dJEvairFzK4HX+4sb/sfKaLySeCd/9+07sHLkwQ+BNLFYLKp7FGFoJs4vdfT3L2q30tvxeoUnalhbA6PLqitkIDPJ7j0o+Mqbo/X1LitAt8wQ41PNJ+Gw6FMMosNgW9o1J9ZlGAuVCNLp9GCdGzRq8K4F8bdGhXNg+bvJfxjYOTH0YMQzbapNN1kfIABnWmm+W4sNOs1dpvEkuzogO8n/gb8LA1rswp8w+Q3SvR5vBoleXPM/koTbmjsVwMDtjIsjJ8a2nLj6ZGAdWGdexUh0IyxKS2ME5WvfNpkBpDeDHqolYpobW4fvFfRFTAGo+NeZEWZkvrqsP/1W9g6l16PSFFhHHqGz8KtC3yjGU+xKRJVFoEPmLlrINAXYig/wQ5ZT5Nh+EOm9pnsrzlrsrMcdpl/WOdPb1GSG8PKouTJLyP7abfWfoI//7/P4jCPDg/F6/VdV+ADLLbASFYfi9drZfFIkyDHiuZBPJY1YrPJaGim1j/RN6IiWFdJMQIu3Fnehqvz+n37pO1TIfSN9udedn+tH9awK/6FaRWQf4lj0aYzZFSLOmxdeIagV2T/xrfLcP9tmSbDZyrrIzc1fFLKA68meNXGQrLJ4j2NvCUG5dTMZ1OciMMmmVikyEA5D0WRTWqc0et1JSmsK0nRA4DEp6tvSBGHF4Eay87ytgDt1KeKdyjOFoeOISQMQVtaGIfHK+iIEoFalcMuBQi9iUqVdQocNombCxIEM8km8FsQmHlZcSIOuw2JyVixCF5T9OI6/s+NR91zSzqpc6ah6vcycN2pmsHiKq/qwaOf6gdqewJot+Df7DYJvqIXwzGa9X3z585Gw79Ofr/vFB6vz7T6jMC7+lPnQFOx26TrYuETl6xNxsx749H56oZj0uMT1sa6khQ2lfpzDhnzc4+FKbWpNN1kl33udpE3qaXrfIBQ8/hgRUE85VU9oiCJ/p0BE9kVCNad7+NeGNMnQ9WLnPx+n9DoD9T30z9ylW2bC3j78BlzLgyLQZHEQaFy7T2qSH6uv/ETwWgT6yM0SDHjSkSEvujsxhXp5uEzcS0aB5sxZhC4LzRVRZL8fH2HTZpkJXtVGDw/avp0pofYp36Bv1H7VAj9+KjpbC3LDEhaFuyQAyL0rIFWMmKxGqHo1ok2KJCmc3MKI2pBeqRIeKZfN1FDa9bzmxtaf3FWFIpN4aU9LQHRqsYzjEdM1E5KcmPMzerTN6+I8IP1egUku2Kk3pXNCFxRYEX06/F7FojnelXGvYGMnd0HOkxB7vMJ4bZbTzRnbRMtGisjYSLWKcnCqrD6DTQmRmNqAYLDrojgmalYLWga//5GjS6YJPOwtCnwcU0P965IZ8yjIsuSya92KHrwmsc35eFq04WD9YD1aeKgm8pmvvumNFPAG0LI7YMF6X5tVpEl00LyavBfbwZanna7bB7UhlZ48uyICYkYWmvZ0mTKq3rISY1A1fSoZ6/GZxYFBhZa17NKoHapafBBZTdlxYnmQd/cecE8ZNw+Mfc7y9vYvb/dXL+lhXEBsErEzGDTBhy5NIbHp3FzfhyqFmjBGuU2774pXS9JKQUcPGMezXS+x0dNZ1Npuhm17PaKVNjBDoWY8FCujHnYtrkAuyNwJnwapj/Oq/rfP8guE+JQWFeSYkb+mmOCHi3vU8X60TRCgmzm2jQ0+p3lbbz+sf+9jXkyFCHhP5B57+gZc5yM5vbBmFuzWFKBDmLwRwq7vdr/SMK1T4XQv3jFzc4PW81/e1VBn9x94JSpIX144uyUKQPWLE8x/15aGGcurMqTooCGVxOYa7BDlP4zMhYavpnCjAhzERnLdCL2KDJG+gJ+A36BqeHXIqyb+UB9v7lZvaqfr+z2+iMAJQm9jJzYGOtKUsxFWV7VQ2f/JTaVppv9NTnJithoRvNpfopqaWFcgDZlxTeN8TI+m7jA3B6VggyL6Wz8admQS7Jj/VGmkrjXQ2tyAgbHmCO3D1RNMvFb0AOJfKLm7s7yk6Loth6AkxY/E0nvlCEIrWNqkzG/Nxg61mbF7u+/zUmIQ2FneRuyJJnjao1UFVlShYM7yObnatvtil5WTzHH6Utrcsw4CrsCiqzw0Jps80DZVJpuZtGsah3GrU9aWXEiB+r7AqAQQ8gE22Xuv81papdGdG9B+mxLNLT4f1v3RXOdGRkpNTRz/ZZXCetFRqzh7bsaJuWh/7ha/NaqoATZJN0SPA3A3TfNxasSYBkWZkQQZJd48Z0mdh84hdXV+Pg9BTy4OovvvXqC7bsaqDo5gCTJpmY8sTls/sDJ9TemgSSYWTabjXUlKQTZZcqKE801ZARHypLEbz9q1VNoB97b0Oi9GuYCkXTc0KuCFQS2sm+CbKJG77qSFFMBM+Ffu8wj63PITpnNS3taWFeSwvK8qWHBv2X7VAh9AFmRAk5gj1djzK1aND1dk9IFgDEw7xzuMA+G8qoeUcPTLiFJMhEzgwmyi0hKCQk0lfyMQEqZle0w0TdgFZzGEls+BS0PRKpZhw0+qu4OvIciUX96RJTm0zV6Y/PbdUda1KwQFJss0iQfPROgZVtT0BZmRJgpYG12hQdXZ5m/syIV5VU9k5zVDkWY17sPdJiCoKw40cTTrakVDE22rDgRH+ianR+CMeImbIqfVlvbOjjJWWgIV7dHDQg2Mv62tSyTB+7IZtOt88xCHb94q4Fxj15XwKLlG+PmcNiwKQqKBD3DowTZ/AfuRIvt1T+52HRrBsEOCVXTAjREQ2bIsp4NFb/1FWwXxWP2VHSxtkQE4Y15VF58pxlN0wRdV1HweH288E6TSSP9/d62ACFr0ydlT0UXq5cmk50igsCsmWEVRTYhkk2l6ZxoHcJhkzhQ36/7OyTGveIdR90+5iWGExKk8O7hDl1wygTZxbXBDomyJUmoTKbxGlbhqFsUJNq2uYBNpemEBCk8vnEBIUEKsq7dv3nw9KSguKrWYcY9IhmiqKermeuhs/+SqGer+xt2H+hAVVUzotbomzHmxVkx5nyJerg+VA0+uyKdPx7rZNyj8nF1tw51inWSFDMDWZb57IoMPR23mIdgh0iBbvRFRlCcH1mfExBo9rUNC3jwzvk6+6rDlCP3rcwETWL3gQ4z+6mid3TzrfN4+T0XM0IdbC3LZPeBDroHL/P3bp+KIipBDoWwUDu79p82PzOKe9hko6CChCyJ8oBrlqXQrBdPMGpiagjt6+sbF1DR3M/a5ak6HAPHmgdEXnxV1KnVENrGmmUpuCyFIAzIZ1NpOjcXxFHdOmg+19BsrUUvQE9TIENt25CeQ0dClv1FPGRJ1C2t02uJ2mSNiJnBnO69hCILrb+ypZ+H78rjcEOfpbiJn4Fjk0Wenf11fdhlUfZN0jRSYsNoOH1O4LbS1AwNuz5+/7Aux3wfRdGwKTItnRdMAaxqojzkzFA7nQOXzPqlwQ4FV9d5NFULFGiy0IZVVZSVGzw/Cpo2qRAGiMNGkUSt2DXLkjnTf5nVS5PYvb+dw40DVLcOWe4rks3ZFFE4RdVEP3yqxsmuc3xtQz5HG3vx+OD+lfNo6hihtes8XhV6hq5gVyQkSUOSxH2az4yAppnUQIOb79Uw53XhvEi6h67S2iXKGyoKZCTMEgU+zp4XFcVswl3u9oo+isA8cQ+7Inwwbj0BXefAZb2+q38wGk6fIzV2Bi2d5/jcbU4KMiJ58+Bptm0u5JZFyRyu76H5zAiyJLOqOImO3ou6/0jinlvm8u6RM9htMmtL0lB9Ki1dF/X3EQpOc+c5QKZB1+D7R0bxqRr3rkgnN202GQlhZsGRls7zHG8ZoLZtGJsicevCRMqWpGBTJD2RnmYWyNlUmk5b93lssrDWPnebk+rWIYLsEt/+fBGpsTN49f2TbCoVZTI3lYoavUYt5iA7tJ69iKaB26dRnBXF4cYBZOBU7yVCgmQaO0T5zerWIb5813zq24fw6e8o5vAc+2p6eWhNNr/6Ywten0awXm1tzbI0DtX3ommanrhP4on7F5MYNZ1/+10ND6/Jpv7UMDlps9m+q4H27vPYFJnOwcs8vCabWdOD2Ffby9YyUVjGKDC0riSFPxw6jQYkz5nOr/4knruiMP5vwtP/1BdR6R68bJaPAyEkPD6NTaXp2G0C79N0PNjt9QdTgH/jKjr1rL59iDG3YNkE22Ue37ggAB4whLeRzMvYlptK000ayc7yNp5/u1FQQTWRzTLYLou0CxaVemtZJt/6/OKA1LQrCuOx22xCs9HpoSGWahPjXqGJSwi66abSdMY9GjER09halmnCCQ/flWNqMF4Vs3ydoZm7fX6IqGxpMg673dR0rdGIkizSEw9fGGPMLcI/H7+nwIRODA1pU2k6//5GrWl2b9tcwNayTB5cnYUsSUiSRIhDwWGDxzcu4F++WMQDd2T5ITdN5ctrc3j6gSIzMZbRH00V1cnGPSrvHunA6/Wy+0AHYx4tYG6MdwExFca7CseojNsrspxKsmwWSFc1jc2fEekBNE1FQ9XXiai5Kkl+015kppTx6JiyqgnrqaJ5UKf/KjokIdJJOGwCC35kfQ42u8K3PreYbZsL+JJFg9yy0omiKDy0JocgmyhoM+5R2VqWaWrSQboj97flrciSTNSsEF7a08JX1uYwI9TBD391nE2lGYy5VVbqZTDH9ENlzCPW8tVxH1vvyKKiqY89FV0oknBsri0R8MiYWxQc37a5gG2bC0znpGEp7jnaQZBdYmuZKCjk9ogaEwDP7KimoqnPhHNKC+PMd3378BlAaFajbpXQYBvBDgVFx/t++5FYg9kps9m2uYCkmBk8/UAReXMjefqBIh6/p4An7ivkm/cvIsgum2U0VYT19l5FpxkQBjAt2M4Dq7Nx+wT0EuKQ+eb9i3hkfQ7p8bOQFTH3D945n5f2tNA/chUkMXfCwof/eL2GuvYhro55mRZsBzTS42exbXMBj23IR5bhkXW5pMfPYvsuEbMaGmwz13JZcSIfVHaj6dbH9l0NaKrYRzNC/76BWfApEfqGc8ifNlnjkfU5vHlQ5KYXkIRQY0tyYwhy2HSTUTZzZPh0q8DK5/b6VK6OecQm0SM8bTLmYhebUSLIDm983Gaao+tKUvjSnWJju/WDZnFmFOMeFUkSWobRLo96sCliYxdmRJA7N5In7ivkaxuEYAwJUoiaFWJuRuNaDbFRk2JmEOyQePqlCn73cRsPrM5i2+YC0uNnodhkvnFvAY+szxE46NFO3D6dCmiJlPzwRDdfXZfDkaZ+7DbJjEYMtsvIsswtC+JMISZJki44JUKCFP75C0XmZv3m/YsItks8fs8CQBRk2b6rgS+WZYIEj23IY8vKTF58p5Ef/qYqYA7HPQJD7h+5SmpsGCFBCjcXJAjBqZ8wNkmMpSTLJm4a5FAIcQj8PFiHn4IdEg+uzsZht7OpNF3PSKoSZIOrY15Gx33s+LCVZ3ZUM6YLokfW52C32fjSnYJ1s6k0HbdXZVNpBiAggDcPnkKSJB5Zn8PeGgH1VLUO41AEFLP1jix9DciEBMl8fWMBj64XwuGJzYXMCHUQGzGNF95uAoSWHRpsw+vz8cs9LciKgKCC7TI7y1t1XncGkmxQRzW+WJaJMymcR9bnmEE+j2zI57cftRLsUJiXGC5yLdllHr8n3zxAN5Wm8+K7zewsb2NdSQohwXY+W5phPiPYIU5Jg1ky5lZNRSHILiFLCuMejV+/b9QhlvjaPfnYFIUvrHKys7wNuyyUgPKqHr7/6xNMD7GzbXMBsqzwtQ35PLRmPi/taWbM7TMdmo+uz2Xb5gJO917kp6/X8MyOavpHrvKtFyroH7nKMzuq+d6rJ7g86kGSxP76x88W6PeVGHOr2Gyy8AkBP9tVR0x4KMEOmcc3FiArMpdHPWzf1cClq27uvSWd1/f603oAPHnfQp7aspBvfW6xqKuhkx+CdJLN6LhqZsb82a46NA1+tqueS1fdPHFfoVmISMypgC+/ui4HSZL43UdtPLI+B8Um89SWhX/3aFz4lMA7F6+4eW53velc9KlwS0E8aXFhVLcOmQJOQ8ArNhlcZ8/zwB3ZHGn054axQgsy4iA44Ro0GT6q/t+X187nloJ4DtT2ioNGE+Z+WbGoNbuzvI3atiHh3LPDIqcwSR02UXqv0BnNsWYBSxyq7xOBSBp0D13lUH0fB+u62V/Xx/zUcArnRfGT1+qobBng1kWJ3LE0FY/Xy+neS5QWxhEXOZ1jTQOMun1suCmNl/a0UHVygOKsGJbnxvLTN+pYlhPL/to+bIoItO8avIokCVO2c+AKX1iVSXJMGB9X97D51gwaTwszdeXiBJo7L9DaLerHfuPeAo409nKkcYCH18xn4y0ZuD0+Ri6P85PX6kiNnUHTmfM6PJZGekIYd96QwpUxD4cb+jne1EflySHuXZFB05lznHAN4rDJrL4hmc6By2b/jzb18tQWkdT1wxPd2GwKMiINxNayTNYsS+Vocz835s1hX10vPlWlpfMCsqwRZFd4YHU2r/zRxcZb5rLQGc2+mrPYFJAlheMtg6LGcd8lNtyURlFWNL/+4CS3FyVxoL6HgowojjUPcMfSZKpPDlF3ShTRkWUB+ciyxO1FyRxq6OMbG/NZnhvL0pxYKl1D3FaUxPzUcI409vO525y8+G4Te2t62VfbzcH6XspPnCV5zgwON4oApJNnz3O4oR+fKqCm+lND1LSdw6tqJgzi6hxBlmUeXpNNSV4cP3mtjsToaWzf1UBi9DT+7XfVfHTiLIosM+b2UdM6xKbSDI67Bjnh6qc4ew5FWdH8trwVCXj4rvkUzotmRWECMbND2VfbzZplqRxqEH6Cg/V95KTOpq59CJss0XB6BIdd5rEN+VS2DOD2ChjEp8LtRUksz43FmRROYvQ0jjYNcO8KAedIksThhn6W58ayv7abisZ+KpoH2HCTqNlbf2qI9PiZPLOjmmNNfZw4KWrZBtkkypakcKCuhyqXqGXs9WlUuYRz93O3OUmPF0n2CjKiuL0oibIlKdhtMkebenngjmxiI6bxcU03dy1Po7Qwgfio6SxyRpnCf8tn5rHjw1buvlFfb4297K/t5sb8eA439HHr4iQO1/cgSTKlCxMoyorG7VV5Zkc1kiTx+IZ8jjb2cbihj/113Rxu6KfhtICmNtyczoH6XkoLEyjJi+VgQ685TvFR0+kevPx3h3c+FUI/yKGwPC8OWRJFmdeVpPDbj9q5uSCeE65+7luZSVXrkAhSsks8sDqb2rZzJEZPp+G0yNMS7FD4zKIEs8iy3QZ2m8LWO7Jp6jiHIgts9OaCOH75XgtzZoeaxaKD7DKfu93JuxWd3FIQz6H6Puw6BVFFMnH8jbek89KeFupPDQdEOdoVGbdXY11JCjfkzOGEnpDqWPMANScH9KLkGkca+0iJncHvPhKayuneS+LQkDQcdgVX53kURUSMHmropSQvjtuLk5kR6iA/PYJjLYNmwXZVgy/flUPVyUEON/SzLHcOGQkz+flbTcLBKkFL10W2lmWyLHcO9afPMWd2KDVt53DYRO4auyLx3JuN1LUN4VU1ms+c47G78zjePMD0EBuvfdxORsJMM3DOsMTq2ofZcNNcatqGWX2DKCl5/8p57DpwivtXzqPx9DlK8uKIj5pORFgQDaeFEzDYodB0ZoT4yGnUtQ/RevYiG25K4+6b0pmfGk5d+zm23pHFK390mWUv0xPCqGkb4rG788lJm019+xD1p0fYcJOoMtbcMYwsySzPi+VAXS917ef4+sZ8pofYOdLYx2dXZJCZNIuTZy+gyDJP3reQ+KjpLM+NZUaog+//upJK1xBfXZfDT9+oY17CLKpbh2jqGEaWFb6xMZ+1JWlkp8xmb00Pde1CkBVlCcF7qL6PILsQLsdbBnl4TTa3FMQzLdhO9clBHr07jxWFCWzfXU9qrFBiirKiqT81TG3bMI+uz6O2bZgntxSyojCe5XmxPP9OE1s+M4/jLUNiDbUJyFKWNWrahvnoxFmKsmIIsisUZ8WY71OcFcMBXYj9470LWDJ/DlUnB3js7nwhSKvPcmNeLKd7LxHiUCjJj+OHv6lieW4sdpvMx9XdYj/ZJB67O58VhfH0j1zlcEO/SJJmV1i5OEnfHzLF2TFUuQbw6X4dmyJht8uULUmhKCuGg/rB6kycxYrCBCqa+zneMsjH1Wf5oPIsh+r7WFEYT5Bd4TsvH0fTNA439JOeEMY9N6cTNs1BkK7xhU1zMO72kRo7g7y5kWQlh/ObD1t56M5sTrj6GffA/NRw5iXO4vm3GvnCqkxq24bZV9vN8ZYBjjb2IaHh8WokRU+npm2Yb2zMpzh7DtWtQ2y4KY3PLE7i0lW3OYaripPJTg7nx6/Xcqi+j7S4MJ7+1Ym/e2H0T4XQBxi6MMov3m5mU2k671V04VN9VLqG2HpHFr/6Uwufu81JY8cwD9yRzcvvCS3w1fdPUlaciOvsRTbcPFfkwpf0KDoVFEmjqeM8qp4182TXearbhtBUaOkcYaV+SHhVjWW5c6g+OWhqBkuy57Cvtg9V1ZOFKfDFsmzSE8LYeEsGS7LncLSpD7si8+Cd2dx5QwrzEsNRNY1D9X2sK0mhpfO8ScMsyIjE1TVC1clBsxzh1rJMCjIiqXQNYbfJgMbmW51UugZRJImDdT0UZ8fw3V9Vsjgzmormfh5cnU1d+xD/eO8CkmPCSJkzg8bT51iQEcnzbzeLNA2ShKyIhHBbVmaSEhvGnNkCR3bYQJaEWd3SeR6HDe5bKbRSRdZYMj+Www091LSdA1RqWof1rIgKdpvQ1Fxd53F1jpgC36hTqmnCYTk67qOiuZ+wUDsv7Wnhy3fN584bUrhjaQrJMdP5+VtNZpm62vZzHGnsZVVxMgfqe6lpG2Lb5kJy0iJIjJ5GevwsyquEpnm4oZ8v3zWfoqxo8uZG8lFVJ7KisPWOLGIjplF+4ixP3leIw67w7M5qNt06jxfeadYFmcyjd+eRGif8HUEOhY6+i+yt6WXLynkUzotmRoid333UhserYtfN+YiZIVy66sbtValuHeSBO7KpbRui0jXI8txYCjIiqW0f5lhzP6oqhPLRxl721fQiyxKVJ8XvDtT20HD6HA/fNZ/0+FnMT5lNRXM/pYUJ3HljOnZZ4tzFMVLjZrIsJ5aY2aEsy51DauwMKl2Dun9BQdMEg+ZoUy8fV3Wzv66bkrw4wqY5uHTVTUleHCsK44mYGcIPf1OFqmpUnhwkOWY6J1yDtJ4VStE/3rsAt1fl4+oe8tMjiJgZwr6abmyKgNaef7uZ/bVC+AXbZSRZ5p+3LmFs3MOh+j6+fNd8fvF2E5om8tMsyYqio+8yG26aS0LUdNweHwfre8hJnc3P32qiplXs5do2Uc1ua1kmLWdGmJc0i9lhwXxUdZantixk1nQ7v/rTSZbliojtcY+PcY+PoQujfPvFY1S3DrGv9qx5UOemRZKRMItDuoXz0p4W/mFdLr/c08zGW9I53jLI/SudHG8Z5AurxH67pTCBRc4oImaG8OzOar64KpPXPm4nb24E33v1OIqs8MR9hQTZFZ7ZUc0TmwtZVZxsWhx/C57+/wp9ve2v7Wb9jXNZkB5J6cJEZs8I4rflrYyOqzR1DDPm1qhvH0ZDNbWszoGL/MPaHH6rb1YVgX+2nh1h3Av3lmZw68JEM7f8o3fncaSxlzGPRkl+HAUZkRRkRPLKH1vEZmrs41hzP8XZMTgTZ9Fy5hyyLPL7pCeEsX1XA8ty5zAj1KHXWe3hcEM/81PD+dFrNRxt7MemSJzqvYiEMKVbOs/T1n0eEMJ24y3CRF65OJFfvtcitMOFSRxrGmD1DSkcbexj1ZIk6k+PMD81nNsWJ/Hj12q5Ou6jrn0IWZYoLUzgX16q4HBDPw+vyeaFd5rZtrmAeUmzqGs/x6Pr8zjWMmDS41Jiw0iMnsZCZzTVbcJSsSkyD62Zz6vvu5B0ZkrVyQEeXD2f2rYBHlqTw+GGfj53m4BkypakkJE4i4VZc/jj0TMmG6S1a4Qxj8qX75rPquJkjjb1ccP8GHbtPy3y0nzYSlzkNGJmhzLu8eFMnEXzmfN8bUM+J1wDyLJEQUYUa5enkZ0cTmqcqGL249dqWZwZTUleLPtqzuJToaZ1gCONA6QnhHGkcQBFhiONA8RFhnK8ZZCirGh++kYdn7/dya/fP8kdS5Jo6TzPPTcLJWF5bqwpSJ7ZUc3dN6Xx6vsniQgL4qU9LaamfsI1xPLcWL7z8nHeP97Fofo+Hl4zn5ffcyFJcP9KJzHhoTz3ZgNPbC6kJC+OI439PLI+l+Mt4mAf96g8vEbAFR+eOMvouI+c1Nn8bFc9Fc2CXRL6AAAgAElEQVR9qD6Nww19fHi8kw+On2FfTS+ZSbPoH7nKj16rYVVxMjGzQ4mLDKWpY4SntizEmTSLhtPDPLVlEQsyItlb08uy3Dlcuurm2y8eo6K5j5K8OM5dHONIYx9jHtWE3f7x3gVC6BXEExsxjR/uqEJG48TJIRZnRnOwoQ9NU6lrH2Hb5gLzALkxP479tT0crOvhWPMAD6/JpnBeNPlzIzjU2Mej63P5w+EOVi9N4vW9p9hXK7R4TcO0vAxrB03UU16WO4fDDf0caxZz13BqmIiwYF7fe4pgu8SN+fE8/Uol7x/vZG9NDwvSI6luHeLhNdlsvCWDrORwfrarjvy5kaTGzWSRM4qkmBnsrenisyuzWJgewfPvNPH1e/IpnBdNYvQ0fv5WEye7zpM/N4L4qOkEORTSYsP4xVtNaGjMS5zF4YYB7l85j5y0CIYujHKwXigkoLMM/0aY/v8KfcSJ/v7xMxyq72NvTQ8H67qpbT/HjfmxbLwlnbnxM6luHcKmSCiyTFFWDJUtfYx7BP5fWpjA/NRw6k8N03JmRJRQ0zWvyhahIde0D4vrTg6yemkSO8vbqG4dorpVmOybStNp7hxh1K1yqL7PpJAVzhM48Z03pLCiMB6A7/zyGCV5cZTkxXG0qZfatmG+WJZpmtbFWTEcbeizBDFpKIrC/Svn8ebB09x9YxqZybP56MRZTrgG+Liqm4fWZDM9xM7+2h4aTo+wqVQ4rZblxrIgI9I0qx+7O4+Ry+Mc1nHcO29I4fbiZLoGBF/66xvzAdhb08PRpj72VveQFhfGf/2hUZjqOjddliSqWgfRNJEAzauKovBVrUPIssyq4mSONPZwvGWIo019zE+ZTeTMEFITw4mYbqfx9Dk23JSG6+wFvnzXfF75o4tlubF8XHWWtp5L2BUjdYVeoetEJ/vr+mg5cw4kWLs8jeLsGAoyonhmRzVxkaFs39VAZtIsZoQ6+KCyk8MN/SxIj2RfrQhuau+5xNc35uOwKRyq72PNMmFRNZ85Z9IP0xNm8tKeZtDA1TWCTxUWiMerEhcZys921bM4M5r9tT20dJ7Hpgj6X356BLNnBAttOzeWqFkhLM+NJT89gtuLkpgeYte1+yhefs/FwfpetukO3qhZISzTcd/8uRHcmC/WRU3bMKULE7h1YSJxkaG8tKeFr2/Mpzgrhn21vdy/ch6P3ltIWswMDtX34UycJSwhWaI4O4bvvHycY80DeHwa81PD2b6rgftXOslJi2Dc7eNok4ABASqa+3h0fR4/fq2Wj6t7cNhkHr5rPq/vbeehO7NJjgnjR6/VUNE8QFFWtD6uggrpTAonf24EB+v7ePK+QgCe3VltrqvDDf18dX0eJ1wDVLcOsjw3jiC7QlFmNA67woG6Xk73irlZWzKX5DnT2XhLBit0TD5smoPlubHcuiiRZblzmB5i5+Pqbhw2aOwYYXTcR8Ppc2wty+S+lYKZlRYXZlI1n3+7mUfW5/LCO80szozm//y2mtFxlYrmPvLnRhIfNZ2hC6Mcqu/joxNdLMuN5XZdO794xc3ssGDy0yMoyorhJ6/XsUw//H/4myo0NJ68byGxEdNITwjjNx+0kjc3gmd3VvPVdbnMCHXw7RcrWJYTa8JNn7T9r9AHjrf0U+ka4uG75pOTOpvmzvOoPpX23kskRU/n1fdPsq4khXtXZLAgI5L/3N3AY3fnU6VDMj/cUUV16xAP3JHF4YZ+7rl5LnWnznH/bU5aOkeoaR3m3hXpvPRuM59dkcE7R87w0J3Z1J8awq7IKDK4Oi8gSfDlu4Sjt+rkINWtg9S0DvGNexcQMTOEILvC//vLCkbHhVVQtjSF4qwYDjb0sjw3lurWIYqzYvjZ7+sY84jEXqd7Be/94TXZ7Piw1dQ689MjhMXS0ItNkaluHWR/bQ9jenDJ8rw48tLE4rulIJ4VhfGU5MXx49cExgiYLJChC6M8s6NaZ/oobN9dz8NrsllVnMzy3Fj+z2+rUWSZDTel0dZ9iUfX51G6MIEDdb2MuX3IsoZdEcmnxj0adpvAZot1GEvTNPbV9LLIGcW4V+NHO6v5wqpMdpafZHRc5c4bUliWK6pWHW0a4O6b0vhiWTaXrrp57s1GbBL8yxeLmJ8aTkleHKWFCQTZFf71lePctTyN/PQIXn7PhSJDpWuIrORwKl1DbNtcQMTMEPbXCStweW4sqXEziZwZQmL0NH6//5QOaQnYan9tj45By3x76xIWzI0QuLtN+F6OuwaRZbgxP46D9SLITJLEYXF51GNitucujhE5M4Rxj49nd1aTnjDTpDZWugb56rocVhQmcHnUw7M7q8lLiyBqVggXr7h5+pVKVhUnU5wVw+GGPhZnRhNkF3Py0J3Z5KZFEjkzxLQuirJiCJ/uYH9dD3fekML81HBq24cFppwSbmq46fGzCJ/h4NX3T5IYPY2f7aoDTeJgQx+HGnp5dH0ezqRw86BaW5JGSmwYeWkRPPdmAysKEyjKjOZQfS8HdOiwyjVAbfsw+XpdhPePn2V+ajjPvdnAP20qMBUJkUn0JF9dl0NlyyCLM6P5zsvHBHGhoZcn7/NDIB19F/nJa3Xkp0cEQCFBDoVxj48gu0KQXeFAXQ/fvH8RJXlxFGVFU9nSx13L04iaFQJA5MwQMpNmkRwTRnnVWe5ansbizGguj3rYV9Mr9khuHD99o4602DB+8nodGhqPbyzgx7+rYUVhAuMeH996oYIPKrs41tzPsZYBNDRKFwqqcHlVF0/et9AU7LctTjK1+6zkcHPcVhQm/E2ZO596od89eJkfv1Yr0iTkxbF9VwOrlyaZVZWKsqKpbh2ipfM8hxt6qHQNomoapQsTKFuSQtSsEObGhVHZMsiGm9NJTwgztZu7bplH7KxgDjf0UukSqYirW4fY8pl5ZCXPpjgrhpL8OA7W9zHq9mHXhW/pwkRK8uI4WNfL1XEBSWzfXU9WcjhrS+aSnhBGbfswpYUJRM0KYc7sEFMbee7NBu5dkU7LmXP0DI/ykA4ZJMeEsaIwAVXTOOEa4NaFibpmGMmamzJ4v+IM6A6xkvw4ImeKxb84M9rUuoxn1bcLrH1VcQqXrrqZEepgWe4crox5+MlrddhkiZq2IQ439JORMJPDDf2m8/MhHQ66vTiZ+SmzqXIJnNVmEwwZr0/jH+9dYG7Y4qwYbswXGzNiZgjf/MURvrI2h1+83YSqiipD81PD2b67ngN1vTxxXyFpcTMJm+YgbJqDiLAgTrQOMWd2CL8tb2VvTQ8VzX1kp8zmuC5AUuNmkjc3grKlKSzOjObHr9eaNEkDq/7hjioO1feRNzeCsGmCPrk8N5b5qbNZWzKXIl3I2mTYdOs8nn+rkZsL4nULcIStd2TRcGoYkChbmkLpwkRS5sygtn2YokwBC339nnyT6WFYHFnJ4fz8rUa+sXEBxdkxzE8N5+dvNXGgroe9NT08dGc2/7m7wTz0PjzRRZHuYDU0xhWFCSzOjGb77nry50bQO3yFV/7oYstnRD8XOaMoyIjkp2/UseGmdEr1qmcG5vzSnhY+rOykum2YYLtETfswmibiMJ68byEL0iN57s0GluXEmtetKEwwIYm02DDTIpkbF8bemh6qTg4iyzJfLMvkP3c3sDw3lqKsaF54p5mvrM3BoWPaj6zPoTh7DjcuTGSaQxH3tSt8XN3DE/cVmsyWcY+PIIdiCuvYiGmmZtw9eJlLV9386yvH+biqWxxAWdFmX7NTwjnSOEBFUx/L8+IIcih0D17mp2/UmYdV18AlfrarjsqWQR7SD8Ef/qaKr64Te27b5gIKMqLYvquOf9pUQNSsEIIcCnlzIzhY34skSTx5n4DiomaJA/3j6h5WFScTNs1BXpo4pObGibF6dmc1/7SpALfHZx7o/xOa/qcmn3734GUz8OGJnx/GpihsvGUuUbNC+M/dDXi9XsY8GiFBCo/dncfPdtUhSzLbNheYp/RX1uYQGzHN/Ptzbzbwg68s55+2H+DqmJdH1udwdUyElb60p0UEUCk2vvtgMZeuuukfucq0YDvP7KgmNEjh+w8tBaCufYiX9rSYecK3bS5geoidGaFCqLk6R3hmRzUhQQpPbVlI/8hVtu9q4JH1OUwLtrN9dz0+n0gqtrUsi+27GghxyCiKwrbNBTy7s5qnPl/Ek//foYAx2VqWyet72/mnTQXms7oHL/PtF48RZANZD5AZHfcR4lB4bEOeuUljwkMDAkl6h6/gTAqne/CyafJeuurm2Z3VfP52Jy/taebR9XnERkzj0lW3+ZtvvnAUTfVnLnlqy0Ke3VnN0w8U0zt8hZ/tqmNrmWDcfGVtDv+5u4EvrHLy8nsuvvtgMWHTHFy84mbbfx1m3KMS7BARpI/dncdzbzbw+dud/PK9Fp7YXMizO6vNOfzWCxVs21zA9149zphb4+kHiswD4Jkd1Xz3wWIA2rrPs31XA08/UGRS6r7/6xOMjvv45ucX89PXqkHDPIw/f7vT/D3At188ZhbGMMYkPmo6VScHSI+fxbdeqMCn+hgdFwFXL+0RPPeQIJnH7s4318FTzx/h+18S6+Wp548iSxJfXSfeBTDH4clfHDHnzJjfpz5fxHdfrkBC4gurnBTOizb3BMAPd1SZicAkhF/KuK/Rvv1ixaR1Yp3nb794jGnBNr73pSXm90b7wW+q0DQNWZZ4YnOh+cwnNhfyg9+cCNCEVU3jew8uMd/H+mxDSBrPNT43xjkkSOapLYvMdfnNF44iIbFts/jN939dyVNbFk26fkaogyd/foRRt49gh8zmW+fx+t52vrI2h2d2VLNtc4E5Ht984SgA33twiTnugNlXo79PP1A8aZwMufHMjuqA9WaskefebDCv+6Ttevn0PxWaPojJGbowitvj40hjP19Y5WTHh62sWZbKisIEE2Z4astCUuNmsjw3ztSAF2dGs0zntBumGMDtxcmEzwrlhqxokw2wfVcDrq7z3L9yHs1nLvCFVU5mhwXznV8e41BDP7cXJVGUFc1x1wClhQlcuurm+XeauPvGNBMS+sVbTfzpeJd5z2d3VvP1e/JNmOmEa0CkZmg/R0VzP1vLBOQkS1DbNoxH16RXFMYzI9TBisIEMtMiyU6cyYrCeIqyoqlyDXK8ZVBogu80maZqkF1hb3UX417BwFhbIvj0hy19T44J419fOU52cjgzQh1864UK7lqeGuCIGvf4+O6vKvnK2hxeeKeZR3RtydDighziPwPn1VSRC2lFYTz3lc1ndNTND35ThYTEPTens6IwgRmhDpJiprN9VwOKArcuTDTn9HCjoO8tyZ7D2pI04qOmW5xoULZEmNPP7KhmWe4cVhUnc+7imGnGx0ZMM/teXnWW7ORwfvCbExyq7yfILnHH0lRTCyvOiuFYcz9fuDOHlOhpZtzBisIEUmLDWOSMYkaogyC7IhgyOqPHYIhkJs3ihXeazbV068JE8tMjeOWPLh7SmVoleXH8+PVa08n3cXUPWcnhJn0yKWY6P3mtjv113ZQuTCTIoQjMuUHg7lUnB2k+c14ExsXNYmlWNFnJ4fzktToWOaPoHb7C0786wYrCeFYVJ3PrwkRuXZhIUVYMP32jznyWMU/LcmIJsit8+8UKExLMS4vgOy8fpyAjkqqTAzy5ZaE5twDf/VUly3JjOdwgnLErChN4dmc1y3JjOVTfx6riZIqyYkyr4Z7PZLJU186N5xrPjpoVQl5aBD/cISig/7e9bw+vqjj3/uV+IwkhBAgg92YSJURCuViiLWBbwVMr9mg1tP0+66WfSvvVXuRej4KAfD3VVtTTEjzCqXiqrfi03upRRLkm5AIhgmO1pUgCGBBIIJDEZH9/rPUOsydr7b322pfssOf3PDzsrLVm5p3bO++8lxmSnB/dVCd2ENX8hJCq2zu7MMts35SkBKzYWI2Fpjst5U/5tnd24Z29jfju1woMg/BLDbj/ZsMNtWTcRdVVVkYySsbm4jtzrsCnJ88a9TP18Cq9tGjRN1kZyZg+Pt/LQ4fmyxdZnvCqItqCRcyrdwAICXb3AWMA/u4vB7CgvFRsrT5qPI29H53EnGkjxUXJKUkJXqoP6niavGkp8XjyT/WYMdHwBV69qdbwzS0ajGdeO4hvzxyH3/75gOEL/rHht5uclIDc7DRcPtKIlly9qRa3zBiH/3rzQ9x/cwmKxwxEydhczCwdJgYGDZasjGSMNY1PSUkJpovaZ0LltP/vp7BwXimmFA1CvzRjR/F27RFMKRyE1NRkdH1u1HVgdhqunjBUMJoHbpuIlKQELKnYjSmFg7B9/zEkJcbhxquNs9MHZKViW30Tyibk47EX64WRcuveJtO/33ANTElOQMu5DrR3dolJOXpoNiaMMVz2SPe8tKJSGLry+qeZjP8oEhOAa0qGoX92Gj49eRY7GgxVDk3Mxet2Ye/fTpq+7WMF46o8eByL5pWKIDBaLFc/V4uu7i4s/s4XkZKUICYcSYK0mI4cnIUlFbsFMykZm4vH/1iPheVGW9Z8eEIsiksqdmP21JGYNWk4Ors9WLlhD+ABZk26zEtCW1pRibdrj2D21JGiXfL6pwlXPmoLYlDD8vphwphcPLm5wUiTlCDUFFkZyRg8IA2PvVAvVEKPvbgP999cghvLxngxmPvmFiM/NwPXTrpMMLwHntyOGRON8VQ4oj9ys9PMQCIPrr9qtPBXJ2YzMDsVa19q8PIXp/dj8rNEn54934mtdU2o+7AZcXHxuHpCPlZsrMaEMblISTLUNHn90zDBbE8yfpJRmtRzxBRzc9JxuOmMYJSkzpGZ4Du1jZhcOEjQ9VbNEexoOIqZpcMFwyepmtqO5lCHOS4JlC8JH+teOYAZE4eh8uAxzCodjhUbqzFj4jBcJ+W7YmM1ykqG4aFnDOmc7AOkmmk51+HVZlQ3uTzKS/4bMASlUBl0NdOHoX7Ysf8YEk0JcXv9UXzhMsN4lpuVYgQdxQNXjB6Alb+vwZaaI3i7thFzpo30GjwA0NrWgXfqGvHB4dO4Z24x+vczOs2YpMPwyz/sxfn2Lhw4ZEQR7v+7ESTTLy0Jy9ZX4d29R7B171HsbGhCHOJw4J+ncf8tJUIFsGJjNa4zmQUAMZhSkhOQnJiA9+obcYcZZHTfXON8lV+9sA/3zR2P3Ow0rHqu1mCY5YY+9NFNdfjL9r/j7dpPUDJ2YA9GI3smlBXnY0fDMSyaNwkdnV1Cov9m2RjTN/qokAxJiiVm2dh8Fss3VOOtmk8wdmg2Hv9jvZDGaHLmZaei4dBnmFI4CA9v2IOxQ7OQm52GLbVHRNDYqzv+ge31R8WRAgAEfR54cGPZGADSDsg0eD72Qj1Sk+JxTYnh+fE/1f/EhQ4j0GnN83VCWl1SsVtIgbQAEIMl6YukwIHZFxlUe2eX+A4AFv92F7q6uxEXH4cppuuqkPaK8zGlcJDQ1dJCl5KUgMXrdmNbfRPeqWtEydhcrNhYjTH5Rju8VfOJWECo3JZzHXjsxX24+xuXe+0QSChobD6LvP5pGJxj2H3erj2CWdKY/caXxyEp3mBMJFUXjczBu/uOigWSxtjidbtQdbAZqcnxuP6qUWIMtpzrwKFjLcIW8asX92HPB82mN40RuUzjSRY2iLHT7oAWT5nxURmd3cDi3+4Uem4r5j9hbK65WzOkYvISol24Km0TSFCzC3ySde7kOVQ0Mscy35z+6Xhlx9+9FnTaAcnSP5Wrlkffq8zdim630EwfEN4M875aiJSkBLxV8wn2mdI3+dmWFuTht38+gPPtXeb57nGYWjTY0shSMi4XE78wEJPHD8XP1m4ThjTACFU3Tt87hbtvuBz7PjqJqg8+xeypIzG9eAiuLhmKXe8fQ3tnN37y7SvxzbLRhjqABrbpqUGQBwkATCkajLWb9wsD0+TCQdhWfxR7PvgUUwoHYWfDMcyfW4zRprFz8IA07P3oBDwegzZiNPI23XAfGy+YOHm+dH3ejXf3HTVjCPYLyfvEmfNiq0x+6Ss2VuN/z2aoPngc1R+eEO6GM0uHY9ak4cjLThW+6uQ6SruFqUWD8e7eJvzghitw29eL8GblIexsOI7CEf2RnGhszxeUl2LOtFFe0ltmejIe3rAHe0zj296PT2JHw1FcPjIHNfyEWExpMhHjJkmbJEJa2EkXbSURtnd2CYYPGGogeIAffWsCHnuxHm/VfGIwDNOLhBhAa1sHtpuh98bvJmH0G5bXD4NzDCl+3PAs7PvoMy8mQ+VPH5+PUflZoh70jh8+heUba4TgQj7rsgQ6dHAm2to6vJjKwOw0seuhsWW4PQ4VAYJUNi1aVR8YMRvFYwairDhfLDxyXUmKn1IoLbQmrUWmOtCOQebmpOPKMQOEOoYOjKNdGH1HC8ryDdXYVt+E6g+avaRuuc9kvFffhDnTRlq+k1UxgLFTqzx4HAvLS73aEgDS05NxVdEgsXDJQgL1N9ni5Ahb2gUDsPXW0S6bIWT6/PApEfhk6K2bMH9usZCu1zxfg50NRmBIwz8+w4++VYLdB4xzZMYMzRID+MSZ81i+oRpVHxg67lHD++PKMcY55g89W4Vt9UcBeHDLjC9gcuEg5OdmYNak4cJbggbQO3sbxQSSt3XtNhIODcZl6yuFTvSbZaMxuXAQMtOTMaVokAj9n2x6ihC9j26qw8N3XYXpVwzGtvomzJk2SqQTW/AxuUIyJ6+EkrG52PX+cdx9w+V45rWDaLvQhSlFg9De0SWkJqKJ7B6/+eM+XOj04LtfK8CIwZmCqQIQLoWDc9Lx6KY6LDJVUU+93IAbpo/GuOFZePZ1jm/N/AKKR+WId7SgkmQrS4gnzpzH7KkjMblwkHAnJE+ZBeUTvSJk6f8JYw3d8CyJ2ZPEvGx9Fd7dZ0ShymXRwju5cBBWbKzGzNLh+NKEYXij8p/i7JSdDcd7SIVkzCZ/7BUbq7GwvNRgPknG4kBS/IY3OBaUTxTPVSlQ/h+4KLnf/Q3DVZd85eXxs2x9JWZNHoFu80ovlamQhEvjksZLybhcMeYBCA8X2YPHSo1BTFBW29AOYqsZnXvD9NGCQcqMLyMjBV1mvWk8Ti4cZB6ncLFdy4ovLi6zTDWWzPCtQOmIIRMDVhdWGgsTxuZiR8NRL2l+aUUl3qr5BK/vPCR2LLIwRuOR7DZPvWzYBkYPzUZj81k8/OwevFVzGFvrmkR9woWYZ/o0OagDqFMpiOKjxtPY2fApvj+nEEUjB2DWJINJ/U91I77ztQLhOQJASMQzSw2j6uSiIeiXmmhG1x0zS4zDqCGZeHJzgzAKnj3fiVXP1aJkbK5wa6t45aAYaDKzJZ2ovAWU9eTErMnT5K0aI6LzvfpGbFFUUgCwrb4J5dcV4cjxFiFZ/+rFfdhScwSzJl1UaZDEL0tfZJwsGTsQu9833FLLivNReeCYkLop3XVTR+LaL16GccOzMPXyIV6MjyZ5UmI8Ht1Uh25PN+ZMGyXqAgC/emEfuj3duObK4Xj4P6uEkR2AYGQnzpwXvw8da8HyjTUoGXexL6nOM0uH99Dh0lhQ9eWErIxkXDYoA3s/Oinc/WQpWDYopiQlCOM47XiIVnlnQBIr7chIp00Mo2hkDm6YPlr4u5Onzjt1TcJ11It2ZdGjHcCEMblYu3k/ZpkLJI2l6ePzMXRwJo41n+2xYMhGWepzK8MiLdykp5fbRVVBys/ld2XFQ72MoqquHjCYfpt5WiW1N5Upe8modganzFNmyKSCkncQcj4U7OW1sJmLzM1fLURSPLzGt7zTkdsvM93YXS7fUI1ujwdxcfFitxxKF00VMc/0yQBFfr008YkhP/ZCvXG+y3sf4x1ahZMSsHXvEbx/6BTmzy3G2s37sWP/Mdw315jAJeMGourgp9hS/YmYOBQuv63eCOC5+4bLhU59x/5j6Py8G5UHj2PIgDRUvHJQ+PrKW0tiFDOlyUuTaUx+Fn75hzpsrWvCiMH9hO+wcQCVYYCNAzBr0nCvwTpkQBqGD87Cv62vxP03l1yMjmw46mWAJYYm66Fpgra2dWB7w1HMv6kYudlpeKeuSRjUZKkuJTnBy91P1qXStp1opslC9aRJNfqyHIwd0g+ftVzwMmYfPXkOj/+xXhieaSE3VEWNuHbSZSJYpmhkDpZvrEHhiP4iHkGW1snoJ6Ox+SzWbt4vDuZSg2ZoN7B6Uy3eqW3EtZNHICPFexewfIPhsSIzZdJ1y77t7Z1dXt5EALB8QzVGDO6Hur8Z0dckVVP/LKnYjZKxuXh4wx7BrEi1RgvZZEWtkpKcgM5u4Gdrt/Vg0rJKQl38ZBdEmdFTe8hSvLojlRm0XN7A7DRLDxX6jpg+/U19OVMao8EaOrfVN+GbpqfVrFLfAVGyPUNeYBISE8TOSV7YafGnPElAobl89YShmDNtZE87j0VbBIuYZ/qNzWexfGMNttU3CX120cgczCwdjtFDs1E4oj82vvkh4hAnDkIiPWx8XBxmlg5HWXG+iAicMMaQlBfOK8W/XDMOJ061iS1ra1sH5kwbhXHDs7DhDY4ZInjHOHKhrDgfj71gBOnkmhGZNIkIshRP0uuEMbn49xf24nx7N75lnnVy9zeM8Hbygikrzvdi+ICh1nrshXpMKRqMr31xuFcEIw16eVEhFQ8tirPMbezyDdXweDzC1dSKuQDeg1b2WCH/bmI0J86cF37zpIqiRaOzG1jw1HZs3dskmPb7h07isRfqRZ1lnf5Dz1bhfHu3UN29XXtEBOfIjDMlOQGDc9KEtClv72nhJbuBarxXv5k9dSSGDs7E3/75mWCccptZSY8zlfa+YfpolIwzPJzaO7vEsRCJ8caZMrKemmwRZaZqT/Ymol2LrKqjdI3NZ4UKUmVwsrFR/psWmC01R7Cl7qJRWPVEMdQ2u7Gl7qLELOchM2iZkdPiJbfF9PH5GNA/Hceaz4q/aQ6SQVhWc6qM0QmzJBWP6grjLHMAAB3oSURBVK3kC2o9GpvPYunvduFL44eIPEgVJS9GNFbILrF8Q7Ww61A7qGMlFIsaIeaDs0inSMfeUrBTemoiVt41DUdPnsPazfsx33R3W7xut3FFoseD719fhGdf5/DAIyI4AcPQc9/c8XjipXq0XegSgVIUzPHUyw0imKjb0w2PxziP5pE7p4kI16UVlV75yn+TCmTlXVcJA2NmejKOnjwngoDI3ZKCmay2wBSw87tFX0XnBUN6a2w+6xXoIj8jmik4ySr4RJYE1fdLKnaLwBUKUpF/yxNdDrSR8xk7KhcfHzrpFfC1bH2VCDZSA1jkoCfAWOgo0EWmkfKhYBuZJqs6ypCDbOjd8TPtWPT0DhG4Jeeh5mlXltouNDasaKCy1fZSy6M8KfBn7c9mIN28l1VOK7eJHFjlry1keijIzepkSDUwifqDgsvk4LqsjGTk5WWiubnVq21INSYHO8nj3a591TZzA7W9qR6r7i0T7emrHJlWgjpurZ6FwnvHV3BWTDB9wLsDAWDR73Zh8XcmATCi+ebfNF4w0VXP1eCHN12Myr1vrqHPN07xAx65y2DcqzfV4se3TMQjG/YgJSkOiYkJuH12objEAYDXJCYmJtNEhj55YMiggb5sfZVgLmpd1LJoIsoRs2NH5YoJRZOJmLsvxm01iYw2q/SimZj46udqBfNQ6VSfWeW9ZN1u/MfCa8UCRSAGwg+f8mpDFeqCooLyoW/VCUdRnHK7WLU/ADz4TBX+zzevEIFd9HxpRaVYNNVIVit6A5nkvr63atvG5rO48vJ8r76XGaYsUNjRB9gzK7v+tCvLLk8AgukT1IVWXWTUxcRqPMkLoN2RxXZj0WohkYUSJ/3mr7/sFqtg4Yvpx8R1iQC8Bk1rWwcS4uMFU0xLjkdGahKW3zEVmenJiI+LR35uBhbNm4T75hrMkXYCkJrS4wEKR+fi+3MK0d7pwe2zC/Hs6xxLKnbj6MlzWFpRidWbjGv/SF0jh6dnZSRjWF4/MSizMigsu1K8B4zrHuffNN5LApSZb2ubwSyXra/C0ZPn0NXdhVW/r8Gi3+3E0orKHu3wwG2GpHvx8AN4lSdL6i3nOkRZ1H4AvCT41raOiwtAHMTVcXKby+ntJkJrWwfOXfgcZ86291jQ6AgEtQ1VZGUkY/kdU712MGo+an3lvxeWl3oxfEojH8NA/bP25zPFjoGet7Z1iHaltl7zfJ2ov/y/3EZOQEzCKh/5nUq72jay1NpyrkME/sl50f9LKnZjaUWlaEfqwyUVu70WDZkutSy1rckO5K+ua56v63GTlAcer3lAZVjlR+9px2M1btQ2taNffn66td0yjRV81dOujHAjJnT6BNKjkYtfR2cXVj1Xi/PtXdix/ximFw8RBkUAwjWTLjkAIP5/eMMexAEYNywb//naQRFNO2FsrriGcEH5RGEwzMpI7nFBAukhZa8CMlTKLmikl3+vvhFlxUOF5wcZiMhlky632FLbaB6WFY/5NxVjWF4/ZGSk4FjzWS9DsWzIogkhu1iqnhoAegT+UDAOBbLcMmMc1r1ywMvriNLI6a10ltRGQ/Iyexge5fcU02CnxyVdvZWftJXtQe4PsveQ8Vcum95T/+TmpKOrs8vLIEoRxnL/ye2o+qfbGfSsoLpGqp4ydkZU2SvGykVRbQNvv/18TC4c5GVfml6c72UPkttELjtQvbRKp3zsguyuKdPspM2s5p5Vm1q9swLFE1i1daDoDe8dV5I+YyyNMfYnxtg2xthrjJkO297fPMgYq2KM7WSMTTGflTLGGhljW81/33ZTvlu0tnUIKfbRTXVYvakWP7ypGKnJ8Zh/03ivOyqzMpKx4s6peOTOaRiW1w9HT57DsvVVQiJ/5M5puH1OIVZu2IN7bxwv+fvXGYYbU9qUBxNJKCQ9y5LW8jumgI3I6aHmaTnXgbWb9yMtJQGL5l28OJm+e3RTHRav243Vm2rFFn3RvFLEJ8Thh+YhYC3nOoR0QlK5LB2R+kKW2OXdhrx9psOwiLbM9GR0e7rx5OYG82C1D3DvjeO9JDxKo6ZXQW3RPzMFD9w20fI7Cu4h6bKx+aylxNvYfBZPvFSPtJQEZKZ7S7a+pDs6EEuWbOX38i5Cfk7/0xiQy1HTqtId7QxUenxJklZSotwvdpKoP129lYROfWxXZ7lN1LJV6drprgYw+joO3lqKQKViKs/XbVRuJG0nbR2tcKveuQfAfs751QA2Algqv2SMlQL4MoCpAG4F8KT5qhTArzjnXzH//cFl+QFDZmzE0BeWl6JfWhIudHRj/asHUfvhp17bQGJULec68NTLDWJhAIwFZMMbHCvvmS50zLSdt9puysyIGIE8uWhBULeixChX3X2Vlx6T3q24cypW3jUNC8tLvdRBj9w5TSwiWRnJgpGSqkFGZnoy0lLie2ybZcjbebk+gHH07oo7p6K0YJA4UVKGFWNSIU+e063tgnFagZiMYX+pFbSptMfHGVcSqgulr2210WZTvNraH6NU6yovqlYqDrU9VHWa2h5WbeSLFqv+s0pv175W+cbHxaO1reeCqY4Juc405+QFNBAGSePbLeQ5Fw6o9VXfOUnfG3DL9MsAvGH+fh3AtRbv3+ScezjnhwEkmruBSQCuZ4y9xxhbzxjLdFl+wJD1si3nLhpQAeMYW3g8eOa1g5aeCMTMN7zBhQRH3i/jxw0U38lSngrKg5iVqkMlxpWZnoz01EQvI6y6iFh5A6x6rraHvUD+HzCYO5Wt0kiTWp6U8m8rieveG8d7tSMtjm4Gszx5+memWBrQ5AWT6hMfF+e14BHWPF/Xw+An188X85b9rQPRuVoxYydpqX/lcWEnyTuhR+4/dUxY6fSd0EdHdAOwzMPKpqDOObc6bLeSNLWpLwHCF3ylsbOhqO+cpI84PB6Pz38FBQV3FBQUNCj/3i4oKCgy38cXFBQcUdIsLSgouEf6+72CgoJxBQUFtxcUFEwyny0pKCj4pa+yOzs/94Qap1oueG5d8qrntqWvevb/rdnj8Xg8h5rOeG5Z9BfPrUte9ZxquWCbdv/fmj2Hms6IfOQ8rX7Lf59queCZt+w1z6GmM17/n2q54DnVcsFz29JXvb61o53oprRE/7/85GXPt5e84jftbUtf9RxqOtOjPPrbab3kMg81nfHKy1cb+nrnKw31GbW/mp9du1t954YGJzTKfWL1Ppj0buiQx5gVLXZtYUeD2vbqt77qHkzdQtEmbtLINAfSTk7LDMc4lGDLV125bDLGXgKwmnNexRjLBrCDcz5eev8jAKmc8zXm33UAvgrgc875afPZ5QCe4JzPsisnlC6bBJKq588tFi6L8kUGdu6AtFUF4OWXnZSajHvXvN3Db1iWhFS3NfV/osvqt0w3ACxet0tc7iLvSEiis9Nd5uVl4uNDJ8XfSyp2Y2F5aQ//cqLdV/vJtgDakdj55KttqMYU2NEqu+7JfUa2E3UX4NT1LZBv/cGKTrt+c1KmVb0C2WXYjSVy16W/rWIpVLWd1U4rmHazaxv5mdqevQ15nsp1t6IzkL6KBMLhsrkDwBzz92wA2yzef50xFs8YGwEgnnN+AsBfyagLYBaAGpfluwbpUMllkXzbyZAKWLvWDcvrhwXlE70YPgChiqDvycXNajvrT7dLedjpcgFg5V1XYcWdU3t4AWWmJ2P1plq/7oz0b/7c4h7bXpl2tf4AerjmkeHYlx5bzouCvwKdHFkZhivlUy83OHIPtAMZ6e2MxMHCapGzst84SW+3/bczzjrR9RNIVaeqewh2ropudddWNIVbvRGKfK1UbWRzUsvqS8Zct0z/aQBXMMa2A7gbwEMAwBhbwxibwjmvgbEQ7ALwJwD3menuAfA4Y2wrgOkAVgRBu2sQc1pYXorVm2qxtKJSSKyyV4hsICR9teyBQ5B96+fPLcaC8ot6c1kqVmE1oa0mospUrZhDa1sHurs8WPVcjd/BR3WR9fu0GMpRkqrBjuwYVu/9LWDEbN3q/Mnrxcr7RS3fCrRT44dP+dXxhmLyqu3lZqGzk7bVRdkXQ162vtKLSZFxVGZossGTrgO0o0nOOxBmp37ja6F2YngNhb48kH6mdlpaUYl717ztSuiIFsRMRC7BagsNXIx8pYlKka0yE6Tv5O1eYkoSFj21XZzCaRzvYLhXkiqD1EdqRKe8ZbQK2Q5EZUFlpCbHY/UPvtQjjZ0qQq4L1c9XWwWqflC/kSNi7ZCUmtwjItdNWeo7+W5eO1WaUzWGqjbxRU+gW39/9bBSOQRDJ30n3w/sL09/kcZW3ztR6+09cNRrrqj5OK27vzb3pcbyly43t1+P8Rlt0BG5JmQpniBPfvk8mqyMZHGhsqxeoTTEJJc8vQP33jgeazfvx5ObjVMvV951lZdEqnoxqP7yctnARR95p9IDlbGgfCLi4y+esSL/b5VG3llQub50+4FK13I5RIsTKXv+/9siJEl6Fow3BL1Tz7SxS+OEkaoStBXsdky+4K+usoTuRE2lqtl8gTyenORp5/5r973TsSzPFRnqrtFffk7q4G8nZZeuf2aK33pEM2JG0qcOpYPKVLczVdJtOed9Dovd2R0klfpilirsDo2S4Ws3Yicp0zEF5F4nH8amSnuqpONEH+xG2rYqx07KJiSlJuPkybMhpc/unUqPr3N71HROJGi5DKeGbKft7HQnqDob+MoTcN7Gge5g/MGfIdfJLjFYOKlTtBmcraAlfXj72qvGRCvVBum46bA1OwlVXfWdGN/kXYAs+dOZJmpaes4PnxLBLlZlyL7pcgCT1SBWbQfyTsYKVhKrvGOy0vGqUpkVg7WSgPtnpvSQZNUYAiu4kfzUPnfq1x0Is1N3c06lc3/fOJWereIe7CAb8lWofRYpHTaNLbf+9nI+/tBX9PLBIGaYPgDBBMmYKDNcK4ZCOk6vA8VMyN/KTNPqO6uJJG/TiUHTsQ8qI6LnFC1KRxFYMVf1VEJ5ovijwR/TlJn3YvOAN1qArFRW8nbcimHYGYWJVvk4A5VhBjP5fcHOUBws5P4JZZ7Bfqu2o3oIn/ydOk7C1QdquYGqO33lEwmaox0xdeAaYAx++cIOu9uD6Fs6qIlu8ZlenO91N+aA/hcP3crKSPZ6Z3VZQsu5noczyTcPWR3AJD+X6SaVAR3gJTNQ+Yq/rIxkr9uT1PtXqQx/8L7+Lh/Ti4eIm63Um6HUg6ysDgST6Zbf0V2pVv1EaeU2DjWc5ikfEBbKfEMNOzrVdrQ60Ez9Tt2pWfWB1Rj3Bfreik553ATTfr4OVgsUvvo90LqHKq2KmL9EJRQgHT9J4iT5OPE0UaVdX8EwgehJ7XTQVnnIwVluvBac0hNIHnbt5ERnGiy9cp+4zacv6HYB33Q6rb/Vd3bPAgnikr93aiPpbdi1Z6B1D1VaGxr1JSqEYCa5FZN68JkqPPT9KY4ZqK/JY6hNdonbsgKlSVadqOXIA9WfsTUc8NfukZz88mLp69Yxf7gUmD4QeoOs28X/UmjPYNoylIZqbcg14Uuv59bIs/bnMy0ZqF1+dkZVwLAhtF3oEpeQ2NXBKi3tROjCi0Bo8KXTD4UO1Ik+NRDDZLCQ9etWu65I6n3JZhGKfNymC3WdA+3DaDSeum2PYATKYA3VThHTkr7cwIFKumSgfXrBLJ/qnUBXfqu7WOV87QKp5Pr4k/SdIhQ7ADdqlN6U+CJJJy3U6vWMbvLx1U+RlvTdItL9blfvYNsz1PS4gZb0JcjMeGlFJRav2wUArrwS7DwdnEqOqrcKANtz3Clfu0Aqei975DiBG+nbafuoHjzB0BIpRJL5ZWV4H30RTD7BLMzBjJVo6DM38DU3A2nP3twhuUXMMH0rBrmgfCLi4+LF3/Sdk+0uTVhf0Xn+1CZWZ5H7S0eMPRSqELVcuwngK40vBDp5AlEzRBOzCYZmeaEOBuFmGFb90xvqsFDB39gMx5iNFsSEesfXds3KeBKIZGpndHSSRzBqoEBh5WVE5QIQl6E4PesnHLQ6Nei5UTuFg2byiHJ6Bk64jeV2CJU6wqkHj1v0RUNutKjGVMS8ekdd1YnRUVi8nXFUhZWUY3X+SiC7BX9l+kIgEiadZ2OFpRWVWL2pNmxHDjuF07IDVTuFUyLzJzE62b31FfhyQohV9MX6xwTTl0EMQD5R006X72srS5PYSr0TiQkeqJqFvIzkBY/eyRfAOynXV6h+pBCI2slu0Q8XLXY0OdlBaUQfLrV+iQmmrxoUKZhJDou30m/707P7msThlgACXVj6Z6aIOsln+FBegdBrZ8DuTTjV0UZSDxtOm4ZGZHAp9ktM6PQBb9dB+Zx8u2+yMrxPtLTT3fU1PaRV3ezgy6UtnItauNs0VPSHms5wtWsoXEsDpctNmmidS6Fwf440Yl6nD3hL3r5cLVX1j5V3DRD9Wz47+qgd1GMbrNL7UpeEGpFsz2jVw0YjXW4k3UtNOo7GfgkGMSPpy/AnhagSv5pG9sQI1ZEBoZTyrM5td3NeSKQ8E1Qa7DyNwlFuMPXrCxIfoCX9UKMv0KklfRNODWpWAU7q71AGb4RSMmo55/zcdsB/TIBdGaGETIM/T6NA0RsePcEg2ugB3Em6l5p0fCkhZph+qCe5PKjdMBb5WSi9fSivQA5uCqRcuzoF267yQqueZ+QWgXj0BJpvOBCtC5HGpYWYYfqB+nY7xenWdq/IWqdlymlCrUYJp5RFdSLY2TyCQajuIA1F1KWK3vT519AIBWKG6QPBHSlgB7qKDrA+D8eODkpDVyH2NemOjN2huNUonAg1TeFmzNHYhoEiWsZytNARbYhJQ66MUBrz1Lx8GUnlb4Ce6qJwHBkQDvfCvkCrGzipVyTpDKadI0UnjWW3x02Ekk4nc88tomF8+oM25PpAKKM01cHlRCpUA6P6il7Xyth9qSDa+iBaoqB9gdoM8L3ri1QdtKrMHjHP9IHw62kD/V4P1t5FNPZBNEZBy5DbzOmxFJGgSaMnYpbp23nPhHtAOtX59xaiWZqMJKKJYWRlJIu7maMZTlyho20xjUXEJNO3kjjkaNzedsnz5+IZLjh1MdWIPC4VRnmp1KMvI6aYvmw0tZI4IuGZ4fTcdfXIh0hsi524mGpcGoiGHa1G78AV02eMpTHG/sQY28YYe40xlmfz3TjGWIP090DG2Jtmuj8wxtLdEh4oVMYVSt/tQOBmCxzJbbFdZG4kjorWiAzI1ba3d7QavQO3kv49APZzzq8GsBHAUvUDxth3Afw3gIHS418A2GSmqwPwA5flB4y+pE8M5EiESCHcDN8Nk+jrTKU36A/0mA436EtzLRbhlumXAXjD/P06gGstvjkF4Msu0oUNbg6NihX0Zl3dMIm+Lk32Fv1ujulwW45GdCLR3weMsTsA3K88Pg7gjPm7FUC2mo5z/oqZXn6c5S+djJycdCQmJvgjMaQ43dqO/pkpON3ajgef2I61P5/p91iAvLzMCFEXHOzoDKSu4UJenvq3Pa39M1OQlwc89cCsXqOX4LbvI02/TKfa1tGEvj6X+gL8Mn3O+XoA6+VnjLGXAFCtMwGcdlhei/n9eSfpTp1qc5htaKBG8T30/SnovNCBZh/H/PaF6DzAP51O6hopBHIMdG/SG4q+jwT9l8oYjRb0BTp9LUpu1Ts7AMwxf88GsC3M6SICVc0Qji1qb6kjrC5wl9EXtuNaV6yhETzcMv2nAVzBGNsO4G4ADwEAY2wNY2yKj3QrANzKGNsB4CoAa12WHzY4OTLZLXpLjxvqM+p7E5rh+8el0M8a4UPMH7hmB6cHNgW61Qvm4Kxg0kbqNqpQoC9sn4HopNNq3EYjnVbQdIYO+sA1FwiXKiEYhh/MLqG3DZ4akYFWgWn4g2b6PhBNE0dPZg2n0GNEwxc00+9D0JNZQ0MjWGimr6GhoRFD0Ey/j0B7ZGhoaIQCmun3AYTD1VMvIhoasQnN9PsAQm3E7evn1mhoaLiHZvp9BKE04mpPIA2N2IVm+r2M3pK2NcPX0IhNaKbfi9BqFg0NjUhDM/1ehFazaGhoRBqa6fcyNMPX0NCIJDTT19DQ0IghaKavoaGhEUPQTF9DQ0MjhqCZvoaGhkYMQTN9DQ0NjRiCZvoaGhoaMQTN9DU0NDRiCJrpa2hoaMQQNNPX0NDQiCFopq+hoaERQ9BMX0NDQyOGoJm+hoaGRgxBM30NDQ2NGIJm+hoaGhoxBM30NTQ0NGIImulrWELf5qWhcWki0U0ixlgagN8DGASgFcD/4pw3W3w3DsDLnPPx5t8DAHwIoMH8ZDPn/NduaNAIH+gaR32rl4bGpQdXTB/APQD2c87/jTF2K4ClAP6v/AFj7Lvms4HS41IAz3POf+iyXI0IQF/jqKFx6cKteqcMwBvm79cBXGvxzSkAX1aeTQJQyhh7lzH2ImMs32X5GmGGZvgaGpcm/Er6jLE7ANyvPD4O4Iz5uxVAtpqOc/6KmV5+/AGAGs75W4yxeQCeAPCvdmXn5KQjMTHBH4m9jry8zN4mwRH6Cp1A36FV0xlaaDrDD79Mn3O+HsB6+Rlj7CUAVOtMAKcdlrcFQJv5ezOAh319fOpUm6/XUYG8vEw0N7f2Nhl+0VfoBPoOrZrO0ELTGTr4WpTcqnd2AJhj/p4NYJvDdBUAvmX+ngWgxmX5GhoaGhou4NaQ+zSADYyx7QA6AJQDAGNsDYA/cs6rbNItBPAMY+xeAOcA3OmyfA0NDQ0NF3DF9DnnbQButnj+gMWzIdLvfwCY4aZMDQ0NDY3goYOzNDQ0NGIImulraGhoxBA009fQ0NCIIWimr6GhoRFDiPN4PL1Ng4aGhoZGhKAlfQ0NDY0Ygmb6GhoaGjEEzfQ1NDQ0Ygia6WtoaGjEEDTT19DQ0IghaKavoaGhEUPQTF9DQ0MjhuD2lM2YgpM7gRljDwK4HsDnAH7MOa9ijA0CsA5ADoAEAN/jnH8cbXRK78oB/JBzflW4aAyWVsbYlTAu3+kC0A6jTY+Hgb54AE8BKDHLuZNz/pH0/i4APzBpW8E5f4UxNhDAJgBpAJoA3G4eThg2uKRzBIBnYMz/OAB3c855tNEpvbsGwHOc88vCSaNbOhljGTBOHh4NIBnGHLI7abjXoSV9Z6A7ga8GsBHGncACjLFSGFdDTgVwK4AnzVdrYAzWa8w0hVFKJ0xmegcMJhAJuKX11zAm1VcAvARgQZjouxFAqrkALgTw7xJtQwD8CMB0AF8HsIoxlgLgFwA2mXWqg8Ecwg03dC4HsNZsw5UAVkUpnWCMXQbgpwCSIkCjWzp/DqDB7Pe7ALAeuUYRNNN3Bn93ApcBeJNz7uGcHwaQyBjLgzE4hjPG3gIwD8DWaKSTMZYLYDWAH4eZPpUWN216K+d8r/lNIoAL4aaPc74bwBeld1MA7OCct3POzwD4CMAEOLs7Ohro/CmAV81vwtmGQdHJGEsF8B8A7o0Afa7phLEAdDDG/gpgGYC/RpDegKHVOwpc3gmcBeCk9Dd9MwrAKc75tYyxX8CQSn8RZXQOAPComdf5UNCmIpRtSlttxtiXAMwHcE3ICb5Y/hnp7y7GWCLn/HOLd0S//Nzy7uhooJNzfgIAmHGB9S9hSLdRRyeAtQB+yTlvVO7ajjY6BwLI4Zx/nTH2PRht+r1IERwoNNNX4PJO4BbpvfzNSQB/Np/9BcAjUUhnNoAvwNBJpgK4nDH2OOc8ZFJ/iNsUjLFvA1gC4HrVDhBCqOXHmxPfF230/DwCuzs60nSCMTYDhu76u+HW57ukswPA1QDGmbadAYyx/+ac3xpldFrN84VhpjEoaPWOM/i7E3gHgK8zxuJNI1m8KU1tl9JdA+D9KKSzinN+hanfvRXAgVAy/BDTeoIx9h0YEv5XOOd/jwR9jLFpAPZL76oAXM0YS2WMZQMoAtAA93dHR5ROk+H/GsB1nPPqCNDohs4qzjnjnH/FHJufRYDhu6GzAZGf50FBS/rO4PdOYMbYNgC7YCyk95npfgqggjF2D4xtYXmU0tkbCJhWxlgCgN8AOAzgJXPL/y7n/MEw0LcZwFcZYzthGLdvZ4z9BMBHnPM/M8Z+A4OpxwNYwjm/wBhbYdbpLgAnEP7+dkvn4zC8TDaYbcg55+E2OgdMZ5jpCRmdjLGVMOb5LgCdiGLVDqCPVtbQ0NCIKWj1joaGhkYMQTN9DQ0NjRiCZvoaGhoaMQTN9DU0NDRiCJrpa2hoaMQQNNPX0NDQiCFopq+hoaERQ/j/7Ix3hsuZ4dsAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(net[2].weight.data.reshape(-1), net[2].weight.grad.reshape(-1), s=1);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loss Functions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order discover a good set of parameters using optimization, we need to specify a loss function to optimize.\n", "\n", "The loss function $\\ell(X_\\text{out}, Y_\\text{tgt})$ compares the actual network output $X_\\text{out}$ with a corresponding target value $Y_\\text{tgt}$ and approaches some minimum value as their agreement improves.\n", "\n", "A loss function must be scalar valued since we need a single gradient for each parameter to implement gradient descent,\n", "$$\\Large\n", "\\theta \\rightarrow \\theta_i - \\eta\\,\\frac{\\partial\\ell}{\\partial\\theta} \\; .\n", "$$\n", "Note that the loss normalization is degenerate with the learning rate $\\eta$.\n", "\n", "Our choice of loss function is primarily driven by the type of problem we are solving: regression or classification. We introduce the most obvious choices below but there are lots of reasonable variations (see [here](https://pytorch.org/docs/stable/nn.html#id51) for the complete PyTorch list)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Regression Loss" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For regression, the $L_2$ norm is a popular choice,\n", "\n", "$$\\Large\n", "L_2 = \\frac{1}{2}\\, \\left|\n", "X_\\text{out} - Y_\\text{tgt}\\right|^2 \\; .\n", "$$\n", "\n", "Optimizing the $L_2$ norm is equivalent to finding the maximum-likelihood (ML) point estimate for the network parameters (weights and biases) if we assume that the uncertainties in $Y_\\text{tgt}$ are \"homoscedastic\" (drawn from the same Gaussian distribution).\n", "\n", "In PyTorch, the $L_2$ norm is implemented as [torch.nn.MSELoss](https://pytorch.org/docs/stable/nn.html#mseloss):" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [], "source": [ "Y = torch.zeros_like(Xout)\n", "loss = torch.nn.MSELoss()(Xout, Y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In case you actually have a reasonable estimate $\\sigma_Y^i$ of the $i$-th sample's target uncertainty, a better loss function is the $\\chi^2$ statistic:\n", "\n", "$$\\Large\n", "\\chi^2 = \\sum_{i=1}^N\\, \\left( \\frac{X_\\text{out}^i - Y_\\text{tgt}^i}{\\sigma_Y^i}\\right)^2 \\; .\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Binary Classification Loss" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For binary classification problems, the L2 norm can also be used but the binary [cross entropy](https://en.wikipedia.org/wiki/Cross_entropy) between the target and output probability distributions is often a better choice:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "\\text{BCE} \\equiv -\\sum_{i=1}^N\\, \\left[\n", "Y_{tgt}^i \\log \\phi_S(X_\\text{out}^i) + (1 - Y_\\text{tgt}^i) \\log (1 - \\phi_S(X_\\text{out}^i)) \\right]\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $\\phi_S$ is the sigmoid (aka logistic) activation function used to coerce arbitrary real values into the range $[0,1]$ required for a probability. An input real value used with sigmoid like this is known as a [logit](https://en.wikipedia.org/wiki/Logit).\n", "\n", "The equivalent PyTorch code uses [torch.nn.BCELoss](https://pytorch.org/docs/stable/nn.html#bceloss):" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "Xout = torch.ones(10)\n", "Y = torch.zeros(10)\n", "loss = torch.nn.BCELoss()(Xout, Y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The cross entropy is inspired by information theory and closely related to the KL divergence we met [earlier](Variational.ipynb). With this approach, our assumptions are that:\n", " - The target values in $Y_{tgt}$ are all either 0 or 1.\n", " - The network output values in $X_{out}$ are continuous and $\\phi_S(y^{out}_i)$ is interpreted as the corresponding probability that the output is 1.\n", " \n", "Note that *something* like the second assumption is necessary to reconcile the different domains of the data and prediction.\n", "\n", "With these assumptions, the likelihood is:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "P(Y_\\text{tgt}\\mid X_\\text{out}) = \\begin{cases}\n", "\\phi_S(X_\\text{out}) & Y_\\text{tgt} = 1 \\\\\n", "1 - \\phi_S(X_\\text{out}) & Y_\\text{tgt} = 0\n", "\\end{cases}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Take a minute to convince yourself that the following expression is equivalent (the case $\\phi_S(X_\\text{out}(\\Theta)) = Y_\\text{tgt} = 0$ requires some care since $0^0$ is indeterminate):" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "P(Y_\\text{tgt}\\mid X_\\text{out}(\\Theta)) = \\left[\\phi_S(X_\\text{out}(\\Theta))\\right]^{Y_\\text{tgt}}\\,\n", "\\left[1 - \\phi_S(X_\\text{out}(\\Theta))\\right]^{1-Y_\\text{tgt}} \\; .\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using this form, you can show that the cross entropy loss equals the negative-log-likelihood of the $N$ samples of training data so optimizing BCE is equivalent to finding the ML point estimate of the network parameters under the assumptions above.\n", "\n", "For fixed training data, optimizing BCE is also equivalent to minimizing the KL divergence of the network's predicted discrete probability distribution with respect to the empirical discrete probability distribution of the training data. Therefore, training a binary classification network using the cross-entropy loss is effectively performing a variational inference (VI) to find the network probabilities that are closest to the empirical training probabilities." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Multi-category Classification Loss" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How can we generalize the binary classification cross-entropy loss to problems with more than two categories? The usual approach is to increase the number of output nodes from 1 to the number of categories $C$,\n", "but we can not directly interpret their values as category probabilities since there is no way to ensure that they sum to one. We could simply require that they are all non-negative and renormalize, but a more more robust approach is to convert the vector of output values $X_\\text{out}$ to a corresponding vector of probabilities $\\mathbf{p}$ for category $j = 1, 2, \\ldots, C$ using the **softmax function**," ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "\\mathbf{p}(X_\\text{out}) \\equiv \\frac{1}{\\sum_{k=1}^C\\, \\exp(X_\\text{out}^k)}\\,\n", "[ \\exp(X_\\text{out}^1), \\exp(X_\\text{out}^2), \\ldots, \\exp(X_\\text{out}^C) ] \\; ,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which works fine with positive or negative outputs $X_\\text{out}^j$. Note that softmax generalizes the sigmoid function $\\phi_S$ in the following sense:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\\Large\n", "\\mathbf{p}([y_1, y_2]) = [\\,\\phi_S(y_1-y_2)\\,,\\, 1 - \\phi_S(y_1-y_2)\\,] \\; .\n", "$$" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [], "source": [ "def softmax(y):\n", " # subtract out max(y) improve the numerical accuracy\n", " expy = np.exp(y - np.max(y))\n", " return expy / expy.sum()" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.26538793, 0.01321289, 0.72139918])" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "softmax([2, -1, 3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The softmax function effectively implements a *winner takes all* policy, similar to the sigmoid activation $\\phi_S$, as illustrated in the plot below where:\n", " - the color scale indicates, from left to right, $p_1, p_2$ and $p_3$ for three categories,\n", " - $y_1$ and $y_2$ are varied over the same range, and\n", " - $y_3$ is fixed to the middle of this range." ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAADRCAYAAABB0v+GAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzsvXmQJNed3/d5mVlH3zODwQBz4JjBUQABEDdA8FgCXBLY5a6oPSzZXllahS1ZEV7ZIflYh2WHHQ7bEVaEQrZ12rqslaxdSdwlwEPguQS4JMhdcrkEiYMoYAbXHMDgmOnu6roz8/mPfC/zZVbW1VdVdb/vxERmZ77Myq6ul/XL7+/3/f6ElBILCwsLCwsLC4vZhzPpC7CwsLCwsLCwsNge2MDOwsLCwsLCwmKPwAZ2FhYWFhYWFhZ7BDaws7CwsLCwsLDYI7CBnYWFhYWFhYXFHoEN7CwsLCwsLCws9gi8SV/AKOi+e8Z6skwrZDjCkND8IVkPg97taqyUYbxO0E3GBH60GnSh21HHRPtltwOdZrSt01LLJrKltjU21LIR/QfkRj1artcJ16JtwaUW3cvRa7dXoynyw2PXs/Rak5X3m9x3/gkx9JeeEriFY3bubBH/xV/9NQD+zt/77W09rxDDP0aCZIw53jHW9Ri93xEiXneFE29L1h0Kjpva7zkuRSf6rBdFtCw4HmWnAMCciJbzToE5tX9BfX0sCY9lGZ1nRTqsqGl9IAgJHVh74Ao+8NxblFq+nTv7DDs1d/Yjgu6FkefOTAR2FlOKPkFdKpDLG2sGdHqfGdBB9LMZ0AEEfhTQQRTUmQEdREGdEdABUVBnBnRqaQZ0AOFag+BSdGz3chgHdPW1Ei/cfpzn77yGxRNNPvi7r+a/FxYWY2BYUDdKQKfHmQGdHm8GdBAFcI7aVnDcVEAHUHS8VEAHUHYKqYAOYE54qYAOYFk6rOjALogCOoBlEVD91AkunVpmfsHn3h+8PuxtsbCw2AbYwM5ifAxg6cYK6owgTmYYO4JuKqCDXpYuFdBBFNSZAR1EQZ0R0EHE0pkBHfSydPW1EgA/qlzHy3cepbTW4ehXL7Am7JSx2DzGCeiy4/uxdGZABzqIyzJ2aZbODOggYunMgA4ils4M6CBi6cyADnpZugPSJ3QErz52jEsnlzny9hofPv0K3spwdt/CwmLrsN9SFhZ98MLtx3n5zqPMrbf5wOffoFW3mRkLi2EIHUH10RNcOrnMFedrfOzbVbxFG9RZWOwWbGBnMRqG1NJtlqmDEevpoDf9uk31dNCbfv3JqRP89M4TlNY6cVC36lqtkcXmMMl6OuhNv25XPR2k068HpM9rnzrGpVMRU/exb1dZWWxSOuAP/f0tLCy2Bzaws7DIwZE313n75AanvnaB8kaXlk3B7ls8+eVvT/oSZgrHX7lEt+jwse9W8QLL1O1n2LkzGdhvK4vB2CpTB71s3WaEEtBbV7dNQglI6uo6BZdO3cP3Jbd+7g3WhEdLeKy6DmvuwLfCYo/i9JmzWzp+s0IJ6GXrNiOU0Nt2QigBEVu3LAKkAwfosHipxslvv8viShuA0gGfwkHLdu9HbHXuWGwONrCzyMe4NiZ5x+mALrVtc0IJANlpQTf6sthOoQRE6dcf3nwd5285xAeeeIPWRlRPp9Ovay6sCcs+WIyOaRFKQG/6dbuEEhCpX1997Bjdossj3/4pC4tJQAdQOOjgHioPfC8sLCy2D7vyGFWpVB6sVCpPZ7b9WqVS+d5uvL6FxTC8cPtxTt9/FCTR/ymBnTuTx1/89c/wF3/9M5O+jKlE6ED10RNcPHmA4VWEuws7dyYPO3cmgx1n7CqVym8Cfx6oG9vuAv4TmLp7wf7GCCxdNGxzTB2o9Kti5VLjBgkldMq1294RoYT2qSutdyK2zhBK6PTrmghZ32XGzs6d6cDy0uJY47dLKKHHbVUoAb3p1+0QSpg+dXlCCZ1+dQ+VcVbmh74n2wk7d6YD484di+3BbqRizwC/AvxLgEqlcgXwvwN/DfjHu/D6FqNgO1KvEAVrPduM9GvgJwGdDvBk2FtP57eRvlrvl37dYj0dwA9vvo7Tdx6ltN7h2s+fjYM6M6ADWBchNbnryj47d2YM21lPp8dstZ4OetOvW6mny/Op+7k/+QneYpiqp9PpV2dlHrG8MPB92QHYuWOxb7Hjqdhqtfp7QBegUqm4wD8F/jpQ2+nXtrAYhMZcgdfuPMLcepvbn3id4sZ0WTLYuWMxrVg/Ns/F61cin7qnq3h5D30ThJ07FvsZuy2euBe4CfiHQBn4QKVS+T+r1epf2+XrsNiutKvGOOlXg6mDjFDCj9g56XfSTB30pl+3IJRY3yjBBtz2hTcoNPye9KvJ1AHUpE+diQZ+du5MMXZCKKH3bVUoAb3p180KJQ5In2W3w/JbHQ493eLIxfWe9KvJ1AGI5QXE4q4zdibs3LHYV9jVwK5arX4fuA2gUqlcD/xrO7ksdhtnbjzCyk83KLYDlt6JAsVp96mzc8di0ggduHDbIRZffBtHwrELq5O+pJFg547FfsN0f5tZbC9GZOmioZvwp4PEwgT619Vppk4zcoGfZuogLZRoqbq5TjtfKLEebZMbjZGFEoeuXOfWr5xlVQV02bo6k6kDqOPT3P0aO4spwPMvnB64f7N1dcOEEnr8VoUS0FtXtxWhxNJCl1tfvMBCjk+dKZTQdXVicQHmd1c8YTEdGDZ3LHYGQsop8nbog+67Z6b/IqcVYwRzySHb1B4s3hcYqlc/MSPWKdl+Qgkd0On0a6uRFkoYAR2MKJS4PxFKFDf8vulXM6ADaEqfhlLpfv3cV2dGVecWjtm5s0PYaaEE9KZfNyOUgN7062aEEhdPHkiEEmHYN/2aCuggCupUYDf/5/5XO3csLDaBoHth5LljGTuLfYEXbj/O6TuPMrfeji1NLCwsBiN0BNVHT3Dp5HIklPh2FW9xuoQSFhYWadjAbq9hEwxddFif4/oxdXpfXnswk6kD1R5MsXNBN8XUQR+hRKueZupA2ZmoVOxGI8XUQX+hxJ/cdC2v3HlsZJ86Uyih06+NsEtTdvPfI4s9jU888gAA33zq+8DuCSX0tq0KJaA3/TqKUGIFnzOPHY8tTUbxqUsJJXT6dX4e5q2f2X5Edu5Y7A5sAz+LPY+lS00WVlvc/sTrlDdscGYxHm6/7UZuv+3GSV/GrkNIWHq/mViaBJapsxgP+3XuTBqWsZtVbJKZ6z3NmHYm5v68vq8y7DEg7qmrG0Uo0WqmmDogYus29HqD8HI0NliNzpcVSoQCNjZKzK21+ODrZ1iXHi3hDTUgNoUSuq6uKbu0QhsU7nfsplACeuvqNiOUgN66ukFCiRV8kLDsdjhRfYPwZcHScjQ/hxkQp4QS8XIRUZ4b+L5ZWFhsH2xgN23YpoBt8EuMkHbVGOZPN0woAT3p1ziga6tUq98ZKJTITb9erqcCOoDWpQKNWhGIhBKXjy5w0787R024IBnZp84USuj0ayvs0g2tKnY/Y1BQtxNCiehYZ1NCibIK9vqlXwcJJc48dpy59Q4nnn81pXyFEXzqDKGETr+K8hwUbWBnYbFbsKlYiz2HF24/zun7j9JYLuGX3UlfjoXFTEALJS6ePEDt8ByhMzMCVgsLCwOzwdjtAou1l9GXoYt29m7LS7vCYKGETrWGYZqpg970q2bq8jpK9BFKQORXF65F44LVdoqpA2jUiqnerx944g2aDYYKJaA3/WoydQDd0Kdjfez2HYalXmG09OtmhBLQm34dJJTQLF1ZeCyobf3Sr1mhxLIIUr1fP/btKstLraFCCchJv5pMHURsXbE89H2cNozyt7cYDZN+L2fB1m07MRuBnYXFCMhampQ3ulPfUcJi+rFeq0/6EnYUoUNkPmxamlihhMU2YK/PnWmF/dbbYxjIzqUH9m4btZOEWVdnMHWQEUroWrqgm66rM5g6GEEooYyIw7Vmqq7OZOouHZzn+TuvSVmajCqUgN66OpOpA+hIn46tsdtX0CzDb/2LL+Tv32GhhN42TCiR1NMpls4pDDUgNnu/vl05yKVTBlM3P4ZQAnrq6lJMHURsna2x25foN3d2G6MyhnuF2bOB3Yxj5EAuGty7bdS0q7EtDtxkmAglzH0qiJMq/Uq3DV0V2HVa6YAOoFGP/gNSd5bYaKQCOuhNv2qhxPpGCW8j4KZvnIO32j0+dcOEEtCbfjUDOoBO6ONnffws9iR2wqduM0IJ6E2/ZoUSZeGlAjoYzaduRUY/LLsdlqoXWVrscMfFc3jzIeVD0TzIFUoszg/0qUsJJXT6tTiHKBQHvaUWFlOBvLk/i8GeFU9YzDTOnTiIVHPxyMtrFDcsq2axvbjhhmu44YZrJn0Z24rQgUvXq2AMqFTftulXi23HXpw7swDL2E0xxmLj+p8kf3teF4l43xB/ulgw4SOzfV8DP83UQcTW5QklFEtHo55i6iAjlLik06+S1mpGKHH/UY4tv8f133uHNeFuSigBvelXk6kD8MOAwIp49jT6MXWf/rmPAvB3//7v7KhQwnWcoT51WaHEnCgMFEos5aRfl0TAa6r366E/aHHN2UvML3VSTB2Ae6CUYuoAxPLiYJ86UyihmDtRKIK6Rov9BXPuzCqEEDPH2tnAzmImYQolrn7u8qQvx8JiJhA68LIhlDh6YXXSl2RhYbHN2JXArlKpPAj8zWq1+nClUrkL+LtAALSBv1CtVi8OOn5bmKv9gGGMUr96OugVSQwwHtbbpGk8rGvt/HaaqYOIrUt1lEgzdrLRyBdKGEwdQGvVo1Er8sLtx2OhxK1PvEm7HtJWbN1mhBLQW1dnMnUAgQzpTqDGbqtzx2I0jGNrMsyAeFyhhOsowYRwhxoQjyqU0EzdAaOubkkEUVCXI5QoH+qmmDoA5+BCmqmD4QbEhlAirqtzCjCBGrutzp1sLaXF5rGV91IyebbMnPOzwN7teGBXqVR+E/jzgNY9/1/Af16tVp+tVCp/Bfhvgf9yp69jz2DUdOCgIGQUkUSOP10qoIO0P50O8Mz2YHrZNn3qmklAV1cfiVo96ShhpF/NgA56fequ/fxZ2vVwy0IJ6E2/mgFdtD8g3OVUrJ07O49JCiXMgA5G86kbRSixJJ04oDuYSb9qn7rHfvhcr1DCCOhApV8Xs2nXIT51plBCp18LRYS7u6lYO3f2DrYjwJ6G4HA3sRviiTPArxg//wfVavVZte4BrV24Bos9AukI5tbb3PbEG/tBKGHnjsU2QRA6IvKpe3pf+NTZuWOxb7HjjF21Wv29SqVyvfHzWwCVSuXDwF8Ffmanr2HqsV1MUB5Ll2tx0sebzky7ZsQTKX86zc6FXaROt+r0q99JMXXQm341mToAudHoI5SIPp4bayUkUG8UOfJHl7jiR6vUu2LbhBLQm341mTqAUIa7Lp6wc2dnsVm2bphP3ahCCZOpg9F86rJCiUXh9gglDgQJU7csAwSw4rT5+HdfQgrB8nyLcrajxIESzopKoWqWbnEesWikYKG/T10hYvtSQgnF3Am3AO7ulnPbuWNhoh/rtxkmT8/vaU7JTkQ8UalU/n3gvwd+oVqtvjv0AKtGjDBKjdeQQA76eNNlVa95/nRGQAdEQZ0Z0IGqp1P1cm0V4DXrifFwoxEHdOG6CuzWmgSr0fFm+nVjLfqy+NEt1xJ6gqu+d4l14UIXLrsOqyqgq22xng56069mQKe3hVMwkceeOxY9GCegE0Lwr377yfjnUdKvm62n84Q71KfODOgAVqTbU0+3EkY+daEDrz16lKtfXeX42zUWF6K5Wj7gUziork2nX1fmkjo6XVe3aChgy2pbsdTrU1coIfS6UU8Xp19dD8TknbXGnTuTboO1F6Dnzna/l9sVVGUDvnECvWkO8HY9sKtUKv8R8FeAh6vV6qXdfn2L2cKLdxzn9F2R+vXwn6xCZ9JXNDnYuTMZrK7WJn0JYyN04OVHI6GEX3S58e13Jn1JE4WdO5PBLM6dvYBdDewqlYoL/B3gTeBzlUoF4FvVavV/2s3rmCi2W1k5QN0aD8kbY6Zd9XkM9k5mWoXhd+JUbCyU6DQN5atKtXbaSN0eLE6/NhKfulo9xdQBBKudHqHExlqJH91yLafvOhqrX+sdwWWVfl1100wdpIUSdcXStaQ/VCgBvelXk6kDCKWcqI+dnTtbx2ZTr4WChyME3a6fGtcv/TquUMJT20pOYahPnWbqlqReOimmDmCFRChx5dtrfPTbL7O00jbSrwL3UMLUgVK9mkwdRGydZup0+rW8kGLqgIityxNK6PSrcGCXxRMm7NyZHAqF6DOg5852YadahAnEnhBa7EpgV61WXwc+pH48tBuvaTHbMJm6W594k/JGl7b6UttPsHNn8vhP//KvAvAP/+G/nfCVDId04OVPRj51h87X+Oi3Xt4PQolc2Lkzeei58/f/wb+Z8JXsL8yGQfF+7NE5sq1J/ri+dXTxtkwXiTBMmDgZ9PjTEXSRfkYo0e2kmDogYut0R4lWVGMnNzZiOxNZa8RMnR8LJRKmrl4rceHEAV6460TM1DUbIU3HZc1J19XlCSVMpg5GE0pAb12dydTp/dNYS2ExGsatq9MYJpTQ48cVSnjCTTF1kO9TNye8VF2dZupWjLo6zdQdDAPevO9KLp1STN23XubAYjTXUnV1h0pppg7yhRLl+TRTB2mhhNfrU5cSSui6OrcAzuRr7MZF9m9vsXls53s5Tq3zZnq/6nvBLDN3sxHY7VfxxAjGzLlp1tSAPsFcJtiTYZhWvYaZtGu3naRdfSP92jYCOlA+dYZQgkj1KteUYCIllFCHrHrUa9GXRa1RYvGVJsdX3mPuxTWajZA1Jz/9mhVK1E1/OrVvFKEE9KZfzYAuep/lVIgnLMbDZgM6yAolkqBumE/dIKGEDuaKjhcHdFoJawol5oQO5tLp15UcocRB9VledjtUnnuLuQWfD716Bm8x7Jt+HUkoUZ5LB3QQCSV0QOeV1LZivlBCb3McxBSIJ8aFNSjeOkS83D5V6qhBYr/79aiih1EDvGlsOTZ7s81iz+LSIWWMKuG6P3yHwt73qbOw2DJCB+pXREFWoRNy1x+/uW/TrxYWFrPC2O0Wpqx1WQ8bl4dBDB3ketIhw8TGRKpH/8BP9pv+dJrF6xr+dLrbRKsJ2tLESL/KDe1Pp5a1ZuJTt9pJMXUAG+tlnr31Gk7ff5Qbf/88R6prXFapV6Bv+jUrlGjKboqpg9GEEtCbfjWZOr1t2p7KLPpjnBZh2fE9LcKilZiti1OxJCnZYUIJk6mDKP2qmbp5R7N0hRRTB1GbMDP9uqymqE6/LimhxHsnljj8tZ9y6FKdhaW0UAJy0q+aqTM7SowilPCKKaYOctKvBlMXvbfOTKZird1JL7b7HrhVVnQQm2bO4zz2btbahI0DG9hZTBwvfvBYLJRYPt+Y9OVYWMwEpAOvfPIEl5VQYlk9PFlYWOxvzEZgN0EmbSTWbKcw6PfONSIOevebPV5Nli42HvaT8drOxO+Ar5i6WDCRCCWkEkrQakAzsTQBkI1mwtTpurpam+Cyqqtbhebl6Im+Xivx4geP8aISStwyQCgBsCYCNhTDaAolzLq6zQgl9LY8oYTeJqWc6WLa/YTtFkp855lnESRsnWbqNDOXravLE0rEPWCNXrAmUxctvRRTB1Fd3bKMXm85NHq/EkRB3allrnxrnY88/TIrCzlCiQMRq+aszKU7SsRMnaqhG1Uo4ZUGGxAbQom4rs5xpsKgeFzsNfHEdtQIj8tifueZZzd13CCY7Nogxs+8Xw9j7wZhFi1QZiKwiwv49wOGBZKD2oaZgaAZzOl0qymYMBWwqPdYq10Dv1co0W6mAzqIVK86oKsbQolatB6uRWnaYK1LR1mCttYKsVDiR7dey5m7rqa03uGaL5yllRFKrOm0q4iucUMGOyKUgN70qxnQwWwrpPYLNutTB4M7Srz4/BmEEdSZAV10rDNUKBGnYEWigC3FCtgk/bqg1hdU+nVZClbC6HpWAslKGHWUeP3RY3FQ99gf/wRvwRRKJAGdMNuELaka1vn5JKBTS1GeSzpK6CAuTyjheklAp4O1TPo1FdCZ42YMey0V6+7g79Mvlfni82eA7QmS9T25398lew3mfM8L8swAb5q7SGwGsznjLGYeoYDVq+Ypr7f5wBNvWKGEhcWI8EsuG1fMcfB8jY98c//61FlYWORjJhi7PWt3Mqo/37D+r8b+HlFEqsuEIY6I+76qbd12wuL5Rg9Y7U/XbiWdJFIdJVR6tqYFE03C1Yip89ei83VXI6YOYGO9RK0ZPf3f8OXz+GWXZlPGHSXW1aPGmghTTB30pl+3SygBvenXPKbO2p1MJ7ZTKGHu16nXz/zSwwjgi098C9dxhvrUZYUSRcdLMXWg/OnUNp1+XTCEEgdU+nUlFKwEUq0HLDtdaHd55GsvUugErMw3U0wdgHeoFDN1zrJi5pYWEAsGS2cydRCxdVooUVIsnymUUOIIUSgZaVf19eG4+UIJk6lzZs9c3NliYf++Qp85+Kd+6eNANHcAwk1kPvS9eBDrF0rZcx/IpmyzWRdHiJ57+jRal2wGlrGz2FW8dMdRLl6/DIATSIp1y9RZTDeuuuoKjlx1xUSvQTrw2sNXUzsYBV9zza5l6iymHkemYO7sR8wGYzetnSd2gkkcUTAhU4xdpobOHBuGsdmw3i8DP21GrJdm39euElJoi5NWMxZKxH1fG01kXW1bV3V16+2Yqetcip6gWmsF6htFfvrBY7x49wnKl9sUXm9RU0/xa65IMXUQ1dWZTB301tVtl1ACeuvqsk93lq2bToxbV2eix9Ykp6OEZm0ExGxdngGx3pYnlCiJQoqpA5gXBeYNpg7SQoklo65uiYAznzzO5VPLlOZCHvyDMywsRUz63MFuzNSZdXUmUwdEbF0slJhLM3UQsXUmUwcRW6dZObOuzmTqIKq7GySUmEG2DvZejd0kEZt558zHoQyZronLYfvy2DyzFi9PaLEf6qVnI7CbVfHEJgI/OTCwMwLcfkGc7E3FSlP5CpHi1Uy7QqqzhOy0E8+6thZMtJKATgdzjSaypv3ptFDCp3M5urG31qMvg9p6iR/fdg2v3n01xfUOx/7deWq4rLnRRFtzJOsimmz9hBLQm37dLqGE3jYs/boXKPq9gs0EdNnAzRyX11FCCyVM93wh8sUTOogrOG5K+QpR+tUM6CAK5hZUynZRdZMwhRIHVPp1iYA3Hz0aCyU+/txLeMsh5ZVoThQOpJWvAGIpI5SAKKhTwVxKKGGkX/OEEjqgE3GA56UCuuj9GiH9OoMCCncGr3naoOfOQPHEiIpVJ2dXaJw2G+SZ6dl+AV6ekCL3EocEhdMmvpiNwM5ipvHSncd49Z6rKa+3qTzxJq3GdHz4LSymGdIhZuoOnq/x4adexluw6VcLC4vBmInAbmbtTuQYKeRRPevyOk2Y7FxWKBF0ez3rAj9h6nRKtt1Ouk10WpGVidoOKv0aM3XRvrBmCiWia+iuOTTWoif/eq3Iazcf5qf3HKe43omDOpOpA1gXMmbqdP/XDenTVOv90q/bJZSA3vRr9gluWp7E9jvGFUoMOi6bfs12lIjYuuQ413HwHDdmckyRREGxU0XhpZg6iHrAmkxdtHRZlroHbHS+pVDETN1KGPD6x67m8qllDr+1zkO//worC62YqSseUtd0sJgwdSuKpUv51On+r3N9Okqo9G2OUAK3kGbqIGLhTKYOertKZFOvM8p8uTPYLWNakL1fDmI/zXtt1pIldU/OOdZk8cLMFHfon5YdhGlj3zaDXQnsKpXKg8DfrFarD1cqlRuBfw5I4HngN6rV6uDHUB18TBu2WmM3rgGxaSysN8kw7U+n6xFN1asZ0EHUEsyP1qU2Im63YgUs7XYc2MmmWjaasT+drKnAbr0TB3Sdtehm3lgrUN+IviBq7SLLLzU4ePU6B773Hs2GZNUV1NRsXFXp17oIUgEdQFP6Q33qtqueDkZLv06iNmPLc2cPYSvpVzBq5/qkX7PGw0KlYs+dvRiti6jGTgd0OpgrOG4czJUcs54uqaubU8fo9OuydOOALk6/hpFPHcCy6HLquXcozoV8/CfVyKdupUvhQPS7uCvRuZ2lEmLJ8KoDxOJiEtDNGerXkpF+LSReddEJCz31dML1htfTxW+0kx5nbgPEBIKkrc4dm4rdPHQ93Pmz7wCJgjwLiUwFc73BVKJc1ePygj0pZTx/9WtHc1qNy0nLmkrZUVOys4Id/+RWKpXfBP4JoB4R+dvA/1CtVj9GlIL/0zt9DRa7j8ZC9OVQ6ATc+NXz1qduE7BzZzrw1Ne+z7e+/oNdea3Qge6cqr273OaBb71q1a+bgJ0704GnvvZ9nvra9yd9GfsOu8HYnQF+BfiX6ud7gW+p9S8DjwKPDzyDThvOKkZJyeawdz3tzExxRLwt0x6sJxVrKGB1qjUI0mlXiFi6jhJPNJvQUilYnX6tt2KmLliPxvlrYcLUrav060aJn9xxnDfvupKbv3iWxYtN1hyX1QFCibr0U0xdtBzuU7ebQokJKam2Pnf2ALbK1IFKtfbxqYMo/SoyjJ7pWWd2k4jTroZIouT07yixYLQKW1Tp1wUco1WYEkrIgDcfPcbGFWV+5smXmGt2WVjqUF7W6VeJtxKdx1mJ4hWxsoBYzAglyuVEKFEyvOnM9GvcKkwxdl5xsFBC5PjUxW+2M41CiS3PHW9G1bw7gXHTkq4x/QZ512XPmx1rMnrJ/TvZ7wh9nLFN9p4rz7NuL2PHZ2C1Wv09wMylimq1qt/hGrCy09dgsXuo3nWU1x64mkIroFCf0hT6jMDOnenAHXffzG133bijryEdeO3R47x7aoXyRpdCZ0otnmYEdu5MB26/+ybuuPumSV/GvsMkxBNmcL0ErA47QPrtnbua3cCgWjro01liBGsT/XMeY6dZOt+HQPWDNfu/dhPRhF5KLZhotpOOEnVdT9cmWI+O8WvR/bGz5sVMXaNRpHrXUV669xjF9Q43PvEGjaak4bqsOkldnSmUqI8olGirpS+DVF1dVzGhQajZuc0JJcx9en+8Pl3dvxUUAAAgAElEQVSeR2PPnb2OcerqsixeyqdOiJidM+1MPMflg/fcjACqP36NgtlZQmiLk0QwURJe7E8XL3Hj3q9L6ll62egosSRDXv/Z46ydWuLwhXUe/PppVuaj+Te/0qF4MPqzeysFnAOKqVvS9XQJY4dhbSLKmr1Ty2LZYOxKaaYO+gsl+gkkYGhdXbJpKhi8seeO/jvvJ/S9341p6afvoberoO6lZ18F8hm51MuIdO2zWWOnr8E1hBCavXNEb32cg4gFFSm7kz0gjhiGSQR2P6pUKg9Xq9WngZ8Hnhp6RHcGArthwZuJYS3CsuOGBXaBYUCsg7hYKGEoYH0VLHbaPYGdbLaQKv1KvUlYV9s3omODmk9X3Qrbyp+uuVFgox59QTx/53Fev/cqiusdrvriORpNybqTpF9rynh4Q4SpgA7SQomGCuI60k8FdJBOv3ZlkAroYPaFEiNg/Lkz4+iXhh1XKGHCyUm/usLp8afTYgn9ilo4kXjWKfWrKFB2kmAuTsGigzmPZSOgg8inbjkICR04/+jROKj7xHdfxJsLmdc+dSshrk6/LpcQixmhxPxclHoFw6duHooqcNPBnJdJv+YJJZxMoGakXXMNiGcjoNMYe+4UbCoWGE9QEN83hV5EK57jEkoZpwjNcWmPufQ9OsxNxcr4/Ik4ojctmw3c9ppAYhAmEdj9V8A/rlQqReCnwO9O4BosthHdgsPblYOU1zvc8MSbNJu22HuHYOfOHkP7YIlL1yxy4NwGD37jNN6cnTs7BDt3LPYNxCzQkc2n/9n0X6SJTaVec/zp4n1Bsi3MMnYy0yosY23S7cbWJlocQbeL1CnYpsHYNZV4ot4mjJk6xZatOnQ2oifYZj1K32zUS2z40RP/u4sRM9BqhrFPXSKSCFmP068BDcXU1RVL15Z+iqmDKP1qMnXQm37dLqGEOT47Lg9+5/zM9BpyC8dma+4obJapg7RQwhzjGOyc3mamX02mDiKWwRMuv/zrn0IAX/qXT1F0en3q5pxCbG2yIDwW1fPyktA+da5haRJdz0oQclB9/oOjLkuXWxyYa7FwIJqPpYPR59w76OIeUkzc8jxCd5TQ6VejVViSfs3pLFEoIQq9rcKEbhVmdJkw06891iZZoYQ5Pv6xP1NXOHLTzMyd2696cCbnzqjYKnM16D6pz/2Zv/CzAHzhX/x+7v03u03/HN+/kal1vS97fN42896fOk8mg5M9Jt6W8/4M+27YyXgq6F4Yee5MFVduMVt49Y4raah2RsUNn6K1NLGwGArpwNsPHMYvRrffA+81cYM9HUNYWFjsImai80RsyTHNCEdUsQ1j61L1dDK1TeYydoYBsR8YjJ26nm43rqeTcV1dJzYepq321VuEjWg9XO8S1JWFSC3p+9pQTF29WeCVe67m9P1HmT/Z5IbH32RDFTisuoJ1JZSoq2qHdRFQl4lgoiF179fEzsRk6kDV06lj+tXV7bZQYhbY7VnHIGuTUevq8vab1ibmNrMWr5+1iVQV2gURGRFrpk7bmpRNwQRubEK8IJO6umU1XRdkyLlPHGP91BLzxYDbvneexYWIpZtf7lBcUUzdSnQO50A5MSBemItq6sAwIJ6D2NJE19XN9RgQi0K6s0SKqYOIrcuwc317wMZv7hh1ddNhgTIWNDO7ZyEGW5EMQr97YXw+Ed1P47mjauzi44XBqmlDYZL1MGebrr8LCFO1dfr12Ob78yyLLGbjk9tqTPoKxsOQVGyuP13e+iAFrP5EB4nq1VTApoK5ruoy0Vap2HYH2dIBXfSlEja6hHUVSK1DpxZ9NFr1aNmoF6m3oy+Ln95zlLP3H6Gw3uHgN99m3XVYVzNt3ZHU0EKJJP2qBRMN2U0FdABt2Y3TrXHaVQapgA5606/bJZTIjsvDLE7uWcMo7cLyxmYDumyrMD1eB25mStYZkIrVHSW+/tvfpuB4kfrV8SiqwK1s+NXNoTtLeCyqgC5ehsRCibc/dZT1U0scOl/j3hdex1sImVtUPnUrPt6KusblaK45S2YwN2e0CNOCifmkVZhOvxaK8XqcfnUK6fSrDuiEmWIdIJSI3+x8QcFAocQMBnVA7E04zdhsYDYOcu99fYJC8z4qpeSbv/MMITJSGBtCiTitKox7trEeqO8QiYzX4/Zhwkm2qU2BlIY4IrmaQchrM7ZX/O5mI7CzmBqcufcqzt5/hNJ6h+s+f5a2FUpYWAxF6MC5Tx2jdmqJ5XN17vvqGbyynTsWFhbbj5kI7OQsMHZbFUyEmaeEIUKJmJkzLU58P2bnEsbOB8XUxSxdu4vUaddGND6oh/gb0RNPZ8OjVY+eVhsN5VPX8ajeeTVv3n8VhfUO137+LM1mGDN1sU8dIXXF1CV2JkEq/WoydRCxdCZTB6P51O2GUMIydTuPcTpL5AklsuOyPWAhsjbJiiccw+KkINxUClYvPcdl5fASReGx/t5G1GUi01liHpd5dcy8dFiMO0pE17UYSi5+8mpqp5Y4eL7GPV95jeVih7mFaF6WlqPPvrckcBVTJ5aUX918GRYU+zY/H1uaaH86USwlKVYvSb/GnnROknIVpjhCs2h6Wz+hRPxG97c22UvpVxPlGWDsBmHbxBE50zPslwkxtwvJ8uElpJSsv1eLBAwZn7rQSLUGhHG6Ne98JnMndV9YbasiRXKd6tyOELH1iSMh2KN9YfMw2zPPYlexcnaDhYtNrv/8WSuUsNg3uP+xO7nr0ds3fbwArji9zsqbG9zz5ddw/b37hWJhYeKeR2/n3sfumPRl7DvMBGNHqzn62HGMgncCQ5k746ber7ZOj8th7KS2LtHn8YOEvet0kXGNnaqra3WRWiDRUtsaPoFKoQaN6Dx+3aW9EX0cmo0CzZZi7PwCvufQlA6dywHHHz/LhiOouw4bDrFQYkM9G9XwaRhCCVAsnVFXZzJ1ejlMKAG9dXVWKDHbGKWmLk8w0W9Mvx6wENXQZceZdXWOYUasjWm1WEIQBWclx1NCCdUXVrF0c8I16uoEC4pGWEACguUw4MBrl7jutUsslTvgwvxCh+Ji9FkvHIh+D3e5kDB1qrMEC3NJD9i5ucSEuGgIJcwesIAw+r7miiPMHrB5QgkTA6xNpsyAeNuhLWx2GjtWJzeGscwo1iX9xoeid7sWReg5V3S8lFBCs28miyekiLcPuq6I2YtbSgDRnA9kmpELMte9HWIIoeQc047ZCOw2NiZ9BfkYNYjMfpAyx8l+SthQpscHQbwtDuCMwE52/Li7hGyrIK4TxAFdaARzfiO6KXcb0Ueg3fRot5RQolVgI4huaq/cdxWXb1rm6JfO0Woplau6n2+IkLruKKGDORKfOrNNWMvwqduMUAJ6069WKLF3sZ2twvS4vM4SOu06uFVYFNqVhEdZuHFAN49Ov7oskAR2y6FEOvDezx5FuoJrvvI68170WZ9X6dfycpeiMrVzl6K55iyXEAs6BWumX5POEiJWwBpCiYwCFreQahUGOZ0lBqVJR+wsMfT4GcfcjKlitxJwDPom04EXpO+rsQAidZ60EELPzKLw0uIIkdzb9T3dnLd62ZVB8kBmLPV+/QqBEUQmKlqRubr9g70xAy12BG/cdyVvP3Al0hVIZ4zHPwuLfQzpwMVPRm3CQlcg7V3WwsJiFzETjyRyoz7pS+iPUVi7IYwdYZJ2TbFFfpAeL2XCyPkJixevdwJkR7N3iuVqh4QtlW5tRMGZ3/TotqKn8ZilaxZodaP1uixw5r4jXHzgMN56l6u+eI56I6Cu065CL8PYn26DhKXTjF07x6dus0IJ6E2/WqHE7GI3e8DqfVk7E7MXrNkDVjN3JeFRdpJUbFmoVKxm6tS4RekY6Vd4/2eP0ji5xMq5DW578g2WRIf5ecXULUSf/cJSiLsUHS8WVQp1oYzQQok5w5tO+dOJYjnpAasZOSPtGgsmDHYuTpc6TrqzxE71gN0jbB1E1jXThO2+O/VLAWe3poQSpp2JsS3xn9P7JL6RMi0Lj4AwTrW6ilMKCOkSxOujXKNO8kKaxdtuH7tZxt6ZhRbbhrP3HebiA4cprHc4+sWzFKxQwsJiKKQTBXXrp5ZYOF/ntiffsEIJCwuLXcd0PZL0wUQZu6wNyUjH5LB4KdFEfwZPxnV1prVJmOxT7FzM0vlhws51QmRXPTkpL+KgJfCb0Z/Z70RxfKfl0VJMXaujBBOBRx2P9lKBc3cfxl3vcvhL59loBuAKNoSMmbqGqo+oS5+GetpqxgbEfoqpg6ibhGbsumGwKaFE9BbKHRFKZMdb7BzGMSHOju/XWSJrbWJuS5ZOD4vnCicWSpj1dpq5KwqXIi6vffMMZSeqr5vDpayOn5d6KVgMoXVigdrJRZbP1bn1yTdZkZFQYq7cpbygTIiXos+8u+DgKKbOWVCM3PwclBUjF9fVlQ1xRCGpo4v7vhbSJsSQ31nCtDgZZm0yAHvV2iQPi1PG2I2KUW088kal2TlzbNr6JCRdY6fXtaVIKCWecDj71GsESIrCJZCCQCRMHaSZJSeHhXcNNjBm6WTSC9qstZNCpMZhmBZnhRR7HTPxyZVrU5CKHeeDkRMMynBQYCcTUUQqsMts8yXS1+IJtfRlHMSFHUHYjaaK39ZLl24n+sLqqGW749H0VUAnVSoWh4bjQD3k0Fcu0K53qTf8OP1aJ2RDBXRa9dpMCSWiL65W6Pe2B5M+XRXEdUI/Dt7GEUrobVYoMZvYTQWsI0TPuKwCFqKUq95WdDwKGfFEUaVeg/faIALKKqibNxSwAPNh5FW3+PoGB79yjuvOXsINJPPz0ed/br5LcVF51S1Hv4ezXECogE771DFXQsyp9bg9WDmtgI1FEaqLhFdMBXTRRkMcYaZXrQJ2LCzJ/C4b04hRZHymCGLQ8dlzSWS8LRXMyWTdJ31f9oUkQBK+1yJEUhYuAQ5d9H1eiSPM+4I0zx8tXeH0dKtwhEh87OztOxf7b7Za5OL9W1YICtHHofRWE8emXy0shkI60Lx1JX58OPRqzaZfLSwsJoqJMHaVSqUA/BZwPRAAf7larb7Ub3y41tr8i+1wSC8HnT9vX6oVrIy3Zc8jAwzGTm3zQfrqSUXb2QUOgUqxBl2B70dPmTrt2u26tFW6ta32taRLSxWAN4TDO/dewbsPHKZ4usbyU2/TcKChBRLqghsioKkYNjP9mvjTqaXs0tV2Jkb6VQsmQhluSigBvenX/SiUGHfuzAK2IpgARrI2ccgXT5hdJmKbE7W/LFyKwuG6X67gIHjv8dPM4TCvnofnhKD+yFHq1y9SR3LkJ5dZwKdciD7rpbJaLvp4i4ptWIhew5kvIOaUaEKlX0W5DEXFyCmWTqTsTIoJK2dam2g2rZ+1CfRPlW7V2mQz4yeEcefOygA5czicXJ4Y+t7Zkq+bGFlxQtq6JFlmxREhCQPok6Rg46WU+Eiu/uWbADj/+Mt0RYgmDf24SwS5fnumH16WkRcypxNNzi9tetsJsb/EFZNKxX4a8KrV6ocrlcqngP8N+NV+g/3L3Z27km0K/PI6huXx4zLMWQ9FErzFSwFSB3Ei3hYGapta+l2HwFdpV9/FV+tdFcR1AoeOSik0VTDXEg4NZV9y8d4rqN13BU6tCz98l5oT1dI1MmnXhgxoyXQ9Xcuop2vHQZyfUsBG25K6ukCGufV08TaSIG4zPnX7oJ5urLkzDRinrm5QyzBzv1lXl92Wm4rtY0Ac+9gJNw7oiuiUrEsZF9dzlSrWZV46lGVkYdJ6+Cjd6xdZOF/nmhcv4SKZc33m56L7VWlOpV8XAtwFFWDOq+BroRwrX/MUsJSMbXEQ58Xrwqyd02lZUwHbk4p18hWwQ7DHFLBjzZ1lOeBzO2W3kGGp2Lz92u0NkkA1r64uqqdTtWrGOB00pQI7HewJ6BLieg4SKOHgkFauZy4k8rlTm2LVrAyTgM4stZC99wWRCfzG8fXLeuDNOiYV2L0MeJVKxQGWgR2M3Cz64dI9h6jddwXuepeFJ8/Rrtv06wzAzp0JQzrQeeQowfWLFM83uPbJ8zb9Ohuwc8diX2BSgd0GER3+EnAY+MVBgzuXeqP77cPmOXU5hI9Ps3PJ2CxhJE3GTj9thIIgEKltYSAIguhJJlTnCwIHX23zQ4dOqAQSKo3QwaGtnqxb6mmp6Qjev+cgjfsOI2pdCl8+x0ajm7B0IqSlLkinXVsyoKEEEpqd68gg1VEC0u3BtGDCl0Gcas0TSgRhmGLqwAolBmCsuTNJDGPqRhFLmOPyFLAOoidVK0ROKtZg7FKqWIOxS0QTirHDoSwivsBBMCcFJQTBw1FQN3e+wdGvnGexGzAnos9/uejHKdjCvFLAzgucecWqacFEqQBzet1YKgWs8LQgopAWTMQqVjfZJjLpVuFsrmVYhoHbg4KJsebOwWB77he70eSy35WGOXMwTyiRSsHGrFzC0sXJJTUwEKhvBsXYCd0pIhrQJVLF6lcvCwchQWSdDJDx67lS9vjcucJJFLTS6Dwh0uycRRpjzdxKpTJvrF+xhdf968BXq9XqzcCdwG9VKpXyFs5nMSbc9S7OWofSk+esUGIXYOfOHoGM5k7hQhTUOZap23HYuWNhMR5GZuwqlcrfBa6pVCovVqvVvwH8L8B/tsnXvUxCg18CCkBfbXljtbjJl9k6cmvnBo3PeYLox9bFTFwoEqZOF7ga24IwYe70uh9qPyCHrpaO49BVz0iapesIQVvV0zVV2V4baL1Ww3ljg7oMQUQsXVM9GbVkSDNm6nQ9XRCvp7pIZOrpfBmkmDqImDntWScNGxOzrm6YUMLcp/fH61PO1E1y7kwrRhVMDD2PUVvjGPV0mivI6zKh2QBPuDFj5+HETF1JL3EoGdVARQTzUrD8R++BI1jsRp/fMgFzqhdsuexTKEefdW9esQ9zLkLX1pXVcq6M0ExdoRAvY086T92aPUMw4RbS3SUgqp0za+tgtF6wA6xNhmIXa+smOXdWdD/uCSMckdkedJczmbthLJ3MbAuM4zUFEAjwdT9Xg73Tb25bCFyZ9G8tSidKjmkLq7i2z4m/8wLCeI7qmj3Ts26gUALBsL+WnsnBHqmjG4RxUrEr1Wr1lyqVyqcrlcr/uMXX/T+Af1apVL4NFIG/Ua1W+5rVbayXtvhy24u84M1Enh7DPCYO6MwgL/blUR8+2evVEyAI1JdSPKkQsRdQVwi66mXaIpl0LbWtfvdB5OEy/lPn6YQSJDRVsWtLBrR0YEcilGgZ4oh2j9rVjxWuOrALZNhjQGyKIyRyPwolJjZ3JoVxjYg1esQSiKGedcmxOu0oUgpZyChgnUQJG5sSC5cCWjyhlsKhLB3C6hreqWW8uw5T/uEl5pXHZFl9SZVFQKkYff6LZR9vLknBAoh5D1FSwVupmCx1QGe2CXNVnOEZggnPVLsabcGijb1CCRjcMmwIpiwFO7G5c0DsXAmeHDFYG4ZB2j/zNVI2qoapb5gMBqLvmDAzLhAiDoZ8bfgrEg86X9LzveMCbQHhS6sEwBwiKk7VsZmh0NWCCxcHV6tYdfpViDgFq2GKJ/T8t352aYwT2HUAqtXqk5VK5Qjwl9jkk1O1Wt0A/uyo47/6Z+7I3X7T829z6qfvAvDHP3OS969a7Blz6N069z/9KgCv33yY6p1Hc8/1qd97HieU1FbKfPfRm3LH3P3M6xy5UAPgW5+5hXa59+279pX3uflP3gLghQeO89bJgz1j5modHvrSaQAuXrfCTz9yvHeiS7jniTOU6z7dksMf/5mbsrsBOPpH77F0Orqmcz9/jPahUmq/BNxzDZyGT3DvFdAOEP/eKUpqQEGNXPRDzj3+MgDlK+c59fD1qfPoMOrZrz3Pe++uAvDor30M4Yiecc/96BVeeDb6/T7+qfs5fs2Rnmt/5+L7fP6JpwG47fYb+chH70z9Yvp8/+SffI52p8uBA0v8uV/7dM97CfDkV77DmTNnAfj1v/AZlpcWesY8/8JpvvnU9wH4xCMPcPttN/aMWa9t8M9/6wsA3HjDNXz65z+W+3qbwMTmjsUW4YA4XIZDJcJWgC3p2XVMbO488+duyd1+7bPvcs0L7wPwwieuYfVo7/1m+WKDO77xJgDnbz3E6/cc6RkD8KHfeRknlNQPlPjxL1yfO+aWp89x6HwUf/7gV26gO5d87+j75NXVy5z8wTsAnP7w1bx7aqXnPOVahw9+/nUA3r9+idc+enXuo3Hl8Tco1n38kkP1z5xMjdGf/yu+/x7lMxsAvPfYMbqHiqnriX7xOs5338FHIu44xPwtB5hTI5RPN6EfcubxnwIwf+UCH3jklDqPjM8nkfzwq8/x3ruXAfi5X/t4/L2TvA+S5370Ms/96BUAHnn0gdT3jr6udy6+zxOPPwXA7XfcyEc/clfmPBH+0T/+Pbpdf2q+d/7O3/vt3Gvoh6GBXaVS+ZVqtfo54P/V26rV6j+vVCrvjvVKW0A/hqzV9ai1ow+UHzi547qBE49p+V7fc623S9EE6xT7jml0Cqy3o8ApUKnS7MRoBS61TvR6ndCLGTgTvnRY9aMxDTUmb4KtyQJNKejiJHR8ho1uOoLQVSyeED00uwTCw0W4cgVZ6xC8WsM5tZTaDxF7VldMnBu6qYmlR0p0Fwk/s6+XYfPDSDCRZdXM102xc5mAzjzXrGIa5s60YVgKdtA4yPes62kf1sezrjcV68QWJ6bdSUntLzkOpY8fg+uXcC80mP/qBeZDKKvP5JxK/pS8gGJRpV+LId58tN+Zi9g3UfaSFKxi7EShkHjWeWqc6VlnplxN1i2TdhWpbUM86+I3cfo966Zh7og+nRrKBZ+lUtTux3PD3HEFN4zHlD2/77mWS22cUCKKou+Y+WKX5VIbKQWuI/GNcbE4wQ1YKkavV3QCnJxzuUKy7HUJJTQdHyEM6aAxfAGfMj4dJDpvpHfr05ZDyaJqdekh4xStME7nSEFRRi8SWabkw820B8uDEGLg/ui1xUgp2f0AMSxVValUniT6m/xGtVp9Y1euKoPvXP2r2/7tvutUuHkM2aeNXvpbkqRb9Uv4QsQ+QoHa5xuUeFeYlHi0bN51kODew1Dr0n7yTaj7tERI26ingyj92tHr0qeja+yM9KtOweoUq9kqLK+ezqyl0wHeZhWw5vjsuDzsZAo26F4Y6cMzDXPHLRybSGTcLxW7GTNiMxXrGulWUPV0at0z+74a6VaIWoYlPWCjZdkpMKdSm3OiwLzavoQHjuDKR66hcN0y1Lt4ZxvMf+cdDoawor7QDiqV+FKhw9JiG4D55Q7lK5R/3cHotZ0DZZyVqP5fLEdZBbG4AAvqCV+1ERNzi1BWLcWKellOvOsKxXSKFqKesEYNXrTRGZyKHRLY7aR3XeHKG2Zm7vz4uj81VU+V45YA5ZX/mOMkIiYektSoiNt96e8vHxF/3+ga7iBTAqTruOMaOyeq6e58+AiBkDjffYcWkpZyXjDruVuG80LLcFwAVQJklP5E+3q/dzSREI0L1O+Z7jcujfVov+z5npDIgd87ekw/TMP3DozA2FWr1U9XKpU/C3yjUqn8U+BvVavVXZVRrsrCbr5cCqMWrmrk/VnzClfNsaFIB3TJtvQ4XyRPI9q5OxDJZOoKSUdt7yLhyjLOvYcJax02vvwGrUYHBHRkEth19NIwII6CuGRiQbqezrQzya2ny4gjsnV1e1koYWIa5s5uY7O1dYPOl7U4gXRtncjsNwUVid1JL4vnCTdm7wo4cW2dJwRztxyicN0y4kIDuVQgPD5PWUIplJTU57soFGNX9Cloxm4uwCmr61ClGqJUSGrrCrpzRCEtkICovk4HZ6Y4Il4vpGvrIPo5G2xlzYhHxDTV1k3D3Flcbu/my8UYRbA3KMjLE+ulAjtDjGeK9aKlkxLrAXRDJw72fJnUeHfUeleKmHXTNXaEAhxonYgeaEoyIiXi8wh9Hhkf6yLw4u3RmyCk6GXpbT3EUIxUY1etVv9tpVL5MpFT9w8rlcpvVKvV7+zspSVYc8cX/U3SOyi+hj4BXW+T5V7n76hlS7TuG+PigC4uZlVBHJEySa93hYT3GohnLtA8VyOod1PBXDsO4tSSgK7UQZ6fUrnqbSOxcxmhBPSydJsRSmTH5WGagjqNSc+dacEoKdhRlLB5x8SqWCOYMwM6UIXZpAM7F2GkYh0K6vgSDuFLq4Sex9LzazR+9ToEUJBQlJKimkdFN1oWCgFeSQkmyhKnpFKrRZ2KLSBUQCcMBWwsnogDPC8V0EUnLKTTr2YKNt42Qgo2u29KU7AmJj135g90Bg/YpdvNIL/UvCAwN7AzvFI1wsBJGLvQ2GZ4pAIUA5F4pRqeqZ7qaNTGQYR6vqmTO0Ao4jRuQUIJw+9OvXmuEHiGEEIHbSmle877HN8r1D5hjNP7JCJ5A/YZRpqtlUrlduA/JKp5PA48WalU/pHpL2QxPXBOJEWcrZcvE9StwfqkYOfODMERlI4pAZYEfnIJsU0mtRbjw84dC4vNYRTxxCrwFvAd4JvA/wy8Afw14HeJ+u/tKC6700W9br4vn9qf+XX6NVkO0w8l+Mg4FRszdiRy8S6S8p2Hmb/nCJ0fXWT9R+/QRcbpVs3IdQkTpk6nWmVorCcpVpO506xcbtrVSL/q9Tjlmkm/zlI93VYwDXNnt7BdKdisZ1WexUl2XDZVawoqzPSr3q+961zh4GkWz3E4/Mi1lK9bpvX1s4Rn6xQQlCQ0iIrCSzJKw5ZUnVDBMwQTJWXPUARRVKxaSaddvZRXXfSCrlEnZ9TIxUydwdyl0q45XSay72E/di3rXTfFmIa5Uzy0zfeVTXpyDEzNDuhHnjquTz9yqbsbxYydiHuT+6rfeBgIox+58kfturjqoccNXZxYFac6I8mEnYvmTvSzLqpypZ6fRtkEImaazP6wwlg395nbLNIYJRV7U7VazVMi/e1KpfKXtvuCLDaPuaC/wo0AACAASURBVDsPs3DPEYJah/rLlyd9ORZ27swEhCO4+uEoqOu8tUF4oTHpS7Kwc8fCYtMYRTwxSF7+y9t4LX2x6kwnIwODyyzyHrRCTCaqd1wimJDxumbkQkymTit8JIGULN11hIV7jtCttTn/5Bna9ag+pCtDujFjl9TV+ZltWZYuZucMoYTu8aqPyapdo+sJU0wd9NbV7UWhRB6mYe5MKzbTZcIca6pn80yLs0/5Qoikxs4s1nYcTjxyksXrD9B5q87618+yEjiAoCij2qDC5Q6elNE6kqKjmDpP2T14AU4h+iyKgohr62IbE9dNRBO6Xtg16uncHCGEYtdERhyRK3DIE0o4/Rm9PPQVTkygtg6mY+54Bzch2ttmp1zZ73x522MmTuZvi+vt9L4QlaRBJWaQfsLYeaqzShgI/LZiuRVj5zgSt6s+810Zv06suA0dAiTlS218BAUJrkzafLjCmIMGE7cbDFzSwWK6vz+2gnEMintQrVar23Uhg1DbFSnEYIxzBTLHQ2iweEL27DMd4AIjUIqDvVgwITlw1xFW7rmKTq3NmSdfprvRMQK3MG6i3DWCNTOg09tMUUQ3I4owU7GxwjUjJwcVxG3CzmQvCCXGwW7NnWnAdt2sTbWriYHednEQ58SKO32OgutyzSMnWbr+AK0LG7S/cY5CkHz5FGT0hXTkKxdii5MCIZ6TBHQAbkHiFKPPo1N0QYsmiur2WiwgTIEEgOcaalcj/aq3DfCuS62Pml7dinfdlGG35o6zsgttZEe5jw0IFtNBnEwvjXWpuqUASD+JwqTqdax7Hssg2R8q7UjYETiuGtfWNjoSxyBcZFcLLhI7lIKUXP/keTaUz2pBgqemrw48PBLvO4GZgo3g0CuUsBiO2Z7hFgC4JY9Orc3rT75Cd2OIksvCwiKCEDhFl/qFGu9//XWwQgkLC4s9gC0xdruFmpgOL+mRuyDI1CJ9jhx2LhqbFU8knFXMgBnrmsULCFn/w9dxf+TRbnUSJk+za8jEa85g30x2DiLmLpedy7ExCY1tWeuSUXzq9qJQwmLrMNOq8bY+jvN54grTIiHrY5frbRdI3v76awDMB06coi3ESyhIycaNS9RCyaFX1ikIiadsTlyVinWLIY6yohNFB+FpBs5NluZ6dHDa0gTy2blUL9g+fWEtdgRipbdF1LZhnJRtmKeQUGnQvPNpuymDpRNhaLgQG8ydYueksZQdLQRS9/GOxFGfdcdT5zM+fmEoYu87vfSloCsFl29apuUIlk7X8IQwyiAiCGmw61IYTF1/Zj76va1oYhBmIrDbkDvvS7ld4ULY50xDgzwpU+OiwC4d7Jmu2FfffQwZSs79+HwUfDW7kRGw9rGTOcpVElWrXjfduhNH7hy16xDnbnNb/DuPkX4dFtBlx1tMDzariB1UW5e3L29bv3ZkWaWsbjMmHMHND9/Ie6ffp3uujiqXU35a0bpeujJav/TAYRwpueqVNVxCXBXYeQVloupKRJxjcpLgzTOWWbWrcEBfuxmsmbV15r5BGNRlYgimyZR42iAWdzCwy0NeAGdC9gZvyc85+6RM1kNjXZUV4AfRf0AEZmAXbdNL4YYIVX6AmX6NX0bE3nfa584LJC6Stx48TCgEB16pRTV26nDtdxdVsybrWeg2YaltfbztLBLYWT2DOHr3MU7ce4Ijt1yJW5gdCwMLi0lCOIJbP1HhihsOc9UHrpr05VhYWFjsCGaCsavvAmPXD6MwSVkMeu7KO19oCA6ScWYKNkltXnP3CY7fe4JmrcWPv/Q8jXYr3u8bQglppGqz7FvfVOsAds5UwOb13xvE1A0SSfR7T1L7LVO3Z7AZdm9YL1lTDauXWVWs57p84BMVDp+8gvUL67z6jdPMCTdODXmIlFoWIrbOjfzrEUTNzj0njIvG44JyD4Ty2hRekooVsdpVJCxeTFXkpF37sWdmt4nsmJ1Mye7zdO+uMXbjMHV9jpGpVKtikiFm5FLsnd4WBMa62tfxEV70fSvV51g6vkEBJQygp0UWQUigFeKazQ5DCsrTTkjwpMQzfOpM5s5k5LKip51m5vS9Za99x0wssKtUKv8d8BmgCPyDarX6Tyd1LbOCa+4+wXX3XUtLBXXtjcn0MrSYLOzcGQ/CEdzxs7dy+OQVrF5Y5dWvniYMwpky7LXYHti5Y7EfMJHArlKpPAx8GPgIMA/814PGN7fI2PWre9sJDGOf8gQYeXYn2bq0K08e5rr7rqW53uSPv/Rj6rWmGi9T7FwP00avJYnJzqWEEGZv10zNnEQm4okBfV/Na7dM3fZj3LkzSexGgbNDr3hCb9fLWx64katOXsnqhTVe/MpLlKWLo0QZMYOAiAu64xo7FLMQ/xziOjKusdOMnfBkIpjwnKS2ztEsnpfPzmW95lI+dsNYvE0EpfucgRt77iwu9t83jGXbDPqdcwTGTuQwdmS2ySBI7Rd+ELF2gOxG37HCdaLPMICTtKLMEnYyDHEVY+cFQdyRIhYW+RJXyrhXrAs4UhrzUl23MQe3er8Q1hYlxqQYu8eA54DHifoA/jeDBjfl5HudjqyIzUGPIW+fdGz2GHPcudffZvG5Rc78+A2aG0n6VUqZBGmG4MIUM2TTrpKcbcY4M2Ab1Z8udd0DVK9570cebFDXF2PNnVnAoBt6VglnqmEHnU8Y4olXf/wmpUKBc98/C6E0AkFSXzTaKFWf2ZMSz/gcusjIv0sHdI4O7IgNuoQjElGEmYrNCCWyxsPJL2woYONfKNNGbBxYVtLEeHOnPLfzVzQ0DZuz3zwmcQTu3W8Gdr6PMNYBCAJkN/puFYZqWzqRZVaesEao13MCw7uxncwJVy0LToinFbmAK2X08JQrnuh9IDMV7vrrIi9wsx53+ZhUYHcYuA74ReAk8IVKpXJLtVq1f54Mlg4tULtUR4aSF595eVfZR4uphJ07I0A4gsXleTrrHbqtLi8/c4ays7nbXeWzr7HC5B8uLbYMO3d2Gfd+9jS12Sjl31OY1Dv+PvBStVrtANVKpdICrgTeyRvcCnf+proTAdMw1indXqyX5br53lPccv8N/ODrP+b8mYtp1sxId/ZbByVwyLB45nl0etVMtZqs2zB/uuz12/TrjmOsubOT2KzNyaDzZW1KRjkm9XQPuK7DA5+6kyuPH+IPv/An1C7VU4IKs9F43NrI8NDSN0Utmii0Q0oiBAGuE7F2kPb0EpqCcPuIInK3ub3ben7BXRRL7H2MNXdEeX53rmoAayeHMnZxbjR/mxZHFAoJU6fZuSCImTrN3OEkHLm+E4swTPzwVPpVdMKEsSvIpAuLq+16JAJJsR1Q1PNO0pN2dYzvg36fbOtZNz4mdZf4DvBzlUpFVCqVY8AC0aSzULj53lPc+sCNNGotLl1cm/TlWEwP7NwZAMcRfOjRuzl26ipW36tRX29u+ZydBY/WgmUd9gDs3NlltBY8Onbu7Dom8o5Xq9UvVSqVnwG+TxRc/ka1Wu3bXqK9C4zdZrAZli+Pkcqe59b7buTWB26kvt7g6Sf+kA0llMgz+k2xa5l1yFiS5NTlperlTOYv55gsUzdq39fs2DxYpm40jDt39gJGNSZ2HMFHHruPE6eu5t3zl/ijJ/8EL3RTQgpIGADNyEX7Eu9V3eo5qruTvPDL1+IAH/lXLyGEjOuIkoNF2sbEG2BtEgsljOs3WDgxLiPnOGlj4hGxH42Jx547O8HYhSNOVWlYlsTH5rBy8fjEziRm+cIQPG1K7KeYOiBi8DTDpj4P5l04Zu7CpEOFKOplgGgnzLVmr7UVkOtIXCF59pdvIATu/P/ORDWtObd585NoubntwcRC6Wq1+pujju1M0MfOxFYEFBr9Ahx97tvvv4nbHriJjfU633j8u9RrzXTQlJfyNIOvrMAhJ4Uamq3A+gRzo/jTZfebY0b5nQcdY9Ef48ydacUg8cOgY/qlZRxH8NHH7uOaG47yzrn3+cFXniUMZEoxm21X5Biu9qY6L15K44tGgKsan8eiCb10MYI44/ocI4jLC9iy20yhw1ZSrjZd2xdjzZ1ieQevRKGvEnZICja7PwySYNAUT2jVq+cZogkjJdtNkyZaxZp6vaAQd6ZAq2c9B+GqzhReIp5wtGLcCVNpVhepfs6qYhPktREbFTZdm4blSKcIjiO4+sSVbKw34qDOwsJiOMrzJQ4dOcDb597je0/+EDe0wY2FhcX+xEwEdp1wOhi7UTAOq9fDZAXwtS88Q7FUpLHRzB2XTZFmt/UIHPqxbxnmL8v25TFxPbYtNv1qMSK2W2ih4ai+sK16m69/7hnazQ5uKHAzqca8dI9jbM+u63Gp4m4hlXNJIpqIz6k965zEaoW833mYP10ORN4xtuPEzmO7GbtRve962Lhs2rWPUCIrpAiDOBUrfD/+u8ohn0HN+GnfO1HwkB3F7LnKrsdz4v7IjifJtjZ2nGiumBSgyYDHKVmRX+i/Xd52+xV2Bk8Bbr/3Jo5dewQAvxukgjoLC4t8OI7DQ4/czdJK1PqpUWsS+Hu63NDCwsJiKGaCseuOWnC6y9hMH1lIM1V3PXAL9zx0G5ffX+fsv/pGLGroN76np2yGXTPr7aA/S5cnjohfo4/xsLk/79ryxubBMnUWo6If0+c4Dp/89Ic4eeMJisUCz3z9h6m6vawFCvT2lM2yAfExsXhCKgNVEET2DUL0shPCFE9EF5e92PT+7DhzfN6xFpNBobRz596UMbHxPTjI5kTX0JksnnDAUfVxToa5y16XF4UFolCIxvlBIgiKlwljJzwQusZOs9nCWKeXDSf+eRO1tpbFG4qZCOx8OZ2BXR7GCVrufvBW7nnoNtbXNnjyiT/oSTlnz5UXpGXHZUURqW2D0rh9grU8sUa/68u+bh5sUGexVZhB3YWz7/CHTz2b3p9z4xeIfHWtUcydLeiOUkeSk995m3n1pWiKJ5IIMKOKjdOyAzpLjOJZt52w6dWxIbwdCOz0d9mghiCZoC9Wubpesl8fb6ZdhVrXn60wTIK8lG+i6jaBUryar+klbcZiFa3nJp0pjMDOFAzpOaHFE9rH7gPPnKMReurBKEFeaGZ2ghkG23FiMGYisNuLuPvBW7nvodtZX9vgi7/7NBu1xqQvycJi6uE4Do/+wkOcvPEE59+8yDe++L34+2yncMXrNZbdzs6+iIXFHsRVb6yxHhQnfRn7DjMR2PlTmordDEIpue2DN3LfQ7eztrrB5z/7zdygLo/5ymPmsscMsh/pl3bNbhtkZ5IdO+h6hx1jsXexU0XPH//kfZy66RrOv3mRr37hGQgkhZxWYYOe/LONyKE/gxAfE+shks9xKhU7CKaP3ajoN972ft09eJsISPJSqOmT9t+lWTPzTy/D5HNosn15rF42Pet3k89RkCNADENE6KlDNCto+N2plGzKm9HVjLNAeEpI4QS9zj2ONPrByvjX6k3FWuwEZiKw22t49fRZrjt1lD/4/R9aps7CYgw8/+wrFIsFvvW1HxD4Ae4upBif+9PX44mQh750esdfy8JiL+F7v3gjgRR88POvT/pS9hVmIrDLExTMGqSUzM+XaTRa1OtNvvT4twaO78fIDRrXT2QxsFZvRKauH+M2ioDEsnUWW4HjOBSLHt22z7vvXOarX3qGgqN7Ug5my4bV7JhhYSKaMPdL2ksFuiLpEZv7kqZB8YA6OpH2SEkvLaYLbmF7zjNqjXi/moL4+8+oscuMFTKpsUv1lx2U7QoDg+VT5w68qM4OYvNi4bq9FinCqCsV/T/KzaUioRw8B8WQ/TsFR4htaTowjZiNwG5U/58pxgMP3c5d99zC5z77Dd595/JIxwz60A1SyvbsHxCk7UTaddCxFhYag9K1cbrUcfj5X/woBw8t8/i//X38dn9Py7hVWF67sZyuFWbQlxcA5oVbuZ3AhsVleZHgTgdzNmW7ZQh3m74e5Wh/a5kX2MmAng+YCHO87pw4sBNBFJBJ18v/nMVp10LSrUJ3qHA9EMqzzkzFCuPBRW0zvRtx1Hl0NxbjQQj1Gzgy/bPFzsG+v7uABx66nQ995E7a7Q6tli3CtrAYBY7j8Ok/9TFuvPlaGvUWfnd2jMotLCwsJoXZYOxmOBX74EN38KGP3Mnaao3P/puvU1uv544bhd3KY+b6HTvIa25YSrffOW3a1WK34DgOv/CZj3HTzddy9o23+eLjTyNDuS01den+lL3bnX77444TvZ9xkagr+ryofYaeOYzK2A39fhryt1cMmnBzxhlsXyJwcJP0rim40JehUshCBki0p51rMHVect36c6kFE4EhlOgan2m9LU7JmhY/yeUOaroCJmtnveh2EjMR2M1qsPChD3+QD3/0LlZXa3z2X3+N2nq9b3A2CIN+/37p2mGB2ahp17xjR309C4vNIAnqruPNN97iS49/C98PetqEjYKtqHOdnM90XlDXO2jIt1vfF3RA2BTq1GDkdPaI4/rVu2UDulSg6PQGfjJMhuiAzKixMxWwQtfdmUGqPliY3nZGDZ1werfF50tSsuZMyHtuSSnIt2A4Zz3rxsdEA7tKpXIE+CHwqWq1+tIkr2W7US4X+eCdN6eCOguL7cJenjtHjhzk5KkTvPnGW3z+c08hg8ne0a98dZ2iY9PAewV7ee5MG46+dplOaB9WdhsTC+wqlUoB+H+AoY1RN8NyTQqauao3W/zO73yZMAhZ32JQN0y5M27adBy2zjJ104dx5s4s4u233+ez//prvPvOJYIg3FT6dZhaFvqnZE2mTiC58btvsVRMamN7RBPDfOy2AuHaNO42Yqy5s93vu8nMDUzfuml2L3sZYZq9ixfxOK1w9ZO0bNBFZqWrrpfuTEGk2u5RwJo+jKltiXgiOXW6XOG275+n1unvB5j2ihSWldsmTPKO8beA/xu4MMFr2Hbcc++trKwsArB6ubbloM7CIgd7bu44jsNHPnIXhUL0pfTWhXfx/b1jTG4xNdhzc8fCIouJBHaVSuUvAu9Wq9WvjjJeqqb0k/4fhOHA/w9+6A5+9pMP8tjPf4RQym37P/Tacv5p5J0v7z1Nvd855+n3d7HYXYw7d2YBjuPwp3/pYT76sbt56MN37tzr0HvDE1LEPlp5+wFeu/8IL99zdDAxp4vJh9UBZtmPTdQNWmwO484d4Xo9/+O/2Vb/u170Xzj5/x03+e8Wov/xPuO/2iZ0faZwc/cPvR5zXM8bIZL/JuLPfP775wh4+Z6jvHb/keg0Y/ytBL32RBajY1J3lf8Y+FSlUnkauAv4F5VK5ep+g7czSNrK/zzowOihD3+Qj/3MPaxervHvvvQH2xpQxq/V59+g9yrvWgelXUdVvtqgbmIYa+5MO3RQV6lcz+uvX+B73/3xaGlUIeJxjhBJgXW/8RnPulGajesRb///7d17jFTVHQfw79zZZSm4KEXEgsCCwlkDFlwRWtmFjWKMbVKfbdKkTVrStDY18Y+mTdNo+k//aVLtH7WtSdUaBZSXUN4gooIgvgBdLBxeC0uRh4Vll+U9O7d/zNzZO8PdmXvv3LnnnjvfT7LJ7uw8zrDzY75znmIoOiYMyz6uj9c9w1tUeKsde/DJSiSMQL9KBi37cLAV9BLGtYEtG+7yAl7e75O2x0z2LZooDJKJhEPItN13Lsy5C10dE4bhhBja9++X/bJ/iOrvAxX5p2SOnZRylvV9tsiekFKeUNGWINwzcwpmzboLZzvPYcGCNRx+pYqJU+0UhrqlSzbCTPMDA1VGnGqHqBhud1Km6TMmewp1bk9tcMvPwoq833toT5T/DqSf7z00Oxfqlix+K7NQQqPeLU9z6wufl30rDY2ec9UK+m9kbWHi8CIyzXT+4xWuszDQt7gitzWJ/XrW3nYG0OuwSMO+iMLpedn3qisiYf99wbqMvOtZd1Xm24ebXnbKUB7spJStqttQjqMdJ3DyxGksXbqRPXUUKt1r5+CBo6irG4ClSzYilep1NQRLFATda4eoGOXBzo2ge7mCUFOTRCrViy+Pf4WX/7U88Pv3ezix2141nvdKKhiGkXtNtbUdQFvbgbz5cqr09+hGovSk70RhzwZ74OIh6O1O7FucOL1GHHrx8nrv0vaNha3bXLtyPJG9vplG36bF9t456yb2NhRuSmzntN2JvV4devZ8zUWlwGgR7PyGnEppnjkVjY3jMH/BGly8eFlJG/wELg67kkqGYeCRR+5FOp3Gin+/G8ieVU4r58odsim8de3FFJK2EydcnT4BlBfwHMe0GBhD5frkCfR/qoRdf38/0zZ0mrs/h5Bn2H5nH4IFMqOv9u+tb6z7NI2+y+0Bz6lNDidO+FV3KYXeNHviw6ZFsIuS5plTMXv2NHR2dqO2tkZZsCPSiRXqGkUD2g8fQzJpoDelxxnQd795MG+DYiJyZ/aKvUU3KKbKYLDzwB7q5s2vzOrXoHvK/Axjs7eOglQY6hYt2oB0bzpy+1QV65/g0BIV5bZ3z6lnr7DX7JrFE/m9d6bRd1ne4xcslEgYBsxe2+PlziFO9d0md3uXvXQepkz4qZlo/Y+gL/btu3RtqOtR3SSiyDMMA48WhDrdTpQ4M2owTo2sV90MIu2cGlmPM6MGq25G1dGix051D9LQoUMwc+admVA3bzW6IhrqyllkovrfmPThZaHDxIlj0dg4zlWoK7XJcCleb+/2bMq9rbcgkTDxwOI2ny0jLXmZ01j07Fcbp569wl68hOG80CLtdJnt8Qvn3dl78GzPJdfzZ7+fYhxPnTDyFk0UnhVr2TmzAaaZwD3zJYwILoKMKy2CnWqdnd1YtGg9Tp/uUh7qgl4hzEBHlbR3bzuWL98Eue+Ikp66UmEvsCELbtVS3dyEwP7CnxX27AHPuj+H2yQSRma1rJs25YZf066Dai74ub2+y1MoKDwcii3ijsm3oaYmUxiH2o8pD3VEOjAMA1O+OTH38+4vDmo3/EpEpCv22PWjpflOzJ49DQ0NI7Fy1eaKPIaq/fnYS0eVYhgGHnvkPjQ2jkOyJokdO/aoblKO18UaTkNHJTsxQu69cDq5gCLK/rdy6nGzD9PaT5Yo3A6lvyFZpyFYt+2xn0ZB2tMi2IUdgFqam3ILJd7b/GkkN0j2gkGOwmAPde3tx/D55/tUN4nIt4SPfdzMwtWq/d+5/Uburmu7Xm641Om2ufvub+i3xD52AWA+VIv//AVampvQmg11r81bxeFXIhcKQ93CRes5/EpEpIAWPXZhaZ55Z2xCHXvpKCyGkcgLdW8sWqfN5sNuTFl9GNdxg2JywamXr2QvntNCCacFFbnfGdfuY1eiPaZp9B1d4bU07c/JYy/mPRv2o4cbFIeOwc6m5/wFnOnsxjzNQx1RmNJpE+cvXMyFulSqN3KbD5dj8NnLqK/jCTNEXtV3XYJ5mZ0MYWOws9m1S2L37gNaDSGxZ46iYM3a95FMJtHbq0/tuJU2EkgbCRjpiNZaAGd6UuXYe/GK9t7113NnX0hR+PtiDMP9QorC27m5zIVya6fcc5+rlZJgJ4SoBfAygAYAdQD+KKVcoaItLc1NuHHYDVi+4h2YphnJUMfwRpao1I41p+7of09g+4eZjXujFOqKvSF4favY/sOJ3KA4BqJQO7mhUbeLLErdn/2YMT8hzuInzLnYu/GtxybnNiim8Kj6qPcjAKellC0AHgTwvIpGWAslRo26CYMGDVTRBCKvlNeOfaHEbbeO8XQSBZFCymuHKAyqhmIXA1hi+zkVdgMKV7+eP3+x5G3Yc0YRoLR2Cle/vrFoXfXWBXfc14232im1Z4fbIVHHuy7Sc1d4pJgTpz3tcu112XNezhC+x9d+QvMtw3SjJNhJKXsAQAhRj0yhPR3m43NLE9KVytoxDAOPPTonL9RFceoCkRPV7ztEYVE261YIMRrAOwBek1IuKHZd0zQD+xoz+uZcqHt13kqc7Trn+rZEUeCldoI0Y/oduL3KQh17GuIl0NpJGNd+eb4LF7cxkvmnUpQrkbSdIevwWKQ9VYsnRgDYAOBJKeXbYT72kY7jWLd+K+S+w+jqYk8d6UVl7Xz4URtqa5P4YPvnVRHqKF5CqR2vK1f93E+poVornLlZTBHUimpOS4gUVXPsfg9gKIBnhBDPZC97UEpZeqKbT+PGjUJ7+zEAwEcf767UwxBVWqi1YxgGRt8yAkc6jiOdTmPzlh1Vt1iiYccpDKwJfRowBS/0951qJz47jksp7qoWNlVz7J4C8FRYjzerpQmts+/G25s+xNZtu8J6WKLAhVk71pw6MXEsFry+BoeyH4yqzag9Z1Bfx5MndBfq+46bBRC5qwa7Bco17QCyQ6/hfzhp2Pc/nLvcd/IEd1wMR+z/na1Q13m2G7u/OKC6OURasELd7Y3jcOTIcXQcPam6SURE5EKsg5091L362krOqSNywR7q2tuP4fWF65BK6T0UWc4O9m1zxuDj1vEBtoaqgs8FFXHycet4tM0Zo7oZVSe2g98MdUTexTHUlat7xCAkEkVWx3LiOBUT1IIKDZ0ZPhimyfoIW2w/TgwZch1DHZFHNTVJXDf4awx1RESaim2P3arVmzFo0EBcuHBJdVOItHHlylXMf30N0mmzakNdbD/tklolFlRUdBGF9fiVwB7ryInV/2EtzU2Yfvfk3M8MdUSlGYaBhx+6Fw0NIwFkwl21hjoiIt3FpscuN6eusxs7d+3F1at8YyIqxT6nrq6uFocPf6m6SfET1CawVH3svXzWxsNpbg5OxcUi2OUtlJi3kqGOyIXChRJL3wz1IAtt3HD8PGqS1TfxnTSXMACofd0OO9mDVC8/2IRN+2DH1a9E3nH1q3uTNh3lBsVEPkzb3J63QTGFQ+tgN2nSrQx1RD7MuW8GQx1R2OK+9QnXUUSC1sFu7952fPLpf7B1206GOiIPtn3wGQYMqMW69dsY6ko4OmkYBtamMH7PV6qbQhS+MuaIHrp9OC5drcHoInUylwAABSBJREFUL04737UJhsEK0HLw+8ZhNwAAenvTWLN2C0MdkQuGYWDo0CEAgJ6eC1i1ejNDnQsdU4dj/+SbVTeDSDv7J9+MjqnDVTej6mgX7Ga1NOGJX3wfEyeMVd0UIm1Yc+rm/uRhDPv69aqbQ0QBSlT50WWUT6tXg7VQoqu7BydPOXftElE++0KJU6fOoKv7vOomERFRhSiZYyeEMAD8HcAUAJcB/ExKeaDYbbj6lch77XD1awVwTpCW/LzvRJ41/61Sp1WQllT12D0MYKCU8tsAfgfg2WJXZqgjyvFUOwx1RDmeaodIV6qCXTOAdQAgpdwOYFqxK0+YMJahjijDU+2MvmUEQx1RhqfaIdJVwjTN0B9UCPEigKVSyrXZnzsAjJdS8p2HqAjWDpE/rB2qFqp67LoB1NvbweIicoW1Q+QPa4eqgqpgtxXAdwBACPEtAG2K2kGkG9YOkT+sHaoKqk6eWAbgfiHENmTWmP1UUTuIdMPaIfKHtUNVQckcOyIiIiIKnlYbFBMRERFR/xjsiIiIiGJC1Rw7V2K5U3iWEGIngK7sj+1SSu3newghZgD4k5SyVQhxG4BXAJgAdgP4lZRSy+3RC55XE4CVAPZnf/0PKeVCda1zxtrRC2snOlg7+ohr3QDl1U6kgx1sO4VnVzE9C+AhxW0qmxBiIABIKVsVNyUwQojfAvgxAOsg0ucAPC2lfFcI8QIyf7dlqtrnl8PzagLwnJQy6rvWs3Y0wdqJHNaOBuJaN0D5tRP1odi47hQ+BcAgIcQGIcSm7H8eujsI4FHbz3cBeC/7/VoAc0JvUTCcntd3hRCbhRAvCSHq+7mdaqwdfbB2ooW1o4e41g1QZu1EPdgNQV+3MQD0CiGi3svoxgUAfwbwAIAnAMzX/XlJKZcCuGq7KCGltJZcnwNwffitKp/D8/oIwG+klLMAHALwByUNK421ownWTuSwdjQQ17oByq+dqP9R47pT+D4AB7Ivwn1CiNMAvgHgqNpmBco+t6EewFlVDQnYMiml9VyWAfirysYUwdrRF2tHLdaOnuJaN4DH2ol6j11cdwqfi8y8DQghRiLzCfG40hYFb6cQojX7/YMAtihsS5DWCyGmZ7+/D8CnKhtTBGtHX6wdtVg7eopr3QAeayfqPXZx3Sn8JQCvCCHeR2YFz9yYfCK0+zWAfwohBgDYA2CJ4vYE5ZcAnhdCXAFwAsDPFbenP6wdfbF21GLt6CmudQN4rB2ePEFEREQUE1EfiiUiIiIilxjsiIiIiGKCwY6IiIgoJhjsiIiIiGKCwY6IiIgoJhjsiIiIiGKCwY6IiIgoJhjsYkgIcZMQoksIYdguWyuEeFxlu4iijrVD5A9rJzoY7GJISnkKmd2pJwOAEOIHAEwpZZx24iYKHGuHyB/WTnTw5ImYEkK8COATAAsA7ABwP4AzAP4C4H4p5WiFzSOKLNYOkT+snWiI+lmx5N8WAPcCmATgZSlle/byuUKIjeqaRRR5rB0if1g7EcBgF19bADwH4EsAdyluC5FOWDtE/rB2IoBz7OLrCIABAJ6UUl5R3RgijbB2iPxh7UQAg118PQVgoZTyPfuFQoi/AWgUQrwghBinpmlEkcbaIfKHtRMBXDwRM0KIRgDLkPnk9LiUskdxk4i0wNoh8oe1Ey0MdkREREQxwaFYIiIiophgsCMiIiKKCQY7IiIiophgsCMiIiKKCQY7IiIiophgsCMiIiKKCQY7IiIiophgsCMiIiKKif8DkVjnRSdRmVUAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plot_softmax(ylo, yhi, n=100):\n", " y_grid = np.linspace(ylo, yhi, n)\n", " y3 = 0.5 * (ylo + yhi)\n", " p_grid = np.array([softmax([y1, y2, y3]) for y1 in y_grid for y2 in y_grid]).reshape(n, n, 3)\n", " _, ax = plt.subplots(1, 3, figsize=(10.5, 3))\n", " for k in range(3):\n", " ax[k].imshow(p_grid[:, :, k], interpolation='none', origin='lower', extent=[ylo, yhi, ylo, yhi])\n", " ax[k].set_xlabel('$y_1$')\n", " ax[k].set_ylabel('$y_2$')\n", " if k != 0: ax[k].axvline(y3, c='gray', ls='--')\n", " if k != 1: ax[k].axhline(y3, c='gray', ls='--')\n", " if k != 2: ax[k].plot([ylo, yhi], [ylo, yhi], c='gray', ls='--')\n", " ax[k].grid(False)\n", " \n", "plot_softmax(0, 15)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The example above assumed output activations that can be large an positive, such as `relu` or `elu`. However, the strength of the *winner takes all* effect depends on how the outputs are scaled, and is relatively weak for output activations that saturate on both sides, such as `sigmoid` or `tanh`, which is why these are generally not used for classification outputs:" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoYAAADUCAYAAAABU+KsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzsvWuUHdd1JvadqtuNJkA8SAAEQQAECUIoPsQ3RYkS9bBISrZk2ZJje7ysrMSeyHHirMyKlZUsOw9PJnEST2LHM/Ysz1qO47FnybJHsk1KsmhalihRfIiSSJEUSYhFggSJJ0ESJNDdAIHuW3Xy49Q55zundtW9/UA3ulEb665b2OdZt29V7fvt/e2ttNbopJNOOumkk0466aSTZLE30EknnXTSSSeddNLJ2SGdYdhJJ5100kknnXTSCYDOMOykk0466aSTTjrppJLOMOykk0466aSTTjrpBEBnGHbSSSeddNJJJ510UklnGHbSSSeddNJJJ510AgDoLebiWZa9G8C/zPP8Q5H+EwB+C0AfwJ/mef7/Zll2HoDPAbgIwASA/zTP89cXeMuddHJWSHftdNLJ7KS7djrppF0WDTHMsuy/B/AnAMYi/QiA3wfwEQAfBPCfZ1l2MYD/EsDTeZ6/H8C/B/A/LeyOO+nk7JDu2umkk9lJd+100slgWUxX8osAfkbQXwVgT57nb+V5PgXgIQDvB3A7gPuqPn8P4M4F2WUnnZx90l07nXQyO+munU46GSCL5krO8/xvsiy7TGhaA+A4/X8CwNpIb3WtMv36i76siy6DNl2WENvKQtCZYy3ohhpb/T9cs6jPU4Z7hC6Dfm79YEw0D68vzc1t7riojymK+nmXA+bpC/OUJWCr6/T71Vgdtlfruc/H9S9oPW3+T2O0oEPh96hLmqeaU/dLwOmpXz8eo92x7lc67Y+1PZUS0H3ljsvqGNWQJ7ddis2vHcP68Uls+/79VePcZCGunXTkEnftKGW2raCC/wNAolRNnygV9AGABCpot/O54wZdEs9N89h1eQ92bKoSuR3UHs8DhVQltbnT6vcz7zsV2u0Z96q1r/75a6EAPPeFZ6q5oz24VYAez+n2GLbbMam2/eB0PTq2bam2Y70+0NljbcfSnNX10tNAWn2ZUw30Kr3TARhBWc1Z6ZRGWunSxOuSJNKlJXo9o0sSjSSt2lONQik8ee2luG7vfpw3Nb2krp2fufQT7tophXYNLWir/kIlslLoL83A/aQ1pLm5X9mgl8bH7bz2sP246lq89zs/fTsA4Ot/8ZDYr21s2x6sfjY6u6Z0DiU04ipypdZufLBfmjv+m2it5XPUoS4eq6P98thi+tCMr51FjTFskHEAq+n/qwEci/RW1yq66NN/okvUGlS2LTLKZm0EVrqBRqA0d9yP9xEbcjVdixEorV0WgP182HhjYzA23poMzIL2Ext3cXu0npaMRWvsRTod7ZENPjgjrvCGn1tXG4NQODZjSjq28wG60MGp6r52d8/SfrVK5Y51qaALVbUrPLFzO36QXYb1mybwkw88gQWQebt2rCgyqljYSIqNwKAfGXaJZNC16AJjkuaJjcDAmKS9Bu2CURsbi9LciTB3Ep9/fK5QSIBAr4J2uPdwTL29B2Htqp2NQL9W1aYRGIuVLebalaZjGssGoRmryYDUgUEIGGMwqXSK3u3HbHVJUgYGodFpKFWNSTQqmxy6p/Hgu67Evks2oBxN8L7dL2ABZN6unRTeEafIiLDSVoRWupYSYUQpGEYp/FgtXLP2szbt9piuh8CYorWqd/vDxoxVwT5SKDdnQudg9yb1g1K1vZt7fHiNKkVzu22HY72B5vcan2OpNRI39XC6kj+ISpeQrmRddd52XwmND87BjtWgteH2b+cvqS2hdgBANJb1dq9zKXd8NhqGPwLwjizLLgQwCeADAH4XwHYAHwPwPQA/AeDBgTMV0/64ZhhGxpeE1rm+7UagH2J1RTifZLy5tgGIIO+hFfVrM94GIHnS2oPmbjIGBSRQMhZ1zVjUwtxepwvZ4HPtfVq3iNC/GBGcKTroDEM+9gagvRjLvoKuruYfXLEdT2SX4fwTb+ODjz4HXSxI1Ma8XTvSAypABNuMRQnpazEeZyKSESghhoGB2GBgSmhkEvcj088bUiECmURrJ9H6yu3R78e+S8YdI4t2lL1RK63cPryByAakb3N71N6QY8QwNhYTrevGYmRA+rW9EeiMReXfUxWig4nyhon9KhiUsGpPNdK0RKEUHrj1KuzbvAGb33gLt+3Zg6Q3+wfcDGTerp2UvutK2LpktHkjh/vVjbfYSAPmbji6dsHwM+cQoVp0zPeJGjqoNSHr9f1wu2Qsmj0lFYoWrqehA2PSGlGSgekMNQWv0+06+2hgA1M08iQjjnR81y+jPycbjm6s8n9jNw803aOsDu4PoaHdvScwEOdwyz1rDMMsy34RwPl5nv9xlmWfBfAPMJ/Dn+Z5fjDLsn8L4M+zLHsIwBSAX1zE7XbSSaM8sdMbhT/+0A+x8uT04EFzkO7a6WQ5SKEUvn3rlcYofP0tfOSxZ5yr+UxJd+100kld1FzgxrNdpvY96U5OtyCGQaxeoKujiLqmK8R+ImIooYO8nzZXMe95IGJY1PtJqN2wsYOM6rH72c43ZDyhDlBEXZ9nUAwhxx7Con/kdq722O4qbnAbW52dbgi3semv6M+m8MxlW/G9d16B80+8jY8++DTOm5x27Tuf/dr8wGYLIL3RLRqQXcVxXGEcJ5gw2iCgjNxf0rWheowYcjxgG4qYqkScx45ndLAnxhjW5+a4wnhMWqGIu37+GgDAi1/cHaCDQRwgo4equd3GFfZoHXYVO9cuoYAcV+j6at/Pj/FoFbuNvU5X67ErmXQWKaS4QquzcYW9tESa1l3JHFf44Lt3Ye/Wi7D5jbfwke8/g9GkgKrGbH7wW0vm2vn09k/5547QLsUMyvF/w41tihMU24eIYdQNbW3zDBurGMf+xWvb9g/84nsAaDzw+UeDfYjxhgPiFtvGSLGItT3OMgaR4wWDdhrbFpcYxxXW5qZ26XO246anDsz42jlrEMNOOlkOsvXIm3h58wa8//s5zn/7NIouh/w5K4ce2T9vLvRzQa44cATTIynu+MFu9MqyK79wDsvuB5+vGTudLJwsa8NwLuSTWaODVjdTIgn3bUPtGtsJJayRRgi1a0IJpbVdTF8zSaWmi2IHtYD0ifMMQS7RfYotdOvF8YKECJJuPuIJzZ/VHvsYQ62B070UI/0S5x87jY9+62lorVAgQVn4uMOlJMMSTpoMn9kwkIF6PGAbA1nar0Q4ide0czeNrZ2rMDfHC8YxiJZscuLAOHoVWpkGY/y6EgOZySVxPKGCZyBz/zhe0LCXzXEPPtZNihcM4wl1pNOBTo4xDOMJJQZyQDSxaGJaAj1AJwoJClz6xpvY9sabBllMAJXqhYovnFcJY/nqIl0x0nUkoYMcs+gP+bssIXz164D72O/gTOISVRQ8ychaSjof52ckJoWoCHlLqhjDNw+8VfVPDBnEL+z2F8QY2u8jxSAyUcWO8XF79f00xRXOlpxSKohxkEw+ib8MHHfo1uB2mrsp3tCMVSJ6PKwsa8NwIPlEIo44xTwagfEeJCIJ920yDCVjcgCpxO1lEMOY3cWAMQYlhvEA1nEt5UxZCu5eMt4CEotdu4FcEs0dpJ5pchtXbW1sY9Mefpy6Ty5iaxhGbmPT37CPX9i+CR/99tM4b2Kq2qJy/ezxUhSJTSz1ASJiRwvbeOB6NM8wDGR+qLLhFxiYEPYYjed+TQxk24/HstFm3tm9DPeeRHP36BNVkBnIcWoaJqm4FDYatT0EpBEtuJIxmIHsdG49MgKJcBIzkJXyD052G7vjapEyVXj4PbswNdLDXY89jV41tzUGmam8lKQH/32UHs7SVTAbA3Kw4RjOUNLfUlo3SEPV4jYOjDLa7WyMxZDwApGQYo1F3guzm2MDzM6tormZOWy/q0xs4b2ExlloOMfklHhMbCDaPcZEFYl8Ajb4aA92bXdP09E8dnxkIM5WluBl10knZ488uetSPHn19gaTqZNzWbZ9fCe2fXznYm/jrJQiUXjoPRn2XbIB0DJ7t5NzV276qetw809dv9jbOGdlmSOGza5k4+JsceOyjueYCzrIbRI6WBsboYRDoIPifppIKv3h3M+D0tFosZ3Qv1gXpKuh/UvkkmCeOhIoo4P2GG6e+XQb2+Mns0vx5JWXYdWJU7jzgWdw3uQUyiotTWnHlModLyVpcxXHhBMpR2Dr3AKKmAwwrQelpqmlmWnYVyuKGMzj1437pRFqyaQTv0dgZOWoG5eIbmEZrfRInx/DhJQ43yETTSz6p2geydU8TGoav553G7vxDq2op6ZJE+9K5tQ09lj3gEfeswv7t6zH5tffwp3ffxYjKKEsrGTH9AC1xF3JIuon6GaDLM4UUWxyQ8upcmhcjLxBzRuKGLu7bSqYsVUrTJ/IlezTsniELkYPnS7KbdiUZkZCG9l9HKa2id3LdV2MHNr1gpyHQC2djRtLSGB9bo8cNrqVAUDY40xkWRuGYYxhS4UQ1jfkGHSqQUag1DZTt7HrExmDg5jF8TxtruLYGBwUO2jfC2u0NRiDzDyOdJoNv9q5DBFDOEzFkshtDBjX8UzdxtDkQrZGHrmSn3jHdjx11XZjFN7/LFaenEJRJM5tXFQGotb+eKkKM5CBwW7h2TKQgXpcoWSYtjGQXZ94nhYjUJqHdTVDTIXGoOSSDmMYQ+ON+4cGnxF2H8fGIjOQ2diLDbq4sknMQFbBeNbpQGdcyZXLGTID2T6AOIYwEZJZJ2mJIlGBUXjX959BT5dIRrRjILu4wiXqSmbD0LtQvQxrBAY58IYYI3glw/kajMYYrfXfhtD93OR2NnutV/EYZCxKibIT0lW9o5g/70q2uRFjI9HOLeU2LP3Ers0ZhIHh5s8/NvgGuZcloxJKcCWTsWglcC+ruo4NxCa3stk35PiEIWVZG4addHIm5MTYKHbv2oLzK6Rw5cmpxd5SJ50sCXlt4xrsu2Q9Ln7tGO747rPoKcnk6aSTThZTlrdhKJFPmhDD2aKD3N7Ufyi3cYOr2G+GkDcJRWxBDEs92A09BNFE9/sCiqjDflIuQmYZu3MJ5xmGXBLnJxTdxsQ6doG4fU3o4OzcxmZbCkWRYMVkgQ8/8CzGTk3jvIlplIQSesSQxixBV/JMSt7NFwPZtg1yG0t7HFTyTmI3x0ifYndvAwO51k9sD9Ea6yaWS+L5vdYrmqgaEshu4R6PjVzFccm7GqKowxrIRlcveRe7kj3BBF5nUURCCS1iGJe8u+T147jj0Wex+Y1jGFWFI6KY9uq4Z5FDQC3BJ1QqXC8M3lgEixG2Qe7lQYhi3J/ROmkPwfcz2m8TmmjJLCGK6OGtmaOI5MYOSBx+jHUTlxahF8geMXoImEdETGJhAggjgg6Zc6fi95Vo/7kO615mQkmgi9YJ90M60Jhqq4MIKSKxBbOXJQjUd9LJ4sjzl12M06PmSbXhzUmcf/L0Iu+ok07OfikShefesdk9tLYdeRO9okMKO+nkbJUl+HtsBlJEwWQAJMKJbov9A5pjB9t0rfGERdhu+wwTQxjPJyGGHEfYMvcgoonm6iXx3NJYKRdhqcN4Q3teLl6wjiI2kktq+QkJHRTiCcPKJlY383jCskjw1JXb8NQ127F141F84OHc9yNySXic+FMol8fvr8bqJEOSTma8zoA4QNe/AXmUjoMxcTwhHTelpqn34/0iGHPipWNQ8MifbW+qhayE1DQpzenWq3RuLMUQWvQvJpz42EH4eWKUMVjbo4QOwVQUY0iEkzRGBxUcYqhHgO/c9g4c2LoeWAFc+/wBFzeoFPxxqpFUTyNFH4DqLT20vYf6nsUYQeomhYMxWjcMYSVEqeV+cX5C3dBm9ZwqRtrXsChiE3FFuTGgMRpHXzwKDXtt1fMdllCN6KHZtw7QQ3deEbLIiKCUA7EJZZTiDttS2HA7I4e1GEMpXpDRRtLFhBQWRWm0ZiPL2zAsK1ey5B5m/XwagbGuKedgbKjR/EMnoeY9NZFK7BqDdINIJe68IsMvNgZjw7Hw+2GCSGwsNpJLBMNQchs7e78M3cb2IxnWbRwahub4yexS/PCaS7Fq8hRufOzlIDehcR/DHVu3sWsvFeJ8YktBmHASu4AHjWFyShsDuankXdOc9v+iW7llnoDxKzKZIegkw4/G0nx1YouZ8/jjrzqjMBHmDkgjWgWuYd9eudCqth65iGWDzvdjwklsOIY6oxxU8i6Bdi47Jpw4VzKVvEsSjSJR+M5tu3Bgq4kpvPqlg4aYQiQT5VzJcE9g6z5WPSX7UM9yGXEGhBfpNAa6j2dgOMbzNa3d5n7mfpLbeW7Gol8vyF0Yu1JhEm4feOwAACBFEhhidjeKS8dFRiJQGU60Xzu3M9TcdAqc09DuMXD70niri93LpZZyMg5wNQdGZ9VGOjHfoaBLFO3HPSKlb8zwsrwNw046maM8deU2ZxTe+c1ncV5HNOmkk4FSJAqPvNcghZuOHMOHv7PbJbDupJNOzm5Z1oah5lJuTikgeU2oX23MAHTQ6QQiCR83EUlq/RoQw7mig/EexNQ0Ouxr29oqlgzKRUgoYoAeVu+DyCViKTvBbVzaLEUWvSsxI7exWVrhh7u24ofXbMeqyVO44xvPYmxyGoV1EQfu47rb2KVf1AkEpP+sl5kQTuYrNU3TfI3tNJ9U8k4iviTCPO68gnQ1dK6E4Hld/di+96oxa2/ehAQKE48fgQrmNBKXvGO3sp3P5x30/Zy7jHSDSt4Nk5omRB6tLiScOPSQSt7ZlDQuDUmi8cj7PFL44e/sxij6QUUT++50vborWS1RV7JNN8Q7d8gZQWsl6ufWhO4NckUDEcpH/2mbZ9B6csqZ4VFEHY0wVBIPfzW5mrfcshUawIHH9leu7Qi1UzRnhB4CcVk7OzefsHJtzj3LKKFtV35uyb1sPyeptF6cwqaecsbvR2s6/2AfGIqQEpfPk9zUM5FFMwyzLEsA/BGA6wGcBvCZPM/3VG03APhX1P09AD4J4HsAngfwTKW/O8/zf71gm+7knJJ1x09i9fjb+LFv7saqk1PijXyhpbtulo6s2rEOADDx+JFF3snCSqKBC46dwPRIapDCogyzJy+SdNfO0pELr1gPwBiGnSy8LCZi+EkAY3me35Zl2XsA/B6AnwaAPM+fBPAhAMiy7OcAHMrz/L4sy+4E8Jd5nv/XQ63gEMOG6iNWhkYC54YOuqUH1S7mPc4k5cwMdDWUsI1Uwihha9xhQ5JqTj9j29pqHDeQS4aqcRzFE5p9+3jBQfGERWF+salS4ZL9x3DxgaegSvNLNI4nNB+TCuIKC1sj2cYY6nmPMTzz1w1JY2xgQ0zgTGVQDGFMbJFS0zCxpKkWclynmNPDiLGDwdi4H+8XtTE8t+2TCnswSJ8/riepZmSSzr/6DnLcYFwLOSacuHkYZYzmTiDXQmbCiRJjDKvjXmnS5KQa1z+3H2WuMJIUQFKhfxYJsShhqgN00G7EooQqVVBzKd9QlwW5dnzS8rqroKS66dKZJQ2Iotg3npuOm+ITpbjEkhA8Ow+nK4rnnw2KGNY4VjRPOIZjEAFTc7zUnkgipbqJ0UPT7lPccHxiTGIBI3mum5yuxp4go3FS8uymuMI2wgr/gdsSYTcRUmpoJM0zG1lMw/B2APcBQJ7nj2ZZdkvcIcuyVQD+BYAPVKqbAdyUZdkDAF4D8M/yPD/cuMI0pRNpM7rs/1lmawRGbaIRyOsNchXzGkPkGmxkN7e4hWVjUofuYruv2jxeJ+UibHQRtxBNmnIStpWygzP8vMFnr/2yrzDIbVxWhJSnr9qG1y5ajfc/kEM5G9cbjUwqMadJhiH8sb0pnQHD8MxfN5BdwMP0BSLjbkDOQmmugW7lltyHUsk7uyeeJxF1ZIgFeQwlY9HruHqJfY/JK2JJvIhwwq5qO9YVAaGxdYMONYZxTDjpuQez1Wn0IqJJCrnkHRNOVOxKTo0ruUgUHrl9F1adOI2bf7gXidJmrCWapN5tbE/AuIqtK5ncxnyC8+tKXpBrxz9UVavrNiwJZ6TJcExiiw6oeTCC/kE/+66JIEHjBGORxzp36aC5I2ORdUwA4XWbXM2qmsdWPrFGtjUwm/IhBmSQamY2OmvGmbC32L0cu42hhDJ6Qr9hXMnxfvj7YO8DwxBSxDyHmL0sZh6NNQCO0/+LLMtiQ/U/A/DFPM/fqP7/HIB/nuf5BwHcA+APz/w2OzlX5Omrt+Lp67Zh8vwxTI2eteG33XXTyVklRaLwnfe9Awe2rcexdatQzi/KN5/SXTuddDKELObTbxzAavp/kud5P+rzaQA/S/+/H8DJ6vhuAP9r6wpcK3lYxLApvUw8z7BpZqR5JLfxsK7ieA+DSCW1fg0ooSWLDHQvRy7pGCWsuYMFF7GIIg5BLqGPz+jknITsNnbbouol7Da2uh9etQ3PXLsNqyZP4ce+vhujE0VQ0cT2i9PRlNoQTOw6BSLyCdR8xyae+eumQZoIJ3NJTdPmkhbzGDIKyPPU0JOQcCKmnKnpQnewGRsieHY+e6wQunlte6rqfSRkMchjSHrAoARMVAEM+ucRR54nHKuAgHDi2olwUkceiWhiUcREB4QTm7PQ5SlMge/evhMHt12ITUeO4YMPPYdRVSKxhBQimthjJpwERJMK4nGu5N68u5IX5NrhiiMqQgTr7t821M+PZSRR6gvU3dDDIH0ARBJLUJO4tmcZRZTQ0RjnZJ3kajbXmr/emiqfcFUVm+LGr2CQ7RoZRJiHK614V3KIKMpu4xD9Dty9hBwO60puI6QshiwmYvgwgI8BQBXv8TQ3Zlm2FsCKPM85+vRPAPxH1fEdAB5fgH12sszl6au34pnrvFG46sRZnZKmu26WiBQnp1GcnB7ccYlKmSg8+v6dOLBtPS569Tg++OBzZ3tFk+7aWSIyfWIa0yeW77VztstiIoZ3A7gry7JHYGzkX86y7LMA9uR5/mUAuwC8HI35DQB/mmXZrwE4AeAzrStIiKGUzFpKGB2MaYgdbOw/ADGU0MGm9YZNQ+OWkuMFfaqYoqbTAxFFr2utWDIgSXVINIFvBzAMuUSscWwRvKKuK0knxRPqUuHNC1bh6WsvxcrJU/jg136Esck+CiRDxRPaNimesE/xL/3ab/s5yZm/btAQGzhHwskwqWliwomUmkasYiKko2H0L045I80dJqEGjUW0x3BMPLfFPI7d+zJsFZSA7NJAOOnBz2/b3dyEYLgk1VU/RhGbCCcxsSUBKMawihcEE008cqiEGEOlgFd2bsDBSwkpRAGkcHWRbT8A9WTWsDGG1TGhgw4lTBTQm1fsYkGuHSZgS/G5AVrXgigympg6HfetxycykhfHJTaRWVx8Gu+LOrgYPQFFZOKKcv09+ldG/biqCiN9YRJqjZe/+jw0PILoklBTWhofB6sC9DBeu418wvMEJJS2MUqOOxRJI7SHYWIMB1VIaUxh4z6L+tyzEbWYcOWZlre/9kfVX1JwDwN1QwuYuxEY60od6u3YFqOz0RgcxCy26w0imkAwBttIJWwMRsZibPi1E014DPwxYCqXuDnt54AgJ+Ewpexit7FtY7dxWYTtL126EeuPTGDViSlwSTv/kXn3MLuNAeM6ltzGBeyNTsF++37yyF/Nq4V4JmXtqivcF1cyDOcrZ2FaWQlNxqI9tv2keVKV1F3ASiGtbpPmuLnd/lF6yjuJvdtXBSXsbBu7hbmv2U+dkJJCufx2TDjpcbvdBxuBdk4yBn31EjtWIpeEOq5uYnS+yonTKWIlJ15niSa9tAyqm2gAL1y1Cbv2vopeQW0jZVDdBEBV8q7SjVY6IpyolI5Hq79LL4GqDMM1f/K1JXPt/M6ln649VPkJwfnl4o4S3sqEDR3o62OZvCC1u3lcabiGdkjt1kAinRZ0wtim+XQ0RpxHxxQdO9bPGecslOYMxgTs5VDYvWzapTHVMRmkWtIJ7cE8sdGp5X46MnzZna11fR7WHZ14fsbXzmK6kjvpZNHkwNYL3A16+96jZ7v7uJMlKCNbzsfIlvMXexvzKmWicGiryc+oAFyZv3q2u487WYKyautqrNq6ZrG3cc7KWUu9nBcp6rWSm8gjQQ1l228QYSXWDYsO8j7aXMU8ZpiUM7V5aOwgZDFGFEsduourMVLFktZchKUOqpK4tkoVpKCJvP1MLpFqHBsXsaTzbmMAQZ5CrYGnr96GZ2/Yil3PHMZ1P9jf6DbWlIvQzmePnSuZ3DcFEjq2iKE/Xooi5SyMCSfDpqaR3Ma19RCuVyOVCJVP4r3ZNjHlTKBDuEfIKWNi8onR0X4JKeT282+7GArAxF+/GLmNPXLIrt045YxBGUOd0rJb2B27/iHhhKubuPUioolxG1drC4QT9IDvfuAKHLz0QrzvwRzb9r8JlWiHFDLhhEknQOVKdpVP4N7dcU95xPDMuZIXRKRaySGRxOtjkzoFahUr+HsuuaG164fW3IcSmYX7JAGKKLRLe/DOm8DV7Pdj1/Tz6UjHR7oas+m2bQCAl764u5Gkwm5zNyNXUnGPHWEM9RvkXmZSSqyT3MZNLuDYtT0bQopYfUVFbuVqD3O5cpa3YdhJJ5E8886tePa6rVg5eQpX5OdWRYpOOpmtlInCYx/YgUOXXoiLDh/H5kPHFntLnXTSyRmS5W0Y9i1zgdAxWFWEIs4XOsjv8dqDkMNhYgh57QExhjNJPTOQVOLmDhHDGko4IEm1+7jceK8bRC7x6CEjgXa7FYJXJDVdWSGCz75zC565bhtWTpzGB/7hOYxNTJu4wCHjCUvUEcM+VHQMd2xPuS8kcj7bRUpNM6f50IA8Rkhe0C4gj3ysBHRwJoSTOPCdyR4OWVN1ZDEmnDCSGLe7PlpIcI2QcFJLe0PIhBRjyISTGooYIYuckgYwhBOubmLfU1XhRUQ40Snw/Q/twOHtF+CiV4/jA9/2RJMkZXSwWk+FsYW2zaOD9l1BpYQSWoik52MM5xRBv0gS/30AoKTvNxNJfBJmGVG0/azwrcQlQOa9IfptAAAgAElEQVR2mkdKjcMom+kzGEX089dRP94Do4duRHjrD9LaDEIPzT5VI0mF53ZoXAM5xZyfR9HtqiUE1A5h1ZQy/nsKumEIKYMSYNs9zIaQIlVGqUdmDi/L2jDUfaK71/IGkiXC7SI5ZQZGoNW1EVrYoKO1ZlOdpG7w6vq+BYax7hft7YIRGBJJSqGfNyKlXIRMLolL2aFUjmjClUuGLWUnuY1Len/22i3Yff1WrJw4jdvvzTF2oo9+6fMTztRtzCQTf0x6dwMDiqX3bBOliXAyTM5CSZqMRdfekPuwtl7wUPHzBO7e2pi6S5vdwoFLOjYWwYZWfW7b7matKpzEhirnQ+xp1tt913Vs8IXkEklnjntENEndAx304LRGoA5K3dmNsVH4wW8/h15ZOoMvUcRaZsKJnccZg9oZhH4ROrGeckQTdiWrJehK5oeqM8RiowHNhmN8xQxyQ0vGYryWmVdjNsaiM6aU/+7Y+xyPjV3NUqUVU1VE0Tx2b9UaIL8pjGGoeDwbpM6AEvIYBsSW6rxoojb2snEV1w1MibUs5TsMDMAGQ862D+tKjvvFRmfNnkD7vXeQLGvDsJNOrJSJwsqJ03j/3z+H87r8WJ10MpRoAGWaYOPh43j/Azl6cSx2J510suxkeRuGfcpjKKWcEdPQ6LCN24dBB23bbIgk8R5ilHBQypnaGF6vrhuaVEIoYb0fu5oh5yL0PyHNWx/u59NMyCVSjeNYp3XkVgYADVz5+CFc8fQR9E5pjwqS25jzEpqttruNmWQSHCuPHqI6zaVIPmlKTSPphk1N0+o2biCc1OZpRAfr8/C51FzE5O6VEDxGCR2qJyCL3M6kEUZmEjcGbozVB/kJ7X5Ix9VN7FjZbawjXUg4iV3J5ptcHVeDODVNkphveooS733geWilMILCIYVMOHFuYyKccHUTowtJJ6YtIpxIruQlGIZh0UFAQP9IEbiAaUwN1WtAE3394IY5o8UZHfPzDUYR/Tr1fbQRVhgR5D01oYd2nvo1pUKksLaHdnKKhP4xcthWNUVRqhwmtljCij3/vi4HElIG5Tk0+x8ORSwihJDrKpv/h9+LmcryNgyL2F+JxTUCxTENRmC8x2GYxZEujA2s92uNHSx16C52/RDpyH3ckItQjh2081S6IWIIpVJ2ktvYHu++dguKXoKrHjuIslRI3jZG4TBu41KHuQiButvYttnrr6/CYzMf0Pf3snNORANTMBBdfzIqm+IKJXcvu4ONrsFFHBiTCMaoYL9+XalMnm1PVUO7Vnj7vv3owbuWU5rTrUd7sA8OzkXILmQ7NyexdvO0GJBc6s4Zr4qTWfu4wkQZdP2xD12OS15+E9v3v4keSkAbF3HiYgbJ4LMPI4orjI1AJKCchbIx6NzGASuZ00UvDbH5JQHUGMb8sA6MvAaDEWg2Gq3ByDGLktt5LsaiZPBpbqe4RHY1m7H1pNd81rGRaPcDaLx+30vVuangu8xuah/fSMad8vO5EnZu/3UjLy6tF38mYYxh3cC0bT2VeAZ29dCqxRWSQSi2x+sFa4RWnoJqd1Or+g+MmcjSC+DopJMh5EfXXYIf3bQFBy6/ENOjS+/h0snSFz0xDT2xtMIWykThsR+7HAcvvxCv7No4h/D1TjqZvRQT0yiW2LWznOTcQAzbyCP2WCCDtKKD0lhmE0tjYmawbXfrRfPE6w1gFpu167phGMaDSCVeh3AejfZchGWIFJp3/1ORK5cMIpfUchZq4y42x1737LVb8NyNW7By4jTed+/zSE+VM2YblzB5Cen0a25jwPxqdzpF6KL9tQdPRFlK0pSzMNYNylnYNKf9v+j6bZknZvzG80i6wEUs5TYMUMToXMDIm99DSE4J2x0DuVet0dcVahgiPEwkCXIR0jtXNwEMuYQRxbrOKGPCCVc3MXNr5wazyIROgcd/7HIcvuwCbDw8jtu++QJ6aYnEooPKj2HCiUUKHWhDPnJmIEsnGBBO4pJ4aeqPl5CMVO8l6qQTzrkHEKJI/eLyd8O4n4HKfSi4nYdFEbXr7+9pEsu5ibDiCIBu3zoYY/fUhB7aPShtvhcaqCppEUu4gcUca+fLvSwSRCLWsm3zrOXqSJczynPYtB6gPRub0MRGIko1XzGHn3UdYtjJspIfXXcJnrvJG4UrJ7uKJp0sjox9agdGP3X5Ym9jKCkThcc/bIzCDYfG8d5vPN9VNOlk0WTDp3Zi46d2LvY2zllZ5ohhFQk2DPo3THUSbidda5oZae64vdLNqjqJG09tQ6Se4fyDMvkkjCN081hUrwEl5PQzAFD2PcLnQzAU1TP2ujZyiVTjmOMJy1Lh8LZ12H3jVpw3cRq3ffV5rKjyFGqKIZxNPKH9lIM8hRRDaE/PkE/ssf0Y6jE8S0FmUgtZSikzVDyhUMVEBYhH+9yM9NWIJBHhRIoDrKeh8chiqKujiAx+1UksPjbK96nHCwYpbAhxYMJJDLIpGh+QSxwqQjr4flzdBDAIjT/WeOHmi/FqhRS+9/7nMaoLg+ipME+hrW7ChJO4yonqaSQWKaRYQ4v+uVhDzlPIqWlSijVMlh52kUYxoECI5A1C/fyY6p368FUUg6llNJ8nQ8weRRwuBtFo47lLLSCLSreihzbFjZ0rVSqo9lNSbx4bk1Oa4g7j3I8S+geNGaWzsW213IbkTRgmzyEQIYsujZQanMLG7d3/XZdkuposyxIAfwTgegCnAXwmz/M91P4HAN4HYKJS/TQMSv95AOcBOATgl/M8P9m4yPSgPIYzMwJrLuK4/1yJJPFempJQzyX/YJOuhVTCMbX18nahMeiMO8u40OT6JVeydwvbU1XeLSwQTtgI9KxjRS5khYv2jeOKJ1/FpbvfwNhk3xmAhU7m5Dbm3ITsNrZt3lhEzXAs/cc3L7Ig1808yGzcxvb/gXHYRkQJdOG6MeFEcjW7BxkRTmoklWBu318qo6d0fYz/f2go2rHsha25rDUfe2NDUXtN595DwkmcszBVvtRdmmjsevpV6FGFa588gF5ZepKJgjtO0zJIYg1Uhp+dm41Bb6EbXeoZyKIxGBiJlpWceiNxHmShrh2JfKLoWdNkMAKy0cjuZyazxNiCgux2nqmxWCI0YoZxNYdb4evJz2n2rHwIAxAYiV4Xn5OKyClmRXYvxwafoj3xdWsNPp8cW7lRgYHozl8wJgMj0BuIEiGlXz3MjJu/GsOP0Fin4J7p7D5uTXoduPuVmyeJvh8zkcX8OfZJAGN5nt8G4DcA/F7UfhOAj+Z5/qHqdRzAbwH4fJ7n7wfwBIBfXdAdd3JWylvrVwIwF/tV3z+ElZPLOmi5u246mRcpE4XxC88DAIxMl7jue/uXu/u4u3Y66WQIWUzD8HYA9wFAnuePArjFNlS/7N4B4I+zLHs4y7J/Go8B8PcA7mxdod/3r6Iwr+lp8yoK32b/Ty89PQ3d75tXUUBH7ejTa7pvXg3terpvXtU8Oh7ft3vrh69+Ae1eZdi3KICihO6HL9BL0oc6DfS1KVsXvwoNXQC6AMopjXJKQ09p6D6g+8ZFXPYNscS+yj5QTiuU08qknymMy9i9CvMqpv1xWSQoiwRFkdR0ZaHQ7yfo96v2CjUsCvMqywQ/uv4SPPDTV+PlK9ajKJV5aftKqpdCgcS8tEIB8+pXrwIJHfs2e1yo6gWDFBYK1GaQwr4yiGK/ehWA62/bl8x1g8qFXLljE6Uc4cS6jqVj+6/NbRzMh3q/YB568VhpnlhnETq7gp8zdCOjOvbrePTOuoldu1sPNA+QaoVUq2iMXdfnNUy0RwF9X/Oype5SHbbF/WzOwqTCOoxO13Xw/VKlqxJ4BiVMEw2kCj+44zI8/FO7cGzDeYZYkpgqJvY4SbTJU1iRTqxbOXhV5JOkp90Jqio1jSl5B6g0/MBUoszLooTV/41/2+h9WzKf7uQFuXbSiviTQru/6Uj16tFxCmBEa4xojbR69fgFuFdqX9q/evEranf9WuZJaHwKVXv1QO3VS9E8PSj0XH9fASVx69Xns9eL0iq4Fuy1Y9PMoJo/hUKqole1rm2Xrm9/LfhrW7qWUyTVi/aqEqQqaZinfu+xe4/bgmM7nu+d9l9E6Bt0z2MJ7oHCfXk2spiG4RoAx+n/RZZl1rW9CsAfAviPAfw4gF/Lsuy6aMwEgLULtNdOzkLJb9iM/OZLcN7Eaaw/PLnY21ko6a6bTuYkZWKMwlcvuwDrXj+B1cdOLfaWFkq6a6eTToaQxSSfjANYTf9P8jy3pUpOAvjXNpYjy7L7YeJC7Ji3q/djrSsw+cRKE3lETFcj6Nr68ZwtaWjCfQwZQ0j6gYmpW3UgHR3bmIsBqWcGVSypJa6mGENON2NTGzDhxBFRGsglLrawVHj+xsooHD+N9/zdC45oEscTAiaR9VziCTlukOMJbZuN1+lTXxtP0gegZ/ejrUnO/HUzhEgpZQbFE1phModU91giosSEk3AeiOvNNJk1xxjy3Nzu9s1rOz2CMfqJNxziwMmq3VjW0bFLQwNKP+PizbRQF9mnq3FpaVC6dDUK2seJJcBTd27HkcvXYcOhcbz7a3swogsTCJ9opBW5JEhgHZBPqvingHwC1w4AqsekE+Xeg3hCwKGFZsOJT2ZtEUKl5jvB9YJcO1z5JImi7zhGUGn+rtb7NMUlmj4CKUTL80jkFRezFo23a/l9cmysdvPV1lYU32bXJ5IK74Dn4wTZZl8aJYDTT75hUtdAVevauX38XZwU27T7uMPaGAgVVBRq8YslnStU4o59vODwhBT7zEIQl1iNoTjAgvcoxCLGFU/MPZbWjv7WOvrezVQW0zB8GMAnAHwhy7L3AHia2nYB+Kssy26C+RxvB/Dn1ZiPAfgzAD8B4MHWFcgwFKuPWBlkBLblEgRkIsmwRiDPO5vqJLV9Dc8w1gVoDKL2OsN4mIolcS5CU9auMsTIGHTGHzGNB5FL7CnkN1yC52/ejPPGT+PWr+zBisk++jZPIbwR6JjIIIMP7WxjLmnnbqzOWIRoLDIhJfoYA908yZm/bhCSR6Q8hW0ilb+TXMwSo5mPZUYws5LDvYb9wrXjeaSchVJOwjhnoW1jwknN4LN990wE88Vzq0gXVy8xLmLfF4iMyWBubwQ6nX3AVDkLy0ThyTsvw5HL12H9oXHc9vUX0NOlL2+XeAPS5SlUvtpJQhVN3HM1gS97Z58mifLtlnDS80agksgnSRLozZh0TsxKQRbk2hmpboJ8h3dVSsioCpjBgsEoGXm+j1DKLjIW43KKXA3D3kt5WjYWPTmDx9O1FZFK+GQlRrNEUimd0cf7MldR+cI4tDLudK6g4vrROlo6V6A2po+wWordo+3YlO/Q3v971ZfflL8LjeAmQgrfE/w6fl/+R4A3INtyEvr7Ks2j6iXylFJzIp8spmF4N4C7six7BOYj++Usyz4LYE+e51/OsuwvADwKYBrAv8/z/Nksy34bwJ9nWfYrAN4A8IuLtflOFkdKBby1aRXOGz+Nd3/lBYxN9gcPWl7SXTedzEqmV6SYuHAM6w9O4Nav7TGl7s4t6a6dTjoZQpSOXaPLSE78P79CP8nidDVNqN4s0MF4DSHnIBC7gOvooNyvDQmcRf5BnzdArmcspJ6ZScUSTjUDmMoksY7HcOUS/9FWv8bKxEPjVON4KkkwvaKH0RN9P491D2ufhqbvkMGwYslM3cacm9D+avbzyW5jqf2/3fe5eYU/zqRsXne1BmRUj13AktuYEcO0+pXdlLPQtkvu5RRJDa1LVRL+Co90Kc3j1q7mMusY6SnvlLZjevQLv0fIY6pCnSWTAFUgvRsDN6YHhfLDm83xNw67QHzAIwKWBGDH8rGZWwcuZMCSDnSoQ+hCtm2+FrLGSGL0U6sS9KYKrNClcxvbyiZp6tPU9HpVW1o6tzFXOUlGLEpIOQupyokaUe7YvCfOhayqEpWql/h0NL0UarSaIE29rjpe9T8vnWvn3k2/oIHQU1DCb19CErlv/EQuVTje9qnVYR4wt7RGKXg0SoT3sbid1/b3TS2uE6Wwrebz6GE8xs4z+uEtAIBT9x+EVvW5w/W847TUPLduHWP7Fa6fdnNo6qeFeaxbWezn9qD93Jrn8bqimtXaFTyP1RW6pP3W5yt0GawZt7989KkZXzuLST7ppJOh5YUbLsarl5m477TQGDu5rFPSdLIcZP0Y9PqxRd1CmSg8+8GtmLjA7GPs7T56xfIFAzpZHpKsH0OyyNfOuSzLu/JJv/DHrZVN6jo9ZL8AHaQ2ER20MkTtYtdVaJ9NYmo/FtSP21XQbuIJw597Zd8nnG5CCW17QbGDtTGczJpQQiaXuLFa4YWbLsbzt2zGymOncMHLk1Clj+EYFE/o+xmZr3hCr6N+yqCG3F6qudWsPNtl2HhCoB6jKCatJoQy0HOMYdUW6sI5Y8KJizdqSWYdp5nx80SIKCAms/YxTSHlxqWwIaQQCAkp85nM2uwbUCnw7J3b8NqOdShHE9x8/14XR2bS0lR9E0IEbWJrJp/YMQmgUq937450Un1OqarHCwbxhNTGyaxVOEbNP/lkQYSR3HqN3xBRU9F3wqBx4bVjSCXhPaRUIXnFrIEoVjH8TkjVV4KqIvbjh38EKYSxh/G+eW7/vSSEzd7faW6Oz4u/tzae0CH7gIs7BOCSY8dxh30ipQAIqpwwMSWutVxCOyTffTrKo36N9ZWFWEWJkKJIF5NPoCjW3n4+GgEyCYTxgrxGExGFz3m2srwNQ4mV3OQeHuRqNh0HG4Gkj8c0kUXse6urONDDzxcbhjp0F5s2HkPvLaQSaATuYnsqtSomXJ2Ey9bxGGIZ2/eYaMLkktISSTTwwk0XY88tmzE2fho3fGWvyatIbuO4lF3sNrYf13y7jZsIJ0XUrqGxFKMgJbexxESWpMlYdO0t8wSEE2EenovdzHXDL1yvTiCpB7QH7YEBGfaLCSdJPBY2YNyPlw06ZhsLBiYZBF6na7oU2ruSlTfsnr7rUrx2+TqsPziBG7691+UoBOByFtpjwBiL9sGraB7HQGbyCRuOlmDCJ+iMxcS/x+STNA2NRWskkisZwg+Ms11SeijXCCLwRmBoqPh2qUzebIzF2OCTWM5MWJErpIRGIgAxy0JoYPK1Y12b/vzYSGQDjdd180afkSUwJrTZEsoZMu5cFJlGdFA31AWmMp1sUxk9dyFUD8KmCineRSwbb3FYDDOVtb1n6dBwBAzYIBFR7MlyGb3ZyPI2DDtZ0vLCTRdjz7sM+/iWL7+I0XOPaNJJJzOWMlHYfdc2vHb5Olx4cALvuu9FF5fYSSeddDJIlrdh2Kd0NVL+QXc8AB1sIpDEczURSQYhgW4PEbIYI4IxOggMJJXYtkGpZ3Rcu7gh9QwTSMxyVB+5IRehRwVNW1Ekvp36OxcyFPZfeSH2vMsghTd/6UWMTBZD1TiO3cb2IzoTbmOzhuw2llDEpSqxCzgmnMw0ZyG3zXfOQv9/djXzmv59NjkL3R6CdRC0s4vYnSfp2A3HY2ebs5BdyQoaz7//Ery2wxiFt9z3oql9bBE8QgkH5SwEUKWo8eih0xPhpDVnYeA2tkhhHSUMchZa3fxWPVkwGbGEAkKjrITf2dA9a9st4ujuK/DfjZK+i4NQxDgVSuxqNm0aMdIXu4xj5JFdpIweOqSMvt+MHhqdDtBDTm3jdHQO9XQ1pq1PNZcBtCKBljzG52IhhqZ8h/4DQGueQ4vqQdfHm3Px+wpqMaN6RkYoYoJ6ahoQishtIQLpkUKrS6K/9UxkWRuGepoICm15Crl9rkag1c0mXtCtQf35eJ6MQHMQGXwDYgetLk5Mrdl9LOQiNF56ayQSc1iH/UoNFNrHHW7cM46NW49h5yOvYrQyCtn4M6egAuPPfgzsNrY6NvjYkANkt7HkFpZdxZzs2rPeggTYWHqyGDkL3VjU54njCsMxdWMxiDucx5yFVlfLWYi6JzU9dLIyQGWXdBxXKLuSIx3Yvezdb5yzcPsPX0c5muCGB15BWhjDLqlYyTPJWQhURp/tl8DlLPQb9w+3meQsdG1SzkIfCLk0YwyVN+yc25iMff/DVgduZSD8QeHcosF33v/wbHM5G2MyGqPQGpfIMYmx0Qlaj8e0xSKW8AaNM4AQuc/t4899f6vOh0+6Tyr8TEzHHsLk2NZI7LvPUTD4KCm2zTDQp8+H+ze5lc3cnJ+x2reicyEDUbOhRjGKZqy/B9mJCq39PZEMaI43tG1kqQQGod1PnNtwJrKsDcNOlp6cWjWCFZPTGJkqcN3X9rmbaCedLDUZfehIUPf1TEqZKEyvSLHy7Smc/9Zp3PCNV5yB0kknS04eOgKN7vu7WLK8DUNmJQsIn2MgiwQPwnmGRQdpjrb2QSXqgn6MEg5iFgMYlH+wCSUcRCpxYwZULIlzEQYuYoFo4ljFpcKemzdh3w0bccPfvYy1R07WchKyC9m8KxR0bOYL3cbVRzJjt3GpiGFMv4R5Tq/zv+LjvF39JXp7a8pZCIQooSRSzkKeU2yLUMemdVjnUTTJ3auidkRj6vtQUbvdV418gpC9HPExDFpo5yZdzORU8XGE8BidrusQ9tMJsPuubZhcP4Z3fWkPxk6aq8WhhIkOCCb2ve42LgO3sX13xz2h/B2TT9il7JBAIp/EiGAv9TkL2W281MknziDX7n7IKLnDp+jG4Fm5Xhl/p027b2sjrgRjAleqRw/dOlGlFYMsDo8e2jExSSWhe22INhLSJ5BTJPcyaucqk1J6xFqmT7rSMbLmkcPQrWz6B6gfgmkCQorPTehdu3bATAgizMqOXfeQUEQFZ5cE8zDyOIcfpUsvgKOTZSkv3rwJe2+9GCOnCoye6HIUdrL0ZeqadTh9zbozukaZKOz+yDa8vmMtxiqkvZNOlrqUV6+DvvrMXjudNMvyRgwLy4oYIm6wRk5pJ4iI/QTkUUIHw33Q/xkdjOYbJuWMfY/zDwbxgg0oYRupJIgxpJQztk3KRcg5CWtEE1CsojZG4Uu3Xoyx8SnccM9LGJ3suwolbTWOS1BKmoZ4Qtv/TMQT2jbbHqau0X49LC8Zhnwym5yFti0O0Oe5h8lZaN457hC1MeHcoHliVLOOqITkk3raG0s2mb5mHRSAlc8eC8cQSjCXnIVIFPKPbMUbO9bigoMTuPHevRgtC0BVKGE1WMpZqJSPI/QkFI/WOEQwpcomwUlUn9Nscxa6TdD3RCCfqCVIPkkVPW9cFR+LjClHhEwVave0FBBQpJmjiDGBxI5h9NCuG6OInI4pRg9Bc/A6iYb7srp+OorBQxXTR0CelOcRGiiuucDodx/zcYcI42p9NZg6cmhiDhWNtPv1KJvpr4N4Q78XjzLWwkGURxxT+ivF5JI41Y0nndRRxkJC/9zZUQ1sQhHbEFUmp8xGlrVhqB0rWUNkHVsZZAQ2uZC5b2NbuxHo5+V5eCy3q9qYQaQS2zaIYSyRSnRkBEoM49gYZJYxUBmG9gvMhJRK99LNm7D31k0YG5/CtffsxchkPyhlFzCL3cfidWz8AXW3cfUxzpvb2BqEfWpzc5PbODAwl6AzWco1KCatHpCz0MowOQvtWD6W8xhGcwtGJx83Gm+1tUPjz77HRmfgKkZoyNl33qNhHAuGKB+Tm8/r6jkLLfmkTBSeI6PwlvteQlrGxmD1sKGchU6XlmLOQuc2ZvJJwFqu5resCfqAHCtZylmoJGOR3MdCgmso5d3KS0h6laHNP5qtKGjPZAUZaNRRRfe0mRiLwxJW2MizY5is0mQkGp3MZGZjs9qEE7v/ngrdy/E8qlrfZnvoafNj3BvC3mBjnV/KG+BJLcaWDVo2xMxxEyGlrF2DKnAr2x3E5JIEoUs6/tswUzkw+CNjkRNcFzwfG4kd+aST5SL90QSHr7wAY+NTuP6evRid7FzInXQyjLx9wSje3HY+1h2cxI337kU6l2y2nXTSSScki2IYZlmWAPgjANcDOA3gM3me76H2XwfwC9V/783z/F9kWaYAHADwQqX/Tp7nv9m60DQlRG4jgHD7HNFBP3e0FhC6imN0MBjP/QjhE9BBvwb3C1G9YVLP1FzJYnUStFYssSXsACpRx65mQgJLDajTGtfesxcA0JvsizkJmVziET7/K9WjgoQs2l9c1H82bmM/j//FTRwftwYXZPcl8epj5kMW7NqpRHIVN5FPpJyFTS5kpxuQs1CaO0b6mEjiftWrcIezzVnIiEkjaSTeo47m1x415HniFDUx8hgimNZ1VSKFxpqjp3DTPS9h1bFT6BUCIqgICVTeXcxpaGLyiVLwVU4klLAHX/aOEcGIaBLkLGSiCecsrNqURD5xLuce5jOP4UJdO2n12Zn7nkWwTJsJS7Dtyt/V3C1dDYUiNhFXZupqllA7nj9GD+3cbSlu5LnhPgd/jZL71gN0hKKZ8T16nPotMhrH4tG4vlAlJfY3DEdICT/7IJ2N/bvRHoMch9qPsRKie7pRx3/LttyG7GqeL/LJYiGGnwQwluf5bVmWvQfA7wH4aQDIsmwHgE8DeDfMJ/RglmV3AzgJ4Ad5nn9ikfbcyTzJvus24IJXJrDq2GmMVShhOYcv8Tkm3bVzjkqZAC+/6yJc+sTrGJkqseaNt01Dd+kMK92100knQ8hiGYa3A7gPAPI8fzTLsluobT+AH8/zvACALMtGAJwCcDOALVmWfRPA2wB+Pc/zvG0R3SesZhj0r6HfTNBBpxPQQd/O41mnav1mknLGtg2bmLqtnnFT6hmpYklTkmrzMXGMIfDyLRfh5Vs3YfUVa3H9374EV/cYIepndUwukWIDfbsfY9v79AuWx0jxhEwg8WPCX9d9Fe4NML/kGHmMYxC10vHXY66yINdOW2LrJvIJi1QtpdYmoYQq1MWIoqI+Tcmsbf9gjFvb72E2yaxtf4d66Prcbr3SJ+jlfr6KSRjXZed0FVCgHWlBJcCej9KjZoYAACAASURBVGzFmzvWQI8k2PXwYYdMJCqMLbTvlmiSpqWYzNrHG1bvPQ/ncE1kZZ8SlMyaT9rVQ2byCdVDdm0cWwgYtJDjCR2hhckp80o+WZBrx6KzJd3PFXldrJRa1xJOaw2HIrbFIjalv2H00N5PORxUjnOL1jALVPOglkaJE2VLJBVOMRSQU8zCdP4m5pD72XNQpa6wOjkhNpS/bvtBvKGdT7nKQIOQQ4mQ4j/HOsoaxw7aOeJayeZe5fcdPzviWspeF6KRpfZ/76ak122VUWYji2UYrgFwnP5fZFnWy/O8n+f5NIA3Kgj//wbwRJ7nz2dZdjGA/zPP8y9mWXY7gM8BeFfrKtYwHNI9XGuXDL5BRmA0p2QEBsdtrmIgMAYHMYuNrj3/YJMxOIhUYk9J1FXf4KJk5nC9fe8tG7Hv1k1YMT6FnV87iD6S0AVsPxLSSczisD34mAIDMsxjaMfOn9vYzSeM0S4Tv7+A50kW5NqR3MaSARj0FVzFknvZCrOAuX9C7RJhxR/zuLBdRbrZ5iwMySek0/6YXchOp4ELv/CKu8EqaR5NLmLtDcY4Z2GZAM9/ZAve3LEG6w5MYud3D5ufW+75oQMXMmCME85dWHMbJ1pwGwu5DYOSd5ELGQiIJsE7l70DjIFnN9xrcB+nQkm8+c1juCDXjv3ck0TT/dLfA6zBGLuVgdhY9G2xscg2DhuLkpHo1hVczSVCI9Hp6Lscu4gNAzkyaIwVZzfUODZ2L7tHLxuMAFZ/8ZXgnl4zg7W/L/fIlQq613KVFKBeRi9emA1E/+fSNI+RfmAE+jFx6TyNsAxe3B7kOazmKVD/IRoYnfQZs7EYu5rnSj6Z159jM5BxAKt5H3meu4DALMvGAPxF1efXKvVjAL4EAHmePwTzK25e7xqdnDl5mYzCd97zMlZ0RJPZSnftnENijMKtOLpjLdYemMS1976MtD+vPzTOJemunU46GUIWCzF8GMAnAHyhivV42jZUF92XANyf5/m/pDH/HMBRAP9XlmXXA9iX53nrHVIzYiiRT6zMJzoIBOOaahdzu3lXra5iIHQXu/Yh8w+6sQ0o4SBSSU0Hv54lmlhSSdAOYP8NG/BKZRRec88rGKnyFPJHNQy5xLeD2uN52JVMOjqerdu4hCYyjD1PTfOFbmXb3p9fxHBBrh2WNvJJ4PoVqqVwW50AUp+HETjbx4zxe1HCPDGaGZBUUP8FLKGVBukLf62HKKLfy0BEEcD0hhWABkbfOF2hjB5JcfMEyCOido0X7vRI4TvvfQUjhU9hkwhEE1vtxHhhtTtmFzJg3MdxzkIoQge5JrJz99LmXLWTeh5DgyLW3csWKQxqIhOKqGrklATozesjakGunV7P/A3YK2OlLJU7vSYUsZ5SV9dc0VSPI0ARJfTQ12kOUSgzD6WjAUgno4dAheA5XdVPtbuXrY7ThYUoo13bfC3661egBJAePR32I4TN/j4qFV/f0YQIkUPJrayjsSU0UvdH8j24nnXsSta0tJTCpqC+wWfvxoTnD8i5DUHr8jMmdjUzGjkbWSzD8G4Ad2VZ9gjMZ/HLWZZ9FsAemO/vBwGsyLLsJ6r+vwngdwB8Lsuyj8N46H5pwXfdyaxk3f5JnH/kJHb9w0GMTU7Pd8zduSbdtbNE5Pidm6EAbPrLl2c1XgHYsGccxWiKd973ikEKO6xqLtJdO0tEJu7cDA1gzX94ebG3ck7KohiGeZ6XAP6LSP0cHY81DP34TNbRXB6qlsA6Rg2rMaybBTro3hnpqxFNFGIQaSbVSXjMIFKJ1Q1KPTOIVGI+jrBiCQAUSMJ297EoFL0ESV9j7OgUrvmbl2ETVmsgSFwNYChySb1dUXoY1PoxMjjTeEKzRx3OzXPSWK38MevN+/xGGC7UtdOWrDpuj5E3HjdMPeR4nTDBNYJ5mhG8EFnkHfEeub2WZqZBFyN5TCSxVU64PY7vsmspGm/6a9J5JNDwtxSSUuOil45j40vHXZxUHFsIGDQwpbg2qwsTV1fruConGmJdZBd3CPfuj1UtXQ2npnEJrIlIEtQ/jtLRGERQIJrwmHmMMVywa4cTXEe/gpXSwX3cIYVDooglIXBxXDgnz47RQ8B6cbRrt/MwguV1cLphUtyYQiMRskZInx3b01pMis0XLKNnPTs26heSS/x9O0QOwztvMyElRhnDpNZJ1O5bIPaTk1r7S4LCJGuxkQn85xf+PUKEkmslN5FPakbGDGR5J7imWJy4+kigY1vQPdG1oKv3G7oiCa0jGYHAYFexZAT6NcmYnAXDuI1UIlUsCdzH7uPxDOR9t2zAG+9Yiyu/tA8jJ/vBx1dC1VzAbLA1kUuGLWWnI10RzD2829jqODdh3ejUdA46GG8+p3mmniyQtLmNY2MxzlkYkE/a8hkGOr+uZFTGbGUWzlnI1U7Y3auor20fJmdhEuyX5zP/IyJvOCbSxzkLgdjAhCOavPiRLdCpwlV/vx+VZ9jbWdDODWaNwTSobFL1S8iAVDooe2f7qWhDTDQhBpBrVz02+MhAFCqaeFaybwtcyEBgLCommki5DZeQpBXLu9RwtPPwh7vva/9u8T3btNkxOiKsWD2qdbRrazIS3TzVHzOl+2rNVRyRVPx2vWXjjT+7oL+JDnIvg8e4k/F76JNaaTKoaK9QoYvYPq7t2CTar5W4BB9XSAnbqmeQYmPcP3+4MopdQWY323b/mbFp6doH5DaMK5sAIQuaGcqmWaOJLDiMLL2rrpOzXvbfsgEHbr0IOlUV+tFJJ50MEmsUvrVjDco06a6dTjrpZFFkWSOGum9/rTEkSIfzjA56nf/1xOggt7ehg9xvJiln3LhZpJ6RSCVSxRJX0QS+LSCa3LIRB2/diNHxKWT3vIJeUPvYSF9JRJLB5BLJ1SwRSWSXtK61D+s25tyE7Da2776daiVrPx9nvV/KIqKIhN5xzkIxDY2QrkZC//yYOrooI3jhGNtPdmf7dgkdrKGI3K7rY7mdkRc+K/v/OB+c0Vmlxksf2YJjO9ZgzYETuOrefRjp+6vCu42BNK5xTDkLWWddxGkaupXtuz1Oen4P7EK278oRTVAjmqBH6CAjgj0h9Qy7iG0/iWjCJJQliBhaV3Lg7mXEh1BEHXmBTIobc8z39kGuZtPWjB7aPXj0sJoDIcJl9q0Aul/GrtT5ci9zzWXOOWtReA1joPQj1NBthZ4ddj/2qyzVVw5dq4R+Vp9fz6UYC8+v/vkIyCN9Tm4VRfWOaU6Xn1LLaWbi3IYhOuj/RlLqGtDfcC6x/MvaMOxkYeXALRucUXjlPa9gxWS/I5p00skAKRNgX2UUrq6Mwi4lTSeddLJYsqwNQ12E8R0AHDoY1yiuo3qYFToYjIeMDgbtLTGEcfuglDOxjucYlHomJpXEFUsAQwSpJbCu+p5ePYJDN27A6PgUdt3zCtLJwhFN4tjAguJDJPTP68Jk1Ywk2nd7qkxCiftx7N9c4gm10g21kD06WLhfe759KUYZzqSyiUg+mUVsoVuP4wQpjY1td6if22MYW1ifO4wtjPfAcVC1tDa8Hu1BQg9jcsr6b7yKVGsXX6io3fZXACa3rXJIYXbvPvT6HnHi2EIASFVZS2ZtiCZVP0pR4+si6yC20OgARXWT3TvHFgJQqWonmkgVTXpEGrH9lYwiKoloEiS4XnqIYTpS3Q803eft3wwhiujuwZw+JXoOMGGlKQbRtjWhh1YnJcquk1QwI3KK0WnnGRoUd+hu2nTPt8ndLaHwwm+8Cg1zrfQUOfX4VkO6GMFLNGqElJKO4ykAuGdgosKk1tabZO8hXAPb0lu5GooSdDERxcxXj18M08z4de35i3WWoRzyKJFTZiPL2zB0rmQghq5qRt6gHINOWddJbGLfJucVdMsOcBW7eYZgFrt+LUagP43QGJQqlvhcg76NXcj24ygB9CamseOr+zE6Po3eZBG4gNn4s2Pl6iOhAclj+2w40tjYmDSGX+jmYBtfY37cxt5g1aLbOGhfgoYhy7Dkk9kwkeVSePW13VjB0OQxgfFGc8ftkkHHjGfuH7OOY6ZyHLyvquPzXjvlyttxlRNvIBom8gWvTGLnfQewft8E0r6mfto9yB3rWIXuYgAV0STSpaULqg8rn9SNRTgdGYlcRy1gJYdEE0Xu4NBYjBjGgrFYYyKLeQztRpaOWNe8Lv29ytali41FRyax7WXdWGTCihIMSGY2NxmJABrzIUoklSYjEbD3OPs/7/bsRfc+di9bVzFXTWGylyONaI2eUui9dso/sjXq3uDYQKRnC2BzFla6qq0H5Y08N1jRpOESdr7wvMMfonaPAWGF+sduaF57UG7D0KUcGX4Iq6EoMrarweIP9mFl6f0c6+SsktevXItixHyNVh96G6OT/QEjOumkkzIB3rhmnXuIXPjSROc+7qSTTs4KWdaIYTnlj+PqI4GOjueKDto+UioZCR3kPm2uYl5rPvIPxq5iHf0CikklXgenO3TLBhy+dSPWbRvHZf94qDEXoeQ29nWM/Rpt5BIpNY2G7Dbu03jXz80zvNuYUT+/xwht1DyfFtDKpehIlpE+K1zZRHQVi1VFpIolwnoR0ld3NYcu5HiMlH8w2Bu3R6k0QiSQ1ov3SEgIoyKBe1lrHPi57VAALv3iK2Ye+0VIgAN3XYLxHauhUoWLf/gmEkIH2ZvriCYWUVA6cCGbd+9CTsg9zHWP4yonKq0TTUBEk6AmsuBKDnUROkh5DBWjf1E/xfWRG/MYLj3sgl30UoiSvaUb1rlFF6t7JKGITFhh9JDnAvz3gL1KMXoIQMyHOJikot3fIHR7VumPBPcyw3sOwSNXsUf9uN2o+spfOwBwyV/vs1PFUze6lYEQwQwrpIQInUgkiWoq+yP7d+HPwq9ne/I9RNNQJvzYfr7yif9sY0IKAuSxmjuqhsKu6Hie2ciyNgx1v34RhcYgWQ6RTuo3jBFo22YaLwhIbuG6jo9nk3+wyRhsih3kj4d1h29Zj1dv3YiR8Slc9Ojrjmks5SKsG28hC62pH8cQlmBjktpb3Ma+fJ2OzsGuWd04MJzbuIR2MYTsLpBiGb0xuTQNQynGUEp2rSRmseAilmIVZV3YznPGx5Lxxn3CGESr5z7hOkkwd9VH18fG80nuZwVAj5hHjXUjKxik0BqFqw+cwEW733LzsQsZAFJ4d3AYT1jppGTWVPouTHBdHXNcIRmO7p1PonpXHFfIBqNZsOZeDozAIIG1H2NOMEp6LcUYLkVWMrmSlWMg28+t4VlExmIcl6i08kxWMl5iY1GXzUai0dXzITbFIlopoZBWG2pzL/fB1yi5Rd091v6o0j70gCZiA7FU5tqJ+9PUoQe4wUCMDb7QmPQGYmjc1UvniQxkFX9WfJ/3RhobifUyejy33w8zlM0a9RyI8XyxMcll8mYjy9ow7OTMiDUKR8ensONL+9Hr3MeddDJQ2Cg8/8AJ7Lx3f+c+7qSTTs46WdaGYVnZK8O6h1k/G3SQ55OIJm0l6rh9UHUS33/4/IO1fnE74jEQdYdvXo8jFVJ42Zf2I53sk4tXzkXodby2ETH/oKSjOQflJGSEz/QPUcL5cBv7ucNqJ3G7puOlLk2uYqniScwCBga7kO07M5HZhWznjX8Hi1VMeIzULlQ5CdFK38YuZKejP2mNnEL52XgvbBRece8B9CKiiR2TkouYXcjm/EMXMmCQw6TGMA6ZyIkrdef7xVVOVE+FLmS7oMhArt57hPoRuUSlzXkMVRNKGOc5DDa5dMSVFQTgSY0eRbRfLpWCnkf+PXQ1W114D2EUkckqTeghADEfYlkq84sF4X3KUn6ayCmxe7lH4/kKZRTO/N+fs8Ra7ivP+Hf7iFFDO4cSjq1KQAcB7T9vaot9OpwXUa6Mwudq9++foVJuw4LcwZzb0CGAQrUUqbwdz8HHcR5DFX2OM5VlbRh2Mv8yOj6N0WNTuOwr+zE62V8mJk8nnZxh0ebaWXXQGIVJhxR20kknZ6nMyDDMsmxlnucnq+P1eZ4fPTPbmh/RhUftxPyD9lhHqCLODDro2+TaxdwO1GMIY9TP9AkRTin/oIwIxjWO7bGgUwpamTXW7JnA+S9NAlq7fISMEtYJJKqO0KEeT1goDOwnxRPWayVr4VzOTDyh7R/kMaTYQsCjjCWFZi+Fa0eqchK0C3GHYo5AmkfSxVVMmubk/0u1i+P+QXoc6sOEkxhFDOora2GsMJ8UlqdovNEbNHDLI69BJwq9svT9qI9FCl0uNpFoUq9ywulqGlPU2HnoBBUhhcKGqv5hHsO4yklITmF0MOwXVDmRaiErBaQ9YZ4Eb0/1sbLa1lK4dpLR6qD0SKG9/FXp0Z8gZRr9feIYRK1QK4/IKCKTVZrQQ8B8J4oi/Dab71IZzqMhklOsNMUdehjOD/AxiHyt2bnhvhP2Y+hp7eLPAfhKKS31laFApAyI7eGu/HoJd6Ae3Df+CcdYofNu6XqsYowsStVL/HNEOV0tb6QihJfusbyePWaP4Fyi24c2DLMs+0MA27Is253n+f8A4H8D8GuzXTjLsgTAHwG4HsBpAJ/J83wPtf8KgF+FeZb/dp7nf5dl2QYAnwdwHoBDAH7ZGqqSFFP+aoq/3LGxOCjHYFs/iU3M/WMjMJhnBq5iyQh0fckYnA3DuM4c9gbdkZvX49TGFdjyj4dcQm83X2QMtuUiZCNPchsPIpe0uY0l97I0di5u4xJ1w083HHNuwwMjr2Ja9ZFl2f8x12tnIa6bWGJXcexGjoOc2eCzwjcy1sVrxExkZ3RFTORgX6jnH1QN7X7sYBeyG6sjnWa3cehCtv0UgLW7j2Fy5xq8cfN6bH7sqOlXarK9KE8hHTsGMTSGSmYdlLKzOrLJejrQu3c+Rt0IBGDcyBIrmZNVx8msY1JJ9a7aSt4FxqTf2O985Xs4cvwEvvW/LJ1rxxnapSaD0LbqoOyqanM1sys5BjbIWJTyImqlUPjsy9V6CmlVrm+Qe9nOU9C6be7lQgOp3TddLzb63Boa/ieR+SwCt3LV3tMa6585hkLRjyw3RDYQ7UdugfiEnkv2+i1Ve+k8NhaZcMKJr4HI1WzHKDag+Vnin6uOMcwu4MgoNWxjbzgCzUmvLXDBx+xSXijyydo8zz+ZZdnHsiz7rVmv6OWTAMbyPL8ty7L3APg9AD8NAFmWXQzgnwG4BcAYgIeyLPtHAL8F4PN5nv9ZlmW/AXMR/37TAv/hx94t6t/54n5ctfcwAODbN2c4sn5t7WfBxjcncPt3nwcA7NmxCc9ctTXsUPX/+FefRFJojK8Zw7c+dLW43k2P7MWmw+MAgG/95FU4PTZSs+W3vXAUux5/FQCw+91bcPjyddFyCueNT+FdX3kRAPDaZWuQv29LbS0N4Ma7X8KKE31Mr0jwxM/tlLaNrd99HetfMHva87GtePvCsaAdAM7ffwIjJ/p4/dYNSE4V2POLO2qfUzJdYusXXwEAnLpoDK/duVn8DC74x8PoHT0NADj689uha8UvgZFnjmHFs8fMXLdfhOKSla7N3T+OnkJyv/nb6Z1rgBvXm33Y/VTvU3fvBfoaavUIxn78UuETAE5851VMH5wEAKz72GVIV47Uer390nEcr/4ua2/ehJU7wr8LAPRPTmH/V83zZdXWNbj4tvC7cuyJk7j1xnfhr7/yNw/Nw7Vzxq+bTuYuOgFObxzD6fUrcPJU3zzEi4HDOolk8tQUfv8/uQM3/sa/WzLXzp9fc5uov/HQPlx75AAA4Os7r8bhNf5eYu83m46P446ndwMAfrRlM564fDsAVbvv/uy3vodEaxxbdR6+9u7r/BzU793f34PNrx0HANx357U4vYKfO+b+e9mLr+Pqpw4BAH54yzYc3H5hYK8BwMqJKdz21RcAAK9uX4vd790SeK3s4jfd8yLGqufOYz/3DvEz2Pbd13DBCxMA2p87l3z7CPpK4egNF+DYNfV7rpoucXGVymbqojG8ecfFtT4awKqvH0ZSPXfe/tnt7scHr6eefQtqt3nuqNs3QW1ehVFqHwVQHj2Ft79p/najO9di7IYN4nqv370H6BdIV4/g4h/fIWJ2h7+zHycOmGfvFR/PMLKqbg8cffEo9j+2HwBw6S2XYsMV62vznDpxGo9/+SkAwIZtF+Ka9++qzfP1v3hI2MFgmUlk7xQA5Hl+L4B9AH5yVit6uR3AfdWcj8JckFZuBfBwnuen8zw/DmAPgOt4DIC/B3DnsItp+9KALhTKQqEsEgM/CX89rVH1qVzB1QTavioUriwViiIhiL6+sO1TlowRxOspFKV5lVq5+e3LXqyFViiQOMQvfgEGtu9DoSTgntutzvbTNBfL2xtX4LVbN2BkfBprdx+TPycAhVLu1fS5F8r8euwL+3AvhSrHoRbPy63nXlrct/11ZtfjmXjOEiFayD2tlNAoqpf0d+MxfV0GGe6tJIn5fTlP186CXDeJUq1u5KBqSeXeZZ09tvNIOltuLkTs6igir8FjQpeucsQVJq/Yl9LKEU6sG9m327krBFBHY6P1YmJJ3E8p4PCdmzG5YzVWHTyB7fceRFroyqWso/kqveJj+7IIoX9Zt7E9tv+3Y1zfxL+g4Hkc1QmqxCBbqqfEE7TooXMV22NlXmFbEr18P4cyKtZRX6tLe8I8CUZ7KZRKltS1478IkT4FVM+8ghgEhN+dpKcrlNe2agPDKe06Jj1DOErScA5eN0k10tT0U5Xe/RnskyXRSNMSaUr+JxVuX8FU3kmSEomymF/0JFMG9U6rp04TVqVg+9mx5u5Zvw8Y5DBtuulCuO7EteR26f9MUhP3Xt1D2nC4YTA6e88ZtJ9h5gxzw84eIYxlIGKYZdnP5Hn+twD+ndVVv55en+PaawAcp/8XWZb18jzvC20TANZGeqtrlJ/7+vfccVyWzsrtjxlUUHQbVx/0zhdfw84XX6vG1+cpobBm/BQ+9rdPBX0A1JJUf/DLz1XrhfNo7V23Vz56CFc+eijclw5dxxtfnsD6vXntlxu7hXunS9z8uT2yq5h0O+49UHMBH7l5PV6/dQNGxqew7csHMDLZx8bvHRVzEtqxK147hYv/8uVwHYL07fsFX3ylNZ4QAEYfOlJzH+toDPaMo9wzHujY3QsAemIaJ/76xdZ4QgB4616zb651HHlv8Nbjr+Lo44fD84sMxskD43j+C8+ihMbxZAJryvOhkwK786eB352Xa+eMXzedzF50Ahy+wxiFvclpjByf7ogms5BvPLsPd1y3Az91yy6nWyrXzi/96DvmQNddwFbueml3FYNYdaUbnf22XHXoEK465MN3TD9/k9QA1p04iX/yze+aoUVYfpX7/sT9P2wsu2qfIdf/YB+u/8E+lKUSM2MAwMX7jmPTK+MobHs1lrNhjEyVeO9f5GJYkwE1jGT3HqBcs/751FcKBz+wCQBwybePYN1TbwXPEzOPf+6MvnYKm/7yZQC+tF4JuDAl684+/4uvuHaOOQ8oAA8dgYamfYXPEwCY3nMcp/ccr/bhnxcFPfiLiWkc+mIetlfHPNverz5fzVP69aIvzb7H9mPv918J9s2lWAHgjf1v4tuff9TNo7V2x7ORYVzJn8my7J8C+K9Ymef5V2e9qpFxAKvp/0l1gUptqwEcI/3bpGuUsqCLKKo+YnTVu1ClJK5TDDTFCNJ6DReTZATW2lCPIQT4gmlPOeP2heFTz7AxyBfeyU1jeP3WDeiNT2Prlw8gOdF35BDfj/cAmhNO7+aWdLSe1+lavzCdTXhhDRNP6OYJ2qN5GuIJ/c3D96/vQa6VrKHxVu8Y3sJxbJreGJiPc7x2zvh1wzJMlROnm2GKGu7blKJmUGyhH4Ogn5Siho8lVENRe1OVEzs2rnJij49dvc4hhadXj+LE1lVuLY4tNGt45MTEE8IdAxhY5SRMPeOPXT9KUSNWOaHYQvfOsYVAGFeYqDC20C1UjZdIJTZFTa8Xjqn1U7wh3P2DF/GlJ/fiN3/mdt8HS+Pa8TGGcDWnNbMj3A1O12IQdaBTTheTVMxXJPgPEkBMcdNETgGsASmcgzWgKCk2p7VJ7TrCs4qJKYW7TpTr5a8EXTuHEgo9rTG5bZU7XSak+HhBf931lPKxjFV7X6FGSOGVg6TV7jNxZx/cryWPVEheqeYekPTaHnOsYZi6xoyJ6ysn8KmJxFQ3qCfPDuMSZy4DDcM8zz+WZdnPA/h6lmX/H4DfpYtpLvIwgE8A+EIV7/E0tX0PwP+eZdkYgBUArgLwTDXmYwD+DMBPAHiwbQHr3pXII0Y/P0Zgbaxu1gOhEej7hEYg93dj7NpsLEY6o7c63yYZdCGq5w3GFUdOY+O3j2Bs/wmoE/2gfF0b0tdEKuG+fj2rqxt081XKrhT7+XkkdLCJbRyPZZKJ/yUYzrN1aguOJ+PYN3oQa/qrQejEXOSMXzcAxJtKExM57juoyslMmMgzrXLidbJBONcqJyC91bljDVzwzDGoVGHj02/hhX9yeaX3LuRgPXjDz7qQAQTGoFTlpMZKjqqcuHc7D1myQ1c5aWAi18gpsXFX9auxkmOiidUxE5na/80v3YWvPbMPv/rH9+JT774Kf/DfLaFrp0ffxfhmVIJ+YURGYtUeGIlOpwLdIJLKMOQUAIagYj92u39iNIcZNSqDj9BBS04xLluLZpmZTNWSql9gILJZZe+o3jAqyexJqt62pJ41EHvao4MJ5fSLDTbQCgA9w1W9LSSf8DEZkcCschuCrv+wPTToGK2UjDxFe2FjsZYjUanQYJmhDBVjmOf5FwDcBOASAI9nWXb7rFf0cjeAU1mWPQITzPvrWZZ9Nsuyn8rz/FUAfwBzEd4P4H/M8/wUgN8G8AtZlj0M4DYA/2Ye9tFJJZOXrnIXytofHcdIV9FkzrK6WI1tp7airwpgfq6d7ro5y0QnwMmthiilNLDhyTc79/E8yEev34G/+uzP4vXjLsyWoAAAIABJREFUJ4Du2umkkwUTpYewKrMseyeA91avn4Qh6vwVgP9mJmkvFlpeuPqjGojiMixqF9UorqeKQdBux8T9hq1IYvp65FF0AUdjGOEfJuUM9zXzVP8npK8JJXzj5gvx5rs2YN1jR3Hh40drrmb7zuOtjlHCQYii0dVRPSmGcFCNY9YV5DaWchK6eQa4jX0sjB7oNvZzC/NU/d5Wp3AqOY1T6SlM9CaOYolcO9dvus04qzjXYPV9SJEE7mX7K7ZXQRcKCml1zEid19l5PNKX0i9ie5woOqYxXgevU/V+1h2itD9OXTv3NdLTvt26pFLABcAnQb+qXWm8fudmnLh8NbbcexDn7zthguYB/OjTO6AAXPO5F9Ejt7F970GjV8FDqdJI7XGF7tmAfwBIq02kaYm0Z3QJ6eyxe+9pJCNVvx58NQ4L0I2q/7+9dw+67azrPD/P2vs9Jyc5JwknycmFEIICCxE7IYFWS9AD3hqm1XFGuyympxxb7RlmptvRHrusVqetKasc6VaL1hksFWtw0CmVhhqQJCaI2hChaBQkgCwDhFzOmyu5nPs5797rmT/Wc/k9l7X25b3uN8/3rbf22r/nutbez1q//f09v98PZeKLOIZrrFAHRkZmznZc+eO1Mcqakq0tcTzyJuQDxqt/NHImZ1e2tubjF9qwNaNRJ4eOObTsoSn/0pMn+fTDT/G3DzzGB/5zszJr57n/7jvsTUCwfp7907lj+9L6ujKsTXYvYiSL9yIGcgji9bYixm+8v15r5cszWbqmU0W8LarV0FpTs3iO2b2I0so1tW3w7KJ8pk2B+/7p1wKKr3/3l5iK51cuy9ZEPDsm4pkVx7i1DpDdPGRbw0b6Syv2GHoGT25Hks8bW9ZG9abaP0Oy5eIZI61Xvn2byLQoc/3p1u9R1Lbv1oUf+sSj/yk1/8zATMawrutngT8GXkv3S+q1wJXAF4D3LDpgwd7DV41SOD65wZHmudkNCubCFw99mccOPs750XkOTQ9BWTv7CrqCJ4xSeOjEWS5d37O6ysrh9b/w+/z0//NnfO6hJ/iHL3shlLVTULBjmMf55GVN0+Q8wX61rusf2+oJbSVyv3os4hzF+ewki7GDTsb8jiRBm1w9cu3t+1QGKdMXsnYqkD19m1cKr/vAw1Rnpu4X2tDeQD+n4f2E8heZZAnlPkLfNmbjhlnGefYTQvdrLGD9RN3+sfP7CWPZVDKGOmQZbzp3E10o1E7WNM0DpvmeXzsSs7KcJPv7erKceBlJedyvPV4my0laL+8EI/cW2vdyb2FffUW3r+upb7+ecy85zKETZ3nhnScYT1pXXmnNoacv0OVA1tFeRbsXyedKnjfLSbDvsC/LCaBGItxJzvkknJApS/cQuhA19jiX5cSyiItkOYllMpeyqnj/T/8AR684DGtdRLl/8m/fsTJrxzKsutWoiDHsZKZisLHaNtaht4Rta44XcVKRew8B2olyX3bn/tGz77Aa+bb2+2b37I9GODbKyioUVHYgyxx6JxU71xYfCLt7cMS7AjWgOPT0BSPVRhQ9MJSO9veFvUC6n7DVIhC2qB/vIUSTdUSZFfQat679sD7bCTPK/RPcfzb+nG1pX/5kx9xaGSrxpl4E8zifDIUH+P6lR94BTCY+yFOS1icyH2+nEij7dG2ierJuzlQcy7v6eSXQ1c/Q7tKs+/RtR3n2tVczOrnBNX/yCOrMNKDaI105MRsnsqj/Tjaf2XiaqRe3nTeVnes74y28lWZjN0dhBpB1rVKYi23IHl8783oiB/KgbtSfUPL6spx0r17Rk6NUmb5z5YGnskh/lyp8IquKnKMOZfLYlnVK4XVOKbzxzhNUEx20VcDX3vEIY3ej9nHfXH9Cf5o7y4l44OeynMiyXJYTqfCljiQqPHYyobzFWU5yTiVVhco5miSyKvSGEeVHL7/MKIsiUJ/Hnl471qNbtTpQCHMyldwcxfEmnVRib+NqTOCcAh3r3UotiW6bhPRebp1CaNQbrYSSaEyYrXIKoVUQlVbeOUUoiHZaIyUmEqh2mq+742GmboVqoeSJZ63wrLZV3dY4YX6u5GUUx74fO4qHnE2sdMrMJzK5jG9vFW1vIs6Wk2ZD0TpW/rqaXpn0MjufqZAHWVV2KPNJgqZpms20L9g9aKC9ZMT45AbX/skjVMXRZEdR1s7qQitFe2DEJSfOcv1dJ4qjyQ6jrJ2Cgu3FphTDvY6pjGM4g/2bFWPQy5ZnB2X9PgeTrg2uLBuGpifkTG8/4gen7OPyjz3F4b95Gi60SZ5iOTfJEqasHjNjES5jNk5km8xxvBVm4zgcja0ng5jGzieyzSrCZjlJ5DkzbpThJC6Pe5kVoka2CVi9qG8lyvOm355cyRE7KCO4uPF1yjaOp5rr7jrRHW94c7CbqwlP8/TLLkcBV99/0rCWtp5/lSFqpLkYcFlNpExVOjAhW1liSo5C1DhizhJwgkVUGcZQ5ULPxOyhfY1YxCw7GNTzjisByyiDMELnjKLiT27vQ5qSkUwhIB1S8owi6U22JWQPrUwwhbatZA9dPBd7JxNsszQv23iXMqwNhhFsK4UNSe3MldOIKXTwTCF0oWwsUxgwh71mZSutePJll6Pp1g7CbOpNycLspLRv7/qbL4RNC0Eu5a6eZ/+nyt+PcrENpUnZP6v9/UBaKNqotSy36GRhP7n8yTqJgRiWS9P2MtjXiqH1kgKhqPWYj7dKCfT1fP2h7CQ5+jpnKpblocKX6Uel9W0/z9x2FNVqDn/aRJO/0Lrg1fEclwlM3UblOmivMzJbb/YewmXMxrbvRYJU+77TfqTZOO1PBwphPO9VQpXcsvrNy+k+QdFGmI9zewKTtkoFN9N4HjnlLRtLUaseZTFVJmUw6/gcrH6lK3jmDddx6f0nOfDQGUYTfD+Rgmn7PvGN1wBw7P7nOiXQnaOt783LM4NZi7JcMOtg7yChMtidRKjwqYzZWI2rjCm58p7IMpi1VAbjYNZSCcztJ5TmY1Wl5e5VKI6rhJG9DqniR7TvUA8qjqa/JfYiMsG51i9qXu5MpcIcbL+5Zl7VqFMOIf/xtEIJjE3JUlmUZmW55qHloW88BsCx+08aVTE81Uprr8hp/7yRAbDj2IfSbGwv91iF+w39+fv7dzx2N1+vRHb9+diGWtSy13Sq/foPlNKEDNKJIjoV9aSZORcUW2Zfyd3D58UKrrqCZfHsbUc5+ZqrOP2KK2jXykdfUDAPdAXPfPv1nH3pEU5//ZUrqOYXFBQUzI99zRhOo/hLQGg+trRr5KHs6i7BDubK0piFaRsrB7Km4k4ej5Oyg339PHvbUU695ipGJze46k8eQU/aJM4TsOmMJfLXUPeqM7K8yXmWc8kyZuN0DpszG+fN0LKffpZxlSA9keNfnrEnsjQhxzLfJmwP1pqpgvLYjJywegNmZnlcRfOJ6+aynMhja7lTFTzzxus5/5LDHDxxlmvueTQyQ4uMJs7cE2Y5qJzM/toPWcDuOJ/lJM1oQkaWmpJjT2QVn2wVmZDNa2BCtmXZLCfWU2EUmpD76gXm5bSekh40OeeUVYJlWFvtaD/3uUSxDfNeyzmZ6TvHIkozs93vOibrnDJkXrZOKBW4rCntRDid9Hgt96HtMSXnzMp2wUx1vMY1Y2BiZNaZa4IKGEHP8Pl7S+o0QuCIAt0lDhxRzFzsYcBGDpmUg3H8NZb3AX9Ofo6eAbTzStPkSU4xxwF2/UR9i36WwQquuoJF8VykFI6Lo0lBwUzoCp41SuGBE2c5dtc61XT5m21BQUHBKuD5wRhm9g2C/4WQDRmTYRa7ugNtsmUhI2fLY1ZPtg8Zuhyj6OsP9dMC515ymJOvuYrRqQ2OfvARODPpEowj9vtGbdy8VSybHXpmGaeS4XpiL19mH+CsNrZe0GaL9hNCyhLmnU9WW5kYDF0TyMjIhljEfFkc1ibs29cNYinachGiRrbNxUNMczyHLOLp11zFBcMUXn3XuotTKPc5SuZR9l2Jz9zuL5R7C7tz1sGWPrm3EDDOJ6YPmRc58tGQeY97Q9S48pQJzMn8fkEVsIfB3kJbT+4ttBPLOp/ELGLELMZ7EKVsheD2ZLZtkHWrk2mU3YSnNXrSOjkwV4ibTkZy09YT7bPYtL7PWfsOW8MiSicUKXOOKqYbGc4m59jhroMyuZjJ7zGM9xt2jXxeZegYwonoP4glKJg3t9wGQthUMoSNmGds0epYQj+OrSCtabn5yNA1tl/Xp1JC39CuXqwbyPzJXubZWumEInMlJw4rqMRSuQj2t2Jo+PB5zcNSLj+wPhOxLOvKwzJbHqsFuRR1nTwej6yXsFQGY0UtLIe1B09z6LPPcMl9z6JPT7Km53mcSuzrPB7GcZtlzMYQKXRLmo3tvLbDbJxrmzqfrKgpOWu69WYKiyrjtSxlXlkSnsqBzB/b+nmzi5yHbSPGjF4DZVGW67BeLFPi+PDfPoMaVxz9+FNUU2ky9sqdIjQhy/H8PDszcmxClrELe4NZx44mWQ/kXL342GmlRiaVQKEgDih+eQcR2UYqgU6jFW0j7TX2RJZOJxaraEq22s6oQjltwL4KZbEVTxSp+M3yZI5l1gA0RihY2l/vIfNyxisZoBrjZLEyJb2WbW5IqSy5U2r9R5pzSJHetu7Hp9Zo5f1spdOWhHY1rFOJ8sd0zynpiCLnL+uh/EdjL9dUniuBzmr68cGjg3qZoNe2fCq0By/z9zUt6lXRuXRtw/uglMXBroFNmZFhnyuGz2dsHD3A6JmLKA1HPvZU8sukoKAgha5gevka42c3qC60XHnvk0vvt7nlj7/svDsLCgrmx2v/+P6ycnYRu6IY1nV9CHg3cAw4BfxwnGGlrut/B7yObo6/1TTNb9d1fRT4e+Czptr7mqZ5e984Nln3vM4j3fHm2EFZBiGrJ9vG7GDQZ8DahX2F8wnLbdtTrz7KmduPcuTDj3Hwy6ez8Qfj8WY5lbh5x+MFDN1wP4uYjSFl95YxG+f62SqzcdxWbkpuM222Aju1diSyoWsWMCFDyDIOOanE5uNYKeuypcRt+k3Its8hltGWqQpOv+F6Lt5wiLX3P8LaMxeNo4lO6sdZTsLxOnZw7cLUZz5RoQnZyiw7CKEJ2b7Ppr+LiDeUdC4RE3MMnZycMAsPZDnJmpxlTEPBBAYmZCPLZjnJmZdzpuZcNpRNYkfXzYG17rVt/QNHOZpMsH7CaCidVGaEuLFlypih9Vh0IW702jCFWfOyYxm1byMdU9wSC0O3eJl5J5jDIbOyRduKNV+1uGwppvWUipFuGV1suzA5yjJn9h4rTab+eWjXqDQBB6Zm6DqKnqGyXD7HZGxDmSoPCJ6n/ooMZ0MJx5Nz8FYG+RrKPCMYl8XnMJWy3cp8sgm8FbivaZpfqOv6h4CfA37CFtZ1/QbgpU3TfHNd1weBz9V1/R7gNuD/bZrmX+zKrFcAp199lDOvuYrq1AbjJ87v9nQKth5l7WwDdAWn33g9F28+zNr6WUYnNzbd54XLxkzRHDxTnL32AMq6WSGcv2yNKYpLytrZFeyWYvg64G3m+E7g56PyjwGfNseabifDBnA7cFtd138JPAH8y6ZpHu0bJMsYulfJ1G0dOxjXk+ygbBuzg7KdL1PRmHF5ONYZoRRebhxNbPDqHLO4iFOJn0PM0A33EwepTusJhm7AucS2C+aYYQcl05cLer2V+wltWW4/oZTprd1juCNrJ5cXOXb6gDz715/RJET+17Hcb6iy5fHYWWZR1M1ujMfnRVYVnH3j9WzcfJgD62d5wV3rjKY67ceRcf7ztFlO5Nys88l93/8SAL753Y3ZT2jH9uxfmBfZy7t56WBvoZ1rfo+hPxdfzx57djDvaCJYwMj5JHEkmSsHckZWKeJJJiFq7N7CHIu4eezIugFQIx8QXMcb3UQIGyrhMpAp7wtxA6AmLVrmZI7qqQn+6S72HdpnnRprXybZQ0C1ijZI8+hZOLCMmZUJ5lDsN+zK+mC9WSpsXmXLHGrD9P3n738paPiW3/9CkFPZO274Z2PnoBKzg935Aj5cnQodUboyd3rBXkN5DvFzUN6jhrKhoHRwj5LBrvv6rqRuIO8zdgzhhNKfBSXPMi6CbVcM67r+UeAnI/HjwHPm+BRwhSxsmuY8cL6u6zXgXXS0/um6rr8A/HXTNB+q6/q/AX4d+IG+sW0S7r7sI/444yASlcN8SmBXNqPvQCnr77NPwcyVn7ntKOdu75TCyz74SOdokp2P7S9U2HJOJb6NVLCG+xlyKpFz2M1UdouajeW8h7yOYyURuhvRsl7Ju7l2LLJZRVC9pmHfxhwPKIvSLNynQOaypSSOJjqdSxX3qdO2io4ptErh2vpZjt45X0iaISON3DCv8GZkaUKGUBm05uNO7svz6e/CScTp79wgoabq5RA4nyihDKbm5Sr0RI6cU4IsJ1nnkwGlUh7H3jLBySyGXV83zitZmI2tIiID4wmvx8C8HJer1imJzot5XM00LyexDyfg7p7SMUWaooF2oqmM+VkqiDLeoTcACwVxDocUCR0oN1o2NQPCSGmm2n+VrcISeyo7pdVOV3oB26+nuPTSgB3HNpSrX2sC87StEGcF66YbKdCivBLHfT9uwcc17ObjlVeZ8aSrr3qzoLj2g3epYWy7Ytg0zTuBd0pZXdfvBY6Yt0eAZ+N2dV2/AHgP8BdN0/ySEX8YOGuO3wf879sx51WDrmDjhkupTm1w+IOPoEqcwn2Bsna2H/qSMZOrDzJeP8uVJU7hvkBZNwUFm8NumZLvBd4MfAJ4E/ARWWg2Cv8Z8CtN0/y+KPod4D8CfwR8O/DXQ4PYHI8d2xRqzwFTN6t8AXYwbRu2s+W5PnOm4nCOpjz+taLh0j89gT44cubjdL4pk7eIU4l9nSf0THgui5uNe+MYZhjDOBRMlv2Tx1toNu6TgTdZyL63CDuyduY1IUtGYND5RJSHv5iJ6oVhbVRULxf2Rh7HDiX2OHaQsfEHq7MTrvjAI1TnpiihFHpfDp30GcculE4nOVgzsjQhgyXgrMyzhnmzMa5eakqWTCFClrKDDJmSZd7jOGwNzMH6qUw9wRzGcQyDeqO0762NY7gj6wbossGA+VBD2kq12rOHTHEcWY9zChgz86RzK1BjS9vNYV62TKJhEdXYP4wCx5RIVo09U1iNvVnZMpnVWAdZUsCydv0OKS4MVOVNu6NRy9TUlbENle7WiqbLP6w1mTh+oSNKHGYuyKXs7s8+fMxQbMNW49l4IZfPQ3k/sjNI+/H5k4PQNPa5q/0zQzKY8T1RIe+tOpDHkCbnzYSs2S3F8B3Au+q6/ihwEXgLQF3Xb6P7xfYtwNcAP17X9Y+bNj8C/Azwu3Vd/4/AGeDHdnriewnnbnkB46cusHbiLGqiUZPJ1qogBXsRZe1sErqCc990DYfue5bRyQ1GhWF/PqCsm4KCObErimHTNGeBH8zI/7U5/ATwaz3N3zDvOBtCp04ZMV8mGbxc/UXYwVzbXN/5PYZh+6AfFcrPv/ooF26/iuqZC1z23oeCXzP5PX3hGJIlDJiu7Bx1KhNhNoYZxTBItR87rBeGesm0XXI/oa03az8hcZtMebyfMJbpYBwv20rs1NqxiPcPQrjfL95v2Mny7VMWUfZjZWF5Gs4m3bck2cgsiygZvgouvOF6pjcfRo0rLvvLx6N+wrYKEqeTTqadLG4j8yK7viNHE/uay4ss2+TyIlvIEDXJtryQyvR7B33naV5kI+9eM6xdT7ia+ZxPfD0V0p9+4tlwNcGus6Wxo+tG7DEk2mMoN7opmOmcAh375z71BfYdeqbQOHa0OgyGDeZma9qYfYV6ol2Aa7nf0DKUrQiKLZnDnEOKZcfsx9hOfe7l6bRKvvNKefbQBoVXSjGy0bfNd6Nbfp6j80c5dtB2rh1bORT0ulLpPsOujWcgE3ZQHOfD1SiI7gkq06ZrF+oWfU4ofVlQfP97eI/hbmISKX8ScQq5tFwcL6AE5vrOm5Kj8US9vlR1tv/zrz7KxduvQp3c4MDd60xJFbKgrZJfMvsaKoOznEpc3RkZS4acSvy8ti+VXdBnYM7NtFnCbJyMgc6ajQPZCmY+mTf9nZQvmv5OiTZ96e+G5iBjF6bKYhi7UKHQFZw3SuHoxFkOf/SJUH/S8liLtr48kSEVRt+mUvDye9c7pxNbnnU+Mf1k0t8F3sbCIhs4ndhXN3FxfTaZ/q4bN6qXS3+XdT6JPpG+OIYW2awqmX5WANYrWWdMyUGarTFzOqdMgzR7MId5edIGCiFYU7MZTpiPEzPlWAVmZZklpZOlafTaiTcX2y+o0to5k9jUeNWoUw67U9bO6u7KVZcS7lUfe5hWV0ZJ8zEJbSYZee9o8ds9AvNyrNCJSytNyq24H9kyd+lV+mNR4z/WqetPiWdCdD2JlUSEXAWyitABxV4TmQUlbitl4Xad5Z87+1ox3I+4IJTCQ3c+AsUMVlAwE7qCi1YpXD/LZXevB3sKtwPXfOUUo+RnYUFBwSxc99BztO3q/SjYL9jXiuE0Yu4kYvZv2Pkk7aOPHczVG2IHXT3V397O4cIrrmDDKIUH73wEfXojaSPHkr9Qlsln7OcmWT3bd57pG3IGmemQkmEM3Vy3OMdxfA7LmI3jOeqIPbT1lg1Xs5voy4ssy+Jy/8t6vrzIso0cd5m8yLn6kuG7+C3XOqbw0ru7OIUyL7JtG7OVcV7kThbGLpQm5E6Wz4vs5ihC1AzlRe6OdfIqzcm2fmqFVW5iS+dFTmQ97GDW+SSiNeO8yK7eyNeLT0I6pKwSDA2mqsozfNKUbDGXcwrAKDUvT6ZoxyJa1rHFulLoceXkPoSNNzV7M7NwRJFMl8uWopM4h0yUN2PbeVfSjGumKo5dSKlWO8c26XziP+YWF9MwE8LG3r5DU2rIFJoRE3Nv5wBi78uGqfPLJBhDhq7xDi1mhiqXDSbPLPbFNOzOIc1pHFyzTN85RlARfXa2rxXMfFKwBMZfOc30RZdx4GNPUJ0ujiYFBfNi7fPPwpri0r94fNuZQou//t6vQQGv/cCXdmS8goL9gnvf9HI08M0fvH+3p/K8xL5WDOWGzlz2EYs+Vk+Wd33kGch5HEmCeiqU27L4cWVD3ehDIzg3gfMTDt5zgpbu3FoxTp4JlGxcbrwME9jjVAL9TJ+rP0eQaluWnCuZtgvkOB7qR9abdz+hlQf1coxgRgZsZ7iaHUduE/N25EWWr7b+MnmRu3E78zFrFVxoGX/1AuMPPxamErZtpGxGXmQnc+N5dlBu86vQnD9yABTO8SS30X5b8iLHsmXzIluZ/Dxz7KDFUF5keZwLat1TrlZwjyFj67nRuuvkrpvMj9znnGIYOscBzREUG0CLXUWq0oFTiq2fOKRMWpSZbsAc2qwhKJ8lZeLnkwuAjWPE/H3aEp3O4YTKuYq00yoI7G6htebMkQPmspg5uctk2Ebduu9OGPbF79mLg163SiVbPivEvTrY+2zqCfYw51Ri+5uVPzn3DFakLGOfE0psdaogCeED/h6rtQ76WhT7WjGciBtaTulyx9ny2SbiuK8+JXCefvLxDjUXbz3K9FUvYHznI6ivXgjGyyt8+fiDsk8rW8bDOJl3pLzlTM1ybn7eoWI5y7lE9rVMTELZx7Jm4/AcfNmQV7Lsc5Ww2diF8eM8ZxaRbRZJf5fMlfQGqyvQx69HX3GAtTtOUJ2bBuVKp2bjZRE7n7j0d8qbleP0d+68ZqS/c/UWSH8nX7sJpqbk4DhW9qQsNi8vmf4uyIYyFLvQyuPxVglBHEOvEFo4hW+Wc8q85mXb79ibdplMA6cUsEqgVwi7NlVgVu7q483H+BiBdq5xnEPIK4iqJfVeVprWfke1DuIbQmfiVaobUQOjSjNthcLjnisqckTxU4fQvJwzbUuT8rweyqHZOFWC4zR50lNZxjTM9RcqtB36sqDEZbJ9LF8WK7jqnj/YuPUo09uvhgvT7r+goGAmdAXt8etpbz4C56awURxACgoKCubFvmYMp+IH8LDzyc6zg0nbiNWaWKXw1EWqOx9Bn5lk4hxmGDxy55qGnolZwiGnEjnGPBlL4jnKuq6fiKGL+4znLc3BeUcandTrcy7ZCrOxHDdnNs6xlauKXOiZnNOJLHdtM8fS3Bv/Mo1jF2ZNzNE4wS99wxRy8xGq9bMcuPsEo2nKKOYQMI+WMfB8yWDswrAfn+MYcGbkOC9yzBzm8yJre7JG5icXWFnjCVURexgziSpnXo4cTUy94HgoL7I74ZRZVCH96V9zx+6aVKC2Jo7hjmIkGE/LFMprJM3LsXNKkA1lhnk5DkhRKZQJYaPHI8E++rA2eVNyjlm0zCOepuqJcwgEGVLsLa9jFu25Ghmty4yiFUF8w+6UdfeVMdbsbl2omY4o/rZs77vS+UT7+hHzJp1PpElZLid7GZ1DiibZelZpkoxkFWTNufK+ZdtIJ5SY/cxZNGR2ZJk3OVe+DPa3YiiOhwJYz6u8yeN5lUDZV59JNy6bvPoo+rZOKVR3PNKluaNPIYrmlVXOSOIPxm1mmYvt+9jDOD6XoViECNk8MQljc+yiqezk/OY1G7s2PcGqZ5mXEwVTKKKrhM3GLoxvS5VKb1TzxC6MWymtepTF7g5vlUK1fpaDd6+jppDZxpR4LQNzpb9zbbFmLN8mjiHt5hztMczFLoTUhCzjP/s6QiYGTkzI0nwsJxebj4MJqSAVoqsXxy4MT0KYlTNKXl/6u0SW6Sc+XhVk9hgGkOblKIaehtS8PJmGQbMhCnqd1uv2EwqzM9DFQzRKmfVYhkAhhFCBpJJr2SuIMs4hdAqi374gfvA4hdDUEybgET5+of3eBXsOVfdea51cxk5pFCZp7DjWJJvGNsx7EOeDXodm3vCs2h6Zr+9Nyj7od7rFS85D3ltyyqQ8rxx8P+lnsAz2tWK4itANAxJuAAAgAElEQVQHKnR9pVMK1ZkSp7CgYC4cPQg3XgbrZ1m7+wRql3dfXPflZwPmsKCgYD7c+ODTK/hzev9gXyuGk4yJ2CLH6MX1hthB+T5hxNQwOyiPk7iDGy3c+TBMNdMzG0HZLM/idN6SyQsxD0sYm3bnYQmz5ueEoQtZtmTeov5QKruwTz+fIeeSfJu82didQyaLSY4ljOeeK181dExfyv5ZdObgkF2sAplpk3EuCduk5TnzcQXJfJxJ+qkLKOOkNZrqYCTpdOL7icoITcheplMZvp50OgHD9KH5uo+fYCTMxrEJeZ7YhUOp8JI0eOJk4tiFSfo74ZU86FwSxy4cdD6RDGVkQpb15Ink0t/lHFJWCSMRm7GK7ryxeTk6x7x5GW/PFCZnFZmctawnsqUEzKFxWAniHWbMy27rwYw4h66sIohzCKAzzidK431mKlyEAeXWWIdXf/rBLsah8uZk8PfdSvlMI1qTeOhW5JxP5s+GIp09bLm7dyjSfjKWKonOJG3qCieUXJsk3iF5pxJXLmI6ujKlNpVxa0VX3v5D+8or0UfWujcnN6AwhQUFs1EBt16FNg+q6onzOxansKCgoGA/Yl8zhp6BErLMfr+ACVtg76Dtb4iNi9vbNsF+g1uOwu1Xo198Fn3nw+F8s3MYYAJVT47jiPWL9/4NOZXkzquPJUxYxgzzmJvPPM4lQ/sJh1nE/B5DN5cl9xP68wvn1Ve+SlgkduE8eZHjNt379JdpHLvQM33pfKoKquM3oG4+AuMK9cmnkv67+YSMI4R7BxfJi9y9hrELHTHn2IxuT+Hf334dSmnqv3m0m4fYW2jfy9iF8R7EXF7kcOK2kNyFTJ1LusFNf9EeRAjzIvfFLoxZPJnPeFbswhzFmSn3LOOqZj4xj9WqFVlDMvsOqwomk97y7L5DyRdEMgXkwtm4T7Jtg1iHXVnbsYbgmEMqNTvOIWKBQKdJ2L7tRypyKruPNGIRHXsmFpnScN833IhuFV//mRPmK2DmJrKiyNiGjnGUTJ9YoxA7pHSYlQ0lCDlDCsnqyZiGcds+JxTP+pHMO6xLcn65vMmeL97cM2dXFMO6rg8B7waOAaeAH26a5smozvuBq4AN4FzTNG+q6/qlwP9Nd/6fBf6npml6Gdyp+IAsckrgLEcUV9ajBMZtg/5Uph/Eh3/LVVS3X40+dZHJf3o0qTfU/6z4g7nx+pTBWeZiSJXJWJZzKrHyuL+cMjXLuSR3XrPMxnHbzZqNZZ9uvFy8xG0yJe/U2rGQNy8ps+iLXRircfL9IrELE33HjiOVwvWzqE99NZiDQoXKXfQc2yr0pb9TSvPIy48C8HWfejQJbg3WVJweh+Wm7wG/DNUbwLrnGAjjCg6YimNZNrB1pL3KANeuzhym4lkBsDeBHV03VjGcTvwTthXXyJqXJ5NQIYRAIbdQVYV2CqRwQrEmYiubCGUy57UMQskTMmlWxiiYA3EO9US7h4L3XhY/WDKp85weGXkqS0cU6JxRlFJ85SXXoFF8w2cfQWvlfkzZe3L3G8YqQ8JE7M4pVAhtWS7oNaINpp0WiqaMaWhPZegL4PtR4rkTBruWc43bSmWzr15XV/6AyCmT6XdpXuzWz7G3Avc1TfN64PeAn8vUeSnwuqZpjjdN8yYj+1Xg50w7BXzfjsx2uyCUwvaOh4v5uGAelLVTwcgohXr9DNxzopiPC2ahrJuCgjmxW6bk1wFvM8d3Aj8vC+u6vha4EvhAXddXAv9H0zR/AtwO/KVo913A+/oGsWpWn/PIImFqujqz2UHZVzqe/8WgXvUCzxTe8TCtUAr7TMWzQs4MjRezVvOwhMPxAv37RWIRun4yZuOZzFw8x5zpNsPgxanqhszGQ/WyrGVkSh4q3yLsyNrJhZ7pi12YOKf0MIqxSVaWLxK7cPxtXinU93RxCpPQM9EcY8hf5tJ8LB1MOplOZfh6OcTp76zjiXQ68XW9bMjRRMYuTEg0YZNPMp5YWRKGRpiFe0LXBO9tvXlS4lUilV0Yjyd9dWZoEaFObUvmkx1ZN4BnAaHzwADvcVG10UOmnyl05z6ZoCJmUYNgI8V30YazmUy7WIawOHOYmJL9MZiPLDIvd+ZnyxSasmo4hI3MPuSYPm3MwgrQWsT/NOfltl742IYydA1+OoJFNG3IOaRI9lBeC18em59lmrwg3V7meSrh29t7Xj6ETb5tyAvOF7Zm+efOtiuGdV3/KPCTkfhx4DlzfAq4Iio/APwK8HbgKHBvXdefAFTTNHqg3cpAP3YO/dXzTD50ojCFBVmUtZOHfvg0HKjQHzoBkfdxQUFZNwUFm8O2K4ZN07wTeKeU1XX9XuCIeXsEeDZq9hjwm03TTIAn6rr+FFATKta5dgFyewwD9msGO5gLDxO3nYcdDMpHinaq4anz8P892MP0mb57spPk5jgUmDqQifHyDOgwSzgrY0nufGKGL5cNJVcv/gyG9hP2sYeQZ/+s3M0nYjB7x1twP6EWfS+K3Vw7Fjm+Zp6g1rFDSkXoVDLUd7D30P667n7ugwb9xZPoL55khHKjSkYxdw4R8dbJohAZKip3MtePZQ7DoNbS6QQht+071oPA6QQiVlFs31NycCJZFXUOSWiarEw6ndhJ2LoxSwiivpiYkAf7AOPA1NmwNnlmUZar3H7CJTKf7Pa6seehx2PBCJpX3RJQeO6chSyTXzk4xhBqQbYUCPKCjAmzoERDuH7GgjW0+/eiDCmeKTSs1RhU5GjS3ens98iz27kQNtIRxdV1fi/aWfi6tQOjUduFrhGXRga9jkPXdFdEBQ5i3Qzz2VBkyJlOpogzm0jYWxFEzKM9NgdaheXzhFWVexrDtgMWCkjC9WwWu7XH8F7gzeb4TcBHovLvAP4IoK7rw8CrgL8DPlXX9fGBdgGm9l/5/4n4n6J7/yd4ZczVl/+m726ZayZJH7LcrMlbjjL6npvQB6tgHDvWVOngX7adGqVLB3PsFB6raGm61Dr238lMXTlHq3TZ+Xu5VwCTa6LbLrZg1PeUlqmZhe3TzVW3TMy/HMPP29ez/1rMb7hci/nYayPqae2UQvenfV9uDlpn5xP3HR8Pl7e0Zg627y3CjqwdJRS0SikXz1DGHfQp7sIbiYqOVSLz/aieOrLfqoK14y9k7fgNnZkWr7B1Cl46f+u8ax1P3LFo6461NyN7mXZm5GQ8tHM66RxNcLELreMJwMHzEw6e72KRWpOYNRlbZ5M4VmH6H+pU9n33H2Y8ccpg7HGspDxS2vr+7UnFMneBrVwl9m2lcu1HqWnYxi6UZVUVnuTWYUfWDdDFMRyNUKOx+2ds/lXVOafY//Go+3fvx669ux7xse2/qjrF3dVXYVvTt1Imo81Y9mP+7WcYtO1MyWrclbnv1rjqvP+7X0NQgRqr7r9S0XezUzoTmfhuV2OxdoJyuOTCBgcvbHTvzY8qmynIevPbaSuVX4sWfu1367n79/cAeR+pdCzTSfm86PpR5t/f9/x9RMrSH85xX/LfRm/oa2NTi8Y/0OfFbu0xfAfwrrquPwpcBN4CUNf124D3NE1zZ13X313X9cfp9Ix/0zTNU3Vd/yvgt+u6PkC3aN+zS/NfGKNbrmJ8+9W0py52wUEv7PaMClYUz6+1U8GB4y9kdPMRputnuifBijiafNv7v0AVBzgu2C08v9bNiuNNH/4M7XQ5paZg81CbiY691/Hvbvqn7uRyoU7kcUwdL2wijt7L8apbruLA7dfQnrrIhTseYnpmEpiJ3RyifmJT6qyQM7KOnFefybjPXGzfDzms5PqUptSh2IZ9JtdZziW5HMfJ/LfBbBzMIXZsieaY7dvIHnnmsytzp3vjjd+lAUb4X52W5RuJX6ojVTm2cGSOFDA2bM9IMINjwn7GKEZRzLYxyrWpKjj0bTcyvvkI7foZNu45wXiqfbkdV/tfuLYskOmwbjcOjMznYq1dI7Qvd2E0tDPQjbGyljXlj212k5HdGF8J2ag1Ms14bI6NbDTS7rgaayozeLWmncxu6Lc5adUY1Jq5ZmP/ah0E1AEzW8v4AIwq1AHTgTUrjkY+r+7aOJHZ7B1qbc07U1hGCWDtgO/Phmcxr2o0hrW1QMZoLanHaAxrB32bUdxmjDKyg6/4tpVZO+fufLu5CbTeBGySAmstZG3bhbQB76TStj6BcDsgm07TvoN6Om0zmfptLRNRNjVOJ5O0rZ603rll0iYy7WTaO6LYl4l2Zmqb+URPlDtVPVUudI1VAtuJojVm46mRaa1cuTUpt61yx9NW0Zr4hlNjh562iimhbGJ4RXsM1vJnZOZe1FkErQymyrbBlctj28/EHWvfFi+bEJZPlZfZ58mEztoX1KOz9sUyX691z86pubjWkgfwpw/ftfDa2S1T8vMGo0gp1MXRpKBgNiq45PgLGRumcOMe62iyOnjihiM8fv3luz2NgoKVw/qxK3j0WPHz2S3s68wncsNmwuoJHXqz7KDvRwcydWSN8a1X0Z66yNk7H0Sf7SKQDoWbkbKYdVsm5EzfuSR9ZxmwuO0wS9g3zmCIl2jOcg7zOJcMsX997F0uDE3S9xwZTeJ5z8qWskoI9wz6fYWuvMfpxJal+wVFW9cuLXds4ouOsPbiy41S+AijzM5t5dqKOfZc6iScjdbJPGRSEdm3dDqRfSTzcaE0uuNPf8vNoODN7/1b53gS1JcBrlW6pS7YZudew32FbsAoi0kY1Fru6/NtVLz/SOwBVNk20R7BeJJDOY77gmfHfQWvo63eZ7gzsMynmhLnSla6cp4WHV9k477YejMcUwagkM+qKYEzCvQ7pDinE5Fb2Ya9Ec4nPn9yRlaJUxEhbAJHFOjo+T5HFFNPac3HbnsZAP/1XZ90Qa8BEeha9YauAYJsKHItT5M7U4o4f7IMdo25XDLYNXTPMXuKWvSTy4Li2mjlHNbmcUzp5qNcH627L4Wf/FZgXyuGE/EdiC9bTmmK6y2iBCblSsPpi5z90CNMn7uAPjNZWHmD1FQs6/X1l8TkmyGb5WEszy+nLPWlx7PzSuMKZuadzWyyXCq7ZF5RnMJlzcZxm9x4Q6bkVUQuduGsbCe52IXdcapgypu27LN98BQX/mId/eCpICSNEnWHNoJXEGQ7icdRor2MU+g9lYWMuF6Y7USRTqSyu9ghcDqBUNeJHU+kTNbNZT4JYhfmvJKlt3E2E0lGQYvjE8ZlTp4+YIPYha6ejE8YnYR1OonHiee5arCmcFXBtHM+SjKcAIoxWmZBsW1VrCTKfHMGufiR+O95qG6E3sogFMRKifKpmbYSiuNUeCObcSZt+D2DvKeyFr+0rKYxQXxvtXvgyNiGqsVpuarSPrYhUvHTvR7K3flr592rhUI3b5q8nYIdsxXbY2R6vK5Mzm34WWLPv9U6+cG+zLwKthDjr70cRmZfwvqZYj4uKJgHFRx4qTcfTb98cuXMxwUFBQWrjn3NGEqaF3c8zA7m2Ky4TlJPZCQ5eMvVXHLbNaj7L+XsRx8N+umbh53LkKlYtp/H7CxlWbYxYglnmYvjNn0s4ZCptc/87GSDbec3G8cMXZxzOTfXQXN3j9k4mUufKXkFGcO+fMhD5WH2ksi8nBkjCKVQweHjN3LgxZdzflQx+cKzwTietUvnUolylZEtEmJiCMKaG8Qu9KEBdfAqYUNs2DZxPSVozTjbiRsUQgpTlCW5kkGYhVOzshJxDJNsJ4EsMh/nzMY5NnKQ6vR9KWk2jthDtYpsIXjGkAlgzMouA4rgo5TqTMuIe3Qr4xzKfsxxLs+yy6PsH+epWRkC5tCy04C25dZ+Oh4JRlGykcYELr5vLr/ypO3yM4OkwcJcyqZMO2bcO6JIk7KqzLzMmgliGwqTsjbCro6ZTyYbijQpS6bQTJFc/uRcTENxWv7YnpYK22P69WxkmDfZli/LTipC07bte6vYzn2tGFpPnpwCGB8vowTG5ZcYpXB66iLnPvUk1tM2pwS6tguYilPTdjzrftnQOPMqb/MEpp7X/Cz3CQ6VZU3DQvmax7ws+99Ks3FuDk5hzJzPKmKWAqgyBovc3sHuuKdcKIWT9TNM7n/O1c+pBqlZWM1U/uTeQvte7i1MZO7Vm41zkHELpYIYhvyzD0JTLtPbSV0sqUdyAVSvEqjyr90E8spfZs9foowFZuGMgigDXAfmzVEo6zNdy/6kkhjLVgl2ztIsrOO7N52SZ5Q6ZZTJYN+hasO6IHTGcaoQSmWR2KwM2X2Hk0y9ydQr55kA2NiYhnQKIRiTtNxvSHcawX5DALGvlkpsmzAvqgrvE3bPoQ10L03Kbt1V/r47FPRaB8Gj/X1gVpo8bxm361wFl8JOf16TtNxrKNPjxf3Ie5F/7s6PobiIs9sWbAkuueVqDhml8NSdD9Ke2djtKRUU7H1UcMQohRvrZzjzoYeL+bigoKBgF7GvGUOpsediBObKHGOmhstlP1IpfO7OB5mcuSjKRdsB1i4YOyMbMhX3yRZhCWeZi/vap7EGM+eQOJ/0MHlzsn87YTaWbYYcShJWc8DsvErIOZ349ypr2u1zOvFtQqlScLlQCk9/6GHGU514KMv+Kkg8jIN52/o6ZB4SU7P2JiQv06kMWc+/5hIKeAajYxFf/2dfYDRqA8eToH6Pw4kv77HESqcTCFjAXMo7M6n8q2zfyw5Ks3J04pU0B6u0be5kJJuYqasClnHxlHi7jpHwNLas31Q6l5jzm24k5mAlWEYtTcq2fWBdzpiS7fEshxRhNiZhBKPvon0/4KlMYF424wpHE9+3l+lK+zaBSVnzjz75GdqpWSOtYNSESbnPQ7mrpwMPZTDMoz0ld03UtjmbyHuVTI+3HZCZpLqx1UxHlSHsa8Vwp6DPTpievMhzdxWmsKBgbmhoz02cUrjfmMLLT553Aa4LCgrmx5VnzpbMJ7uIfa0YDsYxFMeLsINxWYtmcv8znPnys+hpyhx17Xzdob5n7SGcJwyNzWOc9mMZrvR8FmEJ58lYIvuMHUj8HDNM3wznkmX2E8bnt9n9hDnZEMsYj79qmCd2YZyPM7evMCC8RP2zH3sMRsrFKbR7idL2/Q+J/N7AYWZxXnT5Ve28bX/aMRg2ZmEyp0ozrRRtpRi12m2ih7C+YwKVOO5zOolPRoYMyTF5OUZR7jWMPjcV7EXs2VeYZRwjWR/VWYWyLqfyMHu4khgbh5N2SuJIoqVzyVq69zAKZwPRvkPJHMr9hpDsMYyhEPclxzxO/ffI7D/UELCIngnMhLAR30Gd7DsUtL3bgKcIYhtG+267wRVTpWi1YqS122cIBHsN+0LXdDJ7xrPhl5a9d4cOJNbJRTqhxPsJWy32GzqC1TuxdA4r5po5mXBycZfJW2Nk2Jp4PB202frny75WDHNBYjarBFpcess1jK44wMmPnKBttfe6IlWqFu07r5SlslnK5yLK4KAymXUMyfc5KxZhcl4znEuC+QwojrOUvaEg1bkx+8zGiSw65/3ifDLklVwF9fLHeU9moFJc8YYXsfH4WS587ulunMiWk1MmZR/S6QQIAlP3QTqduH4ixVH2Xc34zHwcw5xXcvf+ju+5FYXmez/wKaMriQeiqNfJdKpLBTrZkBIoZDnzcWxWtmVDSt4sr+TYOznoOxOfsE/pk/1JpxMnW0FlcZR5rEqT8kg4lzgTs1EmSS1OijE6VjCnk9CsDL2mZAdhSnXf7sBsbOIYtsopfNJZJfBUjr0vxlXoiGLKku+yjG1YEXgog9eT/+j13wgafugvP+5jG+JNyu0cSp93DPEKn4xp2PWjhbLlnVDkdGOvZK118qNqr6FSalOExAquut3Hpbdcw+HbjrF27FKqS/a1bl1QsHWoFC84fiOXvPhyDtx4eN4f9AUFBQUFO4hd0Wrquj4EvBs4BpwCfrhpmidF+T8Cfsa8VcDrgFcBh4APAPebsnc0TfOHfeMMhZaBfgeU7n2ewbv0lqs5fNsxJqcu8vQdDzA5u5G0mWeMISZQjrusM0nYz3KhZxJZ5rxixmxW+BkYZgll+axwNfOygMkc54hLONOU3NPHEHO5FdiptWPRp7v1OZ3EbRyrV1WdUnjzFVxcP81zH3qIsbgskljoczqB2exgX7YT6XTSycRGdMcmppvTZbgame0kBxeWRtkQNtqYiMPPP5ftJCz3rwlhVs3hdBKV9bODcZuc04h0SEnZSCVj7vQ5ndjXXAibHGTWlC0yL+/kuvGhZxAfoH1OjAnT30Um4tEafaxh1+cMh5QepjCBNVn3OJ94ZtGnx/Ofi6fRlDEfa5ENRW5xyMU27Atd07XR5sJphn45dmvM3lvCmIbdFOZPk5eGq9EzU+clPjVKWu1k3+5Ug/R4tnwzcQzdOYjwOVsVz3C36K63Avc1TfMLdV3/EPBzwE/YwqZp7gLuAqjr+qeBe5um+bu6rn8M+NWmaX5lNyZ9mWEKrVI4LY4mBTuP1Vs7lim8+QouGKVQprkrKNgBrN66KSjYJeyWYvg64G3m+E7g53OV6rq+Efhvgdca0e2duP4+ul9w/0vTNKf6BpF7DIcDWOfZQcn0HLjuUqcUfvWOB9gQIWmGwszE/dsxBoNeZ9mvtLwvMHU8/9mhboadSmbXm80SQsQExue/CeeSWDYUwFq2W2Y/YV8ftjwbCmdrnU92ZO3Mynbi6vVkO5G8xZFXXsUhoxQ++6GHUMJJy+3v05m+e+aRtBXHOf+NzaBCEBxYNiLcV9ib6QRfp8rUS7KdJGFCFAmFaeXkmcNsthMipxM7wdykh4JZ9zmIRE4lvXsDpdMJEGc72eZMJzuybgC3V1KNwIeeMahamBhCYRR4kHQv0hlFpbzPoENK1ZJuPIxgmEJlnFT0ZOK/E30hbJxjiGHbxiN/PhORXznOn5wLel35fuLQNa4fEeFZVbhsKBA6ofQFu+6mOl/+5L5ACNaKEGdBsZcp9qlZFt55xbCfSlryOsj7W1y2Xdh2xbCu6x8FfjISPw48Z45PAVeQx08Bv9Y0zQXz/hPA7zRN89d1Xf8s8G+B/7Vv7NAreUARyzy4Y0Xs3GNneObj65x98CTTMxuDpttlHEnkPPrKh0y6so/5FNEeh4xM34tkLBlS7mS9uZWyoTn0KWIDskW8jWV7OW4w1x6P56Hznxe7uXYkYq/jWEGMVYxYQTzz+a8yHlec/exXoSdOoazf53Riy6XTyeC87avWgRJp2yYyvPI3lO0EpMkqdTqpsuZh4VyS08kqErNaJx9QAi1yDinBhHJtvJIXKGSzvJJzKfFcnxkP40WynWTS6KklnE92fd1Y55K2wpqFnUI3nUReywY555QBZB1SgFDRNMe5H6Y2bmKVcbOSnspZRxTplew0scSUHMQ2lCblPg9lU+63V2i3bpw5WDih9MU07GRpmjx6fnzKLChm1NAcbJoPRc+pIEiPZy9JX3o8YFNxBmchFzd2EWy7Ytg0zTuBd0pZXdfvBY6Yt0eAZ+N2dV1XwD8GflaI39c0ja37PuDXt3zCEQ7ccBnn108DcOrzX93u4QoKHFZ67VSKg8cuZfLYWWg1pz/9ZKJg7ne88vMnskpiwfZipddNAQC3PfhgNotgwc5gt0zJ9wJvpvs19ibgI5k6rwK+0DTNOSH707qu/0XTNJ8Avh3466FBsgzdDPOsfw+X33qMK2+7lmc++SgnP/Okk+fGGOpPZ2SpuTM37zwruUz8wdy4y7KEsjxmCWfFIkzmtoBzySwTcSwbci6ZKZvDbByPkTMbb0OMqR1ZOxZ95mGVKXdHleKa4zdx6KbLefrur3Bh/UzURvafymR/85iGK0Knk7itEu1lZhPpdJKMJ5jD2OkkZhOl04l9VZXmZQ88TjXy1GSc5SR2PskQZgkCE7FkAmN2MA5BMyujyVBZhnEMsp3McjoBsiFs+jAjM8qS2Ll1M5Lm3DAMjWKMzmo8GacSyRxm2iQOKboiMCXrzLWzTie5zCjC+aTXEQWMLVU4ohhZNrZh4pCCoOY08Q3AZkN55WPrtBMVmJNde0jyJ8uYhvb02uiSdRmLfBt77pIphH4zbeCwYu8dIvyNPdV5HT86ttIzit0cpMzMS/mPI+fY0rWx56OEdHnslmL4DuBddV1/FLgIvAWgruu3Ae8xi7AGvhy1eyvwG3VdXwQeA/75dk3QKoWTUxc586Xkx2VBwW5hb6+dSnHs+E1cevMVnF8/zcXHz27LMAUFC2Jvr5uCgj0EtYp5XOfF//DiH3QnF59lXxYSjeaKW49x5W3XsXHqIo/d8SUmxvt4HnYwVzYPO2jHzs3H1R1wJpH9zbXvrqdvOW4+q0rKiG0lSyjnkrBxA/OZyfDNK1tgP2Eim9H36bMPrIwt9b940Zs1dIzgyPxMt79gR4InHKuKEZ4huPb4TRy++UrOr5/myXu+4jKajJT/LTs2RyMR5X+kvcz+Wg3LbVvceIEsYgxH2v/qHWntQuPYbfMjDWPz2TgZMHbHrStbM3TFyLAjI6WpKi+zae8sczgatYzGLR/9ppejFHzrJxqqUUtlJlGZyVZr2snUWFPZuMf2dU2hxubautcKdcAwajbTxLiCkTk+YBqPRzAauWO1ZuRCZo/VmmG1xmNf7uqNwZaPxo7FsiFZWFvzzNYoredex2uo8YHeemqU78eOc+DGf7Aya+f85//cLP6pZ/8sfaXbLkcydMyhKzcLZTrxdW2ZrGfa0rYwNQGn5RiunuzH9D2Z+GNZZvuxwbGnU79hbjp1Diba9TMVfdq2Yj42j/KkDY5tmTbJIPREo003XgZ6orjnFV+P1vAdn/sceqo69hBcmrx2ominlZmiP55MKnN6iqktN041bVsxNfcZK5tSsaH9McAEnz95imJiHVWUl22YY1cGTGy5LQva+B2hGxmZ3W84VRkZmqm5V02kTBxPDKMctulkH3j4gwuvnX0dnXkoJR7kTbVXOqXwAutGKVxECXTHczqS5OY3ZCqWfR1Js5MAABL3SURBVPb1l880Mm+9+coXiUU4KBPnOkuhnSeV3UJjb9JsvGjfq4h50+BRKa47/mIuu/kKzq2f4ul7HkRNdTaDShCn0Hrj9Ywfm59zDivBNISCKLOdqEx53LeMWehkPcOFXslG5kzK3funrjri6stwf70Ou5lBE6eTwANZyGKTq8h2ouRnNcsLOJfeTk5yyBwsy4b6CeYpsp1E5me1oplPnNKshWNH4FzSKb5quoHOOppkspzY6yAzpFgP45EwKbe2nnA+cV4RmWupte/HlFvVomsrvketMBUPxTYUJuXYvJxkQ5EeyuZVV5rHL78cTXfaWnolC5OyNQdLrxDphCJjGkLnhOIDA6RbZKQTyiJmZVvPHs8yJefauGe6Vi67i5UtE5OwAud5vQxWb9VtM8aXHWDj1EWnFBYUFMyGqhSjQ2POrZ/iyXu+EuQNLygoKChYHexrxnDR+IIAj9/7MKNLxkzPTxZmB2W9RdlB2c/8Juu880U8zjws4XA/87OE84arWdSRRI61W2bjoM2cfcfyVUGf0wkQcHbKvp+0PHb3A9BqqsxP3NxvV0mSSVnOSSWoE5EMSoeOJkNth9DNx7B+4jV2OsnFLZRyGZrGvu97jQm58LWHOewLWQP9TN5c7GCOWVQ9FKd0Thml5a6fjCPJkHdNX/tVwsgzfcowfFpmPhFZTpQ1Kw8xh9KpJMmtjA89Mxr7O9VEsIxB11EIm7bttgsIKMQ9K3JEcW3j2IaVCkPXQPhd7cuGIkPXmDHc1wNLJGv3IPTOJzimULKH0glFxjT0Pdpy20+a7SjOeiJ9ZYAgjmGOzVPi9H0bxbIOIe4eK+eCv3SK0Ammm9fmnjmFMQRecMsxrnjl1e799HxPcNCCggKPSnHdt97EoesPA6A32sIUFhQUFKw49jVjOE+A6xfcei1Hb7uOi6cu8Mzff5V2ku4q2OowM7l+ltlDKOWzgl7nnEb6WDFbngu9MitjyaJBqgfnOMcew+3aT9g774G+gzkOnOsqIbdHEKCqKm44/mKO3HwlowMjzj16OgpDk7JaUWSK7DgJgxixg5JRnD1334cMSdPJdCKbhVy2k6rKZz6pHHWBC1+TzK+CJMBv0An5ixUHvQ6YPsnkSToy2qPYs+8wG31b7kscyGgyO/NJmitZBfsS58ylvNchWc6pZfMkcyjYP7HfEPqYQxjcd2gZv8nEfT4de5fJjGJD2LhcyWPHOCIZPxfWRu4n9HsN06DXfh1Jfk63nlHs5iX2LMrvt9xraBe5xjCIKhvT0O8hDINdd6eiaVuV1JdZUOywOYZQ5k32Wxkz9zQRtiZDTCb9EvStxPM7vnq7i+eNYmghH+hHb72Wq4xS+OAd9zOZeGK4L95hrp95vYnzbfNy28ew8paO12sCnbN8M04lfW3mdRpZ1Llk5ry30Wws+xgyG+dM5KuAPqcTgGrklcKz66d4/M8fDMsz/WRT7GmV13uck8qwSUMqi4kCqcNsJ4n5OWhvFUTfRqbBq3pMx2B8PCKnE/vAuvapk17xkxbZXru6Ssszpri0rArlpixJg2fqdv3MUALjDCf2eB6nkozZeGYGE6FMqlkK5l5HTrkTCmKfWRl6HFLaNoqNiFHwjDnZeipXlTftyswoVhnMxTWcTEKF0IznHFEyZmPGUgkcjm04MxtKoi0pqDTXn3oWrNNJbrtFlCbPpdSUyWSi9Vgplc2CIjOVdFPx9wGpNHpzrnYZVnLZUHLm3or5nEgqJDHkfyy709uhZ8m+VgyHcPTWa7n6tuu5eOoCD9/xxeJoUlAwB1SleOHxmzly85WcWT/FY/c80J9w9HmO133y76lGJX1DQcGi+M4vfx5ddnTtGva1YpjLBgJw5GuudErhg3fcz8XTF13ZLAYvHSMtnxVmZlbfi5iK540/mPY3P+O31SxhMI8tdi4J5rvNZmNbf8hsvIpsoYTkGBRw3Wtv4HLDFK7f82XU1McVVJlf1wTt8+ZiKesz68r8yNLpZLuhVOp0IvMjB3WFWVgNmIjDelJuXzPsoEWv80nGgUSWDTmfxBlQ7Di5ScZtZ2U0CczUscl5lO97xeHMuT3MYa9ZGUKHFGcqXoOJIS5c2BqZC7nHQceZjTNtZAgb6YgSyXpdJmIWUTipyDzKysT77MuGkuQH12KBWyITv2bsZHL5kxH1lBbrNbKed+dl17QKsqAQHYVm5Q6S+VOZen3w1gxzKmrxUDTWmhKHuvEsoz2XWYG9hrGvFcNpj1Ly7APPcMm1l/HkZx5nwyiFg4pajzzXd9yXntX3gqZi2WZW/EHZNmcClW1n7R3MtZnnXDYjk2POu4dQ9rXdZuO4ftazekUVw769hU/d9wSjtTFPfvwR9FT37CXMmV9SVNE4Ug6hEjgEue8wbz4Wnod2TxCpN6KMYxinvZPoPB69stjnpfyFl1yPGmle+eB6kv7O9TVgkaUSymHOQ3nAVJwog0mbKm3Tl97Omp9lMMahk1Biv+AAVGCaziiY1Sgfa3Gvwwbtnk56lUPImJVtmb3eNuWdkOVjG2ZS57WtV1DFHkS/RcAG3K7IpckLIPcbAgvFNszF4ezzUAaj8Sg+c/ULYQrf8Pgj2H2G8vRkmryuW3P/FnsN7Rq1ew2DmKNir+FUh9/pCu3uDjnFTYRszMKdijj9PgUwvgdpoHUnsXvPjhVcdcvjwBWXAKBbzaN/9bBTCgsKCvqhKsWBIwcBmJzd4PF7Hy7ex3Pgsy+7kc9+7Yt2exoFBSuHT117E5+64abdnsbzFvuaMZTZR6659TqOvfp6HvqzL3Pqoef6PX5F+3nMgvOYiof6WcRUnPMcjuv1sYhD3sK95XN4JcfzG4yHuAPOJbKvZczGwblmzMZx/VnZUOI6qwZVKW58w0u47NrLeOCDf8/F5y4AISuYJbtyHny2T91flmMgg7ZBXfO6RZdX+n9Yh5RK6cAbOYd0k7thEeNTEdbZZOKVGqRXc8xhUDYU2zDL8vWwg7HTSVxn3liEiSPJDJPzfoJgS5VhB+czKxM6lRiZQsQnDOpFZmhp9hexCZVNO1dVqSNKq5M4hl25GTF2RAHmjW3YDR5ui0iyoUgPZYCK8EenPSX3dTP3XeH1EcYxHHZCsfcl+by1a91mCpHDK1lu5NI72VvAfd/9LKPdSqN76wVtbN9ivpsxDy+Cfb5CO1xz63Vce/sNbJy5yPmnz+32dAoKVgKqUtz0hq/hipuv5Pwz5wrDXlBQUPA8wK4yhnVdfz/wg03TvCVT9uPAf0+3W/YXm6b5k7qurwb+ADgErAM/0jTN2b7+p2iO3Xo9195+AxdOXeBLdzTBw21edjCXIaWPHYz76g8PE/c3P/u3TPzBXP3NOJXI/paJTxjX3yrnknnKt2o/4VB/fQzoVmG7146qFDe/4Wu48uYXcHr9JA/e/SWqljDvLnnmUGXK+/YS5n6ZSuJs3i3UnoTToq2VpXPrxjYsw5yM7lB+5GD+JnuDHSsIVUNEts1iBzNhaLJZTqL9hkl+ZBmmJmkzwOTNkx852Rs4zCL25UdWub2KW+ycst3rBhC5i4WsjzlM4hyCj1komEPH9Mm9m7lwNCKETRyGRlfeEUVmRbHDZeMYVgF72MnkfkK/19A5lYicybOyoQShaxBTM14vdo34ctu3yJCiw/2Grtw08Uy/YBltliIt1oqdIv55UJHGOQxZRMMSyuUmTtWOnM2Qgr8/2ufFvM4olZ8urTgzmQFlM5aqXWMM67p+O/BLuTnUdX0d8C+BbwG+G/iluq4PAv8b8AdN07we+BTdIu7FsVuv4zqjFN5/R8OF0xdp6S7k1JhFu82nLa1RBCa6ZaJbWjRT86/B/VtZa/6nzsDa9WP7ssda9NM3dq6NFmPYYNT237Wl9fOwbXXUToyljdKozViuXPv/3HnlxpZl8fu4fl7W0pr5xnMekum+ucwoD8bXOlEKZftcbEI9Z3+2XLbdauzE2rFK4an1kzx095cC+0pqIU194FSmXp8iKOv3lVfaK3hK49LgSQVwUVTuXzvHE/evcB7JQw4mNni1UwZ7dSJfL5TLf/+g9JNU4X+2bEBhy5mCh7yUbZ+5k7HmZxuMus/BxDqSyP894H28E+sGcNdOjcb+Oo3GMBp3yq85lnL/JRAyd+1GqNG460+2Db88YRv5GZgvs1JVZzYej8Nx5ffI1g8+O9PeBiSf9W8RfJ9sH/67rHLf63hBVyr4tNzXUnxFw/ju2m3lsGvYT8ev5dyalutfQt5n+u41StRbptzXU+K+tHsK2m6u1r8C3tpT9g+Be5umudA0zXPAF4F/ALwOuMvUuRP4jqEBLr/pSqcUFjNYwT7Ctq+dy44d5tT6Sb5y9xeLo0nBfsG2r5uCgv2AbTcl13X9o8BPRuIfaZrmD+u6Pt7T7HLgOfH+FHBFJLeyXvzyb7ytU/D//WJzLijYC9jNtfOLv/yLO7XPed/iX+32BJ6n2M11A7B2zdeWtbNJ/NRuT+B5jm1XDJumeSfwzgWbnQSOiPdHgGeF/JyQFRTsS5S1U1CwOMq6KSjYHHZ/40cenwBeX9f1JXVdXwF8HfBZ4F7gzabOm4CP7NL8Cgr2KsraKShYHGXdFBQY7CnFsK7rn6rr+nubpnkM+A90i/DDwM82TXMe+EXgh+q6vhf4ZuA3dm+2BQV7B2XtFBQsjrJuCgpSqFVN2VVQUFBQUFBQULC12FOMYUFBQUFBQUFBwe6hKIYFBQUFBQUFBQVAUQwLCgoKCgoKCgoMdjUl3nZh0bRHOzy3Q8C7gWN0cbF+uGmaJ6M67weuosuYfq5pmjftwLwq4P8CbgEuAD/WNM0XRfmuXrc55/gf6DIXnDKi7zPBanccdV1/I/DLTdMcj+TfQ5dNYQL8btM0v70L0+tFWTtLzausnS1EWTvbMreydrZvjnti7Wzlutl3jOGSaY92Em8F7jMpln4P+LlMnZcCr2ua5vhOLE6D/xK4pGmabwZ+BvgVW7BHrtvgHA1uA77bXLfju/hg+9fA7wCXRPI14NeA7wK+Dfjn5truCZS1szTK2tkilLWzbShrZxvmaLDra2er182+UwxZLu3RTmIwxVJd19cCVwIfqOv6o3Vd/+OdnlfTNB8HXiPK9sJ1g4E5ml91LwN+q67re+u6/me7MD+LLwH/VUb+dcAXm6Z5pmmai8BHgdfv6MyGUdbOJudV1s6mUdbO9qCsneWxCmtnS9fNypqStzjt0bagZ46PM5xi6QDdL5K3A0eBe+u6/kTTNE9s1zwN4mszret63DTNJFO2rddtAENzvAz4deBXgRHw53Vdf7Jpms/s9CSbpvmPdV3fnCnaE9exrJ0tR1k7W4SydjaPsna2HHt+7Wz1ullZxXCL0x5tC3JzrOv6vWIOufEfA37TfOmeqOv6U0ANbPcCja9NZeaQK9ut1FBDczwLvL1pmrMAdV1/mG5PyI4/3AawJ65jWTtbjrJ2th974jqWtbPlKGtne7HUNdyPpuQh9KU92knMSrH0HcAfAdR1fRh4FfB3Ozmvuq6/CbhPlO2F6wbDc3w58NG6rkdmX8XrgL/Z+SkO4u+Al9V1fbSu6wPAtwIf2+U5zYu98B0oa2d5lLWze9gL34GydpbHKq+dpdbNyjKGi6Cu65+is7O/33gQfYROKbZpj3YS7wDeVdf1R4GLwFvMHN8GvKdpmjvruv7uuq4/DrTAv2ma5qkdmNf7gO+s6/qvAAX8yB67bvPM8feBj9N51f1e0zSf24U5Jqjr+i3A4aZpfsvM90/pruPvNk1zYndnN4w99h0oa2f75ljWzhZjj30HytrZvjnuubWz2XVTUuIVFBQUFBQUFBQAzz9TckFBQUFBQUFBQQ+KYlhQUFBQUFBQUAAUxbCgoKCgoKCgoMCgKIYFBQUFBQUFBQVAUQwLCgoKCgoKCgoMimJYUFBQUFBQUFAAFMWwoKCgoKCgoKDAoCiGBQ51XR+r6/o5kxjcyu6s6/oHdnNeBQV7HWXtFBQsh7J29h6KYljgYBKmP0aXDom6rv8JoJumec+uTqygYI+jrJ2CguVQ1s7eQ8l8UhCgruvfAT4J/AFdzsfvBJ4Gfg34zqZpXrSL0yso2LMoa6egYDmUtbO38LzIlVywED4CvBH4erq8ig8Y+T+r6/pDuzetgoI9j7J2CgqWQ1k7ewhFMSyI8RHgV4F14PZdnktBwSqhrJ2CguVQ1s4eQtljWBDjQeAA8D83TXNxtydTULBCKGunoGA5lLWzh1AUw4IYPwH8YdM0fymFdV3/n8Ar6rr+zbquX7I7Uyso2NMoa6egYDmUtbOHUJxPCgCo6/oVwPvofrn9QNM0p3d5SgUFK4GydgoKlkNZO3sTRTEsKCgoKCgoKCgAiim5oKCgoKCgoKDAoCiGBQUFBQUFBQUFQFEMCwoKCgoKCgoKDIpiWFBQUFBQUFBQABTFsKCgoKCgoKCgwKAohgUFBQUFBQUFBUBRDAsKCgoKCgoKCgz+f/7RpPa7kFWyAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_softmax(-1, +1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that we assume **one-hot encoding** of the vector target values $\\mathbf{y}^{out}$, which is not very efficient (unless using sparse-optimized data structures) compared to a single integer target value $y^{train} = 0, 1, \\ldots, C-1$. However, sklearn has a [convenient utility](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html) to convert integers to one-hot encoded vectors (use `sparse=True` to return vectors in an efficient [scipy sparse array](https://docs.scipy.org/doc/scipy/reference/sparse.html))." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" } }, "nbformat": 4, "nbformat_minor": 2 }