{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Machine Learning and Statistics for Physicists" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Material for a [UC Irvine](https://uci.edu/) course offered by the [Department of Physics and Astronomy](https://www.physics.uci.edu/).\n", "\n", "Content is maintained on [github](github.com/dkirkby/MachineLearningStatistics) and distributed under a [BSD3 license](https://opensource.org/licenses/BSD-3-Clause).\n", "\n", "[Table of contents](Contents.ipynb)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns; sns.set()\n", "import numpy as np\n", "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from mls import nn_unit_draw2d, nn_graph_draw2d" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import torch.nn" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "import mls.torch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Neural Networks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From a user's perspective, a neural network (NN) is a class of models $X_\\text{out} = N(X_\\text{in}; \\Theta)$ that are:\n", " - **Generic:** they are not tailored to any particular application.\n", " - **Flexible:** they can accurately represent a wide range of non-linear $X_\\text{in}\\rightarrow X_\\text{out}$ mappings with a suitable choice of parameters $\\Theta$.\n", " - **Trainable:** a robust optimization algorithm (backpropagation) can learn parameters $\\Theta$ given enough training data $D = (X_\\text{in},Y_\\text{tgt})$.\n", " - **Modular:** it is straightforward to scale the model complexity (and number of parameters) to match the available training data.\n", " - **Efficient:** most of the internal computations are linear and amenable to parallel computation and hardware acceleration.\n", "\n", "The \"neural\" aspect of a NN is tenuous. Their design mimics some aspects of biological neurons, but also differs in fundamental ways.\n", "\n", "In this notebook, we will explore NNs from several different perspectives:\n", " - **Mathematical:** What equations describe a network?\n", " - **Visual:** What does the network graph look like? How is the input space mapped through the network?\n", " - **Data Flow:** What are the tensors that parameterize and flow (forwards and backwards) through a network?\n", " - **Statistical:** What are typical distributions of tensor values?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Mathematical Perspective" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Building Block\n", "\n", "The internal structure of a NN is naturally described by a computation graph that connects simple building blocks. The basic building-block unit is a function of $D$ input features $x_i$,\n", "$$\n", "f(\\mathbf{x}) = \\phi\\left(\\mathbf{x}\\cdot\\mathbf{w} + b\\right)\n", "$$\n", "with $D+1$ parameters consisting of $D$ **weights** $w_i$ and a single **bias** $b$. The corresponding [graph](http://alexlenail.me/NN-SVG/index.html) (with $D=8$) is:\n", "\n", "![NN unit](img/NeuralNetworks/nn_unit.png)\n", "\n", "where the left nodes correspond to the elements of the input $\\mathbf{x}$, the edges correspond to the elements of $\\mathbf{w}$ (thickness ~ strength, red/blue are pos/neg values), and the right node is the output value $f(\\mathbf{x})$. The recipe for obtaining the output value is then:\n", " - propagate each input value $x_i$ with a strength $w_i$,\n", " - sum the values $x_i w_i$,\n", " - apply the activation $\\phi$.\n", "\n", "Note that this building block is mostly linear, except for the **activation function** $\\phi(s)$. This is an application of the kernel trick covered in the [Nonlinear notebook](Nonlinear.ipynb), and allows us to implicitly work in a higher dimensional space where non-linear structure in data is easier to model.\n", "\n", "The building-block equation is straightfoward to implement as code:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def nn_unit(x, w, b, phi):\n", " return phi(np.dot(x, w) + b)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For example, with a 3D input $\\mathbf{x}$, the weight vector $\\mathbf{w}$ should also be 3D:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "-0.9640275800758169" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_unit(x=[0, 1, -1], w=[1, 2, 3], b=-1, phi=np.tanh)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Activation Functions\n", "\n", "The activation function $\\phi$ argument $s$ is always a scalar and, by convention, activation functions are always defined in a standard form, without any parameters (since $\\mathbf{w}$ and $b$ already provide enough learning flexibility).\n", "\n", "Some popular activations are defined below (using [lambda functions](https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions)). For the full list supported in PyTorch see [here](https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity)." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "relu = lambda s: np.maximum(0, s)\n", "elu = lambda s: np.maximum(0, s) + np.minimum(0, np.expm1(s)) # expm1(s) = exp(s) - 1\n", "softplus = lambda s: np.log(1 + np.exp(s))\n", "sigmoid = lambda s: 1 / (1 + np.exp(-s)) # also known as the \"logistic function\"\n", "tanh = lambda s: np.tanh(s)\n", "softsign = lambda s: s / (np.abs(s) + 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These activations divide naturally into two categories depending on their asymptotic behavior as $s\\rightarrow +\\infty$:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAAFcCAYAAADPrf8ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXxkZZ3o/885te/ZKnt3Op3eF+iFpoXx6oz+GFDk50/hNaIzIIvoVbnMZfyBiLgObsjFGfUCjuKo85NRhnFDRhZBLjtt0xu9d6c76exJVVJJ7ds5vz8qqeRk6ySdpZL+vl+vvCrPc06devrJ6XrqW8+m6LquI4QQQgghhBBi2tSFLoAQQgghhBBCLFYSUAkhhBBCCCHEDElAJYQQQgghhBAzJAGVEEIIIYQQQsyQBFRCCCGEEEIIMUMSUAkhhBBCCCHEDBVcQPX973+fK6+8kiuvvJL77rtvzPEjR47wwQ9+kMsvv5zPf/7zZDKZBSilEEKI85G0UUIIIUYrqIDq1Vdf5eWXX+bXv/41v/nNbzh06BDPPvus4Zw77riDL37xizz99NPous5jjz22QKUVQghxPpE2SgghxHgKKqDy+/3cddddWK1WLBYLDQ0NtLe354+3tbWRSCTYsmULAB/84Ad56qmnFqq4QgghziPSRgkhhBiPeaELMNLq1avzvzc1NfGHP/yBf//3f8/ndXd34/f782m/309XV9e8llEIIcT5SdooIYQQ4ymoHqohJ06c4KabbuLOO+9kxYoV+XxN01AUJZ/Wdd2QFkIIIeaatFFCCCFGKqgeKoA333yT2267jbvvvpsrr7zScKyyspKenp58OhAIUF5ePq3r9/VF0TR9Vsp6rkpL3QSDkYUuRkGTOprc+Vo/mqbxetebJNOJfF6Jo5iyXpWB0wfzeRabjWWX/DU2u2MhilnwCun+UVWF4mLXQhfjrOa6jQJppxYTqZ/JSf1MTupncoVUP2drowoqoOro6ODTn/403/nOd7jkkkvGHK+pqcFms/Hmm2+yfft2fvvb3/KOd7xjWq+haXrBNFRAQZWlUEkdTe58rJ8TfacJRnvzaUVRuKBkIy1H/4A+IsgqWr4ai9V+XtbRVEndTN18tFEg7dRiI/UzOamfyUn9TG6x1E9BBVSPPPIIyWSSb37zm/m8a6+9lueff57bbruNzZs3c//993PPPfcQiUTYuHEj119//QKWWAgx3+KZBI39pw15dd7lDJw6YwimFNXMym3biMa0+S6iWKKkjRJCCDEeRdf1xRH6zZJgMFIw0a7f76GnJ7zQxShoUkeTOx/rZ1/3W7RFOvJpq8nKpRU7aHz2t4aAyr18PW97z1+fd/UzHYV0/6iqQmmpe6GLURCknVo8pH4mJ/UzOamfyRVS/ZytjSrIRSmEEGI8fYmQIZgCWFPcQKDxpCGYQjVTs/7CeS6dEEIIIc5HElAJIRYFXdc5FDxqyPNYPVQ5yuk/fdiQ76pdhd3lnM/iCSGEEOI8JQGVEGJRaI20058cMORtLF1L18nj6Kn4cKZqpmb9BfNcOiGEEEKcrySgEkIUvLSW4VjfSUNepasCn8VDX+NBQ76rpgGHS+biCCGEEGJ+SEAlhCh4J/oaSWaS+bSqmFhfsoaOE8cMvVOKaqJa5k4JIYQQYh5JQCWEKGjhVISmgTOGvIaiOmyKldCo3ilHVQNOt/ROCSGEEGL+SEAlhChYuq5zOHiMkbs7OMx2Gnz1dJw8hpaKDZ+sqNRskN4pIYQQQswvCaiEEAWrK9ZNIB405K0vXQsaY+ZOOasbcHo881k8IYQQQggJqIQQhSmrZTnce9yQV+ooodJZTkfjMbRkZPiAosrcKSGEEEIsCAmohBAFqbG/iXh6xIITisLG0nVoujamd8pRtRKX1zvfRRRCCCGEkIBKCFF4YukYjaEmQ16ddzkeq5vOk8fREsO9U4rMnRJCCCHEApKASghRcA4Hj6Hp2XzaarKypmglWS1L70lj75S9sh6X1zffRRRCCCGEACSgEkIUmK5YD12xHkPeupI1WEwWOhtPoiXC+XxFUamW3ikhhBBCLCAJqIQQBSOrZTkcPGbIK7YXU+uuGuydestwzF5Rh9tXNJ9FFEIIIYQwkIBKCFEwTvU3EUuP3FsqtxCFoih0nW5Eiw+MOKRStWHrApRSCCGEEGKYBFRCiIIQS8c4OXohCk8tPpsHTdMIHj9gOGYrr8NTJL1TQgghhFhYElAJIRacruscDB4dsxDF2uJVAGN6p1AUqjdume9iCiGEEEKMIQGVEGLBdcV66IkFDHlDC1FomkbghLF3yu5fjqeoeD6LKIQQQggxLgmohBALKqNlOBw8asgrGVyIAqC76TRarH/4oKJQvUF6p4QQQghRGCSgEkIsqJOh08QziXxaURQ2la1HUZTB3qn9hvNtZcvwlJTOdzGFEEIIIcYlAZUQYsGEUxFO9TcZ8uq9dXisbgC6m0+TjYaGDyoK1RtlZT8hhBBCFA4JqIQQC0LXdQ4GjqDrej7PbrazunglQK53avTKfmXL8ErvlBBCCCEKiARUQogF0RpppzfRZ8jbWLoWs2oGoKelmWzUeLxqvcydEkIIIURhkYBKCDHvUtkUR3tPGPL8zjIqnOVArneq59g+w3FrWS2+srJ5K6MQQgghxFRIQCWEmHdHeo+TyqbyaZNqYlPpOhRFASDQcoZsxNg7VS29U0IIIYQoQBJQCSHmVTDeS2u43ZC3qmglTosTGOydOm5c2c9aWoPPXz5vZRRCCCGEmCoJqIQQ8yarZXkreMSQ57G6Wemry6eDbS1kwkHDOVUbZGU/IYQQQhQmCaiEEPOmsb+JaCpqyNtctgFVyb0VaZpG11Fj75SltJoi6Z0SQgghRIEqyIAqEonwvve9j9bW1jHHvv/97/NXf/VXvP/97+f9738/P//5zxeghEKI6QqnIjSGThvylntrKbYX5dPB9lay4YDhnKp10jslCou0UUIIIUYyL3QBRtu/fz/33HMPTU1N4x4/ePAgDzzwAFu3yocsIRYLXdd5K3AETdfyeTazjbXFq/NpTdPoHt07VVxFcUXFvJVTiLORNkoIIcRoBddD9dhjj/GlL32J8vLxh/gcPHiQH/zgB1x11VV89atfJZlMznMJhRDTdSbcSt+oPac2lKzFarLk070d7WQGegznVMrKfqLASBslhBBitILrofra17424bFoNMr69eu54447qKur46677uLBBx/k9ttvn/L1S0vds1HMWeP3exa6CAVP6mhyhV4/8XSClp4zOF22fF61p5wLlq/KL5MO0PjKIayW4bcku7+atZvXnPPrF3r9LDSpn+mZ6zYKpJ1abKR+Jif1Mzmpn8ktlvopuIBqMi6Xix/+8If59E033cTdd989rcYqGIygafpcFG/a/H4PPT3hhS5GQZM6mtxiqJ83u/bRHx1eiMKkmlleuoJAIJLPC7S3EulqMzyvom7DOf/bFkP9LKRCqh9VVQoukJiu2WijQNqpxUTqZ3JSP5OT+plcIdXP2dqoghvyN5n29nYef/zxfFrXdczmRRUTCnFe6Yh20RntNuStLV6Fw+ww5I1e2c9cVEFJVfWcl0+I2SRtlBBCnJ8WVUBlt9v59re/TUtLC7qu8/Of/5zLLrtsoYslhBhHKpvmUPCoIa/I5qPOW2vIC7a3kwl1GfIqZWU/sQhJGyWEEOenRRFQ3XLLLbz11luUlJTw1a9+lU9+8pNcccUV6LrOjTfeuNDFE0KM40jvMZKZ4Qn5qqJygX9jfs+pIV1H9xrSFl85pdXSOyUWD2mjhBDi/Kboul4YA7XniYxNX1ykjiZXqPXTHQvw5849hrzVxQ2sKW4w5PV2dtDy6h8MeTU7/5qyGmMv1kwVav0UikKqn6Uwh2q2SDu1eEj9TE7qZ3JSP5MrpPpZUnOohBCFL61lOBg4bMjzWN2sKqofc27nkX2GtNlbPmvBlBBCCCHEfJCASggxq471niCeSQxnKMq4Q/36ujpJ93UY8irWXTgfRRRCCCGEmDUSUAkhZk0g3kvzQIshb6W3jiKbb8y5Y3qnPGWUVNfMafmEEEIIIWabBFRCiFmR0TIcCBwy5DktzjHzpgD6urpI9bYb8srXXYiqyluSEEIIIRYX+fQihJgVR3tPEE/HhzMUhQvKNmJSTWPO7Ri1sp/JU0ZpzbK5LqIQQgghxKyTgEoIcc7GG+q3wrucUkfxmHNDPd2kg6N7py6Q3ikhhBBCLEryCUYIcU4mGuq3rnjVuOd3jJo7ZXKXUFazfM7KJ4QQQggxlySgEkKckyO9x8cM9bvQP/5Qv/5AgFSg1ZBXvnaL9E4JIYQQYtGSTzFCiBnrjvVwZsAYIK3wLqfEPnaoH0D7YeNmvyZ3MWXLpHdKCCGEEIuXBFRCiBlJZVMcGLWBr8vqmnCo30BwbO9U2RpZ2U8IIYQQi5t8khFCzMjB4FGSmWQ+rSgKW/ybxh3qB9B+eNTcKVcx5ctXzGURhRBCCCHmnARUQohpa4900hHpNOQ1+OrH3cAXYKA3SDJgXAWwbI2s7CeEEEKIxU8+zQghpiWeSXAwcMSQ57V5WV28csLntB/aB7qeT5ucPsrr6uesjEIIIYQQ80UCKiHElOm6zv6eg6S1dD5PVVS2+DehKuO/nYR7gyQDZwx5pTJ3SgghhBBLhHyiEUJM2emBZoLxXkPe2pJVeKzuCZ/TfmS/oXdKdfqoWDFxb5YQQgghxGIiAZUQYkr6k2GO9Z405JU5Sqn31k34nHAoRKK72fic1TJ3SgghhBBLh3yqEUKcVVbLsq/nLTRdy+dZTBYu8G9EUZQJn9d+aK+xd8rhpbxeeqeEEEIIsXRIQCWEOKsjvceJpCKGvE2l63GY7RM+J9IfIjmqd6p01eYJl1UXQgghhFiMJKASQkyqM9pN84BxyfNaTzXV7spJn9d2eB/6iB4t1eGlomH8TX+FEEIIIRYrCaiEEBOKZ+IcCBwy5DktTjaUrpv0edGBfpKdTYa8EumdEkIIIcQSJAGVEGJcmq6xr/sg6ezwEumKorC1fDMW1Tzpc9sOjeqdsruplN4pIYQQQixBElAJIcbVGDpNb6LPkLe2eDVFNt+kz4sODBDvPG3Ik94pIYQQQixVElAJIcYIxns5HjplyCtzlLLSN/ES6UPaDu+Dkb1TNjeVq9bMehmFEEIIIQqBBFRCCINEJsnenrcMy51bTVYu9G+adIl0gFg4TLzDGIgVr9okvVNCCCGEWLIkoBJC5Gm6xv6egyQzyeFMReFC/ybsZttZnz+6d0qxuahqWDsXRRVCCCGEKAgSUAkh8hpDpwnEg4a8Vb56yp1lZ31uLBIZ2zvVsBGTWXqnhBBCCLF0SUAlhAAgMM68qRJHCauLV07p+e2H96Nr2XxatTqpXrV+VssohBBCCFFoCjKgikQivO9976O1tXXMsSNHjvDBD36Qyy+/nM9//vNkMpkFKKEQS0s8k2Bv94Ex86a2+jehKmd/m4hHI8TaTxryiho2Se+UWJKkjRJCCDHS5JvJLID9+/dzzz330NTUNO7xO+64g3vvvZctW7Zw991389hjj/GRj3xkfgspxBKi6Rp7uw+QyqaGMxWFLf7N2M32KV2jbVTvlGJ1ULVa5k6JpUfaKCHEUqTrWu5L1fzj4O/o+XR+f8n84+Ax9OE0+mByVF4+Sx/8dWQeo87JSelOsn1RxhwYJzlOxsh/3MTHpspigdKJP9cUXED12GOP8aUvfYk777xzzLG2tjYSiQRbtmwB4IMf/CDf/e53pbES4hwcDh6jLxEy5K0uWonfWTql58ejEaJtjYa8ovpNmM2WWSujEIVC2ighxELQdR20DGRS6Nk0ZNK5x2watAx6NgPZTO53LQsjfnQtA5oGejb/qGtDwZM2HEQVCB3QNJ2Qw0o8mkTXdXQdtMFHnVHp/OPwMV3PhWxDvw/FecP5g4HdUEzI8PmMeO5QeWwuN/5VEwdUpi9/+ctfnrMamYF3v/vdVFdX89Of/pQPfOADeL3e/LHGxkb27dvHNddcA0A6neYXv/gF11133ZSvf/DgIYqLS8hms/zud4+jKAplZX7S6TRPPPGfmExmSkvLSCaT/P73v8JisVJSUko8HufJJ3+N3W6nqKiEWCzKk0/+BqfTic9XTDgc5g9/+C0ulxufr4j+/hBPPfU7PB4vXq+Pvr5enn76CXy+YjweD8FggN///rf4fCW43W4CgW6eeeZJysr8OJ0uuro6+eMf/wu/vwKn00lHRzvPPfcHKiqqcDgctLW18PzzT1NVVYPdbqelpZk//ekZamqWY7PZOHPmNH/607MsX74Ci8VKU1MjL7zwR+rqVmKxWGhsPMGLLz5Hff0qzGYzJ04c46WXnqehYQ0mk4ljxw7z8st/YvXqdaiqytGjh3j55RdYt24jAIcPH+C1115i7doNg/W6j127XmXNmtycmf3797BnzxusXr0OgL17d7N//25WDd6Me/bs4q239tHQkNufaPfu1zl06AANDasB2LXrFY4dO8KmTRuIxVK89tpLNDYeZ8WKBgBeeeX/0NTUSF1dbn7Pyy//iTNnmli+vB6AF198jtbWFpYvXwHACy88S0dHO8uW5fZRev75p+np6aKmZjkAzz77X/T1BamurgXgmWeepL8/RFVVDQBPPfU7wuEwVVXVAPzXf/2GWCxGZWUVAL///a9IpZKUl1cC8LvfPU42m8HvrwDgN795DICysvJZvfeqqyvo6gpO+9575pnfU1xcSj9h3mo6QM++FqxeOyabBWfCSuuek1O+9/a+9BxH2tooslkwqwqhjMKpUIi6uvoFv/dcLhuvvPLqjO69+vpVAEv63rNaVX7xi0fn9X1v6N4b/b7ncrlxOq0Uurluo0DaqcXUTh05cohly3L/95fye8VM7z2fz4vd7p7V94qldO9t2XIh0WiSfXveYN+eN1jp96GHe9iz6xUO7NtFnSlCtusku994iUP7drEs2Ua2u5E/793LsWOHWUYfWqiDNw4eobGpiWWWOHq0j9ePnqK5vZNau4aejPL6iVZaA33UukyQTfNqYyftoQi1XhvoOi+fDtAVSVDjc+TuzVM9BKIpqr259AuN3fTFU1QNpp8/2c1AIk2lJzeK5Y8nuhhIpClxWElndZ493sVAPI3LZCKezPDHk930x1KYNQjH0jzX2EVfJImW1ghFUjx3spvegSTJeJruUJwXTvXQG06SimfoCsV4uTlIfzRFOpklGE7yWksvsXiabEqjL5pkV1sfyWQWLaMRiqbY3REik9Egq9MfT7G3qx89q6FoufT+7gFUHVQd+hNp3goMYEbBRC59KBjGqiqYFYX+ZJojvRF8TgcNW7ZP+L5dcD1Uk9E0zbAPjq7rZ90XZzS324bf7yGbzWKxmPB6Hfj9HtLp9GDajt/vIZGwYLGY8Plyx2Mxk+H8SEQZPO7E7/dgtWpYLCaKinJpszljSCtKEovFRHFxLq3rcYB8OpuNDh534fd7SKUGsFhMlJS4BsvjzKfLyjxEo7l0aamb4mIP4fBQ2oXP56G/f/i42+2mt9eBxWKirMyN0+kkEBhO2+12urrsWCwm/H4PFouFjg5HPm0ymWhrGz6eq8exaavVPCJtG5MOhSz5tMtlIxwePu50WonHLSPSNjKDS3f7/R6cTiuKkjWcn04r+bTDYc2fC2C3W7BYLIa0w2E1pJ1O26j08HGbzYzLZRuRtuTvHQCr1TxO2p5PWyymMWmPxz7r9x5Aaalr2veexWLC7NQ4Hj+F3WHFZFKxO6yUFhWx0dHAC6dfmNK9ZzMrpEMdqKqCxWzCajZRtryeSFf3or/3Rh6Xe2/23vdGpke+75WWulnsZqONAmmnFtN7xcCAvFdMdu8N/Xtn871isd57FotKsS1FNh7GlgqiJgfo3/0UlmQUNdCFGo9j7dwPgCkZwpRNYM/2gwJmE5jNKk6XLf+3Mmu6IZ2FEWkziqbl0+bBv8XItFkdvp7ZrGKxmHA6rWg6KKoCikIGyGZ1MppOMq3RF02RyeokUlkGSHOmJ0o2qxFLZAhp0KrGAEimNSKJDMFw7nNcJquRTGWJJnNTAzQNsoM/ACi51zSZTCiqjqoqmE25+dtWszmXNpuwWsyomjZ4PJcmO5g2q1gtZnTVeFxTsobnZxXFcDwDw59hLGZS+mB68LhlMG0+y5xwRdcLqI9vhHe961387Gc/o7a2Np/X1tbGDTfcwLPPPgvA7t27+e53v8vPfvazKV83GIygaYXxT/b7PfT0hBe6GAVN6mhyM62fVDbFy+1vEE/H83mqYuLS6h34bN5Jnml08s3XiTYfzqcVi531V1yDxVIYPQ1y/0yukOpHVZVFFVTNVRsF0k4tJlI/kztf60fXsuixEHqkFy0WQo/3oyciY4bVOV02YtHkBFeZXVlNJ5PVcj+ZEb/n83UGO3UABV1RABWdXHCVK7ky4hi5Ywz+rgz/PnwMUMbJy19rmD7Ol082m4VkcuzCPjpT+aJq6l9m6VM41elycvk1V014fFH1UNXU1GCz2XjzzTfZvn07v/3tb3nHO96x0MUSYlHRdI03uw8YgimATWXrphVMJeIxYi3HDXm++g0FE0wJMd+kjRLi/KRrGnqsD62/Cz0cQIuFcvOXZptqQjFbwWxFMVlg8EcxmcnoKoksJDIKiTTEMzrxlE4irRNL66SyCrqi5n5Q0M0qmFV0RUEfWvR7Bj3qc0EZ7EXyeewk4ilUVcGkKphMKqqioKoM5qmoSu53Vcmdowz+PjJ/6FFRyD2queOKMvyYPzYqbyhtMk3eQ7UoAqpbbrmF2267jc2bN3P//fdzzz33EIlE2LhxI9dff/1CF0+IReVI73F6472GvDrvMpZ5aqZ1nbbDB3ITXQcpFjvVg+PFhTifSBslxPlHz6TQ+jvRQh1oAz25xSFmymRGsblQrA4Uix0sDhSrHcVsB4sNxWIDsxUNlWg8QzieJhpPE4mniURzvyfTkwRwKnO6UZKqKFjMam7ooEnN/W7K/VjMSv733E8uMDKbFMyqismUO27KB01KPrAppB5OVZ082CzYIX9zRYZSLC5SR5Obbv20hNs40HPIkFfiKGFn5bYp7Tc1JBmPc+yZx3MrDA0qWr2Nus1bpnyN+SD3z+QKqX4W25C/uSTt1OIh9TO5pVY/upZFC7Wj9bahDXTlJgNNg2J3o9g9gz9uymqq6I3ouR6nEb1Dmq4TiaUZiKUIR1MMxNKEYymiiQxz/bHdpCrYLCasVlPu0WzCZlGxWkxYLSoWswnb4Jwri0nFaskFQzOZL3o2hXT/nK2NWhQ9VEKIcxeM93EwcMSQ5zDb2VZ+wbSCKYC2I28ZginFYpPeKSGEEEuSFh9ACzSRDbZAJnX2J5Dbj1Fxl6K6ilGcRShOX26Y3ghmrwct3s9AJEUokiQUSdIfzQVR2Vn+UkVVFRxWM3abKf9ot5qxW02DP2ZsFhNm09wER0udBFRCnAei6Rhvdu9D04e/TTOpJrZXbMFmmt6cp2Q8TqTlmCHPW7cei1XmTgkhhFgadF1HDwfIdh5HG+g++xPMNlRfOaq3HNVThmJ1jnvNSDxNXzhJbzhJ5mSQ9s4BtFnodVJVBafNjMtuwWk3535sZpx2Cw5brrdJAqW5IwGVEEtcKpvmz117SY8a331B2cZpLUIxpO3oQWPvlNlGzbpN51xOIYQQYqHpuo7e30mm8zh6pHfScxW7G7W4BrWoKtcLNSpgyWoaoXCK4ECC3oFEbm+lEXOdXC7btIMpp82M22nB7bDidlhwO8y4HRbsNjOqBEwLRgIqIZYwTdfY232AaCpqyF9T3EC1u3La10slEkTOHDXkeerWSe+UEEKIRU+L9JJpfWvyQMpsw1S6DLV0GYrDZ5z7pOmEIkl6QnEC/bkgaqZD9+xWM16XBa/TitdlxeO04nFa8vszicIiAZUQS5Su6xwMHCEQDxryq91VrCpaOaNrju2dskrvlBBCiEVNT8XItB1GC7ZMeI7qKUP1r0AtqkZRh5fQjsTT9ITidPfFCfTHSWemt1AFgNNuochtpchtw+e24nNZsVvlI/pCy2pZMnqWjJYBRacUWZRCiPPOydApWsJthrxiexEXlG2Y0TjqVDJJeHTv1PJ1WG22cyqnEEIIsRB0XUfrOU2m9eCE+0apRVWYKlejukuB3DC+YF+Mrr44nb0xovHpLZduMasUe2wUe+ysqitBS2WwWSff40jMjKZrpLUM6WyatDb0kyGjZUbl5/KG8jN6hnQ2g6YP3xNuq5MVVe+Z8LUkoBJiCWoJt3O8r9GQ57A42F6xBZM6szfutqMH0UesbqSYLNI7JYQQYlHSk1EyTXvQwoFxj6tFVZiq16M6faQzWdq7I3T0xujqjZHJTr0XymEzU+q1U+qzU+q143Fa8l9q+v3uglkWvJDpuk5aS5PS0qSzaVLZVO53LU0qa3xMD52jZciO2CtzrklAJcQS0xML8lbAuNeUxWTh4opt017Rb0g6lSLcbOydci9fh9Vun3E5hRBCiPmm6zpasJlMy1uQHfuBW3H6MNduJu0ooS0Ypf1UB4FQYsqLR9isJvw+B2VFdsp8Dlx2s6yuN8pQgJTMpkhlU7lHbcTvg0GTNaTS2x8mpaWhwLfNlYBKiCUklOznze79ho3/VMXERRVbcFtdM75urncqmU8rJgu16zefU1mFEEKI+aRrWTJn9qMFmsceNFmgaj0dehmtzTGC/WemFESpqkKp1055sYPyIgdel/W8DaCyWpZkNkUymySZTZLIJklmhtLGx6lsUOw02Uhlp7bv11xQFAWzasaiWnDbJv8MJQGVEEtEJBXlz517jV3cisKF/o2U2ItnfN10KsVAs3FDYPeytdI7JYQQYtHQk1HSp/6MHu0z5Gu6TthURLNpJR2NOpoWnOAKw+xWMxUlDiqKnfiLHFjMS3/lvbSWIZFJEM8kSGQTJDJJEpkEiWySRCYXQC1k8DMuRcGqWjCr5vyjxWTBMhgk5YIl4+8jH03K8N5dqjp5kCwBlRBLQDyTYFfnm2PezNaXrJnR8ugjtR87jJ429k7VSO+UEEKIRUILB0g37oLBkRY6kEhmCNOomLYAACAASURBVMU1mtVl9JkrQJl8XpTHaaWy1ElViZNij21J9ULpuk5KSxPPxIml48QzCeKZoccEiUyCtDa9xTdmm1k1YzVZsaoWLCbL4KMV62C+RbXkfky54GkoSJqvv5MEVEIscqlsil2de4hnEob8hqJ6Vvrqzuna6XSK/qbDhjxX7WpsDsc5XVcIIYSYD1qog/SpP4OWJZPVGYim6I8miWg2urwbSZsnHsrldVmpKXNRVebC61zc+y1qukYsHSeaiRFLx4ll4sTSsVwQlYmTnWCVw7liUs3YTNb8Ty5Yyj3m0haq/MWEQymsJiuqUti9gFMOqAKBAC+//DLHjh1jYGAAr9fL2rVr+Yu/+Av8fv9cllEIMYFUNs2uzj1EUhFD/jJPDWuLV53z9duPHUFPDwdqimqmZsMF53xdIeaCtFNCiJGygWYyzXuJxlOEIikisTQ6OjFrGT1F69DUsR+DXQ4LtX43Nf7FF0Tpuk4imyCSihHNRImkokTTsVzwlInP/cIOijIYINmwm2zYzbZ8OvdjzaensuJwkcNDOrI4VkE8a0DV2NjIP//zP/PGG2+wceNGVq5cSVlZGdFolN/97nd84xvfYOfOndx2222sWnXuH+CEEFOT0TK83LyL/uSAIb/SVc6msvXn3M2dTqfoP21cLdC5bA12h/OcrivEbJN2SggxWrz1GKHjewlFkqQyw70vfc4VhJx1MKKNtFlM1PjdLCt3U+Qu/EUldF0nnokTTkUIp6NEUhEi6SiRdGzOlgpXFRW72Y7dbMdhsg3+bsNuGnq0LYqepLly1oDqrrvu4uabb+b+++/Hah0bqadSKZ577jk+//nP88tf/nJOCimEMMpqWf7ctY+EGjXklzpK2OLfPCtvaO3Hjb1TqGZq10nvlCg80k4JIYaEIkk6jhwg03pw1Cp9CgH3asKOagBURaGixMnyCjcVxc6zLjqwUNJahoFkmHA6TDgVYSAVJpyKznrgZFLNOM0OnGY7DosDh9mBw2zHYbZjN9mxmQo/0FxIZw2o/uM//mPS41arlfe85z285z0T7x4shJg9WS3Lm9376Y334nTZ8vnF9mIuOoeNe0fKZNL0nxo9d2oVdpf0TonCI+2UEOc3TdfpCERpbB8g3XWKsshxw3EdlW7vemI2Px6nlboKD7XlLuzWwlpKIJ1N058aIJQcYCAVpj85QCwdm7Xr2812nBYnTrMD1+Cj0+LAaXZgUS0SMJ2Dad1JN998M7feeitbt26lv7+fX//616xfv56dO3fOVfmEECNktSy7u/YRiBuXdfXZvOyo2IJ5nPHgM9Fx4ih6Oj6coZqpWS+9U6LwSTslxPkjncnS1BnmdPsAsWQGV6KL8sgJwzmaYqKnaDMlVbVsq/JQ6rUXROCg6Rq9sRBNA22EEv30JftnJXiyqBZcVicuiwu3xYXL4sz9mJ2z8oWrGN+0Pn0dOHCAjRs3AvDtb3+bI0eO8Oijj/Lxj3+ca665Zk4KKITImSiY8lg9XFy5HYvJMiuvk8mk6Ws8aMhz1TTgcLln5fpCzCVpp4RY+mKJNI3tAzR3hslkc8udO5MBysNHyS2KnmO2WHCsvZQNK1Zgsy5sMJHWMvQlQvQlQvQmQ4QS/didZmLR5NmfPA6LyYLH4sZjdeO2unBb3LgtTmympbWk+2IxrYDKZDJhtVpJp9M899xzPPnkk0QiET7xiU9IQyXEHMpoGXZ37SMY7zXk+2weNpZtwjpLwRRAx4lj6Knh3ilFNVG9/sJZu74Qc0naKSGWrv5IkhOt/bQHoob5UdZ0mPLwEYaCKZfdQpHXTtHGt2MurlqQsqa1DL2JPnrjfQQTvfSnwuOssjeFj+GKgtviwmv14LG68VrdeKwe7BI4FZRpBVRbtmzh0UcfRdM01q5dS0lJCSUlJXR1dc1V+YQ476Wzaf7ctY++hHF3d4/Vwzvr30a4b/Z2Js9msoRG9U45qhpwuqV3SiwO0k4JsbTouk6wP8GJ1n66+sYOiTNpKSoGDmJCw+u2UeyxYbOaMddfhGkegylN1+hPDhCIB+mJ9xJKhtCnu0y5ouCxuPHZvBTZvHitHrxWjwzVWwSmFVB9/vOf57Of/SwtLS3cf//9ADQ3N+NyTbwpmhBi5pLZFLs632QgadyHwWvzsLNyO3azjTCzF1C1nzyClhpusBTVRM3GLbN2fSHmmrRTQiwNuq7T2RvjRGs/vQOJcc9RdI3a6BEqPCo+jw/z4Ep95uUXYiqpnfMyprJpeuIBemIBeuJBUtnptccOsx2vq4hiuw+f1YfP5pm1udBifk3rr7Zs2TIeffRRQ97x48e58sorZ7VQQgiIZ+K80bGHaNq4NLrX5mVn5fZZHeYHud6pvkbjvlOOqpXSOyUWFWmnhFjchlbsO94Soj86cYDicVhYb2qm2KGjKvZ8vqmiAZO/fs7KF8/E6Yz20BXrpjfRN61eKLfVTYm9iBJ7McW2IpZX+QkEInNWVjF/zjkMvuyyy7jssstmoyxCiEHhVIQ/d+4hnjF+K1dsL2JHxdZZW4BipI7GY+jJEcGbolKzQXqnxOIn7ZQQhU/TdVq7I5xo7SccmziQKvHaWV3rw5/uINvaY9igV/VVYKrdNOtli2cSdEa7aI92Ekr0T/l5LouLMkcJpY4SSuzF2EzGffJkDtTScdaA6mc/+xnXXnvtuJslDkmlUvziF7/g+uuvn9XCCXE+6k30sbtrH+ls2pBf5ihle8WFczIcIJvJ0ndy9NyplTg9nll/LSFmm7RTQixemqbT0h3heGuIaDw94XkVJU7W1BZR6rOjRftINxtHVCh2D+b6i1BmYWN7yM1f7oh10RbuoHfUHOaJWFQLZc5S/I5SyhwlOMyOWSmLKHxn/WQWCAS47LLLeOc738mOHTuor6/H5XIRjUZpampi165dvPjii7z//e+fj/IKsaR1RrvY230QTc8a8itdFWzxb5qziamdp46jJUcMO5DeKbGISDslxOIzFEgdawkRS4wfSCmKQnWZizW1Pnzu3Eb2ejZN5vRu0LThE80WLKvehmKe+EuVqdB1nWCilzPhNrqiPWPa4vF4rG7KnX7KnWUU2XyosxTQicXlrAHVP/zDP3DDDTfw61//mscff5zjx48TDofx+XysWbOGd77zndx+++0UFxfPR3mFWJJ0XadpoIXDvcfGLKu63FvLxtJ1c/YmndWy9J58y5DnqKzH5fXOyesJMduknRJi8ZhKIKUqCrXlblbX+vA4h4MkXdfJnNmPnjDOOzLXbUWxz3y+bzKboiXcRku4bUqb6xbbi6h0lVPu9OO2yII3YopzqEpKSrj55pu5+eab6ezs5NSpU5SWlrJ27dq5Lp8QS56maxwOHqN5oGXMsTXFDawqWjmn46w7G0+ijWicFEWVlf3EoiPtlBCFbaqB1PIKD6uX+XDZx84V1oJn0ILGttLkr8dUXDOjMvUnwzQNNNMe6UTTtUnPLbL5qHJXUuUql6F8YoxpTcb4yU9+wn333UdVVRX9/f14vV7uuOMO3vOe98xV+YRY0lLZNHu7DxCIBw35iqKwuWwDyzwzaySmarzeKVvlClxe35y+rhBzRdopIQpLfo5US4joRIGUmguk1tT6cI4TSAHoySiZlgOGPMXhxbRseotQ6LpOdzzAqf5meuO9k57rsDiodVdR7a6SnigxqSkFVA899BDbt2/nRz/6Eb/85S/ZvHkzuq6za9cu7r33XjKZDFddddWsFOiJJ57goYceIpPJ8NGPfpS//du/NRz//ve/z3/+53/iHRyO9Dd/8zdjzhFiMYikouzu2jdmWXSTamZb+WbKnf45L0NX40m0+EA+rcjcKbFISTslRGHRdJ1Tbf28fqBtwsUmVFWhrsLD6toinPaJP5Lquk6meS9kMyOebMK8cgfKFBdq0nWdzlg3J0OnxuztaCiTolLpqmC5p4YSe7GsxCemZEp3YVdXF/feey+BQIA777yTDRs2sH79ejZs2MA3vvEN7rjjjllpqLq6uvjOd77Dr371K6xWK9deey07d+5k1apV+XMOHjzIAw88wNatW8/59YRYKJ3RLvb3HCKjZQz5DouDHRVb8Vjnfu+nrJYlOLp3qrwOt69ozl97tEwmTTQ6QDIZR9POPgl4qrq7VTRt8mEc57P5qB+TyYLb7cPhmNtvd6WdEqIwDC1/frwlhK6q4wZTUw2k8tcMNKEN9BjyzMsuQHWcfa7vUCB1vK+RSGriPZ+cFid13lpq3dVYTWMXt4jHo0Qi/WSzE69EOF3SRk1uPupHVU3YbA5cLi9m88y3pJlSQPXlL38ZgHe9613ceeedtLe3c+zYMZ5++mlOnDhBKpXilltuob6+nhUrVvCRj3xkRoV59dVXedvb3kZRUe4D3eWXX85TTz3Frbfemj/n4MGD/OAHP6CtrY0dO3bw2c9+FpvNNqPXE2K+abrG8b5GGkOnxxwrthezveLCMftUzJXu06cMvVMoCtULMHcqk0nT29uF0+mhpKQSk8k0a98Ims0qmYw0VhOZ6/rRdZ10OkkoFMBstmCxzN29Le2UEAtL03Xae6Icawnl95FyuYz3/XQDKRgc6tdq3NZD9ZajltWd9bmBeC/Hek8QSk68d1SJo4R673LKnWUTLv6UTqcIh/soKirDYrFJGzVP5qONymazJBJRenu7KCmpmHFQNa05VB/+8If58Y9/zLe+9a388IWWlhauvvpqPvCBD3D69Gn27t0744aqu7sbv394mFN5eTkHDgyPl41Go6xfv5477riDuro67rrrLh588EFuv/32Kb9Gaencf/M/HX6/7PNzNkuljhLpBG+07qMrHcA5qpFZUVTL9urNM1oWfSb1o2kaJ/50GKtl+C3AXbuSlauXT/ta56q9vR2324vXOzc9Y2azLGE7mbmuH4vFiaYVkU5Hqa4undPXAmmn5sJSeQ+eK+d7/WiaTktXmIONAfqjKVAUQyDlcuUCkIYaHxtWluJ2TP0Dq67rRA6+idluAnLto2Iy493+3zDZJ+71HkhG2NdxiM5ID5jBaTa2uQpQ461ivX8VxY6zzxlubj6D11uE0+mcctmnStqoyc19G2XCbreiqgq6nsDvL5nRdaYVUH3sYx8jk8lw5ZVXUl9fj8/n49ChQ1x99dW8973vnVEBRtI0zRD167puSLtcLn74wx/m0zfddBN33333tBqqYDCCpulnP3Ee+P0eenomHscrlk4dBeJB9vUcJJlJGvIVRWFD6VrqrMvoDZ59qdbRZlo/HadOEusbsRCGolBUt35B6rqvr5+Skso5+RZKvv2b3HzVj8ViJxgMTXp/qaoyK4GEtFOza6m8B8+V87l+dF2nPRDl6JnhHqnR3G47pW4La5cV4bRbiEcSxCOJKb9GNtBMpvWMIc+8fB29YQ3CY+s9rWU4GTpFU/+Z8VftUxRqXJU0FNXjsbrJRKAncva/XyQSpbR09tspaaMmN5/1Y7U66e3txGIZ/wuSs7VR0wqoFEXhk5/8JNdffz2vv/46gUCAT33qU+zYsWN6pZ5AZWUlu3fvzqd7enooLy/Pp9vb23n11Ve55pprgNx/ZrN5Wv8EIeaVpmuc6DvFyf7TY/aXspltbCu/gBL7/O6No2kaweP7jWUpW46nZO57D8YvTxaTaW42LBaFQVVNszo3bjLSTgkxt7TBQOp4S4iB6PiBlKooLCt3c+nWWuLR5LjnnI2eSY4d6ucpQ/XXj3t+R7SLQ8GjY764HFLpKmdN8aoZzVHWtCzqDEaQiMXDZDq3dmpG7/Iul4t3v/vdM37RiVx66aV873vfo7e3F4fDwTPPPMM//uM/5o/b7Xa+/e1vs3PnTmpra/n5z3/OZZddNuvlEGI2RNMx9vW8RSgxdux2ib2YreWbsZvt816u7qbTZGMjyrRAc6dGklWUlraF+PtKOyXE7BoKpI5N0iOlKAq1fjdrlxfhdlhwO60zDqiybYchM+J1VBPmFdvGvJ8ksykOBo7QGe0a9zoljhLWFa+i2H5uw8qlnVrazvXvW1Bfm1VUVHD77bdz/fXXk06nueaaa7jgggu45ZZbuO2229i8eTNf/epX+eQnP0k6nWbbtm3ceOONC11sIQx0Xacl3Mbh3mNkR3/boSisLqpnVdHKCSe/ziVN0wicGN07tQzvAvVOCbHYSDslzjeartPWk+uRmmogdc6vGeklG2g25Jmq16HYjPOm2iOdHAoeJZUdWy672c76kjVUuSokGBJzTtF1vTAGas8TGZu+uCy2OopnEhwMHKE71jPmmM1sY4t/E2WO2Qtepls/nU2n6NrzgiGv/p3/N97Sslkr03R1djZTWXn21ZpmQsanT24+6+dsf+fZmkO1FEg7tXgs5frJb8jbGppwH6lcIOVizbIiPM6xq3jOpH50XSN99EX0aN/w69g9WDa8C0XNfRGZ0TIcDB6lLdw+5vmqolLvq2NVUT3mKe5RdTZz1U5JGzW5+a6fyf7OZ2ujpv0VeVfX2C7VkydPTvcyQiwpuq5zZqCVF1tfHTeYqnD6eXv122Y1mJouTdMIjJo7ZS2rXdBgSkzukUd+wNvffhHhcSZfi4lJOyXEzGWyGqfaB/jj7hb2nugZN5hSFIVl5R7evb2W7WvLxw2mZkoLNBuCKQDz8gvywdRAKszL7W+MG0z5bF7eXvM21pWsnrVgSsCePbt5+9sv4sUXX1joohhcc81VfO5zn5n0nPkq+5TvtlAoBMAtt9zCv/3bvzHUsZXJZLj11lt56qmn5qaEQhS4aDrGW4HDBOO9Y46ZVDMbStewzF2z4EMOelqayUaMjVT1hm0LVBohZp+0U0LMXDqT5XRHmFPtAyRSmXHPURWF2nI3a5bNztC+0fRMkkzbYeNrltSgenMLv5wJt3IocAxNNw6nVxWV1cUNrPTVLchw+qVu7dp1PPzwv7J8+dyMJpmpr3/929jtjoUuBjCNgOozn/kMr7zyCgA7d+4cvoDZzOWXXz77JROiwGW1LI39p2kMNY95cwcothdxoX8TLsvs71sxXZqm0XNsbO+Ur0x6p8TSIe2UENOXSGVobB+gqWOA9ATDq1RFYVlFLpBy2Wc/kBqSbT9qXIjCZMZcuxlN1zjae4LT/c1jnuO2utlWfsGMVu8TU+Nyudm0afNCF2OMNWvWLXQR8qYcUD3yyCMAfO5zn+Mb3/jGnBVIiMWgK9bD4eAxYumxe0eZVDNri1dR560tmG/KAm1nyEaMPWjV6xd2Zb/J9ITiHGgMTjgBeqpUVZm1uSgep5ULGkrxF03/27BrrrmKSy55O11dHezdu4f16zfwwAPf59/+7V956qkn6e7uoqzMz3vfexXXXXfjhMtsf+1rX+all17gqadeGHP91avX8I1v/K+Z/NOWDGmnhJi6gViKxrZ+WronnrNnUhXqKj2sqvHhnMNACkBLhMn2NBlfv2odGbOFfV37xx1OX+upYWPp2gUZ3jcb7VShtFEAJ0+e4OGHv8fhw4dIJhPU1a3gmmuu5b3vvYo9e3Zz223/na9//X7e8Y6/BODEieM89ND3OHjwABaLmcsuuwKn08Uzz/yBxx9/Asi1Te9+919jsVj4/e9/Szg8wIYNm7jrri/Q0nKGhx/+PmfONFNTU8OnPvX3XHLJX+TL097exiOPPMzevXsIhUKsWFHPhz/8d1x22RX5c0a3fZlMhh//+F946qknCYVCXHDBhVxxxZUzrNHpmfYdKI2UOJ+FUxGO9B6nJxYY93iZo5TNZRtwWgqjCxoGe6eOGnunLKXV+PzlEzxj4e0/GSAywSTohRKOpdh/MsD/ddGyGT3/d7/7FZdf/l6++c3/RSqV5Itf/Byvv/4K1113I5s2bebw4UP85Cc/orX1DF/4wj+e/YJiQtJOCTE+XdcJ9ic42d5P5ySbyZtNKvVVXhpqvNit8xOsZFsPwYjNeBWbi1RJNbvb/0w4ZZxHalJNbC7bQI27al7KNp5Ca6fOpY2KRiPcfvunWbGinnvu+QpWq5Unn/wdX//6VyguLsFmsxnO7+ho59Zbb6GqqobPfe4LKIrCT3/6CM3NzZSUlBjO/c1vHmf9+o3cddcXCAYDPPDAt/jMZ/4Huq5z44234HA4efjh7/GlL93N448/gdfr5cyZJj7+8RspKSnhE5/4NF6vl2eeeYqvfOUeenp6+MhHrhv33/H1r3+FP/3pj3z0ozezYcMmXn/9Fb71rXunXR8zMe3/JVddddW4+U888cQ5F0aIQpXMpjjR18iZcCvjLYxpM9tYX7KGalflgs+VGi3Y1kImHDTkVa3bukClOX85nS7uuONuLBYLe/bs5sUX/8Ttt9/J1Vf/DQA7dryN4uIS7rvva1x99YfYsGHTApd48ZJ2SggjTdNp7YnQ2D5Af2TifaFsFhMrq73UV3mxWuZvI1ttoAct1GHIS1Y28OeuPcTTcWMZzTZ2VGzFZ/POW/mWuqamJvr6ern11v/JpZe+HYCtW7dTUlI6JpgC+OlPc6MB/vmfH8Tny+3vtX37xXzoQ//PmHNV1cQ3v/kAdntu38033niN559/ln/6pwe56KKLAchms3zxi3fR2HiCrVu388gj/0I2m+HBB/+FoqLcYl6XXPJ2EokEjzzyMO973/vxeo1//+bmJp555g/cfPMnuOGGjwFw8cVvIx5P8MQTv56NaprUtAOqL3zhC/nf0+k0Tz75JMuWzewbWyEKXVrLcLq/mdP9zWS0cSbpKgorvMtYU9SAxTS3wyFmQtM0ukf1TllLqimuqFigEk3NhavKZmXI32waGk4xU8uX12Gx5O6RXbteB+Ad7/hLMpnh++q//bd3ct99X+P111+VgOocSDslRE48maG5M0xTZ3jChSYAnHYLDTVe6io8mE3zO1Rd13UyrQcNeVGHmz2xVpJZY/DntXm5qGILDrN9Pos4rkJrp86ljVq5soGSklK+9a2v8cYbr7Fjx0527ryE//E/bgdyK+WNtGvX61x00cX5YArA4/Fw6aVvZ9++PYZzV69ekw+mAEoHVxbeuHF4TlZRUe46Qyva7tmzm+3bd1BW5jcsm37FFVfy0ksvcOjQAS655O2G19m7900A3vnOvzLkX3bZ5YUZUF188cWG9KWXXsq1117LJz/5yVkrlBALLatlaQ630BhqGnfDQIASezEbStcW9Ldkve1tZMLG4YmVBTx3aoi/yMG7t9ee83UKaY+PkcMg+vtzq9F94APvHffcnp6xcwXE1Ek7Jc5nuq7TO5DkdMcA7YEo2iTbjRZ5bKyu8VFV5kJdoNEVWvAMeiyUT4e1FHusGdJZY7krXOVs8W8qmOXQZ6OdKpQ2yuFw8NBDj/DTnz7Cq6++zDPP/AFVVdm+fQf/8A+fHXN+b2+Q4uKSMfml42zD4nK5xuQNveZEBgb6x73WUF44HBn3OQBFRcVnLdNcOOe7sq+vj+7u7tkoixALLq1lODPQyqn+iQMpp8XJ+pLVVDjLC25432hdo+dOFVdRXFG5QKURQ9xuD6qq8tBDP8Y0zrfBI7/1G0lRFLLZsY1vLDbxXAgh7ZQ4P6QzWVq6ozR1DjAQnbjXRFEUKkucNFR7KfXZF7Qd07UM2fYj+XRES/NnSwrNZBxuWOOu4gL/xoJZ6Gkpqqmp5e67v4Su65w61cgrr7zIz372Y77+9S9zyy2fMpzr9foIBoNjrtHXN3b7mJnIXX/sXPVgMPdl41CP1khDgVQwGKSkZLinLhTqG3PuXDinOVS6rtPR0cGHPvShWS2UEPMtlU3RPNDC6YEzpLPjTzK1mqw0FNVT56nFpM7f2PKZCrS1khkwfoisXHfhApVGjLRt23b+/d//jUgkzM6dl+Tzjx07yoMPfpcPf/jvqKqqHvM8p9NFPB5jYGAgP3781KmT+W/mRI60U+J8oes6wYEEZ7oitAWi437hMsRsUlle4WFltXdO9pCaCa37NHoqN0cqoWfYnQmQLa5jZIi33FvLptL1Bf8F5mK2a9frfPWrX+CBB77HmjXraGhYRUPDKo4cOcTRo0fGnL9jx05efvn/MDDQj9frAyAej/Paa6+MO+dqurZt286rr75MINCTn0MF8NRT/4XVamXjxrFD4i+66GIUReGPf3ya1avX5PPnazPiGc+h6u/vp7i4mOLiYhoaGma9YELMh0g6SlP/GVoj7WS1sXtJAZhVM/W+Oup9dVgKZKjBVHQf3WdIW4oqKBnnQ7qYf297219w0UUX85Wv3MN1193I2rXraG9v5Uc/+gGKorB27fpxn/eOd/wljz/+C+6990v8zd98mGAwwI9//C/4fL55/hcUNmmnxFIXT2Zo6Y5wpit81pXmPE4r9VVelpW7sZgLp4dHz6bJdJ4AIKVneTMdIOX0oo6Yj7zCt5wNJWslmJpjGzZswmw285Wv3MMNN3yM0tIyjhw5xGuvvcI111w75vwbb7yFl176P/zP//kprr/+JlTVxKOP/oz+/hAVFee+8uJNN32C1157lU996uN89KM34/X6+OMfn+all17gE5/4NC7X2D3HqqtruPrqD/GLX/x/KIrC9u0XsWfPmzzxxG/OuTxTMe1Ph6Wlpdx5550cOXIERVHYunUr9913H9XV8kFNLA66rtMTD9A80Ep3PAATjC83qWbqvLU0+FZgNVnnuZTnJtjeTrrf2DtVISv7FQxFUfjWt77DT37yI371q/+gp6eLoqJiLrroYj72sf9OcXHxuM/btu0ibr/9Th577FHuuOPvqa1dxsc//mmefvrJef4XFDZpp8RSlMlqdARjtHSH6Qklxl1xdoiqKFSWOllR6cFf5CjIgCTbdRIySbK6xr50kDBZTO7h3ojl3loJpuaJ2+3mn/7pQX74wwf53ve+Qzg8QEVFJTfc8DH+7u9u4MAB4xe0tbXL+O53H+Z//+9/4h//8YvY7XauuOJ9lJX5aWkZu/nydNXVreAHP/hXfvSjCgxSlwAAIABJREFUh/jOd+4jnU6zcmUDX/zivfz1X18x4fP+/u8/g9/v5ze/+RWPPfYoa9as4+67v8g994ydBzbbFH2y/5Hj+NCHPsQHPvABrr76anRd55e//CXPP/88//qv/zpXZZxVweDEG9jNN7/fQ09P+Ownnsdms46S2RRtkXaaB1rH3ZB3iEW1sMK3jBXeOqwFuHLfSBPVz+HnnyQd6sqnLb5yNrz7ffNZtCnr7GymsrJuTq5dKBN+C9V81s/Z/s6qqlBaOvZbx5mQdmr2SDs1ubmuH03T6eqL0dYTpaM3NumQPgCHzcyKSg/LKzw4bAs/omKi+tEzSVJvPYOeSbM/00tnNobqKUP15BYQqHCVs638goKZMzVX7dRibaN279412Au0w5B/001/S3V1Dffee9+svM58189kf+eztVHT/t8Wj8e59trh7r/rrruOxx57bLqXEWJeaLpGTzxIa7iNrljPpN/o2c126n3LWeapXVRD+0br7Wg3BFMA5YtgZT8hZou0U2Ix0zSdnlCctkCUjmCU9Fk+UKpqbpGJusHeqIVarW86sh3HIZuhKRuhMxsD1YTizq0aV2wvZqt/c8EEU2KsEyeO8/DD3+Ommz7O5s0XkkqleOaZP9DYeJK///v/d6GLtyCm/alx5cqV7Nmzh23btgFw/PhxamvPfXljIWaLruuEkv20RzvpiHaRzEy8iSHk9rVY6aujylWxJN7AO0et7Gf2llNWLf9HxflD2imx2KQzGt2hOJ3BKJ29sbMGUQBFbhvLyt3U+t3YrIW/UNIQPRUj23OaXi3J8WxuuXTVU4qi/P/s3XecXGW9+PHPadNn++wm2fRGIIUAIaGJXAsSBAXBSwSk/AQUE0CM/i5REJCL3OsPUMQrIFIuqCCKUpOAglFIwBBCICGk97K979RTfn/MbJlsstkyuzu7+33zGmae55w588yT2fnO95znPEcl6Aowp2T2oJj4aThbsOAybNti2bJXeeqpxzEMg2nTjuOBBx7i+OOH5+kF3U6oDhw4wNe//nWOOeYYdF1n48aNhEKh1lmV5Er0YiA4jkN9vIGy5goONpd3OqQPQFVURgZGMC44mjx37pAZo11bXkaiNv1q8yUys58YZiROicEgHE1QXhuhrCZMVV0EqwvDPL1undKQnzHFQXL9g+vc3hbWwc3ErDgfJqpxHFA0HcWXj6EazCmZnfVD7UXyPODLLruSyy67cqCbkjW6nVB973vD81CeyD62Y1MTraM8XEl5uIJIInLU5/hdfsYEShkdHIV7kE000RVln6SfOKoHiygYVTpArRFiYEicEtnItGxqGqJU1EUor4nQGD7ytaLacxsaI4v8jA75KcwZ2OtG9ZYTayZRuZsPE9XEnOTMukqwCEVROL54Bj7DN8AtFKJnup1QjRs3jpKSkrS6bdu2MXny5Iw1SogjiZhRqiI1VEaqqApXk7A7ny4WktOej/SXMCZYOqSORh2qtryceM2BtLriY2ejqoN/GKMQ3SFxSmQD23Goa4xRVR+lqj5CdX20S0ehIJVEFfoZFfJTlOsZFOdFdYV1cAs7zDpq7ORQfEV3oXhzmZQ3gRJfaIBbJ0TPdTmhqqtLjnO99tprefrpp1tP7jdNk0WLFrF8+fK+aaEY1uJWgvLmCqqjtVRFqmmMN3XpeaqiUuwrYlRgJMXeomExHvvgpg/SylqwiEI5d0oMIxKnxECybJu6xjjVDVGqG6LUNES7dC5UC7/HYGShjxGFPgpyhk4S1cKJNlFXuY3tVkNrnRIspMhXyNR8uU6cGNy6nFAtXryYlStXAjBv3rzWek3TOOecI88JL0R3RM0otbF6aqJ11EZrMctjNDd3PqlEC1XRCPkKGekvodhbhDGMxmHXVVaQqD7k6NS0WXJ0SgwrEqdEf3Ech2jcYk9ZA9v31FDbEKOuKdblI1CQvFZUfo6bknwfIwp8BH3GkB1BARA/uIkNqfOmIHl0yhMo4oQsmh5diJ7qckL12GOPAbBkyRLuueeePmuQGD4s26I+3kBdrIG6WD31sYYOk0n4dHen23BpLop9RRT7QoS8heiDeLrz3jjY4dypQopKxw5Qa4QYGBKnRF9wHIdIzKK+OUZ9U5y6phh1TXGicRO/393lnX4APrdOKN9LcZ6XUJ4XlzH0R09A8ujUzopPqLfbzhtTgkXMKDpuSJ7PLIafbv/63LBhQ+tMSe3JrEniSBzHIWbFaIw305hopD7WSEO8kaZEM3TvutKgKOS5cwh5iwh5C8l15wz7PVv1VVXEq/al1YWmHi9Hp8SwJXFK9JRp2TSGEzQ0x2kIx5P3zXFiCatH23MbGkV5XopyPRTlegh4h/ZRqCOp3/cR28y61rKiuxhVNJmR/pJOniXE4NHthOq2225rfZxIJHj11VcZM2ZMRhslBifHcYhaUZoTYRrjzTQlmmhKhGmKNxG3ujab0eEEXQEKPPkUegso9BTIlKqHOLBxbVpZC+RTNEaOTonhS+KU6EzLcL2mSKLtFk7QGI4Tjpm92rbXrVOY46Ew10NhjmfID+PrCjvSyPrK9VjtdqC6ckcwvfDYAWyVEJnV7YRq7ty5aeXTTjuNBQsWcP3112esUSJ7WbZFxIoSSUQJm2HCZoRwIkxzIkJzIozt9GwvXgtVUclxB8l351HgyWPK6DE01vY8GRvqasorOh6dOkZm9hPDm8QpkTAtwjGLcDRBOGYSjpo0RxM0R0zC0US3znU6Ek1VKMrzMjLPQ37QTUGOB697eA4778y+PaupsdqGRSqGi+mj5+A5ypB+0b8cxxn2yX9v9Povv7a2loqKiky0RQww0zaJWjFiZoyYFSdqRYmYMaJmhKgZI2JFiZldHyveFT7DR547lzx3DnnuXHJcwbQZ+Ty6m0YkoTqSnWvfSytr/nxCY8YNUGtET61du4Ybb/wWP/nJvZx55lkD3ZwhR+LU0GE7DrG4RSxhEY1bROMm0ZhFpN19JGZ2a3a9rtA0lVy/i1y/i7yAm7ygm6DPoKQ4h8rKxoy+1lASb6pjc/XmtLriwimMDsj1EbPJmjWr+f3vn+b++x/M+LYvvvh8pkyZyj333JfxbWeTbidUh45LP3DgAJdccknGGiQyw3EcTMciYSWI23ESlkncjhO3EsSteOpxnFi7m2X3bqhDZzRVI2D4CbqC5LgC5LhyyHEHMYbpJBKZ0FBTTXPZrrS6oqkys58QEqcGB8dxsGyHeMImYVrETZt4ou0+lmi5b7vFE3brdPh9QVEUfB6dXL+LoM9Fjs8gx+/C7zWG3DTm/WHTlrcIt/ttoRpupo+dJ0dCsszzz/+BPXt2DXQzBrVenUOlKAoFBQVMmiTXD8gU27GxbAvLsTAdC8u2MG0LyzExbQuz5d42W2+JQx4n7AQJO9GnQedIDNXAb/jwu/zJBMrw43f58eneYT95RKYd+HgdtPsn1vx5FI+bMHANEiJLSJzqW7btYNk2ppVMiEzLxrRsLMshkbo3LZuEmaxPWMl1E6aVqnOIpx7bGRh61xOGrhLwGuk3n4uAV0eTnVIZEQ/XsbFsU1rduOJjCboCA9QiIfpOj8+h2r9/P6ZpMm5cZocXvfzyyzz00EOYpsmVV17JZZddlrb8k08+4Yc//CHNzc3MmTOHO++8E13v+ttwHAfbadvDZePgOA4ObfVOqs7GblsfB7vdY8exW8u2Y2NzSNmxsRwbJ3WfrLOwWsq2ha/eoK4hjOVY2LadvHcyO0wh4xQFr+bGo3vxGV58evLmN3z4DR+GKifg9ofGmmpiVXtw6W3DI4umDJ2jU3ZDJeaeD3GivRtKk1CVjP1gUzxB9LHHo+aEevT8N954nd/97il27dqJz+fltNM+xfXX30h+fn6HdZcufZmf/OROnnjid0yZckxr/d1338Fbb61g+fIVPX0bw8LQiFOp2OS0lFP3NqkYlBz+5tjtlrWsZ3d8bKXuHTt5AVrbSSZGrctSt5bHpp1Mdrw+N/X1EaxUEtWyPNtpqoLHreP36PjcBl63ht9r4PcY+Dw6Ll2VWNXHtu5eRazdedW64WHq6JMHsEWZlYk4lQ0xatGi61i3Ljm51RlnzOEHP7idk0+ex+OPP8q//rWKmppqPB4vM2fO4vrrb2DixMkAPPbYI7z44p/58Y/v4Ve/+gXbtm0hGMzh7LPn881vLkz7zrNtmyef/A0vv/wCtbU1jBs3nmuuuZ7TT/9URt57Nuh2QrVr1y4WLlxIeXk5juOQn5/PI488kpG9f+Xl5fzsZz/jz3/+My6XiwULFjBv3jwmT57cus73v/99/vM//5PZs2fzgx/8gOeee45LL720y6/x541v0BBt7nVbM8HTYBCNJga6Ga1URcFQXRiKC5facvO0e+zGpbpRUcEEzOQBkmagufVRZpU1xKivCx99xWEmtuP9tCnnVV8uxeMnDmCLMsvcsw4n2jTQzUjjRBsx96zDNePz3X7uc889wy9+cR/nnPNFrrvu21RVVfKb3zzMxx+v59FHn+qD1g5vgz1O/XXNXprC2REb4jY0Z1GcguRU5G6XhtvQ8Lh0vO7kvcel4XXr+Nw6LkMSpoEUbqpmZ80OVKPt32DSiFl4DO8Atiqzsi1O9TRGLV58C/fd91/s3bubu+++l1AoxKJF12EYLhYuvImCgkJ27tzBY489zO23/4CnnvpD699WU1Mjd955K5deegXf+tYi/vGPN3nmmacpKCjka1+7vPU13nlnJQ0N9XznO99DURR+/etfceut/5dnnvkzI0aMzGg/DJRuJ1R33XUX11xzDRdeeCEAzz//PHfeeSdPPdX7HwWrVq3ilFNOIS8vD4AvfOELLF++nEWLFgHJvY3RaJTZs2cD8JWvfIVf/OIX3QpUtY1RasORXrc1E4xwgkSi785bUtFSNx1N0dHQk49bboqRemygYaCitf6RWEAkdUtygGjq1n/8VeFuXTRxONDNMGNq96fVDaWjU0NNONzMo48+xFlnfZZbb72ztX7GjFlceeUC/vKXP3LssdMHsIVDz2CPU8OJqiq4dA1DV3EZKi5dS94byYTJpautyVPLTVUlUcp2m3evwnJsVJKjKDwuL5NGnzjArRKHM2HCRILBIIbhYsaMmWzbtpVQqJgbb/wuU6dOA+CEE06ipqaaJ5/8DTU11RQWFgEQj8dZtOg7fPazZwNw4olzWLVqJW+9tSItofL5/Nx334P4fH4A3G4PN9+8kHXr1nLOOV/s53fcN7qdUFVXV7cGKYCLLrqIJ598MiONqaioIBRqO1RZXFzMRx99dMTloVCI8vLyjLx29lBQUVP/15L3Suo+Vdd2U1EV/ZC6ZAKlosk5S0NUfng37U+eUr05lEwYWueH6GNnY+79ECeSPbNnKd4g+pjju/289es/IhIJc+aZ/4Zptu1AGT16DOPGjefdd1dJQpVhEqf6lqIo6JqCpqpomoKuqWiqgqG1lXVNxdAUdL3lsYqhJ2/6IY/F0NLQVMHeul1pdVNHzsbQXAPToD6SbXGqpzHqUJMnT+GXv/w1juNw4MB+9u/fy549u1mzZjWQvLZfe7NmzU4rFxcXE4mkjyw65phjW5MpgFGjkrM8NjZmR99lQrcTKsuyqKura907V1NTk7HG2Laddoj+0Dnxj7a8K3RDx+XSAaX1P1L/BzVZUpS05Qpq23pKep3SWqemr6+oqUdqqtxSUluXq6io7pbHWts2ZJhCGr9frlXRwkg0ke9Uoxhtf7pT551KSUnuALaqdyoqVHT9kB9VBSW4Cs4emAZlWGNjPQA//vGt/PjHt3ZYPmbMWLTUj0ot9QO0ZQ+8pqX3Tct3Q4f+6oVMbqszqqoSCgX75bUGe5zyed2gqCiqgqIkh2MrioKqJI/oKEpbfUtZbVduqdNSj1UFNFVNPVZQ1bZyyzqaqqClEqPDPda11H277Yuk/vpcDxYfbX8V3Wj7XikI5nLS8WeiaYN3Vt+hHqcOjS1//OOzPPnk41RXV5GTk8uUKVPweJK/xVQ1PU4FAv60vlFVFcdx0ur8fl9a2Uj9hlEU56gxqL9iFPQuTnX703355ZdzySWXMH/+fBRFYenSpVx55ZU9evFDjRgxgjVr1rSWKysrKS4uTlteWVnZWq6qqkpb3hWnF59CPNG7i89mSm6eT84POgrpo3Teg5swPC1fRAqh0SPR80cN6uug2LaNmeFrxrTQdbXPtt1VPl9yRqvvfW8J06Yd22G5Ybior68DSM6OZtqtp8fF42Za+5ubk+P1M/We+rN/bNvu9HOqqgqFhZmZ/WvQx6npxe1PkRwADlgWjgX5oSCVlY3IwOvDC6X6RyRV1+9nZ9mO1nN8DZfGxMJZ1NRkx6kWPdVXcSobYhTQOlGbadq88cbr3HffT/n616/moov+naKi5BH3p556nDVr3sOykn3RMpmGaab3TctkOofWtS9bVvKxbTudvv/+7p/O4tTRYlS3E6pLLrmEsWPH8vbbb2PbNrfffjunnXZadzdzWKeddhoPPvggNTU1eL1eXn/9de66667W5aWlpbjdbt5//31OOukkXnzxRc4888xuvcaUMXlZM0ORfBEfnfRRG7u5lkRVA+S1ndRbNO0E6h3ZU5zNZsyYicvlpqzsIBdccFFrfSQS4bbb/oMTTzy5Q6LVMjSioqK8dVk8Hmfjxo/7r+GD2GCPU4qiDMhlL4ToDcdx2LjnnbQJkwo9uYwcNWsAWyW6ov052B98sBZN0/jGN77ZOlOf4zi8++4qIJl0iI56dPz11FNP5dRTT810WygpKeHmm2/miiuuIJFIcPHFFzNr1iyuvfZabrzxRmbOnMm9997LrbfeSlNTE9OnT+eKK67IeDuEyEbW/o1pZcWfj1FYClXZM8uQ6CgnJ5crrriaJ554lHg8ximnnE4kEuH3v3+KLVs2cemlHb/D5sw5GZ/Pz8MPP5gc2qVqPPfc7zuMXRdHJnFKiP51sH43NQ0H0+pOmHQqqqod4RkiWwSDOdTU1PDOOyuZPn0GL7zwJ372s5/y2c+eTWNjAy+++GfWr/8QgGi0fycnGyyybkDr+eef3+Eq948++mjr42nTpvGnP/2pv5slxICyGyuxGyrS6vTS4+Q8hkHiqquuIRQq5vnn/8ALLzyPx+PlmGOm8fOf/4rjjz+BtWvXpK3v9wf4r/+6j4ceepDbb/8Bubl5nHfelzn55Hn89rdPDsybEK0kTgmRznZsNu35V9rRqRJ3HqPGzqSqWobtZ7sLLriIdes+YMmSxXzjG9/immu+xSuvvMiyZa+Qn1/A8cefwM9//ituuul61q17n0mTJh99o8OM4gyzcQXV1U0y5G8QkT5KHmpPbP4nTlPbifVqsAh96hkUF+cM+v4pK9vNiBGZvfBqi2wZn56t+rN/jvbvnMlzqAY7iVODh/RP0vaqzWzY/HprQqUocOakzzNl5twh0T99FackRnWuv/uns3/no8Uoma9UiCxn1+5PS6YANDk6JYQQIgs0JZrZtO+9tKNTo90F5BYfM4CtEqJ/dXvI3wcffMD9999PfX192kmzL7/8ckYbJoQAxzax9qVPRKDmjUANFA5Qi4TIfhKnhOgfjuPwUdk6zOba1jpDUZk2eh6KXGxeDCPdTqh+9KMf8ZWvfIXjjpM95EL0Nat8O0683fhzRUUbPWPgGiTEICBxSoj+sadxH9VV29OOTh3rG4G3eGhdbF6Io+l2QqXrOldffXVftEUI0Y4Tj2Ad3JxWpxVPRPXIRSSF6IzEKSH6XjgR4ZOKj7HD9a11RaqHMaPnyNEpMex0+xM/ZcoUNm/efPQVhRC9Yu7fCHa7i1DrbrSRMiZdiKOROCVE33Ich/VVG0k0VLQendIVhRn+0WiFYwe4dUL0v24fodq7dy8XXXQRo0aNwu12t9bL2HQhMsduqsGu3pNWp5cei6K7BqhFQgweEqeE6Fvb6nZQ2VSGHWk7OjVVzyNQOl2OTolhqdsJ1c0339wX7RBCpDi2jblnXVqd4stFLeqbqcWFGGokTgnRdyrD1Wyp24HdWNV6dKpAdTPWV4JaOGaAWyfEwOh2QjV37lwA9u/fj2majBsnP/KEyCSrYhtOuzHpAPromSiK7PUToiskTgnRNyJmhHWV63ESMZzU0SmXojLLKEAfdazEKTFsdTuh2rVrFwsXLqS8vBzHccjPz+eRRx5h0iSZ0UWI3nKiTVgHNqXVqYVjUHNCA9QiIQYfiVNCZJ7t2Kyt+Ii4FcdurAQneQHf441CvN581ILSgW6iEAOm27sS7rrrLq655hrWrFnD+++/z/XXX8+dd97ZF20TYlhxHAdzz4eHTEThQh89c+AaJbJe++ssiSSJU0JkluM4fFS1kbpoPU4iihNpBGCylkuh6kGTo1OiE8MhTnX7019dXc2FF17YWr7ooouora3t5BlCiK6wa/ZiN1Sk1emjZ6AY7iM8Qww3v/zlz/niFz/LZz5zOo8//ms2b97Edddd1e3tLF36MmecMYctW4bmTHgSp4TIrM2129jfeAAgeXQKCGkeJmpBFF8eav6ogWyeyCKZiFODMUZ1e8ifZVnU1dWRl5cHQE1NTcYbJcRw48QjmHs3pNWpOcWoMv2sSFm//kOeffa3fPGLX+Lcc8+npGQkf/jD7/jkk48HumlZR+KUEJmzs34P2+t2AuDEwzjRZnyKzky9AEVR0EvlAtoiKVNx6tRTz+Dhh59g7NjB8xuo2wnV5ZdfziWXXML8+fMBWLZsGVdccUXGGybEcOE4DubuD8CMtVWqGvrY4yVIiVZ1dckjLF/4wrkcf/wJA9ya7CZxSojMONBUxsaa5FECB7AbKnEpKie5inApGmqwCCWneGAbKbJGpuJUfn4++fn56LqKadqZal6f6nZCdckllzBu3DjeeustbNvmjjvu4NRTT+2LtgkxLNiVO7Hry9PqtFHHongCA9QikWnbtm3l4YcfZOPGj4nFoowbN56LL17AueeeD0A43MzTTz/JihVvUF5eRlFRiC984VyuvPIb6LrO3XffwbJlrwBw443fAmD+/PNa6844Yw5XX30t3/jGNznjjDl8+9s3ceDAfv72t9dQFIW5c+dxww2LKSoqOmz7HnvsEZ544lGWLfs7wWCwtX7RoutoamriySd/36X3kS0kTgnReweaylhXub51anSiTWiJKCcZxfgVAwBNjk4NGf0Zpz744H0ee+wRtm7djG3bTJ48lcsvv4rTT/8UkBzy95Of3MlTTz3DxIlTAFi7dg2PPvoQW7duJhAI8qUvXUhZ2UEOHNjPL3/569bXuPHG71JWdpC//e11mpqaOOaYaSxceBMzZszq0/7rckJ100038cADD3D++emB8+233wbkgolC9IQdacDcd8hQv2ARWsnwno2sKlLDhupPaI4392o7qqpg25k5Gdbv8jOj8FiKvAXdel5zcxM337yQ8eMncOutd+JyuXj11Zf4yU/uJD+/gJNOOpmFC6/lwIH9XH31tUyaNJkPP1zH008/webNm/jpT3/GVVddw/jxE3jooQf57nf/g6lTp5Gfn088HuONN/7Kww8/QXFx217ip556nAkTJnDrrXdQW1vLI4/8khtuuI4nn3wm7UK3mXwfp556eo+2m0kSp4TIjP1NB1lXuaE1mXJwcBormW0UkqsmLzCv5o1ADRQOZDMHVCbiVDbEKOjfOLV//z6+//2bOOWU07nrrv/GcRz+8IffsWTJYp588hkmTuz4+2fDhvV897uLmDVrNnfc8ROam5v4zW8eprq6muOOm5627uOPP8rs2SewZMltRKNR/ud/HuCWWxbz/POv9Dj+dUWXE6prr70WgNtuu63PGiPEcOLYNubO99Nn9dMM9PEnDvvZkjZUfUJzonfJVKY1x5vZUPUJZ43pXuKwa9cuamtrWLToO5x22hkAnHDCSRQUFOJ2u1m69GW2bt3Cf/3X/ZxxxpkAnHzyKeTk5PKLX9zHu++u4pRTTmPMmOS1lMaPn8CMGcmZHwsKkkecWsotPB4PP/vZr/B4PACMGDGCm29exGuvLeVLX7qQnjja+8gGEqeE6L09DftYX/1J25EpgHADM/ATUr3JsqKgjTp2YBqYJbItTvU0RkH/xqk33nidaDTKV7/6NY4/fjYAxx57HP/7v49hWeZh2/foow8RCpVw330PYhhGanuzuOyyizusW1QU4p577ms9chqNRrn77jtYv/5D5syZ2+2+6aouJ1QzZswAYNy4cZSUlKQt27ZtW2ZbJcQwYB3YiBOuS6vTx85CcfsHqEWiL0ycOImCgkL++7/v5l//eoeTT57HvHmncsMNNwPwwgvPEwgEWoNUi/nzz+MXv7iPtWvf45RTTuvWa37mM59rTaYgGfiCwRzef/+9HidUR3sf2UDilBA95zgOO+p3s6lmyyELbKbHHEZpbbFJLRiD6svr5xaKvtKfcWr69Fl4PB7+4z9u5t/+7bPMmTOPuXNP4YYbvnvY9ROJBOvWvc/FFy9oTaYASktHH3YY38yZs9KGoRYXJ2NBNBrpUvt6qsu7wevq6qirq+Paa6+lvr6+tVxVVcWiRYv6so1CDDlW3QGssq1pdWp+KWrBmAFqUXaZUXQsAVd2nUMWcAWYUdT9PbJer5eHHnqMz33ubFavfpe7776DCy6Yz803L2Tv3j00NNRTUNBx2EwwGMTlctHY2NTt1wyFSjrU5efn09jY0O1ttTja+8gGEqeE6BnbsfmoamOHZEpRFGYpQUY5bT9kUTX00uP6uYXZJ9viVE9jFPRvnBoxYgQPPfQYc+eewptv/pXbb1/C+ed/nltv/b/U1FR3WL++vg7LssjPz++wrLCwY5s8Hm9auSW5ytTQyiPp8hGqxYsXs3LlSgDmzZvXWq9pGuecc07mWybEEOVEmzB3rk2rU1we9HEyq1+LIm8Bnx7dvaMyh5MtMwSVlo7mBz+4PbkHeMd2Vq6DK/H2AAAgAElEQVT8J0899Tg/+ckdhEIlbNq0scNzGhrqicfjrVN/d0fLTEvt1dRUc+yxh/8R1BZwrLT6cDjc5ffx0EOPd7udmSZxSojui1sJ3q/4kJpI+uUFVEXlhPyp5O9Yl1avlUxGcaX/aB2OMhGnsiVGQf/GqSlTjuHHP74Hy7LYvPkT/vGPv/Pss7/FMFzcfvt/pq0bCARRVfWwyVY2XRKjy0eoHnvsMTZt2sSFF17Ipk2bWm8ff/wx9913X1+2UYghw7FNEjtWg5Voq1RU9IlzUfTsOA9FZNbq1e9y3nmfZ8uWTSiKwqRJk7niiv/DySfPo6ysjBNPPImmpibefvufac9bvnwpALNnn3jEbavq4b/CV678J7bdFqT/9a93aGpq4uSTTzns+n5/cihPRUXbbJN1dXXs2rWjy+8jG0icEqJ76mONrDzwrw7JlKEanDziBAprK9PO81UMD9qIKf3dTNHH+jNOvfrqS5x33ueora1B0zSOO24G119/AxMmTKKs7GCH53s8HmbOPJ633vonptl2jlVFRTkff7y+N287o7o9bfpXv/pVrr/+esLhMI7jYNs2+/btY8WKFX3QPCGGjuT1pj7ECden1eujpw/rmZKGuuOOm4Gu69x5561cddU1FBYW8cknH/POOyu5+OIFzJ9/Hn/5y/PcdddtqdmTpvDRR+v47W+fZO7cUzn55HlH3HbLFOd//etypk+fyahRpQDs3r2LJUsWc8EFF1NRUc6vf/0/TJt2HJ/73BcOu50zzvg0//M/D3Dvvf/F1VdfSyQS5umnn8Dnaztn4mjvI5tInBKic47jsKdxHxurt2A76UemfYaPk0tOwJeIk6jZm7ZMG3UsimYghpb+jFMnnHAS8XiCW25ZzKWXXkEwGGT16nfZtm0LN9/8/cNu47rrFnLjjd9k8eIb+epXFxCLRXn88V9jmuYRdyz2t24nVLfddhtf/vKXee2111iwYAFvvPEGZ599dl+0TYghxS7fhl2dfq6JWjAatXh4T5E+1AUCAX7+81/x6KO/4sEHf0ZjYwMlJSO46qpruPzyq9B1nQcffITf/OYhnnnmtzQ01FNSMpIrr/wGl112ZafB4uyz5/OPf7zJ3XffwXnnXcD3vncLAOeffwG2bfOjHy3B7Xbzb//2ea6/fhG6fviv/NLS0dxxx0944olfs2TJYkKhYi655FJ2797FRx992KX3kU0kTglxZAkrwUdVGylrLu+wrMCTz0klx2OoOont76XN9Kd4c1CLxvZnU0U/6e84df/9D/L447/m//2/uwmHw4wePYbFi2/hwgs7ztoHcPzxs7n33gf49a9/xY9+dAs5OblcdNG/s3LlW3i92TH8VHEcp1tnaZ133nm88sor/Pd//zef+9znmD59OhdddBGvvvpqX7Uxo6qrm/r8xLSuCoWCVFY2DnQzstpQ6SOr9gDmjtWHBKcgxrSzULRu79doNRT6p6xsNyNGjOuTbWfT+PT+csYZc/jqV7/GTTctPuq6/dk/R/t3VlWFwsLMnOAtcSpzhsJ3TF8abP1TEa5kfdUnRM1oh2Vjc0ZzXMExaKqGVbkTc3f6uVPG1NNRc4o7PK8zg61/jqSv4tRwjFHd0dI/b721goKCIqZPn9G6LJFI8JWvfJEvfOFcFi36TkZer7N/56PFqG7/kmsZaz927Fi2bt3KSSedlDWH24TIRnZzLebONenX9NAN9EnzepVMCSEOT+KUEOniVoKNNZvZ33igwzJd1ZlZdByjAiMAcMwY5v70CQjUgtJuJ1NCZMqaNatZuvQVvvnNbzN58lSamhr5y1+eJxqN8OUvf2Wgmwf0IKGaOXMm3/nOd7jpppv45je/ya5du444jESI4c6JNWNuezf94r2KijFxLqonOHANE2IIkzglRJLjOBxoLuOTmi3EzFiH5TnuHE4snoXf8LXWWfs2ghlvW0nT0UfP7PBcIfrLwoXfwev18cc/PktlZQVer5dZs2bz8MNPMGZMdgxD7faQP8dx+PDDD5k9ezYrVqxg1apVLFiwgIkTJ/ZVGzNKhlIMLoO5j5xEjMTmt3Ci6e3Xx5+AVjQ+I68xmPunhQz5GzhDdcifxKnMGQrfMX0pm/unPtbIxupN1EQ7XkYBRWFS7nim5E1EU7XWaruphsSmf6Stqo2egd7Dmf2yuX+6Q4b8DYz+7p9+HfK3ePFi/v3f/x2As846i7POOqu7mziiAwcO8P3vf5/q6momTJjAvffe2zp0o8X+/fs577zzGDs2mZEWFRXx2GOPZawNQmSCY8ZJbF3VIZnSRkzNWDIlhDi8vopTEqPEYBA1o2yt28Gexv3pQ81Tgq4As0LTyXPnptU7to25J/28KcUTRJOJk4Q4qm4PKp8zZw73338/n//853nkkUeorKzMWGPuvPNOLr30UpYvX86MGTP41a9+1WGdDRs2cP755/Piiy/y4osvSqASWcexTBLb3sEJ16XVq/mlaHJ1eSH6XF/FKYlRIpvFrQSbarayYt9K9jTs65BMqYrK5LyJnD5qXodkCsAq29Lxsh5jZ6HI+YdCHFW3/0ouvfRSnnvuOR5++GHq6+tZsGABCxcu7HVDEokE7733Hl/4QvI6KV/5yldYvnx5h/XWr1/Pli1b+PKXv8wVV1zB5s2be/3aQmSKY1uY2/+F03TIFedzitEnnISiKAPUsuzWzZHHYpDp73/fvohTEqNEtopbCbbUbmfF3rfZXrcTy7Y6rBPyFfGp0adyTMHktCF+LexwPVZZ+mdVLRgtE1G0I3FqaOvtv2+Pz9KNRqPE43Ecx0HTOv5xdldtbS2BQKD1xOFQKER5ecdrJLjdbr70pS+xYMEC3nrrLRYuXMjSpUtxuVy9boMQveFYJub2f2E3VKTVq4HC5Ix+hwliAlRVw7IsmTRgCLNtC3UAPv+ZjFMSo0S2iZoxdjbsZnfDPizbPOw6PsPHcYXHUOILHXE7jmNj7v4A7LZzVRTDjT52VsbbPFipqoZtW2gyM++QZVm9i1Pd/mQ88cQT/PnPfyYej3PxxRfz3HPPUVRU1K1tLFu2jHvuuSetbty4cR323h9ub/4NN9zQ+vjTn/409913Hzt27GDatGldeu1MnfScKaGQzPR2NIOhj2wzTtOG1ZhWPfjdrfV6IJ/ArLNQ9b77MTUY+qcziUQu8XgYjyevT7av6zJcpTP90T/RaJRAwN9vn9XexqmBjFEgcWqw6e/+qYvUs6V6J3vqD2A7Nm6vBqT/EPTobqYXT2VC3pijXjIguvcTwoTTYlfg2NNwhbr32+5IhsLnJxz2k0hEcbtzMr5tiVGd66/+CYfD5Ofn9vjz2u2EasOGDdx6663MmzevRy8IMH/+fObPn59Wl0gkmDdvHpZloWkalZWVFBd3PNT89NNPc95555Gfnw8kD9F1Z8+2zJ40uAyGPnLMGImt7+A0p8+kpHhzMEaeSKw2BnScrjYTBkP/HI2ieGhqKse2HTweP5qmZWxopMyg1Lm+7h/HcUgkYjQ01JGfX9zpZzWTs/z1Nk4NZIwCiVODSX/1j+3YlDVXsLtxHzWRmiOuZ6gGE/PGMz5nDLqlU13d3Pl2Iw0kPnk/7eiUWlCKRR5k4H0Nlc+PYfipra1AVXUMwy0xqp/0R4yyLItotJlwuJGCgpIjfl4zPsvffffd192ndIlhGMyZM4elS5dy/vnn88ILL3DmmWd2WO+9994jGo1y7bXXsnr1amzbHjRT4Yqhx4k1JyegiKT/ASq+PIypp6Ho7iM8U7TQdYOCghKamxuoqSnDPsz4/55SVRXblmB1JP3RP7puEAzmYxj9N+StL+KUxCgxEJoTYfY27mdv437iVvyI67k0FxNyxzEuZwyG2rWfdo5tYu54Ly2ZQnejjzm+t80ecgzDRTCYT0NDDaaZyNh2JUZ1rj/6R1U13G4vBQUl6LrR4+10+TpUN910Ew888ADnn3/+YZe//PLLPW5Ei/3793PLLbdQXV3NyJEjuf/++8nNzeWZZ56hoqKCm266ifLycm655RYqKytxu93cfffd3RpKIXv+Bpds7iO7uRZz27s4iWhavRIowJh8KkofDvNrkc39kw2kfzqXTf2TiSNUfR2n+iNGgcSpwaQv+iduJTjYXMb+pjJqD3cNqXZ8ho/xOWMZGyw97GQTnTF3r8Oq3JlWp0+cg1YwptttPhL5/HRO+qdz2dQ/R4tRXU6oNmzYwIwZM3jxxRcZOXJk2rKtW7dy2WWX9a6l/UQC1eCSrX1k1R3A3LEGDjmaouaE0CedgtJPJ65ma/9kC+mfzmVT/2QioZI4lXnZ9BnJRpnqH9M2KQ9XcrC5nMpwNbbT+ZH6Am8BE3LGUuwrQlW6f46JVbMfc8fqtDq1cCzGhJO6va3OyOenc9I/ncum/snYkL/Ro0dTV1fHY489xtNPP906vaBpmvzoRz8aNIFKiN5wHAerfCvW/o0dr/GRX4o+4USULg63EEJklsQpMZjErQSVkSrKmsup6EISpas6pYGRjM0ZTY6r5xM9OLFmzD0fpNUpnoDM6idEL3T5l9/ixYtZuXIlAPPmzUNRlNapaM8555w+a6AQ2cKxEpi71mLXHuiwTBsxBa10ulxnSogBJHFKZLumRDOV4SoqwlVUR2u6dO2bfE8+Y4KjGOkvQe/lDjvHtkjseA/anwekqugTT0bRen7+iBDDXZf/Mluu9r5kyZIO08kKMdTZkQbM7atxooccelZU9LGz0EITBqZhQohWEqdEtknYJjWRGqqiNVSEqwgnwl16ns/wMTowktLASHyGLyNtcRwHc/cHHWak1UfPQPX1zWUrhBguuj3w9qabbuKOO+4AYMeOHXz729+mqqoq0+0SIis4joNVuYvEJys6JlO6C2PyKZJMCZFlJE6JgWLZFlWRGrbUbmfVgff46+6/s6Z8Hbvq9xw1mfIaXibmjef00nmcNfp0puRPylgyBWCVb8Wu3ptWp+aNRA3JLJRC9Fa3jx3fcsstfOYznwGgtLSUuXPnsmTJEh599NGMN06IgeQkYpi7P8CuO9hhmeLLw5g0F8XtH4CWCSE6I3FK9Je4laAiXElttJ6aaC11sXpsp+vTPAdcAUp8IUb4i8l15fTZsHGr7kDy3N92FE8QffyJMlRdiAzodkJVW1vLFVdcAYDb7eaqq67ihRdeyHjDhBhIdl1ZcmjEIVOiA6hF49DHzpLJJ4TIUhKnRF+wHZvmRJjaWB31sQZqo/XY5XGam7t+4XZVUSnw5BPyFVHsKyJg9P1OOTtch7nz/fSJlFIjLPrj8h5CDAfd/kVoWRbl5eWUlJQAUFVV1aWTKoUYDJxEFHPveuyafR0Xajr6mFmohWNlj54QWUzilOgt27EJJyLUxxuoj6Vu8UYs20xbz2cc/eLtfpefIk8BIW8hhd6CXk8s0R12tBFz6yqw2rVbUTEmzkXx9O4yBUKINt3+q77qqqu44IIL+NSnPoWiKKxatYrvf//7fdE2IfqN4zjY1Xsw920As+PV6BV/PsaEORKAhBgEJE6J7kjYJk3xJhrijTSm7hviTR2Sp67yGl4KPfkUePIp8hbg1b0ZbnHXOLFmzC0rcRLpR9D0MTNRc0ID0iYhhqpuJ1QXX3wxM2bM4N1330XTNL7xjW8wderUvmibEP3Cbq7F3PsRTlNNx4WKmpwSfeQ0FLX7F08UQvQ/iVPicEzbpCnRTFO8maZEM43xJhoTTUQSkV5tN+gKkO/Jo8CTT747D58xMAlUe048QmLrKpx4+nvTSiajFcskFEJkWo+OO0+bNo1p06ZhWRbLly/nhz/8IX/84x8z3TYh+pQTj2Ae2IhdteewyxVfHvr4E2Q6WSEGIYlTw5Pt2ETNGM2JMM1mMnlqToRpTjQTMTueE9tdLs1FrjuHfHcuue5cppSWUl/b9XOo+kMymVqJE21Kq9eKxqONnjFArRJiaOtRQlVfX88f/vAHfve73xEOh7n88ssz3S4h+oxjxrHKtmBV7AD7MFemVzW0UceilUxCUeSolBCDkcSpocuyLcJmhIgZoTkRIWxGCCfCNJthIolIt2bZ64xLc5HjCpLjDpLnziXXlYNX96SdQ+vSXUD2JFROtInElpU48fQp2tWC0Wjjjpfzf4XoI91KqHbs2MH//u//8tJLL1FaWko0GuXNN98kGAz2VfuEyBjHjGNV7sQq2wpW4rDrqHkj0cfMlOnQhRikJE4NfqZtEjGjqVsk7XHYjBAzM5zAKAp+w0eOK0jQCJDjCpDjDuLRPIMqAbGba0lsfQcO6R81byT6+JNkB6EQfajLCdV1113Hhg0bOPfcc3nqqaeYOXMmn/nMZyRIiaznJKJY5duxKnceMZFSvDmpE3WL+7l1QohMkTiV3RzHwbRNolaMqBkjakVb7yNm2+PEEb6ne01R8Os+AoaPgCtA0BUgYAQIGD40Veub1+wndn05iR2r02fzI5VMTTxZzgEWoo91OaHauHEj06dPZ8qUKYwbNw5gUO25EcOPHWnArtiBVb3n8EP7AMXlQRtxDGpovOy9E2KQkzg1MGzHJm4liNtxYmacmBUjasWIWcnHMbOlHMM6wndxJrk0F37Dh8/wETD8+FP3Pt076BOnQzmOkxzCfuCT9OtMkbpm4rjZEtuE6AddTqhWrFjB66+/zjPPPMPdd9/NWWedRSyWPeOGhQBwHBunvgKrYjt2Q8WRV9RdaCOmohVPkAv0CjFESJzKjJYjSTE7jhJOUNZcS9yOE7cSxKw48dStJWGK24kOP+b7lKLg1T14dS8+3ZtMnnQvPsOLX/dhaEb/tWUAOWYcc9da7LqDHZZpI6ailR4nOxSE6Cdd/iWp6zrnnnsu5557Ltu2bePZZ58lFotx9tlnc/XVV/O1r32tL9spRKecWDNW9R7sqt0dpolNo7vRSiahhSbIFeKFGGIkTnVk2RYJO0HCNolbidTj5K2lnEyQUo9T5ZYLIftq3ISb+zcpVRUVj+7Bo3vw6h58qeQpeUvWqcP8qIvdVI25a22HmfxQFPTRM9BKJg9Mw4QYphSnF5ePj0QivPTSSzz77LP85S9/yWS7+kx1dRO23Y970joRCgWprGwc6GZktc76yDHj2HUHsGv2YTdUdrodxeVDGzEZtWjckDoiJZ+hzkn/dC6b+kdVFQoLM3/h7MEep2zHxrQtzFRSZNomidStpS55S2BaidbHLfe9HWLn82c2oVIVDY/uxqO5k0mT5saju/GmEiiP5satuQfNkZX+/htybBPrwCas8m0djwrqboyJc7LqXOBs+o7JRtI/ncum/jlajOrVL0uv18sll1zCJZdc0pvNCNFljhnHbijHrj2AXV8GdufT4yqBArTiyah5I+WkXCGGocEYp/5V9j710UZM28KyzaM/IQsYmoFLc+HW3LhT957UY4/ubi3rqj5okqVsYzdWYe5ehxPt+ANT8edjTDxZZqgVYoAMnV31Yshy4hHs+vLk0ajGyqMmUagaakEpWmgiqj+/fxophBAZEo6HMz81eDdpqo5LMyjwBvE7Nm7NhUt14dJcqcQpPYEa7kPw+pITa8bc9zF27f7DLteKJ6CNnokyxCbcEGIwkYRKZB3HtnCaarAbKmjYU0e8opPJJdpR/PloReNRC0pRhslJyUII0RlFUTBUA0M1cGnJeyN171J1XJqr3TIXLs3ApRqts+Fl05Cb4cZJxLDKt2JVbD/sjkTF5UMff0JWDfETYriShEoMOMe2cJprsZuqcRorsZtqWqc5N/3uTp+reAKoBaNR80tRvTn90VwhhOg/ioKh6Oiqhq4aGKqOrurt7lsSpJa6VNKk6hiaga5oMsRukHHikWQiVbnriJf80IrGo42ZITsPhcgSklCJfuU4DiQi2M21bUlUuO7ow/jaUTwB1LyRqPmlKL48+bEghBhS5o48CRzQFV3OORomHMfBaa7BqtyZHNp3hJio+POTF6EPFPZzC4UQnZGESvQZx3EgHsYO1+OE61K3epxEtHsbUhQUfz5q7ojk5BKeoPzAEEIMWX7DlzWz0Yq+5cQj2LX7sap24USOPLRScXnQSqejFoyR+CdEFpKESvRaS+LkRBtxok04kUbsaANOpAGsns1Qpbh8KDkhAuPGY1oBuWaUEEKIIcGJh7HrDiZnq22q7vSiyIrLi1YyGbVoPIomP9mEyFby1ym6xLEtiEdwYs3JWzycTJ5iTTix8BHHeXeV4vahBApRA0WowSJw+1EUBVcoiCInRAshhBikHCuRmmipHLuhotMjUS0UTwBtxJTkESmZvU+IrCcJlcCxTEhEcVI34pHk43gEJx6GRAQnEet0L1q3qBqqLw/Fn58cyhcoRHF5M7NtIYQQYoA4jo0TacSJ1OM012A31SRHa3Qlfioqat5ItNB4lGBIhvYJMYhIQjXEOI4DVgKsBI4ZBzOeuo+1K8cgEcMxY8kEqofD8rpEN1C9uSi+XBRvbnISCU9QLrIrhBBi0HJsEycWJl5Vj1lWnhzyHmlMJk/dHLGh+PNR80vRCsegGJ4+arEQoi9JQpUFHMdOfgHbFlhmcnidbYJtJY8e2WaqPnmPZeKkkiasRHIdK4FjxZPLM3UkqTt0F6onmEyWPAEUbw6KNwcMj+xlE0IIMSg4jp2Mp4m2HY+tozYSEZx4BGLh1smVmvxurOZuXoRZUVH9+Sh5I9DyS1Hc/j54J0KI/pSVCdXPf/5zNE3jhhtu6LAsHo/zwx/+kA0bNuDxeLj33nuZNGlSl7ftJKI4lpVMOhwn+eXpOICTmqbUgZa61L2TVm732LZTy5KPk8sssB0cx0rVpe5tKz1xsi3qfDqxxnC3pgwfMIqCYniSX/wuX/KcJ3cAxe1PJlAyaYQQYpjoyxgljs5pidlpsTcVk20r9dhMlc1DdlimylYiuWPSTLTuoHTMeHJHZR/slFS8weTw9pwS1GCRxEwhhpisSqgaGxu55557ePXVV7nmmmsOu87TTz+N1+tl2bJlvPfeeyxZsoTnnnuuy6+R2PQPrEhzpprcK7bpzo5kSlVRdA8Y7mTS5PImjyy5vKmbDwyvDNMTQgxr/RGjAKz9H2PFY9CN3/VOpysfZlmHqkMqUklFU7WXRH0kfZ20VZ209Vvr2pdbH7fVO47dtqw1QWp331p/uB2bTp8kPZmiuDytQ9xVfwFKoEASKCGGOO2OO+64Y6Ab0WLZsmUYhsGMGTNQVZV58+Z1WOdnP/sZV155JaNGjaK0tJRHHnmEz33ucwSDwS69xvr315BrgG07LN1UhqooFPhcmJbNss1l6KpCvs9F3LRZvrkMl6aS53URTVi8tqUcj66R6zUIx01e31KOz9DI8Rg0xUz+urUcv0snx2PQEE3wt60VBN06QbdBXSTOG9sqyPUYBNw6NeE4f9tSRq7bwO/SqW6O8fftlRT6XPhcOpVNMVZsr6TI78ZraJQ3RvnHjkqKA248hsbBhgj/3FHFiKAHt66xvz7CWzurGJXjxeV2sa8pwVs7qxlbOgpXbhF7w/D2jkrGHzsbd8lEdkc1Vu4oY/Kp5+AeM4udjRarPtnB1JM/g1Ewiq0HKlj5/lqmzjgJ1XCzefNG3n57BdOmTQdg48aPeOedtzjmmOMA2LBhHatXr2Lq1GMB+PDDtaxd+y+mTJkGwAcfrOHDD9cwefIxAKxdu5r169cxadJUANaseZePP/6ISZOmALB69Uo2b/6EGTOOIxyO8847b7F9+xbGj0/u6V258h/s2rWdceMmAvD2239nz55djB07AYB//vMN9u3by9ix4wFYseKvHDx4gDFjxgHw5puvUVlZTmnpWAD++tel1NZWM2rUaABef/1V6uvrGDmyFIDly1+isbGRkSNHAbB06QuEw2FGjBgJwCuv/Jl4PEZx8QgAXnrpT1iWSShUAsALLyR/UBUVFWNZFi+99CcURaGoKEQikeDll59H03QKC4uIxWK88sqfMQwXBQWFRCIRXn31L3g8HvLyCgiHm3n11Rfw+XyMGlVCeXk1y5a9iN8fIDc3j/r6OpYvf4lgMIecnFxqa2t47bWXyc3NJxgMUl1dxeuvv0J+fiGBQICqqgpef/1ViopC+Hx+ysvL+NvflhIKleDz+Th48ABvvLGMkpKReL1e9u/fy5tvvsbIkaV4PB727t3N3//+OqWlY3G73ezZs5O///2vjB07HsNwsWvXdlas+Bvjxk3EMAy2b9/KP//5BhMmTEbXdbZu3cxbb73JpElT0TQt9Vn7O1OmTENVVTZt+rjHnz2/383Klat69NmbMGEywJD+7LlcKs8++/seffZyc/NpbGzM2GfP7w/g82X3j87+iFEAH779JjmJOqymWl5duwki9eQTJtFYw9IPNqFG68kjTKyhhmUfbEKPNZDrhInU17B83Wbc8UZynDDNddW8tm4L3kQTQTtMY201r3+4BZ/ZTNAOU19bxV8/2krAaiZgh6mtqeJvH20jxw7jt8NUV1fx2gebybEj+KwwVVVVvLlhOwVKFK8VpqKqir9/vINCNY7HilBWWcWKjTsJ6QncVpQDlVX845NdlLgsXFaUfRXV/HPTHka6HVx2jH0V1by1ZR+lPgXDTrCnspa3tx5gTEBHd0x2Vdaxcns543LcaDjsqGpg1a4qJub7UFWFbVVNvLO7msmFARRFYUtlI+/uqWFqKNnXmyoaeG9vDVOKkuWN5Q2s3V/L5KIAABvK6vnwQD2TCpPl9Qfr2VBWz8RU+cMDdWwsb2BCQXJI3gf7a9lc2cj4Aj8oKmsPNrKtNsa4icegFY5lzYFGdkZ1Jp8yH61wDO+s38jusnLGT0h+twzm74qexqnc3Bw8nkBGvyuGUpyaPft4wuF4j38jDfU4NXbsaOrrm3v8GymTcWrEiJEUFORxJFl1hOqCCy4A4MEHHzziOhUVFYRCodZyKBSirKyMUaNGdek1XC4Nn1/Hsh00TcHl1mfW/okAABRKSURBVPH53SQsO1U28PndaKaVLHuSZSVuomkKbk9yfUfXUuXkcktT08oJVUHTFDweFz6/m5hCsuxNLo+kdq55vAZ+v4ewoqLpGr68XIJ5OYTdYfSKZnJGlpJfkE+4ugGjJk7e5BkUFRYQKa/G1fARRSd8KvmB2n8Qd2Q1Iz59AXn5BUS3bcPd9DYj5n6WQCBA4+bNuCrqKZk2A5/PR63t4Nq1i1BJIR6Ph4oKD4ahEQoFMQyDgwe9rWVN09i/v205QCDQsexy6e3K7g7lujqjtez3u2lsbFvu87mIRIx2ZTemGUv9Gwfx+VwoipW2fiKhtJa9XlfrugAej4FhGGllr9eVVvb53IeU25a73Tp+v7td2SAQaCu7XPphyp7WsmFoHcrBYLJsWRaGoZGT4yUUCpJIJFLl5PJo1MAwNHJzk8vDYS1t/aYmJbXcB0BhoR/D0MjL8xEKBdF1M62sKDEMQyM/P1l2nEha2bKaU2U/oVCQeLwBw9AoKPCn2uNrLRcVBWluTpYLCwPk5wdpbGwp+8nNDVJf37Y8EAhQU5P8LBUVBfD5fFRVtZU9Hg/l5dn52Wu/XD57HT97oVAQl8vO2GevMPUjNpv1R4yC7IxTPr+b5lR7PN7k+p5Ue7wt5VR7vD4XPp8bT7xd2evCHTPbyh4DdzSRKrvxuXXckXjrcq9LxxVOlf0u3LqGqymWKrsxNBVXY7S1rKkK7oYImqbg87uT/eg2UDW1XVlPL7t0tHZlw6Wh6Sr+nACqy4O70calRCmYPB3F7cXvbCXW0MzIs76E6vay9R//pKGhgZEnnQGA72AlSiQi3xXtvita3m8mvyuGUpxq6R+JU4f/7LW0ORviVMtvriNRHKf/j5svW7aMe+65J61u4sSJPPnkk0BbsDrc+PSzzz6b3/zmN4wdm8yaFyxYwC233MLs2bO79NplK1/BikZBARQVUEBpuanJc4VS96Tdtz1WWsrqIfWqdki9BqqCouqpeq1tPVWjqDiXqppIapsyccPhhEJBKuU6VEck/dM56Z/OZVP/qKqSNUnVQMYogIMrns+aoek+v5twdydd6A/tYrPSGnNTcVbVknE69RhVS14UV9Uhda9oyceKZoBmgO5C0VygGd0a3p5Nf0PZSPqnc9I/ncum/jlajBqQI1Tz589n/vz5PXpuSUkJFRUVrcGqqqqK4uLiLj/fmHYWmp0dY69Vw42ixge6GUIIIdoZyBgFoJdOR0kkjr5ip/vhurGTrsMOvbayP99HvC58SL1yyKpK62Ol/Tppy5R2daR2aLasp7RbnrwppO/obLtvt2NTCCGyRFYN+euKT3/607z44ovMmTOHNWvW4Ha7uzWUQgghhOgrmYhRasFoyJIdf+5QEE3Njj3EQgiRrQbFLp5nnnmGBx54AICvf/3rxONxvvjFL3L33Xfz05/+dIBbJ4QQYjiTGCWEEMPbgJxDNZCqq5uws2TPXzaNDc1W0kedk/7pnPRP57Kpf7LpHKqBJnFq8JD+6Zz0T+ekfzqXTf1ztBg1KI5QCSGEEEIIIUQ2koRKCCGEEEIIIXpIEiohhBBCCCGE6CFJqIQQQgghhBCihyShEkIIIYQQQogekoRKCCGEEEIIIXpIEiohhBBCCCGE6CFJqIQQQgghhBCihyShEkIIIYQQQogekoRKCCGEEEIIIXpIEiohhBBCCCGE6CFJqIQQQgghhBCihyShEkIIIYQQQogekoRKCCGEEEIIIXpIEiohhBBCCCGE6CFJqIQQ4v+3c/cxVdb/H8dfgGJWP4fowemaOjTBm7HcasbUzJuV3IkoblhT1JxkqegcDmm5pRTe1HTT9p1zJs4bNEVkppiJczNBVFoNQ0xdbgUkJmmFyY3n8/3DXwyCc8Tra14X8nz8dw7X9Tnv82HjtRfnAwAAABZRqAAAAADAIgoVAAAAAFhEoQIAAAAAiyhUAAAAAGARhQoAAAAALKJQAQAAAIBFFCoAAAAAsIhCBQAAAAAWUagAAAAAwCIKFQAAAABYRKECAAAAAIs62T1AazZs2CA/Pz8tXLiwxdfKy8sVHR2tvn37SpJ69uyprVu3Pu4RAQAdFBkFAGjKUYXqjz/+UEZGhg4fPqy5c+e2es2FCxcUExOjlStXPubpAAAdGRkFAGiNo4785efnq3///po9e7bHa0pKSvTDDz8oNjZWM2fO1KVLlx7jhACAjoqMAgC0xlGFavLkyZo3b578/Pw8XtOlSxdNmjRJOTk5euutt/Tuu++qrq7uMU4JAOiIyCgAQGt8jDHmcb9oXl6eMjIymj0XHByszMxMSdLGjRslqdXz6f80adIkrV27VqGhoY98TgBAx0NGAQAehi1/QxUREaGIiAhL9+7YsUPR0dHq3r27JMkYo06d2v42bt78U273Y++QrXK5/k83bvxh9xiOxh55x/54x/5456T98fX1UY8ez9o9hiR7M0oip9oT9sc79sc79sc7J+3PgzLKUUf+2uLcuXPav3+/JOns2bNyu90KDg62eSoAAMgoAOiIHPVf/jzJyspSVVWVkpOT9d577yk1NVW5ubnq0qWLPvnkE/n6trteCAB4QpBRANCx2fI3VHbiKEX7wh55x/54x/5456T9cdKRP7uRU+0H++Md++Md++Odk/bniTvyBwAAAABOQaECAAAAAIsoVAAAAABgEYUKAAAAACyiUAEAAACARRQqAAAAALCIQgUAAAAAFlGoAAAAAMAiChUAAAAAWEShAgAAAACLKFQAAAAAYBGFCgAAAAAsolABAAAAgEUUKgAAAACwiEIFAAAAABZRqAAAAADAIgoVAAAAAFhEoQIAAAAAiyhUAAAAAGARhQoAAAAALKJQAQAAAIBFFCoAAAAAsIhCBQAAAAAWUagAAAAAwCIKFQAAAABYRKECAAAAAIsoVAAAAABgEYUKAAAAACxyVKEqLi5WfHy8YmNjlZiYqPLy8hbX1NXVKSUlRREREYqLi9PVq1dtmBQA0NGQUQCA1jiqUKWkpCg9PV25ubmKiYlRenp6i2t27Nihrl27Ki8vT2lpaVq+fLkNkwIAOhoyCgDQmk52D/C3uro6JScnKzQ0VJIUEhKinTt3trju5MmTSk5OliS99NJLqq6uVkVFhfr06dOm1/H19Xl0Qz8CTpvHidgj79gf79gf75yyP06Zw5PHlVGS8/bCafM4DfvjHfvjHfvjnVP250FzOKZQ+fv7KzY2VpLkdru1adMmTZgwocV1VVVVcrlcjY9dLpd++eWXNodV9+7PPJqBH5EePZ61ewTHY4+8Y3+8Y3+8Y3/a5nFllEROtTfsj3fsj3fsj3ftZX9sKVR5eXnKyMho9lxwcLAyMzNVV1en1NRUNTQ0KCkpqcW9xhj5+Pg0e+zr66iTiwCAdoyMAgA8DFsKVUREhCIiIlo8X1NTo/nz5ysgIED/+c9/1Llz5xbX9OrVS1VVVerbt68k6ddff1VQUNC/PjMAoGMgowAAD8NRvzZLSUlRv379tGHDBvn7+7d6zZgxY5SbmytJOn/+vLp06fJQRykAALCCjAIAtMbHGGPsHkKSSktLFRcXp4EDB6pTp/sfnAUFBWnLli3KyspSVVWVkpOTVVtbqxUrVujChQvy9/dXenq6hg4davP0AIAnGRkFAPDEMYUKAAAAANobRx35AwAAAID2hEIFAAAAABZRqAAAAADAIgoVAAAAAFhEoXKI0tJSDRs2zO4xHKe4uFjx8fGKjY1VYmKiysvL7R7JEQ4dOqTIyEi99tpr2rVrl93jOM6mTZsUFRWlqKgorV271u5xHGvNmjVKTU21ewy0A2RU68goz8gpz8iotmlPGUWhcoC//vpLq1atUn19vd2jOE5KSorS09OVm5urmJgYpaen2z2S7a5fv67169dr9+7dOnjwoPbu3asrV67YPZZjFBQU6Ouvv1ZOTo4OHjyo77//Xl999ZXdYzlOYWGhcnJy7B4D7QAZ5RkZ1TpyyjMyqm3aW0ZRqBxg9erVSkxMtHsMx6mrq1NycrJCQ0MlSSEhIaqsrLR5KvsVFBTo5ZdfVkBAgJ5++mm9/vrrOnr0qN1jOYbL5VJqaqr8/f3VuXNnDRgwQBUVFXaP5Si3bt3S+vXr9fbbb9s9CtoBMqp1ZJRn5JRnZNSDtceMolDZLD8/X3fv3tXEiRPtHsVx/P39FRsbK0lyu93atGmTJkyYYPNU9quqqpLL5Wp8HBQUpOvXr9s4kbM8//zzeuGFFyRJ165dU15ensaMGWPzVM6yYsUKLVmyRN26dbN7FDgcGeUZGeUZOeUZGfVg7TGjOtk9QEeRl5enjIyMZs8FBwfrzz//VGZmpj1DOYin/cnMzFRdXZ1SU1PV0NCgpKQkmyZ0DrfbLR8fn8bHxphmj3Hf5cuXlZSUpGXLlql///52j+MY+/btU+/evRUeHq4DBw7YPQ4cgozyjox6OOTUg5FRrWuvGeVjjDF2D9FR7du3T5s3b9YzzzwjSSorK1NoaKh27dqlZ5991ubpnKGmpkbz589XQECAPv74Y/n7+9s9ku1ycnJ0/vx5ffjhh5KkTz/9VMYYLViwwObJnKO4uFiLFi1SWlqaoqKi7B7HUWbPnq0bN27Iz89Pt2/f1p07dzR58mSlpaXZPRochox6MDKqdeSUd2SUZ+01oyhUDhISEqJLly7ZPYajvPPOO+rRo4c++OAD+fpyQlW6/8e+06dP1/79+9W1a1clJCRo1apVCgsLs3s0R6isrFRcXJzWr1+v8PBwu8dxtAMHDujs2bNavXq13aOgHSCjWiKjWkdOeUZGtV17yiiO/MGxSktLlZ+fr4EDByouLk7S/XPYW7ZssXkye/Xq1UtLlizRzJkzVV9fr/j4eEKqia1bt6q2trbZD+CEhARNnz7dxqkAPGnIKM/IKc/IqCcTn1ABAAAAgEV8Pg0AAAAAFlGoAAAAAMAiChUAAAAAWEShAgAAAACLKFQAAAAAYBGFCgAAAAAsolABAAAAgEUUKgAAAACwiEIFNFFfX69Ro0Zp7ty5bb5nzpw5qq6uVklJiRYtWmT5tf9eR9L/vFZTj3Kt1jSdGwDw7yKnHh45hX+bjzHG2D0E4BRHjhzRgQMHdOHCBe3atUsDBgx44D0hISEqLCxUYGDg//Taj2qdx83T3D/99JPS09P17bffqqGhQWFhYdq2bZtNUwLAk4GcenjkFP5tfEIFNJGVlaXx48crMjJS27dvb/a1/fv3KyoqSjExMZo5c6YqKyu1fPlySVJiYqIOHjyo6OhoSdLSpUv12WefNd67e/duLV68WG63W+np6Zo2bZoiIyMVERGh4uLiZutUVlaqqKioca29e/cqOjpakyZN0pw5c/Tjjz9KkoqKipSQkKCUlBRNnjxZ0dHRKi4ubvGemq7l6Z6ioiJNmzZNycnJiomJ0bRp03T16tUW9//z8T/nbmrZsmV65ZVXVFBQoIKCAi1YsMDKtwQA0AQ5RU7BgQwAY4wxly9fNkOHDjXV1dXmu+++M2FhYaa6utoYY8zFixfNiBEjTEVFhTHGmG3btpn333/fGGPMoEGDzM2bN82ZM2dMVFSUMcaYwsJCEx0d3bh2fHy8OX36tPnmm2/MwoULzb1794wxxmzevNkkJSU1W8cY07hWQUGBmTBhQuPz2dnZJiIiwrjdbnPmzBkzePBgU1paaowxZuvWrebNN99s8b6azuXpnjNnzpjQ0FBz7tw5Y4wxu3fvNnFxcS3ub+1x07mbGjlypNm+fbupra1t67cAAOAFOUVOwZn4hAr4f1lZWRo7dqy6d++usLAwPffcc/r8888lSYWFhRo1apR69+4tSZo1a5ZWrlzpca0RI0aotrZWJSUlunLliqqrqxUeHq7hw4dr8eLF2rNnj9asWaOjR4+qpqbG4zqnTp1SZGRk4zGFKVOm6Pr16/r5558lSX369NHgwYMlSUOGDNHt27cf+D493RMaGqoXX3xRkjR16lRdvHhRv/322wPX82TdunXKz8/X6NGjlZaWplu3blleCwBATpFTcCoKFSDpzp07ys3NVXFxscaNG6dx48bpxo0b2rlzp+rr6+Xn5ycfH5/G6+/evdt41KA1Pj4+io+PV25urrKzsxUfHy8fHx+dPHlSSUlJkqTx48dr+vTpXudyu90tnjPGqKGhQZL01FNPNXtN04Y/ifR0j5+fX4tr/37fTdetr69/4GtIUnh4uLZv364jR46orKxMOTk5bboPANASOUVOwbkoVICkQ4cOKSAgQKdOndKJEyd04sQJHT9+XHfu3NHRo0c1YsQIFRYWqqqqSpK0Z88erVu3TtL9H+Z/B0dTcXFxOnHihL788ktNmTJFknT69GmNHTtWb7zxhoYNG6bjx4/r3r17HtcZPXq0jhw50vjfibKzsxUQEKB+/fo98j0oKytTWVmZpPvn4YcPH65u3bopMDBQFRUVunnzpowxOnz4cLP7Wpv72LFjunbtmowxqqmp0e+//67Q0NBHPjMAdBTkFDkF5+pk9wCAE2RlZWn27NnNfvvVrVs3zZgxQ5mZmcrOzlZKSkrjv6l1uVz66KOPJEkTJ07UjBkzNGPGjGZrulwuDRkyRA0NDerVq5ckKSEhQUuXLlVMTIwaGho0cuRIHTt2TG63u3GdjRs3Nq4xcuRIzZo1S4mJiXK73QoMDNTmzZvl6/vofxfSs2dPbdiwQeXl5QoMDNTatWslSQMHDlRCQoKmTp0ql8ulV199VSUlJY33NZ170KBBkqTi4mKtXLlSNTU1CgoK0rx58xQeHv7IZwaAjoKcIqfgXPzbdAAqKirSqlWr9MUXX9g9CgAALZBTcDKO/AEAAACARXxCBQAAAAAW8QkVAAAAAFhEoQIAAAAAiyhUAAAAAGARhQoAAAAALKJQAQAAAIBFFCoAAAAAsIhCBQAAAAAWUagAAAAAwKL/AkW+QdsIz8eYAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plot_activations(ax, names, s_range=5, y_range=2):\n", " s = np.linspace(-s_range, +s_range, 101)\n", " for name in names.split(','):\n", " phi = eval(name)\n", " ax.plot(s, phi(s), lw=4, alpha=0.5, label=name)\n", " ax.legend(fontsize='x-large')\n", " ax.set_xlabel('Activation input $s$')\n", " ax.set_ylabel('Activation output $\\phi(s)$')\n", " ax.set_xlim(-s_range, +s_range)\n", " ax.set_ylim(-y_range, +y_range)\n", " ax.axhline(-1, c='gray', ls=':')\n", " ax.axhline(+1, c='gray', ls=':')\n", " \n", "_, ax = plt.subplots(1, 2, figsize=(12, 5))\n", "plot_activations(ax[0], 'relu,elu,softplus')\n", "plot_activations(ax[1], 'sigmoid,tanh,softsign')\n", "plt.tight_layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that all activations saturate (at -1 or 0) for $s\\rightarrow -\\infty$, but differ in their behavior when $s\\rightarrow +\\infty$ (linear vs saturate at +1)." ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden", "solution2_first": true }, "source": [ "**DISCUSS:**\n", " - Which activation would you expect to be the fastest to compute?\n", " - Which activations are better suited for a binary classification problem?" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "The `relu` activation is the fastest to compute since it does not involve any transcendental function calls (exp, log, ...).\n", "\n", "The activations that are bounded on both sides only have a narrow range near $s=0$ where they distinguish between different input values, and otherwise are essentially saturated at one of two values. This is desirable for classification, where the aim is to place $s=0$ close to the \"decision boundary\" (by learning a suitable bias).\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Network Layer\n", "\n", "What happens if we replace the vectors $\\mathbf{x}$ and $\\mathbf{w}$ above with matrices?\n", "$$\n", "F(X) = \\phi\\left( X W + \\mathbf{b}\\right)\n", "$$\n", "If $X$ has shape $(N, D)$ and holds $N$ samples of $D$ features, then $W$ must have shape $(D, M)$ so $F(X)$ converts the $D$ input features into $M$ output features for each sample. We say that $F$ represents a linear network **layer** with $D$ input nodes and $M$ output nodes. Note that the bias is now a vector of $M$ bias values, one for each output value.\n", "\n", "We cannot really add a vector $\\mathbf{b}$ to the matrix $X W$ but we are using the \"broadcasting\" convention that this means add the same vector to each row (sample) of $X W$. We also cannot apply $\\phi(s)$ to a matrix, but we are using the \"elementwise\" convention that this means apply $\\phi$ separately to each element of the matrix.\n", "\n", "To connect this matrix version with our earlier vector version, notice that $F(X)$ transforms a single input sample $\\mathbf{x}_i$ (row of $X$) into $M$ different outputs, $f_m(\\mathbf{x}_i)$ each with their own weight vector and bias value:\n", "$$\n", "f_m(\\mathbf{x}_i) = \\phi\\left(\\mathbf{x}_i\\cdot \\mathbf{w}_m + b_m\\right) \\; ,\n", "$$\n", "where $\\mathbf{w}_m$ is the $m$-th column of $W$ and $b_m$ is the $m$-th element of $\\mathbf{b}$.\n", "\n", "The corresponding graph (with $D=8$ and $M=4$) is:\n", "\n", "![NN layer](img/NeuralNetworks/nn_layer.png)\n", "\n", "The `nn_unit` function we defined above already implements a layer if we pass it matrices $X$ and $W$ and a vector $\\mathbf{b}$. For example:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.73105858, 0.5 , 0.81757448],\n", " [0.5 , 0.88079708, 0.5 ]])" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_unit(x=[[1., 0.5], [-1, 1]], w=[[1, -1, 1], [2, 0, 1]], b=[-1, 1, 0], phi=sigmoid)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A layer with $n_{in}$ inputs and $n_{out}$ outputs has a total of $(n_{in} + 1) n_{out}$ parameters. These can add up quickly when building useful networks!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Network Graph\n", "\n", "Finally, we can build a simple **fully connected graph** by stacking layers horizontally, which corresponds to nested calls of each layer's function. For example, with 3 layers computed by $F$, $G$, $H$ stacked (left to right), the overall graph computation is:\n", "$$\n", "N(X) = H\\left(G\\left(F(X)\\right)\\right) \\; ,\n", "$$\n", "with a corresponding graph:\n", "\n", "![NN graph](img/NeuralNetworks/nn_graph.png)\n", "\n", "Nodes between the input (leftmost) and output (rightmost) nodes are known as **hidden nodes**.\n", "\n", "The corresponding code for arbitrary layers is:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "def nn_graph(X, *layers):\n", " for W, b, phi in layers:\n", " X = nn_unit(X, W, b, phi)\n", " return X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For example, here is a three-layer network with the same architecture as the graph above. Note how the output dimension of one layer must match the input dimension of the next layer." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1., 1.])" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_graph([1, 2, 3, 4, 5, 6, 7, 8],\n", " ([\n", " [11, 12, 13, 14],\n", " [21, 22, 23, 24],\n", " [31, 32, 33, 34],\n", " [41, 42, 43, 44],\n", " [51, 52, 53, 54],\n", " [61, 62, 63, 64],\n", " [71, 72, 73, 74],\n", " [81, 82, 83, 84],\n", " ], [1, 2, 3, 4], tanh), # LYR1: n_in=8, n_out=4\n", " ([\n", " [11, 12, 13],\n", " [21, 22, 23],\n", " [31, 32, 33],\n", " [41, 42, 43],\n", " ], [1, 2, 3], relu), # LYR2: n_in=4, n_out=3\n", " ([\n", " [11, 12],\n", " [21, 22],\n", " [31, 32],\n", " ], [1, 2], sigmoid) # LYR3: n_in=3, n_out=4\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The weight and bias values are chosen to make the tensors easier to read, but would not make sense for a real network. As a result, the final output of `[1., 1.]` is not surprising given how the sigmoid activation saturates for input outside a narrow range." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Visual Perspective" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Activation maps](img/NeuralNetworks/activation_maps.png)" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden", "solution2_first": true }, "source": [ "**EXERCISE:** Identify which activation function was used to make each plot above, which shows the building block \n", "$$\n", "f(\\mathbf{x}) = \\phi\\left(\\mathbf{x}\\cdot\\mathbf{w} + b\\right)\n", "$$\n", "for a 2D $\\mathbf{x}$ with the same $\\mathbf{w}$ and $b$ used in each plot. Red and blue indicate positive and negative values, respectively, with zero displayed as white. For calibration, (a) shows a \"linear\" activation which passes its input straight through." ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "- (a) linear\n", "- (b) tanh\n", "- (c) relu\n", "- (d) softsign\n", "- (e) sigmoid\n", "- (f) elu\n", "\n", "To distinguish between (b) and (d), note that both go asymptotically to constant negative and positive values (so sigmoid is ruled out), but the white transition region is narrower for (d).\n", "\n", "To distinguish between (c) and (f), note that (c) goes asymptotically to zero (white) in the top-left corner, while (e) goes asymptotically to a constant negative value (blue).\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden", "solution2_first": true }, "source": [ "**EXERCISE:** Experiment with the following function to determine how the displayed arrow relates to the three model parameters $w_0, w_1, b$:\n", "```\n", "nn_unit_draw2d(w=[0, 2], b=-1, phi=tanh)\n", "```" ] }, { "cell_type": "markdown", "metadata": { "solution2": "hidden" }, "source": [ "The arrow has the direction and magnitude of the 2D vector $\\mathbf{w}$, with its origin at $\\mathbf{x} = -b \\mathbf{w}\\, / \\, |\\mathbf{w}|^2$ where $s = 0$. The line $s=0$ is perpendicular to the arrow." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "scrolled": true, "solution2": "hidden" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAd5ElEQVR4nO2d36s0yVnHv1XVPXPeH7vvbsCIikRFLwwoakBRDGSNZpOYEAJBROKFId55KYpX3vofRPQmF0EQMWs0CUg0KG4SBMULNRKTm+Bmjbsb47vZ931npruqvOiunurqqj7TNd3V3TPPBw5nzpw558z745nvc+r5fp9mWmsNgiAWB5/7CRAE4YeKkyAWChUnQSwUKk6CWChUnASxULK+T77jQ18CYwwAwDgH4wyMV/XMGWvuy/Ksus0YRCbABYcQHDwTyDcZslxACI4sF8hygXwjkGUc9+7l2G458pxhu2HYboBtDmxzjQd3S2wziRtRYisKbPgBW77HU8W3kZdPkBWPIQ5PwA9PwA87sP1j6NcfQu93ULsd9P4AudtD7g9Q+wMO33kMeSghDyVUKY/vC4XiSdHc1kpDSwV5qG7Lg4JSCgCgCw1Vau9tg7nPvd9GBe4nrpP3lV/x3n+rcrqTFl3/R7VRSjf3K1X9p1ZaV//RlYaUCkpXj1P1x1ppSKVRltV9pdQoJeo3hlJylIqj1NWb1AJSZ5C8elM8hxbmTUBnOSAEmHnLqjduvecZB8/Mi0z14sIFAxf17fp+AGCifoxg4Pz418Qz1vv3ZX+e5f7H8sD9BGEzqK3VylIIXRWktgpTm4JsClO1bitp3qrCVFJDSg0pcXxT9ZsGpGIoFYdUAlILlFpAsqowlcihRAYtMmiRAyIHy3Igy8HyqlB5lh2LVYha6as3ngmrSI+FCABMVEXrYhcby1lTiKEiJIhz6G1rtVJgnFfqaW4rDaC6rbQGVwqaM8hSQmQAOIcsZfM9uKwKt/R8/0NmvzaY29V/9N3B/7qxz+4En6/YbmHKhANQAET9sdofgl/Hs3Y3IAsFgENsqhek6iMOpVRVlDi2rzxjUKUGy5m3jQ3dz3NG7S3Ry+1trdXGhm4bfO2trltZX3tr7vO3tyzQ3h5b2057W7e2vvaWNW1tt73lwn7vtLfm/gHtrfs5am+JGKJOa7vtrWllfe1t3dp62lupqrY21N6WknXa21ILlMi97S1EDmRZuL3lvKe95d721ocpNvOe2ltiCk4qTlNg5rZ9v4urnqZ4ferZHAwNUE/3YKitniKonrxXPXlQPZl9v6OevX+xpJ7EmUTPOW31bA6GfOqp+tRTh9VTBdRT96lnHlTP1qGQo5726a2rnr6DIYDUk5ieYae1Eeppj1WGqScWoZ4Aguo5ZKwCkHoSw+gtTqX6TxONerbGKq56umOVM9XTHqu46ql71LM1VnHUsz1WaatnaKxi46onQYxB7ygFqAqUc9YaqzBmfaw0GEczVgE4wNGMVuyxik0JQAQOXAAgzxnQDEYYzOvILnOesvUtss2d1t2t77DdNWMVF3nwDXrqr7PGKvKgmhENB4fKVTNWMSMTnjEoHN1BZtRyGzRaIVwGmhCOitj62NPaAkfldV1Dpr31uobq9tbvGmJRriEEXUPH9tZ1DZk21ucaMu1t85c4gmuIIFxOKs7b2tvmcZ72Nuga0n7X0LG97bqGSsm8rqGmvfW4hkx763UNndLeDnQN+W4TRAyDT2uHqmfIc6tUj+dWqoSeWx703Da3A+pJnltiSkaJjAU9t6aAB3puZYTntkQe9Nwiy8Ke29ZIZRz1pLEKMQaRDqG2err321BiBd7Pk3oSt9FbnHaxnTpWAaZNrPgsfUtLrABkSiDO59ZRihmZAMexin1/6sTKPmcIvaYsJrECHMcqlFghIjnrd05KrJDnlpiOk43vhuHt7ciJlZDnlhIrxIVx9mlt+sTKMjy3fYkV8twSYxBVnEPU8xoTKwZST+IcRluNmSqx4rX0UWKFuEBOLk630CixEk6s+Gx8lFghhnLrKMXGHqsA8yRWiqK9CGyJiRUotBaBUWKFiGG8tjZRYiXsGlpOYmVs1xBxnUQb3w2pEyshzy0lVohLY9RrpSRJrCT13MYlVshzS4xBv7c2cNHrULH5HztuYiVmSzwlVog1MvpVxqZOrIQtfctJrAxdQu1+ntSTACIuZNTcP1diZeASakqsEGvlpFGKGZF07p8hsVKW7tBjeYkVxp1FYJRYISKY7OK5UyVWYrbEp06s0JZ4YgxGMb4b0iRWIjy3iRMrtCWeGINJLzt/rYkVgDy3xPmc7q0NHAy5UGIlbks8qSfhMqlyGkZNrHgsfUtLrISWUJN6EkMYZnw/4dQWCJ/cGlN8daLJOoZ4KIXqnFNCcv9/vLIMGyCKwjbDA/bZ6z7Ljw/kgO10N6Z421KvzcMO4dNascng24CkCgbkAoCEebbmmfhM8D4DPJniieHe2hNdQ2dHygZ6bnu3xGuEPbc87LlFls5zyzNGrS3RIk1bO2JiJW5LfNrESsot8cTlErdUeiT1bB43JLES8NxSYoW4NJIoJzBeYmU5W+LDiRXaEk+MQfxl5ymxMqrnlhIrhEsy5QTGSayEllAvKbESXEJNiRViAINGKS5DDfH254wp3oxVtFJQnDejFc2OYxUOAYnKFK/EsZBLlK0/QlnYRW6/7jDsDq6l/chW3LQ+Nt+RA+Dbm2as4n61OhSev5USInd+RiGb0YqWrG2Kz1UzTpEnGOIBMsVfC+d7a6eKlI25Jf62SFnAc6tpSzwxI0nbWsM5iZU4z23qxMrwLfG3Qa6h62Ny47thtMTK2J7bCRIrMVviST0Jl1mUE4hPrMRsiafECrFGxrnsfMrEyshb4qdIrMRsibch9SSAGZXTMFQ9Q0uol5RYCS2hpsQKMYSzRik2qRIrXOlmrOKylMSK2JikyryJFWLdjLtUOkFiJWZLPCVWiDUyf1ubYkt88sTK8C3xUyRWqLVdN+MvlZ44sWJ+xqITKyNviffdJi6f2ZUTGKaeMVvi0ydWhm+Jp8QK4TJJcU6aWInZEp86sTLylnhKrFwni1BO4LISK6El1JRYIYYw2ijFZarEiuastQhsiYkVHryEAyVWiNOZ1ls7SWIlZks8JVaI9bGYttZwCYmVGM8tJVYIl1mN74bFbomPTKzEbIkHKLFCtFmccgKUWCHPLQEkKM6xEyvBJdQLSqwMtvRRYoXwsEjlNPjU08fiEisDl1BPlVgJPo4KehVMNkqxGTOxArQXgS0xscI2O89QpiJlYqVv5EIsn3RLpefcEp84sZJySzwlVi6XZbe1K02sxGyJp8QK4ZJ2qfQI6jn2lvgpEitxW+IpsUK0WbRyAuN4bimxQuq5RpIX5zUkVoKWPkqsEANYvHICcIpumKVvjsRKaAl16sSKDann+kgySnE5J7Fibi85scJudq1FYHMlVtxRCyVW1sV83toLTqzEbImnxArhsoq21hDruU2fWBnXc0uJletkkcW5/sTK8C3xlFghXBZZnLdBiRVKrFwDsxbnpSZWQkuoUydWiHUzy2mtzVBTvM8QzziHVhqMo3MZB1lKiKx676MEIAIHLgCQ5wxHQzyD/Xq2F44pvsYY4s3dre+wDZvi5aFrhjewovoziQ0gD6pzcsuB1mmtbZAHTr+EA53cLod1trUDPbdGeUORspDnVqoez60ce0t8f6Qs5LlllpqO4RoilsMiivPiEisxW+InSKy4t5eOgob0vGlcp5LP3tbG4m9vRacdbj1eCCilwBiD4hpMaShWqSTjvKWegrMmsSJEpZpCMggOCG4lVjiH0BzCUU/Oc3BRQosSWhTV75xl2RQkjGpKAS0FuFLgGYdWvP4zGPXU0MocDOmqVZU4nuIC0JJD1S9kt7Wvp+Q952htv7wp8fGn93goPM9HAx99uMXb9rnnKy+XRSgnMLHnNnFiJWpL/ASJlTWNVb54p/QWJgAoBnz+bpH4Gc3PYoozBkqsXE5i5eceZ7inAG8Hq4F3Pr4u1QQWVpzTWPrSJ1aCS6gTJ1aA5ZsSJDRevFPgEw/2eGSOth3e9yjHT11ZSwss8HfOoaZ49xIOZqwC1L9H1qMVzVkzVgHnrdEKl1Xh+gYZh8x+/bK3BaE2xXfZZ3e89wOA2G49m4oqVNAUD/Cs3Q1Ia7SilT6OVZSqWtr6e+sTLuMwx++eEhpfulPis/cO+Jbbzmo0RfrWvcAvP9pM8hyWzuKKMwY3seK7bVBKg6N9OKTrgyHmORwyLS8XrHM4ZMYqgnEIrpvDoaa11Rm4ezhUHwzZ7a05HGJZdTCkpagV/3g4xAWDVua9czjEncTKCb+nz5VY6SvKewp4SyHw5W31wvmMZPjIwxtwn5xeAYsszphIWfexuvndTOnqZNZs6gMXtdICmtetLWdOewsoziCVBpMaQmpIriElA+ftxEomdNPeZqxqbQXLIVgJwXMoUUKpAqxJrBRgMoeWEiyX0FJaiRV5fN9qb8v6Y31sb5WuEyvdYmQ5A1xTwi0XQZqS24ryXY82eMeTHF/NJb68lbhRwEcf3uApfZ2FCSy0OGPQSgGct1xD5v7b1NMUr0897bGKq56lZM1YxVbP1sFQRz1FM1Zx1ZNnx7FKVz2PYxVXPe2xiq2eQ8YqwDTqeWpR3tRF+GOHDH/w6l0IDTytF3UkkpxVF+cQ9Wz23vrUU2loFlBPqSFFQD2VXz3NwZBPPbXIwYT0qufxUKirnu2xSls9uWCQMlw8c6jn0KK0eVZdd1EaVl2cPlzPLVAXpmg7Wm311Eq3TAmnq2fblNCop2tKsNSzGqsUXvV0TQm2erqmhFPUU0ImV89zipJos9jijNkS7/0+tXq6hnhbPRu19Kqnqn739KinlIAUXfWUSkAy5VVPM1bxqSfPMuhSetWTZ6L5s7jqyQQHh+qop11ornqODRXl+Cy2OIH1JlZ2mfPXOlNiBQrHscpEiRUqyulYdHHGEC5Qv+dWKQ0u4PXcMtZNrNjtrQljn+y51ZU5wee5hQh7brUUte+267k1iZUUnlsbKsrpWXxxjtXeNo/ztLftsYrb3h7HKp32VuI4VhFdz23Guu2tEnkzVnHbW5bnzVil297ycHtrjVVObW/d26dCRZmOxRdnDPMnVtqmhLkSK0ywlinhHPUsofGlbYHPPKCiTMUqivNU9ez9Ho4poaWe9c/wqqcKq2fluWUd9bQ9t656KseUYKtn5bmVfvV0PLdt9WSD1dP9PTREU5Q3VJSpWUVxxuCqp3u/jc/SZ9TT2Pe86lknVlz19Fn6bPXsWPpq9fRZ+hr1zLqWPqOeXkufYOCKdSx9p6pnCY0v3i3wmXzfKcr7iuGXHuVUlBOzmuKcwtJn1FMzn6WvVk+Ppc+op/JZ+oTf0mfUU7KsY+kz6smyrqWvUU+Ppe+onsyrnsqJQLoq6TMllNB4MT/gs5s9XuPdonzXLsdzuw021xevTM5qihNYT2Jln7cXgdmkTKww7iwC60msyBz4gjrg09mutyhvzDPMQYvAJmZVxRnDHImVkOc2dWIl5Lm1KaHxD2yPT/M9XnOK/L5ieL7Y4LnDBluPUtKmvmm5mOJcVGIl4LlNnVgJeW5ZzlAWCi+KPf5S7vCacIpSMzx/qIrSKOW1Ltmak4spzj7SJ1YCntsFJFaUZvj7/RN8Sj7Gq7pblO8ut/iFsipKZRXkkpaBXQurK87Q750ucyZWQp7b1IkVe6xSao2/2z3GC08e4VWntb0PhvfiBs/JDW4YI5VcCKsrznNIlVgppe5Y+uZIrDCuUJQan989xp9/5w28Ktse4vtgeC+/wTvVtqOUAF3fc25WWZzLT6x0LX2pEyuSAZ97/Q382bcf4hXH2H+fMbxvcwe/yG6wkQAEJk2sEHGssjiBZSdWiqK9CCxlYqVUGn/96nfwJy+/hv9xkixPMYb3372Hd23vIK+LUBearrGyUFZbnDGkSqxEbYk/M7FSlBKf+eb/4hNffwXf3LfnHk9xjg88fR/P37mLG12Nh6RSkyVWiHFYdXEuNbES8txOkVgpGcNf/ddr+Ph/voT/ftI2KjwtOD74pgd49/17uAGDlgry0D4MGjOxQozLqoszhiSJlYDndszEijww/MXXXsYf/8vX8PIbu9ZzfjoT+JXveRPe++ApbBSgClmZJiZKrDSPodZ2VFZfnEtMrIQ8t2MkVg6HAi/8+9fxh1/8N3zj9cetP8eDPMOvvuXN+MB3P4OtBoonBWShJkusENOy+uKMYerESmgJ9TmJlQLAJ//5K/jY3/4TXvq/N1rP8cEmw4d/5Hvxwe//LmyVhjyUUKVt7Rs3sQKQeqbgIoozJrHimuIro3jV5irOW4mVyj4uwSEgYZviq+9dooT9V1kW9u91drGz+hIOtqX9yFbctD7OABSlxKe+8I/42Aufw0vfetj6/DM3G/zGT/4wPvSDb66LsrBObkuI3PkZhQRyAUBCS2dLfK6a01p54hpNKtBpuYjiBKaLlGnOAp5b3Z6DnrklXjKFEse5517v8Mm/eRF/9Kefwjde+VbreT579wYf+dm34td+/IdwoxSK19+APJQne277tsTrYjlb4q+diynOGKZKrMRsibcTK492Er/127+Pf/2Pr7Sew7P37uCjz70NH/7pH8UdLaEPB8jdviexMnxL/G2QaygdV1GcyRMrEVvi7cTKSy+/0irMZ566h998z9vx62//CdxjGnq/q95uSazEbIlf2jVWrpmrKM4+lphY+b63/ADe/57n8dWvfRXv/vmfwYeffzseyEdghx2we3JyYiVmS/xSrrFCXFhxLiexMnxLvJ1Y2eZ38Xu/+zvYyB3y8gmy4jH0G4fBiZWYLfE2pJ7zQleMqTFX1bavon3b5evtS9fbmxE6l66XqjEmdC9d3758vXvp+vbl64+Xru8kVppL1/Pm8vXupevNaMW0vMDx0vVctC9fP+TS9cByLl9/SVyUcgJxlj6fa8iMVnyWPjNakQH1LcvwwUpR2GZ4wN4WtM+sS6tztKYtxhRvW+q1edjBv19IbI5jFRdVsGasYp6teSY+E7yv1aUUy7RcpHIaFezc7yihUv3/sXT9eVX7aJvxiaWcdjt7HK3Ura1U1eGQqvy2SlpxMtk+HJIaVWurRNPelsghWVY7h7ImUlatM8nAsrxZqdkcDJnt8M6W+PbhELNsfJWquthKaG7zjAUVkhifiyzOGJqCc9pbX2sLHAtb1UVrt7dSqk5ixbS3Zanra6x42tu6tfW1t8fWNtDeZr72lgfb22pLPPO2t9zqMIa0t9TajsvFFudY6tk8zqOejWq66qnC6in71NMcDPnUU4TVk+UB9XS2xI+hnu5tYjoutjhjGKqe5n5XPXWfetYHQ3715FHqyYLqKYLqaQ6GfOrJSD0XwUUX56nq2fs9LGV11dP8jCHqqXrU006suOpZ/d7pV0+E1NNR0LZ6sij1NIVI6jk9F12cMbjq6d5vY49VbPVUPeppj1WmVk97rOKqpz1WcdXTHasApJ5zcHGjFJc5EivKWtI8ZWLFfAe+vWnGKvZX8+AlHCixsgauQjlPaW9PHasAVntrj1Va7a09Vmm3t+2xSre9NWOVTnvbGqs47a01Vum0t85Ypd3ecm9768OooT1WsT8mxucqijOGkFPolPbW6xqq21ufa6jd3nZdQ6V9jRVfezvANXRsb7uuIW4Vqs81dBvkGhqXqy/O89RT+9WzY0qw1dM1JeigKcFWz64p4aierinBqGfIlNBNrLTV03cwBJB6pubqi7MPU2Dmtn2/y0me21vVM43n9jb1BMhzuwSupjhDv3e6DFFPr6VPe8YqY6in19Jn1LNr6WN5PtzSZydWAuppIPWcnqspznMYqp4+S98siRWvpS99YiX4OCroXi5+lGKTKrHClW4tArNJmVhhm/AlHFImVigHGsfVKWeKxErQNZQ4sRL03FJiZRVcXXHGMJbnNnViJey5pcTKGrjK4pw6sWJ+xtyJlaDnlhIrq+AqizOGIeoZ8txSYoXUcwhXW5yTJla8lr70iZWgpY8SK6vgaoszhrUlVnymhDkSKzaknqdzVaMUl6kSK+YSDnMnVtjNrrUIbK7ECu25jePqlXOaxErAc5s4sRLy3FJiZR1cfXHGsJ7EyrieW0qspIWKs4dY9Qx6bpMnVvyeW0qsrAMqzkgosUKJlamh4sT4iZXgEurEiZXQEurUiRUijqs+rbWJMcXbX9ec4ioNxtExxctSQmTVex8lABE4cAGAPGc4GuIZ7NfVvXBM8TXGEG/ubn2HbdgUf7w6dhdWVH8msQHkQXVObjnQOq21DfLA6ZdwoJNbUs6zGHNLfPWx33MrVY/nVo69Jb4/Uhby3DJLTcdwDRFUnC1m3RI/RWIlZkv8BIkV9zZxGlScZ0KJFUqsTAUVp8MY6jn2lvjYxErUlvgJEis0VomDinMExvDcUmKF1NOFitPDpSRWgkuoEydWADIlxECjlABDTfGuId6MVYDqthmtaM6asQo4b41WuKwK1zfIOGT266i9LQi1Kb7LPrvjvR8AxHbr2VRUoYKmeIBn7RcoaY1WtNLHsYpSVUtbf299wmUcyBDfhpRzRM7x3Jq2NhQpS7UlnvW6hljQcwuAPLcjQ8XZw9oTKzFb4imxshyoOEcm1nM7RWIlpee2L7FCnts4qDgjWUViJWJL/BSJFQOp5zCoOCeCEiuknudCxXkLa06shJdQp02shJZQE/3QKOUEhiZWfJdwmCOxssucf96ZEitQOI5VKLFyMqScE7GIxEqEa2iKxMrYrqFrgYrzRFaZWInYEk+JleVAxTkh8ydWxvbcxiVWyHMbBxXnACb13E6QWInZEk+JleVAxTkxlFiJ2xJP6knFOZhpLH3TJFaCS6gpsbIKaJQSwVoSK/u8vQjMJmVihXFnERglVk6ClDMR15xYoS3xcVBxRrKKxErUlvjxEyu0JT4OKs6EpE+skOd2zVBxTgwlVuK2xJN6UnHOQqrESnAJdWL1DC2hJvXsh4rzDMay9F16YgUYZumjxEoFjVLOZMmJlaJoLwKbK7GiSt5aBEaJldMg5ZyBVImVsbfExyZWUm6JvySoOEdgqYmVmC3xlFhZDlScM5EksZLUcxtOrNCW+DioOEdiiYmVmC3x0yRWhntuKbFCxTkrUydWwpa+tImV4BJqSqz0QsU5IotLrAxcQj1dYsW/hJoSK/3QKGVklpRYKUt36DFPYkV5xkCUWLkdUs4FMFViJWZL/DSJleFb4m/jGlxDVJwJSZ9YifDcTpBYidkST4kVKs7FQIkV8ty6UHFOwNhb4qvb602sxGyJt7lW9aTiXBijJlY8lr45EiuhJdSUWOmHTmsnYqgh3v6cbYqvTjRZxxAPpVCdc0pI7v+PV5ZhA0RR2GZ4wD573Wf58YEcsJ3uxhRvW+q1edjBf1orNuYcuWuMVwUDcgFAwjxb80x8JnifAf5STfGknBOSLFI20HPbuyVeI+y55WHPLbJ0nluesatobak4F8aYiZW4LfFTJFaGb4mnxAoV5+TMmlgJeG6TJ1ZG3hLvu32JUHEukLESK2veEk+JFSrOJFx9YmXkLfHXklih4lwoYyRWQkuoUydWQkuoKbHSD41SEjHUEG9/zpjizVhFKwXFeTNa0ew4VuEQkKiM5kocC7lECfufuyzsIrdfoxl2B9fSfmQrblofm+/IAfDtTTNWsb+aBy/hUELkzs8oZDNa0ZK1TfG5asYp8gRDPLBuUzwpZ0Imi5SNuSX+tkhZwHOrR94SzwJLy67Jc0vFuXDOSazEeW7HT6zEeG4psULFuRgmSayM7bmNTKzEbIkHKLFCxbkCYhMrMVviKbGyHKg4E5M0sTLylvjYxErMlnhKrFBxroqh6hlaQp08sTJwCTUlVipolDIDqRIrXOlmrOKSMrHCNuFLOCwlsbJESDlnIkViJWZL/BSJlZRb4i8psULFuSKSbImfILESsyU+dWJlia0tFeeMTJ1YMT9j7sRK3JZ4SqxQca6MIeoZsyWeEivLgYpzZiZNrMRsiZ8gsRK3JZ4SK1ScK2RtiZXQEmpKrPRDo5QFMFViRXPWWgQ2V2KF3exai8AosXIapJwLYZrESsyW+PETKzFb4imxQsW5ataTWBnXc3stiRUqzhUQq55Jt8T3JlaGb4mnxAoV5+qhxMrlem6pOBfE2ImV4BLqxImV0BJqSqz0Q8V5IfjU08cciZXhS6jTJ1aCj5uxoGmUsjDGTKwA7UVgcyVW8vxOaxGY+eolJVb6Ri5zQcq5QGbdEj9FYiVmSzwlVqg4LwVKrFxeYoXpU08hCIJICiknQSwUKk6CWChUnASxUKg4CWKhUHESxEKh4iSIhfL/67ApZltnmrgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nn_unit_draw2d(w=[2, 1], b=+1, phi=tanh)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Study the plots below which show the hidden (left) and output (right) node values for a network with 2 + 2 + 1 nodes. Each graph shows the node value as a function of the 2D input value.\n", "\n", "Note how the hidden nodes divide the input space into two halves, with a dividing line determined by their $\\mathbf{w}$ and $b$ values. The output layer then mixes these halves and can therefore \"select\" any of the four quadrants with an appropriate choice of its $\\mathbf{w}$ and $b$." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAb4AAAG+CAYAAADsjWHpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9eZwkR3km/ERkZlX1PT09l+aURhrdQgjdBxJIIIMOZBtswP5szPpgjTHfYXvX2Cxee/H6WPN5/X34AHxgxGKMbWzAMohbxiBZSEJC9y3NfU9Pd0/XlRGxf0RGZmRmZFXW1Z3VHc/vV7/qyquyaibqieeN931eIoQQsLCwsLCwWCWgy30DFhYWFhYWSwlLfBYWFhYWqwqW+CwsLCwsVhUs8VlYWFhYrCpY4rOwsLCwWFWwxGdhYWFhsargttp57e33LNV9WPQZhBDzdhqf61BKUvsIIdHfVP5Ng23UofKZRq9dz4XjUFDXgeNQOJ58royU4HoOvJID16UolSi8EsXEmINyiaBcAsoeUCkJlDyOisswVa6h7DRRpg2USB0lXkPJr6JSPQ6nUYXTWASpV0HqNaBRAz85C16rg9dqYPUGeL0B3vTRmF8Ea/jhg/sMrOGjWW2CNTkE58GzAGsw8CYHZwKCCXDOIZqyyodVOQCA+1HVj9qnb9O3J8EN22/znw7//qnfOGg8z2J4QKh5vKWPS4w/ktxPwrGrxqYcj0SOOUJAKOA4FIQQUAJ4JQeOQ+A4BNQh8FwC1yWolClKHuC5QMkFSp5A2eMouxwT5TrKjo8ybaBM6piZrKB+4iBGqifgNhfhNKogzRpIowZSr0GcWoBo1CEaDYimD9Fsgvu+HG8+g2AMwmcQnIP7XL7mcnypZwDgPgOA8LX+t+Dc+J3px2Yh65iz7rzLuN0qvlUME+npSA5mkjieEAJKaUiOhBBQR5Kk41A4Lg0HZPigclA6DuA6gOsIUCrgUgGXcjiUwyEMDnw4wofDm6DcB2VNUNYE8X2AMYA1IfwmhO/LQRY8OGPgfvwhByMDZ0KSG+cQTMhByQJiC0hPR5LYgGxys7BoB9MYi+9Pk6c+gVWkRxUxEkSEGJAiJQE5EgKHApRCexZwCECJgEs4XMLgEgaHMHgOgcN9UN4EYT7AGQhjcqzx4JkxQAhAcAghAC4nkOpZJzid9BRM5NSO1PKQXjewxLcCkVftmbYrtacGoa721D7HdeTAUs+OEyg/qfRcz5EPl8bVnkdRDmahlRJQ9gRKnkDF46i4PspuNAMtoY4Sq8Lza3CbVTiNKmi9CjRqIPVFiHoNolaTM85A7bF6A7zhy78TSo81eUSETID5HIJFKk+RnlJ7Oumpv3XS60XtWaws5FV7SbQ6LT4pTU9A1RiXJKdIEHCc4EG1ZyonmPLBjRNMInh8gsmaAGuC+E2AMQjmR6pOKTweTB6FAELCExAirtyS5GUmQLPay4NuyNES3wpDp6TX6lxFerHXISFGz7FZpz77DMIxKgTjuiQYhAifXUcqPZdGs1CHsEDtKcWXHohoSuUnQy5yMIILcH1Axh5auEU9K9LjcdJTSBJgFizprV50G+LMuo6MosRDnACC8RUPcVKHGkOcbjDWTCHOksOME0yH++EEUw9xotmQ4c1EiDMkv2SIU3DAMOaSr9W2VuglxNkOlvhWKaim6JJIzzDjCjAMcZI4yYUhTlOYk8pnOTBNIU4mZ6GJECdhvjnEyfxYmFOFOEVGiFNwESM6FeIEJAHqaBfizEOGFhYKvYY4qRbGBPQQZ3yfCnESGim9VIgzY4JJBAPlzVSIU7QIcQpDiBMhwRUzxKlgiW8FoR9qL0pqSag9fSaqPVNiVnvhOkNC7UWhl07Unm9Wez7rSO0JLjpSe3lg1d7qRbchTsAc5kxHXNqFOGWYMxXipCT8Ox3izJhgCm4McYZhzkSIEyIYOzlDnK1Cmd2GOXshR0t8qxCt1F4SbdVekNmZV+05rdReSHpxtUdZsz9qTxFgQu3pSS1W7VksJfKqPRIM1ZTao2a1RztQew5vgggOwnyz2mPMqPaQqfZahzOXW+0BlvhWDLLUXifntlJ7enmDeqYkofQ0tafWH5Jqz3Wy1Z6bofZIhtoTzWZHak8hqfYAGNVeO5Kzam/1YlBqLxlVMR2jqz1KulB7hgkm4SyYYBrUHjOrPdXYJ6/a63dSSy+wxLfC0apuDzCRXrSWkKzbow4FDbbF6vZUJqfrxOr2vBJFuURR8kisbk/V7FVcJmv29Lo9vwrPr8IJFtpV3R4aNYhqFaJRj9ftNX34Kpsz+WgycJ/H6vZYg4Wkp9ftAZLokqRn1Z5FL2gVVWlHnrrao0FCS1LtkQy157RQe6YJJgSPyhZSak90pPZaE93yqz3AEt+qQid1e3qIU71OhjgjZZc3xKmUniHEGaw5JEOcKqnFFOIUhhBnGOZMhjiZWfWFr9uQmiU9C4VeMjnbqT1TsboeWZFji4ImE8ZcoOQSeJ7M5PRcoOwKlHJOMAnzQRsym1M0GkCjIQvWfZnJGWZPq7X1MJtTpIrVgXRReh611wnptTvWb7O/pXPLasGex96L0alLMLPtnbHtgvvY/9T7wPx5bDn/Q3Dcsfh+wbD/qfdDsEVsPu/3cOrEvTi2+2Op6xNagVfehMkNb8T4zKuN93Dy0F1YOP5v2HLe77S93yMv/jFOnfgONp39AVTGz4mFOZk/h92PvAvrT383JtbfEPsss4e/ivmj/4pGdR8IcVAa3YqpDa/D1KYbtXulOPTCnTj68mf0TwDqjmBkYgc27rwdm868PVbC8I1P3IzaqcOZ9/tD//HP8Mqrb4tCnZQDvIH3vfstuOnmW/GjP/qjkvwMIU4VehGJEGdIeDlDnCqpJUWAWlJL3hCnfmwSNsyZjW99+jas3/5qnHvNf45t56yJ+/7xx9CszeKaH/ksvPJEfD/38e//9BPwGwu4+s1/i4PP340n/+2Dqes73ihGJ7dj+4U/hs27bg23M7+KFx76GA69+FU0qscxOrUdZ1z8Tmzc+fqW9/voN96PXZsXcNlNv4qxNafLjcFQ8xun8MS3Poip7XfgUOMS7V6b2PvEP2D/s3fh1OyLIMTB+PRObD3vh7D13DdB1xpP3/dhPP+g/ntB4JZGMbH2TJx+0Vtw+iveAoJI2f3j/7wO1flDmff75nf/GS6+6jYtzMkBXsevv+fNeO3rb8eP/OhbsyeYggMsI8TpG0KcajyJdKiyG9LrBO1I72TTx5cPHsePbNuQeYwlvhYg1MXM9p/FwWd+E7P7/xYz2/9DbP/c4S+iWX0ZG8/6NVCnEm7feNb7QNRrIcCas5g7/C84+vKfwvEmMTJ5cew6p078O07s+zS8ka0d3J3Asd1/js3n/g6I47U8krMaDj77e6ifeg5TG9+Ame1vBwTH4uzDOPTCR7Bw/D5sueBXQSCvI5XcCM649LfDMCfz5zF36F688ODvozr3PHZd+SthUgsAbDnnjTjn8nfA9aT68zyKkREHngds2rIzVsIAVsOffuiX8MKzT+B1N78xlmFmUntgGWqPZai9YKB24tJi/IY7IEKL7kEdD+e/+v347hd+Gs898GGcd+37Yvt3P/YpLBx/Bq9645/C9UbD7a9645/AUa+FQH3xCHY/9r/w+D0fQGlkGuu2XgMAeOJbv42je7+Nsy79eYxO7cDhl76B73/9V/EK4mDjGTeiFaanp/Hwt/8C57/6v4BSVyM+hiNHjmDbBWUcOhZsa1bxvS+9FycPP47tF74Vu674BQjOcHTPd/D4v34Qh178Gi75gQ/BcUqhanS8UVx5x0eCcSTQqJ3EoRe/ge995QM4efQpXHrzB+R3RORbn37hbbjwuv8QhDOl4iuVCDwH2Lh1Z6yEgft1/Pn/+0t48dkn8Lqbb42SWpIZ05wBnAeEZ3ZpSYc4eZjVaXJpMaEVYfUrxHm80cRXDp7AJdPjLY+zxNcGlfGzMbHudZg/+lWMz9yA8tiZAAC/cRSzB/4B4+tuxMjkhbFzSqM7U+pwZPJC7P7+u7Bw7Fsh8XG2iNkDn8Xc4S+COqPoBISOoFnbj5OHPofpzW8xHRD+eWzPJ1BfeBZbzv9NVMbPCkMw42svxfjMpdj7+G/j6Eufwsaz3hkltRCK8bUXxDw512+/HuXRtdj75J3YuPP1mNl6mRzAhKAytg7rt78q05NTqb0DLz+Kz/zZ+3Bo/0sAAIdwbTD6RrUXzkITag9GpRd/AImCdav2Cok1Gy/G1vPegr1P/QM2n/0mTK2/AABQXTiAFx76KLac+8OY2XJF7JzJdeen1OHMlivxzU/ehAPP/jPWbb0G9cUjOPj8F3HB9f8Vm8++PTymOrcHLz96Z1vio24ZtYV9OPDcv2DL2W9qeewz934IJw8/hstv/3Os2XRRuH39jldjw47r8MC//CKevf9PcO7V/xcAyFIE4mBm8yuD11LdbTn7RpTHZvDMv38UO86/BRt2BJ+bEIxOrMOmHZdkenIqtXfg5UfxmY/8WjjOCHj2BJPJNT7iN4NJZdqlxaj2WqzhLZfae/D4Ai5fO4Gd4yMtj7NrfDkwveWtcLxpHNv9F6G0P7bn46DOONZu+bFc1yDEAyHxecb80a/j1Il7sf6MX8TI5Cs7uie3vAHj627E7MHPoVHdm3mc35zF/NFvYnLjzTHSUxibfhUmN7waJ/bfBeYvBjer3XfCk3PrBT8J6pRx8LnPhyUMAIK/23ty/vX//H8wNjaO3//jvwneihfak9OqvaXDrsvfg/LIOjz5b/89HGdPfef34ZUncfYV/2eua1CnBEqjCAhr1rD13Ddj7ZarYseNTu1AdX5f2+uVKmuxbvurcfC5L6E6vz/zuPriUex7+vPYdv5bYqSnsH77tdh81hvw8mOfjsaZhqQn57lX/SyoW8YLj/xdmNQS7MzlyfmJP5Lj7Hf/fznOwgmm8M2enFxLbhmAJ+egMdvwUWccN21c05b0AEt8uUCdUcxs+yk0qi9h4dg9WDz5MKonH8LM9p/OUGocQjD54E0064dwdPfHIHgdY2uvDY8aXXM5tlzwhxibvspwjfaY2fpjcNwJHH35Y2FqcRK1uccBwTC25pJMT86JdVdD8Aaqs4/KbQHzmTw5S5UJTKw7DyePfD/05CQAiANQwuE60gfQczlc6qPkspgn53t+9Q/xW3/wUWzfvB4A4MIvtCdnElbtDQ5uaRznXvufMX/sKex7+nM4uufbOLr7X3Hedb8Ot5QOXQnBwLkvH6yBxbk9eOJbHwTzq9h05hsBAKNT23Dedb+Gytj68DzOfRzbe2+0btcGW8/9Ybilcbz86J2Zxxzf/wCE8LFu27WpfYqzNu58Hbhfw7G99xuvoSfNeOVxrN10IY7t+57cpzKtISCED4LgEf7NYiUM7/qVP8L7f+8vsHnL5uC8jAmmiqyEoc7BeHK268CQB1mEOtvwcffB4zhSb8bKsFrBhjpzYnTNZRhdcwVO7P8MqDOC8bXXY3TKrNL2fP9dqW1eZSvWnyGTaMJt5Y1d3w9BRMiHX/hDzB/5CiY33AxC4nOZZl0mnbjlaOAna/5KI5tix4KQUPSZPDlLI9NYOP5M5CRBgGfu/wSeuf8TqfucXLsJv/XR74Z1e6fv3AWXLMIlsj0JEdx6clqE2LDjNdhw+k14/sE/heuNYfOu241kAgD3fPKm1Lax6TNx0Y2/g/XbzUlkAPD8A3+CxbndOOfqX851T643iu0XvQ3PP/BnOPzSN7Hh9NekjlFqcGRyS+Z1Ric3h8eqOjwg25OzMjaD2cNPhTWxAPD4dz6Ox7/z8dS1J9duwn/9yHdDT84zzjwLZacGlzfkZzBMMGOenJwNpSfnfJDI8qrpcWwdLbe9joIlvg6wdts7sO+JX4HgBNNbfyLzuI273g9KK3IN7+BnwRrHsP6M96A0sr2j95MzKv0fnKSIbWz6CoyuuQzH930ao2suBXXi//hqcFGa/qeOwidOcGyihCHDkzNUhEEJAwCcftHtuPj6nwnW+OT6w9gIMFLx0i4tQfkCAFDBrCenRQznXvMr+M7fvwU+ITj7ql/KPO7SWz4CxxuF31jAC9/7GGoLB3HRa38bE2t3ZZ7zwvf+Ai99/69x+iveERKq4AxCG2eE0NQ4m970KqzZdAn2PfWPWLPxYtBUQpk8nxjGmQJ1Wv/cJj05o+3BPgBnvfJNuOS1PxsuI3gOQaUMVMqeoW6PZU4wU56cgeLrtydnO/QaFj1ab+LCqTHsmugsR8ISXwdwvWmUKltB3YlU8oqO0siOcH957Czsf+rXcei538Xmc38HjjeV+/0OPvNbqJ96Jnw9PvMarNvxc7E1OACY2fZO7H38l3Fsz8ex/vS42vTKMqW3WT8Kr7LR6NLCAqXnldfFZ6IZnpz1xcOojG0I1xgAYHR8Bht3XBwrWJ8ck01mlXOEXkRLgwFJBbeenBYxlEfXY3z6THiVNankFR0TM+eE+9dsuAj3/dOP46EvvQdX/eCnUB6diR0rBMfT9/4B9jzxt9h2/lux64r3hvseuOtnMXvokfD15rPvwAXXfyA6ORgP2y98Ox7/5m9g92OfwukXvyN2/ZEJqeZqCwcwqqk+vW5vUanCidNi52Z5clbnD2F0YmM0JgkwOrEOm8+4OOzAUPKAsYrswOA6PO7S0mqCmYysCDEQT868fp2tYCLHJuc4UG3gjBzreSZY4hswqFPBuu0/g4PP/jcc2/PX2LDzve1PCrBux8+Bs1r42nHNPwJuaS3Wbnkbju35K5w6EZVKUEowOnUxQFycOn4/RqcuMJ4/d/Q+EFrC+DoZho0KatOenKyxgPljz2DrObeFBevRIntOT074oEIOSOURaD05LXqB443gvOt+HQ/e9XN4+t7/gVfc9LvhPsEZHv3mf8GhF+7GGZf8DM669Odj557/6g/Ab0YJJ6XKtPE9SpU12HLuD2H3Y5/CiQMPxvbNbL0ahLo49OLXsXbzZalzCSU49MI3QJ0yZrZdqbaG+5OenH59DicOPYmdr7gjnHACKhM0vydn1gQz5ckZKr7+eXK2Q7dqjwuBew6fRMkh2DZa7squ0Sa3LAEqE+dhdM1VWJy9D7X5J3Of51U2ozy2M3y45fWZ/8iTG38A5fGzcWzvp2LbHW8CazbejJOH7kZt/mkAcbVXnX0Uswe/hrVbb4XrjaWyPpOenLsf+yQ4a2D7hW8OQ5/yOOT25JTZZSoEw6wnp0VfsPa0S7HxjNfj0ItfwYkDD4Xbn77vQzj0wt0456pfSpEeAIytOR1T688PH7oiSw63Dae/BuPTZ2Lvk5+NbS9V1mDb+T+CPY//HU4ckAkpupA7tvd+7Hvq89hx0VvhlcYjF6UMJ5en7vtLcNbArkvfGnpyqvvptAMDYJpgxkuFIHjfPTn7ofZM+O7xefhC4Np1U117FFvFF6CxuBsnD38xtX1s+mq43pqerz+95W1YPPkgju+9E6ed+8HUGkKvIIRg/envwt7HpSuGzl8z238cjeo+7H3iN7Fm0y0YW/tKEAKcOvEwTuy/C+NrL8GmXT+lXwwCHAsnngiL1FnzJGYPfgf7nv4cdr7yJ7B+2yVh6QIA1BaO4vDu78EL6olKHjBbFii7HOvXTWPr1tNiBrlUNOVbcZ4oYciv9sKHyZ7Mqr1CYv7YM3j5sU+ltm/aeTPKo+t6vv6uK34RR3bfg6fv+wNc+YOfxNyRJ7Dnib/FzNZrMbnhIswefjQ8lhAHU+vP7/g9dlz8k3jinv8W20YoxdlXvhenZl/CA3f9PLZf+Das33YVIASO7r0XLz/6aazbdhXOuToe8RGcYfbg94NQpkCzOouDL3wDzz/89zjvqndi445XxdoRLc4fwYGXHoLnAK4rH6NlAc8RmJlZi21bN8cmmNQwwTR6ciqi65MnZzt0q/Z8LlDnHK/dsAZOD8b8lvgC1E89jfqpp1Pby2Nn9YX4vPIGTK6/GXOH78LCsX/FxLrX9HzNJEojW7HmtDswu/8fwm2yPKGMLef/Ok4e+hrmjnwDJw99GQBQGd+Bzee8G9NbXg9KHRBK4bgOKCXgfhVPfPPd6ioojUxjfHonLn3j/8D2828JTKilGTUhwHMPfwHPPfwF43297ta34j2/9F+irs+sCuLLEC5ljVgJgypfEPV6rIQhWb4QljAE5QvqWZUwKLJjTR4rYUiWLCRJLvk67xqgRT7MHnoYs4ceTm2f2nBRX4hvZGILtp3/Vrz86J3Y/8wXwlq9Y3u/jWN7vx071i2N47U/eU/mtbJ+V0fGT8Oms96AA8/+c2y7443g0ls+jL1P/SP2PfU57H3i7wEAEzNn4YLrfw3bzv9BEEK1DusAay7inr95u3pHVMZmMLnuTFz3lj/CzotuDdUeoVIcPvPQF/DMQ+Zx9tpb3o7/+H//RvsJpqkDg+DZai8sZ2jfU6/fak+/ztF6E2MuxfXre/89JiKrAAzAtbdn/6ewWHrkbTSbNKM2dWBQrYeSHRgIpXA9N92BwXNQrnipDgylEsX4uJvqwFApCUyONI0GueONE8YODKRegzh5It2Bod5AY2Ex1YGB+wyNU/VUBwbBBfyqH8vw1Ov2TKSXRYSDCnPe5keTrJ/6jYM9Xcui/1BraldcUMbJhewf7qlxiu8+2YxtSxpR64otIr20GbXjyLHqJszelUtLpUzDhBbPlWbUJU9gcsRHyWEouz7Kjh9OMMfZbFi+4DYX4yUMC3NB+UIdotGAd/ENqN/7JbBaPTJ/10oYWNNPqT19mQHIX7fXjRn1gs/wz/uP4Yb1UzhtJH/Zwll33mXcbtf4Vhh67cAQtTiJd2AIG80mOzA45g4MtIsODNaT02Lo0Cbc1qqvXrIDAyBJM9l2KHRpSbQdUkktxrZDGZ6cJFR5CZcWiL55crZCN6Tnc4GvHzqBC6fGOiK9VrDENyTIq/Yyz9ddW7RGs8rBRW+AqSe0hAMw0WSWkGg2qie0RIktGYNRZHdgsJ6cFsOOPE1mgewSBnW8HuKMGs1GCS2xpJa8E8zEWIu5tHDRN0/OfluWHa03MV1yccFkZ7V6rWDX+FYQOlF7yddJT87cai/Dk9OlwjgYW3ZgsJ6cFkOEdpPOVmpP/Z1Ue7HJZg5PToegY7Vn8uTEADw5+6EM55s+No2UsGmk1PW1TLCKbwjQi9pTik4/p5XaA2BWe9Ss9lQJQ1rt8Qy116IDQ9M3qr1u7MkUkgXrgFV7Fq2Rt9FsEq1O613twaz2HLPaM3ZgyFJ7PgvCnP3z5OwHZhs+vrD/GE75rG/XVLDEV3B0m9CSPFdPaFHH6Akt1JEd1mkiocVxqDSjduMJLV6Jai4tUUJLyeOo6F2fVcdnXou6PicSWtCoQVSrQVJLlNDCmz58ldGZMqNm0jNQS2hhDaYpPxEjPZXUYkpoScKSnkU79Kr2lCdnWu3BqPZIhtqjJGs5wdCBIVnCoNuTBet7JrWne3IqdKP28pKi4AI+F/jm4VlctnYCY66T67xOYIlvhSFfiDNKaomFOBOenGGI0xTm1FxakiHOpGWS7gpvPTktiow8aq8T0iMkytxU+3RPTkLlWCJUjS0aH1sugesCJZeg5MpsTmlELVDKMcGkzRpoQ04uSaMG0WgAjYYca80meGBILZcLokSyUP0xFiW8AOF2+Xd+95ZOSA8AnppfxFTJxa4uLcnawa7xFRi9hjj1Y3W1p/YnQ5xJT85kUospxBl3aslwaQmazLbtwGA9OS2GFK2SWsLXbUOcMsyZCnG2c2lpNcE0JY8ZPTkRZm8CvXly9gohBM6fHAUTomtnlnawim8FgNI4ybVCW7Wnwp051V47T86Bqj3ryWnRR/Rb7YXbkiFOzZNTD3GqCWcyxJnHk1MPcWZ2YIiVMCSazMYcW3rz5OyWFAUXqDKGL+w/Bl8IeDkz1ruBJb6CYtBqTy9mV8+6J6ephMGk9tp7cibVXnYHBuvJaTGMyFPC0IoQdbVHSWu1J5/Nnpy51Z7Jk1PZlaE3T85eIITAvUfncNpIGaUBkh5giW/o0U7ttSxYN6g9tfaQX+3BrPZiNXs+eunAYD05LQaNbjM5O7lGsmDdpPZIhtpzArXnGgrWsyaY2WrP4MkpRKba68STsxe19/xCDXNNhkvWjOc6pxdY4hti0FazSI3klDVZOLACT07HdSTRuRSO44CqDE79oXlyeh5FuUxRLknbpEoJKHsCFY+j4vopyyS947OyTKL1Kkh9cWg8Oa3as2iFrtUeNau9cH0vofaiCAvvYIKZU+0lsze79OTsFWvLLm7YMAW3D5OQdrDEV0B0G+ZMhjhTfyfq9uT+/C4tyRBnGOpMhDizimhV6MUU4gxVXs4Qp0pqSYY4u3VpsVi9yKv2UiVEeUjPYE+mPDnV8kHSk1NvMqt7clY8ZvTkNE0wSbMmS4WajZgnp1BZnAH5hZ6cSv0J3pEnp46u1Z4QeHLuFCZcB9OlZGf7wcAS35BimD05TSFO68lpsRzolvTyXGfZPTkNIU6V1JL05NQ7rXfjydmLS8vzC1U8PVeN5SEMGpb4CoZ+q71CenJmqD3ryWkxLMij9oD+eXI6FLkmmISzeGTF5NLSgyenaZ8JedVelTHcf3Qe162f6qm/XqewxDeEGHpPzgy1Zz05LYYRS+HJ6dL8E8zMDgzJEgbdpSUYFsnyBbmtdTlDL2rvYLWBXRMjWFdemhCngi1gLxCWsgMDMEBPTt7GkzND7VlPTosiIs/aXrivhdoDkiUM+Tw5VVJLuwmmrvaI3wwmlWm1Z/LklJmeuiVZmgB19COppck5Th+t4PTRSs/X6hRW8RUEvXpymprMqmOK5smZbDJrPTkthhWmcZvlySmXFjr35AxLGHKYvrfqwJCp9hJZnToGpfZ8LvC5vUcx2/C7Or9XWOIbUgy1J6chxGk9OS2WGrkSWxLEFjvFYAHYypNTjbul8uQUjXrKk1NlcaY8OWNZm/EM6k7dW/Kowe/PLmDac7GmtDxBR0t8BUA/XFr0c5bNkzNVRJvhydlhiNN6closB3IvMbTx5FTHUGL25CQkp0tLxgQzGeKEtsYnwnHG43V7GZ6cvYY485DeXNPHU3OLuHJmsu2xg1MuP14AACAASURBVIIlviFCK5eWjtWe9eS0WMXo2aklT4gzCF3GJpwGT07HMXty5ndpaZpdWvTyBb3LgiHEGStkR+tszuiY7sKcIw7FjRunB9JuKC9scssyY8V4cqY6MGR7cnaq9hSsJ6fFUsE0/oxcmdOTM6n2KMlWew7tkydnhtrL68mZRD/U3v5qHSVKsXGJsziTsIpvSNBtBwbAenJatWehYxBqT4epYD3LkzOp9sLWQ33x5Myn9mQ5Q7ba61dSi88Fvn3kJBqs+/KHfsES3xAgjyenUnvWk9N6clr0H+aC9WBfC7WnzlVEF76mWWov7clJSfYEk3A5uSQa4QnmS8LLUnsZnpytw5q9q73HTp7C2pKHzaPltscOGpb4lhH9dGkxhTiL6snZSYjTenJa9BP9Unsh6Wl1eYChA4NDo+WDwAQi9OR0056cnitQcnnoyVmivszmNEwwabMO2lgEaTRkqZCeyVlvyMhK0497cgbr6ElPzpAYB6T2uBDYs1jDFTMTfakB7BV2ja/gWJGenIYQp/XktCgKOilYjx2XIL3QKCK4HCEIlxjkJFNOKpXyozS+hu6QYMy1mGCS0A1JC3GqMKchxAnEQ5mtiK2TlkLtQADctnkGhJBCEJ9VfMuEperAUERPzmy1Zz05LYqPLLWXPo6EpElptJ4nCVCGOAnVQ5wJT85gjHlo5vfkDEKcqkYvHeLksXIGICDAFkNAJI7tFEdqDXz54InM72k5YImvwOiH2iuiJ2eW2rOenBbLjW7VHslQe9EEM1J70ViLJpaR2hNmtadPMAPSM3py6kktLdSeCXlaD5nOablfCNx/bB6njy29LVkrWOJbBqwYT06DZVIeT85spWc9OS0Gh4Fkc2ZcUi9hMKm9sIA9ofYoFVEyC2FwTZ6c+ljLUHvQWg0l1Z4QUWbnoFsPvXyqhqbg2DUx0vG5g4QlviXGoDw5qTM8npypTE7ryWlRQLTM5NSTycJJZVrt0ZDsEmrPiResR2qPhw/HNMFMqD091GkqYcij9jr15+wEJYfi6nVTS9prLw9sckvBkdelRf/benJaQrPoHC0jLrori/pbrZEbPDkdJ1pDd4Oxpntyug4J/DjjnpylYIJZCkqFyrwaTTCbNdBGNfLkVN3V/Wbck1MfayzIola+nGE2ZzzCApFWf72u7R2vN7GxUkr12SN0+RNcrOJbQgzUk9MQ4rSenFbtWeRDy7U9Y4izdUKLJMSoO4Me4lR1e54TD3Gq5YQwxImEJ2fgyxl2WA/GW5S9KaTC84NGs1qIE0CihEH35+yfwlNocI67Dx5ftu4L7WCJr4CwnpzWpcWieJBWY/EQJ9GGKElOPEN7sniIUy9f0EkvLBky2QCKoHzBb0rCS7q0KALUklr0EKe5bEEjPJEd7uwGj82ewuaRMmYyrMl6Xm/tEZb4lgiD9uTMUnt99+Q0WCZZT06LoqPbH1rjuE1GX0hEgFH5gm5PllZ7KqFFre05wTiLqT1VwsAZaKba881qL1mInixYD9ALwWWdW2ccT84t4lXT4y3PX07ys8RXMHTryZml9oroyZmp9qwnp8UyIN/kM3mO2UYwvpSgzKjbqz0nmdQSqD0qWNSBwaT2hDCqPXB9La+12us3yg7Fm7bMYMJrn0KiJuRLDUt8BUIvnpyEkKHx5JRZnNaT02K4QQlAoNmTGdQedSLFR4mu9mBUe3FHpGYwuZQTzVDtpdxZREB08RKGvGov7eiSDHvmT2yZb/p4dHYhF+npWGoCtFmdS4Cl8OQ0hTitJ6eFRedQv7/RWp3aLhlMjjEEWZ2ILSmo7upURVMCQwjPJfA8aN3VpSdn2WVycuk0NU/OGjy/Cq9ZhdusykzOZk2WCjUbEM064DfBGw2IZlM+Al/O0JNTH18J0osntvS3U8LDJxYw2kOfvW7Jr1MFaxVfQdCrS4sxxFlUT05TiNN6closA/KaRrS+RjxTWg9xUir/jkKbyaJ1Q90e140hmiA8aD3E5Zpe6NbCufxbRJNJKCJTE8yMDgwpolBVDT2ovZMNH3sW67hwaqyzL7APSOUytFGQVvENGEvlySn3D9CTM0PtEb8J0bEnp1ntrVZPTt7KKHEFomjFzDqSak/KPRHvq6epPb3VULrfXuTRGW87JGLlC5RwUPAwqSUW4uQsTCBTPfegTS7DpJaEouulxVC32F+t48I1Yyg7xddTlvgKgGXpwLCMnpydqj3rybmy0I7oC0uMLWr31LNSe0RzZUmqPTXe5BiL1vco4XG1l/LkTKi9kPSkGTWAsHZP/h1f20sXqPcvzCmEwHlTY6myiKKi+NQ8xFhKT85MtVdIT84MtWc9OS0giVF/LDeyiFiu+cm/Y+OOyIllGGkJSxkiteeQ4BE4IrmEwYEca3r3BVnC4INwP1R74DyaZHZoRj2oDM97Dp/E/mq9UB0YWsES3zKjE7WXfJ2rA4NjVnsq/GK2J0urPcoz1J6ySzLYk3XagQFAR2rPYnVgqUgwuSRk+hHXw5zqGOXUklR7YXhTC3NSvX6PcC2bM1hHFwxEBHV7aqyF4y0aY6HaY1HdXtKpxZTJ2U+o36Jj9SYO1urYUC4N5H0GAUt8A0Knas+0fqfX6IUlDMGzKl0In7XyhdCIOnjEzKi1EgZVvlBKljCo8gXESxhU+QIatVgJg16+YCphiIyoeUSEQfmCYPESBiBSe3oJQ7KcAbAlDKsR3RJgzz/+pvFsMI3QG82GERaaLlinJrWnTTBbqT0IEZpRI0PttUpS6bQrQzs8fGIBF60Zh7vMbiydwK7xDQDL4cmZFeLsyZOTD8aTE0iHOK0np0Un0Mmv32uCauxE1xbh9rCEwaEgiLqup4zeQzNqWcJQ9gQ8V6DscpRcOcksUT/qdsKq8FgNXrMK2qzL8oWGfECZUTezzajztB4aSJiTAGvKLs6ZGO3tOksMS3zLiL56cppCnEX15MzowGBdWiy6gSLBgSXFEK1OVvPrTBesm82oVYhTJrQEvpwqoSVY13OED4c1w4xO8CiDMwxzqn57iYJ1IMrkbEdk/Q57VhnHZTOTA7v+oGBDnX3GcnlyZqm9InpyZqk9wHpyWnSPXtYATVG6pJOS/pKEz3rtXtqMmpBoXc8hkCFOEh9r0pZMPsf8OMPaPYNbi1a7p2dyAlE2p45uCanV79bRehNf2Hc09r0vlwVZp7CKb5nQb0/OztXeMnpyWrVnsQwQXPTnRzmUeoFdWTjZ1MguZk8Wbz3UNrKS6K4eEp5mRh1f0zOv7Q1afT1yYh4XTo0blXbW91wURWiJryDIo/b07uvquV0HBvmcZU+WVntuhtojGWpPrjv0pwMD1xfgrdqz6AJciJ5CnqkfbBIvZtfDnAR6+FOr2zM4tUTd1VUpAzevoyfVHuehygszOVX3BUMXhiT6kcRCKE1d53i9icO1Jm7YMN3htYqhBi3x9RF5w5zJEEqa9KIBptScSmqhjrQno4Gyo64TqjxpQp3O5CyXKEoeQblEUC7Jrs+VkkDFZagEHZ/LTjNaaG+qjs8yk9NpLIIE2ZyiWoVo1MFr9ciIuunDV9mcKTNqBu5zaTrN5TNrsJD0FAHqmZytSG8lqr2izIIHjaX60euV/EwwhTn1F8lEsqQZNQ3MqKlmRk2hwpxB6yHl0pIgv1jtnpbJqXdhUNCL1geJMdfBjZumhyqTU4clviXGoDw5M0OcBfTkVLCenKsL/SD4vOSZlfCSN9ypsjVj700ABJZlckIaRFEcPZtTXz8HSq6A50gz6pLDUHJkNmeJNEMzatevwfFroM0aqF8HadbDJQU0/SibU7cpS3RhyGo/pD6z/twKhBLjcfpv1VzDR51zbKwMT91eEja5pU9YMZ6cItuT09SBoRtPzn51YLBhztWHpPNPO/Ra9K4K1FUdHwFC+RcVs0fre/FGs1FiCyXpgnWV1BIWrJsyOZkftRjS1vZihtS9fD7DJKDdxOB7J+axb7EeTMyjxzDBKr4lhPXktJ6cFv2DTn6tfqw7DX3msd1Sw5ci7tQSEaCWzRms61Giyhg4HMgQpwpzQs/mTDaa1cKZWWt7YSeGWHJLnBiz1JzxO8g4dq7hY8+pGq7asdFwTufkN+iQbBYs8fUBhfDkzFB7HXly8iXy5MxQe+E2q/YsOoT6P5UnjNltdqdqNksQFbLrlmV6JqckP5nN6ZD0kkKo9jiLvDlDApTOLBBarZ5yatE/h9DIMPb5OiOTrO9C367e48mTp3DeVP86MCyXUrTEt0QYuCdnhtoroidnp2pvpWO1JLfkRS9JMH0rWdCheXMmw5zBbq2MId5zj8ZMqXmk+lQmpwhaD6mee3oXhjCDMypjSIU5W/zfUdmYuZQeaa0I1Xd66cwkBIb//+twBWYLiMJ4crp0aDw5FelZT04LE9pFD/Kcn0Sna33R2ru2UZEe1V6GYU4ShjcpiXdhcEnk1BL23BNK7fma2gt8OVVBuh7mTLQf6hb6b02neHJ2AfOMoeQ66RKqIcvutIqvBxTJkzMrxGk9OYuPYelhloWlbEXTivxMoTl9u77Wl0cZ6vvD5QcV5lTjjkQm1Cqj03MBzxUoOTKj06MMJerDIz5c4sPlTbisAcoaIH7waDZkFwbfB1T2tN5wViW4QK3nZZcsmOrudDWn/s4TeVKoMYYHj83jh7aP5Pq+ig5LfEsA68kZd2nRYV1ahh+DJO5OSDWLzPTtyUQXGQpsl3kdPCNSdGqDsiQj4bqetCWT26PaPdllPe6KJDM6uVR7ggdqj6fVHhCGNjtfv4tI0ER+0WckRsJUePzEKZwxOYLxcj7KKHr43hJflyiaJ2eW2mvryZlSe4P35MxSe9alxSKJLFLNGn9LkeQSruXFElxIsC2ezam8OSkCU2rOQAUPQ52K9MKkFpXYoiW1dFS2QElq3a8V+anvQT82Cc4YDizWcf2m/C4tRVd/lvgGDOvJaT0524EXfHbcLyQdi3pBkhCTRGj6cc9SfXKb2aha2pJpO/WMayJVoMriVJZlSUNqVcJAwcNGsyQkO1XCoGVyGj5fp6pa/t5w7Xswk1/4HQW/MSalRh0Ht+3Y0FZ9L1dpQjewxLfEGJQnZ5baK6Inp1V7qxO9Enwr4lTE0IoA25Ff2/cPQ5wkDHMSLcxJSTzMSUk8zElC4pPhTSK4JMDwocZQwnw6J+kRQmUtn6b64govTn769xNeI/Ed+1zgS7uP4PVbZlBy2q2JDk+upCW+LlBET05viDw5AaxKT06L3mAizuQY00nCNE5N4c0863zh+yUvmajf051a9DAnJRxUcBCI0K1FhjLlg2jkF67vIb/NmOlYUzmD+pxJAiQZ5QzPzZ2C6xCUPSfX95MXy70GaIlvQFhyT05DiLOonpxZpLZaXVqGPatzuUAIyVSRlJL49xpE4fQfd04ByvWsT4BykSryCpWhVsOnsjlVOZ9Oei6NGs46NIiugGlJLVH7Id0cQrUfylWjF8RWBTigfQbwgNhoQGxQk+x4KJJQJ04+YWmV9pUJgUdPLODVp033fc1uudcALfF1iKJ6cppDnC08OQ0hTpXUIqwn55JiuWe/RUarH0jThEGNM/V/UI0nIQQIIUbFB7RXfbEop1bQTmPF6yJybQmL1UXUhSH05uQgSuXJN4/q89r8X4jW7rTXVITKLgTXwp7BdfXPlzxeTbLlPnkPNcaxdXwEp42NLGnJylLAEt8AsCyenJlqr3ienCasVrVn0RrtJgWpsGVirU8nwCT5CS5Sqq8TEAKAxPNeSBDiJEQKRwoBQoQkOqiQpgiTWMJntE4OCcsNGEPwgUBU93VKQBAPaxJKQvUHRAowvF5S1mrlDCT4rsqug+u2zHT8vQwDLPF1gCJ7cmarveJ5clq1F8cwZcMtJ0zjLCs5w0SAJvLLghAibDRrvBdtF6Fx5acIMHwEhBeSn4q7tgtxUwIiCMBJZBIWJr9xCATkp44NySz+/0lwAULi312SCGWRYnSNfQtVfO/wSdy+c1PrexxSWOLrM5bLkzNL7VlPzuKj19Y5KwF5MixbOZVEx8TT9HUCzCI/LgQkD3BwSuU6n5bLwYWAA13WIbxmuIShsjoBmcASPMceQRanIkF5w1rIM/wyqByPiW3yt0KpMo38lFk14ut6SQKMgcd/m+RnoSEhPnJsDrvWjptrPFYALPHlRNE9OUsehsaTE4gI0HpyWgCdk3/SfUVHlLmopfQHZNex8sthyEw1lQckFJ9GekheS6TvW6h7oZLQBBcAFVLZUQqSXN8DB3h0jrzn1oktRqilFTg4sljHyQbD2dOTK25tT8ESXw4MgyenMcRZUE9OG+KMg/s21JlEuzU3liCR2PGMJYgxIBVIkmNMBJEVhKJIrfUp1ce5CIkMSFFWFNJEFPbU6/dCwiMytBkGK1W5grxpgBAIIse0KgokjhO+Z/gpKJHKLogIhabV+tocgrBmcH5YEiEEiCZh1bGxz+NQ0JILcAGv5OL6HRtQqnjZ/wBDDkt8fcRyenKaQ5zWk3MYsFrKGTr13cx1TaPtFokpSEpIGB4ME1uEAOcwkp9aDpNG0NpSXIt70od3SHgEIEnKVIwZ/xCJhUIKEC4VqYqhcvnbIFRSS6D+CHXAfSbJToiYuhNK8an3UbV7TrwmT6lfQikWmY+xSgkz49lm1FkYprVqS3xtMCyenJ2rveXz5GyH1aT2VhP6SfBqbJmSW0wkqMhPgmrrf9F1IhLlEIIAui7iAnBIwIJ9Cv8pkqMauQXhTiKovD8hAtM0BuI68j5IcB9EfgYa/IqHnzv83dHGHucATRCeCgUH5xDHwUN7j8JzHFy1dV36ftsQW5JQiwxLfH3C8ntyZqg968lZePAhmikvN6hax8oi0YyvUuVyRAQoN3AiZCE6iZc3CC7AiYiRYwgRj1hmQQhAKGJFRKSCUIgwxCnVXizcGRCIeld1BwLBOqKyJEsQoFKqhBsmBYYJvE6MhFLUhMBzs6fw9ovOAHXTJCZ4nwi/ALDE1wK9LOwutSdnltqznpzFhy1gbw19maDVJIFSmrIsS363IQGq0Kch5ImkJ6UQkmcEMZKd6hsbvgYJFKN6rWWdEipfq/U9x5HtiFS403EAnwJEZpaGVwkr5gUEJykCJNAmA9qx4elwjIpNTwB6/Ogczlw3hbGRcvpDor2is6HOFY4ienKWPD40npxW7cWxWtb4OoE+6czrV6mTYpIEla1XvIQhUH4yVySwIJP7JNEJICC15E9+WI2QsV3ukwQoQAKVR8GJA0oIOHVAqSvbElEXcASEkEknxHHkdQPyJACERm7SriwgQAdR6YP6nhxJcjEyVNuBGAESR31/FOduWisn3ga1lwck9S0VF5b4ekRRPDmNIc6CenLm3b9awHzW/qAhRbetiPL8z4jV72n/7wghYJxpx8l7oCKYiKpQIFU/1RH5ARyEE3DCADhynZoTCAIIIgmRBSURnItA7ZHAV1oSHeNEZoYKiqAZERhxQam0K+OOBwgB4ZbAARCi6nYdCNaU5McD705XTjDVa8IFIHiUmRkYWotgLTCm6jTSy1Rjwfid9zm8kRFMVEo5vvkEknWIQwBLfBkYOk9OU4jTenJaLDMG1WuQUmL8MScZ4U5dEarJplzTMys/QCMLIcBFUE4QqkAhw5kistrkQkZLlTeLEJIIuZBKjxMHnDrg3JEkRx3Aif8EEwDCFZKcgzU8wpg0pJYpm9J3U5FNQIRGwtOVHbIjC9xneOH4PGa4wKS3OihhdXzKAaFYnpxmtWc9OYuPYVobKQJIUGdnAk1mt6hxqXVn4JxHSTJhQkuc/JRbiwjCnpQLCEoiYhQa2QmAKfLjkfpjhIAJAg4HHEwjPzdQfm7SMRMgFERIlQnqyvU/6gCcyUQYwWOEHBKh/DDRddT3I+LkJzfFv6Pnj86hvM3FaesTHdZXcAjeEp8Bw+nJmaX2rCdn0WHX+PIhKmHInijwBJWERJggQK7cTYJrJslPcBGGNoVAmMYpIILQpgjW7yRRCgEZ4hQCjBOwoGUQFxRMUDhBuJMQDkoZAA8O84J7DD8gwJqA4CDcBbgPIZygXZEDQmXbIqKbXGuhzxAiTXrRFyRSxRiPHDiGW26YAplduQXrSVji6xJF8+TMUntF9OS0ai8Om9WZDX3ctJsgEBIPf+rKMIsABRWJAnZJfoILMKhOCkHypJD/1x0qwnU9wUSg+jSlJwi4IPA5hUsomHAk6UGAEvk3KORaXwAqbxiCUPk3D8iOMan4BIdwWEh0yVAn0b8bfV8SBjJ805UXYXxiDKxcWTXRB0t8CQytJ6czPJ6cSaxmtQfYUGerSEpHHchTa3tabWmgq3QClN6dQdgzQX5CSELkXJIcFYEKRLAOyIVc5wOJ+shy+eBcJbgQ+EGCiy+c0NEFQekEo24s45Nw2a+PQya3EMEhgoSYcK0v6NQudFVnCmu2Ij/t2O8+uxsXbN8E4riAV8ouzV9hUQlLfBqG25MzHeK0npzDgVXfnSHZiaANkp0cdBETGlRreaGEEgjGgmzP4BjO5LgTkY0ZFRSEE+mc0mSgDg0nenAdAByccTDIBBnmczDmwPcFKAGavu72EhTICwKXlOA5FJxQ+MSBRxwwx5XqTsjxSgSDw5qggoG4PqjwASFAuRbeDMhQfkBzI1u1TQTHZH5RAA6fmMPDB07g4osvgChXwCfWyKzRVQBLfF2gkJ6chhCn9eQcDqx2xdcKpjGWNVGgJJ3pmfTnlBcIQvSgUgEGio8jUn+Ccwiq1vLk5A+Ehmt/qqRBWvbJ9T2fBR2Fwr/l+r0vKAh3oC8/ClAQKuAIHwKSdAVxQASDSxoQwpXZpJxJT07BAxUYqTv9b53kdHI0fqfB/odeeAoXn7sLTmUMgroQXiVeRjKEZQp5YYkvgPXktJ6cy4Uir/F105m8n8gzKVDjLkmISX9Oeazekohnkh8XAoQL8GCBjxNpCaYK2wmTpRBMCDBGQKl89oMxSklAfKBoMu03hCLs0EfBIAgBdyio4KCCgQqGoPgARHBQzqJefqo2UUTNbIlB7cm/DWt+Gpq+j0PzVVx/5aXgngfhuOBepe13vVJgia9DFNaTM0PtWU9Oi15QRFJOknFWP75Yh4YEAaonAhm+18kvyuqUpKKyOmViF5drco4sZBdcgLHAaUx7lpNaSYA+pzGrTEFk8TyBgAsGCgZKuHwIWfZAhQxrEkdIohUcRLDwb3nvGuGZFB+Qqfqox/HWO94IQgh8AK7jwi/l78iQap47ZLDEh8GrvaXw5MxSe9aTs/iwoc78pUJANhnr3RWS19U7NMTCn8Frnfw4ly2BeJDxyYWAwwFOZEILEQBhHA6hQZhTqj3GBJo+iXr1Bb8DnqOMtWVRu0MJeEBqUnE6cKS/C2hQ60fAQVW3dqX+hFR88rWu8sykRzJIr1at4p77HsQP3HCtVs7hwXeziS/VXmnIYYmvA7SzX2pZsG5Qe4SQDtUeWnZgOHJgH+aOHsIFO06Dx/UShvxqL3yY7Mms2rMYEDolfxNRJtsR6dclQfgSSKg/CulrqdufcQFBg8mfqt8TJFB9IlR9TARZnwxgVMBnMhxKiSQ/GpCfCnXKEKesIBREGk1TIuAQDiY4KHFAweHC13r6RU1tqWBxlQeEpBh+7gwS1PHIC8/AHR2HXxoNt3Hqoum1V3xZZDpssMSXE6a6PV3thX6AWkmDXsKgyheoS2MlDLGHVsKgyhfKJRIrYQjLFxIlDCcP7cX/8XO/gLn5BezYvBG/8MM3402XnQ/KmmH5gqjXYyUMyfKFsIQhKF9QzyqzTZEda/JYCUOyZCFJcsnXedcAVwtsAXt21CULWUQZZnUmSFBo5BbvzxeRX2y9L7gGD5JbQtWn1vgC1ce56ugOMEZBKeAH3YVooPwaPgEgrdSEIzM9efB5HSLAgjCnExAdI7L0QRZCiJDMKBhUV/jwWdsvX7dWfL7v4/tPv4hb3/gGNJyI6Bj1Yq97wTCEQVc98a0UT85Tc7OYm18AALy8/xD+04fvxJ9snMF7brkOt529BU6BPDmzsBrDnEBxQp2dhBv7jX6Qf7J2L4IiMhZOUHnwWWNuLYxBcAJCRRiZkVEPWQXIQEF8aV4NBCbWjKPRkJmdrsvBGAEvERn69GR2pxAETZ/AdQQcSsN1+pLjSMVHpfIjRMAhQq71AZHaC58hiVAjPf0ZiAgxfJ0IUc7OHcemMy4AndqOee07HxFlzIvJbr/6ocOqJ748GAZPzl3bt+D9734n/r+//lvMnVoEALx06Bh++a8+hw/PTOHd178Ct5y9FdR6clpkoCgEbEIeUk4WsIfbNbWXXNtT+1VX9ti1OLRyBkk4qrRBECpNHAQBZxycyLU+wgHf1+9JJrnwYH2PBX/LZT8Khwj4HHCoJDdJglSGQRGRHyDVIYA08RFd8enfQfr7qUxuxCsu34Q6lySqwISDBu+iM8OQYlUT38ry5Gzi7W98Dd501UX45Ofvxsf/5R7MLdYAAC8dO4n/9I/fwh9Pj+Ndl52NN+zYIE13rSenxZCgXWgzdbxK/tD8PU3kx4Nxmwx7yl53DoTgskxOdWPXShuIkBNExgQIFfCbwTUS8F0Sdmx3qHR2oUFZBaWS7HxOQIHwdajw9JAmiUhOvUsW6SX3AcDuF5/GoQN7cfk1N6XukgmKOrNenRYBhsWTU5UvrKl4eO8dN+GnbngVPnH3t/CXX/su5qp1AMDLJxbwa195CH82OYqfueh03DgzCWI9OS2GGFmlDOF+jQD1RBdFfkC05pe6dnCugAgaISRKGwiF7NIgwBgHY3KccgKwIMEFEGg2CYQjiY9p9bgAier9iCQ9ymVNoFJ78t4RhEKDz4hoOxCRY/gdGDIwhRB4/NFHcP7FV6PB0j/7PqeoGbZ3giTRFhmrlvh6LWHoRO0BMKu9buzJcnZgmCq5+MUfuBo/efl5+Kuv3Y+//vcnqfzqiQAAIABJREFUMFdvAgB2zy3iA99+Ah8bq+AdO0/Da2cmNQWYbU+mkCxYB6za6wWD6lm3EtBpI1ud3Fodo8gP4GHGZ7w1UVDeIAIXFzhBt4aotIFQORaYz0GIXO9jFLJ9UFAQKIQkNC4AV8jJrOzkIO+FEDnWoyxQwKEkyOhUvx1ByFMRnfZZdHJMfXcaER3a/xKaPjC98UzUfT3bXD4zQdFgw9NBvVesSuLLS3qtMjnlNhInPUpD0guL09WzK02oHd2M2nXiRtQlinKJouQRlEtA2QMqJYGSx1FxGSouQ9lpokwbKJE6SryGkl+F50sjaqexCFKvgtRrQKMGUa1iwgHefcU5ePu5W/DJB5/FnY++iPmGXITYc6qGDz76Ij4+UsKPb16HG6bGQAK1x5qSAFmDacpPxEiP+yLlyZlFbpb0LLpBZt+9NoSYJEDl1qLv19f4skOmwTM0tReWNvCg2Wyg+DgFYZJMQydqUPh+0MKIC5kFSmWhOxAUuBME5CeVHwvVoKbygn3y7/Rapunr0I9jKOGsV9yQSW4+I6g1qZFAVyJWJfF1ipXgyTnhOvi5V+7EW886Df/rsZfwN8/sw3xTFsTurTbwe8/vxyfLHt62bhrXjVQA68lpUWDkJcRYeNOw7qeSWsL1PtWWiMjyBiG4JLxA9fFgTFBQMHBwTuE3GSgXcJygnZFDwZiA68pnLtS4D8KaVD47jlJ3garTSiDkNhVFQuw5bpCR+PwG4qrXTsEb3YoSdTC/aFaHDR9YqFrFt2Kxcj05MzowJDw5xxyKnzlvO96yfQP+5qm9+Ls9h7Hgy8G8r97Eh/Ydxt94Ln5kahLXlMqgHFro03pyDgJFzqbsBstZFsED8+hWSKq/1P7AoFqZWYfensHaWayuj4uwto+xuNIDpCIkVI5rlc1Jg4cQBIwIjfSIRnpyGwBN6cWfFcxqL/r7sQe+ipmNZ2DzGa/I/Mw+k+S3WrDqiC8vhs2Ts9MODGOE4B07NuAH10/hH/Yew2cPHcdCMJvd3/TxR0eP4zOOgzePj+NqRxKgglV7Fq3QLyLvlkDD5rN6eULSnUUjv1aJLmrSJwSkdbQQIImudWF5Q+D4EhGh3O8ESwWOQyA44DgEPEiUIUSSIOHKySUiQbA4gUWKL/k9pb8DdcypuWM4fmQ/dr3yFjRbEBvnQKNpVoMrEauK+FayJ2eW2mvnyTlGKX78tLW4fe0E/unQLP7p2CxOBYP9AGP48MmT+Hvq4Icro7iSe3DUj4X15LQYMNplbLZDUv21WstruZ9zwHFCJRjV9JGwxo+Bg4ZLA1LtOQ4NmtRK708hgmbqSkHqxEci0nEcAkCkiC5JcOHvT0Cwpp+3F5+6Hxu3XwImPLBmtD15LONoSYwrDauK+PKindpbbk/OfnRgSHpyjoLibTNTuHViHJ8/NovPzy1gMVgTOcgZ/mRxHp8lFHfQEVxFSrHMMqv2LJYCJvPpdmgX+kyaVmcel0yOCcKdysSaasXu0fqjIj9JYnKNkUJoYUxCJemotT0ZAg32abdDpXVMYvIuMhWa+ijrtlyE0YkZ+G3GJecEzQFORk2qdGlg/oIs8SVgGiRF8uQssSo8vwa3WYXbXITTqILWqyD1xb54co4I4C3jE3hDeRR3zS/gruoiFoO1jYOC4yPsFP4JVdyBCq7wPTjaf6y8npxW7Vn0ijxlC1nnZWV6KqhefKBxNxhZyyeL2QmCzM5A8cEJsjZJlNVJCAEDDya+UiFyzmWUR0QhVkoAoam65JoepVJ0yntNFqlnf9a5Yy9hfM1pAC3Db+O2xDnaHtMT2OAu3Q1WDfGtFE9OU9shMHOIsxdPzlEQvHl0HD/gjeCL9UV8sV4NCfAQOD6KRXzOoXgTr+AKuDECtLBYKuRRgXkSXvTrJa8TcGB0jCI7CnCIMNMz7Agh0muH4ZIHIeAMUaY31UoSVKSJxKNK+q2bfsdMaqpZP4UnH7gLF1zzk/DKZkcWfanGZwT1+uqZkK4a4suDYfDkTIY4VVKLKcTZD0/OMUrxw+VR3EzKuLtRw5d4LSJAwvExZxGfL1PcVi/hCniWAC2WDe3W8Do9Lga1zgep9sIkF+XuEkwgOQVooPxokODJmZDhTdUVghIQgVAFKv6JlVuwiKw5dGVnCMkmcokoAQ6+9BCmN5wDxx1NlSJFp+mmFAS+v7Kyi1thVRBfv9XecntyJtUe8ZsQGWoPffLkHCMUd/AKXocSvszquJvWUA2+1kOU4y9Gavhn3pAE6Hug2kK6DhvmtCgauiJCDTzI2FTX4pRKH0/V1J0rEpTNbSmXBfFwqCQ4IsmMaUkqlAJMI6zYbxhrrWB91sSRvY/h7MveHrtGK8hO8qtnbK4K4suDYfPkTJUwZKi9ZDJLr56cY6C4Q1TwelbCV0Qdd7t1IwHeuljCFQ0bArVYOQi9OzlAnHjPP5XsQrkMPYahUC4dVDgQkh+DtDsD45LwBKBMVigBQBGSIJBe20uSYhoOdl3643BLY7nJjIt211xZWPHEt5QdGICl9+QMSxgy1F679T2gO0/OUVC8ya/gdX4ZX6F1fLkUJ8C/HK/hnxnBbdVySIBW7VkMGr2qt46hRQejJBflAqMlyHB1aKDWNPUHIFxETJEgEBbOq98YZkgUUb9zzK/jyO4HsfGMqzsiMqv4VhB69eTU6/KK7snJa/Uoi7PeAG/68FVGp57J2fDBmgzc533x5BwFwe2NMm5qlPC1UgNf9uqoBl/vYUeEBHjrQgmXN60CtCgeuiHKWAG8EFBp80r1ARH56et9CJ7S6k8rnNdIENB+xxLEpAehFDke3ft91BZn5Tg2dGmIPnOiCB+IZbCudKxo4usUQ+3JaQhxiowQZ5jUkghxAt17cioCvHHRw9fKDXyl0ogR4F9N1XHXWAO3nirh8polQB2r5QenlU1YUdAJCer/bpxHndFlggugyE+6uIjQ0lCOOxJFiYRKYhFhYou8l/R76pmYyVQUzpo4vPsh7LjwDvg+b1mXiITXKefCJresBPTDpUU/p/CenB2GOKVTfDzE2Q9PzlEQ3F4r46Z6KU2AriXA1QwTwQ+CDJc83NkCys6MUBIUt8vtSQWoELttnv5uQoVowMLcAYxMbUFpdEYW17cJXcZ6EIp4UttKx4olvrxo5dLSsdoroCdnptpTBJhQe7E1vh48OUdFRIBf9Rr46qglQIs0kh0TigBTU1rAfI9Zaj1KdCGyDlB1fQ9CpGGmJxBjvkwSjF3cfH9jk9swOrEVIqdwY7FyhnRyy/K5rQweK5L4VqMnZ6dqTyGp9gAY1V63npyVBnBbo4QbFz18fbRpCdDCiKUkwE6ztmWmZSuPz3gSi+l66rg0+ZFEl/gokcXUekk/XseJg49CCI61p13cZehcxM6T99LFZYYEK5L48qLbDgzA8HpyxkoYBqT2TBgVBLedsgRo0RrtWgblQb/CnGri2i1SiS7ETH4AMgkwdj8ZZMi5jyN7HsCWc27J7FOoI4+LzUpfd161xJfHk1OpvZXoycl8HpIda/KQ9DjnIallqb1ePDktAVq0Qz/ILw/06A3QP8IE4nZlClwgJD8AYfZmFgHq96muacLswSdQHlmHyuj6mGLMgokchchu7rsSseKIr58uLaYQZ1E9OTsJceqenCaXFiB/iLNbWAK0aIVBhT7bhTlpggy7Qaq/X6D6AIAH1zYRIIAw+zNxRXlYBpkJzjCz5bK4oXaHNXkyGrSCY5sJrDjiy4MV6clpCHF24snZDoMiQkuAFkUE0SI9CMc8QEBAaPdhUJVxqtSf3BZXaVmkb1JkzK9hauMrwv15zbhXO1YU8S1VB4YienJmq73OPDmBzkiuX62HeiHAKhFwBFCypLjqkduRyVCy1NX7Ja6T1dk9/nea/NQx8l7iBJj1Xpw18fKjn8HW8+5AqTIlt3UZroz3EVz5WFHElwf9UHtF9OTMUnvdenLqGHTYU0enBPhkieGP19QwxQnee2IEm9kKzsG26AqtVFAyzLnUSJIfkCbAcHuCCGcPP47y2Dp45clCloUUGSuG+Fa7J2e20uvNk7MVBtloNi8BbvQpOAFOOAIfXVPD+46NoGyVn4UByd8C88SXpgin/XXzHW9SfUBkopJFgKb34KyJEwcexpZzbouf02U25mqzLFsR0+NBeXJSZ3g8OVOZnH325ExiqbqrKwL870fHcPtCCSPa+vthV+DRCguN6w+4HJ+erPf1/S1WHvKEOcNxT7T1vS7DoVxPOtHrZTlPHJdyEoudF3sIYMOOG1AenenqnlY7Vozi6wR5XVr0v60n5/LOBlspQF3gfWfEx1mNBq6tlZblPi2KhWStrmk9H0CqbCkLWYktqbU+Va8XPHMRmVAnlV/yfkzkp/88Mb+O2qnDGFuzo382Y6vMsmzoFd9APTkNIc6BeXKK4fLkNGEp2g4pAvzpkxVMMQKTAf2dkw087/oDvxeL4UdyzCuoKA8hQBYPdhoS1ZEkmaT6S0KpQS6A4wcfwcLxZ7t+b4sVQHx5YT05l86lZSnwrZEmTjoCpuU8QYC7xjNawFusWLRa2jAf3zrRLXltMuC1YzlWWxOg31zEycOPY81pl8fIMPmwaI2hDnUO2pMzS+1ZT840lrrJ7KV1F0+WGRqG/wJEANdUh/q/tsUAkBXmDPdr7kztrhGiA+mghztboVV3ibnDj2Ni5mx45Yk275X/voDAuWUVEeaq+HXo1pMzS+0V0ZMzU+0tsSfnUuHKmodX1lzUqMEWjQPe6glmrFh0kpqfJ5EtfnyUnZ11PSIr1ju6r07u2WRrFu1Lr/0BwPTmSyG4oQW7RUdY8cTXiycnIWRoPDllFmdxPDmXAmUQlDNatFhY6GhlUtGuhMEU5kwa1neLVuQn90eT1uP778f4zNkoj6zt+v0sJIZ2WrwUnpymEKf15LSwGDwGofaS7cRS+/WxH2Zv6+fH76t19qd5HzfUyiWXJUyonTqCuaNPw3XHwrXArIdFe6xoxderS4sxxFlUT05TiLNAnpwWFkVCnBBbr+0pf079+KWEEALH9t2H6dMuBXW89sd3QX5CiGUlzX52xsiDoSS+pfLklPuHxZPTrPaK5slpYdFPdLu2B5jX98IyhhZLJImNqfftFllhT84acNwRTK47t+f3KCqWmnSHNtTZDsvSgWEZPTk7VXvL7clpYZGFvrciSqg5k9ozQUV88t5jP+47Vd8nZIbnpp2vi4VdLXrD0H2TS+nJman2+uHJKfrtyZmh9gruyWlh0S1alh10qPZ0a8K0sYX5Gt3CtM6nQye/uSNP4vBL9/T8nhZxDB3x5UEnai/5OlcHBses9uTfabUn1/fSao/yDLXnN41qj2eovVYdGIB0U8p2JQwWFisFrXw5TSSnQJOG1gMOcyYhuADz6zhx4EFMb3pl36+/2jFUxNep2jOt3+k1emEJQ/CsShfCZ618wUmUMcTMqLUSBlW+UEqWMKjyBcRLGFT5Ahq1WAmDXr5gKmGIjKh5RIRB+YJg8RIGIFJ7eglDspwBKH4Jg8XKRt5woWnM53VnaqX2kveiH0q06FDre+sPEZ7Y/xBGJrehPLquL9eziDA0yS3L4cmZFeLsyZMz5dLSH09OIB3iHGZPTovVh17WyEyk18qdKVbDm6jJS6776abU4Xal9gZoYzY2fSbc0ljL9kQW3WGoFF8e9NWT0xTiLKonZ0YHhpXg0mJh0Sla2ZOZsjaJNu6jbXGH6qUiHSEE5o4+idLIGrjeaHxfjpo/i/YYCuJbLk/OLLXXtSdnqgND/zw5s9QeMNyenBarA/1Qey2vb+jCoI/9+PVyrO8lr0/ivyG9YHFuN2YPfR+kxc9zMqHNojMMTagzD/rtydm52ltGT06r9ixWCTqt121VuhRuM6m97DcAkO3zGb92Z0TIuY9je76DdduuA6FOrnNM5GdDoq2xoogviTxqLzk48nRgkM9Z9mRptedmqD2SofZEs9m3DgxcKwy1as9ipSFvRqVJ7ZkTZKhRvbX08uxj3WFt/gDKo+swOrWtp+u0U4KrnRgLT3x5w5zJAWCa9ekL29Sh4cI2daQ9GQ2UHXWdUOVJE+p0Jme5RFHyCMolgnIJKHtApSRQcRkqLkPZacoHbaBE6ig1q/D8Kpwgk9NpLIIE2ZyiWoVo1MFr9ciIuunDV9mcKTNqBu5zaTrN5TNrsJD0FAHqmZytSM+qPYvlRC++nMl97dReq3NMSE6UW6m9XslECIHRqW0Ymdza03VyvVeSGEV7slxJGIo1vnYYlCdnZoizgJ6cCtaT02Klolszajnmo21JM2q1Pf47oE+Us91dujGqNkEIgYMv3I3Fub19d66xSKPQxLdiPDlFtienqQNDN56c/erAYMOcFisJecyoFekZzzf5eQ6AmE7NvgC/PoeR8dP6fm2LNApNfHlgPTmtJ6fF8KLbgvVOWg+1MqNO3ku7BrXx43PceA4wv4aje76D9duvz53QYtEbCkt8hfDkzFB7hfTkzFB74Tar9ixWOFrbk5nDlSa1p4dIY9clJJMQW9kgtr9vB+u2XYvK+Kbc51j0hsInt7TCwD05M9ReET05O1V7FhbLjV7syVrtS6q91CS4jdpbStQWDoJQF+PTO5f8vVczCqn4CuPJ6dKh8eRUpGc9OS1WI1qpPdN+ta2d2gt/IVPlED3cbADOmjj80tfhN0/1fjGLjlA44iuSJ2dWiDOXJ2eqbs96clpYKPTTjFpPZiOamqOmSTDVJ8H68kd6fS9UgG1Ir9Mwp7r+8f33ozK+CWNTO1oeb9F/FI748sJ6csZdWnRYlxaLIqPfZtTG4zoIcSbPa3XN9Mb4y7zhUs6aqJ06jJmt1+Q63qK/KNQaX9E8ObPUXltPzlQHhsF7cmapPevSYrGSYRr3sf2GpBY14aXhOen1vbja07oj9CPEyZsg1MGWc37Q1uwtEwpFfHlhPTmtJ+dKwmUXr8HUpJe5/+RcEw88MruEdzQ49OLS0s6eTA9Vms7NoxJN76vvS2xseb2s9zj88r/Bq6zB9KZLOj7fIsL5O0uYGM3+N/33x2qZ+4aS+JIYlCdnltoroienVXvDi6lJD8dPNDL3r50uLeHdDA9atR6S27PVXuyYDALrVI3lCXMunHgRtYWDWLftuo6ubZHGxCjFyQXe/kADCrPGV0RPTq8bT05/eTw5AVhPTotCo19qr9W6vknt6ZZl0fXiHp0dq70uIAvVv4UNZ9wI6mQrfIvBY6gU35J7chpCnEX15MwiNevSYjFsyEN6yUzO6G+zJ6c6R/fkVMelrq0pRdlhXQTn6zfVORl63gg2n307SpXpjs+16C8KofiK6slpDnFaT04LiyIgc9znCHGq44BEB4Y8hGY4pt15Jw5+D6dmX7SkVxAUgvjyYFk8OTPVXvE8OU2was+iKOi3J2e0LduT01TCkAxxtrp27L17iHYuzu3F/JHHUR7d0P1FLPqKZSe+IntyZqu94nlyWrVnsVqQ7LCQ15PT9Nqk9lqSdIchTr+xgCMvfwMbz7gRbmmso3MtBodlJ748WC5Pziy1Zz05LSzyoyienEm1l7In09CJ2mvZnZ26mNl6DUYmtrS+iMWSYlmJr+ienCUPQ+PJCUQEaD05LVYquvXkjJ+TrQj7WVA+e+gRCMExufasvl3Toj9YNuIbBk9OY4izoJ6cNsRpUUQUxZNTz+iWx5D0ee1I00SYGZ9v7sgTmD/2NCgdqsT5VYPChzqX05PTHOK0npwWFv1EHtLLQjeenG1NpHOqv6z91fl9OHHgAWzaeTOoY80HiohlIb5h8eTsXO0tnydnO1i1Z7HU6GfYsBdPTv0YhVQHhnboIKmFswY2nH4jvMqa3OdYLC0KrcOX35MzQ+1ZT04Li4Gh356cNKYIzYlyYcF6D2qP+TUszr6EiXXnRu9tTagLiSVXfL3MApfakzNL7VlPTguL1hik2kvt71Lt5UYOtcdZEwef/xKajbnOrm2xLCjMGl8RPTlLHh8aT06r9iyGEUvpyZlX7bW95wRxCs5w+KWvwStPYfq0y6P3tmqvsChkqLMonpzGEGdBPTnz7rewGDT66dLSb09OmogKmUgvnsnZ/rMJwVEaWYfpTZfY/npDgiVVfEPnyWkKcVpPTguLJcUgPDnzvW/r44UQ2P38A+CsjrWbLwOhTrjPqr1io3CKr1ienGa1Zz05LSx6wyA8OePXa+3JGbteltprASEEdj/3bbDaYdDRs3OdY1EcLJniG05Pziy1Zz05LSxM6HdSS0dqLzERVscB/Vd7e56/FwuzB3HRZbenavWs2is+CqX4iubJmaX2iujJadWexbCgV7VnKliPX6+1J6fu1NKJ2iOEQAgBQgjWzGzH5h2XwvXKAGotz7MoHpaE+IbWk9MZHk/OJKzas1hqDKKEIXzdxpOTGEKbuYvTc0Jwjhee/DrWzGzHzCZzeNOqveHAwIlvuD05tRAnt56cFhZZ6GcHhkF5cqrXpvttd/+c+Xj+8S9DAFiz/gzjMZb0hgeFCXUW0pPTEOK0npwWFt2hV0/O5LVMdXetPDmTpJfl0mJSivtfegCuN4LTz70BVMvetBhODJT4VownZ0rtWU9OCwuFpfLkDBVgIurTV5eWBE7NH0Oj7mLLGZeDUMfW6a0QFMK5pbCenBlqz3pyWlj0jnbjvlVGpknttfLk7EbtLZ7cjUe/+7/bu9MgR876juO/brWO0TU75x6zB7vetXfttY0JsQ0EbGNzmCuBhGBSJEC4khSQvIBKcRRUCCkHUqmkUknlhaECFV44kDiUKUM5BYENMXaCgWDwbYy9613P7o7m1GhGR3fnhWY0uqaluTTSPN9P1dbuSC3pUT0r/ebf3c+/71Yum5Edcgi9HWTLKr6trvY60ZNztWqPnpxAWad6cna62ps5/1PNXPyZXnvrG+SHd7fcnuN7vWXbK752O7GX/123YL1JtWdZ1hqrPa35Cgzl43vtV3uVP83ak1HtwQCtevE2br96tdf8WOHGe3JatiW3lJfv+4okRrXvsjcrPbA3cJzoTdt6cku7vfmqz+aqXsKwvHzBduyaJQw1f6qWMCwvX4hGrJolDJXlC/VLGKqWLzjFXM0ShuXlC34+X7OEoX75QmUJw9LyheW/l5cwLIedW/RqljDUL1moD7n6n9s9Bghslq2u9qr3+LRT7VVrpwI7eUlEqcTKV+D09Fk98dC3ddlVr1L/4OGlcwKCnyMUkl5yZV/gNnM5T488XWg5HnTOlgTfjunJ2WQX5/JJLX6X9eRcDbs5sd1aLVhvduWV1UKvsoShKvRq9/rUXrWl8nxNqr1UwtFMtnz8/bln/lcT40/oyImbpMgezc77klp/dqZnW2/Xn9z2HWuos20VXy/05FztCgz05ATa0+09OUvFvEKRqJxwTCevfavCkXjL94Tet+m/iuysnpyrXIGBnpww3Gaf4djpnpyeW9TTj/9Aj/z4Lsn3tffQNWsOPU5o6V3bUoP3Sk/OVas9enICLXW62mu3J2c+d0FnH/uaCvl5nfiV32j7l3LsHJu6q3OjSxjWUu1Jal7trac9WdNjewFXYCiWmlZ762lPtqx+wbpEtYfu1LkrMDTrv7n+npyFhYzsUFShcFzDB2/Q8auOaibnNWzXzvuj2uttmxZ87YZe0Jmc5dus2tCz7ZUD2suL05f/dspNqEPVzaidUG0j6oitaMRWJGwpGpGiYSkW8RUJe4o5rmKOq2ioqKhdUMTKK+ItKlJaULhUbkQdKuRk5Rdk5RfLZ3MuLMgv5FfO4swX5BVLKi2f0dnQjNqVV/Lke175zE3Pl1twqyo/vyb0vJLf0JNztXAj9NCt2unJWXvb1lV7pcKcZi/8RLnZ0xp9wc3qS40pHEmt8R1hJ9m2Gp+enPTkRO9ppxpqtydn/bq7xl946/f01H72lx9bf3yv+nvE80o6//Q9CoXj2n/529SXGqsa1PreH9Ve79uUim/n9uRc5QoM9OSEgXqlJ2cxP6PZiw9J8jV84AaNHX+rbLt1yzFCzxwdXc7Qaz05N63aoycnNiDsWLr55SOB28zMFvXgT6c7NKK1We8VGKofW3vb6j05J8/dp9zMM0oNn1B6+IqlxzeG3mZfqw+b6/IjEaXiwf9fWjUXCLLh4NvJPTlXq/boyYlOOn8x33KbwYHIlo6hW6s9t5TX4uzTKixmNHzgBiUHL9XA3usUciI6eTSqdCLUfPy25FR9RW1lA+pQSLruZGzV++ns0igVtzWTbTzxaLN0rOJrtxP78jbd0JNzrdUePTlhuk725Jyd+ImymUfUl9qv1NAJSVI0Plq5P50IVb48a56+w9Xe5EzwFzidXTqvI8EXdMHIbu7JaeVzPdOTk2oPW6UbrsAgy1d+flz5+dMq5qc0evj16ksdUnroCoWc2IbW4rX7/ji+t3NsKPh2ek/OZrs4u7UnJ8zWDccB19uTs/4X3so2flHFxSlFE6OaPHtKXimrePqw0iNX6erjCfUn0+Vtl7+HbFVKO1vl7wEpuNrrhtBrtStU2lm7Q7f6+F07trzi6+WenM12cdKTE91ou48DbqRLS739g6eVz51WLpvR0OhhHb38iPxrXqeQEy4/1pIcx9bkjFuzXm/5znJI+ZVtVwbVnRVbq12hkjTYb++YcNzq43ftWHfwbXa1t209OddxBQZ6cqIXtVMV1qv/mJerxplVt3/x1f1Kp8INJ6lVnmz5c7308+z0Wc3PXVR27rzCkbiOnrhJ05m4srErtffwPjnhsKZn3fIvvXJlLX1t2Hbn/s93wy7OzQrH1YyNOrruZKxleLZTrbWy1dVcO7a04uv5npyrVHv05EQvaqcqrFf/C+7ocFQ3v3y4bqOVz7bjWJrIFKruKt/nea5y2QvKL05rcSGjRGqPhnYf15lfPqZorF+JXZcrkRzV9JwnO3pY6aik6ksTqTFIV6/2Kj+u6NL56yX2AAAQyklEQVRqbzO1E46ryRd8zWS9luEZCm3sdbrFuoKvk1dgkLaxJ+cq1R49OWGCZp/zCxNNwnPp8+z7niRLbnFe2dmzyi/OqJCf1cjeKxWNDWji/M8V6xtQsn+/kuk9kqSDR2+UHbIlWQ3P2bj2bjPeVdXzsWC9wU4ItXasOfg22pOz2YHt5W26rSent0hPTmCZ77kqlhZUKuTk+64Sqb2annpas9PPqpifV7GQ1dGTb1GpuKDFhWnF4ruUHjioWN+QQk5Uhy97TW2rsWY9Oeu+XtZS7dU8zqy8whpt+cktPd2Ts8kuTnpyYifx3KKKxQV5bkElN6++viFJ0uTFR1UqLcotLSiZHtPgyHE9/fg35XklOeE+9cWHlOgfUyTar4HhyxSOJBWJJuWEHEXCMSXSu6tepTakas7etMrVXk3oVU5qUyX0LLvq8Vbz0FvZc1T3JgOWUwUxrdozyZqCbzO6tFQ/put7cq5xFyc9ObEVfN+XZVny3KI8ryTPK8n3PUUiSbluUYsLk0u3uYr1DSkSTSpz/mG5Xnn7aDStwZHjOn/2R5rPjstzC/I8V8eu+E3NTp9W5sLDskMROU5U4b1XKxJNyQlHFY31ywnHFIn1S5KOHH9Dw+c4nmx1skzjZ7pyj2U37r6sCr3KdpvcYWUru7SgNwQG333fuKFT49gClsoLe6rfYv82jQVY3WM/u0fZ+YJ839PA0LGl6uoeFQvz8n1P0Vi/Dl96qy6O/1QzU88unfDl6NAlt6hQyGry4qPlJsy2I8fpUySaki9PjhOTbTuKRMvr3XYNHVX/4GHZoYhCdnjptku0a+iSmvFYlqWh0SsaxtkqMJYzrd0zuys/N9nFWf88DZcu20C11w6qvZ3N8n2f0gEAYAyaxAEAjBK4q/NlbzzVqXEARuntwwhAb6PiAwAYheADABiF4AMAGIXgAwAYheADABiF4AMAGIXgAwAYheADABiF4AMAGGXLL0vUC878/MOK91+joQPvrrnd90o699jH5JbmNHb5XyvkJGrv912de+yT8t2c9p34nOan7lfm9B0Nz2/ZMYWje5QevVXJoZc3HcPM+XuUnfxvjZ24veV4L/7yHzQ/9QPtufRTiiUvq7nPLc3qzEN/oOFDf6Tk0K9VjbWkuYvfUXby+younpNl2QrH9is5fKOSg69YujxM2dS5r2pm/OvV76D8HmJjSg3fpOTQjTVNg8/87INyi5Orjnfk8B8rMXCdJKmwcEZTZ+9UPveUJEuxxKUa2P8OhaOjLd83AGwGgi+AZTsaOvg+jT/xZ5o+9y8aOvj7NffPXviWigvPavfRj8sOxSq37z76MVnLP/u+3OK0Zi98UxPP/qNC4bT60lfXPM/81P9o6uydCvftX8PofGVOf0H7jt8uyw6eRs9d1Plf/JUK879QauTVGtj32/J9VwuzDylz+ovKTf1Qo0f+RNZSx/7ye49p97GPVd6D52aVm35QmdN3qLDwTMMvCfGB65UevbXp64ejeyVJpeK0xp/8rMLRfRo++H75vqvp8X/X+BOf0djln5cdiq/h/QPA+hB8LcSSlyo1fIvmJr6t5NANiibKl3ApFSY0/fy/KTn8SvWlT9Y8JhI/0lAd9qVP6vRDH1A28/1K8HluTtPP36XZC99a85e+ZfepuHhO0+Nf18C+3wrcdvLsV1SYf0p7Lv1UZfySFO+/Rn3pF+rCLz6vqef/VYNjb69+AcUSx2qeJ97/IoWctGbO363ErusVS52o3BdydjVsXy+bOSXfd7X76Ecr7zeaOKbnfv5BzU/dr9Twze2+fQBYN47xtWFg7G0KhQeUOf1F+X75yuqZM1+SHUpqcOx32noOywrLsmp/z5ib+E/NT92vkcMfUl/6hWsakxMdVXL4lZo5f7cKC8+tul2pOK3sxCmlRm6pCb1l8f6rlRh4qeYu3CvPXWj5uv173iTLCmsu8901jVeSnMiI+kdfXxPyTmRAlh1TKX9xzc8HAOtB8LXBDsU1dOBdKiw8o2zmlHIz/6eFmR9r6OB7VqnUPPm+W/7jFVXMn9fE6Tvke3klBl9W2Sq+61c1dsXfKDFw/brGNTj2doWclDKnv6DVLqu4OPeIJDcwWOMD18r3C1qce7jla9qhuCLxI8rPP1l3j7/ynmv+eJUtkoMv1a69b64b36PyvQWFY/tavjYAbAZ2dbYpvuvFiu+6VlPnvio71Kfk4CsU728eJmce+kDDbeHYfo0cLp9EU7ktuntDY7JDcQ3uf6cu/vJvNTfxbaVHXtWwTalQrqScyPCqz+NERpa2nWjrdUPhtAoLz9bcNnfxXs1dvLfJtoM6cOXfN30etzSvzOkvyomMKL7O8AeAtSL41mDwwDt19pGPyvcsDez/3VW3233sk7LtWPkY3vhdcgsZjRz+oCJ9B9f0euVqqbqSs2rOvpSkxMC1mp98sabO3ql4/4tqTlBZepalh4ZWfR0r4L52JQZeqvTo65o8d/P/Ym4pq/NPfU6l4qT2HPuEbDuy4TEAQDsIvjVwwgOKxPbLdlINJ69Ui/QdqtwfTRzVucc+ofNP/aX2Hb9doXB/2683/sRnlJ9/ovJzcuhGDR96f8N2gwfepbOPfFSZM1/S8KH31Y55qZpzC5lVlwwsV4WhyFBb4yoVphQKD9TcZjtpRRNH2nz8RDn0ChmNXvKRpsceAWCrEHxbzA7FNHzwvRp/8s+VOfNljR75cNuPHT70fnnuYuXnkJNqup0TGdTA2G2aPPNPmp+6qua+vvRVkhXS/PQPa87CrJabflCWFVZf6oqWY3JL8yosPKtk1bHKtSguPq/xJ/9CvlfQnmOfIPQAdBwnt3RALHVC8V3XKzf9gBbnHm37ceHYPkUTRyp/nOjIqtumhm9RNHGpps7eWXN7yElVlmMsZh9veNzC3MPKZv5LqZFXtbWkYvbCPZJfVHL4prbfxzK3lNX4U7dL8rXnsk8TegC2BRXfkkLutGYufKvh9sTAS+SEd234+QfGblNu5keafO6ftff4ZxuO1W2UZVkaOvhenXvs401fu7h4TuefvF2p0deoL1Ved7gw+5BmL/6H+tJXamDf22of5HtaXD5z0/flleaUm/mxspnvKT16a8OaPbc0vbJ9nZDTr3B0VNPnvia3MKHB/e+U5+ZqtnecXYHBDgCbheBbkp9/XPn5xooomji6KcEXjo4qPfJqzV64p1xhDd+44eesF+nbr/7db9LM+F01t9t2VLuP/qmyme9qbuKU5ia+U94+tl9DB96t5NArGoLY9xY1/vinl36yZDtpRWJjGjn8oabLL3JTDyg39UDTcSWHb9bwwfcoN/MjSdLkc19u2CY18loNHfi9tb5lAFgzy19tAZikl73xVCfHAhjjvm/csN1DAIzFMT4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUZygOy3L6tQ4AADoCCo+AIBRCD4AgFEIPgCAUQg+AIBRCD4AgFEIPgCAUYKXM9jkIgBgZyHZAABGIfgAAEYJ3NVp23RuAQDsLFR8AACjEHwAAKNwVicAwChcnQEAYBRKOgCAUdjVCQAwCskGADBKi4qPY3wAgJ2FXZ0AAKMEd27hrE4AwA5DxQcAMArH+AAARmEBOwDAKIHBF3JCnRoHAAAdEXxyS4jgAwDsLFR8AACjtKj4OKsTALCzBFd8BB8AYIcJrvjY1QkA2GGo+AAARgkMvkgs3KlxAADQEVR8AACjBAafE+YYHwBgZyH4AABGCQy+cITgAwDsLMEVn8MxPgDAzhIYfIkEZ3UCAHaWwOCLRqn4AAA7S/AxvjDX4wMA7CzBFV+E4AMA7Cwtgq9TwwAAoDOCg49zWwAAO0xg8PUn3E6NAzBM4EcPwBYKrvgcr1PjAACgI1oEHxUfAGBnCQy+WKjUqXEAANARwRVfqNipcQAA0BHBF6K1C50aBwAAHRFc8dn5To0DG5Sdm1MyldruYQBA1wsMvlRxqlPjwAZ85FOf1akfPKAPvOsdeu87btvu4aAth7Z7AICxgnt1lhY6NQ6sU2Z6Rt+7735J0t3fvFd/eNuvb/OIAKC7BV+Pr5jr1DiwXgtzlX+WikXmDABaCAy+UIGKr9uFiotVP/nMGQC0EBh8Nl+iXc+qDj7fZ84AoIUWwbcYdDe6gF1YOfPWks+cAUALgcFnzXJWZ7ezsivH+OR5zBkAtBAYfH6e6qHb+VUVn3yfOQOAFgi+Hufnq5sMEHwA0Epg8HmLfIl2u+qg832fOQOAFlpUfPTq7HZ+oaqRuM+cAUArgcHnLtKrs9u51UHn+8wZALQQGHzF2WynxoF1Ks7X7upkzgAgWHDFV+BCtN3OK7orP/g+cwYALQQHH8eLup5bqN7VyZwBQCtUfD3OLaxUfL6o+ACgFYKvx3nFqjnymTMAaCV4HV/JDbobXaB+jpgzAAgWfFZnjlPju10xv7KOz/d95gwAWgje1Vn0OjUOrJNXqpojnzkDgFbY1dnjaoJPPnMGAC1Q8fU4l4oPANaEiq/H+aXq5QzMGQC0QsXX46j4AGBtgiu+ItVDt2s4xsecAUCgFgvYqR66neuuzJHvM2cA0Erw9fg8v1PjwHrVzRFzBgDBgnd1unyJdruaOfKZMwBoJbji40u069VXeMwZAAQLrvg8jhd1u/o5Ys4AIFhwxVekeuh2vl+9q9NnzgCghRYL2PkS7Xb157IwZwAQzN7uAQAA0Ens6uxxvvyqfzNnANAKFR8AwCgc4+txvvyVWWQdHwC0xK7OHlcTfGLOAKAVdnUCAIxC8AEAjMIxvh7nWXW9OpkzAAhExQcAMArBBwAwCsEHADAKyxl6nG/VXZaIOQOAQFR8AACjEHwAAKMQfAAAoxB8AACjEHwAAKMEd27hDMGu59l1nVuYMwAIRMUHADAKwQcAMArBBwAwCsEHADAKwQcAMArBBwAwCsEHADAKwQcAMArBBwAwSmDnFnSvRcvXV1J5Fauux+fK19/tWtBYydZbshFZsrZxhADQnQi+HvVgrKQf9pVqbsvZ0sNRVw9HXV236Gh/KbRNowOA7sWuzh61p9Q4de5SgRf2pQGXqQWAZvh27FGXFG0dLTSfvhtzYSV8dnMCQDMEX4+yZOmN2UjD7WFfenUuvA0jAoDeQPD1sMuKIb2gruq7MRdW2mNaAWA1fEP2MEuWXjdfVfVR7QFASwRfj7uqENKxgq2QL91MtQcALbGcocdZsvSRqfh2DwMAegblAQDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCgEHwDAKAQfAMAoBB8AwCiW7/v+dg8CAIBOoeIDABiF4AMAGIXgAwAYheADABiF4AMAGIXgAwAY5f8Bx84XGHjlCZcAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nn_graph_draw2d(\n", " ([[2, 0],[-1, -2]], [0, 0], tanh), # LYR1\n", " ([[1], [-1]], [-1], tanh) # LYR2\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The histogram on the second layer plot shows the distribution of\n", "$$\n", "s = X W + b\n", "$$\n", "feeding its activation function (shown as the dashed curve). Note how the central histogram peak is higher because both the lower-right and upper-left quadrants of $(x_1, x_2)$ have $Y W \\simeq 0$. The vertical white line shows how our choice of bias $b = -0.5$ places these quadrants in the \"rejected\" (blue) category with $s < 0$.\n", "\n", "Generalizing this example, a layer with $n$ inputs can \"select\" a different $n$-sided (soft-edged) polygon with each of its outputs. To see this in action, try [this demo](https://cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Data Flow Perspective" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The diagram below show the tensors flowing forward (left to right) in a typical fully connected graph. The main flow consists of $N$ input samples flowing from $X_0$ to $X_4$ with a number of features that varies between layers:\n", "$$\n", "X_{n+1} = \\phi\\left( X_n\\cdot W_{n+1} + \\mathbf{b}_{n+1}\\right) \\; .\n", "$$\n", "\n", "The computation of each layer's output is parameterized by the weight and bias tensors shown: note how their shapes are determined by the number of input and output features for each layer. The parameter tensors are usually randomly initialized (more on this soon) so only the input $X_0$ and target $Y$ are needed to drive the calculation (and so must be copied to GPU memory when using hardware acceleration).\n", "\n", "The final output $X_4$ is compared with the target values $Y$ to calculate a \"loss\" $\\ell(X_4, Y)$ that decreases as $X_4$ becomes more similar to $Y$ (more on this soon).\n", "\n", "![forward flow](img/NeuralNetworks/forward_flow.png)\n", "\n", "The diagram below shows the gradient (partial derivative) tensors flowing backwards (\"backpropagation\") through the same graph using the chain rule:\n", "$$\n", "\\frac{\\partial \\ell}{\\partial X_n} = \\frac{\\partial \\ell}{\\partial X_{n+1}} \\frac{\\partial X_{n+1}}{\\partial X_n}\n", "\\quad, \\quad\n", "\\frac{\\partial \\ell}{\\partial W_{n+1}} = \\frac{\\partial \\ell}{\\partial X_{n+1}} \\frac{\\partial X_{n+1}}{\\partial W_{n+1}} \\; .\n", "$$\n", "\n", "![backward flow](img/NeuralNetworks/backward_flow.png)\n", "\n", "Note that these gradient tensors are just numbers, not functions. All of these tensors occupy the (limited) GPU memory when using hardware acceleration but, in most applications, only the final output and the parameter gradients are stored (with 32-bit floating point precision).\n", "\n", "When working with large datasets, the $N$ input samples are usually broken up into fixed-size randomly subsampled \"minibatches\". Optimiztion with the resulting parameter gradients leads to the \"stochastic gradient descent\" (SGD) algorithm." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PyTorch Primer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A fully connected network can be created with a few lines in PyTorch (for a similar high-level API in Tensorflow checkout [Keras](https://www.tensorflow.org/guide/keras)):" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "torch.manual_seed(123)\n", "\n", "net = torch.nn.Sequential(\n", " torch.nn.Linear(8, 4), #0\n", " torch.nn.ReLU(), #1\n", " torch.nn.Linear(4, 3), #2\n", " torch.nn.ReLU(), #3\n", " torch.nn.Linear(3, 2) #4\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As each `Linear` layer is created, its weight and bias tensors are automatically initialized with random values, so we initially set the torch random seed for reproducible results.\n", "\n", "This construction breaks each layer into separate linear and activation \"modules\". Each module can be accessed via its index (0-4 in this example):" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sequential(\n", " (0): Linear(in_features=8, out_features=4, bias=True)\n", " (1): ReLU()\n", " (2): Linear(in_features=4, out_features=3, bias=True)\n", " (3): ReLU()\n", " (4): Linear(in_features=3, out_features=2, bias=True)\n", ")\n" ] } ], "source": [ "print(net)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Parameter containing:\n", "tensor([[-0.4228, -0.1435, -0.3521, 0.0331],\n", " [-0.0934, -0.2682, -0.0455, 0.4737],\n", " [-0.0394, 0.0159, -0.0780, 0.0786]], requires_grad=True)" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net[2].weight" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Parameter containing:\n", "tensor([0.4928, 0.0345], requires_grad=True)" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net[4].bias" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To run our network in the forward direction, we need some data with the expected number of features ($D=8$ in this example):" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "N = 100\n", "D = net[0].in_features\n", "Xin = torch.randn(N, D)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "Xout = net(Xin)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The intermediate tensors ($X_1$, $\\partial\\ell/\\partial X_1$, ...) shown in the data flow diagrams above are usually not preserved, but can be useful to help understand how a network is performing and diagnose problems. To cache these intermediate tensors, use:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "mls.torch.trace(net)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "Xout = net(Xin)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each submodule now has `input` and `output` attributes:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.equal(Xin, net[0].input)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.equal(net[0].output, net[1].input)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use the `verbose` option to watch the flow of tensors through the network:" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0: IN (100, 8) OUT (100, 4)\n", "1: IN (100, 4) OUT (100, 4)\n", "2: IN (100, 4) OUT (100, 3)\n", "3: IN (100, 3) OUT (100, 3)\n", "4: IN (100, 3) OUT (100, 2)\n" ] } ], "source": [ "mls.torch.trace(net, verbose=True)\n", "Xout = net(Xin)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To complete the computational graph we need to calculate a (scalar) loss, for example:" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor(0.1996, grad_fn=)\n" ] } ], "source": [ "loss = torch.mean(Xout ** 2)\n", "print(loss)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now back propagate gradients of this loss through the network:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "4: GRAD (100, 2)\n", "3: GRAD (100, 3)\n", "2: GRAD (100, 3)\n", "1: GRAD (100, 4)\n", "0: GRAD (100, 4)\n" ] } ], "source": [ "loss.backward()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The gradients of each layer's parameters are now computed and stored, ready to \"learn\" better parameters through (stochastic) gradient descent (or one of its variants):" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([-0.0285, -0.0276, -0.0335, 0.0646])" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net[0].bias.grad" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using `mls.torch.trace` we have also captured the gradients of the loss with respect to each module's outputs $\\partial\\ell /\\partial X_n$:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(torch.Size([100, 4]), torch.Size([100, 4]))" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net[0].output.size(), net[0].grad.size()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These gradients can be useful to study since learning of all upstream parameters effectively stops when they become vanishly small (since they multiply those parameter gradients via the chain rule)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Statistical Perspective" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The tensors behind a practical network contain so many values that it is usually not practical to examine them individually. However, we can still gain useful insights if we study their probability distributions.\n", "\n", "Build a network to process a large dataset so we have some distributions to study:" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sequential(\n", " (0): Linear(in_features=100, out_features=200, bias=True)\n", " (1): Tanh()\n", " (2): Linear(in_features=200, out_features=100, bias=True)\n", " (3): ReLU()\n", " (4): Linear(in_features=100, out_features=10, bias=True)\n", ")\n" ] } ], "source": [ "torch.manual_seed(123)\n", "\n", "N, D = 500, 100\n", "Xin = torch.randn(N, D)\n", "\n", "net = torch.nn.Sequential(\n", " torch.nn.Linear(D, 2 * D),\n", " torch.nn.Tanh(),\n", " torch.nn.Linear(2 * D, D),\n", " torch.nn.ReLU(),\n", " torch.nn.Linear(D, 10)\n", ")\n", "print(net)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that our network ends with a `Linear` module instead of an activation, which is typical for regression problems.\n", "\n", "Perform forward and backward passes to capture some values:" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "mls.torch.trace(net, verbose=True)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0: IN (500, 100) OUT (500, 200)\n", "1: IN (500, 200) OUT (500, 200)\n", "2: IN (500, 200) OUT (500, 100)\n", "3: IN (500, 100) OUT (500, 100)\n", "4: IN (500, 100) OUT (500, 10)\n", "4: GRAD (500, 10)\n", "3: GRAD (500, 100)\n", "2: GRAD (500, 100)\n", "1: GRAD (500, 200)\n", "0: GRAD (500, 200)\n" ] } ], "source": [ "Xout = net(Xin)\n", "loss = torch.mean(Xout ** 2)\n", "loss.backward()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First check that the input to the first module has the expected (unit normal) distribution:" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD7CAYAAACCEpQdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAYqElEQVR4nO3df2xT1+H38Y+DkxSa8DCQvaAsYipFjQQqVEMt2R+OJg07q7GgEZoSIoK2R2vKurCir0BpiIgYpWQsD9EQBFUTGxplGiGDJI2CKaITEgobNH+AQiliG2GCIMf8KCWUmCT294+qfkjzy3Z+2OG8X1JE7vG1fe6J74frc+89xxIKhUICABglKd4VAABMPsIfAAxE+AOAgQh/ADAQ4Q8ABiL8AcBAhD8AGMga7wpE6v79RwoGJ++WhDlz0nT3bvekvd9URTtFjraKDO0UuZHaKinJou985/lhnztlwj8YDE1q+H/znhgd7RQ52ioytFPkYm0run0AwECEPwAYiPAHAAMR/gBgIMIfAAxE+AOAgQh/ADDQlLnOH3hWpM+crudSB+96PYE+PfzycRxqBBMR/sAkey7VKs//NA4q/+j/rdTDONQHZqLbBwAMRPgDgIHo9gEmyFB9+zZbepxqAwxE+AMTZKS+fSDe6PYBAAMR/gBgIMIfAAxEnz8wRsPdtAUkMj6xwBiN14ndJ739Q14NxJ2/mAiEP5AgUpKncecvJg19/gBgIMIfAAxE+AOAgQh/ADAQ4Q8ABooo/H//+9/r9ddfl9vt1p/+9CdJUmtrqzwej5xOp2pqasLrXrlyRfn5+XK5XNqyZYv6+vokSZ2dnSoqKlJeXp7Wr1+vR48eTcDmAAAiMWr4nz9/Xv/4xz/U1NSkv/3tbzp06JA+//xzlZeXq7a2Vi0tLWpvb9eZM2ckSZs2bdLWrVt18uRJhUIh1dXVSZK2bdumNWvWyOv1atGiRaqtrZ3YLQPGWfrM6bLZ0gf9AFPRqOH/6quv6s9//rOsVqvu3r2r/v5+ffnll5o3b56ysrJktVrl8Xjk9Xp169Yt9fT0aMmSJZKk/Px8eb1e9fb26sKFC3K5XAPKgankm5u5vv0DTEUR3eSVnJysPXv26I9//KPy8vLU1dUlm80Wftxut8vn8w0qt9ls8vl8un//vtLS0mS1WgeUR2POnLSo1h8PHNVFhnaaeKa1sWnbOxaxtlXEd/hu2LBBv/jFL/TWW2+po6NDFosl/FgoFJLFYlEwGByy/Jt/n/bt5dHcvdutYDAU1XPGwmZLl9/PfZWjMamd4hlIprSxZNZnaqxGaqukJMuIB82jdvv8+9//1pUrVyRJ06dPl9Pp1D//+U/5/f7wOn6/X3a7XRkZGQPK79y5I7vdrtmzZ+vhw4fq7+8fsD4AID5GDf+bN2+qoqJCT5480ZMnT3T69GkVFBTo+vXrunHjhvr7+9Xc3CyHw6HMzEylpqaqra1NktTY2CiHw6Hk5GQtXbpULS0tkqSGhgY5HI6J3TIAwLBG7fbJzc3VpUuXtGrVKk2bNk1Op1Nut1uzZ89WaWmpAoGAcnNzlZeXJ0mqrq5WRUWFuru7tXDhQhUXF0uSKisrVVZWpv3792vu3LnavXv3xG4ZAGBYEfX5l5aWqrS0dEBZTk6OmpqaBq2bnZ2t+vr6QeWZmZk6dOhQjNUEAIwn7vAFAAMR/gBgICZzARIcM3xhIhD+wFMScT5eZvjCREisTzkQZ8PNxytFPycvkMjo8wcAAxH+AGAgwh8ADET4A4CBCH8AMBDhDwAGIvwBwECEPwAYiPAHAAMR/gBgIMIfAAxE+AOAgQh/ADAQ4Q8ABmJIZxgpEcftjxaTvGAspvanH4jRcOP2T6Ux+5nkBWNBtw8AGCii8N+7d6/cbrfcbrd27dolSXr33XfldDq1cuVKrVy5UqdOnZIktba2yuPxyOl0qqamJvwaV65cUX5+vlwul7Zs2aK+vr4J2BwAQCRGDf/W1ladPXtWx48fV0NDgy5fvqxTp06pvb1dH374oRobG9XY2Kjly5erp6dH5eXlqq2tVUtLi9rb23XmzBlJ0qZNm7R161adPHlSoVBIdXV1E75xAIChjRr+NptNZWVlSklJUXJysubPn6/Ozk51dnaqvLxcHo9He/bsUTAY1KVLlzRv3jxlZWXJarXK4/HI6/Xq1q1b6unp0ZIlSyRJ+fn58nq9E75xAIChjXrCd8GCBeHfOzo6dOLECR0+fFjnz59XZWWl0tPTVVJSovr6es2YMUM2my28vt1ul8/nU1dX14Bym80mn883zpsCAIhUxFf7XLt2TSUlJdq8ebNeeOEF7du3L/zY2rVr1dDQIJfLJYvFEi4PhUKyWCwKBoNDlkdjzpy0qNYfD0NdRofBaKfEM9X/JlO9/pMp1raKKPzb2tq0YcMGlZeXy+126+rVq+ro6JDL5ZL0dZhbrVZlZGTI7/eHn+f3+2W32weV37lzR3a7PaqK3r3brWAwFNVzxsJmS5ffzwVzo5mq7fSsh8tU/Jt8Y6p+puJhpLZKSrKMeNA8ap//7du39fbbb6u6ulput1vS12H//vvv68GDB+rt7dWRI0e0fPlyLV68WNevX9eNGzfU39+v5uZmORwOZWZmKjU1VW1tbZKkxsZGORyOWLYVADAORj3yP3DggAKBgKqqqsJlBQUFevPNN1VYWKi+vj45nU6tWLFCklRVVaXS0lIFAgHl5uYqLy9PklRdXa2Kigp1d3dr4cKFKi4unqBNAgCMZtTwr6ioUEVFxZCPFRUVDSrLyclRU1PToPLs7GzV19fHUEUAwHjjDl8AMBDhDwAGIvwBwECM6gk8YxjqGZEg/IFnDEM9IxJ0+wCAgQh/ADAQ4Q8ABiL8AcBAhD8AGIjwBwADEf4AYCCu88czLX3mdD2Xyscc+Db2CjzTnku1DnvDE2Ayun0AwECEPwAYiPAHAAMR/gBgIMIfAAxE+AOAgbjUEzAEk7zgaYQ/YAgmecHTIur22bt3r9xut9xut3bt2iVJam1tlcfjkdPpVE1NTXjdK1euKD8/Xy6XS1u2bFFfX58kqbOzU0VFRcrLy9P69ev16NGjCdgcAEAkRg3/1tZWnT17VsePH1dDQ4MuX76s5uZmlZeXq7a2Vi0tLWpvb9eZM2ckSZs2bdLWrVt18uRJhUIh1dXVSZK2bdumNWvWyOv1atGiRaqtrZ3YLQMADGvU8LfZbCorK1NKSoqSk5M1f/58dXR0aN68ecrKypLVapXH45HX69WtW7fU09OjJUuWSJLy8/Pl9XrV29urCxcuyOVyDSgHAMTHqOG/YMGCcJh3dHToxIkTslgsstls4XXsdrt8Pp+6uroGlNtsNvl8Pt2/f19paWmyWq0DygEA8RHxCd9r166ppKREmzdv1rRp09TR0RF+LBQKyWKxKBgMymKxDCr/5t+nfXt5NHPmpEW1/ngY6soIDEY7TX2J9jdMtPoksljbKqLwb2tr04YNG1ReXi63263z58/L7/eHH/f7/bLb7crIyBhQfufOHdntds2ePVsPHz5Uf3+/pk2bFl4/GnfvdisYDEX1nLGw2dLl93MNxGgSvZ0Ikcgk0t8w0T9TiWSktkpKsox40Dxqt8/t27f19ttvq7q6Wm63W5K0ePFiXb9+XTdu3FB/f7+am5vlcDiUmZmp1NRUtbW1SZIaGxvlcDiUnJyspUuXqqWlRZLU0NAgh8MR9YYCw0mfOV02W/qgHwBDG/XI/8CBAwoEAqqqqgqXFRQUqKqqSqWlpQoEAsrNzVVeXp4kqbq6WhUVFeru7tbChQtVXFwsSaqsrFRZWZn279+vuXPnavfu3RO0STAR4/YD0Rk1/CsqKlRRUTHkY01NTYPKsrOzVV9fP6g8MzNThw4diqGKAIDxxtg+AGAgwh8ADET4A4CBCH8AMBDhDwAGIvwBwECEPwAYiPAHAAMR/gBgIMIfAAzEHL6A4ZjY3UyEP2A4JnY3E90+AGAgwh8ADET4A4CB6PPHlJI+c7qeS+VjC4wVexGmFGbsAsYH3T4AYCDCHwAMRPgDgIEIfwAwEOEPAAaKOPy7u7u1YsUK3bx5U5L07rvvyul0auXKlVq5cqVOnTolSWptbZXH45HT6VRNTU34+VeuXFF+fr5cLpe2bNmivr6+cd4UAECkIgr/ixcvqrCwUB0dHeGy9vZ2ffjhh2psbFRjY6OWL1+unp4elZeXq7a2Vi0tLWpvb9eZM2ckSZs2bdLWrVt18uRJhUIh1dXVTcgGAQBGF1H419XVqbKyUna7XZL0+PFjdXZ2qry8XB6PR3v27FEwGNSlS5c0b948ZWVlyWq1yuPxyOv16tatW+rp6dGSJUskSfn5+fJ6vRO3VQCAEUV0k9eOHTsGLN+5c0fLli1TZWWl0tPTVVJSovr6es2YMUM2my28nt1ul8/nU1dX14Bym80mn883TpsAAIhWTHf4ZmVlad++feHltWvXqqGhQS6XSxaLJVweCoVksVgUDAaHLI/GnDlpsVR1TIYa4xyD0U7Prnj9bflMRS7Wtoop/K9evaqOjg65XC5JX4e51WpVRkaG/H5/eD2/3y+73T6o/M6dO+EupEjdvdutYDAUS3VjYrOly+9nNPPRTHY7EQqTKx77APte5EZqq6Qky4gHzTFd6hkKhfT+++/rwYMH6u3t1ZEjR7R8+XItXrxY169f140bN9Tf36/m5mY5HA5lZmYqNTVVbW1tkqTGxkY5HI5Y3hoAMA5iOvLPzs7Wm2++qcLCQvX19cnpdGrFihWSpKqqKpWWlioQCCg3N1d5eXmSpOrqalVUVKi7u1sLFy5UcXHx+G0FgHHH9I7PtqjC/5NPPgn/XlRUpKKiokHr5OTkqKmpaVB5dna26uvrY6gigHhgesdnG3f4AoCBCH8AMBDhDwAGYiYvJCSmawQmFnsXEhLTNQITi24fADAQ4Q8ABiL8AcBAhD8AGIjwBwADEf4AYCDCHwAMRPgDgIEIfwAwEOEPAAYi/AHAQIQ/ABiIgd0ARGW46R0lpnicSgh/AFEZbnpHiSkepxK6fQDAQIQ/ABiI8AcAA0UU/t3d3VqxYoVu3rwpSWptbZXH45HT6VRNTU14vStXrig/P18ul0tbtmxRX1+fJKmzs1NFRUXKy8vT+vXr9ejRownYFABApEYN/4sXL6qwsFAdHR2SpJ6eHpWXl6u2tlYtLS1qb2/XmTNnJEmbNm3S1q1bdfLkSYVCIdXV1UmStm3bpjVr1sjr9WrRokWqra2duC0CAIxq1PCvq6tTZWWl7Ha7JOnSpUuaN2+esrKyZLVa5fF45PV6devWLfX09GjJkiWSpPz8fHm9XvX29urChQtyuVwDygHp64nabbb0QT8AJtaol3ru2LFjwHJXV5dsNlt42W63y+fzDSq32Wzy+Xy6f/++0tLSZLVaB5QDEhO1A/ES9XX+wWBQFoslvBwKhWSxWIYt/+bfp317ORJz5qRF/Zyx4gg0MrQTnjYenwc+U5GLta2iDv+MjAz5/f7wst/vl91uH1R+584d2e12zZ49Ww8fPlR/f7+mTZsWXj9ad+92KxgMRf28WNls6fL7uV1lNGNtJ3byZ89Y9xv2vciN1FZJSZYRD5qjvtRz8eLFun79um7cuKH+/n41NzfL4XAoMzNTqampamtrkyQ1NjbK4XAoOTlZS5cuVUtLiySpoaFBDocj2rcFAIyjqI/8U1NTVVVVpdLSUgUCAeXm5iovL0+SVF1drYqKCnV3d2vhwoUqLi6WJFVWVqqsrEz79+/X3LlztXv37vHdCgBAVCIO/08++ST8e05Ojpqamgatk52drfr6+kHlmZmZOnToUIxVBACMNwZ2AzBuhhvxk9E+Ew/hD2DcDDfiJ6N9Jh7G9gEAAxH+AGAgwh8ADET4A4CBCH8AMBBX+2BSpM+crudS+bgBiYK9EZOC0TuBxEK3DwAYiPAHAAMR/gBgIMIfAAxE+AOAgQh/ADAQ4Q8ABiL8AcBA3OSFccWdvBgKk7wkHvZSjCvu5MVQmOQl8dDtAwAGIvwBwECEPwAYaEx9/mvXrtW9e/dktX79Mr/5zW/03//+V/v371dfX5/WrVunoqIiSVJra6t27typQCCgn/zkJ9q4cePYaw8AiEnM4R8KhdTR0aG///3v4fD3+XzauHGjjh07ppSUFBUUFOi1117T9773PZWXl+vQoUOaO3euSkpKdObMGeXm5o7bhgAAIhdz+P/nP/+RJP385z/XF198oZ/+9Kd6/vnntWzZMs2aNUuS5HK55PV69eqrr2revHnKysqSJHk8Hnm9XsIfAOIk5j7/L7/8Ujk5Odq3b58OHjyov/71r+rs7JTNZguvY7fb5fP51NXVNWQ5ACA+Yj7yf+WVV/TKK6+El1evXq2dO3dq/fr14bJQKCSLxaJgMCiLxTKoPBpz5qTFWtWYDXVTCgajnRCr4W7++j+zZigleVocajT1xLr/xRz+n376qXp7e5WTkyPp60DPzMyU3+8Pr+P3+2W325WRkTFkeTTu3u1WMBiKtbpRs9nS5fdz+8lovt1O/EeAaIx08xf73+hGyqmkJMuIB80xd/s8fPhQu3btUiAQUHd3t44fP67f/e53OnfunO7du6fHjx/r448/lsPh0OLFi3X9+nXduHFD/f39am5ulsPhiPWtAQBjFPOR/49+9CNdvHhRq1atUjAY1Jo1a/SDH/xAGzduVHFxsXp7e7V69Wq9/PLLkqSqqiqVlpYqEAgoNzdXeXl547YRAIDojOk6/3feeUfvvPPOgDKPxyOPxzNo3ZycHDU1NY3l7ZBAnh7Aja4eYOphYDfEhAHcgKmN4R0AwECEPwAYiPAHAAPR548RMTMX8Gxir8aIOLGLeGDax4lH+ANIOEz7OPHo8wcAAxH+AGAgwh8ADET4A4CBOOELYMrgKqDxQ/iDa/kxZXAV0Phhj8ew1/JLXM8PPKvo8wcAAxH+AGAgun0ATHmcCI4e4W8QTuziWcWJ4OiRBAZhkDaYhm8EwyP8ATyz+EYwPE74AoCBOPIHYBy6gwj/KW24E7iBJ/1KTZkWhxoBUwPdQZMc/h999JH279+vvr4+rVu3TkVFRZP59s+ckU7gcmIXiJ5J3wgmLfx9Pp9qamp07NgxpaSkqKCgQK+99ppefPHFyarClMUlmsDkMOkbwaQlSmtrq5YtW6ZZs2ZJklwul7xer371q19F9PykJMtEVm9C3jMt7TmlRtEtM1J3zf997+NBZQcqnLJ/Z/qQ649X+Xi+1lQvT8Q6JVp5ItZpPMqH+0Yw0j4bCPSpu7tnyMfG03A5NVp+WUKhUGgiKvRtH3zwgb766itt3LhRknT06FFdunRJ27dvn4y3BwA8ZdIu9QwGg7JY/v//RKFQaMAyAGDyTFr4Z2RkyO/3h5f9fr/sdvtkvT0A4CmTFv4//OEPde7cOd27d0+PHz/Wxx9/LIfDMVlvDwB4yqSd8P3ud7+rjRs3qri4WL29vVq9erVefvnlyXp7AMBTJu2ELwAgcTC2DwAYiPAHAAMR/gBgIMIfAAxE+I/is88+06JFi+JdjYTW1tam1atXa+XKlVq3bp1u3boV7yollI8++kivv/66nE6nDh8+HO/qJKy9e/fK7XbL7XZr165d8a7OlPDb3/5WZWVlMT2X8B/B48ePtX37dvX29sa7Kglt06ZNeu+999TY2CiPx6P33nsv3lVKGN8MaPiXv/xFDQ0NOnLkiP71r3/Fu1oJp7W1VWfPntXx48fV0NCgy5cv69SpU/GuVkI7d+6cjh8/HvPzCf8RVFVVad26dfGuRkJ78uSJfv3rXys7O1uS9NJLL+n27dtxrlXieHpAwxkzZoQHNMRANptNZWVlSklJUXJysubPn6/Ozs54VythffHFF6qpqdFbb70V82sQ/sM4ffq0enp6lJeXF++qJLSUlBStXPn1PAHBYFB79+7Vj3/84zjXKnF0dXXJZrOFl+12u3w+XxxrlJgWLFigJUuWSJI6Ojp04sQJ5ebmxrlWiWvr1q3auHGjZs6cGfNrGD9I/IkTJ7Rz584BZS+88IK6u7t18ODB+FQqQQ3XVgcPHtSTJ09UVlamvr4+lZSUxKmGiYcBDaNz7do1lZSUaPPmzfr+978f7+okpKNHj2ru3LnKycnRsWPHYn4d7vAdwtGjR/XBBx/o+eeflyR9/vnnys7O1uHDh5WWlhbn2iWeR48eaf369Zo1a5aqq6uVkpIS7yoljOPHj+vTTz/Vjh07JEn79u1TKBSKeB4Lk7S1tWnDhg0qLy+X2+2Od3US1s9+9jP5/X5NmzZNDx480FdffaVVq1apvLw8qtch/CPw0ksv6erVq/GuRsL65S9/qTlz5mjbtm1KSqIn8Wk+n0+FhYWqr6/X9OnTVVBQoO3btzOu1bfcvn1bb7zxhmpqapSTkxPv6kwZx44d0/nz51VVVRX1c43v9sHYfPbZZzp9+rRefPFFvfHGG5K+7tf+wx/+EOeaJQYGNIzMgQMHFAgEBoRYQUGBCgsL41irZxtH/gBgIL6jA4CBCH8AMBDhDwAGIvwBwECEPwAYiPAHAAMR/gBgIMIfAAz0vwSUhZHaAoxkAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[0].input.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How does torch initialize the parameters (weights and biases) for each layer?" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAYtElEQVR4nO3de3BU5f3H8c/mQgqFXyl011BqkVErlhGpUCylTbRIEhJWYCsVFdFhULAtZagNRUih0lqjQ8VSQOkQZUq1U24hAUOg1opSYMDUcnFi6yAwNWiyuSgJGpLsPr8/mGwJue0mu5vl4f2ayZB9ztlzvufJOZ9z2bMHhzHGCABgpbieLgAAEDmEPABYjJAHAIsR8gBgMUIeACxGyAOAxQh5ALBYQk8XcKmamnPy+0O/dX/gwL6qqqqLQEXdQ12hi9XaqCs01BWartYVF+fQF7/4+XaHx1zI+/2mSyHf/N5YRF2hi9XaqCs01BWaSNTF5RoAsBghDwAWI+QBwGKEPABYjJAHAIsR8gBgMUIeACwWc/fJA0C49Pu/3vpcUuuY6/d/vVV79rMeqCj6CHkA1vpcUoLcjxa0at/x28mq7YF6egKXawDAYoQ8AFiMkAcAixHyAGAxQh4ALHZF3F3T3m1U9eebrpjbqABcma6IkOc2KgBXqisi5IHLAWeciARCvg3tbWznG3xK6hXfqp2N8H8Iqq7jjDN2tbdeS7G/bhPybehoY2Mj7FhPBRU7F0RSe+u1FPvbPyEPK3AUDLSNkAeucJwFRUas9CshD1zhOAuKjFjpV0IeXdLRB1HoGH3XdbFydHw5YU2LIZfTCtzRUQo6Fq6+u3h9cTr7BdpjcX0Jl1g5Or6cBB3yTz31lGpqapSbm6vS0lItWbJE586d0+jRo/X4448rISFBZ86cUXZ2tqqqqjR06FCtWLFCn//85yNZf7c0NPpabBw9LRZXYI46O9dTO+dIry8dbR8270hsE9TWe+DAAeXn5+u2226TJGVnZ+vXv/61Ro4cqcWLF2vTpk2699579fjjj+vee+9VVlaW1qxZo7Vr1yo7OzuS9XdLr8R4jkY7wRF752Jx5xwO7W0fUviW7XI6e71cdRryH3/8sVauXKm5c+fq3XffVVlZmerr6zVy5EhJksfj0apVqzRt2jQdPnxYa9asCbTPmDEjpkP+cmfDBsJ/z3Zls3UHGUs6DfmlS5dqwYIF+vDDDyVJFRUVcjqdgeFOp1Pl5eWqqalR3759lZCQ0KIdkWPDBmLDMqBzsXbZr71LUZfTAVKwOuz1zZs3a9CgQRo7dqy2bdsmSfL7/XI4HIFxjDFyOByBfy926etgDBzYN+T3NOvJ6+sdzTscdYU6jWDGj3Z/RWIZojmdrkyrJ5ahvQBraPSpV2Lrx3JEsqaLxwn1sl8k+7qjS7Wf68LfLFy1RmKb7DDki4qK5PV6NXnyZH3yySf69NNP5XA45PV6A+NUVlbK5XJpwIABqq2tlc/nU3x8vLxer1wuV8gFVVXVye83Ib/P6ewnr7ftY79ohFlH825vWFvjhjL9UMfvSl3h7LtwL0N3ptOVZ5G012fh/LuFYzodBVi46mmvpnAesfdEX3d1RxiuWkNZ55vFxTk6PDju8K/x4osvBn7ftm2bDh06pCeffFKTJk1SSUmJRo0apYKCAqWkpCgxMVGjR49WUVGR3G63tm/frpSUlJALBqLhcn4WSSzo6M6by/mD+q7cjBFrd+ldqku73BUrVignJ0d1dXUaPny4Zs6cKUlatmyZFi1apOeee06DBg3SM888E9ZiY1VH1/cAG3Fn2v+E2hcdXU6LhKBD3uPxyOPxSJKGDRumLVu2tBpn8ODB2rhxY/iqu0ywwseurhxlhWsjjPUjPPSMaOdF7HzcDURAVzaocG2E7PwRCwh5C9l8e1is3YqH2MBZU/vYWnpAqEEV6grc0RHk5f6BIt/ARVs4a2ofId8DQg0qVuArG0ep6A5rQp4NAbZiJ4/usCbkO3uYUk9o/mIFO5/QsdMGwsOakI9Fl9MRWKx9oHk59R3Cj518+MTOVo0exQeaiCXs5MMnrqcLAABEDkfyV5BLT4GjeTrM6TfQMwj5K0hPfjjN6TfQMwh54ArB2dSViZAHgnS5hyRnU1cmQh4IEiGJyxF31wCAxQh5ALAYIQ8AFiPkAcBihDwAWIyQBwCLEfIAYDFCHgAsRsgDgMUIeQCwGCEPABYj5AHAYoQ8AFiMkAcAixHyAGAxQh4ALEbIA4DFCHkAsBghDwAWI+QBwGKEPABYjJAHAIsR8gBgMUIeACwWVMj/7ne/U2ZmprKysvTiiy9Kkvbv3y+32620tDStXLkyMG5paak8Ho/S09O1ZMkSNTU1RaZyAECnOg35Q4cO6eDBgyosLNTWrVu1ceNGvfvuu1q8eLHWrl2roqIiHT9+XHv37pUkZWdna+nSpdq9e7eMMdq0aVPEFwIA0LZOQ37MmDH64x//qISEBFVVVcnn8+ns2bMaMmSIrr76aiUkJMjtdqu4uFhlZWWqr6/XyJEjJUkej0fFxcURXwgAQNuCulyTmJioVatWKSsrS2PHjlVFRYWcTmdguMvlUnl5eat2p9Op8vLy8FcNAAhKQrAj/uQnP9FDDz2kuXPn6tSpU3I4HIFhxhg5HA75/f4220MxcGDfkMYHAFs4nf3CPs1OQ/7EiRNqaGjQjTfeqN69eystLU3FxcWKj48PjOP1euVyuZScnCyv1xtor6yslMvlCqmgqqo6+f0mpPdIkekcAIgmr7c25PfExTk6PDju9HLNBx98oJycHDU0NKihoUF/+9vfNH36dJ08eVKnT5+Wz+fTzp07lZKSosGDByspKUklJSWSpIKCAqWkpIRcNAAgPDo9kk9NTdXRo0c1ZcoUxcfHKy0tTVlZWRowYIDmzZun8+fPKzU1VRkZGZKkFStWKCcnR3V1dRo+fLhmzpwZ8YUAALQtqGvy8+bN07x581q0jR07VoWFha3GHTZsmLZs2RKe6gAA3cI3XgHAYoQ8AFiMkAcAixHyAGAxQh4ALEbIA4DFCHkAsBghDwAWI+QBwGKEPABYjJAHAIsR8gBgMUIeACxGyAOAxQh5ALAYIQ8AFiPkAcBihDwAWIyQBwCLEfIAYDFCHgAsRsgDgMUIeQCwGCEPABYj5AHAYoQ8AFiMkAcAixHyAGAxQh4ALEbIA4DFCHkAsBghDwAWI+QBwGKEPABYjJAHAIsR8gBgMUIeACxGyAOAxYIK+dWrVysrK0tZWVl6+umnJUn79++X2+1WWlqaVq5cGRi3tLRUHo9H6enpWrJkiZqamiJTOQCgU52G/P79+7Vv3z7l5+dr+/bteuedd7Rz504tXrxYa9euVVFRkY4fP669e/dKkrKzs7V06VLt3r1bxhht2rQp4gsBAGhbpyHvdDq1aNEi9erVS4mJibr22mt16tQpDRkyRFdffbUSEhLkdrtVXFyssrIy1dfXa+TIkZIkj8ej4uLiiC8EAKBtCZ2NcP311wd+P3XqlHbt2qUZM2bI6XQG2l0ul8rLy1VRUdGi3el0qry8PKSCBg7sG9L4AGALp7Nf2KfZacg3e++99zRnzhwtXLhQ8fHxOnXqVGCYMUYOh0N+v18Oh6NVeyiqqurk95uQ3iNFpnMAIJq83tqQ3xMX5+jw4DioD15LSkr04IMP6tFHH9XUqVOVnJwsr9d7UWFeuVyuVu2VlZVyuVwhFw0ACI9OQ/7DDz/Uj370I61YsUJZWVmSpJtvvlknT57U6dOn5fP5tHPnTqWkpGjw4MFKSkpSSUmJJKmgoEApKSmRXQIAQLs6vVyTl5en8+fPKzc3N9A2ffp05ebmat68eTp//rxSU1OVkZEhSVqxYoVycnJUV1en4cOHa+bMmZGrHgDQoU5DPicnRzk5OW0OKywsbNU2bNgwbdmypfuVAQC6jW+8AoDFCHkAsBghDwAWI+QBwGKEPABYjJAHAIsR8gBgMUIeACxGyAOAxQh5ALAYIQ8AFiPkAcBihDwAWIyQBwCLEfIAYDFCHgAsRsgDgMUIeQCwGCEPABYj5AHAYoQ8AFiMkAcAixHyAGAxQh4ALEbIA4DFCHkAsBghDwAWI+QBwGKEPABYjJAHAIsR8gBgMUIeACxGyAOAxQh5ALAYIQ8AFiPkAcBihDwAWCzokK+rq9OkSZP0wQcfSJL2798vt9uttLQ0rVy5MjBeaWmpPB6P0tPTtWTJEjU1NYW/agBAUIIK+SNHjuiee+7RqVOnJEn19fVavHix1q5dq6KiIh0/flx79+6VJGVnZ2vp0qXavXu3jDHatGlTxIoHAHQsqJDftGmTli1bJpfLJUk6evSohgwZoquvvloJCQlyu90qLi5WWVmZ6uvrNXLkSEmSx+NRcXFx5KoHAHQoIZiRnnjiiRavKyoq5HQ6A69dLpfKy8tbtTudTpWXl4epVABAqIIK+Uv5/X45HI7Aa2OMHA5Hu+2hGDiwb1dKAoDLntPZL+zT7FLIJycny+v1Bl57vV65XK5W7ZWVlYFLPMGqqqqT329CrikSnQMA0eT11ob8nrg4R4cHx126hfLmm2/WyZMndfr0afl8Pu3cuVMpKSkaPHiwkpKSVFJSIkkqKChQSkpKV2YBAAiDLh3JJyUlKTc3V/PmzdP58+eVmpqqjIwMSdKKFSuUk5Ojuro6DR8+XDNnzgxrwQCA4IUU8q+99lrg97Fjx6qwsLDVOMOGDdOWLVu6XxkAoNv4xisAWIyQBwCLEfIAYDFCHgAsRsgDgMUIeQCwGCEPABYj5AHAYoQ8AFiMkAcAixHyAGAxQh4ALEbIA4DFCHkAsBghDwAWI+QBwGKEPABYjJAHAIsR8gBgMUIeACxGyAOAxQh5ALAYIQ8AFiPkAcBihDwAWIyQBwCLEfIAYDFCHgAsRsgDgMUIeQCwGCEPABYj5AHAYoQ8AFiMkAcAixHyAGAxQh4ALEbIA4DFCHkAsFhEQn7Hjh3KzMxUWlqaXnrppUjMAgAQhIRwT7C8vFwrV67Utm3b1KtXL02fPl233nqrrrvuunDPCgDQibCH/P79+/Wtb31L/fv3lySlp6eruLhYP/7xj4N6f1yco8vzdn2xd8jDaO/5ecdaeyzWFGvtsViTDX3Rlfzr7D0OY4wJeaodWLdunT799FMtWLBAkrR582YdPXpUv/rVr8I5GwBAEMJ+Td7v98vh+N+exRjT4jUAIHrCHvLJycnyer2B116vVy6XK9yzAQAEIewh/+1vf1sHDhxQdXW1PvvsM+3Zs0cpKSnhng0AIAhh/+D1qquu0oIFCzRz5kw1Njbqrrvu0ogRI8I9GwBAEML+wSsAIHbwjVcAsBghDwAWI+QBwGKEPABYLOZD/syZM7rvvvuUkZGhRx55ROfOnWt33H/84x964IEHAq+NMXrqqaeUkZGhzMxMlZSUBIa98MILysjIUHp6uvbs2RORuhoaGpSdna2JEydq6tSpOnHihCRp6dKlmjx5cuDnxhtvVHFxsSRp/PjxLYZ9+OGHUaursbFRt9xyS4v5+3y+DvsxGnWdO3dO8+fPl9vtltvt1iuvvBJ4T1f7q7OH6JWWlsrj8Sg9PV1LlixRU1NTh8tx9uxZPfzww5o4caLuu+++Ft8VCUVX6yopKdFdd92lyZMn64EHHlBZWZkk6dChQ7r11lsD/fPYY49Fta78/Hx95zvfCcx/5cqVkkLbrsNdV1VVVYt15nvf+56+8Y1vSIpefzVbuHChtm3bFngdkfXLxLiHH37Y7Ny50xhjzOrVq83TTz/dahyfz2fy8vLMmDFjzIwZMwLtu3btMg899JDx+Xzm/fffNxMmTDCNjY3myJEjZvLkyaa+vt5UVlaa8ePHm5qamrDXtX79evOLX/zCGGPMoUOHzLRp01qNs3nzZjNr1izj9/tNdXW1SU9PD6mOcNZ17NgxM2vWrFbjt9eP0arrmWeeMbm5ucYYYyorK824ceOM1+vtcn999NFH5vbbbzc1NTXm3Llzxu12m/fee6/FOFlZWebtt982xhjz2GOPmZdeeqnD5Xj88cfNunXrjDHG5Ofnm/nz50e1rttvv92UlpYaYy6sU3PnzjXGGJOXl2eef/75kGsJV13Lly83O3bsaDXNYNaHSNbVzOfzmRkzZpjCwkJjTPT666OPPjJz5swxI0aMMFu3bg20R2L9iukj+cbGRh0+fFjp6emSJI/HEzjivdiJEyd04sSJVs/H2bt3rzIzMxUXF6ehQ4dq0KBBevvtt/XGG29owoQJSkpK0sCBAzVmzBi9/vrrYa/r9ddf15133ilJ+uY3v6nq6mqdOXMmMLympkarVq3S8uXL5XA4dOzYMRljNH36dE2dOlW7du0KuqZw1HXs2DFVV1fL4/HoBz/4gQ4dOiSp/X6MVl1jxozR/fffL0kaOHCg+vfvr8rKyi7318UP0evTp0/gIXrNysrKVF9fr5EjR7aot6PleP311+V2uyVJkyZN0htvvKHGxsag+6g7dTU0NGj+/PkaNmyYJOmGG24InNEcO3ZM+/btk9vt1ty5c0M+M+xOXc3zz8/Pl9vt1s9+9jN98sknQa8Pkayr2datW9W7d+/A3y4a/SVdONIfP368Jk6cGGiL1PoV0yFfU1Ojvn37KiHhwne2nE6nysvLW413/fXX64knntAXvvCFFu0VFRUtHqngdDr10Ucftdse7roqKirkdDrbnc+GDRuUlZWlwYMHS7pwueK73/2uNmzYoN///vfKzc0NXLKIRl0Oh0Pjx4/XX/7yF/3yl7/UggULVF1d3eP9NW7cOH35y1+WJBUVFamhoUHXXXddl/vr0vm4XK4W9bRVR3l5eYfLcfF7EhIS1LdvX1VXVwfdR92pq1evXpo8ebKkC8+OWr16te644w5JUr9+/XT//fdrx44dSk1NDTw4MBp1Nf/+wx/+UIWFhRo0aJCWL18e9PoQybokyefz6fnnn9ejjz4aaItGf0nS7NmzNW3atBZtkVq/wv6N167atWuXnnzyyRZtQ4YMafVws1AedtbWw9Li4uLk9/tbjRsX1/b+rjt1mUseztY8/+batm7dqi1btgSG33HHHYGN8ytf+YomTJigffv26dprr41KXdOnTw+0ff3rX9eIESP0z3/+s91+bEuk+qt52r/5zW+0fv16JSQkhNRfF+vsIXrtDb90vPaWo63ag9HVupo1NDRo0aJFampq0pw5cyRJy5cvDwy/55579Nvf/la1tbXq169fVOpas2ZNoH327NmaMGGCFi5c2K3tOhx1SdKbb76pa665RjfccEOgLRr91Z5IrV8xE/ITJ05sceoiXTh9ufXWW+Xz+RQfHx/yw86Sk5NVUVEReF1ZWSmXy9XmQ9SGDh0a9rquuuoqVVRU6Ktf/WqL+UvS22+/rWuuuUbJycmB8f/+97/rS1/6km666aZAW/NePRp1bd++Xbfcckug3RijxMTEdvsxmv21ceNG5eXlKS8vL7BRhtJfF0tOTtZbb70VeH1pPZeuH811DBgwQLW1tW0uh8vlUmVlpZKTk9XU1KRz584F/k+FYHW1LunCh9OPPPKI+vfvr+eee06JiYny+/1at26dHn74YcXHxwfed/HvkayrtrZWW7du1YMPPijpwvoUHx/fYT9Go65mr776qjIzMwOvo9Vf7YnU+hXTl2sSExM1evRoFRUVSZK2b98e0sPOUlJStGPHDvl8Pp0+fVqnTp3STTfdpJSUFO3Zs0efffaZqqurdfDgQY0dOzbsdaWmpqqgoECS9NZbbykpKSlw2eFf//qXRo0a1WL8srIyrVmzRn6/X5WVlXrttdd02223Ra2uf//733rhhRckSe+//75KS0s1atSodvsxWnW9+uqr2rBhg/785z+3OOrqan919hC9wYMHKykpKXAXUUFBgVJSUjpcjtTUVG3fvl3ShUtKo0ePVmJiYtB91J26JCk7O1tDhgzRs88+q169ekm6cHb617/+Vbt37w7Ue/PNN6tPnz5RqatPnz5av369jhw5Ikn605/+pAkTJnR7uw5Hf0kXtsHRo0cHXkerv9oTsfUr6I9oe8gHH3xgZsyYYSZOnGhmzZplPv74Y2OMMS+//LJ59tlnW4x78ODBFnfX+P1+k5ubazIzM01mZqZ58803A8Py8vJMZmamSUtLM/n5+RGpq76+3ixcuNBkZmaaKVOmmOPHjwfev2zZMvPyyy+3mGZjY6PJyckxEydONOnp6eaVV16Jal21tbVm3rx5Jisry0yaNMkcOHDAGNNxP0ajLrfbbcaNG2fuvPPOwM/Ro0e71V+FhYUmKyvLpKWlmT/84Q/GGGNmz55tjh49aowxprS01Hz/+9836enp5qc//ak5f/58h8tRU1Nj5syZYzIzM83dd99t/vvf/4bcR12t65133jFf+9rXTGZmZqB/Zs+ebYwx5j//+Y+5++67TWZmppkxY4Y5c+ZM1OoyxpjDhw+bKVOmmIyMDDN37lxz9uzZDvsxWnUZY8yIESNMfX19i+lFq7+a/fznP29xd00k1i8eUAYAFovpyzUAgO4h5AHAYoQ8AFiMkAcAixHyAGAxQh4ALEbIA4DFCHkAsNj/Ax/F1QJoplNmAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[0].weight.data.reshape(-1), bins=50);" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAD7CAYAAABOi672AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAY+ElEQVR4nO3de3BU5cHH8V8Skgx5SaXE5VI7FapWbQcogtAUC6UQQjaEYIQCCtoyyKWKSFGKFtHSYsEOxSq20iGWKYitFEJIBQuWi1BguBSBKqUMDA4XgZCghGuW3ef9g2bfhGQvZ7N74PH9fmacIWf3nOd3nnPOz8OS3U0yxhgBAG54ydc7AAAgOhQ2AFiCwgYAS1DYAGAJChsALEFhA4AlKGwAsESTRA9w5sx5BQLOf9U7K6uZKirOJSBR45DLGXI5d6NmI5czseZKTk7SF7/4Pw0+lvDCDgRMTIVds+6NiFzOkMu5GzUbuZyJdy5eEgEAS1DYAGAJChsALEFhA4Aloirs0tJS5efnKz8/X7NmzUp0JgBAAyIW9sWLFzVjxgwtXLhQpaWl2rFjhzZv3uxGNgBALREL2+/3KxAI6OLFi7py5YquXLmi9PR0N7IBAGpJiuYLDBYuXKhf/epXatq0qe699169+uqrSkpKciNfo1X7/EpLTXH8GAD7hbrGbb32I75x5t///reWLl2qdevWKTMzU0899ZSKi4s1atSoqAaoqDgX0y+PezyZKi+vcrxeQ9spmFTa4GNlswsdjxGvXPFGLmdu1FzSjZvNxlyhrv9Yrv145gonOTlJWVnNGn4s0sqbNm1Sdna2srKylJaWpqKiIm3bts1xCABA40Qs7LvuukubN2/WhQsXZIzR2rVr1b59ezeyAQBqifiSyH333aePPvpIRUVFSk1NVfv27TV69Gg3sgEAaonqw59Gjx5NSQPAdcY7HQHAEhQ2AFiCwgYAS1DYAGAJChsALEFhA4AlKGwAsASFDQCWoLABwBIUNgBYgsIGAEtQ2ABgCQobACxBYQOAJShsALAEhQ0Aloj4BQZLlizRokWLgj8fPXpUhYWFmjZtWkKDAQDqiljYgwcP1uDBgyVJBw4c0GOPPabHH3884cEAAHU5eknkhRde0MSJE9WiRYtE5QEAhBB1YW/evFmXLl1SXl5eIvMAAEKI6kt4JelPf/qTfvjDHzoeICurmeN1ang8mTGvm8gx3MgVi1hzVfv8SktNcfxYtNyYr1A5w+W/Xscxmqyft3Ms0eJ1HTs9jyJdO/Ger6gKu7q6Wtu3b9fMmTMdD1BRcU6BgHG8nseTqfLyKsfrNbSdcJyOEa9c8daYXB5PpgomlTb4WNnswkbtr1vzFWofQuW/nscxUtbP4zmWSOFyhbv+Q50XTs+jeF87yclJIW90o3pJZP/+/Wrbtq0yMjIcDw4AiI+oCvvIkSNq3bp1orMAAMKI6iURr9crr9eb6CwAgDB4pyMAWILCBgBLUNgAYAkKGwAsQWEDgCUobACwBIUNAJagsAHAEhQ2AFiCwgYAS1DYAGAJChsALEFhA4AlKGwAsASFDQCWoLABwBIUNgBYIqrCXrt2rYqKipSXl6df/OIXic4EAGhAxMI+cuSInn/+ef32t7/VihUr9NFHH2nDhg1uZAMA1BLxOx3XrFkjr9cb/BLeOXPmKD09PeHBAAB1RbzD/vjjj+X3+zV27FgVFhZq8eLFuummm9zIBgCoJeIdtt/v144dO7Rw4UJlZGRo3LhxKikpUVFRUVQDZGU1izmcx5NZb1m1z6+01JSol8d7jIbWiXemWDW0LzfCdsOt78bchRrf6X65mTVRx7KxIuW6XtdCLPPldB03xogkYmHffPPNys7OVosWLSRJffr00Z49e6Iu7IqKcwoEjONgHk+mysurGlxeMKm03vKy2YUhnx+O0zEaWsdppkQINV/RrhtOY/YhUq54zV24fQh1jJ3ul1tZG3MsEymaXNfjWgiXK5bzIl7PD7VOJMnJSSFvdCO+JNKrVy9t2rRJZ8+eld/v18aNG/WNb3zDcQgAQONEvMPu2LGjRo0apQcffFA+n0/du3fXAw884EY2AEAtEQtbkgYNGqRBgwYlOgsAIAze6QgAlqCwAcASFDYAWILCBgBLUNgAYAkKGwAsQWEDgCUobACwBIUNAJagsAHAEhQ2AFiCwgYAS1DYAGAJChsALEFhA4AlKGwAsASFDQCWiOobZ0aMGKHKyko1aXL16dOnT1fHjh0TGgwAUFfEwjbG6PDhw1q3bl2wsAEA7ov4ksihQ4ckSSNHjtSAAQO0aNGihIcCANQX8Zb57Nmzys7O1nPPPSefz6eHH35Y7dq1U/fu3d3IBwD4r4iF3alTJ3Xq1Cn486BBg7Rhw4aoCzsrq1nM4TyezIQ+v9rnd7yO03GcPLfa51daakrUyxszlhON3W6s68drfxraTqhjf9nnV3oUcx3NGLGo2U7t7TX2vIinWPcz1HzHax8SfR27NUYkEQt7x44d8vl8ys7OlnT1NW0nr2VXVJxTIGAcB/N4MlVeXtXg8lCcPj8tNUUFk0rrLS+bXRg227XjOM0UiseTGTJPpO2Emq9oxw0n1u3WbDvc+vGcOyfbCTfXDS2veczJGKFEynrtnDXmvIinaM6xUPsW7lpr7D6EyxXPvnD6/FDrRJKcnBTyRjfia9hVVVV66aWXdPnyZZ07d04lJSXKyclxHAIA0DgRb5V79eql3bt3a+DAgQoEAnrwwQfrvEQCAHBHVK9tPPnkk3ryyScTnQUAEAbvdAQAS1DYAGAJChsALEFhA4AlKGwAsASFDQCWoLABwBIUNgBYgsIGAEtQ2ABgCQobACxBYQOAJShsALAEhQ0AlqCwAcASFDYAWILCBgBLRF3Ys2bN0pQpUxKZBQAQRlSFvWXLFpWUlCQ6CwAgjIiF/emnn2rOnDkaO3asG3kAACFELOxp06Zp4sSJ+sIXvuBGHgBACGG/NX3JkiVq06aNsrOztWzZspgGyMpqFtN61T6/PJ5MR+s4fX6soh0n1D5U+/xKS02JerxQ27ns8yu91nZqnuN0+5E0dl5jXT9exzPR50UijnO0md0652uPd6OdX7Fuw41+iffxCVvYK1euVHl5uQoLC/XZZ5/pwoULevHFF/Xss89GPUBFxTkFAsZxMI8nUwWTSustL5tdGHKd8vKqBrcTb9eOE2qMtNSUkPvgJGu47TjZfiiR5sjJthradrj1w40dr31I9HnhxnEOpTHHxqmaYxnq2pTCZw2lsfsQ7hyL53kRy3kUy74lJyeFvNENW9h/+MMfgn9etmyZtm3b5qisAQDxw+9hA4Alwt5h11ZUVKSioqJEZgEAhMEdNgBYgsIGAEtQ2ABgCQobACxBYQOAJShsALAEhQ0AlqCwAcASFDYAWILCBgBLUNgAYAkKGwAsQWEDgCUobACwBIUNAJagsAHAEhQ2AFgiqsL+zW9+I6/Xq/z8/Drf8wgAcE/Erwjbtm2btm7dqhUrVujKlSvyer3q2bOnvvrVr7qRDwDwXxHvsLt27ao//vGPatKkiSoqKuT3+5WRkeFGNgBALVG9JJKamqpXXnlF+fn5ys7OVqtWrRKdCwBwjai/Nf2JJ57Qo48+qrFjx+rtt9/WkCFDolovK6tZzOGc8ngyE7r9ap9faakpcRkn0Vkb2n5Nfieqff5Gb8vjyXQ8djzGDbcdtyRybKdz5HR5Q+K9P6H24bLPr/QGMoVa/oXmGQ0udzpuOLHse7znK2JhHzx4UNXV1br77rvVtGlT9e3bV/v37496gIqKcwoEjONgsexoeXlVXLYTSlpqigomldZbXja70PG2Ep011Pad5g+3zw2N0dCY5eVVjsd2Om6ouYvnMYtFIo9zLHMUr2MZL+H2IV7LnY4bSizHMpp5vVZyclLIG92IL4kcPXpUU6dOVXV1taqrq/X3v/9dnTt3dhwCANA4Ee+we/bsqT179mjgwIFKSUlR3759lZ+f70Y2AEAtUb2GPX78eI0fPz7RWQAAYfBORwCwBIUNAJagsAHAEhQ2AFiCwgYAS1DYAGAJChsALEFhA4AlKGwAsASFDQCWoLABwBIUNgBYgsIGAEtQ2ABgCQobACxBYQOAJShsALBEVN84M3fuXK1atUrS1a8Mmzx5ckJDAQDqi3iHvXnzZm3atEklJSVavny5PvzwQ61Zs8aNbACAWiLeYXs8Hk2ZMkVpaWmSpNtuu03Hjx9PeDAAQF0RC/uOO+4I/vnw4cNatWqV3nrrrYSGAgDUF9Vr2JJ04MABjRkzRpMnT1bbtm2jHiArq1ksuRyr9vnl8WS6MlY8JDKrW3PR0BjVPr/SUlMiPi9WNh3n65U1lnGjfb4tc58Isex7vOcrqsLeuXOnnnjiCT377LPKz893NEBFxTkFAsZxMKc7mpaaooJJpfWWl80udDy2G8rLq+oti9fBdWsuQu1DIse26Thfr6yxjNvQsbyWx5Op8vKq/7elHcs1G828Xis5OSnkjW7Ewv7kk0/02GOPac6cOcrOznY8OAAgPiIWdnFxsS5fvqyZM2cGlw0dOlTDhg1LaDAAQF0RC3vq1KmaOnWqG1kAAGHwTkcAsASFDQCWoLABwBIUNgBYgsIGAEtQ2ABgCQobACxBYQOAJShsALAEhQ0AlqCwAcASFDYAWILCBgBLUNgAYAkKGwAsQWEDgCUobACwRNSFfe7cOfXv319Hjx5NZB4AQAhRFfbu3bs1bNgwHT58OMFxAAChRFXYb7/9tp5//nm1bNky0XkAACFE/BJeSZoxY0bMA2RlNYt53c+rap9fHk/m9Y7RaJ+HfUDo8/Gyz6/01JQ6y/6/HvNYr9l4z1dUhd0YFRXnFAgYx+t9nk+MtNQUFUwqrbe8bHbhdUgTu/LyqnrLPs/H7fMq3Pn4eThP4yHWa7ahaySS5OSkkDe6/JYIAFiCwgYAS1DYAGAJR69hr127NlE5AAARcIcNAJagsAHAEhQ2AFiCwgYAS1DYAGAJChsALEFhA4AlKGwAsASFDQCWoLABwBIUNgBYgsIGAEtQ2ABgCQobACxBYQOAJShsALAEhQ0AloiqsMvKyuT1etW3b1+9+eabic4EAGhAxK8IO3nypObMmaNly5YpLS1NQ4cOVbdu3XT77be7kQ8A8F8RC3vz5s361re+pebNm0uScnNz9e677+rxxx+PaoDk5KSYw7X8YtOELndjDNuXh3ss1LG90faB88W95Tdipus5F7H0X7h1kowxJtzK8+bN04ULFzRx4kRJ0pIlS7Rnzx79/Oc/dxwEABC7iK9hBwIBJSX9X+MbY+r8DABwR8TCbt26tcrLy4M/l5eXq2XLlgkNBQCoL2Jhf/vb39aWLVtUWVmpixcvavXq1erRo4cb2QAAtUT8R8dWrVpp4sSJevjhh+Xz+TRo0CB16NDBjWwAgFoi/qMjAODGwDsdAcASFDYAWILCBgBLUNgAYAlXC/v48eN66KGH1K9fP40bN07nz58P+dx//OMfeuSRR4I/G2M0a9Ys9evXT16vVzt37gw+9sYbb6hfv37Kzc3V6tWrE5aturpaTz/9tPLy8nT//ffr4MGDkqRp06apsLAw+N/dd9+td999V5LUu3fvOo998sknruXy+Xy655576ozv9/vDzqUbuc6fP68JEyaooKBABQUFeuedd4LrxDpfkT6gbN++fSoqKlJubq5++tOf6sqVK2H34+zZsxo9erTy8vL00EMP1XkvghOx5tq5c6cGDRqkwsJCPfLIIzp27Jgkadu2berWrVtwfp555hlXc5WUlOi+++4Ljj9nzhxJzq7tRGSrqKioc95873vfU6dOnSS5N2c1Jk+erGXLlgV/jus5Zlw0evRo89e//tUYY8zcuXPNSy+9VO85fr/fFBcXm65du5rhw4cHl69atco8+uijxu/3m0OHDpmcnBzj8/nM7t27TWFhobl06ZI5ffq06d27tzlz5kxCss2fP98899xzxhhjtm3bZgYPHlzvOUuWLDEjR440gUDAVFZWmtzcXMdZ4pVr7969ZuTIkfWeH2ou3cr161//2sycOdMYY8zp06dN9+7dTXl5eczzdeLECdOrVy9z5swZc/78eVNQUGAOHDhQ5zn5+flm165dxhhjnnnmGfPmm2+G3Y+f/exnZt68ecYYY0pKSsyECRNczdWrVy+zb98+Y8zVc2rs2LHGGGOKi4vN66+/7jhLvHJNnz7dlJWV1dtmNOdDorPV8Pv9Zvjw4WbFihXGGPfm7MSJE2bMmDGmQ4cOZunSpcHl8TzHXLvD9vl82r59u3JzcyVJRUVFwbvQ2g4ePKiDBw/W+6ySDRs2yOv1Kjk5We3atVObNm20a9cuvf/++8rJyVF6erqysrLUtWtXrV+/PiHZ1q9frwEDBkiS7r33XlVWVur48ePBx8+cOaNXXnlF06dPV1JSkvbu3StjjIYOHar7779fq1atcjXX3r17VVlZqaKiIn3/+9/Xtm3bJIWeS7dyde3aVSNGjJAkZWVlqXnz5jp9+nTM81X7A8oyMjKCH1BW49ixY7p06ZK++c1v1skbbj/Wr1+vgoICSVL//v31/vvvy+fzRT1HjclVXV2tCRMm6K677pIk3XnnncG/aezdu1ebNm1SQUGBxo4d6/hvbI3JVTN+SUmJCgoK9NRTT+mzzz6L+nxIdLYaS5cuVdOmTYPHz405k67egffu3Vt5eXnBZfE+x1wr7DNnzqhZs2Zq0uTqe3U8Ho9OnjxZ73l33HGHZsyYoZtuuqnO8lOnTtV5S7zH49GJEydCLk9EtlOnTsnj8YQca8GCBcrPz9ctt9wi6epLAt/5zne0YMECvfrqq5o5c2bwZQE3ciUlJal3797685//rBdeeEETJ05UZWVlo+essbm6d++uL33pS5KklStXqrq6WrfffnvM83XtOC1btqyTp6EcJ0+eDLsftddp0qSJmjVrpsrKyqjnqDG50tLSVFhYKOnqZ/nMnTtXffr0kSRlZmZqxIgRKisrU8+ePYMfyuZGrpo//+hHP9KKFSvUpk0bTZ8+PerzIdHZJMnv9+v111/XpEmTgsvcmDNJGjVqlAYPHlxnWbzPsYjvdIzFqlWr9Mtf/rLOsltvvbXeh0Y5+RCphj6EKjk5WYFAoN5zk5ND/3+oMdnMNR98VZOhJt/SpUv1l7/8Jfh4nz59ghfal7/8ZeXk5GjTpk267bbbXMk1dOjQ4LKvf/3r6tChg/75z3+GnMuGJGq+arb94osvav78+WrSpImj+aot0geUhXr82ueF2o+Gskcj1lw1qqurNWXKFF25ckVjxoyRJE2fPj34+LBhwzR79mxVVVUpMzPTlVyvvfZacPmoUaOUk5OjyZMnN+rajlc2Sdq4caPatm2rO++8M7jMjTkLJd7nWEIKOy8vr85fC6SrfzXo1q2b/H6/UlJSHH+IVOvWrXXq1Kngz6dPn1bLli0b/HCqdu3aJSRbq1atdOrUKX3lK1+pk0GSdu3apbZt26p169bB569bt04333yz2rdvH1xW839aN3ItX75c99xzT3C5MUapqakh59LN+Vq4cKGKi4tVXFwcvLiczFdtrVu31o4dO4I/X5vn2nOkJkeLFi1UVVXV4H60bNlSp0+fVuvWrXXlyhWdP38++Jnw0Yo1l3T1H2bHjRun5s2b63e/+51SU1MVCAQ0b948jR49WikpKcH1av85kbmqqqq0dOlS/eAHP5B09XxKSUkJO49ONWbOJOm9996T1+sN/uzWnIUS73PMtZdEUlNT1aVLF61cuVKStHz5ckcfItWjRw+VlZXJ7/fr448/1uHDh9W+fXv16NFDq1ev1sWLF1VZWamtW7cqOzs7Idl69uyp0tJSSdKOHTuUnp4e/Kv9Bx98oM6dO9d5/rFjx/Taa68pEAjo9OnTWrt2rb773e+6lmv//v164403JEmHDh3Svn371Llz55Bz6Vau9957TwsWLNBbb71V504o1vmK9AFlt9xyi9LT04O/DVNaWqoePXqE3Y+ePXtq+fLlkq6+bNOlSxelpqZGPUeNySVJTz/9tG699Va9/PLLSktLk3T1b45r1qzR3/72t2Dejh07KiMjw5VcGRkZmj9/vnbv3i1JWrRokXJychp9bccjW40PPvhAXbp0Cf7s1pyFEvdzzOE/ljbK0aNHzfDhw01eXp4ZOXKk+fTTT40xxixevNi8/PLLdZ67devWOr8lEggEzMyZM43X6zVer9ds3Lgx+FhxcbHxer2mb9++pqSkJGHZLl26ZCZPnmy8Xq8ZOHCg+de//hVc//nnnzeLFy+us02fz2emTp1q8vLyTG5urnnnnXdczVVVVWXGjx9v8vPzTf/+/c2WLVuMMeHn0o1cBQUFpnv37mbAgAHB//bs2dOo+VqxYoXJz883ffv2Nb///e+NMcaMGjXK7NmzxxhjzL59+8wDDzxgcnNzzY9//GNz+fLlsPtx5swZM2bMGOP1es2QIUPMkSNHHM9RrLk+/PBD87Wvfc14vd7g/IwaNcoYY8x//vMfM2TIEOP1es3w4cPN8ePHXctljDHbt283AwcONP369TNjx441Z8+eDTuPbmYzxpgOHTqYS5cu1dmeW3NW4yc/+Umd3xKJ5znGhz8BgCV4pyMAWILCBgBLUNgAYAkKGwAsQWEDgCUobACwBIUNAJagsAHAEv8Lfnya0AVtqf4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[0].bias.data, bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These initial parameter values are sampled from uniform distributions centered on zero with a spread that depends on the number of inputs to the layer:\n", "$$\n", "\\left|W_{ij}\\right|, \\left|b_j\\right| \\le n_{in}^{-1/2} \\; .\n", "$$\n", "This default choice is based on [empirical studies](https://arxiv.org/abs/1502.01852) of image classification problems where the input features (RGB pixel values) were preprocessed to have zero mean and unit variance.\n", "\n", "With this choice of weights, the first `Linear` module mixes up its input values ($X_0$) but generally preserves Gaussian shape while slightly reducing its variance (which helps prevent the subsequent activation module from saturating):" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD7CAYAAACCEpQdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAbL0lEQVR4nO3df0ydZ/3/8eeBc2Dd6FKp5wjBhn32I2LarCySbWhycGYBWnrsPKmmhcgyP2pHJt3INywECIh2KU4+Zc4NNGbZFJ2T4QqM0NMtm1YXZkZJXNMNzdSCKzSHA+0s0PLrnPP9Y+mxlFJ+jMOBc70eSUPv69ynXG/Ofb+4e51zX5clGAwGERERo8REugMiIrL6FP4iIgZS+IuIGEjhLyJiIIW/iIiBFP4iIgZS+IuIGMga6Q4s1vnz4wQC6+eWhM2bExgZGYt0N1aFKbWaUieo1mgQE2PhU5+6ad7H1034BwLBdRX+wLrr7ydhSq2m1AmqNdpp2EdExEAKfxERAyn8RUQMpPAXETGQwl9ExEAKfxERAyn8RUQMtODn/F9++WV+/etfh7bPnDnD7t27uf/++zl06BCTk5Ps2LGDkpISAHp7e6moqGB8fJyMjAxqamqwWq0MDg5SWlrKyMgI//M//0NdXR033TT/DQgia83UtB+7feOc9onJGUYvXIpAj0SWz7KUlbw++OADHnnkEX75y1+yb98+mpqaSE5OZv/+/RQWFpKVlcWuXbs4ePAg6enplJeXs23bNvLz89m/fz9f/epXycvL49lnn+XixYuUlpYuuqMjI2Pr6kYMu30jPt9opLuxKkyp1W7fiOv/tc1pf/X/dkdd/aa8phC9tcbEWNi8OWH+x5fyj33/+9+npKSEDz/8kNTUVLZs2YLVasXlcuHxeBgYGGBiYoL09HQA3G43Ho+H6elpuru7ycnJmdUuIiKRsejpHbq6upiYmGDHjh10dHRgt9tDjzkcDrxeL0NDQ7Pa7XY7Xq+X8+fPk5CQgNVqndW+FNf7DbZWXWuIIFqZVOu1RGP90VjTfEyq9bJFh/9LL73EQw89BEAgEMBisYQeCwaDWCyWedsvf73S1dsL0bDP2hVttW68eQM3xC9t2qtoqh+i7zW9nmitdaFhn0Ud4VNTU3R3d1NbWwtAUlISPp8v9LjP58PhcMxpHx4exuFwkJiYyOjoKH6/n9jY2ND+ImvRDfHWecf2RaLFosb8//73v3PLLbdw4403ArB9+3ZOnz5Nf38/fr+fjo4OnE4nKSkpxMfH09PTA0BbWxtOpxObzUZGRgadnZ0AtLa24nQ6w1SSiIgsZFFX/h9++CFJSUmh7fj4eGpraykuLmZycpKsrCxyc3MBqKuro7KykrGxMbZu3UphYSEA1dXVlJWV0djYSHJyMocPHw5DOSIishiLCv+dO3eyc+fOWW2ZmZm0t7fP2TctLY2WlpY57SkpKTQ1NS2zmyIraznj+iLRREe/GGm+cX3Q2L6YQdM7iIgYSFf+Ip+Qpn2Q9UjhL/IJxdli5/1oaPR9elyihYZ9REQMpPAXETGQwl9ExEAKfxERAyn8RUQMpPAXETGQwl9ExED6nL9ImOjmL1nLFP4S1SI5gZtu/pK1TOEvUU0Ls4hcm8b8RUQMpPAXETGQwl9ExEAKfxERAyn8RUQMpPAXETHQosL/zTffxO12s2PHDg4ePAhAV1cXLpeL7Oxs6uvrQ/v29vbidrvJycmhoqKCmZkZAAYHBykoKCA3N5eioiLGx8fDUI6IiCzGguH/4YcfUl1dTUNDA+3t7bz//vscP36c8vJyGhoa6Ozs5NSpUxw/fhyA0tJSqqqqOHbsGMFgkObmZgBqamrIz8/H4/Gwbds2GhoawluZiIjMa8Hwf/3119m5cydJSUnYbDbq6+vZsGEDqampbNmyBavVisvlwuPxMDAwwMTEBOnp6QC43W48Hg/T09N0d3eTk5Mzq11ERCJjwTt8+/v7sdlsPPzww5w9e5Yvf/nL3HHHHdjt9tA+DocDr9fL0NDQrHa73Y7X6+X8+fMkJCRgtVpntS/F5s0JS9p/LbjWvC7RyqRaV8J6+Hmthz6uFJNqvWzB8Pf7/Zw4cYKmpiZuvPFGioqKuOGGG7BYLKF9gsEgFouFQCBwzfbLX6909fZCRkbGCASCS3pOJNntG/H5zJjBZS3XulZP6rX687psLb+mKy1aa42JsVz3onnB8P/0pz9NZmYmiYmJANx///14PB5iY2ND+/h8PhwOB0lJSfh8vlD78PAwDoeDxMRERkdH8fv9xMbGhvYXEZHIWHDM/7777uOtt97iwoUL+P1+/vznP5Obm8vp06fp7+/H7/fT0dGB0+kkJSWF+Ph4enp6AGhra8PpdGKz2cjIyKCzsxOA1tZWnE5neCsTWaMuT/V89Z+NN2+IdNfEIAte+W/fvp1vf/vb5OfnMz09zZe+9CX27dvHrbfeSnFxMZOTk2RlZZGbmwtAXV0dlZWVjI2NsXXrVgoLCwGorq6mrKyMxsZGkpOTOXz4cHgrE1mjNNWzrAWLmtJ5z5497NmzZ1ZbZmYm7e3tc/ZNS0ujpaVlTntKSgpNTU3L7KaIiKwk3eErImIghb+IiIG0kpdEhUgu1yiyHulskaig5RpFlkbDPiIiBlL4i4gYSOEvImIghb+IiIEU/iIiBlL4i4gYSOEvImIghb+IiIEU/iIiBlL4i4gYSNM7iKwRlxd5udrE5AyjFy5FoEcSzRT+ImuEFnmR1aRhHxERAyn8RUQMpPAXETGQwl9ExECLesP3m9/8JufOncNq/Xj3H/zgB/z73/+msbGRmZkZHnzwQQoKCgDo6uri0KFDTE5OsmPHDkpKSgDo7e2loqKC8fFxMjIyqKmpCf17IiKyuhZM32AwSF9fH3/4wx9CYe31eikpKeGVV14hLi6OvXv3cs899/DZz36W8vJympqaSE5OZv/+/Rw/fpysrCxKS0s5ePAg6enplJeX09zcTH5+ftgLlOii5RpFVsaCZ9G//vUvAL71rW/x0Ucf8Y1vfIObbrqJe++9l02bNgGQk5ODx+Ph7rvvJjU1lS1btgDgcrnweDzcfvvtTExMkJ6eDoDb7ebpp59W+MuSablGkZWx4Jj/hQsXyMzM5Nlnn+WFF17gpZdeYnBwELvdHtrH4XDg9XoZGhpaVLvdbsfr9a5wKSIislgLXvnfdddd3HXXXaHtPXv2cOjQIYqKikJtwWAQi8VCIBDAYrEsun0pNm9OWNL+a8G17taMVibVGgmR+Pma9JqaVOtlC4b/iRMnmJ6eJjMzE/g4uFNSUvD5fKF9fD4fDoeDpKSkRbUPDw/jcDiW1NGRkTECgeCSnhNJdvtGfD4z7stczVpNPEmBVT+WdPyufzExluteNC847DM6OsqTTz7J5OQkY2NjHDlyhB//+Me8/fbbnDt3jkuXLvHaa6/hdDrZvn07p0+fpr+/H7/fT0dHB06nk5SUFOLj4+np6QGgra0Np9O5clWKiMiSLHjlf9999/Huu+/ywAMPEAgEyM/P5wtf+AIlJSUUFhYyPT3Nnj17uPPOOwGora2luLiYyclJsrKyyM3NBaCuro7KykrGxsbYunUrhYWF4a1MRETmtajPzD322GM89thjs9pcLhcul2vOvpmZmbS3t89pT0tLo6WlZZndFBGRlaQ7fEVEDKTwFxExkMJfRMRACn8REQMp/EVEDKTwFxExkMJfRMRAmhtXZI2bmvZfc1qLickZRi9cikCPJBoo/EXWuDhb7LzTWEffjDSyWjTsIyJiIIW/iIiBFP4iIgbSmL+sSVqrVyS8dHbJmqS1ekXCS8M+IiIGUviLiBhI4S8iYiCFv4iIgRT+IiIGUviLiBho0eH/ox/9iLKyMgB6e3txu93k5ORQUVHBzMwMAIODgxQUFJCbm0tRURHj4+MAXLhwge9+97vs2LGDgoICfD5fGEoREZHFWlT4v/322xw5ciS0XVpaSlVVFceOHSMYDNLc3AxATU0N+fn5eDwetm3bRkNDAwBPPfUUGRkZHD16lK9//es88cQTYShFREQWa8Hw/+ijj6ivr+fhhx8GYGBggImJCdLT0wFwu914PB6mp6fp7u4mJydnVjvAH//4R1wuFwC7du3iT3/6E9PT02EpSEREFrZg+FdVVVFSUsLNN98MwNDQEHa7PfS43W7H6/Vy/vx5EhISsFqts9qvfo7VaiUhIYFz586teDEiIrI4153e4eWXXyY5OZnMzExeeeUVAAKBABaLJbRPMBjEYrGEvl7p6u0rnxMTs7T3mjdvTljS/mvBtRbgiFYm1bqWhPPnbtJralKtl103/Ds7O/H5fOzevZv//Oc/XLx4EYvFMusN2+HhYRwOB4mJiYyOjuL3+4mNjcXn8+FwOABwOBwMDw+TlJTEzMwM4+PjbNq0aUkdHRkZIxAILqPEyLDbN+LzmbHURjhqNfFkXI5wHWM6fte/mBjLdS+ar3v5/fzzz9PR0UFbWxsHDhzgK1/5CocOHSI+Pp6enh4A2tracDqd2Gw2MjIy6OzsBKC1tRWn0wlAVlYWra2twMe/UDIyMrDZbCtSoIiILN2yPudfV1fHoUOHyM3N5eLFixQWFgJQXV1Nc3MzO3fu5MSJEzz22GMAPProo/z1r38lLy+PF198kaqqqpWrQERElmzRUzq73W7cbjcAaWlptLS0zNknJSWFpqamOe2bNm3iZz/72SfopoiIrCTd4SsiYiCFv4iIgRT+IiIGUviLiBhIa/iKrFNT0/5r3g8xMTnD6IVLEeiRrCcKf5F1Ks4WO+8i99F3y5KsNA37iIgYSOEvImIghb+IiIE05i8RtfHmDdwQr8NQZLXprJOIuiHeOu+bliISPhr2ERExkMJfRMRACn8REQMp/EVEDKTwFxExkMJfRMRACn8REQMp/EVEDKTwFxEx0KLC/yc/+Qk7d+4kLy+P559/HoCuri5cLhfZ2dnU19eH9u3t7cXtdpOTk0NFRQUzMzMADA4OUlBQQG5uLkVFRYyPj4ehHBERWYwFw/+dd97hL3/5C+3t7fz+97+nqamJv/3tb5SXl9PQ0EBnZyenTp3i+PHjAJSWllJVVcWxY8cIBoM0NzcDUFNTQ35+Ph6Ph23bttHQ0BDeykREZF4Lhv/dd9/Nr371K6xWKyMjI/j9fi5cuEBqaipbtmzBarXicrnweDwMDAwwMTFBeno6AG63G4/Hw/T0NN3d3eTk5MxqFxGRyFjUsI/NZuPpp58mLy+PzMxMhoaGsNvtoccdDgder3dOu91ux+v1cv78eRISErBarbPaRUQkMhY9q+eBAwf4zne+w8MPP0xfXx8WiyX0WDAYxGKxEAgErtl++euVrt5eyObNCUvafy241vqq0cqkWte6+db2nZr2E2eLXfS/Y9JralKtly0Y/v/85z+Zmpri85//PBs2bCA7OxuPx0Ns7H8PIp/Ph8PhICkpCZ/PF2ofHh7G4XCQmJjI6Ogofr+f2NjY0P5LMTIyRiAQXNJzIslu34jPZ8ZKqp+kVhNPunC73tq+i32ddPyufzExluteNC847HPmzBkqKyuZmppiamqKN954g71793L69Gn6+/vx+/10dHTgdDpJSUkhPj6enp4eANra2nA6ndhsNjIyMujs7ASgtbUVp9O5QiWKiMhSLXjln5WVxcmTJ3nggQeIjY0lOzubvLw8EhMTKS4uZnJykqysLHJzcwGoq6ujsrKSsbExtm7dSmFhIQDV1dWUlZXR2NhIcnIyhw8fDm9lIiIyr0WN+RcXF1NcXDyrLTMzk/b29jn7pqWl0dLSMqc9JSWFpqamZXZT1jst1yiytuhslFWh5RpF1hZN7yAiYiCFv4iIgRT+IiIGUviLiBhI4S8iYiCFv4iIgRT+IiIGUviLiBhI4S8iYiCFv4iIgRT+IiIGUviLiBhI4S8iYiDN6iliiPmWd5yYnGH0wqUI9EgiSeEvYojrLe8YfYsYykI07CMiYiCFv4iIgTTsIytKyzWKrA86S2VFablGkfVhUcM+zzzzDHl5eeTl5fHkk08C0NXVhcvlIjs7m/r6+tC+vb29uN1ucnJyqKioYGZmBoDBwUEKCgrIzc2lqKiI8fHxMJQjIiKLsWD4d3V18dZbb3HkyBFaW1t577336OjooLy8nIaGBjo7Ozl16hTHjx8HoLS0lKqqKo4dO0YwGKS5uRmAmpoa8vPz8Xg8bNu2jYaGhvBWJiIi81ow/O12O2VlZcTFxWGz2bjtttvo6+sjNTWVLVu2YLVacblceDweBgYGmJiYID09HQC3243H42F6epru7m5ycnJmtYuISGQsGP533HFHKMz7+vo4evQoFosFu90e2sfhcOD1ehkaGprVbrfb8Xq9nD9/noSEBKxW66x2ERGJjEW/4fvBBx+wf/9+Hn/8cWJjY+nr6ws9FgwGsVgsBAIBLBbLnPbLX6909fZCNm9OWNL+a8G17qaMVibVGo2u9fqZ9JqaVOtliwr/np4eDhw4QHl5OXl5ebzzzjv4fL7Q4z6fD4fDQVJS0qz24eFhHA4HiYmJjI6O4vf7iY2NDe2/FCMjYwQCwSU9J5Ls9o34fGbcN3llrSaeRNHg6mPV1OM3msTEWK570bzgsM/Zs2d55JFHqKurIy8vD4Dt27dz+vRp+vv78fv9dHR04HQ6SUlJIT4+np6eHgDa2tpwOp3YbDYyMjLo7OwEoLW1FafTuRL1iYjIMix45f/cc88xOTlJbW1tqG3v3r3U1tZSXFzM5OQkWVlZ5ObmAlBXV0dlZSVjY2Ns3bqVwsJCAKqrqykrK6OxsZHk5GQOHz4cppJERGQhC4Z/ZWUllZWV13ysvb19TltaWhotLS1z2lNSUmhqalpGF0VEZKVpbh8REQNpegcRw803z7/dvlFz/Ucxhb+I4eab5x801380U/jLslw9e6c+4imyvij8ZVk0e6fI+qY3fEVEDKTwFxExkMJfRMRACn8REQMp/EVEDKTwFxExkMJfRMRACn8REQMp/EVEDKTwFxExkMJfRMRACn8REQMp/EVEDKRZPUVkXvMt9KJFXtY/hb+IzGu+hV60yMv6t+hhn7GxMXbt2sWZM2cA6OrqwuVykZ2dTX19fWi/3t5e3G43OTk5VFRUMDMzA8Dg4CAFBQXk5uZSVFTE+Pj4Cpci4bDx5g3Y7Rvn/BGR9W1R4f/uu++yb98++vr6AJiYmKC8vJyGhgY6Ozs5deoUx48fB6C0tJSqqiqOHTtGMBikubkZgJqaGvLz8/F4PGzbto2GhobwVCQr6vKiLVf/EZH1bVHh39zcTHV1NQ6HA4CTJ0+SmprKli1bsFqtuFwuPB4PAwMDTExMkJ6eDoDb7cbj8TA9PU13dzc5OTmz2kVEJDIWNeb/xBNPzNoeGhrCbreHth0OB16vd0673W7H6/Vy/vx5EhISsFqts9pFRCQylvWGbyAQwGKxhLaDwSAWi2Xe9stfr3T19kI2b05YTlcjSmPjEs2i6fiOploWa1nhn5SUhM/nC237fD4cDsec9uHhYRwOB4mJiYyOjuL3+4mNjQ3tvxQjI2MEAsHldDci7PaN+Hzr//MQJp4UsjjRcHxD9JyrV4uJsVz3onlZN3lt376d06dP09/fj9/vp6OjA6fTSUpKCvHx8fT09ADQ1taG0+nEZrORkZFBZ2cnAK2trTidzuV8axERWQHLuvKPj4+ntraW4uJiJicnycrKIjc3F4C6ujoqKysZGxtj69atFBYWAlBdXU1ZWRmNjY0kJydz+PDhlatCRFaVbv5a/5YU/m+++Wbo75mZmbS3t8/ZJy0tjZaWljntKSkpNDU1LaOLIrLW6Oav9U9z+4iIGEjTOwjw8Z28N8TrcBAxhc52Af57J+/VXv2/3RHojYiEm4Z9REQMpPAXETGQwl9ExEAKfxERA+kNXxFZMbr5a/1Q+BtGH+mUcNLNX+uHUsAw+kiniIDG/EVEjKTwFxExkIZ9RCTs9Ebw2qPwF5Gw0xvBa4/CPwrpEz0ishAlRBSa7xM9oE/1iMjHFP4iEjF6LyByFP7rmIZ3ZL3TewGRo+RYx3TDlogsl8JfRNYcDQeFn8J/HdDwjphGw0Hht6qJ8uqrr9LY2MjMzAwPPvggBQUFq/nt1y0N74h8TP8jWDmrFv5er5f6+npeeeUV4uLi2Lt3L/fccw+33377anVhzdMVvsj16X8EK2fVkqarq4t7772XTZs2AZCTk4PH4+F73/veop4fE2MJZ/fC4uabNxB/jTCfnPITHxd7zef878HX5rQ9V5mN41Mbrrn/UtuX8xzT2tdin/SzuH77fP8jmO9cm5ycYWxsIrS9HvNlIQvVZAkGg8HV6MjPf/5zLl68SElJCQAvv/wyJ0+e5Ic//OFqfHsREbnCqs3qGQgEsFj++5soGAzO2hYRkdWzauGflJSEz+cLbft8PhwOx2p9exERucKqhf8Xv/hF3n77bc6dO8elS5d47bXXcDqdq/XtRUTkCqv2hu9nPvMZSkpKKCwsZHp6mj179nDnnXeu1rcXEZErrNobviIisnZoGUcREQMp/EVEDKTwFxExkMJfRMRACv8w6enpYc+ePezevZsHH3yQgYGBSHcp7J566il++tOfRrobYfHqq6+yc+dOsrOz+c1vfhPp7oTV2NgYu3bt4syZM5HuSlg988wz5OXlkZeXx5NPPhnp7qw6hX+YlJaWcvDgQdra2nC5XBw8eDDSXQqb0dFRysvLef755yPdlbC4PCnhiy++SGtrK7/73e/4xz/+EeluhcW7777Lvn376Ovri3RXwqqrq4u33nqLI0eO0Nraynvvvcfrr78e6W6tKoV/GExNTfHoo4+SlpYGwOc+9znOnj0b4V6FzxtvvMEtt9zCQw89FOmuhMWVkxLeeOONoUkJo1FzczPV1dVRf/e93W6nrKyMuLg4bDYbt912G4ODg5Hu1qrS/MFhEBcXx+7dH8+1HwgEeOaZZ7j//vsj3KvweeCBBwCidshnaGgIu90e2nY4HJw8eTKCPQqfJ554ItJdWBV33HFH6O99fX0cPXqU3/72txHs0epT+H9CR48e5dChQ7Pabr31Vl544QWmpqYoKytjZmaG/fv3R6iHK+d6tUYzTUoYvT744AP279/P448/zi233BLp7qwqhf8ntGPHDnbs2DGnfXx8nKKiIjZt2kRjYyM2my0CvVtZ89Ua7ZKSkjhx4kRoW5MSRoeenh4OHDhAeXk5eXl5ke7OqtOYf5iUlpaSmprKU089RVxcXKS7I5+AJiWMPmfPnuWRRx6hrq7OyOAHXfmHxfvvv88bb7zB7bffzte+9jXg43HiX/ziFxHumSyHJiWMPs899xyTk5PU1taG2vbu3cu+ffsi2KvVpYndREQMpGEfEREDKfxFRAyk8BcRMZDCX0TEQAp/EREDKfxFRAyk8BcRMZDCX0TEQP8fj+xwyB3U1YQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[0].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A scatter plot of the the first `Tanh` activation function's input and output values just traces out the function since it is applied element wise. Note how most of input values do not saturate, which is generally desirable for efficient learning." ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfVRU9f4v8PcMT4qwRG0PkPlsR0tlmeLBvN3pZOoIghjLPD7c6MG4ppbmPdEhbIl6ZFEeFDslnh70eH9lLjklICcYPWm5foY3wUqhXGa6LAWdGRh/BggyMvv+YczPkcfZMLNn9n6/1mrB3vu78fNp1trv+X73wNaIoiiCiIhUSyt3AUREJC8GARGRyjEIiIhUjkFARKRyDAIiIpVjEBARqRyDgIhI5fzlLkCqa9caYLf7xq9ADBoUgtraernL8Aj2qjxq6RNQbq9arQYDBvTr8LjPBoHdLvpMEADwqVp7ir0qj1r6BNTVaysuDRERqRyDgIhI5XocBPX19YiPj8fly5fbHDtz5gySkpJgMBiwdu1a3Lp1CwBQXV2NJUuWYPbs2Vi+fDkaGhp6WgYREUnUoyA4deoUFi1ahIsXL7Z7PDU1FevWrcPBgwchiiLy8vIAABs2bMDixYthNBoxfvx45Obm9qQMIiLqgR4FQV5eHjIyMqDT6docq6qqQlNTEyZOnAgASEpKgtFohM1mQ1lZGQwGg9N+IiKSR48+NZSZmdnhMbPZDEEQHNuCIMBkMuHatWsICQmBv7+/034iIpKH2z4+arfbodFoHNuiKEKj0Ti+3unu7e4YNCikxzV6kiCEyl2Cx7BX5VFLn4D8vV6vv4nDZb/g8SlD0T8kyLH/sqkOO4sqsTRhPO4L790a3RYEERERsFgsju2amhrodDoMHDgQdXV1aGlpgZ+fHywWS7tLS12pra33mc/7CkIoLJY6ucvwCPaqPGrpE3Ct1yu1Ddh35Bz+OP1+hPQNwOGTl1F3oxkmayOWzPodIgfd/gWuuhvNOFZxBRNH34PvfqpxfH1kQiRCgwPb/NySr3/GP784j/qGm4iNGebYv+PT73D6vBXNzd/h5ScnutSXVqvp9M2z24Jg8ODBCAoKwsmTJzF58mQUFhZCr9cjICAA0dHRKC4uRkJCAgoKCqDX691VBhFRt12vv4mcvG9RceEaNACC+/ghwN8P40YMQGjfQDTbWnCh+jrqm1rQv18gzlf/CuAcxgwdgANfXXT8nH1Hzjku1scqruCfX5zH2V+u4fR5q+MrAKcLfatHJkQ6fW31x+n3Azj329fe1etBkJKSglWrVmHChAnIzs7G66+/jvr6eowbNw7JyckAgIyMDKSlpWHHjh2IjIzE1q1be7sMIqJ2/VT1X8jZ9y0am0VoAGg1gJ9WAxEiRFGDW7+tNIgAGppaALTgq4p27mOKQNSogY4Zwc3mFseM4M6LdesFfeLoezBmaI3j690X+lahwYHtBkTkoH4uzwS6S+Orzyzm0pB3Yq/K46t97vrsexxr7wLeTV3NCP733AcxenBY7xXsRrItDRERecLp8zXY9s/Tks69e0bg7++HSb+7Bwseu7/d9XulYhAQkU8p/M8LKLxjPd5VGgBPGX6HPzx0X5tjvjr76SkGARF5tTc+KsOPl6VdnBc8Ngqz21lvJ2cMAiLyOs+9ccTlc4YIfbFh6cNuqEb5GARE5BVcvfg/OKw/Xlk02U3VqAuDgIhk48rF/95BfbApZZobq1EvBgERedTKrUfQ2Ny9sbN/PwQL3PALVOSMQUBEHtHdd//JHXyih9yHQUBEbvPlt5fxHwd/7HLc7+4LRdr/muKBiqg9DAIi6nWZ//drnL/S9ZMHd6VN90A11BUGARH1mjV/O4rrN1o6HSP0D8Sbyx/xUEXUHQwCIuqxvCPnYDxxqdMxfPfvvRgERNQjXd0EZgB4PwYBEUnCAFAOBgERuYQBoDxauQsgIt/RWQj0CWAI+KoezQiKioqwY8cO3Lp1C08//TSWLFniOHbmzBmkpaU5tq1WK/r3749//etfyM/Px5YtWzBo0CAAwB/+8AesWbOmJ6UQkRtxFqBskoPAZDIhJycH+/fvR2BgIBYuXIiYmBiMHj0aAPDAAw+gsLAQANDY2Ignn3wS69evBwBUVlYiLS0N8fHxPe+AiNwq4U+FHR5jACiD5KWh0tJSTJ06FWFhYQgODobBYIDRaGx37LvvvospU6YgOjoaAFBRUYH8/HwkJCTglVdewfXr16WWQURusmLLkU5nAgwB5ZAcBGazGYIgOLZ1Oh1MprbPB62rq0NeXh5efPFFxz5BELBixQocOHAAkZGR2Lhxo9QyiMgNnnvjCJps7R/blTadIaAwkpeG7HY7NBqNY1sURaftVgcOHMCMGTMc9wMAYPv27Y7vn3/+ecycOdPlf7+zBzF7I0EIlbsEj2Gvvq2zpaCiLYkerEQeSnxNuyI5CCIiIlBeXu7Ytlgs0Ol0bcZ9/vnnWLZsmWO7rq4On376KZ555hkAtwPEz8/P5X+/trYedrvoeuEyUNNzUNmr76q70YzVfzvW7rHWGYCS+m2P0l7TVlqtptM3z5KXhqZNm4bjx4/DarWisbERhw4dgl6vdxojiiK+//57PPTQQ459wcHB+OCDD3Dq1CkAwEcffSRpRkBEvee5N450GAJqmAWoneQZQXh4ONasWYPk5GTYbDbMnz8fUVFRSElJwapVqzBhwgRYrVYEBAQgKCjIcZ6fnx+2bduG9evXo6mpCcOHD8fmzZt7pRkich1vCJNGFEXfWF+5C5eGvBN79S0dhcCdAaCEPrtLqb26bWmIiHxbd0KA1IFBQKRCDAG6E4OASGUYAnQ3BgGRijAEqD0MAiKVYAhQRxgERCrAEKDOMAiIFI4hQF1hEBApGEOAuoNBQKRQDAHqLgYBkQIxBMgVDAIihWEIkKsYBEQKwhAgKRgERArBECCpGARECsAQoJ5gEBD5uIydx9vdzxCg7mIQEPm4S5bGNvsYAuSKHgVBUVER4uLiMGvWLOzZs6fN8XfeeQePPfYYEhMTkZiY6Bhz5swZJCUlwWAwYO3atbh161ZPyiBSrfaWhBgC5CrJj6o0mUzIycnB/v37ERgYiIULFyImJgajR492jKmsrMTWrVudnlkMAKmpqdi0aRMmTpyI9PR05OXlYfHixdK7IFIhhgD1FskzgtLSUkydOhVhYWEIDg6GwWCA0Wh0GlNZWYl3330XCQkJ2LhxI27evImqqio0NTVh4sSJAICkpKQ25xFR5zp7zjCRqyQHgdlshiAIjm2dTgeTyeTYbmhowAMPPIDU1FTk5+fj119/RW5ubpvzBEFwOo+IOsdPCFFvk7w0ZLfbodFoHNuiKDpt9+vXD++//75j+7nnnkN6ejr0en2n53VXZw9i9kaCECp3CR7DXj2vaEuiW3++t/TpCWrqtZXkIIiIiEB5eblj22KxQKfTObarq6tRWlqK+fPnA7h9wff390dERAQsFotjXE1NjdN53VVbWw+7XZRavkcJQigsljq5y/AI9upeHd0XcGcdfE19n1ar6fTNs+SloWnTpuH48eOwWq1obGzEoUOHoNfrHcf79OmDv/71r7h06RJEUcSePXswc+ZMDB48GEFBQTh58iQAoLCw0Ok8Imofbw6Tu0ieEYSHh2PNmjVITk6GzWbD/PnzERUVhZSUFKxatQoTJkzAxo0bsXz5cthsNkyaNAnPPvssACA7Oxuvv/466uvrMW7cOCQnJ/daQ0RKxJvD5E4aURR9Y33lLlwa8k7s1T3knA3wNfV9blsaIiLP4JIQuRuDgMiLMQTIExgEREQqxyAg8lKcDZCnMAiIvBBDgDyJQUDkZdoLgfSnJslQCakFg4DIB4weHCZ3CaRgDAIiL8IlIZIDg4DISzAESC4MAiIilWMQEHkBzgZITgwCIpkxBEhuDAIiIpVjEBDJiLMB8gYMAiKZMATIWzAIiIhUrkdBUFRUhLi4OMyaNQt79uxpc/zzzz9HYmIi5s6dixUrVuD69esAgPz8fDzyyCNITExEYmIicnJyelIGkc/hbIC8ieRHVZpMJuTk5GD//v0IDAzEwoULERMTg9GjRwMA6uvrsX79enz66acIDw/HW2+9hbfffhuvv/46KisrkZaWhvj4+F5rhMiXMQRITpJnBKWlpZg6dSrCwsIQHBwMg8EAo9HoOG6z2ZCRkYHw8HAAwJgxY3DlyhUAQEVFBfLz85GQkIBXXnnFMVMgUgM+f5i8jeQgMJvNEATBsa3T6WAymRzbAwYMwMyZMwEATU1NeO+99zBjxgwAgCAIWLFiBQ4cOIDIyEhs3LhRahlEPoVLQuSNJC8N2e12aDQax7Yoik7brerq6rBy5UqMHTsWTzzxBABg+/btjuPPP/+8IzBc0dmDmL2RIITKXYLHsNfumzY+3Cf+f/lCjb1FTb22khwEERERKC8vd2xbLBbodDqnMWazGUuXLsXUqVORnp4O4HYwfPrpp3jmmWcA3A4QPz8/l//92tp62O2i1PI9ShBCYbHUyV2GR7DXjrU3G3g+fpzX///ia+r7tFpNp2+eJS8NTZs2DcePH4fVakVjYyMOHToEvV7vON7S0oIXXngBsbGxWLt2rWO2EBwcjA8++ACnTp0CAHz00UeSZgREvoRLQuTNJM8IwsPDsWbNGiQnJ8Nms2H+/PmIiopCSkoKVq1ahatXr+KHH35AS0sLDh48CAAYP348MjMzsW3bNqxfvx5NTU0YPnw4Nm/e3GsNERGRazSiKPrG+spduDTkndhrW74+G+Br6vvctjRERNL4UgiQOjAIiNyIvzNAvoBBQOQmvr4kROrBICAiUjkGAZEbcDZAvoRBQOQBDAHyZgwCol7GG8TkaxgERL2IS0LkixgEREQqxyAg6iWcDZCvYhAQuQlDgHwFg4CoF/AGMfkyBgFRD3FJiHwdg4CISOUYBEQ9wNkAKQGDgKgXMQTIF/UoCIqKihAXF4dZs2Zhz549bY6fOXMGSUlJMBgMWLt2LW7dugUAqK6uxpIlSzB79mwsX74cDQ0NPSmDSBYJfyqUuwSiXiE5CEwmE3JycvDxxx+joKAA+/btw08//eQ0JjU1FevWrcPBgwchiiLy8vIAABs2bMDixYthNBoxfvx45Obm9qwLIg/jkhApieQgKC0txdSpUxEWFobg4GAYDAYYjUbH8aqqKjQ1NWHixIkAgKSkJBiNRthsNpSVlcFgMDjtJyIieUgOArPZDEEQHNs6nQ4mk6nD44IgwGQy4dq1awgJCYG/v7/TfiJfwdkAKY2/1BPtdjs0Go1jWxRFp+2Ojt89DkCb7e7o7EHM3kgQQuUuwWPU1CsAFG1JlLsEt1PTa6qmXltJDoKIiAiUl5c7ti0WC3Q6ndNxi8Xi2K6pqYFOp8PAgQNRV1eHlpYW+Pn5tTmvu2pr62G3i1LL9yhBCIXFUid3GR6h9F7bmw0ouV9A+a/pnZTaq1ar6fTNs+SloWnTpuH48eOwWq1obGzEoUOHoNfrHccHDx6MoKAgnDx5EgBQWFgIvV6PgIAAREdHo7i4GABQUFDgdB6Rt9r12fdt93FJiBRAchCEh4djzZo1SE5Oxrx58xAfH4+oqCikpKSgoqICAJCdnY2srCzMnj0bN27cQHJyMgAgIyMDeXl5iIuLQ3l5OV5++eXe6YbIjY5V8F4WKZNGFEXfWF+5C5eGvJNSe21vSahoS6Iie72bUl/T9ii1V7ctDRGpGZeESEkYBERd4J+YJqVjEBC5iLMBUhoGAVEnOBsgNWAQEHWAv0FMasEgICJSOQYBUTs4GyA1YRAQdQNDgJSMQUB0F94gJrVhEBDdgUtCpEYMAiIilWMQEP2GswFSKwYBUQcYAqQWDAIi8AYxqRuDgFSPS0KkdgwCIiKVk/zM4urqaqSmpqK2thYjRoxAdnY2+vXr5zTGbDbjtddeQ01NDbRaLV599VU8/PDDsNlsiImJwZAhQxxj9+/fDz8/P+mdEEnA2QBRD2YEGzZswOLFi2E0GjF+/Hjk5ua2GbN582ZMnz4dhYWF2LJlC1555RW0tLTg7NmzeOihh1BYWOj4jyFAnsYQILpNUhDYbDaUlZXBYDAAAJKSkmA0GtuMmzlzJuLj4wEAw4YNw82bN3Hjxg1UVFTAarUiKSkJCxYswIkTJ3rQAhER9YSkpaFr164hJCQE/v63TxcEASZT2wd7twYFAOzcuRMPPPAAQkNDodFo8Pjjj2PZsmU4d+4cUlJSUFRUhIEDB0psg8g1nA0Q/bcug6CkpARZWVlO+4YNGwaNRuO07+7tO+3evRv79u3DRx99BABYuHCh49iDDz6IqKgofPPNN5gxY0a3C+/sQczeSBBC5S7BY3yx16ItiZLO88VepVBLn4C6em3VZRDExsYiNjbWaV/rzd6Wlhb4+fnBYrFAp9O1e/7mzZtx9OhR7NmzBxEREQCAgoICTJo0CUOHDgUAiKKIgIAAlwqvra2H3S66dI5cBCEUFkud3GV4hC/02t5sQErNvtBrb1BLn4Bye9VqNZ2+eZZ0jyAgIADR0dEoLi4GcPvCrtfr24zbvXs3vv76a+zdu9cRAgBw9uxZ7Nq1CwBw4cIFnDlzBpMnT5ZSCpFLuCRE1Jbkj49mZGQgLS0NO3bsQGRkJLZu3QoA2Lt3L8xmM1atWoXt27cjJCQETz31lOO89957DytXrkR6ejri4+Oh0Wjw5ptvIiTEt5Z6iIiUQiOKom+sr9yFS0PeyZt77e3ZgDf32pvU0ieg3F7dsjRE5Gu4JETUMQYBEZHKMQhI8TgbIOocg4AUjSFA1DUGARGRyjEISLE4GyDqHgYBKRJDgKj7GARERCrHICDF4WyAyDUMAlIUhgCR6xgEREQqxyAgxeBsgEgaBgEpAkOASDoGAfm89kKAiLqPQUCKxNkAUfcxCMincUmIqOckP6GsuroaqampqK2txYgRI5CdnY1+/fo5jamqqkJ8fLzj2cT33HMPdu7ciebmZqxduxaVlZXo06cPsrOzMWrUqJ51QqrDECDqHZJnBBs2bMDixYthNBoxfvx45ObmthlTWVmJhIQEFBYWorCwEDt37gQAfPjhh+jbty9KSkqQnp6O1157TXoHpEq8L0DUeyQFgc1mQ1lZGQwGAwAgKSkJRqOxzbiKigr8+OOPSExMRHJyMs6ePQsA+PLLLzF37lwAwJQpU2C1WlFdXS21ByIAnA0QSSUpCK5du4aQkBD4+99eWRIEASaTqc24oKAgzJ07F/n5+Vi6dClWrlyJ5uZmmM1mCILgGCcIAq5evSqxBVIbLgkR9a4u7xGUlJQgKyvLad+wYcOg0Wic9t29DQAvvfSS4/tHH30UW7ZswYULFyCKotN4URSh1bqWSZ09iNkbCUKo3CV4jDt7TfhTYZt9RVsS3fbvdUUtr6ta+gTU1WurLoMgNjYWsbGxTvtsNhtiYmLQ0tICPz8/WCwW6HS6Nud++OGHiI+Px4ABAwDcvuD7+/sjPDwcZrPZcRO5pqam3fM7U1tbD7tddOkcuQhCKCyWOrnL8Ah39trRfQG5/t+q5XVVS5+AcnvVajWdvnmWtDQUEBCA6OhoFBcXAwAKCgqg1+vbjCsrK8Mnn3wCADhx4gTsdjtGjhyJRx99FIWFt9/ZlZeXIygoCPfee6+UUkglOgoBLgkR9ZxGFEVJb6urqqqQlpaG2tpaREZGYuvWrejfvz/27t0Ls9mM1atXw2QyIS0tDRaLBUFBQcjMzMTYsWNx8+ZNrFu3DpWVlQgMDMSmTZswbtw4l/59zgi8k7t69cb7Amp5XdXSJ6DcXruaEUgOArkxCLyTO3r1xhAA1PO6qqVPQLm9umVpiMhTvDUEiJSEQUBeq70QGBXZr52RRNQTDALySh3dHF77dIyHKyFSPgYBeR1+QojIsxgE5FUYAkSexyAgr8EQIJIHg4C8AkOASD4MApIdQ4BIXgwCkhVDgEh+DAKSDUOAyDswCEgWDAEi78EgII9jCBB5FwYBeRRDgMj7MAjIYxgCRN6JQUAewRAg8l4MAnI7hgCRd+vymcUdqa6uRmpqKmprazFixAhkZ2ejXz/nPxH8wgsv4MqVKwAAu92OH3/8EZ988gnGjh2LmJgYDBkyxDF2//798PPzk1oOeaGOAgBgCBB5E8lBsGHDBixevBhz5szB9u3bkZubi9TUVKcxf//73x3fv/XWW5g4cSImTJiAyspKPPTQQ9i5c6f0ysmrMQSIfIekpSGbzYaysjIYDAYAQFJSEoxGY4fjL1y4gIKCAvz5z38GAFRUVMBqtSIpKQkLFizAiRMnpJRBXqqjELh3UB+GAJEXkjQjuHbtGkJCQuDvf/t0QRBgMpk6HJ+bm4ulS5ciJOT2MzM1Gg0ef/xxLFu2DOfOnUNKSgqKioowcOBAKeWQl3gx5whu3Gz/GAOAyHt1GQQlJSXIyspy2jds2DBoNBqnfXdvt7p+/Tq++uorZGZmOvYtXLjQ8f2DDz6IqKgofPPNN5gxY0a3C+/sQczeSBBC5S7BrRL+VNjhsaItiR6sxLOU/rq2UkufgLp6bdVlEMTGxiI2NtZpn81mQ0xMDFpaWuDn5weLxQKdTtfu+UePHoVer0dQUJBjX0FBASZNmoShQ4cCAERRREBAgEuF19bWw24XXTpHLoIQCoulTu4y3CLvyDkYT1zq8PiutOmK7V3Jr+ud1NInoNxetVpNp2+eJd0jCAgIQHR0NIqLiwHcvrDr9fp2x3733XeIjo522nf27Fns2rULwO37B2fOnMHkyZOllEIyeu6NIx2GwK606VwOIvIRkn+PICMjA3l5eYiLi0N5eTlefvllAMDevXvx1ltvOcZdunQJ4eHhTueuXLkSVqsV8fHxWL16Nd58803H/QPyfilvHOGngogURCOKom+sr9yFS0Py6CoAlNRrV9TSq1r6BJTba1dLQ5J/j4DUpbMAADgLIPJlDALqFAOASPkYBNSurgIg0A/4eypDgEgJGATkpKsAADgLIFIaBgEBYAAQqRmDQOUYAETEIFChl7Z9gYamrj96ywAgUgcGgUoYv/4ZeV+c79ZYBgCRujAIFOxKbQPWvv91t8bO/v0QLJh+v5srIiJvxCBQmML/vIDCry52ezzf/RMRg0ABVm49gsbm7o/nxZ+I7sQg8EFr/nYU12+0uHQOL/5E1BEGgZdzdamnFX/zl4i6i0HgRaRe9FtlpsQgclC/3iuIiFSBQeBhX357Gf9x8Mce/xyhfxBef3oKQoMDe6EqIlIzBkEv+6nqv5BbUImmJhv6Bgbg+o1m3CcE4xfzDfQJ1KCp2fVnKPQJ1OL//HEiRg8Oc0PFRKR2PQ6Cbdu2wc/PDy+99FKbY83NzVi7di0qKyvRp08fZGdnY9SoURBFEZs3b8YXX3wBrVaLv/zlL177qMq6G80o/n8/45KpHvP0I/DNWQt+MddhycwxCOkbgMMnLwMAYh4Mx9c/mHD45GU0NN0CADTZbn+U5xfzjdvb3QiBBY+NwuyYYW7qhoioLclBUFdXh6ysLHz22Wd4/vnn2x3z4Ycfom/fvigpKUFZWRlee+015OXl4eDBgzh//jyKi4vx888/Y9myZSguLoa/v3snKHU3mnGs4goemRCJ0OBAp20AThf8ygtWx3kHf3sur/WzJly1NgIA9h05hzFDB+DAb2v6F6/+itPnb5/TJ8gPsNvbnRFA1CD6AR2e/MNoLusQkVeQfOU9fPgwhg8fjmeffbbDMV9++SVWr14NAJgyZQqsViuqq6tx9OhRxMXFQavVYsSIEYiMjMS3336LKVOmSC2nW45VXME/f/szC7Exw5y2gfYv+HP/x3AYfj+kzYzgj9PvR0jfANxsvv0xzpgHwzE8wgQAeHzyfU4XeaU+/o6IlEFyEMybNw8A8Pbbb3c4xmw2QxAEx7YgCLh69SrMZjN0Ol2b/e7W+s6/o6+/NjS3mRHcfVG/e51+3v8c2e73RES+ossgKCkpQVZWltO+kSNHYvfu3V3+cFEUodFonLa1Wi3sdnu7+13R2YOYOyIAGDlsUIfbL97x/cMTh7j88zv9t4XQXv153oy9Ko9a+gTU1WurLoMgNjYWsbGxkn54eHg4zGYzhg4dCgCoqamBTqdDREQEzGazY1zrflfU1tbDbnf9EzhyUNPSEHtVHrX0CSi3V61W0+mbZ9fehrvo0UcfRWFhIQCgvLwcQUFBuPfee6HX61FUVISWlhb8/PPPuHjxIiZMmODOUoiIqAO9/jGdvXv3wmw2Y/Xq1Xjqqaewbt06zJkzB4GBgdi8eTMAYPbs2Th9+jTmzp0LAMjMzESfPn16uxQiIuoGjSiKvrG+chcuDXkn9qo8aukTUG6vsi4NERGR92MQEBGpnM/+rSGtVtP1IC/ia/X2BHtVHrX0CSiz16568tl7BERE1Du4NEREpHIMAiIilWMQEBGpHIOAiEjlGARERCrHICAiUjkGARGRyjEIiIhUjkFARKRyDAIPOXnyJObPn4/ExEQ8/fTTqKqqkrskt9q2bVunjzH1ZUVFRYiLi8OsWbOwZ88euctxq/r6esTHx+Py5ctyl+JW77zzDubMmYM5c+Y4/ly+mjAIPCQ1NRWbNm1CYWEhEhISsGnTJrlLcou6ujqkp6fjH//4h9yluIXJZEJOTg4+/vhjFBQUYN++ffjpp5/kLsstTp06hUWLFuHixYtyl+JWpaWlOHbsGPLz81FQUIDvv/8e//73v+Uuy6MYBB7Q3NyM1atXY+zYsQCAMWPG4MqVKzJX5R6HDx/G8OHD8eyzz8pdiluUlpZi6tSpCAsLQ3BwMAwGA4xGo9xluUVeXh4yMjJcfoysrxEEAWlpaQgMDERAQABGjRqF6upqucvyKJ/966O+JDAwEImJiQAAu92Od955BzNmzJC5KveYN28eACh2WchsNkMQBMe2TqfD6dOnZazIfTIzM+UuwSPuv/9+x/cXL15ESUkJ9u7dK2NFnscg6GUlJSXIyspy2jdy5Ejs3r0bzc3NSEtLw61bt7Bs2TKZKuwdnfWpZHa7HRrNf/9JX1EUnbbJd507dw7Lli3Dq6++iuHDh8tdjkcxCHpZbBgjPu8AAAEISURBVGwsYmNj2+xvaGjA8uXLERYWhh07diAgIECG6npPR30qXUREBMrLyx3bFotF8UsnanDy5EmsWrUK6enpmDNnjtzleBzvEXhIamoqhg0bhm3btiEwMFDuckiiadOm4fjx47BarWhsbMShQ4eg1+vlLot64MqVK1i5ciWys7NVGQIAZwQe8cMPP+Dw4cMYPXo0nnjiCQC315bff/99mSsjV4WHh2PNmjVITk6GzWbD/PnzERUVJXdZ1AM7d+7EzZs38cYbbzj2LVy4EIsWLZKxKs/iE8qIiFSOS0NERCrHICAiUjkGARGRyjEIiIhUjkFARKRyDAIiIpVjEBARqRyDgIhI5f4/3ckeX8+07nUAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(net[1].input.reshape(-1), net[1].output.reshape(-1), s=1);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The non-linear activation distorts and clips the output so it no longer resembles a Gaussian:" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD7CAYAAACCEpQdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcvklEQVR4nO3df1CT9x0H8HcgGH+hVZeIQ8aq7ZXOTnHSKe4utLdCQIg/0HkolTp1olN2ZR1OgcnUWTmL1XOKs6s779RtMio/5DDarR23Dm8ibVV6nPWmsCoaglgFlAjkuz+YmQghCSYh8H2/7jzJN094Ps+Xh3cenueb76MQQggQEZFUfPq7ACIi8jyGPxGRhBj+REQSYvgTEUmI4U9EJCGGPxGRhBj+REQSUvZ3AY66c6cFFovjH0kYN24kbt9udmNFfeettXlrXYD31uatdQHeW5u31gV4b219qcvHR4ExY0bYfH7AhL/FIpwK/0ev8VbeWpu31gV4b23eWhfgvbV5a12A99bm6rp42oeISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkNGDG+RMNNP6jhmGoqvuvWKu5HU33HvRDRUT/x/AncpOhKiX0bxV1az+5ax6a+qEeoscx/GlQsHWUbX7YAdUQ327tto6+bX2f3r6X/6hhPJKnAYfhT4NCb0fZzhx92/o+ffleRN6MF3yJiCTEI3/ySj2dflGr/b3yYunDtg6o1f79XQaRUxj+5JUG0sXSIX6+Nmt1hqtGBz3+fR5/U/LGN07qPwx/Ii/hqje8gfTGSf2H4U80QPU2MonIHu45RANUb0f4RPZwtA8RkYR45E/9qr9OXXCEzv9xGgo5MfypX/XXqQtXjdAZDHiBWE4MfyIP418d5A0Y/jSgDIbgdPavjsGwzeR9HAr/ffv24dSpUwCAiIgIbNiwAZs2bUJlZSWGDRsGAFi/fj0iIyNRXl6OHTt2wGw2IyYmBqmpqQCA6upqZGRkoKWlBWFhYdiyZQuUSr73kHNkPF0j4zaT+9lN3/LycnzyyScoKCiAQqHAqlWr8OGHH6KqqgpHjx6FRqOxLtva2or09HQcOXIEEyZMQHJyMsrKyhAREYG0tDT85je/QWhoKNLT05GXl4elS5e6dePIe3BMOpF3sfvbqFarsXHjRgwZMgQAMHnyZNTV1aGurg7p6ekwGo2IjIzE+vXrcfHiRQQHByMoKAgAoNfrYTAY8Nxzz6G1tRWhoaEAgPj4eOzdu5fhLxGOSe9/PH1Ej7Mb/s8//7z165qaGpw6dQrHjh3DuXPnkJWVBX9/fyQnJyM/Px/Dhw+HWq22Lq/RaGA0GlFfX9+lXa1Ww2g0unhTyBvwCN97uXsOIoDDQwcSh39Lr1y5guTkZGzYsAGTJk3C/v37rc8tW7YMhYWF0Ol0UCgU1nYhBBQKBSwWS4/tzhg3bqRTywPw6qMcb63NFXXxCH/wsLU/9HbPg6EO7EPeuv8D3lubq+tyKPwrKyvxs5/9DOnp6YiNjcXly5dRU1MDnU4HoDPMlUolAgICYDKZrK8zmUzQaDTd2hsaGrpcK3DE7dvNsFiEw8ur1f4wmbxzlLK31uaKurz1F4ec97CtA0P8ut+5zB57+5C37v+A99bWl7p8fBS9HjTbDf+bN29i3bp12L17N8LDwwF0hv3bb7+NWbNmYfjw4Th+/DgWLFiAadOm4dq1a6itrcXEiRNRUlKChQsXIjAwECqVCpWVlZgxYwaKioqg1Wqd2hAi8iyOMhrc7Ib/oUOHYDabkZ2dbW1LSEjA6tWrsWTJErS3tyMqKgpxcXEAgOzsbKSkpMBsNiMiIgLR0dEAgJycHGRmZqK5uRlTpkxBUlKSmzaJiIjssRv+mZmZyMzM7PG5xMTEbm3h4eEoLi7u1h4SEoL8/Pw+lEj9ifO+EA1OHJZBvbI1RPOD7Die3ycawBj+1Cc8H0w9sfVZAv6l6H0Y/kTkMr0dFHjfGBq58WYuREQSYvgTEUmIp32oy4geXsQlkgPDXyK9zcnS28f1iWjwYfhLhDNrUn95chTQo685Cqj/MPyJyO04Csj78IIvEZGEGP5ERBLiaR8i6jf8RHD/YfgTUb/htYD+w9M+REQSYvgTEUmI4U9EJCGGPxGRhBj+REQSYvgTEUmI4U9EJCGGPxGRhBj+REQSYvgTEUmI4U9EJCHO7TMI9XbHLiIigOE/KPGOXURkD8OfiLwOp3p2P4Y/EXkdTvXsfrzgS0QkIYfCf9++fYiNjUVsbCx27twJACgvL4der0dUVBR2795tXba6uhrx8fHQ6XTIyMhAe3s7AKCurg6JiYmIjo7G2rVr0dLS4obNISIiR9gN//LycnzyyScoKChAYWEhvvjiC5SUlCA9PR25ubkoLS1FVVUVysrKAABpaWnYvHkzTp8+DSEE8vLyAABbtmzB0qVLYTAY8NJLLyE3N9e9WyYB/1HDoFb7d/tHRGSP3fBXq9XYuHEjhgwZAj8/P0yePBk1NTUIDg5GUFAQlEol9Ho9DAYDbty4gdbWVoSGhgIA4uPjYTAY0NbWhoqKCuh0ui7t9HQejep58h8RkT12L/g+//zz1q9rampw6tQpvP7661Cr1dZ2jUYDo9GI+vr6Lu1qtRpGoxF37tzByJEjoVQqu7Q7Y9y4kU4t37ke7z0K9ubaiLyVrVFAD9s6MMTP1yXr8NbfTVfX5fBonytXriA5ORkbNmyAr68vampqrM8JIaBQKGCxWKBQKLq1P/r/cU8+tuf27WZYLMLh5dVqf5hM3jkuwFW1eetOSuQuvY0CctXvlDfmRl/q8vFR9HrQ7NAF38rKSixfvhxvvfUWFixYgICAAJhMJuvzJpMJGo2mW3tDQwM0Gg3Gjh2LpqYmdHR0dFmeiIj6h93wv3nzJtatW4ecnBzExsYCAKZNm4Zr166htrYWHR0dKCkpgVarRWBgIFQqFSorKwEARUVF0Gq18PPzQ1hYGEpLSwEAhYWF0Gq1btwsIiLqjd3TPocOHYLZbEZ2dra1LSEhAdnZ2UhJSYHZbEZERASio6MBADk5OcjMzERzczOmTJmCpKQkAEBWVhY2btyIAwcOYMKECXj33XfdtElEJBt+Ith5dsM/MzMTmZmZPT5XXFzcrS0kJAT5+fnd2gMDA3HkyJE+lEhE1Dt+Ith5nN5hAOAsnUTkakyUAYCzdBKRq3FuHyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkxKGeXoTj+YnIU5g0XoTj+YnIU3jah4hIQgx/IiIJ8bQPEQ1anO3TNoY/EQ1anO3TNp72ISKSEMOfiEhCDH8iIgkx/ImIJMTwJyKSEMOfiEhCHOrZD2yNPSYi8hSGfz/obewxEZEn8LQPEZGEGP5ERBLiaR8iko6t624P2zr6oZr+wfAnIunwuhtP+xARSYnhT0QkIYY/EZGEHA7/5uZmxMXF4fr16wCATZs2ISoqCvPmzcO8efPw4YcfAgDKy8uh1+sRFRWF3bt3W19fXV2N+Ph46HQ6ZGRkoL293cWbQkREjnIo/C9cuIAlS5agpqbG2lZVVYWjR4+iqKgIRUVFiIyMRGtrK9LT05Gbm4vS0lJUVVWhrKwMAJCWlobNmzfj9OnTEEIgLy/PLRtERET2ORT+eXl5yMrKgkajAQA8ePAAdXV1SE9Ph16vx969e2GxWHDx4kUEBwcjKCgISqUSer0eBoMBN27cQGtrK0JDQwEA8fHxMBgM7tsqIiLqlUNDPbdv397lcUNDA2bNmoWsrCz4+/sjOTkZ+fn5GD58ONRqtXU5jUYDo9GI+vr6Lu1qtRpGo9GpQseNG+nU8p3r4fw5ROQcb80NV9fVp3H+QUFB2L9/v/XxsmXLUFhYCJ1OB4VCYW0XQkChUMBisfTY7ozbt5thsQiHl1er/WEy9e9dOv1HDcNQFT9KQTSQ9Hdu9KQveebjo+j1oLlPyXT58mXU1NRAp9MB6AxzpVKJgIAAmEwm63ImkwkajaZbe0NDg/UU0mA2VKWU/oMkROSd+jTUUwiBt99+G3fv3kVbWxuOHz+OyMhITJs2DdeuXUNtbS06OjpQUlICrVaLwMBAqFQqVFZWAgCKioqg1WpduiFEROS4Ph35h4SEYPXq1ViyZAna29sRFRWFuLg4AEB2djZSUlJgNpsRERGB6OhoAEBOTg4yMzPR3NyMKVOmICkpyXVbQURETnEq/D/66CPr14mJiUhMTOy2THh4OIqLi7u1h4SEID8/vw8lEhGRq/ETvkREEmL4ExFJiOMQiYj+x9Y8/63mdjTde9APFbkPw5+I6H96m+ff+0b/Px2e9iEikhDDn4hIQgx/IiIJMfyJiCTE8CcikhDDn4hIQgx/IiIJMfyJiCTE8CcikhDDn4hIQgx/IiIJMfyJiCTE8CcikhDDn4hIQpzSmYjIjsE4zz/D3wX8Rw3DUBW7kmiwGozz/DOxXGCoSmlzxyAi8kY8509EJCGGPxGRhBj+REQSYvgTEUmI4U9EJCGGPxGRhBj+REQScij8m5ubERcXh+vXrwMAysvLodfrERUVhd27d1uXq66uRnx8PHQ6HTIyMtDe3g4AqKurQ2JiIqKjo7F27Vq0tLS4YVOIiMhRdsP/woULWLJkCWpqagAAra2tSE9PR25uLkpLS1FVVYWysjIAQFpaGjZv3ozTp09DCIG8vDwAwJYtW7B06VIYDAa89NJLyM3Ndd8WERGRXXbDPy8vD1lZWdBoNACAixcvIjg4GEFBQVAqldDr9TAYDLhx4wZaW1sRGhoKAIiPj4fBYEBbWxsqKiqg0+m6tBMRUf+xO73D9u3buzyur6+HWq22PtZoNDAajd3a1Wo1jEYj7ty5g5EjR0KpVHZpJyKi/uP03D4WiwUKhcL6WAgBhUJhs/3R/4978rEjxo0b6fRrepqFj4jIlTyVM65ej9PhHxAQAJPJZH1sMpmg0Wi6tTc0NECj0WDs2LFoampCR0cHfH19rcs76/btZlgswuHl1Wp/mEyunW+Ps3cS0ZNcnTM96Uue+fgoej1odjrJpk2bhmvXrqG2thYTJ05ESUkJFi5ciMDAQKhUKlRWVmLGjBkoKiqCVquFn58fwsLCUFpaCr1ej8LCQmi1WmdX6xU4eycRDRZOh79KpUJ2djZSUlJgNpsRERGB6OhoAEBOTg4yMzPR3NyMKVOmICkpCQCQlZWFjRs34sCBA5gwYQLeffdd124FERE5xeHw/+ijj6xfh4eHo7i4uNsyISEhyM/P79YeGBiII0eO9LFEIiJyNX7Cl4hIQgx/IiIJMfyJiCTEcYtERH30sK3D5vj7VnM7mu498HBFjmP4ExH10RA/3x6HfwOdQ8Dd/wmAvuNpHyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEK8mUsP/EcNw1AVu4aI+s7WXb685Q5fTLgeDFUpe7w7z8ld8/qhGiIaiGzd5ctb7vDF0z5ERBJi+BMRSYjhT0QkIYY/EZGEGP5ERBJi+BMRSeiphnouW7YMjY2NUCo7v83WrVvxn//8BwcOHEB7ezveeOMNJCYmAgDKy8uxY8cOmM1mxMTEIDU19emrJyKiPulz+AshUFNTg48//tga/kajEampqThx4gSGDBmChIQEzJw5ExMnTkR6ejqOHDmCCRMmIDk5GWVlZYiIiHDZhhARkeP6HP5Xr14FAKxYsQJff/01Fi9ejBEjRmDWrFl45plnAAA6nQ4GgwHf//73ERwcjKCgIACAXq+HwWBg+BMR9ZM+n/O/d+8ewsPDsX//fhw+fBh//vOfUVdXB7VabV1Go9HAaDSivr6+x3YiIuoffT7ynz59OqZPn259vGjRIuzYsQNr1661tgkhoFAoYLFYoFAourU7Y9y4kU7X2NO8GkRE/a0v2eTqPOtz+J8/fx5tbW0IDw8H0BnogYGBMJlM1mVMJhM0Gg0CAgJ6bHfG7dvNsFiEw8ur1f4wmfo2gwbfNIjInZzNpr7kmY+PoteD5j6f9mlqasLOnTthNpvR3NyMgoICvPPOOzh79iwaGxvx4MEDnDlzBlqtFtOmTcO1a9dQW1uLjo4OlJSUQKvV9nXVRET0lPp85P/qq6/iwoULmD9/PiwWC5YuXYoZM2YgNTUVSUlJaGtrw6JFizB16lQAQHZ2NlJSUmA2mxEREYHo6GiXbQQRETnnqcb5v/nmm3jzzTe7tOn1euj1+m7LhoeHo7i4+GlWR0RELsJP+BIRSYjhT0QkIYY/EZGEGP5ERBJi+BMRSYjhT0Qkoaca6jnQ+Y8ahqEqqbuAiCQldfINVSmhf6uoW/vJXfP6oRoiksHDto4ep5BpNbej6d4Dj9UhdfgTEXnaED9fmwedfZuNrG94zp+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkNOjn8+fduoiIuhv0qWjrbl0A79hFRPIa9OFPRDQQ9HZ7R3dg+BMReYHebu/oDrzgS0QkIY+G/8mTJzFnzhxERUXh2LFjnlw1ERE9xmOnfYxGI3bv3o0TJ05gyJAhSEhIwMyZM/Hcc895qgQiIvofj4V/eXk5Zs2ahWeeeQYAoNPpYDAYsH79eode7+OjcHqdj16jGTPM5jK2nmN7/6/b29q9sSZva/fGmgZDXwDOZ6C95RVCCOHUd+yjgwcP4v79+0hNTQUA/OUvf8HFixexbds2T6yeiIge47Fz/haLBQrF/9+JhBBdHhMRked4LPwDAgJgMpmsj00mEzQajadWT0REj/FY+M+ePRtnz55FY2MjHjx4gDNnzkCr1Xpq9URE9BiPXfAdP348UlNTkZSUhLa2NixatAhTp0711OqJiOgxHrvgS0RE3oOf8CUikhDDn4hIQgx/IiIJMfyJiCQ0KKZ03rNnD3x9fZGSktLtuYcPHyIjIwNVVVUYOnQocnJyMHnyZAghsHPnTnz88cfw8fHBtm3bMGPGDJfVVFdXh7S0NNy+fRvPPvsscnJyMGLEiC7LrFmzBjdv3gTQ+SG4L7/8Evn5+QgJCcHMmTMRFBRkXfbEiRPw9fX1SF03btxAXFwcvvWtbwEAvvGNb+DQoUM2+9JVHKmtvr4emzZtQkNDA3x8fLBhwwaEh4ejra3N5X128uRJHDhwAO3t7XjjjTeQmJjY5fnq6mpkZGSgpaUFYWFh2LJlC5RKpUPb8bTs1fbXv/4Vv/3tbyGEwMSJE7Fjxw6MHj0aBQUF2LVrF8aNGwcAeOWVV6yfuvdEXfv27cMHH3yAUaNGAQAWL16MxMREm33pSr3VVl1djY0bN1ofNzY2YvTo0SgpKXF7nwFAc3MzEhIS8Lvf/Q4TJ07s8pzb9jMxgN27d09s2rRJTJ06Vezdu7fHZd5//33xq1/9SgghxLlz58SPfvQjIYQQp06dEj/5yU9ER0eHuHr1qoiMjBRtbW0uq2316tWipKRECCHEvn37xM6dO3tdfs+ePSIzM1MIIcSlS5fEihUrXFaLs3UZDAZrnz3OVl96sra33npLHD16VAghxL///W8xe/Zs0d7e7vI+u3Xrlnj11VfFnTt3REtLi9Dr9eLKlStdlomNjRWfffaZEEKITZs2iWPHjjm8He6srampSfzgBz8Qt27dEkJ07lvbtm0TQgixdetWcfLkSZfW42hdQgiRnJwsPv30026vtdWXnqztkfv374vY2FhRUVEhhHBvnwkhxOeffy7i4uLElClTxFdffdXteXftZwP6tM/f/vY3fPvb38aPf/xjm8v8/e9/x9y5cwEAL7/8MhobG1FXV4eysjLMmTMHPj4+ePbZZzFhwgR89tlnLqmrra0NFRUV0Ol0AID4+HgYDAaby1+9ehWFhYX45S9/CQC4dOkSGhsbER8fj8WLF+PcuXMerevSpUv48ssvMW/ePCQlJeHy5csAbPelJ2uLjIxEXFwcACA4OBhmsxn37993eZ89PhHh8OHDrRMRPnLjxg20trYiNDS0S73O/uzdUVtbWxuysrIwfvx4AMALL7xg/Qvz0qVLKCgogF6vxy9+8QvcvXvXY3UBQFVVFQ4ePAi9Xo+tW7fCbDbb7EtXcqS2Rw4ePIiXX34ZYWFhANzbZwCQl5eHrKysHmc8cOd+NqDDf/78+Vi9enWvf9rX19dDrVZbH6vVaty6dQv19fVdOvtRuyvcuXMHI0eOtP7ZqlarYTQabS6fm5uLlStXYuTIkQAAhUKBH/7whzh+/Dh+/etfIzU1FY2NjR6rS6VSYe7cuSgoKMDKlSuxbt06PHz40GZfuoKjtel0OowePRoAcOjQIbz44ovw9/d3eZ89ua0ajaZLPT31hdFodPpn747axowZg8jISABAa2sr3nvvPbz22mvWen7605+iuLgYEyZMwNatWz1WV0tLC1588UWkpaWhoKAA9+7dQ25urs2+dCV7tT3S1NSEvLy8LrMNu7PPAGD79u3WNxp7dbtyPxsQ5/xPnTqFHTt2dGmbNGkSDh8+bPe14okJ5IQQ8PHx6XGiOR8f598Le6otODi426R1tiaxu3v3Lv75z39i+/bt1raEhATr19/5zncwdepUfPrpp9ZfYHfX9fi1k4iICOzatQtXr1612ZfOeto+A4DDhw/j+PHjOHr0KADX9Nnj7E1EaOv5J5eztx3uqO2RpqYmrFu3DiEhIViwYAEAYP/+/dbnV61aZX2T8ERdI0aMwO9//3vr4xUrViA9PR1ardbtkz462mfFxcV47bXXrOf3Aff2mT3u3M8GRPjHxMQgJiamT68dP3486uvrrRcvGxoaoNFoEBAQgPr6eutyj9pdUduji48dHR3w9fXtdRK7srIyaLVaqFQqa1thYSG+973vWWsWQsDPz89jdR05cgRxcXEYM2aMdf1KpdJmXzrrafts586dKCsrw7FjxxAQEADANX32uICAAJw/f976+Ml6npyo8FFfjB07Fk1NTQ5th7tqAzqPGFeuXIlZs2YhPT0dQOebwQcffIDly5cD6OwjVwwicLSuuro6lJeXY9GiRdb1K5VKm33pSo70GdB5oTw5Odn62N19Zo8797MBfdrHERERESgq6rwp8vnz56FSqfDNb34TWq0WJ0+eREdHB2pra1FTU4Pvfve7Llmnn58fwsLCUFpaCqAzmGxNYvf55593+5Pv8uXL+MMf/gCg83pAdXW1S0YiOVpXRUUF8vPzAQDnzp2DxWLBpEmTbPalKzha2+HDh/Gvf/0Lf/rTn6zBD7i+z+xNRBgYGAiVSoXKykoAQFFREbRarVM/e3fV1tHRgTVr1iAmJgYZGRnWI8Lhw4fj/fffx4ULFwAAR48edelRrL26hg4dinfeeQdfffUVhBA4duwYIiMjbfalKzkysaQQAl988QWmT59ubXN3n9nj1v3MqcvDXmrv3r1dRvv88Y9/FHv27BFCCNHa2io2bNgg5syZI+bPny+qqqqEEEJYLBaRnZ0t5syZI+bMmSP+8Y9/uLSm69evi9dff13ExMSIFStWiK+//rpbbUIIsWrVKlFWVtbltU1NTSIlJUXExsaKuLg4cfbsWY/WdevWLbF8+XIRGxsr4uPjRXV1tRDCdl96qjaLxSLCwsLEK6+8IubOnWv9d+vWLbf0WXFxsYiNjRVRUVHivffeE0J0/rwuXrwohBCiurpaLFy4UOh0OvHzn/9cmM3mXrfDlXqr7cyZM+KFF17o0kfp6elCCCEqKirE/PnzRXR0tFizZo24d++ex+oSonMk2aPnN27caO0zW33pydoaGhrE7Nmzu73O3X32yKuvvmod7eOJ/YwTuxERSWjQn/YhIqLuGP5ERBJi+BMRSYjhT0QkIYY/EZGEGP5ERBJi+BMRSYjhT0Qkof8Cq76f4vhhmAsAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[1].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However, the next `Linear` module restores the Gaussian distribution! How does this happen when neither its inputs nor its parameters have a Gaussian distribution? (Answer: the [central limit theorem](https://en.wikipedia.org/wiki/Central_limit_theorem) which we briefly covered [earlier](https://nbviewer.jupyter.org/github/dkirkby/MachineLearningStatistics/blob/master/notebooks/Statistics.ipynb))." ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD+CAYAAADS3wWuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcs0lEQVR4nO3df1DUZeIH8PfC4qrhTenshkccN3bN0OAETqZyf+yOf7gLLhu2eTcoSZ1dmlM4eQ2NwiZj980YoyhOcbq7rpvIm4njkiUG15y6nGnwUrlOjobrHANKqGVBS5Zjl/3xfP9wXKVdXBbZH/C8X//I5/k86z7P4+fz9sPnx/NRCCEEiIhIKknxbgAREcUew5+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEJTCv/XX38d69evh9FoxFtvvQUA2LNnD/R6PYqKilBUVIQTJ04AANrb22EymaDX61FbWxv4O7q7u2E2m2EwGFBZWQmv1xuF7hAR0ZSIMD799FNRXFwsPB6PGBsbE2vXrhUXLlwQhYWFwm63T6g7NjYmdDqd+Oqrr4TH4xFbt24VH3/8sRBCCKPRKD777DMhhBB79uwRR44cCffVREQUJWGP/FetWoW3334bSqUSw8PD8Pl8mD9/PgYGBlBRUQGTyYS6ujr4/X50dnYiMzMTGRkZUCqVMJlMsNls6O/vh8vlQm5uLgDAbDbDZrNF/T82IiIKbUqnfVJSUlBXVwej0Yi8vDx4vV6sWbMG+/fvR2NjI86ePYumpiYMDg5CrVYHPqfRaGC324PK1Wo17Hb7zPeGiIimRDnVijt37sQTTzyBJ598EqdOncKhQ4cC67Zs2YLm5mYYDAYoFIpAuRACCoUCfr8/ZHkkLl8ehd8f35kolixJxfCwM65tSBQci+s4FtdxLK6L91gkJSlwxx23Tbo+bPhfuHAB4+PjuPfee7FgwQLo9Xq0tbXh9ttvh8FgAHA1zJVKJdLS0uBwOAKfdTgc0Gg0QeVDQ0PQaDQRdcTvF3EP/2vtoKs4FtdxLK7jWFyXyGMR9rTPxYsXYbFYMD4+jvHxcXz44Yd44IEHsH//fnz//ffweDx49913sW7dOuTk5KCnpwd9fX3w+XxobW2FVqtFeno6VCoVOjo6AABWqxVarTbqnSMiotDCHvnrdDp0dnZiw4YNSE5Ohl6vx9NPP4077rgDmzZtgtfrhV6vR2FhIQCguroaZWVlcLvd0Ol0yM/PBwDU1NTAYrHA6XQiOzsbpaWl0e0ZERFNSiHE7JjSeXjYGfdfodTqRXA4RuLahkTBsbiOY3Edx+K6eI9FUpICS5akTr4+hm0hIqIEwfAnIpIQw5+ISEIMfyIiCU35IS8i2S360QLMVwXvMuMeXxxaQ3RrGP5EUzRfpYTpWWtQ+fuvFMWhNUS3huFPdIvGPT6o1YuCyl1uL0aujMWhRUThMfyJbtG8lORJfyPgHe+UqBj+RDeY7Lz+dPA3AkpkDH+iG0x2Xh+I/Nw+fyOgRMZbPYmIJMTwJyKSEMOfiEhCDH8iIgkx/ImIJMTwJyKSEMOfiEhCDH8iIgkx/ImIJMTwJyKSEKd3IIoxzvlDiYDhTxRjnPOHEsGUTvu8/vrrWL9+PYxGI9566y0AQHt7O0wmE/R6PWprawN1u7u7YTabYTAYUFlZCa/XCwAYGBhASUkJ8vPzsWPHDoyOjkahO0RENBVhw//06dP4xz/+gZaWFvztb39DQ0MD/vOf/6CiogL19fVoa2tDV1cXTp48CQAoLy/H3r17cfz4cQgh0NjYCADYt28fNm/eDJvNhuXLl6O+vj66PSMiokmFDf9Vq1bh7bffhlKpxPDwMHw+H65cuYLMzExkZGRAqVTCZDLBZrOhv78fLpcLubm5AACz2QybzQaPx4MzZ87AYDBMKCcioviY0jn/lJQU1NXV4U9/+hPy8/MxODgItVodWK/RaGC324PK1Wo17HY7Ll++jNTUVCiVygnlkViyJDWi+tES6kKdrDgWM28ujOlc6MNMSeSxmPIF3507d+KJJ57Ak08+id7eXigUisA6IQQUCgX8fn/I8mt/3uiHy+EMDzvh94uIPjPT1OpFcDh4SQ6Yu2MR7511to/pXN0upiPeY5GUpLjpQXPY0z4XLlxAd3c3AGDBggXQ6/X49NNP4XA4AnUcDgc0Gg3S0tImlA8NDUGj0WDx4sUYGRmBz+ebUJ+IiOIjbPhfvHgRFosF4+PjGB8fx4cffoji4mL09PSgr68PPp8Pra2t0Gq1SE9Ph0qlQkdHBwDAarVCq9UiJSUFK1euRFtbGwCgubkZWq02uj0jIqJJhT3to9Pp0NnZiQ0bNiA5ORl6vR5GoxGLFy9GWVkZ3G43dDod8vPzAQA1NTWwWCxwOp3Izs5GaWkpAKCqqgq7d+/G4cOHsXTpUrz66qvR7RkREU1qSuf8y8rKUFZWNqEsLy8PLS0tQXWzsrLQ1NQUVJ6eno6GhoZpNpOIiGYS5/YhIpIQp3cgKS360QLMV3HzJ3lx6ycpzVcpJ51fh0gGPO1DRCQhhj8RkYQY/kREEmL4ExFJiBd8aU6bTXf18A1fFEuzY68gmqbZdFcP3/BFscTTPkREEmL4ExFJiOFPRCQhhj8RkYQY/kREEmL4ExFJiOFPRCQhhj8RkYQY/kREEmL4ExFJiOFPRCQhhj8RkYQY/kREEppS+B88eBBGoxFGoxEHDhwAAOzZswd6vR5FRUUoKirCiRMnAADt7e0wmUzQ6/Wora0N/B3d3d0wm80wGAyorKyE1+uNQneIiGgqwoZ/e3s7PvnkExw9ehTNzc34/PPPceLECXR1deGdd96B1WqF1WrFunXr4HK5UFFRgfr6erS1taGrqwsnT54EAJSXl2Pv3r04fvw4hBBobGyMeueIiCi0sOGvVquxe/duzJs3DykpKbj77rsxMDCAgYEBVFRUwGQyoa6uDn6/H52dncjMzERGRgaUSiVMJhNsNhv6+/vhcrmQm5sLADCbzbDZbFHvHBERhRb2ZS733HNP4Ofe3l4cO3YMR44cwenTp1FVVYVFixZh+/btaGpqwsKFC6FWqwP1NRoN7HY7BgcHJ5Sr1WrY7faIGrpkSWpE9aMl1JuWZMWxiJ3ZNNazqa3RlshjMeU3eZ0/fx7bt2/Hc889h2XLluHQoUOBdVu2bEFzczMMBgMUCkWgXAgBhUIBv98fsjwSw8NO+P0ios/MNLV6ERwOvlMJmD1jkcg731SNe3yYl5IcVJ6Ir3ecLdtFLMR7LJKSFDc9aJ5S+Hd0dGDnzp2oqKiA0WjEF198gd7eXhgMBgBXw1ypVCItLQ0OhyPwOYfDAY1GE1Q+NDQEjUYz3T4RBZlN7+qNFF/vSNEQdm/55ptv8NRTT6G2thZ5eXkArob9/v37sWbNGixcuBDvvvsuHnroIeTk5KCnpwd9fX2466670Nraiocffhjp6elQqVTo6OjA/fffD6vVCq1WG/XOkTxm07t6iRJB2PB/88034Xa7UV1dHSgrLi7Gtm3bsGnTJni9Xuj1ehQWFgIAqqurUVZWBrfbDZ1Oh/z8fABATU0NLBYLnE4nsrOzUVpaGqUuERFROGHD32KxwGKxhFxXUlISVJaXl4eWlpag8qysLDQ1NU2jiURENNP4hC8RkYQY/kREEmL4ExFJiOFPRCQhhj8RkYQY/kREEmL4ExFJiOFPRCQhhj8RkYQY/kREEmL4ExFJiOFPRCShuTkBOs1Zc3ne/kiNe3whX1aTiC95ocTDvYhmFc7bfx1f8kK3gqd9iIgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCU0p/A8ePAij0Qij0YgDBw4AANrb22EymaDX61FbWxuo293dDbPZDIPBgMrKSni9XgDAwMAASkpKkJ+fjx07dmB0dDQK3SEioqkIG/7t7e345JNPcPToUTQ3N+Pzzz9Ha2srKioqUF9fj7a2NnR1deHkyZMAgPLycuzduxfHjx+HEAKNjY0AgH379mHz5s2w2WxYvnw56uvro9szIiKaVNjwV6vV2L17N+bNm4eUlBTcfffd6O3tRWZmJjIyMqBUKmEymWCz2dDf3w+Xy4Xc3FwAgNlshs1mg8fjwZkzZ2AwGCaUExFRfISd2+eee+4J/Nzb24tjx47hkUcegVqtDpRrNBrY7XYMDg5OKFer1bDb7bh8+TJSU1OhVConlEdiyZLUiOpHS6iJtGTFsUhc8fy34XZxXSKPxZQndjt//jy2b9+O5557DsnJyejt7Q2sE0JAoVDA7/dDoVAElV/780Y/XA5neNgJv19E9JmZplYvgsPBKbOA+I1FIu9MiSRe2yn3keviPRZJSYqbHjRP6YJvR0cHHnvsMTz77LN46KGHkJaWBofDEVjvcDig0WiCyoeGhqDRaLB48WKMjIzA5/NNqE9ERPERNvy/+eYbPPXUU6ipqYHRaAQA5OTkoKenB319ffD5fGhtbYVWq0V6ejpUKhU6OjoAAFarFVqtFikpKVi5ciXa2toAAM3NzdBqtVHsFhER3UzY0z5vvvkm3G43qqurA2XFxcWorq5GWVkZ3G43dDod8vPzAQA1NTWwWCxwOp3Izs5GaWkpAKCqqgq7d+/G4cOHsXTpUrz66qtR6hIREYUTNvwtFgssFkvIdS0tLUFlWVlZaGpqCipPT09HQ0PDNJpIREQzjU/4EhFJiOFPRCQhhj8RkYQY/kREEmL4ExFJaMpP+BLR7DDu8YV8Etrl9mLkylgcWkSJiOFPNMfMS0mG6VlrUPn7rxSBEy/QNQx/SkiLfrQA81XcPImihXsXJaT5KuWkR69EdOt4wZeISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCTH8iYgkxPAnIpIQw5+ISEIMfyIiCXF6ByJJcLZPuhHDn0gSnO2TbsTTPkREEppy+DudThQWFuLixYsAgD179kCv16OoqAhFRUU4ceIEAKC9vR0mkwl6vR61tbWBz3d3d8NsNsNgMKCyshJer3eGu0JERFM1pfA/d+4cNm3ahN7e3kBZV1cX3nnnHVitVlitVqxbtw4ulwsVFRWor69HW1sburq6cPLkSQBAeXk59u7di+PHj0MIgcbGxqh0iIiIwptS+Dc2NqKqqgoajQYAMDY2hoGBAVRUVMBkMqGurg5+vx+dnZ3IzMxERkYGlEolTCYTbDYb+vv74XK5kJubCwAwm82w2WzR6xUREd3UlC74vvjiixOWh4aGsGbNGlRVVWHRokXYvn07mpqasHDhQqjV6kA9jUYDu92OwcHBCeVqtRp2uz2ihi5ZkhpR/WgJdbeErDgWc8dM/ltyu7gukcdiWnf7ZGRk4NChQ4HlLVu2oLm5GQaDAQqFIlAuhIBCoYDf7w9ZHonhYSf8fjGd5s4YtXoRHA7eFwHM3FjwdY2JYaa2a+4j18V7LJKSFDc9aJ7WXvfFF1+gt7cXBoMBwNUwVyqVSEtLg8PhCNRzOBzQaDRB5UNDQ4FTSCQ3vq6RKD6mdaunEAL79+/H999/D4/Hg3fffRfr1q1DTk4Oenp60NfXB5/Ph9bWVmi1WqSnp0OlUqGjowMAYLVaodVqZ7QjREQ0ddM68s/KysK2bduwadMmeL1e6PV6FBYWAgCqq6tRVlYGt9sNnU6H/Px8AEBNTQ0sFgucTieys7NRWlo6c70gIqKIRBT+H330UeDnkpISlJSUBNXJy8tDS0tLUHlWVhaampqm0UQiIpppfMKXiEhCDH8iIgkx/ImIJMTwJyKSEMOfiEhCDH8iIgkx/ImIJMTwJyKSEMOfiEhCDH8iIgkx/ImIJMSJ1CkmOG8/UWLh3kgxwXn7iRILw59IcuMeX8jXDbrcXoxcGYtDiygWGP5EkpuXkjzpb2V8IePcxQu+REQSYvgTEUmI4U9EJCGGPxGRhBj+REQSYvgTEUmI4U9EJKEphb/T6URhYSEuXrwIAGhvb4fJZIJer0dtbW2gXnd3N8xmMwwGAyorK+H1egEAAwMDKCkpQX5+Pnbs2IHR0dEodIWIiKYqbPifO3cOmzZtQm9vLwDA5XKhoqIC9fX1aGtrQ1dXF06ePAkAKC8vx969e3H8+HEIIdDY2AgA2LdvHzZv3gybzYbly5ejvr4+ej0iIqKwwoZ/Y2MjqqqqoNFoAACdnZ3IzMxERkYGlEolTCYTbDYb+vv74XK5kJubCwAwm82w2WzweDw4c+YMDAbDhHIiIoqfsNM7vPjiixOWBwcHoVarA8sajQZ2uz2oXK1Ww2634/Lly0hNTYVSqZxQHqklS1Ij/kw0hJoDRVYci7lvOv/G3C6uS+SxiHhuH7/fD4VCEVgWQkChUExafu3PG/1weSqGh53w+0XEn5tJavUiOByc7QSYfCw4dfPcEun2zn3kuniPRVKS4qYHzRHvpWlpaXA4HIFlh8MBjUYTVD40NASNRoPFixdjZGQEPp8PycnJgfo0N3HqZqLZIeJbPXNyctDT04O+vj74fD60trZCq9UiPT0dKpUKHR0dAACr1QqtVouUlBSsXLkSbW1tAIDm5mZotdqZ7QUREUUk4iN/lUqF6upqlJWVwe12Q6fTIT8/HwBQU1MDi8UCp9OJ7OxslJaWAgCqqqqwe/duHD58GEuXLsWrr746s70gIqKITDn8P/roo8DPeXl5aGlpCaqTlZWFpqamoPL09HQ0NDRMs4lERDTTeGWOiELiG77mNoY/EYXEN3zNbZzbh4hIQgx/IiIJMfyJiCTE8CcikhDDn4hIQgx/IiIJMfyJiCTE8CcikhAf8iKiiEz25O+1dTQ7MPyJKCKTPfkLcOru2YThT9Nys6M/Ikp8DH+alpvN+0JEiY8XfImIJMTwJyKSEMOfiEhCDH8iIgkx/ImIJMTwJyKSEMOfiEhCDH8iIgnd0kNeW7ZswaVLl6BUXv1rXnjhBXz11Vc4fPgwvF4vHn30UZSUlAAA2tvb8dJLL8HtdqOgoAC7du269dYTEdG0TDv8hRDo7e3F3//+90D42+127Nq1C++99x7mzZuH4uJirF69GnfddRcqKirQ0NCApUuXYvv27Th58iR0Ot2MdYSI4m+yaT9cbi9GrozFoUU0mWmH/5dffgkA2Lp1K7777jv88pe/xG233YY1a9bg9ttvBwAYDAbYbDasWrUKmZmZyMjIAACYTCbYbDaG/yyw6EcLMF/FWUBoam427cdIHNpDk5v2Xn3lyhXk5eXh+eefh8fjQWlpKQoKCqBWqwN1NBoNOjs7MTg4GFRut9sj+r4lS1Kn29QZJeNkZpzDh2aCjPtOIvd52uG/YsUKrFixIrC8ceNGvPTSS9ixY0egTAgBhUIBv98PhUIRVB6J4WEn/H4x3ebOCLV6ERwOuY5fEnnjpdlFxn0nnn1OSlLc9KB52nf7nD17FqdOnQosCyGQnp4Oh8MRKHM4HNBoNEhLSwtZTkRE8THt8B8ZGcGBAwfgdrvhdDpx9OhRvPzyyzh16hQuXbqEsbExfPDBB9BqtcjJyUFPTw/6+vrg8/nQ2toKrVY7k/0gIqIITPu0z9q1a3Hu3Dls2LABfr8fmzdvxv33349du3ahtLQUHo8HGzduxH333QcAqK6uRllZGdxuN3Q6HfLz82esE0REFJlbuo3jmWeewTPPPDOhzGQywWQyBdXNy8tDS0vLrXwdERHNED7hS0QkIYY/EZGE+PQOEUUdn/xNPAx/4lO8FHV88jfxcI8nzFcpQ+6YAJ/kJZqreM6fiEhCDH8iIgnxtA8RxQ0vBMcPw5+I4oYXguOH4S8R3tVDRNcwCSQy2V09vKOHSD684EtEJCGGPxGRhHjaZw7iuX0iCocJMQfx3D7NdrwFNPoY/kSUcHgLaPTxnD8RkYQY/kREEuJpHyKaNXgtYOYw/Gcx3tVDsuG1gJnD5JjFeFcPEU0Xw38W4BE+0c3xdFDkYpoo77//Pg4fPgyv14tHH30UJSUlsfz6WYtH+EQ3N9npoL9VF/I/hUnELPztdjtqa2vx3nvvYd68eSguLsbq1avxs5/9LFZNSHg8wieaWbxGMLmYJU17ezvWrFmD22+/HQBgMBhgs9nw9NNPT+nzSUmKaDZvyiJpR2rqfKhChLl73AfVvOSQn3n8/z4IKnvToofmjgUh60e7PBbfMVvKE7FNiVaeiG0KVT7ZaaLJ9s1Jy91eOJ2ukN8LxDe3wn23QgghYtGQN954A//73/+wa9cuAMBf//pXdHZ24re//W0svp6IiG4Qs4e8/H4/FIrr/xMJISYsExFR7MQs/NPS0uBwOALLDocDGo0mVl9PREQ3iFn4//znP8epU6dw6dIljI2N4YMPPoBWq43V1xMR0Q1idsH3zjvvxK5du1BaWgqPx4ONGzfivvvui9XXExHRDWJ2wZeIiBIHZ/UkIpIQw5+ISEIMfyIiCTH8iYgkxPAnIpIQwz+M1157Db/73e9CrhsfH0d5eTkKCgrw0EMP4cKFCzFuXewMDAygpKQE+fn52LFjB0ZHR4Pq9Pf3Y8WKFSgqKkJRUREef/zxOLQ0et5//32sX78eer0eR44cCVrf3d0Ns9kMg8GAyspKeL3eOLQyNsKNxcGDB7F27drAthCqzlzidDpRWFiIixcvBq1L2O1CUEhXrlwRe/bsEffdd5+oq6sLWeePf/yjeP7554UQQpw+fVr84he/iGUTY2rbtm2itbVVCCHEwYMHxYEDB4Lq2Gy2wHjMNd9++61Yu3atuHz5shgdHRUmk0mcP39+Qh2j0Sg+++wzIYQQe/bsEUeOHIlHU6NuKmOxfft28c9//jNOLYytf/3rX6KwsFBkZ2eLr7/+Omh9om4XPPKfxIcffoif/vSn+NWvfjVpnY8//hgPPvggAOCBBx7ApUuXMDAwEKsmxozH48GZM2dgMBgAAGazGTabLajev//9b/z3v/9FUVERSktL8cUXX8S6qVFz46y0CxcuDMxKe01/fz9cLhdyc3MBTD5Gc0G4sQCArq4uvPHGGzCZTHjhhRfgdrvj1Nroa2xsRFVVVcjpahJ5u2D4T2LDhg3Ytm0bkpNDT70MAIODg1Cr1YFltVqNb7/9NhbNi6nLly8jNTUVSuXVB8LVajXsdntQPZVKhQcffBBHjx7F448/jqeeegrj4+Oxbm5U/PDfWqPRTBiDUNtCqDGaC8KNxejoKO69916Ul5fj6NGjuHLlCurr6+PR1Jh48cUXsXLlypDrEnm7kP7NIceOHcNLL700oWzZsmX485//HPaz4gczkwohkJQ0u/8/DTUemZmZQTOwhpqRtaysLPCzTqfDK6+8gi+//BJZWVnRaWwMhZuVVqZZa8P19bbbbsMf/vCHwPLWrVtRUVERmM5dJom8XUgf/gUFBSgoKJjWZ++8804MDg7iJz/5CQBgaGho1s9UGmo8PB4PVq9eDZ/Ph+Tk5ElnZG1oaEBhYSHuuOMOAFc39Gu/Lcx2aWlpOHv2bGD5h2Pww1lr58K2MJlwYzEwMID29nZs3LgRwNzaDiKVyNvF7D5MjTOdTger9eor4s6ePQuVSoUf//jHcW7VzEtJScHKlSvR1tYGAGhubg45I+uZM2fQ1NQEADh9+jT8fj+WLVsW07ZGS7hZadPT06FSqdDR0QEAsFqtc3bW2nBjMX/+fLz88sv4+uuvIYTAkSNHsG7duji2OH4SeruI26XmWaKurm7C3T5/+ctfxGuvvSaEEMLlconnnntOrF+/XmzYsEF0dXXFq5lRd/HiRfHII4+IgoICsXXrVvHdd98JISaOx7fffisee+wxYTQahdlsFt3d3fFs8oxraWkRRqNR6PV68fvf/14IIcSvf/1r0dnZKYQQoru7Wzz88MPCYDCI3/zmN8LtdsezuVEVbixsNltg/e7du+f0WFyzdu3awN0+s2G74KyeREQS4mkfIiIJMfyJiCTE8CcikhDDn4hIQgx/IiIJMfyJiCTE8CciktD/A6nZGjTIG6KTAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[2].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The next activation is `ReLU`, which effectively piles up all negative values from the previous `Linear` module into the zero bin:" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD7CAYAAAB+B7/XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcFElEQVR4nO3de1xUdd4H8M/gBQXZUJxhzG0xtFSKSbusPK5LDz2vGEEERS3N5NnN0K5s1Fpec9t9mXbxwcral5TlVtAjtSnSZWQ383llkKZWUFIKbqYIMwN44Srg/J4/Ws84zOhwmZlz5szn/Y/9zu8AX76ePv36zZkzGiGEABERqUaQ3AUQEZFnMdiJiFSGwU5EpDIMdiIilWGwExGpDIOdiEhlGOxERCrTX+4CAODUqWbYbPLeTh8RMQT19U2y1qAU7IUde2HHXtjJ3YugIA2GDg295Lwigt1mE7IH+4U66GfshR17Ycde2Cm5F9yKISJSGQY7EZHKMNiJiFSGwU5EpDIMdiIilWGwExGpDIOdiEgGNfXN2PDu16ipb/b492awExHJYOuuIyirasDWXUc8/r0V8QYlIqJAc+dt1wA48u8/PYvBTkQkgxERoXhkzgSvfG9uxRARqQyDnYhIZRjsREQqw2AnIvKixpZ2fLz3GBpb2n32MxnsRERetKe8Bu9+WoU95TU++5m8K4aIyIumxI5w+NMXGOxERF4UFjIQSZOifPozuRVDRKQyDHYiIpVhsBMRqQyDnYjIQ+S4tdEVBjsRkYfIcWujK7wrhojIQ+S4tdEVrtiJiProwodmNLV2IGlSFMJCBspaD4OdiKgPauqbse7tA1770IzeYLATEfXShVBvbO1E2OD+XvnQjN7odrA3NTUhJSUFJ06ccJqrqKhAeno6jEYjVqxYgc7OTo8WSUSkNI0t7Xjp72VSqC+9+yaMiAiVuywA3Qz2b775BvPmzcOPP/7ocn7JkiV48sknsXPnTgghUFBQ4MkaiYgUZ095DWobWqEfNlhRoQ50M9gLCgqwevVq6HQ6p7nq6mq0tbVhwoSfP+IpPT0dJpPJs1USESnMlNgRmJMwGssUFupAN293XLNmzSXnLBYLtFqtNNZqtTCbzX2vjIhIweR4uFd39fk+dpvNBo1GI42FEA7j7oiIGNLXMjxCqw2TuwTFYC/s2As79sJOyb3oc7Dr9XpYrVZpXFdX53LL5nLq65tgs4m+ltInWm0YrNZGWWtQCvbCjr2wYy/s5O5FUJDmsgviPt/uOHLkSAQHB+PAgQMAgMLCQsTHx/f12xIRKcqFNyHV1DfLXYpbvQ72zMxMlJeXAwCef/55rF27FlOnTkVLSwsyMjI8ViARkRJs3XVEUW9CuhyNEELePRBwK0Zp2As79sIu0HtRU9+MrbuO4M7broFhnF7RWzF8CBgRUTeMiAjFI3MmyF1Gt/CRAkRELijl2eq9wWAnInLh4y+O4d1Pq/DxF8fkLqXHGOxERC78ZGl0+NOfcI+diOgildWn8fqHFUibcjX69wtSzBMbe4LBTkR0kdc+OATLqTZs++wo1i2eLHc5vcJgJyLCz7cz5v3jBwQP6AcAuH7UMJkr6j0GOxERgLx//IBDP54GABhGD0Pab6Nlrqj3+OIpEQW8mvpmWE61AQDGXnUFFk6Lkf1zS/uCK3YiCmiNLe3IKfgadWfOYfgvgvHAzFi/DnWAK3YiCnAffXEMdWfOAQAirhjk96EOMNiJKIDV1DdjT3kNACB0UH9kTB0nc0Wewa0YIgpIldWnsf6dr3Gu06a4D6PuK67YiSjgNLa046X3ynCu04bg/kGqCnWAK3YiCjCNLe3Y/OEhNLZ2Imxwfzw826CqUAcY7EQUQBpb2vHKtnL8cPwMxl51hSrugHGFWzFEFDA++uIYfjh+BgDQr59GlaEOMNiJKEBUVp/GroPVAIDQwf0x//axMlfkPQx2IgoIr39YgY5/v1i6XGUvlnbFPXYiUr3GlnaMjxoKmw24d/p4VYc6wGAnIpUrq6rDX7d9i3OdNsxJGI0xI8PlLsnruBVDRKq2+YND0v3qU2JHyF2OTzDYiUiVGlvasf2zo4i5ehhCB/fH/TOvV+1dMF1xK4aIVKemvhnr3/kaDU0/P9xrTsJoGEYPl7kq3+GKnYhU5/UPDkmhHjqof8BswVzAYCciVdl7qBZVNY0AgKAg4A9zDAGzBXNBt4K9qKgIycnJSExMRF5entP8d999h1mzZiE1NRWLFy/G2bNnPV4oEZE7ldWnsWnHIWm8dP6NAXEXTFdug91sNiMnJwf5+fnYvn07tm7disrKSodz1qxZg6ysLOzYsQNXX301Nm/e7LWCiYhcaWxpx/p3vpLGvx6vDchQB7oR7CUlJYiLi0N4eDhCQkJgNBphMpkczrHZbGhubgYAtLa2YtCgQd6plojIhcaWdjz/v1/hXKeQjqn5kQHuuA12i8UCrVYrjXU6Hcxms8M5S5cuxcqVKzFlyhSUlJRg7ty5nq+UiOgSCj49guOWZmmcYbw24PbVL+b2dkebzQaNRiONhRAO47a2NqxYsQJbtmyBwWDAG2+8gSeeeAK5ubndLiIiYkgPy/YOrTZM7hIUg72wYy/slNiLN4q+xefl9sXmkvk3Iv7Gq7z+c5XYiwvcBrter8f+/fulsdVqhU6nk8aHDx9GcHAwDAYDAODOO+/ECy+80KMi6uubYLMJ9yd6kVYbBqu1UdYalIK9sGMv7JTYC9PeY3h/d5U0HqUfgvFXhXu9Trl7ERSkueyC2O1WzOTJk1FaWoqGhga0traiuLgY8fHx0nxUVBRqa2tx9OhRAMAnn3yC2NhYD5RORHRpjS3tKPi0yuFY5vTrZKpGWdyu2CMjI5GdnY2MjAx0dHRg9uzZMBgMyMzMRFZWFmJjY7F27Vo88sgjEEIgIiICTz/9tC9qJ6IA9jfT9w7jxakxqn9qY3dphBDy7oGAWzFKw17YsRd2SupF4WdHUfj5j9J4cWoMJsXoffbz5e6Fu60YPiuGiPxKwa4jMO07Lo0zjNf6NNT9AR8pQER+Y++hWodQjxkVjv+c+EsZK1ImBjsR+Y2LHxcAAItTr5epEmVjsBORX9j4928cxnckjA7oNyFdDoOdiBRv76FaHDxSL41vvCYCUydFyViRsjHYiUjRGlvaHbZgBvbX4KFZN8hYkfLxrhgiUqya+mY8m39QGmsA/HHeRPkK8hNcsRORYm149xucae6Qxn+YYwjYR/H2BIOdiBSprKoO1tNt0nhxakxAfW5pXzDYiUhxyqrqsOHdMmmc9ptRfBNSDzDYiUhRKqtPO4Q6AKT9NlqmavwTg52IFGXd2wcdxrdNvFKmSvwXg52IFOP5dw7g4ucBTomNxN3GcfIV5KcY7ESkCG/v/B6Hjp2RxuGhA3DPND5fvTcY7EQku4JdR7Drq5MOx55aOEmmavwfg52IZHfxExsB4JE5Bj4Hpg8Y7EQkq3Vvf+kwTvvNKN6v3kd8pAARyWb15lIct7ZK49EjQnlrowdwxU5EsthUWO4Q6gCw4r+5r+4JDHYi8rmyqjrsrbA6HFucGiNTNerDYCcin+v6ztKpv76KjwzwIAY7EfnUyldLHMajR4TijtuukakadeKLp0TkM8s2fQ7zqXMOx7iv7nlcsRORT+z+6oRTqHNf3TsY7ETkE2/uPOwwvvaXYdxX9xIGOxF53aJndjmMBw8Elt59i0zVqF+3gr2oqAjJyclITExEXl6e0/zRo0exYMECpKamYuHChThz5oyL70JEgWjZps/RedETGzUAXn70NtnqCQRug91sNiMnJwf5+fnYvn07tm7disrKSmleCIH7778fmZmZ2LFjB8aPH4/c3FyvFk1E/mFTYbnTvvqGrCkyVRM43AZ7SUkJ4uLiEB4ejpCQEBiNRphMJmn+u+++Q0hICOLj4wEA9913H+bPn++9ionIL1RWn3Z6E9KN10Tw4V4+4PZ2R4vFAq1WK411Oh3KyuxvLvjpp58wfPhwLF++HBUVFYiOjsaqVat6VERExJAene8tWm2Y3CUoBnthx17Y9aQX96xz3Fcf/ouBeOo+9azWlXxduA12m80GjUYjjYUQDuPOzk7s27cPb7/9NmJjY7FhwwasW7cO69at63YR9fVNsF38sSky0GrDYLU2ylqDUrAXduyFXU96kf3i/zkde/aBKarppdzXRVCQ5rILYrdbMXq9Hlar/X+nrFYrdDqdNNZqtYiKikJsbCwAICUlxWFFT0SB5Ym/7sGZlvMOx15fyhdLfcltsE+ePBmlpaVoaGhAa2sriouLpf10AJg4cSIaGhrw/fffAwB27dqF667jx1kRBSLT3mOwnml3OBYTdYVM1QQut1sxkZGRyM7ORkZGBjo6OjB79mwYDAZkZmYiKysLsbGxePnll7Fy5Uq0trZCr9fj2Wef9UXtRKQwBZ9WOYzDBgfhj/NukqmawKURQsi7uQ3usSsNe2HHXti560XXF0uDALym0i0Yua+LPu+xExG50/WJjYB6Q90fMNiJqE9e//A7nKxvczh2R8JomaohgMFORH20p9zsMI4cGoypk6JkqoYABjsR9UHXffUBQcDaxb+RqRq6gMFORL3S9YmNALDpce6rKwGDnYh6bPXmUocnNgLA8gU3ylMMOWGwE1GPVFafxnFrq8OxmKgrMGZkuEwVUVcMdiLqkaffOuh0jG9CUhYGOxF1W9cXSwE+B0aJGOxE1C0LnvzQ6RhDXZkY7ETklmnvMZxu7nQ4duM1ETJVQ+4w2InIra4P9wKAh2bdIEMl1B0MdiK6LO6r+x8GOxFdEkPdPzHYicil1ZtLnY4ZoofKUAn1FIOdiFzq+iYkAHjkjokyVEI9xWAnIieutmCK1qfJUAn1BoOdiBxwX93/MdiJSPLg/ziH+uLUGBkqob5gsBMRAKCsqg6t7Y7HNAAmxehlqYd6j8FORACADe+WOR3bzC0Yv8RgJyLuq6sMg50owDHU1YfBThTA1r39pdOxK0L6yVAJeRKDnSiAHT7R6HQsJ+tWGSohT2KwEwUobsGoV7eCvaioCMnJyUhMTEReXt4lz9u9ezduu40XBpHSMdTVrb+7E8xmM3JycvD+++9j4MCBmDt3LiZNmoQxY8Y4nFdXV4dnnnnGa4USkWc8lOMc6ny4l7q4XbGXlJQgLi4O4eHhCAkJgdFohMlkcjpv5cqVeOihh7xSJBF5Tss552N8uJe6uF2xWywWaLVaaazT6VBW5vhGhjfffBMxMTG44YbefaJKRMSQXn2dp2m1YXKXoBjshZ2aejH9sUKnYz15uJeaetFXSu6F22C32WzQaDTSWAjhMD58+DCKi4uxZcsW1NbW9qqI+vom2GyiV1/rKVptGKxW5zsEAhF7YaemXlxqX727v5+aetFXcvciKEhz2QWx260YvV4Pq9Uqja1WK3Q6nTQ2mUywWq2YNWsWFi1aBIvFgrvuuquPZRORJ7kK9eULbpShEvIFt8E+efJklJaWoqGhAa2trSguLkZ8fLw0n5WVhZ07d6KwsBC5ubnQ6XTIz8/3atFE1H0Fu464PD5mZLiPKyFfcRvskZGRyM7ORkZGBmbMmIGUlBQYDAZkZmaivLzcFzUSUR+Y9h13OsZbG9VNI4SQd3Mb3GNXGvbCzt974cn71f29F54kdy/6vMdORP6Jb0IKXAx2IhW67znnUOe/7IGDf9dEKtR+3vnYa1ytBwwGO5HKcAuGGOxEKsJQJ4DBTqQarkJ9aKjbN5eTCjHYiVRs/cPx7k8i1WGwE6kAt2DoYgx2Ij/HUKeuGOxEfsxVqN+RMFqGSkhJGOxEfmrN3/a6PD51UpSPKyGlYbAT+amqmmanY9yCIYDBTuSXuK9Ol8NgJ/IzDHVyh8FO5EdchTpRVwx2Ij/H1Tp1xWAn8hPcgqHuYrAT+QGGOvUEg51I4VyFekzUFTJUQv6CwU7kh/447ya5SyAFY7ATKRi3YKg3GOxECsVQp95isBMpkKtQf2SOQYZKyB8x2IkUZuEl3oRkGD3cx5WQv2KwEymMcHGMWzDUEwx2IgXhvjp5QreCvaioCMnJyUhMTEReXp7T/D//+U+kpaUhNTUVDzzwAM6cOePxQonUjqFOnuI22M1mM3JycpCfn4/t27dj69atqKyslOabmprwpz/9Cbm5udixYwfGjh2Ll156yatFE6kNH+5FnuQ22EtKShAXF4fw8HCEhITAaDTCZDJJ8x0dHVi9ejUiIyMBAGPHjkVNTY33KiYKEFytU2/1d3eCxWKBVquVxjqdDmVlZdJ46NChuP322wEAbW1tyM3NxYIFC3pURETEkB6d7y1abZjcJSgGe2Hn7V5Mf6zQ6VjR+jSv/sze4nVhp+ReuA12m80GjUYjjYUQDuMLGhsb8eCDD2LcuHGYOXNmj4qor2+CzebqXgDf0WrDYLU2ylqDUrAXdt7uxaX21ZXYf14XdnL3IihIc9kFsdutGL1eD6vVKo2tVit0Op3DORaLBXfddRfGjh2LNWvW9KFcosDhKtQHDZChEFIdt8E+efJklJaWoqGhAa2trSguLkZ8fLw0f/78edx3331ISkrCihUrXK7miah7XnmM++rUd263YiIjI5GdnY2MjAx0dHRg9uzZMBgMyMzMRFZWFmpra3Ho0CGcP38eO3fuBABcf/31XLkTXQZvbSRv0ggh5N3cBvfYlYa9sPNGL/w11Hld2Mndiz7vsROR57gK9ReypshQCakZg53IRy71JqSwkIE+roTUjsFOJCN/2IIh/8NgJ/IBf91XJ//EYCfyMoY6+RqDnciL+HAvkgODncjHuFonb2OwE3kJt2BILgx2Ii9gqJOcGOxEHuYq1EOCZSiEAhaDnciDKqtPuzy+MZurdfIdBjuRBz391kGnY9yCIV9jsBN5CPfVSSkY7EQewFAnJWGwE/UR34RESsNgJ/ICrtZJTgx2oj7gFgwpEYOdqJcY6qRUDHaiXuC+OikZg53IQ7haJ6VgsBP1ELdgSOkY7EQ9wFAnf8BgJ+omV6E+mJ9DTQrEYCfqhg0FX7k8/vKjXK2T8jDYibqh7Ogpp2PcgiGl6lawFxUVITk5GYmJicjLy3Oar6ioQHp6OoxGI1asWIHOzk6PF0okF+6rk79xG+xmsxk5OTnIz8/H9u3bsXXrVlRWVjqcs2TJEjz55JPYuXMnhBAoKCjwWsFEvjT9sUKnYwx1Ujq3wV5SUoK4uDiEh4cjJCQERqMRJpNJmq+urkZbWxsmTJgAAEhPT3eYJ/JXfBMS+av+7k6wWCzQarXSWKfToays7JLzWq0WZrPZw2W61tjSjk8OnEBjSzvMDa2Yn3gtRkSESnN7ymswJXYEwkIGuvy6cx3nETygH/7rpl9C22UOACbFROLryjpMGDMcew/9/DtdHz0MWz7+HifrWtC/nwbCJmATwPDwQQgK0qDhbBtm/jYae8prcLKuxanm4AFAx3kAArAJr7WGvISrdfIHboPdZrNBo9FIYyGEw9jdfHdERAzp0fkXfPbpEez4/EdpvG3Pv7D63v+Q5t79tApDQoORnnDNZb8uYmgIoqMioNWGOcydbGjB/goL/lXbiP0VFgDAwSNWKbA7z9uT2Xq6Tfrn93ZXXTK0z3X06lclBShanyZ3CbLTasPkLkExlNwLt8Gu1+uxf/9+aWy1WqHT6RzmrVarNK6rq3OY7476+ibYerF8nRA9DKm/GSWt2GdOuRpWa6M0NydhNCZED5OOdf26Cyv2CdHD/v27NUpzwM8r9qv1YZgwZjiuHBYCgCv2QPX60tucrqNAo9WGBXwPLpC7F0FBmssuiDVCiMvGi9lsxrx58/Dee+9h8ODBmDt3Lv7yl7/AYDBI56SkpOCpp57CTTfdhFWrViEqKgr33ntvt4vsbbB7ktx/UUrCXtixF3bshZ3cvXAX7G5fPI2MjER2djYyMjIwY8YMpKSkwGAwIDMzE+Xl5QCA559/HmvXrsXUqVPR0tKCjIwMz/0GRETUI25X7L7AFbuysBd27IUde2Endy/6vGInIiL/wmAnIlIZBjsRkcow2ImIVMbtfey+EBTUszc0eYtS6lAC9sKOvbBjL+zk7IW7n62Iu2KIiMhzuBVDRKQyDHYiIpVhsBMRqQyDnYhIZRjsREQqw2AnIlIZBjsRkcow2ImIVIbBTkSkMgEf7Bs2bMBLL73kcq69vR1LlixBUlISZs6ciaqqKh9X5xsnT57E/PnzMXXqVNx///1obm52Oqe6uhoTJ05EWloa0tLSsHDhQhkq9Z6ioiIkJycjMTEReXl5TvMVFRVIT0+H0WjEihUr0NnZKUOVvuGuFxs3bkRCQoJ0Lbg6R02ampqQkpKCEydOOM0p9roQAers2bNi2bJlwmAwiBdffNHlOa+99ppYtWqVEEKIffv2iTlz5viyRJ9ZtGiR+OCDD4QQQmzcuFE8++yzTueYTCapF2pTW1srEhISxKlTp0Rzc7OYPn26OHLkiMM506ZNE1999ZUQQohly5aJvLw8OUr1uu70YvHixeLgwYMyVehbX3/9tUhJSRHXXXedOH78uNO8Uq+LgF2xf/LJJxg1ahR+//vfX/Kc3bt3IzU1FQBwyy23oKGhASdPnvRViT7R0dGBL7/8EkajEQCQnp4Ok8nkdF55eTkOHz6MtLQ0ZGRk4IcffvB1qV5TUlKCuLg4hIeHIyQkBEaj0aEH1dXVaGtrw4QJEwBcukdq4K4XAPDtt99i06ZNmD59Ov785z/j3LlzMlXrfQUFBVi9ejV0Op3TnJKvi4AN9hkzZmDRokXo16/fJc+xWCzQarXSWKvVora21hfl+cypU6cwZMgQ9O//84M+tVotzGaz03nBwcFITU3Ftm3bsHDhQjz44INob2/3dble0fXvWafTOfTA1XXgqkdq4K4Xzc3NGD9+PJYsWYJt27bh7NmzeOWVV+Qo1SfWrFmDm2++2eWckq8LRTy215s+/vhjrF271uFYdHQ0tmzZ4vZrhRDQaDQO46Ag//1voateREVFOfyOAJzGAPDwww9L/3zrrbdi/fr1OHr0KMaNG+edYn3IZrM5/T1fPHY3rybuftfQ0FC8+uqr0viee+7B8uXLkZ2d7dM6lUDJ14Xqgz0pKQlJSUm9+trIyEhYLBb86le/AgDU1dW5/F8yf+GqFx0dHZg0aRLOnz+Pfv36wWq1uvwd33rrLaSkpGDo0KEAfr6IL6zy/Z1er8f+/fulcdce6PV6WK1Waezv18HluOvFyZMnUVJSgtmzZwNQ13XQU0q+Lvx3+ekDt956KwoLCwEA+/fvR3BwMK688kqZq/KsAQMG4Oabb8ZHH30EANi+fTvi4+Odzvvyyy/x3nvvAQD27dsHm82G6Ohon9bqLZMnT0ZpaSkaGhrQ2tqK4uJihx6MHDkSwcHBOHDgAACgsLDQZY/UwF0vBg0ahOeeew7Hjx+HEAJ5eXm4/fbbZaxYPoq+LmR72VYhXnzxRYe7YvLz88WGDRuEEEK0tbWJxx9/XCQnJ4sZM2aIb7/9Vq4yverEiRPi7rvvFklJSeKee+4Rp0+fFkI49qK2tlb87ne/E9OmTRPp6emioqJCzpI9bseOHWLatGkiMTFR5ObmCiGEuPfee0VZWZkQQoiKigoxa9YsYTQaxaOPPirOnTsnZ7le5a4XJpNJml+6dKmqe3FBQkKCdFeMP1wX/AQlIiKV4VYMEZHKMNiJiFSGwU5EpDIMdiIilWGwExGpDIOdiEhlGOxERCrDYCciUpn/B2jFkIAeF+iCAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(net[3].input.reshape(-1), net[3].output.reshape(-1), s=1);" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAD7CAYAAACFfIhNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAYr0lEQVR4nO3df0yV993/8dfBg2x84b6t7hwxhJCsa2yGqzYjWbsth7jv5GDhyMQ2U5h06Rat7WzjGjsKROL6tSWWatNUyEyaNIs2k/U7OdPgwW6LNg1N6shagmFtl4ptpYWDmgrMw87hfO4/Gj93KZRzDqKHQ5+PxOD1uX70/e6VXq9zfa7DVYcxxggAAElpyS4AADB3EAoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIDlTHYB1+vy5VFFozP7VYslS7J08eLILFeUXPOxJ4m+Ug19zV1paQ7dcsv/+dL1KR8K0aiZcShc23++mY89SfSVaugrNTF9BACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMBK+d9TuB7/CY/L5cqeNB4ai2j4ytUkVAQAyfWVDoWF6Qvke8w/afzYs+UaTkI9AJBsTB8BACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIAVVyi88MILKi0tVWlpqfbu3StJeuKJJ1RcXKzy8nKVl5fr1VdflSR1dnbK5/OpuLhY+/fvt8fo7e1VRUWFvF6v6urqFIlEJEn9/f2qqqpSSUmJtm3bptHR0dnuEQAQp5ih0NnZqddff11Hjx5VW1ubzp49q1dffVU9PT06dOiQ/H6//H6/1qxZo1AopNraWjU3N6u9vV09PT06ffq0JGnnzp3atWuXOjo6ZIxRa2urJGn37t2qrKxUIBDQihUr1NzcfGM7BgB8qZih4HK5VFNTo4ULFyo9PV233nqr+vv71d/fr9raWvl8Pj3//POKRqPq7u5Wfn6+8vLy5HQ65fP5FAgEdOHCBYVCIa1atUqSVFFRoUAgoHA4rDNnzsjr9U4YBwAkR8x3H91222327319fTpx4oQOHz6sN998Uw0NDcrOztbWrVv1yiuvKDMzUy6Xy27vdrs1MDCgwcHBCeMul0sDAwO6fPmysrKy5HQ6J4wDAJIj7hfivffee9q6dasef/xxffOb39SBAwfsus2bN6utrU1er1cOh8OOG2PkcDgUjUanHL/28/O+uBzLkiVZCW0fr6nenpoqUrn26dBXaqGv1BRXKHR1demRRx5RbW2tSktL9c4776ivr89O+xhj5HQ6lZOTo2AwaPcLBoNyu92TxoeGhuR2u7V48WINDw9rfHxcCxYssNsn4uLFEUWjJqF9rpnu5AaDqfmeVJcrO2Vrnw59pRb6mrvS0hzTfpiO+Uzh448/1sMPP6ympiaVlpZK+iwEnnrqKX366acKh8M6cuSI1qxZo5UrV+rcuXM6f/68xsfHdfz4cXk8HuXm5iojI0NdXV2SJL/fL4/Ho/T0dBUWFqq9vV2S1NbWJo/HMxt9AwBmIOadwosvvqixsTE1NjbasY0bN2rLli3atGmTIpGIiouLVVZWJklqbGzU9u3bNTY2pqKiIpWUlEiSmpqaVF9fr5GRERUUFKi6ulqS1NDQoJqaGrW0tGjZsmXat2/fjegTABAHhzFmZnMvc8T1Th992f9kJ1VvEefD7e1U6Cu10Nfcdd3TRwCArw5CAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAABWXKHwwgsvqLS0VKWlpdq7d68kqbOzUz6fT8XFxdq/f7/dtre3VxUVFfJ6vaqrq1MkEpEk9ff3q6qqSiUlJdq2bZtGR0clSVeuXNGWLVu0du1aVVVVKRgMznaPAIA4xQyFzs5Ovf766zp69Kja2tp09uxZHT9+XLW1tWpublZ7e7t6enp0+vRpSdLOnTu1a9cudXR0yBij1tZWSdLu3btVWVmpQCCgFStWqLm5WZL03HPPqbCwUCdOnNB9992nPXv23MB2AQDTiRkKLpdLNTU1WrhwodLT03Xrrbeqr69P+fn5ysvLk9PplM/nUyAQ0IULFxQKhbRq1SpJUkVFhQKBgMLhsM6cOSOv1zthXJJOnToln88nSSorK9Nrr72mcDh8o/oFAEzDGWuD2267zf69r69PJ06c0M9+9jO5XC477na7NTAwoMHBwQnjLpdLAwMDunz5srKysuR0OieMS5qwj9PpVFZWli5duqSlS5fG1cCSJVlxbZcolyv7hhz3Zkjl2qdDX6mFvlJTzFC45r333tPWrVv1+OOPa8GCBerr67PrjDFyOByKRqNyOByTxq/9/LwvLn9+n7S0+J9/X7w4omjUxL395013coPB4RkdM9lcruyUrX069JVa6GvuSktzTPthOq6rb1dXl37+85/rscce0/r165WTkzPhgXAwGJTb7Z40PjQ0JLfbrcWLF2t4eFjj4+MTtpc+u8sYGhqSJEUiEY2OjmrRokWJdwoAuG4xQ+Hjjz/Www8/rKamJpWWlkqSVq5cqXPnzun8+fMaHx/X8ePH5fF4lJubq4yMDHV1dUmS/H6/PB6P0tPTVVhYqPb2dklSW1ubPB6PJKmoqEhtbW2SpPb2dhUWFio9Pf2GNAsAmF7M6aMXX3xRY2NjamxstGMbN25UY2Ojtm/frrGxMRUVFamkpESS1NTUpPr6eo2MjKigoEDV1dWSpIaGBtXU1KilpUXLli3Tvn37JEmPPvqoampqVFpaquzsbDU1Nd2IPgEAcXAYY2Y2IT9HXO8zBd9j/knjx54tT9l5w/kw5zkV+kot9DV3zcozBQDAVwOhAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAAAr7lAYGRlRWVmZPvroI0nSE088oeLiYpWXl6u8vFyvvvqqJKmzs1M+n0/FxcXav3+/3b+3t1cVFRXyer2qq6tTJBKRJPX396uqqkolJSXatm2bRkdHZ7M/AEAC4gqFt99+W5s2bVJfX58d6+np0aFDh+T3++X3+7VmzRqFQiHV1taqublZ7e3t6unp0enTpyVJO3fu1K5du9TR0SFjjFpbWyVJu3fvVmVlpQKBgFasWKHm5ubZ7xIAEJe4QqG1tVUNDQ1yu92SpKtXr6q/v1+1tbXy+Xx6/vnnFY1G1d3drfz8fOXl5cnpdMrn8ykQCOjChQsKhUJatWqVJKmiokKBQEDhcFhnzpyR1+udMA4ASA5nPBvt2bNnwvLQ0JDuuusuNTQ0KDs7W1u3btUrr7yizMxMuVwuu53b7dbAwIAGBwcnjLtcLg0MDOjy5cvKysqS0+mcMJ6IJUuyEto+Xi5X9g057s2QyrVPh75SC32lprhC4Yvy8vJ04MABu7x582a1tbXJ6/XK4XDYcWOMHA6HotHolOPXfn7eF5djuXhxRNGomUkb057cYHB4RsdMNpcrO2Vrnw59pRb6mrvS0hzTfpie0beP3nnnHXV0dNhlY4ycTqdycnIUDAbteDAYlNvtnjQ+NDQkt9utxYsXa3h4WOPj4xO2BwAkx4xCwRijp556Sp9++qnC4bCOHDmiNWvWaOXKlTp37pzOnz+v8fFxHT9+XB6PR7m5ucrIyFBXV5ckye/3y+PxKD09XYWFhWpvb5cktbW1yePxzF53AICEzGj66Pbbb9eWLVu0adMmRSIRFRcXq6ysTJLU2Nio7du3a2xsTEVFRSopKZEkNTU1qb6+XiMjIyooKFB1dbUkqaGhQTU1NWppadGyZcu0b9++WWoNAJAohzFmZhPyc8T1PlPwPeafNH7s2fKUnTecD3OeU6Gv1EJfc9cNeaYAAJifCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAAArrlAYGRlRWVmZPvroI0lSZ2enfD6fiouLtX//frtdb2+vKioq5PV6VVdXp0gkIknq7+9XVVWVSkpKtG3bNo2OjkqSrly5oi1btmjt2rWqqqpSMBic7f4AAAmIGQpvv/22Nm3apL6+PklSKBRSbW2tmpub1d7erp6eHp0+fVqStHPnTu3atUsdHR0yxqi1tVWStHv3blVWVioQCGjFihVqbm6WJD333HMqLCzUiRMndN9992nPnj03qE0AQDxihkJra6saGhrkdrslSd3d3crPz1deXp6cTqd8Pp8CgYAuXLigUCikVatWSZIqKioUCAQUDod15swZeb3eCeOSdOrUKfl8PklSWVmZXnvtNYXD4RvSKAAgNmesDb746X1wcFAul8suu91uDQwMTBp3uVwaGBjQ5cuXlZWVJafTOWH8i8dyOp3KysrSpUuXtHTp0uvvDACQsJih8EXRaFQOh8MuG2PkcDi+dPzaz8/74vLn90lLS+zZ95IlWQltHy+XK/uGHPdmSOXap0NfqYW+UlPCoZCTkzPhgXAwGJTb7Z40PjQ0JLfbrcWLF2t4eFjj4+NasGCB3V767C5jaGhIOTk5ikQiGh0d1aJFixKq5+LFEUWjJtE2JE1/coPB4RkdM9lcruyUrX069JVa6GvuSktzTPthOuGvpK5cuVLnzp3T+fPnNT4+ruPHj8vj8Sg3N1cZGRnq6uqSJPn9fnk8HqWnp6uwsFDt7e2SpLa2Nnk8HklSUVGR2traJEnt7e0qLCxUenp6wk0CAGZHwncKGRkZamxs1Pbt2zU2NqaioiKVlJRIkpqamlRfX6+RkREVFBSourpaktTQ0KCamhq1tLRo2bJl2rdvnyTp0UcfVU1NjUpLS5Wdna2mpqZZbA0AkCiHMWZmcy9zxPVOH/ke808aP/ZsecreIs6H29up0Fdqoa+5a9anjwAA8xehAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsAgFAIBFKAAALEIBAGARCgAAi1AAAFiEAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAADLeT07b968WZcuXZLT+dlhfvvb3+qDDz5QS0uLIpGI7r//flVVVUmSOjs79fTTT2tsbExr167Vjh07JEm9vb2qq6vT6OioCgsLtXv3bns8AMDNNeM7BWOM+vr65Pf77Z+cnBzt379fL7/8stra2nTkyBH961//UigUUm1trZqbm9Xe3q6enh6dPn1akrRz507t2rVLHR0dMsaotbV11poDACRmxqHw/vvvS5IeeOABrVu3TocOHVJnZ6fuuusuLVq0SJmZmfJ6vQoEAuru7lZ+fr7y8vLkdDrl8/kUCAR04cIFhUIhrVq1SpJUUVGhQCAwO50BABI241C4cuWK7r77bh04cEAvvfSS/vCHP6i/v18ul8tu43a7NTAwoMHBwbjGXS6XBgYGZloSAOA6zXjy/s4779Sdd95pl++99149/fTT2rZtmx0zxsjhcCgajcrhcMQ9noglS7Jm2sK0XK7sG3LcmyGVa58OfaUW+kpNMw6Fv//97wqHw7r77rslfXZBz83NVTAYtNsEg0G53W7l5OTENT40NCS3251QHRcvjigaNTPqYbqTGwwOz+iYyeZyZads7dOhr9RCX3NXWppj2g/TM54+Gh4e1t69ezU2NqaRkREdPXpUzzzzjN544w1dunRJV69e1cmTJ+XxeLRy5UqdO3dO58+f1/j4uI4fPy6Px6Pc3FxlZGSoq6tLkuT3++XxeGZaEgDgOs34TmH16tV6++239ZOf/ETRaFSVlZX67ne/qx07dqi6ulrhcFj33nuv7rjjDklSY2Ojtm/frrGxMRUVFamkpESS1NTUpPr6eo2MjKigoEDV1dWz0xkAIGEOY8zM5l7miOudPvI95p80fuzZ8pS9RZwPt7dToa/UQl9z1w2bPgIAzD+EAgDAIhQAABahAACwCAUAgEUoAAAsQgEAYBEKAACLUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBgEQoAAItQAABYhAIAwCIUAAAWoQAAsJzJLmA+yP6vr+trGZP/VYbGIhq+cjUJFQHAzBAKU/hPeFwuV/ak8bH/jCtj4YIp9/E95p809v8by6Y8DmEBYK4iFKawMH3BlBf5Y8+Wf+l4Isf5srD4stAhRADcLIRCEiQaOtxxALhZCIUUkOgdx3/9d+aUdxzTTX8RMAAkQiGlzWSaa6pxibsRAJ8hFCCJ5x8APkMoYFqz9fyDEAFSA6GAWcVDdCC1EQpIqkSmrVyubMICuMHmRCgcO3ZMLS0tikQiuv/++1VVVZXskpBks/WMg2krIDFJD4WBgQHt379ff/rTn7Rw4UJt3LhR3/ve9/Stb30r2aVhDpqtb1wlGi7TrSNgMJ8kPRQ6Ozt11113adGiRZIkr9erQCCgX/3qV3Htn5bmuK5/vvuWrzM+R2u6keML0xfoF//v5KTxF+uLpxyfbl3Lb/7vdd29XNs30budsbGIRkZCU9Y6F1zvf5tzVar3Fat+hzHG3KRapvS73/1O//73v7Vjxw5J0h//+Ed1d3frySefTGZZAPCVlPRXZ0ejUTkc/5tcxpgJywCAmyfpoZCTk6NgMGiXg8Gg3G53EisCgK+upIfC97//fb3xxhu6dOmSrl69qpMnT8rj8SS7LAD4Skr6g+alS5dqx44dqq6uVjgc1r333qs77rgj2WUBwFdS0h80AwDmjqRPHwEA5g5CAQBgEQoAAItQAABY8z4Ujh07pnvuuUfFxcU6fPjwpPW9vb2qqKiQ1+tVXV2dIpFIEqpMXKy+/vKXv6i8vFzr1q3TQw89pE8//TQJVSYuVl/XnDp1Sj/60Y9uYmXXJ1Zf77//vjZv3qx169bpF7/4xbw5X2fPntWGDRu0bt06bd26VVeuXElClYkbGRlRWVmZPvroo0nrUvWaETczj33yySdm9erV5vLly2Z0dNT4fD7z3nvvTdimtLTU/OMf/zDGGPPEE0+Yw4cPJ6PUhMTqa3h42PzgBz8wn3zyiTHGmOeee848+eSTySo3bvGcL2OMCQaDpqSkxKxevToJVSYuVl/RaNQUFxeb06dPG2OMeeaZZ8zevXuTVW7c4jlfmzZtMqdOnTLGGPP000+bffv2JaPUhLz11lumrKzMFBQUmA8//HDS+lS8ZiRiXt8pfP5le5mZmfZle9dcuHBBoVBIq1atkiRVVFRMWD9XxeorHA6roaFBS5culSQtX75cH3/8cbLKjVusvq6pr6+P+4WJc0Gsvs6ePavMzEz7S5sPPvhgSrw+Pp7zFY1GNTo6Kkm6evWqvva1ryWj1IS0traqoaFhyjcrpOo1IxHzOhQGBwflcrnsstvt1sDAwJeud7lcE9bPVbH6uuWWW7RmzRpJUigU0sGDB/XjH//4pteZqFh9SdLvf/97ffvb39bKlStvdnkzFquvDz74QN/4xjdUW1ur9evXq6GhQZmZmckoNSHxnK+amhrV19frhz/8oTo7O7Vx48abXWbC9uzZo8LCwinXpeo1IxHzOhRivWwvVV/GF2/dw8PD2rJli26//XatX7/+ZpY4I7H6evfdd3Xy5Ek99NBDyShvxmL1FYlE9Oabb2rTpk06evSo8vLy1NjYmIxSExKrr1AopLq6Or300kt6/fXXVVlZqd/85jfJKHXWpOo1IxHzOhRivWzvi+uHhoZS4mV88bxEcHBwUJWVlVq+fLn27Nlzs0uckVh9BQIBBYNBbdiwQVu2bLE9znWx+nK5XMrPz9d3vvMdSVJZWZm6u7tvep2JitXXu+++q4yMDPvamp/+9Kd68803b3qdsylVrxmJmNehEOtle7m5ucrIyFBXV5ckye/3p8TL+GL1NT4+rgcffFBr165VXV1dynySidXXI488oo6ODvn9fh08eFBut1svv/xyEiuOT6y+7rzzTl26dEn//Oc/JUl/+9vfVFBQkKxy4xarr/z8fH3yySd6//33JUl//etfbfClqlS9ZiQkiQ+5b4o///nPprS01BQXF5uDBw8aY4z55S9/abq7u40xxvT29poNGzYYr9drfv3rX5uxsbFklhu36fo6efKkWb58uVm3bp39U1tbm+SK4xPrfF3z4Ycfpsy3j4yJ3ddbb71lNmzYYO655x7zwAMPmKGhoWSWG7dYfZ06dcr4fD5TVlZm7r//fvPBBx8ks9yErF692n77aD5cM+LFC/EAANa8nj4CACSGUAAAWIQCAMAiFAAAFqEAALAIBQCARSgAACxCAQBg/Q8c5QEkQHEfkwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[3].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The final linear layer's output is again roughly Gaussian, thanks to the central limit theorem:" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAX/klEQVR4nO3df0yV5/3/8dcRBLFi25lzijGEZG1TO7dqs6xKl0BcKmDx1MrMoqXSZJuyTnE1jUbhTJf5tRLnZ25GbdrF1VXdpnNVkejRbm4kDXZOsmhcbGeq2Cl6OGKnUOXI4dzfPxrPhvw4PzicHxfPR0LCuc594P32nPvlzXXuc902y7IsAQCMNCLRBQAAhg4hDwAGI+QBwGCEPAAYjJAHAIMR8gBgMEIeAAyWnugC7vfZZ58rEDDn1P1x48aora0j0WUMKdN7NL0/iR5T2YgRNj388AP93p90IR8IWEaFvCTj+umL6T2a3p9Ej6ZiugYADEbIA4DBCHkAMBghDwAGI+QBwGCEPAAYjJAHAIMl3XnyQDxkj83SqMzeL/9On1/tt+4koCJgaBDyGJZGZabL+frBXuOH/m+22hNQDzBUmK4BAIMR8gBgMEIeAAxGyAOAwQh5ADAYIQ8ABiPkAcBgYYX8L3/5Sz3//PMqLS3VO++8I0lqbGyU0+lUUVGRNm3aFNz23LlzKisrU3FxsWpqauT3+4emcgBASCFD/uTJk/rwww9VV1enP/7xj9q5c6c++ugjVVdXa9u2bTp8+LDOnj2rhoYGSdLy5cu1evVqHT16VJZlae/evUPeBACgbyFD/plnntG7776r9PR0tbW1qbu7W7du3VJeXp5yc3OVnp4up9Mpt9utK1euqLOzU1OmTJEklZWVye12D3kTQKzc7eqW3Z7d40v6YhkEIBWFtazByJEjtXnzZv36179WSUmJWltbZbfbg/c7HA55PJ5e43a7XR6PJ/ZVA0MkY2Qayx3AKGGvXbN06VItXLhQP/jBD9Tc3CybzRa8z7Is2Ww2BQKBPscjMW7cmIi2TwX3jgZNRo+pz/T+pOHR4/1Chvwnn3yiu3fv6sknn1RWVpaKiorkdruVlpYW3Mbr9crhcCgnJ0derzc4fv36dTkcjogKamvrMOqK6nZ7trxes48BU7HHaHb2VOsxEqn4HEbK1B5HjLANeHAcck7+8uXLcrlcunv3ru7evas///nPmjdvni5evKhLly6pu7tb9fX1Kigo0IQJE5SZmammpiZJ0sGDB1VQUBC7bgAAEQl5JF9YWKgzZ87oxRdfVFpamoqKilRaWqovfelLqqqqks/nU2FhoUpKSiRJGzdulMvlUkdHhyZNmqSKioohbwIA0Lew5uSrqqpUVVXVYyw/P191dXW9tp04caL27dsXm+oAAIPCJ14BwGCEPAAYjMv/wQj9XbPVd7dbmRlpfTwCGB4IeRhhoGu29jcODAdM1wCAwQh5ADAYIQ8ABiPkAcBghDwAGIyza5BS+jtVEkDf2FuQUgY6VRJAb0zXAIDBCHkAMBghDwAGY04eCMO9C3z3pdPnV/utO3GuCAgPIQ+Eob8LfEtc5BvJjekaADAYIQ8ABiPkAcBghDwAGIyQBwCDEfIAYDBCHgAMRsgDgMHC+jDUli1bdOTIEUlSYWGhVqxYoVWrVqmpqUlZWVmSpCVLlmjGjBlqbGzU+vXr5fP5NHPmTC1btmzoqgcADChkyDc2NuqDDz7Q/v37ZbPZ9P3vf1/vv/++zp49q127dsnhcAS37ezsVHV1tXbu3Knx48ersrJSDQ0NKiwsHNImAAB9CzldY7fbtXLlSmVkZGjkyJF69NFH1dLSopaWFlVXV8vpdGrz5s0KBAI6c+aM8vLylJubq/T0dDmdTrnd7nj0AQDoQ8gj+ccffzz4fXNzs44cOaLdu3fr5MmTWrNmjbKzs1VZWal9+/Zp9OjRstvtwe0dDoc8Hs/QVA4ACCnsBcrOnz+vyspKrVixQl/+8pe1devW4H0LFizQgQMHVFxcLJvNFhy3LKvH7XCMGzcmou1TQX+rF5pkOPQ4EBP6N6GHUIZDj/cLK+Sbmpq0dOlSVVdXq7S0VB9//LGam5tVXFws6YswT09PV05Ojrxeb/BxXq+3x5x9ONraOhQIWBE9JpnZ7dnyes1eozCePSbrTprqzzGv09Q1YoRtwIPjkHPyV69e1eLFi7Vx40aVlpZK+iLU33jjDd28eVNdXV3as2ePZsyYocmTJ+vixYu6dOmSuru7VV9fr4KCgth1AwCISMgj+e3bt8vn86m2tjY4Nm/ePC1atEjz58+X3+9XUVGRZs2aJUmqra1VVVWVfD6fCgsLVVJSMnTVAwAGFDLkXS6XXC5Xn/eVl5f3GsvPz1ddXd3gKwMADBqfeAUAgxHyAGAwQh4ADMaFvJGUssdmaVRmarw873Z193lqp+9utzIz0nqNd/r8ar91Jx6lAYQ8ktOozHQ5Xz/Ya/zQ/81OQDUDyxiZ1m+t/Y2bd7Y2khXTNQBgMEIeAAxGyAOAwQh5ADAYIQ8ABiPkAcBghDwAGIyQBwCD8WEoJFQqfbIVSEXsXUioVPpkK5CKmK4BAIMR8gBgMEIeAAxGyAOAwQh5ADAYIQ8ABiPkAcBghDwAGCyskN+yZYtKS0tVWlqqDRs2SJIaGxvldDpVVFSkTZs2Bbc9d+6cysrKVFxcrJqaGvn9/qGpHAAQUsiQb2xs1AcffKD9+/frwIED+uc//6n6+npVV1dr27ZtOnz4sM6ePauGhgZJ0vLly7V69WodPXpUlmVp7969Q94EAKBvIUPebrdr5cqVysjI0MiRI/Xoo4+qublZeXl5ys3NVXp6upxOp9xut65cuaLOzk5NmTJFklRWVia32z3kTQAmyB6bJbs9u9dX9tisRJeGFBZy7ZrHH388+H1zc7OOHDmil19+WXa7PTjucDjk8XjU2traY9xut8vj8cS4ZCC13e3qlt2e3ed9/a3j0z7URcFYYS9Qdv78eVVWVmrFihVKS0tTc3Nz8D7LsmSz2RQIBGSz2XqNR2LcuDERbZ8K+tuhTTIceoyVjJFpES/KFo9/3+HwHA6HHu8XVsg3NTVp6dKlqq6uVmlpqU6ePCmv1xu83+v1yuFwKCcnp8f49evX5XA4Iiqora1DgYAV0WOSmd2eLa/X7OOwwfQ4HHe6aAz1a4jXaeoaMcI24MFxyDn5q1evavHixdq4caNKS0slSZMnT9bFixd16dIldXd3q76+XgUFBZowYYIyMzPV1NQkSTp48KAKCgpi1AoAIFIhj+S3b98un8+n2tra4Ni8efNUW1urqqoq+Xw+FRYWqqSkRJK0ceNGuVwudXR0aNKkSaqoqBi66gEAAwoZ8i6XSy6Xq8/76urqeo1NnDhR+/btG3xlAIBB4xOvAGAwQh4ADEbIA4DBuJA34iJ7bJZGZfJyA+KNvQ5xMSozPeIPAAEYPKZrAMBghDwAGIyQBwCDEfIAYDBCHgAMxtk1iClOlQSSC3sjYopTJYHkwnQNABiMkAcAgxHyAGAwQh4ADEbIA4DBCHkAMBghDwAGI+QBwGCEPAAYjJAHAIMR8gBgMEIeAAwWdsh3dHRo1qxZunz5siRp1apVKioq0uzZszV79my9//77kqTGxkY5nU4VFRVp06ZNQ1M1ACAsYa1Cefr0ablcLjU3NwfHzp49q127dsnhcATHOjs7VV1drZ07d2r8+PGqrKxUQ0ODCgsLY144ACC0sI7k9+7dqzVr1gQD/c6dO2ppaVF1dbWcTqc2b96sQCCgM2fOKC8vT7m5uUpPT5fT6ZTb7R7SBoDhKntsluz27F5f2WOzEl0akkhYR/Lr1q3rcfv69euaNm2a1qxZo+zsbFVWVmrfvn0aPXq07HZ7cDuHwyGPxxNRQePGjYlo+1Rgt2cnuoQhNxx6TKT+/n37W7t/VBTPx3B4DodDj/eL6qIhubm52rp1a/D2ggULdODAARUXF8tmswXHLcvqcTscbW0dCgSsaMpKSnZ7trze9kSXMaT+t8fhuBPFQ1+voYH+rSN9zQ2316lJRoywDXhwHNXZNR9//LGOHj0avG1ZltLT05WTkyOv1xsc93q9PebsAQDxFVXIW5alN954Qzdv3lRXV5f27NmjGTNmaPLkybp48aIuXbqk7u5u1dfXq6CgINY1AwDCFNV0zcSJE7Vo0SLNnz9ffr9fRUVFmjVrliSptrZWVVVV8vl8KiwsVElJSUwLBgCEL6KQP378ePD78vJylZeX99omPz9fdXV1g68MADBofOIVAAwW1XQNgPi529XNWUuIGiEPJLmMkWn9ng8PhMJ0DQAYjJAHAIMR8gBgMEIeAAxGyAOAwQh5ADAYIQ8ABiPkAcBghDwAGIyQBwCDEfIAYDDWrkFUssdmaVTmf18+LKAFJCdCHlEZlZnOollACmC6BgAMRsgDgMEIeQAwGCEPAAYj5AHAYJxdAximv2vCdvr8ar91JwEVIZEIecAwA10Ttj0B9SCxwpqu6ejo0KxZs3T58mVJUmNjo5xOp4qKirRp06bgdufOnVNZWZmKi4tVU1Mjv98/NFUDAMISMuRPnz6t+fPnq7m5WZLU2dmp6upqbdu2TYcPH9bZs2fV0NAgSVq+fLlWr16to0ePyrIs7d27d0iLBwAMLGTI7927V2vWrJHD4ZAknTlzRnl5ecrNzVV6erqcTqfcbreuXLmizs5OTZkyRZJUVlYmt9s9tNUjJrLHZsluz+7zK3tsVqLLAzAIIefk161b1+N2a2ur7HZ78LbD4ZDH4+k1brfb5fF4Ylgqhkp/SxRIzOMCqS7iN14DgYBsNlvwtmVZstls/Y5Haty4MRE/Jtml8uJd/Z2pgdQ00HM5HJ7n4dDj/SIO+ZycHHm93uBtr9crh8PRa/z69evBKZ5ItLV1KBCwIn5csrLbs+X1Jvex8EAv/IHO1EDq6e+1mAqv08EytccRI2wDHhxH/GGoyZMn6+LFi7p06ZK6u7tVX1+vgoICTZgwQZmZmWpqapIkHTx4UAUFBdFXDiAu7r3vwvsxZor4SD4zM1O1tbWqqqqSz+dTYWGhSkpKJEkbN26Uy+VSR0eHJk2apIqKipgXDCA6A029cV69ucIO+ePHjwe/z8/PV11dXa9tJk6cqH379sWmMgAxxdTb8MTaNQBgMEIeAAxGyAOAwQh5ADAYIQ8ABiPkAcBgrCc/jGSPzdKoTJ5yYDhhjx9G+luIjPOkAXMxXQMABiPkAcBghDwAGIw5eQB96m9Bs06fX+237iSgIkSDkAfQp4EWNGN1ytTBdA0AGIyQBwCDMV0DICIDXXyE+frkQ8gDiEh/c/US8/XJiOkaADAYIQ8ABiPkAcBgzMkbiNUmAdxDEhiI1SaRKHxKNvkQ8gBihk/JJp9BhfyCBQt048YNpad/8WN++tOf6tNPP9Wbb74pv9+vV155ReXl5TEpFAAQuahD3rIsNTc36y9/+Usw5D0ej5YtW6b33ntPGRkZmjdvnqZOnarHHnssZgUDAMIXdchfuHBBkvTd735X//nPf/Sd73xHDzzwgKZNm6aHHnpIklRcXCy3260lS5bEploAQESiPoXy1q1bys/P19atW7Vjxw79/ve/V0tLi+x2e3Abh8Mhj8cTk0IBAJGL+kj+6aef1tNPPx28PXfuXK1fv16vvvpqcMyyLNlstoh+7rhxY6ItKWn1t84HMFz0d9bN3a5uZYxMi1sdw3FfjDrkT506pa6uLuXn50v6ItAnTJggr9cb3Mbr9crhcET0c9vaOhQIWNGWlXTs9mx5vfE9r2A4vpCR3AY66yZe+0ci9sV4GDHCNuDBcdTTNe3t7dqwYYN8Pp86Ojq0f/9+/exnP9OJEyd048YN3blzR8eOHVNBQUG0vwIAMEhRH8lPnz5dp0+f1osvvqhAIKCXXnpJX//617Vs2TJVVFSoq6tLc+fO1VNPPRXLegEAERjUefKvvfaaXnvttR5jTqdTTqdzUEUBAGKDBcoAwGCEPAAYjJAHAIMR8gBgMEIeAAxGyAOAwVhPPoVxBSikOi4yMvRIiBTGFaCQ6vpb7uCPtbMI/xgh5AEkHa4wFTvMyQOAwTiSTyL9zbHzJyqAaBHySWSgOXb+RAUQDUI+BfR3BgIAhELIp4CB3oQCgIEQ8gBSBufVR46QB5AyOLUycoQ8gJTHEX7/CHkAKY8j/P7xYSgAMBhH8gCMdf80zr3vh9M0DiEPwFhM4xDyQyp7bJYk9XpDaDgdRQBILEJ+CLFMAYBEI+QTgGUKgMTqbx/03e1WZkZar/FU/ut7SEL+0KFDevPNN+X3+/XKK6+ovLx8KH5NymKZAiCxBtoHTfvrO+Yh7/F4tGnTJr333nvKyMjQvHnzNHXqVD322GOx/lVJg8vwAWZL5Q9bxTyZGhsbNW3aND300EOSpOLiYrndbi1ZsiSsx48YYYt1STEzZswoZfYT5t/7f8d6jW13FcnxcFaf26fKeDLWlGzjyVgT/xaxHc8YmdbvPv55H5k1UFb0NyXk8/nV0dHZ52MGEiozbZZlWRH/1AG89dZbun37tpYtWyZJ+sMf/qAzZ85o7dq1sfw1AIAwxPwTr4FAQDbbf/9nsSyrx20AQPzEPORzcnLk9XqDt71erxwOR6x/DQAgDDEP+WeffVYnTpzQjRs3dOfOHR07dkwFBQWx/jUAgDDE/I3XRx55RMuWLVNFRYW6uro0d+5cPfXUU7H+NQCAMMT8jVcAQPJgqWEAMBghDwAGI+QBwGCEPAAYjJCPoZaWFpWXl6ukpESvvvqqPv/883637ejo0HPPPae//e1vcaxw8MLpsbW1Vd/73vc0e/ZszZkzRydOnEhApZE7dOiQnn/+eRUVFWn37t297j937pzKyspUXFysmpoa+f3+BFQ5OKF6/NOf/qTZs2frhRde0A9/+EPdvHkzAVUOTqge7/nrX/+qb33rW3GsLEEsxMyiRYus+vp6y7Isa8uWLdaGDRv63XbFihXWN77xDevDDz+MV3kxEU6Pr7/+urVr1y7Lsizrk08+sZ599lnL7/fHtc5IXbt2zZo+fbr12WefWZ9//rnldDqt8+fP99imtLTU+sc//mFZlmWtWrXK2r17dyJKjVqoHtvb261vfvOb1rVr1yzLsqxf/OIX1tq1axNVblTCeR4ty7K8Xq9VUlJiTZ8+PQFVxhdH8jHS1dWlv//97youLpYklZWVye1297nt4cOH9cADD+iJJ56IZ4mDFm6PM2bM0KxZsyRJeXl58vl8un37dlxrjdT/Lqw3evTo4MJ691y5ckWdnZ2aMmWKpIGf32QVqseuri6tWbNGjzzyiCTpiSee0NWrVxNVblRC9XiPy+UKe9HEVEfIx8hnn32mMWPGKD39i8+X2e12eTyeXtu1tLToN7/5jVasWBHvEgct3B6Li4v14IMPSpK2b9+uJ598UtnZyX2RlNbWVtnt9uBth8PRo7f77++v92QWqseHH35YM2bMkCR1dnbq7bff1nPPPRf3OgcjVI+S9O677+orX/mKJk+eHO/yEoJF0KNw5MgRrV+/vsdYXl5er4XY7r8dCARUU1OjH//4xxo1atSQ1zkY0fb4v3bs2KE9e/Zo165dQ1JjLIVaWM+EhffC7aG9vV2LFy/WxIkTNWfOnHiWOGihevzXv/6lY8eOaceOHbp27VoiSow7Qj4KM2fO1MyZM3uMdXV1aerUqeru7lZaWlqfC7NduHBBFy5cUE1NjSTp008/lcvl0tq1azVt2rS41R+OaHu8Z8OGDWpoaNDu3buVk5MTj5IHJScnR6dOnQrevr+3+xfeu379esotvBeqR+m/b5pPmzZN1dXV8S5x0EL16Ha75fV69e1vf1tdXV1qbW3VSy+9pN/+9reJKDc+EvuWgFkWLlxo1dXVWZZlWdu2bbN+8pOfDLj9yy+/nHJvvIbT4zvvvGOVlZVZN2/ejHd5Ubv3hl1bW5t1+/Zt64UXXrBOnz7dY5vS0lLr1KlTlmVZlsvlsn71q18lotSoherR7/dbc+bMsbZu3ZrAKgcnnOfxnn//+9/D4o1X1q6JoStXrmjlypVqa2vT+PHj9fOf/1wPPvigfve736m1tVU/+tGPemy/YMECLVmyRFOnTk1QxZEL1ePSpUv1zDPPaMyYMRo7dmzwcW+//XbwDb1kdejQIb311lvBhfUWLlyohQsXaunSpfra176mjz76SC6XSx0dHZo0aZLWr1+vjIyMRJcdkYF6vHbtmqqqqnqcEPDVr35V69atS2DFkQv1PN5z+fJlVVRU6Pjx4wmsdugR8gBgMM6uAQCDEfIAYDBCHgAMRsgDgMEIeQAwGCEPAAYj5AHAYIQ8ABjs/wMWXCEvHDDtoQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.hist(net[4].output.reshape(-1), bins=50);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So far we have only looked at distributions of the tensors involved in the forward pass, but there is also a lot to learn from the backwards gradient tensors that we do not have time to delve in to. For example, this scatter plot offers some insight into a suitable learning rate for the second `Linear` module's weight parameters:" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAD7CAYAAABwggP9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9e3Rc1Zng+6s69X6oSiVVqfSWLVkyki2EbcAxxhCgbXAIYdzQZHBnVq9J0z30XTc36/awhl7Q6WbWnc7MrKyk031v1uLe7mSt9IU7QHdjcMcY0kBijG0wdsA2smxL1vtRqvf7Xef+UapjPUq2XgY7s3//SKfOOft8+3G+79vf3mdvlSzLMgKBQCAQzEP9ZQsgEAgEghsTYSAEAoFAUBZhIAQCgUBQFmEgBAKBQFAWYSAEAoFAUBZhIAQCgUBQFmEgBAKBQFAWzZctwFoTDMYpFG7cTzuqqiz4/bEvW4xrIuRcW4Sca8/NIuuNLqdaraKy0lz23G+dgSgU5BvaQAA3vHwlhJxri5Bz7blZZL1Z5JyPCDEJBAKBoCzCQAgEAoGgLMJACAQCgaAswkAIBAKBoCzCQAgEAoGgLMJACAQCgaAswkAIBAKBoCzCQAgEAoGgLMJACAQCgaAswkAIBAKBoCzCQAgEAoGgLMJACAQCgaAswkAIBAKBoCzCQAgEAoGgLMJACAQCgaAswkAIBAKBoCzCQAgEAoGgLKsyEAcPHmTv3r3s3r2bl156acH58+fPs2/fPvbs2cNzzz1HLpcDYGJigv379/Pggw/y9NNPE4/HAejv7+eb3/wmjzzyCN/61rcYHx9fjXgCgUAgWAUrNhAej4cf/ehHvPzyyxw4cIBXXnmF/v7+Odc888wzfO973+Ptt99GlmVeffVVAF544QWefPJJDh8+zKZNm/jJT36i/P4nf/InvPnmm+zdu5cf/vCHq8iaQCAQCFbDig3EsWPH2L59O3a7HZPJxJ49ezh8+LByfnx8nFQqRU9PDwD79u3j8OHDZLNZTp48yZ49e+b8DvCzn/2MXbt2USgUmJiYoKKiYjV5EwgEAsEq0Kz0xunpaZxOp3Lscrk4c+bMouedTicej4dgMIjFYkGj0cz5HUCj0RCJRNi7dy+pVIp/+Id/WKl4AoFAIFglKzYQhUIBlUqlHMuyPOd4sfPzrwPmHFdUVHD06FGOHDnC008/zbvvvoskSUuWq6rKspLsfKE4ndYvW4QlIeRcW4Sca8/NIuvNIud8Vmwg3G43n3zyiXLs9XpxuVxzznu9XuXY5/PhcrlwOBxEo1Hy+TySJM2579ChQzz00EOoVCp27dpFKpUiHA7jcDiWLJffH6NQkFeareuO02nF641+2WJcEyHn2iLkXHtuFllvdDnVatWijvWKxyB27NjB8ePHCQQCJJNJ3nnnHXbt2qWcr6+vR6/Xc+rUKQDeeOMNdu3ahVarZdu2bRw6dAiAAwcOKPf99Kc/5Ze//CUAJ06coLKyclnGQSAQCARrh0qW5RW72wcPHuTFF18km83y2GOP8dRTT/HUU0/xne98h82bN9PX18fzzz9PLBajq6uL73//++h0OsbHx3n22Wfx+/3U1tbywx/+EJvNRn9/P3/+539OIpHAarXyF3/xF2zYsGFZMokexNog5FxbhJxrz80i640u59V6EKsyEDciwkCsDULOtUXIufbcLLLe6HJelxCTQCAQCH67EQZCIBAIBGURBkIgEAgEZREGQiAQCARlEQZCIBAIBGURBkIgEAgEZREGQiAQCARlEQZCIBAIBGURBkIgEAgEZREGQiAQCARlEQZCIBAIBGURBkIgEAgEZREGQiAQCARlEQZCIBAIBGURBkIgEAgEZREGQiAQCARlEQZCIBAIBGURBkIgEAgEZREGQiAQCARlWZWBOHjwIHv37mX37t289NJLC86fP3+effv2sWfPHp577jlyuRwAExMT7N+/nwcffJCnn36aeDwOwMDAAPv37+cb3/gGTzzxBOfPn1+NeAKBQCBYBSs2EB6Phx/96Ee8/PLLHDhwgFdeeYX+/v451zzzzDN873vf4+2330aWZV599VUAXnjhBZ588kkOHz7Mpk2b+MlPfgLA888/z1NPPcUbb7zBd7/7Xf7Tf/pPq8iaQCAQCFbDig3EsWPH2L59O3a7HZPJxJ49ezh8+LByfnx8nFQqRU9PDwD79u3j8OHDZLNZTp48yZ49e+b8DvD4449z9913A9DR0cHk5OSKMyYQCASC1aFZ6Y3T09M4nU7l2OVycebMmUXPO51OPB4PwWAQi8WCRqOZ8zsUjUWJv/mbv+GBBx5YtlxVVZZl3/NF43Rav2wRloSQc20Rcq49N4usN4uc81mxgSgUCqhUKuVYluU5x4udn38dsOC6//7f/zufffYZP//5z5ctl98fo1CQl33fF4XTacXrjX7ZYlwTIefaIuRce24WWW90OdVq1aKO9YpDTG63G6/Xqxx7vV5cLtei530+Hy6XC4fDQTQaJZ/PL7gvl8vxH//jf+Ts2bP8/Oc/x2q9Oa2uQCAQ/DawYgOxY8cOjh8/TiAQIJlM8s4777Br1y7lfH19PXq9nlOnTgHwxhtvsGvXLrRaLdu2bePQoUMAHDhwQLnvv/23/0YsFuOnP/2pMA4CgUDwJaOSZXnF8ZiDBw/y4osvks1meeyxx3jqqad46qmn+M53vsPmzZvp6+vj+eefJxaL0dXVxfe//310Oh3j4+M8++yz+P1+amtr+eEPf0g+n2fnzp00NDRgNBqVZ7zxxhvLkkmEmNYGIefaIuRce24WWW90Oa8WYlqVgbgREQZibRByri1CzrXnZpH1RpfzuoxBCAQCgeC3G2EgBAKBQFAWYSAEAoFAUBZhIAQCgUBQFmEgBAKBQFAWYSAEAoFAUBZhIAQCgUBQFmEgBAKBQFAWYSAEAoFAUBZhIAQCgUBQFmEgBAKBQFAWYSAEAoFAUBZhIAQCgUBQFmEgBAKBQFAWYSAEAoFgGUQTGd76aJhoIvNli3LdEQZCIBAIlsHRs5O89v4AR89OftmiXHc0X7YAAoFAcDOxc3PtnL9rTTSR4ejZSXZursVq0l2XZywV0YMQCASCZWA16XjozuYlK+9wLL2skNSN1ENZlYE4ePAge/fuZffu3bz00ksLzp8/f559+/axZ88ennvuOXK5HAATExPs37+fBx98kKeffpp4PD7nvtdee41nn312NaIJbhJu5njujS77auW70fN3s/DuyZFlKfydm2t55K4WIvEMBz64/KWW/4oNhMfj4Uc/+hEvv/wyBw4c4JVXXqG/v3/ONc888wzf+973ePvtt5FlmVdffRWAF154gSeffJLDhw+zadMmfvKTnwCQTqf5wQ9+wF/91V+tIkuCm4kbyVtaLl+k7CtR1quV72aum3J8WQbv/tubePyrrUsOSVlNOvQ6ibc/HuXND4e+1PJfsYE4duwY27dvx263YzKZ2LNnD4cPH1bOj4+Pk0ql6OnpAWDfvn0cPnyYbDbLyZMn2bNnz5zfAU6ePEmhUOCZZ55ZTZ4ENxE7N9cu6+W5kfgiZV+Jsl6tfF903VxvBf5lGTybRb+skBRAT1s1nS12Hryj8Ut9N1Y8SD09PY3T6VSOXS4XZ86cWfS80+nE4/EQDAaxWCxoNJo5vwPs3LmTnTt38s///M8rFYuqKsuK7/2icDqtX7YIS2KpcoZjad49OcL9tzdhs+iX9wxgfXPVCqSblcaXVJ7LlX01cn7j3g1YzPpllfFKy7Yk51rUzXL44P1LvPb+AINTUb77zS1z8rlYG1tOmZbK8I5ONx+cm1pRe10py637D85N0TsU4s5NdV9oHcxnxQaiUCigUqmUY1mW5xwvdn7+dcCC49Xg98coFOQ1S2+tcTqteL3RL1uMazJbzmgiw7unxgC4f2vDAk/orY+Gee39AWLxNA/d2fylyXkjsxZy3r3JTSaZwZu8fiGSxeT8ImbW9Kx30N3q4JPz07zxq0tz2lK5NraSMr17k3vJ7XU5eb7atSuRs2e9g8e/2krPesd1rw+1WrWoY71iA+F2u/nkk0+UY6/Xi8vlmnPe6/Uqxz6fD5fLhcPhIBqNks/nkSRpwX2CG4+jZyd588MhAPQ6acFLdb2n/QmuH0tVNKXwDDCn/tdSUVlNOr79tU4lvdkspY0tVZalttdSni+MBPn21zpXVD5LZb7spZlS1+t5S2XFYxA7duzg+PHjBAIBkskk77zzDrt27VLO19fXo9frOXXqFABvvPEGu3btQqvVsm3bNg4dOgTAgQMH5twnuPEozap45K6Wsi/V/Gl/YvbLzUNJ0fz9L3qZ9Mf55/cvla23xcYj1jquX24K6XKN2LVkWeo01Z2ba+ludXBmIHDNNFc7XnM12cu9T1/U+JD0l3/5l3+5khstFgsOh4Pnn3+el19+mUcffZS9e/fy1FNPsW7dOmpqatiyZQv/+T//Z376059SWVnJn/7pnyJJElu2bOHHP/4xf/d3f0c0GuUv/uIvMBgMStrnz59nfHycBx54YNlyJZMZ5Bs3woTZrCexxoozmsjw3ukx3A4Teq20JmnOllOvldjYXMnG5solpf/e6TFee38Aq0nLhgb7msizFDnh+pTFWnA96n0tcDtMTPhinBkIMOVP8MuTo+g0ajY2V865Tq+V2NBgX1CmbocJq0nLzs21c86tpB4Wu2ex9jS/TBeTZaXotRJd6xyLpjlbXqtJV7Z8ysk5P423TgwTjmUw6iV23960wHCVy/9i9bESVCoVpkWMpUqWb2R1unz+ZxyDKMVUH/9q65p1N1cj52Ie32rCEYuNg8yX83qUxVpwI4+VlOolEs/w9sejPHJXC4/evX5VaS5WD1drA8u9p1SmV0vzWm1uNW1yqW2tXN2X2nP/WJje4aDye7m0SjL2tFXzUW9xQk+5scCVcl3GIAQ3DjfaGMBi8dPVxE2vNQ5S4kYri9ksVRl9WUst7Lq1jjqXlZ71jlXLVCr/nrZq3vpoWLmvXBuYrQDTmTzpTJ5oIqM8Z6Xx+Ggiw4tvfE7vcJB0Js/9WxsW5GElbXK2vKW8LndA++9/0cuZgQAAnc2VNNVY0GklpdzKGYWPej3KOzA0FbnmuMhaIAzEl8xqlcGNtG7LtViN8t65uZZ0Jn/N+0vKpBS3vZHKZanK6IsagCz3vH/38KayPZ3lDlKX6qHkZZfum98GJv1x/vafzjAVSAJF4//a+wNXdQLms1i7Onp2co53/taJYQ5/PEo0nuH37tug3FPOKJXLXynNdCavKOqSjKV8pjN59Drpqu3u6NlJzgwEaG+sQKOW2L+7ndoqs/K8tz4aVp5xYSSoGJLSOGD/WFgZF7ne7WPFYxA3KjfbGMRq4/Wl+3UaNYNTkTWLvV+P2P5icdPZaWey+bLPWWwcZLH47nLL9XqPXZjNeqx6aUE8e/5zo4kMl8bCdDTZubenfsWyLCU/pWt62qqxGLXk8zJtjZXkc/my11TZDAti8fPLef5z548L6LUSFqOWn711nrHpGL/6zTjDnhhmg4bHv9pGW71tyWMaORn+5cPLNNVY6VpXVXZ8RJZl9BqJbRtdvP7BINlcAZ1Wzeb1Vbx3eoymGisT/jivHxks21Zm529kOsZr7w+wrtaKTqvGqNPQVGOdk898XlbScjtMvHVimDP9PnoHA9Q7zXOuNeu1HO/1YDFqlTGft04M8/qRQVpqrWxpd/LA1kZ0GjVqNVSYdOy+o4ltG11rOtZytTEIYSC+YNZ6YK1cw1yLgeG1NGTXUlblXsKlPmcxA7GcwdNSl//90xMLnrtWhsNs1pPL5hcYyJJCKA0Mv3d6jNePDNLdVkXXupV/IFWuvkoDon0jQeqdZqVXUGUzoJHUvH5kEJtFR7PLsiCdKpuBh+5sXmDA55dz6foJX2xmgPfK4G2pLN/9ZIze4RD94xFUKkik82RzBapsBrrWVbGhwU4mm58j67unxuaUE8CvPh3npXcuKs+aXz+ZbJ4jn01wbjDI0FSEYDSD2SDRvb6aEU9MeV+K3n75d9Bi1OILJ3lgayNuhwlfOInFpOPE59P0j0cUQ1DqZayvq1DSOnp2ktePDHJ+KMiF0ZBSF5lsnktjYS6NhfFH0rTUWhmdjuF2mBicjHBhNMSGeht6nURbvQ1PMMkHn00pzyuV0Vo5MlczECLEVIYvMmxzrfjqUu+PJjJK1/Z6cLXw0LXKq6SIFut+l0t7tflYzjhIqcvf3epYdBrn7OtXw2Jl1T8WJprIrKqcZ1MundnjOP1jYfbvblfCK3d21gBwR6d7TmhufjrXKo+dm2uVsEgpBFKSuxQ22XNHI6hkcnmZi6NhOlvstNXbF5VVryuvCO+/vYlT56cWDbfMrteHd7TwL8eGqLYZOfzxKA/e0ahMEy3l891TY6SzefRaiU3rHRw4MkguX+DiWJiOJh8AZwYCtLgreOSuFiW/s8tkfhtOZ/JotBK5bF4Zq5g9/uCyGwGU7y2emAl99Y+H6B0KKelE4hlGPTFl3OOLQvQgynA9pmmWvKeWOtucLvy1rl+q51oK3ywWolmOjG6HiUq7CY8vxnunx7AYtXxwZpJ8XmZ9XcWCdBcrr/nhiVIvp+TxlWSdHSJY7vS95U4fLdezKP32wNZGPu6bnlN2s69fTdkGYxn+z3/8lKlAgjc/HFLKqt5pZsIX49xg8Jre4XLCibPLsVQPGxpsnB8KkEjn8YZTc3oOpR7C8c+n5njl86dvlsqjp62ao2cnuTQWnlOnVpNOmRo6/5p1tVZMBg1f+0oLe+5o5ta2aqwmLft2tbK+roJ3T40pPQZHhYEpf4ItHU7u39qgeOalsFs0keH451Nsba8mHE/zwNZGpYxUKvj5233c2lpNIp3lifs20FxTwfYuN+eHgwyMR2ittymD1qUewOtHBhkYL3rwF0aCDHli+CNp2htsVFr1bOtwYTFqyWTzSJKa+7c2APD5YACdVs3u25v4uG9aeRe61lWxsbmS5jo7/3pymPZGOx/3TfP+6QlcdgPxVI54Kodeq8ZpM3JuMIjFqGVoKsK5wSDtjTb84SQbmyoZ8UQ53uthzBvj1rbqNQ2DihDTMlnr+dRw5eWe34Uvx9VCHlAc2Pv7X/TS7LYuac50Kc1rKbfZ997a7uLg0cu89v4AvnCSI59Nzukmz053sZj5/PBESRmeGQgsGk5armEsxaGXa0jnj29saLArnuBi883nh4OWI+v//eY5fnPRRzSR4f6tDWzrcHH07CSOCgPxZK5s+S0Wzy8XTpz0x3nxzXNcnohwfjhI/1hYiXmX6iEcTzM4FQOKnuujd6+jrd6GTqMmn5epd5rpanPSPxrgzEAAnUZN30hQUdp6rUQmm2dkOsbwVJTXjwzS0WRHr1PPuX5wMsK2DhevvHeJ909P0NFkp7utinQ2z7FzV2Lus8u2FFortbGpQIJffzrBlnan4jzMDj299dEQ734yhj+S5NxgkCqbQWlPpy96GfPGuTwZ5vJEFF84Sdc6B4FIitePXCaRzrFhJnTz2vsDjHgiqFUq9Fo1VqOWUDzDbe3VJFI5EukcVTYDx8558IWT2C16fnFihAujIZBljnw2wYfnPHhDKWWMZnZ5ZrJ5/uv/+wmXxiL4wkke3bkeq0nLo3evx2LUIknQOxTiK5vcdLdVEYqm+ah3muoKA5JaxaXxCO//Zpx0JkckkcUfSZf9TmU1CAOxTNbqI5TZL3hTjRWrSctDd62/Zg/ivdNjvH96gu5WB4/uXL9AjlIX1RdOsr3LPefcfONWkqHkxV2tV+R2mJTGXeMw89klL3qNxCM711Fp1S9qBBaLmZcboOxa51Cesa3DpbxUgUiKF988x8e9Ho58NrlAOZW8d4tRq3h9eq2kxKHL5Wt+zP1q9bmUweG+kWIsWVKr2NxatcCgLDa+8daJYYwGLeFYiulgiu62KkXBTvkTHD07Vbb85hv7kpEYnIywrra4+FspX3//i17ODQYZnIwqXnDpvtk9JItRC8iMTMewGLWsr6vgl5+McPTsFCOeCHff1sDGhqLR6B8Lc/TsFBdGQ1yeCHPrTI/gtfcHaKm1YjZouKW5kk/6vPRsqGLCm1Cu7x0KMDARpbvVwe/e00pTjZX3To3jDaeUmHupLi1GLcNTUdbVWmmtt5HO5kln83S1OLi3p37OeMTgZIQ3PxzCH0kD4LDquWtzLds6XAxPRZHUKib8CQDuvMWF1azlzECACV+Mzy75GfYUDWRrvQ0AmQJ9I2GGpqL4I2nqnCa8oRQWo4aNzZUMjEeoqjBgt2jpGwlTbdeTTOdJpHOEYmkGJ2O0N9qosukx64vlXRr0nvDFmAok+M1FH9U2Ay1uK+O+OPf21FNlM1LvNBOOZehosrNpvYPj56YYmY4RS2RJpHNoNGoS6eIeOslMHodFTyJddCbqneY1m1QhDMQyKb3o85XRcu4tdVtndzc3NNhxVJquGRIpvdCP7ix+rDS/IVTbDVwYCdJSW8G62oqy3nDpt5KSKXlx8w2H22FidDrKD/7Hb2hvtBOOZ3j9yCCeQJwPPpvEG07hrjLx9R3ryn5JXa63VUq73OwSvVZicCqihDVKsd/X3utn2BPHH0nT2VzJdCjBsXMeLoyGmPDFCEbTvH5kkHODfk6e9ype1IZmB7lsTvHYyhmvkrJUZpUM+Lg0y8OefW1HU/EFn1/n0USGwclI0eMbDi0Y3Mxk80qvrxQCshi1/OxQH0fPTnFpNMTO7lrMBg1f6XJz7OwU3nCKLR1OtrQ7lXDM/PAWskz/eIh1tRVYTTpFTpNBw5HPJhUj0Oy2MhWIs3l9Fe2NdrpaHEovpVQPVpOOeqeZ459PEYxmqKs2Me6Lc/TsFADeUIoLIwE6Gu18eHaSc4NBzAYN2VwBfyQ98ywbl8ZCVFn1nOid5tyAn0giSyiaZtyXwGU34q40MuqN47Ib+eYDG/io18MvT47SOxzE7TDidph488MhpvzFXsK5QT+nL/q4rd2JXifx5odDDIxHkGWZO26pUcI/F0ZDyMjKwHZdtYmhqRh6nZpAJM2hEyNs6XDS1eKgvtrEmC+Go8JAKpPj8kSULR1O9Bqp+FdbfI6kVhNP5aiuMLC9qwazQUsilWNoKkZbvY14MsuwJ0Y6kyebl4knc4pxSqTzuOwGOlscfHTeS/94hNMXvFiMGkx6DecGg0hqFf5IGpddz6f9AS6MhhicCOMLpxRjJ0nF8ZLLE1HS6TwFWcaol9i83kEkniGXL5DLy9RWm6itMvLQ9mY+ODO5YNB+pQgDsUxKoYSpQJxffzq5qHd6rWUBSgpk9stfiu0v5tlO+uP8wzsXeGBrI1U2Y9mQ0ZHPJjh9ycfgZHTRGRylMNRXuty4q0zc21NPU411Tsy1lO4bHwwyFUhy+qKXGoeJrhYHv/dABxKy4lGXi7+XvgadPzZRTuZJf5wX3/icUW+MTesdStz4475pXj8ySCJd7FW1uK1oNWr6RsJAMRRyeTKqhDIuTxTn6Je80I0tVfSPBnn9yCCyLHP4xAij3hhVFQbFK+2c8URLiqZ/loddMhqJVI6uluIHYrOnKZYchZKi39rupLPFQSKVo3cwwOBkhI1NlUpsubvVgd2i5/Ujg/QOBRicKspbW23CXWniw3MeLo2FGJwqetdP3LeBrnVVC2bpTPrj/Oyt81waCzPsieMLJ6m2G/inXw1QW2Vm3z3rqbTqSaRyfHrRy/FzHvbv7uCuzbVsmol9lwuZvXVimBO90wDotGpC0QzecEppN95gkuGpKH0jITqbK/mDvRvxBBJU2Qzc3V3L3x08jyeYRK0uhmTC8Sxmg4ZbN1Shk9SM+eI4rHqCsQzxVI4LIyFOX/LhDafQa4rXl5S0N5QkGE0TT+Vob6zAH0pz12Y3F0aDxFM5xSiVwjaSWkXfSIhEOk93q4M/3b+NwfEg5waDTPji5PIyG+pt/O69bRw6MczFsQgjnhjxVA6rUcOju9ZzT089I54YmWwetUrFyHQMrUZNLJlFK6k5ddFHPJWjs8WOy27i1EUfLruRULzo1NU4jIRmlsXI5WXiqRw1DiPJVI4CEI5nGJqKsmm9g1QmR4PLzOh0nGgyq+glfyTNhdEQ6UwWrUZieEZGl93ILS12sjmZLe3VHPlsinS2wC3NdryhFMFoWgllJVI5BsYjtNRa2bSK2W4gZjGtmFxeVhaom7/Uw2JfhPpCSVx2I/XVZt46MczIdFRZwiCdyfPp5QD+YGLRr4Jfee/SzAyHS3z38Z6ys1HS2aIyNRk0i87gKKUzHUzyZ7+/FatJx4EPLvPmh0PKV6WldDc02PjbfzxDNJnj7Y9HcdkNSBpJ+Zx/9syLDz6b4H/93W4AfvzaGaZDSeWZpeUZysn8ynuX6B0O0jscZNIfn5kN4iGdzWMySCRSxTwlUlmGpqJUVxhw2PQ8dm8rl8bC7NxcSyyZJZ48TziWIZsrKHsH7N7WQHerg/6xEAMTUXqHg4xOR+kdCvHIXS1KffW0VfPr34wzHUphNkiK4S7Vhdth5Jv3b6C71UFPWzXvnhrjzQ+HOHRimHgyp5T9pbEwAxOROXn79tc6lZkmm9Y7GJqKcGYgQGdzJahkeodCVFp0dDTauDAaprO5km9/rZNYMsuLb55TPgJMZ/O8+t4lPjgzQXymTNwOI0/ct4Ef/+Nn+MJpfOE05y4HFG+7xI//8TOe+9Y2ZVyqVP4bGmz84P/7DY01FjJK25Fw2Q0cPeuhwqwlEs+iUoEsw7gvpshiNmgx6CXODAT48T+eUcrBH0kRimVob7ShUav58KxHmZETjBd/z+cKDExGlbaaSOXQatT0DgeZCiYIzHjiRr3EdCBVVMJH4A8f7uTvDp6nvbGCdCaPJ5jg88t+AtE0jS4zkgpSMw5FW72d3qEQyZljeeY9DETTSrmoVRBN5viXY0N0NFUqZdbeWAwzZXMFAPKzluhpq7ezvq4Ck14ilckp19dXmxmaipFM53HZjUyHkvzmok9xcABaa61M+OJMBZKYDVqsJi3RRJb5DHmubLXcWleBXqfm4/Ne3A4j7Y12PhvwIeeLM/Ja6yxkcjLxRBZ/OIVWU1xn9c3LA3EAACAASURBVHqvNyZ6EPNQQgkz3kopNjw7XPHpJS8PbGug0qonn5exWYpLCHw+GOBfT40TT+U4e9nP+eEQ3lAKSa2iptLEpD/Ge6fGkSlw+0aXEged3YtodlvxhZM8vKOFj857aKqxKp5/aXaGQSsxOBUlmyvQ2VyJzaxb0BOpthv49JKXQDSjeJClGPq6WiueYJKetmo+7pums9nBfVsb0GnUBKNppkMp+oaDyn0HPrjMsXMeTHqJYCyDL5zk88EAlyevfHEryzKt9bY5IY3ZoaxbW6u5PBnmtvZq9Fo1qWweu1nHv54aJ5uTi6EJh4kxbxynzYAvksIfSWM2FH2YwckI4944J3o9JNI54skszTVWPuv3E46nOTMQwGk3FmedNNpocVfQP16sx1LYzBdO8s0HNnBpLEQwmqHKZqCnrZopf4J8QcYbTnH2sp8xbxydRs3JPi/xVI5srqB4jDqtmv7xK8bBpJf4dw9uBOClX15kwp/gwkiIbz/ciVajJhLPsPuOJsZ8MQbGo0p4pLJCxx231PD//EsvfSNhgtEM1TYDTpuBX86UiUkvsa6ugj96pAuLUcvxc1NEk0VFk8vn0WmlmeWYZVKZAvFUjo97PZzoneL451M47UY+6ZvmzICfvpEQA+MRvKEk2VyBbE5Gq5EIRtOYjRpFwRbTLr5AwWha6U3oNWqSmSvXpGb+r3EY6R0O0d3q4JGd6/h80E84XhxMlaQrMfSNTTbiyZxy3/znpWYMV16WZ8KKCbzhFJ8PBjk/FGDcnyCVyROJZwnNpH+mf5pahxm7VYcnkKAgoww+f9LnpbTLjMthRC5AXbUJSaVCLsgEYxliySwmg4RRJ9FQbabJbWVoKkp7g41qu5E3jg4STeZIZ4sG5PaNLkWujkYbjTVm/OH0HOMAsK6ugkwujz+SxmkzMDkzJqKRVBgNEtlcMUwGUGHSks4WiCYyeIIpNJKKSCLL54M+ook8qWyecW+cYDRDJJ4lmckzOPPe3d1dy/1bG8QYxHJYrYF468Qwb344ROWswa93T42RTOVorbcp3TxPMEG908LrRwb58MwkZy4HCMWuNJZsroDJoKHJaabJZeHjPi/BaLGb6o+kicQz1FaZOXRiZE6YyGrSsb3LzUfnPQs+HrswGuTyRJRJf5wKi47N6x1M+BKcvuRT0ghEUvxfr5/h6JkpQrEMnS12bGY99U4z6+sqyGTzHDs3yaf9fsa8MT74bBJZltnS7mJjcyVd6xxMBeLc0emmUCjG9UuDi001VmqrjLgqTchAMJLGaTcQTRZf2FJM+ZO+aQLRFM0zhu219we4PBFmOpQilc5xfiRMPJUjmszQ4LKwbaOLJ+5ro3c4gDeUorbKoJSVWo0yFmG36kimc8iFPOFEjsoKHb9zZwt33uIiHE/z6N3FsItKBWcvB6itMnJxLIwsy0hqNZcno1Ra9dQ7LXQ02WmusfK3/3iGcV+CuioTwZmeSXWFgYujIcKJLJK66FVXmHQkM3nyBRmdRkUuX0CWIZuXqbTq+R//eonIjJcYTxWV4vFzU0z4E/SPh4repFFDOJZV2sDpi9PIBVm5L5HOodUWY9bVFXqq7Qb6xyOMeWP85pJ3jkEORjMMTkYJxjKkMgWqbQYS6RzJTJ5QLFNUoJd9jHkT2M06CrJMOlsgmyso4wrJdJZcXqZ7vYN0tqAo89mkMjkcVgORRBaX3cimdZV4AkkKMkhqFSadhtvaqzEbtLx7aoxgLINarUKWi/kpKcJAJK0oWgCDVo2MrLyreq0am1lHIJomHCvWfclQ1VabCUbTC+6JxLNcGA0RCCfJ5Io/pjM5JLWKSX+SvCyjVquIJbJk8wXGfQn6xyPUVpnxhlPkCzKZbIF0tkAwlkFSq3DaDCTTWU72ecnn8pQ6FdUVes6PBPEEkzgq9CSSOS6Ohcnmi3lqcVuIxDLIwKQ/gT+SxmHR01xrQa/TzIwlyORyc5VTqUxKz1E6MXLx/8W2UgtG03Q02bl1Db6LEAZiGZS8bH8kTS6fo3coxNGzxa8Yw9EMhYJMKpOnqsLAY/e28sGZCZKZPFpJRWym+93RaEOlKnquNouOvpGQ0tgtRg2ZXNHbi8QymPQSg1MxZFmeM7Nj9vIHpdk+apWKcW+cXEEmmc4TiWcIxjJoNWom/QnODfo5eHQQXzhNIl2Maeq0EsfOeUCWuTQW5v3TY6SzMnqNikAkVWyQssyIJ8bLv7xIa31R9lFvjCOfTjDhi/HAtkaGpiJ8pcvN8XMe+seL3nw2XyCVzVMoyHQ2V2LQS0z6EyTSxfjo6QvTRc88mCA6UzbxVA67RYfFqMEXTuOPpNFr1YRiGU58Po1RJ5HJyYqn2ey2MjXjgXmDcWLJPDMRAfyRNKFYilxe5tg5D55ggosjYS6MhovKVqOmucZK32iIeCqH22HEaNDwy0/GcFYaeOODQRKZPFqNCotJRyiWwVGhL9b9zJsqy2DUSUSTWSRVcTZJOltQ2lh7o40Kk45zg1fW/AGYCiQUZ8Fu1RNNZMnmCoryLJVFPJVDlosKMl+QubWtmha3lWFPDE8wpeTTG0oxH4NOTXNN8ZsBrRpC8SxqVTHMUjwv0ei0EIimCcWLRkitgttvceKPpBUvPpHKolap5xiIlhoz0WROaaudLXae/J12Dh4bVpSaLEMonmE6lODiaNHoq1VXFFulRUsyM1cBQtGTzuTkOe9pvlDsiZfeoRL6GWO8baOLWDK74DxcMSQAkUSWkek4BflK/WmkK883GzQ8sK2BvpEA+cLcdILRYnssGezZMifSeQoz1yfTeaU3VVLguXyBdG5ugslMntHpOIFImqUsMC2poMFlJhLPUpCLdaWaVZ/AnPod80Zpb7TjqDCUS27JiOW+l8GkP87/8fNP5nSB5yOpoarCSCSeQqORiCWLXku+INPstrCxqZILwwGGPHGMOklpTKWXx2bWkskV5jxDI6nI5WXl73231RVjtYEknc2VNNZYOHl+WomtGrRqHDYDgUiKVKawQMbW+gra6m28/fEoUHzhZ8c8Z1NfbWLcl1jwu8OqJxBN43YYmQoklTyqmNtoTYaiIhqZjl213GZj0muwWXVMzjy3cWYwr4ROgkweDDoVqUzxaaUYuVJmasgVivHbSHzuYKtGUvHtr93Cr34zzoXRME6bAW84pZSvQSspYY3ZlNJcKg0uM95ggnS2KJhBq8btMM4p61KM/2rYzTpC8eJSEBqpOJh7LUp5mY8aKGVBq1ErMXYAs1EinlxaHUGxB1coFMdBwvHMVeu3ukJPNJGmTEdk1ZTyoZXUZPOFBW0BFraP2bjsRhLp8gZmPpIKyhTrF4LdrMNk1DBR5n0soZ7Jpwy47Ab+63/YsapnXm25b2Eg5jF79cmVMP+F3Ly+ks8Hg0vyIGZTGrhVq1Uryo9Jry4ar8QVb2SxZDRqleIxz8aol2h0Wbg4Gi57/2xFBHOfsdjzZhuXpVx/NblLSGoWeIMwV4HWVZuY8ieumla5Z5Xk1Ugq8nmZpdSEQatGlmXSuaXXW4VJQyKdL6vwl4peoyKdk6+q4DQSLOFD/hXhsGoJRK9t2G5kZivfm4E7O1388SObVpXG1QzEircc/W2lp60as2HxQR/VYkHBGbK5AhrpykXnLi/dOMxOO5HKo4IVG7tEuqB0VeHqSjY30yuYTzKdZ2hmtk65++fr5NnXLPa82T/Pv16lYkHZLyX75YwDzA09TPiubhwWe1bpp9wSjQNAKlu4qnGY3T5KRBK5VRkHgHSuWI9XS+Z6GQfgpjcOUGwDN4txABibjl3X9IWBmMdHvR5lemE5ltLfmv2iz79cpSp6KUtJ+4tsqIs9K/MF9rVlmauW/W8LqzUEV+NmUm6C1VOaIXW9WJWBOHjwIHv37mX37t289NJLC86fP3+effv2sWfPHp577jlyuWL8b2Jigv379/Pggw/y9NNPE48X47WRSIQ/+qM/4qGHHmL//v14vd7ViLci0mXi0muJLC/NKxYIBIJrodNcI6SxSlZsIDweDz/60Y94+eWXOXDgAK+88gr9/f1zrnnmmWf43ve+x9tvv40sy7z66qsAvPDCCzz55JMcPnyYTZs28ZOf/ASAv/7rv2bbtm289dZbPP744/yX//JfVpG1lRFP3fzdZIFA8D8HpckR14sVG4hjx46xfft27HY7JpOJPXv2cPjwYeX8+Pg4qVSKnp4eAPbt28fhw4fJZrOcPHmSPXv2zPkd4Fe/+hVf//rXAXj44Yc5cuQI2ewXq7B7501XFAgEghuVG7YHMT09jdPpVI5dLhcej2fR806nE4/HQzAYxGKxoNFo5vw+/x6NRoPFYiEQCKxUxBVh0F/fT9cFAoFgrcgsY6bcSljxWkyFQgHVrGk3sizPOV7s/PzrgAXHs+9Rq5dnwxabrrVUste5wAUCgWCtcFj1OJ3W65b+ig2E2+3mk08+UY69Xi8ul2vO+dmDzD6fD5fLhcPhIBqNks/nkSRpzn0ulwufz4fb7SaXyxGPx7Hbl7ej22q/g+hosuGdtQCdQCAQ3Kj4o2m83ui1L7wK1+U7iB07dnD8+HECgQDJZJJ33nmHXbt2Kefr6+vR6/WcOnUKgDfeeINdu3ah1WrZtm0bhw4dAuDAgQPKfffccw8HDhwA4NChQ2zbtg2tVrtSEVeExfDFPk8gEAhWSplPataUVX1JffDgQV588UWy2SyPPfYYTz31FE899RTf+c532Lx5M319fTz//PPEYjG6urr4/ve/j06nY3x8nGeffRa/309tbS0//OEPsdlshEIhnn32WUZHR7FarfzgBz+goaFhWTKttgdRWhJbIBAIbnQaXWZe+Pd3rioNsdTGMogmMnz3b4+uetMhgUBwY7DYmlUrRauG7DLW67peqFXwnce66W5d3YquYqmNZWA16ai2rW51RMFvL9J1eGOutXzLbIy6m+eVXU6+ridr/eV6nWt1E2HWArVaRUEu7t54XZ9zXVO/SckutriPYNH16ZfKF6ngFlvSZKnXllNw16NplOutzjdEJr2ESS8py2dfDUkFVpOG+mpT2fqan/ZaGL1yz7laL1xF+TWpSudKG0WtNXXVJgw6aVlt49/tacdRoQeKK/N6/MufxGLUS9RVm4Dyba2+2qT8Xk4000x5SOpiWoWCjNWo4eEdLcuWZTkIA1EG6yobp157pVgXewkWYzUK+L7b6nBY9Su+32rSXFPe0juvkVS4HcZlPyOZKSjbJUIxv9olfOxjMy+/TsqtzLqUa0sfHzksSy/L+eVWbnb21Z4/2xjZzTrcDtOc86l0fsHOZeXQSSryMkQTOaZDqbJrMzU6zXPkna+MbeYrEzWKyrr4bdDVNi4rPadkbNSquYZHUoPVpJ1z/db2aqRZ2lI161xq3n4Ls5n9m0ZdVPqluprd+290mdFrVXPymkxlSWXyc+q70qpb9Fl2i457b2ugembPhVxOLrtM/Gxa6yqotl1pO5JaxdYOJxO+BN2tDrZtvPJ9WKmss/kCBRn0GjV3dbuBoiEAqLLqqZ151yqtBpLpPGaDRDSZ49zl6/udmDAQ84gmMsSXuKdBOTSSinS2gEmvoaXGsmwDUWq31+qel16+Uvpmg4b7tzXyp9/sobXOqjSuazH7Jc7m5CV1x80GDRUmHVOBuZ7UUnc+3LSuEvWMYpBnnquf0VGLyW3UF1+ka5WLq9KwwOCUymipgYZMrrgBUnJGEWikK0qm9HxXpYE7O11oZsTd3umipeaK4rWbigpCr1VTsoel5+skFUa9mkaXmRZ3MVxRYSoWgEGrIhTPLNifowC0N1Zg0C58ZU2G4m8Oq547umpmfpPmLDtvN+todluotukZ8sTn1LNGKmai1FS3tjtprS3Orb/3tjrMxpISA71OjUmv4ZZm2wI59Bo12zYWp6wb9BryhSveckttxZx9mSvMWgYnY+QL8pUys1zZtCZfkNGq4Vt72hUDVvKid9/RyCN3tVBbbSJXKK7UG4ilaa2vwFGh474t9bQ32BidjpPOyjisBhwzRiAYuyJDS42FzuZKgtEM3a0O/uxbW+huddDgMivXWE1aXnnvEvVOM53NlfzRI5201lVg1M+th1IetBoVT9zfhst+pbeQL8hMBxNFmavMhGZ2zAPQShJfva2OFrcVs0HDH+zdiMWgpbXWSrXNgMOix16hZ2AiSnergz96pJPuVgd3droXlP/1QOwoN4+3Tgzzab9fOdaql7e4XunabL5AaGabwflKrbSN5bXQa9SYDdKcrRqh6PEk0nkcFXriqRxGnZpEOo9Oo6ZrnYNPL/mZ9CfQatQLBuw1kmpOfkpyuOxGKi1aZeex2dfOvseok0jM2lGrrspIXpbJ5uSy5STN2kGtVA4Oq4HpUBK9Rq1sFF9ag99u0VFlMyzYYCeRzCkK1m7Rkc6UN+JGfXEDJ71WjUqWqXcVX8jW+gpCsfSca/VaNRUmzZwNlypMWu69rb64E+BwCPXMXhMFGaxGDX/w0EayuTzfvH8Db344pNRNKlNg3J/g4R0tdLdVYTRIjHiKCnBdrZV4Mqfk1WiQSKSKy7G77EZqKk2MeYsGoaTTjXo1ubyMSa8mO6PMk5liuc/eIQ1m9vPIy9zTU8c3dq7HatKyvbOGvuEgTltxS9h6p4nByRiJdH7OrmRGvUQ0kaXapieeyrNpvYMah4mv37WOKpuB3bc30dFk59TFabI5mXxeJpsv0N1ajVZT3B61lE4qWyCdzpFI56mvMpKd2dIToKetCig6YLIMLW4rI9Mx3A4jG5vtjPsSbO2oxmrUKRs/FWTwBBOKAn/60U1YjFo0kpr7tzZwR1ctn12axmHVE0lkicTT+MJpjHqJ9sZKLoyG0GrURBNZGl0W/JE0rXVWetqqMek1/MHeW9ixyY3VpOXRneuprTKzvcvNxqZKzgz4SKSLe2APjEcYmopy75Z6dm6uI57K8ll/gM7mSrZ0ONFrJDzBJGaDhh1dbkKxDEfPTuGqNCgbFN2+0YUkqTl0YgR/JE1How1JrcIbTpHK5LgwGiabK/BZv4+LY2GCsSt7UAejaVx2A+2Ndno2OLmnpx6n3YgvnGTPHU1YF9kNbqlcbUc50YO4BquZrWCd8QrnG4OlxrHTuQKRxNwdsFrrKrjjlqKXGIkXPZFS8pcnIrx7aoze4eJ6Utl5W6NpNUVFYrfoFM9OI6locVvoWldJTZV5zvXdrVXoNWo2NFQovzXOG6Az6rUkZi3RXQoflbxmFbLikclyMVzhsOnZc0cjt26owqBVo5kZcDMbJHzh4laYLTXFeypmuuDWWWGPXbfW8a097UhqFU/ubld6I1qNmo3NlUBxC8icDL5wise/2kpXiwNZnjtvvNltwWmfG8ox6DTs3d6sbARf2mbSVWng2d/fyp2dbr77eA/nLgdIzOw9rZFUTIeSdLc62LTewYWRINqZrpnZIDEwESUzsxtaUbYrDeLiWJgah5E9dzTS0WhT8p1MF2Y8xk1KiCeRyqHXqNneNdd7LI1L6Gb2NH/ozmaOfz5FPJUjnCi2EWlWV7FnQzU6SUWjy6zsEOewFkMo/kiaNz8c4qPeK8vmtNXbee5b22hvsCk9vKlAnP/l33TT2VL8kLVupu34IkVlNuSJs3NzLZ3Nldx3Wx2eQJKhqSj5QtHQ/u69rTz+1Vb+7Pe3Ks+2GnW0NRR7JtU2A+0NNqYCSVx2A3VVZixGLXqdxJsfDnH07CS9QwGmgymMM93P0nvV5LJyZ2cNLruBbK5AZ4udx77aSnerg3//tU6q7UZ6h4N82u9TyguKm4VFE6W90IsNpaPRxp6ZHktPWzVvfTRMT1s1j3+1lT/+RhdP3LeB/bvb6W51cHd3Le/9ZgKAR+5qwT4T9upa70CnlcjMCk11NFXyvz1+64xMt9DeaFvQNqpter56Wx2dzZVMh1Ic/niUd08VB6U/7fdxZiDAp/0+rieiBzGPeqeZTDbPiCc6s1m9loIsUygs9L6vFTxSU9yMZ/59JWZ71+XQa1U0Os3YrQaqrHqclUb6xyNo1CpSM3sjQ3HbwWgyR3drFaPTUWX/YrNBg6QGSVXcKrRQKCo6u0VLIFp8EQoyhGIZhqaijHuLy65rNWp2dtdy9rK/6LWm82RzBVx2Iw9tb6JvOIi7sriRvcOqIziTlqvSQDSRxe0w8m8faOfcgJ9sXkanKfZwSr2PEU+MSV+coakYuYJMW4ON+7Y2YDFpGfHESKRz6LQa4qkczW4r921t4ME7m/BHkmxtd7L7jiZ+9tZ5YskcU/441TYD2Wyepx/dRCqT58JoCKtRSzpbwGrUsO+eVi5PRNBrJP7NrvUMT0WpdRhpcln55KJvpqzVqFQQTWaZ8MXY2V2LP5xi03oHOo2akek4FqNWMUBnBnzFupiZQtneaKOt3s6xs1OcGwxi0mtwVhqY8CWptulJpPPc21NLMpMjNBPmMBk0ZHMFEqkcLW4rx8550Okk4qkc7Q022hrsBKNpzg0GaW+sIDXTg9BKKhxWAypga0c1nS0OWuttaCQ19U4zeq1Etd3Ap5e8xJI5ulsdPHr3eqX8Kq0GeodD3NtTT0utFb1GYt+96wnH0wyMR+hudWDUa3jzwyF0M0b36NlJPjgzSS4v43YY+cOHu9BpJcKxDB1NdmwWHQPjxc2ldBoVVqMOq1nL6Ys+jAaJ3uEQzW4LyWSWZKaAu8rEzs21HD07STqbZ2A8glqlwmrW0dXi4LF7W0ln80hqFUOeGP3jEawmLTs31yp/u9qcSCoZs0FD/8yzW+us/P7uDj7um+ZkX3Elh690uQnHM7x/eoIqm2FOGiVH4K0Tw7x+ZBCdRs2HZye5PBHF7TDyvz/Rw5Z2l1IGr70/QJXNwEN3XnEijp6d5P3TE3S2OOhuq+Lennom/HE++Gyy6DS0OvmnXw3Q2eJgXa0VnVbNQ9ublR6Lo8LArW3VIMuo1XBrWzWb11fxBw/dwu231LC5tYoRTwRvKEVHk52NzZW4HaYFeVgpV+tBCAMxD71WYtwXVzaht5qubKReTsk7rDp29dSxod5Go8tCLJlVNn/XaFTYLXpiydyccI+KYrw4kc7htBlIpHO01llx2o1s2+jC7TAyOh2nxW1hcCqOTiMx5ouzbaMLvUaibySk7FxXkMFVaeKBbQ1IajVHz06hldQUZBmLUYNJryWSyFJdUXxOPJUjObO1ZYVJqxiZapue2iozwWia39nWgCeYZNwbx6BTUWM3UuMw8cff6OKnv+gjHM+gkdT8zu2NBCLFjd5ddgN/+PVO/JEkg5Mx6qvN7N/dji+cpN5pYcQTw6iXyGQLGGbt0w3Fnsrv3tPG8FSUC6MhOpsreXL3BsLxNPt/p4PbNjhxVBjYsamWrnVV6LUSU/44g1NRzAYtE74Et7RU8o2d67FZdEz5E8XQUjRNJJHDF05y7JwHbzhFMp1l2BOjscaC3aKnfzyCVqMmkysoYaTR6Tj+cIre4aKib661MjAeUV5OgEtjYS6MhpQ2UeMwcvTsFFs6nJgNGv7tA+0k00Vjdfetdey6rYF7e+q4u7sOnUbNulorboeJRCrHdChJR5MdvU6tKKZbmiv5xYkROprsdLdV8dg9bYoiddqKjsKGRhu/v3sjW9pdjE7HeP3IIFaTlg0Ndj467+E3l/x0tzr49tc6+bTfx68/neS2dif39tRjNWnZ1uHiRG/RoPkjSRqcFm7vcvPIjhZGPDEujIZYV1sMBW1osOEPp9jS4eTf772FKpuRt04M8+aHQ3Q02dlzRxM6jZpgLIU/kiGRzpHOFIincmxtd3Jbu5NQNMNUMInLbqS2ysTgZITXjwzS1eJAr1PTOxxiYDyCXqcmnszx5odDfGWTm44mOx1N9hm5dWxosKPXSjgqTdRVGmmqsTIyFcUbTnHnLTV4gkl62qqxGLV0NNm5f2sDTTXWWQbmShol+kaCXBgN0VJrpcKkQ6dV84cPd1FluzIRYzGl7HaY0M10mUsylq59dOd6WhsrGZ0Ks+eOJoKxNL/+dJIqm4ENDXaiiQzvnR6jqcbKlnYXO7vr6G6tZmNzpfIMvVZiXW3FnJCSXistyMNKEQZimZQqXEZmdDqOUS8tOnhbV22mtspMviBjMmh57N5WLo+HCMWz5PIyDS4T9U4zk/6Ekk5rXQUNLjPjvgRVNj339tTjqjRx7JyHLe1OdBqJC6Mh7rylBpNBw+XJ4lorbfU2XJVGLoyGAGhyWQjFM/S0FRVsvdOMTqPGUaEjlc4jyzK+mRjx3bfWsqHexqQ/roQVstkCDS4zboeJ//CNTdx9ax1Wk5b7tzZw6qIXbzCJTqvBG04jSSpUqAjGUoRiGQw6NdmcTN9oCJfdyHQoRaVVj6PCwIXREDIy6Uye372nlSl/ggujISUeXWHSYtJrFCPR3VpFvdPM4GRRCT/+1TbiqSxHz0yyab0DR8XC71LW1VVgNWn5vQfa8fhjPHHfBqwmHUfPTvLrTycY98aVXs83H9iAP5zCG06xtd2JyaDhzECAzhnFNOlPUF2hp9Fl4Q/23kKlVc+kP04wmsEbTqHXSHxlk5v7tzYoL2SprFtqrXS1OHhoezNVNgP3b23gnhklUe80K+W5o6eBXDaPXiuxsbmSkekYvzg+wv1bG+huq2Jbh4tAJE00kcETTM3xRktGsXlGyT14ZxO+cJIzAwHFIFiMWnzhJA9sbcRq0inHT9y3AZ1W4tJYmI4mO9s6XHzcN83OzbV83DfN+6cncDuMDE4WvfRN66vpbK5U8jc4GeXXn04oBnNDvQ1PMInbYaJvJMjAeISWWqviZXetczA8FUajVrN/dzuuSiOSpGZbh4twLI1Oq6ajqZJDs4zfvT31M+1ejSTBucHgnHO3tlVT7zRz9OwkbodJqQOzWU8ikUGvldjcWqXMknr9yCBVNgNf37FOUbSLKdSSgt7W4aJqZgbU5pxVRgAAIABJREFUL06McO9t9dy2wTnnmqYaq1IXs9FrJQanIoqBdjtMvHtqjGQqV+wtB+K8+8kYVTYDPW3Vc+rpvdNjvPb+gFKPi1HqpZQMy1oiDMQy0WslbBYd758aJ5HO0VBtJhTPYJhR8BpJRb3TjNNmRFKrOPa5h/7xyJVBsWRWGby7faOLvV9pwRdO8vBXWrg4GmIqkCQQTZPLy0TiWSS1CrNRS1eLg20dLiUc8tWt9WSyBXL5AsFomrZ6G3dtrmVsOkaVzUCzu4LBySgmvYZ1dRV83DdNPi/zr5+M09ZQwdBUDJfdyM7uWvZub8YTTPJpv1/pYchAJJ6lptLEjk3FuPbIdIymGitf6a5ndCpcNHgTEaZDKS7MGANJrcYfSeMNp/7/9u49uKk6/xv4u2mT3u2NpEkLLb1ya5FLa1WgWEF7o1yKLAiyzOP6U9kdqevISnUWxLWLsvugO4jMsyv7sMMAsyJKQVvgUaEuIspNCxYQI22BtE3SpqVNL2nT7/NHeo65nLRpckpb9/OacaQn5ySfk3Py/XzP5/tNDibHhmFCbCjfw543cyw0+jZUVTdbSj0BUjw0LZq/RA7088ad9h48kj6O7xnOmzkWpy7V4aMvbsBXJsGMZDn+duA7NBg6ca3WgPlp4wSPUdLYUISG+ENvaIcyPACnLtVhWuIY3NK28e+/sbMHyogAFGbG84011xgBQPZ9MYgI8cPq7Al4eOY4hN/jh6u1Bpyp0gKwlM2q69swNTECMZHB+PzCrb7eoSUB1DZY7gk8MSbMofHgYjR1m3Hi4m2EB8n4x7keJpcATl2qw+Evqy3xhvujcG4CEqNDbBpF7vmCA2SYEmcpfZnNDNHyQL6x5xoQ6waltu/qguuZcw0ZV2pZNDsO9U1G6Jo7kRwTCvXtFsREBkPTaETFtxpMTQhHtDwI6tt3IJNKUPFtHWrrW9F0pxONd7owZXw4f2UVHCCDsbMHZ6/qMC4yCD7eEnz0xQ1UVTfh2x8b8dD0aMybOdZm37l9mxgbhtT4CIfeOADBhtTQZsI7H3yLWGUwIkL8kTQ2lE/K0xLH8O+dqdvMHzf7xp17Xq5sxG1vfZUg9Npc0uCe0/rqgjuXuTYhJX4MJsWG2iRl7ji5WioSs6RkjxKEG/5P6feo6bsh+IzkMUhNiEBnVw8MrSbEKYNR3dAGVYQ/rta2YHJsGMyMoaPLDMOdLvhILFMVE1TBWJwZj39/fh2V6ibc1rWhqa/hCvTzQVd3L/x8vVHX2G6p/SZGoKa+FWVnaqFr6USLsQtffFcHVXgAdC2dSIwOwZmqelTVNKPxTheMHd0YKw/E1dpm6Fs6cOKCBtHyQHR09WBWqgoNhnb8ZsEkTE+S840n18v9Tq3nr4p0LZ2Q+Uhw4uItnLigQXCAFFOTFdAb2nFvwhh09/RakkOYH2q1RsQqgxAW5IvGO114IEXZd9kr5T/Qscpg1De2Y8YEOTImR+Kbq1pMT5Kjuv4OfpWViO4eM2Yky3Hxus5Smujr4XK9R5mPBC1GEwytXciYHAl5qD92fVKFWGUwggNkNh/Or76vx97jP6CqugnfXNFaes59DVdC1D2YM1Vl02vmLs+5Hp99PRn4udwwOTYM/1Mwha9bczVorqH4/MItfPTFDVy72czX6oV8fuEW9h7/waaBse/Rclet3hIv3KhvRZC/1OZ42Pca7Xut9nV16wYlJjIYGn0bKtVNfM/cutQSHCBDarylBy6V+uD9z67bPCc3w0ff0oGFs+PQYuzqK0t1IXlsCJbPS7J5/4L8pahvbIefzBsZkyOhb+mAuq90tmh2HN+YCzV09vvF7bdQA/n3w5dx8Qc99C0d/MC9qduMWm0bfrjZzI+haBqNTnvp9s/LvXfWiVnote2ThvXx5I5lnCoYk8eHY0lWEuIjgwWfy9VSkZglJXuUINxwU9fGD7qlJkTAV+aNU5fqMTUhHOOV9+DH23f42mphZjzaO3ugvn0HHSYzmvtmF2VMjoShrcsygBUbhvGqYAQGSBEWJIOmb557j9ky5/6BFCXSJihQcVEDXUsnkseFIEZhOcG48kVXtxmnLzcgeVwIvLy8+mrqPZg3cyzyHhiPiBA/3KhrwY26NvykaUFTqwktxi60d/XgwAk16hvbERIog6G1C9/fMEAR6o8pcWHo7mEIDfbFmSqtZUBzdjzf8Gr0bXwve/HseNQ3tqOqphnpExVW9Wzbmi5X5pmRLEd9UzsOnFCjuv4O6ps6cFvXBrWmFdX1d6DWtELf0oH2Lkuv9oEpSkxNjIDZzHCmqgFTE8Lx+Pxk7Dl+DZXqJr4hsB5QLMhMwI83m/h54qFBvjh92TIDZ85UFRbPiXdo2IH+e2RcL7IwMx4yqTdqtW0Iv8cPNfWtfC3cV+qNIH8pLv/UBGNnj834hD1leABU8iDcN1Hh9APO96D7SiVmM8MXfYOci2fHC25nvQ/2x8C6QfGVemNKXLhDr93+9ZPGhmJSwhh4ezGH5+SuSFqMXVj+cBK/35FhAciaYfuDmtzxV9++w583Gr3luNuXSOx74s6OjVDDPTVZgZv1LXx5Efi54fb18YaupdNq7EL4WAs1vP01/kLvvdBzWsptEZgYG4bwsAC0982Osn4uoX13RV2j0abD5ClKEG6IjQwGYwy+Pt7InBaFmrpWyKQSLJwdZ5nHHBOKR++LsakPczVpby/L/PDJ48Px0LRoyHwk0Bo6cP4HPealxaBwThxkPhJEjQlAgK8PVj2ajIkxYdhz/Bou3zBgakI4EqND8cmZWkxNjMD0JDmSxobix76B0VmpKijC/KG+fadvlpA3epnlQz0xJgz6lg7k3R+L27o2FMyKg765E94SL1TVWHrGE2JC4e0NVNe3obunF9rmTvj7euOh6dFYPDve0jDEhuPHm02oVDfxvezgABm0zR2W3vX4cBQ8GMef2NYnu/WAIPfv+WljLY3LvCQoIwJ+/vvhJCRGh/BlhbQJCtzWGTEhJhRL5ybwVyRcPT04QMb38L29gTnTx2Li2BC+pxsfdQ9kPhK+dOWsB9hfj8z6Ma6x0Ld0WBrsxAhMibPM6T91qQ5nr1qS6tK5Cf02/mlTVOgZ4Bu41q/NJSnueLS2m1B+pgZXaw38TKXB9CpdbZi4gV/75crwAP4qJCLED4vnxEHf0oHH5yc7NFJcD9p6YJlLUPYNqlD5xtl+2a+rUgRj6vhwm9fnjvWj6eMQEeIn2IEZiP35IvR+Daah58ZK7Lk6/mBv1ydVNh0mT1GCGATrAakGQwcqvrX0mr683ABdcyeu1TbjwnW9YE16YmwYUuIicG/iGJuSy436O/hPZR0AIDkmFKl9PQuuxqtv6YChtYvvMf4mfzLi+wZhZ6eq+BoqN5CWNkGB+qZ2RMsD4e/rjXGRwTj8ZTWCA6SYniTH/VOUuPRTI765okV7Vze++K7OZjbIvJlj0dJmsgyET45EcIAUj89PxvQkOf9aE8dH8A2vdQyTYsPQ0NQO/Z1OJESHCNaIp8RF8LX3U5fqMDtVZTOlL2lsKMLv8cP9U5QOJR+hhjg4QMavC1h6+Bp9Gy7fMCAkSIak6BCb3vLE2DB+cLK13cTH4GpvSyjZzZ85ji812fd0uUa8P1wj4WqvkSuVxPSVJqzLWYNtUOz11zA5a8ysr0Jmp6oQEeJvc0zsY9c0Gm3GEOz3hzOY2rr9ukKxWo/TuFuSsU9QAzXkAz0eGOiLBn2bS1dKrrDvMHmKEsQgWJcvJsWG4fqtZsxPG4ebDW0wdlqmiU5NCMf8meOw5/g1vkasDA/gTwAANh8GZXgAPw3v3iQFEqIsXzyz7pVxtWGusRHqxXI9eW4QLG2iAmtyJvGzW4Qar/kzxyGo76cS5s0ci3sTx8BX6s33UB9NH8fPugF+PtllUm/U1N/BtMQx+E9lHf7f2Zt8suTGQKx7MEIneyl3bw3G+MbeGet47Rtie9aNVe6seJh7nPfMXRlgdLYNV7/mGkT7BmcwPXiuMXPWmNjHZL+eda88bYLCYUbPYPTXMDlLEIPZX6F9dLbf7l4FDRSrmAZqyLnHrQfGrdcLDPTFkVM/uXylNBD7DpOnKEEMAle+mBATim+uNECtacUPtQYY2kyYPD4UD0xRYuncBJspgvPTxuLfn1/HiQsah8FerifdYrR8oWhJVhLfoLlSGwYcT1BXBrqse1JCg37c4/YzPLjn9vb2wr8/+5Hv0XMzlqLlQYhVBMHf18emvCDUQyz/uga65k7IpBI8mKLq9313tednP+XQur4rxJUBRmfbmM3M4X1zF9eYOWtshBKC/THmroyExlQGo7+GydNGt7XdxE+p5cZqAOeNrLt1eDFiddVADTn3uLPjEhjoi2Bf7yGbheQpShCDwPWs0yYoYLjThTtGEwxtlt+C+Z+CKXwPnOv9qzWtaDF2oVLdxA+S2g8ucuWBqYkRyEiJsjmpB6pl2pdI3CmZ9NcDsr5i4soySWNDMSneMljJXYFMiLGUhcrO1CJtkgJrcibavL5Qo6sI88f1W814fH4y/10GTxoEodfpYcDHX/7k9PlcHWC0jotLUkJTJgcbM/e846NCYO4xO21sBjO7hbua4Ka4itngeNroWp/r1leNro4r3M1YxebscxYY6IuebvOQzULyFCWIQbDuDRz+spr/ItPi2fEAYNOIcL1/rizCDZLa16WtL0G/vFQHmbeXQ4PT2m7Crk+qHKY1Wn+Agvyl+N///hbfXNEO6gPVX2NjfcVkPQuHG6zkxlYm9n15ylmiEfpwfH2lAd9c0aLF2IVYZTBOXarD9VstHvXK7V/n5Le3HaaQuvN+9DdY6kmPnXvekCAZYvu50cxgSy1CV4Wc4eyVD7aubv8Fv8EQM0F42nEBnB/DkZbI7A1JgtBoNHj22Wfxj3/8A19++SWysrIgk9m+iMlkwoYNG/DWW2/h4MGDSE9PR3h4OBhj2Lp1K15//XXs27cPEydORFRUFL9dQ0MDcnJy8OSTTw46LrFmMXEnOjcgHBMZzDcUGn1bX3L4eR659SCps3LPqUt12Hv8B/47C/aJ4MQFjcO0RusP3J7j1/j55KseSe73C0Cu4hp963IAIHxS99eIOeupc2Ms3D5bz8MXYwAxKTacn5bpSe+sv4atv8cGali4bQcaKxErXmcdDVd52pi5kuis3zP7L445W28oymHWPLmSGch/ZYJ48cUX8dhjj+G1115DTU0NTp8+jVmzZtmss3v3bhiNRuzatQsJCQl47bXXsGzZMhw7dgynT5/G/v37MWvWLDz//PN4/PHHIZFIUFFRgRdeeAEajQbPPffcoOMSI0FYl3G+uarlT5zZqSq+wZP5SHCj/s6gGmduPnzmVJVLM2Lsy0nc7IUn8yfzv4VjXR7ithlM0nC31+PsdexLNfZXWf2NtbjD2bTMwRps8uMM1LBw2w40ViJWvM46Gq6yP+5i9KztWb9n9j894Ww9Z7ODxHpPh/KbyqM5Qbh167Tu7m6cPXsWO3bsAAAUFhbiiSeewPr1623WO3nyJIqKigAA6enpaGpqgkajQUVFBfLy8iCRSBAXFweVSoWLFy8iPT0dH3zwAbZv346CggJ3QhMFd6UAALNTVfz/gwNk+E3+ZMsvUJrM/DrczwUPJDhAhsKsJOh0rVDZ/bS29c8OC8WRm2H59cfnl01zOXZX43KHs9exX269X/b77A53p60OdhtXWZ8fQ8nVfbA/Xz19TW7OPSDe+WQd46lLdX2z+PQO58fdem8B4c8fcTNBGAwGBAUFwcfHsrlcLkdDQ4PDelqtFnL5z7fXk8vlqK+vh1arhUKhcFgOANu3b3cnJFHZf8isTxzub+5347tMZv7fYjZCre0mdJnMWDhrvNMPyLyZY+Er87Z5fKg/VFxDNS1xjODrDPXru5oArRvUoUyad6thcXUfxIyHa7ynJoSLejytY+zvfKFGe/gNmCDKy8uxZcsWm2WxsbHwsrtNmv3fAMAYs1nOGINEIkFvb6/gcjFERDgfCHSVHEB8bP/z9uUAvv2pCf/34ypEhAWg02T5OQsfqTdWZk/qf1t58IAx/OfEdRz+shr/a8Fkp7EIxSmH5T347Gwt5qXHIGQQ91UGgJa2rr5tZYJx/ufEdRw4oUZQoC9+vSDFpZjEtOihJAQF+trs20BxCm0zHFw57s4MxT78fKxtn5OL8268b56eL568p3fTaInT3oAJIjc3F7m5uTbLuru7kZGRAbPZDG9vb+h0OpsrAk5kZCS0Wi1iYmIAAHq9HgqFAkqlElqtll+PWy6GxsY2h9tsDpVp8eFYlpWAafHh/J2ejEYTdLpWp9vI5cH9Pi703K6sb6386xocOKFGm7Fr0D0wblsAmJPi+DV+T+ISy5wUJUwdJug6TE7fT+s4TR0mm22Gg6vHvT9i74PQeWIf53C/b/0R4z29G0Z6nBKJl9OOtVslJqlUirS0NJSVlaGgoACHDh1CZmamw3pz585FaWkp0tLScO7cOfj6+iIqKgqZmZk4ePAgFixYgFu3bqG6uhqpqanuhDKsrC+Bhco9Yj33YE1LHINrtQa+DDQYXPzz0mNgEmgURstl/2iJczjdzRo/GZ3cruts2rQJ77//PvLy8nDu3Dk8//zzAID9+/fjb3/7GwBg9erVMJlMyM/PR0lJCbZu3QoAyMnJQVJSEhYuXIjf/va3KCkpgZ+f401hRhOuQRJ7ENQdntyvltsP65JCa7vJ5n695JdhJJ2zZGTyYkyMbw2MHHezxOQOTy43XZ3JIsasHes4uVLEsqyEEdcrH+mX7xyKU3yjJdaRHqfoJSYyPIZjJgtApQhC/ltRghhFhquhpno+If+dKEGMItRQE0LuJnG+fEAIIeQXhxIEIYQQQZQgCCGECKIEQTxG35Mg5JeJEgTxGDf99tSluuEOhRAiIprFRDxG35Mg5JeJEgTxGE2/JeSXiUpMhBBCBFGCIIQQIogSBCGEEEGUIAghhAiiBEEIIUQQJQhCCCGCKEEQQggRRAmCEEKIILcThEajwapVq5CTk4O1a9fCaDQ6rGMymbB+/Xrk5uZiyZIlUKstd0NjjOHNN99ETk4O8vLycP78eQCA2WzGpk2bsGDBAuTn52P37t3uhkcIIcRDbieIzZs3Y+XKlTh69ChSUlLw7rvvOqyzZ88e+Pv7o7y8HC+//DKKi4sBAMeOHYNarUZZWRl27NiB4uJi9PT04MMPP0RzczMOHz6MAwcO4P3338f333/v/t4RQghxm1sJoru7G2fPnkV2djYAoLCwEEePHnVY7+TJk1i4cCEAID09HU1NTdBoNKioqEBeXh4kEgni4uKgUqlw8eJFJCUl4Xe/+x0kEgkCAgIwbtw41NXRD8ARQshwcCtBGAwGBAUFwcfH8lNOcrkcDQ0NDutptVrI5XL+b7lcjvr6emi1WigUCofl06ZNQ3JyMgDgwoULqKysRHp6ujshEkII8dCAP9ZXXl6OLVu22CyLjY2Fl5eXzTL7vwHLWIP1csYYJBIJent7BZdzzp49i9///vf461//ipCQENf3BkBERNCg1h8OcnnwcIfgEopTXBSn+EZLrKMlTnsDJojc3Fzk5ubaLOvu7kZGRgbMZjO8vb2h0+lsrgg4kZGR0Gq1iImJAQDo9XooFAoolUpotVp+PW45ABw/fhyvvvoq3nrrLWRkZAx6hxob29Dbywa93d0ilwdDp2sd7jAGRHGKi+IU32iJdaTHKZF4Oe1Yu1VikkqlSEtLQ1lZGQDg0KFDyMzMdFhv7ty5KC0tBQCcO3cOvr6+iIqKQmZmJo4cOQKz2YyamhpUV1cjNTUVlZWVePXVV/HPf/7TreRACCFEPG7fD2LTpk3YsGEDdu7cCZVKhW3btgEA9u/fD61Wi6KiIqxevRobN25Efn4+ZDIZtm7dCgDIyclBZWUlP4BdUlICPz8/7Ny5E2azGS+99BL/OuvWrcO8efM82UdCCCFu8GKMjdx6jBuoxCQOilNcFKf4RkusIz1O0UtMhBBCfvkoQRBCCBFECYIQQoggShCEEEIEUYIghBAiiBIEIYQQQZQgCCGECKIEQQghRBAlCEIIIYIoQRBCCBFECYIQQoggShCEEEIEUYIghBAiiBIEIYQQQZQgCCGECKIEQQghRBAlCEIIIYIoQRBCCBHkdoLQaDRYtWoVcnJysHbtWhiNRod1TCYT1q9fj9zcXCxZsgRqtRoAwBjDm2++iZycHOTl5eH8+fMAALPZjI0bN2LBggUoKCjAkSNH3A2PEEKIh9xOEJs3b8bKlStx9OhRpKSk4N1333VYZ8+ePfD390d5eTlefvllFBcXAwCOHTsGtVqNsrIy7NixA8XFxejp6cHhw4fR1taGjz/+GP/617/w+uuvo62tzf29I4QQ4ja3EkR3dzfOnj2L7OxsAEBhYSGOHj3qsN7JkyexcOFCAEB6ejqampqg0WhQUVGBvLw8SCQSxMXFQaVS4eLFi1iyZAm2bt0KANBqtZBKpZBKpe7uGyGEEA+4lSAMBgOCgoLg4+MDAJDL5WhoaHBYT6vVQi6X83/L5XLU19dDq9VCoVA4LAcAHx8fvPLKK3jsscfwq1/9Cr6+vu6ESAghxEM+A61QXl6OLVu22CyLjY2Fl5eXzTL7vwHLWIP1csYYJBIJent7BZdzSkpK8OKLL2L16tWYMWMGZs+e7fIORUQEubzucJHLg4c7BJdQnOKiOMU3WmIdLXHaGzBB5ObmIjc312ZZd3c3MjIyYDab4e3tDZ1OZ3NFwImMjIRWq0VMTAwAQK/XQ6FQQKlUQqvV8utxyy9fvoygoCCMHz8eYWFhmDNnDq5duzaoBNHY2IbeXuby+nebXB4Mna51uMMYEMUpLopTfKMl1pEep0Ti5bRj7VaJSSqVIi0tDWVlZQCAQ4cOITMz02G9uXPnorS0FABw7tw5+Pr6IioqCpmZmThy5AjMZjNqampQXV2N1NRUfPfdd/jLX/6C3t5etLW14dSpU5gxY4Y7IRJCCPHQgFcQzmzatAkbNmzAzp07oVKpsG3bNgDA/v37odVqUVRUhNWrV2Pjxo3Iz8+HTCbjB6BzcnJQWVnJD2CXlJTAz88PK1aswLVr11BQUACJRIJVq1Zh+vTpIuwmIYSQwfJijI3ceowbqMQkDopTXBSn+EZLrCM9TtFLTIQQQn75KEEQQggRRAmCEEKIIEoQhBBCBFGCIIQQIogSBCGEEEGUIAghhAiiBEEIIUQQJQhCCCGCKEEQQggRRAmCEEKIIEoQhBBCBFGCIIQQIogSBCGEEEGUIAghhAiiBEEIIUQQJQhCCCGCKEEQQggRRAmCEEKIILcThEajwapVq5CTk4O1a9fCaDQ6rGMymbB+/Xrk5uZiyZIlUKvVAADGGN58803k5OQgLy8P58+fd9h23bp12L59u7vhEUII8ZDbCWLz5s1YuXIljh49ipSUFLz77rsO6+zZswf+/v4oLy/Hyy+/jOLiYgDAsWPHoFarUVZWhh07dqC4uBg9PT38dh988AG+/vprd0MjhBAiArcSRHd3N86ePYvs7GwAQGFhIY4ePeqw3smTJ7Fw4UIAQHp6OpqamqDRaFBRUYG8vDxIJBLExcVBpVLh4sWLAIDa2lp89NFHWL58ubv7RAghRAQ+7mxkMBgQFBQEHx/L5nK5HA0NDQ7rabVayOVy/m+5XI76+npotVooFAqH5T09PXjllVewefNmlJeXuxMaIiKC3NrubpLLg4c7BJdQnOKiOMU3WmIdLXHaGzBBlJeXY8uWLTbLYmNj4eXlZbPM/m/AMtZgvZwxBolEgt7eXsHl27dvxyOPPILExMRB7winsbENvb3M7e2HmlweDJ2udbjDGBDFKS6KU3yjJdaRHqdE4uW0Yz1ggsjNzUVubq7Nsu7ubmRkZMBsNsPb2xs6nc7mioATGRkJrVaLmJgYAIBer4dCoYBSqYRWq+XX45Zv374dMpkMBw8ehF6vBwD4+/vjqaeeGtTOjnSjIUaA4hQbxSm+0RLrSI6zv9jcKjFJpVKkpaWhrKwMBQUFOHToEDIzMx3Wmzt3LkpLS5GWloZz587B19cXUVFRyMzMxMGDB7FgwQLcunUL1dXVSE1NtRnH4GYwDSY5AEBYWKA7u3RXjYYyGEBxio3iFN9oiXW0xGnPrQQBAJs2bcKGDRuwc+dOqFQqbNu2DQCwf/9+aLVaFBUVYfXq1di4cSPy8/Mhk8mwdetWAEBOTg4qKyv5AeySkhL4+fmJsDuEEELE4sUYG7kFe0IIIcOGvklNCCFEECUIQgghgihBEEIIEUQJghBCiCBKEIQQQgRRgiCEECKIEgQhhBBBlCBE5ul9Mnbs2IHFixcjOzsbhw4dGpFxcnp6erB8+XJ8+OGHQxanp7EajUYUFRWhoKAABQUF+OSTT0SP78iRI8jLy8Ojjz6KvXv3Ojx+5coVFBYWIjs7G6+88gr/0/au7NdIiPP8+fN47LHHsGjRIqxZswa3b98ekXFyqqqqkJKSMqQxehKnVqvF008/jcWLF2PFihW4devWkMfqNkZE9fTTT7OPP/6YMcbYO++8w7Zu3eqwznvvvcf++Mc/MsYY++abb9iyZcsYY4wdOnSIrVy5knV1dTGtVsseeOAB1tLSMuLi5Lz99tvsvvvuYwcPHhySGMWIddu2beyNN95gjDGm1+vZrFmzmE6nEy22+vp6lpWVxQwGAzMajaygoIBdv37dZp38/Hx28eJFxhhjxcXFbO/evS7v10iIMysri125coUxxtiBAwfYs88+OyLjZIyx9vZ2tmLFCpacnDxkMXoa55o1a9i+ffsYY4zt27ePFRUVDWmsnqArCBF5ep+M8vJyPPnkk5DJZJDL5di3b9+Q/ASJp3ECwIULF3D16lVkZWWJHp+Ysd53331YvXo1ACAiIgKhoaH8D0GK4fTp07j//vsRGhqKgIDmwnn5AAAEAklEQVQAZGdn28R3+/ZtdHZ2Ytq0aTbxu7pfwx2nyWRCUVERJk6cCACYMGEC6urqRlycnDfeeANr1qwZsvg8jbOpqQlXr17FihUrAABLly7F888/P+TxuosShIg8vU9GTU0N1Go1fv3rX2PJkiWoqqqCTCYbcXG2tbVhy5Yt+NOf/iR6bGLHOmvWLERFRQEAysrKYDKZPPo5+YFeV6FQ2MQnFFdDQ4PL+zXcccpkMixatAgA0Nvbi3feeQfz588fcXECwGeffYbOzk7k5OQMWXyexnnz5k1ERUXhjTfewNKlS7Fu3TpIpdIhj9ddbv9Y33+7obhPhtlsxrVr17Br1y7o9Xo8/vjjmDx5MsaPHz+i4ty8eTOeeeYZjBkzxu247las1s/95z//Ge+99x7fKItB6N4m1n87e9x+PUB4v4Y7To7JZMKGDRvQ09ODZ555ZsTFqdPpsHPnTuzevXvIYhMjzp6eHlRVVeG5555DcXExDhw4gA0bNmDPnj13Je7BogThpqG4T8aYMWOQk5MDqVQKlUqFe++9F1VVVR4lCLHjlMvl+Oqrr/DDDz9g+/btqKurw5kzZ+Dj48OXeEZKrNx6e/bswa5du7Br1y5MmDDBoxjtKZVKnDt3jv/bPj6lUgmdTsf/zcUVHh6O1tbWAfdruOMELAP9a9euRWhoKHbu3DmkPV534zx58iSam5uxatUq/rFFixZh7969CAoS/6e23Y1TLpcjMDCQL80uWLAAr7/+uujxiYVKTCKyvk8GgAHvkwHA5j4ZWVlZKC8vB2MMBoMBlZWVmDRp0oiKMzo6GqdOnUJpaSlKS0vx8MMPY926dR4nh6GINSoqCp9++il2796N/fv3i54cAODBBx/EV199haamJnR0dOD48eM28UVHR8PX1xfnz58HAJSWliIzM9Pl/RruOAFg/fr1iI2Nxdtvvz0kJU8x4ly2bBk+/fRT/rzkHhuK5OBJnDExMVAqlaioqAAAnDhxAlOmTBmSGEVx98fFf9lu3brFnnjiCZabm8uefPJJ1tzczBizzFZ4++23GWOMdXZ2sj/84Q8sLy+PLV68mF2+fJkxxpjJZGIlJSUsLy+PZWdns/fff39ExmntpZdeGvJZTJ7EWlBQwGbNmsUWLlzI/1dZWSlqfIcPH2b5+fns0UcfZX//+98ZY4w99dRT/OtcuXKFLV26lGVnZ7MXXniBdXV19btfQ8WdOL///nuWnJzM8vLy+PfvqaeeGnFx2hvqWUyexKlWq9kTTzzB8vPz2fLly9mNGzeGPFZ30f0gCCGECKISEyGEEEGUIAghhAiiBEEIIUQQJQhCCCGCKEEQQggRRAmCEEKIIEoQhBBCBFGCIIQQIuj/A9/WD/wi1q7SAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(net[2].weight.data.reshape(-1), net[2].weight.grad.reshape(-1), s=1);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the normalization of the loss function feeds directly into these gradients, so needs to be considered when setting the learning rate:" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0: IN (500, 100) OUT (500, 200)\n", "1: IN (500, 200) OUT (500, 200)\n", "2: IN (500, 200) OUT (500, 100)\n", "3: IN (500, 100) OUT (500, 100)\n", "4: IN (500, 100) OUT (500, 10)\n", "4: GRAD (500, 10)\n", "3: GRAD (500, 100)\n", "2: GRAD (500, 100)\n", "1: GRAD (500, 200)\n", "0: GRAD (500, 200)\n" ] } ], "source": [ "Xout = net(Xin)\n", "loss = 100 * torch.mean(Xout ** 2)\n", "loss.backward()" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD7CAYAAABnoJM0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9aXBc53nv+et9R6MbaKBBrMRGElxEkRS1UbR2WpZkK0p0HUdxYsfWJKmpSt1MEldq6lbFTqpm5kuSO5VyUrmJk0kcOd612JJsKhQtipREiqS4gCBIEPvS3eh937vnw+lz2N1orAS4SOdfxSKA7nPOc97l2Z/nVRSLxSIyZMiQIeNTC+WtJkCGDBkyZNxayIJAhgwZMj7lkAWBDBkyZHzKIQsCGTJkyPiUQxYEMmTIkPEphywIZMiQIeNTDlkQyJAhQ8anHOpbTcBaEQzGKRRuzxKIhgYzfn/sVpOxItwptMp0ri9kOtcXtzudSqUCm8206Od3rCAoFIq3rSAAbmvaqnGn0CrTub6Q6Vxf3Cl01oLsGpIhQ4aMTzlkQSBDhgwZn3LIgkCGDBkyPuWQBYEMGTJkfMohCwIZMmTI+JRDFgQyZMiQ8SmHLAhkyJAh41MOWRDIkCFDRg1EExneOjlJNJG51aRsOGRBIEOGDBk1cPyiix8dHeX4RdetJmXDccdWFsuQIUPGRuLAzpaK/z/JkAWBDBkyZNSAxajlqXs7bzUZNwWya0iGDBkyPuWQBYEMGTJkfMohCwIZMmTIuAW4nbKSZEEgQ4YMGbcAt1NWkhwsliFDhox1QDSR4fhFFwd2tmAxapf9/u2UlSRbBDJkyJCxDlithi9mJa1EaGw0ZEEgQ8YSuJ38uDJubxzY2cILj/SsWsO/HdaYLAhkbDhuh4W+VtxOftzFcKPjeyfPz+2EtWr4t8MakwWBjA3H7bDQ14q1anlrwVoZ8o2O7508P7VwJwm2aCJDOpPn8w923dJYgRwslrHhuJ2CYqvFzawuFRkysKpn3uj43snzUwtrHcdbgeMXXbx+YoIXHum5pbECWRDI2HB8mkr1bwRrZcg3Or6ftPm5kwTb7ULrDbmGfvazn/G5z32OJ598kpdffnnR733jG9/gpz/9qfT73NwcL774Ip/97Gf5wz/8Q+Lx+I2QIUPGJwK3UxbJRmKjXTd30jjeLrSuWRB4PB7+9m//lu9973u8+uqr/OAHP+DatWsLvvMHf/AH/PKXv6z4+7e+9S1+67d+i1/84hfs2LGDv//7v18rGTJkyLjD8EmLSXwSsGZB8P7773PfffdRX1+P0Wjk0KFD/OIXv6j4zs9+9jMee+wxnnrqKelv2WyWjz76iEOHDgHw/PPPL7hOhgwZn1zczAD87Y7bJbC9ZkEwPz+Pw+GQfm9qasLj8VR85+tf/zovvPBCxd+CwSBmsxm1WghPOByOBdfJkCHjkwuLUcuBnS0cv+jaUAZ4uzDZpXC7WEdrDhYXCgUUCoX0e7FYrPh9MdT63kquq0ZDg3nV19xMOByWW03CinGn0CrTub64lXS+d3SEHx0dxWzS8fwjfUt+d610ruYZ64G10PmFh/swm3Q8dk8HVrNuA6haGdYsCJxOJ6dPn5Z+93q9NDU1LXud3W4nGo2Sz+dRqVQrvq4afn+MQqG46utuBhwOC15v9FaTsSKslNbV9lFZb9wpYyrTuTLs7rbzwiM97O62S3TUWmM3QmetZ9woFtsHN0LnQzucZJIZvMmNs1yUSsWSyvOaXUMPPPAAH3zwAYFAgGQyyeHDhzl48OCy12k0Gvbt28ebb74JwKuvvrqi62RsLKKJDK++N8ar743VNKVvFxNWxicDtbJl1nuNrTQjZzUupE/qPlizRdDc3Mwf//Ef8zu/8ztks1l+4zd+g127dvHSSy/xR3/0R+zcuXPRa//iL/6CP//zP+cf/uEfaGlp4W/+5m/WSoaMdYJY2AKg06oW5JXfLvnOMjYOt9rqu1VrbDUFaDebxps1J4pisXh7+leWgewaWh+ItEYTGY6cmQHgsb1ttzyvuRp3ypjeyXS+dXKSHx0d5YVHem6bArObMZ7rsfY3ysW6XnOynGtIriyWAQhm9HMPdd9qMmRsEFbCgJbSdm+1tbCRtFiMWnRaFT86OlrTGl5PrLb9xc2yQGRBIEPGpwAiA7oyFeRrTw/gqPGdpVpN3E79ezaClpvFcFf7nJvV/kMWBDJkfApwYGcLV6aCXBgNcPyii+7OhlVfX/7/rcRG0HKzGO5Sz7mVVpfchlqGjE8BLEYtX3t6QEin7G3kp0dHVlVodTN64qw0e+d26c+z3riVGUmyIJCxZtwJlZsyrkNkoOeu+fjXnw/ddimQn9TUzJXiVrbekAXBHYLbkeneyo17O47HnYIDO1v46jMDt4ThLDVvn/QeRMut2Vtp6ciC4A7B7agt3cqNezuOx6cRqxXIS83b7ejyWS+FI5rI8J03hvjR0VG+88bQbafAyILgDsHtqC3dyo17O47HnYLjF13r5hpajLEvxkA3at42ykJcL4Xj+EUXF0YDOO0GKWB/O0HOGrpDcKecIrUemQ8rucedMh63Iw7sbMFs0rG7274u9yr/X8RiKZ43Mm9LrYuNSm9drwwl8frdvY2cu+Zb8n63IntIFgQy1hU3uiFFE/rCaGDN97gTcCtTBS1GLc8/0rcuFbuLMfaNSPFcam3t7m3kylSQ3b2NNa9d63jfqMJRq2q5pcG05DXVNR83Y33IriEZ64obNf1FE3pXj/2OdfusxE1xK2IcNzPAXstteKPPX2ptnbvm48JogHPXfDWfc6tiSmIPr9dPTKzKfbarx86F0cBNiyfIFoGMdcWNalDlmuTtFDRcDVZiFd2KAq1yuhYrKNtIS0V8fjqTR6dVrZt2Hk1kSGfyfP7BLunAm+rxX8t43+hYiHQd2t+OTqNa0n12YGdLheXwtacHJMv4+EXXhlvGqm9+85vf3NAnbBCSyQy3a7s8k0lHokqKRxMZ3jk7g9NuRKdR3SLKFqIWrbcSOo2Kvrb6BWN0u9G5GEwmHRadCotRw4GdLYvO9WLvuZFw2o0SXbZ6Y83xfOfsDD86OorFqKGvrV76+3LrdyXrW3x+Pl/klWPjC55RCyuZ93fOzvDKsXF29TawfXMDZoMGXzjJ43vbJQau06hw2o0cv+iqSWM5/ZlsnnfOzjAyE66gc6l3rEWnSNeefgfPPrC54ppoIsPITJgtHfU8vLuV4xddvHJsnCvTISxGDds3N7B9s33ZdbRSKBQKjEsIM9kiuElYD9/57dL0S8bSuF0D2Suha7XB35V8Xr52n7q3k2giI1kE64FqmkU30ZYOX4U/fikayz8D+NHRUT7/YFeFK2qtDeN29zby1snJir0ruoxeeKRHOroznclXXHcz15EsCG4SbtQVcDs1/ZLxycVag79LfX7kzAyvn5ggncnz3EPd687gqu+3WOB4KRprfVatdJV/ZzWZbWIraVjcVXWru//KweKbhBvNuZfz5mvjTqgwvhNoXA7Lrd9bWVNSPb7lgeOV0lj+maihH7/owuWPS/cu/0558DmayCzZu6l67y4lRG7VWpFjBBuAjfBnb5RP+Wb53mv5YFcTL1mMzsV82iuhYyP884v5iqtpvJkxo1pj37XJSj6XXxMtK/l+NJHhrQ8nGZ4Kcu9AM2aDBoBWh2nJa6rvu9IYgTi+Truxwve+lrVWfk9fOMnRs3ML1ld5vOX4RRcvH77KlDvKfCi54B2r9+5Sa3a163mlWC5GIAuCDcBGMtf1ZiDltG4kcypf4FPzsVUv9sXGtHxDLkWz+G7VAcD1Ri06a9G4Xht+JXNWa+ytZi2dTeY10bKS74uB0ivTIUkIvH5iAq1aydZO25L3nfPF2L7Zjk6jwmTS4fHFlnxHcXx39zby3cNXOHbeJQWO1zLO5YHcQ/s7aLDqObCzpUKoWIxaibk77UZ8kRTnr/mlYO9SgfbF1mx1AHk996AsCG4BNlIQrLfGUE7rjdx7OYZUvvg7mi2rzoZYbExXaimJ77alo55dvQ3rkomxUjpr0bhSAbYcVmJt1Br7px7sliyC1dJSzngXy8Jx2o1MeSJ4Qym2dAh0XZkOsaWjvkIQlNPa0WxhzhfjwmhAeh+TScfPjo8tui7L3Synhuc5enaOXT12Ht/bzvGLLnb3NkqMfKXjXJ6FdHefQ5q7xfaHTqPiwbvbyGVzNZl49XXl66H8/cXMIVGIrSdkQXALsJGCYL0YiIhyWm/k3ssJkfLFvxY3142OqfhuD+9uZfvmhiXTB29kXBejs/r+6+XqW4m1UWvs7bbr6aOrpUX8vugnr6UBH7/o4sl7BG1635YmZr1xtnTU89jetornvPXhJK8cG0erVnJXb+OClMnl0nHL31XwuWt47kA3p4bn+dHRURqsep66t3PZd6sWSLWet9T+sNuMdDSa2Nppkz4T77mUMKpF/0YoKbIguAXYSEGw3rGCclpv5N7rLaCqsZoxrcXUl3u39bK0FnNlbJTvt9Z7KRQwMhPi8X1t2Ov0i9K51Hiupi5AnPNq95vIhJfSdIenghWWQvX7mEw6ctn8ojUA5TRUu2tWsh5d/jjfeWMITyDB6ycmpBz+WmtlqTVUazzLhRzAuCuyIH6wGP3rDVkQ3ALcKcVPcOMMVsRGF0iths61MN31EmSLuTJWG8u4Ecvk3385zOhclHA8zX3bnTXvmyvCz0+MLfqc6jFciXBdzP221Lu3OkySpbYUg13KLbNapl0eyD5yZpqhiRBKJTy6t21F87/SoLYo5CLxDGdHfFyZDlXEP5ajcz0hC4IlsFHB0U+qIFgpg92IcV0NnWth6uu1IRdzZSxX2SpiPSyHTqcFXzjJMw90cfKyR3pu+X1/dW6Wlw9fXfQ51WO4EroWc78tNbbLjbs476ud08XWoNjU8Nh5F1emQzRa9fgjae7Z2sRzD3Wv6N61xqJW0sW+LU34wknGXFEGOm046g0Mjgel7KbF6NsIniQLgiWwUeb6emnZNwMbwWA3YlxXQ+fN0rJqQXRl1Hr+apjpjVomwWia9y+6effcXE3/s7PRzLQ7XNGGoRzVgquj2YJWrSSfLy6aArracV/J2hfnXby3mLljNmgk2mqliC421u+cneHo2TkGOm3cv8PJ0/d30WDVc+9A87LtJ8Rn1vL5l7sEy11jzx3oxmLU8PzBbvZtbZLm4MiZGcltVB48l9NHV4n1EAQb5dfeaDcG3JgAKb9WqVZVuAjWw/2zmO/4RoTdzbSyboTepVwuK1lv1QxvNTSU++hfPzGBN5xiV4+9xIwq/c8fXHLzy5NTNFj1S6aAimtz++YGxt2Rmqm31eO11PhVZ8kst/arx7M6v3/OFyMYTS+gq7zfkPgu5YHg5w92c1ep8nhqPsakO7rgHtUxD/GZtQLQ5S7BcteYOO6ZbL6iiKw8NtLqMF0fL61q2fjOWiALgiWw3pqjuHDKi3WWw1qF0Y1oDuXXznrjkovAaTfynTeGahbQLIXqjZ/J5pmaj9HRbFmxW2E5rKcgWI7R3wi9b52c4odHRmrmy69mva2FhnIfvfjv1z/TU1Pj7+u0o1IUl1x31WtzsbVaTetStJcHUB/e3brs2hddWFq1knF3RNLGH9/bji+cLPUUWpgSfPyiS2La5XUrYiC4urFc9T1EF9LRs3PSZ4/vba+wBMrXka3eiKJQwBdOcmh/B3f3OZYco/LYSLl18P6gi3FXDH8kyQM71q+LgNx07iZC1HDMJh0P7XAu+d3qRly14PLH+cE7I3zx0b4Fh1nU6o2y0sZ05dc2NJiJxdNSheRazgKo7oNU/fuN9lmKJjK8d3SE3d32Jd9rrfRWY7lDTpaCQlFbO1lsbhb7+2LzK7YqvnegWTrpSrxuuRbe5c/qdliW7fdT3cNnpX2IVjrfK+k5tH/AyZnLbtLZPK+fmACQGtd1OevoctZJB74sRRNUNn8T10B5Y7nye5TvBXGszQZNBb3Vbb0Xa3a3GD3pTJ63PpxkzBWW/ua0GRmaCOG0GZccl/XGDVkEP/vZz/jjP/5j/u3f/g2lUsmuXbsqPr98+TK///u/z7/8y78wPDzMZz7zGZRKJa+88govvfQSr732Gt///vdxu93cf//9q3r27Zg1JGpM5cU6i2ElGp/Yj9wXTkrZHyJqaZcr1SLLr7XbjGyyGSo0PtGVIGI1xWK1NMdatK7G/fLO2ZklA5urxXJWWLk2uVhQbzFs625EpSguaG+wmBtkNdkw5dW6QxMBTl2er8hCWU2K7F39TRUW1mrmY6maiOWUkeWyhKqf8/+9dZmPr/rY3mWv0NjLi76qU1KraRDX5HcPX5HcSfdvdxKOpxdo7+W5/2aDhnqzjllvvKZLrLqt91L1DtVjJAasr81G8EfS2C06Zubj2Oq0TM3H2dnTsGgF9lqwYa4hj8fDn/7pn/LDH/6QF198kW9961vcc8892O3Xz0H96le/yre+9S3+7M/+jCNHjuD1etm5cyc//vGPeeGFF/iLv/gLvvSlL61aCMDtJQjExdPRbGH75oaKYp3FsBKXkJj98cVH+1akCa/GzVTLjbUYI1lNsVj577X83GLq3tsfTUuBzOWYu9NupMVhZv/WpnVx4y3HMKv7yKzGRWO3CQzh+EVXRTuLxYqFVjNnTrsRrVqJSqlg3B3FaTcwOhetSVt5iqQY3C33mzc3mkkkMtL33jo5wXvn3WjVykqfdYkmMd++02mRirXmfDE6nZaKIOtq18piEJnlx1d97Oqx8+uf6anIRBLHQgxelweRyxm+KCTFILE4ZuF4mgujAdyBOPPB6/2BRPobrHrUKmVNt1GtdwnGMvyv1y6W3EeGBe9SHmwemQlz7LyLgU4be7c40KpVTM3HSKRz+MJpnrqvY0Hh3Y1iwwTB22+/jVKp5Omnn0aj0RAIBLh27Rr79+8HYHZ2lh//+Md84xvfAMBqtfLyyy/za7/2a3z7299mbGyMf/7nf+bs2bPce++96PWrC4zcToKgevGvxJ+9kg1hMWq5b7tzRUJgtecViDRrNSquTgdXVTi0UtRiCqImVx7ILBcYtbJCLEYt+7a3kMvWtrLWO/OqPGOmVobIcgeUlAcNt3TUk88X6d5UV7OieTXWkk6jYmunjZ09DViMGr5wYPOSFavlB52IlcCipSNaBOL3/JE0AJtbLHiCyQXzJlqnQxMBvnBgM25/gsHxIG5/okKgl6+V1Qa8qwPJR8/OsW9bE7/xmR5ODc8vWBPlwWsxDjA0EWB0bqGQFOkSx0yMMQyOByvGqFzAbOu01bQaauF/vT7Ix1d9uP0JwqV01+oMJjHYLK4Le52eJ/d3sG9rEyMzIYLRNNl84Za0mFhzjGB+fh6HwyH93tTUxIULFxb93OFw4PF4pJ9/7/d+jz179vA3f/M3/OVf/iV//dd/vVZSbjlu1Ae+HljroRkKRXHZg7LX2j++elyqj+57bG8bQMVh9YBEz0oOsBc1x5Uedr9SgbnSQ0zEz0T/vcmklWIL5RaFTqta8Rgud6Rj+XwsdhD6gZ2VB51EExki8QwDnbaK2If4veGpAFenI0x5Yrz4ZH/FdUfOzNBQp6fJpscdSHLumo/eNitDk0Ham81s764dU6p+j929jRVxjeq5KD+0/YuP9pHO5DGZtJwc8vD6iYkFa6J6fYmf7+qx88VH+zh3zcfu3kZefW8MWHh4/NeeHpBiLmL8oK/NyqnLHtyBJBPuSOl5I3zt6QHpncrnQ3yHLz7WTyaTo6XBtGA/ifTt7m2kyynwwEw2zy9OTQPw3EPd/NGv75JoWUvc70axZkFQKBRQKBTS78ViseL3pT7/9re/Lf3961//Ok888cSqn9/QYF4L2QsQjqU58tEUj93TgdWsW9M9HCw8A9bhsNxUOr7wcB9mk27B9eX3BaSfux0WujsbCMfSjEyHOH15nnNjAZ5/pG/daK4el7d/eZnXT0zwpSf7+a1D2wjH0vzP75/lwmiA3X2NqDUqDu5uI1eAkekgv/aZHr7wcJ9071pj+t7RES6MBti3rUn67lJ0vXd0RAroL/Wui43nYp8d/sWwFMz80pP9mE06GhrMC767knkWr0llcvzn4avL0lrrnmlPlMn5OD2tdTQ0mHnjxBi/LDGea64ou7Y6cTgs0hyJc3H68jzXXFG+8HAfRz6aIpXJSe/1m09swaBTSWupwWasWFf7B5x89+2rnL48j9mkW/Ae4+4opy/Po1KrMehUJNP5irn4wsN90ne02nH62uv5z8NX+dKT/Xz1mQH2Dzg5NeSW3rOc9p8fH6e33cZAt4NnDmzGataxa6uTnx4dkehvsBkrxrF8ff60tC7amky4A0msJg1ffmqAHxwR3ue7b1+lt83Gj46OMu6O8t9/cw9Ws05aT2qNir3bnAx02bk0ERDOQyjtJwcCvzry0RQmk5b/PHyVlkYhGJwrwt+/NsjXnt3BS8/ftWBuD/9imB8dHUWlVvPiZ7cuugZuFGsWBE6nk9OnT0u/e71empqaKj73er3S7z6fj6amJqLRKD/5yU/4yle+AggCQqVavTnv98coFG7MN1SuTcbi6RVrbMtJaYfDgtcbXRUt4ilG1XSsRiN4aIeTTDKDN3ndLVV+33RGyLzwBxPSaUgOh4UvP9HPZqeF3d32VdG9GM21EE1kOHtZ0IaCoSReb5S3Tk5y+vI8TrsBp83Afx6+Si6bZ3QmyOBYAKUC6X20Bi2v/WpkwTjs7rZLWR/id199b2zBe1Z/v7fFwr//fHDJca01nuJ87O5tlOgBuDAivNtdfY3E4xn+8/BV/MGEpNGL91npPD+0w0k0kSGXzS87L7Xu+Q8/OceF0QDnR7yoFBCJC+/Q1Wzi5EUX+wec6KqOpSpfBz88LAi2R/e0MtBpo73ZzP3bHFiMWvz+WMUYiOvqzGW3pJHv7raTSWZ4aIcTlz/Orh47T+5rY7PTQiKR5vtvT3Bofzu7euz0tlzfL19+op9EIsvpy/M0WnR89ZmBimyx3d32inE/ftElPR/ghUd6KuZsd7edzz/YJf0sPqd8vAH8wQSH9rcTT2YIRtOE41lOXXLx5ScETf/05Xk22Y3s6rFz+vI8/8ffvsvd/Y0cvGsTLzzSQ7Go4F9/PsSuHjsz83Ga6g3MzUcZm/RjMWqlNfnZ0juL4zQxG2ZoMkgmc67CQhGtl0RCcNklEulV85RyKJWKJZXnNQuCBx54gL/7u78jEAhgMBg4fPgwf/VXfyV93traik6n48yZM+zdu5fXXnuNgwcPYjQa+ed//mfuvvtu7rrrLv7jP/5jTRbBemAl6ZK1GPFGHBu5WLrgcm6P5QRFuVn68ttXaj57vVw/S+H4RRdXpoU0OW3Jd3pgZ4tkzms1KomhC64LIW1WxJGPpmq6Y5Z690w2v+Cs2OrjA8vdL7FkdtF03fL3qHZdAQxNhNjVY+fPfnsffn8MnVZFOpNf9ohC6f2qjnMsp3U51LrnFx/tI5e7SnuzmQM7W3jrw0kAvOE0cU+c7/xskC8/0V8xfrWe5w7EGZoMsb2MGYu0Xhr3MzQRYqCrns8/2MW9A81s6RDcMeUMTUyrBCQXi06rwh8S6gBaGuakubYYtZLbSatR8fwjfXi9UWmuy5m++POh/e0Ss68eV4tRy2N72zh+0UUsmZUE2A/eGamYv9dPTDDQaWNoMghQwRPK01QB/u//OIM7kOSXp6apMwljpjVoyedytDaaGJ+LMB8SPtdpVBXKSLHqfuVrTjzHGGDCHeFrTw8IQeN1PN95Maw5WGw2m7Hb7fyP//E/+N73vsdzzz3H5z73OV566SU2b95Mc3Mze/bs4S//8i/5l3/5F2w2G3/yJ3+CRqNh165d/NVf/RX/9m//Rjab5Zvf/CY63ercIetZWVydLlmOWgHPpQ6WKM/EWWmF5WKpf2KmgxhUrRWsKi/QqZVuJt73+EUX755zSRkY5aXxKzmYZqUdPRc7iayj2QLFIlqNkqfuE5jN8YsuqUinvD9NrSB5eQFUIJLiH1+7xMkhD+9dcC3IULGatbj9CfzRJO+dd9fMYBHnMJ8vSgHHd8/NLkjXXay3f3lxkdh+od6sY3tPI2qFcP9xV2RBf/rqYjvxGW9/NI03nFpYaVpKN3zrw0nOj/q4NhNe9gQsEBjgAztb2FEa08HxAKOzEbK5Ak31Bnb2NjJaltVUnSYrpnk+uLOFcDzN/dudUt+iK1NBrs1G2NktBK0Hx4PotEruG3DS0WyRDocRg7C7exulIG6xWOTY+Tke39vOB4MevOEUOrWKB8oYXXmKqa3eWNF0TqVUcP8OJw/vbmXcFeHKdAidWsULj/TSZDPw3cNX6HRapL0xPBVk3BWpqAweHPczNhdloKue5w/2SNXGeq2K0dkIA502Xnp2O4CU5qnTKtnTL1hEA1123P4Ee7Y4pNYUrQ4L43NhPhz0MOtLYNKryeYKFItFfOEUO7rthONpLEYtb344JQWEy9d6eVaY2JNosU6oq8WGFpQ9++yzPPvssxV/+6d/+ifp561bt/LjH/94wXX79u3jlVdeuZFHrwtWonHV0rYWu666oKxce6wOxNbSABd7tngAyErcQ8sVKC11r8UsndUEZMvvAVTcT6tRMTQR4uSQB51WtSKrSnyfLzzcJ33vO28M1dTcRJy75lvweXkxlmh2i4VJosbV12ZlPig0axOf/Y+vXWJoMljz4PWWBpNEHwhaZYPNKM396ycmeOGRysre8vERe85cmxHcA7t67Dy2t23B2ijXFEUsd9B5tetj2hMDYKDTRm+blZ/+alRyU4hrotpCEi0ncd7F/0WLzmwQsnDEtfGdN4bY1GDiwmiA/jYrW0uB6R+8M4I7kGRXj51pT6w0NyO8+GS/pA2Xo3p/iUkG/SVLQZzDx/a2MTwZZGgyyLdfuYhaqZTuvaXDJo3ZZ/e3099mJZ7M0d9u5WrJMu1trZfmRlwLdabrwV3xvZx2AxdGA9L6yWTz9LZZK+bq6kyY8yM+uppNmAxqfvuJft4772JoMsjVmbA0x13OukXPHhcPsC93P1ZbtBuFT21l8Up976txm4iT+9g9HWSSmQrXx/GLrjW7kQ+ysy0AACAASURBVMSsCVjINEVzFa6/Uy0GW+0SEYWTo+xei7ktVlpxXJ2ZIh5RWH3NtdkQLz6xBWDRxV7tCiiv1i53e3zuvs4FGShiJkw6m69oOSCO4bWZsLSRy+f3+EUX7kCSkZkwva2CFSUynvQiqavVFar7B5y8dXKyInOoHOVjXE7Trh67pCzEShaa+L+U2VNiKitB+TpIZ/IMTQbpb7PS0Wwmnc3zm09sIZFIS5WwUmZXPCO4fcYC/P4XtlcoEOXfK89m+trTAxLTzOULAKjVCsklU57Jc+z8nDSHLQ0m/vsLuxfMueime/nwVbZ02cnn8pLrBmBoMnh9P5XyUa5Ohzm0vx21WsEXH+3DbNBIWVPpbF4aN9GNBbX3jvhOogAU6T455OHabIihiZB0jU57XUtvc5jx+ONMeOIAHP14FrXyehCmPLtKXK/iOqmuEBex1L5fb3xqew0tVvhSqxBnKZS7DsQGU2JBmU6jWnDikoiVVFgu1t+9HDqNqiKferHCJRFOu3CE4OB4kGKxyIw3ht2srdkjqPqEpaVcaCK9YrOzBqt+gVnb6jAx54sxOB7EF05WnCRV3eyrvM/Llo56FEolTfV66V6pbL6itUB1g7StnTam52MV46JVKykWiwxPhxacLVt9Xmwmm+fjER+eQIJsvsj2LntN11t16+UPLrl5+fDVBY3JxHU17oqUeuxoMRs0kouhvGjw8OkpvKEUBr2KB3a0SPUDd/U2Lrpmql1Y5cVjs944V6ZDNNuMfDDkYXQ2ws7eBuKJDFqNkifv6UCrEYqaookME+4o3nCqwjUBSGtDdGMcOTPD8FSQJpuBeFI4pvGp+zql/Pw5X4zH97ZjNmgw6NQcv+DiwyEPLY1G7htwLngHMaA6Nhvm0niAockgw5NBaf0/eU87WrVSotli1DLrizM6G6G/3cqLT/TzmdLYZrJ55vxxHt7dypQnxpXpEE02PS89u517tjZLh+BUN5arrjt47kA3DVYD4+4I755z0d9mFU5d29rEY3vb6N5Uh8WoQaFUcnp4ngaLjmQmT0OdnuHSMw/sauHgXZtwBxJ0NFvIZPPS+hbdVVOeCDPzMQbHA7z90TTHzrvoarGwp9+x6vqdWpCbzi0CcaKrz12tVYhTDZc/zj++PsjMfIwpj8Bsqg/cXu7Ur+UKykTG1NViQa1SSsxjqXcRtYql+t7rNCrmg0muTIeIxrOcKPOxi8x0yh1lPpSUfKtufwK9VsWUJ7akcBR9nNV+8XImeGh/h9QsTCzcqf5+eRXo85/pwRdO8cMjI9IYl1f7ir7tWoVf1eOytdOGL5ziynRI8g+L333rw0lePzHBlo567upt5J2zM7zxwSTZfBGTXs2XHr/OqMurbBushop5FGMZ1etKvL+4rsSWB4PjQfb0OyoKiMwGDZfG/bzwSC/N9us9ZwKRFO+em6W/vX7BWqieu1lvnGPnXVL8xWLUcHdfI2NzEfZva0KtVvHz9yfwhlL4wkmpg6dWrcQfSdOzyYK9To/VrF1QJS2uFXGfuP0Jjl90s6Wjnnu2NrN9s106e9jtTzAfSnDqshdvOFVR6OW0GyuUrsMfTeOPpPFH0tjqdDTU6Tl4dxuP3t0q+dOn5mO8e86F2aBh3B3h3oFmzAYNjVYD3ZvqFhRxiYLTYtTw209socFqqNkBtUiBZpuRJ+9plwSJKPgy2by0F2OJLMNTIUx6NXv6BXt6aj5GoQjDk0EO7t7Enn4HD93VwshMCE8wxZ5+B+5AQqJnaj4mxf6eeaCLK1MhJtwxrs1GGC21nADQqVU8f3Bp5WulkAXBIigPopZrpLWYWbW29Z03hhgcFwJmWzrqpeCOGLBd7vSnlUDcaCa9mmPnFwZEa71LrU1Q6xpRMx9zRdm3rYln7++SFrtKqWBo8nqLXJ1WyeB4kNHZyJLCUTStH97dyl29jYv2yCmv6jTo1FLgrKPZUhFYnvPFGJ2LSszdF0nx8VXfAqtHnD/RwijfNLWEbavDhFatxF6nlxhHNJHh7ZIWrlLBzm6BnpHpUrVnriC1a3b54/w//3GG6fm4FFQuXx8tTRY22QwSXaLwEgObTfUGnntoM+9dcHHsvIst7VbqzbqKNgnHL7hw+ZNcHPNzd1+j9E6i+0VgrskKq9VpN0rW1pXpEF0tFnRqFXqtMAbbNzfwn0euMuaKYtSr8IaS+MIp6k1aJj0xNrdYuLvfgUmv5tpsBIfVwPuXPJwccnPumr/mqWPiPjEZNIzORqSxiyWznLniRaGASU9MYmxN9Qa+9sw2nA1Gdvc28q9vDnP8oltaV0admtHZCI11eqa9cR7c2YLdKqQWVwt3MchvNmiYcEc4dt7FiQtztDpMnB3x0tdmlYLc1a6X8v1xYGcLwxN+RudikiVbrhRp1UrePj3F8YtuovEME55YhTATK5rv3uJgW0c9O7rtfDDoJhBJc3EsIM2vWKV8/3YnLl9C6gx7csjD2REfAD2b6nBY9WzrspHK5JhwxxacaLZWyIJgGdRqkLa100arwyRpdNWbur+9Hncgzt5+B0/u7yAcz1T0Fv9/f3SOX308R7FYpLOMwdWazOosm7c+nOTCqI90Ns/2LjuH9ncs2kag+vpamnCta8pdVl95ZgdqxXVm7ag3sGeLg+1dQuByT78DrVpJV4uF7V32RV1ZtYRPdQMvUbieGp7n6Nk5dBqlpFWJ1ofojti+2S5ZDN2b6nh0f6eUNVR9Nq2ofdbKfqkep45mC3P+eEW2jCjYLQY1s76EREMgstB6+MfXLjHrS6BRK/nq57Zir9NXMI1JdxStSsFI6YjCMVcUrVpJOpsnEsvgDaek2MmV6RDNdgPHL7ormEpDnXBqVjYntDXevtnOO2dnpEZpVrOW/zo9y5XpEOdGvDjqDfz02CjPPNCFzaJjc4tFCsqOzkakFs539TQy4Y7QZDNy/pofAINOSSpToL3JzJP3dDDlibGlo546k5ZrsxGyuSJOu4Hf+exWtm9ukIRVR7OFu3obaXWYmPLEBJfbVBitWsmJiy6GJkIk0nkGOm3s6LaTSOWYDyUlIfB3P7nAuDsq0WC36HlkTxsNVj2/drCbBquecCzDK++OkcnmmfXFMRs0nBqe58DOFro31aFVK7k2ExayltRKEpk8l8b9fDzil3oJheNpqe+Q2BdJtB539zZyanies1d95PJFNCoFX39moCI7LJ8vcvyiG4B4KsdAp42uFgtGnZon72mnt9WKxajh2YO9dDdb+PdfDnNhNEA0kSGRzlNv1vDh0DyeYIKhiRDDk0E+HvGxpaOe7k11UrbYQKeNrZ023r/kod6sZWwuisWgZno+vqQSuFLIgmARVDeKq2Ya5X35D+xsYcodva71d9gIxzLkC0VGZsIUCkUGuoR2td89fIXBMSG7QqdWkcrml9TOyxnoVMmnLZqIYkqeuAFrxS5qMeDypm+LxTvE74jxjHKNck+/g2cf2FwhGDubLYy7Ioy7IpK7oJbwEV0iCgX83U8ucOryPA1WPc8+sFnyy5Y/S7R4NrdYMOrV0olZ1bGPe3dskrqklqM6DlOrSVz5fJa7o/ZtaeJf3xxmcDxIU72BYCyDxaDmqfs6sdfppTjOk/d0cGp4HqfdyHwoyehshEKhiM2iY9wdobXRxIQ7Qp1JyyvHxvCFk5wY9BBP5djVY6ferOPND6dIpHMAbOmo596BZsGHv6+dRDrL43sFpqJVK9Hr1CgVYDVqyeUKnLnq5dh5F+F4mi8+2sebH0wRT+XQqJVE4lkujfuZno8Tjqf5ylPbGJkJc/yim/52Kw+WgruvHBsnkc4yOhclVLJyTAY1jnoDwWgGo05NKpuXmqypVELzOYNOxdef2S41Uitfb6IQPXbeJQmvIkWee6gbfzjFni0OvvR4H7O+OGdHfJJA/e7hK4zORaUUy1y+yIQ7itmgQa1S0ttqxV6n53tvXyWXLxKOZjhz1Sv1NZpyR9naaePERReD40K21X97tJcJd4QXHuklm8tLQrFQKOCLpJjxJnAH4rx7ziXFb8RzAIx6FelsgY4mM/fvcFbwhVaHSSh6VSrYt7WJJpuBX54SmHexWMQXSXFgZwstTRYSiYzUKLLJZmTWGyeVzpPLF1GgIJHOSWugSIGPrwrZbU31erpbrRSBvlYrZqMWnVrFrC+xZOr4aiALgkVQzUBrNbxy2g2lJlUG5kNJyeQ+dn5OaiE7Ohvh2myEXb0NTLqjHDvvoretDrVSyXMHN+MLpRb4wMtRrr2L+eitDhPxZJYxV5Qpd5Rpb4x3zsxWmNHVvvFyn7QoAN7+aLrimlqCSHRjdTRbSnnStRuGlfu4xQ1JscjUfKwiUC4y4pGZEO5AEqfdwItP9C8QQqLGb9Cp2d5lJ1co8v6gB61aychMmOGpIPu2NEnWkK3eKB0FWF2jUG0hVMd+jpyZkbo92uv0UixAZCQguIxUSgXBWAZPIMGHlzzMeGOolMqKNsSP7W2jWCyiU6swGTS8fmKCCXcEdyCJUafmmYe6ObirBYpFosksXU5LKVAYxxtKMdBVzwuP9EpWUSKd5cJoQAquj7sjvPHBJMFoBrVKyaw/gT+SltwRbn+CCU+URqueu3rtWAxaXnikl0Q6yxcf7SOWzPKTd0eJp3I8uLNFSLGcCqJTq/j8gc2E42lmvHFaGo3s39rEE/e04y/57nOloDgIqbAWo4YvPd7PuWs+aczLA+riPtnVY6ej2SK1VPaHU/S2WTm0X2g/8dbJSfyRNHv7Hezpb6LTacHtT9DWZCKZydNcr+fe7U6p26dWreSVY2OEYkKcrdlmoMNpxlFvkALZg2N+xlxRKduqo9nCY3vbuTjmF/ZugxG1SskvTk6TzQmMYm+/g7v7HVLB26XxAMFomnu2NWHWa+hutUqnu015hDoJgGPn5xiaEOICh/Z3MDgWIJ7K4QkkOD/qp1gssn9HCx5fjFPD8zy+t51fnJwmnsqRyxexGNSE4hlJ8Jn0atyBpBQz8QRTjLuijM5GCMUEl5Kj3sD9O5yLHiy0WsiCYBGUZ1VYjFpJMIiZDr5wUvJR97XVYzVr8YWT1Bm1nBj0SC1k25vMkpk4PicIhZZGExOuKIl0lmPnXQt84IsVA4na99hchLMjPhrrdEzOxxidjUjmo1hM82aJMWvUSinXvNqy8IZT9LTW4ajXYdJpKoqYRJQfYl6e5VMtKMWj9cR2A6OzESLxDB8NeyuE6eB4AJ1axRce2kwknqG/o77CT1/tonn9xAS7eoUNd6XkTjk74uPKdAizQSMxnK5NVt4+NVnxjj86Osq5ES8DXfYFKaTl2UiJVI7R2Qh7tjh4cGcLbn8Clz/B8FSIntY6NGoFU544D+1qwahXk84WuDoTZsIdlVx+on/cE0zwk1+NMutLSD3yH9/XRjie5vMHNuOLpBifizA1H2XSE5M03Rce6a2wLspP2jIbBBqHxgOks3lUCoWgXReL5PJF+tut/G+f306DVS8VPXU5LZy96qOl0cije9qkbJlvv3KBKU+cRquOrzy1TUpR9YZT2Cy6EmPRsLnVxqvvjeFsMNJkM/DGB1OMlhSah3e3Su62jy57+HjELx0J+fqJCVQqmJ2PMTITZt/WJr74aB99bfUUi8VS1pGQpaNVK3n7o2mGp4SUy0Akzc6eBloaTMyHkrxzdo5kOo9ep+b3PrdNysDJ54ucuerDbtFh1GtIZ/NMuGPs6mkgly8QjKaJp3I47QaefXAzPz02SmO9npOXPVJsQLSw0pkcsWSWu/uvW/2zXmHdBaNC7GJbp41ANMWpy/Pk8gXCsQzeUIpzI16iiSzvD3okQWw2aHDYDIRjGSKJrLCH1Sr2b3fyP7//Me+em+PCqF8Yb7MGo17Dzp4GnHaD0O660cRLnx/AqFejU6v40hN92Cw6vOEkyXSeRDqP025g3C20yr67zyELgqVwo4JA1GZEH6wYoLwwGsBsEAJXYpoawHcPX+HCaICeVmspi6Sf+7Y7mfPHeffcHO5AXMp4eOiuTezYbOf+7U784ZSUcVPrcItaEJmuSa8hkc5JJv4Lj/TSvamOI2dmOHXZQyKdJxzLEE9lK7TnjmYLY3Mh/JE0jnodV6cFAVXr2dVHFoqMuq/Nij+cQqWAMZeQndFg1fP8wW5aGkwMTQTwBK+3kg5EUvz1D85x/pqwCRLpLN2brLz54ZTko66Ot4hMMJ8vSq6SMVeURquORDpPkQLRRJZXjo1jNWvZXzr8e3dvI5PuKMFoGk8wKbX+FTNbxHuL4yGmD27vsuMLp3j33JzEBFoaDEy4hQDg5+7vJJMt4A0mCMYydDWb2NRo5qn7Orm7z0Emm+f/+u4ZYqkcWpWC5w524/IlmA8mpcDfT341ypXpEDu7G5gPJsnmCrQ2Gpn1xcnni4zPRYS6CIOGZx/YjNhO+fUTE5KF2VCnR4GCSCJLU72eP/nNu6XspM5SFeyDO52MzISkoKXZoOEfXx/E5U+QzhbQ69Q8vLsVe52ewTE/8VROyohy2o3M+BJ0t1h4eHerZImKmr7FqKXTaeHkkIdYyQXl8ifoarEQT2YZd8UYd0fxR9KY9Gr62+v51zeH8QYTzHgT9LcJ6zWTzfPBkOf6ns3kGRwLcO9AMxdGfUy4oxi0KkKxDFq1kjl/nAM7WzAZ1IzMhNjZbWdoMkgyk0enVvLk/nbOXJknnsph1CkJxrJcGPUx401wZSrER8PzUmxAtLDm/HFOD3uxGLR8MOThynSIYrEoubIGOm3Um7WcvDwPQDyZpVAEg1ZJKJYlGE2zucXClx7vx9lgJBRN819nZmmo0xFJZLFZdPS21vHG++MMTwoCT3T/tDaYmPMnmPLEmA8lSWcL5AsFXnxiC9PzMd49N4c/nMJep8dUigfoNSp2dNtIZQp4gkmGJgLs39Ysu4YWw3qkj5YHGcUApaiRvPGhkEUinnkqmsCNVoOUltfXVo/ZoGFoIlA6Z1Qw4f/bY/3MemO4fEJa3ehshE2NRupMGskCWQpWs5ahiQDzIYHR/v7nd3BXb6OUf3zsvItEOi+YnLGMlI0j5q1nsnk+Gp7HH0mzrcuGXqOiwarnoV0tHDs/x/BUUPLzb+1qoLvMUhAtgXMjXmZ9CcZLWrHbn+D5g0I16z/9fIhJT4yeTXX0t9fTZDPwtz88z3xIyEKpM2kZc0UlTTqREjpYFotFMtk8kUSGsZK1lc8Xef3EBLPeGO1NFnpbrSRSOYJRQaiqlAr2bHGgVquxmbW4Awkm3VFePzFBa6OJZpuR9maz5O7RaZUSI3jq3k4p5XJvv4MHdrYw6Y7S6jChUSnZt7WJZx8QeuuPu6N4ggneH/RIsYId3UIwUVQWjpyZYXBciP/kixCOp6VWCplsHq1GJeWrW81aRueEYGiBIqcuzwtaskaJN1TZSqKvzYo/ksSk1xCKZfBH0rQ0CL77u/sbCUbTmA0ajpyZkeoQzl3zceryPE31Bux1Oo5fcDE0GSKdFQq6kuk8FqMGdyDBR8Nemur1QhCyFOf5ya9GCcfT9LfXc+6aryLbK5rI8N3DV5jxCsVRhUJRWvtnrvow6VVkc0U0agVbO2y8d8HF8FSIYMmVs29rE4f2d3D4lJAOWm/SolJBNlcknspJ1oU/khYSAfa2EY5leOODScbnwlydDjHuimHQqdnd72DCHSGXL3Jp3E8wlsFpN3B3fxPjrijZvMAE7u5r4N7tzgrlwmrWcmk8QDiaIZLMkCoVmPkjaZptRu7f4eSFR3oZmQkzOhtBrVKQK93PVLIMk+k83pBgTSVTOU5cdJPLF0mkBIFBsSi4xMIpABqtetqaTPgjaVLZvHS/HZvtJFJZvvbMAM1244Isr3gyRzKTJ1coMjUf58CuFuLJLO5Acl0yh2RBsAgy2TzBaFoKGoqbbFunjfcHXezsbmBnd0OFxlRv1kk9Q0SGfvyii1OX5+lpsZDJFQhGMwSjKf7roxkptdQbTmHUqRkcD0r5z+WHbFT744+cmeHMVZ/U80R0e4jpg8JiM/LVz23DZtFJ73D8opBf/U8/G5IqKRPJHNPeuOS7FWMG5YUsbaUMKbNBw9XpEO5AglgyJ214QCowGhwXjkgUxjDH4HhQEloASiWE4hma6vU02Qw8treNazNhrs1GUCsVnLwsaHRNNj12i558oSjlTo/ORgjHMsz6BAbUWKdncj6GTqPk6NnZ67EJhODd1Hych/e08tjeNrRqoYrz0P4OzAYNyVSOkZkw7348x9BkCK1axeXJAMfOu0imc0x6YpKFILpbRC3RZFATS+aIp7J0Os1S35d8vsiV6ZCk8R7a38HJITfZXJH5YIIvPd6PJ5RkwhVlPpgkX+qOmykxhP42K18+tFUoSCoFqkWtUKBRSTyVo2eTmXBcCCya9RrB4izl6osa7fBkEIVCmBfR7aVSKDAbNTTW6di/rZkd3XbeO+eizqhhpkyoq1RCN0pPIMnlySCnh70V8R7RWjboVOTyRXo2Wfjyoa2MzUXQqVVSYDYSzzLhjqJQCIFQe6mYqq/VKvjVSxpyKpsHFBQKRZRKBe5AEoNeTTKd5/mD3WzpsPH9I1fJ5opSLMGoV/P5A5s5cmaGcCwrZQU57QZ+7+lteIJJVEoF9SYtwVgGs0HL8weF4q/hqSCvn5hgxhvj9BUvyUyeVCaPVqUgX7y+rooUcfkTnLvqJZHOU2fUksrmMeiUxJKC0NBrVXQ2C62pT1/xSoxdbHycK+uAbNKr+cZv3c3u3kZODrlJpK9Xoxt0KpwNRrZvtnPyskeKyZ294iWeEoSA3aJDr1XitBlw1AsWarnVdyOZQ7IgWARiuqRKqeDjES/vDwpm44lBN/PBFPFklrYmc8lvef00pLNXvcx4hVS2rZ02KdZQKII7kMRiUPO///pdGDRCyqDVrGV7l52n7uvEbNAwPBXkvfOuipOdRH+3Vq1keCrIpTFB89mzxSHlNIvH24lWgD+SxhNI8MXH+rirrFeML5yUNqBRryIcF/yY/e1WGuv1THlidDWb2dQoBKSvzYSZ8cYkmj4c8pDNFWis09NoNWC3aAlGhbYRT97TzqvvjRNP5VAoIFNKLXQHkmxpt6JQQDiexahTEYpnuTYbYXwujC+cIBjNUG/ScndfI+F4Bl9YYPwqhUKwDApFUpk8iXQOq0lDOltgZ7eNB3a2YNAKue17tgh57kMTIclHLAbzxfkRrYw3PpziynSolLqYpagQ2hA01evxBFM01RsIxzMcv+hGpVRwz9Ym6i06+lqtOO1GIiUa64waHtq1iYd3t0oui8/e28HguJ8dpQrmj696SWaEWoNgLIsnkJCEACAxj2a7gQd2tLB9c0OpCaDQbsGkV7PJYZRcVAathmmv0Mp4S2c9va1WvCFhDPvbrISiaa7OhEmk8zRa9dw70IRRr8Fi0nLmio9gNENvq5WzV70MjgfJ5ArSnLY0Grk6HUavVZNM52io0xJJZCkWhTTJsbkwaqWCRCpHuNS6WqFQ8MGgEFj3hlM02Qy0Osy0NhqJJrP4win6261s32yvCPz3t1nZ1dOAVq1Eo1IQTeZoaTCiUytxB5IUCkWuTgWZ9ESZ8yXRqhR85u5Wnnmgiz39Dl5++wqjs1Ga6vX83tPbGJsL4wmm8EeSfHhpvpSlJCgfc/4EU+4oM14hdiHEWAR3lNmgplAsks0XhTTNLpukfIy7oiTSeZQK4bvifBlLSlAuXyQYzUiBa4A6o7A+7XU6Guv0oIA6k5avPLWVwbEA75yZwRVIAqBVK8gXIJXOMutLcvrKvJC6Wyyyp7+JrhYLlyeD6LUqAtE0qUxB2jtXpkJ4gsl1yRySBcEiKDfNxIIXgGIRlEAslePKdIhL435++u4o9w40M+2JSQtiS0c9Wztt14/U29pENJ4taScaxufCvD/okdJAd2xukPKrAUx6FQd3t7Kt08Z751zs7LYzNR/l+EW3ZGJ3Npl59/ws755zSZWPYg57oSBoT2IGkRik/fyBzWhUCuLpLDs225maF7TrZpuRUCxNMJohncszMhORfJn1ZoHZ79nioLfVik6tQqdTCYFvu4mH97Ty5D3tvHz4KhOeKGqVgkJBeIff/exWEukszz3UzbQnjjecksx1EMzwYFR4n2Asg8WkYcItNEBTKhX4win8kTQdzWZpHgqFIoUiuAJJvnCgi962eiLJLA/taiGTLdDqMBGOCjn5GrUwtuXB/7E5wddu1Klob7IwOB4kXBrTNoeJerOWGW9cihP4I2mMejXvD3ow6tWcHJoHBRITUKlga4eNf3h1EE8wxeXJgJCO6E+gUCikFMDnHurmobvb+GjIRTxVGluTln3bHKTSQoGQxahBoYCf/GqUQqFIrlAkFM9gs+hothkZdwsus2abkc2bLPzXmVmiiQwz3oQwj3YDmxrNjLsEt1MinSORynF2xEcwkpaYmUqhwNlgwu1PkEjnMerVhOMZHFbB6kmW5j6ZyVEogMNqkGJcE+4o8VSOrmYT8XReclsA9GyyEIlnhdoHk0aio6FOz6nL8/gjSaKJLKGYsD+TqRyjrijRpCC4Xf6kpClr1EqSGUET16iURJJZ9m1toqPZwrd/epHhqbA0Z+OuiLRu9vY7UCmFgHoqkyebEyzE+VBSylyyGNSSEmQ1aYklhffds0U4crKrxQKForTXxBXb3mSizWFizpeUsnzUKgXlR5+ks0LqbSSeJZLIks4WSKRyREqKRTk/KbVeIif+X9obxWKBSXeUn747RjieIZnJ01RvKMU/1GTzBeKpHBaDmi8f2rJoW/SVQhYEiyAQSXHmipdMLk8mW2BTo5FMNke+AJaSRgoQiWfJ5opcGPUL3RPn4/RsMvPo3jb+/ZfD3L/dibPByGN728jmClyZDhGMphibi9KzqQ4FRSbcMWbmYwxNhiQmHopn0WmVDI0HSwGxHOOuGANd9dgsOvyRNAadWtLucjAW5AAAIABJREFU/ZE0dSYtDqsebyhFo1XPQ3e1SI29xMwQo17N4LgQyN3d56CrxSJlctgtOoLRjLQYbWYNqUyB3b2N3LNN6J3iCSZ595wg2HRqFe3NZh7b28ap4XlJg93cUlequC1K6Y9XpkJMeIR3dtTr2dZZTyItaPg9myzYLXocVj3PHeymUCww60tQKBRprNPT5bTw/Ge6sVl0qJQKPMEkCoUglK9MhZj0RPj4qo+RmRBnrvrwhZKESxkbFAT/+8cj8wxNCJlG12aFQHk2X8SgUwlFa0WBOU5742hUKuKpHCql8IyeFovksnl8b7uUVSTCG0px8rKHUEmgtTaYCMUFwalWKQUfbyqH2aAhGEvzuXsF95Q/kiIQTeMLJwnFslIe/d/95ALzoVSFW0FgwinMRjUuf5I6k5Ypj3CgeTyVo9NpRqdWMeGOsbO7gS0d9VL18IRHYMYis26qNzDQZePt0zPSXGdzBYx6FepS7QEImnS+5L1IZ/MYdCqa6vWolQKD1mpVRBNZlAqFxCjVSiWTnpjUjI1ikXg6S0ujiVlvnEQ6LylLyUyeUMmqMOrV/O5nBfdSPJWjsU5Ps01fcqWmBXeIzYBRp2bKE+P9S0KQWadR4gmmhDVfp6O9ycxj+9q4eE2wmi1GNcWSQG2s09PSYKBY2rfi/HY5TSTSefZvE4TAGx9OoVOr6N5Ux7XZSAVfUKHAYtJQZ9LiCSZpsumJJgR6G6xCq3ylAkmYdTrNKIqC+yuZyVEoCkqMzayhtdFMMpWRhEC9RYvTbiAUy5DOFhiZCZMtfVhnVNPXXk8omhIsvTodCoWCWCpHOJ6W2qKvFbIgWAT/+NolhqdCZEoMv1AokswIP7c5zJLmJJqBhUKRQKkQ56Fdm3j/opvB8SDzwQTNdiP2Oj3jc0IesLfkL3dY9cz4BIZSb9bS3mTG5U/QaNVj1KkYd8ewWXTUm3Vo1Up29zZiMWgIRjPYzVramsx0OS14gomSdppGpRT8yI/va+PQ/g6OnJkhmcrhbDCSSGaZ8cbwhdM02fT87me3cm0mzPnRAP3tVnparYyWLfzOZgv+SJpdPQ1SAZmYh//Y3jbC8YyU9fPw7la0aiWbGo3Eklnu6m1kZ3cDB3aJPVWS0jtfnQmTyRWYDwl50gNddj4a9uKPpAnH0ySSeeZDSUwGNc02A8PTIcLxNL/+mR4arHoujfv5woHN+CMpetvqOD0spEm6/UlMehWxVB69VkG9WY9GrWJkVnAFDHQJwfuTl4WT8awmDWqlgrG5CHP+BIm0UBna2mTEH0mRyQoLKJXJkc0XyeULTHqiBKKpkpUguAeUQKa0YZvqDbzwaC+RRBqjTkM0mSGeytFcr8cdTPKrs7PMeGN88dE+gtEUE+6oFGepM2nZvtlOLJHBH0mhUCrY2y9U5876EhRBWo/BaJpEOidpo7mcwGDF8yTu6m2ks9nCrC9ONJGhySYwmHaHkUgiyz1bm8iWXEKpUufUbK4oCYFGqx6VSiEpPJlcgXS2QCSRlfzV+XxBcCFl8tSbNKSyBRLpHFvarfS21rO1w8a752eZcMcIRgRL0KAV4gqie090jWRzBcZKB7aI1kkwmsFepyOZzhMsuTtHZyN0tQj9fWKJrBSjUCmV+EoWSzielqwFYW6KqJTgqNcz7opJQWGRPwRLjNegUxOMCXMrpFZb8QQSZHIFDDplSWgWcQWSRBMZCkU4sKuF7V120tk8o3NCQDmRLkh7KJPJEStZf5lcQbIc9Bo1rkBCEgLCOhO0/mQmRyKVR6e+bjGkswVmvXFprSTSebK5AnaLDq1GRU+r9YbSSGVBsAimvTGJKWrUSrZ11lNn1NJg1XNtNsK+rU2oFApSmTzJTB6TXk08lcOkV2M1a5hwRUlm8igUcHrYizsQ54NL85KmMNBpY1OjUSqjt9fp6WwW2jS0N5n5+Jofp93ApEcIhk7Px/GHk1yZDhOMpgnGMoy7okRiGaJJYfMqFEKZu92ipVAs8t1fXmF4KsS12QjzwSSRRFbaBK2NJvZva+adszNC9lOdHqfdSFeLhc5mCwadis/e20kym8eoVdHRbCEQSfGvbw6jVMCsL04skWG85Hft3lTH6FyY4xdd+MJpDFo1JoOGNz+YwhNMoikxrGKxyKZGE9PzcUx6NcFoBp1aJZjkapVQubzFgU6jlHz/SoUQXykWi7z63jixVA5vKMX/+eW9HDk7SzAqBBB1GgXprBBLyOUFt0i9SUOoxNyKBWF8RJePUgm+8PVsEaEIq5H/Oj0rbTgQzHWxoEf0HQ902tjV00AknpE2eqNVjzecYsIdEVIoXVGm5+Mk03lC8azkfhJddiqlYC2ICEbTDI4FGJwIolYpSKbzpDN53CVmBFJXZfQaJblCkQd2OkuJDRmMehU6jbqU3ablrQ8n+cWpacHHrVSwqcHAlDdBJltgcDzAfCiJUqWgtcGA2aSVhACASqUgmshSb9ZSLAoMUNxPBq2KaDJLKlMgXyhSKBRRqZSS9hpPZrk0EWR4wo8/miGZzpPNC/GiPX2NjLujtDWaCMYyEqNTqxTSOs6Wccfmer00fwAalYKR2TDheLbCCvGVsnIMGhXxlKCI/P/svWmMXNeV5/mLt8a+ZkZG7iszyUxmkiKpnaRWa7El2yXb7bbd5Z6uZbprBjOoAqbnSwHd1TPA9CwFNBqDKaA+1PQA1VXTtjGQrSq7vLSlNk3JEiXRpEgld+a+RERmxr4vbz7ceC8jciEpLiqWxT9AMDMj3nv33eUs/3POvZVqzaJ/DAOS2TKK1ErjgIiVlatC0S+vCYMl4NEa3klTXECXLWOwboBdFaNh1xROXYwR9Gik81XrejEPNx8mSzarD4uVGkGPZnlpJsTWIeJLmiq3XL8TnLrCYnxzX6vbxQNFsAv6G5lAsgzRjQKrGwXW0yVqdYOnDnahqzLvfBylUBYu8+RQkGRWWH8LsZyV25wpCCvTdIl7wy76Ojx0BBwsrmUtyzKWENkdY31+K8ulze8Q+wk92selhQSJTJnhLg/tfofF22+dSCBc7miiQPPrN0+oNp/OQjzH/GqGFx7p4+pSkkyhwtlr6/SF3Zy7vsFcNMtGpsB0Y/O8U9NR/suvl1hez1uFVKmc4Jwr1TrvTkcbe8qAbBMT9MPLa+SKVUEx1Dfbpso2csUqlWpdBAYTBXRFpiPoQFMl4QW5dN6/KCx3s+WJdGmzSEeTKRSr1nfiiRyrG0WKZRHYM68JB5xWPruwMkuWFS3bROFg3TCo12FyOIjboTLY6UGSaOFyzQ2/1tMl2nxiu+v3LsQ4NtWJ1AhoPz4R4cJcgkyhSm+7E0WRkCSjRanIkmj7sw91c2i0nQ8uxKxYDAhFNdzpIZEtWZZyoVxHVUTcxRrPukHY7+DFR/qYj2XJFSsUSqKY6qNrayzEs5z8aIlqY3oUSkJZmGvCMIThUKmKmoLJoZBl+DjtCrlCFaddQZVtZAtVDENktiiSjWJlsyH1urDuM4Um76QhaZO5CoVSjYBbKL1Epsxgp4dCuUpXu6CKrPs0TVa7JuFxqBwYCXF9JdMyd3c6htz0QorlKvlyjWK5TiJdZP9QiMVY1poLctM8bEZHUFQZNwd8i+VaSywL2PZ7tS7mSCyRp1I1sNlsVBoPiATshHwOfC6VfKFMraE4ak3LVVWF8my+q0OTLa+gXKnvqLha2lSt43frfPNzowS99t2/eBM8UAS7oFypcWk+QSxRoFCqWpOxWBZFWgG3xspGjmpNZA4sreWpVOtINtjb76NQEp7CeH8AbAZXFsUiq1TqLDdS9dbTJcJ+O8nspsXj92i8cXKW1Y0cZ66uUypXsNlEaurMikjFmxgM8uRkJ9eX04z1+qjUDGR5kzaw2ba/j4nedie1hrUcTxVJ58rMRTfd5XiqYFmuBuB16eQKFUvgNwfGdlJCAJ1tThYai9zGpgvudaqN/gmwfzBIphFMc9kVS8GY7r9h1K0gsonNQKdwjRNmdW3daFmk5k9+t8ZApweHptAf8ZDLVzDYFMy1Og1aT3w/kS5yfiZBb9hNoVhjtM+HLElIiINDcvkKtkYQcj1dorvNycyyoIqiiSIX5pNW35jcvfksc0gMQyjl68tpjh/o4uyVuBWQBHDqMtW6YQUvzfvVdxBguWKVy4tJFuO5FmGZL9WYj2YtJdD8/K2QJVvjHAfJShyoVOvoqmR5u6oi4XdpZPKVlriFCYs+bRRabbViKw1ayamLzJdYskilWrMC5ltRrQmvbiNdpFDaeY41w65JlCp1MvkquipRa8yH+Wi2Rcia83Dr+rBBixK4EYa7vEKw1zevLVeNhjdQsfo5mRPFZulcuSkQ3HqvUqVVCYjviK1HrHE3dh878/NCuUYk5Lyn6aM2w7gbe3h++lhfz1K/kSq9Cf7jTy/x5umllr/ZYNvAgXDT2wMOFhoLSZJETvR4f4BqvcblhXSLNeJ2CPc90fAwri1v8vKqIrW4xiYkW6s11Oa1s5YuWumZoz0+ypUaG5kS2XyFrXeQJRvHD3Ry7voGa6kiQa9Gm89BrVa3Cpskm426IQKoRt2wLD+HLlMu16gZImtCkyXWMyWyhYq16M3g7W59ZfaJiZceEdk75Qa3ncyWcWgiGymZKTPW62OsL8DPP1zcUWBsu3+jf3az+naCrtgoV41tY+p1qqTzFVTZts0KvFNIQB0Y7/exEMuRKWx/N/MdpIYEuNE0lm2ieM2Ez6luBsqbnnczODSJQrlO0KuxkW6khUKLNa1INkrV3Rvj1GU0Vb6pUHXqEiM9fi4vJKnX6uxiT+DUJYtvN8fkZugNu6x1uBW6asOGZMVEbgSlMV+b+86hyfzR1w/wZ6+f3/aODl2+JaV1L+BzKfy3r00x0n37ikCSbIRC7l0//8wqgv/h/zrJRpNF2iwUfC6FtsZui6YQtas2ipXN5432+lhLFlru0QwzpiCulShW6gTcKonszSc7CM9BsdkY7vFxZSHFRqbEWK+PSws7H1U40OHG69asc2WbFUubVydfrpEvVrcJbE2RLH4ahJW9dRHYNQlFtllFNta1sg1JtoFhsxaf16miKBL1msiH1mQb5SZJ5neplGsGY73iDNmAV+fs1TWK5e3irL3ByTfDVIyAVQm6VVjaVQmnQ7EE3lZINpGNUtjhmZ9E0dwIkiSs/N2Mi08CXZOoVusio80pUhp36q+bIRJ0MNLt5eS56E2/63OpFv++FVv7+3bQXMUL8MxDXdhsNk5dELu2GsYuc3HLOrwV6IpQcKoElV26zRT0W9sFYv7frL8lm412v040Ubzh926GrQahianhYMuxnp/4vg8Uwc74n/6fU1Zesomtk8Dv0sgUKi3FQUAjI6JmTY52n52AR+fyYqrlHlutiJ0mma5KVuaGid0mQ5tXp1SpbbMybUB/h5vZaHZXwaOrQuDvNNo7XeN3aXhc6qYXtEObzPdxORQiASfXltM7Wtk3E4bmglckm0VN+F2qyPdfy980oHY72M3C6wg6SGaKlJqEjWwDbyNmczvYqe80BcpNw2h6XOZ3FQl2cBxvCzbA2xDsplFyM9hViUjQYZ3BCzDWK/afqtTquyqJT9y2xns7NJlCuYZdlS2jwq7ZKJaN21KmNkDZxfveCl2VsGvytndSZZBlGZ9bI9owPu4Wdhrf3RSOZIP/+fcevaNagpspAmnXT37DMda3ef6s3PDRtwqcZK68TQmAqJ5tHrCXH+vjq88MoyqCPzXvt1XQ7CTQtioB2FkJyBKspUs7Ug0GMBvNWj+DWAjN3GOpsrMSaL5GU4TlD+LdYxt5615b2yRLEHBrKLKNXKHKWkoslErN2MZ56qqEJG396yaS2TJykxIQz68wGxXcuCTZkD/BTJVsoNykCLNQqmHXtt80ulGw0krNZxoGt60EYOfxLG8ZxuYgL8DUSBuq3NpnqnJ7y9UAS8jdSAk0P61YqTMXbaVg1pJF1tKlu6YEOgJ2npyM4LTLVnyomdYplg2r/Z8UBtySEgCxNnZ6p0pNxAzvthKAnZX8bl5HX9htHWZ0r/CZ9QiuLiX53/7qtEUDmJSJItsaGSiNoptGhkpzL2ky23jPra70zayY3az+u42t3P4tXUNr25u53FuBIm8PnN0qduO8b2QhSzbxnp+U0ukI2O/Yld8NWy3+3wTsRq04NJlqvd6SPXW3oGwxEP4+4XEqZPJ/P4P6tWeGefnR/tu+/oFHsAv+9p3ZzcwAm0iTkxrbCjRz5iM93m0W7k7Br60Wxc2m7qc1t29HzW+9xLRUtlqku9n4t6oE7Or26bebLL+RcVc3bo/Xj98jJSBwo1yQTxc3845uFbs5E4XGNg+3AvkTdsv9ogQAsjsoAVWRaPffflrnrWJPj++e3v8zqwgOjrRZP5vC0kqbbKJ0Li+kbyi075/lfu9gvn9H0NHy9ztdogZir/4b4V72b7NcuwFzdVso7yAYb/cRd9q22/XO7gVuNdzzSajATwsG28ewUq1bOwncS3z/xMw9vf992N2fDl4/cf2u3OeTCEOb7e4LnJ3g0Hc2Ae/k0bIEi1vS9nwulYEO923ft1SB9VTxhtdv7V+b7d4oh0/D8DS4vfH/NI3irXGJvy9INyqW+XvErQ7F3fLCTPR27E7r3A18ZhXB774yblkdunr7ky4SdOwqmLbOZcO4vUVt1z5Z+3bLd74TebKTgEjlxN5Gd3Lf4g5FNzeCYdy5J/L3ifuI6dgRd7uu4nbxSdpxP+qMu+mFuR0Kxw903b0b7oA7UgR/8zd/w+c//3leeOEF/uqv/mrb5xcuXOC1117jxRdf5I//+I+pVgXHtry8zLe+9S1eeukl/uAP/oBcbucCkXuJqeE2fI1Ku9InzEtuxupGYVfBdLfC8JX7IOi4UzYQ3L0Uxwd4gNuBfBvJEP/QkC1UeW/65rUfd4LbVgTRaJR/9+/+HX/913/N97//fb7zne9w9erVlu/8y3/5L/lX/+pf8ZOf/ATDMPjud78LwL/5N/+Gb37zm/z4xz9m//79/Nmf/dmdvcVtYGU9R7Z44zS4e0VDfFLcSXbU3UKt/g/bEn+A30zcJw7MPUfpFqql7wS3rQjeeecdHnvsMfx+P06nkxdffJEf//jH1udLS0sUi0UOHhTVcK+99ho//vGPqVQqvP/++7z44ostf/+08Vc/u7RjQK8Z9wsNcT+04QEe4H7H1mSG3yTcak3E7eK2FUEsFqO9vd36PRwOE41Gd/28vb2daDRKIpHA7XajKErL3z9t9IU9n/ozPynux8yJB3iA+xV3s/DrfmACmrG6cW/pc+V2L6zX69iaojSGYbT8vtvnW78HbPv9VnCj4ohbwXOP9PPLj1Z23SHxfsDd2PPmAR7gAT457icvXNds/LNX9tPefu+M19tWBJFIhA8++MD6PR6PEw6HWz6Px+PW72tra4TDYYLBIJlMhlqthizL2667VdxpZfF/+JuP72sl8AAP8ACfPWiKRK1Wb4l9lMoGpz5eIeS+/RPK7lll8RNPPMGvfvUrNjY2KBQK/PSnP+X48ePW593d3ei6zocffgjAD37wA44fP46qqhw5coQf/ehHAHz/+99vue7Twr3Oy32AB3iAB/ikqNbqOOyt9rnPpXJ0svOePve2FUFHRwd/9Ed/xLe//W2+/OUv88orrzA1NcXv//7vc+7cOQD+9E//lH/7b/8tL730Evl8nm9/+9sA/Ot//a/57ne/y+c//3k++OAD/vAP//DuvM0nwOHGcYn/EGAWId0ntT4P8AAPcI9QN7AOLdqEQbZwdzb62w2f2U3n/vQ/nWZ6NnnzL95HMM/svZ9wO5va/UPA3ThH4F7jbp2d8A8JW8fFodkolO/3kbpzDHd6+ON/+vBtX/9g07ld8A8ha2grPm0loCo3d0F+E5UA3F0lcK+2FfmsKQEQ46I1zUtVue0w532P5hya9Wxp9y/eBXxmFcHLj/Xj2GE/+gfYFFz3YlvhTwvyDaSvfgsK7m7iPqgHvKvY6RwHEzbu/X5azfU/6XyFsN/OodG2G1whcD9uRXFDNM2bkFu/p4/6zVWntwBdUyiUWw8c+bTOCbifoSm3dorV/YydDhSyPvtNdWPuELdKh93o2MatZ3fcrWfuBlWxEUsWSd/CYTk7tetGdKssQcBjp1yt3dL97ybMs81VRaIv7OR3vjB+T5/3mTWJT55b2fEQ7ttVAjYg5NFx2WUkGzjtm117PwZ5b9Sk+0EJ3IdddstQJBvjA/77wgLVPsEumM1Tv/lEuTu18G90/Z2qZNNrbT7Z7JP0+43oVkWSWEsVKZRuLc1cU2zWCX93Ar9L49HxME5dKIOJwbY7OqbyVvCZVQRHJzs5OhW5K4vV1jjFbD1TIlesUTcgX9wUpre6H8q9lBtb7y241k92D+lT2kYb7m2g9pPsDHkr1d0eh9Iyj6p1gysLqU8cP9HkVuF7J7vibrZl+98CHn1XWtR8j+ZEjPbAzls3yNKtGTk7GVdup0Jv2NVC093K294Kree031j77UQb7jSv7broo60U6W7CvivkvOH52uZVN1tDNpvByY9WrVMB19N3/6jMrfjMKgKA05fXti3W21EMd4tpcDmFZL6VJtxsMm39fKcm2jVxDuqtvnN9yzbaN7N+7E3m6N3QH38fnlVzQFayQZuvlau12SBTqG6bA83bKN9qs81HmfRkqWLs2se3uv2IOV7mGI/3Bzg21UmhXN9RYO40l5OZnQOVtfqmkfNJU7Gz+SqFUpVSk5DdaQ9/mw1cTXn1PWFhGTvtMr3t4ueAW6W33WltlZ4r1Gjz7X5q2FbaUJFtOyqrVK7KcJcXt1M0zGzebsI+2jigxu/SGv+L9eVvFIIZgK5Im2OyS/sSWUFDmWN2eT6167vcLXxmFcHJcyvkd6gsdjvuPGzius17mEfh3Uyv7BTH6A27GOhwWYfS3IjiMtuXb7i8hgFhv4Nvvzi64/GRO8G0fnZSSOafiuWaJbyHOrdnaamyrWWjsGbh5m0sonafjiLbCLg1asb24zJlSRzEc6eBfxviTNoboW7AWkoIRasZO/SzQ5dp7kYDcOgSXuf2A8gl26aFagqYugHOxjg+Oh5mfMDPaNNRhVLjfGZTUCg3sApUCZy6wuRQEIBIyMlzh3v4xgujhH2i728UWAdxuLuJnRRT0KPh0nd+t90QcGtWX5qo1GC4y81o7+a7GobYnsaEz21HVSTyxRqZYgVFFoKzUKpTqRnYG15UNl9ifMB/w/cy0Rlycmi0rUUwB71C4ScyJbJ54ULqDYVk9oHPpVoK0GmXrXNA/B5xbUfQyTMPdSE17hz0aDw5GWG4S6yFrbLG61QJejYNDfO19zfG7l5C/pM/+ZM/uedPuQcoFMp3ZIlHgk7enV6l2HQAsdMukyvszBuYC9s8ZepGj95pp0CXXcGoGzcU0Kpio74LPW9XJev81p1ukc5VSOYqHJvqJJUtUizXkSX4r18d58pCsmUxy5IQOnVj85m6KrORKVqHuTt0eUfLx3z3TKGC065QrtZRpJ0tqua2FkpVetqdjPUHKJWrYjGXatSqdap1gzavzv6hEEvxXKO/VArlGvmSoNqK5Rp+l4aitAomwxDv4ndr9IbdeBwKyR0Ce7pq2zXd0nyncqWOLNn47RdGWV7LWYqyuf9lSTzT59Ja5g4IZaIqEl6XRqZQFXvlNz7rj3hY3bIpmiIJi3rrPHbaFSaHRV/IkkQiU2Yhvrnp2NauFopDoVJrFdimB1ep1UnlSlRrBka9zunLa0zPJFhez1t9aI6hrkotFrOqSNTrBgG3Sl+Hh7FeH/OxHD1hF9WaQaVap1Cu0RF0UChVAJv1PlvbqSk2fC6dx/dH6OvwcG0pvW0sVFnmtaeGuLqUxGlXKJRqVBoGhwEk0iXrTPFiuWbNO7smUyjX0DSFSrVOtQ7r6aLVlt6wi8Oj7eQKFfKlKv0RN+VKjWrNIJ2rUKnWyZeqhAN2njvcw9eeGWEtVWB2NQtAm8/Ol44OMD2XwK7JlKt1SpU6HodKsVLD77ZbhmXIo6MqMvOxLPPRDPnGPCmUa8ysZvA5NbLFihV0d+gS1ZqBLNtI58XcHe3xcWRvmD3dPpx2le52F7p6+8ee2Ww2nM7dt6j4zCoCXZX59eU462lhldgQysEwDMo7BEvrjS2pFXl3gXIjHJuMgM0gkdkeoDYFxm5KAMDtVLcJnp0wH8si2WxUqnUMA85eXaMjYG8RjiGvbu2zZD6zUK6xni7hcigcnYww1OXbcaE2d3lf2IUqy2Saqh4lm2hrwKNTq9WxIfquWjdI5irk8hU2MmUMw6BW3zycPF+qkcgUqVQNBjpcJLKlbYqoWKlZSkCRbbQHHOQKVVTFRrZQxeNQ+eYLY3w8u0GhVLMsPLsuI9lsLfdr84n2mecsqIqEJtuo1AxWNnLUqgaFcg1F2mx/wKOTb1h9Tl1hcjhIMlu2sju8TpVcsWoJhObWt/t11tMlhjs9BL122n12JofbmFnJtCh5EIZEpVIjV6ySzJZbFJJTl0SsRrKhyptCu9Loa7NvzHc1FZciiWdIksTiWs5qo0OTLKMAWmmT5vs4NYX9QyE0VebqUhq7JpPMlhnvD+B1qsysZkVfNr20z6W2KO1aXcyzXKFKV8jJYjxr3V+RQNdkUrkyH16Oky1Urb5uLlqU5e31E5JNzJ+xXh92VSJTqDQ8CUE3ma9UrdQZ6PLidij0tru5srg5vw0Mhrq8/PMv7ufhvR14nBr9EQ9nrsQplGs8Oh7mymKS6EaBcrWprxUbuirxW0+PML+Sxq5JaKrEkqlkG98zxwEgmSu3GE79HW5UWfRnOGDn6FQnAY+OrspU6wY/enceTZHY2x/gdvFAEdwAXpfGBxeiFjebzlUsJWC63TY2B9Hv0ihX6zfNLDKtKNOK8buEBTC72rqVrCLbGOn2sZbevVjE5ZCpVI065n2eAAAgAElEQVRtSsCuSXQGnSiKRL0h0EC0s9kjqRuQzFVarFPDMLYdBSjZRIwiV6iiKRKRoBMwWE+XWjwghy7TGXSgqTJfe2aEhViW9XQJh2ajWtu0rIe7vCyt5WFL5bGZ3dG8mE3rtVI1cOgydQMy+VbabmscQwTkq4222xrvWWY9VWQ+lrW+pysSpUp9m1Kp1AwrH91M1euPeNBViYmBINPzyZZ7N55KtSbmRKFco1SpkcpVGO318fhEhLPX1sWmYVsmSDhg559/cT8hnx2XQ+X9i8IA6W5z4nZo5EpVS2AG3Cpep843PzfKRqbIerq0zRus1AzqdSG0zbm2tW/Cfge5oohdOHVhLUeCDsb6/cxHs6iNPve6NB4f72C0z49ss1mGkXkfE4VyjatLaYy6garIxJJFxgf8jPT42ciUWE+XCHp0HA0rHqCnzUWiKTNPlTeF/dWltLCmG/O6bjRRY40Ha7LwmNp9dvKlKnZNtvqpeT4YCI9bVyVmozkCLk0ocdlmJQaUKnUS2TLz0SzxZBEDg1RT26o1g5DXTqlS4+y1Na4upri+nOb8TEK0RZXoanMzs5LBaZetALINyJdrXFlIkMoJK7/Z6DLntrkGhru8+JwKdcBpV2n36cxGc7T77HS1ufjysSF+/uEiZ6+uc2khia7IxFNFxvr8DxTBTrgbiuAvf3KJaLKIItt4eF8YRbZRLFW3LWTzObom7XoesAkb0NvhIpktWxM0U6jsmIdsTpCgV6dcriBL272NSNBJOlehzWfnodEQuaIQ1Llijc6Qk/6Im5nVLLIkFvyuRWCNhSPZWgtynLqM3LCE6w3qysDgzNV1Qj47xXKVctWwONNCqUY6L9zr2dU0c1EhdKu1TVrB61QZHwiysp6nXBHWsqbYdg2yNXd3tWZgGHXrfkf2tlOpCityN3SEnGTzFTRZWIalSh23Q0GWbYz2+Unly1RrRgvd1RF00BFw0O7X+dbnRsmXKrz21BCqImMA/R0e5qMZwcU3nhP02C1PSlUkDAwqVQOPU6Mj6GSw04NNsrGWKja+Y0OWbPzWsSEW4zn29Pj40a/mrXukcxUW4jlLuDntCkfGwnw8m+DaUopcvkKhXMPv0Szr2OyroFfHZVfJFiqE/Q7a/HZS2TIOTeL4VCff+NwoH11bJ1+q0tXuwqHJrG4UyBcrOO2qtZ9NoVRDVYXi97g0IkEH6+lSizHR3G+JbJlcscpor4+BiJcfvjuP363R7negqzLLa8ISHuhwN6z6zXHzOjXruQ5dZmokxEKsQQU6BKUT9GiWMqkZwnutN9o51OmlI+gg3gjKNqNSrVOqCoXf1e5ClW1ktsyZZoOmVK7idQqFYSLks/PO+SjXltJcWkiiSJuKURhEBooksX8oiKbYSGTKeJwKxXIdj1PDpcvkSzXavJueo9+t0dXm4qGREJPDIcIBB+9diFOq1CmWa9Tqgl5L5sp0tjm5spBiZjVD2C8oqpcf6yfks/P0we4H1NBOuBuK4ML8BguxHHUDCsUq6038I4BdtbWk39WaOH6vSyUScpLKlvG5VAxj0ypPZstoihDqlWqdNt/mxNj2HuUamXxFZGA0rnc7FKp1w3LpBzu9/NbxIT6e2WC8P0ilJmgc859JA+2mBBTZhl2VqNQMPM5Wd71SM3DZFUqVTU+nUKrhciisrhcsKz3k0VuCey67QiJTthYwgKYKYS9LNi7MJzk2GSFbEAou2xB+dlXCYZd3pN/MtpqKqlY38Dk1fv/VCaLJIvFEK8duLuxH9oaJbuQoVgzr3coNoRBNFCwhVq0ZDHd5UWWJ6EaBRKZEyGenUKySzgsB96N357m2lEZVbNuCmXXDsALkYgFvkuFnr62TzJaYj256ffW6+Dc9u8G56xtcmk8SSxYsD1PXhLfi1GV8bp1UrszUcIhUrkQsWaRQruHUZfb2ByzarM2nc+xAF7//yjiHx9pZSxX4x8/v4fpyiniySLVmEEsWOHagi0q1zrWlNJVKjURWjFUqK5T43v4AmiqRyVdQFYkPL69xbSmNQ1NY3ShYXojPJZRG0KPT1ebCqBsUKzVsNiE4Z1czJDJlZElisSmOUa4Kb8ll35wfR/a1k20ot2rNYGIgSG/YTb5Y5evPjrAUzxJPlXhsvIP5WBbDEHPRnI//+Lk92Gw2utqcZBtcf5vPTthvJ5kTyj7sd7AYz3FsqhNNlZgcDJJpfFdv0GDmXDCVgF2TODbVyWvHhzl/fYNcUdxXkyUS2TIhj05HwM7MqogbLcRyHJvqYqzPj11XiCUKZAsVnj/Sy9RIiN86PoSqSKys5UjnqyQyJTwula88NUw44LAUdJvPTionvBKnLrMYz7N/KEixXOUbz4+SL1UZ6fYxMRi6IyUADxTBrlhZz/G9t65Z/G6uWLVcUptNZKuk81W8LgWHpuB0KOSKVcslLVXq6I3rSpX6Nku++few34HfrSFJNvYNBFjdyO/YJnPxlat1y70tlGs8vj/CD9+ZYy6a5epSmgMjbeQKFXLFKg5NRpJsdAYdVqDJhGnJioCheDeHrtDX4bYsHZdDtmgYM2gFQoGZQr5aM0jmWmMbRycj7O3zU67UCLp1wgEHqxtFPA6FXLHGeH+AgNfO8lqejaZ9UvwejWS2tZ2yJAKMAY++LZMrnipageXoln7raXcx0OnBoSstfG/Asz2Qa1Jj7T4787Es4YCw7tfTJWZWM8STRRKpEi6HSr5Uxe/RSWRKlms/2uNjpMfLQiyH1yUUpxlgNTCo1Y1dlb2pYA/uCbGRLlnKqqcRAEw04gBTw0G+8tQwD+1pt4RFpWawFM9RqRq47DKpXAXZZiOTr7C3L8BTB7s5cXaZk+dWCXp0CuUalWqd6dkNvvB4P2euxMkVBS10cKSN2dUMILKHMjmh/AqlzfTXkM9OR9DBihnYbsRyCuUaiUwJSRJzKV+qbQpjr514atNKDwfs1hgfnYzg1BXiqSLZfIVqvW69//hAEF2VOX1ljYtzCRKZMuMDfuZWs2QLlRYLvlKtk86XOHlulfVUgWS2TCToIJYscvxgF9WqUHYH94R4bCLCc4d7eOahHqZG2hjs8nBlMclQl4+V9TwDHS7CQae1BrwujX/xpf2EfA4GOsV3JwaDnL6yDoh3LzcC4wDDXR58bp3ZlQxnrq5TrwsDq6vNxXOHe9BUmTc/XLKC8Q5dZimeR1NEgdoHF8U5LaJoTCHg1VjdKDI1HMSpK5yfSbCRLvLLj1bwOFX29Nxa9tON8EAR7IK/+OG0ZcE8PhFGlmwtlcbmoi5V6hQrNQYjXgJejfVUCaddpjvkZGm9QJvXTmebk0Qj19q7xeIGwV0nc2UK5RrtfjvRjQJOu2JZlj6XisuhEvLqhHwOEpkShXINuybT0+YkkW3NGpkcCvGVp4d5/0KUTKFKtWbQ0+6mN+wimihgsFm6P94fYP9QkGSmRLFSo1iucWRvmImBINVajWiiaCmMcMDBkbF2NEXiwHCInnY3qiKUlyoL4VM3hNXe3+Hho2sbLMZzyJINTZUIeuwM93jxODTcToUTZ1daqAGAx8Y7kCSse8Em9ebUZZ5qLGpJlmjz6kK5GXBhLsFwl5egR6deF/RPu1/n4nzKEjQAo71e9g+GtgW6zanid+u0++wMd/sY6/Ozp9uHYWD1eb5UZbw/gBkfCTbc/I6Ak688NcxaqsB8NNeg8Qy0Jg/GhMuhUG0IDXvDCnXoMjZjM9fcqct883OjOO0qA50eOkNO5qIZxvr8DQqqQipbIl+q4dBl3HbVojrW0yUuLSRZXssyMRjk4nyCa0tpHpvoYKzPTyZXIZookMqVWIjlcNkVDo608fyRXhYbMZ32gIMXH+7l8nyiYZwUaPPp/Isv7SeeLDKzIhSGadDYNRlNFdSHy6HQF3aJGBBw7ECnxWWP9wf4/VcnAMjkykRCLr50bJDp2Q3iqaK1NkZ7fbT5HJQqNeG1VOt4HAqjvX4+nk3gssv8s8/vo1o3cDsUEpkSh0fbyRZEskHYb2diMIRDl3n5sX6mZxON+9f4J58bA+AHv7zO37w9y/RcgrloloBbJ54q8uhEhEjQycqGoOUKpZolcN+7EOXUhRhTwyHG+vxUa1USGZEQ0ObVOX6gi6DXzk9OLRBPFRnt9WGzCdlxbSmNx6kyH8vyizPLYi7YFUuWyDI8OdnJeqrIobF2ZFni3ekoB4ZDeFwqX392D786HyWeKhLy2XlyspNazbjjjCF4oAh2RX/EY7mBEwNB0vlKS6CsGZINYskisiQ8gOcO9ZAv1YiniuRLVR7ZF0ZXJeLJIv0RN4/t76RSrlCu1luCsrIkcuc7Q05+5wv7WIgKt9pmE8HRZK5Cu89uPUdY4hUr26Q94ODI3jAvPtLHzz9cZHouScCj4barzMWybKRLVOsGHodCqeHpjPR4cegqH88mrHbs6fbhdKisrOdaspgy+Qr5YpX5WBaPS+Xd6RjPHu5BU2TenY5ZgrtuwOxqhlyxilOXSOXFYklkhOCJp4pWNo3TrnBsqhNVsXFgpA1VFcHCeLKIs0EbmBb84xMd+L2Cpy2UasiS2NfG5xQpoZ0hB5cXUpYwCftFfyiKjXSujFNX+PZLe3loTzunL8daTqBz6gp9HS6uLWdYT5eYXc0wMRBEliV8bo3+sBtVldg3EGBuNcNCPE/YbyeeKhH2O5iNZnA7VA6NtnPqYox63cDlUDiyN8x8dDM47XJspiCH/Q7aGoq9WjNIZMu0ee2WpZ8vVThxdgWXXWEumiGWKHJpPsmVxSTvnI+SL9Ws/HQzyO7UZSo1QRHNrGSZj6bxu3SuLqXRVQm/R6cz5GRiIMiLj/Sxup5neT3P7GqGxXiWWr2OLEnMrWYoFKssreWp1gyOHejkHz+3hzNX17iymLDWgkMT8QEzVdTjUMgWqnQGXZbg/8bze9jbH2AtVeAbz4/SGXLx01MLzMWyzKxkCPnsfOnoIIuxLH63xkN72knnyrzzcRQMg6DHjmyzkWyMYTwlaLBKtcZHV9fpDDl4fCLCC4/08dAeQYeN9vr52QeLxJNCaL7wcC/TsxtEE0WW17IkMiV++O68RZ9ODQf54tFB1tMFEukS75yP4nKoFEo1gl6db784RrlS48piirE+P/uHgpw4u8xGumxRSAMRL88c7uZv35kjXxJxkn/60l6Llhvt8RHw6BwZC+N2qMiSjeX1POP9AdoDdqZnk6ynC0zPJZkYCKLIEpcWkqiqxPRsErdD5eXH+lmMZcEGHqfGj96dvytewQNFsAs8To1HxzvAMDh9Oc58bNPiDng0utuclpA0gEjQQTRRYGo4SMjn4OS5VcJ+u6VIvvbMCMtrWaZnk/jcGtOzSbFgvfZG8FNw/qbCSeVKXGxUDDYHUY/sDfPKE/1cmE0QahRTFUo1gj47miLx6hODhHwO/u69OeLJIm67wlojs6daN7DZhBejKjYqVYOFWI5UUwqiy67Q1+Hhjbdnafc7rAVv0hwmPzoQ8TA1HOLIWJhfnFkintz0HIa7PLgdKulchUhIBLPNPPY2r85j4x2EA3aK5Tr/zW/t5+mHejg61cXSWo433p61nnlsMoLHpfJbx4bIlyq8+uQgI90+5lczxFNFK89akiS+9NQIzx/u4aOra0JA6hKrG0VcdoX3L8YpNmi06Eaec9fX+dyRXuZWM4S8Opoik8qXOTbVhSzZiKeKhP12wgEHP2zEBBy6wsX5JOvpIqlGYP/YVCcPjbYT9OpcW0oz1ufnJ6fmyeQ3aY+vPDWMpkhUa3USmRIvPNLPUKeHTK5CLCniEKO9Xtp8dqaGQ1TrddbTpYYAHWV1Pc/5mQT7h4KksyUmBgO8f3HN4tY9Dg2XXdBVYb+DZK7M1HCQsd4AV5fSxJNFQbFoQphcW0pzbSnN1EiIkW4fpy5ELc9mMS4Uf75UpSfsoq/DzexqRvDnqsSZK2ucOLvCvoEAbodC0KOzmigw2usj5LVzZG+Yb70wZgle8/9TF2PMrWY4cXYFt0Pl4nyCxViWZK7MQIebNp+DvX0BHtnXQaVmUK3VOXUhBmAZEMcOdOKyK3zx6CAuu4KmSnzx6CCFSo2zV9fRNYlDo+2EfA4emxB1CJoiMdbn58hYmDNX1/jS0UGrPw1EbUkiU2a0x8e3Xhjj9RPXOT+TsBTDYKeX2dUMxw90cWg0zJunF3n9xAxTIyF+dX6V6dkkhXKNgFtFliWW1/JcWUwSa3h1T052shTPcfLcKgf2tDHS7eONt2cJ+ey8+sQgk8MhPE6V144PcWQsjMep4tAUrjbm0nOHe/A4VZy6Ys2vh/d28MGlGNOzImPo8f2Ru+IVPFAEN8BGushf/ewyyWy5JSXsqYNdfPnYEHMraWySjcfGO/jG86OEfHa+fHSIcMBhBekiISdPH+xmZiXNT99fwGWX6Qt7UGzgc2u4HSr9EY/lRjs0sNlEcMowYLTPh6bIBD0aB0fakGWJd86vshDPUWsUu0SCDmFpJ4usbuR4Yn8ng51e1lIFBrt8zKxkthXvPLE/wlqqaBXKjPX6LPpLlqAj4OSlR/vIFCsE3KqV0w40MoIyTI2EWN3I84szK9Z+SgDHprqYa3gEimRjT6+Prz49zPWVFPFUCVWV+PWVdZ473MMT+zeP2HM7VFY3ckwOBpkcDmEA75yPspEpWhbRgZE2JodDlMpVoom8FdSbHAkx1Onl/MwG6+kSxw50MTkUol6vky0KnrrdJ4KGMysZLs8nSGbL1Op1UjmxVfHxg11cWkiCQUN52mjz2VlPlwh4dDoCzpasl3ypxgsP9zIxGERrlBIfO9DFuZl1KtU6U8Mhhrq8LK/nrOyOFx8f5GfvzTMbzTDa4+PJyU6+/uweHt7XwdvnV5ieTTI1HOQbz49y5uoaDk3k5ZfKVZLZCnZd5snJTr54dIAri0niqSJup/CqvvBEP6lcia8/u4eJwSDlcpVMoYLfrfHc4R4uzSfJFQW19drxIU6eW+HE2RUx71SRQtrm0+kJizTIqeEQ3W1OcqUKQ10+PrgkuOtiqcof/aODPLyvA49TJeDWeefjKLIEH19P8MLDvXSGXOzp8fOjd+d44+1ZutqcTA2HuDiX4J2Po1ZMKRx08M75KPOrGT68HOMXZ1YsGqmtkRY63h8gHHBw4uwKkZATXZP5xRnx83/1yn4uzqxzfibRkkuvqyKI3t3u4j/83QV+cUYooaDXzqWFpJgjU11MjYT46tPDnLoY463Ty4z3B3h8v1Dge/sDeJwqzx3uQVdlkTnlFMdCjvb6rTiNS1cbc8jBN57fw/XlNI/sC/P5x/qZWREZRkf2hrm+nOTwaLt1P9PDmFlJM9TlZWIwRF+HB49T5emD3XicGnt6/PQ3/U1XZfojHsu7WksVef3EzB17BQ8UwQ3w52+ctxb+o+MduO0q8VQRWbJx+lKMaysZCqUaPWEXf/v2HGvJAslsidWNvDVpX360H12V+V/+8gNyRcFdz61mSGTLaIrEfEzwkKZFXmvkf19oZJB4HBqXF1MkMmViiXzDKhUWc6VaJxJ08N99ZYqluOB2D4+2MzEYwuPUeGwiwmCnl9OXNmmQ3rALu6rw4iN9xBJ51tMlxnp9fOXpYeZjGQIencsLaeKpIrOraWYaVMlAxINTVxjt8+HUBeXx3OEegl4771+IWl7GE/sj1Gp1soUyiiKzbyDA+xfjpHNlqwoz5BXCdaDTw9XFFB9civHT9+dJZcv86uMYR/aFefpgNz97f4H1dIlCgyoZ6/Pjc2v85U8vkcyUWN0oWMH5SrXOqQtRLi+kGO8P8K0XRplZSfOzD5YolGoMdHqYHG7jg4txPA6FTKEqaIyicOtzxSrXl9PMx7KWq7+eLvHkZCeyDBfnUxwaa2ek24euyNhsNi4uJFlLFXjqYDczq2lePzFDb4ebf/byPmvh/vxDYUW6HSpHxsL8H//vr5lZEfGJjqCDrz0zgsep8ebpRd46vczUcJDf/cI4v/xohddPzDDc7cNlV5hZyVptmhoJEfLa+fBSHF2VWUuVcNoVbDYbb51eJuSz09fh4fVfXieWLDK7miGVK3F9JWMpmVMXY3S3uTh3fd2q/p0aDvKHXzvII/s66Gx388REB2vpIu9fiDM5FKK73UUskSeZreBxqkwMhogEnXx0bY14qsDqeoF4qshaqsDEYJA3Ty/y8cwGiUwJWbKRL1WYnhP1F6aQP7I3jMsuAqDxZNFqn+lNpHIlvvH8KHv7Ao2D2w2OjIUJ+ewcneykM+xhdinJpYUkBnXWUsUW6/jN04v84oxQdoOdHp6Y7GQxLigon1tvWN2aJeRfOz7EgZE2UUmfLvLzDxaJJQsNYSwEs67KjfUlFKHHqTK7mqG73UW+WOXj2QRjfX4OjYbpbnfhcarMrGY4f30Dp13hqYPdVttePzHDpYWkJch1Vbb+N2EqoZPnVogEnZbX09zuo5Od99Qj+EyfRxAJuqzjKlVZorfDzVq6wOWF1k2ePrwUp1CqsZaG2WiW4S4PLz3SS6lcI5MXls/4YJBTF2IYBrT77cSTRfYPhVjdyFEqiwns1CXypbrYK6i+mc8OMD2XaMk6Gehw4bRrfOuFUdwOlbG+AGN9AZ473ANAJl/m5LkVjk528nuvjvPvv/dRI4OpRixZ5K9/doVYUmR+DHR6+b9/eIHVjQLj/QGefaiL9y7GWN0o4HKIIrKltRyXF1JMDQe5tJBirC/Azz9c5OpiikyhSiTo4He+sI/v//K61WdjvT5iibz1HiON/XD2DwX523dmAXjj7VnrnZbjOb745ABHJzs5eW6FS41+zpdqhP12Hh3v4N9/7yyxZFHsX9NoG8Cluc0YR7VeJ1uocHVRXB/2O/j6s3twO8R+N3t6fPztO7M8PhHhP/70sqUk9/R6yRbL5Is1hrs8TAyGrP6cnk2yEM0y0uNjei7BS4/0Ytdlvv7sHgDr8PCjk514nBovP9rfMkeuLqa4uphiZU1QjE5dUDV/8cNpfvcL4xyd7KS0S2V4Z8hFriC2Zxjp8XFwpI3/9T9+SKZQJejViQQdvPLEAB0Bp9WGk+dWWN0oEPY7eGi0jeMHuhjrW+PoZCc//3CRN96eFf1XFJTSYxMdllAEeO2ZPcTjmZb3Onluxcr4Muf2yXMrvPVrEfgMenUiASdff3YPJ8+t8L23rll7IMmSjY+ubYgis26/NQeOH+jC7VD5+YeLlCo1dFXG7RCC7c/fOM/0bJKBSJTnDvcwu5rmo2sb6Jrc0r/Nn11eSJMtVFhLFfj6s3s4OtnJxzPrTM8mMYDvvHnFWr/XljN4XWKsdhqz77x5hem5BNNzCet7O+H5I72sp4t8dG3D2gJloREXMu/7yESZ6PqHvPLEgNVvB0farDE/OtlJJl/mR+/OsRDN8q0XRCzF/G6pXLPWSnM7dmr3vcBn1iPI5MsW9w3iMJZTF2KWa23XZNL5Cl6XiiLZWjKBEpky2UKFDy+voSkSb/16kdOX1wCYGg7yP/72wzg1iSf2R7gwl+DifJJI0MHEUIj5aJZnDnXj0ERQzKkrfOuFUQCiG3kruPzoRITeDjcj3T5OnhPW41ifn+X1nGU9fO+ta3icKhvpEmeurhMJOvjG86Okc2VAVDiO9nqJBF18eHkNj0Phn768lw8uxVhey+Oyy+QaFvOBYVHw4tQVxgfEJldvvD1LPFW0vJLp2YRlfQEYdYPFhkfVE3YhSTZmVwRl9M75qAjGNjh5EOccmO9wcKSN1fW89Vkk5GR6dsPyKgrlGs8e6qZWrVGq1ukNu6ysrvV0idX1PNNzCcb7A/zR1w8Q8jksy+o7b15pLNqsSEls0AFOXbUqRY9NCfpPV2W6210sr2U5PyMsvakRoSCeOthNtlDhz984TzxRsJTGm6cXiQSdlCs1ZlbSyDJMzyU5NNZOtlAlV6hQqRlEgg6uLWdYXstyaLSd5fUcr5+YYX41g6aIegqfS+cnpxYsrvzQaDurG3l+fWUdl12h3WdnIZYjEnLS1+FhPpalr8NjUQz/5IVRDo2GLWu2XKnxs/cXGgFX4VH+91+dYm9fgJPnBH1y8twKA10+atVai4VqWp8+l8Ybb8+yvJbl+cO9LMaF19jb7ua3XxrjzNU1Do60EfLZeenRPtwOFY9Ls2JlB0baeO9ClLdOL+N2qCyv53j6YDfRRMGiOURmzaYlH00ULI/py0eHLOvX5dKpVmpMDAaZj4qYSK5Q4fpKxvLWJocEFw9w4uwK4/0BDo21MzEQvGEhVn/Ew2I8S8hr59iBTt67ECUSdLZ4G99765pFCS+vZVmM54gEHfzeq+KgmL97d46L8wnW0iXe/XiVSMjJfCxrXffqE4OWl7u6kbeyjUyv6s9/8DG/OLNMd5sTj0vl+cO9lrK+m3hADe0C06U0hYTXpXF1KY3LrvDqkwO8dzEmqhUr4l8k6CDkE9WbamNzqKnhIH63bk2+x/dHePGRPk5djFMsVri+nOad81EcuthHxCymGe3149Bl4skCy+t51lIFwgGnJaRMztRcNMIKVanVDOtvB0faWEsVeHwiwuxKxkoZTOdLgI0rjfTJkM+Ox6mhKzJLa3kiIacVsDo21YXToRFPFBjt86PIEj98dx5ZsvHyY/0i80GGmZWs5aprisRgpwdNlVoC7MVylY+uiRRBXZF5+lA3R8bCYqvhitgzZ7jLa72XyYGa2RWXFzYzL8z01hcf6eO/nFkmmS2jazL9YQ/7h4JMDYdwOVSuLaU5NNbOfDTLxfkEPrfGf/jRRc7PJGjz2cU2CA0aaS1VZF9/wErde/GRPnRVtiyyxycipHIlXmxkppjC4M9/8DHTc0muLqXBMDhxdpm3Ti9bwuz1EzM8PhFhsNPDQizD+GCIKwtJxgf8/N4rE6ylCnx0bcMaR1PhmLULuiJblNRIt49yRWxdMTEQZKjLy7vTMcZ6faylCizFc3io+KUAACAASURBVPzwV3MivXekzRLgmXzZUk4nz63wizPLjA/4mRwKUavXmRpu49TFGN976xprKSFwdU3i8nyiRfCZymBmJY0s2Tg/kyDks/PVpxqJEHNJ6/q1VIHnD/dy5uoatZrYD8cM6DYrleY5a87jo5OdQqk1BLumSrzwcJ8lcJsFoculk8+X0VXZEvifa1BKprdmesZDXV6L/jk0GmZvf+CGdIrHqbGWKvLOx1EuzSd4/2LcSsndGjPwODUmBoN4nCrf+twoIZ+jhfqZHAmxr89vvZtJc3W3u/jLn14Snk5jrHVF5otHB/nOm1esNW9mDpkK4k7TRbfigSLYBVs5w74OscFUIlPm8nyCbNPukS67wh98eT9TwyFOX45TrNQZ7fHR2+GhXjcY7vbhd2tU6wYnP1rhrdOLXFpIkslVrDRQEFbueH8An1vjP3+4RKVax2aD6EaBrjYn+aJIRf0nL4xZAennD/eiqWInw2bu1Ax+racLvPux8GQ8DoWlNWFljw/4G/GAlCUwD422c3Sykz09fjxOVXgs8wmiG3kmGl7ApYWklcf86hODVlDabIewgCVefqwfVZGoVWuoisy3PjeKLMNGusQ/enaEo5Ndlidz/IAI2n3tmRGGurzMr2Y4P5PA7RCBulSujNHI258aDrG8nsPtEJzqaK+fK4tJJofbeG86itMu9jgy36FcqfHDd+e51ODzzYUVatQgjPb52EiXeP3EDNGNPBcXkmLrirEwuipbVt+VxSTXlkWqoxmUy+TLvNfIugGx79D0XNKyWpsDfyfOLnN+JoHTrnD8QCcvPNzHmatrloJ5/nAvIZ/DCjwPdHrQFZnpuQSHGgHGE2eXebuxxYGZwnhhboN4ssDKeoFMriw2V2vsO7OynuMvfjjN6kaeN96etYSWpkgEvXbmYxlLuDx/uJe1VIFXnhggEnJik2S++/Mr24KQpnB7fH+EqZFQixDUFAmnrlj9sLqR4xdnVhjr86NrEh9d20BTJGZW0/R1COrN5NBNCsr0aDxOjcmhkKUY11IFSwk0K7aA30m+Qb+atNLrJ67z9Wf30BlyWeNnxjS28u83w8X5BJcWRJDd9OB24/PN38uVGm+eXuTgSBtuh6gDsesaT06IzerKlRpv/XqRE2dFQZjZ9188OshGukTQayeWKHDi7Iq1J9SRvWF0Rd4WFG/uiwdZQzvgThXBQizD67+4TiJTot3v4JcfreB1aTh1ha89M8Lsarop4Fjl0kKC+dXs5ta9wPnrG5YXYS7geKrIxFAQmwGxpEi987s1MAyK5ToBj47brjDTqPAEkRueyop0w3yxSqVaZyme48TZFUKNStjvvXUNVZGYXU0z2utnpNuHpkisrOdJZEo4NJlcqcZor5cnJzv52jMjZPIVkSUDTAwEefWJQXRVZiGW4Qe/nCGTL/Pux1GrolVUB+c4PNrOo+MdnDy30pIWaE7uSwsiw2cxnuXSQrqxfa+DWKLAYjxHvlThsUbRjikozTJ5XZWJJQtcWkhaNNHrJ2asoGIiLdJqrzaKcx7dF+G5w72sJIqcv74uNgwzDPb0+LmymOLKUrIlHVPEU/zMrIi0yFK5Tk/YzaWFJKFGhlA8WbQWu9uhMj27wepGYRst8ebpRX55dsWqQj6yN8yh0XZLYDUH+UyB/9sv7WMpnmVuNcPrJ2ZI5Up8dG3DsvQ8To29/QH2D4as9EKTn3/r9LIlGMb6/Jy6EGVmZXOHzs42F8emOq2slL/44TQfXdtAttnobndZytocp8Oj7TjtCq88McDrJ67z0bUNAh4dRZZ45nAvTk3aFoQ0x+zIWJjVjTx9HR4rA+atXy/y9vkoHQEn8VSRw6PtPDTaztMHuzk02o7HqZIvVnnj7Vnmo+mGBb8ZgG0W2ma/mULS9Jr29PhbvndgNEx0LcubpxdxO1T+z//vI64tZ5ie3eCRfR3brG/zXW5VgHa3u6w01NeeGrYMrRtd00wZvfrEIAuxbItSNRMDIkEHXzo6SGfIxWMTEd67ELU8iK42J05dYbBLpGm/+EgfqVzZWhemImjuiwdZQzvgThXBn/6nXxNNiArKtZTQzrOrGdoDdiaHQ1xbTuN3awz3+CiV68SSReKpIppso9bYAwWwPAO5sUFVm8+OothYjOVo8+rs6fFx+so6fWGPVdySK1Zbip36OjwsxnN4HAqJrJgMsmTj8f0RjoyFubKQRGsUrF2cS7K6nueZQz1cnE/w7nQU2KxF6Ag4efmxfk5djFmFLWbOsjm5//e//jXRxv4oX//cKMenOq1c8JPnVnmowVN/761ryDIcGm1ndkVY8eMDfh6fiACCjzWF5ECnx6KgvvH86DZBaXLqb55etNoFWF6Oeb+H94bRVKklDQ9g33AbJ88skStW0RVRlGbWJJicbWfIxd7+AHv7A9a2Ar/7yj4GO72srufpDDnZ2+dnvIk7/vmHi3x4ec1SJKcuxizhYQrFzx0RVMQXHh+waCNT0FxZTPH6iRkiISdfPjrEX//ny/zn9xet8Xvxkb4WQRcJOi2hdupizLK4rWc93GNRVPsHQ1YGTLvPwdVGfcDEYAjASjPs6/Dw9vmoZTSYXPs/enYPTx3stvj6qeEgDl1peA8aLz3ca42J+c7me5spmaZ12pz1ZGb9PHe4p0XB7+nxc3UxJbzKJmVroplqMWNcIZ+d5w/3srqRw6EpLfGPo5OdBPxO/ubkdYvWuracweNQ2MiULS/AzOhq7l9zXG4mQM00VJFKqt3QozDH3IyPHJ3stFJED+0L88RExOq/5bXsNg8zEnRaNTJOXWF6LmGlaTd7T81xjU8ra+gzqwj6Ix4uzSd4aLSNoNduCfJ4UlR3zsdEoHEhluORfWHyxQq6JpMtVnE1qmVHe/0ksyVOXYjx8N4wTrvCzEqGRINKyJdq5IqCHgr5dA4Mh8gUKqI0vcfHgZE2HLrMV58eJhJy8tWnRyzOfHouYeXxv/H2LPFkkaBbI5Etc2isnf4OjxUUHO31Wa7l9FzC4nHdDhVdk9nXH+A7b16hzW9vVEsWSOYqPDoe5ne+NMlP3p3le29dY6DTg8uu8PzhXka6fZbbbqb/jfcHGOkRSsXkY798dAi3Q2V2JcO701GePtTNQ3varUVzfmbDshBT2bKVajm7mrY8npcf7bcWgblHjClgTPpjaqSdA0NBK7awFM9xaSFJ2G8nmii2LDiAoNfOc4d7CXrtFm8+u5rhodF2yzOCTWrg8UbdRbP1tTX43PyMv3t3jtdPzDDY6fn/2zv34Kaua/9/bVmSbcn4KfkFfjsQY8C8CVCCgWLMG6cUCmmZm6R5NNOQX6a0YTKdDKUUkklT8mtIJvlBbto04RYmuTgETNIQQgMJ71AgNsQIYzB+PzDyA0uWz+8Pae9sHR1JR7Jky2h/Zhjj4/NYZ59z9lp77bXWxniby+3YxTp8frqGZiKTBKGEmHBU1tzGvEnDcf5qs52vXuyGIJ12fHQ4xufqMHNsCmaNS8W4nASHDoG4bG40dGBkWgxmF6TSTpT1tZPOZPnMLNxo6KA+7ezkYQ5WOulEj12sBwBqnbLnIK5KMlowdpnopOnUvERqfIgnak1mi8NkN3FzHj1fR0eBrItHo1EjSq2gLpb46HD8ZHaOneWujVBS9xeZCyGT/uIOlLyXISHA3z+9jPSkKIfJWWejCXYkQMLGiSttTHY8rtfdsbXTD/MJ7PXVSgUdBc6fPMKhncSGE6tgeYkJJ/RXEcQNC8c8W/bpwRM3MGmUniqD7JQotBpNSIqNwPQxSaht7sL1hg6kxEXCZOlD591eKBWh6Ogy47Ktbn0ek9LfJwh0YRTiu69r6Ua3qdcuK1GlVODYxXrERqmxZHomdRuwLoO4YeG4VNWCzru9mJKXSP3Jh8/W2LKbI/DEstEYnRmH9k4TMpKjMCxShezUaFytacdXF+pw5cZtXKsz4kR5PSqqb+N2p3Wie828+xAbE4mQvj40t3cjXBWGE+WNNLGL+IUj1GEYnRGHuGHh+Pj4dajCQlHb0mn1R9tcEZeq2qhrpfXOXfz5n+dxqqKRJg813b5LP87uu704fqkBeemxmDEmCX//9DLuGxFjN0lLeKv0O1yqasOtRiPmTRxB46uJ4pgxJhnX6+9g3iRrzoMUSXGRdlmo7IfGWmFs58S6h6SiWYgCIaMLEi6YnhINbbg1UzQjOQo3Gzvw1fk6VNli/ZfPzLJr09kFqXZWuTNXh7MOgc2GJcpTPIlM/PXs/S6ZlQNLr0XSSh+ZFkP/kVEZ6x/fdaDcTomxk6bEXZKq09i1M5FVyp/PPh+x8iBRQ0QBV9a041ZTJ03IMnaZ6GQsUQ5id6S4vdg5oeb2bkyzjXDF+7ga0YgtdvGci7PnRbaTb108oe0rV5AYrgjcQD7onNRotHdYXTettiJTxm4zRmfEoXhaOprbu5GRNAwVtoSZljs9tlIFEZg5NhkzxiTTKIBUnRa3jT3WiWZbbRZiuZLoorkTh9OsRPGiE+TjOHy2BkfP38L1+g6MzY6zZZTGw2S2oOxENXUzkRC9//13FZ2v6Ow207UCCnLj0djaTYujjRwRjZzUGOhjI3D8Yh0qb97Gv/9TZ4tksnbY5EMmESEj06wvJfnJhgGyiVJRkSq8VfodquqNSIgOR1bKMChCQ6CLUVPXSqXNffBAfhJOVTRQH7r4gwSAm00d1snuUXpU199xsJR2H/4ehlojKmtuY8r9iZKWk1QWKnF5sFaqlGtAG6FEfUsXdDERuNXcSTtn0qFOGqnHe59doVb86qL7kRClouGM//vvKkwYqYMmPAyr5uRSxfn1pQY6R8K6MUZnxuPyjTaqcN0tRiLunMQRROJOhbRbXGwkjcYRh4/OLrCOQEgnJT4nydCN1lgVMlvugSg2VlmQkQbrUhEnVEl1isAPUUOAdIKWWFG7c++Qe5w3aTiNPBKPCJy5Y5wlg+UOj8H92QlQhAj9duH4yhUkxm8JZbW1tdiwYQNaWlqQmZmJV155BRqNxm4fk8mEF154AZcuXUJ4eDheeeUVZGdnw2w2Y+rUqRgxYgTd96OPPoJC4duQKXcYu0wwmS3Iy4hBCEATnEg0jyBYh7Pnrzbj0UXWuGGVUoEeWwGwqto7+L6mHWqlAuevNuOCoRX6mAia6BRlK0p25Nta62SaRkV9woA1UUatUtCICjZJ7NjFOppgkpcRg4ykYVTuYxfr8H3ND0lvRJ6lMzKQnxWHxrZuu2QjtVKBHltZ6amj9NBGWuPESZJOXnosls7IwNS8RJy/ap+UVDRlBFYWZtOEl5WF2VTeHpPFWq8JsLuvEYlalFe3ITpSSZORAOD81WYkx2vs7ruj2wygkoYCip+PWqnA0hkZUIQpsPeIAT0mC5b/KIvus2pOLhrbLqC+tRuHz9YAsD4zlVJBE6hIu97pNNFkONJmpLPsMVmoTGzHcP5qM006AkCTnUiiT9nJalwwtGJsdhxtF/I3Y5c17LUgJwHnrzbTGH6yPwDsPWLA0hkZdu3KPteyk9VUJvb9cBZrTu6HPBPy09mx4u1SyUtS52xpv4uPj1+HyWzBT+fk2j0TcZuwxxfkJGDXgXIa9eMJM8ck406nCTcbOlCQk+Bwj3Li79l7fHZlgdt95BKtVdNj5DwnOfINJF4rgk2bNmHNmjVYtGgRduzYgTfeeAMbNmyw2+e9995DREQEysrKcPr0aWzcuBF79uzBlStXMH78eOzatavfN9Afjl2sw6FTNwEAOakxWDojAz1mC85eabJmAoeHofLmbRjqjLQDIi+8scuEt0q/o+cqyEnAlRttiIpUovG2taTv//mp9UVruXMXPxqX4vTF7+g2O2QXspmoxi7rxGiP2UKzKXtMFtrhAaCddGVNO1UCjbe7obZ1iKTTj4+JsFobtg5KparCmYpGjM6KQ3K8hsrYYbPCOu+aMUxjLdDHdpRqlbVjZrNAyQcwa1yKdQH3ThMMtvpFeRkxDh0l+b+zD5Iow5WF2QhzYh0lx2uw8eGJDu0H/NBpk44oT2Rdl52sRu7waIzNjoPJLJ3ZSdqaZMUW5CTYdc6uOiJWWUh1zkRG8bFEUfaYLPQ49j5YGcXbnGVAszKw9yd1TjFS53zlf84BAM5VNuFH41KoAREVqaLfArG22eNJpBNQiWdXFnjUaUZFqjBMo0J5dRs1KjztOPvTScs9t1SWsJzr+lM2d3jlGjKbzdi0aRNeeuklhIaGYsSIEXj55Zexbt06u/3+8pe/YN26dUhJSUFqaireeustzJs3D6dOncKZM2ewZ88efPjhh0hPT0dqaqpHMvjCNcT6JudOHI5xOQlIT4zCf642o81oXfmLrLkqdt9YE9KsIWIlD2bj/NVm61A4QoWm9ruYN2kEJo9KpENp8WQmOcfeIwYaj52SEIlhtuxCU28fjl+sQ9GUNFqjXB2mwHTbkHFUeiyNNKiqu+MwWahWhuBGYydSEyIx+f5Eu8krMnyOilRhxvjhUIQIKMhJsPPpfmabiL5r6sXpiiY6pCf7xA0LpxN05KWl99PShWiNCtNtMe0qZSjW/ngk4qMjPH4+ROZxIxOhCBEcfMjsx0NyFEh55JJZWXY+aE2E0rrik60EMBvemZcRJzm5SNo6PzMeo9JjHdwtYncB68qQijJhXRfu/MhsDL44wclZVInUOY1dJlTWtCMjOQqANWSSxOe7ci25mp/ITB5Gyz6LJ77JO6+NUNrlFKiVPxRUWzw9AycrGtxG97Dtyd6v+H2VC3lHSb6DJ8e7CknVaNQ0uklqolqO799f8wOAn+YIWlpasG/fPjz66KMAgIiICPz5z3/Gr371K7v93nrrLaxevRoxMbZIi7IyjBs3Dq2trUhKSsK2bdswduxYPP3001ixYgUiIuR3FL5QBKzv+PDZGly+0Yaqujs0JBOwri40d+JwmolKYEPEyq+3Yt6k4UiKj8T8ySOQmKCB2Wyh/lNnHxrpTPWxkdZFsW2rE8VHh+Po+VvUd26tQthNwzJZiN9UERqC9k4TrXJ4+OwtNLXfRWePGVPzEp36TrtMFhw6cR31LV048M0N6pfWx0WgsuY2fjbvPoxI1NoN8aMilahv7XJQcCEhwPnKJtxq7qITh6SSpJQilPN8iMxxsZFIiY2wixMvO1GNf525gaPn66h/nY3KIKGgUZEqVNXfwcfHr2OCLWqIdLLzJo6goaxkAtIV7ny4Yp+2OMqEICfOnZ2/cNbJy4kqIe+IJty6WFBtcwcm3Z9kV2KCTFhLdcxSskZFqjDFVp2URPOIFRabVUzORYolnqxocBndI9We7P1KzX/IwZVs7nDVUbPRTVIT1XJ8//6aHwB8MEdQVlaGrVu32m1LT09HCCkLyVxIjCAIdtsFQUBoaChWr15Nt+Xl5WHs2LE4d+4c5s2b504cSny8Vva+7vjs0GU6lBuXm4AVD2bjgqEZhpp2dPdYkKKPQlZ6vN0xOgA/L87D79/6GvWt3fjsTA1efOwBAMDVOiN2f/Y94mMjMXdyGrQaNeLjtYjWqvHVkUrsPWKAVmNdDP6CoRU/m38f/mtxHqbkJeFUeT3mTk6DsdOEXfsv4dEl+RieGIWxoxwnUgFgzuR0fPNdPfVjx8dGoqQwF79eNR6b3zmB2uYufFPRhLULRkkev2nnN7hgaEVKgrWgWZhSga8u1aO7x4L61m60d/fiF4vzAVjbXKtRY+7kNACg/4/WWu/ljdJLMHb3IjlBgwfyk6AIC8OD41MRplRAEEKgilDRfQGgvaMHB45fgyCEYPHMTLu/SaHTRUk+s4LcBCybnYtorRo6AFnp8fiIaeeSwlwsm51rJy/Zjzyv//6knLadSxmY49zJKb4my1ci+aSQs48z2jt6cPj0DcydnEblmJKXhF37L+FMRSMOn75Bz9ne0YP3Sq3bS2ZnY9L9esyZnE7vw5kcqo4eaDVqpA+PtXs/SRu1d/TQb0B8/1Jtw8rM7s8+d1fHy0GObM5grwnAQdas9Hj6bojvRdZ7I2Mff+FWERQXF6O4uNhuG5nstVgsUCgUaGpqgl6vdzg2MTERjY2NSEuzNlxzczP0ej327duHCRMm0O2CIECpVHokeEtLB63g2V+6uqxx/wnDwvGfymYkxUZArbDWn89Lj0VBVhyamowOPrz/3v8d7pr7oAkPw4qZmWhqsvrD505OQ0dnDwqy4lD6pfUjOltRj0cX5aEgKw4rC7NRkGWdLFxZmI1po/TUEv1RfhJM3SaoQ4FfLbN2wNeqWxx8h6w/sqax0xbjH42CrDi6/8gRMaht7sK5ino8cL9O0tp9dEk+TKbzmDNhOL44V4POLhM++tI6gbl0RgZqG434fx/9h068EvmIrC0tHSj9shIzxyRj/qThqGkw4r+KR6Kyph3/868rsPRaE+f2HjHA0ttLRxbiSURLb69LX69OF0Xbl31mAJAYG0FlIPfItjM5jsje1P2D64aEfYr39RaxnOJr1rV04p9fVGLx9Ay31xTfg5QP2ZlfmcwJdHT2oHhqOn6Ub+2oV8zMhMnUiyl5SfS6ZSercaaiEWOz49BrtuBMRSNS4q7ReaSWti4snZHhIKv4GlKI79/V36TOJ25PuecWI9VOnhwvviaRtaWtC2qVAstm59Lvwtm9DCahoSEujWevXEMKhQJnz55FaGgoRo4ciX/84x9ISEjA7Nmz7fZrbW3FyZMnMWfOHJw5cwZfffUVnnzySZSWluL06dOYPXs2rl27hp07d+I3v/kNVCr5EyS+Ch8FQN0E8dHhMNyyFty6fOM28tJj8cQy6/qrbHIUcZ+Q0MY5E1IxbXQSHUKPyoiHLkqNYxfrkJqgwXdVLbjZ2OkQPy0e+ouH4OLsVXZISoapZGjN1lmnflBlKM1mFgQB+ZmO1kayPgpjM+JwrrIJR87VYrTNVz67IBW1LdYVxUi4HgkDZOVjwwRb71iT65LiIx0KjIlj1cnvgiBAHabA/Mmuqy6KXQRsaQDi75cKk3Q1xCZJYcqwUKhVCvoc+oNYTjFksrS9swc/nz/K5fXE9yDlmpBKCCP5CFJuBlrKIjYC6Xprx8Ami5FEQeI6IVn3bEYzgXVlENcSiYzypjaOlGvEXXvKRY7/3ZO6PmIX042GOxiV9sOz8qebxxv8lkcwYcIEvPbaa9i5cyeMRiNefPFFhIeHY/fu3fjiiy8wbdo0jB49Gp9//jm2b9+Or7/+Glu3boVer0dBQQE++eQT7NixA4cOHcIf/vAHZGVlub8ogy8VAfngyEpBZOm4B/KTkJUyjHZ2JDmKxNmTjE4yf0BeNrUqFAeOX8fR87W4VnsHbR0mREWEoXhaOi11y8Zak4krcYcv7uzZCTK24JkzfyS5DwB0olkM+dDYGPK0xCgcu1iH3OHRaLnTTcs9iP2y4hhutvNxNikqruh407bQt6s5BGOXCUe+vYU4rYrep8lsoeWNSeflLIvU2YdNckjUYQocPV8rqezknIfdh5R3dgaZLJWKX3eHu8li9vk4K8BG9i+ekUXlZJ+PeKKaLZonlpc9jryr5ddbcaqiUZbvXdyuUspbo1HDcLMNuw6UO2QBs8eLS2XIaTsxUsrC2bNn26m2uQPfft/ssSEykPhNEQwbNgwlJSVYu3Ytli1bhvBwa1bnmDFjMG3aNABAWFgYfvzjH2Pt2rVYvXo1dR+pVCoUFxdjzZo1WLNmDYYPH+7x9X2lCNgHLV46jiQLkZhoUtQMAKrqrJOPrKVEolOu3rqDS9daAFiTuW4be2Ds7kV7Z49dCWPSicZo1XS9ATJxxi6kPTUvEfWtXbSQmasPHbBXbBAEhIaEIDnBWl44Wquys9jYMr/ij7q9sweXqtow3rYqmvhjYi1JUlvI3ctPjnMVfSTmi3M1eP+z751aws7awpUVaOwy0WgrssykuDMVl5NwleBFrhWtVVFLWwoyWUqOkWNBS2UIs+0ppWSdnUucUOYMcj6p8hpSsMET4ixsZ8ix0jUaNf7vnm8lkw7Z40lhRmfnkvNuSrWfOxnVSgVGZ8YhWafFlFH6gOn4xfDMYjeQj/xGvRFjsuPtXDak8ibJmmUrO0pFO6iVClTV38EXZ2vsFkO5eK2VZgaTDoetCZOVMoyuiTvbtszdrgPldEhOCsCxIwMS5RQWFuK0ZopaaXV3XLzWgpPljbRUMxvq5yzcURuhpIvUsLVQpMryevrysx8XG30kZY0D1g80WadFni1805Xrg8VVx8iWZhifq3PZmZKRgziEWOparKUtpw3kWNBywwrFStbVc+kVgE+OX3O5n7PyGs6uTUKUxWsKOEOO4tJo1NBFqSVHUWyNoZzU6H67YqTeZ7nKddLoZPSa3T/3wYIrAjf8x9BMy0eLXTKkguRDD2aDXXdWHxuOaI0KitBQZKUMk+y0Fj+QjnE5CYjWquhLTBb8FneqRIGIyzbkZcQgQhVGFyoh1R5Jnf8rN2+7rJlCfPgke5hUw2RD/ZyFO4YpQh1GPGLEQ/OyE9W4YGhGZU07HXlIWbzsx+Vs/oDt8MiHRorjRUUqkZYYRRcGZ2vysLhSVJ6UESDzEeT/rq7lztJmry/XgvbE3yxXaXx5/pbDKEvquqqwUMRo1Q7vOUFqRC23I5ZjSGg0aoQKAq0xxcLm6HizFoEcl59cY8dXcxn+gq9Z7AbygPWx4SjISYCxy4Qe2wIy5dVtGJ0VR1/AmWOSceVGGy4YWtHYZi2dQLJX2aiEksJcXKtuQdnJavSYLDSTktTikYpgEGecArDLUFxZmG23L8k6JmvDLp6eYZfxCsCunAF7bWdIySAue8DCRv0A9usTk0xm0l7kXOS+xW0mzkCVaiOSsUrmSsj1pJ6BnCxVudEczjKp+0NUpAqPLsqTJa8nsko9QylIZJur/YhMpJ3ZMhJSWbTs8/VnZiwb7UWu6w1ysqqDhaAfEZDJnmu22uF0+UFmhSbWFcKuNHimawAAFglJREFUMJWu16Kq1ojMlGG0/G1UpHUxDTbLkKzeRKwv1uecqtM4+H+Ja4pdT4DNqFUrfyjSFTcs3C5BR6rIFynPK1WIjHURyMl6ZRFb9mQZS1KJVexekppsZn9no6jE+2o0anx64jq1AGfaspbZtvFnZqZcq9wTy9AfE4pyzylO0COIrWRnbjFxIIOrEZ3UeV3hbIUy8bWlEvU8wZeRPf4YEfhqdTKAjwjcIrbMCM6smqhIFbWMtu89j/LqNvzzi0palE5skZGf4vMT3NWPYa0wV5Bzk6JexAp3V+vm8Okb2HvEgCs32uioQS5iS1Usq1qloLWCxBa/1E9xcbMekwU9JguMXSbo4FjzRnw9udawFKyVydbNcXav9yri91FcGJEgVWPJVft7Yn2z+0olWPXnObMMxDPtT/2ggRyxBP2IAJAOn5OjgdlQwPjoCGrVHvn2FpJiIxxWbyLndFcDn7VUWu/cxVul3+FmUwfSXcS5k2sQvynxO5Poo0xbnRlSgoKcJzc9DldvttqNWADfWCPu6uCIfxeHlrLzJuPu09O69P3150pBrEzxZLqnbRHovmKCMznJvABZD8GZ31+qrb2Zk5GC3VdqRBBooZmA8/bszyiVnQzvr6uNTxbLQOpDl/Pxk1BA9iFJhTq6ipd290HtOlCOS1VtMNhWb3I35BaHdJadqLYmwSlDcexivcM54mIjkanXyg6b86RTdPXByilu5q5D8CW0Tr2obg7g2cfsC0XgCyXs7hzO5BQHLvjKxeZJ583uGyiK1dv27I/7yVXBSk/hriEZSNWk92ZYZuwywdhpwrjcBDqRxZ7fk3MRVs3JRW/v9xiRqHUogexMdqlrpOmtcxBSw2mpIbKz4Xd/h6uuSvXKkctfsNcSlwv3lStCDmykF+C9S6A/z2kg7zeQcOXG8bY9+/MOD+Rz4CMCOKaL1zZ3SFqG7vjinHUxl4bWLodFq8Wp+FKWhVSmZE5qNAonDEe+LWxUbJm6q6bIrgUstXSf2JJxlbwkvpf+uGDcVZ0Uy+MuY9efeGLN9teC9SR23xWeVEkVI/d+vR1Je8JAjghcjfzktqcv79+XLjDuGnKCVPyztWzEHVyqaoM2QkkXOZf7cIl/dcL9ekwfnSTp8nD1srnLlJR6Gcm5xbXrCe5eJvGHJieTUnw+b2q0OFtTVozcjN1Aob8dl9i15y2ePndvcFf7yBdupYFUBK46e7nt6c/Itf7AFYETpB6YWqlATWMHrt66g8zkKJpI5Ulm56j0WEwbm+o0y9DVy+Ys0cqZD118bW+sB2cLf+QOj8Y7B8tRwxTFc4YnL7+ncsrN2PXUEvO15Urw1DKUU2/HH/iig3VX+2iojQj60/biml2BUmyOwBWBE5w9MLKwel5GHI2blvtwXbkx3LlcAO+jl7zBmazkun//9DIuVbXhqpNJahZ/vvxEHncZu54UDHO2vy9wZxmKZRosC9IXHaynkUPeECiTxe6QqtkVSPDJYhmIMxWXzsjA3Ik/FMJzNuEjnlwiPnytRk3rvxMGIibYk5hlsaziY1fNyUWv5QrS9FFDYtJQamLNVZu7y2L2hzxSMrmaEBzMNWw5wUXQjghYS4yUf65v7cSxi/W0vo6z9QGcWXOu3BgDMWT0xLqk8hQMx79OVeM70VoLUZEqTM9PluXHl1Ods7+4swylLDG5Pl9fWuXuLEN3uRUs/hwtDDVLO5CQGmkGopwsfETgBNYSO3y2BoBjiKU4NNPYacKhUzfRY7Jg7sTh6DFZsHRGBt2fjByitWqHVY8GIhTSk3AzIs9Xl+qx94gBeRnyOpqhZKXKbfOBDNPzR92gocBQem/ccS/WKAraEQFriZEKoQsfyMD4XJ3DxBcJzVQpQ9F027owTW1LJy1jLB49+Dv5Sc49iXHmL89Nj4MiRMD8ydaqpGxNIymkrFQ2U9pXox2xvP60uHzp1/WlnP70N/dHTm8m2r0d3QSipS010gxEOVn4iEAG568244KhFSPTmu2SiYj1Zuwy0fVbSR0aglSdHF8sQO1rC+rwWWuOQ4/JYlejJ1qrdkikcnVtKSvVH6MdsdXV3tHjkEzHGRy8sYjvpdHNvVh3iisCuH9JnWWdsi+D3Bddbgc/mMNPV9eW+ggGYrKVFMeTkslX3EvuC3/iTad+L3ae9xJB6xoC5IV0ykVufRS5YYW+nlx25r6RktXTa3sauikHsVuEuLD62x7+Din1tYvA3/kO3jCQIZKB7nIhBLqc3DXkAn9Z3cSNIVXSWE5YYUFOAv75RaVXC5w7w9OFWDxpD3HJaDaUFvBN27IurP4gN6TUE9iRhK7fEtpzL05McgKPoFYEUh++L9wDbI1/ucXDWFl+KDpWiWdXFnglw0DCruAFgM6nAIHnE3bV2XvrvvD1/BDLveRbDwS4+0+aoFYEUh++LywwsgxgQU4CRqbZTy47Oz8ry6o5uQAqbT+HBuQee0yWgLZg/eGr9mdnLTeZMdAJFHn5CEuaoFYEUpCPWarks1ykInHE53fVaSTHa+xGAoHyEblCHGF1r1mwrp7BYEyEDoUOjW0zZ/IO9LvNR1jScEUggnzU7PKOvvzQvOk0hsJHT7hXo0MC7RkU5CTgyo02u3UvAg3x0qPsT6l9BqJd79X3s79wRWBDbJkEkuUwmLIMhdHIQBBI7wPgPPclkGDbzFkHHGjtGqyEDrYAgQKxTI5drANgbzmUnayG0U+hYcYuk1fn9/Y4TxG3y0BfP1Ag70OgKMOZY5KxsjA7oDtQOW0WaO0arAT1iIC1dgdr6Crn/FL7DNSQOlCG9Bx7uIuD40u8VgS1tbXYsGEDWlpakJmZiVdeeQUajfQQ9fjx43j77bfxt7/9DQAgCAJefvllHDlyBKGhodi8eTMmTpzorSheI+7MBmPoKuf8UvsM1JB6sIb03CU1sPi7vfnzDGy8dg1t2rQJa9aswaFDh5Cfn4833njDYZ++vj688847eO6559DX10e3f/rppzAYDDh48CB27NiBjRs3ore311tRvEbO8NrfQ1dvh8+DPaT29/WduaRcMdTdVYMlv7HLhF0Hyj1ub0/w5nlyBg6vFIHZbMbp06dRVFQEACgpKcGhQ4cc9jMYDDAYDNi8ebPd9qNHj2LhwoUIDQ1FZmYmkpOT8e2333ojSr8Y7M6U4xxvfOBDvbMZLPmPXazDBUMrxmbH+XXkG+hzGsGMV66htrY2aLVahIVZD9fpdGhoaHDYLzc3F1u2bMHJkyfttjc2NkKv19PfdTod6uvrvRFlwAmWIe5g36dcH7iceZ6hwmDJL47u8Qd8TiOwcasIysrKsHXrVrtt6enpCAkJsdsm/t0VfX19dvsLgoDQUM8GJ/HxWo/295b2jh4cPn0DcyenIVqrxldHKukSjyWFzjN/dbqoAZHPF0jJKvc+BxI5cvq6xIM3ePvsdfB9iQqX17PJOdDX9ZSh8i0NFTmlcKsIiouLUVxcbLfNbDZj6tSpsFgsUCgUaGpqsrPw3ZGUlITGxkb6e3Nzs0fHA0BLSwf6+vpZflQGJLGso7MHxVPTUZAVh5WF2SjIikNTk1HyGJ0uyunfxAy25e1MVjn3OZAMdTkDDS6nbwl0OUNDQ1waz17NESiVSkyaNAkHDx4EAOzbtw+zZs2SffysWbOwf/9+WCwWVFdX4/r16xgzZow3ovgd1rfpj057sPzCZGKyvaNH8u9DZf5kqMjJ4QQyXoePvvjii3j++efx5ptvIjk5Ga+++ioAYPfu3WhsbMT69eudHrtgwQJcuHABS5cuBQBs2bIF4eHh3oriV8SJZb6OnR8svzBRQFqNGj/KTxrQa3MGlsEedXICnxBB6O/yLoPDQLmGWOR+UAM1TOzPB06OXTY7F6buwA+3DPShNyEQ5SQGzMrCbGrABKKcUnA5fYM711BQZxZ7SqBFPvQnu5fcS7RWjaYhoAg43jPUo6k4/ocrgiEM/8A5cgg0A4YTeHBFMIThHziHw/EFvProEGWol1PgcDiBA1cEQ5ShXk6Bw+EEDtw1NETx9fwADzHkcIIXPiIYovg6kYqPMDic4IWPCAKMwbLMeQQShxO88BFBgDFYljkv1cDhBC98RBBgcMucw+EMNFwRBBg8N4DD4Qw03DXE4XA4QQ5XBBwOhxPkcEXA4XA4QQ5XBBwOhxPkcEXA4XA4QQ5XBBwOhxPkcEXA4XA4QQ5XBBwOhxPkcEXA4XA4QQ5XBBwOhxPkcEXA4XA4QQ5XBBwOhxPkcEXA4XA4QQ5XBBwOhxPkcEXA4XA4QQ5XBBwOhxPkeK0IamtrsXbtWixYsABPPfUUOjs7ne57/PhxrFu3jv5uNpsxYcIELFu2jP6zWCzeisIZAIxdJpSdrIaxyzTYonA4HB/jtSLYtGkT1qxZg0OHDiE/Px9vvPGGwz59fX1455138Nxzz6Gvr49uv3LlCsaPH4/S0lL6T6FQeCsKZwAYrLWUORyO//FKEZjNZpw+fRpFRUUAgJKSEhw6dMhhP4PBAIPBgM2bN9ttv3jxIlpbW1FSUoKf/vSnOHXqlDdicAaQmWOSsbIwm6+lzOHcg3i1ZnFbWxu0Wi3CwqyH63Q6NDQ0OOyXm5uLLVu24OTJk3bbQ0JCMHfuXDzxxBOorKzEL3/5S+zfvx9xcXGyZYiP13oj+oCh00UNtgiykSOrDkBWerz/hXElwxBpUy6nb+Fy+h+3iqCsrAxbt26125aeno6QkBC7beLfXbF69Wr6/7y8PIwdOxbnzp3DvHnzZJ+jpaUDfX2C7P0HEp0uCk1NxsEWQxZDRVYup2/hcvqWQJczNDTEpfHsVhEUFxejuLjYbpvZbMbUqVNhsVigUCjQ1NQEvV4vW6h9+/ZhwoQJSEtLAwAIggClUin7eA6Hw+H4Dq/mCJRKJSZNmoSDBw8CsHbss2bNkn38lStX8M477wAArl27hoqKCkycONEbUTgcDofTT7yOGnrxxRexZ88eLFy4EGfOnMGzzz4LANi9ezdee+01l8c+/fTTaG1txeLFi7F+/Xq89NJL0GoD2+fP4XA49yohgiAEpqPdDXyOwDcMFVm5nL6Fy+lbAl1Od3MEPLOYw+FwghyuCDgcDifI8SqPIBAIDZUfrjoYBLp8LENFVi6nb+Fy+pZAltOdbEN2joDD4XA4voG7hjgcDifI4YqAw+FwghyuCDgcDifI4YqAw+FwghyuCDgcDifI4YqAw+FwghyuCDgcDifI4YqAw+FwghyuCDgcDifI4YrAS2pra7F27VosWLAATz31FDo7Ox32MZlM2LBhA4qLi7FixQoYDAYA1oV4duzYgeXLl6OoqAj79u0LSDkJvb29WLVqFT766KOAlLOzsxPr16/HkiVLsGTJEhw4cMAvMu7fvx8LFy7E/Pnz8f777zv8vaKiAiUlJSgqKsILL7yA3t5e2fcWCHKePXsWP/nJT7Bs2TKsW7cOt27dCkg5CeXl5cjPz/erjP2Rs7GxEY8//jiWL1+O1atXo6amxu+yeo3A8YrHH39c+OSTTwRBEITXX39dePnllx322blzp/D73/9eEARBOHXqlLBy5UpBEARh3759wpo1a4Senh6hsbFReOCBB4T29vaAk5Owfft2YcqUKcKHH37oFxn7K+err74qbNu2TRAEQWhubhZmzJghNDU1+VS++vp6obCwUGhraxM6OzuFJUuWCJWVlXb7LFq0SPj2228FQRCEjRs3Cu+//77sewsEOQsLC4WKigpBEARh7969wpNPPhmQcgqCIHR1dQmrV68W7rvvPr/J2F85161bJ3zwwQeCIAjCBx98IKxfv96vsvYHPiLwArPZjNOnT6OoqAgAUFJSgkOHDjns9+WXX2Lp0qUAgMmTJ6O1tRW1tbUoKyvDI488ApVKBZ1Ohw8++ADh4eEBJycAnDt3DpcvX0ZhYaHP5fOVnFOmTMHPf/5zAEB8fDxiYmLQ3NzsUxm//vprTJs2DTExMYiMjERRUZGdjLdu3cLdu3dRUFBgdw9y722w5TSZTFi/fj1GjRoFABg5ciTq6uoCTk7Ctm3bsG7dOr/J1185W1tbcfnyZbo++0MPPUQX7wpEuCLwgra2Nmi1WoSFWYu36nQ6NDQ0OOzX2NgInU5Hf9fpdKivr0d1dTUMBgN+8YtfYMWKFSgvL4dKpQo4OTs6OrB161Zs3rzZ57L5Us4ZM2YgJSUFAHDw4EGYTCbk5OT4VEbxtfV6vZ2MUrI1NDTIvrfBllOlUmHZsmUAgL6+Prz++uuYN29ewMkJAIcPH8bdu3exYMECv8nXXzlv3ryJlJQUbNu2DQ899BCeeeaZgF6XfciWoR4oysrKsHXrVrtt6enpCAmxL+sq/h2wzgWw2wVBQGhoKCwWC65cuYJdu3ahubkZP/vZz5CXl4eMjIyAknPTpk144oknkJCQ4LVcAyEne+4//elP2LlzJ+14fUVfX5/Dtdnfnf1dvB8gfW+DLSfBZDLh+eefR29vL5544omAk7OpqQlvvvkm3n33Xb/J5gs5e3t7UV5ejl//+tfYuHEj9u7di+effx7vvffegMjtKVwRuKG4uBjFxcV228xmM6ZOnQqLxQKFQoGmpibo9XqHYxMTE9HY2Ii0tDQAQHNzM/R6PRISErBgwQIolUokJydj3LhxKC8v75ci8LWcOp0O33zzDb7//nv89a9/RV1dHU6cOIGwsDDqngkEOcl+7733Hnbt2oVdu3Zh5MiRXsvnjKSkJJw5c4b+LpYxKSkJTU1N9HciW1xcHIxGo9t7G2w5Aeuk+1NPPYWYmBi8+eabfrVgvZXzyy+/xO3bt7F27Vr6t2XLluH999/3y7rn3sqp0+mg0WioS3Xx4sX44x//6HP5fAV3DXmBUqnEpEmTcPDgQQDAvn37MGvWLIf9HnzwQZSWlgIAzpw5A7VajZSUFBQWFqKsrAyCIKCtrQ0XLlzA/fffH1Bypqam4tixYygtLUVpaSnmzJmDZ555pl9KwB9ypqSk4PPPP8e7776L3bt3+0UJAMD06dPxzTffoLW1Fd3d3fjss8/sZExNTYVarcbZs2cBAKWlpZg1a5bsextsOQFgw4YNSE9Px/bt2/3iqvSFnCtXrsTnn39O30vyN38ogf7ImZaWhqSkJBw9ehQAcOTIEYwePdovMvqEgZ+fvjeoqakRHn74YaG4uFh45JFHhNu3bwuCYI0O2L59uyAIgnD37l3ht7/9rbBw4UJh+fLlwqVLlwRBEASTySRs2bJFWLhwoVBUVCTs2bMnIOVk+d3vfufXqKH+yLlkyRJhxowZwtKlS+m/Cxcu+FzGjz/+WFi0aJEwf/584e233xYEQRAee+wxeq2KigrhoYceEoqKioTnnntO6OnpcXlv/sIbOb/77jvhvvvuExYuXEjb8LHHHgs4OcX4O2qoP3IaDAbh4YcfFhYtWiSsWrVKqKqq8rus3sJXKONwOJwgh7uGOBwOJ8jhioDD4XCCHK4IOBwOJ8jhioDD4XCCHK4IOBwOJ8jhioDD4XCCHK4IOBwOJ8jhioDD4XCCnP8PMCrqSQQsG2sAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(net[2].weight.data.reshape(-1), net[2].weight.grad.reshape(-1), s=1);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loss Functions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order discover a good set of parameters using optimization, we need to specify a loss function to optimize.\n", "\n", "The loss function $\\ell(X_\\text{out}, Y_\\text{tgt})$ compares the actual network output $X_\\text{out}$ with a corresponding target value $Y_\\text{tgt}$ and approaches some minimum value as their agreement improves.\n", "\n", "A loss function must be scalar valued since we need a single gradient for each parameter to implement gradient descent,\n", "$$\n", "\\theta \\rightarrow \\theta - \\eta\\,\\frac{\\partial\\ell}{\\partial\\theta} \\; .\n", "$$\n", "Note that the loss normalization is degenerate with the learning rate $\\eta$.\n", "\n", "Our choice of loss function is primarily driven by the type of problem we are solving: regression or classification. We introduce the most obvious choices below but there are lots of reasonable variations (see [here](https://pytorch.org/docs/stable/nn.html#id51) for the complete PyTorch list)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Regression Loss" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For regression, the $L_2$ norm is a popular choice,\n", "$$\n", "L_2 = \\frac{1}{2}\\, \\left|\n", "X_\\text{out} - Y_\\text{tgt}\\right|^2 \\; .\n", "$$\n", "Optimizing the $L_2$ norm is equivalent to finding the maximum-likelihood (ML) point estimate for the network parameters (weights and biases) if we assume that the uncertainties in $Y_\\text{tgt}$ are \"homoscedastic\" (drawn from the same Gaussian distribution).\n", "\n", "In PyTorch, the $L_2$ norm is implemented as [torch.nn.MSELoss](https://pytorch.org/docs/stable/nn.html#mseloss):" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [], "source": [ "Y = torch.zeros_like(Xout)\n", "loss = torch.nn.MSELoss()(Xout, Y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In case you actually have a reasonable estimate $\\sigma_Y^i$ of the $i$-th sample's target uncertainty, a better loss function is the $\\chi^2$ statistic:\n", "$$\n", "\\chi^2 = \\sum_{i=1}^N\\, \\left( \\frac{X_\\text{out}^i - Y_\\text{tgt}^i}{\\sigma_Y^i}\\right)^2 \\; .\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Binary Classification Loss" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For binary classification problems, the L2 norm can also be used but the binary [cross entropy](https://en.wikipedia.org/wiki/Cross_entropy) between the target and output probability distributions is often a better choice:\n", "$$\n", "\\text{BCE} \\equiv -\\sum_{i=1}^N\\, \\left[\n", "Y_{tgt}^i \\log \\phi_S(X_\\text{out}^i) + (1 - Y_\\text{tgt}^i) \\log (1 - \\phi_S(X_\\text{out}^i)) \\right]\n", "$$\n", "where $\\phi_S$ is the sigmoid (aka logistic) activation function used to coerce arbitrary real values into the range $[0,1]$ required for a probability. The value $X_\\text{out}^i$ feeding a sigmoid like this is known as a [logit](https://en.wikipedia.org/wiki/Logit).\n", "\n", "The equivalent PyTorch code uses [torch.nn.BCELoss](https://pytorch.org/docs/stable/nn.html#bceloss):" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "Xout = torch.ones(10)\n", "Y = torch.zeros(10)\n", "loss = torch.nn.BCELoss()(Xout, Y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The cross entropy is inspired by information theory and closely related to the KL divergence we met [earlier](Variational.ipynb). With this approach, our assumptions are that:\n", " - The target values in $Y_{tgt}$ are all either 0 or 1.\n", " - The network output values in $X_{out}$ are continuous and $\\phi_S(y^{out}_i)$ is interpreted as the corresponding probability that the output is 1.\n", " \n", "Note that *something* like the second assumption is necessary to reconcile the different domains of the data and prediction.\n", "\n", "With these assumptions, the likelihood is:\n", "$$\n", "P(Y_\\text{tgt}\\mid X_\\text{out}) = \\begin{cases}\n", "\\phi_S(X_\\text{out}) & Y_\\text{tgt} = 1 \\\\\n", "1 - \\phi_S(X_\\text{out}) & Y_\\text{tgt} = 0\n", "\\end{cases}\n", "$$\n", "Take a minute to convince yourself that the following expression is equivalent (the case $\\phi_S(X_\\text{out}(\\Theta)) = Y_\\text{tgt} = 0$ requires some care since $0^0$ is indeterminate):\n", "$$\n", "P(Y_\\text{tgt}\\mid X_\\text{out}(\\Theta)) = \\left[\\phi_S(X_\\text{out}(\\Theta))\\right]^{Y_\\text{tgt}}\\,\n", "\\left[1 - \\phi_S(X_\\text{out}(\\Theta))\\right]^{1-Y_\\text{tgt}} \\; .\n", "$$\n", "Using this form, you can show that the cross entropy loss equals the negative-log-likelihood of the $N$ samples of training data so optimizing BCE is equivalent to finding the ML point estimate of the network parameters under the assumptions above.\n", "\n", "For fixed training data, optimizing BCE is also equivalent to minimizing the KL divergence of the network's predicted discrete probability distribution with respect to the empirical discrete probability distribution of the training data. Therefore, training a binary classification network using the cross-entropy loss is effectively performing a variational inference (VI) to find the network probabilities that are closest to the empirical training probabilities." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Multi-category Classification Loss" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How can we generalize the binary classification cross-entropy loss to problems with more than two categories? The usual approach is to increase the number of output nodes from 1 to the number of categories $C$,\n", "but we can not directly interpret their values as category probabilities since there is no way to ensure that they sum to one. We could simply require that they are all non-negative and renormalize, but a more more robust approach is to convert the vector of output values $X_\\text{out}$ to a corresponding vector of probabilities $\\mathbf{p}$ for category $j = 1, 2, \\ldots, C$ using the **softmax function**,\n", "$$\n", "\\mathbf{p}(X_\\text{out}) \\equiv \\frac{1}{\\sum_{k=1}^C\\, \\exp(X_\\text{out}^k)}\\,\n", "[ \\exp(X_\\text{out}^1), \\exp(X_\\text{out}^2), \\ldots, \\exp(X_\\text{out}^C) ] \\; ,\n", "$$\n", "which works fine with positive or negative outputs $X_\\text{out}^j$. Note that softmax generalizes the sigmoid function $\\phi_S$ in the following sense:\n", "$$\n", "\\mathbf{p}([y_1, y_2]) = [\\,\\phi_S(y_1-y_2)\\,,\\, 1 - \\phi_S(y_1-y_2)\\,] \\; .\n", "$$" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [], "source": [ "def softmax(y):\n", " # subtract out max(y) improve the numerical accuracy\n", " expy = np.exp(y - np.max(y))\n", " return expy / expy.sum()" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.26538793, 0.01321289, 0.72139918])" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "softmax([2, -1, 3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The softmax function effectively implements a *winner takes all* policy, similar to the sigmoid activation $\\phi_S$, as illustrated in the plot below where:\n", " - the color scale indicates, from left to right, $p_1, p_2$ and $p_3$ for three categories,\n", " - $y_1$ and $y_2$ are varied over the same range, and\n", " - $y_3$ is fixed to the middle of this range." ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoYAAADaCAYAAAA7WYPcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9e7glVXnn/1lV+372uXRDc+tuToOAKBiVoEGDmIANDTRpR2MSkzG/zIxmHpNozJOZjGJmMJnRJ5Ob/uaJTia/mUnGSGYkGiHKZUS8InKRCAgNLSDd9AX6dvpc971q/f5Ya1Wt2rv2Ofvczz5nffvZT9Vea1XtdXbvVfXW932/7yuklBIHBwcHBwcHB4cND2+1J+Dg4ODg4ODg4LA24AxDBwcHBwcHBwcHwBmGDg4ODg4ODg4OGs4wdHBwcHBwcHBwAJxh6ODg4ODg4ODgoOEMQwcHBwcHBwcHB2AVDMPp6Wl2797NoUOHAPjIRz7Ctddey549e9izZw/33ntvxzFPP/0073jHO7juuuv46Ec/SqvVWulpOzisKty6cXBYGNzacXCYJ+QK4rHHHpO7d++Wl1xyiTx48KCUUsrdu3fLo0ePznrcjTfeKH/wgx9IKaX8yEc+Im+99dZln6uDw1qBWzcODguDWzsODvPHijKGt912G7fccgtnnHEGANVqlSNHjnDzzTdz00038V/+y38hDMPEMYcPH6ZWq/G6170OgHe84x3cc889KzltB4dVhVs3Dg4Lg1s7Dg7zx4oahh//+Me5/PLLo/cnTpzgiiuu4BOf+AS33XYb3//+9/nCF76QOObYsWNs2bIler9lyxaOHj26YnN2cFhtuHXj4LAwuLXj4DB/ZFbzw7dv386nP/3p6P173vMebr/9dn7hF34hagvDECFE9F5KmXjfC5qnDkEYLH7CDksHGaY3p7XbVRvb/x9lCKFMHitDCFrJzwlbcVurCWFTdTfVlmYNmvV4H5D1OlRnVFu1qrc1ZKWi+qfVNpyqEY6rY5sTIY1JXw2dyjFVzlOr5vCLm3jDl/9j+ncxT6zUugF4y1v3cPjwS4ub8AbGO/7Z2wD4hy99bcnOOdf/oyDut8d6HceJaKwn4vG+8BLH+sKLjs0IP+r3PbXNehlywtf96paS9zIURBaAoqfaCiJDETWurMeVpWBQO64GAxjS7N2QDJg8o8hZk1Octq3M6P/5s9m/lB7h1k5/YDnWzUbE1q1n851v3THv41bVMNy3bx/79+/nuuuuA9QCzGSSUzrrrLM4fvx49P7EiRORW6BnhIEyDBxWH10MQtWVZhRabbZRGBl8YWwQmuODZtyvjUEZNKHZ0OOaSLPf0AZfoxbty5puq0yDNgKJjMEZ5KQyFsMJ1RaM1WieUp9XH88wM5EH4KXsEA+9cTuZmRavemii6989X6zYugEOH36JAwcOLW7CGxgnT54EWJLvcD4GYft42yg044QQUXuaEWgMQE94ZD0/asvo/Zw2+HIiQ9YYf542BkWWpt5vaSMwJIOv91vaGJTSQ+hlnQlCclKt14nzCzx6+emc/+MKV08emfXvng/c2ukPLOW6cZg/VjVdjZSST3ziE0xMTNBsNvn85z/Pzp07E2O2bt1KPp/n0UcfBeCOO+7gqquuWo3pOjj0jMmhAg/tvhCAi752iPnzDd3h1o3DesbYaJlHd57H0MkqP/GDF5f03G7tODjMjVVlDC+++GJ+/dd/nXe/+920Wi2uvfZadu/eDcD73vc+PvjBD/Ka17yGP/3TP+X3f//3mZ6e5pJLLuFXf/VXV3PaDvPFLCyh6l4YUwjafWwzhWaMzRSCYgsj93EjyRQCNKpJphAUS2gxhQBycibBFAI0T4XUx9VSmpnIK6bwmgsJgUvv2E9jPKQyvHTPYG7d9A8++7dfXvQ5enFj9uI+tplC0yc62EGRYAoBsl7sPs54foIpVP2ZBFMIUPKyFHX/gL7NDIoMQ5opHDbbAEYCtX5HZIv6+QWe2bmdTacqvPWbT7OpVCE3VJrz7+8Vbu30B5Zi3TgsHEJKO4BrfaJ58oBzJa8mNphh+O0rXsnY2WVefcd+SqcajIsMrdEz+KWHPjXr97AWcf4Fb3TunFXGRjEMh0TA4+8+n2w94Opv7iXXDBgYrlN85emc/rnb5vwO1hrc2nFYbYyObuPHzz087+NWlTF0WOdYrEEInUahbRCCem8bhABBK2kQQmdcoWUQgo4rtA1CvbUNQlBxhbZBCMm4wumZPKP3vczZpQz1qZCGyDDue0h/1q/CYZ3iTVf8BADfe/CJBR2/UKEJdBqFtkFoxtsGIZgYw9ggNG12XKFtEIKKK7QNQoCiyCQMQoAh6SUMQlBG4YiOKxzxG1z9jafJNlpsKqk1mB9pkV1Ctt2hP7DYdeOwODjD0GHpMYdBqIb0yBIm2hYmNAGQDUt1vIRCE1As4ZHMED96w9nsuO9lplseTIWM++qGNuGDEHN/Jw7rDxdeOArM/wa3VoQm0MkSpglNbIMQFEtoG4SgWELbIATlPq6eX+DktiG2Pv485UwdMsogBMhu8vBG8rN+Fw7rDwtdNw5LA2cYOjgsEpNDBR66RglNthYzMOWMQAeHXjA2WuaZndsZOlmllXHMoIPDWoAzDB2WBj2whGrYwphC0O5jk4vQHjdbPKFxGTfryxJPODlU4L5rLkECl96+n3obUwgwIUIyYt2H8josAZYqntCMW2w8IXS6j5dKaDI2WuaZXbHQZGSgmmAKAfzNBbzBwpzfiYODw9LBGYYOi8dSuI5BGXsdbZb7OGjFBqGVwLojnrBVR7b0fjf38RLEEx7JKPWxBEbvOERjQhmFtkEIMCni/GwODt2wlPGEZsxi4wmh0328FPGEVUt9fP33HyNfClQ8oWUQAnjDJUR56VTJDg4Oc8MZhg4OC4SQUJhp8op7D9OYcO5jh060mu6BIA1eKBk5VuGt33mafMlVpXJIwq2b1YUzDB3mj6VyGxvMx33cVuouITRpKXZQthpJphA63ceLEJocCwfIV1oEgeTSv3+BCa08BuU+tplCgCnZooS7+W1E/N3/uXvOMcshNDF9ixWaQKf7eDFCk9xgSKHSYujUFOd//QTl4XrCfWwzhQBiaABRcozhRkMv68Zh+eCifR0c5oHJoQLffefFPHf5WQBLWtHEwWE9Y2y0zLfefQlHR4cAt3YcHNYqHGPo0Bt6ZAnV0AXkJ4Q4BQ10jys0TKFhBINWkimEpNCkpuMGG/V0ocmkapPTlXkJTQb3TTOuWZL2uEKbKQSYoYXnYgw3JN5y5WUAfOf+f0rtX2hc4VxCEzN+sUIT6IwrXKzQZHRyjIFhtWbb4wptphBAlAeg6MQnGw1zrRuH5YUzDB26Yx7GYHzIElUxifoCS3XcipNZG5dyN6GJMQiN+7hWSQpNLIMQFi40gU73sW0QAlRli4yrvLMhcd55W4HOG9xyC02g0328EKEJdLqPl0poAp3u44RBCFAqQbE463flsP7Qbd04rAycYejgMAcCT/DIDRcAKiWNE5o4OPSG6nCOH+w8j6GTVd76TSc0cXDoBzjD0CHGAhhCdViX47oxhaYvrbydzRSCLm+n2bagmWAKoYvQpDaTZApBp6PRruTpSoIphO5Ck8lpxRie9+2XKEw0OphC6HQf20whQCVskpXN9O/IYUNhpYQmpm2xQhPodB/3KjQZ8hsMTTf4yUf2s/3Fk2wqVdLzFNpCE5spNNviwKzfmYODw9LCiU8cHLpgcqjAy+cNA7D5wDSl8cYqz8jBoT8wNlpm8jTlAn7F88fINR1T6ODQL3CM4UbEApnBztPMMx2N3Z9W91iGHQmsO+IKexGa1KoJphBQbOG02a8QnlJjg3F1vm5CE0LI768xJRTb0i2BtR1XaDOFAFXZpOBiDDckqpVatL+SQhPojCtciNAEOuMKexWanHl0krd+45muQhNIiSu0mUKAUhmRd7WSNxrsdeOw8nCG4XrCEhl8s39ED25jg7nyE84lNIEO93FkENa1q7jVmFVokuo+PjWTMAgBamNZKlM5oE1o8uVDTAX+nHkKbfexbRAC1MImLadK3pD4+y/ei7DcvGlYDqGJOtZbkNCkoI3Fbu7jXoUm1zzxJPnh7kITSHEfWwYhgCgUIetUyRsNf//Fe1d7ChsazjB0cLAwOVTgoWsuBJzQxMFhPhgbLfPMzu1OaOLg0OfYGIahDFeETVuP6MoQqs7OtjS3McwuNDGu4jBMMoXQ6T42TGFaRZMuQhNQ+QrDCTUuGK8nmEKAylSOyek8z19wBpLYKJxLaAKd7mObKQRohi0a0t0kNxqEEFz9M28A4OvffCR9TA/u44UITaDTfTyb0MSwhAWRYUC3dXMfpwlNBv0Gz716mE2nKrz1m0/3JDSBFPexzRQC5IqQ7U9X8lxMsUN3zLVuVhJSytWewopjYxiGDg5zwCz98x87yvDeSXLVgIZwy8Nhcdi67czVnsKyQ6KqmLz2vv0MDDWd0MRh0dgI62YtY8VVydPT0+zevZtDhw4B8PnPf57du3dz00038ZGPfIRGo1P5+aUvfYkrr7ySPXv2sGfPHj75yU+u9LQ3BGQYdry6DEyvZGLHFJoYQv2SMozjCg1bGDTVy4wxQpOgqWIHW40orjBiCxs1/arGbGGtEscVGrZwuqLiCienI7YwGK9HbGFtLBvFFb4shrh712t5aXiQqSBHpSYY1/WPJ/w4rtCwhVOyFcUVGrawEjYjtrAWqlczbGm2UG0XA7du+gu9CE1sJtBmA9vZQtNv+ozQxMQVGrbQ06+s50dsYcbzI7YwJzJRXKFhCwsiQ0FkKAolOhnwsgyQieIKDVs4bF5aaGLiCsNzC+x95w7ym1tsKtYYKVQpDTYioYmJKzRsoRgaiOIKI7YwepURhaJiC3PmVVh0jKFbOw6LgVl/c73WE1aUEnn88cf5/d//ffbv3w/ACy+8wP/4H/+Df/iHf2BgYIAPf/jD/N3f/R2/9mu/ljjuySef5MMf/jC7d+9eyemue8zqJu4c3NnWq9vYaotcxDKMhSZ2nxaXSO0+plmHpr5wG4MQS2hSmVEvQJrKJtNxZZNu7uM0ocm09GlqgxB6E5pAp/vYGIEN3d8IWwTz+a7b4NZN/2A58hQuRGgCne7jdqGJMQiBhPt4rjyFwzoswhaalAYa5PMBhc1qHaQKTcql9DyFtvs4Z1zI2hjMFRHZ7Gxf6axwa8dhpZC29vvVDb2ijOFtt93GLbfcwhlnnAFALpfjlltuoVwuI4Tgoosu4siRIx3H/fCHP+RLX/oSN910E//m3/wbJiYmVnLaDusUk0MFHropFprk12ieQrduHNYaxkbLcUWTrz9NvrU23cdu7Tg4zB8rahh+/OMf5/LLL4/eb926lZ/+6Z8GYGxsjFtvvZVrrrmm47gtW7bwG7/xG/zjP/4jZ599Nn/4h3+4YnPuV6S5hXtyEydPMrfb2IyL+lLcxraL2JwvaCGDJjJoKnFJqw6tBrJVV2xhsx6zhZb7WNa0+9gwhZUZZKWi2ELLfRxOVJX7eKxOMFaneVIm3McveYPcd80lSASvvv1AJDRpdx9PpriPq3J293FD6leoXq0wIFiE+Mmtm7WPbu6kqckZpiYVo90uNElzHxsXs+0+Nvu2+zjeVy/f86K2bu7jgpftcB8PeJ3u4yHpMSQ9BvXLdh8Py4DGaI59u7ax6VSFq7+5l5FClcLmJoXNzdh9PJLHGy4p93FZv4bKba7jNPdxQb+Kmi3MgbdwxtCtnf6EvW76Gf3qYl4T0fVHjx7lve99L+985zv5qZ/6qY7+T3/609H+e9/7Xnbu3LmS03NYhyhUmpx2eIozHxmjNN5gQiew7ie4dbP2cfs/fmO1p7DkKI/XOHP/BG969DklNOnDNINu7axtrMd1009YdcPw+eef573vfS/vec97+Jf/8l929E9NTfHFL34xigGRUuL787uJJ5IpO6RjLkarG0MIie92rsTVpk3aiatNrKFhCiEZV5ioaGLFFkLMFEIirjAYMwmsVYxHbTxDZSrH1GCBRkNQqWc5796XmRA+E8LvqaIJdE9gbccVNvR+S39ngQxpLXG6mpVYNw69YaEVTaAzrnC+FU18T6emEf6cCazjxNVxXKGdkgaImEKAkba4wunTCwydqjBMnTMf3EdpsAEFIqYQwB9RqWW8TQOIsklNo2IIuyawtuMK9b7I5vQXlIXMwhnDNKzU2rEFRg6rA8nqx/jZa75fYg5X1TCcnp7mX/2rf8WHPvQh3v72t6eOKZVK/Pf//t95/etfz2tf+1o+97nPuae3haJXd2Y4ixHTi8gkJT9hwiCEZH5CYyDa5e3Mtm7nKazGBuGMdjNMzcQVTYzQZKyeMAhB5Sk8khniobddSPnlKmf/35cAFi00ASL3MRC5joHIfdwMA8IlvCC4dbM20ItBuHPnFQDce++DSyo0sQ1C6C1PYS9Ck0HpRQbhJp2ncFgG1M7P88OdO5A/PMTZL+5X57SFJpZBCEpoQrm9vF2XPIUJoYllEAJkc+Av3W3KrZ3+gL1uFoOlMMzXgnG50lhVw/ALX/gCJ06c4K//+q/567/+awCuvvpqfvu3f5uPfvSjXH311VxzzTV86lOf4mMf+xi1Wo0dO3bwx3/8x6s5bYc+hF3R5NwHj9Fc5fksBm7d9A/OOvO01Z7CojE2Wmbfzm0MnaxywY+O9qXr2MCtnf7Aelg3/Qwh+4XbXAQax5+P3ZbrDUtV0SWNJUxNUdPmNjZtttvYboc4PyHE/w9hE2ncxcZ93GokmELodB/bTCGAnK4kmEJQ7mPDFE5P5JkcKvCNna9GIrjk9v00J4IFVTQBIqEJ0NV9bDOFAKEMOWf72Tz+1Dc7v881jvMveCMHDhxa7WmsOfTqPn7PP78REHzu1juBJFNoxs23oonveQmmEIiEJkBX93F7RZOy8BmW6njbfWwzhWOj5Uho8tavP62EJqaiyWl6riN5vGHtAjZu43IJUbZcyBAJTdSELfexrmwicoUkUwgIPwv5AfLnxQKSfsEFF17h1s4CodYN/O3n7lzlmXTHYpjElTK7Rke38ePnHp73caseY7gikNKVxJvNPWwwhyEIXXITyjCZ3DotP6FlEALKKLQNQtDxhDpesK4NxOqMSlyNjifUBmGoFWtyokqg08zY7uPpCXWzmarkePDaC5AIRu84SHMi4JTvMa4NwqlFxhNCp/vYNghN2wZ4/toQmH884fzcxwuNJ8wIf848hbZBCDAs/Y54wuEwzlNYKrZ4+G3nMHKqwq6HHydfCCiMtMhuig1CAG+4mDAIAWUUGoOwoNty+c48hdm8MghBGYW2QQjKjSxWvA7DkmA9Jj5eOSRDKpYSS3UtbndTz8dQNH/XWr0vbAzD0GHDQgCvu28/E+RpTqzNXGsODmsR2UbA5Xf/mDNrk+RLbu04OGwUOMOwn9AL6zcfzKIujoakjbHdxuY8FnsobdEJKEZQt0VCk0bVUh5rV3GjjtSMYew+rsQVTaZmEkwhQDDe6BCaTE/keTk3yOGLNnPm98YQ49AUiikEGPeTTCEkhSYzmiWsydacQhPodB/bTCFAKOWi8hg6rD4WWtFkbEwlRrbzFNr97e7j+QpNMrotr3MUAl3dx4YpHLTcxzZTCLApVEKTRiHD1oNTlKcalIfrsft4k8DfHDOFoN3HNlMIii00TKFxHxcGEkwhdLqPE0whKLZwCcUnDv0Bs26WA72ykPNl8wRi3QhV3IpzWHeYHCrw0E4lNNn0xCT5mXUaX+qw5nHX3fd3xBWuZURCkxNVXnnoZbz1cZ9z6DPcdff9qz2FDY2NYRi2V+pY7+g5LU36uK5xhFFbkGwLw5gJlEFHfkKCJrLVJjRpNhJMIaDYQp2OhpqKMZTT01E6GjlViZjCViQ0iZnCmalOoUmlKql4PhNeMq4wTWhiM4XQm9AEOuMKbabQ9K/VWBKHubHQPIVzCU3M+PkKTTLCTzCFkJ6nsCgyibhCwxQOW3GFNlNohCYjpypc9Y1nGBpSaywRV7i5R6FJoZRkCiEpNMlYaWnsuEKbKQTws30bY2hXtHFYG5hP2rCF1D4214J+Zw43hmGYVtZtvaOHhN6pbuLEgC7GYJuxKMMwqToO29zGprQdWEKTKtQtgxB0nkJLaIJSHcsJLThJCE30IeMZZqbUzeal3BAP71RCk+3/eJD6ZMCEl+4+bheazNj5CXVfL0IT6HQf2wah+p4lYX9fJzYkliJx9a5dP41AcM893+0pT+FsQhNjDOa8TGQQGiWyLTQpCmMMJt3HwylCk036t1w7v8C+ncoovP7hx8mXgq7u456EJoVi0iAEJTQxBmEmr9ty6UIT0+Z5iD41DE2Ka4f5Y9cuVbbwnnu+23XMQoyvXg31bgZkr6KRXg1EIcSaJA02hmHosCFQLefwAsnFXz5AfXIDMcQOaxabNw/3hWkwM5Jn6GSVq77+jBOaOKw6Nm8eWu0pbGg4w3CpsMZK7nWwgWmYjSGE1JyEyDBOQ2PKvAWtuN/OT2hYxKaVn9BUO6lVwaSksdzHctrkJ9TbqWqcp3C8kWAKAaYnC4w3C2RaIfmpOq+79Xkm8BJMIXS6j9uFJlXZTDCF0JvQBDrdxzZTaNr63bWwkdBLcHov7uOILxJxWprIlUzsUp5LaGIzhaDcx4YpLHmGJcwmmEJQZe5s9/GQXqLGfTwcBpTyLTKtkK3PHuKS5w8zNFBLMIWQ4j42TKFd0aQXoUkml2AKIcV9bDGF6rv1wOsH07oTduiAw9KnZlksGzvbNdlex2nsYT+WuZsP+pWjd3AAYHK4wHd+6WJeesUIAJ7z2To49IRTo2W+/e5XMXmaMuZ8t3YcHBzYKIxhKFeF0euJtVsuzPb3piayDjr77RrHNksYJa5uxeNNOppWA1qaKYwEJ7HQRGqhCbUKVOOUNACyUo2ZQhNXOFUnOKXjCsehekoxCjNTeSaHC3xz56uQCDjW4pSOxWoXmgBMiIBpzXDaQhM7rnAhQhPTliY0MW3qvbvp9gOWXGiiKUPDFhqm0DCD7XGFaUKTqAayVQvZZgrVNpNgCkHFFQ5J9XlDoVXRJAw4NVrmR1pocnpzmoHBFKHJiGL1vOGiqn0MSmgSMYU6hrBXoUkmP3sCaz+bZApBve/TGENP9B7TttaxFLXe58WeigUc0wNsdm82xtFmE+diD2dDv6aw2RCGoQxb67ckno25DNHZyt7ZhqRtDBp3sS04sRXIaJGJURsHrU6hSb2aNAhBqY6NQThjCU2m1H44odzMwUSTxpg+ZCIbCU2O5Ib4/s5XREKTRpvQZMK4jYWa47QMlkVoAp3u46RB2P8KtY2AheYphNkrmhw7dgph3iMSBqE61ptTaBK5kEWsQM5HCuTYfTyg9we0+3hICoZDNZ/hQDJshCbnFfnRtcoo3PXQ4+SLttAkNgiFcR+XSzCoDD5RKsUGod6KQjGuaGKMwDShiZ+JDUJj7LW5jxMGoT2uD7GeKp/4y/h3pLlijx1TMUNLZViba3K3/4/2OdjrPc1ItA3EtV7FZCHYEIahw/pCPZ/h+ze9AoBX376fhhOaOKxRfOO+h9eUcTB1ZpG9144yeLLGW+7bR77o1o7D2sM37pt/fV+HpcPGMAzXY7qaXvMyzlX/2OrvEJUkqpxY4pKo7rFua9ZjFrFl1UA2+QnrtbiSSaKiiXYvTxnBSZVwXDGFrQl1vua4YgoBpifzTFXzUIGzfnCSTQemaUzGFU0mNbkwIcIEUwid7uOlEppAp/s4jSlcCleMw9JjKYUmdn+a0MT3vDnzFLYLTXJeJsEUgs5PqNuM+3jAEpqMaPfxcCgYDqTeDxjympRPtmjtfZlXPvkSmwqVBFMIkNmcj5hCb0gzg4MDiAGLJbSZQlBsoRGa5DXLaAtNtLhEZPOW21jfejy/TWiSwhRqFrXf4BGHDjjMgl5TyCzA82KuxbMxj6GUHdeBdpdzu9fHE6Ljmr5WU88sBP3L0ztsOEwOF5g8Td2Aznl8jKLOaejgsFZxw41Xcv2NV672NJjYVqJR8PFCyWt+cIhc0zGFDmsX16+RdbNRsTEYwzBce5VPloPB7FFwIhOMYVsMoT02DKNk1aZfBq1kMmuzteseN7UQxaSoqVUjoUlU97hSRc7otkkdVzhZj5jCxph6gqtNZJmZzjE5XOBb174Kvxpw6edfYNIITXyRYApBxRXaTCF0xhUuldAEOuMK258uHVu4NjHfuEIbHWlpUiqaeAgGBwdUdRPNFqYlsDZtaUKTvMgmmEKAkshSsphCSApNBtviCk+Nlnlu11ZqL5zk8u++EAlNipuaEVNoxxXaTCGg2MJIaFJMMoWg2EKbKQTFFhpW0I4rtJlCUHGHswlN+pQthPUVY7jSKA/q9EfW9+enrMc5GToTE5jCNqaxiXYsYppQZaPEi28MwzDoQ/HJAgxHOathaBnG3YxA2elKlrbyGJTi2HYbQ6KyiWzU45yFdSM4qcUGoTEGK1XklMlPaIQmLRqn1I2hNqluJlOTeV4uDPLotUpocua9LzHp+Uz4aqFOeJJJoRZrN6EJdLqPl0poYtrmch+vFxfDesBCDMJ2w88el1bRxAhNzHtjPKaJT4wRmPX8hPIYlPvYNghBGYMD2uVc1tVMbKHJiOU+buwo8Nx1Wxk5VeHNzzxLbiigMKzWRHYkqTwGEINtQhNQRqE2BhNCE8t9nCY0MQahiAzETMIgVN9XD+7jPjWuPBGHDjjMD2nrrcvAaHe2B/C0mt+hdep2I9F2L3czENOEKKlTnMOoXIvilY1hGDr0LSaHCzy6SwlNLr79gBOaODj0iFOjZZ67biuDJ2pced8+ck5o4uDg0AM2hmEYBjHz1S+Q87iI95qzMK3Sic0OtgtNgmZnzsKgFTOFxqVcr8fVTho1lYpGt4N2H0dMoeoLp2yhiZpDc8KjMqGYh5kptd37pq1IRGQU2kwhwKSQEVNo6h9PyxZVvd/NfbxUQhPodB+3P0GupSfBjYz5Ck1mO67dfZwmNIkYRf0+4/kRg2SLTLKaHcuJTIIpBFUD2WYK1dZnSJoayOp8g6FIMIVSwNNvOqnHLIgAACAASURBVJ2RsQpv/tqP2FSoRkxhbrP6G/xNuZgpHNYsYSJPoal/XOxS0US7n1OEJvjZJFMIigW0mUKI2UKDdtdxHzNuvhenInLoHVLKaBXOxbja19r2lDqJa3LKsTaLGLYtcY/ubuXZsBbZv4VgYxiGQTM2YtYKFhtjON8E1nZiatMkw2R+QhOHaauObYMQVEm7ltqXJpF1vRYpkKnXI8NQVvW2Uo3yE8opbRhONiKDsDGhbgaViSwz0+oGM1VXN51zvnWMfHmC+lTAuC+Y0qt5XLuPZ0SQMAgBqrI1Z57CpYonhN7cxxslNmWtYjHuY7BiB7u4j9sTVwvtSj760kllGGq3ojEIjTGY9fzIGMx7djxhHFdY1McY9/GQ9CODMHIfh3GewiGhfvtv+eo+RkpVcgXlPs6q4kD4w+rc3mAeMWjlKgREuRwbhEVLfZy33MfZOFehOmG2I55Q+Jm54wmjL9pLjrPbsIzIPoNd7tChd4RIjr50EojV+2mQyIQx2GmMxcphMy7NWJRSRuvXxCKqNa3HpbiVbaVyry7lfsKK/mqnp6fZvXs3hw4dAuCBBx7gpptu4tprr+WTn/xk6jFHjhzhV37lV9i1axfvf//7mZmZWckpO6wCpoYLPHbDDpp5Hy+Q5KbWmFG/CnBrpz/x4P2P8/B3f7hinzc+Wmbvz24jFFCsNp36GLd2+hEP3v8ED97/xGpPY8NixQzDxx9/nHe/+93s378fgFqtxs0338xnPvMZ7rrrLp588km+9a1vdRz3B3/wB/zyL/8y99xzD5deeimf+cxn5v/hQVO5P9fSywhi5nq16umvsBm/2vuCJrJVT76aDWQr+VKiEf2qV5UbuFFTQpJmQ7+vqldtRr3qdWStiqxVoTKtXzOqikmlgpyeRk7PqNfktHpNzCAnqsiJKsF4nWC8TvNUQP2UT/2Uz8x4jpnxHNNTBV4uDvKd61/JxJYSY4UCE57PuC8Y9wUTnmRcqNeUCNRLtpjWr2r0UkIT4z42bGFDtiK2sBUGkfvYsIWhDCP3sWELzb7UAhMjNDHuY/PUGLYrk5eQLVzVtdPnmEsZKvS/2Y7xhIjGmT7jPjZsodk3/4wK2RfKhWxyFGY9PxKZ5DzlOs572YgtzIsMeZHRTKGqZlI2L+lTlj4DeAxJoV6hZCiUDAcB9R0Fntu1leppeUoDLQYGGxSGmhSGmuQ2SzLDGTLDGbzhAt5wATE8gCjrV0m7kAsFJTQpFBH5AiKvFce5QswWZvO6sol6kclp1lC5jyO20PPVSyi1cSQ0aWcLzTjzfglZttVcOxldyWajv8w66PVl1kjW8yN3fOqr7Tiv7RVlA/A8PLNmrX9e2zo2a7k99+R6KWs4H6yYYXjbbbdxyy23cMYZZwDwxBNPMDo6yvbt28lkMtx0003cc889iWOazSaPPPII1113HQDveMc7OsY4rB9MDRf4pz3nA3DxHQdcnkINt3b6F9fufjPX3HjFsn/O+GiZH143yuCJGm/+vz8i21xnCf0XCLd2+hNvu/FNvO3GN632NDYsVizG8OMf/3ji/bFjx9iyZUv0/owzzuDo0aOJMadOnaJcLpPJqGlu2bKlY0wvUAxZfQGzXgOYLZYQulQ26SE1jXlvi0ukJToBFUsY6HrIdv3jZiw6MVtpBCfVelzRZMbEE9YJJtUxrSnFnjUmMlQmVaxSpZJjarjA/TdcRIjgojsOUJsMqPk+414cV2gLTWZ6FJrU9bYlg0RcYVOLboLQiEsWJjSx+0x/tL9EcYWruXbWM+YTV9jOOtpxhcKKJbPT0WQ8n2KxgICILYxEJ8KkqIkFJ3mRifITRlv8qPbxoH6OH7IrmgQhE6NlfrzrHIbHqrz5q8+yKV+BHJSGG+Q2qd90ZjiLN6JEI8LkiNNsofqgODWNKBjRid4athAiplCdVMcadhOadBOYwJxxhXHT4riL1Vw7voj/vzcCul7v5km4SSkpFNVvLGt9f+25CNs/T4hk7LcdY2jm4FtCklB3eaIzPtBDRIKURLqadSIumQurJj4JwzBxsZVdytKkXZDnDbtM21rEXMafjblK3LWPm8swDKwE1sYIjIQmlgK5pY3NRr3DMJTVGrKmv9+ZKuGMbp9WxwZTLZrj+hCdn7A6nWV6Ri3+6WaWWsUnO9lk07eOUZsMmPRi9fGUTlw9LcKEQQhJoUlFG4EN2UoYhJDMU9iUQcIghP4Smqzo2uljdPt75ys0sWH6PMtY9IXXkZ/QuLfMJxrhSZyzUKuPRZaCFxuDJmfhAMYYzDBkGYSg8hQOBeo3OkwT2ahz2tFpfubBp8nlAkomT+FwiD+szucN5RHlNqFJqahcx2DlKSxBTht+xhjM5CKhiUhJXC38TGz82S5hu9QdJBNYL7NB2A0ruXaynheJjDYq5iPIiK6bIl57Gf39hVJG7k17XDLHYPIaHVrilMQDvv6vjMUlyjhUx5AYb7AeBSazYdUkU2eddRbHjx+P3h8/fjyi+w02b97M1NQUgTZY0sY49DeqA1kkUJhucuE/vEjOuY/nhFs7DgCNsjLQRl6q8Oa7n3VCkx7g1o6Dw9xYNcbwta99LS+88AIHDhxg27ZtfOUrX+Gd73xnYkw2m+Xyyy/nrrvu4qabbuL222/nqquumv+HNa1qHGsdC3Idp+QnjPqCuC1sZwxlW6m7ttQ0zWaUmoZGI2qTxoVctRjDqq58MlMnjJhCzdaNezSm1ZNfdUa5n6Zn8hwdLPP4nh2M7Jtk6/eOM+l7UZ7CuJpJyGTkPg6oaKZwRrOEddlKMIWg3Mc2Uwid7uOlqmhij28ft1xY0bXTh1goUwix0KR9jGexg6bNdh/bTCEQiU1MkLstNgHlNgYoeJlEapqyviSXrcompsTdkF62g6GkOVrk6V3bKXxzP+f8eJyBUoPSkFp3uWH1O88M+/jGfTxUQkQl7qw8hSWTksa4jfNxZZNMnJYm4So2uQg9iwm0y97pto7UNEZokvhyvba3y8tVrOTayer/8/WInpmzLkTrbNdJlR5G7RtXvBSd119JzP6FUnYwhoL4+h1aLKFxIZPSZjOHJs9hKDoZRHX4+i2Tt2qMYT6f54/+6I/4wAc+wA033MD555/Prl27APjoRz/KfffdB8Att9zCbbfdxg033MD3v/99PvShD63WlB2WEFMjeR7fswOA056eWN3J9Bnc2ukfvHzoOEcPnVjSc06ODrB313bKJ2psOTS1pOde73Brpz9w9OAJjh5c2nXj0DuEXO9RlEDt23+LrK3hC2jYowtoLrYwEU8oE20ylTG0Eli3Aosx1PNpNqN4QhnFFTaixNXUdd9MjbCi9sPJJsGMriAyFdc9rmimcKaaZXokz4M3XYhEcP4dB2lqYcq4L5jUj2kzOtpjUgTMyFhwUpGm9nGcuNpmCkHHE+pjusUVrrTQ5Nxzt/Lj5x6edcxaxPkXvJEDBw6t9jR6wlxpaWYb360GsukTbYyhnbzYE3EsmYmJMmITIDWBtWEMS16WAVPZBJ9B3W4SWI+EHsN6uQ4HIZOjAxy87hyGTlb5qbueZ3NO5dcbGG6Q36TWQXaT+lxvcwFv2AhNSrHQpFxW21IpEphElU3yA3ECa80cimyysokw7KAda9jGDs5ZA3k+cYV+juxpo+l9axg3vuGdvHTw5dWexrKhXQzSK7qZHPb5Zr3uWinCuu2bcdE1nVmu88jo3pA4n3Vs2r0hbT6z/a1z3SOWwxQbHd22oPvO+uS521GvQq2y2rPoDXO4kmW7cdjVMJxFgWx48yBWHdsK5IQx2NRVTuralVxvIGvGIFSu5LDSJJzRhtgkNKbUz6o2o7aVmRwzupLJJFl+sOsCJIIzvnyI+lSLSeM+9iRTGKFJ7D42gpOKbCYMQoC6bEbu4shtLIOEQQid7uOlEpq0j0vDBnj2WnXMRxzQzSCEzlJ3ZrxtEAJR3jRIdyXbFU2M0ES5kpVhVNB9RZGhiKlskqGsDcJoGxIJTXJlLzIKr/7mXnLZgGJZl7kbbpEZ1nMcUmvNGywqgQlAqWiVuDOCk1JsEBr3cTaXNAgBvLiyibCrnAjbRTyL0CT6stOFGLO6j/u4ckhOqPyUaxULNezmg9Rrn0j/bPs62m7Q2UIT0yaFdc229gNiI8/sR+XvhBe36aZASktcEs9mNqSVyfOEWDfilI1hGDqsGXghXPDNIzQqksZUn9WvdnCYJ95840/iCcFDdz226HMVpppc8PUjjB4aI5dzQhOH9YsrbrwMgAfufHSVZ7IxsSEMQ1mvItcyY7hYwUnY9pQyh9AkYgbtFDWtVsQOxoxhCzRTGLGE9SbSuI0ranwwE9KaVk9cjekMtRn1lFyp6DyFjQzHRgaY3lKk8Ow0HKlT8z0m9SNblKeQkBnNFMbpaIKE+9hmCkGxhDZTCL3lKVwJoYljCpcf86mBnCY0aR/XXgMZVGqadvGJZ6WoyeoKFxAHy0dVL7I+Hh45oaucWEwhqDyFJX1MSXqUZafQpHVukUw9oHxskvLzJxkq1SkOqHWZH1K//cygwNdMoRjUgpNSAQY0+1cqRSlpIvdxLh+7iNOEJl7sMha2uMSweKatm9Ak+qK7p6bpyhb2MVNokPcyUehAv2GxzJedUqbj3N08MVZ7JpMMwwiRkQAl4eIlZglF2/XYPp/NHEpTF1nPTUgRz1Of2xMiSl3jSQjWcV3kNPT/6nNY85geybN3zw5efNMZBNmNlUvPwWExmB4d4Ee7tnPwDVvWofbRwcFhLWJDMIbUa1Crzj1uPommlwNzMofWraFbbKEZl8IYSpN6xpynFcTsYaOJjGIMdVxhrYk0ApOabqu0CKo6Vq+iztOa8alPq59StZKlWtOMYSvLzEieR/ecjwS2f/kQk6EAXzDtEQlNpvWz2RQtKpbQBDRLaMUV2kyh2c4lNIHOuMLlFJo4pnD50UtMYZrgpNuYNKGJqWzie17HODuu0LOSWRvBSVaoGENTezXvZSiIDCUtNClolrAofCuuUDCgaYxyKJkZHeDIznMYPFHlsq++QDmvWMLSQINcWf3WsyPq7/CHsjFTqCubMFBUTCFAsRgnsY4SWBetiiaabTR1jyFdXGJqIEO60MTGLKlpljstzVpAQcSpiJYLyxYnOI/n97lSz8w2PhSd7SqZteow6X5scYlh/2wWUUgRtc82L8UsRiVNALXmA5lkBIO2eS9FxROhEugs+PiVxMYwDCsVmJ5e7Vkk0asR2v5DbDtOdlMihzI5PgiitsgAtAxD2WhF1U1kXRuBjSAyCEPLGGxV1EW9WVE/n3o1Q72mhSa1LNOBuhie2FTi2T3nIoFNXzlEbarFpL5XTIuQGVPRxBiDxHkK7TJ3NStP4UKEJtDpPnZCk/WLpSx1Z8alVTYxbuPZSt2Zz8iLDAXhRwZhCeM+9hkgNgyH9BoNt5V4eec5lE9UufKrPyJLQEm7jwtDTXIj6rfsD6q15g3lEQPGhWy7j+PKJiKv9/OW0CQbu5DVCbOJUneQUtlkNjdvj5VN5jx+HaBgVbJZ61iMwTLbnSwt/6A6xlx36WwTlmGov7+EuETE13ZzTbfXrdk2ZRA/0Flb029cygGyIyeher9xr+X98at16EtMbhsACVu/cpDGpBOaOGw8nHhxLFHvtVecunCQ4okar/vKC2RxQhOHjYWxF0+t9hQ2NDaEYShnKsjpmdWeRid6YQ3nYAwJY7dxgq1qBcnxUsaMYCtmEaP9RoBsGPZQs2z1kLCm3cUV9UTVqmZo1tSNLmIJq1lqTbU/I7NMeT5CQn7vBFufn2KmKZkxbmNhtmGUn3CamCU0jGE9JU/hQoUm0Ok+dkKT/sVK1kA2fe3paOxayHYNZMMc5kWGgpfh+JNHKYosBaFdyYYptCqbxO5jGJASIeHsrx8k9D2GqVMqa6ZwQOcpHAzxB9XxoqxdwAMFhBGaFA0zmFcvQOQKcQ1kwwhabuNEZRPjIu5W2WS5aiCvI7YQ1P/xwBphDJf66tTNhd3emhCaJPIUxm1xbWPTJzn+5FEgTusUEEauYl8z7AEhTf3Q1O5G7jZH9WmdLGLHfXaDY32tRIdVR2VTjh/94g4qW9RNyK+vctymg0OfoHruAM/+/CjNoo8XQqbp1o6Dg8PKY208ziwzZKW6OoxhexqZno5JuRkkRCfdGUQZxRXaqWnCuE+zgxFL2ApjdrARIpv6yU3nsg5qglZV/URaDfUM0ahlqGmmsNbQgpMgwwwZqptyPL1nFAlMB1DXiaunhYyYwoqOD5mRLSr6aa8aJbBuJZhCUNVMDGPYDIMFCU3UVyiXRWjSPt5h+TCfJNbt49sTWZv+9tQ0dlu89TpYRN+qdmLHGxrmMCd8cvi88saL8RC8ePdzFPEp6ONL0mwF5RBq5w5wYufZlE/UGGy1KPvq918sNCkM6CTWg+o37w94eJop9AY0I1gqQkEzglFcYcESl2TjOEITV5jJJpNYQ3plEztFzVypaWbBek5Nk4aSyFBeI4xhr+g1DUvaqCQ7aI9Npq4JScYYRlVJLPHJRTdeDMCzd+1TfVIQiJgphCSr5aV4AXyLjYxYQhnXQrdjDaVIxh1iJb1uF6JsBPTXr3aBkFMV5MQqupLn88NKMSZlOJthKGNRScIwbGtrSWTLiE/0tiUjIzBsCMKmWmqtutn6NBvqhtfQ23ojQ7WlDUKpXcl4nNpc4NDPbUMCxTsPUZlqRu7jGUKmtUFoVMfVhNBE3fhqYauzvJ1s0dRGYCNsRcbffIQmps0JTfoTK6lA9oToGNeuQAblMjZtOS8TxRGaYPmcdh0bVXJBG4UlS4EMUAqB7SXGdp7NwIkal3/lObLNkFJJG4alJrmyzlU4pP4ObyiL0AahyVNIMY8omrJ2RnVcSCqQI1GJrmKSySUMQtVoiUts97BTIM8bZenRlPOPL11p9MJL2yKS2Y5vP5dERm0JY1DG+y2S1+WWJT4xYq0AjybmOq/FJfZ1QdrnV1tfeB3VUjwh4jyG7vLdFRvCMHRYXtQHsxz6uW0AnPblQ9Snmqs8IweH/kBja4kT155D/mSdi778IlnnPnZwcFhlbAjDMJyqIydqCzhweR8p5GznT+tLlEKWUVv7eWSAxRjqthbIln5SMukMA49Au4iDpqDVUk9nxm3cbPrUtbu4rvtq0qemA+grmhWo1gIyL85QfOIUM9NNKm35CSsioKoZPtt9HOcn1FvZpGnS0VjuYyM4CWW4IKEJdLqPndBkfWAxghOgp9Q0HuniE7vKSZSmRvcXhE9OeFF6jAIeRTxKxC5kgPKJGv5zk2z9zlGGGw0KOfVbzxf0ttwiU9Zsx4D6DK+URRS16ES7j0WhADnNCGqWUCTS0eRiVtBOTWPYvG6paaC7q3exqWkWMr6PMCg9Qtn594Vzk9urhq5Xtvh2E6Fd3JFMPRNv28UlITED2SJ2IUdbKSM3cUHfa5oixJCWrahKCan5Fu18iO0eASFTKiGl/NF2bkMhNp44ZUMYhsFEi/DUMrBYS2Q4plW8S+P3ZZiyH4rY+Iu2AqQxAkXUFga6TW9bTY+gpd3GLZ+W3m9qI7AReDS0K6SqF2hNeFQ8dfzkSA5RC2g0Q/juUaqoeMJKm9u4IgNqMhlPWLPiCeuREdhKKJBVWxxXGMgwNZ4waiM2AheSp9DFE649zCeucLaSd3a/HVfY3pbqSu6SwDrKYyj8yCDMYVzKPgWtn/RQhmJJehT0uvRPy5M9UWdopsX2+w4DUPRblIrqOpUvavfxQIA/oA3UkjbeBgqR8jhNgUzeaouMwEy0L+zYQeNWthXIHa5kL12BPAc2kgI5DWUpkDLVcllTmIujTuvXJpPqF53jbGMwNPkCrXHG6EoYhsZYFLEhl8c8nCUzB7RNROU51E2RalmGsUFoh4rIzuuCaDMc55PXsT0H4nrA+l+dDsuCxkiOEzdtY/zqs1Z7Kg4OaxaVFyZo7p9KtLW2D3Di57YzddnmVZqVg8PaRn3/BPX9E6s9jQ2LDcEYNscFwZhYhie1hfsE5Bz+hCQ7GI9tJ6ykzRiap51QEAQi0RYGgiBQzwGhPl8QeLR0Wyv0aIRaYKLdHw086vrJvqaf1qqeYGpTlokbldBEPHScCU9SidzGITU9IeM2rsmAihaYGHawIYNERRNIlrczgpOWDCJXcZrQJAjDBFMITmiyHjAXU9iL2MQel6ZA9hAdrmYhUlzJFmOYUCVbjGEsOtGMIR4F4dHaN44nfYoIClLgbxugcs3Z5E/W2fLEKUoypCjU77+Qa0Uu5GxJK5BLAq+kWT0jOMlnoWj2ra1WIIuMEZRkk4KTSEXsx22izV0svIWVvGtjADeq4MTGcCDJBIu7bqxExGm3GYYpazBNaJJwIUesYMwSRs4tPTAQRCnbW0AgTKUSNaCJpPmMMgqNil9IEO2ZJJDR5/lSduQ59IUXK5ilVflEJNlBh0641eswLzRHlFEIULjrEN6EE5o4OHSFL9QLCLcPMPW2s8mcrHPOnYfwG05o4uCQCmvdOKw8emYMgyDgvvvuw/M8fuZnfoZMRh169913c/311y/bBJcC1cksrfHcin9uauzgbONTnmC6sYURExjGcSxxqeS4LQhj5tDst0KTD8qjaaT/eDT1U55hCRtCUNfxhFVNuI695UwkkLnrIDOTDRCKJazqJ7OaDKlGTKGJJwyi/UQVk7Z4wpYMEkwhKGbQ5CyUVhoaO65wLqGJ3Wf6o/0VYgoFTb72ta/13dpZi+hVcDLneazYIs+KJzRxSWlVTgwbkRF+xBhm8CKmMG+2eOTxGNh5Lj4Cee8hwqvOIneyzpa7DjNYU7/pAgHFjGYMCy2yBdWeKWn2o+gjTGxhQW+LBYRhCrPZaBvlJNS/L1XZxFQ0ySarm4CKHbRjC6G3WsizpKaZE/OMLQyCgK9/+0Fyw8/23doZDEMypi79KiHskVmf7SpnM4dzsYSyrS2wjjdFUQMBLVPP2GIPDcVQF4LcTkU+VO5+UTUKohtqlG5GeNE9LyCM1qiJWbRzFs4qNEHMWXTSXBOCdRRHOBt6Ngx/7/d+j23btpHJZPirv/or/uRP/oTR0VH+9//+34taoH//93/P5z73uej9oUOH2LNnD//hP/yHqO0v/uIv+OIXv8jQkErk9Qu/8Av8yq/8Ss+fMTOdozmZX/AclxKpwcgW0vQs9jGRQWgbiVFeJpOQszNXU4Ag0De1aFEiolxQTSFo6o+pi3jR1qI2TfN/6wjNrKA22aCq22oyoGYMQ2KhSc0Sl9Q71MatSGFsDMNAhh0JrG1xiUT2pdDkzC0FhAh4+umn+27trAbmm8jaoENsgpgzZ2F8rHGbioRCGdoUyF6sRI6SWgufLEZ8orfCoyA9nccQck0o3X2Yzaca+I2Qgr7JFURAXiuRc4UWmWLsQgYQpQwir42/fC7eGoPQLnPna4MtYwlOMrba2Cprpxo7hSYwe8m7ObDULuSb/9OfsfWcs8kNjvXd2hkSTfJi6T0qskdjby7Mpp20PyORRtdKCh3GgwF1jwnbxgVCRMZUyySMFnEOwpak477jAxV9nmKUcNqLIrciAoRYsOLj4RsVsXEfCxG5kA1s8YlZ/y6fYSd6NgyPHTvGn/3ZnwHw9re/nZtvvpnf+q3fWvQE3vWud/Gud70LgGeffZbf/M3f7Djvk08+yZ//+Z/z+te/ftGf5zB/BMNZWq8exn/oOFRa60p9tRLwM4JQFvjABz7g1s5GQ95HapdY9kQdf5ExZxsNx06M8Z//4Gayp426tePgsELo2TBsNps0Gg1yuRzbt2/nv/7X/8rv/u7v8uyzzy7ZZD72sY/xO7/zO2zenFTrPfnkk/y3//bfOHz4MG94wxv4d//u35HP984APnjVK5g6GZ/zzOcn2PbUSYKM4LEbzusYf/a+U5yz7xSNgs8Prx3t6N/61BhnPj9BbSDL3mu2dfSf+/gJTj8wxcxIjn1Xbe3o3/HoMTYfnmFic4Hnrjy7o//8h44y9HKVibOKvPBTSdWvBM67/2XKJ2uMbxvg4E9u6TDTRr/1MtnxJhOjZY6+bnN0nHniOvO+l8nOtBi/oMzEq0cAk1tK9efvewlRD6ldOEjr4mE4rYAEmqcXCMOQyXsPQCDJvHKE0nnD5IGhKD2B5Id3PgXAlkvP5PRzN+vs9/qpsRnwwJ2P0pAtLv7JV3DWti0JQ7NarfHVrzxAIEOuuPK1nHn2afHfLiVTUzPcdef9SCn52WveyBlnbIpTFgBjYxPcc893CaXkhuuvZPPmYeubg5ePnuSr934PgLf/3M8yODSQ+O4OHzrKfd94GIB3vXMnxVIh0f/CC4f5zv3/BMAv/9L1ZLLJJfTsswf43oNPAPCr77kJgB8+8U+85aqfBei7tbOWMJcLebZxkJ6zsKP8XZechZ2uZC9KUWOnqzHpNfJ45LeXYVMOmiE5IShKKGhWo6idV/lMQC6n3ce5kExJ5ywsKvZPFDKxC1kzhiKbjXMWZvQ4O2eh7TK2Wb82t7FItM2RszD6Elc2Z2FD33ey9N/a2XvtdqZOlqP3W18Y4/xnjtPyPb537YUd48999gSjz52kns/w8NWv6Og/75ljbHvhFJWBLI9edX5H/wVPvszZByeYGsrz2E/v6Oi/6LGXOOPIFBObi/zwiu0d/Rc/cpjNx2YYO2OApy/vvG+9+nuHGTpR5cTWQZ67zLov6evvK799mIFTDU6MDnLwdVuiLnN133HfEXLTLU5cMMTxSzapfkucMnLvS/j1kIlXDsJm9T3716t5lgB574sQSLKv2kx+x7ASn+hjQyR779yrvsfXbGXzuZt0v77vtAIevFNdty+9/ELO3H56xDxKJLVqna/d+T0C4Iorf4Izzz4tmreUkumpCnffeT9Ax30HYGxskrvvUf3d7jv33vsgsHL3HYDTTjuNhWDOlVqpKFL3wx/+MBMTsXy8MdDG1wAAIABJREFUXC7zmc98hg9/+MML+uB2PPDAA9RqtQ73wMzMDK961av4t//23/KlL32JyclJPvOZzyzJZzrMDln04TT1A5Una45znycCHV+047xX0GzGLiW3dtY/MtvLcPU50AzxxuoIpzOZFypVVZDg937rfUxOTUftbu04OCw/hJwjkOrKK6/k/e9/P7/4i78YBf4uBz74wQ9y7bXXsnv37lnH7d27l5tvvpnbb7+953Pfd/kHqB48vtgpzhu9Bv4apP1HpAX+2mNDYcd92G3JcS0RB9iazPGBiAN+m0LS0O1NJAznENdvRwIz9xygqivHNGRIXZ+9YbZWAuuGbNHQ+/WUeEI7HU1qPGGbuKQ9rrAfhCYGO84dYGy8weRkk9HRbfz4uYeX9PwGy7l2zr/gjRw4cGixU+wJC0lRM1uVk/YUNZGoxDBmVpUTEztoxxPmPF3NxPPJ6/g8k5am6OUo6P0BkWUg2vfJbS8z/LPn4o3V8X40QakpKT47xUggGda/9RGhipQPleoMlNV+cVODnCat/E2K/fOGi4hhxS6Icklvy1BS+xR1X6EIBc1C6LQ1ZHOInK6fnMknhCig4w4jIUqcwiYRWxh90bNXOekptnAejOHP7vnn/Ov/55f4+Z+7nkwuT/a0Ts/NUmA5186B695D68jRxU5xXuhF8DhbnHua2NEeH1piRlvsqLZeQuwI0Ay9qLZxS8Yx7g3NSTWJBY4mxrAmhGINAfmcygFaETKKYzex7VUZWELHgHq76NEWONqiR0vsCOr+Y+47RvwYyjC1kpZdZSst5dlsFbXMmG5YjpRoC73vzGnp/c//+T/58z//c/7mb/6GD33oQ9x4440LmuBsaDQaPPLII/zRH/1RR9+RI0d44IEH+Pmf/3lAfXnzNVCnPI8Z3597oMZq5o6K5tDFIOwsUt6Zed52C7escZFBGAUDayMQpQwz+00h8YuCQitg7N4DtCYaCWOwHhmBektAUxojsXPhNWRrVrVxkFLFxCyg9jyFCxGatI9Lw3IsyiMvVTltc56R4RxCGwJLjeVeO2sBvbiQe1Eipx0TqZItoUmcv9AEw3uJ3GiqTViuZI+sVakhV8rBWI3SPUcQjZAh/USWk5KcXkc5X22z2YBMXgtOChIvr13DOeNKziK020hYCuRIfBIpkDMJtbE6YTbpPrZdyFFbDy7k9r4VcCH/1Sf/E5/6y7/hf33+S3zg13+NPb/0a/M+x1xY7rVTHGoSVLqs+xVyvsyWLzfNiEw1DK1cuQZh4EViR9MXBl4iRy5ALhBxrlwrZ25GV9Sq4yFCs970yT1gnzIIZ/RPJ4+V71B/eb4QZCwhibByFZq2buXu1MTVRljjTJ9EdCYN3kCYc8VedNFF/OVf/iWf+MQn+OxnP8vb3/527r///iWdxL59+9ixYwcl8xRsoVAo8Cd/8iccPHgQKSW33norO3fuXNLPd7CQ1ZL/o1XG/uE5WhPLY9BsBDSaIS8drXLseA1PNNzaWecQeu009o0jv/IiohEi8x5B3qWLnS8uPH8Hn/7jj/EfP/Ihbr3tdrd2NhgCt25WFT0/Ar3hDW/g85//PF/96le55ZZb2LZtG7/7u7/LT/zETyx6EgcPHuSss5Iii/e973188IMf5DWveQ1/+Id/yPvf/36azSaXXXYZ/+Jf/It5nX/CF0ytkWSZC69Lqfvb/oxuRcrD5EMRLWTkSo4YQ2K5v3IfZ9m0awe1R49SefYUTSkjd7FhBJuEMVNoXMUytPZjF7HNHBpWMNVtbLmPzX7kMm5zH8/FFNrHto9Lw0pUNKnVAoKwzG/8xm/03dpZCSw0PU3HedrEJWkpatrHtVdDsQUptvjE9Jvchb7wyBCzhwPnDjH8lq00vnqI8HiNLIK8hIlrziaQcM6XD5GXIXkdbJjNWIKTvE6vkQOR06xeXl+as5lErkL1gX5c49h2D0dMocUc2pVN0qqctH+H3di99tyFK4TLX/cabv3//l++8ei+vls72RGJ31ii68sC47tndS2nMoYpx9nVtaKtQAbtjKEg1G6pVsuP2oKWZgr1ttH0I3W+H/p4kcpDV+aScHinEmUO36nqiAdCoH/9+KbeuMX4+zotFFgiM6w8hnS/Djh0Ys4YwxMnTvDUU0+xd+9e9u7dy1NPPcXx48e56KKLGBsb47LLLuPf//t/z8jIyErNed74X2/6EFOHTqz2NIC1axiGw1mGdqk4nuN3v0Brok4TZxguBr4vyOc88nmfTZsG2L7tzL5bOysRY7jY2EJIuoW7GYai3eDzYrexnZ8w6yVjDHMiQ1bvlzwV+5cXGQaEulWdfu5mzr56lNZYjeb/PQSNkEE8RkLBxI1byWjDcFMQMKzL3w3l6wCUh+qUhhUrn9sUktmkzultUrGBYngAUTYxhjqGsFSCkokt1GxXoRTHFkYxhnlExs5zaMUWgi6TlzQWoz6wSuh5XWML46Z5uKJ7wImxUzy97zn2/uh5nn72xzz97At9t3ZOve8XCY+9vDQnW2eGYTNQ/Y3Qp6bdylVt2tU8wY9/TqmRjWFY8VScIcC0VX61YnLmElC1Ciqobauj7GrTiju0w5tmC2tyMYYpuOqqq3jFK17BpZdeyhVXXMH73vc+Lr74YnK5HM1mk7/4i7/gAx/4AH/7t3+7oImvBCY9yYS3/MzQfDHbjNLWc4ht8HSOiwUnsYw/MvywDULzI5cEUpIZzrNl1ygSOHz389S00KQpQ5qRYRjHFbba2tqNwcgItIQmpsaxOSYIw0QcoZpPmDAIoTOucK0KTdKw49wBGs2Qej1Eygyf+tSn+m7trEUspMqJPdYWqaQlvW5nGYQQcYyhxVD4CMrbh7VRWGfyqy8y1BCAICchK02NV7WfRZLzNFOYCfU2wMuq36LIiii2MEpD4/uKNYQ4gbUtGvFtRjApEBF2G12MtzShidedUUxDV6NwAQahwTX/7D2cP7qdSy6+kCt+8nX869/87b5bO/5wBq+ZnXugwRJnfZDdzpfWHhl8Mr0tijc0fSHavkI//yNbsWGYaWoDKhC06ppl14ah50n8pv7NN2X0OWH0GV7E/umlgS9V4mtQTCGoNZixmMCVYADjCiprz55YSsxpGH7/+99PjcEAyGaz/M7v/A6XX375kk9sKTFDyNSKSErSMZ9PlqLzBze7+ER29IVWa2AZWpGxGAlOJDLncd715xECP75rH/WJumX4hVER8qZl7NkGoWmzRSXNtqcvmzGMFMZtqi/QRqBlEKpxc7OEa0FokoYf75+Orh+jpdMT7q9+WTurjaW62Nvl72zMmtswMgI9MoZR020Z4TFw+gDnXnMezbEa1a8eJNOQkeGYlYKsVEHcQkJWSrKEZLzYIATwsxIvp34kXs4HIzrJ6UtzLouwBSYAGT9m9mz3sWmbJXdhYr9X9/BichcuEN+75wuUipr59DIJVXK/rB1vMI9sFuYeuFD0ch2bxdhMGoGyc7zel6GM91uxFSdb+nertzKI+0Mdmh42BJ6vx9VNRR2JZxE1smkEK3qLwNwGjYGSlZARybYMUWpeBLYLWcGjU2ji0BvmNAy7GYU2PvvZzy7JZBxWHmEj4ORjLzN5ZJL6RH21p7Ou0Mt1262d/kV9rMrJx49S3ztGuekC5ZcSkVE4C9zacXBYHixJ7opXv/rVS3GaZcO0CJkSq1zMvFeWSiY2yXOksINqbHuMYcyZRQyctR9ISX64ADnBzPFpZp5+mZYMYybRsHvIONeTxf7Z7CAo5jCVHUyJ1wittvbUM73kKVyr8YQLxVpfO/0C2y0ctVm5C9vbIRmDaKe4sGskm/F22/DWYeR4A78K448dYwA/ZgqjrWIJh/eOUwqlYgyFJKPT1PjaleznQnTYIiLnITKGAfTjrb2vDk6mpIF0djBRC7lLXeQ+xlpfO2KwBMHA3APni/m4nMO0QELtxk07n0kXZrGEIgzjp1ybOdTsoLS2smGEVPo63pB4+rfuZfT5rJ9fGIoo96HZtqRgy1OnALWGADJCWGEcCkJa7L4UFlPY3TOg/m4nOpkL/ZfUbAGoyBbTJiBiGbBU5kbY5UxzGolSJsYpwzBpLNqBsbmRPKM3XEDQCHj0Cz8AqfqNYdiUKQIRYvGI2beTgsZBuSmikjmCd+226G+eh/u4l9rNa9EodFi4Inm22MK0vrS2buX02pXKdpm8087dxAXXXMjk/nFOfUuJclQ+NXW82fpS7Y88N82IDq3wCfG1YZjJqq3wJSLykXmx8Zextu1qY+GBmbtt7NmxhXbfbDC5DW3382JjCx0AEKUitJbBMExDmgFoQ3Yaf/H7lD4p4/3Q2g/0thWoFyAC2zBUbWYr/BChwyew3cfRx4go96HJc5gJJKc/NwnAtEk8L9UL4nyHHkm3cTsEnQ+Gds5Ch+5wK3uDoTBS4OIbXgXAM/fuc4vEwaFHGKOwcrLCkQcO9nRMcyBDvbwhnr8dHJYMjXKGhls3q4YN8c1XZMDMMjKG3dALk9WO2Z770s4XWoKNeJztQo5ds4WRAhfdcDEAj33lSabHZ6L+liU0iZg8Otm/rq7iWdhBW4GcVmZoNqZwNpFJt+8k0e+YwnWDhbCLs5XMg6Qa2WzbVcm+8Dj93E286m0XUz1Z4dl79lFoefj68plBJNTKoNhCH8mha87ipJRcescBMl4YBd1HAfkZEDrHqsjErmQRqY1FzCJGVEmK27gbe2dXO2kfs5wu5XXirl4MxEAJwtryfsh8mMIux8iEq1gz2RAxggn20LQFgbWv+xotREbdZ6X+HUuvZdFPMQOZMSKVICQwCn3Dpoch+685B4Bz/1Gx8hkrT6HNHNqMYLtobLlJD3NtWY/3GLd6NxC2XqKShj72lSepTlRXeTYODv2Dcy87l5mTM/zonn0EjdWNV3ZwcHBYTmwIxrAmW1Hyy/miW9zfcmAu9itNwJKWriYtLk8ieeq7z5J/7AAzU1U9XibYwQ6mj86UMjY7mBCSWDkJw7aYQYmMxSdd8hO2/z2OKdzYWIkAcTvpdSqLaG2funsvUkryLQ9Pi1oiBgMRBcRHMYZoZkMq9sUnxPdkFGNoGEORkbHgJOPFsYWeYREz6exge67BRB7DuVjEBVQxcQzg/FEqgUwpKToXy7cQdDtnD4yhSGEMaWuTQZDoF61AsYaAbKp7q/A99RsG8JrR+dsJQxmG+JoxzARBlPg6Ema1ZLQazS/Vk9Jaj3re1hpc7PVCuLQ2CWwgw7A598BlQs+K5BR0JHTu4k5uP8aMGxgpccmVF/GDrz1FrVqnOtmIzimljI08S7Bii0Ha3caSlDZrnG3w9ZqfMDHvRWSO73aMw/rFbDeEdiWirUae7Xxm3Jk7TmfrhWfxzDeeJTTB9LqsnJ03zYMo0a45c0ZKMtLc4CQ+UuVvMwahZwxDogRtwhOxqMR2JbcJTdoTV8d/sKVAjv6gtjJ488EqlcFbN8gXICgu3/nndCOn9NvHxBmlO/ttw7DVQlj7AAQBsqnuqcJSzUtPGcJpwiShP88LrNyd9XhN+Hqb9cLYMNSf6yO6iE86H+jsDAPmdpFm+Lkch93hHgPXMQZGSlxx0+sZ3DRANrchngEcHJYEZ41u4bKdr6E4WMTz3WXSwcFh42BDWAv1MK6ZuBxYDnfzXKxXsjxeJ8tWHhngp256PRL4zh2PMHVqOlWQ0m0ftECE7uyfcQ/brmKb9ZsrP2H7/J37eONgoWlqZjtfe5qZXo5JsAt6e/boGbzxutcyeXKah+98jFzLQwh1qTQmokphowUnVg41c0EV+nXmY2OURQtPgO8p1hCSOd2EoUD8LqKS1Da/s63jD1xBsYlDB0S+CMt43wFmZQ3lnIxh5NtNbzPikmw2ZgoNOxgEEVNomEO8mKM3V2IRhnE+RO0+Fo0wZgyzMq4C5Jt0S5Ltjx1X/fpMnqTDbexZ94Nuv2yXs3Bh2BCG4UZDeaTET+9R5aK+c8cjTI/PrPKMHBz6A8YonDgxxSN3PU6r0SJnMlAvAMMHptkkVi+MxcGhH3H6gSkAppyJsirYEN96QzapL/eT2zyxEJYxjRFLPU+txsTYFD/49lNMnJpWx3YRpHTbh7aUMilxiYl4QZt5TDmmnSnste5x+9g0OKbQoRt6TWxtUKvWOXF4jEfufQK/JTqEKMKKJUzUZDWxTDJuyyCpjuSoCMHAeB0hZBRHFR8skmloMrOkpomEJtb8LRZQzJcR9LxkYuse4RJb94h8EZYyTVrYoxpeWilnomNTWMFofJyOJmIZwxAyJql1K8EUAopBNAyf/j3YV+GIOQzjCikiZ7YBoh4z54Y9N6mcfE9S26QexrxT8d/ipVzm7V+i4waXDhvCMGzKgMYq5DG0sRgBikE3A8mce2CwSGWmRn2mydduf6Djc1Ndtrbx1i4QSXEBh3Ypuy7GYC/5Cdv77TG9/M2zHeOwvjGbeGS2Y2ZzKw0ODzA9UWH82OT/396dB0lV3nsD/57Tp3uGWZhhBgaQZWQZhBCEiVcFJczlJiyyiOK4YSCGiuH1NZqyKDUxMaasSjSaxCrLpEpT+eO9CorBoBcNm1fFa8Br0AgRg4gsss4MDAyz9nae94+z9OnTp7eZnjm9fD/UVPf02Z5efvRvnhUf/PUfAABF0pabsx5pTRStyWKkiVm/1UcjH2sYAUUSuPK/voQsi8igE+PWA0sSaCmfbEkCnRI++2PWgSJ9aTJmc3Pm+Ir6tyk57kjkJE3I9u1qOJJMWgefGKOOFcUy6MTSpByMfm4SLMmhuVKK11wZBcboZUWG5NEHcymRwSeyMWJfVvH5nFEAgMte/0p7DAIwYy9yPYPTMnipYnNzLP4vkCcGDynD/MbZuOKbX3e7KEQ5ZfS4EVi8Yi7GTR7tdlGIiFxXGDWGaggB1d0aw1SkU6torUmrGFKGb90wCwCw/5NDCFqaHKz72Zt47Y/FDBCJV/tnq3m01zY61QTGTLvD5mNKUaYHqhhkSBg7biQaFl2FC2fbcOLwGce6A6fmKtnyuP2+sV9U53hJ6DPPRAadmOc05iyUI1PlwOk5J5uf0IHkdAxXPBkY3iIgU987qc59GFMbaG82jjPQxD4QRQ2bTclSKGS+ryLJZ9CocTTmPZS8CkRAr1nUR/drK/zotX+KgH1pb9mobrewPmQ2KUvONVuZmtuwkDGKc1zFkDLMv3E2AGDrX95H2/l2l0tElBvG6Enh+bNtePv1DxAMZP8fj0RE/S0ragxXrlyJ1tZWKIpWnMceewzTp083t+/atQuPP/44/H4/rrvuOtx///1pnT+oqlG1aNmgN+soA9E1ZZIk4d8XXw0BYMur7+H8+YsJ949ZU9lWuyfsfQzj1BI6DS4xrxFn4mrrdqeyOe3rhDWF0fo7dnJZoprG0rJBmLvoapw/24Ydr+2CFBIxaylb+zLa11S210aYx5iDT4Q+AS8gQWg/UmztiGQdfALE1sTIcvR2+37W/Z2OJUcDEjeKL3M1hla9mtja8v2XaJoaow+htRZRkgFZ7x8o22oO7eXSX0/Jq/XPFaFwZECVeRupMZQUQDL6GBq16dZwgPN97fde9DVmLWJKXE8MhRA4evQo3nnnHTNIrXp6evDwww/jhRdewMiRI7FmzRrs3LkTDQ0NKV8jLMIIiexKDJ2knfQI4J3tH8LfE8AFW02h/VxOSZ59P/ugkqjHEjVDx0n2nAa7xCuf/bpOmBRGG4jYyUcyJHR39GDn1g/RfLIVwUAQPlkxt9lJkJxHN1s6w9s7xGtNXwJjPmpBif6lah18EskgbaOSzWblBCubpDJnYSblWfPwQMWNpPhSH0mcCuM7LNGCNLak0Rxl7FEi243jrc3Gkn7f+GypaiRJjJo3U1/tBPqIY+s1lcgyeeYoZsUTWRnFkhhaB1wZMWEMPpEkgYkfn9GvI8zrGZxSO+tKRMlwxZPkXI/4w4cPAwBWr16N66+/Hi+++GLU9n379qG2thZjxoyBoihYunQptm7d6kZRs0blkHJMmTYeANB0+lxMUkiFgbGTvtrxl+CSMTUAgGNfnkIw0L/TWFWe6MTQk4zPbMK4yX5DT7Yzblzkeo3hxYsXMWvWLDzyyCMIBoNYtWoVxo0bh2uvvRYA0NzcjGHDhpn719TUoKmpKa1rhEQYoSxrSu4NVQgMqRqMxY0NEAI4cOAowv7YLzanmjenmkH7MYmmj4nXbGx/LNF0NPZ9E5U32TE0MLHjhv7qNF47/hIsXHotmk6fw6njzXGvlajmwbo+siFeDQYAdFQXQ1I8qDjXbQ48AWxNyYlY5zFMVbz9ufYxgAGMG49Xq0VLlVMTcJQEX9dGrZ31rReRNYejahudahXtzcuhYORzFHZoDldVSKqiH2LUSlrmOzRqYq1zcxpLS8oSJEUfiCKHY2dekgXah2prTMtNIfNpxTYlU39xPTGsr69HfX29+XtjYyN27txpBqmqqlF9hYQQ/TZKMdsNqRqMZTfPBQD818Z3EHBICqlwMHZSd+n4UVi49FqcbbmAHZt3Ddh1j8weAUVSMeuNQwN2TUqMcZP9PpulzWM49bVjLpekMLmeGO7ZswfBYBCzZmnTrQghovp9jBgxAi0tLebvLS0tqKmpSesaqmW1jlwk9JrC62+eCwHgtVf+G+dbYweaGOLVCCbaL94glYR9FVOsKYxX45fKABzWFsY3ELGTD8aNH4Xrrp+Nsy0XsPnVdyGCRl+mxF/2yfosRXWMFw6PQZhnkBw61kd2tExwnaAfoRQ9x030LaVswOLG4+17H8NU+8ZLcb7fzO89Sx9D276SiPQxjFpfOVHZ1bClllE/d1iJ1JDqk19LHk/sFDeSpV+t1LePsiTcSdhlScrIohXZyvX/Vdrb2/Hkk0/C7/ejo6MDmzZtwrx588zt06dPx5EjR3Ds2DGEw2G88cYbmDNnTlrXCKtqTv+oQmDEJUMBAby6YQfOnruAsFDj/giImMdUfQSyKoR5XmN5u7BQoUKYP0JEflTbj3Wb9Z91H4Oxn5X1mEScjqVoAxE72UpC/JVM7KuSjJswGmebL+DNv+xEKM6UNMb5nJJFY9UU6/XkOPcjjzlcw7aSneTUPpboIOvB/Un25HWz84DFjeyB5FH6+ONN6QdOP7KsDTpJ9iNHjpEkfZlEj2I5l2Vf44Pr8WrNxYoSfYyxXd8GWdZX75EiA61kWZuzU/+BLAB9YJYki6guF0aIyCL2Meo/rtcYzp07F3v37sUNN9wAVVWxYsUK1NfXY9myZXj++ecxfPhwPPHEE7j33nvh9/vR0NCAhQsXul3sASPLElRVYP8/v8TBz48h0M+d5Sl3MHYSk2VJG7m/40MU+bwQYf6hQYwbomQkUQDVMpd/rQFffXXS7WKkrapqMJYtn4vtW3bj5MnmuPul8hba5zBMdGyiuQaTNUnHO2ehNxvX1o7G4UMful2MtI2feBWOHTuR8fMmasaNmSPQtq/TuseSFJlSxiPJGD9hNL7571fg9Y1vo6O9Cx5ZhkevaTNuFdkDRdJqxozpanyyAp+k3S+WtbnYBsleDNLXTS7Vt5VBQbl+bLnwoELVrl2pt65VhFUMEUF8smwcFFnFNW8eRFlpAKWVfgBA0RCtyU0Z4oGnSutoLw0ugVReqt0v025RWgqU6I8Vl2iPFQ8CfNox8BVrt94iSF6fdl8p0m71GifjPoz7kSUmIs3TsqU9z15baKuhdJxKJ86+GSEr8FbXZv68/cx/9GMg5E+8U1+7OCWa09BybmHdz2ieTrTyiQhDhC3rI4eCkfsAEAoAQf25BQPaIQE/EOjRHuvu1m67uiB6tMdER5d2294F0a49Fr7gR/C8ds1AmxZbXRd92NagLe06Te9j2AYFbfrglQseLdbaZIGL0I5tRwid0J5Xp9DK2iVC6NbXqu7RHwuqIQSE9hyM1dCCamQ6O2OQalhorWrGfae5e+2DL5MN4DT2iac/vv96+73jeo3hQMjFZsmqqgo03jofANDZ1R217Fw6Ej3veH0kkiV2qY42djo21esR9db4CaOx9IYGtDSf71MNe19GRxtNX+P+9wxKFb2/lZzC5zzRkngJLygDUv42/+Yk2SHJjpHiexavv5/HlohHJZqymfBJxn5CjexijCC29DG0jkCWjH6HHkuaYBxs/QPC2ofQ/oeG5Y8I8w88WYr6RnD6W2LKHq0iR+rFd54V5yzsnYJIDHNNVVUFbr5NSwpfeXlbwoEmRBQxwZIU/uXPbyEUcHeaqooz3RjsS1JrRERRqpo7AQDt8LlcksJUEImhMagi2wkhUF5eaiaFL7+0Fa2tbX06Z7KRU+k2+6ZTW8iaQhpIY2tHYtmNc9HSfB6vvrIDgUDIbDZORypTk1j3iB6dbJmrEAJtI0oQUhTzi84sjnnbj6MqJQ9HLbtJkjP3+ltrBhM2P3uiaxftl1ejaw/NG3M/Y4RxyOx6IIWDEPahwx4lemUUaKPmY0YgW+fhjHrMWOFHsoxKFuZta43WfcJ7In6Nf1QMShJrBTOoIBLDXNLR0YUDB47gk3983uekkKiQNDedw/5Pv8T/vPsR/P5gVsw9d/jq4VBkgWvePOh2UYhyxoErtXkMp5046m5BClRBJIbZ0McwWc1dVVUFAoEAOjq68dZb/5ux6yZ73unWDiY7ZyqDTFIpF1GqRo2qQVNTK3p6Ati2dVevagmTcZx+RkgJt0cdnyhHta6VnPAksvN9ykqSrAAeywCQjM2la1nPOMFmjd4P0KhFtG6zdBc0i2ZsF7K5HbKs/W7cN7bLtlpExzW8Jef+suZn3vkpxJwmtd30fd3/gzDXFURiqIrkiZkbjOSouroCt69YiNbWNqxft6V/rpVCwpboNepLgpnOeYjSMXHiGNy4/D+wZ89n+J+dH6d0jLUmUU6p2TjxnIXOxzhdtxeffSaAuUuW9ATLmFA9s++liHc6a75oZHzGQBHb2BQhDHixAAAWZ0lEQVTjMaNoZoIoqZYk0RMZiGKcJxyyJIT6Y5LDbNWybI5iF9aJ3HvBOCp6EnnqD3xdXaYlhdcBALZuGbiluohynZEUNjW1YveufW4Xh4goLxREjaGA+03JTqxJ4fp1W5L2KUy1Vi5VvRmYErU9jfJk4+tPucuaFG54eRsCgSA8OVS7llblkf15WadAyaHnXLAy/R4lqIEUQo2+nr21WUakWdms1bPuZ9QwykDYoanaWiPo9LzMASaJawUl6/YES+KZu/Xx6yPVmn7SFERimK2+PW8mgNSSQiLSKIoHC6+71kwK/f5AVgw0cTLx/dMo93G1IqJ0TPvguNtFKGgFkhgmX5vXDa+//i4GDSrKeFLY2/6UqdbqsT8huSkUCuPll7aivb0LgYD7o4/jXV2WgIrWHpT7AvGPtdessAYwf2RyuhrrwBWnz4hDLWJU7aFqnZjaOCZ2jk+zP6CKyKTX1tpB4xBrGRINPnGarsYarw41i5IkUNGqrZzCeQzdURCJoSpil6pxS3V1Ja6+6uvYum0XOru60dnVPeBl6E3CxmZjclvdxLGoGV6F3bv24uzZCwBSm3MwHqfRi31tcrIf3TqqFN3eItScate2p7L6CdC3BNEpGeF8hgNP9qS25F28VU2s4r1/wtL0a57PIUmULdusTciA1npsvW/csY5Ato5QBrQE0bHtN0GSmIbmS8oBAIOOcnJ4N/B/iwFUXV2J79yxCHV1Y1FeXuJ2cYhyRt3Esbjppm9hUt1YeDy5s/Tb0StqcHDGSLeLQZRTDs4YybhxUUHUGGYDIykEgBfXvYm2to6MXyPTNXW9aX5nbSFlmpEUNjWdw/qXtiIcDmfdXGXJ/sLu1VQ1VBiSrqesc6pZtNfaxQw+ia49FDJi5z+UPTEDTSRZhghbrmeuwx2KHGMen2ItYT93+ciu/xFyG2sMB4A9KTx3jgNNiFJRVxedFPr98fvrERFR3xVEjaHbK594vR709Pjx5407sjYp7MvgHNYSUqrS7RNYXOTD6dNn8fKGbQmTwlQmqk4k3eO5NislZZ3wOZFUV0Rxqlm01yJGLWOC2MEnUY9Zrm/vd2itQbQ8B7Pm0XqeRJxWPrGslWw9faL+tzKDbUAVRGLoluLiIvT0+HH69Fk89/yrridQmR6Z7fbzofxlxM4/Pz2E/Z8dduWzlixZzFhzS5ZOtUMDpC/Jo2xb8s56PodjJElObWk+SbY0H6spD14yE8dU9+/lKijUv9iU3E+qqyuwZk0j/u3fpgJgEkWUqrq6sbj3h7dh7NgRAHI7di577yRm/O2o28Ugyikz/naUceMi1hj2g+rqCqz8zhIAwJEjJ/rlGm7Ny5jLX9KU/erqxqLxpm/jTNM5NDW1ul0cU7qDXYymr9ILAZQXaVNuJK1EGeDak0yv3Uv9yPpeOdX4WZuZrSub2Kezidek7NSEnGp57OsjZ0D5RS1mLqIoY+ek1GVFYvjss89iy5YtAICGhgY8+OCDMdtfffVVDB48GABwyy234I477kj5/GIAJ7i2JoX/+eLmrO1TmComgtmtv2NnINXVjcXNN83DmaZzWL9+S14MNDlbW45ObzFGHs/t/wfyzUDEjRRv2bg4hH20cNwTJ0kSnfa17Gc29zoda547XtN1knkMM0CSgdNjKgAApYd6+uUalJjrieGuXbvw/vvvY9OmTZAkCd///vexY8cOzJs3z9zn008/xe9+9zvU19e7WNLkfD4vVt6RP0khZbd8ip3hw6vzLikEgK+mD4VHFkwMs0g+xU2+OvR1rRvJ9ENH3S1IgXI9MRw2bBh+/OMfw+fTlr6ZMGECTp06FbXPp59+iueeew4nT57ElVdeiYceeghFRdlXxRwIBPH2Ox/i5KnmnE4KWUuYG/IpdpqazuGdd/+Oj/9xAAE/1xam/pOtcSM51C4mrUV0GmjiNCDF3CbHzmOYpDxCyJGlU1Ks1Iy6ntN9ymquv1N1dXWYMWMGAODo0aPYsmULGhoazO2dnZ2YMmUKHnjgAWzatAkXL17EH/7wB7eK66i6ugKX1l4CANj3zy9yOimk3JEPsTNhwhhUVmrLX+3+YF/e1BRS9sqHuCHqT64nhoYvvvgCq1evxoMPPohLL73UfLy0tBR//OMfMWHCBCiKgtWrV2Pnzp3uFdSmuroCq76zFEsWz4GcY38RGfM72n8ot+Rq7NTVjcWtN8/Ht7810+2iFC45vX5w+SQX4kaSZfMnyY6xff6sA1KctifS28+E0+fJpc9YX9c9L2RZ8T/CRx99hDvvvBNr167FjTfeGLXt1KlT2Lhxo/m7EAKK4noLOIBIUggAL23YAjXVzsMDKF7yxwQwP+Rq7FgHmmx+I3uSVUD7Qon3pSKBS2/lg1yMm5QSxHTOJ8lpD5Bx1Jtk0Gnia8oarieGp0+fxj333IPf/OY3WLx4ccz24uJiPPXUUzh+/DiEEFi3bl1UJ2G3WJNCDjQhN+Rq7FiTwnXr/5rXzcdf++8TuOK9w24XgyxyNW4KyRXvHWbcuMj1P4P+9Kc/we/344knnjAfu+222/D222/jvvvuw7Rp0/DYY4/h7rvvRjAYxDe+8Q1873vfc7HEmvoZUwCknhSyho4yLVdjZ9bM6QWRFAJAcWcQJSGHwTRc8cE1AxY3iZpvU10Gz/G0+qAQpxYq+5J4TpzmNDTL6TBgJdE5eiOFz35JpxYz7fD1/jrUa5IogIxlQt3VOHYssxNNS5KE8rISXGzvTGn/AniZKYHa2tE4fOhDt4uRtvETr8po7EiShKIiLwApblJon0zavr6yLEnmPsY2WZIi9y3bPPoXnkeWzfuK3vdKkTzw6vd9svY3sk9SUCR7AQDF+m2RpKBU0u6XStp+ZZKCcqEdWw4Zlap2zYqwFudDwmFUSCE0TajAIG8Qo4+cR9lgP0oqtOfsq9b2UyoUyEMGaeWtKIVUVqrd129RUgKU6I8Vl2iPFZcAvmLtvnHrLYKk6KNmFf3L1OOF5NH/9vcokT5n+usgeZTYL3hJjl2T15bcJGzO7I+57WQF3urazJ+3nwXPHQPUkPPGPiSGkVPEOYf13PaRyVHbtPtCqJEkMRyM7BfWyi7CQfM+Qn79sRDg79Yf0+O4pxMI6Nt7urTHujqBLu2+6Og0b0Wbdl89343Qee2agfPaZ6erzYcD1dp0NRUHtP0uCC/Oe7TPZZtHi7ULskC7Pky6XQqjQ2hl7NRvu0UI3UI7d4+q3frVIAL69oD+3gTVMEJCe51C+usVFirC+msSFqr5/a3q8yELIaCKyH1Amy9ZtX3P27/3E82n3B85Qm+/d1xvSh4IifrZpfNTXVWBO1ddj/LyUqiqiraLHSkfS1TIJtXV4vZbF0JRPPD7g3lfUwgAEgROTq3Ckck1bheFso1Rm2j9SfsUKRwje2IT/b6QPJY1lB2ulSFHJtcwblzkelNyrhhaXYlVK7U+hV4vXzaiVE2qq8XNjfNw5sw5eDwehEIpNlcRFRKnOQkzfZ5kTc1GcpfK0niZGgjDbhVZhxlOCqxJ4f97YTPOnbvgcomIcoM1KXxx/ZsIBDh5NRFRNiuIpuS+qKqqYFJI1AsTJ4yJSgoLofmYqM/SaFbO9BQ2MeWQ5PhNxwNMBhOWgcIawyS6unpw+nQLtr/1AZNCojS0XezAl4dPYNNrbzMpJCLKEUwM46isLEd7exd6evx4acNWt4tDlDOGVlfi7LkLaGk5j5fzIHb6soLCtO3HUOZj8zmlKVP9DXPUVW9/CQAIsI7QFXzVHQytrsTqO2/A4kXfdLsoRDllUl0t1vygEVd842tuFyUr+HrCKPLHmbIEYMd7SqyXI5ZzXZE/lDhuqF8V3icuCetAk7/t+sTl0hDlDutAk0/3H3K7OFnh1GVDcGxitdvFIMopxyZWM25cxKZkC44+Juod++jjQu1TaP9L+/RlQ+CRBWoPnXOlPJQnkkwzk3A1lExdvz/EqTH/qm4oAKBqf3v/XJcSYo2hRWOjth4mk0Ki1JWWDsJNy79V8EkhEVE+YI2hxabX3kYoFGZSSJSGzs5u/HnjDhw/0cSkMNP6ayoSyn/WWkZj4mr7EnlEDgr+f52h1ZWYefXlAICmpnNMColSNKmuFpMmaWvYHvryOJNConxRgANeKKKgawytfQr37juI7u4el0tElBuMPoUnTzXj4MFjbheHiIgypGATQ/tAEyaFRKmxDjR56eXcn6ewP8346xHOY0iZke9zG1rGocza/gUAoLtwUxRXFeSrztHHRL3D0cfp8YQEFE+efpETJdPLPrJKOHnMyALow9zzlEBBJobDR1RDVQVeWPcGk0KiNNTWjmRSmIYTU6tRpIQw/kCL20UhyhmHJw8DAAzb2+ZySQpTQSWGHo+McFjF/v1f4uDBYwgGObM6USqM2Nnx1gfwehXGToqaJlTAIwsmhpS1JEmGQHbVap8cVwWAiaFbCmbo0dDqStxz920YP340APCLjShFk+pq8X//z60YMmQwAMYOEVE+y4rEcPPmzVi0aBHmz5+PdevWxWz/17/+heXLl2PBggX46U9/ilAovS+mqiGDsWrlUiiKBxfbOjJVbCLX9XfsjLt0FG5unIeurh50dXGAVp9JYL+oLNDfcZMTZJnzZJIj1z8VTU1NePrpp7F+/Xq89tpr2LBhAw4dil5n9YEHHsDPf/5zbNu2DUIIvPLKK2ldY/mN3wYA/OcLm3GWfQopTwxE7Cxe9E32KaS8MhBxQ5TLXO9juGvXLsycOROVlZUAgAULFmDr1q344Q9/CAA4efIkenp6MGPGDADA8uXL8cwzz2DFihUpX6OiogJ/2fQWSsvKUFpWlvknQZTEqFEjM37OgYidYFDgvff3YsSImoyUWZKSV5dJtio1+zGyFKl2M/aVpch+srFNkiDrU3x4ZFk/DlAkbRUIj+SBV68x8craf4VeyQOf5AUAFOmPFUsKiiXtfol+bImkoExox5ZCRrmqnbs0LAAAg1QVRVIQ5dXaWsne0TVQygLwDNamrpErtf2kCgVSeZF2v7wEUskg7X5pifZkS0qAomLtsSJtG4oGAT7tGHj1W8UHSfFp9z3eyK2x4oVHidQOGdOeyErsWrWSHDnGfMxhn3j6Y2Jke3n6aCDipj/KbUo0XY1k2yaEbbvDmsuSiJ0KR1UB/bMuyUrkuVhv9fiI3HoByXIfADxFgKR/LmXtcwylBPCUauf2+iF7tdpYj0+PzxIvyquHAACKxmgtfIOEFyH98xvyGPEtIOvP1yupKBbaeUqEtrpLjwihR3/Mr2q3ARFEUN8e1B8LqirC+mMh/VYVAmF93WlVqBD666hC6C+rgKq/tALGayyg2l5vYf8dtvcjwb6Z0NvvHdcTw+bmZgwbNsz8vaamBvv27Yu7fdiwYWhqakrrGvfcczfuuefuvheWKIsMROw89NBaPPTQ2r4XtkBd6XYBKMZAxA0AeIeM7ltBC9h3jTu/dLMUhcv1pmRVVaNqBIQQUb8n205UqBg7ROlj3BAl5npiOGLECLS0RKZyaGlpQU1NTdztZ8+ejdpOVKgYO0TpY9wQJeZ6YnjNNddg9+7daG1tRXd3N7Zv3445c+aY20eNGoWioiJ89NFHAIDXX389ajtRoWLsEKWPcUOUmCT6o8djmjZv3oznnnsOwWAQjY2NuOuuu3DXXXfhvvvuw7Rp03DgwAH87Gc/Q0dHB6ZOnYrHH38cPp/P7WITuY6xQ5Q+xg1RfFmRGBIRERGR+1xvSiYiIiKi7MDEkIiIiIgAMDEkIiIiIh0TQyIiIiICwMSQiIiIiHR5nRhu3rwZixYtwvz587Fu3Tq3i5MxK1euxOLFi7Fs2TIsW7YMe/fudbtIfdLR0YElS5bgxIkTALS1TJcuXYr58+fj6aefdrl0vWd/Xj/5yU8wf/58833bsWOHyyV0lq9xAzB2cgVjJ/swdnJDRmJH5KkzZ86IuXPnivPnz4vOzk6xdOlS8cUXX7hdrD5TVVXMnj1bBINBt4uSEZ988olYsmSJmDp1qjh+/Ljo7u4WDQ0N4quvvhLBYFCsXr1avPvuu24XM2325yWEEEuWLBFNTU0ulyyxfI0bIRg7uYKxk30YO7khU7GTtzWGu3btwsyZM1FZWYmSkhIsWLAAW7dudbtYfXb48GEAwOrVq3H99dfjxRdfdLlEffPKK6/g0UcfNZec2rdvH2prazFmzBgoioKlS5fm5Ptmf17d3d04deoUHn74YSxduhTPPPMMVFV1uZSx8jVuAMZOrmDsZB/GTm7IVOwo/V1QtzQ3N2PYsGHm7zU1Ndi3b5+LJcqMixcvYtasWXjkkUcQDAaxatUqjBs3Dtdee63bReuVX/7yl1G/O71vTU1NA12sPrM/r7Nnz2LmzJl49NFHUV5ejjVr1mDjxo245ZZbXCqhs3yNG4CxkysYO9mHsZMbMhU7eVtjqKoqJEkyfxdCRP2eq+rr6/Hkk0+ivLwcVVVVaGxsxM6dO90uVsbk6/s2ZswY/P73v0dNTQ0GDRqElStXZuX7lq+vP8DYyVWMHfcxdnJTb2MnbxPDESNGoKWlxfy9paXFrF7NZXv27MHu3bvN34UQUJT8qfjN1/ft888/x7Zt28zfs/V9y9fXH2Ds5CrGjvsYO7mpt7GTt4nhNddcg927d6O1tRXd3d3Yvn075syZ43ax+qy9vR1PPvkk/H4/Ojo6sGnTJsybN8/tYmXM9OnTceTIERw7dgzhcBhvvPFGXrxvQgj86le/QltbG4LBIDZs2JCV71u+xg3A2MlVjB33MXZyU29jJ39Sfpvhw4fj/vvvx6pVqxAMBtHY2IjLL7/c7WL12dy5c7F3717ccMMNUFUVK1asQH19vdvFypiioiI88cQTuPfee+H3+9HQ0ICFCxe6Xaw+mzx5Mn7wgx/g9ttvRygUwvz587FkyRK3ixUjX+MGYOzkKsaO+xg7uam3sSMJIcQAlI+IiIiIslzeNiUTERERUXqYGBIRERERACaGRERERKRjYkhEREREAJgYEhEREZGOiSERERERAWBiSEREREQ6JoYUpbOzE1OmTEFzc7P52MGDBzF79mx0dHS4WDKi7MbYIUof4yb7MDGkKKWlpRg/fjw+++wz87Hf/va3WLNmDcrKylwsGVF2Y+wQpY9xk32YGFKMadOmYf/+/QCAv//97zh06BBuvfVWtLe3o7GxEfX19Th48KDLpSTKPowdovQxbrILE0OKMW3aNPOvt6eeego/+tGP4PP5UFxcjOeffx4LFixwuYRE2YmxQ5Q+xk12YWJIMYwg3bZtG3p6esxFt71eL6qqqlwuHVH2YuwQpY9xk12YGFKMyZMno6WlBb/+9a+xdu1ayDI/JkSpYOwQpY9xk10UtwtA2cfn82HSpEkoLS1FQ0OD28UhyhmMHaL0MW6yC9NyihEIBNDa2oq1a9e6XRSinMLYIUof4ya7eH7xi1/8wu1CUHZ55plnUFpaiu9+97sx2+666y58/PHH2LNnDzweD6ZMmeJCCYmyE2OHKH2Mm+wiCSGE24Wg7LB//36sWrUKl112GZ599ll2+iVKEWOHKH2Mm+zExJCIiIiIALCPIRERERHpmBgSEREREQAmhkRERESkY2JIRERERACYGBIRERGRjokhEREREQFgYkhEREREuv8Pzds3ajoYEmgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plot_softmax(ylo, yhi, n=100):\n", " y_grid = np.linspace(ylo, yhi, n)\n", " y3 = 0.5 * (ylo + yhi)\n", " p_grid = np.array([softmax([y1, y2, y3]) for y1 in y_grid for y2 in y_grid]).reshape(n, n, 3)\n", " _, ax = plt.subplots(1, 3, figsize=(10.5, 3))\n", " for k in range(3):\n", " ax[k].imshow(p_grid[:, :, k], interpolation='none', origin='lower', extent=[ylo, yhi, ylo, yhi])\n", " ax[k].set_xlabel('$y_1$')\n", " ax[k].set_ylabel('$y_2$')\n", " if k != 0: ax[k].axvline(y3, c='gray', ls='--')\n", " if k != 1: ax[k].axhline(y3, c='gray', ls='--')\n", " if k != 2: ax[k].plot([ylo, yhi], [ylo, yhi], c='gray', ls='--')\n", " ax[k].grid(False)\n", " \n", "plot_softmax(0, 15)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The example above assumed output activations that can be large and positive, such as `relu` or `elu`. However, the strength of the *winner takes all* effect depends on how the outputs are scaled, and is relatively weak for output activations that saturate on both sides, such as `sigmoid` or `tanh`, which is why these are generally not used for classification outputs:" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAocAAADaCAYAAADUm+jiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9e4wdx3U3+Kvunhk+h++HRFIPi3o/LDl2pMi2bCu2FMly7AhCHg7gIAG0gLGON4bXSQB7nUWAJAiSz/42X2DHm0UQ5FsnG+dLYkn+YomyJUu2XrHelki9Kb7JITkkhzPDmbndXftH9ak6p6r63jsPcng1fYCL23Oq6lT1ndvd5/5O/c5RWmuNRhpppJFGGmmkkUYaAZDM9wIaaaSRRhpppJFGGjl7pHEOG2mkkUYaaaSRRhqx0jiHjTTSSCONNNJII41YaZzDRhpppJFGGmmkkUasNM5hI4000kgjjTTSSCNWGuewkUYaaaSRRhpppBErZ41zODo6ijvuuAN79+4N2nbs2IE777wTt956K7785S8jz/N5WGEjjZyd0lw7jTQyfWmum0YaqZezwjl84YUX8Bu/8Rt4++23o+1f+tKX8NWvfhUPPPAAtNb4zne+c2YX2EgjZ6k0104jjUxfmuumkUbay1nhHH7nO9/BH/3RH2H9+vVB2759+zAxMYFrr70WAHDnnXfi/vvvP9NLbKSRs1Kaa6eRRqYvzXXTSCPtJZvvBQDAn/zJn9S2DQ0NYd26dfbvdevW4dChQ2diWY00ctZLc+000sj0pbluGmmkvZwVzmE7KcsSSin7t9Za/N2NTB3dBZQ54FcKLAt3rEuglO1al7Kdv5vF1evg2dOFN4a3MTu2n2u36yhLY5fbKUs33o4pa9amvfkKoGDr0rJday3nsWv07JSls8PHUL+yDO2w+XRsbFGGuryABtnUVufWU9nL2Wdfus9TF26szr01sjH0L0AJO8ZOkWv3L7D9lB2j2XFZKLSSFEcHl2Ld4ZNI16/FOX//VzhTMhfXzgc/9Ens338QCnKcUgqJtaWC9kTBzpVUbUopq6P+idA527YdKmoHdrxbQeLZVkohVYlrZ/adLglsJ2xt1JYwO0YHZtuNp/dUmaPzP/IuKAB7Hn4bCkDqnWsKJcZmkO2J4rrKNoC0+j4mfD3MJqo+XJfYMQjbq7YMQFJ92TPbX7N+7Ni+a6TVRZFU12eqNNLqOFGVvbREUi0iTar+iUZSTc6P07TA6JIBlH0J1i1fgnX/93/DmZK5uG4A4Ct3/iGGDx5FSfcsJjqic20xXagt2TMkPkd7O7F2bpP3Lb1+wk7NGH9t0nbcFrd33ceuggbw7IM/E/3kOYTr4Gv1bZdaR+dr10+6BdqO8c8vqoP2Ph9vrXXrsecSOT96FkGDf5L0+VI/DY1zz92IHz9yD2YiZ71zuHHjRhw+fNj+feTIkWgooK20JoCiJR0lgDlylQNWeu3CsSoiutAB01ZXSHs64mTZNueg6Xb9yiLUC+eQtQX9irAfdwhjc3eyzR1L7hD6jmBRIOYw6sJzmEsdse10uijclRFrz9m8lXOprZOojQNIutxrz7U91rm9+uyx9adzfmweGGWu7DVa5gq6NPpJZNh2w1U4unIZfmXHHgyWZ3YXx1xcO/v3H8SuXXuFwwVUTh1zCLme/k4iY2z/qHMY6mLOYaKSwBHk6+FtdJwiiTqZaeUq8TZy+hLmlDkdIjplHTI3n1nP4NFVUACO7B1Colg/uH5Oxxw7vp5KRzdrpZU9tv2BwLHMtGvPah1B2PGmnw77aekcZuQAsve+6pGcVo6gVhpaVTrrlWoklVOorRNYAllpJ1SpOT4+OIDvX/Nu9Oc5fu3t3TiTMifPHAAnDg1jeN9h4RSRRB2zLp1I31Gr7TcN59FvjzlLde2dnLH4mJjDKNuHj54LABjaNwStdds1RttrnDV/bXW62NpiuraOnOcc+nNqraOfre/oxZ1ELRzBYI7I9246clbsOWwnmzZtwsDAAJ555hkAwD333IObbrppnlfVSCPtZSpNse2Gq3B41SA++MyrWDLROuNraK6dRnpRRpYuwvdvfDeKJMFHnttxxh9SzXXTSCNnMXJ499134/Of/zyuvvpq/OVf/iW+8pWvYHR0FFdeeSU+85nPTM9YUQBFLsPEQIjs8TCz1YVoog50RbRfFDmMoYR8PRzp83V8zR2RwyLsF0Pvou1tbJcaKHJvPg8ttKHaPGjXAk3UoZ1coona11nbRWWPtdvzYmFjiwaWNcc0xqGINkTMQ8iUyaJU9pgQQl0o9m9TmFIZtr3POIY3/fQVbN5zDEWpoIrph6ZmInN67aAKGrcJq/EQs4/OxWwBEmGM6WJh3tgaYihhDE2M2eHnxVFCfkx2OGLodA6pC0LalZ23/+N1pNXnkzDbHC0k2wk7hyTSrrRDEHl4m/pzxDAYK/pW71oihmZdEjF0Ol3NrQViSDoK71NYOVFOpyo0MUlKpBwxhAkl23alMTo4gAduvAZFmuD2J1/A2lOjwOqlOBMy19cNIcFJ5HsbQ/JUBOjRketIdUQBjaRs3iSCLEmb9fOW0NaWQ6iUtZPaMdpa4mvgY2h+h7RzxE+2P33fC9DQ5lpQyp63Q8SUG6vcNWXblWtP7CkznaLPw5031/Fzpj4J+6BKuWykUA7Vo+tBu2PAXXt8rP/jp9RsHTSOrdHaZu3Q7n9D37cSYHeF6YvSs8Uee0Cm9m0HiqnGOWycw8jx3DuH2887F09dtRU3/fQVXLD/CIoigS6B7NwNuODB/45ekosuvh67d+0LHLPA8WrjHMZC0bNxDkW4mO8pbOMcpiqJ2omFkDN/fyELB0ud23Poj+F7CWMh4lgo2YS069vTiHPIw8vO0UNkLOurXb+5cg4pnEx7D9PE6SiUnKVx5zCpjtNU4/H3bcWeDWtw+1MvYPXJMSSZRrZpA9b/y/+HXpP/7f3/C47sPdx1CHmuws+dws6xecKQbbytnZ2Zhpr9uTuFojuFi6ezH7BuvmCNHULQ/tx83rqQMI31XTC537HN+rUMP8c+5/PO24Q3Xn8SM5GzFjmcS9FlbhyawBnzCSlz5AiSziehANIZ8x1B3redg1bbzhzCgEjCHLQ6hzA2t93jxxxCTjSJ6by9hDri1EXt5IW7CJhOOn0eYSe2v5A7f0w3F/sLzb+VjqtfaLlyS8wVLn79ENYOjWLl8XEUSFAWZh9ieYb3HM6FhESTCOpWgyoG+xAROpExnb/PMLonMYqoROaLOKmc7FI3NjjXiG3qmfB2oQPWXLUeiVI49tLhah8iRD++xlSF7cbJDMfY/YesLSCcQIn9hcprV94xKhuEMjmdFjq3HtJp4RSac3H7C0nHUUKLFjKHMUlL3PjS6xh7czdWTYwDCaBSDZXKh16vCO0nja0+dsXMBGF0h/y7HHOuwuuA96HvYGyfYsrGamEndA5ngiYqz8lKKsTv/Gs2AwB2vbgXpdYOYbPImBZoIa2Xzk9DO8SNjSFdbD30iPB1DjF0n1MMeaSPx87hIZoc9bNjvX97wmz6CKJv29oxJ2zP24xtH/HpJAvCOUSR1xNSYmQSq5hDR5D3BRAll/C+dc5hzKHsQDSxawkcxhrnkDuEEbQxqmMOoY61B+gec+AEsYXmriGceLYlMznUcQRRoITEQqaPvtTQFhGs3nOGCJJz6KGEpr/CpMrw2HsuwbUv78Ky4xNYMTyOskJ6ykJBaxV8/XpFBDJYE6aIkT2iSF4XNysfLWxHNIk5dXVMZ98prGMm89Cw0YVhYx4ijoWdCU0cPG8lAOD4S4crx0raydgnqoCAmZzAIYbcSaQx1D/VCNaQcJ2OIIdgiCA9FKEjyKEjpGTcESTnQWl69lrHRLGwMkcJ7XE1iUqAk8sH8MxVF+KDL76CxTpH/8Q4kozaNVTv/aYCYJyCDCqK5MWugpk4kZ2dR2mhZP/L2Lz8+mwXihaOGVvtTBxGP2RNYeV1560BAOx5cZ91GPlajONd6TwnjGwrz3aiYR1FHqbmjqJbd/U5KLj1am113JH0x/hOIq2RO3aAdATZxxiEn8180vnlIWs/xMzPeabSo5ddI42cPdJKU/zgxiux+5w1OLF88Xwvp5FGekZOLl2EBz5wDQ6tWYHxRQPzvZxGGmmkkgWEHIZhZRPubBPS5To3aHYoIW+LoYTBWA8t7AIljK6nbm9i3l0oulOqGh1tZyigryt5Khu2/tieQmEnRATjKCEdw9o5HSHkVpbiwYqV/IGnXsWmPcdQFkl1Osq+l6WCynvvt5jMZxjXdyKNRO1G0MQ6Ekt0TBs0sROZpS2aKOy4ef1+nGjA21OxRhrvkMYwRBxHLTkJhYeJqY2TYUjH09a4eV27H3ZOWKjO2tEIdZAhZDveohYaqZe2JmVpa+ijN2Hl6jjVOLl0ER780FUo0gS/9MSLWD02DuXFy5MMUNnsEJD5EgorR9G/iG4mCON0kcW6kLQLIdfM4yNwUHOGJvqhb1WFdPl+YR5WdoQLh9T5KKLVMRSR2wbqQ7/Un4eSOWII+KHmUOcjiDQfJ6rQ3DFCiv3+R207BLE2xAwAun7LTzeyMJzDsoAu8qhT1ylv4YwcwVjbdEPIbO3TIpX4dtqFjX2HsNNeQnovyHGrcQg56cTTae78BefSxZ5Cvq+QdH7OQi+EDJgw8nRDyNAsnEyOHgsrT6o+/OCGK3Bk1SDe/8Rr2LJ32JBPqjBgUTmJWpvjpAf3HJJE8xy2ufHMlHwChPsMYze4duQT28e308YRjDKcRdjYc8aUdAijibPhO4iSfEL9Y8xlHkr2HUZOPuEOn+/UmbBxdaxD8okS47lOC50JK1fhZ8TJJz4zWSntEl5HyCejywbw4E3GMbztyRewenQMSZ/bX5hYVo1zKHtNyDl04VQn3TqC/I7RzZhIhFLaq3Ecfaa0+zbIUHRdCNqsNczZ18lhFCFm8b1TvLe3B9CFlSnBuu8okm3uKAIyhAzWFmMZ8/P3nb5OoeaYYwkVCSszh5FEhJpVqONOYl2I2aybGZiBLAznsJFGTodojawo8YGnXsWWvcPzvZpGzkIpi3IWt+d3riRaY/HkFN7/3GtYPT4238tp5CyUsoi5w42cKVkYzmHRkoSUOuRwpighb6/r31UIuSZs7BbDELgYmtgGOSx155B0F+QTnecI0UQt+8XS0XD2sT0XaacbwomfoiYaQmZsZLs5N9cMJZxZCNksS2FC9QEAsqkCH3lkO3SRWPKJRA7dmKJUSP2fiD0ghJC1RQlZ+2yZydTWKYTM12dtewgk71vHevYRP8VDvzXM5KBftN28Dm7byVDEEIEkRDG0aYSzlRVrIwQi42O9sLHyQsQBsqgZqcQijDxVDZuPhZUd6QROR2giQwsJObRoYqIxsbgPiyZbGJw8hTseeQ5pphk5RUPRmIwQRLhQc49JopTINcgxNUKyONLWKdTcCVn0+3PULrYG3uKj83WoIhFcJJroYK7po4kspC2IHRrb799h9DCoIf3MihFAfBQRMI8IjiICEKQQjgxahM6eisyRSJ9rt6HmutyJ/jxyPUwHNqZaaieSSozsUkcg7EZ6N87VSCPzIK0sxUMfuAIPv/9yL/jRSCONtJOTyxbh+7/4bjx79QUAmmunkUbOZunR32TTlLIA8in3d4SEotvtBSQbvI23dyKSxGxylJD36WZPoW8vhhzyfYVtbHcin2ieyNq3HRsby1VYarn/kM7L7h8M0cRawkmQv5ChhJH9hTLJNemmv7+wLBK0shQ/+IUrcWT1crz/idegC4OnEOEEwXHiTqFM0It5Dn2pTVQ9xxvDuq18YvvXIJCxYzEGnm12XJe2JuzH1wsxZuW710NBYeSFIZGChifGdmltYKugcAQxZTbtfJXOjmV7CgkF9Ekobi8hnB0fbRRzO7TQIpmK7TlkJJTURwkVLHKYpBonly3CDz9yJYo0wda9B216GqXgjlONpHoiKfYBqLQ33cgMyqYaAmr2DLJT8/cLAhK164bEItDAmn5+/kJd00Z6nkYmtq5u0cQ6MouyY8DGaGy6dhMAjb3P7zM6Lx9iCVWLIpp1a4Ei2vPyEEaODMZyJNahjbF9iO3S2/B2jiAGew5j+wc56sh0PkmFi7kvB+quZYE4h1VYuZLT7gj6urqchL6zxux3naiar6mOaEJzdNJ1IprY8/KcP98h9J3Hwq2Hk0Z8h7GWcBJxDmMhZOvzlzKETB9JtyFk6Rya4wmV4eEbr8CR1ctx42OvYfOeYS+UDHtcUOUUai9NMK6I3f3PelHRCihtegOAIKy0YybXVUCps0l/R0PMbewIJnCU4YyILub8sbHMXkh2MTYXn7sMCsDoC5XD6Nn2y+PxMLFrr8JpVVvGwsXtyuP5JBTfeZQ6o+xUASWBtuE7TkKxYWVWAYWcw9HlA/jhh41jeOtjL5rKJ30UNnYkFMW8WQolq0z1LMSYQYmwbuw0OoaSp+E8+vbqQtLtQtG8XywEPTuH0c0nchv6YVWYpNwrN60AABx4/oBwxmg1ilcS8RxFoHKe2HrJtnXWrDkFnvOQ1ihCwGw86fxQs19yz66BtcfYzNGwciRE3ImkYtdjH5Gze+AsDOewkUZmKU++dyuOrDGI4eY9DfmkkUa6kVIpPPzBK1CkCW559GdYPdaQTxpppBdkQTiHuqhS2VhFBNGrQ/+CMR1QQquLkEv4cR25JOhXgxzOFiX01xBNW6NlX2prV9mkU65ChiYKFLF670Q4iZa9i4SQeS1k+zFNI4Rspla2Cso1L+zB+W8fwbl7jqOgcLEIJcsQMj+FQidVxLz3IJC6Dc0+CWWu0tbU2attZ/Zi5fFiZJgkYseel0hlw86VIXlOFx7TewYltNTHjTHil8fjIWay5/ISun42dMZ0ncrjdZO2RiKQpJMkFL9mcsoqn/BUNioxKOT1z7+JxRNTWDM2Kiqf0LvVZWFYWSUVetiDkmqXkghgFUIYxBbjstehfJ3C0oCH9rE/2tnpNF88HU33aKL2Rhh6iYPB6sLOqrKtQGFuD71TzKaHIgJ+CTyyzU9Y2TYbquVoIbUrZzsWaqbPKVaGz09vE6ajcevRmp2/WAe6Iqn4pfYS3TnS0056fxNUI42cJmllKV69eCM0gOWjE9i879h8L6mRRnpCTi5bhLe3mPJn5w4dx6qR8XleUSONNDIdWRDIIcqqtnLp/Wby9+11hQjODiW0U3eqdczXOJ10NNPQBWhhO6IJRwvb7kOsSWTNU9NQW7uayDWEk65qInv7C8263f7BTvsLi8JUPnn4/VfgyJrlWHNoFKuGx6s2ub/QfExK7DMsqKYy7TnU1Z7DHv0tVrtXsGaP4HSl055Cn+wSS1vDySZ1tZP9usY8dUx0L6EY6/fj60UwhmyXk4VFDNPIGgzi547DRNYcoWTnX30H+T5Cv3ayT0Kxdjja6NlOEK+dzEkoKrrnsJo7LXFy2SI8/NErUCYKm4eOoa/aDKwSh3RZtDDVAiWkhRBaqFLVs7WVE6joA5ZHEGJXTlKDLEb7+rbZcd1+xdg+xZIheWSHpzLy7c8ETZQ1kRWzI8fQnsRiooCGRlZVSAmJNKoWRTTtLv0N36/oE1vAET3bLZ7Khk6Qk0ZiCbbr9hm2I7Hwf3C7ZNl1JJUAlYS8L05XFoZz2Joy5fPaOV70N5eZOoJeW9QR5PN1ChvzObrIRVjLem4TIo47lFqGjmldgR2ni+UqrA0XtyGf1OUsbFf2Dtb5c04fXf9lrtBtCHkqSfGjD16Oo2uW44Yfv4EVR06h0KZfUShBNDGnyZxDuGO6MZFz2Ith5URJJ6u+n3TWppPTMGarY4i5TW7EWHk8WhO3k0R1zBkTeQ5jDqPT8Son9J5A4fhDe4xTqGrK53kkFB62NjpXDk+Gn0Odzzz2SSiZfTiTTiPzyCcp4uXxOAlF+WHl1IWVR5cvwsM3uz2Gi4rckU9SF0KmEzBhYworsxAyP8GeZSsDfVBtw7iyfJyROucx8b06hGFp0V/0o3fNSBNsXMRh5GNt6LSTbc9h5DpOCuHz1oWddz30JkqY68L8sJJOZl2+REEQqSxzxzNw0CJr80PNfggZKlJyL9Kvm7Cyvx7+faD7QDcklVgexNmElReGc9hII11KK0vwo5uMY3jjY6/h3N3H53tJjTTSE3Jy2QB+dPMVyLMEH/3RS1g12oSSG2mkV2VhOIdlMT3ksC71jLXnIWNMF01BE7MTCyF3Gzb219CJaBL0q0ELiUDSMdTshad9tDAIDUfCxVE0sQvCCfv4jC6es5CHkO2yWJUTHkImXVkoHF2zHMdWLcUv/OR1nPP2CZGuhvr5qWpKDYssag0bPraEFCiUUMhnGX49G6SOhDKbtDXtwtPRPIccDeR2AhRFklCi6WgCnQwNm7ESySN7dKwgQ77UniqFZT+3HgAw9syQJJJYO25MBgmYAQYt4OQVwKCADnnkduRYBQgSim1nJJQQgWTkE0ITEy1IKJTTkFdAUUrjwOaVKLIEv/jwy1g7NmbrKHPyCR1zEoogn1RQjw0rZ4rHBHtKCFFWHjIYhoLboX8sxByJPPgaPyTdDeIHIEpsETWMgzXH0cQYSurjnVwXCzuba01hw3vPhYbGwaf3ixCxyxGqBYqYiCvSfC8DgkjEjmZ2XFhZIovxELJEwUXolyGI3YaV25FU5kMWhnPYSCMdhC7B9UMjuOO7z2HRZN6zewQbOXukb92S+V7CaRe6di599SAu3HMEiyZbzZOlkVnLkvVLZ5mpr5HZyMK4hItcIoexhNexpNL8uG4vYW3/DshhDCWsm6/bFDV2qvj+QZdGpgh0uiOy6HRtK5t0SGQtySdw7QC6IZxEayITkleEupLpYvsLdanQyhI88qHL8K7XDuO8t4+ib7xEgaSr/YXU5u8vBGCRQl2hhlkPOpuE9sVqJ8/UHh9fVzs5msImgkCSxPYz+iQUPx1NzLZMVA02Ft4a5RjftoJDdRTrw/caUn8+JoOzT+3WNkMybCLrqh9HE+tIKD7ZJQHYnsNq/yA4+cQhiCqy51Apk+D6yQ9vxQ0/fQOrj41hST4FpIQSwvYDECa8Bu05rI4zBVXBJ8pueFRA2nvXDWAQtRQIvquAh9q1QRY5qphaHe8b7lfkiJ6/T7GO4GL3q/F1sQ52z14ETeRkFmX7OxSw9Prx6isc8ZOJqum755BEm6iapaxx+2KVQBH9udsRUrgdQUxpN0bF9yFGiSRsDd3sOexUSaU2vY39LJyu2XPYSYoCaE26v6PVRbgjN0tH0NeVWuppbBvHs9Yh7MQ4pvk6kU8QcQjbEU24Q+g5jL7z1558wsfAHQOmwom1SZ8DRM7Cbsre+SFkavNDyAAwlaZ49EOX4eja5XjXjiH45e/cR+ZCxTyEDJgwsh9CpmNzCgoFALZZoWekHRFlJjkN29pp4zD6dnwnU6mQCezbDEK/Yoxbi/LsxCubSBKKTyQh29wR48zkOhIK9ecklJhTF8t9GBJOpI5XQTE6F0LmJBRyOOpIKFQFZXxFHx756GUosgSJNixmOybVSKp+nIRinUJGQnG0Z7YQG1ZOnKPYY5Jp938E6h0v/7rg/ZxDqFl/3leONR+h6+uHooVDaMvIuZZuHUbucpL/qSNjSoQhbekYS/KK6afsdagrAo0hiPCfW0RccTbt14Qtzqmc4xk6gvFQs3AII85jLIQMX+c5kd2GldtVUqljMMcqqMwmk8TCcA4baSQirSzBox82juH1j76BTbuaPIaNNNKNjC4fMI5hmuDmh7Zj1cmGfNJII+8kWRjOYdkyYWX7d5xQImouU79OJBZf1y1KyNfRLmzMx3STjiaww8Z2Qhh9ZLHUMnRcjYlVNmmbq7DUonqJbaNfnDw9TRvCSawmsgkXx3QuhAxA5DHUGpjSDjG8/tE3cM7O4y5dDQsha5arkOzRsQ0rs1BOgYQd069Vc9yLyCGAaE5Dn4TSbdqaWAg5mA9yvoBoEqmQ4q+N2qLpaIQOco2Ip5PxEUYOcvE1pkqJ9nK8BVWhjIk3BghJKH46mhRK5C0EDFITQxPtMUNEnE6DV0Gx83nkExNCruauIaGcWtqHR25xjuHqk2NIM0IMKxyKkU84CcUihjyszHIbWiIKDytnvRlW9msrS0TP9fNJHCkcIuTGOkUsJK1tP4km+ldXDFWcSzSRh53deiSkZdBCD+ZiR7oak49NVTZULXGFh9CtRV5xxT52ImPqUEIgCDVzooqvi4WQOYJYF2K2drpEE9uiksoLMVdr8HPETkcWhnPYSCOeJKXGukMnsXXHIWzefcw6co00Mpcy9uh+4SC8E2RgooX1B0dw+av7sfL4uIsbNtLIHMrBR3c3hJR5lIXhHOaF2XfoEzfqEkzbDjNECfm7mSiOEvK56b2bPYV87g57DqeTlqYj0cTalshhgBZ2SGRtPy473uk6EU4cisgRQVpuheQVSaArK3JJK0swOrAIy09M4Irn9qEsldkn2OX+whIhcphDecewx3TKuVIoZvErbr6EUMPZVkEBQkQQaLO/kCGDnaqg0N/+vsBuSCj+Zni+T8cibCpEGH0SCkcUY+2A2W8YJMGGJKEE+yIZQsFJKD6amLCkvRxBFGlt7NyE7pSiCgq9p6rCjbw9h2PLBrCoaGFJOYVfeOoNs6cwNfsLHUpYzacgqqBQm0MJ6V1BpQwtJKikQgtVloTwV4+I2fPp/i7Z95uTS1yi5jiySP1IYvsV6wgusbQ5HG0zfTqjic5+iP7xNXTak0hrInvdoIjGdpy4wm1bVK6GsGLOT0dsR9A7yOoqpXe9IaLrhqTSKUk2rWEmJJVoBZVZyIJwDnWRQ8fyHJbMG6G/gRrCyjQcQdK1I7lwp47NNZMqJqHTq8N1R5jHOi/at0ccQUkuKSP9GMEkkquQE078sncolSWf8Aon3Za9i4WQS/Y+lab4yUcuxdiyAXz0f7yELC9nFULmxBN3zPT2JgYUqjcJKb7UkVC6yWkYkzqH0bbX5EYM5hMPFmdHhH6DMWF4m4eIRXjadxjBna0YYcW8Fv38BigAU08NVSFk6TD6JJSY8+jruNMnCScxnTnOoEUVFLtG+0AhR9Bt2FITJagAACAASURBVOcklFMr+/HjWy7F4Mgp3PyjHbJdMTYzJ6GQHUZCIaewjoSiyClkYWXVo2HlFEAfnCOQulukFUEuET+OpHQKSdcRXPyQtYLGTBxG61Ap992h+xwfGxJSwoosnFziO4qmn4mhrr7+HADA8FMHKifS2bTnqUKbdYQVoPra6dBONNzbJZt5OiSVTnkQaQ0zIqlEciI2bOVGGukgrSzBT37xUgyvXYb3/ehNZLl/22ykkbmXdPXAnKCu8ymjywfwk1suRZEmuPbZ3fO9nEYWiPSvXjzfS1jQsjCcwzw3r1g6mmiKGi3beHs3KCG1zYRc4q/BRws7paMJxvD5Ql3XRBOGFob9eNgZ8VyF7qekecthf0ZNh3ASq4ns67SWIeZWluAnNzvH8JydJxw6yELIPG+hWWr7EDInnohj5VBEVKfJ0cVeEkIIO9VOjoV5u8lp6Ldb2xwJbFchRaCEoR0SmaIGbo1tkDyOFlp0L4Iw8nZOJOEITWLHwI4hvchfSOthOl4FhcbGQ8ja00kSih9WViyVDSeh2JrJicbo8gE8dtslKNIEH3pwB1adHLNhYkppk6SlCyEzEgqvgmJ0EEQU0+aRUKJh5d67bgCDFKYOQLXCM/OIcDADfgJ0rwZVdPWGa2x6k3OUzNnrjCa6ecJ1tCOxcGSQr6kORSQ74TWlJGIYrKE9YSWGAnIEsV11FaVZ4DtCYqHzz3XZkaTSKQ+iWX93aGLhIYW8DrP5e3Y7MhaGc1gWQD7F/p5HRzA6psYR9NfYDePY08m9gmG/tnsJSy1Dx7YfPB0LJdfkKozvJSQ7la6LPYWxsnexEDI/3n7dJgyvW4b3PPQWNrxtHMNuQsillrkKgTCETG10meZKHht7QK7MayFKLLl1zEm0/ZljWbfPMBb6DfMT1oSLhUMJMUaJ9bp5YyX1qD2N5FiknIa0Kgozp8ymnY+tgR4ePFdhLKchT3Rt7bRxInlZPOvAKp7w2u0z5GHl5z9wPoo0wUd+uAMrR8aRZC5/Id9nyBNdA4BKdeAIIoFzBGv2GdoQskiC3ZuMF9pz6DOPuSMnHL0apxGodxzJaeR7GGMh6Nk4jDGnT/N2tk+Rh53N2DAxNj9r31Gk9bi7MKoShPGQtdvvyBw85ezZcnd2/aGj55fh8z8TuecwdDKpLVOJY2ZXD61gnyFzCqPt/nxiDukMKqj2IWvVhJUbaaStXP7cPqzfM4LV+0fneymNNNJT8nM/3olyUYKVTR7DRhpZULIwnMMidwxcoCaUHCeItEUJY2M5yzg2xmcMU7udr03YODa+Lj9hm351aGEnoonTQdrRaJ+rsJSIoXl3Pxl5hZNOhJMgp6E2oWNz7HQTKsP2927G5c/sR99UgdX7R6fNQi4BW1+5LoQMmF/vVqcYyki/+mDIKWUPhscI7YuGiLvMaejbo/HcPm/rZIejhP4663QiXBzLfSjQRO9cwBE4twZJWJHtxEzWI1NuLcymRRC1tK08Owkc45XGZFoLZDHUGaVPQrGVUSy6wyqaMLRwbLAfb1+2Dtc8txvLTk0ia5VICCVkVVA4CYVXQQEqtJDGMGZy7AQFCcUvn5emPVshpQ+SKMTDtPxJY5FF1s8vlddNKBqowriREHS3aKK2/d09LcZ+riOxWFKgXbcWY2hNdSgirUFpoBiZgoZDEi17uIbd7GvnKtQcJY14bGZqc2zm6kiX08qDWDcfoB1Lm6GKteQUZm+msjCcw0YWlLSyBE989GIcW7cMG3efwPp9I/O9pEYWqEw9drCn8hyODQ7gidsuQZEpbH3tIJaOTnUe1Egjp0FOPLa/yXM4j7IwnMOiMOhhJxSwmyomvJ3p2qagidn22yvdjKqY2PGsrYu0NDw/YZyQIvcVWjuE7tWghTw1DQCUuUP63JYMxeofO107wgmviWzzF3r7C1tZgsc/egmOVXsM1+w5iQJmj+Fs9hfSpyzyGLI9hXR6hpBCx/QxmF+r/r6eXhBCCAPEzyOhxNLNdLW/MFLtRAnko71tjvgF5BKPhBLbFximqHEIo9SFaCIHwUJii6wS4dZQtRPix8cy5IGTUHywTbHxgnBi0RGmg+vHq6AABqlxxxpjgwN48vaLUWQKN217BcvHJw2yp2QeQ6qCwkkofjUUlWkkhBiyvYeEAipWO5nvL7R7DlO297BHkcNEa5tWCJCIXif0z42p3lkf/mn4H03p2XMEiZmjid3tSTRa33apIwij0m1RxFj6G14VqGS9+VifsFK3D9HPDRlDAaExrVQ31BbkPmRRhW7yIAIewmhTTKnO6W3s2t3/dTaZEs4K5/C+++7DN7/5TeR5jt/6rd/Cb/7mb4r2v/7rv8a//uu/YnBwEADwq7/6q0GftpLn0STYQSi5C0cwCBf7/WdLLvHXUpeoejb5Cet0bYgmfJ9tWApPOoTWwSMGhmZhYBZWdiFiOlXlQsQREgp3BB0b2ZFLprIUT37sYhxbtxTX/fBtbNw5Yp3AQiezCiHz3IU8hExtzmFE4DyW1cd3OvIcnvZrZxYykxAy/S0cxHbkFKGT8/oklFjY2T7MGAklIK4I265/rOSe0nJM+v4NAAD92CEZ2mZr4BHZdjkNOQlFsfZAZ98lCcXPaZgqVxbv1Ip+PHH7xSgrx3Dl8VOOeKJgj9O0FImugcr5I9vcIXReutGljpkcdQiFo0hs5dQez7Wc7msn00AfCxsq9qyJkUrs3+yYHD0eiuYEFx9fUIiHoKfrMJaQjkw3YWe5FH49OZtmzcptZ4AMNzsdsOzGc6ChcfLxA5VzJJ0/P9TsO32KrYlft+T0uQTayo4STqI9/4hDKRxB5yTGSCp59TDrJg8iqI9lQitrr21ibBH6V9aOX3p0OjLvzuGhQ4fw9a9/Hf/2b/+G/v5+/Pqv/zquv/56bN261fZ56aWX8LWvfQ3XXXfdPK60kbNdWgMZJpdkuO6hnThn54n5Xs5pl+ba6QEZ7BcPqbNVJpb2QUHjA/e/hpUjp+Z7Oaddmmvn7Jd0sP+sv27eyTLvzuHjjz+OG264AStXrgQA3Hrrrbj//vvxuc99zvZ56aWX8K1vfQv79u3D+973PvzBH/wBBgYGup8kL6o8hzEkrz6UPCOUkNuIoYQ17XaOCGFlWulobD8dGRvTsTliRBPSMbSQE02MTgldLFchJ5AAqKqdyH7dEE7cv4HQxARTiUJaaAyMtHDTv+wAchYiZilr2oWQNZQodwfIEDLPXchDyNXpMzSRE1Yg2ouZ/4iLypm4duryHPoklBhhxZdYnsMoacULNQfh4Bo7vi7xrMdyGsI7rkP3whyKXghZq8gYF36mvId+uhmO+MXK4invGCA0UXs6HerAUUKNtPpmElqYKKDoU8jyEusOncRH/8dL6FeFDQ0TCSVJS0FCCULInJDC09vYtDViQVV71cbQQmXq9Dk9LfI0IIdn4tpJKkIQocmuTJ6TUjkCkQtz8nb3HW2HAnJ7sRB0OzRRMzsl+37z8/DDzqUK0zKVCFHEODIoiSuJF1cvoaxtBZNbNIZe+qFmn2gynVBzCsqm6NZvoxe6jNiJIHl1JBWO/rHPD0CUpIJY6JsjlfT84Ug0CztzJHI2qWxOD14/DRkaGsK6devs3+vXr8ehQ4fs32NjY7j88svxpS99Cf/+7/+OkZERfOMb35iPpTZyFkqrL8GTt12Cl27cAgBI/TjLO1iaa6eR2cjYYD8evutK7L54DYDm2mmunUYacTLvzmFZlsK71VqLv5cuXYq//du/xUUXXYQsy/A7v/M7eOSRR6Y5SQG0Wq5SSlGYV6tlj3WeQ7da5lUU0LZPbl554V6koz4Fa5tqmVdeWDu6pt2NL80rL6DtqzSvqdybz/S17XlpUs+wV6xN6jSQa2j/VVSvlnmVUxqaXjmgcxhySa7Eq8yBsqWqV4IyV/JVuFfRSlC0EpRFgsK+lHm1Ete32l9YFAk7VnZMXiSYSDI8ccvFOL5+KVbvG0NRKvPSCTuOvKBQIKle5u8WFAoYtC+vUMRCKeTVy7YpZcdQYusCBjEslEJe9SOEsVQwOjX3yOEZuXbaCKF6hPQRwkjIHj+OpZ5RfHy1hy9hen/vIdnzK6TYNDGR+cgOJbxO2BglxjiUj6/DRwB5HzF35CXRF5fQmq8h4Tp2TNU1DJqokWqNTJt9bCm0a69IDylMKpsUJoWNSWNTVnqz91ApR0IZH+zDk3dcjDJLsOrouEEElUaSaKRpWe0t1NUL4jhJtU1fk2S8D3tl/G/lXpToml5EOEmUIaFkqXkliXkpZaulzKWciWuH0tjQ/0j+r7T4P1LCbNqnmNrxmv3/zau/emVg/cR87AWHSPsv2+aNJxv2uwhlXzQm0xWCzW3CoHwZ60/IN3/59lKtxCuDvH54GihC9unaldesd23FxsAfoyJ25HWdqgRZ9Ype//ye56+B3df4vYnf33h0BGQ7dk9Uct/0dO63M5F5Dytv3LgRTz/9tP378OHDWL9+vf17//79ePzxx3HXXXcBMBdxlk1z2eQAxtjKJDXs4Y7hZGvK0/EQMJ8rRi7hdmdSxSRYV/fMY12AjYHXHjKPu6ls4ucqNCXwKrKHZnYotMAYyO0IJ1o76HwqzfCft16E4+uX4t0/eBvrd55ATnkM4ULIBQtfdMtCjoWQee5C+kjsWI+k4n2MVjfXhJQzce3Ym3Akj2E7qQv9BqSQCNOZH8eZwpytDKuzc9t+NWFntq6QfBLmLPRzGlIbJ6H466AHE4Yn2QMFgW3l6YKws9aiLwDJauZjPTayT0Kh8N2pFf34zzsuRpEl+OD9r2DFiVOuFF6iHdHE5kB0VVESVvmEJjfOYDU3fb0S5dqJhJIpGUKu+rkQciL0ZkyKuXjQ+XImrh1y5khsNRMWShSMYXaaytum4LebPpGyd+xeZPrQPM6GPa7axI8Ye7/rHHb2iSY8Xh5jOseIK4bN676vZl3mP14OTwIwTgqvtMJDyTzUHJwrEIzJIauq0BqpY10+RLr/Z9WX35TKc88WwAv9MpKK2M5i53HrcoQlOkCUwVzY65vuXzLU7JfT4/fWmci8I4c33ngjnnjiCQwPD+PUqVPYtm0bbrrpJtu+aNEi/MVf/AX27NkDrTW+/e1v42Mf+9g8rriR+RYN4OmPvcs6hhsXAPkkJs210wPy1GEkTx2e71VYafWneLJyDG/8j9ew4tg7n3wSk+baOftl6qkhTP7noc4dGzktMu/I4YYNG/CFL3wBn/nMZ9BqtXDXXXfhmmuuwd13343Pf/7zuPrqq/HHf/zH+OxnP4tWq4X3vOc9+O3f/u1pzaHz3IR2g1Q2dejeDFBC23925JJ4v3aI4AzyE7qcAvH6xzVEE/OuHMW/prIJT0MDGEKJr+NjOhFOLD+G1US+4MUhbNl+BOvfYulqQOikS1HjSCayskm7FDUcEXRoIo11v5458YTacwD0A9Bvn+uw8pm4dnh4g/4GXOgDkEhelGDCkDofveM2o20RtJHbiekcWiB/tXeX0zBch0KIQBDBxB9DN1SBCDKUwOY+ZLrTkdPQ6FhuQ1WRUlo5LvrZQaw9MIpVx8dZuhr3TkQTiyZWIWbAhZcBuDyHWVhHmRNSYHUOOaRQscoSBjMpE1I2k1e6Krw8x3Imrh0TsnX39E71kWW7RIFKxZAlZqMTmujfq+IpbUL0z6DusGOpnROmfPKKQdikHUCiiGSPo9sl5Bpl/kW6dzCdvT61nb0EryDCEEEvF2HGP4twqQJBFOQT345IUcP6Bcigho4QUiSCqAM71E7XfgEtEENrj55LOt4+G0KK0gGN9p0n4//PH0KPHG2cwx53DltZiiMbl2PD7hGR0NraOcudw8HNa3H34/8VvSTvu+aj2LfnwLSdQ+6spZXHUJfwmtp5G9lOkQQh3VQlEKEaT5cyO3buypaZxwjtIeJjMuZQZsw5pKS8GZvDzq0VGw87JoNCedNGc/zIwcpZkM4hL6+WecfGtmbl8yqnTcOGK115PLjyeMRKhral8iZX9EEvTrDy8DiSxLGWU0pobR3C0jqMWVa1paV1BHmpvKTPOYc2vyErlaf6lD0274lzCvtTp6OE11kK1V8ZIOcwS5Gs3oAln++t6wYAHn7v72Jsj0ONS/tt8xnL8n7it5s+cjz18Z3DsoPt2BzCOeR2Is5hbG5339TRebz6B5U97ex4Y8hO/03nQAOYfPQAtApty/lcOmxyDjWbp24M9StsP21taNZPR+yQcxjtZ9egnW3N7ThdUVklv4LbIV3BGNM6Yq/QpZiT2jdtOQc/ef77mInMO3J4RoTIIG0roIQ63WU/4QiytqgjSNJFrWPbdTrpaADpHHKH0I4F68fblWjnKWi4Q+inpfEdQmov2F7CYAxPeM0cQutQsr2HWivkfUm1x3AJPviPr2BgtFWdQuf9ha6fkbnaX+h0rJ8yDiJvL5W5+RTwviPvEOl2fyGAqJMZa2uX8Foh3ONHm7x5fyKhULtFOMS+Kc9ZFbbB7HjOLxAkvBZjqOfSzPWHEk4h4EgqAOY04bVZt0ELxwf78fQvX4SkKPHhf9lukQVDQqn62rQ1bn8hr4pi9yEmgEqd3r7TsU1fo8L9g2J/IWvjCa+VHKOUcmhij0kCjT7EagJL50l53wnjeMlrx5CV5P2jGzRRafmdqEuNw1FEa6eaTkGiiP66uW33vWTOFN3fme06FNHoKiLL0j4AFdqn+fWrRT8am5MdhiDyhNk0xq/NXELbH23201HOwfP3Hzo7bnzQjz4vze8j7jPlCKLde0+fj4ZwQoHqnijxHrHP0Nxv3Zz8nGcqC8M5LEvDTLZ/a6evRPul9KhfG6eO2w/QO24/5giKdbj3tsig0MPZ851DLdFB08bHsPc2RBNoCHSQTiWodsKrmNTkKuTl7ujdJ59wwklJ5BINtPpSPP1L78KJDUtx5YO70TeaC5QwlrOQo4T0cc01Smh1zDkswSun0Bzasph7TcjpihFS2hEFOuU0bGdHhIMjdritWE7DuvlCUkm4yV20CydS9vNJKIk/Fu4mT+PjTh0YMhhxMplT4HRhTsMU2iGH1cMzRYlTK/rw9CcuQpEluH7bG4a5TESTRB4DxmHkOQ0B4yTaEDInpERyGooTtA4jCyH7hJQ0jeY0VAw5RORHRi8IMZID0gicIyidFdceK6k3E4fRd/piVVo4iaU27MwcRcChiuF6nE3q70rpufPjjiJ30vi81q73GdWFmjN7XI1TzD1iB6GzHuZD5CdbV3LPXgjVg7CukopDBOMOnB8F4WX2KCSdaOk8AmGo2XcKzf1r5tfOwnAOG+lZyfsSPH2bcQyv/sEurHtrZL6X1EgjPSHjg/145hMXoUgT3PC91zF4fGGSTxpppJHpy8JwDquwcrRKiT3ugBLW7Rv0bdXtH+yECNo1eAijjwz6KCHQcS8htXVKS6P9Wsc1aWlilU1sPeUiEaFj89HwdDSmrSgS187623By9Ytn/4WrjGP44C6sfetkVzWR/RAyfUSnI4Rs5ghDyHI9pyfP4ZkUPxzs7zPsipDCEEjeFktl46N80k57NNL9zcPOfE73Hg0Xe/18Yotdg5gHop2Hi+15Ml0dCYWqZfB9hnYfIttnSMhTZhEPRj6p3ndduw5FmuDnv/c6Vhw/ZXkdHC2kPYcihMyOAVTpaxyKaPVsn6ELMTtkMB5CJsQwRAvBQ8iko1yHPSgZSmTQFpUikd9ZGaqldtpLau8r4JVU3HexE5rop0npRGKJ7X+MhZ15uJSjiBYxY9/vKHGFoYh1hBXqH6ayMYNzVqMZQFtEkPYM83Oh7T8+sYWvhRbu13Uu2TyE7kGH4825uHX5FVS0VgyhdJ+Zn7YGDE3kbRKJ5IghBGFlJrIgnEPHVo44h2Jf4Rw5gqSbyf5BOwfrz4/nyBE0B57T12EvIel4rkJqa5er0ETsyVFkpBEt+5UaKLTch7jxleNYdnACS49PotAhkSRW9s4PIZOOO33cmQPiIeRYiDgeNjZOozl2G56djtbUexJz6II+c5jT0I5FaMffZyjHhA6j2Ic4hzkNSdeeCV05e0MTlRMaD0/7+wzjYWVPBx5qdqE4ntMQAK54fB+2/uwQlo5MmdAwEVGmkdMQqBw/6pfA5jR0C3cPuOnkNLRtPKeht+dQOIw9JqnSVdlCI3R7T6HZj1stQsyA/FFhQ6TiO+9+fLYLPxuH0huj0HafIt+j6DueYPPxMe32JpZwTo11guCF0unxZ7+/VeehCftJyc/EdMzgyvCZuSunUTiJntOnlHX0iFyWs8+H968LMRvbPH9jtW7FzoU5iZKtLB18viWFDBUsITvfZ8j3H1Ib81SEU2jXMwvxw/uNNDLvkvcleO62CzCydjEAYOnxyXleUSONzEz6nj6CgaePnrH5xgf78ezHL8TUohRJqbF0ZOqMzd1II3MqzxyBfubIfK9iwcqCQA4tWzmC9FlmcpT0wbCeblFCZqNduww1cx3Cfhwt7MQ4BtApBU0dWtiJaGLHdKhs4qejEeHiCPnEso1Lham+FM/ffiFGNizBxlePY/HhSfC0NDycbN6VJXrwqic8hFx9JNMOIVPpOwAirQO36XTu17yfuiGvfvP54aVeEEL7glyE0yGksH6dchrS2JidmM6habHQr/La4Y0J16G8dlpXQEiBZDV7HA2DGpJtpvMZnso/9pAeo9OhDn4/jVODfXjukxeiTBNMLenDkqmWSFvDSSf0HoaQZU5DerfHc5nTEDCoYCyn4TuEkJKp0t4PRbm+6j1KAGH3Cf87bdpdWzsyixgjwqoORbTzeBVZDMLYPYpIY3ziSsLutRJ1ZIhfhLASCzUjONc4USVjbGb2SVe6MB9iBuWFmE1/gf5BmBEkFZeexoV5aUA3pBGfwVyye4v9H8XQRAXrlwg7HIFsc4/uJA1y2MhZI3lfguc/bhzDKx/cjfUN+aSRHpdTN5+D8V8857TPMz7Yj2c/+S6UaYL33vcmlh+bOO1zNtLI6ZTi5nNQ3nz6r51G4rIwkMOyBFq5/bN2H2FAWGlPGon2iyCQMZRQroP9zVFCz1436Wjo3c9PKPYP1qCF7YgmYs8hS0dDbbFchTxnYUA+Adu7qI1j+OzH34UTG5bgigd3Y82bJ22y6nY1kUuwdDU1+wup/+nYX0ht1C7T2mg3HxwK+U6QbggpM8lpSG3+pn1uu5uchuad70NEMEbaBrPjo5shsiIJKWFKHJvTcJHRpP4YhhbMNqfhqcF+vPCpC1CmCX7u3jex8vg4oCq0sBocy2molNtX6IgpDrWxyGDKklyLk6g+p5nmNLSLYN+TCCFFqd7EMBKlkarS7jlLLUKmLDkyVQjuaSkQQZOmjyb6pBIa41dk0QjRRJ6qyUcRwWzweRIN+2W1/bS3Jw/VHj8G6MXyQEID5UBqdw3afYiQ+2xdYvAQQTR7EBUbSet1aJvpr8X+Q7cWhzamwkK1bksQc/8ln3ASVFqhZ0ckvU0RQwHt2bGa2QxNbIes1uWa7VYWhHOo8xI6LxFlI5N0cgTrwsm8b21be0fQ2eV2+FjeroIxnYgm1NaJeRwjmmjPEYwxj32HkLOPgco5pC8xJ6nYm6JxOLOJApc9uAdr3jwpKpsIxrH9WJyOO4BAGEKuPsY5CyGTU5izNmvbtnpOJkLWYi8IhZRjIV/bp0NOQ5JuchrSWH4cz3Po2Y44nvy41oEL5pYOIL37jqcIG0M6c/TO12iYyBFnlB+zkJ/ThTkNOSElVRp9RYHFJ1u48id7sPz4hOcQVg8cltPQ6tIymtPQhpA5IUWwmSv7xKRgH5BlK8dyGqqYw8hCyZEk2KbuX286h1laokh0gDkouJJqAHPSWEfl3dOm4zB2S2Lhjh6N4QSWOkfR6OIMZ+5wVouwQuvPlAw1+3ZUNT/dezNtfpC7b4Fz2rjOTeWc8MRn7LBe0yGplME1qESImVbgE04SyPC0/7/hDGbh9HsOI0+CXXB73FGcY0LKgnAOGzl7Je9LUCYJ+iYLXPn93T3oPjXSyPzI5NIM6akWBsZzvPe7byBLmqunkUYamRtZGM5hngORVDYi3AvMGUrobHtzAe3JJWI878eQvghK6Obg/SS6101amiCsHK1igraVTajcHYBo/WOOFk5lCV78+AXQicK7/+2tKljGYXIEhBOH9Ll+Dh1kCCP98mL9ZxJCdnbcL2/G+7Fz8DqdrnyeHNOLqWyAeNi4jpASy2lYF062ug45DWO2fcSPk0vsr3slVzjTnIYcOaklkvhr1My+dgiJQ2aqdy99jY9ASiSTwlglJgf78MKnLsTat0dw2Y/3V9lkPGRQMURQudAxT1HjE1KUgquGEkMLM7gSeRwZ9MgnIqchJ5/wnIZVm4oRUmz4OXO6HpMk0UgTDYdkGb3ZolDdL7RydzV7S1ddoYl1ZJbphp1j6B2376OIZLtd+pu4bdjPwV2jLJTrgDqGppnxGXucuiVyVI6LQ+XySDUVP+7QHUlFfvYi1Q3939gaRQ5E7caQSJRP1+r4/7Jd7kMedp4rQsrCcA4bOesk7zOO4ciGJbhs2x4oHUTZG2mk5yXbd2rOWX+nBvvxs09dgCJNsOnl4Tm23kgjZ4ek+0+xTTqNnGlZEM6h23PYBQrI2zsiiKytyxQ0rp2P5zoV9JtOOhpq6zZ5dbv6x3VpaWKVTdolsi45IqhDx3D1W6MokAjyifuFJgknsb2Crt2Nofac/ZLlY2L7CzmpxI2Rv7JzJdcG0H7CcIybQwtkspekLgF2HSGFS6yqStAWQwuV1PnIomJ96hJeU38xxs7t1jCThNfU36IfOrRN8y19ftihhayfq3Yi93mRTVspBdoSGVJonBrsx8ufugBlqvCee3dixfCp6pwl+YTeiXySpmU04bXbf1i9Zw7W4TWUFT0pWMJrftK2fjInpLD6ybaN7zUEDGrI9xdakgsnrPQmcpgm49xypwAAIABJREFUpU0lBLjvk9zqroOk1Fo7FLjd3sS61DgcRaT7Kd8eGt/35s1hJqjsIEixxJNpx4grPP2QIKyYidn5mz2IvB+dQ/bccEX+iyfNhnLXbY5YDWtlKwh1QhBjJBWZWkY+B/y9hGTDr61s7lVu3f6zw6+97HQSlSy1+3/XJcaOVVDpVMCgnSwI5xCFBqYK+2e7UHHQHnP6OjmCns2YIyiO24WNAeEQdmIcG137/IR1DmEnogmdUlRXfYuLkjOKw/YSwKsf2oSRDUtw8ba9WPnW6LQIJ37ZO9MuPibhRMo8hzR27kLI1l5kjLYZ+81F3IuEFFQOVDtCiunlOXCeg0dt/s2Ks4N5/4S1x0gs7piPk+3K0800p6EkpDCddsc8nGx17JjGBnY0Cxdr5zTGchoqpbHj41tQpArvufctLB+ecA4ftAgnA8Yh5LkNgxByoiMh5EjuQ1EezwsnA4J8It55iTwAohReVhNKTiPl82YRGptPSRKNLCvZ/dJd//Rj2w8xA77D6Np8h5H7OdxhjDmKdt5I2LmEdBStjn1//XCxYSZ7To2G+1dNI9RsH73caazeqRhPKbaIOA+W7ssZC6vyGBSvpgKEJff8ibmT6P5dmtkxkgtH0I3xy+xpyJJ5frvIg1jZKRD+GBWOJ/uMucPoh50bQkojPScXPHkIa18/gRW7xuZ7KY00clrlxC+dCwVg9f37Z21LaeDiH+1H/1SB5cNNHsNG3tkyduu50ACWPjD7a6eR6cuCcA6DsDLpfeRwrlBCQIyrq3XM2827ahs2BmTo2LZ3mZ/Qjq1BCzsRTQId3HxEPjH5C712AFN9KQ5etRrnPn8UfaMFVoyOuXCw/cg6E05cO1i7b4eHlZmOHc80hFxCM4IMnadm9mSImdrz6tWr0o6QIsLALEzro4SitjJHE4NwcF0I2q1FRez4qKYgriBEUWKopUH85K92iSa6tXREFgGTyA4upMxzGlo7AoGEaE+gMTHYh5Fzl2DTK8ew6sA4UuXS2yQMLXRpa8w3z0RktT3m4WTAhJL9nIZQDCXkNZRt6JctzlZFCfMcGjQxDDUTYihqKDM0UQWElaR3aytnGmXqkEOSslT29OrQxDDlrg7C0qxuh0ATYyiiq+ss0Shjh6WqAZgujiICFZJndVU/1T7UTDqeSkyijTR39TUi4pPfjyFtuUUW+fXtGYREEGMhZu2NLaGR2n+S68HrX/thZc2mjqW3KVhf8dnbMd75I577EGxe/ozxw84clZyJLAjnsJH5lbwvwfY7zsPJDUswuG8cy4dOzfeSGmmkJ2RisA/bqz2G63eOoG+y6DyokUYaaWSWsjCcw1YRRQ5R+uiheRO6GaCE9p0jfgH5RMEHkqZTxYSP6UQ0IV2ntDSdiCbm45CVTQAYMglvtx+LqhzD83Fyw2JcvG0vFg9NVDWSIfYaAuiKcBK2K5Y6BkE/jhBOd3+hWaOWtrlNNlYrd8z15t1gkr2IGxr+QfzXpyCkRPr4e/yELpLKRqCFIgk2hJ16JE8ijHxFfI28PUhBU6MLED0t+wUpaLz9XjSXYuNNf810MuH1xGAfXvnU+ShThWvu3Yn+ybxq02KvIWBQwTQh5NDpZHLrah5bDUUjWkfZ7kOEfXfHKkhlw9PW2CTXjFwi6iV7qWoEMsjJJ3zMLDbVz6ckSYkk1fLZAfN/4fdxixh2iSaWDInz94nzBNs+ighQNEfbdrLDkSyng9V1k/7GFCTxEDaG+Ll9hDqaOJtfsGV1w1TKOCkl4G6g1h4nnLj7tkQQ5V23nqTio40y8XXitbsWRPvFE1+zfYDavfl7JRO4z0/+PyRSyWsr1xFSZiMLwzksNJDzEneuyc8/aHTVgV+6LhjL3rupXMLmiTmCQOewccwRdHMyh3IGzON2RJNYZRMRSrYfj2Mmt/oSvHLHeTi5YTEu2rYPK98aFTkL/XAwd9rqCCfdlr3Tnq4QtrsPIZOO5y4MHU/NzkGL8eZzMnZKP1bUA0Js4U4VUPychoKQ0i7fodC5OWOOpc9i5sJzGvKqKDz0q1hfau8mp2Ei1svtmb8YwVeOqfTK/i1zGgK+k+mOpwYz6xi++96dWHZ00vla0DYkRg5hKiqgVP0S5kQqLUrkUT/lLYiTTxgryLarjDt9zEmMVD5xbGXXJsLJgHAYFSesxHIf9pikqUaRaktHlz/eXT/6v/n3bNNGY7RHYiG9ebdhTHaf9x1Fa6f6Z6bsvhqEjT3iiluu826cA0gTuptop1Az+Bh7Mm4NefWV07pyRFk37kzycDE9rul5kXjrJfHL9fFKKrKtegYpBESSko23/cAcOMF6pnb3mXH30rZ3yH3oV0ABJDuaM5dNs0YdgbAbWRjOYSPzIuOrBzC+ZgBbt+3DqrdOzvdyGmnkjMui3WMz+v1+YtNSlKnCVfe+jWVHJ+d8XY00crbLwO4xf8t/I2dQFoRzqHOgnOKQHzucY5TQ6dyvKI4S8vZ2KCHvN510NHbcDNLSxIgmscomtvIJXBsnnxTVZuUlhyZwzX9/A8lkGYSQcxUjl3QmnMTCzjFySTw8rYP2bkPIPHchDyHTu2tntZW1s2dyjvUecuhLlJDC3B+e0zCaoiaSyiaGAroxIcoYR/LkGOoXD2279hhKGKCJvF2HY3k7R2AUgOUvHrc3WSXGM+SAp69RxsY5O45j7VsjWDRJmzB4CNnUVDbH7p2Hk+mdwsVpKkPM9E7HiSWfaBFOpndlyScIyCfIGErIkcEskpaGh4upnyCfZHKMYu09JiopkWWFu8dy5IehidqLBiWsHjO/t3cKO5u2ehSR1uBQxMoGJNIF0PXi7nN+WHWuQs28RjPPSZtpYPDF4/b+nHvooV0Ke3bQeuirHKvHLMPMDAWtPr/Mph+T5xd+PhEEkn1OdhbF6iMzmzZ/pY6noPFzH0qU0P2PYmltwP6Hswkt9yZe38hZK0Vfglc+eT6GLl8JAMgmm99+jTTSjUwM9uFnv/YunNy4GADQ11w7jTTSyDzJwkAOCw3NSX41NY1DdA8zQgnFeMRRQtHeZk+h394pHY2v4zY6paXxiSZ+ZRPAkEOCJNdV36Ivwat3bMHohsVY9+KwTG7t7RU0BBH5K6uOcFK3b5De6VQ5McXvx/cCzmZ/oVa6pnayQwkL+6vPtWto+wuxl4SQvU4VUKKElBnsNaR3sW+Qpbihdov+WaRS7jUMbcu9hv4a+L6oIOUNn4+tIYYi+oSVoTs2QQE45769NYQUo5sc7MPrnzoPOlXIWoVFengWGZssW5VBwmtDPqn6sfQ1ro6yFnsNjQ5QrM6yfed7DQGoVLUnn8Qqn2SMSEL9VRxNVDHyCUcbVW9iGGmfhs4cIYUiDoJIwlKUaOXuDz6ayEksdXsSqa0ORSRdLJl2SFzBtAgrRqftPb/TPkR702b3fEoATyTDw3dsAgCs/d4+ZIoF9/ithul8JC/RCEgqJTv2TQBub32iZOJriirRPcQs1ejIteBVU1RE55NTjL1wP6NMQePmpfOP1mWGss+XIFn2DGXhOIe5FmFiIOLodcpBaJWhLsYydm3xvIN22g5hY2unC8ax7dfGEXSnIR3CWGUTl4vQtfFwMn0crb4Er9+xBWMbFuOCbfsx+NaoCAdzB5DGxquUSCeSj82588jG+g6lcf5kyIP7+RpzE0J2TqsOQshBe486hyTdElJmwlCOl80L57ZjI84mHyMcOGbbb485dZwJzfv7bGSfwdyuGor9W8twMmAeolODmXUMr7x3F5YenWT9tH2YWzaykqFjABX5xNOlpd1oLyukhA4jrI45irzmmmArS/KJYqFh6TB6zOOIwxgwlN9BeQ6TVBu2Mv2zq9MwGSiqY8WqoVB7GTqMnMSiIk4kZzzXOYoAavMlxogrdY4iQPc4+suFQDPv3sdDzRQ25tVVOAHMEkm0Rsau19QuQkwXOons2QJQTsNKV7VlUM7Rs4MVMyqnIHvyvOWPUTpXQWJh/f2QNJ+7U+5DGV72nD/IqimKOdzV4FmFhnvzJ1kjZ5WUCYRjuLIhnzTSSFcytTTDaxUr+bJ7d2NpQz5ppJFGzgJZEMhh2QJ0zhDBCOLHj2eLElKfWJqZGErI+7QLG/O55iI/oR821t4vIZ9o4nSQulJhxc5RrHvhGAZ3jopayACFi0Odq3vs5mhHOImlrdGIh5BzNt72s3a6DyFz9M+t0UMdNbenI6hl7+Y5bBdSbhs2jlYfiVU2kXNRP474hWFnGU72x8TyE4q18XYvzYZEBNl8/hoZIsLRERFq1iyHIdnxEJhsPMfKXaNY/9IxLDs6IcLJ5vwZ+YSQBaVFONm8u3BywkLFvE6yXw1FpSH5BIx8ImooR8LKUuehhCzPoeIooNdP8XrKtXkOexPDUCmQsLAyiS5d5EMnAN0Z7LODoYmcxMJRRG4LcN8DHl3yUUQA0XyJnYkr2v4PZAi0So0UCTVzmM8ieSxs7NA/3m5UuXLXDscnA8SQo4mRdj8Popk7ROqi5BKvBrM7ov8L/yzcfNST30M0G8pJQNTPVUhxn22Yq1DDT2XjV03hYWmyMxtZEM6hzpV5ccYxtXHvwdPF+nXjCFLbdPcPArEQcajjxzPJT1jnENbtJeQfD9flfQmmBvswMDyF1S8cA+DvC3RjQwdOstPq+vE9hSW4Q8na24SQXV5F7Z0DzVndPNBdCLmEtnsKeeggtrfROZTGSi+GlZW35zDGfqM+XGJl8WKOZlwn27lN/zjmwPE+ck8i6XkfOU8ibFd9dDjWtxcLRSveV5sHD+layzMkrRJ9p3Jc+MhBZkc6gilcaFjuL6x0sYTXrEyeTIJdHfN9hsx5tO/8JKp3xfcZcqfRTBiEmoUjKJJcuzHmBL3E2NE9h7N7yM2XqFTbF+CeFypB/FnEHEZ/n6LSyjFcmQPjO4y6rHcUjS7Ml1i3N5GkhEJaLahdqDkHv0ZZiNTeY+mHlXbbEJgh7iTS/Vt5/ZlpGQ2ucRJ9p086lM5JlA5eWGYvykxW/mel2R3eOWrcUQxL7nHbbj2cuWzmCHMk+vZ8h5KX1JuJLAjnsJG5l6IvwVt3bMbEyn5c9u2dUK2GWdlII74sffMk/B1zk4N9ePuTW9A/0sKl3901L+tqpJGzXZa/edKnCTRyBmVBOIe6AIqp8JdHLFTM9TNBCbm9GPmkXTk73t6pionr331+wqCf3w5/DKK6Vn+CnR/fjPENi7Hlwf1Aq2Th3niuQqfjcxuJ5ieM6ZjNTjkLOdJn+ku0cC5CyM62rIrit2tP38tSFzaOVUbx2cFA53AyvXOGMg8nk13/93C02gkfE2uPVEORqKVr4+Fkq2P/0ICwUm0GX7X9BKW0Q6KBVuUYlqnClh8fQgqJFtL4lIWLeTjZnL8MJwMGQUwC5rFkKCe2LJ7r51dDUZmS4WSaMMpMrt4zhv4xwolK6/Mcqjq00M+DKBbZW6IyjaTfwWCaZ8kgZCwFex65dxl2Jp28g3A0kRNY6lBEANF8iWWpgLIKG7O7FP2oqSOs+KHmjI3nVyhH48zf7pxjbOZcmUwAq14+IfL4BQgyRwEFIuiG+CghoN3nzdr8uzPPmxivoMLPldbvnqGx3IcFCw3z3IcWCYxUVZFooOxXaGnPz3MYi+hMR3rzqmtk3qTok47hirdG53tJjTRy1kqZKZTVHr7JwT68VTmGF9+7B4sb8kkjjdQKv3YaOfPSNXJYFAV++MMfIkkSfPjDH0ZWZbL//ve/j9tuu+20LXAuRBfKvIKNwRKxE0QUnB6U0LXFax3zdiDcU+ijf6aPRDpj+QnjyKBfE5mOI7rqV8nB61ZjfMNibH5wP5btHLX5CjlaGJJKVIjUIdxfWCh07BfbXxjWVtaRczk9+wupv8hzyPYaAkChS5xIRlFMKuR53lPXjl9bWbRF9iFGcwja/TNxnV/tpM4m/ztW69jvL1LnsD6chOKjiaIes46MjdiLbdNT1fh9t5k8h+ffuwf7PrgBZaqw9d49WHp0wvWzY7VFDG2utij5JKyGwlPZ1KavITvsBBVDDCMLqvrLPId+NRRJWOEooewnqqHEaicrBaSyQkoB4JGfvYVFAz/ouedO0gcgc4gh3ThU6VAgkU6N/X/8PYlaORSRhKOJnMBShyIC5jtRFPLbbL5LpbSjESWskNTtQ3RwnBvg9iTya41sw34n6GPItEauFPbdbvIcbrl3b7W/UNNJsA/BnYojaiDaLlfl5kt4B9aD9/WjPhwztFEuHe5d9BHGWJUT9xxRVhfklVQM6WX3WD4fHfsk0plK18jh7//+7+Pll1/Gjh078OlPfxq7du0CAPzTP/3TLJcA3Hfffbj99ttxyy234Nvf/nbQvmPHDtx555249dZb8eUvfxl5nkes1EvRUihzhbIIX0VLBe1FK0HRSlAWCYrqlbfMqyji/azN0rzy3I0tikToTZtibeZVlAp5kSAvzHFRKqsvSlO2rtCJeVW6VplUeoUCiXnR31qhgHnlYO1CRy9z8QqdUsirVwFzo17z7FFsuXcvlu4cY/0MwyxXpk+h5Ni8Gmva4PrDvEhX1vQroO2Lxtj5lAstu37MdmQshZDLoN28rE47ljGVv+Njgv7Vy/bV5rW77wDGk1MYyUd77tohIYdMKRUNKZPe6qo+fkiZO1Wk8+cghjI5bwrVjVWpIKRs1wU+xr1i7W4sgjEQY90aEg1LTKGQsh0vjrUNKdt1VmO3PHQAF9+zG0uOTrK5NSiXYcqOlXIhZSKbkA9GCa+JoZxUZBOKwCr+N/XLtNRTtLZ6OZ1yziC9iITiv7LUhZTtRMqFlMnxYy+VZVBZ5nTEULYvOkmzoP/jX5/Ajn1HevK5ozIFlcG+Enr1a6jMvaw+0+b/xIgspEsybfMmilfQViK1L/fdsf97BdZuXkmikab0Il2JNNFIE2ZDmR8v5uXaSafA2qvvc8a+01n1SqvtE/T9T7V58Wsx09qGsOmHVgrz4teY0xmkK4O5Nun6VGy8+X3k7gOZNi95rbu/FbtvKK3sNhR7T/DHsHsev7c4nXd/jOR6pTZ7LH4It+nHf3ize/EZCSsPDQ3hC1/4An73d38X/+W//Bd85StfwVNPPTXjiUkOHTqEr3/96/jHf/xHfPe738U///M/44033hB9vvSlL+GrX/0qHnjgAWit8Z3vfGfW8zbSvRR9Cvs+tAH5ohSqBJYcPDXfS+opaakcG/J1uHDJ+c21s8CkTBWmVvRBJ0DfqQKLh6fme0k9JYdPnsLn7rixee400sgZlq7Dyq1WC1NTU+jv78eWLVvwzW9+E1/84hfx+uuvz2oBjz/+OG644QasXGlq8d566624//778bnPfQ4AsG/fPkxMTODaa68FANx55534q7/6K3z605/ueo6Hrr0co+dttNDwBXuP4LKdB5CnCX7w/qtsP2p/184hXPT2ECb6M/z4xstkI4Ctrx/CebuPYmzxAJ68cato1wAu2X4Q5+w5jpODi/DsL1xQ7YF1Hvwlzx/A+v0ncWzVYrz8C1sCyP6S/zyAlYfGcWzDErz28+e6hmov7SWP7ceyo5MY3rQUO39uPfyduFsf2YdFx1sYPn8Z9l271l8+LvzhfgyM5jiydTkOX7k6aN+ybT+yiQLDlw7i2OUrMLl6AGV/gvENi5BOljjn+/uQ5BrHrliBsYuWB+PXfm8fAODkNSsxcd5SCckXGivu3w+tgLFrVyPftFi2T5RY/NABFNBovXctivWL+OkDYzmSRw8ayPz6ddCrB2y7AoCRKZSPHUIBjf73b4Qa7K/GViHe4UmM/+chaGgsvelcJEv6xPpbh8dx4ulDAICVN29BMiC5pqf2j2L4BdO+8ZYLkaSJWP/I7uM48tIQAOCC2y8GAOx+5hDOu+4irF23tueunXYh5dmGk2mcb72b9DXK6hC2e+FkevdzGvKwDE9f06kaij+vArdZtWsgH8wwtXYAGgqtZX3oPzGFoEIKwELI/Njp2lVD4eHlxOvH09dAyXCyaffCyW5BVbty77FqKO3yHPIQMg8vt0tVk2aBnamiRCsvsBjouWvngQuvxOjKc+z9/eJjQ7j6yD60oHDf1mtdx6r90kMHcNnQAUwk/XjgiqtEGwBcsWc/Ljo0hJP9i/Cjqy83zaz96rf2YvOBYZxYsgiPX3txNVZZM1fv2IONB09geMVSPHPdheKZBShc8/xurB4aw5F1y/Cza7ewdmPj6if3YPnRUzh87nK8du05bPmm/YpH9mLJ8UkcPn8Qb797nbANAJf+cC8GRls4snUlDly5yj99XPiAee4cu3QFjl62AhNrzb19zyc2Qytg03+Y586JK1fgpPfc0XDPnYlrVmKSnjv0vc41lj6wH5kGTl23BsW5i8X8erJA+tABcx/4ubXQ6xehz320yMZamHr0AEqt0H/9eixePSDu+8XIFEYe3w8FYMX7z0VqnzvmNTV8CkeeMu0bbzoP2VL5XBofGsP+p/cCAM6/eSvSRak4v+P7TmD3c3vM/+GXrkCSJmL84d1HsfMF0379J9x3a9Ua9zlPVzo6h+Pj41iyZAn+8A//ECdOnMC6deafvmzZMnzjG9/A9773vRlPDhhEkmwCwPr16/Hiiy/Wtq9btw6HDh2a1ZyNdCdlqqxj2H9sCulkk1hgOpIXObI0w5Vbr0Sr1bL65tp558vUYB/2/vJmaCgsOjKBgZFW50GNWDk1lWNxf4Yv3v4+jJyaxGClb66dRho5M9LRObzlllvw2c9+Fr/2a79mNwOTpGmKT37yk7NaQFmWYq8Sr+nYTXs38qGfvop8/yFRpaRAAlUAH334pcoum6NUKJGgb7zEzT/YLtqJPFIgwcDJHB964BWZMJTadYLFwy28/3++Xs0JYSfXCZYfmcQN970RrWJSaIXBA6fw3nvejKajKaCwYu84rtn7NkuO6ewUUFixawzLd41XOkk0yaGw4o1RLH9zrNJxAgkw1a9w4pJBlH0JznnwAJbuHBX9CgDLdoxg2Y6RIN0MEUQW/ew4Fv3sOIJE1sq89z8/jP7nQ8IJpZ1Jnz4Cxey5vX/VWp8a8ubmBBFg4rGDgnjip6g58eg+d86RFDVHHtoNR2LRYg0AsHfbW7Z/rLbytke2YV1rDVYUKwAAk5vH7f+oV64dwO0ppGOuB4DZpK/hfevS1/jVUMTc4GMg+sXS1/BjsS+QjeXIIgCRBJsn/vWrodDx1GAf9n9iM3SqsO7pIxgYy+1cnHxi5jB7sOjYVjEhXYdqKDItjTu2/Vj6mmg1lDYoYW36msxLUcOrocSIJpS+JsvkmKCfgzd/+b9+F3d/+GrcecPl6F+xHFx65dq5bd92lIcOOhul2TjXrzXufOO5aiJWQaUEkAFLyil86qVnK1117yfiSgosb03gE88+ZyJJXsEGnSqsmhjHx598wRRaID0RTVKFNSfHcMujL4k2+2xMFNYfPYmPPLgjWoihVAk2HDiJdftPBsUXCq1QKoX1u0ewbtdIdUpm7zvgEMYNbxzH2jeo3REsqGDB2ldPYM2rJ3D00hUoFbDq1RFouKIJq14+gVUvn0CuJOmRdoQOvngcePE4ciXJjjTfwPPDwPOsDYxkqAA8c0SQDOnZYYk1Tw2hUFpU1yJJlMLJxw9YImNZ7UMHAKqWMvToHkFalGltgJ0PvemeO9U/ScPdB1++f7tbl213a/jpfc9X/48SGzZvAP4UM5KOzuHf/d3f4Wtf+xr+/u//Hr/3e7+Hj3/84zObqUY2btyIp59+2v59+PBhrF+/XrQfPnzY/n3kyBHR3o0QoYQkVtZO5DT0HEGuizmCnVjGvh6QjqDrI5nEvL8dQ3Nzh9HTGT3pXBu/kKiNO4Q8l2Hel6IYSLH+BweweOeoKHXXiVHcKVehm490Oug3V2Xv4rkNnR1+EXdiIftj7c1DuxsAn2fL5CYM9R3FkewYVrdWYbVei7mUM3Ht+EHfOoZy0A/tq6FMh6E83WooThd3CmdbDQVMTzp7rAE9kEIVGhf8z31YNDxpy4bxCincSbR+FzS6rYYSsJW9aij2newwb7braig1DGURTjYLkw5e1S9gKxP5hMaQjjOUK/03f/tj+G/bnsX/+/gO/K+f/DB+5WOYUzkj104KqH6F4GZUgv3K4O3ativLVuY6JXQiX2ICq4sxnHnFFb80X1HA/v/td54xnWWmDcdqpmdhUdlO4X7UFJUlU92k6md/TCn2YSRwd1eax1QDWfvqCRTsWUXXETmJmTakRJonYX35uYDNALBnuArb+Fh5TPcJuudPP/ch31wm253zaHpp+7zl+Q7D0qMu/6Sf8xDAjIAALh0JKZdccgn+5m/+Bn/6p3+Kf/iHf8CnPvUp/OQnP5nVpFxuvPFGPPHEExgeHsapU6ewbds23HTTTbZ906ZNGBgYwDPPPAMAuOeee0R7I3MrZaagFZCNF9jyr7uwbGeTx3CmMqAHsGXqXGyYWo/j2Qm8cOKF5tp5B0vZb26ni4cmcNE/78Si4Unki1Lki/waKY10kq0bV+H/+swv4v+864P4x4efaZ47C1Caa2d+RWnt0yHay7Zt2/Dnf/7n2Lx5M774xS/immuumfUi7rvvPnzrW99Cq9XCXXfdhbvvvht33303Pv/5z+Pqq6/GK6+8gq985SsYHR3FlVdeiT/7sz9Df39/1/Z3fvQzaO0dsn/X1TQOq49AtNMYv1+3lUtMXwbVR8LBsSomdixUoI/lJyS9sVP9zRC/WrSwX2H/bZuRjUxh/Y8OVYigP58cTzqOFnaT0zCG+JU8TGDf29dE5rqChZBjOQvbhZBjYygtDZ0D9echZGc7YoehkhpAuj7F+Mrxnrp2bnvvr+DA3kNBCDlFIkLN9Gs2qyAMBYW0OuaIndORHYf4peyXMR0nih2zMU4Hp1NhPwqNKO2OU9vO+xrJtGuvIrImXQYLIbt+dKxRVKHk1c8NY9XLJ5Cjd8isAAAgAElEQVRpjRTAm7+8BQrAxffstmk8jE3znkEjq2AiShECAGmF8qWJRpJUumoRaVoizYwuYTo6tu+ZRtJX9csARSFmAur6FVRafeZETMkUVH9a6aqzzRJ33JdBUVg5pfbUhZP7q238aWrDz7atr8/lN6QtSmlq9IBBEAlFrNpVmkENrsajU5t77rkz8r//JvTQQYb+ORRQx47prXR9ecg5qMfMw8b2XTE7SuoBkc+3LJRts88lFmq27ZFqXkWhgrByqYGSKq2w51hB87FoF4WaTcg2scfUrwDw6ifPA6Bw8T27UbDnV6waFw8x5+yZ5efALRSPOvGxFSrpPloXLWNIHt+axJ831FZ6/QrtniHRdvaMiYWiKWzMdZq1WXu6RFH9Rc+dAiU2btqAe346M5Z9R+fwyJEjePnll7F9+3Zs374dL7/8Mg4fPoxLLrkEw8PDeM97/v/23j3Ijuq8F/2t1b1nJI0GEKARQkICETtATGLnwjEnD3S4iZEHxCPEqdy8SGxi+7pcFOW4SPEol5PcclGxTYoQyqk4SdkVivgk8S0s2wcpuYdr1ck5+B4eNse8MQaBHuiFRug9M7t73T+611rfWuvr3j17HntvzfpNTe2e9eqve/bq/e3v+37f9/P43Oc+Z1hf/YioHHZWDrOWwK7r1uLUqiUY+69vY/kbx6JyOEvlcEpNY1JO4qScxKScRHuoDZWqgdo7UTnsrByq0RR7b1iLPJG44Ds7seTQVFQOZ6kcvnNyGi/teQcvvz2Bl/cdwcv7jw/c505UDqNyOMjKYceYw6uvvhoXX3wx3ve+9+Gqq67Cxz/+cVxyySUYGhrC9PQ0HnroIdx+++14+OGHuxJgIaATUHNVSuwx2P6ZKoKmDQJ+5RLA3SDsHG4cuPn677ANCJU6V0ETTlveEtg9vhaTpWK4dMdxsxl5hdJFDtTGF9LNRxVCGldo5/qKV71C2SS+EIBJaK3beCWTU/rcdapqK1NroW5/fekODKkhDOVDGM6X4MLlq/Gf/8c3BmrvaHSqhhLE+9WkrynaEPT76+rjbqqhhON4YoyX3cWJU+TPZ/skilrJ+zavgSoVw2WHJm2/2fjKxBtyhJSZVkNx4hCrqqEARSJlcoEBIcUVqOwLYwqD9DVcNRStMM6kGorfRo433fdPuGjsLFy2diU+eOmF+D/vGbzPHZFIqFRCeMqhypWJKXQecPpVKHuPTZ8yx048omkTpo1WXKGxiACQt4V5s+t/fVUcokzsXP1+09VVkgSmYodukxCA1CfSSqIy3yG0rDmARG8uJchFGomgd6vS+0PBvmf0thIKbryfuwpAPnv17VSA/h7UJuP9mEIosLWX6Tj9+e7GHup9bU9rq6KgQ7/9BLf/G3vNureq3rJR0nWb91ydKToqh08//TSWLVvG9rVaLXzmM5/BFVdcMSsh5htZWa0kKAHkWQvnUxGka5o53jg6lrMM+u3FeF4RNOOZb1kuOxjY+6vnY3LVEpzz+F4M7zjufLPy9OXAShi0eesXbc2shBkzzp/btOwdtRIauebBSmhk9FzIALD+5IWQRdgwcqWwLBkxbqlB2TtcNRTdTsHlOfQDmgVR9KhS5xNFBKyyR88imbW5fofBTHIbhkpfmC9RV0Px5eeUSCTAvuutYrjk0JTTL8hvcX5ldqFZj+hQuhqKbge0IliONUoiJZVohZCsQ/ochZCeFLwiGDCTTRtR4Lz8hYIjmkgJwZFPgjbpMmTK9v/+f/0+lg4VlkYxei6WEDfyoOwdY3ElSiGAQlmsUhT1K1UUAXRLXPFLxsqUWAe13iGBnGpKKKzitAxfbpTCUsVRgiiKpcUqF0Yp1EqiUMISVoiSqMVKBBHEUe+sElVMU0TRI5+15f1RSliCCSmzZ1akt5Ec23XKY3KvqDS+4lmUq9PGAV9qLXlxadoiyPbDKnuSXJerABYjrUJp27Q8GWmnxJTZkFI6KodViiHFP/zDP3QtQETvceazhzDyyhEsjeSTOYXszPeKe2eAIdsKK555B0sPnMKSWPlkTrF0qNVxTNw7ERHzh8YVUupw2WWXzcUy84YsF06MhIZvBfRjKADX0mfburcS0vFVcYXFHJg+NkVNRTqaynXIF0+FwpV8/ILlWPrGMbT2nkICEmPhrOO+NokpnCsXctAmwpgNdp0GMYXdupCpJZO6kmneKp+QoudkIG+YEv2+dwAE1VBMO+fSZeqFVrmT6RpA6E725zjWPW9tQfp5N7AIVHUJ1FZDMedXrtWxfUYL2bIErbdP4oxXjiBV1E1crqMKd9jKFw6ZNbX1sBhnX5tWQ6GpauqqoTj5Dp08iOU16GBJYk0UjOVQcGlp2GoooTWRtRI642y8omNtpEkagSIGUfBfsvp974hEQBHLIXUvV1kRizaED9kcrhVRtxGLoZ5LrYgm14t+9hCrM3U163yYNOUNSstgLgWkdopq12XmWQwNrMUQKNLcaIuhY0GsdDHrVonVL0wU/dCmPfOBWl4LcT8JZeeb9Zqlt8lhnwP6c1VCmbZM2OcRzSdoP2/0s8i6uanlk3oqcm827dco2tx1pBDm88TIIoSXI9Ht9zkNM8WcKIf9DpVL5LkMElH7x3OlCNpxdrzy/lGVsYLBOewblva7Sh+zjgjH63XaLYED42swNbYEY/+0A+nRdqNYQtPGnGMmuQrDNj2uc0xhNy5kvTaXyJoqf924kMP1lKMUUrnpg2VQUJXn0FfkwrjBcA0aP8jFJJq5grp7Xbe2Xtt/7LG5FpWoUBhDhZImvPavgSqR2RktHNy8BkIBy//zGxB5uY6nZOq1z33tqJPk2vPsOq7mjgmvSR+X8NqJJYSrEBYX4Sp9gnEhi1QybmVpSSg04TVVCP2E11QR5OILqStZyLDfvCZWjkFDwruVkatqRTFoK9fiFMYOsYlow7CqZupqLtymxDWs37mlXDIpFETA/qsocqII+m5lqjBSFzPd80COVa8ddsgqXugipFJWmVP288YqemFuROpC1rc7FW78ob1++/z2z13IaxXJYj2b+1CRUfqeZsruf0cxDQxCKlBGMzKOupwdF7OX55DK2A06+70iTivkRDE8+/G9SI+2O0+KiIhA+4wWDmxeA5VKrNq2235w12ByeYrJ5YviO3hExJxicnkr7p0eYlHc+SwXaGdWD/arjhRt1Jpo+7uxEnJ9YdqacI5uB8C6jYt2/zyhlbBqnXZL4J1SMVzx+F4M7ThmvA7OXOq1CNZzrWj+uCbpaOx1+VZAdCScdONCDmWYnQuZd0nTdXhr4wxTivYF9LdT33rnM5SpO9lvs3Pc+YD2bAqn33cpB9a9GpczPZaePP5YrhoKPaal8rIzWnhHK4bf3YVhh3xCKp941VDe+JXzIQC8b8ubrpXQWAvJPamohhJWPgHTFrqVfYay8C9Weu7k8tVxJ+s+thqKZi8Q614VC1n3+ZVUyDhBWTUcYWXQkCTFb/ktwvxfiGXQtyICKEkqXFu5LmdNpC7ntjaJgSWs1LmaNTFFAqa6St4mRJQKNnMV8gq3Mudi1psoU8Uef+VX1kIBuHzLDqQA2uWAtJS7DeFYBq2lzz5bQiIJHHIKUNxih5xSyqIPHatknXvZOY+9x/qo6vlnLYFaLt9dDHNWfx2NYh1v7fmukBJx+uDUBSNGMYzkk4iI5jh+2ZlGMRyK5JOIiIjTHIvGcpg5sYWhlZBNJ8NYGIuxNXPYPtcyp/t96x6d71rqOMuiHV+3Dl1v+I1jOPdf3oQ8Mm2KklOODjenm7Q03RBN6seR2D4mLrDTHD3OmTNH8YVAaC3kCSnKJCkdRFTFGgKetVCPZ8YBnDWR7/NT3rhr27FOrkXdT9LX0LlcvsSwJnRoTZQKOOt/HsToC4ex5Mg0mWv7vXA/h3xStCsa9gcut6FLSCnbpHJiDfUcn7dB6yRXpq8x/aFFkGuz8YPCsSI6sYZ6HI011IKxhBTfmuhZGP2YRNo2YBCpBFJpkk4b5ApCB+UpBdXOTTuACpIK14bgoa3ayiY0z+2aneIQ89KaSIkptM2QV8plaKobjuxh7oEoazeDjzn04w+LSboOc1FPOIVCm6zv5BokFjiz3WrS20ia3obI6Xu2CmuhPY8eQL1qnDw0rY1e16wpBNE3lBnnfyrQesu2zVptKTGF1lYOSCyzijhcJMphrmyWdoC6ZKnCaMfXs4g7KYlun+73HYpcxZKi3T8fWPYwVQh9ZY32t4cEDv/vqzHyw0NI9p8CjkwHWeP1Ma94+u5ZNGIe+3O6cSEDnlLXpQtZyzUfLmRubkhIKdryAXUru27c0GUhGTYzbbMKE2EwO232WI/nXTBUDj2HnNN7dRRG2q/ccX6b9tDlZ7Rw5JfHcPb39qF1vI3kaNueV1kFT8B1J9PzCXNcuJR9dzLNbViZ8Nonn7DMZG6cf2w007KNKoJESaxR/njSCJ1DFUGj1ZK5ngbrM5QpEcVgMJVDSFlUlDEagX7NrcKYk08Uqvx1Yjj7bTp0PAVRspS933WuZoatDAAyhWnzFSrKZtalgqjCZC4pt/9SjqRCWbjmC6hSUMImwaZELgpVjgA00UTYYxSfU5ScQuWn4yDsv0bfroxeKxy9tVxH2PyEdByTGFv3Z0R7sG32uabIOOldSwYVPAdpm58QG3CJKd1gQHddRBPkLYGJD6/B1NplyJYtiu8BERFzguyMFg5dvwbTZw8jH4qPyYiIiMWFRaExZCBUfNhvCZwVsDienZWQ9gGudY/O9a2EzpqO9c5dy5XH7ddz2y2Bdz+8BtNjSzD6/+5F681jrMWQnq8T0cTI7Z/PsdTVrzMTFzIQWvm6cSFz68yVC9mfSwOVc2/O4NkNLbhSTDNxJwOutbGOuOK7kn3VTIqK9DYV7mS9Zp21kfblZ7Tw7vVrgFTi3O/uwvDEFLEM2vF+NRT3fIXFY82zB801SOG6kwHXlQy47mT9N1sqzzPAQVDCCRHMWOqocMRFXFMNhXU/05yHxCLouJPLNrYaCudq5tzOTtWU2TjHeoihtMjjqD9whDGXEesfcSBS4kqH9De6T5QuaZWSJciHgyothqyr2VgblZ1DySrmtrtpXWxb+RexINa5mDXynOx5mcNUVSlnZ5BIVI4L/9eBwvImtAVNP2Op+9R+Huo9St3BjtsZKBbyPkNpP/0co7kPaVk9AE6+X3tH6qumuOejMlhvA31126xl0O/zryGjbfNZISVi8JCnrmI4/MYxqAF9vkZELCSy0RRHri9Yyed+dxdasySfnP3mMcOujIiIaI6Vbx5xjDoRC4tFoRxmyiOkmFdqsZs7K6E/jloJ6VzfSkjn2T7hndPvD7/BZJmCODaN5c8fRrqjsBhWWRhnQjSxMviWuvp1/ETW4ThiqashnOh5joyMlZBa/LjE2HMZX6j7/PhCV9YijU1el/OhTyFBCAiw30S5OETaX135xAX/LZnGHwq23z83a2EkY9lgeYR1lMVkjvTdaYz+fwcMK9knnxRtdjfpaihUNk1IOXHWEBIoLDs8VcYX6nNbK6BbR9m2AzqWEOa4eEVFzCFMvx2nj62VkCefEGugR0gJyCWNaiYzbVLAFzJIX6NjDTlr4oBByCKVjfID33LlWhHL8Yrpr0p/AwCinRcVWFATh6g/4Ukcov6sE6myfdSKCEDkAnnbvsdh9ln5bBP288+xIJL4w6KvCprhIqHrMGsLoiotfsfPHEYGgZHDk04NZkvmsJ+NBWnFtxICDjmlvAxKTin6zOU5sYf0GvzPQfqMqquaAqGcZxRNiF21tqS6AX3O6HMQYkp1tRT32dkNFoVymEOgDb5KCR0TKE9eP9BMESz6OqztKGbVa1YpmVx/PlSWazqZYen2fQDqS+L5ihxHNLFzqJJVv04d0YTK0MuydzN1IVO5q9jI/hyzXql4+gy0QQFbfQSi0k1s55THNQojdRFXKZFcVZWAfKJCWaS/pgrnmg/mkRTyZIZ0Mse5/2U3mqDu0auD6F/buAYCCj//7Tcg4ZbKA1yFULuSi3bbz5fKc4XwS+WZk7jaqm0HHEKKIAph6GqWLkPZI6w41VBYQkqNYkmPfQaNczEDiDQB0sQqbloZoYnzCBPScTX7/SI3iqJhN6eyo6s5yI3YBszTk5JVqFsaQN5WkKUrmiqJNB+idQYTJbEBSYVCOQqOolPx4sa1AIArv/0TZMq+lbXS4jOYjeKqxaXsYP32JLeeOrP93If0Sa0UHFe1HuBXDwPosyf8vJTkuOoLLuCWWaUKLK2MUowXldVSzPxZKIgDvPMiKPKWwJFNa3BkfE10IUdEzADZGS0cuWEtjv/SWK9FiYiIiOgLLArLYQaJDMJxEwOexQ6o75+BlTCcG7p+VcWanNvYlbHsJ99aVEvg6KY1yMaWYNn39hblNZlvNZ3S0nQimpjzMev41kL3WmbuQq7Mc8hYDv0UMawVkB7PoQu5qg2w7gu99iBSUkx6iRp3Mv2GWUtIIf3uN2d449yUN747mUuJQ499kok+9kkzUgHqjBaOX1eQT5Y+NxHIW4xTwZp+bkNKROGgXcrUnQxoQ5xus9ZD3oUMMy50K1OLIUhbaCVEnVuZ1kn2U9oADax/ghlHLIh+nkNnXMK4oqV7nkGClGX1GNd8JXJlrYjIYGxlFYQVoHQ5tws/kEi1+a6Bq1lbFEtrokjth5FDVvHaZGothjK1LmZt0ZSpcqqpANp6V01SMUVcpHXzJkmOrBxLcx8KVewVhaJesVJg8vy55BQ/BZ1Te9k8n21qmbrch7mCtcqTdvp5SJ9HWoJwHVtv2Ulboz93lf3MoJZM/5koQJ+tymn34bufu8WiUA5PZ/iKYeuNYwOogkRELDxyohie+V92IY2VTyIiIiIALBLlsA1RWg5dBFVMPE27WyshN5dbm485dOc764iw/cRVK5GNLcHS7+2FdMgnYawcJZoYuUw/sXixMqqwjaTgqLcsuoms7bndcW4aGGaud6zX89uaEE64+EL4c5h+P77Qb1POeWybf32DBC6w2Y83bFJHGeCsiXQd3eb2h6luwngYapVkrYnE0qeDwE/86moglVj+mFUMORKLAAIiStGmTJs/x9ZRpvFDLvlEv3J1lOkcro6yBk1fE4TpuSZNG0toFw/rKJftxStjvatIZdOMkGLHCdcMagVnU9kMqOWwjDmEF3NIA98E0JGwAhRWQPNfn0EcorUYlmSPXLkJs4HyYVvOKeMMVVuZJNg0/lBbKnOSOJtaEDmSiraSaa5RntlazVkmg/e8ENaKqBPHCyGQaEJf+d4otp+11dkjzkqoF1fGalmXGFuKMO6wmGMtkYGVkBzzqWwE4H0GCGZOMc/VLaqIKVXVUuz63VsPF4lyiJKQ4sIvNxf2k+MZKILc2rxb2TsfGVdV1s5XUltPvwP55jGInSdMcCrn2uWYx75C2IloYsZ2qGxSRzSxcs1f2TtnTce1y8zpwoUcnAMqcCEDnlt5QNnKflZ+LrchNw4AozCGfYLMqSqVZ9tCGWhuw1BhdHMb+sznpf9tH9JcIT00xSuRxH1F3clBG6jSaOdIAVz0zP6CiKL7WUJKuQ5TKs9hIRPvrENE0a9GcHKdsyyVV5zXG8eVymMJKd5/pCrPoQZbfYX7zw4GhEygGLeyU44rRUPCSgaRau2qoau5nTtKIaDdzuXpiCs5cFmmwnEx02oqRVtYci9vW9exfoMKpQzBRJfRk0mhIBaXrJAkXr8oyse999m3oZQsFTWbs1BXnKHPjhw29MNxNftKHbm11L2ck+eR7jO3XoRfGBXsvzUz6wnymRC6gF1FEaRdOG0SLilF3xNaLcWfS9u40J1usCiUw9MNqiUwdfkKtJ49BHkqA3ae6LVIEREDgXy0hfYFyzD04rtID07O+wPw7N3HkQRfDSMiIjph5Z6jyPPB/GJwOmBRKIe5EMgqqpTYMa6bmesHmlkJuXF1VkIzjnEbu+cBspbA5LVrkI8tgdhzAnLfyWAOPRf9ptJN/WMrG7Xu6bV5i18dQaQjSYWxHBpZ57gmsn8N3biQfRmVZ0XU4/x8jYMCAdcCyKalcVzMZRtDGuHqKNM59m+/nrOeY+FxLwKroO6jlr58tIXJMsZw6CfHIE9lNkieXq+/jnLdyUWbm9uQupOLtsIddvScJZBC4Yx3Tjm+C5q+pq6OcnGsglfqWtbjQ4+sMIJ1XUc5aKuwErKEFM+86ddRNuMSO86/CEpSGTQkEiJNraWPupU1hAgsiyFhBQCS0NXczqCMNVFbH3MTQKVSadptehvrdrYuZ0JOoRYvU1VFBXkQ0RbWpa3lltSlW4pKjk26qVwZohslpNh/cw7kEu+evRRZLnDmoZOglbb149t1q7oWw/KMgeu3IIXo53JpsbPbxDkHTWtjSS6lhIKrGsNbGKtyHhbXENZAdu4Zs7ZrGbTPIJawMgtSyoDuusUJRRTDoe/tRbLvVK9FiogYCGjFUKUSyx7bVVjcFwCv/uL5eOUX1yzIuSIiTic8d9UFeOE/ru21GIsWi8JymJW/XJUSjSrrHu0HeCshPe5kbaQxhSFBho97zOErhm9D7DhWXBM5D28RpFY57nyMRbCCaAJUW/zM+AaJrHVfcK1g5s6gJnLdOnRc0/hC3e6M4yyDTBuA0yKVjQYX2DwfdZTpqx7fTR3l4rwF1GgLU9ethUollnrkExpraNo61FE2beZ8CjTWsJBbt+u/VVBHGYCbvob0z7qOst/WbR1l3Ub/n5yVUKOujjI95hJfV/QLP3ZxUJCmRSobbW3X10TrKeeCWP2IZbG01Jn3cIPE2QCg2rB9UjlEFT0+IKm0c4hSE3AsiLq6CIStptK28nBJsmEsY/Y5rQ2ehoQCaegjeSad5O8a+tlJ94Qwt6m0OqrcvHfclDA2hs9PjJ0LEYSASnI+GntoiCbEisgRTfR6neotc5/BAqG1sYqY4hdRkECQ3gewz1ilwvs6EywS5VCgLfgqJeYYnGLW2V3sr1WlCDZZh8+HWCgy+RlDyFcMIfne21AlK1nPDZU+Pj8hXVO3dcM8DuT2FDjO7Uxls3K7ymUnwgldq5uchXSNbl3I7jXYvjq2spFzlpu1F/CVs5nkNvQ/0jkXCZ0zk1J5gZwIH7L6OBtbApUIjDy2CwkhnwgVupC7hU9IMaXyhHUx+6XyzHV1KJVnxs2gVB59LQQM3crOsa/w0Tbf1dxlqTynakpdbkPdbs432/9Oj2DyHFqlUMMofYxb2SGsNHU163VT6+ZFO3OIKoBWBK1SWMyRjou5GA/rSobNIahl9fMgArySKHKErGahkOv3qFJO/kOgcPcKoaDzHCZSIcuJ0mM+V4RHTrGiA66rmXNzU/dyU+ay60IOFWG/pF5O5SI5D7n1XKW2QFW1FL+Pzvfbu8WiUA4HGUoAUIB8ZxLpP70BMT2IkWsREQsPvXeSnxxF660TSKbi3omIiIhogkWhHGYCJv8fhWv5W3grYTDXt261BKavvQB48yjkC4eBds7kQWQseeCuNUxL41sL64gm9BxNKpv4MtKxZh3PUuev6ctNXcM8uUYF46oIJ3PhQqbn5VLZ+OsNsluZS0vDEVFov5nLHFPXr28J9HMbsu5m7zxBQPloC+raNVBP7Eey5wSS6Zy1LPpwLJDacgDiHq7JbeiuY2siAzAuZb+Osm9B5OsoK32xZZsVzvG4+gJJz4roWxQF52r2yCflOOe4ro6yueDQwihcM6h95Y7NPZGASDCQSJLCtawthvQeUVezT1hxqqZ0cDUTN3IxTkCU6W0UzbEIm/KGdytzFkZtgYQ1V1XkQQTgVFLRj7rCwqivtWxDbiqoKAEn/2Fxyap4y4jSsihUkcrFq70MIpYiqV5gnreUkKLseM8CRwkp1L1Mt5O+jYakohCEoUkF+PWWJVw3sQZ9buk5lJjiW0E52zmtpkzrLLv93VvdF4dyCB1z6N4oTrkD6hU4etxUEaRrVbl3/b6sJZBvWgusXALx4kSpTFUpRZ5crIKGID+hP6eT61j/7TOP/Wupy1UI0tYkZ2HgmmXXDpUxVjlu6EI2cyoSWndyNQdKZqmMDmKeQ+k9YDh3clVuQ//BJEX4qGqS29CfJZSoUBjLcaMtqPG1QCohT2aFIsvo5UTHIrGHnUvlmbnQLi07x/eAvvfJPUiEdSnTUnkAnNyGQOhOpjmi7RjSRk4cuJOpK5kK57uSHYFEyHDkchu6F0FczIyiV1UqL2hj1vGPBwlJ4sQcOqCuZi/HngJCV3M7KxJqkzY3MXY4rogvJC5oAEW+xFIx00xmwFEKAVeJhKR72SqJNA8iUCiJNpSBfOkxSmE5jriDE9j8hqZUZ3ntlz+7EyoXkLL48u3fRqHsHKGUoygV5wlzH/LMYsUmxnZdvu5V5RVtdrx1L9vE4GG4F5WDPls4hZJeFwe7Tvg/6AaLQjkcNCiqGG4vyCcRERENMNpCXiqGrcd2Qk5M91ScFftOIJWD96UgIqLXOPfAMeS+eS5iwbAolMNMFISUercyOUZ4zFkJ6d+BZcxzY/PkDObcAlDXrgFWLgG2v41sx1FH3k6M41BuatFz0cRa6Lt5m1gLWVd0YKlzrW2B3GR8Xdk7d00rTx3hhJ/Du5DNNaj6cWzuROX2DyIhBdAWv9AKqFG4hrX1w7pLfNZwVW5DzgLJuWypsSywJgIQSxNjMUwf24l0Yso5EyWi2HW8PrjuZNumwjbYcZSIApQWPyhMrBqBFApn7z8elMrTr53JJ8ppo2ODknnkYvzchkGpPMJWriWc+LkNawkp1FLpuZPpOHohTqm8xJk7sDkOgdJymAL+lwPiSubuCe9qhvVtEvez8NzPio4jVVUcC2JJYnHyITKuZhOG0CEPoumTcPIgAoBiCClCwfJoJEzmAWH2WIGD5y5Hlkuce+CYcS0D9rkrha1IohQC5i5PSGleNYUSQHS/eXYIhOswHu5sHhUAACAASURBVCuKwj1djiXEFG5OkA8RPNHE9JOcj6ZPCMxm9wzwzjs9IRQgXj8KbH8biBbDiIjmOJkBO44ifWwnxMRUr6UBALz6H1bj5SvP77UYEREDhx+9fx2e/0DMc9grLArLYQ7Xh19HKAFmFkuo16uzyvnz9Rwn/iAVwFlDyA+eAl4+7Fo1WRlqLIKioiayZ/3zYwHriCbcdVVZCwNrI2OB5ORpQjipiy+stybyMYdGli7jC+31uXJx/YNISAktdNQSx8caBuMEP6f4O/yG6qfPsRa/0MUkAWC0BZkD8ngbePIgqe/qWQS9tm7rKBevbm5DY6AzVg2atoaPNfT7JCGs1NVRdgXXneBuZEg4KU5erufFJAJuHeWq3Ia+NY/WUe6U25AzdTL91tqYDHDMYVrE/5nqIjo4L4djiW23K/vZOERKQvHaBAAu1Y35T+a5kwux6MsL6yFgLIiQonMeRJANAhTahF5b/0tJDWbzL/WsicaKRjZZ0VRUFDEEFf2kL0vqKaGc3IfG8kgtfmSP6hUpWQToXDXFSUeDENS6R3Me+nOriCnW+odAbncsguvj6ixbu/HsPm96rhzu2bMHd955J9555x1cdNFF+PKXv4yRkRFnzO7du7F582asW7cOAHDuuefi7//+7xufIxPGyg2AVwQ7kVNMX4Ui6M911hPMOiBvgFRAbFoDnDWM7F9eB0jKDeoi5dbvlJ+QO1+VQtjJdQyECqXfxhFNdLu/HqdQdSKccNfVyYXsz52tC5muac7H5VMM3MqYUyzE3gHcBxht06jKbeircvTvmeQ2DHQech4x2oIYvwA4mUF95y1HR6JEFAkEpfLmCjqPIUCUOlhFD9CeWeaB7+U2DMknoduZJ/1WJbmuOAbg5h2scRv7bWzya0+DpUmwzZgGbmM2SXb4pWC2WJC9I0vF1uQOJPdIu5rbbVcpBNjrFVJCGSWSEFO0u1i3tYlCybGZAaLokTbqYkapZNbkQVRtZT4ULKuZfGlhyuwZXdJjMFNyClAQVERJXlMCJSFFmC9UNEE2zX1o3LzmmlylUPdxibFB5qCcp4iySXMe6kupcyHbdQT53HETYlNZ/blU4awaV4ylXyI4hbL7vdPzr2R/+qd/it/+7d/Gtm3b8L73vQ9f+cpXgjHPP/88brjhBmzZsgVbtmyZ8YdbXyMVEJvWAiuXIv8fex3FMCKiDot+72jFMBXAf987i8dgxGLDot87EREd0FPlcHp6Gk899RQ2bdoEALjllluwbdu2YNxzzz2HV199FTfddBNuvfVWvPLKKzM6Twab69D/baP41a5nf2zb+81gXYRZ+avXaIOMo7+w7t82FNrlPEUUw2z7HrR3HDPjsor1M6Gc35z2kTnc+TJV/GpyRKboOewcen3O/PI7UF5a5hwZVY62yo0L2R2XO2P1PdRycOsVc3JnnHYhO/enPCd13eag15gXv8jNeRQ5p/khY+vG0TVpW5P+2aYWoFiovWNSRZQ/UtgchALU2uenvHGPKTlFfzOWXj9dW3pjaJuAgBxtIS0VQ7V1F+TEVOFtVe5cKmNwbcy4Yr6CzmNYtKmwDXYcB5vXUN9HVaatcX+Lscq4lLWl0LcYmtyGpXUxSAlILjqwIuq/aY5C0yfd30JYbZbRF0LmkLFmnGT7hS59R89Lhfd/JclgJ5Lil8o1R1iovVOUz2N+08SmudEl9uivrqyi+9MUEAIiTYvfJIHQ49LEW8/+CiHs3/o+JuT/kxa/pj2R9r1TupWLfvdYv59EWvzaTSEg0sK6SN+z5l+cqOLXe4/LpPi1bYXFvNi4dL/ot5KbGkpXU2Hf3tD7uxhT7Flvz8NsLQe0339uCfqcKedqL0fdu9XOFyYlV5NnVTFXOL91z7Xid3Zfl3uqHE5MTGD58uVIS7P6ypUrsW/fvmDc8PAwbrzxRjz66KO47bbb8OlPfxpTU/0RcD4biMtWGMVQRfJJxAyw2PeO/A8rjWKIiclei1OJn/n+Tlz+P9/qtRgRBIt97wwK/rcfvoGff2ZHr8VYtFiwmMOtW7fivvvuc9rWr18fJFwNErACuP32283xxo0bcf/99+P111/HJZdc0ujc2kKo4cTCkdN1IpX4Y/RcVdXvxRfQ/hwAnjsE8fYJ5AdOMnPLtSuqmHAy1iWvdtpMDERVPGNdzKBLNOHW5lLr+HGBXNUUbpz/P6iLL2TJIFruiprHdfGFledrEF9IoS2T3RZC7+XeAfhvkE0SX/skFWpB7LS2E4uo0z+QMfm/7wWWpUgOT5uz0lhD7hqMpZLEHlIiSjEXQWJsunZV4mtKRAFpXzFxElLmgDGyebGJtEIKCa8T9OTw2iRs8JMZL+DHFAZtlIiihdBj/dhDgIz34v64uEA/eTWb8kbya5B+p4qKufDuK6T0cu8IIaCcCinlq9L+Hj1QXzNpY+oxO8co4wKdqiqAUz8khVstxTuFWScl8Yc6ns+rpGKOy/eHSgHhkU+KJ51+H5E4WSa9DSWnmLGGC6OgBHD2u8eRZxIQQJLkyMqqKrYGs7XM+2ltijsiQEljhYR81RSajqZoE0EFFAqJMB6QtlFiCu3nSCnB2iTG0Z1b/fkhgSCVz2yxYMrh+Pg4xsfHnbbp6Wl88IMfRJZlSJIEBw4cwNjYWDD34YcfxubNm7FixQoAxZtCf+trghyFgkj/1uCUI+7v2SiC9FilAq3/uArZMweQn2gDB056BJlQsWLZuN556bkpO5jK12kdbk6gUHZQCDl5ucomdM2ZEE7Cfq5UXqiIURmpomiuhVEe65jHTfp9hnNVZvtO6OXeMfm2iHtZo4qIUowLj902UdtP1zV60mgLyc+dA/X9/RBTOTA1ZWRgK6AoMpcek7UFGVu0NSuV5+c2pEQUwCp9+88fhRQKY28fcVzJrsuYKoh+P0dSIR+uRJljCSmmkyqKFUqafqXu5ao2Z46AL6Sgmi6tisLlOeSqpbAJHGeOXu4dyAQiSY3WoygJJSnXETnCPIh5oAiyIBWH7DuIlN4DzCe8oyQyCqLdQEwllXZOlMLyC0M7LwgvAF9mT//bGAazkKqSnAKgqHaiBN4eOxMqFzhv37tATgkp+gsNyUXoMZf1OnqHa3GqqqaYNq2UCdrm5jzU/U1QfLEs58KtllK0gbQVqFIC/eckVSL5HIgi+II+E/TUrdxqtXDFFVfgscceAwB861vfwtVXXx2Me+qpp/DNb34TAPDkk08iz3Ns2LBhQWWdE6QCQ5vWQl58BuQ5w72WJmKAsdj2jhhtYWh8HZJ1y4GRnidZaIxX378ar/zc6l6LEUGw2PbOoOK5Sy/A85fFPIe9Qs+fsp///Odx11134a//+q+xevVq/MVf/AUA4Bvf+Ab279+PO+64A/feey/uuusubNmyBcPDw7j//vshZxCk3BahCxUIrXP+t4FurYT6b8ealkoMb1oLsXIpprbvwfTO447L2MjgreO7VTulo6FjqFxNrIVcGpnQRRpa/Pzz+LkKOdmq3K+c1dHtD928gfzz4EJ2ZPAtlZ6M7NqzcCtXYSH2DsC7aausgD4RJWxzwX279YO6xWgLw+PrgFRgeusuJEemA/c0XZt3B889aG5Dacgl5bmJlRA17mTJWBON8IAlosAzooWmTNvFVUXx2otXakEMXc12onVJO6ls6lzI9Jirt8zk4ymsjUyqG1+eOcCC7B1NFNGWrNKnGLiaM52ippzHmaWcAtzV+7F4p/AuZqCwIHIuZjPfqcVcHksBEIshACgvD2IxDuaDgEtvYyyNUoB+glmLYfmal/uiHCVKs7+1DJZ7R9pnup/WBoBTNSUjW8taEfXfJA0OcS9X5Tw0l6qtn9qrqEL39Exyj9Bz+NtbNVjLrzQ18HkO16xZg4cffjho/63f+i1zvGrVKnzta19bSLHmFqViKEvFkJbEi4joFoth74jRFoY/vA6iVAxVH5NPIgYHi2HvRETMBj1XDhcCOk1KYN0jX8BmayW064TfGPJEoJUInNq+G+03jwLCP4c7l7b51rc6S15HORjLILtODYmlk7Ww6jyB1dZJSu3KTGVoQjipswJWWfHmKr7Ql7uuqkq3MYe9hE/yaEJE0X2+fYOtesL0O5bIJAHaOab+nz2QE1PBmm78YHX8IW23xBQVyOHEKZK1KRGFruFDEAuiTrFRtFvroTOeJsEOQ/fclDXm1VoE/ZQ17gVXxBkS8klAxCBWQMHOkWDjBusqpDBrV44LikYn6HH0U/eQLba2slDSxiGiDfMxrPS4DmSVGrg2OS/+EKgmqRgiCqnFXH5AUkKKrbfMtElyKW1iVvMt36mqJqeU44p4QQUhBJJEmcTYAJzYQ7vfhPHEcVVT6F7OGqR48est04TYKG8XTYgNeLGLZB2uWoqZo4TxOjQhqxTyaAup/VwT3n9+LrAolEOde9C/dZziBMxSEaT9LQC5Qj7VxrHv7gDUzBU4IHQb03FV6wWEiw5tTYgmem1OYaoqpaflCkkjjNwNCCdNy94FcnkklG5dyP4c7nyVbuUBVA4BXwks2ypcxVw/54Lm+uk6ckgWCeEnJjH16Bvlm1KY8VbBq5EbcKqi+OcRZL5V/uAQUYLzESWRKn6C+d/KMlebgHUz+9VOimNlXn1CCh3LcjQoMYVjK1MWMluxhFHSAiKJ10eJKB4E1XQpEaXqYihJpcplO8d5DhcMSQIkLSCbLv72K6EAEEhdogpQKJTCVxQps6ME58LHzFzMQOEipkQUoHjvWOUxCwgpaOfu+wzgGcyKfNvS2kYb5H2rzAcOdS8L7VtVZe5DRRQ9o/wpc9k+c7m4fusuVkSpa1pSb6Fg3cnC/E1L6RV9VLb6zxF9/blSDdTgaiwK5bAnSCWWfmgt1Kk2Tnxv91wr9RERpy3kaAsj4+sx/dIEsh8dGui98/Pf38FaDCMiIurxC8/+2MT9RSw8FoVyqOB+E+hkJaxKdeP3VaagSSVGPrQWydhSHN++27W2VcihZalzG9P5jdPacJbICmthJ9exP6fKWljndq1yRZu22rnNXch1qWw6uZDNHMatXEVwoW2sW3keCCkLAc7S5/fRfmplbJTT0CejjLYwOr4eIhHIdh73ZODX1X28O7g8nqNbTzy7Tm5DSkShr6NHTjlkFEpEoeOA0nLicT0cwWluQ/9mcrWVAeIiDl3MgpJPuDyHVa5kzoXMWSVrTZ52LUFdyJ4VUQyq1RAoXcoSQKv4W7uNBfk0EqJwM4M8o3OaB5Ga28pjri6zqbtsP9JDFzPgWBD1+xKA0v3al5omxLJIrZKEkFKT3oayKxxyStlHySnmthD3spDAWSdOFn0CJvch4LqXq9LaAKUF0klr426dTvWWuZyHlDRijvVlCXc+ynWtVdKts6z7u7VSCrhubr32XFk9F4VyqEutaVQqeN0ogn5/KrH8QxcgGVuKY9t3YfrNo5VuXjN3Bm7j0M1dn5+QttWdp6kC1yR5dVNXNI0brOtj3cREAWviaqbrz6ULmZPBKI0+ixmDiU5KoJuyWrchmFMcV/cno0MYHV8PJAInt+5EPjGJlMzh1qcxkZ0UQBprqP+msYZBm3m1LmQORZ5DrfSVbWW5rz1rz4KUOdbsPly0U3da+bdDRPXczoJquGZMlSIo+FcADsuYUR7pa6CQ+WxlK4h9ZdnKHPO4xoXssJmTsG3gIF0XsfKf3igUvVKxE2XuQycOUeTuWIDojWmoFFKFEb6LGWDjENvMuHZmFXQmSXYVg9mJP0RxGU78IQCQOFtIEkJRvghZvOXfGjsHeQ6s23fI5D4E4LiXq5jLxenCxNjKSTBtnwNUUTRtRtGjXnK9z4VzK7T4Td3TNPYw10n+BTW2lPeCygMtd3PMpoTeolAOFxIjG89HShTDiIiIBkgElm9aByQCx7e9BTFxepQpe/Wy8yAEjHIYERHRDM9tWAuoQjmMWHgsCuUwgyZpuOCsZfTYL13HHfvrnHj2AORrh3HqzSOkn8ytsd4552ba6tzGVW0zsRZ2ch1XzQ9zETLXEBBSKix6Da2AC+FCpnOqXMicDFUu6EF0K9vKKC50kXnAtwzyRBQ7R3ht5bfoTOHkMweQH56EmJhkmcuOu5hY+gKZ9XjlWioDt7Oy7iTbpsI20HH2lSs+YC0ZJQEFAOCSUZzxFSQU21/hlaVEFMCxBnLl8Uqh+Fc6v9JKSF3M3oVL6hoW4VzuYriqKM7QGpf1oCBJXMthRgkn5TVl04Fr2KmqQt3Lfj7ENni3sj7uRFIhLmQElkHvvaj/rmEww3E1l+cl5BOneoomkkhl5zjuZW1GLPdITixrxL1cxVwuxvFVU3zPgIKYNwIKfVbRUnrzAWGemfrcs4vXHNBd12dIJYYuPhMA0H7nFKaixTAiohGS0SGk548AAKbfOIIs5jGMiIiI6DkWieWwIs8hOe7GSpij+AY1+qEL0BpbhsmDJ5C9a91hdfkCq9buFFPYJEVNVW3lTmlk/HNUWQubVDaha/qkEisjY/HrQDjpJr7Qv77ZxhdybXXWRr32XBVEX2g0yW0YVjkJ4wwdw5cQSEaHcNb4hRAADv/fr0GWX9+F8tcK1wxl5GIF6y2MTSFg8yFWVUWhxBIjk5PKRgfNl1ZCn4hSnKgzEcW/GJpOhLPocZZFGnvo/d+EE5tYEWdYVw2FVkXhTJ5e1ZSgKsqgWgk5pGlZQ9kjlyhKOGmFsYheqhvAi0OkFkQafwgEMYc+BMhzyVggM/s+KuMRFeBYE61FkElvQ96DKohDJOZ7E5An4OQ+9OJwi5MLiJLgofN/2lhDYgU0lkE3rU3Rpq+4M+zW0s9uVFZLQSmyH1+YKxJ/aAytlthSkFi0L0G3EeKLuU3WK0NT2vjnU86cuf9sWSTKoZshqltFsHgl/4RU4qxSMTy8fSemDk+SuXTdUOlr6jam7VxbJwV0JgphrULJkkX4NTvlKgyuqwPhxJGnRnnspPDVJbLmzlnlQg7avGuuJqQMnnJY5aKQzBj/mGc4F0hGh7Bi/EKIRODYtrccAiddv4qMQokoAJzk1VWgRBSzjqc80rVlh/+XzXPIsZWZ8ST43vfCFm0q1KccvaxOESRtnCuZK4vXseydCMc5wteQS7j8hVWKn1NeL2HaBlRhlCncHI/EvZwQwolxN5esZkwHSwmkUL6SmbVdFzNQ6VY2IG5V8+52XMhlnsNcGKWPElgcBrPPyEilS04p+4L3slJ0k3ll9UJduchzaG+fNITQzoqfJYtYpY/mPCzWUUThssQUKq7PVlZaa+1jcIUIZoJFoRzOB0QqceaH1qE1tgzvbt+FUzuOdJ4UERHhKIbvbnsTOE3IJxyueuI1NpYwIiKiHv/puZdYgnfEwmBRKIdKAFmFtbCKlFL8XWHJUwrDq5ehtXIpDm/fiZNEMayrXOK311kE6Xm7JZi463SXliZoY67Lt5x1Sk0D1FsLaX+nVDZNrYGBjA3yFnZ0K1esUWXBHERCClDtmKkiovhz/FyJS99zFkQiMLFtB8TEFJPH0HVZB+7iDlbCqqoolIhStJHgdGNVDAPWaSobWhWFg81pWJBQRk5OQiahwFxVFLffvgaGM9mAiOL1VVsJ/TkckYSSVEKrpKD5eKqIKPqVS2/DgVZXGVDLoUgSKEo+MRbBFG6pPM9dnLRQZT0EGpBUKiyGAbT7uoKQYi2MtpSe/b9Yc5ooXcmKVE2h4Q5c7sOqtDbFHAUoYHTqFPJ2te2r2GPls8PLeViI0LykXpjKRnUssxfwbAT13tG1zaU6pfR0/2zyHJprIKl15irf4aJQDucDk28dxYFv/hjZ8XATR0REVOPYD/Zj8tXDyI9PI50lo67f8da6cyCkwvqd7/RalIiIgcJPVo0hz4CL9x7otSiLEotCOdSpbOqTXFdbCTVEKnHWr6zDsecOYGrPcUwdJ+QTM7feSkjPUZsYm7WChf1Vyat9+TunwaknmnQe19laCHgWQf/6Z0E48dvqklzTed3EF1atofvZNDkDajn0o1a4KJaqqijUfpGODuHcq9fiyL/vRnZ0Gjn5UmXi/ZhSWdJbk+sv5ArjFOeyKgologAIqqJwhBQhgNfeswoQwIW7DkIy44KqKEEKEdGZiEL+Lo6l7asiomgBOaHrEl5XkUY8okllrCAlogDwq6IMdEUUHyLx0tKUkDnQLt//lYQVvUZo/6klqcgcYSCih9JiKEriimq37XuiKr2NIYuUVrc0sdfTJvWY/XrLXGJsadfx09qYdXLgxQvOB5TAT+0/YKqmAC4xpSohdiFqs3rLWcVzQnsT/Gop+jb5PJtuYQktpRVUUI9eAfp88/vmC4tEOdRs5RpljPngpi0qFTh303oMrVyGo6+8g7an3DR2SRtFhx/bKQ9inXuXrtFMGa0gaTBrz6SySZ2CR8c1VszqZKhSxmraZsJCpvPpeR1ZK5jQ/vXPB6NsoeCzkUOl0RtPWlqjQ1g5fhFkKo0ListjaOfybmPXRWyJKLVy61elHEVSzw3aYBXAuqooAHVfhUQUrp6yQzipJKx4gsMjougT+uBIKo5A3Byr6DlKWSe2Mlc+z6zJMI9nUhWFKbknBtStXBBSrIvYKHVZG0hL8klOS+kxhJUasCQVAK6yWR5zX0x1XkXJUK8og5klp1C2stHGAreyk/uQupermMtlvw21UGbfGNcwIaZU5Tws2sKSeqj4AkqrpZRndV3D5fSsxtEhAaeUnr4lVaX0ADiV2+YaXF7ZmWBRKIezhUglzikVw4Pb33JiDCMiIqqRloqhSCXe2foG8sOnL/kkIiIi4nTBolAOc3i1lTu4au3fgEgFVhLF8MSOd2dsJaTtnJWwbm6VdbKb/ITcebu1FtJ+31rYKVdhINsMCCed3MV+Wx3hpGNbAxeyfw6OeDLIFkOg2lUsmH76PTVdPoRV4xuMYphNTCIx36Lp+uHadL0mbmIJl4jizxVkPq2AQokowfmIBdEnovhWRUpE0a+izHNI8xf61VB8QgpjOAvguIupRdC3EvrpaTpVPqnrYyyPTlWUTkQUAGx6myo4FsgBjUtN9PW6KWoEUiiWhssQTagFkZkTkFSUhONWVsx91kQUroIKIaRUklOA0q9KyCllG5v7MCCpgJjoFPwHgKmaok9a7h9tMaTu5aqch/rycu+WFZWN7Bx97dRiCFS7bB0Si352kNQ4+lKbkkEKq6W1LBYy0LZSLmH/HRzZpZijr0eQ1u6xKJTD2UC1FaYnTuHoCwdxYse7vRYnImJgkE9mmD50Ckef2Yf2xORpTj2JiIiIOH2wKJRDXZ3C16OrqpUoKIhUQg4nmD4+jYNP7Gbn1Fkd/b4mVkJ9bk4eei11cui+RnF4FWvT8/LVV0LL2FxaC6ksgVWuRp6Olr6mbTOILwzaatYeTEKKfnVVuzAptv07Xd6COJlBTec48F/fNIxkP26xmMedk6/bTOfwsYLzB0pEKV5dKyEHIRR++YmXkSSKJaMUg2isFd/PmVb9+ENBrGtOX23aGs7q2IFwwvVLGjdYY/KER0Tx1hOnW4UUkXiEk9KCKHKIrLAiKja2kEtvQ8cxGTKMhZDGGVZYDetS3RirogBNfs3WXtaWNxp7SPO66NeApEKshV5am2IcAAlsevF55BlMahtrMSxfvXrLgnm+BLHAQpDPhvKShGBJKVXVUoo20TH+0FxeeazsJdbO7RcsCuWwqnwewLttRSqxatNFkEtS7Hz0FSB3VZImLGOgObnEPz/t49zGdM2q9fiKJE3HNeufSa7C2jZyrZ2U2iZl72Z07lm6kGeydu5e2sCgack8ABgaHcbq8Q2Y3H8CE9t3lnPIWgwRRbuGqtQCqgD6c1l59ecT4FRFEUy/vzbNaWjaKk7nspXLNuNeLv5eMtVGktgPPeOWrvLCMicNiCgOM7lCEdRC6A9m+r/qxA7mSuFRIetcw5wSya3jyEmqonjKpRjoCikJRNKyTwSHcFIoiiKbrlEQAUdJ1PeBVlLRzOOEuJdzPY4oioYpUaEY6nXK/mI0dSFbcgpQuo3rch8S97Lvag6qplDmcvmqpMLSfAoqE8XflK1M3MvaNUy1LUpMoTkPgYKYYr+jheEylJgyExezHqePO7mVq5RHAMiVsEpvw/WqzsE9o2cyP4JApBLnbboIwyuX4dAzbw/uJ3pExAKjNTqE1WWM4bv/a3+vxekL/OTCMby2fqzXYkREDBxeHluNl887r9diLFosCsthDpcyXpV/kCqGe7e/ieM73p2xlZCOm6mVkK7T3H3NEzL88zSxFtav09xa2DSVzUzJJfRcvXIhO3Marq3bw8CG/of0AqM1hHfcGh3CmvGLIVKJt7e+jnxisnaOXT9cm7PoBfM8Y4NQvKt5pijkKS195NUnotS5kgFNNAFev2gMAgrv2bnPtPuvvmHOfa1yFzN3kyOimL4G7uLiApg+6vt2LpasnYT9Zh0mvU0H9zM7f9CQpEBbQpSWPkUrpJBqKJ1dzHCJJkEtZti0NElqnzJtYm10lvbS2+R5UZOZQKCanGLm+rkPpXDT2gDue7WqagpNa1OeQ0jglVWrAQVcun+vcS0X88tXBWMxpFZESkyhOQ/tlel+vU5YFcmvjmKse/qWiXoroSCXb+cIdEsSEURyaml08iB6NaNnmyYnWg4JzrlyNZYQxTAiIqIZzrtmvVEMpydO9VqciIiIiIhZYJFYDot4w05WwAPPvI2jb72L47uPsmuY4zlIQcOt001MIW3vlBibI5JUWcd0P5eWpVNlk5kmsq6VsUHM4XzFF1bKXbO2I2OwzuBZDgEvzrDieN9/ewtCCMdiWEUmoa/ceQJLomclpJbFzrLbNWi6mqJNBW2dwFVFkZInm0hRWiRK8kplDWU/CbCzCPib5RNRHItfBbmkrkIKtSBycUo0TrGm8knnCilhbWXhxCk2rL08CJCysB5m2qpHLYjECkjiD4EqCyJQG4eoXX9Q9AAAFHhJREFULX/ttvn/FFY8poKKJqqY2sqpsTyCWv4oOSV33zt8Ymy7j6idTuXWsljIRUkqCM1yuo1sXl01xYeNKXQTYheXopDnIhhPq6Xo03KWQlpn2YY2Ms80ktKGMVAG68JZ2xJk8uDu9RaLQjnUhBQNR0lMJc75uVV459m9mJ5qY3q3TXBdlQ+RW6cpy5ify7frNeoVuPB8le7Qhv2zIZpUzWlKJJkp4aSj3PPoQqZrVDGW9Xq5Cl3mgwD9IA3YykKgNTqE0YvOwrs/OoDpw5MAYJnJzBqcsiiU4HUfQ1ypd29QhTFQIpVbFSVwRTvztZJo59CSebLCjQyUvA+PiFJVTk/Qk3MXYxQ32h665cI+6baXfUHJvHJssU4HRdCvhKKPmxBNGBdyx0onRKEUjpLZfVB9TyFT1yVOlMQqFzNQQVLJc8JYpq7m0rVcKpYgjGGngopWCDkGc7vtKoXl+Qw5hXEhI6WKYH3uw45VUwKNSYAy9x0FEvbt5pfUM+U3adEZbz9KIdhqKbSiSSGKfQ5QxdG6dpWpxMIxjznXr0QzYokENQ7Z56m5vAX6HBngr2Wzh0gl1m7agLMvH8OSlSO9FiciYmDQGh3CBeM/hRXvG0OydFF8x4yIiIhYNFgUT3W/QgpQKIZrNm3A0pUj2L19B46+bS2GnSx54fphf6cUNJ3WnonbuGl+wnC95pa/ubYWOnLMMeHEkXeeXch6fOhCdvsHGfQbpECRrmb9+HsgU4mdW19DdrKNhEtRw6xV5TqmbVV2IlpPmRJR5htChEQUmufQGUtcxEIq/Or3n4dMVGBcc8fRdv3KWAk1KgkpDKmE9tURUvxKKfo8nJD+3E6VTxyXte9+Tvi1TwMIIXkXcdaudjEDLknFuI1bQLu0DpqUNjSnYQVpx7iQmTk0vQ0lp3htlTQK35pYkftQyLw8poQUakXU68GuJ4EbXnsWqvScK9g9o4Xh6i2DjBOK7FfPk15cl97TwqmWAu/IdTEXoBZAwYyrgvVqlJciZp6mRntV/DQ41tqor0V0TPtVh0WiHLrxdbIlse7aQjHc+b03cGTH4XJcjbJW0W76O7iNZ6oIdjpv0/yEdC7nDqVzO8UScnOaXMts2ug5m8YU0rXm24Xsj2cZ18y6gwKrzBWvQ6PDuPC690CUiuHUxCmjGFJwJfWqFELuAUYfok1UBydEiZyXlsyjsYZBG+g4/XCt/n8VTEirMFaxl9MsRyJzoFQWK+MOUaEjSaIgcszlGrdxoBAGc2Q4p6oUnnZFV7l5uZjDBixjJ/E1p2TKpLM7ul+RJBBZtYIIMC5m3afvty6PR9r43IdMmb08t/GHJCbRhguU45QEV1LPgZMcG5hR7kMuT2cVcxmATozdgjLxijrukF4eLalXLFs+Z0nsod6jOvbQyUlKYg8z5b6nJZR5OnDKG0npyMJcCrn8KiXQfwYpALm5iN59ZgzorpsdWiNDGDpjGLuIYhgREdEZw2cvAaTArlIxjKjHyxetxkvrz++1GBERA4fnzl2D51at6bUYixaLw3KoFDKlGU8Kpw6fwqv/8gKyttXlm1oJORdhE7dx3TozcRtzjGJ/XJU1sY5FXNnfgK3sy1ebL3EBCCd0rW5cyM61Mi5kf3ynqil6zCDmOdTQe+fom+/i+O4XIDM4FkPW6FVTLk+o6j6OuOLMdcaWr3N0ayknhJbMoyxlDmHge2FN3LH2XAgBXPrmHis8iJWQCi5FrZmVsyA6fXW5D1lrX4WV0CeicOXy6LjKyic+uaSD+/l0gyzK54nSStjMxQyXaFK2FeQSOG1s7kMaAkByF4pMT5EhOSVXQZ7Dor88o09OAdA092FxcjdEIqiaQpnLACABlSn8eMUYoIDLD+x2SCna+q4IE8TNc1hPTNHPJfp5a8rilX20nJ6g/WU7ZS1bb7hdu9raqD0xqnKcM0evTeRdKHrWItihBWRL4sLx92Dl+4uM66rdKTogIiICKFzJ7/n1yzB6wZkAgDzunYiIiIjTGn1jOXzggQeQJAluv/32oG9qagr33nsvnn/+eSxZsgRf/vKXcfHFFzdeW6UC66/9KSwbG8GBF/YhI98ZmloJ/dizYjwq+5tY/vyP2G5jCpvmJ+TGz4ZoQtfrJn+hP36uCCdN+ucqvrBuPV+e+Yo5nM+90xoZwsW/+NOQqcTUsakKa2D4LVow/VWxhdw3VGpAaxpUbY1xiszVbaFsxblLa0NDq25dPWVH/rLKgz6Xk8YGntGtk5WQSVHDVkPx4g+Deso0hU0wp8ai16SechArWG9NrKqnLLjYxXmKOZzPfQMAQiZQJO6y0oIY5EEEbE5DYkE0Fj8ay8mlqiHpbfwUNUpacgqtnqJPx+Y5lI4VsWij8YU29tAQTUiN5U5VU5y0NrCiCVHcC71HbL9e2xK9hHLjD01/OcVa/Im1UVczUmSvaBFhPw8kwjyIrjWxtBbS7UYuVZ+ZraQC+3zUnxdNCSrSioucXBmtlDIbK2PPLYdHjx7FPffcg6997WuVYx5++GEsXboUW7duxT333IO77757Rue44JfXY9nYCHZ873VM7JgoCSpF/kP9k6nc5KJrqxxtlRuWcwabwljB5k3My9/MOFuLdfRa+liRdarOzc1R5ByaUKN/zVzkVg49V3nzyLlUqTiq8lymX9lf7rq4c9M+/29/PN+WIy/l9WWua1NVsnTod87vKWv++lzuQtVwPd+FPF9K4ULsnQuv2QCZSux47MeYmjhp2v2HDseME8y4KmWQjq/ql8oqeULBlMyjSuBMIc2vMmQU8ytgmMp1pBOd4NoohJV6kR3nttNf+2FphRTuL9tXo7RxbuE69rJek7sY7YrWCaurSCeaXEJ/+4RYshD7BkCh7CapvU9JWriZhTTHtN2+CUibuXcJRJIW69G57pvHnUP/B+WbWQhZuJDT1D0vfR/p8c7/rpyvk5Z3+tVw3k96DfteFtz72t/QUjgPBfO2JG9RNwe8MmEdeg9bcexe5vY03f8U9DlT9awRZFw3/XacIM+l3ilpPbccPv7447jwwgvx0Y9+tHLM9u3bcccddwAArrzyShw6dAh79uzB+ec3C/RetWE1XvzX5zHUbmFs7Zhpr7KGVcUQcvP8Md1YCWvToFTIqJjxtVYwhOeonsvIXyOjXzeYtyzCQwfrZsP4SLpCU7ayf6zPE8YXutegj/xzBGsz9wcA1qxZjbnEQuyds1aciV3//ibOGBkFRkZJQmv7jTeBgF+DOYEIkl8nEKaWgn4mJ7CxiwmZqx/AqbLzU/KtPiHzUf5tzl3e78QZp8xYbVlMARPJlZTvegmFRCgzByg+bJJSodOvQua2Lc2N9VCWJ0ySHEIqLD97BSCAZPUqJC0FUfYbw1mqIEtDj0gERFJaRfSTORFAKynbpGkzx2WsmEjJh3Kqx6fmGFK67YWQVrnT49K0aC8urJyb2PMkdM2WXYeOBYDWECD1HDJe9+u6wJLMTVIzx5FhjmsrL8S+AVDIrVIYdcIJqiOUW22NK98UQqaAKu1HsnxN2rYt00mn20U7bUNu4w+zDMjL+EQdX59n1oKnq6LkmTtHv5rz5HZslts5bdKvz5GXbVNtMjf31s5tUF87N2uKqaJfZYBqKyxfcXYRxjh2HlRb2VDLdrlHMmFuicgEMF0+b0rLqMgEcn1czslzYfqT0ropM2nYwdpCmCmBpHyiZBAmGbWx5woBe8fLPmFjCjX/PBPWopjBHrfLZ0tbgKxjbcb6mI7Xc/TnDi3uQY/153emFM46bwW6Rc+Vw5tvvhkA8Fd/9VeVY/bv34+VK1eav1euXIm9e/c23qgfu+1jwG2zkzMiot+wEHvn45/6BPCp2cm5mPE7vRYgIsBC7BsAaK1Y272QEfg/ei3AIseCKYdbt27Ffffd57Rt2LABX//61zvOVUo58TNKKcjFwHaLiEDcOxER3SDum4iI7rFgyuH4+DjGx8e7mrtq1Srs378f69atAwAcPHgQY2NjHWZFRJweiHsnImLmiPsmIqJ7DMRXoY0bN2LLli0AgKeffhrDw8Mzi/2IiFikiHsnImLmiPsmYrGjb5XDb3zjG/jLv/xLAMDv/d7vYWpqCtdffz2+8IUv4Itf/GKPpYuI6F/EvRMRMXPEfRMRYSHUIBZ8jYiIiIiIiIiImBf0reUwIiIiIiIiIiJi4RGVw4iIiIiIiIiICIOoHEZERERERERERBhE5TAiIiIiIiIiIsIgKocREREREREREREGp61y+MADD1SWR5qamsKdd96J8fFx/Nqv/Rp+8pOfLKhse/bswe/8zu/gwx/+MD71qU/h+PHjwZjdu3fjAx/4AG666SbcdNNNuO22han/953vfAfXXXcdrr32WjzyyCNB/0svvYRbbrkFmzZtwr333ou2rrm5gOgk40MPPYRrrrnG3DtuzHzj2LFj2Lx5M3bt2hX09cM9rEPcO92h3/fOIOwbIO6d+UK/7p1+3zdNZDwt9446zXDkyBF19913q5/92Z9VDz74IDvm7/7u79TnPvc5pZRSTz75pPqN3/iNhRRRfeITn1Df/e53lVJKPfTQQ+qLX/xiMGbbtm1GxoXC3r171TXXXKMmJibU8ePH1Q033KB+/OMfO2Ouv/569cMf/lAppdTdd9+tHnnkkb6T8ZOf/KT6wQ9+sKByUTz77LNq8+bN6md+5mfUzp07g/5e38MqxL3TPfp97wzCvlEq7p35RD/unX7fN01lPB33zmlnOXz88cdx4YUX4qMf/WjlmO3bt+PGG28EAFx55ZU4dOgQ9uzZsyDyTU9P46mnnsKmTZsAALfccgu2bdsWjHvuuefw6quv4qabbsKtt96KV155Zd5le+KJJ3DVVVfhrLPOwrJly7Bp0yZHtt27d+PUqVN4//vfXyt7L2UEgOeffx5/8zd/gxtuuAF/9md/hsnJyQWV8Z//+Z/x+c9/ni231Q/3sApx73SPft87g7BvgLh35gv9unf6fd80kRE4PffOaacc3nzzzfjEJz6BJEkqx+zfvx8rV640f69cuRJ79+5dCPEwMTGB5cuXI01Tc+59+/YF44aHh3HjjTfi0UcfxW233YZPf/rTmJqamlfZ/PsyNjbmyMbdN072Xsp4/PhxXHrppbjzzjvx6KOP4siRI/jKV76yoDJ+4QtfwBVXXMH29cM9rELcO92j3/fOIOwbIO6d+UK/7p1+3zdNZDxd9046p9ItILZu3Yr77rvPaduwYQO+/vWvd5yrlIIQwvlbyrnXkzkZ169f75wbQPA3ANx+++3meOPGjbj//vvx+uuv45JLLplzOTXyPA/uC/27U/9CoJMMIyMj+Nu//Vvz98c+9jHcc889+MxnPrOgclahH+5h3Dtzj37fO4O+b4De30Mg7p25Rr/vmyYynK57Z2CVw/HxcYyPj3c1d9WqVdi/fz/WrVsHADh48CBrip0tOBmnp6fxwQ9+EFmWIUkSHDhwgD33ww8/jM2bN2PFihUAin+m/tY3XzjvvPPw9NNPm7992c477zwcOHDA/D1f9202Mu7ZswdPPPEEPvKRjwBYmPs2E/TDPYx7Z+7R73tn0PcN0Pt7CMS9M9fo933TRMbTde+cdm7lJti4cSO2bNkCAHj66acxPDyM888/f0HO3Wq1cMUVV+Cxxx4DAHzrW9/C1VdfHYx76qmn8M1vfhMA8OSTTyLPc2zYsGFeZfuFX/gFfP/738ehQ4dw8uRJ/Nu//Zsj25o1azA8PIxnnnkGALBlyxZW9l7KuGTJEnzpS1/Czp07oZTCI488gg996EMLKmMd+uEezgZx7/Do970z6PsG6P09nC3i3gnR7/umiYyn7d7pnhvT33jwwQcd1tg//uM/qgceeEAppdSpU6fUH//xH6vrrrtO3Xzzzer5559fUNl27dqlfvd3f1eNj4+rj33sY+rw4cOBjHv37lV/8Ad/oK6//np1yy23qJdeemlBZPv2t7+trr/+enXttdeqr371q0oppf7wD/9Q/ehHP1JKKfXSSy+pX//1X1ebNm1Sf/RHf6QmJycXRK6ZyLht2zbTf9ddd/VERqWUuuaaawxrrN/uYR3i3ukO/b53BmXfKBX3znygX/dOv++bJjKejntHKKXUAiiuERERERERERERA4BF6VaOiIiIiIiIiIjgEZXDiIiIiIiIiIgIg6gcRkREREREREREGETlMCIiIiIiIiIiwiAqhxEREREREREREQZROYyIiIiIiIiIiDCIymFERERERERERIRBVA4jHOgi4vv37zdtr776Kn7pl34Jx44d66FkERH9jbh3IiK6Q9w7/YeoHEY4GBkZwYYNG/Diiy+atvvvvx+f/OQnsXz58h5KFhHR34h7JyKiO8S903+IymFEgMsvvxwvvPACgKLW5muvvYbf/M3fxNGjR/GRj3wEH/jAB/Dqq6/2WMqIiP5D3DsREd0h7p3+QlQOIwJcfvnl5hvcl770Jdxxxx0YGhrCkiVL8NWvfhWbNm3qsYQREf2JuHciIrpD3Dv9hagcRgTQm/Rf//VfcerUKWzevBkA0Gq1cPbZZ/dYuoiI/kXcOxER3SHunf5CVA4jAlxyySU4cOAA/vzP/xyf/exnIWV8m0RENEHcOxER3SHunf5C2msBIvoPQ0NDeO9734uRkRFs3Lix1+JERAwM4t6JiOgOce/0F6JqHhFgamoKhw4dwmc/+9leixIRMVCIeyciojvEvdNfSP7kT/7kT3otRER/4cEHH8TIyAh+//d/P+j7+Mc/jh/84Ad4+umnkSQJLr300h5IGBHRn4h7JyKiO8S9018QSinVayEi+gMvvPACbr31Vvz0T/80HnrooRgEHBHREHHvRER0h7h3+hNROYyIiIiIiIiIiDCIMYcREREREREREREGUTmMiIiIiIiIiIgwiMphRERERERERESEQVQOIyIiIiIiIiIiDKJyGBERERERERERYRCVw4iIiIiIiIiICIOoHEZERERERERERBj8/+9k6a0LUnB8AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_softmax(-1, +1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that we assume **one-hot encoding** of the vector target values $\\mathbf{y}^{out}$, which is not very efficient (unless using sparse-optimized data structures) compared to a single integer target value $y^{train} = 0, 1, \\ldots, C-1$. However, sklearn has a [convenient utility](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html) to convert integers to one-hot encoded vectors (use `sparse=True` to return vectors in an efficient [scipy sparse array](https://docs.scipy.org/doc/scipy/reference/sparse.html))." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.5" } }, "nbformat": 4, "nbformat_minor": 2 }