{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# The DistributionLambda Layer\n", "\n", "> In this post, we will introduce the most direct way of incorporating distribution object into a deep learning model with distribution lambda layer. This is the summary of lecture \"Probabilistic Deep Learning with Tensorflow 2\" from Imperial College London.\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- author: Chanseok Kang\n", "- categories: [Python, Coursera, Tensorflow_probability, ICL]\n", "- image: images/distributionlambda_layer.png" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Packages" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import tensorflow as tf\n", "import tensorflow_probability as tfp\n", "\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "tfd = tfp.distributions\n", "tfpl = tfp.layers\n", "\n", "plt.rcParams['figure.figsize'] = (10, 6)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.layers import Dense\n", "from tensorflow.keras.optimizers import RMSprop" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tensorflow Version: 2.5.0\n", "Tensorflow Probability Version: 0.13.0\n" ] } ], "source": [ "print(\"Tensorflow Version: \", tf.__version__)\n", "print(\"Tensorflow Probability Version: \", tfp.__version__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Overview" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Linear model with Sequential API" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.layers import Dense\n", "\n", "model = Sequential([\n", " Dense(1, input_shape=(2, ))\n", "])\n", "\n", "model.compile(loss='mse', optimizer='rmsprop')\n", "model.fit(X_train, y_train, epochs=10)\n", "\n", "# y = + b + eps\n", "# eps ~ N(0, sigma^2)\n", "\n", "# Likelihood\n", "# theta = (w, b)\n", "# theta^* = argmax_{theta} P(D | theta)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### DistributionLambda Layer in Linear Model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "import tensorflow_probability as tfp\n", "tfd = tfp.distributions\n", "tfpl = tfp.layers\n", "\n", "model = Sequential([\n", " Dense(1, input_shape=(2, )),\n", " # Final layer includes normal distribution\n", " # Form is a sort of Lambda function, that can give the output as a mean\n", " # in this case, output tensor is go through normal distribution\n", " # And normal distriubiton is trainable\n", " tfpl.DistributionLambda(\n", " lambda t: tfd.Normal(loc=t, scale=1)\n", " )\n", "])\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loss function for Distritubion Object\n", "Negative Log-Likelihood Loss function\n", "```python\n", "def nll(y_true, y_pred):\n", " # Since y_pred is distribution object, we can call log_prob for sample data\n", " return -y_pred.log_prob(y_true)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Converting Tensor function\n", "In this case, output will be tensor object, not distribution object.\n", "```python\n", "model = Sequential([\n", " Dense(1, input_shape=(2, )),\n", " tfpl.DistributionLambda(\n", " lambda t:tfd.Normal(loc=t, scale=1),\n", " convert_to_tensor_fn=tfd.Distribution.sample\n", " )\n", "])\n", "# tfd.Distribution.mean\n", "# tfd.Distribution.mode\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "model = tf.keras.Sequential([\n", " tf.keras.layers.Dense(8, input_shape=(12, )),\n", " tfpl.DistributionLambda(lambda t: tfd.Bernoulli(logits=t))\n", "])" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model(np.ones((3, 12), dtype=np.float32)).sample(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Its input layer has 8 nodes and passes through the distributionLambda layer. So its event_shape has 8. And input data is entered as 3 batches. We generate 5 samples. As a result, the output tensor shape will be `(5, 3, 8)`." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense (Dense) (None, 8) 104 \n", "_________________________________________________________________\n", "distribution_lambda (Distrib multiple 0 \n", "=================================================================\n", "Total params: 104\n", "Trainable params: 104\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tutorial " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a probabilistic model using the `DistributionLambda` layer\n", "\n", "Create a model whose first layer represents:\n", "\n", "$$\n", "y = \\text{sigmoid}(x) = \\frac{1}{1 + \\exp(-x)}.\n", "$$" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "# Create a sigmoid model, first deterministic, then probabilistic\n", "\n", "model = Sequential([\n", " Dense(units=1, input_shape=(1, ), activation='sigmoid',\n", " kernel_initializer=tf.constant_initializer(1),\n", " bias_initializer=tf.constant_initializer(0))\n", "], name='deterministic_sigmoid_model')" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"deterministic_sigmoid_model\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense_4 (Dense) (None, 1) 2 \n", "=================================================================\n", "Total params: 2\n", "Trainable params: 2\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAF1CAYAAADMXG9eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABB2klEQVR4nO3dd3jV5f3/8ec7OUlOdggJCRDCkr1EIqKAo6iAClq1rq+obS0d2mqtdmiHteNXa2utiq2j1lVXbbXUUUedUVEBBWQTEnZCyD5JTub9+yMnaQgBAhkn4/W4Lq4r+ZzPOZ/75Ch5cd/v+/0x5xwiIiIicnRCgj0AERERkZ5MYUpERESkHRSmRERERNpBYUpERESkHRSmRERERNpBYUpERESkHRSmROQAZvZnM/tJR5/b4nnpZuYzs9AjH+EhX3etmZ3aka/Z3uua2almtvMQz51pZpsDP4/zOmmIrV13tplt7KrrifRWpj5TIp3LzHKAFKAWqAPWAY8BDzjn6tvw/FOBJ5xzaZ03ys4X+Dlc7Zx7o52vEw78P+BiIAHYB7zgnLu+nUPsNIf7DM3sv8BS59wfO3kcDhjlnNvSmdcR6Ws0MyXSNRY452KBocBvgB8Af+mKC5uZpyuu04V+BGQA04FY4FRgZTAH1AGGAmuDPQgROToKUyJdyDlX4pxbSsOsypVmNhHAzCLM7Hdmtt3M8gJLZ5FmFg28AgwKLAH5zGyQmYWY2Q/NLMvMCszsWTNLDLzWMDNzZvZVM9sOvGlmV5nZ+2b2BzMrNrOtZnZS4PgOM9trZlc2jtPMHjGzXwa+PtXMdprZ9wLn7TGzLx/k3CQzezFwjUIzey8w1seBdODfgffw/Wbj9ASem2hmfzWz3WZWZGYvHOTHeDzwvHNut2uQ45x7rNl4cszs9MDXkWb2aOD11geuu7PFuTeZ2WozKzezv5hZipm9YmZlZvaGmfVrdv7CwHJesZm9bWbjDnHdRwLXXRcYc6vMLAsY0exnE9H8tQLn3GpmT7T4fK8M/Peyz8xuaXZuqJndHPhvo8zMVpjZEDN7N3DKqsB1Lm65/Ghm4wLvqzjwPhe2+JyXmNlLgdf9yMxGHux9ifQlClMiQeCc+xjYCcwOHPoNMBo4FjgGGAz81DlXDswHdjvnYgJ/dgPfBs4DTgEGAUXAkhaXOQUYB8wNfH8CsBroDzwJPE3DL/ljgMuBe80s5iBDTgXiA+P6KrCkecho5nuB95VMw9LmzQ1v1y0CttMwQxfjnPttK899HIgCJgADgD8cZCzLgBvM7FtmNsnM7CDnAfwMGEZDWDkj8D5buiDw2GhgAQ3h9ebAewgBvgNgZqOBp4DrA4+9TEMACj/IdUcG/swFrmzlHACccyPZ/2dTdYj309wsYAwwB/hps2B3A3ApcBYQB3wFqHDOnRx4fErgOs80fzEzCwP+DbxGw8//28DfzGxMs9MuAX4O9AO2AL9q41hFejWFKZHg2Q0kBsLAYuC7zrlC51wZ8GsafnEdzDeAW5xzOwO/fG8FLrT9l/Rudc6VO+cqA99nO+f+6pyrA54BhgC3OeeqnHOvAdU0BKvW1ATOrXHOvQz4aPhF3tp5A4GhgXPfc20ozDSzgTSExm8454oCz33nIKf/P+B24P+A5cCu5rNqLVwE/DrwmjuBu1s55x7nXJ5zbhfwHvCRc+5T55wfeB6YGjjvYuAl59zrzrka4HdAJHDSQa77q8DnueMg122vnzvnKp1zq4BVwJTA8auBHzvnNgZm7lY55wra8HozgBjgN865aufcm8CLNASzRs875z52ztUCf6Mh/Iv0eQpTIsEzGCikYZYjClgRWF4pBv4TOH4wQ4Hnm52/nobi9pRm5+xo8Zy8Zl9XAjjnWh472MxUQeAXaKOKg5x7Bw0zFq8FlhJ/eIj30NwQoNA5V3S4E51zdc65Jc65mTQUoP8KeLj5klszg9j/59DyZwIH/lwO9jMZBGxrNo76wOsNbsN1t7VyTnvlNvu6+ecxBMg6itcbBOxosSliG/u/v4NdU6RPU5gSCQIzO56GX1KZNOxGqwQmOOcSAn/inXONv6ham9nZAcxvdn6Cc84bmF3hEM/rVM65Mufc95xzI4CFNCzHzWnDeHbQMEuXcITXq3TOLaFhmXN8K6fsAZrvoBtyJK/fwm4aQiwAgRnFIcCuVs7d0+Ja6Ud4rXIaAnaj1CN47g4alheP1G5giJk1/72QTuvvT0SaUZgS6UJmFmdm59BQr/SEc25NYCbgQeAPZjYgcN5gM2usdcoD+ptZfLOX+jPwKzMbGjg/2czO7bp30jozO8fMjgkEjRIaZssaZzryaKhdOoBzbg8NtUr3mVk/Mwszs5NbO9fMrg8UTkeamSewxBcLfNrK6c8CPwq85mDg2na8vWeBs81sTqC+6HtAFfDBYa6bRkP90ZH4DLgk8HPIAC48guc+BPzCzEZZg8lm1j/w2EE/A+AjGmabvh+47qk01JA9fYRjF+lzFKZEusa/zayMhlmDW4A7gS83e/wHNCyPLTOzUuANAjVJzrkNNBQ+bw0s6w0C/ggspWE5rYyGouwTuurNHMIoGsbuAz4E7nPOvRV47P8BPw68hxtbee4iGmquNgB7aSj0bk0F8Hsalpz2AdcAFzjntrZy7m00FMRnB8b1HA0B6Ig55zbSUMB+T+C6C2goGq9u5fSf07BElk1DQffjR3i5n9Awu1QUeK0nj+C5d9IQ5l4DSmlowREZeOxW4NHAZ3BR8ycF3scCGmrX9gH3AVcE/vsTkUNQ004R6TPM7JvAJc65U4I9FhHpPTQzJSK9lpkNtIZbtYQEtvh/j4YdeiIiHaa3dUYWEWkuHLgfGA4U01D/c18wByQivY+W+URERETaQct8IiIiIu2gMCUiIiLSDkGrmUpKSnLDhg0L1uVFRERE2mzFihX7nHOt3pkiaGFq2LBhLF++PFiXFxEREWkzMzvobaG0zCciIiLSDgpTIiIiIu2gMCUiIiLSDgpTIiIiIu2gMCUiIiLSDgpTIiIiIu2gMCUiIiLSDocNU2b2sJntNbPPD/K4mdndZrbFzFab2XEdP0wRERGR7qktM1OPAPMO8fh8YFTgz2LgT+0floiIiEjPcNgO6M65d81s2CFOORd4zDnngGVmlmBmA51zezpqkCIiIiItZef7yMwqIK/ET0q8l1kj+zM8OabLx9ERt5MZDOxo9v3OwLEDwpSZLaZh9or09PQOuLSIiIj0BK0FH6BNx4Ynxxzw/GH9Inl/ayEJkWGkxnvx+Wt5ZvlOLs5I6/JA1aX35nPOPQA8AJCRkeG68toiIiLS8doSkloLPve/uxUzGJoYfchjzyzfycwRiXy4KY/+IXUMs1qqcnJZ+nouY+M9DIk0ylMHEzJgYNN1e2KY2gUMafZ9WuCYiIiI9ABHO2t02JAUG05VQRHPZK5hYqwxLMLhqSjHU1lByNY9RFT5mRAXisdfgcdfycjdBYRX+0n3Qqi/Eo+/EldZSU2ZjzGhRmiINY35xIoaPKFGdISHTV+6iuyzLyTG6yG3xN/lP7+OCFNLgWvN7GngBKBE9VIiIiLBdSQB6ZnlOw89a1RZw8OvfU50eSkjQmuYXF1OXVExn2/dzVRPDalUE+YrI6yinMI9+Xj95aRaDZ7KCgBOaxZ8Gg2trAEgJi6KuohIaiOjKKs2qsMjqI7pT21yKnURXmoivKzI9zNpZCr1Xi+13kjqwyNYta+KfXXGSePTqExOAcDnryUl3tuVP2agDWHKzJ4CTgWSzGwn8DMgDMA592fgZeAsYAtQAXy5swYrIiIiB2pLPVFrS2j/yNxMv4pijikvpX9lKeElRUSUFDFo626iy0sYaX7Cy0oILyvB52uY8Yn1/i86jKmoISQslIj+idTExFITHUNxbD8qk9OoG5ZKTXQMtVHRfF5SRx5hTJ84lNrIKGqionlnZzlV3khmjx/c9Hpvb9wLBqeOHtB0rLSyhs93l1A/KJ64yLCm4xX5PrbllZGWlkSM14OvsobiyhrmTkjpgp/4/tqym+/SwzzugGs6bEQiIiJyUG0KTu9lM7mfh0EVFXgL9pJeuI/Y9dnElRYyJqQSb+E+vMWFUFFBeVXtfiHFhXoIC4nEFxOPP30gpUNHUh0Tx8pShy86lokThlEdF09NdByZ+dXsrg1hwZS0pue3FogK8n1szitjwLBA8PHX4h0QSaQ1hKXGY4nR4ViLY8WVNVw8LY33txYCNB0PCQ1h8azh5BRVkhv4WcydkNJjd/OJiIhIOx1pIfegqBDqc7bx5gsbmUwZQyuLiNyXR+S+PGZv20VsdcV+y2ppVXWUxcQTOjQNX9ow9k2aRmVCIstKjeFjhhKaMoCqhH7URsXw9qb8AwLR8kBISm12rL/52JtXdthA1Frw+frJI5re36GONQaktMSoVo/P7ooP5zCsYWKp62VkZLjly5cH5doiIiLBdKjZpcYAklNQToirZywVDCjMJXTnDvat28xofwGpxfl4C/PBOUoC9UiRMVFUDEjF338AGy2anVH9GHvsaPyJyVT2T+a1vBpcaOgBS2iVNbVU1boDrt24JHioY8WVNcwckUhOUWWb2hv0ZGa2wjmX0dpjmpkSERHpRG2tZ5oe4WdkRT6xu7YRvXsHx67fRNK+PfQLrW96rb11oZSlDKZo1DgqUuZQMSCVZVWRbI2I59STxoM17HbLyfexPq+MkJH/W1brF1ve+hJaRsMS3eFmiA41a9Ta7FBPD09HQmFKRESkAxxsma7lTrm/vLmRmfWFjC3dTeyOHGJ3ZHPKhs1E11U1Lcv5E5PIi0xi69TRpB03gfLUwZQPHMI7+TXsq6hmweT/FW3X5/uozCuj1F97xMtqzWuMWgs/bT3W1ylMiYiItFN2vu+A0PTM8p1EUcvwvJ0M3rWVuJwtxOds4YwtWwgziI7wUBsVTdmQ4WzMOJmsfgMZO/M4fIPTqYuMarWQe2C1j2J/7WHrkTRj1LUUpkRERI5Qy1mogjI/CZFhJFeW0G/1OhK2bCBq8wZCN28iMbBRrjo2ntJhx7Dq1PFsShzMhNNmUJk0AMyaluWiB/9vm39bC7m7UyF2X6UwJSIicggHrXnyehhZnk/UR6uJWPYJGUU5xBcXAFAfFk7JsGP46Pg5xGUcR8248fgTk5qC08a8MpJiEokBfJU1R7wsp+DUvShMiYiIBByuWLx+5y4+eeJ9Zu/bwoicDYSXFAGQFxrF5qGjiZn/RYpGj6dsyAhKahyVNbXsaNwpx8GDk5blejaFKREREVqve/rrG+s5zbeNSdvWkvT5p0Tm51JSUUNVfDwFx2VQOH4KhWMmkhPZj3c3F3DmhNTD7pTT7FLvozAlIiJ90sHqnlKL8xjw2cckrVnBnE8/I4I6vPGxFI6dxLYzF/J6TDobo5L26/rtraxh9ugkogM32m3LTjnpPRSmRESkz9lvFio2nLD164h+5XVOzV1LYv4eAHyD0/lk1lw+HTKOiWedivM0VJKH5/vwtOj63TgLpdDUNylMiYhIr3fALFRJBcfs2MQxqz5kwMplRBQXUlpdz7ZhY8m7/Fz2Tp2Ov/8AcvJ9bM8rY0gNxIS6bndPOOkeFKZERKRXa5qF8noYm59D4r/eYcI7bzK4xkeI10v+5Azypp3I+uETeWOX/391T0dRLC59k8KUiIj0Ki1nofzZ2zhx+XuM/PhtIvfl4TweVqaP491JJzLw7DOo80YCEFJZw+zo2lbrnhSc5FAUpkREpNdonIVKsloy1ixjwDtvEL7mM2K9YZRMPJYt513G3uNmsLs2hHc37eNM5yHGOdU9SbsoTImISI/Vchaqfu1avvDhfxm2/D1Cq/xUpAzizTMuYvWxM5l2/Lim5x1u953IkVCYEhGRHqlxFqp/SB0Zqz9k4OsvEbpxPVExUeTOOo2ds8+k+JixFJX5yd60j1HafSedRGFKRES6vZYzULNG9mf5x+uZ/cpSjvngDTyV5fgGp/Ofc67k8yknceKU4U3P9YZ5NAslnUphSkREurWWncnD1q9jzx9e4MSVH+ANCyXv+Jlsn3M2RaMnUFnmZ9emfeoBJV1KYUpERLq1zKyChpsKb1jJiJf+TsKWDVSFe/ngxLmUn3chYYMHNZ2rWSgJBoUpERHpVvZb0osNp+7V15j34UvE7sihMimFDf+3mO2zTufzwhpiI8NI0CyUBJnClIiIdBuNS3r9wo1pqzMZ+u9nYft2/IOHkLP4BvaccAouNJTSyhrGDopk1sj+rd5EWKQrKUyJiEjQtHabl4lrljHl5WeIyttNWfpwPvjaTfxnwDhOGjWAmJAQfJU1FFfWNAUnhScJNoUpEREJiv0Ky+MiiP3oA4Y//hdGleZSOWwkK6/7CfnHTqceGLK7VHVQ0m0pTImISFBkZhWQEBnG0O0bGf3sIyRkbWBbdH+eOu8bDPjiORASAoCvsoaxg+JYNGNokEcs0jqFKRER6RItl/T2rNnE2e/8g9QVH1DVrz9rv/xtPp08k3ezijizqo4YrzUVlc+dkBLs4YsclMKUiIh0uuZLemlhdQx54iFmvPICXm84m8+/nJy551Ef4SVCt3mRHkhhSkREOl1mVgEJEaGMX/YGo//xOGHlZaydcQpPH38Ok48dRUy4p6mwXK0NpKdRmBIRkQ7Xcklv34rVLHz5URK2bqJozEQ2XHo1xUNH0k+F5dILKEyJiEiHar6kNzi8nqGP/ZmTXllKSEI8axbfwO4TTwMzFZZLr6EwJSIiHapxl96otR8z7on7iSgpYvXJZ/Lk8QuYNnEoMbBfryiRnk5hSkREOlTJzlxO+9dfSV3+AWXpI/j0Oz+maPgoBmhJT3ophSkREWmXpvqo4kqO3fAxCx/9E+HVfjZdeCU588/HhYZqSU96NYUpERE5ao31USnVPs565gGSVnzIjkEj+O9FXydu3JgDbv8i0hspTImIyFHLzCpg/IblTP/bfYRWV7H50q+y+uSzCaur15Ke9BkKUyIi0mbNWx4MinAMfGAJUz99h9Jhx7Dq6zdSMTCNGOfwlfi1pCd9hsKUiIi0SfOWB2MKdzDhT3fA9h0sn38+hZd/GecJA8DnryUl3hvk0Yp0HYUpERFpk8Yu5lP++wKj/vkE1fEJvHL9bbwdO4SZNRAT6nQvPemTFKZERKRNinbnc+bTS0hes4Lc42ex9qpriIiKIV0tD6SPU5gSEZFWNa+PGrNvG+fc82uiykpYd8U32XHaWepiLhKgMCUiIgdoqo/yepj+yRuMefJBCqPi+dfinxI5ZbK6mIs0ozAlIiIHyMwqIMlqOfHRe0n96F3ypxzPB4u+DeGRWtITaUFhSkREDuDbuo35j/yO2F3b2XzhFWw960IizKhTywORAyhMiYj0cc1ro1LivXyhYifn/eFmXE0NK67/KfsmZwANy3pqeSByIIUpEZE+rHnvqNR4L8mvvYT97X7Chg7h2cu+S8iwYQ1NONXyQOSgFKZERPqwzKwCEiLDiA8zxj5xP+n/fZHdE6by+TXfZ/7koWRmFag+SuQwFKZERPqwvBI/gyMcx/3xdpLWrCB7/gVsuPAKcsuqGZ4co/Ak0gYKUyIifVh6fTlTf3EribnbWfvlb7PzlLmqjRI5QgpTIiJ9SPNi85G+vZx+z22U7yvi3W/eQnnGdPWOEjkKClMiIn1E82LzCbs2MuXuX1HsCafwjrvx9RvctJtPtVEiR0ZhSkSkj2gsNh+9+kMmP3An5amDeO9bPyak30D1jhJph5C2nGRm88xso5ltMbMftvJ4upm9ZWafmtlqMzur44cqIiLtkVfiZ9yHrzPlz3dQfMxYPrr5t4QOGkheiT/YQxPp0Q47M2VmocAS4AxgJ/CJmS11zq1rdtqPgWedc38ys/HAy8CwThiviIgcpRMz/82oZx8lf+p0Prvmh9SHR6jYXKQDtGWZbzqwxTm3FcDMngbOBZqHKQfEBb6OB3Z35CBFROTINRWbF1dyymtPM/m1F1g9dSaff+27RIeFq9hcpIO0JUwNBnY0+34ncEKLc24FXjOzbwPRwOkdMjoRETkqTcXmEaGc/sJfGPTWf1g+80w8N91EVEmVGnGKdKCOKkC/FHjEOfd7MzsReNzMJjrn6pufZGaLgcUA6enpHXRpERFpKTOrgH7hIcx87I8M/Ohdti68mI1nXUJ0SZWKzUU6WFsK0HcBQ5p9nxY41txXgWcBnHMfAl4gqeULOececM5lOOcykpOTj27EIiJyWHsLy5uC1KYLr2TzhVcQExmmYnORTtCWMPUJMMrMhptZOHAJsLTFOduBOQBmNo6GMJXfkQMVEZE2qq1l7tP3MGDZu2y66Mtkn/MlAHz+WhWbi3SCwy7zOedqzexa4FUgFHjYObfWzG4DljvnlgLfAx40s+/SUIx+lXPOdebARUTkfxqLzfcW+pj35D0M/WwZb517BbtPW0iMc/j8tSo2F+kkbaqZcs69TEO7g+bHftrs63XAzI4dmoiItEVjsXliGMx/4o8krfiQt869kn5fvYKSokoVm4t0MnVAFxHp4RqKzY1Zf72TlE+XsX7RN9g9ax4lRZUqNhfpAgpTIiI9XF5RBXOfvY+UFR+y4f8Ws/2MBcQ4R66KzUW6RJtuJyMiIt2Uc5z+wl9I+eBtNl14JdvOWAio2FykK2lmSkSkh2ne2fzUl59gzPtv8MHpXyR7znkqNhcJAs1MiYj0II3F5uX+Wk7+73Okv/ovPj5xHlHfvoZor4fcEj/RXg8XZ6Sp2Fyki2hmSkSkB8nMKiAhMowpb/6LkS8+y87T5rH54quJLvar2FwkSBSmRER6kLwSPxmr3mX03x9hzwkns/bKa4gxU7G5SBBpmU9EpAeZtHUV4/96L/smHsear30XQkJUbC4SZApTIiI9xapVnPLw79k7aBjvffVG6kI9lFbWUFxZw6yR/YM9OpE+S8t8IiLdWOPOPf+GzSy45yfEpqYQe9e9RBTUq7O5SDehMCUi0k017twbWFnMWQ/8iurQMP566Y2cndCPRaMUnkS6Cy3ziYh0U5lZBSTX+zltyS/w+P2suukXeNIGk5lVEOyhiUgzClMiIt1UfkEZsx+6g6i8Pay87seUpQ8nxushTzv3RLoVhSkRke7IOeY8/xAJ61bz+Ve/Q9HYSYBuEyPSHSlMiYh0R3/5C2OXv8uKeV9iw9TZ1DunnXsi3ZQK0EVEuonGnXtRb7zGqX+7h5BzzmHsTd9l39ZC7dwT6cYUpkREuoHGnXvH7NjIyc/8idxRE3np5Ev5kpluEyPSzWmZT0SkG8jMKmBwWT6zHridyuRU1l5/C3GxUdq5J9IDKEyJiHQDhbkFnPzn34AZK757K7XRsdq5J9JDKEyJiARbfT1nPnUP3j27+PTam6kckApo555IT6EwJSISbPfey4iNq/jg/K+wbehY7dwT6WFUgC4iEgSNO/eiX/sPpzz1EHbhhRz7na+QmVWgnXsiPYzClIhIF2vcuTdidxaz/34/uaMm8uIJ53MRaOeeSA+kZT4RkS6WmVVAqr+U2Q/cjj8xmXXX3Ux8TKR27on0UApTIiJdLH9fKbMf+i2h1VWsvO4n1MTEaeeeSA+mMCUi0sVOe/FxYrdsZM3V11M+OB3Qzj2RnkxhSkSkK/3rX4xf9gafzTmPzROma+eeSC+gAnQRkU7WuHOvdtUazrn350Qcn8Gon91EXnaRdu6J9AIKUyIinahx515KTTlnPvp7yuP68cRZX+O8kBDt3BPpJbTMJyLSiTKzCugXHsKsv95JuK+UNdf9mMikRO3cE+lFFKZERDpRXomf4156ksQNa1h71bWUDR2pnXsivYzClIhIJ5q0dRVDX/oHO06bz+6ZcwDt3BPpbRSmREQ6y+7dnPy3e9k7eDgff/Eq7dwT6aVUgC4i0hmqq+H73ycyLJS4P/6eyPJw7dwT6aUUpkREOkhjC4S8Ej9zXvgL4z9fR8Tdd5E+aRSLgj04Eek0WuYTEekAjS0Qyv21TF23jMFvvsK7M+aTPe64YA9NRDqZwpSISAfIzCogITKMgUV7mPToEkrHTmTrl65UCwSRPkBhSkSkA+SV+IkLqePYJbdTFx7Bqm9+n+joCLVAEOkDFKZERDpASryXkY/dT8yubaxZfANV/fqrBYJIH6ECdBGRDnD6jlXwzqusn38Beycehy/QAmHuhJRgD01EOpnClIhIe+3cycB7f0/Z8VPJuegKtUAQ6WMUpkREjkJjG4T8gjLOvfdnDKpzxP7+t1w+cGCwhyYiXUw1UyIiR6h5G4RTXnuG2K2beH7B1WR7YoM9NBEJAoUpEZEj1NgGYeTGTxn22r/YdcYCSk+YqTYIIn2UlvlERI5QXomfoa6CSQ/dRVn6cDZe8hViPB5y1QZBpE/SzJSIyBFKiQ1n3J9/T2iVn1Xf+D71YeFqgyDShylMiYgcobmf/Zf+az9jxZe+StnANEoDbRBmjewf7KGJSBBomU9E5Ehs2MCAxx6ieO7p5J8+nzy1QRDp8xSmREQOo7ENQkFeEef/4YckxcST8OvbWBQXF+yhiUg3oGU+EZFDaN4G4bSXHsebu5tnzl1MdpX++hSRBvrbQETkEBrbIIz6/CPS3nudbQsuomrSsWqDICJN2hSmzGyemW00sy1m9sODnHORma0zs7Vm9mTHDlNEJDjySvz0ryhmwiP3UjJiNFvOu4wYr4c8tUEQkYDD1kyZWSiwBDgD2Al8YmZLnXPrmp0zCvgRMNM5V2RmAzprwCIiXSklNpzxt99JSG0tqxd/D+fx4KusURsEEWnSlpmp6cAW59xW51w18DRwbotzvgYscc4VATjn9nbsMEVEgmPu6rdIXLeKlRd+BV/KILVBEJEDtCVMDQZ2NPt+Z+BYc6OB0Wb2vpktM7N5HTVAEZGg2bKFAY8+SNTpp5E/Zx65JX6ivR4uzkhTGwQRadJRrRE8wCjgVCANeNfMJjnnipufZGaLgcUA6enpHXRpEZGO09gGIX9fKef/8WZSIqKI//VtLOrXL9hDE5Fuqi0zU7uAIc2+Twsca24nsNQ5V+OcywY20RCu9uOce8A5l+Gcy0hOTj7aMYuIdIrmbRBOfv1Zorbn8NyCq8muDQv20ESkG2tLmPoEGGVmw80sHLgEWNrinBdomJXCzJJoWPbb2nHDFBHpfI1tEIZvXcvw1/7FrjMWUD5tutogiMghHTZMOedqgWuBV4H1wLPOubVmdpuZLQyc9ipQYGbrgLeAm5xz+ttHRHqUvBI//eoqmfTQHygfmMbGi65SGwQROaw21Uw5514GXm5x7KfNvnbADYE/IiI9Ukq8l2OW3E14STErv3ML9RFetUEQkcNSB3QRkYDTd6xi0LJ3WDv/QoqHjVIbBBFpE93oWEQEID+fgX+6i7LjJrPji5eSW+InJd7L3AkpaoMgIoekMCUi4hzcdhtUVRF7x2+4XK1bROQIKEyJSJ/V2FMq/sUXmPnGO9R970YGKEiJyBFSzZSI9EmNPaVs23Zm/PsJ9oyZzMMDM8jO9wV7aCLSwyhMiUiflJlVQL/wEE58/B6cx8PGr99AQnSEekqJyBFTmBKRPimvxM+kt/5FQtYG1i36JlWJSeopJSJHRWFKRPqkY0p2M/KffyMvYyZ7ZpwCgM9fq55SInLEFKZEpO+pqeH0p++jwhvNsosXUw/qKSUiR027+USk73nwQWK2Z1N26y8JS0pUTykRaReFKRHpExrbINStWs059z5A6DlnM3DhPBYFe2Ai0uNpmU9Eer3GNgj+0nK+8NQSyuP68fCM89UGQUQ6hMKUiPR6mVkFJESGMe3FJ4nO282Gxd8lOjFBbRBEpENomU9Eer28Ej/jd29i6OtL2T7nHAomHEuMc+SqDYKIdADNTIlIrzcowjH+obuoSBnEpi9dBagNgoh0HIUpEen15r3+FBH5e1l2+bXURESoDYKIdCiFKRHp3T78kMRXXyL8qivwT5xEbomfaK+HizPS1AZBRDqEaqZEpPcqK4Nf/AJGjKD/jdexKDw82CMSkV5IYUpEep3GnlIj7/0to3N243/gIdIUpESkk2iZT0R6lcaeUnEffcCYFe+xbt4FPFEWo55SItJpFKZEpFfJzCpgQF0l05/8E2Xpw9l1wWUkRIapp5SIdBqFKRHpVfJK/Ex/9kHCyn2sufoGnCeMGK+HPPWUEpFOojAlIr3KsRs/YcBH77HlvEspSx8OqKeUiHQuhSkR6T0KC5n53EPsTRvBqtPOpd459ZQSkU6n3Xwi0js4B7/+NZG11cTf+SuiaiPILfGTEu9l7oQU9ZQSkU6jMCUivcMrr8Dbb8N11zEkYyKLgj0eEekzFKZEpEfLzvex/JONnPCjn1OdPpLQM89leLAHJSJ9imqmRKTHys738cwnOxj70N14XR3LLr+WZ1buVk8pEelSClMi0mNlZhVw7Mp3GLT+UzZd9GVChg1VTykR6XIKUyLSY/m2bmfqP/5K4bjJbJ9zNoB6SolIl1OYEpGeqb6eLzx3P3X1js+/ch2ENPx1pp5SItLVFKZEpGd67jmGbV3HsoWLyItJVE8pEQkahSkR6Xl27IC77ybi5FlM/fZVRHs95Jb4ifZ6uDgjTT2lRKRLqTWCiPQI2fk+MrMKyCuq4Lz7bmUwIUT++McMHxDL8AGxwR6eiPRhmpkSkW4vO9/HM8t3Uu6v5cSPXiVu0zr+feb/kW1RwR6aiIjClIh0f5lZBSREhjGwaA+j//k4BdNmUDh7jlogiEi3oDAlIt1eXomf2DBj8oN/oC7Cy9qrvk1MZJhaIIhIt6AwJSLdXkq8l8EvPE1c9mbWXnkN1fEJaoEgIt2GCtBFpNs7rX4fIUufIWf6bPZkzMQXaIEwd0JKsIcmIqKZKRHp5qqrSbvrdhIGD2DzVd9UCwQR6XY0MyUi3duf/wxbtxJ1991cctLEYI9GROQAClMi0u009pSq//Qzzv7zXwj54nkknnRSsIclItIqLfOJSLfS2FPKX+LjC08vwdcviYczziM73xfsoYmItEphSkS6lcaeUscvfZyo/DzWL76BmH5x6iklIt2WlvlEpFvJK/Ezcdtahrz5MjnzvkjR2EnEOEeuekqJSDelmSkR6VbSQqsZ/9Bd+Aans/mCRQDqKSUi3ZrClIh0K/NefJTQ0hI+uOI6aj1hlAZ6Ss0a2T/YQxMRaZWW+USk+3j1VRIy36bmm1+ndvRo8kr8pMR7mTshRT2lRKTbUpgSkaBqbINQtm0X5//2VqLHjCP52q+zKDQ02EMTEWkTLfOJSNA0tkEor6hmzrN/xlXX8Ni8r5BdWBnsoYmItJnClIgETWMbhIkfvk7Sus/YctnVhKanqw2CiPQobQpTZjbPzDaa2RYz++EhzrvAzJyZZXTcEEWkt8or8ZNasJsxz/yFfZOmseO0+cR4PeSpDYKI9CCHDVNmFgosAeYD44FLzWx8K+fFAtcBH3X0IEWkd0qNDmXCfXdQFxHJmquvBzO1QRCRHqctM1PTgS3Oua3OuWrgaeDcVs77BXA7oH9SikibzH/vBWK2b+Wj//sW/rgEtUEQkR6pLWFqMLCj2fc7A8eamNlxwBDn3EsdODYR6c1WrqT/P58h4ksXUDpjJrklfqK9Hi7OSFMbBBHpUdrdGsHMQoA7gavacO5iYDFAenp6ey8tIj1MYxuEwj37uOB336dfciqJP/4Bi6Kigj00EZGj1paZqV3AkGbfpwWONYoFJgJvm1kOMANY2loRunPuAedchnMuIzk5+ehHLSI9TlMbBH8tX1j6KOGF+3hqwdfILq8P9tBERNqlLWHqE2CUmQ03s3DgEmBp44POuRLnXJJzbphzbhiwDFjonFveKSMWkR6psQ3C6NUfMmjZ22Sfdxm14yaoDYKI9HiHDVPOuVrgWuBVYD3wrHNurZndZmYLO3uAItI75JX4SfYVMuGRJRSPHMvWBRepDYKI9Aptqplyzr0MvNzi2E8Pcu6p7R+WiPQ2qTFhjP/VbzFXz+qv34gLDcVXWaM2CCLS46kDuoh0iXnLXiRh83o+ueTrlCenqA2CiPQaClMi0vlWrSLpmSfwLjyHwlPmqA2CiPQq7W6NICLSmv3aIPz+B/Trn0y/237CoujoYA9NRKRDaWZKRDpcUxuEyhrmvPAw4QX5PLXw62RXuGAPTUSkwylMiUiHa2yDMHblewz8+D22XnC52iCISK+lZT4R6XB5JX5GVuxj/ON/omjMRLae/SVizMhVGwQR6YU0MyUiHW5gVAgT772d+rAwVn39RggJweevVRsEEemVFKZEpMPNf+0pYrZvZdkV36GyX3+1QRCRXk1hSkQ61ptvkvjKUsKvXETZ8TPUBkFEej3VTIlIuzW2QfBt3cYXf3cz0aNG0/8HN7AoLCzYQxMR6XSamRKRdmlsg1BR7ufMv91Nfb3j0QXfILu4KthDExHpEgpTItIujW0Qpr30NAlbN7H+6uvwpA1WGwQR6TO0zCci7ZJX4mdS9mqGv/IPdpw2n7zjZxHjnNogiEifoZkpEWmXYdUlTHzgTsrSR7Dhsq8BqA2CiPQpClMicvSqq5n7+F3U1dbx3tU3UesJUxsEEelztMwnIkfvrruI3boJ322/gNQ0ckv8pMR7mTshRW0QRKTPUJgSkSPS2AbB++YbfOHxJ+Dyyxl43nwWBXtgIiJBomU+EWmzxjYIITk5nPLs/ewdMZaHppxFdr4v2EMTEQkahSkRabPMrAL6h9Qx66E7qAuPYN21PyQ+JlJtEESkT1OYEpE2yyuu5MQn7yNmz05Wf+MmqhKTiPF6yFMbBBHpwxSmRKTNZiz7DwOWvcumC66gYMKxgNogiIgoTIlI23zyCccvfYKtk6az6rSF1DunNggiIihMiUhb5ObCj35ExDEjSP39/yM6MozcEj/RXg8XZ6SpDYKI9GlqjSAiB5Wd7+ODDXs49tYbSSoso+qOexg2dADDhgZ7ZCIi3YdmpkSkVdn5Pp75ZAejH15C6u5sll11PU/mmtogiIi0oDAlIq3KzCpg2kevM2LZm2QtvITyGTNJiAxTGwQRkRYUpkSkVfbxx0z9+8PkHzudLeddBqA2CCIirVCYEpEDbd/O6Y/eSXHKYFZ94yYIafirQm0QREQOpDAlIvvz+eCGG4iPjuA/X/0BRc6jNggiIoeg3Xwi0nTz4ryiCs75628Zlr2NyAfv5+who8nMKiC3xE9KvJe5E1LUBkFEpAWFKZE+rvHmxQmRYZz66lMkrlrOy1/6GlOHjGZ4cozCk4jIYWiZT6SPy8wqICEyjHGfvM3wV19g1xkL2Hv62dq1JyLSRpqZEunj8kr8TNi5gQmP3EvBhGPZcOnVxISGkqtdeyIibaKZKZE+bqRvL1Pu/hXlA9P47Jof4Twe7doTETkCClMifVlBAXPv/xV+Tzhvf+sWqiOjtGtPROQIaZlPpI9p3LlXsLeYBX+6jcElJYTffQ8hocnatScichQUpkT6kKadexGhnPn0vcRmb+YfX7mJk0aPZZHCk4jIUdEyn0gfkplVQILXwwnP/5WUTz9i8+WLKZt+knbuiYi0g2amRPqQvBI/J723lPQ3XmTbmeey7YyFxDinnXsiIu2gmSmRPmTaqnc55u+PsefEU9lwyVcB3W9PRKS9NDMl0os13SamxM+kras46Yn72DRmCisu+xZRZvgCO/fmTkgJ9lBFRHoshSmRXqr5bWLG7t3KlCW/YVtKOv7bfkmk37RzT0SkgyhMifRSjbeJGVi4h+Pvuo3qpAF8fP3PCPMbi2YMDfbwRER6DdVMifRSeSV+BpTs5fg7fkxdeATLb/wF4cn9yVOxuYhIh1KYEumlhtaWMu03txBSW8MnN/2SyuQUFZuLiHQChSmR3qiggLPu+wVW7uPtb/+MskFDdJsYEZFOopopkV6icede0e58Fi65ldTyQkLuvpsa70DyVGwuItJpFKZEeoHGnXtJVHPOg7/Gm7ebp792M6dMmKjbxIiIdDIt84n0AplZBSRRw2n3/ZK4HTms/s4t+KdM1W1iRES6gGamRHqBwtwCznnw18TnZPHZNT9g35TjdZsYEZEuojAl0tOVlXHO/b8iZlsWn337ZvYeNwPQbWJERLpKm5b5zGyemW00sy1m9sNWHr/BzNaZ2Woz+6+ZqSOgSCfKzvfx+LJt3P3PFWRddBXJe7bxxlXfY8u4adQ7p517IiJd6LBhysxCgSXAfGA8cKmZjW9x2qdAhnNuMvAc8NuOHqiINGgsNq8uKGLBA78kevtWll7+XdLPnUu010NuiZ9or4eLM9K0c09EpAu0ZZlvOrDFObcVwMyeBs4F1jWe4Jx7q9n5y4DLO3KQIvI/mVkFpNSUc9qS24jes5NPr/8JvlFTyCmq1G1iRESCoC1hajCwo9n3O4ETDnH+V4FX2jMoETk439btnP3Ar/AWFbDyup9QMPE4FZuLiARRhxagm9nlQAZwykEeXwwsBkhPT+/IS4v0DTk5LLz3p1BZyfKbfknxqHGAis1FRIKpLWFqFzCk2fdpgWP7MbPTgVuAU5xzVa29kHPuAeABgIyMDHfEoxXpgxo7m9esWcv8B35NVLiHZ6+5FZd2DDHO4fPXUlxZw9wJKcEeqohIn9SW3XyfAKPMbLiZhQOXAEubn2BmU4H7gYXOub0dP0yRvqmx2Dxy9Wecfd/PqQoL56nFP2X87ONUbC4i0k0cdmbKOVdrZtcCrwKhwMPOubVmdhuw3Dm3FLgDiAH+bmYA251zCztx3CJ9QmZWARPWfcyMR/9IZXIqK278BSGR8So2FxHpRtpUM+Wcexl4ucWxnzb7+vQOHpeIOEfiP59lxr8fp/iYsay87ifUxMar2FxEpJtRB3SRbqSxPiqvqILTXnqMjNf+zfZpM9n4rRupD48AVGwuItLdKEyJdBON9VH9Q+s5+7E76b/yIz6aeRYfnn0Z6XUhKjYXEemmFKZEuonMrAJSq8s4+YHbic/ezPpF32DXrHkk19Q2FZunxHuZOyFFxeYiIt2IwpRIN1G7+nPOfOR3hJX7+PTam9k77cSG2aiqWhWbi4h0YwpTIkHSVB9V4mfqumXMfeguquP7sfLHd1CWPgJQfZSISE+gMCUSBI31UQkRoZzy2tMMffE5stPH8J9F15OSPFD1USIiPYjClEgQZGYVkOyqmPngPSR/9jE7vjCfVRd8hThnqo8SEelhFKZEgqD683Wc+didRBbsY92ib7JjztlEOUdpiV/1USIiPYzClEgna14blRIXwZmbPmTBH3+HPzqWj3/0G92sWESkh1OYEulETbVRkWEM8sLo+++k/v03cdOn849zv07kgCTVR4mI9HAKUyKdKDOrgITIMAYW7eHYJbcTs3s7ny+8lO0XXMZ5o5LJzCpQfZSISA+nMCXSifKKK5m+8i3GPf0QdRGRLP/ez8mfMJW8Ej/Dk2MUnkREegGFKZEO1Lw+aohV8YUH7yTt8xUUTp7Gmq99l+r4fvgqa1QbJSLSiyhMiXSQ5vVRE7etZcKDd1JfWsabZ11O8YLziYkKx1dZo9ooEZFeRmFKpINkZhWQZLVM+/ujpL/5Er7B6Xxw3c/JTUkjKSpctVEiIr2UwpTIUdqv5UG8F9+777Ng6cNEFuaz7YyFbPrSldSFhVOn3lEiIr2awpTIUWi+pDc4vJ4RD91N8hsvUz4ojdU3307xqPEAqo8SEekDFKZEjkJjy4MRm1cx4ZF78RYXsvLM83jyuLM4IW2QekeJiPQhClMibdBySW/n+mwW/vdpUld8QPmgISy7+bcUjRzDoN2lureeiEgfozAlchjNl/QGRntI/fdzTH/2caI9xuYLFpE9/3ycJwxfZQ1jB8WpPkpEpI9RmBI5jMYlvfQdm5jw2J+I2ZlD1vgp3HvyxUyYNo6YUI9aHoiI9GEKUyIttFzS2712C2e/8xwDP3kff2ISn377FnKnnkDcnjIt6YmIiMKUSHMtd+kNefJhTnj5eSIjPGw57zJy5p1PnderJT0REWmiMCXSTGZWAf3CjXHL3mDU838jvLSYdcfP4unpC5g0dTQxEVrSExGR/SlMSZ+235JebDj1r7zCWe++QNTeXIpGjWfFd39K8bBRJGiXnoiIHITClPRZTUt6Xg9Tsj5l+N8fJyR7K6XDRrDh+p+RPyUDzLSkJyIih6QwJX1Gy8LygtJKxm36lMmvPkdc9mYqUgaR+ZUbeGXgRE4aNYAY0JKeiIgclsKU9AnNC8tTY8NJeO8thj/zBKNKc6lJGcjnX/kOu2fOoS4khCFa0hMRkSOgMCV9QmZWAYkex5hP3mbES38nKm83OTHJPLPwaySfvxAXGgqgJT0RETliClPSKzVf0ksLqSLmheeZs+K/RJQUU5Y+nM+u+SFrRk3l3S2FnFldT4w3RPfSExGRo6IwJb1O45LekOJc5rzzMgMz36DCV8muydMo/NoNFEyYCmZ4K2uYPTpJS3oiItIuClPSo7UsKp+VHseW515mwX9fJnXTGurDwtg1+3Q+mDaHDy2BmSOT9issvzgjTeFJRETaRWFKeqzmReXDq0sY8PTT8O5rjC8txqUOZPMFi9hxyjxq4uKJc450FZaLiEgnUJiSHqPlLFRxQQmTPl/OqI/fov/61WDGzvHH8caE2USfPIu46Iim5/r8tSosFxGRTqEwJd1Sy+A0rF8k728tJMHrYVzuFpKfeYPod98iKaSWmpSBZC28hJ0nn0FFYjKFu0upqaqDkBpivB4VlouISKdSmJJuZ7+eUPENNxX+9+sfcsaOVYxf9zHegnzqIrwsm3QCz0+dxYgzZkNICPC/1gazRvYnM6tAS3oiItLpFKYk6A7oTF7mJ8HrYXD+DlI/ziT14/eoydlOSFgovmnT2XzBIvKOO5E91bBy0z6SquqI8dp+M1DDk2MUnkREpEsoTEmXOujyXWQYA2PCiFj1GQmvvs7Je9YSX1wAISEUjJ3E29PmsmzYZE4/YXTTa3mdWhuIiEjwKUxJlzlg+c5fy2OvrmFO8RYmbFtH8qpPCCv3UVwLWSMnEX7+5eRPOZ7q+AT25fuoySujtHL/Oii1NhARkWBTmJJO09ryXb/wENJ2byVpzUqS1qzgjM8/JywEwvsnkj/lePYeN4P16eN4c3s5Z05IbQhOlTWEhIaweNZwcooqNQslIiLdisKUdIhD7b47piyPmMyVxH+wjOPys4isqQIzSoaP4qPTz2dl2limzp3VVETuqaxh9ujIVpfvZgf5fYqIiLSkMCVH7HB1T54N61n9/secUpjDsJ1bCC8tBmBHVCLLxxxP3OyTKBg3hZq4eHLyfeTmlVHaoohcy3ciItJTKEzJQR1wq5aR/QH2q3uq3lfIW89/zPG+3YzMyyYhayOh/kpKKmooTxrAvmOnUjh2MgXjJ7M9PI53N+3T8p2IiPQqClMCHHq2qbFY/J/vbSQ1dxvH78ohZXc28Vs3EbV3DyUVNYR6QnHHjGTXSV+gaMwEXg9LJccTzYLJg5uucagbC2v5TkREeiqFqT7osMGpsoanPljLjLpCRpXmEbcjm7icLUTs2UV5VS1xkWFU9etP8Ygx7Dx1Hu+GD2BtwmDmZYxoukZsvg+Pdt+JiEgfoDDVixxsWe6gwSkugtq8vbz+78+ZWlfE8LK9xOzaTuyObM7cV4wn1IiO8FCZlELp0JHsnDmHN1wCaTOOIyIluem6dfk+XIvgpOU7ERHpK8w5F5QLZ2RkuOXLlwfl2r3BoWaXGgNNTkE5ZjAyKoTk0nxCt29n9+ebGVddyKDSfcTs2YGnopySiho8oUZEQjy+wUMoGzKcFWH92RSfyrTTjqc2KhqA0soaKmtqqap1+12nuLKGmSMSySmq3C/IKTiJiEhvYWYrnHMZrT2mmalu5ohnl+K9lJf7eSZzDcd5KhhZ6yMyP5eo/Fwmbc6mX3E+KbUVTa9fUlFDZWJ/6kYMZ8+MU/ANSmdZfQybopM59aTxYAZAXr6P3XllDLdwYpzbb5mucTyqexIREVGY6nBtCUOHOtayQ/iDb20iqryEUfiZUl6M5eWRs3k7s+t9pPpL8Bbk4y0u5AxfVdOyHCEh+PslsTcsnvXHTKFk4mgqBgykInUwb5aHk1vDfoXh1fk+KvPKKPXXtnmZTrNOIiIiDbTM14qjDUSHWmobmhjddGxbfhmRFWWMDKulX5WP+n0F1O0rIK68mJjyMuIqSokoLiCiuIiq/H3gINb7v9xbUO2oTEwiKm0Q/v7JVCal8GmNlx0RcUw7cSL+xCScJ4y3N+4Fg1NHD2h6bk6+j/V5ZcwcmaRlOhERkTY61DJfrwxTrYWh4ckxR7yEdqhAlFNQjuEYGRVCQr2fmsIS1m3cydioeoZ46ggr9xFWVsqubXuIrPQxwlNLeHkZ4aXF+AuKDghIdfWOshqHNzWZ6rgEqvr1x5+QyKcVoZTFxDNu0kj8iUn4E5N4N6+afRXV+80utRaSWhu3gpOIiMiR61NhqvnNdFsGiMbbm8SF1uMv8bEnt5DwmiqGeo1YV01NiY9N2XmMjDZSwxwefwWeykp27dxLhL+SYZEQVlGOp6IcX0ExEVUVxIWHNl27sZA7OqIhJDmPhz1EUBEZTf9BA6iOiaM6Lp5VPsMXHcu4cUOpjo2nKr4f/rh4XsnxMeOYAcRFhjW9Zntnl+DAGTUFJxERkSPT7gJ0M5sH/BEIBR5yzv2mxeMRwGPANKAAuNg5l9OeQR+tzKwCEiLDGLptA2OefYTQKj/1lZX4S31cEVJPRG011NcDUOavBfafIZrYPBCFhFATGQW1HvzeSCwiEX9iEjVpQ1lXWo/fG8mokYOoiY6hNiqaFcX17HEeZh83kproWOoivLy9Kf+AMPR+ICD1a3astLKGKUPDKa6sAWgKSYnR4ZjR5rYDrRWBKzyJiIh0nsOGKTMLBZYAZwA7gU/MbKlzbl2z074KFDnnjjGzS4DbgYs7Y8CHk1fiJzXeiwv1UBsVhb9fIrXhXtYU+Bk7bAD14RHUeiOp80ayPK+CqvAIpo4eRJ03ktrIKD7OrWRPXQinHzec+vAIMGt1dqjxWFizY7X5PvbllbE3qh8xEQcPQ60dO9hOua+fPOKAY9o9JyIi0n20ZWZqOrDFObcVwMyeBs4Fmoepc4FbA18/B9xrZuaCsIaYEtgFFzJqHMtv/AXQEFo+311C2aD4/ZbQ1gUC0ZBmgSgy0kdVXhnF9aHEAL7KmjYHotZmjFoLQ4cKSND6TJJml0RERLqntoSpwcCOZt/vBE442DnOuVozKwH6A/uan2Rmi4HFAOnp6Uc55EObNbI/zyzfCbD/rM+0NN7fWrjf8c4KRG1dalNAEhER6fm6tM+Uc+4B4AFoKEDvjGsMT47h4oy0VkNOWmKUApGIiIh0qLaEqV3AkGbfpwWOtXbOTjPzAPE0FKIHxfDkmIMGHwUiERER6UghbTjnE2CUmQ03s3DgEmBpi3OWAlcGvr4QeDMY9VIiIiIiXe2wM1OBGqhrgVdpaI3wsHNurZndBix3zi0F/gI8bmZbgEIaApeIiIhIr9emminn3MvAyy2O/bTZ137gSx07NBEREZHury3LfCIiIiJyEApTIiIiIu2gMCUiIiLSDgpTIiIiIu2gMCUiIiLSDgpTIiIiIu2gMCUiIiLSDhasRuVmlg9sC8rFe6YkWtw4WroFfS7djz6T7kmfS/ejz+TIDHXOJbf2QNDClBwZM1vunMsI9jhkf/pcuh99Jt2TPpfuR59Jx9Eyn4iIiEg7KEyJiIiItIPCVM/xQLAHIK3S59L96DPpnvS5dD/6TDqIaqZERERE2kEzUyIiIiLtoDDVA5nZ98zMmVlSsMfS15nZHWa2wcxWm9nzZpYQ7DH1ZWY2z8w2mtkWM/thsMfT15nZEDN7y8zWmdlaM7su2GOSBmYWamafmtmLwR5Lb6Aw1cOY2RDgTGB7sMciALwOTHTOTQY2AT8K8nj6LDMLBZYA84HxwKVmNj64o+rzaoHvOefGAzOAa/SZdBvXAeuDPYjeQmGq5/kD8H1AxW7dgHPuNedcbeDbZUBaMMfTx00HtjjntjrnqoGngXODPKY+zTm3xzm3MvB1GQ2/vAcHd1RiZmnA2cBDwR5Lb6Ew1YOY2bnALufcqmCPRVr1FeCVYA+iDxsM7Gj2/U70i7vbMLNhwFTgoyAPReAuGv5RXh/kcfQanmAPQPZnZm8Aqa08dAtwMw1LfNKFDvWZOOf+FTjnFhqWNP7WlWMT6QnMLAb4B3C9c6402OPpy8zsHGCvc26FmZ0a5OH0GgpT3Yxz7vTWjpvZJGA4sMrMoGE5aaWZTXfO5XbhEPucg30mjczsKuAcYI5Tr5Fg2gUMafZ9WuCYBJGZhdEQpP7mnPtnsMcjzAQWmtlZgBeIM7MnnHOXB3lcPZr6TPVQZpYDZDjndJPKIDKzecCdwCnOufxgj6cvMzMPDZsA5tAQoj4BLnPOrQ3qwPowa/iX36NAoXPu+iAPR1oIzEzd6Jw7J8hD6fFUMyXSPvcCscDrZvaZmf052APqqwIbAa4FXqWh0PlZBamgmwksAr4Q+P/js8CMiEivopkpERERkXbQzJSIiIhIOyhMiYiIiLSDwpSIiIhIOyhMiYiIiLSDwpSIiIhIOyhMiYiIiLSDwpSIiIhIOyhMiYiIiLTD/wejR7NNm4ISHQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot the deterministic model\n", "X = np.linspace(-5, 5, 100)\n", "plt.scatter(X, model.predict(X), alpha=0.4)\n", "plt.plot(X, 1 / (1 + np.exp(-X)), color='r', alpha=0.8)\n", "plt.title('Deterministic Sigmoid function')\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0]])" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a constant input for this model\n", "x = np.array([[0]])\n", "x" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Explore the feedforward object\n", "y = model(x)\n", "y" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[0.5]]\n", "[[0.5]]\n", "[[0.5]]\n", "[[0.5]]\n", "[[0.5]]\n" ] } ], "source": [ "# ... and its behavior under repeated calls\n", "for _ in range(5):\n", " print(model.predict(x))" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"probabilistic_sigmoid_model\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense_5 (Dense) (None, 1) 2 \n", "_________________________________________________________________\n", "distribution_lambda_1 (Distr multiple 0 \n", "=================================================================\n", "Total params: 2\n", "Trainable params: 2\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model = Sequential([\n", " Dense(units=1, input_shape=(1, ), activation='sigmoid',\n", " kernel_initializer=tf.constant_initializer(1),\n", " bias_initializer=tf.constant_initializer(0)),\n", " tfpl.DistributionLambda(lambda t: tfd.Bernoulli(probs=t),\n", " convert_to_tensor_fn=tfd.Distribution.sample)\n", "], name='probabilistic_sigmoid_model')\n", "\n", "model.summary()" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAF1CAYAAADMXG9eAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA05ElEQVR4nO3deZhcVZ3/8fc3a2ffCWRh30EECagwgoIwGBDEUVnGhUFhFHAAAUVQQGQHFZBNQGQEFFFHJwr+XBEHESQsohCWECALELKQfeukz++PUzGd0J10p6r7VlW/X89zn+66davu997qdD59zrnnRkoJSZIkbZxuRRcgSZJUywxTkiRJZTBMSZIklcEwJUmSVAbDlCRJUhkMU5IkSWUwTEk1LCJSRGy7ka99OSLe38pz74mI51raNiLOiYhbN3KfiyJi64157Xre81cR8alKvme5+42ILUufTY9Wnt8hIp6MiIUR8V8dW+la+9289Bl076x9Sl1Bi//QJXWciHgZGAmsAhYDvwJOSSktKrKu5lJK/wfs0Mpzl7TlPSLij8CdKaV/Bq+UUv+NqScizgFOAEYA84A/p5SOKr3nBzbmPctV5n6/CNyfUtq9QuW0qPSz9pmU0u8AUkpTgY36DCS1zpYpqRgfLAWLdwDjgK+su0FrrRpdTan15xPA+0vnbBzw+2KrKtsWwNNFFyGpMgxTUoFSSjPILVO7wj+77U6OiBeAF0rrToiIyRExNyImRMSodd5mfERMiYjZEXFlRHQrvW6biPhDRMwpPXdXRAxe57V7RcQzEfFmRHwvIhpKr31vRExvqeaIuCAi7ix93xARd5b2MS8iHo2IkRFxMfAe4LpSt9J1zY5v29L3fSLiGxHxSkTMj4gHI6JPC7vcC/h1SunF0jl7PaV0c7N6/hgRnyl93730nrMj4qWIOKV5d1tp24si4qFSXb+IiGGlc7OgVP+Wzd57n9K6+aWv+6xnv1eV9jsFOLSlc1fa9g/A+5qdm+2bv1dpm+Mi4sFmj1NEfDYiXiid5+sjIpo9f0JETCp1Gz4TEe+IiDuAzYFflPbzxXW7HyNiVOlnam7pZ+yEdT7neyLi+6X3fToixrV2XFJXZpiSChQRY4HxwBPNVn8IeCewc0QcAFwKfAzYDHgFuHudtzmS3FrzDuAI4PjVb1967ShgJ2AscME6r/134F+BbYDtaaGFbAM+BQwqvfcw4LPA0pTSucD/kbsv+6eUTmnhtVcBewL7AEPJXV9NLWz3MPDJiDgrIsZtYLzPCcAHgN3J5+NDLWxzNLmlazT5uP8CfK9UwyTgfICIGArcC1xbOrZvAvdGxLBW9nsYsAf5s/hIawWmlA5g7XPz/HqOp7nDyMFyN/LPw7+W6vwo+XP9JDAQOByYk1L6BDCVUitoSumKFt7zbmA6+WfkI8AlpZ+51Q4vbTMYmABc18ZapS7FMCUV4+cRMQ94EHgAaD4O6dKU0tyU0lJy2LktpfR4Smk58GXg3c1bT4DLS9tPBa4GjgFIKU1OKf02pbQ8pTSLHAb2X6eO61JK01JKc4GLV7+2HRrJQWPblNKqlNJjKaUFG3pRqfXseODUlNKM0msfKh3jWlJKdwKfJ4eHB4A3IuJLrbz1x4BrUkrTU0pvApe1sM33UkovppTmk1sFX0wp/S6ltBL4MTkQQW5deiGldEdKaWVK6YfAs8AHW9nv1c3O5aUbOgcb4bKU0rzS53w/OTACfAa4IqX0aMomp5Re2dCblYL8vsCXUkrLUkpPAreSQ9lqD6aU7ksprQLuAN5eweOR6oZjMqRifGj1oOAWTGv2/Sjg8dUPUkqLImIOuVXl5Ra2f6X0GiJiJHANubttAPmPpzfXs69/vrYd7iC3St1d6kK8Ezg3pdS4gdcNBxqAF9uyk5TSXcBdEdGT3Np0V0Q8mVL69TqbjmLtY5rGW81s9v3SFh6vHqA9inxOmnuFfO7Xte5+NxhmNsLrzb5fwpo6x9LG87iOUcDclNLCZuteIbestbbPhojoUQqekkpsmZKqT2r2/avkwcoAREQ/ckvQjGbbjG32/eal10Bu7UrA21JKA4GPk7v+aMNr21ZoSo0ppa+llHYmd9cdxpqWjdT6K5kNLCN3s7V3fz8GnqI0zmwdrwFjmj0e28I2bbXWuS/ZnLXPffP9rnsu22Mx0LfZ403b8dpptH4e1/cZvAoMjYgBzda1dnyS1sMwJVW3HwL/ERG7R0RvckB6JKX0crNtzoqIIaVum1OBH5XWDwAWAfMjYjRwVgvvf3JEjCmNDzq32WvbJCLeFxFvK41jWkDu9ls97mkm0OKcUimlJuA24JulQdDdI+LdpWNcdx/HRcShETEgIrpFxAeAXYBHWnjre4BTI2J0qaWste7AtrgP2D4ijo2IHhFxFLAz8MtW9vtfpXM5BDi7nft6EvhwRPSNPED/0+147a3AmRGxZ2TbRsTqELi+z2Aa8BBwaeQLCXYr7ffOdtYudXmGKamKlboCvwr8lNz6sQ15AHVz/ws8Rv4P+V7gu6X1XyMPwp5fWv8/LeziB8BvgCnkrqKL2lnipsBPyEFqEnlM0x2l564BPhL5SsFrW3jtmcDfgUeBucDltPw7aQFwDnkw9TzgCuBzKaUHW9j2ltLxPEUe1H8fsJI8p1e7pJTmkFvazgDmkAfIH5ZSmt3Kfn8N/I3cLdvSuV6fbwEryOHnv4G72lHnj8nj3X4ALAR+Th5MD3ns1ldKVwCe2cLLjwG2JLdS/Qw4fz3dz5JaESmtrxVYkmpXqRXrppTSut11klQxtkxJqhuR564aX+qWG02e5uBnRdclqb7ZMiWpbkREX3JX447kK/PuJU+/sMHpGiRpYxmmJEmSymA3nyRJUhkMU5IkSWUobAb04cOHpy233LKo3UuSJLXZY489NjulNKKl5woLU1tuuSUTJ04saveSJEltFhGt3ibKbj5JkqQyGKYkSZLKYJiSJEkqg2FKkiSpDIYpSZKkMhimJEmSymCYkiRJKoNhSpIkqQyGKUmSpDJscAb0iLgNOAx4I6W0awvPB3ANMB5YAhyXUnq80oVKteClWYt48MU5zJy/jJGDGviXbYax1Yj+RZe1Qa3V3dJ6oE3rKn3c7Tm35XwObT3m9hxfOeexnHUt1die46v051/p81BujZ1xbttTY2ecs2o7jx1RYxG/cyOltP4NIvYDFgHfbyVMjQc+Tw5T7wSuSSm9c0M7HjduXPJ2MqonL81axI8mTmdwn570b+jBomUrmbe0kaPGjanqQNVa3ftuPZQ/T5m71vqX5ywmArYY2m+96yp93O05t+V8Di29ttzja+t7VnpdSzW25/gq/flX+jyUW2NnnNv21NgZ56zazmNH1NiRv3Mj4rGU0riWnttgN19K6U/A3PVscgQ5aKWU0sPA4IjYbONKlWrXgy/OYXCfngzs05NuEQzs05PBfXry4Itzii5tvVqr+0ePTX/L+rmLVzBn8YoNrqv0cbfn3JbzObT02nKPr63vWel1LdXYnuOr9Odf6fNQbo2dcW7bU2NnnLNqO48dUWNRv3MrMWZqNDCt2ePppXVvEREnRsTEiJg4a9asCuxaqh4z5y+jf8PaPef9G3owc/6ygipqm9bqfm3+0resX7GyieWNqza4rtLH3Z5zW87n0NJryz2+tr5npde1VGN7jq/Sn3+lz0O5NXbGuW1PjZ1xzqrtPHZEjUX9zt3gmKlKSindDNwMuZuvM/ctdbSRgxpYtGwlA/v0/Oe6RctWMnJQQ4FVbVhrdW82qM9b1vfq0Q1i7de3tK7Sx92ec1vO59DSa8s9vra+Z6XXtVRje46v0p9/pc9DuTV2xrltT42dcc6q7Ty2p8ZRA3qzbN5CBndrovvyZXRfsZwtXn+VXo3LGdE4ncWbjWbJyFGF/c6tRJiaAYxt9nhMaZ3UpfzLNsP40cTpAGv13//rLiMLrmz9Wqv7qD3H8Ocpc9daP7RfLyJgwdLG9a6r9HG359yW8zm09Npyj6+t71npdS3V2J7jq/TnX+nzUG6NnXFu21NjZ5yzTj+PS1YwsNsqVsxbwDaLZtGwbAl95r/EwFXLaFywiANmv0nv5UsZ+ZdV9GtcRtPixWwxbyG9li9lMCtpaFwGS5fCkqX8e6xi0fKVdO8WdO8WrGpKvGNlEwT06t6NyR87jqcO/FBhv3M3OAAdICK2BH7ZygD0Q4FTWDMA/dqU0t4bek8HoKseeTWfV/OV856VXufVfMWf23q5mm/Tft15z7DudJs/j78/PZXFM2ezSVrOzn2b6LZgATNemUnj3DcZtHIZm8UKui1cyKLZc1m1opGe3bvRv3duu1m0fCWNq5rWrItgHj1Y3LOB7v37MXT4YFLfvkxfDvOjFw0D+7P56KEM22QIsxqDSfMamd3UnQGDB7DL1puQGhp4bNZypvYcwKAxm3bo79z1DUBvy9V8PwTeCwwHZgLnAz0BUko3laZGuA44hDw1wn+klDaYkgxTkiQVaNUqmDMHZs9ee5k7Ny9z5qz5fvHi1t+nb18YPBgGDlx7GTAgLwMHQv/+LS8NDdCtNqa8XF+Y2mA3X0rpmA08n4CTN7I2SZJUaatWwcyZ8PrreXnjjbzMnLnm65tvQksNKgMHwrBhMHQo7LQTDBmy9jJ4cF4GDcrb9uz51vfoYjp1ALokSaqAlHIr0owZa5bp0+HVV+G112DWLGhqWvs1AwfCJpvkZYcdYMSIvAwfvmYZOhR6GA3ayzMmSVK1WrAAXn45L9OmwdSpeZk2DZY1mwIgAkaOhFGjYNw42HRT2GyzvGy6aQ5QffoUdRR1zzAlSVLRFiyAyZPz8tJLa5Y5zSag7N4dRo+GzTeHvfaCsWNhzJi8btNNoVev4urv4gxTkiR1lqam3B333HPw7LPwwgs5QL3xxppt+veHrbaCffeFLbfM32+xRW51sguuKvmpSJLUEVLKY5j+8Q94+ukcnp59FpYsyc/36JGD0p57wnbbwbbb5mXEiNxtp5phmJIkqRKWLMnB6W9/WxOg5s3Lz/XuDdtvD4ceCjvumJett/ZKuDphmJIkaWO8+SY88QQ8+WRenn02d+NF5Ban/faDXXfNy9Zb20VXx/xkJUlqi0WL4PHHYeJEePTRPN4J8sDvXXeF446D3XeH3XbL457UZRimJElqSVMTTJoEDz0Ef/lL7rprasrhaffd4aST8jQEO+1kd10XZ5iSJGm1BQvgwQfhz3+Ghx+G+fNzt90uu8Dxx8Pee+dWKKchUDOGKUlS1/bqq/DAA3l5/PHc+jR0KLznPbDPPvDOd+Zbp0itMExJkrqe6dPht7/Ny/PP53Vbbw2f+hTsvz/svHPN3IBXxTNMSZK6hldfhd/9LgeoSZPyut12g9NOywFq7NhCy1PtMkxJkurX4sU5QN17b+7Cgzz+6fTT4cAD821YpDIZpiRJ9aWpCR55BH75S7j/flixIt/P7qST4JBD8m1ZpAoyTEmS6sPs2TBhAvzsZ/DaazBwIBxxBBx2WB4D5S1a1EEMU5Kk2tXUBH/9K/z0p/lqvKamPH3BqafmGcidwkCdwDAlSao9S5fmcVA//CG88goMGQIf/zh86EO5S0/qRIYpSVLteP11uOee3JW3cGHuvvv61/NgcluhVBDDlCSp+r34Itx+O/z61/nxAQfAscfC297mWCgVzjAlSape//gHfO97eTxUnz5w9NFwzDGw2WZFVyb9k2FKklR9Hn8cbrkFHn00X5V34olw1FHe1kVVyTAlSaoeTz8NN96YbzI8bFienfzDH4a+fYuuTGqVYUqSVLznn4ebboI//QkGD84zlH/kI9C7d9GVSRtkmJIkFee11+D66+H//T8YMCDPUn700bZEqaYYpiRJnW/x4jyw/Ac/yI+PPx4+8YkcqKQaY5iSJHWeVavyHFHf+Q68+SaMHw8nnwwjRxZdmbTRDFOSpM7xxBNw2WV5zqh3vAOuvRZ22qnoqqSyGaYkSR1r7ly4+mq47z7YdFO48kp473udbFN1wzAlSeoYq1blGxDfcAMsW5bHRR1/PDQ0FF2ZVFGGKUlS5T3/fL5n3qRJ8M53wpe+5A2IVbcMU5KkylmxAr773XwfvUGD4NJL4f3vt0tPdc0wJUmqjH/8Ay68EKZMgUMPhTPOyLeCkeqcYUqSVJ4VK/K4qLvughEj8lV6++xTdFVSpzFMSZI23gsvwFe+kqc7+PCH4dRToV+/oquSOpVhSpLUfk1N8MMfwnXX5VnLbY1SF2aYkiS1z8yZcMEF8OijsP/+uWVqyJCiq5IKY5iSJLXdn/6Ug1RjI3z1q3D44V6ppy7PMCVJ2rCVK3OX3p13wg475CkPnDdKAgxTkqQNmTkTvvxleOop+OhH4fTToVevoquSqoZhSpLUuoceyt15jY1wySVw8MFFVyRVHcOUJOmtmpryTObf+Q5stx1cfrndelIrDFOSpLUtWQLnnw/33w/jx8O550Lv3kVXJVUtw5QkaY3p0+ELX4CXX85fjznGq/WkDTBMSZKyRx7JA80hX7m3997F1iPViG5FFyBJqgL33AOf/zxssgnccYdBSmoHW6YkqStraoKrr4Yf/AD22w8uugj69i26KqmmGKYkqataujTfCuaBB/LYqNNPh252WEjtZZiSpK5ozpwcnp59Fs46C446quiKpJplmJKkruall+C//gvefBOuuip370naaG1qz42IQyLiuYiYHBFnt/D85hFxf0Q8ERFPRcT4ypcqSSrb00/DZz4Dy5fDLbcYpKQK2GCYiojuwPXAB4CdgWMiYud1NvsKcE9KaQ/gaOCGShcqSSrTI4/AZz8L/fvDbbfBTjsVXZFUF9rSMrU3MDmlNCWltAK4GzhinW0SMLD0/SDg1cqVKEkq2+9+B6eeCmPG5NvEjBlTdEVS3WjLmKnRwLRmj6cD71xnmwuA30TE54F+wPsrUp0kqXw//Slcdhm8/e3wrW/BgAFFVyTVlUpdA3sMcHtKaQwwHrgjIt7y3hFxYkRMjIiJs2bNqtCuJUmtuv12uPRS2HffPKu5QUqquLaEqRnA2GaPx5TWNfdp4B6AlNJfgAZg+LpvlFK6OaU0LqU0bsSIERtXsSRpw1KCm2/OAeqQQ/JVew0NRVcl1aW2hKlHge0iYquI6EUeYD5hnW2mAgcCRMRO5DBl05MkFSEluPHGHKYOOwwuvBB6OBOO1FE2+K8rpbQyIk4Bfg10B25LKT0dERcCE1NKE4AzgFsi4nTyYPTjUkqpIwuXJLUgJfj2t+H734cjj8w3LnZWc6lDtelPlZTSfcB966w7r9n3zwD7VrY0SVK7pATf/Cb88IfwsY/BmWcapKROYLuvJNWDlODKK+Gee+DYY/OtYiKKrkrqEgxTklTrUoKrr85B6uMfz/NJGaSkTmP7ryTVuhtvhLvuyl17Bimp0xmmJKmWffe7+dYwRx6Zx0gZpKROZ5iSpFr1/e/nVqnx471qTyqQ//IkqRbdcw9cey0cfDCcf75BSiqQ//okqdb86ldwxRWw//55Qs7u3YuuSOrSDFOSVEseegguuAD23DPfc8+ZzaXCGaYkqVY89RScdRZst12enLNXr6IrkoRhSpJqw4sv5mkPRo7MY6X69Su6IkklhilJqnavvgqnnAINDXDddTB0aNEVSWrGznZJqmbz5uUgtXw53HILjBpVdEWS1mGYkqRqtXw5fOEL8PrreT6pbbYpuiJJLTBMSVI1amqC886Dv/8dLrsM3v72oiuS1ArHTElSNbr2Wvj97+G00+DAA4uuRtJ6GKYkqdr86Edw551w1FFw7LFFVyNpAwxTklRNHngArroqz25+xhneuFiqAYYpSaoWkybBOefAzjvDxRd7vz2pRvgvVZKqwaxZ+cq9IUPgW9/Kc0pJqglezSdJRVu2LHfpLVoEt93mpJxSjTFMSVKRmpryjYsnTYJvfCPfd09STbGbT5KKdOut8Lvfwec/D/vtV3Q1kjaCYUqSivKb38DNN8MHPwif+ETR1UjaSIYpSSrCpEm5e2/33eHLX3YKBKmGGaYkqbPNnQtnnpkHml9xBfTqVXRFksrgAHRJ6kwrV8LZZ8Obb3rlnlQnbJmSpM70rW/B44/DV74CO+5YdDWSKsAwJUmd5Re/yPfdO/ZYGD++6GokVYhhSpI6wzPPwKWXwl57wamnFl2NpAoyTElSR1s94HzYsByouncvuiJJFeQAdEnqSKtW5ZsXz5sHt98OgwcXXJCkSrNlSpI60o03wsSJOVBtv33R1UjqAIYpSeooDzyQW6M+/GE47LCiq5HUQQxTktQRpk+H88+HnXfO46Uk1S3DlCRV2vLl8MUvQrducNllznAu1TkHoEtSJaWUA9QLL8A118CoUUVXJKmD2TIlSZU0YUKenPOEE2CffYquRlInMExJUqVMngyXXw577w2f+UzR1UjqJIYpSaqEJUvgS1+CgQPhoovyeClJXYJjpiSpXKvHSU2blueVGjq06IokdSL/dJKkck2YAPfdByeeCHvuWXQ1kjqZYUqSytF8nNTxxxddjaQCGKYkaWMtWQJnnw0DBjhOSurCHDMlSRvr8sth6lS44QbHSUldmH9GSdLGuO8+uPde+PSnYdy4oquRVCDDlCS117Rp+eq9PfZwPilJhilJapfGRjj3XOjRA77+dejeveiKJBXMMVOS1B433ADPPANXXgmbblp0NZKqgC1TktRWDz0Ed9wBH/kIvO99RVcjqUoYpiSpLebMgQsugG22gdNPL7oaSVXEbj5J2pCmphykFi+Gm26C3r2LrkhSFWlTy1REHBIRz0XE5Ig4u5VtPhYRz0TE0xHxg8qWKUkFuvtu+MtfcovU1lsXXY2kKrPBlqmI6A5cDxwETAcejYgJKaVnmm2zHfBlYN+U0psRsUlHFSxJneqFF+Db34b99oN/+7eiq5FUhdrSMrU3MDmlNCWltAK4GzhinW1OAK5PKb0JkFJ6o7JlSlIBli/P0yAMHAhf/SpEFF2RpCrUljA1GpjW7PH00rrmtge2j4g/R8TDEXFIS28UESdGxMSImDhr1qyNq1iSOss118CUKfC1r8GQIUVXI6lKVepqvh7AdsB7gWOAWyJi8LobpZRuTimNSymNGzFiRIV2LUkd4MEH4Z574Nhj4V3vKroaSVWsLWFqBjC22eMxpXXNTQcmpJQaU0ovAc+Tw5Uk1Z65c3Nr1HbbwSmnFF2NpCrXljD1KLBdRGwVEb2Ao4EJ62zzc3KrFBExnNztN6VyZUpSJ0lpzTQIF18MvXoVXZGkKrfBMJVSWgmcAvwamATck1J6OiIujIjDS5v9GpgTEc8A9wNnpZTmdFTRktRhfvKTPNP5aac5DYKkNomUUiE7HjduXJo4cWIh+5akFr3ySh4jteeeefC5V+9JKomIx1JK41p6ztvJSBLAypV5+oOGBjjvPIOUpDbzdjKSBHDLLfDMM3DFFTB8eNHVSKohtkxJ0lNPwfe+Bx/8IBxwQNHVSKoxhilJXduSJbl7b9NN4cwzi65GUg2ym09S13bVVfDaa7mbr1+/oquRVINsmZLUdT3wAEyYAMcdB29/e9HVSKpRhilJXdPcuXDRRbDDDnDCCUVXI6mGGaYkdT0p5dnNFy+Gr38devYsuiJJNcwwJanr+cUvchffySc7y7mkshmmJHUtr76aB53vuSccc0zR1UiqA4YpSV1HU1O+iXFE/trNX4GSyudvEkldx113weOPw1lnwWabFV2NpDphmJLUNbz4ItxwA7zvfXDooUVXI6mOGKYk1b/GxjzL+YABcM453sRYUkU5A7qk+nfLLfD88/DNb8KQIUVXI6nO2DIlqb79/e9w++1w+OGw335FVyOpDhmmJNWvZcvg/PNh5Eg444yiq5FUp+zmk1S/rr0Wpk6Fm27yJsaSOowtU5Lq01//CvfcA8ceC+PGFV2NpDpmmJJUfxYuzJNybrllvmWMJHUgu/kk1Z8rr4TZs/PA8969i65GUp2zZUpSfbn/frjvPvj0p2HnnYuuRlIXYJiSVD/mzoVLLoEdd8xhSpI6gWFKUn1ICS6+GBYvhgsvhB6OYpDUOQxTkurDvffCAw/ASSfB1lsXXY2kLsQwJan2vf56HnS+xx55KgRJ6kSGKUm1rakpd+s1NeXpELr5a01S5/K3jqTa9uMf5wk6Tz8dRo8uuhpJXZBhSlLteuWVfMuYffaBI48suhpJXZRhSlJtWrUKzjsvT8p53nkQUXRFkroow5Sk2nT77fD003D22TB8eNHVSOrCDFOSas+zz8LNN8PBB+dFkgpkmJJUW1asyN16Q4bkVilJKphTBEuqLTfeCFOm5IHnAwcWXY0k2TIlqYY8/jjceSd8+MP5Cj5JqgKGKUm1YfHiPCnnqFFw2mlFVyNJ/2Q3n6Ta8I1v5NvG3Hor9O1bdDWS9E+2TEmqfg88ABMmwKc+BbvtVnQ1krQWw5Sk6jZ3Llx0EWy/PZx4YtHVSNJb2M0nqXqlBBdfDIsWwU03Qc+eRVckSW9hy5Sk6vWLX+QuvpNPhm22KboaSWqRYUpSdXr1VbjqKnjHO+DYY4uuRpJaZZiSVH2amvIs55CnQ+jmrypJ1cvfUJKqz/e/D08+CV/6Up5XSpKqmGFKUnV57rk82PzAA2H8+KKrkaQNMkxJqh7Ll8NXvgKDB8M550BE0RVJ0gY5NYKk6nHttfDSS3DddTBoUNHVSFKb2DIlqTo8/DD86Edw9NHwrncVXY0ktZlhSlLx5s/PV+1tvTV8/vNFVyNJ7WI3n6RirZ7lfN48uOYa6N276IokqV3a1DIVEYdExHMRMTkizl7Pdv8WESkixlWuREl1bcIE+MMf4KSTYIcdiq5Gktptg2EqIroD1wMfAHYGjomInVvYbgBwKvBIpYuUVKemToUrr4S99oKPf7zoaiRpo7SlZWpvYHJKaUpKaQVwN3BEC9t9HbgcWFbB+iTVq5Ur8zQIvXrB177mLOeSalZbfnuNBqY1ezy9tO6fIuIdwNiU0r3re6OIODEiJkbExFmzZrW7WEl15DvfgWeeyYFqk02KrkaSNlrZfwpGRDfgm8AZG9o2pXRzSmlcSmnciBEjyt21pFr1+ONw++1wxBFwwAFFVyNJZWlLmJoBjG32eExp3WoDgF2BP0bEy8C7gAkOQpfUogUL4KtfhbFj4YwN/g0mSVWvLVMjPApsFxFbkUPU0cCxq59MKc0Hhq9+HBF/BM5MKU2sbKmSal5KcOmlMHs2fO970Ldv0RVJUtk22DKVUloJnAL8GpgE3JNSejoiLoyIwzu6QEl15H//F377W/jc52Dnt1wULEk1qU2TdqaU7gPuW2fdea1s+97yy5JUd6ZMydMg7L03fPKTRVcjSRXjtciSOt6KFXDOOblb78ILnQZBUl3xdjKSOt7VV8PkyXDttTB8+AY3l6Ra4p+HkjrWH/8I99wD//7vsM8+RVcjSRVnmJLUcWbOzN16O+4IJ59cdDWS1CEMU5I6xqpVeT6plSvhkkvybWMkqQ4ZpiR1jO98J890fvbZsPnmRVcjSR3GMCWp8v7ylzwp5xFHwPjxRVcjSR3KMCWpsmbNgvPOg623hrPOKroaSepwhilJlbNqFZx7LixdCpddBg0NRVckSR3OMCWpcm6+OY+T+vKXYautiq5GkjqFYUpSZTz8MNx2Gxx+OBx6aNHVSFKnMUxJKt8bb+RpELbaCr74xaKrkaROZZiSVJ7Gxjz9wbJlcPnljpOS1OV4bz5J5bnmGnjqqTwxp+OkJHVBtkxJ2ni/+Q3cfTcccwwcfHDR1UhSIQxTkjbOlCnw9a/D298Op55adDWSVBjDlKT2W7IkDzTv0wcuvRR6OGJAUtflb0BJ7ZNSbpGaOhVuuAE22aToiiSpULZMSWqfO++E3/4WTjoJxo0ruhpJKpxhSlLbPfIIfPvbcOCB8KlPFV2NJFUFw5SktpkxY81tYs4/HyKKrkiSqoJhStKGLV0KZ5yRx0t94xvQt2/RFUlS1XAAuqT1SwkuvDBPhXDNNTBmTNEVSVJVsWVK0vp9//t5wPnJJ8O73110NZJUdQxTklr30ENw/fVw0EHwyU8WXY0kVSXDlKSWTZmSb2C87bZw3nkOOJekVhimJL3VvHlw2ml5hvNvfSt/lSS1yAHoktbW2AhnnQWzZ8Mtt8DIkUVXJElVzTAlaY2U8r32nngCLrkEdtml6IokqerZzSdpjbvuggkT4IQT4OCDi65GkmqCYUpS9qc/5Xmk3v/+HKYkSW1imJIEzzyTbxWz005wwQXQzV8NktRW/saUuroZM/KVe8OGwdVXQ0ND0RVJUk0xTEld2fz58F//BStXwre/DUOHFl2RJNUcr+aTuqoVK/LNi197DW64AbbYouiKJKkmGaakrqipCc4/H558Mk+FsPvuRVckSTXLbj6pq0kpX7X329/mLr6DDiq6IkmqaYYpqav57//O80kddRR84hNFVyNJNc8wJXUlP/85XHcdHHJIHi/lzYslqWyGKamr+MMf8i1i9tknj5dyLilJqgh/m0pdwcSJcO65sOuucPnl0LNn0RVJUt0wTEn1btIk+MIXYOzYPClnnz5FVyRJdcUwJdWzyZPhlFNg0KA8VmrgwKIrkqS6Y5iS6tXLL8NJJ0Hv3nDjjbDJJkVXJEl1yTAl1aNp0+Czn83f33gjjBlTbD2SVMecAV2qN6+9Bp/7HDQ2ws03e5sYSepghimpnrzxRm6RWrwYbroJttmm6Iokqe4ZpqR6MXNmDlJvvpm79nbYoeiKJKlLcMyUVA9efRVOOAHmzs1X7e2yS9EVSVKXYcuUVOtmzID//M/ctXfDDQYpSepkbWqZiohDIuK5iJgcEWe38PwXIuKZiHgqIn4fEY54lTrD1Km5RWrJkty1Z5CSpE63wTAVEd2B64EPADsDx0TEzuts9gQwLqW0G/AT4IpKFyppHS+9BCeeCCtWwHe+AzvuWHRFktQltaVlam9gckppSkppBXA3cETzDVJK96eUlpQePgw4qY3UkSZNykGqqSlPf7DddkVXJEldVlvC1GhgWrPH00vrWvNp4FflFCVpPSZOzGOkGhrg1lth662LrkiSurSKXs0XER8HxgFXtvL8iRExMSImzpo1q5K7lrqGP/wBPv952HRT+O53YfPNi65Ikrq8toSpGcDYZo/HlNatJSLeD5wLHJ5SWt7SG6WUbk4pjUspjRsxYsTG1Ct1XT//OZx9dh4bdeut3mtPkqpEW8LUo8B2EbFVRPQCjgYmNN8gIvYAvkMOUm9UvkypC0sJbr8dLroI3vWuPP3BwIFFVyVJKtngPFMppZURcQrwa6A7cFtK6emIuBCYmFKaQO7W6w/8OCIApqaUDu/AuqWuYdUquOIK+OlP4eCD4Wtfg549i65KktRMmybtTCndB9y3zrrzmn3//grXJWnJktyt99BD8KlPwcknQzdvWiBJ1cYZ0KVq9MYbcNppMHkynHsuHHlk0RVJklphmJKqzfPPw6mn5tvDXHMNvPvdRVckSVoP+wykavLHP8JnPgMReeoDg5QkVT1bpqRq0NSUpzu4+eZ8f72rrgKnD5GkmmCYkoq2eDGcdx488AB88IPw5S9Dr15FVyVJaiPDlFSkqVPhjDPglVfgzDPhqKNyF58kqWYYpqSi/OlPcP75ebqD66+HvfYquiJJ0kYwTEmdbeVKuO46uPNO2GEHuPJKGDWq6KokSRvJMCV1ppkz85iop56Cj34UTj/d8VGSVOMMU1Jneegh+OpXobERLrkk3x5GklTzDFNSR1uxIt+c+M47Ybvt4PLLYfPNi65KklQhhimpI02eDF/5Sv76kY/kbr3evYuuSpJUQYYpqSM0NcEPf5gHmg8YkG8Ls+++RVclSeoAhimp0mbOhAsugEcfhf33zy1TQ4YUXZUkqYMYpqRKaWqCn/0st0KllEPUEUc4Cack1TnDlFQJ06bB178Ojz8Oe+8N554Lo0cXXZUkqRMYpqRyNDXBD34AN94IPXrYGiVJXZBhStpYTz8Nl10GkybBfvvB2WfDJpsUXZUkqZMZpqT2mj8/30vvZz+DYcPyBJwHHWRrlCR1UYYpqa2ammDCBPj2t2HhQjjmGPjP/4R+/YquTJJUIMOU1BZPPgnf/CY88wzssQd86Uuw7bZFVyVJqgKGKWl9pk3LLVF/+EMeD3XhhfCBD9ilJ0n6J8OU1JIFC+DWW+Gee6BnT/jsZ+HjH4eGhqIrkyRVGcOU1NySJfk2MHfckb8//PAcpIYPL7oySVKVMkxJAMuWwY9/DLffnq/W239/+NznHBclSdogw5S6tuXL4ec/h9tugzlz4N3vzi1Ru+xSdGWSpBphmFLXtGgR/OQnefbyuXPhHe/IE3DusUfRlUmSaoxhSl3L3Llw9915YPmiRbDPPvAf/2GIkiRtNMOUuoYpU/LA8nvvhcZGOOCAHKJ23LHoyiRJNc4wpfrV1AQPPZRD1COPQK9ecOiheYqDLbYoujpJUp0wTKn+zJ8Pv/wl/PSnMHUqjBgBJ58MRx4JgwcXXZ0kqc4YplQfUoInnoD/+R/4/e9zV97b3gYXXwwHHgg9/FGXJHUM/4dRbZs1C371K/jFL+Cll/JNh488Mi/bbVd0dZKkLsAwpdqzdCn88Y95MPlf/5rHRu22G5x3Hhx0EPTpU3SFkqQuxDCl2rBiBTz8MPzudzlILVkCo0bB8cfD+PGw+eZFVyhJ6qIMU6pejY255ek3v8kBavFiGDgwtz4deijsvjt061Z0lZKkLs4wpeqycCH8+c/wwAP565Il0L9/nhfqoINg770dTC5Jqir+r6RipZSnL3joIfi//4PHHoNVq2DoUDj44HzD4Xe+M88RJUlSFTJMqfMtWQKPPpoD1F/+Aq++mtdvsUWeUHP//WHXXe3CkyTVBMOUOt7y5fC3v8HEiXn5xz/yFXh9+sBee8EnPwnvfjeMHl10pZIktZthSpW3eDE89VSeRPPJJ+Hvf8+Dybt1g112geOOyyFq992hZ8+Ci5UkqTyGKZWnqQmmT8+tTf/4Rw5Pkyfn9d26wQ47wFFH5fC0xx7Qt2/RFUuSVFGGKbVdSjBzJjz7LEyaBE8/nZeFC/PzffrkW7h85jO51WnXXQ1PkqS6Z5hSyxob4ZVXcivT88/Dc8/lEDV/fn6+WzfYdts8XcEuu+TgtNVWDhqXJHU5hqmubuVKmDYt39fu5ZfhxRdzgHr55TxFAeRxTdtsA+97H+y4Y1623RYaGoqsXJKkqmCY6gpSgtmzc2iaOnXN8sor+evq0ASw6aY5KL3nPflGwdtum6cscKJMSZJa5P+Q9WLRInjttTXLjBl5YPiMGXlZtmzNtj17wtixOSS99725e27LLfPiGCdJktrFMFULlizJLUszZ8Ibb6z99fXXc3hatGjt1/Tpk+dtGj06zyA+Zky+GfDmm+fWJ8c2SZJUEYapoixbBm++mZe5c/MyZ86ar7Nnr1mWLHnr6wcNgk02ycFojz3y1802y8uoUTBkCER0/nFJktTFGKbKtXJlnhpg4cJ8pdvChbBgQV7mzcvrmi+rA1Tzbrfm+vaFYcNg+PA80Hv48DXLJpusWRz8LUlSVeiaYWrVKli6NLf4LFuWvzZfFi/Ozy9alL9f/XX19wsXrvm6fPn699W/f25FGjQIBg/O45KGDFl7GTo0L8OGQe/enXEGJElShbQpTEXEIcA1QHfg1pTSZes83xv4PrAnMAc4KqX0cmVLbbuXZi3imfseYPMffI/+qZFNeib6pZUsXbiYRfMWsmr5Cnp270b/3vnwFy1fSeOqphbXRb9+DBg6iNS3L6+v7M7CHr3pMXg4o7fbmaZ+/Xh+STA7etF3+GB22m40Tf0H8Nc3V/Lqqp4M3mwE+26/CQAPvjiHmfOXMXJQA/+yzbC11y1q4F9GDmSr3r15adai9W9b8LqtRvS3xg3su6Wfx2qqsa11l6vSx9daja2d342tp63notLv1xHaem7qbd+doSOOr97PWT2LlNL6N4joDjwPHARMBx4FjkkpPdNsm5OA3VJKn42Io4EjU0pHre99x40blyZOnFhu/W/x0qxF/GjidLaZMZm333c3S7v3YnG3noweOZgpi1bRs19fegzox6JuvXhteWJlQx+GDhtEr4H9WdCtFy8vaaKxTx82GzmU3oMHsmhFEy/PWUwEbDG0H/0berBo2cqKr5u3tJF9tx7Kn6fMZXCfnh22H2vs2H0fNW7MWr/8Vv88Nq+l2s5jS3VX6t9hpY6vtRpbO79t+RzKOReVfr+O0NZzU2/77gwdcXz1fs7qQUQ8llIa19JzbWmZ2huYnFKaUnqzu4EjgGeabXMEcEHp+58A10VEpA0ltQ7w4ItzGNynJ41v242Jb9sNgAVLG/nBq/PZddQgBvZZc2Pd/3vuDQh47/absLi07sXSuq2HDQFgYJ/uzF28AgLeNnpwaV3Piq8D+NFj09eqsSP2Y40du+8HX5yz1i++1T+PzWuptvPYUt3laum4O6LG1s5vWz6Hcs5Fpd+vI7T13NTbvjtDRxxfvZ+zeteW6+NHA9OaPZ5eWtfiNimllcB8YNi6bxQRJ0bExIiYOGvWrI2reANmzl9G/4a1M2L/hh68Nn/pW9avWNnE8sZVVbHOGmu/xv4NPZg5f+0LC2rh57GlusvV0nF3RI2tnd+2fA7lnItKv19HaOu5qbd9d4aOOL56P2f1rlMHoKeUbgZuhtzN1xH7GDmogUXLVq7VArVo2Uo2G9TnLet79egG68weUNQ6a6z9GhctW8nIQWtfZVkLP48t1V2ulo67I2ps7fy25XMo51xU+v06QlvPTb3tuzN0xPHV+zmrd21pmZoBjG32eExpXYvbREQPYBB5IHqn+5dthjFvaSMLljbSlBILljbmfuc9x7xl/dB+vRjWr1dVrLPG2q9x3tLGfw5CrqWfx5bq7oh/hx1RY2vnty2fQznnotLv1xHaem7qbd+doSOOr97PWb1rywD0HuQB6AeSQ9OjwLEppaebbXMy8LZmA9A/nFL62Pret6MGoENtXD3V0jprrP0avZpv/cfdETV6NV/7avRqvsrwar6uZ30D0DcYpkpvMB64mjw1wm0ppYsj4kJgYkppQkQ0AHcAewBzgaNXD1hvTUeGKUmSpEoq92o+Ukr3Afets+68Zt8vAz5aTpGSJEm1yLvdSpIklcEwJUmSVAbDlCRJUhkMU5IkSWUwTEmSJJXBMCVJklQGw5QkSVIZDFOSJEllMExJkiSVoU23k+mQHUfMAl4pZOe1aTgwu+gi9BZ+LtXHz6Q6+blUHz+T9tkipTSipScKC1Nqn4iY2No9gVQcP5fq42dSnfxcqo+fSeXYzSdJklQGw5QkSVIZDFO14+aiC1CL/Fyqj59JdfJzqT5+JhXimClJkqQy2DIlSZJUBsNUDYqIMyIiRcTwomvp6iLiyoh4NiKeioifRcTgomvqyiLikIh4LiImR8TZRdfT1UXE2Ii4PyKeiYinI+LUomtSFhHdI+KJiPhl0bXUA8NUjYmIscDBwNSiaxEAvwV2TSntBjwPfLngerqsiOgOXA98ANgZOCYidi62qi5vJXBGSmln4F3AyX4mVeNUYFLRRdQLw1Tt+RbwRcDBblUgpfSblNLK0sOHgTFF1tPF7Q1MTilNSSmtAO4Gjii4pi4tpfRaSunx0vcLyf95jy62KkXEGOBQ4Naia6kXhqkaEhFHADNSSn8ruha16HjgV0UX0YWNBqY1ezwd/+OuGhGxJbAH8EjBpQiuJv9R3lRwHXWjR9EFaG0R8Ttg0xaeOhc4h9zFp060vs8kpfS/pW3OJXdp3NWZtUm1ICL6Az8FTkspLSi6nq4sIg4D3kgpPRYR7y24nLphmKoyKaX3t7Q+It4GbAX8LSIgdyc9HhF7p5Re78QSu5zWPpPVIuI44DDgwORcI0WaAYxt9nhMaZ0KFBE9yUHqrpTS/xRdj9gXODwixgMNwMCIuDOl9PGC66ppzjNVoyLiZWBcSsmbVBYoIg4Bvgnsn1KaVXQ9XVlE9CBfBHAgOUQ9ChybUnq60MK6sMh/+f03MDeldFrB5WgdpZapM1NKhxVcSs1zzJRUnuuAAcBvI+LJiLip6IK6qtKFAKcAvyYPdL7HIFW4fYFPAAeU/n08WWoRkeqKLVOSJEllsGVKkiSpDIYpSZKkMhimJEmSymCYkiRJKoNhSpIkqQyGKUmSpDIYpiRJkspgmJIkSSrD/wdU8LUT8ae3xgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot the probabilistic model\n", "X = np.linspace(-5, 5, 100)\n", "plt.scatter(X, model.predict(X), alpha=0.4)\n", "plt.plot(X, 1 / (1 + np.exp(-X)), color='r', alpha=0.8)\n", "plt.title('Probabilistic Sigmoid function')\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0]])" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a constant input for this model\n", "x = np.array([[0]])\n", "x" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Explore the feedforward object\n", "y = model(x)\n", "y" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[0]]\n", "[[0]]\n", "[[1]]\n", "[[0]]\n", "[[1]]\n" ] } ], "source": [ "# ... and its behavior under repeated calls\n", "for _ in range(5):\n", " print(model.predict(x))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Use the forward model to create probabilistic training data" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "# Use the model to create 500 training points\n", "X_train = np.linspace(-5, 5, 500)[:, np.newaxis]\n", "y_train = model.predict(X_train)" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAFlCAYAAADPim3FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA7A0lEQVR4nO3deZxT1f3/8ddJZmMYFhFcEBBQUUCstgOi+PvWuuLS8m21Vax+W63aBai7UhfAutX16wKiVlGpWqVaLSpfWxXrrshiRVAQEcfBBQRZZzJbzu+PM8dkQoDMTDI3ybyfj0ceSW5uzvmcezPJZ84991xjrUVEREREWiYUdAAiIiIiuUzJlIiIiEgrKJkSERERaQUlUyIiIiKtoGRKREREpBWUTImIiIi0QkFQFXfv3t327ds3qOpFREREUjZv3ryvrbU9kr0WWDLVt29f5s6dG1T1IiIiIikzxny6tdd0mE9ERESkFZRMiYiIiLSCkikRERGRVlAyJSIiItIKSqZEREREWkHJlIiIiEgrKJkSERERaQUlUyIiIiKtoGRKREREpBW2OwO6MWYacDywylq7b5LXDXAbcCxQBfzSWjs/3YG2VEMDRCJQXw8FBVBSAuFw0FHltkxs06D3Uyr1Z6rdmzfDpk3ucUGBu1nr7ouLXT0Qq9sYt251tVteVubWqatLHls64k4so7DQ1VdTE1tWXBxbvrW6fHvXr3fllZS4+MNh9x7fvvp6V3ZdnSuzoMAt88+Li2PLampcWcZAaakrD9w2ja8Dkm8z34b4en15xcWxW0Hjt6Vfp77exd2hQyx+vxxi26SgoOn+6tAhFt/mzbF6fHsiEYhGk++HcLhpG6urXdk+9m29N1lZifVubRvG74P4OpKV0Zz6E+tKtj0SPwPx+6Yl9W6r/c0tI1OyObZMamm7/WepSxfo2DGY3/hULifzADAZmL6V148B9mq8HQhMbbwPXEMDbNzoNmxhYex5p05KqFoqE9s06P2USv2Zave6dVBV5cqorobaWvdaly7uyyQUcl8oAEVF7vmaNe6Hp1s39761a11ZO+7o1omPDVofd2Lba2th9WpXRk2NW6e+3iWAq1dD165bxhEOx9q7caMrIxRy7Vi71v2gFhW5stavd+vW1bkvxrVr3fpFRdC5s3u9sDAWXyTiyg+F3HtWr3bLO3Rw623a5JaFw9CjR9Nt1rWra0NDQ6xeX5617ou8Y8fYD75XXR1LpFaujCVUPmHyOnSIJVLdurllFRVuWVGRqzsadXE3NLgywmEXc+KPSDjstqVvYzjsyty0Cb75JrZOsvcm8j8+0WisXv/5StyGoRB89ZXbB/Hx+cQrvozm1B/fFki+Pfw+958BH6e1qW2z5rS/uWVkSjbHlkktbbf/LPn31dW5v+u2/o3fbjJlrX3FGNN3G6uMAqZbay3wljGmqzFmV2vtF+kKsqX8l6LfqP4+EnFfkNJ8mdimQe+nVOrPVLvr6tyPg//v3/9ARKNuuX8O7odt82b3mu9VKSpyz/0PUFFR09h8rK2JO7Htvp6NG2NJhE+cfMyJcXTsGGtvNOraGg67H09jXLJhjFsWjbofzpKSWHJSX+/KrK52/7lGIrEvXK+kxG0fv72Ki2M/0LW17os6cZv5Nvi4fDILLh5/v2mTez+45Ndvu02bmsbZsaN7Hdz6mzbFYquri5VXW+vi9OvHJ6V+W/v1vaKi2LZqaIiVH4269X1Snuy9iYqKYjHEb99k29BvB99z6OuIb8f2Yk9Wf3xbIPn28Ps8vn7/3lS2WXPa39wyMiWbY8uklrbbfx7ivweD+I1Px4WOdwM+i3te2bhsi2TKGHM2cDZAnz590lD1ttXXN/0PFvL/A5lpmdimQe+nVOrPVLv94TxfvrWuJ8B/kSSW79/jkxZo+p5ksbU27sS2+x+0jRtjPTY+MSora1p2fF0+dt/m+Njr6txj3xtXV+d6JDZujNXte+k6dnTLIZaYWbtlu6yN3UejTbeRr9fHHF+vL8/X6XtB/LK6ui0TRR+P79FKbHvnzrHHoVDsiz9+fc8f7k1cHh9L/PYoLHSPt/XeRImfF79Nkm1Dv20S44Mty2hO/fFtgeTbI76N8TElSrXe+HJbGnumZXNsmdTSdvvPUvwtsZy2kI5kKmXW2nuAewDKy8sz/rHwYxUSx2wUtGmr80smtmnQ+ymV+jPVbj/+yZef+NzfJ76ntjb2AxP/nmSxtTbuxLYXFLj6i4tjyxsa3PP4uBLr8rH7eH3vRkODe098z5Q/nBjfo+N75Hwdvgz/I+zL8dsrvmfJJzA+Fl+vb0NBQaxeX56P35hYfNC0jvh44tf3j/3y+HV8Yudj9ut7W1sejcZi8b2XJSWxxGxb700Uv828+H0Rvw39Nk4WX2IZzak/vi1be298L2x8nIlSrXdb7W9uGZmSzbFlUirtNvV1hGqqCde6WyhSRWkoQkdThem7O6Z/b4wJ5jc+HVWuBHrHPe/VuCxwJSVN/2NsaIh1j0vLZGKbBr2fUqk/U+2ORNxhDT+2x/eYhEKxH/v4Xgd/GKaqytXd0BD7zyz+Bz0+ttbGndj2wkIXgx8z5b/8OnVy6/myE+vy7Q2FYt35/r9RP74J3DJ/eK5jR/dD6r8c/aFOP74KYoczIxH3uj9U5+vx5fltFL/NfBv8NveHXKFpz1b8mKmystiYqbIyN9YqfsyU31+RiHvdj5ny28H3LPpDWNY2TUz9OJHEMR8NDW5d30Yflz95obh46+9N5PdL/PiUrW1D/0NWUJD6mKlU6o9vCyTfHn6f+8+ALzfVbdac9je3jEzJ5thawtTXEa7eRLh6EwURdx+u2hh73HgrrNlEcbQaE6nCRFyyVFBbTXRzFaGaandrSNLlZCAcgvX/M47aPX9BYWHspJ02badNod+wcczUM1s5m+84YCzubL4DgduttcO2V2Z5ebmdO3duswNuLj+gVGfzpU8mtmnQ+ymV+jPVbp3N59bR2Xw6m09n8zlZGVt9PaFNGwhvXEd403pCGxLuN64nvHEdoY3rCG/aQGjzRkJVmzB1NdstOlpSSrS0DDqUYko7QGkHosWl1BV0oKGoA7akA9GSUndf3IFoh1JscYdvn5uOpRT12YVOu3fL6Nl8xph51trypK9tL5kyxvwVOBToDnwFTAQKAay1dzVOjTAZGImbGuF0a+12s6S2SqZEREQkiWjUnQq6Zg18/XXT+/jHa9fGzqZIpqjInULnb507u27fsrLYLfF5/C1+UF4W21YylcrZfKO387oFxrQwNhEREUm3aNQlQ199BV9+Gbv556tXu0QqWZdXx45urpXu3WGffZomSv7WpUvsPojjallGQ7FFRERyjbXuePlnn0FlZew+PmmKH80N7rjqLru42z77uITJJ03du8eeKzlqNiVTIiIi2WrzZvjkE3eLT5w++6zpoTdjYKedoGdP+M53YknTzjvHHifOPitpo2RKREQkaBs2uIRp+fLY/fLlsGpVbJ1w2CVLvXvDkCHQq5d73KsX7LZb09NcpU0pmRIREWkr1sIXX8DSpbBkibt9+GHTpKmkBPr3h6FD3X2/fu7Ws6dOR89SSqZEREQywSdO77/vbkuXutuGDe71UAh23x2++10YMAD22MMlTbvskjNnuImjZEpERCQdqqrggw9g4cLYbe1a91pREey1FxxxBOy9t7vtuacGe+cJJVMiIiItsXEjLFgA8+fDvHnukJ2faqBPHzjoIDe2acgQ1+uka5nlLe1ZERGRVPjkae5clzwtXRq7PtC++8Lpp7sz6QYPdvMvSbuhZEpERCSZaNQlTG+84W7vveeWFRXBfvvBWWdBeblLpHQmXbumZEpERMRbvx7eegtef93d+zFPAwe6nqcDD1TyJFtQMiUiIu3bV1/Bv/8NL73kxj9Fo+4w3UEHwcEHw/Dh0K1b0FFKFlMyJSIi7U9FBbz4okugFi92y/r3h1/+Er7/fdcTpekJJEVKpkREpH1YtQqefx6ee85NYQBusPjYsfCDH7g5n0RaQMmUiIjkrw0bYPZsl0DNm+fOvhs0CM4/3835tNNOQUcoeUDJlIiI5Jdo1CVOTz3lDuPV1rp5n846C0aOdI9F0kjJlIiI5IfVq+Hpp+Ef/4CVK6FTJ/jv/4bjj3djoIwJOkLJU0qmREQkd1kLc+bAY4/Ba6+5XqnvfQ9+8xs47DAoLg46QmkHlEyJiEjuiURg1ix49FFYvhx22AFOOw1GjdJhPGlzSqZERCR3fPklzJjhxkNt2AADBsDEiXD00ZpIUwKjZEpERLLf8uXwwAPurDyAQw+F0aNh//01FkoCp2RKRESy1wcfwP33u+kNSkrg5JNdErXrrkFHJvItJVMiIpJ9FiyAadPgzTfdWXlnnukSqa5dg45MZAtKpkREJHu8/z7ceac7Q69bNxg3Dk48ETp2DDoyka1SMiUiIsFbtgymToWXX3Zn5p1/PvzkJ+7QnkiWUzIlIiLB+fxzl0Q995zrffrd79zhvNLSoCMTSZmSKRERaXubN8N998Ff/wqhEPziF/A//wOdOwcdmUizKZkSEZG2E426OaLuugvWroXjjoMxY3TBYclpSqZERKRtzJsHN97oxkcdcADceisMGhR0VCKtpmRKREQya+1alzjNmgU9e8INN8APfqDJNiVvKJkSEZHMiEbhiSdgyhSoqXFzRZ1+ui4+LHlHyZSIiKTfkiVwzTWweDEMGwaXXAK77x50VCIZoWRKRETSp7YW7r3XXUdvhx1cQnXUUTqkJ3lNyZSIiKTHokVw5ZXuosTHH+8m3tRUB9IOKJkSEZHWqamBu++Ghx6C7t3h9tvh4IODjkqkzSiZEhGRllu6FC6/3PVG/fjHcM45UFYWdFQibUrJlIiINF80Co884s7U69IFJk+G4cODjkokEEqmRESkeVatgkmTYM4cOPRQ1zPVtWvAQYkER8mUiIik7uWX3SDz2lqXRI0apTP1pN1TMiUiIttXXw933AEPPwwDB7opD/r0CToqkaygZEpERLbtq6/gD3+A996Dk06Cc8+FwsKgoxLJGkqmRERk69580x3Oq6uD666DI48MOiKRrKNkSkREthSNupnM//xn2GMPd3FiHdYTSUrJlIiINFVVBVdc4QabH388jB8PJSVBRyWStZRMiYhITGWluwzMihVw4YVujJTO1hPZJiVTIiLizJnjeqHATcI5bFiw8YjkiFDQAYiISMCshUcfhbFj3bX1pk9XIiXSDOqZEhFpz6JRuPlmeOwx+P734aqroLQ06KhEcoqSKRGR9qq6Gi67DF55BX7+c3eR4pAOWIg0l5IpEZH2aO1aN/nmhx/CxRfDz34WdEQiOUvJlIhIe7NiBfz+97BmDdx0E/zXfwUdkUhOS6k/1xgz0hizxBizzBgzPsnrfYwxLxljFhhj3jPGHJv+UEVEpNUWLoQzzoBIxE3IqURKpNW2m0wZY8LAFOAYYBAw2hgzKGG1y4EZ1toDgJOBO9MdqIiItNLbb8NvfwtdusADD8CgxK9yEWmJVHqmhgHLrLXLrbW1wKPAqIR1LNC58XEX4PP0hSgiIq32wgtugHmfPnDffdCzZ9ARieSNVJKp3YDP4p5XNi6LNwk41RhTCcwCxqUlOhERab0nn4Q//AH23Rfuvhu6dQs6IpG8kq5zYEcDD1hrewHHAn8xxmxRtjHmbGPMXGPM3NWrV6epahER2aoHH4RrroERI9ys5p06BR2RSN5JJZlaCfSOe96rcVm8XwEzAKy1bwIlQPfEgqy191hry6215T169GhZxCIisn3Wul6oO+6AkSPdWXu6WLFIRqSSTL0D7GWM6WeMKcINMJ+ZsE4FcDiAMWYgLplS15OISBCshalT3dl6o0bBH/8IBZoJRyRTtvvXZa2tN8aMBf4JhIFp1tpFxpg/AnOttTOBC4A/G2POww1G/6W11mYycBERScJa1xs1fTr85CfuwsWa1Vwko1L6V8VaOws3sDx+2YS4x4uBEekNTUREmsVa+N//hUcecTOaX3QRGBN0VCJ5T/2+IiL5wFo3Luqxx2D0aDj/fCVSIm1EyZSISK7zPVKPPQannurmk1IiJdJmdCBdRCSXWQt33ukO7Y0erURKJABKpkREctm0aXD//W6wuQ7tiQRCyZSISK566CE3BcJxx7mz9pRIiQRCyZSISC6aMQNuvRWOPBImTND0ByIB0l+fiEiumTULbrgB/uu/4KqrIBwOOiKRdk3JlIhILnntNZg0CYYOhT/9STObi2QBJVMiIrnivffgkktg773h5puhqCjoiEQEJVMiIrnh44/dtAc77wy33QalpUFHJCKNlEyJiGS7L76AsWOhuBgmT4Zu3YKOSETi6GC7iEg2W7fOJVKRCPz5z9CzZ9ARiUgCJVMiItmqpsZNxPnFF26W8z33DDoiEUlCyZSISDaKRmHiRFi4EK6/HvbfP+iIRGQrNGZKRCQbTZ4ML7zgBp0fdljQ0YjINiiZEhHJNk88AdOnw09/Cj//edDRiMh2KJkSEckmr7/uDusdcghceKGutyeSA5RMiYhkiyVL3AWL99oLrr1Wl4kRyRFKpkREssGqVXDuudC5s7uAsSblFMkZOptPRCRokQicdx5s3gzTpkGPHkFHJCLNoGRKRCRI1sKVV8LSpa5HSnNJieQcHeYTEQnSfffB88/DuHEwYkTQ0YhICyiZEhEJyuzZcNddcOyxcNppQUcjIi2kZEpEJAhLl8KECTB4MFx+uaZAEMlhSqZERNra2rVwwQXQqRPcfDMUFQUdkYi0ggagi4i0pbo6uOQSWLMG7r0XuncPOiIRaSUlUyIibemmm2DBArjmGhg0KOhoRCQNdJhPRKStzJzprrv3P/8DRx8ddDQikiZKpkRE2sIHH8Cf/gRDh8KYMUFHIyJppGRKRCTT1q+Hiy+GHXbQNfdE8pDGTImIZFI0CpdeCl9/7Sbo3GGHoCMSkTRTz5SISCbddRe8/bbrmdKAc5G8pGRKRCRTXn7ZXbh41Cj48Y+DjkZEMkTJlIhIJlRUuBnOBw1y80qJSN5SMiUikm6RCFx0ERQUwPXXa4ZzkTynAegiIul2443w8cdwxx2w665BRyMiGaaeKRGRdJo1C/7xDzjjDDjooKCjEZE2oGRKRCRdli9380h997vw618HHY2ItBElUyIi6VBdDePHQ4cO7rp7mphTpN3QmCkRkXS44Qb45BOYPBl69Ag6GhFpQ+qZEhFpraefdrdf/QoOPDDoaESkjSmZEhFpjY8/dhcwLi+Hs88OOhoRCYCSKRGRlqqqchNyduzoxkmF9JUq0h5pzJSISEtdf72b6XzKFNhxx6CjEZGA6N8oEZGW+L//g2efdeOkhg4NOhoRCZCSKRGR5lq5Eq67Dr7zHTjzzKCjEZGAKZkSEWmO+nq49FI3PurqqzWflIhozJSISLPcfTcsWuTO4NN190QE9UyJiKTunXfggQfgv/8bjjgi6GhEJEsomRIRScW6dTBhAuy+O1xwQdDRiEgW0WE+EZHtsRb++EeXUN12m7v+nohIo5R6powxI40xS4wxy4wx47eyzs+MMYuNMYuMMY+kN0wRkQD97W/wyiswbhwMGBB0NCKSZbbbM2WMCQNTgCOBSuAdY8xMa+3iuHX2Av4AjLDWfmOM2SlTAYuItKlly+DWW+Hgg2H06KCjEZEslErP1DBgmbV2ubW2FngUGJWwzlnAFGvtNwDW2lXpDVNEJAA1NW4ahLIymDQJjAk6IhHJQqkkU7sBn8U9r2xcFm8AMMAY87ox5i1jzMhkBRljzjbGzDXGzF29enXLIhYRaSu33grLl7vxUt26BR2NiGSpdJ3NVwDsBRwKjAb+bIzpmriStfYea225tba8R48eaapaRCQD3njDjZX6+c9h+PCgoxGRLJZKMrUS6B33vFfjsniVwExrbZ219hNgKS65EhHJPevXw5VXwh57wJgxQUcjIlkulWTqHWAvY0w/Y0wRcDIwM2Gdp3C9UhhjuuMO+y1PX5giIm3EWnfdvfXr4aqroKgo6IhEJMttN5my1tYDY4F/Ah8AM6y1i4wxfzTG/KhxtX8Ca4wxi4GXgIustWsyFbSISMY89xy88AL8+teaBkFEUmKstYFUXF5ebufOnRtI3SIiSX31FZx0EvTvD/fe6y5mLCICGGPmWWvLk72mbwoREYBo1I2TamhwZ+8pkRKRFOnbQkQEYMYMmDMHzjsPevUKOhoRySFKpkREVqyA22+HQw6BH/846GhEJMcomRKR9q2+HiZMcBcvvuIKzXIuIs223WvziYjktWnTYPFiuOEG2HHHoKMRkRyknikRab8WLXJn7R17LBx2WNDRiEiOUjIlIu1TJOIO63XvDhddFHQ0IpLDdJhPRNqn22+Higq4807o1CnoaEQkh6lnSkTan7ffdlMhjB4Nw4YFHY2I5DglUyLSvmzY4Cbn7NcPxo4NOhoRyQM6zCci7csNN8CaNXDzzVBcHHQ0IpIH1DMlIu3H88+7CxmfdRYMHBh0NCKSJ5RMiUj7sHo1XHcd7LsvnH560NGISB5RMiUi+c9ad/Himhp3Hw4HHZGI5BElUyKS/554At58E845B/r0CToaEckzSqZEJL9VVMCtt8Lw4fDTnwYdjYjkISVTIpK/GhrcRYyLity9LmIsIhmgZEpE8teDD8L778P48bDTTkFHIyJ5SsmUiOSnDz+Eu++Go45yNxGRDFEyJSL5p7bWHdbr1s31SomIZJBmQBeR/DN5Mixf7u47dw46GhHJc+qZEpH8MncuPPII/Oxn7gw+EZEMUzIlIvlj0yaYONHNJTVuXNDRiEg7ocN8IpI/brzRXTZm2jTo0CHoaESknVDPlIjkh9mz4dln4Ywz3PX3RETaiJIpEcl9a9bAtdfCwIFw5plBRyMi7YySKRHJbdbC1VdDVZW7iHGBRi+ISNtSMiUiue0f/4BXX3UDzvv1CzoaEWmHlEyJSO5auRJuuQWGDYOTTgo6GhFpp5RMiUhuikbdLOehkJsOIaSvMxEJhr59RCQ3/eUv8J//wMUXw847Bx2NiLRjSqZEJPcsXQpTp8Jhh8ExxwQdjYi0c0qmRCS3+IsYd+4Ml14KxgQdkYi0czqHWERyy113wbJlcOut0LVr0NGIiKhnSkRyyIIFbqzUT34ChxwSdDQiIoCSKRHJFZs3u7P2dtsNzj036GhERL6lw3wikhtuvhm+/BLuvRdKS4OORkTkW+qZEpHs99JLMHMmnH467Ldf0NGIiDShZEpEstuaNXDNNe4ixmedFXQ0IiJbUDIlItnLWnfxYl3EWESymJIpEcleTz4Jr78O55yjixiLSNZSMiUi2amiwl3E+MAD4ac/DToaEZGtUjIlItmnocHNcl5UpIsYi0jW0zeUiGSf+++H99+H8eNhp52CjkZEZJuUTIlIdlm8GO65B0aOhKOOCjoaEZHtUjIlItkjEoErroDu3eHii4OORkQkJTrPWESyx+23w6efwp13QufOQUcjIpIS9UyJSHZ44w2YMQNOOQWGDQs6GhGRlCmZEpHgrV/vJuXs3x/Gjg06GhGRZtFhPhEJlrXucjHr1sFtt7npEEREckhKPVPGmJHGmCXGmGXGmPHbWO8EY4w1xpSnL0QRyWv/+AfMng2/+x3svXfQ0YiINNt2kyljTBiYAhwDDAJGG2MGJVmvE3AO8Ha6gxSRPPXpp3DTTW6M1KmnBh2NiEiLpNIzNQxYZq1dbq2tBR4FRiVZ7yrgeiCSxvhEJF/V1sKll0JxMUyapFnORSRnpfLttRvwWdzzysZl3zLGfBfoba19dlsFGWPONsbMNcbMXb16dbODFZE8MnUqLFniLhujWc5FJIe1+l9BY0wIuAW4YHvrWmvvsdaWW2vLe/To0dqqRSRXzZkDf/kLnHACfP/7QUcjItIqqSRTK4Hecc97NS7zOgH7Av82xqwAhgMzNQhdRJJat871RvXrB+edF3Q0IiKtlsrUCO8Aexlj+uGSqJOBU/yL1tr1QHf/3Bjzb+BCa+3c9IYqIjnPWjef1Pr1brbzkpKgIxIRabXt9kxZa+uBscA/gQ+AGdbaRcaYPxpjfpTpAEUkjzzxBLzyCowbBwMGBB2NiEhapDRpp7V2FjArYdmErax7aOvDEpG8s3w53HILHHwwnHxy0NGIiKSNzkUWkcyrrYXLLoOOHTUNgojkHV1ORkQy7/bb4aOP3OViunULOhoRkbTSv4cikln//jc8+iiMHg0jRgQdjYhI2imZEpHM+eILuPJKGDQIfv/7oKMREckIJVMikhn19e5yMdEoXHstFBYGHZGISEZozJSIZMadd8LChfCnP0GvXkFHIyKSMeqZEpH0e+01mD4dTjwRjjgi6GhERDJKyZSIpNeqVTBxopuU8/zzg45GRCTjlEyJSPo0NLhxUrW17vBeUVHQEYmIZJySKRFJn7vvhnffdQlVnz5BRyMi0iaUTIlIerz9Ntx/P4waBcccE3Q0IiJtRsmUiLTeqlVw+eXQrx9cdFHQ0YiItCklUyLSOnV1cMklUFMDN9wAJSVBRyQi0qY0z5SItM6tt7r5pK6/Hvr2DToaEZE2p54pEWm5f/4THnsMfv5zOPzwoKMREQmEkikRaZnly+Gqq2D//WHcuKCjEREJjJIpEWm+zZvhwguhtNTNJ1WgEQMi0n7pG1BEmsdauPJKqKyEu+6C7t2DjkhEJFDqmRKR5nn4YZg92x3a++53g45GRCRwSqZEJHVz5sDtt8Nhh8GppwYdjYhIVlAyJSKpWbkSxo930x9MmgTGBB2RiEhWUDIlIttXVQXnn+8e33KLG3guIiKABqCLyPZY63qiPvkE7rgDevUKOiIRkayinikR2bZp09yA83POgQMPDDoaEZGso2RKRLbulVdg6lQ49lg45ZSgoxERyUpKpkQkuU8+gcsvh0GD4LLLNOBcRGQrlEyJyJbWr4cLLoCSErjxRiguDjoiEZGspQHoItJUXR1cdBF8+aWb4XznnYOOSEQkqymZEpEYa+Gaa2D+fHe/335BRyQikvV0mE9EYu6/H555Bn79azj66KCjERHJCUqmRMR5/nm480535t6ZZwYdjYhIzlAyJSKwcCFMnAj77+/O4NOZeyIiKVMyJdLeff65O3OvRw+46SYoKgo6IhGRnKIB6CLt2YYNcO657gy+e+6Brl2DjkhEJOcomRJpr2pr4cILoaICJk+Gvn2DjkhEJCcpmRJpj6JRNzZq/ny49looLw86IhGRnKUxUyLtjbVubNTs2XD++XDUUUFHJCKS05RMibQ3DzwAM2bAaafp4sUiImmgZEqkPXn6aZgyBY45BsaNCzoaEZG8oGRKpL145RW46ioYNgwmTICQ/vxFRNJB36Yi7cGcOTB+POyzjxsvVVgYdEQiInlDyZRIvlu40E3K2bs33HEHlJYGHZGISF5RMiWSzz76CH7/e9hxRzdWqkuXoCMSEck7SqZE8lVFBYwZAx06uAsYd+8edEQiInlJyZRIPvr8c/jd79ycUlOnQs+eQUckIpK3NAO6SL754gv4zW+gqgruugt23z3oiERE8pqSKZF88sUX8Otfw8aNrkdqwICgIxIRyXs6zCeSL778MpZI3XmnmwZBREQyTj1TIvngyy/h7LNhwwaXSA0cGHREIiLthnqmRHKdP7TnE6lBg4KOSESkXUkpmTLGjDTGLDHGLDPGjE/y+vnGmMXGmPeMMS8aYzTiVaQtVFTAmWe6RGrKFCVSIiIB2G4yZYwJA1OAY4BBwGhjTOI39gKg3Fq7H/A4cEO6AxWRBMuWuUSqthbuvhsGDw46IhGRdimVnqlhwDJr7XJrbS3wKDAqfgVr7UvW2qrGp28BvdIbpog0sWiRGyNVUAB//rPO2hMRCVAqydRuwGdxzysbl23Nr4D/a01QIrIN8+fDb38LZWVw773Qt2/QEYmItGtpPZvPGHMqUA58fyuvnw2cDdCnT590Vi3SPrz2GlxyiZvRfMoU2GmnoCMSEWn3UumZWgn0jnveq3FZE8aYI4DLgB9Za2uSFWStvcdaW26tLe/Ro0dL4hVpv556Cs4/H/r3h3vuUSIlIpIlUkmm3gH2Msb0M8YUAScDM+NXMMYcANyNS6RWpT9MkXbMWjcu6uqrYfhwN9h8hx2CjkpERBpt9zCftbbeGDMW+CcQBqZZaxcZY/4IzLXWzgRuBMqAvxljACqstT/KYNwi7UNDA/zpT/Dkk/DDH8Jll7lB5yIikjVS+la21s4CZiUsmxD3+Ig0xyUikQhceim88gqccYYbdO7+WRERkSyif3FFstGqVXDBBbBkCYwfDyeeGHREIiKyFUqmRLLN4sVuoHlVFdx8M/y//xd0RCIisg26Np9INnnxRTjrLCgshGnTlEiJiOQA9UyJZANrXfI0dSrstx/cdBN06xZ0VCIikgIlUyJBq6qCq66C55+HY4+Fyy+HoqKgoxIRkRQpmRIJ0ooVcNFF8Omn8Pvfw2mn6Yw9EZEco2RKJCizZ8OkSVBc7C4NM3Ro0BGJiEgLKJkSaWsNDS55mj4dBg+GG26AnXcOOioREWkhJVMibWnVKjcmav58OOEEN5eUxkeJiOQ0JVMibeWVV9xhvbo6uPJKOO64oCMSEZE0UDIlkmm1tXDrrTBjBuy9N1x3HfTpE3RUIiKSJkqmRDLpk0/cxYmXLoVTToGxY3VYT0QkzyiZEsmEaBQefRQmT4bSUtczdcghQUclIiIZoGRKJN0+/9yNjZo/310O5rLLoHv3oKMSEZEMUTIlki7WwpNPul4ogAkT4Ic/1CScIiJ5TsmUSDqsXOkGlr/1Fgwb5hKpXXYJOioREWkDSqZEWqO+Hh55BO6+G8JhuOQSN39UKBR0ZCIi0kaUTIm01KJFcPXV8NFHcOihcPHFsNNOQUclIiJtTMmUSHNt2AB33QV/+5sbWH7jjfCDHwQdlYiIBETJlEiqolF46il3Xb2NG+GnP4UxY6Bjx6AjExGRACmZEknFu++6CxIvXQrf/S5ceCEMGBB0VCIikgWUTIlsy8qVrifqX/9y46GuvRaOPFLTHYiIyLeUTIkk8803cN998Pjj7iy9M8+EX/wCOnQIOjIREckySqZE4lVXu6kOHnwQIhEYNQrOPht69Ag6MhERyVJKpkTAJVF//7tLotaudWfnjRkDffsGHZmIiGQ5JVPSvlVVuUN5f/mLO7Q3bBj85jew335BRyYiIjlCyZS0T1VVMGMGPPQQrFsHw4fDWWfBd74TdGQiIpJjlExJ+/L11y6JevxxN/nmwQe7weXqiRIRkRZSMiXtw0cfuYHlzz3nrqd36KHwy1/C4MFBRyYiIjlOyZTkr2gU3noLHn4Y3n4bSkrgxz+GU06BXr2Cjk5ERPKEkinJP998A08/7c7Oq6x00xqMHQs/+Ql07hx0dCIikmeUTEl+sBYWLIAnnoDZs6Guzl325Te/gcMPh8LCoCMUEQlEXV0dlZWVRCKRoEPJCSUlJfTq1YvCZvxuKJmS3LZqlRsH9fTT8Mkn0KkTnHCC64Xq3z/o6EREAldZWUmnTp3o27cvRpfC2iZrLWvWrKGyspJ+/fql/D4lU5J7qqrgpZfg2WfhnXdcr9R++8HEie66eSUlQUcoIpI1IpGIEqkUGWPYcccdWb16dbPep2RKckNtLcyZ4y44PHu2u9RLz55uWoNjjoE+fYKOUEQkaymRSl1LtpWSKcletbXw5pvw4ovw8suwebM7jHfssXDcca43Sl8QIiISMCVTkl02bnQJ1Msvw6uvukN6nTu7QeRHHAFDh2owuYiIZBUlUxIsa2H5cnj9dXjtNXj3XTc/VNeucPTRLokqL4cCfVRFRHLRihUrGDlyJMOHD+eNN95g6NChnH766UycOJFVq1bx8MMPM3jwYMaNG8f7779PXV0dkyZNYtSoUaxYsYLTTjuNzZs3AzB58mQOPvhg/v3vfzNp0iS6d+/O+++/z/e+9z0eeuihwA5n6hdK2t7GjW4agzfecEnUF1+45QMGuFnJDzkE9t0XQqFAwxQRyTs33wxLlqS3zL33hgsu2OYqy5Yt429/+xvTpk1j6NChPPLII7z22mvMnDmTa6+9lkGDBnHYYYcxbdo01q1bx7BhwzjiiCPYaaedeP755ykpKeGjjz5i9OjRzJ07F4AFCxawaNEievbsyYgRI3j99dc55JBD0tu2FCmZksyrrob//MedeffOO/Dhh673qUMHGDYMzjgDRoyAnXYKOlIREcmAfv36MWTIEAAGDx7M4YcfjjGGIUOGsGLFCiorK5k5cyY33XQT4M5ArKiooGfPnowdO5Z3332XcDjM0qVLvy1z2LBh9Gq8msX+++/PihUrlExJHtm4ERYuhPfeg3nz3OP6egiHYcgQdwZeebnrfSoqCjpaEZH2Yzs9SJlSXFz87eNQKPTt81AoRH19PeFwmCeeeIK99967yfsmTZrEzjvvzH/+8x+i0SglcVPfxJcZDoepr6/PcCu2TsmUtI618Nlnrufpvffc/SefuOWhkOv+PeUUN3B8//1db5SIiEico48+mjvuuIM77rgDYwwLFizggAMOYP369fTq1YtQKMSDDz5IQ0ND0KEmpWRKUmctrFwJH3zgDtX5+w0b3OudOrmep6OPdtMWDB4MpaXBxiwiIlnviiuu4Nxzz2W//fYjGo3Sr18/nnnmGX73u99xwgknMH36dEaOHEnHjh2DDjUpY60NpOLy8nLrB5FJFqqthU8/hY8/doMVfeK0aZN7vaAA9twT9tkHBg1yvU59+2rQuIhIlvnggw8YOHBg0GHklGTbzBgzz1pbnmx99Uy1dw0NUFHhpif4+OPYraLCDRIHN65pzz1dj9M++8DAge66dxrvJCIiomSqXYhG4auv3Nimigp3849XrnSDw8HNJt67t0uUDj8c9tjDPe7bV/M8iYiIbIV+IfOBtbB+vZuv6Ysv4MsvY48rKqCy0h2284qL3bXs9tgDfvADlzDtsYdLmuLOjhAREZHtUzKV7ax1A7y//jp2W7Vqy6QpEmn6vtJS2GUX19M0YoS779PH3Xr00DXtRERE0kTJVBCsdRftXbcudlu7NpYsrV7dNHmqq9uyjK5dYdddoV8/OOgg9zj+1qmTEiYREZE2oGSqNax1s3tv2uQmqvT3Gze63iSfKH3zTdPEad262DilRJ07Q/fu7nbAAe6+R4/YMv88buIyERERCU77S6asdYfEqqvdrapq2483bWqaJMUnTZs2xc54S8YYlxx17Qo77AC9erlZv/3zrl1jtx12cImSzpATERFpkUMPPZSbbrqJ8vKkMxhkTErJlDFmJHAbEAbutdb+KeH1YmA68D1gDXCStXZFekNNXUMDVL8+H6ZMxlZFCNdWUVhbBTUR7OZqGhpSn1srWlJKtLSMaMdO7r60B9Fd+xPd0z9vfM2/3njfUNaFaFnn1OddWtd4S6Nw2I0nLyhwHWGRyLZzvyDLTVeZLS0nlfdlqt2lpVBW5h7X17ubMe6+psbVY0ysbmvdun4y+epqd++fb9rkjiL72NIRd2IZdXVQWNh0WU1NbPnW6vLt7dLFda5GIi7ehgb3Ht++ggJXdmGhK7O+3i1LfF5S4soEV8bmzbGp0MrKmtaRuI0ikaZtiK/Xd/xGIq5dNTWxzmS/TkFB43dNdSx+vxxi28RfSSlxf5WVgZ9/MBJJ3qZkGhpi6/sj+T727b13a+X490Lybbitbd6c2BPrT6wr1e3RmnrTWYaIt91kyhgTBqYARwKVwDvGmJnW2sVxq/0K+MZau6cx5mTgeuCkTAS8PQ0N7ihaZHMBHUPF0KMrNUWl1BSUYotKMGWlREKlVDWUUF9USkNxKdGiEhpKSon6x8Wx5S2ahNICGxtvAfE/WtGo2ybhsLttrzMtiHLTVWZLy0nlfZlqd6dO7odq7Vr3Y+I7Jtevj33Zh8Pux9IPndthB7duRYV73qOHu6+ocO8pKnIdqxs3uh/b1sad2PaiItfhun69S0a8ujr3Y7h2rTt5NLEu395o1L3uyyoqcgmGP+G0Sxf3A2etGz7Ytatbp7a26fOGBtfGTp3c+9atc+3229DHUVTkEhk/pVpDg3seCrkhiYWFLjZfbygUO58jHG76g+/58qqr3ft8QpV4taTq6lgiFb+/wmG3/fz+LSpyccS3KdkPu389HHZxrlvnlnft6rbrtt67tXJ8vevWufuamqbbsKEBdtwx+TZPLKM59Sfur1S3R7LYU613W+1vbhn5LN2J5ubNm/nZz35GZWUlDQ0NXHHFFSxZsoSnn36a6upqDj74YO6++26MMRx66KEccMABvPrqq2zevJnp06dz3XXXsXDhQk466SSuvvpqVqxYwciRI/ne977H/PnzGTx4MNOnT6c04Yob//rXv5g4cSI1NTXsscce3H///ZSVlTF+/HhmzpxJQUEBRx111LcXV26NVHqmhgHLrLXLAYwxjwKjgPhkahQwqfHx48BkY4yxAUyvHom4L/Wavfej9tqphMOxo3PgPhDGxP4z3SYLZOdlgLbL//j4yxj5/47D4eTj2YMsN11ltrScVN6XqXYb435wi4vdvTGuTJ9weMa4Hzmf1GzaFOuV8PUbE4sxFIp91lsbd2LbffJTXOz+hnySVlIS63XxPWzxdfn2RqNNe3N8b1ZdnVsWjbp1N2509/69iff+y93/HScu9/UUFLhlvoevttbF7nv7IhF37+v1cfgy6upc26qq3LLS0lhyVVLiHpeUuDZs2hS7glJVVaynJXF/+SRy40a3jk/C4tuU7KoZPtnwSZ5PROLX39p7t1aOr7euzpXpe0l9+/02SLbNE8toTv2JZaa6PZLFnmq922p/c8vIV5lINJ977jl69uzJs88+C8D69es58sgjmTBhAgCnnXYazzzzDD/84Q8BKCoqYu7cudx2222MGjWKefPm0a1bN/bYYw/OO+88AJYsWcJ9993HiBEjOOOMM7jzzju58MILv63z66+/5uqrr+aFF16gY8eOXH/99dxyyy2MGTOGJ598kg8//BBjDOv8fyStlEq3y27AZ3HPKxuXJV3HWlsPrAd2TCzIGHO2MWauMWbu6tWrWxbxdvj/aP2hEBdT0y8F/5pfLx9voVDsB8HzP3rZVm66ymxpOam8L1Pt9j1OoVDss+l7R/xyvz7Enkci7nEoFEtcQqEtP+PpiDuxDN9z4xOP+B9bH1eyuuLb4t/n2xu/jn/ue0j84bX6+qbPoemhUd9mz9pYHf79fhv5en0b4uuN/w6JL9cv84lHfJvjk8b4NvrlifsrPnYfX2KbkvExxj+OX39b791aOfHbK3F5/D5Jts0Ty2hO/Yn7K9Xt0Zp601lGvkqWaPrPcEsNGTKE559/nksuuYRXX32VLl268NJLL3HggQcyZMgQZs+ezaJFi75d/0c/+tG37xs8eDC77rorxcXF9O/fn88+c+lI7969GTFiBACnnnoqr732WpM633rrLRYvXsyIESPYf//9efDBB/n000/p0qULJSUl/OpXv+Lvf//7Fr1ZLdWmA9CttfcA94C7Nl8m6igocP8BGhP7Ijcm9gXvnzc05PfMAf6wSvyPoH/emnZnotx0ldnSclJ5X6babYz77y8adffxPVN+efyXvn+PH8cCsTE60WjTHikfW2vjTmx7NOrqr62N/ecaDrvn8XEl1uVj921OjM/H7MciFRfH6vDtjH8OsfWh6d+8fx4KxdpZUBDbRr5eX140GqvX3/v4E78vGhpiMfg2+/X9PvSP/fJk+8vXlbgf4tuUyMcWDjeNM7HM7YkvxzNmy+Xx+yRZfIllNKf+xP2V6vZIFnuq9aazjHxVX9/0bwxaf0RjwIABzJ8/n1mzZnH55Zdz+OGHM2XKFObOnUvv3r2ZNGkSkbhsrbhx8uhQKPTtY/+8vjHjNQkflMTn1lqOPPJI/vrXv24Rz5w5c3jxxRd5/PHHmTx5MrNnz2554xql8tFZCfSOe96rcVmydSqNMQVAF9xA9DbnB5yGQk0PefjucD+eJPEPKd80NGx9rExr2p2JctNVZkvLSeV9mWq3tbHxNx06xL7E/GfWf79Y65IL/3kuK3PjWSD2Hmtjh4/8Z3xbY6ZSjTux7eBijT/MBK4nqWNH96XrD63F1+Xb69vm2+IPj/nXfE9Yp07u8IL/x7Gw0NXpnzc0xGIDt63ix+CEQrGxUT4B8tvIJ1S+9yu+3uLipv98+fj8skjEbf9kY6bKymLbo6wsNmYqcX81NLjvJ3/oxO+bxDYlKimJDVkoKWk6Zmp7791aOb7ewkIXT/yYKd+b6D9jiXUkltGc+hP3V6rbI1nsqdabzjLyVSYSzc8//5xu3bpx6qmn0rVrV+69914AunfvzqZNm3j88cc58cQTm1VmRUUFb775JgcddBCPPPIIhxxySJPXhw8fzpgxY1i2bBl77rknmzdvZuXKlfTs2ZOqqiqOPfZYRowYQf/+/VvesDipbJ53gL2MMf1wSdPJwCkJ68wEfgG8CZwIzA5ivBS4D0DXru6P3w9oLCtzk4GD+1IsKGj6pZevwuEtz+LyA0ezrdx0ldnSclJ5X6banexsvp133v7ZfH36uDL8OCX/PPEzno64E8uoq3MzeSQ7m2/nnbdel29v4tl8HTq49RLPqvMD9P24qp49Y88LClxZ/kvf/9378Uzdum15Nl/8NopEmrbB1+uTCoiNofHtgeRn83XokPxsvuLi2CGl+P3V0OC2Qfw4oGRtSrYfOnWKDQ7u0sUt9z1r23rv1srx9Xbt6l6LH3Dvt+HWtnliGc2pP3F/pbo9ksWear3pLCNfZSLRXLhwIRdddBGhUIjCwkKmTp3KU089xb777ssuu+zC0KFDm13m3nvvzZQpUzjjjDMYNGgQv/3tb5u83qNHDx544AFGjx5NTU0NAFdffTWdOnVi1KhRRCIRrLXccsstLW9YHJNKzmOMORa4FQgD06y11xhj/gjMtdbONMaUAH8BDgDWAif7AetbU15ebufOndva+EVERGQbPvjgAwYOHJjy+g0N2T1txIoVKzj++ON5//33M1ZHsm1mjJlnrS1Ptn5KHXfW2lnArIRlE+IeR4CfNjtaERERySrhsM5qbK4WTKIkIiIiEoy+fftmtFeqJZRMiYiIiLSCkikREZE8F9A5YTmpJdtKyZSIiEgeKykpYc2aNUqoUmCtZc2aNZT4i1WmSFOUiYiI5LFevXpRWVlJpq48km9KSkro1atXs96jZEpERCSPFRYW0q9fv6DDyGs6zCciIiLSCkqmRERERFpByZSIiIhIK6R0OZmMVGzMauDTQCrPTd2Br4MOQrag/ZJ9tE+yk/ZL9tE+aZ7drbU9kr0QWDIlzWOMmbu1awJJcLRfso/2SXbSfsk+2ifpo8N8IiIiIq2gZEpERESkFZRM5Y57gg5AktJ+yT7aJ9lJ+yX7aJ+kicZMiYiIiLSCeqZEREREWkHJVA4yxlxgjLHGmO5Bx9LeGWNuNMZ8aIx5zxjzpDGma9AxtWfGmJHGmCXGmGXGmPFBx9PeGWN6G2NeMsYsNsYsMsacE3RM4hhjwsaYBcaYZ4KOJR8omcoxxpjewFFARdCxCADPA/taa/cDlgJ/CDiedssYEwamAMcAg4DRxphBwUbV7tUDF1hrBwHDgTHaJ1njHOCDoIPIF0qmcs//AhcDGuyWBay1/7LW1jc+fQto3qXGJZ2GAcustcuttbXAo8CogGNq16y1X1hr5zc+3oj78d4t2KjEGNMLOA64N+hY8oWSqRxijBkFrLTW/ifoWCSpM4D/CzqIdmw34LO455XohztrGGP6AgcAbwccisCtuH/KowHHkTcKgg5AmjLGvADskuSly4BLcYf4pA1ta59Ya//RuM5luEMaD7dlbCK5wBhTBjwBnGut3RB0PO2ZMeZ4YJW1dp4x5tCAw8kbSqayjLX2iGTLjTFDgH7Af4wx4A4nzTfGDLPWftmGIbY7W9snnjHml8DxwOFWc40EaSXQO+55r8ZlEiBjTCEukXrYWvv3oOMRRgA/MsYcC5QAnY0xD1lrTw04rpymeaZylDFmBVBurdVFKgNkjBkJ3AJ831q7Ouh42jNjTAHuJIDDcUnUO8Ap1tpFgQbWjhn3n9+DwFpr7bkBhyMJGnumLrTWHh9wKDlPY6ZEWmcy0Al43hjzrjHmrqADaq8aTwQYC/wTN9B5hhKpwI0ATgMOa/z7eLexR0Qkr6hnSkRERKQV1DMlIiIi0gpKpkRERERaQcmUiIiISCsomRIRERFpBSVTIiIiIq2gZEpERESkFZRMiYiIiLSCkikRERGRVvj/bi8AZ2bV5zkAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot the data and the mean of the distribution\n", "plt.scatter(X_train, y_train, alpha=0.04, color='blue', label='samples')\n", "plt.plot(X_train, model(X_train).mean().numpy().flatten(), color='red', alpha=0.8, label='mean')\n", "plt.legend(loc='best')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a new probabilistic model with the wrong weights" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"probabilistic_sigmoid_model\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense_5 (Dense) (None, 1) 2 \n", "_________________________________________________________________\n", "distribution_lambda_1 (Distr multiple 0 \n", "=================================================================\n", "Total params: 2\n", "Trainable params: 2\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model_untrained = Sequential([\n", " Dense(units=1, input_shape=(1, ), activation='sigmoid',\n", " kernel_initializer=tf.constant_initializer(2),\n", " bias_initializer=tf.constant_initializer(2)),\n", " tfpl.DistributionLambda(lambda t:tfd.Bernoulli(probs=t),\n", " convert_to_tensor_fn=tfd.Distribution.sample)\n", "], name='wrong_probabilistic_model')\n", "\n", "model.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train the new model with the negative log likelihoood" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "# Define negative log likelihood,\n", "def nll(y_true, y_pred):\n", " return -y_pred.log_prob(y_true)" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "# Compile untrained model\n", "\n", "model_untrained.compile(loss=nll, optimizer=RMSprop(learning_rate=1e-2))" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "# Train the model, record weights after each epoch\n", "epochs = [0]\n", "training_weights = [model_untrained.weights[0].numpy()[0, 0]]\n", "training_bias = [model_untrained.weights[1].numpy()[0]]\n", "\n", "for e in range(100):\n", " model_untrained.fit(X_train, y_train, epochs=1, verbose=False)\n", " epochs.append(e)\n", " \n", " training_weights.append(model_untrained.weights[0].numpy()[0, 0])\n", " training_bias.append(model_untrained.weights[1].numpy()[0])" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAFzCAYAAAANJxyKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABUvUlEQVR4nO3deXxU1d3H8c/JZN9XtiSQAGFfAgQUWYwoitZdVFxatVrqVmr71Fbbqn1qbX0qbV1aa61abLWuaEVbrYIbiAoBwr6vCWsWsu/Jef64kxAgQICZTJbv+/Wa1+Quc+c3mZk73zn3zLnGWouIiIiIeJafrwsQERER6YwUskRERES8QCFLRERExAsUskRERES8QCFLRERExAsUskRERES8wN/XBbQkPj7epqSk+LoMERERkRNatmxZvrU24cj57TJkpaSkkJWV5esyRERERE7IGLOzpfk6XCgiIiLiBQpZIiIiIl6gkCUiIiLiBe2yT5aIiIi0rLa2ltzcXKqqqnxdSpcTHBxMUlISAQEBrVpfIUtERKQDyc3NJSIigpSUFIwxvi6ny7DWUlBQQG5uLqmpqa26jQ4XioiIdCBVVVXExcUpYLUxYwxxcXEn1YKokCUiItLBKGD5xsn+3xWyREREpE3cdtttrFu37rjr3Hzzzbz55ptHzd+xYwf//Oc/vVWaVyhkiYiISJt47rnnGDJkyCndtlOGLGNMsjHmE2PMOmPMWmPM91tYxxhjnjTGbDHGrDLGjG627CZjzGb35SZPPwARERFpW4899hhPPvkkAD/4wQ+YMmUKAB9//DE33HADH374IePHj2f06NFcffXVlJWVAZCZmdl0Rpfnn3+eAQMGMG7cOL7zne9w9913N23/888/56yzzqJv375NrVr33XcfCxcuJD09nT/84Q9t+XBPWWt+XVgH/I+1drkxJgJYZoz5yFrbvL3vQiDNfTkD+DNwhjEmFngIyACs+7bzrLUHPfooREREuqD/fXct6/aUeHSbQ3pF8tAlQ4+7zqRJk/jd737HrFmzyMrKorq6mtraWhYuXMiIESP41a9+xfz58wkLC+P//u//+P3vf8+DDz7YdPs9e/bw8MMPs3z5ciIiIpgyZQojR45sWr53714WLVrEhg0buPTSS5k+fTqPPvoos2fP5r333vPo4/WmE7ZkWWv3WmuXu/8uBdYDiUesdhnwd+v4Cog2xvQELgA+stYWuoPVR8A0jz6CU7D+6/+yOXuhr8sQERHpkMaMGcOyZcsoKSkhKCiI8ePHk5WVxcKFCwkJCWHdunVMmDCB9PR0XnzxRXbuPPzUfkuWLOHss88mNjaWgIAArr766sOWX3755fj5+TFkyBD279/flg/No05qnCxjTAowCvj6iEWJQE6z6Vz3vGPNb2nbM4GZAL179z6Zsk5a4Ic/oSSoJ6S/79X7ERER8aYTtTh5S0BAAKmpqcyZM4ezzjqLESNG8Mknn7BlyxZSU1OZOnUqr7zyyilvPygoqOlva60nSvaJVnd8N8aEA3OBe6y1nm2bBKy1z1prM6y1GQkJCZ7evIiIiHjQpEmTmD17NpMnT2bSpEk888wzjBo1ijPPPJMvvviCLVu2AFBeXs6mTZsOu+3YsWP57LPPOHjwIHV1dcydO/eE9xcREUFpaalXHou3tCpkGWMCcALWy9bat1pYZTeQ3Gw6yT3vWPNFRESkA5s0aRJ79+5l/PjxdO/eneDgYCZNmkRCQgJz5szhuuuuY8SIEYwfP54NGzYcdtvExER++tOfMm7cOCZMmEBKSgpRUVHHvb8RI0bgcrkYOXJkh+n4bk7UDGeckbdeBAqttfccY51vAHcDF+F0fH/SWjvO3fF9GdD4a8PlwBhrbeHx7jMjI8M2/vrAG7Y+nE5JUE9G/ViHC0VEpGNZv349gwcP9nUZp62srIzw8HDq6uq44oor+Pa3v80VV1zh67JOqKX/vzFmmbU248h1W9MnawLwTWC1MSbbPe+nQG8Aa+0zwH9wAtYWoAK4xb2s0BjzMLDUfbtfnihgiYiISOf3i1/8gvnz51NVVcX555/P5Zdf7uuSPO6EIctauwg47jjy1mkOu+sYy14AXjil6kRERKRTmj17tq9L8DqN+C4iIiLiBQpZIiIiIl6gkCUiIiLiBQpZIiIiIl6gkCUiIiInZceOHQwbNuyo+bfddhvr1q1r4RZd00mdVkdERETkWJ577jlfl9CuqCVLRERETlpdXR033HADgwcPZvr06VRUVJCZmUnjYOJ33HEHGRkZDB06lIceeqjpdvfddx9DhgxhxIgR/OhHP/JV+W1CLVkiIiId1fv3wb7Vnt1mj+Fw4aMnXG3jxo08//zzTJgwgW9/+9s8/fTThy1/5JFHiI2Npb6+nnPPPZdVq1aRmJjI22+/zYYNGzDGUFRU5Nna2xm1ZImIiMhJS05OZsKECQDceOONLFq06LDlr7/+OqNHj2bUqFGsXbuWdevWERUVRXBwMLfeeitvvfUWoaGhvii9zaglS0REpKNqRYuTtzinNm55evv27cyePZulS5cSExPDzTffTFVVFf7+/ixZsoQFCxbw5ptv8sc//pGPP/64rUtvM2rJEhERkZO2a9cuvvzySwD++c9/MnHixKZlJSUlhIWFERUVxf79+3n//fcB56TQxcXFXHTRRfzhD39g5cqVPqm9rXTJkGWAmroGX5chIiLSYQ0cOJA//elPDB48mIMHD3LHHXc0LRs5ciSjRo1i0KBBXH/99U2HFUtLS7n44osZMWIEEydO5Pe//72vym8TXfJwYVRIAFuKa/l6WwFn9I3zdTkiIiIdSkpKChs2bDhq/qefftr095w5c1q87ZIlS7xUVfvTJVuyYkIDCfL343cfbcJa6+tyREREpBPqkiHLzxhS48NZsr2QL7YU+LocERER6YS6ZMgCSIwJoVdUML/7aKNas0RERMTjumzIchnD3VPSWLGriE835vm6HBEREelkumzIArg6I4nk2BB+r75ZIiIi4mFdOmQFuPyYNSWN1buL+XDdfl+XIyIiIp1Ilw5ZAFeMSqRvfBh/+GgTDQ1qzRIRERHP6PIhy9/lx/fPS2PDvlL+s2avr8sRERFp94qKio46IXR7kZWVxaxZs467zo4dOxg2bFiLy+bMmcOePXs8UkuXD1kAl4zoxYDu4Tw+fzP1as0SERE5rmOFrLq6Oh9Uc7iMjAyefPLJU769QpaH+fkZZp2bxpYDZfxntVqzRESk48jMzGwaXb22tpbMzExeeuklACoqKsjMzOS1114DoLi4mMzMTN566y0A8vPzyczM5N133wVg3759rbrP++67j61bt5Kens7YsWOZNGkSl156KUOGDDmqlWj27Nn84he/AGDr1q1MmzaNMWPGMGnSpBZHjQeor68nNTUVay1FRUW4XC4+//xzACZPnszmzZspLy/n29/+NuPGjWPUqFG88847gDPq/MUXXwxAXl4eU6dOZejQodx222306dOH/Pz8pvv4zne+w9ChQzn//POprKzkzTffJCsrixtuuIH09HQqKytb9f84FoUst4uG9SStWzhPfbxZfbNERESO49FHH6Vfv35kZ2fz2GOPsXz5cp544gk2bdp03NvNnDmTp556imXLljF79mzuvPPOFtdzuVwMHDiQdevWsWjRIkaPHs3ChQuprq4mJyeHtLQ0HnnkEaZMmcKSJUv45JNPuPfeeykvLz9sO//7v//LlClTWLt2LdOnT2fXrl1NyzZv3sxdd93F2rVriY6OZu7cuUyfPp2MjAxefvllsrOzCQkJOa3/U5c8d2FL/PwM3zs3jVmvrOD9Nfv4xoievi5JRETkhJqfLzAgIOCw6dDQ0MOmo6KiDpuOj48/bLpHjx6nVMO4ceNITU097jplZWUsXryYq6++umledXX1MdefNGkSn3/+Odu3b+f+++/nr3/9K2effTZjx44F4MMPP2TevHnMnj0bgKqqqsNCFMCiRYt4++23AZg2bRoxMTFNy1JTU0lPTwdgzJgx7Nixo9WPt7XUktXMN4b3pF9CGE8uUGuWiIhIa4WFhTX97e/vT0NDQ9N0VVUVAA0NDURHR5Odnd10Wb9+/TG3OXnyZBYuXMiSJUu46KKLKCoq4tNPP2XSpEkAWGuZO3du07Z27drF4MGDW11zUFBQ098ul8sr/ckUsppxuftmbdxfyn/Xtu64tIiISFcTERFBaWlpi8u6d+/OgQMHKCgooLq6mvfeew+AyMhIUlNTeeONNwAnJK1cufKY9zFu3DgWL16Mn58fwcHBpKen85e//IXJkycDcMEFF/DUU081DSa+YsWKo7YxYcIEXn/9dcBp+Tp48OBpPbaTpZB1hItH9KJvQhhPqDVLRESkRXFxcUyYMIFhw4Zx7733HrYsICCABx98kHHjxjF16lQGDRrUtOzll1/m+eefZ+TIkQwdOrSps3pLgoKCSE5O5swzzwScw4elpaUMHz4cgAceeIDa2lpGjBjB0KFDeeCBB47axkMPPcSHH37IsGHDeOONN+jRowcRERHHfWw333wzt99+u0c6vpsTnU7GGPMCcDFwwFp71KASxph7gRvck/7AYCDBWltojNkBlAL1QJ21NqM1RWVkZNisrKxWP4iT9vfLoDwf7viixcVvr8jlB6+t5JkbxzBt2KkdnxYREfGG9evXn9Rhsa6suroal8uFv78/X375JXfccQfZ2dmntc2W/v/GmGUtZZzWtGTNAaYda6G19jFrbbq1Nh24H/jMWlvYbJVz3MtbFbDaRMpE2L8GygtaXHzJiF6kxqs1S0REpCPbtWsXY8eOZeTIkcyaNYu//vWvbXr/JwxZ1trPgcITred2HfDKaVXUFlLPdq53LGxxsb/Lj7vP6c/6vSV8tF7nNBQREfGWRx55hPT09MMujzzyiEe2nZaWxooVK1i5ciVLly5t+mViW/HYEA7GmFCcFq+7m822wIfGGAv8xVr7rKfu77T0GgWB4bD9cxh6eYurXJbei6c+3swT8zczdXB3/PxM29YoIiLSBfzsZz/jZz/7ma/L8ApPdny/BPjiiEOFE621o4ELgbuMMZOPdWNjzExjTJYxJisvL8+DZbXAFQB9Jjgh6xj8XX78YOoA1u0t4c3lud6tR0RERDodT4asGRxxqNBau9t9fQB4Gxh3rBtba5+11mZYazMSEhI8WNYxpE6Ggs1QcuzzE106sheje0fz2w82UlpV6/2aREREpNPwSMgyxkQBZwPvNJsXZoyJaPwbOB9Y44n784hUd6Pa9pb7ZQEYY/jFpUPJL6vmjx9vaaPCREREpDM4YcgyxrwCfAkMNMbkGmNuNcbcboy5vdlqVwAfWmubnzSoO7DIGLMSWAL821r7gSeLPy3dh0FIDGz/7LirjUiK5uoxSbzwxXa25ZW1UXEiIiLS0bXm14XXWWt7WmsDrLVJ1trnrbXPWGufabbOHGvtjCNut81aO9J9GWqt9cxPBTzFzw9SJjn9sk4wVti90wYS5O/ikX8fe/h/ERGRrqKoqIinn37aq/dx88038+abbx41Pysri1mzZnn1vj2la4/4njoZinPg4PbjrtYtIpjvTenPgg0H+HTjgTYqTkREpH06Vsjyxvn/jpSRkcGTTz7p9fvxhC4estzjZR3nV4aNbpmQSmp8GL98bx219Q0nXF9ERKQtZGZmHnVpDEAVFRUtLp8zZw4A+fn5Ry1rjfvuu4+tW7eSnp7O2LFjmTRpEpdeeilDhgxhx44dDBt26AQxs2fP5he/+AUAW7duZdq0aYwZM4ZJkyaxYcOG497P/PnzycjIYMCAAU3nQPz000+5+OKLAViyZAnjx49n1KhRnHXWWWzcuBGAtWvXMm7cONLT0xkxYgSbN29u7b/To7p2yIpPg/AerQpZgf5+/Pwbg9mWV86Li3d4vzYREZF26tFHH6Vfv35kZ2fz2GOPsXz5cp544gk2bdp03NvNnDmTp556imXLljF79mzuvPPO466/Y8cOlixZwr///W9uv/12qqqqDls+aNAgFi5cyIoVK/jlL3/JT3/6UwCeeeYZvv/975OdnU1WVhZJSUmn94BPkccGI+2QjIG+Z8PWj51+Web4A45OGdSNswck8MT8zVw7NpmI4IA2KlRERKRln3766TGXhYaGHnd5fHz8cZe31rhx40hNTT3uOmVlZSxevJirr766aV51dfVxb3PNNdfg5+dHWloaffv2Parlq7i4mJtuuonNmzdjjKG21hluafz48TzyyCPk5uZy5ZVXkpaWdoqP7PR07ZYscPplledB3vGbLMEZ0mHWuWmUVtfx/up9bVCciIhI+xcWFtb0t7+/Pw0Nh7rVNLY+NTQ0EB0dTXZ2dtNl/frj/6DMHNH4ceT0Aw88wDnnnMOaNWt49913m+7r+uuvZ968eYSEhHDRRRfx8ccfn9bjO1UKWU3jZZ34kCHA6N7RpMaHMVejwIuISBcVERFBaWlpi8u6d+/OgQMHKCgooLq6uqkvVWRkJKmpqbzxxhsAWGtZuXLlce/njTfeoKGhga1bt7Jt2zYGDhx42PLi4mISExMBmvqZAWzbto2+ffsya9YsLrvsMlatWnWqD/W0KGRF94aYlFaHLGMMV45K5OvtheQUVni3NhERkXYoLi6OCRMmMGzYMO69997DlgUEBPDggw8ybtw4pk6dyqBBg5qWvfzyyzz//POMHDmSoUOH8s477xy56cP07t2bcePGceGFF/LMM88QHBx82PIf//jH3H///YwaNeqwXza+/vrrDBs2jPT0dNasWcO3vvUtDzzqk2fsCcaI8oWMjAyblZXVdnc473uw7h348Xbwc51w9dyDFUz8v0/44dQBzDrXN8d5RUSka1q/fj2DBw/2dRldVkv/f2PMMmttxpHrqiULnKEcqoph7/GbLRslxYRyZt9Y3lqeS3sMqSIiIuJ7ClngjPwOrT5kCHDV6CR2FFSwfNdBLxUlIiLS+T3yyCOkp6cfdnnkkfZ1kphT1bWHcGgU0R0SBjkha+I9rbrJhcN78uA7a3lz2W7G9In1bn0iIiLNWGuP+qVdR/Wzn/2Mn/3sZ74uo1VO9uiVWrIapU6GXV9BfW2rVg8P8mfasB68t2oPVbX1Xi5ORETEERwcTEFBgbqrtDFrLQUFBUd1vj8etWQ16jMBljwLe7IheWyrbnLV6CTeXrGb+ev3c/GIXt6tT0REBEhKSiI3N5e8vDxfl9LlBAcHn9To8QpZjfpMcK53LGx1yBrfL44ekcHMXZarkCUiIm0iICDghKOrS/ugw4WNwhMgYTDsWNTqm7j8DFeMTuTzzfkcKK068Q1ERESky1DIai5l4kn1ywK4anQi9Q2Wedl7vFiYiIiIdDQKWc2lTITa8laPlwXQv1sEI5OimLt8txcLExERkY5GIau55v2yTsJVY5JYv7eE1bnFXihKREREOiKFrObCE5zxsk6iXxbA5aMSCQt08cIX271UmIiIiHQ0CllHOoV+WZHBAVw7tjfvrtzDvmJ1gBcRERGFrKOlTISaspPqlwVwy4QUGqzlxS93eKcuERER6VAUso7UZ6JzfZL9spJjQ7lgaA/++fUuKmrqvFCYiIiIdCQKWUdq6pf1xUnf9LZJqRRX1jJ3Wa4XChMREZGORCGrJX0mwK4vof7kWqRG944hPTma5xdtp6FB55QSERHpyhSyWnKK/bKMMdw2KZUdBRUs2HDAS8WJiIhIR6CQ1ZKUU+uXBTBtaA8So0N4buE2DxclIiIiHYlCVkvCu0H8wJMeLwvA3+XHzWel8PX2Qtbs1uCkIiIiXZVC1rGkTDylflkA145LJizQxfOLNDipiIhIV3XCkGWMecEYc8AYs+YYyzONMcXGmGz35cFmy6YZYzYaY7YYY+7zZOFelzLhlPplweGDk+4trvRCcSIiItLetaYlaw4w7QTrLLTWprsvvwQwxriAPwEXAkOA64wxQ06n2DbVOF7WzpM/ZAjO4KR+xvDIv9d7sCgRERHpKE4Ysqy1nwOFp7DtccAWa+02a20N8Cpw2SlsxzciukP8gFPqlwXO4KR3ndOf91bt5ZON+qWhiIhIV+OpPlnjjTErjTHvG2OGuuclAjnN1sl1z+s4UibCzsUndR7D5m7P7Eu/hDB+/vYajQIvIiLSxXgiZC0H+lhrRwJPAf86lY0YY2YaY7KMMVl5eXkeKMsDUs92+mXtXn5KNw/yd/GbK0ewu6iSx+dv9nBxIiIi0p6ddsiy1pZYa8vcf/8HCDDGxAO7geRmqya55x1rO89aazOstRkJCQmnW5ZnpE4GDGz79JQ3MS41lhljk3l+0XbW7tGQDiIiIl3FaYcsY0wPY4xx/z3Ovc0CYCmQZoxJNcYEAjOAead7f20qNBZ6pZ9WyAK4/8LBxIQGcP9bq6nX6XZERES6hNYM4fAK8CUw0BiTa4y51RhzuzHmdvcq04E1xpiVwJPADOuoA+4G/gusB1631q71zsPwor6ZkLsEqstOeRNRoQE8cPEQVuUW8/cvd3isNBEREWm/jLXtr2UlIyPDZmVl+boMx7ZP4e+XwfVvwIDzT3kz1lpu+ttSlu0o5KMfnk2v6BDP1SgiIiI+Y4xZZq3NOHK+Rnw/keQzwT/4tA8ZGmN45PJh1FvLb97f4JnaREREpN1SyDqRgGDofeZphyxwxs6aObkf767cw7KdpzL0mIiIiHQUClmt0TcTDqyF0v2nvanbz+5Lj8hg/vfddTSoE7yIiEinpZDVGn0znevtn5/2pkID/fnJhQNZlVvM2yuOOaKFiIiIdHAKWa3RYwSExMC2TzyyuctGJjIyOZrf/ncD5dUaCV5ERKQzUshqDT+XM/r7tk/BA7/G9PMzPHjxEPaXVPOXz7aefn0iIiLS7ihktVbfTCjZDQVbPLK5MX1iuCy9F3/5fBu5Bys8sk0RERFpPxSyWquxX5YHfmXY6CfTBmEM/N8HGz22TREREWkfFLJaKzYVovt4NGT1ig5pGtIha4eGdBAREelMFLJORt9M2L4Q6j3XWb1xSIeH39OQDiIiIp2JQtbJ6JsJ1cWwN9tjmwwN9OfeCwayMreYd1ZqSAcREZHOQiHrZKSe7Vx7aCiHRleMSmREUhS//WAjlTX1Ht22iIiI+IZC1skIi3PGzNr2mUc36+dn+Pk3hrC3uIpnP9/m0W2LiIiIbyhknax+58Cur6Cm3KObHZcay0XDe/DMZ1vZV1zl0W2LiIhI21PIOln9zoWGWo+3ZgHcN20w9Q2Wx/6rIR1EREQ6OoWsk9V7PARFwqYPPL/puFBumZjC3OW5rM4t9vj2RUREpO0oZJ0s/0DnkOHmDz1yip0j3XVOf+LCAnn4vXVYL2xfRERE2oZC1qlIuwBK98K+VR7fdGRwAD+YOoAlOwp5f80+j29fRERE2oZC1qlImwoY2PShVzY/Y2wyg3pEcN/cVWzeX+qV+xARERHvUsg6FeHdIHE0bP6vVzbv7/Ljr9/KICjAxU0vLGFvcaVX7kdERES8RyHrVKVdALlZUJ7vlc0nx4Yy55axlFTVcfMLSymuqPXK/YiIiIh3KGSdqgEXABY2f+S1uxjaK4pnvzmGbfllfOfvWVTVajR4ERGRjkIh61T1HAnhPbwylENzZ/WP5/fXpLNkRyH3vJpNvU4iLSIi0iEoZJ0qY2DA+bD1Y6j37qG8S0b24oGLh/DB2n38/F+rFbREREQ6AIWs05F2AVSXOKfZ8bJbJ6Zy1zn9eGVJDrNeWUF1nQ4dioiItGcKWaejbya4Ar1+yLDRvRcM4mcXDebfq/dy8wtLKalSZ3gREZH2SiHrdASFQ8pEZ/T3NvKdyX15/Np0lu4o5Nq/fMWBEp1MWkREpD1SyDpdaRdA/iYo3NZmd3n5qESev3ksOwvKufLPi9mWV9Zm9y0iIiKto5B1ugac71x7afT3Yzl7QAKvfOdMKmvqueSpRfzm/fXkl1W3aQ0iIiJybCcMWcaYF4wxB4wxa46x/AZjzCpjzGpjzGJjzMhmy3a452cbY7I8WXi7EdsX4gd4bfT34xmZHM2/7prAlMHdefbzbUz8v4/55bvr2K9DiCIiIj7XmpasOcC04yzfDpxtrR0OPAw8e8Tyc6y16dbajFMrsQNIOx92LILqtj/PYHJsKE9dN4r5PzybbwzvxYtf7mDSbz/hf99dS01dQ5vXIyIiIo4Thixr7edA4XGWL7bWHnRPfgUkeai2jmPABVBf4wQtH+mXEM7vrhnJJ/+TyZWjEvnbFzu48+VlGupBRETERzzdJ+tW4P1m0xb40BizzBgz83g3NMbMNMZkGWOy8vLyPFyWlyWfAQFhsGWBryuhd1woj141gocvG8r89Qe4/R/LdDoeERERH/BYyDLGnIMTsn7SbPZEa+1o4ELgLmPM5GPd3lr7rLU2w1qbkZCQ4Kmy2oZ/kDOUw9aPfV1Jk2+OT+HXVwznk415fFdBS0REpM15JGQZY0YAzwGXWWsLGudba3e7rw8AbwPjPHF/7VK/KVC4FQ7u8HUlTa4/oze/vWoEn2/O4zt/z6KyRkFLRESkrZx2yDLG9AbeAr5prd3UbH6YMSai8W/gfKDFXyh2Cv3Pda7bUWsWwDVjk3ls+kgWbcnn23OWUqpR4kVERNpEa4ZweAX4EhhojMk1xtxqjLndGHO7e5UHgTjg6SOGaugOLDLGrASWAP+21rbN+Wd8Ia4/RCW3i35ZR5o+JonfXzOSJTsKueLpxWzPL/d1SSIiIp2esdb6uoajZGRk2KysDjis1rxZsPZt+PF2cPn7upqjfLm1gDtfXkZ9g+WP149m8oAO1vdNRESkHTLGLGtpqCqN+O5J/aZAdQnsXubrSlo0vl8c8+6eSK/oEG7+2xKeW7iN9hiyRUREOgOFLE/qezYYP9ja/g4ZNkqODWXuHWcxdUh3fvXv9fzojVU6HY+IiIgXKGR5UkgMJI5pd53fjxQW5M+fbxjDPeelMXd5Lmf8egG3zlnKf1bv1VAPIiIiHtL+Og51dP2mwOePQeVBJ3S1U35+hnvOG8A3hvdk7vLdvL0ilwUbDhAZ7M8lI3sxfUwS6cnRGGN8XaqIiEiHpI7vnrbra3jhfLj6RRh6ua+rabX6BsvirfnMXZbLB2v3UVXbQFq3cK7OSOKKUUkkRAT5ukQREZF26Vgd3xWyPK2+Dn7bF4ZeBpc+5etqTklpVS3/XrWX17NyWL6rCJef4ZyB3Zh1bn9GJEX7ujwREZF2Rb8ubCsuf+g7GbZ+Au0wwLZGRHAAM8b15q07JzD/h2dz26RUsnMOctWfF/Pi4h36RaKIiEgrKGR5Q78pUJwDBVt8Xclp698tnPsvHMz8H57NpLQEHpq3lu+9soKy6jpflyYiItKuKWR5Q78pznU7HP39VEWHBvLctzL48bSB/Gf1Xi794yI27iv1dVkiIiLtlkKWN8SkQGy/dj+Uw8ny8zPcmdmff37nTEqr6rjsT4t4J3u3r8sSERFplxSyvKXfFNixEOo630CfZ/aN49+zJjIyKZrvv5rNkws2q5+WiIjIERSyvKX/uVBbAbu+8nUlXtEtIph/3HoGV45K5PcfbeInc1dRW9/g67JERETaDQ1G6i0pE8HPH7Z94pxupxMK9Pfjd9eMJCk2lCcXbGZvcRVP3zCaiOCAU9regZIqtuWXM6ZPDAEu5X/xrvoGy8rcIj7bmMdnm/LYX1LFeYO7c8nIXmT0icHPr+MPxJtXWs3yXQdZsauIjftKGNMnhqvGJNEzKsTXpYl0CRony5v+dhHUlMF3P/d1JV73+tIcfvr2avp3C+dXlw+juq6BgxU1FFXUUlRRQ4OFXtEhJMWEkBgdQs+oYOqtJWvHQT7f5HzIbXB3pO8ZFczNZ6UwY1xvokJOLbCJtKSqtp6P1u3nv2v3sXBzPsWVtfgZGJkcTbeIID7blEdVbQM9o4K5eERPMgd2o7Sqlr3FVewrrmJvcRX5ZdW4/AyBLj8CXH4E+vsREuAiIyWG8wZ3JyYs0GePr6Sqlk82HODjDQdYtvMguQcrAQhwGZJjQ9mWV44xMCktgWsykpg6pDtB/q4TbtdaS3VdAxU19ZRX11FRU09FTR0FZTXkHKxgV2EFOYWV5BRWUNvQwOS0BKYM6sYZfWNbtf2uoqSqlrzSamJDA4kKCfBYkK+pa2DzgVLW7i5h9e5iNu0vJcDlR3iQPxHB/kQEBxAe7E9ksD/hQf6Eu68jgv2JDg0kPiyIyBD/pjN8WGvZU1zFil0Hyd5VxIqcIvYVV+HvMrj8DP5+BpefHxFB/gzoEc7AHpEM7hHBgB4RRJ7il+yOToOR+sLnj8HHv4J7t0JYvK+r8bqFm/O446XlrRreweXnvFlr6hoIcBnG9Ilh8oAEkmNC+efXu/hyWwFhgS6uGZvMtyekkhwb2gaPwDeqauuxFkIC9WHkDdZalu8q4s1luby3ag+lVXUkRASROSCBswcmMLF/PNGhTjAqr65j/vr9zMvew+eb86itP7R/DHT50SMqmISIIBqspba+gZo651JSVUdheQ0uP8OZfWOZNqwnFwzpTrfI4FbXWVVbz4Z9pazeXcza3cXsLa4CoPHMVgYIDnCRHBtKcmwofWJD6R0bSlCAHwvWH+DDdfv5cms+tfWW+PBAxqbEMqp3NKN7xzAsMYrgABc7C8qZuyyXN5flsqe4iqiQAHpEBlPb0EBdvaWuvoHaBue6rt4emt9w/M+JsECnrqSYUBqsc/aIqtoGwgJdTEpLYMrgbkwZ1I348K555oja+gb+/uVOHv9oE6Xu/aMxEB0SQExoID2jgxnUI5LBPSMZ3DOC/t3CjwqnDQ2WkqpadhRUsCO/nO355ewoKGdrXhmb9pVR4+6uER7kz4Du4QCUVtVRVl3XdH08AS5DbFggcWFB5JdVc6DU6U8c5O/H8MQoesc6z229hXr36+JgRQ0b9pVSWnVo20kxIYxMiiY9OZqRydEMT4wiJNBFfYMlp7CCLQfK2HygjD1FlZzZN47zhnTrFEFcIcsXdi+Dv06Bq56H4dN9XU2byCmsYO2eYqJDA4kJDSQmNICoUOebzd6iKnIPVrK7qILcg5VU1dZzRmoc4/vFERZ0+JHrNbuLeX7Rdt5duYd6a+kbH8bIJOdNOyIpisE9IwkOOPqNWVlTT35ZNfll1RSU1VBQXk1pVR2VNfVU1NZTWVNPVW09EcH+DE+KZmSSs/PwxDka6+obmnZoJVW1lFbVYa2zM/UzpunDcvfBSjbuL2Xz/lI27S8j52AF1kJooIv48CDiwp0d3ZCeEUwfk0zvOO8HzIqaOlbsKuLr7YUs33mQqtp6ggNcBAf4ERTgIsjfjyB/F4EuQ6D/oRaciGDnQ7pHlHPpFhHk80O91lp2FVawZncJq3YX8eHa/WzPLyckwMWFw3pw1ZgkxveNO2ErQlFFDStzi4kLC6RnVDCxYYHHfJ1Ya1m9u5gP1uzjgzX72JZfDjjjzI1IiiI9OZoRSdEM7hkBwM6CCrbllbE1z/mQXL/XeT00hpno0ADnddm4ffd1eXUdOQcrqak7uv9jSlwoFwztwflDuzMq+fiHOxtPo/VO9h7KqurwdzmtE/4uPwJcBn8/P/xdhgCXX9P8IH+nZSQ00EVooD+hQS5iQgNJjgk56n9TVVvP4q35LFh/gAXrD7CvpApjYFRyNOcO7s7UId1J6xbeqvedtbZDn0N18dZ8fjFvLZv2lzEpLZ7L0xMprqzlYEWNcymvJedgBRv3lVLtfl79/QzdI4Oprmuguq6e6tqGphDVyBjoFRVC34QwhvSMZFhiFMMSo+gTG9ric9/QYCmvccJWWVUdpe59VVFFDXml1RSU11BQVk1+WQ2Rwf6M6h3DqN7RDOoRSaD/sd/T1lr2FlexYV8J6/eWsm5PCdk5RewuclpRXX6GxOgQ9pVUHfa6DQlwUVlbT0xoAFeMSuKasUkM6hEJwO6iSr7eVsDX2wpZurOQ6toGIoL9iQwOcLfM+TO0VxRXjE48bnDfXVTJmt3FXDC0R+ufsFOkkOULDfXwWD8YeBFc/rSvq+mQ9hZX8tby3U6zdU4x+WXOtyuXnyHA1di07f4Qshy1I2ouwGUIDnAREuCiqLK26Q0fFRLAiKQo+iWEEx7kT0igi7BAF6FB/sSGBjIyObrFczfW1DWwaEse767cyycbD1BUUdvqx+XvZ+ibEEZa9wgGdIsg0N/PHQydnV1eaTWb9pfSYGF83ziuHZvMtGE9WgyWR2posGwvKGdnQTm7CirYVVjJrsIK9pVUEuTvIizIn/AgF+FB/gS4/Fi7p4Q1u4upa7D4GRjUI5KokACq6+qpqm2gyr2Tr65roLa++eXofYcxEBcWREKEc4kPDyQhIogekcH0TQinb3wYidEhh30IHCitYu1up4bNB8qoqWug3lqstdS7Q0ePqGBS4sLoExdGanwYfeJCqaipZ3ez0L67qJKN+0pZs7uYEvc368ZW0itHJ3HR8J6EB3m/G6q1li0Hyvhw3X6W7zzIytwi8stqmuqpb7A0bxjqERlMWvdwhidGMdz9QZkUE3LMYNHQYDlQWs2uwgp2FpRTUlXHpLT4VoeWtmatZe2eEhasP8D89ftZvbsYcFo8xqbEkp7stHoM7ul8mNfUNbAyt4ivthbw5bYClu082BQ6ukUG0T0ymO6RwSRGh5ASH0ZqXBiJMSG4TvPQW119A7kHK9maV8a2vHK25Zexp6iKHpHBpCaEkdLstXe892FNXQOlVbXklVXz1Mdb+PeqvSTFhPDAxUM4f0j3Yz5H9Q2W7fnlrN9bwoZ9JewtriLI3/miE+z+ohMe5E/v2FBS48NIjj1+Hb6WV1rNypwiVuYWsTWvjKSYUPonhNOvWzj9uzn72oWb83gjK5cP1+2jtt4ytFckJVW15BQ6AS0y2J9xqbFEhgRQWlVHqfvLa3FlLbkHK/H3M0wd0p0Z43ozsX88Lj/D7qJK3l+9l3+v3suKXUUE+vuR/eBUQgO9+95XyPKVN252fmH4w/WH2v3llDR+Y1qVW8TaPSVU1zU43/QNGJyWoohgf+LDnQ/3+PAgYsOcvg/BAa7DWlhq6xvY6D40syq3mFW5RewsqKCipo6Wjoz0iQtlTO8YRveJoVd0MB+t28/7a/ZRVFFLZLA/U4f0IDk2hIjgACLdfSAigv3xMwZrLRZosBZrnT5nKfFhJ2zx2Vtcydxlubyelcuuwgoig/05f2gPhidGMbRXJIN6RjaFhoKyahZtyeezjXl8vjmv6UMdnBay3rGh9IgKpq7eUlpdR7n722xlbT1p3cIZlxrL2NRYxvSJaXWfisbDF439lfaVOH2WDpQ4/ZbySp1vxXml1YeF3+AAP1LiwkiICGLjvtKmwxLgfPCGBfpjjBOk/YzBYtlTVEVheU1LZTQJCXDRv1s4w9xhZXhiFAN6HH3Ypa019m9ZlVPE6t3F+Lv86JcQRr+EcFLjw45qxe3s9hVXsWDDfj7bmMeKnCLy3M9/oL8ffePD2FFQTlWt83oZ3DOSM1Jj8TOG/aXOa2t/STX7S6qaWn3gUJ+zqJAA9xeCencrUAN17tde87e1tc77EfcXtAZrqalrOOywaKy7BXNfcRUFR7z2AlxOn7xAf7+mlt2q2npKq+oOqyvI3487M/vz3bP7tutA5GuF5TW8vWI3/161h4SIIM5IjeOMvrEM6hF5zPC85UApry7JYe7yXA5W1JIYHUJCRBDZOUUADO0VyUXDe/KN4T1JiQ/z+mNQyPKV5X+Hed+DO7+CboN9XY2cwJEdfPeXVLF810GW7XQujeElNNDF1CHduXRkLyalJRy3Of10NTRYvtpewOtLc/hsUx4Hm7WYpcSFEhbkz7q9JVgLMaEBTB6QwIT+8fTvFk7v2FDijnOYqy1Yaykor2Gb+9BY42GyvNJq0rqFMzQximG9IhnSK/K4v0wtrqxlZ4HTF2VXQQUhgS73DylCSYwJISY0oF225MixNQbQlTlFZOcUsWFfKX3jwxjfL45xKbHH/BGBtZa80uqmfknb851+SuU1dQT5uwgK8Gs6xO3vZw7r19bIuA/h+xmDAYLc4b9vQjj9EsKa+umB02G9sR9UTmEFFTX1Tn+8Zv3yggJc7i9Y7o7mQf6M7xdHr2j9ktObquucH7O8tjSHkspazh/ao82CVXMKWb5SlAOPD4MLfg3j7/J1NXIarLXkuA+9jekT45OO6tZa9pdUs25vMev2lLBubwkHy2sZ3y+OswckMCwx6rQPm4iIyMk5VsjqWu3UvhCdDPEDnFPsKGR1aMYYeseFtklH9OPV0NjJfMqg7j6rQ0RETkwjPraFflNgxxdQW+XrSkRERKSNKGS1hX5ToK4ScjrnKXZERETkaApZbaHPBPALgC0LfF2JiIiItBGFrLYQFA69z4Stn/i6EhEREWkjClltpd85sH81lO73dSUiIiLSBhSy2kq/Kc71tk99WoaIiIi0DYWsttJjJITEOkM5iIiISKfXqpBljHnBGHPAGLPmGMuNMeZJY8wWY8wqY8zoZstuMsZsdl9u8lThHY6fn3PIcOvHzjkdREREpFNrbUvWHGDacZZfCKS5LzOBPwMYY2KBh4AzgHHAQ8aYmFMttsPrNwXKD8D+tb6uRERERLysVSHLWvs5UHicVS4D/m4dXwHRxpiewAXAR9baQmvtQeAjjh/WANi4cSNz5swBoLa2lszMTF566SUAKioqyMzM5LXXXgOguLiYzMxM3nrrLQDy8/PJzMzk3XffBWDfvn1kZmbywQcfAJCTk0NmZibz588HYNu2bWRmZvLZZ5813XdmZiaLFy8GYM2aNWRmZrJ06VIAsrOzyczMJDs7G4ClS5eSmZnJmjVOI9/ixYvJzMxk48aNAHz22WdkZmaybds26DeF+dvqyLzwcnJycgD44IMPyMzMZN++fQC8++67ZGZmkp+fD8Bbb71FZmYmxcXOmetfe+01MjMzqaioAOCll14iMzOT2lrnfHZz5swhMzOz6X/517/+lfPOO69p+umnn+bCCy9smn7iiSe49NJLm6Znz57NVVdd1TT96KOPMmPGjKbphx9+mBtvvLFp+sEHH+SWW25pmr7//vuZOXNm0/SPfvQj7rrr0Ej399xzD/fcc0/T9F133cWPfvSjpumZM2dy//33N03fcsstPPjgg03TN954Iw8//HDT9IwZM3j00Uebpq+66ipmz57dNH3ppZfyxBNPNE1feOGFPP30003T5513Hn/961+bpjMzMzvnaw+YP38+mZmZeu256bWn114jvfb02vPUa+9InuqTlQjkNJvOdc871vyjGGNmGmOyjDFZjcV3OpG9ICYFKg/6uhIRERHxslafINoYkwK8Z60d1sKy94BHrbWL3NMLgJ8AmUCwtfZX7vkPAJXW2tlHbqO5TnWC6CPN/wUsfgp+vA2Co3xdjYiIiJymY50g2lMtWbuB5GbTSe55x5rfdaWdDw11GspBRESkk/NUyJoHfMv9K8MzgWJr7V7gv8D5xpgYd4f3893zuq6kcRAUBZs+9HUlIiIi4kX+rVnJGPMKzqG/eGNMLs4vBgMArLXPAP8BLgK2ABXALe5lhcaYh4Gl7k390lp7vA70nZ/LH/pPgS0fQUODM7SDiIiIdDqtClnW2utOsNwCdx1j2QvACydfWieWdgGsfRv2rYJe6b6uRkRERLxAzSi+0N/90+LNH/m2DhEREfEahSxfCE+AXqNhs/pliYiIdFYKWb6Sdj7kLoXyAl9XIiIiIl6gkOUrA84HLGxd4OtKRERExAsUsnyl5ygIjdchQxERkU5KIctX/PwgbSpsmQ8N9b6uRkRERDxMIcuX0qY65zHM7aSnEBIREenCFLJ8qd8UMC4dMhQREemEFLJ8KSQGks9QyBIREemEFLJ8LW2qM/J7yV5fVyIiIiIepJDla2nnO9dbNPq7iIhIZ6KQ5Wvdh0JUb+dchiIiItJpKGT5mjEwcgZs/QSKd/u6GhEREfEQhaz2IP06wMLKV3xdiYiIiHiIQlZ7ENsX+kyA7H+Ctb6uRkRERDxAIau9SL8BCrdCzte+rkREREQ8QCGrvRhyGQSEQfbLvq5EREREPEAhq70ICoehl8Oat6GmwtfViIiIyGlSyGpP0q+HmlJY/66vKxEREZHTpJDVnvQ+C2JSdMhQRESkE1DIak/8/GDk9bD9cyja5etqRERE5DQoZLU3TWNmverrSkREROQ0KGS1N9G9IXWyc8iwocHX1YiIiMgpUshqj9JvhIM7YNeXvq5ERERETpFCVns0+BIIjFAHeBERkQ5MIas9CgyFoZfBunc0ZpaIiEgHpZDVXo2YATVlsPE/vq5EREREToFCVnvVZwJEJetXhiIiIh1Uq0KWMWaaMWajMWaLMea+Fpb/wRiT7b5sMsYUNVtW32zZPA/W3rn5+cHwq2Hrx1B2wNfViIiIyEk6YcgyxriAPwEXAkOA64wxQ5qvY639gbU23VqbDjwFvNVscWXjMmvtpZ4rvQsYOQNsPax+09eViIiIyElqTUvWOGCLtXabtbYGeBW47DjrXwe84oniuryEgdAzHVbpkKGIiEhH05qQlQjkNJvOdc87ijGmD5AKfNxsdrAxJssY85Ux5vJj3YkxZqZ7vay8vLxWlNVFjJwBe1fCgQ2+rkREREROgqc7vs8A3rTW1jeb18damwFcDzxujOnX0g2ttc9aazOstRkJCQkeLqsDGzYdjEutWSIiIh1Ma0LWbiC52XSSe15LZnDEoUJr7W739TbgU2DUSVfZlYUnQP9zYdUbOs2OiIhIB9KakLUUSDPGpBpjAnGC1FG/EjTGDAJigC+bzYsxxgS5/44HJgDrPFF4lzLiWijJhZ2LfF2JiIiItNIJQ5a1tg64G/gvsB543Vq71hjzS2NM818LzgBetdbaZvMGA1nGmJXAJ8Cj1lqFrJM18CLnNDsrX/N1JSIiItJK5vBM1D5kZGTYrKwsX5fRvvzrTlg3D360yTntjoiIiLQLxphl7v7nh9GI7x3FiGuhplSn2REREekgFLI6ipRJEJkIq3TIUEREpCNQyOoo/Pxg+HTnNDsVhb6uRkRERE5AIasjGXYVNNTBep0CUkREpL1TyOpIeoyAuP6wZq6vKxEREZETUMjqSIxxWrO2L4TSfb6uRkRERI5DIaujGXolYGHdO76uRERERI5DIauj6TYIug/TIUMREZF2TiGrIxp2JeR8DUW7fF2JiIiIHINCVkc09Erneu3bvq1DREREjkkhqyOKTYXEMTpkKCIi0o4pZHVUw66CvSshf4uvKxEREZEWKGR1VEOvAAysfcvXlYiIiEgLFLI6qshe0OcsWP0mWOvrakREROQIClkd2bArIX8jHFjn60pERETkCApZHdngy8C41AFeRESkHVLI6sjCE6Dv2TpkKCIi0g4pZHV0w66Cop2we5mvKxEREZFmFLI6usGXgCvIac0SERGRdkMhq6MLjoK0qc5QDg31vq5GRERE3BSyOoPh06FsP+xY6OtKRERExE0hqzMYMA0Cw3XIUEREpB1RyOoMAkJg0MWwfh7UVfu6GhEREUEhq/MYPh2qimHLfF9XIiIiIihkdR59MyE0TocMRURE2gmFrM7CFQBDLoeN70N1ma+rERER6fIUsjqT4dOhrhI2/sfXlYiIiHR5ClmdSfKZEJmkQ4YiIiLtQKtCljFmmjFmozFmizHmvhaW32yMyTPGZLsvtzVbdpMxZrP7cpMni5cj+PnBsCth6wKoKPR1NSIiIl3aCUOWMcYF/Am4EBgCXGeMGdLCqq9Za9Pdl+fct40FHgLOAMYBDxljYjxWvRxt+HRoqIN1//J1JSIiIl1aa1qyxgFbrLXbrLU1wKvAZa3c/gXAR9baQmvtQeAjYNqplSqt0mMExA+A1XN9XYmIiEiX1pqQlQjkNJvOdc870lXGmFXGmDeNMckneVvxFGNg+NWw8wso3O7rakRERLosT3V8fxdIsdaOwGmtevFkN2CMmWmMyTLGZOXl5XmorC5q1I3g54Klz/m6EhERkS6rNSFrN5DcbDrJPa+JtbbAWtt4PpfngDGtvW2zbTxrrc2w1mYkJCS0pnY5lsheMOQyWP4PjZklIiLiI60JWUuBNGNMqjEmEJgBzGu+gjGmZ7PJS4H17r//C5xvjIlxd3g/3z1PvG3cd6G6GFa95utKREREuqQThixrbR1wN044Wg+8bq1da4z5pTHmUvdqs4wxa40xK4FZwM3u2xYCD+MEtaXAL93zxNuSx0HPdPj6L2Ctr6sRERHpcoxthx/AGRkZNisry9dldHzZr8C/bodv/gv6nePrakRERDolY8wya23GkfM14ntnNuxKCI2HJc/6uhIREZEuRyGrM/MPgoxbnJNGazgHERGRNqWQ1dllfFvDOYiIiPiAQlZnp+EcREREfEIhqyvQcA4iIiJtTiGrK9BwDiIiIm1OIasrMAbOvAPyN8K6f/m6GhERkS5BIaurGDYdug+DDx+AmgpfVyMiItLpKWR1FS5/uPC3UJwDXzzu62pEREQ6PYWsriRlAgy7ChY9Dgd3+LoaERGRTk0hq6uZ+rAzbtaHP/d1JSIiIp2aQlZXE5UIk/4H1r8LWz/xdTUiIiKdlkJWVzT+bohJhfd/AvW1vq5GRESkU1LI6ooCgmHab5whHXTyaBEREa9QyOqqBkyD/lPh00eh7ICvqxEREel0FLK6KmOc1qyaMvjqaV9XIyIi0ukoZHVl8Wkw+FJY+gJUl/q6GhERkU5FIaurmzDLOXn0shd9XYmIiEinopDV1SWOgZRJziFD/dJQRETEYxSyBM6aBSW7Yc1cX1ciIiLSaShkCaRNhW5D4IsnwVpfVyMiItIpKGSJ80vDs74HB9bClgW+rkZERKRTUMgSx7DpENELvnjc15WIiIh0CgpZ4vAPhDPvgB0LYfdyX1cjIiLS4SlkySFjboagSFj8pK8rERER6fAUsuSQ4EjIuAXWvQOF231djYiISIemkCWHO+MO8AtwzmkoIiIip0whSw4X2RPG3wmrXoXdy3xdjYiISIfVqpBljJlmjNlojNlijLmvheU/NMasM8asMsYsMMb0abas3hiT7b7M82Tx4iUTfwhhCfDBTzVuloiIyCk6YcgyxriAPwEXAkOA64wxQ45YbQWQYa0dAbwJ/LbZskprbbr7cqmH6hZvCo6EKQ9Azlew9m1fVyMiItIhtaYlaxywxVq7zVpbA7wKXNZ8BWvtJ9baCvfkV0CSZ8uUNjfqRug+HOY/BLVVvq5GRESkw2lNyEoEcppN57rnHcutwPvNpoONMVnGmK+MMZeffIniE34uuOARKNrlnDxaRERETopHO74bY24EMoDHms3uY63NAK4HHjfG9DvGbWe6w1hWXl6eJ8uSU9X3bBj4DVj4Oyjd7+tqREREOpTWhKzdQHKz6ST3vMMYY84DfgZcaq2tbpxvrd3tvt4GfAqMaulOrLXPWmszrLUZCQkJrX4A4mXnPwx11fDJr3xdiYiISIfSmpC1FEgzxqQaYwKBGcBhvxI0xowC/oITsA40mx9jjAly/x0PTADWeap4aQNx/WDcTFj+D9i7ytfViIiIdBgnDFnW2jrgbuC/wHrgdWvtWmPML40xjb8WfAwIB944YqiGwUCWMWYl8AnwqLVWIaujOfteCImB93+sIR1ERERaydh2+KGZkZFhs7KyfF2GNLf87zDve3DZ0zDqBl9XIyIi0m4YY5a5+58fRiO+S+uk3wjJZ8BHD0BFoa+rERERafcUsqR1/PzgG7+HyiJY8EtfVyMiItLuKWRJ6/UYBmfcDsvmQK4O54qIiByPQpacnHPuh4ge8N4PoKHe19WIiIi0WwpZcnKCImDab2DfKlj6vK+rERERabcUsuTkDbkc+k2Bjx+G0n2+rkZERKRdUsiSk2cMXDTbGQn+3XugocHXFYmIiLQ7CllyauL6Oafc2fQ+fKxfG4qIiBzJ39cFSAc2biYcWA+L/gDxAyH9Ol9XJCIi0m6oJUtOnTFw0WOQOhnenQU7v/R1RSIiIu2GQpacHlcAXP0iRCXDazfAwZ2+rkhERKRdUMiS0xcaC9e/Dg118M9roarE1xWJiIj4nEKWeEZ8f7jm75C/Cf55DRTn+roiERERn1LIEs/pmwlXPgt7V8HT4yH7n2Ctr6sSERHxCYUs8azh0+GOL6DHcPjXHfDq9VC639dViYiItDmFLPG82FS46T244NewZQE8fSasel2DloqISJeikCXe4ecH4++C2xdCTAq89R340zhY/g+oq/F1dSIiIl6nkCXelTAQbpsP0/8GASEw7254YiR8+SeoLvN1dSIiIl6jkCXe5+eCYVfCdz+HG+c6p+T570/hD0NhwcPqsyUiIp2SQpa0HWOg/3lw83tw63xInQQLfwePD4N534O8jb6uUERExGN07kLxjeSxcO1LULDVOXSY/TIs/zukXQBjb4V+54JLL08REem4jG2H4xhlZGTYrKwsX5chbak8H5Y+D0ufg/IDENELRt3oXGL6+Lo6ERGRYzLGLLPWZhw1XyFL2pX6Wtj0ASx7EbbMd+alTobkM5xO9AmDIK4/BAT7tk4RERG3Y4UsHY+R9sUVAIMvcS7FubDiZVgzFxbOBuseZ8v4QUwqDL8azviuc+5EERGRdkYtWdIx1FVDwRanc3zeRtid5bR0BYRBxi0w/m6I7OnrKkVEpAtSS5Z0bP5B0H2oc2l0YD0s+gN89WdY8iyk3wBn3uEcVmwr5fmw6jXY8G+oKoH6GqivdgZcbaiD7kMgZSKkTIJeo8E/8MTbbKh3WvFqyiEwDIIiIDC8dbc9ntL9ULgVCrfDwe3OdXEOBEdBVBJEJkJUsvN3j+EQHHl69yciHYO1UFUEJXuhdK+zH0gYcOLb1Nee/n6pk1NLlnR8hdth8ZOw4iUn5CSOgfTrYdhVEBJz+Lq1lU6LWOE2KMpxQkZRDhTvcsJN92FOwOg5AnqMaPlQZEO9c7qgFf+Aje9DQ62zblSSc7jTFXRox7MnG/avcf72D4HkcRCd7Pwd4L74B0NFgVNTwVYnANW3MCq+XwCExUNsX+cS1w9i+0F0bycoBUU6wcgV4NS4fy3kfA27vnKui3MObcv4OTvS6N7OzrV4N1QWHn5fKRNhwDQYcIFzqqRGlUVOrYXbnPtMnaw+ctL5WQtVxc77JTQegsJ9XdGJNdTDjkWw9i1nX1Vb6exvmu97qkuccFVXefht+0x0fuk96OLDg9TBnc5p0la96uxLw7s7Z/WI7uP8SCm8u3PkobYSasuhpgLqqg7fN7qCnH1GdB/nS3Fsv2PvQxrqnf2VMV77N3mCOr5L51eWB6tfd/pxHVgLrkAYeBFE9oL8Tc6lKAdo9poPDHeCRlSy8ybetxpKdh9aHhrvtKL5+Ts7Cb8AJxCVH4DQOBgxw/kFZPchx66rohB2fuHs7HZ+AeUFzg6tttLZ+YCz02kKTu7roEinNaumHGpKneuyA4fCWPmBlu/PPxgwh3aaET2dHw4knwHxA5zAFJV89DfQmgrnsR/cCds/c36AkL/JWRY/0GlRK9x2eBhr/B+mnQ+DL3augyKcx5a30WltPLAODu6AmjJnlP+acudvrFNT30xIPdsJn0eyFmorICDU+zvZ2krn/uDQfRmXZ76p15Q7oXffKudDIzTOCcxhCc5rrLbC+cDK3wT5m52/G+qc8N5rFPRKd547P5ezvboaKM+Dsv3OB1rimPbRotD4Wj+44/BLZZHzOPqMh97jnVbd9hrMKw/Crq9h12LYs8LZr1QUOBdbf2i90LhD+47GLzquQOf91xgk6qqc7VUedMJZZZHz2gqNd57/xmtwvgQV73ZasYtznfsD92vRONcBoc4XtdTJzpegyF6H126ts4/I3wjr34V17zivkYBQ58tSeA/3vqfKec3VVTnv38hezn4isqezTu5SyHoeinY5oWn0Tc6yVa/Dri+d++oz0Xk+S/ZC0U5nv1GSe6jvLDj7y4BQZx/aUOu8butrnL8PY5yAFtffuX3lQee1VFkE1cVODcnjIPlMZ5/Rc4SzzXZEIUu6DmudD7PsV5zQVVvpvHnjB7gvaU6Qie7ttHQd+eFdXgD7VjqBq3C70yTeUHvo2hUEQy6FARee/gdbQ4Ozo/MPOvQB2lpVJU7oKc51vo1WlzrzqkucD/Je6c4OKbr3qQeUgq2w+UPnUl97qPUstq8T1kr2wvp5zuHSinznfxPZy9npNu5sXUHODjQo0vn2HxjuHAatq3aHzjxnvdh+zgdwbQWU7nMOW5Tucz4UGoNib/dOtsdwJ/SerpK9sOZNWPka7F/d8joBYc7rJDQGQmKd1s3QOPffce6/o53/T21Fs2Bc5gTNfauc0NT8w+d4gqOd16hxOa/B2nJ3HaFOa2l5/tFBNyjS+RAdfIkz4G9g2PHvo7IIVr7iXOpr3Y+j2ePyD3LeR7bh0CWyJ6RMdtfW7PVkrftD+QVY85ZzuByc0BGT4lwCw2H3Msjb4CxzBTrPYVCE80Hs5++8/v1cTtivLnG/lkudLxjB0e4w4z6cHZXsvKe7DT69H75Ulx76IrB3lRMgDqxzlvkFODVG9nIHIvdzHRTpfMEpynFCSNEuJyA1fmFqiX+I8xoKiXb+l43P4ZGvicbHGZXoBDCD+zuhdf7PlYVOjVXFzvqx/SBxtBNIjqzDP9j50jPsSmf8wcDQk/vfNLbYL33Oef9jnf3niGthxDXOfuVI9bVOLQHBzuv1WO/RhgbnvXJwu/Meyd/sfMEo2OLcJsT9Xmv8nx3c4bTGH9zh3N4V5OyLwhKcABbezbkEReD805rxc8Hob53cYz8FpxWyjDHTgCcAF/CctfbRI5YHAX8HxgAFwLXW2h3uZfcDtwL1wCxr7X9PdH8KWeIxDe6dmJ9ObuBVDfXOTnD9u07o6zbYfRnqBLJjDSxrrfOhtu0z2Pap80EcHOWEqogeziUkxvlw3vW1c1gX3IEj2d0a1KxFICDUfWih2cU/8FCwCwxz/i7YAitfdVrsbAMkZjgfSP5BHNbSWV/ntEBUFDofcIddHzx83ZZEJh069Nx4HRDifMiW5znBtDzfud+4NCfAhMYdCjEN9c4H0N5sp1WlZDeEdXM+WCK6O9cN9U6r48b/OK0f/sHQ9xyn5ajxeYhJdZ6DPSuc8ehWv+mE116jnRBRUXiotaalD/9Dn/ZOS0fqZOdSVwXL5jiHxAMjnA/fEdc6/XmOPFQPzheYnK+doLA32wna9bVOq11DnfNYAkPdgTzCeS0Ehjs1Fec6waZ0z+H1hfdwWpK7DXH+H36uw5//hjon9DYPwOV5sH/dodcTOPUnj4XeZzktNL1Gn1wwqa9z98d0X+qrneciOLrlVruGeifsVuS7XyuJrTsE2VDv/L93LILtC50gHp7ghJ7o3hDV2/lS0+csd+jwgKJdTiDtNsS3h+1K90PuEuc1VLjdaaUrO+Bcjjzc2cg/BH6+z+ulnXLIMsa4gE3AVCAXWApcZ61d12ydO4ER1trbjTEzgCustdcaY4YArwDjgF7AfGCAtc3bXI+mkCUiLSre7exgc5ceOqRSnuduGWhF6Gkuuo/7W/m1EN//5GtpqHdaFCoKnPt2BTitXoGhTpgLCGvbQ3j1dU542fCe0/JQuJ2m/4cryAllRbucIDp8OmTc6rR2HvW43C1Xjf1gjHHCcOE22P457FjoXDe2QPYcCRnfhmHT26afUn2dEzbzNzsB/cB6p3tA3sbjtyYZl/t5CXVav5p/Eeg22Hk96MtYx2Ttoe4IRzLG+bLmZafz68JxwBZr7Tb3hl4FLgPWNVvnMuAX7r/fBP5ojDHu+a9aa6uB7caYLe7tfXm8O9y4cSOZmZmHzbvmmmu48847qaio4KKLLjrqNjfffDM333wz+fn5TJ8+/ajld9xxB9deey05OTl885vfPGr5//zP/3DJJZewceNGvvvd7x61/Oc//znnnXce2dnZ3HPPPUct//Wvf81ZZ53F4sWL+elPf3rU8scff5z09HTmz5/Pr371q6OW/+Uvf2HgwIG8++67/O53vztq+T/+8Q+Sk5N57bXX+POf/3zU8jfffJP4+HjmzJnDnDlzjlr+n//8h9DQUJ5++mlef/31o5Z/+umnAMyePZv33nvvsGUhISG8//77ADz88MMsWLDgsOVxcXHMnTsXgPvvv58vvzz86U1KSuKll14C4J577iE7O/uw5QMGDODZZ58FYObMmWzatOmw5enp6Tz++OMA3HjjjeTm5h62fPz48fzmN78B4KqrrqKgoOCw5eeeey4PPPAAABdeeCGVlYd/47n44ov50Y9+BHDU6w702mv/r72lANz/k5/w5VdfHupXhSWpV09eevZJqCnnnvseInvNOidA+PlDUAQD1u3k2SlOwOpcr70osMO5Y8Y3uHZ8b3LWfMU3f/0aBLsPrXy0Grin9a+9lSuPeO0lQW0sv75vFmddcqPz2rvg4qNu3yavvQnu1978v7lbrg8F7f/M/SehYeE8/fxLvD73rSNuvY1PP30B0H6v8+/3Zhy13Nv7vUatCVmJQLOfJZELnHGsday1dcaYYiDOPf+rI26b2NKdGGNmAjMBgoLaV4c2EekA/BoPETWb5x90aPy08ATn8FNXYVxOv7n0ayFuEjy3yrPbDwiFuL6e3eZpMUf3awyLh9BQnQdVfKY1hwunA9Ostbe5p78JnGGtvbvZOmvc6+S6p7fiBLFfAF9Za19yz38eeN9a++bx7lOHC0VERKSjONbhwtYcgN4NNP9tdZJ7XovrGGP8gSicDvCtua2IiIhIp9OakLUUSDPGpBpjAoEZwLwj1pkH3OT+ezrwsXWayOYBM4wxQcaYVCANWOKZ0kVERETarxMeqHb3sbob+C/OEA4vWGvXGmN+CWRZa+cBzwP/cHdsL8QJYrjXex2nk3wdcNeJflkoIiIi0hloMFIRERGR03A6fbJERERE5CQpZImIiIh4gUKWiIiIiBcoZImIiIh4gUKWiIiIiBcoZImIiIh4gUKWiIiIiBcoZImIiIh4QbscjNQYkwfs9PLdxAP5Xr4POXl6XtovPTftk56X9kvPTfvkjeelj7U24ciZ7TJktQVjTFZLo7OKb+l5ab/03LRPel7aLz037VNbPi86XCgiIiLiBQpZIiIiIl7QlUPWs74uQFqk56X90nPTPul5ab/03LRPbfa8dNk+WSIiIiLe1JVbskRERES8psuFLGPMNGPMRmPMFmPMfb6upyszxiQbYz4xxqwzxqw1xnzfPT/WGPORMWaz+zrG17V2RcYYlzFmhTHmPfd0qjHma/d75zVjTKCva+yKjDHRxpg3jTEbjDHrjTHj9Z7xPWPMD9z7sTXGmFeMMcF6z/iGMeYFY8wBY8yaZvNafI8Yx5Pu52iVMWa0J2vpUiHLGOMC/gRcCAwBrjPGDPFtVV1aHfA/1tohwJnAXe7n4z5ggbU2DVjgnpa2931gfbPp/wP+YK3tDxwEbvVJVfIE8IG1dhAwEuc50nvGh4wxicAsIMNaOwxwATPQe8ZX5gDTjph3rPfIhUCa+zIT+LMnC+lSIQsYB2yx1m6z1tYArwKX+bimLstau9dau9z9dynOh0UiznPyonu1F4HLfVJgF2aMSQK+ATznnjbAFOBN9yp6XnzAGBMFTAaeB7DW1lhri9B7pj3wB0KMMf5AKLAXvWd8wlr7OVB4xOxjvUcuA/5uHV8B0caYnp6qpauFrEQgp9l0rnue+JgxJgUYBXwNdLfW7nUv2gd091VdXdjjwI+BBvd0HFBkra1zT+u94xupQB7wN/eh3OeMMWHoPeNT1trdwGxgF064KgaWofdMe3Ks94hXc0FXC1nSDhljwoG5wD3W2pLmy6zz81f9BLYNGWMuBg5Ya5f5uhY5ij8wGviztXYUUM4Rhwb1nml77v49l+GE4F5AGEcfrpJ2oi3fI10tZO0GkptNJ7nniY8YYwJwAtbL1tq33LP3NzbXuq8P+Kq+LmoCcKkxZgfOIfUpOP2Aot2HQkDvHV/JBXKttV+7p9/ECV16z/jWecB2a22etbYWeAvnfaT3TPtxrPeIV3NBVwtZS4E09y8+AnE6Js7zcU1dlrufz/PAemvt75stmgfc5P77JuCdtq6tK7PW3m+tTbLWpuC8Rz621t4AfAJMd6+m58UHrLX7gBxjzED3rHOBdeg942u7gDONMaHu/Vrj86L3TPtxrPfIPOBb7l8ZngkUNzuseNq63GCkxpiLcPqbuIAXrLWP+LairssYMxFYCKzmUN+fn+L0y3od6A3sBK6x1h7ZiVHagDEmE/iRtfZiY0xfnJatWGAFcKO1ttqH5XVJxph0nB8kBALbgFtwvjDrPeNDxpj/Ba7F+dX0CuA2nL49es+0MWPMK0AmEA/sBx4C/kUL7xF3KP4jzuHdCuAWa22Wx2rpaiFLREREpC10tcOFIiIiIm1CIUtERETECxSyRERERLxAIUtERETECxSyRERERLxAIUtEOhRjTL0xJrvZxWMnQzbGpBhj1nhqeyLStfmfeBURkXal0lqb7usiRERORC1ZItIpGGN2GGN+a4xZbYxZYozp756fYoz52BizyhizwBjT2z2/uzHmbWPMSvflLPemXMaYvxpj1hpjPjTGhPjsQYlIh6aQJSIdTcgRhwuvbbas2Fo7HGcE58fd854CXrTWjgBeBp50z38S+MxaOxLn/H9r3fPTgD9Za4cCRcBVXn00ItJpacR3EelQjDFl1trwFubvAKZYa7e5Tzy+z1obZ4zJB3paa2vd8/daa+ONMXlAUvPTnBhjUoCPrLVp7umfAAHW2l+1wUMTkU5GLVki0pnYY/x9MpqfW64e9V0VkVOkkCUincm1za6/dP+9GJjh/vsGnJOSAywA7gAwxriMMVFtVaSIdA36hiYiHU2IMSa72fQH1trGYRxijDGrcFqjrnPP+x7wN2PMvUAecIt7/veBZ40xt+K0WN0B7PV28SLSdahPloh0Cu4+WRnW2nxf1yIiAjpcKCIiIuIVaskSERER8QK1ZImIiIh4gUKWiIiIiBcoZImIiIh4gUKWiIiIiBcoZImIiIh4gUKWiIiIiBf8P9DELoAo1B8PAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot the model weights as they train, converging to the correct values\n", "\n", "plt.plot(epochs, training_weights, label='weight')\n", "plt.plot(epochs, training_bias, label='bias')\n", "plt.axhline(y=1, label='true_weight', color='k', linestyle=':')\n", "plt.axhline(y=0, label='true_bias', color='k', linestyle='--')\n", "plt.xlabel('Epoch')\n", "plt.legend(loc='best')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This make sense that the training data is not perfect. It contains some errors itself. This kind of uncertainty is called [epistemic uncertainty](https://en.wikipedia.org/wiki/Uncertainty_quantification#Aleatoric_and_epistemic_uncertainty). We will cover it in later notebook." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 4 }