{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Chapter 11 – Training Deep Neural Networks**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_This notebook contains all the sample code and solutions to the exercises in chapter 11._"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
" \n",
" ![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\") \n",
" | \n",
" \n",
" ![](\"https://kaggle.com/static/images/open-in-kaggle.svg\") \n",
" | \n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"# Setup"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This project requires Python 3.7 or above:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"\n",
"assert sys.version_info >= (3, 7)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And TensorFlow ≥ 2.8:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from packaging import version\n",
"import tensorflow as tf\n",
"\n",
"assert version.parse(tf.__version__) >= version.parse(\"2.8.0\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As we did in previous chapters, let's define the default font sizes to make the figures prettier:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"plt.rc('font', size=14)\n",
"plt.rc('axes', labelsize=14, titlesize=14)\n",
"plt.rc('legend', fontsize=14)\n",
"plt.rc('xtick', labelsize=10)\n",
"plt.rc('ytick', labelsize=10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"And let's create the `images/deep` folder (if it doesn't already exist), and define the `save_fig()` function which is used through this notebook to save the figures in high-res for the book:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"\n",
"IMAGES_PATH = Path() / \"images\" / \"deep\"\n",
"IMAGES_PATH.mkdir(parents=True, exist_ok=True)\n",
"\n",
"def save_fig(fig_id, tight_layout=True, fig_extension=\"png\", resolution=300):\n",
" path = IMAGES_PATH / f\"{fig_id}.{fig_extension}\"\n",
" if tight_layout:\n",
" plt.tight_layout()\n",
" plt.savefig(path, format=fig_extension, dpi=resolution)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Vanishing/Exploding Gradients Problem"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAEQCAYAAAD2/KAsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABNlUlEQVR4nO3de3yO9f/A8ddn56PzDHNYclw5HxIxpxWRYyo5ppIi9ZUvRSchfSUmyuEnEXKMnJVkkiijEWE5m83ZZrOT3ffn98c1y2xj7N6ue9v7+Xhcj+2+Pp/7+rx3ue2967o+B6W1RgghhMhrDmYHIIQQonCSBCSEEMIUkoCEEEKYQhKQEEIIU0gCEkIIYQpJQEIIIUxhkwSklJqrlLqglDqQRXkvpdT+1O03pVQdW7QrhBAi/7LVFdA8oN0dyk8AgVrr2sBYYLaN2hVCCJFPOdniIFrrX5RS/nco/+2Wl7uA8rZoVwghRP5lxjOgF4GNJrQrhBDCjtjkCii7lFKtMBLQY3eoMxAYCODu7t6gQoUKeRRd9litVhwcpO/G3ch5urszZ86gtaZixYpmh2L38vLzdMN6g9MJpwGo6F4RZwfnPGnXFuzx/114ePglrbVPZmV5loCUUrWBOUB7rfXlrOpprWeT+oyoYcOGOjQ0NI8izJ6QkBBatmxpdhh2T87T3bVs2ZLo6GjCwsLMDsXu5dXn6XzceR6Z8wjFkouxrf82Hir9UK63aUv2+P9OKXUqq7I8SZVKqYrASqCP1jo8L9oUQoh7VdKjJB2qduCH3j/ku+STH9nkCkgptRhoCZRSSkUAHwDOAFrrmcD7QEngS6UUQIrWuqEt2hZCiJy6mnCVxJREynqX5YsOX5gdTqFhq15wPe9S/hLwki3aEkIIW4pNiuXJb58kNimWsEFhODnk6aPxQk3OtBCi0Eq4kUDnJZ3ZfXY3K55ZIcknj8nZFkIUSsmWZHos70HIyRAWdF1AlxpdzA6p0JEEJIQolN7f+j7r/1nPzA4z6VW7l9nhFEr5PgFdu3aNCxcucOPGjTxpr2jRohw6dChP2srPCuN5cnZ2pnTp0hQpUsTsUEQ2DG86nId8HqJPnT5mh1Jo5esEdO3aNc6fP4+fnx/u7u6k9rDLVbGxsXh7e+d6O/ldYTtPWmsSEhI4e/YsgCQhO6W1Zv6++fR8uCelPEpJ8jGZfQ2ZvUcXLlzAz88PDw+PPEk+QmRFKYWHhwd+fn5cuHDB7HBEFt7f+j4vrH6Bb/Z9Y3YognyegG7cuIG7u7vZYQiRxt3dPc9uB4t7M3HHRMZtH8dL9V7ipfoyKsQe5OsEBMiVj7Ar8nm0T1/u/pKRP42k58M9mdlxpvw72Yl8n4CEEOJOriRc4b2t7/FUtaeY32U+jg6OZockUuXrTghCCHE3JdxLsGPADvyL+ePsmH9mti4M5ApICFEgbTq6if/9+j8AapSqgZuTm8kRidtJAhL5TkREBK+//jqPPvpoWg/IkydPmh2WsCO/nPqFrku7suTgEhJuJJgdjsiCJCCR7xw9epRly5ZRvHhxmjdvbnY4ws7sPrubjt92xL+YPz/2/hF3Z+kpa68kAYl8p0WLFpw/f54NGzbQo0cPs8MRduTAhQO0W9SOUh6l+KnPT/h4ZroQp7ATkoBEvmNvSw4L+xF2LgxPZ09+6vsTfkX8zA5H3IX0ghNC5HsWqwVHB0d61+5N1xpd8XTxNDskkQ3yp6QQIl87F3eOBrMbsOnoJgBJPvmIJKB87vXXX+epp57Kdv0pU6ZQu3ZtrFZrLkYlRN64knCFxxc8ztErRyniKhPA5jeSgPKxY8eOMWvWLD744INsv2fQoEFcuHCB+fPn52JkQuS+2KRY2i9qz5HLR1j93GqaVmhqdkjiHkkCyseCg4OpU6cODRs2zPZ73N3d6du3L5MmTcrFyITIXYkpiTy1+Cn2RO5heY/ltKncxuyQxH2QBGSHrl+/zsiRI6lSpQouLi4opdJtn332GUlJSSxcuJDnn38+3Xt37NiRof7N7dVXXwXgueee4++//+a3334z48cTIsecHZypWaomC7ouoFP1TmaHI+6T9IKzM1prunXrxo4dOxg9ejQNGzZk586djBkzBn9/f3r27MmTTz7Jrl27iI6OzjAQs0aNGuzcuTPdvk8++YSNGzfyzDPPAFC3bl2KFCnCpk2baNo089sWWmssFstd41VK4eiY95M7rlixAoA9e/YAsHHjRnx8fPDx8SEwMDDP4xF5I8WawqX4S5TxKsOMjjPMDkfklNY6xxswF7gAHMiiXAGfA0eB/UD97By3QYMG+k7+/vvvO5bnhmvXruXq8b/44gutlNI//vhjuv1du3bVpUqV0larVWut9SeffKKVUjopKemOx3vvvfe0m5ub3rBhQ7r9jz32mA4KCsryfVu3btXAXbfAwMBM35/b5+le48lL2f1cBgYG6jp16uRuMAXE1q1btcVq0f1W9dMVp1TU0QnRZodkl7Zu3Wp2CBkAoTqL3/G2ugKaB0wHslpmsD1QNXV7BJiR+lXc5uuvvyYoKIigoKB0+2vUqMGaNWvS1jGJjIykSJEiuLi4ZHmst99+m88//5y1a9fStm3bdGU+Pj6Eh4dn+d4GDRqwe/fuu8Zr1rLbxudaFBZaa4ZuHMr8ffMZ03IMRd2Kmh2SsAGbJCCt9S9KKf87VOkMfJOaDXcppYoppcpqraNs0f6tcn+dqez9wr2f34/nz58nNDSUKVOmZCiLioqiTJkyaa8TExNxdXXN8lhvvfUWs2bNYsOGDbRs2TJDubu7OwkJWU/S6OXlRd26de8asyzsJfLCnBNz+PbMtwx/dDjvtXjP7HCEjeTVMyA/4MwtryNS990xAR05ciTDL89nnnmG1157jfj4eM6dO5fJtCzVbRBuzsXFxXH27NkM+ytUqICHhwfXrl0jKir9j79//34AypYtS3R0NOfPnwfAYrGwbt06Hn/8cZKTk3FxccHDw4OrV69y5MiRdMeoXLkyw4YNY968ecyePZuyZcumq1OlShUcHR05d+4cXl5eGd5fvbpx/latWkX37t3v+nM2atSIBQsWAODo6EiVKlUAuHjxIpGRkenqOjs7U7lyZQBOnz6dIQG6urri7+8PwMmTJ0lKSkpX7u7uTsWKFQE4fvx4hqWvPT09KV++PGBMWHr7Myxvb2/KlSsHQHh4eIarqKJFi6Yl+dvPC0Dx4sUpXbo0FouFo0ePZigvWbIkpUqVwmKxZJr0X331VZ599lnOnDlDnz59CAsLIyUlJa3uW2+9xVNPPcWRI0d45ZVXMrz/3XffpW3btoSFhfHmm29mKP/4449p2rQpv/32G6NGjcpQHhwcTN26dfnpp58YN25chvJZs2ZRvXp11q5dy2effZahfMGCBVSoUIGlS5cyY0bG5y8rVqygVKlSzJs3j3nz5mUo37BhAx4eHnz55ZcsW7YsQ3lISAgAkyZNYt26dWn7o8pGEV4jnFcavMLEoImMGzeOLVu2pHtvyZIl+e677wB45513MjwHLV++PAsXLgTgzTffJCwsLF15tWrVmD17NgADBw7McHegbt26BAcHA9C7d28iIiLSlT/66KNMmDABgO7du3P58uV05W3atOG994zE2b59+wyf/Y4dOzJ8+HCATD87t/7ee/LJJzOU9+/fn/79+xMTE5Otz97tcvuzl5W8SkCZ/Zmc6TWCUmogMBCMX1jR0dHpysPDwwkJCSExMREXFxdSUlLSlZ85E0HRokWxWCyZJoBixYpRpEgRbty4kSEBAJQoUQIvLy+SkpLSEsCtihcvjre3N4mJiVy4cCFDuY+PD+7u7sTHJ2SIDYwebhaLhfj4+AzlHh4eAOzbt48WLVqklc+ePZuYmBi6d+9OXFwczs7O+Pv7c+PGDSIiItJ+aWqteeWVV1i5ciWLFi3iwQcfzNBGXFwcDg4OnDp1ioCAgAzlsbGxgJGIlixZkq7MwcGB0qVLAxATE0NCQgKenp5px9BaExsbS5EihWNA4IEDB9K9TkxMJDY2lqSkpAyfW4CDBw8SEhLChQsXiI6OJiUlBa11Wt2//voLb29vTp8+nen79+3bh5OTE0ePHs20fO/evSQnJ3PgwIFMy0NDQ4mOjmbfvn2Zlv/+++9ERUXx119/ZVq+c+dOjh07xsGDBzMt37FjB0WLFuXw4cOZlv/yyy+4ubkRHh6eafnNBHTs2LF05S7XXSiny/FMi2fYtm0bJ06cyPB+q9Wa9v7Mzp+zs3NaeURERIbyyMjItPLIyMgM5REREWnl58+fz1B++vTptPKLFy9y7dq1dOUnTpxIK79y5UqGP66OHTuWVp7Zubn1915m5YcPHyYkJIS4uLhsffZud/tnT2uwWt2wWt3R2p0ffzzD8eOhHD8exZkzNbBaXbFaXdHaDavVjU8/tVKixAnOnHHh4MGX0PpmuTOwLUN7Nylb3UtPvQW3Tmv9cCZls4AQrfXi1NdHgJZ3uwXXsGFDHRoammX5oUOHqFmzZo7ivlexsbG59txDGx0vCA8PZ+LEiTz44IOsWbOGGTNmMG3aNAYPHpxW9+TJkzzwwAN89913dOvWDYChQ4fy5ZdfMnXqVBo0aJDu2AEBAWmJITo6mhIlSjB79mxeeumlXPlZcvM82bvsfi5btmxJdHR0hr/GheHHYz/SvGJz3J3dCQkJyfQve5FeSEgILVq0JCYGrlwxtsuXja/R0XDtGsTGGtvN7zPbFxsLtpssRe3RWmc6WDGvroDWAEOUUkswOh/E5Mbzn/xOKcX333/PkCFDGDFiBFarlQYNGrB69eoM0+34+/vTuHFj1q5dS7du3dBaM3/+fCwWC0OGDMlw7CNHjqQloPXr1+Pi4kLXrl1z7Wc5e/YsX3zxBaGhoezbt4+EhAROnDiRdostv+jbt2/ardHo6GguXbpETEyMKV3PC5PlB5fz3HfP8XaztxnfZrzZ4ZjKYoFLl+DcOTh/3vh6c7t4MX2SOX++GXFxtkkebm7g6QkeHuDu/u/Xm9utrzMrc3MDV1fo1SvrNmySgJRSi4GWQCmlVATwAeAMoLWeCWwAnsTohh0PvGCLdguiihUrsmbNmmzVffXVV3njjTf44osv8PDwICYmJlvvW7hwIT169KBkyZI5CfWOjh8/zrJly2jQoAHNmzfnxx9/zLW2ctM33xgdO0+ePEnnzp2ZNWuWJJ9ctuGfDTy/8nmaVmjKqObZf56QH924AZGRcOoUnD6dfouKMpLMhQv3klCcAShSBEqUgJIlja8lSkDx4uDtbZR5e9/5e29vcHa+/59r2bJl+Pr6EhgYmPsJSGvd8y7lGhh8pzri3vXp04eJEyfy5Zdfpj3AvJuwsDC2bt2a4fmFrTVr1iztGdqcOXPuOwH5+/vTv39/PvzwQxtGZ6hfvz6nT5/OtOzPP/+kQoUKgHFLrUePHsyaNYtmzZrZPA7xr5CTIXRf1p06vnVY13NdgZjZ+soVCA+Hf/4xvh47ZiSYU6eM5JOd5FKyJJQp8+/m62t89fH5N8mULAmHDu2gY8dmOUoeOREbG8uLL77I8uXLGT58+F0HhctMCPmYo6Mjc+fOZe/evdl+z7lz5/j666/TeqvlFjMXjdNaM3fuXGbNmsWBAwcoUaIEPXr04OOPP8bd/d/lmbNz3vbs2UPfvn1ZtGhRtrqli/t3w3KDAasHULl4ZTb13pSvxvpYrUZC+esvYzty5N+kc+VK1u9TCvz8oGJFY6tUyfhaoYKx/2aSucNwv3TOnbthWvIJDQ2lU6dOXEn9ge80RvEmSUD5XJMmTWjSpEm267dr1y4Xo7EPL730EosWLWLYsGGMHz+e8PBwRo0aRWJiYqbdh7Oybds2hgwZwqpVq6hWrVouRiwAnB2dWf/8eoq6FaWURymzw8nStWvw55+wf/+/CefAAYiLy7y+lxdUrQrVqhlfq1QBf38j0fj5ZT+52Cur1cqECRMYP358uu7ldxqneJMkIGE3dBbzz1mt1nTdxe80/9w333zD3Llz0/UODAoKIjExkVGjRjF9+vRsP8N5+umncXNzS5tDz9HRkV27duFs1p+YBVT45XBWHVrFiGYjqOmTt71a7yY52Ug0f/wBu3cbXw8dynygua8v1KplbAEB/yacMmXyYoC8OaKioujevXtaR6ObHBwccHK6e3qRBCTsxrZt22jVqlWG/WPHjmXs2LFprwMDA9PGTNxu3LhxtGjRgk6dOqVLWgEBASQnJxMZGZn2bOduLl68eG8/gLhnp6JP0fabtiSmJNK/bn98vXxNjefyZfjlF2PbudO40klOTl/HxQXq1DG2WrXg4YeNrz4+5sRslrVr19K7d+9MxzQ6ODhk6w81SUDCbmQ2/1ynTp3o2LEjAwcOTNuX1fiiEydO8M8///DPP/9k+eEvWjT/PFco6KJio2jzTRtik2MJ6RdiSvK5cgV+/hlCQmDbNuNW2u1q1IDGjf/datc2uhcXVomJiQwdOpSFCxdmOZ2XXAGJfMfb2zvD4nouLi6UK1cuW4vu3Zz54uuvv+bhhzOMh8bBwaHQzNJg7y7FXyJoQRDn4s7xU9+fqFOmTp60a7EYt9I2bYIffjBuqd3aC83VFZo0gZYt4bHHoGFDKFYsT0LLF/7++286depEZGTkHeeSVEpJAhKFi5+fH2A8/LyXVWJF3tsVsYtTMadY23MtTcpnvxPN/YiNhY0b4fvvjcRz9eq/Zc7OEBgIrVoZXxs3NgZQiowOHjxIvXr1MszBmBm5BSdMl9eLxvn7+9OqVSveeOMNLly4QJ06dYiPj+fEiRNs3ryZlStXmto9XBgdTZRSdKzWkRNvnMi13m6XL8OaNbBqFfz4I9w69VrlytCunbG1amX0UhN3V6VKFd59910mTpyIxWIhMTExy7pKKUlAwlw9evRI9/q1114D7tyJICeUUixfvpwxY8YwdepUIiMjKVq0KDVq1ODpp5+W5GOypJQknlnxDC/UfYEuNbrYPPnExRlXOQsXwk8/GbfbwOiB1qwZdO0KnToZPdPEvXN1deX999/n9ddfZ/z48UyZMgVrFqNo5RacMJ0tJro9efLkPdUvWbIkn3/+OZ9//nmO2xa2k2JNoed3PVlzZA1dqnex3XFTYMsWI+msWgXXrxv7nZzg8cehWzfo3NnoCi1so3jx4jzyyCO4u7tz/eYJv41cAQkh7IJVW3lh9QusOryKqe2m8kK9nE8FeeoU/N//wVdfGfOl3dSsGfTuDc88Y0xPI2wvJSWFYcOGZUg+zs7OWK1WLBaLXAEJIcyntWbw+sEs3L+Qca3GMfSRofd9LIvF6EwwcyZs2PDvYNCqVaFPH2PW5dT1DkUu+uabb7h6a08OjMUgP/vsM9atW8eWLVtISkqSKyAhhPlcnVwZ2Wzkfc9sffUqLFpUkb594UzqusouLvD00zBokNFduqDONGBvkpKSePvttzNc/dSsWZOBAwfyyiuvEBYWxpgxY6iajYdtkoCEELkiOjGaYm7FmPLEFMB4LnAvTp2CKVNgzhy4ft24tHnwQXjlFejfv/DNPGAPZs2aRXx8fLp9Hh4eTJ06Ne3ft27duqxatSpbx5NuQUIImwveFcxDXz7EqehTKKXuKfmEhRm30x58EKZONToWNGx4hU2bjBmm//tfST5muH79Ou+//366qx+lFI0aNaJp06b3dcx8n4BstaS4ELYgn0f4au9X/OeH//Bo+UfxK+KX7ff9+Sc89RTUqwfffmvs69XL2P/pp/t54gmQnvTmCQ4OzjAI1c3NjeDg4Ps+Zr7+53R2dr7jdBBC5LWEhIRCPVv20gNLeXnty7Sr0o5F3Rbh5HD3u/x//WV0l65fH9atM5ZzfvNNY+G2hQtBlmEyX0xMDJ988km6228ODg60adMmR+tk5etnQKVLl+bs2bP4+fnh7u5+z/eYhbAVrTUJCQmcPXsWX19zZ3Q2S8jJEHqv6k3zSs357pnvcHW684ydhw/DBx/AsmXGazc3eO01GDkSSpfOg4BFtk2YMCHDjNcuLi5MmjQpR8fN1wno5sSSkZGR2ZqfyBYSExNxk8mi7qownidnZ2d8fX0L7YSnDco24LWGrzG29Vg8nD2yrHfpkpF4Zs0yula7uBgdC955B8qWzcOARbZcvHiRadOmpZt6x8nJia5du1K9evUcHTtfJyAwklBe/ocPCQmhXr16edZefiXnqfDYd24fD5Z4EG9Xb6a2n5plveRkmDYNxo6FmBjjec7AgfDee1C+fB4GLO7JmDFjMiwU6eTkxIQJE3J87Hz9DEgIYa6wc2EEzgvk1fWvZllHa2OanIAAGD7cSD5PPGGsNDprliQfexYREcHcuXNJumU2VxcXF/r27UulSpVyfPx8fwUkhDDH4UuHeXzB4xRxLcK4VuMyrXPsGLz6KmzebLyuWRM++wzat8/DQMV9GzVqVIZnP46Ojnz00Uc2Ob5cAQkh7tmJqydo+01bHJQDW/puoVKx9H8NJyfDhAnGctWbN0Px4sbtt337JPnkF0ePHmX58uXpnq+7ubnx2muv2ayjjU0SkFKqnVLqiFLqqFLq7UzKiyql1iql9imlDiqlcj4boRDCFFpreq/qTfyNeDb32UzVkumnXNmxw+hSPWoUJCYak4MePgxDhhgLwIn8YcSIERk6dzk6OjJ69GibtZHjW3BKKUfgCyAIiAB2K6XWaK3/vqXaYOBvrfVTSikf4IhSapHWOjmn7Qsh8pZSinmd5xGdGE0t31pp+69dgxEjjOc6YMxkMHMmtG1rUqDivl28eJHvv/8+3cBqd3d3RowYQfHixW3Wji2ugBoDR7XWx1MTyhKg8211NOCtjIE6XsAVIAUhRL4RkxhD8K5gtNZULVmVRn6N0sq2bYPatY3k4+QEo0cbA0wl+eRPPj4+rFy5kkqVKuHp6QkYnQ+GDRtm03ZskYD8gDO3vI5I3Xer6UBNIBL4C3hDa535UnpCCLtzPfk6Hb7twIjNIzhw4UDa/sREeOstY2nrU6eMW29//gnjxoG7u4kBixzr0qULx44dY+bMmZQrV45x48bhZeP1y1VO565SSvUAntBav5T6ug/QWGv9+i11ngaaAcOAB4HNQB2t9bVMjjcQGAjg6+vbYMmSJTmKz9bi4uJs/o9QEMl5urs333wTi8XCtGnTzA7ljpKtyYz6axR/Rv/J+wHvE+gTCMCRI15MmFCTU6c8cXDQ9Op1ir59T+HkZPv58OTzlD32eJ5atWq1R2vdMNNCrXWONuBR4IdbXr8DvHNbnfVA81te/4yRpO547AYNGmh7s3XrVrNDyBfkPN1dYGCgrlOnjtlh3FFySrLutLiT5kP0vD/naa21tli0Hj9eaycnrUHratW0/v333I1DPk/ZY4/nCQjVWfyOt8UtuN1AVaXUA0opF+A5YM1tdU4DbQCUUr5AdeC4DdoWQuSiPVF72PjPRqa3n06/uv24cAHatTOe8aSkwNChxi23xo3NjlTkRznuBae1TlFKDQF+AByBuVrrg0qpQanlM4GxwDyl1F+AAkZqrS/ltG0hRO5qUr4JR4Yc4YHiDxASAs8/D1FRUKoULFhgJCMh7pdNZkLQWm8ANty2b+Yt30cCj9uiLSFE7tJa8/ZPb1OvbD2ee/g5KhZ5gLFj4cMPwWqF5s1h8WLwy/5SPyIXtWzZkocffpjp06ebHco9k5kQhBDpjNk2hom/TWRXxC4uXjSuct5/35jTbfRo+Pnn/J98Ll68yGuvvYa/vz+urq74+vrSpk0bNt+cM+guQkJCUEpx6VLe3ciZN29eph0MVq5caZOJQc0gc8EJIdJ89ttnjNk2hv51+9PbZzING8Lp08YS2AsXwuMF5D5G9+7diY+P56uvvqJKlSpcuHCBbdu2cfny5TyPJTk5GRcXl/t+f4kSJWwYTd6SKyAhBACz98xm+Obh9AjoQdu4r2jR3IHTp6FJEwgLKzjJJzo6mu3bt/PJJ5/Qpk0bKlWqRKNGjRg+fDjPPfccAAsXLqRRo0Z4e3tTunRpevTowdmzZwE4efIkrVq1AowBm0op+vfvDxi3w4YMGZKuvf79+9OxY8e01y1btuTVV19l+PDh+Pj40KxZMwAmT55M7dq18fT0xM/Pj5deeono6GjAuOJ64YUXuH79OkoplFJ8+OGHmbbp7+/PuHHjeOWVVyhSpAjly5fn008/TRdTeHg4gYGBuLm5Ub16dTZs2ICXlxfz5s2zyTnOLklAQggATsecpv2DHan4x2J693YgIQEGDICQEChXzuzobMfLywsvLy/WrFmTbpG1WyUnJzNmzBj27dvHunXruHTpEj179gSgQoUKfPfddwAcPHiQqKgopk7Neh2kzCxcuBCtNdu3b+ebb74BjCWug4ODOXjwIN9++y1//PEHr79uDKds2rQpwcHBeHh4EBUVRVRUFMOHD8/y+FOmTKFWrVrs3buXkSNHMmLECHbu3AmA1Wqla9euODk5sWvXLubNm8eYMWPSLbmQV+QWnBCFXFJKEq5OrrxVbxzP/c/KZz864OQEwcHGEtkFbaV7Jycn5s2bx8svv8zs2bOpV68ezZo1o0ePHjzyyCMADBgwIK1+5cqVmTFjBjVr1iQiIoLy5cun3fYqXbo0pUqVuucYHnjgAT777LN0+95888207/39/Zk4cSKdO3dm/vz5uLi4ULRoUZRSlClT5q7Hf/zxx9Ouil5//XU+//xztmzZwqOPPsrmzZs5cuQIP/74I36pD/OmTJmSdiWWl+QKSIhCbPOxzVSdVpWNv//DI4/Ajz86UKoU/PQTDB5c8JLPTd27dycyMpK1a9fSvn17fvvtN5o0acLHH38MwN69e+ncuTOVKlXC29ubhg2NgfynT5+2SfsNGjTIsO/nn38mKCiI8uXL4+3tTbdu3UhOTubcuXP3fPzatWune12uXDkuXLgAwOHDhylXrlxa8gFo1KgRDg55nw4kAQlRSO04vYMuS7vgEtmK3h0e5J9/oE4dCA2FwECzo8t9bm5uBAUF8f777/Pbb7/x4osv8uGHHxITE8MTTzyBh4cHCxYsYPfu3WzatAkwbs3diYODQ7oZpIEMSxoAaRN83nTq1Ck6dOhAzZo1Wb58OXv27GHu3LnZajMzzrete6GUwmo1pt/UWqPs5C8LSUBCFEJ7o/by5LdPUvSfl4mYNo8rlx148knYvh1ssNJyvhQQEEBKSgphYWFcunSJjz/+mBYtWlCjRo20q4ebbvZas1gs6fb7+PgQFRWVbt++ffvu2nZoaCjJyclMmTKFRx99lGrVqhEZGZmhzdvbux81a9bk7Nmz6Y4fGhqalqDykiQgIQqZfy7/Q9A3j+Pw6yii5gWTlKR49VVYvRq8vc2OLvddvnyZ1q1bs3DhQvbv38+JEydYvnw5EydOpE2bNgQEBODq6sr06dM5fvw469ev57333kt3jEqVKqGUYv369Vy8eJG4uDgAWrduzcaNG1mzZg1Hjhxh2LBhnDlzJrMw0qlatSpWq5Xg4GBOnDjB4sWLCQ4OTlfH39+fxMRENm/ezKVLl4iPj7+vnz8oKIjq1avTr18/9u3bx65duxg2bBhOTk55fmUkCUiIQqa0ux8lflpB9PqRKAWTJsEXXxjr+BQGXl5eNGnShKlTpxIYGMhDDz3EqFGjeP7551m6dCk+Pj7Mnz+f77//noCAAMaMGcPkyZPTHcPPz48xY8YwevRofH190x74DxgwIG1r1qwZXl5edO3a9a4x1a5dm6lTpzJ58mQCAgKYM2cOkyZNSlenadOmDBo0iJ49e+Lj48PEiRPv6+d3cHBg1apVJCUl0bhxY/r168fo0aNRSuHm5nZfx7xvWc1Sag+bzIadf8l5uru8ng377LWzOvJytO7QwZjF2s1N6xUr8qz5HJHPU/bc73kKCwvTgA4NDbVtQPrOs2EXkr95hCjcLly/QOCMLlz8v7nEHC1KyZKwbp0xyFQUPqtWrcLT05OqVaty8uRJhg0bRp06dahfv36exiEJSIgC7mrCVVpN68Wx4Lno8w9Tvjxs3gw1apgdmTBLbGwsI0eO5MyZMxQvXpyWLVsyZcqUPH8GJAlIiAIsLjmO1lMG8ven/wfR/tSoAT/+CBUqmB2ZMFPfvn3p27ev2WFIAhKiIHtu2v8I+3g6XPelcWNYv95Yy0cIeyAJSIgCascO+GXMGLjuQFAQrFwJmczmL4RppBu2EAWMxWphxOyNPPGEJjbWgaefhrVrJfkI+yMJSIgCxKqtdPjocz4d3JLr1xV9+xqrl7q6mh2ZEBlJAhKigNBa0/nDufwwdjCkuPPKK/D114VngKnIfyQBCVFAdH9/GevG9QOrC0OHambMABMmOBYi2+TjKUQBMGnGBVaNfxqszowcqQkOVgV2KQVRcEgCEiKfW7AARgwuDdqRD8dYmTBBko/IH2ySgJRS7ZRSR5RSR5VSb2dRp6VSKkwpdVAptc0W7QpR2L368Q769dNoDePHwwfvO0jyEflGjh9PKqUcgS+AICAC2K2UWqO1/vuWOsWAL4F2WuvTSqnSOW1XiMLuP5/uZua7TUArxo61MmqU3NAQ+YstPrGNgaNa6+Na62RgCdD5tjrPAyu11qcBtNYXEELct3eC9xE8sj5oR979IIl335XkI/IfW3xq/YBbV1yKSN13q2pAcaVUiFJqj1LK/EmIhMinPpx+iE+GPQzakbffTWDshzLIR+RPthghkNkdZ33bayegAdAGcAd2KqV2aa3DMxxMqYHAQABfX19CQkJsEKLtxMXF2V1M9kjO091FR0djsVju6TyFhPjw0dgaoB15ptdhnmhzjsJwmuXzlD357TzZIgFFALfOrVseiMykziWt9XXgulLqF6AOkCEBaa1nA7MBGjZsqFu2bGmDEG0nJCQEe4vJHsl5urtixYoRHR2d7fO0bLmVceMc0FYY8U4yn4yvgVKFY00F+TxlT347T7a4BbcbqKqUekAp5QI8B6y5rc5qoLlSykkp5QE8AhyyQdtCFAozF0by7HMWLBYYNQo+Ge8ivd1EvpfjKyCtdYpSagjwA+AIzNVaH1RKDUotn6m1PqSU2gTsB6zAHK31gZy2LURhsHDVBV7tXxKszgwYcolx40pJ8hEFgk1midJabwA23LZv5m2vPwU+tUV7QhQWq3+4Qt9nvcHiyrMDLjDn89KSfESBIX03hbBTP/1yjW5dXNA33On47Hm+/T9JPqJgkXlyhbBD+/fDM128sSYqWnY8x/eLysjEoqLAkY+0EHYm7K9E2rS1cvWqonNn+HFlGRwdzY5KCNuTBCSEHTlyNJkmgXFcuuhAUJCVpUvB2dnsqITIHZKAhLATJ0+n0KDZVZKulqJqvSi+/95BVjIVBZokICHsQNQ5K3WaXuT6BV8qBkQRGlIWDw+zoxIid0kCEsJkV65AvWYXuXa2LL6Vz/Hn9rIUKWJ2VELkPklAQpjIYvGkXTs4f9wXn4pX2PebLyVKmB2VEHlDumELYRKLxY1/IiaSFA0PPADbt5fA19fsqITIO3IFJIQJEhNhz9F3SYpuTDGfOLZsAb/bFzERooCTBCREHrtxAx5td5qEc4+h3C+wPcSVBx4wOyoh8p4kICHykMUCLTudJmxbRZTrVR6s9CoPB8hAH1E4yTMgIfKI1Qr9BiTx26aKOLpd5+GHRkDKMbPDEsI0cgUkRB7QGoYOhUXfuOLmbmHdOk0xr38y1Dt48CDTpk1jw4YNhIeHk5ycbEK0QuQNuQISIpdpDX0HR7JwRjlcXWHtGkfatvHik7EZ64aHhzN06FCKFCmC1WolISGB4sWL4+/vT0BAAA8//DBVqlShSpUqVK5cGU9Pz7z/gYSwEUlAQuSywSPPsXBGOXC4waIlFtq2dcuybpcuXXj44Yc5cODf9RovXbrEpUuXCA0NxdnZGXd3d7TWJCQk4OnpSYUKFXj55ZcZOnRoXvw4QtiM3IITIhe9/dFFZnxaBpSF6V9dpXuXrJMPgFKKWbNm4ZHFPDw3btzg2rVrxMbGkpKSQkxMDOHh4ShZKEjkQ5KAhMgl4z67zP8+8AFgwufnGdy/dLbe17RpU1q0aIFjNtZgcHR0pGHDhgwePDhHsQphBklAQuSC+fPhveElARj1SQRvDyl3T++fNm0aztlYh8Hd3Z2lS5fiIKvViXxIPrVC2NiSJZoBA4zvP/w4jvEjy9/zMapUqUK/fv1wvct6DBaLhe+++w6r1Xo/oQphKklAQtjQ0pXxPN/bgtUKH30EH7zjdd/HGj9+PE5Od+4nlJCQwKhRo2jWrBlnzpy577aEMIMkICFsZO3GRHo+64S2ONH95aO8+27OjleyZElGjx6dZYeEm+Lj49m9ezcBAQHMnz8frXXOGhYij9gkASml2imljiiljiql3r5DvUZKKYtS6mlbtCuEvfhpazJduoBOcaHd8+Esn1UFW3RM+89//pNhrI+bmxtubul701ksFuLi4hg8eDDt27fn4sWLOW9ciFyW4wSklHIEvgDaAwFAT6VUQBb1/gf8kNM2hbAnIdtSaNfegjXZjcc6h7N+QTWbJB8wks2UKVPSkpCrqysdO3akV69euLu7Z6h//fp1fv75Z6pWrcrq1attE4QQucQWV0CNgaNa6+Na62RgCdA5k3qvA98BF2zQphB24ddfoWMHRyxJ7jR68m9CvquGrTuk9ezZkwoVKgBQtGhRvvrqK+bMmcOqVasoUaIELi4u6erfuHGDmJgYnn/+eXr27Mm1a9dsG5AQNmKL/yp+wK1PPyNS96VRSvkBXYGZNmhPCLvw66+a9u01168r+vTR7FwTQDaG7twzBwcHZs6ciVKKZcuWUSR1ve4nnniCo0eP0qFDh0yfE8XHx7Nq1SqqVKlCSEiI7QMTIodsMRVPZjcbbn8KGgyM1Fpb7jZiWyk1EBgI4Ovra3f/ceLi4uwuJntU0M/TX395M+y/AaQkuRPYOoJ+/Y6yffu9HSM6OhqLxZLt87R06VK01hnqDx06lFq1ajFp0iSSkpKwWCxpZUlJSVy8eJF27drxxBNP8Nprr921a7c9KuifJ1vJd+dJa52jDXgU+OGW1+8A79xW5wRwMnWLw7gN1+Vux27QoIG2N1u3bjU7hHyhIJ+nnTu1dvVI1KB11cDf9Y0b1vs6TmBgoK5Tp47N4oqKitKtWrXSnp6eGuOPwHSbu7u7rlChgt69e7fN2swrBfnzZEv2eJ6AUJ3F73hb3ILbDVRVSj2glHIBngPW3JbkHtBa+2ut/YEVwGta6+9t0LYQeer336FV2ySS4l3xb76TA5sb4uRkH/OwlSlThi1btqR1Wrh9doSEhATOnDlDixYtGDVqFDdu3DApUiEMOU5AWusUYAhG77ZDwDKt9UGl1CCl1KCcHl8Ie/HHH9CqbTKJ113xe/RXDv3UCBdn+xpKp5Ti5Zdf5q+//qJu3bqZLteQkJDA1KlTqV27NocOHTIhSiEMNvnfo7XeoLWuprV+UGs9PnXfTK11hk4HWuv+WusVtmhXiLyyYwcEBUFCnAtVHvuTIz83ws3FflczeeCBB/jjjz947733cHd3zzBbdnx8PEeOHKFBgwZMmjRJpvIRprCvP99EtrVs2ZIhQ4aYHUahsHUrBD1u4do1eOYZ+Pvneni62f+DfEdHR0aOHMnu3bupWrVqhp5yOnVNoQ8++IAmTZpw6tQpkyIVhVWhSkAXL17ktddew9/fH1dXV3x9fWnTpg2bN2/O1vtDQkJo1aoVly5dyuVI/zVv3jy8vDLOJ7Zy5UomTJiQZ3EUVps2Qbv2FhLiHXk4aA/ffgvZmKTarjz00EP89ddfvP7665kOXo2Pj2fv3r089NBDfPXVVzKVj8gzhSoBde/enT/++IOvvvqK8PBw1q1bR/v27bl8+XKex5KcnJyj95coUQJvb28bRSMys3o1dOpsJTnJkeKPLePnlf65Ms4nL7i4uPDJJ5+wdetW/Pz8MiQii8XC9evXGTp0KEFBQZw/f96kSEWhklX3OHvYbNkN++rVqxrQmzdvzrLOggULdMOGDbWXl5f28fHRTz/9tI6IiNBaa33ixIkM3Vr79euntTa60w4ePDjdsfr166c7dOiQ9jowMFAPGjRIv/XWW7pUqVK6YcOGWmutP/vsM12rVi3t4eGhy5Urp1988UV99epVrbXRpfL2Nj/44INM26xUqZIeO3asHjhwoPb29tZ+fn564sSJ6WI6cuSIbtGihXZ1ddXVqlXT69ev156envrrr7++n1N6R/bYHfReLF2qtZOTVYPWRQO/1mdjIm3ehq27YWfX9evX9csvv6w9PDwy7a7t7OysixQpolesWJHnsWUlv3+e8oo9nidyuRt2vuDl5YWXlxdr1qwhMTEx0zrJycmMGTOGffv2sW7dOi5dukTPnj0BqFChAt999x0ABw8eJCoqiqlTp95TDAsXLkRrzfbt2/nmm28AY5R7cHAwBw8e5Ntvv+WPP/7g9ddfB4yVMYODg/Hw8CAqKoqoqCiGDx+e5fGnTJlCrVq12Lt3LyNHjmTEiBHs3LkTAKvVSteuXXFycmLXrl3MmzePMWPGkJSUdE8/Q2Ewdy707KlJSVF4t5lO2KpWlCtS1uywbMbDw4PZs2ezZs0aSpUqlWFg6s1lv/v27cvTTz9NdHS0OYGKgi+rzGQPm60Hoq5YsUIXL15cu7q66iZNmui33npL79q1K8v6hw4d0oA+c+aM1vrfK5KLFy+mq5fdK6BatWrdNcaNGzdqFxcXbbFYtNZaf/3119rT0zNDvcyugJ577rl0dapUqaLHjh2rtdZ606ZN2tHRMe2KTmutd+zYoQG5AkpltWr9v/9pDcb2yvAIffjikVxrz6wroFtdvXpV9+jRI8urIVdXV12yZMk73jnIC/nx82QGezxPyBWQoXv37kRGRrJ27Vrat2/Pb7/9RpMmTfj4448B2Lt3L507d6ZSpUp4e3vTsGFDAE6fPm2T9hs0aJBh388//0xQUBDly5fH29ubbt26kZyczLlz5+75+LVr1073uly5cly4YMz9evjwYcqVK4ef37/T9DVq1EiWck5ltcJ//wsjR4JSmunTYeanflQvVc3s0HJVsWLFWLZsGQsWLKBIkSIZlgFPSkri8uXLdOrUiYEDBxIfH29SpKIgKnS/fdzc3AgKCuL999/nt99+48UXX+TDDz8kJiaGJ554Ag8PDxYsWMDu3bvZtGkTcPcOAw4ODhl6DmU2yvz2QYGnTp2iQ4cO1KxZk+XLl7Nnzx7mzp2brTYzc/svD6VU2vgOrXWGsSDCcOMGDBgAn30GON7A5ZkX6NLnrNlh5alu3boRHh5OYGBgphObJiQksGDBAqpVq8bvv/9uQoSiICp0Ceh2AQEBpKSkEBYWxqVLl/j4449p0aIFNWrUSLt6uOnmtPe3TvYI4OPjQ1RUVLp9+/btu2vboaGhJCcnM2XKFB599FGqVatGZGRkhjZvb+9+1KxZk7Nnz6Y7fmhoaKEfgJiQAN26wfz54OCSgHPvrvzwyQv4FfG7+5sLGF9fX3788UemTZuGp6cnjrd1+UtMTOTs2bO0aNGCrVu3mhSlKEgKTQK6fPkyrVu3ZuHChezfv58TJ06wfPlyJk6cSJs2bQgICMDV1ZXp06dz/Phx1q9fz3vvvZfuGJUqVUIpxfr167l48SJxcXEAtG7dmo0bN7JmzRqOHDnCsGHDOHPmTGZhpFO1alWsVivBwcGcOHGCxYsXExwcnK6Ov78/iYmJbN68mUuXLt33LZCgoCCqV69Ov3792LdvH7t27WLYsGE4OTkV2iujy5fh8cdh3Tpw8oxB9WvL6tGDCfQPNDs00yilGDBgAAcPHqRBgwaZTuXj5eVF3bp18z44UeAUmgTk5eVFkyZNmDp1KoGBgTz00EOMGjWK559/nqVLl+Lj48P8+fP5/vvvCQgIYMyYMUyePDndMfz8/Ojfvz+jR4/G19c3bSaCAQMGpG3NmjXDy8uLrl273jWm2rVrM3XqVCZPnkxAQABz5sxh0qRJ6eo0bdqUQYMG0bNnT3x8fJg4ceJ9/fwODg6sWrWKpKQkGjduTL9+/Rg9ejRKqQzLOxcGR4/Co48aC8qV9E1Av9CcJW/+h/ZV25sdml2oVKkSO3fuZMyYMXh4eKT9keLh4cG3335L8eLFTY5QFAhZ9U6wh02WY8hdYWFhGtChoaE2P7Y9n6cdO7QuVcro6VanjtYREVofu3Isz+Owh15w2XHo0CFds2ZN7ezsrF944QVTYrDnz5M9scfzxB16wdnvbIrC5latWoWnpydVq1bl5MmTDBs2jDp16lC/fn2zQ8szy5dDnz6QlATl6//F2K8v4+fXEqhsdmh2q0aNGuzbt4/FixfTvXt3s8MRBUihuQUnIDY2liFDhhAQEECvXr2oWbMmP/zwQ6F4BqQ1TJxoTCaalAQ124UQ0aEeh2L/MDu0fMHZ2Zm+fftm+kxIiPslV0CFSN++fenbt6/ZYeS5pCR47TVjhgOA5i+uZXv5TrzT/B1GNBthbnBCFGJyBSQKtMhICAw0ko+bGzz9wTK2V+jE64+8zvjW480Or9Dx9/fP0NFGFF5yBSQKrF27jDE+UVFQsSKsXKmZHrGR/vQnuF1wobj1aIb+/ftz6dIl1q1bl6Fs9+7dchtPpJEEJAqkuXPh1VchORlatICFi5OoUM6Vr+ob6904KLn4N4OPj4/ZIQDGTCM3B5YL8xS4/4WRkZFZznYtCr7kZHj9dXjxReP7wYNhwORvaf3dw5yJOYODcsDRIZ8u6lMA3H4LTinF7Nmz6dGjB56enlSuXJmFCxeme8/Zs2f56KOPKF68OMWLF6dDhw78888/aeXHjh2jc+fOlClTBk9PT+rXr5/h6svf358PP/yQAQMGUKxYMXr16pW7P6jIlgKVgFJSUqhbty4+Pj7897//zdZsBKLgOHECHnsMpk8HFxeYMweChqzmxfV98fP2o5RHKbNDFJn46KOP6Ny5M/v27ePZZ59lwIABacuDx8fH06pVK1xcXNi2bRs7d+6kbNmytG3bNm1WkLi4ONq3b8/mzZvZt28f3bt3p1u3bhw+fDhdO5MnT6ZGjRqEhoamTUAszFWgEtDq1atJTEwkLi6Ozz//nKpVqxbKXl+F0apVUK8e7N4NlSrBL79AxZabeWbFMzQo14C1Pdfi7pxxOWphvj59+tC7d2+qVKnC2LFjcXJyYvv27QAsWbIErTUjR46kdu3a1KhRg1mzZhEXF5d2lVOnTh0GDRpErVq1qFKlCqNHj6Z+/fqsWLEiXTuBgYGMGDGCKlWqULVq1Tz/OUVGBSoBjR8/ntjYWMC4x5uUlJRhck9RsCQnw5tvGp0NYmKgc2f4809Q5f+gy9Iu1ChVg429NuLtKsuX26tblxFxcnLCx8cnbSLgPXv2cOLECZ588sm0RSWLFi3K1atXOXbsGADXr19nxIgRBAQEULx4cby8vAgNDc2wjMrN5VWE/bBJJwSlVDtgKuAIzNFaf3JbeS9gZOrLOOBVrfXdp4u+B/v37+fIkSPp9nl5efHOO+/YshlhR44fh2efhdBQcHY2Bpq+8QYoBVXcqtC5ememPDGFEu4lzA5V3MGdlhGxWq3UrVuX//znPzzyyCPp6pUoYfy7Dh8+nE2bNjFp0iSqVq2Kh4cHffv2zbCkifS+sz85TkBKKUfgCyAIiAB2K6XWaK3/vqXaCSBQa31VKdUemA08kvFo92/ixIkZOh+UKFGC1q1b27IZYQe0Np7v/Oc/cP06+PvD0qXQuDEcv3ocP28/SriX4Nvu35odqsih+vXrs3jxYooWLUqVKlUyrfPrr7/St2/ftGmCEhMTOXbsGNWqFezFBAsCW1wBNQaOaq2PAyillgCdgbQEpLX+7Zb6u4DyNmg3zeXLl/nuu+/SrW3j6enJyJEjZaxHARMVBS+9BBs2GK+feQZmzoTixeHYlWM0/7o5bSu35Zuu35gbaCF37do1wsLC0u0rVqzYPR+nV69eTJo0idGjR+Pt7U3FihU5c+YMq1evZtCgQVStWpVq1aqxatUqOnfujLOzM2PGjJGesPmELRKQH3Brd7MI7nx18yKw0Qbtppk9e3aGRKO1lg4IBczy5TBoEFy5AsWKwYwZ8NxzRlnEtQjaLmhLkiWJkc1G3vE4Ivdt376devXqpdt3PxOZenh48Msvv9C/f3969OhBTEwM5cqVo1WrVmlLQkyePJkXX3yR5s2bU7x4cd58801JQPmE0rctJX3PB1CqB/CE1vql1Nd9gMZa69czqdsK+BJ4TGt9OYvjDQQGAvj6+jZYsmTJHdu3WCx0796dmJiYtH2Ojo489dRTvPHGG/f5U2UtLi4OLy8vmx+3oLHleYqJcWbatCps2eILQKNGV/jvfw/j42Pc47+afJU3wt7gcvJlJteZTHXv6jZpN7e9+eabWCwWpk2bZnYodk/+32WPPZ6nVq1a7dFaZ94DJKt1GrK7AY8CP9zy+h3gnUzq1QaOAdWye+zsrAe0cuVK7e3trYG0zc3NTR87ljvru9jjehv2yBbnyWrV+uuvtS5Z0li7x8ND6xkzjP23aj2/tXYf565/OflLjtvMS/llPSB7IP/vsscezxO5vB7QbqCqUuoB4CzwHPD8rRWUUhWBlUAfrXW4DdpM8/HHH6d1vb6pSZMmVK4s67vkZ0eOGLfbQkKM123aGM96MnsOPeWJKZyPO0/zSs3zNEYhRM7keByQ1joFGAL8ABwClmmtDyqlBimlBqVWex8oCXyplApTSoXmtF2AAwcOcPDgwXT7vLy8GDVqlC0OL0yQlAQffQS1axvJp1Qp+OYb2Lw5ffJJTElkwb4FaK2p7VuboAeDTItZCHF/bDIOSGu9Adhw276Zt3z/EvCSLdq61cSJEzP09S9WrBht27a1dVMil2kNa9bAf/8LN6f5GjDAGNtTsmT6ujcsN+ixvAfrw9fzUOmHqF+28KzoKkRBkm9nw7569SrLly/HYrGk7fPw8GDEiBHS9Tqf2bfPGNOzdavxukYN43ZbYGDGuharhT6r+rAufB0zOsyQ5CNEPpZvp+L5v//7PxwcMobfv3//vA9G3Jdz5+Dll4053LZuNcbyTJ0K+/dnnnys2sor615h6cGlTGw7kUENB2WsJITIN/LlFZDFYmHSpElps+GCMYdU37598faWOb/sXXQ0TJkCkydDXBw4ORnLJrz/PpS4w6w5u8/u5uuwr3mvxXv8t9l/8yxeIUTuyJcJaP369RkGmjk5OTFs2DCTIhLZERtrXOF89pmRhAA6doRJk6B6NobuPFL+EUJfDqVumbq5GaYQIo/ky1twmXW9btSokUyxbqeuX4f//Q8eeADee89IPi1bGksmrF179+QzZecU1oevB6Be2XryjE+IAiLfJaC///6b/fv3p9snXa/t08WLMGaMMVno22/D5cvQrBn8/LPxzKd5NobtzNg9g2E/DmPJwTvPiCGEyH/y3S24SZMmZeh6XaRIER5//HGTIhK3O3oUpkypyubNkJBg7HvkEWN8T1CQsVxCdizYt4DXNrxGx2odmdtpbu4FLIQwRb5KQDExMSxZsiTTrteZ9YgTeUdr2LHDeMazciVYrX4APPmkMbYnMDD7iQdg5aGV9F/dn9YPtGZ5j+U4Ozrf/U1CiHzFrn9rR0RE8MUXXxAXFwfAnDlzMp31+oUXXjAjPIGxCun06VCrlnFLbcUKcHSEdu2iOHAA1q83nvfc62ObbSe30divMaufW42bk1uuxC6EMJddXwFdvXqVt956ixEjRtCvXz9WrlyZoet1r169KFKkiIlRFj5aG6uQzpoFixfDzX8SX1948UWjS3V4+BEeeqjsPR/bqq04KAeC2wUTfyMeTxdZxVKIgsquE5CjoyMJqQ8R5syZg5NT+nCdnZ0ZPny4GaEVSqdOwaJFsHAhHDr07/42bYyJQzt1AhcXY1/4fUw5GxoZSv/v+7Py2ZVUK1lNko8QBZzdJ6Cbbty4wY0bN9KV+/r6Uq5cubwOq1C5etW4rbZwodFt+iYfH+jdG155JXtjeO7mwIUDPLHwCYq6FsXTWRKPEIVBvklAmblw4QJlypShd+/eDB8+XMYB2UhkJKxebXQm2LoVbvb5cHODLl2gTx+jN5uzjfoF/HP5H4IWBOHq6MpPfX/Cr4ifbQ4shLBrdp2Abr/ldrubz4PmzJnD3LlzuXjx4n2tO1/YaQ1//w0bNsCqVbBz579ljo7GLbbevaFbN7D147abS2mnWFPY1n8blYvLOk5CFBZ2nYCcs/kntpubG4sXL5bkcw+io+Gnn2DTJvjhB4iI+LfMzQ0ef9xIOB07ZlwOwZa8XbypVboWH7X6iACfgNxrSAhhd+w6ATk6OuLk5ERKSkqm5U5OThQtWpSffvqJunXr5m1w+cy1a8Y4nW3bjG337n9vrYHRg+3xx42OBO3aQW4vK3814SquTq4UdSvKuufX5W5jQgi7ZPcJyMXFJdME5Obmhr+/P1u2bJGOCLfR2rii+eMP43batm2wdy9Yrf/WcXIyBoe2awdPPAF16kBejeWNTYql3aJ2FHMrxqZem2RuNyEKKbtPQJl1RPDw8KBFixasWLECT0/pMXX1qjEu548//t3OnUtfx8kJGjc2kk5goDEnmxnDpxJuJPDU4qfYE7mHlc+ulOQjRCFm9wno9il2PDw8GDhwIJ999lmhm34nJcVYrvqvv4ztwAHj67FjGesWL24knEaNoEULePTR3L+tdjfJlmS6L+vOL6d+YVG3RXSq3sncgIQQprLrBHR7Lzh3d3eCg4N5+eWXTYoob8TFGYnmn3+MAZ3h4UaiOXQIkpIy1nd1hfr1jYRzc3vwwXuf/ia3Dd04lI1HNzK742x61uppdjhCCJPZdQJydHTEarWilMLLy4s1a9bQsmVLs8PKsZQUY6zN6dPGduoUHD/+b8KJisr6vZUqGfOu3bpVr267MTm56c0mb1KvTD1eblCw/4AQQmSP3Seg2NhYypUrx9atW6lWrZrZId1VfLzx/OXWLSLCSDI3E87Zs+l7oN3OxQWqVIGqVf/dHn7Y2PLbtHdaa9aFr6NjtY7UKFWDGqVqmB2SEMJO2HUCcnJy4u2332b48OGUzM3BKFmwWIzZni9fhitXjO3XX33Zv9/Yd/kynD//b6I5f95Ydjo7ypY1rmYqVjQ2f3+oVs1INhUqGANAC4LRP49mwq8TWPXsKrrU6GJ2OEIIO2KTBKSUagdMBRyBOVrrT24rV6nlTwLxQH+t9d5sHJcJEybcUyxaG89JEhKMq5HYWGO7di3z729/HRPzb7K5etU4Xno179i+qyuUKWOMq7n5tUKFfxNNpUrg52fUK+gmbJ/AhF8n8EqDV+hcvbPZ4Qgh7EyOE5BSyhH4AggCIoDdSqk1Wuu/b6nWHqiauj0CzEj9ekdnz8LQoUYyyc4WHw+JiZkljftXtKgxE0CJEsaWknKemjV9KVHC2H8z0dzciha1v4f/Zlh1dhWfH/2cXrV68WWHL6W7tRAiA1tcATUGjmqtjwMopZYAnYFbE1Bn4ButtQZ2KaWKKaXKaq3v8LjduK01bdq9B+TsDO7u4OEB3t7GcxNv7+x9X6QIacmlWDFj/MytQkIO0bKl770HVYicij7Fl8e+pHP1znzd+WscVOHqLi+EyB5bJCA/4MwtryPIeHWTWR0/4I4JyMXlIhUqTMPBIQlHxyQcHJJxcEhK2xwdM3+tlDXDsbQ2brVdu3ZfP2Oa6OhomXMuGyo5VuLK1isEzQwyOxS7FRYWRkpKSoHo2Znb5P9d9uS382SLBJTZvZXbb4Jlp45RUamBwEAwJiP18voq00a1Nroz5zWLxUJ0dHTeN5wPXPO9htXRSrHIYrhb3LnmmMNsX8ClpKSgtZbPUzbI/7vsyW/nyRYJKAKocMvr8kDkfdQBQGs9G5gN0LBhQx0aGmqDEG0nJCRE/mLNRMjJENovak+t0rXY+eJOtv+yXc7TXbRs2ZLo6GjCwsLMDsXuyf+77LHH83Sn57+2uDm/G6iqlHpAKeUCPAesua3OGqCvMjQBYu72/EfkH3+c/YOnFj9F5eKV2dBrA44OBaQPuRAiV+X4CkhrnaKUGgL8gNENe67W+qBSalBq+UxgA0YX7KMY3bBfyGm7wj7sP7+fdgvbUdqzNJv7bKaURymzQxJC5BM2GQektd6AkWRu3Tfzlu81MNgWbQn7svrwajycPdjSdwvlvGVZDCFE9kn/WHFfdOpgq3dbvMufr/yJfzF/cwMSQuQ7koDEPYuKjSJwXiAHLhxAKYWPp4/ZIQkh8iG7ngtO2J/L8ZcJWhDEyeiTxCXHmR2OECIfkwQksu1a0jXaLWrH0StH2dBrA03KNzE7JCFEPiYJSGRL/I14On7bkbBzYXz/7Pe0fqC12SEJIfI5eQYkssWqrbg7u7Oo2yI6VOtgdjhCiAJAroDEHaVYU0hMScTLxYtNvTbJrNZCCJuRKyCRJau2MmD1AIIWBJFsSZbkI4SwKUlAIlNaa4ZsGMKC/QvoWLUjLo4uZockhChgJAGJDLTWvP3T28wIncGIpiMY1XyU2SEJIQogSUAig6m/T2XibxN5teGrfNL2E7n1JoTIFdIJQWTQqXonomKjmNB2giQfIUSukSsgkWZXxC601lQuXpn/Bf1PltIWQuQq+Q0jAFh6YClNv2rK9D+mmx2KEKKQkAQkWBe+jt6revNYxcd4sf6LZocjhCgkJAEVcj+f+Jmnlz1N3TJ1Wff8OjycPcwOSQhRSEgCKsRik2LpsbwHVUpUYVOvTRRxLWJ2SEKIQkR6wRVi3q7erOixghqlalDSo6TZ4QghChm5AiqEDl86zNIDSwFo9UArynqXNTkiIURhJFdAhcyJqydo+01brNpKh2od8HLxMjskIUQhJQmoEImMjaTtgrbE34hnW/9tknyEEKaSBFRIXIq/RNCCIC5cv8CWvluo5VvL7JCEEIVcjp4BKaVKKKU2K6X+Sf1aPJM6FZRSW5VSh5RSB5VSb+SkTXF/Vvy9guNXj7Ou5zoa+zU2OxwhhMhxJ4S3gS1a66rAltTXt0sB3tJa1wSaAIOVUgE5bFfco0ENB3HwtYME+geaHYoQQgA5T0Cdgfmp388HutxeQWsdpbXem/p9LHAI8MthuyIbElMSef6759kbtReAysUrmxyREEL8K6cJyFdrHQVGogFK36myUsofqAf8nsN2xV3csNzguRXPsfjAYg5eOGh2OEIIkYHSWt+5glI/AWUyKRoNzNdaF7ul7lWtdYbnQKllXsA2YLzWeuUd2hsIDEx9WR04cscA814p4JLZQeQDcp6yR85T9sh5yh57PE+VtNY+mRXcNQHdiVLqCNBSax2llCoLhGitq2dSzxlYB/ygtZ583w3aAaVUqNa6odlx2Ds5T9kj5yl75DxlT347Tzm9BbcG6Jf6fT9g9e0VlLGi2VfAofyefIQQQthOThPQJ0CQUuofICj1NUqpckqpDal1mgF9gNZKqbDU7ckctiuEECKfy9FAVK31ZaBNJvsjgSdTv/8VKEjrOs82O4B8Qs5T9sh5yh45T9mTr85Tjp4BCSGEEPdLZsMWQghhCklA90kpNVwppZVSpcyOxR4ppT5VSh1WSu1XSq1SShUzOyZ7opRqp5Q6opQ6qpTKbAaRQk+m8bo3SilHpdSfSql1ZseSXZKA7oNSqgJGp4vTZsdixzYDD2utawPhwDsmx2M3lFKOwBdAeyAA6CnTU2VKpvG6N29gzDSTb0gCuj9TgBGAPEDLgtb6R611SurLXUB5M+OxM42Bo1rr41rrZGAJxrRW4hYyjVf2KaXKAx2AOWbHci8kAd0jpVQn4KzWep/ZseQjA4CNZgdhR/yAM7e8jkB+sd6RTON1V8EYfxRbTY7jnsh6QJm4y/RDo4DH8zYi+3Sn86S1Xp1aZzTGrZRFeRmbnctsWIJcTWchdRqv74A3tdbXzI7H3iilOgIXtNZ7lFItTQ7nnkgCyoTWum1m+5VStYAHgH3GBA+UB/YqpRprrc/lYYh2IavzdJNSqh/QEWijpb//rSKACre8Lg9EmhSLXUudxus7YNGd5pAs5JoBnVIH+LsBRZRSC7XWvU2O665kHFAOKKVOAg211vY2+Z/plFLtgMlAoNb6otnx2BOllBNGx4w2wFlgN/C81lqmLb9F6jRe84ErWus3TQ4nX0i9Ahqute5ocijZIs+ARG6ZDngDm1OnX5ppdkD2IrVzxhDgB4wH68sk+WRKpvEq4OQKSAghhCnkCkgIIYQpJAEJIYQwhSQgIYQQppAEJIQQwhSSgIQQQphCEpAQQghTSAISQghhCklAQgghTCEJSIg8ppQakbqY4e3bR2bHJkRekpkQhMhjSilvwPOWXcOBXkBzrfVRc6ISIu9JAhLCREqpkcBQoLXW+ojZ8QiRl2Q5BiFMopR6B2NS0lZa63Cz4xEir0kCEsIEqQv1DcJYrkJuu4lCSRKQEHlMKfUe8DLQUmt9zOx4hDCLJCAh8lDqlc8bQCfgulLq5pLm0VrrRPMiEyLvSScEIfJI6gqf0UCRTIrbaq235G1EQphLEpAQQghTyEBUIYQQppAEJIQQwhSSgIQQQphCEpAQQghTSAISQghhCklAQgghTCEJSAghhCkkAQkhhDCFJCAhhBCm+H+7xj5SCcm/twAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# extra code – this cell generates and saves Figure 11–1\n",
"\n",
"import numpy as np\n",
"\n",
"def sigmoid(z):\n",
" return 1 / (1 + np.exp(-z))\n",
"\n",
"z = np.linspace(-5, 5, 200)\n",
"\n",
"plt.plot([-5, 5], [0, 0], 'k-')\n",
"plt.plot([-5, 5], [1, 1], 'k--')\n",
"plt.plot([0, 0], [-0.2, 1.2], 'k-')\n",
"plt.plot([-5, 5], [-3/4, 7/4], 'g--')\n",
"plt.plot(z, sigmoid(z), \"b-\", linewidth=2,\n",
" label=r\"$\\sigma(z) = \\dfrac{1}{1+e^{-z}}$\")\n",
"props = dict(facecolor='black', shrink=0.1)\n",
"plt.annotate('Saturating', xytext=(3.5, 0.7), xy=(5, 1), arrowprops=props,\n",
" fontsize=14, ha=\"center\")\n",
"plt.annotate('Saturating', xytext=(-3.5, 0.3), xy=(-5, 0), arrowprops=props,\n",
" fontsize=14, ha=\"center\")\n",
"plt.annotate('Linear', xytext=(2, 0.2), xy=(0, 0.5), arrowprops=props,\n",
" fontsize=14, ha=\"center\")\n",
"plt.grid(True)\n",
"plt.axis([-5, 5, -0.2, 1.2])\n",
"plt.xlabel(\"$z$\")\n",
"plt.legend(loc=\"upper left\", fontsize=16)\n",
"\n",
"save_fig(\"sigmoid_saturation_plot\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Xavier and He Initialization"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"dense = tf.keras.layers.Dense(50, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\")"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"he_avg_init = tf.keras.initializers.VarianceScaling(scale=2., mode=\"fan_avg\",\n",
" distribution=\"uniform\")\n",
"dense = tf.keras.layers.Dense(50, activation=\"sigmoid\",\n",
" kernel_initializer=he_avg_init)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Nonsaturating Activation Functions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Leaky ReLU"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAADkCAYAAADeiPCXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAg9ElEQVR4nO3deXhU9fk28PuZJEASEkBCggEFDLuILBGoCA2kUMrqqwgIBamWaHHtJQj059JXARdc3rpc1pZaqlB/gtVWLVqhJULQBhITQJIQliIFQ1LAGLLQJDPP+wdkZJjJZCaznDMz9+e65oI5y/c888zJ3DkzJ2dEVUFERBRsFqMLICKiyMQAIiIiQzCAiIjIEAwgIiIyBAOIiIgMwQAiIiJDRBux0aSkJO3Zs6cRm3arpqYG8fHxRpcRMtgvzx04cABWqxUDBw40upSQEan7V00NUFoK2GxAp05Ar16AiCfrmbNf+fn5p1S1i6t5hgRQz549kZeXZ8Sm3crOzkZGRobRZYQM9stzGRkZqKysNOV+b1aRuH/985/AxInnw2f2bGD9eiDaw1dps/ZLRL5qbh7fgiMiMoGm8Dl71vvwCVUMICIig0Vi+AAMICIiQ0Vq+AAMICIiw0Ry+AAMICIiQ0R6+AAGnQXXkqqqKlRUVKChoSGo2+3QoQOKi4uDus1Qxn557rHHHoOqhlW/YmJikJycjMTERKNLCTkMn/NM95CrqqpQXl6Obt26ITY2FuLJCfB+cvbsWSQkJARte6GO/fKcxWJBY2MjBgwYYHQpfqGqqKurw4kTJwCAIeQFhs93TPcWXEVFBbp164a4uLighg8ReU5EEBcXh27duqGiosLockIGw8eR6QKooaEBsbGxRpdBRB6IjY0N+lvloYrh48x0AQSARz5EIYI/q55h+LhmygAiIgoXDJ/mMYCIiAKE4eMeA4iIKAAYPi1jAIWYZcuWYcKECUaX4VfffPMNUlJScPjwYY+WnzlzJp5//vkAVxV83vYh0MK1z8HA8PEMA8jPxo8fjwULFgRs/MLCQgwZMsSnMcaPHw8RgYggJiYGffr0wW9/+1uvx5k6darbx9q5c2esWbPGafojjzyCHj162O+vXr0akydPRlpamkfbfeyxx7By5Up8++23XtdsZt72IdDCtc+BxvDxHAPIzwoKCjB8+PCAjb9nzx4MHTrUpzEKCgqwevVqlJWV4dChQ5gzZw7uvPNOFBQUeDXO3r17m32sR48exZkzZ5Cenu40Ly8vz75ebW0t1q5dizvuuMPj7V5zzTW46qqrsH79eq/qNbPW9CHQwrHPgcbw8Q4DyI8OHz6MysrKZl+UT5w4gQULFqBz587o2LEjbr75ZpSXlzsss3LlSgwePBjt27dHly5dsHDhQtTV1QEATp48ifLycvsRUE1NDebMmYNhw4bh6NGjAIDu3bs7vW2yb98+tGvXDkVFRfYaJ02ahK5du6JHjx648847oarYv3+/x/W29Fjz8/MhIhg2bJjLeU3BtHnzZlgsFowePdphmWeeecZ+lHbx7dFHHwUATJ8+HW+99ZbLbfvi+PHjEBG8/fbbGD9+POLi4nDttdeipKQEeXl5GDt2LOLi4jBixAgcO3bMvp675w0APv74YwwdOhRfffXdd3Pdf//9SEtLQ3l5ebN9AM73KzMzE7Gxsejduze2b9+OjRs32pdtqVfN8WS9QPU5HDF8WkFVg34bPny4NqeoqKjZeYFWVVXl0/pvv/22WiwWPXv2rNO8I0eOaHJysi5fvlyLioq0oKBAx44dqzfeeKPDco899pjm5OTo0aNHdcuWLZqamqqrV69WVdXNmzdrbGysNjY2aklJiQ4cOFDnzZuntbW19vVnzpypc+bMcRhz/Pjxevfdd9trTExM1MbGRlVVLSsr0zlz5qjFYtH9+/d7XK+7x6qqumLFCu3bt6/T9KNHjyoA/fjjj1VV9b777tMJEyY4LVdVVaVlZWX224MPPqhdu3bVgwcPqqrqRx99pDExMQ6PvcmqVas0Pj7e7W379u0u6/7ggw8UgH7/+9/X7Oxs3bt3r/br109Hjhyp48aN0x07dmhhYaH26tVL77//fvt67p43VdXi4mIdOHCg/vSnP1VV1TVr1miXLl20tLTUbR927dqlsbGx+sQTT2hpaanOmzdPMzIydPDgwfr3v//do141x5P13PW5SaB+Zrdt2xaQcQPh889VExJUAdXZs1UbGoJfg1n7BSBPm8mCkAggwJibtx566CHt37+/y3kTJ07UFStWOEzbsmWLJiQkuB1z0aJFumDBAlVVXb16tY4YMULfeecdveyyy/SFF15wWv65557TtLQ0+/333ntPO3XqpKdOnbLXaLFYND4+XmNjYxWAtmnTxmmslup96KGHXAbMxevPnTvXafo777yjAOz1zJgxw/74mvPUU09pamqqlpSU2Kft2bNHAeihQ4eclj99+rQePHjQ7a25F9SVK1dqhw4d9OTJk/Zp99xzjyYlJdlrVlVduHChzpo1q9maL37eVFVLSkr0tdde0+joaH3yySe1ffv2umvXLvv85vowZswYh+389a9/VYvFohkZGS6366pXnmhuPXd9bhLpAWSG8FE1b78YQEEKoMzMTJ03b57T9K+++koBaGxsrMNv4e3atdPLLrvMvtyxY8f03nvv1UGDBmmnTp00Pj5eo6Oj9Re/+IWqqs6aNUs7deqkiYmJmp2d7bKGnTt3KgA9ffq0njt3TtPS0hzCJTMzU7OysvTgwYOan5+vP/zhD+1HR97Um5mZ6fYFOCkpSZ9//nmn6StWrNAePXrY70+cOFGzsrKaHWf16tWampqqBw4ccJheWlqqAHTfvn3NrtsaM2fOdHoOp0+f7lTj+PHjddmyZara8vOmej6AvvzyS/3e976nUVFRunnzZofxXPWhrKxMATi8sGzZskUB6I4dO5xqb65XLXG3nid9juQAMkv4qJq3X+4CKCTeoVQNznZ8vbpzQUEBHn74YafphYWFSExMRH5+vtO8Nm3aAABOnz6N6667DmPHjsWzzz6L7t27IyoqCtddd539M5/CwkLcdNNN+OMf/4jTp0+7rGH48OFo06YN8vLyUFBQgOjoaNx9990ONc6fPx+9e/cGALz22mvo1asX7rzzTlxzzTUe11tQUIAlS5a4rOH48eM4deoUBg0a5DRvy5YtDp9zJCUl4ZtvvnE5zqpVq/DrX/8an376qb3eJmfOnAEAdOnSxWm91atXY/Xq1S7HbPLRRx9hzJgxTtP37NmD++67z2FaQUEBHn/8caflsrKyPHremuTm5mLPnj1QVaSkpDjMc9WHpq9uuO666+zTDhw4gH79+uGGG25wWNZdr9xpaT13fY50/MzHD5pLJk9vANoB2AVgD4D9AP5vS+uE42dAR44cUQAuj0w2b96sUVFRzX5eoqr6hz/8QTt06KA2m80+bd26dQpAS0tLtaamRi0Wi+7atUs3bNig8fHxmp+f73KskSNH6j333KMJCQn6wQcfONV46XrDhg3TpUuXelxv0ziX/hZ/6fy//OUvDtNzc3NVRBymr1mzRq+++mqnMR5//HG94oormn3rZ+3atZqamupyXmvfgquurlaLxaI5OTkOYwHQwsJC+7Rjx44pAD1w4ECLz1uTP//5z5qQkKC/+93v9KabbtKJEyc6bNtVH959910VEXutVVVVevnll+vQoUO96lVzPFnPXZ+bROIRkJmOfJqYtV8I5FtwAARA+wv/jwGQC2CUu3XCMYA2bdqkAHTnzp26b98++62oqEjPnDmjSUlJeuONN+oXX3yhhw4d0k8++UQXL16sVqtVVVU//PBDjYqK0nfffVcPHjyoL774onbt2lUTEhLUZrPpZ599plFRUfYXo4cfflhTU1P1+PHjTrU88MADKiJOL3KbNm1Si8Xi9OK7bNky7d27t/1+S/U2PdYtW7Y4PVZVVZvNpgMGDNBBgwbpJ598onv27NHXX39dU1NTddq0aQ7b3rt3r1osFofPV1auXKmdO3fWnTt3OnxIXldXZ1/mtttu09tvv701T1WzPvvsM6cTK7Zu3apt27bV+vp6+7T3339f27dvr1artcXnTfX8iRfJycn2tzr37dunIuLwguGqD01B/sgjj2hJSYlOmzZNhw0bpomJifZw86RXrni6nid9jrQAMmP4qJq3XwENIIfBgDgAXwAY6W65cAyg5cuXKwCn26BBg1RVdffu3Tpu3Djt0KGDtm/fXgcPHuxwlpTNZtPFixdrQkKCdunSRe+991598MEHdfTo0aqq+uqrrzqc4GCz2fSWW27RYcOGaU1NjUMt69at06ioKP3yyy+dauzTp49T7Vu3blUADsu7q7elx6p6/sXz5ptv1qSkJE1ISNAhQ4bor371K/3vf//rtP1Ro0bpyy+/bH9ciYmJLsffunWrqqrW1dVpYmKifv755x48M5579dVXtV+/fg7T1qxZo5fur0888YT9eWnpeTt9+rT2799fZ82a5dDfWbNm6ahRo5rtQ5NVq1ZpUlKStm3bVufOnatnzpzR66+/XpOSkjzq1e9//3sFoP/617/sY3qynqrnfY6kADJr+Kias1+qQQggAFEACgFUA3i6peXDMYDMZMKECbp48eKAb8df/froo4+0b9++9lPDW/Lyyy+7PGXZzJpOQnDH2z544tFHH9WBAwdqQyteKT3tc6QEkJnDR9V8/WriLoD88pGZqloBDBGRjgDeE5FBqvrlxcuISBaALABISUlBdna2y7E6dOiAs2fP+qMsr1mtVsO27SubzYZTp05hw4YN2Lt3L9auXRvwx+Kvfo0ePRp33HEHSkpKcOWVV3q03aeeeiqknqvGxkYAcFuzt33wxAcffIBnnnnG4Y9iPeVpn8+dO9fsz7MvqqurAzJuaxQVJWLp0sGorY3GuHEVWLSoGDk5QTo7ykNm6pfHmkum1t4APAZgibtleATkf9u2bVMR0f79++vOnTuDss1Q7leweXIEFKrC/QjI7Ec+TczSr0shkEdAItIFQIOqVopILIAfAHja13HJOxkZGbDZbEaXQRRWeKp1YPmjlZcD+IOIROH8teU2quqHfhiXiMgwDJ/A87mdqroXgG+XZyYiMhGGT3DwathERBdh+AQPA4iI6AKGT3CZMoD4YTpRaAinn1WGT/CZLoDi4+Nx4sQJ1NfXN53WTUQmo6qor6/HiRMnEB8fb3Q5PmP4GMN0Le7evTtOnTqFr776yv7He8Fy7tw5tGvXLqjbDGXsl+dOnjwJVYXFYrrf+VotOjoaHTp0QFJSktGl+IThYxzTtdlisSA5ORnJyclB33Z2djaGDuUJfZ5ivzz3s5/9DJWVlSgsLDS6FLoIw8dY4fPrGBGRFxg+xmMAEVHEYfiYAwOIiCIKw8c8GEBEFDEYPubCACKiiMDwMR8GEBGFPYaPOTGAiCisMXzMiwFERGGL4WNuDCAiCksMH/NjABFR2GH4hAYGEBGFFYZP6GAAEVHYYPiEFgYQEYUFhk/oYQARUchj+IQmBhARhTSGT+hiABFRyGL4hDYGEBGFJIZP6GMAEVHIYfiEBwYQEYUUhk/4YAARUchg+IQXBhARhQSGT/hhABGR6TF8whMDiIhMjeETvnwOIBG5QkS2iUixiOwXkfv9URgREcMnvPnjCKgRwIOqOgDAKAB3i8hAP4xLRBGsqCiR4RPmfA4gVS1T1S8u/P8sgGIA3Xwdl4gi1z//CSxdOpjhE+b8+hmQiPQEMBRArj/HJaLI0fS2W21tNMMnzPntaRWR9gD+BOABVa1yMT8LQBYApKSkIDs721+b9pvq6mpT1mVW7JfnKisrYbVa2a8WFBUlYunSwaitjcaYMSewaNEh5OSo0WWFhFD8eRRV359cEYkB8CGAv6nq8y0tn56ernl5eT5v19+ys7ORkZFhdBkhg/3yXEZGBiorK1FYWGh0KaZ16QkHixZ9iszM7xtdVsgw68+jiOSrarqref44C04A/A5AsSfhQ0R0KVdnu0VF8cgn3PnjM6DRAOYDGC8ihRduk/0wLhFFAJ5qHbl8fppVNQeA+KEWIoowDJ/IxishEJEhGD7EACKioGP4EMAAIqIgY/hQEwYQEQUNw4cuxgAioqBg+NClGEBEFHAMH3KFAUREAcXwoeYwgIgoYBg+5A4DiIgCguFDLWEAEZHfMXzIEwwgIvIrhg95igFERH7D8CFvMICIyC8YPuQtBhAR+YzhQ63BACIinzB8qLUYQETUagwf8gUDiIhaheFDvmIAEZHXGD7kDwwgIvIKw4f8hQFERB5j+JA/MYCIyCMMH/I3BhARtYjhQ4HAACIitxg+FCgMICJqFsOHAokBREQuMXwo0BhAROSE4UPBwAAiIgcMHwoWBhAR2TF8KJgYQEQEgOFDweeXABKR10WkQkS+9Md4RBRcDB8ygr+OgNYBmOSnsYgoiBg+ZBS/BJCqbgdwxh9jEVHwMHzISPwMiChCMXzIaEHb3UQkC0AWAKSkpCA7OztYm/ZYdXW1KesyK/bLc5WVlbBarabpV1FRIpYuHYza2miMG1eBRYuKkZOjRpflgPuXd0KxX6Lqn51ORHoC+FBVB7W0bHp6uubl5fllu/6UnZ2NjIwMo8sIGeyX5zIyMlBZWYnCwkKjSwmZIx/uX94xa79EJF9V013N41twRBEkVMKHIoO/TsN+C8DnAPqJyHERucMf4xKR/zB8yGz8svup6q3+GIeIAoPhQ2bEt+CIwhzDh8yKAUQUxhg+ZGYMIKIwxfAhs2MAGUhE8M477xhdBoUhhg+FAgaQGwsXLsTUqVONLoPIKwwfChUMIKIwwvChUMIAaqWioiJMmTIFCQkJSE5Oxq233oqTJ0/a5+/evRsTJ05EUlISEhMTccMNN+Dzzz93O+bTTz+NpKQk5ObmBrp8CkMMHwo1DKBWKCsrw9ixYzFo0CDs2rULW7duRXV1NaZPnw6bzQYAOHv2LObPn48dO3Zg165dGDJkCCZPnoxTp045jaeqWLJkCV566SV8+umnGDlyZLAfEoU4hg+FIu6irfDqq6/i2muvxdNPP22f9sYbb+Cyyy5DXl4eRowYgfHjxzus89JLL+FPf/oTPv74Y/z4xz+2T7darbj99tuxc+dO5OTkoGfPnsF6GBQmGD4UqribtkJ+fj62b9+O9u3bO807fPgwRowYgYqKCjzyyCPYtm0bysvLYbVaUVdXh2PHjjksv2TJEkRHRyM3NxfJycnBeggUJhg+FMq4q7aCzWbDlClT8OyzzzrNS0lJAQDcdtttKC8vxwsvvICePXuibdu2yMzMRH19vcPyEyZMwFtvvYXNmzdj4cKFwSifwgTDh0Idd9dWGDZsGDZu3IgePXogJibG5TI5OTl48cUXMWXKFABAeXk5ysrKnJabPHkybrrpJtxyyy0QEdx2220BrZ3CA8OHwgFPQmhBVVUVCgsLHW5TpkzBt99+i9mzZyM3NxdHjhzB1q1bkZWVhbNnzwIA+vbti/Xr16OoqAi7d+/GnDlz0KZNG5fbmDp1KjZt2oS77roLb7zxRjAfHoUghg+FC+62LdixYweGDh3qMO3mm2/Gzp07sWLFCkyaNAnnzp3DlVdeiYkTJ6Jt27YAgNdffx1ZWVkYPnw4UlNT8ctf/hL/+c9/mt3O1KlTsXHjRsyaNQsAsGDBgsA9KApZDB8KJ9x13Vi3bh3WrVvX7Hx3l9G59tprnf6eZ/78+Q73L/022mnTpqGurs77QikiMHwo3PAtOKIQwPChcMQAIjI5hg+FKwYQkYkxfCicRWQANTY24uDBg0aXQeQWw4fCXcQFUFFREQYPHoyrr74axcXFRpdD5BLDhyJBxASQ1WrFk08+ifT0dJSUlKCxsREzZ85EY2Oj0aUROWD4UKSIiAAqLS3FsGHDsHLlStTV1UFVoao4evQoVq1aZXR5RHYMH4okYR1ANpsNzz33HIYMGYJ9+/ahtrbWYX5tbS2efPJJNDQ0GFQh0XcYPhRpwnb3PnLkCGbPno3i4mKXf9wZGxuLzp07Y9OmTc1ez40oWBg+FInC7ghIVfHKK6/gmmuuwRdffIGamhqnZWJjY/GTn/wEpaWlGDVqlAFVEn2H4UORKqx283//+9+YM2cO9uzZ4/R2GwC0a9cOHTt2xMaNGzFmzBgDKiRyxPChSBYWR0CqirVr12LAgAHIzc1t9qhn7ty5OHjwIMOHTIHhQ5Eu5Hf3r7/+GvPmzcPu3btdBk/btm2RmJiIt956C5mZmQZUSOSM4UMUwkdAqoo333wT/fr1Q05OjsvwiYuLw8yZM3Ho0CGGD5kGw4foPL8EkIhMEpEDInJIRJb7Y0yr1YrFixejvLzcaV5FRQV+9KMf4a677kJ1dbXTH5O2bdsWnTp1wqZNm7B+/XokJib6oyQinzF8iL7jcwCJSBSAVwD8CMBAALeKyEBfx127di1ee+01zJ8/3+F7czZt2oQ+ffrgH//4h8sTDeLi4jBt2jQcPnwYkydP9rUMIr+prY1m+BBdxB9HQCMAHFLVI6paD+B/AczwZcCqqiosW7YMNpsNO3fuxBtvvIHTp09jxowZWLhwIaqqqpz+eLRNmzbo0KEDNmzYgE2bNqFTp06+lEDkF6pARQVw5Ahw6FA8w4foInLpt3J6PYDITACTVPWnF+7PBzBSVe9pbp2EhAQdPnx4s2MePnwYX3/9NWw2GwDAYrHAYrHAarU6fYto0/yOHTuif//+Pv1RaWVlJTp27Njq9SNNpPersRGorwf++1/Hfy+ddn6XLQQAdO06BH37AiJGVh4aIn3/8pZZ+/Xpp5/mq2q6q3n++B3M1Y+SU0qISBaALACIiYlBZWWly8Hq6+tx4sQJh6Cx2Wz2MLqUxWJB9+7d0alTJ5cnInjDarU2Wxc5C9d+Wa2ChgYLGhoEjY3n/z1/33GaqmcpIqIQUbRpY0VKSiW+/TbADyBMhOv+FSih2C9/BNBxAFdcdL87gK8vXUhVfwPgNwCQnp6ueXl5LgebMWMGSktLYbVa3W40Pj4e119/Pd58802kpKS0tnYH2dnZyMjI8MtYkSCU+qUKVFYCZWXA119/9+/F/2/699w5z8aMjwdSU7+7XX65479N/09IADIyMlBZWYnCwsJAPsywEkr7lxmYtV/i5nDfHwG0G0AfEekF4ASAOQDmtmag3NxcbNmyxW34REdHo127dnjllVcwf/58tw+Owp/RwUJEredzAKlqo4jcA+BvAKIAvK6q+1sxDhYtWuTywqEXa9OmDYqLi9G9e/fWFUwhgcFCFP78ch6Oqm4GsNmXMd5++20cOXKkxeVEBO+99x7uvfdeXzZHBmGwEFETU5wIeu7cOdx3330enURQU1OD5cuXY/LkyUhLSwtCdeQJBgsRecsUAfTss896fAZbTEwMGhoa8POf/xzvv/9+gCsjVeDbb12HSWHhQDQ2MliIqHUMD6Dy8nKsWrUK5y68eokI4uLiEB0djfr6ejQ0NKBLly7o0aMH+vfvjwEDBiAtLQ3p6S5PKycPuQuWS6c1HyzJDvcYLETkDcMDKDs7G7GxsUhLS0OfPn1w9dVXo3fv3rjqqqvQq1cvdOvWDRZLyF4zNej8EyyOmguWb74pwg9+MJDBQkStYngAzZ49G7Nnzza6DNMLZrB4esSSnV2BjAyfL/tHRBHK8ACKdGYMFiKiYGAABQiDhYjIPQaQlxgsRET+wQC6QBWoro5GURGDhYgoGMI+gLw7YrnBozEZLEREvgvZAArEW2Ht2llxxRVRDBYioiAwXQAZ+RlLfv4OU17OnIgoHBkSQGfPAm++yc9YiIgimSEBVFoKLFjQ/HwGCxFR+DMkgNq3B6ZNY7AQEUUyQwKoXz/gj380YstERGQWvMonEREZggFERESGYAAREZEhGEBERGQIBhARERmCAURERIZgABERkSEYQEREZAgGEBERGYIBREREhmAAERGRIRhARERkCAYQEREZwqcAEpFbRGS/iNhEJN1fRRERUfjz9QjoSwA3Adjuh1qIiCiC+PR9QKpaDAAi4p9qiIgoYvAzICIiMkSLR0AishVAVxez/kdV/+LphkQkC0DWhbvVInLA03WDKAnAKaOLCCHsl3eSRIT98hz3L++YtV89mpshqurz6CKSDWCJqub5PJiBRCRPVXkyhYfYL++wX95hv7wTiv3iW3BERGQIX0/D/j8ichzA9wD8VUT+5p+yiIgo3Pl6Ftx7AN7zUy1m8BujCwgx7Jd32C/vsF/eCbl++eUzICIiIm/xMyAiIjIEA6gZIrJERFREkoyuxcxEZI2IlIjIXhF5T0Q6Gl2T2YjIJBE5ICKHRGS50fWYnYhcISLbRKT4wqW+7je6JrMTkSgRKRCRD42uxRsMIBdE5AoAEwAcM7qWELAFwCBVHQygFMAKg+sxFRGJAvAKgB8BGAjgVhEZaGxVptcI4EFVHQBgFIC72bMW3Q+g2OgivMUAcu0FAA8B4AdkLVDVT1S18cLdfwLobmQ9JjQCwCFVPaKq9QD+F8AMg2syNVUtU9UvLvz/LM6/sHYztirzEpHuAKYAWGt0Ld5iAF1CRKYDOKGqe4yuJQTdDuAjo4swmW4A/n3R/ePgi6nHRKQngKEAcg0uxcz+H87/wmwzuA6v+XQadqhyd3khAL8AMDG4FZmbJ5djEpH/wfm3TjYEs7YQ4OpKvTyy9oCItAfwJwAPqGqV0fWYkYhMBVChqvkikmFwOV6LyABS1R+4mi4i1wDoBWDPhSt8dwfwhYiMUNWTQSzRVJrrVxMRuQ3AVACZyvP6L3UcwBUX3e8O4GuDagkZIhKD8+GzQVXfNboeExsNYLqITAbQDkCiiKxX1R8bXJdH+HdAbojIUQDpqmrGC/yZgohMAvA8gO+r6n+MrsdsRCQa50/OyARwAsBuAHNVdb+hhZmYnP/t7w8AzqjqAwaXEzIuHAEtUdWpBpfiMX4GRL56GUACgC0iUigivza6IDO5cILGPQD+hvMfpm9k+LRoNID5AMZf2KcKL/yGT2GGR0BERGQIHgEREZEhGEBERGQIBhARERmCAURERIZgABERkSEYQEREZAgGEBERGYIBREREhmAAEQWQiDx04YsNL709bnRtREbjlRCIAkhEEgDEXzRpCYB5AMao6iFjqiIyBwYQUZCIyDIA9wEYr6oHjK6HyGgR+XUMRMEmIitw/qKk41S11Oh6iMyAAUQUYBe+rO8unP/KCr7tRnQBA4gogETkEQCLAGSo6mGj6yEyEwYQUYBcOPK5H8B0ADUi0vS15pWqes64yojMgSchEAXAhW/1rASQ6GL2D1T178GtiMh8GEBERGQI/iEqEREZggFERESGYAAREZEhGEBERGQIBhARERmCAURERIZgABERkSEYQEREZAgGEBERGeL/AzODoCvOmQ/oAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# extra code – this cell generates and saves Figure 11–2\n",
"\n",
"def leaky_relu(z, alpha):\n",
" return np.maximum(alpha * z, z)\n",
"\n",
"z = np.linspace(-5, 5, 200)\n",
"plt.plot(z, leaky_relu(z, 0.1), \"b-\", linewidth=2, label=r\"$LeakyReLU(z) = max(\\alpha z, z)$\")\n",
"plt.plot([-5, 5], [0, 0], 'k-')\n",
"plt.plot([0, 0], [-1, 3.7], 'k-')\n",
"plt.grid(True)\n",
"props = dict(facecolor='black', shrink=0.1)\n",
"plt.annotate('Leak', xytext=(-3.5, 0.5), xy=(-5, -0.3), arrowprops=props,\n",
" fontsize=14, ha=\"center\")\n",
"plt.xlabel(\"$z$\")\n",
"plt.axis([-5, 5, -1, 3.7])\n",
"plt.gca().set_aspect(\"equal\")\n",
"plt.legend()\n",
"\n",
"save_fig(\"leaky_relu_plot\")\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"leaky_relu = tf.keras.layers.LeakyReLU(alpha=0.2) # defaults to alpha=0.3\n",
"dense = tf.keras.layers.Dense(50, activation=leaky_relu,\n",
" kernel_initializer=\"he_normal\")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2021-12-16 11:22:41.636848: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA\n",
"To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
]
}
],
"source": [
"model = tf.keras.models.Sequential([\n",
" # [...] # more layers\n",
" tf.keras.layers.Dense(50, kernel_initializer=\"he_normal\"), # no activation\n",
" tf.keras.layers.LeakyReLU(alpha=0.2), # activation as a separate layer\n",
" # [...] # more layers\n",
"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ELU"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Implementing ELU in TensorFlow is trivial, just specify the activation function when building each layer, and use He initialization:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"dense = tf.keras.layers.Dense(50, activation=\"elu\",\n",
" kernel_initializer=\"he_normal\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"### SELU"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By default, the SELU hyperparameters (`scale` and `alpha`) are tuned in such a way that the mean output of each neuron remains close to 0, and the standard deviation remains close to 1 (assuming the inputs are standardized with mean 0 and standard deviation 1 too, and other constraints are respected, as explained in the book). Using this activation function, even a 1,000 layer deep neural network preserves roughly mean 0 and standard deviation 1 across all layers, avoiding the exploding/vanishing gradients problem:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAD+CAYAAAB8xdFqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA5cElEQVR4nO3deXgUVfbw8e/NDmQjBEKIbIogIMgmIo4QDJuIuKCijAqjA4KM44bj6wY6DKg4MuMAIowO4k9Fx3EQVEBhIICKQoCIrBJWCSAQCCQkIUuf949KmjTphIR0Ut3J+TxPPemuulV96nanT1fVrXuNiKCUUkpVNz+7A1BKKVU7aQJSSillC01ASimlbKEJSCmllC00ASmllLKFJiCllFK2CLDjRaOjo6VFixZ2vHSZzpw5Q7169ewOw2dofZXfzp07KSgooF27dnaH4jO88vPlcICfX4lZKSmQkQEBAdCmDYSEVH9oXllfwIYNG46LSEN3y2xJQC1atCApKcmOly5TYmIi8fHxdofhM7S+yi8+Pp709HSv/Nx7K6/7fE2YAAsXwpIlEBMDQHY2DBkCmzZZs1asALt+Y3hdfRUyxuwvbZmeglNKqQuZOhUmTYKtW2HDBuBc8lm+3P7k46s0ASmlVFnefBOefhqMgXffhUGDNPl4SKUTkDEmxBizzhjzozFmqzHmJU8EppRStps3D8aNsx7PmgX33qvJx4M8cQ3oLHCDiGQaYwKBb4wxS0Tkew9sWyml7PHJJ/DAA9bjv/4VHnpIk4+HVToBidWbaWbh08DCSXs4VUr5trVrrSZuL74ITz6pyacKGE/0hm2M8Qc2AK2AmSLytJsyo4HRADExMV0/+uijUrcXFBRESEgIxphKx1YRIlLtr+nLtL7K7+TJk4gIUVFRdofiM6rq8yUi5OTkkJube6GCRK1bx4nu3Tmb689zz13Jhg1R1K+fy7RpybRokeXx2CojMzOT0NBQu8MooU+fPhtEpJu7ZR5JQM6NGRMJLAAeEZEtpZXr1q2blNYc9cCBAxhjiImJITAwsFq/4DIyMggLC6u21/N1Wl/lt3PnTvLz82nfvr3dofiMqvh8iQh5eXn8+uuviAjNmjVzLbBxIzRtCg3P3bbiK0c+XtwMu9QE5NFWcCKSDiQCAy92G2fOnCEuLo6goCD9da2U8ihjDEFBQcTFxXHmzBnXhcnJkJAAvXvDsWOA7yQfX+WJVnANC498MMbUAfoCOyoVlJ+2DldKVZ0S3zHbt0P//pCeDm3bQv36mnyqgSdawcUC8wqvA/kB/xaRLzywXaWUqnp79kDfvtZRz8CB8OGHZOcFaPKpBp5oBbcZ6OyBWJRSqnodPGiddjt0yDr19umnZDuCNflUEz3XpZSqnRwO68hn3z7o3h0+/5xsU1eTTzXSBFQDnTx5kpiYGHbv3l2u8nfccQfTpk2r4qjKVtGYi/Tu3ZtOnTrRqVMnAgICWLt2bRVFWFJ5623kyJE89NBDzucOh4OHHnqIBg0aYIwhMTGxCqP0DiNHjmTw4MF2h+HKzw8GD4aOHWHJErIDwjT5VDcRqfapa9euUppt27aVuqyqnT59ulLrjxgxQrBuwnWZrrnmGufym266qdT1e/fuLePGjSsxf+7cuVKvXr1yxzF+/HgZOXJkuctv3rxZ6tevL+np6eVeR6Ty9VVcRWM+30svvSQPP/ywx+IREVm1apXcfPPN0qRJEwFk7ty5LsvLW2/p6emybt062bJli4iIfP755xIYGCjffvutHD58WM6ePevRuCti5syZ0qJFCwkODpYuXbrI6tWrq+R1LvTZd8eTny93tm3bJuJwiGRkSFaWSN++IiASEyOydWuVvnSVWLlypd0huAUkSSm5QI+APKxv374cPnzYZVq8eHG1vX5WVhZvv/02Dz74YLnX6dChA5deeinvv/9+FUZWuouJubjp06eza9cuZsyY4dG4MjMzufLKK3njjTeoU6dOieXlrbeIiAjCw8Odz1NSUoiNjaVnz540btyYoKAgj8Z9vtOnT5Oenl5i/scff8yjjz7Ks88+y6ZNm+jZsyc33ngjBw4cqNJ4bFVQAPv3Q16e9dwYsv1D9cjHJpqAPCw4OJjGjRu7TNV59/vixYvx8/Pjuuuuc5k/depUjDElpgkTJgAwZMgQ5s+fXyUxbdiwgYSEBOrUqUOrVq1YvXo1//73v50xlhYzQGpqKvfffz8NGjQgMjKSoUOH8uuvvzqXv/fee6xYsYK5c+d6/L6xQYMGMWXKFO64445Sbw0oT70VPwU3cuRIHn/8cecN12UNzHih96wsBQUFfPXVVwwfPpzGjRvz448/ligzbdo0Ro4cyahRo2jbti3Tp08nNjaWWbNmXXD7xYkIU6dO5bLLLqNOnTp06NChzKS8evVqevToQWhoKBEREVxzzTVs2bKlxPY6duxYru1BOevK4bBavB07Zv1F7/OxmyagGmbNmjV07dq1xJfx2LFjXY7KnnzySRo3bsz9998PQPfu3Vm3bh3Z2dkltjllyhRCQ0NLTLGxsc7Ha9ascRvP+vXruf766+nTpw+bN2+mR48eTJw4kcmTJzNp0qQyY967dy9dunQhLi6Ob775hsTERI4fP86YMWMAWLBgAR988AEfffQRAQG2jK1YZr2588YbbzBhwgQuueQSDh8+zPr160ste6H3zJ2tW7fypz/9iWbNmjFs2DDq1avH0qVL6dWrl0u53NxcNmzYQP/+/V3m9+/fn++++65c+1Lk+eef55133mHmzJls27aNZ555hoceeogvv/yyRNn8/HxuueUWfvOb3/Djjz/yww8/8Oijj+Lv719ie6+//voFt1fkgnUlAnv3wqlT1rClzZohosnHbvb819ZgS5cuLdEf07hx43j11Ver5fX3799PbGxsiflhYWHObk1effVV5s+fT2JiIq1atQKgSZMm5OXlcejQIS677DKXdceMGcNdd91VYpvF+56Ki4tzG8+TTz7JzTffzPPPPw/A8OHDufnmm+nVqxc33HBDmTGPGTOGBx98kClTpjjnvfDCC9x+++0APPDAAzRs2JBrrrkGgIkTJ3LbbbeVUTueV1a9uRMREUFYWBj+/v40bty4zLIXes+KpKWl8cEHH/Dee++xefNmBg4cyN///neGDBlCcHCw220fP36cgoICYgpH9iwSExPD8uXLL7gfRc6cOcO0adP4+uuvuf766wFo2bIl69atY+bMmdx0000u5YtOB958883O+rriiivcbq9Tp06EhYWVub1y1ZWI1dLt5Enw94fWrXEE1+HoUU0+dvOJBFR9PfK49jt1Md3k9erVizlz5rjMi4yMrERMFZOdnV3iS6W4l19+mRkzZrBy5Upat27tnF90jcPdL/moqCi3pxEv1FfXkSNHWLNmDStXrnTOCwoKwuFwOI9+Sov5wIEDfP3116xZs4Z//OMfzvkFBQXUrVsXsFrOXcjzzz/P5MmTyyyzcuXKi+5Dq6x685TS3rMi06dP56WXXqJnz57s2rWL5s2bl3vb5x91SgU7AN22bRs5OTkMHDjQZb28vDy3pxejoqIYOXIkAwYMICEhgYSEBO68806aNm16Uds7X4m6EoEDByAtzWr1dvnlOELqkpICOTmafOzmEwnIl9StW7fEL9TyCg8P59SpUyXmp6enExERUa5tREdHl/rFPHnyZN566y1WrVpVIsYTJ04A0LBYJ4xFpkyZ4nIU4s6SJUucv4CLbN++HYCrr77aOW/nzp20adOG3/zmN2XGnJycTHh4OBsKhz8uriIX7R977DHuvffeMsuU6JCyAsqqN08o6z0rMnr0aAIDA3nvvfdo3749t912G/fddx8JCQkup7aKi46Oxt/fnyNHjrjMP3r0aJk/YM7ncDgA+Pzzz0vUY2BgoNt15s6dy2OPPcbSpUtZtGgRzz33HJ999hkDBgxw2V5UVJTL2YTStlfEbV2dOmVd8zEGWrXCUTeUlBQ4fdrKR5p87OUTCciDHXaXye7endu0acPixYtL/ArduHEjbdq0cSm7adMmHnvsMVJTUxk/fjwrVqzg73//O507d+bdd98tse1Jkybxz3/+k8TERLenirZs2UKTJk3cfvlc7Cm49PR0jDHOC/gZGRlMnjy5xKkndzEHBgZy5swZGjduXKku5qOjo4mOjr7o9S+krHqrrAu9Z0WaNGnCc889x3PPPcf333/PvHnzuPvuuwkODmb48OHce++9dO7s2llJUFAQXbt2ZdmyZdx5553O+cuWLWPo0KHljrFdu3YEBwezf/9+5ynV8rjqqqu46qqrePrpp7nxxhuZN28eAwYMcNne1VdfXe7/x1LrKiICYmOhXj0coeHO5BMYCI0ba/Kxm08kIF9y9uzZEr8q/f39nb+QT58+TXJyssvyyMhIWrRowdixY5kxYwaPPPIIo0aNIiQkhMWLFzN//nwWLlzoLJ+VlcWwYcP44osviIqKonPnzgwePJgmTZowYMAAnn76adLS0mjQoAFg/TJ84403WLRoEfXq1XPGFxkZSUhICGA1BBg40H0n5hd7Cq5Tp06ICC+//DK//e1veeqpp4iNjSUlJYVdu3Zx+eWXA7iNuUePHtSvX5/77ruPCRMmEB4ezp49e/jss8+YPn16tXRYm5mZSUpKCmD90j9w4ADJyclERUU5f+2XVW+VUZ73zJ0ePXrQo0cP/v73v/P5558zb948unfvzooVK0ocoT7xxBPcd999dO/eneuuu4633nqLQ4cOORt5lEdYWBjjx49n/PjxiAi9evUiMzOT77//Hj8/P0aPHu1Sfu/evcyePZshQ4YQFxfHnj172Lx5M2PHji2xvezsbPr371/m9kqtK4eDyKgoq67i4nA4cEk+rVtbl4WUzUq7Qagqp9p2I2pcXFyZy4cOHercxrp166R///7SqFEjCQ8Pl+7du8uCBQtcXuezzz6TBx54wPk8ODhYUlJSnM979OghM2bMEBERh8Mh4eHhbl93+fLlIiKSnZ0t4eHhsnbt2grtb3nqa/LkyRIdHS3BwcEyfPhwOXHihPTs2VOio6NdyhWPucj69eulT58+EhERIaGhodKxY0eZMmVKhWKsjJUrV7qttxEjRohI+ettxIgR0rt3b+eNqK+99po0b9681PLlec8q4tdff5Xjx4+7XTZz5kxp3ry5BAUFSZcuXWTVqlUuy+fOnSuA7N27t8x4//GPf0jbtm0lKChIoqOjpW/fvvL11187yxTdiHrkyBG57bbbpEmTJhIUFCRNmzaVp556SnJzc0tsr02bNqVur3jZUutqyRIRESkoENm5U2T9epHkZJGsLGtdO79rqoIv3oiqCaiYqr7z2lNeeukl55f1jh07pHv37i7LlyxZIq1bt5b8/PxybW/GjBnSr1+/CsfhyfqqaMzeoCL1tmPHDmcC8iUTJkyQdu3aSV5eXrW/9kV9vo4ftzLN+vUix46VmnxENAFVl7ISkN4H5IMCAwNJTU1FRHj22WfJz893WT5w4EDGjRvHwYMHy7296dOnV0Wo5VbRmL2BN9RbVVu8eDEzZsyw7T6rCjl50rrXB6zTblHRJU67uenQQtlIE5APuvfee1m6dClXXHEFgwcPpn79+iU6xfzjH/9Y7ua4o0ePLtHIwQ4VidkbeEu9VaX169fTp08fu8O4sFOnnL0bEBuLIyZWk48P8IGfNep8TZs2ZePGjc7nv/vd72yMRimbZWRYLQxEICYGR+Mmmnx8hB4BKaV8W0aGlXyio3E0uYSU3UaTj4/QIyCllG+LjYU6dXCER2ry8TF6BKSU8j05OZCbaz02BkdEfU0+PkiPgJRSvuXsWfj5Z6t7nTZtcAQE6TUfH6VHQEop35GbayWf3FwIDMRh/DX5+DBNQEop35CXZyWfs2ehbl0cl7UiZa+/Jh8fpglIKeX98vNh1y7r2k+dOjhatSZlb4AmHx+nCUgp5d0cDiv5ZGVBcLCVfPZp8qkJtBGCUsq7GQPh4ZCXh+Py1qTsD9TkU0NoAlJKeTdjrL7dGsbokU8No6fglFLeR4Sgo0ed9/o4HGjyqYE0AdVgJ0+eJCYmht27d5er/B133FGiU1PlfWr8+yoC+/YRfPIk7NqFo0C0qXUNpQnIg44dO8bDDz9MixYtCA4OJiYmhoSEBJYtWwbAyJEjMcaUmHr06OHcxsiRIxk8eHCprxEfH88f/vCHEvPffffdEkNXT5kyhUGDBpU5nHNxEydO5C9/+QunTp0qV/mKWL16tXMUTGOM22HD3XnzzTdp2bIlISEhdO3alTVr1rgsf/HFF0vU5/lDfhep7vqvKt70vnqcCPzyC6SlIcbgaNpceziowSqdgIwxTY0xK40x240xW40xj3oiMF80dOhQ1q1bxzvvvMPPP//MF198wY033khaWpqzTN++fTl8+LDLtHjxYo/HkpWVxdtvv82DDz5Y7nU6dOjApZdeyvvvv+/xeDIzM7nyyit54403qFPOb5CPP/6YRx99lGeffZZNmzbRs2dPbrzxRg4cOOBSrk2bNi71+dNPP5W6zeqq/6ribe+rR4lAaiocPQrGkNUkjpQjoZp8ajBPHAHlA0+KSFugBzDOGNPOA9v1Kenp6axZs4ZXXnmFhIQEmjdvztVXX8348eO5++67neWCg4Np3LixyxQVFeXxeBYvXoyfnx/XXXedy/ypU6e6PQqYMGECAEOGDGH+/Pkej2fQoEFMmTKFO+64Az+/8n3spk2bxsiRIxk1ahRt27Zl+vTpxMbGMmvWLJdyAQEBLvXZsGHDUrdZXfVfVbztffWoI0esyRgcLS9j/8loTT41XKUTkIgcFpGNhY8zgO1AXGW362tCQ0MJDQ1l0aJF5OTk2B0Oa9asoWvXrhhjXOaPHTvW5df/k08+SePGjbn//vsB6N69O+vWrSM7O7vENqdMmeLcz9DQUGJjY12eh4aGljhFdrFyc3PZsGED/fv3d5nfv39/vvvuO5d5e/bsIS4ujpYtW3L33Xezp2hgshqoKt5Xr5CZaR39AI7mLUk5HklWVoAmnxrOo82wjTEtgM7AD26WjQZGA8TExJCYmOh2GxEREWRkZJSYHxYeXurr5rzxBnmFg7IFzp1LyKOlnwXMOH3a+bhur174Jyefe41SypXXrFmzeOSRR5gzZw4dO3akR48e3HrrrVx99dUA5OXlsXTp0hLXCkaNGsWf//xnZ5n8/Hy3dQBQUFBAbm5uieVFSa9o/u7du4mOjna7nXr16gHwt7/9jQ8//JAvv/ySmJgYMjIyiIiIIC8vj59//plLL73UZb3f/va3DBo0yPnc4XCUOJpp0qRJqbGfH29Z5Q4fPkxBQQFhYWEu5SIjIzl06JBzXocOHZg1axatW7fm2LFjvPbaa1x77bX88MMPNGjQwGWb1Vn/w4YNY+3atfTu3Zv/+7//cw6bfv56+/fvZ9y4cfz6668YY1iyZAlHjhxh9OjRzjK7du3iX//6F4MHD66S99UriBAUHY3DP4C9R8PJygJ/fwdxcdnk5zsox0eqwnJyckr9HvJFmZmZPrc/HktAxphQ4FPgMREp8e0tInOAOQDdunWT+Ph4t9vZvn07YWFhbpeVJiQkhJCidUJCyizrsu0yTgVVNAawhsq+4447WLNmDWvXrmXp0qVMnz6dyZMn8+yzzxIYGEivXr2YM2eOy3qRkZHO1wsMDCQgIKDU1/f39ycoKKjE8pDC/S6an5eXR1hYWKnbefnll5k9ezaJiYm0bt3aOT86Otr5OuevGxYW5jJkdkZGxkXVU1G8Za1b9AVbr149l3JBQUEusQ0dOtRlvRtuuIFLL72U//73vzzxxBMuy6qz/p966ikyMzOZN28eYWFhBAQEkJ+fX2K9cePGMWnSJHr16sWpU6cICQmhRYsWbN682VkPLVu2ZMiQIdSrV69K3ldbORzO/0NHaDgpKVaHB4GBEBeXTXR0vSp76ZCQEDp37lxl269uiYmJlPa96q080grOGBOIlXw+EJH/emKbJYiUPhX7tcjo0WWXLW7DBpdlGadPuy9XASEhIfTr148JEybw3Xff8eCDD/Liiy+SW3g/Q926dWnVqpXLVPTlUB7h4eFuWzOlp6cTERHhfB4dHc3JkyfdbmPy5Mm8+eabrFq1yuVLCuDEiRMAbq+jVOcpuOjoaPz9/Tly5IjL/KNHjxITE1PqeqGhobRv355du3a5XV5d9d+nT58LftFv3brVmRTBOvoPDg52KbNw4UL69u3rPLqpivfVNunpsG0bnD1r3edzXlPr4GCH3RGqKuaJVnAGeAfYLiI+dLNB9WjXrh35+fkeuy7Upk0bNm7ciJyXJDdu3EibNm2czzt37sy2bdtKrD9p0iTnL+RWrVqVWL5lyxaaNGni9kt+zJgxJCcnO6dvvvnG5XlycjLdunXzwF5aRzpdu3Z1NmEvsmzZMnr27Fnqejk5OezYsYPY2FiPxHG+8tZ/eezatYuwsDBuueUWOnfu7GwwUNzHH3/MsGHDnM+r4n21xenTsHs35OTgSDuh9/nUUp44BXcdcB/wkzEmuXDesyLiO21bPSAtLY0777yTBx54gI4dOxIWFkZSUhJTp04lISGB8MJrWGfPni3xq97f39/ll+np06dJLnZtCqzTRC1atGDs2LHMmDGDRx55hFGjRhESEsLixYuZP38+CxcudJYfMGAATz/9NGlpac5rIZMnT+aNN95g0aJF1KtXzxlHZGSk8xTSmjVrGDhwoNt9jIqKcmkxVpFTcJmZmaSkpADWtaMDBw6QnJxMVFQUzZo1A2DGjBnMmDGDHTt2APDEE09w33330b17d6677jreeustDh06xJgxY5zbHT9+PDfffDPNmjXj6NGjTJo0iTNnzjBixAi3cVRX/ZdHfn4+iYmJJCcn06RJE2699VYWLFjAbbfdBlhHVd9//z2ffPKJc52qeF+rXUaGdbgjgjRsREpGY05naPKplUSk2qeuXbtKabZt21bqsqp2+vTpi143JydHnnnmGenWrZtERkZKnTp1pFWrVvL4449LWlqaiIiMGDFCgBJTXFycczullRk6dKizzLp166R///7SqFEjCQ8Pl+7du8uCBQtKxNSjRw+ZMWOGiIg4HA4JDw93u+3ly5eLiEh2draEh4fL2rVry7XPFamvlStXun3tESNGOMtMnDhRrI/kOTNnzpTmzZtLUFCQdOnSRVatWuWyfNiwYRIbGyuBgYHSpEkTuf3222Xr1q1uY6ju+l+5cqVzvR07dsiWLVtclq9du1YSEhKcz998802ZMGGC8/k777wj9913X4ntVvX7WqUyM0U2bhRZv14ce/bKzh0OWb9eJDlZJCvLtWhl/h/Lw87vmqqwcuVKu0NwC0iSUnKBJqBiqvoDX92WLFkirVu3lvz8/HKVnzFjhvTr16/c269p9eVpF0pA+fn50qlTJzl+/Lg4HA4ZNmyYfPTRR87l/fv3ly+//LLEdqv6fa0yWVkimzZZySclpczkI6IJqKJ8MQFpVzw12MCBAxk3bhwHDx4sV/nAwECmT59exVHVDn379uXOO+9k8eLFXHLJJWzatAmATp06Ocv4+/vz6quv0qdPHzp27Eh0dDR33XUXAMePHyc5OZl+/fqV2LbPvq+nTkF+PhIRwa78lpzOMHrarZYzUokWXxerW7dukpSU5HbZ9u3badu2bTVHZKlMs+LaSOur/Hbu3El+fj7t27e3OxRbOdJOsPt4JKcy/C6YfKr682Xnd01V8NZm2MaYDSLitnWSjgeklKo6eXnWbQ1BQVZT67QobXCgnDQBKaWqRn4+/PwzFBTguLwNKb8Ea1Nr5UKvASmlPK8o+WRnI35+7N3vp8lHlaAJSCnlWQUFFPWpI8HB7AlozcnMQE0+qgSvTEB2NIxQSnmAw2H1cJCZiQQGsTewNSczg7wu+eh3jHfwugQUGBjovV3GK6VKJwJ79sDp00hAAPuCWnMiM9jrkg9AdnY2gYGBdodR63ldAmrUqBGpqalkZWXprxSlfIkxUK8e4u/P/uDWpJ0J8brkIyJkZWWRmppKo0aN7A6n1vO6VnBFfaYdOnSIvLy8an3tnJwcZ99Z6sK0vsrvyJEj1p3f5RwN1leJwPH0QLLO7sfPDxo3hn37Lm5bVfX5CgwMJCYmxvldo+zjdQkIrCRkx4cjMTGxRo0PUtW0vspv7NixpKenl+jk1OeJwJQpMHw42Y1bMmQILF8OMTGwYgW0a3fxm9bPV81Xs3+OKaWq1gsvwPPP40joy+2Dcz2WfFTtoAlIKXVxXnkFJk9G/P15Kex1lq4I0uSjKkQTkFKq4mbMgGeeQYzhlSvm8efNt2ryURWmCUgpVTFz58IjjwDwtytm8+zW32ryURdFE5BSqvy2b4ff/x6AWZdP48ntozT5qIvmla3glFJeqm1bcl95nQ/fyuDhXY9r8lGVoglIKXVhBQXg7092Ngz5+jGW79HWbqry9BScUqps330HHTqQ89Muj97no5QmIKVU6TZuhEGDYPt2vrzpTU0+yqM0ASml3Nu2Dfr3h1OnWN1oKMN+eU2Tj/IoTUBKqZJ274a+fSEtjXUNbqTv0Q+JjgnQ5KM8ShOQUsrVL79AQgIcPkxy/Xh6p31KVEyQJh/lcZqAlFKuvvoK9u9ne/g1XH9yERExdTT5qCqhCUgp5SL7t79nypUf0vP0EurFhGnyUVVG7wNSSsHp03DiBNkxLaym1lvu0QYHqsppAlKqtsvKgptvRnal8IcWy1m+tq0mH1UtNAEpVZudPQu33w6rV3M8OI4Vh0M0+ahq45FrQMaYfxljjhpjtnhie0qpapCfD/fcA199xcnAhlx/djnZMS01+ahq46lGCO8CAz20LaVUVXM4YORIWLCAjIBI4vOWkR5zhSYfVa08koBEZDVwwhPbUkpVMREYOxY++IAs/1D65S/h15irNPmoaldt14CMMaOB0QAxMTEkJiZW10uXW2ZmplfG5a20vsovPT2dgoICr6mvxiaIFqYOgwo+5+f6XZj2yjqOHs3i6FG7IztHP18V44v1ZUTEMxsypgXwhYhceaGy3bp1k6SkJI+8riclJiYSHx9vdxg+Q+ur/OLj40lPTyc5OdnuUKwhFYbAtuWpFMTEee2Rj36+KsZb68sYs0FEurlbpjeiKlVbzJ5Nzo87nUMqeHPyUbWDNsNWqjaYNQsefpgzQY35IXcnMTHhmnyU7TzVDHs+sBZoY4w5aIx50BPbVUp5wHvvwcMPA/Bc7kTqavJRXsIjR0Aico8ntqOU8rBPP0V+9zsMMJ7X+CxmjCYf5TX0GpBSNdWSJcg992AcDl5kIu/HjNfko7yKJiClaqLUVGToUExeHn/lSd5qNFGTj/I62ghBqRooOyqOOc2nEbhjM39t9BorVhpNPsrraAJSqiYRITvHWE2td4zRjkWVV9NTcErVFDt24LiqM3+4YRvLl6PJR3k9TUBK1QR79yIJffH76UcSvv+LJh/lEzQBKeXrUlNx3JCAOZTKaq7nuUZva/JRPkETkFK+7OhRHAl98du3l3VczQMNv+DLlXU1+SifoAlIKV918iSOfgPw27mDzXTg/oZLWZQYrslH+QxNQEr5qNwvl+G3OZmdtGZ49DL+mxilyUf5FG2GrZQPys6GIfPuIpoCdkb/hn+vitHko3yOJiClfEluLjm7Uxnyx5aFTa3v0QYHymfpKTilfEV+PgXDhpN91TUcW56sTa2Vz9MEpJQvcDjIH/kg/p99isnLpWGUQ5OP8nmagJTydiLkj/kDAR+8Ryb1uLf+Yt5Y00WTj/J5moCU8mYi5D35NAH/nEUOwYyMXMjUb3pq8lE1giYgpbxY3sS/EPi318gjgFGR/+HP3yZo8lE1hiYgpbxUdjbM+CSGPAL4Q8T7PPPtYE0+qkbRZthKeaHsbAqHVBjN/Oh+vLuqpSYfVePoEZBSXubsB5/wxxu2OIdU0OSjaio9AlLKi5z9aAEB993DKxLB99Fb+XhFY00+qsbSIyClvMTZRV9hht+NvxTwXr2xfLxKk4+q2TQBKeUFzi5bjdx2G0GSy5y6jzLgh0mafFSNpwlIKZvlrF5H/o2DCXFk836d3/ObdX+jXXtjd1hKVTlNQErZKPtwOmf7DqJeQQb/DbmHLj+8pclH1RraCEEpm2Rnw5D7I2mS9zp3Bi/iiu/n0a6Dv91hKVVt9AhIKRtkZ4l1n89y+CpmBJdu+A/trgq0OyylqpUmIKWqWfbuQ+xpcp3rkAp62k3VQpqAlKpG2QeO8etV/Wh/ai3/CBqvQyqoWs0jCcgYM9AYs9MYk2KM+X+e2KZSNY3kFXDwygG0OLONHf7tiVnxkSYfVatVuhGCMcYfmAn0Aw4C640xi0RkW2W3rVRN4cgXAnfs53JHBnv8W+H3v2W0vi7a7rCUspUnWsF1B1JEZA+AMeYj4Bag1AS0c+dO4uPjPfDSnpWenk5kZKTdYfgMra/yKSiAjd99T6Dkci1BONo3oM7Ee+wOy+vp56tifLG+PJGA4oBfij0/CFxzfiFjzGhgNEBgYCDp6ekeeGnPKigo8Mq4vJXW14UVFBgOpfgRKLkAnGneAj+Tw9n0HJsj8376+aoYX6wvTyQgd813pMQMkTnAHIBu3bpJUlKSB17asxITE73yyMxbaX2V7eRJGDAATuRATEAH4qLS2bBvp91h+Qz9fFWMt9aXMaW38PREAjoINC32/BLgkAe2q5TPOnQIBg2CH3+Eli2hUaMG5OToTaZKFeeJVnDrgcuNMS2NMUHA3cAiD2xXKZ+0bRtcey3k/LiD25omsWoVhITYHZVS3qfSR0Aikm+M+QPwFeAP/EtEtlY6MqV80OrVcMstkJ4ubAgbQ+eDqzHffmh3WEp5JY/cByQii0WktYhcJiKTPbFNpXzN/PnQrx+kp8PMTm/TJWMVpkED6N/f7tCU8kraE4JSlZSfD089BcOHQ24uPDvyEGP3PmUt/Mc/ICrK3gCV8lLaG7ZSlZCWBsOGwf/+BwEBMO114Q8rx2FOnYKbboK777Y7RKW8liYgpS7Shg1wxx2wbx80agSffAK9jn4Kn30GYWEwaxaU0QRVqdpOE5BSFeRwwLRp8OyzkJcHV18Nn34KTeMccOUEq9Crr0LTpmVvSKlaThOQUhVw+DCMGAHLllnPH3kEpk4tambtZ42tMGsWPPSQnWEq5RM0ASlVTv/5D4wdC8ePQ3Q0zJ0LgwefV6hxY3jpJVviU8rXaCs4pS7gyBEYOhTuvNNKPn37wubNxZLPgQMwZYrVHE4pVW56BKRUKUTgvffg8cetft1CQ+G112D0aPAr+ulWUAD33WfdgZqZaSUipVS5aAJSyo1Nm6zrO99+az0fOBBmz4Zmzc4rOHWqlXxiYqxMpZQqNz0Fp1QxaWnWdZ5u3azk06gRzJsHixe7ST7r1sGEwlZv8+ZBw4bVHq9SvkyPgJQCsrJg+nR45RWrKx1/f3jsMXjxRYiIcLPCsWPWTUD5+fDoo9a4C0qpCtEEpGq1/HyrNduLL1pDKAAkJMAbb0D79mWsdM898MsvcM011j0/SqkK0wSkaqXcXKuBwSuvwO7d1rzOna3n/fpdoAODrCyrFUKjRlbb7ODgaolZqZpGE5CqVbKy4F//stoO/FI4kHyrVjBpEtx1V7HWbWUJD4clS2DvXrjkkiqNV6maTBOQqhUOHoSZM2HOHDhxwprXvr3Vnc5dd1kdiV7Qvn0QG2sd8fj7W5lLKXXRNAGpGksEVq2ymk9/8ol1yw5Yfbc984w1cFy5jnjA6oOnd2+49FJYsAAiI6sqbKVqDU1AqsZJTYV337UaFxRd3/H3t450HnsMevSoYCfVmZlWtwcHDkBcnF7zUcpDNAGpGiEry7pXZ+5cWLrU6rEarEs0I0davRdcVOfU+fnWgD8bN1qn3BYuhDp1PBm6UrWWJiDls4qSziefwJdfwpkz1vzAQKvvtgcesFq0+ftf5AuIWN0hLF4MDRpYDQ/0ZlOlPEYTkPIpJ07A11/Df/9rJZ2srHPLrr7aGhb73nut3qor7fnn4a23rFNuixZpowOlPEwTkPJqDoc18uiSJda0bt2502tg3Qd6551WpwTNm3vwhQsKYNcu6/Dpo4+gZ08PblwpBZqAlJdxOGD7dqv12qpVsHKl1etNkcBAiI+Hm26ykk6J/tk8xd8fPvzQyniafJSqEpqAlK1yciA5Gb7/3upUevVqq0PQ4po1gxtvtKYbboCwsCoMaMEC68JRaKh1c5AmH6WqjCYgVW3y8mDnTkhKsg4s1q2DH38sOY5bkybWLTdFU5s2FWw2fbH++ld46im47jpITCzn3alKqYul/2HK40Sse3E2b4affjo3bd9uJaHijIErr7QaEPzmN+fu9ayWhFM84BdegMmTref33KPJR6lqoP9l6qKdOePPhg3Wtfri044d1pAG7lx2GXTqZDUe6N4dunSp4lNqF+JwWMMpzJhhXfeZO9ca4VQpVeU0ASm3RKwkcuCANf3yy7m/+/dDSgr8+uv1pa4fFQUdO0KHDuf+tm9vXVrxGmfPwoMPwgcfQFAQfPwx3Hqr3VEpVWtoAqplHA7rXppff3U/paaeSzZFN3aWJjDQQevWflx+OS5T69ZWn53VehrtYsyZYyWfevWsHg4SEuyOSKlaRROQj3I4ICPDSiYnT577W/xx8XlpaXD0qDUVdcp5IaGhVgu0Zs2sbmyK/73sMkhJWc0NN8RX6X5WqYcfhi1brDG4O3WyOxqlap1KJSBjzJ3Ai0BboLuIJHkiqJpAxDrDk5MD2dkX/puZaU0ZGdbk7nHxeRc6OilL/foQE+N+io09l2giIso+itmz5+JjsM2KFdb5wIYNrWs+s2fbHZFStVZlj4C2ALcDFfovLiiA48etX/EFBdZUXY/z8s5Nubmuz3fvvpSFC90vK2s9d0klJ6eSNVsO9epZ11qioqyk4u5v8ceNGllTUFDVx+Z1HA74y1+ssbevvdZKRNqrtVL2EpFKT0Ai0K0C5cWapNg0uHDeomLzZhfOG1VsXmrhvNjz1u9SOD+p2LyJhfMmFpuXVDivy3nrxxbOTy02b1ThvNnF5i0qnDf4vPVL7pMx1j5FRi6S5s1FrrhC5JJLrH2KjR0lN94ocvvtIrfeau1TvXqxMnGiyF//KvLWWyItWlj79NZbSbJ+vcj27SKPP27t0wsvTJQiSUnWPnXp0kWKi4219ik1NdU5b9Qoa59mz57tnLdokbVPgwcPdlm/aJ+KGzzY2qdFixbJypUrRURk9mxrn0aNGuUsl5qaWrifsS7rd+li7VNSUpJz3sSJ1j5NnFhF+3TsmMiAASLO9wiRggK3+1TE0/vUu3dvufzyy215n6pqn0Ts++zpPpXcp6L/R2/bJyBJSskF1XYNyBgzGhhdfF5kZC7GgL+/cPq0g9xciIo6S926WRgDZ86c5cQJCAvLo3HjDPz9hfz8DFJSICBAaN8+HT8/wd9f2LatgKws6NgxnbCw4/j5CQcOnGH/fmjZMpNWrY7g5wdnzhzjm28gMjKPAQMO4O8vBAQIn3xSQHY23H//XiIizhAQIPzvf+ls3gw33XSIq6/eQkCA8PPPe3nvPWjf/hRjx24kIEAICnLwwAPWPi1c+A1BQQ6Cghw8/3waa9fCU0/9RM+eVlvjzz/fybRp0KXLIcaPTwTg+PHjfPYZhITkEh+f6KyfwMCMwkdJZGZmkJkJJ07sA+DAgX0kJlpld+7cCUBGRoZzHkBubi4A3333HdGFvXMeOnTIuU5R2Z9++gmAtLQ0l/WLFJ+XVthNwU8//UTHjh1JTEx0vv6hQ4ecZY8fP+6Mofj6GRnWPiUlJTkf79u3z/nX0/t0au9ectq3J+ToUfLCw+H0aWufVq92u09hhW3CPb1P6enpOAo7savu96mq9qn4NnWf7N+nuLg4EhMTvXKfSmOsBFVGAWOWA43dLHpORBYWlkkExks5rwF169ZNkpK873JRYmIi8fHxdofhM7y+viZMsE67iVij0P373xc5KFDlxcfHk56eTnJysi2v74u8/vPlZby1vowxG0Skm7tlFzwCEpG+ng9JqWoQHGy1onjmGZg4sZZe/FLKe2kzbFVznD0L27ZB587W86eftnow7dLF3riUUm75VWZlY8xtxpiDwLXAl8aYrzwTllIVtGaNdS9PQoJ1sxNY/blp8lHKa1UqAYnIAhG5RESCRSRGRAZ4KjClyuXXX+H3v4devaxO6Bo2hCNH7I5KKVUOlUpAStnm7FmYOtXq++edd6zrOxMnWuM7dOxod3RKqXLQa0DKN917L/znP9bjm26C11+3Bg5SSvkMPQJSvkHEeQ8PYA2hcOWV8NVX8MUXmnyU8kGagJR3czisBNO9O867fcEave7HH6F/f/tiU0pVip6CU94pLw/mz7eu82zdas07eNAapCgy0nrup7+flPJl+h+svEt6Orz6qjXew4gRVvKJi7Ou8ezefS75KKV8nh4BKe9y9iy88IJ1BNS2LfzpTzB8uPZioFQNpAlI2ef4cfjoI/jyS+s6j7+/NSjRyy9byWfgQD3NplQNpglIVa/MTFi0CD780GrBlp9vzV+61GpODfDkk/bFp5SqNpqAVPU4fRrGjIGFCyEry5rn72/11TZyJPTrZ2t4SqnqpwlIVY3Dh+G772DoUOt5WBisXWsln549res6d95pDdGqlKqVNAEpzzh7Fr7/Hv73P+t02vr11vxDhyA21hoW4d13oVkzaNnS1lCVUt5BE5CqnD17YOxYqzfq7Oxz80NCoG9fOHXKSkAAvXvbE6NSyitpAlLlc/y4dYSzdq117ebPf7bmR0dbRz0FBVbXOAkJ56a6de2NWSnl1TQBKfd+/NFqHp2cDJs2QUrKuWWNGsFLL1mPw8Otcp06WU2olVKqnDQB1WanT8POndY4Olu3wi23wLXXWstWrIDnnjtXNiTE6o/t2mutRgQi55YN0GGglFIVpwmopsvNde1F4JFHYMsWK/EcPuxatm7dcwkoPh4ee8w6srnqKmjfHgIDqylopVRtoAmoJjh82DplduAA7N0L+/ad+5ufb12/KfLVV7Brl/U4JARat4YrrrB6Hih+JNO5szUppVQV0QTkzdatg2PH4OhRK8kcOnRueuIJuOsuq9znn8NDD7nfRmCg1ftAaKj1/K9/heBga/ycZs20qxullG00AVUFhwMyMqwmyOnp0Lw5RERYy779FlatspadOgVpadYRyvHjVplvvjm3nX79XAdhK27nznOP27Sxmjw3awYtWpybWra0mkD7+58rO2SIZ/dVKaUuUu1LQHl5cPKkdePkeVNkcjL06GGdmgLrdNWOHXDmjDVlZZ173K7duYv0x45ZF+azsqyjjYwM14v0ixbBzTdbj5cvhxdfdB9bVJTr8z59rNiio6FJk3NTbKx16qxI7956j41SyufYk4COHLF6PM7PLznde++5aw+LF1u9JZ9fJi/PShILFpzb5q23Wtc8cnNLJpdHH4VJk6xyK1eW2mqrE1hHCJdeas345z/h00/d78MNN5xLQEFBrs2UwTrlFRlpHdUUv3h//fXw9NPW/IgIaNDASjBFU3GffVZWLSqllG8TkWqfAME6RnBOgwvnLXrySSkye+hQAWRUsXKpheVijZHiugQHCyBJxcpOLCw7sUcPZ7mkt98WQLoEBoq0bClyxRUiV10lsYGBAkjqunXOsqN69RJAZg8cKDJpksjrr8uihx8WQAZfe+25Fy8oEOc+nTghkp8vIiKDBw+29mnRonP7NHu2tU+jRjnnpaamWvsUG+u6T126WPuUlOScN3HiRGufJk48t09JSdY+denisn5sbKy1T6mp5/Zp1Chrn2bPds5btGiRtU+DB7us79ynYorv08qVK2vcPhXx9D717t1bLr/88hq1TyI1733y5X0q+n/0tn0CkqSUXGDvFejnn7dOR/3lL1ZLLDh39AHWnfVgnYr6+GPraORf/7LmnX+6qmi9jz+2WnkdOADjx1vz+vY9V65TJ+tvhw5WNzLbt1s3WxYdfcTFnSvbpo3197bbrFifeMIaowasI5cixS/k16/ves1FKaWUW0aKX6uoJt26dZOkpKRqf90LSUxMJD4+3u4wfIbWV/nFx8eTnp5OcnKy3aH4DP18VYy31pcxZoOIdHO3TNvgKqWUsoUmIKWUUrbQBKSUUsoWmoCUUkrZolIJyBjzmjFmhzFmszFmgTEm0kNxKaWUquEqewS0DLhSRDoCPwPPVD4kpZRStUGlEpCIfC0i+YVPvwcuqXxISimlagNPdsXzAPBxaQuNMaOB0QAxMTEkJiZ68KU9IzMz0yvj8lZaX+WXnp5OQUGB1lcF6OerYnyxvi6YgIwxy4HGbhY9JyILC8s8B+QDH5S2HRGZA8wB60ZUb7xhyltv5PJWWl/lFxkZSXp6utZXBejnq2J8sb4umIBEpG9Zy40xI4DBQILY0a2CUkopn1SpU3DGmIHA00BvEcnyTEhKKaVqg8q2gpsBhAHLjDHJxpi3PBCTUkqpWqBSR0Ai0spTgSillKpdtCcEpZRSttAEpJRSyha2jAdkjDkG7K/2F76waOC43UH4EK2vitH6qhitr4rx1vpqLiIN3S2wJQF5K2NMUmkDJ6mStL4qRuurYrS+KsYX60tPwSmllLKFJiCllFK20ATkao7dAfgYra+K0fqqGK2vivG5+tJrQEoppWyhR0BKKaVsoQmoFMaY8cYYMcZE2x2LN9NRcS/MGDPQGLPTGJNijPl/dsfj7YwxTY0xK40x240xW40xj9odk7czxvgbYzYZY76wO5aK0ATkhjGmKdAPOGB3LD5AR8UtgzHGH5gJ3Ai0A+4xxrSzNyqvlw88KSJtgR7AOK2zC3oU2G53EBWlCci9vwF/AvQC2QXoqLgX1B1IEZE9IpILfATcYnNMXk1EDovIxsLHGVhfrHH2RuW9jDGXADcBb9sdS0VpAjqPMWYIkCoiP9odiw96AFhidxBeJg74pdjzg+iXabkZY1oAnYEfbA7Fm/0d6wezw+Y4KsyTQ3L7jLJGeQWeBfpXb0TezVOj4tZSxs08PbIuB2NMKPAp8JiInLY7Hm9kjBkMHBWRDcaYeJvDqbBamYBKG+XVGNMBaAn8aIwB63TSRmNMdxE5Uo0hehUdFbdSDgJNiz2/BDhkUyw+wxgTiJV8PhCR/9odjxe7DhhijBkEhADhxpj3ReRem+MqF70PqAzGmH1ANxHxxg7+vELhqLjTsEbFPWZ3PN7GGBOA1TgjAUgF1gPDRWSrrYF5MWP9+psHnBCRx2wOx2cUHgGNF5HBNodSbnoNSFWWjopbhsIGGn8AvsK6mP5vTT4XdB1wH3BD4WcqufAXvqph9AhIKaWULfQISCmllC00ASmllLKFJiCllFK20ASklFLKFpqAlFJK2UITkFJKKVtoAlJKKWULTUBKKaVsoQlIqSpkjPlT4cCG509/tjs2peymPSEoVYWMMWFAvWKzxgO/Ba4XkRR7olLKO2gCUqqaGGOeBv4I3CAiO+2ORym71crhGJSqbsaYZ7A6Je0jIj/bHY9S3kATkFJVrHCwvjFYQ1boaTelCmkCUqoKGWNeAEYB8SKy2+54lPImmoCUqiKFRz6PAkOAM8aYomHN00Ukx77IlPIO2ghBqSpQOKpnOhDuZnFfEflf9UaklPfRBKSUUsoWeiOqUkopW2gCUkopZQtNQEoppWyhCUgppZQtNAEppZSyhSYgpZRSttAEpJRSyhaagJRSStlCE5BSSilb/H9ma6MAFqxXiAAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# extra code – this cell generates and saves Figure 11–3\n",
"\n",
"from scipy.special import erfc\n",
"\n",
"# alpha and scale to self normalize with mean 0 and standard deviation 1\n",
"# (see equation 14 in the paper):\n",
"alpha_0_1 = -np.sqrt(2 / np.pi) / (erfc(1 / np.sqrt(2)) * np.exp(1 / 2) - 1)\n",
"scale_0_1 = (\n",
" (1 - erfc(1 / np.sqrt(2)) * np.sqrt(np.e))\n",
" * np.sqrt(2 * np.pi)\n",
" * (\n",
" 2 * erfc(np.sqrt(2)) * np.e ** 2\n",
" + np.pi * erfc(1 / np.sqrt(2)) ** 2 * np.e\n",
" - 2 * (2 + np.pi) * erfc(1 / np.sqrt(2)) * np.sqrt(np.e)\n",
" + np.pi\n",
" + 2\n",
" ) ** (-1 / 2)\n",
")\n",
"\n",
"def elu(z, alpha=1):\n",
" return np.where(z < 0, alpha * (np.exp(z) - 1), z)\n",
"\n",
"def selu(z, scale=scale_0_1, alpha=alpha_0_1):\n",
" return scale * elu(z, alpha)\n",
"\n",
"z = np.linspace(-5, 5, 200)\n",
"plt.plot(z, elu(z), \"b-\", linewidth=2, label=r\"ELU$_\\alpha(z) = \\alpha (e^z - 1)$ if $z < 0$, else $z$\")\n",
"plt.plot(z, selu(z), \"r--\", linewidth=2, label=r\"SELU$(z) = 1.05 \\, $ELU$_{1.67}(z)$\")\n",
"plt.plot([-5, 5], [0, 0], 'k-')\n",
"plt.plot([-5, 5], [-1, -1], 'k:', linewidth=2)\n",
"plt.plot([-5, 5], [-1.758, -1.758], 'k:', linewidth=2)\n",
"plt.plot([0, 0], [-2.2, 3.2], 'k-')\n",
"plt.grid(True)\n",
"plt.axis([-5, 5, -2.2, 3.2])\n",
"plt.xlabel(\"$z$\")\n",
"plt.gca().set_aspect(\"equal\")\n",
"plt.legend()\n",
"\n",
"save_fig(\"elu_selu_plot\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using SELU is straightforward:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"dense = tf.keras.layers.Dense(50, activation=\"selu\",\n",
" kernel_initializer=\"lecun_normal\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Extra material – an example of a self-regularized network using SELU**\n",
"\n",
"Let's create a neural net for Fashion MNIST with 100 hidden layers, using the SELU activation function:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42)\n",
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=[28, 28]))\n",
"for layer in range(100):\n",
" model.add(tf.keras.layers.Dense(100, activation=\"selu\",\n",
" kernel_initializer=\"lecun_normal\"))\n",
"model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=tf.keras.optimizers.SGD(learning_rate=0.001),\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's train it. Do not forget to scale the inputs to mean 0 and standard deviation 1:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"fashion_mnist = tf.keras.datasets.fashion_mnist.load_data()\n",
"(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist\n",
"X_train, y_train = X_train_full[:-5000], y_train_full[:-5000]\n",
"X_valid, y_valid = X_train_full[-5000:], y_train_full[-5000:]\n",
"X_train, X_valid, X_test = X_train / 255, X_valid / 255, X_test / 255"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"class_names = [\"T-shirt/top\", \"Trouser\", \"Pullover\", \"Dress\", \"Coat\",\n",
" \"Sandal\", \"Shirt\", \"Sneaker\", \"Bag\", \"Ankle boot\"]"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"pixel_means = X_train.mean(axis=0, keepdims=True)\n",
"pixel_stds = X_train.std(axis=0, keepdims=True)\n",
"X_train_scaled = (X_train - pixel_means) / pixel_stds\n",
"X_valid_scaled = (X_valid - pixel_means) / pixel_stds\n",
"X_test_scaled = (X_test - pixel_means) / pixel_stds"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2021-12-16 11:22:44.499697: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/5\n",
"1719/1719 [==============================] - 13s 7ms/step - loss: 1.3735 - accuracy: 0.4548 - val_loss: 0.9599 - val_accuracy: 0.6444\n",
"Epoch 2/5\n",
"1719/1719 [==============================] - 12s 7ms/step - loss: 0.7783 - accuracy: 0.7073 - val_loss: 0.6529 - val_accuracy: 0.7664\n",
"Epoch 3/5\n",
"1719/1719 [==============================] - 12s 7ms/step - loss: 0.6462 - accuracy: 0.7611 - val_loss: 0.6048 - val_accuracy: 0.7748\n",
"Epoch 4/5\n",
"1719/1719 [==============================] - 11s 6ms/step - loss: 0.5821 - accuracy: 0.7863 - val_loss: 0.5737 - val_accuracy: 0.7944\n",
"Epoch 5/5\n",
"1719/1719 [==============================] - 12s 7ms/step - loss: 0.5401 - accuracy: 0.8041 - val_loss: 0.5333 - val_accuracy: 0.8046\n"
]
}
],
"source": [
"history = model.fit(X_train_scaled, y_train, epochs=5,\n",
" validation_data=(X_valid_scaled, y_valid))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The network managed to learn, despite how deep it is. Now look at what happens if we try to use the ReLU activation function instead:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=[28, 28]))\n",
"for layer in range(100):\n",
" model.add(tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"))\n",
"model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=tf.keras.optimizers.SGD(learning_rate=0.001),\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/5\n",
"1719/1719 [==============================] - 12s 6ms/step - loss: 1.6932 - accuracy: 0.3071 - val_loss: 1.2058 - val_accuracy: 0.5106\n",
"Epoch 2/5\n",
"1719/1719 [==============================] - 11s 6ms/step - loss: 1.1132 - accuracy: 0.5297 - val_loss: 0.9682 - val_accuracy: 0.5718\n",
"Epoch 3/5\n",
"1719/1719 [==============================] - 10s 6ms/step - loss: 0.9480 - accuracy: 0.6117 - val_loss: 1.0552 - val_accuracy: 0.5102\n",
"Epoch 4/5\n",
"1719/1719 [==============================] - 10s 6ms/step - loss: 0.9763 - accuracy: 0.6003 - val_loss: 0.7764 - val_accuracy: 0.7070\n",
"Epoch 5/5\n",
"1719/1719 [==============================] - 11s 6ms/step - loss: 0.7892 - accuracy: 0.6875 - val_loss: 0.7485 - val_accuracy: 0.7054\n"
]
}
],
"source": [
"history = model.fit(X_train_scaled, y_train, epochs=5,\n",
" validation_data=(X_valid_scaled, y_valid))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Not great at all, we suffered from the vanishing/exploding gradients problem."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### GELU, Swish and Mish"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAADsCAYAAAAy23L6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABbDElEQVR4nO2dd3hURdfAf5PeSCMkEECKIUiXYoiAEHpVFLChIIIiCIIKSrGAH0XBV1+VgIKgqPBSFUSqoCRUKUoLHamBQEhICOltvj9udrObbJINKZsyv+e5z+6de+7Mmbt377kzc+aMkFKiUCgUCkVpY2VpBRQKhUJROVEGSKFQKBQWQRkghUKhUFgEZYAUCoVCYRGUAVIoFAqFRVAGSKFQKBQWocgGSAhRWwixUwhxWghxUggx3oSMEEJ8JYS4IIQ4LoRoVdRyFQqFQlG+sSmGPNKBCVLKf4QQVYC/hRDbpZSnDGR6Aw2ytrbA11mfCoVCoaikFLkFJKWMkFL+k/X9HnAaqJlDrD/wo9T4C3AXQtQoatkKhUKhKL8URwtIjxCiLtASOJDjUE3gmsF+eFZahIk8RgIjARwcHFo/8MADxalimSczMxMrq8o1NKfqXPG5du0aUkoq2/8ZKt9vDXDu3LkoKWW1guSKzQAJIVyAn4E3pZRxOQ+bOMVkDCAp5SJgEUDDhg3l2bNni0vFckFISAhBQUGWVqNUUXWu+AQFBREbG8vRo0ctrUqpU9l+awAhxBVz5IrFLAshbNGMz3Ip5S8mRMKB2gb7tYAbxVG2QqFQKMoOUVHmyxaHF5wAlgCnpZSf5yG2ARia5Q0XCNyVUubqflMoFApF+eXqVWjZ0nz54uiCaw8MAU4IIY5mpU0FHgCQUn4DbAb6ABeARODlYihXoVAoFGWEyEjo3h3Cw80/p8gGSEq5B9NjPIYyEhhT1LIUCoVCUba4GHORLadDWTL2Zc6dgxYt4Ngx884tVi84hUKhUFQeIu5F0O3H7lyKvQjOt/Dzm8y2bVC9unnni7K8IF1+XnCZmZmEh4eTkJBQylqVLMnJyTg4OFhajVJF1bnic/PmTaSU1KhhPP3P1tYWb29vXF1dLaRZyVORveCG/PISy078qO2kOxD6zEk6NquPEOJvKWWbgs4vty2gqKgohBA0bNiwQvnY37t3jypVqlhajVJF1bniY2VlRXp6Oo0aNdKnSSlJSkri+vXrABXaCFVEMjMh/ddgENegzi6CO66hY7P6hcqj3D65Y2Nj8fHxqVDGR6GoTAghcHJyombNmkRGRlpaHUUhkBLeegtW/lgFp3WbmR/4O2O69yt0PuW2BZSRkYGtra2l1VAoFEXE0dGRtLQ0S6uhKAQzZsBXX4GdHfz6swPdunW5r3zKrQEC7Q1KoVCUb9T/uHwgpWTKH1NIOdGPL6Z1wMoKVqyAbt3uP0/Vf6VQKBSKAvko9CPm7J3DF9E9oMFmFi2CAQOKlqcyQAqFQqHIl+jEaL7c+422Y5tEq5dWMmJE0fNVBkihUCgU+XL2aFWS5u+BmLrUz+jF/smLiyVfZYAUCoVCkSdhYdC3L6RE+PF84j6OvbcWO2u73IJ//glnzhQqb2WAFADExMTg4+PDv//+a5b8oEGD+PzzvGLPlj0qev0UiuJGSsmVK9CzJ8TGwpNPwo8LauBi75xb+K+/4PHHoUMHuHzZ7DKUAbIAt27d4q233qJBgwY4ODjg7e1Nu3btmDdvHvHx8QAMGzYMIUSuLTAwUJ/PsGHD6Ncvb9/7oKAgxo4dmyt96dKluLi4GKXNnj2bPn368OCDD5pVh2nTpjFz5kzu3r1rlnxJcvjwYYQQXM7nxi/P9VMoSpujN48SsLA9nfuHc+MGdOqkebzZmPKbPnkS+vSBxETo1w8KseigMkClzOXLl2nVqhVbt25lxowZ/PPPP/z5559MnDiRP/74g82bN+tlu3XrRkREhNFmeLy4SExMZPHixYwoxKhis2bNqF+/PsuWLSt2fczl1KlTDB06lEGDBgHa9Ro+fDg5wzeV1/opFJbgwp0L9PypF4dv7edS5/Y81P4cv/4KJiNHXb4MPXpATAw88QQsXgyFCA5QYQyQEJbZCsvo0aOxsrLi8OHDPPfcczRu3JimTZsyYMAA1q9fz9NPP62Xtbe3p3r16kabp6dnMV41jc2bN2NlZUX79u2N0ufOnWuyFfbhhx8C8MQTT7BixYpi16egcgF++eUXWrRoQUJCAm+++SYAU6ZMITo6mmbNmrF+/foyWz+Foixz9PpJbsffAcDK6S7BC5NwczMheOuWtv6Crom0cmUeTaS8qTAGqDxw584dtm3bxpgxY3B2NtGPimUm5e3evZvWrVvnKnv06NFGra8JEyZQvXp1hg4dCkBAQAAHDx4kKSkpV56zZ8/GxcUl32337t0m9Smo3NTUVF577TV69uzJzz//TIcOHQDo2rUrv/76K926dWPkyJH62fUlUT+FoiKSkQGrZ/RHLv8NkeTF//ptpGuTFrkF09K0brcLF7QV6H79FRwdC11euY6EYEgZDuqt5/z580gpadiwoVF6rVq1iI2NBeDZZ59lyZIlAGzdujXXWM2YMWOYM2dOsep15cqVXFGKAapUqaIPmDlnzhxWrFhBSEgIfn5+APj6+pKWlsaNGzdyja2MGjWKZ555Jt9ya9asaTK9oHJPnDhBVFQUQ4YMMXn+kCFD2LJlCydOnKBVq1YlUj+FoqIhJYwbB2vWgKtrT7YMukS7Ni6mhW1t4fXX4T//ga1bMd1EKpgKY4DKM7t37yYjI4ORI0eSnJysT+/YsSOLFi0yknV3dy/28pOSkvDx8cnz+Mcff0xwcDA7d+7E399fn+6Y9cZjqoXg6elZ5O7CvMrVLSGSV2tRF6BWJ1cS9VMoKgqpGalYCStmzbBhwQKwt4cNG8jb+OgYMQJefFE74T5RXXCliJ+fH0IIzuTwla9Xrx5+fn44OTkZpTs5OeHn52e0eXl5mV2eq6urSS+u2NhY3AzeWLy8vIiJiTGZx6xZs1iwYAGhoaFGD2fQuhQBqlWrluu8onTBFVRu8+bNqVq1KsuXLzd57rJly/Dy8qJZs2YlVj+FoiKQkZnB0HVDaTVnINNnJmFlpQ3ldOpkQjgzEyZMgOPHs9OKYHxAtYBKlapVq9KjRw+Cg4N54403cnWvFTcNGzZk8+bNSCmNWgv//POPUTdgy5YtWbp0aa7zZ8yYwbfffktISIjJLqiwsDB8fX1Nti6K0gVXULl2dnYsWLCAwYMH8+yzz/LYY48BEBoayttvv83WrVtZuXIldnZ2JVY/haK8I6XkjS1vsOrkKi3hxd4Et9/Ck0+aGMuRUjM+X3wBq1ZpYz/FsKCiMkClzIIFC2jfvj2tW7dm+vTptGjRAhsbG/7++2+OHTtG586d9bIpKSncvHnT6Hxra2ujN/K4uDiOHj1qJOPu7k7dunUZPXq03ti9+uqrODg4sHnzZlasWMGvv/6ql+/ZsyeTJk0iOjqaqlWrAlrL4Msvv2TDhg04Ozvr9XB3d9ev5Ll792569eplsp732wVnTrkAzzzzDI0aNeKTTz5h7ty5gDZ3p2PHjhw7dozGjRuXaP0UiorAnYjsRQAfrd+MUSPyMCqzZ2vGx9YWvvuuWIwPoFnBsrr5+/vLvDh16lSex8o6ERERcty4cfLBBx+UdnZ20tnZWbZp00bOnj1bXr9+XUop5UsvvSSBXFvNmjX1+eQlM3DgQL3MwYMHZY8ePaS3t7d0dXWVAQEBct26dbl0CgwMlMHBwVJKKTMzM6Wrq6vJvHfs2CGllDIpKUm6urrK/fv3F/l6xMXFmV2uKQ4dOiQBeenSpTxlLFk/U+jqXFk4c+aMDAsLy/N4ef4/F8TOnTstrYJJDh6U0tlZStrPkQ9NHSzTMzJMC379tZQgpRBSrlplVt7AYWnGM97iRia/raIaoPyw1INpy5Yt0t/fX6anp5slHxwcLLt3714sZZdGnS1ZP1MoA2RMRf0/S1k2DdCZM1J6eWkW4MUXpUxPzzQtuHKlZnhAym++KTjjjAwpk5PNNkDF4oQghPhOCBEphAjL43iQEOKuEOJo1vahKTmF5ejVqxdjxowhPDzcLHlbW1vmzZtXwloVHxW9fgqFOVyOvcz161rwgqgo6N1b61GztjbhURoeDkOHauM/s2bBa6/ln3lGBrzyihY0zkyKawxoKRAM/JiPzG4pZeEXDVeUGuPGjTNbduTIkSWoSclQ0eunUOTHriu76PlTT6ocn8Ttq9N49FHBmjXasI5JatXSrNOxYzBlSsEFCKF5yoWGmq1TsbSApJS7gDvFkZdCoVAoipeTkSd5/H+Pk5yRzO0mH1Ft4Cw2bgSTAVkyM7O/v/ACzJ1rXtwxKytYsgQOHTJbr9L0gntUCHEMuAFMlFKeNCUkhBgJjARt/kVISIjJzNzc3Lh3714JqWo5MjIyKmS98kPVueKTnp4OkGedk5OT8/yvl3fi4+MtXrd7KcmI649AtT+wSqjOh089xPHjuXVyDA+n6QcfcGbyZO7liNhiCqvUVGqtWcO1Z55B5tmUygdzBorM2YC6QFgex1wBl6zvfYDz5uSpnBAqB6rOFR/lhGA5MjI0RwOsU6TDoJHy17+OmRa8elXKBx7QHA6eesq8zJ9/XpMfMcIomdJ0QjDDyMVJKeOzvm8GbIUQ5k/pVygUCkWhkRImToRly8DZwY5d7y7kibbNcwtGRkK3bnD1Kjz6KPyY33C+ARMnQoMGWhC5+6BUuuCEENWBW1JKKYQIQBt7ii6NshUKhaKyEZ8aT1hkGKHLA/nvfzVHg3Xr4JFHTAjHxmrLnp47By1awKZNkF+UFimzx4RatYJTpwq9DIOO4nLDXgHsBxoKIcKFECOEEKOEEKOyRAYBYVljQF8Bz2U10xQKhUJRjKSkpzBg1QAeWxLE5KXrEQJ++klbuicXCQnQty8cPQr+/rBtG3h45J15dDR07AiGY1r3aXygmFpAUsrnCzgejOamrVAoFIoSZMLvE9h+cbu288wgpnqG8eyzD5kW3rUL/voLateG7duhoLiH33wDe/bAm2/CP/8UavVTU6hYcAqFQlGBeMx6IgvubEN6XqATHzJzfB7GB7SZqGvXQpMm8MADBWc+ZQokJWlrARXR+IAyQAqFQlFhOHYMRj5TF5mxh0dfW8afn76dWygzE65cgXr1tP2nnso/0/BwqFZNW3rBygpmziw2fdV6QOWcYcOG0a+feQEmgoKCGDt2bIFyMTEx+Pj48O+//5qV76BBg/j888/NkrUkFbVeCgXAxYuaL0FcHAzq7cPuuROwssoxgVRmLXvaqhXs319wphcuQLt28Oyz2jLcxYwyQKXM7du3ef3116lbty729vb4+PjQtWtXtm/ffl/5ffnllyxbtqxYdZw9ezZ9+vQxexnqadOmMXPmTJOL35UlKmq9FJWbn0/9zKHzl+nRA27dgi5dNLdra2sTwu+/D/PnQ2KithVEYiLEx8Pt21rXWzGjuuBKmYEDB5KYmMiSJUvw8/MjMjKS0NBQoqPvzyvd7T7XYs+LxMREFi9ezG+//Wb2Oc2aNaN+/fosW7aMMWPGFKs+xUVFrZeicvPb2d94du2zWCX6kBb3O61bN2H9+jwWKp07V1vXx9oaVq+Grl0LLqB5c81RoW7d/F2z75OK1QISIu9t0aJsuUWL8pc1pHVr8+TMIDY2lt27d/PJJ5/QtWtX6tSpwyOPPMLEiRN57rnn2LJlC76+vvqwJefPn0cIwejRo/V5vPfee3Q38KfM2QW3a9cuAgMDcXFxwc3NjbZt2xIWlh2kPDMzk6lTp+Ll5YW3tzcTJ04k0yD20+bNm7GysqJ9+/b6tLlz5yKEyLV9+GF2UPMnnniCFStWFPqa5Ic55eo4ePAg3bt3p1q1arnk//333zJVL4WiOLibfJeh64aSITNIc7yB48A32LwZqlQxIfzNNzBpkvbc+uEH6N8/74z/+ENzx9bRtGmJGB+oaAaojOPi4oKLiwsbNmwgOTk51/HHHnuM5ORkDh8+DEBISAheXl7s3LlTLxMSEkJQUJDJ/NPT0+nfvz8dOnTg2LFjHDhwgPHjx2Nt0BZfvnw5NjY27Nu3j+DgYL744gtWrVqlP757925at25ttIT36NGjiYiI0G8TJkygevXqDB06VC8TEBDAwYMHSTLRTJ89e7a+7qa2GjVqsHv37lznmVMuaEtnBwUF0ahRI0JCQvjzzz+pXr06AQEBLFu2jPr165dIvRQKS+Ji60bTk79AShWs79YndNz/8PY2Ibh8uea1BvD111qA0bw4fhz69NEcE86cKRG9jTAnXo+ltooYC27t2rXSw8ND2tvby8DAQDlhwgT5119/6Y+3bt1azp49W0op5eDBg+X06dOlg4ODvHHjhkxISJC2trZyz549evmXXnpJ9u3bV0opZXR0tARkSEiIybI7deokAwMDjdK6desmRxjEcerfv78cOnRonvp/8skn0tfXV545c8Yo/dixYxKQFy5cyHVOdHS0PH/+fJ7bkSNHZGJiYp5l5leulFJ26dJFDhgwwCht8uTJ0s/Pr0TrVRRULDhjyuv/2RxKIhZcZqaUr76qhWFz8T8sN+3L5/5csUJKa2sp58wxL+OXX5Zy7FgtiNx9gpmx4NQYUCkzcOBA+vbty+7du9m/fz9bt27ls88+Y9asWUydOpXHHnuMkJAQpkyZQmhoKOPHj+fPP//Ut4ZsbW0JCAgwmbenpyfDhg2jZ8+edO3ala5du/L0009Tu3ZtvUzz5sZxoHx9fYmMjNTvJyUl4ZPHZLSPP/6Y4OBgdu7cib+/v9ExR0dH/fmm9PL09Mzzmty7d09/fmHLjYqKIjQ0lG2GXQaAs7OzUWunJOqlUFiKDz6Ab78FBwfY+l1r2j+aj/Bzz2ljOY0b5y2Tnq5FNBBCy9jK6r6GGQqL6oKzAA4ODnTv3p0PP/yQffv2MWLECKZPn05qaiodOnRg7969nDp1inv37tG6dWuCgoLYuXMnISEhtGvXDtt8wp5///33HDhwgI4dO7Jhwwb8/f2NHs45zxVCGI0BeXl5ERMTkyvfWbNmsWDBAkJDQ3M9pAHu3NGWg6pWrVquY/fbBWdOuX///TcZGRm0aNHCKP3w4cM8YhD4qiTqpVCUJneS7jBuyzjmfpHArFmaL8GaNWAwrJnN9u1w8GD2fl7GR0qYPh0efxxSU7U0a+tSMT6gvODKBI0bNyY9PZ3k5GTatWtHSkoKc+fOpUOHDlhbWxMUFMTIkSPx9vamT58+BebXokULWrRowaRJk+jduzc//PADPXv2NEuXli1bsnTpUqO0GTNm8O233xISEpKnC3NYWBi+vr4mWxmjRo3imWeeybPM+Ph4GppYe8SccjMyMgDjFsqFCxfYtm0b69atK9F6KRSlRUJqAv3+14/94fvh2mFw3MiSrz0xOQUwNFRzMrCxgb//1qJV50VkJCxYoMV427NH8+EuRVQLqBSJjo6mS5cuLFu2jOPHj3Pp0iXWrFnD3Llz6dq1K66urri4uNCqVSuWLVtG586dAXj00Ue5du0aBw4cyNMBAeDSpUtMnjyZffv2ceXKFXbu3Mnx48dpnF/TOwc9e/bk9OnTerfwWbNm8eWXX7Jy5UqcnZ25efMmN2/ezOVEsXv3bnr16mUyT09PT/z8/PLcHnzwwVxdcOaW27ZtW5ycnHj33Xc5ffo027Zto2/fvjz33HNG+pREvRSK0mL9mfWa8QGovZ9h03bz0ksmBPft04KLJiVpk0cLmvPm4wNbt2pNqVI2PoByQihNkpOT5ZQpU2SbNm2ku7u7dHR0lH5+fvKtt96S0dHRUkptcHrSpEkSkIcOHdKf26lTJ+nk5CRTU1ON8jR0Qrh586Z86qmnpK+vr7Szs5O1a9eW77zzjv6cTp06yTFjxuR5vo7AwEAZHBwsMzMzpaurqwRybTt27NDLJyUlSVdXV7l///77ui45B+TNLVfHpk2bZMOGDaWtra2sW7eunDFjhkxLS8slV9r1KkydKzrKCaFo7NsnpW2HLyXTkV2nfmVa6NAhKV1dNc+EF1+UMj3dtFxSkpZhCYKZTggWNzL5bRXNAJlDWXgwbdmyRfr7+8v0vG7gHAQHB8vu3bvfd3mlVefSrld+lIXfuTRRBuj+CQuT0sNDe1r3H3lcZmaaEDp6NFvo6aelNPECJqWUMjlZyq5dpbS3l/LPP4ukV36Ya4BUF5wiF7169WLMmDGEh4ebJW9ra8u8efNKWKuiU1Hrpai4XL2qxXeLiYEnnoC185vl9g9ISNCiWsfEaGM/y5fnvUaPnR089JC25o+X5RelFpqxKps0bNhQnj171uSx06dP06hRo1LWqOS5d+8eVUxOZa64qDpXfM6ePUt6ejpNmjQxebyi/p8h/8njefHVga+4EHmDbe98zLmzgsce04IT5Dlb4eeftWW0V6/OIw6PARkZcPMm1KxZKJ0KgxDibyllm4LklBecQqFQlCGWHV/G+K3jtZ2GUTRz+IYNG2xyG5/MzOw1eQYOhAEDTLtP//MPfPKJZqAcHDQ36xI0PoVBdcEpFApFGUFKyS+n1uv37WudZt1vqbi75xC8dk2LU7l3b3aaKeOTkQGDB2tebp99VhIqFwllgBQKhaKMIKXAZv1KOPoS1tFN2fv6Rh6s7WQsdOOGFsn66FEtJEJ+wyi62aqvvQbvvFOiut8PqgtOoVAoygBSwvjxsGaVDS5VvmPj9ru0buJhLHTjBgQFwfnz8PDD2nLaplo+Z8+CbnJ3s2ZaNOwyiGoBKRQKhQVJSU8BtJWug4M1R7XfNljRqa0J49O5s2Z8WrSAHTsgZ4xFKbVlF5o2hd9/L6Ua3D/FYoCEEN8JISKFEGF5HBdCiK+EEBeEEMeFEK2Ko1yFQqEoz1y9e5UmC5ow4ouVfPih5lOwYoXWyDEiIkIzPufOacbnjz+gatXcGQqhjfuAtoppGae4WkBLgfzilfQGGmRtI4Gvi6lchUKhKJfcir9F95+682/Mv3wXOxhaLuGbbzRntlwcOQIXL+ZvfHTMnasFIs1v3Z8yQrEYICnlLuBOPiL9gR+zJsn+BbgLIWoUR9kKhUJRHsmQGaQkZQ3DZ9jy8oAHePXVPIT79IGNG00bn/BwGDZMm5AKWjOqZcuSUrtYKS0nhJrANYP98Ky0iJyCQoiRaK0kqlWrRkhIiMkM3dzcuHfvXrEramkyMjIKVa9Ro0YRHR3NmjVrCpTt06cPjRo14rMy5o6ZV51jYmJo06YN27dvp379+gXmM2TIENq2bcvYsWNLQs1io7D1Kg1K+trplpnP695OTk7O879e3omPjzdZt7Nnq3Drkz9gwJMEZoxgyBBbIzm76Gjsb93ini6YsL09nDiRK5+Hx4/H/fhxwuPiuDBuXAnVooQwJ16PORtQFwjL49gmoIPB/h9A64LyrIix4CIjI+Xo0aNlnTp1pJ2dnfT29pZdunSRv//+u5Sy8DHCYmNjZUxMjFmypoKRlgXyqvPEiRPlsGHDzM7n+PHj0sPDQ8bGxhaXaiXCxIkT5QsvvJArff78+bJu3brS3t5etmrVSu7atavAvG7cuCGHDh0qvby8pL29vWzUqFGeK+LmR0lfOxULzpizZ6X08tJCtz0/OCP34qMREVI+9JCULi5SGqyYbJIzZ6Ts31/KrIDGZQHKWCy4cKC2wX4t4EYplV2mGDhwIAcPHmTJkiWcO3eOjRs30rt3b/0yAYXFzc0N91yz1Mo/iYmJLF68mBEjRph9TrNmzahfvz7Lli0rQc2Khq5eQ4cONUpftWoV48ePZ+rUqRw5coR27drRu3dvrl69mmdesbGxtG/fHiklmzZt4vTp08ybNw9vb+9C61Uerl15Jz0znSuxV7h+HXr0gKgo6NULln5vpQ9oAGhhcjp3hjNnoF4900sqnDmT/b1hQ1i/PrdHXHnAHCtlzkb+LaC+wBZAAIHAQXPyrGgtoJiYGAnI7du3mzy+efNm6eLiol9K4Ny5cxKQo0aN0stMnTpVduvWTb+fczmF0NBQ2bZtW+ns7CxdXV1lQECAPHHihJRSawGNHj1aTpkyRVatWlVWq1ZNTpgwQWYUYe334sBUC2jNmjXS09NTZuYI/TtnzhyTyyh88MEHUkopP/roI9m+ffti17GgcnUcOHBAduvWTXp5eeWSvXDhgr5ed+/eNTovICBAvvLKK0Zpfn5+cvLkyXnqNGXKFNmuXbti07+krp2UqgWUkZkhh/wyRHrNqSbrtzsiQcrAQCnj43MI37ihtXxAymbNpIyMzJ3hBx9IaWUl5YYNpaL//UBptoCEECuA/UBDIUS4EGKEEGKUEGJUlshm4CJwAfgWeL04yi1v6Jag3rBhQ66FzwAee+wxkpOTOXz4MKAFMfTy8mLnzp16mfwCG6anp9O/f386dOjAsWPHOHDgAOPHj8fa2lovs3z5cmxsbNi3bx/BwcF88cUXrFq1qngrWgzs3r2b1q1bI3JMshs9ejQRERH6bcKECVSvXl3foggICODgwYNGK6TqKGhpcBcXlzyXBi+oXNBWTw0KCqJRo0aEhITw559/Ur16dQICAli2bBn169c3Wa/U1FT+/vtvevToYVRmjx492LdvX57XaP369bRt25Znn30Wb29vHn74YYKDg3UvfYXWP79rpygak7ZP4qfjPxGVdJuLHYNoEHCRjRvB2dlA6No16NhRa900bao5HJhaCl7XXIqMLBXdSxRzrJSltkK3gLRpWMZp/fppaYZvCwsXammvvpqddv26llajhvH5rVpp6YcPZ6dNm5a7HDNZu3at9PDwkPb29jIwMFBOmDBB/mXQx9u6dWs5e/ZsKaWUgwcPltOnT5cODg7yxo0bMiEhQdra2so9e/bo5Q1bQNHR0RLIcwygU6dOMjAw0CitW7ducsSIEUZpb7/9tmzWrJn09/eX69atu696btq0Sfr7+0s/Pz85f/78POUuXbokO3bsKBs1aiQbN24sb9++LaWUsn///nLo0KH5lvHJJ59IX19feebMGX3asWPH9K2NnERHR8vz58/nuyUmJhZYN1PlSillly5d5IABA4zSJk+eLP38/PT7unoZtvquX78uARkaGmp07kcffSTz+w/Y29tLe3t7OXnyZPnPP//I7777Tjo7O8t58+bdl/75XbuiUtlbQDvP75M277lLpiOrDB4pw8NzLOqTmiplw4bac+Xhh6XM+h+YJDNTyiNHSlTnokIZGwNSZDFw4EBu3LjBb7/9Ru/evdm3bx+BgYHMnj0b0FpBOk+Y0NBQevfuTUBAACEhIezduxdbW1sCAgJM5u3p6cmwYcPo2bMnffv25fPPP+fatWtGMs2bNzfa9/X1JdLgTWrdunVcv36dY8eO8dVXX7Fly5ZC1zE9PZ3x48ezY8cOjh07RnBwMBERuRweAXjppZeYMmUKp06dYt++ffolCpKSknBwcMizjI8//pivvvqKnTt30lAXcgT0S3ubeosvaGlwPz+/XEuDm1tuVFQUoaGhvP66cePe2dnZqLWTX71ytvaklLnSDMnMzKRVq1Z8/PHHtGzZkpdffplx48Yxf/78QusP+V87xf2Tni747K1HSV8cglPYGP75vwXUrJnjd7W11SJWt28Pf/5pvFbPhQtatOu4OG1fCC0MTwWgYhkgXRvIkN9+09Iefzw7beRILW3Rouw0X18t7UYO34i//9bSW7fOTps+Pf8AgAXg4OBA9+7d+fDDD9m3bx8jRoxg+vTppKam0qFDB/bu3cupU6e4d+8erVu3JigoiJ07dxISEkK7du2wtbXNM+/vv/+eAwcO0LFjRzZs2IC/vz/btm3TH895rhCCzMxM/f7GjRt5+eWXSUhIYNGiRTz55JOFrt/Bgwdp3LgxtWvXxsnJiaeeeoqNGzfmkjt58iS2tra0b98e0Bwq7LPWMvHy8iImJsZk/rNmzWLBggWEhobi7+9vdOzOHW06WjUTXRdF6YIrqNy///6bjIwMWrRoYZR++PBhHnnkEf2+qXp5eXlhbW3NzZs3jdIjIyPx8fHJU58aNWrQWOeim0WjRo3ydFzIT3/I/9op7o/MTPjkk4fYuBE801pwcHowfg9md4mTmpr9/cknYdcubbE4HVLC8OHwyy9a4NEKRsUyQOWUxo0bk56eTnJyMu3atSMlJYW5c+fSoUMHrK2tjQyQOQtbtWjRgkmTJunlf/jhB7N1OXLkCNeuXcPLy4ubN2/SvXt3o+Nt27bl4YcfzrX9+++/epkbN25Qu3a202OtWrW4fv16rrLOnz9PlSpVeO6552jZsiUffvih/ljLli05depUrnNmzJjBwoULCQkJwc/PL9fxsLAwfH19TT64R40axdGjR/Pd2rQxvYZWQeVmZIU/MWw9XLhwgW3btjFkyJB862VnZ0fr1q3Zvn27Ufr27dtp166dSX0A2rdvT84FG8+dO0edOnUKrT/kf+0UhWPD2Q3EJMUydiz88YcPLi6wdSsYrcd39Cj4+4PhS49VjkeyELB0KQwZogWLq2iY009nqa2iecFFRUXJzp07y59++kkeO3ZMXrx4Ua5evVr6+PjoPdvi4uJkQECAtLa2lp9++qmUUsqkpCRpb28vra2tjcZ/pDQeA7p48aKcNGmS3Lt3r7x8+bL8888/pa+vr5wxY4aU0vQ8IMPz09LSZL169fR69OnTRy5ZsqTQ9Vy9erVROQsWLJDTp0/PJbdmzRrp7u4uw8LCZGpqquzTp4/85ZdfpJTavBQrKysZFRWll585c6asWrWq3Lt3r4yIiNBvSUlJRvUZPnx4oXXOD3PKjYqKkk5OTvK5556Tp06dklu3bpX+/v655jHp6nXp0iWj9JUrV0pbW1v57bffylOnTslx48ZJZ2dnefnyZb3MvHnzZMOGDfX7Bw8elDY2NnLmzJny/PnzcvXq1dLV1VUGBwcXWn8pS+ba6ahMY0D/O/4/KaYLWf3DVhLHKGlrmyFzTQU6eFBKDw+tz+aZZ4yPpadLmYenbHkBM8eALG5k8tsqmgFKTk6WU6ZMkW3atJHu7u7S0dFR+vn5ybfeektGZ00ii4uLk5MmTZKAPHTokP7cTp06SScnJ5mammqUp6EBuXnzpnzqqaekr6+vtLOzk7Vr15bvvPOO/pyCDNCxY8dk79699cemTZuWywAFBATIFi1a5NoMB6737t0rn3jiCf3+1KlT5aJFi3Jdj/3798uuXbvqB+QXLFggP/zwQ/3xwMBA/cM0MzNTurq6mnQl3rFjh5RSM9Surq5y//79ef8IhcSccnVs2rRJNmzYUNra2sq6devKGTNm6F3qDQkMDJT/+c9/cqXPnz9fP0G5VatWuZwSpk2bJsnh/LJx40bZvHlzaW9vLxs0aCC//PJLI9d1c/UviWtnSGUxQJdiLkmb/7ORTEcyHSkGvChnzTpuLLRnj5Surtrjt39/KZOTs49lZEj5+OPasbVrS1X34kQZoHJKYSMhFCc//vij7Ny5s8zMzJT37t2TLVu2lNevXy90PmlpafLBBx+UV69elYmJibJRo0Ym80lPT5cPP/ywvHTpkszMzJTPPvusXLlypf74li1bpL+/v0xPTzer3ODgYNm9e/dC61vabNmyRfr5+Zldr9KgpK9dZTFAUko55L+LJdOE5PUm8psfbxtHQti5U0pn5+yWT44XSimllJ9+KqWnp5T3EdGirGCuAVJjQAo9R48eJSYmhvr169O5c2c++OADfH19C52PjY0N//3vf+natSvNmjVj9OjRRvk8nOXBY21tzZw5c+jXrx/NmzfHy8uLZ555Ri/Xq1cvxowZQ3h4uFnl2traMm/evELrW9r06tWLV1991ex6lQbl5dqVdVauhGVvj4DVa/m40Q5eG2LgzbZ1K/TurQUNHTIEli/XvN8AsmLlATBhApw6BZ06la7ylsAcK2WpTbWASpcuXbrIc+fOlXq5lqyzpahsda4MLaCNG6W0sdEaN1lT+aSUBrHgNm/WBF59VRoFf/vxRy3qQRmK5VZUUC0gRWG5cOECD5qKO6VQKEyy/9p+3tz6JjtDMhk0SGvIvPsuTJ5sQrh3b22dnoULs73d0tPhiy+0KNcrV5am6mWC0lqOQVEOuHLliqVVUCjKDYdvHKbX8l7EpcQx/2QM6alLGDnShk8+0bynkRLmzsXdxiZ7idOc6/TY2GhzfLZvh1deKe0qWBzVAlIoFIr74MdjPxKXokUnSK+7hf5DrrNggYHxeecdmDyZptOmgWG0+6NH4dNPs/fr1KmUxgdUC0ihUCjui+G+X7AoLImU+r/Q7cYO1i6tg7U1Wrfaa6/Bd9+BjQ1n336bJrpVTO/c0ZwL4uK0Wal9+li0DpZGGSCFQqEoJKdPQ4/uVqRELaTrgPfYvKIuNjZASgoMHqx1qzk6wi+/cNsw9p+npxbR4MgR6NLFYvqXFVQXnEKhUJjB3eS7AJw/D127wu3b0LOHFZuW19W8qe/e1Vo0v/wCbm7auE6vXtjExWkWS8fYsbBkCeQTbLeyoAyQQqFQFMDxW8dpMK8Bn/y+mK5dISJCW7T0l18gK36uZmT27AEfHwgN1SJbX7pE61GjNA+4qChNToisgSKF6oJTKBSKfDgZeZIuP3QhOimaKftfharWtH/gZTZsACcnA8HAQM0iNW6sLaUNUKMGaW5uODo7a91zCiNUC0ihUCjywbeKLzWd62o7ya408mrC5s3g4gLs3QubNmUL9+2rLe2iW2bBwYETs2drEa9r1ixt1cs8ygApFApFPiREe3BvwXa42JUGf21nz6oAXF2BdeugWzd4+mk4eVITvnxZ63p79139+WkeHppDgiIXygApFApFHly7ps0hvXTKg4dPbGf/2gA8PYH587VVSpOTYehQ0K0ue+sWHD8Ov/6qOSUo8kWNASkUCoUB+6/tJ1NmUpv2dO4MFy9Cq1awfbvA0z0TprynLZ8Nmkv11KnZTgVt22rjQO3aaZ5winxRBkihUCiyCLkcQr//9cMKa6qs+4MbF9vQurXmUe3hkATPD4PVq8HaGhYv1tyu+/aFadM04wPQr59F61CeUF1wZZxhw4bRz8wbOigoiLFjxxYoFxMTg4+Pj9Ey2nkxaNAgPv/8c7PKL4+UxDUzh8zMTF577TWqVq2KEIKQkJD7zqsw94i5FLa+pu6T8nbvpGak8vKvL5OQlsC9tDhutB1Cm0cy2LEDPDyACxdg40aoUkX7HDYMgoNhyxZtbo+Ulq5CuaNYDJAQopcQ4qwQ4oIQIlccWCFEkBDirhDiaNb2YXGUWx4ZNmwYQgheMRH76d1338XV1dXoYfLll1+ybNmyYtVh9uzZ9OnTx6zI19OmTWPmzJnctWB/trlGoiQpzDUzh23btvH999/z22+/ERERQbt27cpEPXUUtr6m7pOycO8UBjtrOxY89itWyZ4Q50uzU+vY/rs17u5ZAs2awZo1sG8f9Oqlpb3/PowZA+vXq7k990GRDZAQwhqYD/QGGgPPCyEamxDdLaV8OGv7v6KWW56pXbs2q1atIiEhQZ+Wnp7OTz/9RO3atY1k3dzccNf/A4pOYmIiixcvZsSIEWbJN2vWjPr16xe7ESxPFPaamcPFixepUaMG7dq1o3r16tjZ2RVb3kXlfupr6j4pb/fOxYswekBzMn/4nRZHdrF73UO479tsvExCs2aaA4JuTo+dndYKUi7W90VxtIACgAtSyotSylRgJdC/GPKtsDRv3pwGDRqwevVqfdqmTZtwcHCgQ4cORrKG3Su7du0iMDAQFxcX3NzcaNu2LWFhYUbymZmZTJ06FS8vL7y9vZk4cSKZmZn645s3b8bKyor27dvr0+bOnYsQItf24YdaQ/WJJ55gxYoVxX4ddOW6urqaLFdX/9DQUObPn68/fvnyZbZu3cpjjz2Gh4cHnp6e9OzZk9OG4U7QWk6vv/56vtfjfq9Zfr9FSkoKb775Jj4+Pjg4OBAYGMiePXuM6jRlyhSuXr2KEIK6devmWc+goCBGjRrF+PHj8fDwwMPDg3feeSdXHXLWO2dLKmc3XX76m6qv4e9VmPukpO6d4iIlXTMkp0/DY4/BlSsQWKc1u9Y/iNuP8+Dxx+Gll7QVSqWEAQPgm2805wNFkSkOJ4SawDWD/XCgrQm5R4UQx4AbwEQp5UlTmQkhRgIjAapVq5Zn37ibmxv37t0zSpu9bzaf/KV5p0wOnMzUdlONjk8NnUrw38EAzOo4izfavGF0fNz2cSw9sRSAL7t9ycvNXzY6PnzTcNaeXQvA4t6LeabRMxSWtLQ00tPTeeGFF/j2228ZNGgQAIsWLWLw4MFcunSJ9PR0fd108jExMfTv358hQ4awcOFC0tPTOXr0KMnJyXrZjIwMli9fzqhRo/j99985ceIEI0aMoHHjxjz99NMA/PHHH7Ro0YL4+Hi9Ti+++CIDBgzQ73/11VesXr2aAQMGcO/ePZo2bcrMmTOJjIzE0cR8hv/85z989tln+db7559/pl27dkZpunIzMjKwtrbOVS7AzJkzOX36NP7+/kybNg0Ad3d3oqKieO2112jSpAnJycl8+umn9O3bl0OHDulbE+Zcj/u5Zunp6fn+Fu+++y7r1q0jODiYevXqERwcTK9evThy5AjVq1dn5syZ+Pj4sHz5ckJCQrC2tsbW1tZkPXX6DR48mB07dhAWFsa4cePw9PTUGxndPWJ4H6Smphr9PwxlCtLf1D1yv/eJLi0+Ph4HB4dc/1kdycnJRRoHux/Whq9lY8RGxrosYuaUIO7etaN581g+mHKEey99hev69QBcfu45Lt+8CZGRVBk+nDr29pxt3Zo0M/WNj48v9bqVF4rDAJnq+Mw5GvcPUEdKGS+E6AOsBxqYykxKuQhYBNCwYUMZpFvIKQenT5+mSpUqRmn2+qBM2vecx+1s7fI9bqtbnx1wcHDI97ijo2Ou4+Zga2uLjY0Nw4cP5/333+fmzZtUqVKFHTt28PXXXzNlyhRsbGz0eevkpZTExsYycOBAWrRoAUDr1q2N8ra2tqZx48bMmTMHgFatWrFs2TL27dvH8OHDAYiIiKB27dpGuht+nzNnDj///DOhoaE0zJrb4OfnR1paGvfu3cPb2ztXncaPH8+QIUPyrXfNmjVzGS9duffu3WPBggW5ytXJODo64ubmhp+fnz79xRdfNMrrp59+wtXVldOnT+tbkeZcj/u5Znfu3Mnzt0hISGDJkiUsXrxYb8CWLFnC7t27+eGHH5g5cyZVqlTB1dUVGxsbozqZqqe1tTU1atTgm2++QQhB69atuXbtGvPnz2fKlClA9j2i08/a2ho7Ozuj39VQJj/9TdU35+8F5t8nurQ7d+7g6+ub53/GwcGBljkXaytBpu2cxvx/5wMw6fK7ZKb+Se/enqxdmIHT8NmwY4fWvTZjBnWrV6euLnJ1UBCMHo1XIcoKCQkhr+dYZac4DFA4YDhwUQutlaNHShln8H2zEGKBEMJLShlVDOWXSzw8PHjqqaf47rvvcHd3JygoiAceeCBPeU9PT4YNG0bPnj3p2rUrXbt25emnn841ZtS8eXOjfV9fXyIjI/X7SUlJ+Pj4mCzj448/Jjg4mJ07d+Lv769P1xmOpKSkPHXz9PTMv8L58Nlnn/Htt9/mKjc//v33Xz744AMOHDjA7du3yczMJDMzk6tXrxrJFXQ9zJHJec3y+y3+/fdf0tLSjLqvrK2tefTRRzl16pRZdctJYGAgwmCA+9FHH+WDDz4gLi4OV1fXQudX0L2U3z0ChbtPdGnJycmF1rMkaVmjJVZYkUkmmckuPPG4FWsnncC24xNaNANvb1i0CJ5/HtLSoGlTbTKQolgpjjGgQ0ADIUQ9IYQd8BywwVBACFFdZP2DhBABWeVG58qpiEwPmo6cJpHTJNODpuc6/lnPz/THJ7SbkOv4oscX6Y+PbD0y1/H/Dfyf/vjgZoOLrO/w4cP58ccf+e677/Rv2/nx/fffc+DAATp27MiGDRvw9/dn27ZtRjKGrTQAIYTReIGXlxcxMTG58p41axYLFiwgNDQ0lxG4c+cOoHWJmmL27Nm4uLjku+3evdvkubNmzWLx4sUmy82Pxx9/nNu3b7Nw4UIOHDjAkSNHsLGxIVUXgyuLgq6HOTKmrllev4XMcsU1NBiG+ZYGVlZWej10pKWlGe3ndy/ldY9A4e8TXVpRXlBKAnH2ScSmhXD2cV6Uv/PzcndsHW20lUsfeQQOH4b+/eHVV+GFF8CgVaooPopsgKSU6cBYYBtwGlgtpTwphBglhBiVJTYICMsaA/oKeE7m/IdUQrp27YqdnR1RUVE8+eSTZp3TokULJk2apG/W//DDD4Uqs2XLlrnexGfMmMHChQsJCQkx6v7RERYWhq+vb55vxaNGjeLo0aP5bm3atMl1nq7cTZs2mSxXh52dHRkZGfr96OhoTp8+zdSpU+nWrRuNGjXSj22UBKauGZj+Lfz8/LCzszNyOsjIyGD//v00bmzKOTSbnPXUceDAASOD8tdff+Hr65tn66datWpEREQYpR07dsws/fOr7/3cJ7o0L6/CdFqVLEuWaFF0Mg69wljPdfyw2ElbTK5RI21C6ZIloOtZ+PxzWLoU7qOlqSiYYpkHJKXcLKX0l1I+KKWclZX2jZTym6zvwVLKJlLKFlLKQCnlvuIot7wjhOD48eNcunTJaPzKFJcuXWLy5Mns27ePK1eusHPnTo4fP17gQy0nOm+x6Kw16mfNmsWXX37JypUrcXZ25ubNm9y8edOoy2T37t300s17MIGnpyd+fn75bjnHf8wpV0fdunU5ePAgly9fJioqCg8PD7y8vPj222+5cOECoaGhjBo1ChubkgnskfOa5fdbODs7M3r0aCZPnszmzZs5ffo0o0eP5tatW7z++uv5lpOznrpW2I0bN3jzzTc5e/Ysa9eu5dNPP+Wtt97KM58uXbqwZcsWNmzYwNmzZ3n77be5di3bT6igeylnfeH+75OC7p3S4Hrcdd7Y/AYp6anMmgWvvAIZGTDr7Wi+OtsLq28XaoJLl8LEiTBihNbtBlrEA0WJoULxWBhzHRmcnJw4d+4cTz/9NFFRUfj4+PDCCy8wadKkQpXXrFkzAgICWLlyJa+//jpz584lLi4ul8vtjh076Nq1K8nJyaxbty5XV19RkFIWWK4hEydO5KWXXqJx48YkJSVx6dIlVq1axbhx42jatCl+fn589tlnDBw4sNh0NMTwmo0ZM6bA30Ln0PDyyy8TGxtLy5Yt2bp1KzVq1Mi3HFP1BHjhhRfIyMigbdu2CCEYMWJEvgZo+PDhHD9+XN+t+/rrr/PUU08RlbUgWkH656yvOb+XqfukJO6dwnIu+hw9furBlbtX2LY7mvNzliGEFSsmH+PZlU/BpUtw5owWUPTJJ+Hjj7VxH2V4SgcpZZnd/P39ZV6cOnUqz2Plmbi4uBIvY8uWLdLf31+mp6cXKBscHCy7d+9eovqURp2LSmGumTmYW+dOnTrJMWPGFEuZhaGw9TV1nximnTlzRoaFheV5fkn9n7/Y/4VkOtr2gY20qXNQHnr9OykdHaUEKR94QMrLl7NPSEkpdh127txZ7HmWdYDD0oxnvIoFVwnp1asXY8aMITw8vEBZW1tb5s2bVwpalW0Kc80qAoWtr6n7pCzcO0MbjqPG9ZGQ6oTH+jVce2g+bRYMh6QkqF4drl6FDQY+U2UoIkVlQHXBVVLGjRtnltzIkbm9ASsr5l6zikJh6mvqPrH0vXPxIvTrJ4g4O59qjd7g7AOz8diyQlsc7uuvNceCqVMhIMCielZmlAFSKMowaga9+aRmpLIybCVDmg9h3z7Bk09CVBQ0aWLDpk1N8fjnaTh/SFvJtGlT7aR+/SCHG76i9FBdcAqFotxzJ+kOvZb14qX1LzE4+D906QJJUfEEN5rPvr2SOgmntKWzb90ydqlWxseiqBaQQqEo9/x3/3/ZeXknACvvvMtDXt6EJszC+/R5WGkDI0dqgUXr1oUyNCepsqMMkEKhKPe83eYDvtkWQpTTHjr+0ZM/IoZjIzPhoYegQwdtrZ61a5V7dRmjXBsgKWWphTdRKBQlgyxiUJTz52HAADtsLs1j1gNDmXraYN5Rs2bQpIn2XRmfMke5HQOytrbOFd9KoVCUP5KSknLF48uP+NR4fj3zKwC//QZt2oBN2BFOJ3Vh6ukTWiDRpUuhd2/4v0q99mWZp9y2gNzd3bl16xY1a9bEyqrc2lGFotIipSQpKYnr16/nG33bkHPR5xi0ehBhkWE8m7GRlTP6YEsqExtvwfW6hPZ94LvvwMdHW0hOUaYptwbIy8uL8PBwzp49a2lVipXk5GQcHBwsrUapoupc8bl586Y28z3Hy6KtrS0+Pj5mLysx5Y8pnIg8AcDatKfxdP6b/xtXhcFfzUYkJGgtHjONmcLylFsDZGVlle/6OeWVkJCQUl2Yqyyg6lzxGT16NLGxsRw9erRI+bzo9g0b4vcjHG7x9bZE+gZ8RPXZK8A/GGrUgByLNCrKNuXWACkUispDWhpM+1By/ZMt/F4zBveMTFrE2GP1344gJQwbZmkVFfeBMkAKhaJM8uuZX8mUmTSzfYp3B1zg9ROj6MYfcD1L4KXnYPRoi+qoKBrKACkUijJFYloib297m4V/L8RBuNBy4Tf8cWMEjqSQ5lYV2/9+qi0Y162bpVVVFBFlgBQKRZljx4UQAJJlPK6dh8AqZ1KfHIjd11+qSAYVCOW/rFAoygxSwsavohn1+QOIDBtszw0k+GBTHIY9j92385XxqWCoFpBCobAoMUkxeDh6cP54Etd6jmDgzVVYk0mTZQ/RLHQ1tbzToIAl6xXlE9UCUigUFiEjM4PP9n1G7f/W5v+GzsKmRWO63FyBNZnceaAFvUJ+p1ZtK2V8KjDKACkUCovwzvZ3mbh9IglpCaxwfZ8aNpe5VaU+ia+Ow/PCIcQDtS2toqKEUQZIoVCUOocPg+dEe1xStH3rDBsOvjUTnztncVr0pVqnp5JQLAZICNFLCHFWCHFBCDHZxHEhhPgq6/hxIUSr4ihXoVCUL678+S+/P/QGQY/EsybsWT7eZscb51pxcEYkHee+BzZqWLoyUeRfWwhhDcwHugPhwCEhxAYp5SkDsd5Ag6ytLfB11me+pKVZce0aWFlpkdRNfeZME0LbFApF2eDa3WucCD+Ke2QCtbs2oA6Sd629iH97GkPG3MKtjrulVVRYiOJ43QgALkgpLwIIIVYC/QFDA9Qf+FFqC3/8JYRwF0LUkFJG5JfxpUvhPPBA0H0rZmiMDD8LSsv5PedmZVW4dHPO0X0mJsbj6uqiN666TSej2yoSsbGxuLu7W1qNUqWy1Dny9i3OxJ9B3oIYoKOARLtq2D+0GfuDO+l/0NIaljyV5be+H4rDANUErhnsh5O7dWNKpiaQywAJIUYCI7U9Z2xtM9HWqxLo1q3K+Wl4zBApMZletnExS0ozYDLLMEn9vmasZJahkllbQd/zOi5LpTWZkZFBbGxsyRdUhqjodU6JScfh+nU8M2OxqwkpgASu1vbG06MGSaSQFJtiaTVLhYr+WxeF4jBAph5ROR/75shoiVIuAhYBNGzYUJ49u6tQykgJmZmQkaF9Gn7P6zOvY7otLQ3S07XPonzP63haGqSmQnIy3LwZg6OjBykpkJKipRl+6jZD45qRUahLVCgcHcHVtXCbmxt4emZvjo75lxESEkJQUFDJVaIMUhHrHH9PsvK7WBZ/70jI1eq4c5d0rPikYSdm2/1FTceanA87b2k1S52K+FsXhLkrVReHAQoHDP0lawE37kMmF7Fpsaw4sYKGXg1pVcM8vwUhtLGg8rr6bkjIsQJvVimzDVZehiopCRIStC0xsfDf4+Ph3j0tn6QkuHXr/utkb29skDw8jPejony5eVP7XrWqtqBltWpQiZbLKbdICQd3JvD3R6NZXucX3G7V5sCx0yx1GE2Hh6KoN3ME7/Vpy/bOQdyNvWtpdRVljOIwQIeABkKIemhxap8DBueQ2QCMzRofagvcLWj8ByAyOZLBvwxm+EV3lpxvpL1au7mBuzvzhzflg50f4ubgxmsuQUz2ejL7uJsbuxNOcfDuadycPAioGUBzn+bFUNWygRDaQ72k5+dJqRmjuLjCbbGxEBMDd+5AdLRmECMitM00/iZTq1TRDJG3d7ZRyuuzWjWwsyupK6EwREo4fRp2BJ/BZflCHnaYx5jXtWa4XZ0zfDnoLCOHfGz0AiFMdoIoKjtFNkBSynQhxFhgG2ANfCelPCmEGJV1/BtgM9AHuAAkAi8Xpgy3W7Gwf392gpMT0c9PIiY5hpjkGOL3/gZrlhqds7krfPKY9n1m55maATp1Ct59Fzw8mFTzNN85nqaqlQvvew3gxTqPQ8eO4KKNwey+vIsb8RFUdapKM+9m+LhUvlUWhQBnZ22rUeP+8pBSa0HpDJJuM9wPC7uBo6Mv0dGawYqMhNu3tRbYvXtw8aJ5ZXl4aHrWqAHVq2d/N9yqV9e6CZWnZOFIT9f+gr/+ChErQnj1xjTGoXWPyzgIvCb4q7Yk3daKKm324eDQ0MIaK8oDxeJ0L6XcjGZkDNO+MfgugTGFzdfNxpWedTvSuk17eL8D3L2rbamp3E0+oZdzr/sQ9PPIPn73LrFuEUCqlo+DmyZ49Sps2gRAZH+IaglRmYmkfPsNHPkGLlzQG6CvvxrCCrerAPz0Tx1eTGyQ3X/UsiUT653nyM0jeDtW452qj9OqQUejfqOIexE42jriZu9mdn9oRUQIcHLStpo1TcuEhJwjKMjXKE1K7ae8fTvbIEVGGn83/Lx9WzNqMTHae0Z+ODqaNkw507y8Kp7HoblkZMDJk/Dnn/DHDsnpkFv8m+jNM37DeCdjBa1IJ8XGiageL1B90ku873WbJSd+ZFaXWTSq1sjS6ivKCWV61pePY3VWvfSbyWOfZKQx5bEpxCbH4u7gDk7GUXK7n/4Fxyu7iU2KoZl3My2xVSvYsAHu3OFO+KeQfhIAzzYdwNvZKNJudGaC/nvVU1fgwpXszB9/nEMD7rLrivYG+MoPq+BS1jFnZ6hWjYH9b7PfIwFbYcOu4bsJrBUIJ07ApUtQrRpL43aR4exE9Wr16Fi3E1XsqxTtYlUwhAB3d21r0KBg+cxMrfWk6+rTbTdv5k5LTNRaVQW1rKytwcdHM055bTrj5WKe82KZJD0d/v1XMzh//w1//QWHDkHVe5d4iaV8znIO1Elh+EAHVrteIOksrE94Evsfl1LTTXu56wv0bfykReuhKH+UaQOUH7bWtng5eeHlZDo8+4BGAxjQaIBxorc3PP44AGsynudO0h2iE6Op5VoLdK2kLIIGvE2Vq39xJ/42tb4ZB+me2f1GtWtz++qk7Gxr+UNKvPYanjWSH5nloZYm0zUDCfDjj/Cf/wAw7U24mpV8bkU1qtRvA5uzG5FD5gRia+dADbdaTG7zJlVqP6g9jStxayo/rKyyx4Ka5zPcJ6XWrZeXcTJMv3MHbtzQtoJwdjZtoLy8NMcKnYOF7ntpTw9ISoLr1+HaNQgPh8uXtZbiqVNw5ozm1ALwEKcZwC/M5Rda84/+fGnjRbqrthTpbw3hxGvTaeHmZqIkhcJ8yq0BKip21nZUd6lOdZfqJo9P6Tg13/NXR9bjxr0bRCZEUndyf7Cvoj1V4uIgKgrn9b1wTgwnITMZb2dv7aSGDaFPH+TtSCJd/kbniV7j0m1widTnnZmRzsqEA6QnA3HwwWvLIR1tlN3Hh4zZs/C+8SbVnKrhK1zZkfkiVjV89a/rmT7eXMuIoYarL3bWamTeECGy3cUbFjBMkZKief/dvJn/FhGhvXf8+6+2mYOtbUe9cXJzyx5rc3Ex/u7oqEWnsbHRWmS6z8xMzWjoXPjT0jQPyLt3NScQ3WdsrKZjVFT++jzwALzg/TsTTvZkVVPwPwYIe+0itG2L/5499Fn9JHuv7WX4w8Op5lzNvIoqFPlQaQ1QUWnq3ZSm3k2NE4XQe+Edm6DNd0hMS8TRJmsizCuvwCuvkJGZztgdU4hMjCQmPgqX819rf/Qsou9GkJ7lRu6eZoND/fraUyQuDq5d43bmPe4k3eFO0h2ihDNW08YbqRHpAnUnat8beTXi1JisQZElS0BK4qu58Zf9bWrVbkLNOs2o4uJZ7NenImBvrz2YH3ggfznde4cpwxQdne0NqPvUPAOtCvAMLF5sbbUxuFq1oJlXBJ1Sf+eRqK24elpjv3oZVbau4Z2fhlCjN6RZg8vo8Qx9ZhYcOwaPPgpC8HXfr/F09MTZzrl0lFZUeJQBKmGcbJ1ypdlY2fBpj0/zPMelihcbnttARHwEqRmpMHOsdiBrQs7NtGv6MafqjtVg1BDtiZf1uh7ODbS551pXpZ7p0yE8nFM1ofurwAFocROO/lwN3nlH24BrJ/awefOX1KxaDz/fpjz0YFvlPpYPBu8dBbaqdGzduoumTTsSHa0Zr4QEbe6Vbi6W7ntiYvaE6PT07E9ra82o2NpqDWNbW81gZs1SMNp8rKOoFrYTq727ITQU9hxHkjU73NERbJOhfn2q3UkhLevF50frMIY6O0O7dnqda7up5REUxYsyQGUQR1tHHm/4uIkDjlC3Li1kHaLeiSIiPoKU9BTwbW0kFn9pJzV+HszNhFva+JaO4cPh6lWupxwDjgBQKw5t7MqAQwfXMyp5LVyHfn/CbysMyq9enUO/LWTtjR3Ucq1FmzNxPJrmYzwi7+2twukXgINDJrVqaS2SYiUtTfMmcHEBPz8tbdEv8NprxNnDmsbw62CBc4pkxaVWsHKlZrlateKFL/9kyqZuPOL7CIMaD0JKWak9OBUljzJA5RAhBFWdqlLVqarJ40H1OnNjYgRpGWkkpGV78/HRRwA4XdjGY7tncf3edeoP7g2fvW80q/W6a/ZDp6ZzdXjQOdt97PJl9t8NY+6+uQCMvurDo98Zh0lY0wTWPmxPzTpN6TtiDl3rd9UGIZYtMzZUulaV4v64d0+bERoWprmvHT6sdZmlpMCbb8J//5sdGqNVK+L6d+YV+RkgcUyDRLv6OPn56Vu1NVt35kqDK8YvLQpFCaIMUAXG1toWd2v3XOk9/XrS069nnuc91KoHw+3uEH4vnFb9BsK3WbFh4+Ph1i3CLy7Uy9Z8sCW8XMPIfexgzVusbpAC/I3X9QOaATp3Dt56C4CZHeHXhlrra/RxOx6LrQo7dkDjxgBkbtmM1fUbxsbKx6dyrhWTlqa5rl26pPmNv/xy9nXo1Qv27TMSX9UEQpu7csh9NTuSp+O2ZgOMGwddulDrw//w0PxNnIk6Q5It7P6/V+iZo4WjjI+iNKmE/2hFQXR/sDvdH+ye+4CLC7i40Ne6L672roTHhRPY+Gmo39VI7Pra5+HkSsDggVa1KowdCzdvEuYVwuHqURyuCQNOp+IYEaHF3cli8O+vscMunFp/w+fboMsltLf0qlWhTx+uz5uNo60jHpn2iPffz/Zt1vk6677XqFF2A8rpZtrevKm1PuvW1dLPnYOZMzWf6UuXtMnThtFmO3YksX5tTt8+jW9rf2rEx0OjRtocN3t7gq9OZ49rLBBH6JVQnujbFwICoF8/kJKxj4wlKT2JJx96Ej9Pv9Kvt0JhgDJAikLTqW4nOtXtlOfxd9q/S1//foTHhdO2ZtbKHA0bwrx5AFz//jG4ugeAWis2ceBIDG19syMhhFezJzoNop1APPggJMZrIQ+ioiAhgWfWPsO+a/uwETbsXZNOwHXj8n9ooQ2we7/7EY8NmqB5bc2dC19/rXX5VamSvTk6gq8vfPxxdgaffaY99B0cNONg2PJ65JHsiUYXL0JIiHGY8pQUzS86JQVmz84eCxs1Cg4e1Efq6Bgbm21YXnkFvv1W+56UBD/9RJw9pFlB1UyhDRTVqwf16/PO0bl8tvJ7JJJ5XaYwdvwv8OCD2rmbNhGwOZY9WX4Df1z8gycaPgEHDujVHxNQ6IAkCkWJoQyQothpWaMlLWu0zPP4yoEruXL3CuFx4TSv25YTkSeMwpdHuEht9TKg1sotULWB5voVFQVSEr4qEIB0mY7n+MkQa5Xt53znDlPb7OKGYxqcn8blxJc0AxQRoc2+BNqNAJtMcE+GlSvAqY6f3gBlZGawetX7OMUl45wG3XJES8iY8wn3/B/ASlhh/dcunEeMMDqeKSDJBlJsgHfG4OldVztw7hwcOcItZ9j7ANytDd7p9vRNqqWFd8rip+QDvP6RPfEyhfH+Q/niqYVGrTifff9BntHmj4Ut+RjqxcNXX2kHu3XjqUvv4FHfirYNu/JIzUfM+bkUCouhDJCi1KnpWpOarnkEhgPOjT1HZEIk4XHh1HGvoyXa2ED16kgpcbFzwdXelbiUOHzGTdUmAWchpSRyph1kavv6ScAffQRvvEHG3Rj2b2ijl7dfuAScs+PoJKQlMLhvMgAumbbcixhm1AV21d+H+nM0g1HXsQaXhg3LDk1ub89h+9u0tVkKwCPrBnHwtcPaiV98ASkp/JNwlIGh2phajwc70ffFbUZ1d6pSlXipudBfJEYzPqtXQ3AwvPEGTVs2xUpY0cCxFtVEtLETh709HcbOpUOeV1ahKFsoA6Qoc1hbWVOjSg1qVMkdglsIwcnXtRh+yenJ2Fsbr0mRlpnGGwFvcCvhFnEpcTjaZk0Czgp/cDfpjrY4COBm74b1y8ONzk9MS9R/d6riAR8tMjqeeedfOKZ9t3JwhO+/Nzpud/MoLFwKQIpMyz6Q1W3ndi0NQrWku8m518fxCdeafnZWdkjdmo1Xr8Lu3VCvHl0GfEvC1AQcrOxgApU3WqqiQlCmDVCV8+dh4kR9/DQSErRxhDp14PnnswWlVBMkKyEONrkdDOys7fi85+d5nuNq78rR144SkxxDcnpyruPWwprnmj5HYloiVexMB4h1s3cjQ2aYnGTsYOOAo40j9jb2Js/3reJL/4b9SYpNotMDAbB1K3Trph9nCliylTsbwP2LrxDPv6adNHCgNgbUtasKraSoUJRpA4SUxgPA167BlCnaBDtDA9SqlTY+sHNn9uS7nTu1fvdOneChh0pXb0WZxcbKhhbVW+R5vJpzNVYMXJHn8Qc9HyR2cmyexx/yeojE9xJzH0hPh6tXqVu/PuufW68t0/zKK/DvTDhyBB5+GAC7gc9g55kjomq9etqmUFQwynT7Pd7PDyZPzk5wcdFaREOHGgtevaqF+DWMzrtiheZ5tHNndtr27dos/ZEjjc+fPx/WrtUiPCoURSUpSbsndSQmavdm06bavB4dnTtroW4SDQzWM89o3nqPPlp6+ioUFqJMt4CklZUWzEpHrVrwqYkYateva/MpqhpEBujYUTMoWW+Wernbt7UHhI5797T5KY6OWleHjoEDtZAmP/wAbbNciY8d09b0ad1am3uhqNykpWnhr+3soH59Le2vv6BDB+0e0bk/Ozlprt7p6dqLkq41o3O9VigqKWW6BWQ2Dg7aRD7DAdkXX4TFi43fJIcM0RZ3mTMnOy09HUaP1lpVhuNI587B2bPaw0XHunVaHv/7X3ZaWJjWqurf31inefNg4UJt3EpHamrpLwSjKB6OHdNeRi5dyk4LDtZeRD77LDvN31/7lNL4tz5+XDtXdaUpFHoqhgEyF2trbXa8waRHPDxgwQL45htj2V27NONi2NJp0gSefRZaGsxxuXVLa1XFxRmf/957WhegYZfLyJGasVxhMMZw5Ai8+y788kt2Wmam1tIyZyU0RdFIS8u9WM7UqdoLhWHX2Ny5MGyYNvFUR5Mm2ouPoSu0p6cWsujgQeMXGkfHElBeoSjflOkuOIvi4WE0QRCAp5/WNkOCgrTuP4P1fMjMhDfe0AyT4bjU3btaK8gg7Ax//611K778MgzIWsH1zh1tENrDQ/uuY/hwbTJlcLA+bhoHD8LRo1o3YYuswfWUFK1sd/fyvVb0/ZKerk1MdXXNfvD/8w9s2aI5rPTuraWdPAnNmmnXMiws+/w1a+DCBa3F0qSJlta5s9aiqW2wJEGPHsYtIh1lNfyPQlHGqFwtoJLA2loLlGm4apmVFcyaBYsWGb8Fr1undcn16JGd9sgj2ix8nfEBbYyqcePc3nsHDmhOFYbOEuvXw2uvwW+/ZacdPao9KDt3Nj6/c2do397YqP30E7z9thZJWcfly1rk6v37s9MyM7XW2smTxnlGR2vGzrCll5mp7RvGMMvMzF7gxpC4OGzv3NGMhmH5e/ZoY3Y6rl7VnEV+/tk4z4EDoWdP4+6uvn21AKaGrZVDh+D9941bmj4+2qehngD/93+aEappMFn2lVe0rtdu3VAoFMVDkQyQEMJTCLFdCHE+69MjD7nLQogTQoijQojDpmQqDU5OxuNKLVponn79+mWn1a6tPehzRDpm1SotarRuwBugTRsYMUIb9NaRnq51M1bPsdz4gQNangZLL7Blixa2/9y57LS//tLGunQhXkAzHq1aZTtk6Bg4UBsD27MnO+2zz7Q6TpqUnXb2rNYaa9XK+PxWrWg/cKCx19jnn8Njj2meiYbnjx2rdZfqsLLS9P/9d2PD5u0NXl7GrdI2bbTr3LdvdlrVqprM6dPGOj3/PAwaZOwAo1Aoip2idsFNBv6QUn4ihJictT8pD9nOUsoCVqZX5EvTptpmyIABxq0n0Fo513NE6ATNsNy9qxlBHUOHag9nQ8NQu7bmxBEYmJ0mpeZRmHMsw9VVe9gbpltba5uhU4i1tSaTs3vKzY1UDw/sDFshfn6a84i3d3ZavXramJquS0yHbkE1Q6P+44+5Jya3bm1spEGTUQvnKRQWQ8gieGUJIc4CQVLKCCFEDSBESplrUWIhxGWgTWENUMOGDeXZs2fvW7/ySEhICEFBQZZWo1RRda74BAUFERsby9GjRy2tSqlT2X5rACHE31LKNgXJFbUF5COljADIMkLeechJ4HchhAQWSikX5SGHEGIkMBKgWrVqhBj241cC4uPjVZ0rAZWtzrGxsWRkZFSqOuuobL91YSjQAAkhdgDVTRx6rxDltJdS3sgyUNuFEGeklLtMCWYZp0WgtYAq25tDZXxbUnWu+Li7uxMbG1up6qyjsv3WhaFAAySlzNPtRwhxSwhRw6ALLjKPPG5kfUYKIdYBAYBJA6RQKBSKykFR3bA3AC9lfX8J+DWngBDCWQhRRfcd6AGE5ZRTKBQKReWiqAboE6C7EOI80D1rHyGErxBic5aMD7BHCHEMOAhsklJuLWK5CoVCoSjnFMkJQUoZDXQ1kX4D6JP1/SKQd/x7hUKhUFRKVCQEhUKhUFgEZYAUCoVCYRGUAVIoFAqFRVAGSKFQKBQWQRkghUKhUFgEZYAUCoVCYRGUAVIoFAqFRVAGSKFQKBQWQRkghUKhUFgEZYAUCoVCYRGUAVIoFAqFRVAGSKFQKBQWQRkghUKhUFgEZYAUCoVCYRGUAVIoFAqFRVAGSKFQKBQWQRkghUKhUFgEZYAUCoVCYRGUAVIoFAqFRVAGSKFQKBQWQRkghUKhUFgEZYAUCoVCYRGKZICEEE8LIU4KITKFEG3ykeslhDgrhLgghJhclDIVCoVCUTEoagsoDBgA7MpLQAhhDcwHegONgeeFEI2LWK5CoVAoyjk2RTlZSnkaQAiRn1gAcEFKeTFLdiXQHzhVlLIVCoVCUb4pkgEyk5rANYP9cKBtXsJCiJHAyKzdFCFEWAnqVhbxAqIsrUQpo+pcOfASQlS2OkPl/K0bmiNUoAESQuwAqps49J6U8lczyjDVPJJ5CUspFwGLsso+LKXMc2ypIqLqXDlQda48VMZ6CyEOmyNXoAGSUnYroi7hQG2D/VrAjSLmqVAoFIpyTmm4YR8CGggh6gkh7IDngA2lUK5CoVAoyjBFdcN+SggRDjwKbBJCbMtK9xVCbAaQUqYDY4FtwGlgtZTypJlFLCqKfuUUVefKgapz5aEy1tusOgsp8xyOUSgUCoWixFCREBQKhUJhEZQBUigUCoVFKBcGSAgxUQghhRBeltalNBBCzBBCHBdCHBVC/C6E8LW0TiWNEOJTIcSZrHqvE0K4W1qnksbcUFYVgcoYjksI8Z0QIrKyzGUUQtQWQuwUQpzOuq/HF3ROmTdAQojaQHfgqqV1KUU+lVI2l1I+DGwEPrSwPqXBdqCplLI5cA6YYmF9SoMCQ1lVBCpxOK6lQC9LK1GKpAMTpJSNgEBgTEG/c5k3QMB/gXfJZ/JqRUNKGWew60wlqLuU8vcsj0mAv9Dmi1VopJSnpZRnLa1HKaAPxyWlTAV04bgqNFLKXcAdS+tRWkgpI6SU/2R9v4fm9Vwzv3NKIxTPfSOEeAK4LqU8VkC8uQqHEGIWMBS4C3S2sDqlzXBglaWVUBQbhQrHpSj/CCHqAi2BA/nJWdwA5RfqB5gK9ChdjUqHgkIcSSnfA94TQkxBm0c1rVQVLAHMCeskhHgPrSm/vDR1KymKIZRVRaBQ4bgU5RshhAvwM/Bmjt6cXFjcAOUV6kcI0QyoB+haP7WAf4QQAVLKm6WoYolQiBBH/wM2UQEMUEF1FkK8BPQDusoKMkGtGEJZVQRUOK5KghDCFs34LJdS/lKQvMUNUF5IKU8A3rp9IcRloI2UssJHlRVCNJBSns/afQI4Y0l9SgMhRC9gEtBJSploaX0UxYo+HBdwHS0c12DLqqQoboTWUlgCnJZSfm7OOeXBCaEy8okQIkwIcRytC7JAd8YKQDBQBdie5X7+jaUVKmnyCmVV0ShiOK5yixBiBbAfaCiECBdCjLC0TiVMe2AI0CXrP3xUCNEnvxNUKB6FQqFQWATVAlIoFAqFRVAGSKFQKBQWQRkghUKhUFgEZYAUCoVCYRGUAVIoFAqFRVAGSKFQKBQWQRkghUKhUFgEZYAUCoVCYRGUAVIoSgkhxLtZCyvm3P7P0ropFJZARUJQKEoJIUQVtPWddEwEXgAek1JesIxWCoXlUAZIobAAQohJwDigSyVZlE6hyEWZjYatUFRUDNZ46iylPGdpfRQKS6EMkEJRimQtuDcKbdkJ1e2mqNQoA6RQlBJCiA+AV4EgKeW/ltZHobA0ygApFKVAVstnPNoCgwlCCN0y3bFSymTLaaZQWA7lhKBQlDBZK0XGAq4mDneTUv5RuhopFGUDZYAUCoVCYRHURFSFQqFQWARlgBQKhUJhEZQBUigUCoVFUAZIoVAoFBZBGSCFQqFQWARlgBQKhUJhEZQBUigUCoVFUAZIoVAoFBbh/wGYvos0Dz25tAAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# extra code – this cell generates and saves Figure 11–4\n",
"\n",
"def swish(z, beta=1):\n",
" return z * sigmoid(beta * z)\n",
"\n",
"def approx_gelu(z):\n",
" return swish(z, beta=1.702)\n",
"\n",
"def softplus(z):\n",
" return np.log(1 + np.exp(z))\n",
"\n",
"def mish(z):\n",
" return z * np.tanh(softplus(z))\n",
"\n",
"z = np.linspace(-4, 2, 200)\n",
"\n",
"beta = 0.6\n",
"plt.plot(z, approx_gelu(z), \"b-\", linewidth=2,\n",
" label=r\"GELU$(z) = z\\,\\Phi(z)$\")\n",
"plt.plot(z, swish(z), \"r--\", linewidth=2,\n",
" label=r\"Swish$(z) = z\\,\\sigma(z)$\")\n",
"plt.plot(z, swish(z, beta), \"r:\", linewidth=2,\n",
" label=fr\"Swish$_{{\\beta={beta}}}(z)=z\\,\\sigma({beta}\\,z)$\")\n",
"plt.plot(z, mish(z), \"g:\", linewidth=3,\n",
" label=fr\"Mish$(z) = z\\,\\tanh($softplus$(z))$\")\n",
"plt.plot([-4, 2], [0, 0], 'k-')\n",
"plt.plot([0, 0], [-2.2, 3.2], 'k-')\n",
"plt.grid(True)\n",
"plt.axis([-4, 2, -1, 2])\n",
"plt.gca().set_aspect(\"equal\")\n",
"plt.xlabel(\"$z$\")\n",
"plt.legend(loc=\"upper left\")\n",
"\n",
"save_fig(\"gelu_swish_mish_plot\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Batch Normalization"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"# extra code - clear the name counters and set the random seed\n",
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(42)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Dense(300, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Dense(10, activation=\"softmax\")\n",
"])"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"sequential\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"flatten (Flatten) (None, 784) 0 \n",
"_________________________________________________________________\n",
"batch_normalization (BatchNo (None, 784) 3136 \n",
"_________________________________________________________________\n",
"dense (Dense) (None, 300) 235500 \n",
"_________________________________________________________________\n",
"batch_normalization_1 (Batch (None, 300) 1200 \n",
"_________________________________________________________________\n",
"dense_1 (Dense) (None, 100) 30100 \n",
"_________________________________________________________________\n",
"batch_normalization_2 (Batch (None, 100) 400 \n",
"_________________________________________________________________\n",
"dense_2 (Dense) (None, 10) 1010 \n",
"=================================================================\n",
"Total params: 271,346\n",
"Trainable params: 268,978\n",
"Non-trainable params: 2,368\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[('batch_normalization/gamma:0', True),\n",
" ('batch_normalization/beta:0', True),\n",
" ('batch_normalization/moving_mean:0', False),\n",
" ('batch_normalization/moving_variance:0', False)]"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"[(var.name, var.trainable) for var in model.layers[1].variables]"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/2\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.5559 - accuracy: 0.8094 - val_loss: 0.4016 - val_accuracy: 0.8558\n",
"Epoch 2/2\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.4083 - accuracy: 0.8561 - val_loss: 0.3676 - val_accuracy: 0.8650\n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# extra code – just show that the model works! 😊\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=\"sgd\",\n",
" metrics=\"accuracy\")\n",
"model.fit(X_train, y_train, epochs=2, validation_data=(X_valid, y_valid))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Sometimes applying BN before the activation function works better (there's a debate on this topic). Moreover, the layer before a `BatchNormalization` layer does not need to have bias terms, since the `BatchNormalization` layer some as well, it would be a waste of parameters, so you can set `use_bias=False` when creating those layers:"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"# extra code - clear the name counters and set the random seed\n",
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(42)"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
" tf.keras.layers.Dense(300, kernel_initializer=\"he_normal\", use_bias=False),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation(\"relu\"),\n",
" tf.keras.layers.Dense(100, kernel_initializer=\"he_normal\", use_bias=False),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation(\"relu\"),\n",
" tf.keras.layers.Dense(10, activation=\"softmax\")\n",
"])"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/2\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.6063 - accuracy: 0.7993 - val_loss: 0.4296 - val_accuracy: 0.8418\n",
"Epoch 2/2\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4275 - accuracy: 0.8500 - val_loss: 0.3752 - val_accuracy: 0.8646\n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# extra code – just show that the model works! 😊\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=\"sgd\",\n",
" metrics=\"accuracy\")\n",
"model.fit(X_train, y_train, epochs=2, validation_data=(X_valid, y_valid))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Gradient Clipping"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"All `tf.keras.optimizers` accept `clipnorm` or `clipvalue` arguments:"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.SGD(clipvalue=1.0)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer)"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.SGD(clipnorm=1.0)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Reusing Pretrained Layers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Reusing a Keras model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's split the fashion MNIST training set in two:\n",
"* `X_train_A`: all images of all items except for T-shirts/tops and pullovers (classes 0 and 2).\n",
"* `X_train_B`: a much smaller training set of just the first 200 images of T-shirts/tops and pullovers.\n",
"\n",
"The validation set and the test set are also split this way, but without restricting the number of images.\n",
"\n",
"We will train a model on set A (classification task with 8 classes), and try to reuse it to tackle set B (binary classification). We hope to transfer a little bit of knowledge from task A to task B, since classes in set A (trousers, dresses, coats, sandals, shirts, sneakers, bags, and ankle boots) are somewhat similar to classes in set B (T-shirts/tops and pullovers). However, since we are using `Dense` layers, only patterns that occur at the same location can be reused (in contrast, convolutional layers will transfer much better, since learned patterns can be detected anywhere on the image, as we will see in the chapter 14)."
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/20\n",
"1376/1376 [==============================] - 1s 908us/step - loss: 1.1385 - accuracy: 0.6260 - val_loss: 0.7101 - val_accuracy: 0.7603\n",
"Epoch 2/20\n",
"1376/1376 [==============================] - 1s 869us/step - loss: 0.6221 - accuracy: 0.7911 - val_loss: 0.5293 - val_accuracy: 0.8315\n",
"Epoch 3/20\n",
"1376/1376 [==============================] - 1s 852us/step - loss: 0.5016 - accuracy: 0.8394 - val_loss: 0.4515 - val_accuracy: 0.8581\n",
"Epoch 4/20\n",
"1376/1376 [==============================] - 1s 852us/step - loss: 0.4381 - accuracy: 0.8583 - val_loss: 0.4055 - val_accuracy: 0.8669\n",
"Epoch 5/20\n",
"1376/1376 [==============================] - 1s 844us/step - loss: 0.3979 - accuracy: 0.8692 - val_loss: 0.3748 - val_accuracy: 0.8706\n",
"Epoch 6/20\n",
"1376/1376 [==============================] - 1s 882us/step - loss: 0.3693 - accuracy: 0.8782 - val_loss: 0.3538 - val_accuracy: 0.8787\n",
"Epoch 7/20\n",
"1376/1376 [==============================] - 1s 863us/step - loss: 0.3487 - accuracy: 0.8825 - val_loss: 0.3376 - val_accuracy: 0.8834\n",
"Epoch 8/20\n",
"1376/1376 [==============================] - 2s 1ms/step - loss: 0.3324 - accuracy: 0.8879 - val_loss: 0.3315 - val_accuracy: 0.8847\n",
"Epoch 9/20\n",
"1376/1376 [==============================] - 1s 1ms/step - loss: 0.3198 - accuracy: 0.8920 - val_loss: 0.3174 - val_accuracy: 0.8879\n",
"Epoch 10/20\n",
"1376/1376 [==============================] - 2s 1ms/step - loss: 0.3088 - accuracy: 0.8947 - val_loss: 0.3118 - val_accuracy: 0.8904\n",
"Epoch 11/20\n",
"1376/1376 [==============================] - 1s 1ms/step - loss: 0.2994 - accuracy: 0.8979 - val_loss: 0.3039 - val_accuracy: 0.8925\n",
"Epoch 12/20\n",
"1376/1376 [==============================] - 1s 837us/step - loss: 0.2918 - accuracy: 0.8999 - val_loss: 0.2998 - val_accuracy: 0.8952\n",
"Epoch 13/20\n",
"1376/1376 [==============================] - 1s 840us/step - loss: 0.2852 - accuracy: 0.9016 - val_loss: 0.2932 - val_accuracy: 0.8980\n",
"Epoch 14/20\n",
"1376/1376 [==============================] - 1s 799us/step - loss: 0.2788 - accuracy: 0.9034 - val_loss: 0.2865 - val_accuracy: 0.8990\n",
"Epoch 15/20\n",
"1376/1376 [==============================] - 1s 922us/step - loss: 0.2736 - accuracy: 0.9052 - val_loss: 0.2824 - val_accuracy: 0.9015\n",
"Epoch 16/20\n",
"1376/1376 [==============================] - 1s 835us/step - loss: 0.2686 - accuracy: 0.9068 - val_loss: 0.2796 - val_accuracy: 0.9015\n",
"Epoch 17/20\n",
"1376/1376 [==============================] - 1s 863us/step - loss: 0.2641 - accuracy: 0.9085 - val_loss: 0.2748 - val_accuracy: 0.9015\n",
"Epoch 18/20\n",
"1376/1376 [==============================] - 1s 913us/step - loss: 0.2596 - accuracy: 0.9101 - val_loss: 0.2729 - val_accuracy: 0.9037\n",
"Epoch 19/20\n",
"1376/1376 [==============================] - 1s 909us/step - loss: 0.2558 - accuracy: 0.9119 - val_loss: 0.2715 - val_accuracy: 0.9040\n",
"Epoch 20/20\n",
"1376/1376 [==============================] - 1s 859us/step - loss: 0.2520 - accuracy: 0.9125 - val_loss: 0.2728 - val_accuracy: 0.9027\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2021-12-15 16:22:23.274500: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: my_model_A/assets\n"
]
}
],
"source": [
"# extra code – split Fashion MNIST into tasks A and B, then train and save\n",
"# model A to \"my_model_A\".\n",
"\n",
"pos_class_id = class_names.index(\"Pullover\")\n",
"neg_class_id = class_names.index(\"T-shirt/top\")\n",
"\n",
"def split_dataset(X, y):\n",
" y_for_B = (y == pos_class_id) | (y == neg_class_id)\n",
" y_A = y[~y_for_B]\n",
" y_B = (y[y_for_B] == pos_class_id).astype(np.float32)\n",
" old_class_ids = list(set(range(10)) - set([neg_class_id, pos_class_id]))\n",
" for old_class_id, new_class_id in zip(old_class_ids, range(8)):\n",
" y_A[y_A == old_class_id] = new_class_id # reorder class ids for A\n",
" return ((X[~y_for_B], y_A), (X[y_for_B], y_B))\n",
"\n",
"(X_train_A, y_train_A), (X_train_B, y_train_B) = split_dataset(X_train, y_train)\n",
"(X_valid_A, y_valid_A), (X_valid_B, y_valid_B) = split_dataset(X_valid, y_valid)\n",
"(X_test_A, y_test_A), (X_test_B, y_test_B) = split_dataset(X_test, y_test)\n",
"X_train_B = X_train_B[:200]\n",
"y_train_B = y_train_B[:200]\n",
"\n",
"tf.random.set_seed(42)\n",
"\n",
"model_A = tf.keras.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(8, activation=\"softmax\")\n",
"])\n",
"\n",
"model_A.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=tf.keras.optimizers.SGD(learning_rate=0.001),\n",
" metrics=[\"accuracy\"])\n",
"history = model_A.fit(X_train_A, y_train_A, epochs=20,\n",
" validation_data=(X_valid_A, y_valid_A))\n",
"model_A.save(\"my_model_A\")"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/20\n",
"7/7 [==============================] - 0s 20ms/step - loss: 0.7167 - accuracy: 0.5450 - val_loss: 0.7052 - val_accuracy: 0.5272\n",
"Epoch 2/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.6805 - accuracy: 0.5800 - val_loss: 0.6758 - val_accuracy: 0.6004\n",
"Epoch 3/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.6532 - accuracy: 0.6650 - val_loss: 0.6530 - val_accuracy: 0.6746\n",
"Epoch 4/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.6289 - accuracy: 0.7150 - val_loss: 0.6317 - val_accuracy: 0.7517\n",
"Epoch 5/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.6079 - accuracy: 0.7800 - val_loss: 0.6105 - val_accuracy: 0.8091\n",
"Epoch 6/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.5866 - accuracy: 0.8400 - val_loss: 0.5913 - val_accuracy: 0.8447\n",
"Epoch 7/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.5670 - accuracy: 0.8850 - val_loss: 0.5728 - val_accuracy: 0.8833\n",
"Epoch 8/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.5499 - accuracy: 0.8900 - val_loss: 0.5571 - val_accuracy: 0.8971\n",
"Epoch 9/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.5331 - accuracy: 0.9150 - val_loss: 0.5427 - val_accuracy: 0.9050\n",
"Epoch 10/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.5180 - accuracy: 0.9250 - val_loss: 0.5290 - val_accuracy: 0.9080\n",
"Epoch 11/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.5038 - accuracy: 0.9350 - val_loss: 0.5160 - val_accuracy: 0.9189\n",
"Epoch 12/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.4903 - accuracy: 0.9350 - val_loss: 0.5032 - val_accuracy: 0.9228\n",
"Epoch 13/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.4770 - accuracy: 0.9400 - val_loss: 0.4925 - val_accuracy: 0.9228\n",
"Epoch 14/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.4656 - accuracy: 0.9450 - val_loss: 0.4817 - val_accuracy: 0.9258\n",
"Epoch 15/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.4546 - accuracy: 0.9550 - val_loss: 0.4708 - val_accuracy: 0.9298\n",
"Epoch 16/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.4435 - accuracy: 0.9550 - val_loss: 0.4608 - val_accuracy: 0.9318\n",
"Epoch 17/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.4330 - accuracy: 0.9600 - val_loss: 0.4510 - val_accuracy: 0.9337\n",
"Epoch 18/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.4226 - accuracy: 0.9600 - val_loss: 0.4406 - val_accuracy: 0.9367\n",
"Epoch 19/20\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.4119 - accuracy: 0.9600 - val_loss: 0.4311 - val_accuracy: 0.9377\n",
"Epoch 20/20\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.4025 - accuracy: 0.9600 - val_loss: 0.4225 - val_accuracy: 0.9367\n",
"63/63 [==============================] - 0s 728us/step - loss: 0.4317 - accuracy: 0.9185\n"
]
},
{
"data": {
"text/plain": [
"[0.43168652057647705, 0.9185000061988831]"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# extra code – train and evaluate model B, without reusing model A\n",
"\n",
"tf.random.set_seed(42)\n",
"model_B = tf.keras.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(1, activation=\"sigmoid\")\n",
"])\n",
"\n",
"model_B.compile(loss=\"binary_crossentropy\",\n",
" optimizer=tf.keras.optimizers.SGD(learning_rate=0.001),\n",
" metrics=[\"accuracy\"])\n",
"history = model_B.fit(X_train_B, y_train_B, epochs=20,\n",
" validation_data=(X_valid_B, y_valid_B))\n",
"model_B.evaluate(X_test_B, y_test_B)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Model B reaches 91.85% accuracy on the test set. Now let's try reusing the pretrained model A."
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [],
"source": [
"model_A = tf.keras.models.load_model(\"my_model_A\")\n",
"model_B_on_A = tf.keras.Sequential(model_A.layers[:-1])\n",
"model_B_on_A.add(tf.keras.layers.Dense(1, activation=\"sigmoid\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that `model_B_on_A` and `model_A` actually share layers now, so when we train one, it will update both models. If we want to avoid that, we need to build `model_B_on_A` on top of a *clone* of `model_A`:"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42) # extra code – ensure reproducibility"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [],
"source": [
"model_A_clone = tf.keras.models.clone_model(model_A)\n",
"model_A_clone.set_weights(model_A.get_weights())"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [],
"source": [
"# extra code – creating model_B_on_A just like in the previous cell\n",
"model_B_on_A = tf.keras.Sequential(model_A_clone.layers[:-1])\n",
"model_B_on_A.add(tf.keras.layers.Dense(1, activation=\"sigmoid\"))"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [],
"source": [
"for layer in model_B_on_A.layers[:-1]:\n",
" layer.trainable = False\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.001)\n",
"model_B_on_A.compile(loss=\"binary_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/4\n",
"7/7 [==============================] - 0s 23ms/step - loss: 1.7893 - accuracy: 0.5550 - val_loss: 1.3324 - val_accuracy: 0.5084\n",
"Epoch 2/4\n",
"7/7 [==============================] - 0s 7ms/step - loss: 1.1235 - accuracy: 0.5350 - val_loss: 0.9199 - val_accuracy: 0.4807\n",
"Epoch 3/4\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.8836 - accuracy: 0.5000 - val_loss: 0.8266 - val_accuracy: 0.4837\n",
"Epoch 4/4\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.8202 - accuracy: 0.5250 - val_loss: 0.7795 - val_accuracy: 0.4985\n",
"Epoch 1/16\n",
"7/7 [==============================] - 0s 21ms/step - loss: 0.7348 - accuracy: 0.6050 - val_loss: 0.6372 - val_accuracy: 0.6914\n",
"Epoch 2/16\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.6055 - accuracy: 0.7600 - val_loss: 0.5283 - val_accuracy: 0.8229\n",
"Epoch 3/16\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.4992 - accuracy: 0.8400 - val_loss: 0.4742 - val_accuracy: 0.8180\n",
"Epoch 4/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.4297 - accuracy: 0.8700 - val_loss: 0.4212 - val_accuracy: 0.8773\n",
"Epoch 5/16\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.3825 - accuracy: 0.9050 - val_loss: 0.3797 - val_accuracy: 0.9031\n",
"Epoch 6/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.3438 - accuracy: 0.9250 - val_loss: 0.3534 - val_accuracy: 0.9149\n",
"Epoch 7/16\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.3148 - accuracy: 0.9500 - val_loss: 0.3384 - val_accuracy: 0.9001\n",
"Epoch 8/16\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.3012 - accuracy: 0.9450 - val_loss: 0.3179 - val_accuracy: 0.9209\n",
"Epoch 9/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.2767 - accuracy: 0.9650 - val_loss: 0.3043 - val_accuracy: 0.9298\n",
"Epoch 10/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.2623 - accuracy: 0.9550 - val_loss: 0.2929 - val_accuracy: 0.9308\n",
"Epoch 11/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.2512 - accuracy: 0.9600 - val_loss: 0.2830 - val_accuracy: 0.9327\n",
"Epoch 12/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.2397 - accuracy: 0.9600 - val_loss: 0.2744 - val_accuracy: 0.9318\n",
"Epoch 13/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.2295 - accuracy: 0.9600 - val_loss: 0.2675 - val_accuracy: 0.9327\n",
"Epoch 14/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.2225 - accuracy: 0.9600 - val_loss: 0.2598 - val_accuracy: 0.9347\n",
"Epoch 15/16\n",
"7/7 [==============================] - 0s 6ms/step - loss: 0.2147 - accuracy: 0.9600 - val_loss: 0.2542 - val_accuracy: 0.9357\n",
"Epoch 16/16\n",
"7/7 [==============================] - 0s 7ms/step - loss: 0.2077 - accuracy: 0.9600 - val_loss: 0.2492 - val_accuracy: 0.9377\n"
]
}
],
"source": [
"history = model_B_on_A.fit(X_train_B, y_train_B, epochs=4,\n",
" validation_data=(X_valid_B, y_valid_B))\n",
"\n",
"for layer in model_B_on_A.layers[:-1]:\n",
" layer.trainable = True\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.001)\n",
"model_B_on_A.compile(loss=\"binary_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
"history = model_B_on_A.fit(X_train_B, y_train_B, epochs=16,\n",
" validation_data=(X_valid_B, y_valid_B))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So, what's the final verdict?"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"63/63 [==============================] - 0s 667us/step - loss: 0.2546 - accuracy: 0.9385\n"
]
},
{
"data": {
"text/plain": [
"[0.2546142041683197, 0.9384999871253967]"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model_B_on_A.evaluate(X_test_B, y_test_B)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Great! We got a bit of transfer: the model's accuracy went up 2 percentage points, from 91.85% to 93.85%. This means the error rate dropped by almost 25%:"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.24539877300613477"
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"1 - (100 - 93.85) / (100 - 91.85)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Faster Optimizers"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {},
"outputs": [],
"source": [
"# extra code – a little function to test an optimizer on Fashion MNIST\n",
"\n",
"def build_model(seed=42):\n",
" tf.random.set_seed(seed)\n",
" return tf.keras.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dense(10, activation=\"softmax\")\n",
" ])\n",
"\n",
"def build_and_train_model(optimizer):\n",
" model = build_model()\n",
" model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
" return model.fit(X_train, y_train, epochs=10,\n",
" validation_data=(X_valid, y_valid))"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.001, momentum=0.9)"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.6877 - accuracy: 0.7677 - val_loss: 0.4960 - val_accuracy: 0.8172\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 948us/step - loss: 0.4619 - accuracy: 0.8378 - val_loss: 0.4421 - val_accuracy: 0.8404\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 1s 868us/step - loss: 0.4179 - accuracy: 0.8525 - val_loss: 0.4188 - val_accuracy: 0.8538\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 1s 866us/step - loss: 0.3902 - accuracy: 0.8621 - val_loss: 0.3814 - val_accuracy: 0.8604\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 1s 869us/step - loss: 0.3686 - accuracy: 0.8691 - val_loss: 0.3665 - val_accuracy: 0.8656\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 925us/step - loss: 0.3553 - accuracy: 0.8732 - val_loss: 0.3643 - val_accuracy: 0.8720\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 908us/step - loss: 0.3385 - accuracy: 0.8778 - val_loss: 0.3611 - val_accuracy: 0.8684\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 926us/step - loss: 0.3297 - accuracy: 0.8796 - val_loss: 0.3490 - val_accuracy: 0.8726\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 893us/step - loss: 0.3200 - accuracy: 0.8850 - val_loss: 0.3625 - val_accuracy: 0.8666\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 886us/step - loss: 0.3097 - accuracy: 0.8881 - val_loss: 0.3656 - val_accuracy: 0.8672\n"
]
}
],
"source": [
"history_sgd = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Momentum optimization"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.001, momentum=0.9)"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 941us/step - loss: 0.6877 - accuracy: 0.7677 - val_loss: 0.4960 - val_accuracy: 0.8172\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 878us/step - loss: 0.4619 - accuracy: 0.8378 - val_loss: 0.4421 - val_accuracy: 0.8404\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 898us/step - loss: 0.4179 - accuracy: 0.8525 - val_loss: 0.4188 - val_accuracy: 0.8538\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 934us/step - loss: 0.3902 - accuracy: 0.8621 - val_loss: 0.3814 - val_accuracy: 0.8604\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 910us/step - loss: 0.3686 - accuracy: 0.8691 - val_loss: 0.3665 - val_accuracy: 0.8656\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 913us/step - loss: 0.3553 - accuracy: 0.8732 - val_loss: 0.3643 - val_accuracy: 0.8720\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 893us/step - loss: 0.3385 - accuracy: 0.8778 - val_loss: 0.3611 - val_accuracy: 0.8684\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 968us/step - loss: 0.3297 - accuracy: 0.8796 - val_loss: 0.3490 - val_accuracy: 0.8726\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 913us/step - loss: 0.3200 - accuracy: 0.8850 - val_loss: 0.3625 - val_accuracy: 0.8666\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 1s 858us/step - loss: 0.3097 - accuracy: 0.8881 - val_loss: 0.3656 - val_accuracy: 0.8672\n"
]
}
],
"source": [
"history_momentum = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Nesterov Accelerated Gradient"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.001, momentum=0.9,\n",
" nesterov=True)"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 907us/step - loss: 0.6777 - accuracy: 0.7711 - val_loss: 0.4796 - val_accuracy: 0.8260\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 898us/step - loss: 0.4570 - accuracy: 0.8398 - val_loss: 0.4358 - val_accuracy: 0.8396\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 1s 872us/step - loss: 0.4140 - accuracy: 0.8537 - val_loss: 0.4013 - val_accuracy: 0.8566\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 902us/step - loss: 0.3882 - accuracy: 0.8629 - val_loss: 0.3802 - val_accuracy: 0.8616\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 913us/step - loss: 0.3666 - accuracy: 0.8703 - val_loss: 0.3689 - val_accuracy: 0.8638\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 882us/step - loss: 0.3531 - accuracy: 0.8732 - val_loss: 0.3681 - val_accuracy: 0.8688\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 958us/step - loss: 0.3375 - accuracy: 0.8784 - val_loss: 0.3658 - val_accuracy: 0.8670\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 895us/step - loss: 0.3278 - accuracy: 0.8815 - val_loss: 0.3598 - val_accuracy: 0.8682\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 878us/step - loss: 0.3183 - accuracy: 0.8855 - val_loss: 0.3472 - val_accuracy: 0.8720\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 921us/step - loss: 0.3081 - accuracy: 0.8891 - val_loss: 0.3624 - val_accuracy: 0.8708\n"
]
}
],
"source": [
"history_nesterov = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## AdaGrad"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.Adagrad(learning_rate=0.001)"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 1.0003 - accuracy: 0.6822 - val_loss: 0.6876 - val_accuracy: 0.7744\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 912us/step - loss: 0.6389 - accuracy: 0.7904 - val_loss: 0.5837 - val_accuracy: 0.8048\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 930us/step - loss: 0.5682 - accuracy: 0.8105 - val_loss: 0.5379 - val_accuracy: 0.8154\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 878us/step - loss: 0.5316 - accuracy: 0.8215 - val_loss: 0.5135 - val_accuracy: 0.8244\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 1s 855us/step - loss: 0.5076 - accuracy: 0.8295 - val_loss: 0.4937 - val_accuracy: 0.8288\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 1s 868us/step - loss: 0.4905 - accuracy: 0.8338 - val_loss: 0.4821 - val_accuracy: 0.8312\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 940us/step - loss: 0.4776 - accuracy: 0.8371 - val_loss: 0.4705 - val_accuracy: 0.8348\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 966us/step - loss: 0.4674 - accuracy: 0.8409 - val_loss: 0.4611 - val_accuracy: 0.8362\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 892us/step - loss: 0.4587 - accuracy: 0.8435 - val_loss: 0.4548 - val_accuracy: 0.8406\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 873us/step - loss: 0.4511 - accuracy: 0.8458 - val_loss: 0.4469 - val_accuracy: 0.8424\n"
]
}
],
"source": [
"history_adagrad = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## RMSProp"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.001, rho=0.9)"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.5138 - accuracy: 0.8135 - val_loss: 0.4413 - val_accuracy: 0.8338\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 942us/step - loss: 0.3932 - accuracy: 0.8590 - val_loss: 0.4518 - val_accuracy: 0.8370\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 948us/step - loss: 0.3711 - accuracy: 0.8692 - val_loss: 0.3914 - val_accuracy: 0.8686\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 949us/step - loss: 0.3643 - accuracy: 0.8735 - val_loss: 0.4176 - val_accuracy: 0.8644\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 970us/step - loss: 0.3578 - accuracy: 0.8769 - val_loss: 0.3874 - val_accuracy: 0.8696\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3561 - accuracy: 0.8775 - val_loss: 0.4650 - val_accuracy: 0.8590\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3528 - accuracy: 0.8783 - val_loss: 0.4122 - val_accuracy: 0.8774\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 989us/step - loss: 0.3491 - accuracy: 0.8811 - val_loss: 0.5151 - val_accuracy: 0.8586\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3479 - accuracy: 0.8829 - val_loss: 0.4457 - val_accuracy: 0.8856\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 1000us/step - loss: 0.3437 - accuracy: 0.8830 - val_loss: 0.4781 - val_accuracy: 0.8636\n"
]
}
],
"source": [
"history_rmsprop = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Adam Optimization"
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9,\n",
" beta_2=0.999)"
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4949 - accuracy: 0.8220 - val_loss: 0.4110 - val_accuracy: 0.8428\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3727 - accuracy: 0.8637 - val_loss: 0.4153 - val_accuracy: 0.8370\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3372 - accuracy: 0.8756 - val_loss: 0.3600 - val_accuracy: 0.8708\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3126 - accuracy: 0.8833 - val_loss: 0.3498 - val_accuracy: 0.8760\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2965 - accuracy: 0.8901 - val_loss: 0.3264 - val_accuracy: 0.8794\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2821 - accuracy: 0.8947 - val_loss: 0.3295 - val_accuracy: 0.8782\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2672 - accuracy: 0.8993 - val_loss: 0.3473 - val_accuracy: 0.8790\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2587 - accuracy: 0.9020 - val_loss: 0.3230 - val_accuracy: 0.8818\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2500 - accuracy: 0.9057 - val_loss: 0.3676 - val_accuracy: 0.8744\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2428 - accuracy: 0.9073 - val_loss: 0.3879 - val_accuracy: 0.8696\n"
]
}
],
"source": [
"history_adam = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Adamax Optimization**"
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.Adamax(learning_rate=0.001, beta_1=0.9,\n",
" beta_2=0.999)"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.5327 - accuracy: 0.8151 - val_loss: 0.4402 - val_accuracy: 0.8340\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 935us/step - loss: 0.3950 - accuracy: 0.8591 - val_loss: 0.3907 - val_accuracy: 0.8512\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 933us/step - loss: 0.3563 - accuracy: 0.8715 - val_loss: 0.3730 - val_accuracy: 0.8676\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 942us/step - loss: 0.3335 - accuracy: 0.8797 - val_loss: 0.3453 - val_accuracy: 0.8738\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 993us/step - loss: 0.3129 - accuracy: 0.8853 - val_loss: 0.3270 - val_accuracy: 0.8792\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 926us/step - loss: 0.2986 - accuracy: 0.8913 - val_loss: 0.3396 - val_accuracy: 0.8772\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 939us/step - loss: 0.2854 - accuracy: 0.8949 - val_loss: 0.3390 - val_accuracy: 0.8770\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 949us/step - loss: 0.2757 - accuracy: 0.8984 - val_loss: 0.3147 - val_accuracy: 0.8854\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 952us/step - loss: 0.2662 - accuracy: 0.9020 - val_loss: 0.3341 - val_accuracy: 0.8760\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 957us/step - loss: 0.2542 - accuracy: 0.9063 - val_loss: 0.3282 - val_accuracy: 0.8780\n"
]
}
],
"source": [
"history_adamax = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"**Nadam Optimization**"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.Nadam(learning_rate=0.001, beta_1=0.9,\n",
" beta_2=0.999)"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.4826 - accuracy: 0.8284 - val_loss: 0.4092 - val_accuracy: 0.8456\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3610 - accuracy: 0.8667 - val_loss: 0.3893 - val_accuracy: 0.8592\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3270 - accuracy: 0.8784 - val_loss: 0.3653 - val_accuracy: 0.8712\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3049 - accuracy: 0.8874 - val_loss: 0.3444 - val_accuracy: 0.8726\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2897 - accuracy: 0.8905 - val_loss: 0.3174 - val_accuracy: 0.8810\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2753 - accuracy: 0.8981 - val_loss: 0.3389 - val_accuracy: 0.8830\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2652 - accuracy: 0.9000 - val_loss: 0.3725 - val_accuracy: 0.8734\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2563 - accuracy: 0.9034 - val_loss: 0.3229 - val_accuracy: 0.8828\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2463 - accuracy: 0.9079 - val_loss: 0.3353 - val_accuracy: 0.8818\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2402 - accuracy: 0.9091 - val_loss: 0.3813 - val_accuracy: 0.8740\n"
]
}
],
"source": [
"history_nadam = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**AdamW Optimization**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note: Since TF 1.12, `AdamW` is no longer experimental. It is available at `tf.keras.optimizers.AdamW` instead of `tf.keras.optimizers.experimental.AdamW`."
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.AdamW(weight_decay=1e-5, learning_rate=0.001,\n",
" beta_1=0.9, beta_2=0.999)"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.4945 - accuracy: 0.8220 - val_loss: 0.4203 - val_accuracy: 0.8424\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3735 - accuracy: 0.8629 - val_loss: 0.4014 - val_accuracy: 0.8474\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3391 - accuracy: 0.8753 - val_loss: 0.3347 - val_accuracy: 0.8760\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3155 - accuracy: 0.8827 - val_loss: 0.3441 - val_accuracy: 0.8720\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2989 - accuracy: 0.8892 - val_loss: 0.3218 - val_accuracy: 0.8786\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2862 - accuracy: 0.8931 - val_loss: 0.3423 - val_accuracy: 0.8814\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2738 - accuracy: 0.8970 - val_loss: 0.3593 - val_accuracy: 0.8764\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2648 - accuracy: 0.8993 - val_loss: 0.3263 - val_accuracy: 0.8856\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2583 - accuracy: 0.9035 - val_loss: 0.3642 - val_accuracy: 0.8680\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2483 - accuracy: 0.9054 - val_loss: 0.3696 - val_accuracy: 0.8702\n"
]
}
],
"source": [
"history_adamw = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtgAAAHoCAYAAABzQZg1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAD0SElEQVR4nOzdd5gUVdbA4d+t6jzdk4c0wBBmQJAgoIiCSjChILqgIihmXOPnrqirmLMsxtXVlaC4LiYUs5hRMYEkRVByhsl5Ond9f3RPzzQzYKMwifM+zzx0Vd2qvl2Oevpy6hxlGAZCCCGEEEKIA0Nr7AkIIYQQQgjRkkiALYQQQgghxAEkAbYQQgghhBAHkATYQgghhBBCHEASYAshhBBCCHEASYAthBBCCCHEAdSgAbZS6lSl1G9KqfVKqX/Uc/xGpdSKyM8qpVRQKZXakHMUQgghhBDiz1ANVQdbKaUDa4GTgO3AEuA8wzBW72X8aOBvhmEMb5AJCiGEEEIIcQA05Ar2QGC9YRgbDcPwAa8AY/Yx/jzg5QaZmRBCCCGEEAdIQwbYmcC2WtvbI/vqUEo5gFOBNxpgXkIIIYQQQhwwpgZ8L1XPvr3lp4wGvjEMo6jeCyk1GZgMYLPZBnTs2PHAzLAZ0Coq0QsLATBsVgKtW8d9bigUQtPkudbfI/cpPnKf4if3Kj5yn+In9yo+cp/iI/cpfmvXri0wDCPj98Y1ZIC9HehQa7s9sHMvY8ezj/QQwzCeA54D6N69u/Hbb78dqDk2ef68PNYff0J4w2ym23ffoTsT4jp34cKFDB069OBNroWQ+xQfuU/xk3sVH7lP8ZN7FR+5T/GR+xQ/pdSWeMY15NeVJUCOUqqzUspCOIh+Z89BSqkk4ATg7QacW7NhbtUKa48e4Q2/n6rvv2vcCQkhhBBCiBgNFmAbhhEArgE+AtYArxmG8YtS6q9Kqb/WGnoW8LFhGJUNNbfmxnnccdHXFV8vasSZCCGEEEKIPTVkigiGYXwAfLDHvmf32H4BeKHhZtX8OI8bQuFzzwFQ8fVXGIaBUvWluAshhBBCiIYmGe3NkP2II9CcTgACO3fh27ChkWckhBBCCCGqNf8Au4Ea5TQlymwm4dhjo9uSJiKEEEII0XQ0aIrIwWCv2MKviz/hsIEn1Xu8rKyMvLw8/H5/A8/s4ApNuoDgmDMA2Gm1krdmze+ek5SUxJo4xrUkCQkJtG/fXsoPCSGEEKLBNPsA20QA95ePQz0BdllZGbm5uWRmZmK321tUnnLI78dbXZ5QKWzduqF0fZ/nlJeX43K5GmB2TUMoFGLHjh0UFBTQqlWrxp6OEEIIIQ4RLWJZr2/FN2xbt7LO/ry8PDIzM3E4HC0quAbQzGY0qy28YRiEKqXoyp40TaN169aUlpY29lSEEEIIcQhpEQG2pgx2Lni0zn6/34/dbm+EGTUMzeWMvg5VVDTiTJous9lMIBBo7GkIIYQQ4hDSIgJsgL4F71OYu73O/pa2cl1bdSURgGB5BcYh+MDn72nJ//yFEEII0TQ1+wDbgxUAm/Kz9r3HG3cyDUxzOFCRh/cMvw/D52vkGQkhhBBCiOYfYJuTo68P2/YK7sryxptMA1OahpZQK02k/ND57EIIIYQQTVWzD7CLTB6WmcIVIlIo56f3n2nkGR0Y+fn5XHXVVXTq1Amr1Urr1q0ZMWIEn3zySXTMxo0b+ettU+l+8skk9+9Px8MPZ9iwYcyZMwdfrdVspRRKKRITE3E4HHTp0oUJEyawaJHUzxZCCCGEONCafZk+d8jNC1m96L/hcwAy18wmGPg7uql5f7SxY8dSVVXFrFmzyM7OJi8vjy+//JLCwkIAfvzxR0aMGEGPww7jkVtuoXvnzlR5vWz0eJgZOWfw4MHR682YMYOhQ4diNpvZuHEjc+bM4fjjj+fhhx/mxhtvbKyPKYQQQgjR4jTvKDTiW7ayRXOSFaqgvbGL5Z/9j36nXNjY0/rDSkpK+Prrr/nkk08YMWIEAFlZWRx11FEAGIbBhRdeSE5ODt9+9x2+DRswvF4AjsrKYsLEiXUeeExOTqZ169a4XC6ysrIYNmwY7dq145ZbbuGss84iOzu7YT+kEEIIIUQL1exTRAC8IR/PZPaLbtuXNO80EafTidPp5J133sHj8dQ5vmLFClavXs2UKVPQNA3dWTsPO1yuL57qGTfccAOhUIi33nrrgM1dCCGEEOJQ1yJWsAG+shVSgplk/BwWWMOvP3wMiR3qjOv0j/cbYXZhmx86Pa5xJpOJF154gcsvv5znnnuOfv36MXjwYM4++2yOPvpo1q5dC0D37t0B0FwuSjdvJnvECFAKlOLWW2/l1ltv3ef7pKWl0apVKzZu3PjnPpgQQgghhIhq9ivYJhX+jlAeqOCp1jWr2O4vH2+kGR0YY8eOZefOnbz77ruMHDmSb7/9lkGDBvHAAw/UGas5HLhcLr6fN4/vX3+ddm3bxjzkuC+GYUitaCGEEEKIA6jZB9iJemL09WfJXvyR130rvyXgb951oW02GyeddBJ33HEH3377LZdeeil33XUXnTp1AuDXX38FwuX6zC4XXTt2pGvHjljifMCzoKCA/Px8unTpcrA+ghBCCCHEIafZp4gkaAmk2lIp8hRR4Cvm2ZTeXFv8M5oyCLrL6oyPN02jKerZsyeBQIDDDjuMHj16MG3aNM455xx0XUdzOglG6mAboVBc13vkkUfQNI0xY8YczGkLIYQQQhxSmv0KtkJxfo/zo9sftLJSHV5aQu5muYpdWFjI8OHDeemll/jpp5/YtGkTr7/+OtOmTWPEiBEkJSXxwgsvsGHDBo455hjefvtt1u/axa8bN/L8G2+wY9cuNC32H21JSQm5ubls3bqVL774gosuuoiHH36Yhx56SCqICCGEEEIcQM1+BRvgnO7nMPPnmVQFqtjuy+MVZxcmVGxEYeApycWZUfdhx6bM6XQyaNAgnnjiCdavX4/X6yUzM5MJEyZw2223ATBw4ECWLVvGgw8+yLXXXsvu3buxW6306taNO6+9liuuujrmmpdffjkAVquVtm3bMmjQIBYuXMjxxx/f4J9PCCGEEKIlaxEBdpI1ibO7nc2c1XMAmNc2gwnrwpUxbP4iQsF2aLremFPcL1arlQceeKDeBxpry87OZtasWdFt/65dBCKNaEy1Vu6ra2KXl5fjcrkOwoyFEEIIIUS1Zp8iUu2Cnhdg0sLfF9YFdvGptTUAJkJUleY15tQajFarHnYwUg9bCCGEEEI0rBYTYLdOaM2oLqOi2y9kdo6+tngK63Q2bIm0hIRwHWzA8HkJxVmqTwghhBBCHDgtJsAGuPjwi1GEA8yVxk58KvzxLPipKitszKk1CKVp4SA7IhSpKiKEEEIIIRpOiwqwuyR3YViHYdHtct0cfa1X5h8Sq9h6rRzrUIWkiQghhBBCNLQWFWADXNL7kuhrHyG8kRVtGx48lXXrYrc0MXnYlZVx18QWQgghhBAHRosLsPtm9GVA6wEAGBjkme3RY0ZFbmNNq8FoVivKYglvhEKEqqoad0JCCCGEEIeYFhdgA1zSq2YVu1wLEoi8tgcr8XpafsCp11rFDkk1ESGEEEKIBtUiA+zjMo8jJyUHCNeAzjPZgHCBDX/p7sacWoPQauVhByvkQUchhBBCiIbUIgNspRQXH35xdLtUJ9o+3REoa5bt0/dHTLk+r5TrE0IIIYRoSC0ywAY4tfOp6Fq4e2PICJGvWwHQVLh9ektWp1yfVBMRQgghhGgwLTbANmtmnOaaXOQSk0Z1kb5w+/Rg40wsDhdddBFKKS677LI6x2666SaUUowaNaqeM2vE5GE3YoCtlGLevHmN9v5CCCGEEA2txQbYAHaTPbqKHTCCFGnhutjNoX16hw4dePXVV6msrIzuCwQC/Pe//6Vjx46/e762Rz1sKdcnhBBCCNEwWnSArSmNVFtqdLvQZIquYjf19ul9+vQhJyeH1157Lbrv/fffx2azMXTo0Oi+UCjEvffeS4cOHbBarfTu3Zu3334bZbGgzGa27NiB/fDDefnFFxk5ciR2u51+/frx008/sWrVKo499lgSEhIYMmQImzZtipnDu+++y4ABA7DZbHTu3JmpU6fiq5XP3alTJ+677z6uuOIKEhMTad++Pf/85z9jjgOcffbZKKWi23fddRe9evWKea8XXngBZ61V9+oxc+bMoVOnTjidTi6++GJ8Ph///ve/6dChA2lpafz9738nJF8ehBBCCNGEtOgAGyDVloqKPPDnJ0iZCq9oN4f26ZdeeimzZ8+Obs+ePZuLL744+nkAnnjiCf75z3/y8MMP8/PPP3PWWWfxl7/8hZUrV8asYt91331cf/31LF++nOTkZCZMmMC1117L/fffz+LFi/F4PFx33XXR8R999BETJ07kmmuu4ZdffmH27NnMmzePW2+9NWaOjz32GL1792bZsmXcfPPN3HTTTXz33XcALFmyBIAZM2awa9eu6Ha8Nm/ezNtvv817773HG2+8weuvv86YMWNYsmQJH3/8MTNnzuRf//oX8+fP36/rCiGEEEIcTKbGnsDBZtJMpFhTKPIUAZD0n+OixxL2dtLBclfpfg2fMGECU6ZMYd26dbhcLhYsWMC//vUv7rjjjuiY6dOnM2XKFCZMmADAPffcw1dffcX06dOZ8+9/R8ddd+GFnHLKKbhcLm644QZGjx7NG2+8wbBh4dby11xzDddcc010/P3338+NN97IxReHq7F07dqVhx9+mPPPP59//vOf0SD/5JNPjp537bXX8uSTT/LZZ59xzDHHkJGRAUBycjJt2rTZ37tFMBjk+eefJykpiV69enHqqafy5ZdfsmPHDiwWCz169GDw4MF88cUXjB07dr+vL4QQQghxMLT4ABsgzZ4WDbCbk5SUFM466yxmz55NcnIyQ4cOjcm/LisrY+fOnQwePDjmvCFDhvDBBx/ElOvr1bkzBMItd1q3bg1A7969o+e0bt2ayspKqqqqcDgcLF26lMWLF/Pwww9Hx4RCIdxuN7t376Zt27ZAOJWltnbt2pGXd2Dy2zt27EhSUlLMHLt164alulNlZN+Bej8hhBBCiAPhkAiwLbqFJGsSpd79W0FuCi655BIuvPBCnE4n99xzT71jaqeM1N6ndB1lD7eKN5nNKI8nZrzZbK5zjep85lAoxJ133snZZ59d59rVK9N7XqP6Or+XE61pWp38d7/fX2dcfdeub1+wCVeEEUIIIcSh55AIsAHS7emUekv55ZpvAcj2+bFiYBjgS+uO1eZo5BnWb8SIEVgsFgoKCjjzzDNjjiUmJtKuXTsWLVrE8OHDo/sXLVpEz549AdAdNYkwmtsd9/v279+fX3/9lezs7D81f7PZXCcAzsjIIDc3F8MwooH9ihUr/tT7CCGEEEI0FYdMgG0z2XBanFT4wjWhc01WOgY80fbpVluXRp5h/ZRS/PTTTxiGgdVqrXP8xhtv5I477iAnJ4cBAwbw0ksv8fXXX7N06VIANGdNgK3cboJlZXG97x133MGoUaPIysrinHPOwWQysWrVKhYvXsy0adPinn+nTp347LPPOOGEE7BaraSkpDB06FCKiop44IEHGD9+PAsXLpRa2UIIIYRoMVp8FZHa0u3p0dcVWojqpARHoAx/E26f7nK5SExMrPfYddddx4033shNN91Er169mD9/Pm+88QZHHHEEAKpWvjKGgW/rVvxx5CyfcsopvP/++3zxxRcMHDiQgQMH8tBDD8VVg7u2Rx55hC+++IIOHTrQr18/AHr06MEzzzzDc889R58+ffjkk0/qVCcRQgghhGiuVFOuBR2P7t27G7/99lu9x9asWUOPHj2i24ZhsKlsE25/OFUiOQiZwXBgXWFOx5nR4eBPuBGEqqrwbd2KEXnIEUCZLZjbZ6InNHgtlQa35+/B71m4cGFMrXFRP7lP8ZN7FR+5T/GTexUfuU/xkfsUP6XUUsMwjvy9cYfUCrZSinRbzSp2ma6ozg62+YsIBgP1n9jMaQ4H1uxsDEdNnrnh9+HbtAn/7t3S5VEIIYQQ4gA6pAJsAJfFhUUPp02EMCjQa9qnu0vzG3NqB5UymQimp2Nu3x6l69H9gYICfBs2ENqPByCFEEIIIcTeHXIBtlIqJhe7WNeoXr+1eAqadPv0P00pTMnJWLKz0Wq1JQ95vXg3bsSfn9+yP78QQgghRAM45AJsgCRrEiYtXEAliEGxVt0+PdDk26cfCJrZjCUrC3PbtqAivwKGQSA3F9/GTYS83sadoBBCCCFEM3ZIBtia0kizp0W3C0xmqtdt9cq8Q2IVVymFKS0Na3ZXtEgzGoCQuwrvhg0EiooOifsghBBCCHGgHZIBNkCKNQUtsnobIESZFn5tw4unMr5a0S2BZrVi6dIFU6tW0bbqhEL4d+7Et2ULoXo6LAohhBBCiL07ZANsXdNJtaVGt/N0S3QV26jIbZxJNRKlFOZWrbB26YKq1cwmVFGBb/16gqXNr8W8EEIIIURjOWQDbIBUe2q0VbdPhaiKvHaEKvF6qhpzao1Cs9uxdu2KKa0mfcYIBvFt24Zv27aYOtpCCCGEEKJ+h3SAbdbMJFuTo9u5ppquh4HS3Y0wo8anNA1z27ZYOnVGmc3R/cHSUrzr1xMsL2/E2QkhhBBCNH2HdIANxDzs6FYG7sgqtr2Jt08/2HRnAtbsbPTklOg+IxDAt2ULvp07MYLBfZwthBBCCHHoOuQDbKtuJdGaGN3OizSe0ZSBt6RxcrEvuugilFLcd999MfsXLlyIUoqCgoI//R7xXEvpOpb2mVg6dkTppuj+YFER3g0bCFYdemk0QgghhBC/55APsIGYxjMVGlSvWzdm+3Sbzca0adPIz2/87pJ6YiLWnGz0xJovIobPh2/jRvy5ufikbrYQQgghRJQE2IDdZCfBnBDdzmsC7dOHDRtGp06duPfee/c6ZvXq1Zx++um4XC5atWrFeeedx+7dNbnjP//8MyNGjCAxMRGXy8Wxxx7LF198webNmxk2bBgAGRkZKKW46KKLADAMg2nTptG1a1fsdju9e/fmpZdeQplMmDt0YEcwiKN3b1774ANGXnopiVlZ/PuBBwhUVXHvvffSoUMHrFYrvXv35u23347O5ZhjjuGGG26ImX9ZWRl2u5358+cfwDsnhBBCCNG4JMCOqL2KXaYrqtetG6t9uqZpPPTQQzz77LNs2LChzvFdu3Zx/PHH06tXLxYvXsynn35KRUUFZ5xxBqFQuPn7hAkTaNu2LYsXL2b58uXccsst2Gw2OnTowBtvvAHAL7/8wq5du3jiiScAuO2225g1axZPP/00q1ev5pZbbuGKK67g/fffDzenSUoC4I4nn2Tyueey7K23OP2443j07rv557RpPPTQQ/z888+cddZZ/OUvf2HFihUAnH/++bzyyivRuQG88cYb2O12Tj/99IN5K4UQQgghGpTp94e0LL3n9G609/75wp/3a/xpp53G4MGDmTp1Kq+88krMsWeeeYa+ffvy8MMPR/e9+OKLpKam8uOPPzJw4EC2bNnClClTOOywwwBo3bo1LpcLgNTUcA3wVq1akZ4e/nJRWVnJo48+yscff8xxxx0HQOfOnVm8eDFPP/10TCB87fXXc86kSfhzc8EwePyFF/i/SZMYd8wxmNu355577uGrr75i+vTpvPTSS4wfP56//e1vfPHFF4wYMQKA//3vf5x99tlYLDXVW4QQQgghmjtZwW7ipk2bxuuvv86PP/4Ys3/p0qV89dVXOJ3O6E+HDh0Aoivef//737nssssYPnw4999/P2vXrt3ne61evRqPx8Opp54ac91nnnmmzir6UUcdhSk9HWvXrlT4A+zKy+OYfv0IVVXhW7+eQFERgwcPZvXq1QCkpaVxyimn8L///Q8Ir8B/8cUXnH/++QfkPgkhhBBCNBUSYDdxRx11FGPHjuXmm2+O2R8KhTj99NNZsWJFzM+6desYNWoUAHfddRerV6/mzDPP5Ntvv+WYY45h9uzZe32v6vSNd999N+aav/zyCx9//HHM2ISEcM66ZrNh6dwJABVpPW9EWq2HyspQtc45//zzeeONN/B4PLz88st06NCBIUOG/JnbI4QQQgjR5BxyKSK/l6ZR5CliV8UuAHQU3XxeNKBKS8DRplsDzLCuBx54gJ49e7JgwYLovv79+/Paa6+RlZWFuVZDmD3l5OSQk5PDddddx2WXXcbMmTO55JJLomkZwVr1rHv27InVamXLli0MHz487vklJSfTrl07Fm/exLDjhmD4wnVYvvn+e7q3b0+wtBQ9KYkxY8YwefJk3nvvPf73v/8xceLEaCdNIYQQQoiWQlaw95BsTUbXdACCGJRp4VvUmO3Ts7OzmTx5cvRBRICrr76a0tJSzj33XH744Qc2btzIp59+yuTJkykvL8ftdnP11VezcOFCNm/ezA8//MB3331Hz549AcjKykIpxfvvv09+fj4VFRW4XC6mTJnClClTmD17NuvXr2fFihU8++yzPPfcc/uc44033sgjTzzBm0uWsLG4hHueeopvli3jukmTwq3Wt2/Hajbzl7/8hfvuu49ly5ZJeogQQgghWiQJsPegKY00W013xzzdTHUNkcZsn37HHXdgMtX8hUO7du345ptv0DSNU089lcMPP5yrr74aq9WK1WpF13WKi4u58MIL6d69O2eddRYDBw7k0UcfBSAzM5O7776bqVOn0rp1a6655hoA7r33Xu666y6mT5/O4YcfzkknncQbb7xB586d9zm/6667jhtvvJGb//EP+o0Yzntff83LTz5J38gDlsGSErzr1zNh3DhWrlxJ//796dGjx0G6W0IIIYQQjeeQSxGJR4othQJ3ASEjhF8ZVCgNlxGKtE/3YjZbD+r7v/DCC3X2tWrVivLy8ph9OTk5zJs3b6/XmTt3bsx2eXl5tIoIwO23387tt98eM0YpxbXXXsu1115b7zU7depUb9lCTdPqXM8IBvHv2kWwpCS87fczJCsL386dmFq33uu8hRBCCCGaM1nBrodJM5FiS4lu50VWjsPt0/Maa1rNTrjVenssHTqgdD26P1BYiHfDBkLSal0IIYQQLZAE2HuRZkuLPoDnUVAVed2Y7dObKz0pCUt2Nnqt1XPD68W7cRP+3DyMWs1nhBBCCCGauwYNsJVSpyqlflNKrVdK/WMvY4YqpVYopX5RSn3ZkPOrzaybSbImRbfz9PAqdmO2T2/ONLMZc8eOmNu1Q2nVv3YGgfw8fJs2EfJ6G3V+QgghhBAHSoMF2EopHXgaGAn0BM5TSvXcY0wy8G/gDMMwDgfObqj51SfdVtM+vVJTeCKr2BZPgay6/gFKKUypqViys9Ecjuj+kNuNd/16AgWN05ZeCCGEEOJAasgV7IHAesMwNhqG4QNeAcbsMWYC8KZhGFsBDMNo1IRnq8mKy1KT1pAfySO2EKCqvKixptXsaRYLls6dMbduDdV1sA0D/+7d+DZvJhSpoy2EEEII0RyphloxVEqNA041DOOyyPYFwNGGYVxTa8zjgBk4HHABTxiG8WI915oMTAbIyMgY8Nprr9X7nklJSWRnZ/+peXtDXnL9udHtHJ8PC+DGit/Zvlk1SgkGg+i1HjZsEnw+9IJClL9WUK00gqkpGAkJNQH4n7B+/XpKS0vjHl9RUYHT6fzT79vSyX2Kn9yr+Mh9ip/cq/jIfYqP3Kf4DRs2bKlhGEf+3riGLNNXX6S0Z3RvAgYAIwA78J1S6nvDMNbGnGQYzwHPAXTv3t0YOnRovW+4Zs2amLJ0f4QLF+Wl5VT5wxUvCnWdtsEgdrygGdidiX/q+g1pzzJ9TYWRkkIgL59AQSS33QihFxai+/3hnG3Tn/s1tdls9OvXL+7xCxcuZG+/U6KG3Kf4yb2Kj9yn+Mm9io/cp/jIfTrwGjJFZDvQodZ2e2BnPWMWGIZRaRhGAfAV0LeB5rdX6faaXOxiXae6hohRkVv/CWK/KE3D3KY1ls5dUJEW7gDBsjK869cTLCtrxNkJIYQQQuyfhgywlwA5SqnOSikLMB54Z48xbwPHKaVMSikHcDSwpgHnWC+n2YnVFG4uYwBFkTQLR6gSr7uyEWfWsugJDqxdu6Knpkb3GYEAvq1b8e3YgREMNuLshBBCCCHi02ABtmEYAeAa4CPCQfNrhmH8opT6q1Lqr5Exa4AFwE/AYmCmYRirGmqOe6OUiqkoUqjrVNcQCZQ1jVXsefPmNZt88FGjRnHRRRfVe0zpOpZ27bBkZcWkhgSLi8Or2ZXyhUYIIYQQTVuD1sE2DOMDwzC6GYbR1TCM+yP7njUM49laY/5pGEZPwzB6GYbxeEPOb18SrYmYNTMAIaA4Usu5un36wbB8+XJ0XWfw4MEH5foAGzdu5LLLLiMrKwur1Uq7du0YNmwYc+bMwdeI1Tx0lwtrdjZ6Uk0tcsPvx7dpE/5du6VMohBCCCGaLOnkGCdNaaTZ06LbBboJg+r26QdnFXvGjBlcddVVrFq1ijVrDnymzI8//ki/fv1YtWoV//rXv/j555/54IMPmDx5MnPmzGHJkiV7Pdfv9x/w+exJmUxYOnTA3L79Hq3WC/Bt2EDI7T7ocxBCCCGE2F8SYO+HZGsyuhYO9AIKyiKr2DZ/8QFvn+52u5k7dy6XX34548aNY9asWTHHX3zxRbKysnA4HIwaNYrc3Nggf8OGDYwZM4Y2bdqQkJBA//79+fDDD6PHDcPgwgsvJCcnh2+//ZYzzjiDbt26ccQRR3Deeefx+eefc+yxxwKwefNmlFK8/PLLDB8+HLvdzn/+8x8KCws577zzaN++PXa7ncMPP5znn38+Zh5VVVVcdNFFOJ1OWrduzQMPPLDf98KUnBxuTlOrhFDI68W7cSP+vHxpTiOEEEKIJkUC7P2gazqptpoH8PJ1HYNI+/SSA9s+fd68eWRlZdGnTx8uuOACXnzxxeiq8Q8//MBFF13E5MmTWbFiBaNHj+aOO+6IOb+iooKRI0fyySefsHLlSsaOHcv555/Pr7/+CsCKFStYvXo1U6ZMQdPq/zXYM6f7lltu4aqrrmL16tWceeaZeDwe+vfvz3vvvccvv/zC//3f/3HFFVfw2WefRc+ZMmUKn3zyCW+88QafffYZy5cv56uvvtrv+6GZzViysjC3bQvV8zUMAnm5+DZulFbrQgghhGgyGrIOdpOw5rAef/oa1eGoH/h1P87r8Wv8aR4zZ87kggsuAOCEE07A4XDwzjvvMHbsWJ544glGjBjB1KlTAejWrRtLliyJWeXu27cvffvWVDicOnUqb731FvPmzeO2225j7dpwafHu3btHx5SWlpKZmRndvvXWW7n11luj29deey3jxo2LmeeNN94YfT158mQ+//xzXn75ZUaMGEFFRQWzZs1i9uzZnHLKKQA8//zztG/fPu77UJtSClNaGprTiX/79miKSMjtxrthA+bWrdFTU5vNw55CCCGEaJlkBbsJWr9+Pd988w0TJkwAwoHlxIkTmTlzJhBuoHPMMcfEnLPndmVlJTfddBM9e/YkJSUFp9PJ8uXL2bp1617f1+VysWLFClasWEG7du3qPOR45JGxjYuCwSD3338/ffr0IS0tDafTyZtvvhl9jw0bNuDz+WLm5nQ66d27937ekVia1YqlSxdMrWq1Wg+F8O/ahW/LFkINkB8uhBBCCLE3h9wKdnMwc+ZMgsEgHTt2jO6rzjPetm1bXDnHU6ZMYcGCBUyfPp2cnBwcDgcTJ06MBs3dunUD4Ndff412OdQ0Ldpa3lKr4Uu1hISEmO3p06fzyCOP8MQTT9C7d2+cTie33noreXl5MXM+GJRSmFtloLsiq9mRFJFQRQW+9esxtW2LKTn5oL2/EEIIIcTeHHIB9v6kaezL9vLtlHpLAXCFQnQMhB9ydCd2we5M2tep+xQIBJgzZw4PPvggo0aNijl2wQUX8Pzzz9OzZ0++//77mGN7bi9atIhJkyYxduxYADweD5s2baJHj3CKzBFHHEGPHj2YNm0a55xzDnqtKh3xWrRoEaNHj46mshiGwdq1a0mOBLbZ2dmYzWa+//57unTpAoRX1letWkXXrl33+/3qo9ntWLp2JZCXR6CgIDyPYDAcdJeXh3O2hRBCCCEa0CEXYB8o6fb0aIBdrml4UVgxMMpz4U8E2O+//z4FBQVcfvnlpKWlxRwbP348zzzzDHPnzmXIkCE8+OCDjBs3joULFzJ//vyYsd26dWP+/PmMGTMGs9nM3XffjbfWg4BKKV544QVOPPFEjjnmGKZOnUqPHj0IBoN88803bN++/XeD7m7duvHqq6+yaNEi0tPT+de//sWmTZuiK+JOp5NLL72Um2++mYyMDNq1a8c999xD8AB3ZAy3Wm+D5nLh374dI5IiEiwtJVRZScjjOaDvJ4QQQgixL5KD/QfZTDaclpqycYWm8K10GH+uffqsWbMYNmxYneAa4Oyzz2bLli3RhwefeeYZ+vTpw5tvvsldd90VM/bRRx+lVatWHHfccYwcOZJBgwbVydMeOHAgy5Yto3fv3lx77bX06tWLQYMGMWfOHO6//35uuummfc71tttuY+DAgYwcOZLjjz+ehIQEJk6cGDNm+vTpDBs2jLPOOothw4bRq1cvjj/++D92c36HnpAQbk6TkhLdZwQCBIuK2HXXXYSkC6QQQgghGoCsYP8J6fZ0KnwVAJRoOhkEMRNun261d/lD13znnXf2eqxLly4xec0XX3xxzPFrrrkm+jorK4tPP/005vgVV1yBy+WK2ZednV2nxvaeOnXqVG8+dUpKCm+++eY+z01ISODFF1/kxRdf3Oe4A0XpOpbMTIIuF/6dOzEiqTslr7xK5bff0e6hh3D079cgcxFCCCHEoUlWsP8Eh8mB3WwHwAAKIykVB7N9uoiPnpgYXs1OTIzu82/dypbzzyfv0ccwGrENvBBCCCFaNgmw/wSlFOm29Oh2sa4T5OC2TxfxUyYT5g4d0JOTa7pAhkIUPvccm845F89vaxt3gkIIIYRokSTA/pNcFhcWPVzSLgQU6QevfbrYf0opNIeDLu+8jWPQoOh+76+/snncOApnzcI4wA9dCiGEEOLQJgH2n6SUIt1es4pdqJkIcXDap4s/ztyuHR1nz6L1rbeirFYADL+fvH9OZ8ukC/Ft29bIMxRCCCFESyEB9gGQZE3CpIWfFw0qKNHCt9XiLcAIhRpzaqIWpWmkTrqAzm++ga1Xr+h+99KlbBpzJsWvv35Qm+MIIYQQ4tAgAfYBoCmNNHtNWb0CXccALASoKi9svImJelm7dqXTy3NJv+YaiDyYGqqqYvftd7D9yqvQSksbeYZCCCGEaM4kwD5AUqwpaCp8O/1KUR5ZxdYr82VVtAlSZjMZ11xNp1dextK5c3R/xcKFpN11N7vuuJOqJUvkbyCEEEIIsd8kwD5AdE0n1ZYa3c6PrGLb8OKpLGu8iYl9svfuTef5b5Iy6YLoPs3tpuS119hywSTWn3gieY88gmetVBwRQgghRHwkwD6AUu2pKKUA8ChFVeS1US4l+5oyzWajza230vGF5zFndYw5Fti5i8IZM9l0xhg2jjmTwpkz8e/a1UgzFUIIIURzIAH2AWTWzCRbk6PbBZH83j/bPl00jIRBg+j64YcUTbmB5HPPRU9Kijnu/e038qY/wvrhI9hywSSKX3uNoORrCyGEEGIPEmAfYLUfdqzQNNyRVexAWfyr2BdddBFKKZRSmEwmOnbsyJVXXklxcXF0TKdOnVBK8dJLL9U5f+DAgSilmD59enTfpk2buOyyy2jfvj1Wq5V27dpx+umns3z58jrXVErhcDjo1asX//nPf/br8zd3StPwZ2fT9u67yPn6K9r/+2lcI0+NlvYDwDCoWrKE3Xfcybohx7HtmmsoW/ARIa907xRCCCGEBNgHnFW3kmitac8d0z7dF38AduKJJ7Jr1y42b97MzJkzeffdd7nqqqtixnTo0IFZs2bF7Fu1ahW//PILaWk1gb7f7+ekk06ioKCA1157jbVr1zJv3jwGDhxIUVFRzPl33HEHu3bt4qeffuLMM8/kr3/9K6+++mq9c/S18HbjymLBNXw47R97jJxvFtH2wQdJOPZY0Gr+tTH8fio+/Ywd11/PuiHHsXPqVCq//16a1wghhBCHMAmwD4LajWdKNQ0fkfbppfGvYlutVtq0aUP79u05+eSTOffcc/n4449jxkyYMIHvvvuOjRs3RvfNmjWLcePG4axuDQ788ssvbNiwgUceeYRjjz2WrKwsjj32WO68805GjBgRc02Xy0WbNm3Izs7mvvvuIycnh7feeguAoUOHcuWVVzJlyhQyMjIYPHgwAF999RVHH300NpuN1q1b87e//S0m+B46dCh//etf+b//+z9SUlJISUnhxhtvJNSMKnToTifJZ51Jx9mzyF74Ba1v+Qe2ww+PGRMqL6f0jTfZetHFrB82nNyHp+FZvVqqyAghhBCHGAmwDwK7yU6COSG6XZ2L/Ufbp2/cuJEFCxZgNptj9qenpzN69Gief/55ILyi/NJLL3HppZfGjMvIyEDTNN555x0Cgf17f5vNht/vj26/9NJLGIbB119/zYsvvsiOHTsYOXIk/fr1Y/ny5cyaNYuXX36ZW265JeY6//vf/wiFQnz33Xf85z//4bnnnuPxxx/fr7k0FeZWrUi98EI6vzGPLh98QPpVV2Lu0CFmTCAvj6Lnn2fTX8aycdRoCp59Ft/27Y00YyGEEEI0JFNjT6ChPf3Xzxvtva++I4WKknycaW1/d+yCBQtwOp0Eg0E8Hg8Ajz76aJ1xl1xyCVdccQV3330377zzDsnJyRx//PExYzIzM3nyySe56aabmDZtGgMGDOD4449n/PjxHL7HKmy1QCDASy+9xM8//8yVV14Z3d+5c2ceeeSR6PbUqVNp27Yt//73v9E0jR49evDQQw9xxRVXcO+99+JwOABo27YtTz75JEopDjvsMNauXcujjz7K3//+99+/cU2YtUtnMq67jvRrr8WzciWl775H2YcfEqyVeuPbsIH8x58g//EnsPfrR+LoUSSOHIkpJaURZy6EEEKIg0VWsBtYvO3Tjz/+eFasWMHixYu59tprOe2007juuuvqjDvllFMwDINPPvmEWbNmcckll9R7vauvvpp169Yxd+5chgwZwttvv80RRxzBf//735hxU6dOxel0Yrfbufrqq7nxxhu54ooroscHDBgQM37NmjUcc8wxaLXykocMGYLP52P9+vXRfYMGDYqWMAQ45phj2LFjB2VlLaNGuFIK+xFH0Ob228j5ciEdnvsPiaNHo+z2mHHu5cvJvede1h13PNv+eiWl779PyO1upFkLIYQQ4mCQALsBBYm0Ty/7/fbpDoeD7OxsevfuzZNPPklVVRX33ntvnXGapnHhhRfywAMP8Pnnn3PhhRfu9Zoul4szzjiD+++/n5UrVzJs2DBuv/32mDF///vfWbFiBVu2bKGiooJp06bFBM8JCQkx4w3DiAmca9vb/pZOmc04jz+ezH9Oo9uir2n3z2kkHH9ctC07AIEAFQsXsvOGKawbPISdN99MxdeLMPYzhUcIIYQQTc8hlyJy9bPDG+y9DMNgfcl6fMHwA3/FgQDpoRB6VT5GUvp+BaB33nknI0eOZPLkybRr1y7m2CWXXMIDDzzAaaedVufY3lSnaixbtixmf1paGtnZ2XHPq2fPnrz22muEQqFoIL5o0SIsFgtdu3aNjvvhhx9igvHvv/+edu3akZiYWO91WwotIYGk0aNJGj2aQGEhZR8uoOzdd3GvXBkdE6qqovTtdyh9+x309HQSR44kafQobL17H7JfUoQQQojmTFawDyKlVExd7EJdJ0R1+/T9a1AydOhQDj/8cO677746x7p06UJBQQGvv/56veeuWLGCMWPG8NZbb7F69WrWr1/PrFmzmD17NmedddZ+zWNPV111FTt37uSqq65izZo1vP/++/zjH//gmmuuieZfA+zcuZPrr7+e3377jXnz5vHPf/6Tv/3tb3/qvZsbU1oaqedPpNOrr9D1449Iv+5aLJ07x4wJFhRQ/N//svmcc9l46kjyn3oa35YtjTRjIYQQQvwRh9wKdkNLtiaTV5VHMBQkoBRlmkZyKIRRngfO5P261t///ncuvvhibr755jrHUlNT93pe+/bt6dKlCw8//DBbt24lFArRsWNHpkyZwj/+8Y/9/UgxMjMz+fDDD7nxxhs54ogjSE5OZsKECTzwwAMx4yZOnEgwGOToo49GKcWll156yAXYtVk6diTjqqtIv/JKPL+spuzddyn94H2C+QXRMb4tWyh46ikKnnoKW58+JI0aReJpIzGlp+/jykIIIYRobBJgH2Sa0kizpZFXlQeES/YlhULR9ulWe0Kdc1544YV6rzVhwgQmTJgAwObNm/f5vrWPp6en89hjj1FeXo7L5YrrnPosXLiw3v3HH388P/zwwz7PNZlMPPXUUzz11FP7HHeoUUph73U49l6H0+qmG6n8/nvK3n2P8k8+IVRZGR3n+eknPD/9RO7DD5NwzDEkjR6F68QT0RLq/v4IIYQQonFJgN0AUmwpFLgLCBkhvEpRoTRcRohAWS5We5fGnp5oIpSu4xw8GOfgwYTuupOKL76g9N33qPj6a6iuRR4MUrloEZWLFqFsNlwjRpA4ehTOwYNRe9RJF0IIIUTjkAC7AZg0Eym2FArd4eohBSYNlz8UbZ9utlgbeYaiqdFsNhJHjiRx5EgCxcWUf/Qxpe+9i/vHpdExhsdD2fvvU/b+++gpKSSOPJXEUaOx9ztCHo4UQgghGpEE2A0kzZZGkacIwzCoUhpVSuHAoKo0F3NGx8ae3kG1t9QSER9TSgop488lZfy5+HfsoPS99yl7712862rqjAeLiyme+zLFc1/G3L49iaNOJ2n0aKy1KrkIIYQQomFIFZEGYtbNJFmTotvV7dPtf7B9ujg0mTMzSb9iMp3feYfOb80n9dJLMLVpEzPGv307hc/+h42nj2LjX/5C4ezn8efmNdKMhRBCiEOPBNgNKN1WU/2hXNPwKIVOCHeJBD9i/yilsB12GK1vvJHszz+j45w5JJ89Dm2Ph1i9q9eQN20a64cOZcvFF1PyxpsEy8sbadZCCCHEoUEC7AZkNVlxWWoCoEI9fPst3sK42qcLUR+laSQcPZC2995LzjeLyPzXk7hOPhllsdQMMgyqvvueXVOnsm7wELb/3/WUf/opIZ+v8SYuhBBCtFCSg93A0u3plPvCK4glmk4GQSwEqCwrJCE5o5FnJ5o7zWIh8aSTSDzpJIJlZZR//DGl775H1eLFYBgAGD4f5R99RPlHH6ElJZF48skkjh6F48gjUZp85xZCCCH+LAmwG5jD7MBhdlDlrwLC3R3bBoN/qH26EPuiJyaSPG4cyePG4d+9m7L3P6D0vffwrlkTHRMqLaXk9dcpef11TG3bkjTqdBJHjcbWvVsjzlwIIYRo3mS5qhGk22tysUt0jQB/rH26EPEyt2lD2qWX0GX+m3R5713SrrgCc2ZmzJjArl0UzpjJpjFj2HjGGApmzMC/c2cjzVgIIYRoviTAbgROsxOrKVz7OoSiKFJRxCj/4w87zps3T1a/RVys2dm0+tv1dP30E7Lm/o/k8eeiJyXFjPGuXUv+I4+yfvgItpx/AcWvvkawVL4ACiGEEPFo9gG22Q1GsHk9IKiUiqkoUqRrhCDaPr3a8uXL0XWdwYMHN8IsRUunlMLRvz9t77qLnK+/ov2//03iaSNRNlvMuKoff2T3nXeydshxbLv6GsoWLCDk8TTSrIUQQoimr9nnYGt+RelHm0k+rXm1HE+0JpJXlYc/5CeIoljTSAvFtk+fMWMGV111FS+++CJr1qyhR48ejTxr0VIpiwXX8GG4hg8jWFFJ+aefUPbue1R+9x1UV7jx+6n47DMqPvsMzekksXdvSisrcQwYgHmPWtxCCCHEoazZr2ADVHy1A/evRY09jf2iKY00e1p0u1DXMQB7oBS/z4vb7Wbu3LlcfvnljBs3jlmzZsWc/+KLL5KVlYXD4WDUqFHk5ubGHN+wYQNjxoyhTZs2JCQk0L9/fz788MOYMZ06deKee+7hoosuwuVy0aFDB1599VVKSkoYP348TqeTnJwcPv7444N2H0TTozsTSD7zTDrOmknOlwtpfest2Hr1ihkTqqjA/t137LxhCuuHDmP98BHsuPEmil95Bc/atVJ2UgghxCGtRQTYAMWv/UagxNvY09gvydZkdC2cf+1XijJNQ1PgLc1l3rx5ZGVl0adPHy644AJefPFF/H4/AD/88AMXXXQRkydPZsWKFYwePZo77rgj5toVFRWMHDmSTz75hJUrVzJ27FjOP/98fv3115hxjz/+OAMHDmTZsmWcc845XHjhhUyYMIHTTjuNFStWcPzxx3P++efjkZSAQ5IpI4PUSZPoPO91unz4AelXXYW5Y8c64/w7d1L27rvsvutuNp0xhrWDjmHbFX+l4LkZVC1dSsjbvP7dFEIIIf6MZp8iQuS5vlBVgKKXfyVjcm+UvvfvDY+cO6qBJlbXDa++F7OtazqptlTyq/KBcPv0xFAIu7+YGTNmcMEFFwBwwgkn4HA4eOeddxg7dixPPPEEI0aMYOrUqQB069aNJUuWxKxy9+3bl759+0a3p06dyltvvcW8efO47bbbovtPOeUUrrrqKgDuvvtuHn30UbKzs5k0aRIAt99+O7Nnz2bVqlUceeSRB+GuiObC2rkzGdddS/q11+D56SdWzXmRVkVFuFeuxHC7Y8aGysqo+PJLKr78EgBlNmPr1QvHgP7Y+w/A3u8ITCkpjfExhBBCiIOu2QfYfocRXocPgW9LGWWfbCHp1M6NPa24pdpSKXAXYBgGHqWoVIrdGzfz7bff8sorrwDhh9EmTpzIzJkzGTt2LGvWrGH06NEx1znmmGNiAuzKykruvvtu3nvvPXbt2oXf78fj8dCvX7+Y8/r06RN97XQ6cTgc9O7dO7qvdevWAOTlSTt3EaaUwt63L5VnjCZr6FAMvx/Pr79StXQp7qXLqFq2jGBhYcw5ht+Pe/ly3MuXA+HfU0t2Vxz9B4SD7gEDMGdmSiUcIYQQLUKzD7BDOiSelEXZR1sAKF+4HWvnJGzdUxt5ZvExaSZSrCkUecI55AW6zsyX3yIYDNKx1l/FG5EufNu2bYu+3pcpU6awYMECpk+fTk5ODg6Hg4kTJ+LbozW22WyO2VZKxeyrDnhCklMr9kKZzdh798beuzdcdBGGYeDfsoWqpcuoWrYU97Ll+DZtqnOeb/0GfOs3UPLaawCYWrXCPqB/NOi2duuGMjX7/0QJIYQ4BLWI/3u5TuiAd2Mp3nUlABS99hut/69/vWP3TNNoCtLsadEAuzQU4oXX3+PBW65l+KhxOBJrvihccMEFPP/88/Ts2ZPvv/8+5hp7bi9atIhJkyYxduxYADweD5s2bZJKJOKgU0ph6dQJS6dOJI/9CwCBwkLcy5dHg27PL6shEIg5L5CXR/mHCyj/cAEAmsOB/YgjwkH3gAHY+/RBczga/PMIIYQQ+6tFBNhKU6Se253cJ5YTKvcRqgxQ+PJvcJz5909uAiy6hSRrEqXeUr765CsKi0q4fMJZJKSmY23bI7qKPH78eJ555hnmzp3LkCFDePDBBxk3bhwLFy5k/vz5Mdfs1q0b8+fPZ8yYMZjNZu6++2688qCZaCSmtDRcJ56I68QTAQi53bh/+hn3sqVULV2Ge/lyQpWVMeeEqqqo/PZbKr/9NrxD17H17Imjf//ISnd/TOnpe76VEEII0ehaRIANoDstpI7vTsHMn8EA36ZSQkclN/a04pZuT6fUW8qb/3uTo4YchTMtBavhxV1Zit2ZDMDZZ5/NP/7xDyoqKpg1axZ33nkn99xzD0OHDuWuu+7i2muvjV7v0Ucf5dJLL+W4444jJSWF66+/noqKikb6dELE0ux2Eo4eSMLRAwEwgkG8a9eGg+1lS6n6cSmBPfP+g0E8P/+M5+efYc4cACxZWdgHRPK4+/fH0qmT5HELIYRodC0mwAawdU0mcURHyj7dCkDIEyDkCaDZmv7HtJlsOC1OnnrpKQAKQ0HaBYIY5bkQCbC7dOkSk3998cUXx1zjmmuuib7Oysri008/jTl+xRVX4HK5otubN2+uM489g3CbzRZXzrcQf4bSdWw9emDr0QPOnxjO496xMxxsL1uGe+kyvOvW1TnPt2ULvi1bKH3zTQD01FTs/ftF87htPXqgLJaG/jhCCCEOcU0/8txPruEd8W4qxbuhFIBAkQdza8c+S/c1Fen2dCp84QC3RNPJIIjDqMLrrsRqT2jk2QnRcJRSWNpnYmmfSdIZZwAQLCmhasWKaKUSz08/YURqw1cLFhVR8elnVHz6Wfg6Nhv2Pn2iD0/a+x2B7nQ2+OcRQghxaGlxAbbSFKnjDyP3iWXhHSGDQJEHU7q9yf/VscPkwG624/a7MQh3d2wTDBIo243V3rWxpydEo9KTk3ENHYpr6FAAQl4vnl9+qSkPuHw5odLSmHMMj4eqxYupWryYQgBNw9q9O47+/WvKA0ZKUQohhBAHSvMPsOvJXtBdFlLP7U5h/ubwEG+QULkPPdHasHPbT0op0m3pbPNvA6BY18gIBrEHyvD7vJgtTXv+QjQkzWoNB8r9+8PlYIRC+DZsqCkPuHQZ/h07Yk8KhfCuWYN3zRqK//c/AMyZmdGHJu39+2PNzkZpTf9vvIQQQjRdzT7A9pRCRbEHZ4otZr8tJwWtfHt0O1jmQ1n0Jp+P7bK4sOgWfEEfIRRFukZGMERVaS7mjLotqoUQYUrTsObkYM3JIWX8uQD4c3NxL1sWDbq9v/4Ge9R09+/YgX/HDsreeRcALTERR79+0Ycnbb16oVnly60QQoj4Ne1oMw5GCN57+if+MqU/lj2CZ81mQll1DG8QgECxB3Orpp2PrZQi3Z7OzoqdABRpOmnBcPv0YLAdut7s/5EJ0WDMrVtjHjmSxJEjAQhWVOBesbKmPODKlRgeT8w59bZ57907WqnE0a8fenJyQ38UIYQQzUiLiNYKt1fwyaxfGHllHzStVp61AlOKDX9eFYQMCDaPfOwkaxJ5VXkEQgECSlGiaaSGQlSU5OFMa9fY0xOi2dKdTpxDBuMcMhgIt3D3rFlTUx5w6TKCRUUx5xh+P+5ly3AvWxbdZ83Jxl7d5r3/AMyZ7Zr0f1OEEEI0rBYRYANs/rmQb+etZ8g5OTH7lUnDlGojUOAGqvOx/eiJTbd0l6Y00uxp5FbmAuGHHVNCIazeQkKhNmiSHyrEAaHM5nCVkT594OJwm3ff5s3RtBL30qX4tmypc5533Xq869ZT8uqrAJhat44pD2jt3h2l6w39cYQQQjQRzT7ANtVKvV75+TaSW9vpdUL7mDGazYTmshAq9wEQLPOirBqatel+/BRrCvlV+YSMED6lKNc0EkMBKssKSUjOaOzpCdEiKaWwdu6MtXNnkseOBSBQUEDV8uU15QFX19PmPTc3ts17QkJNm/f+A1B7pKEIIYRo2ZpuhBknswO69Mtg4/J8AL56dR2J6XY6Hp4WM05PtGB4gxi+SD52UdPOx9Y1nVRbKgXuAgAKdA1XKIRelY+RlC5/HS1EAzGlp5N40kkknnQSEG7h7v7p52ilEvfy5YSqqmLOCVVWUvnNN1R+8w0ArYB1badhzc6u9dMVS9dsdKfUuBdCiJam2QfYACde3JO3ipaRt6UcI2Tw0YxV/OXGATFjlFKYUvfIxy72YkqzNdlgNdWeSqGnEMMwcCuNKqVIMLy4K0qxu5Jjxs6bN4+zzz5bui4KcZBpDgcJg44mYdDRABiBQLTNe3XQXafNOxDYtYvArl1Ufv11zH5Tu7ZYu0rgLYQQLUmLCLDNFp3TrurDvId+pKLYi88T5P2nf2LAxOSYccqkYUqxESiM5GN7AoQq/OiuppmPvWrlKo488kj6DOjDSx+8RIGukxAIYFTkwh4BthCicSiTCVvPnth69iT1gvMjbd531ORxL1uGZ8MG1B7lAasFdu4isHMvgXd2dk3wnZONpUtXCbyFEKIZaNAAWyl1KvAEoAMzDcN4aI/jQ4G3gU2RXW8ahnFPPNdOSLJy+tV9ePOfy/B7g5QXefCU+zBCBqpWZRHNbkJzWghVRPKxS73h+tjWpvdA0owZM7jiyiv474v/ZcPaDXTt1hW3UtI+XYgmLNzmvT2W9u2jbd4Xfvopx3TqhHd9+OFI74YNeNevw7d5S5187mrRwPurvQTe2TlYu3bFmpONtWtXtAT574EQQjQVDZaArJTSgaeBkUBP4DylVM96hn5tGMYRkZ+4gutq6e1dnHzZ4VRnfAQDBmWFnjppE3qSBWWpCagDRR6MYP2rS43F7XYzd+5c/jr5r5x+5um8+b83gXBFEYAXZz5DVlYWDoeDUaNGkZubG3P+hg0bGDNmDG3atCEhIYH+/fvz4Ycfxozp1KkT99xzDxdddBEul4sOHTrw6quvUlJSwvjx43E6neTk5PDxxx9HzwkGg1x66aV07twZu91OTk4O06ZNIxRZnfN4PPTq1YtLLrkkes7OnTtJT09n+vTpB+VeCdHkmUxYs7NJPPVUMq69hvaPP0bX997jsGVL6fLeu2Q+/hjpV1+N65RTsGR3BdPe1z6qg+6i2bPZNXUqm885l98GHMn64SPYOnkyudP+Scmb83H/9BOhysoG/JBCCCGqNeQK9kBgvWEYGwGUUq8AY4DVB/JNOvVOZ/DZOSx6bR0A3io/laUazuSaTmx187FDTS4fe968eWRlZdGnTx8uvvBiJoyfwPW3XU+p2czmZT9zxf/dxF133cn48efxxRdfcOutt8acX1FRwciRI7nvvvuw2+28+uqrnH/++fTp04fDDjssOu7xxx/nvvvuY+rUqTz77LNceOGFDB8+nPHjx3Pffffx4IMPcv7557N161ZsNhuhUIjMzExee+01MjIyWLx4MZMnTyYtLY1LL70Um83G3LlzGThwICNHjmTcuHFMmjSJvn37csMNNzT0bRSiSVMWSzT3mlNr9hs+H74tW8Ir3evWh1e+N6zf54q3f+dO/Dt31lnxNrdrhyUnNtXE2qWLrHgLIcRB1JABdiawrdb2duDoesYdo5RaCewEphiG8cv+vlGfYe0pza0Cwk/2V5V60U0Ku9PC9n98ve+TD6L2Dx0X99iZM2dywQUXAHDy8JOx2+0sXLCQk0afxCOzX2HEkIH8/cqLcWZ0pFu3bixZsoRZs2ZFz+/bty99+/aNbk+dOpW33nqLefPmcdttt0X3n3LKKVx11VUA3H333Tz66KNkZ2czadIkAG6//XZmz57NqlXhfHCz2cw999T8xUKnTp1YtmwZL7/8MpdeeikAffr04aGHHmLy5Ml89913LF++nJ9++qnJfHkRoqlTFku07Xu9gff69XjXb4j8uT5cq/v3Au8vv4rZHw28q/O8JfAWQogDpiED7Pqiqz1LXiwDsgzDqFBKnQa8BeTseZJSajIwGSAjI4OFCxfWvXBrA6UnR7fLCz14fY1bi7a8vDyucRs2bOCbb75hxowZ0XPGnTOON156g5NGn8Sv6zYx9sTjsPuKKS1NQtM0+vXrF/MelZWVPPTQQyxYsIDc3Fz8fn80faN6jGEYdO/ePWZeDoeDnJyc6D6HwwHA5s2b6d69OwCzZs3ixRdfZOvWrXg8Hvx+Px06dIi5ziWXXML8+fN57LHHmDNnDomJiXF//gPN4/HU+zuyNxUVFfs1/lAl9yl+B/xe2e3Qu1f4ByAQQM/Lw7RzF6ZdOzHt2o1p5070vLy9Ply5t8A7mJZGoG3b8E+78J/BNm0wbLZ6r3Mgye9U/ORexUfuU3zkPh14DRlgbwc61NpuT3iVOsowjLJarz9QSv1bKZVuGEbBHuOeA54DyO7c0Rg6dGi9b7h69Wp0s0bQH/4fjL+ycVdQXS5XXONeeeUVgsEgPXvWpKhX55Hv2rELwzCoUgpdhdADlTjT2mGL/M+v+j1uuukmFixYwPTp08nJycHhcDBx4kQMw4iOUUrhdDpj5qWUwuVyRfeZzWYAbDYbLpeLV199lX/84x9Mnz6dY489lsTERJ5++mnmz58fc538/HzWrl2Lruvs2LEj7s9+MNhstugXkHgsXLiQvf1OiRpyn+LXWPcqZsU7mmqyAd/mzRAM1nuOXliIXliIddWqmP3mzEys2dlYsruGH7DMzsbatQta5Ev4gSC/U/GTexUfuU/xkft04DVkgL0EyFFKdQZ2AOOBCbUHKKXaALmGYRhKqYGEH8Is3NdFNVMe65YuIGfAqXWOKaVIbuWgeHcloaCBETKwX9uPlDYOtFoNZoxACH9uFUSCWM1uQk9tnHzsQCDAnDlzePDBBxk1alTMsQnnT+Ctl9+ia/eu/LD8Z4IQbZ/+/fffx4xdtGgRkyZNYmykG53H42HTpk306NHjT81v0aJFHH300VxzzTXRfRs2bKgz7rLLLqNr16489dRTnHfeeZx88skMGDCgzjghxMETk2oysma/4fPh3bwZX0yO974Db/+OHfh37IAvv4zZ3xCBtxBCNDcNFmAbhhFQSl0DfES4TN9swzB+UUr9NXL8WWAccKVSKgC4gfHG73ROUZrB2s03kJjWkdad6hYl0U0aSRl2inPdYBgEAyFK890kt3ZEA2hl0tBTrASLwikkIXcAVelHdzZ8fez333+fgoICLr/8ctLSYrtRThg/gaf+/RQPPfsQk0ZN4s6nnufC00bw0eufMH/+/Jix3bp1Y/78+YwZMwaz2czdd9+N1+v90/Pr1q0bL7zwAh9++CHZ2dm88sorfPnll6SkpETHPPvssyxcuJAVK1bQuXNnLrroIiZMmMDy5cujKSdCiMajLBZs3bph69at/sC7vhzvPxh4W3OysVTneUvgLYQ4RDRoHWzDMD4APthj37O1Xj8FPLW/17WlePjhmwsZkbyAhOS0OsfNVhOJaTbKCsINZvzeIOWFHly1qoboDjOGN0io0g/Uqo9dq5xfQ5g1axbDhg2rE1wDnHPOOdxyyy1UVVZxz+P38O9pT/PIY89x/DFHceedd3LddddFxz766KNceumlHHfccaSkpHD99ddTUVHxp+d3xRVXsGLFCiZMmIBhGIwdO5YbbriB2bNnA/Dbb79xww038Mwzz9C5c2cgXKmkf//+/O1vf+M///nPn56DEOLgiAm8awn5fPiigXdN8B1P4F1RO/BWKhx4R+p3W6rreXfpLIG3EKJFUc29tXb37lbj38+0B6BqezYjz3sPUyRveM2aNTEpEZWlXipLalZxE5KtJCTVlO8zQgaB/CqMSM42Jg1zK0dMo5rGFjJCrC1eSzAU/p9aZiBAciiE29W5Tvv0PZWXlzdqLnRj2fP34PdILlp85D7Fr6Xeq5DPh2/TZnwb1sc00dlX4F2vSOBdkZxMmz59MLdvjzkzM/LTDj05WaoQ7aGl/k4daHKf4iP3KX5KqaWGYRz5e+Oafav0UNAZfe1ov57PXpvMyRNm1/sfY0eihaA/hCeySl1Z4kU3adgSwgG50hR6qp1AXiQfOxAiWOxptHzs+mhKI82WRl5VHgAFuk5SKCTt04UQDU6zWLB174atez0r3nsG3uvX49u6tf7A2zDwb9+Odft2ivd4uBJAS0ioFXBHftpnYom81hITm8x/o4UQAlpAgK3p6ZgCPQiYfgZAb/0V3753P4NH31ZnrFIKV5qNYCCE3xv+j3xZoQfdpGGOtErXzE0nH3tvUmwpFLgLCBkhvEpRoTRc0j5dCNFE/F7g7V2/LuYBy70G3tXnVVbiXbsW79q19b+f01lr1btdOPCutQquH4J/cyeEaFzNPsAGGDx8Lgs/HoGy5aE0qDTN4ZfvuqEl964zVikVfuhxdxXBQAgMg9L8KlLaJKCbwpVFmko+9t6YNBMpthQK3eECKwUmDZc/RKBsN1Z710aenRBC1G/fgfcmln/wAd2SkqP52/4dO/Dt2IFRVbXP64YqKvD++iveX3+t/30TE2tWvNtVr4DXCsCdsjAhhDiwWkSAbTI5OPaEeSz66hR0qxvdGmJr3p20T3i53vGarpHUKhxkGyGDUNCgNK+K5DYJaJF8az3JiuELhvOxDQgWeVBNKB87zZZGkacoUhNbo1Ip7IEy/D4vZov19y8ghBBNRDjw7o531y7S9sgDNQyDYEkJ/u3VQff2aODt37ED//YdGJ59NxELlZXhXV2Gd/Waeo/rSUl75H1HVsLbt8fcrp10txRC7LdmH2AHIu2BHQmZ9O//PMtXTkQzBbG4fPh9RQT9PnRz3fQOk1knKcNOSW54ZSTgD1GW7yaplR2lVCQf20YgL1zezwiECJZ40VOsTSLXz6ybSbImUeIpAaBQ1+loBKgq3Y05I6txJyeEEAeIUgpTSgqmlBTs1Z0razEMg2BRUc2K9/btkdc7o/uM3ylRGiwtJVhaiueXX+o9rqek1Fr1boc5M7MmDaVdOzS7/YB8ViFEy9HsA2y32x2tEpHe+ii6Zt3Lxu23ohQoPUR52UYSU7qhaVqdcy02E640G+WF4dUPnydARbEXV2q4K6Jm1tGTrQSLI/nYVX6UVUePPBTZ2NJt6dEAu1zT8CiF3VdCMJCJbmr2/2iFEOJ3KaUwpaVhSkvD3qdPneOGYRAsKKhZ9d6+IyYFxb9jB4bfv8/3CBYXEywuxlPPA5gAelpaOOhun1n3Ycx27dAaoM28EKJpaRFR2Jtvvskll1xC27Zt6dL9XMpL1lFQ+TwAmtlPefEmElO71LvybHeGK4tUlfkAcJf70E0ajsTwqreeEMnHrorkY5d4URYNzdz4+dhWkxWXxUW5rxyAQl0j0whSUZqHM61dI89OCCEan1IKU0YGpowM7EccUee4EQoRyC+oFXBvj8n/9u/cBb8XgBcWEiwsxPPTT/Ue1zPSsdST+129Gq5Zms5D9EKIA6NFBNh+v5+5c+dy+eWXk5iYSJ+BU/nms3XR48pcRUXJDlwp7es9PyHZSjBg4I0E0RXFHnSTwuoIr1TryZF87MhDkcHCppOPnW5PjwbYpZpOBsFo+/T6Vu2FEELUUJqGuXUrzK1bQf9+dY4bwSCB/Py9p6Ds2gWRVMW9CeYX4M4vwL1yZb3HTa1axZQfjKagZGZibtsWJQG4EM1Osw+wq1ely8vLefnll7n44ouxWCwcO3wmK1d8Ex1naMVUlVtxuDLqvUZimo3iQIiAL1K+r8BDchsNs0UP52On1c3HNqU2/l/7OcwOHGYHVf4qDMK52G2DASrLCklIrvtZhRBCxE/pOuY2bTC3aQMDBtQ5bgQCBPLy9p6Csnv37zbdCeTlEcjLw718eT0TUJhat45Z8bZkZmLJzcPTti2mVq2kEY8QTVCzD7Dt9vBDiYZhsGvXLubPn8/ZZ5+Nppmx2jIwQgqlBUGBP7gbr9uK1Z5Y5zpKU+HKIruqCAVDGIZBaZ6blDYOdJNWbz52sInkY6fb09nq3wpAia6REQyiV+VjJKXX+Y/uNddcw6pVq1i4cGEjzFQIIVoWZTJhbtcOc7t2OI46qs5xIxDAvzu3JuDeXisFZecOArtzIRTa+xsYBoHduwns3o176dLo7hRg07/+Fd4wmzFlpGPKyMDcqlU4Jab6z1qv9ZQUlPzNphANotkH2Lquc9ppp/H+++8D4bbYn3/+OSeeeCJKaSQ4O1NZuR6lDJQGHs82dFM2JnPdUna6rpFcXb7PMAgFQ5Tmu0lu7UDTFJrDhOE1H/R87Isuuog5c+Zw7733ctttNQ1zFi5cyLBhw8jPzyc9PT2632l2YjVZ8Qa8hFAU6Tqtgl7cFaW/2z5dCCHEwaNMJiztww9A1sfw+fDn5u41BSWQmxvuLLwvfj+BnbsI7NzFPgsWmkyY0tNrgu9WGbFBeSQY11NTJRAX4k9q9gE2wFFHHUVBQQE//PADAIsWLSI9PR2r1YrJZMNu64DHG17hVXqIivKNJCZ1Q9PrBsYmi05ihp3SvEj5Pl+Q8kI3ienhlfI6+dhFHlTGgc/HttlsTJs2jSuuuIKMjH2neiilSLels6NiBwBFukZ6MCjt04UQoolTFguWDh2wdOhQ7/GQz0dg167YFJSdOylY+xtOfzg9JVRREd+bBQLR1fB90vVwIF57JbxVRt2V8bQ0VD3/HxVCtJAAG+CUU06hqKiIdevCDze+8847nHXWWQBYrEkEA63xB3MB0EwByks2kpiaXW/emtVuwplqoyLSLt1bFaCyxIszxVZTHzs/ko/tDxEs9WJKObD52MOGDWP79u3ce++9PPnkk3WOB4NBJk+ezOeff87u3btp3749Z048k0lXTQJNo1jTSPaX87frr+OFOf8F4LzzzsPYYyVkwYIF3H///axatQqlFEcddRSPP/44PXr0AGDz5s107tyZl19+mWeeeYbFixdz2GGHMWfOHDRNY/LkyaxcuZJ+/frx3//+l86dOx/Q+yCEEIcyzWLBkpWFJSuL2u1u1i1cSN9IU55QVRWB/PzwT15e9LW/+nVeeDtUVhbfmwaDBHJzw6vn+5ycFi6RuGdayp5/pqWipHSsOMS0mN94TdMYN24cs2bNIi8vj1AoRGVlJYFAAJPJhD2hFcEyDyFKAVBmD+UlW0lMqb8pi8MVLt/nLg+X76sq86GbNexOC5pFR0+yECwJNy8IVUbysR0HLh9b0zQeeughzjzzTP7v//6Prl1jW6CHQiEyMzN57bXXyMjIYPHixVw++XJcyS7Gnj+WQl1n5lOzmT37eWbMnEWfPn147LHHePnll+nfv3/0OpWVlVx//fX06dMHt9vNfffdx+jRo1m9ejWWWk+u33nnnTz22GN06dKFK6+8kgkTJpCRkcH9999Pq1atuPDCC7nuuut49913D9g9EEII8fs0hyMahO9LyOOJDcTz8gnk1/ozPx9/Xj6h0tL43jgUil5v3xPU0NNSw+koGa0iq+GRP/dcETc3/nNNQhwILSbABrBarUyYMIEZM2ZQWVmJYRgUFRWRnp6OpmkkuDpw992zG21+d911136NP+200xg8eDBTp07llVdeiTlmNpu55557otudOnXix6U/8uH8Dxl7/lj8SvH4zJe58cpJnHXmGZgt4ZSTL774IuY6Y8eOjdl+/vnnSUxMZPHixQwZMiS6/+9//zunnXYaADfccAOjR4/mjTfeYNiwYUD44clrrrlmvz6fEEKIhqPZbPtMR6kW8noJ5BfUrIbv+WfkdbCkJL43DoUI5hcQzC/AS/3t6gFQCj01NRJ01+SK13lwMz1dAnHR5LWoABsgOTmZ8ePH88ILLwDhVupFRUWkpaU1yzJG06ZNY9CgQUyZMqXOsWeffZaZM2eyZcsW3G43fr+f9h3Dtb7Ly8rZnZvPMQP64C3NxZyRhaZpHH300Wzbti16jQ0bNnD77bfzww8/kJ+fTygUIhQKsXXr1pj36lOrQ1rr1q0B6N27d8y+yspKqqqqcDgcB/QeCCGEaDia1brPBzOrhXw+gnumo9RKSakOyINFRfG9sWFEm/Z49xGHA+FAfC854ubq/bWKAQjR0FpcgA3QoUMHzjzzTCoiD374fD5KS0tJSkpq5Jntv6OOOoqxY8dy8803c/vtt0f3v/rqq1x//fVMnz6dY489lsTERJ5++mnmz58f80Widvv0+owePZrMzEz+85//kJmZiclkomfPnvh8vphx5lqrBdXXr29faF/lpoQQQrQYmsWCFmmIY9/HOMPnI1BYGBN0+2NWxcOr5cGiot+vmBIRLCoiWFSE97ff9jkuIyGBDa1bo6emYEpNq/VnKqa0VPTUNEypKehpaehJSfLQpjhgWmSADeHV1eW1ivZXVVVhMpmiaRp+XwVV7k1Ux6JGUMfpykE31f/XTn5vkJLcquhDgmarTnKkm2OwwhfNxwbQU20HNB/7gQceoGfPnixYsCC6b9GiRRx99NExaRkbNmwAIMWagpFokNE6g4XLV3H64KOoKM3DMDtZvHgxbdu2BaCwsJA1a9bw9NNPR1M9li1bRuB3upIJIYQQ8VIWC+a2bTFH/t+zN4bfT6CoKDYlJSZPPB9/fh7BwqJ91w6vRausxLdxI2yMZ6IKPSUlNghPTQ3nj6fuEYynpIQDcilnKPaixQbYEC51Z7PZ8HjC1UDKysowmUzYbDbMFifWQDt8gZ0AKD1IRdlGEpNz6v0XxmzVSUy3UZrvBsIBd3mRB1eaDS3BjOENEnKHA9NgsRfNoqNMB+ZfvOzsbCZPnswTTzwR3detWzdeeOEFPvzwQ7Kzs3nllVf48ssvSUlJIc2eRpGniPMnn89/npjJEZ070Ld7d55++UN27doVDbBTUlJIT09nxowZdOjQgR07dnDjjTdikqe9hRBCNDBlNmNu3RpzJA1xb4xAgEBhUb154TF/FhTEHYiHL2xEV8Z9bPj98bqOnpISCb6rg/FIEF69Wh4Jxk1paWguV7NMVRV/TIuPpJKTkyksLMTvDzeHKS4uJj09HbPZjM2RRrDcQ9AI54cpk4/y0s24kjvXX77PYSYhOURlZLXaU+lHN2skJFnRU2yE/FUQqY8dKPRgamU/YP8y3XHHHcyZMye6fcUVV7BixQomTJiAYRiMHTuWG264gdmzZ2PRLSRZk7jwqgspyCvgqin3ojA4Z9w4Jk6cyJo14eQ2TdN49dVXue666+jVqxfZ2dk88sgjdR58FEIIIZoKZTJhbt0Kc+tWwOF7HWcEg3z1wQcMOuwwAoVFBIuLwn8WFREoKiRYVBz+M7IvGG/1lGrBIMGCAoIFBfGNN5sxpYRXwKN/7iUY11NT0RISJCBvxlp8gK1pGqmpqRQUFBAMBmMqi+i6ToIrk/JSL4aqDJ+gV1JZthNnUv05y45EC8FACE9FOGCvLPGimzRsCWZMqTYC+VVggOEPhutjJ+9/fezqBzRra9WqFeXl5TH7Zs2axaxZs2L23XHHHUC4fXqpt5Sb77uZm++7mWy/H2Xo+B2tsSfWPPgxfPhwVq1aFXONilpNCzp16lSndvaRRx5ZZ9+pp55aZ58QQgjRWJSuY7hcWHNysOb8/njD7ydYUkIgsoodDsYLw9uFRQSKi6LBeKCoiNAe/0/+XX5/JO0lD+/vj0ZZLHsPxmsH5ZHUFU0KDDQpLT7AhnA79eog2zAMgsFgtLKIpmk4EztRVroOpYUf7AupItyVVuwJdZ9AVkrhSrURDBj4PeGUkLJCD7pJYbaa0BOtBEsj9bEr/ISsOpq94csJ2Uw2nBYnFb5wsFyoa7QLBLBU7cBftZsqSwrWpFaY62kZL4QQQhxqlNkcrckdj5DPR7C4mGBhIYGi4nqD8eqV8mBhIaGqqv2ajxHp4hnYtSu++dvt0WD8dx/qTE1Fsx3YBnki1iERYEO44kVKSgpFkXJBfr+fkpISUlJSUErD5epKeflalBYEwBfYhe6xYbE561xLKUVSuo3i3VUEIykhpXluUtokoDnNhLxBjEjwHSj2YjYfuHzs/ZFuT48G2CWaTgZBzICZIGZfAaG8Air1RHRXK2wJrgafnxBCCNFcaRYLWhw549VCHk9k9TsSjNeXrlJUHbAXYUSeH4uX4Xbjd7vx79wZ3/wdjmgwnmwY7PzoY/Tk5Pp/UpIxJSejajWgE/t2yATYEH7oMTExkbJIu1iPx0N5eTmJiYlouomEhM5UVm1AKQOlwO3Zgm7KQTfV/YXSdI2kVnaKd1dhhAxCIYOS/CpS2iRgSrHizwtBMAQhg0CRB1PGgcvHjpfD5MButuP2uzGAnaYE2gU8mAl/idAUJITKoLQMT6mNoCMde2J4VV8IIYQQB45ms6G1a4e5Xbu4xoeqqmqlq+yRMx6TT15EsLAQI/KsWbxCVVWEqqrwb9uGFSj96eff/wwOxz6C8JQ6QbmenHzI5pLHFWArpXoCQcMwfotsnwRcCPwCTDMMI3jwpnhgJSQkEAgEqIr8VU1FRQUmkwmHw4HJbMdm7YDXuxUUKC1ERfkGXEnd0LS6tTFNZp2kDDsleeG866A/RFm+m6RW9pp8bMDwBQmW+jAlN2w6hlKKdFs62/zhxjIVmp+tjgSchgWXuwIHNd+ObXigajv+ql1UWVIj6SPyTVUIIYRoDJrDgcXhgPbtf3esYRiEKiujq981gXf9wXiguBj+QEneaFAe5yo5AGYzelISenISenJyOI1lb0F69U9SEqqZVzSLd/azgCeA35RS7YG3gYXA1UAicMtBmd1BoJQiKSmJYDCI1xvOlS4pKcFkMmGxWLDakggGWhEI5YXH6wHKSzeSmJxd7zcwi81EYqqdssJw+T6fJ0BFkRdnqhU9qXY+ti+Sj92wvzAuiwubyYYnEA6mPQEPHjwU23RcejpOjxdXoAJNRep7E8Tsyw+nj5hc4fQRh6SPCCGEEE2VUgrd6UR3OrFkZf3ueMMwCJWVRYPxlV99zWHtMwmWlIR/iktqXlf/lJZC8A+sp/r9+1dtJUJLTKwVdCfVCcJN9ayaa/Z9tTxqWPFGez2AZZHXZwM/GIZxmlJqGPA8zSjAhvAvYkpKCgUFBdGmKtWVRUwmEw5na8pLPRgqnEqidA8VZdtwJXWs93o2p5lAIERVJJh2V/jQzRp215752B7MZkeD5mMrpeiY2JHcylzKvGUYhAPpYChISaiMEh2ctmQcfkj2ldVKHzFICJZBSRnuEhshSR8RQgghWgSlVHhVOSkJOnfGW15OytCh+zzHCIUIVVTEBt3FxeHKK3sG4yWl0deG2/2H5hgqKyNUVoZ/69b4P5fVWm/+eL3BeXVQnph4UBoGxRtg60B17+wRwAeR1xuA+LL7m5jq8n35+fnhb3KhUDTIDlcW6Uh5yXrQwyu/hiqlqiIXh7P+j5uQZCHoD+GtCudAVRSHK4tYUqz484IQNBotH9usmWnvak+JUYLf5KfYU4w/VJOrVeGvpAIosdtxGhYSvVUkGDXpI/Y90kdsSa0wSfqIEEIIcchQmoaemIiemAgd619wrE/I46m7Gr6XlfJASTHBklJCZWXwB0r/Gl4vgdxcArm58Z9U/bniSV1JSY77svEG2KuAK5VS7xEOsKtXrDOB/Vvzb0JMJhOpqakUFhYCEAgEKC4uJjU1FaUUzqQulJeuRemRFehgHl63Das9qc61lFIkptkoCYbwe8OrwGUFHpJbOyL52OFvcIYvSLDMhymp4cvj6Uon2ZFMuj2dcl85xd7iaJURAF/QRxE+SiwaTj0Vp9dHUqACLfJdQNJHhBBCCLE/NJsNrU0bzG3axH2OEQwSLCurP1Wl1sr5nj/7+6AnAKFQ9PwDKd4A+2bgLWAKMMcwjOpHTc8AFh/QGTUwq9VKcnIyJZEb6/V6KSsrIykpCU3Tcbq6UFGxHqWFQIHHuw3dZMFkrpvnozRFUoY9Wr7PMAxK892ktHHE5mOXR/KxbY2TwK+UItGaSKI1EW/AS5G3iBJPCSEj3FI2ZIQoC1RQpkOxNZGEgCLFV4GlvvSR0kj6iEvSR4QQQgjx5yldx5SSgiklJe5zDMPAqKqqm7KytyA98hOq1VzvQIorwjMM4yulVAaQaBhGca1D/wH2r3J6E+RwOAgEAtEOhpWVlZhMJhISEtBNVuz2jrg9m1EKlGZQWbEJV2I3NL3u7dN0LRxk50bK9wVDlOa7SW7tQNXOxy7yYG514PKx582bx9lnn73f3RStJittTW1pZW9Fqa+UIk8R3kBNjyl3wIMbKLFZcGIlyevGadQctxseqNyOv1LSR4QQQgjROJRSqIQEtIQEzJn1d+Ouj+H3Eywt3Uc+eWxOOb/9Gtd1415CjZTiK458CDswGFhnGEZe3J+iCXO5XAQCATyRwu6lpaXouo7NZsNidREMtMUfDHdTUnqQ8rINJCbnoFTdANlk0UlKj5TvAwK+IOUFblxpNgL7kY+9fPlyjjzySAYNGsQ333xzkD55mK7ppNpSSbGmUBWooshTRLm3PPpQZCAUpIQqjjhiKBdePolbrr2ClEAlmoJ7H5vBHdOfYd5z/+Ss006MpI+05tLLr2Dbtm189dVXB3XuQgghhBB/hDKbMaWnY0qv2727/hPie4YuruVTpdQLSqmrIq8thNNCPiZctm9kfDNq2pRSJCcnYzbXtDUvLi7GH8nnsSeko5FaM173UVG6ea/Xs9hNuFJr2pB63QGqynyYau2rzsfemxkzZnDVVVexatUq1qxZ80c+1n5TSpFgTqCDqwM5KTlkODIwaTXfwwYOGcg3X3/Lbt3PBruTXZZEPv9uKR0z2/DFt0ui6SO2knV88fmnDD7maIxQqEHmLoQQQgjRFMSbn3AK8H3k9RmAC2gD3BX5aRGqK4tU5xIbhkFRURHBSN1HZ2ImBB3R8YZWSWX53out210WHK6adImqMh9eXwg9sWZfqNxHyFO32Lvb7Wbu3LlcfvnljBs3jlmzZsUcf/HFF8nKysLhcDBq1Chy93hidsOGDYwZM4Y2bdqQkJBA//79+fDDD2PGdOrUiXvuuYeLLroIl8tFhw4dePXVVykpKWH8+PGkJKUwuO9gNi3ZRHtXexxmBwOHDGTFkhX4vD58IR+7vWV8v+wnrvjb1Xz27dLotddu2MKu3XmcOrA7gd2/UFGwnYB/718mhBBCCCFaingD7BSgOhXkVOCNSGrIK0DPgzGxxqLrerSKCEAwGKSoqCia2+xK7owRqFnlDhqFeKoK93q9hBQr1lrNZcqLPATNOspa0xkyUOTBCMau8s6bN4+srCz69OnDBRdcwIsvvhhdTf/hhx+46KKLmDx5MitWrGD06NHccccdMedXVFQwcuRIPvnkE1auXMnYsWM5//zz+fXX2Nyhxx9/nIEDB7Js2TLOOeccLrzwQiZMmMBpp53GihUrOP7445l0wSSshpXOSZ0ZN3IcHreHVctWAbDyx5UkpSRx8tmnsmHzVhaXecnTE/js2yXYbTYG9e+NmQBOXz5a3moqczfiqTo4DxQIIYQQQjQF8eZg7wZ6KaV2EV7NnhzZ7wT+QE2UxvPZ510b7b17d10OQGmBm5QMO4bfA6Fa+djpNfnYM2fO5IILLgDghBNOwOFw8M477zB27FieeOIJRowYwdSpUwHo1q0bS5YsiVnl7tu3L3379o1uT506lbfeeot58+Zx2223RfefcsopXHXVVQDcfffdPProo2RnZzNp0iQAbr/9dmbPns2qVas48sgjOfyww2nfvj3rl67ntBNPY+m3Szlq8FHYHXZ69u3JV18vIuGskbz3wwoGDOyPYbVDTPWRUigpxV1qJ+RIx+FKPSgF3oUQQgghGku8kc1s4FXC9bCDwGeR/UcD8T1OKdD0cPBshAxKCz3oyTW1sA1vkFB5OIVi/fr1fPPNN0yYMAEI50VPnDiRmTNnArBmzRqOOeaYmGvvuV1ZWclNN91Ez549SUlJwel0snz5crbu0RGpT58+0ddOpxOHw0Hv3r2j+1q3DjfWycureZZ12LBhfLnwS9Lsafz0/U+cOuJUXBYXRx17FIu/CVdt/GHRD/QfciQbrSY2W12UKCu165vYDTcJldsi6SM7CPyR2pVCCCGEEE1QvGX67lFK/QJ0BF43DKM6mTYAPHywJtfSJGU4KMmtwjAMgoEQZWU+XC5LNLAOlvlQFp2ZM2cSDAbpWKtTUnWKyrZt2+IqxTdlyhQWLFjA9OnTycnJweFwMHHiRHy+2Dzo2g91QjiYr72vekU9VOtBxeHDh/PXv/6V4uJifvjhB2bPnk3HxI6cefKZXH/99Wxet5nC/EIGDhmIYRhU4qXSDPlaAq6AQUbAjR4Jt80EMPvyCOXlU2lKxJTYCqvduT+3VQghhBCiSdmfMn1v1LNvzoGdzsE3YviG/RpvGAaFhYUxgWl6ejoWS/hBxcry3QSN/JoTQnZcSV33WnovMd1GaaSro98bpFLXcFh1jEj3R09+BXPmzOHBBx9k1KhRMedecMEFPP/88/Ts2ZPvv/8+5tie24sWLWLSpEmMHTs2fF2Ph02bNtGjR4/9+vz1GTZsGF6vl0ceeYSMjAy6dg2n3Qw9YShbNm/h2/e+xel0ctTAo/AbNSvTvpCfQg2KrVachok0nxvHnukjxaW4S+wYjnTskj4ihBBCiGYo7gBbKdWHcCfHnoABrAam1+rq2CIppUhJSaGgoCBaTaSoqIj09PRwMxpXG8qKPaCXh0/Q3FSWb8eZ2KHe61kdZpwpISqKw81avFV+TC4LZi0EIYMPPl5AQUEBl112Gel71GQcP348zzzzDHPnzmXIkCE8+OCDjBs3joULFzJ//vyYsd26dWP+/PmMGTMGs9nM3Xffjdfr5UDIysqic+fOPPnkk4wZMya6v7payb/+9S+OO+44uqV1wx1wU+QpotRbGl15DxkhyvBRZtGxKxtJfj+pIR/VX0nshhsqt+Gv3IXXkootuRUmk7memQghhBBCND3x1sE+A1gGdAA+BBYQThdZppQaffCm1zTsWVkkFApRVFQUTZtwJWdhBGryqUOU4K7ce/8du8uC3VlTqq+y3EfIEf6u88IrL3LCMceRYnHVOe/ss89my5YtVFRUMGvWLJ555hn69OnDm2++yV133RUz9tFHH6VVq1Ycd9xxjBw5kkGDBtXJ0/4zhg0bRnl5OUOHDo3ZP3ToUMrLyxk+fHj4s5rsZDoz6ZbSjdYJrbHosV0e3Yaf3SZYa7Wz02THT83Kf7j6SB5a7i9U5m7E65bqI0IIIYRo+lQ8+bxKqZ+A+YZh3LnH/nuAMYZh9K3/zIOve/fuxm+//VbvsTVr1hyQlIhqXq+XwsKaknxWqzUaeIeCQcpL16JMkZrWBthsWVisifVeyzAMSvPc+KprYCtITjCDu6YmtinDjmaN+y8Zfld5eTkuV93AvSEZhkGFv4IiTxEVvroBs1IKh2Ei3e/BaQTrHHcrO4YjA3ti6l7TcPa0v78HCxcurPPFQdQl9yl+cq/iI/cpfnKv4iP3KT5yn+KnlFpqGMaRvzcu3gTXbsB/69n/X6D7/kysObNarSQlJUW3vV4vZWVlAGi6ToKrC0YwEvQpcHu2Egh46r2WUorEdBu6OfKPwICyqgCY910fu7lTSuGyuMhKzCInJYc0exq6VvOZww9F+tli1llndZCvWah9B+yGG0flVvy7VlFRuINAQKqPCCGEEKJpiTfAzgMG1LN/AJBbz/4WKyEhgYSEhOh2ZWUllZWVAJjMVmy2LKr/UkApg8qKjYRCdTs1Ami6RnKGA02rTj0xqAiEILJN0CBQ7I2rakhzZNEttEloQ7eUbrRztsNmssUc9xkB8kzwm9XGdpMdb630EQsBnN48VO4vVORuwuuubOjpCyGEEELUK978gxnAf5RS2cC3hB9yHEL4ocd/HqS5NVmJiYkEAoHoQ4OlpaWYTCasVitWu4tgoA0BYzcASgtSUbYBV1K3elMadLNGUis7xbluMAwCgRAei44tUsbO8AQIVfjRXZY657YUmtJIsaWQbE2OPhRZ5iuLeSiyVINSixk7JtL8XhKNIArQlYEzWALFJZHqI/uXPiKEEEIIcaDFG2DfB1QANwD3RvbtBO4EnjwI82rSalcWCQTCq9PVlUXMZjMOVwblJR4MrSR8guajomwzrqTO9V7PbDWRmGajrCBcvs/jC2Ky6pgC4eSIYKkXZdHRarVXb4mUUjjMDhxmB4FQgGJPMcWeYvyhmjQQNwG2m3VMWEgOBkkP+qi+K+HqI1vxVe7EZ03DlpQh1UeEEEII0eDiShExwh4zDKM9kAQkGYbR3jCMJ4wmnr9wsKanaRqpqalokTrNhmFQVFQULeXnTGqPEXDUzENVUFWxa6/XsyWYSUiqqURS4Q1i6DWrsC0xH3tfTJqJDEcGOSk5dHB1IMGcEHM8QJACHX6zWNlqsuFWe6aP5KJyf6F890YC/gNTnlAIIYQQIh773cXDMIxywzDKD8ZkDjSz2Yzb7T5o1zeZTKSmpka3g8EgxcXFGIYRfpgvuTMhf80KaiBUgNddtNfrOZIs2BJqxpf7QlAdOAZDLTofe2+UUiRaE+mU1Ins5GxS7aloqubX1sCgXAux0WxmvcVOkaZHH4rUlYHVX4KtZAM/PziUFZ++TDBQfz68EEIIIcSBstcUEaXUz0Bc0ZxhGH0O2IwOoFatWrFjxw4yMzOx2+0HJS/XYrGQkpJCcXExAD6fj9LSUpKSktA0DVdSF8rL1qGZwmGf17cDzWTFvMeKLEQqbKTaCAZC+L1BQkBl0CChutDIIZCPvS9Wk5W2pra0srei1FdKkacIb6BmddpLkF0mnTzMJAZDpAW85BVXkLT5fXK8y2HRX9nxzZ1syz6fHqddRVJK+j7eTQghhBDij9lXDva8BpvFQZKYGK5BvXPnTvz+g1vOzePx4PHUlOSz2+1YreGUj6Dfj89fiNIiDy4aeVitrdG0+nOqjZBBZZkPIxgeb9GI5hmjQHdaUKb9byHu8Xiw2Wy/P7AZCQQDVPor8QQ8GLW+D+4gvLpdULqOTjs/IgNQQKaRS+a6R6h6/Cl+yDidNideS9Zh/Rtr+kIIIYRogfYaYBuGcXdDTuRgSUxMjAbaB5NhGLz11lusXLkyum/8+PEcdthhAKxZ/Brb8qeiW8Ir2aGqNgw79TNMpvoD3uLdlbwxbSneqgAKGJZqwRUKB5B6ipXW1/VHs+9fE5qFCxfSr1+/P/Dpmr78qnzmrZvH67+9Tr47P/Zg21TahdpzXlEeZ1cUk2AYOJSXowvehFfe5CfbAIyBk+l9wtloest+kFQIIYQQB9/+L4OKeimlGD16NB07dozue+ONN9i1K/xgY4+B5+AMXYgRSRDWHLv55rMJe82pTmmTwKlX9EbTFAbwfYmP6h42wWIvRfPWHnL52PuS4cjgyr5X8tG4j5h+wnSObB3bZGmnVsUj6U6GdurCzWkd2Wiu+XLSx7OUvl9dwc77evL93HvxSUt2IYQQQvwJEmAfQCaTiXPPPZeUlBQA/H4/c+fOjXZ7PGbkVEIFx0fHB8wr+XHRjXu9XvvuKZwwMdwosyoESytqHtDz/FJI5Xd7r0pyqDJrZk7pdArPn/o8b57xJud2Pxe7yR497sHPB4kwpn07zm2Xw8d2B9V3tb2xm0FrpzP0+wtZ9eAJfDdnKutWfE0oWLdluxBCCCHE3kiAfYAlJCQwYcKEaP51eXk5L7/8Mj6fD6UUw8f+B/fOrtHxZf75/PbzjL1er+fgdvQ/JbwqvstvsNFbE+yVvL8R3w5Zbd2bnJQcbht0G5+d/Rn/GPgPOu9Rh3y11csNbdIZnpXNk8npFERKLlpUgF7eFRyz6Sly3hpF6b1ZLJ0+hsVvPMauLb81xkcRQgghRDMiAfZBkJGRwTnnnBOtWrJr1y7mz59PKBTCZLYw/IxXcOfWlPfbtvthdm37Yq/XGzSmK136ZQDwiztESeThR4IGhXPXEPJI6bl9cVlcTOwxkbfHvM2Mk2cwouOImFJ/xZqPGSkOTszqyNWtslhis1L7kdgUyhlQsZCBP99F2+cHsu3uHvzw1MUs//glykoKG/4DCSGEEKJJkwD7IOnatSunnXZadHvNmjV88UU4iHYkpjJw8It4S8IPOCrd4OdfrqKsdH2911Ka4sSLe9Iqy0UI+LEygD+Sfx0s9FD85jrJx46DUopBbQfx+LDHWfCXBVze+3JSbbXqmBPiqwSDS9q2ZlDnLozPPIwnklqxwmqJCbg7GDs5uuBN+n17NY7Hsvn1/mP4btYUfv3hY/w+aWojhBBCHOriKkOhlJq0l0MG4AHWG4ax/IDNqoU46qijKCgo4IcffgDg66+/Ji0tjSOOOIJWHXuQU/gIG3Zeh9keRLf4+OGbcxl64ueYLUl1rmW26Jx2VR/mPfQjFcVeVlQFOSoh/I/P/VMBlV124xzUtkE/X3PW1tmW6/pfx1/7/pWPt3zMK7++wsr8mgowPgL8YgnwS6qNmbTBgk53r8ZxlYUc66mip9eHGTCpEIf5V8O21bBtBhUf2FmV0A9fx+Np238kHbL7oDT5HiuEEEIcSuKt8/Y0YAHMEG2Up0F0Yc+slFoOnGoYRn495x+yTjnlFIqKili3bh0A77zzDikpKWRlZZHT71RK8q+lJPAEmslAs5Ww6POzOeHkD9C0uv9oEpKsnH51X97851J2eoNs8gbpbA2XlSt5bwOWji4s7ZwN+vmaO4tuYVSXUYzqMorVhauZt3YeX2z8goJAQcw4H0F+tgb52ZrIv0nEamj09Bic4C7iKI+Xnl4fJsCp3PSr+hZ+/RZ+fYjdpLM1eSBaznC6DhxFSoZ8CRJCCCFauniX1s4BlgODAVvkZzCwFDgL6Ee4j8ejB2GOzZqmaYwdO5ZWrVoBEAqFeOWVVygqCrdMP+rka9GKa1JJQpYN/PDVFXtN+Uhv7+Tkyw5HKVjlDlFanY8dMCia+yshr+Rj/1E903pyxzF3cGfmnXw09iPuH3I/Z2afSaYzs85Yrwqx3G7weGoKE9u1YVCnTlzSOpPZSS5+tliilUnaUMDAkg84cskUkp7qwfp7+/Pdf65l1ddv43FXNuwHFEIIIUSDiHcF+1HgIsMwfqi17zul1N+B5w3D6KGUugH47wGfYQtgs9k477zzmDlzJpWVlbjdbubOncull16K3W5n2LjH+PB/W7G1/xmAqtBCfln2ML0G/KPe63Xqnc6Qc3L4+tV1/FgZ4ASXCZNSBArcFM9fT+q53Q9KW/hDSTtnO85wnsEZXc8AYEfFDn7c/SOLdy9mye4l7KqMLZHoVSGWOHSWOMIlGm0hRX+Ph0GeKga6vXT3+TApg+zgBrJ3bYBdL+L+1MJP9t5UtT+OjL4j6XL4QEknEUIIIVqAeAPsTkBVPfurIscANgEpf35KLVNKSgrjx4/nhRdeIBgMUlBQwOuvv87EiRPRdZ0RY1/i4zdPISFzNwC7i2eQuKE7HbueVe/1+gzrQEmum58XbmdlVZAB1fnYK/Kp6ppMwlFtGuyzHQoynZlkZmcyJnsMEA64F+9azI+54aB7d+XumPEezeBbh5VvHeFyjfYQHOl2c7THw5EeD4f5/NiVjz6epbB+Kax/nMI3ktiUeBShzkPpNHAUrTI77zkNIYQQQjQD8QbYi4FHlVIXGIaxG0Ap1QaYDlSvaucA2w/8FFuODh06cOaZZ/LGG28AsHHjRj788ENOP/10bAlOhoz4H998PQZHRgVKg9823IwrqRMp6fW3Nx9ydjal+W62/lJIujdEljW8+lnyzgYsHVyY2yQ02Gc71GQ6Mzkr5yzOyjkLwzDYXrGdH3f/yJLdS1i8ezG5Vbkx490afJ1g5+uEcNObhJDBkR4PR7k9HOXx0N3nJ41S0so+hZWfwsrb2Kx1YHf6MdgPG0H2UaeS4EpuhE8qhBBCiP0Vb4B9GfAWsFUptZNw9ZBMYC1wZmRMAnDfAZ5fi9O7d28KCwtZuHAhAD/++CPp6ekMGjSI1Had6H3406zZcBkWlx/NFOTHJZMYMvRj7Pa6D8dpusYplx3OG/9cys87K0kxKRJ1heEPUTh3Da2u6Ydm0Rv4Ex56lFJ0cHWgg6tDTcBdvp0luUuiAXdeVV7MOZWa4kuHnS8d4YDbGQyFA26PNxpwdwpto1PeNsh7Dd+XOqutPSltO5jU3qeQfcTx6KZ4//UVQgghREOK6//QhmGsU0r1Ak4GuhN+oHEN8IkReRrPMIy3fu86SqlTgScAHZhpGMZDexl3FPA9cK5hGPPimWNzcsIJJ1BQUMCqVasA+Oijj0hNTaVbt2506jWEkvxbyfPci24NoVmr+PbLsxl60ifour3OtSx2E6df3Yd5Dy9lSYWvJh87z03JW+tJPad7Q3+8Q55Sig6JHeiQ2IG/5PwFwzDYVr4tGmwv2b2EfHdssZ0KXWNhgoOFCQ4AEoMhBkQC7oEeDzk+Pz19P8OWn2HLs5S9l8D6hP74s06g/ZGnkdnl8Mb4qEIIIYSoR9xLYJFA+qPIz35TSumEy/2dRDiVZIlS6h3DMFbXM+7hP/o+zYFSijFjxlBSUsL27dsxDIN58+Zx6aWX0rp1a44YNomFr68lYH4ZpQGWXXz35SQGD3sVpeo+BJeYZuf0K/sw/9Fl/FQVpH8kH7tqWR7WrskkDGjdwJ9Q1KaUomNiRzomdmRst7EYhsHW8q3RYHvJ7iUUuGPLApbpGl8kOPgiEnAnBYMMiATbR7q95Pgr6V/5Naz+Glbfx07Vmm2pgzDnDKfrwNNJSs1ojI8qhBBCCPYjwFZKHQ2MAFqxR3k/wzCui+MSAwk3pNkYud4rwBhg9R7jrgXeAI6Kd27NkdlsZvz48cyYMYPS0lJ8Ph9z587l8ssvx+l0csLYe/jgxU3YOn4PgJdlrPjhH/QbNK3e67XunMiJF/XkoxmrSPeF6GiJ5GO/tR5Leyfm1pKP3VQopchKzCIrMYuzu52NYRhsLtvMkt1LopVKCj2xLdhLdZ3PExx8Hgm4k4NBjvR4I3ncXrL9ubQrfBsK3yb43fWsNedQ2HowiYefTM6A4Vistsb4qEIIIcQhScXTYlspNQWYBqwHqnOwqxmGYQyP4xrjCDeiuSyyfQFwtGEY19QakwnMBYYDs4D36ksRUUpNBiYDZGRkDHjttdd+9zM0VRUVFSxfvpxgMAhAYmIiffv2Rdd1gn4fxRvvJSVnZ3R8wH0u1oST93q9/NUGhT8ZnOAy4dLDpfq8ToPtx4Qod1fgdEojmt9TUdG498kwDHIDuaz3rGetZy3rPespD5Xv85yU6oDb7WGgx0tXv5/qQo2VhpXVpp7kJvaFtv1ISO94QMoBNvZ9ak7kXsVH7lP85F7FR+5TfOQ+xW/YsGFLDcM48vfGxRtgbwMeNgzjqT86IaXU2cApewTYAw3DuLbWmNeBRwzD+F4p9QJ7CbBr6969u/Hbb7/90Wk1CWvXruXll1+ONpfp1asXY8eORSlFaf5uvvxoFAntigEwQorePWfSut3Qeq9lGAaf//dXdny/ixNcJvRIPeyEo9qwMm0HQ4fWf56osXDhwiZ1nwzDYFPppmhKyY+5P1LkKdrnOanVKSWRKiVd/IFowJ1HKluSjkJ1HU6ngaeT3qbDH5pXU7tPTZncq/jIfYqf3Kv4yH2Kj9yn+Cml4gqw400RSQQ++HNTYjtQ+//k7Qmvhtd2JPBKpElKOnCaUioQzwOUzVm3bt045ZRTWLBgAQCrVq0iPT2doUOHkpTRhv4DZ7Dip/Oxp3pQmsFPP1/JINe7uFzZda6llGLohO68k+/mpy2l9HOE/xFXLtlNYk+FETJQmjShaU6UUnRJ7kKX5C6MP2w8hmGwoWRDtErJj7t/pNhbHHNOka7zSYKDTyIpJanBYKQkoJejPGUcWfoRatlHsOxmNmqdyMs4BkePk8g56mTsCa7G+JhCCCFEixFvgP0ycCrw7z/xXkuAHKVUZ2AHMB6YUHuAYRjRzhq1VrDf+hPv2WwcffTRFBQU8OOPPwLhb5NpaWn07t2b9t36UZL//+39d3gk933n+75/VdU5A42MyZEzQ3LIYRBJicHKlhxkeS3Ta3ttX9nrPetd+5yzd9Nzn72795x7dn033LN+du/xsWVpj5MsWpa1CrRkUdIokAoMEjWcyOFE5Bw6d1X97h9VaHSjG0DPEANght/X8+Dp7qrqRqE5BD744lvf3//K0Ow/IxB1MAIVvvf8z/P4j32FYLB5bR/TMnj/b97Np//di1zPV9nh92N3nzEY+90XiZ7oJnZ/D1a2eSqJ2P6UUuzP7Gd/Zj9PH34aV7te4PYvmHxp/CXmynMNz5kxTb4cj/HluNeL32k7PFgbCzjEw+OfRI1/ksrXLV4LH2Ox/+1k730f++5+FMOUUY9CCCHEjWg3YF8H/o1S6jHgR0C1fqfW+j+t9wJaa1sp9Vt400FM4ONa69NKqd/09//+DZ35HUYpxfvf/35mZma4dOkSAJ/97GdJp9Ps2LGDY499iLm/fp2i/QcYlkYFZ/nON5/mHT/2OQwj2PR64ViAD/7Wcf76//MSGUcT9/uxnfkyi1+7zuLXrmP0x0i+rY/ovV0YIZmpfLsylMGBzAEOZA7wC3f9Aq52uTh3sSFwz5fnG54zbZl8KR7jS37gztYCd4kHi69x9PIPUZf/C7OfTXApfgJnz1PsfPAD9O48sBVfohBCCHFbabcH+/Iau7XWeu/GndKNuRN6sOsVi0X+6I/+iKkpb2xbLBbjox/9KJlMBq01f/Pf/gGhXV+pHZ8I/RgPPvoH+G01TUZen+Vv/vMP2R9QDAYMQi3aQxwFpc4IoXuydD3cRywVujVf3G3iTutFc7XL67OvNwTuhcrCms/ptm0e8Be9ebBYZqft9XBfV/2MdL6N4MF3MuVmePf7PrA5X8Rt7k77N3WryPvUPnmv2iPvU3vkfWrfhvZg17duiFsrEonwC7/wC/zhH/4hxWKRfD7PJz/5SX7t136NcDjMu//uf+aL/+3DJPafBWCx/DXOnfr33HXPP235ev0HMrzrH9zDd/76DU4P5egNKHYEDXosheGHclNDbKoIX7vO0FeuMWEaVHcm6DiYoWdPiq4dcSxZEfK2ZSiDQx2HONRxiF888ou42uXC7IWGwL1YaZxSMmFZPBu3eNavcHfbttdOUpznwbnPseOFz+Bogzde3MV06hgMnKDz4CPsOnw/VqD5LypCCCHEW4n0BWxDHR0d/PzP/zx//Md/jOM4TExM8OlPf5qnn36aQDDEu/7OH/OVv/4gyd3jAAxP/p8krh5kcNdPt3y9XUc72XW0k69+5esc2nEv41cWOHdxDuvaIr1akzSXq9oxU7EHDdcWmHxjjhcrLmMOpAfj9OxJeh+7k6S7o3Kx5G3KUAaHOw5zuOMwv3Tkl3BcpyFwvzz+MovV5sD9xbjFF/3A3eMH7ntLkxzLfZmDpz5P8BQUdZDXgweY77iHwM4H6L3rMfp3H9qQsYBCCCHE7WLVgK2U+j3gX2it8/79VbW50Iy4Abt27eInfuIn+OxnPwvAxYsX+du//Vve//73E8908MhTH+f73/k5Yr15lIJzF/4pseRuMpnjq76mGVAMHu5g8HAH4I1/W5guMvmDSSo/miI6VcCq6xjqChh0BQxsrRmeLHBtOMdr3xgGIBS16N69HLh79iSJxKVyeTsyDZO7Ou/irs67+OWjv4zjOpyfPd8QuHPVXMNzxi2LL8QtvuAH7oDWHC5XOFaucHflMsdmzrNr/JMYL8IsCa6FD1PoOk50z0PsOPYYHd0DW/GlCiGEEJtirQr23UCg7v5q1m/iFjfl+PHjTE9P861vfQuA733ve2SzWR588EF69xzm0OR/5I3hf0QoVUWZDi9//5d5+xNfJhzua+v1lVKkslFS794F796Ftl3yr02x8MIozrWF2txkSyl2hRS7QgY5R3O94nK9aHP9zAzXzyzPY052RWphu2dPkq7BBGZAKpe3G9MwOdJ5hCOdR/h7R/8ejutwbvYcL46+yIvjXuDOV/MNz6kqxalwiFPhEJ/EG/MXd12OlivcXS5zrPwad4+8Qvf1P4RvwojqZjR2hGrvfST3v43dxx4hGk9txZcrhBBCbLhVA7bW+qlW98Xmeuqpp5iamuLsWa/n+tlnn6Wjo4N9+/Zx6KF3M/v532a+/J+wQi4qkOc73/o53vHUl7CsG18aXVkG8ePdxI934yyUKfxggvxL49iTxdoxcVNxV8TkcNhg0vbC9mhV4wALk0UWJou8/qLXumJYiq4diYbQncxGVr0gU2xPpmFytPMoRzuP8ivHfgXbtTk/c55PvfApSukSr029xvXF603PyxkG34uE+V5keZn2btv2qtzlEsfK3+HoG98gcVHj/I3isrmLydRRdP8JsoceYefhEwSCb+0LboUQQtyepAd7mzMMgw996EPMzc0xOjqK1ppnnnmGj370o3R1dfHwB3+Tv/3ji+j+z6JMcM0Rvv/Cr/LIOz6JUjd/YaKZDJF4YgfxxwepXF+k8PI4hVcn0SVvSXelFN0BRXfAwAaGqy5XSy6zzvIfNFxbM355gfHLC/B1b1s4HmgI3D27k4SigRZnILYry7A4mj3KjyV/jCcffxKAudIcr02/xqmpU5yeOs2pqVMtV5ucsCy+Zll8zV8AB2BPpepXuac4lv8Kh05/keBpr5/7YnA/85m7sXY+QN+Rx+jffZf0cwshhNj22g7YSqmPAO8EuoGGn3Ba65/c4PMSdYLBIE8//TQf+9jHWFhYoFwu82d/9mf8+q//OrFYjHf93X/H5z92ncTBlwEo2i/zo1f+Jfee+N03/bmVUoR2JgntTJL+4F6KZ6bJvzRO+eJcrTnIAnYFDHYFDJyoxXQ0wMW5CpNTpabXK+WqXH1tmquvTde2pXuiDb3cnYNxTFNC1O0kHU7z9oG38/aBtwNef/9ofpRTU6d4bcoL3memz1C0i03PvRwMcDkY4HP+ApKW1hyu+P3c5ascm73A7olPYbwEc8S5Fr6LfNe9RHY/yI5jb6ezZ3Azv1QhhBBiXW0FbKXUvwd+B68OOYL0XW+6ZDLJ008/zcc//nGq1Spzc3N86lOf4pd/+ZexrADv/bsf42/+/KdIHbgGwNT8p7n0+iH2Hvi1DTsHFTCJ3ttN9N5u7PkyhVfGKbw8gT21HJrMgk13waZbQeD+LKW+OGOuZuLqIuNXFigX7KbXnRsvMDde4Px3x7zXCBhea8me5Up3oiMsrSW3EaUU/fF++uP9vHf3ewFwXIdL85dqgfu1qde4MHsBRzsNz7WV4rVQiNdCIf7C37bUz32sXObu8mscG3mFnusfg2/BKF2Mxo9Q6b2PxN6H2X33o8QS6c39goUQQog67Vawfxl4Wmv96Vt5MmJtfX19fPjDH+Yv/sKLHdeuXePzn/88P/3TP00kkeTxD3yCb3/1wyR2zAFw6er/Rjy1l+7uJzf8XKxUiORTO0k8uYPK1QUKL09Q+NEkuuyHJQ3VS/OYl+YZDFscvDdL9Kf3UgiaTFzx2kbGrywwdT2H6zb+vuZUXcYuzTN2aXn1wUgy2NhasitJMCIdTrcT0zBrK05+6MCHACjZJc7NnGsI3dcWrzU9d7V+7uWLKL/L0UvfJHlR43xZcdncyWTyKHppPvddD0g/txBCiE3TbkIxgB/ewvMQbTp8+DDvfve7+cpXvNUcX331VbLZLO94xzvIDu7mnuP/hdPnP0o0W0IZmh/98B/wtkc+Rzxxa5a4VkoR2p0itDtF6if2Ujw9TeFlv4XEp0s2+e+Nkf/eGFZ3lP4TPez/wB7MZBC74jB5Pcf45XnG/eC9ON3cWlJcqHDlR1Nc+dGU/4kh0xujt67K3dEXw5DWkttK2ApzvPs4x7uP17bNleY4PX26ob1ktX7uCcvi63X93LsrVe4uVzhWnuZY8TkOnXmW0GkofSbAG4H9zHXcjbXDm889sPeI9HMLIYS4JdoN2H8A/CLwr2/dqYh2Pfroo0xNTfGDH/wAgK9+9at0dnZy5MgR9t77CLNj/5KJ/P+LYMxGWRW+/92nefsTf3vLz8sImsTu6yZ2Xzf2bInCKxPkXx7HmVkOzPZEgfm/ucz8ly4TPpgh+kAPvXd10rdveURbYaHih+15xi8vMHFlgUqpsY0ADbOjeWZH85x9YRQAK2jQvStZV+lOEc9I1fJ2kw6neWzgMR4beAzw+rnH8mMNgfv09OmW/dxXggGuBAN8PuFN0bG05lCtn/sax2ZfZ8/EMxgvwzwxroYPk8/eS2T3Qwwee4xs785N/VqFEELcmdoN2GngF5RS7wZ+BFTrd8pCM5tLKcUHPvABZmdnuXLlCgCf+cxnSKVSDAwMcOK9f5ev/ulFnOCfYAY02pzle8//Apr/edPO0cqESb5zJ4mndlC5skD+5XGKpybRFdc7QEPp/Cyl87MYUYvIvV3ETvQQGIgTTQbZc0+WPfdkvUNdzexYgfEr87XWkunhPHpFa4ldcRl5fY6R1+dq22LpUMMFlN27kgRCsuz77UQpRV+8j754H+/Z/R6gsZ97KXS/Pvs6tm7s8beV4nQoxOlQiE/522IN/dxnODb6Q3qGPo76NozRxUjsLio9x0nsexu77n6UeDKzyV+xEEKI2127AfsIyy0ih1fskwset4BlWfzcz/0cH/vYx5iZmcG2bT75yU/y67/+66RSKZ56+v/B5//gCrGD30QpqOjXcSr/lrHxEtnOH7upOdk3QxmK0N4Uob0p3J/cR/HUFPmXx6lcXu6vdgs2+e+Mkv/OKIHeKNETPUTv68b0V4ZUhqKjP0ZHf4y7Hu0HoFp2mLy26I8B9NpLcrPlps+fnytz6QeTXPrBpPdaCjr64w0XUGZ6Yxiy7PttZb1+7temveB9deFq03PzhsH3I2G+X9fP3dXQz/19jl75FqlLv4f7t4or5g4mksfQ/ffRcfARdt31IMFQuOl1hRBCiCVtBWxZaGZ7ikaj/MIv/AIf+9jHKJVK5HI5PvnJT/Krv/qrhEIh3vf3/guf//jPkjlyAQAzeJXTp38HtEUm9XYGdnyIzs6nNi1sGyGT2AM9xB7owZ4ukn9lgsLL4zhzy8G4OlZg/ouXmf+bK4QPZYg90EP4UAfKauyVDYRM+g+k6T+Qrm3Lz5X9Crdf6b66iF1ubC3RGqaHc0wP5zjz7RHvtcKm11riV7rLi5pSvkowYknwvo206ueeL8/X5nIvVbqnS9NNz520LE5aFidX9HMfq1Q4Vp7hWPE5Dp99ltAZKP91gPOBfcxm7sbacYKeux5lYO8xDFP+MiKEEMIjYxhuc9lslo985CP8yZ/8Ca7rMjY2xmc+8xk+8pGPEIrGeOeH/4ivfObv0HnX2PKTlM3swklmT58EHaAj8zj9Az9FNvsUphld9XNtJKszQurdu0i+cyflS/MUXh6n+NoUuuq3kLia0tkZSmdnMGIW0ePdRE/0EOyPr/qasXSIvfd1sfe+Lu8lXM3saJ7xywuM+f3cM6P5pr+5VEsOw+dnGT4/W9t28Yve8vTBiEUoYhGKWYSiFqFoYPk24m+LLW8PRwPec2KWzPLeBlKhFI8OPMqjA48CXj/3eGGcU1OnaqH79NRpCnah6blL/dxfiC/3cx+sVPyLKK9z99xFdk/+JeYrsECMq6FD5Jb6uY8+RrZ/16Z+rUIIIbaPVQO2UupzwC9qrRf8+6uShWa21p49e/jABz7A5z//eQDOnz/Pc889x3ve8x4yff2886c+ybc/83sslp4nvWeeSGddK4WqMjP3VWbmvgoE6Ox4kr7+nyTb+eSmhG1lKML704T3p3F/ym8heWmcytWF2jFu3ib3/Ai550cI9MWIPtBD9Hg3ZmztFSANQ9E5EKdzIM6Rt3utJZWS7c3k9gP3+OUFCguVVV+jUrSpFG0Wm4dYrMsKmYSjK4J5xL+/MrCv2G4FpBp6Kyil6I310hvr5d273g14/dyX5y/XLp48NXWKCzMXWvZznwmFOFPXzx2t9XNXuLt8lrvHXqVn+BOo52GcToZjRyj33MdcNc7VngTdOw8SiSU2+asWQgix2daqYE+zXOtr/puq2FZOnDjB9PQ0L7zwAgAvvPAC2WyW+++/n87BnfzUP/4PPPflL9EbNLnwyucp2N8nvXehMWxTZXrmK0zPfAVFkM7Op+jt+wk/bEdu+ddghC1iD/YSe7CX6lTRW579lXGc+eUAXB3NM//5S8w/e5nI4Q6iJ3oIH8qg2qwWB8MWg4cyDB7yLlzTWpObLTf0ck+NzqNci0qxeVGcG2GXHXJlp2Vv+HrMgNFeII9ahOsq6MGIRSBkyqI8N8A0TPZn9rM/s7/Wz112yk3zuVv1cxcMgxcjYV6s6+futB2vl7tS4e7yixy98m0ecTV86n8DYIo0U1YfuegA1cQOzI7dRHv20jFwkO7BvViB4OZ84UIIIW6ZVQO21vpXW90X29e73vUupqenOX/+PABf+MIXyGQy7NmzBwArFObYk09y7Kl3k5+b5cL3nuf173+BivoB6X2LRDqWg6CmwtT0l5ma/jKKENnsO+nt/QCdnU9sStgOZCOk3rub5Lt3Ub44500hOT0Ntt9C4miKp6cpnp7GiAeI3tftTSHpvbF+cqUUiY4wiY4w+090A3Dy5EmefPJxXFdTKdqUCzblQtW/rb/fvK3k368UbPSbuPzXqboU5isU5levrq/GMNWKdpYVLS0rA3ut1SVAMCzhHCBkhri3617u7bq3tm2+PM/p6dO10H1qsnU/97RlctKKNvRz99g2A7bNYNVm0LYZtK8yWH6DXXmb7LCLOuUdZ2uDESPLbKCPfGwQJ7WTQOce4r37yO44SGf3oMzuFkKI24D0YN9BDMPgZ37mZ/jEJz7B2NgYruvyqU99io9+9KNks9mGY2PpDPe994Pc994PsjA1wfkXvsXF57+IEzrtVbY7loOdpszk1LNMTj2LIkxX9zvp6V4K27d2moIyFOGDGcIHM7hFm8KPJim8PE7l2mLtGDdXJfetYXLfGiYwGCd2oofovV0Y0bVbSNZjGIpwLEA4FgBu7JcK7WqqZYdSq2CetykXWwV2/37eblrd8ka4jqa4WKW4WF3/4BWUguAqgTzs3w/623NjmomrC7XHwcid3XeeCqV4tP9RHu1v7Oeur3K/NvVay37uccti3LJ4pcX/LmHXpd92vOBdtRmwiwzaFxjMnWFwziZ6ZfnfQlEHmTB7mAv1UYoNojO7CWb3kurbR9fOQyTTnbfs6xdCCNG+tgO2Uuop4GlgJ9DwN0yt9Y9t8HmJmxQKhXj66af5wz/8Q3K5HKVSiT//8z/nox/96KrPSWa7efAnP8yDP/lhZkaGOP/Ct3jj619EJV4nvW+BcKY+bJeYmPgiExNfxFARurreSU/PB+joeALTvLWLuhgRi/jDfcQf7qM6UaDw8jj5VyZwF+taSIZyzA3lmPvCJSJHOr0WkgMZlLm5VVllKIIRL3Ryg5lHa41dcRsq5aW8F8ArRbsutNcH9uXHztKFojdBa7zXy7fXHnP15EsNj62QWRfMvVaWYNQiFFluYVlucWkM7Lfb1Jb6fu537XoX4PVzX1m40hC4z8+ex3ZXfz9LhsGloMGlYOtfCDscxw/efvW7OsOgPcGOuZfpmXIwX18+dp4Yk2YvC+EBKokdqI7dRLr2kh7YT/eOA4QjmzMxSAgh3uraCthKqV8Bfh/4a+BJ4L8DB4E9wJ/eonMTNymVSvH000/ziU98Atu2mZmZ4ZlnnmHnzvVXqevoH+SRn32at33455m8epmzL3yDy899iUDmCum9jWHb1UXGJ77A+MQXvLDd/W56un+cjo7Hb3nYDnRHSb1/D8n37KZ0cdabQnJ6Ghy/2udoiqemKJ6awkgEid7vt5B0b86UlDdDKUUgZBIImcRvYo0Tu+osB+9C1Q/fKwJ5oUrJD+z17S0rxxre8OcuO9hlh/zcjfedgzcyMVQXuOt7zoP1wXwpsNdtC4Yt1BYHdNMw2Zfex770Pn56/08DUHWqfPZrn2Xg6ABDi0MM5YYYWhxiODfM0OIQC5WFNV9zxjSZMU1+RPP/U5bW9PnBe2Cp/aQ6wmD1Gvsmvk1yzKX+HZmgg+lAL/nIAHZyJ0bnHmI9++gcPEBX/x5MS/6oKYQQG6Hd76b/BPgtrfXHlFKLwL/QWl9SSv0XIHfrTk/crIGBAT70oQ/xl3/5lwBcuXKF+fl5ent72b9/P8Hg2hdSKaXo3r2X7t17eVz/CqOvn+PsC9/gyne/TLh7mPS+RcLpFWF7/HOMj38Ow4jS3fVuurt/nI6Od9zSsK1MReRQB5FDHbiFKoVXJ8m/PE51aPmfpbtYIfeNIXLfGCK4I+FNIbmnCyNyZ4YJK2BipUxiqRt/3x3bXQ7jxca2lYaQXrQZH54kEoov96kX7Te97FS15FAt3dyFoSjvItamMO4H8oYwXt/+4m8PhMxbEtADZoCuQFettWSlhcoCw4vDteBdC98573at6retFNcDAa4HAi27mBKO64XvWv93hUH7EgOFC/Qv2ASHlo+taJNRo4vZYB9Fv/87mN1Lom8/2cEDZLJ90v8thBBtajdh7AWe8++XgaVhxP8FOAn88409LbERjh49yvT0NF/72tcAmJ2d5ZlnniEQCHDw4EGOHDnCgQMH2grb/Qfvov/gXbjur3P99CnOv/ANLn/7q0T6x0jvXRG23QJj4/+dsfH/7oft99Dd8+N0drwdw7h1YduIBog/0k/8kX6qY3nyL49T+MEEbm65F7lyfZHK9UXmPn+JyNFOYid6CO1Pb3nlc7swLYNoMkg0uf4kC+9i0Idqj5f6zldWzOsDeKWw3H/utbssbbPf9NQW9PJYRW5irKJSNLSr1F8E2lg9twg29Ki/uektyWCSZGeSuzrvatrnuA6TxUmuL16vVb+XKt9Di0MtL7Kst2ganDWDnA01//dUWtPjOMuVb9tmsLrAoD3D3vlX6ZxxUZeXjy/oEONmL/PhfkqxQcjsIpTdQ3rgAN07DxFLpG/4axdCiDtVuwF7Glga3joMHAN+hNddeutHSoib9o53vIPFxUVefPHF2rZqtcrp06c5ffo0gUCAAwcOcOTIEQ4ePLhu2DYMk113H2fX3cdx7P+BK6/+gHPPn+TiN04S3zFFZt8CodRyoPXC9mcZG/8sphHz20g+QEfHY7c0bAd6Y6Q/sJfU+3ZTOu+3kJybWW4hsV2Kr05SfHUSMxUken8P0fu7CXRt/xaS7aq+7zzRcePPd11NtbQyjDdWzZe31Qd3b3+19ObaW7SmdrHpzVCGalk9D0Ysxidcvj3xOoapUKbCMBSG6X0oY+mx4W0z/G3+fsMwyJq76TL2cCKkMCIKo9c7tuyWmS5PMVEaZ6I0zlhxjNHCCKPFEUYKIxScPK5y0MrF9T9Q3v8DWinGLIsxy+LlFl9P2HX9vm+nNv1kwJ5gsDrCwanvEp3UcGH5+FmSTFq9LIb7qSR2YnTsItq9l/TAAXp2HJDl5YUQbyntBuxvAe8BTgHPAL+nlHo38E7gK7fo3MQGUErxgQ98gAceeIBnn32WfD7P1NRUbX+1WuXMmTOcOXMGy7IawnYotHYANq0A+048xL4TD1Et/2MuvfIi557/BteufZvkrhnSexvDtuPmGRv7LGNjn8U0E3VtJI9hGLdm9q8yDSJHOokc6cTJVyn80FuevTqSXz6v+QqLX7/O4tevE9yVJBlTVEbzBHqiUtneRIah/JaNm5v+4joulZJTF76rzWHcD+Stgnr1Tfafa1dTylcp5VtPb5l5/fqbev31dROjm/3cw/41jtIshW3H/3DrAvhSGHeWH+OSVy5nlcNp5aL9fRY2YRzC2ibqf8R1lbi2iY1UMRliUV0lx3MM41BUIUpmjGowjo6kMWMZQqksyWwviUwnpmUyf01z+UdTWJaBGTSwAgamZWAFDUzL9B4HDSzLkP83hRDbWrsB+7eApfLDvwVs4DG8sP2/3oLzEhusp6eHPXv28MQTTzAxMcGZM2c4ffp0Q9i2bZuzZ89y9uxZLMti//79HD16tK2wHQiFOfTIOzj0yDsoF/JcfPG7nHv+JFdHvktqz3xz2HYWGR37DKNjn8Eyk3R1vZvu7vff0rBtxgIkHhsg8dgAlZGct5DNDydw6yZmVK4u0I3BxJlXUCGT4I4EwZ0JgjuThHYm3vToP3HrGKZBOGb4YxVvnOO4VIveaMX6QF5fKa8F8+Ly9JaKH+Ttys1Pb9lMCgNTG5h6Y69BKPofkzf8zCn/wzP0wo/aepZhKi9wBwysgIlZu2+sut0MeOG8FtiD3rblEO8/P2hiWiue57+eYSmZFS+EWNe632GVUhbw88BnAbTWLvC7t/a0xK2ilKKnp4eenh6eeuqphrA9Obn8o9G2bc6dO8e5c+cwTbMhbIfDa/+pNxSNcfSJd3L0iXdSWJjnwnef5/wL3+DK5Euk9y6S3rdAKLkctm1ngdGxv2J07K+wrCRdXe/xwnbm0VsWtoP9cYL9cVLv30Pp/Az5l8YpnZ+Buoykyw7li3OUL87VtlldEYI7kwR3JgjtSmJ1S5X7TmGaBmbcIBy/yYBuu0395kth/dzZC+zbuw/X1biORvu3rqNxXY32b13HXfG47tbRaNdd8bhxf+25/ussP14+7k7hOpqK40DJAW585vtNU3gV9obgbjYEe7NFyG+uypu1QN/y+KbtBsYdPGdeiDvNugFba20rpf498MVNOB+xybq7u+nu7ubJJ59kcnKyFrYnJiZqxziOw/nz5zl//jymabJv3z6OHj3KoUOH1g3b0WSK4+/5cY6/58dZnJ7i/He+xbnnv8HCwinS+xa9ynZ92LYXGB39NKOjn8ayUnR1vYee7h8nk3kEw9j46rGyDCJHs0SOZnEWKxR+NMnw998gWQg1zNeund9kEXvSW8YdaKxy70oS2iFV7rcq0zKIJIJEEs2/FE46r3P8yfXHZG4GXR/cl0J9q9Df9NjFcbzAbtsOs8U5pvJTTBdmmC7MMFucZbY4z3xxnmKliKFNDG2g/FtDmyhtePcxMVwDxfI+QxsYrknADRB1TKKORdi1CLoWphvAcAO4Ooijg9gEgC0Kmxrsqov9JubN3yxlLFftHddl+OvfxQwYBIJ+yA8uV+CXKu5msG5/wPCPMWu35srnBJePk0AvxM1r92+E3wVOAFdv4bmILdbV1cUTTzzBE088wdTUVC1sj4+P145xHIcLFy5w4cKFWtg+cuQIhw4dIhJZ+3rXRGeWBz74IR744IeYHRvh/PPf5Ny3vkGhfIHMvgXSexcJJurD9jyjo3/J6OhfYllpbxpJ94+TybztloRtMxEk8dgAY9XXOfTEQzhzZSrXFqhcXaR8bcHr216xwuKaVe5dCUI7pcotthdlKExDYb7pV+rGWw6hWaFaYDg3vDzxZMXs75JTuqnPmHIcdldtdleq7KpqdlUUgxVFT8XAIIitAyzoJOPGDnKRQZz4TqzUDuIdO4jFszgOOFUHu+ri+CG56dZ2sCsuju16t1UX23ZxKg627eLaW/dXgKVJPUvXC8wVmlcN3UiGodYM4A2hful+cLkCbwWXg3wt3K+xX75PijtJuwH7D4H/oJTaCbwM5Ot3aq1f2egTE1srm83y+OOP8/jjj9fC9pkzZxgbG6sdUx+2DcOohe3Dhw+vG7Yzvf287cM/z8M/8xGmrl/l/Avf5NxzJ6nqq6T3LZDeu0AwsdwbbdtzjIw+w8joM8thu+cDZNJvwzA2fp61UgorE8bKhIne2w2ArjpUhnO1wF25toDbYjnyllXuuj7uoFS5xR0uGohyIHOAA5kDTfu01kyXphuC99L9i1MXmXfmV33dedPkVdPk1XDjNSGG1gzYNrureXZX59hdvcSeapXdC1Wysy7qijdmcDiwk7nYXnTnIWJ9d5Hdcw99uw/f0AI72tVe4K6F8tUD+1KYb97v1D1/Kdi72BV/+1K4t+v2Vxz0Jmd719W4/mz6zWjDMSxFYGUAr7s1byK0F6Y0Y5fnvek8ypvQowxaP/a3Gf42tTTRR9U9lv570Sal1/g/Vin1ceB3gLk1XkNrrd98MeQmHTp0SJ8/f36rPv1txZtb/OSbeo3p6ela2B4dHW15jGEY7N27txa2o9H2Rt9prRl74wLnnv8m57/zTXRghPTe5rBdLxDooKvLG/2XTj+8IWG73fdJa40z61e5r61e5W7F6o4Q3HF7V7k34t/TW4W8V+05efIkDz72IFcWrnBl/krD7dWFqxTt4g2/Zsx12V2tepVv/3ZPtcrOqk1Ea8o6wJA1yGx0L9WOA4T67qJz9z307z1KIHhrV6S9UY7jh/KKy7e/9TwPnngIu+KH8IpTu3UaHvu/BFSWg7q96n6n9vr2FgT624LCD9zLoXs5rK/xuG7bqo9rAZ/VHxsKQzXur32ets+r+XOcv3COu+89SiBoYgW9lYSXfnkJBE2skHftgPyCAUqpl7XWD6x73DoB2wH6WGfWtdZ6y1pHJGC3b6N/yM/MzNTC9sjISMtjDMNgz549HDlyhLvuuqvtsO26DsNnT3Pu+W9y4XvfxohOLle242uFba9n+82E7TfzPrVb5V6pqcq9M7ntV5qU0Ng+ea/as9b75GqXicIEl+cvNwXw0fwo+iaWEe2z/dBdWQ7fu+0qvbaDo02GzX5monsoZw4S6D1Mx6576N93jHAk9ia/0jfvVv+b0trrv6+F8RYB3K7WhfM19jtNvwSsCPWVrelpFzdGKWp/IfACuB/GgwZWyMQKmARCdaHc374U2uufF2h6He95xm1QaGo3YK/3E1zB1gZosX11dHTw9re/nbe//e3Mzs7Werbrw7brurzxxhu88cYbfOELX2DPnj0cPXqUw4cPE4ut/kPKMEx2HL2HHUfv4cd+7Te5euoHnH/+m7z+V98hkJrxppGsCNvV6gwjI3/ByMhfEAh00N39Prq73k8m8zBKbc4fWVTAJLQ7RWh3igTtV7l12aH8+hzl1+dY9LdZ3RE/cHuVbqvr9qtyC7FRDGXQG+ulN9bLI/2PNOwr2SWuLlxtWfnOVXOrvuaoZTFqWXxnRQkp7LrsrNrsrhbYXX2V3TMvsWfcpvulKgEXrht9TEV2U0ofwOo5THrX3Qzsv4doPHUrvvQtoZTCtBSmZRDahF/2tdbLLTOrBHC7qfrut+esUsWvVhzmZudJxBPehbraa/HR/v2li3mXPtz6/S0ev9Ur+lpTuwag2Ebh6GaYftvPWqHcCpkEAl5VvVVYXyvcG+bmtfm083/NW/yflGhHJpPhscce47HHHmNubq4WtoeHh2vHaK25dOkSly5d4gtf+AK7d++uhe14PL7qa5uWxd77HmTvfQ/yrkqZyz94iXPPf4Pzz3yfcMcC6X0LpPYuEow1hu3h4T9nePjPCQQ6vbDd/X4y6Yc2LWyD38vdEcbqCBM97vVyuxWH6nCOyrUFylcXvSp3rkUv90QRe6JI4SW/lzu8NLEkSWhX0uvl3uZVbiE2Q9gKc6jjEIc6DjVsX+r3vjx/2QvgS+F74QpDi0M4uvXiQiXD4EIoyIUWS8xnbcevdp9ld+4Ue2ar9JyyCdo2I6qbyfBuiqkDmN2HSOw8Rv/+4yTTnbfk676TKKVqFVE28A8EXqX/wQ15La0bQ/rK0O4Fdv84V9cFeP8vAnVBfTm0a1x3ndfU/mu4za+x6udo+UsEuLpx/9L90dExOtJZ7xeUskN16a8WFS9QL10TcKstXZtQzt/cirrrUYbyKu5B0w/gxnIl3m+LWW6TMVpW6dvVzk/nsfXS/lb2YIvtJ51O8+ijj/Loo48yNzfH2bNnOX36NENDQ7VjtNZcvnyZy5cv88UvfpFdu3Zx9OhR7rrrrjXDdiAY4uDDj3Hw4ccoFwq88dJ3OffCNzn3yVeIdOX8nu1FAg1he5rh4T9jePjPCAazdHW9j57u95NOP7ipYXuJETQJ7UkR2tNc5S5f9Srd1dFcw0xuAF1aUeVWYHVFazO5gzulyi1EPaUU2UiWbCTLg72NIavqVLmeu95U8b4yf4XZ8uyqrzllmUxZJi9FGkeUWlr7Ve9L7C6fZ/elz7LnfBX3WZsJN814aDf51H5U1yESO7zgnc723pKvW9waSimUAu7A77EnT07w5JP3rHmM63h/KVgK3HbFD+L+Yy+UO1TLdfsqrr9/+XG19nip/Wj5/q0u6WpXUyk5VEpvbuXedrQTsH+DtS9yFGJV6XSaRx55hEceeYT5+flaz/b168vLRmutuXLlCleuXOHZZ59l165dtZ7tRCKx6muHolGOPP5jHHn8xygszHPx+9/h3PPf4PSfnSLWU/B6tvcsEIgt/49UqUwxPPynDA//KcFglu6u99Pd/eOk0ye2JGzDGlXuoZzfx71KlVuDPVHAnijUVbktL3D7fdzBnQmMsFS5hVgpYAbYm9rL3tTepn3z5fmWvd7XFq9RdVv/adxWikvBAJeCzROC0o7D7uowu6tX2D30N+y+bBP+apVSNcZ0cBe55H7IHiQ2eJTe/cfp7B5EGTKDWmwvhmkQNA2Ct+hnylKb0MogXgvgK4J5dUWw9yrvdaG9RdB32xhCsFHaeZc+r7WeWP8wIdaWSqVqYXthYaEWtq9du1Y7ZrWwfeTIkTXDdjSZ4p53vY973vU+cjPTnP/Otzn/wjc5/cI5Yr1Fr7LdImwPDf8JQ8N/QjDY5beRfABvsdKtZQRNQntThPZ6PZ1aa5yZUq2Pe/Uqt035wizlC34FToHVHfX6uP3FcKxsRKrcQqwhFUpxvPs4x7uPN2x3XIeR3AiXFy43tJtcmb/CZHH1ReLnTJMfmiY/bDlecJLd1VF2T3yN3cM2xW9Xma8GKBs7yCX243YeJDpwlO5999IzsFeCt7hjNbQJ3SKO4y5X3MsOdrUuiPuP7XJdaG+ouHv72v562pkisp0DtkwRad92nWSwsLDA2bNnOXPmDFevrn497c6dO2thO5lMtvXac+Nj3oztF77J1PXLxHoL3gqSexYIRFf7E1GcbPZtpFMnSKVPkEwcwzC217guWKpyL1K+tkjFby1x821MLKmvci/1ct9ERWK7/nvajuS9as/t/D7lKjmuLlxtDN/zV7i6cIWSU77h14u7LrtWjBfsqRhY9FGK7mPaiZIe2I+V7Cac6iHe2U+ys49UR88NzfW+093O/6Y2k7xP7dvQKSJC3ErJZJKHH36Yhx9+mMXFxVrP9sqwfe3aNa5du8aXvvQlduzYUQvbqdTqV+6ne3p5+EM/x8Mf+rnlBW2e/ybDz48Q7/XaSFJ7FleE7RxTU88xNfUcAEoFSSaPkUqd8EJ36n6Cwa2/aMmrcqcJ7U0DUuUWYivFg3GOZo9yNHu0YburXcbz403B+8qCN15wNTnD4HQoxOnQyl/uy/TZP6LXtsnOfpPOKYes49Ble7cZ2yXoRjF1klKgg3Kwg2q4Ex3LYsa7CSS7iXT0kejoJ93VRzSWlKq4ELfAmgFbay3/14lNlUgkeOihh3jooYdYXFzk3LlztbBd/9eW69evc/36db785S8zODhYu0AynU6v+trZHbvIfuSXePTnfpHxSxe9SSTf+RZDz08R7yuQ3utNIwlEGivbWleYn3+F+flXuMYfAhCJ7K5VuFOp+4lF96HU1v7vopTC6oxgdUaI3lffy71OlVuDPV7AHi+Qf9FbqVNFLII73nyVW4i3OkMZ9MX76Iv38Wj/ow37inaRawvXmsL35blLFJzVF9VZGi+4FqU1GXeSTmfMC985h855l+w1P5A7DmXHIW6b2DpJzspQDGQohzpxIllUPIuV6CaU6iHa0U+qs49UtnfbLb4jxHYlPzHFtpVIJHjwwQd58MEHyeVytTaSK1euNITtoaEhhoaG+PKXv8zAwABHjx7lyJEjq4ZtpRS9+w7Qu+8AT/zirzF87gznXvgmF777bYa+PU8oXSHWWyDeWyTaUyScrjS9RrF4hWLxCqNjfwWAZaVIpe4nnbqfVOoEyeQ9mObay8VvhtWq3MuBe4HqWL65yl1cpcrtTysJ7kzKAE8h3qSIFVl3vGCt4j1/mTdm32Ck0N6iOlopZkyTGdPk9eZpgw1CrkvWydHpzJN13iBbcchOOmTHHLKOS9Z2iDkOOA5zxJk30uStDKVghmo4ixvNYsS7CCS7Cad7SXT2kcwOkEx1SHVcvGVJwBa3hXg8Xgvb+Xy+FrYvX77cELaHh4cZHh7mb//2b+nv76+F7Uwm0/J1lWEweOQYg0eO8dSv/AbXz5zi+Wc/j5FfZOhbr+M6NlbYJtZbJNpT8G67Shhm4w84255nevrrTE9/3XtdZZFIHCXlB+506gShUPete4PaVF/ljtVVuSvXF2vTSirXFnBXziCtr3J/36ty7wkYTF46RXAgTmAgTnAggZkJyVK6QrxJ640XHMoN8dx3nmPw0CBTxSkmi5NMF6e9+4VJJvMTzFcX2l7dsmwYDBsGw4H1I0HKccg6NllnnKwzQtZxyM45ZKe9ynjScUk4DknXpapN5lSKBTNDIZChHOrACXdCrAtjqTqe6SWR7Sed7dsWK2QKsVEkYIvbTiwW44EHHuCBBx4gn89z7tw5zpw5w6VLlxrC9sjICCMjI3zlK1+hv7+/1rPd0dHR8nVNy2L3PfdxZWaeJ598kmqlzNjr5xk6d5qhs6cZ/eE5RssllOkSyZaI9RaJ9xSI9hZbtJXYLCy8ysLCq1y//gkAwuFBv4fbayuJxw9u2WjAekbQJLwvTXhfGvCr3NOl5RGBV/0q94qf1WZV1eZy114ravlhW0K3ELdCwAywJ7WHA+EDPLnnyVWPq7pVZkuzTBWn1vyYLExSckptf/5502TeNHljneMsrb3w7Thk7XmyzgxZ53WyBYfsokN22KntD/nfWxZ1hHkjXWtXqYQ7caNZVMyrjofSPSQ6+klm+0l1dGOYW//9U4jVSMAWt7VYLMaJEyc4ceIEhUKhIWy77nLfw1LYfu655+jr6+PIkSMcPXp01bAN3qI2S8u1Azi2zcSVNxg+e5qhc6cZPneGyVcXAU0wWSXeW6hVuiMdzW0lpdIQY6Uhxsb/OwCmGSeVus+vcN9PMnkvlrX6IjubRSmFlY1gZSPE7u8BwC07VIb8CvfVRSrXW1S5AbdgS+gWYhsIGAG6o910R9f/y1mhWmCyONkQvKeL07VtS9Xx6dI0bptjTG2lGLMsxiwL1mnbTjjuchh3KnQ6I3Q518lWXLJFh45xr2c87bosNZw4WjGlUiwaafKBNKVgJ3akE6JdGIkugskewpke8rMTlAo5wtGt/94q3lokYIs7RjQa5f777+f++++nUChw/vx5zpw5wxtvvNEQtkdHRxkdHeWrX/0qvb29tbDd2bn2ZBDTsujbf4i+/Yd44Cd+Bu26TA9dY+jcGYbOvsbwudPMXJj2jg05xPyWklhPkWhXESPQWAJ2nBwzM99iZuZb/haDRPwuv63kftLpBwiH+zf0PbpZRqi5yv3833yD+wePesu+D+eoDOXQJQndQtxuooEouwK72JXcteZxjuswW55dbkdZEcDrW1Vy1Vzbn3/RNFg0DS7TvEhPPVNrOh2HTselyw/knc4sWXuKLuc82XmH7IxLp+MQ9f+aeRjg1d9kgShzRoZFq4NSKEs10oWOdWOl+gil+4h1DpDqGiCT7ZMxh2JDyL8icUeKRqPcd9993HfffRSLRc6fP8/p06ebwvbY2BhjY2N87Wtfo6enhyNHjpDL5XBdF2Odi3OUYZDduZvszt0cf8+Po7VmfmKc4XOna4F79Psj/rGaSGfJD9xe8K5fzt3jspg7zWLuNEPDfwJAKNTrXzzptZbE43dhGFv/v61SCjsK0Xu64J4uoG5MoB+4q28qdCcIDsQldAuxjZiGWesNP8ShNY8t2sVa2F7rY7o4ja2bv0e04ijFhGUxYcHZdY6Nua4fwL1AnrUdss4inc4cWft1snMundPe/vpYv1QZnzM7yQc6KIe7cKJdqEQPgVQvkY4BEtkBMt2DxOIpuYhTrGrrf1ILcYtFIhGOHz/O8ePHKRaLXLhwoRa2HWe5d3p8fJzxcW/J8VOnTtHf38/g4CADAwMMDg4Sj6/9J0alFOmeXtI9vRx94p0A5OdmGTq7HLgnX7vC5KkOQBNMVIn1FIn5rSXhjjIrs2S5PMbExLNMTDwLgGFESCXvJZX2LpxMJu8jEGhv0Z1brWFMoIRuId7SIlaEwcQgg4nBNY9ztct8eX7NAL5UKV+oLLT9+fOGQd4wuBpYuyoOkHT8aSm1QO7Q6UySdcbIlh06Cw7ZcYeM4zaEpoIOMWukWbQ6KQSzVCNZ3Fg3ZrKXYLqPWEc/qe4dZLr6ZbzhW5AEbPGWEolEuPfee7n33nsplUq1NpKLFy82hO1yuczly5e5fPlybVsqlWoI3H19fQTW+eYdS2c49MjbOfTI2wEo5XOMnD/L0NnXGDp3mvE3Xmf2ordQjhFw/MDtVbmjPUXMFW0lrltkdu67zM5919+iiMUO1BbASaVOEIns3DYB9NaG7kQtfEvoFuL2ZCiDTDhDJpzhQObAmsdWnEpD4F6tV3yyOEnVXX9V2yULpsmCaXJpnRYVb7a4Wwvh3gjDMp3OdTqdK3TmXK9N5XpjvzjALEnmjAy5QAflUBY72g2JHqxkL+FMH4nOAdLdgyQzXVIVv0NIwBZvWeFwuCFsX7hwgbNnz3Lx4kWq1eZvzvPz88zPz3P69GkADMOgp6enFrgHBgbo7Oxcs7UkHIuz9/4H2Xu/N3qrWi4x+vqFWoV75PVzLA75yyorTaSj7FW4/Up3MLEyhGry+Qvk8xcYHvkkAMFgtjapJJ06QSJxFMNYZxDuJpLQLYS4GUEzWFu0Zy1aaxYqC0wVp/jqd7/KjkM7GgL5VMm7nS5OM1OawdHOmq9Xe9362eLrHGtqTYffntJZ6xef86epnKfTH23Y6bgkXbe2bHZFW8yoDAtWhnwwSyXsVcWNRA/BdD+Rjj6S2UE6ugfkws1tTgK2EHhh+5577uGee+7h61//OsePH2d4eJihoSGGh4cZHR3FthvDnuu6tQsmX3rppdrrrGwticVWn+0aCIXZeewedh6rm1Ry+Y1ahXv43GmmToeZOu0fH6s29HFHOkusXECyUplicvLLTE5+GQDDCJJI3LO88mTyPoLB1aenbAUJ3UKIjaKUIhVKkQqluB6+vuY4Q8d1mCvPMV2argXw+qkpS8F8pjTDbGm27dnijlJMWhaTbaSsQO3izeVWlQ5nnKw9QmfJIZt3yY56+2Na18L48oWbncsXbsZ7sJI9hNJ9xLODtQs3ZaTh5pOALcQKSikymQyZTIZjx44B4DgO4+PjtcA9NDTE9PR003NLpRKXLl3i0qVLtW3pdJrBwcFa6O7t7V21tcS0LPoOHKLvwCEe/MkPo12XqaFrXoXbHw8498YMc294fdeG5RLtLi73cvcUMUONY7Rct8L8/EvMz78E17xt0eje2gI4qdT9RKN7t13wlNAthLjVTMOkM9JJZ6STg5mDax5ru3bDbPGmUF5arpLfSL94tX6k4TrCrruiKl4g6+TIOpfozDl0zjt0XvOCesSfpGJrg2mVYt7sWL5wM9aNivcQSPUR6egjNz1FITdPNJ5q+7zF2iRgC9EG0zTp7++nv395bF6xWKytHLkUvAuFQtNz5+bmmJub47XXXgO81pLe3t6GKndHR0fLkKcMg66du+nauZv73vtBb1LJ+FhDhXtuZJTcyFKVXBPOlL0qt1/pDqWa210KhUsUCpcYHf1LAAKBjNfDnbyfVPoEycTdmGZ4A965jSWhWwixVSzDoivaRVe0a91jK06FmdJMY2tKXTCfLk7XAnq+mm/7HEo3sOpmzK0P4i6dzgxZZ5JO54w3znDSqV3YeQTg1D9kUUeYNTtYtLIUw13Y0R5I9BJI9xPpGCDZtYOO3h0SxNsgAVuImxSJRNi/fz/79+8HvKA3Ozvb1FpSf/EkeK0lSwvf1L/WwMBAQz93NBpt+pxKKdK9faR7+zj21LsByM1M18L20NnTTF2/Smk2zPRZb3l4K2LX9XEXiWSLGCv+WlitzjI19VWmpr7qf54AicTR5ZUn0ycIBbMb9t5tpLZC99AileH8zYfuwThmWkK3EKI9QTNIb6yX3ljvusfWjzSsD961UF7XM34jq24uTVK51sYklYTj0u3YZB2XbrtE1rlKt3OJ7KJD96xD12UvjC9VxReIMmt0sBjIUgp1eRdtJvtqQTzVvYOOnp1EYom2z/dOIwFbiA2ilKKjo4OOjg7uvvtuAGzbbmotmZmZaXpusVjk4sWLXLx4sbato6OjIXD39vZitfgTYryjk8OPPs7hRx/3Xiu3yMj5MwydPc3w2dOMX77I/GWL+cteW4kyXaJdpYaFcKympd6rLCz8kIWFH8L1PwIgEt5JKn1/rbVEt7mi21a41aE7OOgtkCOhWwjxZrU70lBrTb6abw7gfo/4ytYV221vvjgsLfYT5I11jkv4i/x4H0W67Ct0OW/QtejQNevQdckL4tFaEI/5QbyTYrgbO9qNSvQRSA8Q7ewn2bWTzt4dd+QFmxKwhbiFLMuqVaaXFAqFptaSYrHY9NyZmRlmZmY4deoU4LWprGwtyWQyTQEvEk+w78TD7DvxMADVUomR188x7Fe5Ry6cJz9mkB+LwqsAmlCq4reVeJXucKZ5qfdi6RrFsWuMjX126TPx0st3EYvuJxbzPqLR/YTDfaiVV15uA+uG7qEc1WEJ3UKI7UkpRTwYJx6Mr7vq5tIklVYXbL6ZSSpLq26uN9Iw7nqL+3Q7DlmnQLe9SNZ5g+4Fh+ysQ/aSF9Lrg/jMUkU83IUd7UUlewmm+4l2DJDs3klH7w7CkdWHBmw3ErCF2GTRaJQDBw5w4IA381VrzczMTEOVe2xsrGHFSfAutFwK5vWvtbK1JBKJNDwvEA6z6+7j7Lr7uPc6dpXxS2/URgMOnz9DeT5PeT7EzPk0AGbY9lpK/Isno10lDGvl1fNF5udfYX7+lYatphklGt3rhe668B0O79gWq1DWaz9059Cl5h9Aa4buwURtKfg2Bw8IIcSGqJ+ksje9d81jXe3yxa99kYP3HWSiMMFUcYqJwgSTxUkmC96M8YnixA1VxXOGQS5ocGWdIB5zXbpsL2xnnQLdzmW67It0LXgV8Y43vH0xP4jPE2PW6FwO4rEeVKKPYGY5iHf27iAUbm6x3Gzb66edEG9BSik6Ozvp7Ozk3nvvBaBarTI2NtZQ5Z6dnW16bqFQ4PXXX+f115ensnZ2djYE7p6enobWEtMK0H/wMP0HD8NP/aw3qeT6Vf/CyTMMn32N/NwsC1cTLFz1+ueUoYlkSw293IFo62+0jlNgcfE1FhdfW/F1BolGdzcF72h0N4axfVY5WzN0Dy1dSHljoXuvaTD+o1ewshGsjjBWZwSz079NBlGGVLyFEFvDUAYJM8GhjkMc6ji06nGudpkrzzFZmKyF74bbuvvtBvG8YZBvI4hHXb81xXbocvJ0OQt02a/TNe/QNeOQueTtWxpjOEfcC+LBLOWQF8SNZB+BdB/RzkG/R/zWBnEJ2EJsQ4FAgB07drBjx47atnw+3xC4h4eHKZWaL3iZnp5menqaH/3oR4DXWtLX19fQWpJOp2ttDMow6Nq1h65de7jvfT+B1pq5sRHvwsmzZxg69xrz42MUJiIUJiJMAqAJxGxC6TLhTIVwpux/VLDCrf/UqHWltihOI4NIZGctcHvhex/R6D4sa3v8ObAhdN9746HbcBTV0TzV0RbTAixVC91WR9gL4f59MxNCmduv3UYI8dZjKIOOcAcd4Q4OsXoQ11p7QbxVCF9x2+6KmwXD4KphcHWdCzYjDUE8R5czT5dzoRbE0/6+uB/EZ0kwtxTEw8tBPJjuI9I5SLp7Jx09OwiGbnyqlgRsIW4TsViMgwcPcvCgN6vVdd2m1pLx8fGWrSVDQ0MMDQ01vNbK1pJw2PsGopQi0zdApm+Au596DwCLM1P+HG6vwj11/SrVfIBqPkBuuOHTYYZtL3Snl0N3KFMmGFutouFSLF6hWLzC1NRzDXvCoX6vyl0L3/uIxfYTCKRv/o3cIKuG7un6kYGr93TX2Bp7oog90dyHjwFmOozlV7sbbjvCqIAsHiGE2F6UUmTCGTLhzJqzxbXWzJfnV62C199W3ObrglopGgbX2picEnG9BX28ML5IlzNHl3OerjmHrmmHlD/eMOEuBfEks0YHueD6IxqXSMAW4jZlGAbZbJZsNsvx48cBr7VkdHS0odI9NzfX9Nx8Ps+FCxe4cGG5mpzNZhuq3N3d3Zj+6l+JjiyHH3uCw489AUBxcYGvfPYzDGY7mRm+zszIEDMjQ+RnZ3BKFvlRi/xo45/ejKBDOL1c7Q7594OJKqtdE1gqj1AqjzA9882G7cFgllh0ZfA+QDCY3dILDJVSXgU62xi6v/WVb/Dw4fuxp0s400Xs6RK2f+vm16jguODMlHBmSg0tJ0vMVBCzI9IygBth+fYuhNi+lFKkw2nS4TQHMgdWPW7pgs31QvhkcZKyU27rcxcNg+uGwfV1gnjYD+LdjkPWXqTbmWv765PvwELcQQKBADt37mTnzp21bblcrqm1pFxu/iY0NTXF1NQUP/zhDwFvAkp/f39DlTuVSqGUIpJIktyxm/uffLLhNcqFvBe2h4eWb4evMzc+iluh1mZST1ku4ZRX5Q5nyrUQHkpVmpaBX1KpTFGpTDE7992G7ZaVbOjxjsb2EYse2NLJJkop3CCEdiYJ7Uw27XdLdkPgXrp1Zoo482tXbZz5Cs58hcrl+aZ9RizQELjNusq3EQvIpBMhxG2h/oLN/Zn9qx6ntWaxurhuW8pkYbLteeIlw2DIMBhqY5b4ShKwhbjDxeNxDh06xKFDXs+c67pMT083tZZo3TjqwrZtrl27xrVr1xpeaylwz87Osri4SDwer4W1UDRG3/5D9O1v7M9zbJv5ibHG4D1ynZnhIcqFPMXpMMXpxh43ZWiCybr+7qXgna60mGiydM4LLSebGEaEWGxf3cWVXsV7O0w2McIWQX8FyZV01cGeKbUO4HMlWGMUuZuvUslXqVxbbNqnQmbDBZdeEPfuGwm56FIIcftRSpEMJkkGk+xL71v1OK01uWquFrjrJ6fU304WJynaLVr32iQBW4i3GMMw6Orqoquri/vuuw+ASqXS1FoyP99cFc3lcpw/f57z588D8OqrrxIMBmtTUOo/Ojo6aiMDTcuio3+Qjv7GhRS01hTm55bbTJYC+MgQC5MTlOdClOdCzF+ue5LSBBNVv8fbr3z7981g68TpusU2J5t4wXu7TDZRAZNAT4xAT/PFntpxcWbLXuCeKWFP+bf+Y+zVZwPqskPVX1a+iWUsV747wljZMJbfhmKmwyhTwrcQ4vallCIRTJAIJtYcYbi0sM9EcYKpwlTt9lf51bY+z6YGbKXU+4D/DJjAx7TW/27F/p8C/he8uowN/I7W+tubeY5CvBUFg0F27drFrl3LixcsLi42tZZUKs0tC0vhfHR0tGlfLBZrGb4zmQyBgNemEEtniKUz7Dh6T8Nzq6USs2MjTA9frwXv2eHrzIwOU1lQVBaCLFyrf8Yqk03SlaaVKmvPuJ0nm5hGrd97Je1qnIUK9nQRp1b1XqqAl9CVNRaVsF3s8QL2eKF5n6GwMqHldpP620wYFZCJJ0KIO0P9wj57U8tBfNsFbKWUCfxX4N3AEPCiUupzWuszdYd9Ffic1lorpe4BngEOb9Y5CiGWJRIJDh8+zOHD3v+CrusyNTVVC9wXLlygUqm07Odeks/nyefzDW0mS9LpNB0dHU3hO51OYxgGgXCY7t176d7dWGFwXYfFqcla6K4P4MWF+TUmm5TrLrK8MyebLFGGwkqHsNIhWPGXUq01bq7aVPVeCuJuYY2JJ66uhfSm/+oKzGTIC9zZCOaK9hMjJH8wFUK8dWzmd7yHgIta60sASqm/AH4KqAVsrXX93ytjyPpnQmwbhmHQ3d1Nd3c3999/PydPnuSJJ56gUCjUZm+v/HCc1Sulc3NzzM3NcenSpYbtpmmSyWRaVr7j8TiGYZLq7iXV3cue+x5oeG5xcYGZkeFaf7fXdnKd+fFxf7JJY/V5syabbCdKKcxEEDMRJLSrxUWXRbup33up7cRdWOOiSw3OfBlnvkz5UouLLuOBhhGDS/O+zY4wRtSSiy6FEHeUzQzYA8D1usdDwMMrD1JKfQj4t0A38IHNOTUhxM1QShGLxYjFYg2TS8CreC8sLLQM3nNzc00XVS5xHKc20WSl1fq9Ozs7CYfDRBJJBg4lGTh0V8Pz7GqVubGR5gknI0MUJortTzZJVlCrjJ5ea7KJ43Zy6tSnCUcGCIcHiYQHCPsfltV8ceNWMiKWt8z7YKJpn1txcGaaL7i0p4s4c+U1SyJurkolV6VydaFpnwpbWJ1hehzFfPGyV/3uCGNlwpjpEMqS1hMhxO1FrfZDbsM/kVJ/B3iv1vqj/uNfAh7SWv+jVY5/HPhXWut3tdj3G8BvAHR1dZ145plnbt2J30FyuRzx+Pb6Yb4dyfvUnjfzPrmuS7FYpFgsUigUGm5b9Xm3IxAIEI1GiUQiDbfhcLg2z3slrTXVfI7S3Ayl2Wn/dobS3AzVfPMFgDcz2WR9MSALdKLoBNWJIru8Td26pXw3lAuBIgQKECioFbeg9M1VqDUaOwx2BKoRTTXaeN8JAW/B4rd8n2qPvE/tkfepfU899dTLWusH1jtuMwP2I8C/1lq/13/8LwC01v92jedcBh7UWjeXsnyHDh3SSxMNxNpOnjzJkyvmFotm8j6151a9T6VSiZmZmZaV77X6vdeSTqdbVr1TqRSG0bo6Wi4UmB0dbppwMjs6guus6FNeMdnEC91rTzZpl2Ul/Gr3IOHwgF/9HiQc7icSGcSy0tu+vUK7Gme+3HChZX3ft66+iffIVF6lOxPyl5evq37fwe0n8n2qPfI+tUfep/YppdoK2JvZIvIicEAptQcYBn4e+IX6A5RS+4E3/Isc7weCwPQmnqMQYouFw2H6+/vp7+9v2K61Jp/PtwzeMzMzbfV7v/HGGw3bTdNseaFlZ2cnsViM3n0H6N3XuMKY6zjeTO+60O1daHmdhWv5lpNNgomq9xGv1u4H/PuGuXaRw7YXyeXOkcuda7nfNGOEw/2NATyyfD8Q6NzygKkMLwRbmTCsWCeidtHldJHXnv8h+7t348yWsGdKOLMlnIXK2lfjONq7WHOq2HzhJf7Mbz9sW5mQf+uFcLMjjBGU5eaFEBtv0wK21tpWSv0W8GW8MX0f11qfVkr9pr//94EPA7+slKoCReAjerNK7EKIbU0pRTweJx6PN4wTBK/lZH5+ftV+79U4jsPk5CSTk5NN+0KhUMvZ3p2dnWT6Bsj0DbDvxPJlJFprigvzdbO8vQstp4eHWBifID/W6luZxoraTeG7/vF6rSeOkyeff518/vWW+w0j7FfA+4mEBxuq3+HwAMFg15atcgmNF10uXtGknmz8b6ttF3uu7PV+z5Zqt0sB3M2vMfUEf+b3WJ7qWL7lfiMWqIXvpgq49H8LIW7Sps5N0lo/Czy7Ytvv193/XeB3N/OchBC3P8MwyGQyZDIZ9u9vLJFWq1VmZ2dbVr1zuRYLrfjK5TIjIyOMjIw07VttvndHRweDR44xeORYw/Ff++pXOX7XYebGR5gbG/U/RpgbH2N+YoxCIUBhvNVZaKyw41W8W4TvUKKKEVi7vcJ1SxQKb1AovNFyv1JBP3wvX3gZjgz6YbyfUKgHtdqVnZtAWQaBbIRAi3nfAG7Z9hbcmfFD94oArivrvD/5Km6+SvV684qXS6MHzY5QQ9V7KYDLqpdCiNXIYFIhxB0tEAjUxguutFq/99TU1JoXW64231spRSqVagre5UqFVE8vHf0DTa/lOg6L05PMLgXv8eUAPj8+hl2qYJcsmGwVMDVmyGnZghJM2oQSNkZgnQqvrtRmfreilEUo1NfUerLUFx4K9W7pcvNGyMLotQj0tljtUmvcfHU5gNdVwJ2ZEvZcGZw1/kJQN3qwcrl5+gmWwkqHW1bAzcyd2/8thFifBGwhxFvWRvd7a61X7fd+8cUXyWQydHR01G6XPtIdWVLdvXDPfY2v57rk5mYaq95jo8yNjzE3PkKlWMQpWxTLFsWp1hVeM+jU+r2XPkJJm3BaE4iXMay1p7ZobVMqXadUug5zzfuVMgkFe/zw3e+PIRys6wvvwzCCa36OW0UphRkPYsaDBHc0jx1cWvHS8avfKwO4s7hO/7ct/d9CiNYkYAshxAq3ot/bdd3aca0+XyqVagjdSx+ZTIYdHVl2HLm74TlLPd+zde0m3q0Xxks5r+XBqZg4MyalmXDL8zICzor+b5top0E47WBGiiizuOZ7pbVTW3ynNUUo1FNrP6mvfkcig4RC/ZhmaM3PcavUr3gZ2ptq2q9t1wvbq1TA11z1kjb6v+OBugAebmxFSYdQpvR/C3G7koAthBA34Gb6vcfGxtZsOamvfK9c2RIgmUw2he6l+wOH7mpaWAeglMv5YXtkufXED9/5udnacW7VpDRrUpptHcCV5S4H8HiVeLdFtEN5F2CG8mijRe9y41dHuTxGuTzG/PzLLY8IBrsIhwdw3QAXL35vuRc83L+li/EoyyDQFSXQ1XoWuVuysWfLtQr40vSTpQC+3vjBpcV3WK3/OxWqu+gy1ND/LescC7G9ScAWQogNslq/98mTJ3nkkUeYnZ1lZmam6WNhoUV/b52FhQUWFha4cuVK075YLNay8t3R0dFyzCBApVRs7Peu9X2PsjjdOFFF2wbluRDlOa/KPH22fm8GZbr+KMIKie4A8e4A4YwmEC1DYAGXOdZLg5XKJJWK93mvXnuxab9lperaTgaaAnggkNmSXmcjbBHss6Bv9f7v5eBdbgzgs2Vw1+n/nivjzJWpXG5een4fBiPPfxczEcTwp7CYiQBGvO6+v12FTOkFF2KTScAWQohNEAqF6O3tpbe3t2lftVplbm6uZfhea1l5WL7g8vr16037IpHIquG7a9ceunfvbXqOXakwPzFem3gyOzbKvB/A5yfH0W5jVVY7BpWFIJWFILnh+j1RIIoyegjEqsS7LJL9EWJZg1DKwQwX0eYctjMNrD7DHMC251nMzbOYO91yv2lGCYX6iUT84B3yw7f/OBTs3vRRhPX93+xMNu33+r+Xqt9l7JliQyuKu7B2b7xC4eaquLkqjLZuQakdGzC8sB1fDt0N95dCejwgYwmF2CASsIUQYosFAgG6urro6upq2mfbNvPz8y3D9+zsLK67ehtCsVhkeHiY4eHhpn3BYHDV8N0xMEjn4I6m5zi2zcLURNO0k7mxUeYnxnDs5p5k7Soqi0FmFmHmUv2lgCbQCaqDaMYkszMBsTK9u5IEElWMYB5XzVKpjuG6a4dNxylQKFykULjYcr9SAcKhvlrgrq9+R8ID/iSUzb0Q0+v/DmOlw4Saf89BV13sufqxg+WGBXjW6/9e+VqOP8JwPUbU8qvgfgD3K+JG0gvgS2HciFgyolCINUjAFkKIbcyyrNq4v5WWLrhcCtsrA7jdIvAuqVQqjI2NMTY21vJzrha+U929ZHr7m57jug65melaq8nsUvAeH2V2fBR7tWXutaIw41KY8doghp+vD4ExgpGjdOzoJL0jRrwrQDjtEoiW0dY8VXuCUmkEx1l9njmA1lWKpWsUS9dWOWLpQszlFTFXzgY3zdZTWm4VFVi7//vk107y9hOP4C5WvEkouQruYhVnseJtyy3fv5Gl6N2CjVuwsSfWOdBQLargfgD3A3otjMu0FPEWJAFbCCFuU/UXXK7kui65XK5l5XtmZmbNiy5t22ZiYoKJieaUZZpm7XOuDN/pTCfJbDc7j93b8BytNfm52bppJ6MNE0/KhdVbHCrFEmMXhhm70LwvFEuS6T1EeqCTVF+EaKdBMFHFDBWoupOUSsOUSiNUqzNrvIvQeCHmKy2PCAQ66qrezf3glpXc3D5nA6xUCFJrT2DRWqMrDs6CH8BzFT94+wHcf+wsVnFz64wlrLc04nChQnWdQ1XQxEwGMfwKeC2Mx1f0j8eCKFOq4uLOIAFbCCHuQIZhkEwmSSaT7N69u2Hf0pzvVi0nMzMzFIurj+ZzHIepqSmmpqaa9imlSKfTLSvf6XSawbuOMXhX4yqXWmtKuUVmR0f4zsmv0ZtOMTc2wuzoCHNjI2uG73I+z9gbFxl7o7k1JJxIkundT6bvCdK9WeI9ASJpCMTKdeHbC+Dl8jjrJctqdYZqdYbFxdda7jfNeEPoblgZMzxAMJjdkgsNlVKokIXRZUFzB1ID7foL8yxWcP0KeH1F3F2s1Lbp0tp98w2vW3Gwp4owtfbIR5S/dH28rhpea1NprJarsFy4KbY3CdhCCPEWUz/ne+fOnU37C4XCqhNP8vnVA6/WmtnZWWZnZ5sW2gEaZn2vrID3HzxM58gYjz35ZMPrFRfmmR0d8VtORpbvj45QLa/eU1xaXGB0cYHR18837Yum0mT6+kn3Pkamt5+uvh5iWYtgoortTNWFby+Al8qjaL12ndZxcuTzF8jnW5TaAcMIehdiNvSAD9YCeCjUs6UrYoLXF74UYNejqw6OXxF3/Qr4ckW8MYyvuVpmw4uyfOFmc+dSI0st94cnGltS6gO6av/3ACE2lARsIYQQDaLRKNFolIGB5qXdy+Xyqm0ni4trz8Sen59nfn6ey5cvN+2Lx+MYhsHExASJRIJ4PE4ikfDuZzrZu3M3kUikVrWstZ34gXt2dLhW9Z4bG8Wurt4CU5ifozA/x/C5M83n0dFJprefdN8OMr0P09XXT3qwh2jGoup4Pd/1AbzoP3bdtauzrrvekvT+ipj1AbxuafqtXJCnFRUwsTpMrI7W89OXaK3RRbuhJ7wxmC9vcwvV9ltUbF0bY7iWfZgMf+s7dSMMAw1TUxou5IwFpEVFbBgJ2EIIIdoWCoXo6+ujr6+vaV+lUll13OD8/Pya4wZzOe9CxTNnmkPvEtM0G4J3LYhnuujfuZeD/r5IOExudqax4u3fnx8fbTntpHYeM9PkZqa5fuZU4w6lSGa7SPf2k+ntJ9P3AOnen2Tnzn6SXd24OkepNFQL4MX6CnhpGNtunmVdr2FFzPnmWeAAwWC2FsBdt8zlK6cJBrMEA53ebbCTYLAT02x9YeRWUEqhogGMaIBA99rnpR3Xb1Gpv1hzuV+8Pozryg20qJRs7JINk220qEQDDRNUloL4yos4ZYqKWI8EbCGEEBsiGAy2XGgHvAsn15r1vda4wSWO49Sq4GsxDGNFBbyHzM797EwkiMWiGLaNnVugODvNfG3iyQjzE+O4zirBTWsWJidYmJzg2qkfNuxShkGyq9sP3gNeCO87Tk+vF74N08S2F1uE7+UAvrTQzloqlSkqlSkWFl4F4NKlr7Y8zjSjBAKdfuDOEgx01O4Hatu8/d4iPdtj9rUyDcxkCDO5fqXeLTtNF2g6C5Wm/nFnsYzSbQZhDW6+ipuvwlhh7WNXTFGpXcDZYpss9PPWJAFbCCHELWdZFtlslmw227RvKTh/4xvfYN++fSwuLpLL5VhcXGy4v9bkk3qu69ZWv1zLUi96oqOf+M6DZGMxAqaBqlZwC3kqi3MUpybJjY+wODmJ1q1/CdCuy/z4GPPjY1x5tXEKiWFapLp7/J7vper33fT0vZfE3izKMPz3oEy5PLIifC8H8HJ5DK3bq9o6TgHHKVAqNS8+1MwgGOyoVcEDfhU8GMguB/RgZy2wm+baLSGbxQiZGKEIVufa4xNPfv0k73josVoYr1XDc8tTVW6qReVGpqjUL/SzZptKABWQkYZ3CgnYQgghtpRpmrULH++5555VjyuXyy2D98r75dVmbq+gta49Z42zQ2V3Ett5mEg4TNA0MFwHyiXs3AKl2RnKs9Mou4KybdSKhOY6tt8f3rzYjxkIkO7p8yveXvj27h+mv7ejFr4BXNemUpmgWByiVB7h7NnvsXNHmkplmkp12q9uT1OpTKN1e7+I+K9cq4yTb74gtOmczbgXyP0qeCBY356yXBkPBrP+6MItro4rMGMBzFiAQE/zkvb1tLM8RaUWvBump1RrF3Hq0q1Z6EeFzYaLNOvbVBpHGgZQ5vb4y4NoTQK2EEKI20IoFCIUCrVcdKdepVKpBe61Avla4wjraa3J5fPkmiaoWJDu9j58QcvEApTtVcGdQg5lV1F2FaPq3SqnitIap1pleuga00PNC+BYwRDp3r7l4N23VP0+QLrnQc6fS7N//5Mtz9Vxcg2Buz6AVyv+fX+bba9d5V/JcXIUizmKxdUW7VmmlEUg0LEcwGvhu65aXusf78AwtvYiTmUqzGQQM9nOFBV3eaZ4rsUCPze50I8uOdilNkYaAkZsadXNxpaUhr7xuNf/Lv3im08CthBCiDtK/TLwa6lWq+RyuYbg3SqIFwrr9OPWqdgOXv3YhGjS+2hBOQ6qWl4O3/7t0odrV5i8fpWpa1eanhsIRzBCYUa/9izRZJJIMkU0ma7d9x6niCb3kOi4F9MKrHq+rluhUp2hUpnyw3d9AJ/2t8/Utml9A5Vb7VXdK5X1loX0WFbCC94N/eN1bSrBTkL+fstKbGlfswoYWJkwVqaNKSoVZ5VFfpqr5W2PNATcvI2bt7HH1+sXZzmI19pU6qvkAQJ5cCuOrLq5gSRgCyGEeEsKBAKrroRZz7btpiDe6v5aM8JX0qaJbmfah2M3hG+jWqFiV1GOTf76da8abtsox2a1uBmKxogkk0STaT98J+tCeKoulB8llU1hBVoHcq01tr2wXB2vTtcF88ZQXqlMr7uE/Uq2vYhtLwLNYxxXUirot6p0rugfb2xT0XoG285jmtGtXegnZGFl1+4Xr400XNGO0jjS0H+cv5F+cXAXKrhr9IvvwmTkWy9gxCzMdBgzFcJKhzDrPqx0CCMelGp4myRgCyGEEGuwLIt0Ok06nV7zOMdxyOfz6/aI5/P5NUcWNjAtXNOC0NrhDK29kG3bdaHbu604VXK5Isb8Qm07rrtqIA9Gon7wbq6OL4fyDiLJPXR2p7CCrVsqHKdMdUV/eGMon6ZS9VtXqjNtX8TpfbmV2vL26/nGN//vKGViWcnaR6B2P4EVSNU9TmIFko2PreSmXNzZMNKwZ+1jtaNxCy1aU+r7xpf6xYvt/9XBq4rnqA6v8suRqTBToVUDuJkOYYQkWoIEbCGEEGJDmKZZW55+La7r1oL4elXxtoO4UmgrgLYCwDph3DuJpiBu+JXxql2lkMuj5uaWQ/sa5dJAOEI0lSKaqAvlqTTRxHIojyQHiaeOEsmmCASbe621dqlW55oDeENl3O8hr07jOO3/tcB7fYdqdZZqdfaGnrfEMIKtA3qgVWhfGdITGMb6fd03Qpk3sOqm7TbOE28xQSU/MU+gYqzfouLo2gWbq11Kq8LWquHbTIcwE6G3xII+ErCFEEKITWQYRm1G91pc16VQKLQM3leuXCEajZLP58nn821fsFl3EmgjhA60eWFhrVWluUJedaoUF3LMzs6ibBtcZ9XqOEAgFPZDuB++/WAe9YN5JJkkmjhIMpki2p0iEGquHjtOgUpdb3i1rjVl+aLOKfL5CQyjjOuuP8FjLa5bWZ62chMMI+IF7pWBvM2ArtTN90Yry8BKhyG9ehX+5MmTPPH4O3BzVey5Um2VTGeujD1Xxpkv48yVcPPrV8N1yaY6ZlMdW+WXIIU377xV+E6HsVJBVMS67WeHS8AWQgghtqGlBXPi8Ti9vb0N+06ePMmTTz5Ze+w4DoVCoRa46z9yuVzTNnuN1SxbqrWqtHGsdhuDuF8ZV7aN4VSx7SqlxUXmZme87etU6a1QqBbEo8mkH8JTRBL1oXyQTDJFpNcL5EvhbOl9ct1yrce7Wp3Hthew7QWq9oK33d9W9bfX9le9W63Xm3a9NtctUq4UKVfGb+r5phm/gUDu7w+ksKwEphlvK6wqo26Kys5Vvo6K44ftuvBdC+Bl7LkS2OtUwTXe8fNluLrKuQTNlgHc8kO4mQyirO09plACthBCCHGbM02zrao4eBfTVSqVlmG81UehUGi/VQVAGehAEB1osy3CsRsC+HIo9+7bjk15foG5mZk1L+ZcYgWCRPzqeNF2KJ5+xWtZWWpdSaaIpvqJJO+iM7N6D/kSrbUf0OeXA3jVC+ZVezmst9xmL/oBvf3+8pZvkZPzLhotj9zEsw0sK0HASmEFEi1bXVw9wfhEkVCwi1Com2Cwu2XfuRE0MbqiBLpaX6CrtT9LvEUA9+6XcBfX/2VFVxzsiQL2RIGWU+2VNxmldStK2OsFj25tFVwCthBCCPEWopSqzRRfb5QheK0qxWJx3ar40ke7K27WmBbatNBBWHdidO1iztbtKoZTxbFtKvPzLExPo7TLmWtrTyUJRiLehJWUf0Fn3W3tws6lYJ7Yi2HeWLuGN5s8XwvcVXuhVjH3quaLfkCfb6yq14L6WgshtcP1X2se1uiUee21P2t47I1N7CYU7CIYWrrtIhTsJlgXxOtHJiqlvFUp40EYbP3LnrbdusDdHMCd2fL6s8M1uIsVKosVuN76/VEBwwveqdVaUUKowK2rgkvAFkIIIcSqDMMgFosRi629EuKSarXadnU8n8/juu0vxNJ4MWcbXKflnPHlxxXKlQrl8VHmxkfb+vyReGK5V3xFIF8O6t79UCyGUgrLimNZ8fa/zjpaO9j2UkCvr6Iv+oF8RcW8Wr9t8YYvCF2y1FJTKLyx5nGGEfICdy2Idy8H8bpAHgx2oJTp9YR3rr7M/dK4QntlH/hSb/h8GWehsu6YQl11sSeL2JOrX59gxANe+E6trIT7VfDYzS/SIwFbCCGEEBsmEAi0NdYQvDBVKpXWrIjXf5RKN3ixomGigyZOcJ0xe9pdddGflY+LiwsUF9tbAdMwLW/EYUOLSpJIi2AeTaYIhJvPUymTQCBJIJAEBm/s6wdc18Zx6nvPm6voV6+eoTMboFKZpFyeoFKZarvv3HXLlEpDlEpDax7nfR2dhEJda1TGuwmFshjREMFoAPpb/1KiHRdnodIygC8Fc11evy3HzVVxc1WqQ2uMJVwRwNslAVsIIYQQW0IpRSQSIRKJ0NXVte7xtm2vejFnq4s7HafN3mdloAPeVJU16+kNLSpVDLuyehjXGtexyc3OkJudaes0vAs607UJK/W94yvbVSLJFKa1fowzDAvDyBAIrL6g0vXrJ7n3nifrvkx/bGItcE9QrkxRKU9QrkxQKU96t5VJHKe9lU61dupW9jy99vtgpQnVV79XVMRDoW6CiS6C6eSqfdZuyW7sA/dDeO3xQnn9niRH40yXcKZvfAqNBGwhhBBC3BYsy2pr1jh41fHnnnuO48ePN4w4bHVbLre8lK7ZihaVteK7chzUGgF8KZzXL/pjl8ssTI6zMNnetJFwLO5Xx5N1feMrq+P+ap3xBMpor+dYKcNfKbODePzQmsfads4P4pN+EJ9sCuLl8iS2PdfW5/Zecw7bniOff33N4wwjslwRD/m94cHu2rZgvItQRxfhQA9KNX7t2tXeXPBa9bvSOKJwvoxbuMFpO3UkYAshhBDijqOUIhAI0NXVtW51vFKpkMvl1gzhi4uLFArtVWsBtGmizci6q3AqrTEcG6plVLWyamtKqwkqpXyOUj7HbBvDRZRheKMNG6riXjCfHBvjQiRAOJYgHI8TjieIxBNYodC6kziW+suj0T1rHue6ZSqV6eWK+FIVvOyH8soklfIklepU21NXXLdIsXiNYvHa2l+7sggGs34V3G9LWaqMR7sJZbxwHgvuwjCW+/vdstMwgtCZK8PvtnVqErCFEEII8dYWDAbp6OhYd6qK4zhtBfEbuXhTK4VjBcAKrLkIpwJMvDCuKmXcUrFWIW8M461X3tSuS2F+jsL8XMvXv/aNrzRtMy2LcDzhf3jBuz6Ee49jdcd4+0KRaFO13DBChMP9hMP9a78f2qFSnW3ZjlKrkvu3rtveXx60timXxyiXx2DNoSyKQCCzoj+8m1Cki2Dau4CzXRKwhRBCCCHaYJomqVSKVCq15nH1q3CuF8bbXfRHAzYKzABEAhBZfSpJwFB+GHf8MF7ALRaae8fXmW/u2Db5uVnycze2xLxSBqF4nEg83hzIV96P1d+PY5gmoWCWUDBLgiOrvx9aY9uLfuD2A3itPaXutjJxA6MONdXqDNXqDOTP39DXvJIEbCGEEEKIDVS/CudatNaUy+V1e8QXFxfb7xMHqq6mCqBMCEW9jxa/E1imQdA0ccslQpYFjo2ulHErZexSAV2polzH+3AclGuD4z9eI5xr7VJaXKDU5rSVesFItCmIR2rV81ZBvYNkYhdWYPXRjY5TqusPX9Ga4t+WyxNesF5v/l+bJGALIYQQQmwBpRThcJhwOLxun3i1Wq0F7rXCeD7f/txr23GxHReUScnRgAnBqPcRX33qiHfuYCmFAbUAru2qH9Ar/jYb5TrePHJnOajXHrcIs5VigUqx0PaFnkusUMgL4ytCeCi23FPuhfI+wvGDpFLe9kAoXOszd12banV6uSJeVwVf2gaX2jufGzp7IYQQQgix6QKBQNt94vl8ft2qeC6Xu7FFflbQGqq1KrYBpuG1r4RaL6PeikJjaO2FcNtGL13M6TrLlXL/tvZ4ZTXdfy27XCZXLpObnrqhr8Mwraa2lUhD+8pewvF7SMUThDMJ4K/bel0J2EIIIYQQdwjTNNsaZei6LsVikVwux/PPP8/hw4cplUqUSiXK5XLDbattbyacL9EoHKXqwvnaE1dW+UJq1fJVK+VrVNMdx17z4s+bJQFbCCGEEOItxjAMYrEYsViMjo4OjhxZ/YLClbTWVKvVptC9ViBvtU2vc5Flm18I2jBqs8lvSl11nBWBvBbUl+63SQK2EEIIIYRom1KKYDBIMBgkkUjc1GtoralUKm8qpJfL5Y0J6abpzS1/869UIwFbCCGEEEJsKqUUoVCIUCjU1sqcrbiuuyEh/VaQgC2EEEIIIW47hmHUprCsN5t8Na7r1oJ2O4G8XRKwhRBCCCHEW5JhGEQiESKRm7jAcq3X3dBXE0IIIYQQ4i1OArYQQgghhBAbSAK2EEIIIYQQG0gCthBCCCGEEBtIArYQQgghhBAbSAK2EEIIIYQQG0gCthBCCCGEEBtIArYQQgghhBAbSAK2EEIIIYQQG0gCthBCCCGEEBtIArYQQgghhBAbSAK2EEIIIYQQG0gCthBCCCGEEBtIArYQQgghhBAbSAK2EEIIIYQQG0gCthBCCCGEEBtIArYQQgghhBAbSAK2EEIIIYQQG0gCthBCCCGEEBtIArYQQgghhBAbSAK2EEIIIYQQG0gCthBCCCGEEBtIArYQQgghhBAbaFMDtlLqfUqp80qpi0qpf95i/99VSv3I/3hBKXXvZp6fEEIIIYQQb9amBWyllAn8V+D9wBHgaaXUkRWHXQae0FrfA/wvwB9s1vkJIYQQQgixETazgv0QcFFrfUlrXQH+Avip+gO01i9orWf9h98FBjfx/IQQQgghhHjTlNZ6cz6RUj8LvE9r/VH/8S8BD2utf2uV4/8JcHjp+BX7fgP4DYCurq4TzzzzzK078TtILpcjHo9v9Wlse/I+tUfep/bJe9UeeZ/aJ+9Ve+R9ao+8T+176qmnXtZaP7DecdZmnIxPtdjWMt0rpZ4C/m/A21vt11r/AX77yKFDh/STTz65Qad4Zzt58iTyXq1P3qf2yPvUPnmv2iPvU/vkvWqPvE/tkfdp421mwB4CdtQ9HgRGVh6klLoH+Bjwfq319CadmxBCCCGEEBtiM3uwXwQOKKX2KKWCwM8Dn6s/QCm1E/gM8Eta6wubeG5CCCGEEEJsiE2rYGutbaXUbwFfBkzg41rr00qp3/T3/z7wr4BO4P+nlAKw2+lzEUIIIYQQYrvYzBYRtNbPAs+u2Pb7dfc/CjRd1CiEEEIIIcTtQlZyFEIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA0nAFkIIIYQQYgNJwBZCCCGEEGIDScAWQgghhBBiA21qwFZKvU8pdV4pdVEp9c9b7D+slPqOUqqslPonm3luQgghhBBCbARrsz6RUsoE/ivwbmAIeFEp9Tmt9Zm6w2aAfwz89GadlxBCCCGEEBtpMyvYDwEXtdaXtNYV4C+An6o/QGs9obV+Eahu4nkJIYQQQgixYTYzYA8A1+seD/nbhBBCCCGEuGNsWosIoFps0zf1Qkr9BvAb/sOyUuq1mz6rt5YsMLXVJ3EbkPepPfI+tU/eq/bI+9Q+ea/aI+9Te+R9at+hdg7azIA9BOyoezwIjNzMC2mt/wD4AwCl1Eta6wfe/Ond+eS9ao+8T+2R96l98l61R96n9sl71R55n9oj71P7lFIvtXPcZraIvAgcUErtUUoFgZ8HPreJn18IIYQQQohbbtMq2FprWyn1W8CXARP4uNb6tFLqN/39v6+U6gVeApKAq5T6HeCI1nphs85TCCGEEEKIN2MzW0TQWj8LPLti2+/X3R/Dax25EX+wAaf2ViHvVXvkfWqPvE/tk/eqPfI+tU/eq/bI+9QeeZ/a19Z7pbS+qesMhRBCCCGEEC3IUulCCCGEEEJsoNs6YK+39LrwKKU+rpSakHGGa1NK7VBKfV0pdVYpdVop9dtbfU7bkVIqrJT6vlLqVf99+jdbfU7bmVLKVEr9QCn1ha0+l+1MKXVFKXVKKfXDdq/SfytSSqWVUp9WSp3zv1c9stXntB0ppQ75/5aWPhb867rECkqp/9H/Xv6aUuqTSqnwVp/TdqSU+m3/PTrdzr+l27ZFxF96/QJ1S68DT69Yel0ASqnHgRzwx1rrY1t9PtuVUqoP6NNav6KUSgAvAz8t/6YaKaUUENNa55RSAeDbwG9rrb+7xae2LSml/ifgASCptf7gVp/PdqWUugI8oLWWWbxrUEr9X8C3tNYf8ydyRbXWc1t8WtuanxeGgYe11le3+ny2E6XUAN738CNa66JS6hngWa31f9vaM9telFLH8FYgfwioAF8C/oHW+vXVnnM7V7DXXXpdeLTW3wRmtvo8tjut9ajW+hX//iJwFllttIn25PyHAf/j9vxN/RZTSg0CHwA+ttXnIm5/Sqkk8DjwRwBa64qE67a8E3hDwvWqLCCilLKAKDe5Rskd7i7gu1rrgtbaBr4BfGitJ9zOAVuWXhe3jFJqN3Af8L0tPpVtyW97+CEwAXxFay3vU2v/O/BPAXeLz+N2oIG/VUq97K/WK5rtBSaBT/htRx9TSsW2+qRuAz8PfHKrT2I70loPA/8BuAaMAvNa67/d2rPall4DHldKdSqlosCP07h4YpPbOWBv2NLrQtRTSsWBvwJ+R2awt6a1drTWx/HGaj7k//lM1FFKfRCY0Fq/vNXncpt4TGt9P/B+4B/6rW2ikQXcD/wfWuv7gDwg1x+twW+j+UngL7f6XLYjpVQG76//e4B+IKaU+sWtPavtR2t9Fvhd4Ct47SGvAvZaz7mdA/aGLb0uxBK/p/ivgD/TWn9mq89nu/P/PH0SeN/Wnsm29Bjwk35v8V8AP6aU+tOtPaXtS2s94t9OAH+N1wYoGg0BQ3V/Mfo0XuAWq3s/8IrWenyrT2SbehdwWWs9qbWuAp8BHt3ic9qWtNZ/pLW+X2v9OF7b7ar913B7B2xZel1sKP/ivT8Czmqt/9NWn892pZTqUkql/fsRvG/Q57b0pLYhrfW/0FoPaq13431/+prWWipDLSilYv6FxfgtD+/B+5OsqOMvxnZdKXXI3/ROQC7CXtvTSHvIWq4Bb1NKRf2fge/Eu/5IrKCU6vZvdwI/wzr/rjZ1JceNtNrS61t8WtuSUuqTwJNAVik1BPw/tdZ/tLVntS09BvwScMrvLwb4l/4KpGJZH/B/+VfmG8AzWmsZQSfejB7gr72f71jAn2utv7S1p7Rt/SPgz/zC0iXgV7f4fLYtv1f23cDf3+pz2a601t9TSn0aeAWv5eEHyKqOq/krpVQnUAX+odZ6dq2Db9sxfUIIIYQQQmxHt3OLiBBCCCGEENuOBGwhhBBCCCE2kARsIYQQQgghNpAEbCGEEEIIITaQBGwhhBBCCCE2kARsIYQQLSmltFLqZ7f6PIQQ4nYjAVsIIbYhpdR/8wPuyo/vbvW5CSGEWNttu9CMEEK8BTyHt/hRvcpWnIgQQoj2SQVbCCG2r7LWemzFxwzU2jd+Syn1RaVUQSl1VSnVsBS7UupupdRzSqmiUmrGr4qnVhzz95RSp5RSZaXUuFLqv604hw6l1F8qpfJKqUstPse/8j93WSk1ppT641vxRgghxO1EArYQQty+/g3wOeA43vLGf6yUegBqy0R/CcgBDwEfAh4FPr70ZKXU3wf+T+ATwD3AjwOnV3yOfwX8d+Be4FPAx5VSu/znfxj4J8D/ABwAPgh8f+O/TCGEuL3IUulCCLEN+ZXkXwRKK3b9V631P1NKaeBjWutfr3vOc8CY1voXlVK/DvwHYFBrvejvfxL4OnBAa31RKTUE/KnW+p+vcg4a+Hda63/hP7aABeA3tNZ/qpT6n4C/DxzTWlc36msXQojbnfRgCyHE9vVN4DdWbJuru/+dFfu+A3zAv38X8KOlcO17AXCBI0qpBWAA+Oo65/CjpTtaa1spNQl0+5v+Evht4LJS6st4FfPPaa3L67ymEELc0aRFRAghtq+C1vriio+pNp+rgNX+RKn9/e1YWZnW+D87tNbXgUN4VewF4D8CLyulYm2+thBC3JEkYAshxO3rbS0en/XvnwHuVUol6vY/ivd9/6zWehwYBt75Zk5Aa13SWn9Ra/0/Ag8CR4HH3sxrCiHE7U5aRIQQYvsKKaV6V2xztNaT/v2fUUq9CJwEfhYvLD/s7/szvIsg/1gp9a+ADN4FjZ/RWl/0j/l/A/9fpdQ48EUgCrxTa/0f2zk5pdSv4P0c+R7exZQfwat4v36DX6cQQtxRJGALIcT29S5gdMW2YWDQv/+vgQ8DvwdMAr+qtX4RQGtdUEq9F/jf8SZ7lPCmgfz20gtprf8PpVQF+J+B3wVmgGdv4PzmgH+GdzFlAK9q/jNa68s38BpCCHHHkSkiQghxG/InfPwdrfWnt/pchBBCNJIebCGEEEIIITaQBGwhhBBCCCE2kLSICCGEEEIIsYGkgi2EEEIIIcQGkoAthBBCCCHEBpKALYQQQgghxAaSgC2EEEIIIcQGkoAthBBCCCHEBpKALYQQQgghxAb6/wPr1k6vXFAjhgAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtgAAAHoCAYAAABzQZg1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAADxsklEQVR4nOzdd3hUVf7H8fednslMei+kQICEXkQURIqKBdsiFixgw13XtmsXe5dVd9Xdn+4KKqxrRbGLlaig0ntCD6GkkN6n398fkwwZkkCAkPp9PQ8PmXvPvXPmEjKfnPnecxRVVRFCCCGEEEK0DU1Hd0AIIYQQQojuRAK2EEIIIYQQbUgCthBCCCGEEG1IArYQQgghhBBtSAK2EEIIIYQQbUgCthBCCCGEEG2oXQO2oihnK4qyVVGUHYqi3NfM/rsVRVlX/2eToihuRVHC2rOPQgghhBBCHA+lvebBVhRFC2wDzgT2ASuBK1RVzWqh/fnAX1RVndguHRRCCCGEEKINtOcI9ihgh6qqu1RVdQDvARcepv0VwLvt0jMhhBBCCCHaSHsG7Hhgb6PH++q3NaEoihk4G/ioHfolhBBCCCFEm9G143MpzWxrqT7lfGCZqqqlzZ5IUWYBswBMJtOIXr16tU0PuzmPx4NGI/e1Holcp9aR69R6cq1aR65T68m1ah25Tq0j16n1tm3bVqyqauSR2rVnwN4HJDZ6nADktdD2cg5THqKq6n+A/wD069dP3bp1a1v1sVvLzMxk/PjxHd2NTk+uU+vIdWo9uVatI9ep9eRatY5cp9aR69R6iqLktqZde/66shJIUxQlRVEUA94Q/dmhjRRFCQZOBz5tx74JIYQQQgjRJtptBFtVVZeiKLcA3wBa4A1VVTcrivLH+v2v1Te9GPhWVdWa9uqbEEIIIYQQbaU9S0RQVfUr4KtDtr12yOO3gLfar1dCCCGEEEK0HaloF0IIIYQQog11/YDdTgvlCCGEEEII0RrtWiJyIugKCnCVlaELDW12f2VlJQcOHMDpdLZzzzqf4OBgsrOzO7ob7SowMJCEhASZfkgIIYQQ7abLB2zF4WTPzGvp9dabTUJ2ZWUlhYWFxMfHExAQgKI0NxV3z1FVVYXVau3obrQbj8fD/v37KS4uJioqqqO7I4QQQogeolsM69m3bmXPddfjLi/3237gwAHi4+Mxm809Plz3RBqNhujoaCoqKjq6K0IIIYToQbp+wK7PzfbsbG/IbhSmnE4nAQEBHdQx0Rno9XpcLldHd0MIIYQQPUiXD9jusHCoH522ZWWx5/obcFdW+vbLyHXPJv/+QgghhGhvXT5geyyBxD75hO+xbdMm9txwI+6qqg7slRBCCCGE6Km6fMAGCJk6lZjHH/M9tm3YwN4bbkT1eDqwV0IIIYQQoifqFgEbIPTSS4l59BHf47r163GXlqK63R3Yq2NXVFTEzTffTHJyMkajkejoaCZNmsR3333na7Nr1y5uuOEGkpKSMBqNxMXFMWHCBObPn4/D4fC1UxQFRVEICgrCbDaTmprK9OnTWbp0aUe8NCGEEEKIbq3LT9PXWOjll6O63RQ+8SQAqsOBIzcXQ1ISilbbwb07OlOnTqW2tpZ58+bRp08fDhw4wE8//URJSQkAq1atYtKkSaSnp/PKK6/Qv39/amtryc7O5vXXX6dPnz6MGTPGd77XX3+d8ePHo9fr2bVrF/Pnz2fcuHE899xz3H333R31MoUQQgghup1uFbABwq68EjwqhU89BYCnthZH7h4MSb26TMguLy/nl19+4bvvvmPSpEkAJCUlcdJJJwGgqiozZswgLS2NX3/91W8RlaFDh3LFFVegHrLCZUhICNHR0VitVpKSkpgwYQJxcXHcf//9XHzxxfTp06f9XqAQQgghRDfWbUpEGgu7+iqi77/P99hTW4Njz54uU5NtsViwWCx89tln2Gy2JvvXrVtHVlYWd911V4srFLZm9ow777wTj8fDJ598crxdFkIIIYQQ9brdCHaDsBkzyFu50vfYU+MN2f1e29xhfdr97HmtaqfT6Xjrrbe48cYb+c9//sOwYcMYM2YM06ZN4+STT2bbtm0A9OvXz3dMRUUF8fHxvscPPPAADzzwwGGfJzw8nKioKHbt2nUMr0YIIYQQQjSnW45gN9BaLOiio32PPdXVHdibozN16lTy8vL4/PPPOeecc/j1118ZPXo0Tz/9dLPtrVYr69atY926dcTFxfnd5Hg4qqrKXNFCCCGEEG2oWwdsAH1kJLqo6CM37IRMJhNnnnkmDz/8ML/++ivXX389jz76KMnJyQBs2bLF11aj0dCnTx/69OmDwWBo1fmLi4spKioiNTX1RHRfCCGEEKJH6rYlIo3poyIBFdeBA2RflQSAxmrFkJiI0kINc2eUkZGBy+Wif//+pKenM2fOHC699FK0x3jz5gsvvIBGo+HCCy9s454KIYQQQvRcPSJgA+ijokAFV9EBADxVVTj37kXfCUN2SUkJ06ZN47rrrmPw4MFYrVZWrVrFnDlzmDRpEsHBwbz11lucccYZnHLKKcyePZv09HTcbjfLli1j3759TUJ3eXk5hYWFlJWVsXPnTubPn8+CBQuYM2eOzCAihBBCCNGGekzABtA1jGQXFQF4l1Pfuw99YkKnCtkWi4XRo0fz0ksvsWPHDux2O/Hx8UyfPp0HH3wQgFGjRrFmzRqeeeYZbr31VgoKCggICGDw4ME89dRT3HDDDX7nvPHGGwEwGo3ExsYyevRoMjMzGTduXLu/PiGEEEKI7qxHBWxFUdA1jGQXN4TsSti3D31C5wnZRqORp59+usUbGhv06dOHefPmHfF8DXNiV1VVYbVa26SPQgghhBCieZ0jUbYjRVHQRUehi4jwbXNXVuLct6/LzJMthBBCCCE6rx4XsKEhZEejCz8kZO/f32QFRCGEEEIIIY5GjwzYUB+yY6LRhYf7trkrKrwj2RKyhRBCCCHEMeqxARsaQnZMMyFbRrKFEEIIIcSx6dEBGxqF7LAw3zZ3RbmUiwghhBBCiGPS4wM21Ifs2Fi0jUN2eTnO/XkSsoUQQgghxFGRgF1PURT0sbFoQ0N929zlZTjzJGQLIYQQQojWk4DdiKIo6OPi/EN2WRnOvHwJ2UIIIYQQolUkYB/CF7JDQnzb3GWlOPMlZAshhBBCiCOTgN0MRVHQx8f7h+zSUlwSsoUQQgghxBFIwG6BL2QHB/u2uUpLcRUUnPCQPXPmTBRF4YYbbmiy75577kFRFKZMmXJC+9BWFEVh4cKFHd0NIYQQQoh2IwH7MBRFQZ+Q4B+yS0raJWQnJiby/vvvU1NTc/C5XS7++9//0qtXrxP63EIIIYQQ4thJwD4CX8gOOiRkFxae0JA9ePBg0tLS+OCDD3zbvvzyS0wmE+PHj/dt83g8PPHEEyQmJmI0Ghk0aBCffvqpb//u3btRFIX33nuPc845h4CAAIYNG8aGDRvYtGkTp556KoGBgYwdO5acnBy/Pnz++eeMGDECk8lESkoKs2fPxuFw+PYnJyfz5JNPctNNNxEUFERCQgJ/+9vf/PYDTJs2DUVRfI8fffRRBg4c6Pdcb731FhaLxfe4oc38+fNJTk7GYrFw7bXX4nA4+L//+z8SExMJDw/nr3/9Kx6P55ivsxBCCCFEW5OA3QrekB2PNijIt81VXHzCQ/b111/PG2+84Xv8xhtvcO2116Ioim/bSy+9xN/+9jeee+45Nm7cyMUXX8wf/vAH1q1b53euRx55hDvuuIO1a9cSEhLC9OnTufXWW3nqqadYsWIFNpuN2267zdf+m2++4corr+SWW25h8+bNvPHGGyxcuJAHHnjA77x///vfGTRoEGvWrOHee+/lnnvu4bfffgNg5cqVALz++uvk5+f7HrfW7t27+fTTT/niiy/46KOP+PDDD7nwwgtZuXIl3377LXPnzuWVV15h0aJFR3VeIYQQQogTSdfRHWh3jwYfuU0zFMBw3M9dcVTNp0+fzl133cX27duxWq0sXryYV155hYcfftjX5vnnn+euu+5i+vTpADz++OP8/PPPPP/887z99tu+dn/961+ZPHkyVquVO++8k/PPP5+PPvqICRMmAHDLLbdwyy23+No/9dRT3H333Vx77bUA9O7dm+eee46rrrqKv/3tb76Qf9ZZZ/mOu/XWW3n55Zf54YcfOOWUU4iMjAQgJCSEmJiYo71auN1u3nzzTYKDgxk4cCBnn302P/30E/v378dgMJCens6YMWNYsmQJU6dOPerzCyGEEEKcCD0vYHchoaGhXHzxxbzxxhuEhIQwfvx4v/rryspK8vLyGDNmjN9xY8eO5auvvvLbNnjwYN/X0dHRAAwaNMhvW01NDbW1tZjNZlavXs2KFSt47rnnfG08Hg91dXUUFBQQGxvb5LwAcXFxHDhw4DhfuVevXr0IblT/Hh0dTd++fTEYDH7b2ur5hBBCCCHaggTsTu66665jxowZWCwWHn/88WbbNC4ZaWmbXq9vsq+5bQ31zB6Ph0ceeYRp06Y1OXfDyPSh52g4z5FqojUaTZPSGqfT2aRdc+dubpvb7T7s8wkhhBBCtKeeF7CPskyjJarHg2PPHjzV1b5tuqgo9FFRbXL+BpMmTcJgMFBcXMxFF13kty8oKIi4uDiWLl3KxIkTfduXLl1KRkbGcT3v8OHD2bJlC3369Dmu8+j1+iYBODIyksL6+vWGYH9ozbgQQgghRFfV8wJ2G1E0Ggy9evmFbNeBA6Ao6BuN8B738ygKGzZsQFVVjEZjk/133303Dz/8MGlpaYwYMYK3336bX375hdWrVx/X8z788MNMmTKFpKQkLr30UnQ6HZs2bWLFihXMmTOn1edJTk7mhx9+4PTTT8doNBIaGsr48eMpLS3l6aef5vLLLyczM1PmyhZCCCFEtyGziByHhpCtaTS9nKuwEFdRUZs+j9VqJajRDCaN3Xbbbdx9993cc889DBw4kEWLFvHRRx8xdOjQ43rOyZMn8+WXX7JkyRJGjRrFqFGjePbZZ496Du4XXniBJUuWkJiYyLBhwwBIT0/n1Vdf5T//+Q+DBw/mu+++azI7iRBCCCFEV6V09aW/+/Xrp27durXZfdnZ2aSnp5/wPqgeD47cXDyNFoXRx8Sgi4g44c99NKqqqrBarR3djXZ3tN8HmZmZfnONi+bJdWo9uVatI9ep9eRatY5cp9aR69R6iqKsVlV15JHayQh2G/CNZAcG+rY5CwpwFRd3YK+EEEIIIURHkIDdRhSt1huyzWbfNmdBAa6Skg7slRBCCCGEaG8SsNuQotViSEryD9n5+RKyhRBCCCF6EAnYbcwXsgMOCdmlpR3YKyGEEEII0V4kYJ8AilaLITkJTUCAb5szL09CthBCCCFEDyAB+wTxhuzkpiG7rKwDeyWEEEIIIU40CdgnkC9kmxqF7P37JWQLIYQQQnRjErBPMF+5iMnk2+bcvx9XeXnHdUoIIYQQQpwwErDbgaLT1Y9kNwrZ+/ZJyBZCCCGE6IYkYLcTX8g2+odsd0VFB/ZKCCGEEEK0NQnY7UjR6TCkJKMxGn3bHHubhuyZM2eiKApPPvmk3/bMzEwURaG4DVaIbMtzCSGEEEKIgyRgtzNvyE5B8YVstdmQbTKZmDNnDkVFRe3fyaPkdDo7ugtCCCGEEJ2GBOwOoOh0GJOT/UP2vn24Kyt9bSZMmEBycjJPPPFEi+fJysrivPPOw2q1EhUVxRVXXEFBQYFv/8aNG5k0aRJBQUFYrVZOPfVUlixZwu7du5kwYQIAkZGRKIrCzJkzvT1RVebMmUPv3r0JCAhg0KBBvP32275z7t69G0VRePfdd5k4cSIBAQH8+9//xuPx8MQTT5CYmIjRaGTQoEF8+umnvuNOOeUU7rzzTr/+V1ZWEhAQwKJFi471UgohhBBCdDoSsDuIotdjSE5GMRi8G1QVx969vpCt0Wh49tlnee2119i5c2eT4/Pz8xk3bhwDBw5kxYoVfP/991RXV3PBBRfg8XgAmD59OrGxsaxYsYK1a9dy//33YzKZSExM5KOPPgJg8+bN5Ofn89JLLwHw4IMPMm/ePP71r3+RlZXF/fffz0033cSXX37p9/z3338/N998M1lZWVx00UW89NJL/O1vf+O5555j48aNXHzxxfzhD39g3bp1AFx11VW89957vr4BfPTRRwQEBHDeeee16bUVQgghhOhIuo7uQHsbNH9Qhz33xhkb/R5r9HoMKSk4cnJQHQ5fyFbrSy7OPfdcxowZw+zZs3nvvff8jn311VcZMmQIzz33nG/bggULCAsLY9WqVYwaNYrc3Fzuuusu+vfvD0B0dDRWqxWAsLAwAKKiooiIiACgpqaGF198kW+//ZbTTjsNgJSUFFasWMG//vUvvyB86623cskll/geP//889x1111Mnz4dgMcff5yff/6Z559/nrfffpvLL7+cv/zlLyxZsoRJkyYB8L///Y9p06ZhaPglQwghhBCiG5AR7A7WELIbj2R7ampQXS4A5syZw4cffsiqVav8jlu9ejU///wzFovF9ycxMRHAN+L917/+lRtuuIGJEyfy1FNPsW3btsP2JSsrC5vNxtlnn+133ldffbXJKPrIkSN9X1dWVpKXl8eYMWP82owdO5asrCwAwsPDmTx5Mv/73/8A7wj8kiVLuOqqq47mcgkhhBBCdHoSsDsBTUO5iL5RyK6rw11VxUknncTUqVO59957/Y7xeDycd955rFu3zu/P9u3bmTJlCgCPPvqor4Tj119/5ZRTTuGNN95osR8N5Ruff/653zk3b97Mt99+69c2MDCwyfGKohx221VXXcVHH32EzWbj3XffJTExkbFjx7buIgkhhBBCdBE9rkTk0DKNzkJjMGBIScaRk+Pb5tizB0NSEk8//TQZGRksXrzYt2/48OF88MEHJCUlodfrWzxvWloaaWlp3Hbbbdxwww3MnTuX6667zleW4Xa7fW0zMjIwGo3k5uYyceLEVvc9KCiIuLg4li5d6nfc0qVLycjI8D2+8MILmTVrFl988QX/+9//uPLKK5sN5UIIIYQQXVmPC9idmTdkp4Cm/oMFVcWRm0tKUhKzZs3y3YgI8Oc//5nXX3+dyy67jHvvvZfIyEh27drFBx98wAsvvIBOp+Ouu+5i2rRpJCcnU1hYyG+//cYpp5wCQFJSEoqi8OWXX3L++ecTEBCA1Wrlrrvu4q677kJVVcaNG0d1dTW///47Go2GWbNmtdj3u+++m4cffpi0tDRGjBjB22+/zS+//MLq1at9bUwmE3/4wx948sknWb9+vd/sJEIIIYQQ3YWUiHQyGoMBjdV6cGRXVXHk7mH2nXei0x38fSguLo5ly5ah0Wg4++yzGTBgAH/+858xGo0YjUa0Wi1lZWXMmDGDfv36cfHFFzNq1ChefPFFAOLj43nssceYPXs20dHR3HLLLQA88cQTPProozz//PMMGDCAM888k48++oiUlJTD9vu2227j7rvv5p577mHgwIEsWrSIjz76iKFDh/q1u/rqq1m/fj3Dhw8nPT297S6cEEIIIUQnISPYndD8BQvw2O04cnajupygegipq6O8oABto9rntLQ0Fi5c2OJ53nnnHb/HVVVVvllEAB566CEeeughvzaKonDrrbdy6623NnvO5ORkVFVtsl2j0TR7vkNNnDix2eOFEEIIIboLGcHupDRGI4aUZJSGUWuPB0duLu6amo7tmBBCCCGEOCwJ2J2YN2Sn+IVsZ24u7traju2YEEIIIYRoUbsGbEVRzlYUZauiKDsURbmvhTbjFUVZpyjKZkVRfmrP/nVGh4Zs1ePBuXs3HgnZQgghhBCdUrsFbEVRtMC/gHOADOAKRVEyDmkTAvwfcIGqqgOAae3Vv85MYzR658nWHgzZDgnZQgghhBCdUnuOYI8CdqiquktVVQfwHnDhIW2mAx+rqroHQFXVA+3Yv05NYzJ5a7K1WqA+ZOfm4qmr6+CeCSGEEEKIxpT2mtFBUZRLgLNVVb2h/vHVwMmqqt7SqM0/AD0wALACL6mquqCZc80CZgFERkaO+OCDD5p9zuDgYPr06dPGr6SDORzoCguhftVFNBpcUdFgNBzxULfbjbY+oPckO3bsoKKiotXtq6ursVgsJ7BH3YNcp9aTa9U6cp1aT65V68h1ah25Tq03YcKE1aqqjjxSu/acpq+5JfsOTfc6YAQwCQgAflMU5XdVVbf5HaSq/wH+A9CvXz91/PjxzT5hdna237R03YXHbMaxezeq2w0eD/qiAxiSk9EEBBz2uEOn6espTCYTw4YNa3X7zMxMWvqeEgfJdWo9uVatI9ep9eRatY5cp9aR69T22rNEZB+Q2OhxApDXTJvFqqrWqKpaDPwMDGmn/nUZmoCA+prs+nIRt9tbk22zdXDPhBBCCCFEewbslUCaoigpiqIYgMuBzw5p8ylwmqIoOkVRzMDJQHY79rHL8IVsTaOQnZMjIVsIIYQQooO1W8BWVdUF3AJ8gzc0f6Cq6mZFUf6oKMof69tkA4uBDcAKYK6qqpvaq49djTdkJ6FoNCz69lsCMjJw5HT+kewpU6Ywc+bMju6GEEIIIcQJ0a7zYKuq+pWqqn1VVe2tqupT9dteU1X1tUZt/qaqaoaqqgNVVf1He/avs1m7di1arZYxY8a02EZjNmNITgbFW+Kuul3echG7vVXPsWvXLm644QaSkpIwGo3ExcUxYcIE5s+fj8PhaIuXIYQQQgjRo8hKjp3Y66+/zs0338ymTZvIzm65UkZjNqOLjPI9Vl0ub7nIEUL2qlWrGDZsGJs2beKVV15h48aNfPXVV8yaNYv58+ezcuXKFo91Op1H/4KEEEIIIXoACdidVF1dHe+88w433ngjl1xyCfPmzfPbv2DBApKSkjCbzUyZMoWiinLvDo33n3RnTg4XnnsuMTExBAYGMnz4cL7++mvf8aqqMmPGDNLS0vj111+54IIL6Nu3L0OHDuWKK67gxx9/5NRTTwVg9+7dKIrCu+++y8SJEwkICODf//43JSUlXHHFFSQkJBAQEMCAAQN48803/fpZW1vLzJkzsVgsREdH8/TTT5+4iyaEEEII0QlIwO6kFi5cSFJSEoMHD+bqq69mwYIFvlHj5cuXM3PmTGbNmsW6des4//zzefjhhwEwJCWBRkN1bS1njhnDF//5D2tXrmTq1KlcddVVbNmyBYB169aRlZXFXXfdhUbT/LeBovjPrHj//fdz8803k5WVxUUXXYTNZmP48OF88cUXbN68mdtvv52bbrqJH374wXfMXXfdxXfffcdHH33EDz/8wNq1a/n5559PxCUTQgghhOgU2nMe7E4hu396hz13+pbWT4gyd+5crr76agBOP/10zGYzn332GVOnTuWll15i0qRJzJ49G4C+ffuycuVK5s2bhzYwEEOvJAarMLhfPwAUrZb7776bTz75hIULF/Lggw+ybZt3avF+9W0AKioqiI+P9z1+4IEHeOCBB3yPb731Vi655BK/ft59992+r2fNmsWPP/7Iu+++y6RJk6iurmbevHm88cYbTJ48GYA333yThISEVl8HIYQQQoiuRkawO6EdO3awbNkypk+fDnhHkq+88krmzp0LeBfQOeWUU/yOafxYawnEGRHO7BdfZPiFFxJ70kkEhYaydu1a9uzZ0+LzWq1W1q1bx7p164iLi2tyk+PIkf4LF7ndbp566ikGDx5MeHg4FouFjz/+2PccO3fuxOFw+PXNYrEwaNCgY7gqQgghhBBdQ48bwe4K5s6di9vtplevXr5tDUva7927l9Ysb3/vo4+y+McfefqOO+idmIg5IIAbHngAW3kFHoeDvn37ArBlyxbfKocajca3tLzB0HTp9cDAQL/Hzz//PC+88AIvvfQSgwYNwmKx8MADD3DgwAG/PgshhBBC9CQ9LmAfTZlGR3C5XMyfP59nnnmGKVOm+O27+uqrefPNN8nIyOD333/323fo46VLl3LNjBlcev31OPbswWazkbN3L2lJSdi3bSMjLIz0fv2YM2cOl156Kdr6VSGPxtKlSzn//PN9pSyqqrJt2zZCQkIA6NOnD3q9nt9//53U1FQAampq2LRpE7179z7q5xNCCCGE6Ap6XMDu7L788kuKi4u58cYbCQ8P99t3+eWX8+qrr/LOO+8wduxYnnnmGS655BIyMzNZtGiRX9u+ffuyaNEiLrzwQjROJ48/8gi2RiUfnqoqXnvkEabceCOjR41i9kMPkZGRgdvtZtmyZezbt++Iobtv3768//77LF26lIiICF555RVycnJ8I+IWi4Xrr7+ee++9l8jISOLi4nj88cdxu91tdLWEEEIIITofqcHuZObNm8eECROahGuAadOmkZub67t58NVXX2Xw4MF8/PHHPProo35tX3zxRaKiojjttNOYMnUqp0yaxCmjR4Ne72tz0qBB/PrBB2QkJXHrzTczcOBARo8ezfz583nqqae45557DtvXBx98kFGjRnHOOecwbtw4AgMDufLKK/3aPP/880yYMIGLL76YCRMmMHDgQMaNG3fsF0gIIYQQopOTEexO5rPPPmtxX2pqql9d87XXXuu3/5ZbbvF9nZSUxPfff++3/49//CNWqxWPzYarpAR3eTm9e/Xitccf97VRNBq0oaFow8PR1NdhJycnN1tPHRoayscff3zY1xMYGMiCBQtYsGDBYdsJIYQQQnQXErB7II3JhCE+HjUqCldpGe7SEtT6sg3V48FVUoKrpARtUDC6iHA0ZnMH91gIIYQQouuQgN2DKXo9+ugodJERuMvLcZWUoDZaXt1dWYG7sgJNgNkbtIOCmiw+I4QQQggh/EnAFigaDbqwMLShoXiqq3GVlOCprvbt99TV4thbi6LXowsPRxsainIMs44IIYQQQvQEErCFj6IoaK1WtA112sXFuCsqoL7+WnU6cRYU4DpwAG1YGNqwMF+dthBCCCGE8JKALZqlMZkwJCTgiY7GXVqKu7TUv067uBhXcQna4CB04VKnLYQQQgjRQAK2OCyNXo8mOhpdZKS3Tru4BNXRUKet4q6owF1RgcZsRhcRgcZqlTptIYQQQvRoErBFqzSp0y4uxlNT49vvqa3FsWcPisHgrdMOCZE6bSGEEEL0SBKwxVHxq9Ouq/POp924TtvhwJmf763TbphPu9HiNkIIIYQQ3Z0EbHHMNAEBB+u0S0pxlzWq03a7vXXajefTDgjo4B4LIYQQQpx4ErDFcdPo9Whiov3n03Y4vDtVFXdFOe6KcjTmQG/QljptIYQQQnRjmo7ugGhq5syZKIqCoijodDp69erFn/70J8rKynxtkpOTURSFt99+u8nxo0aNQlEUnn/+ed+2nJwcbrjhBhISEjAajcTFxXHeeeexdu3aJudUFAWz2czAgQP597//3ep+K1otuvBwjGlpGHr1ajKziKe2BseePdi3b/eG8PrRbiGEEEKI7kQCdid1xhlnkJ+fz+7du5k7dy6ff/45N998s1+bxMRE5s2b57dt06ZNbN68mfDwcN82p9PJmWeeSXFxMR988AHbtm1j4cKFjBo1itLSUr/jH374YfLz89mwYQMXXXQRf/zjH3n//feb7aOjYZT6EIqioA0KwpiaijG1N9rgYODgiHVDnbZ92zachYV4nM6juTRCCCGEEJ2alIh0UkajkZiYGAASEhK47LLLeOutt/zaTJ8+nX/84x/s2rWL1NRUAObNm8cll1zCTz/95Gu3efNmdu7cycKFCxk6dCgASUlJnHrqqU2e12q1+p73ySef5IMPPuCTTz7hsssuY/z48aSnpxMYGMj8+fNJTk5m5cqV/Pzzz9x9992sX7+e4OBgpk+fznPPPYfBYEBjDuCsc6+mX1oaBlXl7Q8/BFVl5tSpPPmXv6AWFeEqLkYbHOydT1vqtIUQQgjRxckIdhewa9cuFi9ejP6Q2TgiIiI4//zzefPNNwHviPLbb7/N9ddf79cuMjISjUbDZ599hsvlOqrnNplMOBuNML/99tuoqsovv/zCggUL2L9/P+eccw7Dhg1j7dq1zJs3j3fffZf777/f7zzvvPceBATw66+/8n9//ztvLFzIP//7X+9OVcVdXo59507sOTm4q6pQ62clEUIIIYToanrcCPa//vhjhz33n1+b2Oq2ixcvxmKx4Ha7sdlsALz44otN2l133XXcdNNNPPbYY3z22WeEhIQwbtw4vzbx8fG8/PLL3HPPPcyZM4cRI0Ywbtw4Lr/8cgYMGNDs87tcLt5++202btzIn/70J9/2lJQUXnjhBd/j2bNnExsby//93/+h0WhIT0/n2Wef5aabbuKJJ57AXF+HHRsby8svv4yiKKQPGMCOvDxeee017vjTn/DU1vrO56mpwVFTg2I0HpxPWyO/BwohhBCi65Dk0kmNGzeOdevWsWLFCm699VbOPfdcbrvttibtJk+ejKqqfPfdd8ybN4/rrruu2fP9+c9/Zvv27bzzzjuMHTuWTz/9lKFDh/LfhlHkerNnz8ZisRAQEMCf//xn7r77bm666Sbf/hEjRvi1z87O5pRTTkHTKASPHTsWh8PBjh07fNtGjx7tN3PIqaeeyv68POwRERhTU5vWadvtOPPysG/dirOwEFXqtIUQQgjRRUjA7qTMZjN9+vRh0KBBvPzyy9TW1vLEE080aafRaJgxYwZPP/00P/74IzNmzGjxnFarlQsuuICnnnqK9evXM2HCBB566CG/Nn/9619Zt24dubm5VFdXM2fOHL/wHBgY6NdeVdUWp9xr7VR8GrMZQ2Iixr5p6MLD/UasVbcbV1ERtm3bcOzbj6d+NF8IIYQQorPqcSUiR1Om0Zk88sgjnHPOOcyaNYu4uDi/fddddx1PP/005557bpN9LVEUhf79+7NmzRq/7eHh4fTp06fV/crIyOCDDz7A4/H4gvjSpUsxGAz07t3b12758uV+Yfz3338nLi6OoKAgXxuNwYAmNhZdVBTusjLvVH4NI9eqiru8DHd5GRqLxXtDpMUi82kLIYQQotOREewuYvz48QwYMIAnn3yyyb7U1FSKi4v58MMPmz123bp1XHjhhXzyySdkZWWxY8cO5s2bxxtvvMHFF198XP26+eabycvL4+abbyY7O5svv/yS++67j1tuucVXfw2Ql5fHHXfcwdatW1m4cCF/+9vf+Mtf/tLsORWtFl1EBMa+fTEkJjaZWcRTXY0jNxf7jh24SktRPZ7jeg1CCCGEEG2px41gd2V//etfufbaa7n33nub7AsLC2vxuISEBFJTU3nuuefYs2cPHo+HXr16cdddd3HfffcdV5/i4+P5+uuvufvuuxk6dCghISFMnz6dp59+2q/dlVdeidvt5uSTT0ZRFK6//voWA3YDRVHQBgejDQ7GXVuLu7gYd2Wlb39Dnbar8ADa8DB0YWEoOvmWFkIIIUTHkjTSCR0633WD6dOnM336dAB279592HM03h8REcHf//53qqqqsFqtrTqmOZmZmc1uHzduHMuXLz/ssTqdjn/+85/885//PGy7lmjNZrS9euFxOHCXlOAuK/ONXKtuF64DB3AVFaENCfGWj5hMx/Q8QgghhBDHSwK26FL86rRLy3CVHlKnXVaGu6y+TjsiAs0hN2UKIYQQQpxoErBFl6RotegiI9CGh+GurMRdUoKnrs6331NdjaO6Go3RhKe2Fo/DgcZg6MAeCyGEEKKnkIAtTriWSkvagqLRoAsJQRscjKe21ls+0qhO22O34S4vZ8fESYRdOZ2Qyy9HFxp6wvojhBBCCCGziIhuQVEUtIGBGHr1wpiWhi4sHBrNp+0uLqbopZfZMWEi+Y8+in1XTgf2VgghhBDdmQRs0e1ojEb0cbGY+vZFFx3tF7RVm43y995n17nnsvePf6Lmd+/83EIIIYQQbUUCtui2FJ0OfWQk+uho4v42B2NGut/+6sxM9sycSc4fplLx6aeoDkcH9VQIIYQQ3YkEbNH9KQrB559Pykcf0WvBfCwTJvjttmdnk3fvfeyYdAbF//4PSk1NB3VUCCGEEN2B3OQoegxFUQgcNYrAUaOw5+RQumABFYs+QbXZAHAVFVH0978TqdGQ8+ZbmEeMIGDkCMwjRqA7zEI+QgghhBCNScAWPZIxJYXYRx4h8rbbKH//A0r/9zbuomIAFI8H26ZN2DZtgvnzATCkpmIeMQLzyBEEjBiJPj4ORVE68iUIIYQQopOSgN1NLFy4kGnTpskNe0dJFxpKxB9vIuy6a6n86ivK/vcOdZs2oRxyHR27duHYtYvyDz/0HhcT4x3hHjEc84iRGNP6oGik4koIIYQQErA7tbVr1zJy5EhGjx7NsmXLOro73ZrGYCDkoosIuegifvrqK4abzdStXk3tqtXUbdoEDatF1nMVFFD55ZdUfvml9/jgYMzDhtWPcI8gYMAAFFnYRgghhOiRJGB3Yq+//jo333wzCxYsIDs7m/T09CMfJI6bajZjHT8e6/jxAHhsNuo2bDgYuNeuxVNb63eMp6KC6sxMqusX1VFMJgIGD/YFbvPQobJsuxBCCNFDyGfanVRdXR3vvPMON954I5dccgnz5s3z279gwQKSkpIwm81MmTKFwsJCv/07d+7kwgsvJCYmhsDAQIYPH87XX3/t1yY5OZnHH3+cmTNnYrVaSUxM5P3336e8vJzLL78ci8VCWloa33777Ql/vZ2ZxmQicNQoIv70J3rNm0vfFctJ/mgh0Q/cj/Wss9CGhzc5RrXZqF2xguL/e5W919/A1lEnk3PJNAqfeZbK777DVVraAa9ECCGEEO1BAnYntXDhQpKSkhg8eDBXX301CxYswFlfprB8+XJmzpzJrFmzWLduHeeffz4PP/yw3/HV1dWcc845fPfdd6xfv56pU6dy1VVXsWXLFr92//jHPxg1ahRr1qzh0ksvZcaMGUyfPp1zzz2XdevWMW7cOK666ips9TNtCO/82gEDBhB2zTUkvPwSaUt/IfXrr4h98gmCL7oIfWJi04PcbmybNlE6fz77b72N7aeOYee555H/0MNUfPopjn37pX5eCCGE6CZ6XInIC5dN6bDnvvP9L1rddu7cuVx99dUAnH766ZjNZj777DOmTp3KSy+9xKRJk5g9ezYAffv2ZeXKlX6j3EOGDGHIkCG+x7Nnz+aTTz5h4cKFPPjgg77tkydP5uabbwbgscce48UXX6RPnz5cc801ADz00EO88cYbbNq0iZEjRx77i+/GFEXBmJKCMSWFkEsuAcBZeIC61auoXbWa2tWrsW/bBq25cXL48PqpAeXGSSGEEKKr6nEBuyvYsWMHy5Yt49133wW8Ae7KK69k7ty5TJ06lezsbM4//3y/Y0455RS/gF1TU8Njjz3GF198QX5+Pk6nE5vNxrBhw/yOGzx4sO9ri8WC2Wxm0KBBvm3R0dEAHDhwoM1fZ3emj45Cf+65BJ17LgDuykpq16w58o2TX31F5VdfAXLjpBBCCNFVScDuhObOnYvb7aZXr16+bQ3lA3v37m1VKcFdd93F4sWLef7550lLS8NsNnPllVfiOGQ5cL1e7/dYURS/bQ1zPXs8nmN+PQK0QUFy46QQQgjRQ/S4gH00ZRodweVyMX/+fJ555hmmTPEvZ7n66qt58803ycjI4Pfff/fbd+jjpUuXcs011zB16lQAbDYbOTk5MhNJJ9Fw42TgqFEAqC4Xtq1bfYG7dvVq3CUlfsc03DhZu2KFd4NWiyk9XVacFEIIITqZHhewO7svv/yS4uJibrzxRsIPmZ3i8ssv59VXX+Wdd95h7NixPPPMM1xyySVkZmayaNEiv7Z9+/Zl0aJFXHjhhej1eh577DHsdnt7vhRxFBpunGy4eVJVVRy7d/sFbufevf4H1d846bfiZErKwRHukSPRx8fLipNCCCFEO5OA3cnMmzePCRMmNAnXANOmTeO+++6jurqaefPm8cgjj/D4448zfvx4Hn30UW699VZf2xdffJHrr7+e0047jdDQUO644w6qq6vb86WI43DMN07m5ODIyaH8w4UA6KKjG41wy42TQgghRHuQgN3JfPbZZy3uS01N9au/vvbaa/3233LLLb6vk5KS+P777/3233TTTVitVt/j3bt3N3mOQ0O4yWSS6eM6iWO6cbKwUG6cFEIIIdqZBGwhuii5cVIIIYTonCRgC9FNyI2TQgghROcgAVuIbkpunBRC9FQet4dtKwspz1Fxj/Gg1cu9J6J9ScAWoodo8xsng4KoCwvD2LcvGpOp3V+PEEI0x+P28N2bWexY5V0g7cN9K5k0I4PIXtYjHClE25GALUQPdjw3TgYBu997D7RajKkpmDIyMKanY0rPwJTeH21QUAe8IiFET3ZouAYo2V/DwmdXMeLcZEack4RWK6PZ4sSTgC2E8DmWGydxu7Fv34F9+w749OAsOPrEREzp6Zgy0jFlZGBKT0cXGdmOr0YI0ZN43B6+PyRcowAqeDwqK7/IYfeGYibNSCc83tJh/RQ9gwRsIUSLDnfjZO633xJUXIIjN7dJWQmAc+9enHv3UvXtt75t2siI+tCd4R3pzkhHn5AgNd1CiOPSEK63NwrXg06Ppy4wj+otQRTsqgCgaE8VHzyzklFTUhh2Zi80MpotThAJ2EKIVmt84+SGXr0YNn48npoabFu3YsvKxpaVhS07G/uOHU1KSwDcRcXUFP1Czc+/+LZprFZv6K4f7Tamp2NMTUXRyY8nIcSRedwevn8ru0m4Pu3yvvz0Uz5n3jWc9d/vZflnu3C7PHhcKr9/souc9d7R7NAYmZpUtD15BxNCHBdNYCDm4cMxDx/u2+ZxOLBv3449O9sbvLOzsW3ZglpX1+R4T1WV/1SBgGI0YuzX7+Bod0a692ZKo7FdXpMQomvwheuVhb5tA+vDdcMnYxqNwrCzepE0MJwf5mdxILcKgMKcSt5/aiWjL0xlyMREFI18kibajgTsbmLhwoVMmzZNVl0UnYLGYPCNdDdQ3W4cubn1gTsLW1YW9qxs3BUVTY5X7XZsGzZg27Dh4EatFmNq6sHAXT/qrbXKzABC9EQthetxjcJ1Y2FxgUy9ZwRrvtnDyi9z8LhV3E4PyxbuYNe6IibNSCc40tyeL0F0YxKwO7G1a9cycuRIRo8ezbJlyzq6O0IcF6U+IBtTUwmech4Aqqriys/3lpY0jHRnZ+MqKGh6ArfbOyq+fTsVn37q26zv1cuvxMSUkYEuIqK9XpYQogM0G67HtRyuG2i0Gkaem0zy4HC+fyubkn3VAOTvqOC9J1Zw6h/6MHBcvIxmi+PWrgFbUZSzgZcALTBXVdVnD9k/HvgUyKnf9LGqqo+3Zx87k9dff52bb76ZBQsWkJ2dTXp6ekd3SYg2pSgK+rg49HFxWM84w7fdVVrqG+m2Z2dj25zlvZmyGc49e3Du2UPVN9/4tukiIzFmpDcqMcmQBXKE6CY8HrX5cH3F4cN1YxEJVqbdN5JVX+1m9eJcVI+Ky+Hh5/e2sWtdEROvSccaJvP7i2PXbrfPKoqiBf4FnANkAFcoipLRTNNfVFUdWv+nx4bruro63nnnHW688UYuueQS5s2b57d/wYIFJCUlYTabmTJlCoWFhX77d+7cyYUXXkhMTAyBgYEMHz6cr7/+2q9NcnIyjz/+ODNnzsRqtZKYmMj7779PeXk5l19+ORaLhbS0NL5tNAuE2+3m+uuvJyUlhYCAANLS0pgzZw4ejwcAm83GwIEDue6663zH5OXlERERwfPPP9/Wl0l0U7qwMCxjxxBx443Ev/givb9ZTN9Vq0j639tEz55N8MUXY+zfH1q4EdJVVETNTz9T8tq/2X/b7ew840y2nTya3BkzKXz2OSo++wz79u2oLlc7vzIhxPHweFTvbCFHOXLdHK1Ow8kXpHLJvSMIjT14o+O+LWW8+/hyspblSdmlOGbtOYI9CtihquouAEVR3gMuBLLasQ9dxsKFC0lKSmLw4MFcffXVXHrppTzzzDPo9XqWL1/OzJkzeeKJJ5g2bRpLlizhgQce8Du+urqac845hyeffJKAgADef/99rrrqKgYPHkz//v197f7xj3/w5JNPMnv2bF577TVmzJjBxIkTufzyy3nyySd55plnuOqqq9izZw8mkwmPx0N8fDwffPABkZGRrFixglmzZhEeHs7111+PyWTinXfeYdSoUZxzzjlccsklXHPNNQwZMoQ777yzvS+j6Ea0lkDMI0ZgHjHCt63hZkpbVtbBGyq3bm3+ZsrKSmqXL6d2+XLfNsVkwtivb32JiXek29g3TW6mFKIT8nhUfnjLP1wPaAjXx1HSEZUUxKUPjGTF5zms+24PqgpOm5sl/93CrrVFTLiqP4Eh8jNBHJ32DNjxwN5Gj/cBJzfT7hRFUdYDecBdqqpubstO7LvvlyM3OkESnj2t1W3nzp3L1VdfDcDpp5+O2Wzms88+Y+rUqbz00ktMmjSJ2bNnA9C3b19WrlzpN8o9ZMgQhgwZ4ns8e/ZsPvnkExYuXMiDDz7o2z558mRuvvlmAB577DFefPFF+vTpwzXXXAPAQw89xBtvvMGmTZsYOXIker2exx8/+MFCcnIya9as4d133+X6668HYPDgwTz77LPMmjWL3377jbVr17Jhwwb5eF60uRZvpty9+2BNd/3UgZ7mbqa02bCt34BtfaObKXU6782U6emYBngXyDGmp6O1yMIUQnSUhnC9bYV/uD79OMN1A51ey6l/6EPq0Ei+fyuLigPeX9JzN5Xw7uPLOe2yvvQdFS3vY6LV2jNgN/ddeehnL2uAJFVVqxVFORf4BEhrciJFmQXMAoiMjCQzM7PZJwwODqaqquo4uty2WtuXnTt3smzZMl5//XXfMdOmTeO1117jrLPOYvPmzZx99tl+5xs2bJjfc9TU1PDss8+yePFiCgsLcTqdvvKNhjaqqtKvXz+/85jNZtLS0nzbzGbvHdW7d++mX79+AMybN48FCxawZ88ebDYbTqeTxMREv/Ncd911LFq0iL///e/Mnz+foKCgDvu3sNlsLX6PNKe6uvqo2vdUnf46WS0w6iTvH1VFU1qKfs9edHv3ot/r/VtbXt70OJcL+7Zt2Ldt87uZ0hUZiSsxEWevRFyJ3j+eVi4H3+mvVSch16n1etK1Uj0q+5erVDS6DSO0N6ixefz0c/5hjz2W6xQ3TkW7AUq3eR/ba118/2YWy7/NIu4kBZ2p+4XsnvT91F7aM2DvAxIbPU7AO0rto6pqZaOvv1IU5f8URYlQVbX4kHb/Af4DkBFvVeODNaQNG9fkCbOzs7EeMoVX0zGs9nNoX1ry3nvv4Xa7ycg4WKLeUAdWXl6OoigYjUa/85lMJr/nuOeee1i8eDHPP/88aWlpmM1mrrzySlRV9bVRFAWLxeJ3HkVRsFqtvm16vd53fqvVyvvvv899993H888/z6mnnkpQUBD/+te/WLRokd95ioqK2LZtG1qtlv3797f6tZ8IJpPJ9wtIa2RmZjK+fqlw0bLucJ1cJSWNZi/xTh3ozN3TbFtdURG6oiJMa9Yc3BYV5R3hzji4OqU+Pq7JKFd3uFbtQa5T6/WUa+XxqPwwP4uK3EYj16fFcfoV/Vo1cn3M1+kM2L+tjB8XZFNZbAOgaj/klus4fXo/+oyIOvpzdmI95fupPbVnwF4JpCmKkgLsBy4HpjduoChKDFCoqqqqKMoovDdhlhzupGa1lrRPz2fN9+MIP/8xkvoPP1zzoyrT6Agul4v58+fzzDPPMGXKFL99V199NW+++SYZGRn8/vvvfvsOfbx06VKuueYapk6dCnhHcXNyco57JpKlS5dy8sknc8stt/i27dy5s0m7G264gd69e/PPf/6TK664grPOOosRjWpnhegMdOHhWE4bi+W0sb5t7upq7Fu2+K9MuXMnNHNDpOvAAaoPHKD6p5982zTBwZj69/fN121KTwe3u11ejxDdicej8uP8bLYtPxiuM44iXB+v+L6hXPbgKH79eCebf94PgK3GyTevb2Ln2ijGXd6XAIvhhPdDdE3tFrBVVXUpinIL8A3eafreUFV1s6Iof6zf/xpwCfAnRVFcQB1wudrKW3iH1/yM+92JrAyZTPzFjxOX3O8EvZIT68svv6S4uJgbb7yR8PBwv32XX345r776Ku+88w5jx47lmWee4ZJLLiEzM5NFixb5te3bty+LFi3iwgsvRK/X89hjj2G324+7f3379uWtt97i66+/pk+fPrz33nv89NNPhIaG+tq89tprZGZmsm7dOlJSUpg5cybTp09n7dq1vpITITorrcWCeeRIzCNH+rZ57Hbs23dgy9rsDdwNN1PabE2O91RUNLmZMkpR2B4ZiS42Bn1MLPqYGHQxMehjY7xfx8aii4hA0Wrb5TUK0dk1hOutyw/OiZ9xWhzj2ylcNzCYdIyf3o/eQyP58b/ZVJd530d3rDrA/m3ljJ/ej9Shke3WH9F1tOs82KqqfgV8dci21xp9/U/gn0dzzmrl4I1HWkXlpIrFON78juWRFxE4/i/H2eP2N2/ePCZMmNAkXIO3Dvu+++6jurqaefPm8cgjj/D4448zfvx4Hn30UW699VZf2xdffJHrr7+e0047jdDQUO644w6qq6uPu3833XQT69atY/r06aiqytSpU7nzzjt54403ANi6dSt33nknr776KikpKYB3ppLhw4fzl7/8hX//+9/H3Qch2pvGaCRg4AACBh5yM2VOTv2NlI1upqysbHK8oqq4DhzAdeCA/w2Vjel06KIiDwbwhjAeG4Ou/m9tWJjcZCW6PY9H5ccFh4Trse0frhtLzAjj8odPZtmH28n+1Vv3XVfp4OvXNtLv5BjGXpqGKVDfIX0TnZPS1ed47Nevn/rlB/Oo/fpRBttW+u3bPPlDkvr0xxQai04n3/hVVVUdWgvdUY52kR6pRWsduU5NqaqKc3+e31Lwti1bcB44gNIGP2sVg8E78h0dfUgAj0Ef6w3mmuDgLhvC5Xuq9brrtfJ4VJYsyGbL74eE6+nHFq5PxHXavbGYJW9vobbC4dsWGGxgwjXpJA1oOjjWFXTX76cTQVGU1aqqjjxSu26xVHqfIWNhyPdk/fY1yo+Pk+70Tq2tQcXiLMF9oIxqQzgBoTFotd3iJQshOiFFUTAkxGNIiCfozDN92zO//55T+/fHlZ+Ps6AAZ34BroIC79cF+bjyC3CXlR3x/KrD4Vu5ssU+BASgry8/0TU7Gh4jUw6KTqnZcD0m9pjD9YmSPCiCKx4+mV/e3+abNrCmwsEXr6wnY0wsYy5JwxAgWaOn61bfARmnnIN68mTWZy7Esuxp33YtHiyOIpyFpdQZIzGHRqHRSK2jEKKd6HQYEhIwJCS02MRjsx0M3fkFuAryceYfDODOggI8rZjqUq2rw5GTgyMnp8U2GovlsAFcHxuLxiTLRIv24/GoLPlvM+H6yv6dKlw3MAXqOfO6AfQeFkXmO1uoq3ICkLUsnz3ZpUy6Jp2E/mEd3EvRkbpVwAZQNBqGTLwUz+lT2bh+NQ70GPB+4+txo7cX4CgoxhkQhTkkqst+lCqE6F40JhOG5GQMyckttnFX1+AqbDmAOwsKUGtrj/hcnupq7Nt3YN++o8U22pAQdPVlJ43rwHXR0d5ylOhoFIPMoCCOny9c/9Y1wnVjqcMiie0TzE/vbGXn2iIAqkvtfPqPdQw6PZ5T/tAHvVEG9HqibhewG2i0WgwBVnQx/aipKMJQdwA93mm2DLgw1OVhryvGFRiNOShcgrYQotPTWgLRWnpj7N272f2qquKprKwfBc/3jog3hPGCQl8YVx2OZo9vzF1ejru8HHt2dsv9iYhoUo7iF8YjI1F03fZtRrQB1aOy5O0tfuE6vYuE6wYBVgOTZw1kx6oD/PTeVuw13qyx8af95GaVMmlGOnF9Qjq2k6LddfuffBqNhsDQaDzBEVSXFWKyF6PDOyetEQfGmr3Yag7gscYSYAmRoC2E6LIURUEbHIw2OBhTv+anKlVVFXdZWdMAXj8K7srPx3ngQLPzfh/KXVyMu7gY26ZNzTfQaNBFRR0sQ4k+JIDHxHinJ9Rojudliy5K9aj8+PYWtvx6cDXG9DGxTOhC4bqBoiiknRRNXN8QMv+3ld0bvOvjVRbVseiFNQyZlMjoC1LRGWQ0u6fo9gG7gUajxRIeh9sdRXVZAQH2ErSKBwATdqjaTV11AATFERDYuuWPhRCiq1EUBV1YGLqwMBgwoNk2qtuNq7jEN/LdJIAXFOAqKgKP5/BP5vHgKvDe0Mm6Ftro9eijotDFxhCkaDiwerXfrCi62Fi0ITL40d00G65P7ZrhurHAYCPn/mkQW38v4JcPtuOoc4EK67/fy55NJUyckU5MSnBHd1O0gx4TsBtotTosEQm4nNFUl+VhdpahUbzTZwWodVCxk9rKQLQhcRgD5E57IUTPo2i16KOj0EdHETCk+Taq04mrqKhJOYqvJrywEHdx8ZGfzOnEuX8/zv37CQBKVq5s0kQxmeqnJoz1Bu9DpibUxcbKzChdSIvh+qquHa4bKIpC/1NiSegfyo//3cLerFIAygpq+XjOaoZNTmLUeSlo9fLJTXfW5QN2njOP+Zvnc1Gfiwg2tv63Qp1ejyUqCacjhtqyPMyuchr+X5vVGtTS7dRorehD4jCYZPVBIYRoTNHr0cfFoY+La7GNx+HAVVjYfACvHw13V1Qc8blUmw1Hbi6O3NwW28jMKF2Dr+a6Ubju343CdWOWUBPn3zqErKV5LFu4A6fdjarCmsW55G4sZtKMDCJ79by1KXqKLh+wXaqL51c9z7/W/YspqVO4ov8VpIWmtfp4vcGIPjoFh60OZ/l+zO4qFAUUBQI9VaglW6nWBWMMjUNvkB/OQgjRWhqDAUNiIobExBbbeGprfWUoGzMz6R0c3CSMe2pqjvhcrZ4ZJS625dUyo6NQ9LIo2YnSEK6zDwnXE7thuG6gKAoDTosnMT2MHxdks39bOQAl+2tY+OwqRpybzIhzktBqZTS7M6sus7FrXTE564tafUyXD9gN6lx1fLjtQz7c9iGjYkYxvf90YtXYVh9vMAVgiOmDva4ad0UeZo/3B7qigMVdgaeokmp9qHdVSH3XnZrqlltuYdOmTWRmZnZ0V4QQAo3ZjDE1BWNqCjaHg8hmVpNzV1W1OAruzM/DVVCIarcf8bl8M6NktTAziqKgi4hodnrChlIUXUQEilZuVDtaqkdlyf96VrhuLCgigAvvGMbGn/bx28c7cTk9eDwqK7/IYfeGYibNSCc8XsqcOpPS/Bp2rSsiZ10RB3KPvAbBobp8wA7ThdE3tC/byrb5tq0oWMGKghW8MvAVouqiCDGGoNO07qUaAywQ0Je6mkqozPPWZQMaRcXiKsV9oJxqQxgBobFoT9D0UzNnzmT+/Pk88cQTPPjgg77tmZmZTJgwgaKiIiIiIk7IcwshRGejtVrRWq3Qt2+z+9tsZhRVxVVUhKuoCNuGDc230enqb8psZlrC+lIUbWio3JTZiC9cL2sUrk+J6THhuoGiURg8IZFeGeH8MD+bgl3e8qiiPVV88MxKRk1JYdiZvdDIaHaHUD0qhbmV5KwrZte6IsoLj7ymwOF0+YBt0VhYeP5CVheu5p0t7/Djnh9xq95p+NweN4U1hRyoPUCwMZhwUzgmXevKPAICg1DNVmqrK9BU5XlnGgG0igeLsxjXgVKqjZGYQ6LRnIDRDJPJxJw5c7jpppuIjIxs8/MLIUR3cdQzozQeBW+YFSU/H1dxMajq4Z/M5cKZl4czL4+6lvpjNKKLifYvRYmN8w/h1p5Re6t6VDKbCdcTrk7vUeG6sZBoMxffNZz1P+xl+ae7cLs8eFwqv3+yi5z13tHs0JjAju5mj+B2e8jbVu4dqV5fTE1585+EaTQK8f1CSB0aCf9u3bm7fMAG7w/XkTEjGRkzkoKaAt7f+j4Lty307VdVlXJbOeW2csx6M2GmMIIMQUccYVAUBbM1BNUSTE1lKbqaAox4F2jQ4cFiL8RZWEytybsqpKYN53KdMGEC+/bt44knnuDll19ust/tdjNr1ix+/PFHCgoKSEhI4MYbb+Suu+7y9cPtdnPvvfcyb948AK644grUQ948Fi9ezFNPPcWmTZtQFIWTTjqJf/zjH6SnpwOwe/duUlJSePfdd3n11VdZsWIF/fv3Z/78+Wg0GmbNmsX69esZNmwY//3vf0lJSWmzayCEEG3Ff2aU5qdGUR0OnAeKWlgl0/u1u6zsiM+l2u04c/fgzN3TYhtNYGDTGzEb14PHxqAJCDjm19sZNITrrMbherQ3XGt6aLhuoNEoDDuzF0kDw/nhrSxfCUJhTiXvP7WS0RemMnhiYo+/TieC0+5mT1YJu9YVkbuxBHtt859s6QwakgaEkzI0kqSB4ZgCj+7+jG4RsBuLCYzh9uG388chf2TT5k2YdCZsLptvf62zllpnLXqNnlBTKKGm0COWjyiKQmBwOGpQGDXlRejrCjHUrwqpx43elu9dft0cjTk4ok0+GtRoNDz77LNcdNFF3H777fQ+ZOU2j8dDfHw8H3zwAZGRkaxYsYJZs2YRHh7O9ddfD8ALL7zA66+/zuuvv87gwYP5+9//zrvvvsvw4cN956mpqeGOO+5g8ODB1NXV8eSTT3L++eeTlZWFodEyyI888gh///vfSU1N5U9/+hPTp08nMjKSp556iqioKGbMmMFtt93G559/ftyvXQghOoJiMGBIiMeQEN9iG4/N5i1DKSjAmZfvPytKfTD3VFcf8bk8NTU4duzEsWNni220wcG+UpTGo+D6mBi0hQdwV1ejCQzslOUoqkcl852tfuG63+gYJlwj4bqxsNhApt4zgjXf7mHlFzl43Cpup4dlC3ewa10Rk2akExwpM5kdL1u1k5wN3psU92SV4nY2P4e/KVBP8uBwUodGkpgedlwLA3W7gN3AqDVi1ptJDU6lzlVHia2ESnslH/7zww7r06OPPnpU7c8991zGjBnD7Nmzee+99/z26fV6Hn/8cd/j5ORk1qxZw7vvvusL2P/4xz+45557uPTSSwGYM2cOS5Ys8TvP1KlT/R6/+eabBAUFsWLFCsaOHevb/te//pVzzz0XgDvvvJPzzz+fjz76iAkTJgDemydvueWWo3p9QgjR1WhMJgzJyRiSk1ts466u9i3I09LNmarN1uLxvvNUVOCuqMC+ZUuTfRHAtkce8ZajhIejjYxAFx7h/Toi3HuzZngEuohwtOHh6CIj2y2M+8L10jzftn6jY5go4bpZGq2Gkeckkzwogh/mZ1G81/sLWv6OCt57YgWn/qEPA8fF99iSmmNVVWojZ30Ru9YVkbe9AtXTfPmXJcxI6pBIUodGEtsnuM1q4LttwG6gKApmvRmz3ozT7Ozo7hy1OXPmMHr0aO66664m+1577TXmzp1Lbm4udXV1OJ1OkpKSAKioqCA/P59TTjnF116j0XDyySezd+9e37adO3fy0EMPsXz5coqKivB4PHg8Hvbs8f9oc/Dgwb6vo6OjARg0aJDftpqaGmprazGb5bdtIUTPpbVY0KalYUxrfspYVVVxl5cfPoQfOADOI79nqXa7ryb8SNojjKselcx3DwnXJ0u4bo2IBAuX3DuSVV/tZvXiXFSPisvh4ef3trFzbRETr+lPUHjXLhs6kVRVpSy/ll3rvKG6aE/LM3+ExQWSOtQbqiMSLSfkF89uH7Ab02u73vymJ510ElOnTuXee+/loYce8m1///33ueOOO3j++ec59dRTCQoK4l//+heLFi06qvOff/75xMfH8+9//5v4+Hh0Oh0ZGRk4HA6/dvpGc8M2fCM2t81zpKWThRCih1MUBV1oKLrQUEwZGc22UT0eXMXFLc6KUrV/H/rqmlaNhPvOeSxhPCKiPoAfOYyrHpWf3t1K1i+HhOsZEq5bS6vTcPIFqaQMieD7t7Ipy/dOGbx/axnvPbGCsZekkT4mtlOWBXUE1aNSuLvSd5Pi4Wb+iEkNIqV+pDok+sQPBLYqYCuKEgmgqmpR/eNBwGXAZlVV3z1x3Wt7jcs0VFWlzlVHqa2USkdlkxsAFUUhyBBEuCmcAP2Rf2t0Oh3Yy/IwO8t9y683qNFYsNfVYAw4+juDn376aTIyMli8eLFv29KlSzn55JP9yjJ27jxYyxccHExsbCy///47EydO9L3eFStWEBvrnR+8pKSE7Oxs/vWvf/lKPdasWYPrSFNZCSFEG7PXubBXqaiqKuEBUDQa9FFR6KOiCGj0CWKDzMxMTj/9dDw1tbhLinGVlOAqKsZVUoy7uMQbzktKcNf/7SouPqFhXBseTnavi9lrOvgLQ3JkLSNj9mJbU9vuZSpdXVRSEJc+MJKVX+Sw9ts9qCo4bW6WvL2FnWuLmHBVfyyhxo7uZodwuzzs31ZGTv3CLzUVjmbbaTQK8f1DSR0aScqQCAKD2/d6tXYE+wPgv8AbiqJEAD8DecCtiqLEqar6wonq4InkVz7idlJmL6PMVobL4w2YqqpSYa+gwl5BgC6AsADv7CMapfn6HL3egD4qGYfdhrM8D7OrgoafI4GeatTSbdTogtCHxGMwtn5VyD59+jBr1ixeeukl37a+ffvy1ltv8fXXX9OnTx/ee+89fvrpJ0JDQ31tbr/9dp555hn69u3LoEGD+Mc//kF+fr4vYIeGhhIREcHrr79OYmIi+/fv5+6770Z3gub3FkKI5mxfVciSt7fgtKnUbF3D6AtTiUsLPfKBPZyiKGgtgWgtgRjqywNboqpqy2G8PoAfaxj32B1sCRzH/kbhOrpgBSmZC8j/8JCBq5ZGxsPrHzeMjEdEoLGcmI/uuwqdXsspF/chZUgkP8zP9o3O7tlcwntPLOe0S9Poe3JMj7hGTrubPZu9M3/s3liCo66FmT+MWpIGhJFaP/OH0dxxlQutTVKDgd/rv74E2KGq6kmKolwI/A3okgG7Mb1WT5Q5ioiACCodlZTaSqlzHpzltM5Vx/6q/RRqCr2zjxhDWyw5MRhNGKJTsdtqcZXvJ9DjvWFBUSDQXYmn2LsqpDE0Fr2+db9RPfzww8yfP9/3+KabbmLdunVMnz4dVVWZOnUqd955J2+88YavzZ133klBQQE33HADAJdddhlXXnkl2dneVcw0Gg3vv/8+t912GwMHDqRPnz688MILTW58FEKIE8E7W8J2Nv6037ctf0cFi15YS0L/UE6+IJWY1OAO7GH3caLCuLO4hG2JF7I//jTf8dEFK8jYsgCFpjeVHXOZSn3o7olhPCY1mEtnn8TyT3axfsleUMFe6+L7t7LZubaI8Vf2xxzUdVeYbkldtYPdG4rZta6YvdmHmfnDoidlcAQpQyNJ7B96XDN/tCXl0LKIZhspSi3QX1XVPYqiLATWq6r6hKIoicA2VVU7rOq+X79+6tatW5vdl52d7ZvP+VjUOetnHzlM+UiYKYwAXcBh/2PbaqvwVORhVv1rg9yqQp0hzLv8uu7E/5ZVVVWFtYcsbtDY0X4fZGZmMr6Z5ZqFP7lOrSfXqqmKojq+eX3TYW9EAkgaFM7J56cS2avn/ew6nM7wPaV6VH56byubfz4YllMSPYxKPoBacvwj40ejpTCeU1JCv2HD0QZZ0VitaIOC0FqtaIKCvOUqbbh+RXvI217GD/OzqSw+eB1NgXrGXdGXtJHRx3zezvD9BFBZUucr/cjbXt7iuk+WMKPvJsXY3m0380drKIqyWlXVkUdq19oR7O3AHxRF+Qg4C++oNUA0UH5MPewCAvQBJOgTcHqclNvKKbWVNls+YtKZCDeFE2RsvnzEZLaCuR911RVQlUeA6v2PoVVULM4S3AfKqDaEExAag1Yr5RlCiO5t17oifpif7fcxb+qwSIgqxlAZzdbfC3xvrLkbS8jdWELvYZGcdH4K4XGWDuq1aExVVX5+b5tfuO47KppJMzNavKGxych4cXF9AD/+MN7SyHgQkP9BC9PzKoo3dFssaBoFb+/fVrTWoPpgHoTGajn4uKGNxYJyAlZyPpy4tFAue3AUv328k00/ez/5sdU4+XbuZnatLWLcFX0JsHSd0WxVVSnNq6mfTq+4w2f+aEutTXOPAe/iLQX5QVXV5fXbJwNrT0THOhO9Rk+kOZLwgHCqHFWU2kqpdR4cjba5bOyv3k9BbQGhplDCjGHNlo8EWIJRA4OorSpDU52PqX5VSC0eLI4inIWl1Jki61eF7BwfcQghRFtxuzz8tmgn6384OFWoRqtw6tQ+DJ6QwE8//cT4izMYPjmJlV/uZvuqQhqqDHauLWLnuiLSRkYzakpKu8wCIJqnqio/v7vNF/AA0k46fLiG4yxTacMw3uhJ8FRW4qmshFaUrDRHY7F4w7ilaSj3+9tiPWS7Fa3VinIM9zwZTDpOn96P1GGR/Lggm+oy7/LeO1YfYP+2MsZf2d+7pHcn5Zv5Y613Or2KorrmGyoQkxJMytAI78wfUV3r/3yr/mVVVf1YUZReQBywvtGu74GPTkTHOiONoiHYGEywMdg7+0hdKRWOCl/5iNvjpri2mJLaEqxGq3f2kUPKRxRFwRwUhmoNpaaiGH1tIQa8c516V4Us8K4KGeBdfr2z/4YmhBCtUVVq45vXN1GYU+nbZg0zMfnGgUSnBPm1DY0J5KzrBzDi7CRWfJHDrrVF3h0qbF9ZyI7VB+g3OoaTzk0mKELmBW5PLYXrM649fLg+Wm0VxvdkZRETZMVTWYW7qgpPZaXvb09ty1O6tZanuhpPdTUu8o/cuBkas7l+RNziDd/NjKJ7/25a4pLQ28LlD5/Msg+3k/2r9/nrqpx8/dpG+p0cw9hL0456ee8Txe3ysH9rmW86vdrKFmb+0Cok9AslpYNm/mhLrf7VSVXVQqCw4bGiKH3w1mKfmIKqTi5AF0C8NZ5oTzRltjL/8hFUKu2VVNorMelMhJnCCDYG+5WPKIpCYEgknqBwaiqKMNQdQF+//LoBF4a6POx1xbgCozEHhUvQFkJ0Wbs3FvP9W1nYaw6WhCQPjmDSjPTDBoDweAvn3DSIA7mVrPg8h9xNJYB3BGzLr/lsW15A+pg4Rp6ThCW09TMziWPTUBbSJFzP7Nh5rg8XxjdnZjKihdpi1eXCU12Nu6oKd2UlHr+/q7x/HxLK3VVVuKsq8VRV46k6/P0DreGprcVTW4ur4NiOVwICSLRaCYgczObws7BrvFMBb11eQO6qvYxKLSYhQX8wwDceRQ8KQmM4ceUkDpuLPZtL2bWuiNxNR5r5I5zUYREkDYzAGNA9SmVbOw/208BWVVXnK96k9y0wCahQFOXsRiUjPY5OoyPSHOk3+8ih5SN51XkU1hYSagwlzORfPqLRaAgMjcYTHEF1WSEmezE63AAYcWCs2Yut5gAeaywBlhAJ2kKILsPj9rD8sxzWfJPr26ZoFE65qDdDz0xs9c+zqKQgptwyhIJdFSz/bBf7tpTVn19l88/72fJrPgPGxTHi7ORuOZtCZ+AL1z81E67b8QaztqTodGhDQtCGhBzT8arbjaempj6MV/r/XV3l99hdVek/il5d7S1NacVEE4ftQ10drro6Qg58zyjdr2zrM43CmFEA2Nx6ft4eS+zPy0jb8TE6d9PxUMVoRBNkJVyrY3fc62hDQ9GGBKMLDa3/OqT+71C0od5rpQ0ObvHm0LoqBzkbislZV8Te7DLcrsPM/DEkgtQhkSSkh6LTd7+y2Nb+mnAl3oVlAM4BhgKj67c/C0xo8551MYqi+MpHbC4bpbZSyu3l/uUjdcUU1xUTZPTOPmLWmX1vMBqNFkt4HG53FNVlBQTYS9Aq3m9ME3ao2k1ddQAExREQGNRiP4QQojOoLrPz7bxN5O+o8G0LDDEy+YYBxPYJOaZzxqQGc+Edw9i/tYzln+/yndvt8rDhx31kLc1j8IQEhp2ZhMnSOT4a7w5UVeWXbhau24Ki1XpLNoKCgPijPl71eLwj2IeOkFd6R8gPhvLKpiUu9aPruN2+8+ldtQzYMp+o4nVs6XsFToN35p382DGUhqaTvuVtwsr9Z11T7XbcRXZ0QF1BK4fRNRq0wcG+8G0PiafQlEq+J5YSmwWV5n9xtoaZvDcpDosgpndIt1/ds7UBOxrYV//1ucAHqqquUBSlFFh1QnrWhZl0JuIscUSZoyi3l1NaV4rT4/TtbygfMeqMhJvC/cpHtFodlogEXM5oqsvyMDvLfKtCBqh1ULGT2qpAtMFxGAP876ZXVRVVBY/Lg8et4nZ7//a4VVSPihsV1SKrpAkhTqy9WaV89+Zm6qoO/tzrlRHGGddmEGA9/hHm+H6hXNx3OHuzSln+2S4O5Ho/qnc5PKz5Zg8bf9rPkEmJDJ2U2KELTXQHDeG68VzlaSOjeny4bguKRoPWYkFrsXAs36WqqqLW1jYpcYmrqqJPSQ0rt8C+Cm/ItpvCWDf0NnrVbaJv4XcolaW4KyvhGFZuVj0eKhwBFHn6U6QMptrTC1ooZw+s3k9k8XoiSzYQrKtGtzYExyeh7D/OkfKuoLUBuwRIwhuyzwLub3S8pLUW6DQ6IgIiCDcdnH2kxlnj22932b3lIzWFhJhCCDOFYdB633x0ej2WqCScjhhqy/IIcFWiosWjatF4FFzFRdQq1aC3gKocDNJH+LipzFmLNcyI3tg9apyEEJ2Hx6Oy8sscVn212zf7h6LAqAtSGTE5CaWNb4LrNSCcxIwwdm8sYflnuyjZ513Uy2lzs+rL3Wxcso+hZ/Zi8IQEDCb5mXe0VFXll/e3Nw3X12ZIuO4EFEVBCQxEExiIPibGb18wcIGqsmPVAX56b6vv/oc9AQMpHzmSSTMyiO0TjGqz4a6s4vcfvmdEWhqusjLcZeW4y8txl5XhLivDVV6Gq7ycktpACrWJHAjOoC6ghVlKVA/BlTlEFq0nong9Zluxb5cHcJSVQU5O617gISPlx1u+0t5a+xPnI+AdRVG2AWHA4vrtQ4EdJ6Bf3YqiKAQZgwgyBvnKRyrsFXhUbwmI6oGKmiqqq+oI0Jgxa81oVV2jUegIaj3hzZ/c3Xx9U0tcDjdlBbWYAvUEhhrRyg9JIUQbqK108O28zezfWubbZg4ycNb1A4jvd+KWPVcUhZTBESQPDGfn2iJWfL6LsgLvcJq91sXyT3ex/oe9DJ+cxKDT4zvNKm+dnS9cZ+7zbZNw3bUoikLaSdHE9Q0h839b2b3BG3Yri20senENQyYmMvrCVPTRUbjj4jCfdJLf8W6Xh32NZv6oa2nmD0UlOtRJfGA5Mep+9FVFuIKcuMPjcJeZcZeVHdsNoR6PL+R3xVDe2oD9VyAX6AXco6pqwzBsLPBqm/eqG1BV1Teq7HE3lGyoeFwqZncIRneQt/i/mQFnJypOnE13HMbnX33CDTfPoDC3Aq1Wg0aroNFq0GoVNDrvCHfjaXFsNU7stS4CQwwEWA1SNiKEOGb7t5bx7bzNfj9j4vuFctb1A9rtpkNFo9BnRBSpwyLZvrKQFV/kUFk/v66t2smvH+1g3fd7GHF2MgPGxqHVS0hsiaqq/PKBf7juI+G6ywoMNnLunwax9fcCfvlgu3c2DxXW/7CX3E0lTJp5cKVjh81F7qYSchpm/rC5mz2n3qglaWA4qUMjSRoYjuEIM3+oTifuigpvYC4vb3Gk3Pu4vFOH8tZq7TzYLryLzBy6/e+tfqZuQlVVPJ764Ozy+IVo9yGB+nht3LSesy4Yz4hhJ/HFR996A7NOQaNVUD0uFGclRurQ4Mai8f5mGq7Noc4YRkBILNpDJrB3Kw5w6LDXHVyNsrrMTl21E2uo6Yj/QQBiY2O55ZZbmD17tm/bE088wcMPP8zChQuZOnWqb/uVV17J3r17+fnnn4/7WgghOh/Vo7J6cS4rPt91cDIEBU46N5mR56V0yE1MGo1Cv5Nj6DMyiq2/F7DyyxyqS70LcdRWOPjl/W2s/TaXk85Lod8pMfIp3iF84XqJf7g+U8J1l6YoCv1PiSWhfyhL/ruFPVmlAJQX1vLxnNWE9IYvNq9n32Fm/giw6kkZHEHK0EgS+h/dzB+KXo8uIgJdRESrj+nUobwVWl2UpihKNPBnIAPvuGsW8C9VVQ+0WW86kKp6bwRsCMyNbxD0uD24XQe/bmsNo81oVBzYsXnqcCsu3vjwNa6YOZ1PP/yE3/dnMnTgUMJMYQTqvbOPqKrVt/y6VvH+lqlVPFgcxbgOlFJtjMQcEo2mfilXjVbBGmXGXueiutTm+0/kdnooP1CL0azDEmI67MjOhAkTWLJkiV/AzszMpFevXixZssQvYGdmZnLDDTe0+fUSQnS8umoH37+R5XujBu8b8JnXDiAxI6wDe+al1WrIGBNHv1ExZC3LY9XXu6mt8I6wV5fZWfL2FlYv3s2oKSmkjYrp9jMatIaqqiw9NFyPkHDdnVhCTUy5dQhZS/NYtnAHTrsbVYWyHVBGSZP21nCTb3nymN7B7fr/pFOH8lZo7TzYY/DWXRcCv9VvvhL4i6Iok1VV/a3FgzsB74hzo8Dsajra3JobBI+WRuMNzpr6UWdfyUb915r6r/3LMyy4PW7yy/P5fNHnvPXZW9Taavj4fx/T+7HeVDmqMGgNfPfRdzz3xHMUFRUxceJEJo4f5/fcubtz+etjt7N87Saqamrp168/9913H5deeinGAB2GuECSk5K5fNpV5Obu5ovFnxEcFMwjs5/kvCnncOc9t/PFl18QGxvLv/71L8466yzAG7Bvv/127HY7RqMRu93Ob7/9xosvvsgrr7zie/5t27aRl5fHxIkT2/SaCiE6Xv6Ocr6Zu5macrtvW2yfYCbfMJDAkM618ppWr2HQ+ATST41l08/7WfNNrm92k8piG9+/lc3qxbmcNCWFPsOj2vRGzK5EVVWWfridDYeG6+skXHc3iqIw4LR4EtPD+PG/2ezfWu63PzzeQurQCFKHRRIeb+lSJaQnKpS7y8u9wbysHLZuadV5WzuC/TzwLvBHVfXemacoigZ4DW/pyKmtfiVtTPV4f9jXVDioKbdTU1H/p9xBwmiFoj1VbR6cFY03GGsbheSGIK1tFJ6P9ZtSq9Gy5MslpCSncOYpZ+KodvDnmX/mjgfvQK/Xs2rFKm6ZdQu33n8r06ZNY91v63jkwUcAqAmIR19XSHVNHedMGMOT99xMgMnIO5/9wFVXXUXf3ikMHjYcjUaLolF4/c1XmX3fw9xxy13M/98b3PbXP/LBwnFcfOElPPTgI7zw979x1VVXsWfPHkwmExMmTKCuro7ly5czbtw4fv/9d8LDw7nmmmu4/fbbKSwsJDo6miVLlhAQEMDo0aPb8tILITqQqqqs+24vv32yE9Vz8Ofq8MlJnHxBSqcOYjqDlqFn9CJjbBwbM/ex9ts92Gu95XJlBbV8O3czq+NzGXV+CilDIrpUqDhevnD948Fw3Xu4hOvuLigigAtvH0bWsjzW/LSVgSf3IXVoJMGRAR3dtXZ11KG8lT8bWhuwhwIzG8I1gKqqHkVRXgTWtvIcJ4S7Aj5+fk2z++JOCm0SrjfuHNYe3WrWpIk7W9127ty5XH311VgNVi479zLuDbyXVT+uYszZY3j7P29z8riTmfWXWQBMmDaBpb8t5f3/vo85JBI1OILepggyBqSjr18V8uHbr+Wr75bwxftvMThGh00xoXpcTJo4ntvu+BOqR0tcwoO8NvefJCenMu3iywG440938+abb7Jp0yZGjhxJnz59SEhIYMmSJYwbN44lS5Ywfvx4zGYzI0aMIDMzk8suu4wlS5Zw6qmnYjR2rtEsIcSxsdU4+WF+tm8mAgBjoI4zZmaQPKj1o0UdzWDSMeLsZAaensD67/ew7oe9OOtv5CrZX83Xr20kKsnKqAtS6ZUR1u2DtqqqLPtwR5Nwfdb1Eq57AkXjHc0ucm9n2PheHd2dbqW1/3sqgJRmtqcA5W3Wm2MQqFUYb9URpz/MD0EFNDoNemPXmJ5px44dLFu2jOnTpwPej3OuuvIqPnr7I/qG9mXvjr2MOGmE3zEZwzO8x5bvoMxehkcfyP0vLqD/hMsIzTgdS9oYVm3IZk9ePhoFTNhQVA/D0+LRF2dBaTbhlmrMAWYy+mf4zhsc5K2l3L1zH576EauGOmzAF7ABxo8f79uemZkp5SFCdBOFOZV88NRKv3AdnRLEZbNHdalw3ZgxQMeo81O55slTGT45CZ3h4NvhgdwqvnhlPYueX+M37WB30xCu1/+417et9/AozpRwLcRxa+0I9nvAPEVR7gF+xXuT41i8y6S/e4L61mrBWoWTAnU4jFrqkoLR9A0hMNREpaeAiAQLiqZRuUZWx/a1NebOnYvb7aZXr4O/TTaMxOftz0OjaAg1hZIUlESJrYRqR7WvncPtoKCmgCfufoJfl/zKnDlz6NsnDZ3q5MabbqbO4T/ljl7v/RYw4MLgqURRVEIM5Zg0ldg8QSj16wjZahyU7q8mMNTIhAkT+NOf/kRZWRnLly/njTfeAOD000/n9ttvJzs7m8LCQiZMmHBCr5MQ4sRSVZUNS/bx60c7/GZGGnJGIqdc1ButruuHMJNFzykX92bIpETWfJPLpp/2+24Az99ZwSd/X0t8v1BOviCV2N7BHdzbtqOqKssWHhquIznz+gyZWUWINtDagH0P3hUb32h0jBPvHNj3nYB+HROD3Y1hWynaAzVYxydQE6Q0+S38aMo0OoLL5WL+/Pk888wzTJkyxW/f1VdfzZtvvklGRgbLly/HYrBgMViwu+1sXb/Vr+2a5Ws4b9p5DJo4CIvBghkzOXv2kT5wMK7IgTjqqlAVLU70eFTFtxw7gAaVIG0RAZoKipWDy7F7PCpVJTaGpo/Abrfz/PPPExkZSe/evQEYO3Ysu3bt4n//+x8Wi4WTDpm0XgjRddjrXCxZkM3OtUW+bYYAHZNmpJM6tIVV3Lowc5CBsdPSGHpGL1Yv3k3W0jzfLxX7t5bx8d9W02tAOCdfkEJUUlAH9/b4+ML1D4eG6wESroVoI62dB9sB3K4oyv1Ab7xhe4eqqi2sPt9+HFYV68REqpflodq9o7Pucjvln+zEfZEVd5UDTaC+y9wZ/uWXX1JcXMyNN95IeLj/6o2XX345r776Ku+88w5jx47lmWee4ZJLLiEzM5NvPv8GgNjAWEptpSSlJvHDVz8w8eyJ6PQ6Xv3bq9TZ6qhz1eHASYA1FEWjRR8UBbGDsdlqcNVVAwpuvKU0esVBiDYfAA0HR77jY5PplZjEKy+/zHmTJ1FdvB+tyUJAgIXhw4fz8ssvc9ppp6HTydLEQnRFRXuqWPz6Jt9CLQCRvaxMvnFgt78ByhJq5PQr+jHsrF6s+mo3W34r8N3QuWdzCXs2l5A6NJJR56cQHm85wtk6H1VVWfbRIeF6mIRrIdraUf1vUlW1VlXVjaqqbugM4RpAVSD4rGRi7x9F0OQkNOaDoU71qLgr7DgLanBX2v3ueu+s5s2bx4QJE5qEa4Bp06aRm5tLdXU18+bN49VXX2Xw4MF8/PHHPProowCEBYTRO6Q3L/39JaIio5hxwQz+dPmfGDxyMMNHD8fuspNTkcO2sm24VTcOtwNFUTCZrVjCY0FR0IYm4IxIpyYwgRq9d4njQG0xZk0ZDUtPjjnlNKqqqzl51CQ0dXZMFbugYANjRw6kqqqKsaeejNNhb/IahBCdl6qqbPp5Px/NWe0XrgeNT2Dq3SO6fbhuLCg8gIlXpzP90ZPpe3I0NBqj2bWuiPeeXMG3czdRVlDT8kk6GV+4/v6QcH2DhGsh2prS0hR2iqJ81tqTqKp6QZv16Cj169dP3br1YHmEx+GmZnkBVT/vo2SCgX69+hxsrChoLHq0Fj1KD/lh4nA7KLWVUm4vx+1pYclTjZ4gYxDBhmBMOlOzd8173G7sddW46mpw2PS4Pf6zg2gVB1ZNMQZNnd92JzocmgBUfSC6AAsGUyAaTfte++zsbNLT04/csF5mZqbvxk3RMrlOrdcVrpXD5iLzf1vZvrLQt01v0jLhqv6kjYxulz505utUmlfDii9y2LnGf201RYF+J8cw8ryUdv0F5Givlaqq9cvF96xw3Zm/pzoTuU6tpyjKalVVRx6p3eE+w2+6pE8XoDFosZ4Wj2V0LGWbs0CrgYbVF1UVT5UDT7UTTaAerbX7B22D1kBMYAzR5miKKotwaV1UOir9wrbT46SkroSSuhL0Wj3BhmCCjEGYtAfDtkarJcASDBbvTT72WidVpTZfjaJbNVDujsPoqcaiLUGreOeX1eNC76kCexXYwa0q1GlMuHVmNEYLBrMVnU7fzldFCNFYyf5qFv9nE+WFBz+YDE+wcPaNAwmJNndgzzqPsLhAzp41kKK9Vaz4PMc3o4qqwpbfC9i2opD+Y2IZeU4y1jBTB/fWX3PhOrUHhGshOlKLAVtV1WvbsyNtTdFr0Bi16GPMeGpdeKocqK5GQbvagafGgcasR2s1oHSDu+EPR1EUAjQBWC1WYtQYap21VDgqqLRX4jk4vTlOt5PiumKK64oxaA0EGYIINgZj1Br9RraNZj2GAB21lQ5qKxy+WU7sqgW7KxCDthqrUoxW8V9aXquoBKh14KwDZwlUgx09Lq0ZDIHoAqwYjAHdfu5ZITqL7F/z+PndbbicB/+vZoyN47RL09AZusbUpu0pMtHKeTcPpjCnkuWf72Jv/VLxHo9K1i95bPktnwGnxTPi7CQCgzt+HQBVVfn1451NwvVZEq6FOKG6/V1oiqKgDdSjMevw1NUH7YY3EhU8NU48NU40Zj0aqx6Nvvu/oWgUjW8GktjAWGqcNVTaK6l0+Idth9vhF7aDjcEEGYJ8YVtRFAKDjZgC9dSU27HVOOuPVHC4rZRpgzGZVfBUozhr0HvqMOBq0h8jTozuCqirgDpwocGhBODRm9GaLBgCLGi13f5bVYh25bS7+fndrWz5vcC3TWfQMP7K/vQ7OaYDe9Y1RKcEccFtQ8nbXs7yz3aRt70cAI9LZeOSfWQvzWPg+ASGn9WLAKuhQ/roC9ff7fFtSx0q4VqI9tBjUouiKGjNejQBOlSbG3eVA7XRnNCeWieeWieaAB0aqwFNDxm50SgarAYrVoOVWNUbtivsFVQ5qpqE7aLaIopqizBqjb6abaPOiFanISgiAJNFT3WZHVf9dfW4PdRWgsEUjCU8Cp1ei9Nh904RaK9G56rFoNo5dIIXHR50ag04asBRhFoBNsWIS2dGMQRiCLCiN3auj2CF6EpK82v45vVNlOYdvEEvLC6QyTcOJCw2sAN71vXEpYVw0V+HsW9LGcs/20VhTiUALqeHdd/tYfPP+xk8MYGhZ/TCFNh+5XCqqvKbhGshOky3D9iqqvqVGyiKghKgQzFpUe31QdveKGjXufDUuVBMOrRBPSdog3/Y9qgeqh3VVDoqm4Rtu9t+MGzrjN4yEkMwRpOR0Bgttmon1eUHZ21x2FyU5rkwWw2YQ4wEBkcA3tXfPG43dXXVuG3VaFw1GDw2dPjfjKkoYMIOLju4yqC28c2TZnQmK4aA5m+ebOkmXiF6qq3LC8h8ZyuuRj/3+o+OYdwV/brMaredjaIoJKaHkdA/lNxNJSz/bBfFe70LgDntblZ/ncvGzP0MPSORIRMTMQSc2LfehnC9tlG4ThkS4Q3X3bwcUojOolsHbL1eT11dHWZz05t0FEVBMenQmHR47C7cVU5U28HyBdXmwmVzoRi1aIMMKAZtj6oL1igagoxBBBmDfGG7wlFBtaPaP2y77BS5vGHbpDMRZAgiyBxEuDmQmnIHddUOX9vaKge2WieBId6yEkVRmtw8qaoqDnsdzrpqVEc1encdRhxN+ud/82QhnvLGN096R7l1egNOp1Pm4xYCcDnc/PLBdrKW5vm2afUaTr+iL+mnxnVgz7oPRVFIHhRB0sBwdq0rYsXnOb5PCRx1LlZ8nsOGH/cxbHIvBo1PQH8CBnBUVeW3RU3D9eQbB0q4FqIddevkERUVxf79+4mPjycgoOUb5zRGHRqjDo/D7Z1lpK5R0La7cRXVoRjqg7axZwVt8A/bbo+baufBke3GI8Q2lw2by8aB2gPesB0QhDXQiq3cjdPeUDbiXQ2yrsqJNczUZMRMURQMJjMGkxmIAsDlcuKorcJjr0FbP8qtVfxHpjXN3DxpU3XsK7NTnreL3a4KevUdhkYrI3Si5ykvrGXx65so2Vft2xYSbWbyjQOJSOh6i6V0doqi0HtYFClDItmxupAVn+dQccA7hamtxukt3fh+LyPOTmLAaXHo2ujeH1+4/lbCtRAdrdUBW1EUMzAUb+rx+5+qqurHbdutthEU5F3ONi8vD6fTeYTWB6luD6rNjcfRdN5oRatBY9Ki6LV+Cw90BTabDZOp7WqXVVXF7rZT56rD7rKj0nw5hl6jx4QZxa5FbTypyE7QG7UYzbqjWmlTVQ24nA48Thu4Heg8DrQ0M8e36iGwZAMj1sxB86uLSgLJMWVQGz0Ca9oYUoaMI9AacnQvWoguZsfqA/z432yctoP/R9JGRjH+qv4YTN16jKXDaTQKfU+Koc/wKLYuL2TllzlUldgAqKt0sPSD7az7bg8jzkkm/dTY4wrBqqry+ycSroXoLFr101VRlDOAd4Gmywt6l/brwGHB/WzcdCtWSzoWSzoWazpGQ7RvlDkoKMgXtI+Wq7iOqp/2UbOmENz+4VEXbSZoQiIBgyJRtF0jaWdmZjJs2LATcu5qRzWZ+zL5Zvc3LNu/DKen6S80OreBs0uvJGHXEPAcvGaGAB2jzk9h0OnxaI7x5puivN3s3ZCJI+d3QkvWkuLcjkHxD91B1DDEthJyV0Lua7i/U9ihS6UkdAjapNHEDRpPbK+0Y3p+ITobt9PDso92sDFzn2+bRqdw2qV9GXBaXI/7JK4jabQa0k+Npe+oaLJ/zWfVV7upKfeudFtdZuend7ay9ttcRp6bQr+To4/656A3XO9izTcSroXoLFo7fPES8CXwgKqqeUdq3L6cHDjwFQcOfOXboteHYrH0x2JJx1r/d2BgHzSao5sqSRcRQOjUNKyTelH98z6qVxRA/VzarsJaSt/biva7XILGJ2IeFtXt59I+HIvBwpTUKUxJnUKVo4ole5fwze5v+DXvV1web8mNS+vgi8g3CbJEMGb3H0gqHwB4axOX1teGjrusL/H9Qo/6+SPjkomMmwnMBMBWV8OWDcso37YUY/4qkmo3Ekal3zFaRaWPeyd9indC8cewGooIRa/PYJ27gAGnXYze0PHz2ApxtCqL6/jm9U0cyK3ybQuKMHH2rEFE9rJ2YM96Nq1Ow8Bx8fQ/JYbNv+SxenEudZXee0wqi238uCCbNd/kctKUZNJGRLfqk72D4TrXty15sIRrITpai0ul+zVSlBpgsKqqO098l45Ov35G9f9eTThiO0XRExjYu1HwTsdi6Y/B0NygfPPcVQ6qlu6n5rd8vyn+ALTBRqynJxB4UrS3fKQT6oilUCvsFfy450e+yf2G5XnLcakH69t7lWUwZvcfCLZF+h3TZ0QUp07t06aroakeD3m7s8nbkIln73KiytaT5M5Fo7T8/V9KENsiJxN+6jX0GTIWpZ2XeO9sPG4PRXur2b+1jP1byyjYV0py/2iSB0WQmBHWrlOQdTXt+X9v17oiflyQjb324P+13sMimXBNOsYTPHvF8eppyzU77W42/rSPtd/sabSOgFdYXCAnn59KytCIZj9tyMzM5PTTT+f3T3exZrF/uD57loTrBj3te+pYyXVqvdYuld7agP0t8A9VVb86YuN21q9fsvrjj89SXb2Fqupsqqu34HZXH/nAekZDNBarN3RbLP2xWtIxm1NQlJZDsqfWSdWyPKqX5fnNPAKgseixnpZA4OgYNMb2fzNTPR6cDjtOmw2n3Y7TbsNps6F6PGzZu49JZ57V7n1qUG4r58e9P7I4ZzErClbgVt1oPFoG509gxL6z0HsajRbrVAadGcup5/ZrsxuADlVZXkLu+p+o3vErlgOrSbFlY1Hqmm2bq0kgL+lCUiZcS0wPKSNRPSrF+w8G6rzt5ThszdS6A4pGIbZ3MEkDw0kaFE5YbKCUIDTSHm9ebreH3xbtZH2jFfs0WoVTp/Zh8ISELvHv0VPf5B11Ltb/uJd13+/FUef/nhLZy8qo81NIGhju92+4ZMkSTBW9JFwfQU/9njpacp1ar60D9h+AJ4EXgY2A36/aqqquOcZ+Hrd+/fqpW7dubdwXbLZ9VFVnUV21herqbKqqt2Cz7T3MWfxpNCYsgX29o93WdKyWDCyWfuh0/h+temwuqn/Pp3rpfjzV/qMPSoAO65g4LKfGoTEfHNlTVRW3y4XTbsPVKAA77f6BuOFrXxu7Daet0deHPPa2s+Ny2Ft8XVqjidMuu4ohZ52HTt+xo42ltlJ+2PMD3+R8w8rClQTYrIzecwFpxf7fs3ZzNVFneDh3wjjCAsJOaJ/cLhc5m39n17f/YWjNL0RR2qSNR1XINg6iJn0a6ROvwhp8YvvUnlRVpSy/lv3byti3tYz928qw1zRdebM1rOEmkgeGkzQ4gvi+ISfsl6Su4kS/eVWV2vh27iYKdh0sg7KGmZh840CiU47tHpSO0NPf5G01TtZ9v4cNP+7zzbzUIDoliJMvTCWhvoTug1eWUJx1cL+E6+b19O+p1pLr1HptHbA9h9mtqqraYe+ehwbslrhcVVRVewN3dZV3pLu6ZiseT8uB9FA6TRQGTS+0ahwadyyqPRK3zYy7zklAoYnQ4lD0bv86b5fqZI9rCztq11FTV47T7h1N7khBkVGMufQq+o89HY2m44NPcV0xP+T+wDe537BvWyljcv5ARK1/2c/ekC3UjtrF+EGnMqnXJIKNwSesP5mZmZw2dixZv36JbfX/GFCeiVlp+n1iU/VsDjoN3fDpDBh7ITp9xyyHfKxUVaWiqM43Qr1vW7mvHrQlgSFGEvqFEt8vhF37txJlTiZ3UwmFuytpYRIZdAYNCf3DSB4UTtLACCyhPa+u/US+ee3eWMz3b2X5/TKUPDiCSTPSu1zZjrzJe9VVOVjz7R42Zu7D7fR/v4hLCyE0xszmXw7eDpU8OIKzbxyIVi/h+lDyPdU6cp1ar60DdtLh9quqmnu4/SdS37Q09feflhzjKHAdHk0JmoBitOZy9EGVGINr0Qe2fko/t11DXamRuhIT9pIAQuoG0Fc9A6s22q+dy+NkV9V6tlSsoM5d1cLZ2obOYERvNKI3mdAbTeiNRqrLy6guKfZrF9krmdOmzyR56IhO8/FxUW0R3+Z8x7qfdhOTNQiT6+CyzW7FzcaYTNYn/sDwXkM5O/lsJiROaPOwfegPmtrqCrJ+fAdD1ocMqFvTZA5ugGJC2BE1mYgx19B70Kmdtl67sqSO/VvL2b/NG6qryw7/C2aAVU98v1Di+4aS0C+U4KiD88k3vk61lQ72bC5h98Zi9maVtlhKAhCRaCFpYDjJgyKISg5CcxRTNHZVJ+LNy+P2sPyzHL+b2xSNwikX9WbomYmd5v/00ZA3eX81FXZWL85l8y/78biaf69OHhTO2bMGSbhugXxPtY5cp9Zr04DdmSWGhah3nDm2Tc+pNbkICLcTEG7z/W0KsXOYsmw/qgc8VVYCq3tjqe2DsSoRU1UvtPYQPKqH3JpN7LCtx6mzozeZ0BkPBmHvn/rHJu/XusaPDY2Ds+mQIG1CZzA0G+5cTicfvfoyxRtWY6vyn00jMWMQp105k9g+/dri8rWZPQf2882Ha7BvDEBpNPV6jb6C35M+Y3vEanRaLafGncrZyWczPnE8VsPxz5BwuB80RXm72fnjW0TtWkSqZ3ezbXZrepGffBGpE2cSndD7uPtzPGoq7AdHqLeWUVlsO2x7o1lHfN9Qb6juF3LYWuqWrpPb5SF/Rzm7N5WQu7GE8sLaFp/PZNF767YHhtMrIwyjuWuNuLZWW7951ZTb+XbeZvK2l/u2BYYYmXzDAGL7hLTZ87Q3eZNvXlWpjVVf72bLsnw8noPv2RKuj0y+p1pHrlPrtXnAVhRlMHAXkIH3w+As4HlVVTceT0ePV3xMjHrzpRejtdWisdWisdtQWvqs+ijoGgddoxG9SY8p1IExpAadtRKtuQyNqQS0zd8U1xytw4qxKtH7p6YXIbFDiRgzGmNsyHH3tzUyMzM5ZdQoVn3xMau+WITL7j962ffkMYy5/BrC4uLbpT+tVbSnih/e3URJjv+1LrDksDRlIcUW7zy/eo2eMfFjmJw8mQmJEwjUBzZ3uiNq7Q+aXZuWc2DpfHoXfEUkZU32e1SFLNMQ6tKnkT7xSixBRz/94NGqq3Z4R6jra6jLCloOtwB6k5b4tBDfKHVEgqXVi/609jqVH6gld6N3dDtvezked/P/PzUahdg+wSQNjCB5cDgh0eYuOQrbnLZ889qbVcp3b26mrurgJ229MsI449oMAqxdq0zpUPImf3gVRXWs+jKHnI3FBES5uPyv4yVcH4F8T7WOXKfWa+sSkQuAj4FfgKX1m8fW//mDqqqfH0dfj0tcXJw6a9Ys32MFlQCNhkC9DmuAgRBzICHWQIwBZr9R4UNHiRuPHLc0CnwoVVWxOwrra7qzfbOY1Nbm0GJB6qE8WkzuXgSFDyQ4apBvGkGDoe1vnmv8H6i6rJTfP3qPDT8s9qsJVzQaBk+azOipV2AJ7Tw38KmqyvaVhfz60Q5qKg7WCat4yI76jRW9vsSmr/FtN2gMnJZwGpOTJ3N6wumY9eZWP9fR/qBxu1xkLfsM++p3yKj4udl67TrVwObgcRiGXUHG2AvarF7bXuskb3u596bEreWU7D/8DDo6vYbYtBDi+4aQ0C+MyF6WY17c51h+IDtsLvZml3oD96aSw9Z8B0WYSB4UQdKgcOLTQrt0kGiLNy+PR2XVlzms/Gq378eLosCo81MZcXbSUa2G2lnJm3zrybVqHblOrSPXqfVaG7BbO4/ck8BTqqo+csiTPF6/r8MC9qFUFGo9KrV2J0V2J5TXoCgKkZGRxMbGEhMS4f07Jua4lw1XFAWTMQaTMYaIiAm+7W53HdU126iuyjp4Y2X11uanD9S4sWlysFXmcKDy4GU0GmMOmbM7HbM56bDTBx4NS2gYZ9xwM8PPvZBl7y1g2/JlgHeav/Xffc3mn39k5HkXMfL8qRjNrQ+nJ4qiKPQdFUPy4AhWf72bdd/vxeNWUdCQcWAMaaUj+T3hc7JilqEqHhweBz/s+YEf9vyASWvyhe3T4k87qrDdGlqdjkGn/wFO/wM1VeWs/OF/mLI/ZIBtnW+e7QDFwcjK7+Gn7yn+KYQd0ecQOeYaUgeOPqp6bYfNRf6OCl/JR/HeKg73O7JGpxCbGlxf8hFKdHJQh84yYDDp6D0sit7DolA9KkV7q9i9sYTcjcV+i6KAd+GNDUv2sWHJPnRGLYn9Q72Be2A4gSE960bJ2koH372xmX1bDn5SYg4ycNb1A45pYSYhhBAnVmtHsG3AQFVVdxyyPQ3YqKpq260IcpR69+6tvvHGG+Tn55Ofn09FRUWrjw0LCyM2NtYXuGNjYwkMPLaygiNRVY93+sD60e6Koo1UV2Th0B1o9Tk0mgAslr5NFsvR6SytOv5wv6Hmb9/Kz++8yb6sTX7bA6xBjJ56OYPPOKfDp/ZrrLywlqUfbid3U4nfdiXMzpq0r1muWdLscQG6AE5POJ3JyZMZGz8Wk67pt25b/SZfuG8nOT++RfTuT0nxNH8fcI4micKUi0ideC1R8SlN9rscbvJ3VfjqqA/srvKrwTyURqMQlRxEQv9Q4vuGEJMajM5wYmaKafO64go7uZtKyN1Uwt6s0ibTlDUW2ct68EbJJGunH709nmu1f1sZ387dTG2j0f74fqGcdf0AzEFduyTkUDKK1npyrVpHrlPryHVqvbYuEdkD3K2q6vuHbL8ceE5V1cPOMnIiHTpNX21tLQUFBb7AnZ+fT0lJyWHO4C8oKMgXuhuCd1BQ0AmrBa3dm0/Rb0upLN6I3bIXu3Uvdss+VG3rZzIJMPXyLZbTMNptMsU36fOR/gOpqsrudav5+Z23KN6z229fcFS0d2q/Mad3qhkydm8o5pcPt1NZ5F+fHTs4kMIhG/m2+Et2lO9o9lizzsz4xPFMTp7MmPgxGLXeUdG2/kGjejzs2vQ7RcsW0KfwayIob9LGoypsNg2ltv+lRKZeQPFep3e1xJyKFmcPAG+JQGQvq2+EOrZ3MAZT+yxwdCJ/ILudHvJ2lPtqtyuKWr7XIcDacKNkBL0ywjB0wtUKj+VaqR6V1d/ksuKzXQc/pVBg5LnJnHReSrecfUXe5FtPrlXryHVqHblOrdfWAfsh4E7gb8CveCsAx+K96fFvqqo+1cpOnQ28BGiBuaqqPttCu5OA34HLVFVdeLhztmYebLvdTmFhoV/oLioqwtPK+ajNZrNf6I6NjSU0NLRNQ7fzQC1VmXupXXcAVXXjMBdit+7Bbt2DIyIPe/A+nJ7iI5+onk5nxRLY3y94r15dzIQJE494rMfjZsvSn1j6/n+pKi7y2xeZnMq4K2aQNGR4p7kBze30sO6HPaz6ajcux8F/U51Bw4hzkgka6eL7/d+yePdicipymj1HoD6QCYkTODv5bBw7HJw54cwT0leX00HW0s9wrH2XARU/Y8TFAWdv9jsGsd8xiHxHOi4OX/4QnmAhoa93lo+4tJAOm3mjPX8glxfWsntjMbs3lpC/vbzFUXyNRiE2LYTkQd7R7ZDoji9vgqO/VnXVDr5/M4s9mw8udhRg1XPmtQNIzOg890a0NXmTbz25Vq0j16l15Dq1XlsHbAW4A2/IjqvfnIc3cL+stuIkird4eBtwJrAPWAlcoapqVjPtvgNswBttEbCb43Q6OXDggN9od2FhIS5X61auMxqNvrKShj/h4eFotcf3cbyrpI6qn/dRs6oQDp1tIdb5/+39d5yk2V3ffX9O5Vydc5wO0xM2x9mo1UpaBRCSCBJBRhiQ/djwyAHbgG2SwcDrNrfxc9/YGIPICCQhQCChVVxt3p3VpsnTPZ1zV4fK8brO80dVV3d1rJ6p6e6Z+b1fr3pVuqrqmjPVVd861++cg+XBBNnGGWLxfG13PHEFrctdba+Fe+/5HwSDd5e3L5kMb33ty7z8hb8iFSutj+04eTuP/tCP0dRzeJYNjy2nePELVxg8PVdye6DezSPf30fnyRquhK/w9OjTPD36NKOR0S2fx67snKg7kT/V5s+7Al1Y1LX33JumZnEyxuSlZcbPzTM9uIxp7PyecTkjNPTXcPzUAC39Vbh9h6Ms4KA+kNPJHBPnlxg7G2Ls7GLJbBobBevdxYGSLX1VB1Z/vpe2mrkS5qu/f7ZkjvLm3iDv+fGTN/0iPfIlXz5pq/JIO5VH2ql8120ebKWUH0BrvafVUpRSp4Bf0lo/Vbj+c4Xn+fUN2/0r8kux3wf8w/UK2FsxDINQKMTMzExJ8M5kdl7dbpXNZqOxsbEkdDc0NGCz7f2QtRFO54P2q7PoDSt5WWtc+B9vw3tPI9qSJR6/kl8aPnaRWDS/NHwut7LNMyva236UI0f+DTZbefXm6USc01/8a77zpb/btBT70VOP8vDHPk51U8s2j95/04PLPPuXg5tm1Og4UcujP9BHVaMHrTWXly/z9OjTfGX0K0xEJ3Z8Tq/dy/Ha45ysPcnxuvx5q29zGc5GWmuWpuP55ccvLjM9uEI6sfMPooB1hjbHGVodZ2h1nMNrzQ9su2LtZqH7Q/S+88eoazmwqqyiw/CBrE3N/FiU0TP5sL0wvv3Hkt1lpf1YfkXJjhO1eIP7F1bLaSutNW9+bYKX//ZKSQ/93U918sAHu696tpcbyWF4T90opK3KI+1UHmmn8h26hWaUUt8HvFdr/ROF6x8HHtBa/9S6bVqBvwDeCfwB2wRspdQngU8C1NfX3/PZz372uu231ppkMkksFiMajRKLxYjFYmSz5dVIK6Xwer34fD58Ph9+vx+fz1d2T7c1DcExRdWYwmKUhrmcU7PcrYm0a9YvVp//P10GJtBMgJ5AcwZYH47rsKh/glInytoPgEw8xsxrLxK6cIaSqSssFuqP307zPaewe67PING90qZm6QosnNEY634fKQvU9EP9CYXVnm9PrTWT2Ulej7/Om4k3CeXKK8XxWrx0ODryJ2cHnY5OAtYAmSjE5yE+p4nPg7HzYonYPeBtAG+jwlOnScWG8Ex8i7viz1OrIpu2N7TiLdttTNS/A2/3Q9ic7rLbpZJisRg+X3kDbPdLNqmJTUN0WhOfA3OH3zKuGvC3KPwt4KrmupY87dZWRkYz9YomOrV2m9UBrQ8q/C2HoxRrPxzG99RhJW1VHmmn8kg7le+JJ564toCtlHobeFxrvayUOsMOEztrrW/f9YWU+n7gqQ0B+36t9U+v2+ZzwG9prV9WSv0R+9yDXS6tNZFIpKSme2Zmhmi0/E79urq6ktlLmpubcbu3D0pmIkvspRliL0xhbugBtXjt+B5pxXeqGcs2A9ySySlefOlfAKWzhDQ3fz99vT+H3V7+cuOLUxO88Jd/yuCrL5bcbne6uPe7P8y93/VhHO7DUfuajGV45e+GOff8dMk72BN08NBHeum/v3FTsPriN75I1dEqzoXOcXbxLGdDZ1lKLbEdf6qGlkgfreE+2qJH8aQDO+6TJ+AoLOxSRdtANYE695bhLpfNcO65vyX3xmc4EXkOl9r8oy6hnZyregeue36Y4w99AOtVHC25Woe9x8PImkwNLhenAdxpFUtPwJEfKHlbLe3Haio+UHSntpobjfD0/zlLdHFt/xq7Azz1kyfx1xzYBE0H4rC/pw4TaavySDuVR9qpfNfcg62U+kXyAxgTSqlfYueA/ctl7NCuJSJKqRFgNWnUAQngk1rrv93ueQ8iYG8nFottmsFkeXnzCn/bqaqq2jRtoN9fuvS3mTaIvzJD9NlJzFhp4FIuG76HmvE93IrVu3ng27e+9S0GBla4PPirJSUkDkc9R4/+Mg31T+3p3zt9+SLP/vkfMnXxXMnt7kCQU9/7MW5/13ux2g7H1H4L41Ge/cvLzA6XTuPY3BPk0Y/2U9+x1s4bP2i01swl5oqB+9LEMOGRLNWLbbRG+gika3d87Yw9QaYxTHWPg/4Tbdx99AQB584hfKNoeImL3/hTPBc/z4nM21tuM08NV5rfT9Ojn6D7+H17ev6rcSN9IGut8wMl315k7GyI6aEweruBklZFS19VsXa7quHafyxu1VZaa848M8kLnx8qWd3yjne1c+pDPQc6X/lBuZHeUwdN2qo80k7lkXYq32EsEbGRH+T4JDBFfpDjD2mtz22z/R9xSHuw9yKVSm0K3aFQiHLb3efzbZo2sKqqCnIm8dfmiH57EmOltAZBOSx4H2zG/2gb1nVLJ6/+AaUzIS5f/mXm579c8riGhvfT3/+LOB11Zf/7tNYMv36a5z/zx4QmSud6DjY28chHP87RU48eiqn9tNZcfnWOF78wRCK8vm4ETjzayoMfPILLZ9/0QZOIZJi6vMzU5fwS5CtzOy8/nrYmmQkMMRUcZDowyKJnBlTp/3dXoKs4iPJk3UkGagZw28or9ZgZu8TYt/6Y5vG/o9Oc3HKbK9YjLBz5ML1PfoK6po6ynnevbuQP5HQiy/j5/IqSY+cWScW2L/mqavTQeVstXSdrae69uoGSG9sqnczxrT+9wJXX12bpcbhtPPmjxzhyZ/2en/9mcSO/p/abtFV5pJ3KI+1UvkrPIvJN8kuir2y4PQD8rdZ697nf8tu/H/ht8tP0fVpr/WtKqX8OoLX+3Q3b/hE3QcDeSiaTYW5uriR4z8/PYxjbL6yxnsvlWgvcDU1ULzuwnY5gLG0o9rUpvPc14X+8DVuVa9Mf0MLCV7l46RfIZNa+5G22IP19/4mmpg/vqSbVNA0uPPcML/zVnxFdLJ3ar6G7h8d+6MfovP3Osp/vesqkcrz2pVHe+uZESc+h02PjgQ8eYS49yJHGk4Xlx5dZmo7v8Gxgc1pp6Q3S3BfE0pZk3DHI+eVznAud4+LSRTLm7oNkLcpCT1UPJ2tPFkN3X3UfDuv2M4Zo02ToredZfPFP6F94mhq2rtc+576XzMkf4MQTP4jb69/ima7OzfKBbJqa+dFIcaBkaGL75eYdLivtx2vouq2OjhO1ZS/0sr6tFsajfOX/nC2Zu72+w89TP3mSYP3B1NMfFjfLe2o/SFuVR9qpPNJO5at0wDaBJq31/IbbG4AprfWB1QHciAF7K7lcjoWFhZIZTGZnZ8seTOlwOKj31VAdc1MTd1Fr+qnWXixYwKLw3N3AINPc+9RDWHz2YnjOZsMMDf0G0zOlA0Vrax5jYODXcLn2NjtILpPhjaf/gVf/5rOk4qVBpfP2u3j0hz5BY3fPnp7zelmejfP85wZL5houh9VmoaknSNvRKlqP1tDQ5ce6zQwPWTPL0PIQ5xbPcTZ0lvOL5xlcHiRXxtSKNouNo9VHi4H7eO1xeqp6sFk21wdnM2nOPfc3GG98hpPRF3BuUa8d024uVL8D970/xPFTH8ByjVNK3qwfyLHlFGNnFxk9s8jkhSVy2W3my1fQ0Bkozrld1+7b9kfpM888w+OPP86556Z5/rODGLm157zt8VYe/r4+rPaDP8pz0G7W99T1IG1VHmmn8kg7la8iAVsptTph8mvAe4D1ScQKPAX8hNa66+p39drcLAF7K6Zpsri4WOzlXg3eqdT2g7XWs2hFjfZRa/qp037qzSA12ofNY8fW6MXe5MHe5MXe6CHqfItLI/+ZVGqt5MBq9dLT8+9oa/1h1B7ngE7FYrz6xc/zxpe/SC5b2oM78PDjPPzRj1PV2LSn57wetNaMvh3i+c8NbjsIzmJVNHYHaO2vpu1oNY1HAtjsVx9OU7kUl5cvczZ0lnOL+Z7u4fAwevthDkUuq4uBmoFi4D5Zd5LOQGfJHN3h5RCXvvmn+C79NcczZ7Z8nlnqGGl5Py2PfoLOY/dc1b/jVvhAzmUMpi6vMHYmxOjZxZKBiBt5g6sDJetoG6guGSj5ja99C2O8oWSOdrvLyhM/MkDfvY3X9d9wI7kV3lOVIm1VHmmn8kg7la9SAdtkbXDjVl0zSeCntdafvqq9rICbOWBvRWvNysrKphlM4vGdyxhW2bWVBjNIoxmkSVdRbwaxkw+Lqtok1P83hAL/UFIzHAzey7GBX8frPbLn/Y0uhnjxc3/BuWe+jtZrvXYWq4073vM+HvzIx/AEyp/B5HrJZQ3e/NoEb3xtnEwqR0NngLaj+dUSm3uqsDuvrbd3N/FsnAuLF4qB++zi2V3n5l7ls/s4Xnu8uCjOybqTtHhbUEoxPXqJsWf+kLaxv6NdT2/5+CFrD6Gej9D35CeobWwre59vtQ9krTVLM/Hi8u2zw5HtB0raFK391XTdVkt1s5ev/MGbZNZNMlTb6uO9nzx5aFaaPCxutffUtZC2Ko+0U3mknXZmGgaLUxPMj1zh5DveVZGA3Uk+WA8D9wPri2szwLzWurzC4evkVgvY24lGo5tCdzgc3vVxSitqtY8ms4pGs4pGM4gKTjB74tNkfNPrtrPTYv4obQ2fwNkUwFbnRu1h4YvFyXGe+8yfcOW1l0tut7vc3PfBj3DPBz6Ew3Xw9adaa771zWd455NPHPSuEE6HObd4jvOL54u93bPx2bIeW+2s5nhdPnSfrD3J8ZpjhC9fZPnFP6E/9FWq2TylZE5bOOe5l+zJH+DkEz+Iy7PznKi3+gdyKp5l4vxSvnb73CLpeHkrqh5/pIVHf6APm+P6/mi7Ed3q76m9kLYqj7RTeaSd1hi5HIuT48yNDDE3fIX54SEWxkaKR+N/5rNfOlyziFwvErC3l0gkimUl09PTDA4OlrUqZcB004iPxs43UJ3fBstaz7Mz0knTuX+KK9GFvd6NrcmLvTFfZmJv8mKtcqIs2w+OnLp4nmf//A+Zvnyh5HZPsIpT3/dD3PbO9+zrXM5bOcwfNKFkiHOhc8Wa7nOL53aco3u9BncDx+uOc6xqAN/UEp2XX+JU5BUcanMwjGo3F6qfwHvfj3DswfduWa99mNtpv5mmZm44zOjZ/Jzbi1ObjyjZHBbe8cMDHH3g4EujDit5T5VP2qo80k7luVXbychlCY2PFcL0EPMjV1gYH8XYYfxbxQN2YZq9+4EOoGTovNb6T8p6kutAAnb5vvWtb3HnnXcyPj7OxMQE4+PjzM/P7/gYr3eJo/0v4/Uvrt1oWqgZfT+1wx/EYpbOoqAclnx9d6MnH7wLdd7rB1ZqrbnynVd57i/+iKWp0jKIqqZmHvnYj9L/4MPXdWW9ndxIHzRaa2bjsyWB+9ziOaKZzT3UW2n2NNGadnF0cZp3Jic4ns7g2/CZMEs9I60foOWxH6Pz6J3F22+kdtpv0aXVgZIhpi6vYPMafPinHqCm5XCsdHpYyXuqfNJW5ZF2Ks+t0E65TIbQ+Gg+TI9cYW54iND4GKZR3tFHX20djd29fPjf/+eyAnZZXYVKqQHg74Fu8iUjRuGxWfLrbx9YwBblU0pRXV1NdXU1d9xxBwDJZLIYtsfHx5mamiqZLjAer+H1N95La9sFOjvfwmo1wGKydOQfWGh8Cff5j9GyfCdO8hPJ6IxJdiJKdqI04Fk8trWBlY1e2psG+Piv/DYXTn+bFz/758SW8gF+ZXaGf/jt36Cpp49Hf+jH6Di56yKhtzSlFM2+Zpp9zbyr811APnSPR8eLtdznQue4sHSBZC656fEziVlmgNeq4M+r8oPt2rKaO9MJTqQznEynOZoJcWrqj+Azf8SgrY/Fno/Q/+SP7t8/8gbkr3Fx8rFWTj7WCuS/vCRcCyHE/shm0iyMjjA/cqXYO704OY5Z5nTIgfoGGrt7aejuofFIL43dPXiCVfk7//1/Lus5yj0W/9vAd4A7gdnCeRD4X8B/KvM5xCHkdrvp7++nv78fyE8XODMzUwzc4+PjJJNJpiZPsLTYTl/fSwSr8r3eVu8i6Xt/hxemjxKZeIz6TB0NmQBNugqfdqHWjYs1EzkyI2EyI6V14XXBIB++59+yFJ/m0qWXWIxOEskuMntlkM/9l5+n6857ePQHf5SGrr0PsLxVKaXoDHTSGejk/UfeD4BhGoyER4qB+9xifo7urLn5MNikXTFp9/IPvnwgtGpNTybLiUyGk+kZToz8d1y/89/w2u7gpeEvYa/vJdA6QGP3CYLV5S9SJIQQQlyrbCrF/NhIocQj3zu9ODmONreZYnWDYGMTjV09NBzppfFILw1dR7acfGFubo6XXnqp7P0qN2DfBzyutY4XZhaxaa1fV0r9e+D/AaSb8SZhs9lob2+nvb2dhx9+GK01oVBoXS93J/MLr9Dd/To2WxaloLX1ErW1kwwOPsiF5fy82T6nh2ZXHY1GkPqIl+qMOz8n9wZGOIMRzuDDyz2Bd0Eg3wMbyy0TzoQIjy7wzK/8T2pv6+aej30vVc1Sv3o1rBYrvdW99Fb38qHeDwGQNbIMrgyuzVwSOsvQyhDGhnHLhlJcdjq47HTwN4V1auxa05eZojU1TuNIjqZBg8ZvGLhzTiyqEZuzE6qP4GjoI9g6QPORE3j9Vfv7jxZCCHFTySQTzI8O5wcfFsL00tRkySxlO6lqaqaxuxCku3to7O7F5dt+QL/WmitXrvDSSy9x5cqVPe1ruQFbAavrQy8ArcAlYBLo3dMrihuKUor6+nrq6+u5++78tOixWIyRke8wO/dbWCz5le5drji33fYNZmd7GBm+h1gaBtPjDAJYwOF30FzTSIu7jiYjSG3UAwsZMDaPAVBK4bfX4LfX0ObN96wzB5Hfvsii6wz+niZc7cFCjffuAyvF1uxWO8drj3O89jjf3//9QH6O7otLF4uh+9ziOUbCI5vm6M4qxXmnk/POrZ45gdLnqTPO0Dhp0Dhq0PhtA59hx6mrcTuaqPL30th4gvr2kzR1Ha/oCpNCCCFufOlEvFDika+Xnhu5wvLMFJQzdlApqptbaezuyZ8KgdrpKa9UL5fLcebMGV566aVdx6ptp9yAfRa4g/x0fa8C/0EpZQA/CQxd1SuLG5bP5+O22x7n5MnHmJv7Ipcu/wq53AoATU1XqKmZZmjoPhZDncXHZLIZxuYmGCM/qFEpRVNHE231LTS762g0q3AtabJzCXKLSbZac8WirDjTVjLnV8icXynevuXAykYvFr/9wAZK3qhcNhd3NtzJnQ13Fm+LZWJcWLpQMnvJZGxy+ycBtFIs2Gws2GycLQnhSWAkf5r7GjXTBo0vGFTlLPi1F5+thip3K43VfXS33cnxvgcJ+mquw79UCCHEYZGKxZgbyc/iMTc8xPzoFZZntl67YSOlLNS0ttHY3UNDdy+NR3po6DqCw733dQYSiQSnT5/m1Vdf3bS+iFKKY8eOlf1c5QbsXwNWY/9/Av4B+BYQAn6g7FcTNxWlFE1N30NNzcNcvvxfmJv/BwAcjiTHjz+LzXo/odB7GBtbJhKJlDxWa12cr3tVdXU17Ufa6Xi0nWZvPcG0i9x8ktjQLKmpCC62/mMpd2ClvcmDvcGDxWOvcEvc3HwOH/c13cd9TfcVbwunw3zuW5+jub+ZucQcc/E5ZuIzTK2MM5+YI2zEyliXEpasVpasVnBCfrz0DOgZWHoNlj4Db0PA0FSbNoLKR5WjlgZfO+0NR+lvv5PWYBuNnkY8dlmwRQghbgTJaKTYKz0/PMTc6BXCc+Wt8aAsFmrbOtaF6V4aOruxu1zXtE+hUIiXX36ZN998k1yudFYRu93O3XffzQMPPEBNTQ0f/ehHy3rOsgK21vrpdZeHgeNKqRpgWd/oE2mLa+Zw1HHy5P+gceG7uXTpF0hn8stB54xXqau/xKmHfh6368liHffExARzc3Obnmd5eZnl5WXefvttID8As729nY67Omj/4ACxsTHO/e1XsUQ0QXsdQUc9QUc9LuvW4Wq7gZXWgCM/f/dq8G70YGvwYJGFP8oWdAbpdfXyjiPv2PL+rJFlIblQDN/T0SlG5y4ztTJKKDnHkhEhrLLoMg4wRKyKiNUAwmCGITIMkW+XHDvzaCvVFh+1znpaqjroqu+nxd9Co6eRRm8jjZ5GfI6dF84RQghRWYlIOF/eUZhjem5kiMhCeSUXFqs1H6aP9BZn9Kjv7MLuvLYwvUprzdjYGC+99BJbTffs9/t54IEHuOeee3C7974Q3lWv6KG1Lm91C3HLqK9/F1VV9zN05TeYnv4rAHK5MBcu/Adqqh9hYODXuP32/HjYZDLJ5ORkyfSAG381JpNJLl++zOXLlwGwWq20nOjGrQ3GzrxObuEZlGHgtHgIOuppbzlOb8+9ODJOsrMJdGbr6XiMSAYjkiF9eXntRgW2Ghe2Ri/VGUW6O4KjzY+ySonJ1bBb7bT4WmjxtWy7Tc7MEUqGmApPcnnsDcbnLzAfGWM5vUDYjLJizbJkhVwZZT4JZZDQYaZSYd6eHYLZb27axq2c1LvraQm20eRtosnblA/g60J4wBGQsiJxqBmGQSwWwyxzhgQh9kt8ZblQK51fAXFuZIjYYqisx1qsNuo6OotT4jV291LX0YXN4dj9wXtkGAbnz5/nxRdfLDmKvqqpqYmHHnqI48ePY7uGhe+2faRS6ltsWQm7mdb6nVe9B+KmYrcHODbwX2ls+C4uXvyPJFPjACwtP88rr76PniM/Q1vbx3G73fT19dHX1wesTQ+4fk7uRCJR8tyGYTAxUViYproJqpuwZlKk4lFiyRizE9/gteF/pOfu+3j4x/8JNYFmsnMJsrNxcrPx/OX5xJYDK9GQW0yRW0xRi4WFobdQTivOnipcfVU4e6vyy8NL+KoYm8VWDLr3tGw9Z38iEeXS0GlGJl5ndukSK/EJYrkQcRUnbM0xa7Mxb7OSLeP/JanTjCcmGU9sXz/usrrWgnchdG+8XuWskveB2FeGYTAyMsL58+e5ePEiiUQCu92Ow+Hgvvvuw+nccrSxENeF1prY8mKxXnq1dzq2XF6/q9Vup76ja90c073Utndis1/f8s1UKsV3vvMdXnnllU1lqwD9/f2cOnWKrq6uinzG7xTNz667bAV+mPwc2K8UbrsfaAb+7Jr3Qtx0amoe4oEHvsTw8G8zPvGHgIlhJLg8+CvMzf8DxwZ+Ha93bQKa9dMDPvTQQ2itWVxcLIbtiYkJFhcXN72O4XBhOFxkq+sBULkM5+YWufgbv0rfwDHe9YM/QtVAe3F7bWhyi0myc3Gyswlyc/ngnQttHlip0wap84ukzudf11rlxNlbhauvGmdvFVav1HJfbx6Pn7tufyd33b75N3wyHmV29DxLExcIzZ0nEh0kmZkmo5eI29LMWW3M2azM2azMWq2kLZunidwoZaQYjYwyGhnddhuHxbFt+F49r3HVYFG7v54Q28nlcly5coXz589z6dIlUqlUyf3ZbJavf/3rPP/88zz44IPcf//9eDwyFkFUltaa6GIoPwBxeG0FxER4pazH2+wO6ru6i4MPG7t7qW3rwHoNPcN7tby8zCuvvMLrr79OJpMp3T+bjTvuuIMHH3yQ+vr6bZ9Da41hJDCMWNmvW9ZS6Uqp/04+ZH9qfc21Uuq3C8/xqbJfscJkqfTyHdRSqOHwm1y4+LPE44PF25RycKT7p+no+EkslvKCaiwWK+nhnpmZ2f0wqWlQ5fVw4s67ONLbR1tb25a9PTprkJ1Pkp2NM/biRapjLoxwZosnXP0HgL3Fh6u3CmdfFc7OIMp+awWqw7y0biyyzOzIecJTF8nMD2JdHkalxtF6hpQ1xazNxpzVWgzgc1YbszYryTJCeDlsFtta+YmnkXAoTGtLKxZlQaGwKEv+slJYWHd5l/utylqyzepjLKy7XMZrbNz+ap6jnNcoub+M13ju28/xzidu3QOi2WyWoaEhzp8/z+XLl0mn01tuZ7FYNn32rfZmnzp1Ct8O8/reag7z59Rh8swzz/D4448TWZgvmc1jbuQKyUh49ycAbE4nDZ1H8r3ShWnxalvbsVgPZnzTxMQEL7/8bQYHz2C1ZrBac1ht+XOvx0L3kWba2uqwWjLkjBhGLl5ynsvFMYwYuVwMw4iz2gv3rieHy1oqvdyAvQic0lpf3nB7P/Cy1vrA5tGSgF2+g/ygMc00o6O/y+jY/0TrtVprn+84x479OgH/yT0/ZyaTYXp6uqSXe7svpFX5mU+a8oMnOzro6OggEAiUbLP6QZNbSJIeXCY1tEL6Snjbmm4AbBac3YFi77a9yXvTz819o35xhZdDzI/mw3d2fgj7yhUCiXEaclNYLIm1nm+rlTnb+sv5IB613lo/pPabVVmxW+zYLDbsFnv+ZN1wfd39Nqtt023lPM5u3eK21ccoW8n9Wz1u9bpVWa/pcHI6nS4J1dns5tVVAYLBIMePH+fYsWM0Nzfzuc99joWFBZaXl0u2s9ls3HPPPTz00EMEg5tXo7vV3KifU/slk0xw7tvf4PTXvkJ2ZYlULLr7gwC7y01D15G1mukjvVS3tGKxXFuY1lpjmsmScLtV+DW2uT2Xi5FKrZDLRlGWDEpVfh6OcgP2XhaauQ24vOH22/a6Y+LWZLE4OXLkUzQ0vJfzF/4D0egZAGKx87z22kfo6PhJurt+Gqu1/NHBDoeDrq4uurq6ADBNk/n5ecbHx7l09gxjY2PkNhymXz894KuvvgpAVVVVMWy3t7ejtUYphb0hP62f7+FWtGGSmYiSGlwhPbhMZjIK6zuQcibpwRXSgyv5f6/Pni8n6a3G1VeFNSg1kodFsLqOYPVjcNdjm+5bCc1ijJ7DOXWJ1oUhusIjBKLjNOWm8KskAHGlCmUntnU94IUwXgji4QPqsbkZGNrAMAzY4ffsYaJQu4f5Dfc7TAfuZTeOkAPrkhVlbh3QrV4rvjYfgfYAvjofCWuCt7JvcX7qPMvVyzzyzkdYHFnkwmsXWF7MB+1cLscrr7zC6dOnueuuu3j44YepqZG55EWp8Pwsb3zl7znzza+RSSZ23Nbh9hSmxesp9k5XN7WgCkf88qE4RTa3jJGLkjPihQC8/jx/+2pvcOl5bN15nNIv16tzjTl/S0o5UZSfUcrtwf5vwI8Dvwm8XLj5QeDfA3+otf63e9/VypAe7PIdll/yppljYvKPGB7+vzHNtR5nj6ebYwO/QVXVrj8My6K15s1vf5Pnv/R3RNI5DI8P0+mGXXqbrFYrHR0dtLS0FE9VVaUD28xUjvSVlXzgHlrJ13DvwNbgxtVbnS8nORLE4ty/+rPr5bC8n/aDNk0W56dYGDtPdPoSxsIQzsgIweQEzblpPKr0yElSqXUlKDbSCjQKk/xXh1ZgFq5rwCxcX7u87v5119ffX3w+VXgO8gv8FF8DMNW61yhc1+Sz6/r7DRQmCkOpwmULhgITS/FxJmrdeeH1lSo+jy7ZF4Uu2S/QhfVA8ydduE+vu64xKvDFeljZDTvNiWZaE600JhqxsnUCiNqjTHommfJOEXaE891bu9HQkmjh2MoxqjJVG+7SGE0Gjh4H/ho/PrsPn8OHz+7D7/CvnRdu89q92Cw3/ufTqlvpc2o3Wmsmz5/h9X/8Ildee7W4vLiyaJxVaaxOA5ffQXVLHYHGanx1fjxVbmxOMMzEul7j9eE5H5S1Ppy/iE3DisW0YDEUqnCy5sBiaKw5jc0wseVM7LkcDsPEkcviNHK4DAOrobEVtrMamtXuOvXLkYqWiFiAnwE+RX5gI8AM8D+A39IH2LISsMt32D5oEolRLlz8eVZWXim5va314/T0/Aw2W2XqCI1cjrPf+iovfu4viEejGG4vhseH4fZhevzoMg7vut1uWlpaaG1tLYbu9aUlueUU6cEVUkPLpIdWMBO57Z/MonB0+PPlJH1VOFpvzOkAD9v76aBo0yQ0O8786HniM5cwQkO4wiNUpyZoMmZwqa0P+YvNVsN/VimyCrIocquXlSKLIomNuMVOUtlJYSNtsZGy2EkrGxmLlYzFRtZiJausZK1WshYrhrKSs1rIWSzkLApTWTAsipyl8KNCgWkBA42hwFAmJpocGgODHAaGNsnpHDkzR9bMkjWz5Mwcxg5ffw7DQUuihdZ4Kw3JBixsXV4UtoeZ8k4x6Z0kao+WF6q3acCmZBMDKwPUpms33KWZ8kxxseoiYefONbVumxu/vRC6HT78dj9eu7cYyH2OtXBeDOyF7VbP7dbDMQhcPqcgl8lw8YVv8/o/fpGFsZHCrRpPQ4rq/jA1fTGsjsPzOWUxdDHc2nJrl62GWbyeNDxMGG1M5FrJGU5yhh3DsGHk7NgMk9uNy9yXO0OQ+O4vuEcVDdglD1AqAKC13jzHyQGQgF2+w/hBo7XJ9PRfMTj0GyWjc13OFgYGfpXa2scr9lrZVIrvfPnvOP3Fz5NJ5nucNQrT5aHq6HFcLR3MLYQ2LY+6HZ/PV9LL3dLSgs/nQ5ua7HQsX7s9uEx6NLL11IAFyrU2HaCrtxprreuGmAbuML6fDhvTMFiYGeX0S89z/NgARjaNkctgZDOYuSxmLo2Zy1/WRgYzlwEjf1kbWchl0GYWjPxJGRkwcygzWzxZCtctOoelcG5dPS+cbDqLFaN4bieHTeewkcOhDmfP02GV0VYyOMgoOxkcpJSDtLKTtDrIWOxElIcZs4kFo4GIEWC7tGxzZrH7s9ircuAB02rBtFjRVoVpsWBa80cKTGvhyIQFTKXJYZAxMkzOTWL324llYsQyMaLZKMlc4UiahvpUPQMrAzSkGja99ox7hotVF1lyXb/lLJxW5+ZQbl/rKd/Yi14M8g5vcTuX9do/C2/lz6nY0iJvfe3LvPX1rxQHKtp9War7wtT0hXFV7zCQf48s5mqPsC7p+d0YjG2b7tsYojWWbb4uNTBEFy9yDyN0bLq/mhVO8Tp3cg4HO3RylUEDUxYnV+xORhwOxux2JuxWJu0Wnv7Js9cnYB82ErDLd5g/aFKpGS5d+gVCi6ULhDQ1fZj+vv+I3V5dsddKRMK88jef5c2nv4RprP0RKmXh+OPvJF3bTO+JE8zMzDI9Pc309PSmKbK2EwwGN4Vup9VBZjRCanCZ9OAK2dmdA7y12lkcLOnqrTq0S7sf5vfTYXOY20qbJrlcllw2QyaTxsimyWUz5LIZjGwKI5cll0ljGtniDwQzm8E0suhcuvBDIf/DwMwVfhis/kAwsmDmryszV/yBYCn+OMiizBwWnf+hoLNJnBYTm85g1xnsZgY7WRxkcOoszkN6NCCClwv0cZ5exmhju1DdwizHGeQ4g9RQ3swMG+W0hTQOllQVYWcTCXcLhr8VW3U7jrp27A3NuOrqyFpMopkoU5NTDL8xTHhq8+tlghlCLSFCrhCxbD6ox7KxQuHOwbMpW0kQ99q9m4L6+vKWTYHd4efV51/liSeeOOh/yr5IxMLMjV9m4s2XGf3Oa4SmFtEaLDaT4JEINf1hfC2JLasknWkDV2qrIGxuE5pLe5i3C8VXI63tZLCRUQ6y2Eni4rLq4bzuI4x/0/bV1gRd7igNbgPsTrA6weZE2ZwouwvL6rndidXhwmp3Y3W4sDncpCw5Zs0ws9kQM5kFplOzTManGY+Nr/1g3eDsJ64xYCul3gYe11ovK6XOsMOiM1rr23d7oetFAnb5DvOXPOTrw+bm/p7Lg/+FbHatZ8Vur+Xo0V+iof59Fe3ZDc/P8sJf/RkXnn9m031KWfAEg3ira/BWVWMNVJGzu0hqiKTSLEeiZHPl/UKurq4uhu3W1lYa/LUwniRVmKHEjOwyHWCrb61+uzOAsh2OWSwO+/vpMJG2Ks9u7WQaBplMinQqSTadIJNKksskyaZT5NIJcpkURiaJkUliZlPFk86m0Lk05FKQS6FyaZSRxmKkUUYGq5m/bDMzWM3MWsAvnBxkcOgsTrJYCrMSrODnAr2cp58Jtl+xtJ1pjnOZYwxRRXkzNGz6dwNJt5W4J39KuSz44gb1ixlc6a3r1kNUsWRrIOZsIuNrJeJsZSTmYmZp8w/89vZ2HnvsMXp7e9FoEtkEsWyMaCZKPBsnmokWr68G8eJ92WgxmK+/f6fSmf1kV3Za/a20+Fpo9jbT6mul2Zc/b/G2UO+pv2HmrDdyORZmRlmcuEx8bghjcQR7ZBxfYoqa7AzLURuvL7UwnQyC0vhaEtT0hwl2R7DaN0c4a86kIZSheS5NVThLZkOwzSoHOWUnZ8mfGxZH4eTEtDoxLQ601YG2OtG23YOtxebG5swHW5vDic3pwe5wYXe6sTvdOF1uHA5XcfBkPB7n9OnTnD59etORZaUUx48f59SpU7S1te3YbqlcivHoOGORMcYiY4yGR4uXl9PLOz52K5UI2L8I/F9a60Th8ra01r+85z2sEAnY5btRvuQzmUUuD/4qc3NfLLm9vu7dHD36Kzidmw95Xou5kSs8/5k/ZvSt18t+jAZMhwvT7UUFqjHdXjJWe9n9PnV1dcXQ3eiuIbjixByJkR5eQWe2H+il7BYc3cF8OUlfNbZGz4GVk9wo76fDQNqqPIe9nRZDIc6eO8uF8xeYnZvbdrvagJvmKheNfgtOlcXM5oP9ashfDfgql8JiZrCshnwzhXYk0a4UhidLzp0j6zFJe0BvM+1nIJKlPpShIZTGk9p9kOgCNTzHfZzRA+gNwTLgtnKyv5vjt99NQ9sRHM7yZ0xYT2tNykgVS1fWl7GsD+yrQX1973kxqGdiZMzKlTBsx2ax0extpsXbkg/hhfC9GsYbPA37OugzGl5ifuwi4ZkrZBauoFbGcMcmqE5P0WjO41ClHTvJnI23V5p4c7mFWM6JsypNTX+Y6r4wDt8WnUAabFEvlkQvdvv9eBuPUdPWz8XhSZ5817v36V+5s4WFBV5++WXeeustchs6shwOB3fffTcPPPAA1dVrR7ZzZo7p2DSjkbXwvHp5Nj57Vfvhd/jpDnTTGejMn4KddPo7OV53XEpERKnD/uW10ULoG1y69Auk02t/HDabn77e/0hz8/dVPFiOnXmTV//2s0wPDZJL7TwryFY0YDrdGC4vptuTP3e5oYzeEaUUtTU1tLa20uSuoybhITCjMKYSOxw7Aovfnu/dLqwwaQ049rzfV+tGez8dJGmr8hzGdgqFQpw/f57z588zO7v1F7VSiq6uLo4fP87AwAB+/+bD2OuZZpZkcoxYfJB4fIh4fJB4fJBEYgStr74MxpawUbWQo3MhTDCR3XGs5BJBnuc+3uQ45oYZTeoJ8Yg+TaNaIGyrJ+ZqIutrRVW146ztxN/YRV1bH4Gq2mJv4/WQMTJrAXw1hK8L7avnm0L7ul717Q7zl8uqrDR6GjcF72ZfM63eVpq8TXsa0JnLZpifGmFp8hKJuSsYiyM4ouP4k1PU52aoLvMoRyjt4Y2lFs6HG9AOTXVvhOr+MN6GrcsZXa5u2lq/n8am78HlbNp0/0H/7WmtGR0d5cUXX2RwcHDT/YFAgAceeID2Y+3MpmdLgvRYZIzJ6CQ5vfe6a5fVRUegYy1EBzrpCnTRGeikylm1Zc5QSknAFqUO+g/oauRyUYaGfpOp6c+U3F5T/TADA7+G292+zSOv3jPPPMOjjzxMfGWZ2NIS8ZUlYstLxJeX113On++2wpVWanPoLmOqQACFxmOzUWcP0KRqaUrVUJvwbjsTAYCt0VNYXbI6Px2g4/rNx3wjvp8OirRVeQ5DO2mtWVhYKIbq+fn5LbezWCwcOXKE48ePc/ToUbxe76ZtTDNDIjlWCNDrg/TonoO009GI19uH19uL09nA0JUvodSlkoW71nPYm3FwEh1txFzIYo1M4UrMEEjPUWfMFed1D+PjRe7lO9xGbsPSGDUs8winuZ0L2LaYQjGuXSxY6wk7mkh5W9D+Nmw17Xgauqlu7qauuQu742DXAPjHb/4jPXf1MB2bXjvF1y5fTYnAegpFvae+JHxXKT/uWAb3cgz/8iKu8ASe+AQ1mWkazBD2qxxYrDW8HWvlzZU2FhN2Ah0xqvtXCHTEtpz32W6vobHxu2lu+jB+/8kdO6UO6m/PMAzOnj3LSy+9tOUPWGvQSrw1zhXXFcaiY1f1g8mqrLT6WksDdDB/3uBp2HOJULkBe9vjHrvVXa93kDXY4uZms/kZGPhVGho/wMWLP08yOQ7A0vILvPzK++jp+be0t/0TlKpskLTa7ATqGgjU7VyOYuSyxFdWiK/kA3hsuRDIS4L5EolwCFbyf05aWTBd+dBtuDyYbi+mw7UpdGsU8ZxBPLfMGIUvAYeJJwt1pp92SwtN1BDUXiyFvqrcXILYXILYC9NgAXuHH3d/Da6+auytvpt+dUkhrobWmtnZ2WKoXlxc3HI7q9VKT09PMVS73W4gH6RjsUuFEF0I0omhQo/03nrVnM6mQpDOh2mftw+Ppxe7vXTF2ZGRozz88F2EQt9kfuErLC09V7KuQCY7Q4YZcIGzt4na+ndTX//PqQreh8ViI7KyyOLUEOHZEWpCYzyyOMXkSo7RTE0xaC9RzRd5D8/wII/wGndxFvu6FYC8KoXXnIDUBKSARWB0bR8NrZhVtSzbG4i7msn6WrFUt+Oq6yDQ2E1dWx/+4PVdBMdtcdNf3U9/df+W9yeyCWbiM5vDd+E8lAzt+PwazXxinvnEPG/wxpbb1HoNWp05mnM5WnJ+WnI5WnI5WnM5mnMGnnUdnWltZ9bayIqzhZS3HV3dhTXYzspCnLHX3yCjR6m5Y4kTPRFs7s1BXSkHdXXvpLnpw9TWPo7FcjgHyS/HlnnmpWe48MYFMonSUiCNZsYzw2BgkJArBEnyp100eBqKvc/re6Jbfa0HMm3kToVFn9+3vRBiFzXVp3jg/i8zPPLbjI9/GjAxzSSDg7/K/NyXGDj26/i8ffu+X/kgXk+grn7H7UzDIB5eXgvhhR7wfDBfIrK8RDieJG6YGE43htuLdmxR/2ixkHDCOHHGyR9Gs5hQZbhopo4GHaROBwhoN8pUZEejZEejRL46Ro4scXeUTI2BarHjbqnGW12Nr7oWb3X1gfc0CbGftNZMT08XQ/XGJcdX2Ww2+vr6OHbsGL29XZjmDPH4INMz3yqG6WRydM8LbTidzYUA3Y/X21sM1DbbzuUl69ntQZqbP0xz84fJ5eIsLj3LwsLThELfKpn2NJ2eZXLyT5mc/FPs9mrq6t5FQ/1TdB57CMuJB0qeM5FI8Morr/DKK68UZ0+KEODLvJOvq8fot05wp/EmLebMpgWWNrIqTRMhmrIhyJ6HKPkVNNaJ4GHR0kDE2UjK24oZaMNe046voZvqliPUNXVitV2/GmiP3UNPVQ89VT3F27RpshyaITQ5yOL0BeZCF1iOjxDNzhNTYcLWDLN2K1M2G/NW665rKSzarCzarLzN1p+xPuWiwVFHa7CDrtqe/KBMbwtVKScLL7zJlS//Hb72WZrfsf3UeoHAXTQ3fZjGxg9gt1dddXtUUtbMMh2bLhlYOLkwiR7TNCw1YNOl/685lWPMN8ZQcIiYPbblcwYcAbqCXXQFuujwdxR7ojv8HXjsnv34Z5Vt23ftQQ5cFGIrVqubvt6fo6Hh/Vy48LPE45cBCEfe4NVXP0h317+ks/OfHcpf7BarFX9NHf6auh23Mw2DRCRMfHmJxbk5pqcmmZtfYCkSIZLOkN3imJJpgSVLiiUmOcckAHZtod7Mh+1600+dDuDTLoLJGpgCpiCWXWIo+TpzyVHmUmNY3Da8VTX4qmuKs6eshu/V23xVNdhdVzfwSYiDZpomk5OTXLhwgfPnzxMOb13i5XAoBgaq6eiwEQjESaW/TTz++7z8ytieg7TL2YLX14fX04u3GKZ79hSky2GzeWlseB+NDe/DMNIsL7/I/MLThEJfJ5td+/GQzS4zM/M5ZmY+h9Xqo67uCerrn6K25jFsNi8ej4cnnniCU6dO8dprr/Hiiy+SSOSX0s5oG2dz3VxxH+eBBx7gWP8RYqFJIrPDpBfHITyJIzaJNzVLTW6eenYvvwiQIGCOQnI030sZAobX7s9qK/OqhmVHEwlXE1l/K5aqDtx1nQSbuqlr68Hrr9pze6WSceYnBlmeGiQ1fwW9NIozOk4wNUWjMUuNSlFO33oWmLVZmbbZiqcJm4Mxu5tZu5Uli4G5y4HDmE4RS08yPD/Jc3Mv0rTk5OSEj5PeHLX9Yfq+d+up9az2ehoaP0hH60fxeXs2b7APtM734K8fVLhVXXRNqoa+cB+tiVbUhtEBKWuKK/4rDAeGyVgzuKwu+gP9Jb3Qq5erXFUH8K+8OlKDfQs5DPWNlWKaGUbH/jejo79TUsvo8w1wbOA3CARuu+rnPsztFIvFmJmZYWpqksnxCaZnZkgky6tJc2k7dWaAOu2n3gxQZwbwFnpUTG2ynJ5lNjXKXHKUxdQU5jbLVjvcHnzVNWSVhYF77qPl6HFa+gfwBIIV+3febA7ze+owuR7tZJom4+PjnD9/ngsXLhCNrg0iU8rA4wnj8YTx+6PUN+Rwu1cwjBnY47LtLldrsRe6WOLh6anYirQbldtWppljJXyahfmnWVj4KunM1rOfWCxOamseo77+Kerq3ondnv97zmQyvP7667zwwgslbQfgdDq57777OHXq1JY16OlUgtD0CMszwyQXxsgtjWONTuJOzBDMztFgzFdkpdMVfCxaG4g4m8h4W9DBNuw1Hfgaurl48QKtAQvZ0Ai28BjexCS12Rnq9VJxysW9MrViQdWw6Ggh7mkjF+zEXtuNr7mXuvaj1Da0Fgd/5swcC4kFpmJTzMRnSs9jM0zHp8mZOawGdE97eDhpp7MtQfDI1lPrpU14M2nldNzGlbQFjcJj89Diy8+CsjobyvrLNa6aXScF2O39FE6HiwF6/TR349Ht54tGQ2uilb5w36aVRQESrgS59hyNPY10V6/N1nE1ddH7qeKDHJVSPwb8INABlExVoLU+cjU7WQkSsMt3M37Jx2KXuHDx54hE3lp3q4XOjp+gu/tTWK1772290dopEokUF8RZPa32OO3Gox350G36qdf50O3GQdbMsJCaYC45ymxylEh25zpEgOrmVlr6j9FydIDWo8epaWm7rjMM3EhutPfUQalUOxmGwdjYWDFUJ5Nh3O4IHu9KIVDnz93uGGqPIcvlai/WRq+GaY+nB5ttc8C8nq6mrbQ2iUTeYn7haRbmnyaZGt9yO6VsVFefoqH+Kerq343TUUcul+PNN9/k+eefZ2VlpWR7u93OPffcw0MPPUQgENjyObfcn0IpxuL0MLG59b3gU/jT+V7w2qtckOdaxbSbOVsTEVcraV87qqYbd0MP1W19NLT34XRVphwhshTi5a/+HkvzXyLYGcLh31yzrzVcSlk4nbBxJmklo/c2lsZlddHsa94yfLf4Wqhz1/Hst5/l/ofvZyI6sWmqu/HIOCvplbJfz2ba6Ix20hfpw5vb/HfR0tnC4488Tn9v/w2xavFGFQ3YSql/B/wc8L+Bfw38T6AXeAz4b1rrX7223b16ErDLd7N+yWttMDHxx1wZ/i1Mc22KIre7i2MDv0519f17er4bvZ201oTDYaampkpCdzq9c73kKp925QO3GaBO58M35FjITjEducxMfJiUsfty8i6vj+b+AVr6j9F69BhNPf23bHnJjf6e2i/X0k65XI4rVy4wOPgsM7NvYLfN58O0N4zLFS1n4p51FO5CkC7tle7Bat3fOs90Ik5oYpzFiTFChVN4fg5td3Di1MO0Dpygpe8oDvfe9ktrTSx2kYWFp5lfeLpYcreZoip4L/UNT1Ff9x4cjibOnj3Lc889RyhU+sPbarVy11138fDDD5fMUXwtUokYC1NXWJkZJRUaJbc8ji06hTs5Q1VmlgYztGlu6HIYWjFnqWfJ0ULC04ZR1Ymj7gj+5j7q2/upqm28rh0Ek5de4+zp/5es7TU8DVv3ADsdnbS1/QCOqkdZzJpMxfO93qu94NOxaaZiU9c8FaHdYseFi6h5dQshBZ3BfAmHo4vq+WqSI0mMbGkpldVq5bbbbuPUqVM0NjZe0/4etEoH7MvAz2utP6+UigJ3aK2HlVL/GejQWv/kte/y1ZGAXb6b/Us+kRjj4sWfZ3nl5ZLbW1t/mN6ef1d2zePN2E6mabK8vLyppzubLe/wbMB0F8N2vRmgrqqGRUIob4aJ2QuMjbxFLrfzohDKYqGh6wgtR48VQvdx/LU716TfLG7G99T1UG47GUaCePwKkchFpqe/w9LyOQxjAqfzKoK0u32tNtrTW6iX7sFqdV/tP+Oq5DIZFqcmSoJ0aHyM6OLCro9d/dtqPXqc1mMnaD16HG/V3gJuIjHC/PzTLCw8TST69rbb+f235Xu2697N2FiaZ599lrkNi+4opbj99tt59NFHqau7vn/jpmGwtDDF0vQw0dkRsktjEJ7EGZ/Cn57DMDVxbztpfyeWmi48jUeoaeunoa133wd257JJzr7ye0xNfh5H7fSWU+sp7aGp8YO0dXwUv/+2XXt4tdaE0+GSqQc3Xo5mri44r+eyukrnig4WaqP9nSSXk7z44oucO3cO0ywtrXK73dx7773cf//9u84Pf6OodMBOAANa63Gl1DzwHq31m0qpXuBVrfX1nWdnBxKwy3crfMlrrZme+SyDg/+1ZBS909nEwNFfpa7uiV2f41ZoJ8iH7lAoVBK4Z2dnN6ycpbHZMtjtKRyOFHZ7Eocjid2ewmMFZ7Sd6pUBWhLHcAfcJGwxFuPTTM5eYH5ldNeebn9t/brAfYz6zm4s1us3d/dBuVXeU9dqYzvlcnESiSvFeaRjhXmkU6kpypxFtkDhdncUa6NXyzs8np6rKiO7FqZhsDw7XRqkJ8ZZmZlG673Vfe+kurmF1oETxdBd1dhc9uH4VGqahYWvMr/wNCsrp9murb3ePurrniKROMrLL08yNTW9aZsTJ07w6KOP0tS0eXGT/XDQf3taa0Jzr3Dhzd8hpU9jdW7u1NCmwmO/m56BH6e+/gkslsouGBbJRPL13luE7+nYdLH8w6ZstPnbSoL0dnXRpmkyODjISy+9xOjo6KbXrKmp4dSpU9xxxx04HPu3ANp+qHTAHga+T2v9ulLqNPBprfX/Ukq9F/hzrfXm6vV9IgG7fAf9QbOfUulZLl36RUKhr5fc3tT4PfT1/Sccju1/E97s7aS1JpeLkMmE1p0WyGRCpNILxGLTJJNzZLOLKKIoy+6zJuRyNtLRRqzhToIrR2lauQ1XpgbToUlaYizGp5hdGmYlM08ku4ixzdzANqeT5t6jtBZCd3P/AC7v9Rkktp9u9vfUtTLNDLH4ZV577W9pb7cVA3UqNbmn59FaAfX4fP3U1d6Gz9dfCNJH9j1Ia62JLi4QGl8fpMdYmprAKPPIEYDFaqOmtY269s78qaOTYH0jzz79FQIWzfTFcyxMjOULdXfgrarOh+2B47QOnCj7x2w6EyK08HUWFp5mafmlbRfIcbnacTgeYPCyn8uXs7BhpoijR4/y6KOP0tbWVva/vRIO6m8vlZ5l+NIfMz31eZRzacttzGQDra0/QN+JT2C3V6ak5moksgm+8u2v8N3v/G7su8zClc1meeutt3j55Zc3lQgBdHZ2curUKfr7+7HcpGNwrnmhmQ2+CXwQeB34A+C/K6V+ALgb+OxV76UQ14nL2cTtt/0u8/Nf4tLlXyabzX/Azc79HYtLz3G0/xdpaPjADTnAYis7heZMZpFMJkR63XWtdy7nANjLZ6PNlsNWPQXVU6R5kTEgmwygIu34V/qpXTlGm/1dWEwHGk1SxVlKTLOcmmUls8BKZp5ELkIunWbi3NtMnCscolaK2tZ2Wo8eL/R0D1DV1HLT/L/dilbDdDRyhkj0LNHoWWKxy8X35PjW4+5KaK1IJv0kEkFMs4m6uts50v0InZ33Y7Ptf51/IhImND5aEqQXJ8bIlDnDDwBKUdXYVAjRXcVAXdXUsuU80DV9A8XgmIrFmB68wNSFc0xdOs/s0GWMXOmP2PjKMpdfeYHLr7wAgN3lpqV/gNaB47QNnKCptx+7c3PbOR11tLZ+jNbWj5HNRggtfpOFhadZXHy2ZMxLKjVBKjVBYxO0tNYSiRxh+EqQlZVGwMKlS5e4dOkSR44c4bHHHqOrq6v8trlBGEaC+bmnGR76Y5LZMygFakMVSjbhwMUDnLz3U9Q13XUwO7qBx+6hxlazY7iOxWKcPn2a06dPbxpEr5TixIkTnDp1itbW1uu9uzeMHQO2UupJrfU3gE9Cfn1mrfXvKqWWgYeBvyY/8FGIQ0cpRWPjd1Fd/RCDg7/G7NzfApDNLnH23Keom/t7Bo7+Ck7n4RxwkQ/N0XVB+dpD89WwWr04HLU4HPU4HHU4HHVYrdVcvjyEyzlHzhjEZtu8KIDdHQH3OZKN55jkb9CmBR1twRvuIbhylIZwL63JvuKcqFmdYSU9T7gQuPPnCyxOjrM4Oc7b3/gKAO5AsFhS0tJ/jMYjvdhuskOQN4vVFQ6j0bNbhundFIN0vIpEIkgiESQer8Jj76C/7Ri3d/XSUF2HKhy6NsZTGCq91nmqFCjWfpCptduAtZVNN9yOUmu13Ku3K0U2nWRpZoql6UmWpify51OTJCIr6NUyCr16SWNVtsJNmvz6dPl7vDU1m4J0TWvblgG3HC6fjyN33ceRu+4D8vXcs8ODxcA9fekC6URpuVY2lWTs7TcYezu/+qDFaqPxSE++rGTgBK1Hj+H2l84IYrcHaG76EM1NH8IwEiwuPsfCwtMshL5RUpJnGIt4vYvcdjuYppv5+RYWQx0sLzczPDzM8PAwHR0dPPbYY/T09NzQP5i1NlleeYXpyc8zN/+PUFh8Z/0/ycgq0vMttLZ/lNve8xM31KJe8/PzvPTSS7z99tsYRunRTKfTyd13380DDzxAVVXVwezgIbZjiYhSyiS/8OkfAH+otd5cYHXApESkfLf6YepQ6FtcvPSfSKdni7fZbH56e3+OluYfKH7IX8922jo0rwbngw3NqyfnhuvbzZqw2k5aayKRUYaHv878wqtkMhdxOuewWstYkCPjwRnuwb/ShyvcgyvSjTVX+nqx7EoxbIez+fAdyy4Xw4rVZqPhSG++l7swa8leB3hdb7fC355pponFLhfC9Jl1Ybq8cohsMkA4VkU8UUUing/TyWQArfNlDNWmj26jnm6zgWp945cNAesCPWuJbDXcq8Idau3+4u0WRcyeofHODpzdQRwdfizO7fvLTNNgcWKcyYvnmLp4nqmL54gtbb0c/Hq1bR0lAycD9Q1bhmHTTLO0/FJ+ru3Q14tHDDfK5WwsL7USCnWytNSCadppaWnhscceu24lBdfrby+RGGFm5gtMT/81mezmucW1htiUF4dxDyfv+xe0n7j3UP+QWN9OWmtGRkZ48cUXGRoa2rRtMBjkwQcf5K677sJ1C84MVZEabKXUMeDHgR8BaoGvAv8H+Hu91+WsrhMJ2OW7Fb7kd5PLRRm68n8xNfXnJbdXV5/i2MB/xe3u2HM7lROa14Lz4Q7Ne7FTO62sLHLlyreZnX2JRPIcLtc0Hk8ZI9m1wh5vwh3uxb3Sgyt8BGe8FaVLa0VzZpZINpQP3avhOzNP2swfkq9qbM6H7UJpSV1bx4HOyX2z/e3lw/QlItGzRFbOEA2fIZ4aRFPedGnZZIBorIZwrJpYtIZYrJZcbnOvXq3pp9tooMusp0rv7zzTNxQL2Jt9OLsCOLqCOLsCWP3bH9XRWhNZmGeqELgnL55jaWpi15fx1dbRejRfUtI6cJy69s5Nf1emmSMcfi0/1/bCV0s6NEq3s7C81EJosYOlxTZqatp59NFHOXHiREWDdiX/9rLZFebmv8zMzF8Tiby55TapZQeR0Xqamj/E3e/6YYINh/MI6UbPPPMMjzzyCGfPnuWll17aNDMMQGtrK6dOneLYsWNYb8LB6OWq9CBHG/ka7H8KPAUsAn9MfrDjgaZbCdjlu9m+5K/F8vIrXLj4cySTY8XbLBYXPUf+LVeudPGOdzxxA4Xm+kJo3t9pxcp9P2mtWVxc5Mrw20xNPU8sdga3exa/P4Tdvnu7qZwTV6QLd7gH10oP7nAPtkzVltsmc7Fi2F7J5sN3JLOI3eOiue9occaS5r6jOFz711436t+e1ppcLEFk/iyRpbeIxs4Tz10gaRlFlzn3cC4ZJBqtZSUWJBarJRar2TJMr2p01tDrbeOIr42gzZfvClz9mtI6P5Zv9Xtr9bJevamw7bptVrc3sjmMTJpcJkMuk8HIZPN1yloDqliqpArdxIVr+Utq9RaF1WrDYrVisVixWK0oZcmXqGx8fdbtl14rH1m7//qy1blxdAVwdgVwdgWx1rp27EFNRMJMX7rA1KXzTF04x9zIEKaxcz+a0+stjI/Ih+7Gnj5s9rU6Xq1NItEzLMw/zfzCV0o+b9czTUU43EQo1IFp3MapU+/l9ttvr0iIu9a/PdPMsrj0LDMzXyAU+saWR2RyKSvLQwFyS72cOPUjnHj8yX39fLlaWmtSqRQrKyt89atfZWFhgVhsc7nfwMAAp06doqOj41D3wu+Xiq/kuO6JW4BPAD8GHAFe0Fo/djU7WQkSsMt3o37JXy+GkWJk5H8wNv77lC6L7EWp7C0Zmvfiat9PpmkyOzvLlStDTEy8RjT6Fh7vPAF/CK9vuayV9WzJWtzhI/myknAPrkgnFnPrHjtTm0Szi8XBlOHMAuHsIr62OlqO5nu5W/uP4a+rv25fHof1b09rjRnLkltOYaykyS5FiEYuEcucJ64vkXRcIe2dhDJmkgEwklXEorUsxYLEYjW7hmmnw0lrawstra20trYyPj7OU089dU3/pkQkvDYF3roZPDLJ8lY3BfIDDhuaqF2duaO9g7qOLqqbW7Dadp5lYS+03vyDYO1HBGz8wVDcJG3w5tOv0ONpJTMaITsb3zW0W/x2nF1BHJ0BnN1B7E1elHX793s2nWJm8DJTl/K93NOXL5JN7Txo02q309TTXxw4uX4WIK018fjlQs/208RiF7dpE4hE6onHjtLf/wPcffd7sNuvvs2vbsVLTTR2jpmZLzA39/dblrxoA8LjPpYvB6kKPsI97/sIXXfcfahWrzVNk3g8zsrKCuFweMvzTGbr7zm7zc6dt93Ogw89RG39gU0Udyhdt4BdePIq4OPALwFVWusDO1YgAbt8h/VL/qBFIm9z4cLPEotf/fvoZgnNe1Gp91M2m2VycpLh4WFGRvID4nz+BQL+EP5ACKezjGCkrbii7bhW8oHbHT6CPdFY7IfcSsZIrQ2mzC6QcaXxdTfQfOwoLf3HqO86suXsDVfjoP72tKkxohmM5RTGcprcSuF8OUU2HCOeHSLtHSYVGCUVGCPt20OYTlSTiNWyGAsSjVXvGqatVitNTU20FsJ0a2srNTU1JeUAe2mnTDLB4uTEukVZ8rN4JMIrZT1+la+6Zl2Qzp9q2zoO/aqj69vKTOZIj0fIjERIj4bJTEYht/N3u3JacXT486G7K4Cj3Y/Fsf1XuWkYLIyNMHXxXLGWe9e2Vor6jq7i1ICtA8fx1+QXnkkkRgpzbX9123KL/HZ1BALv4I7bP0FV1bGdX28Le3lPpdNzzM7+HTOzXyAeH9x6f+ZdLF0OEp2oZ+CB93DXez9IbVv7nverEgzDIBKJbBugw+HwpoGJu/FoB8dz7QwYrbjI/7BRdgvKZcPismJx2VCr5878ucVlLd6vnJu3s7isYLPcNL3f1yVgK6XeRb5M5ENACvgM8Pta6zeucj+vmQTs8knA3p5pZhgb+z1Gx34H08z/or8VQ/NeXK/3UyqVYnR0tBC4RwiHR/EHQvj9IQKBED7fYlkDKK2GH3esB+dCd768JNyNNbd7HW80u0w4M0/EWEbV2fEdqafxtj6ajx7D7bu6lciuV1tpQ2OE0xgrKXLLaYzlwvnq9XAaDI1pyZD2TZIOjBbC9Chp31TZYdpM1ZCM17MUDbASCe4apgHq6+tLwnRDQwO2XX6wbNVOuWyW5enJkinwQuNjRBY214juxOn1UtfeVRqkOzqv+v/0oO30ntJZk8xUlPRohMxohPRoBJ3apaTHqnC0+gplJfmebqt3+55jrTUrs9PFGu7pS+dZntl9HoRgQ+O6gZMnqGltI52eZWHhq8zO/SORyGts3x3fTFvbd9Pc/AH8vhNlBbbd/vYMI8nCwteYmf0CS0svUHo0My8Ts7E8GGTpchCHtY07n/oAtz351HV/72QymW17nsPhMNFolKvpJF3Ppi34tAu/dtNtNNJjNmLlOvTCWxUWp7U0qDs3B/Gtg3rhNod1beafA1SxgK2U6iBfDvIJoBN4Fvh94PNa69QOD90XErDLJwF7d4aR4tln/57HHvuuWzI078V+vZ8ikQgjIyOMjIwwPDxMNLqCx7OCPxAi4F/AHwjh8UTKei6n0Y471otzvhNXqBtnrG3TAMqt5MwM4UyIpD2Btd6Jr6eBhrv6qOkurybxattK50yMlXSxhCO3vNYDbawUAvSGj/DVMJ0KjBYD9V7CNLk6Uqkmllf8LCx5iUVrMYydp0EMBoO0tLQUw3Rzc/OeZhcwDYNcNsM3v/IVeloai6sbLk6MsTwztWst8Ho2h5Pato610o5CkPZV1940PWiwt/eUNjW5+QTpkXAhdIcxwruXwNkaPPmBk92FgZNVzh3bML6yXKzhnrp0nvmR4V1Xp3T5A4WBk/le7qrWahZCX+fK8OfJ5c5gsWz9eIejhcbG99JQ/xTB4N3FqRo32qqdtDZZWTnNzOzfMD//jyVTDK4ysorwSICly0Fi0x5a+k9wz/s/SO99pyqy2qzWmmQyuWOA3jjf9NVwahs+7canXSUnf+HcqezY6z2sqAR1VTWYaQOdymGmDMxUDp029mXMQFkUKId1XRBfDePrgriz9P5N4d1pRdmu7QdERRaaUUp9DXgCmCc/qPEPtNab52wR4iZhtbpQql7C9SESCAS44447uOOOO4oDJlfD9sjICKlUCpstjc+/WCgrWdh2AGXaOkE6OAFBoA8syoVH9+OJ9WKfbccx3YkjtXmVT5vFQa2rJX9lKX9Knh5nyDhP2pXGWu/E39NI7e3duFqCKGt5H+BmxsgH5Y09z8spcitpzGhmxy8305Ih7b/6MK1UI5lMM8tLfubmXESjNbuGabvNRrXPS5XHTcDlxOewYUOTS0dJXTrD5bPf4Xwmg5HNkstmMLIZcpnVy4XzTIZcLpc/z2bQ5lqIOlfWnoPFaqW6ubWkN7quvZNgQyMWy607w8FWlEVhb/Jib/LiO9WC1hpjJZ0P24XQnZvfHOZy8wly8wnir+ZnArEGHcVZSpzdQWwNnpIeRW9VNf0PPEz/Aw8D+TKe6cFL+dlKLpxjZugyuUy65DVS0QhXXnuZK6+9DOR/IDX3HaV14Htp7f1XzMdfZW7hq/j9oyVHrjKZaSYmPs3ExKdxOOqpr3839fVPUV31AJZtFkxJJEaYmf1bZmf/dttVQqNTHpYuBwmP+MF0MvDQo9z9U99D45HePbR4vv45FovtGKC3q3/eC492bhuevdqFY13MU3YL9hYv9mYv9hYfjhYftkYPFoeVi888w8A7Tm56fq01OmNgptaCdzGAp3Po1SBeOC/evyGoY1Qgpev8+AMjbUAZPxC3ZbNsLmNZ17u+FtTXlcOsL3cp92V2uT8JfAT40mGZlk8IcetSSlFXV0ddXR333XdfccDk6uIV4+Pj5MZzgMblihZ6ufOh2+tdxmIp/ZA3dYoYbxPzvQ29QC84bI34OIYj0o11sgnHVBuO7NalJW6rD3fWB9PAdJzF585iYpJ1ZrDWu/D1NuLrrsMzD7EXp0tqoI3lNGa8/CWziz3TwRHS/tUwPV12mM4kvMQj1USiNYQT9cRidbuGaUwTSyqBNRnHmopjTcZR2TRJ8l8OM2Xv/bUJrq5w2N5ZrJeubm4tmbFClE8pha3aha3ahfeuBgCMeJbMWKGGezRCZjIGZunfixHOkHxrgeRbC/nncdkKUwPmZytxtPlLegcdbg9dt99F1+35FQuNXJa54Sv5Xu5CHXcqVjp9Zy5Tupqrslio7zpOpu3dxB3DeHyD1NZOYrOt/e1kMgtMTf0FU1N/gc0WpL7uSerrn6Km5hG0jjM59RfMzv4N4fDrW7ZHasXB0uUgy4NBsjE7nmAV93/wfdzx7vfhq978gxsgl8sRiUS2DdCRSGTP9c8bWZQFv9OLDxferANv2lEMz6sBertyDovXjr3Fi6PFVwjVPmx17j2XWCil8uUaThsEr36BHJ0zSwP4TkE9vfV2OlOhGJozMWMmZqz8z9+rcVWDHA8TKREpn5SIlEfaqTyHsZ3WD5gcHh5menq6WKNoseTw+ZbwFwZP+v0hXK74Ls8ISlnxuo/iYQDLfBN6rB7bbDXunK+4Ul+l5XumJ0j6R4j7h0gHRjH887DN4fKNUhEXsXB+WrxIqolYon73MK01lnQSayqOJZkP05Z0CrUPx4eVsuRX47TZaOnpo669oxika9s6bogpz/bb9f77MzMGmYlooYY7TGYsunvAsSkcbYWBk90BnB0BLO7t/0a0abI0PVms4566eH7H2noN5PzVmC2teGqXqKsbp7Z2Aodj62pVq9WDYWRgi/nZcykLy1eCLF8Okph3AYr6riPc/b4PMvDQY5iw4+wb0WgZ8/rvwm63UxWsIuD24bO48eWceOI23CsKb8qOGyeWHQZqr7LVuQu90oWe6WYfFr99TyVRh/HzfCNtanR6u57y1dtWg3r+vvW966uP3aLMfk/af/Oxay8REUKIG4ndbqe7u5vu7m6efPLJTQMmFxZsRCINMFXY3pHI93AXQ/ciVmvpl7HWBrHEeWKcBx9wAmx3BAn4b8eW7sSYqsa4UoVlwYnH9OO1B8veX1MbxM0QEe8QSf8w2eAEunoOSzCCspQXbFMRJ7GVfJiOJpuIphp2D9OAyqSLvdK2TAqnmcNms2GzO7C5HVgDPqw2OzaHHavdgc1ux2Z3YHXkL1vtDmzrL9vthfscWO32dbc7sDryj920TeHyak3rjfAlf6uwOKy4eqpw9VQB+QG12dk46ZFwvqd7JLy5BzCn873foxF4BlBgb/IWB046uwJY1/WCKouF2rYOats6uP1d7wUguhgqzFRynumL51iYGCtOYagAe3QZfWmZtC/IcN1tDLnvJxBcoK52nNq6cVyutVIXwygte9GmIjLuZfFykPBkFabVhbY7qbmjj+rOI+SsNl64dIUvvfIdksmdpyQsh9vtpqqqimAwSFVVFQGvH5/hwpu04QpbsM5lyM0kyy+fsCnsjet7pfOnnVbxvJkoi0K5bTv+aNuN1hqdNdeVtKz1mm8qc1kX4tcH9XLdGv8rQohbksvlYmBggIGBAWDzgMlIBBYXO1hc7Cg8wsTrXSkG7qqqZVyuzXPg5nJhlpafA56DauBe8Hi6sbiPk4g0kBp2kx5yYFlW+Gw1mDpHPBsmrpdI+yYwqqexVC/iqIviqk6zOj5rt8rtXCpIOlFHPFbLcrSGxZVgWWHa5XTSUFdLU0MDzc3NtLa04q8KFkKwXWqWxa7U6iwjrT54pDVfx72YIj0aLs5WkgttCKUasjNxsjNx4i/lC4qsNa51ZSVBbPXukp5Wf20dAw8/zsDDjwOQisWYHrxQHDg5O3QZI5fDFgtjjYUxvAHitc1EwvcxPHwvPt9ivme7bry4emx8yc/CVCvzc51ktR/T4YDetff8VMZkanDvw8v8fn9JgF5/7lduWMySnYqRnYmRORPHWEwBidWm2XHdU+W24SiUdqyWetjq3WWP7xBbU0qhHFZwWLEGdv/s3NJ/Lm8zCdhCiFtGOQMm4/Ea4vEaZmf7AbBaM8XAXV8fxeOZR6nNsw4kEiMkEiP5K+1g6XTi9p3AQhcLc6P4gis4M2M4Ka+O0OHoADpIxOuZn3czMQGZzO5B2G6350P0uinyqqqqbqoZNMTBU0phq3Njq3PjvbcJACOaKc5Skh6NkJ2ObRqkayylSCylSLw+D4DFa8PRuTZw0t7iLQmRLp+PI3fdx5G77gMgl8kwOzxYDNzTly5gG79Ezu0jU9dMjDpisTpGR+/C5YqhtSKdzi92wx7ylMVi2TI4r54HAgFsNlt+hpaFBNmZOJnpONm3Y2SnQyzGy+/ptFY7sTf78oG60DttDe48Y4s4/CRgCyFuSVsNmJyZmSkG7vHx8fxMF4aDlZUWVlZamBiH/ADKGMHgEs0tSfz+EDDBxv4o00wTibwOvI7NDaltB70rXK5OFJ0kEvUshDyMj0MysfthY6UUjY2NJVPk1dfXV2SJaSH2yup34LmtDs9t+cVkzHSOzHg0X1YyGiEzEUVnSwtgzXiO1PlFUucXgfxMF44Of3G2EkdHAItz7f1sczhoGzhB28CJ/ONNg8WJ8WIN98jQIGGHh5y/mlRq+3mqHQ7HjgHa5/OVLIIE+br07Gyc7OU40ekRMjNxcrPxTf+mbVkU9kZPcRYPe7MXR7MXi0cG696MJGALIQT5HqvVkPrII4/sMGBSkUr5SaX8zBXGYyllUFMTpb3DoKpqCYtljGx2qzk2FG5XFxZLF8lUI4shL+PjEImsT9/bB+vq6uqSnummpiYcjqs8zCnEdWZx2nD1VePqqwbyM0lkpmPFxW8yo2HMxIYxD1mT9JUw6SthogAWsLf4cBaWeHd0BrD6197zFouV+s5u6ju7ueup70JrTWRhnnOvvcrb584TTSbpO36Cprb2kgDtdrt37CE2YhlS03Ey07F8mct0LF8CU2a5tHJa8wF63Swe9kbPNc/BLG4cErCFEGILGwdMJpNJxsbG1g2YXChuq7WVxcUqFhcBaoE+qqosdB+Burok83NhLJYjTE4qFhbWzz6w/VyuXq+3JEy3tLTg8Xiu1z9XiOtO2Sw4O/Kzi/gfKyyAE0oWe7jTo2GM5dJ5sjEhOxkjOxkj9kJ+pUhbnbt04GStqxiWlVIEGxp56P3fzUPv/+5dB85qU5NbSpFdF6Qz0/H8HPRlsgYcxdKO1VIPa7XrUKw6KA6OBGwhhCiD2+3ecsDkauCOREpXk1xZMXnjdQAn0ABsrtteZbfbS8o8WltbCQaDUoMpbmrKorA3eLA3eOCBZgBy4XSxhjszEiE7F9/Ua5wLJcmFkiReyx9Csvjt+akBC6Hb3uzdMtzqrEl2Lk52Ok5mJkZ2Oj8As+z5lRXY6j1r80sXZvGw+uQokthMArYQQlyFrQZMrobt1QGTW7FYLDQ2NpaE6bq6uk31nkLcimxBJ7Y7GvDckV8Ax0zmSI+tDZzMTEQ3TWtnRrMkz4RIngkB+fIMR0d+Pu6qccXS3CUy0zFyC4my50BWdktprXSLD3uTB2WX8Q2iPBKwhRDiGq0fMHn//feXDJicmJhgeXmZu+++u1g3bZcVCIUoi8Vtwz1Qg3sgv6KizppkpqKkRwqheyyCTpX2QOu0QXpwhfTgCnVYSDC/82v47IUFWtZm8bDV7n3VQyHWk4AthBAVtn7AJOQXUHnwwQcPeK+EuPEpu6VQex0E2tGmJjuXWFdWEsaIbF8/batzrw06bPHiaPZd/XzIQuxAArYQQgghbkjKonAUprvznWrJL4CznC4s7x5hemqaznv68j3TTZ5bZtVDcfDknSaEEEKIm4JSCluNC1uNC+/djbz1zBQnTrUc9G6JW5CMqhFCCCGEEKKCJGALIYQQQghRQRKwhRBCCCGEqCAJ2EIIIYQQQlTQvgZspdR7lVKXlFJDSqmf3eL+71FKva2UelMp9ZpS6pH93D8hhBBCCCGu1b7NIqKUsgK/A7wbmAROK6W+qLU+v26zbwBf1FprpdTtwGeBgf3aRyGEEEIIIa7VfvZg3w8Maa2HtdYZ4C+B71m/gdY6prVeXQPVC2iEEEIIIYS4gexnwG4FJtZdnyzcVkIp9WGl1EXgS8A/3ad9E0IIIYQQoiLUWofxdX4hpb4feEpr/ROF6x8H7tda//Q22z8G/ILW+l1b3PdJ4JMA9fX193z2s5+9fjt+E4nFYvh8voPejUNP2qk80k7lk7Yqj7RT+aStyiPtVB5pp/I98cQT39Fa37vbdvu5kuMk0L7uehswvd3GWutnlVI9Sqk6rXVow32/B/wewNGjR/U73vGO67C7N59nnnkGaavdSTuVR9qpfNJW5ZF2Kp+0VXmkncoj7VR5+1kichroU0p1K6UcwMeAL67fQCnVq5RShct3Aw5gcR/3UQghhBBCiGuybz3YWuucUuqngKcBK/BprfU5pdQ/L9z/u8D3Av9EKZUFksBH9X7VsAghhBBCCFEB+1kigtb6y8CXN9z2u+su/ybwm/u5T0IIIYQQQlSSrOQohBBCCCFEBUnAFkIIIYQQooIkYAshhBBCCFFBErCFEEIIIYSoIAnYQgghhBBCVJAEbCGEEEIIISpIArYQQgghhBAVJAFbCCGEEEKICpKALYQQQgghRAVJwBZCCCGEEKKCJGALIYQQQghRQRKwhRBCCCGEqCAJ2EIIIYQQQlSQBGwhhBBCCCEqSAK2EEIIIYQQFSQBWwghhBBCiAqSgC2EEEIIIUQFScAWQgghhBCigiRgCyGEEEIIUUESsIUQQgghhKggCdhCCCGEEEJUkARsIYQQQgghKkgCthBCCCGEEBUkAVsIIYQQQogKkoAthBBCCCFEBUnAFkIIIYQQooIkYAshhBBCCFFBErCFEEIIIYSoIAnYQgghhBBCVJAEbCGEEEIIISpIArYQQgghhBAVJAFbCCGEEEKICpKALYQQQgghRAVJwBZCCCGEEKKCJGALIYQQQghRQRKwhRBCCCGEqCAJ2EIIIYQQQlSQBGwhhBBCCCEqSAK2EEIIIYQQFSQBWwghhBBCiAqSgC2EEEIIIUQFScAWQgghhBCigiRgCyGEEEIIUUESsIUQQgghhKggCdhCCCGEEEJUkARsIYQQQgghKkgCthBCCCGEEBUkAVsIIYQQQogKkoAthBBCCCFEBUnAFkIIIYQQooIkYAshhBBCCFFBErCFEEIIIYSoIAnYQgghhBBCVJAEbCGEEEIIISpIArYQQgghhBAVJAFbCCGEEEKICpKALYQQQgghRAVJwBZCCCGEEKKCJGALIYQQQghRQRKwhRBCCCGEqCAJ2EIIIYQQQlSQBGwhhBBCCCEqSAK2EEIIIYQQFSQBWwghhBBCiAra14CtlHqvUuqSUmpIKfWzW9z/w0qptwunF5VSd+zn/gkhhBBCCHGt9i1gK6WswO8A7wOOAz+olDq+YbMR4HGt9e3AfwF+b7/2TwghhBBCiErYzx7s+4EhrfWw1joD/CXwPes30Fq/qLVeLlx9GWjbx/0TQgghhBDimimt9f68kFLfB7xXa/0ThesfBx7QWv/UNtv/DDCwuv2G+z4JfBKgvr7+ns9+9rPXb8dvIrFYDJ/Pd9C7cehJO5VH2ql80lblkXYqn7RVeaSdyiPtVL4nnnjiO1rre3fbzrYfO1Ogtrhty3SvlHoC+HHgka3u11r/HoXykaNHj+p3vOMdFdrFm9szzzyDtNXupJ3KI+1UPmmr8kg7lU/aqjzSTuWRdqq8/QzYk0D7uuttwPTGjZRStwO/D7xPa724T/smhBBCCCFERexnDfZpoE8p1a2UcgAfA764fgOlVAfwBeDjWuvL+7hvQgghhBBCVMS+9WBrrXNKqZ8CngaswKe11ueUUv+8cP/vAr8A1AL/UykFkCunzkUIIYQQQojDYj9LRNBafxn48obbfnfd5Z8ANg1qFEIIIYQQ4kYhKzkKIYQQQghRQRKwhRBCCCGEqCAJ2EIIIYQQQlSQBGwhhBBCCCEqSAK2EEIIIYQQFSQBWwghhBBCiAqSgC2EEEIIIUQFScAWQgghhBCigiRgCyGEEEIIUUESsIUQQgghhKggCdhCCCGEEEJUkARsIYQQQgghKkgCthBCCCGEEBUkAVsIIYQQQogKkoAthBBCCCFEBUnAFkIIIYQQooIkYAshhBBCCFFBErCFEEIIIYSoIAnYQgghhBBCVJAEbCGEEEIIISpIArYQQgghhBAVJAFbCCGEEEKICpKALYQQQgghRAVJwBZCCCGEEKKCJGALIYQQQghRQRKwhRBCCCGEqCAJ2EIIIYQQQlSQBGwhhBBCCCEqSAK2EEIIIYQQFSQBWwghhBBCiAqSgC2EEEIIIUQFScAWQgghhBCigiRgCyGEEEIIUUESsIUQQgghhKggCdhCCCGEEEJUkARsIYQQQgghKkgCthBCCCGEEBUkAVsIIYQQQogKkoAthBBCCCFEBUnAFkIIIYQQooIkYAshhBBCCFFBErCFEEIIIYSoIAnYQgghhBBCVJAEbCGEEEIIISpIArYQQgghhBAVJAFbCCGEEEKICpKALYQQQgghRAVJwBZCCCGEEKKCJGALIYQQQghRQRKwhRBCCCGEqCAJ2EIIIYQQQlSQBGwhhBBCCCEqSAK2EEIIIYQQFSQBWwghhBBCiAqSgC2EEEIIIUQFScAWQgghhBCigiRgCyGEEEIIUUESsIUQQgghhKggCdhCCCGEEEJUkARsIYQQQgghKkgCthBCCCGEEBUkAVsIIYQQQogKkoAthBBCCCFEBe1rwFZKvVcpdUkpNaSU+tkt7h9QSr2klEorpX5mP/dNCCGEEEKISrDt1wsppazA7wDvBiaB00qpL2qtz6/bbAn4/wIf2q/9EkIIIYQQopL2swf7fmBIaz2stc4Afwl8z/oNtNbzWuvTQHYf90sIIYQQQoiK2c+A3QpMrLs+WbhNCCGEEEKIm8a+lYgAaovb9FU9kVKfBD5ZuJpWSp296r26tdQBoYPeiRuAtFN5pJ3KJ21VHmmn8klblUfaqTzSTuU7Ws5G+xmwJ4H2ddfbgOmreSKt9e8BvweglHpNa33vte/ezU/aqjzSTuWRdiqftFV5pJ3KJ21VHmmn8kg7lU8p9Vo52+1nichpoE8p1a2UcgAfA764j68vhBBCCCHEdbdvPdha65xS6qeApwEr8Gmt9Tml1D8v3P+7Sqkm4DUgAJhKqX8FHNdaR/ZrP4UQQgghhLgW+1kigtb6y8CXN9z2u+suz5IvHdmL36vArt0qpK3KI+1UHmmn8klblUfaqXzSVuWRdiqPtFP5ymorpfVVjTMUQgghhBBCbEGWShdCCCGEEKKCbuiAvdvS6yJPKfVppdS8TGe4M6VUu1LqW0qpC0qpc0qpTx30Ph1GSimXUupVpdRbhXb65YPep8NMKWVVSr2hlPqHg96Xw0wpNaqUOqOUerPcUfq3IqVUlVLq80qpi4XPqlMHvU+HkVLqaOG9tHqKFMZ1iQ2UUv+68Fl+Vin1GaWU66D36TBSSn2q0Ebnynkv3bAlIoWl1y+zbul14Ac3LL0uAKXUY0AM+BOt9cmD3p/DSinVDDRrrV9XSvmB7wAfkvdUKaWUArxa65hSyg48D3xKa/3yAe/aoaSU+jfAvUBAa/1dB70/h5VSahS4V2stc/HuQCn1x8BzWuvfL8zI5dFarxzwbh1qhbwwBTygtR476P05TJRSreQ/w49rrZNKqc8CX9Za/9HB7tnhopQ6SX4F8vuBDPAV4P+jtR7c7jE3cg/2rkuvizyt9bPA0kHvx2GntZ7RWr9euBwFLiCrjW6i82KFq/bC6cb8pX6dKaXagA8Av3/Q+yJufEqpAPAY8AcAWuuMhOuyPAlckXC9LRvgVkrZAA9XuUbJTe4Y8LLWOqG1zgHfBj680wNu5IAtS6+L60Yp1QXcBbxywLtyKBXKHt4E5oGvaa2lnbb228C/B8wD3o8bgQa+qpT6TmG1XrHZEWAB+MNC2dHvK6W8B71TN4CPAZ856J04jLTWU8B/A8aBGSCstf7qwe7VoXQWeEwpVauU8gDvp3TxxE1u5IBdsaXXhVhPKeUD/hr4VzIH+9a01obW+k7y02reXzh8JtZRSn0XMK+1/s5B78sN4mGt9d3A+4B/WShtE6VswN3A/9Ja3wXEARl/tINCGc0Hgc8d9L4cRkqpavJH/7uBFsCrlPqRg92rw0drfQH4TeBr5MtD3gJyOz3mRg7YFVt6XYhVhZrivwb+XGv9hYPen8OucHj6GeC9B7snh9LDwAcLtcV/CbxTKfVnB7tLh5fWerpwPg/8DfkyQFFqEphcd8To8+QDt9je+4DXtdZzB70jh9S7gBGt9YLWOgt8AXjogPfpUNJa/4HW+m6t9WPky263rb+GGztgy9LroqIKg/f+ALigtf6/D3p/DiulVL1Sqqpw2U3+A/rige7UIaS1/jmtdZvWuov859M3tdbSM7QFpZS3MLCYQsnDe8gfkhXrFBZjm1BKHS3c9CQgg7B39oNIechOxoEHlVKewnfgk+THH4kNlFINhfMO4CPs8r7a15UcK2m7pdcPeLcOJaXUZ4B3AHVKqUngF7XWf3Cwe3UoPQx8HDhTqC8G+PnCCqRiTTPwx4WR+Rbgs1prmYJOXItG4G/y3+/YgL/QWn/lYHfp0Ppp4M8LHUvDwI8d8P4cWoVa2XcD/+yg9+Ww0lq/opT6PPA6+ZKHN5BVHbfz10qpWiAL/Eut9fJOG9+w0/QJIYQQQghxGN3IJSJCCCGEEEIcOhKwhRBCCCGEqCAJ2EIIIYQQQlSQBGwhhBBCCCEqSAK2EEIIIYQQFSQBWwghxJaUUlop9X0HvR9CCHGjkYAthBCHkFLqjwoBd+Pp5YPeNyGEEDu7YReaEUKIW8DXyS9+tF7mIHZECCFE+aQHWwghDq+01np2w2kJiuUbP6WU+pJSKqGUGlNKlSzFrpS6TSn1daVUUim1VOgVD27Y5keVUmeUUmml1JxS6o827EONUupzSqm4Ump4i9f4hcJrp5VSs0qpP7keDSGEEDcSCdhCCHHj+mXgi8Cd5Jc3/hOl1L1QXCb6K0AMuB/4MPAQ8OnVByul/hnwv4E/BG4H3g+c2/AavwD8HXAH8FfAp5VSnYXHfy/wM8C/APqA7wJerfw/UwghbiyyVLoQQhxChZ7kHwFSG+76Ha31f1BKaeD3tdY/ue4xXwdmtdY/opT6SeC/AW1a62jh/ncA3wL6tNZDSqlJ4M+01j+7zT5o4De01j9XuG4DIsAntdZ/ppT6N8A/A05qrbOV+rcLIcSNTmqwhRDi8HoW+OSG21bWXX5pw30vAR8oXD4GvL0argteBEzguFIqArQC39hlH95evaC1zimlFoCGwk2fAz4FjCilnibfY/5FrXV6l+cUQoibmpSICCHE4ZXQWg9tOIXKfKwCtjtEqQv3l2Njz7Sm8N2htZ4AjpLvxY4AvwV8RynlLfO5hRDipiQBWwghblwPbnH9QuHyeeAOpZR/3f0Pkf/cv6C1ngOmgCevZQe01imt9Ze01v8auA84ATx8Lc8phBA3OikREUKIw8uplGracJuhtV4oXP6IUuo08AzwfeTD8gOF+/6c/CDIP1FK/QJQTX5A4xe01kOFbX4N+O9KqTngS4AHeFJr/Vvl7JxS6hPkv0deIT+Y8qPke7wH9/jvFEKIm4oEbCGEOLzeBcxsuG0KaCtc/iXge4H/H7AA/JjW+jSA1jqhlHoK+G3yM3ukyM8G8qnVJ9Ja/y+lVAb4t8BvAkvAl/ewfyvAfyA/mNJOvtf8I1rrkT08hxBC3HRkFhEhhLgBFWb4+H6t9ecPel+EEEKUkhpsIYQQQgghKkgCthBCCCGEEBUkJSJCCCGEEEJUkPRgCyGEEEIIUUESsIUQQgghhKggCdhCCCGEEEJUkARsIYQQQgghKkgCthBCCCGEEBUkAVsIIYQQQogK+v8DUmYc3Z6AKJwAAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# extra code – visualize the learning curves of all the optimizers\n",
"\n",
"for loss in (\"loss\", \"val_loss\"):\n",
" plt.figure(figsize=(12, 8))\n",
" opt_names = \"SGD Momentum Nesterov AdaGrad RMSProp Adam Adamax Nadam AdamW\"\n",
" for history, opt_name in zip((history_sgd, history_momentum, history_nesterov,\n",
" history_adagrad, history_rmsprop, history_adam,\n",
" history_adamax, history_nadam, history_adamw),\n",
" opt_names.split()):\n",
" plt.plot(history.history[loss], label=f\"{opt_name}\", linewidth=3)\n",
"\n",
" plt.grid()\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel({\"loss\": \"Training loss\", \"val_loss\": \"Validation loss\"}[loss])\n",
" plt.legend(loc=\"upper left\")\n",
" plt.axis([0, 9, 0.1, 0.7])\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Learning Rate Scheduling"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Power Scheduling"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```python\n",
"learning_rate = initial_learning_rate / (1 + step / decay_steps)**power\n",
"```\n",
"\n",
"Keras uses `power = 1`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Note**: The `decay` argument in optimizers is deprecated. The old optimizers which implement the `decay` argument are still available in `tf.keras.optimizers.legacy`, but you should use the schedulers in `tf.keras.optimizers.schedules` instead."
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {},
"outputs": [],
"source": [
"# DEPRECATED:\n",
"optimizer = tf.keras.optimizers.legacy.SGD(learning_rate=0.01, decay=1e-4)"
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {},
"outputs": [],
"source": [
"# RECOMMENDED:\n",
"lr_schedule = tf.keras.optimizers.schedules.InverseTimeDecay(\n",
" initial_learning_rate=0.01,\n",
" decay_steps=10_000,\n",
" decay_rate=1.0,\n",
" staircase=False\n",
")\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=lr_schedule)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `InverseTimeDecay` scheduler uses `learning_rate = initial_learning_rate / (1 + decay_rate * step / decay_steps)`. If you set `staircase=True`, then it replaces `step / decay_step` with `floor(step / decay_step)`."
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.7004 - accuracy: 0.7588 - val_loss: 0.4991 - val_accuracy: 0.8206\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4781 - accuracy: 0.8316 - val_loss: 0.4477 - val_accuracy: 0.8372\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4293 - accuracy: 0.8487 - val_loss: 0.4177 - val_accuracy: 0.8498\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4053 - accuracy: 0.8563 - val_loss: 0.3987 - val_accuracy: 0.8602\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3864 - accuracy: 0.8633 - val_loss: 0.3859 - val_accuracy: 0.8612\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3720 - accuracy: 0.8675 - val_loss: 0.3942 - val_accuracy: 0.8584\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3616 - accuracy: 0.8709 - val_loss: 0.3706 - val_accuracy: 0.8670\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3529 - accuracy: 0.8741 - val_loss: 0.3758 - val_accuracy: 0.8638\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3452 - accuracy: 0.8765 - val_loss: 0.3587 - val_accuracy: 0.8680\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.3379 - accuracy: 0.8793 - val_loss: 0.3569 - val_accuracy: 0.8714\n"
]
}
],
"source": [
"history_power_scheduling = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAHNCAYAAADolfQeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/SrBM8AAAACXBIWXMAAA9hAAAPYQGoP6dpAACEuklEQVR4nO3dd1gU1/4G8HfYXZYO0os0WyxYwV6w91iiCZqE6FVzf8ZYiTFqTCwpxhi92I1Go97kGk00RiMaURE1YgXsXRBFEOkdFnZ+fyAbV4orLuwC7+d5iHDmzMx35mTl6zlnzgiiKIogIiIiohcy0HUARERERNUFEyciIiIiDTFxIiIiItIQEyciIiIiDTFxIiIiItIQEyciIiIiDTFxIiIiItIQEyciIiIiDTFxIiIiItIQEyciogqIjo6GIAgYO3asTs7fvXt3CILwSsco6xrGjh0LQRAQHR39SscnqomYOBHVUMW/FJ/9MjQ0hKurK95++21cunRJ1yFWiezsbHz99ddo06YNzMzMYGRkhLp166Jr166YM2cO7t69q+sQiagakeo6ACKqXPXr18e7774LAMjMzMTp06exfft27N69G0ePHkWnTp10HGHlycjIQJcuXXDp0iU0aNAA7777LqysrPDgwQNcvXoV33zzDerXr4/69evrOlS9snjxYsyePRsuLi66DoVI7zBxIqrhGjRogAULFqiVzZs3D1999RU+/fRThISE6CawKhAYGIhLly5h/Pjx2LhxY4mhraioKOTl5ekoOv3l5OQEJycnXYdBpJc4VEdUC02ZMgUAcO7cOVVZQUEB/vOf/6Bly5YwNjaGpaUlevTogf3796vtGxkZCUEQMH36dLXyX3/9FYIgwNTUFPn5+WrbHB0d0aRJE7UyURSxefNmdO7cGRYWFjAxMYGPjw82b95cIt4FCxZAEAQcO3YMW7duhbe3N0xMTNC9e/dyrzMsLAwAMHny5FLnA3l6eqJx48YlyhMSEjBz5ky89tprMDIygrW1NTp06IBly5aVep579+5h5MiRqFOnDkxNTdG7d29cvHix1LoJCQmYMWMGGjRoALlcDltbW4wYMQJXrlwptf7Jkyfh6+sLU1NT2NjYwM/PDw8ePCi1bnlzk569hy9S2nGOHTsGQRCwYMEChIeHo1+/fjA3N4elpSWGDx9e5nyo3bt3w8fHB8bGxnBwcMD777+PlJQUeHh4wMPD44WxEOkbJk5EtdDzSYQoivDz80NAQAByc3Px4YcfquZBDR48GCtXrlTVbdmyJaytrUv0VBX/Qs7OzsaZM2dU5devX8fjx4/Ro0cPtfO9++67GD9+PBITE/H2229jwoQJyMrKwvjx4zFz5sxS4166dCk++OADNGzYEFOnTkWXLl3KvU5ra2sAwJ07d158U566ffs22rRpg2XLlsHe3h7Tpk3D22+/DSMjI3z11Vcl6kdHR6N9+/Z48uQJxo0bhz59+uDIkSPo0aMHHj9+rFb37t278Pb2xooVK9CgQQNMmTIFAwcOxMGDB9GhQwe1+wYAR44cQc+ePXHmzBmMHDkS//73vxEVFYXOnTsjJSVF42vSpvPnz6Nr166QSqX4v//7P/j4+GDPnj3o3bs3cnNz1epu3rwZI0aMwN27d/Hee+9hzJgxCAsLQ58+faBQKHQSP9ErE4moRoqKihIBiP369Sux7dNPPxUBiN27dxdFURS3bdsmAhB9fX3FvLw8Vb0HDx6I9vb2okwmE+/du6cqHz58uCgIgvjkyRNVWZMmTcTu3buLEolEXLhwoap8zZo1IgBx586dqrINGzaIAMTx48eLCoVCVZ6Xlye+/vrrIgDx/PnzqvL58+eLAERTU1Px0qVLGt+DPXv2iABECwsL8ZNPPhGPHDkiJicnl7tPu3btRADihg0bSmx78OCB6vvi+wtA/Oabb9TqzZs3TwQgLl68WK28U6dOolQqFQ8dOqRWfvPmTdHc3Fxs3ry5qqywsFCsV6+eKAiCeOLECVW5UqkU3377bdW5nzVmzBgRgBgVFVUi9uJ7GBISUuIaxowZ88LjhISEqM75yy+/qNX39/cXAYjbt29XlaWkpIhmZmaiubm5ePfuXVW5QqEQe/fuLQIQ3d3dS8RJpO+YOBHVUMW/FOvXry/Onz9fnD9/vvjRRx+JnTt3FgGIRkZG4qlTp0RRFMWePXuKAMQzZ86UOM7ixYtFAOIXX3yhKluxYoUIQPz1119FURTF+Ph4EYD4n//8R2zXrp3o6+urqjty5EgRgPj48WNVWYsWLURTU1MxJyenxPkuXbokAhA/+ugjVVnxL/0ZM2a89H349ttvRTMzM9Uv/eJ78uGHH4q3bt1Sq3v27FkRgNitW7cXHrf4/np6eoqFhYWlbnvjjTdUZeHh4apksTQBAQEiAPHy5cuiKIpiaGioCEB8/fXXS9SNjo4WJRKJThKn0u5N8baAgABV2ZYtW8pss7CwMCZOVG1xcjhRDXf37l0sXLgQACCTyeDg4IC3334bs2fPRvPmzQEAERERMDY2Rrt27UrsXzyPKDIyUlVWPOwWEhKCkSNHqobtevTogfj4eAQGBiI3NxdyuRyhoaFo1qwZ7O3tARQN5V2+fBnOzs745ptvSpyveAjnxo0bJbaVFt+LfPzxx5g4cSIOHjyIU6dO4fz58zhz5gzWrFmDTZs2YceOHRgyZAgA4OzZswCAvn37anz8li1bwsBAfdZD3bp1AQCpqamqstOnTwMA4uPjS0zWB/653hs3bsDLy0s1R6pr164l6rq7u8PV1VUn6yy1adOmRFlp11scf2lPbbZr1w5SKX/9UPXE/3OJarh+/frh4MGD5dZJT0+Hq6trqdscHR0BAGlpaaoyLy8v2NnZqRKmkJAQ2NjYoEWLFoiPj8eSJUtw6tQp2NnZ4cmTJ/Dz81Ptm5KSAlEUERsbq0roSpOVlVWizMHBodzrKIu5uTnefPNNvPnmm6prmTt3LtauXYvx48cjNjYWhoaGql/8L/MYvqWlZYmy4qSgsLBQVZacnAwA2L9/f4kJ988qvu7i+12ccD7PwcFBJ4mTptebnp4OALCzsytR38DAALa2tpUUIVHl4uRwIoKFhUWJiczFisstLCxUZYIgwNfXF9evX0d8fDyOHTsGX19fCIKALl26QCaTISQkRDVh/NmJ4cXH8fb2hlg0XaDUr9KWSXjVlbKLWVpaYvXq1XB3d0diYiIuX74MALCysgIAxMbGauU8zyq+7lWrVpV73WPGjFHFCBQ9hVea0tqruOeroKCgxLZnE9+qUHy9T548KbFNqVQiMTGxSuMh0hYmTkSE1q1bIycnRzVU9azQ0FAAQKtWrdTKi4fwfv75Z9y6dQs9e/YEAJiamqJdu3Y4evQoQkJCVElWMXNzczRp0gTXr19XG9qpaoIgwMTERK2seCjw0KFDWj9f+/btAfyzRMKLtGzZEgBw4sSJEtvu379f6pIEderUAVB64hcREaFxrNpQHP+pU6dKbDt79mypyR1RdcDEiYhUvRxz5sxRe0w8NjYWy5cvh1QqxTvvvKO2T3Ev0pIlS9R+Lv7+3LlzCAkJQfPmzWFjY6O279SpU5GdnY3333+/1CG5qKgorQxDff/992prVT1r9+7duHHjBqysrODl5QUAaNu2Ldq1a4fjx49j48aNJfZ5lZ6odu3aoX379ti+fTt27NhRYrtSqVQlqQDQpUsXeHp64s8//8TJkydV5aIoYu7cuWrDYsV8fHwAAFu2bFEr/+2339SOXRWGDh0KMzMz/PDDD4iKilKVFxQU4LPPPqvSWIi0iXOciAj+/v7YvXs3/vjjD7Ro0QKDBw9GVlYWdu7ciaSkJCxbtgz16tVT26dp06ZwcHDA48eP4eDggKZNm6q29ejRA19++SVSU1NVSdmz/u///g+nT5/G1q1b8ffff6N3795wdnbG48ePcePGDZw5cwb/+9//XnmBxAMHDmDixIlo0KABOnfuDGdnZ2RmZiIyMhInTpyAgYEB1q5dC7lcrtrnp59+Qvfu3fHvf/8b//3vf9GxY0fk5ubi6tWriIiIQFJSUoXj2b59O3r06IFRo0YhMDAQ3t7eMDIyQkxMDMLCwvDkyRPVWkgGBgbYsGEDBg4ciN69e8PPzw/Ozs44evQo4uLi0KJFixLvGxw2bBg8PT2xZcsWPHjwAK1bt8b169dx9OhRDBw4EEFBQRWO/WVZWVlh+fLl+Pe//402bdrAz88PlpaWCAoKglwuh7Ozc4lJ9UTVAf+vJSIIgoDffvsN3333HWQyGVatWoWffvoJXl5e+OOPPxAQEFDqfsXDdc+v4N2pUydVMlLa6t6CIGDLli3YsWMHmjVrhj///BPLly9HcHAwjIyM8N1336F3796vfF1LlizBt99+C09PTxw/fhz/+c9/sHHjRjx69AhjxozB2bNnMXr0aLV9GjZsiPDwcEybNg2xsbEIDAzETz/9hMzMTMybN++V4vH09ERERATmzZuHzMxMbN68Gd9//z0iIyPRrVs3bN++Xa1+7969ceTIEbRv3x6//vorNmzYAHd3d5w8eVI1LPcsY2NjHDlyBEOHDsXZs2exbt065Obm4vjx42jbtu0rxV4R77//Pn799VdVMrdlyxZ06NABhw4dQnp6utq8OaLqQhBFUdR1EEREVHvcuXMHDRs2xFtvvVXqsCWRPmOPExERVYqUlJQSL1HOycnBjBkzABQNLRJVN5zjRERElSI0NBTjx49H37594ebmhsTERBw9ehTR0dHo2bOn2vpeRNUFh+qIiKhS3L59G5999hlOnTqlWs+pQYMG8PPzw8yZM2FkZKTjCIleHhMnIiIiIg1xjhMRERGRhpg4EREREWmIk8O1SKlU4tGjRzA3N9faO7WIiIiocomiiIyMDI0WZmXipEWPHj0q8w3zREREpN8ePHiAunXrlluHiZMWmZubAyh6z5a1tbWOo6ndFAoFDh06hL59+0Imk+k6nFqP7aE/2Bb6g22hP9LT0+Hq6qr6PV4eJk5aVDw8Z25uzlcJ6JhCoYCJiQksLCz4F5IeYHvoD7aF/mBb6B9NptlwcjgRERGRhpg4EREREWmIiRMRERGRhpg4EREREWmIiRMRERGRhpg4EREREWmIyxEQEemAQqFAYWGhTs4rlUqRm5urk/PTP9gWlUsmk0EikWj9uEyciIiqUHp6OhITE5GXl6eT84uiCEdHRzx48ICvhtIxtkXlEgQBlpaWcHR01Or9ZeJERFRF0tPTERsbCzMzM9ja2kImk1X5L0ylUonMzEyYmZm98J1cVLnYFpVHFEVkZWXhyZMnMDY2hpWVldaOzcSJiKiKJCYmwszMDHXr1tVZD4NSqUR+fj6MjIz4y1rH2BaVy9jYGHl5eUhISIClpaXWPnN62VJr166Fp6cnjIyM4O3tjRMnTpRbPzQ0FN7e3jAyMkK9evWwfv16te1Xr17FiBEj4OHhAUEQEBgYqJXzEhFpSqFQIC8vT6t/gRNR+SwsLFBYWKjVOWR6lzjt2LED06dPx6effoqIiAh07doVAwYMQExMTKn1o6KiMHDgQHTt2hURERGYO3cupk6dil27dqnqZGdno169evjmm2/g6OiolfMSEb2M4r+4+U4yoqojlRYNrBUUFGjtmHqXOC1fvhzjx4/HhAkT0KRJEwQGBsLV1RXr1q0rtf769evh5uaGwMBANGnSBBMmTMC4cePw3Xffqeq0bdsWS5cuxahRoyCXy7VyXiKiimBvE1HVqYzPm17NccrPz8eFCxcwe/ZstfK+ffvi1KlTpe4TFhaGvn37qpX169cPmzZtgkKh0OhfdxU5b3kKrgcB1pYvvZ/eMJAAHl0Ao2p8DURERJVArxKnxMREFBYWwsHBQa3cwcEB8fHxpe4THx9fav2CggIkJibCycmpUs4LAHl5eWqPFKenpwMAjPd/CMir978qlY0GovDNbboOo8IUCoXan6RbbI+iaxdFEUqlEkqlUmdxiKKo+lOXcRDboioolUqIogiFQlHumk4v83eTXiVOxZ7vWhNFsdzuttLql1au7fMuXrwYCxcuLFH+wLABHE31bhRUI4aFWTDPfYS0hzdwPChI1+G8suDgYF2HQM+oze0hlUrh6OiIzMxM5Ofn6zocZGRk6DqESvfNN99gyZIl2LdvH7p06aLrcMpUE9siJiYGLVu2xOjRo7F27VqdxZGfn4+cnBwcP3683HlO2dnZGh9TrxInW1tbSCSSEr08CQkJJXqDijk6OpZaXyqVwsbGptLOCwBz5sxBQECA6uf09HS4urriz1bfI2BwK43OrW+E24eAnW/D0tISAwcO1HU4FaZQKBAcHIw+ffpwMq4eYHsAubm5ePDgAczMzGBkZKSzOERRREZGBszNzfVuvlV0dDTq16+P9957Dz/++OMrH694TquJiQksLCxe+Xjapi9tsWXLFowfP77M7b6+vjh69OhLHdPMzAxA0cMQurz3ubm5MDY2Rrdu3cr93BWPGGlCrxInQ0NDeHt7Izg4GMOHD1eVBwcHY+jQoaXu07FjR+zbt0+t7NChQ/Dx8dH4L+iKnBco+lCWNtn8YVpe9f3l8PQJBANBgEF1vYZnyGSy6tsWNVBtbo/CwkIIggADAwOdrtlTPCRUHIs+KY5HW7FNmTIFo0ePhpubm95dK6A/bVF87l69epXaM+fh4fHS8Wm7LSvKwMAAgiC88O+el/l7Sa8SJwAICAiAv78/fHx80LFjR2zYsAExMTGYOHEigKJentjYWGzbVjT/ZuLEiVi9ejUCAgLw/vvvIywsDJs2bcL27dtVx8zPz8e1a9dU38fGxiIyMhJmZmZo0KCBRud9GTHJmnf5ERFR5bC1tYWtra2uw6g2evfuXeIhKSpJ71JwPz8/BAYGYtGiRWjVqhWOHz+OoKAguLu7AwDi4uLU1lby9PREUFAQjh07hlatWuGLL77AypUrMWLECFWdR48eoXXr1mjdujXi4uLw3XffoXXr1pgwYYLG530Z95OYOBERPW/Xrl3w9fWFvb09jIyM4Orqiv79+2PPnj3YsmULPD09AQBbt26FIAiqr2PHjgEo+rt8/vz56NChA+zt7SGXy+Hh4YFJkyYhISGhxPkWLFigtj9QNBwoCALGjh2LGzdu4I033oCtrS0EQUB0dLSq3t69e9GvXz/Y2NjAyMgIHh4e8Pf3x5UrV1R1bt26hVmzZqFNmzaqeo0aNcLs2bORmZlZIp64uDhMmzYNDRs2hLGxMWxtbdGpUyd8+OGHJYaK8vPzsXz5crRp0wampqYwNzdH165dsXfv3ldogVezefNmDB06FB4eHjAyMoK1tTX69euHkJAQjY/x/D2wtrZG8+bNMWnSpGpxDwA97HECgEmTJmHSpEmlbtuyZUuJMl9fX4SHh5d5PA8PD9WE8Yqe92UkZyuQlqOApXF1HpJ48f0iItLUunXrMGnSJDg5OWH48OGwsbFBXFwczp49iz179mD69OmYNm0aVqxYgZYtW2LYsGGqfT08PAAAx48fx7Jly9CrVy+0b98eMpkMERERWLduHf766y+Eh4fD0lKzZVTu3LmDDh06oFmzZhgzZgySk5NhaGgIAJg1axaWLl0Ka2trDBs2DPb29njw4AEOHz4Mb29veHl5AQB2796NTZs2oUePHujevTuUSiVOnz6NJUuWIDQ0FMePH1cNAWVnZ6Nz586Ijo5G3759MXz4cOTl5eHWrVvYunUrPvnkE9VcoLy8PPTv3x/Hjh1D69atMX78eCgUCuzfvx9Dhw7FqlWrMHnyZC21jOY+/PBDtGzZEr1794adnR1iY2OxZ88e9O7dG7t37y53agtQ+j3Iz8/HvXv3sGXLFsyaNUvv7wGgp4lTTRCdmIWWrla6DoOIqgFRFJGj0N4rIcqjVCqRk18IaX7BS809MZZJXmkC8w8//ABDQ0NcvHgRdnZ2atuSkpJgY2OD6dOnY8WKFWjVqhUWLFhQ4hg9e/ZEfHy8auJxsW3btmHMmDFYvXo1Pv30U43i+fvvv/HZZ59h0aJFauVBQUFYunQpmjdvjpCQELWHjAoKCpCUlKT62d/fHwEBAaqEq9iiRYswf/587Ny5E++88w4A4MiRI4iKisKMGTOwfPlyAEVtkZ6eDkEQYGxsrLb/sWPHsGDBAnz++eeq+56RkYGePXvio48+whtvvAFnZ2cAQGRkJPbs2aPRdQNFiejYsWNLlB8+fBi5ubklyidOnAhHR0dcu3ZN1StYLC4uDj4+Pvj4449fmDiVdg+KZWRkqM0Zftl7UJWYOFWSqGqbOOnXUzZEtUGOohBNP/9L12GU69qifjAxfLVfGWVN0NX0CWh7e/tSy/39/TFlyhQcPnxY48TJ0dER8+bNK1G+Zs0aAMCKFStKxCWVStWetHZxcSn12JMnT8b8+fNx+PBhVeJU7NkEqZi5ubkqiVUqlVi3bh0aNGigljAU1/v8888xZMgQ7N69W9XjEhkZWerSOGXx9fUtNXE6cuQIjhw5UqJ82LBhcHR0LJE0AYCTkxNGjBiBVatW4f79+xpNbynrHhSryD2oSkycKklUYpauQyAi0htvvfUWZs+eDS8vL4waNQrdu3dHly5dYGVl9VLH2b17N77//nuEh4cjJSVF7eWtjx490vg4LVu2LNFTBABnz56FXC6Hr6/vC48hiiJ+/PFHbNmyBVeuXEFaWpraQpbPxtOtWzc4Ojpi8eLFiIyMxKBBg9CpU6cSicbNmzeRkpICZ2fnUpOhJ0+eAABu3LihKhs7dmypidDLWrx4cbmTw+/du4fFixfj6NGjiI2NVVsAGii63vISp9LuQZcuXdC8eXO15Kgi96AqMXGqJEyciEhTxjIJri3qVyXnUiqVyEjPgLmF+UsP1b2KWbNmwcbGBuvXr8fy5cuxbNkySKVSDBw4EIGBgaX2Zjxv2bJlmDlzJuzs7NC3b1/UrVtX1XsRGBhY4hd5ecpaoy81NRUuLi4a3ZupU6di9erVcHV1xZAhQ+Dk5KQablq4cKFaPJaWlggLC8P8+fOxb98+BD1dYNjZ2Rlz587Fhx9+CABITk4GAFy9ehVXr14t89xZWVX7O+bOnTto164d0tPT0aNHD7z++uuwsLCAgYEBjh07htDQ0Bfe/7LuQd26dTFnzhzVHGN9vQfFmDhVEiZORKQpQRBeeRhMU0qlEgWGEpgYSqt0fR1BEDBhwgRMmDABSUlJOHHiBLZv346dO3fi9u3buHz5crn7FxQU4IsvvoCzszMiIyPV5kmJoohvv/32peMpjZWVFeLj46FUKsu9PwkJCVizZg1atGiBsLAwmJiYqLbFx8eX2lPi4eGBrVu3orCwEJcvX8Zff/2FlStXYvLkybC2tsbo0aNVk6NHjBiB3377TaNr0dYcp/L85z//QUpKCn766acSw48TJ05EaGioxud+9h4cOnQIK1euxIcffog6depU+B5UJSZOlSQqMeuFr2zRaxo8hUhEVBE2NjYYNmwYhg0bhsTERBw9ehR37txRJR/PDr8VS0xMRFpaGnr16lVicvn58+eRk5OjldjatWuHoKAghIaGokePHmXWu3fvHkRRRO/evdWSJgA4ceJEueeQSCRo1aoVWrRogebNm2PQoEHYu3cvRo8ejSZNmsDCwgLnz5/X+EX12prjVJ67d+8CAIYMGaJWrlQq8ffff7/UsYB/7kGrVq3QsWNHdOvW7ZXuQVXSu3WcagIDAcjMK8CTTM27jYmIarK//vqrxLvCFAqFaljG2NgYderUgSAIePjwYYn97e3tYWxsjPDwcLX3iqWkpGDKlClai7N4yGzatGmq2IoVFBTg8ePHAKCay3Pq1Cm1eU0PHz4sdZ7QlStXcP/+/RLlxfN1ioccpVIpPvjgA9y/fx8zZ84s9eWzV65cUVu3auzYsRBFUeOvZ9e10lTx9Z48eVKtfMmSJWprW5WnrHtQfE9f5R5UJfY4VQJHSyPE5wDRidmwN9fdO6kqpLr2kBGRXvPz84OJiQm6dOkCd3d31fsLr127Bj8/P7i5uQEA2rZti+PHj+Nf//oXGjZsCAMDA7z99ttwc3PDpEmTsGzZMrRs2RKvv/460tPTceDAAbi7u2vtsfSBAwdi5syZ+O6779CwYUMMHz4c9vb2iI2NxZEjRzBz5kxMnz5d9TTZrl274OPjg169euHx48f4888/0bNnT9y7d0/tuIcPH8ZHH32Ezp07o3HjxrCxscHdu3exb98+GBsbqz0dtnDhQoSHh2PlypXYv38/fH19VesmXb58GRcvXkRYWFiZTxlWhokTJ+LHH3/EG2+8AT8/P9jY2OD06dMIDw/HoEGDsH///hceo7R7cO/ePezdu7da3INiTJwqgbu1CeJjcxGVmIl2nta6DoeISOcWL16MgwcP4uzZs9i3bx9MTU3RoEEDfP/99xg3bpyq3n//+1/MmDEDe/bsQVpaGkRRRIcOHeDm5obFixfD2toaW7Zswdq1a+Hg4IBRo0Zh4cKFqkUptWHp0qXo2LEjVq9ejd9++w25ublwcnJCz5490adPH1W9LVu2wMPDA7t27cKqVavg5uaGgIAAfPLJJyWe2OvXrx+io6Nx/Phx7N69G5mZmXBxccEbb7yBuXPnolmzZqq6crkcBw4cwKZNm7Bt2zb89ttvyMvLg4ODA5o2bYqJEyeiefPmWrteTbRu3RqHDh3CvHnzsHv3bkgkEnTq1Al///039u7dq1HiVNY9GDVqFGbNmoUmTZqo6urjPSgmiJosqU0aSU9Ph6WlJT7++RR2XkrG/3WrhzkDm7x4R31yOxj4eSTg1Ar4P80m++kjhUKBoKAgDBw4UO/Gx2sjtkfRW9qjoqLg6elZ7lvaK1vxoovFT0SR7rAtKp+mn7vi399paWmqyellYUtVAg+bonHau0/4ZB0REVFNwsSpEnjamgIA7iRk6DiSV8GOSCIioucxcaoE9WyKEqeY5GzkVtH7p4iIiKjyMXGqBDZmhrAwkkIpVseFMPlUHRERUVmYOFUCQRDQ0KHohYV3EjJ1HA0RERFpCxOnStLAzgwAcJuJExERUY3BxKmSNHQoSpyq9wRxIiIiehYTp0pS3744caqmPU5c3ouIiKgEJk6VpOHTxCkqMQsFhcoX1CYiIqLqgIlTJXG2NIaJoQSKQhH3k7NfvIO+4EN1REREZWLiVEkMDATUL54g/riaDtcRERGRGiZOlah4uO7uEyZORERENQETp0pUPEH89mM+WUdERFQTMHGqRMU9TtVzLSc+VUdE1duCBQsgCAKOHTum61CoBmHiVIkaPDNUp1QyESEiKk90dDQEQcDYsWN1HUqtcOzYMQiCoPFX9+7ddR2yXpDqOoCazM3aBIZSA+QqlHiQkg33py//1W98rI6IaobJkydj1KhRcHNz03UoesnDwwPz589XK4uOjsbWrVvRsmVLDBs2rER9YuJUqaQSAzSwM8O1uHTciM+oJokTEVHNYGtrC1tbW12Hobc8PDywYMECtbJjx45h69ataNWqVYltVIRDdZWssVPRy35vxnOCOBHVbrt27YKvry/s7e1hZGQEV1dX9O/fH3v27MGWLVvg6ekJANi6davaEFHxHKVHjx5h/vz56NChA+zt7SGXy+Hh4YFJkyYhISGhxPlKm+P07HDgjRs38MYbb8DW1haCICA6OlpVb+/evejXrx9sbGxgZGQEDw8P+Pv748qVK6o6t27dwqxZs9CmTRtVvUaNGmH27NnIzCw5tzUuLg7Tpk1Dw4YNYWxsDFtbW3Tq1Akffvgh0tPT1erm5+dj+fLlaNOmDUxNTWFubo6uXbti7969r9ACFbdlyxYIgoAtW7Zg//796Nq1K8zNzVW9UM9uf17xkGBpiVhUVBQmTJgANzc3yOVyODk5YezYsbh//37lXtArYI9TJWviaAEgFjfi019Yl4ioplq3bh0mTZoEJycnDB8+HDY2NoiLi8PZs2exZ88eTJ8+HdOmTcOKFStKDBMV/3I+fvw4li1bhl69eqF9+/aQyWSIiIjAunXr8NdffyE8PByWlpYaxXPnzh106NABzZo1w5gxY5CcnAxDQ0MAwKxZs7B06VJYW1tj2LBhsLe3x4MHD3D48GF4e3vDy8sLALB7925s2rQJPXr0QPfu3aFUKnH69GksWbIEoaGhOH78OGQyGQAgOzsbnTt3RnR0NPr27Yvhw4cjLy8Pt27dwtatW/HJJ5/AwsICAJCXl4f+/fvj2LFjaN26NcaPHw+FQoH9+/dj6NChWLVqFSZPnqyllnk5v/76Kw4dOoTBgwdj0qRJyMioeKfAmTNn0K9fP2RlZeH1119HgwYNEB0djZ9//hkHDhxAWFgY6tWrp8XotYOJUyV7zbGox+lGdetx4lx2oqojioCiit4woFQWnStfAhi8xKCDzAQQKj4H8ocffoChoSEuXrwIOzs7tW1JSUmwsbHB9OnTsWLFijKHiXr27In4+HiYmZmplW/btg1jxozB6tWr8emnn2oUz99//43PPvsMixYtUisPCgrC0qVL0bx5c4SEhMDGxka1raCgAElJSaqf/f39ERAQoEq4ii1atAjz58/Hzp078c477wAAjhw5gqioKMyYMQPLly8HACiVSqSnp0MQBBgbG6vtf+zYMSxYsACff/45hKf3PSMjAz179sRHH32EN954A87OzgCAyMhI7NmzR6PrBooS0YpOwD9w4AAOHTqE3r17V2j/YgqFAqNGjYJSqcT58+fRsmVL1baTJ0+ie/fumDZtGvbt2/dK56kMTJwqWfFQXXRiFnIVhTCSSXQcERHpHUU28LVzlZzKAIBVRXac+wgwfLV5mjKZTNUD86xnk5Py2Nvbl1ru7++PKVOm4PDhwxonTo6Ojpg3b16J8jVr1gAAVqxYUSIuqVQKBwcH1c8uLi6lHnvy5MmYP38+Dh8+rEqcij2bIBUzNzeHwdMkVqlUYt26dWjQoIFa0lRc7/PPP8eQIUOwe/duVa9TZGQkFi5cqMllAwB8fX0rnDgNGzbslZMmAPjzzz8RHR2NL774Qi1pAoAuXbpg6NCh2LNnD9LT01U9cfqCiVMlszOTw9rUEMlZ+bj9OBPN62rWjawzr/AvSiKisrz11luYPXs2vLy8MGrUKHTv3h1dunSBlZXVSx1n9+7d+P777xEeHo6UlBQUFhaqtj169Ejj47Rs2bJETxEAnD17FnK5HL6+vi88hiiK+PHHH7FlyxZcuXIFaWlpUCr/ean7s/F069YNjo6OWLx4MSIjIzFo0CB06tQJ7u7uase8efMmUlJS4OzsXGoy9OTJEwDAjRs3VGVjx46tsiUc2rVrp5XjnD59GkDRdZTWuxgfHw+lUolbt27Bx8dHK+fUFiZOlUwQBDR2NMepu0m4Hp+u/4kTEVU9mUlRj04VUCqVSM/IgMUzvRwakZm80nlnzZoFGxsbrF+/HsuXL8eyZcsglUoxcOBABAYGqiaGl2fZsmWYOXMm7Ozs0LdvX9StW1fVgxMYGIi8vDyN43m25+hZqampcHFx0ejeTJ06FatXr4arqyuGDBkCJycnyOVyAMDChQvV4rG0tERYWBjmz5+Pffv2ISgoCADg7OyMuXPn4sMPPwQAJCcnAwCuXr2Kq1evlnnurKwszS5Uy8q6by+r+Dp//vnncuvp6jrLw8SpCjR2tMCpu0m4EVfN5jkRUdUQhFceBtOYUgnICovO9zKJ0ysSBAETJkzAhAkTkJSUhBMnTmD79u3YuXMnbt++jcuXL5e7f0FBAb744gs4OzsjMjJSbZ6UKIr49ttvXzqe0lhZWal6O8pLnhISErBmzRq0aNECYWFhMDH5J7GMj48vtbfIw8MDW7duRWFhIS5fvoy//voLK1euxOTJk2FtbY3Ro0erhqVGjBiB3377TaNrqco5TmXdt+J7VVBQUGJbWlpaibLi69y3bx8GDx5coVh0hYlTFWj8dIL4zcd8so6IyMbGBsOGDcOwYcOQmJiIo0eP4s6dO6rk49nht2KJiYlIS0tDr169SkwuP3/+PHJycrQSW7t27RAUFITQ0FD06NGjzHr37t2DKIro3bu3WtIEACdOnCj3HBKJBK1atUKLFi3QvHlzDBo0CHv37sXo0aPRpEkTWFhY4Pz581AoFKXOCXteVc5xKkudOnUAALGxsSW2RURElChr3749ACAsLKzaJU5cx6kKFE8Qr149Tnysjoi056+//irRG6FQKFRDNsbGxqhTpw4EQcDDhw9L7G9vbw9jY2OEh4cjO/ufJxBTUlIwZcoUrcVZPGQ2bdo0VWzFCgoK8PjxYwBQzU06deqU2rymhw8fYvbs2SWOe+XKlVLXJiqes1Q85CiVSvHBBx/g/v37mDlzJhQKRanHenbdqrFjx0IURY2/KuPdfW3atIEgCPjll1+Qm5urKr99+zZWrFhRov7QoUPh5uaG5cuX4/jx4yW2KxQKnDx5UutxagN7nKpAQ3tzCAKQlJWPJxl5sDOX6zokIqIq5efnBxMTE3Tp0gXu7u5QKBQIDg7GtWvX4Ofnp3otStu2bXH8+HH861//QsOGDWFgYIC3334bbm5umDRpEpYtW4aWLVvi9ddfR3p6Og4cOAB3d3fVo/mvauDAgZg5cya+++47NGzYEMOHD4e9vT1iY2Nx5MgRzJw5E9OnT4eTkxNGjBiBXbt2wcfHB7169cLjx4/x559/omfPnrh3757acQ8fPoyPPvoInTt3RuPGjWFjY4O7d+9i3759MDY2VluXaeHChQgPD8fKlSuxf/9++Pr6ws7ODrGxsbh8+TIuXryIsLCwMp8y1AUXFxf4+fnhl19+gbe3N/r374+EhAT8/vvv6N+/P3bt2qVWXy6X47fffsOAAQPg6+uLXr16qdbHiomJwYkTJ2BjY6M2CV5fMHGqAsaGEnjamOJeYhZuxKfDztzuxTvpDJ+qIyLtW7x4MQ4ePIizZ89i3759MDU1RYMGDfD9999j3Lhxqnr//e9/MWPGDOzZswdpaWkQRREdOnSAm5sbFi9eDGtra2zZsgVr166Fg4MDRo0ahYULF6p+6WrD0qVL0bFjR6xevRq//fYbcnNz4eTkhJ49e6JPnz6qelu2bIGHhwd27dqFVatWwc3NDQEBAfjkk09KPLHXr18/REdH4/jx49i9ezcyMzPh4uKCN954A3PnzkWzZs1UdeVyOQ4cOIBNmzZh27Zt+O2335CXlwcHBwc0bdoUEydORPPmzbV2vdqyadMm2NnZYefOnVizZg1ee+01bNiwAc7OziUSJ6AoSb548SKWLl2KoKAgnDx5EnK5HC4uLhg2bBhGjx6tg6t4MUEURY7JaEl6ejosLS2RmJhYYv2PD366gANX4jFvUBNM6Kp/K6Gq3A0B/jsMcPACPvhb19FUmEKhQFBQEAYOHKjRHAGqXGwPIDc3F1FRUfD09ISRkZHO4ihedNHCwuLlnqojrWNbVD5NP3fFv7/T0tJeuG4UW6qKNHYsaohrcZwgTkREVF0xcaoizZyfJk6PmDgRERFVV0ycqoiXS9HCl7cTMpGrKPmord7hCC4REVEJTJyqiIOFHLZmhihUitXvhb9EREQEgIlTlREEAc2ci3qdrsSWXEWViIiI9B8TpypUPM/p6iM9Tpz4kl8iIqIyMXGqQsXznK5ygjhRrcUVYIiqTmV83pg4VSGvp0N1N+IyoChUvqA2EdUkEokEAEp9hQYRVY7i1/xIpdpb75uJUxVytTaGuZEU+YVK3H6cqetwiKgKyWQyyOVy1WrYRFT50tPTIZFIVP9w0Qa+cqUKCYIAL2dLhN1LwpVHaWjqXP7qpLrFv9iJtM3W1haxsbF4+PAhLC0tIZPJIFTxvEKlUon8/Hzk5uZytWodY1tUHlEUkZWVhfT0dDg5OWn1c8bEqYp5uVgg7F4SrsamAT6uug6HiKpQ8ascEhMTERsbq5MYRFFETk4OjI2NqzxpI3Vsi8olCAKsrKxgaWmp1eMycapixRPEr+jtBHF+eIkqk4WFBSwsLKBQKFBYWPWL4SoUChw/fhzdunWrte8N1Bdsi8olk8m0OkRXjIlTFStekuB6XDoKlSIkBkxUiGojmUymk1+WEokEBQUFMDIy4i9rHWNbVE8cVK1inrZmMJZJkJ1fiKhEThAnIiKqTpg4VTGJgQAvl6Jep4sP9HghTCIiIiqBiZMOtKxrBQCIfJCq0zjKxceliYiISmDipAOt3KwAABcfpuo0DiIiIno5TJx0oLjH6XpcOnIVVf9UTbn4SCwREVGZmDjpQN06xrAxNYSiUMS1OH1dloCIiIiex8RJBwRBQCtXKwDARX2e50RERERqmDjpSMuniZNeTxAnIiIiNXqZOK1duxaenp4wMjKCt7c3Tpw4UW790NBQeHt7w8jICPXq1cP69etL1Nm1axeaNm0KuVyOpk2b4vfff1fbXlBQgHnz5sHT0xPGxsaoV68eFi1aBKVSqdVrK6b/PU58qo6IiOh5epc47dixA9OnT8enn36KiIgIdO3aFQMGDEBMTEyp9aOiojBw4EB07doVERERmDt3LqZOnYpdu3ap6oSFhcHPzw/+/v64ePEi/P398dZbb+HMmTOqOkuWLMH69euxevVqXL9+Hd9++y2WLl2KVatWVcp1tqhb9OqV6KRspGTlV8o5iIiISLv0LnFavnw5xo8fjwkTJqBJkyYIDAyEq6sr1q1bV2r99evXw83NDYGBgWjSpAkmTJiAcePG4bvvvlPVCQwMRJ8+fTBnzhw0btwYc+bMQa9evRAYGKiqExYWhqFDh2LQoEHw8PDAyJEj0bdvX5w/f75SrtPKxBCetqYA9G1ZAj5VR0REVBa9elddfn4+Lly4gNmzZ6uV9+3bF6dOnSp1n7CwMPTt21etrF+/fti0aRMUCgVkMhnCwsIwY8aMEnWeTZy6dOmC9evX49atW2jUqBEuXryIkydPqtV5Xl5eHvLy8lQ/p6cXPSGnUCigUCheeL0tXCwQlZiF8OhkdK5X54X1q4JQWAApit7aXaDBNeir4vuvSTtQ5WN76A+2hf5gW+iPl2kDvUqcEhMTUVhYCAcHB7VyBwcHxMfHl7pPfHx8qfULCgqQmJgIJyenMus8e8xPPvkEaWlpaNy4MSQSCQoLC/HVV19h9OjRZca7ePFiLFy4sER5SEgITExMXni90jQBgASHI26jfu7NF9avCjYZ19EFQEZmJkKCgnQdzisLDg7WdQj0DLaH/mBb6A+2he5lZ2drXFevEqdiwnOLMIqiWKLsRfWfL3/RMXfs2IGffvoJ//vf/9CsWTNERkZi+vTpcHZ2xpgxY0o975w5cxAQEKD6OT09Ha6urujRowdsbGxecJWAy8M07Pr+DB7lydG/f3cYGOh+mEy4bwHcAczNzDBw4EBdh1NhCoUCwcHB6NOnD986rgfYHvqDbaE/2Bb6o3jESBN6lTjZ2tpCIpGU6F1KSEgo0WNUzNHRsdT6UqlUlbyUVefZY3788ceYPXs2Ro0aBQBo3rw57t+/j8WLF5eZOMnlcsjl8hLlMplMow9BC1dryKUGSM1R4EFaHhrYm79wn0onKfpfQgBqxAdZ07agqsH20B9sC/3BttC9l7n/ejU53NDQEN7e3iW6LYODg9GpU6dS9+nYsWOJ+ocOHYKPj4/qRpRV59ljZmdnw8BA/XZIJJJKW44AAAylBqplCc5Hp1TaeYiIiEg79CpxAoCAgAD88MMP2Lx5M65fv44ZM2YgJiYGEydOBFA0PPbee++p6k+cOBH3799HQEAArl+/js2bN2PTpk2YOXOmqs60adNw6NAhLFmyBDdu3MCSJUtw+PBhTJ8+XVXn9ddfx1dffYX9+/cjOjoav//+O5YvX47hw4dX6vX6eBRNCj+nL4kT31VHRERUJr0aqgMAPz8/JCUlYdGiRYiLi4OXlxeCgoLg7u4OAIiLi1Nb08nT0xNBQUGYMWMG1qxZA2dnZ6xcuRIjRoxQ1enUqRN++eUXzJs3D5999hnq16+PHTt2oH379qo6q1atwmeffYZJkyYhISEBzs7O+L//+z98/vnnlXq9Pu7WAO7iwv3kSj0PERERvTq9S5wAYNKkSZg0aVKp27Zs2VKizNfXF+Hh4eUec+TIkRg5cmSZ283NzREYGFju8gOVoY1bHQhC0UKYTzLyYGdecs4UERER6Qe9G6qrbSxNZGj0dFI4e52IiIj0GxMnPeD9dJ6Tfk0Q57vqiIiInsfESQ+0LZ4gfl+fEiciIiJ6HhMnPVA0QRy4GpuGnPxCHUfDp+qIiIjKwsRJD9StYwwHCzkKlKKevfCXiIiInsXESQ8IgqDqdTofzQniRERE+oqJk54onud0JoqJExERkb5i4qQnOtQveq/e+egU5BdU3mteNCbyqToiIqLnMXHSE43szVHHRIYcRSEucZ4TERGRXmLipCcMDAR0qFfU63T6XpLuAuG76oiIiMrExEmPdHw6XBemy8SJiIiIysTESY8U9zhduJ+CvAJdr+dEREREz2PipEca2pvBxtQQuQolLj5I03U4RERE9BwmTnpEEP6Z5xR2V9fDdXyqjoiI6HlMnPRM8bIEOp0gTkRERKVi4qRnOhbPc4pJQa5CF/Oc+FQdERFRWZg46Zn6dqawM5cjv0CJiJhUXYdDREREz2DipGcEQVD1Op26m6jjaIiIiOhZTJz0UNeGtgCA47eZOBEREekTJk56qGtDOwDApYepSMnK100QfFcdERFRCUyc9JCjpREaOZhBFIG/OVxHRESkN5g46aniXqcTt6o4ceK76oiIiMrExElPdWv0NHG6/QQih82IiIj0AhMnPdXOwxqGUgM8SsvF3SdZug6HiIiIoKXEKTk5GQ8ePNDGoegpY0MJ2nlYAyjqdSIiIiLdq3DilJaWhmnTpsHBwQF2dnbw9PRUbTtz5gwGDhyICxcuaCXI2qp4WYITOlmWgMODREREz6tQ4pScnIz27dtj1apVcHV1RZMmTdTm4bRo0QJ///03fv75Z60FWhsVTxAPu5uEvAJdvH6FiIiInlWhxGnBggW4desWtm/fjvPnz+PNN99U225sbAxfX18cPXpUK0HWVo0dzWFrJkeOohDno1Oq6Kx8qo6IiKgsFUqc9u7di8GDB8PPz6/MOu7u7nj48GGFAyPAwEBAj9eKep2OXE/QcTRERERUocQpLi4OTZs2LbeOkZERsrL4NNir6tXEHgBw5MZjLktARESkYxVKnGxsbF74FN2NGzfg5ORUoaDoH10a2kEmEXA/KRv3EpmIEhER6VKFEqdu3bph7969iI2NLXX7tWvXcPDgQfTu3fuVgiPATC5Fh3o2AICjVTlcx94tIiKiEiqUOH366acoKChA586d8b///Q+JiUWPy1+/fh2bNm1Cz549IZfL8fHHH2s12NqqV+N/huuIiIhId6QV2al58+bYsWMH3nvvPfj7+wMARFGEl5cXRFGEubk5du7ciYYNG2o12NqqZ2MHLNh3DeeiU5CWo4ClsazyTsZ31REREZWpQokTAAwZMgT37t3D1q1bcebMGSQnJ8PCwgLt27fHv/71L9ja2mozzlrNzcYEDe3NcDshE6G3nmBIS2ddh0RERFQrVThxAgBra2vMmDFDW7FQOXo2scfthEwcvf6YiRMREZGOVGiO07hx47B3795y6wQFBWHcuHEVCopK6t3EAQBw7NYTFBQqdRwNERFR7VShxGnLli2IjIwst87ly5exdevWihyeStHa1QpWJjKkZitwrkpWEedTdURERM+r8Et+XyQ3NxdS6SuNBNIzpBIDVa/TX1fjdRwNERFR7VThxEko4+krURTx4MEDBAUFwdmZc3G0aYCXIwDg4JV4KJWV1SPEp+qIiIjKonHiZGBgAIlEAolEAqDoRb/FPz/7JZVK4eHhgXPnzmHUqFGVFnht1LmBLUwNJYhPz8XFh6m6DoeIiKjW0XgsrVu3bqpepuPHj8PNzQ0eHh4l6kkkElhbW6Nnz554//33tRYoAUYyCXo2ccC+i49w8Eo8WrvV0XVIREREtYrGidOxY8dU3xsYGOBf//oXPv/888qIicrRv5ljUeJ0NR6zBzQuc8iUiIiItK9Cs7eVSj4OryvdX7ODXGqA+0nZuB6XgabOFpVzIr6rjoiIqIRKe6qOKoepXIpujewAAAf5dB0REVGVqvB6AYWFhdi5cycOHz6MR48eIS8vr0QdQRBw5MiRVwqQShrg5Yjga49x8EocAvo00u7BOfRHRERUpgolTllZWejbty9Onz4NURQhCALEZ4Z2in/m/JvK0auxA6QGAm49zsSdhAw0sDfXdUhERES1QoWG6r788kuEhYVh4cKFSExMhCiKWLBgAeLi4rBjxw54enpi5MiRpfZC0auzNJGphuv2XozTcTRERES1R4USp927d6NDhw6YN28erK2tVeUODg548803cezYMRw5cgRLly7VWqCkrvhFv3sjY9V6+4iIiKjyVChxiomJQYcOHf45iIGBWu9S3bp1MWjQIL6rrhL1aeoAI5kBopOycTk2rRLOwGSMiIjoeRVKnExNTWFg8M+ulpaWiItTHzJydHRETEzMq0VHZTKVS9Hr6bvr9kY+0nE0REREtUOFEid3d3e1pMjLywtHjx5V9TqJoogjR47AyclJO1FSqYY+Ha7bd+kRCrX27jpO6CciIipLhRKnXr16ISQkBAUFBQCAMWPGICYmBh07dsTHH3+MLl26IDIyEiNGjNBqsKTO9zU7WBhJ8Tg9D2ejknUdDhERUY1XoeUI3n//fdjY2ODJkydwcnLCuHHjEBERgbVr1yIyMhIAMGLECCxYsECLodLz5FIJ+ns5Yuf5h9h78RE61rfRdUhEREQ1WoV6nBo2bIhPPvlEbShu1apViI+PR1hYGB49eoRff/0VJiYmWguUSjekpQsAIOhyHPIL+CocIiKiyqTVV67Y2dmhffv2cHR0BADcuXNHm4enUnSsbwM7cznSchQ4djNBewfmQ3VEREQlVMq76u7fv49x48ahWbNmlXF4eobEQMDw1kW9Tr9eeKjjaIiIiGq2l57jFBoaigsXLkAqlaJz587w9vZWbYuLi8PChQvx448/QqFQwMXFRavBUulGetfFhuP3EHIjAYmZebA1k1f8YHyojoiIqEwaJ075+fkYNmwY/vrrL7XyKVOmIDAwEJs3b8a0adOQlZUFZ2dnzJ49G//+97+1HjCV1MjBHC3rWuLiwzTsiYjFhK71dB0SERFRjaTxUN2KFStw8OBBODk54YMPPsDEiRPh4OCAVatW4aOPPsKECRNgaGiIwMBA3L17F5MnT4ahoWGFglq7di08PT1hZGQEb29vnDhxotz6oaGh8Pb2hpGREerVq4f169eXqLNr1y40bdoUcrkcTZs2xe+//16iTmxsLN59913Y2NjAxMQErVq1woULFyp0DVVtpI8rAOC3Cw/5ChYiIqJKonHitGPHDtja2uLy5ctYvXo11qxZg4sXL6JOnToIDAxE27ZtcfPmTUydOhVyecWHinbs2IHp06fj008/RUREBLp27YoBAwaUuQp5VFQUBg4ciK5duyIiIgJz587F1KlTsWvXLlWdsLAw+Pn5wd/fHxcvXoS/vz/eeustnDlzRlUnJSUFnTt3hkwmw4EDB3Dt2jUsW7YMVlZWFb6WqjSkhTMMpQa4EZ+Bq4/SdR0OERFRjaRx4nTr1i0MGzYMderUUZXZ2dlh+PDhAIp6iWxtbV85oOXLl2P8+PGYMGECmjRpgsDAQLi6umLdunWl1l+/fj3c3NwQGBiIJk2aYMKECRg3bhy+++47VZ3AwED06dMHc+bMQePGjTFnzhz06tULgYGBqjpLliyBq6srfvzxR7Rr1w4eHh7o1asX6tev/8rXVBUsTWTo27ToFSy/aWWSOHutiIiInqdx4pSZmQlnZ+cS5cUTwFu2bPnKweTn5+PChQvo27evWnnfvn1x6tSpUvcJCwsrUb9fv344f/48FApFuXWePebevXvh4+ODN998E/b29mjdujU2btz4ytdUld58Oly3JzIWeQWFOo6GiIio5nmpp+qefbFvMUEoegxLKq3QIuRqEhMTUVhYCAcHB7VyBwcHxMfHl7pPfHx8qfULCgqQmJgIJyenMus8e8x79+5h3bp1CAgIwNy5c3H27FnVsON7771X6rnz8vJU7+cDgPT0oiEyhUKhStqqUnt3SzhYyPE4PQ8HLz3CwOaOL30MoaAQUhS9b7BAB9egLcX3XxftQCWxPfQH20J/sC30x8u0wUtlOw8fPsTZs2dLlAHAuXPnSp2U3K5du5c5BYB/krFioiiWKHtR/efLX3RMpVIJHx8ffP311wCA1q1b4+rVq1i3bl2ZidPixYuxcOHCEuUhISE6WzW9lbkB/ko3wOqDkcCDl19J3Cr7HnwB5OTkIDgoSOvxVbXg4GBdh0DPYHvoD7aF/mBb6F52drbGdV8qcdq0aRM2bdpUolwURXTo0KHUfQoLNR8ysrW1hUQiKdG7lJCQUKLHqJijo2Op9aVSKWxsbMqt8+wxnZyc0LRpU7U6TZo0UZtk/rw5c+YgICBA9XN6ejpcXV3Ro0cP1bmrWqvUHAQvP4Hb6QZo3LYr6tmZvtT+wqMI4CZgbGyMgQMHVlKUlU+hUCA4OBh9+vSBTCbTdTi1HttDf7At9AfbQn8UjxhpQuPEacyYMRUK5mUYGhrC29sbwcHBqknnQFE2PnTo0FL36dixI/bt26dWdujQIfj4+Kj+R+zYsSOCg4MxY8YMtTqdOnVS/dy5c2fcvHlT7Ti3bt2Cu7t7mfHK5fJSnyCUyWQ6+xC428nQs7E9Dl9PwM7wR/hscNMX7/Ssp0OugiDUiA+yLtuCSmJ76A+2hf5gW+jey9x/jROnH3/8sULBvKyAgAD4+/vDx8cHHTt2xIYNGxATE4OJEycCKOrliY2NxbZt2wAAEydOxOrVqxEQEID3338fYWFh2LRpE7Zv36465rRp09CtWzcsWbIEQ4cOxR9//IHDhw/j5MmTqjozZsxAp06d8PXXX+Ott97C2bNnsWHDBmzYsKFKrlub3mnvjsPXE/DbhYf4uN9rMJJJXv4gXAuKiIiohEp5V92r8PPzQ2BgIBYtWoRWrVrh+PHjCAoKUvX8xMXFqa3p5OnpiaCgIBw7dgytWrXCF198gZUrV2LEiBGqOp06dcIvv/yCH3/8ES1atMCWLVuwY8cOtG/fXlWnbdu2+P3337F9+3Z4eXnhiy++QGBgIN55552qu3gt6dbIDi5WxkjLUWD/pThdh0NERFRjvPqjcJVg0qRJmDRpUqnbtmzZUqLM19cX4eHh5R5z5MiRGDlyZLl1Bg8ejMGDB2scp76SGAh4u70blv51Ez+fuY8R3nU137mcSfhERES1nd71OJF2vOlTF1IDAeExqbjGlcSJiIi0golTDWVvboR+zYrWcdoWFq3bYIiIiGoIJk412JhOHgCA3yNikZyVr9tgiIiIagAmTjVYW486aO5iibwCJf535v5L7s2n6oiIiJ7HxKkGEwQB47p4AAC2hd1HfsHLryRORERE/2DiVMMNau4Me3M5EjLyEHRZk6UJ+FQdERFRWSq0HMG4ceNeWMfAwAAWFhZ47bXXMHjwYLi4uFTkVPSKDKUGeK+jO747dAubTkZhaCvnct/7R0RERGWrUOK0ZcsW1S/f0l7sKwiCWvmUKVPw+eefY968eRUMk17F2+3dseroHVyOTcP5+ylo62Gt65CIiIiqpQoN1d29exeDBw+Gg4MDFi9ejNDQUNy4cQOhoaH4+uuv4eDggCFDhuDMmTPYsGEDnJ2dMX/+fOzYsUPb8ZMGrE0N8Uaboh6/H07c03E0RERE1VeFepx27NiBs2fP4uLFi7C3t1eVN2rUCF27dsXYsWPRqlUrhISEYNasWRgwYACaNm2KtWvXws/PT2vBk+bGdfbE9rMPcOjaY9xJyEQDe7Pyd+C76oiIiEqoUI/Tpk2b8Oabb6olTc9ydHTEm2++iY0bNwIAXFxcMHjwYFy8eLHikdIraehgjj5NHSCKwPehd3UdDhERUbVUocTp4cOHkMvl5dYxMjLCw4cPVT+7ubkhNze3IqcjLZnUvT6AogUxY1NzSq/EieNERERlqlDi5OLigj/++AN5eXmlbs/Ly8Mff/yh9iRdQkIC6tSpU7EoSStau9VBx3o2KFCK2Hicc52IiIheVoUSp/Hjx+POnTvw9fXF/v37kZycDABITk7Gn3/+iW7duuHu3btqyxacOHECLVu21E7UVGGTehT1Ov1yLgZJmaUnvkRERFS6Ck0OnzVrFq5fv46ffvoJQ4YMAVC0bpNSWbQytSiKeOeddzB79mwAwOPHjzFo0CD0799fS2FTRXVpYIvmLpa4HJuGLaei8VHf13QdEhERUbVRocRJIpFg27ZtGDNmDH766SdcunQJ6enpsLCwQMuWLfHOO++gV69eqvoODg74z3/+o7WgqeIEQcCk7vXxwc/h2HoqGv/uVg/mRrJSavKpOiIioudVKHEq1qtXL7UEiaqHfs0cUd/OFHefZOHHv6MxtVdDXYdERERULfBddbWQgYGAab0bAQA2nriHtBzFM1v5VB0REVFZXqnHKT4+HhcuXEBqaioKCwtLrfPee++9yimokgxu7oTVR2/j1uNMbDoZhYA+jXQdEhERkd6rUOKUm5uL999/H9u3by/1XXVA0QRxQRCYOOkpAwMBM3o3wgc/h2PzySj8q5MH6pga6josIiIivVahxOmTTz7Bzz//jEaNGmH06NGoW7cupNJX6rwiHejXzBFNnCxwPS4dG0/cw6z+jXUdEhERkV6rULbz66+/omnTprhw4cILVxAn/VXU69QQ//7vBWw5FY3xXTxhU7yR76ojIiIqoUKTw1NTU9G/f38mTTVAn6YOaO5iiez8Qqw7xnfYERERladCiVOTJk3w+PFjbcdCOiAIAj7qWzQxfFvYfcSnczVxIiKislQocfrkk0/wxx9/4M6dO9qOh3TAt5EdOtW3QX6hEtvConUdDhERkd6q0BwnR0dH9O/fH+3atcP06dPRunVrWFpallq3W7durxQgVT5BEDB3YBMMXnUSITefYBZHYImIiEpVocSpe/fuEAQBoihiwYIFEISyF00sa30n0i9eLpYY1soZNy/eB1D0whUuhUlERKSuQonT559/Xm6yRNXTR31fw4eXzwAACvKyIbu2V8cRVZxQWAjrzNuAOEDXoRARUQ1SocRpwYIFWg6D9IGrtQkGta4LXAFkinRgp7+uQ6owKYCuAAruNgeaMHkiIiLt4KqVpMZvQG/8eqU3PJQxcLcxhb1Z9ZzwJD65ASE3FciI03UoRERUgzBxIjVWpkbI6vMd3tx3DXXSZAh5vzusTKrfq1jEn/0g3D6IotlaRERE2qFR4lSvXj0IgoDDhw/D09MT9erV0+jggiDg7l0uqljdvNvBHdvPPsDNxxlYdugWvhjmpeuQiIiI9IJG6zgplUoolUq1n0VRfOHXs/tQ9SGVGGD+kKYAgJ/P3Me1R+k6jqgCih9e4KtjiIhIizTqcYqOji73Z6p5OtW3xaDmTth/OQ4L9l7Fjv/rwCcpiYio1qvQyuFUO8wd1ARGMgOcjU7G3ouPdB3OSxKe/pc9TkREpD1MnKhMLlbG+LB7AwDAV/uvIy1HoeOIiIiIdKvCT9Xl5+djz549OHfuHFJTU0tdIVwQBGzatOmVAiTder9bPfweEYt7iVn49uANfDW8ua5D0gznOBERUSWoUOJ0//599OnTB3fv3oVYzi8mJk7Vn5FMgq+GN8fojafx85kYDG/tAh8Pa12HRUREpBMVSpxmzJiBO3fuwN/fH+PGjUPdunUhlXJJqJqqY30bvOVTFzvPP8Sc3Zexf2pXGEr1fZS3eCI7e5yIiEh7KpTtHD16FL169cLWrVu1HQ/pqbkDm+DI9QTcTsjE96F3MaVXQ12HREREVOUq1G2gVCrRunVrbcdCeszKxBCfv160ttOqo3dw90mmjiN6AdUcJ92GQURENUuFEqeOHTvi+vXr2o6F9NyQls7o1sgO+YVKfPzrRRQqmZUQEVHtUqHE6ZtvvkFISAh+++03bcdDekwQBHw93AtmcinCY1Lxw4l7ug6pHJzjRERE2lehOU779u1Djx494OfnB19fX7Ru3RqWlpYl6gmCgM8+++yVgyT9UbeOCT4b3ASf7LqMZYduoUdjezRyMNd1WERERFWiQonTggULVN8fO3YMx44dK7UeE6ea6S0fVxy8Eo+Qm0/w0c6L2D2pE2QSPXvKjus4ERFRJahQ4hQSEqLtOKgaEQQB34xogb7/OY7LsWlYG3IX03rzKTsiIqr5KpQ4CYIACwsLtGrVSsvhUHXhYGGERUObYdovkVh19DZ6NrZH87olh2t1jz1ORESkPRUaX+nRowc2btyo7ViomhnS0hmDmjuhQCli6i8RyMwr0HVIRERElapCiZO9vT0MDQ21HQtVM4Ig4KvhXnC2NEJUYhbm/3FV1yH9Q+BTdUREpH0VSpz69euH0NDQct9TR7WDlYkhAke1hoEA7Ap/iD0RsboOiYiIqNJUKHH6+uuvkZSUhH//+99ITk7WdkxUzbTztMbUp69gmbfnCu4nZek4IkC1jhOTeyIi0qIKTQ5/9913YWVlhc2bN+Onn36Cp6cnHBwcIKiGR4oIgoAjR45oJVDSb1N6NsSpu0k4G5WMKdsj8NvETtXgRcBEREQvp0KJ07PrNuXl5eHGjRu4ceNGiXrPJ1JUc0kMBAT6tcKAFSdw6WEavg66jgVDmukuIM5xIiKiSlDhl/xq8lVYWKjteEmPOVsZY/lbLQEAW05Fc74TERHVOBxLIa3q1cQBk3s0AADM2X0ZN+LTdRQJ5zgREZH2MXEirZvRpxG6NrRFjqIQE/97Aem5Cl2HREREpBUVmuNU7OHDhwgJCcGjR4+Ql5dXYjvfVVc7SQwErBjVGq+vOonopGzM3HkR3/t7V+2cN85xIiKiSlDhxOnjjz/GihUr1OYxiaKo+uVY/D0Tp9rJ2tQQa99pgzfXh+HQtcdYceQ2pvdupOuwiIiIXkmFhuo2btyIZcuWoUePHvjtt98giiLGjBmD7du3Y+LEiZBKpRg5ciSOHj2q7XipGmnpaoUvh3kBAAIP38aflx5V4dk5x4mIiLSvQonThg0b4OHhgQMHDmD48OEAAA8PD/j5+WHNmjU4dOgQ9uzZgydPnmg1WKp+3mrrigldPAEAH+28iIsPUnUbEBER0SuoUOJ048YN9O/fHwYG/+xeUPDPC159fX0xaNAgfPfddxUKau3atfD09ISRkRG8vb1x4sSJcuuHhobC29sbRkZGqFevHtavX1+izq5du9C0aVPI5XI0bdoUv//+e5nHW7x4MQRBwPTp0ysUP6mbM7AJerxmh7wCJd7fdh7xabmVf1LOcSIiokpQ4afqrKysVN+bmpoiKSlJbftrr72Gq1df/qWvO3bswPTp0/Hpp58iIiICXbt2xYABAxATE1Nq/aioKAwcOBBdu3ZFREQE5s6di6lTp2LXrl2qOmFhYfDz84O/vz8uXrwIf39/vPXWWzhz5kyJ4507dw4bNmxAixYtXjp2Kp3EQMDK0a3RyMEMCRl5mLDtHHLyucYXERFVPxVKnFxcXPDw4UPVz/Xr1y+RhFy5cgWmpqYvfezly5dj/PjxmDBhApo0aYLAwEC4urpi3bp1pdZfv3493NzcEBgYiCZNmmDChAkYN26cWm9XYGAg+vTpgzlz5qBx48aYM2cOevXqhcDAQLVjZWZm4p133sHGjRtRp06dl46dymZuJMOmMW1hbWqIK7HpmLI9AgWFyko8I+c4ERGR9lXoqbrOnTurDZ8NHToUX375JSZOnIjXX38dJ0+exIEDBzBixIiXOm5+fj4uXLiA2bNnq5X37dsXp06dKnWfsLAw9O3bV62sX79+2LRpExQKBWQyGcLCwjBjxowSdZ5PnD788EMMGjQIvXv3xpdffvnCePPy8tSWYUhPL1rsUaFQQKHg2kXPczSXYe3olhiz5QIOX3+MT3+/jC+GNKmUZQoEpRIGAAoLlVCyLXSu+PPAz4XusS30B9tCf7xMG1QocfL398ejR49w//59uLu74+OPP8aff/6JDRs2YOPGjRBFER4eHli6dOlLHTcxMRGFhYVwcHBQK3dwcEB8fHyp+8THx5dav6CgAImJiXByciqzzrPH/OWXXxAeHo5z585pHO/ixYuxcOHCEuUhISEwMTHR+Di1zbv1BWy+aYAd5x8iLf4+Brhqv1eodVwc3ADcvn0Ld9KDtH58qpjg4GBdh0BPsS30B9tC97KzszWuW6HEqXv37ujevbvqZzMzM5w+fRp//PEH7t69C3d3d7z++usVGqoDSr4c+Nn1oTSt/3x5ecd88OABpk2bhkOHDsHIyEjjOOfMmYOAgADVz+np6XB1dUWPHj1gY2Oj8XFqm4EA3M4+wPx913HwoQRdvJvCz6euVs8h/PEnkAw0bNgQjboM1Oqx6eUpFAoEBwejT58+kMlkug6nVmNb6A+2hf4oHjHSxCutHP4smUyGkSNHvtIxbG1tIZFISvQuJSQklOgxKubo6FhqfalUqkpeyqpTfMwLFy4gISEB3t7equ2FhYU4fvw4Vq9ejby8PEgkkhLnlsvlkMvlJcplMhk/BC8wpnM9JGYpsOroHXy+9xocLE3Qp2npbVwRSoOi9pIYGEDCttAb/GzoD7aF/mBb6N7L3P9XflfdtWvXsHv3bvz3v/991UPB0NAQ3t7eJbotg4OD0alTp1L36dixY4n6hw4dgo+Pj+pGlFWn+Ji9evXC5cuXERkZqfry8fHBO++8g8jIyFKTJnp1AX0a4S2fulCKwIf/C8ffdxJ1HRIREVG5Kpw4nTt3Dq1atULz5s3x5ptvYuzYsaptx48fh4mJCfbu3fvSxw0ICMAPP/yAzZs34/r165gxYwZiYmIwceJEAEXDY++9956q/sSJE3H//n0EBATg+vXr2Lx5MzZt2oSZM2eq6hQPwy1ZsgQ3btzAkiVLcPjwYdU6Tebm5vDy8lL7MjU1hY2NDby8vCp2g+iFBEHA18Obo29TB+QXKDFh63mci07W1tGf/smn6oiISHsqlDhdvXoVPXv2RFRUFGbMmIEBAwaobe/atStsbW3x66+/vvSx/fz8EBgYiEWLFqFVq1Y4fvw4goKC4O7uDgCIi4tTW9PJ09MTQUFBOHbsGFq1aoUvvvgCK1euVHuir1OnTvjll1/w448/okWLFtiyZQt27NiB9u3bV+TySYukEgOsers1fBvZIUdRiH/9eI6rixMRkd6q0Byn+fPnAyiaG9SgQQMsXLgQBw4cUG0XBAEdO3Z8qSfUnjVp0iRMmjSp1G1btmwpUebr64vw8PByjzly5MiXmoN17NgxjevSq5FLJVj/rjf+teUsTt9Lxnubz+KXf3dAEyeLih9U4DpORESkfRXqcQoNDcWIESPQoEGDMuu4ubkhLi6uwoFR7WJsKMEPY9qijZsV0nIUePeHM7j1OEPXYREREampUOKUkZEBe3v7cuvk5uaisJCv1SDNmcml+PFf7eDlYoGkrHyM2nAa1x5p/oho6djjRERE2lOhxMnV1RVXrlwpt86FCxdQv379CgVFtZelsQw/jW+PFnUtkZyVj9EbT+PSw1Rdh0VERASggonT4MGDcejQIRw9erTU7Tt37sTp06cxbNiwV4mNaikrE0P8NKG9atjunY1ncOF+yssdhHOciIioElQocZo7dy6cnJwwYMAA/Pvf/8b58+cBAGvXroW/vz/efvtteHh4qK2qTfQyLIxk2Da+Pdp5WiMjrwDvbTqDM/eSdB0WERHVchVKnOzs7BAaGgofHx/88MMP2L9/P0RRxOTJk/Hzzz+jbdu2OHr0KCwtLbUdL9UiZnIptvyrLTo3sEFWfiHe23wWh6891nBvruNERETaV+FXrtSrVw9///03IiMjcfr0aSQnJ8PCwgLt27dH27ZttRkj1WImhlJsGtMWH/4cjiM3EvB/P13A4jea4y0fV12HRkREtdArv6uuVatWaNWqVYnyVatWISQkBLt3737VU1AtZySTYL2/N+bsvozfLjzErN8uITEzDx/41i/z5c8i5zgREVEleOV31ZUlPDwcf/zxR2UdnmoZmcQAS0e2wETfoic1vz14E4v+vAalkokRERFVnUpLnIi0TRAEzB7QGJ8NbgoA+PHvaEzZHoFcRWnrhXGOExERaR8TJ6p2xnfxxIpRrSCTCNh/OQ5+G04jISNX12EREVEtwMSJqqWhrVzw3/HtYWUiw8UHqRi+5hSuxz2zyjjnOBERUSVg4kTVVod6Nvh9Umd42poiNjUHI9edQsiNBF2HRURENRgTJ6rWPG1N8fukTuhYr2itp/Fbz+GHE/fAOU5ERFQZNF6OYODAgS914MuXL790MEQVYWViiK3j2uGzPVew4/wDfLn/Oho5JKGbrgMjIqIaR+PE6eDBgy998LLW2CHSNkOpAb4Z0RxNnMzx5f7riE7KQTcpUHDlD0hS7uk6vFfTsC/QcpSuoyAiIrxE4hQVFVWZcRC9MkEQMLazJ5o6WyJy225ABOQpt4CUW7oO7dXcCGLiRESkJzROnNzd3SszDiKtaedpDed/L8K3G62QnZcHQQC6NbSDbyM7GFSnXtC8DCDkS6AgR9eREBHRU6/8yhUifeRga4f6zbvivNID2889xI/XgQ551lgxqjUcLIx0HZ5mMhOKEiciItIbfKqOaiypAbBoSFP8x68lTAwlOH0vGQNWnEDIzeqyZMEzvWNcj4qISC8wcaIab3jruvhzShc0dbJAclY+/vXjOXwddB35BUpdh1Y+gYkTEZG+YeJEtUI9OzPsntQJYzt5AAA2HL+HN9efwr0nmboNrFzPzsdi4kREpA+YOFGtYSSTYMGQZvje3xuWxjJcfJiGgStPYFtYNER97NFhjxMRkd5h4kS1Tr9mjjgwrSs6N7BBrkKJz/+4ivc2n0Vcmj4/vcbEiYhIHzBxolrJ2coY/x3XHguHNIORzAAnbiei73+OY09ErP70PrHHiYhI7zBxolrLwEDAmE4e2D+1K1q6WiEjtwDTd0Ri4k8X8Dg9V9fhgXOciIj0DxMnqvXq25lh18SO+KhPI0gNBPx19TF6Lw/F9rMxUCp1mLCwx4mISO8wcSICIJUYYEqvhtg3pQta1rVERm4B5uy+jNEbT+vwyTv2OBER6RsmTkTPaOJkgd2TOuOzwU1hLJPgTFQy+q84gTUhd6AorOJ1n9jjRESkd5g4ET1HYiBgfBdPHJrRDd0a2SG/QImlf93EwBUncOpuYhVGwh4nIiJ9w8SJqAyu1ibY+q+2+I9fS1ibGuJ2Qibe3ngGk/8XXjVLF7DHiYhI7zBxIiqHIAgY3rouQj7qjvc6usNAAP68FIdey0KxPvRuJb+2hT1ORET6hokTkQYsTWRYNNQL+6Z0gbd7HWTnF+KbAzfQf8VxhNxMqJy1n9jjRESkd5g4Eb2EZs6W+G1iRyx7syVszeS49yQL//rxHN7bfBbXHqVr+WzscSIi0jdMnIhekiAIGOFdF0dn+uL9rp4wlBStPD5o1Ql8/OtF7S2eKTzz8WSPExGRXmDiRFRBFkYyfDqoKQ4H+GJwCyeIIvDrhYfovvQYlgffQlZewaudQGCPExGRvmHiRPSK3GxMsPrtNtg9qRO83esgR1GIlUduw3dpCDafjEKuorCCR+YcJyIifcPEiUhL2rjVwW8TO2LdO23gbmOCxMx8LPrzGnp8dwz/OxPz8gtoqvU4ERGRPmDiRKRFgiBgQHMnHA7wxdfDm8PJ0ghxabmY+/tl9FoWit3hD1Go8fvv2ONERKRvmDgRVQKZxABvt3dDyMzu+HxwU9iaGSImORsBOy+iX+Bx/BEZ++IEinOciIj0DhMnokpkJJNgXBdPHJ/VA7P6vwZLYxnuJGRi2i+R6L08FDvPPSh7EU2u40REpHeYOBFVARNDKSZ1b4ATn/RAQJ9GsDKRISoxC7N2XUKP745hW1j0CyaRM3EiItIHTJyIqpCFkQxTezXE35/0xKcDm8DOXI7Y1Bx8/sdVdFkSgu9D7yIjV/HMHk97ndjjRESkF5g4EemAqVyK97vVw4lZPfDF0GZwsTJGYmYeFh+4gU6Lj+Kr/dcQm5rzzHAdEyciIn0g1XUARLWZkUwC/44eGNXODXsiYrE+9C7uPsnCxhNR2Px3NG4ZAhIAuPUXYFxH1+FWmGBgCEH5iguCEhHpASZORHpAJjHAmz6uGNGmLkJvPcHGE/dw6m4SFKIEEkEJ7Juq6xBfiRRAQ6cRAIboOhQiolfCxIlIjxgYCOjR2B49Gtvj6qM0/PXH+3CJOwLx6VCdXGoAO3Mj2JoZQmZQTUba0x4C6Q9hlJ+s60iIiF4ZEyciPdXM2RLNPliM+LRcbAuLxs9nYpCWrQCyAcMkAwxu4YT3OnqgZV1LCPq8yvjxpcDRLyFwnhYR1QBMnIj0nKOlEWb1b4wpPRti78VYbAu7j6uP0rE7PBa7w2PR3MUS/h3c8XpLZxgbSnQdbik4wZ2Iag4mTkTVhLGhBH5t3fCWjysiH6Tiv6fv489Lcbgcm4ZZuy7hq6DrGN7aBW/5uKKps4Wuw/2HUDSkKHBJBSKqAZg4EVUzgiCgtVsdtHarg3mDmmLn+Qf46fR9PEzJwZZT0dhyKhpeLhbw83HFkJYusDSR6Trgoj/Y40RENQATJ6JqzNrUEBN96+P9rvVw4vYT/Hr+IQ5di8eV2HRcib2KL/dfR38vR7zl44qO9WxgYKCDuVBC8SR2Jk5EVP0xcSKqASQGArq/Zo/ur9kjOSsfv0fEYue5B7j5OAN/RD7CH5GPULeOMd5o7YKhrV1Q386s6oLjUB0R1SBMnIhqGGtTQ4zv4olxnT1w6WEadp5/gL2Rj/AwJQcrj97ByqN30KKuJYa1csHglk6wNzeq5Ig4OZyIag4mTkQ1lCAIaOlqhZauVpg3qCn+uhqPPZGxOHE7EZcepuHSwzR8uf8aOjewxfDWLujbzBFm8kr4K6G4xwlK7R+biKiKMXEiqgWMDSUY1toFw1q7IDEzD/svxeH3iFhEPkjFiduJOHE7EUayy+jdxAGDmjuh+2v22lvaoHiOEzuciKgGYOJEVMvYmskxppMHxnTyQHRi1tM5ULG4l5iFPy/F4c9LcTCWSdCzsT0GNndCj8Z2MDF8hb8qVE/VsceJiKo/Jk5EtZiHrSmm9W6Iqb0a4NLDNARdjsP+y3F4mJKD/U+/N5IZoMdr9hjQ3Am9GtvD9GWH8/hUHRHVIEyciEhtPtTsAY1xOTYNQZfjEXQ5DjHJ2ThwJR4HrsRDLjVAt0Z26NPEAT2b2MPWTK75OSoxfiKiqqKXbwldu3YtPD09YWRkBG9vb5w4caLc+qGhofD29oaRkRHq1auH9evXl6iza9cuNG3aFHK5HE2bNsXvv/+utn3x4sVo27YtzM3NYW9vj2HDhuHmzZtavS6i6kAQBLSoW5RAhX7cHX9O6YJJ3evDw8YEeQVKBF97jFm7LqHtV4cxYt0prDt2F3cSMiCWtdyAao4Th+qIqPrTu8Rpx44dmD59Oj799FNERESga9euGDBgAGJiYkqtHxUVhYEDB6Jr166IiIjA3LlzMXXqVOzatUtVJywsDH5+fvD398fFixfh7++Pt956C2fOnFHVCQ0NxYcffojTp08jODgYBQUF6Nu3L7Kysir9mon0lSAI8HKxxKz+jREyszuCpnbFjN6N0NzFEqIIXLifgiUHb6D38uPouSwUX+2/hjP3klBQ+EySpHqqjkN1RFT9CWKZ/0zUjfbt26NNmzZYt26dqqxJkyYYNmwYFi9eXKL+J598gr179+L69euqsokTJ+LixYsICwsDAPj5+SE9PR0HDhxQ1enfvz/q1KmD7du3lxrHkydPYG9vj9DQUHTr1k2j2NPT02FpaYnExETY2NhotA9VDoVCgaCgIAwcOBAymY5fOVJDxaXl4PD1BBy+9hhhd5OQ/0yyZGUiQ7eGdvBtZIc+uQdgETwTcZatYTs5mO2hY/xs6A+2hf4o/v2dlpYGC4vy3/WpV3Oc8vPzceHCBcyePVutvG/fvjh16lSp+4SFhaFv375qZf369cOmTZugUCggk8kQFhaGGTNmlKgTGBhYZixpaWkAAGtr6zLr5OXlIS8vT/Vzeno6gKIPg0KhKHM/qnzF95/tUHlsTaQY5e2MUd7OyMwrwMk7STh6IwEhNxORmq3A3ouPsPfiI7wluYlvZYBZ+h1kbBoGS2MZBKF6zngSbRtC2fuLZya8Vz/8bOgPtoX+eJk20KvEKTExEYWFhXBwcFArd3BwQHx8fKn7xMfHl1q/oKAAiYmJcHJyKrNOWccURREBAQHo0qULvLy8yox38eLFWLhwYYnykJAQmJiYlLkfVZ3g4GBdh1CrdDcGurYEojOAG6kGuJ4qICGnDgDAXMwA4k/qOMJXdO8oQtPdkGHsqutIXhk/G/qDbaF72dnZGtfVq8Sp2PP/GhVFsdx/oZZW//nylznm5MmTcenSJZw8Wf5f8nPmzEFAQIDq5/T0dLi6uqJHjx4cqtMxhUKB4OBg9OnTh13gOpaU0Q3HTnng0uVLSMiXIiuvUG27nZkhGtiboYG9Kerbmb38cgdVRBL8KYScFHTr2A6ic2tdh1Nh/GzoD7aF/igeMdKEXv0NZWtrC4lEUqInKCEhoUSPUTFHR8dS60ulUlXyUlad0o45ZcoU7N27F8ePH0fdunXLjVcul0MuL/k4tkwm44dAT7AtdM/RWgabfm8hrdAM/9d/AG4+ycaxm08QeusJImJSoEwHkA7gTlH9pk4W6FTfBp0b2KKtp3XlvAamIkK/AXJSIJVIgBrw/xQ/G/qDbaF7L3P/9eRvpCKGhobw9vZGcHAwhg8frioPDg7G0KFDS92nY8eO2Ldvn1rZoUOH4OPjo7oRHTt2RHBwsNo8p0OHDqFTp06qn0VRxJQpU/D777/j2LFj8PT01OalEREAA4OipQ5a1LXC1F4NkZ6rwJl7yfj7TiLC7ibh5uMMXItLx7W4dPxwMgpSg6L1pTrXt0GH+jZo7VpHe6+Cefngi/4UC8uvR0Q1ml4lTgAQEBAAf39/+Pj4oGPHjtiwYQNiYmIwceJEAEXDY7Gxsdi2bRuAoifoVq9ejYCAALz//vsICwvDpk2b1J6WmzZtGrp164YlS5Zg6NCh+OOPP3D48GG1obgPP/wQ//vf//DHH3/A3Nxc1UNlaWkJY2PjKrwDRLWHhZEMfZo6oE/Tot7fJxl5OHW3KIn6+24iHiTn4ML9FFy4n4KVR+9AJhHQ3MUSbT2t0c7DGj7u1rA0qaJ/qXM9KiKCHiZOfn5+SEpKwqJFixAXFwcvLy8EBQXB3d0dABAXF6e2ppOnpyeCgoIwY8YMrFmzBs7Ozli5ciVGjBihqtOpUyf88ssvmDdvHj777DPUr18fO3bsQPv27VV1ipc/6N69u1o8P/74I8aOHVt5F0xEKnbmcgxt5YKhrVwAAA+Ss3HqbiL+vpOEM1FJeJyeh/CYVITHpOL70HsQBOA1B3O087RGWw9rtPO0hoOFUeUEJzzt6VKyx4moNtO7xAkAJk2ahEmTJpW6bcuWLSXKfH19ER4eXu4xR44ciZEjR5a5Xc+WsyIiAK7WJvCzdoNfWzeIoogHyTk4G52Mc1HJOBedjHuJWbgRn4Eb8RnYFnYfAOBmbYK2HtZo426FNm510MjBHBIDLSx/wB4nIoKeJk5ERM8TBAFuNiZwszHBSO+iBzcSMnJxPjoFZ58mUtfi0hGTnI2Y5GzsCn8IADA1lKClqxVauxUlUq1crWDzEu/YUzF42uPEOU5EtRoTJyKqtuzNjTCwuRMGNncCAKTnKormREWnIOJBCiJjUpGVX4hTd5Nw6m6Saj93GxO0caujSqZeczSHTPKCRS3Z40REYOJERDWIhZEMPV6zR4/X7AEAhUoRtxMyEBGTivD7KYh4kIo7CZm4n5SN+0nZ+D0iFgAglxqgqbMFWrhYonldK7Soa4n6dmbqQ3zFiZOSiRNRbcbEiYhqLImBgMaOFmjsaIHR7dwAAGnZCkQ+TEVETArCY1IRGZOC9NwCRMSkIiImFUDRXCljmQTNnC3QvK4lWtS1xIBCwAhgjxNRLcfEiYhqFUsTGXwbFb2AGACUShFRSVm4EpuGSw/TcDk2DVdj05CVX4jz91Nw/n4KAMDTMAutDADl/94CYAAIQHF/VHV6857UQIJG9kMADNR1KETVEhMnIqrVDAwE1LczQ307M9UyCIVKEVGJmbhcnEw9TENkXGO0wl0YQARQCFTTB3GFwkI4pZ7XdRhE1RYTJyKi50gMBDSwN0cDe3MMb130BF+hsiPu3o/G9UepuB6XhluPM3E7IRNZeQWlHsPJ0ggN7c3RyMEMDe3N0MjRHM6WxjDQxtIIFRV9Etg1HgI43EhUUUyciIg0IDEQUN/TE/U9gcFPy5RKEQ9TclSvibn+9OthSg6epAGX0gqA26kAUgEULY3Q0MEcrzmYo6GDGRo5mOM1R3PYm8vLfZG51pjaAgAEztMiqjAmTkREFWRg8M/aUv29HFXlaTkK3FBLpjJw83EGsvILEfkgFZEPUtWOY2EkRSMHczRyNEcjezPV97YVWW+q3ICL/soXuBYVUYUxcSIi0jJLYxna17NB+3o2qrKCQiXuJWbh1uOMomG+x0XJ1P2kbKTnFqhNRC9mbWqIhvZmeM3RHA3tzVDv6VwsB4sK9lA9fW0Mh+qIKo6JExFRFZBKDIp6khzM1cpzFYW49yQLtxMycOtxBm7GZ+J2QgZikrORnJWPM1HJOBOVrLaPqaHkaRJlqkqm6tubwsPGFEYySdlBqHqcqunMdiI9wMSJiEiHjGQSNHW2QFNnC7XynPxC3EnIfNpDlYG7TzJx90kWYpKzkZVfiMuxRUsnPEsQgLp1jFHP9p9kquh7U9iZyyEYFC3iyaE6oopj4kREpIeMDSVoXtcSzetaqpXnFygRk5yFu0+yipKphCzcS8zE3YRMpOcW4EFyDh4k5yD01hO1/UwMJehpGY/VACSKTNzf8j7MjQ1hbiSFsUyCqpibrlUmtkDXjwC5ma4joVqGiRMRUTViKDVQLZXwLFEUkZSVj7sJmbiXmPXPn08y8SA5G9n5hYh4AsAIkCMPDWJ/180FaJNdY6Cln66joFqGiRMRUQ0gCAJszeSwNZOrTUoHinqpHqZkIzopC4euKpB6+wwEuSlSsxVIyymAspw5T1IDAZbGMliayGBlLIOlsSEsjKWwMjaEpbEMRrIXvBy5Mlz+DXhyHchLr/pzU63HxImIqIYzlBqgnl3RU3mK+mMRFGSPgQMHQiaTQVGoxKPUHEQnZeN+UhaiE5/+mZSFB8k5yFcoAQWAMnIUcyMpXOuYoG4dY7ham8C1+E/rojITw0r4NfP4alHipORcLap6TJyIiGoxmcQA7jamcLcxBWCntq1QKeJRag7uJ2UjJjkbD1Ky8SA5Gw9TcvAwJRuJmfnIyC1QLQBaGhtTQ9R9mlDVfZpgOVsZwdnKGM5WxrAwkr180JKn+ygVL78v0Sti4kRERKWSGAiq3qPSZOcX4GFKjiqZeqBKrnLwICUbGbkFSMrKR1JWPi4+t+hnMXO5FM5WxnB6mky5WD1NrCyLEisHCyMYSp8bDny6rAKUpb/uhqgyMXEiIqIKMTGUlro2VbG0HMXTpOqfZOpRai4epebgUVoOUrMVyMgrwM2ni4GWRhAAe3N5UQ+VZVFSNfJJDl4DkBIVDgPjXTAzkkJS7R4LLHrhslF+yosrkl5h4kRERJXC0lgGSxdLeLlYlro9O7/gn0Sq+CstV+37/AIlHqfn4XF6HiKevvPPWZqF16RAnbt7gbt7q/CKtEsKoJvMGoC/rkOhl8DEiYiIdMLEUIoG9mZoYF/6WkzFSyz8k1gVJVVxCSMQEfcEBgXZUBQqoek66BJBgKHUADJJ0ZehVICh6vvicgECqqD3SlkAxJ6HsSIZCmUhgArM9SKdYOJERER66dklFlrUtXpmS1MARes3FRQqkZSVj/i0XMSn5+Jxem6J7x+n5yEz7+l8qLwXnbNoQrutmRz2FkawM5PDzrzoy95c/XszubRi7wwEgLwMYHHdou+VCgBGFTsOVTkmTkREVG1JJQZwsDCCg4URWpZTLzOvAI/Tc/H4aVIVr/Z9Hh6n5eJJZh4KlSISM/ORmJmPG/Glz7sqZiQzKEqkzOSwNzdSJVXPJ1m2ZnLIJM9NcJcY/vN9IZ8OrE6YOBERUY1nJpfC7OkLkctSqBSRkp2PJxl5SMjIe/pnLp6ovs9D4tPvM/IKkKtQql5x8yJWJjLYmBrCxkwOWzND2JrIsOjpttv7A2FiZglTuRSmcknRK3CqYrhQW2TGQLNhgFHpc9lqGiZOREREKFp+oXhosIlT+XWz8wuQmJGPJ5m5SEjPw5PMp8nVs99n5CIxMx+FShGp2QqkZitw90mW6hiz5XKYCHloen1FJV9ZFUiJBnrP13UUVYKJExER0UsyMZTCzUYKN5vS17gqplSKSM7OR3JWPhIz85CUmY+kzDwkZeVjb8zHcI47DAOpHHmFSuQVFKKgUNOp7v8onvSu+pI892cp30sMtNSj9eQW8PgykPlYO8erBpg4ERERVRKDZ3qxnl/vSqGYjqCgRqrX3wBAXkEhkrPyixKsrKdJVmY+ErPUk66kzKJELK9AWXSw/JeLy1gmgbWpIaxMZLA2NUQdE0PUMZGhTvH3poawNvlnu7WpIYxkkpIHClsL/DUHKHjBrPsahIkTERGRnpBLJXCyNIaTpfEL64qiiOz8QiRl5iM5Ox8p2flIySrq3UrNVhSVZRWX//NzgVJEjqIQsak5iE198fysYkYyA1gZFyVTFsZFL33un/MEbwBIuXsOD7d9BCOpBHKZAeRSAxjJJJBLJZBLDaD365Nm5WpclYkTERFRNSQIwtMJ5S8eMiwmiiIy8wr+SaQ0SLZSs/OhKBSRq1AiXlH0JGIxQwMF3jAE6uTcR517P1TWpVa+PM2HSJk4ERER1RKCIMDcSAZzI1mFkq20HAVSc/KL/sxWIDPTHSHRBZDmPEF+gRJ5CiXyCpTILyxEnkKJAuXLz9l6loFB0SKlcqkBZM/M1ZI/O3errLlcUgNIDTRc0DQ7D0CgRjExcSIiIqIyPZtsle6bMvfNKyhEWo4CadkKVbJVlHwpkJad/8/3z2wr+j4fr5hzAQCkBgLMjaSwMJbBwkgGC2MpLIxkRWVGsqflUkhNc8HEiYiIiHRKLpXA3lwCe/OXWxldFEVk5BUgI7cA6TmKoq+n32fk/vN9eq4C6TkFSM9VFNXN/aduoVJEgVJESrYCKdnlLzKqzMvWODYmTkRERKRXBEEo6hEyksHF6sUT5Z9XPHFePZl6Lsl6piwxJRUPNDw2EyciIiKqUZ6dOO9o+eLervT0dOycotmxDV5chYiIiIgAJk5EREREGmPiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhvUyc1q5dC09PTxgZGcHb2xsnTpwot35oaCi8vb1hZGSEevXqYf369SXq7Nq1C02bNoVcLkfTpk3x+++/v/J5iYiIqHbRu8Rpx44dmD59Oj799FNERESga9euGDBgAGJiYkqtHxUVhYEDB6Jr166IiIjA3LlzMXXqVOzatUtVJywsDH5+fvD398fFixfh7++Pt956C2fOnKnweYmIiKj20bvEafny5Rg/fjwmTJiAJk2aIDAwEK6urli3bl2p9devXw83NzcEBgaiSZMmmDBhAsaNG4fvvvtOVScwMBB9+vTBnDlz0LhxY8yZMwe9evVCYGBghc9LREREtY9eJU75+fm4cOEC+vbtq1bet29fnDp1qtR9wsLCStTv168fzp8/D4VCUW6d4mNW5LxERERU+0h1HcCzEhMTUVhYCAcHB7VyBwcHxMfHl7pPfHx8qfULCgqQmJgIJyenMusUH7Mi5wWAvLw85OXlqX5OS0sDACQnJ7/gSqmyKRQKZGdnIykpCTKZTNfh1HpsD/3BttAfbAv9kZGRAQAQRfGFdfUqcSomCILaz6Iolih7Uf3nyzU55sued/HixVi4cGGJ8kaNGpW5DxEREemnjIwMWFpalltHrxInW1tbSCSSEr08CQkJJXqDijk6OpZaXyqVwsbGptw6xcesyHkBYM6cOQgICFD9nJqaCnd3d8TExLzwxlPlSk9Ph6urKx48eAALCwtdh1PrsT30B9tCf7At9IcoisjIyICzs/ML6+pV4mRoaAhvb28EBwdj+PDhqvLg4GAMHTq01H06duyIffv2qZUdOnQIPj4+qq7Pjh07Ijg4GDNmzFCr06lTpwqfFwDkcjnkcnmJcktLS34I9ISFhQXbQo+wPfQH20J/sC30g6YdHnqVOAFAQEAA/P394ePjg44dO2LDhg2IiYnBxIkTART18sTGxmLbtm0AgIkTJ2L16tUICAjA+++/j7CwMGzatAnbt29XHXPatGno1q0blixZgqFDh+KPP/7A4cOHcfLkSY3PS0RERKR3iZOfnx+SkpKwaNEixMXFwcvLC0FBQXB3dwcAxMXFqa2t5OnpiaCgIMyYMQNr1qyBs7MzVq5ciREjRqjqdOrUCb/88gvmzZuHzz77DPXr18eOHTvQvn17jc9LREREJIiaTCEnjeTl5WHx4sWYM2dOqUN4VHXYFvqF7aE/2Bb6g21RPTFxIiIiItKQXi2ASURERKTPmDgRERERaYiJExEREZGGmDgRERERaYiJkxatXbsWnp6eMDIygre3N06cOKHrkKqNxYsXo23btjA3N4e9vT2GDRuGmzdvqtURRRELFiyAs7MzjI2N0b17d1y9elWtTl5eHqZMmQJbW1uYmppiyJAhePjwoVqdlJQU+Pv7w9LSEpaWlvD390dqaqpanZiYGLz++uswNTWFra0tpk6divz8/Eq5dn23ePFiCIKA6dOnq8rYFlUrNjYW7777LmxsbGBiYoJWrVrhwoULqu1sj6pRUFCAefPmwdPTE8bGxqhXrx4WLVoEpVKpqsO2qAVE0opffvlFlMlk4saNG8Vr166J06ZNE01NTcX79+/rOrRqoV+/fuKPP/4oXrlyRYyMjBQHDRokurm5iZmZmao633zzjWhubi7u2rVLvHz5sujn5yc6OTmJ6enpqjoTJ04UXVxcxODgYDE8PFzs0aOH2LJlS7GgoEBVp3///qKXl5d46tQp8dSpU6KXl5c4ePBg1faCggLRy8tL7NGjhxgeHi4GBweLzs7O4uTJk6vmZuiRs2fPih4eHmKLFi3EadOmqcrZFlUnOTlZdHd3F8eOHSueOXNGjIqKEg8fPizeuXNHVYftUTW+/PJL0cbGRvzzzz/FqKgo8ddffxXNzMzEwMBAVR22Rc3HxElL2rVrJ06cOFGtrHHjxuLs2bN1FFH1lpCQIAIQQ0NDRVEURaVSKTo6OorffPONqk5ubq5oaWkprl+/XhRFUUxNTRVlMpn4yy+/qOrExsaKBgYG4sGDB0VRFMVr166JAMTTp0+r6oSFhYkAxBs3boiiKIpBQUGigYGBGBsbq6qzfft2US6Xi2lpaZV30XomIyNDbNiwoRgcHCz6+vqqEie2RdX65JNPxC5dupS5ne1RdQYNGiSOGzdOreyNN94Q3333XVEU2Ra1BYfqtCA/Px8XLlxA37591cr79u2LU6dO6Siq6i0tLQ0AYG1tDQCIiopCfHy82j2Wy+Xw9fVV3eMLFy5AoVCo1XF2doaXl5eqTlhYGCwtLdVWje/QoQMsLS3V6nh5eam97LFfv37Iy8tTGx6p6T788EMMGjQIvXv3VitnW1StvXv3wsfHB2+++Sbs7e3RunVrbNy4UbWd7VF1unTpgiNHjuDWrVsAgIsXL+LkyZMYOHAgALZFbaF3r1ypjhITE1FYWAgHBwe1cgcHB8THx+soqupLFEUEBASgS5cu8PLyAgDVfSztHt+/f19Vx9DQEHXq1ClRp3j/+Ph42Nvblzinvb29Wp3nz1OnTh0YGhrWmvb85ZdfEB4ejnPnzpXYxraoWvfu3cO6desQEBCAuXPn4uzZs5g6dSrkcjnee+89tkcV+uSTT5CWlobGjRtDIpGgsLAQX331FUaPHg2An43agomTFgmCoPazKIolyujFJk+ejEuXLqm9hLlYRe7x83VKq1+ROjXVgwcPMG3aNBw6dAhGRkZl1mNbVA2lUgkfHx98/fXXAIDWrVvj6tWrWLduHd577z1VPbZH5duxYwd++ukn/O9//0OzZs0QGRmJ6dOnw9nZGWPGjFHVY1vUbByq0wJbW1tIJJISWX5CQkKJfxFQ+aZMmYK9e/ciJCQEdevWVZU7OjoCQLn32NHREfn5+UhJSSm3zuPHj0uc98mTJ2p1nj9PSkoKFApFrWjPCxcuICEhAd7e3pBKpZBKpQgNDcXKlSshlUpV94BtUTWcnJzQtGlTtbImTZqoXnbOz0bV+fjjjzF79myMGjUKzZs3h7+/P2bMmIHFixcDYFvUFkyctMDQ0BDe3t4IDg5WKw8ODkanTp10FFX1IooiJk+ejN27d+Po0aPw9PRU2+7p6QlHR0e1e5yfn4/Q0FDVPfb29oZMJlOrExcXhytXrqjqdOzYEWlpaTh79qyqzpkzZ5CWlqZW58qVK4iLi1PVOXToEORyOby9vbV/8XqmV69euHz5MiIjI1VfPj4+eOeddxAZGYl69eqxLapQ586dSyzNcevWLbi7uwPgZ6MqZWdnw8BA/demRCJRLUfAtqglqngyeo1VvBzBpk2bxGvXronTp08XTU1NxejoaF2HVi188MEHoqWlpXjs2DExLi5O9ZWdna2q880334iWlpbi7t27xcuXL4ujR48u9THfunXriocPHxbDw8PFnj17lvqYb4sWLcSwsDAxLCxMbN68eamP+fbq1UsMDw8XDx8+LNatW7dWP+b77FN1osi2qEpnz54VpVKp+NVXX4m3b98Wf/75Z9HExET86aefVHXYHlVjzJgxoouLi2o5gt27d4u2trbirFmzVHXYFjUfEyctWrNmjeju7i4aGhqKbdq0UT1KTy8GoNSvH3/8UVVHqVSK8+fPFx0dHUW5XC5269ZNvHz5stpxcnJyxMmTJ4vW1taisbGxOHjwYDEmJkatTlJSkvjOO++I5ubmorm5ufjOO++IKSkpanXu378vDho0SDQ2Nhatra3FyZMni7m5uZV1+Xrv+cSJbVG19u3bJ3p5eYlyuVxs3LixuGHDBrXtbI+qkZ6eLk6bNk10c3MTjYyMxHr16omffvqpmJeXp6rDtqj5BFEURV32eBERERFVF5zjRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiRERERKQhJk5EREREGmLiREQ1VnZ2Nr7++mu0adMGZmZmMDIyQt26ddG1a1fMmTMHd+/eVdX18PCAh4eH7oIlompBqusAiIgqQ0ZGBrp06YJLly6hQYMGePfdd2FlZYUHDx7g6tWr+Oabb1C/fn3Ur19f16ESUTXCxImIaqTAwEBcunQJ48ePx8aNGyEIgtr2qKgo5OXl6Sg6IqquOFRHRDVSWFgYAGDy5MklkiYA8PT0ROPGjREdHQ1BEHD//n3cv38fgiCovhYsWKC2z/Hjx/H666/D1tYWcrkcDRs2xLx585Cdna1W79ixY6r9jx8/Dl9fX5iZmcHa2hpvv/02Hj58WGnXTUSVi4kTEdVI1tbWAIA7d+6UW8/Kygrz58+HpaUlLC0tMX/+fNVX9+7dVfXWr1+P7t2749SpUxg8eDCmTp0KFxcXfPXVV+jTpw/y8/NLHPv06dPo06cPbGxsMHXqVLRr1w7bt29Hp06d8PjxY61eLxFVEZGIqAbas2ePCEC0sLAQP/nkE/HIkSNicnJymfXd3d1Fd3f3UrddvXpVlEqlYuvWrcWkpCS1bYsXLxYBiN99952qLCQkRAQgAhB/+OEHtfoLFy4UAYjjxo2r+MURkc4IoiiKOs3ciIgqydKlS7Fo0SJkZmaqyurXr4/+/ftj2rRpaNiwoaq8+Im66OjoEseZNm0aVq5ciRMnTqBLly5q25RKJRwdHeHm5obz588DKBqq69GjB1577TVcv35dbagwJycH7u7uyMzMRGpqKgwNDbV4xURU2Tg5nIhqrI8//hgTJ07EwYMHcerUKZw/fx5nzpzBmjVrsGnTJuzYsQNDhgx54XFOnz4NADh48CAOHz5cYrtMJsONGzdKlHfu3LnE/CpjY2N4e3vj4MGDuHXrFry8vCp4dUSkC0yciKhGMzc3x5tvvok333wTAJCWloa5c+di7dq1GD9+PGJjY1/Y65OcnAwA+Oqrr17q3Pb29qWWOzg4qGIhouqFk8OJqFaxtLTE6tWr4e7ujsTERFy+fPmF+1hYWAAA0tPTIYpimV/PS0hIKPV4xRPDLS0tX+FKiEgXmDgRUa0jCAJMTEzUyiQSCQoLC0ut3759ewD/DNlp6u+//y6RUOXk5ODChQswNjZGo0aNXup4RKR7TJyIqEb6/vvvce7cuVK37d69Gzdu3ICVlZVqjpG1tTUSExORm5tbov6kSZMglUoxZcoUPHjwoMT21NRURERElCi/efMmNm/erFa2dOlSPHnyBKNHj+bEcKJqiHOciKhGOnDgACZOnIgGDRqgc+fOcHZ2RmZmJiIjI3HixAkYGBhg7dq1kMvlAICePXvi/PnzeP3119G1a1cYGhqiS5cu6NKlC7y8vLB27Vp88MEHeO211zBw4EDUr18f6enpuHfvHkJDQzF27FisX79eLYa+ffti0qRJ2L9/Pxo3bozw8HD89ddfcHV1xddff62L20JEr4jLERBRjXTz5k3s3bsXwcHBuHPnDuLi4gAALi4u6NKlC6ZMmQJvb29V/czMTAQEBODPP//E48ePoVQqMX/+fLXVw8+dO4fly5fj+PHjePLkCSwtLeHm5oa+fftizJgxaNy4MYB/liOYP38+evbsiXnz5uHChQswNDRE//798e2338LV1bVK7wcRaQcTJyIiLXs2cXr+tS1EVL1xjhMRERGRhpg4EREREWmIiRMRERGRhjjHiYiIiEhD7HEiIiIi0hATJyIiIiINMXEiIiIi0hATJyIiIiINMXEiIiIi0hATJyIiIiINMXEiIiIi0hATJyIiIiINMXEiIiIi0tD/A5tqHJKQl0hTAAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# extra code – this cell plots power scheduling with staircase=True or False\n",
"\n",
"initial_learning_rate = 0.01\n",
"decay_rate = 1.0\n",
"decay_steps = 10_000\n",
"\n",
"steps = np.arange(100_000)\n",
"lrs = initial_learning_rate / (1 + decay_rate * steps / decay_steps)\n",
"lrs2 = initial_learning_rate / (1 + decay_rate * np.floor(steps / decay_steps))\n",
"\n",
"plt.plot(steps, lrs, \"-\", label=\"staircase=False\")\n",
"plt.plot(steps, lrs2, \"-\", label=\"staircase=True\")\n",
"plt.axis([0, steps.max(), 0, 0.0105])\n",
"plt.xlabel(\"Step\")\n",
"plt.ylabel(\"Learning Rate\")\n",
"plt.title(\"Power Scheduling\", fontsize=14)\n",
"plt.legend()\n",
"plt.grid(True)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Exponential Scheduling"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```python\n",
"learning_rate = initial_learning_rate * decay_rate ** (step / decay_steps)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {},
"outputs": [],
"source": [
"lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay(\n",
" initial_learning_rate=0.01,\n",
" decay_steps=20_000,\n",
" decay_rate=0.1,\n",
" staircase=False\n",
")\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=lr_schedule)"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.6916 - accuracy: 0.7632 - val_loss: 0.5030 - val_accuracy: 0.8254\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4832 - accuracy: 0.8311 - val_loss: 0.4601 - val_accuracy: 0.8358\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4372 - accuracy: 0.8449 - val_loss: 0.4256 - val_accuracy: 0.8524\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.4131 - accuracy: 0.8546 - val_loss: 0.4037 - val_accuracy: 0.8568\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3952 - accuracy: 0.8596 - val_loss: 0.3950 - val_accuracy: 0.8598\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3825 - accuracy: 0.8640 - val_loss: 0.4010 - val_accuracy: 0.8584\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3739 - accuracy: 0.8667 - val_loss: 0.3851 - val_accuracy: 0.8650\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.3664 - accuracy: 0.8696 - val_loss: 0.3811 - val_accuracy: 0.8616\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.3606 - accuracy: 0.8720 - val_loss: 0.3749 - val_accuracy: 0.8662\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.3555 - accuracy: 0.8743 - val_loss: 0.3706 - val_accuracy: 0.8662\n"
]
}
],
"source": [
"history_exponential_scheduling = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAHNCAYAAADolfQeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/SrBM8AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB99klEQVR4nO3dd1wU194G8GcLLB2kF2n23iB2xQpGTTTRRE1CNJZ7DdGgpFhiYklRY/TFEjUmRjRFzVVjiWjEAliwIsZuVBRFEBGlCizsvH8gGze74LIs7ALP9/Ph3nD27MxvZ0Qe55w5IxIEQQARERERPZfY0AUQERER1RQMTkRERERaYnAiIiIi0hKDExEREZGWGJyIiIiItMTgRERERKQlBiciIiIiLTE4EREREWmJwYmIiIhISwxORFRr+fj4wMfHp1LbmDNnDkQiEaKjo/VSU2Xo4/PoSl/HQdNniIiIgEgkQkRERKW2TVQdGJyIqsmtW7cgEonK/WrXrp2hy6xRxowZA5FIhFu3bhm6FCVBEPDzzz+jT58+cHBwgKmpKVxcXNC+fXuEhIQgJibG0CUSUSVIDV0AUV3TsGFDvPXWWxpfc3V1reZqarcDBw5U+z7Hjh2LiIgI1KtXD4MHD4a7uzvS09Nx7do1rF27FllZWQgICKj2uozZK6+8gs6dO8PNzc3QpRA9F4MTUTVr1KgR5syZY+gy6oSGDRtW6/4OHz6MiIgItGvXDjExMbCxsVF5/fHjx7h06VK11lQT2NrawtbW1tBlEGmFQ3VERuqLL76ASCTC5MmT1V4rnW8ydepUtbbo6Gh8//33aNmyJczMzODl5YUZM2YgPz9f437++OMP9O7dG7a2tjA3N0e7du0QHh6O4uJilX6lQ41jxozBzZs3MXz4cNSrVw+Wlpbo168fzp07p3H7aWlpmDp1Kho1agSZTAZHR0cMGzYMFy5cUOtbOv8lNzcXYWFh8PDwgEwmQ5s2bbBlyxa1vuvXrwcA+Pr6Koc7e/Xqpba9Z927dw+zZ89G586d4ezsDJlMBh8fH4SEhCAtLU3jZ9BWXFwcAGD06NFqoQkA7Ozs0LVrV7X2wsJCLF26FB07doS1tTWsrKzQokULhIWF4dGjR2r9tTk+z257yZIl6NChAywtLWFtbY0ePXpg586dGvvfuXMHo0aNgr29PaysrBAQEIDY2FiNfcubmxQdHQ2RSKTVPxLK2k7p+Xzw4AHGjh0LZ2dnmJubo3PnzmXOtfrrr78wcOBAWFtbw9bWFgMHDsSFCxeMcliXaiZecSIyUjNnzkRUVBRWrFiBwMBAvPTSSwCAo0eP4osvvkCbNm2wYMECtfctXrwY0dHRGDFiBAYPHozIyEgsWLAAZ8+exZ49eyASiZR9ly5diilTpsDe3h5vvPEGLC0tsWvXLkydOhWHDx/Gli1bVPoDJQGqU6dOaNGiBcaOHYsbN25gx44d6N27Ny5fvgwXFxdl3xs3bqBXr15ITk5GYGAghg4dirS0NGzduhV//vknDhw4gE6dOqlsXy6XIzAwEBkZGXj11VeRl5eHTZs24fXXX8fevXsRGBgIAJgyZQoiIiJw7tw5hIaGws7ODgCeO3k6NjYWixcvRt++fdGpUyeYmJjg7NmzWLVqFf7880/Ex8frfPXD3t4eAHD9+nWt35Ofn4+goCDExsaicePGeOeddyCTyfD3339j9erVePvtt1GvXj1lf22PDwAUFBRgwIABiI6ORvv27TFu3DjI5XLs3r0bQ4YMwfLlyzFp0iRl/5SUFHTp0gXJyckICgpChw4dcPnyZfTv3x+9e/fW6ZhU1uPHj9GtWzfY2NjgzTffRFpaGjZv3oygoCCcOXMGrVq1UvY9d+4cevTogby8PLz66qto1KgRzpw5g+7du6Nt27YGqZ9qIYGIqkViYqIAQGjYsKEwe/ZsjV979uxReU9SUpJQr149wdHRUbh3757w+PFjwcfHRzA3NxcuXryo0nf27NkCAMHMzEy4cOGCsl0ulwv9+/cXAAgbNmxQtt+4cUOQSqWCs7OzkJSUpGwvKCgQAgICBADCTz/9pFY/AGHBggUq+541a5YAQJg/f75Ke9euXQWpVCrs27dPpf3q1auCtbW10Lp1a5V2b29vAYAwZMgQoaCgQNm+f/9+AYAQFBSk0n/06NECACExMVHteJduz9vbW6Xt/v37QnZ2tlrf9evXCwCEL774QqW99LgeOnRI4z6elZSUJFhbWwtisVh4++23hd9//13l2Gry0UcfCQCE4OBgoaioSOW1x48fq9Ra0eMzc+ZMAYAwZ84cQaFQKNuzsrIEf39/wdTUVEhOTla2lx7Pfx+D7777Tnnunz0O69atEwAI69atU/tchw4dEgAIs2fPVmnXdE7K2k7pPkNCQoTi4mJl+w8//CAAEP773/+q9O/evbsAQPjf//6n0l56Dsv7s0KkLQYnomrybPAo6ys0NFTtfVu2bBEACP369RNGjhwpABBWrlyp1q/0l8OECRPUXjt16pQAQOjbt6+ybd68eQIAYeHChWr94+Li1PqX1u/r66vyS+zZ11599VVlW3x8vABAGDdunMbjERYWJgAQzp8/r2wrDQY3b95U6+/t7S3Y29urtOkSnMqiUCgEGxsboVevXirtFQlOgiAIe/fuFTw9PVXOq5OTk/D6668LBw4cUOlbVFQk2NjYCLa2tkJGRsZzt12R41NcXCzUq1dPaNSokUpoKrVz504BgLB8+XJBEEoCs5mZmeDs7Cw8efJEpW9xcbHQpEkTgwQnS0tLtaArl8sFqVQqdOjQQdl269YtAYDQvn17tVpyc3MFe3t7BifSCw7VEVWzoKAg7N27V+v+w4YNw/jx4/HDDz8AAIYMGYJ33323zP49evRQa/P394e5uTkSEhKUbWfPngUAlTlBpTp37qzWv1Tbtm0hFqtOj6xfvz6AkmGVUsePHwcApKamapzncuXKFeX/PzvcYmdnB19fX7X+9evXV84hqqxt27bhu+++Q3x8PB49eqQyn+vevXuV2nZQUBBu3ryJ6OhoxMbG4syZMzhy5Ah+++03/Pbbb5gxYwa++uorACWfPSsrC/369VMZjiuPtsfn6tWrePToEdzd3TF37ly1/g8ePFDWUNo/Pz8fffr0gZmZmUpfsViMrl274tq1a9odBD1q3LgxrKysVNqkUilcXFxU/ryVzrHTNIfMwsICbdu2xaFDh6q0VqobGJyIaoBXX31VGZzee++9cvs6OzuX2Z6cnKz8PisrCwBU5iSV17+Upvk/UmnJXyXPBpCMjAwAwO7du7F79+4y683NzX3u9kv3oVAoytyOthYvXowPP/wQTk5OCAwMRP369WFubg4ACA8PR0FBQaX3IZVK0a9fP/Tr1w8AUFRUhIiICLz77ruYP38+hg8fjg4dOih/8Xt4eGi9bW2PT+nxv3jxIi5evFjm9kqPf2ZmJoCy//yU9eekqpX3eZ/981b659nJyUljf0PVT7UPgxORkcvIyMB//vMfWFlZQS6XY9KkSYiPj4elpaXG/mXdGZaWlqbyS6j0rq/79+/D29tbY39Nd4Zpq/S9/56AbEhFRUX4/PPP4e7ujoSEBJVfsoIg4Ouvv66S/UqlUowfPx6HDx/Ghg0bcOjQIXTo0EE5oV1TQK2s0uM/bNiwMu+4e1bpn42y/vzcv39fra30ymNRUZHaa6VBrLqUft7SK2n/pql+Il1wOQIiIzdhwgTcvXsXK1aswIIFC3Dt2jWEhoaW2f/w4cNqbadPn8aTJ09UViZv3749AGi8rfvkyZNq/Suq9G45fQ2vaSKRSABAbemEsqSnpyMzMxOdO3dWuzJReoyq0r/DbtOmTWFjY4NTp05pXHagMpo3bw4bGxucPn0acrn8uf2bNm0KMzMznD59Wm3pCoVCgWPHjqm9p3R4UVPwKx0Kri6ld81pqjMvL6/M5TKIKorBiciIff/999i2bRtGjBiB0aNHIzQ0FEFBQVi7dm2ZVxF++uknlaGZoqIizJw5E0DJ+kKl3njjDUilUixZskRlXo9cLsf06dMBlDzSRFcdO3ZEp06dsHHjRmzevFntdYVCUenHj5Te/n/37l2t+peuAxQfH4+8vDxl+6NHjzSul1VRe/fuxY4dOzRegbl27ZrynHXv3h1AyZWo//73v8jMzERoaKhaAMzMzEROTo5OtUilUrz77ru4ffs2PvzwQ43h6cKFC8orTKampnj99deRlpaGxYsXq/T74YcfNM5v6tChA0QiETZt2qQStv7++28sXbpUp7p15e3tjW7duuHs2bNqPxuLFi1SDl0SVRaH6oiq2fXr18tdFLD0tatXr2LKlCnw8vLC6tWrAUC5SGCbNm3wn//8B506dYKnp6fK+/v164fOnTtj5MiRsLe3R2RkJC5cuICgoCCVR700bNgQCxcuxAcffIA2bdrg9ddfh6WlJf744w9cuXIFQ4YMKfPRMNrauHEjevfujZEjRyI8PBx+fn4wMzNDUlIS4uLi8ODBgzIX5tRGnz598M033+C///0vXnvtNVhaWsLLywtvvPGGxv5isRghISFYvHgx2rZti5deeglZWVnYs2cPvL294e7urnMtQMlE66lTp8LR0RE9e/ZEw4YNIQgCrl+/jsjISBQWFuLdd99VWbtq3rx5OH78OH766SccP34cL774ImQyGW7evIm9e/fiyJEjOl/5mzt3LuLj47Fs2TLs3r0bAQEBcHJyQnJyMs6fP49z584hLi5OOa9pwYIFOHDgAGbNmoUjR46gffv2uHz5MiIjIxEYGIh9+/apbN/DwwMjRozApk2b4OfnhwEDBiAtLQ2///47BgwYgK1bt+p8LHWxfPly9OzZEyNHjsSwYcPQsGFDxMfH4/jx4+jZsydiY2PVbmwgqjBD39ZHVFdosxxB6Y9kQUGB0KFDB0EsFgsxMTFq29q1a5cAQOjZs6dyaYBnb5v/7rvvhBYtWggymUyoX7++MH36dCEvL09jXTt27BACAgIEa2trQSaTCa1btxYWL14syOVyjfWPHj1a43YACAEBAWrtGRkZwqxZs4RWrVoJ5ubmgpWVldC4cWPhjTfeELZt26bSt7zlA0rXlvq3r7/+WmjcuLFgYmKiVoOm7RUWFgpffvml0LhxY0EmkwleXl5CWFiYkJ2drbF/RZYjSEtLE77//nth+PDhQtOmTQVra2vBxMREcHNzEwYPHixs2bJF4/vy8/OFb775RmjXrp3yGLVo0UL44IMPhEePHlXq+BQVFQnfffed0K1bN8HGxkb5mQcMGCCsWrVKyMnJUel/+/ZtYcSIEYKdnZ1gYWEh9OjRQ4iJiSnzOOTm5gqTJ08WXFxcBJlMJrRp00b45Zdf9LYcgaY/U+Udi7NnzwpBQUGClZWVYG1tLbz44ovC+fPnhcGDBwsAVI4nkS5EgiAI1RnUiKhqzJkzB3PnzsWhQ4c0LjFAVFcVFxejYcOGePLkCSeJU6XxmiUREdUKRUVFSE9PV2tfsGABbt++jaFDh1Z/UVTrcI4TERHVCjk5OfDw8ED//v3RpEkTyOVynDhxAqdOnYKbm5tWDxwmeh4GJyIiqhUsLCwwbtw4HDx4ELGxscjPz4ebmxv++9//4tNPP4Wbm5uhS6RagHOciIiIiLTEOU5EREREWmJwIiIiItIS5zjpkUKhwL1792BtbQ2RSGTocoiIiEgLgiAgOzsb7u7uz10klcFJj+7du6e2ijMRERHVDHfu3EH9+vXL7cPgpEfW1tYAgMTEROUztMgw5HI59u3bh8DAQJiYmBi6nDqP58N48FwYD54L45GVlQVPT0/l7/HyMDjpUenwnLW1NWxsbAxcTd0ml8thYWEBGxsb/oVkBHg+jAfPhfHguTA+2kyz4eRwIiIiIi0xOBERERFpicGJiIiISEsMTkRERERaYnAiIiIi0hKDExEREZGWuBwBEZEByOVyFBcXG2S/UqkU+fn5Btk//YPnomqZmJhAIpHofbsMTkRE1SgrKwvp6ekoKCgwyP4FQYCrqyvu3LnDR0MZGM9F1RKJRLC1tYWrq6tejy+DExFRNcnKykJycjKsrKzg6OgIExOTav+FqVAokJOTAysrq+c+k4uqFs9F1REEAbm5uXjw4AHMzc1hZ2ent20zOBERVZP09HRYWVmhfv36BrvCoFAoUFhYCDMzM/6yNjCei6plbm6OgoICpKWlwdbWVm8/c0Z5plauXAlfX1+YmZnBz88Phw8fLrd/TEwM/Pz8YGZmhgYNGmD16tUqr1+8eBHDhg2Dj48PRCIRwsPD9bJfIiJtyeVyFBQU6PUvcCIqn42NDYqLi/U6h8zogtPmzZsxZcoUfPLJJzh79ix69OiBF198EUlJSRr7JyYmYuDAgejRowfOnj2LmTNn4v3338fWrVuVffLy8tCgQQMsWLAArq6uetkvEVFFlP7FzWeSEVUfqbRkYK2oqEhv2zS64LRkyRKMGzcO48ePR/PmzREeHg5PT0+sWrVKY//Vq1fDy8sL4eHhaN68OcaPH4+xY8fim2++UfZ54YUXsGjRIowcORIymUwv+yUi0gWvNhFVn6r4eTOqOU6FhYU4c+YMpk+frtIeGBiIY8eOaXxPXFwcAgMDVdqCgoKwdu1ayOVyrf51p8t+yyO/FAk42Fb4faQ/ouJi2Of8DQgvGroUIiKqRYwqOKWnp6O4uBguLi4q7S4uLkhNTdX4ntTUVI39i4qKkJ6eDjc3tyrZLwAUFBSo3FKclZUFALCIfA+Q8V+VhiQF0ANA/rXmQNMBhi6nzpPL5Sr/XxfJ5XIIggCFQgGFQmGwOgRBUP6/IesgnovqoFAoIAgC5HJ5uWs6VeTvJqMKTqX+fWlNEIRyL7dp6q+pXd/7nT9/PubOnavWfte0EVwsjW4UtE6xzr8H0+JcXD5xEEk3+BeSsYiKijJ0CQYjlUrh6uqKnJwcFBYWGrocZGdnG7qEKrdgwQIsXLgQu3btQvfu3Q1dTplq47lISkpC27ZtMWrUKKxcudJgdRQWFuLJkyeIjY0td55TXl6e1ts0quDk6OgIiUSidpUnLS1N7WpQKVdXV439pVIpHBwcqmy/ADBjxgyEhYUpv8/KyoKnpyeiOv6AkMDWWu2bqoZo85vA9T/RokULtPIfaOhy6jy5XI6oqCj079+/zk6Ozs/Px507d2BlZQUzMzOD1SEIArKzs2FtbW10861u3bqFhg0b4u2338a6desqvb3SOa0WFhawsbGp9Pb0zVjORUREBMaNG1fm6wEBATh48GCFtmllZQWg5GYIQx77/Px8mJubo2fPnuX+3JWOGGnDqIKTqakp/Pz8EBUVhVdeeUXZHhUVhSFDhmh8T5cuXbBr1y6Vtn379sHf31/rv6B12S9Q8kOpabJ5anZhnf3lYCwUT/8SkkgkkPJcGA0TE5M6+7NRXFwMkUgEsVhs0DV7SoeESmsxJqX16Ku2yZMnY9SoUfDy8jK6zwoYz7ko3Xffvn01Xpnz8fGpcH36Ppe6EovFEIlEz/27pyJ/LxlVcAKAsLAwBAcHw9/fH126dMGaNWuQlJSEiRMnAii5ypOcnIwNGzYAACZOnIgVK1YgLCwMEyZMQFxcHNauXYuNGzcqt1lYWIhLly4p/zs5ORkJCQmwsrJCo0aNtNpvRdx7/KSyh4GIiCrJ0dERjo6Ohi6jxujXr5/aTVKkzugi+IgRIxAeHo558+ahXbt2iI2NRWRkJLy9vQEAKSkpKmsr+fr6IjIyEtHR0WjXrh0+//xzLFu2DMOGDVP2uXfvHtq3b4/27dsjJSUF33zzDdq3b4/x48drvd+KuPc4vxJHgIiodtq6dSsCAgLg7OwMMzMzeHp6YsCAAdi+fTsiIiLg6+sLAFi/fj1EIpHyKzo6GkDJ3+WzZ89G586d4ezsDJlMBh8fH4SEhCAtLU1tf3PmzFF5P1AyHCgSiTBmzBhcuXIFr776KhwdHSESiXDr1i1lv507dyIoKAgODg4wMzODj48PgoODceHCBWWfa9eu4eOPP0aHDh2U/Zo0aYLp06cjJydHrZ6UlBSEhoaicePGMDc3h6OjI7p27Yr33ntPbaiosLAQS5YsQYcOHWBpaQlra2v06NEDO3furMQZqJwff/wRQ4YMgY+PD8zMzGBvb4+goCAcOnRI6238+xjY29ujdevWCAkJqRHHADDCK04AEBISgpCQEI2vRUREqLUFBAQgPj6+zO35+PgoJ4zrut+KuJfJK05ERM9atWoVQkJC4ObmhldeeQUODg5ISUnByZMnsX37dkyZMgWhoaFYunQp2rZti6FDhyrf6+PjAwCIjY3F4sWL0bdvX3Tq1AkmJiY4e/YsVq1ahT///BPx8fGwtdVuKZjr16+jc+fOaNmyJUaPHo2MjAyYmpoCAD7++GMsWrQI9vb2GDp0KJydnXHnzh3s378ffn5+aNWqFQBg27ZtWLt2LXr37o1evXpBoVDg+PHjWLhwIWJiYhAbG6scAsrLy0O3bt1w69YtBAYG4pVXXkFBQQGuXbuG9evXY9q0acq5QAUFBRgwYACio6PRvn17jBs3DnK5HLt378aQIUOwfPlyTJo0SU9nRnvvvfce2rZti379+sHJyQnJycnYvn07+vXrh23btpU7tQXQfAwKCwtx8+ZNRERE4OOPPzb6YwAYaXCq6bLzi5H5RA5b87o5l4OIKkYQBDyR6++REOVRKBR4UlgMaWFRheaemJtIKjWB+YcffoCpqSnOnTsHJycnldcePnwIBwcHTJkyBUuXLkW7du0wZ84ctW306dMHqampyonHpTZs2IDRo0djxYoV+OSTT7Sq5+jRo/j0008xb948lfbIyEgsWrQIrVu3xqFDh1RuMioqKsLDhw+V3wcHByMsLEwZuErNmzcPs2fPxm+//YY333wTAHDgwAEkJiZi6tSpWLJkCYCSc5GVlQWRSARzc3OV90dHR2POnDn47LPPlMc9Ozsbffr0wQcffIBXX30V7u7uAICEhARs375dq88NlATRMWPGqLXv378f+fnqIyYTJ06Eq6srLl26pLwqWColJQX+/v746KOPnhucNB2DUtnZ2Spzhit6DKoTg1MVufsoD7bmXASTiJ7vibwYLT7709BllOvSvCBYmFbuV0ZZE3S1vQPa2dlZY3twcDAmT56M/fv3ax2cXF1dMWvWLLX2b7/9FgCwdOlStbqkUqnKndYeHh4atz1p0iTMnj0b+/fvVwanUs8GpFLW1tbKEKtQKLBq1So0atRIJTCU9vvss8/w8ssvY9u2bcorLgkJCRqXxilLQECAxuB04MABHDhwQK196NChcHV1VQtNAODm5oZhw4Zh+fLluH37tlbTW8o6BqV0OQbVicGpitzJeIKW7gxOREQA8Prrr2P69Olo1aoVRo4ciV69eqF79+6ws7Or0Ha2bduG7777DvHx8Xj06JHKw1vv3bun9Xbatm2rdqUIAE6ePAmZTIaAgIDnbkMQBKxbtw4RERG4cOECMjMzVRayfLaenj17wtXVFfPnz0dCQgIGDRqErl27qgWNq1ev4tGjR3B3d9cYhh48eAAAuHLlirJtzJgxGoNQRc2fP7/cyeE3b97E/PnzcfDgQSQnJ6ssAA2UfN7ygpOmY9C9e3e0bt1aJRzpcgyqE4NTFbn7SPvFtIiobjM3keDSvKBq2ZdCoUB2VjasbawrPFRXGR9//DEcHBywevVqLFmyBIsXL4ZUKsXAgQMRHh6u8WrGvy1evBgffvghnJycEBgYiPr16yuvXoSHh6v9Ii9PWWv0PX78GB4eHlodm/fffx8rVqyAp6cnXn75Zbi5uSmHm+bOnatSj62tLeLi4jB79mzs2rULkZGRAAB3d3fMnDkT7733HgAgIyMDAHDx4kVcvHixzH3n5uZq90H15Pr16+jYsSOysrLQu3dvvPTSS7CxsYFYLEZ0dDRiYmKee/zLOgb169fHjBkzlHOMjfUYlGJwqiJ3H3GCuHF4/k0BRIYmEokqPQymLYVCgSJTCSxMpdW6vo5IJML48eMxfvx4PHz4EIcPH8bGjRvx22+/4e+//8b58+fLfX9RURE+//xzuLu7IyEhQWWelCAI+PrrrytcjyZ2dnZITU2FQqEo9/ikpaXh22+/RZs2bRAXFwcLCwvla6mpqRqvlPj4+GD9+vUoLi7G+fPn8eeff2LZsmWYNGkS7O3tMWrUKOXk6GHDhmHLli1afRZ9zXEqz//93//h0aNH+Pnnn9WGHydOnIiYmBit9/3sMdi3bx+WLVuG9957D/Xq1dP5GFQnBqcqcieDV5yIiDRxcHDA0KFDMXToUKSnp+PgwYO4fv26Mnw8O/xWKj09HZmZmejbt6/a5PLTp0/jyRP9/GO1Y8eOiIyMRExMDHr37l1mv5s3b0IQBPTr108lNAHA4cOHy92HRCJBu3bt0KZNG7Ru3RqDBg3Czp07MWrUKDRv3hw2NjY4ffq01g+q19ccp/LcuHEDAPDyyy+rtCsUChw9erRC2wL+OQbt2rVDly5d0LNnz0odg+pkdOs41Ra84mRgRvYoCaK67s8//1R7VphcLlcOy5ibm6NevXoQiUS4e/eu2vudnZ1hbm6O+Ph4leeKPXr0CJMnT9ZbnaVDZqGhocraShUVFeH+/fsAoJzLc+zYMZV5TXfv3tU4T+jChQu4ffu2WnvpfJ3SIUepVIp3330Xt2/fxocffqjx4bMXLlxQWbdqzJgxEARB669n17XSVunnPXLkiEr7woULVda2Kk9Zx6D0mFbmGFQnXnGqInce5T33IcFERHXFiBEjYGFhge7du8Pb21v5/MJLly5hxIgR8PLyAgC88MILiI2NxTvvvIPGjRtDLBbjjTfegJeXF0JCQrB48WK0bdsWL730ErKysrBnzx54e3vr7bb0gQMH4sMPP8Q333yDxo0b45VXXoGzszOSk5Nx4MABfPjhh5gyZYrybrKtW7fC398fffv2xf379/HHH3+gT58+uHnzpsp29+/fjw8++ADdunVDs2bN4ODggBs3bmDXrl0wNzdXuTts7ty5iI+Px7Jly7B7924EBAQo1006f/48zp07h7i4uDLvMqwKEydOxLp16/Dqq69ixIgRcHBwwPHjxxEfH49BgwZh9+7dz92GpmNw8+ZN7Ny5s0Ycg1IMTlUkr7AYj/LksLdUv2uDiKiumT9/Pvbu3YuTJ09i165dsLS0RKNGjfDdd99h7Nixyn4//fQTpk6diu3btyMzMxOCIKBz587w8vLC/PnzYW9vj4iICKxcuRIuLi4YOXIk5s6dq1yUUh8WLVqELl26YMWKFdiyZQvy8/Ph5uaGPn36oH///sp+ERER8PHxwdatW7F8+XJ4eXkhLCwM06ZNU7tjLygoCLdu3UJsbCy2bduGnJwceHh44NVXX8XMmTPRsmVLZV+ZTIY9e/Zg7dq12LBhA7Zs2YKCggK4uLigRYsWmDhxIlq3rt4Hybdv3x779u3DrFmzsG3bNkgkEnTt2hVHjx7Fzp07tQpOZR2DkSNH4uOPP0bz5s2VfY3xGJQSCdosqU1aycrKgq2tLTrM2o6Hcil2vNcNbT3tDF1WnaT4dSTE1/agaOASSDuW/dRvqh5yuRyRkZEYOHCg0c1XqC75+flITEyEr69vuU9pr2qliy6W3hFFhsNzUfW0/bkr/f2dmZmpnJxeFp6pKuBuVzJOy3lOREREtQuDUxVwty1Zx+MO13IiIiKqVRicqoB7vdIrTgxOREREtQmDUxVwty0ZR72TwaE6IiKi2oTBqQp42PGKExERUW3E4FQF3OxKrjjdffQEvGnRwHj8iYhIjxicqoCrtRnEIqCgSIEHOdo/dJKIiIiMG4NTFTCViuFqw3lOhsUV24mISP8YnKpIffuShz5ynhMREVHtweBURbyeBqfbDxmciIiIagsGpyrizeBERERU6zA4VREvh5LgdCeDwYmIiKi2YHCqIt4OlgCA2xm5Bq6EiIiI9IXBqYqUDtXdzypAvrzYwNUQEdU9c+bMgUgkQnR0tKFLoVqEwamK2FmYwNpMCgBI4nAdEdFz3bp1CyKRCGPGjDF0KXVCdHQ0RCKR1l+9evUydMlGQWroAmorkUgEbwcLXEjOwu2HeWjiYm3okoiI6pRJkyZh5MiR8PLyMnQpRsnHxwezZ89Wabt16xbWr1+Ptm3bYujQoWr9icGpSnnbWz4NTpznZCgi8JErRHWVo6MjHB0dDV2G0fLx8cGcOXNU2qKjo7F+/Xq0a9dO7TUqwaG6KlR6Zx2H6oiIgK1btyIgIADOzs4wMzODp6cnBgwYgO3btyMiIgK+vr4AgPXr16sMEZXOUbp37x5mz56Nzp07w9nZGTKZDD4+PggJCUFaWpra/jTNcXp2OPDKlSt49dVX4ejoCJFIhFu3bin77dy5E0FBQXBwcICZmRl8fHwQHByMCxcuKPtcu3YNH3/8MTp06KDs16RJE0yfPh05OTlq9aSkpCA0NBSNGzeGubk5HB0d0bVrV7z33nvIyspS6VtYWIglS5agQ4cOsLS0hLW1NXr06IGdO3dW4gzoLiIiAiKRCBEREdi9ezd69OgBa2tr5VWoZ1//t9IhQU1BLDExEePHj4eXlxdkMhnc3NwwZswY3L59u2o/UCXwilMV4lpOBiTiI1eIjMmqVasQEhICNzc3vPLKK3BwcEBKSgpOnjyJ7du3Y8qUKQgNDcXSpUvVholKfznHxsZi8eLF6Nu3Lzp16gQTExOcPXsWq1atwp9//on4+HjY2tpqVc/169fRuXNntGzZEqNHj0ZGRgZMTU0BAB9//DEWLVoEe3t7DB06FM7Ozrhz5w72798PPz8/tGrVCgCwbds2rF27Fr1790avXr2gUChw/PhxLFy4EDExMYiNjYWJiQkAIC8vD926dcOtW7cQGBiIV155BQUFBbh27RrWr1+PadOmwcbGBgBQUFCAAQMGIDo6Gu3bt8e4ceMgl8uxe/duDBkyBMuXL8ekSZP0dGYq5n//+x/27duHwYMHIyQkBNnZ2Tpv68SJEwgKCkJubi5eeuklNGrUCLdu3cIvv/yCPXv2IC4uDg0aNNBj9frB4FSFeMWJiLQiCIC8mv6eUChK9lUoAcQVGHQwsajUP0h++OEHmJqa4ty5c3ByclJ57eHDh3BwcMCUKVOwdOnSMoeJ+vTpg9TUVFhZWam0b9iwAaNHj8aKFSvwySefaFXP0aNH8emnn2LevHkq7ZGRkVi0aBFat26NQ4cOwcHBQflaUVERHj58qPw+ODgYYWFhysBVat68eZg9ezZ+++03vPnmmwCAAwcOIDExEVOnTsWSJUsAAAqFAllZWRCJRDA3N1d5f3R0NObMmYPPPvsMoqfHPTs7G3369MEHH3yAV199Fe7u7gCAhIQEbN++XavPDZQEUV0n4O/Zswf79u1Dv379dHp/KblcjpEjR0KhUOD06dNo27at8rUjR46gV69eCA0Nxa5duyq1n6rA4FSFvJ55Xl2xQoBEzKsgRKSBPA/4yr1adiUGYKfLG2feA0wtK7VvExMT5RWYZz0bTsrj7OyssT04OBiTJ0/G/v37tQ5Orq6umDVrllr7t99+CwBYunSpWl1SqRQuLi7K7z08PDRue9KkSZg9ezb279+vDE6lng1IpaytrSF+GmIVCgVWrVqFRo0aqYSm0n6fffYZXn75ZWzbtk151SkhIQFz587V5mMDAAICAnQOTkOHDq10aAKAP/74A7du3cLnn3+uEpoAoHv37hgyZAi2b9+OrKws5ZU4Y8HgVIXcbM1hIhFBXizg3uMn8HwapIiI6prXX38d06dPR6tWrTBy5Ej06tUL3bt3h52dXYW2s23bNnz33XeIj4/Ho0ePUFz8zzp59+7d03o7bdu2VbtSBAAnT56ETCZDQEDAc7chCALWrVuHiIgIXLhwAZmZmVAoFBrr6dmzJ1xdXTF//nwkJCRg0KBB6Nq1K7y9vVW2efXqVTx69Aju7u4aw9CDBw8AAFeuXFG2jRkzptqWcOjYsaNetnP8+HEAJZ9D09XF1NRUKBQKXLt2Df7+/nrZp74wOFUhiVgEz3oWuJmei6SMPAYnItLMxKLkik41UCgUyMrOhs0zVzm0YlK5v78+/vhjODg4YPXq1ViyZAkWL14MqVSKgQMHIjw8XDkxvDyLFy/Ghx9+CCcnJwQGBqJ+/frKKzjh4eEoKCjQup5nrxw96/Hjx/Dw8NDq2Lz//vtYsWIFPD098fLLL8PNzQ0ymQwAMHfuXJV6bG1tERcXh9mzZ2PXrl2IjIwEALi7u2PmzJl47733AAAZGRkAgIsXL+LixYtl7js31zB3a5d13Cqq9HP+8ssv5fYz1OcsD4NTFfNyKAlOtx/moVsjQ1dDREZJJKr0MJjWFArApLhkfxUJTpUkEokwfvx4jB8/Hg8fPsThw4exceNG/Pbbb/j7779x/vz5ct9fVFSEzz//HO7u7khISFCZJyUIAr7++usK16OJnZ2d8mpHeeEpLS0N3377Ldq0aYO4uDhYWPwTLFNTUzVeLfLx8cH69etRXFyM8+fP488//8SyZcswadIk2NvbY9SoUcphqWHDhmHLli1afZbqnONU1nErPVZFRUVqr2VmZqq1lX7OXbt2YfDgwTrVYigMTlWs9M46ThAnIirh4OCAoUOHYujQoUhPT8fBgwdx/fp1Zfh4dvitVHp6OjIzM9G3b1+1yeWnT5/GkydP9FJbx44dERkZiZiYGPTu3bvMfjdv3oQgCOjXr59KaAKAw4cPl7sPiUSCdu3aoU2bNmjdujUGDRqEnTt3YtSoUWjevDlsbGxw+vRpyOVyjXPC/q065ziVpV69egCA5ORktdfOnj2r1tapUycAQFxcXI0LTlzHqYp5PX3YbxIf9ktEddiff/6pdjVCLpcrh2zMzc1Rr149iEQi3L17V+39zs7OMDc3R3x8PPLy/vmH6KNHjzB58mS91Vk6ZBYaGqqsrVRRURHu378PAMq5SceOHVOZ13T37l1Mnz5dbbsXLlzQuDZR6Zyl0iFHqVSKd999F7dv38aHH34IuVyucVvPrls1ZswYCIKg9VdVPLuvQ4cOEIlE2LRpE/Lz85Xtf//9N5YuXarWf8iQIfDy8sKSJUsQGxur9rpcLseRI0f0Xqc+8IpTFeNaTkREwIgRI2BhYYHu3bvD29sbcrkcUVFRuHTpEkaMGKF8LMoLL7yA2NhYvPPOO2jcuDHEYjHeeOMNeHl5ISQkBIsXL0bbtm3x0ksvISsrC3v27IG3t7fy1vzKGjhwID788EN88803aNy4MV555RU4OzsjOTkZBw4cwIcffogpU6bAzc0Nw4YNw9atW+Hv74++ffvi/v37+OOPP9CnTx/cvHlTZbv79+/HBx98gG7duqFZs2ZwcHDAjRs3sGvXLpibm6usyzR37lzEx8dj2bJl2L17NwICAuDk5ITk5GScP38e586dQ1xcXJl3GRqCh4cHRowYgU2bNsHPzw8DBgxAWloafv/9dwwYMABbt25V6S+TybBlyxa8+OKLCAgIQN++fZXrYyUlJeHw4cNwcHBQmQRvLBicqph36VpOD/MgCEKZ48NURQQ+coXIGMyfPx979+7FyZMnsWvXLlhaWqJRo0b47rvvMHbsWGW/n376CVOnTsX27duRmZkJQRDQuXNneHl5Yf78+bC3t0dERARWrlwJFxcXjBw5EnPnzlX+0tWHRYsWoUuXLlixYgW2bNmC/Px8uLm5oU+fPujfv7+yX0REBHx8fLB161YsX74cXl5eCAsLw7Rp09Tu2AsKCsKtW7cQGxuLbdu2IScnBx4eHnj11Vcxc+ZMtGzZUtlXJpNhz549WLt2LTZs2IAtW7agoKAALi4uaNGiBSZOnIjWrVvr7fPqy9q1a+Hk5ITffvsN3377LZo2bYo1a9bA3d1dLTgBJSH53LlzWLRoESIjI3HkyBHIZDJ4eHhg6NChGDVqlAE+xfOJBIG/WfQlKysLtra2SE9PV67/kS8vRrNP9wIA4j/tD3tL9dtfSf8UG9+A+OpuFA9YBEnn/xi6nDpPLpcjMjISAwcO1GrORm2Un5+PxMRE+Pr6wszMzGB1lC66aGNjU7G76kjveC6qnrY/d6W/vzMzM5+7bhTPVBUzM5HA1abkZPFhv9WJV/aIiEj/GJyqAR+9QkREVDswOFWD0gnit9IZnIiIiGoyBqdq4ONYsiTBLQ7VERER1WgMTtWgwdPglJjO4ERERFSTMThVAx8GJyIiolqBwaka+DxdPTzziRyPcgsNXA0RGRJXgCGqPlXx88bgVA3MTSVwty1ZkuAmrzoR1UkSiQQAND5Cg4iqRuljfqRS/a33zeBUTThcR1S3mZiYQCaTKVfDJqKql5WVBYlEovyHiz7wkSvVxNfREsduPMQtBqdqxl9QZDwcHR2RnJyMu3fvwtbWFiYmJtX+GCaFQoHCwkLk5+dztWoD47moOoIgIDc3F1lZWXBzc9PrzxmDUzXx5RUnojqv9FEO6enpSE5ONkgNgiDgyZMnMDc357MzDYznomqJRCLY2dnB1tZWr9tlcKomDE7VjH8JkZGysbGBjY0N5HI5iouLq33/crkcsbGx6NmzZ519bqCx4LmoWiYmJnodoivF4FRNng1OgiDwXxdEdZyJiYlBfllKJBIUFRXBzMyMv6wNjOeiZuKgajXxtLeARCzCE3kx7mcVGLocIiIi0gGDUzUxkYjhWc8cAIfriIiIaioGp2rEJQmIiIhqNganauTLh/0SERHVaAxO1aj0Yb83HzA4ERER1UQMTtXon6G6HANXQkRERLpgcKpGpUN1SRl5KFZwRWsiIqKahsGpGrnbmsNUKoa8WEDyoyeGLqduYD4lIiI9MsrgtHLlSvj6+sLMzAx+fn44fPhwuf1jYmLg5+cHMzMzNGjQAKtXr1brs3XrVrRo0QIymQwtWrTA77//rvJ6UVERZs2aBV9fX5ibm6NBgwaYN28eFAqF3j6XWCyCj4MFACCRE8SJiIhqHKMLTps3b8aUKVPwySef4OzZs+jRowdefPFFJCUlaeyfmJiIgQMHokePHjh79ixmzpyJ999/H1u3blX2iYuLw4gRIxAcHIxz584hODgYr7/+Ok6cOKHss3DhQqxevRorVqzA5cuX8fXXX2PRokVYvny5Xj+fcgXxB5znVLW4MjsREemf0QWnJUuWYNy4cRg/fjyaN2+O8PBweHp6YtWqVRr7r169Gl5eXggPD0fz5s0xfvx4jB07Ft98842yT3h4OPr3748ZM2agWbNmmDFjBvr27Yvw8HBln7i4OAwZMgSDBg2Cj48Phg8fjsDAQJw+fVqvn49rOREREdVcRvWsusLCQpw5cwbTp09XaQ8MDMSxY8c0vicuLg6BgYEqbUFBQVi7di3kcjlMTEwQFxeHqVOnqvV5Njh1794dq1evxrVr19CkSROcO3cOR44cUenzbwUFBSgo+OfxKVlZWQBKHtwol8s1vsf76erh19NyyuxDlScSFBADKFYUQ8HjbHClf9b5Z97weC6MB8+F8ajIOTCq4JSeno7i4mK4uLiotLu4uCA1NVXje1JTUzX2LyoqQnp6Otzc3Mrs8+w2p02bhszMTDRr1gwSiQTFxcX48ssvMWrUqDLrnT9/PubOnavWfujQIVhYWGh8T1o2AEhx8U46IiMjy9w2VY7//fvwAHD58mXcSudxNhZRUVGGLoGe4rkwHjwXhpeXl6d1X6MKTqVEItX5KYIgqLU9r/+/25+3zc2bN+Pnn3/Gr7/+ipYtWyIhIQFTpkyBu7s7Ro8erXG/M2bMQFhYmPL7rKwseHp6onfv3nBwcND4nsd5coRfOITHhSL07BsIK5lRnoIaT7TlN+Ax0Lx5c7ToONDQ5dR5crkcUVFR6N+/P58Cb2A8F8aD58J4lI4YacOofms7OjpCIpGoXV1KS0tTu2JUytXVVWN/qVSqDC9l9Xl2mx999BGmT5+OkSNHAgBat26N27dvY/78+WUGJ5lMBplMptZuYmJS5g+Bk60JHK1MkZ5TiDuPC9CmvrnGflQ5ClHJ9D2JWAIJ/0IyGuX9bFD14rkwHjwXhleR429Uk8NNTU3h5+endtkyKioKXbt21fieLl26qPXft28f/P39lQeirD7PbjMvLw9iserhkEgkel2OoFRDJysAwA3eWUdERFSjGNUVJwAICwtDcHAw/P390aVLF6xZswZJSUmYOHEigJLhseTkZGzYsAEAMHHiRKxYsQJhYWGYMGEC4uLisHbtWmzcuFG5zdDQUPTs2RMLFy7EkCFDsGPHDuzfvx9HjhxR9nnppZfw5ZdfwsvLCy1btsTZs2exZMkSjB07Vu+fsaGzFU4kZuB6GoMTERFRTWJ0wWnEiBF4+PAh5s2bh5SUFLRq1QqRkZHw9vYGAKSkpKis6eTr64vIyEhMnToV3377Ldzd3bFs2TIMGzZM2adr167YtGkTZs2ahU8//RQNGzbE5s2b0alTJ2Wf5cuX49NPP0VISAjS0tLg7u6O//73v/jss8/0/hkblV5xSuOSBFWPS4cTEZH+GF1wAoCQkBCEhIRofC0iIkKtLSAgAPHx8eVuc/jw4Rg+fHiZr1tbWyM8PLzc5Qf0paEzh+qIiIhqIqOa41RXNHoanG49zEVRsf7nUBGAcu7CJCIi0hWDkwG42ZjB3EQCebGApAzt144gIiIiw2JwMgCxWISGziWPXuEEcSIiopqDwclA/lmSgBPEiYiIagoGJwMpDU684kRERFRzMDgZSCPeWUdERFTjMDgZiHKoLi1H+Ww9IiIiMm4MTgbi42gBsQjILijCg+wCQ5dDREREWmBwMhCZVAIvewsAnOdERERUUzA4GRDnOVUDDoMSEZEeMTgZEJckICIiqlkYnAyo9Jl1HKojIiKqGRicDKh0qO7a/WwDV0JERETaYHAyoMZPg1NadgEe5xUauBoiIiJ6HgYnA7I2M4GHnTkA4Np9DtcREREZOwYnA2viwuE6IiKimoLBycCauFoDYHAiIiKqCRicDKyJM4MTERFRTcHgZGBNlVecOMeJiIjI2DE4GVhDJyuIREBGbiHSc/jMOiIiImPG4GRg5qYSeD99Zt21VA7X6R8fuUJERPrD4GQEGrtwnhMREVFNwOBkBJo+DU5XOc9Jf0QiQ1dARES1EIOTEWj8dC2nv3nFiYiIyKjpJThlZGTgzp07+thUnVR6Z93V+9kQBM7JISIiMlY6B6fMzEyEhobCxcUFTk5O8PX1Vb524sQJDBw4EGfOnNFLkbWdr6MlJGIRsvOLcD+Ld9YREREZK52CU0ZGBjp16oTly5fD09MTzZs3V7lS0qZNGxw9ehS//PKL3gqtzWRSCXwdLQGUXHUiIiIi46RTcJozZw6uXbuGjRs34vTp03jttddUXjc3N0dAQAAOHjyolyLrAuUz67gkARERkdHSKTjt3LkTgwcPxogRI8rs4+3tjbt37+pcWF3ThEsSEBERGT2dglNKSgpatGhRbh8zMzPk5ubqVFRdxOBERERk/HQKTg4ODs+9i+7KlStwc3PTqai66J/glAOFgnfWERERGSOdglPPnj2xc+dOJCcna3z90qVL2Lt3L/r161ep4uoSHwcLyKRiPJEX43ZGnqHLqUUYQomISH90Ck6ffPIJioqK0K1bN/z6669IT08HAFy+fBlr165Fnz59IJPJ8NFHH+m12NpMKhEr13O6nJJl4GqIiIhIE6kub2rdujU2b96Mt99+G8HBwQAAQRDQqlUrCIIAa2tr/Pbbb2jcuLFei63tmrva4K+7mbiSkoWBrTnMWTl85AoREemfTsEJAF5++WXcvHkT69evx4kTJ5CRkQEbGxt06tQJ77zzDhwdHfVZZ53QzK3kitOlFE4QJyIiMkY6BycAsLe3x9SpU/VVS53X3M0GAIfqiIiIjJVOc5zGjh2LnTt3ltsnMjISY8eO1amouqq5a0lwSn78BJlP5AauhoiIiP5Np+AUERGBhISEcvucP38e69ev12XzdZathQk87MwBAFd41YmIiMjo6PyQ3+fJz8+HVFqpkcA6qbkb76wjIiIyVjoHJ5FI811LgiDgzp07iIyMhLu7u86F1VWl85yu8Jl1RERERkfr4CQWiyGRSCCRSACUPOi39Ptnv6RSKXx8fHDq1CmMHDmyygqvrZq5coI4ERGRsdJ6LK1nz57Kq0yxsbHw8vKCj4+PWj+JRAJ7e3v06dMHEyZM0FuhdUXpUN3V+9koVgiQiLkeERERkbHQOjhFR0cr/1ssFuOdd97BZ599VhU11WneDpYwN5HgibwYiem5aORsZeiSajaBj1whIiL90Wn2tkKh0Hcd9JRELEJTV2sk3HmMyylZDE5ERERGpMruqiPdcSFMPSjj5gUiIqLK0Hm9gOLiYvz222/Yv38/7t27h4KCArU+IpEIBw4cqFSBdVHpPCfeWUdERGRcdApOubm5CAwMxPHjxyEIAkQiEYRn5pKUfl/WkgVUPl5xIiIiMk46DdV98cUXiIuLw9y5c5Geng5BEDBnzhykpKRg8+bN8PX1xfDhwzVehaLna+ZacsUpJTMfj3ILDVwNERERldIpOG3btg2dO3fGrFmzYG9vr2x3cXHBa6+9hujoaBw4cACLFi3SW6F1ibWZCbwdLAAAF+/xqhMREZGx0Ck4JSUloXPnzv9sRCxWubpUv359DBo0iM+qq4RWHrYAgAv3Mg1cCREREZXSKThZWlpCLP7nrba2tkhJSVHp4+rqiqSkpMpVV4e1ci8JTueTGZyIiIiMhU7BydvbWyUUtWrVCgcPHlRedRIEAQcOHICbm5t+qqyDWnmUTBC/yOBERERkNHQKTn379sWhQ4dQVFQEABg9ejSSkpLQpUsXfPTRR+jevTsSEhIwbNgwvRZbl5Recbr1MA9Z+XIDV0NERESAjssRTJgwAQ4ODnjw4AHc3NwwduxYnD17FitXrkRCQgIAYNiwYZgzZ44eS61b6lmawsPOHMmPn+Bicha6NHQwdEk1FB+5QkRE+qPTFafGjRtj2rRpKkNxy5cvR2pqKuLi4nDv3j3873//g4WFhd4KrYtaP50gfpETxImIiIyCXh+54uTkhE6dOsHV1RUAcP36dX1uvs4pnefECeK64OKrRESkf1XyrLrbt29j7NixaNmyZVVsvs5QLknA4ERERGQUKjzHKSYmBmfOnIFUKkW3bt3g5+enfC0lJQVz587FunXrIJfL4eHhoddi65qWTyeI30zPRW5BESxlOj9akIiIiPRA69/EhYWFGDp0KP7880+V9smTJyM8PBw//vgjQkNDkZubC3d3d0yfPh3/+c9/9F5wXeJkLYOrjRlSs/JxKSULL/jYP/9NREREVGW0HqpbunQp9u7dCzc3N7z77ruYOHEiXFxcsHz5cnzwwQcYP348TE1NER4ejhs3bmDSpEkwNTXVqaiVK1fC19cXZmZm8PPzw+HDh8vtHxMTAz8/P5iZmaFBgwZYvXq1Wp+tW7eiRYsWkMlkaNGiBX7//Xe1PsnJyXjrrbfg4OAACwsLtGvXDmfOnNHpM+gLh+uIiIiMh9bBafPmzXB0dMT58+exYsUKfPvttzh37hzq1auH8PBwvPDCC7h69Sref/99yGQynQvavHkzpkyZgk8++QRnz55Fjx498OKLL5a5CnliYiIGDhyIHj164OzZs5g5cybef/99bN26VdknLi4OI0aMQHBwMM6dO4fg4GC8/vrrOHHihLLPo0eP0K1bN5iYmGDPnj24dOkSFi9eDDs7O50/iz5wgjgREZHx0Do4Xbt2DUOHDkW9evWUbU5OTnjllVcAlFwlcnR0rHRBS5Yswbhx4zB+/Hg0b94c4eHh8PT0xKpVqzT2X716Nby8vBAeHo7mzZtj/PjxGDt2LL755htln/DwcPTv3x8zZsxAs2bNMGPGDPTt2xfh4eHKPgsXLoSnpyfWrVuHjh07wsfHB3379kXDhg0r/ZkqQ7kkQTIf9ktERGRoWgennJwcuLu7q7WXTgBv27ZtpYspLCzEmTNnEBgYqNIeGBiIY8eOaXxPXFycWv+goCCcPn0acrm83D7PbnPnzp3w9/fHa6+9BmdnZ7Rv3x7ff/99pT9TZZUO1f2dlo0nhcUGroaIiKhuq9BtWs8+2LeUSFSyXo5UWvk7vtLT01FcXAwXFxeVdhcXF6Smpmp8T2pqqsb+RUVFSE9Ph5ubW5l9nt3mzZs3sWrVKoSFhWHmzJk4efKkctjx7bff1rjvgoIC5fP5ACArq+SqkFwuV4a2yqpnJoajlSnScwpx/m4G2nva6WW7tZ1IoYAYQHFxMRR6Oheku9KfB339XJDueC6MB8+F8ajIOahQ2rl79y5Onjyp1gYAp06dgiCoP96iY8eOFdkFgH/CWClBENTantf/3+3P26ZCoYC/vz+++uorAED79u1x8eJFrFq1qszgNH/+fMydO1et/dChQ3pdNd3FRIx0iLFpXxxS3PgIEW10SE2FJ4CrV6/i5qNIQ5dDT0VFRRm6BHqK58J48FwYXl5entZ9KxSc1q5di7Vr16q1C4KAzp07a3xPcbH2w0uOjo6QSCRqV5fS0tLUrhiVcnV11dhfKpXCwcGh3D7PbtPNzQ0tWrRQ6dO8eXOVSeb/NmPGDISFhSm/z8rKgqenJ3r37q3ctz7cNL+BiwdvoMimPgYObK237dZmom2/A4+Apk2aoFnXgYYup86Ty+WIiopC//79YWJiYuhy6jSeC+PBc2E8SkeMtKF1cBo9erROxVSEqakp/Pz8EBUVpZx0DpSk8SFDhmh8T5cuXbBr1y6Vtn379sHf31/5B7FLly6IiorC1KlTVfp07dpV+X23bt1w9epVle1cu3YN3t7eZdYrk8k03kFoYmKi1x+CDj4OAG7g/L0s/nBpSSGRAAAkEgkkPGZGQ98/G6Q7ngvjwXNheBU5/loHp3Xr1ulUTEWFhYUhODgY/v7+6NKlC9asWYOkpCRMnDgRQMlVnuTkZGzYsAEAMHHiRKxYsQJhYWGYMGEC4uLisHbtWmzcuFG5zdDQUPTs2RMLFy7EkCFDsGPHDuzfvx9HjhxR9pk6dSq6du2Kr776Cq+//jpOnjyJNWvWYM2aNdXyucvTtn7JBPHE9Fw8ziuEnYVu62MRERFR5VTJs+oqY8SIEQgPD8e8efPQrl07xMbGIjIyUnnlJyUlRWVNJ19fX0RGRiI6Ohrt2rXD559/jmXLlmHYsGHKPl27dsWmTZuwbt06tGnTBhEREdi8eTM6deqk7PPCCy/g999/x8aNG9GqVSt8/vnnCA8Px5tvvll9H74Mdham8HW0BACcu8v1nIiIiAzFKB9+FhISgpCQEI2vRUREqLUFBAQgPj6+3G0OHz4cw4cPL7fP4MGDMXjwYK3rrE5t69siMT0X5+48RkATJ0OXQ0REVCcZ3RUn0qzt02UIzt15bNA6iIiI6jIGpxpCGZzuPta47AMRERFVPQanGqKFmw2kYhHScwqR/PiJocshIiKqkxicaggzEwmau5U88DeBw3VEREQGweBUg7TjPCciIiKDYnCqQf6ZIM4lCbTH+WBERKQ/Oi1HMHbs2Of2EYvFsLGxQdOmTTF48GB4eHjosit6RjvPkoUwzydnoqhYAamEuZeIiKg66RScIiIilA/I1XSHl0gkUmmfPHkyPvvsM8yaNUvHMgkAGjhawUomRU5BEf5Oy1HOeSJNyn4oNBERka50umRx48YNDB48GC4uLpg/fz5iYmJw5coVxMTE4KuvvoKLiwtefvllnDhxAmvWrIG7uztmz56NzZs367v+OkUsFqHN08evnE16bNhiiIiI6iCdgtPmzZtx8uRJJCQkYNq0aejRoweaNGmCHj16YPr06YiPj8fx48dx6NAhjB8/HkePHoWVlRVWrlyp7/rrnA5e9QAA8UmPDFwJERFR3aNTcFq7di1ee+01ODs7a3zd1dUVr732Gr7//nsAgIeHBwYPHoxz587pXikBAPy8S4LTmdsMTkRERNVNp+B09+5dyGSycvuYmZnh7t27yu+9vLyQn5+vy+7oGe297AAAiem5eJhTYNhiiIiI6hidgpOHhwd27NiBggLNv7gLCgqwY8cOlTvp0tLSUK9ePd2qJCU7C1M0drYCAMRznhMREVG10ik4jRs3DtevX0dAQAB2796NjIwMAEBGRgb++OMP9OzZEzdu3FBZtuDw4cNo27atfqqu40qH607fzjBwJURERHWLTssRfPzxx7h8+TJ+/vlnvPzyywBK1m1SKBQASpYoePPNNzF9+nQAwP379zFo0CAMGDBAT2XXbX7e9bDp1B3Ec54TERFRtdIpOEkkEmzYsAGjR4/Gzz//jL/++gtZWVmwsbFB27Zt8eabb6Jv377K/i4uLvi///s/vRVd15VecTp3NxMFRcWQSSUGroiIiKhu0Ck4lerbt69KQKLq4etoCXtLU2TkFuLivSzlEgWkgYYFWomIiHTFZ3bUQCKR6J/1nDhcR0REVG0qdcUpNTUVZ86cwePHj1FcXKyxz9tvv12ZXVAZ/LzrYf/l+zh96xHG9zB0NUZIxEeuEBGR/ukUnPLz8zFhwgRs3LhR47PqgJIJ4iKRiMGpivj7PF0IM+mR8lgTERFR1dIpOE2bNg2//PILmjRpglGjRqF+/fqQSit18YoqqLWHLUwkIjzILsCdjCfwcrAwdElERES1nk5p53//+x9atGiBM2fOPHcFcaoaZiYStPKwxdmkxziTlMHgREREVA10mhz++PFjDBgwgKHJwPyeThA/dYsTxImIiKqDTsGpefPmuH//vr5roQp6wdceAHAykSuIExERVQedgtO0adOwY8cOXL9+Xd/1UAV09CkJTtfTcpDOB/4SERFVOZ3mOLm6umLAgAHo2LEjpkyZgvbt28PW1lZj3549e1aqQCpbPUtTNHO1xpXUbJxMzMDA1m6GLomIiKhW0yk49erVCyKRCIIgYM6cOeXeCl/W+k6kH5187RmcysWVw4mISH90Ck6fffYZ1w0yEh19HbA+7jaO33xo6FKIiIhqPZ2C05w5c/RcBumq49MJ4lfvZ+NxXiHsLEwNXJGxYLAnIiL947PqajgnaxkaOllCEHh3HRERUVVjcKoFOjVwAMDgREREVNW0Gqpr0KABRCIR9u/fD19fXzRo0ECrjYtEIty4caNSBdLzdfK1x68nknCCwYmIiKhKaRWcFAqFymTwf39flrIeAEz61cm35IrTxXuZyMqXw8bMxMAVERER1U5aBadbt26V+z0ZlqutGbwdLHD7YR7O3HqE3s2cDV0SERFRrcQ5TrVEp6d313G4joiIqOowONUSpcN1cVzPiYiIqMrotI4TABQWFmL79u04deoUHj9+rHGFcJFIhLVr11aqQNJO10Ylwen83cfIfCKHrTnnOREREembTsHp9u3b6N+/P27cuFHuBHAGp+rjZmuOBk6WuPkgFyduPkRgS1dDl2QceIMCERHpkU7BaerUqbh+/TqCg4MxduxY1K9fH1KpzhevSE+6N3LEzQe5OHo9ncGJiIioCuiUdg4ePIi+ffti/fr1+q6HKqFrQ0dsiLuNI9fTDV2K4fFZikREVAV0mhyuUCjQvn17fddCldSlgQPEIuDGg1ykZuYbuhwiIqJaR6fg1KVLF1y+fFnftVAl2VqYoHV9OwDAUV51IiIi0judgtOCBQtw6NAhbNmyRd/1UCV1a1hydx2DExERkf7pNMdp165d6N27N0aMGIGAgAC0b98etra2av1EIhE+/fTTShdJ2uveyBEro2/g6I10CIKg1aNxiIiISDs6Bac5c+Yo/zs6OhrR0dEa+zE4Vb8O3vUgk4pxP6sANx7koJGztaFLIiIiqjV0Ck6HDh3Sdx2kJ2YmErzgY48j19Nx5O90BiciIiI90ik4iUQi2NjYoF27dnouh/ShWyNHHLmejqM3HmJMN19Dl0NERFRr6DQ5vHfv3vj+++/1XQvpSbenj185fuMh5MUKA1dDRERUe+gUnJydnWFqaqrvWkhPWrnbwt7SFNkFRYi//cjQ5RgYH7lCRET6o1NwCgoKQkxMTLnPqSPDEYtF6NnYEQAQfe2BgashIiKqPXQKTl999RUePnyI//znP8jIyNB3TaQHvZo6AwCir9bV4MRlGIiISP90mhz+1ltvwc7ODj/++CN+/vln+Pr6wsXFRW3NIJFIhAMHDuilUKqYHo0dIRIBl1OycD8rHy42ZoYuiYiIqMbTKTg9u25TQUEBrly5gitXrqj14+KLhuNgJUMbD1ucu5uJmGsP8Lq/p6FLIiIiqvF0fsivNl/FxcX6rpcqIODpcF1MnR2uIyIi0i+dghPVDL2aOgEADv/9AEVcloCIiKjSGJxqsbb17VDPwgRZ+UU4e+exocshIiKq8XSa41Tq7t27OHToEO7du4eCggK11/msOsOSiEXo0dgJO8/dQ/TVNLzgY2/okoiIiGo0nYPTRx99hKVLl6rMYxIEQTkhvPS/GZwMK6BJSXCKufYAHwU1M3Q5RERENZpOQ3Xff/89Fi9ejN69e2PLli0QBAGjR4/Gxo0bMXHiREilUgwfPhwHDx7Ud71UQT2blMxzupCchbSsfANXQ0REVLPpFJzWrFkDHx8f7NmzB6+88goAwMfHByNGjMC3336Lffv2Yfv27XjwgHdzGZqTtQxt69sCAA5eSTNwNQbAxe2JiEiPdApOV65cwYABAyAW//P2oqIi5X8HBARg0KBB+Oabb3QqauXKlfD19YWZmRn8/Pxw+PDhcvvHxMTAz88PZmZmaNCgAVavXq3WZ+vWrWjRogVkMhlatGiB33//vcztzZ8/HyKRCFOmTNGpfmPTv4ULACDq0n0DV0JERFSz6XxXnZ2dnfK/LS0t8fDhQ5XXmzZtiosXL1Z4u5s3b8aUKVPwySef4OzZs+jRowdefPFFJCUlaeyfmJiIgQMHokePHjh79ixmzpyJ999/H1u3blX2iYuLw4gRIxAcHIxz584hODgYr7/+Ok6cOKG2vVOnTmHNmjVo06ZNhWs3Vv2eBqcj19ORV1j0nN61BRdfJSIi/dMpOHl4eODu3bvK7xs2bKgWQi5cuABLS8sKb3vJkiUYN24cxo8fj+bNmyM8PByenp5YtWqVxv6rV6+Gl5cXwsPD0bx5c4wfPx5jx45VudoVHh6O/v37Y8aMGWjWrBlmzJiBvn37Ijw8XGVbOTk5ePPNN/H999+jXr16Fa7dWDV1sYanvTkKihQ48ne6ocshIiKqsXS6q65bt24qw2dDhgzBF198gYkTJ+Kll17CkSNHsGfPHgwbNqxC2y0sLMSZM2cwffp0lfbAwEAcO3ZM43vi4uIQGBio0hYUFIS1a9dCLpfDxMQEcXFxmDp1qlqffwen9957D4MGDUK/fv3wxRdfPLfegoIClWUYsrKyAAByuRxyufy5769OfZo6YX1cEv68mILeTRwMXU6VEwkKiAEUK4qhMLJzUReV/jwY289FXcRzYTx4LoxHRc6BTsEpODgY9+7dw+3bt+Ht7Y2PPvoIf/zxB9asWYPvv/8egiDAx8cHixYtqtB209PTUVxcDBcXF5V2FxcXpKamanxPamqqxv5FRUVIT0+Hm5tbmX2e3eamTZsQHx+PU6dOaV3v/PnzMXfuXLX2Q4cOwcLCQuvtVAerTBEACf78KxndTZMgruUjWe2Sk+EN4O+//8b1rEhDl0NPRUVFGboEeornwnjwXBheXl6e1n11Ck69evVCr169lN9bWVnh+PHj2LFjB27cuAFvb2+89NJLOg3VAeoPB352fSht+/+7vbxt3rlzB6Ghodi3bx/MzMy0rnPGjBkICwtTfp+VlQVPT0/07t0bDg7GdVWnf7ECPy2IRlZ+EdxadYGfd+0ZitREtDMSyAAaN26MJt0HGrqcOk8ulyMqKgr9+/eHiYmJocup03gujAfPhfEoHTHSRqVWDn+WiYkJhg8fXqltODo6QiKRqF1dSktLU7tiVMrV1VVjf6lUqgwvZfUp3eaZM2eQlpYGPz8/5evFxcWIjY3FihUrUFBQAIlEorZvmUwGmUym1m5iYmJ0PwQmJkDvZs7YkXAPh/5+iM6NnA1dUpVSiEqm70nEEkiM7FzUZcb4s1FX8VwYD54Lw6vI8a/0s+ouXbqEbdu24aeffqrspmBqago/Pz+1y5ZRUVHo2rWrxvd06dJFrf++ffvg7++vPBBl9SndZt++fXH+/HkkJCQov/z9/fHmm28iISFBY2iqiUqXJdjPZQmIiIh0ovMVp1OnTmHChAk4f/68si04OBgAEBsbiwEDBmDTpk14+eWXK7TdsLAwBAcHw9/fH126dMGaNWuQlJSEiRMnAigZHktOTsaGDRsAABMnTsSKFSsQFhaGCRMmIC4uDmvXrsXGjRuV2wwNDUXPnj2xcOFCDBkyBDt27MD+/ftx5MgRAIC1tTVatWqlUoelpSUcHBzU2muynk2cYCIR4caDXNx8kIMGTlaGLomIiKhG0emK08WLF9GnTx8kJiZi6tSpePHFF1Ve79GjBxwdHfG///2vwtseMWIEwsPDMW/ePLRr1w6xsbGIjIyEt7c3ACAlJUVlTSdfX19ERkYiOjoa7dq1w+eff45ly5ap3NHXtWtXbNq0CevWrUObNm0QERGBzZs3o1OnTrp8/BrLxswEnRuUDF/uvah5sj0RERGVTacrTrNnzwZQMjeoUaNGmDt3Lvbs2aN8XSQSoUuXLhW6Q+1ZISEhCAkJ0fhaRESEWltAQADi4+PL3ebw4cMrNAcrOjpa6741yYut3HD473TsOZ+KkF6NDF1ONeAzV4iISH90uuIUExODYcOGoVGjsn/xenl5ISUlRefCqGoEtXSBWAScT85E0kPtb78kIiIiHYNTdnY2nJ3LvysrPz8fxcXFOhVFVcfBSqYcrttzoRYH23KWryAiItKVTsHJ09MTFy5cKLfPmTNn0LBhQ52Koqo1sLUbACDyfC0OTkRERFVAp+A0ePBg7Nu3DwcPHtT4+m+//Ybjx49j6NChlamNqkhQS1eIRcC5u5m4+4jDdURERNrSKTjNnDkTbm5uePHFF/Gf//wHp0+fBgCsXLkSwcHBeOONN+Dj46OyqjYZDydrGTr62gMA9pzn3XVERETa0ik4OTk5ISYmBv7+/vjhhx+we/duCIKASZMm4ZdffsELL7yAgwcPwtbWVt/1kp4MKh2uq83znIiIiPRM5wUwGzRogKNHjyIhIQHHjx9HRkYGbGxs0KlTJ7zwwgv6rJGqQFArV3y28yLOJj3GvcdP4G5nbuiSiIiIjF6ln1XXrl07tGvXTq19+fLlOHToELZt21bZXVAVcLY2wws+9jiZmIHI8ykY36OBoUsiIiIyepV+Vl1Z4uPjsWPHjqraPOnB4DYlw3U7z90zcCVEREQ1Q5UFJzJ+A1u7QSIW4a+7mbj5IMfQ5RARERk9Bqc6zNFKhh6NHQEA2xNq6VUngY9cISIi/WFwquOGtvMAAOxISIbAkEFERFQuBqc6rn8LF5ibSHD7YR4S7jw2dDlERERGjcGpjrOUSRHY0gUAsKO2DtcRERHpidbLEQwcOLBCGz5//nyFiyHDGNrOAzsS7uGPv+5h1qDmkEqYp4mIiDTROjjt3bu3whsX8Qn1NUL3xo6wtzRFek4hjlxPR6+mzoYuiYiIyChpHZwSExOrsg4yIBOJGIPbuGFD3G3sSLjH4ERERFQGrYOTt7d3VdZBBvZKew9siLuNPRdSMG9IS1ibmRi6JCIiIqPDySwEAGjnaYdGzlbIlyuw6xwf/EtERKQJgxMBKJmPNsLfEwDw2+k7Bq6GiIjIODE4kdLQ9h6QikVIuPMY1+5nG7ocIiIio8PgREpO1jL0aVYyMfx/teaqE1dDJyIi/WFwIhWvPx2u2xafDHmxwsDVEBERGRcGJ1LRq6kTnKxleJhbiINX0gxdju64hhgREVUBBidSIZWI8WqHkgf//naqtgzXERER6QeDE6l5za9kuO7Q1TTce/zEwNUQEREZDwYnUtPI2QqdfO2hEIBNJ5MMXQ4REZHRYHAijYK7lKwUv/HUHU4SJyIieorBiTQKbOEKJ2sZHmQXYN/F+4Yuh4iIyCgwOJFGplIxRr5QMtfpp+O3DFsMERGRkWBwojKN6ugFsQg4fjMDf3MlcSIiIgYnKpu7nTn6NXcBAPx8/LaBqyEiIjI8Bicq11udSyaJb4tPRm5BkYGr0YHAR64QEZH+MDhRubo3coSPgwWyC4qwNf6uocshIiIyKAYnKpdYLMI73XwBAD8eSUSxoqZcweEjV4iISP8YnOi5hvvVh42ZFLce5uHAZS5NQEREdReDEz2XpUyKNzqVzHX64UiigashIiIyHAYn0srort6QikU4mZiBv+4+NnQ5REREBsHgRFpxszXHS23dAQBredWJiIjqKAYn0tq47iWTxHf/lYJ7j58YuBoiIqLqx+BEWmvlYYvODexRpBCw7iivOhERUd3D4EQV8t+eDQEAv5xIwqPcQgNXQ0REVL0YnKhCejV1Qkt3G+QVFvOqExER1TkMTlQhIpEIk3o3AgBEHLuF7Hy5gSt6npqyYCcREdUEDE5UYUEtXdHI2QpZ+UX4iQ//JSKiOoTBiSpMLBYhpFfJXKe1hxPxpLDYwBWpE0R85AoREekfgxPp5OW27vC0N8fD3EJsOpVk6HKIiIiqBYMT6UQqEWNiQMlVp1XRN5AvN76rTkRERPrG4EQ6e83PEx525kjLLsBPcZzrREREtR+DE+nMVCpGaL/GAIBVMTeQU1Bk4IqIiIiqFoMTVcqr7T3QwNESGbmF+JHPsCMiolqOwYkqRSoRY0r/JgCA72Nv4nEeVxMnIqLai8GJKm1wazc0c7VGdkERvou9aehyiIiIqgyDE1WaWCzCB4FNAQARR28hLSvfwBU9Q+DK4UREpD8MTqQX/Zo7o4OXHZ7Ii7F43zVDl0NERFQlGJxIL0QiET4Z1AIA8NuZO7h0L8vQFRl4/0REVBsxOJHe+HnXw+A2bhAE4MvISxA4TEZERLUMgxPp1bQBzWAqEePo9Yc4dDXN0OUQERHpFYMT6ZWnvQXe6e4DAPhy92XIixWGLYiIiEiPjDI4rVy5Er6+vjAzM4Ofnx8OHz5cbv+YmBj4+fnBzMwMDRo0wOrVq9X6bN26FS1atIBMJkOLFi3w+++/q7w+f/58vPDCC7C2toazszOGDh2Kq1ev6vVz1RXv9W4Ee0tT3HiQi1+O81EsRERUexhdcNq8eTOmTJmCTz75BGfPnkWPHj3w4osvIikpSWP/xMREDBw4ED169MDZs2cxc+ZMvP/++9i6dauyT1xcHEaMGIHg4GCcO3cOwcHBeP3113HixAlln5iYGLz33ns4fvw4oqKiUFRUhMDAQOTm5lb5Z65tbMxMEPZ0UczF+64hLduIlicgIiKqBKMLTkuWLMG4ceMwfvx4NG/eHOHh4fD09MSqVas09l+9ejW8vLwQHh6O5s2bY/z48Rg7diy++eYbZZ/w8HD0798fM2bMQLNmzTBjxgz07dsX4eHhyj579+7FmDFj0LJlS7Rt2xbr1q1DUlISzpw5U9UfuVYa1dELberbIrugCPMjrxi6HCIiIr2QGrqAZxUWFuLMmTOYPn26SntgYCCOHTum8T1xcXEIDAxUaQsKCsLatWshl8thYmKCuLg4TJ06Va3Ps8Hp3zIzMwEA9vb2ZfYpKChAQUGB8vusrJJb8OVyOeRyeZnvqytmD2qG4WtO4PezyRjW3g2dfMs+lnqnUEACoFhRDAXPhcGV/jzw58LweC6MB8+F8ajIOTCq4JSeno7i4mK4uLiotLu4uCA1NVXje1JTUzX2LyoqQnp6Otzc3MrsU9Y2BUFAWFgYunfvjlatWpVZ7/z58zF37ly19kOHDsHCwqLM99UlXZ3FOHpfjA83nsLHbYohqaZrnG3u3oUvgBs3buBabmT17JSeKyoqytAl0FM8F8aD58Lw8vLytO5rVMGplEikunihIAhqbc/r/+/2imxz0qRJ+Ouvv3DkyJFy65wxYwbCwsKU32dlZcHT0xO9e/eGg4NDue+tK7rmyRG49AhS8+S4b9cC45/ecVfl/ogCHgINGzREo14Dq2efVCa5XI6oqCj0798fJiYmhi6nTuO5MB48F8ajdMRIG0YVnBwdHSGRSNSuBKWlpaldMSrl6uqqsb9UKlWGl7L6aNrm5MmTsXPnTsTGxqJ+/frl1iuTySCTydTaTUxM+EPwlJOtCWYMbI6Pt/yFpQevY2Abd3g7WFb5fovFJZe2JBIxJDwXRoM/G8aD58J48FwYXkWOv1FNDjc1NYWfn5/aZcuoqCh07dpV43u6dOmi1n/fvn3w9/dXHoiy+jy7TUEQMGnSJGzbtg0HDx6Er6+vPj4SAXjNrz66NnRAvlyB6VvPQ6GohhXFy7lCSUREpCujCk4AEBYWhh9++AE//vgjLl++jKlTpyIpKQkTJ04EUDI89vbbbyv7T5w4Ebdv30ZYWBguX76MH3/8EWvXrsWHH36o7BMaGop9+/Zh4cKFuHLlChYuXIj9+/djypQpyj7vvfcefv75Z/z666+wtrZGamoqUlNT8eTJk2r77LWVSCTCglfbwNxEgribD7HxlOalJYiIiIyd0QWnESNGIDw8HPPmzUO7du0QGxuLyMhIeHt7AwBSUlJU1nTy9fVFZGQkoqOj0a5dO3z++edYtmwZhg0bpuzTtWtXbNq0CevWrUObNm0QERGBzZs3o1OnTso+q1atQmZmJnr16gU3Nzfl1+bNm6vvw9diXg4W+CioKQBgfuQVJD9mICUioprHqOY4lQoJCUFISIjG1yIiItTaAgICEB8fX+42hw8fjuHDh5f5Oh9IW/VGd/XB7vMpOHP7EWZuO4+Id14od9I/ERGRsTG6K05Ue0nEIiwc1gamUjFirj3AxpN3DF0SERFRhTA4UbVq5GyFjwJLhuw+/+MSbjzIMXBFRERE2mNwomo3rrsvujVywBN5MaZsSkBhkcLQJREREWmFwYmqnVgswuLX2sHOwgTnkzPxf/uvGbokIiIirTA4kUG42pphwattAACrY24g7sZDA1dERET0fAxOZDADWrlihL8nBAEI3XQWadn5+t8J75YkIiI9YnAig5r9cgs0drZCWnYB3t94FkXFnO9ERETGi8GJDMrCVIpVb/nB0lSC4zcz8M0+fc134vpQRESkfwxOZHCNnK2wcPg/852iLt03cEVERESaMTiRURjcxh3vdPMBAIT9loCbXN+JiIiMEIMTGY0ZLzaHn3c9ZOcXYdz608jMkxu6JCIiIhUMTmQ0TKVirH7LDx525khMz0XIr2cg52RxIiIyIgxOZFScrGX4YbQ/LEwlOHr9IebuumjokoiIiJQYnMjoNHezwdKR7SESAT8fT8K6o4mGLomIiAgAgxMZqf4tXDBtQDMAwLw/LuGPv+4ZuCIiIiIGJzJi/+3ZAMGdvSEIwNTNCTh6Pd3QJRERUR3H4ERGSyQSYc7LLTGwtSvkxQL+s+E0LiRnVnArfOQKERHpD4MTGTWJWIT/G9EOXRo4ILewGGPWneQaT0REZDAMTmT0ZFIJ1rzth5buNkjPKcSo748jMT23/DeJ+MgVIiLSPwYnqhGszUywYWxHNHGxwv2sAoxacxy3Hz4nPBEREekZgxPVGA5WMvwyvjMaOVshNSsfo9YcR9LDPEOXRUREdQiDE9UoTtYy/DqhExo6WeJeZj5GronjnCciIqo2DE5U4zhbm2HjhM5o8DQ8vbY6Toe77YiIiCqOwYlqJGcbM/z23y5o6W6Dh7mFGLXmOE7cfGjosoiIqJZjcKIay9FKho3/6YyOvvbILijC2z+exP5L9w1dFhER1WIMTlSj2Ty9265vM2cUFCnwn59OlzzbTuDCl0REpH8MTlTjmZlIsDrYDyP8PaEQgLm7LuHU7ceGLouIiGohqaELINIHE4kYC4a1RgMnSyzYewXX7uegsxRQ3IyBBApDl1fnicUyyORuhi6DiKjSGJyo1hCJRPhvQEN4O1jizm8bAQAmySeB5JMGrowkABo6DwIwytClEBFVCoMT1ToDWrnivHgmftgshri4AFKxCN0aOaKhk5WhS6ubkk8Dd09BWszFSomo5mNwolqpWePG+Kv1MOx+5IxjNzKAS8CYrj6YObA5TKWc2letYr4G7p4Cnx5IRLUBf4NQrWVlAvz4th8m9W4EAIg4dgvDVh3DDa40Xs1KIxPvdCSimo/BiWo1iViED4Oa4oe3/WFnYYLzyZkYvOwIfjlxGwKXLKgeyktNPN5EVPMxOFGd0K+FC/aG9kS3Rg54Ii/GJ79fwIQNZ5CeU2Do0uoA0TP/S0RUszE4UZ3hamuGn8Z2wqxBzWEqEWP/5fvovyQG2+Lv8upTVRI9jUw8xkRUCzA4UZ0iFoswvkcDbH+vG5q72eBRnhxhv53D6HWncCeDd31VLQYnIqr5GJyoTmrhboOdk7rho6CmMJWKEXvtAQL/Lxbfx95EYREXzNQvDtURUe3B4ER1lolEjPd6N8Le0B7o6GuPJ/JifBl5GQOWxiL6apqhy6s9RIxMRFR7MDhRndfAyQqbJnTG18PawNHKFDcf5GLMulMYG3EKiem5hi6vFuByBERUezA4EaFk7tPrL3ji4Ie9MKGHL6RiEQ5eSUP/JTGYtf080rLyDV1izSVicCKi2oPBiegZNmYm+GRQC/w5tSd6NXVCkULAz8eT0HPRIczfcxmP8woNXWIN9HSOE3MTEdUCDE5EGjR0skLEOx2xcUJndPCyQ75cge9ibqLHwkNYEnUNj3IZoLTGK05EVIswOBGVo0tDB2x9tyvWjvZHM1drZBcUYdmBv9F1wUHM23UJKZlPDF1iDcDJ4URUe/Ahv0TPIRKJ0Le5C3o3dUbkhRSsir6Bi/ey8OPRRPx0/BZeae+Bd7r5ormbjaFLNU684kREtQiDE5GWxGIRBrdxx6DWboj9Ox2roq/j+M0M/Hb6Ln47fRcdfe0xuosPAlu6wETCi7n/KF3HicGJiGo+BieiChKJRAho4oSAJk6IT3qEHw7fxJ8X7+NkYgZOJmbAxUaGUR298Jq/JzzszA1druEpH7li2DKIiPSBwYmoEjp41cPKN/2QmpmPX0/cxq8n7+B+VgHC9/+NpQf+RreGjhjuVx9BLV1hbioxdLkGwqE6Iqo9GJyI9MDV1gxhgU0xqU9j7LmQgl9PJOFEYgaOXE/HkevpsJJJMbiNG15u646OvvaQ1sGhPA7VEVFtwOBEpEemUjGGtPPAkHYeuJORh63xd7HlzF3cffQEm07dwaZTd+BgaYqgVq4Y1NoNnepCiOIjV4ioFmFwIqoinvYWmNKvCd7v0xgnb2Xg9/hk/HkpFQ9zC/HriST8eiIJ9pam6N/cBb2bOaN7Y0dYyWrjjySH6oio9qiNf0sTGRWxWITODRzQuYEDvihuhbgbD7HnQgr2XkhFRm4hNp++g82n78BEIkJHX3v0buqMXk2d0dDJEqLacLVGVHpXHRFRzcfgRFSNTCRi9GzihJ5NnPD5kFY4fjMDB67cx6Erabj1MA9Hrz/E0esP8cXuy/CwM38auOzRpaED6tezMHT5lSPwihMR1XwMTkQGIpWI0b2xI7o3dsTsl1oiMT0XB6+kIfpqGk7czEDy4yfYGn8XW+PvAgA87c3R2dcBnRo4oL2XHXwdLCEW14DrOFwAk4hqEQYnIiPh62iJcd19Ma67L/IKi3D61iPE3XyI4zcf4q+7mbiT8QR3Mu7if2dKgpStuQnaetqhvacd2nvZoZ2nHewsTA38KTThUB0R1R4MTkRGyMJUqhzSA4CcgiKcupWB4zce4sztRzifnInMJ3LEXnuA2GsPlO/ztDdHCzcbNHezQQs3G7Rwt4GHnblh50rxihMR1SIMTkQ1gJVMit5NndG7qTMAQF6swJWUbJy98wgJSY9x9s5jJKbnPr0q9QR/XryvfK+NmRTN3WzQ1NUaDZ2sSr6cLeFqY1ZNgYrBiYhqDwYnohrIRCJG6/q2aF3fFm93KWl7nFeISylZuHQvC5dTsnEpJQvX07KRlV+EE4kZOJGYobINC1MJGjhZKsOUt4MF6tezgKe9OZysZPoLVbXhzkAioqcYnIhqCTsLU3Rt6IiuDR2VbYVFCtx4kINL97Jw/UEObqTl4MaDHNx+mIe8wmJcSM7CheQstW2ZmYhLQlQ9c3jaW8CzngXq1zOHi60ZXG3M4GQtq8CDjPmsOiKqPRiciGoxU6kYzZ/OeXqWvFiBpIy8p0EqFzce5OBORh7uPnqClMwnyJcrcD0tB9fTcjRuVyQCHK1kcLGRwdXGDC42JYHKxdYMjlamsLeUwcHSFA5WpihdRIGPXCGi2sAog9PKlSuxaNEipKSkoGXLlggPD0ePHj3K7B8TE4OwsDBcvHgR7u7u+PjjjzFx4kSVPlu3bsWnn36KGzduoGHDhvjyyy/xyiuvVGq/RDWViUSsHKL7t8IiBVIyS+ZK3XmUhzsZebjz6AmSH+XhflYB0rLzIS8W8CC7AA+yCzResXrWm6YX8KUYsMm6jAcL20MiFim/xKKSr5L/LlksVCwSQSIScYSvikhFYnia9wAw0NClENVIRhecNm/ejClTpmDlypXo1q0bvvvuO7z44ou4dOkSvLy81PonJiZi4MCBmDBhAn7++WccPXoUISEhcHJywrBhwwAAcXFxGDFiBD7//HO88sor+P333/H666/jyJEj6NSpk077JaqtTKVieDtYwtvBUuPrCoWAjLxCpGbm435WPlKz8nE/8+n/ZxXgYW4BMnIKkZ5biMIiBa7LXQAZYIECWBTdqeZPQ/8mAtDAPB/AfEOXQlQjiQTBuJbz7dSpEzp06IBVq1Yp25o3b46hQ4di/nz1H/Rp06Zh586duHz5srJt4sSJOHfuHOLi4gAAI0aMQFZWFvbs2aPsM2DAANSrVw8bN27Uab+aZGVlwdbWFunp6XBwcKjYBye9ksvliIyMxMCBA2FiYmLocuokQRCQW1iMjJxCZCb9hbPHY+Hm3RC5hcXIeiJHdn4RcguLkVdQhJzCIuQVFiO3oAhPCourpB6pRARTiRgmUjFMJGKYSsQwlYphKhHBVCqBiURU8r1UUtImKelnIhVDKhZBKim5EiaViJX/bSIRQyIWQSoWQSIRwUT89HuJCBKRGFJJyUKnUtHTNnHJtp69uiYWARKxCKLqWOnq/gVgz8fIlrnB7MPz/NkwMP49ZTxKf39nZmbCxsam3L5GdcWpsLAQZ86cwfTp01XaAwMDcezYMY3viYuLQ2BgoEpbUFAQ1q5dC7lcDhMTE8TFxWHq1KlqfcLDw3XeLxGVTyQSwUomhZVMCrmNH24m3Uev/s//BVFUrEB2fhGy8uXIfCJH1pMiZD6RI6dAjtyCYuQVlgSu3IIile/zCoqQU1ASwPIKS157In8mhBU9/Sqoyk8t4J9Z8BULgKUBSiwqCWLip8OZUvE/w5nKL5HqcKdU8q8+T/9bJCo5D2IRIBaJ0KDAArMAmBRk4K//ewWifw2Jip7+T2mIK31NrV35/dPXNTyPUPRM/2f7quxLrUFUdh+RhrYytyUq+zX1LmVsT3WH5cVaDaVrFYMFQYBPZi7ihWSIxdrdbFGReF1lw90V2HCF6q1YEdr31KJrTt4TrbdnVMEpPT0dxcXFcHFxUWl3cXFBamqqxvekpqZq7F9UVIT09HS4ubmV2ad0m7rsFwAKCgpQUPDP38KZmZkAgIyMjLLeQtVELpcjLy8PDx8+5L/kjIAu58MKgJU54GEuAmD69KtiihUCCoqKUSBXoKBYgcKiYuTLFciXK1BYrEBBkQKFcgXyi4qRX6SAvKgY+UVCSVtxyfsKixQoUiggLxZQVKyAXPH0/4uh/F5erIBcoUBxccl/Fz7tW1QslLxXIaCoWIC8uOwL/AqU5Lqq5Iw8TJIBYlE+GhfEVvHeSBs2AHCC/0A3tKyCkp9NbQbhjCo4lfr3+jGCIJS7poym/v9u12abFd3v/PnzMXfuXLX2Jk2alPkeIiJDuQOgnqGLIDJi2dnZsLW1LbePUQUnR0dHSCQStas8aWlpaleDSrm6umrsL5VKlfOMyupTuk1d9gsAM2bMQFhYmPL7x48fw9vbG0lJSc898FS1srKy4OnpiTt37jx3vJqqHs+H8eC5MB48F8ZDEARkZ2fD3d39uX2NKjiZmprCz88PUVFRKksFREVFYciQIRrf06VLF+zatUulbd++ffD391cOCXTp0gVRUVEq85z27duHrl276rxfAJDJZJDJZGrttra2/CEwEjY2NjwXRoTnw3jwXBgPngvjoO0FD6MKTgAQFhaG4OBg+Pv7o0uXLlizZg2SkpKU6zLNmDEDycnJ2LBhA4CSO+hWrFiBsLAwTJgwAXFxcVi7dq3ybjkACA0NRc+ePbFw4UIMGTIEO3bswP79+3HkyBGt90tERERkdMFpxIgRePjwIebNm4eUlBS0atUKkZGR8Pb2BgCkpKQgKSlJ2d/X1xeRkZGYOnUqvv32W7i7u2PZsmXKNZwAoGvXrti0aRNmzZqFTz/9FA0bNsTmzZuVazhps18iIiIio1vHqSYrKCjA/PnzMWPGDI1DeFR9eC6MC8+H8eC5MB48FzUTgxMRERGRlrR9vDkRERFRncfgRERERKQlBiciIiIiLTE4EREREWmJwUmPVq5cCV9fX5iZmcHPzw+HDx82dEk1xvz58/HCCy/A2toazs7OGDp0KK5evarSRxAEzJkzB+7u7jA3N0evXr1w8eJFlT4FBQWYPHkyHB0dYWlpiZdffhl3795V6fPo0SMEBwfD1tYWtra2CA4OxuPHj1X6JCUl4aWXXoKlpSUcHR3x/vvvo7CwsEo+u7GbP38+RCIRpkyZomzjuaheycnJeOutt+Dg4AALCwu0a9cOZ86cUb7O81E9ioqKMGvWLPj6+sLc3BwNGjTAvHnzoFAolH14LuoAgfRi06ZNgomJifD9998Lly5dEkJDQwVLS0vh9u3bhi6tRggKChLWrVsnXLhwQUhISBAGDRokeHl5CTk5Oco+CxYsEKytrYWtW7cK58+fF0aMGCG4ubkJWVlZyj4TJ04UPDw8hKioKCE+Pl7o3bu30LZtW6GoqEjZZ8CAAUKrVq2EY8eOCceOHRNatWolDB48WPl6UVGR0KpVK6F3795CfHy8EBUVJbi7uwuTJk2qnoNhRE6ePCn4+PgIbdq0EUJDQ5XtPBfVJyMjQ/D29hbGjBkjnDhxQkhMTBT2798vXL9+XdmH56N6fPHFF4KDg4Pwxx9/CImJicL//vc/wcrKSggPD1f24bmo/Ric9KRjx47CxIkTVdqaNWsmTJ8+3UAV1WxpaWkCACEmJkYQBEFQKBSCq6ursGDBAmWf/Px8wdbWVli9erUgCILw+PFjwcTERNi0aZOyT3JysiAWi4W9e/cKgiAIly5dEgAIx48fV/aJi4sTAAhXrlwRBEEQIiMjBbFYLCQnJyv7bNy4UZDJZEJmZmbVfWgjk52dLTRu3FiIiooSAgIClMGJ56J6TZs2TejevXuZr/N8VJ9BgwYJY8eOVWl79dVXhbfeeksQBJ6LuoJDdXpQWFiIM2fOIDAwUKU9MDAQx44dM1BVNVtmZiYAwN7eHgCQmJiI1NRUlWMsk8kQEBCgPMZnzpyBXC5X6ePu7o5WrVop+8TFxcHW1lZl1fjOnTvD1tZWpU+rVq1UHvYYFBSEgoICleGR2u69997DoEGD0K9fP5V2novqtXPnTvj7++O1116Ds7Mz2rdvj++//175Os9H9enevTsOHDiAa9euAQDOnTuHI0eOYODAgQB4LuoKo3vkSk2Unp6O4uJiuLi4qLS7uLggNTXVQFXVXIIgICwsDN27d0erVq0AQHkcNR3j27dvK/uYmpqiXr16an1K35+amgpnZ2e1fTo7O6v0+fd+6tWrB1NT0zpzPjdt2oT4+HicOnVK7TWei+p18+ZNrFq1CmFhYZg5cyZOnjyJ999/HzKZDG+//TbPRzWaNm0aMjMz0axZM0gkEhQXF+PLL7/EqFGjAPBno65gcNIjkUik8r0gCGpt9HyTJk3CX3/9pfIQ5lK6HON/99HUX5c+tdWdO3cQGhqKffv2wczMrMx+PBfVQ6FQwN/fH1999RUAoH379rh48SJWrVqFt99+W9mP56Pqbd68GT///DN+/fVXtGzZEgkJCZgyZQrc3d0xevRoZT+ei9qNQ3V64OjoCIlEopby09LS1P5FQOWbPHkydu7ciUOHDqF+/frKdldXVwAo9xi7urqisLAQjx49KrfP/fv31fb74MEDlT7/3s+jR48gl8vrxPk8c+YM0tLS4OfnB6lUCqlUipiYGCxbtgxSqVR5DHguqoebmxtatGih0ta8eXPlw875s1F9PvroI0yfPh0jR45E69atERwcjKlTp2L+/PkAeC7qCgYnPTA1NYWfnx+ioqJU2qOiotC1a1cDVVWzCIKASZMmYdu2bTh48CB8fX1VXvf19YWrq6vKMS4sLERMTIzyGPv5+cHExESlT0pKCi5cuKDs06VLF2RmZuLkyZPKPidOnEBmZqZKnwsXLiAlJUXZZ9++fZDJZPDz89P/hzcyffv2xfnz55GQkKD88vf3x5tvvomEhAQ0aNCA56IadevWTW1pjmvXrsHb2xsAfzaqU15eHsRi1V+bEolEuRwBz0UdUc2T0Wut0uUI1q5dK1y6dEmYMmWKYGlpKdy6dcvQpdUI7777rmBraytER0cLKSkpyq+8vDxlnwULFgi2trbCtm3bhPPnzwujRo3SeJtv/fr1hf379wvx8fFCnz59NN7m26ZNGyEuLk6Ii4sTWrdurfE23759+wrx8fHC/v37hfr169fp23yfvatOEHguqtPJkycFqVQqfPnll8Lff/8t/PLLL4KFhYXw888/K/vwfFSP0aNHCx4eHsrlCLZt2yY4OjoKH3/8sbIPz0Xtx+CkR99++63g7e0tmJqaCh06dFDeSk/PB0Dj17p165R9FAqFMHv2bMHV1VWQyWRCz549hfPnz6ts58mTJ8KkSZMEe3t7wdzcXBg8eLCQlJSk0ufhw4fCm2++KVhbWwvW1tbCm2++KTx69Eilz+3bt4VBgwYJ5ubmgr29vTBp0iQhPz+/qj6+0ft3cOK5qF67du0SWrVqJchkMqFZs2bCmjVrVF7n+ageWVlZQmhoqODl5SWYmZkJDRo0ED755BOhoKBA2YfnovYTCYIgGPKKFxEREVFNwTlORERERFpicCIiIiLSEoMTERERkZYYnIiIiIi0xOBEREREpCUGJyIiIiItMTgRERERaYnBiYiIiEhLDE5EVGvl5eXhq6++QocOHWBlZQUzMzPUr18fPXr0wIwZM3Djxg1lXx8fH/j4+BiuWCKqEaSGLoCIqCpkZ2eje/fu+Ouvv9CoUSO89dZbsLOzw507d3Dx4kUsWLAADRs2RMOGDQ1dKhHVIAxORFQrhYeH46+//sK4cePw/fffQyQSqbyemJiIgoICA1VHRDUVh+qIqFaKi4sDAEyaNEktNAGAr68vmjVrhlu3bkEkEuH27du4ffs2RCKR8mvOnDkq74mNjcVLL70ER0dHyGQyNG7cGLNmzUJeXp5Kv+joaOX7Y2NjERAQACsrK9jb2+ONN97A3bt3q+xzE1HVYnAiolrJ3t4eAHD9+vVy+9nZ2WH27NmwtbWFra0tZs+erfzq1auXst/q1avRq1cvHDt2DIMHD8b7778PDw8PfPnll+jfvz8KCwvVtn38+HH0798fDg4OeP/999GxY0ds3LgRXbt2xf379/X6eYmomghERLXQ9u3bBQCCjY2NMG3aNOHAgQNCRkZGmf29vb0Fb29vja9dvHhRkEqlQvv27YWHDx+qvDZ//nwBgPDNN98o2w4dOiQAEAAIP/zwg0r/uXPnCgCEsWPH6v7hiMhgRIIgCAZNbkREVWTRokWYN28ecnJylG0NGzbEgAEDEBoaisaNGyvbS++ou3Xrltp2QkNDsWzZMhw+fBjdu3dXeU2hUMDV1RVeXl44ffo0gJKhut69e6Np06a4fPmyylDhkydP4O3tjZycHDx+/BimpqZ6/MREVNU4OZyIaq2PPvoIEydOxN69e3Hs2DGcPn0aJ06cwLfffou1a9di8+bNePnll5+7nePHjwMA9u7di/3796u9bmJigitXrqi1d+vWTW1+lbm5Ofz8/LB3715cu3YNrVq10vHTEZEhMDgRUa1mbW2N1157Da+99hoAIDMzEzNnzsTKlSsxbtw4JCcnP/eqT0ZGBgDgyy+/rNC+nZ2dNba7uLgoayGimoWTw4moTrG1tcWKFSvg7e2N9PR0nD9//rnvsbGxAQBkZWVBEIQyv/4tLS1N4/ZKJ4bb2tpW4pMQkSEwOBFRnSMSiWBhYaHSJpFIUFxcrLF/p06dAPwzZKeto0ePqgWqJ0+e4MyZMzA3N0eTJk0qtD0iMjwGJyKqlb777jucOnVK42vbtm3DlStXYGdnp5xjZG9vj/T0dOTn56v1DwkJgVQqxeTJk3Hnzh211x8/foyzZ8+qtV+9ehU//vijStuiRYvw4MEDjBo1ihPDiWogznEiolppz549mDhxIho1aoRu3brB3d0dOTk5SEhIwOHDhyEWi7Fy5UrIZDIAQJ8+fXD69Gm89NJL6NGjB0xNTdG9e3d0794drVq1wsqVK/Huu++iadOmGDhwIBo2bIisrCzcvHkTMTExGDNmDFavXq1SQ2BgIEJCQrB79240a9YM8fHx+PPPP+Hp6YmvvvrKEIeFiCqJyxEQUa109epV7Ny5E1FRUbh+/TpSUlIAAB4eHujevTsmT54MPz8/Zf+cnByEhYXhjz/+wP3796FQKDB79myV1cNPnTqFJUuWIDY2Fg8ePICtrS28vLwQGBiI0aNHo1mzZgD+WY5g9uzZ6NOnD2bNmoUzZ87A1NQUAwYMwNdffw1PT89qPR5EpB8MTkREevZscPr3Y1uIqGbjHCciIiIiLTE4EREREWmJwYmIiIhIS5zjRERERKQlXnEiIiIi0hKDExEREZGWGJyIiIiItMTgRERERKQlBiciIiIiLTE4EREREWmJwYmIiIhISwxORERERFpicCIiIiLS0v8DpANmHCxio2MAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# extra code – this cell plots exponential scheduling\n",
"\n",
"initial_learning_rate = 0.01\n",
"decay_rate = 0.1\n",
"decay_steps = 20_000\n",
"\n",
"steps = np.arange(100_000)\n",
"lrs = initial_learning_rate * decay_rate ** (steps / decay_steps)\n",
"lrs2 = initial_learning_rate * decay_rate ** np.floor(steps / decay_steps)\n",
"\n",
"plt.plot(steps, lrs, \"-\", label=\"staircase=False\")\n",
"plt.plot(steps, lrs2, \"-\", label=\"staircase=True\")\n",
"plt.axis([0, steps.max(), 0, 0.0105])\n",
"plt.xlabel(\"Step\")\n",
"plt.ylabel(\"Learning Rate\")\n",
"plt.title(\"Exponential Scheduling\", fontsize=14)\n",
"plt.legend()\n",
"plt.grid(True)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Keras also provides a `LearningRateScheduler` callback class that lets you define your own scheduling function. Let's see how you could use it to implement exponential decay. Note that in this case the learning rate only changes at each epoch, not at each step:"
]
},
{
"cell_type": "code",
"execution_count": 72,
"metadata": {},
"outputs": [],
"source": [
"def exponential_decay_fn(epoch):\n",
" return 0.01 * 0.1 ** (epoch / 20)"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {},
"outputs": [],
"source": [
"def exponential_decay(lr0, s):\n",
" def exponential_decay_fn(epoch):\n",
" return lr0 * 0.1 ** (epoch / s)\n",
" return exponential_decay_fn\n",
"\n",
"exponential_decay_fn = exponential_decay(lr0=0.01, s=20)"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [],
"source": [
"# extra code – build and compile a model for Fashion MNIST\n",
"\n",
"tf.random.set_seed(42)\n",
"model = build_model()\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.001)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.6905 - accuracy: 0.7643 - val_loss: 0.4814 - val_accuracy: 0.8330 - lr: 0.0100\n",
"Epoch 2/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4672 - accuracy: 0.8357 - val_loss: 0.4488 - val_accuracy: 0.8374 - lr: 0.0089\n",
"Epoch 3/25\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.4212 - accuracy: 0.8503 - val_loss: 0.4118 - val_accuracy: 0.8532 - lr: 0.0079\n",
"Epoch 4/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3975 - accuracy: 0.8593 - val_loss: 0.3884 - val_accuracy: 0.8636 - lr: 0.0071\n",
"Epoch 5/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3781 - accuracy: 0.8657 - val_loss: 0.3772 - val_accuracy: 0.8642 - lr: 0.0063\n",
"Epoch 6/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3634 - accuracy: 0.8710 - val_loss: 0.3779 - val_accuracy: 0.8662 - lr: 0.0056\n",
"Epoch 7/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3530 - accuracy: 0.8744 - val_loss: 0.3674 - val_accuracy: 0.8652 - lr: 0.0050\n",
"Epoch 8/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3437 - accuracy: 0.8771 - val_loss: 0.3616 - val_accuracy: 0.8686 - lr: 0.0045\n",
"Epoch 9/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3359 - accuracy: 0.8801 - val_loss: 0.3509 - val_accuracy: 0.8728 - lr: 0.0040\n",
"Epoch 10/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3290 - accuracy: 0.8826 - val_loss: 0.3504 - val_accuracy: 0.8720 - lr: 0.0035\n",
"Epoch 11/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3236 - accuracy: 0.8844 - val_loss: 0.3458 - val_accuracy: 0.8736 - lr: 0.0032\n",
"Epoch 12/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3186 - accuracy: 0.8869 - val_loss: 0.3459 - val_accuracy: 0.8752 - lr: 0.0028\n",
"Epoch 13/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3147 - accuracy: 0.8878 - val_loss: 0.3359 - val_accuracy: 0.8770 - lr: 0.0025\n",
"Epoch 14/25\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.3109 - accuracy: 0.8890 - val_loss: 0.3404 - val_accuracy: 0.8762 - lr: 0.0022\n",
"Epoch 15/25\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3076 - accuracy: 0.8902 - val_loss: 0.3398 - val_accuracy: 0.8790 - lr: 0.0020\n",
"Epoch 16/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3043 - accuracy: 0.8915 - val_loss: 0.3331 - val_accuracy: 0.8784 - lr: 0.0018\n",
"Epoch 17/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3020 - accuracy: 0.8924 - val_loss: 0.3363 - val_accuracy: 0.8774 - lr: 0.0016\n",
"Epoch 18/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2998 - accuracy: 0.8927 - val_loss: 0.3356 - val_accuracy: 0.8778 - lr: 0.0014\n",
"Epoch 19/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2979 - accuracy: 0.8935 - val_loss: 0.3309 - val_accuracy: 0.8796 - lr: 0.0013\n",
"Epoch 20/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2961 - accuracy: 0.8940 - val_loss: 0.3308 - val_accuracy: 0.8782 - lr: 0.0011\n",
"Epoch 21/25\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.2944 - accuracy: 0.8951 - val_loss: 0.3286 - val_accuracy: 0.8802 - lr: 0.0010\n",
"Epoch 22/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2930 - accuracy: 0.8953 - val_loss: 0.3313 - val_accuracy: 0.8804 - lr: 8.9125e-04\n",
"Epoch 23/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2916 - accuracy: 0.8957 - val_loss: 0.3285 - val_accuracy: 0.8796 - lr: 7.9433e-04\n",
"Epoch 24/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2904 - accuracy: 0.8961 - val_loss: 0.3313 - val_accuracy: 0.8786 - lr: 7.0795e-04\n",
"Epoch 25/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2896 - accuracy: 0.8962 - val_loss: 0.3296 - val_accuracy: 0.8812 - lr: 6.3096e-04\n"
]
}
],
"source": [
"n_epochs = 20\n",
"\n",
"lr_scheduler = tf.keras.callbacks.LearningRateScheduler(exponential_decay_fn)\n",
"history = model.fit(X_train, y_train, epochs=n_epochs,\n",
" validation_data=(X_valid, y_valid),\n",
" callbacks=[lr_scheduler])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternatively, the schedule function can take the current learning rate as a second argument:"
]
},
{
"cell_type": "code",
"execution_count": 76,
"metadata": {},
"outputs": [],
"source": [
"def exponential_decay_fn(epoch, lr):\n",
" return lr * 0.1 ** (1 / 20)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Extra material**: if you want to use a custom scheduling function that updates the learning rate at each iteration rather than at each epoch, you can write your own callback class like this:"
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {},
"outputs": [],
"source": [
"K = tf.keras.backend\n",
"\n",
"class ExponentialDecay(tf.keras.callbacks.Callback):\n",
" def __init__(self, n_steps=40_000):\n",
" super().__init__()\n",
" self.n_steps = n_steps\n",
"\n",
" def on_batch_begin(self, batch, logs=None):\n",
" # Note: the `batch` argument is reset at each epoch\n",
" lr = K.get_value(self.model.optimizer.learning_rate)\n",
" new_learning_rate = lr * 0.1 ** (1 / self.n_steps)\n",
" K.set_value(self.model.optimizer.learning_rate, new_learning_rate)\n",
"\n",
" def on_epoch_end(self, epoch, logs=None):\n",
" logs = logs or {}\n",
" logs['lr'] = K.get_value(self.model.optimizer.learning_rate)"
]
},
{
"cell_type": "code",
"execution_count": 78,
"metadata": {},
"outputs": [],
"source": [
"lr0 = 0.01\n",
"model = build_model()\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=lr0)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": 79,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.6947 - accuracy: 0.7635 - val_loss: 0.5014 - val_accuracy: 0.8224 - lr: 0.0091\n",
"Epoch 2/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4718 - accuracy: 0.8349 - val_loss: 0.4530 - val_accuracy: 0.8382 - lr: 0.0083\n",
"Epoch 3/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4255 - accuracy: 0.8500 - val_loss: 0.4216 - val_accuracy: 0.8526 - lr: 0.0076\n",
"Epoch 4/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4025 - accuracy: 0.8587 - val_loss: 0.3954 - val_accuracy: 0.8618 - lr: 0.0069\n",
"Epoch 5/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3840 - accuracy: 0.8643 - val_loss: 0.3847 - val_accuracy: 0.8612 - lr: 0.0063\n",
"Epoch 6/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3696 - accuracy: 0.8689 - val_loss: 0.3908 - val_accuracy: 0.8558 - lr: 0.0058\n",
"Epoch 7/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.3590 - accuracy: 0.8722 - val_loss: 0.3744 - val_accuracy: 0.8670 - lr: 0.0052\n",
"Epoch 8/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3498 - accuracy: 0.8749 - val_loss: 0.3754 - val_accuracy: 0.8640 - lr: 0.0048\n",
"Epoch 9/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3415 - accuracy: 0.8783 - val_loss: 0.3592 - val_accuracy: 0.8700 - lr: 0.0044\n",
"Epoch 10/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3340 - accuracy: 0.8803 - val_loss: 0.3575 - val_accuracy: 0.8724 - lr: 0.0040\n",
"Epoch 11/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3281 - accuracy: 0.8833 - val_loss: 0.3573 - val_accuracy: 0.8718 - lr: 0.0036\n",
"Epoch 12/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3228 - accuracy: 0.8847 - val_loss: 0.3579 - val_accuracy: 0.8688 - lr: 0.0033\n",
"Epoch 13/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3182 - accuracy: 0.8865 - val_loss: 0.3421 - val_accuracy: 0.8756 - lr: 0.0030\n",
"Epoch 14/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3138 - accuracy: 0.8882 - val_loss: 0.3468 - val_accuracy: 0.8766 - lr: 0.0028\n",
"Epoch 15/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3101 - accuracy: 0.8889 - val_loss: 0.3471 - val_accuracy: 0.8766 - lr: 0.0025\n",
"Epoch 16/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3064 - accuracy: 0.8898 - val_loss: 0.3386 - val_accuracy: 0.8752 - lr: 0.0023\n",
"Epoch 17/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3035 - accuracy: 0.8903 - val_loss: 0.3417 - val_accuracy: 0.8758 - lr: 0.0021\n",
"Epoch 18/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3005 - accuracy: 0.8919 - val_loss: 0.3398 - val_accuracy: 0.8768 - lr: 0.0019\n",
"Epoch 19/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2983 - accuracy: 0.8929 - val_loss: 0.3357 - val_accuracy: 0.8766 - lr: 0.0017\n",
"Epoch 20/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2959 - accuracy: 0.8939 - val_loss: 0.3370 - val_accuracy: 0.8752 - lr: 0.0016\n",
"Epoch 21/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2940 - accuracy: 0.8938 - val_loss: 0.3346 - val_accuracy: 0.8782 - lr: 0.0014\n",
"Epoch 22/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2917 - accuracy: 0.8949 - val_loss: 0.3361 - val_accuracy: 0.8766 - lr: 0.0013\n",
"Epoch 23/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2902 - accuracy: 0.8955 - val_loss: 0.3349 - val_accuracy: 0.8796 - lr: 0.0012\n",
"Epoch 24/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2884 - accuracy: 0.8959 - val_loss: 0.3364 - val_accuracy: 0.8796 - lr: 0.0011\n",
"Epoch 25/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2871 - accuracy: 0.8969 - val_loss: 0.3352 - val_accuracy: 0.8802 - lr: 1.0000e-03\n"
]
}
],
"source": [
"import math\n",
"\n",
"batch_size = 32\n",
"n_steps = n_epochs * math.ceil(len(X_train) / batch_size)\n",
"exp_decay = ExponentialDecay(n_steps)\n",
"history = model.fit(X_train, y_train, epochs=n_epochs,\n",
" validation_data=(X_valid, y_valid),\n",
" callbacks=[exp_decay])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Piecewise Constant Scheduling"
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {},
"outputs": [],
"source": [
"lr_schedule = tf.keras.optimizers.schedules.PiecewiseConstantDecay(\n",
" boundaries=[50_000, 80_000],\n",
" values=[0.01, 0.005, 0.001]\n",
")\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=lr_schedule)"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.6942 - accuracy: 0.7617 - val_loss: 0.4892 - val_accuracy: 0.8318\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4751 - accuracy: 0.8340 - val_loss: 0.4603 - val_accuracy: 0.8346\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4280 - accuracy: 0.8500 - val_loss: 0.4245 - val_accuracy: 0.8542\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4035 - accuracy: 0.8581 - val_loss: 0.3867 - val_accuracy: 0.8626\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3828 - accuracy: 0.8650 - val_loss: 0.3827 - val_accuracy: 0.8634\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3665 - accuracy: 0.8700 - val_loss: 0.3880 - val_accuracy: 0.8608\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.3539 - accuracy: 0.8730 - val_loss: 0.3669 - val_accuracy: 0.8688\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3423 - accuracy: 0.8773 - val_loss: 0.3583 - val_accuracy: 0.8708\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3322 - accuracy: 0.8807 - val_loss: 0.3447 - val_accuracy: 0.8758\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3218 - accuracy: 0.8832 - val_loss: 0.3488 - val_accuracy: 0.8716\n"
]
}
],
"source": [
"history_piecewise_scheduling = build_and_train_model(optimizer) # extra code"
]
},
{
"cell_type": "code",
"execution_count": 82,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAHNCAYAAADolfQeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/SrBM8AAAACXBIWXMAAA9hAAAPYQGoP6dpAABTFUlEQVR4nO3deVxU1f8/8NfIMoAKKiiIsrokRLmAKcqikbgvpUlqpLn0JVJR+pjikssnJXP5kCmSRi4tSqWlFZqoiBtuLGauH01FCURcQCVhgPP7wx/34zQDXkZkBng9Hw8fOWfec++ZOYO8uvfccxVCCAEiIiIieqJ6+u4AERERUU3B4EREREQkE4MTERERkUwMTkREREQyMTgRERERycTgRERERCQTgxMRERGRTAxORERERDIxOBERERHJxOBEBMDZ2RnOzs767sYz06NHDygUCn13g2qZefPmQaFQYN++fdW+7ytXrkChUGDMmDFPtZ3y3oNCoUCPHj2eattUOzE4Ua1U9o/q439MTU3h4OCAkSNH4vfff9d3F+u0CxcuYNKkSXj++edhaWkJpVIJR0dHDBs2DFu2bEFpaam+u6ihqn5R62r9+vVQKBRYv359pV/7xx9/YPTo0XB2doZSqYSVlRVat26N1157DZ9++il45y0i+Yz13QGiZ6lVq1Z48803AQD379/HkSNHsGnTJmzduhV79+5Ft27dAAB79uzRZzefuY0bN6KgoEDf3QAALFu2DNOnT0dpaSl8fHzQq1cvWFhY4Nq1a9i9eze2bNmCsWPHIjY2Vt9drRUSEhIwYMAAFBcXIyAgAK+++ioA4M8//8ShQ4fw448/4r333oOxMX8dPO7s2bOwsLDQdzfIAPEnhWq11q1bY968eWpts2fPxsKFCzFr1iwkJiYCeBSwajNHR0d9dwEAsGbNGvzrX/+Cs7MztmzZgk6dOqk9X1xcjA0bNuDAgQN66mHt8+6776KkpAS7d+9Gz5491Z4TQmDXrl0wMjLSU+8MV7t27fTdBTJQPFVHdc6kSZMAAMePH5faypvjJITAl19+ie7du8PS0hIWFhbw8vLCl19+qXXbQghs2LABfn5+aNSoESwsLNCmTRuEhIQgIyNDrfbevXuYO3cunn/+eZibm6NRo0bo06cPDh48qFY3ZcoUKBQKpKenq7X3798fCoUC48ePV2vfsWMHFAoFFi9eLLVpm+NUWlqKL774Ai+99BKaNGkCCwsLODs7Y8iQIdi/f7/Ge9u/fz8GDhwIGxsbKJVKtGnTBrNnz5Z9JCsvLw/Tpk2Dqakpfv31V43QBADGxsYYN24cPv/8c7X2goICzJs3D+3atYOZmRmaNGmC/v374/DhwxrbeHzOynfffYdOnTrB3NwczZs3x+TJk/H3339rvGbLli3w9/dHs2bNYGZmBgcHB/Tp0wc//fQTgEenyVxcXAAAGzZsUDsFXDY35q+//sLcuXPRtWtXNGvWDEqlEs7OzggNDUVOTo7GPseMGQOFQoErV64gOjoabm5uMDMzg5OTE+bPn692unLMmDF4++23AQBvv/222v4rkpOTg0uXLsHDw0MjNAGP5vH07t1b63YOHDiAV199Fba2tlAqlXBwcMBrr72m8f0sI/ezBir3XSopKcHixYvRunVrmJmZoXXr1oiMjCz3dG5Fc5MqM5dR23YqM2ZlCgoK8MEHH8DBwQFmZmbw8PDA2rVrsW/fPigUCo3/sSPDxyNOVOfInSQthMCbb76Jb7/9Fm3btsXIkSNhamqKhIQEjBs3DmfOnMHSpUvV6keMGIG4uDi0aNECI0aMgKWlJa5cuYK4uDj06dNHOvJz+/Zt+Pn54fTp0/D19UXv3r2Rl5eHbdu2oWfPnvj+++8xZMgQAEDPnj3x6aefIjExER06dADw6JdJ2S+wsqNmZcp+kWv7Rfm4iIgIfPLJJ2jVqhVGjhyJhg0bIjMzEwcOHMDevXvh5+cn1cbExCA0NBSNGzfGwIED0bRpUxw/fhwLFy5EYmIiEhMTYWpqWuH+vv/+e+Tn52PkyJFwd3evsFapVEp/LywsREBAAI4cOYJOnTphypQpyMnJQVxcHHbt2oW4uDi89tprGttYtWoVduzYgcGDB6NHjx7YuXMnPvvsM9y6dQvffPONVLd69WqEhoaiefPmePXVV2FtbY2srCwcO3YMP/30E4YMGYIOHTogLCwMn376Kdq3by+NDQDpF/H+/fuxbNkyBAQEoEuXLjAxMUFaWhpWr16N3377DampqbCystLo57Rp07Bv3z4MGDAAgYGB+OmnnzBv3jwUFRVh4cKFAIAhQ4bg7t272LZtGwYPHix9D57EysoKRkZGyMrKwoMHD1C/fn1Zr1u1ahUmTZoEc3NzvPrqq3B0dERmZiYOHjyIH374AT4+Pjp91kDlv0vvvPMOvvzyS7i4uOC9997Dw4cPsXz5cq2hubrIGTPg0c/pgAEDkJiYiPbt22PkyJG4ffs23n//fU48r8kEUS10+fJlAUD07t1b47lZs2YJAKJHjx5Sm5OTk3ByclKrW7NmjQAgxo0bJ1QqldReWFgoBg4cKACIEydOSO2rVq0SAERAQIAoKChQ21ZBQYG4deuW9HjkyJECgPjyyy/V6rKzs4WDg4No2rSp+Pvvv4UQQty5c0fUq1dPDBw4UKo7evSotC8A4urVq9JznTt3Fg0bNhTFxcVSm7+/v/jnj3uTJk1EixYtxIMHD9TaS0tL1fp6+vRpYWxsLDp27KjWLoQQkZGRAoBYunSpeJIxY8YIAOKLL754Yu3jFixYIACIUaNGidLSUqn95MmTQqlUisaNG4v8/Hypfe7cuQKAsLKyEufOnZPaCwoKRNu2bYVCoRCZmZlSe6dOnYSpqanIycnR2Hdubq7097Lv1OjRo7X288aNG+LevXsa7Rs2bBAAxEcffaTWPnr0aAFAuLi4iL/++ktqv3nzpmjUqJFo2LChKCwslNrXrVsnAIh169Zp3X95hgwZIgCIDh06iOjoaJGeni6KiorKrf/999+FkZGRsLe3F5cvX1Z7rrS0VO2zq+xnXdnvUmJiogAg2rdvL+7fvy+1X79+XdjY2GgdDwDC399f63vT9nNe9h4SExOfuJ3KjtkXX3whAIhBgwaJkpISqf3s2bPCzMxMABBz587V2lcyXDxVR7XaxYsXMW/ePMybNw//+te/4OPjg4ULF8LMzAyLFi2q8LUrV65E/fr1sXLlSrWJs6amptL/VW7atElqX7VqFYyMjLB69WqYm5urbcvc3BxNmjQBAOTm5iIuLg4BAQHS6Zcytra2mDZtGm7evIndu3cDABo1aoT27dtj//79KCkpAfDoKNPjh/n37t0LAMjPz0dqaip8fX1lzVsxNTXVmBSsUCikvgLA559/juLiYqxYsUKtHQA++OADNG3aVO1zKE92djYAoGXLlk+sfdz69ethYmKCjz/+WO1o4YsvvogxY8bgzp072LZtm8brwsLC8Nxzz0mPzc3NMWLECAghkJKSolZrYmICExMTjW1YW1vL7mezZs3QoEEDjfbg4GBYWlpK4/lPc+bMQfPmzaXHNjY2GDx4MO7du4fz58/L3n951q5di/79+yM9PR2hoaHo0KEDGjRogO7du2PFihUap9NiYmJQUlKCjz76SOO0lkKhgL29vcY+5H7Wlf0ubdy4EQDw4Ycfqh0ta9GiBcLCwir/YVQRuWP29ddfAwD+/e9/o169//26bdeuHUaPHl19HaYqxVN1VKtdunQJ8+fPB/Dol6OtrS1GjhyJGTNm4IUXXij3dQUFBTh16hTs7e3x8ccfazyvUqkAAOfOnQMAPHjwAGfOnEHr1q3Rpk2bCvt0/PhxlJSU4OHDh1rnN/z3v/+Vtj1gwAAAj067paWlITU1FZ07d5YO/fv4+MDOzg6JiYkYM2aMFK6edJoOAIYPH46YmBh4eHggKCgI/v7+8Pb21jidc+TIEQDAzp07tf7yNzExkT6Hqpafn48///wTbm5uWgNXjx498PnnnyM9PV26erKMtjlUZdu4e/eu1DZ8+HDMmDEDHh4eeOONN9CjRw/4+PigUaNGle7v1q1b8fnnnyM1NRV37tyRgi7waA6UNnL7qSsbGxv88ssvuHDhAn777TccO3YMR44cweHDh3H48GGsXbsWSUlJUpA5duwYACAwMFD2PuS+h8p+l06ePAkA8PX11ajV1lZd5L7fkydPon79+njxxRc16rt166Yxl49qBgYnqtV69+6NnTt3Vvp1d+7cgRACmZmZUvDS5sGDBwD+949lixYtnrjt27dvAwAOHTqEQ4cOPXHbwKPgtHz5ciQmJqJjx444dOgQJkyYAOBReCib51T2XznBacWKFXB1dcX69evx0Ucf4aOPPoKZmRmGDx+OZcuWwcbGRq2/j8/d0IWdnR0AIDMzU/Zr8vPzATw6ElfRNvPy8jSe0zafqOzo2uOB5oMPPoC1tTViYmKwfPlyLFu2DMbGxujXrx+ioqKkSeFPsmzZMvzrX/9C06ZNERgYiJYtW0pHHqOiolBYWKj1dXL7+bTatm2Ltm3bSo/LwuYff/yB+fPn49NPPwXw6LusUCjUjqg8idz3UNnvUl5eHurVqyd9Fx9X3neiOsh9v/n5+XBwcNC6DX32n54OT9URaWFpaQkA8PT0hBCi3D9lQaXsH1I5oaBs2++//36F2547d670Gj8/PxgZGSExMRHHjx/H/fv3pXDUs2dPXLt2DZcuXcK+fftgZWWFjh07PrEfJiYmmDZtGk6fPo3MzEx8++238PX1xcaNGzFq1CiN/ubn51fY3yfp3r07gMqtmVW27xs3bmh9vqy9rE4XZVcmnjhxAjdv3sSPP/6I1157Ddu3b0f//v1lhZfi4mL8+9//hr29PU6fPo1vvvkGixcvxrx58zB37lwUFRXp3L9npUOHDvjss88A/O9UL/Do1LAQAllZWVW+z8p+l6ysrFBaWorc3FyNbZX3nVAoFCguLtb6nLaA/SxZWlri5s2bWp8rr/9k+BiciLRo2LAh3NzccPbsWVmnSxo0aAB3d3dcvnxZOtVWns6dO0OhUCA5OVl2fywtLdGxY0ccPHhQWnen7Kq3l19+GcCj00Tp6enw8/NTm08hh729PUaMGIGdO3eiTZs22L17tzT3pUuXLgD+d5pFV8OGDYOlpSW2bNnyxFN7ZUdnLC0t4erqiosXL2oNpUlJSQAg+yqzJ7G2tsaQIUMQFxeHl19+GWfPnsXFixcBQJozpi1I5ebmIi8vD127dkXTpk3Vnjtx4kS5l+VXRkX715W2q+xeeuklAMCuXbuqbD9lKvtdat++PQBoXdervLW+GjdurPW7cuXKlSo59VkZ7du3x4MHD7TeqUCfVwXS02FwIirH5MmTUVBQgAkTJqidNitz+fJlXLlyRXr83nvvoaSkBKGhoRq/KB8+fCidprCzs8Pw4cNx+PBhLFmyROvRmqNHj2qsadOzZ0/cv38fq1atQqdOnaSjXK1bt0bLli2xZMkSlJaWyjpNV1hYiL1792rs+8GDB7h37x5MTEykX9ShoaEwNjbGpEmTcO3aNY1t3b17F2lpaU/cZ6NGjbBkyRIUFhZKk5X/qaSkBBs2bEBISIjUNnr0aKhUKkRERKj1948//sC6detgZWWltjxAZf32228aRyhUKpU0XmWn2xo3bgyFQoHr169rbKNZs2YwNzdHamqq2rjduXNHWjfsaZXNQdK2//I8ePAACxcu1HrEpri4GJ988gkAqC0vEBISAiMjI8yePRtXr15Ve83THomq7HfprbfeAgAsWLBA7WcwMzNTOrX4T15eXrhy5YraveeKiooQHh6uc791VXbkds6cOWprPJ07dw4bNmyo9v5Q1eAcJ6Jy/N///R+OHDmCDRs24NChQ3jllVdgb2+PGzdu4Ny5czh69Ci+/fZb6cqjd999F0lJSfjuu+/Qpk0bDBo0CJaWlsjIyMBvv/2G2NhY6Rd8dHQ0zp8/jw8++ABfffUVvL29YWVlhWvXriElJQX//e9/kZWVpXbLh549e2LJkiW4efOmxtV4PXv2xFdffSX9/Un+/vtvBAQEwNXVFV26dIGjoyPu37+PX375BdnZ2Zg+fbq0lo6Hhweio6Px7rvv4rnnnkO/fv3QqlUraeJ2UlISxowZg5iYmCfu95133kF+fj5mzJiBTp06wc/PDx07doS5uTkyMzOxZ88eZGZmqi3q+cEHH+DXX3/FV199hbNnzyIgIAA3b95EXFwcVCoVNm7ciIYNGz5x3+UJCgqChYUFfHx84OTkBJVKhYSEBJw5cwZBQUHS2lsNGjRA586dsX//frz99tto06YN6tWrh5EjR8LR0RGhoaFYtmwZ2rdvj4EDByI/Px87duyAk5OT1ivRKsvb2xvm5uaIiopCfn6+dGRrxowZ5b5GpVJh9uzZmDdvHry9vdG+fXtYWlrixo0b2LlzJzIzM+Hi4qJ2WviFF15AVFQUJk+ejOeffx5DhgyBk5MTsrOzsX//fvTv3x9RUVE6vYfKfpd69OiBt99+G+vWrcMLL7yAV199FYWFhYiLi0PXrl3xyy+/aOxj6tSp2LVrF/r3748RI0bAwsICCQkJaNSoUaXmbVWFt99+G1999RW2b98OT09P9O7dG7dv38bmzZvRq1cv/Pzzz5U+OkwG4NmudkCkHxWt46SNtvVdysTFxYlXXnlFNG7cWJiYmIgWLVqIHj16iGXLlombN2+q1ZaWloovvvhCdO3aVdSvX19YWFiINm3aiJCQEJGRkaFWW1BQID755BPh6ekp6tevL8zNzYWLi4sYMmSI2Lhxo9raUUIIce/ePWFsbCwAiB07dqg99+WXXwoAonHjxmrrxZT55zpORUVFYvHixSIwMFC0bNlSmJqaCltbW+Hv7y82b96s9XM4duyYeOONN4S9vb0wMTERNjY2olOnTmLGjBni7Nmz5X622pw/f15MnDhRuLu7iwYNGkif65AhQ8QPP/ygtl6TEELcv39fzJkzR7Rt21aYmpqKRo0aib59+4oDBw5obLu8dXmE0L4WUnR0tBg0aJBwcnISZmZmwtraWnTp0kV8/vnnGmNw/vx50a9fP9GoUSOhUCjU9lNUVCQWLlwo2rRpI5RKpXB0dBTh4eHi3r17Wr9fZWsC/XOtpIrew6+//io6d+4szM3NBQCNtbn+qaSkRMTHx4uwsDDh6ekpbG1thbGxsbC0tBReXl5i/vz54u7du1pfm5iYKAYMGCCaNGkiTE1NRcuWLcXQoUPFoUOHnthPISped6oy36Xi4mIRGRkpXF1dhampqXB1dRWLFi0SFy9eLHddrbi4OPHCCy8IU1NTYWdnJyZNmlTuOOiyjlNlxuz+/fvi/fffF/b29kKpVAp3d3exZs0a8cMPPwgA4j//+Y/GtsiwKYTgbbGJiIiqU9k9M+Pj49G3b199d4cqgcGJiIjoGcnKytI4RXjmzBl07doVRkZGyMzMVDslT4aPc5yIiIiekXfffRdXrlzBSy+9hMaNG+PSpUv4+eefoVKpEBsby9BUA/GIExER0TPyzTffICYmBmfPnkVeXp50kcH777+P3r1767t7pAMGJyIiIiKZeB0kERERkUwMTkREREQycXJ4FSotLcVff/2Fhg0bQqFQ6Ls7REREJIMQAvfu3YO9vf0TFyVlcKpCf/31V7l3wiYiIiLDdu3aNbRs2bLCGganKlR224fLly9L95Ui/VCpVNi1axcCAwNhYmKi7+7UeRwPw8GxMBwcC8ORn58PBwcHWbdvYnCqQmWn5xo2bAhLS0s996ZuU6lUsLCwgKWlJf9BMgAcD8PBsTAcHAvDI2eaDSeHExEREcnE4EREREQkE4MTERERkUwMTkREREQyMTgRERERycTgRERERCQTgxMRERGRTAxORERERDIxOBERERHJxOBEREREJJNBBqfo6Gi4uLjAzMwMnp6eOHDgQIX1SUlJ8PT0hJmZGVxdXRETE6P2/OnTpzF06FA4OztDoVAgKiqqSvZLREREdYvBBae4uDhMmTIFs2bNQlpaGnx9fdG3b19kZGRorb98+TL69esHX19fpKWlYebMmZg8eTK2bNki1RQUFMDV1RUff/wx7OzsqmS/REREVPcYXHBavnw5xo0bh/Hjx8PNzQ1RUVFwcHDA6tWrtdbHxMTA0dERUVFRcHNzw/jx4zF27FgsXbpUquncuTOWLFmCN954A0qlskr2S0RERHWPsb478LiioiKkpKRgxowZau2BgYE4fPiw1tckJycjMDBQra13796IjY2FSqWSdcdpXfZbkT1nc2DZqKjSr6OqU1xcgpO3FDA6fQPGxkb67k6dV1vGw8WmAZ6za6jvbhCRHhlUcMrNzUVJSQlsbW3V2m1tbZGdna31NdnZ2Vrri4uLkZubi+bNmz+T/QJAYWEhCgsLpcf5+fkAgH9t+QP1lBZP3C89a0b48sJJfXeCJDV/POopgAPT/NGsofYj1zWBSqVS+y/pD8fCcFRmDAwqOJVRKBRqj4UQGm1PqtfWXtX7jYyMxPz58zXanRoIGJuJSu2biAxbxn2gRCjw0849aFlf3715egkJCfruAv1/HAv9KygokF1rUMHJxsYGRkZGGkd5cnJyNI4GlbGzs9Nab2xsDGtr62e2XwCIiIhAeHi49Dg/Px8ODg74/j0/2fumZ0OlUiEhIQG9evWSdbqWnq3aMB4+S5JwI78Q3bv74Hl7S313R2e1YSxqC46F4Sg7YySHQQUnU1NTeHp6IiEhAa+++qrUnpCQgMGDB2t9jbe3N37++We1tl27dsHLy0v2F1GX/QKAUqnUOtncxMSEPwQGgmNhWGryeCjw6OizsbFxjX0Pj6vJY1HbcCz0rzKfv0EFJwAIDw9HcHAwvLy84O3tjTVr1iAjIwMhISEAHh3lyczMxMaNGwEAISEhWLlyJcLDwzFhwgQkJycjNjYWmzZtkrZZVFSEM2fOSH/PzMxEeno6GjRogNatW8vaLxEREZHBBaegoCDcunULCxYsQFZWFjw8PBAfHw8nJycAQFZWltraSi4uLoiPj8fUqVOxatUq2NvbY8WKFRg6dKhU89dff6Fjx47S46VLl2Lp0qXw9/fHvn37ZO2XiIiIyOCCEwCEhoYiNDRU63Pr16/XaPP390dqamq523N2dpYmjOu6XyIiIiKDWwCTiIiIyFAxOBERyVDJ1U2IqJZicCIiIiKSicGJiIiISCYGJyIiIiKZGJyIiCpBxgW6RFSLMTgRERERycTgREQkAy+qIyKAwYmIiIhINgYnIiIiIpkYnIiIiIhkYnAiIqoEAV5WR1SXMTgRERERycTgREQkg4I3qyMiMDgRERERycbgRERERCQTgxMRERGRTAxORESVwHvVEdVtDE5EREREMjE4EREREcnE4EREREQkE4MTERERkUwMTkREREQyMTgREVUCL6ojqtsYnIiIiIhkYnAiIpKBt6ojIoDBiYiIiEg2BiciIiIimRiciIiIiGRicCIiqgTBm9UR1WkMTkREREQyMTgREcnAq+qICGBwIiIiIpKNwYmIiIhIJgYnIiIiIpkYnIiIKoHX1BHVbQxORERERDIxOBERyaAAL6sjIgYnIiIiItkYnIiIiIhkYnAiIiIikonBiYioEnirOqK6jcGJiIiISCYGJyIiIiKZGJyIiGTgTX6JCGBwIiIiIpKNwYmIiIhIJgYnIiIiIpkYnIiIKoXrERDVZQxORERERDIxOBERycCL6ogIYHAiIiIiko3BiYiIiEgmBiciIiIimRiciIgqgTf5JarbGJyIiIiIZGJwIiKSQcGb1RERGJyIiIiIZGNwIiIiIpKJwYmIiIhIJoMMTtHR0XBxcYGZmRk8PT1x4MCBCuuTkpLg6ekJMzMzuLq6IiYmRqNmy5YtcHd3h1KphLu7O3788Ue154uLizF79my4uLjA3Nwcrq6uWLBgAUpLS6v0vRFRzcaL6ojqNoMLTnFxcZgyZQpmzZqFtLQ0+Pr6om/fvsjIyNBaf/nyZfTr1w++vr5IS0vDzJkzMXnyZGzZskWqSU5ORlBQEIKDg3Hy5EkEBwdj+PDhOHr0qFSzePFixMTEYOXKlTh79iw++eQTLFmyBJ999tkzf89ERERUMxhccFq+fDnGjRuH8ePHw83NDVFRUXBwcMDq1au11sfExMDR0RFRUVFwc3PD+PHjMXbsWCxdulSqiYqKQq9evRAREYF27dohIiICAQEBiIqKkmqSk5MxePBg9O/fH87Ozhg2bBgCAwNx4sSJZ/2WiagG4DV1RAQAxvruwOOKioqQkpKCGTNmqLUHBgbi8OHDWl+TnJyMwMBAtbbevXsjNjYWKpUKJiYmSE5OxtSpUzVqHg9OPj4+iImJwYULF9C2bVucPHkSBw8eVKv5p8LCQhQWFkqP8/PzAQAqlQoqlUrOW6ZnpOzz5zgYhtowHuL/r3xZXFxco99HbRiL2oJjYTgqMwYGFZxyc3NRUlICW1tbtXZbW1tkZ2drfU12drbW+uLiYuTm5qJ58+bl1jy+zenTpyMvLw/t2rWDkZERSkpKsHDhQowYMaLc/kZGRmL+/Pka7YmJibCwsHji+6VnLyEhQd9doMfU5PF48MAIgALJycnIOa3v3jy9mjwWtQ3HQv8KCgpk1xpUcCrzz4XmhBAVLj6nrf6f7U/aZlxcHL7++mt8++23eP7555Geno4pU6bA3t4eo0eP1rrfiIgIhIeHS4/z8/Ph4OCAnj17wtra+gnvkp4llUqFhIQE9OrVCyYmJvruTp1XG8Yj6sJB4GEBvL294eXUWN/d0VltGIvagmNhOMrOGMlhUMHJxsYGRkZGGkeXcnJyNI4YlbGzs9Nab2xsLIWX8moe3+a0adMwY8YMvPHGGwCAF154AVevXkVkZGS5wUmpVEKpVGq0m5iY8IfAQHAsDEtNHo+y/9EyMjKuse/hcTV5LGobjoX+VebzN6jJ4aampvD09NQ4bJmQkIBu3bppfY23t7dG/a5du+Dl5SV9EOXVPL7NgoIC1Kun/nEYGRlxOQIiIiKSGNQRJwAIDw9HcHAwvLy84O3tjTVr1iAjIwMhISEAHp0ey8zMxMaNGwEAISEhWLlyJcLDwzFhwgQkJycjNjYWmzZtkrYZFhYGPz8/LF68GIMHD8a2bduwe/duHDx4UKoZOHAgFi5cCEdHRzz//PNIS0vD8uXLMXbs2Or9AIjIMPGyOiKCAQanoKAg3Lp1CwsWLEBWVhY8PDwQHx8PJycnAEBWVpbamk4uLi6Ij4/H1KlTsWrVKtjb22PFihUYOnSoVNOtWzds3rwZs2fPxpw5c9CqVSvExcWhS5cuUs1nn32GOXPmIDQ0FDk5ObC3t8f//d//4cMPP6y+N09EREQGzeCCEwCEhoYiNDRU63Pr16/XaPP390dqamqF2xw2bBiGDRtW7vMNGzZEVFRUhcsPEBERUd1mUHOciIiIiAwZgxMRUSWULXdCRHUTgxMRERGRTAxOREQy8KI6IgIYnIiIiIhkY3AiIiIikonBiYiIiEgmBiciokrgNXVEdRuDExEREZFMDE5ERDIoFLyujogYnIiIiIhkY3AiIiIikonBiYiIiEgmBiciokrgreqI6jYGJyIiIiKZGJyIiGTgNXVEBDA4EREREcnG4EREREQkE4MTERERkUwMTkRElSB4tzqiOo3BiYiIiEgmBiciIhl4qzoiAhiciIiIiGRjcCIiIiKSqUqC0+3bt3Ht2rWq2BQRERGRwdI5OOXl5SEsLAy2trZo2rQpXFxcpOeOHj2Kfv36ISUlpUo6SURkMHhRHVGdplNwun37Nrp06YLPPvsMDg4OcHNzg3jszpcvvvgiDh06hG+++abKOkpERESkbzoFp3nz5uHChQvYtGkTTpw4gddff13teXNzc/j7+2Pv3r1V0kkiIn1T8G51RAQdg9P27dsxYMAABAUFlVvj5OSE69ev69wxIiIiIkOjU3DKysqCu7t7hTVmZmZ48OCBTp0iIiIiMkQ6BSdra+snXkV37tw5NG/eXKdOERERERkinYKTn58ftm/fjszMTK3PnzlzBjt37sQrr7zyVJ0jIjI0vKiOqG7TKTjNmjULxcXF6N69O7799lvk5uYCAM6ePYvY2Fi8/PLLUCqVmDZtWpV2loiIiEifjHV50QsvvIC4uDi89dZbCA4OBgAIIeDh4QEhBBo2bIjvvvsObdq0qdLOEhHpC+9VR0SAjsEJAAYNGoQ///wTGzZswNGjR3H79m1YWlqiS5cuePvtt2FjY1OV/SQiIiLSO52DEwA0adIEU6dOraq+EBERERk0neY4jR07Ftu3b6+wJj4+HmPHjtWpU0RERESGSKfgtH79eqSnp1dYc+rUKWzYsEGXzRMRGSzBy+qI6jSdb/L7JA8fPoSx8VOdCSQiIiIyKDonG0U5l5gIIXD9+nXEx8fD3t5e544RERERGRrZR5zq1asHIyMjGBkZAXh0o9+yx4//MTY2hrOzM44fP4433njjmXWciIiIqLrJPuLk5+cnHWXav38/HB0d4ezsrFFnZGSEJk2a4OWXX8aECROqrKNERERE+iY7OO3bt0/6e7169fD222/jww8/fBZ9IiIiIjJIOs1xKi0trep+EBHVCIJ3qyOq057ZVXVEREREtY3OV9WVlJTgu+++w+7du/HXX3+hsLBQo0ahUGDPnj1P1UEiIkNQ3pXERFS36BScHjx4gMDAQBw5cgRCCCgUCojHVoUre8x/aIiIiKg20elU3UcffYTk5GTMnz8fubm5EEJg3rx5yMrKQlxcHFxcXDBs2DCtR6GIiIiIaiqdgtPWrVvRtWtXzJ49G02aNJHabW1t8frrr2Pfvn3Ys2cPlixZUmUdJSIiItI3nYJTRkYGunbt+r+N1KundnSpZcuW6N+/P+9VR0S1Du9VR1S36RSc6tevj3r1/vdSKysrZGVlqdXY2dkhIyPj6XpHREREZEB0Ck5OTk5qocjDwwN79+6VjjoJIbBnzx40b968anpJRKRnvNSFiAAdg1NAQAASExNRXFwMABg9ejQyMjLg7e2NadOmwcfHB+np6Rg6dGiVdpaIiIhIn3RajmDChAmwtrbGzZs30bx5c4wdOxZpaWmIjo5Geno6AGDo0KGYN29eFXaViIiISL90Ck5t2rTB9OnT1do+++wzfPjhh/jzzz/h5OQEOzu7KukgERERkaHQeeVwbZo2bYqmTZtKjy9evIjWrVtX5S6IiPSKF9UR1W3P5F51V69exdixY/H8888/i80TERER6UWljzglJSUhJSUFxsbG6N69Ozw9PaXnsrKyMH/+fKxbtw4qlQotWrSo0s4SEekL7yBFREAlglNRURGGDBmC3377Ta190qRJiIqKwpdffomwsDA8ePAA9vb2mDFjBt55550q7zARERGRvsg+Vffpp59i586daN68Od59912EhITA1tYWn332Gd5//32MHz8epqamiIqKwqVLlzBx4kSYmprq1Kno6Gi4uLjAzMwMnp6eOHDgQIX1SUlJ8PT0hJmZGVxdXRETE6NRs2XLFri7u0OpVMLd3R0//vijRk1mZibefPNNWFtbw8LCAh06dEBKSopO74GIiIhqH9nBKS4uDjY2Njh16hRWrlyJVatW4eTJk2jcuDGioqLQuXNnnD9/HpMnT4ZSqdS5Q3FxcZgyZQpmzZqFtLQ0+Pr6om/fvuWuQn758mX069cPvr6+SEtLw8yZMzF58mRs2bJFqklOTkZQUBCCg4Nx8uRJBAcHY/jw4Th69KhUc+fOHXTv3h0mJibYsWMHzpw5g2XLlqFRo0Y6vxciIiKqXWQHpwsXLmDIkCFo3Lix1Na0aVO8+uqrAB4dJbKxsXnqDi1fvhzjxo3D+PHj4ebmhqioKDg4OGD16tVa62NiYuDo6IioqCi4ublh/PjxGDt2LJYuXSrVREVFoVevXoiIiEC7du0QERGBgIAAREVFSTWLFy+Gg4MD1q1bh5deegnOzs4ICAhAq1atnvo9EVHtIXizOqI6TXZwun//Puzt7TXayyaAt2/f/qk7U1RUhJSUFAQGBqq1BwYG4vDhw1pfk5ycrFHfu3dvnDhxAiqVqsKax7e5fft2eHl54fXXX0ezZs3QsWNHrF279qnfExEREdUelbqq7vEb+5ZR/P9LTYyNn35JqNzcXJSUlMDW1lat3dbWFtnZ2Vpfk52drbW+uLgYubm5aN68ebk1j2/zzz//xOrVqxEeHo6ZM2fi2LFj0mnHt956S+u+CwsLpfvzAUB+fj4AQKVSSaGN9KPs8+c4GIbaMR6PjjSVlJTU6PdRO8aiduBYGI7KjEGl0s7169dx7NgxjTYAOH78uNZD2C+99FJldgHgf2GsjBBCo+1J9f9sf9I2S0tL4eXlhUWLFgEAOnbsiNOnT2P16tXlBqfIyEjMnz9foz0xMREWFhbl9peqT0JCgr67QI+pyeORl2cEQIHjx47j/n9r/um6mjwWtQ3HQv8KCgpk11YqOMXGxiI2NlajXQiBrl27an1NSUmJ7O3b2NjAyMhI4+hSTk6OxhGjMnZ2dlrrjY2NYW1tXWHN49ts3rw53N3d1Wrc3NzUJpn/U0REBMLDw6XH+fn5cHBwQM+ePaV9k36oVCokJCSgV69eMDEx0Xd36rzaMB5rribj+oN76PxSZ/i1efr5nPpSG8aituBYGI6yM0ZyyA5Oo0eP1qkzlWFqagpPT08kJCRIk86BR2l88ODBWl/j7e2Nn3/+Wa1t165d8PLykr6I3t7eSEhIwNSpU9VqunXrJj3u3r07zp8/r7adCxcuwMnJqdz+KpVKrVcQmpiY8IfAQHAsDEtNHo+yI9RGRkY19j08riaPRW3DsdC/ynz+soPTunXrdOpMZYWHhyM4OBheXl7w9vbGmjVrkJGRgZCQEACPjvJkZmZi48aNAICQkBCsXLkS4eHhmDBhApKTkxEbG4tNmzZJ2wwLC4Ofnx8WL16MwYMHY9u2bdi9ezcOHjwo1UydOhXdunXDokWLMHz4cBw7dgxr1qzBmjVrquV9E1HNUPNP0hHR06jSm/xWhaCgINy6dQsLFixAVlYWPDw8EB8fLx35ycrKUlvTycXFBfHx8Zg6dSpWrVoFe3t7rFixAkOHDpVqunXrhs2bN2P27NmYM2cOWrVqhbi4OHTp0kWq6dy5M3788UdERERgwYIFcHFxQVRUFEaNGlV9b56IiIgMmsEFJwAIDQ1FaGio1ufWr1+v0ebv74/U1NQKtzls2DAMGzaswpoBAwZgwIABsvtJRHWHArxZHRFVYh0nIiIiorqOwYmIiIhIJgYnIiIiIpkYnIiIKoOX1RHVaQxORERERDIxOBERyVDBXZ+IqA7RaTmCsWPHPrGmXr16sLS0xHPPPYcBAwagRYsWuuyKiIiIyGDoFJzWr18v3X5A2419FQqFWvukSZPw4YcfYvbs2Tp2k4iIiEj/dDpVd+nSJQwYMAC2traIjIxEUlISzp07h6SkJCxatAi2trYYNGgQjh49ijVr1sDe3h5z585FXFxcVfefiIiIqNrodMQpLi4Ox44dw8mTJ9GsWTOpvW3btvD19cWYMWPQoUMHJCYm4oMPPkDfvn3h7u6O6OhoBAUFVVnniYiqm+BldUR1mk5HnGJjY/H666+rhabH2dnZ4fXXX8fatWsBAC1atMCAAQNw8uRJ3XtKREREpGc6Bafr169DqVRWWGNmZobr169Ljx0dHfHw4UNddkdEpHe8qI6IAB2DU4sWLbBt2zYUFhZqfb6wsBDbtm1Tu5IuJycHjRs31q2XRERERAZAp+A0btw4XLx4Ef7+/vj1119x+/ZtAMDt27fxyy+/wM/PD5cuXVJbtuDAgQNo37591fSaiIiISA90mhz+wQcf4OzZs/j6668xaNAgAI/WbSotLQXwaImCUaNGYcaMGQCAGzduoH///ujTp08VdZuIiIio+ukUnIyMjLBx40aMHj0aX3/9NX7//Xfk5+fD0tIS7du3x6hRoxAQECDV29ra4j//+U+VdZqISF+0LF1HRHWITsGpTEBAgFpAIiIiIqrNeK86IiI5eLM6IsJTHnHKzs5GSkoK7t69i5KSEq01b7311tPsgoiIiMhg6BScHj58iAkTJmDTpk1a71UHPJogrlAoGJyIiIio1tApOE2fPh3ffPMN2rZtixEjRqBly5YwNn6qg1dEREREBk+ntPP999/D3d0dKSkpT1xBnIioNuFVdUR1m06Tw+/evYs+ffowNBEREVGdolNwcnNzw40bN6q6L0REBovX1BERoGNwmj59OrZt24aLFy9WdX+IiIiIDJZOc5zs7OzQp08fvPTSS5gyZQo6duwIKysrrbV+fn5P1UEiIiIiQ6FTcOrRowcUCgWEEJg3bx4UFSwMV976TkREREQ1jU7B6cMPP6wwLBER1VYpGXdQ/P9vaF4TFReX4OQtBYxO34CxsZG+u1OnFReX4E6hvntBlaVTcJo3b14Vd4OIyLAZ13v0P4ur913Sc0+qghG+vHBS350gAI1MjTDqVX33giqDq1YSEcnwjp8r6h28jNLSmr2QkxACt+/cQZPGjXnmQI9UJaU4eT0PeUX67glVFoMTEZEMgc/bIfB5O31346mpVCrEx8ejX7+XYGJiou/u1Fk37xWi88Ld+u4G6UBWcHJ1dYVCocDu3bvh4uICV1dXWRtXKBS4dKk2HNYmIiIikhmcSktL1Q7p/vNxecq7ATARERFRTSQrOF25cqXCx0RERFR5gmvS1zg6rRxOREREVBcxOBEREVUzXtBYc+l8VV1RURF++uknHD9+HHfv3tW6QrhCoUBsbOxTdZCIiIjIUOgUnK5evYpevXrh0qVLFU4AZ3AiIiKi2kSn4DR16lRcvHgRwcHBGDt2LFq2bAljYy4JRURERLWbTmln7969CAgIwIYNG6q6P0REREQGS6fJ4aWlpejYsWNV94WIiKhO4Nzwmkun4OTt7Y2zZ89WdV+IiIiIDJpOwenjjz9GYmIifvjhh6ruDxEREZHB0mmO088//4yePXsiKCgI/v7+6NixI6ysrDTqFAoF5syZ89SdJCIiqq14e7KaRafgNG/ePOnv+/btw759+7TWMTgRERFRbaJTcEpMTKzqfhAREdUZCi4dXmPpFJwUCgUsLS3RoUOHKu4OERERkeHSaXJ4z549sXbt2qruCxERUZ3DKU41i07BqVmzZjA1Na3qvhAREREZNJ2CU+/evZGUlMQrAYiIiKhO0Sk4LVq0CLdu3cI777yD27dvV3WfiIiIajVODa+5dJoc/uabb6JRo0b48ssv8fXXX8PFxQW2trYaVwkoFArs2bOnSjpKREREpG86BafH120qLCzEuXPncO7cOY06Xm5JRERUMU56qVl0Ck6lpaVV3Q8iIiIig6fTHCciIiLSHU/I1FwMTkREREQy6XSqrsz169eRmJiIv/76C4WFhRrP8151REREVJvoHJymTZuGTz/9FCUlJVKbEEKaEF72dwYnIiKi8nFNxJpFp1N1a9euxbJly9CzZ0/88MMPEEJg9OjR2LRpE0JCQmBsbIxhw4Zh7969Vd1fIiIiIr3R6YjTmjVr4OzsjB07dqBevUfZy9nZGUFBQQgKCsLw4cPRq1cvDB8+vEo7S0REVBsouARmjaXTEadz586hT58+UmgCgOLiYunv/v7+6N+/P5YuXapTp6Kjo+Hi4gIzMzN4enriwIEDFdYnJSXB09MTZmZmcHV1RUxMjEbNli1b4O7uDqVSCXd3d/z444/lbi8yMhIKhQJTpkzRqf9ERERUO+l8VV2jRo2kv9evXx+3bt1Se/65557D6dOnK73duLg4TJkyBbNmzUJaWhp8fX3Rt29fZGRkaK2/fPky+vXrB19fX6SlpWHmzJmYPHkytmzZItUkJycjKCgIwcHBOHnyJIKDgzF8+HAcPXpUY3vHjx/HmjVr8OKLL1a670RERFS76RScWrRogevXr0uPW7VqpRFC/vjjD9SvX7/S216+fDnGjRuH8ePHw83NDVFRUXBwcMDq1au11sfExMDR0RFRUVFwc3PD+PHjMXbsWLWjXVFRUejVqxciIiLQrl07REREICAgAFFRUWrbun//PkaNGoW1a9eicePGle47ERFRZXFqeM2i0xyn7t27q50+Gzx4MD766COEhIRg4MCBOHjwIHbs2IGhQ4dWartFRUVISUnBjBkz1NoDAwNx+PBhra9JTk5GYGCgWlvv3r0RGxsLlUoFExMTJCcnY+rUqRo1/wxO7733Hvr3749XXnkFH3300RP7W1hYqLYMQ35+PgBApVJBpVI98fX07JR9/hwHw8DxMBwcC8OgKv7f569SqaAy4rKK+lSZnwedglNwcDD++usvXL16FU5OTpg2bRp++eUXrFmzBmvXroUQAs7OzliyZEmltpubm4uSkhLY2tqqtdva2iI7O1vra7Kzs7XWFxcXIzc3F82bNy+35vFtbt68GampqTh+/Ljs/kZGRmL+/Pka7YmJibCwsJC9HXp2EhIS9N0FegzHw3BwLPSroBgo+xW8Z/ceMDfpV0FBgexanYJTjx490KNHD+lxgwYNcOTIEWzbtg2XLl2Ck5MTBg4cqNOpOkDz5sCPrw8lt/6f7RVt89q1awgLC8OuXbtgZmYmu58REREIDw+XHufn58PBwQE9e/aEtbW17O1Q1VOpVEhISECvXr1gYmKi7+7UeRwPw8GxMAz5f6sQcTwRABDwSgAszJR67lHdVnbGSI6nWjn8cSYmJhg2bNhTbcPGxgZGRkYaR5dycnI0jhiVsbOz01pvbGwshZfyasq2mZKSgpycHHh6ekrPl5SUYP/+/Vi5ciUKCwthZGSksW+lUgmlUvPLbmJiwn+QDATHwrBwPAwHx0K/jP93ITrHwgBU5vN/6oODZ86cwdatW/HVV1897aZgamoKT09PjUPICQkJ6Natm9bXeHt7a9Tv2rULXl5e0gdRXk3ZNgMCAnDq1Cmkp6dLf7y8vDBq1Cikp6drDU1ERERVgQuH1yw6H3E6fvw4JkyYgFOnTkltwcHBAID9+/ejT58+2Lx5MwYNGlSp7YaHhyM4OBheXl7w9vbGmjVrkJGRgZCQEACPTo9lZmZi48aNAICQkBCsXLkS4eHhmDBhApKTkxEbG4tNmzZJ2wwLC4Ofnx8WL16MwYMHY9u2bdi9ezcOHjwIAGjYsCE8PDzU+lG/fn1YW1trtBMREVHdpdMRp9OnT+Pll1/G5cuXMXXqVPTt21fteV9fX9jY2OD777+v9LaDgoIQFRWFBQsWoEOHDti/fz/i4+Ph5OQEAMjKylJb08nFxQXx8fHYt28fOnTogH//+99YsWKF2hV93bp1w+bNm7Fu3Tq8+OKLWL9+PeLi4tClSxdd3j4REdFTqWDaLhk4nY44zZ07F8CjuUGtW7fG/PnzsWPHDul5hUIBb2/vSl2h9rjQ0FCEhoZqfW79+vUabf7+/khNTa1wm8OGDavUHKx9+/bJriUiIqK6QacjTklJSRg6dChat25dbo2joyOysrJ07hgRERGRodEpON27dw/NmjWrsObhw4coKSnRqVNERER1BeeG1yw6BScHBwf88ccfFdakpKSgVatWOnWKiIiIyBDpFJwGDBiAXbt2Ye/evVqf/+6773DkyBEMGTLkafpGRERUK3FueM2l0+TwmTNn4ocffkDfvn0xevRoaS5TdHQ0kpOTsWnTJjg7O6utqk1ERERU0+kUnJo2bYqkpCQEBwfjiy++kNonTpwIAOjSpQs2bdoEKyurquklERFRbcUVMGsUnRfAdHV1xaFDh5Ceno4jR47g9u3bsLS0RJcuXdC5c+eq7CMRERGRQXjqe9V16NABHTp00Gj/7LPPkJiYiK1btz7tLoiIiGqVim5cT4btqe9VV57U1FRs27btWW2eiIiIqNo9s+BEREREVNswOBEREekRp4bXLAxORERERDIxOBEREVUzTg2vuRiciIiIiGSSvRxBv379KrXhU6dOVbozRERERIZMdnDauXNnpTfOdSqIiIgqxoXDaxbZweny5cvPsh9EREREBk92cHJycnqW/SAiIqozeEKm5uLkcCIiIiKZGJyIiIiIZGJwIiIi0iPBtcNrFAYnIiIiIpkYnIiIiKqZgmuH11gMTkREREQyMTgRERERycTgREREpEdcObxmYXAiIiIikonBiYiIqJpx5fCai8GJiIiISCYGJyIiIiKZGJyIiIj0iHPDaxYGJyIiIiKZGJyIiIiIZGJwIiIiIpKJwYmIiEiPuABmzcLgRERERCQTgxMREVE14wKYNReDExEREZFMDE5EREREMjE4ERER6RVnh9ckDE5EREREMjE4ERERVTMFODu8pmJwIiIiIpKJwYmIiIhIJgYnIiIiPeLK4TULgxMRERGRTAxORERE1Ywrh9dcDE5EREREMjE4EREREcnE4ERERKRHnBteszA4EREREcnE4ERERFTNODe85mJwIiIiIpKJwYmIiIhIJgYnIiIiPeLK4TULgxMRERGRTAxORERE1UzBpcNrLAYnIiIiIpkYnIiIiPRIcAnMGsUgg1N0dDRcXFxgZmYGT09PHDhwoML6pKQkeHp6wszMDK6uroiJidGo2bJlC9zd3aFUKuHu7o4ff/xR7fnIyEh07twZDRs2RLNmzTBkyBCcP3++St8XERER1WwGF5zi4uIwZcoUzJo1C2lpafD19UXfvn2RkZGhtf7y5cvo168ffH19kZaWhpkzZ2Ly5MnYsmWLVJOcnIygoCAEBwfj5MmTCA4OxvDhw3H06FGpJikpCe+99x6OHDmChIQEFBcXIzAwEA8ePHjm75mIiOoWznCquRRCGNaFkF26dEGnTp2wevVqqc3NzQ1DhgxBZGSkRv306dOxfft2nD17VmoLCQnByZMnkZycDAAICgpCfn4+duzYIdX06dMHjRs3xqZNm7T24+bNm2jWrBmSkpLg5+cnq+/5+fmwsrJCbm4urK2tZb2Gng2VSoX4+Hj069cPJiYm+u5OncfxMBwcC8NQWirgOjMeAHAsogeaWdXXc4/qtrLf33l5ebC0tKyw1ria+iRLUVERUlJSMGPGDLX2wMBAHD58WOtrkpOTERgYqNbWu3dvxMbGQqVSwcTEBMnJyZg6dapGTVRUVLl9ycvLAwA0adKk3JrCwkIUFhZKj/Pz8wE8+odJpVKV+zp69so+f46DYeB4GA6OhWEoLf3fMYtiVTHHQ88q8/kbVHDKzc1FSUkJbG1t1dptbW2RnZ2t9TXZ2dla64uLi5Gbm4vmzZuXW1PeNoUQCA8Ph4+PDzw8PMrtb2RkJObPn6/RnpiYCAsLi3JfR9UnISFB312gx3A8DAfHQr8e5aZHv4IT9+1DAx7806uCggLZtQYVnMr8c30LIUSFa15oq/9ne2W2OXHiRPz+++84ePBghf2MiIhAeHi49Dg/Px8ODg7o2bMnT9XpmUqlQkJCAnr16sXTEQaA42E4OBaGobRUYOqRR+G1Rw9/nqrTs7IzRnIYVHCysbGBkZGRxpGgnJwcjSNGZezs7LTWGxsbS+GlvBpt25w0aRK2b9+O/fv3o2XLlhX2V6lUQqlUarSbmJjwHyQDwbEwLBwPw8Gx0K/HpxcbG3Ms9K0yn79BXVVnamoKT09PjUPICQkJ6Natm9bXeHt7a9Tv2rULXl5e0gdRXs3j2xRCYOLEidi6dSv27t0LFxeXqnhLREREVIsY1BEnAAgPD0dwcDC8vLzg7e2NNWvWICMjAyEhIQAenR7LzMzExo0bATy6gm7lypUIDw/HhAkTkJycjNjYWLWr5cLCwuDn54fFixdj8ODB2LZtG3bv3q12Ku69997Dt99+i23btqFhw4bSESorKyuYm5tX4ydAREREhsrgglNQUBBu3bqFBQsWICsrCx4eHoiPj4eTkxMAICsrS21NJxcXF8THx2Pq1KlYtWoV7O3tsWLFCgwdOlSq6datGzZv3ozZs2djzpw5aNWqFeLi4tClSxeppmz5gx49eqj1Z926dRgzZsyze8NERFSnGdSaQPREBhecACA0NBShoaFan1u/fr1Gm7+/P1JTUyvc5rBhwzBs2LBynzew5ayIiIjIABnUHCciIqK6oKIrxcmwMTgRERERycTgRERERCQTgxMREZE+cY5tjcLgRERERCQTgxMRERGRTAxORERERDIxOBERERHJxOBERESkR5waXrMwOBERERHJxOBERESkB1w8vGZicCIiIiKSicGJiIiISCYGJyIiIj3iwuE1C4MTERERkUwMTkRERHrAueE1E4MTERERkUwMTkRERHrEKU41C4MTERERkUwMTkRERHqg4AqYNRKDExEREZFMDE5EREREMjE4ERER6ZHgCpg1CoMTERERkUwMTkRERHrAqeE1E4MTERERkUwMTkREREQyGeu7A0RERHXZ5weuwNLcRN/dqNMePrgvu5bBiYiISA8sTI2Q/7AYXx3J0HdX6rzSwgLZtQxOREREevCf4S9i464TcHZxRr16nDmjTw8L7uNjmbUMTkRERHrg18YG9/9bin792sHEhKfq9Ck/P192cGLEJSIiIpKJwYmIiIhIJgYnIiIiIpkYnIiIiIhkYnAiIiIikonBiYiIiEgmBiciIiIimRiciIiIiGRicCIiIiKSicGJiIiISCYGJyIiIiKZGJyIiIiIZGJwIiIiIpKJwYmIiIhIJgYnIiIiIpkYnIiIiIhkYnAiIiIikonBiYiIiEgmBiciIiIimRiciIiIiGRicCIiIiKSicGJiIiISCYGJyIiIiKZGJyIiIiIZGJwIiIiIpKJwYmIiIhIJgYnIiIiIpkYnIiIiIhkMsjgFB0dDRcXF5iZmcHT0xMHDhyosD4pKQmenp4wMzODq6srYmJiNGq2bNkCd3d3KJVKuLu748cff3zq/RIREVHdYnDBKS4uDlOmTMGsWbOQlpYGX19f9O3bFxkZGVrrL1++jH79+sHX1xdpaWmYOXMmJk+ejC1btkg1ycnJCAoKQnBwME6ePIng4GAMHz4cR48e1Xm/REREVPcYXHBavnw5xo0bh/Hjx8PNzQ1RUVFwcHDA6tWrtdbHxMTA0dERUVFRcHNzw/jx4zF27FgsXbpUqomKikKvXr0QERGBdu3aISIiAgEBAYiKitJ5v0RERFT3GFRwKioqQkpKCgIDA9XaAwMDcfjwYa2vSU5O1qjv3bs3Tpw4AZVKVWFN2TZ12S8RERHVPcb67sDjcnNzUVJSAltbW7V2W1tbZGdna31Ndna21vri4mLk5uaiefPm5daUbVOX/QJAYWEhCgsLpcd5eXkAgNu3bz/hndKzplKpUFBQgFu3bsHExETf3anzOB6Gg2NhODgWhuPevXsAACHEE2sNKjiVUSgUao+FEBptT6r/Z7ucbVZ2v5GRkZg/f75Ge9u2bct9DRERERmme/fuwcrKqsIagwpONjY2MDIy0jjKk5OTo3E0qIydnZ3WemNjY1hbW1dYU7ZNXfYLABEREQgPD5ce3717F05OTsjIyHjiB0/PVn5+PhwcHHDt2jVYWlrquzt1HsfDcHAsDAfHwnAIIXDv3j3Y29s/sdaggpOpqSk8PT2RkJCAV199VWpPSEjA4MGDtb7G29sbP//8s1rbrl274OXlJR369Pb2RkJCAqZOnapW061bN533CwBKpRJKpVKj3crKij8EBsLS0pJjYUA4HoaDY2E4OBaGQe4BD4MKTgAQHh6O4OBgeHl5wdvbG2vWrEFGRgZCQkIAPDrKk5mZiY0bNwIAQkJCsHLlSoSHh2PChAlITk5GbGwsNm3aJG0zLCwMfn5+WLx4MQYPHoxt27Zh9+7dOHjwoOz9EhERERlccAoKCsKtW7ewYMECZGVlwcPDA/Hx8XBycgIAZGVlqa2t5OLigvj4eEydOhWrVq2Cvb09VqxYgaFDh0o13bp1w+bNmzF79mzMmTMHrVq1QlxcHLp06SJ7v0REREQKIWcKOclSWFiIyMhIREREaD2FR9WHY2FYOB6Gg2NhODgWNRODExEREZFMBrUAJhEREZEhY3AiIiIikonBiYiIiEgmBiciIiIimRicqlB0dDRcXFxgZmYGT09PHDhwQN9dqjEiIyPRuXNnNGzYEM2aNcOQIUNw/vx5tRohBObNmwd7e3uYm5ujR48eOH36tFpNYWEhJk2aBBsbG9SvXx+DBg3C9evX1Wru3LmD4OBgWFlZwcrKCsHBwbh7965aTUZGBgYOHIj69evDxsYGkydPRlFR0TN574YuMjISCoUCU6ZMkdo4FtUrMzMTb775JqytrWFhYYEOHTogJSVFep7jUT2Ki4sxe/ZsuLi4wNzcHK6urliwYAFKS0ulGo5FHSCoSmzevFmYmJiItWvXijNnzoiwsDBRv359cfXqVX13rUbo3bu3WLdunfjjjz9Eenq66N+/v3B0dBT379+Xaj7++GPRsGFDsWXLFnHq1CkRFBQkmjdvLvLz86WakJAQ0aJFC5GQkCBSU1NFz549Rfv27UVxcbFU06dPH+Hh4SEOHz4sDh8+LDw8PMSAAQOk54uLi4WHh4fo2bOnSE1NFQkJCcLe3l5MnDixej4MA3Ls2DHh7OwsXnzxRREWFia1cyyqz+3bt4WTk5MYM2aMOHr0qLh8+bLYvXu3uHjxolTD8ageH330kbC2tha//PKLuHz5svj+++9FgwYNRFRUlFTDsaj9GJyqyEsvvSRCQkLU2tq1aydmzJihpx7VbDk5OQKASEpKEkIIUVpaKuzs7MTHH38s1Tx8+FBYWVmJmJgYIYQQd+/eFSYmJmLz5s1STWZmpqhXr57YuXOnEEKIM2fOCADiyJEjUk1ycrIAIM6dOyeEECI+Pl7Uq1dPZGZmSjWbNm0SSqVS5OXlPbs3bWDu3bsn2rRpIxISEoS/v78UnDgW1Wv69OnCx8en3Oc5HtWnf//+YuzYsWptr732mnjzzTeFEByLuoKn6qpAUVERUlJSEBgYqNYeGBiIw4cP66lXNVteXh4AoEmTJgCAy5cvIzs7W+0zViqV8Pf3lz7jlJQUqFQqtRp7e3t4eHhINcnJybCyslJbNb5r166wsrJSq/Hw8FC72WPv3r1RWFiodnqktnvvvffQv39/vPLKK2rtHIvqtX37dnh5eeH1119Hs2bN0LFjR6xdu1Z6nuNRfXx8fLBnzx5cuHABAHDy5EkcPHgQ/fr1A8CxqCsM7pYrNVFubi5KSkpga2ur1m5ra4vs7Gw99armEkIgPDwcPj4+8PDwAADpc9T2GV+9elWqMTU1RePGjTVqyl6fnZ2NZs2aaeyzWbNmajX/3E/jxo1hampaZ8Zz8+bNSE1NxfHjxzWe41hUrz///BOrV69GeHg4Zs6ciWPHjmHy5MlQKpV46623OB7VaPr06cjLy0O7du1gZGSEkpISLFy4ECNGjADAn426gsGpCikUCrXHQgiNNnqyiRMn4vfff1e7CXMZXT7jf9Zoq9elpra6du0awsLCsGvXLpiZmZVbx7GoHqWlpfDy8sKiRYsAAB07dsTp06exevVqvPXWW1Idx+PZi4uLw9dff41vv/0Wzz//PNLT0zFlyhTY29tj9OjRUh3HonbjqboqYGNjAyMjI42Un5OTo/F/BFSxSZMmYfv27UhMTETLli2ldjs7OwCo8DO2s7NDUVER7ty5U2HNjRs3NPZ78+ZNtZp/7ufOnTtQqVR1YjxTUlKQk5MDT09PGBsbw9jYGElJSVixYgWMjY2lz4BjUT2aN28Od3d3tTY3NzfpZuf82ag+06ZNw4wZM/DGG2/ghRdeQHBwMKZOnYrIyEgAHIu6gsGpCpiamsLT0xMJCQlq7QkJCejWrZueelWzCCEwceJEbN26FXv37oWLi4va8y4uLrCzs1P7jIuKipCUlCR9xp6enjAxMVGrycrKwh9//CHVeHt7Iy8vD8eOHZNqjh49iry8PLWaP/74A1lZWVLNrl27oFQq4enpWfVv3sAEBATg1KlTSE9Pl/54eXlh1KhRSE9Ph6urK8eiGnXv3l1jaY4LFy7AyckJAH82qlNBQQHq1VP/tWlkZCQtR8CxqCOqeTJ6rVW2HEFsbKw4c+aMmDJliqhfv764cuWKvrtWI7z77rvCyspK7Nu3T2RlZUl/CgoKpJqPP/5YWFlZia1bt4pTp06JESNGaL3Mt2XLlmL37t0iNTVVvPzyy1ov833xxRdFcnKySE5OFi+88ILWy3wDAgJEamqq2L17t2jZsmWdvsz38avqhOBYVKdjx44JY2NjsXDhQvHf//5XfPPNN8LCwkJ8/fXXUg3Ho3qMHj1atGjRQlqOYOvWrcLGxkZ88MEHUg3HovZjcKpCq1atEk5OTsLU1FR06tRJupSengyA1j/r1q2TakpLS8XcuXOFnZ2dUCqVws/PT5w6dUptO3///beYOHGiaNKkiTA3NxcDBgwQGRkZajW3bt0So0aNEg0bNhQNGzYUo0aNEnfu3FGruXr1qujfv78wNzcXTZo0ERMnThQPHz58Vm/f4P0zOHEsqtfPP/8sPDw8hFKpFO3atRNr1qxRe57jUT3y8/NFWFiYcHR0FGZmZsLV1VXMmjVLFBYWSjUci9pPIYQQ+jziRURERFRTcI4TERERkUwMTkREREQyMTgRERERycTgRERERCQTgxMRERGRTAxORERERDIxOBERERHJxOBEREREJBODExHVWgUFBVi0aBE6deqEBg0awMzMDC1btoSvry8iIiJw6dIlqdbZ2RnOzs766ywR1QjG+u4AEdGzcO/ePfj4+OD3339H69at8eabb6JRo0a4du0aTp8+jY8//hitWrVCq1at9N1VIqpBGJyIqFaKiorC77//jnHjxmHt2rVQKBRqz1++fBmFhYV66h0R1VQ8VUdEtVJycjIAYOLEiRqhCQBcXFzQrl07XLlyBQqFAlevXsXVq1ehUCikP/PmzVN7zf79+zFw4EDY2NhAqVSiTZs2mD17NgoKCtTq9u3bJ71+//798Pf3R4MGDdCkSROMHDkS169ff2bvm4ieLQYnIqqVmjRpAgC4ePFihXWNGjXC3LlzYWVlBSsrK8ydO1f606NHD6kuJiYGPXr0wOHDhzFgwABMnjwZLVq0wMKFC9GrVy8UFRVpbPvIkSPo1asXrK2tMXnyZLz00kvYtGkTunXrhhs3blTp+yWiaiKIiGqhn376SQAQlpaWYvr06WLPnj3i9u3b5dY7OTkJJycnrc+dPn1aGBsbi44dO4pbt26pPRcZGSkAiKVLl0ptiYmJAoAAIL744gu1+vnz5wsAYuzYsbq/OSLSG4UQQug1uRERPSNLlizBggULcP/+famtVatW6NOnD8LCwtCmTRupveyKuitXrmhsJywsDCtWrMCBAwfg4+Oj9lxpaSns7Ozg6OiIEydOAHh0qq5nz5547rnncPbsWbVThX///TecnJxw//593L17F6amplX4jonoWePkcCKqtaZNm4aQkBDs3LkThw8fxokTJ3D06FGsWrUKsbGxiIuLw6BBg564nSNHjgAAdu7cid27d2s8b2JignPnzmm0d+/eXWN+lbm5OTw9PbFz505cuHABHh4eOr47ItIHBiciqtUaNmyI119/Ha+//joAIC8vDzNnzkR0dDTGjRuHzMzMJx71uX37NgBg4cKFldp3s2bNtLbb2tpKfSGimoWTw4moTrGyssLKlSvh5OSE3NxcnDp16omvsbS0BADk5+dDCFHun3/KycnRur2yieFWVlZP8U6ISB8YnIiozlEoFLCwsFBrMzIyQklJidb6Ll26APjfKTu5Dh06pBGo/v77b6SkpMDc3Bxt27at1PaISP8YnIioVvr8889x/Phxrc9t3boV586dQ6NGjaQ5Rk2aNEFubi4ePnyoUR8aGgpjY2NMmjQJ165d03j+7t27SEtL02g/f/48vvzyS7W2JUuW4ObNmxgxYgQnhhPVQJzjRES10o4dOxASEoLWrVuje/fusLe3x/3795Geno4DBw6gXr16iI6OhlKpBAC8/PLLOHHiBAYOHAhfX1+YmprCx8cHPj4+8PDwQHR0NN59910899xz6NevH1q1aoX8/Hz8+eefSEpKwpgxYxATE6PWh8DAQISGhuLXX39Fu3btkJqait9++w0ODg5YtGiRPj4WInpKXI6AiGql8+fPY/v27UhISMDFixeRlZUFAGjRogV8fHwwadIkeHp6SvX3799HeHg4fvnlF9y4cQOlpaWYO3eu2urhx48fx/Lly7F//37cvHkTVlZWcHR0RGBgIEaPHo127doB+N9yBHPnzsXLL7+M2bNnIyUlBaampujTpw8++eQTODg4VOvnQURVg8GJiKiKPR6c/nnbFiKq2TjHiYiIiEgmBiciIiIimRiciIiIiGTiHCciIiIimXjEiYiIiEgmBiciIiIimRiciIiIiGRicCIiIiKSicGJiIiISCYGJyIiIiKZGJyIiIiIZGJwIiIiIpKJwYmIiIhIpv8HdbIvqflshwgAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# extra code – this cell plots piecewise constant scheduling\n",
"\n",
"boundaries = [50_000, 80_000]\n",
"values = [0.01, 0.005, 0.001]\n",
"\n",
"steps = np.arange(100_000)\n",
"\n",
"lrs = np.full(len(steps), values[0])\n",
"for boundary, value in zip(boundaries, values[1:]):\n",
" lrs[boundary:] = value\n",
"\n",
"plt.plot(steps, lrs, \"-\")\n",
"plt.axis([0, steps.max(), 0, 0.0105])\n",
"plt.xlabel(\"Step\")\n",
"plt.ylabel(\"Learning Rate\")\n",
"plt.title(\"Piecewise Constant Scheduling\", fontsize=14)\n",
"plt.grid(True)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Just like we did with exponential scheduling, we could also implement piecewise constant scheduling manually:"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [],
"source": [
"def piecewise_constant_fn(epoch):\n",
" if epoch < 5:\n",
" return 0.01\n",
" elif epoch < 15:\n",
" return 0.005\n",
" else:\n",
" return 0.001"
]
},
{
"cell_type": "code",
"execution_count": 84,
"metadata": {},
"outputs": [],
"source": [
"# extra code – this cell demonstrates a more general way to define\n",
"# piecewise constant scheduling.\n",
"\n",
"def piecewise_constant(boundaries, values):\n",
" boundaries = np.array([0] + boundaries)\n",
" values = np.array(values)\n",
" def piecewise_constant_fn(epoch):\n",
" return values[(boundaries > epoch).argmax() - 1]\n",
" return piecewise_constant_fn\n",
"\n",
"piecewise_constant_fn = piecewise_constant([5, 15], [0.01, 0.005, 0.001])"
]
},
{
"cell_type": "code",
"execution_count": 85,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/25\n",
"1719/1719 [==============================] - 5s 2ms/step - loss: 0.5433 - accuracy: 0.8087 - val_loss: 0.4586 - val_accuracy: 0.8288 - lr: 0.0100\n",
"Epoch 2/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4487 - accuracy: 0.8439 - val_loss: 0.4608 - val_accuracy: 0.8350 - lr: 0.0100\n",
"Epoch 3/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4263 - accuracy: 0.8502 - val_loss: 0.4234 - val_accuracy: 0.8568 - lr: 0.0100\n",
"Epoch 4/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4241 - accuracy: 0.8537 - val_loss: 0.4359 - val_accuracy: 0.8490 - lr: 0.0100\n",
"Epoch 5/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.4080 - accuracy: 0.8584 - val_loss: 0.4165 - val_accuracy: 0.8560 - lr: 0.0100\n",
"Epoch 6/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3544 - accuracy: 0.8738 - val_loss: 0.3830 - val_accuracy: 0.8662 - lr: 0.0050\n",
"Epoch 7/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3464 - accuracy: 0.8761 - val_loss: 0.4026 - val_accuracy: 0.8652 - lr: 0.0050\n",
"Epoch 8/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3426 - accuracy: 0.8772 - val_loss: 0.4212 - val_accuracy: 0.8544 - lr: 0.0050\n",
"Epoch 9/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3417 - accuracy: 0.8793 - val_loss: 0.4116 - val_accuracy: 0.8612 - lr: 0.0050\n",
"Epoch 10/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3339 - accuracy: 0.8804 - val_loss: 0.4090 - val_accuracy: 0.8618 - lr: 0.0050\n",
"Epoch 11/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.3309 - accuracy: 0.8819 - val_loss: 0.4033 - val_accuracy: 0.8746 - lr: 0.0050\n",
"Epoch 12/25\n",
"1719/1719 [==============================] - 5s 3ms/step - loss: 0.3270 - accuracy: 0.8826 - val_loss: 0.4518 - val_accuracy: 0.8630 - lr: 0.0050\n",
"Epoch 13/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.3270 - accuracy: 0.8837 - val_loss: 0.3714 - val_accuracy: 0.8674 - lr: 0.0050\n",
"Epoch 14/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3247 - accuracy: 0.8844 - val_loss: 0.4026 - val_accuracy: 0.8652 - lr: 0.0050\n",
"Epoch 15/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.3204 - accuracy: 0.8852 - val_loss: 0.3993 - val_accuracy: 0.8724 - lr: 0.0050\n",
"Epoch 16/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2859 - accuracy: 0.8963 - val_loss: 0.3930 - val_accuracy: 0.8736 - lr: 0.0010\n",
"Epoch 17/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2781 - accuracy: 0.8978 - val_loss: 0.4021 - val_accuracy: 0.8714 - lr: 0.0010\n",
"Epoch 18/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2743 - accuracy: 0.8984 - val_loss: 0.3955 - val_accuracy: 0.8754 - lr: 0.0010\n",
"Epoch 19/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2704 - accuracy: 0.8999 - val_loss: 0.4015 - val_accuracy: 0.8756 - lr: 0.0010\n",
"Epoch 20/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2683 - accuracy: 0.9015 - val_loss: 0.4161 - val_accuracy: 0.8756 - lr: 0.0010\n",
"Epoch 21/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2655 - accuracy: 0.9020 - val_loss: 0.4207 - val_accuracy: 0.8740 - lr: 0.0010\n",
"Epoch 22/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.2646 - accuracy: 0.9020 - val_loss: 0.4497 - val_accuracy: 0.8746 - lr: 0.0010\n",
"Epoch 23/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2626 - accuracy: 0.9032 - val_loss: 0.4429 - val_accuracy: 0.8762 - lr: 0.0010\n",
"Epoch 24/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.2608 - accuracy: 0.9038 - val_loss: 0.4566 - val_accuracy: 0.8748 - lr: 0.0010\n",
"Epoch 25/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.2587 - accuracy: 0.9038 - val_loss: 0.4726 - val_accuracy: 0.8770 - lr: 0.0010\n"
]
}
],
"source": [
"# extra code – use a tf.keras.callbacks.LearningRateScheduler like earlier\n",
"\n",
"n_epochs = 25\n",
"\n",
"lr_scheduler = tf.keras.callbacks.LearningRateScheduler(piecewise_constant_fn)\n",
"\n",
"model = build_model()\n",
"optimizer = tf.keras.optimizers.Nadam(learning_rate=lr0)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
"history = model.fit(X_train, y_train, epochs=n_epochs,\n",
" validation_data=(X_valid, y_valid),\n",
" callbacks=[lr_scheduler])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We've looked at `InverseTimeDecay`, `ExponentialDecay`, and `PiecewiseConstantDecay`. A few more schedulers are available in `tf.keras.optimizers.schedules`, here is the full list:"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"• CosineDecay – A LearningRateSchedule that uses a cosine decay with optional warmup.\n",
"• CosineDecayRestarts – A LearningRateSchedule that uses a cosine decay schedule with restarts.\n",
"• ExponentialDecay – A LearningRateSchedule that uses an exponential decay schedule.\n",
"• InverseTimeDecay – A LearningRateSchedule that uses an inverse time decay schedule.\n",
"• LearningRateSchedule – The learning rate schedule base class.\n",
"• PiecewiseConstantDecay – A LearningRateSchedule that uses a piecewise constant decay schedule.\n",
"• PolynomialDecay – A LearningRateSchedule that uses a polynomial decay schedule.\n"
]
}
],
"source": [
"for name in sorted(dir(tf.keras.optimizers.schedules)):\n",
" if name[0] == name[0].lower(): # must start with capital letter\n",
" continue\n",
" scheduler_class = getattr(tf.keras.optimizers.schedules, name)\n",
" print(f\"• {name} – {scheduler_class.__doc__.splitlines()[0]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Performance Scheduling"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {},
"outputs": [],
"source": [
"# extra code – build and compile the model\n",
"\n",
"model = build_model()\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=lr0)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": 88,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.6807 - accuracy: 0.7679 - val_loss: 0.4814 - val_accuracy: 0.8310 - lr: 0.0100\n",
"Epoch 2/25\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4659 - accuracy: 0.8343 - val_loss: 0.4615 - val_accuracy: 0.8306 - lr: 0.0100\n",
"Epoch 3/25\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.4201 - accuracy: 0.8505 - val_loss: 0.4199 - val_accuracy: 0.8490 - lr: 0.0100\n",
"Epoch 4/25\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3957 - accuracy: 0.8590 - val_loss: 0.3845 - val_accuracy: 0.8614 - lr: 0.0100\n",
"Epoch 5/25\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.3754 - accuracy: 0.8658 - val_loss: 0.3742 - val_accuracy: 0.8614 - lr: 0.0100\n",
"Epoch 6/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3588 - accuracy: 0.8709 - val_loss: 0.3853 - val_accuracy: 0.8628 - lr: 0.0100\n",
"Epoch 7/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3469 - accuracy: 0.8740 - val_loss: 0.3627 - val_accuracy: 0.8690 - lr: 0.0100\n",
"Epoch 8/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.3346 - accuracy: 0.8785 - val_loss: 0.3574 - val_accuracy: 0.8680 - lr: 0.0100\n",
"Epoch 9/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.3244 - accuracy: 0.8828 - val_loss: 0.3410 - val_accuracy: 0.8748 - lr: 0.0100\n",
"Epoch 10/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3149 - accuracy: 0.8850 - val_loss: 0.3410 - val_accuracy: 0.8720 - lr: 0.0100\n",
"Epoch 11/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.3074 - accuracy: 0.8879 - val_loss: 0.3629 - val_accuracy: 0.8678 - lr: 0.0100\n",
"Epoch 12/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2990 - accuracy: 0.8920 - val_loss: 0.3379 - val_accuracy: 0.8746 - lr: 0.0100\n",
"Epoch 13/25\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.2929 - accuracy: 0.8938 - val_loss: 0.3223 - val_accuracy: 0.8808 - lr: 0.0100\n",
"Epoch 14/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2867 - accuracy: 0.8947 - val_loss: 0.3405 - val_accuracy: 0.8754 - lr: 0.0100\n",
"Epoch 15/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2807 - accuracy: 0.8972 - val_loss: 0.3480 - val_accuracy: 0.8730 - lr: 0.0100\n",
"Epoch 16/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.2743 - accuracy: 0.8998 - val_loss: 0.3350 - val_accuracy: 0.8766 - lr: 0.0100\n",
"Epoch 17/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2694 - accuracy: 0.9019 - val_loss: 0.3421 - val_accuracy: 0.8764 - lr: 0.0100\n",
"Epoch 18/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2631 - accuracy: 0.9032 - val_loss: 0.3360 - val_accuracy: 0.8772 - lr: 0.0100\n",
"Epoch 19/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2445 - accuracy: 0.9110 - val_loss: 0.3162 - val_accuracy: 0.8874 - lr: 0.0050\n",
"Epoch 20/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2410 - accuracy: 0.9131 - val_loss: 0.3221 - val_accuracy: 0.8812 - lr: 0.0050\n",
"Epoch 21/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2380 - accuracy: 0.9137 - val_loss: 0.3166 - val_accuracy: 0.8828 - lr: 0.0050\n",
"Epoch 22/25\n",
"1719/1719 [==============================] - 4s 2ms/step - loss: 0.2351 - accuracy: 0.9148 - val_loss: 0.3146 - val_accuracy: 0.8854 - lr: 0.0050\n",
"Epoch 23/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2330 - accuracy: 0.9160 - val_loss: 0.3191 - val_accuracy: 0.8836 - lr: 0.0050\n",
"Epoch 24/25\n",
"1719/1719 [==============================] - 3s 1ms/step - loss: 0.2300 - accuracy: 0.9161 - val_loss: 0.3175 - val_accuracy: 0.8878 - lr: 0.0050\n",
"Epoch 25/25\n",
"1719/1719 [==============================] - 3s 2ms/step - loss: 0.2276 - accuracy: 0.9174 - val_loss: 0.3205 - val_accuracy: 0.8868 - lr: 0.0050\n"
]
}
],
"source": [
"lr_scheduler = tf.keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=5)\n",
"history = model.fit(X_train, y_train, epochs=n_epochs,\n",
" validation_data=(X_valid, y_valid),\n",
" callbacks=[lr_scheduler])"
]
},
{
"cell_type": "code",
"execution_count": 89,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAcgAAAEbCAYAAABeCxRrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABM20lEQVR4nO2dd5hUVdKH34JhSCoyYkBRQMS4q4iIcRVUFFlXxIS45lVEBcHVNawJRXfVNaEoiIrKigHFwKcYkTGsCVQyIogJGMGMCAoz1PdH3ZE7Pd3Tt2d6unum632e+/QN55xb99LMr885dapEVXEcx3EcpyINsm2A4ziO4+QiLpCO4ziOEwcXSMdxHMeJgwuk4ziO48TBBdJxHMdx4uAC6TiO4zhxcIF06jQislJETsu2HfUNEflcRC7Kth2Ok01cIJ1aR0QeFBENtlIR+VJERopIy2zblg5EpFvwbK0SXB8aev51IrJURMaJyNaZtjWw57SQPSoiJSIyXkTa17DNlem003GyjQukkyleBVoD7YAzgb8Ad2fToAwzH3v+NkBf4I/A+CzasyqwZ0vgRKATMFFEGmbRJsfJKVwgnUzxm6p+raqLVfVl4HHg0HABETldROaKyK8i8omIXCAiDULXtxOR4uD6fBE5IqZ+u6BH1CXmvIrIsaHjLYMe3HciskpEpotI99D1v4jIB8F9PhOR60WksIbPXxo8/1JVfRO4F9hbRDaqqpKIHC0is0TkNxH5SkQuFxEJXf9cRK4QkXtEZIWILBaRf0SwRwN7SlR1CnAN8AdguwR2/F1EZorILyKyRETuE5GNg2vdgAeA5qFe6dDgWqGI3BjY9YuITBWRw0LtNhSR+4P3vFpEFojIxTH/7g+KyHMx9gwVkdkRntNxqk1Btg1w8g8R2RboCawNnTsLuBYYBHyA/bG+NygzIviD+TTwA7AP0AwYDjRO8d7NgdeB5UAfYAmwW+j6YcA4YDDwBrANMCq4T1rm5ERkC+BooCzYEpXbA3gCuC6waU/gHmAFcGeo6AXA1cB/gMOBO0TkLVV9JwWzVgefjRJcXwcMARYBbYP73wmcDLwdXPsX0CEoXz7c+kBw7kRgMdAL+D8R2VNVZ2A/0pcAxwPfAF2B0cB3wP0p2O846UdVffOtVjfgQaAU+6O5GtBguyBU5kvg5Jh6Q4C5wf6hmJhsE7q+f9DOacFxu+C4S0w7Chwb7J8F/Ay0SmDrG8CVMeeOCmyXBHW6BfdI1ObQwPaV2NBm+fMPT/LexgGvxWlrcej4c+DRmDILgCuqaPc0YGXouA3wDvAVUBhq96Iq2ugJ/AY0iNdmcK4DJqzbxJx/Bri7irZvAF6N+f48F+c9zM72d9u3+r15D9LJFG8A/YGmmEh1AO4AEJFNga2Be0RkZKhOAVA+nLgTsERVvwxdfw/7A5wKuwMzVfXbBNf3ALqKyCWhcw0Cu7cASlK8XzmfYr2nxkBv4Bjgn0nq7AQ8H3PuLeBqEdlIVVcE52bGlFkKbJak7eaBU41gvfEPgaNVdU28wiJyEHBZYFMLoCFQiL2TpQnu0Tlof25oVBjsHbwWansANi/dFnvPjYAvktjvOLWOC6STKVap6sJg/3wRmQJcifUEyuebBmDDdfGQBOfDlItleI4udsgwWTsNsPm4J+Jc+yaCDYlYE3r+OSLSEbgL63klQrCeZjzC59fGuZbMv2AV5pizDlimqr8kNEKkLSbU9wJXYcOfnYFHMZFMRIPAlj3j2Lg6aLsvcDs2fP02Nnx8Hjb8Xc46Kv+7JRoKdpy04QLpZItrgBdEZLSqLhWRJUAHVR2boPxcYCsR2VpVvwrOdaWiEJQLWOvQuU4x7XwInCQirRL0Ij8EdgyJWW0xDJgvIneq6gcJyszFhpHD7I8Nsf5cw/trCs/YBRPCC1S1DCDWQQpYg/Uqw3yECdsWao5A8dgfeE9VR5SfEJEOMWW+ofK/Y+yx46Qd92J1soKqFgNzgCuCU0OBiwPP1R1E5A8icoqIXBZcfxX4GBgrIp1EZB/gNmxus7zN1cC7wCUisouI7AvcHHPrRzAHnWdE5E8i0l5Ejgx5sV4LnCgi1wY27Cgix4rITREe6w+BbeEt7v8xVV0ETMSEMhG3AAcGHpvbi8hfgQuBKLakkwXY34ohwfvqh80Ph/kcaCIiPUSklYg0U9VPsHnUB4N3uK2IdBGRi0Tk6KDeJ0BnETlcRDqKyJXAgTFtvwbsLiJniHkyXwzsV0vP6jjryfYkqG/1fyOOk0Vw/kTM0aNtcNwP68H9inmrvgWcECq/PeaB+hv2R/tIzPHltFCZnYD/YUOIs4A/EXLSCcq0wZaZ/BiU+wjoFrp+KPBmcG0FMA0YWMXzdWO9403stgEJHEqAfYMy+1bR9tHBc6zBnGguJ+QsRBxnGqAYGFFFm6cR41ATp0yFdoHzMW/T1cBkzOtUgXahMiOBb4PzQ4NzjYLnXxQ8w9fYD4M9guuFmLfqD8G/x/3YMO7nMfYMxeZ/f8LWz/4r3jv1zbd0bqKaaIrDcRzHcfIXH2J1HMdxnDi4QDqO4zhOHFwgHcdxHCcOLpCO4ziOE4e8XAfZoEEDbdq0abbNyDnWrVtHgwb+mymMv5P4+HuJT31/L6tWrVJVrb8PGENeCmRhYSG//JIwcEjeUlxcTLdu3bJtRk7h7yQ+/l7iU9/fi4isTl6q/pA3vwQcx3EcJxVcIB3HcRwnDi6QjuM4jhMHF0jHcRzHiYMLpOM4juPEIaMCKUJPEeaLsFCES+NcFxHuCK7PFKFz6NoYEZaLMDumTpEIr4iwIPhsmcyOtr+tZc+tv2bcuGh2jxsH7dpBgwb2WV/rHXTQgXXCzkzU83dSdb1cfy+OkxYyFRUdtCHop6DbghaCzgDdOaZML9AXQAV0b9D3QtcOAO0MOjumzk2glwb7l4LemMyWPUBHcK42a6b68MNaJQ8/rNqsmSqs37xe/a5XF2z0ernJlClTsm1CrQL8olnOsJHJLWPZPETYBxiqymHB8WUm0Pw7VOYeoFiVR4Pj+UA3VUqC43bAc6r8IVTn9zIitA7q71CVLV1E9A2asi2L+LXFFpx/fuKyd9wBP/1U+XyLFni9elqvLtiYz/XatoXPP09cL5vkwTrIVaraPNt2ZIpMCuSxQE9VzgyOTwb2UmVgqMxzwA2qvBUcTwYuUWVacNyOygL5oyobh45/UK08zCpCf6A/wB7IHm9RyP2cyUBGIJLYbns98Qqo16un9eqCjflcT0R57bXXE1fMIitXrmSDDTbIthm1Rvfu3fNKIDPWVQU9DvS+0PHJoHfGlHkedP/Q8WTQPULH7eIMsf4Yc/xDMlv2CMZrfqGpdmlTEjuKUIG2bbXCEE/51rZtldW8Xh2uVxds9Hq5Sd4PsUJPhfkKCxUuraLcngplWp7IHHZQmB7aVigMCa4NVVgSutarShvSuGXkJvZedR/Ql0LHl4FeFlPmHtB+oeP5oK1Dx/EE8vcyoK1B5yezpVwgV1OoHx98bpVfiLoy7+L10levLtjo9XKTvBZIaKjwqcK2CoUKMxR2TlDuNYVJvwtk5etfK7QNjocqXJTwvrWpWxm7EVoAugi0PeuddHaJKfNnKjrpvB9zPZ5A/oeKTjo3JbNlj/D/uE6dkn4pHn7YfrWK2GfU/6B1r966OmJn7dfzd5KsXubeS8uW9l+1TZvcFkfVvBfIfRReCh1fpnBZnHJDFM5TeDCBQB6q8L/Qcf0XSHu32gv0E8yb9fLg3ADQAcG+gN4VXJ8F2iVU91HQEtC1oItB/xac3wQbil0QfBYls2PzggJ79NdfT+GrUf+p7/+5q4O/k/hk8r2MG2f/XefPz9gtq019/760gt8UpoW2/rpeyI5VuC90fLLCCA3//YWtFF4PeomJBHKMwsDQ8VCFzxVmBtdaVqpTS1tGs3moMgmYFHNuVGhfgfMS1O2X4Px3wMGp2LGiYUMoLIRHHoEDDkilquM4GaaoyD6//z67djjwLZSi2iXB5fieWBW5HbgE1bK4HloihcCRYKscAkYCw4K2hgG3AGekZHg1yctIOusAeveGJ56ANWuybY7jOFVQLpDffZddO5ykLAa2Dh23AZbGlOkCPIbI58CxwN2IHBW6fjjwIarLfj+jugzVMlTXAfcCXdNvenzyUiABOPFE+0n68svZtsRxnCrwHmSdYSrQEZH2QU/wBGBihRKq7VFth2o74EngXFSfCZXoB7YO/ndEWoeO+kDFaGq1Sf4K5KGH2v+8Rx7JtiWO41SBC2QdQbUUGAi8BMwDxqM6B5EBiAxIWl+kGdADeCrmyk2IzEJkJtAduCC9hicmo3OQOUVhIRx3HPz3v7ByJdTjxb2OU5dp0QJEXCDrBKqV/ExQHZWg7Gkxx6uATeKUOzlN1qVM/vYgwYZZV62CiROTl3UcJys0bAgbb+wC6WSe/BbI/feHNm18mNVxcpyiIhdIJ/Pkt0A2aAD9+sFLL8G332bbGsdxEuAC6WSD/BZIsGHW0lJ48slsW+I4TgJcIJ1s4AK5226w004+zOo4OYwLpJMNXCBFrBf55pvw5ZfZtsZxnDhssokLpJN5XCDB5iEBHnssu3Y4jhOXoiL44QdYty7bljj5hAskQIcOsNdePszqODlKUZGl3/npp2xb4uQTLpDlnHgizJgBc+Zk2xLHcWLwaDpONnCBLOf4423Zx6OPJi/rOE5GcYF0soELZDlbbAEHH2zDrBqbocVxnGziAulkAxfIMCeeCJ99Bu+9l21LHMcJ4SmvnGzgAhmmTx9o3NiddRwnx/AepJMNXCDDtGgBRxwBjz9u0XUcx8kJWra0TxdIJ5O4QMZy4omwfDm89lq2LXEcJ6CgADbayAXSySwukLH06mX/E32Y1XFyCg8352QaF8hYmjSBY46Bp56C1auzbY3jOAEukE6mcYGMx4knws8/w/PPZ9sSx3ECXCCdTOMCGY/u3WHzzX2Y1XFyCBdIJ9O4QMajYUM44QTrQf74Y7atcRwHF0gn87hAJuLEE2HNGpuLdBwn65QLpAe6ymFEeiIyH5GFiFxaRbk9ESlD5NjQuc8RmYXIdESmhc4XIfIKIguCz5a1+gwhXCATseeeluXDh1kdJyfYZBMoKzP3ACcHEWkI3AUcDuwM9ENk5wTlbgReitNKd1Q7odoldO5SYDKqHYHJwXFGcIFMRHki5ddeg5KSbFvjOHmPR9PJeboCC1FdhOoa4DGgd5xyg4AJwPKI7fYGHgr2HwKOqqGdkXGBrIp+/Ww85/HHs22J4+Q9LpDZpxUUIDIttPUPXd4K+Cp0vDg4tx6RrYA+wKg4zSvwMiIfxLS7OarWS7HPzWr+JNEoyNSN6iQ77QS7727DrEOGZNsax8lrXCCzz7dQGjP8GUbinIudMb4duATVMqRS8f1QXYrIZsAriHyM6hs1s7hmeA8yGSeeCFOnwoIF2bbEcfIaF8icZzGwdei4DbA0pkwX4DFEPgeOBe5G5CgAVJcGn8uBp7EhW4BliLQGCD6jDs3WGBfIZJxwgs1HeiJlx8kqnvIq55kKdESkPSKFwAnAxAolVNuj2g7VdsCTwLmoPoNIc0Q2BECkOXAoMDuoNRE4Ndg/FXi2th+knIwKpAg9RZgvwkKRyp5IIogIdwTXZ4rQOVldEXYT4R0RZonwfyJslFaj27SBAw6AsWPhwAPh66/T2rzjONHwjB45jmopMBDzTp0HjEd1DiIDEBmQpPbmwFuIzADeB55H9cXg2g1AD0QWAD2C44yQsTlIEcpdgHtgXfGpIkxUZW6o2OFAx2DbCxgJ7JWk7n3ARaq8LsIZwD+AK9Nq/Iknwtlnw6JFMGwY3HVXWpt3HCc5jRtD8+YukDmN6iRgUsy5eA45oHpaaH8RsFuCct8BB6fHwNTIZA+yK7BQlUWqJHIB7g2MVUVVeRfYWITWSeruAJRP5L4CHJN2y/fbzz5V4YEHvBfpOFnCo+k4mSSTXqzxXID3ilBmqyR1ZwNHYuPSx1Fxkvh3ROgP9AcoKBCKi4sjG97xttvYUgRRZd3atZQMGMCCeujVunLlypTeSz7g7yQ+2XovhYVd+OSTXykunp28cBbw70v9IpMCGcUFOFGZquqeAdwhwlXYZO6aeDdXZTQwGqBJE9Vu3bpFMBkLEvDyy7/Ht2pQWspWL7/MVqNGwRZbRGujjlBcXEzk95In+DuJT7beyzbbwNq1G+Tsv4l/X+oXmRxijeICnKhMwrqqfKzKoarsATwKfJpWq4cNg3XrKp4rK7PzjuNkFB9idTJJJgVyKtBRhPYixHcBtuNTAm/WvYGfVCmpqq6IRVUQoQFwBfEjNFSfd96xoOVh1qyBt99O620cx0mOC6STSTImkKpUcgFWZY4IA0QodwGeBCwCFgL3AudWVTeo00+ET4CPsV7lA2k1/KOPbHhV1aIkb7SRebV+9FFab+M4TnI8o4eTSTIaak6VSi7Aqut7fKoocF7UusH54cDw9FqagA02gNNPh7vvhltuqXdzkI6T6xQV2QDOqlW25MNxahOPpJMq554La9fC6NHZtsRx8g4PN+dkEhfIVNl+ezjsMBg1yoTScZyMsckm9ukC6WQCF8jqMHCgLf94+ulsW+I4eYX3IJ1qIdKoOtVcIKvD4YdD+/YwYkS2LXGcvMIF0kmKyPmIHBM6vh9Yjch8RHZIpSkXyOrQsKHNRb75JsyYkW1rHCdv8IweTgTOB74BQOQA4HjgRGA6cEsqDblAVpczzoCmTT1wueNkEO9BOhHYCvg82P8L8ASq44GhwN6pNOQCWV2KiuCvf4WHH4Yffsi2NY6TFzRtCk2auEA6VbIC2DTY7wFMDvbXAk1SaSiyQIpwuAjPiTBXxMK+iXCmSHbSkOQE550Hq1dbhg/HcTKCR9NxkvAycG8w97gd8EJwfhfgs1QaiiSQIvwVGA8sANoD5R5BDYGLU7lhvaJTJ9h/fxtmjY3X6jhOreAC6SThPOB/QCvgWFTLvy2dsXjdkYnag7wYOEuVC4DS0Pl3gU6p3LDeMXCgJVJ+8cXkZR3HqTEukE6VqK5AdRCqvVF9MXT+alT/lUpTUQWyI/BOnPMrgY1SuWG9o08faN3al3w4ToZwgXSqRGTnCss5RHog8jAilyHSMJWmogrkUmD7OOcPIN3ppeoahYVw9tnwwguwcGG2rXGceo8LpJOE+4HdARBpAzwLFGFDr9el0lBUgRyNJSXeLzjeWoRTgZuAkancsF7Svz8UFFgQc8dxahUXSCcJOwEfBvvHAe+h2gs4GeiXSkORBFKVm4CngFeA5sAULO/iKFV8IWDr1nDssTBmDPzyS7atcZx6TVGROY+vXp1tS5xKiPQMItYsROTSKsrtiUgZIscGx1sjMgWReYjMQWRwqOxQRJYgMj3YeiWxoiFQnsT3YNZngfoU2DyVx4m8zEOVyzGvoK7YYstNVbkylZvVawYOhJ9+snWRjuPUGuXBAnz5cY5h83t3AYcDOwP9ENk5Qbkbsfy+5ZQCF6K6E6Yv58XUvQ3VTsFWKe1hDLOBcxD5EyaQ5Y46WwHfpvJIUZd5jBFhQ1VWqTJNlfdVWSlCcxHGpHLDesu++9qyjxEjPJur49QiHk0nZ+kKLER1EaprgMeA3nHKDQImAMt/P6NaguqHwf7PwDxM0KrDJcBZQDHwKKqzgvNHAu+n0lDUHuSpQNM455sCp6Ryw3qLCAwaBLNnwxtvZNsax6m3uEBmj1ZQgMi00NY/dHkr4KvQ8WJiRU5kK6APNkUXH5F2mJPNe6GzAxGZicgYRFpWaaTqG1gknVaonhG6cg9wTpV1Y6hSIEUoEmETQICWwXH5tilwBLAslRvWa/r1s/+9vuTDcWoNzwmZPb6FUlS7hLZw5niJUyV2OO124BJUy+LeQGQDrHc5BNUVwdmRQAdszX0JUQKOW/urEfkDIrsg0gTVz1FdnrRuiIIk17/FHlCBufHMAK5O5Yb1mqZN4W9/g1tvhcWLoU2bbFvkOPUO70HmLIvBwpAGtMGWCIbpAjyGCJhPSy9ESlF9JsjZOAEYh+pTv9dQXd8JE7kXeK5KK0QKgH8DA4FCTLh/Q+RO4HJUI2e6TzbE2h2b5BTgWOCg0LY/sI0q10e9WV5wzjkWdu6ee7JtiePUSzzlVc4yFeiISHtECoETgIkVSqi2R7Udqu2AJ4FzA3EUbP3iPFRvrVBHpHXoqA/mhFMVNwEnAQOw9fsdsaHVkzHhjEyVPUhVXjf7aA98pYoHHE1G+/ZwxBEwejRccQU0bpxtixynXtG8OTRq5D3InEO1FJGBmHdqQ2AMqnMQGRBcTzzvCPthAjYLkenBuX8GHqs3IdIJG7H8HDg7iSUnAmfEeLt+isg3wH3ARVEfKdkQKwCqfAEgwpbANli3NXzdvVLCDBwI//d/8OSTlhLLcZy0IeLBAnIWE6VJMefiC6PqaaH9t4g/hwmqJ6doRQviR3j7FNg4lYaiLvPYUoRibIz5f5j77JTQ5oQ55BDYfnt31nGcWsIF0qmCGcD5cc4PBqan0lDUZR63A2XY4s9VwJ+wED7zgJ6p3DAvaNDAckW++y5Mm5Ztaxyn3uEC6VTBxcCpiHyCyEOIPIjIfGxe8h+pNBRVIA8ELlHlY2wc+BtVnsIWZA5L5YZ5w6mn2mTJf/4DBx4IX3+dbYscp97gAukkxNZBbg88AWyAZZx6AtghGMqNTFSBbMr6ED3fA5sF+3OBXVO5Yd7QogWccorNQ771Fgzz3xGOky5cIJ0qUV2K6uWoHoPq0aheATRCZHwqzUQVyI+BHYP96cAAEdpi6UOWpHLDvOL4423Jx7p18MAD3ot0nDThAulUg42BY1KpEFUghwNbBPvXAocCi4BzgX+mcsO8Yvx4ggWxUFbmvUjHSRNFRbByJaxZk7ys41SXqOmuxqnyYLD/IdAO2BMLFPBErVlXlykpsV5jeeDyNWu8F+k4acIzejiZIHK6qzBBVo8PgV9ESJzzK58ZNsyGVsN4L9Jx0oKHm3MyQdJAASK0AvYC1gKTVSkToRE2/3gZFjHhhig3E6EnNlzbELhPtWI9ESS43gtbTnJaIMQJ64rQCYsM3wTLKXauamopTWqFd96pPP6zZg28/XZ27HGceoQLpFMJkYlJSmyUapPJsnnsCywA/g94AfifCDsCM7FAsMOwyDpJEaFSMk0RYpNpHo7FzesI9MeiuCerexNwjSqdgKuC4+zz0Uc2vKoKI0fauRkz7LzjODXCBdKJw3dJts+Asak0mKwHOQyLq3cdcAYwBIukfi3wX9VKqUyqoiuwUJVFACK/J9MMZwnpDYwN2n1XhI1FaI3NeSaqq6z/ZdCCytHjs8/RR1vggPHjYVdfFeM4NcUF0qmE6unpbjKZQO4GHKjKHBGuwEL1XFZNx5x4yTT3ilBmqyR1hwAviXAz1iPeN97NReiP9UopKBCKi4ur8QjVZ9fdd6fJgw/y/sEHr/dszTFWrlyZ8feS6/g7iU+238vKlQ2BP/H++wtp23Zx1uyIJdvvxUkvyQSyCPgGzDFHhFVAdccIoyTTTFSmqrrnABeoMkGE47GUKYdUKqyMBkYDNGmi2q1bt4hmp4kBA+Css+jWogV07pzZe0ekuLiYjL+XHMffSXyy/V5UoWFDaNlyO7p12y5rdsSS7ffipJcoXqwtRSgSYROC4czg+Pct4r2iJNNMVKaquqcC5ck1n8CGcnOPPn2goAAefzzbljhOnUcEWrb0IVandokikHOxXuRyLK7d1OD4Gyz83DcR7zUV6ChCexHiJ9O041NEEBH2Bn5SpSRJ3aVYrFiwRM4LItqTWTbZxLJ8jB+/fm2k4zjVxqPpOLVNsiHW7um6kSqlIlRIphnMbQ4Iro/C8oj1AhZiyzxOr6pu0PRZwHARCoBfCeYZc5K+feH002HqVOiamx1dx6kruEA6tU2VAqnK6+m8mSqVkmkGwli+r9j6ykh1g/NvAXuk085a46ij4OyzbZjVBdJxakRRESxblm0rnJxEpBnQCUusUXGkVPWpODXikjRQgJNGNt4YDjvMhln/8x/LG+k4TrUoKoJ587JthZNziBwCPApsEueqYqOQkfC/0Jnm+ONh8WJLpuw4TrXxIVYnAcOB54E2qDaI2SKLI7hAZp4jj4TGjd2b1XFqSFER/PQTlJZm2xInx2gHDEO1xkFjXCAzzUYbQa9e8MQTFrzccZxqUR5N58cfs2qGE0akJyLzEVmISOJEFiJ7IlKGyLFJ64oUIfIKIguCz5ZJrPgfsEMNnwRwgcwOfftaOqy33sq2JY5TZ/FwczmGSKWY2YjExtsuL3cjtiohSt1LgcmodgQmB8dVMQq4GZEzEdkLkc4VthSI5KQjwpgElxRbWrEQeFw1B+Og5iJ//jM0bWrOOgcemLy84ziVcIHMOboCC1FdBIBIvHjbAIOACVhO4Sh1ewPdgnIPAcXAJVXY8WTwOTrOtZScdKJ6sW4K/AlYB8wOzv0BCwH3AXA0cK0If1JletSb5y0bbABHHAFPPgnDh1uEHcdxUsIFMvO0ggJEpoVOjUa1XIiSx9sW2QrogwV1CQtkVXU3R7UEANUSRDZLYmb75E8Sjah/mf8HrAT+psoqABGaAfcCM7DF/WOBW4CD02VcvaZvX5uHfP11ONhfmeOkigtk5vkWSlHtkuBylHjbtwOXoFoWk7QhSt1oqH5RrXpxiDoHORi4tlwczQZWAddjgcLXYGPKndJlWL2nVy9o3ty9WR2nmrhA5hxR4m13AR5D5HPgWOBuRI5KUncZIq0Bgs/lSS0R2RWRsYhMQ2QqIg8h8sdUHyiqQG4AtI5zfovgGsAKPPBAdJo2tSUfTz0Fa9dm2xrHqXNsvLEFLf/uu2xb4gRMBToi0h6R+PG2Vduj2g7Vdthc4bmoPpOk7kQsKQXB57NVWiFyJPAhJrgvAC8C2wAfIvKXVB4oqkA+DdwvwnEitBOhrQjHYamlysP2dAU+SeXmeU/fvva/+7XXsm2J49Q5GjY0kfQeZI6gWgq/x8yeB4xHdQ4iAxAZUK26xg1AD0QWAD2C46q4Drge1e6oXhls3YF/B9ciE7XHNwC4FXg4VKcUGANcFBzPwwKHO1Hp2dPWRT7+uIWgcxwnJTyaTo6hWjlmtuqoBGVPS1rXzn9Har4t2wP/jXP+v8DFKbQTrQepyipVBmAJlHcHOgNFqpyjyi9BmenuwZoijRtbAPOnn4Y1a7JtjePUOVwgnTgsJ34Ciz2AlMLbpxQoQJVfVJmpyoxyYXRqyPHHWyiQV17JtiWOU+dwgXTicC9wDyKXI9IdkW6IXIEFEIi3NjIhUQMFNME8WQ8mTvoQVXZN5aZOiB49LDX6449bAAHHcSJTVASffpptK5wc4zpsWeKFwLDg3FLgauCOVBqKOgd5N7a48wngbaq7PsWpTGEh9OljayJ//RWaNMm2RY5TZ/AepFMJVQVuA25DZMPg3M/VaSqqQB4FHKfKq9W5iZOEvn1hzBh48UWbk3QcJxJFRfDDD7BunadXdeJQTWEsJ6pArqJiGCAnnXTvDptsYrFZXSAdJzJFRaBqaa9aJsvx4NRfRGYCB6L6AyKzqGqUUzXylGBUgbwJ+LsI56iyLmrjTkQaNYJjjoFx42DVKmjWLNsWOU6dIBxNxwUyr5kA/BbaT8s0YFSB7IEFK+8pwlygQugXVY5MhzF5Td++MHo0TJoExx6bvLzjOBUEskOH7NriZBHVa0L7Q9PVbNRR+2+xaDqvAV8D38VsTk058EDYbDOPzeo4KeDxWJ1KiLyGyMZxzm+ESEphyyL1IFU5PZVGnWrQsKH1HB94AFautJRYjuNUiQukE4duQGGc802wkdDIuN9XLtG3L6xeDc89l21LHKdO4ALp/I5IZ0Q6B0e7/n5s255Af2BJKk0m7EGKMBM4UJUfRKjSK8gDBaSJ/feHLbe0YdYTTsi2NY6T85Q75rhAOsA0TKcUeDnO9dXAoFQarGqINewV9GQqjTrVpEEDOO44GDUKVqywQOaO4ySkUSPYcENPeeUA0B5LvLwIyy71TejaGmA5qmWpNJhQIFW5Jt6+U8scfzwMHw4TJ8JJJ2XbGsfJeTbZxHuQDqD6RbCXtqlDT3Cca+y9N2y9NTz0ENx7rw23brFFtq1ynJzFw805lRApwHqR2xDrsKM6NmozUYOVFwHXkzhYuY8FposGDawXeeutli592DC4665sW+U4OYsLpFMBkR2B/2P9kGsZpnVrsWnDyAIZtSt6P3A48CAwBJvoDG9OOjn4YIuftW6dLfv4+utsW+Q4OYsLpBPD7cAHQAssTOpOQBdgOnBMKg1FHWI9GOihynupNO5Uk4kTrfeoCqWl3ot0nCpwgXRi2BOLy/oLIuuAAlQ/RORi4E6Ivuoiag9yOZZfy6ltSkrgwQdNHAHWrrVMH96LdJy4lAukehI+xxCs5wjmybpVsL8Y2C6VhqIK5OXAtSLUKLyLCD1FmC/CQhEujXNdRLgjuD5ThM7J6orwuAjTg+1zEabXxMasM2yYDa2G+fVXuPzy7NjjODlOURGUlcHPNUps5NQjZgO7BfvvA5cgciBwDbAwlYaiCuQVwKHAchHmBeL1+xalAREaAndhc5k7A/1E2Dmm2OFAx2DrD4xMVleVvqp0UqUTtnbzqYjPlJu88w6sWVP5/GOPWaaP2qKkhE6DB3tP1alzeDSdHEKkJyLzEVmISKVOECK9EZmJyHREpiGyf3B+h+Bc+bYCkSHBtaGILAld65XEiuuxXiSYdm0NTME07PxUHifqHGQ6AgV0BRaqsghAhMeA3sDcUJnewFhVFHhXhI1FaA20S1ZXBAGOBw5Kg63Z46OPKp976imL09qvH0yYAAW1sDrnmmtoMWuWz3c6dY6wQLZrl1VT8huR8o5MD2w4cyoiE1EN/42fDExEVRHZFRgP7IjqfKBTqJ0lWIKMcm5D9eZIdqi+FNpfBOyMSBHwA5raQHzSv7QiNAKaA3ep8kWy8lWwFRWTLi8G9opQZquIdf8ELFNlQbybi9Af65VSUCAUFxenaH4WKSpiy0GD2P6OO1h69NF8csEF5sSTDlRpM348He65x/yh77+f9w4+mDXlf3XynJUrV9at70qGyKX38sUXLYDdmTJlBitW/JBVW3LpvWSBrsDCQJRApHInSDXsy9Kc+CFMDwY+DS38rzmq1RpfSCqQqqwV4Rzg7urcIES8v+ixLydRmSh1+wGPJrq5KqOB0QBNmqh269YtoaE5Sbdu0Lw5W/7732zZtStccUXN2/zhB+jfH55cP0DQUJV9J0/2XmRAcXExde67kgFy6b1suql9br31bmTbpFx6L7VBKyhAZFro1GhURwf7UToyINIH+De2pv7PcW5zApX/lg9E5BQs3uqFqFb8JSQyhahJklUjjzJGnYN8mZoPXS7GxoLLaQMsjVimyroiFABHA/U7meL118Mpp8CVV9r6yJrwv/9Bp07w9NMVh2zXrPG1l06dwucgM8e3UIpql9A2OnQ5SkcGVJ9GdUfgKGBYhWsihcCRwBOhsyOBDtgQbAlwS5z7zAbmBNvHwB6YYC8Oti2Dc/OSPWOYqAI5GfiXCLeLcLIIR4e3iG1MBTqK0F6EQuxXwsSYMhOBUwJv1r2Bn1QpiVD3EOBjVRZHtKVuIgL33QeHHgpnnQWTJqXeRmkpXHMNHHCACWPv3ha9J7bMsGHx6ztOjuEZPXKGKJ2g9ai+AXRApFXo7OHAh6guC5VbhmoZquuAe7Gh3Ni2Bv2+WbSch7C5zVOCbUfgAaA0lQeK6u0xIviM5wGkQMNkDahSKsJA4KWg/BhV5ogwILg+CpgE9MJccVeBJWpOVDfUfLwuef2kUSMbEu3WzTJ/FBfDnntGq/vllxYA/c037fOuu+DAAyt7za5daz1Mx6kDNGkCzZq5QOYAU4GOiLTHnGxOAE6sUEJkO2x+UYPcjYVAOBdL5akykdaolgRHfbDeYlWcAuwTxyHnbuBdYHDUB4okkKrpiY6uyiRMBMPnRoX2FTgvat3QtdPSYV+dYcMN4fnnYd994c9/hrffhu2SrH+dMAHOPNN6h2PHwskn2/mQ12xxcTHdli6Fv/4VBnkEQafuUFTkKa+yjmopIhU6MqjOQWRAcH0UFurtFETWYvkZ+/4uZCLNMA/Ys2NavgmRTlhn7PM412MR4I/AJzHn/5jqI3k2j7rKFlvAiy/CfvvBYYeZSG6+eeVyq1bBkCGWGWTPPeGRR6oW03794O674bLL4JhjYOONa+sJHCdteLi5HEG1ckfGhLF8/0bgxgR1VwGbxDl/copWjAHuQ6Qj1mME2Bu4GBtmjUxkgQwyevQkTvoQVa5N5aZOmth+e3juOejeHY44AqZMgQ1CwY5mzDDBmzcPLr7Y5hULCxO3BzbPeccd0KULXHutZRVxnBzHc0I6IS7GwqMOBv4VnCsBbiC+g09Coqa72ht4Hpv83BQbX24dHH8OLpBZY6+9YPx4c7Y57ji45x4bPj30UBPEli3hlVfgkEOit9m5szkB3Xmnfe60U+3Z7zhpoKjIfgc6TuDMcxM2NLtRcG5FdZqKOrf4H2Ac5jb7K7bkYxtsTUr87rKTOY44woTxxRdNCN94w9ZJHnIIzJyZmjiWc9110Ly5Dc96FGgnx/EhVicuqiuqK44QXSB3BUYETjRlQGNVlgGXAEOre3MnjZx5Jlx4ISwIAgk1amTzjuWrqFNl001tiPXlly39luPkMJ7RI8+x+K4tg/1ZwXH8LQWizkGG1wEsA9piCy5XYgswnVzgl1+gYUNLbSBivcCaRMQ55xzrmf797+YI1KRJ+mx1nDRSVGSrlVatsoEPJ++YgE35QXpihwPRBfJDLAnlJ0AxcJ0ImwMnQbRsHk4tU55HsqzMjssj4lx5pXm8VodGjWD4cOjRw5x1/vnPtJnrOOkkHE3HBTIPUb0m7n4NSSUfZHlEhCuwJJR3Ai0JAoA7WSZeHsmysppHxDnkEDj6aAtzt7h+Bypy6i4ebs6pDaIGCpgW2v8GCwfk5BLx8kiuWWPrI2vKzTdbYIJLLoFx42renuOkGRfIPEdkFtGDle8atdmUAgWI0AULGvucKr+I0Bz4TTW1+HZOLRAvj2S6aN9+/TrKc8+14ASOk0O4QOY9aZt3DBN1HeTmWHDwPTGV7ggsAm7Fln1Ejm3n1FEuucTmNAcNgqlTzRmoJpSUwAknwOOPV3+O1HECXCDznDTOO4aJOgd5G/A1FgZoVej8E8Ch6TbKyUGaN7eh1o8+gjFjataWKvzjH/DWW541xEkLLpBObRBVIA8GLlclNl33p1jAACcfOP54S5P1z39asuVU+fVX64X+8Y82l7luneeedNJC06bQuLELpBMgcjoiLyPyMSKLKmwpEFUgm1JxLWQ5m2JDrE4+UB6n9fvvLadkVJYsscg+W28NZ5xhw6vlpMPT1sl7RDyjhxMg8g8s5uoHQDvgGSxFVhEWyDwyUQXyDaiQUkpFaIhF0pmcyg2dOs5uu8HZZ8OIETBnTuJyquZZ268ftGsH//qXOfc88YSt5i6nfL2m9yKdGuLh5pyAs4D+qF4GrAVGoHokJpptU2koqkBeDJwlwitA4+BGc4H9gMtSuaFTDxg2DDbaCAYPrhzb67ff4OGHLYj6vvvCCy/A+efDwoXwzDPw2mu1s16zNigpodPgwS7edQQXSCegDfB+sL8a2CjYfxTLRxmZSAKpylws2eTbwMtAE8xBZ3dVPk3lhk49YJNNTNAmTzaHnQMPtKDo11wDbdtaNpEVKyzM3eLFcMstsO22Vrc212umm2HDaDFrVm6Kt1MJF0gn4GugVbD/BbBPsL8dUddKBkTtQaLK16pcrcoRqvRS5QqgUITxqdzQqSecfbY52wwZYtlDOnWCoUNhjz0sq8jcubZmMpyfEswLVtW25cvNs+Lss2t3HWd1KCmBMWMQVfsR4L3InMdzQjoBrwFHBvv3A7ciMgV4HHgqlYYiC2QCNibFLqtTTygogKuugpUr7bhBA1u28fzzFti8QYSv1qabWm9z7Njc86649FIbLgbr4XovMufxHmSeI3JwsNcfuA4A1VGY/8wsLGTquak0WVOBdPKZ116zgOZggQMeeST1NgYPhtWrYfTo9NpWExYtgv/+d/3xunVw//3ei8xxiorsq7R6dbYtcbLEK8EyjsuAzX4/q/o4quejOgLVtak06ALpVI+SEvM+XRt836rrjfqHP1hA9BEj1reVTVStBxzrfLR2rfcic5zyYAHVWaLr1At2wYZQBwFfIPI8IkchUu2wXy6QTvVIZ/aQCy6ApUvhyVoJp5ga111nHrexrFsHr76aeXucyHg0nRxApCci8xFZiMilca73DhIXT0dkGiL7h659HiQ7tmvrzxch8goiC4LPlnHvrToP1YswL9a+mEPOE8ASRG5EZIdUH6dKgRRhYlUbMDzVGzr1hHR6o/bsCdtvD7fdlt2U8E8+afOqp5xigqhK8ZQp8OOP9te3ffvs2eYkxQUyy1hP7S4s29POQD9Edo4pNRnYDdVOwBnAfTHXu6PaCdUuoXOXApNR7RjUryy8YVRLUX0K1SOwdY93AEcDcxF5I5VHStaD/C7J9hkwNpUbOvWEsDdqeKuON2qDBjYXOXWqCW82+OADE8Z99oF77rHQLOW0aGHh9V56CaZMyY59TlJcILNOV2AhqotQXQM8BvSuUEJ1Jfr7r+DmRFt20Rt4KNh/CDgqskWqS4G7MZH8EVu7H5kqBVKV06NsqdzQceJyyimw8cZw++2Zv3dJCfTubV61Tz8NTZpULnPeedCmDVx2WXZ7uemkpMTWsNYT5yMXyNqnFRQEQ6PlW//Q5a2Ar0LHi4NzFRHpg8jHwPNYL7IcBV5G5IOYdjdH1eJT2udmREHkEEQeAZYC12CC3aXqShXxOUgnN9hgA+jfHyZMgC++yNx9V682cfzxR5g4ETbfPH65Jk0sEMJ778Gzz2bOvtpk2LB6lVHFBbL2+RZKUe0S2sLu5xKnSuVfk6pPo7oj1hMMf/n2Q7UzNkR7HiIHpGygyDaIXI3IZ1hQmy2xZR9bonoeqikNcblAOrnDeefZ0OaIEZm5n6oFT5861cLj7bZb1eVPOQV23NGGW8vKMmNjbbFokS1dqUcZVTbYwJbnukBmjcXA1qHjNljvLT6qbwAdEGkVHC8NPpcDT2NDtgDLEGkNEHwuj9ueyCtYnuKzsd7i9qh2Q/VhVKuVVMMF0skdttkGjjkG7r13fQCC2uT66+GxxyyQ+lFHJS9fUGB15s2z4AZ1jS++gLvvhiOOMKeociererKEpTyjhwtk1pgKdESkPSKFwAnAxAolRLZDggl+kc5AIfAdIs0R2TA43xzLMzw7qDURODXYPxVINISzGnPG2RrVy1CN446eGi6QTm5xwQXw00/w0EPJy9aECRPgyivhpJMsak5U+vSBPfeEq6+2/Ja5QKK5xLVr4fXX4ZJLbL1pu3bWS589u2K50tJ604v0lFdZRLUUGAi8BMwDxqM6B5EBiAwISh0DzEZkOubx2jdw2tkceAuRGVig8edRfTGocwPQA5EFQI/gON79j0R1IqppG95xgXRyi733tkwgw4dXXmeZLj780ELc7b239VYl3tRJAkTghhvgq69g5MjasS9VwnOJy5fbj4u+fc3pqFs3Wz6zxRZw663w8cdw+OEW+ShMPQmn5z3ILKM6CdXtUe2A6vXBuVFByDdQvRHVXYKlHPug+lZwfhGquwXbLr/XtWvfoXowqh2Dz4z9C2dUIEXoKcJ8ERaKVF7LIoKIcEdwfaYInaPUFWFQcG2OCDdl4lmcWmTIEFiwACZNSn/bJSVw5JHQqpWl34rnsZqMgw6CHj1suHXFirSbmBLlEY3WrTPB3nxzOO00ePNNOPZYeOop+PZbC3JwwQWwww7w7ruV17CWlVnowDqOC6STTjImkEGC5QqLSEWIXUR6ONAx2PoDI5PVFaE7tk5mV1V2AW6u/adxapVjjrElFele8rF6tc01/vBD1R6rUfjXv2ws75Zb0mZetRg2rKLYde1qazqXLIH77rMh4Y02qlgndg3r11/bEpsttqjzS1hcIJ10kskeZFdgoSqLVIm/iNSOx6qiqrwLbCxC6yR1zwFuUOU3ANUEHk5O3aFRIxg40PJNzpyZnjaXLrVcle+/bx6rnTrVrL0uXeC440wgl2fpKxfuPYKJ26xZsOWWqQ0bb7453HgjFBdXDNJeB3GBdNJJQQbvFW8R6V4RymyVpO72wJ9EuB74FbhIlamxNxehP9YrpaBAKC4urvaD1FdWrlyZM++lYKed2KdJE5ZfeinzL764xu3tfs45tPjmG37acUc+atnSxCACVb2TpkccQdcJE1hyzjksHDSoxjamSsfbbqP1mjUVfuWuW7uWkgEDWDBkSGqNbbcdu++yC03PP5/3W7SgtEWLKovn0nclzIoVbVm5sj2vvPI6jRplvjecq+/FqSYadNdqewM9DvS+0PHJoHfGlHkedP/Q8WTQPaqqCzob9A5QAe0K+hmoVGVL48aN1anMlClTsm1CRc45R7VxY9Vly6rfxm+/qQ4cuH5AsWlT1ZKSyNWTvpOzzlJt1Ej1s8+qb2N12W23eMH+VDt1ql57M2aoNmyoeuaZSYvm3Hcl4K677BV8/XV27p+r7yVdAL9ohjQjF7ZMDrFGWUSaqExVdRcDTwXP8z6wDmiVRrudbHH++Za0eNSo6tWfPdvm5EaMWJ/AuboZRxJx1VXmEXr11elrMyqXX26fzz5b83i4ALvuCn//u81dvvVW+uzMIB5Nx0knmRTIqUBHEdqLEH8RqR2fEniz7g38pEpJkrrPAAcBiLA9tvD021p/Gqf22XFHW5Jw990mlFEpK4Obb4Y99oDFi6GwcP08XXXzViaiTRsYNMjm7mLXF9Y2t98OHTrAn/+cvjavvtoCNgwYUNnTtQ7gAumkk4wJpCqVFpGqMkeEASKULyKdhIUKWgjcC5xbVd2gzhhgWxFmY847p6pGihDv1AWGDIFly+Dxx6OV/+wzW4bxj39Ar17wl79ULpPuXuSll5qnaHmPLhO8/76lFjv//MprGmtC8+bW454zx9ZP1jFcIJ10kkknHVSZhIlg+Nyo0L4C50WtG5xfA5yUXkudnKFHD9h5Z+stnXxyYu9MVesZDh5sZR580GKndu6cvryViSgqgosvNoF8+23Yd9/0tZ2I4cNhww1tzWO6+ctfbHnINdfA8cfXqTyYLpBOOvFIOk5uI2K9yI8+gjcS5Dpdvtz+oP/tb7b8YtYsOPVUq5vOvJVVMXiwLZe49NLaX0u4ZAmMH2/PG7vGMV0MH24904ED69TaSBdIJ524QDq5z0knwSabxA8c8OyzFmf0xRdtTeLkybbeMdM0b24OO2++abbUJiNH2jBxbS4t2XpruPZai2b01FPpaTMD+Sc32sj8sVwgnXTgAunkPk2bmtPIs89amLQDD7RQdGecYZFxttrKosf8/e/rvVWzwZlnwrbbwkUXwQEH1I4QrF5tXr29e9u9apNBgyygwvnnpyekXgbyTzZoAC1bukA66cEF0qkbnHuuDfn172+9tM6dLSj35ZdbEuNddsm2heYte911MHdu7QnBuHEW4m7w4PS3HUtBAdxzj/X8rrqqZm19+qktH8lA/kmPpuOkCxdIp26w5ZYWZHzWLJsT++UX61Fed50JU65wwAE291nuNJROIVC1YebddrNedCbo2hXOOQfuvNOyoKTKTz/Bv/9tw+Br19q51attXrmW8JRXTrpwgXTqDuHlDI0awQsvZM+WRFx//Xo7051CavJkW34xZEhqsVZryvXXw2abwdln29xnFL791vJttm0L//xnZU/ixx+3+K+14ADkPUgnXbhAOnWDkhL4v/9bf5zuBf/poDx4eGmpHZeV2bBiumwcPtyE6oQT0tNeVDbe2NZETpuWPAfm0qVw4YUmjNddBwcfbGm3CmJWlImYx+8xx6RdzVwgnXThAunUDYYNq5xAOd0L/mtKPBvXrDEhqCkLFsBzz5mzUnVyWNaUvn1tTeo//2kiGMtnn9lQbPv2JuRHH2293QkTYOHCyj1IVRs2f+45GzJOtISnGrhAOunCBdKpG7zzTu0v+K8p8WwEE4moQ5OJuOMOG1Y+55yatVNdRCzk35o1cPbZdBo82HrGH39sa047doQxYyxwwSefWOi9nYN0r4nWoi5ZYv9+TZpA9+4W5q68910Diops6jMNTTl5jgukUzfI1IL/mhDPxvvug5Urbb6tuvz4ow3d9utnSY2zxXbbwRVXwHPP0WLmTNhvPxPBJ56wJSGLFpnXayrLT7p0Meefk06ydZfdusEXX9TIzPJgAT/+WKNmHMcF0nFqlTPOsOHJq66yHmZ1GDPGvHYzsbQjGSedBCIImCAOGmSCdtttth61Omy4oS3ZefhhmDHD1l5OmFBtEzfZxD59mNWpKS6QjlObiFivapttrAeYaremtNSGVw84wNZ+ZpubblrvcFNYaPZtuml62v7rX2H6dBuuPfZY85pdtSrlCDwebi6LiPREZD4iCxGpPPku0huRmYhMR2QaIvsH57dGZAoi8xCZg8jgUJ2hiCwJ6kxHpFemHscF0nFqmxYt4NFHbc6tf//UljZMnGg9tFzoPZZ76ZavZ6wNT+IOHSzIwiWXwOjRsOee9uwpBF5wgcwSIg2Bu4DDgZ2BfojsHFNqMrAbqp2AM4D7gvOlwIWo7gTsDZwXU/c2VDsFW6WkFbWFC6TjZIK99rJlD088YfOSUbn9dmjXzkLLZZtMeRIXFsINN8DLL8M339g7SyECjwtk1ugKLER1EaprsPSDFb+4qivR338hNocgNaFqCaofBvs/Y2kNqzlmnz5cIB0nU/zjH3DIIdYjmjs3efkPP7SweoMGpTfnY3XJtCdxjx6WDLo8KMJvv5mTUBLyRiAzEPw9llZQEAyNlm/9Q5e3Ar4KHS8mnsiJ9EHkY+B5rBcZe70dsDvwXujswGBodgwiLWv+JNFwgXScTNGgAYwdCxtsYI47q1dXXX74cCv7t79lxr5khLx0i6dMqX1P4pISeOyx9UPS69aZw9Krr1ZZbeON7bPeC2QGgr/H8i2UotoltI0OXY4X3qnyfILq06juCBwFVDReZANgAjAE1fII+SOBDkAnoAS4pYaPERkXSMfJJK1bm0jOnm0RZxLx9dc2b3naaTaHmY/EG9JVhcMOM8enBHO5DRuaSNZbgSwttfi2I0fa+0lntKaasRjYOnTcBogTVSJA9Q2gAyKtABBphInjOFSfCpVbhmoZquuAe7Gh3IzgAuk4maZnTxPHkSPh6afjlxk50pxhzj8/s7blEokCLzRvbhGFTjrJ1pjGod5G03n1Vdh9d4toVD70vGaNzXHPnp1d22Aq0BGR9ogUAicAEyuUENkOCQwX6QwUAt8F5+4H5qF6a0yd1qGjPkDGHtQF0nGywb/+ZYvk//Y3+PLLitd+/dUE8ogjbMlDvpIoOMSPP1rv8rHH7B3OmlWpar3L6LFwoeU+7dHDwgQVFlbsQX/5Jey6K5x1lg1NZwPVUmAg8BLmZDMe1TmIDEBkQFDqGGA2ItMxj9e+gdPOfsDJwEFxlnPchMgsRGYC3YELMvVILpCOkw0KC20Ide1aW/8Xjov26KPmvVmLKaHqNA0amLPOq6+aWOy1l3m4hqg3PcgVK+Diiy1i0eTJNrTas2flcoWF8Mc/WsCF7baDa65J2LuuVVQnobo9qh1QvT44NwrVUcH+jajuEizX2AfVt4Lzb6EqqO5aaTmH6smo/jG4diSqGfsF4ALpONliu+1g1KiKjhaq5pzzhz/AQQdl175cp3t362XuvbdFLDrtNIs4RD0QyLIyuP9+G0H4z3/sR9Qnn1jg+6lT43sTN2gA8+aZ5+/QoVb3vvtqHgc4j3GBdJxs8te/WrDv666Dp56yMGszZthSkEzmfKyrbLEFvPKKhfIbO9YSPM+bV7cF8s037TnOPNN+RE2daj3k1sFUXFVxiTt0gPHj4X//s8wqZ51l36kXXlg/JJuF5SF1FRdIx8k2I0bYH8JTToGZM6FxYxNOJxoNG9qQ4ksv2dB0ly50X/Iwjb8vQQ+oA0JQLlhTp9rynwMOgOXLbaj9rbdsnjVV9t3XRPKJJyxcX69ecOihFsovC8tD6ioukI6TbTbYAO688/fhQcrKbG7NSY0ePUwA9tiDY589mVc5qHpCkOke1lVXWa9xn30sKfjQoTB/viXGrskogojFtJ03z4LJf/ihecCOHm3LQ+6/3wLOR6WkhB0gC8lIs0dB8iKO49Q6zzxjPaGyMptLGjYM7ror21bVPbbcEl57jbmH/52dX73TlqnffbfFtC0qgo02Wr9tuGH844ceMsHKxL/Bo49WDD345puwxx7pvUdhoTl8nXqqzdvOmGHnf/vNhmRbtbJg+m3b2md4a9sWNtvMxHbYMDbIs06VC6TjZJvyIODlzhTlQcCvvDK7+R/rKgUFNG1UyhoKKKTUfnA0a2YxbX/+GZYtgwULzEP0559tCDIeo0db0PRttkm/jT/+aGthx4xZf65hQztOt0CW8+uv1jMNU1AAhx8O335r115+ef1IRjmNG9v8Z+xypDwgr34NOE5Okqkg4PlCSQnbTH7AxBHs3X71lUXfee01mDbNPEK//trEYO1a8+j54gs4/nho1MjqlZba2sLJk9Nr3/PPm5fygw+uTx0GtZMdJUy871mDBtZznjQJ5syxHwzff29D1c8+a0P/559v4p1KFpp6gguk42SbTAcBr+8MG4ZoCj84CgqgZUsTxokT16fzAutlHnKIOc8sWVIzu77/3hyxjjjC7tenjwlUVDtrSpTvmYjZtttucOSRMHAgXHCBPbsLpOM4Gacqt30ndd55hwZrq/GDI14Pq1Ejy0k5cSLsuCPcfHNFAY3Ks8/CLrvYnOOVV1ov9tNPM/vDqLrfs3jvJU9wgXQcp37x0UesXaMIyrBrUxCCRD2stWtt+LF7d0tZ1qkTFBdHs+Xbb6FfPwsTt/nmtpTj2mttXq+u/DBKFBM3D3CBdByn3tGokU2tpRQsoCrB2nZb60VOnGhOPd27W7D0quKePvmkhYibMMFEcepUE9e6Rui9fAAJPJrqJxkVSBF6ijBfhIUiXBrnuohwR3B9pgidk9UVYagIS0SYHmy9Ytt1HCe/GDfOdOz22815ddy46PXatbOpwbj1/vIXS3Z95ZW2CH+HHeD223lkbCl7tilBul/NoVvO5ou9joPjjjMP2A8+sPLlzj+p3K+6dtZSPXMHziNUNSMbaEPQT0G3BS0EnQG6c0yZXqAvgAro3qDvJasLOhT0olRsady4sTqVmTJlSrZNyDn8ncQnl9/Lww+rNmtWsRvYrJmdT2u9BQtUe/ZUBZ0lf9QJHKVliK6isf5KoX54/L9V167Nvp1prddMNUOakQubqGbGM0mEfYChqhwWHF9mAs2/Q2XuAYpVeTQ4ng90A9olqivCUGClKjdHtaVJkyb666+/puGp6hfFxcV069Yt22bkFP5O4pPL76VdO1uxEUtBAWy/feJ6n3xSMalKpHqq7DT/GW5fN4g2mJdrGUJ3pvBOwYHpv1/W6zVH9Ze8CRKcyUABWwFfhY4XA3tFKLNVhLoDRTgFmAZcqMoPsTcXoT/QH6CgQCiOOsmeR6xcudLfSwz+TuKTy+/lyy8PBCr/DS8tVTbd9JuE9ebO3bRa9SbMO4pDmcQZPEABZaylEX0Zz5ulB9TK/XKlXl6Qqa4q6HGg94WOTwa9M6bM86D7h44ng+5RVV3QzYMh2Aag14OOSWaLD7HGJ5eHzbKFv5P45PJ7adu24jBi+da2be3U67LVUl1FkwqVfqGpdmlTklN2pqdefg2xZtJJZzGwdei4DbA0YpmEdVVZpkqZKuuAe4GuabbbcZw6xPXXV3YladbMztdGvXE7DkOouE6wAWU8vEPVC/4zbWc66+UNmVJi0ALQRaDtQ442u8SU+XOMk877yeqCtg7VvwD0sWS2eA8yPrncK8gW/k7ik+vv5eGHrecjYp/JHFFqVK9Tp/hds06dcsvONNTLtx5kxpx0AIIlGLcDDYExqlwvwgATakaJIMAIoCe23uZ0VaYlqhuc/y/QCVDgc+BsVapYnOROOonIZceLbOHvJD7+XuJT39+LiKxS1ebZtiNTZDSbhyqTgEkx50aF9hU4L2rd4PzJaTbTcRzHcTySjuM4jpMmRHoiMh+RhYhUCgaDSG9EZiIyHZFpiOyftK5IESKvILIg+GyZkWfBBdJxHMdJByINgbuAw4GdgX6I7BxTajKwG6qdgDOA+yLUvRSYjGrHoH5l4a0lXCAdx3GcdNAVWIjqIlTXAI8BvSuUUF3JeseX5pjvSLK6vYGHgv2HgKNq7QlicIF0HMdxItEKCoKh0fKtf+hyokAvFRHpg8jHwPNYLzJZ3c1RNcdL+9wsHc8ShYw66eQKv/32m4rI6mzbkYMUAHGCUeU1/k7i4+8lPvX9vTRFtUuCa/HC7VReJqH6NPA0IgcAw4BDItfNMHkpkMCHmvgfOW8RkWn+Xiri7yQ+/l7ik+fvJUowmPWovoFIB0RaJam7DJHWqJYg0hpYnl6zE+NDrI7jOE46mAp0RKQ9IoXACcDECiVEtkNEgv3OQCHwXZK6E4FTg/1TgWdr+Tl+J197kI7jOE46US1FZCDwEkFAF1TnIDIguD4KOAY4BZG1wGqgb+C0E7+ucQMwHpG/AV8Cx2XqkTIaSSdXEJH+qjo623bkGv5eKuPvJD7+XuLj76V+kZcC6TiO4zjJ8DlIx3Ecx4mDC6TjOI7jxCGvBFJEeorIfBFZKPHiBOYpIvK5iMwSkekiMi3b9mQLERkjIstFZHboXJGIvCIiC4LPjMWBzBUSvJehIrIk+M5MF5Fe2bQx04jI1iIyRUTmicgcERkcnM/770t9Im8EUuLE+pPKcQLzme6q2imP13ABPIilWgtzKTBZsxAHMod4kMrvBeC24DvTSVUrZdqp55QCF6rqTsDewHnB3xP/vtQj8kYgCWL9qeoiTRQn0MlrVPUN4PuY01mLA5krJHgveY2qlqjqh8H+z8A8LDRa3n9f6hP5JJDR4gTmJwq8LCIfSMXYig5srkEcSM1wHMg6wEARmRkMwebtUKKItAN2B97Dvy/1inwSyJyM9Zcj7KeqnbHh5/PEYiQ6TlWMBDoAnYAS4JasWpMlRGQDYAIwRFVXZNseJ73kk0CmFicwj1DVpcHncuBpbDjaMZaJxX9EMhwHMpdR1WWqWqaq64B7ycPvjIg0wsRxnKo+FZz270s9Ip8EcirQUUTaS6I4gXmIiDQXkQ3L94FDgdlV18orshYHMpcpF4GAPuTZd0Ysnuj9wDxVvTV0yb8v9Yi8iqQTuKLfThDrT1Wvz65F2UdEtsV6jWCxeR/J1/ciIo8C3YBWwDLgauAZYDywDUEcSFXNK4eVBO+lGza8qsDnwNnlc2/5gIjsD7wJzALWBaf/ic1D5vX3pT6RVwLpOI7jOFHJpyFWx3Ecx4mMC6TjOI7jxMEF0nEcx3Hi4ALpOI7jOHFwgXQcx3GcOLhAOk4dR0RURI7Nth2OU99wgXScGiAiDwYCFbu9m23bHMepGQXZNsBx6gGvAifHnFuTDUMcx0kf3oN0nJrzm6p+HbN9D78Pfw4UkedFZJWIfCEiJ4Uri8gfReRVEVktIt8HvdIWMWVODZJa/yYiy0TkwRgbikTkCRH5RUQWxd7DcZzUcYF0nNrnGixGZydgNDBWRLoAiEgz4EVgJRbwuw+wLzCmvLKInA3cAzwA7Ar0AubE3OMqLO7nbsDjwBgRaVtrT+Q4eYCHmnOcGhD05E4Cfo25dJeqXiIiCtynqmeF6rwKfK2qJ4nIWcDNQJsg8S4i0g2YAnRU1YUishh4WFXjZqcP7nGDql4WHBcAK4D+qvpw+p7WcfILn4N0nJrzBhCbaPrH0P47MdfeAf4c7O8EzCwXx4C3sQDYO4vICiyx9+QkNsws31HVUhH5Bk/W6zg1wgXScWrOKlVdWM26QuLE3Ur8RN/xWBunrk+hOE4N8P9AjlP77B3neF6wPxfYrTwnZ8C+2P/Neaq6DFgCHFzrVjqOUwHvQTpOzWksIlvEnCtT1W+C/aNFZCpQDByLid1ewbVxmBPPWBG5CmiJOeQ8FeqVXg/cJiLLgOeBZsDBqnpLbT2Q4zgukI6TDg4BYpMFLwHaBPtDgWOAO4BvgNNVdSqAqq4SkcOwRN7vY84+zwKDyxtS1ZEisga4ELgR+B6YVEvP4jhOgHuxOk4tEniYHqeqT2bbFsdxUsPnIB3HcRwnDi6QjuM4jhMHH2J1HMdxnDh4D9JxHMdx4uAC6TiO4zhxcIF0HMdxnDi4QDqO4zhOHFwgHcdxHCcO/w/PifgHVc6qJwAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# extra code – this cell plots performance scheduling\n",
"\n",
"plt.plot(history.epoch, history.history[\"lr\"], \"bo-\")\n",
"plt.xlabel(\"Epoch\")\n",
"plt.ylabel(\"Learning Rate\", color='b')\n",
"plt.tick_params('y', colors='b')\n",
"plt.gca().set_xlim(0, n_epochs - 1)\n",
"plt.grid(True)\n",
"\n",
"ax2 = plt.gca().twinx()\n",
"ax2.plot(history.epoch, history.history[\"val_loss\"], \"r^-\")\n",
"ax2.set_ylabel('Validation Loss', color='r')\n",
"ax2.tick_params('y', colors='r')\n",
"\n",
"plt.title(\"Reduce LR on Plateau\", fontsize=14)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1Cycle scheduling"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `ExponentialLearningRate` custom callback updates the learning rate during training, at the end of each batch. It multiplies it by a constant `factor`. It also saves the learning rate and loss at each batch. Since `logs[\"loss\"]` is actually the mean loss since the start of the epoch, and we want to save the batch loss instead, we must compute the mean times the number of batches since the beginning of the epoch to get the total loss so far, then we subtract the total loss at the previous batch to get the current batch's loss."
]
},
{
"cell_type": "code",
"execution_count": 90,
"metadata": {},
"outputs": [],
"source": [
"K = tf.keras.backend\n",
"\n",
"class ExponentialLearningRate(tf.keras.callbacks.Callback):\n",
" def __init__(self, factor):\n",
" self.factor = factor\n",
" self.rates = []\n",
" self.losses = []\n",
"\n",
" def on_epoch_begin(self, epoch, logs=None):\n",
" self.sum_of_epoch_losses = 0\n",
"\n",
" def on_batch_end(self, batch, logs=None):\n",
" mean_epoch_loss = logs[\"loss\"] # the epoch's mean loss so far \n",
" new_sum_of_epoch_losses = mean_epoch_loss * (batch + 1)\n",
" batch_loss = new_sum_of_epoch_losses - self.sum_of_epoch_losses\n",
" self.sum_of_epoch_losses = new_sum_of_epoch_losses\n",
" self.rates.append(K.get_value(self.model.optimizer.learning_rate))\n",
" self.losses.append(batch_loss)\n",
" K.set_value(self.model.optimizer.learning_rate,\n",
" self.model.optimizer.learning_rate * self.factor)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `find_learning_rate()` function trains the model using the `ExponentialLearningRate` callback, and it returns the learning rates and corresponding batch losses. At the end, it restores the model and its optimizer to their initial state."
]
},
{
"cell_type": "code",
"execution_count": 91,
"metadata": {},
"outputs": [],
"source": [
"def find_learning_rate(model, X, y, epochs=1, batch_size=32, min_rate=1e-4,\n",
" max_rate=1):\n",
" init_weights = model.get_weights()\n",
" iterations = math.ceil(len(X) / batch_size) * epochs\n",
" factor = (max_rate / min_rate) ** (1 / iterations)\n",
" init_lr = K.get_value(model.optimizer.learning_rate)\n",
" K.set_value(model.optimizer.learning_rate, min_rate)\n",
" exp_lr = ExponentialLearningRate(factor)\n",
" history = model.fit(X, y, epochs=epochs, batch_size=batch_size,\n",
" callbacks=[exp_lr])\n",
" K.set_value(model.optimizer.learning_rate, init_lr)\n",
" model.set_weights(init_weights)\n",
" return exp_lr.rates, exp_lr.losses"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `plot_lr_vs_loss()` function plots the learning rates vs the losses. The optimal learning rate to use as the maximum learning rate in 1cycle is near the bottom of the curve."
]
},
{
"cell_type": "code",
"execution_count": 92,
"metadata": {},
"outputs": [],
"source": [
"def plot_lr_vs_loss(rates, losses):\n",
" plt.plot(rates, losses, \"b\")\n",
" plt.gca().set_xscale('log')\n",
" max_loss = losses[0] + min(losses)\n",
" plt.hlines(min(losses), min(rates), max(rates), color=\"k\")\n",
" plt.axis([min(rates), max(rates), 0, max_loss])\n",
" plt.xlabel(\"Learning rate\")\n",
" plt.ylabel(\"Loss\")\n",
" plt.grid()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's build a simple Fashion MNIST model and compile it:"
]
},
{
"cell_type": "code",
"execution_count": 93,
"metadata": {},
"outputs": [],
"source": [
"model = build_model()\n",
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=tf.keras.optimizers.SGD(learning_rate=0.001),\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's find the optimal max learning rate for 1cycle:"
]
},
{
"cell_type": "code",
"execution_count": 94,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"430/430 [==============================] - 1s 1ms/step - loss: 1.7725 - accuracy: 0.4122\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEOCAYAAACNY7BQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABAZUlEQVR4nO2dd5gc1bG331qlVRaKIKEECmQQQQiTBDY52aRLtMGAjAEb30sONsaYZAwPl2CCsT6wScYYc4UQwWANiCBAAiQQEkpIKKEcUFhJqz3fHzWH7untSavZ2Zndep9nnk6ne870zvZvqupUHXHOYRiGYRieiobugGEYhlFamDAYhmEYKZgwGIZhGCmYMBiGYRgpmDAYhmEYKZgwGIZhGCk0b+gObC2dOnVyAwYMaOhuNBrWrVtH27ZtG7objQa7n4Wl3O7nzJmwejW0bw+DBgX7P/kEunSBpUuhVy/Ydtvi923ixInLnHPd4o6VvTD06NGDCRMmNHQ3Gg2JRILhw4c3dDcaDXY/C0u53c/jj4eXX4b994d//zvY37YtnHEG3H8/XHIJXH998fsmInPTHTNXkmEYRj3h84ejecTOQfPkz/KamuL2KRdMGAzDMOoJ/9CPPvxragJh2LKluH3KBRMGwzCMeiKTxdCsma6bMBiGYTQhvKUQFYaaGhBRcTBXkmEYRhPCC0L04e8cVFToq0lbDCJSKSIfisgkEZkiIjfHtBERuU9EZorIZBHZu1j9MwzDKDS5WAylKAzFHK66ETjcObdWRFoA74jIK8658aE2xwADk6/9gYeSS8MwjLIjU4zBXEmAU9YmN1skX9HJIE4C/ppsOx7oJCLbFauPhmEYhSRuVJIXiVJ2JRU1wU1EmgETgQHAg865DyJNegHzQtvzk/sWRa4zAhgB0K1bNxKJRH11ucmxdu1au58FxO5nYSm3+7ly5V5AJ1avXk0i8QngRWI4c+d+hXPbM3fuYhKJmQ3Yy9oUVRicc1uAvUSkE/AvEdnNOfd5qInEnRZznUeBRwEGDx7syikTstQpt8zSUsfuZ2Ept/vZoYMu27fv+F2/vYWwww79adUKevbcnuHDt2+YDqahQUYlOedWAQng6Mih+UDv0Pb2wMLi9MowDKOwxI1K8usipetKKuaopG5JSwERaQ38AJgWaTYK+HFydNIwYLVzbhGGYRhlSNyopHCMoVSDz8V0JW0HPJGMM1QAzznnRovIxQDOuYeBMcCxwExgPXB+EftnGIZRUOJGJZWDxVA0YXDOTQaGxOx/OLTugEuL1SfDMIz6JNuopFLNY7DMZ8MwjHoim8VQqq4kEwbDMIx6IlOMoZRdSSYMhmEY9UTcqCRzJRmGYTRhzJVkGIZhpJBtuKq5kgzDMJoY2RLczJVkGIbRxCjXBDcTBsMwjHqiXBPcTBgMwzDqiUwJbhZ8NgzDaILEWQwWfDYMw2jCxMUYLPhsGIbRhMklwc1cSYZhGE2IbBaDuZIMwzCaGNliDOZKMgzDaGLEjUqykhiGYRhNGBuVZBiGYaRgo5IMwzCMFDKNSjJXUpnw6quwcmVD98IwjMaCVVctccaNg8ceS398zRo45hg4/vji9ckwjMaNzcdQQsyeDUcfDUuWBPsOPRQuuggWLIg/Z9UqXb73Xr13zzCMJkK2BDezGIrIY4/Ba6/BHXcE+/wf4y9/CfatWQObN+u6FwbDMIxCYcHnEmLhQl2OHq3Ldev0jwAwcWLQrmNHOOUUXTdhMAyj0OSS4GaupCLxySe6nDEDVq+GadOCP4YPLvvtl17S5erVwfnffqvLhQvhhhtKU9ENwyh9siW4mSupSGzcCF98AYMH6/bs2TB1qq7vvDOsWKHr69ennhe2GGbP1uUhh8BttwXnG4Zh5EMmi8FcSYCI9BaRsSIyVUSmiMjlMW2Gi8hqEfk0+fpNtuuuXt0iZfubb6C6Go44QrdnzYJFi3R9zz0DYfBWgScsDAsWqHDMmqXbURGJY9Qo6NsXNmzI3tYwjKaBTe2ZnWrgCufczsAw4FIR2SWm3Tjn3F7J1++yXXTJksqUG+sf8HvvrctZs2D5cmjRAnr3VpHo0iU11hA+D2DZMnVDedauzf7h3nsPvv4avvoqdf/rr6cfCWUYRuMmblSSuZJCOOcWOec+Tq5/C0wFem39dWHp0mDbP+D79IGuXdUttGKFikHnznpsxQq49dbgnC1bUmMMS5fC4sXBdtS6iGPePF1+/XWwb/16OOooHTqbjbffhsmTs7czDKN8yMViaNLCEEZE+gFDgA9iDh8gIpNE5BUR2TWX6911F9xyi/6y9w/4Tp1gxx3hyy9VCDp3DoQB4IPQO69YoYKy3XZqWRRKGD77TJdh6yPMDTfAm2/q+qGHqqvLMIzGQ7kmuDUv9huKSDvgn8CvnHNrIoc/Bvo659aKyLHAi8DAmGuMAEbo1j7cfbeuvf76UpYsqQTaM23aeLbffnteemk7Bg5cS/PmjkWLFgCqNeE/xujRHzFjRj9atmxDhw7NmTx5BatXrwd2BODcc+Gppxax556refrpPjz22ARatkz9a86YsT/QmnHj5tKr19dMntyRpUsrgUF07FhFIjE+pX1NDdx223Buuw2OPPIbYFsAEokEX3zRgfXrm7Hvvpnrc3zxRQcmTerImWfOy9guH9auXUsikSjY9Zo6dj8LS7ndz+rqg4FmVFdvIZEYB8CXX7YH9mHKlM9YtGgbNm7sQSLxboP2sxbOuaK9gBbAa8D/5Nh+DtA1c5t9nOpx6mvpUudeeinYPukk5958s3Y7cO6uu3Q5bJhze+zh3IknOnfllfFtwbl33nHuxhudmzfPOeecq652rnlzPXb66c61a6fru++uy/79tV1NjXMjRzq3YIH2L+7aNTXB+pYtLiPNmmm7b7/N3C4fxo4dW7iLGXY/C0y53c/WrfV/tLIy2PfBB7pv9GjnLr/cuY4dG6ZvwASX5rlazFFJAvwFmOqcuydNm22T7RCRoaira3ku1+/RI3W7Y0d1zzRrpttRV1KYq67S5eLF0K1b4Erq0ydIjAtz773w+9/DJZcE51VX6/qrrwbBau9KWrRIH/XTpsFPfwq77ZbqqgoTDoKffHL6dvPnB+v+fQzDKC2siF52DgTOBQ4PDUc9VkQuFpGLk21OBT4XkUnAfcAZSWXLyjnnpG63aAHt28OuyShFly4qFpn4/e9ThWHbbaGyUo8NGwavvAK9esHzz+u+xYs1fuDjC127apmNMJ07Q1UV3H8//Oc/um/lSnjqqfg+TJkSrP/f/8G//lW7zYwZOsLKf6E+/TTz5zIMo2HINiqpyQefnXPvOOfEObeHC4ajjnHOPeycezjZ5gHn3K7OuT2dc8Occ1lL2nXvXsWf/6yJaHfdVfv4zjvrsnNnfdC3agU33RQc79JFl+edB2edpUHrmTN1mGnXrkEtpbPP1tFFBxwQnPvhhzBoEBx7rG6fdFLt9//JT3R5+eXw5JNq2bRrBw89pPt32AGOPDJo74PR++2ny+Uhe+nrr+Hjj/kupuJ5773UXySGYZQGmSyGUg4+l33mc6dOm7nwQmjZEg47rPbxvn11uWkTtG6tv97/+7+D4wMGpLbbYYfgWGVl4CLq2lWXp59e+z180typpwb7fv1rXR5zTLBv/HgYPhyOOy61mutrrwVJeK+/rsvHHlMLx7uSxo3TPu6zDzzySOr7P/mkWiRxjB2rSX+eKVM0O9wwjPrHpvYsAXr2rL1v33112a5dsK9Dh2C9f39d9uuny8su03yCf/4ztTqrtyxOOCH+vSsrU4Xpd7/TP/hBB8HA0Liq7t1TxcJft0cP2H57FYpmzVSwevRQYbjrLrj22vj3veYaGDIE/v731P1PPgnf+x4cfrgKEei1dtsNfvGL+GsZhlE4tmwJHvrmSmpAunevve/UU9VP/8tfBvvCAWUvCN5iaN0adt9dA7/hB7q3GCorNZv66ad1+/JkYY9mzdRNdeut8OKLuq+iQq8Xjid07qxi4WnePOiTdyn94AfQpo1+niVL4Oqr088T0bOnTi40fryKg5+MaORIeP99XZ80SZfLlunyrbfir2UYRuGYNk2X/fplTnDz4xBLiUYlDH4EUhgR+OEPNRgdx847a5uBtbIlUvHCAOpuOvNMdTN5C2LTJl1ef33tWEOnTsH6NtukuqvCeGE491xdeoshzFFHwd/+Flyza1etC1VTA2ecoZMRrV8P74aGRXfurIH1Aw/U7bj7ZBhGYfFld3y8MBqI9iUxwvtKhaInuNU3H32kI4uyceed6ms/6yzNON5++8ztvcsnTLNmml0NQZA6jm22CdY7d9YvxNVX1x4Ke/rp6kLydZ569NAYgadXLx0OC+oqeu01HXnlA+yed98NhAp0pJSPeYCargsXqrVRXa2jnKLXMAxj65gwAdq2Df63nNP/+ajFAPo/WUo/2BqdMPiYQjauvjpYz6UURZs28ft799ZlOKAdJTxM1udS3Hln7XYiGlz29OgRBLYhVXx8PKVFCxWt9u2D0h2+3Mett6qQjBuX+j7Tp6vIbN6slsTNN6vZ60uVG4ax9Xz2GeyxR/DAj7MY/DGzGMqMt99W/306mjXTUtstW6ZvE3Zjha2HbERjJuHRRPfcow/yI4/UL9gOOwSxhI8/1uUVV8DQoUEJ8ijz5gUiEp7DwjCMrWf9ev3RFnYX+ZgCpLqSSi0A3ahiDPXBwQcHmdHpqKwM/sDZSJd9HUd0lFXYndWpk45I8u/rR1eBCkOXLhoM32239NefNSu4pp+cyDCMwlBdrULgXcZRiyHqSkpHTQ3813/BO+/UX1+jmDAUmXwsBu+mArUMxoxJ3zYc0J47VyvFgib1pWP27OALOW2ajmI67LDhOc1YN306nHiiJgOCTlS0ww42UZFheKqrddRhVBjiLIaaGq24cNZZtas5f/MNPPdc+qHy9YEJQ5HJRxjCAfFbbsns6jnwwFRrxAsDaPwhjtmzdTgs6PSlf/yjrr/8spbZCE9QNGOGWjBXX63njBmj82Xvt59+0ceP10mKwmXHDaMps2WLCkN05FG64PPvfgfPPKNDzcP46geZBrgUGhOGIpMpFhElPEQ2m6CcfLJ+gbxrKCwMt98ef86sWYEwTJoUDK+76ipNmttrL53PAuCNNzQ7+667dC7s6dN1/6pVam34WerCxf1AR0f9+tepxQHD/UqXn2EY5U46iyFd8Dnd0FU/EZkJQyMk3a/2TISHs+ZqafjkubD1cOml8T7M11+Hzz/XEUpr16oLyjNwoA5zHTYMEgkVjS5dNLHvyy+11pPPpUgkdPgr1J7GdNQoHfl0zTWp+2tq4MYbNSfDMBoj6WIMcRZDdXUgDNFkN5+Y6svzFAMThiIxYwY5+e7Tka0yrOf883UZDYZXVOgX7OJkHduf/SzIdTjzzPD76M4//1lHLPXooQUEJ07UobSnnx7kbhx9tLq7rr5aLQqItxigdnB71arAr2oYjRFvMUQtgbDF4Ks3b9wYCEg6i6GYQ1pNGIpEjx6w0051Pz9d5naUE0+EJ57QDOwoXboEcYthw4JM686dtQjf3/8Ov/71F5x2mloe/ftrFvbChRpz2Htv/YVz1llBnx56KDUW8cEHajVcfLGWC/EF/N54A154IWjn/abelWUYjY1sweeKikAYqqqyu5IgNXG1PrE8hhJn3DjN5s4VEfjxj9Mfv/JKdUude67GIUaN0tFPfj6LRGIVV1wRtA8PmR00SJc/+YkGw08+Wes0HXWUZmGDXm/UqOCccI2q007TobR77hmYx2YxGI2VqMUQF2MIC0O6ZDf/vwL6Q6tPn/rrs8eEocQ56KDUontbS6tWwcxzRx2lQWTvGoqjV69g3Rcc3HHHwH8KWsbDC0OU8CRCLVtq8Pr88wP3lQmD0Vjxo5KiLqLwcNVWrXS9qipoFw0yh/9HFi4sjjCYK6mJM3Bg5uS8OGGA1LouF12kyzvuCNb9qKhPPlG31ezZmoE9aZJWj/Vf9uXLixtUM4xi4X88RbOb07mSvCCsW5d6naVLA9FYvbp+++wxi8HISNiVlK7Q4J57po6kOOMM/SIffrgm6/Ttq/GKXXcNigCGWb689pzdhlHueFdS69a6XVWly3SuJJ8cun596nWWL9f/oTlzaie/1RdmMRgZCZcMzzUAfvjhqdaFF5ddd40fi33rrcGvqQ0bgppPhlHOeGHwBTj9gz+dxeCPRy2GjRuDitHhgR71iQmDkZFoafBcCVsAu+6qy3R1m+6/X8uIg+Y87LUX/OlPdXtfwygVohaDtwSyWQxRYdi8OchjMovBKBkee0ynOs2HcJlyXwo905wPvgy5n2Do7rth5Uqd37rUKk8aRi744LP/X/DCEGcxbNyY3pW0eXOQsGoWg1EyXHCBDk2tK95iaNUqmJ1uv/00MW7ZMrjpJs2m3rhR59sGrbt0002aD3HTTfpP9sYbmqdhwWqj1KmpUQFo1qy2Kylfi6G6Wuesb9483mL45htNPo0ml2YiKj5RTBiMeqd5aIjDI4/A449r0b0779SkuwED9J/l/vvVSjj6aP2nevRRPefWWzVT+4gjtHDfhx9quzvuMJEwShP/vYxzJeUbY9i8WeN77drFC8Nf/qL5QQ89lHv/jjsu83ETBqPeWLy49pzVrVtrglx4iOyAAbq86iq1Kn71K93euBH+5390yOxzzwXt//1vuOEGuO46GD26Xj+CYdSJsDDkajH4UUtxrqQWLbTeWpwrKZwXkY6XXoIHHgi2/Xulw4TBqDe6d689C10cAwcG62+8oXkOnksuUTeW/6XUurUKw8qVuj1tWuq1nNOyxSNH1i5GZhjFIheLIZrgls1iCE/fCzBliv5gCk/847nvvtQpi088EX7xi2DbhMEoecKVYA88UP2yo0frQ3/HHVMnIbr0UnVD+QmCXnpJq8B6Xn5ZYyIXXGDDXo2Gww+YyBZ8rqjQh35dXEn7768zu/kh4GGL4csvMxftLBlhEJHeIjJWRKaKyBQRuTymjYjIfSIyU0Qmi8jexeqf0XCIwIgROm+D/3Ifd1wwMVFYGI47Tv/pJkzQ7ffe01FPo0apZXHzzUFbK9BnNBTeYsgWfAZ1J6VLcHMu1WJ4/XWtOQaBgHirOWwxbNyo56WbwyE8f3wcxcx8rgaucM59LCLtgYki8m/n3BehNscAA5Ov/YGHkkujkfPII+mPheezPuAA/WfbsgUuvFBN8YcegpNOCtr8+Mfw17/qiKfp0zVjOzx81jDqm1yDz1BbGNatC3IgvOXhLQaA55/XZd++OoeKt4zDFoN/8K9fH1+yv2QsBufcIufcx8n1b4GpQK9Is5OAvzplPNBJRLbDaNJ4YTjtNBWCN9/U2MOvf60BtaFDU9uffbYuFy9Wq+Oww2pfc+HC/Ib3GUY+hIWhRQtd5mIxdO+ux7/6So/5X/wtWgQC4/F1zGbMSL0epApDGC9K2YShQWoliUg/YAjwQeRQL2BeaHt+ct+iyPkjgBEA3bp1I5FI1FdXmxxr164tyfv5xBNt6NGjikRC/6tOO00L882eDb16DQC0kNM222yiouJ9RA7hlVcWA9vy4YcwZsw4mjev4brr9uDMM7/mqqv2pLJyC6+8Mq5e+12q97NcKZf7uWhRJTCMGTOmkkgspmXLg5g+fRGJxCymTdsOGMz48e8xa9YmnBvKnDlr2bixO/37L2fJki4899xnHHjgctatawYczNy5s/jyy86ApkC/+WaC5cv3Bjp8957Tp88lkVBFWbBgN6Ar//nPeHr1qgKGJ897i+bNHevXH5z5AzjnivoC2gETgZNjjr0MHBTafhPYJ9P1Bg0a5IzCMXbs2IbuQt7cd59z4NzZZzv3wQe6r3Nn5wYN0v3+tc02qdtQ/30rx/tZypTL/ZwxQ79fTz6p2z16ODdihK4//LAeW7BAt3ff3bmjjtJ9N9ygy9tv12PLlun2vfc6N2BA8L1dvty5wYNTv8tXXBG8v7/e5Mm67dusWeNcTY1zIs4BE1ya52pRLQYRaQH8E3jKOfdCTJP5QO/Q9vbAwmL0zShfLrxQA81XXQUdkj+gunTR+AKoyb1gQTDE1dO8uWZa77573WtCGUYc4eAzaIwrWkQv7EpasULXe/TQmNgXychr2JU0dGgwGm/58to5Df76kOpKCg/brqpSd2y2odzFHJUkwF+Aqc65e9I0GwX8ODk6aRiw2jm3KE1bwwDU93rLLYEoAHTtGhxLJLQUuGf58iBres894d57i9lboykQjjGAfg8zBZ99ImjHjhoX8z9q/HVatNB52P/4R91evrx2FnQ4bhAWhvAIpHAiXSaKmcdwIHAucLiIfJp8HSsiF4tIcop6xgCzgZnAn4FLitg/oxHRpYsud9xRM6ufeUaF4/TTNW9i222DtmXgsjbKjKgwtGkTX10V9Bd8WBi23TYYah22GNq0CWZzzGYxhLOow3kRVVXZh6pCEYPPzrl3gIwGu/rCuLQ4PTIaM/6fITwt6rJlgWkfLgu+caP+I37ve3DPPZolahhbQ5zFEDcfA6jF4B/WHTvqyKQ4YYAgGXTBAhWYsODEWQzr1tUWhlKzGAyjaPikuOuuC/a1aBH8M4aFYd48+M1vYNYsLaVhGFtLOPMZMlsMvl4SBMKwbp22jwqDt4TnzNFleFbFdDGGsGWxYYMJg9GEufdeHd+dbuL0sDB88YVWfIXa5QgMoy7kEnwOWwyeDh2C2dqWLq0tDJ066Xlz5+p279BQHX/9OXNSs6jrYjHYnM9Go6Rdu6Bqaxz+n8+zcaOa6WPHwosv6nSkffqkxiIMI1cyBZ+jFoMfKAGBxQAqDF5EvDBUVOhsbnHCUFWlohCuFFBXYTCLwWiStGihBfnCZbt//nN1AfzoR3DooXDjjQ3XP6O8yRR8jloM24VqO3TsGPxoWbKktsXg2/is/agr6ZtvUvthFoNh5ImvT//xx1qCIDqsb/z4humXUf5kCj5HLYawMLRoEVgMS5ZA27bBfk+nTkF9pKjF0DzyRI/GGHIdlWQWg9HkGTJEK7Oecgr87/8G+7/4oniTrxuNi7gYQ9x8DJAqDJDqSvIWQ/iB37FjENyOWgzRGQ1tVJJhbCUtW8IvfwmDBum2c/DRRw3bJ6M8iRuVtGmT7s/kSgK1Eior07uSOnUK1vv2DdbjHvoWYzCMAvGLX8CVV+o/4O236z/y1VfDJ5/o8Y0btYLrnXcGbgGjaTJkCJxwQu39ca4k0F/1mVxJfn+XLprEli7G4NlxRzjrLNhnH712eMgqqDCEhWD0aL1uNizGYBgRLrtMlz176pzTY8bAXXfBn/6k/tovv4Snn9bX8cfDrrs2bH+NhuPTT/UVJS74DKm1i7zFsM02tc/v0kXrJ4VLYni8xdChg1oWTz0F11+vcYdswvDSS/rKhlkMhpGGH/1Il34SoXXr4O9/Tx35scgqeRkxpBOGOIshroBj587ZLYbwlLitW+t7RstkRGsl5YpZDIaRhr59tTJr+BfWGWfoXLseX+PGMMJEg8/hWdyiFgPAu+8GWc2g61OnZo4xhBPj/PV9lVaPF4bwqKhc2GqLIVlK2zAaHSJwyCHB9rXX6vDAD0LTS5kwGHHEBZ9BH9RRiwG0Tpef4xxysxhatgz2+eunE4ZWrfLrf17CICK/FJFTQtt/ATaIyJciMjjDqYZRllx1lS67dNFA9Hnn6XazZvqPacJgxJEp+BwdrhqHjzFkshjihCEcWK6sVPdnnDBUZHny52sx/BJYCiAihwCnA2cBnwJ353ktwyh5hgyB116DV1/VbT+UdcsWrbdkwmDEkSn4HGcxROncWUXBTy4VFgY/70g2Ydhmm/QWQ3hkUxz5CkMvYE5y/QTgH86554DfAsPyvJZhlAVHHgn77qvrYXPfhKFpk2kWtEzB51wtBgi+X2Fh8OdncyV16hQIQ2UlvPEGDByox8KJcXHkKwxrAF9+7Ah0TmaAzUBl7BmG0YjwFgNohqoJQ9Ml02ifbMHnbK4cP+LIj4ALC4N/uJ9zTrAvnSvJD1dt1Qq+/32NZUDhheF14M/J2MIA4JXk/l2Br/K8lmGUHd4EP/54tRg++UTzGzxjx8Lhh2v9JaNxkymDOFvwOdsc41GLIVwSo29fvc4FFwT74oSheXMVrw0bAleSj08UWhguBd4FugKnOue84bI38Eye1zKMsmTNGnjhBTjzTN2+7Tb9FXjbbSoKY8fCs882bB+N+icXiyFd8DlXiyHOlRS+nifOleTFYOXKYN0X5ctWTj6vPAbn3BrgFzH7b8rnOoZRzrRvr8sjjtAJ2i+6SMsaLF6seQ7TpsGHHzZsH42688QT+ov85z/P3C5sMTiXagVkCz5nsxj8d2zFCm3rXVLp8Nf/9lu1as8+W92e77yjwuCrsPpcBn/9dOQ7XHWX8LBUETlCRJ4UketEJEvXDaPxcdRRuly8WGvWPPaYzjM9cWLgTjDKi/POg0suyd4uLAzRqqbpYgx+as3KLBHZsDBErYU4vCUA6mp68MHAbbRyZfB+vqBeuH0c+bqS/gIMARCR7YH/AzqjLqbf53ktwyh7eveG55+H6dO1Zk3btrDfflqaYNq0hu6dUZ+EhSHqVqquVneRtwwqKtSds369ikPUFRSlXbvgurkIg7cYILh22L3kXUk+D+eYYzJfL19h2BnwYbXTgA+cc8cC5wJn5nktw2gUnHJKMFIEYOhQXZo7qXETFoPw+sqVWnQxWnnXz8mQizC0aBE8zPMVBm8d+H01NcG1DjhA3V7h6T/jyFcYmgGbkuvfB8Yk12cBPWLPMIwmxqBBmoQ0ciTMnRv8xy5bVrvImVFa+EzjXAhbDJs2BetjxqRue3y9olyEAQJ3Ui7C0KJF7UB3WCzqtSQG8DnwcxE5GBWGZD4ovYBleV7LMBolFRWa4/DOO3DFFXt+t79bt2AcuVGa5DJXgSedK8lbCkOGpLbPx2KAwJ2ULR4Rvj40jDBcA1wEJIBnnHOfJfefCJjhbBhJzj9flytXtsQ5mD1btz/7LL9fpUbhWb0aPv88/tjSpblfJ50w+MDzCy+ktm/Tpm4WQ9euufXHC0HUlRTelyt5CYNz7m0087mrc+6noUOPABkHd4nISBFZIiKxfxIRGS4iq0Xk0+TrN/n0zTBKieuug8svh2bNHI8/rjNted57r8G6ZaDDjHffPf5YWBiyjSpLF2OIK3wHKgZ1sRjyFYaGsBhwzm1BK6ruJiK7ikilc26Oc25JllMfB47O0macc26v5Ot3+fbNMEoFERgwADZvrvhuoh/P++83TJ8Mxc/jHbXcpkzRshGesEVQVVU7mJzOYkgnDGFXUi6/4PO1GHz/+vUL3s9T32W3m4vIXcBKYBLwGbBSRP6QbV6GpLWxIlMbw2hM+LIDEyfq8mc/0zIaM2c2XJ+MgPXrU7ejZUx8Mpj/hX/99anH8xUGH3yuqqofV9KcObrcYw9dhnMV6tti+ANwDnAxMAgYiLqQzgVuz/NacRwgIpNE5BURsZl0jbLGC0N1tbqVHn5YrQgThtLAJ3t5vOvG44XBjyR77LHU4/m6kvINPnurIldh8HhhCL9HvsKQ79SeZwE/dc6NCe2bJSJLgceAK/O8XpiPgb7OubUicizwIio8tRCREcAIgG7dupFIJLbibY0wa9eutftZIFasaAnoMKRWraaRSHxDu3Y7MXHiNiQSjcOf9O67Xais3MI++6wqyvsV5vs5HICxYz+gV69gvstPP+0G7Mq++65gwoTOJBIf0Lv3BlatagEcyObNm0kk3v2u/ZQpvQENHk2YMJlmzdQh8uWXfYAdeO+9t2jZMqjNvWbNTqxY0YlNmypYuXIZicT0jL2cP38XoDsrVkwnkViY8+f66qsEc+em7ps3bwaJxIIcrpHEOZfzC9gADI7ZvxOwIYfz+wGf5/hec9Agd8Z2gwYNckbhGDt2bEN3odGwZYtzbdpsdh06OLdihe77/e+dA+fWrQva1dQ0TP8Kwd57O3fkkcV7v0J8PzXFy7lPP03d/+STuv+OO1KPL1ig2507p7a/5ZbgWv/6V7D/t7/VfVu2pLb/2c+c697dufbtnfvVr7L388QT9TrPPpvb57rjDufOPDN1X8+eeo1HHqndHpjg0jxX83UlTUJncYtyefJYnRGRbUU0gVxEhqJurjxGFRtGaVFRAU8/PZ5ly3Q2LVBXEsDbb8Ott8Kdd2q7WbNqBzfLgaqqzOWnS5lojMEPM/W+fe9KiktWg8wxhoqK2hVU83Ul+etnK3jnueYaePrp1H2+eF59u5KuBsaIyBHA+4ADDgB6Ahmrb4jIM6hd01VE5gM3AS0AnHMPA6eiyXPVqGVyRlLVDKNs6dixOsXXfPDBuozWqhkwQIPTl16qmbPXXFO8Pm4NVVWZy0+XGuEnSjTG4GMDfupMLwx+f/RplCnGEJet3KZNEK/IRRj8NfPNQQjTuzd88EH2Mt9R8i27/baIDEKL5u0ECPAPtDTGr4B3MpybsZaSc+4B4IF8+mMY5UbPnvpPGmcdPPII3w1tPfvs7JOplAIbN5aXxfDtt8F61GLIVxgyWQxxwhAWg1yE4bLL4K230udc5IL/Di1alN95dcljWOicu8E5d4pz7mTn3I3AOuCUfK9lGE2RJ5+Eww6Dk07S7d69YVhkxvTx44vfr7pQbsKwalWwns5iiLqS0mWqb9gQCECuFoMnFyvg1FNVjLp1y942HYceqsuePfM7L29hMAxj6zjzTPjPf4L5ow84ICiH7CmXJLhyizGsXBmsp4sxRC0GH2OIWgwbNgRzHtSHxVAIfvhDzc84++z8zjNhMIwGwpc+7tMHBg8O9n/veyoML7wQPJxKlXKzGMLCkG+MIUq+whA3Z0IxGDIk+4xxUUwYDKOB8KUL+vSBnXbS9eHD1YJ4/32d5+Guuxqqd9mpqdGHYDkJw+rVwXq+MYYoGzboNJpQ2sJQF3IKPovIqCxNOhSgL4bRpNhrLx3Guv/+Ojn7Sy/BgQeqm8lTyiN+fN9KuY9RwmIQtRiyDVeNcyW1aaNDQcPiWEqupLqS66ikbPkEy4GvtrIvhtGk2G47nXbRc/zxugwHotONoS8FvCBUVelDM193RUMQFoY4i6FZsyAwnG1U0vr10Lmzto8KQ/OYJ2ujsxicc+fXd0cMw1B69YJjj9V8hj/+UauBlmKVkugMZvkmURWCiy/Wh/Ntt+XW3otBs2bxMYYWLYL5mXNxJbVuHS8M5W4xWIzBMEqQl1+G/fbT9bfeSh1/XyqEXUgNFWd45BG4PY/ynV4YunbVIcHhYHT4gV5Zmd1iyFcYwhZDqeeomDAYRonSIzSL+pQpDdePdETnKygHvDDU1MCkSWpxeKqrAxdQZWUgfOnceVsjDPnmFRQbEwbDKFG6dAnWP/ssfbuGohQshnzxD3M/U1s4xhN+oLdqFXy+fF1J1dWZhWFrMpmLhQmDYZQo4RyGUheGfEcmzZsHX39d2P7kwvr1+oDeay/dDk+5mk0Y1q7VALsfvZSvxdCzJzz6aOqos1LFhMEwSpTwmPsnnoDHH4d771U3yB13ZJ64fv58eOON+u3f1riS+vSBvn0L259cWL9eH+bvv68WWVjQwq6k8BDUqCtp9WqNN/iZ2HIVBoCLLsp/4p2GIN/qqoZhFInrrtMA6bPPajXW85NjAysq9NikSfDMM/HnHnCAisOWLflX1syVcnQleYuhslJzSNIlpoVjDFFX0sqVgVvIC0O6IHa5YhaDYZQohx6qRd+OPhrGjQtq7T+QrEH8xhtBNdYo8+frMvzAKjSFCD6vKNAs8Fu2xO+/4AJ4881g2wsDZP6lnynGsGpV4ObL12IoF0wYDKMMOOggLb53yCEwY4buW7ZMR9XMnp3+vG++qft7Pvus+tQXL44/XgiLoUsXrTa7tUST1UD7NHIkvPZasM9nK0Pqwx/SC0PUlbRypQmDYRglxMkn19732GPpZ3/LJgw1NSowcTz4oC6nTYs/XihXUl1jIeG8Aj8BThhfYjs6B0M6YYgOV/WfKc6VZMJgGEbJECcMt9+umbyTJ9c+lk0Ybr9d6/0vjJlr3j9405W6SDdRTZh167JPWVrXh2jYfRTNYobAjRYWjUzCkKsryYTBMIySondvuPHGoNzzww8H4+J9Elz4l3Q2YXjxRV3Om1f7mL9ONHj96KM6XWQ2i8E5aNcOfvzjzH2o60M07OKJsxi8MORqMZgwBJgwGEaZccstMGECnHYanHNOMC7eD18ND3PNJgzNmukyLnjrhcGP2/dceaUGvbMFn/2D9amnapeTCFMIYcjHYvB1iuIS0+JcSRZjMAyjLNhxR3juOWjbVovIVVQEwrBkSdDuj3+EDz9Mfx1vDcRNCORdQNGH3rff6sMxm8UQ/qX+6afp+9AYLIZNm1Sov/99EwbDMEqAigod3eOFIZr4NmxYenHwwrBmTe1j/ld++KHvH7a5CEP4YT12bPr+17cwhI/VZVRSLsIA8Pzzar2ZMBiGURJ07x5YCl4YRo3SDF/n0o/88a6kXIXB5x2sWKH7M1kc4V/q772Xvu91TcDL1ZXk+1FTk58w5ONKCtMYhMEynw2jEdCtG7z6qo7Z93GFffbR+jwDB2pMIg4vDHFlvb0rKfzQ98LgLYY2bXQZl0fgf6l37QrvvhtM5hONWdR1BrhsFoMfruqP+Qd9rsNVN27UPkcthuXLAyHt0KG2MKQroldOmMVgGI2Abt30AX700SoMImpFAOy7b3phyNeVFBYGH8ht1y7+F7t/IB92mPbJWzTRX+CFEIZsFkPYamrXTpfhshdQ22Lw+6LCsGBBIDrbbFNbGCB+BrdywoTBMBoBLVsG6zNmqFD4h9OwYToc9a9/rX2ef+i99VbtDOpMwrBunf5y7tBBA+BxD2Zvhey0ky69JVMoYQg/sP2DOowXhupqfY+LL4bddoPTT9f9meZq9sJQVVVbGJYu1c/SsqWKQpwwmMVgGEaDE36ov/UWbLttsH3RRfC978G119Y+zz/QX389tQQ1BMIQ50oCLZudSRi8FeKv60trRIUhU9b0rFnpy1SHr7M8Zlb6cJ2o6dNh0SIYMSKobtqqlbrLvGsr6koCFZTw+3TrpsspUzSXRMSEYasQkZEiskREPk9zXETkPhGZKSKTRWTvYvXNMMqde+6BXXbR9QULUoWhdWv40Y/0wRgtWhd9oN99tz4M//GPIGEuzmIAmDsX2revLQxbtsAJJ8BPf6rbAwbo0gtD1ELIZDHsvLMOAY0j/MCOK0EeFoa339blkCHBPm8VhEcfRS2GjRtTLQYvcp99FiQZRmMmYMKQD48DR2c4fgwwMPkaATxUhD4ZRqNg2DD45JMgmLzddqnHvWh88UXq/mjQ9sorVWROPz14IKYThsWL4y2GV16B0aODbS8MdXEl+T7EtfHXadYsvt7TypVBPOHtt/XX/Z57BsfrIgz+Pi5erPEFCO5h+J6bMOSIc+5tIFOR3ZOAvzplPNBJRLbL0N4wjBAtWwYPvvB80QC77qrLqDDEuYB8mQxPOlcSqMXQrl2qwETLa3Tvru6WdK6kXGIMixbV3uev07Nneouhd29d/+gj6NdP++vJRRiqqvR9DjgArr8e7rorON9bDGeeCTffDH/4Q3AsHPMpR0opdt4LCH+l5if31fpKiMgI1KqgW7duJBKJYvSvSbB27Vq7nwWk2Pfz4IN78vHHg5g9ex6JxKzv9tfUQGXlwbz22iIGDZoZ6t8hRH8fRpPhvvpqIYnEdACmTt2TTp3asmpVy+T5C1m/vjlLl7YlkfgIgAkT+gL9vzv/rbcSdOw4jMmTV5FITGPmzLbAft8d/+ablSQSk9J8ouEA9O8PI0bM4oQTgvv58cddgd1o3341Cxe2JpEIkiWqq4V16w6lbdsVQGcWLKihX791JBITv2szZ862wE4kEuPZbrsqNm06mEWLFpBIzGbGjC7A7rz77gSWLx9Ehw6bOeKIz5g8GTp0OJA1a1qwadMSEglV2kMOgQ8/7AzsAcD8+Z+TSKQpW1sOOOeK9gL6AZ+nOfYycFBo+01gn2zXHDRokDMKx9ixYxu6C42KYt/PzZudu/lm5+bNq31sv/2c+8EPgu1Nm5zTEHPm1znnBOf07+/ccccFx666yrmf/MS5Pn2CNpdfnnq+c87tv79zRxyh6x9+mHp8r72cW7Ys/vO0ahW0Gzgw9X4++6zuP/lk5yoqnNuyJThvyRI9duGFwfkHHZR67aef1v1Tp+p28+bOXXedrr/yih577z3nhgxx7oQTgvMOOkiP/exnqdd7663gvcrh3wiY4NI8V0tpVNJ8oHdoe3sgphiwYRjpaN4cfvMb2H772sd22UUDyr5gnncjHXaYJsGlw7uStmxRN9Huuwejd+KCz8uWqXsnTI8eQYzBu278aJ5PP9WRQiNH1nZDdegQrEev6V1JvXqpRRQONvv13qEnio83eMKuJOdSE9PCrqQ1a/QzerxbzruSPOE2XbpQ1pSSMIwCfpwcnTQMWO2ci/EsGoZRF3bZRX31zZtr7SI/E9wZZ8C996Y/zwefFy3Sh2ffvsFDMC74vGyZPqzDdOsWBIj9A/3VV+GUU4I2F1wAffqkZlGHH+Y+Yc8TjjH49/V4YejTJ9gXfnBDqjB4sYwOV92wQYfl9usXnLfDDsGxMD6jGkwYckZEngHeBwaLyHwRuUBELhaRi5NNxgCzgZnAn4FLitU3w2gK+F+6AIcfDkOH6nrbtsHY/s6da5/nhWHuXF2GhcEHn6uqgofr8uXB9Txduuh+54IHeqtWwcM5zPjxwXo4UB0NUvvRQl6EwgHoOIshkzD4a0Uthjlz9FhYGLw1Fp3cqDEJQ9GCz865M7Mcd8ClReqOYTQ5wsIQZscdgwdZixbwwx+mjkzyv4zjhMFbDKBWw4IFMHWqZjt/+mkwfLZLF33Ir1sXPOx95nCU8BzTYevBJ8ytWKEWyIEH6rZPOguX9fDC0LOnlv2oqantSgonsUWFwc/Z4EdxhYXhhBPgiCN0JFKYsPDECV45UUquJMMw6pF+/eCf/9QRNKC/6q+4QnMg/MP1/PPhX/+CSaFBQu+9B7/6leYnVFRkFoZddtFl1646dHa33fSYF57ly1OFIe4BGhWG889XEfATEE2dqg/6ceN0u2PHoK3HC8M22wSCkM5iqKoK3FD+Wt5t9cEHuuzbNzivfXvNFPc5DZ6wxVDulNJwVcMw6pmTT4Yf/EB//e+6a1BEr0MHfZj6YO8ee+iv/5/+VCu2/u//6v7rr9cHrH/ItmwZrIcnCIqW0g4Lg3cJtWxZOxGsefNAGLZs0bb9++vy6afhn//sxfDhqef4IHDY5+9rJ3XqpA/yaAAZgof/7NmBxbCHjjalc2ftny8+GBaGdMRZP+WKCYNhNDE6dAjmiQ4THWXTs2eQ09CjB1x1lVoOEPj1a2qCB+5ee6W+R5h0FkP4Yd6vnz6gvTB4C6BNm+CX/AMPDKw1Dal/r6jF4AvceYsh6krq21dHY736Kuy/v2ZGe3ebiH7+OXP0s3vXUibqOq9EKdKIPophGIXGP5DnzFG3k48ZPPQQ/P73MHx4bRfK3XfD1Ven7vPB6DPOgInJHLNWrYKM6XvuUfdVjx6ZhQF0Dukw/v3DIrNkSWAR+GznqMUAcMwxOkLr/fdVJMJt/GinnXeufV5jx4TBMIy0/Oc/+oq6STp3hhtu0F/JPj7h+clPav/C9hbDihXw4IO63rJlIAw77KC//MP5Dl4Y2rZNFYY1a1JHG3lhCFsMYWFIF2MAda1VVam7bO9I2U5f+ygaS8jEZZfFlzcvN8yVZBhGWvr311cm9tlH/fR+fH/ckNe4fWFh8A/vHj00yD1zZuqMa2FhgKA+UlWVXkektsXgK8x6iyHqSoIgEA+Bm8zjXUP5WAz3359721LGLAbDMLaa/v3h+OPVHSNS+3jcjGYtWwb7fZzAP8zPOivVlRS8jypJz57w1Vc6akhE29TFYhCBMWO0ON7++6ce8wHscJJcU8GEwTCMgjBqlE6Ik4nwr+8WLbQMxk03qdUBcGkyk2n+/FRXkh9+OnToCnbYQYfBbrttkKTXpk1gMTinwuArzGaKMYDGGa68svb+ESP8e2b+TI0RcyUZhlEQ4iyFMCtXasB5zz21HEdFhWYR//a3QZvtt9cZ5156KSiz0aYNXHihzkx3+unzGTmyT638h9atAyFZs0ZHPkUthjhXUiZOPTWYxa6pYRaDYRhFoVMnfYBPmKCTCqWjY0dNZgu7krp31wBx586baNMmGB3lCVsMflRTLq4kIx4TBsMwikqHDqk5D3HHN2wIah/l8kAPWww+0S6X4apGPOZKMgyjpPCB6Fde0fyHXIK/YYvhzTd16YebnnCCVoaNzmpnpMeEwTCMksIPTR09Gk47LbeM4tat4Y03tCDg7Nl6nq/TNHiwJt0ZuWOuJMMwSopwOY3DD8/tHD+kdfZsrXz6t79lD4Yb6TFhMAyjpAgLQ67JZV4YTjhBXVDlXva6oTFhMAyjpAhnOedS1RSCEhyDB9cesWTkjwmDYRglRdhiiE4Rmg4/IilcQ8moOyYMhmGUFGGLITpfQzqWL9eln3bT2DpMGAzDKCmicznkgp+BzYShMJT9cNV58+YxPDqlk1FnVq1aRafojC1GnbH7mT9ahiIBUOt/O939nDXrT8Au/PKXp1NZuaTWcSM/yl4YDMNoXIhA69Zf06PH6zmfs8suv2PZsoNo1cpEoRCIK/MqUYMHD3ZffvllQ3ej0ZBIJMwCKyB2PwuL3c/CISITnXP7xh2zGINhGIaRggmDYRiGkYIJg2EYhpFCUYVBRI4WkS9FZKaIXBtzfLiIrBaRT5Ov3xSzf4ZhGEYRRyWJSDPgQeAIYD7wkYiMcs59EWk6zjl3fLH6ZRiGYaRSTIthKDDTOTfbObcJeBY4qYjvbxiGYeRAMYWhFzAvtD0/uS/KASIySUReEZFdi9M1wzAMw1PMBLe46ujRJIqPgb7OubUicizwIjCw1oVERgAjALp160YikShsT5swa9eutftZQOx+Fha7n8WhmMIwHwjXPtweWBhu4JxbE1ofIyJ/EpGuzrllkXaPAo+CJrhZwkvhsASiwmL3s7DY/SwOxXQlfQQMFJH+ItISOAMYFW4gItuK6LxLIjI02b/lReyjYRhGk6doFoNzrlpELgNeA5oBI51zU0Tk4uTxh4FTgZ+LSDWwATjDlXvNDsMwjDKjqEX0nHNjgDGRfQ+H1h8AHihmnwzDMIxULPPZMAzDSMGEwTAMw0jBhMEwDMNIwYTBMAzDSMGEwTAMw0jBhMEwDMNIwYTBMAzDSMGEwTAMw0jBhMEwDMNIwYTBMAzDSMGEwTAMw0jBhMEwDMNIwYTBMAzDSMGEwTAMw0jBhMEwDMNIwYTBMAzDSMGEwTAMw0jBhMEwDMNIwYTBMAzDSMGEwTAMw0jBhMEwDMNIwYTBMAzDSMGEwTAMw0jBhMEwDMNIwYTBMAzDSMGEwTAMw0ihqMIgIkeLyJciMlNEro05LiJyX/L4ZBHZu5j9MwzDMIooDCLSDHgQOAbYBThTRHaJNDsGGJh8jQAeKlb/DMMwDKWYFsNQYKZzbrZzbhPwLHBSpM1JwF+dMh7oJCLbFbGPhmEYTZ7mRXyvXsC80PZ8YP8c2vQCFoUbicgI1KIA2Cginxe2q7XoCKwuwrnZ2qY7ns/+6L7odldgWdaebh12PwuL3c/CUoz7mUu7+r6ffdO+s3OuKC/gNOCx0Pa5wP2RNi8DB4W23wT2yXLdCUXo+6PFODdb23TH89kf3RezbffT7qfdz3o+N5d2xbif6V7FdCXNB3qHtrcHFtahTUPwUpHOzdY23fF89kf3bc1nqyt2PwuL3c/CUoz7mUu7BrufklSRekdEmgPTge8DC4CPgLOcc1NCbY4DLgOORd1M9znnhma57gTn3L711vEmht3PwmL3s7DY/SwORYsxOOeqReQy4DWgGTDSOTdFRC5OHn8YGIOKwkxgPXB+Dpd+tJ663FSx+1lY7H4WFrufRaBoFoNhGIZRHljms2EYhpGCCYNhGIaRggmDYRiGkUKjFwYRaSsiE0Xk+IbuS7kjIjuLyMMi8ryI/Lyh+1PuiMgPReTPIvJ/InJkQ/en3BGRHUTkLyLyfEP3pdwpWWEQkZEisiSa1ZytEF8M1wDP1U8vy4dC3E/n3FTn3MXA6UCTHjJYoPv5onPuIuA84L/qsbslT4Hu52zn3AX129OmQcmOShKRQ4C1aO2k3ZL7mqG5EEegyXAfAWeiw19vj1zip8AeaAp9JbDMOTe6OL0vPQpxP51zS0TkROBa4AHn3NPF6n+pUaj7mTzvbuAp59zHRep+yVHg+/m8c+7UYvW9MVLMWkl54Zx7W0T6RXZ/V4gPQESeBU5yzt0O1HIVichhQFu0musGERnjnKup356XJoW4n8nrjAJGicjLQJMVhgJ9PwW4A3ilKYsCFO77aRSGkhWGNORSiO87nHM3AIjIeajF0CRFIQN53U8RGQ6cDLRCkxGNVPK6n8AvgB8AHUVkQDLJ0wjI9/vZBbgVGCIi1yUFxKgD5SYMErMvqy/MOfd44bvSKMjrfjrnEkCivjrTCMj3ft4H3Fd/3Sl78r2fy4GL6687TYeSDT6noVSL7JUrdj8Li93PwmL3s4EoN2H4CBgoIv1FpCVwBjCqgftUztj9LCx2PwuL3c8GomSFQUSeAd4HBovIfBG5wDlXjVZffQ2YCjwXrs5qpMfuZ2Gx+1lY7H6WFiU7XNUwDMNoGErWYjAMwzAaBhMGwzAMIwUTBsMwDCMFEwbDMAwjBRMGwzAMIwUTBsMwDCMFEwbD2EpE5LfRctGGUc5YHoNRFojI40BX51zJVdUUkXZAq2StnpJERBxwmnPOJrExsmIWg2GkIVmGISvOubUNIQoiUpGcs8AwCooJg9EoEJFdRORlEfk2ORPYMyKybej4fiLyuogsE5E1IvKOiBwQuYYTkUtF5AURWQfc5t1EInKGiMxKXv9FEekaOi/FlSQij4vIaBG5XEQWiMhKEfl/ItIm1KatiPxVRNaKyGIRuS55zuMZPuN5yfbHJt9vE7Bzts8mInOSq/9IfsY5oWMniE59WyUiX4nIrbkKotF4MWEwyh4R2Q54G/gcndzlB0A7dEIh/x1vD/wNODjZ5lNgTPgBn+QmdK6J3YEHk/v6oVNv/gg4EhiC1v3PxMHAbsm++HMvDx2/Gzg0uf9wYM/kOdmoBG4EfoZOQDU3h8+2X3J5EbCd3xaRo4CngAeAXdFZD08FbsuhH0ZjxjlnL3uV/At4HBid5tjvgDcj+7ZBa/cPTXOOAIuAc0L7HHB/pN1vgSqgY2jfDejMYuE2n0f6Og9oHtr3Z+CN5Ho79Nf+GaHjbYGVwOMZ7sF5yT7uk+Vepftsp0bavQ38OrLvh+gUm9LQf3N7NdzLLAajMbAPcEjSzbJWRNYSzPy1I4CIdBeRR0RkuoisBr4FugN9IteaEHP9uc651aHthclzM/GF0+qgcefsCLQAPvQHnXPrUIsnG9WoRfAdeXy2KPsAN0Tu29OoSG2b+VSjMVNuM7gZRhwVwMvAlTHHFieXTwA9gP8G5gAbgTeBqD99Xcw1Nke2HdndsJnOkdC+fNnonNsS2ZfrZ4tSAdwM/CPm2NI69M1oJJgwGI2Bj4HT0V/20Qey5yDgl865lwFEpAfqb28IZqLCMRT4KtmfNmhMYlYdrpfLZ9sMREcwfQzs5JybWYf3NBoxJgxGOdFBRPaK7FuFBokvAv4uIneiv3Z3QMXiCufct8B04BwR+QB1lfwB9fMXHefcWhEZCdwpIsvQeMCN6C/4ulgRuXy2OcD3ReQt1OpYicZmRovIXOA51E21GxqXuboO/TAaCRZjMMqJg4FPIq8/OucWAgcCNcCrwBRULDYmX6AjbtoBE4FngZHow7KhuBIYh05VORaYjMY3qupwrVw+2xXAYWjs5RMA59xrwHHJ/R8mX9cCX9ehD0YjwjKfDaMEEJFW6NDTu5xzdzd0f4ymjbmSDKMBEJEhwM7or/T2wDXJ5d8bsl+GASYMhtGQ/A8wmGAI6iHOufkN2iPDwFxJhmEYRgQLPhuGYRgpmDAYhmEYKZgwGIZhGCmYMBiGYRgpmDAYhmEYKZgwGIZhGCn8f6VhC2DS56ADAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"batch_size = 128\n",
"rates, losses = find_learning_rate(model, X_train, y_train, epochs=1,\n",
" batch_size=batch_size)\n",
"plot_lr_vs_loss(rates, losses)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Looks like the max learning rate to use for 1cycle is around 10–1."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `OneCycleScheduler` custom callback updates the learning rate at the beginning of each batch. It applies the logic described in the book: increase the learning rate linearly during about half of training, then reduce it linearly back to the initial learning rate, and lastly reduce it down to close to zero linearly for the very last part of training."
]
},
{
"cell_type": "code",
"execution_count": 95,
"metadata": {},
"outputs": [],
"source": [
"class OneCycleScheduler(tf.keras.callbacks.Callback):\n",
" def __init__(self, iterations, max_lr=1e-3, start_lr=None,\n",
" last_iterations=None, last_lr=None):\n",
" self.iterations = iterations\n",
" self.max_lr = max_lr\n",
" self.start_lr = start_lr or max_lr / 10\n",
" self.last_iterations = last_iterations or iterations // 10 + 1\n",
" self.half_iteration = (iterations - self.last_iterations) // 2\n",
" self.last_lr = last_lr or self.start_lr / 1000\n",
" self.iteration = 0\n",
"\n",
" def _interpolate(self, iter1, iter2, lr1, lr2):\n",
" return (lr2 - lr1) * (self.iteration - iter1) / (iter2 - iter1) + lr1\n",
"\n",
" def on_batch_begin(self, batch, logs):\n",
" if self.iteration < self.half_iteration:\n",
" lr = self._interpolate(0, self.half_iteration, self.start_lr,\n",
" self.max_lr)\n",
" elif self.iteration < 2 * self.half_iteration:\n",
" lr = self._interpolate(self.half_iteration, 2 * self.half_iteration,\n",
" self.max_lr, self.start_lr)\n",
" else:\n",
" lr = self._interpolate(2 * self.half_iteration, self.iterations,\n",
" self.start_lr, self.last_lr)\n",
" self.iteration += 1\n",
" K.set_value(self.model.optimizer.learning_rate, lr)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's build and compile a simple Fashion MNIST model, then train it using the `OneCycleScheduler` callback:"
]
},
{
"cell_type": "code",
"execution_count": 96,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/25\n",
"430/430 [==============================] - 1s 2ms/step - loss: 0.9502 - accuracy: 0.6913 - val_loss: 0.6003 - val_accuracy: 0.7874\n",
"Epoch 2/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.5695 - accuracy: 0.8025 - val_loss: 0.4918 - val_accuracy: 0.8248\n",
"Epoch 3/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.4954 - accuracy: 0.8252 - val_loss: 0.4762 - val_accuracy: 0.8264\n",
"Epoch 4/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.4515 - accuracy: 0.8402 - val_loss: 0.4261 - val_accuracy: 0.8478\n",
"Epoch 5/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.4225 - accuracy: 0.8492 - val_loss: 0.4066 - val_accuracy: 0.8486\n",
"Epoch 6/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.3958 - accuracy: 0.8571 - val_loss: 0.4787 - val_accuracy: 0.8224\n",
"Epoch 7/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.3787 - accuracy: 0.8626 - val_loss: 0.3917 - val_accuracy: 0.8566\n",
"Epoch 8/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.3630 - accuracy: 0.8683 - val_loss: 0.4719 - val_accuracy: 0.8296\n",
"Epoch 9/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.3512 - accuracy: 0.8724 - val_loss: 0.3673 - val_accuracy: 0.8652\n",
"Epoch 10/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.3360 - accuracy: 0.8766 - val_loss: 0.4957 - val_accuracy: 0.8466\n",
"Epoch 11/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.3287 - accuracy: 0.8786 - val_loss: 0.4187 - val_accuracy: 0.8370\n",
"Epoch 12/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.3173 - accuracy: 0.8815 - val_loss: 0.3425 - val_accuracy: 0.8728\n",
"Epoch 13/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2961 - accuracy: 0.8910 - val_loss: 0.3217 - val_accuracy: 0.8792\n",
"Epoch 14/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2818 - accuracy: 0.8958 - val_loss: 0.3734 - val_accuracy: 0.8692\n",
"Epoch 15/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2675 - accuracy: 0.9003 - val_loss: 0.3261 - val_accuracy: 0.8844\n",
"Epoch 16/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2558 - accuracy: 0.9055 - val_loss: 0.3205 - val_accuracy: 0.8820\n",
"Epoch 17/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2464 - accuracy: 0.9091 - val_loss: 0.3089 - val_accuracy: 0.8894\n",
"Epoch 18/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2368 - accuracy: 0.9115 - val_loss: 0.3130 - val_accuracy: 0.8870\n",
"Epoch 19/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2292 - accuracy: 0.9145 - val_loss: 0.3078 - val_accuracy: 0.8854\n",
"Epoch 20/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2205 - accuracy: 0.9186 - val_loss: 0.3092 - val_accuracy: 0.8886\n",
"Epoch 21/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2138 - accuracy: 0.9209 - val_loss: 0.3022 - val_accuracy: 0.8914\n",
"Epoch 22/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2073 - accuracy: 0.9232 - val_loss: 0.3054 - val_accuracy: 0.8914\n",
"Epoch 23/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.2020 - accuracy: 0.9261 - val_loss: 0.3026 - val_accuracy: 0.8896\n",
"Epoch 24/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.1989 - accuracy: 0.9273 - val_loss: 0.3020 - val_accuracy: 0.8922\n",
"Epoch 25/25\n",
"430/430 [==============================] - 1s 1ms/step - loss: 0.1967 - accuracy: 0.9276 - val_loss: 0.3016 - val_accuracy: 0.8920\n"
]
}
],
"source": [
"model = build_model()\n",
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=tf.keras.optimizers.SGD(),\n",
" metrics=[\"accuracy\"])\n",
"n_epochs = 25\n",
"onecycle = OneCycleScheduler(math.ceil(len(X_train) / batch_size) * n_epochs,\n",
" max_lr=0.1)\n",
"history = model.fit(X_train, y_train, epochs=n_epochs, batch_size=batch_size,\n",
" validation_data=(X_valid, y_valid),\n",
" callbacks=[onecycle])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Avoiding Overfitting Through Regularization"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## $\\ell_1$ and $\\ell_2$ regularization"
]
},
{
"cell_type": "code",
"execution_count": 97,
"metadata": {},
"outputs": [],
"source": [
"layer = tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\",\n",
" kernel_regularizer=tf.keras.regularizers.l2(0.01))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Or use `l1(0.1)` for ℓ1 regularization with a factor of 0.1, or `l1_l2(0.1, 0.01)` for both ℓ1 and ℓ2 regularization, with factors 0.1 and 0.01 respectively."
]
},
{
"cell_type": "code",
"execution_count": 98,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42) # extra code – for reproducibility"
]
},
{
"cell_type": "code",
"execution_count": 99,
"metadata": {},
"outputs": [],
"source": [
"from functools import partial\n",
"\n",
"RegularizedDense = partial(tf.keras.layers.Dense,\n",
" activation=\"relu\",\n",
" kernel_initializer=\"he_normal\",\n",
" kernel_regularizer=tf.keras.regularizers.l2(0.01))\n",
"\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
" RegularizedDense(100),\n",
" RegularizedDense(100),\n",
" RegularizedDense(10, activation=\"softmax\")\n",
"])"
]
},
{
"cell_type": "code",
"execution_count": 100,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/2\n",
"1719/1719 [==============================] - 2s 878us/step - loss: 3.1224 - accuracy: 0.7748 - val_loss: 1.8602 - val_accuracy: 0.8264\n",
"Epoch 2/2\n",
"1719/1719 [==============================] - 1s 814us/step - loss: 1.4263 - accuracy: 0.8159 - val_loss: 1.1269 - val_accuracy: 0.8182\n"
]
}
],
"source": [
"# extra code – compile and train the model\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.02)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
"history = model.fit(X_train, y_train, epochs=2,\n",
" validation_data=(X_valid, y_valid))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Dropout"
]
},
{
"cell_type": "code",
"execution_count": 101,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42) # extra code – for reproducibility"
]
},
{
"cell_type": "code",
"execution_count": 102,
"metadata": {},
"outputs": [],
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
" tf.keras.layers.Dropout(rate=0.2),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dropout(rate=0.2),\n",
" tf.keras.layers.Dense(100, activation=\"relu\",\n",
" kernel_initializer=\"he_normal\"),\n",
" tf.keras.layers.Dropout(rate=0.2),\n",
" tf.keras.layers.Dense(10, activation=\"softmax\")\n",
"])"
]
},
{
"cell_type": "code",
"execution_count": 103,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.6703 - accuracy: 0.7536 - val_loss: 0.4498 - val_accuracy: 0.8342\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 996us/step - loss: 0.5103 - accuracy: 0.8136 - val_loss: 0.4401 - val_accuracy: 0.8296\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 998us/step - loss: 0.4712 - accuracy: 0.8263 - val_loss: 0.3806 - val_accuracy: 0.8554\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 977us/step - loss: 0.4488 - accuracy: 0.8337 - val_loss: 0.3711 - val_accuracy: 0.8608\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.4342 - accuracy: 0.8409 - val_loss: 0.3672 - val_accuracy: 0.8606\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 983us/step - loss: 0.4245 - accuracy: 0.8427 - val_loss: 0.3706 - val_accuracy: 0.8600\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 995us/step - loss: 0.4131 - accuracy: 0.8467 - val_loss: 0.3582 - val_accuracy: 0.8650\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 959us/step - loss: 0.4074 - accuracy: 0.8484 - val_loss: 0.3478 - val_accuracy: 0.8708\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 997us/step - loss: 0.4024 - accuracy: 0.8533 - val_loss: 0.3556 - val_accuracy: 0.8690\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 998us/step - loss: 0.3903 - accuracy: 0.8552 - val_loss: 0.3453 - val_accuracy: 0.8732\n"
]
}
],
"source": [
"# extra code – compile and train the model\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.01, momentum=0.9)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
"history = model.fit(X_train, y_train, epochs=10,\n",
" validation_data=(X_valid, y_valid))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The training accuracy looks like it's lower than the validation accuracy, but that's just because dropout is only active during training. If we evaluate the model on the training set after training (i.e., with dropout turned off), we get the \"real\" training accuracy, which is very slightly higher than the validation accuracy and the test accuracy:"
]
},
{
"cell_type": "code",
"execution_count": 104,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1719/1719 [==============================] - 1s 578us/step - loss: 0.3082 - accuracy: 0.8849\n"
]
},
{
"data": {
"text/plain": [
"[0.30816400051116943, 0.8849090933799744]"
]
},
"execution_count": 104,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.evaluate(X_train, y_train)"
]
},
{
"cell_type": "code",
"execution_count": 105,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"313/313 [==============================] - 0s 588us/step - loss: 0.3629 - accuracy: 0.8700\n"
]
},
{
"data": {
"text/plain": [
"[0.3628920316696167, 0.8700000047683716]"
]
},
"execution_count": 105,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.evaluate(X_test, y_test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Note**: make sure to use `AlphaDropout` instead of `Dropout` if you want to build a self-normalizing neural net using SELU."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## MC Dropout"
]
},
{
"cell_type": "code",
"execution_count": 106,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42) # extra code – for reproducibility"
]
},
{
"cell_type": "code",
"execution_count": 107,
"metadata": {},
"outputs": [],
"source": [
"y_probas = np.stack([model(X_test, training=True)\n",
" for sample in range(100)])\n",
"y_proba = y_probas.mean(axis=0)"
]
},
{
"cell_type": "code",
"execution_count": 108,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[0. , 0. , 0. , 0. , 0. , 0.024, 0. , 0.132, 0. ,\n",
" 0.844]], dtype=float32)"
]
},
"execution_count": 108,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.predict(X_test[:1]).round(3)"
]
},
{
"cell_type": "code",
"execution_count": 109,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([0. , 0. , 0. , 0. , 0. , 0.067, 0. , 0.209, 0.001,\n",
" 0.723], dtype=float32)"
]
},
"execution_count": 109,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_proba[0].round(3)"
]
},
{
"cell_type": "code",
"execution_count": 110,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([0. , 0. , 0. , 0.001, 0. , 0.096, 0. , 0.162, 0.001,\n",
" 0.183], dtype=float32)"
]
},
"execution_count": 110,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_std = y_probas.std(axis=0)\n",
"y_std[0].round(3)"
]
},
{
"cell_type": "code",
"execution_count": 111,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.8717"
]
},
"execution_count": 111,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y_pred = y_proba.argmax(axis=1)\n",
"accuracy = (y_pred == y_test).sum() / len(y_test)\n",
"accuracy"
]
},
{
"cell_type": "code",
"execution_count": 112,
"metadata": {},
"outputs": [],
"source": [
"class MCDropout(tf.keras.layers.Dropout):\n",
" def call(self, inputs, training=None):\n",
" return super().call(inputs, training=True)"
]
},
{
"cell_type": "code",
"execution_count": 113,
"metadata": {},
"outputs": [],
"source": [
"# extra code – shows how to convert Dropout to MCDropout in a Sequential model\n",
"Dropout = tf.keras.layers.Dropout\n",
"mc_model = tf.keras.Sequential([\n",
" MCDropout(layer.rate) if isinstance(layer, Dropout) else layer\n",
" for layer in model.layers\n",
"])\n",
"mc_model.set_weights(model.get_weights())"
]
},
{
"cell_type": "code",
"execution_count": 114,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"sequential_25\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"flatten_22 (Flatten) (None, 784) 0 \n",
"_________________________________________________________________\n",
"mc_dropout (MCDropout) (None, 784) 0 \n",
"_________________________________________________________________\n",
"dense_89 (Dense) (None, 100) 78500 \n",
"_________________________________________________________________\n",
"mc_dropout_1 (MCDropout) (None, 100) 0 \n",
"_________________________________________________________________\n",
"dense_90 (Dense) (None, 100) 10100 \n",
"_________________________________________________________________\n",
"mc_dropout_2 (MCDropout) (None, 100) 0 \n",
"_________________________________________________________________\n",
"dense_91 (Dense) (None, 10) 1010 \n",
"=================================================================\n",
"Total params: 89,610\n",
"Trainable params: 89,610\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"mc_model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can use the model with MC Dropout:"
]
},
{
"cell_type": "code",
"execution_count": 115,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[0. , 0. , 0. , 0. , 0. , 0.07, 0. , 0.17, 0. , 0.76]],\n",
" dtype=float32)"
]
},
"execution_count": 115,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# extra code – shows that the model works without retraining\n",
"tf.random.set_seed(42)\n",
"np.mean([mc_model.predict(X_test[:1])\n",
" for sample in range(100)], axis=0).round(2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Max norm"
]
},
{
"cell_type": "code",
"execution_count": 116,
"metadata": {},
"outputs": [],
"source": [
"dense = tf.keras.layers.Dense(\n",
" 100, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
" kernel_constraint=tf.keras.constraints.max_norm(1.))"
]
},
{
"cell_type": "code",
"execution_count": 117,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1719/1719 [==============================] - 2s 1ms/step - loss: 0.5500 - accuracy: 0.8015 - val_loss: 0.4510 - val_accuracy: 0.8242\n",
"Epoch 2/10\n",
"1719/1719 [==============================] - 2s 960us/step - loss: 0.4089 - accuracy: 0.8499 - val_loss: 0.3956 - val_accuracy: 0.8504\n",
"Epoch 3/10\n",
"1719/1719 [==============================] - 2s 974us/step - loss: 0.3777 - accuracy: 0.8604 - val_loss: 0.3693 - val_accuracy: 0.8680\n",
"Epoch 4/10\n",
"1719/1719 [==============================] - 2s 943us/step - loss: 0.3581 - accuracy: 0.8690 - val_loss: 0.3517 - val_accuracy: 0.8716\n",
"Epoch 5/10\n",
"1719/1719 [==============================] - 2s 949us/step - loss: 0.3416 - accuracy: 0.8729 - val_loss: 0.3433 - val_accuracy: 0.8682\n",
"Epoch 6/10\n",
"1719/1719 [==============================] - 2s 951us/step - loss: 0.3368 - accuracy: 0.8756 - val_loss: 0.4045 - val_accuracy: 0.8582\n",
"Epoch 7/10\n",
"1719/1719 [==============================] - 2s 935us/step - loss: 0.3293 - accuracy: 0.8767 - val_loss: 0.4168 - val_accuracy: 0.8476\n",
"Epoch 8/10\n",
"1719/1719 [==============================] - 2s 951us/step - loss: 0.3258 - accuracy: 0.8779 - val_loss: 0.3570 - val_accuracy: 0.8674\n",
"Epoch 9/10\n",
"1719/1719 [==============================] - 2s 970us/step - loss: 0.3269 - accuracy: 0.8787 - val_loss: 0.3702 - val_accuracy: 0.8578\n",
"Epoch 10/10\n",
"1719/1719 [==============================] - 2s 948us/step - loss: 0.3169 - accuracy: 0.8809 - val_loss: 0.3907 - val_accuracy: 0.8578\n"
]
}
],
"source": [
"# extra code – shows how to apply max norm to every hidden layer in a model\n",
"\n",
"MaxNormDense = partial(tf.keras.layers.Dense,\n",
" activation=\"relu\", kernel_initializer=\"he_normal\",\n",
" kernel_constraint=tf.keras.constraints.max_norm(1.))\n",
"\n",
"tf.random.set_seed(42)\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
" MaxNormDense(100),\n",
" MaxNormDense(100),\n",
" tf.keras.layers.Dense(10, activation=\"softmax\")\n",
"])\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.01, momentum=0.9)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
"history = model.fit(X_train, y_train, epochs=10,\n",
" validation_data=(X_valid, y_valid))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercises"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. to 7."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Glorot initialization and He initialization were designed to make the output standard deviation as close as possible to the input standard deviation, at least at the beginning of training. This reduces the vanishing/exploding gradients problem.\n",
"2. No, all weights should be sampled independently; they should not all have the same initial value. One important goal of sampling weights randomly is to break symmetry: if all the weights have the same initial value, even if that value is not zero, then symmetry is not broken (i.e., all neurons in a given layer are equivalent), and backpropagation will be unable to break it. Concretely, this means that all the neurons in any given layer will always have the same weights. It's like having just one neuron per layer, and much slower. It is virtually impossible for such a configuration to converge to a good solution.\n",
"3. It is perfectly fine to initialize the bias terms to zero. Some people like to initialize them just like weights, and that's OK too; it does not make much difference.\n",
"4. ReLU is usually a good default for the hidden layers, as it is fast and yields good results. Its ability to output precisely zero can also be useful in some cases (e.g., see Chapter 17). Moreover, it can sometimes benefit from optimized implementations as well as from hardware acceleration. The leaky ReLU variants of ReLU can improve the model's quality without hindering its speed too much compared to ReLU. For large neural nets and more complex problems, GLU, Swish and Mish can give you a slightly higher quality model, but they have a computational cost. The hyperbolic tangent (tanh) can be useful in the output layer if you need to output a number in a fixed range (by default between –1 and 1), but nowadays it is not used much in hidden layers, except in recurrent nets. The sigmoid activation function is also useful in the output layer when you need to estimate a probability (e.g., for binary classification), but it is rarely used in hidden layers (there are exceptions—for example, for the coding layer of variational autoencoders; see Chapter 17). The softplus activation function is useful in the output layer when you need to ensure that the output will always be positive. The softmax activation function is useful in the output layer to estimate probabilities for mutually exclusive classes, but it is rarely (if ever) used in hidden layers.\n",
"5. If you set the `momentum` hyperparameter too close to 1 (e.g., 0.99999) when using an `SGD` optimizer, then the algorithm will likely pick up a lot of speed, hopefully moving roughly toward the global minimum, but its momentum will carry it right past the minimum. Then it will slow down and come back, accelerate again, overshoot again, and so on. It may oscillate this way many times before converging, so overall it will take much longer to converge than with a smaller `momentum` value.\n",
"6. One way to produce a sparse model (i.e., with most weights equal to zero) is to train the model normally, then zero out tiny weights. For more sparsity, you can apply ℓ1 regularization during training, which pushes the optimizer toward sparsity. A third option is to use the TensorFlow Model Optimization Toolkit.\n",
"7. Yes, dropout does slow down training, in general roughly by a factor of two. However, it has no impact on inference speed since it is only turned on during training. MC Dropout is exactly like dropout during training, but it is still active during inference, so each inference is slowed down slightly. More importantly, when using MC Dropout you generally want to run inference 10 times or more to get better predictions. This means that making predictions is slowed down by a factor of 10 or more."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 8. Deep Learning on CIFAR10"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### a.\n",
"*Exercise: Build a DNN with 20 hidden layers of 100 neurons each (that's too many, but it's the point of this exercise). Use He initialization and the Swish activation function.*"
]
},
{
"cell_type": "code",
"execution_count": 118,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42)\n",
"\n",
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=[32, 32, 3]))\n",
"for _ in range(20):\n",
" model.add(tf.keras.layers.Dense(100,\n",
" activation=\"swish\",\n",
" kernel_initializer=\"he_normal\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### b.\n",
"*Exercise: Using Nadam optimization and early stopping, train the network on the CIFAR10 dataset. You can load it with `tf.keras.datasets.cifar10.load_data()`. The dataset is composed of 60,000 32 × 32–pixel color images (50,000 for training, 10,000 for testing) with 10 classes, so you'll need a softmax output layer with 10 neurons. Remember to search for the right learning rate each time you change the model's architecture or hyperparameters.*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's add the output layer to the model:"
]
},
{
"cell_type": "code",
"execution_count": 119,
"metadata": {},
"outputs": [],
"source": [
"model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's use a Nadam optimizer with a learning rate of 5e-5. I tried learning rates 1e-5, 3e-5, 1e-4, 3e-4, 1e-3, 3e-3 and 1e-2, and I compared their learning curves for 10 epochs each (using the TensorBoard callback, below). The learning rates 3e-5 and 1e-4 were pretty good, so I tried 5e-5, which turned out to be slightly better."
]
},
{
"cell_type": "code",
"execution_count": 120,
"metadata": {},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.Nadam(learning_rate=5e-5)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=optimizer,\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's load the CIFAR10 dataset. We also want to use early stopping, so we need a validation set. Let's use the first 5,000 images of the original training set as the validation set:"
]
},
{
"cell_type": "code",
"execution_count": 121,
"metadata": {},
"outputs": [],
"source": [
"cifar10 = tf.keras.datasets.cifar10.load_data()\n",
"(X_train_full, y_train_full), (X_test, y_test) = cifar10\n",
"\n",
"X_train = X_train_full[5000:]\n",
"y_train = y_train_full[5000:]\n",
"X_valid = X_train_full[:5000]\n",
"y_valid = y_train_full[:5000]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can create the callbacks we need and train the model:"
]
},
{
"cell_type": "code",
"execution_count": 122,
"metadata": {},
"outputs": [],
"source": [
"early_stopping_cb = tf.keras.callbacks.EarlyStopping(patience=20,\n",
" restore_best_weights=True)\n",
"model_checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\"my_cifar10_model\",\n",
" save_best_only=True)\n",
"run_index = 1 # increment every time you train the model\n",
"run_logdir = Path() / \"my_cifar10_logs\" / f\"run_{run_index:03d}\"\n",
"tensorboard_cb = tf.keras.callbacks.TensorBoard(run_logdir)\n",
"callbacks = [early_stopping_cb, model_checkpoint_cb, tensorboard_cb]"
]
},
{
"cell_type": "code",
"execution_count": 123,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%load_ext tensorboard\n",
"%tensorboard --logdir=./my_cifar10_logs"
]
},
{
"cell_type": "code",
"execution_count": 124,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/100\n",
"1404/1407 [============================>.] - ETA: 0s - loss: 4.0493 - accuracy: 0.1598INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 17s 10ms/step - loss: 4.0462 - accuracy: 0.1597 - val_loss: 2.1441 - val_accuracy: 0.2036\n",
"Epoch 2/100\n",
"1407/1407 [==============================] - ETA: 0s - loss: 2.0667 - accuracy: 0.2320INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 12s 9ms/step - loss: 2.0667 - accuracy: 0.2320 - val_loss: 2.0134 - val_accuracy: 0.2472\n",
"Epoch 3/100\n",
"1407/1407 [==============================] - ETA: 0s - loss: 1.9472 - accuracy: 0.2819INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.9472 - accuracy: 0.2819 - val_loss: 1.9427 - val_accuracy: 0.2796\n",
"Epoch 4/100\n",
"1405/1407 [============================>.] - ETA: 0s - loss: 1.8636 - accuracy: 0.3182INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.8637 - accuracy: 0.3182 - val_loss: 1.8934 - val_accuracy: 0.3222\n",
"Epoch 5/100\n",
"1402/1407 [============================>.] - ETA: 0s - loss: 1.7975 - accuracy: 0.3464INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.7974 - accuracy: 0.3465 - val_loss: 1.8389 - val_accuracy: 0.3284\n",
"Epoch 6/100\n",
"1407/1407 [==============================] - 9s 7ms/step - loss: 1.7446 - accuracy: 0.3664 - val_loss: 2.0006 - val_accuracy: 0.3030\n",
"Epoch 7/100\n",
"1407/1407 [==============================] - ETA: 0s - loss: 1.6974 - accuracy: 0.3852INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 1.6974 - accuracy: 0.3852 - val_loss: 1.7075 - val_accuracy: 0.3738\n",
"Epoch 8/100\n",
"1405/1407 [============================>.] - ETA: 0s - loss: 1.6605 - accuracy: 0.3984INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 1.6604 - accuracy: 0.3984 - val_loss: 1.6788 - val_accuracy: 0.3836\n",
"Epoch 9/100\n",
"1405/1407 [============================>.] - ETA: 0s - loss: 1.6322 - accuracy: 0.4114INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.6321 - accuracy: 0.4114 - val_loss: 1.6477 - val_accuracy: 0.4014\n",
"Epoch 10/100\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 1.6065 - accuracy: 0.4205 - val_loss: 1.6623 - val_accuracy: 0.3980\n",
"Epoch 11/100\n",
"1401/1407 [============================>.] - ETA: 0s - loss: 1.5843 - accuracy: 0.4287INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.5845 - accuracy: 0.4285 - val_loss: 1.6032 - val_accuracy: 0.4198\n",
"Epoch 12/100\n",
"1407/1407 [==============================] - 9s 6ms/step - loss: 1.5634 - accuracy: 0.4367 - val_loss: 1.6063 - val_accuracy: 0.4258\n",
"Epoch 13/100\n",
"1401/1407 [============================>.] - ETA: 0s - loss: 1.5443 - accuracy: 0.4420INFO:tensorflow:Assets written to: my_cifar10_model/assets\n",
"<<47 more lines>>\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 1.3247 - accuracy: 0.5256 - val_loss: 1.5130 - val_accuracy: 0.4616\n",
"Epoch 33/100\n",
"1407/1407 [==============================] - 13s 9ms/step - loss: 1.3164 - accuracy: 0.5286 - val_loss: 1.5284 - val_accuracy: 0.4686\n",
"Epoch 34/100\n",
"1407/1407 [==============================] - 12s 9ms/step - loss: 1.3091 - accuracy: 0.5303 - val_loss: 1.5208 - val_accuracy: 0.4682\n",
"Epoch 35/100\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 1.3026 - accuracy: 0.5319 - val_loss: 1.5479 - val_accuracy: 0.4604\n",
"Epoch 36/100\n",
"1407/1407 [==============================] - 11s 8ms/step - loss: 1.2930 - accuracy: 0.5378 - val_loss: 1.5443 - val_accuracy: 0.4580\n",
"Epoch 37/100\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 1.2833 - accuracy: 0.5406 - val_loss: 1.5165 - val_accuracy: 0.4710\n",
"Epoch 38/100\n",
"1407/1407 [==============================] - 11s 8ms/step - loss: 1.2763 - accuracy: 0.5433 - val_loss: 1.5345 - val_accuracy: 0.4672\n",
"Epoch 39/100\n",
"1407/1407 [==============================] - 12s 9ms/step - loss: 1.2687 - accuracy: 0.5437 - val_loss: 1.5162 - val_accuracy: 0.4712\n",
"Epoch 40/100\n",
"1407/1407 [==============================] - 11s 7ms/step - loss: 1.2623 - accuracy: 0.5490 - val_loss: 1.5717 - val_accuracy: 0.4566\n",
"Epoch 41/100\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.2580 - accuracy: 0.5467 - val_loss: 1.5296 - val_accuracy: 0.4738\n",
"Epoch 42/100\n",
"1407/1407 [==============================] - 13s 9ms/step - loss: 1.2469 - accuracy: 0.5532 - val_loss: 1.5179 - val_accuracy: 0.4690\n",
"Epoch 43/100\n",
"1407/1407 [==============================] - 11s 8ms/step - loss: 1.2404 - accuracy: 0.5542 - val_loss: 1.5542 - val_accuracy: 0.4566\n",
"Epoch 44/100\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 1.2292 - accuracy: 0.5605 - val_loss: 1.5536 - val_accuracy: 0.4608\n",
"Epoch 45/100\n",
"1407/1407 [==============================] - 12s 9ms/step - loss: 1.2276 - accuracy: 0.5606 - val_loss: 1.5522 - val_accuracy: 0.4624\n",
"Epoch 46/100\n",
"1407/1407 [==============================] - 13s 9ms/step - loss: 1.2200 - accuracy: 0.5637 - val_loss: 1.5339 - val_accuracy: 0.4794\n",
"Epoch 47/100\n",
"1407/1407 [==============================] - 13s 9ms/step - loss: 1.2080 - accuracy: 0.5677 - val_loss: 1.5451 - val_accuracy: 0.4688\n",
"Epoch 48/100\n",
"1407/1407 [==============================] - 15s 10ms/step - loss: 1.2050 - accuracy: 0.5675 - val_loss: 1.5209 - val_accuracy: 0.4770\n",
"Epoch 49/100\n",
"1407/1407 [==============================] - 10s 7ms/step - loss: 1.1947 - accuracy: 0.5718 - val_loss: 1.5435 - val_accuracy: 0.4736\n"
]
},
{
"data": {
"text/plain": [
""
]
},
"execution_count": 124,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.fit(X_train, y_train, epochs=100,\n",
" validation_data=(X_valid, y_valid),\n",
" callbacks=callbacks)"
]
},
{
"cell_type": "code",
"execution_count": 125,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"157/157 [==============================] - 0s 2ms/step - loss: 1.5062 - accuracy: 0.4676\n"
]
},
{
"data": {
"text/plain": [
"[1.5061508417129517, 0.4675999879837036]"
]
},
"execution_count": 125,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.evaluate(X_valid, y_valid)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The model with the lowest validation loss gets about 46.8% accuracy on the validation set. It took 29 epochs to reach the lowest validation loss, with roughly 10 seconds per epoch on my laptop (without a GPU). Let's see if we can improve the model using Batch Normalization."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### c.\n",
"*Exercise: Now try adding Batch Normalization and compare the learning curves: Is it converging faster than before? Does it produce a better model? How does it affect training speed?*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The code below is very similar to the code above, with a few changes:\n",
"\n",
"* I added a BN layer after every Dense layer (before the activation function), except for the output layer.\n",
"* I changed the learning rate to 5e-4. I experimented with 1e-5, 3e-5, 5e-5, 1e-4, 3e-4, 5e-4, 1e-3 and 3e-3, and I chose the one with the best validation performance after 20 epochs.\n",
"* I renamed the run directories to run_bn_* and the model file name to `my_cifar10_bn_model`."
]
},
{
"cell_type": "code",
"execution_count": 126,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/100\n",
"1403/1407 [============================>.] - ETA: 0s - loss: 2.0377 - accuracy: 0.2523INFO:tensorflow:Assets written to: my_cifar10_bn_model/assets\n",
"1407/1407 [==============================] - 32s 18ms/step - loss: 2.0374 - accuracy: 0.2525 - val_loss: 1.8766 - val_accuracy: 0.3154\n",
"Epoch 2/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.7874 - accuracy: 0.3542 - val_loss: 1.8784 - val_accuracy: 0.3268\n",
"Epoch 3/100\n",
"1407/1407 [==============================] - 20s 15ms/step - loss: 1.6806 - accuracy: 0.3969 - val_loss: 1.9764 - val_accuracy: 0.3252\n",
"Epoch 4/100\n",
"1403/1407 [============================>.] - ETA: 0s - loss: 1.6111 - accuracy: 0.4229INFO:tensorflow:Assets written to: my_cifar10_bn_model/assets\n",
"1407/1407 [==============================] - 24s 17ms/step - loss: 1.6112 - accuracy: 0.4228 - val_loss: 1.7087 - val_accuracy: 0.3750\n",
"Epoch 5/100\n",
"1402/1407 [============================>.] - ETA: 0s - loss: 1.5520 - accuracy: 0.4478INFO:tensorflow:Assets written to: my_cifar10_bn_model/assets\n",
"1407/1407 [==============================] - 21s 15ms/step - loss: 1.5521 - accuracy: 0.4476 - val_loss: 1.6272 - val_accuracy: 0.4176\n",
"Epoch 6/100\n",
"1406/1407 [============================>.] - ETA: 0s - loss: 1.5030 - accuracy: 0.4659INFO:tensorflow:Assets written to: my_cifar10_bn_model/assets\n",
"1407/1407 [==============================] - 23s 16ms/step - loss: 1.5030 - accuracy: 0.4660 - val_loss: 1.5401 - val_accuracy: 0.4452\n",
"Epoch 7/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.4559 - accuracy: 0.4812 - val_loss: 1.6990 - val_accuracy: 0.3952\n",
"Epoch 8/100\n",
"1403/1407 [============================>.] - ETA: 0s - loss: 1.4169 - accuracy: 0.4987INFO:tensorflow:Assets written to: my_cifar10_bn_model/assets\n",
"1407/1407 [==============================] - 21s 15ms/step - loss: 1.4168 - accuracy: 0.4987 - val_loss: 1.5078 - val_accuracy: 0.4652\n",
"Epoch 9/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.3863 - accuracy: 0.5123 - val_loss: 1.5513 - val_accuracy: 0.4470\n",
"Epoch 10/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.3514 - accuracy: 0.5216 - val_loss: 1.5208 - val_accuracy: 0.4562\n",
"Epoch 11/100\n",
"1407/1407 [==============================] - 16s 12ms/step - loss: 1.3220 - accuracy: 0.5314 - val_loss: 1.7301 - val_accuracy: 0.4206\n",
"Epoch 12/100\n",
"1404/1407 [============================>.] - ETA: 0s - loss: 1.2933 - accuracy: 0.5410INFO:tensorflow:Assets written to: my_cifar10_bn_model/assets\n",
"1407/1407 [==============================] - 25s 18ms/step - loss: 1.2931 - accuracy: 0.5410 - val_loss: 1.4909 - val_accuracy: 0.4734\n",
"Epoch 13/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.2702 - accuracy: 0.5490 - val_loss: 1.5256 - val_accuracy: 0.4636\n",
"Epoch 14/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.2424 - accuracy: 0.5591 - val_loss: 1.5569 - val_accuracy: 0.4624\n",
"Epoch 15/100\n",
"<<12 more lines>>\n",
"Epoch 21/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.1174 - accuracy: 0.6066 - val_loss: 1.5241 - val_accuracy: 0.4828\n",
"Epoch 22/100\n",
"1407/1407 [==============================] - 18s 13ms/step - loss: 1.0978 - accuracy: 0.6128 - val_loss: 1.5313 - val_accuracy: 0.4772\n",
"Epoch 23/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.0844 - accuracy: 0.6198 - val_loss: 1.4993 - val_accuracy: 0.4924\n",
"Epoch 24/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.0677 - accuracy: 0.6244 - val_loss: 1.4622 - val_accuracy: 0.5078\n",
"Epoch 25/100\n",
"1407/1407 [==============================] - 18s 13ms/step - loss: 1.0571 - accuracy: 0.6297 - val_loss: 1.4917 - val_accuracy: 0.4990\n",
"Epoch 26/100\n",
"1407/1407 [==============================] - 19s 14ms/step - loss: 1.0395 - accuracy: 0.6327 - val_loss: 1.4888 - val_accuracy: 0.4896\n",
"Epoch 27/100\n",
"1407/1407 [==============================] - 18s 13ms/step - loss: 1.0298 - accuracy: 0.6370 - val_loss: 1.5358 - val_accuracy: 0.5024\n",
"Epoch 28/100\n",
"1407/1407 [==============================] - 18s 13ms/step - loss: 1.0150 - accuracy: 0.6444 - val_loss: 1.5219 - val_accuracy: 0.5030\n",
"Epoch 29/100\n",
"1407/1407 [==============================] - 16s 12ms/step - loss: 1.0100 - accuracy: 0.6456 - val_loss: 1.4933 - val_accuracy: 0.5098\n",
"Epoch 30/100\n",
"1407/1407 [==============================] - 20s 14ms/step - loss: 0.9956 - accuracy: 0.6492 - val_loss: 1.4756 - val_accuracy: 0.5012\n",
"Epoch 31/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 0.9787 - accuracy: 0.6576 - val_loss: 1.5181 - val_accuracy: 0.4936\n",
"Epoch 32/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 0.9710 - accuracy: 0.6565 - val_loss: 1.7510 - val_accuracy: 0.4568\n",
"Epoch 33/100\n",
"1407/1407 [==============================] - 20s 14ms/step - loss: 0.9613 - accuracy: 0.6628 - val_loss: 1.5576 - val_accuracy: 0.4910\n",
"Epoch 34/100\n",
"1407/1407 [==============================] - 19s 14ms/step - loss: 0.9530 - accuracy: 0.6651 - val_loss: 1.5087 - val_accuracy: 0.5046\n",
"Epoch 35/100\n",
"1407/1407 [==============================] - 19s 13ms/step - loss: 0.9388 - accuracy: 0.6701 - val_loss: 1.5534 - val_accuracy: 0.4950\n",
"Epoch 36/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 0.9331 - accuracy: 0.6743 - val_loss: 1.5033 - val_accuracy: 0.5046\n",
"Epoch 37/100\n",
"1407/1407 [==============================] - 19s 14ms/step - loss: 0.9144 - accuracy: 0.6808 - val_loss: 1.5679 - val_accuracy: 0.5028\n",
"157/157 [==============================] - 0s 2ms/step - loss: 1.4236 - accuracy: 0.5074\n"
]
},
{
"data": {
"text/plain": [
"[1.4236289262771606, 0.5073999762535095]"
]
},
"execution_count": 126,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tf.random.set_seed(42)\n",
"\n",
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=[32, 32, 3]))\n",
"for _ in range(20):\n",
" model.add(tf.keras.layers.Dense(100, kernel_initializer=\"he_normal\"))\n",
" model.add(tf.keras.layers.BatchNormalization())\n",
" model.add(tf.keras.layers.Activation(\"swish\"))\n",
"\n",
"model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))\n",
"\n",
"optimizer = tf.keras.optimizers.Nadam(learning_rate=5e-4)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
"\n",
"early_stopping_cb = tf.keras.callbacks.EarlyStopping(patience=20,\n",
" restore_best_weights=True)\n",
"model_checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\"my_cifar10_bn_model\",\n",
" save_best_only=True)\n",
"run_index = 1 # increment every time you train the model\n",
"run_logdir = Path() / \"my_cifar10_logs\" / f\"run_bn_{run_index:03d}\"\n",
"tensorboard_cb = tf.keras.callbacks.TensorBoard(run_logdir)\n",
"callbacks = [early_stopping_cb, model_checkpoint_cb, tensorboard_cb]\n",
"\n",
"model.fit(X_train, y_train, epochs=100,\n",
" validation_data=(X_valid, y_valid),\n",
" callbacks=callbacks)\n",
"\n",
"model.evaluate(X_valid, y_valid)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* *Is the model converging faster than before?* Much faster! The previous model took 29 epochs to reach the lowest validation loss, while the new model achieved that same loss in just 12 epochs and continued to make progress until the 17th epoch. The BN layers stabilized training and allowed us to use a much larger learning rate, so convergence was faster.\n",
"* *Does BN produce a better model?* Yes! The final model is also much better, with 50.7% validation accuracy instead of 46.7%. It's still not a very good model, but at least it's much better than before (a Convolutional Neural Network would do much better, but that's a different topic, see chapter 14).\n",
"* *How does BN affect training speed?* Although the model converged much faster, each epoch took about 15s instead of 10s, because of the extra computations required by the BN layers. But overall the training time (wall time) to reach the best model was shortened by about 10%."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### d.\n",
"*Exercise: Try replacing Batch Normalization with SELU, and make the necessary adjustements to ensure the network self-normalizes (i.e., standardize the input features, use LeCun normal initialization, make sure the DNN contains only a sequence of dense layers, etc.).*"
]
},
{
"cell_type": "code",
"execution_count": 127,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/100\n",
"1403/1407 [============================>.] - ETA: 0s - loss: 1.9386 - accuracy: 0.3045INFO:tensorflow:Assets written to: my_cifar10_selu_model/assets\n",
"1407/1407 [==============================] - 20s 13ms/step - loss: 1.9385 - accuracy: 0.3046 - val_loss: 1.8175 - val_accuracy: 0.3510\n",
"Epoch 2/100\n",
"1405/1407 [============================>.] - ETA: 0s - loss: 1.7241 - accuracy: 0.3869INFO:tensorflow:Assets written to: my_cifar10_selu_model/assets\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.7241 - accuracy: 0.3869 - val_loss: 1.7677 - val_accuracy: 0.3614\n",
"Epoch 3/100\n",
"1407/1407 [==============================] - ETA: 0s - loss: 1.6272 - accuracy: 0.4263INFO:tensorflow:Assets written to: my_cifar10_selu_model/assets\n",
"1407/1407 [==============================] - 18s 13ms/step - loss: 1.6272 - accuracy: 0.4263 - val_loss: 1.6878 - val_accuracy: 0.4054\n",
"Epoch 4/100\n",
"1406/1407 [============================>.] - ETA: 0s - loss: 1.5644 - accuracy: 0.4492INFO:tensorflow:Assets written to: my_cifar10_selu_model/assets\n",
"1407/1407 [==============================] - 18s 13ms/step - loss: 1.5643 - accuracy: 0.4492 - val_loss: 1.6589 - val_accuracy: 0.4304\n",
"Epoch 5/100\n",
"1404/1407 [============================>.] - ETA: 0s - loss: 1.5080 - accuracy: 0.4712INFO:tensorflow:Assets written to: my_cifar10_selu_model/assets\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.5080 - accuracy: 0.4712 - val_loss: 1.5651 - val_accuracy: 0.4538\n",
"Epoch 6/100\n",
"1404/1407 [============================>.] - ETA: 0s - loss: 1.4611 - accuracy: 0.4873INFO:tensorflow:Assets written to: my_cifar10_selu_model/assets\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.4613 - accuracy: 0.4872 - val_loss: 1.5305 - val_accuracy: 0.4678\n",
"Epoch 7/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.4174 - accuracy: 0.5077 - val_loss: 1.5346 - val_accuracy: 0.4558\n",
"Epoch 8/100\n",
"1406/1407 [============================>.] - ETA: 0s - loss: 1.3781 - accuracy: 0.5175INFO:tensorflow:Assets written to: my_cifar10_selu_model/assets\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.3781 - accuracy: 0.5175 - val_loss: 1.4773 - val_accuracy: 0.4882\n",
"Epoch 9/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.3413 - accuracy: 0.5345 - val_loss: 1.5021 - val_accuracy: 0.4764\n",
"Epoch 10/100\n",
"1407/1407 [==============================] - 15s 10ms/step - loss: 1.3182 - accuracy: 0.5422 - val_loss: 1.5709 - val_accuracy: 0.4762\n",
"Epoch 11/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.2832 - accuracy: 0.5571 - val_loss: 1.5345 - val_accuracy: 0.4868\n",
"Epoch 12/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.2557 - accuracy: 0.5667 - val_loss: 1.5024 - val_accuracy: 0.4900\n",
"Epoch 13/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.2373 - accuracy: 0.5710 - val_loss: 1.5114 - val_accuracy: 0.5028\n",
"Epoch 14/100\n",
"1404/1407 [============================>.] - ETA: 0s - loss: 1.2071 - accuracy: 0.5846INFO:tensorflow:Assets written to: my_cifar10_selu_model/assets\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.2073 - accuracy: 0.5847 - val_loss: 1.4608 - val_accuracy: 0.5026\n",
"Epoch 15/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.1843 - accuracy: 0.5940 - val_loss: 1.4962 - val_accuracy: 0.5038\n",
"Epoch 16/100\n",
"1407/1407 [==============================] - 16s 12ms/step - loss: 1.1617 - accuracy: 0.6026 - val_loss: 1.5255 - val_accuracy: 0.5062\n",
"Epoch 17/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.1452 - accuracy: 0.6084 - val_loss: 1.5057 - val_accuracy: 0.5036\n",
"Epoch 18/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 1.1297 - accuracy: 0.6145 - val_loss: 1.5097 - val_accuracy: 0.5010\n",
"Epoch 19/100\n",
"1407/1407 [==============================] - 16s 12ms/step - loss: 1.1004 - accuracy: 0.6245 - val_loss: 1.5218 - val_accuracy: 0.5014\n",
"Epoch 20/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.0971 - accuracy: 0.6304 - val_loss: 1.5253 - val_accuracy: 0.5090\n",
"Epoch 21/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.0670 - accuracy: 0.6345 - val_loss: 1.5006 - val_accuracy: 0.5034\n",
"Epoch 22/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.0544 - accuracy: 0.6407 - val_loss: 1.5244 - val_accuracy: 0.5010\n",
"Epoch 23/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.0338 - accuracy: 0.6502 - val_loss: 1.5355 - val_accuracy: 0.5096\n",
"Epoch 24/100\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.0281 - accuracy: 0.6514 - val_loss: 1.5257 - val_accuracy: 0.5164\n",
"Epoch 25/100\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.4097 - accuracy: 0.6478 - val_loss: 1.8203 - val_accuracy: 0.3514\n",
"Epoch 26/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.3733 - accuracy: 0.5157 - val_loss: 1.5600 - val_accuracy: 0.4664\n",
"Epoch 27/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.2032 - accuracy: 0.5814 - val_loss: 1.5367 - val_accuracy: 0.4944\n",
"Epoch 28/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.1291 - accuracy: 0.6121 - val_loss: 1.5333 - val_accuracy: 0.4852\n",
"Epoch 29/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.0734 - accuracy: 0.6317 - val_loss: 1.5475 - val_accuracy: 0.5032\n",
"Epoch 30/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.0294 - accuracy: 0.6469 - val_loss: 1.5400 - val_accuracy: 0.5052\n",
"Epoch 31/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.0081 - accuracy: 0.6605 - val_loss: 1.5617 - val_accuracy: 0.4856\n",
"Epoch 32/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.0109 - accuracy: 0.6603 - val_loss: 1.5727 - val_accuracy: 0.5124\n",
"Epoch 33/100\n",
"1407/1407 [==============================] - 17s 12ms/step - loss: 0.9646 - accuracy: 0.6762 - val_loss: 1.5333 - val_accuracy: 0.5174\n",
"Epoch 34/100\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 0.9597 - accuracy: 0.6789 - val_loss: 1.5601 - val_accuracy: 0.5016\n",
"157/157 [==============================] - 0s 1ms/step - loss: 1.4608 - accuracy: 0.5026\n"
]
},
{
"data": {
"text/plain": [
"[1.4607702493667603, 0.5026000142097473]"
]
},
"execution_count": 127,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tf.random.set_seed(42)\n",
"\n",
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=[32, 32, 3]))\n",
"for _ in range(20):\n",
" model.add(tf.keras.layers.Dense(100,\n",
" kernel_initializer=\"lecun_normal\",\n",
" activation=\"selu\"))\n",
"\n",
"model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))\n",
"\n",
"optimizer = tf.keras.optimizers.Nadam(learning_rate=7e-4)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
"\n",
"early_stopping_cb = tf.keras.callbacks.EarlyStopping(\n",
" patience=20, restore_best_weights=True)\n",
"model_checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\n",
" \"my_cifar10_selu_model\", save_best_only=True)\n",
"run_index = 1 # increment every time you train the model\n",
"run_logdir = Path() / \"my_cifar10_logs\" / f\"run_selu_{run_index:03d}\"\n",
"tensorboard_cb = tf.keras.callbacks.TensorBoard(run_logdir)\n",
"callbacks = [early_stopping_cb, model_checkpoint_cb, tensorboard_cb]\n",
"\n",
"X_means = X_train.mean(axis=0)\n",
"X_stds = X_train.std(axis=0)\n",
"X_train_scaled = (X_train - X_means) / X_stds\n",
"X_valid_scaled = (X_valid - X_means) / X_stds\n",
"X_test_scaled = (X_test - X_means) / X_stds\n",
"\n",
"model.fit(X_train_scaled, y_train, epochs=100,\n",
" validation_data=(X_valid_scaled, y_valid),\n",
" callbacks=callbacks)\n",
"\n",
"model.evaluate(X_valid_scaled, y_valid)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This model reached the first model's validation loss in just 8 epochs. After 14 epochs, it reached its lowest validation loss, with about 50.3% accuracy, which is better than the original model (46.7%), but not quite as good as the model using batch normalization (50.7%). Each epoch took only 9 seconds. So it's the fastest model to train so far."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### e.\n",
"*Exercise: Try regularizing the model with alpha dropout. Then, without retraining your model, see if you can achieve better accuracy using MC Dropout.*"
]
},
{
"cell_type": "code",
"execution_count": 128,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/100\n",
"1405/1407 [============================>.] - ETA: 0s - loss: 1.8953 - accuracy: 0.3240INFO:tensorflow:Assets written to: my_cifar10_alpha_dropout_model/assets\n",
"1407/1407 [==============================] - 18s 11ms/step - loss: 1.8950 - accuracy: 0.3239 - val_loss: 1.7556 - val_accuracy: 0.3812\n",
"Epoch 2/100\n",
"1403/1407 [============================>.] - ETA: 0s - loss: 1.6618 - accuracy: 0.4129INFO:tensorflow:Assets written to: my_cifar10_alpha_dropout_model/assets\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.6618 - accuracy: 0.4130 - val_loss: 1.6563 - val_accuracy: 0.4114\n",
"Epoch 3/100\n",
"1402/1407 [============================>.] - ETA: 0s - loss: 1.5772 - accuracy: 0.4431INFO:tensorflow:Assets written to: my_cifar10_alpha_dropout_model/assets\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.5770 - accuracy: 0.4432 - val_loss: 1.6507 - val_accuracy: 0.4232\n",
"Epoch 4/100\n",
"1406/1407 [============================>.] - ETA: 0s - loss: 1.5081 - accuracy: 0.4673INFO:tensorflow:Assets written to: my_cifar10_alpha_dropout_model/assets\n",
"1407/1407 [==============================] - 15s 10ms/step - loss: 1.5081 - accuracy: 0.4672 - val_loss: 1.5892 - val_accuracy: 0.4566\n",
"Epoch 5/100\n",
"1403/1407 [============================>.] - ETA: 0s - loss: 1.4560 - accuracy: 0.4902INFO:tensorflow:Assets written to: my_cifar10_alpha_dropout_model/assets\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.4561 - accuracy: 0.4902 - val_loss: 1.5382 - val_accuracy: 0.4696\n",
"Epoch 6/100\n",
"1401/1407 [============================>.] - ETA: 0s - loss: 1.4095 - accuracy: 0.5050INFO:tensorflow:Assets written to: my_cifar10_alpha_dropout_model/assets\n",
"1407/1407 [==============================] - 16s 11ms/step - loss: 1.4094 - accuracy: 0.5050 - val_loss: 1.5236 - val_accuracy: 0.4818\n",
"Epoch 7/100\n",
"1401/1407 [============================>.] - ETA: 0s - loss: 1.3634 - accuracy: 0.5234INFO:tensorflow:Assets written to: my_cifar10_alpha_dropout_model/assets\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.3636 - accuracy: 0.5232 - val_loss: 1.5139 - val_accuracy: 0.4840\n",
"Epoch 8/100\n",
"1405/1407 [============================>.] - ETA: 0s - loss: 1.3297 - accuracy: 0.5377INFO:tensorflow:Assets written to: my_cifar10_alpha_dropout_model/assets\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.3296 - accuracy: 0.5378 - val_loss: 1.4780 - val_accuracy: 0.4982\n",
"Epoch 9/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.2907 - accuracy: 0.5485 - val_loss: 1.5151 - val_accuracy: 0.4854\n",
"Epoch 10/100\n",
"1407/1407 [==============================] - 13s 10ms/step - loss: 1.2559 - accuracy: 0.5646 - val_loss: 1.4980 - val_accuracy: 0.4976\n",
"Epoch 11/100\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.2221 - accuracy: 0.5767 - val_loss: 1.5199 - val_accuracy: 0.4990\n",
"Epoch 12/100\n",
"1407/1407 [==============================] - 13s 9ms/step - loss: 1.1960 - accuracy: 0.5870 - val_loss: 1.5167 - val_accuracy: 0.5030\n",
"Epoch 13/100\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 1.1684 - accuracy: 0.5955 - val_loss: 1.5815 - val_accuracy: 0.5014\n",
"Epoch 14/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.1463 - accuracy: 0.6025 - val_loss: 1.5427 - val_accuracy: 0.5112\n",
"Epoch 15/100\n",
"1407/1407 [==============================] - 13s 9ms/step - loss: 1.1125 - accuracy: 0.6169 - val_loss: 1.5868 - val_accuracy: 0.5212\n",
"Epoch 16/100\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 1.0854 - accuracy: 0.6243 - val_loss: 1.6234 - val_accuracy: 0.5090\n",
"Epoch 17/100\n",
"1407/1407 [==============================] - 15s 11ms/step - loss: 1.0668 - accuracy: 0.6328 - val_loss: 1.6162 - val_accuracy: 0.5072\n",
"Epoch 18/100\n",
"1407/1407 [==============================] - 15s 10ms/step - loss: 1.0440 - accuracy: 0.6442 - val_loss: 1.5748 - val_accuracy: 0.5162\n",
"Epoch 19/100\n",
"1407/1407 [==============================] - 12s 9ms/step - loss: 1.0272 - accuracy: 0.6477 - val_loss: 1.6518 - val_accuracy: 0.5200\n",
"Epoch 20/100\n",
"1407/1407 [==============================] - 13s 10ms/step - loss: 1.0007 - accuracy: 0.6594 - val_loss: 1.6224 - val_accuracy: 0.5186\n",
"Epoch 21/100\n",
"1407/1407 [==============================] - 15s 10ms/step - loss: 0.9824 - accuracy: 0.6639 - val_loss: 1.6972 - val_accuracy: 0.5136\n",
"Epoch 22/100\n",
"1407/1407 [==============================] - 12s 9ms/step - loss: 0.9660 - accuracy: 0.6714 - val_loss: 1.7210 - val_accuracy: 0.5278\n",
"Epoch 23/100\n",
"1407/1407 [==============================] - 13s 10ms/step - loss: 0.9472 - accuracy: 0.6780 - val_loss: 1.6436 - val_accuracy: 0.5006\n",
"Epoch 24/100\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 0.9314 - accuracy: 0.6819 - val_loss: 1.7059 - val_accuracy: 0.5160\n",
"Epoch 25/100\n",
"1407/1407 [==============================] - 13s 9ms/step - loss: 0.9172 - accuracy: 0.6888 - val_loss: 1.6926 - val_accuracy: 0.5200\n",
"Epoch 26/100\n",
"1407/1407 [==============================] - 14s 10ms/step - loss: 0.8990 - accuracy: 0.6947 - val_loss: 1.7705 - val_accuracy: 0.5148\n",
"Epoch 27/100\n",
"1407/1407 [==============================] - 13s 9ms/step - loss: 0.8758 - accuracy: 0.7028 - val_loss: 1.7023 - val_accuracy: 0.5198\n",
"Epoch 28/100\n",
"1407/1407 [==============================] - 12s 8ms/step - loss: 0.8622 - accuracy: 0.7090 - val_loss: 1.7567 - val_accuracy: 0.5184\n",
"157/157 [==============================] - 0s 1ms/step - loss: 1.4780 - accuracy: 0.4982\n"
]
},
{
"data": {
"text/plain": [
"[1.4779616594314575, 0.498199999332428]"
]
},
"execution_count": 128,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tf.random.set_seed(42)\n",
"\n",
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=[32, 32, 3]))\n",
"for _ in range(20):\n",
" model.add(tf.keras.layers.Dense(100,\n",
" kernel_initializer=\"lecun_normal\",\n",
" activation=\"selu\"))\n",
"\n",
"model.add(tf.keras.layers.AlphaDropout(rate=0.1))\n",
"model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))\n",
"\n",
"optimizer = tf.keras.optimizers.Nadam(learning_rate=5e-4)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=optimizer,\n",
" metrics=[\"accuracy\"])\n",
"\n",
"early_stopping_cb = tf.keras.callbacks.EarlyStopping(\n",
" patience=20, restore_best_weights=True)\n",
"model_checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\n",
" \"my_cifar10_alpha_dropout_model\", save_best_only=True)\n",
"run_index = 1 # increment every time you train the model\n",
"run_logdir = Path() / \"my_cifar10_logs\" / f\"run_alpha_dropout_{run_index:03d}\"\n",
"tensorboard_cb = tf.keras.callbacks.TensorBoard(run_logdir)\n",
"callbacks = [early_stopping_cb, model_checkpoint_cb, tensorboard_cb]\n",
"\n",
"X_means = X_train.mean(axis=0)\n",
"X_stds = X_train.std(axis=0)\n",
"X_train_scaled = (X_train - X_means) / X_stds\n",
"X_valid_scaled = (X_valid - X_means) / X_stds\n",
"X_test_scaled = (X_test - X_means) / X_stds\n",
"\n",
"model.fit(X_train_scaled, y_train, epochs=100,\n",
" validation_data=(X_valid_scaled, y_valid),\n",
" callbacks=callbacks)\n",
"\n",
"model.evaluate(X_valid_scaled, y_valid)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The model reaches 48.1% accuracy on the validation set. That's worse than without dropout (50.3%). With an extensive hyperparameter search, it might be possible to do better (I tried dropout rates of 5%, 10%, 20% and 40%, and learning rates 1e-4, 3e-4, 5e-4, and 1e-3), but probably not much better in this case."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's use MC Dropout now. We will need the `MCAlphaDropout` class we used earlier, so let's just copy it here for convenience:"
]
},
{
"cell_type": "code",
"execution_count": 129,
"metadata": {},
"outputs": [],
"source": [
"class MCAlphaDropout(tf.keras.layers.AlphaDropout):\n",
" def call(self, inputs):\n",
" return super().call(inputs, training=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's create a new model, identical to the one we just trained (with the same weights), but with `MCAlphaDropout` dropout layers instead of `AlphaDropout` layers:"
]
},
{
"cell_type": "code",
"execution_count": 130,
"metadata": {},
"outputs": [],
"source": [
"mc_model = tf.keras.Sequential([\n",
" (\n",
" MCAlphaDropout(layer.rate)\n",
" if isinstance(layer, tf.keras.layers.AlphaDropout)\n",
" else layer\n",
" )\n",
" for layer in model.layers\n",
"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then let's add a couple utility functions. The first will run the model many times (10 by default) and it will return the mean predicted class probabilities. The second will use these mean probabilities to predict the most likely class for each instance:"
]
},
{
"cell_type": "code",
"execution_count": 131,
"metadata": {},
"outputs": [],
"source": [
"def mc_dropout_predict_probas(mc_model, X, n_samples=10):\n",
" Y_probas = [mc_model.predict(X) for sample in range(n_samples)]\n",
" return np.mean(Y_probas, axis=0)\n",
"\n",
"def mc_dropout_predict_classes(mc_model, X, n_samples=10):\n",
" Y_probas = mc_dropout_predict_probas(mc_model, X, n_samples)\n",
" return Y_probas.argmax(axis=1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's make predictions for all the instances in the validation set, and compute the accuracy:"
]
},
{
"cell_type": "code",
"execution_count": 132,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.4984"
]
},
"execution_count": 132,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tf.random.set_seed(42)\n",
"\n",
"y_pred = mc_dropout_predict_classes(mc_model, X_valid_scaled)\n",
"accuracy = (y_pred == y_valid[:, 0]).mean()\n",
"accuracy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We get back to roughly the accuracy of the model without dropout in this case (about 50.3% accuracy).\n",
"\n",
"So the best model we got in this exercise is the Batch Normalization model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### f.\n",
"*Exercise: Retrain your model using 1cycle scheduling and see if it improves training speed and model accuracy.*"
]
},
{
"cell_type": "code",
"execution_count": 133,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42)\n",
"\n",
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=[32, 32, 3]))\n",
"for _ in range(20):\n",
" model.add(tf.keras.layers.Dense(100,\n",
" kernel_initializer=\"lecun_normal\",\n",
" activation=\"selu\"))\n",
"\n",
"model.add(tf.keras.layers.AlphaDropout(rate=0.1))\n",
"model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))\n",
"\n",
"optimizer = tf.keras.optimizers.SGD()\n",
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=optimizer,\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": 134,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"352/352 [==============================] - 3s 8ms/step - loss: nan - accuracy: 0.1706\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEOCAYAAACKDawAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAu0klEQVR4nO3deXxU5bkH8N8TguwQE5IQdhDBIIoIolcqm0VxrVtdqN66VCrKtVbrVtuqvS612l4XXFFBiytaXBBQSwkoWhBEWYxBEFBZw04WAiTv/eOZ13NmMjOZCZMzkzm/7+eTz2xn5rxzkjznPc+7iTEGRESU/jKSXQAiIvIGAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPZCa7ANFkZWWZXr16JbsYaaO8vBytWrVKdjHSAo9lYiXzeJaUAJWVQHU1kJkJ9O+flGIkzOLFi7caY3LDvZbSAT8/Px+LFi1KdjHSRlFREYYPH57sYqQFHsvESubxHDYMKC4GSkuBrCygsYccEVkX6TWmdIjI14wBMnwSCX3yNYmIwqupAZo00fvpPvEAAz4R+ZoxDPhERL7gTukw4BMRpTGmdIiIfII1fCIin2AOn4jIJ9wpnXTHgE9EvsaUDhGRTzClQ0TkE+ylQ0TkE0zpEBH5BFM6REQ+wZQOEZFPMKVDROQTnB6ZiMgnmNIhIvIJNtoSEfkEc/hERD7BlA4RkU8wpUNE5BOs4RMR+QS7ZRIR+QRTOkREPlFTwxo+EZEv7NsHNG+e7FJ4gwGfiHytqooBn4jIFxjwiYh8wBhg716gRYtkl8QbDPhE5Fv79+stAz4RUZqrqtJbpnSIiNIcAz4RkU/s3au3TOkQEaU51vCJiHzCBvxmzZJbDq8w4BORbzHgNzARaSIiS0Rkutf7JiJyszl8BvyG8xsAxUnYLxFRkNAc/hVXJK8sXsj0cmci0hnAGQDuBXCjl/smIgrlTulUVQGZnkZE73n99R4GcAuANpE2EJGxAMYCQG5uLoqKijwpmB+UlZXxeCYIj2ViJet4LlqUA+AoLFu2CPv2lXm+f695FvBF5EwAW4wxi0VkeKTtjDHPAHgGAPr06WOGD4+4KcWpqKgIPJ6JwWOZWMk6nqWlejtkyCD06+f57j3nZQ5/CICzRWQtgFcBjBSRKR7un4goiG20ZT/8BDPG3G6M6WyM6Q7gYgD/NsZc6tX+iYhCsVsmEZFP+C3gJ6VN2hhTBKAoGfsmIrL8FvBZwyci32IOn4jIJ2wN/5BDklsOrzDgE5FvVVVpsBdJdkm8wYBPRL5VVeWf/D3AgE9EPrZ3LwM+EZEvVFX5p8EWYMAnIh9jSoeIyCcY8ImIfII5fCIin2AOn4jIJ5jSISLyCQZ8IiKfYA6fiMgn9u5lDp+IyBf27AHatUt2KbzDgE9EvrVrF9C2bbJL4R0GfCLypQMHgIoKBnwiorS3Z4/eMuATEaW53bv1lgGfiCjNMeATEfkEAz4RkU8w4BMR+YQN+OyHT0SU5ljDb2SM0R8iongx4DcyZ54JtGyZ7FIQUWO0ezcgArRqleySeCcz2QU4GDNmJLsERNRY7d4NtGkDZDTqam98Uvqr7tuXgexsYM0aYMcOYMqU8NsxrUNE8fLbPDpAigf8qqoM7NgBlJQAkyYBl10GfP+9vnbggLOdHSJNRBSr3bsZ8FNKdbUA0DPx6tX63MaNert+vbPdjh3O/TvvBF57zaMCElHK2bEDuPtuoLo6+nYM+CmmpqZ2wN+0SW/XrXO2cwf8CROA55/3qIBElHLefRe46y5g6dLo223f7q8++ECKB3x3Df/bb/W5zZv1du1aZzsb8Kuq9Jf4zTfelZGIUsuWLXpru12GU1GhJ4Sjj/amTKmiUQT87dudAB+thu9+rarKmzISUWopLdXbaG178+cD+/cDI0d6U6ZU0SgC/ooV+ssBnKBuG28BPSHU1Div1dQ4VwRuX3/NHj1E6S6WGv6cOUBmJvCTn3hTplSR0gHf5vCXLHGesymdrVuBrl31/tVXAxdc4DToAsDKlcGf9e67QGEh8MorDVhgIko6G/Cj1fAXLAAGDABat/amTKnCs4AvIs1FZKGIfCkiK0Tk7rreY2v4P/ygj7t2dWrx27YB3bsDTZro4w8+CO658803zlUBADz5pN5+8snBfhMiSmWxpHRKS4GOHb0pTyrxsoZfBWCkMaY/gGMAjBaRE6K9wQZ8a9Cg4Bp++/aavgGA8nKgqEiHSnfooIO0DjkEePppfc+sWbrdnDnAffcBe/cm8qsRUaqIJaWzYwdw6KHelCeVeBbwjSoLPGwa+ImaUXcH/Pz82jX8nJzgnPw77wB5ecCDDwJffqnPzZ6tDb7GaErnq6+AO+4Apk1L2FcjohQSWsOfMwcYNy54mx07gOxsb8uVCjydS0dEmgBYDKAXgMeNMQvCbDMWwFh9NPDH59u124OKii0oKzsM06d/hK1bh6C8/HsA3QAAmZk12LcvA61bl6FTp0W44YaOePjh3ti0qRSzZ28CcBT69fsexcVdAADPPrsZBQXFDfp9U01ZWRmKioqSXYy0wGOZWIk6npWVGaioGAoAKCnZiKKiEjz55GGYOrULBg78DJs2tcDxx29Defkw7Ny5BkVF6+r4xDRjjPH8B0AWgDkA+kXfbqBp0UInQT7nHGPeflvvz5yptw89ZCdINuaii/T2xBPNj04+2ZgTTjDm2Wf1tZISY+67z5jzzzcmK8uYfftMVI88YsykSdG3SaQZM4yprGy4z58zZ07DfbjP8FgmVqKO55o1Tkz4+c/1uauuch7n5hqzaZM+fvzxhOwy5QBYZCLE1KT00jHG7ARQBGB0Xdva6Y87dwb699f7//633ubkAC+/DDz0EPDii9ow++CDznsLCrTnjs3pdekC3H47MGYMsHNn3Q24f/878MwzcXyxg7B8OXD66cD48d7sjygd2f91wMnh79rlvLZzp3bjBpjDb1AikisiWYH7LQD8FMDX0d7Tps0BnH++3s/O1hx+Vpbm5QFttL3kEuCmm7SB9pprgBNPdN5vA/7mzToNaosW+vyoUbr9u+9G3ndlJfDdd9rf/803gc8/r9/3jpX945w3r2H3Q5TONmzQ23btnBz+zp16u3Wr9tyzvfkY8BtWAYA5IrIUwGcAPjTGTI/6hoLKH4N027baA6d/fyf45uTUscMCYN8+nW0zL895vk0bYPjw6AF/9Wq9MNywAbjiCr0yaEj2j9JdQyGi+BQHmuUGDXICvq3h28bcNWv01o+Ntl720llqjBlgjDnaGNPPGPPn2N6nt/aXY9M6QN0B3/az/fLL4IAP6GpZK1fqL7+8HPjVr4CBA7UXz9y5wFNP6XY1NfqH8/HHevKoj/vv155B0djLTPvHSUTx++orTf927Fg7pbNtm97agO/HGn7Kr3j1pz/p4KoxY/Tx2WcDjz6q99u3j/7eggK93bgRGDw4+DU7pHrBAr0CeP55TRcdc0zwgC2rogJYuDD+odjPPgv8/vd6/957I29nA77dF5duJIpfcTHQt69mBEJr+Ha6ZNbwU1hOjjaeNmumj92THWVlRX+vDfhA7Rp+v35A8+bAZ59p0D/qKG04HTtWXwvnyiu1T2883nxTb0WcQWLhuAM+Z/skil9NjRPw27TRGr4xta+a7TxbdcWPdJTyAT+UiI6avfHGutei7NRJG2cBIDc3+LWmTXUujYULNeXTv79eBk6Y4Myjba8gWrUC7rlHA/Frr8WX2ikp0VtjdL6fG24Iv5293AR0kjciis933+nVcWGhBvwDB7SWHzqqfu1avQKw07L4SaML+ABw6qnA3/5W93YtWwK33KL3beB3GzxYc/MbNgTPiy2ijairVukfRu/emoMfNEhPBnl5wCmnBE/BvH69pm/ctfjKSv3jOuoofTxtmtOl1M0YDfj2KsQu9kJEsbPdrPv3d1aysvNwuW3Z4s90DtBIA3487rwT+L//A37969qvnX66c9/dGAxot6527TTI27x99+7Af/6jl4gffgi89JLWIMaP14aiq6/W9JC1apUG8xEjnOfcE7wBerWQkaGzeHbtqvMArV6ttRUiit20afr/c9xxzkpW7oWS3Bjw60lEmiaiIA0lM1PTKB061H5t1Ci99ANqB3zrww+Bhx/W+z16BM/ds3y5jgl4/HGnveDzz/Xnlluc1Iw74G/frjV/y04GB2h7xWGHaQNyt27Av/4FfPQRcPzxwe8homCVlcCMGcA552gFyqZjI10t9+rlWdFSSlwBX0SuF5HzXY+fA1ApIiUi0ifhpWtgIppjf+212o26VkaG01bQvbveNmumDbslJc68+8XF+ke2ZAnw3HM64nfqVH1t6NDgz7SDQwCnbzCgtY7DDnMeP/YY8Prr2s5g2wISYckS4OKLNcdJlA6WL9f8/Smn6GP7/xypA8QRR3hTrlQTbw3/egClACAiQwFcCGAMgC8AxJBVTz0FBcCFF8a2bY8eenvEEdowtHKl/kHl5ekl5LHHau3eNvpOnao9BrKzgy8h3Wkd90CrVq2Cax7TpzuDw9as0auL4uKDT/dMn64nOfeqYUSN2datemuv5G0njVWrwm/PgB+bTgDWBu6fBWCqMeZ1AHcBiDq3fTqwNfzCQqBPHw3Cy5droy6gvX6WLQMWLXLec/31eutebOHPf9aAu2dPcEpn0yadBtp+ljHO2r1r1miaqG9fTfEczFKN9grDvW9ArzZC2xiIGgPbrdkOxqwr4PdpdPmIxIg34O8GYDs4jgIQmNUG+wE0T1ShUlX37trz59hjNchXV2sj7uGH6+tDh2qaZO9eHcB12GHAf/+3vtarl9NbZ/ZsTam0bQv89rfO57dpo8Ec0EVa3I3K776r4xEAPTF88YUzHUMkGzaEPzFECvjjxmkOlKixsd2a7ZV0ixZ6xWwDfmgvPVtJ85t4A/4HACYGcve9AMwMPH8kgDWJLFgqatFCa/TXXx98SWj/eEa75v586in9Y7NzAT3zDPD++87rF1+stzt26JiAKVM0Z9+/v3b3HD1aR+gOGKAnmqIiPdnYKcOPPVaHhkcaCLZ6tb7PLu3oZmvxofP2fPutpqM++ghYvLju40FUl5dfbriJB/fvB4YN0xRluBkw8/K0wtO8uXPlbPltLVsr3oB/HYD5ANoDuMAYY8eHHgvAF8uD9+ihjbYDBzrdNW3wz8gAnnhCaxm2Nm/l5gaP/H35ZeC88/R+Xh7wi184l6O2NnLiifrPcuSR+viss2pP7XDrreHL+dxz+g/x1FO1a/mRavgbN2o30aFDtTsq5/Whg2GM/l0PHFj3tvv3S9yVjHXrdHbZs87SGn5WVvBgKpvWOeIIpzfelCnBKVe/iSvgG2N2G2P+xxjzM2PMLNfzdxpj7kt88VJXRobWrt95BzjjDOf5ceP0jy/SXDjTpztr79org7omcbI18nPPDf6DPvts7Wd8ySXBI3gPHAAmT9ZazLJlTg2rsjIDTz6pgR0IDvjV1bVr/A88EL1cVD9ff+2PNZVDKxTRPPxwbwwaFF8bkm3fAoD582tPpmgD/pFHOv+P/frFdgJKV/F2y+zr7n4pIqNEZIqI3B5YvtBXMjO1dtE0jpEIZ5yhl6GA03BUURH9PTffrCeIU0/Vx3PnApMmab6/tFRPOk884QTymTP1/qOP6gnCrt87a1YHXHut87nuf8jS0uBRwk2bandQSqzKSk3TTZyY7JIk3v3363xTVl0jxjds0DEqW7YA8+drtHbPKVUXd8BfsqT2YKrmgVZFd8D3+6SE8aZ0ngMwAABEpDOAtwFkQ1M99yS2aOnP1vB37Ii+3ZgxGoztcPGhQ4HLL9fBWYCeMPbv13x9dbWmcfLzgUsv1bTQxIk6Kdy8ecETCrlr9HZxeEAvf08/PXi8AKBtC5dd5iwQD+jlcb9+weMJ3P7yF+2VRGr7dq3dp+NI6jlztLJh2YAvEn77RYv0avezz4BduzSPWdf/gtu6dfrZNl0TWsO3XTX79nUCfatWsX9+Ooo34BcCsE0wPwewwBhzOoDLAFySyIL5gQ347jl54tG1q3M/J0cD/dFH64jDX/1Ka+mjR2tgnzgR+OILJ3fUpYvW8KdM0bYEe3UwcKA2KHftWvvy+osvdPtjjnGuBqZNA1as0BHJobZsAe66S6egiNfChToSOt1SH6Fzs6eT7dv1d26nIbYB31ZUQtlj4a5sxFvD79jRaS8LreHb/fbsyRq+FW/AbwLAzhV5MoAZgfurAeSHfQdFlJOjPX7cvXfiYWv4gM61X1qqI3InTdJAC2jeH3DGAVxwgZ5oRozQgD9tmv7YWvvUqdqjqFMnnV62rMzZh11NCADee09v7YRVc+fWLt8TT+jJbONGPal07gx8+mls3+3993VqCTuSOZJNm5x5zxsD25U2HQP+tm1aEbBXezbgl5WF7x5sFyhx95WPFPDffrv2/8m6dfo/YDtNhNbwJ04Enn5ar0BZw1fxBvzlAMaJyEnQgG8bbjsB2JrIgvmBCPDII/EvqmJ17Kg5+sxMzZ2OGaOfd/nl+hygg8TKy4EPPgD69NmNRx7Rk8Jhh+nl8xdf6HbTA4tN2pGK9gThruUXF2tjdatW2m5w4ICT5w8N+JWVGvAzMjQgT5umn/W730X+PhUVOv6grMxJedi5y8OpqdF2DDu4rTFI5xq+/U72atEG/OpqJ7i7hVvHOVLAv+WW2qlBG/Dt6PTQ6dLz8zWVKaIBPzMzvva2dBRvwL8VwNUAigC8YoxZFnj+bABs4vNYZqbWxHv00D/kl14Crruu9nYtW2rD1VNPff5jILeXwTagzp+v00PYcQOdOumtO4//1Vf6OaNHa672yy81SA8cqCcR+48OaFlKS/XkYx1yiNbwI+X733tPp6G++urYAv6CBbqdHZswb17qB9J0reHv2+dcadkUjbuy8P77tQcK2pOfvUoEwgf8ffv05OGe6ri6WqcG6dbN6WMf6e8K0BSl+4rYr+LtljkPOtK2vTHG1R6PpwGMS2TBKDb/9V+1J2eL9X2h3BO32YC/fr02CF90kV4FFBYCp52mz9vc/I036q27pjZ1ql5qX+Jq2bnuOr20z8vTSeEAvRK47DKdk8imj1591UkxRQv4tvfR2rU69cSwYc7YhkTbskWDhnv66/pI1xq+O1DbE//27U7t+6KLanfzddf6MzNrkJUVPuCvXq0BfsMGp31g40a9wuzWzVm+NNqV8s03czAhUI/pkY0x1dAZMvuJyJEi0twYs9YYs6XON1PCvfqqLrwSrw4dnLmB7DiAxx93XrcB/4EHtH3ABuiePYHhw/X+yy/rVcZ552lPCVvT3r9fF5YZNSp4DqFzz3Uaqt95x7mdMgX44x+Da2i2y2i0gD9jhjMrop2ZdPnyGL58PSxcqDXK2bPr3jYaW8vdvv3g5kPyWk2Ntqncdlv4Fd/cJ7BNm3Sb8nJn2hGg9ohb98C+Ll0q0KFD+IBv244OHHB6ltkumd26aY5+/Xrgmmsil/+QQ5w58v0s3n74mSLyIIAdAL4EsAzADhH5a6rPi0+1nXii3s6fr7n8E1zT39mh58uXA3ffrWmje+/VhWR69NA8/ubNelXQvLnWrmwef9EiTfUMGxY8urhHD93PwIHOP64N0G3aaDe60DxspIC/Z4+mmK66StNZr72mzzfUP/WKFXr71VcH9znuBbV379ZAecop4VdmAvQkfNZZB7fPeFRU1J42e9EiDayjRmkFINz4DHfA37jR6V7pnv3V3Z0XCK7h9+hRjuzs8AHfveSnneHVHfABrVhE6v5Jjnhr+H8FcCmAawD0BnA4NJVzGYD7E1s0amjXXquNqIWF4ReAeeABHUwzdKgG+9//Xq8KMjKc6R5sjf2kk7QmtnOnU9MfOlSHuzdvrlcCBQXaRmC7fJaWOnMBbd+uAb9jR2dx6V69NFVTXa2plOefd8r2+edaQz7pJO1nbWuP0QL+/v31OkwAnEAfS8DfsqX2SktVVVpedx572zbtg/7hh3rM5szR7rLu8RHvvacN7om+Gvjf/w1emMcaMEDHTlg7d2qbTWamM9FfuHSUfU5Ea/g2cLsD/ubNwYP93DX8goK9yM4O3w/fHfDtidG28TAvH594A/4YAFcZY14wxqwO/EwG8CsAv0h46ahBDRmiC7VEcsstegk/d25wLh7Qy2jACfj20n3dOt2+Xz8d2i6igb5LF2daiI4dNSB27apXF4D+A5eW6iIyffvqcyNGaGqguFivPq66ytm/nQ9l4MDgeYvc3e62bAF++lOtcb71lqauQqePiJWt4RcXO3nkSHr3dtZOAPRE1qqVnjBDA74NgGvWaGrrhx80dWKVlOgxSPS8Rn/6k55k3J+7b592g125Un9v99yj33vbNh21bXtDhQv4NsD37Kk5d/vY3S4EOGtFAME1/Kys/RFr+EuW6NxOgBPw163Tbph+72YZr3gDfjton/tQqwFkHXRpqNGwQdYGfDsIbPVqzd/b6SMAnULCXhEA2j5QUeEMqho8WP+Bt27VgP/SSzpw7Kab9HV3l88NGzRoL1qk+8zLc04+QPC4gU8/1Zz7/Pna1lFerlcCS5YAf/1r5O9mR8NaNTUa6LOz9flI66RaNoja1Mj48XqSePppfc2mHkID/qxZwd+3qsrZl91u5crgNJcxehVW37aLjz927tuToU2b/PGPTgNs165OP/doNfwrrtDj+9Zb+ti9vkOTJjrX1LJA377du3XW1+7dgREjtoQN+Dt36knnrLN00kJ3Soe1+/jFG/C/hK56Feo3gdfIJ2y+f8AAvbUB/623NLDahl1AG3dffNF57G7IXbVKJ4ErLdVafm6uBoCJE/Vk0rmzLvBuXXCBbr98uZOGctfw3TVo2y3wu+80LQLo+8aN01lGw9Waly3TwGZPNoDWKisqdL9AcIohGhusbY29pkbLZxvE3QF/xgw9mTVr5qTEVq1yRjTbYNynT3Ctec8evXKYNEnL1a+fkybbtk2DdrilLG2wdJ9MbVncK6HZgF9QoO06TZtGDviHHKJpn/x84KGH9PnsbC37p58Cb76pFQI7/cKuXdpbbM0aICdnH7Kz9STgTr0tWKAntSFD9G+BAf/gxBvwbwHwSxFZKSIviMhkESmB5vWjDKmhdHPCCfpPd9xx+jgvT//h//lPfezuKnroocEzgtqA16KFpj7sP+6GDc7i04DWhIcOddI+gAaApUu1lmsDX6SAby//33rLyQ0/95zTtTI0cL/xhtO1zy4tCTgNhPY7/fADUFLSBmefHX2U74QJ2pV12zYNVrt26QnHTpq3fr0TZO3t+PFai//+++BRxuFmnqysdALgmjW62M6KFXoCMEZ/F/fcE34+ensFM2WK0y5h9+FuQJ43T/P37dvr7yMnxwn469c7jeXbtulrLVsCJ5/svD87W0/izZoBP/uZvr5li5Zv9+7gNhfbwO8e+/HJJ9pmdPzx2h6wcqWzEhwDfvzq0w+/N4CpAFoDaBu4fyrC1/wpjbnn8snI0Dx9ebnm4CMtCg84Nfy+ffV97s+xU9paY8cGP66p0UBXUaH5YkCD6THH6D5373ZqxTZw2bTFkUdqkLezKLqnigA0UHbtqnn/ykrneRvwBw/W8paUANdcMxDvvht8MnIfC0BHPdseNnYK7T179Aqmc2ftteJuUxgwQHtBAVobdi9cH9r2cOCAfudxgdEvs2drA/AJJ+gqbJ995izg7R4QB2jA3L5dZ1+trnbaRmzAd8/t9MYb2oXXfqf27Z1JyS65ROddWr/eCfhA8PKBofPo5OXpd6ms1O/gft3+Pt0pKzs5X+vW2rng66/1iq283L/LFB6M+vTD32CMucMYc74x5jxjzB8AlAM4P/HFo8akSxe9dadzwrEB3+bejz7aec1dwwe0LeC773QulVC2YVRE88a33abBzNa6bcA3Rrt9jhqlj2+9Va9Giot1da/XX9ca7+rVOqbg1FM1qNl8su0R0qOHpivcvYUWL9bUjfuKoFmz2mU97TTnfrt2GqyXLAmuuZ92mjZ+9++v4wpKSpyG782bg3vqLF6std2PPtLHtgH05pv1dsWKyAHfzq46cqSeYBYu1KuPSPPXu7vWumv49nM//jg44LtXgwvtZpuXp/ux5a0r4K9a5bQTHXGEnijsnDqFheHLS5HFHfCJIrE1dXeDbTjt2jm1Q0C7Ydp0Sbhg2aVLcMC0bICwbHdOm9ZxpyaOOEI/46ijtLfJ4Yc7/fivukrv19Q4C9QDTkrFti20bKnpKG14NejcWU8WPXpofn/jRg2k7qsDq39/Xf2pc2dNGw0YoAF97VrtadSkiTNK+LzzNJXxn//oVVBOjtaKy8udzwt3AhRxTrYbN0YO+PZElp2tPaFqavTEERrwberN3ebiDvg2WNspLcIF/FD5+fpdbPuJO6XTubOmj9xz8Kxd6/yebYC3I6wZ8OPHgE8J07OnBp26Ar6INuS61wCePFn71Lvzv25Nm9aeDdGOFLZs8Ni5U2vD7oDfp48OcFq6VANdYaHm1+2UDpMn63aFhU6N8tFHNQ309NPOycwGwby8KgwZEtzN8OOPnZrrcccBF17ofN9OnTRf/v33OuLYTjG9e7eeiLZvd1ZistNefP21ltumQdx91MMF/J499btlZel3t4HTBvyaGp3byLZ9HHqo7qtZM21UtycIy/4u3FddOTkahKdOda58QgO+e3RtqLw8rcHbhXjcNfzMTM3Lz5ypv4/167WrqC2vDfCzZum+QtN/VDcGfEqY8eN1EFHogtGx6NFDA0e0hjj7uS1aaF45dG5zW8OfNUtrq5WVzuyfobXOMWP0NjdX0w6PPaa3vXtr4GzSRHsH2Tx/aMDv0qXix8A2YYKWZeZMp7/+uHF6wgA0JRI6S6Od/wXQIOgOfLbnE6ABPz9fv4+7QTrcADDb9bWgQHPftmHWBvynngJeeMHpBZOdre0Zv/iFBnA75bV10kl66+6Vk5OjKaELL9S0V2am01ffBnw7AV84tp3l3//WOZRC54Gyo7GvuAL4+9/1ORvwc3Odk0/nzhxZWx8xBXwReSfaD4BHGric1Ajk5ESuoSdCfr4G4lGjtIYcygb8224DfvlLvX/88Xob2sB37rlaq1661Olp1KGDBqumTXVKg3/8w1lXwOai3QH/ppv0SuXaazVVNGmSc3XTrp0Gt6wsp23DrXNnLaf7M6327Z332IC/aVPtUag2wNq2DDtgraBAG20BHZhkA77trmnZnlPPPhs8jxKgtf4hQ/S+PT6Ac4ytESP0aurAgeArsOnTnfYFN3dlYMKE2o269qTWurU2egPB3VDt+Im62okovMwYt6trbr9tANYcZFmIoureXVMVL70UfqoBdzD64AN9fOed2svGBi83G/CeeUa3c88lZHvLFBfrazbouAN+VpYzAvnKK4Nn0mzbVmug559fO6Bb992n+Xp3jd469lhN//TurVc9b77p1LTPPVdPTFdeqT2KBg/WK5ZLL9XX3Y2sI0c6s0QuXaptCXZOG7tClIh+3zfe0MD90Ud6wiosdNZOsEInThs61FntzB3wba+kUO7eW+FWwrrzTr0yuv12LTsQfMK84gpNzYWeeCg2MQV8Y8wVDV0Qoro88IA29tmJ3UK5+/oDwA03aDBdsiT65x59tNMQGKqwUN9vrxBs8OvZszxou7FjNQhddJE+tu0J0WYyFQmuPbuNHKmDlXr00BTT/v3OaNqHHtLnqqv1WPTsqf3tLRvwu3TRK48ZM7Tn0jffAH/4gxPw3cerSRNNs8yc6QR8wGnPsK67TtNAn3yi4xvc7TWhSwyGY2v4kdJ+Z56pP4C2aWzb5kzJYUU6gVLdYq3hHzQR6QLgRQAdANQAeMYYw1QQxSw3N3pD3aGHaorl5JM1x21nAz1Y7vTRkCHae6aiovYwXXf7Q6R1XGM1frwuBJOZ6fRSsTV1G6ibNNGyhAZAG/B79tT71dV6QjNGrxyscCdOW+5I5c/O1q6fVVWau3dPjhbaqB6OXXA89EQSzqef1j1vEcXHs4AP4ACAm4wxn4tIGwCLReRDY8xBTjhL5LAplmgDvw6GiLYL2OkP3NwB/2Cnac7IcBo/bcC3I2bdwdg9R5FlJxTr2FF7ALVr57RpuMc8hGv0tJ9dV/mbNXNOHm3bakNsLAF/wACdfsHOvBlN06ZckjDRPAv4xpiNADYG7u8RkWLoWrgM+JQW6spP11fXrlqb//57DcShKY5Q9sQzerSmoGbO1JRPQYG2gyxbFnmdgVgDvluXLlrbjyXgZ2Y6vW/Ie17W8H8kIt0BDABQa8E4ERkLYCwA5ObmoihcVYrqpaysjMczQSIfy+EAgAULihLabTA//3hs2NACzZvvRVHRf6Ju27w58NxzrdC1a/mPVyJ2BK5dhrJt2/BXKXv2ZAL4CSorN6KoqKT2BmG0anUUgBwsWzYXTZrUb+J+/m16Q4zH66yJSGsAcwHca4z5Z7RtW7ZsaQa7OyzTQdm5cyey2L0hISIdy7lziwAAw4YNT+j+vvzyIezcOQgtW67FccddntDPdjOmCebNm41Ond5Ar14TYnrPN9/8BqWlI3DiiefUe7/820ycuXPnLjbGDAr3mqc1/MAyiG8CeKmuYE/UGOXnz8KOHWH/1w5Kt24vIiNjH9q1W1r3xgdBpBrt289DVtYXMb+nW7d/ID///YYrFCWMZzV8EREALwDYboy5IZb39OnTx5SUxHZZSXUrKirCcI5YSQgey8Ti8UwcEYlYw/dyaoUh0LVvR4rIF4Gf0z3cPxGRr3nZS+djAJz9gogoSTh5GhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU94FvBF5HkR2SIiy73aJxERObys4U8GMNrD/RERkYtnAd8YMw/Adq/2R0REwTKTXYBQIjIWwFgAyM3NRVFRUXILlEbKysp4PBOExzKxeDy9IcYY73Ym0h3AdGNMv1i279OnjykpKWnYQvlIUVERhg8fnuxipAUey8Ti8UwcEVlsjBkU7jX20iEi8gkGfCIin/CyW+YrAD4F0EdEfhCRq7zaNxERedhoa4y5xKt9ERFRbUzpEBH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+QQDPhGRTzDgExH5BAM+EZFPMOATEfkEAz4RkU8w4BMR+YSnAV9ERotIiYisEpHbvNw3EZHfeRbwRaQJgMcBnAagL4BLRKSvV/snIvI7L2v4gwGsMsZ8a4zZB+BVAD/zcP9ERL6W6eG+OgH43vX4BwDHh24kImMBjA08rBKR5Q1crnYAdjXwe+vaLtrrkV4LfT7cdqHPtQewNWpJD15jPJ71ec6LYxmpHIl+L49nYt+XjOPpftwt4p6NMZ78APg5gGddjy8D8Fgd71nkQbmeaej31rVdtNcjvRb6fLjtwmzD4xnDcYrlOS+OJY9n6hzPeN6XjOMZa/m8TOn8AKCL63FnABs83H8k73rw3rq2i/Z6pNdCnw+33cF8t/pqjMfzYJ5raDyeiVXffcbzvmQcz5jKJ4GzQ4MTkUwAKwGcDGA9gM8AjDHGrIjynkXGmEGeFNAHeDwTh8cysXg8veFZDt8Yc0BExgN4H0ATAM9HC/YBzzR8yXyFxzNxeCwTi8fTA57V8ImIKLk40paIyCcY8ImIfIIBn4jIJxptwBeRViKyWETOTHZZGjsRKRSRp0TkDREZl+zyNHYico6ITBSRt0XklGSXp7ETkZ4i8pyIvJHssjR2ngd8EXleRLaEjqCtx8RqtwJ4vWFK2Xgk4ngaY4qNMdcAuBCAr7vGJeh4vmWMuRrA5QAuasDiprwEHc9vjTFXNWxJ/cHzXjoiMhRAGYAXjTH9As81gfbRHwUdoPUZgEug3TfvD/mIKwEcDR2K3RzAVmPMdG9Kn3oScTyNMVtE5GwAtwGYYIx52avyp5pEHc/A+/4G4CVjzOceFT/lJPh4vmGMucCrsqcjL+fSAQAYY+aJSPeQp3+cWA0ARORVAD8zxtwPoFbKRkRGAGgFnXWzUkRmGGNqGrbkqSkRxzPwOe8AeEdE3gPg24CfoL9PAfAXADP9HOyBxP19UmJ4HvAjiGliNcsYcwcAiMjl0Bq+L4N9FHEdTxEZDuA8AM0AzGjIgjVScR1PAP8D4KcA2olIL2PMUw1ZuEYo3r/PHAD3AhggIrcHTgxUD6kS8CXMc3XmmowxkxNflLQQ1/E0xhQBKGqowqSBeI/nowAebbjiNHrxHs9tAK5puOL4R6r00knVidUaKx7PxOLxTCwezyRJlYD/GYDDRaSHiBwC4GIA7yS5TI0Zj2di8XgmFo9nkiSjW+YrAD4F0EdEfhCRq4wxBwDYidWKAbwew8RqBB7PROPxTCwez9TCydOIiHwiVVI6RETUwBjwiYh8ggGfiMgnGPCJiHyCAZ+IyCcY8ImIfIIBnygCEbkrdFpfosaM/fApqURkMoD2xpiUmyVRRFoDaBaYyyUliYgB8HNjDBcHoTqxhk++ExjOXydjTFkygr2IZATmjCdKKAZ8Smki0ldE3hORPYGVk14RkQ6u148TkQ9EZKuI7BaRj0Xkv0I+w4jIdSLyTxEpB3CfTdeIyMUisjrw+W+JSHvX+4JSOiIyWUSmi8hvRGS9iOwQkUki0tK1TSsReVFEykRks4jcHnjP5Cjf8fLA9qcH9rcPQGFd301E1gbuTg18x7Wu184SXQJ0r4isEZF7Yz3RUfpiwKeUJSIFAOYBWA5dNOOnAFpDF2qxf7ttAPwDwEmBbb4AMMMduAPuhM71fxSAxwPPdYcuQXgugFMADIDOux7NSQD6Bcpi3/sb1+t/AzAs8PxIAP0D76lLcwB/APBr6MI+62L4bscFbq8GUGAfi8ipAF4CMAHAkdBV4i4AcF8M5aB0ZozhD3+S9gNgMoDpEV77M4DZIc8dCp07fXCE9wiAjQAudT1nADwWst1dAPYCaOd67g7oSkzubZaHlPV7AJmu5yYC+Ffgfmto7fxi1+utAOwAMDnKMbg8UMaBdRyrSN/tgpDt5gH4Y8hz50CXGpRk/875k7wf1vAplQ0EMDSQ7igTkTI4KyUdBgAikiciT4vIShHZBWAPgDwAXUM+a1GYz19njNnlerwh8N5ovjI622O49xwGoCmAhfZFY0w59AqlLgegNfgfxfHdQg0EcEfIcXsZevLpEP2tlM5SZcUronAyALwH4HdhXtscuH0BQD6A3wJYC6AKwGwAofnq8jCfsT/ksUHdac5o7xHXc/GqMsZUhzwX63cLlQHgbgBTw7xWWo+yUZpgwKdU9jmAC6E18dBAa/0EwPXGmPcAQETyofnsZFgFPSEMBrAmUJ6W0Jz/6np8XizfbT+A0B49nwM4whizqh77pDTGgE+poK2IHBPy3E5o4+rVAF4TkQegtdOe0JPATcaYPQBWArhURBZAUxZ/hebRPWeMKROR5wE8ICJbofn2P0Br3PWp9cfy3dYCOFlE5kKvEnZA2z6mi8g6AK9D00X9oO0et9SjHJQmmMOnVHASgCUhPw8ZYzYAGAKgBsAsACugJ4GqwA+gPVBaA1gM4FUAz0ODYLL8DsBH0CX75gBYCm0/2FuPz4rlu90EYAS0bWMJABhj3gdwRuD5hYGf2wB8V48yUBrhSFuiBiQizaBdLB80xvwt2eUhf2NKhyiBRGQAgEJorboNgFsDt68ls1xEAAM+UUO4EUAfOF0thxpjfkhqiYjAlA4RkW+w0ZaIyCcY8ImIfIIBn4jIJxjwiYh8ggGfiMgnGPCJiHzi/wEx1+eWoFC05AAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"batch_size = 128\n",
"rates, losses = find_learning_rate(model, X_train_scaled, y_train, epochs=1,\n",
" batch_size=batch_size)\n",
"plot_lr_vs_loss(rates, losses)"
]
},
{
"cell_type": "code",
"execution_count": 135,
"metadata": {},
"outputs": [],
"source": [
"tf.random.set_seed(42)\n",
"\n",
"model = tf.keras.Sequential()\n",
"model.add(tf.keras.layers.Flatten(input_shape=[32, 32, 3]))\n",
"for _ in range(20):\n",
" model.add(tf.keras.layers.Dense(100,\n",
" kernel_initializer=\"lecun_normal\",\n",
" activation=\"selu\"))\n",
"\n",
"model.add(tf.keras.layers.AlphaDropout(rate=0.1))\n",
"model.add(tf.keras.layers.Dense(10, activation=\"softmax\"))\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=2e-2)\n",
"model.compile(loss=\"sparse_categorical_crossentropy\",\n",
" optimizer=optimizer,\n",
" metrics=[\"accuracy\"])"
]
},
{
"cell_type": "code",
"execution_count": 136,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 2.0559 - accuracy: 0.2839 - val_loss: 1.7917 - val_accuracy: 0.3768\n",
"Epoch 2/15\n",
"352/352 [==============================] - 3s 8ms/step - loss: 1.7596 - accuracy: 0.3797 - val_loss: 1.6566 - val_accuracy: 0.4258\n",
"Epoch 3/15\n",
"352/352 [==============================] - 3s 8ms/step - loss: 1.6199 - accuracy: 0.4247 - val_loss: 1.6395 - val_accuracy: 0.4260\n",
"Epoch 4/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 1.5451 - accuracy: 0.4524 - val_loss: 1.6202 - val_accuracy: 0.4408\n",
"Epoch 5/15\n",
"352/352 [==============================] - 3s 8ms/step - loss: 1.4952 - accuracy: 0.4691 - val_loss: 1.5981 - val_accuracy: 0.4488\n",
"Epoch 6/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 1.4541 - accuracy: 0.4842 - val_loss: 1.5720 - val_accuracy: 0.4490\n",
"Epoch 7/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 1.4171 - accuracy: 0.4967 - val_loss: 1.6035 - val_accuracy: 0.4470\n",
"Epoch 8/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 1.3497 - accuracy: 0.5194 - val_loss: 1.4918 - val_accuracy: 0.4864\n",
"Epoch 9/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 1.2788 - accuracy: 0.5459 - val_loss: 1.5597 - val_accuracy: 0.4672\n",
"Epoch 10/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 1.2070 - accuracy: 0.5707 - val_loss: 1.5845 - val_accuracy: 0.4864\n",
"Epoch 11/15\n",
"352/352 [==============================] - 3s 10ms/step - loss: 1.1433 - accuracy: 0.5926 - val_loss: 1.5293 - val_accuracy: 0.4998\n",
"Epoch 12/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 1.0745 - accuracy: 0.6182 - val_loss: 1.5118 - val_accuracy: 0.5072\n",
"Epoch 13/15\n",
"352/352 [==============================] - 3s 10ms/step - loss: 1.0030 - accuracy: 0.6413 - val_loss: 1.5388 - val_accuracy: 0.5204\n",
"Epoch 14/15\n",
"352/352 [==============================] - 3s 10ms/step - loss: 0.9388 - accuracy: 0.6654 - val_loss: 1.5547 - val_accuracy: 0.5210\n",
"Epoch 15/15\n",
"352/352 [==============================] - 3s 9ms/step - loss: 0.8989 - accuracy: 0.6805 - val_loss: 1.5835 - val_accuracy: 0.5242\n"
]
}
],
"source": [
"n_epochs = 15\n",
"n_iterations = math.ceil(len(X_train_scaled) / batch_size) * n_epochs\n",
"onecycle = OneCycleScheduler(n_iterations, max_lr=0.05)\n",
"history = model.fit(X_train_scaled, y_train, epochs=n_epochs, batch_size=batch_size,\n",
" validation_data=(X_valid_scaled, y_valid),\n",
" callbacks=[onecycle])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"One cycle allowed us to train the model in just 15 epochs, each taking only 2 seconds (thanks to the larger batch size). This is several times faster than the fastest model we trained so far. Moreover, we improved the model's performance (from 50.7% to 52.0%)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
},
"nav_menu": {
"height": "360px",
"width": "416px"
},
"toc": {
"navigate_menu": true,
"number_sections": true,
"sideBar": true,
"threshold": 6,
"toc_cell": false,
"toc_section_display": "block",
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 4
}