{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST classification using keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal of this exercise is to train a simple MLP on the MNIST digit classification dataset, the \"Hello World!\" of machine learning. MNIST was created by Yann LeCun to benchmark supervised learning algorithms. State-of the art is at 99.7% accuracy on the test set (using convolutional deep networks). See this link to see the different approaches : <http://yann.lecun.com/exdb/mnist>\n",
    "\n",
    "In MNIST, each input is a 28x28 grayscale image representing digits between 0 and 9. The training set has 60.000 examples, the test set 10.000.\n",
    "\n",
    "Instead of programming explicitly the MLP like in the previous exercise, we will now use **Keras** <https://keras.io>, a high-level API to **tensorflow** <https://tensorflow.org>.\n",
    "\n",
    "You need first to install `tensorflow` 2.x if not done already. On Colab, tensorflow is already installed. Even if you are using Anaconda, it is recommended to install tensorflow with pip:\n",
    "\n",
    "```bash\n",
    "pip install tensorflow\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.7.0\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:** Read the documentation of keras at <https://keras.io/api/> to get an overview of its structure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You are provided with a basic poor-performing keras model to get you started. The goal is to extend this model in order to obtain a satisfying accuracy on the test set. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data preprocessing\n",
    "\n",
    "The first step is to download the MNIST dataset. You could download the raw data from <http://yann.lecun.com/exdb/mnist> and process it, but that would take a while.\n",
    "\n",
    "Fortunately, keras comes with a utility to automatically download MNIST, split it into training and test set and create nice numpy arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "(X_train, t_train), (X_test, t_test) = tf.keras.datasets.mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Have a look at the doc of `tf.keras.datasets` to see what other datasets you can simply use.\n",
    "\n",
    "**Q:** Print the shape of the four numpy arrays `(X_train, t_train), (X_test, t_test)` and visualize some training examples to better understand what you are going to work on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data: (60000, 28, 28) (60000,)\n",
      "Test data: (10000, 28, 28) (10000,)\n",
      "x: [[  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0  83 149   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0 104 245   9   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0  32 254  40   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0  66  68   0   0   0   0   0  15 232 144   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0 166 128   0   0   0   0   0   0 135 208   4\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0 181 128   0   0   0   0   0   0  64 254  88\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0 240 105   0   0   0   0   0   0  24 245 126\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0 232  30   0   0   0   0   0   0   0 136 190\n",
      "    4   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0  36 230   0   0   0   0   0   0   0   0  96 255\n",
      "   41   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0  60 236   0   0   0   0   0   0   0   0  23 254\n",
      "   92   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0 102 236   0   0   0   0   0  20   1  21  85 254\n",
      "   92   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0 133 244 104 150 177 222 252 227 213 188 189 167\n",
      "  165   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0  58 251 205 191 157 117  44   7   0   0   0 129\n",
      "  227   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0  10   3   0   0   0   0   0   0   0   0 129\n",
      "  241   6   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 129\n",
      "  254  59   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 129\n",
      "  254  59   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 129\n",
      "  247  31   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 129\n",
      "  240   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 129\n",
      "  246  27   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  93\n",
      "  193   8   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]\n",
      " [  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0]]\n",
      "x (shape): (28, 28)\n",
      "t: 4\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATcAAAEkCAYAAABKYHYHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAVbElEQVR4nO3df6hc5Z3H8fcnicmCv4hEYxqjcdu4NPZHXFJbyCKxYo1SiNJa4oJElEQhggFL/fFPLSVFlqpbyhq4rqlZahsD6hrUaqMIrv2hSSRoYmINGu1NbnM3GhqhGkn87h9zLjuNM8/cmTs/zjzzecFwZ85zzpxvp/rxOec55zmKCMzMcjOp1wWYmXWCw83MsuRwM7MsOdzMLEsONzPLksPNzLI0pdcFmFl5LVmyJA4ePNj0dtu2bXs2IpZ0oKRxc7iZWV0HDx5ky5YtTW83adKkGR0op7kael2AmZVbRDT9SpE0R9ILknZJ2inplmL5XZL2SdpevK6o2uYOSXskvSnpsvHU7Z6bmSV14C6mo8CtEfGqpJOBbZI2F233RcRPq1eWNB9YBpwPfA54TtJ5EXEstROHm5nVNZ6eWAvfOQKMFO8/lLQLmJ3YZCmwISKOAO9I2gNcCPwhtR8flppZUouHpTMkba16raz13ZLmAhcALxeLbpb0mqR1kqYXy2YDf67abJh0GALuuZlZAy323A5GxMLUCpJOAh4FVkfEYUlrgR8DUfy9B7geUK2yGhXgcDOzpE7MHCTpBCrB9nBEPFbs50BV+wPAk8XHYWBO1eZnAfsb7cOHpWaW1IHRUgEPArsi4t6q5bOqVrsK2FG83wQskzRN0rnAPOCVRnW752Zm3bYIuBZ4XdL2YtmdwDWSFlA55NwL3AgQETslbQTeoDLSuqrRSCk43MwsoUOjpS9R+zza04lt1gBrmtmPw83Mkvp1tm6Hm5klOdzMLEsONzPLksPNzLLTiQGFbnG4mVmSw83MsuRwM7MsOdzMLDs+52Zm2XK4mVmWHG5mliWHm5llaSDDTdIS4GfAZOA/I+LuBuv3569kloGIqDUTR6NtBi/cJE0G/gO4lMpMmVskbYqIN9pVnJn1Xr+G20Rm4r0Q2BMRb0fEJ8AGKk+pMTPruYkcltZ6Is3XJ1aOmZVNv/bcJhJu43oiTfFIr5qP9TKz8hvEcBvXE2kiYggYAg8omPWjQQy3LcC84mk0+6g87v5f21KVmZXCQI6WRsRRSTcDz1K5FGRdROxsW2VmVgoDF24AEfE0iSfWmFn/G8hwM7P8OdzMLEsONzPLzkAOKJjZYHC4mVmWHG5mliWHm5llyeFmZtnxgIKZZcvhZmZZ6tdwm8hklWZmpeWem5kl9WvPzeFmZkkONzPLjkdLzSxbDjczy5LDzcyy5HAzsyw53MwsOx5QMLNsOdzMLEsONzPLUr+Gm+8tNbOksfNuzbxSJM2R9IKkXZJ2SrqlWH6apM2S3ir+Tq/a5g5JeyS9Kemy8dTtnpuN23e+851k+ze/+c1k+6pVq9pZjnVBhwYUjgK3RsSrkk4GtknaDFwHPB8Rd0u6HbgduE3SfGAZcD7wOeA5SedFxLHUTtxzM7OkdvfcImIkIl4t3n8I7AJmA0uB9cVq64Eri/dLgQ0RcSQi3gH2ABc2qts9NzNL6uQ5N0lzgQuAl4GZETFS7HNE0hnFarOBP1ZtNlwsS3K4mVlSi+E2Q9LWqs9DETFUvYKkk4BHgdURcVhSve+q1dCwKIebmXXCwYhYWK9R0glUgu3hiHisWHxA0qyi1zYLGC2WDwNzqjY/C9jfqACfczOzpA6Mlgp4ENgVEfdWNW0ClhfvlwNPVC1fJmmapHOBecArjep2z83M6urQaOki4FrgdUnbi2V3AncDGyXdALwHXF3UsFPSRuANKiOtqxqNlILDzcwaaHe4RcRL1D6PBnBJnW3WAGua2c+Ewk3SXuBD4BhwNHWMbf3v9NNPT7Zfd911yfbvf//7yfaPPvqo2ZKsC/r1DoV29NwujoiDbfgeMyuhQQ43M8tYv4bbREdLA/itpG2SVrajIDMrj1ZGSssShhPtuS2KiP3FlcSbJe2OiBerVyhCz8Fn1qfKElbNmlDPLSL2F39Hgcepcb9XRAxFxEIPNpj1p37tubUcbpJOLO7oR9KJwLeAHe0qzMzKoV/DbSKHpTOBx4v7waYAv4qIZ9pSlZmVRlnCqlkth1tEvA18tY21WMk980z6v133339/sv2cc85Jtu/evbvpmqyzytQTa5YvBTGzJIebmWXJ4WZmWXK4mVmW+jXcPJ+bmWXJPTczq8ujpWaWLYebcfHFFyfbJ0+enGx/7rnn2llO2x06dCjZPjo6mmy/6aabku2rV69utiTrAoebmWXJ4WZmWXK4mVl2PKBgZtlyuJlZlhxuZpYlh5uZZcnhZqxYsSLZ/qc//SnZXvbr3P76178m23/xi18k2y+//PJk+5Qp6X8cjx49mmy39vOAgplly+FmZllyuJlZlhxuZpYlh5uZZccDCmaWrX4NN8/Ea2ZZcs+tjU499dRel9BTzz77bLL9tttuS7Z/97vfTbZv2LCh6Zps4vq15+ZwM7Mkh5uZZcnhZmbZ8WipmWXL4WZmWXK4mVmWHG5mlp2sz7lJWgd8GxiNiC8Vy04DHgHmAnuB70VE+qGWA2DRokXJ9ieffLJLlfTG7t27k+0fffRRsv28885rZznWJv0abuO5Q+EhYMlxy24Hno+IecDzxWczy9BY762ZVxk0DLeIeBH44LjFS4H1xfv1wJXtLcvMyqJfw63Vc24zI2IEICJGJJ3RxprMrETKElbN6viN85JWStoqaWun92Vm7dVKr208YShpnaRRSTuqlt0laZ+k7cXriqq2OyTtkfSmpMvGU3ur4XZA0qxip7OA0XorRsRQRCyMiIUt7svMeqhDh6UP8dlz+QD3RcSC4vU0gKT5wDLg/GKb+yVNbrSDVsNtE7C8eL8ceKLF7zGzkutEuNU5l1/PUmBDRByJiHeAPcCFjTZqGG6Sfg38AfgnScOSbgDuBi6V9BZwafHZzDLU5QGFmyW9Vhy2Ti+WzQb+XLXOcLEsqeGAQkRcU6fpkoZlDpiPP/442f7UU091qZLe+Mtf/pJs//nPf55sX7x4cbL9Jz/5SbLdzzUtlRnHnWcfioihBtusBX4MRPH3HuB6QDXWbZigvkPBzJJa7IkdbPY8e0QcGHsv6QFg7Kr3YWBO1apnAfsbfZ+nGTezujo1WlrL2CBl4SpgbCR1E7BM0jRJ5wLzgFcafZ97bmaW1Inr3Ipz+YupHL4OAz8EFktaQOWQcy9wY7H/nZI2Am8AR4FVEXGs0T4cbmaW1Ilwq3Mu/8HE+muANc3sw+FmZkn9eoeCw83MkhxuZpadMt0I3yyHWxO+8IUvJNs//fTTZPt7773XznL6TqP53ho917TR79/o+601Djczy5LDzcyy5HAzsyw53MwsOx5QMLNsOdzMLEsONzPLksNtAJx55pnJ9qlTp3apkv70m9/8Jtne6F+ihQvTM+j4OrfO6Ndw85RHZpYl99zMrC6PlppZthxuZpYlh5uZZcnhZmZZcriZWXY8oDAg5s+f3+sSOmrGjBnJ9vfff39C21900UVN12S953Azsyw53MwsSw43M8uSw83MsuMBBTPLlsPNzLLkcDOzLGUbbpLWAd8GRiPiS8Wyu4AVwP8Wq90ZEU93qsiyOOecc5Lt06ZNS7bfdNNNyfavfOUryfbZs2cn27/85S8n288+++xk+8jISLL98OHDE9peUrL9kUceSbZffvnlyfZf/vKXyXZrTb+G23jmc3sIWFJj+X0RsaB4ZR9sZoNobECh2VcZNAy3iHgR+KALtZiZtc1EZuK9WdJrktZJmt62isysVLLtudWxFvg8sAAYAe6pt6KklZK2Stra4r7MrIf6NdxaGi2NiANj7yU9ADyZWHcIGCrWLcf/ajMbt7KEVbNaCjdJsyJibGjsKmBH+0oyszLJNtwk/RpYDMyQNAz8EFgsaQEQwF7gxs6VaGa9UqbDzGY1DLeIuKbG4gc7UEvpPfXUU8n2H/zgB8n2FStWJNsbXUf29ttvJ9vXrl2bbG/03NC33nor2X7kyJFk+0T96Ec/SrZff/31Hd2/1ZZtuJnZYHO4mVmWHG5mliWHm5llJ+sBBTMbbA43M8uSw83MsuRwGwC///3vk+2nnnpqsr3RdWLHjh1ruqZBcsoppyTbp06dmmz/5JNP2lnOwOjXcJvIrCBmlrlOzedWzCY0KmlH1bLTJG2W9Fbxd3pV2x2S9kh6U9Jl46nd4WZmSR2aFeQhPjsJ7u3A8xExD3i++Iyk+cAy4Pxim/slTW60A4ebmXVdnUlwlwLri/frgSurlm+IiCMR8Q6wB7iw0T58zs3Mkrp4zm3m2GxDETEi6Yxi+Wzgj1XrDRfLkhxuZpbUYrjNOG6C2qFibsdW1HqyUMOiHG5mltRiuB2MiIVNbnNgbK5ISbOA0WL5MDCnar2zgP2Nvszn3Mysri4//WoTsLx4vxx4omr5MknTJJ0LzANeafRl7rm10d/+9rdel5C1k08+Odn+ta99Ldn+u9/9rp3lDIxOnHOrMwnu3cBGSTcA7wFXF/vfKWkj8AZwFFgVEQ0vCnW4mVlSJ8KtziS4AJfUWX8NsKaZfTjczCypX+9QcLiZWZLDzcyy4/nczCxbDjczy5LDzcyy5HAzsyw53MwsOx5QMLNsOdzMLEsONzPLUr+Gm2cFMbMsuedmZkn92nNzuJlZXVmPlkqaA/wXcCbwKZXpgn8m6TTgEWAusBf4XkQc6lypNugaPfd1eHi4S5UMln4Nt/GcczsK3BoRXwS+AawqHrVV8zFcZpaXLs7E21YNwy0iRiLi1eL9h8AuKk+eqfcYLjPLSL+GW1Pn3CTNBS4AXqb+Y7jMLBNlCqtmjTvcJJ0EPAqsjojDUq2nbdXcbiWwsrXyzKzXsg43SSdQCbaHI+KxYnG9x3D9neJZhUPF9/Tnr2Q2wPo13Bqec1Oli/YgsCsi7q1qqvcYLjPLSM7n3BYB1wKvS9peLLuTOo/hMrO8lCWsmtUw3CLiJWo/zh7qPIbLrBMaXef27rvvdqmSwVGmnlizfIeCmSU53MwsSw43M8uSw83MsuRwM7Ps9POAgierNLMsuedmZkn92nNzuFnfmDp1arJ99uzZyfZ9+/a1s5yB4XAzsyw53MwsSw43M8tOP4+WOtzMLMnhZmZZcriZWZYcbmaWJYeb2QTt2bMn2b5ly5Zk+/vvv9/OcgwPKJhZxhxuZpYlh5uZZcnhZmZZcriZWXY8oGBm2XK4mZk1QdJe4EPgGHA0IhZKOg14BJgL7AW+FxGHWvr+bqaypP78T4BZBiKi3vOH6zrxxBNj/vz5Te9r69at2yJiYWqdItwWRsTBqmX/BnwQEXdLuh2YHhG3NV0AnmbczBoYO+/WzGsClgLri/frgStb/SKHm5kldTDcAvitpG2SVhbLZkbESLHfEeCMVuv2OTczq2sCPbEZkrZWfR6KiKHj1lkUEfslnQFslrS75UJrcLiZWVKL4Xaw0Tm3iNhf/B2V9DhwIXBA0qyIGJE0CxhtZefgw1Iza6ATh6WSTpR08th74FvADmATsLxYbTnwRKt1u+dmZkkduqJiJvC4JKjk0K8i4hlJW4CNkm4A3gOubnUHDjczS+pEuEXE28BXayx/H7ikHftoeFgqaY6kFyTtkrRT0i3F8rsk7ZO0vXhd0Y6CzKw8WjkkLcsdDePpuR0Fbo2IV4tj5G2SNhdt90XETztXnpn1WlnCqlkNw6241mTsupMPJe0C0o/2NrNs9Gu4NTVaKmkucAHwcrHoZkmvSVonaXq7izOz3uvXw9Jxh5ukk4BHgdURcRhYC3weWEClZ3dPne1WStp63AV9ZtYn+jXcxjVaKukEKsH2cEQ8BhARB6raHwCerLVtcVXyULFeOf5Xm9m4lCmsmtUw3FS5EOVBYFdE3Fu1fNbYPWDAVVQuwDOzzGQbbsAi4FrgdUnbi2V3AtdIWkDl5te9wI0dqM/MeizbcIuIl4Ba80A93f5yzMzaw3comFlStj03MxtsDjczy07Wo6VmNtgcbmaWJYebmWXJ4WZmWXK4mVl2PKBgZtlyuJlZlhxuZpYlh5uZZcnhZmbZ8YCCmWXL4WZmWXK4jc9B4N2qzzOKZWXl+iamzPWVuTZof33ntLqhw20cIuL06s+StkbEwm7W0AzXNzFlrq/MtUH56+sHPiw1syT33MwsOx4tbd1Qj/ffiOubmDLXV+baoET19Wu4qV8LN7POmzJlSpxyyilNb3fo0KFtvT5n2Ouem5mVXL92gCb1YqeSlkh6U9IeSbf3ooYUSXslvS5pu6StJahnnaRRSTuqlp0mabOkt4q/00tW312S9hW/4XZJV/SwvjmSXpC0S9JOSbcUy0vxGybqK8VvOHberZlXGXQ93CRNBv4DuByYT+XhzvO7Xcc4XBwRC3rdtS48BCw5btntwPMRMQ94vvjcKw/x2foA7it+wwUR0cvn3B4Fbo2ILwLfAFYV/8yV5TesVx/0+DdsJdgGNtyAC4E9EfF2RHwCbACW9qCOvhERLwIfHLd4KbC+eL8euLKbNVWrU19pRMRIRLxavP8Q2AXMpiS/YaK+UnC4jd9s4M9Vn4cp0f+RhQB+K2mbpJW9LqaOmRExApV/OYAzelxPLTdLeq04bO3ZYXM1SXOBC4CXKeFveFx9UILf0OE2fqqxrBy/xv9bFBH/TOXQeZWki3pdUB9aC3weWACMAPf0tBpA0knAo8DqiDjc63qOV6O+UvyGDrfxGwbmVH0+C9jfgzrqioj9xd9R4HEqh9Jlc0DSLIDi72iP6/k7EXEgIo5FxKfAA/T4N5R0ApXgeDgiHisWl+Y3rFVfWX5Dh9v4bQHmSTpX0lRgGbCpB3XUJOlESSePvQe+BexIb9UTm4DlxfvlwBM9rOUzxkKjcBU9/A0lCXgQ2BUR91Y1leI3rFdfGX7Dfh5Q6MlFvMWQ9r8Dk4F1EbGm60XUIekfqfTWoHId4K96XZ+kXwOLqcwUcQD4IfDfwEbgbOA94OqI6MlJ/Tr1LaZyOBXAXuDGsfNbPajvX4D/AV4HPi0W30nlvFbPf8NEfdfQ499w0qRJMW3atKa3+/jjj3t+Ea/vUDCzuiZNmhRTp05tersjR470PNx8h4KZJfVrB8jhZmZJDjczy06ZBgia1ZN7S83MOs3hZmZJnbgUpBuTZ/iw1MyS2n1YWjV5xqVULurfImlTRLzRzv2452ZmSR3ouXVl8gz33MwsqQMDCrUmz/h6u3ficDOzlGep3HnSrH84bqLXoYgYey5EVybPcLiZWV0RUWsS0onqyuQZPudmZt3Wlckz3HMzs66KiKOSbqZyyDs2ecbOdu/HN86bWZZ8WGpmWXK4mVmWHG5mliWHm5llyeFmZllyuJlZlhxuZpYlh5uZZen/AO11BQav3K+DAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(\"Training data:\", X_train.shape, t_train.shape)\n",
    "print(\"Test data:\", X_test.shape, t_test.shape)\n",
    "\n",
    "idx = 682 # for example\n",
    "x = X_train[idx, :]\n",
    "t = t_train[idx]\n",
    "print(\"x:\", x)\n",
    "print(\"x (shape):\", x.shape)\n",
    "print(\"t:\", t)\n",
    "\n",
    "plt.figure(figsize=(5, 5))\n",
    "plt.imshow(x, cmap=\"gray\")\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, we are going to use a regular MLP (with fully-connected layers). Convolutional layers will be seen next time.\n",
    "\n",
    "We therefore need to transform the 28x28 input matrix into a 784 vector. Additionally, pixel values are integers between 0 and 255. We have to rescale them to floating values in [0, 1]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = X_train.reshape(X_train.shape[0], 784).astype('float32') / 255.\n",
    "X_test = X_test.reshape(X_test.shape[0], 784).astype('float32') / 255."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We saw in the last exercise that **mean removal** is crucial when training a neural network. The following cell removes the mean image of the training set from all examples.\n",
    "\n",
    "*Note:* you could also divide by the standard deviation to have a full normalization, but it does not bring much here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATEAAAEvCAYAAAAtufaDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAATLElEQVR4nO3dX2id933H8c/XihP/t+XacR3VibPgjIXB0mHCIGNklJa0N0kuOpqLkkHBvWiggV4s9Ka5GYTRP7sZBYeEZpC2FJKsuShbQyhkhRHihJA48/KnjVPLduU6smzJ/yV/d6EnoLqSn490Hunoq7xfYHT06Ovf+R09Rx8959H3+Z3ITAFAVav6PQEA6AUhBqA0QgxAaYQYgNIIMQClEWIASrtuKe8sIujnALBQJzNz+9UbezoSi4h7I+KdiHg/Ih7tZSwAaPHhbBsXHGIRMSDp3yR9UdIdkh6MiDsWOh4ALEQvR2J3SXo/M3+bmZck/VTSfd1MCwA8vYTYkKQjMz4fbrYBwJLp5cR+zLLtT07cR8Q+Sft6uB8AmFMvITYsadeMzz8j6djVRZm5X9J+ib9OAuheLy8nX5W0JyJujYjrJX1F0gvdTAsAPAs+EsvMyYh4WNJ/SRqQ9FRmvt3ZzADAEEu5nhgvJwH04LXM3Hv1Ri47AlAaIQagNEIMQGmEGIDSCDEApRFiAEojxACURogBKI0QA1Daki5Pjf6LmG3xkcWt68d9ds29suXKlSudjdV13UrFkRiA0ggxAKURYgBKI8QAlEaIASiNEANQGiEGoDRCDEBphBiA0ujY76MuO9RXrfJ+H7l1AwMDVt1117U/hZya+dS5c3NMTU1ZdZOTk1bd5cuXO6mR/Lk5dSu5q58jMQClEWIASiPEAJRGiAEojRADUBohBqA0QgxAaYQYgNIIMQCl0bG/CNxO/C674m+44QZrrLVr11p169evt+o2btzYWrNlyxZrrA0bNlh1zmN11rqXpHPnzll1p0+fturOnDnT2VgTExNW3cWLF1tr3KsE3O/bcroCgCMxAKURYgBKI8QAlEaIASiNEANQGiEGoDRCDEBphBiA0ggxAKXRsT9Pzhr17lrxbpf9unXrWms2b95sjbV161arbufOnVbd0NBQJzWStH37dqvO+X6469OPjY1ZdUePHrXqjhw50lozPDxsjfX73//eqjt16lRrjXtlgtP9L/nvObAUnf09hVhEHJY0LmlK0mRm7u1iUgDg6uJI7O8z82QH4wDAvHFODEBpvYZYSvplRLwWEftmK4iIfRFxICIO9HhfAPAnen05eXdmHouIGyW9GBH/l5kvzyzIzP2S9ktSRCyf9TsArAg9HYll5rHm4wlJz0u6q4tJAYBrwSEWEesjYuPHtyV9QdLBriYGAI5eXk7ukPR8s4rpdZJ+nJn/2cmsAMC04BDLzN9K+qsO59JXThOr1O1S0c7SzpLXoPrpT3/aGmvXrl1W3a233tpZ3S233GKNtWPHDqvOaXZ1mzFHR0etOqeJVZIGBwdba9asWWON5S5z7jSUustOu03CXY/XC1osAJRGiAEojRADUBohBqA0QgxAaYQYgNIIMQClEWIASiPEAJS24pendrueu+zYX7t2rTXWpk2brDpn2Wa3E//222+36vbs2WPV7d69u7XGXera/X6sXr26teby5cvWWO6+GhgYsOqc+3WXij5z5oxVNz4+3lpz9uxZa6wLFy5Yde7P1VLgSAxAaYQYgNIIMQClEWIASiPEAJRGiAEojRADUBohBqA0QgxAaXTsN9yOfadb3F1DvcuO/aGhIWusm2++2aq76aabrLrNmze31rjrrLvr3Ttryne5Pr3k7XfJe98Ed79v2LDBqnPe08G94mA5deK7OBIDUBohBqA0QgxAaYQYgNIIMQClEWIASiPEAJRGiAEojWbXhtvs6tS5jZFdLmP9qU99yhpry5YtVp2zDLckjY2NtdZMTExYY7nLNjv71P3eug2lLrex1+E24l65cqWTGsmfvzu3pcCRGIDSCDEApRFiAEojxACURogBKI0QA1AaIQagNEIMQGmEGIDSVnzHvsvtQHa6xd2rBLrs7Hc71N0rE8bHx626U6dOtdacPHnSGuvixYtW3bp161prBgcHrbFc7vLO58+fb61xr0y4cOGCVTc5OdlJjeR39pfq2I+IpyLiREQcnLFta0S8GBHvNR+7fcYAgMn5tfwjSfdete1RSS9l5h5JLzWfA8CSaw2xzHxZ0tVvQ3OfpKeb209Lur/baQGAZ6En9ndk5nFJaj7e2N2UAMC36Cf2I2KfpH2LfT8APpkWeiQ2EhE7Jan5eGKuwszcn5l7M3PvAu8LAOa00BB7QdJDze2HJP28m+kAwPw4LRY/kfQ/kv48IoYj4muSHpf0+Yh4T9Lnm88BYMm1nhPLzAfn+NLnOp4LAMzbiu/YdzuLu6xzu+Ldjv01a9a01txwww3WWG5H9unTp626kZGRzsZyvx/r169vrbn++uutsdw6t+Pd6dh333Pg7NmzVt2lS5daa/rxc7BUuHYSQGmEGIDSCDEApRFiAEojxACURogBKI0QA1AaIQagNEIMQGl07M+zzuF27Lvd4s76+e5Ybse+u9791NRUa43TYS9JW7dutep27NjRWrN9+3ZrLOdqCEkaHb16XdDZOeviux37Xe6DrtGxDwAdIcQAlEaIASiNEANQGiEGoDRCDEBphBiA0ggxAKWt+GbXrjmNrG7jqdto6Sw9PTAwYI3lNru6Nm7c2FrjNrEODQ1ZdU6zq9tg6yztLEljY2NWndPs6t5nl/vKbcB26yLCqluKpliOxACURogBKI0QA1AaIQagNEIMQGmEGIDSCDEApRFiAEojxACUtuI79rvuGHY6492O/dWrV1t1The1u0SxW3fddd5TY9u2ba01u3fvtsZyO/YHBwdbayYnJ62xTp06ZdW54zld9m63u3sVhlPn3qdbt5xwJAagNEIMQGmEGIDSCDEApRFiAEojxACURogBKI0QA1AaIQagtBXfse9yO5WdTna32929T6db3FnbXfLXUHevJnDW2N+8ebM11rp166w6x7lz56y6iYmJTsdzOvbdTnz3eeTs04qd+K7WRx8RT0XEiYg4OGPbYxFxNCLeaP59aXGnCQCzc34t/0jSvbNs/0Fm3tn8+0W30wIAT2uIZebLkkaXYC4AMG+9nNh/OCLebF5uzrmsQETsi4gDEXGgh/sCgFktNMR+KOk2SXdKOi7pe3MVZub+zNybmXsXeF8AMKcFhVhmjmTmVGZekfSEpLu6nRYAeBYUYhGxc8anD0g6OFctACym1kaUiPiJpHskbYuIYUnfkXRPRNwpKSUdlvT1xZsiAMytNcQy88FZNj+5CHNZFG6Tn9sE6jQgus2MTmOk5DVanj592hrLXWbZbTy9ePFia407N3dZb2efnj171hprdNT7w7s7nrNPu2xidXW9TLv7c9X1/c6Gy44AlEaIASiNEANQGiEGoDRCDEBphBiA0ggxAKURYgBKI8QAlMby1A23O7rLpYCnpqasOqdb3F1O+tKlS53WOdzv7eXLl60657G68x8fH7fq3Lk53C529+oKp859ri1Fh33XOBIDUBohBqA0QgxAaYQYgNIIMQClEWIASiPEAJRGiAEojRADUNqK79h3u+fddfGd9dHdNdRdTre421Hu1jlr50ve1QRuV/yaNWusOmf9f7fzvOv3YHDut8tOfLeOjn0AWKYIMQClEWIASiPEAJRGiAEojRADUBohBqA0QgxAaYQYgNLo2G+4HftOnTuWuy6+08m+du1aa6z169dbde54GzdubK3ZsGGDNZY7N6djv+srE65cuWLVnT9/vrXGucpBks6dO2fVOe8n4Hb/u49zOXX2cyQGoDRCDEBphBiA0ggxAKURYgBKI8QAlEaIASiNEANQGs2uDbd5z1nm171Pdxlrp6F0cHDQGmvr1q1W3aZNm6y6LVu2tNZs27at0/t0vr9nzpyxxnKbXcfGxqy6jz76qLVmdHTUGuv06dNWndNg6zb/us2uy0nrkVhE7IqIX0XEoYh4OyK+2WzfGhEvRsR7zUfvpwgAOuS8nJyU9K3M/AtJfyPpGxFxh6RHJb2UmXskvdR8DgBLqjXEMvN4Zr7e3B6XdEjSkKT7JD3dlD0t6f5FmiMAzGleJ/YjYrekz0p6RdKOzDwuTQedpBs7nx0AtLBP7EfEBknPSnokM8+4J68jYp+kfQubHgBcm3UkFhGrNR1gz2Tmc83mkYjY2Xx9p6QTs/3fzNyfmXszc28XEwaAmZy/ToakJyUdyszvz/jSC5Ieam4/JOnn3U8PAK7NeTl5t6SvSnorIt5otn1b0uOSfhYRX5P0O0lfXpQZAsA1tIZYZv5a0lwnwD7X7XQAYH5WfMd+l534ktf5fOHChU7v01nu2unql/zu+e3bt1t1zhUAmzdvtsZatcr7Y/nExERrjdvtfvToUavu8OHDVt2RI0daa0ZGRqyx3MfgLGPtLk/t/rywPDUAdIQQA1AaIQagNEIMQGmEGIDSCDEApRFiAEojxACURogBKG3Fd+y7a4a7a5A73eLuGupul72zfr67dr7bie8uteRwOsol/0qHY8eOtdb85je/scZ65513rLp3333Xqvvwww9ba/7whz9YY42Pj1t1zvsEuFeHrMg19gFgOSPEAJRGiAEojRADUBohBqA0QgxAaYQYgNIIMQClrfhmV3cZXXf5Xqdx8+TJk9ZYXTaUus2Mly5dsurcRstNmza11rj7wF2OeXh4uLXmgw8+sMZyl512l7H+6KOPWmuchmnJa2KVvOeu+/xYTstOuzgSA1AaIQagNEIMQGmEGIDSCDEApRFiAEojxACURogBKI0QA1Daiu/Yd7kdzU4XtbvEb5fd88ePH7fGcpdjdpbElqQ1a9a01rhd4O4y1mNjY601p06dssZyrxJwr2A4f/58a427392rSJznW8VOfBdHYgBKI8QAlEaIASiNEANQGiEGoDRCDEBphBiA0ggxAKURYgBKo2N/npzu6MuXL3c2ljueu277yMiIVbd69WqrbtWq7n4Puh3qTp27D7quc+bm7ne3biV34ztan4ERsSsifhURhyLi7Yj4ZrP9sYg4GhFvNP++tPjTBYA/5hyJTUr6Vma+HhEbJb0WES82X/tBZn538aYHANfWGmKZeVzS8eb2eEQckjS02BMDAMe8TmhExG5Jn5X0SrPp4Yh4MyKeighv2QMA6JAdYhGxQdKzkh7JzDOSfijpNkl3avpI7Xtz/L99EXEgIg70Pl0A+GNWiEXEak0H2DOZ+ZwkZeZIZk5l5hVJT0i6a7b/m5n7M3NvZu7tatIA8DHnr5Mh6UlJhzLz+zO275xR9oCkg91PDwCuzfnr5N2SvirprYh4o9n2bUkPRsSdklLSYUlfX4T5AcA1OX+d/LWkmOVLv+h+OgAwP3TsLwK309rlrOvvdpS769i7nfjTZxu64XaeO9/ffnXFf9LXu+8Hrp0EUBohBqA0QgxAaYQYgNIIMQClEWIASiPEAJRGiAEojWbXPuq6IXO56rIh1kVD6ScHR2IASiPEAJRGiAEojRADUBohBqA0QgxAaYQYgNIIMQClEWIASlvqjv2Tkj68atu2ZntV1ecvLfJjWILuefZB/y3F/G+ZbWP0+/KMiDhQ+T0pq89fqv8Yqs9fqv8Y+jl/Xk4CKI0QA1Dacgix/f2eQI+qz1+q/xiqz1+q/xj6Nv++nxMDgF4shyMxAFiwvoVYRNwbEe9ExPsR8Wi/5tGLiDgcEW9FxBsRcaDf83FExFMRcSIiDs7YtjUiXoyI95qPg/2c47XMMf/HIuJosx/eiIgv9XOO1xIRuyLiVxFxKCLejohvNtsr7YO5HkNf9kNfXk5GxICkdyV9XtKwpFclPZiZ/7vkk+lBRByWtDczy/T3RMTfSZqQ9O+Z+ZfNtn+RNJqZjze/UAYz85/6Oc+5zDH/xyRNZOZ3+zk3R0TslLQzM1+PiI2SXpN0v6R/VJ19MNdj+Af1YT/060jsLknvZ+ZvM/OSpJ9Kuq9Pc/lEycyXJY1etfk+SU83t5/W9BNyWZpj/mVk5vHMfL25PS7pkKQh1doHcz2GvuhXiA1JOjLj82H18ZvQg5T0y4h4LSL29XsyPdiRmcel6SeopBv7PJ+FeDgi3mxebi7bl2IzRcRuSZ+V9IqK7oOrHoPUh/3QrxCb7Z0jKv6Z9O7M/GtJX5T0jealDpbeDyXdJulOScclfa+vszFExAZJz0p6JDPP9Hs+CzHLY+jLfuhXiA1L2jXj889IOtanuSxYZh5rPp6Q9LymXyZXNNKc5/j4fMeJPs9nXjJzJDOnMvOKpCe0zPdDRKzW9A//M5n5XLO51D6Y7TH0az/0K8RelbQnIm6NiOslfUXSC32ay4JExPrmpKYiYr2kL0g6eO3/tWy9IOmh5vZDkn7ex7nM28c//I0HtIz3Q0y/f92Tkg5l5vdnfKnMPpjrMfRrP/St2bX58+u/ShqQ9FRm/nNfJrJAEfFnmj76kqZXA/lxhccQET+RdI+mVx0YkfQdSf8h6WeSbpb0O0lfzsxlefJ8jvnfo+mXMCnpsKSvf3x+abmJiL+V9N+S3pL08RuKflvT55Sq7IO5HsOD6sN+oGMfQGl07AMojRADUBohBqA0QgxAaYQYgNIIMQClEWIASiPEAJT2/3rBGrzHHUyEAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_mean = np.mean(X_train, axis=0)\n",
    "X_train -= X_mean\n",
    "X_test -= X_mean\n",
    "\n",
    "\n",
    "plt.figure(figsize=(5, 5))\n",
    "plt.imshow(X_mean.reshape((28, 28))*255, cmap=\"gray\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last preprocessing step is to perform **one-hot encoding** of the output labels. We want for example the digit 4 (index 5 in the outputs t) to be represented by the vector:\n",
    "\n",
    "[0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]\n",
    "\n",
    "`keras` offers the utility `utils.to_categorical` to do that on the whole data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]\n"
     ]
    }
   ],
   "source": [
    "T_train = tf.keras.utils.to_categorical(t_train, 10)\n",
    "T_test = tf.keras.utils.to_categorical(t_test, 10)\n",
    "\n",
    "print(T_train[idx])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All set! The data is ready to be learned by a neural network. You should normally not have to re-run those cells again. If you do, do not forget to run all of them sequentially."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model definition\n",
    "\n",
    "Let's now define a simple MLP with keras. When using a notebook, you can recreate models by simply re-running the cell, but this does not delete the previous networks which may end up filling your RAM. It is therefore good practice to start by telling tensorflow to delete all previous models (if this is what you want):\n",
    "\n",
    "```python\n",
    "tf.keras.backend.clear_session()\n",
    "```\n",
    "\n",
    "One way to define a neural network in keras is by stacking layers in a `Sequential()` model (you can later have a look at the doc of `Model()` for directed acyclic graphs).\n",
    "\n",
    "```python\n",
    "model = tf.keras.models.Sequential()\n",
    "```\n",
    "\n",
    "The input layer has 784 neurons, one per pixel in the input image. We only need to define a placeholder of the correct size to represent inputs and `add()` it to the model as its first layer:\n",
    "\n",
    "```python\n",
    "model.add(tf.keras.layers.Input(shape=(784,)))\n",
    "```\n",
    "\n",
    "The input layer goes into a hidden, fully-connected, layer of 100 neurons using the logistic (or sigmoid) transfer function This can be specified by adding to the model a `Dense` layer (in the sense \"fully-connected\") with 100 units (another name for neuron), followed by an `Activation` layer using the 'sigmoid' function:\n",
    "\n",
    "```python\n",
    "model.add(tf.keras.layers.Dense(units=100))\n",
    "model.add(tf.keras.layers.Activation('sigmoid')) \n",
    "```\n",
    "\n",
    "The weight matrix and the biases are intialized automatically using the Glorot uniform scheme (seen in the last exercise) for the weights and zeros for the biases. Check the doc of the `Dense` layer to see how to change this: <https://keras.io/layers/core/#dense>.\n",
    "\n",
    "We then add a softmax layer as output layer (classification problem), with 10 units (one per digit):\n",
    "\n",
    "```python\n",
    "model.add(tf.keras.layers.Dense(units=10))\n",
    "model.add(tf.keras.layers.Activation('softmax')) \n",
    "```\n",
    "\n",
    "Weights and biases are initialized in the same manner. That's all, keras now knows how to transform the input vector into class probabilities using randomly initialized weights!\n",
    "\n",
    "For training, we need to choose an optimizer (learning rule). Several optimizers are available (<https://keras.io/optimizers/>). We pick simply **Stochastic Gradient Descent** with a learning rate of 0.01:\n",
    "\n",
    "```python\n",
    "optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)\n",
    "```\n",
    "\n",
    "The last step is to **compile** the network, so that keras computes how to implement the backpropagation algorithm. You need to specify:\n",
    "\n",
    "1. which loss function you want to minimize. The full list is at <https://keras.io/losses/>. Here, we will use the cross-entropy loss function as we have a clasification problem with softmax outputs.\n",
    "2. which optimizer you want to use.\n",
    "3. which metrics (accuracy, error, etc. - <https://keras.io/metrics/>) you want to track during learning.\n",
    "\n",
    "After the call to `compile()`, the neural network is instantiated and ready to learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Metal device set to: Apple M1 Pro\n",
      "\n",
      "systemMemory: 16.00 GB\n",
      "maxCacheSize: 5.33 GB\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-03-20 18:09:54.683119: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.\n",
      "2023-03-20 18:09:54.683274: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)\n"
     ]
    }
   ],
   "source": [
    "# Delete all previous models to free memory\n",
    "tf.keras.backend.clear_session()\n",
    "\n",
    "# Sequential model\n",
    "model = tf.keras.models.Sequential()\n",
    "\n",
    "# Input layer representing the 784 pixels\n",
    "model.add(tf.keras.layers.Input(shape=(784,)))\n",
    "\n",
    "# Hidden layer with 100 logistic neurons\n",
    "model.add(tf.keras.layers.Dense(units=100))\n",
    "model.add(tf.keras.layers.Activation('sigmoid')) \n",
    "\n",
    "# Softmax output layer over 10 classes\n",
    "model.add(tf.keras.layers.Dense(units=10))\n",
    "model.add(tf.keras.layers.Activation('softmax')) \n",
    "\n",
    "# Learning rule\n",
    "optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)\n",
    "\n",
    "# Loss function\n",
    "model.compile(\n",
    "    loss='categorical_crossentropy', # loss function\n",
    "    optimizer=optimizer, # learning rule\n",
    "    metrics=['accuracy'] # show accuracy\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A good practice after creating the model is to call `model.summary()` to see how many layers you have created and how many parameters each layer has.\n",
    "\n",
    "**Q:** Explain why you obtain this numbers of parameters in each layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense (Dense)               (None, 100)               78500     \n",
      "                                                                 \n",
      " activation (Activation)     (None, 100)               0         \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 10)                1010      \n",
      "                                                                 \n",
      " activation_1 (Activation)   (None, 10)                0         \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 79,510\n",
      "Trainable params: 79,510\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(model.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A:** The hidden layer has a weight matrix of size 784x100 and 100 biases, what makes 78500 free parameters. The output layer has a weight matrix of size 100x10 and 10 biases, so 1010 parameters.\n",
    "\n",
    "Note that we have more free parameters than training examples, we are going to have to regularize quite hard..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model training\n",
    "\n",
    "Now is time to train the network on MNIST. The following cell creates a `History()` object that will record the progress of your network.\n",
    "\n",
    "It then calls the `model.fit()` method, which tells the network to learn the MNIST dataset defined by the `(X_train, Y_train)` arrays. You have to specify:\n",
    "\n",
    "1. the batch size, i.e. the number of training examples in each minibatch used by SGD.\n",
    "2. the maximal number of epochs for training\n",
    "3. the size of the validation, taken from the training set to track the progress (this is not the test set!). Here we reserve 10% of the training data to validate. If you do not have much data, you could set it to 0.\n",
    "4. a callback, which will be called at the end of each epoch. Here it will save the metrics defined in `model.compile()` in the `History()` object.\n",
    "\n",
    "The training process can take a while depending on how big your network is and how many data samples you have. You can interrupt the kernel using the menu if you want to stop the processing in the cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-03-20 18:10:02.477640: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz\n",
      "2023-03-20 18:10:02.579396: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "422/422 [==============================] - ETA: 0s - loss: 2.0908 - accuracy: 0.4240"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-03-20 18:10:07.488093: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "422/422 [==============================] - 5s 10ms/step - loss: 2.0908 - accuracy: 0.4240 - val_loss: 1.8238 - val_accuracy: 0.6538\n",
      "Epoch 2/20\n",
      "422/422 [==============================] - 4s 9ms/step - loss: 1.6363 - accuracy: 0.6766 - val_loss: 1.4138 - val_accuracy: 0.7585\n",
      "Epoch 3/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 1.3001 - accuracy: 0.7496 - val_loss: 1.1166 - val_accuracy: 0.8143\n",
      "Epoch 4/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 1.0651 - accuracy: 0.7899 - val_loss: 0.9142 - val_accuracy: 0.8458\n",
      "Epoch 5/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.9056 - accuracy: 0.8143 - val_loss: 0.7770 - val_accuracy: 0.8630\n",
      "Epoch 6/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.7954 - accuracy: 0.8303 - val_loss: 0.6808 - val_accuracy: 0.8742\n",
      "Epoch 7/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.7162 - accuracy: 0.8421 - val_loss: 0.6113 - val_accuracy: 0.8785\n",
      "Epoch 8/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.6573 - accuracy: 0.8503 - val_loss: 0.5586 - val_accuracy: 0.8848\n",
      "Epoch 9/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.6119 - accuracy: 0.8569 - val_loss: 0.5179 - val_accuracy: 0.8910\n",
      "Epoch 10/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.5759 - accuracy: 0.8620 - val_loss: 0.4859 - val_accuracy: 0.8967\n",
      "Epoch 11/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.5468 - accuracy: 0.8670 - val_loss: 0.4597 - val_accuracy: 0.8983\n",
      "Epoch 12/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.5228 - accuracy: 0.8705 - val_loss: 0.4380 - val_accuracy: 0.9012\n",
      "Epoch 13/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.5027 - accuracy: 0.8734 - val_loss: 0.4199 - val_accuracy: 0.9032\n",
      "Epoch 14/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.4855 - accuracy: 0.8761 - val_loss: 0.4045 - val_accuracy: 0.9050\n",
      "Epoch 15/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.4707 - accuracy: 0.8783 - val_loss: 0.3912 - val_accuracy: 0.9062\n",
      "Epoch 16/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.4578 - accuracy: 0.8801 - val_loss: 0.3800 - val_accuracy: 0.9067\n",
      "Epoch 17/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.4465 - accuracy: 0.8827 - val_loss: 0.3698 - val_accuracy: 0.9092\n",
      "Epoch 18/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.4365 - accuracy: 0.8841 - val_loss: 0.3611 - val_accuracy: 0.9097\n",
      "Epoch 19/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.4275 - accuracy: 0.8859 - val_loss: 0.3533 - val_accuracy: 0.9108\n",
      "Epoch 20/20\n",
      "422/422 [==============================] - 4s 10ms/step - loss: 0.4194 - accuracy: 0.8872 - val_loss: 0.3463 - val_accuracy: 0.9117\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x16a66a760>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# History tracks the evolution of the metrics during learning\n",
    "history = tf.keras.callbacks.History()\n",
    "\n",
    "# Training procedure\n",
    "model.fit(\n",
    "    X_train, T_train, # training data\n",
    "    batch_size=128,  # batch size\n",
    "    epochs=20, # Maximum number of epochs\n",
    "    validation_split=0.1, # Percentage of training data used for validation\n",
    "    callbacks=[history] # Track the metrics at the end of each epoch\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The training has now run for 20 epochs on the training set. You see the evolution of loss function and accuracy for both the training and validation sets.\n",
    "\n",
    "To test your trained model on the test set, you can call `model.evaluate()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test loss: 0.39123424887657166\n",
      "Test accuracy: 0.8955000638961792\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(X_test, T_test, verbose=0)\n",
    "\n",
    "print('Test loss:', score[0])\n",
    "print('Test accuracy:', score[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use the `History()` object to visualize the evolution of the the training and validation accuracy during learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA34AAAFzCAYAAABhKNvjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABn+0lEQVR4nO3dd3iUZdbH8e8hlNCkdxBQQcVCEVEREIkFK4INXsUu4uraVte21tVddy1rxy52dNeyFrAr9hVQFBBFQMAIUkW6ELjfP87EDCEhk2R6fp/req5n5mlzMgx5cuYux0IIiIiIiIiISPaqluoAREREREREJLGU+ImIiIiIiGQ5JX4iIiIiIiJZTomfiIiIiIhIllPiJyIiIiIikuWU+ImIiIiIiGS56qkOIJ6aNm0aOnTokOowREQkwSZNmrQkhNAs1XFkCt0fRUSqjtLukVmV+HXo0IGJEyemOgwREUkwM5ub6hgyie6PIiJVR2n3SHX1FBERERERyXJK/ERERERERLKcEj8REREREZEsl1Vj/EREUmnDhg3k5+ezbt26VIeSNXJzc2nbti01atRIdShZR5/X+NPnVUTSmRI/EZE4yc/Pp379+nTo0AEzS3U4GS+EwNKlS8nPz6djx46pDifr6PMaX/q8iki6U1dPEZE4WbduHU2aNNEf0XFiZjRp0kQtUgmiz2t86fMqIulOiZ+ISBzpj+j40vuZWHp/40vvp4ikMyV+IiJZYunSpXTr1o1u3brRsmVL2rRp8/vz9evXb/XciRMnct5555X5Gr17945XuFLF6fMqIpJcGuMnIpIlmjRpwuTJkwG49tprqVevHhdffPHv+wsKCqheveRf+z179qRnz55lvsYnn3wSl1hF9HkVEUkutfiJiGSxU045hYsuuoj999+fSy+9lM8//5zevXvTvXt3evfuzXfffQfA+++/z+GHHw74H+GnnXYa/fv3Z7vttuPOO+/8/Xr16tX7/fj+/ftzzDHHsNNOO3HCCScQQgBg7Nix7LTTTvTp04fzzjvv9+uKlEWfVxGRxFGLn4hIIlxwAURaM+KmWze4/fZynzZjxgzefvttcnJyWLFiBR988AHVq1fn7bff5oorruD555/f4pxvv/2W9957j5UrV7Ljjjty9tlnbzFF/Zdffsm0adNo3bo1++67Lx9//DE9e/bkrLPO4oMPPqBjx44MGzasgj+sJJU+r/q8ikjWU+JXKAT47DPIzYXu3VMdjYhI3Bx77LHk5OQA8Ouvv3LyySfz/fffY2Zs2LChxHMOO+wwatWqRa1atWjevDkLFy6kbdu2mx3Tq1ev37d169aNOXPmUK9ePbbbbrvfp7MfNmwYDzzwQAJ/Osk2+ryKSLYpKIDffita1q3b/Hn0csghUC1BfTKV+EU7+mjYf3946qlURyIima4CLR2JUrdu3d8fX3XVVey///68+OKLzJkzh/79+5d4Tq1atX5/nJOTQ0FBQUzHFHafkwyjz6uIZLkQYONG2LDBE7GCAk/A1qzZ+rJ2bWzHbC2x27Qp9jjXrvV2qERQ4lfIDPr0gQ8/THUkIiIJ8+uvv9KmTRsARo8eHffr77TTTsyePZs5c+bQoUMHnn322bi/hlQd+ryKVC0FBbB0KSxeDIsW+Tr68aJFsGSJJ1PRCVxBwebPS9q3cWPlYqtRA+rU2XKpXRuaNYNatTZfcnO33BbL/mK91ONKiV+0vn3h3/+GefNg221THY2ISNz9+c9/5uSTT+a2225jwIABcb9+7dq1uffeexk4cCBNmzalV69ecX8NqTr0eRVJL4WtZmUlWyUlXr/95klbSclc4bZly/w1ijODJk08wWraFBo2hOrVfalRo+hx8eelPS5ccnNLTuZKSu4SmZAli2VTN4eePXuGiRMnVvwCX34JPXp4V8//+7/4BSYiVcL06dPZeeedUx1Gyq1atYp69eoRQuCcc86hU6dOXHjhhRW+Xknvq5lNCiGUPZ+/ACXfH/V5dcn4vIokw4YN8OuvsHy5r1euLLuLYizL2rWbJ3HxEJ3INWsGzZtv+Th6W5MmEBn6KzEo7R6pFr9ou+8O9et7d08lfiIiFfLggw/y2GOPsX79erp3785ZZ52V6pAyipkNBO4AcoCHQgg3FdvfCHgE2B5YB5wWQpia9ECzhD6vki7WroVffilK3goTuFgfr1kT+2uV1m2xTh1o2XLz57m5sbeqbe15zZqewDVvrkQuVZT4RcvJgd694aOPUh2JiEjGuvDCCyvVYlKVmVkOcA9wIJAPTDCzl0MI30QddgUwOYQw2Mx2ihyfl/xos4M+rxJP69Z58lbSsmxZ6ft++cXP3Zrq1b2LY+HSoAG0bl30OHp7w4bellG3bvZ2W5TyU+JXXJ8+cNVV/j+wUaNURyMiIlVLL2BmCGE2gJmNAQYB0YlfF+DvACGEb82sg5m1CCEsTHq0IlXAmjXw88+wYIGvC5fiz5ct81a7ralf3/+8bNzY1zvt5OviS0mJXO3a3kVSpKKU+BXXp4+vP/4YDj88tbGIiEhV0wb4Mep5PrBXsWO+AoYAH5lZL6A90BZQ4icSgw0bYMUKX5Yvh4ULt57UrVy55TWqVYMWLbxbZMuWsNtu3n2xMKEraSmckEQkVfTxK65XL2///ugjJX4iIpJsJX2fX3wWtpuAO8xsMjAF+BLYYsoFMxsBjADYVjNVSxZZtQpmzSpK3n79tehxSc+Lb9taq9w223gi16qVz/dXmNi1alX0uGVLn1lSY9Qk0yjxK65OHdhjD43zExGRVMgH2kU9bwvMjz4ghLACOBXAzAz4IbJQ7LgHgAfAZ/VMULwiCfXzzz7p+uTJReuZM0ue8h+8Ja5BA0/gCpfmzWGHHYqeR+9v0KAomWvRwv8MFMlWSvxK0qcP3Hmnj7LNzU11NCIiMenfvz+XX345Bx988O/bbr/9dmbMmMG9995b4vG33HILPXv25NBDD+Xpp5+mYcOGmx1z7bXXUq9ePS6++OJSX/ell16ic+fOdOnSBYCrr76afv36ccABB8TnB6taJgCdzKwj8BMwFNhsmmkzawisCSGsB84APogkgxlFn1eJtmmTJ3TRCd6XX3o3zEIdO0K3bjB8eNHYuOLJXJ06GgcnUholfiXp2xduuQUmTPDHIiIZYNiwYYwZM2azP6THjBnDzTffXOa5Y8eOrfDrvvTSSxx++OG//yF9/fXXV/haVV0IocDMzgXewMs5PBJCmGZmIyP77wN2Bh43s434pC+npyzgStDntepatw6mTt08yfvqK1i92vdXrw677AIDB0L37p7sde3qY+REpOKqJerCZtbOzN4zs+lmNs3Mzi/hGDOzO81sppl9bWY9ovYNNLPvIvsuS1ScJerd29fq7ikiGeSYY47h1Vdf5bfffgNgzpw5zJ8/n6effpqePXuyyy67cM0115R4bocOHViyZAkAN954IzvuuCMHHHAA33333e/HPPjgg+y555507dqVo48+mjVr1vDJJ5/w8ssvc8kll9CtWzdmzZrFKaecwn/+8x8A3nnnHbp3785uu+3Gaaed9ntsHTp04JprrqFHjx7stttufPvtt4l8azJKCGFsCKFzCGH7EMKNkW33RZI+QgifhhA6hRB2CiEMCSH8ktqIK0af16ph3Tr43//g7rvh5JN9EpR69WDPPeHMM+GJJ3ys3GmnwSOPwBdf+Bi+yZNh9Gg4/3zYbz8lfSLxkMgWvwLgTyGEL8ysPjDJzN4qVovoEKBTZNkLGAXsFWMdo8Rp2hR23tkLuV9+eVJeUkSyywUX+B8u8dStG9x+e+n7mzRpQq9evXj99dcZNGgQY8aM4fjjj+fyyy+ncePGbNy4kby8PL7++mt23333Eq8xadIkxowZw5dffklBQQE9evRgjz32AGDIkCGceeaZAPzlL3/h4Ycf5o9//CNHHnkkhx9+OMccc8xm11q3bh2nnHIK77zzDp07d+akk05i1KhRXHDBBQA0bdqUL774gnvvvZdbbrmFhx56qLJvkVSQPq/6vMbDhg0wbRpMnOidpiZMgClToCAy9VCLFj6NwqBB/vno3t27b1ZLWDOEiERLWOIXQlgALIg8Xmlm0/FpqqOTt0HA4yGEAHxmZg3NrBXQgbLrGCVW377w7LOwcaOmbRKRjFHYfa7wD+lHHnmE5557jgceeICCggIWLFjAN998U+of0h9++CGDBw+mTmSGgyOPPPL3fVOnTuUvf/kLy5cvZ9WqVZt10SvJd999R8eOHencuTMAJ598Mvfcc8/vf0gPGTIEgD322IMXXnihsj+6ZCB9XjPXpk0wY8bmSd6XXxYVIW/YEHr2hEsu8da9nj2hbVuNv5MqKgT/z7F6tS+rVhU9Lr6ce27C/qMkZYyfmXUAugP/K7arpHpFbUrZXryOUWL16QMPPOBfXZVywxERKc3WWjoS6aijjuKiiy7iiy++YO3atTRq1IhbbrmFCRMm0KhRI0455RTWFf5lVgor5YZzyimn8NJLL9G1a1dGjx7N+++/v9XrhNKm3YuoVasWADk5ORQUbFGNQJJIn1d9XrcmBJg715O7wkRv0iQvjQBQt66XPjj7bE/y9twTtt9eSZ5kuI0bi2qBlLWsXLl58lY8sVuzxr8ticXppydsetmEJ35mVg94HrighFnHSqtXFEsdo8LrJ6ZOUWEh9w8/VOInIhmjXr169O/fn9NOO41hw4axYsUK6tatS4MGDVi4cCHjxo2jf//+pZ7fr18/TjnlFC677DIKCgp45ZVXOOusswBYuXIlrVq1YsOGDTz11FO0adMGgPr167OyhArHO+20E3PmzGHmzJnssMMOPPHEE+y3334J+bklM+nzmp6WLvXk7vPPi5bFi31fzZo+0cqJJxa15O28szpHSZrYtMkTrZUrPfkqaR39eGuJ3apVZb9erVo+pWz9+v4NSN26Poi1RYui52Ut9ept/rx27YS9PQlN/MysBp70PRVCKKlfRGn1imqWsn0LCatT1KEDtGnjE7ycc07cLisikmjDhg1jyJAhjBkzhp122onu3buzyy67sN1227Hvvvtu9dwePXpw/PHH061bN9q3b0/fqJmN//rXv7LXXnvRvn17dtttt9//eB46dChnnnkmd9555++TZADk5uby6KOPcuyxx1JQUMCee+7JyJEjE/NDS8bS5zW11q71LprRSd6sWb7PzGfXPPxw6NXLE73ddvPkTyShVq2CRYt8Wbhw88eLFxclZsUTu8KpYWORm1tUC6RwadVq8+fFl4YNN38e6QmQKaysrg0VvrD3vXgMWBZCuKCUYw4DzgUOxbty3hlC6GVm1YEZQB5ex2gC8H8hhGlbe82ePXuGiRMnxu+HGDrUE78ff1R/BREp0/Tp09l5551THUbWKel9NbNJIYSeKQop45R0f9TnNTHS+X3duBG+/XbzJO/rr4smX2nXzhO8wmWPPbwhQ6TS1q+H5ctLT+aKb1uzpuTrNGgAzZp5Alavnn9AK7KuV8/rhmSp0u6RifyJ9wWGA1PMbHJk2xXAtvB7LaKxeNI3E1gDnBrZV2IdowTGWrI+fXyCl7lzvQVQREREJANs2uTfW0+cWJTkTZxY1HutQQNvwbv00qLWvFatUhuzpKlNm/yD8+uvnrxVZL12bcnXzsmB5s19adECOncuehy9LlwyrIUt3SRyVs+PKHmsXvQxASixH2UIYSyeGKZOYZeRjz5S4iciIiJpJQT4+WefXfP77zdfZs0q+lu7Zk0vn3DKKUWteZ06qYxClROCt6QtW+YDOYsvJW1ftgx++aXsiUlyc7fsDrnttps/b9hwy2SuUSN9EJMoe9s442HXXb3v70cf+ShmERERkSQKwYc0FU/svv8eZs7cfP6JGjV8Ns1OneCgg3zdo4dPxqKGkiwUgo9tW7hwy2XRIv/gFE/kfvut9OvVqweNG0OTJr60b+/rxo03T95KWusDlhGU+G1NTg707u0ze4qIxCCEUOr08lJ+iRqHLk6f1/iq7Oc1BJg8GV5+2cfiFSZ4K6LmRM/J8aLnnTvDfvt5cle4bLutZtfMeJs2eQtbSYlcSdtKKndi5glbs2a+3m4778tbmNBFJ3fR25S8ZT0lfmXp2xeuvNK/JWnSJNXRiEgay83NZenSpTRp0kR/TMdBCIGlS5eSm5ub6lCykj6v8VWZz+vChfDUUzB6NEyZ4j3f2rf3ZG7vvTdP7jp08JY9yTDr13u/3AULtlxHP164EDZs2PL8wrFwLVr4svPOmz8vXJo394QviycukYrTp6IshfX8PvkEjjgitbGISFpr27Yt+fn5LC4seCWVlpubS9u2bVMdRlbS5zX+yvN5/e03eOUVeOwxGDfOZ9zs1QvuvReOP94bYCQDrF8PP/0E8+bB/PklJ3MLFvhYueLMPElr1QpatvTaGS1b+lI8mWvcWGPhpNKU+JVlzz39q7UPP1TiJyJbVaNGDTp27JjqMERios9r8oXgM2uOHg3PPOM9+lq3hosvhpNP9kYcSSMheI+vefNKXn780ZO64l18a9b0ZK5VK++T269f0fOWLYseN2um5ltJKiV+Zald25O/jz5KdSQiIiKSgebPhyef9IRv+nSfAHHwYE/2DjhA4/JSpqAA5swpPbGbN2/LMgS5uT6YctttYeDAosft2nkW36qVT3ii7tOShpT4xaJPH/jXv/w/f+3aqY5GRERE0tzatfDf/3pXzjff9Dk79t0XHngAjjvOJ0KUJNm4EX74AaZN82XqVF9/992Ws1y2auWJ3G67wWGHFSV2hUvTpkrqJGMp8YtFnz7wz3/ChAneXC8iIiJSTAjw2Wfesvfss163ul07uOIKOOkkn5xFEmjTJm/BK0zwCpfp0zef/bJ9ex9Pd/DB3r+2Y0dP6tq00cyWktWU+MVi3319/eGHSvxERERkMxs3wiOPwM03e/mFOnXg6KO9YHr//pqTI+5C8G6YxVvwpk/3AuWF2rb1BG///X29666e6NWvn7rYRVJIiV8sGjf2Xxga5yciIiJR3n8fLrgAvvoK9trLE8BjjlFuERch+ADJwsSuMMn75pvNK9e3bu1/p40Y4etddoEuXdSfVqQYJX6x6tPHp+DauFGjsEVERKq42bN9Ns4XX/Seg8895wmfhn9VQAhejLx4gjdtmveXLdSihSd1p55a1ILXpQs0apS62EUyiBK/WPXtC/ff75VVu3VLdTQiIiKSAitWwN/+5nO+1agBN9wAF12kud9itnTp5old4eOlS4uOadzYk7r/+z9fF7biNW2aurhFsoASv1gVFnL/6CMlfiIiIlXMxo0+acuVV8LChV6K4W9/816GUop16+DLL+HTT33Wm88+89p3hbbZxhO6IUOKWvB22cVb9tR0KhJ3Svxite22Pkj4o4/g3HNTHY2IiIgkyYcfwvnnew7Tuze88oqX+JUohROuFCZ5n37qb9iGDb6/fXufLG+PPTzB23VXn0VTCZ5I0ijxi5WZd/ccP95/uekXlYiISFabMwf+/Gf497+9LMPTT8PQofoTAPDZMydOLEryPvsMfv7Z99Wu7ZnxhRfCPvv4rDetWqU2XhFR4lcuhRO8zJnjNV9EREQk66xaBX//O9x6q5diuO46n8ilTp1UR5YiIcCsWZsneV995f1fAXbYAQ44wJO8vff24uc1aqQ2ZhHZghK/8oge56fET0REJKts2gRPPAGXXw4LFsAJJ8BNN/lIjyrn55/hrbfgzTd9vXChb69Xz1vwLrvMk7y999akKyIZQolfeeyyi9eE+fBDGD481dGIiIhInHz8sdfjmzgRevWCF17wnKbKWLfOv9h+80144w34+mvf3qwZHHgg7Left+h16aKyViIZSolfeeTk+MBkFXIXERHJCvn5cMklMGaMzzXyxBNeRaBatVRHlmAheCmFN9/0Zfx4T/5q1PAeTjfdBAcdBF27VoE3Q6RqUOJXXn36wNixsGSJujaIiIhksNmzfd62Zcvgqqvg0kuhbt1UR5VAixfD228XJXvz5/v2nXeGs87yRG+//bL8TRCpupT4lVfhOL+PP4ZBg1Ibi4iIiFRIfr7PR7JuHXz+uc9HknXWr4dPPvGum2++CV984dsbN/bumwcd5Ot27VIbp4gkhRK/8tpzT6hZ07t7KvETERHJOIsWedK3ZAm8+26WJX0hwKRJ8OCDPhP5ypVQvboXILzhBk/2evTQOD2RKkiJX3nl5nryp3F+IiIiGeeXXzz3mTfPG8J69kx1RHHy66/w1FOe8E2e7LX0jjsOjj4a+veH+vVTHaGIpJgSv2IKCvyLsa3q08eL+6xZU4WL+oiIiGSWlSvh0ENh+nR45RUf35fRQvCunA8+CM89B2vXQvfucO+9PkNNgwapjlBE0oimaYrYtAm6dfOZvcrUt69niJ9/nuiwREREJA7WroUjj4QJE+DZZ73VL2MtXQr/+peXmerTx2tPnHSS16L44gs4+2wlfSKyBbX4RVSrBk2aeF//MvXu7euPPvLuEyIiIpK21q+HY47xigVPPAFHHZXqiCpg0yZ4/31v3XvhBf+h9t4bHn7Yu3TWq5fqCEUkzSnxi5KXB1de6YO+mzffyoGNGsGuu3ohdxEREUlbBQVw4oleien+++GEE1IdUTn9/DOMHg0PPQSzZkHDhl564cwzs2xWGhFJNHX1jJKX5+v33ovh4L59vV99QUFCYxIREZGK2bQJzjgD/v1vH5o/YkSqI4rRxo0wbhwMGeKlFi6/HNq29ebK+fPhzjuV9IlIuSnxi7LHHrDNNvDOOzEc3KcPrFoFU6YkPC4REREpnxDgvPPgscfg2mvhootSHVEMNmyAUaNgu+18FpqPPoILL4TvvvNuniee6LN1iohUgBK/KNWr+5C9mBM/UHdPERGRNBOCN5Ldcw9cfDFcfXWqIyrDpk0+40yXLvCHP3gr33PPeZX5f/4TOndOdYQikgWU+BWTlwezZ8OcOWUcuO22vqien4iISFr529/gH/+AkSM9bzJLdURb8dZbXh946FBvzXv1Vf9S+dhjoWbNVEcnIllEiV8xAwb4OuZWv48+8q8WRUREJOXuuAP+8hfvFXnPPWmc9E2cCAcc4HUlli6Fxx+HL7+Eww5L46BFJJMp8Stml12gRYsYyzr06QMLFngToYiIiKTUww/DBRfA4MHw6KNeqintzJjh5Rf23BO++gpuv93H8A0fDjk5qY5ORLJYOv5KTCkzb/V7990YGvL69vW1unuKiIik1JgxXuFg4EB45hkft59W5s/3Mgxdunhtiauv9vIM558PtWqlOjoRqQISlviZ2SNmtsjMppay/xIzmxxZpprZRjNrHNk3x8ymRPZNTFSMpcnL87I533xTxoFdung9HSV+IiIiKfPKK95g1rcvPP98muVRy5f7TDM77ODNkGef7Qnfddf5VOIiIkmSyBa/0cDA0naGEG4OIXQLIXQDLgfGhxCWRR2yf2R/zwTGWKLCen5ljvOrVg323Vcze4qIiKTIO+/4PCjdu3sCWKdOqiOKWLsWbr7ZSzPcdJP3P/32W7jrLh9TIiKSZAlL/EIIHwDLyjzQDQOeSVQs5dWhg/+ejmmCl759vW/+4sWJDktERESifPwxHHmkVzt4/fU0aUArKPDBhp07w5//DHvt5ZO2PPWU/3EhIpIiKR/jZ2Z18JbB56M2B+BNM5tkZiPKOH+EmU00s4mL45h85eV5rdSCgjIOLKzn9/HHcXttERER2bovvvAa523aeEWExo1THFAI8OKLsPvucMYZ0Lo1vPcejBsH3bqlODgRkTRI/IAjgI+LdfPcN4TQAzgEOMfM+pV2cgjhgRBCzxBCz2bNmsUtqLw8WLECJk0q48CePX0wgbp7ioiIJMXKlV71oFEj752T8p6TK1fCkCG+bNrkAw0/+wz6909xYCIiRdIh8RtKsW6eIYT5kfUi4EWgV7KD2n9/X5dZ1qFWLejVSxO8iIiIJMltt/kkbGPGQLt2KQ5m1izYZx8fYHjLLTB1qieAqsUnImkmpYmfmTUA9gP+G7WtrpnVL3wMHASUODNoIjVvDrvtVo5C7l98AatXJzwuERGRqmzRIs+vhgyBvfdOcTBvveX1+BYsgDffhD/9KQ3rSIiIuESWc3gG+BTY0czyzex0MxtpZiOjDhsMvBlCiM6YWgAfmdlXwOfAayGE1xMV59bk5fnQvXXryjiwTx8fDPi//yUlLhERkarqxht9wsy//S2FQYQA//qXFw1s2xYmTPAiwCIiaSxhX0uFEIbFcMxovOxD9LbZQNfERFU+eXlw++3wySdl/D7v3du7dHz0kX7xi4iIJMgPP8CoUXDaabDjjikKYt06L8T++OPe7PjYY1CvXoqCERGJXTqM8Utb/fpBTk4M3T0bNvR+oRrnJyIikjBXXeX35WuuSVEAP/0E++3nSd/118O//62kT0QyhhK/rdhmG5+3JeZxfp9+GkP9BxERkdKZ2UAz+87MZprZZSXsb2Bmr5jZV2Y2zcxOTUWcyfbVV/D003D++V7CIek++8zH833zjZdtuOoqqKY/o0Qkc+g3Vhny8rzr/q+/lnFg376wapXfmURERCrAzHKAe/ByRl2AYWbWpdhh5wDfhBC6Av2BW82sZlIDTYHLL/cONpdemoIXf/RRb+mrXdu/5D3qqBQEISJSOUr8yjBggJfk+eCDMg4sLOSu7p4iIlJxvYCZIYTZIYT1wBhgULFjAlDfzAyoBywDsrq7yfvvex30yy/32n1JU1AAF1zggwr79fNvgnfdNYkBiIjEjxK/MuyzD+TmxtDds21baN9ehdxFRKQy2gA/Rj3Pj2yLdjewMzAfmAKcH0LYVPxCZjbCzCaa2cTFixcnKt6EC8Fb+dq2hXPPTeILL10KBx8Md9wBF17omWfjxkkMQEQkvpT4lSE31xvzYhrn17evt/iFkPC4REQkK5VU9bv4TeVgYDLQGugG3G1m22xxUggPhBB6hhB6NmvWLN5xJs2LL8Lnn8O113pPy6SYMsXH8338MYwe7RXjVZ9PRDKcEr8Y5OXB1KmwcGEZB/bp4wfNmpWUuEREJOvkA+2inrfFW/ainQq8ENxM4AdgpyTFl1QFBXDFFbDzznDyyUl60Rde8O4+69bB+PFJfGERkcRS4heDvDxfv/tuGQcWjvNTd08REamYCUAnM+sYmbBlKPBysWPmAXkAZtYC2BGYndQok+TRR+G777xYe8Ib3DZt8mbFo4/2cXwTJ8JeeyX4RUVEkkeJXwx69PCZxMrs7rnzzt7/XxO8iIhIBYQQCoBzgTeA6cBzIYRpZjbSzEZGDvsr0NvMpgDvAJeGEJakJuLEWbPG87B99oFBxae3ibeVKz3hu+46b+F7/31o3TrBLyoiklzqsB6DnBzo3z+GxK9aNdh3XyV+IiJSYSGEscDYYtvui3o8Hzgo2XEl2113wfz58MwzYCWNfIyXWbM8s/z2W7j9djjvvAS/oIhIaqjFL0Z5eTBnDvzwQxkH9ukDM2bEMCBQRERESrJsGfz973DYYV5FIWGWL4cDDvAM8/XXvTq8kj4RyVJK/GI0YICvy2z1Kxzn9/HHCY1HREQkW910E6xY4clfwoQAZ5wB+fnw2mueAIqIZDElfjHaeWdo1SqGxG+PPbwGhLp7ioiIlFt+vnfzPPFE2G23BL7QqFHw/PM+c8w++yTwhURE0oMSvxiZeavfu++WUaavVi3o1csHhouIiEi5XHutT7B5/fUJfJEvv/Si7IccAn/6UwJfSEQkfSjxK4e8PFi0yGv6bdUhh/hNZd68pMQlIiKSDb75xks4/OEP0KFDgl5k5Uo47jho2hQef9wnZhMRqQL0264cCuv5ldndc/BgX7/0UiLDERERySpXXgl16/o6IUKAs86C2bN9utCmTRP0QiIi6UeJXzlsuy3ssEMMid+OO0KXLvDii0mJS0REJNN9+ql/X/rnPycwH3v4YU/4rrsuwdOFioikHyV+5ZSXB+PHQ0FBGQcOHgwffACLFyclLhERkUwVAlx6KbRo4UPvEmLqVPjjH332zssvT9CLiIikLyV+5ZSX58MDJk4s48AhQ3x0+iuvJCUuERGRTDV2LHz4IVx9tXf1jLvVq31cX4MG8MQTkJOTgBcREUlvSvzKqX9/X5fZ3bN7d2jfHl54IdEhiYiIZKyNG70Bbvvt4cwzE/Qi554L334LTz4JLVsm6EVERNKbEr9yatYMunaNIfEz8+6eb73lTYQiIiKyhaefhilT4MYboUaNBLzA44/D6NE+Y4yKtItIFabErwLy8uCTT2Dt2jIOHDIE1q/3PiwiIiKymd9+g6uugh494NhjE/AC337rtSH69YNrrknAC4iIZA4lfhWQl+c3q48/LuPA3r29iVCze4qIiGxh1CiYOxduuikB5fTWroXjj4fcXG9WrF49zi8gIpJZlPhVQL9+fv8os7tnTg4cdRS89hqsW5eM0ERERDLCihXevfOAA+DAAxPwAhdeCF9/7ZO5tGmTgBcQEcksSvwqoF492GsvePfdGA4ePBhWrYK33054XCIiIpnilltgyRJv7Yu7Z5+F++/3ooCHHJKAFxARyTxK/CooL89LOixfXsaBAwbANtuou6eIiEjEzz/Drbd6hYU99ojzxWfN8ulB994bbrghzhcXEclcSvwqaMAAL9M3fnwZB9aqBYcfDv/9bwxV30VERLLfX//qc5/FPS/77TfPJnNyYMyYBE0TKiKSmZT4VdDee0Pt2jGM8wPv7rl0KXz0UcLjEhERSWczZ8IDD3ijXKdOcb74n/8MX3wBjz7qtXRFROR3SvwqqFYt6Ns3xsRv4ECfVUzF3EVEpIr7y1+gZk24+uo4X/ill+DOO+H8831iNRER2YwSv0rIy4NvvoEFC8o4sF49OOggH+cXQlJiExERSTezZvm8KxdcAC1bxvHCc+fCqaf6gMF//COOFxYRyR5K/CohL8/X770Xw8FDhkB+vs8IIyIiUgW99pqvTzstjhfdsAGGDvWB988+611yRERkC0r8KqFbN2jUKMbunkcc4YPNNbuniIhUUePG+bi+7beP40WvvBI++wwefDDOFxYRyS5K/CohJwf2398TvzJ7cDZuDP37w/PPq7uniIhUOWvWwPvvw6GHxvGiY8fCzTfDyJE+m6eIiJQqYYmfmT1iZovMbGop+/ub2a9mNjmyXB21b6CZfWdmM83sskTFGA95eT60YPbsGA4eMgRmzIDp0xMel4iISDp5/31Yty6O9dTz8+Gkk2D33eG22+J0URGR7JXIFr/RwMAyjvkwhNAtslwPYGY5wD3AIUAXYJiZdUlgnJUyYICvY+ruOWiQr9XdU0REqphx47wM0n77xeFiIXjSt24dPPecX1hERLYqYYlfCOEDYFkFTu0FzAwhzA4hrAfGAIPiGlwc7bgjtG4dY+LXpo0XAFRZBxERqWLGjfPhEbm5cbjYhAk+s9qNN/qNWEREypTqMX77mNlXZjbOzHaJbGsD/Bh1TH5kW1oy8+6e777rE4qVafBgLy47d27CYxMREUkH33/vpRziNr5v1CioW9dLOIiISExSmfh9AbQPIXQF7gJeimy3Eo4tdTYUMxthZhPNbOLixYvjH2UM8vJgyRKYWuJoxmIGD/a1unuKiEgVMXasr+Myvu+XX2DMGDjxRNhmmzhcUESkakhZ4hdCWBFCWBV5PBaoYWZN8Ra+dlGHtgXmb+U6D4QQeoYQejZr1iyhMZemsJ5fTN09O3WCXXdV4iciIlXGuHHQuTNst10cLvbYYz627+yz43AxEZGqI2WJn5m1NDOLPO4ViWUpMAHoZGYdzawmMBR4OVVxxqJtW7+hxZT4gc/u+eGHsGhRQuMSERFJtcIyDnFp7QsB7rvPx8t37RqHC4qIVB2JLOfwDPApsKOZ5ZvZ6WY20sxGRg45BphqZl8BdwJDgysAzgXeAKYDz4UQpiUqznjJy4Px42HDhhgOHjLEb14vp3U+KyIiUmnvvw+//RanxO+99+C779TaJyJSAdUTdeEQwrAy9t8N3F3KvrHA2ETElSgDBvhY8wkToHfvMg7efXfo2NFn9zzjjKTEJyIikgpxLeMwahQ0bqxi7SIiFZDqWT2zxv77+wyfMXX3NPNWv7ffhl9/TXhsIiIiqRCCT+wyYEAcyjgsWAAvveQzecalJoSISNWixC9OmjSBbt3KMc5v8GDvFzo2oxo2RUREYvb99zB7dpy6eT70EBQUwFlnxeFiIiJVjxK/OMrLg08/9YHsZdpnH2jZUsXcRUQka40b5+tKJ34FBfDAA3DggT47toiIlJsSvzjKy4P16+Hjj2M4uFo1OOoovyuuXZvo0ERERJJu3DjYccc4lHF47TXIz9ekLiIilaDEL4769oUaNcrZ3XP1anjrrYTGJSIikmxxLeMwahS0aQNHHBGHi4mIVE1K/OKobl0vLRRz4te/PzRsqGLuIiKSdd57L05lHGbNgjfegDPPhOoJm4xcRCTrKfGLs7w8mDQJfvklhoNr1oTDD/d6fgUFCY9NREQkWcaNgzp1oF+/Sl7o/vshJ0flj0REKkmJX5wNGODTV7//fownDBkCy5bBBx8kMiwREZGkCcETv0qXcfjtN3jkETjySO/qKSIiFabEL8722su/4Yy5u+fBB3tlW83uKSIiWSJuZRz+8x9YulSTuoiIxIESvzirWRP228+/6QwhhhPq1IGBA32c36ZNCY9PREQk0QpL1FY68Rs1CnbYwcdRiIhIpSjxS4Bjj/VvOj//PMYThgyB+fNhwoSExiUiIpIMhWUcOnasxEWmTPH6SCNHegkkERGpFP0mTYCjj/YxDU88EeMJhx3mM5Wpu6eIiGS4NWtg/Pg4tfbVqgWnnBKPsEREqjwlfgmwzTYwaBCMGQMbNsRwQqNGPgL+hRdi7B8qIiKSngrLOBx6aCUusnKlf3t6/PHQpEncYhMRqcqU+CXIiSf6ePQ33ojxhMGDYeZMmDYtoXGJiIgkUlzKODz1FKxapUldRETiSIlfghx8sH9J+eSTMZ4waBCYqZi7iIhkrBB8YpcBA7yXZoUvMmoUdOvmU2WLiEhcKPFLkBo1YOhQ+O9/4ddfYzihVSvYZx+N8xMRkYw1Ywb88EMlx/d9+il8/bVP6mIWt9hERKo6JX4JNHw4rFtXjlxuyBCYPNnvmiIiIhlm3DhfVyrxGzUK6teHE06IS0wiIuKU+CVQr15efijm7p6DB/ta3T1FRCQDjRsHO+1UiTIOS5bAc8/5N6f16sU1NhGRqk6JXwKZ+SQv770H+fkxnLDddtC1qxI/ERHJOKtXw/vvV7K179FHYf16TeoiIpIASvwS7IQTfJz600/HeMKQIV6w9uefExqXiIhIPL33nudsFU78Nm2C+++HPn1g113jGpuIiCjxS7gddvA5W8rV3TMEnxVGRESqHDMbaGbfmdlMM7ushP2XmNnkyDLVzDaaWeNUxBqt0mUc3n4bZs1Sa5+ISIIo8UuCE0+EKVN8krIy7bqrZ4vq7ikiUuWYWQ5wD3AI0AUYZmZdoo8JIdwcQugWQugGXA6MDyEsS3qwm8XkiV9eXiXKOIwaBc2awdFHxzU2ERFxSvyS4LjjoHp1eOKJGA4281a/d96B5csTHZqIiCSAmR1uZhW5x/YCZoYQZocQ1gNjgEFbOX4Y8ExFYoynSpdxyM+Hl1+G006rROYoIiJbo8QvCZo29Zvh00/Dxo0xnDBkCBQUwGuvJTw2ERFJiKHA92b2TzPbuRzntQF+jHqeH9m2BTOrAwwEni9l/wgzm2hmExcvXlyOEMpv7FhfVzjxe/BBbzYcMSJuMYmIyOaU+CXJ8OEwf77PeFamXr2gdWsVcxcRyVAhhBOB7sAs4FEz+zSSiNUv49SSKpaHUo49Avi4tG6eIYQHQgg9Qwg9mzVrFnPsFVFYxqFDhwqcvGGDJ34HH+yzW4uISEIo8UuSww+HbbaJcZKXatXgqKPg9ddhzZpEhyYiIgkQQliBt8aNAVoBg4EvzOyPWzktH2gX9bwtML+UY4eSBt08V6+G8ePh0EMreIGXX4YFCzSpi4hIginxS5LateGYY+A//4kxlxsyxA98882ExyYiIvFlZkeY2YvAu0ANoFcI4RCgK3DxVk6dAHQys45mVhNP7l4u4foNgP2AlE8BXekyDqNGQbt2cNhhcY1LREQ2p8QviYYPh1Wr/MvNMvXrB40aaXZPEZHMdCzwrxDC7pFZOBcBhBDWAKeVdlIIoQA4F3gDmA48F0KYZmYjzWxk1KGDgTdDCKsT9yPEZtw4qFsX+vatwMkzZvhkZiNGQE5O3GMTEZEi1VMdQFXSrx+0bevdPYcOLePgGjXgyCO9nt+6dZCbm5QYRUQkLq4BFhQ+MbPaQIsQwpwQwjtbOzGEMBYYW2zbfcWejwZGxyvYigrBJ3YZMKCCk3Hed59Pe33GGXGPTURENqcWvySqVg1OOMGH7i1aFMMJJ5/sJR1iqgMhIiJp5N/ApqjnGyPbssp338GcORUc37d2LYwe7SWMWraMc2QiIlKcEr8kO/FEL+nw7LMxHNy/P/ToAbfeCps2lXm4iIikjeqROnwARB7XTGE8CTFunK8rNL7v2Wfhl180qYuISJIo8UuyXXeFbt1inN3TDC65xL9SffXVRIcmIiLxs9jMjix8YmaDgCUpjCchxo2DnXeG9u0rcPJ993kNiP794x2WiIiUQIlfCpx4Inz+uY9pL9Mxx/gd9eabEx6XiIjEzUjgCjObZ2Y/ApcCZ6U4prhatcrLOFSote/LL+F//4ORI/1LThERSTglfikwbJjf52Jq9ateHS68ED76CD77LOGxiYhI5YUQZoUQ9ga6AF1CCL1DCDNTHVc8VaqMw6hRXufopJPiHpeIiJQspsTPzOqaWbXI485mdqSZ1SjjnEfMbJGZTS1l/wlm9nVk+cTMukbtm2NmU8xssplNLM8PlAlat4a8PE/8QojhhNNPh4YN4ZZbEh2aiIjEiZkdBvwBuNDMrjazq1MdUzxVuIzDr7/CU0/59NaNGiUkNhER2VKsLX4fALlm1gZ4BziVsqeRHg0M3Mr+H4D9Qgi7A38FHii2f/8QQrcQQs8YY8wow4fDDz/Ap5/GcHC9ej74/YUXYGZWfWEsIpKVzOw+4Hjgj4Dhdf0qMhIuLYXgiV9eXgXKODzxBKxZo0ldRESSLNbEzyJFZ4cAd4UQBuPdV0oVQvgAWLaV/Z+EEH6JPP0MaBtjLFlh8GDv5RJzpYY//tFr+912W0LjEhGRuOgdQjgJ+CWEcB2wD9AuxTHFTWEZh3J38wzBu3nusQfsuWciQhMRkVLEnPiZ2T7ACcBrkW3xLP5+OjAu6nkA3jSzSWY2oozARpjZRDObuHjx4jiGlFj168NRR/ls1uvXl3k4tGrlzYSPPgoZ9HOKiFRR6yLrNWbWGtgAdExhPHE1NlJevtyJ3/ffwzffwGmnxT0mERHZulgTvwuAy4EXQwjTzGw74L14BGBm++OJ36VRm/cNIfQADgHOMbN+pZ0fQngghNAzhNCzWbNm8QgpaYYP9xJG48aVfSwAf/oTrFsH996b0LhERKTSXjGzhsDNwBfAHOCZVAYUT+PGQZcuFSjjMGeOr3fdNd4hiYhIGWJK/EII40MIR4YQ/hGZ5GVJCOG8yr64me0OPAQMCiEsjXq9+ZH1IuBFoFdlXysdHXggNGsW4+ye4MWSDj8c7r7bx0eIiEjaidwn3wkhLA8hPI+P7dsphJAVk7usWgUffFDB2TznzfP1ttvGNSYRESlbrLN6Pm1m25hZXeAb4Dszu6QyL2xm2wIvAMNDCDOittc1s/qFj4GDgBJnBs101at7aYdXXoHly2M86ZJLYMkSePzxRIYmIiIVFELYBNwa9fy3EMKvKQwpripVxmHuXKhWDdq0iXtcIiKydbF29ewSQlgBHAWMBbYFhm/tBDN7BvgU2NHM8s3sdDMbaWYjI4dcDTQB7i1WtqEF8JGZfQV8DrwWQni9XD9VBjnxRPjtN/jPf2I8oW9fHxB/662wcWNCYxMRkQp708yONsu+6uRjx3oZhz59KnDyvHle06jGVitCiYhIAsQ6QUuNSN2+o4C7QwgbzGyrFehCCMPK2H8GcEYJ22cDXbc8Izv17Ak77ujdPc/Y4t0ogZm3+h13HPz3vzBkSMJjFBGRcrsIqAsUmNk6vKRDCCFsk9qwKqewjMMBB1SgjAN44qduniIiKRFri9/9+MD0usAHZtYeWJGooKoSM2/1Gz/ee8DEZMgQ6NhRBd1FRNJUCKF+CKFaCKFmCGGbyPOMTvoAvv3W71UV6uYJfnK5Z4QREZF4iHVylztDCG1CCIcGNxfYP8GxVRn/93++fvrpGE/IyYGLLvLq7x9/nLC4RESkYsysX0lLquOqrMJZqCuU+G3cCPn5avETEUmRWCd3aWBmtxXWyzOzW/HWP4mD7baDfff1Yu5hqx1oo5x6KjRuDDffnNDYRESkQi6JWq4CXgGuTWVA8VBYxqFCudvChbBhgxI/EZEUibWr5yPASuC4yLICeDRRQVVFw4fD9OkweXKMJ9StC+ecAy+/DN99l8jQRESknEIIR0QtBwK7AgtTHVdlVKqMAxSNZ1BXTxGRlIg18ds+hHBNCGF2ZLkO2C6RgVU1xx7rk5w98UQ5TjrnHKhZE267LWFxiYhIXOTjyV/GevddL+Nw6KEVvIBq+ImIpFSsid9aM/t94mYz2xdYm5iQqqbGjeGww+CZZ6CgIMaTWrSAk0+Gxx7zLjQiIpIWzOwuM7szstwNfAh8leq4KmPcOKhXr4JlHKAo8VOLn4hISsSa+I0E7jGzOWY2B7gbOCthUVVRJ54IP//s36rG7KKL/CvYu+9OWFwiIlJuE4FJkeVT4NIQwompDaniCss45OV5R5MKmTsXGjSAbTJ+clMRkYwU66yeX4UQugK7A7uHELoDAxIaWRV02GHQsKHX9IvZjjvCkUfCvffC6tWJCk1ERMrnP8CTIYTHQghPAZ+ZWZ1UB1VR06dXsowDqIafiEiKxdriB0AIYUUIobB+30UJiKdKy831sX4vvFDOHO6SS2DZMnhU8+2IiKSJd4DaUc9rA2+nKJZKmzLFx6FXOvFTN08RkZQpV+JXjMUtCvndiSd60vfSS+U4ad99YZ99fJKXmAcIiohIAuWGEFYVPok8ztgWv+OPh19+qWSD3dy5avETEUmhyiR+sVack3Lo08e/EC1Xd0+Aiy+GH36AF19MSFwiIlIuq82sR+ETM9uDDJ8UrW5lqveuWAHLlyvxExFJoa0mfma20sxWlLCsBFonKcYqpVo1OOEEePNNn+glZoMGwQ47eEH3mKvAi4hIglwA/NvMPjSzD4FngXNTG1IK/fijr9XVU0QkZbaa+IUQ6ocQtilhqR9CqJ6sIKuaE06ATZtgzJhynJSTA3/6E0yY4BV2RUQkZUIIE4CdgLOBPwA7hxAmpTaqFCos3q4WPxGRlKlMV09JkC5doEePCnT3PPlkaNoUbrklIXGJiEhszOwcoG4IYWoIYQpQz8z+kOq4UkbF20VEUk6JX5oaPhwmTfIptGNWuzacey68+ip8803CYhMRkTKdGUJYXvgkhPALcGbqwkmxefOgenVo1SrVkYiIVFlK/NLU0KE+3q/crX7nnON1IW69NSFxiYhITKqZ2e+zX5tZDlDR0ueZb+5caNvWhyWIiEhKKPFLUy1bwoEHeuK3cWM5TmzaFE491U9csCBh8YmIyFa9ATxnZnlmNgB4BhiX4phSR8XbRURSTolfGhsxwu+VTz1VzhMvugg2bIC77kpIXCIiUqZL8SLuZwPnAF+zeUH3qkXF20VEUk6JXxo76iif5OXqq+G338px4g47wJAhMGoUrFyZqPBERKQUIYRNwGfAbKAnkAeUZ9R29igogJ9+UoufiEiKKfFLY9Wqwd//7kMjHnignCdffLEXy33kkUSEJiIiJTCzzmZ2tZlNB+4GfgQIIewfQrg7tdGlyPz5PmZBiZ+ISEop8UtzBx4I/fvDX/8Kq1aV48S994Y+feBf//JvW0VEJBm+xVv3jggh9Akh3AWUZ6R29iks5aCuniIiKaXEL82Zeavf4sVw++3lPPmSS7y58N//TkRoIiKypaOBn4H3zOxBM8sDrIxzspuKt4uIpAUlfhlg771h0CC4+WZYurQcJx5+OOy4oxd0DyFh8YmIiAshvBhCOB7YCXgfuBBoYWajzOyglAaXKireLiKSFpT4ZYgbb/R5Wm66qRwnVasGf/oTfPEFvPdewmITEZHNhRBWhxCeCiEcDrQFJgOXpTaqFJk3D5o0gbp1Ux2JiEiVpsQvQ+yyCwwfDnffDfn55Thx+HBo0cKbC0VEJOlCCMtCCPeHEAakOpaUmDtXrX0iImlAiV8Gue46nxjt+uvLcVJuLvzxj/D66/D55wmLTUREpEQq3i4ikhaU+GWQDh1g5Eiv0DBjRjlOPPdcaNMGTj21nAUBRUREKiEEb/HTjJ4iIimnxC/DXHmlN+JddVU5TmrQAB58EL75Bq69NlGhiYiIbG75cq9FpBY/EZGUU+KXYVq0gAsvhOee8zlbYnbIIXD66fDPf8L//pew+ERERH6nGT1FRNKGEr8MdPHF0LgxXHFFOU+89Vbv8nnKKbB2bSJCExERKaLi7SIiaUOJXwZq0AAuvxzeeAPGjy/niQ8/DN9+C1dfnbD4REREABVvFxFJI0r8MtQ553jj3eWXl7M2+4EHwllneevfJ58kLD4RERHmzYNataB581RHIiJS5Snxy1C1a8M118Cnn8Irr5Tz5Jtv9m9fTzkF1qxJRHgiIiKe+LVrB9X054aISKol7DexmT1iZovMbGop+83M7jSzmWb2tZn1iNo30My+i+y7LFExZrpTT4VOnXymz40by3Fi/fpeE+L77/1kERGRRFDxdhGRtJHIr+BGAwO3sv8QoFNkGQGMAjCzHOCeyP4uwDAz65LAODNW9epwww0wdSo8/XQ5Tx4wAP7wB7jjDvjgg4TEJyIiVdy8eZrYRUQkTSQs8QshfAAs28ohg4DHg/sMaGhmrYBewMwQwuwQwnpgTORYKcExx0D37t7tc/36cp78j394VfhTT4XVqxMRnoiIVFXr18OCBWrxExFJE6nsdN8G+DHqeX5kW2nbpQTVqsHf/gY//OA12sulXj149FGYPdtniREREYmX/HyffUyJn4hIWkhl4mclbAtb2V7yRcxGmNlEM5u4ePHiuAWXSQ4+GPbbD/761wo03O23H5x3Htx1F7z/fiLCExGRqkg1/ERE0koqE798oF3U87bA/K1sL1EI4YEQQs8QQs9mzZolJNB0ZwZ//zssXOhD9srtb3+DHXbwLp+rVsU9PhERiV0sE5yZWX8zm2xm08ysPBVdk6cw8VOLn4hIWkhl4vcycFJkds+9gV9DCAuACUAnM+toZjWBoZFjZSv22QeOPBL++U9YtrWRlSWpW9e7fM6dC3/+c0LiExGRssUywZmZNQTuBY4MIewCHJvsOGNSWLy9XbutHyciIkmRyHIOzwCfAjuaWb6ZnW5mI81sZOSQscBsYCbwIPAHgBBCAXAu8AYwHXguhDAtUXFmkxtvhBUrfM6WcuvTBy68EEaNgrffjntsIiISk1gmOPs/4IUQwjyAEMKiJMcYm3nzoEULyM1NdSQiIgJUT9SFQwjDytgfgHNK2TcWTwylHHbdFU48Ee6804fttSnvlDg33ACvvgqnnw5TpsA22yQkThERKVVJE5ztVeyYzkANM3sfqA/cEUJ4vPiFzGwEXi6JbVPR3XLePHXzFBFJI6ns6ikJcN11Xsz9r3+twMm1a8Po0T4T28UXxzs0EREpWywTnFUH9gAOAw4GrjKzzluclOox8CreLiKSVpT4ZZmOHWHECHjoIfj++wpcYJ994E9/8toQb7wR9/hERGSrYpngLB94PYSwOoSwBPgA6Jqk+GITgoq3i4ikGSV+Wegvf4FateDqqyt4geuvh5128i6fy5fHMzQREdm6WCY4+y/Q18yqm1kdvCvo9CTHuXVLl8LatWrxExFJI0r8slDLlnDBBTBmDEyeXIEL5ObCY4/BggVw0UVxjk5EREpT2gRn0ZOjhRCmA68DXwOfAw+FEKamKuYSFc7oqcRPRCRtKPHLUpdcAo0awRVXVPACvXrBpZd6mYfXXotrbCIiUroQwtgQQucQwvYhhBsj2+4LIdwXdczNIYQuIYRdQwi3pyzY0qh4u4hI2lHil6UaNoTLLoNx4+DDDyt4kWuugV12gTPPhF9+iWd4IiKSzdTiJyKSdpT4ZbFzz4XWreHyy32cfbnVquVdPhctgvPPj3t8IiKSpebN85mimzRJdSQiIhKhxC+L1anjE7x8/HElemvusYf3F33iCXi5+PwCIiIiJSic0dNKqk4hIiKpoMQvy512Guywg+dumzZV8CJ/+QvsvjucdZbP1CYiIrI1quEnIpJ2lPhluRo14IYbYMoU+Mc/KniRmjW9y+eSJXDeeXGNT0REstC8eUr8RETSjBK/KuC442DoUG+4e/vtCl6kWze/wNNPexIoIiJSkrVrfWy4ZvQUEUkrSvyqADN48EHYeWcYNqxolu1yu+IK6N/fC7v/5z/xDFFERLLFjz/6Wi1+IiJpRYlfFVGvHrzwAvz2GxxzjK/LrUYNeOUV2GsvzyA12YuIiBRX+O2iEj8RkbSixK8K6dzZe2lOmFCJ6gz16sHYsdC9Oxx7LLz+elxjFBGRDKfi7SIiaUmJXxUzeDBceincfz+MHl3BizRoAG+8AV26wFFHwTvvxDFCERHJaHPn+hiDNm1SHYmIiERR4lcF3XADDBgAZ58NX35ZwYs0agRvvQWdOsERR8AHH8Q1RhERyVDz5kHr1j4jtIiIpA0lflVQ9erwzDPQtCkcfTQsW1bBCzVt6tOEbrstHHYYfPppXOMUEZEMpFIOIiJpSYlfFdW8uU/MmZ8Pw4dXorh7ixbe1bNFCxg4ECZOjGucIiKSYVS8XUQkLSnxq8L22gvuuMPnarnhhkpcqE0bePddaNwYDjoIvvoqbjGKiEgG2bTJyzloYhcRkbSjxK+KGzkSTjoJrr0Wxo2rxIW23daTv7p14YADYNq0eIUoIiKZYtEiWL9eLX4iImlIiV8VZwajRsHuu8MJJ8APP1TiYh07evJXvTrk5cGMGXGLU0REMsDcub5W4iciknaU+Al16sDzz3sPnaOPhrVrK3GxTp18zN+mTT516KxZcYtTRETSnGr4iYikLSV+AsD228OTT3p5h3POgRAqcbEuXXy2z7VrPfkr/AZYRESyW2HipxY/EZG0o8RPfnf44XDVVfDoo/DQQ5W82O67e52/X3/15O+nn+ISo4iIpLG5c6F+fWjQINWRiIhIMUr8ZDPXXAMHHwznngsTJlTyYj16wBtvwOLFnvz9/HNcYhQRkTQ1b5538zRLdSQiIlKMEj/ZTE4OPPUUtGrl4/2WLKnkBffay+tF5Of7hC+LF8clThERSUMq3i4ikraU+MkWmjTx4u6LFsGwYbBxYyUv2KcPvPIKzJ4NBx4Iy5bFJU4REUkzKt4uIpK2lPhJiXr2hHvu8TlarrkmDhccMABeegmmT/e+pL/+GoeLiohI2li1yr/Y04yeIiJpSYmflOr00+GMM+DGG+Hll+NwwYMP9qbEyZPhkENg5co4XFRERNLCjz/6Wi1+IiJpSYmfbNVdd8Eee8Dw4fD993G44BFHwLPPwuefQ9++KvIuIpItVLxdRCStKfGTrcrN9Ua66tV9spfVq+Nw0SFDvAnxxx995s8nn4zDRUVEJKVUvF1EJK0p8ZMydegATz8NU6fCyJGVLO5e6NBD4auvPPEbPhxOOy1OWaWIiKTEvHk+NXSrVqmORERESqDET2Jy8MFw/fXeOHf11XFK/tq2hXff9arxo0fDnnvClClxuLCIiCTd3LnQpo13ERERkbSjxE9idsUVcOqpcMMNcNZZUFAQh4tWr+4Z5VtvwS+/QK9ecP/9ccosRUQkaQqLt4uISFpKaOJnZgPN7Dszm2lml5Ww/xIzmxxZpprZRjNrHNk3x8ymRPZNTGScEptq1eDhh+HKK+HBB33M35o1cbp4Xp7P9tmvn/cnHTpUJR9ERDKJireLiKS1hCV+ZpYD3AMcAnQBhplZl+hjQgg3hxC6hRC6AZcD40MI0dW994/s75moOKV8zLzF7+67vSZ7XOuxt2gB48bB3/8Ozz/v4/8mTIjTxUVEJGE2boT8fCV+IiJpLJEtfr2AmSGE2SGE9cAYYNBWjh8GPJPAeCSOzjkHnnsOJk6EPn2KJnOrtGrV4LLL4IMPvC/pvvvCbbep66eISDpbsMB/Z6urp4hI2kpk4tcG+DHqeX5k2xbMrA4wEHg+anMA3jSzSWY2orQXMbMRZjbRzCYuXrw4DmFLrI45Bt58E+bPh332ifO8LL17w5dfwmGHwZ/+5PX/liyJ4wuIiEjcFH77pxY/EZG0lcjEz0rYVlqzzRHAx8W6ee4bQuiBdxU9x8z6lXRiCOGBEELPEELPZs2aVS5iKbf99oMPP/THffvC+PFxvHjjxvDCC3DnnT75S7duRS8mIiLpQ8XbRUTSXiITv3ygXdTztsD8Uo4dSrFuniGE+ZH1IuBFvOuopKHddoNPP/XSTQcf7MPz4sYM/vhHf4HataF/f/jrX308iYiIpAe1+ImIpL1EJn4TgE5m1tHMauLJ3cvFDzKzBsB+wH+jttU1s/qFj4GDgKkJjFUqadtt4aOPfD6WY4+Fe+6J8wv06AFffOGzfV59NRx0kI8pERGR1Js3Dxo1gvr1Ux2JiIiUImGJXwihADgXeAOYDjwXQphmZiPNbGTUoYOBN0MIq6O2tQA+MrOvgM+B10IIrycqVomPJk3g7bd9ON6553rZh7jOyVK/vleQf/hhbwHs2hXeeCOOLyAiIhUyd65a+0RE0lxC6/iFEMaGEDqHELYPIdwY2XZfCOG+qGNGhxCGFjtvdgiha2TZpfBcSX916nhXzzPPhL/9DU4/HTZsiOMLmMFpp/l0oi1awMCBcNFFqvknIpJKKt4uIpL2Epr4SdVUvTrcfz9ccw08+igcdRSsXl3maeXTpQv8739w1lnwr3/B9tvDHXfA+vVxfiERESmTireLiKQ9JX6SEGZw7bVw333w+uswYEACqjHUqeMvMGmSd/u84AJPCP/9b9X9ExFJll9/9UUtfiIiaU2JnyTUWWd518+vv/Za7D/8kIAX6dHDBxeOHeszfx53nNcB/PjjBLyYiIhsRjN6iohkBCV+knBHHeV52eLFno9NnpyAFzGDQw7xiz/0kE800KcPDBkCM2Yk4AVFRARQ4icikiGU+ElS7Luvl3uoUQP69YN3303QC+Xk+Iwy338P11/vhd+7dIFzzoFFixL0oiIiVVhh8XZ19RQRSWtK/CRpunSBTz7xvw0GDvSqDAkbile3Llx1FcycCSNG+GwzO+wAN94Ia9Yk6EVFRKqgefP8W70WLVIdiYiIbIUSP0mqtm3hgw+8F+YZZ3gCOGdOAl+wRQu4916YOtVnmPnLX6BzZ59udOPGBL6wiEgVMW8etGsH1fQnhYhIOtNvaUm6Ro18zN/dd3sL4K67wl13waZNCXzRnXaCl17yrLNNG68F2L27CsCLiFTW3Lnq5ikikgGU+ElKVKvmw+6mToW+feG883w9fXqCX7hvX/jsM3j2WS8uOHAgHHRQgmacERGpAlTDT0QkIyjxk5Rq396rMDz+OHz7LXTrBn/7G2zYkMAXNfOSD99848XfJ03ykhDDhysBFBEpjw0bYP58JX4iIhlAiZ+knJnnXN9846UfrrwS9twTvvgiwS9cq5YXfZ81Cy65BF54wbt/9unjLYIJzT5FRLLATz95P3119RQRSXtK/CRttGjh+daLL8LChdCrF1x2Gaxdm+AXbtgQ/vEP/wPmttvg559h6FD/Q+b66/25iIhsSTX8REQyhhI/STtHHeWtf6ec4vlY164+J0vCNWwIF17oBd9fe837nV5zjf9Bc8IJ8OmnCaw/ISKSgZT4iYhkDCV+kpYaNYKHHvLZPwsKYL/94A9/gBUrkvDi1arBoYf64MMZM3wWmldfhd69vQ/q6NGwbl0SAhGRqsjMBprZd2Y208wuK2F/fzP71cwmR5arUxEnUFS8XYmfiEjaU+InaS0vD6ZM8Ya4++7z0g9jxyYxgE6dfAKYn36CUaM84Tv1VC9IePnlRd92i4jEgZnlAPcAhwBdgGFm1qWEQz8MIXSLLNcnNcho8+ZBs2ZQu3bKQhARkdgo8ZO0V7euD7375BOoXx8OO8wng1myJIlB1KsHI0d6Fvruu9CvH/zzn9CxIwwZAu+9p26gIhIPvYCZIYTZIYT1wBhgUIpjKp1KOYiIZAwlfpIx9t7bZ/q8+moYMwa6dPHJYJKab5nB/vv7DKA//AB//rMPQBwwwJsjR42CVauSGJCIZJk2wI9Rz/Mj24rbx8y+MrNxZrZLckIrgYq3i4hkDCV+klFq1YLrrvMEsH17n3zz0ENhwoQUBLPttvD3v0N+Pjz6KOTm+kDENm18ZpqxY2H9+hQEJiIZzErYVvzrrS+A9iGErsBdwEslXshshJlNNLOJixcvjm+U4N+6qcVPRCRjKPGTjLTbbj7J5q23wuefe+mHQw/1bUmXm+uJ3sSJ3h91yBB46SXvk9qiBZx2Grz+uuoCikgs8oF2Uc/bAvOjDwghrAghrIo8HgvUMLOmxS8UQngghNAzhNCzWbNm8Y/0l19g9WolfiIiGUKJn2Ss6tXhootgzhy46SZv9evdGw48ED78MAUBmcE++3jr36JFPhPokUfC88/DIYd4EnjGGfDmm0oCRaQ0E4BOZtbRzGoCQ4GXow8ws5ZmZpHHvfB7+dKkR1o4o6e6eoqIZAQlfpLx6teHSy/1BPCWW+Drr33ulf33T+GcKzVreovfY495Evjyy/78uefg4IOhVSs480x46y2vVyEiAoQQCoBzgTeA6cBzIYRpZjbSzEZGDjsGmGpmXwF3AkNDSMFvOtXwExHJKEr8JGvUrQt/+pPPuXL77fDddz7nSr9+nl+lbNLNWrXgiCPgiSc8Cfzvf2HgQJ+h5qCDPAk86yx45x0lgSJCCGFsCKFzCGH7EMKNkW33hRDuizy+O4SwSwihawhh7xDCJykJVImfiEhGUeInWadOHTj/fJg9G+6+21sCDzrIu4GOG5fiqgu5ud7988knPQl88UXvm/rUU3DAAdC6NZx9tjdVbtyYwkBFRMowd67/TkvE+EEREYk7JX6StXJz4ZxzYOZML/4+f75PANOrF7zyShqU3atdG446Cp5+GhYv9rGAAwbA44/7unVrLxb/9NOeJIqIpJPCGT2tpIlIRUQk3Sjxk6xXq5b3pPz+e3joIVi61Bvd9tjDG9w2bUp1hHgSOGSId/9cvBj+/W8fpPjyy3DCCT4xTI8ePpjxnXdg3bpURywiVZ1KOYiIZBQlflJl1KwJp5/uY/8efRRWrvRcq1s3z7PSIgEE76t6zDGeBC5a5PUqbrgBttkGbrvNu4Q2buzjBG+9FaZMSYPmSxGpclS8XUQkoyjxkyqnRg0vuzd9us+3sn49HHccdOnis4IuXJjqCKPk5MCee8KVV8L778OyZd5P9cwz/Y+uiy+G3Xf3bqEnneRjB3/+OdVRi0i2++03/12jFj8RkYyhxE+qrOrV4cQTYdo0eOYZaNQILrkE2rSBQYO8BnvaldurXx8OPxzuuMMz13nz4OGHoX9/GDsWhg/3WUK7dvWk8M03Ye3aVEctItkmP9/XavETEckYSvykysvJgaFD4dNP4ZtvvCTE55/D4MGeBF50kfemTEvt2sFpp3nmumgRTJwIf/87NGkCd93lNQMbNfLE8PLLvbVwyZJURy0ima6weLta/EREMoYSP5EoO+8M//gH/PgjvPoq9O3rJSF23917XN57L/zyS6qjLEW1aj5jzWWXwbvverfQsWPhD3+ANWu8H+uRR/rU6507w8knw/33e8V7lY4QkfJQDT8RkYxTPdUBiKSj6tXhsMN8WbLEy+w9+qiXh7joIq/CcOqpPs9KTk6qoy1F3bpwyCG+gCd/kybBJ5948+brr3vpCPAupHvtBfvs48vee3tLoYhISebN8zIObdumOhIREYmREj+RMjRt6gXhzz8fvvzSE8CnnoJnn/W/eU46yZPAHXZIdaRlqFPHmzD79vXnIXiV+08/9eWTT+DGG4umN915Z696X5gM7rSTtyqKiMydCy1ber0cERHJCBayaBr4nj17hokTJ6Y6DKkCfvvNS+w9+ii88YbnSn37egJ47LFQr16qI6ygVatgwoSiVsFPP/UuowANG0KvXl7/ols36N4dOnVK4yZPyWZmNimE0DPVcWSKuN8fDzzQa+J89ln8rikiInFR2j0yoYmfmQ0E7gBygIdCCDcV298f+C/wQ2TTCyGE62M5tyRK/CQVfvrJy0I8+ijMmOENawcf7JNvHnaY117PWCH4D1XYIjhxIkydWjTdae3aPgAyOhncbTd/E0QSSIlf+cT9/ti5s/9/f/bZ+F1TRETiIumJn5nlADOAA4F8YAIwLITwTdQx/YGLQwiHl/fckijxk1QKwfOjJ5/0yTMLZzvv1QuOOMITwa5dfVhMRlu/Hr79FiZP9r6vkyf7sny5769Wzf8oLEwGCxPC5s1TFLBkIyV+5RPX+2MI/qXPH/8IN98cn2uKiEjclHaPTOQYv17AzBDC7EgAY4BBwFaTtzicK5ISZj4krndvuOce+Oornxn0lVfgqqt8advWE8AjjoABAyA3N9VRV0DNmt7Kt/vuPsAR/A/BefOKksAvv/QseMyYovNatSpKBLt29TGEnTtn6JsgUoUtXuz93TWjp4hIRklk4tcG+DHqeT6wVwnH7WNmXwHz8da/aeU4VyQtmRXlOH/5C/z8s1dWePVV7xZ6333eG/KAAzwJPOwwz4sylpkXcm7fHgYNKtr+yy+eAUcnhG+9BQUFvr9aNdhuO08Co5eddoIGDVLxk4hIWQpr+Kl4u4hIRklk4ldSh7bi/Uq/ANqHEFaZ2aHAS0CnGM/1FzEbAYwA2FbfPkqaatnS66yfdhqsWwfvv1/UGvjyy35Mz55FrYHdu2dBl1AoKh7fv3/Rtt9+866i06dvvrzxhncjLdS69ZYJ4c47+6DJrHhzRDKUaviJiGSkRCZ++UC7qOdt8Va934UQVkQ9Hmtm95pZ01jOjTrvAeAB8DEM8QldJHFyc2HgQF/uusvnSnnlFU8Er7sOrr3Wc57DD/cWwX79MnyCmOJq1fKunl27br69oAB++KEoEfzmG18/9pjPHlioYcPNE8EddvBlu+00qYxIMijxExHJSIlM/CYAncysI/ATMBT4v+gDzKwlsDCEEMysF1ANWAosL+tckWxg5pNg7rYbXHEFLFoE48Z5Ivj00/DAA37cjjvCfvv50q9fltZMrl7dy0N06gRHHlm0PQSfOrV4C+Grr8Ijj2x+jVatPAncfntfoh83bpzcn0ckW82d6zVrGjVKdSQiIlIOCUv8QggFZnYu8AZekuGREMI0MxsZ2X8fcAxwtpkVAGuBocGnGS3x3ETFKpIumjeHk0/2ZcMG+OILGD/elzFjihLB7bbbPBHs0CGLez+aeabbtq3XDou2bBnMmlW0zJzp6zffhPnFOgk0alRyQrjDDp4wZu0bKBJn8+Z5a5/+z4iIZBQVcBfJEBs3+jwp48fDBx/4UlhbvV27oiRwv/280azK/022Zg3Mnr15Qli4njvX39BCtWsXTU4TvWy7ra9bt/YWSUkbKudQPnG9P/bsCc2aefcEERFJO6ko5yAicZSTAz16+HLhhbBpE0yb5gng+PHeyPXkk35sy5ZFSeB++/lQuGrVUht/0tWpA7vu6ktxGzZ48hfdUjh3ri+TJsGSJZsfn5PjLY7FE8Lo57VrJ+fnEkm1uXM9+RMRkYyixE8kQ1WrVjQ+8JxzfCjcjBlFXUPHj4fnnvNjGzSAPfbwpWdPXzp2rMKtgjVqFE0KU5LVq70727x5RQlh4fL++z7mcNOmzc9p3twTwHbtoE0bX9q2LXrcpg3UrZvwH00kodas8S9GNLGLiEjGUeInkiXMfBKYHXeEESM8EfzhB08AP/8cJk6EO+4oqpjQqJEngNHJoIbtRNStWzRraEk2bPAxhMWTwrlzvVTFO+/AihVbntegweaJYElL8+ZVsHlWMsaPkRK7SvxERDKOEj+RLGXmk8Bstx2ceqpvW7/ey0dMnFi03HJLUT31pk03TwR79vRcRMlgMTVqFHXzLM2qVd4yWNryzTewYMGWLYfVq/tkM23aeJ/d0pYWLbw2iEgyqXi7iEjGUuInUoXUrFk0TnDECN+2bh1MmbJ5MnjTTUVznzRvXpQEdu8Ou+ziyWROTup+joxQr15RE2xpNm6EhQtLTgznz/fxhx9/DIsXl3x+w4ZbTw4LE8SmTTU5jcSHaviJiGQs/SUgUsXl5sKee/pSaO1an0E0Ohl8/fWixqncXO8FucsuPnfKLrv40r69eimWS06OzxjauvXm/wDFbdjgyd/PP5e+TJrk6+hi99EaN/YsvlmzLdfFtzVposxeSjZvnv8nb9061ZGIiEg5KfETkS3Urg177+1LodWrvZvotGm+TJ0K771XNJMo+NC4Ll2KEsHCpLBtW3UXrZQaNYoSxLKsXu2tiNFJ4eLFsGhR0Xr6dB/8uXSpDwYtzsyTv+JJYdOmvr1wiX6+zTb6R64K5s71bsg1aqQ6EhERKSclfiISk7p1Ya+9fIm2fHlRMliYEI4bB6NHFx2zzTZFyeAuu3hy2LmzT4CphqU4q1u3aHBnWTZu9OSveGK4ePHmj6dO9ce//FJyogjelbS0pLD48+23926oknkKi7eLiEjGUeInIpXSsCHsu68v0ZYuLUoEC5PCF1+Ehx4qOqZmTc8BdtjBi85HL23bqttowuXkeGte8+aekZdl40bP9Jcu9WXJkqLHxZ/PmFH0eMOGza9z3XVw9dUJ+ZEkwebN2/LbHxERyQhK/EQkIZo08SLy/foVbQuhqKfh999vvrz1lk80Uyg315PCwkQwOjls3VpJYUrk5BS12sUqBJ/hNDop7NgxcTFK4mza5OUcjjsu1ZGIiEgFKPETkaQx80kmW7SA/v0337dpE+Tnw8yZmyeE330HY8cW1R8EH4NYmAh26FC0tG/v6222SdqPJGUxg/r1fVHCl9l+/tlbb9XVU0QkIynxE5G0UK2a/z257bYwYMDm+zZu9IaG77/fPDH85htPCqNbCsG7nxZPBgvXHTr4fs1DIlJOhaUcVMNPRCQjKfETkbSXk1OUtB144Ob7CruPzp0Lc+b4Uvi4sAvp6tWbn1O//pbJYNu2PllhmzbelbRWrcT/XCIZpbB4u1r8REQykhI/Eclo0d1He/Xacn8IPqysMBksniCOH19y6bumTYuSwMKEsPjSpIlaDqUKUfF2EZGMpsRPRLKamSdxTZvCHntsuT8En6jyp59KXyZN8lbF4mrV2jIxbN3aKxVEL40bK0GULDBvnveT1iBaEZGMpMRPRKo0M2jUyJdddy39uPXrfW6LrSWHL78Ma9dueW6NGlsmgy1bQqtWW26rXTtxP6tIpcydq9Y+EZEMpsRPRCQGNWsWTT5TmhC82+iCBZ4kFi7Rz+fNg88/9xbEkmqhN2jgCWDz5tCs2daXpk09LpGkUPF2EZGMpsRPRCROzLwX3DbbwI47bv3YggJYvLj0BHHhQq93+MEHPkaxpCQRPFHcWmLYrFlR6b3Gjf141UCUCpk3D/r0SXUUIiJSQUr8RERSoHp17+rZqlXZx27cCMuWeaJYfFmypOjxnDkwYYI/Ligo+VrVqnkCWJgIFiaFZT2vUyeuP75kmpUr4Zdf1OInIpLBlPiJiKS5nJyiVrxYhAC//lqUGC5d6suyZUWPC5/n58NXX/nj4mUvouXm+rweheMhS3pc2v769TW5TcbTjJ4iIhlPiZ+ISJYx88SrYUPo1Cn289atK0oOS0oSf/nFl+XLvTvq9On+ePny0ruigrcyFsbTqBGccw6cemolfkBJPhVvFxHJeEr8REQE8Fa91q19KY9Nm2DFCk8Ao5PD0h7XqhX/2CXBcnNhwADo0CHVkYiISAUp8RMRkUqJbtFTXpCl9t/fFxERyVia201ERERERCTLKfETERERERHJckr8REREREREspwSPxERERERkSynxE9ERERERCTLKfETERERERHJckr8REREREREspwSPxERERERkSynxE9ERERERCTLKfETERFJI2Y20My+M7OZZnbZVo7b08w2mtkxyYxPREQykxI/ERGRNGFmOcA9wCFAF2CYmXUp5bh/AG8kN0IREclUSvxERETSRy9gZghhdghhPTAGGFTCcX8EngcWJTM4ERHJXEr8RERE0kcb4Meo5/mRbb8zszbAYOC+JMYlIiIZrnqqA4inSZMmLTGzuZW8TFNgSTziSaJMjBkyM+5MjBkyM+5MjBkyM+5MjLl9qgNIECthWyj2/Hbg0hDCRrOSDo9cyGwEMCLydJWZfVfJ2DLxcwKZGXcmxgyZGXcmxgyZGXcmxgyZGXeJ98isSvxCCM0qew0zmxhC6BmPeJIlE2OGzIw7E2OGzIw7E2OGzIw7E2POYvlAu6jnbYH5xY7pCYyJJH1NgUPNrCCE8FL0QSGEB4AH4hVYpn5OMjHuTIwZMjPuTIwZMjPuTIwZMjfukmRV4iciIpLhJgCdzKwj8BMwFPi/6ANCCB0LH5vZaODV4kmfiIhIcUr8RERE0kQIocDMzsVn68wBHgkhTDOzkZH9GtcnIiIVosRvS3HrFpNEmRgzZGbcmRgzZGbcmRgzZGbcmRhz1gohjAXGFttWYsIXQjglGTFFZOrnJBPjzsSYITPjzsSYITPjzsSYIXPj3oKFUHzMuIiIiIiIiGQTlXMQERERERHJclUy8TOzgWb2nZnNNLPLSthvZnZnZP/XZtYjFXEWi6mdmb1nZtPNbJqZnV/CMf3N7FczmxxZrk5FrMWZ2RwzmxKJaWIJ+9Pq/TazHaPew8lmtsLMLih2TFq812b2iJktMrOpUdsam9lbZvZ9ZN2olHO3+v8gyTHfbGbfRv79XzSzhqWcu9XPUiKVEve1ZvZT1Ofg0FLOTaf3+tmoeOeY2eRSzk3Zey2ppXtk8mTa/TESU0bcIzPx/hh57Yy7R2bi/THy2lXvHhlCqFILPlh+FrAdUBP4CuhS7JhDgXF4PaW9gf+lQdytgB6Rx/WBGSXE3R+f3S3l73OxuOYATbeyP+3e72Kfl5+B9un4XgP9gB7A1Kht/wQuizy+DPhHKT/XVv8fJDnmg4Dqkcf/KCnmWD5LKYj7WuDiGD5DafNeF9t/K3B1ur3XWlK36B6Z9Lgz9v4Y9XlJy3tkJt4ftxJ3Wt8jM/H+WFrcxfZn3T2yKrb49QJmhhBmhxDWA2OAQcWOGQQ8HtxnQEMza5XsQKOFEBaEEL6IPF4JTAfapDKmOEq79ztKHjArhDA31YGUJITwAbCs2OZBwGORx48BR5Vwaiz/DxKipJhDCG+GEAoiTz/Da5ellVLe61ik1XtdyMwMOA54JhmxSMbQPTK9pN17XUza3iMz8f4ImXmPzMT7I1TNe2RVTPzaAD9GPc9ny5tDLMekjJl1ALoD/yth9z5m9pWZjTOzXZIbWakC8KaZTTKzESXsT+f3eyil/6dPx/caoEUIYQH4H0NA8xKOSef3/DT8G+6SlPVZSoVzI91vHiml21C6vtd9gYUhhO9L2Z+O77Uknu6RyZXJ90fIvHtkpt8fIbPukZl6f4QsvUdWxcTPSthWfGrTWI5JCTOrBzwPXBBCWFFs9xd4d4uuwF3AS0kOrzT7hhB6AIcA55hZv2L70/L9NrOawJHAv0vYna7vdazS9T2/EigAnirlkLI+S8k2Ctge6AYswLuFFJeW7zUwjK1/k5lu77Ukh+6RyZWR90fI6ntkOr/nmXSPzOT7I2TpPbIqJn75QLuo522B+RU4JunMrAZ+Q3sqhPBC8f0hhBUhhFWRx2OBGmbWNMlhbiGEMD+yXgS8iDftR0vL9xv/z/xFCGFh8R3p+l5HLCzsChRZLyrhmLR7z83sZOBw4IQQQom/+GP4LCVVCGFhCGFjCGET8GAp8aTje10dGAI8W9ox6fZeS9LoHplEGXx/hMy8R2bk/REy7x6ZqfdHyO57ZFVM/CYAncysY+TbqqHAy8WOeRk4ydzewK+FXQNSJdLX+GFgegjhtlKOaRk5DjPrhf/7Lk1elCXGVNfM6hc+xgcoTy12WNq93xGlftuTju91lJeBkyOPTwb+W8Ixsfw/SBozGwhcChwZQlhTyjGxfJaSqthYm8GUHE9avdcRBwDfhhDyS9qZju+1JI3ukUmS4fdHyMx7ZMbdHyEz75EZfH+EbL5HljbrSzYv+CxZM/CZhK6MbBsJjIw8NuCeyP4pQM80iLkP3vz9NTA5shxaLO5zgWn4rEifAb3TIO7tIvF8FYktU97vOvhNqkHUtrR7r/Gb7gJgA/7N2elAE+Ad4PvIunHk2NbA2Khzt/h/kMKYZ+L9/As/2/cVj7m0z1KK434i8pn9Gr9ZtUr39zqyfXThZznq2LR5r7Wkdinp85oBv7Mz7h5Z2v+zdH+vI3Gl/T2ylN/ZaX1/3ErcaX2PLCXmtL4/lhZ3ZPtosvQeaZEfQERERERERLJUVezqKSIiIiIiUqUo8RMREREREclySvxERERERESynBI/ERERERGRLKfET0REREREJMsp8RNJMjPbaGaTo5bL4njtDmYWcy2ZSC2atyKPP4oULRUREUk63R9FEksfYpHkWxtC6JbqICL2AT4zs0bA6hBCQaoDEhGRKkv3R5EEUoufSJowszlm9g8z+zyy7BDZ3t7M3jGzryPrbSPbW5jZi2b2VWTpHblUjpk9aGbTzOxNM6tdwmttb2aTgSeB/wMmAV0j37A2T85PLCIiUjbdH0XiQ4mfSPLVLtaV5fiofStCCL2Au4HbI9vuBh4PIewOPAXcGdl+JzA+hNAV6AFMi2zvBNwTQtgFWA4cXTyAEMKsyLeqk4BewOPA6SGEbiGERfH7UUVERGKm+6NIAlkIIdUxiFQpZrYqhFCvhO1zgAEhhNlmVgP4OYTQxMyWAK1CCBsi2xeEEJqa2WKgbQjht6hrdADeCiF0ijy/FKgRQrihlFgmhBD2NLPngfNCCD/F++cVERGJhe6PIomlFj+R9BJKeVzaMSX5LerxRkoYy2tm90UGuXeKdGkZCLxmZheWI1YREZFk0f1RpJKU+Imkl+Oj1p9GHn8CDI08PgH4KPL4HeBsADPLMbNtYn2REMJI4Drgr8BRwGuRbiz/qlT0IiIiiaH7o0glaVZPkeSrHfkWsdDrIYTCKatrmdn/8C9lhkW2nQc8YmaXAIuBUyPbzwceMLPT8W8uzwYWlCOO/fCxC32B8RX5QUREROJI90eRBNIYP5E0ERnD0DOEsCTVsYiIiKQL3R9F4kNdPUVERERERLKcWvxERERERESynFr8REREREREspwSPxERERERkSynxE9ERERERCTLKfETERERERHJckr8REREREREspwSPxERERERkSz3/6lGS5aeRTpTAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1080x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(15, 6))\n",
    "\n",
    "plt.subplot(121)\n",
    "plt.plot(history.history['loss'], '-r', label=\"Training\")\n",
    "plt.plot(history.history['val_loss'], '-b', label=\"Validation\")\n",
    "plt.xlabel('Epoch #')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.plot(history.history['accuracy'], '-r', label=\"Training\")\n",
    "plt.plot(history.history['val_accuracy'], '-b', label=\"Validation\")\n",
    "plt.xlabel('Epoch #')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Q:** Did overfitting occur during learning? Why? Looking at the curves, does it make sense to continue learning for much more epochs?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A:**  No, the training accuracy is always below the validation accuracy. The model is too small to overfit. The accuracy has started saturating, it will not get much better (perhaps one percent or two) or very slowly (you can let it learn for 500 epochs or more to see it)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following cell makes predictions on the test set (`model.predict(X_test)`), computes the predicted classes by looking at the maximum probability for each example and displays some misclassified examples. The title of each subplot denotes the predicted class and the ground truth.\n",
    "\n",
    "**Q:** Are some mistakes understandable?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-03-20 18:11:34.792866: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj8AAAHOCAYAAAB3imgWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA4JElEQVR4nO3dedyXU/7H8c8nJdpkazS0WKYxRCOVoRIzkSVE9miMNLaiwfRDZUsNoaGMLJkMY80SMpGtSOg3IbKESWnXvmiv8/vje5tf55zrvr/b9V3P6/l43I+Z9+Wc6zrV6bo/Xd9zn0uNMQIAABCKaoUeAAAAQD5R/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKAUTfGjqrNUdZ2qrlHVRao6SlXr5OA6V6rqd6r6o6p+qarNMjjHuIpxrlHVTaq6cZt8f4xjvUBVt2xz7jWqelRc5y83zKGk13xLVY2qVs/F+ctFPuaRqr6tqotVdZWqTlPVUzI8T77uRc1V9TVVXaKqbA6Xgnzdjyqu1aHi7/atGfbP1zyqqap/VdX5qrpcVe9T1RpxnT8dRVP8VDjJGFNHRFqKSGsR6R/nyVX1IhHpISInikgdEeksIkvSPY8x5nhjTJ2KsT4uIkN+ysaYS+Ics4i8v8256xhjJsR8/nLDHIqgqt1EhKIndTmdRyJypYg0NMbUE5E/isg/VbVhuifJ4zzaJCLPSGLuI3W5nkdSUTzcIyIfZnqOPM6ja0WklYg0F5Fmkvh9if33JBXFVvyIiIgxZp6IjJPEb1AsVLWaiNwoIn8yxnxhEv5jjFlWSft2qroirusjv5hD1jh2ksS4+xZ6LKUmF/Oo4ryfGmM2/xRFpIaINIpqWwzzyBgzwxjzsIh8XshxlKpczaMKV4vIeBH5qqpGxTCPROQkERlmjFlmjFksIsNE5MJCDKQoix9VbSQiJ4jIx5X897GquqKSr7GVnHaviq/mqjqn4mOLmyu+oXmMMZOMMfVj+vW0q2K8K1S1XRXdD6l41Py1qg7gI4vUMIcsg0VkhIgsjGMsIcnRPNq273pJ/It9goj8O6pdEc0jZChX80hVm0iieLgl2RiKZB5pxde2ea+Kf6DlVbF9Ix2jqptFZKWIvCKJm7bHGNM5g3PvVfG/x4rIQSJSXxLV8lwReSiD86XMGDOp4nrpekcS/1KYLSIHisjTIrJZRP4S2+DKD3NoG6raSkTaSuJjlr2SNMf/y+U8+m/fio8sOorI/saYrZmeK41rZnovQmZyPY+GicgAY8waVU3aOC5ZzKNxInKlqr4tItuJyBUVx2tJ4vcob4qt+OlijHkjR+deV/G/Q4wxK0Rkhao+IIlqPKffuDJljJm5TfxMVW8RkT8LxU9VmEMVKp5I3SciVxpjNufz5lgGcjmP/ssYs0lExmliEf1/jDEv5fqayKuczSNVPUlE6hpjns7F+XNkkCSKpk9EZIMk7puHiMgP+R5IUX7slYzaK9Pdr3GVdJshIhsl8fl6Xqlq+yrGu0ZV26d4KiP2I0NkKJA5VE8SiwufVtWFIvK/FcfnpjHnUIUM51GU6iKyb67G+ZMY70WIUYbz6Hci0kpVF1b8/T5LRPqo6ot5GG9G88gYs84Y08sYs6cxZh8RWSoiU40xW3I9ZlexPflJiTHm+Az6rFXVp0Wkr6p+LCI7iUhPEbkjqr0mfqT8bWNM1sWGMeZdSfxkUFpU9XgR+cgYs0hV9xeRASIyOtvxIJg5tFJEfr5NbiQiU0TkUBFZnO2YkNk8qvi7vLck1vlslsQ3rSOlkgXpRTCPRBOPDWuKyPYVeYfE6cyGbMeEzOaRJL4f3LZNvkdE5ovIwKjGRTKP9pTEPx4XiMhhkvg1FOQnCEuy+MlCLxF5UBITZIUkHrn9vZK2jUTk/fwMq1K/E5FHNLE3xCIR+adU8pkx8qZk5pAxxsg2i5wrvmGJiCza5ieNkH8qIjeJyAEiskVEvhGRs4wxH1XSvhjuRU1E5Ltt8jpJrEVsWpDRQIwxq0Vk9U9ZVdeJyI+V/fSpFMc82ldEHhWRBiIyR0SuNcaML8RANHF/hEtVR4rIaGPMa4UeC0oTcwhxYB4hDswjG8UPAAAISkkueAYAAMgUxQ8AAAgKxQ8AAAgKxQ8AAAhKWj/qrqqsjg5AHPtAVIY5FIwlxpjdc3Vy5lEYuBchBpH3Ip78AMiF2YUeAABIJfciih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABCU6oUeQDnaeeedrdy4ceO0zzF79mzv2J/+9CcrT58+3Wvz9ddfW3natGlpXxv/r379+lZetWqVlbdu3ZrH0aBUjRgxwsp//OMfrfzMM894fS688EIrr1u3Lv6BAYHiyQ8AAAgKxQ8AAAgKxQ8AAAgKxQ8AAAiKGmNSb6yaeuMydeKJJ3rHTj75ZCsfddRRVt5vv/3Svo67cFlEpEmTJlauWbNm0vNst912aV/bGKNpd0pRqc2h119/3co//vijlUeOHOn1GTt2bE7HFKcGDRp4x5YtW2blzZs3Z3LqqcaYVpmNKrlinkcNGzb0jr333ntWTuWHII455hgrv/3229kNrARxL0IMIu9FPPkBAABBofgBAABBofgBAABBCXqTw3333dc7dvnll1u5Z8+eVt5xxx29PqrxfyzdrFmz2M+J9H300UdW/vOf/2zliRMn5nM4sevTp493rEaNGlZ2f82o2oIFC7xjCxcutHIqa36uv/56K//73/+28urVqzMYHUrV0KFDrez+3XXvVSIis2bNsvKee+5pZXctmojIxx9/bOUJEyZYOWp+l+Jmrzz5AQAAQaH4AQAAQaH4AQAAQQl6zc9ee+3lHbvyyisLMBKRr776ysqff/55QcYB25w5cwo9hFi5e8dcddVVXpvtt9/eyqz5yd7kyZOt3KZNm6R9jj76aCt37tzZyk8++WT2A0NR6tixo3esS5cuVu7atauV169fn7RP3bp1rfyHP/zB6+Mec/t88MEHXp9u3bpZuRTumzz5AQAAQaH4AQAAQaH4AQAAQaH4AQAAQSnZBc+77bablaMWKrsbOL366qtW3rBhg9dn5cqVVnZfZFm7dm2vz/jx4608ffp0K3/44YdeH3cjqXXr1lV5XRTGpZdeWughxMp96a67uFkkerM0ZMe9R/Tq1cvK1asnvxW3bt3ayix4Ll8XXnihd2zevHlWfuGFF5KeZ9y4cVmP5eqrr7byzJkzvTbuy5BLAU9+AABAUCh+AABAUCh+AABAUEpizU8q62xatGjhtTn11FOrPG/UZk0tW7a0svtiuKgXEs6dO9fKpfiSN4g0b97cO+a+CLDUuZscRrnlllvyMJKwuPcrdx1g27Ztk57jrLPOsvIDDzzgtZkxY0YGo0Oxifp+NmXKlAKMROSuu+4qyHVzjSc/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKEW54NndeO2JJ57w2rgLwgYPHuy1eeONN9K+trvA2fX999+nfU6UhsMPP9w7Vq9evSr7FPNmlDVr1vSO1ahRw8ru5poiIu+8807OxoSEW2+91cqpbEb3s5/9zMrPPvus1+aggw7KbmAoCPfvatTfXXfzXGSHJz8AACAoFD8AACAoFD8AACAoRbHmp06dOla+7rrrrNy5c2evz5IlS6x85513em3Wrl0bw+hQrtx5577AL4r7MsEHH3ww1jHF6ZRTTvGOuWvlHnroIa/NihUrcjUkVHBfurx8+XKvzc4771zlOXbaaSfvmLtGbdWqVRmMDvnmbqb685//3GsT9eeNzPHkBwAABIXiBwAABIXiBwAABKUo1vx06dLFytdee62Vo/bWad++vZVXrlwZ+7hQ3v76179auVmzZkn7lNJLPy+88MJCDwGVcNcjDh061GszcODAKs8R9dLdDh06WPnll1/OYHTIt5kzZ1r5pZde8tpcc801Vh4+fLiVFy9eHMtYhgwZYuXXX3+9ylyqePIDAACCQvEDAACCQvEDAACCQvEDAACCUhQLno844ogq//vHH3/sHZs7d26uhoMydPLJJ3vHzjjjjKT93BfdzpgxI64hxc7dBK1BgwYFGgnSddddd3nHTjjhBCtHvXjX1b9/fytPnjzZykuXLs1gdMi3b775xju2ww47WNm9pz388MOxXPvQQw+1sqpamQXPAAAAJYjiBwAABIXiBwAABKUo1vycfvrpVf734447zjt24403WvnFF1/02nzyySdZjQuly33B44ABA5K2iXLaaadZef369dkNLIf23ntvK//6179O2ieudQLIzsaNG71jGzZssLK79qJaNf/frq1atbJyw4YNrcyan9Lw0UcfJW3jrgHKRNS6x/3339/KI0eOzPo6xYgnPwAAICgUPwAAICgUPwAAIChFseZn9913t/LWrVutXLNmTa/PDTfcYGV3fwsRkfvvv9/KH3zwgZUbN27s9fn222+t/Pnnn0eM2HbggQda+f3337cyexLln7vHjbt3RZQXXnjBO/bZZ5/FNqZixBqQ4uW+0NkYY2X3PhnV5qSTTrLy9OnTYxodcilqDet7771nZfcF4I888ojX58cff6zyOsccc4x3zF0n9t1331V5jlLFkx8AABAUih8AABAUih8AABAUih8AABCUoljwfOedd1r5qquuSvscURt+XXbZZVXmXFm8eLGVJ0yY4LU5++yz8zKWUHTq1MnKgwYNStrHXdzeq1cvr82WLVus7G40V7t27VSH+F+bNm3yjtWoUaPKPlELF93FralwF3XPnDkz7XMgP8aMGWPl7t27p30O9+/FHXfc4bXZvHlz2udFbkUtZh89erSV7777biv37dvX63PTTTdZ2d0YsXXr1l4f9x5XrnjyAwAAgkLxAwAAgkLxAwAAglIUa37czZqefvppKz/xxBNen+rV7aE3atTIaxO1Digf3E0bo17c6m7KeOutt+Z0TOXulFNOsXLLli2T9nE3z7z++uuT9nHn3cUXX5y0j/sZetRLCw855JAqz9GtWzfv2NixY6187LHHJh3LihUrrJzJuiHkx7/+9S8rf/nll1Y+4IADkp6jXbt2Vu7Tp4/Xxl1zieI0bNgwK3ft2tXKUS9vbtOmjZXdtYWzZs3y+rRo0SLDEZYWnvwAAICgUPwAAICgUPwAAICgFMWaH3cvlX//+99WbtasWdJz/O53v/OOuZ9vunseRO1xkAtR+yak8qJNpM79/DsV7jqxyy+/PK7hWNx1NXXr1vXazJkzx8qrVq2y8uOPP+71mTp1qpXdtWZRRowYkbQNioO7/87IkSOtPHTo0LTP2blzZ+8Ya35KU5cuXazco0cPr02TJk2sPHDgQCtHvRA8FDz5AQAAQaH4AQAAQaH4AQAAQaH4AQAAQSmKBc9xePPNN5O2+fWvf23lqAXP7iLDUaNGWfmhhx7y+rgbh5177rlJx4J43XLLLVYePnx40j7ff/+9lRctWuS1WbNmTZXneOutt7xj7oJ915QpU7xjtWrVsvKSJUusfOSRR3p9evfubWV3Ef20adO8Pt98802VY0PxmjFjRtbnOPjgg71j7qLY2bNnZ30d5N7y5cutnMnC9ah7RCh48gMAAIJC8QMAAIJC8QMAAIJSNmt+UjF+/HgrDxo0yGvjvriyZ8+eVt5vv/28PkcddVTaY5k7d27afVC5Bx54wMoffvhh0j4LFy608rJly7w2a9euzW5gKXI/v3e9/vrr3rGTTz65yj7vv/++d2zlypXpDQxF49VXX7Xy//7v/3ptWrVqVeU56tWr5x1z145dc801GYwOKC08+QEAAEGh+AEAAEGh+AEAAEEJas3Pl19+aeVnnnnGa3PmmWdWeY6jjz466XXcF7W+8sorXptrr7026XmQOnd/pmR77ZQDd2+ju+++28oLFizI42iQb1H3lWRrfqIcfvjhcQwHJahjx46FHkLB8OQHAAAEheIHAAAEheIHAAAEheIHAAAEJagFz+vWrbOy+0JSEZE6depY2V1A2KBBA6/PrFmzrPzYY49Z+aabbkp9kECKFi9eXGVGeXv88ce9YzfeeGPa53nhhRfiGA5KkPu9S0Tkhx9+sPInn3ySn8HkGU9+AABAUCh+AABAUCh+AABAUIJa8+NatGiRd+ykk06y8vnnn2/l3/zmN16fm2++2cruZ6YAELc5c+Z4x6666ior9+vXz8q1atXy+rz11lvxDgwl49tvv/WO7brrrlZu3ry5lctlA1me/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKCoMSb1xqqpN0bJMsZors7NHArGVGNM+q8YTxHzKAzci3KrR48e3rHhw4dbuWXLllb+6quvcjqmHIi8F/HkBwAABIXiBwAABIXiBwAABCXoTQ4BAAjVnnvu6R1bvny5lUtwjU9KePIDAACCQvEDAACCQvEDAACCwj4/8LC3BmLAPj/IGvcixIB9fgAAACh+AABAUCh+AABAUCh+AABAUCh+AABAUCh+AABAUCh+AABAUCh+AABAUNJ9sekSEZmdi4GgaDTJ8fmZQ2FgHiFbzCHEIXIepbXDMwAAQKnjYy8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABAUih8AABCUoil+VHWWqq5T1TWqukhVR6lqnRjP30BVn1TV+aq6UlXfU9XDMjzXuIpxrlHVTaq6cZt8f4xjPltVZ1SM9wdV/Yeq1ovr/OWGOVTptfZR1bGqulpVl6jqkDjPX27yMI8ab/Nn/dOXUdWrMzhXvu5Fv1fVqaq6SlXnquoQVa0e1/nLUa7nUcU1jlDVKRV/tz9V1XYZnidf8+h+Z95vUNXVcZ0/LcaYovgSkVki0rHi/+8pItNF5LYYz7+PiFwlIg1FZDsR+aOILBGROlme9xERuTVHvyeNRGS3iv9fR0QeF5Fhhf6zKtYv5lDkubcXkf9UjLu2iOwgIgcX+s+qmL9yPY8irre3iGwRkaZZnieX8+hSEWlfMZ/2FJGpInJtof+sivkrD/ejXSruP2dU3I/OE5HlIrJzsc6jSq7190L8+RTNk59tGWPmicg4EWke4zlnGmOGGmMWGGO2GGMelMRf5F9GtVfVdqq6Iq7rZ8IYM8cYs2SbQ1tEZL9CjaeUMIf+6wIRmV8x7h+NMeuNMZ8WeEwlIxfzKEJ3EXnHGDMr6j8WwzwyxowwxrxrjNlY8XvyuIi0LeSYSkmO5tERIrLIGDO64n70TxFZLCKnRTUuhnm0LVWtLSJdReQfhbh+URY/qtpIRE4QkY8r+e9jVXVFJV9jU7zGryXxjevbqP9ujJlkjKmf2a/Au1a7Ksa7oqpHlRV9V4rIaklMlLvjGFO5Yw79129EZFbFY+0lqjpBVQ+KY0whyMc8kkTxU+k3gCKZR64jReTzOMYUghzNI634co9FFlhFOI+6SqJYeyeOMaWr2D6zHaOqm0VkpYi8IiKDoxoZYzpncxFNrJt5TERuNsaszOZcqTDGTBKR+ln03UlV9xSRnpJ4lIrKMYdse4nI0SJysoi8KSJXisiLqrq/MWZjfCMsO/maR+1F5Gci8mw250lVNvein6jqH0SklYhcFMeYylwu59FkEfm5qp4jiflzrojsKyK1MhxryuKYRyLyexF51FR8/pVvxVb8dDHGvJHLC6jqjiLysoh8YIz5Sy6vFSdjzDxVfVVEnhKRloUeTxFjDtnWicgkY8w4ERFVvVNE+ovIr0RkWiEHVuRyPo8q/F5EnjPGrMnDtbKmql1E5DZJrGVZkqQ5cjiPjDFLVfUUEblTRP4mIq+JyBsiMjcX14tTxZOwDpL4B31BFOXHXsk4K9Pdr3FV9KspImNEZJ6IXJzH8bavYrxrKv71l4rqkqjskaWA5tCnIlKQf1mFINN5VNF3R0ksVs3bmods7kWqepyIPCQiJxljPsvXmEOQ6Twyxkw0xrQ2xuwiIudLYv3hlDyMN9vvad1FZLIxZmaux1qZYnvykxJjzPHp9lHVGpJ4NLhORLobY7YmaX+UiLxtjHE/U02bMeZdSfy0VlpUtZuIvCsic0SksYgMksRHF8hSKHNIRP4pIlerakcReVtErpDET4h8me2YkNk82sapIrJCEn8ulSqGeaSqv5XEIudTjTE5/+YamkznkaoeIomfIttRRG4RkbnGmNcqaXuUFP5+9JPuInJ7tuPIRkk++cnQESLSWUSOFZEVKVSojUTk/byNLtoBkvhcd42IvCciM6SAjwlRenPIGDNDEj8Ce78kfgz2FBE5mfU+RSHVNQ8Fn0ciMkBEdhKRf6X6ZAt50VcS/5iZI4ktOE6tom0xzCNR1cMlsRZxdEHHUaC1RkVPVUeKyOjKqmggGeYQ4sA8QhyYRzaKHwAAEJSQPvYCAACg+AEAAGGh+AEAAEGh+AEAAEFJa58fVWV1dADi2AeiMsyhYCwxxuyeq5Mzj8LAvQgxiLwX8eQHQC7MLvQAAEAquRdR/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKCktc8PELpq1ex/L/To0cNrc/DBB1u5d+/eOR0TACA9PPkBAABBofgBAABBofgBAABBCWrNz2GHHWbl22+/3WszYsQIK48dO9bKP/74Y/wDQ8lo1qyZld35IiLy8ssv52s4AIAM8OQHAAAEheIHAAAEheIHAAAEJag1P1dddZWV27Vr57Vp3769lZ977jkr33zzzV6fzz//PIbRoRS48yHK9OnT8zASAECmePIDAACCQvEDAACCQvEDAACCQvEDAACCEtSC58mTJ1u5S5cuXpvq1e3fktNOO83KHTp08Pr079/fyqNGjbLy5s2b0xkmCqRGjRresaFDh1r5F7/4hZWHDx/u9bnppptiHRcAIF48+QEAAEGh+AEAAEGh+AEAAEEJas3PPffcY2V3fY+ISJ8+fazcsGFDK++6665eH/fllu66EPe6IiLz5s2rcqzIvxNPPNE7dumll1q5Z8+eVnbXdwEAih9PfgAAQFAofgAAQFAofgAAQFCCWvPjuuuuu7xjc+fOtbK75mfffff1+rjrQtwXqNapU8frc9lll6U8TuTHGWec4R2bPXu2lVN5sWm+7LbbblY2xlh56dKl+RwOKkStJWzatKmVzzvvPCtH3SOSef75571jX3zxhZVXrFiR9nlReP369fOOHXrooVYeNGiQlb/55huvz6pVq6y8ww47WPnYY4/1+vz973+3cseOHa38ySef+AMuQTz5AQAAQaH4AQAAQaH4AQAAQaH4AQAAQVF3kWSVjVVTb1ymdtxxR+/Y9ddfb+VrrrnGyu4iMxGRG264wcoDBw6MYXTxMMZors5dTHPo+OOPt/IzzzzjtXEXHg4bNiynY/pJrVq1rPznP//Za9O7d+8qzzFx4kTvWNeuXbMbWOqmGmNa5erkhZpHLVq08I516tTJyp07d/batG3bNmdj2tbXX39t5euuu87KY8aMycs44hLKvci1ZcsW71iy79Vffvmld2zx4sVWrl27tpXdRdRRnnzySSuff/75SfsUmch7EU9+AABAUCh+AABAUCh+AABAUFjzkwPu5+onnXSS12by5MlWdjeS2rBhQ+zjSlUon7O7L6Q9+OCDvTb5WqvRrl07K7tzqH79+mmfc/Pmzd4xd55NmjQp7fOmqCzW/Pzxj3+08gUXXOC1Oeyww6y8cOFCr82//vUvK7sb1K1ZsybpWBo0aGDl0047zWtz4403WtmdA6NHj/b6dO/ePem1CyWUe5ErkzU/qVC1fztTOeemTZusHLXuzV1rVmRY8wMAAEDxAwAAgkLxAwAAgkLxAwAAghL0W91z5ZxzzrFy1GLGww8/3Mr16tWzsrs5FeLXrVs3K7/44ot5ua670ZiIyB133GFld4Hzs88+6/V54IEHrOxuwNisWTOvT9Qmnfh/HTp0sPLgwYOtXLNmTa9Pr169rPzwww97bTZu3Jj12JYsWWJl9w3uIiJLly618r333mvlU0891evjbnQ3derUTIeIDLkbZaaiZ8+eVna/p4iItG/f3spR94RkatSoYeXq1cujbODJDwAACArFDwAACArFDwAACEp5fHhXZM4+++xCDwGO5s2be8dWrlxp5UsvvTQvY3niiSe8Y61bt7ay+5LV8847z+uzdetWK69evdrK69ev9/q8/vrrKY8zRC+//LKV3fVZN998s9fH3SyzkEaOHGlldwPDNm3aeH123333nI4Jye2zzz5p9xk7dqyVR40a5bXZZZddrLzHHnskPe8777xj5Uw2WC0FPPkBAABBofgBAABBofgBAABBYc1PmqL2Sbn++uutfPXVVyc9z/jx4628fPny7AaGKvXt29c7tnbtWiun8nLJTLh7x5x88slemylTpli5d+/eVnbX90SZNm2alaPWOaFqdevWtXKydVXFxn0JZSFfkIzUVatWrcoskto9wLVs2bIqcxT3paruy1DddUSliic/AAAgKBQ/AAAgKBQ/AAAgKBQ/AAAgKGWz4Nl9qaOIyOWXX27l+fPnW3nMmDFenxUrVljZ3SguarFqq1atqhxb1AsI3ZfSbd68ucpzIDstWrTwjr3xxht5uXb//v2tHLVw8dZbb7Wy+4LKVLi/xny9qLWcXHLJJVaeMWOGlT/77LN8Didt++67r5Xdl5a690ARkUmTJuV0TEjOvSdE3SOMMXkZi3sdN5911llen1KcQzz5AQAAQaH4AQAAQaH4AQAAQSmbNT9Rmw+6n1U2bNjQylEvsnQ3dErlc9YffvjByn/729+sHPXCuXnz5iU9L3KrWbNmebnO3nvvbeWoP/sJEyakfd6DDz7YygcddJCV33zzzbTPGboHHnig0EPIivsC3Fq1aln5vvvu8/rkanNPoJjx5AcAAASF4gcAAASF4gcAAASlbNb89OrVyzv2/PPPW7l169ZWPvvss70+DRo0sHL9+vWTXvuZZ56xsrtnC4rTW2+9lZPz7r///lbeeeedrRy110qyF2bWqVPHO/bII49Y2V33NmLEiCrPidJ27bXXesduuOEGK99zzz1Wdl/CjOKwYMECK0fdI9w1q8gOT34AAEBQKH4AAEBQKH4AAEBQKH4AAEBQymbB84YNG7xj48aNqzLfcsstXp969epZ2V1A2KdPH6/PmWeeaeU77rjDynPnzvUHjILr1KmTlf/yl7/Ecl5388SddtrJylHzLhn3Jb0i/iaH7gL/ZcuWpX0dFK9jjjnGyoMGDfLafPXVV1a+6667rLxly5b4B4asuS/Z/vrrr7027qa869aty+WQyh5PfgAAQFAofgAAQFAofgAAQFDKZs1PXFatWmXlJ554wsoXX3yx12f33Xe3srtuCMUpXy82zcQpp5xi5YEDBybt467vWL9+faxjQn65mxheccUVVh4/frzXp2fPnlbmBcql6YsvvvCO9e7dOy/Xdl/u7eZywZMfAAAQFIofAAAQFIofAAAQlJJY89O/f3/v2DvvvFNlzpVq1fx6sVw/Ey0n999/v3fMfelj1LqaAQMGxD6WVq1aecfcl6EOHTrUylEvPnXXBX300UcxjA754L5AeciQIV6bc88918qvvfaalbt06eL1YR8fZMsYU2UuFzz5AQAAQaH4AQAAQaH4AQAAQaH4AQAAQSnKBc+nn366lfv27eu1GTVqVNbX2W+//bxj7kZiRx99tJVr1qzp9SnXBWHlZMSIEd6xY4891soXXXSR1+ZnP/uZla+++morRy1E3nXXXasci7uQVUTknHPOqbJP1MLrSZMmVdkHxevZZ5+1ctu2bb02n376qZVHjhxp5bp163p96tevb+VZs2ZlNsA07bLLLt6xRo0aWXnatGl5GQuQCp78AACAoFD8AACAoFD8AACAoBTFmh9348DzzjvPyrVq1fL6bNq0ycrupmEiIjvuuKOVL7jggiqziEjjxo2t7K7nWbt2rdfn3nvvtfLXX3/ttUHxee6556x86KGHem0uvPBCK//yl7+08saNG70+hxxySAyjs82cOTP2cyI3otbiuC+d/c1vfpP0PAcffLCVn3/+eSsvWbLE6+PeS9esWWPl6tXTv+XPnz/fO/bqq69a+dJLL/XauPffqN8XoFB48gMAAIJC8QMAAIJC8QMAAIJSFGt+dtttNyt37tw5aZ8FCxZYOa69diZOnGjlMWPGWPmNN97w+nzxxRexXBv59c9//tPK7r4qIv66sJ49e1rZXdcgIvLdd99Z+dFHH7XyTjvt5PX57W9/a+VBgwZZefTo0V4fFKeo9Tw9evSwsvvnG9ceOKtWrbLycccdZ+WoFzPXrl3byu49sF+/fl6fevXqWdldgymSfL8roJB48gMAAIJC8QMAAIJC8QMAAIJC8QMAAIKi6SwUVtWcvMHT3XjrpZdesrL7AsqKsVg56tfhvvx0/fr1Vn7qqae8Pu+9917Vgw2AMUaTt8pMruYQis5UY0yrXJ28mOdR1EaCTZs2tfLy5cutvHTp0lwOKStRC/RXrlxpZfcFwCL+pobffvtt2tfmXpR/ixYtsrL7A0nTp0/3+rRo0SKnY8pS5L2IJz8AACAoFD8AACAoFD8AACAoRbHmB8WFz9kRg2DX/CA+3Ivyb+HChVZ21/xEyeSFuXnEmh8AAACKHwAAEBSKHwAAEBSKHwAAEBSKHwAAEBSKHwAAEBSKHwAAEBSKHwAAEBSKHwAAEBSKHwAAEBSKHwAAEBSKHwAAEJSifhsZAADIn06dOln5mWeesfKUKVPyOZyc4ckPAAAICsUPAAAICsUPAAAICsUPAAAIihpjUm+smnpjlCxjjObq3MyhYEw1xrTK1cmZR2HgXoQYRN6LePIDAACCQvEDAACCQvEDAACCku4mh0tEZHYuBoKi0STH52cOhYF5hGwxhxCHyHmU1oJnAACAUsfHXgAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAICgUPwAAIChFU/yo6ixVXaeqa1R1kaqOUtU6ObpWB1U1qnprhv3HVYxzjapuUtWN2+T7YxxnTVX9q6rOV9XlqnqfqtaI6/zlJh9zSFUHqupnqrpZVW/K4jz5mkP3b3PeNaq6QVVXx3X+csS9KPI6zKM0cT+KvM7vVXWqqq5S1bmqOkRVq8d1/nQUTfFT4SRjTB0RaSkirUWkf9wXqCge7hGRDzM9hzHmeGNMnYqxPi4iQ37KxphL4hqriFwrIq1EpLmINJPE70vsvydlJtdz6FsR6Ssir2RzknzNIWPMJduct46IPCkio+M6fxnjXmRfh3mUGe5Htloi0kdEdhORw0TkdyJyTYznT1mxFT8iImKMmSci4yTxTT9uV4vIeBH5qqpGqtpOVVfk4PrpOElEhhljlhljFovIMBG5sMBjKgm5mkPGmH8YY8aJSNJ/9RbJHPovVa0tIl1F5B+FHkup4F7kYx6lj/tRgjFmhDHmXWPMxorfk8dFpG0hxlKUxY+qNhKRE0Tk40r++1hVXVHJ19gqzttEEsXDLcnGYIyZZIypn+Evwb1uuyrGu0JV21XWteJr27yXqu4Ux7jKWa7mUDqKZA5tq6uILBaRd+IYUwi4F0ViHqWJ+1GljhSRz+MYU7oK8llbFcao6mYRWSmJx3iDoxoZYzpneP5hIjLAGLNGVZM2josxZpKI1M+g6zgRuVJV3xaR7UTkiorjtSTxewRfrudQQWQxh7b1exF51Bhjsh9R2eNeVDnmUeq4H1VCVf8giWUdF8UxpnQVW/HTxRjzRi5OrKoniUhdY8zTuTh/jgySxAT7REQ2iMhDInKIiPxQuCEVvZzNoVJW8S/PDiLSs9BjKRHciyIwj9LG/SiCqnYRkdtEpKMxZkkhxlCUH3sl46xMd7/GVdLtdyLSSlUXqupCETlLRPqo6ot5GG/7Ksa7RlXbR/UzxqwzxvQyxuxpjNlHRJaKyFRjzJZcj7ncZTiHCibTObSN7iIy2RgzMx/jDUUo96JtMI9yIKT7kaoeJ4l/yJ9kjPksf6O2FduTn5QYY47PoNsASVSaP7lHROaLyMCoxqp6lIi8bYzJ+pm0MeZdEUn7RxxVdU8RMSKyQBIr4weISI9sx4OM59BPP6GznST+4VBdVXcQkU1RBWkxzKFtdBeR27MdB2yh3Iu2wTzKgVDuR6r6W0kscj7VGDMl23FkoySf/GTCGLPaGLPwpy8RWSciPxpjllXSpZGIvJ+/EUbaV0Qmi8iPkvjJimuNMeMLO6TgPSSJuXOOiPSr+P/nV9K2GOaQqOrhIrKX8KPJRaFE70XMo+JUavejASKyk4j8q9BPtpQ1a9FUdaSIjDbGvFbosaA0MYcQB+YR4sA8slH8AACAoATzsRcAAIAIxQ8AAAgMxQ8AAAgKxQ8AAAhKWvv8qCqrowMQxz4QlWEOBWOJMWb3XJ2ceRQG7kWIQeS9iCc/AHJhdqEHAABSyb2I4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASF4gcAAASleqEHAAAAslO9uv/t3Bhj5S1btuRrOEWPJz8AACAoFD8AACAoFD8AACAorPlJYvvtt7fyFVdc4bW58cYbrbx06VIrN2jQwOtzzDHHWPm9996zcpMmTbw+5557rpVvv/12r83WrVu9Y8hc06ZNrdysWTMrn3feeV6fvffe28ozZ8608j777OP1efzxx608atQoK2/YsCHpWAGEo3PnzlZ+9NFHvTbu96LBgwdb+R//+IfXJ5TvITz5AQAAQaH4AQAAQaH4AQAAQVF3H4AqG6um3rhEVatm14PDhw+38gEHHOD1eeSRR6z89NNPW/m+++7z+vTr18/K69evt/KUKVO8PjvssIOVW7Zs6bVZvHixdyxdxhjN+iSVKOY59POf/9w7NnnyZCs3atTIyqr+b1U6f6cqO88NN9xg5VtvvTXtcxbYVGNMq1ydPJN5NGLECCt/+OGHXhv37zIKK9R7USrc9Yi9evXy2pxxxhlW3muvvazsrjUVEbnooous/PXXX2c4wqIReS/iyQ8AAAgKxQ8AAAgKxQ8AAAgKxQ8AAAhK0Aue69Wr5x1zN5dbuHChlR977DGvzwcffFDlddq1a+cdcxdbTpw40cruwloRkeOPP97K06dPr/K6mQp1keGOO+7oHbv++uurzBs3bvT6PPjgg1b+/vvvrTxkyBCvj7vged68eVbO1eL2HCq6Bc/uve6HH37w2ribj3766afpXqbkHXjggVY+//zzvTbuBqvLly/PyVhCvRflyrHHHmvlp556ymtTo0YNK++///5Wdu9NJYAFzwAAABQ/AAAgKBQ/AAAgKEG/2PS4447zju25555WdjeOWrBgQdrXmTRpknfs6KOPtrL7Oesll1zi9cnVGh8krFu3zjt25513WtldizV27Fivz7PPPmvlPfbYw8pRa35c7vojd4NLpG/lypVW3m233bw2Z511lpW//fZbK69duzb+geXRLrvs4h0755xzrOy+qHnXXXf1+rhz+oILLsh+cMi58ePHW7lnz55eG3eT3oMOOsjKJbjmJxJPfgAAQFAofgAAQFAofgAAQFAofgAAQFCCXvD8y1/+0jv2zjvvWDmTBc4ud9MwEZGXXnrJyqtXr7bynDlzsr4usucuks1kYae7ODTqTfCuVNogPd26dbOy+3dQROTaa6+18i9+8Qsr33bbbV4fdyPU+fPnZzrEtDRu3Ng71qZNGyu7G6N26NDB67P33nunfe22bdum3QfF57nnnvOOuW9xP+SQQ6z86quv5nRM+cKTHwAAEBSKHwAAEBSKHwAAEJSg1vzccccdVj7yyCO9Nu3bt4/9updddpl3zN3E7g9/+IOVQ3yhYrnq2LGjlaNeJuyu8dl+++2tHPXSVaTntddes3LU2oVOnTpZuWvXrlY+8cQTvT6bNm2yctTLbt11QDVr1rTy6NGjI0Zs23nnna3cvXt3r02dOnWSnicOY8aMyct1QuVutisicuaZZ1r5qKOOsnLUy2XdefXee+8lvY47h1544YUqx1qqePIDAACCQvEDAACCQvEDAACCUtZrfs4+++wqc9Q+P1Gf1yfjvixw8ODBVo56edzQoUOt7L4ME+XjV7/6Vdp9PvvsMyu7e28gfZs3b7Zy1PqdgQMHWvniiy+2ctRLPlN56WxUv23169cv6TlS4a7PcNcwRr3M1bVlyxYrX3fddV6bhx56KIPRQUSkbt263rHTTz/dynfffbfXxl0XuGTJEitXr+5/O//9739vZXffuqj95D788EMrf/XVV16bcsCTHwAAEBSKHwAAEBSKHwAAEBSKHwAAEJSyWfActdjr9ttvt/KAAQOsvHbt2qTnrVbNrg9/+9vfem1uuOEGK7svHPzTn/7k9Rk2bFjSa6M0uZsauhvlpSLqhYPIPfce8corr1jZ/aEJEX+zwQ0bNnht3MWpBxxwgJXdRcZRRo0aZeU333zTa+NulprKpofuQvCjjz7aypMnT056DqTO/f0VETnuuOOsHLXg+eGHH7by999/b2V3Y1QRf9NO92W+DRs29PqsWrXKyk2aNLHy7NmzvT6liCc/AAAgKBQ/AAAgKBQ/AAAgKGWz5qd58+beMXcd0NixY6281157eX2aNm1q5W7duln5kksu8fq4GyOefPLJVnZfqIjy5n7OXq9evbTP4b6AEIXxwQcfVJlFRPr06ZP0PEcccYSV3XWBy5YtS3qON954w8pRawmPPfbYpOdxuWsWWeMTr7Zt21r5scce89qce+65VnbXmqUiaoPeZGu+Zs6c6R1zX/jtbnp4yimneH3cNqWAJz8AACAoFD8AACAoFD8AACAoaoxJvbFq6o3zbP/99/eOTZs2zcqff/65lVu0aOH1mTVrlpVXrlyZ9DrTp0+3cps2baoca7EzxmjyVpkp5jmUiaiXFLpry9q1a5f0PH/961+t3LdvXytv3bo1g9EV1FRjTKtcnbzc5lEq3BdbTpgwwWuTbK4tXbrUO7bffvtZ2d3npZDK4V7krhOL2o+nQ4cOVl69enXa1zn++OO9YyNHjrTyokWLkvY57LDDrOzuORS1N9Ahhxxi5SJ7GWrkvYgnPwAAICgUPwAAICgUPwAAICgUPwAAIChls8lh1AIrd/MxdyMpd3MvEZG//e1vVv6f//kfK0cteJ40aVKqw0SZ6dGjh3cslQXOrkGDBlm5BBc4I8d+9atfWTmVebZixQorn3baaV6bYlrgXI7mzp1r5aiF6pkscHY3ynzwwQe9NuvXr7fyCSecYGV3AbSI//LTL774wsrjx4/3+owZM8bK7mab7ktYiwFPfgAAQFAofgAAQFAofgAAQFDKZpPDuFxxxRVWdjd4inqxobtB1aZNm2IfVz6Vw8Zi+fLuu+96x9yXWLqiPpu/9NJLYxtTkWCTw5g9/vjjVj777LO9NmvWrLFyv379rHzvvffGP7AcKod7UatW9l+DZ5991mvTu3dvK7/88stemxNPPNHKd911l5XdP3sRf458++23VQ82Be6mmCL+OiD3JauHHnqo1+fHH3/MeiwpYpNDAAAAih8AABAUih8AABCUoNf8NGrUyDv2n//8x8ru/gvNmzf3+ixYsCDegRVYOXzOni+p7Mfjvhz3gAMO8NqU2xwS1vxkbeedd7ayuyfLdttt5/Xp37+/lf/yl7/EP7A8Ksd70S233OIdc//cnn/+ea9N586drTx//nwrR+375LbJFXcd0EcffWTliRMnen3OPPNMK69bty7+gSWw5gcAAIDiBwAABIXiBwAABIXiBwAABKVsXmyaCneB4FtvveW1cV+Q6r6gbeHChfEPDCWjY8eOSdu4P0TgblhWhoubkQN9+/a1ctQCZxcvxC1+Q4YM8Y4deeSRVm7WrJnX5pJLLrGyuyi6kC+odTdPPOOMM6wctbGju2FwmzZtvDYbNmyIYXTRePIDAACCQvEDAACCQvEDAACCEtSaH/flcXvssYfXpm7duvkaDkpA7dq1rTx8+PC0z3HnnXfGNRyUqZYtW3rHrrnmmrTPU8h1H0hN1AtIjzvuOCtHrd1yXxZazF577TUrt27d2mvjboQ4duxYr02nTp2sHOeaNp78AACAoFD8AACAoFD8AACAoJT1mp+jjjrKygMHDrTy7bffnsfRoBS5a8Ci9t8AsuW+UFlEZObMmVZ2Xx4ZZdq0abGNCfmzfv36Qg8hp9z980REevToYeXHHnvMa9OrVy8rDxs2LLYx8eQHAAAEheIHAAAEheIHAAAEheIHAAAEpWwWPHfv3t07dscdd1h5zJgxVr7ttttyOSSUIVWtMgOZWLt2bUrHthX10sfp06fHNiYgl5588kkrN2jQwGszdOhQK8+ZM8fKL7zwQsbX58kPAAAICsUPAAAICsUPAAAIihpjUm+smnrjPPvss8+8Y+5LKQ888EArr1u3LqdjKlXGmJwtZCnmORTF3WTrnnvusXLUmp+XX37Zyl27drXy5s2bYxpdUZtqjGmVq5OX2jxKplUr/7fqww8/rLLPU0895R3r1q1bbGMqBtyLwrHddtt5xyZMmGDlxo0bW7lJkyapnDryXsSTHwAAEBSKHwAAEBSKHwAAEBSKHwAAEJSy2eQwyqhRo6zMAmeky11I776duGnTpl6fRo0aWTmQBc7IM3eTOKCUbdmyxTvWoUMHK1erFt/zGp78AACAoFD8AACAoFD8AACAoJTsmp899tjDyoMHD/baPP300/kaDsrUxIkTrdyxY0crX3nllV6fGTNm5HRMKD/fffedd8zd5PCggw6y8ty5c3M6JqDQtm7dWmXOBk9+AABAUCh+AABAUCh+AABAUMrmxaaIDy8TRAx4sWmWdtllFyvvvvvuVg5hbRn3IsSAF5sCAABQ/AAAgKBQ/AAAgKBQ/AAAgKCU7CaHAFDOli1bVmUGkDme/AAAgKBQ/AAAgKBQ/AAAgKCku+ZniYjMzsVAUDSa5Pj8zKEwMI+QLeYQ4hA5j9La4RkAAKDU8bEXAAAICsUPAAAICsUPAAAICsUPAAAICsUPAAAICsUPAAAICsUPAAAICsUPAAAICsUPAAAIyv8BfPxT+01mazgAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 720x576 with 12 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "Y_test = model.predict(X_test)\n",
    "c_test = np.argmax(Y_test, axis=-1)\n",
    "\n",
    "misclassification = (c_test != t_test).nonzero()[0]\n",
    "\n",
    "plt.figure(figsize=(10, 8))\n",
    "for i in range(12):\n",
    "    plt.subplot(3, 4, i+1)\n",
    "    plt.imshow((X_test[misclassification[i], :] + X_mean).reshape((28, 28)), cmap=plt.cm.gray, interpolation='nearest')\n",
    "    plt.title('P = ' + str(c_test[misclassification[i]]) + ' ; T = ' + str(t_test[misclassification[i]]))\n",
    "    plt.xticks([]); plt.yticks([])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Questions\n",
    "\n",
    "With the provided model, you probably obtained a final accuracy on the test set around 90%. That is lame. The state-of-the-art performance is 99.7%.\n",
    "\n",
    "The goal of this exercise is now to modify the network in order to obtain an accuracy of **98%** (or more) in 20 epochs only.\n",
    "\n",
    "You are free to use any improvement on the basic model, using the doc of Keras. Here are some suggestions:\n",
    "\n",
    "* Change the learning rate of SGD.\n",
    "\n",
    "* Change the number of neurons in the hidden layer.\n",
    "\n",
    "* Change the number of hidden layers (just stack another `Dense` layer in the model).\n",
    "\n",
    "**Beware:** you do not have three weeks in front of you, so keep the complexity of your model in a reasonable range.\n",
    "\n",
    "* Change the transfer function of the hidden neurons. See <https://keras.io/activations/> for the different possibilities in `keras`. Check in particular the Rectifier Linear Unit (ReLU).\n",
    "\n",
    "* Change the learning rule. Instead of the regular SGD, use for example the Nesterov Momentum method:\n",
    "\n",
    "```python\n",
    "optimizer = tf.keras.optimizers.SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)\n",
    "```\n",
    "\n",
    "or the Adam learning rule:\n",
    "\n",
    "```python\n",
    "optimizer = tf.keras.optimizers.Adam(lr=0.01)\n",
    "```\n",
    "\n",
    "* Change the batch size. What impact does it have on training time?\n",
    "\n",
    "* Apply **L2- or L1-regularization** to the weight updates to avoid overfitting <https://keras.io/regularizers/>:\n",
    "\n",
    "```python\n",
    "model.add(tf.keras.layers.Dense(50, kernel_regularizer=tf.keras.regularizers.l2(0.0001)))\n",
    "```\n",
    "\n",
    "* Apply **dropout** regularization after each layer. Find a good level of dropout.\n",
    "\n",
    "```python\n",
    "model.add(tf.keras.layers.Dropout(0.5))\n",
    "```\n",
    "\n",
    "* Add **Batch normalization** between the fully-connected layer and the transfer function.\n",
    "\n",
    "```python\n",
    "model.add(tf.keras.layers.Dense(100)) # Weights\n",
    "model.add(tf.keras.layers.BatchNormalization()) # Batch normalization\n",
    "model.add(tf.keras.layers.Activation('relu')) # Transfer function\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solution\n",
    "\n",
    "Here is a proposal which obtains 98% accuracy in a reasonable time. There are many other possible solutions, perhaps you found even better... "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " dense (Dense)               (None, 150)               117750    \n",
      "                                                                 \n",
      " batch_normalization (BatchN  (None, 150)              600       \n",
      " ormalization)                                                   \n",
      "                                                                 \n",
      " activation (Activation)     (None, 150)               0         \n",
      "                                                                 \n",
      " dropout (Dropout)           (None, 150)               0         \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 100)               15100     \n",
      "                                                                 \n",
      " batch_normalization_1 (Batc  (None, 100)              400       \n",
      " hNormalization)                                                 \n",
      "                                                                 \n",
      " activation_1 (Activation)   (None, 100)               0         \n",
      "                                                                 \n",
      " dropout_1 (Dropout)         (None, 100)               0         \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 10)                1010      \n",
      "                                                                 \n",
      " activation_2 (Activation)   (None, 10)                0         \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 134,860\n",
      "Trainable params: 134,360\n",
      "Non-trainable params: 500\n",
      "_________________________________________________________________\n",
      "None\n",
      "Epoch 1/20\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-03-20 18:11:37.357186: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "422/422 [==============================] - ETA: 0s - loss: 0.4516 - accuracy: 0.8682"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-03-20 18:11:45.140012: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "422/422 [==============================] - 9s 17ms/step - loss: 0.4516 - accuracy: 0.8682 - val_loss: 0.1409 - val_accuracy: 0.9613\n",
      "Epoch 2/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.2019 - accuracy: 0.9401 - val_loss: 0.0954 - val_accuracy: 0.9723\n",
      "Epoch 3/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.1556 - accuracy: 0.9539 - val_loss: 0.0801 - val_accuracy: 0.9762\n",
      "Epoch 4/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.1285 - accuracy: 0.9609 - val_loss: 0.0742 - val_accuracy: 0.9782\n",
      "Epoch 5/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.1094 - accuracy: 0.9663 - val_loss: 0.0698 - val_accuracy: 0.9790\n",
      "Epoch 6/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0985 - accuracy: 0.9692 - val_loss: 0.0670 - val_accuracy: 0.9805\n",
      "Epoch 7/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0901 - accuracy: 0.9715 - val_loss: 0.0636 - val_accuracy: 0.9807\n",
      "Epoch 8/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0804 - accuracy: 0.9741 - val_loss: 0.0645 - val_accuracy: 0.9805\n",
      "Epoch 9/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0773 - accuracy: 0.9753 - val_loss: 0.0598 - val_accuracy: 0.9828\n",
      "Epoch 10/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0705 - accuracy: 0.9771 - val_loss: 0.0580 - val_accuracy: 0.9827\n",
      "Epoch 11/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0671 - accuracy: 0.9784 - val_loss: 0.0603 - val_accuracy: 0.9817\n",
      "Epoch 12/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0615 - accuracy: 0.9804 - val_loss: 0.0600 - val_accuracy: 0.9817\n",
      "Epoch 13/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0586 - accuracy: 0.9813 - val_loss: 0.0551 - val_accuracy: 0.9850\n",
      "Epoch 14/20\n",
      "422/422 [==============================] - 7s 17ms/step - loss: 0.0558 - accuracy: 0.9809 - val_loss: 0.0621 - val_accuracy: 0.9812\n",
      "Epoch 15/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0525 - accuracy: 0.9828 - val_loss: 0.0575 - val_accuracy: 0.9833\n",
      "Epoch 16/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0502 - accuracy: 0.9838 - val_loss: 0.0615 - val_accuracy: 0.9828\n",
      "Epoch 17/20\n",
      "422/422 [==============================] - 7s 15ms/step - loss: 0.0502 - accuracy: 0.9836 - val_loss: 0.0581 - val_accuracy: 0.9833\n",
      "Epoch 18/20\n",
      "422/422 [==============================] - 7s 16ms/step - loss: 0.0494 - accuracy: 0.9834 - val_loss: 0.0579 - val_accuracy: 0.9837\n",
      "Epoch 19/20\n",
      "422/422 [==============================] - 7s 15ms/step - loss: 0.0445 - accuracy: 0.9845 - val_loss: 0.0537 - val_accuracy: 0.9852\n",
      "Epoch 20/20\n",
      "422/422 [==============================] - 7s 15ms/step - loss: 0.0437 - accuracy: 0.9852 - val_loss: 0.0513 - val_accuracy: 0.9857\n",
      "Test loss: 0.059500761330127716\n",
      "Test accuracy: 0.9824000597000122\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA34AAAFzCAYAAABhKNvjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABnyklEQVR4nO3deXxU1f3/8deHsAQSdlAJQQKKLIosRtwVRUWtiqIWqV+FuiDWatWf+0pdWrVo1Va0qGhttVSrULAoKi64yyqKgLIpIYhhTdhJOL8/zgwZwgQmyUzuTPJ+Ph7zyMydeyefDIHLe86552POOURERERERKTmqhN0ASIiIiIiIpJYCn4iIiIiIiI1nIKfiIiIiIhIDafgJyIiIiIiUsMp+ImIiIiIiNRwCn4iIiIiIiI1XN2gC4inVq1auZycnKDLEBGRBJsxY8Yq51zroOtIFTo/iojUHuWdI2tU8MvJyWH69OlBlyEiIglmZj8EXUMq0flRRKT2KO8cqameIiIiIiIiNZyCn4iIiIiISA2n4CciIiIiIlLD1ahr/EREgrR9+3by8vLYsmVL0KXUGOnp6WRnZ1OvXr2gSxEREUlpCn4iInGSl5dH48aNycnJwcyCLiflOedYvXo1eXl5dOjQIehyREREUpqmeoqIxMmWLVto2bKlQl+cmBktW7bUCKqIiEgcKPiJiMSRQl986f0UERGJDwU/EZEaYvXq1fTs2ZOePXuy33770bZt252Pt23btsdjp0+fzrXXXrvX73H00UfHq1wRERGpRrrGT0SkhmjZsiWzZ88GYMSIEWRmZnLjjTfufL64uJi6daP/s5+bm0tubu5ev8enn34al1pFRESkemnET0SkBhs6dCg33HADJ554IrfccgtffvklRx99NL169eLoo49mwYIFAHzwwQeceeaZgA+Nl156KX379qVjx4488cQTO18vMzNz5/59+/bl/PPPp0uXLlx00UU45wCYNGkSXbp04dhjj+Xaa6/d+boiIiISnISO+JnZacDjQBrwrHPuwXL2Oxz4HBjknPtPaNtSoAgoAYqdc3v/KFpEJFlcdx2ERt/ipmdPeOyxCh/23Xff8e6775KWlkZhYSFTp06lbt26vPvuu9x+++289tprux0zf/583n//fYqKiujcuTNXXXXVbi0VZs2axdy5c8nKyuKYY47hk08+ITc3lyuvvJKpU6fSoUMHBg8eXMkfVkREROIpYcHPzNKAJ4FTgDxgmplNcM59G2W/h4DJUV7mROfcqkTVuJupU6FJE/+fKxGRGuKCCy4gLS0NgPXr1zNkyBC+//57zIzt27dHPeYXv/gFDRo0oEGDBuyzzz6sXLmS7OzsXfbp06fPzm09e/Zk6dKlZGZm0rFjx53tFwYPHszo0aMT+NOJiIikpq1bYc0af1u71t/OOitx3y+RI359gIXOucUAZjYWGAB8W2a/a4DXgMMTWEtsLroI+vWDF14IuhIRSXWVGJlLlIyMjJ3377rrLk488UTGjRvH0qVL6du3b9RjGjRosPN+WloaxcXFMe0Tnu4pIiJSGzgHRUW7Brjw/fK2hR9v2rT7623aBA0bJqbWRAa/tsCyiMd5wBGRO5hZW+Bc4CR2D34OeNvMHPA351ziPzLOyoIVKxL+bUREgrJ+/Xratm0LwAsJ+JCrS5cuLF68mKVLl5KTk8O///3vuH8PEZGa6Oef/dhDu3YwaBDU0UocSWnlSpg2Db780n+dNg1Wry5//4bpO2jeuJgWDTfTov4GDqiznubpq2nReiUtNi+nxYYfaL5pOS1YQwvWUK/4YyAxyS+RwS9a86WyHwU/BtzinCuJ0qvpGOdcvpntA7xjZvOdc1N3+yZmw4BhAPvvv3/VKs7KgoULq/YaIiJJ7Oabb2bIkCE8+uijnHTSSXF//YYNGzJq1ChOO+00WrVqRZ8+feL+PUREapKFC+GRR+D55/3UP4AHH4Q//AHOOAPUzjQ469fDjBm7Br1loWGtOnUcB3cuZsDhBXRtvIyW23+ixeZ8mhf9SIu1i2ix6juaF3xHwy2bYUvEi6alQZs2PndkZUHbtpDVo/Rxg7SE/TyWqGk5ZnYUMMI51z/0+DYA59wfI/ZZQmlAbAVsAoY558aXea0RwAbn3Mg9fc/c3Fw3ffr0yhf9m9/Av/+959guIlKOefPm0bVr16DLCNyGDRvIzMzEOcfVV19Np06duP766yv9etHeVzOboUW/Ylfl86OIxN306fDww/Daa1CvHgwZAjfc4NcFu/NOHwiPPRb++Ef/VRJryxb46qtdR/Lmzy99vuO+G+iz7w8cXu8rDt/4Ab3zJpCxYeWuL9K6dWmA2xnqytxv3dqHvwQq7xyZyBG/aUAnM+sALAcuBH4VuYNzrkNEgS8AbzjnxptZBlDHOVcUun8qcG8Ca/WysvyE261bIeLaFRERid0zzzzD3//+d7Zt20avXr248sorgy5JRCQpOAdvvw0PPQTvvw9Nm8Itt8C118J++/l9OneGgQNhzBj4/e/huOPgzDPhgQfg0EODrb9Ctm+HH3+EJUtg8WJYvtzPX61fv3K3evVK76en+wvhGjSIaU6scz7Ybdjgbxs3+uvy5s0rHc2bM8dRXOzHo/bLKOTwzPlc1PJTDl/7Nrk7vqDlyjWwEsjOhi5d4OQL/NcuXeCAA/woXpLnh4QFP+dcsZn9Fr9aZxowxjk318yGh55/eg+H7wuMC03/rAu87Jx7K1G17pSV5b+uWAE5OQn/diIiNdH1119fpRE+EZGaprjYTyp7+GGYM8cP/owcCcOGQePGu+9frx5ceSVcfDH85S9+6mfPnn4dwt//Hjp2THzNzsHMmfDNNz6gtmhRemveHBqmOygo8KFu8eLSgBf+umwZ7NhR+e8PbKYha2keuvrN39bSnCIas4FMNpDJxrQmbKjTlA1pTdhgjdkQfs41YqNrxIaShmwobsiOctqXN00rIrfOLG4s/oQ+fMnhTKPttgKsVSfo2hW6HAZdLvIB76CDov+BpYiE9vFzzk0CJpXZFjXwOeeGRtxfDPRIZG1RhYNffr6Cn4iIiEgcbd7sFzCJNpBTUxcy2bgRnnsOHn0UfvgBunXz1/L96lf+Z9+bRo38iOCwYX6U8PHHfYC88ko/HXTffeNb7+bNMGUKTJwIb7zh/0tcnnS20ILttCCD5mTTgka0aNiR5s2gxT5ptOjWkBbtG9P8gBa06LIPjTu2prDIWFNQzNqCEtasKmHNKsea1c6vcrkW1qytw5r1dVhbmMaawrps3b7nKZH16pSQWW8rmXW3kll3M5lpm8mos5m2Vkim5fto6IrI2LGBzJL1/rZ9LZnF68jYvpaOTdfQ6eD61OnaOTR6dxl0+ZPPAXUTGpMCUfN+oqpo08Z/1cqeIiIiIlVWWOgDxOuvw5tvRl++HvwlT7HM8AvfGjf2o14HHuhn2R14oF8NM1n+r15QAH/9q7+tWeOv0fvrX/1iLZUJuc2b+1G/a66B++6Dp57yAfL66+HGG/2IXGXl5/s/o4kTYcoUx+bNRuP0bfRvPo2z0l/kyC3vs5EMP+KWnsXalp1Y0ySHNY2yWVNvX9bWacWa7R1ZsrkhM9bV8W0KVgBflfcdd028mZn+5wuPJnY9YNfH4RHGyPtNmkBGBtSvnwY0Ct2aV+wHd67WrZyTJH89kkTkiJ+IiIiIVNjq1TBhgl+05J13YNs2/9n60KHQu7ef9rh9u99emdumTf5ysTff9NdthdWtCx06lAbByFDYoUP1XH61eLEf3Rszxo+eDRgAN98MRx8dn9dv2xaeftovAnP33XD//TBqFNx+O1x9tb/0bW+cg1mzfNCbONGvWgmQk1nA5WkTOYt/ccKWD6nfsB2cdTIcfw906uTfxFatYgpLW7fu2rtu7Vr/IUB4ymg4yDVvHtvIZ0LUstAHCn67atnSf6yk4CciIiISs59+gvHjfdh7/30oKYH27X0YOe88OOqo+E/n3LHD/5dt0SK/Amb468KF8OmnPmiEmfkRwXAQDH9t397/16+q1q3zgeyVV/zo5SWX+JG4Ll2q/trRHHQQjB0LN93kQ9+NN8Jjj8GIEX510LIjn5s3w3vvhaZwTtzB8vw6GDs4stEc/sC/OYuJHFxvBXZyP79oycl/q9KFhA0a+MVqwgvWSHJQ8ItUp47/DdVUTxFJQX379uW2226jf//+O7c99thjfPfdd4waNSrq/iNHjiQ3N5czzjiDl19+mWbNmu2yz4gRI8jMzOTGG28s9/uOHz+egw46iG7dugFw9913c/zxx3PyySfH5wcTkaT0449+Cudrr8Enn/iRpIMO8iNc553nR/cSOahSp45fYDE7G044YdfnnINVq3YPhYsW+YBaUBD/epo08QHsd78rnURWISUlfiizpMTPf2zc2CeoPbyJhx0Gkyf7UHfbbXD55X7RmAcegCOPhP/9DyZO2MG77zg2b00jM20Tp+6YzFn8lzPqT2GfozrDySfDKX/3q8ckuM2ABEvBr6ysLI34iUhKGjx4MGPHjt0l+I0dO5Y//elPez120qRJe92nPOPHj+fMM8/cGfzuvTfx3XdSnZmdBjyOX/X6Wefcg2Webw6MAQ7At/691Dn3Tei564HL8YvefQ382jkX2R5YJGG+/7407E2b5rd17w733OPD3sEHJ8cMOjPfLq11ax+Ayios9CFw2TKfs6oqLc2Hzz1ea1dS4v+PuXTprrclS/zXZcv8PNiyL9y4cWkQLOfrSZmZfP7LxozvfQi3TziC884rLWR/W86l7r+cxUT6HrKOBv37wskXwbFP+ZYIUmso+JWVlQXffRd0FSIiFXb++edz5513snXrVho0aMDSpUvJz8/n5Zdf5vrrr2fz5s2cf/75/P73v9/t2JycHKZPn06rVq144IEHePHFF2nXrh2tW7fmsMMOA3x/vtGjR7Nt2zYOPPBA/vGPfzB79mwmTJjAhx9+yP33389rr73Gfffdx5lnnsn555/PlClTuPHGGykuLubwww/nqaeeokGDBuTk5DBkyBAmTpzI9u3befXVV+mSqDlRScbM0oAngVOAPGCamU1wzn0bsdvtwGzn3Llm1iW0fz8zawtcC3Rzzm02s1fwfXJfqNYfQhJmzRrfW2zz5vKvcYv1+rjt2/2Uv2iLo8TaLq1+fZ89Pv7Yh72vv/Z15ub6xuLnnecv/0o1TZpAr17+Fjc7dsDyFbuGucjbjz/6P5RIbdr4FSSPPBIGD/ZzT+vX903mNmwo/+uqVaVN6YqKYPNmDDgXOIs0/sVg8sni9DZf0f2MdtgpJ8NJ//RJWGotBb+y2rSBDz8MugoRSXHXXQezZ8f3NXv29NdwlKdly5b06dOHt956iwEDBjB27FgGDRrEbbfdRosWLSgpKaFfv37MmTOHQ8vpAjxjxgzGjh3LrFmzKC4upnfv3juD38CBA7niiisAuPPOO3nuuee45pprOPvss3cGvUhbtmxh6NChTJkyhYMOOohLLrmEp556iuuuuw6AVq1aMXPmTEaNGsXIkSN59tlnq/oWpYo+wMJQ6yLMbCwwAIgMft2APwI45+abWY6ZhRdurws0NLPt+KXsNE0lRW3a5BfZ+PJLP3o2bZqfjlhRdetGD3B16/pBpvICY0WYwTHHwJ//7JuL779/xeuscdas8Rc0vvuu/7/jokW7v7H77usXRTn8cLjgAh/ycnL8tv33j201lliUlOwMgnWLirh4wwa/ckrHjskxBCtJQcGvrKws/xd5y5b4/WUUEakm4eme4eA3ZswYXnnlFUaPHk1xcTErVqzg22+/LTf4ffTRR5x77rk0atQIgLPPPnvnc9988w133nkn69atY8OGDbtMKY1mwYIFdOjQgYMOOgiAIUOG8OSTT+4MfgMHDgTgsMMO4/XXX6/qj55K2gLLIh7nAUeU2ecrYCDwsZn1AdoD2c65GWY2EvgR2Ay87Zx7uxpqliravt03wg4HvC+/hLlzS6cZZmf7bHDppdCjh5/FF+voXGUWTXHOzyqMdSXNLl1Ku17VWps3+4sZ333XN7ubMcO/kZmZcPzxcNZZpaEuJ8cHu9C/pQmXlubnmValr4PUeAp+ZYWvxl2xwv/FFRGphD2NzCXSOeecww033MDMmTPZvHkzzZs3Z+TIkUybNo3mzZszdOhQtmzZ8+VgVs6nw0OHDmX8+PH06NGDF154gQ8++GCPr+Oc2+PzDUJrq6elpVFc9rqWmi3aG1z2zXoQeNzMZuOv45sFFIeu/RsAdADWAa+a2f855/652zcxGwYMA9hfwzPVascOfy1cOORNm+ZH9sJ/9Zo3hz594Oyzfdg7/PDqD1VmPjTWq+f7oUkUJSUwc6YPeu++60Pf1q1+KPWoo/yFjSef7P8w47E0qEiCKfiVFdnEXcFPRFJMZmYmffv25dJLL2Xw4MEUFhaSkZFB06ZNWblyJW+++SZ9+/Yt9/jjjz+eoUOHcuutt1JcXMzEiRO58sorASgqKqJNmzZs376dl156ibZt2wLQuHFjioqKdnutLl26sHTpUhYuXLjzmsATyi69VzvlAe0iHmdTZrqmc64Q+DWA+SS+JHTrDyxxzhWEnnsdOBrYLfg550YDowFyc3P3nMKlUrZs8etxLFvmL99asMCHvOnTYf16v0+jRn51y6uu8vng8MM1+y5pOecTezjovf++79MAcOihvjdFv35+dC8zM9BSRSpDwa8sNXEXkRQ3ePBgBg4cyNixY+nSpQu9evXi4IMPpmPHjhxzzDF7PLZ3794MGjSInj170r59e4477ridz913330cccQRtG/fnu7du+8MexdeeCFXXHEFTzzxBP/5z3927p+ens7zzz/PBRdcsHNxl+HDhyfmh04t04BOZtYBWI5fnOVXkTuYWTNgk3NuG34Fz6nOuUIz+xE40swa4ad69gOmV2fxtcWOHb433Y8/lga78Nfw/Z9/3vWYunV9PrjwQh/w+vSBrl1376kmSeSnn/y0zfD0zWWhWdj77+9XrunXD046yV+rJ5LibG9TcVJJbm6umz69iue/Vav8ikePPw7XXhufwkSkVpg3bx5du3YNuowaJ9r7amYznHO5AZVUZWZ2BvAYvp3DGOfcA2Y2HMA597SZHQW8CJTgF325zDm3NnTs74FBQDF+Cujlzrmte/p+cTk/1kDFxfDFF36lyshA9+OPvp1a2QUYGzf2eaBdu12/hu9nZ/u2a5KEduzwK23Onr3rLS/PP9+ihQ94J5/sw94BB2hYVlJWeedIfQZVVsuWfp62mriLiEiCOOcmAZPKbHs64v5nQNRF8p1z9wD3JLTAGmzpUt/wevJkP8BTWOi3163rg9v++8Oxx0YPd1o3I0Vs2eJXzokMeF995dsegF8IpUsX33ivVy848US/bHJlVskRSSEKfmWZ+ev8NNVTREQk5W3cCB98UBr2wq16998fBg2C/v19C7X99vN5QFLM6tW7j+LNm1e6XGpmpl8m9ZJLfLjr2dN3mVfjcqmFFPyiycpS8BMREUlBzsGcOaVB7+OPfTuChg39AM9VV8Fpp0HnzprJl3K2b/cr53zwAXz2mQ95yyI6o7Rt64PdgAGlIa9jR43kiYQo+EXTpk3pR4IiIhXgnCu3HYJUXE26Dl0Sp6AA3nnHB7233/brdQAccghcc40f1TvuOLXnTTnbtpUGvQ8+8O0UNm3yz3Xt6lfXDAe8Hj38Gg0iUi4Fv2iysvw/MCIiFZCens7q1atp2bKlwl8cOOdYvXo16frfukQxYwaMGwdvveVbrTnn1+c45RQf9E491Q8ASQrZts33wwgHvU8/LQ163bvDZZdB374+8LVqFWChIqlJwS+arCxYuxY2b9YccBGJWXZ2Nnl5eRQUFARdSo2Rnp5OdnZ20GVIEvn6a7jjDpg40V+Td8QR8Pvf+7B32GG6Ti+lbN26e9DbvNk/d+ihcPnlPugdd5yCnkgcKPhFE9nEvWPHYGsRkZRRr149OnToEHQZIjXSkiVwzz3wz39Ckybwhz/46/WaNQu6MonZli27Tt2MDHo9esCwYf5CzOOP96usi0hcKfhFE27iruAnIiISqJ9/hgcegKee8qN5N90Et9zip3VKktq4EebPh2+/3fW2eLHvp2dWGvTCI3oKeiIJp+AXTTj4aWVPERGRQBQWwiOP+NuWLXDppXD33b7XniSJwkLfOqFswFu6tHSfevXgoIN8v7yLLvJfjztOyV0kAAp+0YSneir4iYiIVKstW/zo3gMP+BZtv/wl3Hefzw4SkKIif3Hlt9/6xujhgJeXV7pPgwa+KfpRR/lFWLp187cDDvDhT0QCp+AXTcuW/h+pFSuCrkRERKRWKC6Gf/zDX8e3bJlfnfMPf4Dc3KArq6W2b/f9MV58ESZM8AuxgF/0rmtXP0Xz4INLA16HDlpZRyTJKfhFY6Ym7iIiItXAOfjvf+H22/2swcMPh+efh379gq6sFnIOZs3yYe/ll32DxNatYfhwOPlkH/Tat1dDdJEUpeBXnjZtFPxEREQS6IMP4NZb4Ysv/CzB116Dc8/1n79KNVq+HF56yQe+uXOhfn04+2wYMsT3ydBUTZEaQcGvPFlZfkUqERERiauZM/0I3+TJfrGW556DSy6BuvpfSfXZuBHGjfNh7913/Wjf0UfD00/7CyubNw+6QhGJM/0TW56sLHjvvaCrEBERqTG2bPE9uV96yS/q+Mgj8JvfQHp60JXVEjt2+GHWF1/0w6sbNkBODtx1F1x8MRx4YNAVikgCKfiVp00bWLfONxZt2DDoakRERFLevff60Hfbbb4XX9OmQVdUS8yf78PeP//pV85p0gQuvNAPsx5zjK7ZE6klEvo33cxOM7MFZrbQzG7dw36Hm1mJmZ1f0WMTJrKJu4iIiFTJjBnw8MPw61/71ToV+hIsLw/+8hfo08evwvnww9C9O4wdCz/9BM884/vpKfSJ1BoJG/EzszTgSeAUIA+YZmYTnHPfRtnvIWByRY9NqMgm7h07Vtu3FRERqWm2bfMN2PfZx0/vlARwDubM8Uuk/ve//kJKgJ494dFHYfBg2G+/QEsUkWAlcqpnH2Chc24xgJmNBQYAZcPbNcBrwOGVODZx1MRdREQkLh56qDSTaM2QONq+HT76yL+xEybA0qV+SdQjj4QHH4QBA/xyqSIiJDb4tQWWRTzOA46I3MHM2gLnAiexa/Db67ERrzEMGAaw//77V7nonTTVU0REpMq++Qbuu88POJ19dtDV1ACFhX451P/+F/73P78eQXq677N3xx1w1lmw775BVykiSSiRwS9aFx5X5vFjwC3OuRLbtWlPLMf6jc6NBkYD5ObmRt2nUlq08H1sNOInIiJSKcXFfopn06bw+ONBV5PCli/3I3r//S+8/76fO9uypR/RGzAATj0VMjKCrlJEklwig18e0C7icTZQNkXlAmNDoa8VcIaZFcd4bGKZqYm7iIhIFfz5zzBtml9PpHXroKtJIc75odLw9XrTp/vtBx4I11zjw97RR0NaWrB1ikhKSWTwmwZ0MrMOwHLgQuBXkTs45zqE75vZC8AbzrnxZlZ3b8dWi6wsTfUUERGphO++g7vvhnPO8f3AJQbLl8Ozz/rWC4sX+21HHOGXQR0wwK/OadEmRYmI7F3Cgp9zrtjMfotfrTMNGOOcm2tmw0PPP13RYxNVa7mysuDb6ltPRkREpCbYsQMuu8xfejZqlLLKHpWUwNtvw9/+Bm+84R+fcopvdHjWWaWLzYmIVFFCG7g75yYBk8psixr4nHND93ZstWvTBt59N9ASREREUs2oUfDxx/DCC8ot5frpJxgzBkaPhh9+8L0ubroJrrhCbaREJCESGvxSXlYWrF8PmzZBo0ZBVyMiIpL0li6FW2+F006DSy4Jupoks2MHTJniR/f++1+/+s1JJ/nm6uec4xeVExFJEAW/PYls6XDAAcHWIiIikuSc8wNWZj7baIpnSEEBPP+8H91btMivyHnddf7NOuigoKsTkVpCwW9PIpu4K/iJiIjs0Zgx/gqJp56CeLbWTUnOwYcf+gT82mu+2frxx8O998LAgf4CSBGRaqTgtydq4i4iIhKT5cvhhhugb18YNizoagK0Zg38/e8+8C1YAM2awW9+A1de6VflFBEJiILfnoSDn3r5iYiIlMs5GD7cD2o9+yzUqRN0RQFYsADuvx9efRW2bvV99v7+d7jgAmjYMOjqREQU/PaoeXNo0EDBT0REZA9eftl3Inj00Vp6ZcS//uWv16tTBy6/3I/ude8edFUiIrtQ8NsTM3+dn6Z6ioiIRLVyJVx7LRx5pP9aq2zZ4ue3PvUUHHssjB0LbdsGXZWISFS1cTJGxWRlacRPRESkHNdcAxs2+IVd0tKCrqYaLV4MxxzjQ99NN8F77yn0iUhS04jf3rRpA3PnBl2FiIhI0nntNX9J2x/+UMvWLRk/HoYO9TOD/vtfOPvsoCsSEdkrjfjtTVaWpnqKiIiUsWYNXH019OoFN94YdDXVZPt2/8Oeey506gQzZyr0iUjK0Ijf3mRlwfr1sHEjZGQEXY2IiEhSuP56WL0a3noL6tULuppqkJcHgwbBp5/69gyPPuoXgBMRSREa8dubcBN3jfqJiIgAMGkSvPgi3HYb9OwZdDXV4O23/dDmnDl+Bc8nn1ToE5GUo+C3N2riLiIistP69b5bwcEHwx13BF1NgpWUwN13w2mnwX77wfTpcOGFQVclIlIpmuq5N2riLiIistPNN/tT4muv1fBBr5Ur4aKLYMoUv5DLk09Co0ZBVyUiUmkKfnsTnuqp4CciIrXce+/B6NF+fZM+fYKuJoGmTvUje2vXwnPPwaWXBl2RiEiVaarn3jRv7j/S1FRPERGpxTZuhMsv94tZ3ntv0NUkyI4d8NBDcNJJkJkJX3yh0CciNYZG/PbGTE3cRUSk1rvjDliyxA+GNWwYdDUJsGYNDBkCb7wBF1wAzz4LTZoEXZWISNwo+MWiTRsFPxERqbU++QSeeAJ++1s47rigq0mAL7+EX/7Sn+v/8hffoNAs6KpEROJKUz1joSbuIiISR2Z2mpktMLOFZnZrlOebm9k4M5tjZl+a2SERzzUzs/+Y2Xwzm2dmRyW63gYN/MKWf/xjor9TAJ55Bo49FpyDjz/26VahT0RqIAW/WGiqp4iIxImZpQFPAqcD3YDBZtatzG63A7Odc4cClwCPRzz3OPCWc64L0AOYl+iac3N9777MzER/p2r2yCMwbBj06wezZtXwFWtEpLZT8ItFmzZQWAgbNgRdiYiIpL4+wELn3GLn3DZgLDCgzD7dgCkAzrn5QI6Z7WtmTYDjgedCz21zzq2rtsprkvvv98uTXnABTJgALVoEXZGISEIp+MVCTdxFRCR+2gLLIh7nhbZF+goYCGBmfYD2QDbQESgAnjezWWb2rJllRPsmZjbMzKab2fSCgoJ4/wypyzm480646y64+GJ4+WWoVy/oqkREEk7BLxYKfiIiEj/RLiBzZR4/CDQ3s9nANcAsoBi/KFtv4CnnXC9gI7DbNYIAzrnRzrlc51xu69at41V7anPOj/I98ABccQW88ALU1Tp3IlI76F+7WKiJu4iIxE8e0C7icTawywnGOVcI/BrAzAxYEro1AvKcc1+Edv0P5QQ/KWPHDrjmGhg1yn99/HEt4iIitYpG/GIRHvFT8BMRkaqbBnQysw5mVh+4EJgQuUNo5c76oYeXA1Odc4XOuZ+AZWbWOfRcP+Db6io8ZZWU+BG+UaPg5psV+kSkVtKIXyyaNYP0dE31FBGRKnPOFZvZb4HJQBowxjk318yGh55/GugKvGhmJfhgd1nES1wDvBQKhosJjQxKOYqLfWP2l1+Ge+7xN4U+EamFFPxiYaYm7iIiEjfOuUnApDLbno64/xnQqZxjZwO5iayvxti2DQYPhtdf900Ib9WsWBGpvRI61TOGBrUDQs1pZ4dWHjs24rmlZvZ1+LlE1hkT9fITERFJHVu2wMCBPvT9+c8KfSJS6yVsxC+iQe0p+AvZp5nZBOdc5LUIU4AJzjlnZocCrwBdIp4/0Tm3KlE1VkhWFsyZE3QVIiIisjebNsE558A778BTT8Hw4UFXJCISuESO+O21Qa1zboNzLryEdQa7L2edPDTVU0REJPkVFcHpp8OUKfD88wp9IiIhiQx+sTSoxczONbP5wP+ASyOecsDbZjbDzIaV902qrUFtVpY/mWzYkLjvISIiIpW3bh2ceip88gm89BIMHRp0RSIiSSORwS+WBrU458Y557oA5wD3RTx1jHOuN3A6cLWZHR/tm1Rbg1o1cRcREUleq1dDv34wYwa8+ipceGHQFYmIJJVEBr+9NqiN5JybChxgZq1Cj/NDX38GxuGnjgZHTdxFRESS08qVcOKJMHcujB8P554bdEUiIkknkcEvlga1B5r5Zjpm1huoD6w2swwzaxzangGcCnyTwFr3Tk3cRUREks/y5dC3LyxaBP/7H5xxRtAViYgkpYSt6hljg9rzgEvMbDuwGRgUWuFzX2BcKBPWBV52zr2VqFpjoqmeIiIiyeWHH/z0zp9/hrfeguOOC7oiEZGkldAG7jE0qH0IeCjKcYuBHomsrcKaNoX0dI34iYiIJINFi+Ckk6Cw0LdtOOKIoCsSEUlqCQ1+NYqZmriLiIgkgy1b4OSTYeNGeO896NUr6IpERJKegl9FZGVpqqeIiEjQRo2CpUt9rz6FPhGRmCRycZeaR03cRUREglVYCH/4A5xyip/qKSIiMVHwqwhN9RQREQnWY4/5nn0PPBB0JSIiKUXBryKysmDDBigqCroSERGR2mf1ahg50vfpO/zwoKsREUkpCn4VEW7iruv8REREqt9DD/kPYO+/P+hKRERSjoJfRaiJu4iISDDy8+Evf4GLL4Zu3YKuRkQk5Sj4VYSauIuIiATj/vuhpARGjAi6EhGRlKTgVxHhqZ4a8RMREak+ixfDM8/AFVdAhw5BVyMikpIU/CqiaVNo2FDBT0REpDrdcw/Uqwd33hl0JSIiKUvBryLM1MRdRESkOn3zDbz0ElxzTenMGxERqTAFv4pSE3cREZHqc9dd0Lgx3Hxz0JWIiKQ0Bb+KUhN3ERGR6vHllzB+PNx0E7RsGXQ1IiIpTcGvojTVU0REpHrccQe0bg2/+13QlYiIpDwFv4pq08Y3jy0qCroSERGRmuu99+Ddd+H22/1UTxERqRIFv4pSE3cREZHEcs6P9mVnw/DhQVcjIlIjKPhVlJq4i4iIJNbEifD5576NQ3p60NWIiNQICn4VpSbuIiIiibNjhx/t69QJhgwJuhoRkRqjbtAFpBxN9RQREUmcsWN9775//cs3bRcRkbjQiF9FNWkCjRppqqeIiEi8bd8Od98NPXrAL38ZdDUiIjWKRvwqyky9/ERERBLh+edh0SJ44w2oo8+mRUTiSf+qVkabNgp+IiIi8bR5M9x7Lxx9NJxxRtDViIjUOBrxq4ysLJg5M+gqREREao6nnoLly+Gll/zsGhERiSuN+FVGeKqnc0FXIiIikvoKC+EPf4BTT4UTTgi6GhGRGknBrzLatIGNG6GoKOhKREREUt+f/wyrV8MDDwRdiYhIjaXgVxlq6SAiIhIfq1bBI4/AwIGQmxt0NSIiNZaCX2WEg59aOoiIiFTNQw/5WTT33Rd0JSIiNVpCg5+ZnWZmC8xsoZndGuX5AWY2x8xmm9l0Mzs21mMD1aaN/6oRPxERkcpbvhz++le4+GLo1i3oakREarSEBT8zSwOeBE4HugGDzazsv+pTgB7OuZ7ApcCzFTg2OJrqKSIiUnX33w8lJXDPPUFXIiJS4yVyxK8PsNA5t9g5tw0YCwyI3ME5t8G5nUtjZgAu1mMD1bgxZGRoqqeIiEhlLVoEzz4Lw4ZBhw5BVyMiUuMlMvi1BZZFPM4LbduFmZ1rZvOB/+FH/WI+NjBmauIuIiJSFSNGQL16cMcdQVciIlIrJDL4Reu+ulvjO+fcOOdcF+AcIHxld0zHApjZsND1gdMLCgoqW2vFhXv5iYiISMV8841v1H7ttaXXzYuISEIlMvjlAe0iHmcD5SYl59xU4AAza1WRY51zo51zuc653NatW1e96lhlZWmqp4iIVEoMi581N7NxoQXQvjSzQ8o8n2Zms8zsjeqrOo7uvBOaNIGbbw66EhGRWiORwW8a0MnMOphZfeBCYELkDmZ2oJlZ6H5voD6wOpZjAxee6umiDkSKiIhEFeMCZrcDs51zhwKXAI+Xef53wLxE15oQX3wB//0v3HQTtGgRdDUiIrVGwoKfc64Y+C0wGX9yesU5N9fMhpvZ8NBu5wHfmNls/ElwkPOiHpuoWislKws2bYLCwqArERGR1BLLAmbd8Ctf45ybD+SY2b4AZpYN/ILQStgp5447oHVr+N3vgq5ERKRWqZvIF3fOTQImldn2dMT9h4CHYj02qUQ2cW/aNNhaREQklURbwOyIMvt8BQwEPjazPkB7/GUPK4HHgJuBxgmvNN6WLIEpU3zT9szMoKsREalVEtrAvUZTE3cREamcWBYwexBoHpoRcw0wCyg2szOBn51zM/b6TYJa/GxPfvzRfz3ssGDrEBGphRI64lejqYm7iIhUzl4XMHPOFQK/BghdC78kdLsQONvMzgDSgSZm9k/n3P+V/SbOudHAaIDc3NzkuCA9vCiaVvIUEal2GvGrrMipniIiIrGLZfGzZqHnAC4HpjrnCp1ztznnsp1zOaHj3osW+pJW+MPS8DlURESqjUb8KqtxY8jI0IifiIhUiHOu2MzCC5ilAWPCi5+Fnn8a6Aq8aGYlwLfAZYEVHE/5+dCwoa6NFxEJgIJfVaiJu4iIVEIMi599BnTay2t8AHyQgPISZ8UKP83Tol3mKCIiiaSpnlWhJu4iIiKxy8/XNE8RkYAo+FVFuIm7iIiI7J2Cn4hIYBT8qiI81dMlx2JpIiIiSS081VNERKqdgl9VZGXB5s1QWBh0JSIiIsmtqMjfNOInIhIIBb+qUBN3ERGR2ISviVfwExEJhIJfVaiJu4iISGzUvF1EJFAKflWhJu4iIiKxUfN2EZFAKfhVhaZ6iojUamZ2ppnpXBoLBT8RkUDpZFUVjRtDZqaCn4hI7XUh8L2ZPWxmXYMuJqmtWAENG0KTJkFXIiJSKyn4VZWauIuI1FrOuf8DegGLgOfN7DMzG2ZmjQMuLfmEe/iZBV2JiEitpOBXVWriLiJSqznnCoHXgLFAG+BcYKaZXRNoYclGzdtFRAKl4FdV4SbuIiJS65jZWWY2DngPqAf0cc6dDvQAbgy0uGSj5u0iIoGqG3QBKS881dM5TV8REal9LgD+7JybGrnRObfJzC4NqKbklJ8PZ5wRdBUiIrWWRvyqqk0b2LwZ1q8PuhIREal+9wBfhh+YWUMzywFwzk0JqqikU1QEGzZoqqeISIAU/KpKTdxFRGqzV4EdEY9LQtskkpq3i4gETsGvqtTEXUSkNqvrnNsWfhC6Xz/AepKTeviJiAROwa+q1MRdRKQ2KzCzs8MPzGwAsCrAepKTgp+ISOC0uEtVKfiJiNRmw4GXzOyvgAHLgEuCLSkJaaqniEjgFPyqqnFjf9NUTxGRWsc5twg40swyAXPOFQVdU1LKz4dGjaBJk6ArERGptWIKfmaWAWx2zu0ws4OALsCbzrntCa0uVaiJu4hIrWVmvwAOBtIt1NbHOXdvoEUlm3DzdrU9EhEJTKzX+E3Fn9DaAlOAXwMvJKqolKMm7iIitZKZPQ0MAq7BT/W8AGgfaFHJSM3bRUQCF2vwM+fcJmAg8Bfn3LlAt8SVlWIU/EREaqujnXOXAGudc78HjgLaBVxT8gmP+ImISGBiDn5mdhRwEfC/0DZdHxjWpo3/NNO5oCsREZHqtSX0dZOZZQHbgQ4B1pOcFPxERAIXa/C7DrgNGOecm2tmHYH393aQmZ1mZgvMbKGZ3Rrl+YvMbE7o9qmZ9Yh4bqmZfW1ms81seox1BiMrC7ZsgXXrgq5ERESq10Qzawb8CZgJLAX+FWRBSaeoCDZu1FRPEZGAxTRq55z7EPgQwMzqAKucc9fu6RgzSwOeBE4B8oBpZjbBOfdtxG5LgBOcc2vN7HRgNHBExPMnOueSvx9S+FPM/Hxo3jzYWkREpFqEzodTnHPrgNfM7A0g3Tm3PtjKkox6+ImIJIWYRvzM7GUzaxJa3fNbYIGZ3bSXw/oAC51zi51z24CxwIDIHZxznzrn1oYefg5kV6z8JBH+FFMtHUREag3n3A7gkYjHWxX6olDwExFJCrFO9ezmnCsEzgEmAfsDF+/lmLb4RrZheaFt5bkMeDPisQPeNrMZZjasvIPMbJiZTTez6QUFBXspKUEiR/xERKQ2edvMzjNTn4JyqXm7iEhSiHWBlnpmVg8f/P7qnNtuZntbySTaSTDqMWZ2Ij74HRux+RjnXL6Z7QO8Y2bznXNTd3tB50bjp4iSm5sbzOoq4ZOZgp+ISG1zA5ABFJvZFvy5zznn1Kk8TCN+IiJJIdYRv7/hL1jPAKaaWXugcC/H5LHrktbZwG7JyMwOBZ4FBjjnVoe3O+fyQ19/Bsbhp44mp8xMaNxYUz1FRGoZ51xj51wd51x951yT0GOFvkj5+ZCR4c+TIiISmFgXd3kCeCJi0w+hUbo9mQZ0MrMOwHLgQuBXkTuY2f7A68DFzrnvIrZnAHWcc0Wh+6cC98ZSa2DUy09EpNYxs+OjbY82Q6XWCjdv12xYEZFAxRT8zKwpcA8QPsF9iA9i5V7E7pwrNrPfApOBNGBMqBXE8NDzTwN3Ay2BUaHLI4qdc7nAvsC40La6wMvOubcq/uNVIwU/EZHaKHKhs3T87JQZwEnBlJOE1MNPRCQpxHqN3xjgG+CXoccXA88DA/d0kHNuEn4xmMhtT0fcvxy4PMpxi4EeZbcntTZt4LPPgq5CRESqkXPurMjHZtYOeDigcpJTfj7k5gZdhYhIrRdr8DvAOXdexOPfm9nsBNSTusIjfs5pOouISO2VBxwSdBFJw7nSqZ4iIhKoWIPfZjM71jn3MYCZHQNsTlxZKSgrC7ZuhbVroUWLoKsREZFqYGZ/oXTF6jpAT+CrwApKNkVFsHGjpnqKiCSBWIPfcODF0LV+AGuBIYkpKUVFNnFX8BMRqS2mR9wvBv7lnPskqGKSjlo5iIgkjVhX9fwK6GFmTUKPC83sOmBOAmtLLZFN3A8+ONhaRESkuvwH2OKcKwEwszQza+Sc2xRwXclBzdtFRJJGrH38AB/4nHPh/n03JKCe1BUZ/EREpLaYAjSMeNwQeDegWpKPRvxERJJGhYJfGVrBJFLkVE8REakt0p1zG8IPQvcbBVhPclHwExFJGlUJfm7vu9QiGRnQpIlG/EREapeNZtY7/MDMDkOLn5VascKfHxs3DroSEZFab4/X+JlZEdEDnrHr1BYBNXEXEal9rgNeNbPwP/5tgEHBlZNk1LxdRCRp7DH4Oef0EV1FtGmjqZ4iIrWIc26amXUBOuM/FJ3vnNsecFnJQ8FPRCRpVGWqp5SlET8RkVrFzK4GMpxz3zjnvgYyzew3QdeVNNS8XUQkaSj4xVM4+Dld/igiUktc4ZxbF37gnFsLXLG3g8zsNDNbYGYLzezWKM83N7NxZjbHzL40s0NC29uZ2ftmNs/M5prZ7+L5w8SVcxrxExFJIgp+8dSmDWzbBmvXBl2JiIhUjzpmtnOVazNLA+rv6YDQPk8CpwPdgMFm1q3MbrcDs51zhwKXAI+HthcD/8851xU4Erg6yrHJobAQNm1S8BMRSRIKfvGkXn4iIrXNZOAVM+tnZicB/wLe3MsxfYCFzrnFzrltwFhgQJl9uuF7BOKcmw/kmNm+zrkVzrmZoe1FwDygbfx+nDhS83YRkaSi4BdPCn4iIrXNLfiAdhVwNTCHva963RZYFvE4j93D21fAQAAz6wO0B7IjdzCzHKAX8EW0b2Jmw8xsuplNLygoiOVniS/18BMRSSoKfvGkJu4iIrWKc24H8DmwGMgF+uFH4fbEomwre3H4g0BzM5sNXAPMwk/z9C9glgm8BlznnCssp7bRzrlc51xu69atY/hp4kzBT0QkqeyxnYNUUDj4acRPRKRGM7ODgAuBwcBq4N8AzrkTYzg8D2gX8Tgb2OXEEQpzvw59LwOWhG6YWT186HvJOfd6lX6QRNJUTxGRpKIRv3jKyICmTRX8RERqvvn40b2znHPHOuf+ApTEeOw0oJOZdTCz+vgAOSFyBzNrFnoO4HJgqnOuMBQCnwPmOecejctPkij5+ZCZCY3VElhEJBko+MWbmriLiNQG5wE/Ae+b2TNm1o/oUzh345wrBn6LXxhmHvCKc26umQ03s+Gh3boCc81sPn71z3DbhmOAi4GTzGx26HZG/H6sOFIrBxGRpKKpnvGmJu4iIjWec24cMM7MMoBzgOuBfc3sKWCcc+7tvRw/CZhUZtvTEfc/AzpFOe5jYgyYgVPzdhGRpKIRv3hT8BMRqTWccxudcy85587EX6s3G9itIXutpBE/EZGkouAXb+Gpnq7sAm0iIlKTOefWOOf+5pw7KehaAuecgp+ISJJR8Iu3rCzYtg3WrAm6EhERkWAUFsLmzZrqKSKSRBT84k1N3EVEpLZTDz8RkaSj4BdvauIuIiK1nYKfiEjSUfCLN434iYhIbafm7SIiSUfBL97CJzkFPxERqa3C50AFPxGRpKHgF2+NGkHTpprqKSIitVd+PjRu7G8iIpIUEhr8zOw0M1tgZgvNbLe+RmZ2kZnNCd0+NbMesR6b1NTLT0REajM1bxcRSToJC35mlgY8CZwOdAMGm1m3MrstAU5wzh0K3AeMrsCxySsrC378MegqREREgqEefiIiSSeRI359gIXOucXOuW3AWGBA5A7OuU+dc2tDDz8HsmM9NqkdfzxMnw7TpgVdiYiISPVT8BMRSTqJDH5tgWURj/NC28pzGfBmRY81s2FmNt3MphcUFFSh3Di67jpo3RpuvhmcC7oaERGR6uOcpnqKiCShRAY/i7ItagoysxPxwe+Wih7rnBvtnMt1zuW2bt26UoXGXZMmcNdd8MEH8NZbQVcjIiJSfdavh82bNeInIpJkEhn88oB2EY+zgd1WPDGzQ4FngQHOudUVOTapXXkldOwIt9wCJSVBVyMiIlI91LxdRCQpJTL4TQM6mVkHM6sPXAhMiNzBzPYHXgcuds59V5Fjk179+vDAA/D11/DSS0FXIyIiUj3UvF1EJCklLPg554qB3wKTgXnAK865uWY23MyGh3a7G2gJjDKz2WY2fU/HJqrWhPnlL+Gww/y0zy1bgq5GREQk8TTiJyKSlOom8sWdc5OASWW2PR1x/3Lg8liPTTl16sDDD0O/fvDkk/D//l/QFYmIiCRWOPhpxE9EJKkktIG7ACedBP37+2mf69YFXY2IiEhi5ef7Rc4yM4OuREREIij4VYeHHvKh78EHg65EREQksdTKQUQkKSn4VYcePeCii+DxxyEvL+hqREREEkfN20VEkpKCX3W57z7YsQPuuSfoSkRERBJHwU9EJCkp+FWXnBy4+mp44QWYm3oLlIqIiOyVc5rqKSKSpBT8qtMdd/iL3W+7LehKRERE4m/dOt++SCN+IiJJR8GvOrVsCbfeChMnwkcfBV2NiIhIfKmHn4hI0lLwq26/+50/Id58s58SIyIiUlOsWOG/aqqniEjSUfCrbo0awe9/D59/DuPHB12NiIhI/GjET0QkaSn4BWHoUOja1V/rV1wcdDUiIiLxEQ5+GvETEUk6Cn5BqFvXN3NfsACeey7oakREROJjxQpo0gQyMoKuREREylDwC8pZZ8Exx8CIEbBxY9DViIiIVJ16+ImIJC0Fv6CYwcMPw08/wZ//HHQ1IiIiVafgJyKStBT8gnT00XDOOT4AFhQEXY2IiEjVqHm7iEjSUvAL2h//6Kd63n9/0JWIiIhUnnMa8RMRSWIKfkHr0gUuuwyeegoWLw66GhERkcpZuxa2blXwExFJUgp+yWDECL/S5513Bl2JiIhI5ah5u4hIUlPwSwZZWXD99fCvf8HMmUFXIyIiUnFq3i4iktQU/JLFzTdDy5Zwyy1BVyIiIlJxCn4iIklNwS9ZNG3qp3q++y68807Q1YiIiFSMpnqKiCQ1Bb9kctVVkJPjR/127Ai6GhERkdjl5/sPMRs1CroSERGJQsEvmTRo4Ns6zJoFY8cGXY2IiEjs1MpBRCSpKfglm8GDoVcvuOMOvyy2iIhIKlDzdhGRpKbgl2zq1IGHHoKlS+Hpp4OuRkREJDYa8RMRSWoKfsnolFPg5JPhvvtg/fqgqxERkTgzs9PMbIGZLTSzW6M839zMxpnZHDP70swOifXYQDin4CcikuQU/JLVQw/B6tXw8MNBVyIiInFkZmnAk8DpQDdgsJl1K7Pb7cBs59yhwCXA4xU4tvqtXQvbtmmqp4hIElPwS1a9e/vr/R59FD77LOhqREQkfvoAC51zi51z24CxwIAy+3QDpgA45+YDOWa2b4zHVj/18BMRSXoJDX4xTGXpYmafmdlWM7uxzHNLzexrM5ttZtMTWWfS+vOfITsbfvEL+OaboKsREZH4aAssi3icF9oW6StgIICZ9QHaA9kxHlv9FPxERJJewoJfjNNR1gDXAiPLeZkTnXM9nXO5iaozqe27L7z9NjRsCKeeCkuWBF2RiIhUnUXZ5so8fhBobmazgWuAWUBxjMf6b2I2zMymm9n0goKCKpQbAzVvFxFJeokc8dvrdBTn3M/OuWnA9gTWkdo6dIDJk2HLFr/oy8qVQVckIiJVkwe0i3icDeRH7uCcK3TO/do51xN/jV9rYEksx0a8xmjnXK5zLrd169ZxLD+K8Iifgp+ISNJKZPCr6nQUB7xtZjPMbFhcK0s1hxwC//uf/0S1f39Yty7oikREpPKmAZ3MrIOZ1QcuBCZE7mBmzULPAVwOTHXOFcZybCDy86FZM2jUKOhKRESkHIkMfjFPRynHMc653vipoleb2fFRv0l1TmUJ0lFHweuvw7ffwtlnw+bNQVckIiKV4JwrBn4LTAbmAa845+aa2XAzGx7arSsw18zm48+Dv9vTsdX9M+xGzdtFRJJe3QS+dszTUaJxzuWHvv5sZuPwU0enRtlvNDAaIDc3tyLBMvX07w//+Idf7fOXv/RBsF69oKsSEZEKcs5NAiaV2fZ0xP3PgE6xHhs49fATEUl6iRzxq/R0FDPLMLPG4fvAqYCWtQQYNAiefBLeeAMuvRR27Ai6IhERqe0U/EREkl7CRvycc8VmFp6OkgaMCU9lCT3/tJntB0wHmgA7zOw6/AqgrYBxZhau8WXn3FuJqjXlXHWVb+5+113QsqVv+2DRZtaKiIgkmHOa6ikikgISOdUzlqksP+GngJZVCPRIZG0p7447YNUqePxxaN3aPxYREalua9bAtm0a8RMRSXIJDX6pxDl49VXYbz84PuoyMknGDB591J9w77wTWrTwI4EiIiLVSc3bRURSgoJfyPbtcOut0LgxzJwJaWlBVxSDOnXguedg7Vq4+mof/gYNCroqERGpTdS8XUQkJSRycZeUUr8+/PGPMGeOXzgzZdSrB6+8AsceCxdf7Ju9i4iIVBeN+ImIpAQFvwi//CX06eNnTm7aFHQ1FdCwIUycCN26wcCB8PnnQVckIiK1RTj4acRPRCSpKfhFMIORI2H5cnjssaCrqaCmTf1oX1YWnHEGzA2+n6+IiNQCK1ZAs2b+Q0gREUlaCn5lHHccDBgADz4IP/8cdDUVtO++8PbbkJ4Op54KS5cGXZGIiNR06uEnIpISFPyieOghP9Xz3nuDrqQSOnTw4W/zZjjlFFi5MuiKRESkJlPwExFJCQp+UXTuDMOGwd/+Bt99F3Q1lXDIIfC///mT8Wmnwfr1QVckIiI1lZq3i4ikBAW/ctxzj58xeeutQVdSSUcdBa+/7q/1O/tsPwIoIiIST85pxE9EJEUo+JVj333hlltg3Dj4+OOgq6mk/v19b4qPPvL9/bZuDboiERGpSVav9o1wFfxERJKegt8e3HCDP5fddJP/UDMlDRoETz7p2z0cd5wWfBERkfhR83YRkZSh4LcHjRrBfff5tnj/+U/Q1VTBVVfBa6/BggXQuze88UbQFYmISE2g5u0iIilDwW8vhgyB7t3htttg27agq6mCgQNh5kzIyYGzzvIXLxYXB12ViIikMgU/EZGUoeC3F2lp8PDDsGgRPPVU0NVU0QEHwKefwpVX+p4VJ51UetIWERGpKE31FBFJGQp+MejfH04+2ff1W7cu6GqqKD0dnn7aL/oyYwb06gVTpgRdlYiIpKL8fGje3J9bREQkqSn4xcAM/vQnWLsW/vjHoKuJk//7P5g2DVq29I3e77sPduwIuioREUklauUgIpIyFPxi1LMnXHwxPP44/PBD0NXESbdu8OWX8Ktfwd13w+mnQ0FB0FWJiEiqUPN2EZGUoeBXAfff70f/7rwz6EriKDPTT/v829/gww/91M9PPw26KhERSQUa8RMRSRkKfhXQrh1cdx38859+gcwawwyGDYPPPvPXaZxwAjzySAo3LxQRkYRzzo/4KfiJiKQEBb8KuvVWf1lcSjd1L0+vXn7Bl7POghtv9C0gUn41GxERSYjVq2H7dk31FBFJEQp+FdS0qb8c7r334M03g64mAZo29c3e//xn3+i9d28fBkVERCKph5+ISEpR8KuE4cPhwAPh5ptraA90Mz+ndepU/2nu0Uf7FhA1bohTREQqTcFPRCSlKPhVQv36vq3D3LnwwgtBV5NARx0Fs2ZBv35w1VW+BcSGDUFXJSIiyUDN20VEUoqCXyWdd57PRXffDRs3Bl1NArVq5ad83n8/jB0Lhx8O776r0T8RkdouPOKn4CcikhIU/CrJDEaO9B94PvJI0NUkWJ06cMcdPvAVFvqG7927w+jRsGlT0NWJiEgQ8vOhRQu/GrSIiCQ9Bb8qOPpov/Dlww/DTz8FXU01OPFEWLTIz2+tXx+uvNL3uLj1Vli2LOjqRESkOql5u4hISlHwq6IHH4StW2HEiKArqSbp6TBkiF/p88MPoW9f+NOfoEMHGDTIN3/XNFARkZpPzdtFRFKKgl8VderkV/l89lmYNy/oaqqRGRx/vG/9sGgRXH89TJ4MxxwDffrASy/Btm1BVykiIomi4CciklISGvzM7DQzW2BmC83s1ijPdzGzz8xsq5ndWJFjk8ndd0NGBtxyS9CVBCQnx4/65eXBk09CUZFfATQnB+67D37+OegKRUQknnbs8Nc4aKqniEjKSFjwM7M04EngdKAbMNjMupXZbQ1wLTCyEscmjdat/WVuEyf62Y+1VmYm/OY38O23vrv9oYf6VLz//vDrX8Ps2UFXKCIi8bB6te/zqhE/EZGUkcgRvz7AQufcYufcNmAsMCByB+fcz865acD2ih6bbK67DrKz4cYb/QehtVqdOnDaafDWW37+66WXwiuvQK9e/prAceOgpCToKkVEpLLUvF1EJOUkMvi1BSKXeswLbYvrsWY2zMymm9n0goKCShUaDw0b+lZ306fDv/8dWBnJp0sXGDXKTwP9059gyRK/FOqBB8IDD8DChUFXKCIiFaXm7SIiKSeRwc+ibIt1uceYj3XOjXbO5Trnclu3bh1zcYnwf/8HPXrA7bf7lT4lQvPmfjh00SL4z3/89M877/Sr4xx2GDz0kA+FIiKS/DTiJyKSchIZ/PKAdhGPs4H8ajg2MGlpflBr6VJ47LGgq0lSdevCeef5iyF/+AEeecRvu/VW6NgRjjjCb1NfQBGR5BUOfhrxExFJGYkMftOATmbWwczqAxcCE6rh2ECdcgr84hc+x1x8MaxcGXRFSWz//eGGG+CLL2DxYj/qV1zsRwb339+3hnj88dL/YIiISHJYsQJatIAGDYKuREREYpSw4OecKwZ+C0wG5gGvOOfmmtlwMxsOYGb7mVkecANwp5nlmVmT8o5NVK3x9uqrcMcd/lq/zp39JW5ay2QvOnSAm2/2jeG//95f/7dhQ+mqOSec4N9IJWkRkeCph5+ISMox52K97C755ebmuunTpwddxk4LFvjuBu+9B7m58NRT/qtUwPz5fkXQf//bt4moU8evDDpokF8kplWroCsUkQCY2QznnP5FjVHcz49HHAHNmsHkyfF7TRERiYvyzpEJbeBe23XuDO++Cy+/7Be17NMHrr4a1q0LurIU0qWL7wU4dy58/bUfSs3LgyuvhP32g/794YUX/OigiIhUjxUrdH2fiEiKUfBLMDMYPNgPXP32t/D00z4Q/vOfUIMGW6vHIYfAvff6N3PWLD81dOFC3xy+TRu44gr47DO9sSIiibRjhw9+muopIpJSFPyqSdOm8MQTMG0a5OT4hV/69fP9zaWCzKBnT/jDH3zw+/hjuOAC+Ne/4Oij4eCDYeRIXQ8oIknLzE4zswVmttDMbo3yfFMzm2hmX5nZXDP7dcRz14e2fWNm/zKz9GotftUqvxCXgp+ISEpR8KtmvXvDp5/66/1mzSrt+7dpU9CVpSgzv/rnmDH+E+hnn/U9A2+6yS8Kc+65MHGi/0+KiEgSMLM04EngdKAbMNjMupXZ7WrgW+dcD6Av8IiZ1TeztsC1QK5z7hAgDb/ydfVR83YRkZSk4BeAtDQYPtwv/vKrX8Ef/wjduvl8IlXQuDFcdhl88olfCOb6633KPvtsaNfO99j47rugqxQR6QMsdM4tds5tA8YCA8rs44DGZmZAJrAGCH+CVRdoaGZ1gUZUd59bNW8XEUlJCn4B2mcfvy7Jhx9CRobPJwMG+L7mUkVdu8LDD/uFYMaPh8MP99M/O3eG447TgjAiEqS2wLKIx3mhbZH+CnTFh7qvgd8553Y455YDI4EfgRXAeufc29G+iZkNM7PpZja9oKAgftUr+ImIpCQFvyRw/PF+2udDD/lVQLt2hQcfhG3bgq6sBqhXz6fpCRNg2TL/Jv/8c+mCMJdfrgVhRKS6WZRtZf8R6g/MBrKAnsBfzayJmTXHjw52CD2XYWb/F+2bOOdGO+dynXO5rVu3jlftpcFvv/3i95oiIpJwCn5Jon59v0jlt9/6DgW33ebXL3nrLQXAuGnTxr/J8+eXLggzdmzpgjD33ut7Bk6bBgUFCoMikih5QLuIx9nsPl3z18DrzlsILAG6ACcDS5xzBc657cDrwNHVUHOpFSugZUto0KBav62IiFRN3aALkF21bw/jxsEbb8A118Dpp0PDhr5X7nHH+dtRR0FmZtCVprDwgjDHHAOPP+7D3pgxcM89u+6XkeGXYM3JgQ4ddr/fvLl/LRGRipkGdDKzDsBy/OIsvyqzz49AP+AjM9sX6Awsxo8WHmlmjYDNoX3i2Jk9Bvn5muYpIpKCFPyS1JlnwkknwZtvwkcf+dsDD/j2SWlp0KtXaRA89liI5yyeWiW8IMxll0FhISxdWnpbsqT060cf+efLHhstEB5wgF+tJy2tmn8YEUkFzrliM/stMBm/KucY59xcMxseev5p4D7gBTP7Gh/2bnHOrQJWmdl/gJn4xV5mAaOr9QfIz9eKniIiKchcDZrOlpub66ZPr94PPqtTYaG/HC0cBL/4ArZu9c+F1ywJ33JyNBgVd+vWlYbByGAYvh+5WEyLFr5R4ymn+FtOThAVi9RYZjbDOZcbdB2pIq7nx3bt4OST4fnn4/N6IiISV+WdIzXil0KaNPHX//Xv7x9v3QozZpQGwf/8x7exA2jbdtcgePDBUEdXdFZNs2Z+qLVXr92fcw7WrPEhcN48mDIF3nkHXn3VP9+pkw+Ap54KJ57o/zBFRFLNjh3+Gj9N9RQRSTkKfimsQQO/LsnRR8Mtt/jz8TfflAbBqVP92iXgM8uhh0L37qW3Qw5R/ogbM7/YQcuWcNhh8H//58PgvHk+AL7zDvz97zBqlJ8CesQRPgSecgr06QN19VdRRFJAQQGUlGiqp4hU2Pbt28nLy2PLli1Bl1JjpKenk52dTb169WLaX1M9azDnSi9P++wzmDPHB8OiotJ92rffNQx27+6njcb4+yMVsW2b/4N4+20fBKdP939ITZr4CzrDI4IHHKB5uiJ7oameFRO38+Ps2X7Ww2uvwcCBVX89Eak1lixZQuPGjWnZsiWm/+dUmXOO1atXU1RURIcOHXZ5TlM9ayEz6NjR34YM8duc8w3iv/5619tbb0Fxsd+nXj3o0mX3QNiunfJIldSvDyec4G8PPACrV8N77/kQ+PbbvtE8+OsBw9cG9uzp03n9+gEWLiISoubtIlJJW7ZsIScnR6EvTsyMli1bUlBQEPMxCn61jFnpIpRnnVW6fetWWLBg1zD40Ufw8sul+zRt6qeHdugA2dm731q31nWEFdKype8leMEFPpEvXFgaAv/9b3jmGb9fnTo+dR9wgL917Fh6/4AD/B+MiEh1CAc/TfUUkUpQ6Iuvir6fCn4C+OsFDz3U3yKtW+enh4bDYPgawuXLS0cIw+rV84vKRAuF4dt++6nLQVRmfgGYTp3gN7+B7dth5kyfxhctKr2NH++vsYnUosWuQTAyHGZlKY2LSPysWOG/7rdfsHWIiFTQ6tWr6devHwA//fQTaWlptA71Q/vyyy+pv4fZVdOnT+fFF1/kiSee2OP3OProo/n000/jV3ScKfjJHjVr5vsEHnvsrtt37ICff4a8vOi3adN8I/pwu4mwtDT/QXF2dmnLuwMOgAMP9F/btNF0UsCn6COO8Leyiopg8eJdA+GiRfDll34V0ZKS0n3T0/0QbffucNRR/tarl6aOikjl5OdDq1b+00IRkRTSsmVLZs+eDcCIESPIzMzkxhtv3Pl8cXExdctZbC83N5fc3L1fVp7MoQ8U/KSS6tTxH/jutx+U9/fAOX8ZW2QgXL7cf122LHpOadhw98GrcChs316LXwK+cXyPHv5W1vbt8OOPu4fCzz+HV17x+zRo4FceDQfBo47S9ToiEhs1bxeRGmTo0KG0aNGCWbNm0bt3bwYNGsR1113H5s2badiwIc8//zydO3fmgw8+YOTIkbzxxhuMGDGCH3/8kcWLF/Pjjz9y3XXXce211wKQmZnJhg0b+OCDDxgxYgStWrXim2++4bDDDuOf//wnZsakSZO44YYbaNWqFb1792bx4sW88cYb1fLz6r/RkjBm/oPhVq38GiXRbN/uF5sJ55OFC0u/Tp4MkSv+pqVFHyVs2RI2b4ZNm/zXsrfytpd9Li3ND4z17Fl6S7nZTPXqlb5BZS1f7lcVDd/+8hd45BH/3P777xoEe/bUqKCI7E49/EQkHq67zq8SHE89e8Jjj1X4sO+++453332XtLQ0CgsLmTp1KnXr1uXdd9/l9ttv57XXXtvtmPnz5/P+++9TVFRE586dueqqq3ZrqTBr1izmzp1LVlYWxxxzDJ988gm5ublceeWVTJ06lQ4dOjB48OBK/rCVo+AngapXzwe4Aw/c/blwn+DIQBi+/+WX/vrDWNSt60cSw7dGjUrvZ2T4YNqwoZ+W+uWXfl2VsH333TUI9uzpL8NLyesU27aF88/3N/A/8KxZpUHwk09Kf/j09N1HBZPsU37nfJbdd1+1HxGpNvn5fpUvEZEa4oILLiAt9B+79evXM2TIEL7//nvMjO3bt0c95he/+AUNGjSgQYMG7LPPPqxcuZLs7Oxd9unTp8/ObT179mTp0qVkZmbSsWPHne0XBg8ezOjRoxP40+1KwU+SVp06Pqu0bQvHH7/782vW+BBYWLhrsCsb8Co6PXTdOt/zcPbs0tujj/rRSfCvWXZksHt3yMyswg8bhAYN4Mgj/e366/22vLxdRwWfeAJGjvTPtW/vrw/cf3+/ymj4a7t2PhRWQxpetcovfDp5sl/8dMUKPzB58MF+5mvPnqWzYJs3T3g5FBf7yy3nzYP58/0tLc1fVhlupdKxo/9wQdeuSsorKYGffkq6D4FEJAVVYmQuUTIyMnbev+uuuzjxxBMZN24cS5cupW/fvlGPaRBxnXNaWhrFZVc8LGefoPunK/hJymrRAvr0if/rNmvmg2Zk2Ny2zf+nPjIMvvoqhD+kCS/KGQ6B++3ng0eLFrveGjWKbwDYscOHoZUrd7399NOuj7du9YGkbt1dv+5+P5u0tAtIS7uAuu0grV0JaevXUHfNz6StWkmrj5bQc+Mn9NryLJ34njR2+ELS0nxCDwfBssGwXbtKpZ/iYn954uTJ/hbued+8uW9zeMwx/nrRr76CSZPghRdKj23XrjQEhm8HHli5RU43bCgNdvPnlwa9778v/UAA/P+HnfPvf6TMTB8AywbCjh399OX09IrXJFLtVq3y4U9TPUWkhlq/fj1t27YF4IXI/1TESZcuXVi8eDFLly4lJyeHf0dOM6sGCn4iMahfv7TdxSWX+G3O+dARGQanTStdQ6W812nRInooLLu9WTM/mrmnQFdQsOviOJHfZ7/9/DTI7GwfLEpK/K24ePevW7aU93waJSWtKS5uTUnJwRQUwbZtlwGQ0bCEHjnr6bVPPr0zv6MXszi46HPqT5sGr7/u03Kk9PRdg2C0gNikCT/84EPeW2/BlCn+PahTxw9MjhgB/fv7BYWiDTD+9JMPgbNn+69ffQVvvln6HmVk+GAeGQa7d/fr5YQDW9lwN2+eHwgNS0vzAbJLFzj7bP+1a1fo3Lm0peLGjbB0KSxZ4kcEw7dwq8ZNm3atOytr1zDYoYP/swsH8jp1dr1VZFvDhrDPPhpxlDhQ83YRqeFuvvlmhgwZwqOPPspJJ50U99dv2LAho0aN4rTTTqNVq1b0ScQIxh5Y0EOO8ZSbm+umT58edBlSy23a5Kehlndbuzb69g0b9vy6DRr4MLDvvqWhLvIWua1p08T8R3/7dh+EZs70lwfOnOlDVrj2evX85T+9ejp6dyqi1z7L6dHwOzIKlvqUvGyZX3V02TI/T3PHDjbRkA/oy2T6M7nO6SzYcRAA7TLW0L/LD/Q/cj39TqlD825tfDis4PDYli0wd25pEAwHw/XrS/fJyfF/LpHbMjN9qAsHu/DXAw6o2ro3zvlWKJGBcPHi0pCYl+f3iafmzf102MjbIYf4QJiqzGyGc27va2sLEKfz4//+B2ee6aeBH3lkfAoTkVpj3rx5dO3aNegyArdhwwYyMzNxznH11VfTqVMnrg9fclMJ0d7X8s6RGvETibNGjfytzDW+e7Vtm7++MDIgNm5cGuqaNAl+1KZevdKRz6FD/bYdO/yiO5FhcMJEY8yqJkATzLrSubO/PLBXLvQe5kc0p7xdwuQ3tvHRlw3Yuq0O6XW3c0LW91zZ5Dn6l7xJ14Kp2IwCmAE8GVFE69a7jhaG50926OBvjRvvUnN4nZrDDivd5pzPn+EgOHeuryky4LVtm5j326w0oB911O7Pb93qV7otKPDvbdlbScnet0U+Liz0Yf2bb/xo9Nq1pd+rVatdg2D4fsuW8f+5pQYIN2/XiJ+ISKU988wz/P3vf2fbtm306tWLK6+8stq+t0b8RCTunPMjV7NmlYbBWbP8QF+kbt381M3+/f01lQ0blnmhLVtKGz+GRwojbz/+6JNNpNatd7+QLhwOs7NTdEnW+AhPZ5071wfBuXNL7xcVle637767h8Fu3apnwZxYacSvYuJyfrz3XrjnHv/phNq9iEgFacQvMZJmxM/MTgMeB9KAZ51zD5Z53kLPnwFsAoY652aGnlsKFAElQLFO8CKpw6x0UO7ss0u3r1rlA+DPP/ug167dXl4oPb38fh/gk8zatbvPm1y8GL74wg9xRV4EWa+eX500Wihs29YPgdXgYGjmF6Bp0wZOPrl0ezioRwbBuXPhuef89YphGRn+2P32K32daLeWLYMfnZYEyM/3f0cU+kREUlLCgp+ZpeEnaJ0C5AHTzGyCc+7biN1OBzqFbkcAT4W+hp3onFuVqBpFpHq1auVX44wbs9LVcHKjfDZUXOxHBqMFwxkzYPXqXfevU8ePGIYvmNxvv/Lvt2hRY9JNZFA/7bTS7Tt2+EHVb77xC93k5/vZfitW+Cmyb72160hhWL16/m0qLxh27+6vq5QUo+btIiIpLZEjfn2Ahc65xQBmNhYYAEQGvwHAi87PN/3czJqZWRvn3IoE1iUitUXduqXX/vXrt/vz69eXrqqyYkXpsqk//VS6xOdPP+2+Qmn4tSNX1Ql/jegHVK5YptjXqeNHPCtza9AgLiOXder4gJaT49f0iGbjxtIwGH4LIx8vWQKffupHe8PuvRfuuqvK5Ul1y89X8BMRSWGJDH5tgcgrevLYdTSvvH3aAisAB7xtZg74m3Mualt7MxsGDAPYf//941O5iNQOTZv65os9e5a/j3M+IIbDYGRfjfC2n37yS4X+/LMfZUwG9er5ENi0qe9b0bu3X2Gnd2+/KE6cRiszMvY8Gzds2zb/9qxY4fOxpKD8fD9cKyIiKSmRwS/a/yrKfsy9p32Occ7lm9k+wDtmNt85N3W3nX0gHA3+4vWqFCwishsz31SxWTO/5OeehJfUjPV196S42C+isXWrX+SmsrdVq3wofeut0tpatPABMDIMVra7fYzq1/dr61R0tVtJEiUl/sMOjfiJSIrq27cvt912G/3799+57bHHHuO7775j1KhRUfcfOXIkubm5nHHGGbz88ss0a9Zsl31GjBhBZmYmN954Y7nfd/z48Rx00EF069YNgLvvvpvjjz+ekyMvtq8miQx+eUDk0g3ZQH6s+zjnwl9/NrNx+KmjuwU/EZGkEe6cHg9161a4Z+Eebd4MX3/tl1gN3x57rHQaa2ZmaQgMB8KuXX0dIgUFPvwp+IlIiho8eDBjx47dJfiNHTuWP/3pT3s9dtKkSZX+vuPHj+fMM8/cGfzuvffeSr9WVSXu412YBnQysw5mVh+4EJhQZp8JwCXmHQmsd86tMLMMM2sMYGYZwKnANwmsVUSkZmvYEPr0geHDYfRomD7dr8wyezaMGeMbM5aUwDPPwJAhvllj48ZwxBFw1VXw9NMwdiyMHw+TJ8OHH/qVU+fMge++84voFBT410yW6a4SP/mhz23btAm2DhGRSjr//PN544032Lp1KwBLly4lPz+fl19+mdzcXA4++GDuueeeqMfm5OSwKnSx+gMPPEDnzp05+eSTWbBgwc59nnnmGQ4//HB69OjBeeedx6ZNm/j000+ZMGECN910Ez179mTRokUMHTqU//znPwBMmTKFXr160b17dy699NKdteXk5HDPPffQu3dvunfvzvz58+PyHiTso1znXLGZ/RaYjG/nMMY5N9fMhoeefxqYhG/lsBDfzuHXocP3Bcb5bg/UBV52zr2VqFpFRGql+vX99X89esCvQ//8lpT4IBduvjhzJvzrXz74VURamg+b6emlX8P3hw8v/X6SGtS8XUTi6Lrr/OeO8dSzp5/IUp6WLVvSp08f3nrrLQYMGMDYsWMZNGgQt912Gy1atKCkpIR+/foxZ84cDj300KivMWPGDMaOHcusWbMoLi6md+/eHHbYYQAMHDiQK664AoA777yT5557jmuuuYazzz6bM888k/PPP3+X19qyZQtDhw5lypQpHHTQQVxyySU89dRTXHfddQC0atWKmTNnMmrUKEaOHMmzzz5b1bcosX38nHOT8OEuctvTEfcdcHWU4xYDPRJZm4iIRJGW5qd4du0KF13ktzkHy5fDhg1+ymj4+sHw/fK+lvdcgwbB/oxScRkZfmXcvTbfFBFJXuHpnuHgN2bMGF555RVGjx5NcXExK1as4Ntvvy03+H300Uece+65NGrUCICzI5oVf/PNN9x5552sW7eODRs27DKlNJoFCxbQoUMHDjroIACGDBnCk08+uTP4DRw4EIDDDjuM119/vao/OpDg4CciIjWAmVZlqe369vU3EZE42NPIXCKdc8453HDDDcycOZPNmzfTvHlzRo4cybRp02jevDlDhw5ly5Yte3wNK2dxtqFDhzJ+/Hh69OjBCy+8wAcffLDH13F7ae3UIPQhaVpaGsVxuoQikdf4iYiIiIiIJIXMzEz69u3LpZdeyuDBgyksLCQjI4OmTZuycuVK3nzzzT0ef/zxxzNu3Dg2b95MUVEREydO3PlcUVERbdq0Yfv27bz00ks7tzdu3JiioqLdXqtLly4sXbqUhQsXAvCPf/yDE044IU4/aXQa8RMRERERkVph8ODBDBw4kLFjx9KlSxd69erFwQcfTMeOHTnmmGP2eGzv3r0ZNGgQPXv2pH379hx33HE7n7vvvvs44ogjaN++Pd27d98Z9i688EKuuOIKnnjiiZ2LugCkp6fz/PPPc8EFF1BcXMzhhx/O8OHDE/NDh9jehhlTSW5urps+fXrQZYiISIKZ2QznXG7QdaQKnR9FJGjz5s2ja9euQZdR40R7X8s7R2qqp4iIiIiISA2n4CciIiIiIlLDKfiJiIiIiIjUcAp+IiIiIiKScDVpbZFkUNH3U8FPREREREQSKj09ndWrVyv8xYlzjtWrV5Oenh7zMWrnICIiIiIiCZWdnU1eXh4FBQVBl1JjpKenk52dHfP+Cn4iIiIiIpJQ9erVo0OHDkGXUatpqqeIiIiIiEgNp+AnIiIiIiJSwyn4iYiIiIiI1HBWk1bWMbMC4IcqvkwrYFUcyqlOqVgzpGbdqVgzpGbdqVgzpGbdqVhze+dc66CLSBW1+PwIqVl3KtYMqVl3KtYMqVl3KtYMqVl31HNkjQp+8WBm051zuUHXURGpWDOkZt2pWDOkZt2pWDOkZt2pWLNUv1T9PUnFulOxZkjNulOxZkjNulOxZkjduqPRVE8REREREZEaTsFPRERERESkhlPw293ooAuohFSsGVKz7lSsGVKz7lSsGVKz7lSsWapfqv6epGLdqVgzpGbdqVgzpGbdqVgzpG7du9E1fiIiIiIiIjWcRvxERERERERquFoZ/MzsNDNbYGYLzezWKM+bmT0Ren6OmfUOos4yNbUzs/fNbJ6ZzTWz30XZp6+ZrTez2aHb3UHUWpaZLTWzr0M1TY/yfFK932bWOeI9nG1mhWZ2XZl9kuK9NrMxZvazmX0Tsa2Fmb1jZt+HvjYv59g9/j2o5pr/ZGbzQ3/+48ysWTnH7vF3KZHKqXuEmS2P+D04o5xjk+m9/ndEvUvNbHY5xwb2XkuwdI6sPql2fgzVlBLnyFQ8P4a+d8qdI1Px/Bj63rXvHOmcq1U3IA1YBHQE6gNfAd3K7HMG8CZgwJHAF0lQdxugd+h+Y+C7KHX3Bd4IutYotS8FWu3h+aR7v8v8vvyE74eSdO81cDzQG/gmYtvDwK2h+7cCD5Xzc+3x70E113wqUDd0/6FoNcfyuxRA3SOAG2P4HUqa97rM848Adyfbe61bcDedI6u97pQ9P0b8viTlOTIVz497qDupz5GpeH4sr+4yz9e4c2RtHPHrAyx0zi12zm0DxgIDyuwzAHjReZ8DzcysTXUXGsk5t8I5NzN0vwiYB7QNsqY4Srr3O0I/YJFzrqqNjxPCOTcVWFNm8wDg76H7fwfOiXJoLH8PEiJazc65t51zxaGHnwPZ1VFLRZTzXsciqd7rMDMz4JfAv6qjFkkZOkcml6R7r8tI2nNkKp4fITXPkal4foTaeY6sjcGvLbAs4nEeu58cYtknMGaWA/QCvojy9FFm9pWZvWlmB1dvZeVywNtmNsPMhkV5Ppnf7wsp/y99Mr7XAPs651aA/88QsE+UfZL5Pb8U/wl3NHv7XQrCb0PTb8aUM20oWd/r44CVzrnvy3k+Gd9rSTydI6tXKp8fIfXOkal+foTUOkem6vkRaug5sjYGP4uyrezSprHsEwgzywReA65zzhWWeXomfrpFD+AvwPhqLq88xzjnegOnA1eb2fFlnk/K99vM6gNnA69GeTpZ3+tYJet7fgdQDLxUzi57+12qbk8BBwA9gRX4aSFlJeV7DQxmz59kJtt7LdVD58jqlZLnR6jR58hkfs9T6RyZyudHqKHnyNoY/PKAdhGPs4H8SuxT7cysHv6E9pJz7vWyzzvnCp1zG0L3JwH1zKxVNZe5G+dcfujrz8A4/NB+pKR8v/F/mWc651aWfSJZ3+uQleGpQKGvP0fZJ+neczMbApwJXOSci/oPfwy/S9XKObfSOVfinNsBPFNOPcn4XtcFBgL/Lm+fZHuvpdroHFmNUvj8CKl5jkzJ8yOk3jkyVc+PULPPkbUx+E0DOplZh9CnVRcCE8rsMwG4xLwjgfXhqQFBCc01fg6Y55x7tJx99gvth5n1wf/5rq6+KqPWlGFmjcP38Rcof1Nmt6R7v0PK/bQnGd/rCBOAIaH7Q4D/Rtknlr8H1cbMTgNuAc52zm0qZ59YfpeqVZlrbc4lej1J9V6HnAzMd87lRXsyGd9rqTY6R1aTFD8/QmqeI1Pu/AipeY5M4fMj1ORzZHmrvtTkG36VrO/wKwndEdo2HBgeum/Ak6HnvwZyk6DmY/HD33OA2aHbGWXq/i0wF78q0ufA0UlQd8dQPV+FakuV97sR/iTVNGJb0r3X+JPuCmA7/pOzy4CWwBTg+9DXFqF9s4BJEcfu9vcgwJoX4uf5h3+3ny5bc3m/SwHX/Y/Q7+wc/MmqTbK/16HtL4R/lyP2TZr3Wrdgb9F+X1Pg3+yUO0eW9/cs2d/rUF1Jf44s59/spD4/7qHupD5HllNzUp8fy6s7tP0Faug50kI/gIiIiIiIiNRQtXGqp4iIiIiISK2i4CciIiIiIlLDKfiJiIiIiIjUcAp+IiIiIiIiNZyCn4iIiIiISA2n4CdSzcysxMxmR9xujeNr55hZzL1kQr1o3gnd/zjUtFRERKTa6fwoklj6JRapfpudcz2DLiLkKOBzM2sObHTOFQddkIiI1Fo6P4okkEb8RJKEmS01s4fM7MvQ7cDQ9vZmNsXM5oS+7h/avq+ZjTOzr0K3o0MvlWZmz5jZXDN728waRvleB5jZbOCfwK+AGUCP0Ces+1TPTywiIrJ3Oj+KxIeCn0j1a1hmKsugiOcKnXN9gL8Cj4W2/RV40Tl3KPAS8ERo+xPAh865HkBvYG5oeyfgSefcwcA64LyyBTjnFoU+VZ0B9AFeBC5zzvV0zv0cvx9VREQkZjo/iiSQOeeCrkGkVjGzDc65zCjblwInOecWm1k94CfnXEszWwW0cc5tD21f4ZxrZWYFQLZzbmvEa+QA7zjnOoUe3wLUc87dX04t05xzh5vZa8C1zrnl8f55RUREYqHzo0hiacRPJLm4cu6Xt080WyPulxDlWl4zezp0kXun0JSW04D/mdn1FahVRESkuuj8KFJFCn4iyWVQxNfPQvc/BS4M3b8I+Dh0fwpwFYCZpZlZk1i/iXNuOPB74D7gHOB/oWksf65S9SIiIomh86NIFWlVT5Hq1zD0KWLYW8658JLVDczsC/yHMoND264FxpjZTUAB8OvQ9t8Bo83sMvwnl1cBKypQxwn4axeOAz6szA8iIiISRzo/iiSQrvETSRKhaxhynXOrgq5FREQkWej8KBIfmuopIiIiIiJSw2nET0REREREpIbTiJ+IiIiIiEgNp+AnIiIiIiJSwyn4iYiIiIiI1HAKfiIiIiIiIjWcgp+IiIiIiEgNp+AnIiIiIiJSw/1/gjHapCahRdsAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1080x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Delete all previous models to free memory\n",
    "tf.keras.backend.clear_session()\n",
    "\n",
    "# Sequential model\n",
    "model = tf.keras.models.Sequential()\n",
    "\n",
    "# Input layer representing the 784 pixels\n",
    "model.add(tf.keras.layers.Input(shape=(784,)))\n",
    "\n",
    "# Hidden layer with 150 relu neurons, BN and dropout\n",
    "model.add(tf.keras.layers.Dense(units=150))\n",
    "model.add(tf.keras.layers.BatchNormalization()) \n",
    "model.add(tf.keras.layers.Activation('relu'))\n",
    "model.add(tf.keras.layers.Dropout(0.3)) \n",
    "\n",
    "\n",
    "# Second hidden layer with 100 relu neurons, BN and dropout\n",
    "model.add(tf.keras.layers.Dense(100))\n",
    "model.add(tf.keras.layers.BatchNormalization())\n",
    "model.add(tf.keras.layers.Activation('relu'))\n",
    "model.add(tf.keras.layers.Dropout(0.3))\n",
    "\n",
    "# Softmax output layer over 10 classes\n",
    "model.add(tf.keras.layers.Dense(10))\n",
    "model.add(tf.keras.layers.Activation('softmax'))\n",
    "\n",
    "# Learning rule\n",
    "optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)\n",
    "\n",
    "# Loss function\n",
    "model.compile(\n",
    "    loss='categorical_crossentropy', # loss function\n",
    "    optimizer=optimizer, # learning rule\n",
    "    metrics=['accuracy'] # show accuracy\n",
    ")\n",
    "\n",
    "print(model.summary())\n",
    "\n",
    "# Training\n",
    "history = tf.keras.callbacks.History()\n",
    "model.fit(\n",
    "    X_train, T_train,\n",
    "    batch_size=128, \n",
    "    epochs=20,\n",
    "    validation_split=0.1,\n",
    "    callbacks=[history]\n",
    ")\n",
    "\n",
    "# Testing\n",
    "score = model.evaluate(X_test, T_test, verbose=0)\n",
    "print('Test loss:', score[0])\n",
    "print('Test accuracy:', score[1])\n",
    "\n",
    "plt.figure(figsize=(15, 6))\n",
    "plt.subplot(121)\n",
    "plt.plot(history.history['loss'], '-r', label=\"Training\")\n",
    "plt.plot(history.history['val_loss'], '-b', label=\"Validation\")\n",
    "plt.xlabel('Epoch #')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.subplot(122)\n",
    "plt.plot(history.history['accuracy'], '-r', label=\"Training\")\n",
    "plt.plot(history.history['val_accuracy'], '-b', label=\"Validation\")\n",
    "plt.xlabel('Epoch #')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.9 ('tf')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  },
  "vscode": {
   "interpreter": {
    "hash": "0623a8ec4bd597eab065752ed1de30aa39c6d40efdfbc7bcfd7fe40f99c37f5e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}