{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aSzAgc72qFJy"
   },
   "source": [
    "# Exercise 18.1\n",
    "## Generation of fashion images using Generative Adversarial Networks\n",
    "\n",
    "Let us start to learn how to implement (Generative Adversarial Networks) GAN, as introduced by [Goodfellow et al.](https://arxiv.org/abs/1406.2661) in Keras, to gain more insights into the training procedure. \n",
    "\n",
    "As a simple example, we will use GAN to generate small greyscale images.\n",
    "We will use the fashion MNIST data for training the GAN.\n",
    "\n",
    " 1. Pre-train the discriminator and evaluate the test accuracy.\n",
    " 2. Set up the main loop for training the adversarial framework. Use uniform distributed noise as prior distribution for the generator.\n",
    " 3. Generate some images after each training epoch. Plot the training losses for the generator and discriminator and generated images.\n",
    "\n",
    "Training GANs can be computationally demanding, thus, we recommend to use a GPU for this task."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HEJtbW1uqFJ0"
   },
   "source": [
    "### Software\n",
    "First we have to import our software."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dHlLzqA_qFJ1",
    "outputId": "f868ea87-d18c-4e56-d9e4-20941fe5e4e5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "keras version 2.4.0\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "import matplotlib.pyplot as plt\n",
    "layers = keras.layers\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0T9qjpe0qFJ4"
   },
   "source": [
    "### Data\n",
    "Let us download Fashion MNIST and normalitze it to [0,1]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "-8MQX38LqFJ4"
   },
   "outputs": [],
   "source": [
    "fashion_mnist = keras.datasets.fashion_mnist\n",
    "\n",
    "(train_images, train_labels), _ = fashion_mnist.load_data()\n",
    "train_images = train_images[...,np.newaxis] / 255."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VsxmdTz4qFJ5"
   },
   "source": [
    "Let us inspect the data!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 589
    },
    "id": "p-fWgwLbqFJ6",
    "outputId": "9686432a-6912-4380-8511-05d336f2169b"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAAI8CAYAAAD1D3GaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd7xcVdn91xYp6eWm15tAKKaYkNAiSBGBUF6IgALS9AfKawP1BRQFu4IYQZCiIKJCBIFQRQgQAoGEkoRUAoE0SCHJTW+AwPn9cSdhnrV3Zp+ZO7fMyfp+Pny468w5z9kzZ88+O2eveR6XJAmEEEIIIbLMJxq7AUIIIYQQ9Y0mPEIIIYTIPJrwCCGEECLzaMIjhBBCiMyjCY8QQgghMo8mPEIIIYTIPJ8sZucOHTok1dXV9dQU0VgsXLgQNTU1riHOVSl96N133/W2vfXWW0a3a9fO6ObNmxvtnP+R8jY+z5o1a4zeddddvRhdunQxeqeddvL2aWgasg8BTbcfffDBB0bX1NR4+1RVVRm98847l70dmzdvNjrUn7n/hvprQ7MjjkXvvfee0Rs3bjR67dq13jH8nec+xWNR6PrzWLNhwwajP/EJ+zykffv2XoyOHTt625oCU6ZMqUmSxGtcUROe6upqTJ48uXytEk2CYcOGNdi56qMPcS6pcgzcc+bM8bZ961vfMvqLX/yi0UOGDDF6l1128WJ88pP2Kzd79myj77//fqP79u3rxbjkkkuMbtu2rbdPQ9OQfQhoumPRihUrjL799tu9fc4++2yjeQJbDqZNm2b0a6+95u1z8sknG10fE69iqfSxqBQWLFhg9DPPPGP0gw8+6B3Dk4+zzjrL6H333dfo0PW/7777jH7yySeNbtGihdFnnnmmF+NrX/uat60p4JxbFNquJS0hhBBCZJ6invAI0RjEnuCkeaLzyiuvGH333Xcbzf/aCS0T8aPmyy67zOjVq1dH2xFjzz33NHr69OnePr/5zW+M5icERx99tNHf//73vRgDBw4stYkiD+4TDz30kNF///vfvWPuuusuo3lZgJ8Mhp688Hl5WeTtt982+qSTTvJicB8/9dRTvX1E3fjPf/7jbbvmmmuMbtasmdHvv/++0bvttpsXY+HChUafdtppRi9fvtzo0NIdP23u2rWr0W3atDH63nvv9WJce+21Rh955JFGX3fddd4xjYme8AghhBAi82jCI4QQQojMowmPEEIIITKPPDyiyRPz6Kxfv95o/hUM4Hth2BfUsmVLo3ldHfB/xsseCP5J8rp167wY/HNRjpHGj7T//vsbzT85nThxotHjx4/3Yhx88MFG33HHHdHzCh/uN+x7uPLKK71jfvWrXxnNv6Bh/wX7cwD/l3mtWrUymr0Uxx57rBeDfUCi7sybN8/o0aNHe/uwf27Lli1Gf/TRR0bzz8MBoGfPnka3bt26YLtC4wqPPRyDvWPs+QGAgw46yOjFixcbzf7BUaNGFWxnfaMnPEIIIYTIPJrwCCGEECLzaMIjhBBCiMyjCY8QQgghMs8Oa1oupRwB1xp57rnnjB4xYkTR5/3www+NDhnDioXPEaIp1M0pFyNHjjSaa14BQOfOnY3m98/XIU19Kj6GP3eubxM6hklz7Rg2WHOistC1njBhgtFcSmOfffYpuh3CNxiHyn5885vfNPr66683muunpTEtDx061OivfOUrRnOiOqDp1kGqZNiUm+YzZpMy/wghNBbxfaJPnz5Gs3k+VEuLx4VQPyt0TgD473//azQnOJw1a5bRjzzyiBfj+OOPL3jecqInPEIIIYTIPJrwCCGEECLzaMIjhBBCiMyzw3p4eN2U10nffPNN75hbb73VaPZOtGjRwuhQ0TdOGhfz7IQ8Hdx23ieNDyjmJWnKTJkyxWj27HTo0ME7hpMCMpz8a8mSJdF9+Drw5x76jENJxPLhwoGhwpGcZK5Hjx4F2xGC28F9u7EThFUqfG1qamq8fXr37m00f9bc91auXOnFYK8E93k+b6j/l+IXE4U599xzjeZCoYDv62F/IXtFQ2MAwwVnQ32G4USDnBQ1DXzetWvXGs1jU0P6dULoCY8QQgghMo8mPEIIIYTIPJrwCCGEECLz7LAenljelXHjxnnHPPHEE0ZzATfOY7B582YvxtixY40+//zzjY7liwm1leGigCHfSCnrtU2Fp59+2mj+3EM5J/gzYP8N5z757W9/68Xo2rWr0Xz9ly5dWnD/0Hl5fZ49PKECj1OnTjX6uuuuM5o9ApwrA/A/j/vuu89oeXhKI03+plWrVhV8nf04Xbp08fbhsYV9P2mK0mYpF1dTgT2aXFwTAB588EGjDzjgAKPZbxW6j7Rv395o9tLwGBDyk3JcHic4l8+KFSu8GAz7HEPFcxsTPeERQgghRObRhEcIIYQQmUcTHiGEEEJknh3Ww8NrnszLL7/sbeN6NOzHYH3UUUd5MV555RWjL7nkEqOHDRtm9MCBA70YXOfopZdeMprbPnz4cC9G/tpypeXkuffee41mvwJfB8DPTcPr17xezd4qwPdfcT6gr371q0b/6U9/8mL079/faPYb8bXo1KmTF+O73/2u0TfeeKPRvBYf8jRxzqjXXnvN6Llz5xq95557ejGET5oafdxf+ZpzLpP6akcsN5WoO9/5zne8bddee63RnJeJ/Tf8XQV8Dybn1GFC15rPw/vwOBI6x7p164zmepKxdjU0esIjhBBCiMyjCY8QQgghMo8mPEIIIYTIPJrwCCGEECLz7DCm5ZiJj5MKTp482YvBBqxNmzYZzUZP1gCw3377Gb3HHnsYzYnmJk6c6MUYM2aM0WzI5eRXt9xyixcj37QdSmzVlJk+fbrRnAAwZMLm5IQMm+9CHH300Ua3bNnS6Dlz5hj9u9/9zosxcuRIox9++GGj2Tg4ZMgQLwYnHowZskOJJ3kbf4aTJk0yWqbldPD3N9TvOAkc91e+NqH+HCv8GftBBRA2s4u6wd/fUCHf559/3ugf/ehHBWOGksRywlJO+MeFrUN9iI/h5KuhPsPwPieccEL0mMZET3iEEEIIkXk04RFCCCFE5tGERwghhBCZJxMenth6dhouv/xyo5ctWxY9hr0SnFCM10QB4LnnnjOavULsLdp33329GP369St43j/+8Y9Gz58/34uRXyyyVatW3utNiZkzZxrNCbNiidxC23j9movxhZg9e7bRfH25z4TW5rmv8lo8v85emhBcpJSLmIYKWnI/4zX/Z5991uhzzjkn2g7hezhCYxNvYx8Evx6KETuGvSMhP0alJRytBEKeHYa/r3379jV6wYIFRocKf/KYzb4vPiZ0/dmDuHLlSqPT9KFevXp525oyesIjhBBCiMyjCY8QQgghMo8mPEIIIYTIPJnw8IQK4xVLu3btjA55eNjnwDk2uNga5+QA/LVV9pLwe2HPD+Dn5uH1++XLlxt9zDHHeDEqiauuuspo/sy4uF5oHZ39Vnwd2EsTysO0atUqo1evXm00X3++DqHzcDvef/99o0OFJO+++26j16xZYzT301AM3ofbzoVRRTrY5xDKocLemZgfJ+TBYmJjYMhPKJoGfL35vhHKo8X3Hvb08DgS8gHFCmin6Xeh4sZNGT3hEUIIIUTm0YRHCCGEEJlHEx4hhBBCZJ5MeHjKAXs8QjkqeK2dfRBdunQxuqqqyouxcOFCo3l9ltdz09RA4Ri89rp48WIvRiUxfPhwo9kb8+abbxodqovF15dzGfFneMABB3gx+HPlY1iH8lawVyaWPyV0/bmmG9e54hpvoXbwebt162b0SSed5B0j4qSpP1Rs7aw0MRnOBxTy8IQ8ZqK8hK4dX+/u3bsbPWPGjGgMvp4ck+ukhXxAvA/fz9j3U1NT48Xo0aOHty2fNLXFGhI94RFCCCFE5tGERwghhBCZRxMeIYQQQmQeTXiEEEIIkXkyYVqOFeMDfMMpJ3figoshkx8nauLkTnwMJ8QDfEMtG5vZXMvnAPyib+vXrzd64MCBRrOJFbCJ9ficTY1vfOMbBTUn3nvjjTe8GDfddJPR48ePN5qLh/JnCABt27Y1mq9NKeZSJk1fZjMh96lBgwYZPXr06Dq3S4ThvseG41DhT04SWI5+w6ZUNouGEs/x955NrKFjRPmprq42mvtQ6B7A/a53795GszmYk6YCfrJdPobvd6G+3Ngm5GLREx4hhBBCZB5NeIQQQgiReTThEUIIIUTmqawFuO3Aa+KhZG3s4eECjFwstGPHjl4MTvjHMdkr89Zbb3kxuHgkF4HjNVFOVBdqByeE+uY3v2n0tGnTvBj5a/yhtdlKgtei999/f28f9leNGzfOaO5DfF0A//qyTyKU3Ivhz5o1xwi1g/sQey84UaOoP7hfsS6lsHGaY9J4vfIJjYlt2rQxWp6dxoELzKYp2hlLcpom8SCPmytXrjQ6VPyaCfmLmjJ6wiOEEEKIzKMJjxBCCCEyjyY8QgghhMg8mfDwsJeC8weEGDBggNG89h7yzvA6OK+1rlixwujQmjjne+G283lDOXR47bVnz55Gc96Viy++2Itx4IEHbvs7lC+oKcP+Bf7MQteffRGtWrUyOnZtQzFi7SrFv5GGmF+D8wWFiBWwrK+2Z400/sHGII0nTdQ/aXx97Ntk/2hoPON7AMNjQCgGe0E7d+5sNHt6Ku0+EUJPeIQQQgiReTThEUIIIUTm0YRHCCGEEJmnrB6eUD4XXtNm/wEfwzlGgPg6aCn1PEaMGGE016dq1qyZd0ws5wCvvbI/B/DzI8T8RqH3xp8Hf8YzZswwmvNtVDrsTwj1GWb33Xc3unXr1kaX4gPjdtSHhyfUjlg/THO9+XuYJveH8Il5dkJjV7G1s8oRI019Qd4njf9EFCb0ufPnyrUQuU5W6F4Uqo2VD9+LQvUSuQZfbMwLvZdQrrl8mlqtLfVoIYQQQmQeTXiEEEIIkXk04RFCCCFE5tGERwghhBCZp06OojTJ2hrCtPTss88afd9993n7PPfcc0ZzwbaqqiqjQ4m62ITK741jhgyNHJdNzHyONMme2MTKx4wZM8Y75oQTTojGrRTSGHDZ+MeJJvk6hIzQnOAwZlIOmfiLLfoYSl7JBkSOKQNywxH7/oaM67HrxX2ilGSGaQz0vI3HERUTrTtpjN9sMO7fv7/RvXr18o7hMYCv1fLly40OGZJ79+5dMAabqbt27erFWLJkibetKaMnPEIIIYTIPJrwCCGEECLzaMIjhBBCiMxTJ4NNKV6B1atXG7106VKj586d6x3D+7AnhY9hfwbgr4uzz4UTOXXr1s2LwWuc7OngddNQO3jtdfjw4UZv2LDB6AkTJngxeF2YE82x/+SFF17wYmSJNAn++DOLFctM472ItSNNcjiOGTsHEE88mcY3oOKg5SF2/ULXs9gitOUgTcxikxmK8sBjPCdJZa8N4N+LuBgy30fWrl3rxWDPKft8+L4bgu95XEC7U6dORqdJxFif6AmPEEIIITKPJjxCCCGEyDya8AghhBAi89TJwzNp0iSjr7jiCm+flStXGs1ribx+F1rja9u2rdHsHeL1y5B3htewOS8Le2nuvvtuL8Z+++1nNOcp4HXVhQsXejEYLvS5ceNGo3v06OEdw/4j9gVt2rSp6HbsaPD6NPexNLlPSvHfFEsoJnu0eJ9Q0VpRP5SSIydGmnxOTCzvTqhPcNvVb+pOmgKsb7/9ttGvvvqq0X379jWai4kCvud0jz32MJrvAfPnz/ditGvXzmi+n6WBi26PHj3a6Isuusjoxi5Iqyc8QgghhMg8mvAIIYQQIvNowiOEEEKIzFO0hyd/3ffCCy80r4V+t8/1pngNL02tKK4/xf4b1iHWrVtn9KJFi4z+wQ9+EI150003Gc21RdjDc8QRR3gxOMfCG2+8YTSvzYZqOvFaO68b82fOuRCyRik5ZWI5pLiuEOD33XLkYIl5L0LtYI9aGr9GrB2iNGJ1sdLkc4rlv0lzrcqRA4rHyNatW0djCEsaj8rjjz9u9Kc+9SmjuT5b6Drw/at79+5Gv/baa0aHxjv2h7KftHPnzkbzvQnwfUBcW4vvb/369fNiNCR6wiOEEEKIzKMJjxBCCCEyjyY8QgghhMg8mvAIIYQQIvMUZVquqanB3/72t22ajVOcMAnwEyBxUbOQEYphEyab69h8xQYuANiyZYvRbMg655xzjH7ggQe8GCeccILRCxYsMJrf65QpU7wYTz/9tNGc/IsNqWzYBsJG1nzYtBzaPz/5VSxeFuHPmU2eIZNfLKlYGvMwm9D5GO4PIcMqX18mVChQ1A9cQJj7SDmSBpaDUJ/h87BZVtQPbA4eNGiQ0dyHQuNz6L6QT5ofLsTGL/4RDidMBHxDNWueI8i0LIQQQghRz2jCI4QQQojMowmPEEIIITJPUR6enXfe2SSxY+8M+3MA3yvRq1evgsfwmjjgFzVr37690b179462g9cjWbNnY+TIkV6MgQMHGs1FOdmPFCpiykUq2dPB7dhll128GLymW2xCPACYO3futr9j68FZJJZ4MESxXotQYcmY/yZN4Ujeh/sQ+9XSnEeUBnsl+HqF+kBDfPaxfgbEi9CKusM+T8BPWMveKS7IGfLj8PgV+86H+gPfN2L3gebNm3vb3nnnHaPZP8vFwxsbPeERQgghRObRhEcIIYQQmUcTHiGEEEJknqI9PPm+HV4D7Nmzp3cM56bhNT32tHTs2NGLwdt4TZPXHkNrnrxOunHjRqN5rb2qqsqL8eqrrxrNa63sT+LCaqF28HvjtdZQ8VDeh9dveV21TZs2Xoxp06Zt+3vz5s3e61kn5K2IUaz3ohRPRJoikLwPr+fviNezsYjlsAr1GR430xT6rCuhdvDYwmO1qDuh3DV8/fl+xX0qlB+J7wEh72s+a9asicbgMZHb1adPHy8GFwflGJwzb/Xq1V4M9uTWJ3rCI4QQQojMowmPEEIIITKPJjxCCCGEyDxFeXiaN2+OwYMHb9Ocq+avf/2rd0y3bt2M3n333Y3mfDjsrQH8NU32rPD6ZcjDw+fhfXiNO5RzgPMn8FoseylC7WDPEucM4tw9vH9oG+fq4bX5UC6I/FpiIZ9QJVGOvCbl8FGk8ezEvENp8vBwW2Nr8aL+4LGJr18o/0lD5LtJUxuOv/fz5s0zesiQIeVv2A5G6B7A14bvNezBC/lzeMzn68v3plBuOu6bfO9ZsmSJ0cOGDfNiPPvss0bzPZLff8hLJA+PEEIIIUQZ0YRHCCGEEJlHEx4hhBBCZB5NeIQQQgiReYoyLTOXXXaZ0fmG5q387ne/M5oNtJx4L2TSZVMXm7448WDItBkr8sdmw5DZLGaejp0jBO/D7zVkNuPkTWxQ48SDgwYN8mKceeaZ2/6+9tpro+1syhRb1BPwTX9pCm4ysQRyIaMo78MxmNB74ffL50ljWlbx0PKwdOnSgq+nSRzJfYCvX5prFetXobGITasdOnSInkcUBxeUBvz7CN8DZ82aZXRobOJkshyTr22aHwPxD3tmzJhh9HHHHefF4Ps1x2STcui+2pDoCY8QQgghMo8mPEIIIYTIPJrwCCGEECLzFO3hyV8r5nXiY4891tuft40bN85o9gEtXLjQi8EFyHg9mte8Q4maeE2TY3Tq1Mno0Lp5fuFUwF/z5GKipSSAY29JKAEir9d//vOfN3qfffYxevjw4UW3Y0cjjf+G+wwfE9NAOm9FPqF+GEuSqMSDDQePATz2hK4fX59yeLI4iSAfE/KKsa+Dix+LusPFsgH/+8uFqteuXWt06PpzQl/2znDh6hYtWkTbEYPvb6HzcH/n8y5btsyLsddeexXVjrqgJzxCCCGEyDya8AghhBAi82jCI4QQQojMU7SHJ5Y3JMYRRxxh9AsvvBA95rXXXjOa10V5HXHx4sVejN69exvNXhkuaioqh1JyyvAa+BtvvGF0qOgj933WvI4e+q5wW2PFJkN+tBjKw9Nw7L///kbPnTvXaPZjAL7vh2FPT6gvFnv9Qt4J7p8N6aXYUdi0aZO3jX2ZoYKa+bz77rveNr5/cX4bvkdyrp9Q2/gY1lxcFoh7ErmfhvLKNSR6wiOEEEKIzKMJjxBCCCEyjyY8QgghhMg8daql1VDsvffeBTUzYMCA+myOyADsreCcJCHvDNfFYa8M57UoxX/Dfo2QH4fzQXGtndBaO1NsTS8Rhv0YZ599ttFPP/20d0xNTY3R7KVgPwbn2AnB/YT7UXV1tXcM+ylDOb9E3WBvIAD06dPH6JBHJ59QvpzNmzcbzb4wzr02evRoLwb3s8997nMFzxtqB4+j3If69u1r9OGHH+7FaEg0ygkhhBAi82jCI4QQQojMowmPEEIIITKPJjxCCCGEyDwVYVoWohCxZFch9t13X6P79+9vdNu2bb1jYiZkNvWFiu1x22JJ5kJmYjaxsnGQk+GFkEm5PPD1Y/PoiBEjojFWr15t9DvvvGM0F08G/H7UpUuXgjqW7BAo7XskCnPjjTd62/g7zuPGl770JaNDP0LgRLpvv/220WyMHjZsWLyxxMknnxzd59RTTy06bmOiUU8IIYQQmUcTHiGEEEJkHk14hBBCCJF5HK/bFtzZuZUAFtVfc0Qj0TtJEr+6XD2gPpRZGqwPAepHGUZjkSgHwX5U1IRHCCGEEKIS0ZKWEEIIITKPJjxCCCGEyDwVOeFxzp3knEucc4WriH68/0LnXIfA9o2h/QvEKWr/AnHOdc51K0csURrOuSrn3LTcf+8455bk6V0KHFftnJu1ndd+7pw7cjuvedfcOXeac+5HzrnDnHPDQ8eJpov6kGgMnHMf5vrYbOfcdOfc951zFXkvb2gqNfHg6QCey/3/J43cllI4F8AsAEsbuR07LEmSrAIwGACccz8FsDFJkt/VMeYVoe3OuZ0QvuYjAFwH4AQAGwFMrMv5RcOiPiQaiS1Jkmztd50AjAbQGnQvdM59MkmSDwLH77BU3KzQOdcSwMEA/h+A0/K2H+acG++cu9c595pz7k5HqUKdc82cc/9xzp0fiHuxc+5l59wM59zPCpz/mtzM+innXMfctsHOuRdyx97vnGu3ve3OuVMADANwZ26W3qwsH4woO865/s65l3LXaYZzrl/upZ2cc7fk+sHYrdfQOXd77vpufap4lXNuKmon5uaa5/rmYACrAVwA4Lu51w7JPQEYlzvnU865Xnnxb3bOTXbOzXXOHd/Qn4koDvUhUZ8kSbICwNcAfMvVcq5z7iHn3DgATznnWjjnbsv1wVeccycC4X6Z2/ffuadGs5xzXyp48gqk4iY8AE4E8FiSJHMBrHLODc17bQiAiwB8CkBfAJ/Je60lgIcB/DNJklvyAzrnjgLQD8D+qB1AhjrnPhs4dwsAk5Mk6Q/gGXw8o/47gEuTJBkEYGah7UmS3AtgMoAvJ0kyOEmSLaV8CKJBuADAH3L/mhoGYHFuez8AN+T6wVoA28vBvipJkn2TJLkD/jUfAmB6kiQLANwM4JrcaxMAXA/gb7l+cydq/wW/lWrU9tPjANzsnIvXDBCNifqQqFeSJJkPYCcAnXKb9gVwSpIkhwL4EYBxSZLsD+BwAFc751og3C+PAbA0SZJPJ0kyAMBjDfxW6p1KnPCcDuCu3N935fRWXkqSZHGSJB8BmIbaL/ZWHgTw1yRJ/h6IeVTuv1cATAWwN2oHJOYjAHfn/r4DwMHOuTYA2iZJ8kxu+98AfHZ721O/S9EUmATgMufcpajN67B1crogSZJpub+nwPazfO7eznagdnD5z3ZeOwi1j6kB4B+ofaK5lX8lSfJRkiRvAJiP2r4qmi7qQ6KheSJJkq0F2o4C8APn3DQA4wHsBqAXwv1yJoDP554qHpIkiV/ErcKpqAmPc649gCMA3OqcWwjgYgBfzFu6ei9v9w9hPUrPAziGl7m2hgbwm9y/jgYnSbJHkiR/SdEkJTHKEM65ke5j0+mwJElGA/gfAFsAPOqcOyK3a6F+ls+mAqc7CsDYEprJfU59sAmhPiQaGudcX9T2oRW5Tfl9xgE4Oe/e1itJkjmhfplbNdkXtROfXzrngn6ySqaiJjwATgHwjyRJeidJUp0kSU8ACwAckuLYKwCsAXBD4LXHAXzV1fqD4Jzr7mrNYMwncm0AgDMAPJebBa9xzm1tw1kAntne9tzfGwC0StFm0YAkSXJ/3sAwOTeQzE+S5DrUPiEcVIfw26557unfJ3OmV/Najon42J/2ZQAT8l471Tn3Cefc7qhdtn29Dm0SZUZ9SDQkrtZHejOAPybhLMKPA/j21n/oO+eG5P7v9UtX+wvAzbnl06tRO/nJFJU24TkdwP207T7YZa1CXAigmXPut/kbkyQZi9rHv5OcczMB3IvwhGQTgP1d7U9KjwDw89z2c1C7NjoDtR6g2PbbUbt2LtNy0+aLAGblHgcPQK0nq1RuR+6ao/ZfVk/mvfYwgK1PBg4B8G0AX8n1m7NQ22+38haAl1C7lHFBkiTv1qFNov5RHxLlplnuOs9GbR8YC2B7P7T5BYCdAczI7f+L3PZQvxwI4KXctp8A+GU9vodGQaUlhGhgnHO3Arg1SZIXijzudgCP5IzvYgdGfUiI4qnUPDxCVCxJkpzX2G0QlY36kBDFoyc8QgghhMg8lebhEUIIIYQoGk14hBBCCJF5NOERQgghRObRhEcIIYQQmUcTHiGEEEJkHk14hBBCCJF5NOERQgghRObRhEcIIYQQmUcTHiGEEEJknqJKS3To0CGprq6up6ZUPv/973+N3nnnnRupJcWxcOFC1NTUuIY4V1PtQx988IHR69ev9/apqakxeqeddjJ6t912M/oTn/D/PcHn2bRpk9EtWrQwunv37l6MUNzGpiH7ENB0+5GoGxqL0rFhwwajd911V6N32WWXomO+9957Rm/evNnodu3aFR2zsZgyZUpNkiQdeXtRE57q6mpMnjy5fK3KGEuXLjW6W7dujdSS4hg2bFiDnascfYjLoThX9/FxxYoVRo8bN87b55ZbbjG6bdu2Ru+zzz5G8yAEAGvWrDF60qRJRh944IFG//rXv/ZiNGvWzNtWiFD5mHJ8Zvk0ZB8CNBZllUobi5g0pZrK8d175plnjNloZbEAACAASURBVN59992N7tGjR9ExFyxYYDR/NqeeemrRMRsL59yi0Pam909FIYQQQogyU5HV0o844gij+V/NHTp08I7hf52X8iiTn+AcfvjhRm/ZssXoXr16eTEef/xxo3kJQ/gU+0SHl54A4A9/+IPRTz75pNHvvvuu0aHr8v777xv98ssvGz1mzJiC7QL8ZU5esnrxxReNHj58uBejffv2Rh966KFGf/vb3za6kh5FC1HJ8FiVZvl58eLFRt92221Gjxo1yjsmtORebrjtZ511lrfPVVddZfSFF15Y9Hk++uijguctJ3rCI4QQQojMowmPEEIIITKPJjxCCCGEyDwV6eHhNT/2bCxZssQ7ZuDAgUa3bNnS6FNOOcXoO+64w4vx4YcfGs0/Q+Zf7fBPBwF5duqDefPmGX388cd7+3Tp0sVovlbsreGfnAP+r674FyUbN24sOgb7glauXGk0/4wd8H8++sQTTxj9/PPPG/31r3/di/GFL3zB2yaEKI5S/CdDhgwx+o033jCav9/Nmzf3YvB4xh5E9u3xeAcAy5YtM5o9qPxrUD4HAPzf//2f0fyr0s997nNGjx492ovBn1l9enr0hEcIIYQQmUcTHiGEEEJkHk14hBBCCJF5NOERQgghROapSNMyJ17jlNhVVVXeMatXrzb6nXfeMfr66683evr06V6MGTNmGM3GMK6lFWqHKJ5YosEf/vCHRnft2tXbh68Vm4H5HJ/8pP/V4KRibFJmQ3KotASblLmWFpunQ+1gszyb/PgcN9xwgxfjqKOOMppN/EIIn2ITCx500EHetlmzZhnduXNno/n7Gxr/eB8eJ/j+xgZlwDclc/0tNinzuBPaxuPqP//5T6O5PhcAPPDAA0bzZ1rOUkJ6wiOEEEKIzKMJjxBCCCEyjyY8QgghhMg8Fenh2X333Y1+4YUXjE6T8C1GqLjohAkTjO7WrZvRnLgptF4p6g6vR/N6devWrb1j2F/Fa958rdhbA/iJJ7mfsQ6t7/O6OJ+Xj2FPT+g87L/hdfXQe3nooYeMPuOMM7x9hBCWmH/k/vvvN5rvTQDQs2dPo9mDx2NV6Jy8jTWPgeyDCZ035k8KJR7k8/J4xQW0uXg2APznP/8xesSIEQXPURf0hEcIIYQQmUcTHiGEEEJkHk14hBBCCJF5KtLDs88++xjNa5GhNT8u2sk5BzjHTgj2RvCaJ6+9hrwkou6sWbPGaPbwhDxcXJCPvTN8DOe5AOL5IdL0w1Ax0ELHhNbe2X/EBUc7dOhgdOi9PPnkk0bLwyOEhT17QHhsyYeL8vJ3EfCLSscKGfN9BfDHiVhesVIKcKY5JjYm8tgTKmJ67LHHGs0eTS6UGhpDQ/nKQugJjxBCCCEyjyY8QgghhMg8mvAIIYQQIvNUpIenR48eRvP6XWjtldcSud7SkCFDjA75b/i87Nlg2rRpU/B1URrst+I1Xfb0AP61Ys3+LM6xBPj5nzhXU/PmzY3mWjWA7yXj9Xr2Gs2cOdOL8fDDDxc8z9q1a43mml9AODePEOJjYn4dADjxxBONZo9KqEbdwoULCx6TJhcXE7rnlZuQp4e9QvyZ8TjLYyTgj73jx483+rTTTit4jmLQEx4hhBBCZB5NeIQQQgiReTThEUIIIUTm0YRHCCGEEJmnIk3LbDhmU1coWRsbnfiY/v37Gx1K9sQGLDYls+E0ZmoWpcEmtkMOOcToO++80ztm1qxZRl922WVG77333kW3g5MXcvFY1oBvFuaCfGxqDiUE/M1vfmP0fvvtZzSbtkNGwfnz53vbhBDFMWnSpIKv8z0hRCzBX5rioUzoHlhX0rQjVoA0dF/lMfDll182msf7uhQT1RMeIYQQQmQeTXiEEEIIkXk04RFCCCFE5qlID0/Hjh2N5kROIT9GsYU/Q8S8QrxeyQVKRXm45JJLjObP/fDDD/eO4cSS69evN5r7TGgNnJNRVlVVGR0rAgjE17zXrVtnNHuPAGCPPfYwmj1LnOyM2wkAu+66q7dN1J003gnuA5w0jvtzmiK0aYsn5sMew1IKTDI8jnK76uK/aIpw0k9OcJvmunCfSVM8NE2y3XxCyfr4+sf6bpqEf9wv+b4b8jSxb3H06NFGjxo1KnretOgJjxBCCCEyjyY8QgghhMg8mvAIIYQQIvNUpIenS5cuBV8PrUXyOmgsR06adXPWvI7arl27gucQpXH00Ucb/dRTTxl93333eceMHTvW6HPOOcfoG2+80Wj20gDAm2++aTQX5eQ+w/0D8Psh+7zYR3HmmWd6MVq1amX0lVdeaTT7c0L9cMyYMUZPnDjR6Pbt23vHiDileFR4vEoTo1jPDvdvAPjlL39p9NKlS4uKGSJNoctKZvr06UavXLnSaM7NxjlmAP87z/uw7yXkz4kV6eTXS8mhE3s9dF7ul3zMmjVrvBg8XpXiR0uLnvAIIYQQIvNowiOEEEKIzKMJjxBCCCEyT0V6eBhe8ywFXq8MrXnyumhsvZLztojy8IMf/MBovg7dunXzjtlnn32Mfuihh4z++c9/Hj0v+xN47TnNujm3lX0+7PHh2luAn+/ngAMOMJo9bqG8RJzLR56d+iHke+B+UYpngXOVTJs2zeh77rnH6NAYyfnMTj/9dKP/+c9/Ft0uzkPz29/+1ugf//jHRcdsSvD3NeYFZZ8f4Pv0uI/wOdJ4Z3ifWG6nNDHS5N2JHcPtCHm8uG2LFy+OnrdU9IRHCCGEEJlHEx4hhBBCZB5NeIQQQgiReTThEUIIIUTmyYRpuZRkX8UmXQptY9MXv85F0UR5GDlypNGceHDKlCneMSNGjDD6f/7nf4xesWKF0b169fJisAGPDcZbtmwpuH8INqw2b97c6JDJb8OGDUYvWrTI6Guuuabg6wAwfvx4o7m4KmsRJjZupBmb3njjDaPZcDxp0iTvGE6k2bdvX6N79OhhNCerBPyiy48++mi0rTHuuusuo1988cU6x2xKTJ061Wg2aadJ1seJB7kAKf9QIU0yRz5vmgK0vA+PV2kSqcbGOH6dx0jAN89z8WPuQ/wjjWLQEx4hhBBCZB5NeIQQQgiReTThEUIIIUTmyYSHJ7ROWuwxvD4ZSgYWS+bEmn0hojzMmTPHaPa9hIrLHnjggUY///zzRs+cOdPo0Jp3sQVnQzFifTVNwjB+f2eccYbRgwcPNrpPnz5ejJ49exq91157FWxXFuDrx58t+zHYaxEi5tFZu3att+2yyy4z+u677zaavX9du3b1Yuy///5Gs59s8+bNRu+9995ejCVLlhh9+eWXe/vkExrPuO3f+973jH7ttdeMDvnrhg4dWvC8TQn+/sYS75VSTJVjhmJwwdFYwr9SxiImFOO9994zmouncuLFkA+I3x/HvPbaa40uJSHmVvSERwghhBCZRxMeIYQQQmQeTXiEEEIIkXnk4ckRy6kTOobXI3ktkvNciPIwb948o3m9+u233/aOYd9LLN8N54IA/OvPPq80/ptYng72XoTW79lLwe+F183ZqwH43pJ33nnHaM7rUmmExoTYOJHGs8NwDqj77rvPaC7yCfiFWvv3728096t169Z5MdavX28053JhH9DkyZO9GPyduPPOO42++uqrC54DAAYOHGg0+y/YaxLKB1RJhMaFfNIUy+R+Fiv8mYaYv7AUuF2h7wePIzy+sS+OCx8Dftv5PNyH6oKe8AghhBAi82jCI4QQQojMowmPEEIIITLPDuPh4XXC2DGh13l9ktdaOReCPDz1A1+b3XbbzehQDiX2DrBXhv02oTVxvt4xH1jIw8PHxM7La+ChfTp06ODtk8/q1au9bew/W7p0qdGV7uEJefD4+xnjuuuu87bddNNNRi9fvtxozm80YMAALwb3T47BpKnrF+tXXK8I8H1AzPDhw42+//77C+4PAL/85S+NvuGGG4zu3bu3d8wdd9yx7W/2ADU1fv3rXxvNHp1YThnA/z5WVVUZXYontT7g8S7k4eHvFL9fzg8V8kDxWMyexAceeMDoNPfm7aEnPEIIIYTIPJrwCCGEECLzaMIjhBBCiMyjCY8QQgghMk9Fmpbnzp1rNBs7Syn8GCsKl+YYNiPW1NREY4jiiRnQQ2bhdu3aGb1ly5aCx4T6S8wYV0rx0JjJMVRsj9vWuXNno9nEHTLrctwNGzZ4+1QSU6dONfqJJ57w9nn99deN5oRmbNwOfSacOK1Hjx5Gc5LAkGk1lEgwHzZthvpArM+zWTT0neBEgtxvXnzxRaNDRUw3bdpkdPfu3Y3ec889jWaDKgDccsst2/5euXKl93pTYv78+UbvuuuuRvP1Dv3ogI3b/Jk0FdMyE2oXj1/8neF+mObezGbp6urqaIy06AmPEEIIITKPJjxCCCGEyDya8AghhBAi81Skh2fOnDlG8zp6KEFSaO04n1hSuTTH8HouF2QEgIkTJxrNyb1E8bDHIbTGy4USY/0hRMw3we0I+YB4G2v226QpJMj9LpYQEYgXPm3qrFixAn/84x+36TFjxpjX2aMFxIshst+AC3CGYnChVu4T7McBfB9QrP+GiidyO9g7wtc89HlwXPabtGnTxuiQF4y9cezp4PNWklcsVHSX3w8n/UxT/Jf7SKzocJpEe3xtQt6hGHxejhlqB4897E/j7xj7xAC/T/DY9NZbb22nxcWjJzxCCCGEyDya8AghhBAi82jCI4QQQojMU5EenqeeespoXs9M452IrZOm+a0/x+Rj9thjD+8YLj4oD0/xlJKHgb0Godwm+aQp/MlrzbH+sL1thWKGvDXcDvYVsEckTUHGkE+kKVNVVYWzzjprm95vv/3M688//7x3zKxZs4xetGiR0ewlWLNmjReDfT6xPrBixQovBufnink6Qn4MbkfMcxgq2sgeJfZbsIcj1HfZk8FtZQ8Tez4A4Ljjjtv294MPPui93lhMmDAhuk/MOxPy8PBnxsVE+TqkKYRbbI6wcsHXk/sU9232vAH+WMyfTyyHXjHoCY8QQgghMo8mPEIIIYTIPJrwCCGEECLzVKSH54UXXjCa19HT1J5hSslDwmuL7IMIrVdzHh7ROPC1SpNzIpZDh0mzbh6r4RXqQ2vXrjWaPTz9+vUzetq0aV4M9gk01fo9hchv84ABA8xrBxxwQPR49jYtWLDA6DfffNM7ZuHChUZz/S3uV2n6EfeBqqoqo1u1auXF4H3Yt8U5dPh1wPfXhHIG5RPyEsX6DeepCeU2yv+ehLxGjUXIf8Pw9zNW5w/wv79874n5woD4uME69F6KvSeGxjOOy/4bfj3ki4u9/3KiJzxCCCGEyDya8AghhBAi82jCI4QQQojMowmPEEIIITJPRZqW2TjISeXSFFtj0hi0YqQp2McFRdk4GTKpCgubODmZVZpEVXxt2FwXKpQYSkaYD/eZUD/kbbEEiKFCuLHioL169TJ68uTJXgzuZ5VWPHSnnXYyRtxNmzaZ15ctW+YdEzNptm/f3ujDDjvM24dNyTFja+hzjSVK5XOEYrCBmBMRcoxQwreVK1cazYkXOWbovXL/5eKZ/F0NGVJ79+697e/QmNlYHHroodF9+FryGBF6vzFDccwIHTovXwfWzZo182Lw9Y0VLg6NVdz2WCFUPmfovPWJnvAIIYQQIvNowiOEEEKIzKMJjxBCCCEyT0V4eDhZEa89d+rUyehQscRYoqbYWizgewBY83mPOuooL8a//vUvo6dMmWK0ion6sF8h5pVp3bp1NGasCGQIPm9sDTxNMj9e8+YYoX4YW7+vrq4u2M5Q3NA+lQQntAsluIvB/pE0fgP2xvAYkOZz5T7AY1Maj0PMgxhKXti9e3ejY/6yNJ9HrD+Hrku3bt22/R3ymjQW//73v6P7sMeONd+rAKBz584Fj+HrkOb7y59zKT6gWD8M+VqLLfwZ8qNx2+vT06MnPEIIIYTIPJrwCCGEECLzaMIjhBBCiMxTER6eV155peDr7L8I5XKIeXg4b0WoUB6vLfKaJuckeP31170YvIY5Z84co+Xh8eHPOealYW9CiJjnIZZzB4h7ukIeHo5brLcM8Ps750/h4qFpPACVWDy03LB/JI2fhHOAiezw2GOPRffh7yJ7Z/i7CQA33XST0V/+8peN5u9rqKAqf3/ZB8Svp8kHFTsH3yND29atW2c05zJatGiRFyNU2LYQy5cv97axL2p76AmPEEIIITKPJjxCCCGEyDya8AghhBAi81SEh+eRRx4xukOHDkaXUgeJ82ekqXHEx3C+F17P5bpZobbNnDnT20cUJlaLKD+vx/YopeZLLE9FKb4ffi+l5PLhdfP+/fsXbGdomzw8QlhC+dw4nxHXDkszBowcOdLo73znO0aPHj3a6JAPaPXq1UZ37drV6FDbmWI9qaF6bBzjgAMOMPrCCy80+plnnvFixMZA5qGHHvK2nX/++QWP2Yqe8AghhBAi82jCI4QQQojMowmPEEIIITKPJjxCCCGEyDwVYVqeN2+e0WziYnNwKMlSVVVVwWMefvhho48//ngvBiciY8NaKEEUw8fMnj07eoywxIqH9u7dOxqDE4R17NjR6FCxxZiZjk3raczCTKwgLeAn+2IzYZrEi/xeQiZ9IXZkQon5+N5TbNK8EFdeeWVBnQYeE7idoR8lxBK6cjLDNEWZSyFWLJULkvK9GpBpWQghhBBiG5rwCCGEECLzaMIjhBBCiMxTER4e9tOMHz/eaF4DDCV/Yu8Mk8Z/wx4NTswU2x/w1yMHDhwYPe+OTpqinPmE/DcMe2N4DTx0bVetWmU0X99SkgYy7PEJrZtv2rTJ6GXLlhnNfSzkA2LPTqhYrhA7Mn/5y1+8bWPGjDGav4ulJCMtB/ydZ91UqK6u9ratXLnSaPZF8dj8mc98puTz6wmPEEIIITKPJjxCCCGEyDya8AghhBAi81SEh4d/Y/+1r33NaPZKcM4dIJ5DJc1aKxctXbt2rdGct2D9+vVeDN7GxdWED+dV4s+5FO/MKaecYjRfF87LE2pHrE+F8kHF/EjcD0M+sDZt2hg9bNiwgu0I+ZG47aG2CrEjE8qxs2jRIqOHDx9uNI8jZ5xxRvkbBt8rFNOhnEJMbJ/QPZK3xYojH3PMMV6MW2+91WjOK3bccccZfemllxZsZyH0hEcIIYQQmUcTHiGEEEJkHk14hBBCCJF5KsLDw8yYMcPoQYMGRY/h2knMihUrojG4/hbnB2AfBNczAYDHH3/c6DR1n3Z0tmzZYnRsvZq9VSF++MMf1r1hFUJobb6Uz0yIHZ1evXoZzfmreMxfvHhxNCbn8mnRokX0GPbONFS+nxh8D2QP4uDBg71jeB/28HzrW98qU+v0hEcIIYQQOwCa8AghhBAi82jCI4QQQojMowmPEEIIITJPRZqWueAmJzuaMGGCd8ycOXOMHjdunNFpCpKxeYqNzl/60peMPvbYY6MxRZz27dsbveeeexrds2dPow844IBozFhywjSJuiqFUPKzBQsWGD106NCGao4QFQuPG1dffbXRPFZ17do1GjP2g5pKIjZuhhK6NmvWzGj+PMppyNYTHiGEEEJkHk14hBBCCJF5NOERQgghROZxaQotbtvZuZUAFkV3FJVG7yRJ/MXVekB9KLM0WB8C1I8yjMYiUQ6C/aioCY8QQgghRCWiJS0hhBBCZB5NeIQQQgiReSpuwuOcq3LOTcv9945zbkme3qXAcdXOuVnbee3nzrkjt/Pauc65brTtNOfcj5xzhznnhtftHYmmhHPuw1xfmuWcu8c51zyy/3jn3LDc3wudcx0apqWiKZMbH2Y752bk+lM8OVT62Ic55x4pVzzRNNFYVH4qbsKTJMmqJEkGJ0kyGMDNAK7ZqpMkeT92/HZiXpEkyZO83Tm3E4BzAXSjl0YAeAzAYQA04ckWW3J9aQCA9wFc0NgNAgBXS8V9X3dEnHMHATgewL5JkgwCcCSAtxu3VbU45yoy2ewOisaiMlORjY7hnOvvnHspNzue4Zzrl3tpJ+fcLbl/eY11zjXL7X+7c+6U3N8LnXNXOeemAjgdwDAAd+ZiNXO1qSQHA1iN2g743dxrh+SeIo3LnfMp51yvvPg3O+cmO+fmOueOb+jPRJTEBAB78L+onXN/dM6dW+hA59z3cv8ym+Wcuyi37Urn3Dfz9vmpc+7/cn9f7Jx7Odd3fpbbVu2ce90593cAswD0DJ1LNDm6AqhJkuQ9AEiSpCZJkqW5seVnzrmpzrmZzrm9AcA518I5d1tuzHrFOXdibnu1c25Cbv+poafJzrn9csfs7pwb6px7xjk3xTn3uHOua26f8c65a51zkwFc2HAfgygjGovKQCYnPKidiPwh9xRoGIDFue39ANyQJEl/AGsBnLyd41clSbJvkiR3AJgM4Mu5mfYWAEMATE+SZAHsE6YJAK4H8Lfcv+ruBHBdXsxqAPsDOA7Azc653cr4fkWZyf1LeASAmSUcOxTAVwAcAOBAAOc754YAuBvAF/N2/SKAu51zR6G2b+6P2sn0UOfcZ3P79ANwY5Ik/ZMk0U9oK4OxAHrm/nFzo3Pu0LzXapIk2RfATQD+L7ftRwDGJUmyP4DDAVztnGsBYAWAz+f2/xLseILcBOhmACcCeAu1488pSZIMBXAbgF/l7b5LkiTDkiQZVe43K+oXjUXlI6sTnkkALnPOXYra3+NvyW1fkCTJtNzfU1A7CQlxd4HYxwD4z3ZeOwjA6Nzf/wBwcN5r/0qS5KMkSd4AMB/A3oXfgmgkmjnnpqF2ovsWgL+UEONgAPcnSbIpSZKNAMYAOCRJklcAdHLOdXPOfRrAmiRJ3gZwVO6/VwBMRW3f2PpUclGSJC/U7S2JhiR3zYcC+BqAlai9kZybe3lM7v/5489RAH6Q63fjAewGoBeAnQHc4pybCeAeAJ/KO80+AP4M4IQkSd4CsBeAAQCeyMX5MYAeefsXGtNE00RjUZnJxHquc24kgJ/k5HlJkox2zr2I2qcpjzrnvo7aScZ7eYd9CMBWLfuYTQVOdxS2/2SoEJzwSAmQmiZbck8Gt+Gc+wD2Hwd1eTp3D4BTAHTBxzchB+A3SZL8ic5bjcJ9UTRRkiT5ELWTl/G5Ccs5uZe2jkEf4uPx1wE4OUmS1/NjOOd+CmA5gE+jtv+9m/fyMtT2wyEAluZizE6S5KDtNEn9qPLQWFRmMvGEJ0mS+/OMy5Odc30BzE+S5DoADwIYVIfwGwC0AgDnXBsAn0ySZBW/lmMigNNyf38ZteuuWznVOfcJ59zuAPoCMIObaNIsAvAp59yuzrm2AD4X2X8CgJOcc81zSxMj8XFfuBu1feQU1A44APA4gK8651oCgHOuu3OuU7nfhGgYnHN7uY99g0Dt0kChJYDHAXzbudpS07klBwBoA2BZkiQfATgLwE55x6xF7T/ofuOcOwy140lHV2uYhnNuZ+dc/3K8H9Gk0FhUBzLxhCfAFwGc5Zz7L4B3APwaQOsSY92OWs/NFgCjAOT/muthAPfmTIbfzv33V+fcxah9lP2VvH3fAvBSrh0XJEmS/6810YRJkuRt59y/UGvWW4Dax72F9p/qnLsdtdcbAG7NPUJGkiSznXOtACxJkmRZbttY59w+ACbl7nkbAZyJ2qcAovJoCeD63A3pAwBvonZ5a3s/VvgFgGsBzHC1v35ZkNv3RgD3OefORu2vQs2/sJMkWe5qfwDxHwBfRe2N67qt/zDLxZxd5vcmGhGNRXVDpSWKwDl3K2o7TFHrmLkO90iSJPfWS8OEEEIIUZCsPuGpF5IkOa+x2yCEEEKI4tETHiGEEEJknkyYloUQQgghCqEJjxBCCCEyjyY8QgghhMg8mvAIIYQQIvNowiOEEEKIzKMJjxBCCCEyjyY8QgghhMg8mvAIIYQQIvNowiOEEEKIzFNUaYkOHTok1dXV9dQU0VgsXLgQNTU1riHOpT6UTRqyDwGN148++ugjo5csWWL0pk2mvieqqqq8GB07dix/wyKsWbPG21ZTU2N069a2vnLnzp3rtU0hNBbVD+++69eqXr9+vdE77bST0Z/4hH0e0rJlSy/GzjvvXIbWlZ8pU6bUJEnifdGKmvBUV1dj8uTJ5WuVaBIMGzaswc6lPpRNGrIPAY3Xj3hCc/nllxs9ceJEo88++2wvxje+8Y3yNyzCPffc42279dZbjR4xYoTRF110Ub22KYTGovrh9ddf97Y99thjRrdv397o3Xbbzejhw4d7Mbp3716G1lm43FWuantROOcWhbZrSUsIIYQQmUfV0oUQIsAFF1zgbXvmmWeM5iUuXgbiJ0AAcN111xnds2dPo/v162d0mzZtvBirV682mp8svf/++0bz8gUAdO3a1eibbrrJ6IcfftjoW265xYvRt29fb5uof4p9CvK///u/3raXXnrJ6A8++MDo9957L9qO8847z+jp06cbvXnzZqM/+9nPejFGjRpldLNmzYz+8MMPjealt2LQEx4hhBBCZB5NeIQQQgiReTThEUIIIUTm2WE9PLwGymvx/JM8IL5OyjFDlOI4Z3i9nt3z7Mjfc88966UdwtJQ179YzjzzTG/b9773PaP33Xdfo3n9ftdddy1/w5oY48aNM3rBggXePkOGDDGavTE8jnz605/2YqxcudLoefPmGc2/BAv9cmnGjBlGf/KTdijv0KGD0dxuAFixYoXRffr0MXrt2rVGf//73/di3H///d42Uf8U6+F55513vG3t2rUzmn1fu+yyi9HcHwDgjjvuMJp//s4/W589e7YXg/sue9y4XezxKQY94RFCCCFE5tGERwghhBCZRxMeIYQQQmQeTXiEEEIIkXl2WNNyjBLTWdf5vOPHjzd65syZ3j5vvPGG0ZdddpnRbGgbO3asF2NHMKEWopT05bFjWIdMzMWe97///a+3jY2A3EdOOeUUo+fOnevF2Lhxo9EPPPBAUe3KIk888YTRoTpLbObma8HXi83DgG/S5D7BidZC15Q7rAAAIABJREFURk82bnKdo1atWhnNNb8AoHnz5gXb0aNHD6NDyQufe+45ow8++GBvH1F+Yj+yYaPvW2+95cVo0aKF0Zx4kM3zoVpabHxmoz8bn0Nj4ne/+11vWz6hHxCVip7wCCGEECLzaMIjhBBCiMyjCY8QQgghMk8mPDyl+DF4n1IKkv397383+sADDzR6woQJ3jGcVKlbt25Gc/G1UNJAThJ37bXXGj148ODttFhsJY3fJnYMey0YXmcH/HVy9mLwMewRAYBnn33W6JEjRxrN6+Z77723F+OGG24ItLjwebPO0qVLjW7durW3T8zDw30iVICRrw97I9h/EYLHK/bXcNFG9uuEzsteCX4voXFVHp76JzQ2hcaWfDiJJvtxAN/nFYsZ6pccl/s7e9oGDRoUjcFJErt06RJtZ1qfj57wCCGEECLzaMIjhBBCiMyjCY8QQgghMk8mPDz1wZw5c7xt7L/gnDmTJ082evXq1V6Mc845x+hDDz3UaPbncMzQNvYEvPnmm0bvscceXgxhKSXvTMz3FXo95o3htei3337b2+fYY481mtfi2XsxatQoL0b37t2NLsUHV+mwF4B9MG3atPGO4W1cLJEJ5VHi68M5kXic4e93KC6/F44R8j1wjN12283bJ59QnwjleBLlJfS5h/pEPi+//LLR7IMBgLZt2xrNRaf5vCEfGBfCZdgHd+KJJ3r7cJ64oUOHGs1tT+O33B56wiOEEEKIzKMJjxBCCCEyjyY8QgghhMg8mfDwlOI34DwVEydONDq05snr91/96leNvuaaa4xmnwQAfO973zN6xYoVRvN7CeVQmTp1qtFcA4jX4uXhiROrTZOG5cuXGx3ycK1atcroKVOmFIzBXgwAaN++vdHcV9etW2f0sGHDttPiHRuu+8N9YMuWLd4x7EngWkLsi9mwYYMXg2tpcX4T9iiEfEK8D3vDuN+EfA881nCfD3k2mFCNLlFe0lw75umnn47GZQ/P5z//eaPnz58fPSd7eDgH3LRp04wO5fI5+eSTje7du/d2WlxLKTnztqInPEIIIYTIPJrwCCGEECLzaMIjhBBCiMyjCY8QQgghMk8mTMucyCtkOGXDFSf72nXXXY2eNWuWF4MTDf7pT38y+rHHHjP66KOPDjc4j06dOhV8nU3NgG9aZePgbbfdZvRnPvMZL8aAAQOibduRSNOH5s2bZ/RFF11k9Nq1a43mhIAAMHv2bKO5eOyrr75q9GGHHebFYDM8F+zjvhwyPpeDWPHUps6yZcuM5s8t1AfYQMoGS/5MQn2AY/BYxAbkUDt4HzZLc1HakNGTk9d17drVaC7qyO0EgKqqKqPZxNqxY0fvGFEcoe8ZG98ZNhzzj3QA4IUXXjCa7yvcT0OJOHl8Wrx4sdGnn3660b/+9a/DDS5w3nImQdUTHiGEEEJkHk14hBBCCJF5NOERQgghRObJhIeH17jTrPnxGjevG44bN8475swzzzT65ptvTtvEkuFEdYBf5JCLrfHaPHs8OG59eTwqiVhRTwDYfffdjb799tuNZj9DOQh5IDgRHfuxvvSlLxnNPiEg7lni10PJz2I+gqYOf7fYw8IJHAHg2WefNfrLX/6y0fxZs08I8L+PPBbFCkMC/mfPx/B3OnSt2D/Ing7uE/vss48Xg8ei1157zWh5eOpOmkR7EyZMMJq9nyHPJvf/NWvWGM1JNUOFQjnpKReuDvWZxkRPeIQQQgiReTThEUIIIUTm0YRHCCGEEJmnshfhc5TyO33Oj/HZz362oA7BxQW5aGeadsVyDoQ8ALy2ygUNR4wYEY2xaNGibX+HCrqJOOzZ4eKTofwZabxC+Rx++OHetvvuu89o7g/PPPOM0ZdeeqkXI+YLSOMbqHQfGHsSuNBnqAAjH8PFX3ncmDFjhheDizay/4b7UajP8HeWP3/2eXFOHcDPIcTFQl988cWCMQGgR48eRk+fPt3oQw45xDtGFEea+8gdd9xhNN9XQt9PHr/YSxYrSBs6hjn11FON5uLZAPD73//eaH6/5czLoyc8QgghhMg8mvAIIYQQIvNowiOEEEKIzJMJD085SFNLidfWY6+n8UHECOU+aNmypdG8xsntCNXAyc/LEcqxIuLE1pbT+HVi+VLOPvts75h77rmnYDs4FwZ7zYD42jvX9PrmN7/p7ZNf0+vtt98uGK8pct555xn9+c9/3miujQYA1113ndFct47z0LCvD/A9O+zH4bGH62QB/jXnmOzHYX8SALz00ktGc79ibwXX7AP8XGRcj0wUD9+L0txHxo4dazT7c0LXjutrcT9Lkw8qlCcun7POOsvo0Hs58cQTjX7wwQeNVi0tIYQQQogi0IRHCCGEEJlHEx4hhBBCZB5NeIQQQgiReWRazpHGGMb7sPEzlGiOKTaJUihh2N/+9jejjz/+eKPPOOMMo9nkDNi2hwzaIk45zHSxz56vLeAnGmRzLSeifOqpp7wYPXv2NHrkyJEF28GFBQFg9OjR2/6eOXNmweMrAU7EN2bMmOgxXJSRizhyYj6g+B8JhH4sEUtyyX0gZC7lY9q3b2/0L3/5y6LaKcpDmnGFE1ouXLjQ6D59+hgdKiDNhnoeE+bPn290/o8UthIbv/g79fzzz3v7cAHe+kR3OiGEEEJkHk14hBBCCJF5NOERQgghROapCA9POYuH1Sfs8Unj6Yl5hziBFAAMGTLE6MmTJxv99a9/3eh58+Z5MYYPH77tb3l40lFsPwx5NcrRl3mtnZPKrV692ugTTjghGrNz585Gc9LEUBHTrl27bnf/SiCWsDPkneH3OXDgQKPZLxe6vhw3lnwyzfeTY/B5OREhUHyyyHKMZyJOmuvNiQa5X3ICyFDySu4znKCWfT/dunXzYnBiXG7HW2+9ZfTll1/uxWDOPfdco2+//fboMWnRnU4IIYQQmUcTHiGEEEJkHk14hBBCCJF5KsLD01Q9OzFKWc+eNm2a0Z/+9Ke9fU4//XSjH3nkEaMff/xxo7k4IWB9IGmKxImGybuThunTpxs9aNAgo5ctW2b0XXfd5cVYv3690VdccYXRvJ7PhTWzAF9P/r6muVahHFf5hL5b7777rtHs2WGvTBofELeVz9GiRYtUbStE6POo1LG5KRErFhrKocNFbAcPHmz066+/bnToHsDXv1WrVgXb2aFDB28b+0Nj+aBCfhzO1TN+/Hij+f4Wyk2WFj3hEUIIIUTm0YRHCCGEEJlHEx4hhBBCZJ6K8PBUCrG12BBXXXWV0ZxD5YILLvCO+cc//mE05+o59thjjeY6K4Bdv9U6fHlIk2OHc19wH+FjQrl8OMcGr70XW6sJAH71q18ZzR6RU089teiYlU4o7wz7bTi/Cb8eyk/E/iiuycf+m1AMvj583i1bthgd8uvsueee3rZChPqVxo66E7tPhGqacQ6ltm3bGs25utjTA/i+npDPKwa3PeYt474O+B4lrvH16KOPGs3+Q8CvH7k99IRHCCGEEJlHEx4hhBBCZB5NeIQQQgiReTThEUIIIUTmkWm5jLCBK2QW/ulPf2o0m1g7depk9H333efF6Nevn9FsnFy6dKnRWU8sGCsCGdqHYXNdfRVU5bgx0+ewYcO8bVzIkxNNpoENi2zQ5WRgoaRjAliyZInRbDBmA3KITZs2FYwRgvtRzDwd+k5wjMWLFxvdo0cPo0sxwwufmLGXCSXrY5Mym5g54R8XuQWAN9980+g1a9YYXV1dbTSb7YHiiwazmR7wf3RxxBFHGH3jjTcWdY5C6AmPEEIIITKPJjxCCCGEyDya8AghhBAi8zS4h6eU5HwNAbcrtF7NvofmzZsbPWfOHKMvvvhiLwYn++K111GjRhmdJrEXFxydP3++0QcddFA0RmORJllfbJ9YEcimRGy9/gtf+ILRXBgUAP76178WjJHGI8DeMfaRDBkypOA5dgTSfPcmTZpkNHsaQkUb+fpwIkn2OfDrgN/H+RhOIsfXO3TMihUrjGYPT8gH1JS/a41BmvEsNgY8/PDDRrO3CvA9PHwtOTnfhg0bvBhcmJr76qJFi4wOJeLkdvB7Yy9ZiL59+xr9l7/8JXpMqegJjxBCCCEyjyY8QgghhMg8mvAIIYQQIvM0uIcntuabJtdDfRSs43aF1ivZs8M5OH7/+98bzfkEAODFF180+p577imqnSH48+C2c7ubEmmKZZbjer/22mtG33bbbUaz36pjx47RmDGvTCgHCxfG+/GPf2z0ypUrjR4zZky0HUyaHEK8D7+X3XffPRoj63lZ0nyOnMsk5q0BfK8Ee3RiOXVC52G474XGAD4PF5jcd999jVah0Djl+IyuuOIKozkfDuDnYmMvKF//UE645557zmj2l3L/f/rpp70Y3De5OCj3sRBpclXlU5d7hJ7wCCGEECLzaMIjhBBCiMyjCY8QQgghMk+Tq6XVUOvEsXwJafJLcF2sbt26GT1jxgzvmLvvvjtlC9PDa601NTVGN6VaWkmSmHVdvg4h3wSvE7Pv5dZbbzW6S5cu0XYsWLDA6AcffNBo9jOE4Lbye2G/DuCvtbOH69FHH42el30hvG6eJg8P183h/n/wwQdH25E1D08p+YvYc8V+nDR5pZiYxwfwvxMck8evUA4dzhkU6/P1VV8uy6SpYfbyyy8bzXnVQn5CPobz4fTp08foPfbYw4vBuXmmTp1qdMuWLY0OjQkvvPCC0dx3+d4T+j60adPG21aIuswR1IOFEEIIkXk04RFCCCFE5tGERwghhBCZRxMeIYQQQmSeBjctx8zCa9eu9Y5Zvny50cuWLTP6sMMOK7odpRiffvKTnxjNxkE2Kd9///1FnyNU5I/h83KiQTYtNyWcc55ZsljYXMf9I3Rt2TzYqVMno7lwIhfwA4ATTjihYLvS9KnTTz/d6GOOOcboNAn/2KRcCu+8847RXGxy+PDhdT5HpZHGhM1FGauqqozmftS6dWsvBptF+fsQSnrK8DjBbecYob7JMTiJIhMyLacplpkleByJGd3TGL0vvfRSo/nHDqHPlPfhJLicaDD0A4q99trL6E996lNG87jKxUQBYMCAAUZzglc2z4d+QNOuXTtvW32hJzxCCCGEyDya8AghhBAi82jCI4QQQojM0+Aentga76uvvupt42RtvC6+efNmo8tRLJPXRAFg4sSJRnPRswkTJtT5vPz5pFkD5mPeeuutOrejvti4cSOeffbZbZrbesopp3jH8Poze7iYUCIrXidmHwx7WC688EIvRszDw5x44onettmzZxvNCQ8binXr1hldyncma4kH07wfHovY08Pfxffee8+LwcnZ+BguuJimCC3H4OSUrVq18mKwF5D9FdyOkPeOPSxpErY2Vfj6hwpf8mdUSjLGq6++2mhO3nfooYcazfcdwP+cecxjf1bovfA4yv4zhhO8An7bOWki36tD7UhTqLlc6AmPEEIIITKPJjxCCCGEyDya8AghhBAi8xTt4clf5ywl50Isb0NTyf9x/vnne9vmzp1r9COPPFL28/LabBpfAa8jcy6EpsR7772H+fPnb9Nf//rXzeuXX365dwwXsWN/Fb8e8hqw94Jj8GcYKvp3ySWXGH3eeecZzfk0nn76aS/GkUceaTTncWkoeP0+5PGIkfWcKyH4u8Uenvbt2xvNRVqBuFeG/RdpPDwck/OZhb4THIPPwz6vDh06eDGy5OPi/pym6DLnO2JP4vXXX+8dc8011xh90EEHGc05skL3RM5Fxrmd0hSxjfmPHnroIaNDHsZYseNYgWUgXjy0nLme9IRHCCGEEJlHEx4hhBBCZB5NeIQQQgiReYr28NR13T52fGiN79hjjzWa/Rc/+MEPjD7jjDOKbtfPf/5zox977DFvn4suusjogQMHFn2e+oDXkUO+gaZCVVUVzj333G36z3/+s3k9lIeJ3w+vC3fp0sXojRs3ejHY08B+BPYvhPoh589gzfkkQjWvfvazn3nb8onV5ikX/Hm0bdu26Bj11bamzOrVq43mPDuc24Z9MIDv24rVvQr5ydjnwz42vr4hjxbH5XawlyTk4cky9957r7ftK1/5itFp/FYMe1Y4N9fQoUON5hqNgF9zb9asWQXbFfJw8fXl2o9p8o6lqfuWT+j+361bt4LHlDPX0443YgkhhBBih0MTHiGEEEJkHk14hBBCCJF5NOERQgghROYpyrS8YcMGjB8/fpvmxEwhYxwn4uIijZwgiZNhhba9+eabRo8aNcpoTu4GAJ06dTJ67NixRv/hD38w+rDDDvNiXHnlld62+iaNSZxNXfyZNmWqq6uN5mJ0ANCrVy+jufji8uXLjQ6ZPNnUyWbTNJ8zFyCNfc5spgbiRvdyJPPj9xYyT7OZtnPnzgVjpkl+V+mkSaK3YMECo0Nm0HxCBvq+ffsaHSowmk/I+Mx9kcdiPi8XEwX8/svfG05mFyJLiQc5GefFF1/s7cOmdB5X0sBGX77+kyZNMvrAAw/0YuQnbw21gwuBbtq0yYsxcuRIo0866aTttHj7xJIzssE49EOH2A8mytnH9IRHCCGEEJlHEx4hhBBCZB5NeIQQQgiReYry8Lz//vtYuHDhNp3/N+CvGwL+OjCvefNadCipUM+ePY0+88wzjR40aJDRTz75pBdj4sSJRs+cOdPogw8+2Gj2BQH+eiWvvTaWd4Y9GkcffXSjtKMUfvjDHxr9z3/+09uHC3/ymi57x1q3bu3FiBXT40RdXNAxdN6Y52H06NFeDKY+Eg2mWfNmT07MwxPyRe2I8PjEPib2yoTGBB4D2ZPGfgtOdggAffr0KRiDCSWI4/fCfb4U/2Alw8UyQ587+/L4WvHnHPK+xb7z/P19+eWXvRg9evQwetiwYUZzcVG+VwPAmDFjvG35pLm/sSeXifVLID72lBM94RFCCCFE5tGERwghhBCZRxMeIYQQQmSeojw8XPixFFatWmX04sWLjQ6tm/I+vMa5aNEio9mvAwDr1683mguScsFR9g2FaCr5btjD8/vf/97oyy+/vCGbUxSclybkP+FCrldccYXRvMbN17qhOOSQQ4w+/PDDG6UdaXxA/B2JFfArR36gLMA+PvbBsGeB838B/vXhcYRjhHyNnN9s8+bNRrO3IvS9il3TNHmWslRA9uyzzzb6X//6l7fPnDlzjOZ8R2kKv8Zy0/B4Hooxb948o9k/y8Vjn376aS9GDM45FCKWh4pjsFcSiOcyYl9UmnZtj+z0ViGEEEKI7aAJjxBCCCEyjyY8QgghhMg8pS+GlUhVVVVBLYqH61F961vfapyG1BPHHHNMQc3MnTvX2zZlyhSjZ8yYYfSSJUuMDnnJeH2+e/fuRt98880F2wX4Xor68ECk8ZZdcsklRu+1114F94/VzNlR4L7FXgn2NKxZs8aLwdvYs8M+x5AnjesJcj25adOmGX3QQQd5Mdh/wn1zR7vm7J156qmnvH3YT3r77bcb/e9//9tozocDpMtNUyyc7+fRRx81OlQbshz069ev4Ov8/eA6cgDQv3//gjFCHrZS0RMeIYQQQmQeTXiEEEIIkXk04RFCCCFE5tGERwghhBCZp8FNy6L++cUvftHYTWhU9txzz+i2008/vaGaY2iIBH5pznHkkUcWFbOcxsGmShoDORdprKmpMZoTDYYM5B07djSaP9ulS5cW1AAwdOhQo7nQIydjDfWJ5s2bG81GZy6UGSJLiQfTwEU7f/zjHxfUIdj4Pn/+fKPZ1M5JJgHf/BszD9cXF198sdH77bef0fx9CL2X2A+X6pJokNmxeqsQQgghdkg04RFCCCFE5tGERwghhBCZx4WKym13Z+dWAlgU3VFUGr2TJOkY363uqA9llgbrQ4D6UYbRWCTKQbAfFTXhEUIIIYSoRLSkJYQQQojMowmPEEIIITJPZic8zrkfOedmO+dmOOemOecOKEPM8c65YXXdR1QO6kdiezjnqnJ9Yppz7h3n3JI8vd3Km865aufcrO289nPnXDBJknPuXOdcN9p2Wq6PHuacG163dyQaC+fcSc65xDm3d8r9FzrnOgS2bwztXyBOUfsXiOP1zaZIJhMPOucOAnA8gH2TJHkv1zF2rNK/os6oH4lCJEmyCsBgAHDO/RTAxiRJflfHmFeEtjvndgJwLoBZAPIzEY4AcB2AEwBsBDCxLucXjcbpAJ7L/f8njdyWUjgXft9scmT1CU9XADVJkrwHAEmS1CRJstQ5d4Vz7mXn3Czn3J9dLv1o7l/TVznnXnLOzXXOHZLb3sw5d5dzbo5z7n4AzbaewDl3k3Nucu5f/z9rjDcp6h31I1EnnHP9c/1hWu4p4daUuDs5527JXfexzrlmuf1vd86dkvt7Ya4/TUXtjXAYgDtzsZrl+t1gAKsBXADgu7nXDsk9RRqXO+dTzrleefFvzvW5uc654xv6MxEW51xLAAcD+H8ATsvbflhuTLnXOfeac+7OrWNN3j7NnHP/cc6dH4h7cW6cmlFobHHOXZPrh0855zrmtg12zr2QO/Z+51y77W3P9VfTN8vywdQDWZ3wjAXQM/eFvtE5d2hu+x+TJNkvSZIBqL3p5H/ZP5kkyf4ALsLHM+z/BbA5SZJ9ctvy87n/KEmSYQAGATjUOTeoPt+QaBTUj0RduQDAH5IkGYzam8Li3PZ+AG5IkqQ/gLUATt7O8auSJNk3SZI7AEwG8OUkSQYnSbIFwBAA05MkWQDgZgDX5F6bAOB6AH9LkmQQgDtR+xRoK9UA9gdwHICbnXO7lfH9iuI5EcBjSZLMBbDKOZc/PgxB7VjyKQB9AXwm77WWAB4G8M8kSW7JD+icOwq1fWx/1E6KhzrnPhs4dwsAk3P98Bl8PGb9HcCluf4zs9D2JEnuhd83mySZnPAkSbIRtTeVrwFY+f/bu/doK6tyj+O/B7QEFQhERTE3aeIFDQUsT6Ao5cBOpqdjoqVSVnZqYDgsPQ6Peo6XossoCRO1OoiCZqfUYVFKkQPvFy4qaIgYIF4R5KKYkpd5/ljvxjWfd+691t7sC0y/nzEYrOdd75rr3WvN9a653/nsZ0r6jZl9WdKRZvaQmS2UdJSkA6oedkvx/zxVTgiSdLik6UWbCyQtqNr/xOI3r0eKdvZvlx8GnYZ+hDbwgKTzzew/VakN0vhlsCyE0Lh4VXVf8X7TTNujJd3exH2HSbqxuD1NlSsIjf4vhPBuCGGJpKWS6sobQbs5WdJNxe2birjRwyGE50II70p6VHE/uU3StSGE6xNtHl38e0TSfFXe49SCW+/qvT42XdJwM+spqVcI4a5i+3WSDm9qe90/5RYgyxweSQohvCNptqTZxRfTN1T5LXpoCOHZYs69+jebxtX33lGN18XMBkj6rqRhIYS1ZjbVtYVM0I/QEmb2b3rvt+GvhRBuNLOHVLma8icz+4Yqg4zq1T7fUdU0p/N6M093tJq+MtQcX3yNYmydxMx6q/JL04FmFiR1lRTMrHFVTt9Pqs8p90kabWY3hnJBPZM0IYRwTQsPKeu+kOUVHjMbWDVXLlUu6S0ubq8u5kxPqKOpuyV9sWhzkCpfdJLUQ5UT0Xoz20WVxEFkhn6Elgoh3Fpc1h8cQphrZh+RtDSEMEmV38g3Z8ryNUk7SlLx2/Y2ReJ0dF/hfr2XD/IlSfdU3fcFM+tiZnupMk2yWOgsJ0iaFkLYM4TQEELYQ9IySSPqeOxFktZKujJx30xJpxfnKJnZ7ma2c2K/LnrvHPZFSfeGENZLWtuYgyjpVEl3NbW9uO373xYp1ys8O0i6wsx6SXpb0tOqTEusUyWT/CVJc+po5ypJ15rZIkmLVLn0rBDCY2b2iKQnJT2rykgb+aEfYXOdKOlUM3tLlf7yfVUGuq0xVZWcmzck/UTSrKr7/iDpd2Z2nKQzi3/XFlcKVkn6StW+KyQ9XBzHf4QQ3mzl8WDznSzph27bzcX25qYzG42XNMXMfhRCOLdxYwjhz2a2n6QHijznDZJOkfSye/zrkg41swuK+8YU28eq0te6q3JF8is1tk/Ve33zsC01j4elJQBgK2Nmv5L0qxDCgy183FRJM4pEU+B9JdcrPACQrRDC1zr7GICtDVd4AABA9rJMWgYAAKjGgAcAAGSPAQ8AAMgeAx4AAJA9BjwAACB7DHgAAED2GPAAAIDsMeABAADZY8ADAACy16KlJXbaaafQ0NDQToeCzrJ8+XKtXr3aOuK56EN56sg+JNGPcsW5CG1h3rx5q0MIff32Fg14GhoaNHfu3LY7KmwRhg4d2mHPRR/KU0f2IYl+lCvORWgLZvZMajtTWgAAIHsMeAAAQPYY8AAAgOwx4AEAANljwAMAALLHgAcAAGSPAQ8AAMgeAx4AAJA9BjwAACB7DHgAAED2GPAAAIDsMeABAADZY8ADAACyx4AHAABkjwEPAADIHgMeAACQPQY8AAAgewx4AABA9rbp7AMAAADt57e//W1p29VXXx3FBxxwQBSPGjUqio877ri2P7AOxhUeAACQPQY8AAAgewx4AABA9sjhaYHly5dH8XPPPRfFw4cP78CjAQCgtgcffLC07bXXXoviOXPmRPEVV1wRxePHjy+1MXHixM0+ttdffz2KL7vssiheuXJlFF9zzTWlNrbddtu6nosrPAAAIHsMeAAAQPYY8AAAgOwx4AEAANkjabkJqUJNF154YRSPHj06inv16hXFgwYNavsDq8P06dOjeJ999intc+ihh3bU4QAA2tA777wTxV27dm12//vuu6+0rWfPnlHsk5hHjhwZxT/72c9KbZx66qlRPGTIkGaPY926daVt/nleeeWVKP7HP/4RxWPHji21ccQRRzT7vI24wgMAALLHgAcAAGSPAQ8AAMheljk87777bmlbly7x2O7555+P4m9/+9vN3i9JH/nIR6J4wYIFUXzGGWdE8f3331/7YJ0NGzZE8ZQpU0r7rF69OorfeOONKN5hhx2ieLfddmvxcbxs3aM5AAAW8klEQVTfhBCi2Mxa3MakSZOi+JBDDints/POO0exLwjWt2/fKD7ooINKbey+++4tPraWmjBhQhT7hQUl6XOf+1y7HweAslrnpzVr1kTxsmXLSvvsu+++Ubxx48Yo7tGjRxTvvffepTaGDh0axSeccEIUf/jDH47in/70p6U2BgwYEMW77rprFL/66qtR3KdPn1Ib9eIKDwAAyB4DHgAAkD0GPAAAIHtZ5vD4fIwUP8e5ePHiKG5oaCg9plb+xapVq6LY18ORpCOPPDKKZ8yYEcW33nprFPv8HEkaMWJEFPu6BJ1V/2dr5utabLNN7Y/GrFmzovikk06KYp+PI5Xf30cffTSKu3fvHsWTJ08uteFzyYYNGxbFvhbGfvvtV2rDL4T717/+NYqfeeaZKE71Q3J4Oo8/x/m+6PvIXnvtVbON1uStoXP4nFTvxhtvjGJfI04q57r6c56vh+PPTZI0cODAKL799tuj2C8MmjoXffCDH4zi9evXR7Gvw+MX7Zbq/87jCg8AAMgeAx4AAJA9BjwAACB7W1wOT6qGjp9brjXXXGtdEUk68MADo7h3795R/MQTT5Qe86EPfSiKfa6En/M888wzS230798/ij/2sY9F8Xe+850oTs1N9uvXr7Stmp+bf/vtt0v7bLvtts22kRvfr/wcuJ+/XrRoUakNv76an0v+05/+FMW+v0jl193XqfDH4de7SW179tlno3jOnDlRnMol8s9z4oknRvGLL74YxU899VSpDbRPHszSpUuj+JJLLint43MM77rrrig+9thjo/jss88utdEROTs///nPo3jw4MGlfYYPH97ux/F+873vfS+KU+cRX9/GnxN8337zzTdLbfh99thjjyj2fWzHHXcsteHzA/3391tvvRXFPndWKq9r2RSu8AAAgOwx4AEAANljwAMAALLHgAcAAGSvw5OWayWP1iqo1F5+/OMfR/GoUaNK+9x2221R7Bfp9Amou+yyS6kNn8R3xBFHtOg46+ETxXJLUPaJcrViqXYi+x133BHFl19+eWmfcePGRbEv7lZPYu/KlSuj2L9XvsjW9ttvX2rDf4a6devW7P2+n0rSF77whSj2nzufCL127dpSG9VJ2//85z9L929t6ilY2tI/oPAJl1L5DyJ+//vfR7FPGE9ZuHBhFPsikP79uueee0pt+AKmrTFv3rwo/ta3vhXF/jiPP/74UhskLTcv1S99v/OLg7700ktR7BfklMpJyD5p2d+fOg7/GF/A1Z8X/B8Hpfg2/bnpgQceqNlGU7jCAwAAsseABwAAZI8BDwAAyF6Lc3iq5/HqKVzl5/1q5ej4uUdJmjZtWhT7BcruvPPOmsdRy8c//vEo9oXYUs/r5xpr5VZI5eJ1tXJ4/JyoVF5cbcOGDVHsCzm98MILpTaqi+L5vJEtXa0+lSpe6ReH9YveXXzxxVE8ZcqUUht+IbwBAwZE8SmnnNLEEddv3bp1UTxz5szSPn7BUV+ozuf9pBaO9Avd+twi3ydSeWDVOTypXJX2Vt0P6ikAWOt81RaF+FasWBHF559/fmkf3z997p8vKpjKe/AF3HwekF8s0i9aK0kPPfRQFPfp0yeK/Xv+5JNPltrwP+8nP/nJKPaL1D7++OOlNhDz52+/uKZU7qu+OKUvNtqjR49SG74f+rienDb//eTPA76fps4Tfpv/+X0O4uzZs2seV1O4wgMAALLHgAcAAGSPAQ8AAMjeZtXh8XN+rZkDP+uss6L44YcfLu3j5x99jQlf+2Hy5MktPg7vmmuuKW379a9/HcV+LtHPV7/22mulNq677roo9vVOPv3pT0exX6BSKi/65hcH9blFqfyLj370o5tu+xygzlRPzQmfs+P7TKpmjK9/dNRRR0XxH//4xyj276VUztHx+VheatFW/954PvdizJgxpX38Np8XceWVV0bxX/7yl1Ibft7c53n5+hmphVA7W3W/aM25x5+//Odg9erVpcf4nJU1a9ZE8ZIlS6LYL6YolRcM9jlZPi8idT7z78enPvWp0j7VPvCBD5S2+XOLPxf5PpDKJfG5Ij736zOf+UwU+1wxKc6NS+Xf5c6f81K5n94f/vCHKJ46dWoU77333lHsvzOk2ouF+s9U6tzlH7PddttFse8zqXzRWjW8fF9/+umnS/ukch1TuMIDAACyx4AHAABkjwEPAADIXotzeDZ33tw74IADoviGG24o7VOdbyKV5yd9jYnzzjuv1IavdVFLKu/Fz737ufWNGzdGsa9JIUkHH3xwFB944IFR7GtwHHrooaU2/PN4fq71lVdeKe2z8847b7qdWq+pPVXP07d0bSJJuuqqq6LY59v4PiVJI0eOjGKf1+Lvv/fee0tt+HyEffbZp9njTP0stebJ66kn402aNCmKfT5OqpaTzy/zc/w+lyi1Htduu+226XZnr9fmcz/qqRnj8298fatUvoGvK+LztPz7t//++5fauPvuu6PY17/xa/BVf1Yb+ferf//+pX2q+RpSUvl85mtA+fNCKtfPv0a+NlXPnj2jOJWjWZ0rlcp76yypfMJ61u2rlqo719Jz3oQJE0rbLr300ijed999o9i/jqlzgM+v8bk09dThqZU/6j+XqTUN/bZa50CfJyRJjz32WM1jlbjCAwAA3gcY8AAAgOwx4AEAANljwAMAALLXoqTlEEKU2OQT1nyCmlQ7IevrX/96FPviflI5ofSiiy6K4k984hNRnCpC5J/HF9l68MEHo9gvyChJb775ZhQfdNBBUTxs2LAoTiUD+4TjPffcM4rnzp3b7HFK5eRCX9zMJ4qlihdWJ+C2RfJ5S9RaQLYWn6DpE8FTiY8+8X3QoEFR7F+jQw45pNSG3ydViK1aKkGvlnreC9+/f/nLX0bx6NGjo/ipp54qtbHTTjtFcffu3aPYf5ZTP0tnJi2vWbNG06dP3xT7P1Q4/fTTS4+ptaCiTx72r5FUTt72i7D65/CJ0ZI0ePDgKPZ90y+eOG7cuFIbPrHTJxT780wqaTVVoLPayy+/HMWpQoz+fffHMX/+/ChOFfTcUrVmAdrW8Au/nnvuuVHsFz6Wygnn/pxaawFOqXye9I/xCcj1vB7+POH7XeoPbvy5xz/GJ1OnCjOmPmcpXOEBAADZY8ADAACyx4AHAABkr0U5PBs3btTf//73TbHPJUgVKvL5JH7+zc8t+jwZqVxY0D/GL4x3xhlnlNrwc3x+8UTfhi/kJJXnH32uxJw5c6J49913L7Xh+WJnI0aMiOIFCxaUHjNq1Kgo9rklfm524MCBpTaq53w7Oodnc9WzWKjnFy30xat8LpUv3icp6vv1SOUSvfjii1Hs3//nn38+ilMF4/yx3XzzzVHsi2ymFv70+WU+F8N/HlJF+Krn+Du6D/Xo0UPHHHNM8likdIE7v8hqLb7IoFTOc1i2bFkU++NI5Sz4NnzscxhS/cj3G9+Gz+lI5Vj5c6DPafJ9JJUHUut993luqcVD582bt+l2qp9tyXxR11mzZkWxXxhWkmbMmBHFvl/6gqY+N1Qqv3f+/fef33qKCHq1igqm+D7j+38qF9C364/dH4f/2VOPaQpXeAAAQPYY8AAAgOwx4AEAANlrUQ5Ply5donlcn+eSWkzRL+Ln54n9nG6qfoavU+HnzcePHx/Fxx9/fKkNX1fF/22/n2tesmRJqQ2f57Fw4cIo9gsupubv/fP6+Uj/vL5NSbrnnnui2Ndk8HlTqXyU6gUJ65mbbSsbNmyIFua85ZZbovv79etXeox/DXzdBp/nksoD87ljvl7IokWLojg15+3rHd1xxx1RXM+ctz9Wn2vhH5Oq5eQ/d74NX7vpb3/7W6kN3+987HM+fE0WSfrqV7+66XZHL/poZtExnnTSSdH9Pm4v/nXz70XqHOBfW9/XUv3X87kQvk0fb8l5etUL2frvh440e/bsKL7kkktK+/jPlv9cVNemktILrvrPtM/b9O+V/85I7eNzWPziwKk2/GP8a+/7tv9ekcrnVb+PP87U95nPWfOx/0ylXtPDDjustC2FKzwAACB7DHgAAED2GPAAAIDstTiHp3rOztcMSa1n8eqrr0Zx7969o9jXUPFr0UjleVO/Fs2KFSuiOLUOks+38bVJfM0NPxcrlXNh/Fy7r+2Smr/32/ycp//5U2vg+HWOXnrppSj2+RSpfJTqPJjNXduqJbp16xatY+X7kI+lcq2LXXbZJYp93k8qB8L3Q78ukH9fUvPE/nW87LLLorihoSGKUzVFauW6+OdIvXe+L/v+4ONU/kaqlkW1/fffP4pT78tpp53W7HG2p65du0Y5B/799bFU/iz5PBifP5X67NVaH82/1qlzgD9P+DZTuRJePf2k1v3+eXzsjz3Vd2u9Hr7NVC5JdT5hPT97W3nrrbeiHNJvfvOb0f2p2i5+fTUf+z6UWl/St+u/N1N5e16t96416/j57wF/HKnziK/N5M8rvu5Y6v31+bP+vOn7YWotrcMPP7y0LYUrPAAAIHsMeAAAQPYY8AAAgOwx4AEAANlrUdKyTxT0SU6polE+ScknKPlk4T59+pTa8AmkvpCaf45UkTRfvNAn4NVKJpbKRZN8ATiffJYqoucLMfrXwx9HKunNJ377JF2/eKR/TilONmtNgltrde3aNXodx4wZ0+I2fDKp//lTC276PuTfX5/kmkpY9Al5Polv3bp1zT6HVE6G9wmsvl+mEp/98/jH+KJzqQRs/577z64v7ti/f/9SG9X9MLU4ZUfyx9+ZBezQeh1ZBHXVqlWaPHnyptgnVKfOvbWKQtbznejPT/6z6O9PJQv718k/r0/sTZ3j/c/iz1f+HJBKOPd/UOT/gGbXXXeN4tR3ov/O92MCf+z+/CfVl+gtcYUHAAC8DzDgAQAA2WPAAwAAsteiCVMzi3Ib/Bzn4sWLS4/xuRC+0OD69euj2BcykspFs2oVp0oVVfP5NbUKz/nF11LPW2sBxhQ/p+vnXn2hJj9HKpXnVn0ekJ/z9YUaU8+7NfFzun7+NjWfW13cDADMLMqF8bkz/jwrlXPVfB6MP6/6nBap/H3mz9c+r8+f31PP47+b/P2p3KNaC936YqOpYp4jR46M4ksvvTSKZ86cGcWpApj+NfU5mj4H0X93t8TW+60HAABQJwY8AAAgewx4AABA9jar6MHnP//5KE7l1ixZsiSK/Zymr5mzdOnSUhs+J8PPR9azYJuvVTJgwIAorrWYqFTOHfE1BXwbrcmT8fOXqZpCfs7Tz73640y9HgDwftavXz9deOGFm+JVq1ZF9995552lx/jvL39u9d+BqZxU/31V6/ss9b3qt9Wqy5Nqwy/K7B9z9tlnR/FZZ51VaqOWadOmRXGqDk+tWn0+NzZVD6heXOEBAADZY8ADAACyx4AHAABkr00XLknlrAwcOLDZGLFaeUIAgLY3adKkKE7VVZs4cWIUX3/99VHsa9f4unNSOSfV54v62nV+XazUsdWqoZNq44ILLoji888/v7TP5lqwYEEUr1ixorSP/87z6wf27ds3ileuXFlqo968Hq7wAACA7DHgAQAA2WPAAwAAsseABwAAZK9Nk5YBANga+QJ4vsCrJJ1zzjnNxl6qeOH8+fOj+PHHH4/iZ555JorXrVtXasMnIftifePGjYvi8847r9njrEeqeGGt4ro/+MEPoji1IKtP0vbFGnv16hXFQ4YMafY5m8MVHgAAkD0GPAAAIHsMeAAAQPbI4QEAvO+1ZrHnWo466qi6tm0NWvP6jB07th2OpPW4wgMAALLHgAcAAGSPAQ8AAMgeAx4AAJA9BjwAACB7DHgAAED2GPAAAIDsMeABAADZY8ADAACyx4AHAABkjwEPAADIHgMeAACQPQY8AAAgewx4AABA9hjwAACA7DHgAQAA2bMQQv07m62S9Ez7HQ46yZ4hhL4d8UT0oWx1WB+S6EcZ41yEtpDsRy0a8AAAAGyNmNICAADZY8ADAACyl+2Ax8z+y8yeMLMFZvaomX28DdqcbWZDN3cfbD3oR2iN9ug3VW2PNLMZbdUeOp+ZHW9mwcz2rXP/5Wa2U2L7hhY+b4v2b6adL5vZbm3RVnvaprMPoD2Y2WGSPivpkBDCxqJjfKCTDwtbGfoRWmNL7jdmtk0I4e3OPg6UnCzp3uL//+7kY2mNL0t6XNILnXwczcr1Ck8/SatDCBslKYSwOoTwgpldZGZzzOxxM/uFmZm06bfpH5rZw2b2lJmNKLZ3M7ObzGyRmd0qqVvjE5jZVWY2t/gt7uLO+CHR7uhHaI2m+s1yM7vYzOab2cLG3+bNbHszm1L0m0fM7Lhie4OZ3VPsP9/M/sU/kZkNKx6zl5kNMbO7zGyemc00s37FPrPNbKKZzZU0vuNeBtTDzHaQNFzSVyWdVLV9ZPHe/c7MnjSzGxrPNVX7dDOz283s64l2zynOUwuaO7eY2eXF+eevZta32DbYzB4sHnurmX2oqe1mdoKkoZJuKK5mdmvquTpdCCG7f5J2kPSopKckTZZ0RLG9d9U+0yQdW9yeLeknxe3PSJpV3D5b0pTi9kGS3pY0tLotSV2Lxx9U1dbQzn4N+Ec/4t8W12+WSzqzuP0tSb8qbn9f0inF7V7F47aX1F3SdsX2j0qaW9weKWmGpH+RNE/ShyVtK+l+SX2LfcZU9bnZkiZ39uvCvyb7y5ck/W9x+35JQ6re5/WS+qtyceIBScOr+lKDpFmSTqtqa0Px/9GSfiHJisfOkHR44rmDpC8Vty+S9PPi9oKqfnuJpIk1tm8V56ssr/CEEDZIGiLpDEmrJP3GzL4s6Ugze8jMFko6StIBVQ+7pfh/niodSZIOlzS9aHOBKm92oxPNbL6kR4p29m+XHwadhn6E1mim30jp/nG0pPPM7FFVvji203uDmF8W/ey3ivvGfqp8oR0bQlghaaCkQZL+UrRzgSpflI1+03Y/IdrYyZJuKm7fVMSNHg4hPBdCeFeVQXRD1X23Sbo2hHB9os2ji3+PSJovaV9VBs3eu3qvb0yXNNzMekrqFUK4q9h+naTDm9pe90+5Bcgyh0eSQgjvqHLymF2cML6hym/XQ0MIz5rZ/6hyYmm0sfj/HdV4XcxsgKTvShoWQlhrZlNdW8gE/Qitkeg3Y4u7Uv3DJP17CGFxdRtF31op6WOq/Jb+ZtXdL6rSVw5WJW/CJD0RQjisiUN6fTN+HLQTM+utyi9NB5pZUOVKbzCzc4pdNlbt7s8p90kabWY3huIyS3XTkiaEEK5p4SFlXZgvyys8ZjbQzKpHs4MlNZ5MVhdzpifU0dTdkr5YtDlIlS86SeqhyglkvZntIumYNjlwbFHoR2iNJvpNcxV9Z0o6syoX7OBie09JLxa/3Z+qypdho3WS/lXSBDMbqUq/7GuVhGmZ2bZmVn3lEVumEyRNCyHsGUJoCCHsIWmZpBF1PPYiSWslXZm4b6ak04tzlMxsdzPbObFfF713DvuipHtDCOslrW3MQVSl793V1Pbi9muSdqzjmDtVrld4dpB0hZn1UiVf4mlVLi+vUyWT/CVJc+po5ypJ15rZIkmLVLkMrRDCY2b2iKQnJT2rykgb+aEfoTWa6jefbWL/SyVNlLTAzLqo8oX3WVXyf242s9Mk3SF3lSaEsNLMPivpdkmnq/LFNamYetimaPOJNv7Z0LZOlvRDt+3mYns905DjJU0xsx+FEM5t3BhC+LOZ7SfpgWIcvUHSKZJedo9/XdKhZnZBcd+YYvtYSVebWXdJSyV9pcb2qcX2NyQdFkJ4o45j73AsLQEAALKX5ZQWAABANQY8AAAgewx4AABA9hjwAACA7DHgAQAA2WPAAwAAsseABwAAZI8BDwAAyN7/AwX6eAL2ymwMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 16 Axes>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
    "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']\n",
    "\n",
    "plt.figure(figsize=(10,10))\n",
    "\n",
    "for i in range(16):\n",
    "    plt.subplot(4,4,i+1)\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "    plt.grid(False)\n",
    "    plt.imshow((255 * train_images[i]).astype(np.int).squeeze(), cmap=plt.cm.binary)\n",
    "    plt.xlabel(class_names[train_labels[i]])\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "C8Jho-4DqFJ6"
   },
   "source": [
    "### Generator\n",
    "For training GANs we need to further define our generator and discriminator network. We start by defining our generator network, which should map from our noise + label space into the space of images (latent-vector size --> image size). Adding the label the input of to both the generator and discriminator should enforce the generator to produce samples from the according class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0WSwlefNqFJ6"
   },
   "source": [
    "#### Task\n",
    "Design a meaningful generator model! Remember to check the latent and image dimensions.  \n",
    "You can make use of the 'DCGAN guidelines'. Use a meaningful final activation function!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "f3pnZBM9qFJ8"
   },
   "outputs": [],
   "source": [
    "def generator_model(latent_size):\n",
    "    \"\"\" Generator network \"\"\"\n",
    "    latent = layers.Input(shape=(latent_size,), name=\"noise\")\n",
    "\n",
    "    return keras.models.Model(latent, z, name=\"generator\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AxNmxzWdqFJ9"
   },
   "source": [
    "Build and check the shapes of our generator!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "cllaLqb5qFJ9",
    "outputId": "60036b2e-6343-4f61-b0f3-c367e4072346"
   },
   "outputs": [],
   "source": [
    "latent_size = 100\n",
    "g = generator_model(latent_size)\n",
    "g.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1xyByAO3qFJ-"
   },
   "source": [
    "We can further plot the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "RsseR3gDqFJ-",
    "outputId": "cfb0e6ac-c250-4076-e11b-39ebf0deb1dc"
   },
   "outputs": [],
   "source": [
    "keras.utils.plot_model(g, show_shapes=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8-E3K81SqFJ-"
   },
   "source": [
    "### Discriminator\n",
    "The task of the discriminator is to measure the similarity between the fake images (output of the generator) and the real images. So, the network maps from the image space into a 1D space where we can measure the 'distance' between the distributions of the real and generated images (image size --> scalar). Also, here we may add the class label to the discriminator.\n",
    "\n",
    "#### Task\n",
    "Design a power- and meaningful discriminator model!  \n",
    "\n",
    "Remember that you can make use of the DCGAN guidelines (use convolutions!) and check the image dimensions (compare Sec. 18.2.3). We need a `softmax` as last activation function in the discriminator!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1IRfIv3YqFJ_"
   },
   "outputs": [],
   "source": [
    "def discriminator_model(drop_rate=0.25):\n",
    "    \"\"\" Discriminator network \"\"\"\n",
    "    image = layers.Input(shape=(28,28,1), name=\"images\")\n",
    "\n",
    "    return keras.models.Model(image, x, name=\"discriminator\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "q_R8G8zZqFJ_",
    "outputId": "41d0aa92-82cc-40d3-d792-1382c94f1f54"
   },
   "outputs": [],
   "source": [
    "d = discriminator_model()\n",
    "d.summary()\n",
    "d_opt = keras.optimizers.Adam(lr=2e-4, beta_1=0.5, decay=0.0005)\n",
    "d.compile(loss='binary_crossentropy', optimizer=d_opt, metrics=[\"acc\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WbEIOHHgqFJ_"
   },
   "source": [
    "We can further plot the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "ouMCmFGyqFJ_",
    "outputId": "ff2fffc0-d3ca-484b-dd8d-c406d8d6b322"
   },
   "outputs": [],
   "source": [
    "keras.utils.plot_model(d, show_shapes=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1jJONRzcqFKA"
   },
   "source": [
    "### Training preparations\n",
    "After building the generator and discriminator, we have to compile it. But before, we have to freeze the weights of the discriminator. (Remember that we have to fix the discriminator weights for training the generator because we want to fool the discriminator by drawing excellent images, not by making our discriminator a worse classifier)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7vgJsz8iqFKA"
   },
   "outputs": [],
   "source": [
    "def make_trainable(model, trainable):\n",
    "    ''' Freezes/unfreezes the weights in the given model '''\n",
    "    for layer in model.layers:\n",
    "        # print(type(layer))\n",
    "        if type(layer) is layers.BatchNormalization:\n",
    "            layer.trainable = True\n",
    "        else:\n",
    "            layer.trainable = trainable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mqxKb1iJqFKA"
   },
   "source": [
    "Note that after we compiled a model, calling `make_trainable` will have no effect until compiling the model again.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EOkbdZFHqFKA"
   },
   "outputs": [],
   "source": [
    "make_trainable(d, False)  # freeze the critic during the generator training\n",
    "make_trainable(g, True)  # unfreeze the generator during the generator training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c8jDE6QBqFKB"
   },
   "source": [
    "We build the pipeline for the generator training by stacking the generator on the discriminator (with frozen weights)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_VHegmLDqFKB",
    "outputId": "377c2e48-7b61-4cd3-fd97-118f1b567854"
   },
   "outputs": [],
   "source": [
    "gen_input = g.inputs\n",
    "generator_training = keras.models.Model(gen_input, d(g(gen_input)))\n",
    "generator_training.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 312
    },
    "id": "vK63LBbRqFKB",
    "outputId": "1adfac20-9227-41a4-c2c0-07bfe7da805b"
   },
   "outputs": [],
   "source": [
    "keras.utils.plot_model(generator_training, show_shapes=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "I41Pj5aPqFKB",
    "outputId": "eae3757c-82a7-494e-91eb-06019331d139"
   },
   "outputs": [],
   "source": [
    "g_opt = keras.optimizers.Adam(lr=2e-4, beta_1=0.5, decay=0.0005)\n",
    "generator_training.compile(loss='binary_crossentropy', optimizer=g_opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WhPhsGKxqFKC"
   },
   "source": [
    "We pre-train the discriminator using 5000 real and 5000 fakes samples (pure noise, since the generator wasn't updated yet)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UBZULrGqqFKC",
    "outputId": "d9d9bcca-f476-4268-dd06-56970962acac"
   },
   "outputs": [],
   "source": [
    "ntrain = 5000\n",
    "no = np.random.choice(len(train_images), size=ntrain, replace='False')\n",
    "real_train = train_images[no,:,:,:]   # sample real images from training set\n",
    "noise_gen = np.random.uniform(0,1,size=[ntrain, latent_size])\n",
    "generated_images = g.predict(noise_gen)  # generate fake images with untrained generator\n",
    "\n",
    "X = np.concatenate((real_train, generated_images))\n",
    "y = np.zeros([2*ntrain, 2])   # class vector: one-hot encoding\n",
    "y[:ntrain, 1] = 1             # class 1 for real images\n",
    "y[ntrain:, 0] = 1             # class 0 for generated images\n",
    "\n",
    "# - Train the discriminator for 1 epoch on this dataset.\n",
    "d.fit(X,y, epochs=1, batch_size=64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lEABiVwzqFKC"
   },
   "source": [
    "####  Task\n",
    "Set up the adversarial training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "QQsqSquUqFKD",
    "outputId": "0026f531-51ef-4e79-dd40-daba7b882f7d"
   },
   "outputs": [],
   "source": [
    "losses = {\"d\":[], \"g\":[]}\n",
    "discriminator_acc = []\n",
    "batch_size = 64\n",
    "nsamples = len(train_images)\n",
    "iterations_per_epoch = nsamples / batch_size   # Number of training steps per epoch len(train_images)\n",
    "epochs = 15\n",
    "\n",
    "iters = 0\n",
    "\n",
    "for epoch in range(epochs):\n",
    "    print(\"Epoch: {0:2d}/{1:2d}\".format(epoch, epochs))\n",
    "    perm = np.random.choice(nsamples, size=nsamples, replace='False')\n",
    "\n",
    "    for i in range(int(iterations_per_epoch)):\n",
    "        \n",
    "        # Create a mini-batch of data (X: real images + fake images, y: corresponding class vectors)\n",
    "        image_batch = train_images[perm[i*batch_size:(i+1)*batch_size],:,:,:]    # real images   \n",
    "        noise_gen = np.random.uniform(0.,1.,size=[batch_size, latent_size])\n",
    "        \n",
    "        # Generate images using the generator (set correct labels)\n",
    "        generated_images = g.predict(noise_gen)\n",
    "\n",
    "        X = np.concatenate((image_batch, generated_images))\n",
    "        y = \n",
    "\n",
    "    \n",
    "        # Train the discriminator on the mini-batch\n",
    "        d_loss, d_acc  = d.train_on_batch(X,y)\n",
    "        losses[\"d\"].append(d_loss)\n",
    "        discriminator_acc.append(d_acc)\n",
    "    \n",
    "        # Create a new mini-batch of data (X_: noise, y_: class vectors pretending that these produce real images)\n",
    "        X_ = \n",
    "        y_ =\n",
    "    \n",
    "        # Train the generator part of the GAN on the mini-batch\n",
    "        g_loss = generator_training.train_on_batch(X_, y_)\n",
    "        losses[\"g\"].append(g_loss)\n",
    "\n",
    "        iters +=1\n",
    "\n",
    "        if iters % 1000 == 1: \n",
    "          # Plot some fake images    \n",
    "          noise = np.random.uniform(0.,1.,size=[16,latent_size])\n",
    "          generated_images = g.predict(noise)\n",
    "          plt.figure(figsize=(5, 5))\n",
    "          \n",
    "          for i in range(4):\n",
    "              plt.subplot(2,2,i+1)\n",
    "              plt.xticks([])\n",
    "              plt.yticks([])\n",
    "              plt.grid(False)\n",
    "              img = plt.imshow((255 * generated_images[i]).astype(np.int).squeeze(), cmap=plt.cm.binary)\n",
    "      \n",
    "          plt.suptitle(\"Iteration %i\" %iters)\n",
    "          plt.savefig(\"./fake_fMNIST_iteration_%.6i.png\" % iters)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<em>After roughly 3,000 iterations, we can recognize the basic shapes of the clothes. Above iteration 10,000, the images are of good quality.</em>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ngULajAvqFKD"
   },
   "source": [
    "## Plot the training history\n",
    "Plot the loss of the discriminator and the generator as function of iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 497
    },
    "id": "ev-wq-QvqFKD",
    "outputId": "c565f878-82bc-4888-e362-cc2f949ff2a2"
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(10,8))\n",
    "plt.semilogy(losses[\"d\"], label='discriminator loss')\n",
    "plt.semilogy(losses[\"g\"], label='generator loss')\n",
    "plt.ylabel(\"loss\")\n",
    "plt.xlabel(\"iterations\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1hZ724HIqFKD"
   },
   "source": [
    "Plot the accuracy of the discriminator as function of iterations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 497
    },
    "id": "sMc75W9BqFKD",
    "outputId": "edd58065-b044-4725-fa14-0c8e10eb9d64"
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(10,8))\n",
    "plt.semilogy(discriminator_acc, label='discriminator')\n",
    "plt.ylabel(\"accuracy\")\n",
    "plt.xlabel(\"iterations\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Pd28_f_AqFKD"
   },
   "source": [
    "| Further questions |\n",
    "|:-------------| \n",
    "|Check the loss and the generated images.\n",
    "Does the image quality correlate with the discriminator or the generator loss?\n",
    "Is the generator able to produce all classes of the dataset?\n",
    "How can you improve the performance?|"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "Vanilla_GAN.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}