{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise 4.2 - Solution\n",
"## Linear regression\n",
"In this task we will design and train a linear model using [Keras](https://keras.io/).\n",
"\n",
"### Tasks\n",
"1. Complete the implemetation of the `LinearLayer`\n",
"2. Define a meaningful objective\n",
"3. Implement gradient descent and train the linear model for 80 epochs."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"from tensorflow import keras\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"layers = keras.layers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Simulation of data\n",
"Let's first simulate some noisy data"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"x.shape: (100, 1)\n",
"y.shape: (100,)\n"
]
}
],
"source": [
"np.random.seed(1904)\n",
"x = np.float32(np.linspace(-1, 1, 100)[:,np.newaxis])\n",
"y = np.float32(2 * x[:,0] + 0.3 * np.random.randn(100))\n",
"print(\"x.shape:\", x.shape)\n",
"print(\"y.shape:\", y.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Implement linear model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we have to design a linear layer that maps from the input $x$ to the output $y$ using a single adaptive weight $w$:\n",
" \n",
"$$y = w \\cdot x$$\n",
"\n",
"### Task 1\n",
"Complete the implementation of the `LinearLayer` by adding the linear transformation in the `call` function."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"class LinearLayer(layers.Layer):\n",
"\n",
" def __init__(self, units=1, input_dim=1): # when intializing the layer the weights have to be initialized\n",
" super(LinearLayer, self).__init__()\n",
" w_init = tf.random_normal_initializer()\n",
" self.w = tf.Variable(initial_value=w_init(shape=(input_dim, units), dtype=\"float32\"),\n",
" trainable=True)\n",
"\n",
" def call(self, inputs): # when calling the layer the linear transformation has to be performed\n",
" return tf.matmul(inputs, self.w)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Build a model using the implemented layer."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"model = keras.models.Sequential()\n",
"model.add(LinearLayer(units=1, input_dim=1))"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"sequential\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"linear_layer (LinearLayer) (None, 1) 1 \n",
"=================================================================\n",
"Total params: 1\n",
"Trainable params: 1\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n",
"None\n"
]
}
],
"source": [
"model.build((None, 1))\n",
"print(model.summary())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Performance before the training\n",
"Plot data and model before the training"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAm/UlEQVR4nO3dfXRV9Zkv8O+TCAkpiBiaDIUmAVux+AIIKKKC0MxUnTv40hdhUoovLG516rrTSqd2Ml3YsbG3lXvtslqnFB0diMWXGa1WvUqGxNax9gqzUHkRhQgYriUYHSVCIibP/WOfA/uc7H3O3vvs13O+n7XOysk+++zzZHM4z/n9fs/vt0VVQUREFDdlUQdARERkhQmKiIhiiQmKiIhiiQmKiIhiiQmKiIhi6YSoA/Bi7Nix2tDQUNAxPvroI3zqU5/yJ6AAJSVOIDmxMk7/JSVWxuk/P2LdvHnzu6r66SEPqGribjNmzNBCtbe3F3yMMCQlTtXkxMo4/ZeUWBmn//yIFcAmtfisZxcfERHFEhMUERHFEhMUERHFEhMUERHFEhMUERHFEhMUERHFEhMUEVEJa20FGhqAsjLjZ2tr1BEdl8iJukREVLjWVmD5cuDwYeP3vXuN3wGgqSm6uNLYgiIiKlHNzceTU9rhw8b2OGCCIiIqUfv2udseNiYoIqISVVfnbnvYmKCIiEpUSwtQVZW5rarK2B4HTFBERCWqqQlYvRqorwdEjJ+rV8ejQAJgFR8RUUlraopPQsrGFhQREcUSExQREcUSExQREcUSExQREcUSExQREcUSExQREcUSExQREcUSExQREbmWvkzHggXzArtMByfqEhGRK5mX6ZDALtPBFhQREbkS1mU6mKCIiMiVsC7TwQRFRESuhHWZDiYoIiJyJazLdDBBERGRpXSlXlkZMir1Mi/ToYFdpoMJioiIhkhX6u3dC6jiWKWeOUnt2QNs3Pg89uwJ5pIdTFBEREXOriWUS1iVerkwQRERxUxbW43rhGInX0vITliVerkwQRERxUhrK7Bq1WTXCcWO15ZQWJV6uTBBERHFSHMz0N9fnrGtkK41ry2hsCr1cok8QYnIZ0WkXUS2i8g2EfkfUcdERBQVv7vWvLaEMiv1EFilXi6RJygAnwC4SVWnAJgN4G9EZErEMRERRcLvrrVCWkLpSr3BQQRWqZdL5AlKVd9R1f9M3T8EYAeA8dFGRUQUjZYWoKJiIGNbIV1rcWgJeRV5gjITkQYA0wH8MeJQiIgi0dQErFix09eE4rQl5KUcPUiiqtFGkCIiIwE8D6BFVf/N4vHlAJYDQG1t7Yz169cX9Hq9vb0YOXJkQccIQ1LiBJITK+P0X1JiZZz22tpqsGrV5IwCjYqKAaxYsRONjd22z/Mj1vnz529W1ZlDHlDVyG8AhgF4FsB3nOw/Y8YMLVR7e3vBxwhDUuJUTU6sjNN/SYmVcdqrr1c1Ctszb/X1uZ/nR6wANqnFZ33kXXwiIgDuBbBDVf931PEQEZWiOEzMzRZ5ggJwPoAlABaIyJbU7dKogyIiKiVxmJibLfJLvqvqCwAk6jiIiEpZS4v5Mu6GsCfmZotDC4qIiCKWrxw9igq/yFtQREQUD01N1iXo6QVn062r9PqAADA+wFmrbEERERWBIFs4UV16gy0oIqKEy9XC8WPFiKgq/NiCIiJKuKBbOFFV+DFBERElhF03XtAtnKguvcEERUSUALmujBt0CyeqBWeZoIiIEiBXN55VC2fYMKC317+iiSguvcEERUTkkyAr6XJ142W3cKqrjZ89Pf5cNj4qTFBERD7I1QXn5LnmxNbWVjNkn3zdeOYWzsiRwMcfZ+4XRlm435igiIh84LWSziqxrVo1eUhic1OoEMeFX71ggiIiQuHdc16TglVi6+8vH5LY3BQqxHHhVy+YoIio5BXSPZfmNSm4SWxOCxXytbbiduVcO0xQRFTy/Jjo6nWuUBCtnVytLT+ScViYoIiopFi1HvwYs/E6V8gqsVVUDBQ8CdautRXVunpecC0+IioZbW01uOOOoWvWnXyyUZKdzW0rxm418HzPAYwEsW+f8Zpf//pONDVNcXcgh5JUQMEWFBEVNXOL6cc/Ps2y9QAEv5RPdsvthhuO/56ebJtu7TQ2dvv3wlmSVEDBBEVERSt7vGVw0Poj7733glnKJ52URIAlSzLHfe65J5pxoKjW1fOCCYqIipbVeIuVurrcFXJeqt7MyREwElEuYY0DRbWunhccgyKiouVkXCVf68HrtZacJkezsMaBvIyVRYEtKCIqWnbjKuXlzlsPXqvevCSbOI4DRYkJioiKlt14ywMPOF+V22vVm9tkE9dxoCgxQRFR0coeb6mt7XM93uK16s0qOYoYP+vrgeuvT8Y4UJQ4BkVERc083tLR8RIuuugiV89vackcgwKctXas5je1tDAJucEWFBFRDvmq3nJV+EVxkb9iwhYUEVEedlVvXiv8yBm2oIiIPIp6XbukrEruFVtQREQeRbmuXSm03tiCIiLyKMp17aJuvYWBCYqIyKMo17VL0qrkXjFBERF5FOW6dklaldwrJigiogJEVUqepFXJvWKCIiKKMbtKvSStSu4VExQRhc6P8uhiL7EGhl7PKvu6UcU+EZgJiohCle9DN6xjJEEpVOrlwgRFRKHy+qFrbjEtXVoaE2RLoVIvF07UJaJQefnQzZ6UOjDg/hh+CXOCbF3d8SvyZm8vBbFoQYnIfSLSLSJbo46FiILlpTzazaXbgxZmt1spVOrlEosEBeB+ABdHHQQRBc/Lh64fl253y64bL8xut1Ko1MslFl18qvo7EWmIOg4iCp6X6yTZdXWVlxsVbH5faylXN17Y3W52K6mXgri0oIiohLgtj/bj0u1u5OrGK/VutzCJqkYdAwAg1YL6raqeYfP4cgDLAaC2tnbG+vXrC3q93t5ejBw5sqBjhCEpcQLJiZVx+i+MWNvaarBmzSR0d1egpqYfy5Z1orGx29UxnMa5YME8qMqQ7SKKjRuf9yUWP+KMAz9inT9//mZVnTnkAVWNxQ1AA4CtTvadMWOGFqq9vb3gY4QhKXGqJidWxlmYdetU6+tVRYyf69bFN9ZsTuOsr1c1Zlhl3urrg4zuuKScT1V/YgWwSS0+69nFR0SO2U2QbWuriTo0X7EbLx5ikaBE5NcA/gBgsoh0ich1UcdEREPZjc2sWTMpmoACUurVc3ERiwSlqotVdZyqDlPVCap6b9QxEZUytyXW3d0VYYUWmmJf5y4JYlFmTkTx4aXEuqamH0BlaDFSaYhFC4qI4sNLifWyZZ3hBWghu8V3ww3O1sorhRXRk4wtKCLKkGulBLtJtuPHdwOYElqMZlYtvnvuOf64uQU4fnzu5wW1ph55wxYUEWXIt1Ze3MZmnKzTZ7VWXqlfyiIJmKCIKEPSSqydroGXvV+pX8oiCZigiChDECXWbsZ63I4LOV0DL3s/L6uqU7iYoIhoCD+78dxc/dbLlXKtWnzZrFqASWspliImKCLyLN3aWbBgnm1rx81Yj5dxIasW3/XX528BcjJu/LGKj4g8yayCE9sqODdjPV7HhbxekqKUL2WRBGxBEZEnTls7bsZ6OC5EZkxQROSJ09aOm7EejguRGRMUEXnitLXjZqyH40JkxjEoIvKkpSVzJQbAvrXjZqyH40KUxhYUUYSSvBZcZmtH2doh3zFBEUXEy5yfuEnPl9q48flYLHtExYUJiigiXAuOKDcmKKKIcC24/JLcBUqFY4IiikhS5vxElSSKoQuUCsMERRSRJMz5iTJJsAuUmKCIIpKEOT+FJIlCW17sAiXOgyKKUNzn/HhNEn5crbauznie1XYqDWxBEZEtr+NkfnTPJaELlILFBEVEtt1xXpOEH91zSegCpWCxi4+oxDnpjmtuNpJLXZ2RnPIlCb+65+LeBUrBYguKqMTl647zcnVdNy0vznUiO0xQRCUuiGo5p91znOtEuTBBESVAkK0MvyYMZ8cI5G95ca4T5ZI3QYnIBhGZGkYwREkVZAIJqpWRjnnvXqOVY+a2Wq6trcZTjJzrRLk4aUF9D8DPROSfRWRc0AERJU3Q3VR+tjLSSUkEWLLkeCGD6vEk5aVabs2aSZ5iTMpyTxSNvAlKVf9TVecD+C2A/yMiK0VkRPChESVD0N1UfrUyzIkUMJKSmaqRnLxcNqO7u8JTjJzrRLk4GoMSEQGwE8A9AG4E8KaILAkyMKKkCLqbyq9WhlUizeY15pqafsvt+WLkXCfKxckY1H8A2A/gDgDjAVwN4CIA54jI6iCDI0qCoLup/GplOEk+XmNetqzTc4xeytipNDhpQS0HMF5V/1xVf6Cqv1XVXap6I4ALA46PKPaC7qbK18pwWqCRL/kUEnNjYzdbQuQ7J2NQ21Sze6uP+Uuf4yFKnDC6qexaGW4KNKwSaSGFEU5jJPKqoHlQqtrpVyBESRbVh7ObAg2rRLp2rZHYmFAojjhRlyjB8hVoeJk8SxQXTFBECZarQIPLCFHSxSJBicjFIrJTRHaJyM1Rx0NUqLAWQM1VoMFlhCjpIk9QIlIO4G4AlwCYAmCxiEwJ9EWPHEFZfz/Q1wf09wMffwwcPQoMDBh9H7Y1IUT5+dVyaWursU1y6QS4ZAkwYgRQXT20QIPLCFHSxeF6UOcA2JUuuBCR9QAuA7A9sFc84wzM7XRQ3yFi3MrK8t/3sq95e/bjqd9nHT4MjBrl7DXdPOZl3zx/7+Q//cn45HRybry+nptzbHOccW+8AbzxhqNz88KLZVj/kODgu4KTx5Zh8V8L5s7Lfd6e+47ggsOCQZRBIcbtsODJm8rQNMFZrE8/I9jw00GMPvoKzoBA9wp+tkwwukugUoZVKwXD+8pwCgTaI6isFKz5X2W4/IrUMd4WnPMZwdv7jdc3xzJhvAA9Ht4z5m1EIRD7CvKQAhD5CoCLVXVZ6vclAM5V1W/ZPWfmzJm6adMm7y/6y1+ic/NmTJo40fiKa76lW1D57pt/93IMh/se7O7Gp8eOzX0cL4/ZxW/1tzk8fn9fHyqGD899fqyeb3deBge9/xtT8EwJa1AEZV6/IHn9IufhGB98+CFGjxnjPRaHX9Yc72vz2NtdXfhsXZ2nL2Bhfxn848AAzi2w2kZENqvqzOztcWhBOSIiy2FMGkZtbS06Ojq8H2zyZPSOH499I0f6E1yAent7MTIBcQIBxmpKXJL+fXBwyP32jWNx152n4OOPy9LtFowYfhTfuuENzL3w4LH9P+rtxaeqqiDm42a/xuAgvvPtqejpGY4yDB47XhkGMfbkPvz0J69aP1cVP/iHKXj//cznCRRjx/ThLy/dj0cenoBPjsqx7cOHfYKvXvk2pk19/9hr/+AfTgeAIcfIjsX8WDkG8L2/ez0jph3bR+I/XhiLjw6V4cRRR3H+nIM4bfKh4+ctHTsw9HykviCYtwOAmLrB0/eP9vdj+LBh9sfyeP/YscyxWMSV8356/8FBDJSV4f3eXsu/If1ek/QXo+z7uV4j9e8mpuOYX99ye9bxzccaNziIgezjZr+e+X6Eqq6/Hh3jxwdy7Di0oM4DcIuqfin1+/cBQFV/bPecgltQADo6OnDRRRcVdIwwJCVOIPpY05eOyJZeADXNaZxlZcc/t8xEcjfusi+hDhiFC6tXGwUKTmLM9bcAzo4Rpqj/7Z0qyjj96q2wOkb2cSzuv7BrFy74q78q6O+NcwvqZQCfF5GJMNb8WwTgr6MNiZLIbvB/717jA7+lxd28n7o660TgZAFUwEhG+/YZ+6dfe4nNEsvZsbe0ANddN4D+/vJj28xLEVklQK4AXqLKoq11++TAgcCOHXkVn6p+AuBbAJ4FsAPAw6q6LdqoKIlyJQ4vlXSFrLFnt7KE04Vlm5qAFSt2Wi6fxBXAqVREnqAAQFWfVtVTVfUUVeX3QPLEKqGYuZ0DFEQicJP0Ghu7bVd94Lp3VAri0MVH5Atz15pV1xzgfg5QusXil1zdf0SUKRYtKCK/pFsW6WKCbH5eStzrahFs/RA5wwRFRSnoazRxnTui4DFBUVEKupCA69wRBY8JiopWrq60Qhdz5Tp3RMFjgqKS09ZWU3D3nNNycSLyjgmKSs6aNZM8d8+lW1579w5dM5WTZYn8xQRFiealq667u8Jye77uOXNhBGC0vtJJipNlifzHeVCUWNlr3qW76oDciaKmph8HDlQO2Z6ve86qMEI12jXwiIoZW1CUWF4r6ZYt6/RUgs7CCKJwMUFRYnlNGI2N3Z5K0JNSGBHW5eaJgsYERYnlNmGkP7gXLJiH5majxeRmNYegJ//6gROIqZgwQVFiuUkYmR/c4umDOwmriHMCMRUTJihKLDcJw68P7qDX0TN3zy1aNJsTiKmksYqPEs3pauNJ+ODOrko8cKDSUVWimdeLLBLFEVtQFHt+DPpHWeDgNH4/WnlJGCcjcooJimLNr0H/qD643cTvRysvCeNkRE4xQVGs+Tl2dPyDW0P74HYTv1+tPF5viooFExTFmp9jR+kP7o0bn8/5we3nPCI38bN7jigTExTFWthjR37PI3ITf3b3XG1tH7vnqKQxQVGshd2q8Hsekdv4zd1z69e/xOREJY0JimIt7EF/v8vRWbRA5B3nQVHsOZ3r5FVrq9FC2rfPGHcaGBi6TyFdikHHT1Ss2IIi3yVpsdLsMSer5MRCBaJosAVFvvJ6jaaoWI05AUB5uTEOVFdnJKc4xk5U7JigyFe5igzi+CFvN7Y0OGjciCg67OIjXyVhzTuzpFzjiagUMUGRr+w+2FX9G4/yc4yLk2OJ4osJinxl9YGf5sfF8/yeSMsycKL4YoIiX5k/8K0UevG8IC7Ix7XriOKJCYp8l/7AF7F+vJDxqKSNcRGRd0xQFBi78aiyMu/jRyxqICodTFAUGLvxqIEB7+NHLGogKh1MUBSY7AKE8vKh+7gdP2JRA1Hp4ERdCpR5Hboym69DbsePuLYdUWlgC4pCw/EjInKDCYpCw/EjInIj0gQlIl8VkW0iMigiM6OMhYLH8SMiciPqFtRWAFcC+F3EcSRGki5lYYWTYonIqUiLJFR1BwCI3YxOypC0S1kQERUi6hYUuRDEMj9xYm4djh1r3JLaUiSiwomqBvsCIm0A/szioWZV/U1qnw4AK1R1U47jLAewHABqa2tnrF+/vqC4ent7MXLkyIKOEQZznAsWzIPq0NamiGLjxudDjautrQZr1kxCd3cFamr6sWxZJ2bP7vR8TtvaarBq1WT091tMlgJQUTGAFSt2orGxu5CwASTz3z7ukhIr4/SfH7HOnz9/s6oOrUNQ1chvADoAzHS6/4wZM7RQ7e3tBR8jDOY46+tVjTUYMm/19UOft26dsV3E+LlunX8xrVunWlWVGUNVlWpz8zbPx7T72/L9nV4k8d8+7pISK+P0nx+xAtikFp/17OKLIXNX16JFs491bzkt0/b7khTZ7Loa16yZ5PmYTibrckFYotISdZn5FSLSBeA8AE+JyLNRxhMH2cnlwIHKY8klu0y7uhoYMQJYsiRznCbosSq7RNHdXeH5mE4m63JCL1FpiTRBqepjqjpBVStUtVZVvxRlPHGQL7mky7TXrgWOHAF6eoa2koK+JIVdoqip6fd8zFwXOgQ4oZeoFLGLL2acJpdciSzoJYXsuhqXLev0fEyr1mF1NSf0EpUyJqiYcZpcciWyoJcUslsRotAKO/Mk3nffNW6c0EtUupigYsZpcsmVyPItKeTHahRcEYKIgsYEFTPZyaW2ts+yeytfIrNLIEFU+KUT3oIF8ziploh8wwQVQ+bksn79S5atE68Lr7qp8HPS0spMeOIq4SV9XUEiChYTVIJ56WZzWoRh1dK65pqhyw95LWkPeq4WESUfE1SJsRu7Us0/l+ro0aFl7Xv3Wh8vX0l7sa8rSESFY4IqMbnmGzmZS2V2+DBQbr10Xt6S9qDnahFR8jFBlRjz2JWVw4eBpUuNVpITAwPeStp5+XciyocJKiJRFgikx67sLsM1MOD8WOniDKNYQx0Xa/Dy70SUDxOUz9xXvtkXIATNTWuluhoYPjxzWzqhpBPexo3POy7W4OXfiSgfJigfOa1Mc1qAEHSSyrf+XZqIsarDfff5m1A42ZeIcmGC8pHTyjSnBQhBV7Rlt2LyFTwwoRBRmJigfOS0Ms1p11oYFW3mpPPAAxwXIqL4YILykdPKNKdda2FXtHFciIjihAnKR04r06wuLWFXgBA2duMRUVycEHUAxST9Yd7cbHTP1dUdr3Kz2te8Pb1sUPbzOjpCCZ2IAnT06FF0dXWhr6/P0f6jR4/Gjh07Ao7KH25iraysxIQJEzBs2DBH+zNB+Sw78QT9PCfskh8RhaOrqwujRo1CQ0MDxG4CosmhQ4cwatSoECIrnNNYVRU9PT3o6urCxIkTHR2bXXxFjouyEkWvr68P1dXVjpJTsRIRVFdXO25FAkxQRY+LshLFQyknpzS354AJqshxUVYiSiomKBvFcjE9LspKlDwPP3xCoJ8/t9xyC1atWmX7+OOPP47t27f7+6IeMEFZKKZxGy7KSpQsra3AjTdWRvr5wwQVY8U0bsPJt0TJ0twMHDmSOVbjx+dPS0sLTj31VFxwwQXYuXMnAOBXv/oVZs2ahalTp+LLX/4yDh8+jBdffBFPPPEEvvvd72LatGnYvXu35X5hYIKyYDc+s3dvMrv8OPmWKDmCGDfevHkz1q9fjy1btuDpp5/Gyy+/DAC48sor8fLLL+OVV17BF77wBdx7772YM2cOFi5ciNtvvx1btmzBKaecYrlfGJigLOQanwm7yd3WVlMUY2FE5EwQ48a///3vccUVV6CqqgonnngiFi5cCADYunUrLrzwQpx55plobW3Ftm3bLJ/vdD+/MUFZcLJWnpMmd6GFFq2twKpVk4tiLIyInGlpAUaMyLykdVDjxldffTXuuusuvPbaa1i5cqXtHCWn+/mNCcpC9riNnX377JOQH4UWzc1Af3/mNTCSOhZGRM40NQE//3mfr+PGc+fOxeOPP44jR47g0KFDePLJJwEYq0CMGzcOR48eRavpw2nUqFE4dOjQsd/t9gtaySWodEJZsGBezlaNedymvt56H1VgyRLrJORHoQXnMBGVpq997RNfx43PPvtsXHXVVZg6dSouueQSzJo1CwBw66234txzz8X555+P00477dj+ixYtwu23347p06dj9+7dtvsFTlUTd5sxY4Z6sW6dalWVqpFOjFtVlbHd7fNy3errVUXsHxcx9sn3uvX19sePo/b29qhDcIRx+i8psUYV5/bt213t/+GHHwYUif/cxmp1LgBsUovP+pJqQXlt1Zi7/JxIL8pqx2mXX0sLUFExkLGNc5iIqFSUVIIqpMss3eXnZCmp9IrhhRZaNDUBK1bs5BwmIipJJZWg/CjfzLdvuoXjptAil8bGbs5hIqKSVFIJyo9lf6yOkU5A2S0cJ4UWXBOPiMhaSSWozFaNoroaGDHCqMRzOk/JaumgtWuNcaVcLRyuiUdE5E5JJSjgeKvm7/9+B44cAXp63M9T8rJ0ENfEIyJyp+QSVNqaNZNCXxCWa+IRUdI1NDTg3XffLXgfJyJNUCJyu4i8LiKvishjInJSWK/d3V1huT2pC8ISERWbEyJ+/Q0Avq+qn4jITwB8H8D3wnjhmpp+HDhQafmYucsPYEuHiHz0t38LbNmSc5cRAwNAeXnOfTJMmwb87Ge2D+/ZswcXX3wxZs+ejRdffBGzZs3CNddcg5UrV6K7uxutra343Oc+h2uvvRadnZ2oqqrC6tWrcdZZZ6GnpweLFy/G/v37cd5558GYV2tYt24d7rjjDgwMDODcc8/FL37xC5S7iTuPSFtQqvqcqn6S+vUlABPCeu1lyzp9WRCWiCgJdu3ahZtuugmvv/46Xn/9dTz44IN44YUXsGrVKtx2221YuXIlpk+fjldffRW33XYbvvGNbwAAfvjDH+KCCy7Atm3bcMUVV2Bfam7Mjh078NBDD2HDhg3YsmULysvLfV+nL+oWlNm1AB6ye1BElgNYDgC1tbXo6Ogo6MVmz+7Ft79tjEV1d1fA+FIwdMLSvn2Kjo7nC3qtQvT29jr+W9vaao79PTU1/Vi2rBONjd3BBmjiJtYoMU7/JSXWqOIcPXr08cVXb7017/4DAwPuWyKmxV2z9fb2or6+Hg0NDfjoo49w6qmnYs6cOejt7cXEiRPR2dmJt956C2vXrsWhQ4cwa9YsvPvuu9i/fz86Ojqwbt06HDp0CHPnzsVJJ52E3t5ePPXUU9i0aRPmzZsHEcGRI0eO/Z2qit7eXlRUDB1K6evrc/5vYLX+kZ83AG0AtlrcLjPt0wzgMQDi5Jhe1+Izy16TK67r3jldO8zrOoN+4nps/kpKnKrJibVU1+J766239PTTTz/2+9KlS/WRRx7JeGzatGm6e/fuY/tMmDBBP/jgA506dWrG9jFjxujBgwf1zjvv1Jtvvtky1vr6ej148KBlLLFai09VG1X1DIvbbwBARK4G8N8ANKUCjUTS5ykV02XqiSh8F1544bEuuo6ODowdOxYnnngi5s6diwcffBAA8Mwzz+D9998HAHzxi1/Eo48+ioMHDwIA3nvvPezdu9fXmCLt4hORiwH8HYB5qhrORe5tpAshmpuPL/aaXrIoCXhpDiIqxC233IJrr70WZ511FqqqqvDAAw8AAFauXInFixfj9NNPx5w5c1CXWv5mypQp+NGPfoTLL78cADBs2DDcfffdqHe6qrYDUY9B3QWgAsAGMdYLeklVvxlVME1NyUlI2erqjMpDq+1EVNoaGhqwdevWY7/ff//9lo89/vjjQ55bXV2N5557zvK4V111FS699FKMGjUqY/uePXsKjhmIOEGp6ueifP1i0tJilMWbu/mS1EVJRJStZFeSKDZcSomIik3UXXzkoyR3URIVO1WFOLmgXBFzWwfHFhQRUcAqKyvR09Pj+gO6mKgqenp6UFlpvYKPFbagiIgCNmHCBHR1dR0ryc6nr6/P1Qd5lNzEWllZiQkTnC8YxARFRBSwYcOGYeLEiY737+jowPTp0wOMyD9BxsouPiIiiiUmKCIiiiUmKCIiiiVJYlWJiBwEUOiiT2MBFH7Jx+AlJU4gObEyTv8lJVbG6T8/Yq1X1U9nb0xkgvKDiGxS1ZlRx5FPUuIEkhMr4/RfUmJlnP4LMlZ28RERUSwxQRERUSyVcoJaHXUADiUlTiA5sTJO/yUlVsbpv8BiLdkxKCIiirdSbkEREVGMMUEREVEsFXWCEpGvisg2ERkUEdsySBG5WER2isguEbnZtH2iiPwxtf0hERkeUJwni8gGEXkz9XOMxT7zRWSL6dYnIpenHrtfRN4yPTYtqjhT+w2YYnnCtD2U8+k0VhGZJiJ/SL1HXhWRq0yPBXpO7d5zpscrUudoV+qcNZge+35q+04R+ZKfcXmI8zsisj11/v5dROpNj1m+DyKM9WoROWiKaZnpsaWp98qbIrI04jjvMMX4hoj8l+mx0M6piNwnIt0istXmcRGRO1N/x6sicrbpMX/Op6oW7Q3AFwBMBtABYKbNPuUAdgOYBGA4gFcATEk99jCARan7/wTg+oDi/CmAm1P3bwbwkzz7nwzgPQBVqd/vB/CVEM6nozgB9NpsD+V8Oo0VwKkAPp+6/xkA7wA4Kehzmus9Z9rnBgD/lLq/CMBDqftTUvtXAJiYOk55hHHON70Pr0/Hmet9EGGsVwO4y+K5JwPoTP0ck7o/Jqo4s/a/EcB9EZ3TuQDOBrDV5vFLATwDQADMBvBHv89nUbegVHWHqu7Ms9s5AHapaqeqfgxgPYDLREQALADwaGq/BwBcHlCol6WO7/R1vgLgGVU9nGc/v7mN85iQzyfgIFZVfUNV30zd/38AugEMmc0eAMv3XNY+5vgfBfDF1Dm8DMB6Ve1X1bcA7EodL5I4VbXd9D58CYDzayn4y8k5tfMlABtU9T1VfR/ABgAXxyTOxQB+HVAsOanq72B8EbZzGYB/UcNLAE4SkXHw8XwWdYJyaDyAt02/d6W2VQP4L1X9JGt7EGpV9Z3U/T8BqM2z/yIMfdO2pJrZd4hIhe8RGpzGWSkim0TkpXQ3JMI9n4DLcyoi58D4RrvbtDmoc2r3nrPcJ3XOPoBxDp08N8w4za6D8Y06zep9EBSnsX459W/6qIh81uVz/eD4tVLdpRMBbDRtDvOc5mP3t/h2PhN/PSgRaQPwZxYPNavqb8KOx06uOM2/qKqKiG3tf+obypkAnjVt/j6MD+HhMOYkfA/AP0YYZ72q7heRSQA2ishrMD5gfeXzOV0LYKmqDqY2+3ZOS4GIfB3ATADzTJuHvA9Udbf1EULxJIBfq2q/iPx3GC3UBRHGk88iAI+q6oBpW9zOaaASn6BUtbHAQ+wH8FnT7xNS23pgNFlPSH2DTW/3JFecInJARMap6jupD8vuHIf6GoDHVPWo6djplkK/iPwzgBVRxqmq+1M/O0WkA8B0AP8KH8+nX7GKyIkAnoLxheYl07F9O6cW7N5zVvt0icgJAEbDeE86eW6YcUJEGmF8KZinqv3p7Tbvg6A+TPPGqqo9pl/XwBinTD/3oqzndvge4fHXcvrvtwjA35g3hHxO87H7W3w7n+ziA14G8HkxKsyGw3hTPKHGaF87jPEeAFgKIKgW2ROp4zt5nSF90qkP4PQ4z+UALKtufJA3ThEZk+4OE5GxAM4HsD3k8+k01uEAHoPRj/5o1mNBnlPL91yO+L8CYGPqHD4BYJEYVX4TAXwewP/1MTZXcYrIdAC/BLBQVbtN2y3fBwHF6TTWcaZfFwLYkbr/LIC/SMU8BsBfILOHItQ4U7GeBqPA4A+mbWGf03yeAPCNVDXfbAAfpL7Y+Xc+g6j+iMsNwBUw+j/7ARwA8Gxq+2cAPG3a71IAb8D4JtJs2j4Jxn/+XQAeAVARUJzVAP4dwJsA2gCcnNo+E8Aa034NML6dlGU9fyOA12B8iK4DMDKqOAHMScXySurndWGfTxexfh3AUQBbTLdpYZxTq/ccjC7Ehan7lalztCt1ziaZntucet5OAJcE/H8oX5xtqf9b6fP3RL73QYSx/hjAtlRM7QBOMz332tS53gXgmijjTP1+C4D/mfW8UM8pjC/C76T+j3TBGGP8JoBvph4XAHen/o7XYKqU9ut8cqkjIiKKJXbxERFRLDFBERFRLDFBERFRLDFBERFRLDFBERFRLDFBERFRLDFBERFRLDFBEcWMiLSLyJ+n7v9IRH4edUxEUUj8WnxERWglgH8UkRoYa60tjDgeokhwJQmiGBKR5wGMBHCRqh6KOh6iKLCLjyhmRORMAOMAfMzkRKWMCYooRlIrbrfCuFppr4gEdWVXothjgiKKCRGpAvBvAG5S1R0AboUxHkVUkjgGRUREscQWFBERxRITFBERxRITFBERxRITFBERxRITFBERxRITFBERxRITFBERxdL/B58SBf4q516gAAAAAElFTkSuQmCC\n",
"text/plain": [
"