{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"ExecuteTime": {
"end_time": "2019-04-07T02:19:02.272939Z",
"start_time": "2019-04-07T02:19:02.268259Z"
},
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"***\n",
"***\n",
"\n",
"# Introduction to Neural Network\n",
"\n",
"\n",
"***\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {
"ExecuteTime": {
"end_time": "2019-04-07T05:41:09.971531Z",
"start_time": "2019-04-07T05:41:09.967172Z"
},
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"\n",
"\n",
"\n",
"http://playground.tensorflow.org/"
]
},
{
"cell_type": "markdown",
"metadata": {
"ExecuteTime": {
"end_time": "2019-04-07T09:37:45.438709Z",
"start_time": "2019-04-07T09:37:45.434406Z"
},
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"- Deep Learning http://www.deeplearningbook.org\n",
" - An MIT Press book by Ian Goodfellow, Yoshua Bengio and Aaron Courville\n",
"\n",
"- Neural Networks and Deep Learning http://neuralnetworksanddeeplearning.com/index.html\n",
"\n",
" - A free online book explaining the core ideas behind artificial neural networks and deep learning. [Code](https://github.com/mnielsen/neural-networks-and-deep-learning). By [Michael Nielsen](http://michaelnielsen.org/) / Dec 2017"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## House Price\n",
"\n",
"Let’s start with a simple example. \n",
"- Say you’re helping a friend who wants to buy a house.\n",
"\n",
"- She was quoted $400,000 for a 2000 sq ft house (185 meters). \n",
"\n",
"Is this a good price or not?"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"So you ask your friends who have bought houses in that same neighborhoods, and you end up with three data points:\n",
"\n",
"\n",
"\n",
"| Area (sq ft) (x) | Price (y) | \n",
"| -------------|:-------------:|\n",
"|2,104|399,900|\n",
"|1,600|329,900|\n",
"|2,400|369,000|"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"\n",
"$$y = f(X) = W X$$\n",
"\n",
"- Calculating the prediction is simple multiplication.\n",
"- But before that, we need to think about the weight we’ll be multiplying by. \n",
"- “training” a neural network just means finding the weights we use to calculate the prediction.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"A simple predictive model (“regression model”)\n",
"- takes an input, \n",
"- does a calculation, \n",
"- and gives an output \n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"ExecuteTime": {
"end_time": "2019-04-07T03:15:14.317623Z",
"start_time": "2019-04-07T03:15:14.313438Z"
},
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Model Evaluation\n",
"- If we apply our model to the three data points we have, how good of a job would it do?"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"**Loss Function (also, cost function)**\n",
"\n",
"- For each point, the error is measured by the difference between the **actual value** and the **predicted value**, raised to the power of 2. \n",
"- This is called **Mean Square Error**. "
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"- We can't improve much on the model by varying the weight any more. \n",
"- But if we add a bias (intercept) we can find values that improve the model.\n",
"\n",
"\n",
"\n",
"$$y = 0.1 X + 150$$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"**Gradient Descent**\n",
"\n",
"- Automatically get the correct weight and bias values \n",
"- minimize the loss function.\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## softmax\n",
"\n",
"The softmax function, also known as softargmax or normalized exponential function, is a function that takes as input a vector of K real numbers, and normalizes it into a probability distribution consisting of K probabilities. \n",
"\n",
"$$softmax = \\frac{e^x}{\\sum e^x}$$\n"
]
},
{
"cell_type": "code",
"execution_count": 128,
"metadata": {
"ExecuteTime": {
"end_time": "2019-04-07T13:57:33.466056Z",
"start_time": "2019-04-07T13:57:33.459580Z"
},
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"array([0.02364054, 0.06426166, 0.1746813 , 0.474833 , 0.02364054,\n",
" 0.06426166, 0.1746813 ])"
]
},
"execution_count": 128,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def softmax(s):\n",
" return np.exp(s) / np.sum(np.exp(s), axis=0)\n",
"\n",
"softmax([1.0, 2.0, 3.0, 4.0, 1.0, 2.0, 3.0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"\n",
"That is, prior to applying softmax, some vector components could be negative, or greater than one; and might not sum to 1; but after applying softmax, each component will be in the interval (0,1), and the components will add up to 1, so that they can be interpreted as probabilities. \n",
"\n",
"Furthermore, the larger input components will correspond to larger probabilities. \n",
"\n",
"Softmax is often used in neural networks, to map the non-normalized output of a network to a probability distribution over predicted output classes."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"## Activation Function\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"ExecuteTime": {
"end_time": "2019-04-07T03:59:39.599371Z",
"start_time": "2019-04-07T03:59:39.593104Z"
},
"code_folding": [],
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"0.6224593312018546"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"def sigmoid(x):\n",
" return 1/(1 + np.exp(-x))\n",
"\n",
"sigmoid(0.5)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"ExecuteTime": {
"end_time": "2019-04-07T04:01:24.575040Z",
"start_time": "2019-04-07T04:01:24.567328Z"
},
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/plain": [
"0.5"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Naive scalar relu implementation. \n",
"# In the real world, most calculations are done on vectors\n",
"def relu(x):\n",
" if x < 0:\n",
" return 0\n",
" else:\n",
" return x\n",
"\n",
"relu(0.5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# WHAT IS PYTORCH?\n",
"\n",
"It’s a Python-based scientific computing package targeted at two sets of audiences:\n",
"\n",
"- A replacement for NumPy to use the power of GPUs\n",
"- a deep learning research platform that provides maximum flexibility and speed"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"ExecuteTime": {
"end_time": "2019-06-15T03:13:38.257567Z",
"start_time": "2019-06-15T03:13:37.133622Z"
},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import torch\n",
"from torch import nn, optim\n",
"from torch.autograd import Variable\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"ExecuteTime": {
"end_time": "2019-06-15T03:13:54.278106Z",
"start_time": "2019-06-15T03:13:54.075090Z"
},
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD+CAYAAADfwXXpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAFNlJREFUeJzt3X+Q3PV93/HnC4FiEv9QLS4jIQvJwlHBSceMdZ3xtWQQwgy4k2TGeGoS0kAcG9nYUxg8MZ4y9ozDlIHB/ie4bjJKaJk6OImdgt2YGqpSq65dOZODGKt2LBnLxkgV6iEgbkNVLN27f3y/stfKSbcn3WlPH56PmZ29/eznu/u6O3jx4bPf201VIUlq0xmjDiBJWjiWvCQ1zJKXpIZZ8pLUMEtekhpmyUtSwyx5SWqYJS9JDbPkJalhZ446wDnnnFNr164ddQxJOq08+uijz1TV2GzzRl7ya9euZXJyctQxJOm0kuTJYea5XSNJDbPkJalhlrwkNcySl6SGWfKS1LChSj7J0iTfTPKH/e2bknw/yc4kbxmYd2eSPUl2JNmwUKElScMZ9hTKW4HvASQ5H3gf8PPAauA/J1kD/CJwMbAWuBS4B7hofuNKAmD7dti2DTZuhImJUafRIjZrySe5EPiHwKfpSvytwKer6n8D30zyPWADcBVwb1UdArYmGUuyoqqeXrD00kvR9u1w2WXw4ouwdCk88ohFr2M67nZNkgB3AzcNDK8GBk/C3wOsnGF8bz8+0+NuTjKZZHJqaupEcksvXdu2dQV/+HB3vW3bqBNpEZttT/49wLaqemJgbCkwPXB7Gjh8nPG/o6q2VNV4VY2Pjc36V7mSBm3c2K3glyzprjduHHUiLWKzbdf8BvCKJP8UeDXwM3Qr+1UDc14DPAXsO2r8XLpVvqT5NDHRbdG4J68hHLfkq+ofHfk6yW/S7cl/Hvhkko/Rvcj6auBrwIPAe5PcB2wCdlXVswsTW3qJm5iw3DWUOb9BWVU9muSPgG8AB4F3VVUleQC4BNgNHACumdekkqQ5S1WNNMD4+Hj5LpSSNDdJHq2q8dnm+RevktQwS16SGmbJS1LDLHlJapglL0kNs+QlqWGWvCQ1zJKXpIZZ8pLUMEtekhpmyUtSwyx5SWqYJS9JDbPkJalhlrwkNcySl6SGWfKS1DBLXpIaZslLUsNmLfkkZyTZmmRXkp1JrujH7k7y7SQ7kkwMzL8zyZ5+fMPCxpckHc+ZQ8wp4Nqq2pfkSuB24FzgNcAFwBuA+5K8HrgUuBhY2399D3DRAuSWJA1h1pV8dfb1N9cAjwPjwBeq6nBVPQYcAtYBVwH3VtWhqtoKjCVZsUDZJUmzGGpPPsktSQ4ANwO3Ad8AfjnJWUkuBF4LjAGrgScHDt0LrJzh8TYnmUwyOTU1dbLfgyTpGIYq+aq6q6qWA7cCDwN/QFfgXwc+AHwLOAAsBaYHDp0GDs/weFuqaryqxsfGxk7uO5AkHdMwe/I/UlX3J7kbeFVV3QCQ5KeAXcBTwD5g1cAh5wJ75imrJGmOhjm7Zt2RffX+LJqDwN8mWZokwIeBz1bVQeBB4LokS5JcDuyqqmcXML8k6TiGWckvAx5KsgTYD1wNnAd8Afgp4IvAu/u5DwCXALvptm+ume/AkqThzVry/dkz62e4a90Mc6eBG/uLJGnE/ItXSWqYJS9JDbPkJalhlrwkNcySl6SGWfKS1DBLXpIaZslLUsMseUlqmCUvSQ2z5CWpYZa8JDXMkpekhlnyktQwS16SGmbJS1LDLHlJapglL0kNs+QlqWGzlnySM5JsTbIryc4kV/TjH03ynSTfTvK2gfl3JtmTZEeSDQsZXpJ0fLN+kDdQwLVVtS/JlcDtSQ4CFwMX0H2g95eAf59kUz++FrgUuAe4aCGCS5JmN+tKvjr7+ptrgMeB/9cfexh4GbC/v/8q4N6qOlRVW4GxJCvmP7YkaRjDrORJcgvwQWAKuKKqnkzyIPBV4Czg1/qpq4HPDRy6F1gJPH3U420GNgOcd955J5NfknQcQ73wWlV3VdVy4Fbg4SSrgUuA9wCfBm7qpy4FpgcOnaZb7R/9eFuqaryqxsfGxk4mvyTpOOZ0dk1V3Q+8HPgd4IGqeqyq7gDelORCYB+wauCQc4E98xVWkjQ3w5xds+7IvnqSCeAgXXGPp7MSWAH8AHgQuC7JkiSXA7uq6tmFiy9JOp5h9uSXAQ8lWUL3AuvVwG7gPuB7wAvAv6iqvUkeoNvG2Q0cAK5ZiNCSpOHMWvJV9Riwfoa7/skMc6eBG/uLJGnE/ItXSWqYJS9JDbPkJalhlrwkNcySl6SGWfKS1DBLXpIaZslLUsMseUlqmCUvSQ2z5CWpYZa8JDXMkpekhlnyktQwS16SGmbJS1LDLHlJapglL0kNs+QlqWGWvCQ1bNaST3JGkq1JdiXZmeSKJB9P8sTA5XCS1/fz70yyJ8mOJBsW/luQJB3LmUPMKeDaqtqX5Erg9qoaP3JnkvXAp6rqm0k2ARcDa4FLgXuAi+Y/tiRpGLOu5Kuzr7+5Bnj8qCnvAv5N//VVwL1VdaiqtgJjSVbMW1pJ0pwMs5InyS3AB4Ep4IqB8bOAt/Pj1fpq4HMDh+4FVgJPH/V4m4HNAOedd94JRpckzWaoF16r6q6qWg7cCjycJP1dvwJ8paqe728vBaYHDp0GDs/weFuqaryqxsfGxk48vSSdjrZvhzvu6K4X2FAr+SOq6v4kdwPLgWeA64G7BqbsA1YN3D4X2HOyISWpGdu3w2WXwYsvwtKl8MgjMDGxYE83zNk1647sqyeZAA5W1TNJ1gDnA18cmP4gcF2SJUkuB3ZV1bMLEVySTkvbtnUFf/hwd71t24I+3TAr+WXAQ0mWAPuBq/vxdwL/rqpqYO4DwCXAbuAAcM08ZpWk09/Gjd0K/shKfuPGBX26/GRHn3rj4+M1OTk50gySdEpt396t4DduPOGtmiSPDp7Ofixz2pOXJM2DiYkF3Ycf5NsaSFLDLHlJapglL0kNs+QlqWGWvCQ1zJKXpIZZ8pLUMEtekhpmyUtSwyx5SWqYJS9JDbPkJalhlrwkNcySl6SGWfKS1DBLXpIaZslLUsMseUlq2Kwln+SMJFuT7EqyM8kV/fiqJA8leSrJ9oH5dybZk2RHkg0LGV6SdHzDrOQLuLaq1gM3Abf3438M3FdVq4FNAEk2ARcDa4H3A/fMd2BJ0vBmLfnq7OtvrgEe71foqapP9nP+b3//VcC9VXWoqrYCY0lWLERwSdLshtqTT3JLkgPAzcBtwEXA3n4b51tJfrufuhp4cuDQvcDKGR5vc5LJJJNTU1Mn9x1Iko5pqJKvqruqajlwK/Aw8LPABcDb6bZnbkjyBmApMD1w6DRweIbH21JV41U1PjY2dpLfgiTpWOZ0dk1V3Q+8nK68v1RVz1XVM8BXgPXAPmDVwCHnAnvmKaskaY6GObtm3ZF99SQTwEG6F10vS/LKJMuANwF/BTwIXJdkSZLLgV1V9ezCxZckHc+ZQ8xZBjyUZAmwH7i6qr6f5GPAXwIB7qyqJ5LsBi4BdgMHgGsWKLckaQipqpEGGB8fr8nJyZFmkKTTTZJHq2p8tnn+xaskNcySl6SGWfKS1DBLXpIaZslLUsMseUlqmCUvSQ2z5CWpYZa8JDXMkpekhlnyktQwS16SGmbJS1LDLHlJapglL0kNs+QlqWGWvCQ1zJKXpIZZ8pLUsFlLPskZSbYm2ZVkZ5Ir+vEfJnmiv/zJwPw7k+xJsiPJhoUML0k6vjOHmFPAtVW1L8mVwO3Aw8Deqnrd4MQkm4CLgbXApcA9wEXzmliSNLRZV/LV2dffXAM8fpzpVwH3VtWhqtoKjCVZMQ85JUknYKg9+SS3JDkA3Azc1g8vT/KdJF9MMt6PrQaeHDh0L7ByhsfbnGQyyeTU1NRJxJckHc9QJV9Vd1XVcuBW4OEkqapXVNX5wL8GHuinLgWmBw6dBg7P8Hhbqmq8qsbHxsZO7juQJB3TnM6uqar7gZcDywfGPgOcnWQZsA9YNXDIucCeecgpSToBw5xds+7IvnqSCeBg//Wy/votwIGqeh54ELguyZIklwO7qurZBUsvSTquYc6uWQY8lGQJsB+4mm6f/T8kmaZbvb+9n/sAcAmwGzgAXDPviSVJQ5u15KvqMWD9DHe9doa508CN/UWSNGL+xaskNcySl6SGWfKS1DBLXpIaZslLUsMseUlqmCUvSQ2z5CWpYZa8JDXMkpekhlnyktQwS16SGmbJS1LDLHlJapglL0kNs+QlqWGWvCQ1zJKXpIZZ8pLUsFlLPskZSbYm2ZVkZ5IrBu5bnmR/kg8NjN2ZZE+SHUk2LFRwSdLshlnJF3BtVa0HbgJuH7jvY8BjR24k2QRcDKwF3g/cM29JJUlzNmvJV2dff3MN8DhAkjcDh4C/GJh+FXBvVR2qqq3AWJIV85xZkjSkofbkk9yS5ABwM3BbkrOB24Bbjpq6Gnhy4PZeYOUMj7c5yWSSyampqRNLLkma1VAlX1V3VdVy4FbgYeAjwCeq6rmjpi4FpgduTwOHZ3i8LVU1XlXjY2NjJxRckjS7M+cyuaruT3I3cCOwM8kHgBVAJfkusA9YNXDIucCe+QorSZqbWUs+yTrghap6OskEcLCqzh64/yPAoaq6L8lB4L1J7gM2Abuq6tkFyi5JmsUwK/llwENJlgD7gauPM/cB4BJgN3AAuOakE0qSTliqaqQBxsfHa3JycqQZJOl0k+TRqhqfbZ5/8SpJDbPkJalhlrwkNcySl6SGWfKS1DBLXpIaZslLUsMseUlqmCUvSQ2z5CWpYZa8JDXMkpekhlnyktQwS16SGmbJS1LDLHlJapglL0kNs+QlqWGWvCQ1bNaST3JGkq1JdiXZmeSKJOck+XKSbyd5PMkbB+bfmWRPkh1JNixsfEnS8Qyzki/g2qpaD9wE3A68APxSVf0c8HvArQBJNgEXA2uB9wP3LEBmSdKQZi356uzrb64BHq+qF6rq+SRLgNXA4/39VwH3VtWhqtoKjCVZsSDJt2+HO+7oriVJMzpzmElJbgE+CEwBV/Rjvwu8A/gmcGU/dTXwuYFD9wIrgafnKW9n+3a47DJ48UVYuhQeeQQmJub1KSSpBUO98FpVd1XVcrptmYeTpKpuApYBnwI+009dCkwPHDoNHD768ZJsTjKZZHJqamruqbdt6wr+8OHuetu2uT+GJL0EzOnsmqq6H3g5sLy/PU23J/+mfso+YNXAIecCe2Z4nC1VNV5V42NjY3NPvXFjt4JfsqS73rhx7o8hSS8Bs27XJFkHvFBVTyeZAA4CK5L8sKr+Bngr8Gg//UHgvUnuAzYBu6rq2XlPPTHRbdFs29YVvFs1kjSjYfbklwEP9S+y7geuptt7//Mkh4HvAu/s5z4AXALsBg4A18x74iMmJix3SZrFrCVfVY8B62e467UzzJ0GbuwvkqQR8y9eJalhlrwkNcySl6SGWfKS1DBLXpIalqoabYBkCnjyBA8/B3hmHuPMF3PNjbnmxlxz02quNVU161+TjrzkT0aSyaoaH3WOo5lrbsw1N+aam5d6LrdrJKlhlrwkNex0L/ktow5wDOaaG3PNjbnm5iWd67Tek5ckHd/pvpKXJB3Hoi35JGcnmemN0UbKXHOzWHOdTpK8KsnfeUNAaRiLruSTvDLJZ+ne1viWgfFVSR5K8lSS7QPjdybZk2RHkg392JlJ7k2yN8lX5+NfkJlyJfl4kicGLoeTvH7Uufrxjyb5TpJvJ3nbwPgof15nJLm7z7Sj/3yCU53rZUm2JNmV5MkkN/fjNyX5fpKdSd6yGHIlWZvkP9F9GM+vHzV/lLmWJ/nT/vf4nSS/ukhynZPky32ux5O8cTHkGrhveZL9ST50SnNV1aK60H3y1GXAu4A/HBj/EvAb/ddn99ebgC/TvWXy5cDX+vHfAv4ECHA98NmFyjVw/3pgcjHkontP/+3AWcDfB/YvklzvAO4HlgBvBP66f85TmWs58Lb+Mc+h+4/QJcAu4BXA64H/2f/sRp1rNTAB/A7woYG5o871i8DG/v7XAc8vkp/XGmBZf/97gD9bJD+v1f19/xb4wpHf5anKdVLf0EJegN/kx+WwAfhvM8z5V8C7Bm7vBVYAnwfe3I/9NPB/FiLXUeN3Ae9dDLnoPo7xL+j+T+0NwNcXSa5PANcP3LcDOH8UuQaeaxL4MPAvB8b+e/8zHHWuf9B//RF+suQXRa6BsSnglYslF90i4nbgw4vl5wW8GfiDwd/lqcq16LZrjuEiYG+SrUm+leS3+/HV/ORbIuwFVg6OV9ULwAtJ/t5ChUtyFvB2ug81H3muqvoq3UcxfhW4F/i1xZAL+Abwy0nOSnIh3QfPjI0qV5JfAF5Gt+IafP49Rz//iHL9j2NMWTS5+q2tx6rqB4shV5LfBZ6j+7/Ij/dTRp3rCeA2BrZTT2Wu06Xkfxa4gK5ILwZuSPIGYCkwPTBvGjh8nPGF8ivAV6rq+f72SHMlWU23BfEe4NPATYshF91KZi/wdeADwLfoPibylOdKcg7wSbotpLk+/ynJVf1SbgaLIleS1wEfBd69WHJV1U10H1n6KeAziyEX3er9E1X13FHTTkmu06Xk/xfwpap6rqqeAb5Ctwe+D1g1MO9culXYj8aTnA0s6VcaC+V64J6B26PO9c+BB6rqsaq6A3hTv3Ieaa6q+mFV3VBVFwI30K3inzrVufpV0eeBW6vqL2d4/tcsklzHMvJcSdYAfwZcW1XfWyy54EcfQ/p7dFtuiyHXNcAHknyNbuH1viS/fqpynS4lvxW4LN0ZG8vofnl/RbclcV2SJUkuB3ZV1bP9+Dv6Y/8Z8LmFCtb/w34+8MWB4VHnOgiMp7OSbp/vB6POle50yqVJQrcP/tmqOngqcyV5JfDndHvwX+iHHwR+NclPpzs76tXA1xZBrmMZaa4kq+heQL++us+AXiy5fiHJq/opbwUeXQy5qmp1VV1UVRcBv0+3qr/vlOWa7xca5uGFilfQ7WHtB/6m//rS/pveSXcWxG/1c88A7qbbv3oMuKAffxnwx3Srsf8KrFjAXLfRv8AzMHfUud4K/Mf++f8aeOciyfVuYDfdls0fAT8zglwfAv62z3Pksg64Ffhu//P6x4sk18/318/SbWs9AfzcIsj1XwZ+p0cuSxdBrvf1v8Mn6BaG5y+S3+O6gfs/wo9feD0luXxbA0lq2OmyXSNJOgGWvCQ1zJKXpIZZ8pLUMEtekhpmyUtSwyx5SWqYJS9JDbPkJalh/x/AETkZpfVIUQAAAABJRU5ErkJggg==\n",
"text/plain": [
"