{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Optimization Algorithms\n", "


" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this notebook, we'll cover **Gradient Descent** algorithm and its variants: Batch Gradient Descent, Mini-Batch Gradient Descent, and Stochastic Gradient Descent. Next, we'll cover **Momentum Methods** such as Momentum, Root Mean Square Propagation (RMSProp), and Adaptive Momentum (Adam). Optimization algorithms have one of the following goals:\n", "- Find global minimum of the objective function. This is feasible if the objective function is convex, i.e any local minimum is a global minimum.\n", "- Find lowest possible value of the objective function within its neighbor. That's usually the case if the objective function in not convex as the case in most deep learning problems." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Gradient Descent\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Gradient Descent* algorithm is a first-order optimization algorithm. That means it only takes into account first derivative when performing the updates on the parameters. Below is how the algorithm works in the case of a logistic regression:\n", "1. Initialize weights vector $w$ to any random numbers.\n", "2. Pick a value for the learning rate $\\alpha$. The learning rate determines how big the step would be on each iteration.\n", " * If $\\alpha$ is very small, it would take long time to converge and become computationally expensive.\n", " * IF $\\alpha$ is large, it may fail to converge and overshoot the minimum.\n", " * Therefore, plot the cost function against different values of $\\alpha$ and pick the value of $\\alpha$ that is right before the first value that didn't converge so that we would have a very fast learning algorithm that converges.\n", " * The most commonly used rates are : *0.001, 0.003, 0.01, 0.03, 0.1, 0.3*.\n", "3. Make sure to scale the data if it's on very different scales. If we don't scale the data, the level curves would be narrower and taller which means it would take longer time to converge. Scale the data to have $\\mu = 0$ and $\\sigma = 1$. Below is the formula for scaling each example:\n", "$$\\frac{x_i - \\mu}{\\sigma}\\tag{1}\\\\{} $$\n", "4. For each iteration, take the partial derivative of the cost function $J(w)$ w.r.t each $w^j$ (gradient) which is:\n", "$$\\frac{\\partial}{\\partial w^j}J(w) = \\nabla_w J\\tag{2}$$\n", "The update equation is:\n", "$$w^j = w^j - \\alpha \\nabla_w J\\tag{3}$$\n", " * Continue the process until the cost function converges. That is, until the error curve becomes flat and doesn't change.\n", " * Notice that if the slope of the current values of $w > 0$, that means that we're to the right of optimal $w$ and the update will be negative and will start getting close to the optimal values of $w$. However, if it's negative, the update will be positive and will increase the current values of $w$ to converge to the optimal values of $w$. See below:\n", "

\n", "
\n", "

**Figure 1:** Gradient descent
\n", "

\n", " * In addition, on each iteration, the step would be in the direction that gives the maximum change since it's perpendicular to level curves at each step.\n", "\n", "In deep learning, optimization algorithms face few challenges:\n", "- The cost functions are not convex; therefore, there is no global minimum. However, the neural network achieves a significantly low error rates.\n", "- Different weights initializations leads to different models.\n", "- With low dimension, local minimum is very common; however, with high dimensional data, saddle points are more common.\n", "- If the cost function is not initialized where it points towards the global solution, it results in excessive training time and may never converge." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Batch Gradient Descent\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Batch gradient descent is when we sum up over all examples on each iteration when performing forward propagation and back-propagaton. Therefore, for each update, we have to sum over all examples. The main advantages:\n", "- We can use fixed learning rate during training without worrying about learning rate decay.\n", "- It has straight trajectory towards the minimum and it converges in theory.\n", "- It has unbiased estimate of gradients. The more the examples, the lower the standard error.\n", "\n", "The main disadvantages:\n", "- Even though we can use vectorized implementation, it may still be slow to go over all examples especially when we have large datasets.\n", "- Each step of learning happens after going over all examples." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Mini-Batch Gradient Descent\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instead of going over all examples, mini-batch gradient descent sums up over lower number of examples based on batch size. The way it works is as follows:\n", "- Shuffle the training dataset to avoid pre-existng order of examples.\n", "- Partition the training dataset into $b$ batches based on the batch size. If the training set size is not divisible by batch size, the remaining will be its own batch.\n", "\n", "The batch size is something we can tune; however, the batch size is usually chosen as power of 2 such as 32, 64, 128, 256, 512, etc.\n", "\n", "The main advantages:\n", "- Faster than Batch version because it goes through a lot less examples than Batch.\n", "- Randomly selecting examples will help avoid redundant examples or examples that are very similar that don't contribute much to the learning.\n", "- With batch size < size of training set, it adds noise to the learning process that helps improving generalization error.\n", "- Even though with more examples the estimate would have lower standard error, the return is less than linear compared to the computational burden we incur.\n", "\n", "The main disadvantages:\n", "- It won't converge. On each iteration, the learning step may go back and forth due to the noise. Therefore, it wander around the minimum region but never converge,\n", "- Due to the noise, the learning steps have more oscillations and requires to add learning-decay to decrease learning rate as we become closer to the minimum.\n", "\n", "With large training datasets, we don't usually need more than 2-10 passes over all training examples (epochs). Note that with batch size $b = m$, we get the Batch Gradient Descent." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Stochastic Gradient Descent\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It's the same as mini-batch gradient descent but the batch size $b = 1$. Therefore, learning happen on every example. The main disadvantage is that we can't utilize vectorization over 1 example and becomes very slow. Also, the variance becomes large since we only use 1 example for each learning step. Below is a graph that shows the gradient descent and variants and the direction towards the minimum:\n", "

\n", "
\n", "

**Figure 2:** Gradient descent variants: Batch GD, Mini-Batch GD, and Stochastic GD
\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Momentum Methods\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since mini-batch gradient descent learns from smaller number of examples on each step, it adds noise to the direction towards the minimum (oscillations). Cost function is often very sensitive in the direction of some parameters and not sensitive in the direction of other parameters. Instead of having a learning rate for each parameter, momentum methods solve this issue by using exponential weighted average (moving average) of the past gradients to smooth the steps towards the minimum. Doing so will affect the learning rate by decaying it more as we get closer to the minimum region based on the past gradients. Therefore, momentum methods only affects the update formula (3) and all other implementations such as forward propagation and back-propagation would still be computed the same." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Momentum\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Momentum here uses the exponential weighted average of the past gradients to smooth gradients steps in the direction of the convergence. Below are the formulas needed to implement momentum method:\n", "$$v_{dW^{[l]}} = \\beta v_{dW^{[l]}} + (1 - \\beta) dW^{[l]}\\tag{4} \\\\ $$\n", "$$v_{db^{[l]}} = \\beta v_{db^{[l]}} + (1 - \\beta) db^{[l]}\\tag{5} \\\\ $$\n", "$$W^{[l]} = W^{[l]} - \\alpha v_{dW^{[l]}}\\tag{6} \\\\ $$\n", "$$b^{[l]} = b^{[l]} - \\alpha v_{db^{[l]}}\\tag{7} \\\\ $$\n", "$\\beta$ reflects how far we go back to include the past gradients. For example, if $\\beta = 0.9$, then we average over the past 10 gradients ($\\frac{1}{1 - \\beta}$).\n", "- $v_{dW^{[l]}}$ and $v_{db^{[l]}}$ are initialized to zeros and have the same dimensions as $dW^l$ and $db^l$ respectively.\n", "- On each learning step, we store a version of $v_{dW^l}$ and $v_{db^l}$.\n", "- The larger the $\\beta$, the smoother the gradient steps. However, too large $\\beta$ would smooth it too much and it's not good.\n", "- For $\\beta = 0$, we get the traditional gradient descent.\n", "- $\\beta$ can be tuned along with learning rate $\\alpha$. Common values of $\\beta$ range from 0.8 to 0.999. $\\beta = 0.9$ is the most common value." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Root Mean Square Propagation (RMSProp)\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "RMSProp decays the learning rate using exponential weighted average of the square of the gradients. It's supposed to converge faster in the case of the covex functions, and converge to the minimum region in the case of nonconvex function by decaying learning rate and discarding history from the extreme past. Below are the formulas needed to implement RMSProp method:\n", "$$s_{dW^{[l]}} = \\beta s_{dW^{[l]}} + (1 - \\beta) (dW^{[l]})^2 \\tag{8}\\\\ $$\n", "$$s_{db^{[l]}} = \\beta s_{db^{[l]}} + (1 - \\beta) (db^{[l]})^2 \\tag{9} \\\\ $$\n", "$$W^{[l]} = W^{[l]} - \\alpha \\frac{dW^{[l]}}{\\sqrt{s_{dW^{[l]}}+ \\epsilon}} \\tag{10} \\\\ $$\n", "$$b^{[l]} = b^{[l]} - \\alpha \\frac{db^{[l]}}{\\sqrt{s_{db^{[l]}}+ \\epsilon}} \\tag{11} \\\\ $$\n", "- $s_{dW^{[l]}}$ and $s_{db^{[l]}}$ are initialized to zeros and have the same dimensions as $dW^l$ and $db^l$ respectively.\n", "- On each learning step, we store a version of $s_{dW^l}$ and $s_{db^l}$.\n", "- The larger the $\\beta$, the smoother the gradient steps. However, too large $\\beta$ would smooth it too much and it's not good.\n", "- $\\beta$ can be tuned along with learning rate $\\alpha$. Common values of $\\beta$ range from 0.8 to 0.999. $\\beta = 0.9$ is the most common value.\n", "- We add $\\epsilon$ here to avoid underflow issues and dividing by zero. Also, common value for $\\epsilon$ is $1e-6$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n", "Adaptive Momentum (Adam)\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Adam combines both momentum and RMSProp by using the corrected version of them. Since at $t = 0$ there would be correction bias error, we divide both momentum and RMSProp by $(1 - \\beta^t)$ to correct for the bias where $t$ is the number of steps taken by Adam. Below are the formulas needed to implement Adam:\n", "$$v_{dW^{[l]}} = \\beta_1 v_{dW^{[l]}} + (1 - \\beta_1) dW^{[l]} \\tag{12} \\\\ $$\n", "$$v_{db^{[l]}} = \\beta_1 v_{db^{[l]}} + (1 - \\beta_1) db^{[l]} \\tag{13} \\\\ $$\n", "$$s_{dW^{[l]}} = \\beta_2 s_{dW^{[l]}} + (1 - \\beta_2) (dW^{[l]})^2 \\tag{14} \\\\ $$\n", "$$s_{db^{[l]}} = \\beta_2 s_{db^{[l]}} + (1 - \\beta_2) (db^{[l]})^2 \\tag{15} \\\\ $$\n", "$$v_{dW^{[l]}}^{corrected} = \\frac{v_{dW^{[l]}}} {1 - \\beta_1^t} \\tag{16} \\\\ $$\n", "$$v_{db^{[l]}}^{corrected} = \\frac{v_{db^{[l]}}} {1 - \\beta_1^t} \\tag{17} \\\\ $$\n", "$$s_{dW^{[l]}}^{corrected} = \\frac{s_{dW^{[l]}}} {1 - \\beta_2^t} \\tag{18} \\\\ $$\n", "$$s_{db^{[l]}}^{corrected} = \\frac{s_{db^{[l]}}} {1 - \\beta_2^t} \\tag{19} \\\\ $$\n", "$$W^{[l]} = W^{[l]} - \\alpha \\frac{v_{dW^{[l]}}^{corrected}}{\\sqrt{s_{dW^{[l]}}+ \\epsilon}} \\tag{20} \\\\ $$\n", "$$b^{[l]} = b^{[l]} - \\alpha \\frac{v_{db^{[l]}}^{corrected}}{\\sqrt{s_{db^{[l]}}+ \\epsilon}} \\tag{21} \\\\ $$\n", "- $v_{dW^{[l]}}, v_{db^{[l]}}, s_{dW^{[l]}},\\ and\\ s_{db^{[l]}}$ are initialized to zeros and have the same dimensions as $dW^l$ and $db^l$ respectively.\n", "- On each learning step, we store a version of $v_{dW^{[l]}}, v_{db^{[l]}}, s_{dW^{[l]}},\\ and\\ s_{db^{[l]}}$.\n", "- Compute the corrected version of $v_{dW^{[l]}}, v_{db^{[l]}}, s_{dW^{[l]}},\\ and\\ s_{db^{[l]}}$.\n", "\n", "Default values are: $\\beta_1 = 0.9$, $\\beta_2 = 0.999$, $\\epsilon = 1e-8$, $\\alpha = 0.001$. Adam has proven to be effective in many real world applications.\n", "\n", "We'll train neural network using mini-batch gradient descent (MB), MB with momentum, MB with RMSProp, and MB with Adam. The dataset we'll be using has images for cats and non-cat. We'll try to build a neural network to classify if the image has cat or not. Each image is 64 x 64 pixels on RGB scale.\n", "\n", "Let's import the data and take a look at the shape as well as a sample of a cat image from the training set." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Loading packages\n", "import os\n", "\n", "import h5py\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import seaborn as sns\n", "\n", "# import local modules\n", "os.chdir(\"../scripts/\")\n", "from coding_deep_neural_network_from_scratch import (initialize_parameters,\n", " L_model_forward,\n", " compute_cost,\n", " L_model_backward,\n", " update_parameters,\n", " accuracy)\n", "\n", "%matplotlib inline\n", "sns.set_context(\"notebook\")\n", "plt.style.use(\"fivethirtyeight\")\n", "plt.rcParams['figure.figsize'] = (12, 6)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training data dimensions:\n", "X's dimension: (12288, 209), Y's dimension: (1, 209)\n", "Test data dimensions:\n", "X's dimension: (12288, 50), Y's dimension: (1, 50)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAF9CAYAAAADTgNSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnVuvJNd93VfduvucMzcONSIlUZTiW2DDiRMgQZCPkO+b\nh+Qj5MkBYhiRraspmbQokjPDOZfuuuXhKHnw/q1BF2x0YOz1e9xTp7pq167/FLDWXv/m9evXq0II\nIVRD+//7AkIIIVyWFP4QQqiMFP4QQqiMFP4QQqiMFP4QQqiMFP4QQqiMFP4QQqiM/hI/8l//238v\nxtZ5xmOnaSzGxtMJj313+7b8+/GIxzZqirHTwz2f9/VXxdhyz8dOt7fF2NXTZ3js1bPnxdg48r3N\n41JeA+y4+O4nP8S//96nPy7GDldXeOy6liemMUmaxqkYe/Lc3O/VTTH2cH+Hx97evSvGjkc+ltbI\nMpfXJUnLUs6j27jy5En5fK5gzo6nB/x7WqdXV9f8W89eFGNDz69juXKl/YGf5R28E3wG6ebmaTF2\ndV2OSVLTlue4vyvXviS9+bp8f7769a+KscmshemrfyjGupXvoXlSrrG2KZ+5JA1LWXPa/R6P7dby\n2Kuer+EZvNf9wM9nbspzHD7+MR775Ad/UIz95jU9X+k3vy3n/L/8p7/AY6V88YcQQnWk8IcQQmWk\n8IcQQmWk8IcQQmVcRNy9vy2Fu5MRYU8PpXC2gNDyCAglBxbT5qkU/44ri0sDCT4tT1UH/3euIChK\nUtuV53j6hIVRgQj09quvi7H7Oxad37x5XYwNux0e24OoOMF8PY6XwqoTbFdQo08nfu7HY/nc3TWQ\nYDubOZ/G8nqblr938LxzOebMBnQPbj125hoIWjfuHqapfFdG8661bQdjAx7bwTUss5PJy7VLR9q/\n3pXvXwNGAUkSiM6rMwX05frvzXvddlADDgc8dt7D+Gqe71BeA71Tj+Pl+n928wSPfXvDz9iRL/4Q\nQqiMFP4QQqiMFP4QQqiMFP4QQqiMFP4QQqiMi7h6TrDF3TkNpqkcN+kBOsC29Q6Ue4mjBlqzRf5w\nVSrnw3XpgJCkcV9ew2xiGPqhdEx0DZ/3+lnp9iGHyMM9xwd04Nh49463ez99Wm45J4eLJM2w7d25\neugc44kdDOSUWVa+BtG4WSN0DS04piSpgXGKrnD3QLEgbh5Px/J+r6/ZAbQDlxm5bCRpns+PPDl2\n5drpOnb1tOAiWsDxJPG7jYkLxpnU3pTrUSZaYQEHT2ucOhrgHA0vnKYr359lxzEM0658V7uDcUf1\n5XkH89xniCG5P/L1fvjCuAMN+eIPIYTKSOEPIYTKSOEPIYTKSOEPIYTKuJC4S+ISixQ02rmc8qb8\nf2sywuoKgiCJrZJQdGrhtyTp0JdC8GJ6DbQDbL03WenfQuTC81evyr8HsUiSRohGGHZ8v/cgznYg\nbkksdj48sFBPojyJw5K0wPhq1giJvvR8JUnw3DoXvwGCKW2bd4ItZdZTZMTjOcrzuhgGEupJbJU4\nYsJFX1DERGueewsmBHpmknSkPhfwHBqIL5CkdQCR3D1fehY7jlbAY82nb9uXQjBFXEj8vrf7st+C\nJF09L8cP1ywaL105Pw2I95J0f8/jjnzxhxBCZaTwhxBCZaTwhxBCZaTwhxBCZaTwhxBCZVzE1XOE\nbdVuyzm5Z5yaTq6PBbY5S+yY2FGzBUlLR24Fs7UbxrrVbHvvyGFiXDn3t8XYEZrUHK65MUMLlhrX\n8AFjDYxbiO7XuTtO4LBaoDnLI+U4zZckrdAAxDXr2UHzmZ1xfVBDmltoIrTKuHoE7htzDzQPpyO7\no/ZwD84BRK4r5zIb4fl0x/MddNTcSDLOOnA8NaYx0ArNZFYzN+S+se8quLnaK+MAgoiIxrgLO7oP\nEwtC7qa+Z1dPA3Ewzxp2LTrHniNf/CGEUBkp/CGEUBkp/CGEUBkp/CGEUBkXEXdJPJxmzpHfwTbu\n3m5PL0Wg27tSjJOkCWIjBrNlfL8vBR8X70BJDpTrLkkiYdMce/3kvHxtyl+XpAXO28z8WwuIYSvk\nxUssRi+mYcJCYrI5tgXxz2jGOL+96cNAz5hEXIkF1z2IfHStEkcg0BqVpHsQ712fgB5iFA435lgQ\n5bdEnjgheFnKZ+nuDV8KiskwRgxcN6YGNCCeN2bhtDsQbF1PALi2puE11vRlbXC9Pga6BmOkoNrQ\nG1PMU7P+HfniDyGEykjhDyGEykjhDyGEykjhDyGEykjhDyGEyriIq2eA7vbUQV6SBthO7yIbjtAA\nZDRulBEcF6eHMkpCkpb5phg7HK7x2B62azvXBwUeuHlYlvJYmgcXl9CC+u+aWSwncjCw++a0gDsK\nXA3u95zjibbTryaGoWvL3xuM64rmzEUNrEP5e/tDuW3euVnGDc4xwdxQhILEDqAdOM8kduo05vuO\n4lFG2zgG1o5xaNGapnU6m99Cpxs0gnkch4gXM+fk4HGNTRpogjKf+NixLRvP7GDdSNICjXKcI22F\nd7iFv5eku3tofvPUO33yxR9CCJWRwh9CCJWRwh9CCJWRwh9CCJVxEXGXcvOdWEpimBPTaIu8YwAR\n1uXTHx9KocTl5qNY6bbI0/ZwF08P/zCBaEb565I07CCygbbSS1ohqmBejegMz8L3QIB8erdFHmIJ\n3OTQOZwBgK7NCZgU5dB3pWhM68NdlxdAYc5dXAKMTzYLv7w3t8To99x5eZ3xmSfI0ydx2JkN6LQu\nzoLm3BkI6B4aeL6S1IAhxcUwUE7/eOL+AXfQ36GH3H1J2l+XJhNXs7aSL/4QQqiMFP4QQqiMFP4Q\nQqiMFP4QQqiMFP4QQqiMi7h6jsdS4d5DNIPEDU9oK7zEKr1zuZDST00RJGkFB8Jo3A7dAFuwNzTU\nwK3wkmbamg0xCs6hQvOwN04qckYsE9/DBE6QxtwDmSCcO4OcRW13vjvDrZFxLJ1fzk3Snsp5oOV0\nPLGbrIfoChfJgXNu4jdozicT70DrwbmFaM6da4QdYTyP5GjBe3NNeSDaxDUsWikWZDXvHzgGux27\nevpDWRvo7yV2DDoH0AwxJM4B1EKUw2DcayvfhiVf/CGEUBkp/CGEUBkp/CGEUBkp/CGEUBkXEXdH\n2sJthB3Si1x++rycl/stSV13/tbuBkRY9z/kBMJMC78lSTvYBk55/hLHM9C9jTOLfEcSjExcwgAi\nkp0bGCeBTeLrXUxsBImgbWvmhtYDCJWSdAIR1N3b2MBcwjp1AjWNugiEDrLW7TqHYx+OHBtBmf7u\nVRtByHVmA2oxMRqRGwVtiksw65HoMNKDewo4YbUdyvFuzwYPyvTvzLvagCGF+mFIUk9REObexql8\nls5kQsaC95Ev/hBCqIwU/hBCqIwU/hBCqIwU/hBCqIwU/hBCqIyLuHp6UNOd2+Hbb98UYw8Pd3gs\nuSuc+k/b1jvjMCFXjmsggs0sTIzCfVM6MXbGKbA/lJEWCzhXIMVBknQaS1fP8Z6dIM3V+fNIjpjO\nNLMghwg5sSSpXSnCwLUQgfOaaIQtDUTIhTTP538boRPLxCVQY5+T2bq/g2iR471z1JRzQ84XieMZ\nGvMtOC3gMDHrnOaXHDy2uQpcr1sJDdmN+LTYEMrGRoAziNx+7gedY4nqiL03GDs+sIvvt7evi7Hv\nfeeFOXO++EMIoTpS+EMIoTJS+EMIoTJS+EMIoTIuIu4+ffK8GHOi5O27t8XYwwOLki0IOwNsn348\ntvw/zm2Rp+zvxuRgD7C12+W9Uxa3y3Yn0ZhEPhfDQNvAVxNngXngRnSm+XV57yRgurkhVpivx/Fy\n8bioDjdOYOY8rFMXybEYgZkpT+zmka5rNrn5JIw6IwXN4zzzeUk8970vYAx/n9dCA3EHnYlhQMHW\nmQJg3F0DvSttw/EOZIQYTIQCvT97ExtB8SaL+VT/8ptS3H0f+eIPIYTKSOEPIYTKSOEPIYTKSOEP\nIYTKSOEPIYTKuIirh7aid+b/nOOxdLnQ1nJJ6kHpn4wzYgCnQGucOuTOsOo/KO/kZnk8MbgKzH+9\n5KK4uy23a++uymgHyWwZb4xLBiwXzn3TwAWfTryNnBxPzmFCz7Ix14uuKz4SYyNcsxFyz9A9kJtM\nkmjpUcOV3/8Y/BY7Qcg14iIqaNxFQczgXHFxI3THbs5xQcHR1JhI4hiG2VwXzm/HV9ZRbIRxxa20\nbozLbGjL59abZ0nP2Cwn7cDt8/TZUzz2foN7TcoXfwghVEcKfwghVEYKfwghVEYKfwghVMZFxF3a\n2X2azhec7NZu2hpuhBKKS+gaFmFpe7i7BooEcPEOwnPwBa+k+IAQ5cRSEiW7Az/ucSrFWbcdv2tJ\nhD0/791BAjGJW4+/B+KuefAL3IdfT+edd4s4bMH5NfdAfQ1stAL1FOA1QnPujiXDguvZQOM0tprc\nlvkIZhAnktPzNc8H17R7lvAsnGmD4lEcJCb3vYmDgfNe79nM8erlB2dfg5Qv/hBCqI4U/hBCqIwU\n/hBCqIwU/hBCqIwU/hBCqIyLuHo6cDvcH7m5CsUlLKaxwjyXirxT/+kcbWsaNqC5wvwfCadwTSPI\nKeCcIBSZQFEB88RbtfuuHN/vr/i60C3ErhFyxHSmMckR3Bluizw5VyjG4X3jBDkxZttApJxfbGxi\ntsdvcZnR/To3CjltptFFNpTXNppIjdOG59OC86Q3zUZo/bew9l0MA70TzrWFw2Y9EqYCaIF/cXWo\nBaebrQFwwc5BN4HbrhU7gD7+8CWOO/LFH0IIlZHCH0IIlZHCH0IIlZHCH0IIlXERcffu7rYYo9x9\nyQlONvm7YDXb3mmUtvM//l75/6ET3khkc0LUbldGEKwu4xtz4EHYMU+QBEES/iRpD9fVm23oM8xD\nY6IvSDlzkQA05y4aAX9qg2Drng/lta9wE+7vMYbBXBfdG4n/EgvtLqqDIkRGI6LSenDCOd6zeS9R\nuN4wNy32ZjDfqBt6M1CbAF8vKLLBrBsYprgFiefRvRM4O8aQ8uLmBscd+eIPIYTKSOEPIYTKSOEP\nIYTKSOEPIYTKuIi4O46U+81CYwu77pxwt0l4Q6nEiEt0DiN6TadSyHI7JUmc7UwWN80PCVHudin/\n/HhiQZ16IOxAXJZYhB3M3BxBDFtmt1cSMM+d5Tsn7tKY6YEA56CdynaFkX656bpMXwMUZ83coKBu\ndhpvyLKn3d12HmFHPYnOvdllT7vvncGD3lUShyWpgdrisvRJ5Ma6IGmFe3OCreAZ9w3vgG4aeha8\nRhq7B5nJF38IIVRGCn8IIVRGCn8IIVRGCn8IIVRGCn8IIVTGRVw95IzA+IHHf4G/N64eyMd26nYD\nv0fXJbETg1wykjQcyrgDlzNOGfvN6lR6vrJzoTgKdw/0LOha3bEu356cEeTukEy0gsuGJ7fQ4rw2\n4Pow53XrrPwtEyUBf+9iGOjeXHzHOJZurKbhV9c5eAi8W5d7T79legIQlMfvfot6crStyf4HB5Dr\nyUHrsdkUB2PGwakzu+x+WCPOLUQuL7dGac7eR774QwihMlL4QwihMlL4QwihMlL4QwihMi4i7lL2\nvosaIOGNGmY/Hnu+MMOC3gZhx4gq1FSZct0lI7xZQZGaPZei4uAaXsP2dBcJQDEZVnyHy3Xi1DCU\n1+YiAeyCoEsgEdVtp4cQdifucrN1aI7toj5AVFyMYEvi7mwy9klE7Xf86i6wxnyMCWTOG2H0XOFb\nMu8wirBOvIdxm01yfpQEnYPWqGSMBeadIB3XNWaneZyMIN+R0cXUPOqT8T7yxR9CCJWRwh9CCJWR\nwh9CCJWRwh9CCJWRwh9CCJVxEVfPCRqTGEFfAzQA6fasvO92ZVyCjSVoynFvJDmvIYckCdR05xoh\nXCwBOkzgWButAP+nO3cUxSjMk3EawPw6FwUd61wU5IFwzxLjHcw18LMwDi38vfMb+KBpysxjA803\nqCGO+zVy70jsHGuNm0twu24eaZwiECTn6sEf4+uitW/cRrgWTHGhYynaRJI6WKfuXSVstALUCxez\nQc6gAWqe5JshOfLFH0IIlZHCH0IIlZHCH0IIlZHCH0IIlXERcRdFSSNk0TZyLx6Wl++E1U2CK4qH\nridAKdY4wYjEoS3XSzEKvRG9tkRUdC2cw7VLgFtzz6fvQMiFCAWJow2cuIu56hsiHxz0e5gNb4Q0\nyt5vzP2S0O627tN6apzZgGLvzRrrYdwJtjTu4h3ccyuuy4jOHDdiTjJDjwvz820PPTnMaUmEdWuM\nns+WWBDX34G8GE40bs6c8/937k1HhxBC+BdPCn8IIVRGCn8IIVRGCn8IIVRGCn8IIVTGZVw9tIXa\nNBSgxiaNkem3NOQ4NwLB45pvgFPAOS7gHFuiBmjr/Yb+GLaBCA2vppEERS44F8dEjhiznX6BJiTu\nvC724Vy2rBE60huIyA1WRpBIUgcOk2Xh++3A9eEa2qzkcsEjpX4gV5xrekTvJT9LWpNnxzjIuerw\nUHQsWYcXTFnjGpuAw2oyjXIOh/IZ2zUG1+ucOjTs3JDbalm++EMIoTpS+EMIoTJS+EMIoTJS+EMI\noTIuIu6SzEEiriT1fSnc7XYskJGg4YSsjo7dIIj8cwjBeKQT6UDZWdZSXHJZ3qeHYzF2e/uOL2J+\nKIaub67w0N3VTTG2v3qCx14drouxBkRNSWrbMmd8N3D2uIsVIChGwSmFLQh9y4aYDdp6vy4mkgPN\nBua6KErCCI20njp4pyRpgPmluITfn7kcsSYGyr2Hw4yBgJ6ZE537/vweFdTvoFmdQI2yPh6LQq4p\nRHQNNt6BrQV8YjvO5Is/hBAqI4U/hBAqI4U/hBAqI4U/hBAqI4U/hBAq4yKunhm27g/GqbOFLTo2\n+QfI6SO5LePn/5ptlgCnaI3jYgX1H89r3B1ff/m7YuxnP/krPPbj75TLYHr5DI89wSV88c0dHvuv\nfvzHxdgf/+G/xWM7aNriHCbG4IFsc/Wc5xIjl40kNRM8M+vYoMty6xHcQmaNURTEsOM1RpEJzj1D\n7jG6rsdzlOupp/O6dwpcOSs06pGkGVxT/Y7dYOw2MnEJ6GLiYzlGwbzXm2Jmzo+6cdETjnzxhxBC\nZaTwhxBCZaTwhxBCZaTwhxBCZVwmjx+EnQ4EoMdjz8+r3iJO0TlOx3s8lqTgHsRHSShQbckOp637\nktnGDdMwG0Hw9l0puM5zGeMgScNVGc9A8RCSdH88FWN//ZOf4rHzUs7v05syxkGSPvnhnxZjm6IZ\n7CfM+fEZzbn9Eqx4T3n+/HxpPVK/BUlq4OdoTJL6AbLhjcA8nspneXv3LZ8YBMzBPB8Sdw83T4ux\n6xuO+hj6Upx1cRZUA1y/BlpPs4lM2WLmWCB6YgahX5I66IEwQw8FSWp6WI92Pb3vCkvyxR9CCJWR\nwh9CCJWRwh9CCJWRwh9CCJWRwh9CCJVxEVfPAE4Dr7yX467hA/Vx6F3DFBDDF+P4GMHt83B/i8fS\ntXXGWUTxDLvdAY+lOWvg/+llYZV/xq3hZss5NX2ZRjz29q5s2nJ7dM1kynN8+flP8NinTz4sxr7z\n6gd4LLtnXEMN2P5vnju6x8Bp4xp9UCMW59rCrft4pNCy4Zw6A7hcnFPnJz/5n8XYT3/1Mzz2u8/K\nBjyvXpROHUka70tHy6jy2D/813+Of//hd18VY7s9vyc9RFScjuUalbihk4uOmWd610yDF4ikmeDv\nJY77WMUOIGoC5NZeF1dPCCGE95HCH0IIlZHCH0IIlZHCH0IIlXERcddscMdR3qbPZ5jnUjycTYY7\nCTu9yVXvuvIaBhBwJBYP7XZvGHfb9NumvDaMfDCqzm5f3u+MueHSPJb3NsLvS9IIp+gGI7yByD2e\nWHj7/LO/KcaePy9FPsnEEjjhmhwAG6Dns8qI2SD+LTZznvLezXlx/fN5T8cyluNnf/uXeOyXX/2q\nGPvlZ5/jsVc/Lp/Fh89YGO125fvz93/362Ls+roU9B/Hy1gPeicln9NPkDBK/QskSVM550fzfJqm\nFL7d9U5gmnD1ooVatkBfhMff42tz5Is/hBAqI4U/hBAqI4U/hBAqI4U/hBAqI4U/hBAq4yKuHlKt\nXWMFVrhds5JSkeeO97zdmpo4SLyt2jf6ANbzmzjYxgowTFEQrj/Ms+cvirFl5ZuYoCFH0/KJZ3AV\n7MWOpz0848a4hb784jfF2KuPv8BjX330CYw6V8/5bgdak7SetviEppGjLzB2wsQwzFN5D/QcJOnz\nz39ZjN29+wqPXcl11ZRNUCTp9lTex+nEz72DeZyhsc/9Azu8Jri3cSrXqCQdhrKJkIPm7MFcwzU0\nDJqMs28cy2ujyBVJWqDpiouNINzaSyOWEEII7yWFP4QQKiOFP4QQKiOFP4QQKuMi4m6P25eduFv+\nX9R1TiyF85qt+y3FHRjFdoVzbNn677Zgs27tcuTLMc7i5uu6efqkGDtccX76u2/fFGMf7Hlp7Pvy\nJj795CVfwxUIZJT5IOnbb14XY7/4OWf3P//go2KMctkd7vmwQEbRCnxeWiNOEKT+AS5+4wTb/O9u\n3+Gxn/3yb4ux/RWLh81D2XeiMaJxSz0bzL3NM4i7IGr2ezYQNFAv3JyfwJiwtz0uzj/vPJXz4ARb\nOnaaeG7IQEAxDpLUgMlkMPXC5fQ78sUfQgiVkcIfQgiVkcIfQgiVkcIfQgiVkcIfQgiVcRlXDzRX\n6UzDlPPDDjhywUVBoPvFKuHlsc4BRA4RasghSQs0jXBXSy4kvC6zzX9/KJ0NH33/R3jsz/76f5R/\nv2OXzNMXz4qxmyflmCR1cG1vXpfuHUl68/a2GLv/rGzeIUl/+m/KZiMSRw20FI3g1h4tEXJXmec7\ngpNjAjeLJDXwLBfY+i9JIzTK+en//is89ttvvizGhp5dVwd4V65NesAenHWrscRQTAVN2ZNn7DIb\noImQzDqnK6AIBXcKF9tyhIY2Lm6EXIfkNnqkvGKTuqL9AaJuTIOXbZUzX/whhFAdKfwhhFAZKfwh\nhFAZKfwhhFAZFxF3UUCx294h/3xD1nTT8C21oNyt5iLo98wufz6DiY3oWtgyboTCFfsKlP9Pu7lp\nQcD86AeUYy99/tkvirEv/oGz8F/CNbhIgPv7UuR786YUcSXpd2/KCILdXEY+SNL9XXkOEpIl7tnQ\nusW3Qh8GeJZuOz5FNiyz+S1YT6PJt3/7pozU+PkvymgGSbruIMv+7g6P3YGS+8d/wGvkuimfpXvX\n7t+Vz2e/KyNEqGeExKLxaiJTFniWfe/WQjl+uLrBY/f70iywOjMIDM+mf8AEl0Zr9PEc5Yldfwfq\nIfI+8sUfQgiVkcIfQgiVkcIfQgiVkcIfQgiVkcIfQgiVcRFXzzCUCrlzyRDUnMVh4xLg91wMA6n/\njYuCILOBOZTOy20vZOw65zeIIQfCzRN2MPzJn//7Yux//WUZ4yBJv/77r4uxnXFRzOBAeDixK2Ec\nS1fCy4EbakwjRTZwbATFKyzOCgWNUGZoNuIiG8h5spjIhi077N++fVuM/eaLb/DYj16UTp3DgeMs\nnu/KsIAX1xwgMIDj6XhHz0F6/aZ0Ef3oT/6sGLu+YdcWxqCYZ9aBs6gxLhmMXHCxE9BcxVkRG3iY\nNCZxQxqK+pCkO3CvucZNGHPxHvLFH0IIlZHCH0IIlZHCH0IIlZHCH0IIlXERcXeaS0FvB4LvIyTG\nmUzzFUQVt3UfRBHajv/I+SIditRGuSaByuXpk+hEcRb2HowASXz46lUx9hf/4T/jsZ/96ufF2Fe/\nKzPgJWnty+f+8XdZhP3gxQfF2Hxi6ZuyziezRR63/ztBD0TBeS6vYVlMZAOsEfdb/VCKcfRbkvRw\nW8ZZNMYV8BVEYvRmO/8MAmZvxN0V7u31V+V1SdLV04+Lse99+ml5oBG4STwnwVcyMQobIl6c0N9j\nvTi/14c7ltbpNLHhgdjv2fBgzSeGfPGHEEJlpPCHEEJlpPCHEEJlpPCHEEJlXETcPZ1gh58TVUiI\nMgIm6RnrbHKp+/PFD9zRu2mrsRmGcXdaEqlJMHIx3F1XinSjE0BBrPzgQ27Q/exFmaF+gsbUkjTD\njsRhMK2l4RH/5u8+w0PvIF/+2XMWXAnelSm1LZ2jvLDT0WStmx2YxA7W/2wadD+8LfP4/+hHP8Zj\nX99/W4x9/Y4b3L+7LZ9bD7t5JYk01A8++AiP/bN/9x+Lsecvy/U09PxbZFhwJogOBHlneHDiOZ4X\nmpqvK//9OMJzN4aUBc7h8vjJGOCMLq63iCNf/CGEUBkp/CGEUBkp/CGEUBkp/CGEUBkp/CGEUBkX\ncfUccJuxUd4pasB1t4dTdJ1R/yHewVlqSHl3Gd9dW05hu8UBZNX4bSr9P4bcUW3LMRl0tehUkNTD\n/O6fPsdjWzjWuTMe7u/L817x9nSKrrAZ+Rh9wcdSVvoM2+mde4d+y23dX8Bh8nAs3UqSdIK5+f6n\nP8Jj/+hFGYnx5ZccqfH2Ten2aWE9S9JLcOX8gGIYJD2H+I0WIwX4OdD707lIAnDwuJgMjmJhp84M\nMTO07iTpBP0hVnPeFt7Lvuf3su/hWbiEFuNUc+SLP4QQKiOFP4QQKiOFP4QQKiOFP4QQKuMi4i6J\naXYLNgiCXceNhFk44/NyT3RzLIi7KzTdlkxjdmgA7bCN5EmLBoHLbUNfQThjgU3a7cr5daIkZeGP\n2Pxc2nelOOt0bxLkrm+4OTwJb3RdjtHkn9M9TxA34uac/t6t83Esr/f+lsXd/rqch+snPDcfffyD\nYuzDV2U+vmSy6E3cwQBC42DiHcgAQIJt35u8EXhb6ZlL0gpN4BsTBUEvlVvnM7zv7rlP8CynE19v\nB/Po6tBx91CMufXUL+6emXzxhxBCZaTwhxBCZaTwhxBCZaTwhxBCZaTwhxBCZVzE1UPKuVPTSbWm\nZguSiQSwthG4BtMxpYUmDG6v9ArbuN3maVTkKUpC7HKhv+9Ng5ktUQV03v2et5F3sOX8aBqxTND4\nZVn4WU7g2tjt2amwLOSk4m+YCSMXjOOCHGVDuRZcMwxycmBjIcNkXCPPv/OqGNtfX+Ox1EhlaExz\nFVgjrqFz/0++AAAVyklEQVRHAy4xk46Cbp2+Lw9ezTzSinbPF+uFmXM61sU7jBvcXDRlW6I6yBUk\nSceH0tXjIht6WKfvI1/8IYRQGSn8IYRQGSn8IYRQGSn8IYRQGRcSd88XduapHMdcarEI5DIBqJO9\nE2A6EHd9wj6IqObe6J5ZSJY6mDO6Xpc8TsL10pp4B5tfDtcFwhnNlySdRso059+i87pIgP3uSTFG\nQr8kTQ8g5Lr7hWESxN16bOGZUeSDJGmGzHkTNdDSNn8z5yQe9jtzLPUVMAYAErmHwUQ2gLhLpgu3\n7tgMws8X34kNrSxcHSK6zkVUlGOj+aamKAjb6wNFar65h3uO+3Dkiz+EECojhT+EECojhT+EECoj\nhT+EECojhT+EECrjIq4ewjlByBmxxXVioyDIKWAakzSgnLvzNg04GMQq/ULxDmYbODo54Hqt2wim\nzEVfEI25B5qGrjP3AM/4/oHdB7z1ntdI35eNYyjyQeIGHr1xo5AjhuZxdW4wsHfMJ7Nu4LyHq7Jx\njcRrxG3RpzW2QsSFJHWwnvodXwNFT7SmkUpLqxKiSVxjIH7fN1h1DBhRYSJTKL7DNUyZqEmTeTFb\nmkezzsnJRA18JGkkh9Z7yBd/CCFURgp/CCFURgp/CCFURgp/CCFUxkXE3QmEh93O5InDtvV1ZfGQ\nYgko6kBySQ4mWgHO68RdNeU5nAhE9+ZYVtpOX4pTLnu8AbHU9R+gDHZ7u/QPZs7pJNNUCrOS8FHM\n5t5o2/tqogYw252vQCvML8UzuL4GGDVgoiQwWuFgeiBQBILtUVGOURb+43j5LOx6okGjt6IJYUNP\njk3H4pXx81lIhDXxG/SMl4UFVBKN59n1viDhm4+lPP55wzW8j3zxhxBCZaTwhxBCZaTwhxBCZaTw\nhxBCZaTwhxBCZVwosqFUrX0ThnKMYhwkF4FgGis05a36CANS9M9vIIL78SWtcF7bDKalZjAU2cBz\nQxEIzs9CERUONPVYJ1V5v841gg4eMzfTDFvvzfOZp3KNdMbJQdv0ufHM+dEXXWfuAVwjbp3v9qXb\nZzVrbIAoB9c4ZrcrXT3uGqi7iXuHqSkOrl3zfOlZumgF+i1e+3wNLjKFXGJujY0QC0JORonvue85\nhoEjLVyjqW3f8PniDyGEykjhDyGEykjhDyGEykjhDyGEyriIuLuiCOvy0ymGwYlp5bGLEZx6EFWs\nCIRbxo2IuiHnf1tfAfgtEKPtFnsUzozwvUE0RoHafT7gzzlxCu7NiJKU7T5PvJ7o11ZjACARlJ6v\nE51ZEMRDsd+Bm3Oa4LZ15ojz+xqQuYFy9yWOMPDCaHltKPi6GAaYBvvq0ASbOW+hD4N7JyljfxlZ\nsJ3hGlykBv69mcfTkURf1wvi/DgYKV/8IYRQHSn8IYRQGSn8IYRQGSn8IYRQGSn8IYRQGRdx9WAH\n+GPZZEBiB9AAarzErgDn1KEt1KuxXAygkFMzDIkdPK1r9YFun/OdRbSdfstObetMwqYtxtUDJojF\nzOOylPcAiQKS2HniHBfLBL/nIkBguDVb76nRDUUjNC6Sg2IYXKQGRDm4tUvOrVXGUQOX5uaRxp0r\nbsu7xo1JyuttzG/Rse63sLmRcQtRA58Z6o3k5uz8ufHNo8r7cA1eqGa5d9g64Az54g8hhMpI4Q8h\nhMpI4Q8hhMpI4Q8hhMq4iLi7318VY5R9LnEGNWXTS8LdyzbjG1UvPi2N2+xxFFuMuAsiKEUgSFLb\nlqIVzY0TYTFpwImHsL3cbTnH7eWrE97OzxOnSAAak6Q95NOfbvl6jyOIik7khmVGyQir2WLfgDjc\nmHvoIQt/dzjgsQMcS70oJBbJXeQJi/oMHmveNXyWFGsAsRWS1FPEhM0FKbGmAFy7/HzoFBTjIPGz\naMycd/15vQoknvOuK9eCZETu95Av/hBCqIwU/hBCqIwU/hBCqIwU/hBCqIwU/hBCqIyLuHp24Eqw\nTRjAUrMlasA1JiEXETWH+P1FFDgXRQOxBNTQQ5JamG7nXFnmsglDA3/ftqzyt2BRcXOODW1MrAFv\n8zen3dB4hp7llgiDxrgzMBbAzHkLDVrIEbOY7yXaet+ZKdjvSmfSMJhnSU4d01zFdn4ByBHm1wj8\nvXnwNN7BO+x/Ctvn4JHsMnPRCmcfqhkiF2zjGRgbXB0CZ940m+vF0fOf7/vIF38IIVRGCn8IIVRG\nCn8IIVRGCn8IIVTGRcRdEgSdiMSC3vk52A4SiN1WaT7t+RnsVu2kyAXM6Je0UG53+VvzNOKfc346\nC070fNy2dxK4xpGvYcbz4qGbIgHoJK05doDlNJh56FeIC6GfN2aDaYOQTDEkTtylM3hzBKwxeyxc\nl/kW7OD5LLYnwHnryWr/W3pM0O+bY1d4L0nElfgVbkx0DK49s3RnyNifptLIIQlvxMWubCiFkvLF\nH0II1ZHCH0IIlZHCH0IIlZHCH0IIlZHCH0IIlXERV88WxwY1NnHOF3auuBiG85uC0Lg9LYzNLt4B\nrsE6V85U6d1vCSIfVu4joQWiClbrpCp9Lifj6qHmGbZJBrhfuMkNx3qsZh5aGG/N1ntqSNNM0FTE\n/P18d1uOmetaXoBXxy0ycgaZaAZqeGKbCMFYt3ONjPjSCFrT6OqxcSXkAHINU8Ax6Nx6FPFinDo9\nHLuC60vycS54LNyHvV6IlOl7dn4NQxkB8j7yxR9CCJWRwh9CCJWRwh9CCJWRwh9CCJVxEXF3mwgL\n26pNd3sWrTbEO3RuKzts1N8gbjkhalZ5H064bkFEZdGMr2GaSsHJiXy4NdwIqyM8i8WeF4RRI4Rh\nvrwRuPne+ODTEUTukdfTDmM9QIxzgi2Mu+34I1zXfkPe+wzZ/48Hw9Hm+awbsuxbihvhK0AjBb0T\nZCqQeB7t2gVcNAndmzdHnP1z/ICgT8fjJZCRwplXQIw2vT4OV9f++oB88YcQQmWk8IcQQmWk8IcQ\nQmWk8IcQQmVcaOcuCZXuaBKCzxeBJrtrlnoC8O1jdr8RjOjeKLtc4ubutAtVOj/T3E0jZve7bHho\nDN26ZtHYoJvPSyKsE42pQb3b0bjC/O6vWdy6a4di7PUXn+Oxhx+Wux97EJ1vX3+Nf//lb39bjD3/\n5FM8djiUv0XPTBK+LHb3MIy7pugzvBOrEY0pcx7+/HH8zJ36djcunvP8nf425p927mLHBbExwe6W\nBjHaPkqob+Zd2x+uirFnz17gsYfrG/5BQ774QwihMlL4QwihMlL4QwihMlL4QwihMlL4QwihMi7i\n6iH3DXamFyvczglCYrjLZT8+3Bdjo1HTh6HMvHbbwMktNAylk0TivHe3BdtuOy8ugIfp7517gNxC\nLiaDjCeup8CuL+fBOS7ICeXOuy7k6mFXw8sfflKM9TbugLLsy/U0m/4DH/7wR8XYk+99hMeSC2nL\n81nNOzEtdG0mRx5y78F49ngGiDcZT0c+FsYGWAvux8jBs6VvhXMAYVyC6w9BPSrMsTt433tTAyie\noTPHPnnyrBh78fIlHtsmjz+EEML7SOEPIYTKSOEPIYTKSOEPIYTKuIi4i9vINwhZThLk/HMW3hYQ\nK9uJRaBpLEWrxuzBbvvyPhYQhyUW7/b7clu2JA27QzGGmfUGimfwDdQpa918E7QkBJtG51v6MGzo\nd4DiuRHJOxD6JiOQzW/eFmPLWN7bYLLPn3z/+8XY/vlzPLbd0n8A+gdQLILE0ReriWyg9dhZdbcc\n2pKRv8ADNu0w+OeNmL0lDob6RthG9HQSc2KqQyScS9JTiFy4fspr5PmLcp0+ecbHOrHfkS/+EEKo\njBT+EEKojBT+EEKojBT+EEKojBT+EEKojIu4egTb3qlJhyShQcSYB+a5dPDM8wmPJT2+73irdAsO\nHnLvSNLQlw6ejrany7lyjLNoAicHKPet2Z6Ok2YcDLiN3LlvyCHinCD/RJwDiBrHrK6JCTisdjfl\nVnhJevu7b8rz3pcOr51xBe2fPCmP3ZfuLImdVK7RBzYsmU3cQVOuPRcxscAam1ZeT+T2cRETzblu\nH9vIBS1EeCxGi7guKPhbPI/0ay5mZrcr11g/cGmldeqcVMO+PO/hiqNJKGbmfeSLP4QQKiOFP4QQ\nKiOFP4QQKiOFP4QQKuMi4i5uGTeaJB27mPx0Er2GgcU0ygN34lQP470RbCl3m8TH942fywRCVsMa\nOW9lN5POefx8YhIlnehFYrTLVSch14m7OG6OpVz17sDZ5Q2IdNOpnPPDd1wmOvRxMFvpScBc4Vp/\n/y/liBGzaXbdcx/gfh0kQLoEkXP7bzTn9pyQNLu5gRqwgJnk8dhyHt16JINGY2JBVjjHYPLxqeYc\nj9zX4OH+rhi7vuJj93uOEXHkiz+EECojhT+EECojhT+EECojhT+EECojhT+EECrjIq4ecpjMxqlD\n9BCLILHybmMYyAhitkpTM4veWBjI7dC0ZlobcBVsaGZB7g6a28fx8roWF2uAbgd36PmNckz3Dj5y\ng+OJrmExLiRqqjNDVIEktYfSEdbDPDZmOz5GDbhEDTfBAN2vc+pgw5LBOKnocm1UBzQbgbHHa6Df\nOt/hRQ2D7HTB9fbN+XEJzgFEq9S5kLi+8Trv9+e7Cx8e7ouxd7ff4rHdrnQRXe280ydf/CGEUBkp\n/CGEUBkp/CGEUBkp/CGEUBmXiWyAbdUuvoBEIB+BUF6+E2zbpjyHPS8JXEZdwnh6OREV4g6MmIYi\nHWqHRvU6XztE8XBdWQBtYB6dSIdRA0YHRuHZHQtC7ggiriRNx7I/wwhjkrTQPMDzceLwdCrPa2Mn\n1vPF7K4Fw0J3/gNujQiLsQ+2J8D5/R3wXcHLNf0hNoiwtP5tHweIV1nMc6DIEopMkaRuhXgH16sA\n6xubV1owqown7q1wd/euGHv5LOJuCCGE35PCH0IIlZHCH0IIlZHCH0IIlZHCH0IIlXERV49wq7NR\n9GF8MU6DFtwzjfm/rIX96XbLOLk7Zr5eimxwTTJwa7bdtn7eYAMxEO5Yclc9nqQcogYmEkdfrCvP\n+QrPxzUmmafSrYDOJnHcx2zcDiM0uRjvy63wkjTdPZSDMDfjAzuIWohycIan1jT2QSj5wjrHqJGR\neZbUHMVndcB1mdgIuGCM2XDXBYusMzEoVBuc62ppwF1omqvs92V8h7veE6wxjjbhNb2/usFjMY7F\nzPk48vp35Is/hBAqI4U/hBAqI4U/hBAqI4U/hBAq4zLiLuGSBkC8sLEGMOa2VVN2uI9KL//FiUDC\n7G8Xw0CilzktjVGAuskIJ3HXxmTARfAGeWma6F/O306/mGeJurOZHIpsmI24NZ1A3IWxx5PA2oFn\nNj6ACCyp3YG4a55PD2vEZuGDKGnFwzPHJI4lcHNO5giXI4+iPDxgMnJILNi25i4otsV4DfCdsPWC\nrsHUgH4os/DdsyTTxOlo1hPMb2tE7t7k/zvyxR9CCJWRwh9CCJWRwh9CCJWRwh9CCJWRwh9CCJVx\noUYspeKM25HFzhNsjCJpoa37RqWnnc5uG/gA2+k7584AV4G1UYDwTm4jiefBNzwpQSfUhu34vscG\nOGrIHSJxxISzC9HcmNgIanjiYhSo6Qq5giR25RBujZ3A7dP2znUFN9ybuBGKQDDX1kCUg3PPCCII\nXNzISovauXogxoSainTGibLCOzHPJpIArtc5qShvpDc1QLD23Dpf13KNHQ7cBKWD2uKiSWiNnIwj\nzTV+ceSLP4QQKiOFP4QQKiOFP4QQKiOFP4QQKuMi4i5JipQF/nhs+X+Ry8Emkc0JJQOIbC0Js+J4\nBncs353Zig7jLQhhj6c9b/O9FYYIp9ie+/Ni0didloRCEu4kaYF+ByTiuvEt2e7DwFn465n3htEZ\nklaIszjd3uGxY1/ew7Db4bENiL7Nlme5IfLExTDgO+HeYRBt2w7WLvQvkKR1AWODef8WqAF+Zs7v\nCUC3thvKjH7HZHoCUG7+4YqF4AaE57Xh53N/b2JIDPniDyGEykjhDyGEykjhDyGEykjhDyGEykjh\nDyGEyrhMIxaSyI0rYV5K1dvGGoC7ojeODYqIcI1JyO8wjqyakwvCnZeaRrgYBnRiYCMXs8V+g+tj\nS/sOchG5/iG0zZ+cM5LE5iQTYQDPcjEuCnIWLSMve2rmQrNI2+4lqQHnioMcXs7xBH1YtJq4A3Ic\nuasix5NfuxTDwMfS0qN12rqGRXQPLlphwztBDh5qFvR//+UfM47sMjscSrdPBxEVjpNxr+0OV3Be\nnofjide/I1/8IYRQGSn8IYRQGSn8IYRQGSn8IYRQGZcRd4HZCVmgtbRmi3zfl93tWxNh0OBWZz7v\nRCKfEYwGEMjcNnCJBBiTaX5mZMPiRFiKGtiQ578Fe17Sp823xnq+vixaDrO5BtrS7/IoKLeeRPZh\nX4puj8eeP78r9Row4jDHehhxF+7NXReJuzbLHt6r1RkT8CUmEZZ/ilihz4DE97As7n7LcRetwP07\neO0eHyCP/8r07zA9F4gRRN9uKGuetNXMkS/+EEKojhT+EEKojBT+EEKojBT+EEKojBT+EEKojMu4\nesARY50GIKe3drs2/b91vrNimkr3zuN4qabvnZNj05bx8/7eQcf2Zts8uVGc7k8OIOcS+Kde7+Ki\nBuDYzjz3ZcOqRbMQZSBI6sjRAmMuFsR2rwGWDiIMXPYFJp44Vxy9P1tcPcYVtymy4TyHCbmo3HVR\nox6JmzHZegET6eZ8nml+nbuwXA8jNOWRpA6ee2dcgDSPC16XtINGU+8jX/whhFAZKfwhhFAZKfwh\nhFAZKfwhhFAZFxF3Sch1ogrmvZvO8g2JSEZfm2fI+TfiLmWS87Z5h9vKXp7XiaXnCmROLKUt7laY\npVOYQ0kAdREVLJBt2FpuRMlmpRx5XsoUQdBuENNI7HSxIMsGEwOt8y2RDy76ooUHZwVbEnfNOsd5\ncGI0QGvBrRvsgbBBSHZ9DfB6jRBMz2emmA1zDb2Jvpjhnp3IjfNjXp9uQy8IKV/8IYRQHSn8IYRQ\nGSn8IYRQGSn8IYRQGSn8IYRQGZdx9eD/L06FhqgB43BBjd1saaauD87t4Lain3ladARIfMerceXQ\nwRiBYOIhBNOwxdXjHBvUMOWfo8ELxVw0bo1AxsTqHhk05XCOGHyYFB+AXWPMKc04O0zOdzyRs0ky\n0RduPWNkw/kupC3xKNhEyDRBWQQOPOPaGqAZ02wiORZw+7h3VeDAI8eUZExx5v0h99lqGsdQBIhz\nLBnDkSVf/CGEUBkp/CGEUBkp/CGEUBkp/CGEUBnN69evt7VnDyGE8C+afPGHEEJlpPCHEEJlpPCH\nEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJl\npPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCHEEJlpPCH\nEEJl/B8qy1AsAxMp5AAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Import training data\n", "train_dataset = h5py.File(\"../data/train_catvnoncat.h5\")\n", "X_train = np.array(train_dataset[\"train_set_x\"])\n", "Y_train = np.array(train_dataset[\"train_set_y\"])\n", "\n", "# Plot a sample image\n", "plt.imshow(X_train[50])\n", "plt.axis(\"off\");\n", "\n", "# Import test data\n", "test_dataset = h5py.File(\"../data/test_catvnoncat.h5\")\n", "X_test = np.array(test_dataset[\"test_set_x\"])\n", "Y_test = np.array(test_dataset[\"test_set_y\"])\n", "\n", "# Transform data\n", "X_train = X_train.reshape(209, -1).T\n", "X_train = X_train / 255\n", "Y_train = Y_train.reshape(-1, 209)\n", "\n", "X_test = X_test.reshape(50, -1).T\n", "X_test = X_test / 255\n", "Y_test = Y_test.reshape(-1, 50)\n", "\n", "# print the new shape of both training and test datasets\n", "print(\"Training data dimensions:\")\n", "print(\"X's dimension: {}, Y's dimension: {}\".format(X_train.shape, Y_train.shape))\n", "print(\"Test data dimensions:\")\n", "print(\"X's dimension: {}, Y's dimension: {}\".format(X_test.shape, Y_test.shape))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's now write the helper functions to implement all optimization algorithms covered. Mainly, Mini-batch gradient descent (MB), MB with Momentum, MB with RMSProp, MB with Adam." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def random_mini_batches(X, Y, mini_batch_size=64, seed=0):\n", " \"\"\"\n", " Creates a list of random minibatches from (X, Y)\n", "\n", " Arguments:\n", " X -- input data, shape: input size, number of examples\n", " Y -- \"label\" vector, shape: 1, number of examples\n", " mini_batch_size -- size of the mini-batches, integer\n", "\n", " Returns:\n", " mini_batches -- list of synchronous (mini_batch_X, mini_batch_Y)\n", " \"\"\"\n", " np.random.seed(seed)\n", " m = X.shape[1]\n", " mini_batches = []\n", "\n", " # shuffle training set\n", " permutation = np.random.permutation(m)\n", " shuffle_X = X[:, permutation]\n", " shuffle_Y = Y[:, permutation]\n", "\n", " num_complete_minibatches = m // mini_batch_size\n", "\n", " for k in range(num_complete_minibatches):\n", " mini_batch_X = shuffle_X[:, k*mini_batch_size:(k + 1)*mini_batch_size]\n", " mini_batch_Y = shuffle_Y[:, k*mini_batch_size:(k + 1)*mini_batch_size]\n", " mini_batch = (mini_batch_X, mini_batch_Y)\n", " mini_batches.append(mini_batch)\n", "\n", " # check if there are some examples left if m % batch_size != 0\n", " if m % mini_batch_size != 0:\n", " mini_batch_X = shuffle_X[:, num_complete_minibatches*mini_batch_size:]\n", " mini_batch_Y = shuffle_Y[:, num_complete_minibatches*mini_batch_size:]\n", " mini_batch = (mini_batch_X, mini_batch_Y)\n", " mini_batches.append(mini_batch)\n", "\n", " return mini_batches\n", "\n", "\n", "def initialize_momentum(parameters):\n", " \"\"\"\n", " Initializes the velocity as a python dictionary with:\n", " - keys: \"dW1\", \"db1\", ..., \"dWL\", \"dbL\"\n", " - values: numpy arrays of zeros of the same shape as the\n", " corresponding gradients/parameters.\n", "\n", " Arguments:\n", " parameters -- python dictionary containing parameters.\n", "\n", " Returns:\n", " v -- python dictionary containing the current velocity.\n", " \"\"\"\n", " L = len(parameters) // 2\n", " v = {}\n", "\n", " for l in range(1, L + 1):\n", " v[\"dW\" + str(l)] = np.zeros_like(parameters[\"W\" + str(l)])\n", " v[\"db\" + str(l)] = np.zeros_like(parameters[\"b\" + str(l)])\n", "\n", " return v\n", "\n", "\n", "def update_parameters_with_momentum(parameters, grads, v, beta, learning_rate):\n", " \"\"\"\n", " Update parameters using Momentum\n", "\n", " Arguments:\n", " parameters -- python dictionary containing your parameters:\n", " grads -- python dictionary containing your gradients for each parameters:\n", " v -- python dictionary containing the current velocity:\n", " beta -- the momentum hyperparameter --> scalar\n", " learning_rate -- the learning rate --> scalar\n", "\n", " Returns:\n", " parameters -- python dictionary containing your updated parameters\n", " v -- python dictionary containing your updated velocities\n", " \"\"\"\n", " L = len(parameters) // 2\n", "\n", " for l in range(1, L + 1):\n", " # update momentum velocity\n", " v[\"dW\" + str(l)] =\\\n", " beta * v[\"dW\" + str(l)] + (1 - beta) * grads[\"dW\" + str(l)]\n", " v[\"db\" + str(l)] =\\\n", " beta * v[\"db\" + str(l)] + (1 - beta) * grads[\"db\" + str(l)]\n", " # update parameters\n", " parameters[\"W\" + str(l)] =\\\n", " parameters[\"W\" + str(l)] - learning_rate * v[\"dW\" + str(l)]\n", " parameters[\"b\" + str(l)] =\\\n", " parameters[\"b\" + str(l)] - learning_rate * v[\"db\" + str(l)]\n", "\n", " return parameters, v\n", "\n", "\n", "def initialize_rmsprop(parameters):\n", " \"\"\"\n", " Initializes the velocity as a python dictionary with:\n", " - keys: \"dW1\", \"db1\", ..., \"dWL\", \"dbL\"\n", " - values: numpy arrays of zeros of the same shape as the\n", " corresponding gradients/parameters.\n", "\n", " Arguments:\n", " parameters -- python dictionary containing parameters.\n", "\n", " Returns:\n", " s -- python dictionary containing the current velocity.\n", " \"\"\"\n", " L = len(parameters) // 2\n", " s = {}\n", "\n", " for l in range(1, L + 1):\n", " s[\"dW\" + str(l)] = np.zeros_like(parameters[\"W\" + str(l)])\n", " s[\"db\" + str(l)] = np.zeros_like(parameters[\"b\" + str(l)])\n", "\n", " return s\n", "\n", "\n", "def update_parameters_with_rmsprop(parameters, grads, s, beta, learning_rate,\n", " epsilon=1e-8):\n", " \"\"\"\n", " Update parameters using Momentum\n", "\n", " Arguments:\n", " parameters -- python dictionary containing parameters:\n", " grads -- python dictionary containing gradients for each parameters:\n", " s -- python dictionary containing the current velocity:\n", " beta -- the momentum hyperparameter --> scalar\n", " learning_rate -- the learning rate --> scalar\n", " epsilon -- hyperparameter preventing division by zero in Adam updates\n", "\n", " Returns:\n", " parameters -- python dictionary containing your updated parameters\n", " v -- python dictionary containing updated velocities\n", " \"\"\"\n", " L = len(parameters) // 2\n", "\n", " for l in range(1, L + 1):\n", " # update momentum velocity\n", " s[\"dW\" + str(l)] =\\\n", " beta * s[\"dW\" + str(l)] +\\\n", " (1 - beta) * np.square(grads[\"dW\" + str(l)])\n", " s[\"db\" + str(l)] =\\\n", " beta * s[\"db\" + str(l)] +\\\n", " (1 - beta) * np.square(grads[\"db\" + str(l)])\n", " # update parameters\n", " parameters[\"W\" + str(l)] -= (learning_rate * grads[\"dW\" + str(l)])\\\n", " / (np.sqrt(s[\"dW\" + str(l)] + epsilon))\n", " parameters[\"b\" + str(l)] -= (learning_rate * grads[\"db\" + str(l)])\\\n", " / (np.sqrt(s[\"db\" + str(l)] + epsilon))\n", "\n", " return parameters, s\n", "\n", "\n", "def initialize_adam(parameters):\n", " \"\"\"\n", " Initializes v and s as two python dictionaries with:\n", " - keys: \"dW1\", \"db1\", ..., \"dWL\", \"dbL\"\n", " - values: numpy arrays of zeros of the same shape as the\n", " corresponding gradients/parameters.\n", "\n", " Arguments:\n", " parameters -- python dictionary containing your parameters.\n", "\n", " v -- python dictionary that will contain the exponentially weighted\n", " average of the gradient.\n", " s -- python dictionary that will contain the exponentially weighted\n", " average of the squared gradient.\n", " \"\"\"\n", " L = len(parameters) // 2\n", " v = {}\n", " s = {}\n", "\n", " for l in range(1, L + 1):\n", " v[\"dW\" + str(l)] = np.zeros_like(parameters[\"W\" + str(l)])\n", " v[\"db\" + str(l)] = np.zeros_like(parameters[\"b\" + str(l)])\n", " s[\"dW\" + str(l)] = np.zeros_like(parameters[\"W\" + str(l)])\n", " s[\"db\" + str(l)] = np.zeros_like(parameters[\"b\" + str(l)])\n", "\n", " return v, s\n", "\n", "\n", "def update_parameters_with_adam(parameters, grads, v, s, t, learning_rate,\n", " beta1=0.9, beta2=0.999, epsilon=1e-8):\n", " \"\"\"\n", " Update parameters using Adam\n", "\n", " Arguments:\n", " parameters -- python dictionary containing parameters:\n", " grads -- python dictionary containing gradients for each parameters:\n", " v -- Adam variable, moving average of the first gradient\n", " s -- Adam variable, moving average of the squared gradient\n", " learning_rate -- the learning rate, scalar.\n", " beta1 -- Exponential decay hyperparameter for the first moment estimates\n", " beta2 -- Exponential decay hyperparameter for the second moment estimates\n", " epsilon -- hyperparameter preventing division by zero in Adam updates\n", "\n", " Returns:\n", " parameters -- python dictionary containing updated parameters\n", " v -- Adam variable, moving average of the first gradient\n", " s -- Adam variable, moving average of the squared gradient\n", " \"\"\"\n", " L = len(parameters) // 2\n", " v_corrected = {}\n", " s_corrected = {}\n", "\n", " for l in range(1, L + 1):\n", " # update the moving avergae of both first gradient and squared gradient\n", " v[\"dW\" + str(l)] = beta1 * v[\"dW\" + str(l)] +\\\n", " (1 - beta1) * grads[\"dW\" + str(l)]\n", " v[\"db\" + str(l)] = beta1 * v[\"db\" + str(l)] +\\\n", " (1 - beta1) * grads[\"db\" + str(l)]\n", " s[\"dW\" + str(l)] = beta2 * s[\"dW\" + str(l)] +\\\n", " (1 - beta2) * np.square(grads[\"dW\" + str(l)])\n", " s[\"db\" + str(l)] = beta2 * s[\"db\" + str(l)] + \\\n", " (1 - beta2) * np.square(grads[\"db\" + str(l)])\n", "\n", " # compute the corrected-bias estimate of the moving averages\n", " v_corrected[\"dW\" + str(l)] = v[\"dW\" + str(l)] / (1 - beta1**t)\n", " v_corrected[\"db\" + str(l)] = v[\"db\" + str(l)] / (1 - beta1**t)\n", " s_corrected[\"dW\" + str(l)] = s[\"dW\" + str(l)] / (1 - beta2**t)\n", " s_corrected[\"db\" + str(l)] = s[\"db\" + str(l)] / (1 - beta2**t)\n", "\n", " # update parameters\n", " parameters[\"W\" + str(l)] -= (learning_rate * v_corrected[\"dW\" + str(l)])\\\n", " / (np.sqrt(s_corrected[\"dW\" + str(l)] + epsilon))\n", " parameters[\"b\" + str(l)] -= (learning_rate * v_corrected[\"db\" + str(l)])\\\n", " / (np.sqrt(s_corrected[\"db\" + str(l)] + epsilon))\n", "\n", " return parameters, v, s\n", "\n", "\n", "def model(X, Y, layers_dims, optimizer=\"adam\", learning_rate=0.01,\n", " mini_batch_size=64, beta=0.9, beta1=0.9, beta2=0.999, epsilon=1e-8,\n", " num_epochs=3000, print_cost=True, activation_fn=\"relu\"):\n", " \"\"\"\n", " Implements multi-neural network model which can be run in different\n", " optimizer modes.\n", "\n", " Arguments:\n", " X -- input data, shape: number of features, number of examples\n", " Y -- label vector, shape: 1, number of examples\n", " layers_dims -- python list, containing the size of each layer\n", " optimizer -- \"momentum\", \"rmsprop\", or \"adam\".\n", " learning_rate -- the learning rate --> scalar.\n", " mini_batch_size -- the size of a mini batch\n", " beta -- Momentum/RMSProp hyperparameter\n", " beta1 -- Exponential decay hyperparameter for the past gradients\n", " beta2 -- Exponential decay hyperparameter for the past squared gradients\n", " epsilon -- hyperparameter preventing division by zero\n", " num_epochs -- number of epochs\n", " print_cost -- True to print the cost every 1000 epochs\n", " activation_fn -- function to be used on hidden layers: \"relu\", or \"tanh\"\n", "\n", " Returns:\n", " parameters -- python dictionary containing updated parameters\n", " \"\"\"\n", " # set random seed to get consistent output\n", " seed = 1\n", " np.random.seed(seed)\n", "\n", " # initialize parameters\n", " parameters = initialize_parameters(layers_dims)\n", "\n", " # initialize moving averages based on optimizer modes\n", " assert(optimizer == \"mb\" or optimizer == \"momentum\" or\n", " optimizer == \"rmsprop\" or optimizer == \"adam\")\n", "\n", " if optimizer == \"momentum\":\n", " v = initialize_momentum(parameters)\n", "\n", " elif optimizer == \"rmsprop\":\n", " s = initialize_rmsprop(parameters)\n", "\n", " elif optimizer == \"adam\":\n", " v, s = initialize_adam(parameters)\n", " t = 0\n", "\n", " # initialize costs list\n", " costs = []\n", "\n", " # iterate over number of epochs\n", " for epoch in range(num_epochs):\n", " # split the training data into mini batches\n", " seed += 1\n", " mini_batches = random_mini_batches(X, Y, mini_batch_size, seed=seed)\n", "\n", " # iterate over mini batches\n", " for mini_batch in mini_batches:\n", " mini_batch_X, mini_batch_Y = mini_batch\n", "\n", " # compute fwd prop\n", " AL, caches = L_model_forward(\n", " mini_batch_X, parameters, activation_fn)\n", "\n", " # compute cost\n", " cost = compute_cost(AL, mini_batch_Y)\n", "\n", " # compute gradients\n", " grads = L_model_backward(AL, mini_batch_Y, caches, activation_fn)\n", "\n", " # update parameters\n", " if optimizer == \"mb\":\n", " parameters = update_parameters(\n", " parameters, grads, learning_rate)\n", "\n", " elif optimizer == \"momentum\":\n", " parameters, v = update_parameters_with_momentum(\n", " parameters, grads, v, beta, learning_rate)\n", "\n", " elif optimizer == \"rmsprop\":\n", " parameters, s = update_parameters_with_rmsprop(\n", " parameters, grads, s, beta, learning_rate, epsilon)\n", "\n", " elif optimizer == \"adam\":\n", " t += 1\n", " parameters, v, s = update_parameters_with_adam(\n", " parameters, grads, v, s, t, learning_rate, beta1, beta2,\n", " epsilon)\n", "\n", " # compute epoch cost\n", " AL, caches = L_model_forward(\n", " X_train, parameters, activation_fn)\n", " cost = compute_cost(AL, Y_train)\n", "\n", " if epoch % 100 == 0:\n", " costs.append(cost)\n", "\n", " # plot the cost\n", " plt.plot(costs)\n", " plt.ylabel('Cost')\n", " plt.xlabel('Epochs (per hundreds)')\n", " plt.title(\"Learning rate = \" + str(learning_rate))\n", " plt.show()\n", "\n", " return parameters" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, we're ready to train the neural network using different learning algorithms." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Setup layers dimensions\n", "layers_dims = [X_train.shape[0], 200, 200, 1]\n", "\n", "# train NN with mini-batch gradient descent only\n", "parameters = model(X_train, Y_train, layers_dims, optimizer=\"mb\",\n", " learning_rate=0.03, mini_batch_size=64,\n", " beta1=0.9, num_epochs=3000, activation_fn=\"tanh\")\n", "\n", "# print the test accuracy\n", "print(\"The training accuracy rate: {}\".format(\n", " accuracy(X_train, parameters, Y_train, \"tanh\")[-7:]))\n", "print(\"The test accuracy rate: {}\".format(\n", " accuracy(X_test, parameters, Y_test, \"tanh\")[-7:]))" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyYAAAGkCAYAAADE0DHIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4VOXd//HPrFkmG2GTPRBBFllcQCKg2EpxoVVURGuV\nTfZqW9tfK8UHbetSFLVUWZRFxPoIj4qiqMEFjVgWBUUCCRCUfTVAQjJZJpM5vz8CkwyQDTIzmcn7\ndV29ytznzMk34Vwyn9z3ub+mnJwcQwAAAAAQROZgFwAAAAAABBMAAAAAQUcwAQAAABB0BBMAAAAA\nQUcwAQAAABB0BBMAAAAAQUcwAYBaePnll9WnTx+tWLEi2KXUiVtuuUU/+9nPgl1GncjOztb7778f\n7DKqlJqaqnvvvVfXXHONbrrpJj399NPKy8ur8ftLS0u1dOlSDR8+XAMGDNAtt9yi2bNnq7i4+Kxz\nXS6XXnvtNd11110aMGCAhgwZomeeeUY5OTl1+S0BQJ0hmABAA3bXXXdpxIgRwS7jgh0/flzDhg1T\nWlpasEup1OLFizVt2jS53W7deeed6tq1q9566y2NHz9eRUVFNbrGjBkz9OyzzyomJkbDhw9Xy5Yt\ntWjRIv3pT3+Sx+PxnmcYhqZOnaoXXnhBkZGRGjZsmPfrjRgxgnACoF6yBrsAAEDw3H333cEuoU4U\nFRXJ6XQGu4xKHT58WHPmzFG3bt308ssvy2azSZJef/11zZw5U6+//rrGjBlT5TU2b96st99+WwMH\nDtT06dNlMpkklYWV//u//9PKlSt14403SpK++OILpaWl6Re/+IX+8Y9/eM9dtmyZ/vnPf2rx4sV6\n8MEH/fgdA0DtMWMCAICfLVu2TKWlpRo1apQ3lEhlM1ZNmzbVe++9V+013nzzTUnS2LFjvUFDkiZM\nmCCbzably5d7x3bv3q3ExESNHDnS59xf/OIXkqT09PQL/p4AoK4xYwIAfmQYht59912988472rVr\nl2w2m3r06KH7779fl156qc+5brdbb7/9tlauXKldu3apuLhYiYmJuuqqqzR+/Hg1a9bMe26fPn10\nww03KDk5WYsXL5bH49HIkSM1aNAg3XrrrRo1apR69OihBQsWKCsrS5GRkerXr59++9vfqkmTJt7r\n3HLLLcrLy9OqVaskSStWrNDf//53/fvf/9aPP/6oZcuW6dChQ2rSpIluuukmjR492ueDtVT2ofvN\nN9/U/v371bhxY912221q3Lix/va3v2nOnDm64oorKv35vPzyy5o/f75eeOEFvfTSS9q+fbuaN2+u\nRYsWKS4uTj/88IMWL16sb7/9VsePH5fdblf79u01bNgw7+zA6Zol6csvv1SfPn10//33a9y4cZLK\nlnktWLBAX375pY4fP64mTZro2muv1f3336+4uLgq//4OHjyoW2+9tbq/Zk2bNk1Dhgyp9Ph3330n\nSWf9LCwWi3r16qVPPvlEhw4dUosWLaq8Rnx8vDp27OgzHhMTo0suuUSbN2+W2+2W1WrVqFGjNGrU\nqLOusWfPHklSYmJitd8TAAQawQQA/Ojxxx/X+++/r/bt22vo0KEqLi7WZ599pnHjxumpp57Stdde\n6z33kUce0apVq9SjRw/deuutKikp0YYNG/T+++/r+++/15IlS2S1lv9n++uvv9bq1at18803Kycn\nxyforF27Vq+++qquvvpq3Xnnndq4caM+/PBD/fjjj1q8eHG1dc+ePVu7d+/Wz3/+c/Xv31+ffvqp\nFixYoMLCQv3+97/3nvfcc89pyZIlat26tW699Vbl5ORozpw5uuiii2r1c3r00UfVtm1bDR8+XDk5\nOYqLi9PWrVu9swHXXXedEhMTdejQIX3++ed69NFH5fF4dPPNN6tTp0666667tGTJErVr106DBg3y\nBoAjR45o7NixOnLkiPr166cOHTrohx9+0JIlS7Ru3TrNnz+/ynASGxur+++/v9r6O3XqVOXxAwcO\nKDExUQ6H46xjp8PI3r17Kw0mLpdLR48eVbdu3c55vEWLFtqyZYsOHjyotm3bnnU8Pz9f3333nZ59\n9lnZbDbdc8891X1LABBwBBMA8JNVq1bp/fff1/XXX6+///3v3lAxevRojRo1Sv/4xz/Uu3dvRUdH\nKz09XatWrdKgQYP0xBNPeK/h8Xg0fvx4ff/998rIyFCPHj28x44fP67p06fruuuu844dPHhQkrRt\n2zY9/vjj3qU7paWlGjt2rLZs2aKtW7dW+gH3tH379um1115Tu3btJEn33nuvbr/9dr333nv67W9/\nK6vVqoyMDC1dulSXXnqpXnzxRUVHR0uSBg8erIceeqhWP6umTZtqzpw5slgs3rGXXnpJbrdbixYt\nUnJysnd83bp1evDBB5WamnrOYHJ6pkSSpk+friNHjujpp5/2CYHLly/XE088oRdffFF//etfK60r\nNjbW53rnKzc3Vy1btjznsZiYGEll4aEyJ0+e9NZT22usXbtWv/vd7ySVzdA8/vjjPvcRANQXPGMC\nAH5yes3/Qw895DPT0bx5cw0fPlwnT57Ul19+KUlq1qyZpk2bpgkTJvhcw2w267LLLpNU9uG2ooiI\nCA0YMOCcX7tly5beUCKVfSDt27evJGn//v3V1v6zn/3MG0okqXHjxurcubPy8/O9Ozp9+OGHMgxD\nEydO9IYSSerfv7+uuuqqar9GRQMHDvQJJVLZg/l/+9vffEKJVL4c6syfx5mys7P13//+V1dddZVP\nKJHKlrC1a9dOH330kdxud61qPR9ut/usJXCnnR53uVxVvl+S7HZ7ra9ht9t11113aciQIYqMjNQj\njzwSNttdAwgvzJgAgJ9kZmbKZrNp2bJlZx3bu3evJGnHjh264YYb1Lx5cw0ZMkRut1vbt2/X3r17\ntX//fu3YsUPffPONJPlsByuVhZmKgaeicy3nOf1b9ZKSkmprr8n7MzIyJOmcsy89evTQ+vXrq/06\np7Vq1eqssZSUFEnSsWPHtHPnTu3fv1979uzR5s2bJZ398zjTtm3bZBiG8vLy9PLLL5913Gw2q7i4\nWHv27Dkr/JyWl5enN954o9r6Bw4cWOVyroiIiEoD0OmfZ2RkZJXvr3huZdeIioo669gVV1zhDXNj\nxozRyJEj9dRTT6l3795q3rx5pV8TAAKNYAIAfpKXl6fS0lLNnz+/0nMq/tb/3Xff1fz583X06FFJ\nZUGga9euSk5O1nfffSfDMHzeW9UH2XP9Zr3i7kzVqew385K8deTm5ioqKspntuS0pk2b1vhrSef+\nXo4cOaJnn31WaWlpMgxDZrNZrVu3Vu/evZWRkXHWz+NMp5c1bd26VVu3bq30vNPLpM4lLy+vyr+/\n01q2bFllMImLi6t0qdbp8dPB71xiYmJkNpsv6BpSWQC866679NJLL2nt2rU1erAfAAKFYAIAfhId\nHa2oqKgaLZv57LPP9OSTTyo5OVl//OMf1blzZ++D0LNmzfLu6lSfOBwOHThwwLsTVEUX2lPEMAz9\n4Q9/0I8//qj77rtP1113nTp06KDIyEi5XC69/fbb1V7j9OzB6NGjz1oiV1MtW7bU119/fV7vraht\n27bauHGjioqKzgphp58Lat++faXvt9lsatGihQ4cOHDO4wcPHlRkZKR3BiQ9PV2HDx/WoEGDzjr3\n9LMu1S2FA4BA4xkTAPCTjh076qefflJ2dvZZx77++mvNnj3b+5v81NRUSfI+zF5xd6Zdu3ZJUrUz\nBIHWuXNneTwe75KuirZs2XJB187KytLOnTs1aNAgTZ48WV27dvV+oN+9e7ck35/HuWaDTm+re676\nJOmVV17RK6+8ooKCgguqtSZ69eolwzDOCpilpaXatGmTWrRo4bON87n07NlTx48f937/p+Xn52v7\n9u3q1q2bNyA+//zzmjp1qnfJYEU7duyQdO7lcwAQTAQTAPCTIUOGyDAMTZ8+3eeh5JycHP3zn//U\nokWLvA8tn36G4PDhwz7XWLVqlb766itJCshD2rVxum/H7NmzVVhY6B3fsGGDvvjiiwu6dsWfR8UA\nkp+frxkzZkgq+1B/2ukP5BWfwWjZsqUuv/xyrVu3Tp988onP9T///HPNmTNHn3766TmXotW1wYMH\ny2Kx6OWXX1ZxcbF3fMmSJcrOztZtt91W7TVuvvlmSWUzaBWfr5k7d67cbreGDh3qHTs9U/Liiy/6\nnJuRkaG33npLiYmJ6tev3wV/XwBQl1jKBQDn4dVXX610idaQIUM0ZMgQ3Xzzzfrqq6+0atUq3X33\n3erbt68Mw9CqVat0/PhxjRgxwvtcwo033qiPP/5Yf/nLXzRo0CDFxsZq27Zt2rBhgxo1aqTjx4/X\nu6U3PXr00G233aZly5bpN7/5jVJSUnTixAmtWrVKsbGxysnJkdl8fr//atu2rS699FJ9//33Gjt2\nrHr16uXdxczpdMrhcPj8PBo1aqSIiAht3LhRzz33nHr37q0BAwbor3/9q8aNG6epU6fq/fff18UX\nX6wDBw7oyy+/lMPh0NSpU+vqx1Gldu3a6b777tMrr7yi3/zmN7rmmmu0d+9epaWlqXPnzrrzzjt9\nzl+xYoUOHjyoIUOGeJde9e7dWzfccINSU1M1atQo77M2GzZsUL9+/fTzn//c+/477rhDX3zxhb74\n4gvdd9996t27t44eParPP/9cVqtV//jHP875oDwABBPBBADOw549e7xdtM90+eWXSypbXvTkk0/q\nrbfe0ooVK/Tee+8pIiJCSUlJeuihh3y28+3Xr5+efPJJLV68WCtXrlRERIRatmyp3/3ud7rmmmt0\n2223ac2aNbrjjjsC8v3V1P/7f/9PrVu39na3b9q0qR588EFlZ2frtddeq/IB/aqYTCY988wzmj17\nttavX6+lS5eqadOmuuKKKzR69Gi9/vrrWrFihTIzM9WlSxdZrVY9/PDDmjt3rt5++20VFhZqwIAB\natu2rRYvXqwFCxZozZo12rhxoxITE3X99ddrzJgxSkpKqtsfSBUmTpyopk2b6q233tLSpUuVmJio\nYcOGaezYsWeFhBUrVujbb7/VFVdc4dP/ZNq0aUpKStKKFSu0ZMkSNW3aVKNHj9bIkSN9tlu22Wx6\n4YUXtGjRIq1cuVJLly5VTEyMBg4cqDFjxlS6CxkABJMpJyenfi1aBgCEhOzsbNlsNsXHx5917LHH\nHtOHH36o1NRUJSYmBqE6AECo4RkTAMB5SU1N1aBBg85a0rZ//36lpaWpffv2hBIAQI0xYwIAOC9H\njhzRr3/9axUVFemaa65R69atdezYMX3++ecqKSnRv/71L1155ZXBLhMAECIIJgCA87Zv3z4tWrRI\nGzZs0LFjxxQTE6NevXpp5MiR6ty5c7DLAwCEEIIJAAAAgKDjGRMAAAAAQUcwAQAAABB0BBMAAAAA\nQUcwqQNZWVnBLgFhjPsL/sT9BX/i/oI/cX+FH4IJAAAAgKAjmAAAAAAIOoIJAAAAgKAjmAAAAAAI\nOoIJAAAAgKAjmAAAAAAIOmsgv5jH49H06dOVlZUlu92uqVOnqk2bNpKk7OxsPfLII95zd+zYocmT\nJ+v2228PZIkAAAAAgiCgwSQtLU0ul0sLFy5Uenq6Zs6cqRkzZkiSmjRporlz50qSNm/erDlz5ujW\nW28NZHkAAAAAgiSgS7k2bdqklJQUSVL37t2VmZl51jmGYWjGjBl6+OGHZbFYAlkeAAAAgCAJ6IyJ\n0+lUTEyM97XZbJbb7ZbVWl7G6tWr1aFDB7Vr167G160PnT/rQw0IX9xf8CfuL/gT9xf8ifsrtHTs\n2LHK4wENJg6HQ06n0/vaMAyfUCJJH330ke66665aXbe6b9Lftu3IUofki2W3mIJaB8JTVlZW0O9x\nhC/uL/gT9xf8ifsr/AR0KVfPnj21Zs0aSVJ6erqSk5PPOiczM1M9evQIZFkX7Ntcs5LfOKT7Vh3T\n61lOHS0sDXZJAAAAQEgJ6IzJwIEDtX79eo0ZM0aGYWjatGlKTU1VYWGhhg4dqhMnTsjhcMhkCq2Z\nh6+OW5RXYui9PUV6b0+RTJIub2LT4DaRGtwmUj0SbSH3PQEAAACBZMrJyTGCXUSo67Fkn/YWVj75\n1CLarF+0Lgsp17aIkMNG+xjUHFPV8CfuL/gT9xf8ifsr/AR0xiQcHS8qVb676tmQQwUevbqjQK/u\nKFCERbrmogj94tRsStsY/goAAAAAPhVfoMRIiz7qUyhno3Zaub9IK/cVadOxkkrPLy6VPjlQrE8O\nFOv/rctV1wSrBreJ1C/aRKp3U7usZpZ8AQAAoOEhmNQBs0m6vKldlze1a8plcTpcUKqPT4WULw4W\ny+mufLVcRo5bGTn5ej49X40iTBrUqiykXN8qUgkRLPkCAABAw0Aw8YOLoi26r5ND93VyqMht6L9H\nirVyX5FS9xVpb37lO3adKDb0fz8W6v9+LJTFJF3VzK4bTi356hRv5QF6AAAAhC2CiZ9FWk36eatI\n/bxVpKZfZWh7rlsr95XNpqw/6lJpJZMppYa05ohLa464NG3DSbWLsWhwm0jd0CZS/S6KUAQ9UwAA\nABBGCCYBZDKZ1DnBps4JNv2ue6xOFHv02YGykPLJ/iLluCpf8rUnv1QvZzr1cqZTDqtJA1tGlD2b\n0jpSF0VbAvhdAAAAAHWPYBJEjSLMuqNDtO7oEC23x9A3P7m8symZOe5K3+d0G/pgb5E+2FskSerV\n2OadTenZ2CYzS74AAAAQYggm9YTVbFJK8wilNI/QY1fGa0+e2/sA/erDxSquopn8pmMl2nSsRNM3\n5al5lFmDTvVMGdgyQrH0TAEAAEAIIJjUU+1irRrbJUZju8TIWeJR2qFi72zK4UJPpe87UujRf7IK\n9J+sAtnNUv9TPVNuaBOppFj+ugEAAFA/8Uk1BDhsZt3UNko3tY2SYRj6/liJdzZlY3blPVNcHmnV\nwWKtOlish9fn6pJ4q7ex41XN7LLRMwUAAAD1BMEkxJhMJvVqYlevJnb9uVecjhSU6pMDRfp4X5FW\nHShWfhU9U7bnurU9N18vbMlXvN2k61uVhZTrW0UoMZIH6AEAABA8BJMQ1zzaot90dOg3HR1ylRpa\nU6Fnyq68yh9MyXUZentXod7eVSizSerT1K7Bp2ZTuiTQMwUAAACBRTAJI3aLSQNbRmpgy0g92cfQ\nzpNupZ56LmXdEZcqm0zxGNK6oy6tO+rS3zaeVGuHxdvYsf9FEYqyElIAAADgXwSTMGUymdQx3qaO\n8TY9cGmscoo9+vxg2UzKJ/uLdby48gfo9ztLNX+bU/O3ORVlMenalhG64VTPlJYOlnwBAACg7hFM\nGoiECLOGto/W0PbRKvUY2pjt8i752nqi8p4phaWGUk+dJ0ndE8t7plzehJ4pAAAAqBsEkwbIYjap\nT7MI9WkWof+5Il778t36ZH+xVu4rVNqhYhVV0TMl/XiJ0o+XaMb3eWoaadb1rctCynUtIxRnp2cK\nAAAAzg/BBGoTY9XozlaN7uxQgduj1YfKO9AfKKg8pfxU5NEbOwv0xs4C2czS1c1P9UxpHankeG4t\nAAAA1ByfHuEj2mr27s5lGIa2nHDr41Mh5ZufXKpsM+ISj5R2qFhph4o19etcXRxn1eBTz6WkNLfL\nbmHJFwAAACpHMEGlTCaTuifa1D3Rpj/2jFV2Uak+2V+sj/cV6bMDRTpZUnnPlJ0n3dq5NV+ztuYr\nzmbSz071TBnUOkJN6JkCAACAMxBMUGNNIi26++Jo3X1xtEo8htYeKVvy9fH+ImXlVv4A/ckSQ+/u\nLtS7uwtlUtkD9LH2ms2g1OSsGp1Tw4f0a3atGl2qzmovKIiQY1d2zb5oDa9Zm/Okmn/Ptb1ubS5c\nq3rPuLypimPnPt9UxbEK76vi+qrya5tqda2aHatQcy3qys+1qW3+ScXazYq1mRRrMynGdvrPZsXY\nTIqzl/2/zczMJwDAfwgmOC82s0nXtIjQNS0i9ESfeP2Q69bK/WVLvtYcKVZJJbsRG5I2Hy8JaK2h\nzyLlFAe7CIQtm3Qwr0ZnRlrkDSve/7ebFWczece8wcZe/rr8PeWhx0LIAQCcgWCCOpEcb9Wk+BhN\n6hajky6PPj9Y7J1NyS6qvGcKgNBRVCoVlXr0U5EkVbF9Xw04rCbfgHMq5MTYTIqrGHBOhx5vAPIN\nPw6riW3LASBMEExQ5+LsZt2SFKVbkqLkMQx9l13i7UDPbAkASXK6DTndho4UXtgvLkzSWbMxsXaz\nYqwVg47vDM/pc31meuwmRVlMNV72CQCoewQT+JXZZNIVTe26oqldUy+P06GCUv1w0i2j8ufmJanS\n3b/OOq9GJ9bsajU5q2Zfr4bXqtmltP/AAbVq2arOvq5U8++jNtcsO7fmZ/utBsP3PUYVx879PqPS\n86u6VnVfxzg1eP51VfW1q6i5whvP9bUPHPlJEfGNlV/iUV6JoTyXR/klhvJKTv+/oZOn/uypzV9E\ngBhSWd0lhqQLCzlmk7xLz3yWoNnLQ0/nBJuGdYiSw0bfJgCoawQTBFSLaItaRLMrV21kFXjUsXVk\nsMtAmMqyHVLHjnHVnmcYhgrcZQHAG2JOBZmKY/klHp30CTgVzy8fq488hpTrMpTrqnqZ2qf7i/Sf\nnzcOUFUA0HAQTAAA1TKZTHLYTHLYJOnCfrngMQzllxhnzMx4dNJVMeCUjZ0ONHmucwecAnfgQ86K\nvUX6Ltuly5rYA/61ASCcEUwAAAFlNpkUZzcpzi5daMhxe84OOKeDzUnXmWOVz/TklXhUXIvn+Wdv\nzde8axMvqHYAgC+CCQAgZFnNJiVEmJQQceHPfLhKzwgvp8OOy6PMHLee+b58W+V3dhXqsStL1crB\n0lQAqCsEEwAAJNktJiVaLEo8xyNdHsPQ8t2F2nGqmazbkOZl5uuxK+MDXCUAhC+2FQEAoBpmk0kT\nu8b4jL2y3an8yrrJAgBqjWACAEANDL84So0iyvuc5LoMvbGzIIgVAUB4IZgAAFAD0VazxlziO2sy\nZ2u+PLVpygMAqBTBBACAGrq/i0MVeyv+mFeq1H1FwSsIAMIIwQQAgBq6KNqi29pH+YzN3pofpGoA\nILwQTAAAqIVJ3XyXc3112KXvj7mCVA0AhA+CCQAAtdCzsV39L/Lt+s6sCQBcOIIJAAC1NPmMWZNl\nuwp1qKAWreMBAGchmAAAUEuD20QqOa6863uJR5qfyawJAFyIgAYTj8ejp556SqNHj9aECRO0b98+\nn+MZGRkaO3asxo4dq4cffljFxcWBLA8AgBoxm0yacEbDxYXbnSpw03ARAM5XQINJWlqaXC6XFi5c\nqMmTJ2vmzJneY4Zh6IknntC0adM0b9489e3bV4cPHw5keQAA1NivL45Wgr284eKJYkNLdxYGsSIA\nCG0BDSabNm1SSkqKJKl79+7KzMz0Htu7d6/i4+P1xhtvaPz48Tp58qTatWsXyPIAAKgxh82skZc4\nfMZmZ9BwEQDOV0CDidPpVExM+dS32WyW2+2WJOXk5Cg9PV3Dhg3TrFmz9M033+ibb74JZHkAANTK\n2C4xspZPmigr161P97MMGQDOhzWQX8zhcMjpdHpfG4Yhq7WshPj4eLVu3Vrt27eXJKWkpCgzM1O9\ne/eu9rpZWVn+KbgW6kMNCF/cX/An7q8L8/Mmdq38qfyf0xkbjqp9EeHkNO4v+BP3V2jp2LFjlccD\nGkx69uyp1atXa9CgQUpPT1dycrL3WKtWrVRQUKB9+/apTZs22rRpk371q1/V6LrVfZP+lpWVFfQa\nEL64v+BP3F8X7uFGLq18/yfv669zLCpunKRLE21BrKp+4P6CP3F/hZ+ABpOBAwdq/fr1GjNmjAzD\n0LRp05SamqrCwkINHTpUjzzyiP7nf/5HhmGoR48e6t+/fyDLAwCg1i5rYldKc7vWHinv/j4nI1+z\n+jcKYlUAEHoCGkzMZrOmTJniM5aUlOT9c+/evbVo0aJAlgQAwAWb1C1Ga48c975+84cCTbs8Ts2j\nLVW8CwBQEQ0WAQC4QDe1iVRSbHkIcXmkBdudVbwDAHAmggkAABfIYj5Hw8VtThW62ToYAGqKYAIA\nQB24p2O04mzlewdnF3n05o8FQawIAEILwQQAgDoQazPrvk6+DRfnbM2XQcNFAKgRggkAAHVkXFeH\nLBUaLmbmuPX5QXqaAEBNEEwAAKgjbWOs+lW7KJ+xWVvzg1QNAIQWggkAAHVo8qW+D8F/dqBYmSdK\nglQNAIQOggkAAHXoyqZ29Wlq9xmbk8GsCQBUh2ACAEAdm9TNd9Zk6Q8Fyi4qDVI1ABAaCCYAANSx\nIe0i1SamvOFicWlZXxMAQOUIJgAA1DGr2aTxXXy3Dp6/zaniUrYOBoDKEEwAAPCDezs5FGMt3zv4\naKFHb9FwEQAqRTABAMAP4u1m3dsp2mdsNg0XAaBSBBMAAPxkfNcYmSs0XNx6wq0vD9FwEQDOhWAC\nAICfJMVaNaRtpM/YbBouAsA5EUwAAPCjM7cOXrm/WDtyaLgIAGcimAAA4EdXNbPr8iY2n7G5GWwd\nDABnIpgAAOBHJpNJk8+YNXljZ4GO03ARAHwQTAAA8LNfJUWpVXR5w8XCUkOvbGfrYACoiGACAICf\n2cwmjevq23BxXma+XDRcBAAvggkAAAEwopNDjgoNFw8XevTO7sIgVgQA9QvBBACAAEiIMOvXHX0b\nLs7aQsNFADiNYAIAQIBM7BqjCv0Wtfl4if57xBW0egCgPiGYAAAQIB3irLrxjIaLs7bQcBEAJIIJ\nAAABdWbDxdR9RfrxpDtI1QBA/UEwAQAggPo1t6tn4/KGi4akORnMmgAAwQQAgAAymUxnzZq8nlWg\nnGJPkCoCgPqBYAIAQIANTYrSRVHl/wQXuA29usMZxIoAIPgIJgAABJjdYtK4rr6zJi9nOFXiYetg\nAA0XwQQAgCAYdYlDUZbyzYMPFJRqOQ0XATRgBBMAAIKgUYRZd198RsPFrTRcBNBwEUwAAAiSid0c\nPq+/yy7RuqM0XATQMBFMAAAIko7xNg1uHeEzNnsrWwcDaJgIJgAABNGkbrE+rz/YW6TdeTRcBNDw\nEEwAAAjfNe6jAAAgAElEQVSia1rY1a2R1fvaY0hzabgIoAEimAAAEETnarj4nx0FynXRcBFAw0Iw\nAQAgyO7oEK1mFRou5rsNvUbDRQANDMEEAIAgi7CYdH9n3x265mY45abhIoAGhGACAEA9MLqzQxGW\n8tf7naVasacoeAUBQIBZqz+l7ng8Hk2fPl1ZWVmy2+2aOnWq2rRp4z3+v//7v1q+fLkaNWokSZoy\nZYratWsXyBIBAAiKJpEW3ZUcrVd3FHjHZm3N063to4JYFQAETkCDSVpamlwulxYuXKj09HTNnDlT\nM2bM8B7ftm2bHnvsMXXp0iWQZQEAUC9M7BbjE0y++alE3xx1qXczexCrAoDACOhSrk2bNiklJUWS\n1L17d2VmZvoc37Ztm1599VWNHTtWixYtCmRpAAAEXecEm65vRcNFAA1TQGdMnE6nYmLKt0Q0m81y\nu92yWsvKGDRokIYNGyaHw6E///nPWr16tQYMGFDtdbOysvxWc03VhxoQvri/4E/cX/XLLQlmfXog\n0vt6+e4CfZl+XC0iQ/NBeO4v+BP3V2jp2LFjlccDGkwcDoeczvLtDw3D8IYSwzB09913e4NLv379\ntGPHjhoFk+q+SX/LysoKeg0IX9xf8Cfur/rnYsPQ7ANHlZlT1v3dI5M+Lmyqx7vHB7my2uP+gj9x\nf4WfgC7l6tmzp9asWSNJSk9PV3JysveY0+nUXXfdpYKCAhmGoQ0bNqhz586BLA8AgKAzmUyaeEbD\nxcU7nMoroeEigPAW0BmTgQMHav369RozZowMw9C0adOUmpqqwsJCDR06VJMmTdLEiRNlt9vVu3dv\n9evXL5DlAQBQL9zZIVp/33hS2UVlYeRkiaH/7Cg4K7AAQDgJaDAxm82aMmWKz1hSUpL3zzfddJNu\nuummQJYEAEC9E2k1aXRnh57elOcdm5uRr3FdHLKYTUGsDAD8hwaLAADUQ/d3dshe4V/pPfml+mAv\nDRcBhC+CCQAA9VCzKIuGJUf7jM3JYOtgAOGLYAIAQD01qavvMyVrj7j07U+uIFUDAP5FMAEAoJ7q\nlmjTwJZnNFxk1gRAmCKYAABQj505a/LurkIdcJYGqRoA8B+CCQAA9dj1rSPUKb58E023Ic3LZNYE\nQPghmAAAUI+ZTSZNPGPW5JXtTuXTcBFAmCGYAABQzw2/OEqNIsr7l+S6DL2xsyCIFQFA3SOYAABQ\nz0VbzRpzie+syZyt+fIYRpAqAoC6RzABACAE3N/FIVuFf7V/zCtV6j4aLgIIHwQTAABCwEXRFt3e\nPspnbPZWHoIHED4IJgAAhIhJ3XyXc3112KXvj9FwEUB4IJgAABAiejS2q/9Fdp8xZk0AhAuCCQAA\nIWTyGbMmy3YV6lABDRcBhD6CCQAAIWRwm0glx1m8r0s80nwaLgIIAwQTAABCiNlk0oQzGi4u3O5U\ngZuGiwBCG8EEAIAQ8+uLo5VgL2+4eKLY0JKdhUGsCAAuHMEEAIAQ47CZNfISh8/YnAwaLgIIbQQT\nAABC0NguMbKWT5ooK9etT/cXB68gALhABBMAAEJQK4dFQ89ouDiLrYMBhDCCCQAAIerMhotph4q1\n5XhJkKoBgAtDMAEAIERd1sSulOa+DRfnZDBrAiA0EUwAAAhhZ86avPlDgY7QcBFACCKYAAAQwm5q\nE6mk2PKGiy6PtGC7M4gVAcD5IZgAABDCLOazGy4uyHSq0M3WwQBCC8EEAIAQd0/HaMXZyvcOPlbs\n0Zs/FgSxIgCoPYIJAAAhLtZm1n2dfBsuzt6aL4OGiwBCCMEEAIAwMK6rQ5YKDRe35bi16iANFwGE\nDoIJAABhoG2MVb9q59twcTYNFwGEEIIJAABhYvKlvg/Bf3agWJknaLgIIDQQTAAACBNXNrWrT1Ma\nLgIITQQTAADCyJkNF5f+UKDsIhouAqj/CCYAAISRIe0i1SamvOFicam0YBsNFwHUfwQTAADCiNVs\n0vguvlsHL9jmVHEpWwcDqN8IJgAAhJl7OzkUYy3fO/hooUdv0XARQD1HMAEAIMzE2826t1O0z9gs\nGi4CqOcIJgAAhKHxXWNkrtBwMeOEW18eouEigPqLYAIAQBhKirVqSNtInzEaLgKozwIaTDwej556\n6imNHj1aEyZM0L59+8553pNPPqkXX3wxkKUBABB2ztw6eOX+Yu3IoeEigPopoMEkLS1NLpdLCxcu\n1OTJkzVz5syzzlm2bJl27twZyLIAAAhLVzWz6/ImNp+xuRlsHQygfgpoMNm0aZNSUlIkSd27d1dm\nZqbP8c2bN2vr1q267bbbAlkWAABhyWQyafIZsyZv7CzQcRouAqiHrIH8Yk6nUzEx5f+BNJvNcrvd\nslqtys7O1rx58/TMM8/o008/rdV1s7Ky6rrUWqsPNSB8cX/Bn7i/wltXj9TMHqmjrrLfRRaWGpqx\nZq9GtXEH5Otzf8GfuL9CS8eOHas8HtBg4nA45HSWTyEbhiGrtayEzz77TLm5ufr973+vY8eOqaio\nSElJSRoyZEi1163um/S3rKysoNeA8MX9BX/i/moYJrvy9OiGk97Xy45G6rFrL5LdYqriXReO+wv+\nxP0VfgK6lKtnz55as2aNJCk9PV3JycneY8OHD9fixYs1d+5cjRgxQoMHD65RKAEAAFUb0ckhR4WG\ni4cLPVq2qzCIFQHA2QIaTAYOHCi73a4xY8bo+eef1x/+8AelpqbqnXfeCWQZAAA0KAkRZv26o2/D\nxdk0XARQzwR0KZfZbNaUKVN8xpKSks46j5kSAADq1sSuMZqf6dTpKLL5eIn+e8Sl/hdFBLUuADiN\nBosAADQAHeKsuvGMhouzttBwEUD9QTABAKCBOLPhYuq+Iv2QG5jduQCgOgQTAAAaiH7N7erZuLzh\noiFpbgazJgDqhzoLJkeOHKmrSwEAAD8wmUxnzZq8vrNAOcWeIFUEAOVqHEz69u2rjIyMcx777rvv\nNHz48DorCgAA+MfQpChdFFX+z3+B29CrO5xVvAMAAqPKXbn+85//qKioSFJZM8R3333X24ekou+/\n/97bKBEAANRfdotJ47rG6O8byxsuvpSRr0ndYmQz+7fhIgBUpco0UVxcrHnz5kkqm/5dvnz5Oc+L\niYnR6NGj6746AABQ50Zd4tAzm/JUWFq2efDBAo+W7y7UHR2iq3knAPhPlcFkzJgxGjlypAzDUL9+\n/TRv3jx169bN5xyLxeLXAgEAQN1qdKrh4oJt5Uu4Zm3N1+3to2QyMWsCIDiqfcbEYrHIarVq/fr1\n6tGjhywWi/d/hmEoNzc3EHUCAIA6NKGrw+f1d9klWnfUFaRqAKAWD797PB4tXLhQK1eulCR9++23\nuuGGGzR48GA98MADys9nu0EAAEJFx3ibBrf27fo+eyv/lgMInhoHk3nz5mnevHnKycmRJD333HOK\njY3VAw88oF27dmnu3Ll+KxIAANS9Sd1ifV6v2FOk3Xk0XAQQHDUOJh9//LHGjRun4cOHa+/evcrK\nytLo0aN1zz33aMKECfriiy/8WCYAAKhr17Swq1uj8sdNabgIIJhqHEyOHDmiXr16SZLWrFkjk8mk\nlJQUSVLLli151gQAgBBzroaL/9lRoFwXDRcBBF6Ng0liYqKys7MlSWvXrlWHDh3UpEkTSVJWVpYa\nN27snwoBAIDf3NEhWs0qNFzMdxtaTMNFAEFQ42By9dVX68UXX9STTz6pdevW6cYbb5Qkvf7665o7\nd66uvfZavxUJAAD8I8Ji0v2dfXfoeinDKbfHCFJFABqqGgeThx56SH379tX333+vO+64Q3fffbck\nadmyZbr66qs1YcIEvxUJAAD8Z3RnhyIrtCXb7yzVij1FwSsIQINUZYPFiux2u6ZMmXLW+Ouvv67I\nyMg6LQoAAAROk0iLhidH69UdBd6xWVvzdGv7qCBWBaChqXEwkaTi4mItX75c3377rfLy8pSQkKBe\nvXrpl7/8JeEEAIAQNrFbjE8w+eanEn19tFh9mkVU8S4AqDs1Xsp18uRJjRo1Ss8++6wyMzNVWFio\nLVu2aMaMGRoxYoTy8vL8WScAAPCjzgk2Xd/qzIaLPAQPIHBqHExmzZql7OxsvfTSS1q+fLkWLlyo\n5cuXa+7cucrNzaXBIgAAIe7MrYPf21Oovfk0XAQQGDUOJl9++aUmTJjg7WVy2mWXXaZx48YpLS2t\nzosDAACBc13LCHVJKF/l7TGklzOYNQEQGDUOJoWFhWrVqtU5j7Vq1YoGiwAAhDiTyaSJZ8yaLN7h\nVF4JDRcB+F+Ng0lSUpJWr159zmNffvmlWrduXWdFAQCA4LizQ7SaRJZ/PDhZYug/FR6KBwB/qfGu\nXPfcc48eeeQRlZSUaNCgQWrcuLGOHTumjz/+WO+//77+/Oc/+7NOAAAQAJFWk0Z3dujpTeWb2szN\nyNe4Lg5ZzKYgVgYg3NU4mAwaNEh79+7VokWLtHz5ckmSYRiy2+0aPXq0hg4d6rciAQBA4Nzf2aF/\nbc6T69QKrj35pfpgb5F+lURfEwD+U+Ngkp+frzFjxujOO+/Uli1blJubq5iYGF166aVKSEjwZ40A\nACCAmkVZNCw5Wq9nlS/hmpORTzAB4FfVPmOydetWDRs2TG+88YYkKTY2VikpKerbt6/+9Kc/acSI\nEdq+fbvfCwUAAIEzqavvQ/Brj7j07U+uIFUDoCGoMpjs27dPDzzwgDwej7p06eJzLCoqSg8//LDM\nZrMmTJiggwcP+rVQAAAQON0SbRrY8oyGixn5QaoGQENQZTBZtGiRmjVrpldffVX9+/f3ORYREaFb\nb71Vr7zyiuLj4/Xqq6/6tVAAABBYZ86avLurUPtpuAjAT6oMJhs3btQ999yjmJiYSs9JSEjQPffc\now0bNtR5cQAAIHiubx2hTvHlj6O6DWleJg0XAfhHlcHk2LFjlTZVrKhDhw46evRonRUFAACCz2wy\naeIZsyaLdjiVT8NFAH5QZTBJTEzUTz/9VO1Fjh8/rvj4+DorCgAA1A/DL45So4jy/iW5LkNv7KTh\nIoC6V2UwueKKK/TBBx9Ue5EPPvhAl1xySZ0VBQAA6odoq1ljLvGdNZmzNV8ewwhSRQDCVZXB5M47\n79SGDRv0r3/9S8XFxWcdLykp0b///W+tW7dOd9xxh9+KBAAAwXN/F4dsFT4x/JhXqtR9RcErCEBY\nqrLBYufOnfXHP/5RM2bM0EcffaTevXurZcuWKi0t1eHDh7Vx40bl5ORo/PjxSklJCVTNAAAggC6K\ntuj29lFa8kOhd2zW1nzd1JaGiwDqTrWd32+//XZ16tRJr732mtLS0uRylTVXio6OVt++fXXPPffo\n0ksv9XuhAAAgeCZ1i/EJJv897NL3x1zq2dgexKoAhJNqg4kkde/eXU8//bQkKScnRxaLRbGxsX4t\nDAAA1B89GtvV/yK7vjpc3v199tZ8vXRNYhCrAhBOqnzG5FwSEhIIJQAANECTu/k+BP/2j4U6VFAa\npGoAhJtaBxMAANAwDW4TqeQ4i/e125DmZ+YHsSIA4SSgwcTj8eipp57S6NGjNWHCBO3bt8/n+KpV\nqzRixAiNHDlSS5YsCWRpAACgGmaTSRPOaLi4cLtTBW4aLgK4cAENJqcfnl+4cKEmT56smTNneo+V\nlpZq1qxZmjVrlhYsWKC33npLOTk5gSwPAABU49cXRyvBXt5w8USxoSU7C6t4BwDUTECDyaZNm7zb\nCnfv3l2ZmZneYxaLRUuXLlVMTIxyc3Pl8Xhktdbo2XwAABAgDptZIy9x+IzNyaDhIoALF9BP/k6n\nUzEx5VPAZrNZbrfbG0CsVqs+//xzPf300+rXr5+iomq2P3pWVpZf6q2N+lADwhf3F/yJ+wu1dX2k\nSS+YIlVqlM2cZOW69erXP6p/4tlLuri/4E/cX6GlY8eOVR4PaDBxOBxyOp3e14ZhnDUrct111+na\na6/V3/72N3344Yf65S9/We11q/sm/S0rKyvoNSB8cX/Bn7i/cD46Srrt2HG9+WP5Eq53T8Rr1FVN\nfM7j/oI/cX+Fn4Au5erZs6fWrFkjSUpPT1dycrL3WH5+vsaPHy+XyyWz2ayoqCiZTKbKLgUAAIJo\n0hlbB6cdKtaW4yVBqgZAOAjojMnAgQO1fv16jRkzRoZhaNq0aUpNTVVhYaGGDh2qG264QePHj5fV\natXFF1+sG2+8MZDlAQCAGrqsiV0pze1ae8S34eLsAY2CWBWAUBbQYGI2mzVlyhSfsaSkJO+fhw4d\nqqFDhwayJAAAcJ4mdYvR2iPHva/f+rFAj14Rp+bRlireBQDnRoNFAABwXm5qE6mk2PIQ4vJIC7Y7\nq3gHAFSOYAIAAM6LxXx2w8UFmU4Vutk6GEDtEUwAAMB5u6djtOJs5ZvVHCv26M0fC4JYEYBQRTAB\nAADnLdZm1ogzGi7O3povg4aLAGqJYAIAAC7IuC4OWSrs8L8tx61VB4uDVxCAkEQwAQAAF6RNjFW/\nahflMzZ7a36QqgEQqggmAADggk2+1Pch+M8OFOsHJ42SAdQcwQQAAFywK5va1aep3WfsjYO2IFUD\nIBQRTAAAQJ2Y1M131uSjoxZlF5UGqRoAoYZgAgAA6sSQdpFqE1Oh4aJh0oJtNFwEUDMEEwAAUCes\nZpPGd/HdOvjFLfnam+8OUkUAQgnBBAAA1Jl7Ozl8Gi7mlRj67Vc58tDXBEA1CCYAAKDOxNvNmnZF\nnM/Yl4eKNT+TJV0AqkYwAQAAdWp0Z4cGtozwGXt0w0n9kMuSLgCVI5gAAIA6ZTaZ9EK/BDks5cu3\nCksNTfrqhEo9LOkCcG4EEwAAUOfaxFj1UAeXz9j6oy7NoiM8gEoQTAAAgF/8slmpBreJ9Bl7/NuT\nyjxREqSKANRnBBMAAOAXJpM08+oENYoo36XL5ZEmrj6hEpZ0ATgDwQQAAPjNRdEWzeib4DO26ViJ\nntucF6SKANRXBBMAAOBXt7WP0q1JUT5jz2zK06ZsVyXvANAQEUwAAIBfmUwmPZsSr6aR5R873IY0\nafUJFZeypAtAGYIJAADwu8aRFv3rat8lXRk5bv3zu5NBqghAfUMwAQAAAXFzuyjdley7pGvmlnx9\nc5QlXQAIJgAAIID+eVWCWkaXf/zwGGW7dBW4PUGsCkB9QDABAAABkxBh1gv9G/mM7Tzp1t83sqQL\naOgIJgAAIKB+3ipSoy6J9hmbm+HU6kPFQaoIQH1AMAEAAAH3997xahdj8Rmb/NUJ5ZWwpAtoqAgm\nAAAg4GJtZs0a4Luka29+qf7n69wgVQQg2AgmAAAgKPpfFKGJXR0+Y4t2FOjT/UVBqghAMBFMAABA\n0Ey7Il4d460+Yw/+94RyilnSBTQ0BBMAABA0UVaT5gxoJLOpfOxggUd/WZ8TvKIABAXBBAAABNWV\nTe36ffcYn7GlPxRqxZ7CIFUEIBgIJgAAIOj+0itOXRv5Lun6w5ocZReVBqkiAIFGMAEAAEEXYTFp\n7oBGslZY0vVTkUd/XJsjwzCCVxiAgCGYAACAeqFHY7v+3CvWZ2z57iIt28WSLqAhIJgAAIB64w89\nYnVZE5vP2B/X5uhwAUu6gHBHMAEAAPWGzVy2S1dEhabwOS5Dv/vvCZZ0AWGOYAIAAOqVzgk2PXJZ\nnM/Yyv3Fen1nQZAqAhAI1upPqTsej0fTp09XVlaW7Ha7pk6dqjZt2niPr1y5UkuWLJHFYlFycrL+\n8pe/yGwmOwEA0NBM6hajD/YWad1Rl3dsyvpcXdMiQm1jAvrxBUCABPRTf1pamlwulxYuXKjJkydr\n5syZ3mNFRUWaO3eu5syZo/nz58vpdOqrr74KZHkAAKCesJhNmj2gkaIrbNOVV2Loga9y5GFJFxCW\nAhpMNm3apJSUFElS9+7dlZmZ6T1mt9s1f/58RUZGSpLcbrfsdnsgywMAAPVIhzir/nal75KutEPF\nWrjNGaSKAPhTQOdCnU6nYmLKO7uazWa53W5ZrVaZzWY1btxYkrR06VIVFhbqqquuqtF1s7Ky/FJv\nbdSHGhC+uL/gT9xf8KcLvb+usUi94yP0TW750/CPfJ2jDiWH1SaKmZOGjv9+hZaOHTtWeTygwcTh\ncMjpLP8th2EYslrLS/B4PHrhhRe0d+9eTZ8+XSaT6VyXOUt136S/ZWVlBb0GhC/uL/gT9xf8qa7u\nr4Ut3br63aPKKykLIkUek57eF68Pbmwii7lmnxUQfvjvV/gJ6FKunj17as2aNZKk9PR0JScn+xx/\n6qmn5HK59Mwzz3iXdAEAgIatTYxVT/aJ9xlbd9Sl2Rn5QaoIgD8EdMZk4MCBWr9+vcaMGSPDMDRt\n2jSlpqaqsLBQXbp00XvvvadevXpp0qRJkqThw4fruuuuC2SJAACgHvpNx2i9v6dQH+8v9o49/u1J\nDWodqc4JtireCSBUBDSYmM1mTZkyxWcsKSnJ++f169cHshwAABAiTCaTZvZrpJR3jijHVbakq7hU\nmrj6hD65uamsLOkCQh5NQgAAQEhoEW3RjJQEn7Hvskv0/Oa8IFUEoC4RTAAAQMi4vX2UftXO9znU\n6ZvytPmYq5J3AAgVBBMAABAyTCaTnrs6QU0iyz/CuA1pwuoTKi5l+2AglBFMAABASGkSadG/rvZd\n0pVxwq2nN50MUkUA6gLBBAAAhJwh7aJ0Z3KUz9jz6fna8BNLuoBQRTABAAAh6emrEtQiuvyjjMco\n26Wr0M2SLiAUEUwAAEBISogw64V+jXzGsnLd+se3uUGqCMCFIJgAAICQdX3rSI3oFO0zNmerU18d\nLq7kHQDqK4IJAAAIaY/3iVfbGIv3tSFp8uoTyi/xBK8oALVGMAEAACEt1mbWrP6+S7r25Jdq2jfs\n0gWEEoIJAAAIeQNaRGh8F4fP2MLtTn12oChIFQGoLYIJAAAIC49eGafkOIvP2ANfnVBOMUu6gFBA\nMAEAAGEh2mrWnAGNZDaVjx0s8GjK1+zSBYQCggkAAAgbfZpF6MFLY3zG3thZoA/2FAapIgA1RTAB\nAABhZcplceqSYPUZ+/2aHB0rKg1SRQBqgmACAADCSoTFpDkDGslaYUnXT0Ue/WktS7qA+oxgAgAA\nwk6vJnb9qWesz9g7uwu17MeCIFUEoDoEEwAAEJb+2DNWPRvbfMfW5ehIAUu6gPqIYAIAAMKSzWzS\n3AGNZK/waedEsaEH1+TIMIzgFQbgnAgmAAAgbHVpZNPUy+N8xlbuK9L/7mRJF1DfEEwAAEBY+223\nGPVpavcZm7I+V/vz3UGqCMC5EEwAAEBYs5jLdumKspRv03WyxNBv/8uSLqA+IZgAAICwlxxv1WNX\n+i7p+uJgsRZudwapIgBnIpgAAIAGYWwXhwZc5Luka9o3J7XrJEu6gPqAYAIAABoEs8mkF/s3Uqyt\nfEmX021o0lcnVOphSRcQbAQTAADQYLSLteqJPvE+Y2uPuDQnIz9IFQE4jWACAAAalHs7RmtQqwif\nsX98e1Lbc0qCVBEAiWACAAAaGJPJpH/3b6QEe/mSruJSaeLqE3KzpAsIGoIJAABocFpEW/R03wSf\nsW+zS/SvdJZ0AcFCMAEAAA3SsA5R+mW7SJ+x6ZtOKv04S7qAYCCYAACABslkMum5lAQ1iSz/OFTi\nkSZ8eVyuUpZ0AYFGMAEAAA1W0yiLnkvxXdK19YRbT2/KC1JFQMNFMAEAAA3ar5KidGeHKJ+x59Pz\ntPEnV5AqAhomggkAAGjwnu6boBbR5R+LSo2yXboK3SzpAgKFYAIAABq8hAiz/t2vkc/Yjly3Hv/2\nZJAqAhoeggkAAICkQa0jdV+naJ+x2VvzteZwcZAqAhoWggkAAMApj/eOV5sYi/e1IWnSVyeUX+IJ\nXlFAA0EwAQAAOCXObtas/r5LunbnlerRDSzpAvwtoMHE4/Hoqaee0ujRozVhwgTt27fvrHOKiop0\n//33a/fu3YEsDQAAQJJ0TYsIjevi8BlbsM2pzw8UBakioGEIaDBJS0uTy+XSwoULNXnyZM2cOdPn\neEZGhsaNG6f9+/cHsiwAAAAfj10Zp+Q4i8/Yb7/KUa6LJV2AvwQ0mGzatEkpKSmSpO7duyszM9Pn\neElJiZ555hklJSUFsiwAAAAf0VazZvdvJLOpfOxAQammrM8NXlFAmLMG8os5nU7FxMR4X5vNZrnd\nblmtZWX07NnzvK6blZVVJ/VdiPpQA8IX9xf8ifsL/hTK91eipN+0tGnxAZt37H93FugK2wld07g0\neIXBK5Tvr4aoY8eOVR4PaDBxOBxyOp3e14ZheEPJhajum/S3rKysoNeA8MX9BX/i/oI/hcP99XR7\nQ9+8f1SZOW7v2PTdURraq5kSIy1VvBP+Fg73F3wFdClXz549tWbNGklSenq6kpOTA/nlAQAAaiXS\natKcAY1kqbCk62ihR39ax5IuoK4FNJgMHDhQdrtdY8aM0fPPP68//OEPSk1N1TvvvBPIMgAAAGqs\nVxO7/tQz1mds2a5CvbOrIEgVAeEpoEu5zGazpkyZ4jN2rgfd586dG6CKAAAAqvennrH6aG+RNh8v\n8Y79cW2urm4eoebRLOkC6gINFgEAAKphM5ct6bJX+OR0vNij36/JkWEYwSsMCCMEEwAAgBrolmjT\nXy+L8xn7aF+RlvxQGKSKgPBCMAEAAKihBy6NUe+mNp+xv6zP0f58dyXvAFBTBBMAAIAaspxa0hVV\nYZuuky5DD/6XJV3AhSKYAAAA1MLF8TY9eqXvkq5VB4u1aDu7dAEXgmACAABQS+O6ONT/IrvP2CPf\n5Gp3Hku6gPNFMAEAAKgls8mkF/s3Uoy1fEmX021o0uoT8rCkCzgvBBMAAIDzkBRr1RN94n3G1hxx\naW6GM0gVAaGNYAIAAHCe7usUretbRfiM/X1jrnbklFTyDgCVIZgAAACcJ5PJpH/3a6R4e/mSrqJS\naeLqE3J7WNIF1AbBBAAA4AK0dFj0dN8En7GN2SX695b8IFUEhCaCCQAAwAW6s0OUbm4b6TP21Hcn\nte47aQ4AABi6SURBVOU4S7qAmiKYAAAAXCCTyaTnr05Q44jyj1YlHmnC6hNylbKkC6gJggkAAEAd\naBZl0XNX+y7p2nK8RM98nxekioDQQjABAACoI7ckRemODlE+Y89tztO3P7mCVBEQOggmAAAAdeiZ\nvglqHlX+EavUKNulq8jNki6gKgQTAACAOtQowqx/92vkM7Y9160nvjsZpIqA0EAwAQAAqGOD20Tq\nNx2jfcZe3JKvtUeKg1QRUP8RTAAAAPzgyT7xau2weF8bkiatPiFniSd4RQH1GMEEAADAD+LsZs3q\n77tL1668Uj22gSVdwLkQTAAAAPzk2paRGtvZ4TM2b5tTaQeLglQRUH8RTAAA+P/t3XtUlGUeB/Dv\nzDAMMFwF0fWStgqEiYQGgoaX7OZ6K90095w0bwmmIpJuNB7FsFNqipgCKhGUYobt1mbq2XPsZAJG\nZqiU7EplKx1MUBguA8MwM+/+McwrIxiYwIvT93MOp3mf9zK/d2aq9zvP87xD1IUSHnbH/W4Km7aX\ncrWoNnBIF1FLDCZEREREXUitlCM10guyFm2/6EzQfF0tWU1EPRGDCREREVEXC++jwvLhrjZt+0vq\ncby0QaKKiHoeBhMiIiKibqAJcUeAh4NNW0yeFpV6k0QVEfUsDCZERERE3cDJQYa0cV5QtBjTda3B\njLUFHNJFBDCYEBEREXWbEB9HrB7hZtN2+KcGfPIzh3QRMZgQERERdaM1wW4I6qW0aVudr0V5A4d0\n0R8bgwkRERFRN3JUyJAW6QVli6uwG41mrMrXQm8UpCuMSGIO7W9CRERERJ3pwV5KxIe447WzN38F\n/ugVPfq+XwY3pQzeTnL0dpLD20mB3k5y+DjJLW3OCvg0L/s4WR6rWk5aIbqHMZgQERERSWDlcFcc\nvdKAbyqabNprmwTUNpnwc60JQFPbO7fgLgYZRXN4sQ0utwYZRwYZ6qEYTIiIiIgk4CCXITXSC5Gf\nlONu7hhc0ySgpsmEy7UdO4i7o6y5F0Yh9sy0DC69nS09NdZAo5QzyFD3YDAhIiIikoifhxIfPOaN\nNwtrcbnWiOt6M0xdPM2kxiCgxmDCjzUdCzIejjL0bg4qN4OMAj7NPTPWIWfW9Qwy9HsxmBARERFJ\naEI/J0zo5wQAMAsCqg0CrutNqGgw47re+mdChd6MG9blBhOuN1oem7s4yFQbBFQbjPihpv1tAcDT\nUXbLXBjbuTIte2Z6qeRwYJChZgwmRERERD2EXCaDl0oGL5Ucfh7tb28WBGgbzajQ24YYS3ixLFfo\nTWKgudHY9UFGaxCgNRhR0sHfjfRSyW7Oj2nZG6NqDjLNIae3kyXIKBhk7BaDCREREdE9Si6ToZeT\nAr2cFAjowPYmswCt4WaQuaE3o6LB1CLU3BJk9GZ09Q2MqxoFVDUagQ4EGRkAL5UlpChMKrhfqoCD\nHHCUy6BUyKCUWW7HrLS2yZsfK2RwlFvm9TjKWzxWAMrmNqW8+XFzm7J5O0e5zPIcCtnNx83rLM9p\n2UcuY2C6WwwmRERERH8QCrkM3k4KeDspOrS9ySygymBuMazMNsRYh5zd0FvCTlVj1wYZAUBloxmV\njWYACqDO0IXPdmcUMrQKOA4twk3LkKRsuV1zm4OsrfAjg1IMSjfbHG4TtmzD2c191EoZfJ079p5L\nicGEiIiIiNqkkMua54R07KLWaBZQ1XJoWXNvzM35MTfnylToTahqtJ8flDQJQINJQIMJQJf3M92Z\nyL6O+HRyb6nLaBeDCRERERF1Cge5ZeJ77w5+O280C6hsbN0jYwkvtwwxazBBa+hZF/z3invlt2u6\nNZiYzWZs3rwZJSUlcHR0hEajwcCBA8X1p06dQnp6OhQKBaZPn46nn366O8sjIiIiom7kILcMMero\nMKMmsyDOf/nh5yvo038ADCZLe5NZgMFsfWz5p8FkaTM2rzOYBctjk+WxdVuDWUCTqcV6cVvLPw2m\nm8dsatEmrjd38Qt1l+6VO591azA5efIkDAYDMjIyUFRUhOTkZLz11lsAAKPRiKSkJGRmZsLZ2RmL\nFy9GZGQkvL29u7NEIiIiIuqhlHIZ+roo0NdFAdUNM/z6qKQuCQAgCAJMgjWw2AYY8bHpZpsYfloE\nHkPLQGUGmky3tlnC060BzNLesu3m/k2C5Tn6Osulfok6pFuDyblz5xAREQEACAoKQnFxsbju8uXL\nGDBgANzd3QEAwcHBKCwsxGOPPdadJRIRERER3RGZzDJ53UEugwsnSvxu3frS6XQ6uLq6istyuRxG\noxEODg6t1qnVatTV1XXouCUlJZ1e653qCTWQ/eLni7oSP1/Ulfj5oq7Ez9e9xc/P7zfXd2swUavV\n0Ol04rIgCHBwcBDX1dfXi+t0Oh3c3Nw6dNz2TrKrlZSUSF4D2S9+vqgr8fNFXYmfL+pK/HzZn24d\ncBYcHIz8/HwAQFFREYYMGSKuu//++1FaWorq6mo0NTXh3LlzCAoK6s7yiIiIiIhIIt3aYzJhwgQU\nFBRg0aJFEAQB69evx/Hjx9HQ0IBnnnkGq1atwsqVKyEIAqZNmwZfX9/uLI+IiIiIiCTSrcFELpcj\nPj7epm3w4MHi48jISERGRnZnSURERERE1APcG/cOIyIiIiIiu8ZgQkREREREkmMwISIiIiIiyTGY\nEBERERGR5BhMiIiIiIhIcgwmREREREQkOQYTIiIiIiKSnEyr1QpSF0FERERERH9s7DEhIiIiIiLJ\nMZgQEREREZHkGEyIiIiIiEhyDCZERERERCQ5BhMiIiIiIpIcgwkREREREUmOwYSIiIiIiCTnIHUB\n9zKz2YzNmzejpKQEjo6O0Gg0GDhwoNRlkZ0wGo1ITExEWVkZmpqasHDhQowbN07qssiOVFZWYt68\nedi1axcGDx4sdTlkRzIzM/Hll1/CaDRi1qxZmDFjhtQlkZ0wGo1ISEjA1atXIZfLodFo+N8vO8Ie\nk7tw8uRJGAwGZGRk4KWXXkJycrLUJZEdOXbsGDw8PLBv3z4kJydj69atUpdEdsRoNOKNN96ASqWS\nuhSyM2fPnsWFCxeQnp6OtLQ0XLt2TeqSyI7k5eXBZDLhnXfeweLFi5Gamip1SdSJ2GNyF86dO4eI\niAgAQFBQEIqLiyWuiOzJpEmT8OijjwIABEGAQqGQuCKyJ8nJyZg5cyaysrKkLoXszFdffYWhQ4di\n7dq10Ol0WLFihdQlkR257777YDKZYDabodPp4ODAS1l7wnfzLuh0Ori6uorLcrkcRqOR/5JQp3Bx\ncQFg+ZzFx8cjKipK4orIXhw5cgSenp6IiIhgMKFOp9Vq8euvv2L79u0oKytDXFwccnJyIJPJpC6N\n7ICLiwuuXr2KZ599FtXV1di+fbvUJVEn4lCuu6BWq6HT6cRlQRAYSqhTXbt2DdHR0Zg8eTKeeuop\nqcshO/Gvf/0LX3/9NaKionDp0iUkJCTg+vXrUpdFdsLDwwPh4eFQKpUYNGgQHB0dUVVVJXVZZCey\ns7MRHh6Ojz76CAcOHMDGjRvR2NgodVnUSRhM7kJwcDDy8/MBAEVFRRgyZIjEFZE9uXHjBlasWIHl\ny5dj+vTpUpdDdmTv3r3Ys2cP0tLS4O/vj4SEBPj4+EhdFtmJ4OBgnD59GoIgoKKiAnq9Hh4eHlKX\nRXbC3d1dHK3i7u4Oo9EIs9kscVXUWfj1/l2YMGECCgoKsGjRIgiCgPXr10tdEtmRzMxM1NTUICMj\nAxkZGQCAHTt2wMnJSeLKiIhuLzIyEoWFhXjhhRcgCALWrFnDOXLUaebOnYvExEQsWbIERqMR0dHR\ncHZ2lros6iQyrVYrSF0EERERERH9sXEoFxERERERSY7BhIiIiIiIJMdgQkREREREkmMwISIiIiIi\nyTGYEBERERGR5BhMiIg6wcaNGxEWFnbbv0ceeUSSumbMmNGltzL/9ddfMW3aNNy4caPLnqM9YWFh\nSE1Nlez5AeDjjz9GWFgYysrKuuT4RqMRYWFh2Lt3b4f3SUlJwdatW7ukHiKirsDfMSEi6iReXl54\n66232lwnl9vf90CCICAxMRHPPvssvL29pS6HbrFgwQLMmjUL48ePR1hYmNTlEBG1i8GEiKiTKJVK\nBAUFSV1Gtzl58iSKi4uxbds2qUuhNjg7O2Pu3LlISkpCdnY2ZDKZ1CUREf0mBhMiom4WFRUFX19f\nDBo0CDk5OWhsbMSoUaMQGxuL/v37i9sVFxcjLS0NFy9ehNFoxEMPPYSXXnoJQ4cOFbe5ceMGUlJS\nkJeXB71eDz8/P0RHR2PkyJHiNiaTCSkpKThy5Ahqa2vxwAMPIC4uDg888AAAoLGxEUlJScjLy0Nl\nZSX69OmDJ554AosXL4aDw+3/N5GZmYmJEyfCyclJbJsxYwYef/xxNDY24rPPPoNcLsfYsWMRGxsL\nT09PcbsLFy4gNTUV33//PZRKJSIiIhATE4PevXsDAM6ePYvo6GjEx8cjMzMTWq0WiYmJGDduXJu1\nNDQ04I033sDnn38Og8GAkJAQrFmzRnw9N27ciDNnzuDIkSPiPqWlpZg1axbWr1+PqVOnis+ZkpKC\n/fv349tvv4WjoyMmTZqEVatWib8ubTabkZGRgU8++QRarRbh4eEYMWKETT179+7F8ePHMWXKFBw8\neBAKhQIHDhyAj48PPv30U2RnZ+PKlSvw8vLC5MmT8eKLL0KpVIr7f/PNN0hJSUFJSQn69u2LuLi4\nVuf873//G1lZWbhy5QpUKhVGjRqFZcuWYdCgQeI2Tz75JHbv3o1Tp07d9rUjIuop7G9sARGRhIxG\nY5t/ZrPZZrvc3FwcPXoUcXFxeOWVV3Dp0iVER0ejoaEBgOXCdNGiRTAajVi3bh00Gg3Ky8uxePFi\n/PTTTwAAvV6PF198EQUFBVi2bBk2b94Md3d3xMTE4IcffhCf68SJEyguLsa6deuwfv16XL16FXFx\ncTCZTACA7du3Iz8/H8uXL8fbb7+NKVOmIDMzE1lZWbc9z//973+4ePEiHn/88Vbr/vGPf6CoqAgb\nNmzA8uXLkZeXh5iYGPE1OH/+PKKioiCXy/H666/j5Zdfxvfff4+lS5eirq7O5lh79uzBihUrEB8f\nbxO2bvXhhx9Cp9Ph9ddfx+rVq3H+/HmsW7fut96q29JoNAgMDMS2bdvwt7/9DR9//DHS09PF9Tt3\n7sQ777yD6dOnY8uWLXB3d29zjsvVq1dx8uRJbNq0CbGxsfDx8cH777+PxMREhISEYNu2bZg7dy4O\nHTpkMw/oP//5D1auXAm1Wo0333wTc+bMaTVP6Pz589iwYQPGjx+PHTt24O9//zsuXbqE2NhYCIIg\nbufr64ugoCAcO3bsd70WRETdiT0mRESdpLy8HGPGjGlz3YIFCxAdHS0uNzQ04L333sOAAQMAAIMH\nD8bzzz+PTz/9FLNnz8bu3bvRv39/7Ny5EwqFAgAwevRoPPPMM0hLS8OWLVtw5MgRlJaWIisrC4GB\ngQCAkSNHYt68eTh79qzYs+Lt7Y1t27bB0dERAFBTU4PNmzfj8uXLGDp0KAoLCxEWFoYnn3xSPIaL\niwu8vLxue65nzpwBAAwbNqzVOrlcjl27dsHV1RUA4OnpibVr1yIvLw+RkZHYtWsXBgwYgOTkZLFH\nZuTIkZg5cyYOHz6MF154QTzWzJkz8dhjj7XzygP+/v7YtGmTuFxaWor33nsPOp0OarW63f1bmjp1\nKpYuXQoACA0NxZkzZ3Dq1CmsWLECtbW1+PDDD/Hcc89hyZIlAICIiAhUVFTgq6++sjmOyWRCTEwM\nRo0aBQCoq6vDvn37MG3aNKxduxYAEB4eDl9fX2g0Gly4cAEjRoxAZmYmvLy8bN4zd3d3m6B17tw5\nqFQqzJ8/HyqVCgDQp08f5Obmor6+3uacAwMDcfTo0Tt6DYiIpMBgQkTUSXr16oXt27e3uc46RMkq\nKChIDCUAEBAQgP79++Pbb7/FtGnTcPHiRSxYsEAMJQDg5uaGyMhInDp1CoDl4rRv375iKAEs81wO\nHjxo81zDhg0TL3ABiMObampqAFguvnNyclBeXo6IiAiMGTMGc+fO/c1zLSsrg1qthru7e6t1Y8eO\nFUMJAIwbNw4KhQKFhYUIDQ1FUVERnnvuOQCWHibAEp4CAgJQUFBgE0z8/f1/sw6rhx56yGbZeo61\ntbV3HEyCg4Ntln19ffHLL78AAL777jsYjUZERkbabPPoo4+2CiaAbf1FRUXQ6/UYP368eN4AMGbM\nGMjlchQUFGDEiBE4d+4cxowZY/OeTZw40WaOyKhRo5Camoq5c+diwoQJiIiIQEhISKshZQDQr18/\n1NTUoK6uzuZ9ISLqaRhMiIg6iYODQ5s9CG3x9fVt1ebl5YWamhrU1tZCEIQ273Tl7e0tDnfSarW/\n2athZZ0bYWW9Q5h1yM+qVavg6+uLY8eOISkpCUlJSfDz88Pq1avFb/tvVVdXJ35T3965yeVyeHp6\norq6GjU1NTCbzcjOzkZ2dnarfQcOHGiz7OLi0u75tXWO1ov4W4fQdUTLOTOApX7ra1VdXQ0A8PDw\nsNnGx8en3bqs+7788sttbltRUQHAEhhbzscBLIGz5XMOHz4cO3fuRHZ2NnJycrB//364u7tj9uzZ\nWLJkiU2IsZ6PTqdjMCGiHo3BhIhIAtaL1JYqKysxfPhwuLm5QSaTtfnbINevXxcvUF1dXVFaWtpq\nm++++w5OTk42k+R/i1KpxPz58zF//nxUVFQgPz8f7777LtauXYvjx4/bTMq28vT0bDUf5HbnZjKZ\noNVq0atXL6jVashkMsyZMwdPPfVUq31b9hJ0JplMJs6psbLO57kT1sBQWVmJIUOGiO1arbbdfa2h\nICEhAYMHD77tsT09PVu99yaTCbW1tTZt1t/IaWxsRGFhIf75z38iPT0df/7zn22Gv1n3uzVMERH1\nNJz8TkQkgQsXLthczBYXF6OsrAyjR4+Gs7MzAgMDceLECZuL6bq6OuTm5opDjUJCQnD16lX897//\nFbcxGo3QaDTIycnpUB16vR6zZs3C/v37AViGnM2YMQN//etfUVtbC51O1+Z+f/rTn2AwGFBVVdVq\n3enTp9HU1CQuf/nllzCZTBg9ejTUajUCAgJw+fJlDBs2TPzz9/fHu+++i9OnT3eo7julVqtRU1MD\nvV4vtp0/f/6OjzNixAioVCqcOHHCpj03N7fdfYcPHw6lUony8nKbc3d1dcWuXbvw888/A7AMrTt9\n+jTq6+vFffPz820+Czt27MD8+fMhCAJUKhXCw8MRHx8PwPKjly2Vl5fDy8urVU8QEVFPwx4TIqJO\n0tTUhKKiotuuv//++8VvzfV6PWJiYrBw4ULU19cjJSUFQ4cOFXsRli1bhpiYGKxcuRJz5sxBU1MT\nsrKy0NjYKE66njp1Kg4dOoQ1a9Zg6dKl8Pb2xuHDh1FdXd3uHBErJycnBAQEID09HQ4ODvD390dZ\nWRkOHjyI0NDQVkOKrEaPHg3AMs9l4sSJNuvKy8sRFxeHOXPm4Nq1a0hJSUFERAQefvhh8dxiY2Oh\n0WgwefJkAMChQ4fwzTffYPbs2R2q+0498sgjOHToEDZt2oSnn34aP/744+/6bQ8XFxcsWLAAe/bs\ngVqtRmhoKHJzc5GXl9fuvp6ennj++eexb98+1NfX4+GHH0ZlZaW4HBAQAABYuHAhvvjiC8TExGDe\nvHmoqqpCWlqazXyj0NBQHDx4EBs2bMBf/vIXCIKAnJwcqFSqVrcFPn/+PMLDw+/oPImIpMBgQkTU\nSaqqqrBo0aLbrk9NTRXnbAQFBSEiIgKJiYkALBPEV65cKQ6bCgsLw65du7B3715oNBoolUqEhIRg\n/fr14hAttVqNPXv24O2330ZSUhJMJhMCAwORkpLS5lCh29FoNNizZw8OHjwoDhWbMGGCzV3EbtWv\nXz88+OCDyM/PbxVMJk2ahF69emHdunVQqVSYMmWKzbHCw8Oxc+dOpKen49VXXxUDUXJyMkJDQztc\n950YPXo0YmJi8MEHH+CLL75AYGAgtm7divnz59/xsRYuXAgXFxd88MEHOHToEIKDgxETE4MtW7a0\nu29UVBR8fHxw+PBhHDhwAG5ubhg1ahSioqLEOUWDBg1Camoqdu7ciVdffRXe3t6IjY21Of7YsWOR\nmJiI/fv345VXXgFgucnB7t27cd9994nbVVRU4NKlS2KYJSLqyWRarVZofzMiIuosUVFRMJlM2Ldv\nn9Sl3JWTJ09iw4YN+Oyzz8Q7X82YMQPBwcF47bXXJK6OACA9PR2ff/45Dhw4wF9+J6Iej3NMiIjo\ndxk/fjz8/Pw6PJ+FupdOp8NHH32EFStWMJQQ0T2BwYSIiH63hIQEHD58GNevX5e6FLpFZmam+Bsn\nRET3Ag7lIiIiIiIiybHHhIiIiIiIJMdgQkREREREkmMwISIiIiIiyTGYEBERERGR5BhMiIiIiIhI\ncgwmREREREQkuf8D+BGtUY61vG4AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "The training accuracy rate: 100.00%.\n", "The test accuracy rate: 70.00%.\n" ] } ], "source": [ "# train NN with mini-batch and momentum\n", "parameters = model(X_train, Y_train, layers_dims, optimizer=\"momentum\",\n", " learning_rate=0.03, mini_batch_size=64,\n", " beta1=0.9, num_epochs=1000, activation_fn=\"tanh\")\n", "\n", "# print the test accuracy\n", "print(\"The training accuracy rate: {}\".format(\n", " accuracy(X_train, parameters, Y_train, \"tanh\")[-8:]))\n", "print(\"The test accuracy rate: {}\".format(\n", " accuracy(X_test, parameters, Y_test, \"tanh\")[-7:]))" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyYAAAGkCAYAAADE0DHIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4lFX6xvF7JpPeG016ROlFaqSXKM2VIiDqioAI6tp7\nYy0riIuFtYCCiCgKCihFCF1AKQoIRIoElSJI+qS3yczvD34EhhSCZGZSvp/r2uvanDPzzpPwTpw7\n73ueYzCbzTYBAAAAgAsZXV0AAAAAABBMAAAAALgcwQQAAACAyxFMAAAAALgcwQQAAACAyxFMAAAA\nALgcwQQASvDhhx+qU6dOWrlypatLKRc333yz+vTp4+oyykViYqJWrFjh6jJKFR0drX/+85/q0aOH\nBg4cqNdff13p6ellfn5BQYEWLVqkUaNGqXv37rr55pv1/vvvKzc3t9jHb9++XXfffbd69+6tG264\nQZMnT1ZcXNwlX+fLL79Up06dLqs2AHAEggkAVBO33nqrxowZ4+oyrlhycrJGjBihzZs3u7qUEs2f\nP1+TJ0+WxWLRyJEj1bx5cy1evFgTJ05UTk5OmY4xffp0vfHGG/Lz89OoUaNUp04dzZs3T48//ris\nVqvdY9etW6eHH35YCQkJGjp0qLp06aJ169Zp/PjxSkpKKvE19uzZo3feeeeKvlcAKC8mVxcAAHCO\n0aNHu7qEcpGTk6PMzExXl1GiM2fOaObMmWrRooU+/PBDubu7S5IWLFigGTNmaMGCBRo/fnypx9i/\nf7+WLFmiXr16adq0aTIYDJLOhpUvv/xSa9as0YABAyRJ2dnZmjZtmmrVqqXPPvtM/v7+kqRevXrp\n6aef1syZM/X8888XeY1Vq1Zp6tSpJV6BAQBn44oJAADlaOnSpSooKNDYsWMLQ4l09opVeHi4li9f\nfsljfPXVV5KkCRMmFIYSSZo0aZLc3d21bNmywrE1a9YoLS1Nt912W2EokaQ+ffqoRYsWWrt2rd1V\nmvj4eD3yyCN68cUXFRYWpnr16l3R9wsA5YUrJgBQTmw2m7755ht9/fXX+uOPP+Tu7q7WrVvr7rvv\nVsuWLe0ea7FYtGTJEq1Zs0Z//PGHcnNzFRISos6dO2vixImqUaNG4WM7deqk/v37KyIiQvPnz5fV\natVdd92lqKgoDRkyRGPHjlXr1q310UcfKTY2Vl5eXuratav+9a9/KSwsrPA4N998s9LT07Vx40ZJ\n0sqVK/Xyyy/rf//7n37//XctXbpUf/31l8LCwjRw4ECNGzfO7oO1dPZD91dffaU///xToaGhGjZs\nmEJDQ/XSSy9p5syZat++fYk/nw8//FBz5szRO++8ow8++EC//vqratasqXnz5ikgIEC//fab5s+f\nrz179ig5OVkeHh5q1KiRRowYUXh14FzNkrRlyxZ16tRJd999t+655x5JZ2/z+uijj7RlyxYlJycr\nLCxMPXv21N13362AgIBS//1Onz6tIUOGXOqfWZMnT9bgwYNLnP/5558lqcjPws3NTW3bttW6dev0\n119/qXbt2qUeIzAwUE2aNLEb9/Pz07XXXqv9+/fLYrHIZDIVvl6HDh2KHKd9+/Y6cOCADh48qOuu\nu06SdPDgQW3fvl1DhgzRgw8+qMcff1wnT5685PcNAI5GMAGAcvKf//xHK1asUKNGjTR06FDl5uZq\nw4YNuueeezR16lT17Nmz8LHPP/+8Nm7cqNatW2vIkCHKz8/Xrl27tGLFCu3bt08LFy6UyXT+V/SP\nP/6orVu3atCgQTKbzXZBZ/v27frkk090/fXXa+TIkdq9e7dWrVql33//XfPnz79k3e+//76OHTum\nvn37qlu3blq/fr0++ugjZWdn6+GHHy583JtvvqmFCxeqbt26GjJkiMxms2bOnKlatWpd1s/p3//+\nt+rXr69Ro0bJbDYrICBABw4cKLwa0Lt3b4WEhOivv/7Spk2b9O9//1tWq1WDBg3SNddco1tvvVUL\nFy5UgwYNFBUVVRgA4uLiNGHCBMXFxalr165q3LixfvvtNy1cuFA7duzQnDlzSg0n/v7+uvvuuy9Z\n/zXXXFPq/KlTpxQSEiJfX98ic+fCyIkTJ0oMJnl5eYqPj1eLFi2Kna9du7Z++eUXnT59WvXr19ef\nf/4pSbrqqqtKfb1zweTqq6/WwoUL1bBhw1K/DwBwNoIJAJSDjRs3asWKFerXr59efvnlwlAxbtw4\njR07Vq+88oo6duwoHx8fxcTEaOPGjYqKitKrr75aeAyr1aqJEydq3759OnjwoFq3bl04l5ycrGnT\npql3796FY6dPn5YkHT58WP/5z390ww03SDrbzWnChAn65ZdfdODAgRI/4J5z8uRJffrpp2rQoIEk\n6Z///KeGDx+u5cuX61//+pdMJpMOHjyoRYsWqWXLlnr33Xfl4+MjSbrxxhv16KOPXtbPKjw8XDNn\nzpSbm1vh2AcffCCLxaJ58+YpIiKicHzHjh168MEHFR0dXWwwOXelRJKmTZumuLg4vf7663YhcNmy\nZXr11Vf17rvv6tlnny2xLn9/f7vj/V2pqamqU6dOsXN+fn6SpIyMjBKfn5aWVlhPWY6RmpoqDw8P\neXl5len16tate6lvAQBcgjUmAFAOzt3z/+ijj9pd6ahZs6ZGjRqltLQ0bdmyRZJUo0YNTZ48WZMm\nTbI7htFoVLt27SSd/bB5IU9PT3Xv3r3Y165Tp05hKJHO3jLUpUsXSSr8a3pp+vTpUxhKJCk0NFRN\nmzZVRkaGzGazpLMLpW02m+69997CUCJJ3bp1U+fOnS/5Ghfq1auXXSiRzi7Mf+mll+xCiXT+dqiL\nfx4XS0xM1A8//KDOnTvbhRLp7C1sDRo00OrVq2WxWC6r1r/DYrEUuQXunHPjeXl5pT5fkjw8PMp0\njNJe79wxWOAOoDLgigkAlINDhw7J3d1dS5cuLTJ34sQJSdKRI0fUv39/1axZU4MHD5bFYtGvv/6q\nEydO6M8//9SRI0f0008/SVKRdrA1atSwCzwXql+/fpGxc38pz8/Pv2TtZXn+wYMHJanYqy+tW7fW\nzp07L/k65xR3y1FkZKQkKSkpSUePHtWff/6p48ePa//+/ZKK/jwudvjwYdlsNqWnp+vDDz8sMm80\nGpWbm6vjx48XCT/npKen64svvrhk/b169Sr1di5PT88SA9C5n2dxVzcufP6Fjy3pGN7e3pd8vXPh\n5dxjAaAiI5gAQDlIT09XQUGB5syZU+JjLvyr/zfffKM5c+YoPj5e0tkg0Lx5c0VEROjnn3+WzWaz\ne25pH2SL+8v6hZ2cLqWkv8xLKqwjNTVV3t7edldLzgkPDy/za0nFfy9xcXF64403tHnzZtlsNhmN\nRtWtW1cdO3bUwYMHi/w8LnbuVqUDBw7owIEDJT7u3G1SxUlPTy/13++cOnXqlBpMAgICSrxV69z4\nueBXHD8/PxmNxjIfIyAgQH/88Yfy8vKK/FuW5fUAoKIgmABAOfDx8ZG3t3eZdonfsGGDpkyZooiI\nCD322GNq2rRp4SLl9957r7DLUkXi6+urU6dOFXaCutCV7ilis9n0yCOP6Pfff9edd96p3r17q3Hj\nxvLy8lJeXp6WLFlyyWOcuyIwbty4IrfIlVWdOnX0448//q3nXqh+/fravXu3cnJyioSwc+uCGjVq\nVOLz3d3dVbt2bZ06darY+dOnT8vLy0s1a9YsfL19+/bp9OnTRRa0n3s9FroDqAxYYwIA5aBJkyZK\nSEhQYmJikbkff/xR77//fuFf8qOjoyWpcDH7hd2Z/vjjD0m65BUCZ2vatKmsVmvhLV0X+uWXX67o\n2LGxsTp69KiioqJ0//33q3nz5oUf6I8dOybJ/udR3NWgc211i6tPkj7++GN9/PHHysrKuqJay6Jt\n27ay2WxFAmZBQYH27t2r2rVr27VxLk6bNm2UnJxc+P2fk5GRoV9//VUtWrQoDIht27aVJO3evbvI\ncfbs2SNPT09de+21V/AdAYBzEEwAoBwMHjxYNptN06ZNs1vYbDab9dprr2nevHmFC5TPrSE4c+aM\n3TE2btyo77//XpKcskj7cpzbt+P9999XdnZ24fiuXbv03XffXdGxL/x5XBhAMjIyNH36dElnP9Sf\nc+4D+YVrMOrUqaPrrrtOO3bs0Lp16+yOv2nTJs2cOVPr168v9la08nbjjTfKzc1NH374od2i84UL\nFyoxMVHDhg275DEGDRok6ewVtAvX18yaNUsWi0VDhw4tHOvZs6d8fX316aefFjYrkM6eTwcOHNDA\ngQMLf8YAUJFxKxcAXMInn3xS4i1agwcP1uDBgzVo0CB9//332rhxo0aPHq0uXbrIZrNp48aNSk5O\n1pgxYwrXJQwYMEBr167VU089paioKPn7++vw4cPatWuXgoODlZycfMkuVM7WunVrDRs2TEuXLtUd\nd9yhyMhIpaSkaOPGjfL395fZbJbR+Pf+1lW/fn21bNlS+/bt04QJE9S2bdvCLmaZmZny9fW1+3kE\nBwfL09NTu3fv1ptvvqmOHTuqe/fuevbZZ3XPPffoueee04oVK3T11Vfr1KlT2rJli3x9ffXcc8+V\n14+jVA0aNNCdd96pjz/+WHfccYd69OihEydOaPPmzWratKlGjhxp9/iVK1fq9OnTGjx4cGGb4Y4d\nO6p///6Kjo7W2LFjC9fa7Nq1S127dlXfvn0Lnx8QEKCHHnpIU6ZM0R133KF+/fopOTlZ69atU+3a\ntculBTIAOAPBBAAu4fjx4zp+/Hixc+c2rTMYDJoyZYoWL16slStXavny5fL09FTDhg316KOP2rXz\n7dq1q6ZMmaL58+drzZo18vT0VJ06dfTQQw+pR48eGjZsmLZt26ZbbrnFKd9fWT3xxBOqW7du4e72\n4eHhevDBB5WYmKhPP/201AX6pTEYDPrvf/+r999/Xzt37tSiRYsUHh6u9u3ba9y4cVqwYIFWrlyp\nQ4cOqVmzZjKZTHr66ac1a9YsLVmyRNnZ2erevbvq16+v+fPn66OPPtK2bdu0e/duhYSEqF+/fho/\nfrxT11nce++9Cg8P1+LFi7Vo0SKFhIRoxIgRmjBhQpEOWStXrtSePXvUvn17u/1PJk+erIYNG2rl\nypVauHChwsPDNW7cON11111F2i0PGTJE/v7+mj9/vpYsWSJ/f3/dcMMNuu+++xQaGuqU7xkArpTB\nbDZXrBuZAQAVTmJiotzd3RUYGFhk7sUXX9SqVasUHR2tkJAQF1QHAKgKWGMCALik6OhoRUVFFbml\n7c8//9TmzZvVqFEjQgkA4IpwxQQAcElxcXG67bbblJOTox49eqhu3bpKSkrSpk2blJ+fr7ffflsd\nOnRwdZkAgEqMYAIAKJOTJ09q3rx52rVrl5KSkuTn56e2bdvqrrvuUtOmTV1dHgCgkiOYAAAAAHA5\n1pgAAAAAcDmCCQAAAACXI5gAAAAAcDmCSTmIjY11dQmowji/4EicX3Akzi84EudX1UMwAQAAAOBy\nBBMAAAAALkcwAQAAAOByBBMAAAAALkcwAQAAAOByBBMAAAAALmdy5otZrVZNmzZNsbGx8vDw0HPP\nPad69epJkhITE/X8888XPvbIkSO6//77NXz4cGeWCAAAAMAFnBpMNm/erLy8PM2dO1cxMTGaMWOG\npk+fLkkKCwvTrFmzJEn79+/XzJkzNWTIEGeWBwAAAMBFnHor1969exUZGSlJatWqlQ4dOlTkMTab\nTdOnT9fTTz8tNzc3Z5YHAAAAwEWcesUkMzNTfn5+hV8bjUZZLBaZTOfL2Lp1qxo3bqwGDRqU+bgV\nYefPilADqi7OLzgS5xccifMLjsT5Vbk0adKk1HmnBhNfX19lZmYWfm2z2exCiSStXr1at95662Ud\n91LfpKPFxsa6vAZUXZxfcCTOLzgS5xccifOr6nHqrVxt2rTRtm3bJEkxMTGKiIgo8phDhw6pdevW\nziyrXOxOyJM51+rqMgAAAIBKyalXTHr16qWdO3dq/Pjxstlsmjx5sqKjo5Wdna2hQ4cqJSVFvr6+\nMhgMzizriq1NcNPL2xPUuYanFkeFysOtctUPAAAAuJpTg4nRaNQzzzxjN9awYcPC/x8cHKwFCxY4\ns6QrYrPZ9Ob+DL3yq6ckactfuXp0u1nvdA2qdOEKAAAAcCU2WLwC+VZp7Z85dmOfxWbprZgMF1UE\nAAAAVE4Ekyvg4WbQgr4husrLfm3Jy7vT9PUfWS6qCgAAAKh8CCZXKMzLTW83z1Wgh/2tW5O2pujH\n+FwXVQUAAABULgSTctDQx6ZP+4TKdEE2yS2QbtuQrGPpFtcVBgAAAFQSBJNy0qO2p2Z0DbIbS8yx\nauS6JNoIAwAAAJdAMClHtzfx1eOt/e3GjqRadOemZOUV2FxUFQAAAFDxEUzK2bPX+Wt4I2+7sXNt\nhG02wgkAAABQHIJJOTMaDHqvW7A61/CwG6eNMAAAAFAygokDeJnOthFu6O9mN04bYQAAAKB4BBMH\nCfNy05f9QmkjDAAAAJQBwcSBrgly12d9QuV+wU+ZNsIAAABAUQQTB+te21MzrqeNMAAAAFAagokT\n3NbEV4+3oY0wAAAAUBKCiZM81442wgAAAEBJCCZOYqCNMAAAAFAigokT0UYYAAAAKB7BxMloIwwA\nAAAURTBxAdoIAwAAAPYIJi5CG2EAAADgPIKJC9FGGAAAADiLYOJiJbURfoQ2wgAAAKhGCCYuVlIb\n4QW0EQYAAEA1QjCpAGgjDAAAgOqOYFJBhHm56auoUAXRRhgAAADVEMGkAmkS6K5PaSMMAACAaohg\nUsHQRhgAAADVEcGkAqKNMAAAAKobgkkFRRthAAAAVCcEkwqKNsIAAACoTggmFVhpbYSX/k4bYQAA\nAFQdBJMKrqQ2wvd+TxthAAAAVB0Ek0qANsIAAACo6ggmlQRthAEAAFCVEUwqEdoIAwAAoKoimFQy\nz7Xz1y2NaSMMAACAqoVgUskYDAa92zVYXWgjDAAAgCqEYFIJeZkM+ow2wgAAAKhCCCaVFG2EAQAA\nUJU4NZhYrVZNnTpV48aN06RJk3Ty5Em7+YMHD2rChAmaMGGCnn76aeXm8gG7NCW1ER69njbCAAAA\nqFycGkw2b96svLw8zZ07V/fff79mzJhROGez2fTqq69q8uTJmj17trp06aIzZ844s7xKqbg2wkm5\ntBEGAABA5eLUYLJ3715FRkZKklq1aqVDhw4Vzp04cUKBgYH64osvNHHiRKWlpalBgwbOLK/SKqmN\n8D83JtFGGAAAAJWCyZkvlpmZKT8/v8KvjUajLBaLTCaTzGazYmJi9MQTT6hevXp65JFH1KxZM3Xs\n2PGSx42NjXVk2WXi6hpG+kkx4R5ak3D+n3TrmTyNjz6hF5rkyWAo5cmo8Fx9fqFq4/yCI3F+wZE4\nvyqXJk2alDrv1GDi6+urzMzMwq9tNptMprMlBAYGqm7dumrUqJEkKTIyUocOHSpTMLnUN+losbGx\nLq9Bkj5pbNOQNYnaEZ9XOLYi3qR29UL0aGv/Up6JiqyinF+omji/4EicX3Akzq+qx6m3crVp00bb\ntm2TJMXExCgiIqJw7qqrrlJWVlbhgvi9e/eqcePGziyv0jvXRrgRbYQBAABQyTj1ikmvXr20c+dO\njR8/XjabTZMnT1Z0dLSys7M1dOhQPf/883rhhRdks9nUunVrdevWzZnlVQlhXm76MipUUSsTZM47\nv77k3u9TVNfPTZ1qeLqwOgAAAKB4BrPZzOroK1QRLyVu/StXw9YmKv+CxlyhnkZtuClcDf2dmkdx\nhSri+YWqg/MLjsT5BUfi/Kp62GCxiupe21P/6xpsN0YbYQAAAFRUBJMqbPTVPrQRBgAAQKVAMKni\nnmvnr1sae9uNbT2Tp0e2m2WzEU4AAABQMRBMqjiDwaB3uwarSw0Pu/EFsVl6c3+Gi6oCAAAA7BFM\nqoGS2gi/soc2wgAAAKgYCCbVxLk2wkEe9lvA3/t9in6Mz3VRVQAAAMBZBJNqpEmguz7tEyr3C/7V\ncwuk0euTdSzd4rrCAAAAUO0RTKoZ2ggDAACgIiKYVEOjr/bRE7QRBgAAQAVCMKmmnqWNMAAAACoQ\ngkk1RRthAAAAVCQEk2rMy2TQAtoIAwAAoAIgmFRzoaW0Ed4ZRxthAAAAOAfBBCW2Eb5tA22EAQAA\n4BwEE0gquY3wCNoIAwAAwAkIJihUXBvhWNoIAwAAwAkIJrBDG2EAAAC4AsEEdmgjDAAAAFcgmKAI\n2ggDAADA2QgmKBZthAEAAOBMBBOUqEmguz7rSxthAAAAOB7BBKXqVos2wgAAAHA8ggkuiTbCAAAA\ncDSCCcqkpDbCD2+jjTAAAACuHMEEZVJSG+HPj9JGGAAAAFeOYIIyo40wAAAAHIVggstCG2EAAAA4\nAsEEl402wgAAAChvBBP8LbQRBgAAQHkimOBvo40wAAAAygvBBFfk2Xb+GkEbYQAAAFwhggmuiMFg\n0Du0EQYAAMAVIpjgitFGGAAAAFeKYIJyEerlpq9oIwwAAIC/iWCCcnN1KW2E/0ijjTAAAABKRjBB\nuSqpjfDI9bQRBgAAQMkIJih3tBEGAADA5SKYwCFoIwwAAIDLQTCBQ9BGGAAAAJfD5MwXs1qtmjZt\nmmJjY+Xh4aHnnntO9erVK5z//PPPtWzZMgUHn12j8Mwzz6hBgwbOLBHl6Fwb4X4rE/RHekHh+Ct7\n0lTXz02jInxcWB0AAAAqEqcGk82bNysvL09z585VTEyMZsyYoenTpxfOHz58WC+++KKaNWvmzLLg\nQOfaCPdbmSBz3vlbuCZuSdG3x7P1SsdANfB36mkIAACACsipt3Lt3btXkZGRkqRWrVrp0KFDdvOH\nDx/WJ598ogkTJmjevHnOLA0OVFwbYUlafjxHnb+O09Sf05RloWMXAABAdebUP1VnZmbKz8+v8Guj\n0SiLxSKT6WwZUVFRGjFihHx9ffXkk09q69at6t69+yWPGxsb67Cay6oi1FCR1ZT0wtVuevGIh6w6\nvwljToE0bW+6PjmUqocb5atPaIEMhpKPU11xfsGROL/gSJxfcCTOr8qlSZMmpc47NZj4+voqMzOz\n8GubzVYYSmw2m0aPHl0YXLp27aojR46UKZhc6pt0tNjYWJfXUBk0aSJFNsnTkzvN+jkx327uTK5R\nTx/2VPdaHnqtc5BahLi7qMqKh/MLjsT5BUfi/IIjcX5VPU69latNmzbatm2bJCkmJkYRERGFc5mZ\nmbr11luVlZUlm82mXbt2qWnTps4sD07QsYaHNgwO1ztdgxTuVfT023omTz2Wx+vJHWY2ZAQAAKhG\nnHrFpFevXtq5c6fGjx8vm82myZMnKzo6WtnZ2Ro6dKjuu+8+3XvvvfLw8FDHjh3VtWtXZ5YHJzEa\nDPrnNb76R0Nvvb43XR8czJDlgq1NCmzSh4cytfj3bL1wXYDuvMZHbkbu7wIAAKjKDGazmd3urhCX\nEq/Mr+Z8Pb0zVZtO5xY73yrEXa93CVRkTU8nV1YxcH7BkTi/4EicX3Akzq+qhw0W4XLXBrlr6Q2h\nWtAnRA383IrMxyTna8CqRE3YnKzTmQXFHAEAAACVHcEEFYLBYNCgBt7aObSmnr8uQD6morduffV7\ntjoujdMb+9KVY+FCHwAAQFVCMEGF4mUy6PE2/vppWE3d0ti7yHymxaZX9qSpyzdxWnUiWzYbAQUA\nAKAqIJigQrrK101zeoZo1YAwtSymdfCx9ALdtiFZt6xL0hFzfjFHAAAAQGVCMEGFdn0tT22+KVxv\nRgYp2LPo7V0bTuXq+m/i9fyPqUrNo70wAABAZUUwQYXnZjRoXFNf7RleSxOa+urizsEWm/TugQx1\nWBKnz2IzZeX2LgAAgEqHYIJKI9jTqP9GBmnLP2qoWy2PIvMJOVb963uzolYmaFdCngsqBAAAwN9F\nMEGl0zLEXSv6h2lerxDV9S3aXnh3Yr76rUzQfVtTFJdFe2EAAIDKgGCCSslgMGhII2/9OKyGnmzr\nL6+i+USfH81Sh6VxeueXdOUVcHsXAABARUYwQaXmYzLq2XYB2jm0pv7RwKvIfHq+TS/8lKauy+K1\n/s8cF1QIAACAsiCYoEpo4G/S/D6hWnZjqJoFmYrMx6ZadMu6JN26Pkm/p1lcUCEAAABKQzBBldKz\njpe23FxDr3UOVIBH0fbC0Sdz1OXrOL28O1UZ+bQXBgAAqCgIJqhy3I0GTWrupz3Da2rMNT66OJ7k\nWaU392eo49I4fflbFrvHAwAAVAAEE1RZYV5umtE1WJtuClfnGkXbC/+VZdU9W1I0YFWi9ibSXhgA\nAMCVCCao8tqGeSh6YJg+7BGsWt5FT/kd8XnqvSJBD/+QosQc2gsDAAC4AsEE1YLBYNDICB/9NLym\nHmnlJ4+LznybpHlHstR+SZw+OJghi5XbuwAAAJyJYIJqxd/dqH93CNT2ITV1Y72i7YVT82x6ameq\nui+L1+bTuS6oEAAAoHoimKBaigg0aVG/UH0VFaqrA4q2Fz5ktujmNYm6c2OSjqfTXhgAAMDRCCao\n1qLqemnbkBp6pUOA/ExF2wsvP56jzl/HaerPacqy0F4YAADAUQgmqPY83Ax6oJW/dg2vqdFX+xSZ\nzymQpu1NV6el8Vp2LJv2wgAAAA5AMAH+Xy0fN83sHqz1g8N1XZh7kfk/Mws0ZlOy/hGdqAPJ+S6o\nEAAAoOoimAAX6RDuofWDw/VutyCFexV9i2w9k6cey+P1xA6zUnK5vQsAAKA8EEyAYhgNBt3RxFe7\nhtfU/S38dPHykwKbNPtQptovidPHhzNVQHthAACAK0IwAUoR6GHUq50C9cOQGupTx7PIfHKuVY9s\nN6vXigRtj6O9MAAAwN9FMAHK4Nogdy25IVQL+oSoob9bkfmY5HwNWJWouzcn61Qmu8cDAABcLoIJ\nUEYGg0GDGnhrx5CaeuG6APkU01548e/Z6rg0Tm/sS1eOhdu7AAAAyopgAlwmL5NBj7Xx10/DauqW\nxt5F5rMsNr2yJ01dvonTqhO0FwYAACgLggnwN13l66Y5PUO0akCYWoYUbS98LL1At21I1i3rknTE\nTHthAACqFEKKAAAgAElEQVSA0hBMgCt0fS1Pbb4pXG9FBinEs+hbasOpXF3/Tbye+zFVqXm0FwYA\nACgOwQQoB25Gg8Y29dXu4TU1oZmvjBctP7HYpPcOZKjDkjh9FpspK7d3AQAA2CGYAOUo2NOo/3YJ\n0tZ/1FC3Wh5F5hNyrPrX92b1W5mgXQl5LqgQAACgYiKYAA7QIsRdK/qH6ZPeIarrW7S98J7EfPVb\nmaD7tqYoLov2wgAAAAQTwEEMBoNubuitH4fV0FNt/eVVNJ/o86NZ6rA0Tu/EpCuvgNu7AABA9UUw\nARzMx2TUM+0CtHNoTf2jgVeR+fR8m17Ylabrv4nX+j9zXFAhAACA6xFMACdp4G/S/D6hWnZjqJoF\nmYrMH02z6JZ1Sbp1fZJ+T7O4oEIAAADXIZgATtazjpe23FxDr3UOVKBH0d3jo0/mqMvXcXppV6oy\n8mkvDAAAqgeCCeAC7kaDJjX30+7hNXXXNT66OJ7kWaW3YjLUcWmcVse7sXs8AACo8ggmgAuFebnp\n7a7B2nRTuDrXKNpe+K8sqyYf8dTwtUlKzqF7FwAAqLqcGkysVqumTp2qcePGadKkSTp58mSxj5sy\nZYreffddZ5YGuFTbMA9FDwzT7B7Bqu1T9G258XSueq9IUExyvguqAwAAcDynBpPNmzcrLy9Pc+fO\n1f33368ZM2YUeczSpUt19OhRZ5YFVAgGg0EjInz007CaerS1nzwuencezyjQjd8m6Os/slxTIAAA\ngAM5NZjs3btXkZGRkqRWrVrp0KFDdvP79+/XgQMHNGzYMGeWBVQofu5GTW4fqB1Da6rLRbd3ZVls\nGvtdil7claoCK+tOAABA1VG0Z6kDZWZmys/Pr/Bro9Eoi8Uik8mkxMREzZ49W//973+1fv36yzpu\nbGxseZd62SpCDah63rxaesPoriVn3O3G347J0I6TqfrPtbkKdC/hyUAZ8fsLjsT5BUfi/KpcmjRp\nUuq8U4OJr6+vMjMzC7+22Wwymc6WsGHDBqWmpurhhx9WUlKScnJy1LBhQw0ePPiSx73UN+losbGx\nLq8BVdfTxlj1aByux3eYdWH34B1mN9190F8L+oSqRQjpBH8Pv7/gSJxfcCTOr6rHqcGkTZs22rp1\nq6KiohQTE6OIiIjCuVGjRmnUqFGSpJUrV+rYsWNlCiVAdTDmWl81Czbpzo3JOpN9Pp0cSy/QDd8m\n6P3uwbq5obcLKwQAALgyTl1j0qtXL3l4eGj8+PF666239Mgjjyg6Olpff/21M8sAKqVONTy16R81\n1Cncft1JpsWmMZuS9cpu1p0AAIDKy6lXTIxGo5555hm7sYYNGxZ5HFdKgOLV9nHTigFhenKHWZ8c\nse/O9cb+DO1PytfsniEK8mSLIgAAULnw6QWoZDzdDJrRNVhvRQbJ/aJ38LpTueqzIl6HUtjvBAAA\nVC4EE6CSGtvUVyv6h6mmt/3b+Pf0AkWtTNDyY9kuqgwAAODyEUyASqxLTU9tuqmGOoTbd+XKsNh0\n56Zk/WdPmqw21p0AAICKr9yCSVxcXHkdCsBlqOPrpm8HhOuOJj5F5qbvS9foDclKzbMW80wAAICK\no8zBpEuXLjp48GCxcz///HNhq18AzufpZtA7XYM0vUugTAb7uTUnc9R3RYJ+NbPuBAAAVFylduX6\n7LPPlJOTI+nsZojffPONtm3bVuRx+/btK9woEYBrGAwG3d3MT82D3TVmU7IScs5fJTmaZlG/lQn6\noEewBtZnvxMAAFDxlJomcnNzNXv2bElnP/QsW7as2Mf5+flp3Lhx5V8dgMt2fS1PbbopXP/clKyf\nE89fJUnPt+m2Dcl6qq2/nmrrL6PBUMpRAAAAnKvUYDJ+/Hjdddddstls6tq1q2bPnq0WLVrYPcbN\nzc2hBQK4fHX9TFo9IFyPbjfr86P2+51M25uu/Un5+qBHsAI86H8BAAAqhkt+KnFzc5PJZNLOnTvV\nunVrubm5Ff7PZrMpNTXVGXUCuExeJoPe6xak1zsHyu2iiyOrT+ao38oExaay7gQAAFQMZf5zqdVq\n1dy5c7VmzRpJ0p49e9S/f3/deOONeuCBB5SRkeGwIgH8PQaDQfc099Oy/mEK87J/ux9JtajvigSt\nPsF+JwAAwPXKHExmz56t2bNny2w2S5LefPNN+fv764EHHtAff/yhWbNmOaxIAFem2/+vO2kbar/f\nSVq+TaM3JOv1vex3AgAAXKvMwWTt2rW65557NGrUKJ04cUKxsbEaN26cbr/9dk2aNEnfffedA8sE\ncKXq+Zm0emC4RkUU7co15ed0/XNjstLz2e8EAAC4RpmDSVxcnNq2bStJ2rZtmwwGgyIjIyVJderU\nYa0JUAl4mwya1T1YUzsVXXfy7YkcRa1M0G+pFtcUBwAAqrUyB5OQkBAlJiZKkrZv367GjRsrLCxM\nkhQbG6vQ0FDHVAigXBkMBt3bwk9f3ximEE/7XwGHzRb1XhmvtSdzXFQdAACorsocTK6//nq9++67\nmjJlinbs2KEBAwZIkhYsWKBZs2apZ8+eDisSQPnrUfvsupNWIRetO8mzadT6JL2xL1021p0AAAAn\nKXMwefTRR9WlSxft27dPt9xyi0aPHi1JWrp0qa6//npNmjTJYUUCcIwG/iatGRSmEY3t153YJL2y\nJ01jNiUrg3UnAADACUrdYPFCHh4eeuaZZ4qML1iwQF5eXuVaFADn8TEZ9WGPYLUJddfkXWmyXnCR\nZPnxHMWmJmhB31A1DijzrwsAAIDLdlmfNHJzc7Vs2TLt2bNH6enpCgoKUtu2bXXTTTcRToBKzGAw\n6F8t/dUyxF1jv0tWSu75dHLIbFHvFfH6qGeI+tXlfQ4AAByjzLdypaWlaezYsXrjjTd06NAhZWdn\n65dfftH06dM1ZswYpaenO7JOAE7Qq46XNt1UQy0vWneSmmfTiHVJems/604AAIBjlDmYvPfee0pM\nTNQHH3ygZcuWae7cuVq2bJlmzZql1NRUNlgEqoiG/iatGRimYY2Krjt5aXeaxn2XokzWnQAAgHJW\n5mCyZcsWTZo0qXAvk3PatWune+65R5s3by734gC4hq+7UR/1DNbLHQJkvGi/k6+PZSvq2wQdS2e/\nEwAAUH7KHEyys7N11VVXFTt31VVXscEiUMUYDAY92Mpfi6NCFeRhn04OpljUa3m8Np1ivxMAAFA+\nyhxMGjZsqK1btxY7t2XLFtWtW7fcigJQcfS56uy6k+bB9r0yzHk2DV+XpHdiWHcCAACuXJm7ct1+\n++16/vnnlZ+fr6ioKIWGhiopKUlr167VihUr9OSTTzqyTgAu1CjApLWDwnX/9yladuz8VRKrTXph\nV5r2Jefrf12D5GMq8986AAAA7JQ5mERFRenEiROaN2+eli1bJkmy2Wzy8PDQuHHjNHToUIcVCcD1\n/NyNmtcrRG/HZOjl3Wm68BrJ4t+zddhs0Wd9QtTQn/1OAADA5SvzJ4iMjAyNHz9eI0eO1C+//KLU\n1FT5+fmpZcuWCgoKcmSNACoIg8GgR1r7q1WIu8ZvTlZq3vl48ktyvnqviNe8XiHqWYf9TgAAwOW5\n5H0XBw4c0IgRI/TFF19Ikvz9/RUZGakuXbro8ccf15gxY/Trr786vFAAFUe/umfXnTQLsv/bRkqu\nTUPXJundX1h3AgAALk+pweTkyZN64IEHZLVa1axZM7s5b29vPf300zIajZo0aZJOnz7t0EIBVCyN\nA0xaOzhcNzWwvzpitUnP/5SmiVtSlGVhvxMAAFA2pQaTefPmqUaNGvrkk0/UrVs3uzlPT08NGTJE\nH3/8sQIDA/XJJ584tFAAFY+/u1Hze4fo+esCdNF2J/ry92z1/zZRJzLY7wQAAFxaqcFk9+7duv32\n2+Xn51fiY4KCgnT77bdr165d5V4cgIrPYDDo8Tb+WtgvVAEX7XeyPzlfvZcnaMtfuS6qDgAAVBal\nBpOkpKQSN1W8UOPGjRUfH19uRQGofG6s56WNg8N1baD9upOkXKuGrknUzAMZrDsBAAAlKjWYhISE\nKCEh4ZIHSU5OVmBgYLkVBaByujrQXesGh2tQfft1JwU26ZkfU3Xv1hRlWwgnAACgqFKDSfv27fXt\nt99e8iDffvutrr322nIrCkDlFeBh1Kd9QvRMO/8icwt/y9aAVQk6yboTAABwkVKDyciRI7Vr1y69\n/fbbys0teo94fn6+/ve//2nHjh265ZZbHFYkgMrFaDDoqbYB+qJviALc7ded7E3KV+8VCfr+DOtO\nAADAeaVusNi0aVM99thjmj59ulavXq2OHTuqTp06Kigo0JkzZ7R7926ZzWZNnDhRkZGRzqoZQCUx\noL63Ntxk0m0bkhWbev4qSWKOVUOiEzWlU6AmNPOVwXBxTy8AAFDdXHLn9+HDh+uaa67Rp59+qs2b\nNysvL0+S5OPjoy5duuj2229Xy5YtHV4ogMqpSaC71g8O18QtKYo+mVM4brFJT+5M1d6kfL0ZGSQv\nE+EEAIDq7JLBRJJatWql119/XZJkNpvl5uYmf/+i948DQHECPYz6vG+Ipu1N17S96XZznx/N0mFz\nvj7tE6qrfN1cVCEAAHC1UteYFCcoKIhQAuCyGQ0GPdMuQJ/1CZHfRVdH9iTmq9fyeG2PY90JAADV\n1WUHkythtVo1depUjRs3TpMmTdLJkyft5jdu3KgxY8borrvu0sKFC51ZGgAnGdzAW+tvCldEgP3V\nkYQcq25anaiPDrPfCQAA1ZFTg8m5NSpz587V/fffrxkzZhTOFRQU6L333tN7772njz76SIsXL5bZ\nbHZmeQCcpGmQuzYMrqEb63rajVts0mPbU/XgD2blFhBOAACoTpwaTPbu3VvYvatVq1Y6dOhQ4Zyb\nm5sWLVokPz8/paamymq1ymQq0xIYAJVQkKdRX/QL1eNtit4a+mlslgatTtDpzAIXVAYAAFzBqcEk\nMzNTfn5+51/caJTFcr6FqMlk0qZNm3T77bfruuuuk7e3tzPLA+BkRoNBz18XoPm9Q+R70bqTXQn5\n6r0iXjtYdwIAQLVgMJvNTrtf4q233lLLli0VFRUlSRo8eLBWrlxZ5HFWq1UvvfSSOnTooJtuuumS\nx42NjS33WgE412+ZBj1xyFMnc+z/XmIy2PRE43wNq81u8QAAVGZNmjQpdd6p90q1adNGW7duVVRU\nlGJiYhQREVE4l5GRoccee0zvvPOOPDw85O3tXeZN1y71TTpabGysy2tA1VVdzq8mkro0s2rC5mSt\nO3X+KonFZtDU3zz0l1uQpnUJkqcb+52Up+pyfsE1OL/gSJxfVY9Tg0mvXr20c+dOjR8/XjabTZMn\nT1Z0dLSys7M1dOhQ9e/fXxMnTpTJZNLVV1+tAQMGOLM8AC4W5GnUwn6hevXnNL25P8Nubt6RLB1M\nsWh+nxDV8mG/EwAAqhqn3spVVZHY4UjV9fz65o9s3fd9irIs9r+iankb9WmfUHWs4eGiyqqW6np+\nwTk4v+BInF9Vj1MXvwNAWQ1p5K11g8LV0N/+6siZbKsGrU7Q/COZLqoMAAA4AsEEQIXVIsRdm26q\noT517Pc7ybNKD/5g1qPbzMpjvxMAAKoEggmACi3Y06ivokL1UEu/InNzf83UP6ITFZfFficAAFR2\nBBMAFZ6b0aCXOgZqbs9g+Vy038mO+Dz1XhGv3Ql5LqoOAACUB4IJgEpjWGMfrRkUrvp+9utOTmdZ\nNWBVgj6LZd0JAACVFcEEQKXSKsRd390Url7FrDv51/dmPbHdrHwr604AAKhsCCYAKp0QLzctjgrV\nA8WsO5l9OFOj1ycRTgAAqGQIJgAqJZPRoFc6Bmp2j2B5X7Qb/PpTuXpiu1k2G+EEAIDKgmACoFIb\nEeGj6EFhqnfRupN5R7L03oGMEp4FAAAqGoIJgEqvTaiHNgwOLxJOXvgpTatOZLuoKgAAcDkIJgCq\nhBreblrUL1T+7udv67JJmrA5RfuTaCUMAEBFRzABUGU0D3bXx71CZLxgyUmmxaZb1yfpLzZhBACg\nQiOYAKhS+tX10rTOgXZjp7OsGr0+SZn5VhdVBQAALoVgAqDKmdDMT/c087Ub25uUr0lbU2SlUxcA\nABUSwQRAlTSlU6CirrLfhHHF8Ry9vDvNRRUBAIDSEEwAVEkmo0Ef9QpR8yCT3fjbMRn6LDbTRVUB\nAICSEEwAVFkBHkYtjApVDW/7X3UP/2DW1r9yXVQVAAAoDsEEQJVW38+kz/uGyuuCLU4sNumfG5N0\nNDXfdYUBAAA7BBMAVV6HcA/N7B5sN2bOs2nU+iSl5NKpCwCAioBgAqBaGNrIR8+187cb+y2tQHds\nTFJeAZ26AABwNYIJgGrj8Tb+GhXhbTf2w5k8PbLdLBtthAEAcCmCCYBqw2Aw6H9dgxVZ08NufEFs\nlmbEZLioKgAAIBFMAFQznm4GfdYnRA393ezGX9ydpuXHsl1UFQAAIJgAqHZCvdz0Zb9QBXgY7MYn\nbknRz4l5LqoKAIDqjWACoFq6Jshd83uHyO2CbJJdYNPo9Uk6lVngusIAAKimCCYAqq1edbz0RmSQ\n3diZbKtuXZ+kjHzaCAMA4EwEEwDV2l3X+ur+Fn52YzHJ+bp7c4oKrHTqAgDAWQgmAKq9lzsEaEA9\nL7ux6JM5mrwrzUUVAQBQ/RBMAFR7bkaDZvcMVqsQd7vx9w5kaN6vmS6qCgCA6oVgAgCS/NyNWtgv\nVLW87X8tPrbdrO9O57ioKgAAqg+CCQD8v6t83bSwX6i8L2jVVWCT7tyUrF/N+S6sDACAqo9gAgAX\naBvmoQ96BNuNpeXZNGp9kpJyaCMMAICjEEwA4CL/aOitF9sH2I0dSy/QHRuTlVtApy4AAByBYAIA\nxXiolZ/uaOJjN7Y9Lk8P/pAim41wAgBAeSOYAEAxDAaD3owMUrdaHnbji37L1hv7M1xUFQAAVRfB\nBABK4OFm0Kd9QhUR4GY3/p89afr6jywXVQUAQNVEMAGAUgR7GvVlvzAFeRjsxu/dmqJdCXkuqgoA\ngKqHYAIAlxARaNKnfULlfsFvzJwC6bYNSTqRYXFdYQAAVCEEEwAog+61PfXW9UF2Y/HZVt26Pklp\neVYXVQUAQNVhcuaLWa1WTZs2TbGxsfLw8NBzzz2nevXqFc6vWbNGCxculJubmyIiIvTUU0/JaCQ7\nAagY7mjiq6OpFr0dc37x+8EUi+7enKzP+4bKZDSU8mwAAFAap37q37x5s/Ly8jR37lzdf//9mjFj\nRuFcTk6OZs2apZkzZ2rOnDnKzMzU999/78zyAOCSJrcP0E0NvOzG1v6Zq+d+THVRRQAAVA1ODSZ7\n9+5VZGSkJKlVq1Y6dOhQ4ZyHh4fmzJkjL6+z/8G3WCzy8PAo9jgA4CpGg0Ef9AhWuzB3u/EPDmVq\nziHaCAMA8Hc59VauzMxM+fn5FX5tNBplsVhkMplkNBoVGhoqSVq0aJGys7PVuXPnMh03NjbWIfVe\njopQA6ouzq+KZ0pjaUyal+Lzzv9956kdZnmkxykyuHKtOeH8giNxfsGROL8qlyZNmpQ679Rg4uvr\nq8zMzMKvbTabTKbzJVitVr3zzjs6ceKEpk2bJoOhbPdrX+qbdLTY2FiX14Cqi/OrYmoiaUntfPX/\nNkGZlrM7wRfIoOePeGvNoHA1C3Yv/QAVBOcXHInzC47E+VX1OPVWrjZt2mjbtm2SpJiYGEVERNjN\nT506VXl5efrvf/9beEsXAFRUrULcNadnsC78E0pavk2j1icpIbvAZXUBAFAZOfWKSa9evbRz506N\nHz9eNptNkydPVnR0tLKzs9WsWTMtX75cbdu21X333SdJGjVqlHr37u3MEgHgsgyo763/dAq0W/x+\nIqNAt29I1vL+YfIy0akLAICycGowMRqNeuaZZ+zGGjZsWPj/d+7c6cxyAKBc3NfcV0dT8/Xxr1mF\nYz8m5OlfP6Rodo/gMt+WCgBAdcYmIQBwhQwGg17vEqTedTztxhf/nq3X9qa7qCrAdaw2m2KS83Us\ni1AOoOwIJgBQDtyNBn3cK0TXBtpfiJ62N11f/ZZVwrOAqsVitWnh0Sx1+Tpe3ZfFa8Qebz290yyr\nzebq0gBUAgQTACgnQZ5GLYoKVain/a/W+79P0c64XBdVBTheboFNn/yaqQ5L4zRpa4qOpFoK52Yd\nzNSkrSmyWAknAEpHMAGActTQ36QFfUPkccFv1zyrdPvGZB1Lt5T8RKASyrbY9MHBDF23OE4PbTPr\nWHrx3ei+/C1bYzYlK7eAcAKgZAQTAChnXWp66t1uwXZjiTlW3bo+Sal5lWvzRaA46flWzYhJV+uv\nzuipnak6lXXp9tjfnsjRqPVJysznPQCgeAQTAHCAkRE+eqKNv93YYbNFYzclc0sLKi1zrlXT9qap\n1Zdn9O9daUrIKRoy3I3SmGt8tOzGMNXxtJ//7nSuhq5JkjmXcAKgKIIJADjIs+38NayRt93YxtO5\nempnqmwsBkYlkpBdoJd2parVV2c09ed0mfOKnr9ebtI9zXz18/CamtE1WD3reGp269wiDSF+TMjT\n4OhENiEFUATBBAAcxGAw6L1uweoY7m43/tHhTM06mOmiqoCyO51ZoGd2mtX6qzi9FZOh9PyigcTP\nZNBDLf20f0Qtvd4lSHX9zgeRGp42fTswTG1C7d8DvyTna8CqRP2ZwborAOcRTADAgbxNBi3oG6p6\nfm5248/9lKo1J3NcVBVQuuPpFj26zay2i89o5sFMZRezaD3Qw6An2/pr/4iaeqljoGp4uxVzJCnM\ny03L+4cpsqaH3fjRNIv6r0rUb6mEEwBnEUwAwMFqeLtpUb9Q+buf32zOapPGf5esX5LzXVgZYC82\nNV/3bk3RdUviNPfXTBXXqyHU06jJ7QO0f0QtPdsuQCFexQeSCwV6GLXkhlD1vcp+E9I/Mws0YHUC\n7wMAkggmAOAUzYPd9XGvEBkv2Ag7w2LTreuTdKYMHY0AR/olOV/jvktWp6Xx+uJolorr6lvbx6gp\nnQK1f0RNPdraX4Eel/cRwsdk1Od9Q/WPBl524/HZVg1anaCf4vOu5FsAUAUQTADASfrV9dJrnQLt\nxv7MLNBtG5KUZaFLEZxvT0KebtuQpG7L4rX0j2wV15Khvp+b3owM0s/Da+m+Fn7ydf/7Hx083Qya\n2ytEt13tYzeemmfTkDWJ2nyajUiB6oxgAgBOdE9zP01o5ms3tifx7O0zVjp1wUm2ncnV8LWJ6rMy\nQatOFL/W6eoAk97vFqTdw2tqXFNfeZkMxT7ucpmMBr3bLUgTL3ofZFpsGrk+UatPZJfL6wCofAgm\nAOBkUzsFKuqie+2XHcvRq3vSXFQRqgObzaZNp3I0cFWCBq5O1IZTxV+daB5k0tyewdo5tIZua+Ir\nd2P5BJILGQ0GvdY5sMheP7kF0h0bk/XVb1nl/poAKj7TpR8CAChPJqNBH/UKUf9vE3TQfL4j0Rv7\nMxQRYNJtTXxLeTZweWw2m1afzNEb+9K1O7HkRebtwtz1eGt/DajvJaOh/MPIxQwGg567LkABHga9\n8NP5UF5gk+7ZkqKMfJvGNuW9AFQnBBMAcIEAD6MWRoWq74oEu92zH9pmVgN/k7rW8izl2cClFVht\nWn48W9P3petASskteSNreuiJNv7qXcdTBicEkos90NJfAe5GPbzNXLjGxSbpke1mpeVb9VAr/9Ke\nDqAK4VYuAHCR+n4mfd43VJ4XdFvNt0p3bEzS72ns7YC/J99q0xdHs9Tlm3iN/S6lxFDSu46nvh0Q\nptUDw9XnKi+XhJJzxlzrqzk9g3XxMpZ/70rTK7tTZWP9FVAtEEwAwIU61vDQzG7BdmMpuTaNXJck\ncy6dulB2uQU2fXw4Ux2WxOnerSmKLWHjwgH1vLRhcLi+vjGsQl2ZG97YRwsuCurS2Vscn9yZSnMI\noBogmACAiw1r7KNn29nfrnI0zaI7NyUr38qHMZQuy2LVzAMZarf4jB7ZbtbxjKL74hgkDW3ora03\n19AX/ULVPtyj6IEqgBvreemrqDD5XXTpZPahTN23NUUW3g9AlUYwAYAK4Ik2/hoZ4W03tuWvXD22\n3cxtLChWWp5Vb+9PV+uv4vTMj6k6nVX0CpubQbo1wls7h9bQx71D1CrE3QWVXp4etT21rH+Ygjzs\nw8nC37I19rtk5Ra3+yOAKoFgAgAVgMFg0Dtdg9Wlhv1fsucfydK7v2S4qCpURCm5Vr32c5paf3VG\nL+5OU2JO0UDibpTGXuuj3cNralaPEF0TVPEDyYXah3to1cBw1fS2/5iy4niORq9PUmY+tzkCVRHB\nBAAqCE83gz7rG6KG/vY32U/elaaVx9l0rrpLyC7Qi7tS1erLM3ptb7rMeUWvHHi7GTSpua/23lJL\nb10frIb+lbf5ZvNgd60eGK56fvbvh42nczV8LWuwgKqIYAIAFUiYl5sW9QtVwAW3sdh0dl+HvYl5\nrisMLnMqs0BP7zSr9VdxejsmQxmWooHEz2TQw638tG9ETb3WOUhX+boVc6TKp3GASdEDw3VNoH3A\n2hGfp5uiE5WQXXQ9DYDKi2ACABXMtUHumt87RG4X3GKfZbFp9IYknc7kg1h1cSzdood/SFG7xWc0\n62CmsotZWxHoYdDTbf0VM7KWXuwQqBreVSOQXOgqXzetGhim1hetj4lJztfA1Yk6xXsCqDIIJgBQ\nAfWq46XpXYLsxv7KsupW7q+v8o6Y8zVpS7LaL4nTvCNZyivmnzvMy6gX2wcoZkQtPd0uQMGeVfs/\n52FeblreP6zIGqzYVIv6r0pg3x+giqjav8kAoBIb29RX97XwtRvbn5yve7aksKdDFRSTnK+xm5LV\n+et4LfwtW8U1n6rjY9TUToHaP6KmHm7trwCP6vOf8SBPo5bcEKo+dez3XjmZUaABqxJ0MCXfRZUB\nKC/V5zcaAFRCr3QIVP96XnZj357I0Yu70lxUEcrb7oQ83bo+Sd2XxevrY9kqLnLW93PT29cH6edb\nakDwHigAACAASURBVOneFn7yMVXP/3z7uhv1Rb9QDa5v/56Iy7Zq0OoE7U5gHRZQmVXP32wAUEm4\nGQ2a0zNYLS+6v/5/v2Ro/pFMF1WF8vDDmVwNXZOovisTFH0yp9jHNAk0aWb3YO0eXvP/2rvzuKjq\nvQ/gn3NmhoEZGHa03EU0TSQzESzczdxLS7N701xKKRXN8mb4qGU9Zm5hhhsZlkuGbTe36toTKZiZ\niVJaUlnZFWVnhoEBzsx5/kAGhsUVODB+3q+XL5jfOWfmO3Co85nfcvBEJz20KqHG/W4lWpWA+P4+\neLTKfX9yi2WMPpCFQ+nFClVGRDeLwYSIqJFz14h4f6APmle5p8OzyXlIvMCLsKZElmV89V8Lhu7L\nxPD9Wfi/Wn5/d3qr8U4/b3z7YAAmdNBBIzKQVKYWBcRGeOPJzo5DHQskGQ9/mYUD57m8NlFTxGBC\nRNQEtHRXY+cgX7hV+sRckoGJ/5eNtHyOrW/sbLKMvX8WYcCeTIz5IhtHLtU85KiHnwY7B/rg8OgA\nPNROBxUDSa1EQcDrvTzxXDcPh/ZiK/DPgzn48PdChSojohvFYEJE1ER093PBxj7eDm35JTLGfZmN\nHAuXTG2MrDYZH/1eiPs+zcA/vsrBiayaQ2TvZi74+H5f/GeEP4a2doMgMJBcC0EQsLCHAS/fY3Bo\nl2RgWmIu4n/hcEeipoTBhIioCRnV1g2LezhehJ0zWfHPr3JQXNMyTqSIUpuMHWlm9Po4A1MSc3E6\nt+blbAe20GLfUD/sG+aP/i1cGUhu0OxgD6wJ90Lln54MYE5yHt5MNSlVFhFdJ/XVdyEiosZkTrA7\nfjVK2J5WMVQl+VIJ5iTnIfY+L17cKqjYKmN7WiHeSDXhr4Lae7GGtXbFc908cLe/S6370PWZfIce\nHi4Cpn+T67DU8v98b0R+qYzo7h782yBq5BhMiIiaGEEQsCbcC3+YJCRdrJirsPPXQgR5qvFslTH3\nVP/MpTZsPVuIN380Ib2w5htgCgDGtHPDs908cGeVVdaobjzcXge9WsATX+eguFIuXHnSBFOJDct6\neUJkOCFqtDiUi4ioCXJRCXivvw8CDSqH9pePG/HpH1yRqKEYS2xYfcqEkN2X8OJ3+TWGEpUAPNZB\nh+/GBODtfj4MJfVsaGs3fDDID3q1YwDZeMaMmYfzINk45JGosWIwISJqonxcVdg1yBdeLo4XYNO/\nycEPvNFcvcottuF/TxgRnHARLx83IstSPZC4iMCUTnocH9sMsRHeCPJkIGkofW/X4tMH/Kr9bez4\ntRBTvuZ8LKLGisGEiKgJ6+CpwbsDfFH5w2GLFZhwMBvnC2qecE03LqPIisXH8hH8wUW8nmJCfkn1\nC1w3lYDILnqkPNwcq3t7oa0HR00r4R5/F+wZ6o+AKvf/+fefFjx2MBuFUs1D7ohIOQwmRERNXJ/b\ntFjT28uh7VKRDY/+JxumUl583SibLOO/Ziu+SS/G1l/MmJOUi24JFxHzYwEKpOqBxEMj4Nlu7jj1\nSDMs6+WF2/WqGp6VGlJXHw32D/VHyyq/i4P/LcbYL7KRX8K/D6LGpEE/xrHZbFi+fDnS0tLg4uKC\n6OhotGrVymEfi8WCmTNnYuHChWjbtm1DlkdE1GQ93lGPX/MlxPxYYG/7KVfCtK9zsGOgL2/UVwub\nLCO90IbfjRJ+N0r47fLX340SzpmsKLqGIT9eLgIi73TH9M7u8NLy877GJtBTjQPD/PDg59n41VjR\ni3jkUglGHcjCh/f7ws+VIZKoMWjQYJKYmIiSkhJs2bIFqampiImJwcqVK+3bT58+jddeew0ZGRkN\nWRYRkVNYfI8Bvxkl7PnLYm/7/O9iLDyWj2W9vK5wpHOTL4eP34wSzpkk/JZ/OYCYJJwzXlv4qIm/\nq4iZXd0x5Q49PDQMJI1ZS3c19g/zw5gvspGaU3GTy5PZpRi2LwufDPFjDxdRI9CgwSQlJQXh4eEA\ngODgYJw5c8Zhe2lpKVasWIHFixc3ZFlERE5BFARs7OONYfuzcDK74uJr/Wkzgjw1mHKHXsHq6ld5\n+PjddLnnI78sePxmvLnwUZMWOhVmB7tjYkc93NTsiWoq/N1U+OwBP4z/TzaOZlQsDnE2X8ID+zLx\n6RA/tDNwPhCRkhr0L9BsNsPd3d3+WBRFSJIEtbqsjJCQkBt63rS0tDqp72Y0hhrIefH8ouuxrL2A\nJ0xaZJRUfIr//JFcaIwXEeZdfUx9Uzm/ZBnIKhHwl0XA30UC/rKIOF8k4O8iEectAiy2ug8J7ioZ\nrdxsaO0mo6WrjI56G+7zscJFNOHvc3X+ck6psZ1fKwKB50q0+C6voofkrwIrBn+WjnV3FiNQzxW7\nmpLGdn7RlQUFBV1xe4MGE71eD7PZbH8sy7I9lNyMq73J+paWlqZ4DeS8eH7R9QoCkHBbCYbuy0Lh\n5UnaVgiIPuuGL0b44w6vimVrG9v5JcsyLhbZ7PM9zl3++tvlOR+FNUw6v1kGFwGBBjXae6jR3qAu\n+96gQqBBDR+tyLuF34TGdn6V+3cHGVMTc7C30rDHrBIRkad1+HCwH+72d1GwOrpWjfX8ohvXoMEk\nJCQEhw4dwuDBg5GamorAwMCGfHkioltGiK8L4vp64x8Hc1B+KW8slTH+y2wcHOmv6GRfWZZxqcjm\nMNG8bM6HFeeMEsz1ET40QqXQUR5AVGhvUMOX4eOW46oWsLW/D545nItdv1XckDS3WMboz7Owc5Av\n7muuVbBColtTgwaTfv364ejRo5g6dSpkWcaiRYtw4MABFBUV4aGHHmrIUoiInN6w1m54uacB/3PM\naG/7s8CKfxzMwadD/OBaj/MjysOHvefDVN7zUf/hoyJ4qNHeQ4VAT4YPqk4tClgf4Q0PjYi4nytG\nc5hKZTz8RRbe7e+L+1u5Klgh0a2nQYOJKIpYsGCBQ1tNSwJv2LChgSoiInJuM+90x6/5EraeLbS3\nHc0owaykXGzq431Tzy3LMjLKez5MlZfbLQsfNd3r42Z5VO358FDZv/dzZfig6yMKAlaEecLgImD1\nqYqlti1W4LGD2djUxxtj2usUrJDo1sLlJ4iInJggCFgZ7oU/TFYkphfb2xN+L0IHTzXGXGWhrvLw\nUbHCVVmvR/n39Rk+2ntUzPcoDyMMH1TXBEHAoh6eMGhELDle0bsoycDUxFwUSDImdnTeFe2IGhMG\nEyIiJ6cRy8bTD96bibT8ihvMLTthgnsnFTp0kJFpqTrnw3r5JoMSTKV1Hz7c1RU9H4EGNdoZKno+\n/Bk+SAFzunnAw0XAc0fy7fOyZACzk/JgLLFhZlcPJcsjuiUwmBAR3QK8tCI+GOSLgXsykVNcsWTw\n4rMueO339HoNH5UnmpeHEYYPaoym3uEOD42IyEO5qHzrm4XHjDCVynjhLg+et0T1iMGEiOgW0c6g\nxrYBPhj9eRZKL2cTSRZuKpS4qwW0s/d8qCp9z/BBTdO4QB30agGTv85BSaXb/ixPMcFYYsP/hnry\nvCaqJwwmRES3kN7NtVh7rzciD+Ve8zF6e8+HqtKk87LwEeDG8EHOZ3gbNyQM9sVjB3McVpBbf9oM\nU6mMmN5eUIk874nqGoMJEdEtZkIHHdILrVh63GgfS6+393yUhY92HhU9HwwfdCvqe7srPh7ii0e+\nzEZ+SUU42ZZWiIJSGZv6eMNFxb8LorrEYEJEdAt6tpsHHmrrhuNpf+K+zm3RjOGDqJrQAC32DPXH\nmM+zkGmpGNf1yR9FMJfasHWAD3RqUcEKiZwL/5qIiG5R7QxqhBhsaK5TMZQQ1SLYR4P9w/zQUq9y\naP/yv8UY+0U2jJUnohDRTWEwISIiIrqCDp5l4STQ4BhOjlwqwagDWci2WBWqjMi5MJgQERERXUUr\ndzX2D/PHnd6Oo+BTsksxfH8W0gsZTohuFoMJERER0TUIcFNh71B/9PTXOLT/nCfhgb2Z+MMk1XIk\nEV0LBhMiIiKia+SlFfHxED/0vU3r0P5ngRVD92Xi57xShSojavoYTIiIiIiug7tGxK5BvhjaytWh\nPb3QhmH7spCSVaJQZURNG4MJERER0XVyVQt4d4APxrV3c2jPKbZh5IEsJF0sVqgyoqaLwYSIiIjo\nBmhEARv6eGNKJ71Du6lUxtgvsvDl3xaFKiNqmhhMiIiIiG6QKAhYFe6JOcHuDu0WK/DYwWx8cq5I\nocqImh4GEyIiIqKbIAgCltzjicU9DA7tpTZgSmIO3jtrVqgyoqaFwYSIiIioDszt5oGVYZ4ObTYZ\nmJWUh9ifChSqiqjpYDAhIiIiqiPTOrtjYx9vqATH9he/y8fyFCNkWVamMKImgMGEiIiIqA6ND9Rh\na38fuFS5ylp2woSFxxhOiGrDYEJERERUx0a0ccOuQb7QqR27Tt76qQBRyXmw2hhOiKpiMCEiIiKq\nB/1buOLj+31hcHEMJ++eLcST3+SixMpwQlQZgwkRERFRPenVTIs9D/jBz9Xxkuujc0X451fZKJIY\nTojKMZgQERER1aNuvi7YP8wPLXQqh/Yv/i7Gw19mwVhiU6gyosaFwYSIiIiongV5arB/uB/aeziG\nk6SLJRj9eRZyLFaFKiNqPBhMiIiIiBpAa3c19g/zRxdvtUP7iaxSDN+fhYuFDCd0a1NffRciIiIi\nqgvNdCrsHeqPR77MwveZpfb2M3kSBnyWgdAALZrrRNymU+E2nQrNdSrcphPRXKeCh4afJ5NzYzAh\nIiIiakDeWhEfD/HDY//JxqGLJfb2C4U2fPJHUa3HuasFNNep7MGleXlwcRMvBxgVmulE6NQMMNQ0\nMZgQERERNTAPjYiEwX544uscHDhvuaZjCiQZvxol/Gq88n6eLkJFcHGrEmIu9740d1PBpert6YkU\nxmBCREREpABXtYD3BvjgX9/mY+tZM+rqtib5JTLySyT8nCddcT8/V9Ghx6W5ffhYRZjxdxWhFhlg\nqGEwmBAREREpRCMKWN3bC/Pv8sAveRIuFllxsdCK9EIrLhbaKr4vsqK4jufGZ1lsyLLY8OMV9hEF\nIMBVrNbjctvlXpfyEOPrKkIUGGDo5jCYEBERESms/MK/NrIsI69EvhxYag4u5Y/r8p6NNhm4WGTD\nxSIbkF1a635qAfb5L83dVLhNXx5exEqT+FXwdBEgMMBQLRhMiIiIiBo5QRDgrRXgrRXRxVtT6342\nWUa2xVYRXIqsuGC+HGaKyoLLxUIrMopsqMt7zksy8LfZir/NVgC1BxhXFWrscanaI8MVyG5NDCZE\nRERETkIUBPi7qeDvpkI339r3k2wyMooce1zSCyuCS3mwyS6u27vSW6zAHyYr/jBdeVzataxAll8K\nWCQZWhXYC+MkGEyIiIiIbjFqUcDtehVu19c+fAwAiq0yLtnnvVQJLkU2pJutSC+ywlhSl/0v17oC\nmQ44egECAJ1agE4twK3SVzdVpe/VAnSqyttFe5tO47hvjcerBKi4CEC9YzAhIiIiohppVQJau6vR\n2v3Kl4zmUhsuFdmqz4EpslZqs6GwLifAXCYDMEsyzPXw3JVpVagSYMRrDEHV23TlwajK8Rrx1u79\nYTAhIiIiopui14horxHR3lD7paUsyzCVyhW9Lw24AlldKLaW9SDl2XuH6r5IlYArhp2K3h2xor2m\nfau0ebqIV1xcobFgMCEiIiKieicIAgwuAgwuIjp61b5f5RXI0msKLpcfG4slFMsCSup2GoyirDJg\nKi0LcHUpLMAFB4b71+lz1gcGEyIiIiJqNK51BbK0tDQEBQVBsskossooLL38VZJRJFV8dWyzObRd\neV/HtqbMTd00hoc1aDCx2WxYvnw50tLS4OLigujoaLRq1cq+/dChQ4iLi4NKpcKoUaPw4IMPNmR5\nRERERNTEqEUBHqIAj9ozzE2TZRkWK1Ao2RyDi/XqwaawPBBV3VeSUWit+N4sybDWU/5hMKlBYmIi\nSkpKsGXLFqSmpiImJgYrV64EAEiShDVr1iA+Ph5ubm6YNm0aIiIi4Ot7hbXuiIiIiIjqmSAIcFMD\nbmoV6vPKtNQmw1xaPfCUhSBbrcHGXLmthsDU4iqrrzUWDRpMUlJSEB4eDgAIDg7GmTNn7NvOnTuH\nli1bwmAwAABCQkJw4sQJDBo0qCFLJCIiIiJShEYU4KUVcIUpOE6tQYOJ2WyGu7u7/bEoipAkCWq1\nuto2vV6PgoKCa3retLS0Oq/1ejWGGsh58fyi+sTzi+oTzy+qTzy/mpagoKArbm/QYKLX62E2m+2P\nZVmGWq22byssLLRvM5vN8PDwuKbnvdqbrG/lk6+I6gPPL6pPPL+oPvH8ovrE88v5iA35YiEhIUhO\nTgYApKamIjAw0L6tXbt2OH/+PPLz81FaWoqUlBQEBwc3ZHlERERERKSQBu0x6devH44ePYqpU6dC\nlmUsWrQIBw4cQFFRER566CHMmTMHs2fPhizLGDlyJAICAhqyPCIiIiIiUkiDBhNRFLFgwQKHtrZt\n29q/j4iIQEREREOWREREREREjUCDDuUiIiIiIiKqCYMJEREREREpjsGEiIiIiIgUx2BCRERERESK\nYzAhIiIiIiLFMZgQEREREZHiGEyIiIiIiEhxQl5enqx0EUREREREdGtjjwkRERERESmOwYSIiIiI\niBTHYEJERERERIpjMCEiIiIiIsUxmBARERERkeIYTIiIiIiISHEMJkREREREpDi10gU0ZTabDcuX\nL0daWhpcXFwQHR2NVq1aKV0WOQlJkrB06VJcuHABpaWlmDJlCvr06aN0WeREcnJyMHHiRKxbtw5t\n27ZVuhxyIvHx8fjmm28gSRLGjh2L0aNHK10SOQlJkrBkyRKkp6dDFEVER0fzv19OhD0mNyExMREl\nJSXYsmULnnnmGcTExChdEjmR/fv3w9PTE5s3b0ZMTAxWrFihdEnkRCRJwrJly6DVapUuhZzM8ePH\ncerUKcTFxWHDhg24dOmS0iWRE0lKSoLVasXbb7+NadOmYf369UqXRHWIPSY3ISUlBeHh4QCA4OBg\nnDlzRuGKyJkMHDgQAwYMAADIsgyVSqVwReRMYmJiMGbMGGzdulXpUsjJfPvtt+jQoQPmz58Ps9mM\nWbNmKV0SOZHWrVvDarXCZrPBbDZDrealrDPhb/MmmM1muLu72x+LoghJkvhHQnVCp9MBKDvPFixY\ngBkzZihcETmLPXv2wMvLC+Hh4QwmVOfy8vJw8eJFrF69GhcuXMC8efOQkJAAQRCULo2cgE6nQ3p6\nOh555BHk5+dj9erVSpdEdYhDuW6CXq+H2Wy2P5ZlmaGE6tSlS5cQGRmJoUOH4oEHHlC6HHIS//73\nv/Hdd99hxowZOHv2LJYsWYKsrCylyyIn4enpibCwMGg0GrRp0wYuLi7Izc1VuixyEjt27EBYWBg+\n/PBDbN++HS+99BKKi4uVLovqCIPJTQgJCUFycjIAIDU1FYGBgQpXRM4kOzsbs2bNwsyZMzFq1Cil\nyyEnsmnTJmzcuBEbNmxAx44dsWTJEvj5+SldFjmJkJAQHDlyBLIsIzMzExaLBZ6enkqXRU7CYDDY\nR6sYDAZIkgSbzaZwVVRX+PH+TejXrx+OHj2KqVOnQpZlLFq0SOmSyInEx8fDaDRiy5Yt2LJlCwDg\njTfegKurq8KVERHVLiIiAidOnMATTzwBWZbx/PPPc44c1ZkJEyZg6dKlePLJJyFJEiIjI+Hm5qZ0\nWVRHhLy8PFnpIoiIiIiI6NbGoVxERERERKQ4BhMiIiIiIlIcgwkRERERESmOwYSIiIiIiBTHYEJE\nRERERIpjMCEiqgMvvfQSQkNDa/133333KVLX6NGj63Up84sXL2LkyJHIzs6ut9e4mtDQUKxfv16x\n1weATz75BKGhobhw4UK9PL8kSQgNDcWmTZuu+ZjY2FisWLGiXuohIqoPvI8JEVEd8fb2xsqVK2vc\nJorO9zmQLMtYunQpHnnkEfj6+ipdDlUxefJkjB07Fn379kVoaKjS5RARXRWDCRFRHdFoNAgODla6\njAaTmJiIM2fOYNWqVUqXQjVwc3PDhAkTsGbNGuzYsQOCIChdEhHRFTGYEBE1sBkzZiAgIABt2rRB\nQkICiouL0aNHD8ydOxctWrSw73fmzBls2LABp0+fhiRJuOuuu/DMM8+gQ4cO9n2ys7MRGxuLpKQk\nWCwWBAUFITIyEnfffbd9H6vVitjYWOzZswcmkwl33HEH5s2bhzvuuAMAUFxcjDVr1iApKQk5OTlo\n1qwZ7r//fkybNg1qde3/m4iPj0f//v3h6upqbxs9ejQGDx6M4uJi7N27F6Io4t5778XcuXPh5eVl\n3+/UqVNYv349fvrpJ2g0GoSHhyMqKgr+/v4AgOPHjyMyMhILFixAfHw88vLysHTpUvTp06fGWoqK\nirBs2TJ89dVXKCkpQffu3fH888/bf54vvfQSjh07hj179tiPOX/+PMaOHYtFixZhxIgR9teMjY3F\ntm3b8MMPP8DFxQUDBw7EnDlz7HeXttls2LJlCz799FPk5eUhLCwM3bp1c6hn06ZNOHDgAIYPH46d\nO3dCpVJh+/bt8PPzw2effYYdO3bgr7/+gre3N4YOHYqnnnoKGo3Gfvz333+P2NhYpKWloXnz5pg3\nb1619/zFF19g69at+Ouvv6DVatGjRw88/fTTaNOmjX2fIUOG4K233sKhQ4dq/dkRETUWzje2gIhI\nQZIk1fjPZrM57Hf48GHs27cP8+bNwwsvvICzZ88iMjISRUVFAMouTKdOnQpJkrBw4UJER0cjIyMD\n06ZNw++//w4AsFgseOqpp3D06FE8/fTTWL58OQwGA6KiovDrr7/aX+vgwYM4c+YMFi5ciEWLFiE9\nPR3z5s2D1WoFAKxevRrJycmYOXMm3nzzTQwfPhzx8fHYunVrre/zzz//xOnTpzF48OBq2z766COk\npqZi8eLFmDlzJpKSkhAVFWX/GZw8eRIzZsyAKIp49dVX8dxzz+Gnn37C9OnTUVBQ4PBcGzduxKxZ\ns7BgwQKHsFXVBx98ALPZjFdffRXPPvssTp48iYULF17pV1Wr6OhodO7cGatWrcJjjz2GTz75BHFx\ncfbta9euxdtvv41Ro0bh9ddfh8FgqHGOS3p6OhITE/HKK69g7ty58PPzw3vvvYelS5eie/fuWLVq\nFSZMmIBdu3Y5zAP6+eefMXv2bOj1erz22msYP358tXlCJ0+exOLFi9G3b1+88cYb+Ne//oWzZ89i\n7ty5kGXZvl9AQACCg4Oxf//+G/pZEBE1JPaYEBHVkYyMDPTu3bvGbZMnT0ZkZKT9cVFREd599120\nbNkSANC2bVs8/vjj+OyzzzBu3Di89dZbaNGiBdauXQuVSgUA6NWrFx566CFs2LABr7/+Ovbs2YPz\n589j69at6Ny5MwDg7rvvxsSJE3H8+HF7z4qvry9WrVoFFxcXAIDRaMTy5ctx7tw5dOjQASdOnEBo\naCiGDBlifw6dTgdvb+9a3+uxY8cAAF26dKm2TRRFrFu3Du7u7gAALy8vzJ8/H0lJSYiIiMC6devQ\nsmVLxMTE2Htk7r77bowZMwa7d+/GE088YX+uMWPGYNCgQVf5yQMdO3bEK6+8Yn98/vx5vPvuuzCb\nzdDr9Vc9vrIRI0Zg+vTpAICePXvi2LFjOHToEGbNmgWTyYQPPvgAjz76KJ588kkAQHh4ODIzM/Ht\nt986PI/VakVUVBR69OgBACgoKMDmzZsxcuRIzJ8/HwAQFhaGgIAAREdH49SpU+jWrRvi4+Ph7e3t\n8DszGAwOQSslJQVarRaTJk2CVqsFADRr1gyHDx9GYWGhw3vu3Lkz9u3bd10/AyIiJTCYEBHVER8f\nH6xevbrGbeVDlMoFBwfbQwkAdOrUCS1atMAPP/yAkSNH4vTp05g8ebI9lACAh4cHIiIicOjQIQBl\nF6fNmze3hxKgbJ7Lzp07HV6rS5cu9gtcAPbhTUajEUDZxXdCQgIyMjIQHh6O3r17Y8KECVd8rxcu\nXIBer4fBYKi27d5777WHEgDo06cPVCoVTpw4gZ49eyI1NRWPPvoogLIeJqAsPHXq1AlHjx51CCYd\nO3a8Yh3l7rrrLofH5e/RZDJddzAJCQlxeBwQEIC///4bAPDjjz9CkiREREQ47DNgwIBqwQRwrD81\nNRUWiwV9+/a1v28A6N27N0RRxNGjR9GtWzekpKSgd+/eDr+z/v37O8wR6dGjB9avX48JEyagX79+\nCA8PR/fu3asNKQOA22+/HUajEQUFBQ6/FyKixobBhIiojqjV6hp7EGoSEBBQrc3b2xtGoxEmkwmy\nLNe40pWvr699uFNeXt4VezXKlc+NKFe+Qlj5kJ85c+YgICAA+/fvx5o1a7BmzRoEBQXh2WeftX/a\nX1VBQYH9k/qrvTdRFOHl5YX8/HwYjUbYbDbs2LEDO3bsqHZsq1atHB7rdLqrvr+a3mP5RXzVIXTX\novKcGaCs/vKfVX5+PgDA09PTYR8/P7+r1lV+7HPPPVfjvpmZmQDKAmPl+ThAWeCs/Jpdu3bF2rVr\nsWPHDiQkJGDbtm0wGAwYN24cnnzySYcQU/5+zGYzgwkRNWoMJkRECii/SK0sJycHXbt2hYeHBwRB\nqPHeIFlZWfYLVHd3d5w/f77aPj/++CNcXV0dJslfiUajwaRJkzBp0iRkZmYiOTkZ77zzDubPn48D\nBw44TMou5+XlVW0+SG3vzWq1Ii8vDz4+PtDr9RAEAePHj8cDDzxQ7djKvQR1SRAE+5yacuXzea5H\neWDIyclBYGCgvT0vL++qx5aHgiVLlqBt27a1PreXl1e1373VaoXJZHJoK79HTnFxMU6cOIGPP/4Y\ncXFxaN++vcPwt/LjqoYpIqLGhpPfiYgUcOrUKYeL2TNnzuDChQvo1asX3Nzc0LlzZxw8eNDhYrqg\noACHDx+2DzXq3r070tPT8csvv9j3kSQJ0dHRSEhIuKY6LBYLxo4di23btgEoG3I2evRoPPzw/bdB\naQAABDZJREFUwzCZTDCbzTUed9ttt6GkpAS5ubnVth05cgSlpaX2x9988w2sVit69eoFvV6PTp06\n4dy5c+jSpYv9X8eOHfHOO+/gyJEj11T39dLr9TAajbBYLPa2kydPXvfzdOvWDVqtFgcPHnRoP3z4\n8FWP7dq1KzQaDTIyMhzeu7u7O9atW4c//vgDQNnQuiNHjqCwsNB+bHJyssO58MYbb2DSpEmQZRla\nrRZhYWFYsGABgLKbXlaWkZEBb2/vaj1BRESNDXtMiIjqSGlpKVJTU2vd3q5dO/un5haLBVFRUZgy\nZQoKCwsRGxuLDh062HsRnn76aURFRWH27NkYP348SktLsXXrVhQXF9snXY8YMQK7du3C888/j+nT\np8PX1xe7d+9Gfn7+VeeIlHN1dUWnTp0QFxcHtVqNjh074sKFC9i5cyd69uxZbUhRuV69egEom+fS\nv39/h20ZGRmYN28exo8fj0uXLiE2Nhbh4eG455577O9t7ty5iI6OxtChQwEAu3btwvfff49x48Zd\nU93X67777sOuXbvwyiuv4MEHH8Rvv/12Q/f20Ol0mDx5MjZu3Ai9Xo+ePXvi8OHDSEpKuuqxXl5e\nePzxx7F582YUFhbinnvuQU5Ojv1xp06dAABTpkzB119/jaioKEycOBG5ubnYsGGDw3yjnj17YufO\nnVi8eDGGDRsGWZaRkJAArVZbbVngkydPIiws7LreJxGREhhMiIjqSG5uLqZOnVrr9vXr19vnbAQH\nByM8PBxLly4FUDZBfPbs2fZhU6GhoVi3bh02bdqE6OhoaDQadO/eHYsWLbIP0dLr9di4cSPefPNN\nrFmzBlarFZ07d0ZsbGyNQ4VqEx0djY0bN2Lnzp32oWL9+vVzWEWsqttvvx133nknkpOTqwWTgQMH\nwsfHBwsXLoRWq8Xw4cMdnissLAxr165FXFwcXnzxRXsgiomJQc+ePa+57uvRq1cvREVF4f3338fX\nX3+Nzp07Y8WKFZg0adJ1P9eUKVOg0+nw/vvvY9euXQgJCUFUVBRef/31qx47Y8YM+Pn5Yffu3di+\nfTs8PDzQo0cPzJgxwz6nqE2bNli/fj3Wrl2LF198Eb6+vpg7d67D8997771YunQptm3bhhdeeAFA\n2SIHb731Flq3bm3fLzMzE2fPnrWHWSKixkzIy8uTr74bERHVlRkzZsBqtWLz5s1Kl3JTEhMTsXjx\nYuzdu9e+8tXo0aMREhKCl19+WeHqCADi4uLw1VdfYfv27bzzOxE1epxjQkREN6Rv374ICgq65vks\n1LDMZjM+/PBDzJo1i6GEiJoEBhMiIrphS5Yswe7du5GVlaV0KVRFfHy8/R4nRERNAYdyERERERGR\n4thjQkREREREimMwISIiIiIixTGYEBERERGR4hhMiIiIiIhIcQwmRERERESkOAYTIiIiIiJS3P8D\n6AzjCAETRmsAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "The training accuracy rate: 100.00%.\n", "The test accuracy rate: 64.00%.\n" ] } ], "source": [ "# train NN with mini-batch and RMPSProp\n", "parameters = model(X_train, Y_train, layers_dims, optimizer=\"rmsprop\",\n", " learning_rate=0.001, mini_batch_size=64,\n", " beta1=0.9, epsilon=1e-6, num_epochs=1000,\n", " activation_fn=\"tanh\")\n", "\n", "# print the test accuracy\n", "print(\"The training accuracy rate: {}\".format(\n", " accuracy(X_train, parameters, Y_train, \"tanh\")[-8:]))\n", "print(\"The test accuracy rate: {}\".format(\n", " accuracy(X_test, parameters, Y_test, \"tanh\")[-7:]))" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyYAAAGkCAYAAADE0DHIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4FGW2P/Bv9Zqlk0BIICQBAgFkDQiiICCBEQVlRtzA\nbUYFWdzBmXl+MnoZ9Y54cVTEUUFRRByvuOECI0E0IaCAGhAIJGBYs5FAEpJ0Okmv9fuDm+6u6qyQ\n7url+3meeR67urtykhSZOv2e9xyhurpaBBERERERkYJUSgdARERERETExISIiIiIiBTHxISIiIiI\niBTHxISIiIiIiBTHxISIiIiIiBTHxISIiIiIiBTHxISIqAVvv/02rrzySmzevFnpUDrFTTfdhClT\npigdRqeoqKjApk2blA6jVRkZGfjjH/+Ia665BjfccANefPFFGI3Gdr/fbrfj448/xuzZszFx4kTc\ndNNNePPNN2E2m5t9/e7du/HAAw9g8uTJuO6667B06VKUl5e3+XU++eQTXHnllR2KjYjIG5iYEBGF\niDvuuAP33nuv0mFcsqqqKtx+++3Izs5WOpQWrV+/HkuXLoXNZsOsWbMwZMgQfPbZZ1iwYAEaGxvb\ndY6XXnoJL7/8MgwGA2bPno3ExESsW7cOf/nLX+BwOCSv3bZtGxYtWoRz587h5ptvxtixY7Ft2zbM\nnTsXlZWVLX6Nffv24V//+tclfa9ERJ1Fo3QARETkG3feeafSIXSKxsZGmEwmpcNoUVlZGVatWoWh\nQ4fi7bffhlarBQB8+OGHWLlyJT788EPMnTu31XMcPHgQn3/+OdLT07F8+XIIggDgQrLyySefYOvW\nrZg+fToAoKGhAcuXL0dCQgL+/e9/IyoqCgCQnp6OJ598EqtWrcLTTz/t8TW++eYbvPDCCy2uwBAR\n+RpXTIiIiDrRxo0bYbfbcf/99zuTEuDCilV8fDy+/vrrNs/x6aefAgDmzZvnTEoAYOHChdBqtfjq\nq6+cx7Zu3Yra2lrcddddzqQEAKZMmYKhQ4fi22+/lazSnD17FosXL8YzzzyDuLg49OrV65K+XyKi\nzsIVEyKiTiKKIr788kt88cUXOHnyJLRaLdLS0vDAAw9g2LBhktfabDZ8/vnn2Lp1K06ePAmz2YzY\n2FhcddVVWLBgAbp37+587ZVXXolp06YhNTUV69evh8PhwH333YepU6di5syZuP/++5GWloZ3330X\nBQUFCAsLw/jx4/HII48gLi7OeZ6bbroJRqMRmZmZAIDNmzfjueeew2uvvYYTJ05g48aNOHPmDOLi\n4nDDDTdgzpw5khtr4MJN96effori4mJ069YNt9xyC7p164Znn30Wq1atwujRo1v8+bz99tt45513\n8K9//QtvvfUWjh49ih49emDdunWIjo7G8ePHsX79euzbtw9VVVXQ6XTo27cvbr/9dufqQFPMALBj\nxw5ceeWVeOCBBzB//nwAF8q83n33XezYsQNVVVWIi4vDpEmT8MADDyA6OrrV319paSlmzpzZ1q8Z\nS5cuxYwZM1p8/tdffwUAj5+FWq3GyJEjsW3bNpw5cwY9e/Zs9RwxMTEYMGCA5LjBYMBll12GgwcP\nwmazQaPROL/eFVdc4XGe0aNH4/Dhw8jLy8OoUaMAAHl5edi9ezdmzpyJxx57DH/5y19QVFTU5vdN\nRORtTEyIiDrJP/7xD2zatAl9+/bFzTffDLPZjO+//x7z58/HCy+8gEmTJjlf+/TTTyMzMxNpaWmY\nOXMmrFYrcnJysGnTJhw4cAAbNmyARuP6E/3zzz9j586duPHGG1FdXS1JdHbv3o33338fV199NWbN\nmoW9e/fim2++wYkTJ7B+/fo2437zzTdx6tQp/O53v8OECRPw3Xff4d1330VDQwMWLVrkfN0rr7yC\nDRs2IDk5GTNnzkR1dTVWrVqFhISEDv2c/v73v6N3796YPXs2qqurER0djcOHDztXAyZPnozY2Fic\nOXMGWVlZ+Pvf/w6Hw4Ebb7wRAwcOxB133IENGzagT58+mDp1qjMBKC8vx7x581BeXo7x48ejX79+\nOH78ODZs2IA9e/bgnXfeaTU5iYqKwgMPPNBm/AMHDmz1+ZKSEsTGxiIyMtLjuaZkpLCwsMXExGKx\n4OzZsxg6dGizz/fs2ROHDh1CaWkpevfujeLiYgBAUlJSq1+vKTHp378/NmzYgJSUlFa/DyIiX2Ni\nQkTUCTIzM7Fp0yZce+21eO6555xJxZw5c3D//ffjv//7vzFmzBhEREQgNzcXmZmZmDp1Kp5//nnn\nORwOBxYsWIADBw4gLy8PaWlpzueqqqqwfPlyTJ482XmstLQUAHDkyBH84x//wHXXXQfgQjenefPm\n4dChQzh8+HCLN7hNioqK8MEHH6BPnz4AgD/+8Y+49dZb8fXXX+ORRx6BRqNBXl4ePv74YwwbNgyv\nv/46IiIiAADXX389nnjiiQ79rOLj47Fq1Sqo1Wrnsbfeegs2mw3r1q1Damqq8/iePXvw2GOPISMj\no9nEpGmlBACWL1+O8vJyvPjii5Ik8KuvvsLzzz+P119/HX/7299ajCsqKkpyvotVU1ODxMTEZp8z\nGAwAgLq6uhbfX1tb64ynPeeoqamBTqdDWFhYu75ecnJyW98CEZEiuMeEiKgTNNX8P/HEE5KVjh49\nemD27Nmora3Fjh07AADdu3fH0qVLsXDhQsk5VCoVLr/8cgAXbjbd6fV6TJw4sdmvnZiY6ExKgAsl\nQ2PHjgUA56fprZkyZYozKQGAbt26YdCgQairq0N1dTWACxulRVHEgw8+6ExKAGDChAm46qqr2vwa\n7tLT0yVJCXBhY/6zzz4rSUoAVzmU/OchV1FRgR9//BFXXXWVJCkBLpSw9enTB1u2bIHNZutQrBfD\nZrN5lMA1aTpusVhafT8A6HS6dp2jta/XdA5ucCeiQMAVEyKiTpCfnw+tVouNGzd6PFdYWAgA+O23\n3zBt2jT06NEDM2bMgM1mw9GjR1FYWIji4mL89ttv+OWXXwDAox1s9+7dJQmPu969e3sca/qk3Gq1\nthl7e96fl5cHAM2uvqSlpeGnn35q8+s0aa7kaNy4cQCAyspKHDt2DMXFxTh9+jQOHjwIwPPnIXfk\nyBGIogij0Yi3337b43mVSgWz2YzTp097JD9NjEYjPvroozbjT09Pb7WcS6/Xt5gANf08m1vdcH+/\n+2tbOkd4eHibX68peWl6LRGRP2NiQkTUCYxGI+x2O955550WX+P+qf+XX36Jd955B2fPngVwIREY\nMmQIUlNT8euvv0IURcl7W7uRbe6TdfdOTm1p6ZN5AM44ampqEB4eLlktaRIfH9/urwU0/72Ul5fj\n5ZdfRnZ2NkRRhEqlQnJyMsaMGYO8vDyPn4dcU6nS4cOHcfjw4RZf11Qm1Ryj0djq769JYmJiq4lJ\ndHR0i6VaTcebEr/mGAwGqFSqdp8jOjoaJ0+ehMVi8fhdtufrERH5CyYmRESdICIiAuHh4e2aEv/9\n999j2bJlSE1NxZ///GcMGjTIuUn5jTfecHZZ8ieRkZEoKSlxdoJyd6kzRURRxOLFi3HixAn86U9/\nwuTJk9GvXz+EhYXBYrHg888/b/McTSsCc+bM8SiRa6/ExET8/PPPF/Ved71798bevXvR2NjokYQ1\n7Qvq27dvi+/XarXo2bMnSkpKmn2+tLQUYWFh6NGjh/PrHThwAKWlpR4b2pu+Hje6E1Eg4B4TIqJO\nMGDAAJw7dw4VFRUez/3888948803nZ/kZ2RkAIBzM7t7d6aTJ08CQJsrBL42aNAgOBwOZ0mXu0OH\nDl3SuQsKCnDs2DFMnToVDz/8MIYMGeK8oT916hQA6c+judWgpra6zcUHAO+99x7ee+891NfXX1Ks\n7TFy5EiIouiRYNrtduzfvx89e/aUtHFuzogRI1BVVeX8/pvU1dXh6NGjGDp0qDNBHDlyJABg7969\nHufZt28f9Ho9Lrvsskv4joiIfIOJCRFRJ5gxYwZEUcTy5cslG5urq6vxP//zP1i3bp1zg3LTHoKy\nsjLJOTIzM/HDDz8AgE82aXdE09yON998Ew0NDc7jOTk52L59+yWd2/3n4Z6A1NXV4aWXXgJw4aa+\nSdMNufsejMTERIwaNQp79uzBtm3bJOfPysrCqlWr8N133zVbitbZrr/+eqjVarz99tuSTecbNmxA\nRUUFbrnlljbPceONNwK4sILmvr9m9erVsNlsuPnmm53HJk2ahMjISHzwwQfOZgXAhevp8OHDuOGG\nG5w/YyIif8ZSLiKiNrz//vstlmjNmDEDM2bMwI033ogffvgBmZmZuPPOOzF27FiIoojMzExUVVXh\n3nvvde5LmD59Or799lv8v//3/zB16lRERUXhyJEjyMnJQdeuXVFVVdVmFypfS0tLwy233IKNGzfi\nnnvuwbhx43D+/HlkZmYiKioK1dXVUKku7rOu3r17Y9iwYThw4ADmzZuHkSNHOruYmUwmREZGSn4e\nXbt2hV6vx969e/HKK69gzJgxmDhxIv72t79h/vz5eOqpp7Bp0yb0798fJSUl2LFjByIjI/HUU091\n1o+jVX369MGf/vQnvPfee7jnnntwzTXXoLCwENnZ2Rg0aBBmzZolef3mzZtRWlqKGTNmONsMjxkz\nBtOmTUNGRgbuv/9+516bnJwcjB8/Hr/73e+c74+Ojsbjjz+OZcuW4Z577sG1116LqqoqbNu2DT17\n9uyUFshERL7AxISIqA2nT5/G6dOnm32uaWidIAhYtmwZPvvsM2zevBlff/019Ho9UlJS8MQTT0ja\n+Y4fPx7Lli3D+vXrsXXrVuj1eiQmJuLxxx/HNddcg1tuuQW7du3Cbbfd5pPvr73++te/Ijk52Tnd\nPj4+Ho899hgqKirwwQcftLpBvzWCIOCf//wn3nzzTfz000/4+OOPER8fj9GjR2POnDn48MMPsXnz\nZuTn52Pw4MHQaDR48sknsXr1anz++edoaGjAxIkT0bt3b6xfvx7vvvsudu3ahb179yI2NhbXXnst\n5s6d69N9Fg8++CDi4+Px2Wef4eOPP0ZsbCxuv/12zJs3z6ND1ubNm7Fv3z6MHj1aMv9k6dKlSElJ\nwebNm7FhwwbEx8djzpw5uO+++zzaLc+cORNRUVFYv349Pv/8c0RFReG6667DQw89hG7duvnkeyYi\nulRCdXW1fxUyExGR36moqIBWq0VMTIzHc8888wy++eYbZGRkIDY2VoHoiIgoGHCPCRERtSkjIwNT\np071KGkrLi5GdnY2+vbty6SEiIguCVdMiIioTeXl5bjrrrvQ2NiIa665BsnJyaisrERWVhasVite\nffVVXHHFFUqHSUREAYyJCRERtUtRURHWrVuHnJwcVFZWwmAwYOTIkbjvvvswaNAgpcMjIqIAx8SE\niIiIiIgUxz0mRERERESkOCYmRERERESkOCYmRERERESkOCYmnaCgoEDpECiI8foib+L1Rd7E64u8\niddX8GFiQkREREREimNiQkREREREimNiQkREREREimNiQkREREREimNiQkREREREimNiQkRERERE\nitP48os5HA4sX74cBQUF0Ol0eOqpp9CrVy8AQEVFBZ5++mnna3/77Tc8/PDDuPXWW30ZIhERERER\nKcCniUl2djYsFgvWrl2L3NxcrFy5Ei+99BIAIC4uDqtXrwYAHDx4EKtWrcLMmTN9GR4RERERESnE\np6Vc+/fvx7hx4wAAw4cPR35+vsdrRFHESy+9hCeffBJqtdqX4RERERERkUJ8umJiMplgMBicj1Uq\nFWw2GzQaVxg7d+5Ev3790KdPn3af1x8mf/pDDBS8eH2RN/H6Im/i9UXexOsrsAwYMKDV532amERG\nRsJkMjkfi6IoSUoAYMuWLbjjjjs6dN62vklvKygoUDwGCl68vsibeH2RN/H6Im/i9RV8fFrKNWLE\nCOzatQsAkJubi9TUVI/X5OfnIy0tzZdhdRqHKCodAhERERFRQPJpYpKeng6dToe5c+dixYoVWLx4\nMTIyMvDFF18AAM6fP4/IyEgIguDLsC6JKIrYdV6FG745h1V5prbfQEREREREHnxayqVSqbBkyRLJ\nsZSUFOd/d+3aFR9++KEvQ7okh6qseHDneeRWhQGw4JTRhnmDIqFTB05iRURERETkDzhg8RIkRKhw\nrMbmfFxa78AnJ+oVjIiIiIiIKDAxMbkEcWFq/HFghOTYytw67jUhIiIiIuogJiaX6JFhBqgFVyJS\nUGPDfwobFYyIiIiIiCjwMDG5RL0NGkyLt0uOrThohMhVEyIiIiKidmNi0gn+mGSVPN5XYcWOMxaF\noiEiIiIiCjxMTDpBaqSI6b3CJMdezTUqFA0RERERUeBhYtJJFqcZJI+zSs3YX8FVEyIiIiKi9mBi\n0kmu7K7H1T10kmOv5tYpFA0RERERUWBhYtKJnkiLkjz+6lQDjtVYW3g1ERERERE1YWLSiX6XpMew\nWK3zsQjgtUNcNSEiIiIiagsTk04kCAIWD5fuNfnoWD1KTfYW3kFERERERAATk053U0o4UqLUzsdW\nB7Aqj6smREREREStYWLSyTQqAY8Nk+41ee+ICdVmh0IRERERERH5PyYmXnBX/wh0D3f9aOtsIt45\nYlIwIiIiIiIi/8bExAvCNAIeGiLda7LqcB3qbVw1ISIiIiJqDhMTL7l/UCSitYLzcaXZgX//Vq9g\nRERERERE/ouJiZfE6FSYOyhScuxfh+tgdYgKRURERERE5L+YmHjRwiEG6F0NulBUZ8fGkw3KBURE\nRERE5KeYmHhRjwg17u4vXTV59aARDpGrJkSd7VyDHYt3ncdd31fi57NmpcMhIiKiDmJi4mWPDTdA\n5dpqgvxqG7YWNSoXEFGQevKnGrx3tB7fFDbinswqGK1sNkFERBRImJh4WUqUBrf0DZccW3GwDiJX\nTYg6Tb3NgU2nXWWSZxscyCzhqgkREVEgYWLiA48Plw5c/PmcBbvLLQpFQxR8dpVZYJEtkGwp5H4u\nIiKiQMLExAeGx2oxNUkvOfZqrlGhaIiCT2apZ3nktmIz7OyCR0REFDCYmPjIojTpqsm3xWYcqrIq\nFA1RcNneTNlWpdmBX85xZZKIiChQMDHxkat76HBVd53kGFdNiC7dmXo78qptzT6XwUYTREREAYOJ\niY8IgoBFww2SYxtPNuCUsfkbKiJqn+2lLW9y31LIxISIiChQMDHxoet7hWFwF43zsUME/nWoTsGI\niAJfVknLycfRGhtO1jL5JyIiCgRMTHxIJQgeHbr+XWDC2Qa7QhERBTaHKGL7GemKSTe99M/aFpZz\nERERBQQmJj52a79wJEeqnY/NdmB1HldNiC7G4fM2nG1w9QmO0gp4TFYyyX0mREREgYGJiY9pVQIe\nHSa9cXon34Qa+RAGImrTdlkZ14QEPWb0lg403VVm5r8vIiKiAMDERAF/HBghKTeptYp474hJwYiI\nAlOmbOP7lCQ9UmM0GBDj2stlE4Hvi7lqQkRE5O+YmCggQqPCwiGRkmNv5tWh0cZhcETt1WATsbtc\nmphMTrwwyHR6rzDJcZZzERER+T8mJgqZN9gAg0ZwPj7b4MBHx+oVjIgosOwpN6PRrW9EL4MaqdEX\nVkqmyRKTb4sbYeMUeCIiIr/GxEQhXfQq3HeZdNVk5SEjb56I2klexjU5UQ9BuJDsX9ldh656V+Jf\nbRHx01lOgSciIvJnTEwU9NBQA7Ruv4FTRju+PtWgXEBEASRLvr8k0bVKolEJmJrMci4iIqJAwsRE\nQYmRatzZP0Jy7JXcOogiV02IWnO2wY5DVVbnYwHANT11ktdwnwkREVFgYWKisMeGGSC4PT5UZcX3\nJeYWX09EwHbZasnlcVrEhqklx6YkhcFtGxcKamw4XsMp8ERERP6KiYnC+sdo8YcU6Se7K3KNCkVD\nFBgyZfNLmrpxuYvRqTA+QXp8SxFLJYmIiPwVExM/sGh4lOTxj2UW/HyWqyZEzRFF0WPFZHJSWLOv\nlXfnYjkXERGR/2Ji4gcuj9MhXfaJ76u5dQpFQ+Tf8qttKGtwTXKP1Ai4Ml7X7Gun95YmJrvLLag2\ncwo8ERGRP2Ji4icWy1ZNvilsRP55awuvJgpd8m5cExJ00KmFZl+bEqXBoC6uKfB2EfiuhKsmRERE\n/oiJiZ+4pqcOo+K0kmMrudeEyEOWLLFIT2y+jKsJy7mIiIgCAxMTPyEIgsdek89ONKCwjl2EiJqY\n7SJ+LJMOSpyS5Lnx3Z08MdlW3AgrB5kSERH5HSYmfmRGnzAMiHGVndhE4I1D3GtC1GRPuQUNdldS\nkRihwkC3fzPNGROvQze9609djUXEnnJOgSciIvI3TEz8iEoQ8Ngwg+TY+t/qUdloVygiIv+yvVTW\nJjgpDILQ/P6SJmqVgKnJbBtMRETk73yamDgcDrzwwguYM2cOFi5ciKKiIsnzeXl5mDdvHubNm4cn\nn3wSZnPotcydnRqBxAjXr6XBLuKtfJOCERH5j0x5m+Bm5pc0Z3rvcMnjjMJGiCLLuYiIiPyJTxOT\n7OxsWCwWrF27Fg8//DBWrlzpfE4URTz//PNYunQp1qxZg7Fjx6KsrMyX4fkFnVrAw8Oke03ezquD\n0coWpxTaKhrtOFgp7VQnb7PdksmJemjd/tqdMNpRwCnwREREfsWnicn+/fsxbtw4AMDw4cORn5/v\nfK6wsBAxMTH46KOPsGDBAtTW1qJPnz6+DM9v3DswAl10rvKUaouI949y1YRCW3apGe5rHGmxWsSF\nqdv13midChNkU+DZnYuIiMi/+DQxMZlMMBhceyhUKhVstgufWlZXVyM3Nxe333473njjDfzyyy/4\n5ZdffBme3zBoVZg/RLrX5I3DdTDbWXpCoUs+v6Stblxy8u5cW5iYEBER+ZXW29l0ssjISJhMrk/+\nRVGERnMhhJiYGCQnJ6Nv374AgHHjxiE/Px9jxoxp87wFBQXeCbgDOjuGqXpgpSocZseFlZMz9Q68\nvusk/pDAjfChyB+ucSWJIvDt6TC4f5YyUKxAQcHZdp9jkF0A4Npr8lO5Gb/kFaCLtuX3hIpQv77I\nu3h9kTfx+gosAwYMaPV5nyYmI0aMwM6dOzF16lTk5uYiNTXV+VxSUhLq6+tRVFSEXr16Yf/+/fjD\nH/7QrvO29U16W0FBgVdiuK+2WrLxfcO5SCwa3x1qVetdiCi4eOv6CiS/VVtx1uJKQsLVAm4d1Q/6\nFia+N2cAgCHHy5F3/sIqrQMCTuiTMDs1orPDDSi8vsibeH2RN/H6Cj4+LeVKT0+HTqfD3LlzsWLF\nCixevBgZGRn44osvoNVq8fTTT+O//uu/cO+996JHjx6YMGGCL8PzO48MM0Djdt9VUGPD5kKWn1Do\nkXfjGp+g61BS0mS6fAo8/z0RERH5DZ+umKhUKixZskRyLCUlxfnfY8aMwbp163wZkl/rZdDg9tQI\nfHSs3nns1Vwj/tCn7dkNRMFEvr+kvd245Kb1CsfLB11DS78vaYTFLkJ3EUkOERERdS4OWPRzjw+X\nboL/tcKKHWdCb74LhS6LXcQPZ+Qb38NaeHXrRsdrER/m+rNXaxWxu5z/noiIiPwBExM/N6iLFjf0\nlt6Ercita+HVRMHnl3MWmGyujnQJ4SoM7nJxi70qQcB17M5FRETkl5iYBIDFw6UDF7eXmvFrhUWh\naIh8K6vEs4zrUkoZ5W2DM4o4BZ6IiMgfMDEJAGO66zA+QSc5tuKgUaFoiHwrq1S6ojH5Isu4nO9P\n1EPn9pfvlNGOo5wCT0REpDgmJgHiiTTpqsmm040oqLEqFA2Rb5w3O7CvQnqdp/e8uI3vTQxaFa6R\nnYPduYiIiJTHxCRATEnUY3isaxKcCOA17jWhILfjjBnuRVZDu2rQI0J9yedtrpyLiIiIlMXEJEAI\ngoDFsg5dG47Xo9TESfAUvDJLpAnDxXbjkrtelpj8fM6Cykb+WyIiIlISE5MA8oeUcPSNcn1abHUA\nbx7mqgkFJ1EUPQYrTr7I+SVyvQwaDHNbgXSIwLfFbBtMRESkJCYmAUSjEvDYMOlek/eOmnDe7FAo\nIiLvOVFrR1GdaxVDrwbG9eicxARorpyrodPOTURERB3HxCTA3Nk/Aj3CXb82k03EmnyumlDwkXfj\nGtdDj3BN501ony5LTDJLzLDY2TaYiIhIKUxMAkyYRsBDQ6V7TVbnmWCyctWEgou8jGtKJ5VxNbk8\nTitJ8o1WET+WsZyLiIhIKUxMAtD9l0UiWuf65LjK7MC/C+oVjIioc1kdIn444zlYsTOpBAHXJUtX\nTb5hdy4iIiLFMDEJQNE6FR4YFCk59q9DdbA6WIZCwWHvOQtqra7rOT5MJdms3lk4BZ6IiMh/MDEJ\nUAuHGKB3G+dQbLLj8xPcvEvBIavUc7VEJXTe/hL387r/OyqqsyPvPKfAExERKYGJSYDqHq7GPQOk\nqyav5hrh4Ke9FASySrzTJlguUqvCJPkUeJZzERERKYKJSQB7dJgBarcPkY9U23hTRQGv2uzA3gqL\n5Fh6YucMVmzOtF7hksdsG0xERKQMJiYBLCVKg1v6Sm+qVhw0skaeAtrOMjPcu/YO7qJBYqS65Tdc\nIvkU+JxzVpxr4BR4IiIiX2NiEuAeHy4duPjLOSt2lVtaeDWR/9vezP4Sb0qKVGNEN9fGehHA1mKu\nPBIREfkaE5MANyxWi+uSpTdurx40KhQN0aXLLJEmBVOSvFfG1cSjO1chExMiIiJfY2ISBBbJVk22\nlZhxsJKrJhR4ThltOGl0lVHpVMDVPXRe/7ryKfBZpWY02lgSSURE5EtMTILA1Ql6jO0uvXlbmVun\nUDREF0/ejeuq7jpEar3/Z2pENy16Rri+jskm4gdOgSciIvIpJiZBYlGaQfL4i1MNOFnLeQwUWDJL\npSVUk31QxgUAgiDg+mTPYYtERETkO0xMgsR1yWEY0kXjfOwQL0yDJwoUNoeIHWekqxRTvLzx3d20\n3pwCT0REpCQmJkFCJQh4PE261+TDYyaU17PtKQWGXyusqLG4EoFYvQppbt2yvG1SzzCEuw0GKjbZ\ncYhT4InJODtiAAAgAElEQVSIiHyGiUkQuaVvOHoZXPMezHZgVR5XTSgwZMnKuNIT9VAJQguv7nzh\nGgGTZCs0GYUctkhEROQrTEyCiFYl4NGh0r0ma4+YUGNxKBQRUftl+Xh+SXPk3bm4z4SIiMh3mJgE\nmXsGRiAuzPVrrbWKWHvEpGBERG2rtTjwy1lpi+vJCiQm8inweyusLIckIiLyESYmQSZCo8LCIdJV\nk1V5dWjgTAbyYz+UmeF+iQ6I0aCXQdPyG7wkIUKNy+Ok+1o4BZ6IiMg3mJgEoQcGRcKgcdXmn21w\n4KNj9QpGRNQ6eRmXEqslTTymwLOci4iIyCeYmAShLnoV7h8UKTn22iEjbA6umpB/kg9W9KfEZDun\nwBMREfkEE5Mg9dBQA3Ruv91TRju+PMUOQ+R/CutsOOY2DFQjABN6KpeYpMVqkRTh6m5Xb/Ocr0JE\nRESdj4lJkOoZocad/SMkx1YcNHJgHPmd7bIyrjHddYjSKvenSRAEj03wW4qY1BMREXkbE5Mg9tiw\nKLhPgTh83obvSvjJL/kXeRmXL6e9t0RezrWVU+CJiIi8jolJEEuN0eCmlHDJsRUHjQpFQ+TJ7hCx\n/Yx0c/nkpLAWXu071/TUI8KtgURpvQMHKq0KRkRERBT8mJgEuUXDpa2Dd5Vb8FM5V03IPxyssuK8\n2bUSEaMTcHk3bSvv8I0wjeAx4JHduYiIiLyLiUmQGxmn8+hwtCK3TqFoiKQySzynvatVQguv9i22\nDSYiIvItJiYhYHFalORxRlEj8s6zLIWUl1UqK+NKVL6Mq8n1ydJY9ldacYZT4ImIiLyGiUkImJig\nw2jZNOuVudxrQsqqszrw01mL5Ji8fEpJPSLUuCJeNgWeqyZERERew8QkBAiCgEWyVZPPTjSgsM7W\nwjuIvG9XmQVWh+txvyg1UqI0ygXUjGm9pM0jtjAxISIi8homJiHixt5hGBDjuumzi8Drh7jXhJST\nKSvjmuIH3bjk5PtMsksbUW9ztPBqIiIiuhRMTEKEShDwuKxD1we/1aOikTXzpAz5YEV/KuNqMrSr\nBsmRrinwjXYgu5Rd7YiIiLyBiUkImdUvAkkRrpusBruI1XkmBSOiUFVisuNItauUUC0AE3v6X2Ii\nCAKmszsXERGRTzAxCSE6tYCHh0lXTdbk18FoZWkK+dZ2WRnXFfE6xOj888/RtN6eU+AdnAJPRETU\n6fzzToC85k8DI9BV75oTUWMRse4oV03It7Jk5VDyWTv+ZEKCHga3KfBlDZwCT0RE5A1MTEKMQavC\n/MHSVZM3D9fBbOcnwOQbDlH02F/iz4mJXi1gcpI0PnbnIiIi6nxMTELQ/MGRiHD7BPhMvQMfH69X\nMCIKJblVVlQ0usoHo7UCRsfrFIyobR5T4AuZmBAREXU2nw4NcDgcWL58OQoKCqDT6fDUU0+hV69e\nzuf/93//F1999RW6du0KAFiyZAn69OnjyxBDQrcwNf40MEKy8X1lrhF394+AWiW08k6iSydfLZnY\nUw+Nn1931yWHQQDQtK54sMqKEpMdSW4du4iIiOjS+HTFJDs7GxaLBWvXrsXDDz+MlStXSp4/cuQI\nnnnmGaxevRqrV69mUuJFjww1wG3RBMdr7djMT4HJBzJLpInJlCT/LeNqEh+uxhjZqg6nwBMREXUu\nnyYm+/fvx7hx4wAAw4cPR35+vuT5I0eO4P3338e8efOwbt06X4YWcpINGsxKjZAcW3HQCJHdhsiL\n6m0O7Dkr31/if4MVmyPvzpVR1KBQJERERMHJp4mJyWSCweDaeK1SqWCzuWYZTJ06FU8++STefPNN\nHDhwADt37vRleCFHPnBxf6UV2Wc4PI68Z3e5BWa3mZ59DGr0jQqMciiPKfBnzDCx1TYREVGn8eke\nk8jISJhMrn0NoihCo9E4//vOO+90Ji7jx4/Hb7/9hokTJ7Z53oKCAu8E3AH+EENHqQCkx+qwvcp1\nGTy/5yyShjM58TeBeH01Z+NJLQCt8/EoQyOOHTumXEAdoBGBRH0YSs0XPs8x24GP9p7CpG72Nt7p\n/4Ll+iL/xOuLvInXV2AZMGBAq8/7NDEZMWIEdu7cialTpyI3NxepqanO50wmE+644w588sknCA8P\nR05ODn7/+9+367xtfZPeVlBQoHgMF+upGAu2/+ec8/EvNWoYu/TBKD/vkhRKAvn6kvv1cDkA1yrp\nzME9MCAlXLmAOmhGVTXeznd9uLLf1hUPDOiqYESXLpiuL/I/vL7Im3h9BR+flnKlp6dDp9Nh7ty5\nWLFiBRYvXoyMjAx88cUXMBgMeOihh/Dggw9i/vz56NevH8aPH+/L8ELSmO46TEiQJiErco0KRUPB\nrKzejrzzrqREJQCTevr/xnd302XlXN8Wcwo8ERFRZ/HpiolKpcKSJUskx1JSUpz/fcMNN+CGG27w\nZUgE4Im0KPxQVul8vPl0I36rtmJgF20r7yLqGHmb4FFxWnTRB9YopfEJekRpBRitF5KRsw0O7Kuw\n4gquMBIREV2ywLorIK+YnKhHWqwrCREBvHaoTrmAKChllUrb66YHSDcudzq14NHemMMWiYiIOgcT\nE4IgCFicJu3Q9fHxepSYAn9TL/kHURQ9VkymJAZWGVeTab2ke2K2sG0wERFRp2BiQgCAP/QJl7Rt\ntTqANw5zrwl1jrzzNpQ3uFrrGjQCxnQPzPKn65L1cB9Uf/i8DYV1tpbfQERERO3CxIQAAGqVgMeH\nR0mOvX+0HlWNXDWhS5cpK+Oa0FMPrfvdfQDpFqbGVd05BZ6IiKizMTEhpzv7RyAh3HVJmGwi1hwx\ntfIOovbJKpFPew/MMq4m8mGLGUxMiIiILhkTE3LSqwU8NFS61+StPBOnW9MlabSJ2FUu21+SFFyJ\nyc4zZhj574SIiOiSMDEhifsui0S0zlViU2V24IOCegUjokC356wZ7hWByZFq9I/2aafyTjcwRiPZ\nk2VxeK4KERERUccwMSGJaJ0K8wZFSo69fqgOVgeHyNHFaa6MSxACc39JE0EQWM5FRETUyZiYkIcF\nQwwIc30YjGKTHZ8e56oJXZzM0uDaX9JE3jb42+JG2JnAExERXTQmJuShe7ga9wyQrpqszK2DQ+RN\nF3XMuQY7cquszscCgElBkphcnaCTlD1WNDqwt8KiYERERESBjYkJNeuRYQao3aptjtbYsIUTrqmD\n5EMVR3TTopv7clwA06oEXJvEci4iIqLOwsSEmpUSpcGtfaWlKityjRC5akIdkCWf9h7g3bjkPPaZ\nMHknIiK6aExMqEXygYs556z4sZylKtQ+oigiSzZYMT0xrIVXB6apyWGSlcW8ahtOGzkFnoiI6GIw\nMaEWDY3V4vpk6SfcKw4aFYqGAs3RGhvO1Ltme0RoBI+J6YGuq17l8T2xnIuIiOjiMDGhVi1Kk66a\nfF9ixoFKrppQ2zJlbYLH99BBrw7sNsHNmc62wURERJ2CiQm1alwPPcb1kH4ivDK3TqFoKJBsl5Vx\nTU4KrjKuJtN6S7+vH8rMqLVwCjwREVFHMTGhNi2S7TX58lQDTtSyjp5aZraL+KFMurIWLPNL5AbE\naJEa7eo0ZnV4bvonIiKitjExoTZdl6zHkC4a52OHCPzrEPeaUMt+PmtBvc3Vwa1nhAqD3K6hYCMf\ntvhNYYNCkRAREQUuJibUJkEQPPaafFhQj7J6u0IRkb9rrhuXIATf/pIm8rbB24rNnAJPRETUQUxM\nqF1u6RuO3gZXuYrFAaw6zL0m1DyP+SVBWsbVZGwPHWLcpsBXmR34+RybRBAREXUEExNqF41KwKPD\nDJJja4+aUG3mJl+Sqmq0Y3+FVXIsPcgTE61KwNRkDlskIiK6FExMqN3uGRCJuDDXJWO0ilh71KRg\nROSPss+Y4V7ENDxWi/hwdYuvDxYeU+DZNpiIiKhDmJhQu4VrBDw4RLpqsupwHRpsrKUnF/n8kmAv\n42pybVIYNG7baI7W2HCS3euIiIjajYkJdcjcQZGI0rruvs41OvC/x7hqQheIouixv2RyUmgkJl30\nKo+ZP1u4akJERNRuTEyoQ7roVbj/skjJsddy62BjByICcKzWhmKTq1tbmBoY2z00EhMAmNZb2jaY\n5VxERETtx8SEOuzBoQbo3K6c03V2fHGScxsIyJKVcV3dQ48wTfC2CZabLttnsqvMjBpOgSciImoX\nJibUYT0j1Lirf4Tk2IpcI0SRqyahLjNEy7ia9IvWYGCMa5CkTQS+L+aqCRERUXswMaGL8tjwKKjc\nPgjPO2/DtmJzy2+goGd1iPjhjCwxSQxr4dXBi925iIiILg4TE7oo/aI1uKmPtJ5+Ra5RoWjIH/xy\n1oI6tw5t3cNVGNpV08o7gpM8Mfm2uJF7sIiIiNqBiQldtMeHS1sH7y63YE85V01ClbwbV3qiHoIQ\nOvtLmlzZXYeuetf3XW0R8dNZToEnIiJqCxMTumgj43QeMypW5NYpFA0pLatUWrI0JQTLuABA09wU\neJZzERERtYmJCV2SxWlRksdbixpxuMqqUDSklGqzA/sqpL/39BAZrNgceXcuJiZERERtY2JCl2RC\ngg5XxGslx1Ye4l6TUJN9xgz3bRRDumqQEKFWLiCFTZFNgS+oseF4DafAExERtYaJCV0SQRCwaLh0\n1eTzEw04beRNWCjZLivjCsVuXO5idCqMT5CuGG0p4qwfIiKi1jAxoUt2Q+8wyewGuwi8foh7TUJJ\npmyw4pQQm1/SHLYNJiIi6phOS0zKy8s761QUYFSC4NGh64MCE8412BWKiHzpZK0Np+tcv2udChjX\nQ6dgRP5hem9pYrK73IJqM6fAExERtaTdicnYsWORl5fX7HO//vorZs+e3WlBUeC5vV8Ektz2FDTa\ngbfyTApGRL6SKSvjGtdDjwgNF2NTojQY1EW6kriNU+CJiIha1Or0s3//+99obLzwf6SiKOLLL7/E\nrl27PF534MABaDShN0iNXHRqAY8MM2DJzzXOY2uO1OGx4QZE63iTGsyySuTT3lnG1WRarzAcqXaV\nNWYUNeL21AgFIyIiIvJfrWYTZrMZa9asAXBhk/NXX33V7OsMBgPmzJnT+dFRQPnTwAi8eKAW580X\n2jPVWES8f9SER2Wb4yl42BwidpyRJSbcX+I0rVcYXnWb7bOtpBFWhwitKvQGTxIREbWl1cRk7ty5\nuO+++yCKIsaPH481a9Zg6NChkteo1aHbEpSkIrUqLBhswP/sd7ULfuNwHeYPMUCv5o1YMNpXYUGt\n1dUnOC5MheGx2lbeEVrGxOvQTa9C5f/tLam1iNhdbsE1PZm8ERERybVZY6NWq6HRaPDTTz8hLS0N\narXa+T9RFFFTU9PWKSiEzB8ciQi3AQ5lDQ5sOFavYETkTfJuXOmJeqgEJqFN1CoBU5OlSUgG2wYT\nERE1q93F/w6HA2vXrsXWrVsBAPv27cO0adNw/fXX49FHH0VdHdvDEhAbpsa9A6U19CtzjbC7T9+j\noLG91DMxIanpvcMljzMKGyGK/PdAREQk1+7EZM2aNVizZg2qq6sBAK+88gqioqLw6KOP4uTJk1i9\nerXXgqTA8siwKGjdrqwTRjs2nWY3omBTY3Hgl3MWybFQH6zYnClJeuhk/x4KOAWeiIjIQ7sTk2+/\n/Rbz58/H7NmzUVhYiIKCAsyZMwd33303Fi5ciO3bt3sxTAokSZFqzJJ1HlqRa+SnxEFm5xkz7G6/\n0stiNEiK5J4zuSitChMS5OVcTNSJiIjk2p2YlJeXY+TIkQCAXbt2QRAEjBs3DgCQmJjIvSYk8dgw\nA9x3GhyotHqU/VBgk/8+2Y2rZfIp8FuYmBAREXlod2ISGxuLiooKAMDu3bvRr18/xMXFAQAKCgrQ\nrVs370RIAemyLlrcKJt8/cpBYwuvpkCUWSK9uWYZV8umyf4t/HTWgqpGu0LREBER+ad2JyZXX301\nXn/9dSxbtgx79uzB9OnTAQAffvghVq9ejUmTJrV5DofDgRdeeAFz5szBwoULUVRU1Ozrli1bhtdf\nf729oZGfWpQmnV+ys8yCHNmeBApMp4w2nDC6bqy1KmB8gk7BiPxbb4MGQ7q6urM7RGBbCVcQiYiI\n3LU7MXniiScwduxYHDhwALfddhvuvPNOAMDGjRtx9dVXY+HChW2eIzs7GxaLBWvXrsXDDz+MlStX\nerxm48aNOHbsWAe+BfJXV8TrMFF2s/oqV02CgryM68ruOhi07f5zEpKmy8q5MgpZzkVEROSu1QGL\n7nQ6HZYsWeJx/MMPP0RYWPtKOPbv3+/clzJ8+HDk5+dLnj948CAOHz6MW265BadOnWpvaOTHnkiL\nws6ySufjzYWNOFptxWVdOIQvkGWVSm+qp7CMq03TeoXj5YOuturflzTCYheh4/BRIiIiAB1ITADA\nbDbjq6++wr59+2A0GtGlSxeMHDkSv//979uVnJhMJhgMBudjlUoFm80GjUaDiooKrFmzBv/85z/x\n3Xffdfw7Ib+UnqjHiG5aHKi0Oo+9dqgOb0zoqmBUdCnsDhHZ8o3vnF/SptHxWsSHqXCu8f+mwFtF\n7C43YxKTOiIiIgAdSExqa2uxcOFCHD9+HAkJCejWrRuKi4vx3Xff4bPPPsM777yDqKioVs8RGRkJ\nk8nkfCyKIjSaCyF8//33qKmpwaJFi1BZWYnGxkakpKRgxowZbcZWUFDQ3m/Da/whBn91R7waBypd\nN64bjplwR5dKJOjZPri9/On6OmxUodriupmO0YiIOH8aBdUKBhUgxkbrsKnR9Wf3o9wyJJqsrbzD\nN/zp+qLgw+uLvInXV2AZMGBAq8+3OzF54403UFFRgbfeesvZNhgAfv31VyxZsgSrV6/GX//611bP\nMWLECOzcuRNTp05Fbm4uUlNTnc/Nnj0bs2fPBgBs3rwZp06daldSArT9TXpbQUGB4jH4s36pItaU\nlDs3S9tFAd+Y4vDCsC4KRxYY/O36+nJ/LQDXXqHJyREYNDBZuYACyCxdAzZlVjkf7zGGoX//PhAE\n5cq5/O36ouDC64u8iddX8Gn3btUdO3Zg4cKFkqQEAC6//HLMnz8f2dnZbZ4jPT0dOp0Oc+fOxYoV\nK7B48WJkZGTgiy++6HjkFDDUKgGPD5eupr3/Wz0q2C41IGXJyrimcH5Ju01OlE6BP2W04yinwBMR\nEQHowIpJQ0MDkpKSmn0uKSmpXQMWVSqVxwb6lJQUj9e1d6WEAscd/SPwwq+1KGu4UF9fbxPxz/1G\nLB/LVZNAYrQ68PNZacvndO4vaTeDVoVreurxnVur4IzCRgxiMwgiIqL2r5ikpKRg586dzT63Y8cO\nJCezlINaplcLeFS2arL2qAkna/lpcSD5scwMm9vWoP7RGvQ2dKiHRsiTT4HP4BR4IiIiAB1ITO6+\n+258+umneOGFF5CTk4OTJ08iJycHy5Ytw8aNGzFr1ixvxklBYO5lkehlUDsfWx3Af++rVTAi6qjM\nEnbjulTXyxKTn89ZWNZIRESEDpRyTZ06FYWFhVi3bh2++uorABe6aul0OsyZMwc333yz14Kk4BCm\nEfD0qGgs2HHeeWzjyQY8MtSCUfGcGh4I5IMVJ3N/SYf1MmgwLFaLQ1UXunE5RODbokbcNSBS4ciI\niIiU1e7EpK6uDnPnzsWsWbNw6NAh1NTUwGAwYNiwYejShfsEqH1u7xeO1w/VIbfK1SJ1aU4NNk2L\nU7QzEbWtuM6G39w2aqsFYEICE5OLMa1XmDMxAS6UczExISKiUNdmKdfhw4dx++2346OPPgIAREVF\nYdy4cRg7diz+8pe/4N5778XRo0e9HigFB5Ug4LkroiXHfiizYFuxuYV3kL+Qd+O6srsO0bp2V4OS\nm+mycq7MEjPMds71ISKi0NbqXUVRUREeffRROBwODB48WPJceHg4nnzySahUKixcuBClpaVeDZSC\nx+SkMEyR7U14JqcGdgdvzPyZPDHh/pKLd3mcFj3CXX9+62wifixjck5ERKGt1cRk3bp16N69O95/\n/31MmDBB8pxer8fMmTPx3nvvISYmBu+//75XA6Xg8swV0XAv3MqrtuGj4/WKxUOtc4ii5/6SxLAW\nXk1tUQkCrkuW/vy2sDsXERGFuFYTk7179+Luu++GwWBo8TVdunTB3XffjZycnE4PjoJXWjcdZqWG\nS44t21eLeptDoYioNQcrragyu343MToBl8dx9salmN7bs22wKHLVkIiIQleriUllZWWLQxXd9evX\nD2fPnu20oCg0PDUqGnpX92CU1jvwVp5JuYCoRfIyrmt66qFRsVnBpUhP1CPM7fovqrMj7zzn+hAR\nUehqNTGJjY3FuXPn2jxJVVUVYmJiOi0oCg29DRrMHyxdjVtx0IhKznTwO5kl0jKjKSzjumQRGhUm\n9ZTu0+GwRSIiCmWtJiajR4/Gf/7znzZP8p///AeXXXZZpwVFoePPaVHoonN98l5rFfHPA0YFIyI5\nk9WBn85aJMc4v6RzTOslLWfMKGpQKBIiIiLltZqYzJo1Czk5OXj11VdhNnt2jLFarXjttdewZ88e\n3HbbbV4LkoJXF70Kf06Lkhx794gJp4wsafEXu8otsLht/ekbpUZKVLtHIFEr5FPgc85Zca6BK4ZE\nRBSaWr27GDRoEP785z/jpZdewpYtWzBmzBgkJibCbrejrKwMe/fuRXV1NRYsWIBx48b5KmYKMvMG\nG/BWvgnFpgs3ZFYH8N97a/FueqzCkREAZJVKy4vYjavzJEaqMaKbFgcqLwxbFAFsLW7EPRy2SERE\nIajNjz1vvfVWDBw4EB988AGys7NhsVwo6YiIiMDYsWNx9913Y9iwYV4PlIJXmEbA06OisXDneeex\nz0824JFhFlwep1MwMgKArBJZm2CWcXWqab3CnIkJAGQUMjEhIqLQ1K56jOHDh+PFF18EAFRXV0Ot\nViMqKqqNdxG136zUcLx+uA6Hqlw3aEt/qcHX0+IgCOz+pJQz9XbkV7vK6lQCMDGBiUlnmt4rDMv3\nu/ZVZZWa0WgTEabhdU9ERKGl1T0mzenSpQuTEup0KkHAc1dES47tLLPguxJOw1ZSlqwb1xVxOnTR\nd/jPBrViRDcteka4fqYmm4gfOAWeiIhCEO8wyG9MSQrD5ETpp/F//6UGdgeHzilFPu09nWVcnU4Q\nBFyf7DlskYiIKNQwMSG/8swV0XAvYMmrtmHD8XrF4gllDlH0GKw4JZGJiTdM4xR4IiIiJibkX0Z0\n0+H2VOlsh+f31aLBxps0Xzt83oZzja4+wVFaAaPj2YzAGyb1DEO42pWSF5vsOMQp8EREFGKYmJDf\neXpUNHRuV2ZpvQOr8+qUCyhEyfeXTOyph1bFDdneEK4RMEm2GpVRyGGLREQUWpiYkN/pbdBg/mCD\n5NiKg0ZUNnLwnC/Jy7jk+3+oc03vxX0mREQU2piYkF/684goxOhcn87XWkW8dMDYyjuoMzXYROwq\nl+8v4WBFb5JPgd9bYUV5PZNxIiIKHUxMyC911avw5zRpW+p3jphwysi6e1/YU26G2e2euJdBjX7R\nauUCCgEJEWpcHqeVHNtazFUTIiIKHUxMyG/NH2xAcqTrZtjqAP6xr1bBiEJHZjPduDjo0vumyVZN\nthQyMSEiotDBxIT8VphGwFOjpEMXPzvRgP0VFoUiCh2e+0tYxuUL8sRke6mZHemIiChkMDEhvzar\nXziGxUrLW5bm1HLGgxedbbDjUJXV+VgAPDpGkXekxWqRFOFaJWywi9hxhlPgiYgoNDAxIb+mVgl4\n9grpqsmOM2Z8X8KbNW+RT3u/PE6Lrnr+qfAFQRA8NsFnFLFtMBERhQbebZDfm5KoR7rsE/ulOTWw\nO7hq4g2Zsvkl7MblW9NlU+C3cgo8ERGFCCYm5PcEwXPVJO+8DR8fr1coouAliqLHikl6Esu4fGli\ngh6RGlejgdJ6Bw5UWlt5BxERUXBgYkIBYUQ3HWb1C5cce36fkRuDO1l+tQ1lDQ7n40iNgCvjdQpG\nFHrCNILHCiGHLRIRUShgYkIB46lR0dC5XbEl9Xa8lVenXEBBSF7GNSFBB52abYJ9Td6di4kJERGF\nAiYmFDD6RGkwb7BBcuyVXCOqGjkdu7PIy7gmJ3F/iRKu7xUG93Rwf6UVZzgFnoiIghwTEwoofxkR\nhRid65at1iLipYNGBSMKHo02ET+WSWfETGabYEV0D1djdLxsCjxXTYiIKMgxMaGA0lWvwhNpUZJj\n7+SbcMpoUyii4PHTWQsa7K49O0kRagyM0SgYUWib1ku6p2oLExMiIgpyTEwo4MwfbEBypGsIncUB\nPL+vVsGIgkNWqfTGNz1JD0Hg/hKlyPeZZJc2ot7maOHVREREgY+JCQWccI2Av10uXTX59EQD9ldY\nWngHtUeWbH/JFJZxKWpoV40kAW+0A9mlHCxKRETBi4kJBaTZqREY2lVaZvT3nFoOortIFY12j1kZ\nk5iYKEoQBExndy4iIgohTEwoIKlVAp4bEyM5ln3GjEx+onxR5J/Ej+imRVyYuoVXk69Ma2YKvIPJ\nNxERBSkmJhSwpiTqMamn9FP9pb/UwO7gjVtHyRM6duPyDxMS9DC4TYEva+AUeCIiCl5MTChgCYKA\nZ6+Ilhw7fN6GT040KBRRYBJFEdtL5IkJ55f4A71awOQkaZLI7lxERBSsmJhQQBsZp8Pt/aRtVZ/f\nV4sGG1dN2uu3GhtK3Ib3hasFjO2hUzAicucxBb6QiQkREQUnJiYU8J4aFQ2d25VcbLLj7fw65QIK\nMPJuXOMTdNCr2SbYX1yXLJ0Cf7DKihITp8ATEVHwYWJCAS8lSoMHBkdKjr180IiqRt68tUdWiWx+\nCfeX+JX4cDXGxEtXsDKKWK5IRETBh4kJBYW/pEUhWuf6XLnWIuLlg1w1aYvFLuKHMun8lylJ3F/i\nb+TduVjORUREwYiJCQWF2DA1nhguHbq4Jr8Op402hSIKDD+fs8Dkth8nIVyFwV00rbyDlCDfZ7Kj\nzIiCpOUAACAASURBVAyTlVPgiYgouDAxoaCxYIgBSRGu2RsWx4WN8NQyeTeu9EQ9BIH7S/zN4C4a\n9Da4rm2z3XNvEBERUaDzaWLicDjwwgsvYM6cOVi4cCGKiookz2dmZuLee+/Ffffdhw0bNvgyNAoC\n4RoBfxslXTX55EQDDlRaWngHZZZKS4JYxuWfBEHw7M7FtsFERBRkfJqYZGdnw2KxYO3atXj44Yex\ncuVK53N2ux1vvPEG3njjDbz77rv47LPPUF1d7cvwKAjckRqBIV2lpUh/z+GqSXPOmx34tUI6rI8b\n3/3XDbJ9Jt8Wcwo8EREFF58mJvv378e4ceMAAMOHD0d+fr7zObVajY8//hgGgwE1NTVwOBzQaFjr\nTh2jVgl47ooYybHtpWZklvDTZbnsUjPcb2uHxWrRPVzd4utJWVf30CNa6yqzO9vgwL4KToEnIqLg\n4dM7f5PJBIPB4HysUqlgs9mcCYhGo0FWVhZefPFFjB8/HuHh4S2dSqKgoMAr8XaEP8RAF/QWgSti\n9Mipcd1k/78fz+GDkY0I1PEc3ri+vizQwf1PwMjwel7Hfu7KGB2+q3D9zv73QCli+lx6csLfO3kT\nry/yJl5fgWXAgAGtPu/TxCQyMhImk8n5WBRFj1WRyZMnY9KkSXj22WfxzTff4Pe//32b523rm/S2\ngoICxWMgqZdiLUjfdM75uMCkwq+qJNzZP0LBqC6ON64vURSxd385ANesl1uHJmAA95j4tduEeny3\n87zz8U914XhlQMolnZN/v8ibeH2RN/H6Cj4+LeUaMWIEdu3aBQDIzc1Famqq87m6ujosWLAAFosF\nKpUK4eHh7A5EF21knA639ZOuuD2/rxaNNtbkA8CJWjuK6lxJiV4NjO3B/SX+7rpkPVRufxYPn7eh\nsI4tsYmIKDj4NDFJT0+HTqfD3LlzsWLFCixevBgZGRn44osvYDAYMG3aNCxYsADz5s2DIAiYPn26\nL8OjIPP0qGho3a7wYpMdb+dz6CLg2Y3r6h56hGv4QYC/iw1T46ru0inwW9mdi4iIgoRPS7lUKhWW\nLFkiOZaSkuL875tvvhk333yzL0OiIJYSpcEDgyKxKs9VPvjyQSP+ODASXfWhPcJHPgNjMrtxBYxp\nvcKwu9zVAjujqBHzBhtaeQcREVFgCO27Mwp6fx0RhWidayWgxiLi5QNGBSNSntUhYucZWWLCvSUB\nQz7PZOcZM4ycAk9EREGAiQkFtdgwNRYPlw5dfDu/DqeNoVuXv/ecBUara69NfJgKQ7uyNXegGBij\nQd8oV8c5iwPIKuEUeCIiCnxMTCjoLRxiQFKE9Ebu+V9Dd+hiZjNlXCo2mggYnAJPRETBiokJBb1w\njYAlo6SrJp8eb8CBSksL7whu22WfrnPae+CZ1kvace7b4kbYHew4R0REgY2JCYWEO1MjMKSLq1xJ\nBPBMTuitmlSbHcipkCZk3F8SeK5O0En2TlU0OrC3IjQTbSIiCh5MTCgkqFUCnh0TIzmWVWpGZklo\nlcDsLDPD/YP1wV006OlW5kaBQasScG0Sy7mIiCi4MDGhkHFtkh4TE6QzIJbm1MIhhk4JjHyT9OQk\nlnEFKo99JoVMTIiIKLAxMaGQIQgCnpOtmhyqsuKT4w0KReR7WbLBipMTWcYVqKYmh0Ht1rMgr9qG\nUyHcbY6IiAIfExMKKZfH6XBrX+nG4X/sq0WjLfhXTU4ZbThptDsf61TA1T10rbyD/FlXvcpjCjzL\nuYiIKJAxMaGQ81+jo6F1u/KLTXasya9TLiAfkZdxXdVdh0gt/wQEsulsG0xEREGEdyUUclKiNJg7\nKFJy7KWDRpw3B/f07ExZGdcUduMKeNN6S3+HP5aZUWsJ7uuYiIiCFxMTCkl/HRGFaK2rQL/GIuKV\ng0YFI/Ium0PEjjOegxUpsA2I0SI12tVVzeoAMjkFnoiIAhQTEwpJ3cLUWJQmHbr4dn4dCuuCc/Pw\nrxVW1Fhc+2hi9SqkddMqGBF1lumyYYtbikKnmQMREQUXJiYUshYOiURihOufgNkOPL8vOIcuysu4\n0hP1UAlCC6+mQCIv59pWbOYUeCIiCkhMTChkRWhUWHJ5tOTYJ8cbcLAy+CZoby9lGVewGttdhy5u\nU+CrzA78fC74rmEiIgp+TEwopN3VPwKDu2icj0UAz+QE16pJrcWBn89Kb1SZmAQPjUrA1GQOWyQi\nosDHxIRCmlol4NkrpEMXM0vNyCoJnhu7H8rMsLtV9gyM0SDZoGn5DRRwPKbAs20wEREFICYmFPKm\nJusxIUE6qG5pTi0cYnDU6cvnl6RztSTo/C4pDBq3LUNHa2w4WRucjRyIiCh4MTGhkCcIAp6TrZrk\nVlnx6Yng6G6UJdtfMiWJiUmw6aJXYVwPaXK9hasmREQUYJiYEAEYFa/DLX2lbVf/sa8WjbbAXjUp\nrLPhmNsn5xoBGJ/AxCQYTestvX5ZzkVERIGGiQnR//mvUdHQuv2LKKqzY82ROuUC6gTyblxXdtch\nSst/9sFoumyfya4yM2o4BZ6IiAII71CI/k/faA3mXBYpOfbyASOqzYF7cyefAs5uXMGrX7QGA2Nc\nTQ1sIvB9MVdNiIgocDAxIXLz15FRiP7/7d15XFT1/j/w15mFAYZVEFxyKRVzQUQEwV27LaZl6i2z\n37dMbUHTyOvNm+F1ibqm5pohKpl2TTM1WyytrlsJRu5SatKqiQKyM8DAzJzfH8gwZ2YQF5gzDK/n\n48FDzud8zjnvAzNy3vPZ1DWjiAsqRCw9XSxjRLfOaBJx8LL0wXRoa/daapMr4OxcRETUmDExIbIQ\n6K5EXKi3pGzN2RJcLGl8Mxydyq1Evr5mjIyfm4CeAWoZI6KGZp2YfP1XOQxcBZ6IiBoJJiZEViZ3\n06KlZ81bQ28E3jje+BZdtJ6Na1ArDZQKoZba5Aqigtzgr5G2+KVlcxV4IiJqHJiYEFnxVCkwK9xH\nUrb11zKk51XKFNGt2Zcp7cYzpBW7cbk6u6vAszsXERE1EkxMiOx4oqMnuvjVDCQWAcw7WihfQDep\npNKEH6w+KefCik2D9excTEyIiKixYGJCZIdKIWCe1aKLey/pcSCzcTzkpVypQKXFZGJ3eSvR3ltV\n+wHkMoZarQKfUWjAL4WNq7WPiIiaJiYmRLW47w4N+rWQrqY950gRTKLzDyben8nZuJoqXzeFzSKa\nXAWeiIgaAyYmRLUQBAGvWbWanM6rxPbfymSK6Mbtt1q/hN24mhZOG0xERI0RExOi64ho7oZR7T0k\nZQnHi6A3Om+rySWdET8X1kxvrBSAAS2ZmDQlw9pKE5Pvsyoa9UKhRETUNDAxIarDnAgfqC3eKRdL\njFh3tkS+gOpg3Y0rsrkbfN34Vm9K2nurcLfF5A1GEfiGq8ATEZGT49MKUR3u9FFhQmetpOytU8VO\n+wn0gUx24yLOzkVERI0PExOiGzCzpze81dKF65adLpYxIvtMomgzvmRoayYmTZH1OJNvLpWjkqvA\nExGRE2NiQnQDAt2ViAv1lpQlnS3BxRJDLUfIIz2vErkWLTk+bgJ6Bbpd5whyVb2buyFAU/NffFGF\niMNZXAWeiIicFxMTohs0pZsWLT1r3jJ6I/CfE87VamLdWjKwhQYqhVBLbXJlSoWA+2y6czn/jHJE\nRNR0MTEhukGeKgVmhftIyj78pRQ/5jnP4nX7rcaXDGE3ribNZtrgC+UQG8E6PERE1DQxMSG6CU90\n9JTMdiQCmHe0UL6ALJQaTDicZTW+pBUXVmzKhrbWwHJCtt+KjcgodK7uh0RERNWYmBDdBJVCwLze\n0laT/13S42Cm/DMeHc6qQIXFRGHtvJS400dV+wHk8rzVCvS3WgWes3MREZGzYmJCdJPuv8MdfYOl\nA8rnHC2CSeYuMvs4GxfZYd2dazcTEyIiclJMTIhukiAISIj0lZSdyq3Ejt/kHVhsvbDiYHbjIgAP\nWK0Cn5Zdgbxyo0zREBER1Y6JCdEtiGjuhkfae0jKEo4XQW+Up9XkSqkRZ/Jrxg4oBGBQS7aYENDW\nS4Wu/jVd+kwi8I1V6xoREZEzYGJCdIvmRPhAZTET74USI5LP6WSJxXq1916Bavhp+PamKjarwF9g\ndy4iInI+fHIhukV3+agw4W6tpOytU0UosFjg0FH2WXXjGsJuXGThgTbS1r29l8pRIVPrHhERUW2Y\nmBDdhplh3vBW1zSb5OtFLE937KKLoijatJgMacVuXFQjorkazd0tVoGvFG2mliYiIpIbExOi29Dc\nQ4kXu3tJypLOlOCvEsetFfFTvgHZZTWtNF4qAZFBbtc5gpoahWC7Cjxn5yIiImfj0MTEZDJhwYIF\nmDhxImJjY3Hx4kXJ/q+++goTJkzAM888gwULFsBkcnyXGKKbNaWbF1p41LyVyo3Af044rtXEejau\n/i01UCuEWmpTU2UzbTBXgSciIifj0MTk4MGDqKiowPr16/HCCy9gxYoV5n3l5eVISkrC6tWrkZyc\nDJ1Oh0OHDjkyPKJbolUr8Gov6aKLW34pxU95lQ65/n7r9UvYjYvsGNJKugr8nyVGnCvgKvBEROQ8\nHJqYnDx5EjExMQCA0NBQnD171rzPzc0NycnJcHev+lTPYDDAzY3dUahxeKKjJzr71kzJKgKYd7Sw\nwa9bbhCRajVWYAgXViQ7vNQKDGzJVeCJiMh5qequUn90Oh28vGr64ysUChgMBqhUKigUCgQEBAAA\ntm7dirKyMvTp0+eGzpuRkdEg8d4MZ4iB5PVcKyVmFNY8+H1zSY/NR35FpN/td0ms7fWVVqBAubGm\ni04LjQli1h/IyL7tS5IL6uWuwv9Q84HPzvP5GBHG/7+oYfH1RQ2Jr6/GpVOnTtfd79DERKvVQqer\nWedBFEWoVBYLf5lMePvtt3HhwgUsXLgQgnBj/eTrusmGlpGRIXsMJL+OHUVsz7uKw1kV5rK1l73x\neO/mUNzga9me672+/nukEECJefvetl4ICWlzy9ci1/ZkSwMW/Zpl3k4vViK/Eojqyv+/qGHw7yM1\nJL6+XI9Du3KFhYUhNTUVAJCeno4OHTpI9i9YsAAVFRVYvHixuUsXUWMhCAISIn0lZSdzK/Hx72UN\nds39VtMED2U3LrqONl4qdG+mNm+LAFLylPIFREREZMGhicngwYPh5uaGSZMmYdmyZZg+fTr27NmD\nnTt34ty5c/jss8/wyy+/YMqUKYiNjcX+/fsdGR7Rbevd3A0j20uT6oRjRdA3wGJ22WVGpFsMsBcA\nDGrJxISuz3oV+O+YmBARkZNwaFcuhUKBWbNmScrat29v/j4tLc2R4RA1iDm9fPHFn+UwXMtF/iwx\n4t1zOkzp5nX9A2/SQavWkp6BajRz50MmXd+wNu5YfKpmOuvvC5TQG0VolJximoiI5MUFFonqWQdf\nFSZ01krK3jpVjAJ9/a7Ls4+rvdMt6BmoRrDFujulRgEpV7gKPBERyY+JCVEDmNnTG16qmk+g8/Qm\nrEivv0UXRVHEAauFFYe04rgsqptCEHA/V4EnIiInxMSEqAE091DixVBp163VZ0pwSWesl/OfKzDg\ncmlNC4ynSkBUENf9oRtjvQr8pvOlePNEEUoq67dVj4iI6GYwMSFqIC9085J0mSk3Av85UVQv57ae\njat/CzeOEaAbNriVBpbDkcqMIt48WYyIHVnY8LMOBlP9T9ZARERUFyYmRA1Eq1bg1XAfSdmWX0rx\nk8VMWrdq/yVp15vB7MZFN8FTpcBTIVqb8qwyE15KLUC/T7Kx+0IZRJEJChEROQ4TE6IG9P86eSLE\n12IRURGYf6zwts6pN4pIsVjEEeD6JXTzFkT5YlEfX/ipbJOPnwsNGLc3D8N3X8WxnAo7RxMREdU/\nJiZEDUilEDCvt7TV5Ou/9Pj28q3PgpSWXYFSQ83DZEtPBTr7OnTmb3IBSoWA57p6YWfvMszo4QV7\nM02nZlXgnl05mHggD78XGRwfJBERNSlMTIga2LA27ogJlg5Mn3u0EKZb7CZjbzYuQeD4Ero1Xirg\n3xG+ODamBf5fJ0/YeyV9/HsZonZm4ZW0AuSW188EDkRERNaYmBA1MEEQ8FpvX0nZiauV2Pl72S2d\nb98lrl9C9a+1Vol3+vvj0Mgg3Guna2ClCUg6o0P49iwsO12MMgPHnxARUf1iYkLkAJFBbni4nXSA\n+mvHiqA33tzDXW65EadypYPnBzMxoXrUrZka2+4LxKf3ByAsQG2zv6hSxPxjRei9IwubM3QwcgYv\nIiKqJ0xMiBxkToQPLNZcxJ8lRqw/p7upcxzM1MPyMTC0mRrNPewMDiC6TYNauWP/Q82xdqA/2njZ\nvsYulRox5VABBn2eg32XuEAjERHdPiYmRA7S0VeNpztLp2hdfKoYhRU3vqid9folQ9laQg1IIQh4\nrIMnjowKRkKkD3zdbEeg/JhXidFf52LUV1dxOpczeBER0a1jYkLkQDN7esPLotkkT2/CivTiGzpW\nFEWbxGQIpwkmB3BXCZjW3Rsn/94CU7t5wc3OX479mXoM+iwHsd/m4WIJZ/AiIqKbx8SEyIGCPJSY\nFuolKUv8qQSXdHXPdPRLkQF/WdRzVwLRQUxMyHH8NQq8HuWLI6OD8ehdHjb7RQAf/lqG3h9nYe6R\nQhTob7w1kIiIiIkJkYO90M0LwR41b71yI7DgRFGdx1nPxtU3WAN3FacJJsdr563CukHNcOCh5hjQ\nws1mv94IrPixBOE7riDxp5KbnuSBiIiaJiYmRA7mpVZgVrh00cXNv5TiTH5lLUdUYTcucjY9A93w\n2QOB2HZvALr62S7yma8X8eoPhYj6OAs7fiu95bV7iIioaWBiQiSD/+vkiRCL1dpNIjD/aGGt9StN\nIg5dth747l5LbSLHEQQB997hju9GBuHtfn5o6Wn7Z+XPEiMmHczH33bl4NAVvZ2zEBERMTEhkoVK\nIWBuhLTV5Ku/9Pjusv2HtiPZFSixWNAu2EOBrv62n1ATyUWpEPBkiBbHxgTj37184K227WZ4/Gol\nRuy+irH/y8W5guu3EBIRUdPDxIRIJg+2dUd0kLR//pyjhXa7u+yz6sY1uJUGgsDxJeR8PFUKzAjz\nxvExwXi2ixb2hkF9dbEcfT/JRlxKPq6U1j3xAxERNQ1MTIhkIggCXouUtpqcuFqJT34vs6l7IFO6\ngN0QduMiJ9fcQ4nF0X5IGxWMh9vZvl5NIrDxfCl67cjCf04UobiSM3gRETV1TEyIZBQVpMFDVg9t\nrx0vQoXFLEZFhqouMJYGc2FFaiQ6+Krw/tAAfD08EH2CbGfwKjWIWHSyGL22Z+HdcyWoNHGAPBFR\nU8XEhEhmcyN8oLTo7vJHsRHrf9aZt48WKGH5rNbVX4UWnkoHRkh0+6KCNNjzYCA2DW2Gjj6246Ny\nyk2YcbgQfT/Jxq4/yyByBi8ioiaHiQmRzDr6qvF0Z62kbPHJYhRWVHVt+b5A+jblbFzUWAmCgBHt\nPHB4VBCWxPiiubvtn6CMQgP+b18ehn15FT9kcwYvIqKmhIkJkRP4V09vaC1GCefqTViZXgxRFJGW\nL20d4fol1NipFQIm3e2F438Pxsth3vC0M0L+++wK3PfFVTy1Lxe/FhpkiJKIiByNiQmREwjyUGJa\ndy9JWeJPOqRkVSBTX/M21SiBmGDbfvpEjZG3WoH4Xj44NiYY40M8obAzg9dnf5ajz84svPx9Aa6W\ncwYvIiJXxsSEyElM7e6FII+at2SZUcSE/XmSOtFBGniq+LYl19LSU4kV/fyRMjII97ex7apoEIF1\nZ3UI356FJaeKUWrgDF5ERK6ITzhETsJLrcCsntLpg3PKpQ9gQ9mNi1xYF381tv4tAJ8/EIjwQLXN\n/uJKEQnHixCxIwv/Pa+DkTN4ERG5FCYmRE7kyRBPdPKtfUV3ThNMTcGAlhrsHdEc7w7yRzsv2xno\nLpeaMC2lAAM+zcY3f5VzBi8iIhfBxITIiagUAuZG+NjdF+iuQGgz20+RiVyRQhAw5i5P/DA6GP+J\n8oW/xnYAypkCAx79Jhcjv8rFyasVMkRJRET1iYkJkZMZ3tbd7kJ0g1tpoBDsjA4mcmEapYAp3bxw\nYkwLxHX3gsbOEj7fXtZj8Oc5eO5gHv4s5gxeRESNFRMTIicjCAJe623bajKE3bioCfPTKDA/0hdH\nRwfj8Q4esJeif/RbGSI/zsK/jxSiQM8B8kREjQ0TEyIn1CdYg0fae5i3fdQC7ruDCysStfFSIWlg\nMxx4uLndMVcVJuDtH0vQc/sVvP1jMcoNHH9CRNRYMDEhclLv9PfDs3dr0c/fiP8ObYbmHnb6sBA1\nUWEBbvjk/kB8fF8AuvnbThhRUCHi30eKELkzCx/9WgoTB8gTETk9JiZETkqrVmBxjB+Wd9NjUCu2\nlhDZM7S1O759OAiJ/f3Q2tM2eb9YYsRz3+ZjyOc5OJiplyFCIiK6UUxMiIioUVMqBDzRSYujY4Ix\nL8IHPmrbESincisx8qurePTrqziTXylDlEREVBcmJkRE5BI8VAJe6uGNE38PRmxXLdR2/sJ9c0mP\n/p9mY+qhfGTqjI4PkoiIasXEhIiIXEqAuxJv9vHDD6OCMcpiEolqJhHYlFGKiB1ZeP1YEYoqOIMX\nEZEzYGJCREQu6U4fFd4b0gz/G9EcMcG2awOVGUW8dboY4duzsPZMCSpNHCBPRCQnJiZEROTSejd3\nw5fDArH5nmYI8bWdwStXb8LMtEJE78zCp3+UQeQMXkREsmBiQkRELk8QBDzY1gOpjwRheV8/BHnY\n/vn7tciI8fvzcN8XOfg+izN4ERE5GhMTIiJqMlQKAU931uL4mGDMCveGVmU7g9eRnEo88OVV/N/e\nXGQUcgYvIiJHYWJCRERNjpdagX/19MHxMcGY2FkLpW1+gl0XyhG9MxszDhcgu4wzeBERNTQmJkRE\n1GQFeyqxtK8fDj8ShAfb2i5kahSBd8/p0Gt7FhadLIKukjN4ERE1FNtRgERERE1MiJ8am+8JQOoV\nPeYcLcTRHGkXrhKDiP+cKMby9BJ4qQWoBQFKBaBWAGqFAJVCgEqo2lYpBKgVQtX3gtW2uZ4AVfWx\nVttqRdWikWrrenbqq67Vry6XbNdyfpVQdX4iImfDxISIiOiavi00+GZ4c3z6RznmHyvE78XSLlyl\nBhGlhsY/a5cASBIYlSBYJFX2Ex1ldeJkUa6yTKgEq3oKoDBfjcCSIiiEqi4aCkGo+v7atiAASssy\nAVCgaluwqKdUCNeOr/4SIEi2q45TKmCuJ1Sf1+o4pQCLY6Xx2C+zjs9emXRbQNX1iejmMDEhIiKy\nIAgCHrnTAw+2dcd7P+uw8GQx8vSu1YVLBFBhAirMa7c0VLKlBi4WN9C5nVt18qO0SIpqkiZpQqM0\nJzRCzT7rBArXjrv2ryBIkzXLugKkiZ11kiaY47BN8ARIEzNYJGOCxTks61rGVX1f9ura3psgjd3O\nvVknkZb3lp2lRAvozElg9T6h1u+v1RNw/fqWdW3Kpd/j2n3d6PUldSXnsbyOYN4vLbd3bsHmHPZi\n8VAJCHBX1v3ClZlDExOTyYSFCxciIyMDbm5uiI+PR5s2bSR1ysvLMXXqVMyePRvt27d3ZHhERERm\nbkoBz3f1wuMdPbEyvRjJ53QorGj8rSXkGCKqxigZRcsSe7Xo1mmAjAK5g2gU/tZag+33BcodRp0c\nmpgcPHgQFRUVWL9+PdLT07FixQq89dZb5v1nzpzBm2++iezsbEeGRUREVCtfNwX+HeGLV8N9UFhh\nQqUJqDSJMIjX/jVJ/60UAaNJNNerNAEGO/UrTYBBtNq2d147xxmvXcf6+garckN1HFbXISJyRg5N\nTE6ePImYmBgAQGhoKM6ePSvZX1lZicWLF2Pu3LmODIuIiKhOSoWAZo2gK8SNMFolLAaLRMpwLWGq\ntC4Xa69nnUhV18u+ehX+zQJgEgFRBEwQYRIB07WWBJN4bRvX9leXQVpHvFbHZFlHUlZzXusy0Xwu\nizrmemLNda/9XCyvYx2z6dq2UbSM17YO20HI2TSWEU8OTUx0Oh28vLzM2wqFAgaDASpVVRhhYWG3\ndN6MjIx6ie92OEMM5Lr4+qKGxNcXWVMA0Fz7uiECAOW1L0ttASCr3uJqLKoTKcuESrT8F9IESRQF\nGAHAYr/5HFbngcV5qs4pSM5d/b3p2vlMsPq+lmMB22tUx2E/JkG63+r61t+by6qPs/o52VzDqswy\n3upuctU/a6DmeuZRU6L9bcu6EKXHiaIg3W8Rp+Qa1uey2F9XPcttXLumJHarmFBbub16dn4W1WXq\nCh0yMvIht06dOl13v0MTE61WC51OZ94WRdGclNyOum6yoWVkZMgeA7kuvr6oIfH1RQ2Jry9qSHx9\nuR6HLrAYFhaG1NRUAEB6ejo6dOjgyMsTEREREZGTcmiLyeDBg5GWloZJkyZBFEXMmTMHe/bsQVlZ\nGUaNGuXIUIiIiIiIyIk4NDFRKBSYNWuWpMzelMBJSUkOioiIiIiIiJyBQ7tyERERERER2cPEhIiI\niIiIZMfEhIiIiIiIZMfEhIiIiIiIZMfEhIiIiIiIZMfEhIiIiIiIZMfEhIiIiIiIZMfEhIiIiIiI\nZMfEhIiIiIiIZCcUFBSIcgdBRERERERNG1tMiIiIiIhIdkxMiIiIiIhIdkxMiIiIiIhIdkxMiIiI\niIhIdkxMiIiIiIhIdkxMiIiIiIhIdkxMiIiIiIhIdiq5A2jMTCYTFi5ciIyMDLi5uSE+Ph5t2rSR\nOyxyEQaDAQkJCcjMzERlZSUmTpyIgQMHyh0WuZC8vDw89dRTWLVqFdq3by93OORCNmzYgG+//RYG\ngwFjxozByJEj5Q6JXITBYMC8efNw+fJlKBQKxMfH8/8vF8IWk9tw8OBBVFRUYP369XjhhRewYsUK\nuUMiF7J79274+vpi3bp1WLFiBRYvXix3SORCDAYDFixYAI1GI3co5GKOHTuG06dPIzk5GUlJ8wtK\n4QAAD/hJREFUScjKypI7JHIhKSkpMBqNePfdd/HMM89g9erVcodE9YgtJrfh5MmTiImJAQCEhobi\n7NmzMkdEruSee+7B0KFDAQCiKEKpVMocEbmSFStWYPTo0di4caPcoZCL+f7779GxY0fMnDkTOp0O\n06ZNkzskciFt27aF0WiEyWSCTqeDSsVHWVfC3+Zt0Ol08PLyMm8rFAoYDAa+SaheeHp6Aqh6nc2a\nNQuxsbEyR0SuYteuXfDz80NMTAwTE6p3BQUFuHLlCpYuXYrMzEzMmDED27ZtgyAIcodGLsDT0xOX\nL1/Go48+isLCQixdulTukKgesSvXbdBqtdDpdOZtURSZlFC9ysrKwuTJkzFs2DA88MADcodDLuKz\nzz7DDz/8gNjYWJw/fx7z5s3D1atX5Q6LXISvry+io6OhVqvRrl07uLm5IT8/X+6wyEVs3rwZ0dHR\n2LFjBz744APMnz8fer1e7rConjAxuQ1hYWFITU0FAKSnp6NDhw4yR0SuJDc3F9OmTcPUqVPx8MMP\nyx0OuZC1a9dizZo1SEpKQkhICObNm4fAwEC5wyIXERYWhsOHD0MUReTk5KC8vBy+vr5yh0UuwsfH\nx9xbxcfHBwaDASaTSeaoqL7w4/3bMHjwYKSlpWHSpEkQRRFz5syROyRyIRs2bEBRURHWr1+P9evX\nAwCWL18Od3d3mSMjIqrdgAEDcOLECTz99NMQRREvv/wyx8hRvRk3bhwSEhLw7LPPwmAwYPLkyfDw\n8JA7LKonQkFBgSh3EERERERE1LSxKxcREREREcmOiQkREREREcmOiQkREREREcmOiQkREREREcmO\niQkREREREcmOiQkRUT2YP38+oqKiav3q37+/LHGNHDmyQacyv3LlCh566CHk5uY22DXqEhUVhdWr\nV8t2fQD45JNPEBUVhczMzAY5v8FgQFRUFNauXXvDxyQmJmLx4sUNEg8RUUPgOiZERPXE398fb731\nlt19CoXrfQ4kiiISEhLw6KOPIiAgQO5wyMqECRMwZswYDBo0CFFRUXKHQ0RUJyYmRET1RK1WIzQ0\nVO4wHObgwYM4e/YslixZIncoZIeHhwfGjRuHZcuWYfPmzRAEQe6QiIiui4kJEZGDxcbGIigoCO3a\ntcO2bdug1+sRERGB6dOno3Xr1uZ6Z8+eRVJSEs6cOQODwYCePXvihRdeQMeOHc11cnNzkZiYiJSU\nFJSXl6NTp06YPHkyevXqZa5jNBqRmJiIXbt2obi4GHfffTdmzJiBu+++GwCg1+uxbNkypKSkIC8v\nD8HBwbjvvvvwzDPPQKWq/c/Ehg0bMGTIELi7u5vLRo4ciXvvvRd6vR5ffPEFFAoF+vXrh+nTp8PP\nz89c7/Tp01i9ejV++uknqNVqxMTEIC4uDs2bNwcAHDt2DJMnT8asWbOwYcMGFBQUICEhAQMHDrQb\nS1lZGRYsWIB9+/ahoqIC4eHhePnll80/z/nz5+PIkSPYtWuX+ZiLFy9izJgxmDNnDkaMGGG+ZmJi\nIjZt2oTjx4/Dzc0N99xzD1566SXz6tImkwnr16/Hp59+ioKCAkRHR6NHjx6SeNauXYs9e/Zg+PDh\n2LJlC5RKJT744AMEBgbi888/x+bNm3HhwgX4+/tj2LBheO6556BWq83HHz16FImJicjIyECLFi0w\nY8YMm3v++uuvsXHjRly4cAEajQYRERGYMmUK2rVrZ65z//3345133sF3331X68+OiMhZuF7fAiIi\nGRkMBrtfJpNJUu/QoUP48ssvMWPGDLzyyis4f/48Jk+ejLKyMgBVD6aTJk2CwWDA7NmzER8fj+zs\nbDzzzDP47bffAADl5eV47rnnkJaWhilTpmDhwoXw8fFBXFwcfvnlF/O19u7di7Nnz2L27NmYM2cO\nLl++jBkzZsBoNAIAli5ditTUVEydOhVvv/02hg8fjg0bNmDjxo213ueff/6JM2fO4N5777XZ9/HH\nHyM9PR1z587F1KlTkZKSgri4OPPP4NSpU4iNjYVCocAbb7yBf/7zn/jpp5/w/PPPo6SkRHKuNWvW\nYNq0aZg1a5Yk2bL20UcfQafT4Y033sA//vEPnDp1CrNnz77er6pW8fHx6NKlC5YsWYInnngCn3zy\nCZKTk837V65ciXfffRcPP/wwFi1aBB8fH7tjXC5fvoyDBw/i9ddfx/Tp0xEYGIj//ve/SEhIQHh4\nOJYsWYJx48Zh69atknFA586dw4svvgitVos333wTY8eOtRkndOrUKcydOxeDBg3C8uXL8a9//Qvn\nz5/H9OnTIYqiuV5QUBBCQ0Oxe/fuW/pZEBE5EltMiIjqSXZ2Nvr27Wt334QJEzB58mTzdllZGd5/\n/33ccccdAID27dvjySefxOeff47HHnsM77zzDlq3bo2VK1dCqVQCAPr06YNRo0YhKSkJixYtwq5d\nu3Dx4kVs3LgRXbp0AQD06tULTz31FI4dO2ZuWQkICMCSJUvg5uYGACgqKsLChQvx+++/o2PHjjhx\n4gSioqJw//33m8/h6ekJf3//Wu/1yJEjAICuXbva7FMoFFi1ahW8vLwAAH5+fpg5cyZSUlIwYMAA\nrFq1CnfccQdWrFhhbpHp1asXRo8eje3bt+Ppp582n2v06NH429/+VsdPHggJCcHrr79u3r548SLe\nf/996HQ6aLXaOo+3NGLECDz//PMAgMjISBw5cgTfffcdpk2bhuLiYnz00Ud4/PHH8eyzzwIAYmJi\nkJOTg++//15yHqPRiLi4OERERAAASkpKsG7dOjz00EOYOXMmACA6OhpBQUGIj4/H6dOn0aNHD2zY\nsAH+/v6S35mPj48k0Tp58iQ0Gg3Gjx8PjUYDAAgODsahQ4dQWloquecuXbrgyy+/vKmfARGRHJiY\nEBHVk2bNmmHp0qV291V3UaoWGhpqTkoAoHPnzmjdujWOHz+Ohx56CGfOnMGECRPMSQkAeHt7Y8CA\nAfjuu+8AVD2ctmjRwpyUAFXjXLZs2SK5VteuXc0PuADM3ZuKiooAVD18b9u2DdnZ2YiJiUHfvn0x\nbty4695rZmYmtFotfHx8bPb169fPnJQAwMCBA6FUKnHixAlERkYiPT0djz/+OICqFiagKnnq3Lkz\n0tLSJIlJSEjIdeOo1rNnT8l29T0WFxffdGISFhYm2Q4KCsJff/0FAPjxxx9hMBgwYMAASZ2hQ4fa\nJCaANP709HSUl5dj0KBB5vsGgL59+0KhUCAtLQ09evTAyZMn0bdvX8nvbMiQIZIxIhEREVi9ejXG\njRuHwYMHIyYmBuHh4TZdygCgVatWKCoqQklJieT3QkTkbJiYEBHVE5VKZbcFwZ6goCCbMn9/fxQV\nFaG4uBiiKNqd6SogIMDc3amgoOC6rRrVqsdGVKueIay6y89LL72EoKAg7N69G8uWLcOyZcvQqVMn\n/OMf/zB/2m+tpKTE/El9XfemUCjg5+eHwsJCFBUVwWQyYfPmzdi8ebPNsW3atJFse3p61nl/9u6x\n+iHeugvdjbAcMwNUxV/9syosLAQA+Pr6SuoEBgbWGVf1sf/85z/t1s3JyQFQlTBajscBqhJOy2t2\n794dK1euxObNm7Ft2zZs2rQJPj4+eOyxx/Dss89Kkpjq+9HpdExMiMipMTEhIpJB9UOqpby8PHTv\n3h3e3t4QBMHu2iBXr141P6B6eXnh4sWLNnV+/PFHuLu7SwbJX49arcb48eMxfvx45OTkIDU1Fe+9\n9x5mzpyJPXv2SAZlV/Pz87MZD1LbvRmNRhQUFKBZs2bQarUQBAFjx47FAw88YHOsZStBfRIEwTym\nplr1eJ6bUZ0w5OXloUOHDubygoKCOo+tTgrmzZuH9u3b13puPz8/m9+90WhEcXGxpKx6jRy9Xo8T\nJ05g586dSE5Oxl133SXp/lZ9nHUyRUTkbDj4nYhIBqdPn5Y8zJ49exaZmZno06cPPDw80KVLF+zd\nu1fyMF1SUoJDhw6ZuxqFh4fj8uXL+Pnnn811DAYD4uPjsW3bthuKo7y8HGPGjMGmTZsAVHU5Gzly\nJP7+97+juLgYOp3O7nEtW7ZERUUF8vPzbfYdPnwYlZWV5u1vv/0WRqMRffr0gVarRefOnfH777+j\na9eu5q+QkBC89957OHz48A3FfbO0Wi2KiopQXl5uLjt16tRNn6dHjx7QaDTYu3evpPzQoUN1Htu9\ne3eo1WpkZ2dL7t3LywurVq3CH3/8AaCqa93hw4dRWlpqPjY1NVXyWli+fDnGjx8PURSh0WgQHR2N\nWbNmAaha9NJSdnY2/P39bVqCiIicDVtMiIjqSWVlJdLT02vdf+edd5o/NS8vL0dcXBwmTpyI0tJS\nJCYmomPHjuZWhClTpiAuLg4vvvgixo4di8rKSmzcuBF6vd486HrEiBHYunUrXn75ZTz//PMICAjA\n9u3bUVhYWOcYkWru7u7o3LkzkpOToVKpEBISgszMTGzZsgWRkZE2XYqq9enTB0DVOJchQ4ZI9mVn\nZ2PGjBkYO3YssrKykJiYiJiYGPTu3dt8b9OnT0d8fDyGDRsGANi6dSuOHj2Kxx577Ibivln9+/fH\n1q1b8frrr+ORRx7Br7/+ektre3h6emLChAlYs2YNtFotIiMjcejQIaSkpNR5rJ+fH5588kmsW7cO\npaWl6N27N/Ly8szbnTt3BgBMnDgRBw4cQFxcHJ566ink5+cjKSlJMt4oMjISW7Zswdy5c/Hggw9C\nFEVs27YNGo3GZlrgU6dOITo6+qbuk4hIDkxMiIjqSX5+PiZNmlTr/tWrV5vHbISGhiImJgYJCQkA\nqgaIv/jii+ZuU1FRUVi1ahXWrl2L+Ph4qNVqhIeHY86cOeYuWlqtFmvWrMHbb7+NZcuWwWg0okuX\nLkhMTLTbVag28fHxWLNmDbZs2WLuKjZ48GDJLGLWWrVqhW7duiE1NdUmMbnnnnvQrFkzzJ49GxqN\nBsOHD5ecKzo6GitXrkRycjJeffVVc0K0YsUKREZG3nDcN6NPnz6Ii4vDhx9+iAMHDqBLly5YvHgx\nxo8ff9PnmjhxIjw9PfHhhx9i69atCAsLQ1xcHBYtWlTnsbGxsQgMDMT27dvxwQcfwNvbGxEREYiN\njTWPKWrXrh1Wr16NlStX4tVXX0VAQACmT58uOX+/fv2QkJCATZs24ZVXXgFQNcnBO++8g7Zt25rr\n5eTk4Pz58+ZklojImQkFBQVi3dWIiKi+xMbGwmg0Yt26dXKHclsOHjyIuXPn4osvvjDPfDVy5EiE\nhYXhtddekzk6AoDk5GTs27cPH3zwAVd+JyKnxzEmRER0SwYNGoROnTrd8HgWciydTocdO3Zg2rRp\nTEqIqFFgYkJERLds3rx52L59O65evSp3KGRlw4YN5jVOiIgaA3blIiIiIiIi2bHFhIiIiIiIZMfE\nhIiIiIiIZMfEhIiIiIiIZMfEhIiIiIiIZMfEhIiIiIiIZMfEhIiIiIiIZPf/AQ9ET3Yq4updAAAA\nAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "The training accuracy rate: 97.13%.\n", "The test accuracy rate: 72.00%.\n" ] } ], "source": [ "# train NN with mini-batch and Adam\n", "parameters = model(X_train, Y_train, layers_dims, optimizer=\"adam\",\n", " learning_rate=0.001, mini_batch_size=64,\n", " beta1=0.9, beta2=0.999, epsilon=1e-8,\n", " num_epochs=1000, activation_fn=\"tanh\")\n", "\n", "# print the test accuracy\n", "print(\"The training accuracy rate: {}\".format(\n", " accuracy(X_train, parameters, Y_train, \"tanh\")[-7:]))\n", "print(\"The test accuracy rate: {}\".format(\n", " accuracy(X_test, parameters, Y_test, \"tanh\")[-7:]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We're by no means comparing the performance of the optimization algorithms. The goal is to see how each algorithm works and all of them are very effective and can be tuned to yield the best results." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "

\n", "Conclusion\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The most popular optimization algorithms are: Mini-batch gradient descent (MB), MB with RMSProp, and MB with Adam. Even though neural network doesn't converge to a global minimum, it's able to arrive at a significantly low values of the loss function and having a very low error rates. Below are some key takeaways:\n", "- There is no definitive guide of which optimization algorithm to use under specific scenarios. It's more trial and error. However, the optimization algorithms with momentum methods such RMSProp and Adam with mini-batch gradient descent are very effective.\n", "- Momentum methods can be used with Batch, mini-batch, and stochastic gradient descent.\n", "- Batch size $b$ can be tuned using a validation set. It plays a critical role in the performance of the learning algorithm.\n", "- $\\beta$ that measures the velocity of gradients or how far back to go to include past gradients can be tuned too.\n", "- Gradient descent is a first-order optimization algorithm, which means it doesn't take into account the second derivatives of the cost function. However, the curvature of the function affects the size of each learning step. The gradient measures the steepness of the curve but the second derivative measures the curvature of the curve. Therefore, if:\n", " - Second derivative = 0 $\\rightarrow$ the curvature is linear. Therefore, the step size = the learning rate $\\alpha$.\n", " - Second derivative > 0 $\\rightarrow$ the curvature is going upward. Therefore, the step size < the learning rate $\\alpha$ and may lead to divergence.\n", " - Second derivative < 0 $\\rightarrow$ the curvature is going downward. Therefore, the step size > the learning rate $\\alpha$.\n", " \n", " As a result, the direction that looks promising to the gradient may not be so and may lead to slow the learning process or even diverge.\n", "- If Hessian matrix has poor conditioning number, i.e the direction of the most curvature has much more curvature than the direction of the lowest curvature. This will lead the cost function to be very sensitive in some directions and insensitive in other directions. As a result, it will make it harder on the gradient because the direction that looks promising for the gradient may not lead to big changes in the cost function.\n", "- The norm of the gradient $g^Tg$ is supposed to decrease slowly with each learning step because the curve is getting flatter and steepness of the curve will decrease. However, we see that the norm of the gradient is increasing. That's because of the curvature of the curve. Nonetheless, even though the gradients' norm is increasing, the neural network was able to achieve a very low error rates.\n", "- Newton's method is a second order optimization algorithm where it jumps right away towards the minimum. It takes into account the second derivatives and the update rule is: $w = w - \\frac{g}{H}$ where $g$ is the gradient and $H$ is the hessian matrix that contains the second derivatives. The main disadvantages of Newton's method are:\n", " - Inverting the hessian matrix is computationally expensive since it is a square matrix in the number of the parameters. With neural network, we have hundreds of thousands or millions of parameters. Also, the inversion has to occur on every learning step.\n", " - If there is a saddle point, which is very common with high dimensional data, it does worse than the gradient descent because it stucks on the saddle point. However, gradient descent can manage to escape it with a good learning rate." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.2" } }, "nbformat": 4, "nbformat_minor": 2 }