{ "cells": [ { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

Table of Contents

\n", "
" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# code for loading the format for the notebook\n", "import os\n", "\n", "# path : store the current path to convert back to it later\n", "path = os.getcwd()\n", "os.chdir(os.path.join('..', 'notebook_format'))\n", "\n", "from formats import load_style\n", "load_style(plot_style = False)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Ethen 2018-09-15 15:33:48 \n", "\n", "CPython 3.6.4\n", "IPython 6.4.0\n", "\n", "numpy 1.14.1\n", "matplotlib 2.2.2\n", "keras 2.2.2\n", "tensorflow 1.7.0\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Using TensorFlow backend.\n" ] } ], "source": [ "os.chdir(path)\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "# 1. magic for inline plot\n", "# 2. magic to print version\n", "# 3. magic so that the notebook will reload external python modules\n", "# 4. magic to enable retina (high resolution) plots\n", "# https://gist.github.com/minrk/3301035\n", "%matplotlib inline\n", "%load_ext watermark\n", "%load_ext autoreload\n", "%autoreload 2\n", "%config InlineBackend.figure_format = 'retina'\n", "\n", "import tensorflow as tf\n", "from keras.datasets import mnist\n", "from keras.utils import np_utils\n", "\n", "%watermark -a 'Ethen' -d -t -v -p numpy,matplotlib,keras,tensorflow" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Tensorflow" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "TensorFlow provides multiple APIs. The lowest level API--TensorFlow Core-- provides you with complete programming control. We recommend TensorFlow Core for machine learning researchers and others who require fine levels of control over their models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Hello World" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can think of TensorFlow Core programs as consisting of two discrete sections:\n", "\n", "- Building the computational graph.\n", "- Running the computational graph." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# note that this is simply telling tensorflow to \n", "# create a constant operation, nothing gets\n", "# executed until we start a session and run it\n", "hello = tf.constant('Hello, TensorFlow!')\n", "hello" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "b'Hello, TensorFlow!'\n" ] } ], "source": [ "# start the session and run the graph\n", "with tf.Session() as sess:\n", " print(sess.run(hello))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can think of tensorflow as a system to define our computation, and using the operation that we've defined it will construct a computation graph (where each operation becomes a node in the graph). The computation graph that we've defined will not be `run` unless we give it some context and explicitly tell it to do so. In this case, we create the `Session` that encapsulates the environment in which the objects are evaluated (execute the operations that are defined in the graph).\n", "\n", "Consider another example that simply add and multiply two constant numbers." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mutiply: 6.0\n", "add: 5.0\n", "add: 5.0\n" ] } ], "source": [ "a = tf.constant(2.0, tf.float32)\n", "b = tf.constant(3.0) # also tf.float32 implicitly\n", "c = a + b\n", "\n", "with tf.Session() as sess:\n", " print('mutiply: ', sess.run(a * b))\n", " print('add: ', sess.run(c)) # note that we can define the add operation outside \n", " print('add: ', sess.run(a + b)) # or inside the .run()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The example above is not especially interesting because it always produces a constant result. A graph can be parameterized to accept external inputs, known as `placeholders`. Think of it as the input data we would give to machine learning algorithm at some point.\n", "\n", "We can do the same operation as above by first defining a `placeholder` (note that we must specify the data type). Then `feed` in values using `feed_dict` when we `run` it." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mutiply: 6.0\n", "add: 5.0\n" ] } ], "source": [ "a = tf.placeholder(tf.float32)\n", "b = tf.placeholder(tf.float32)\n", "\n", "# define some operations\n", "add = a + b\n", "mul = a * b\n", "\n", "with tf.Session() as sess:\n", " print('mutiply: ', sess.run(mul, feed_dict = {a: 2, b: 3}))\n", " print('add: ', sess.run(add, feed_dict = {a: 2, b: 3}))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Some matrix operations are the same compared to numpy. e.g. \t" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[3. 4.]\n", " [5. 6.]\n", " [6. 7.]]\n", "[3.5 5.5 6.5]\n", "[1 1 1]\n", "[3.5 5.5 6.5]\n", "[1 1 1]\n" ] } ], "source": [ "c = np.array([[3.,4], [5.,6], [6.,7]])\n", "print(c)\n", "print(np.mean(c, axis = 1))\n", "print(np.argmax(c, axis = 1))\n", "\n", "with tf.Session() as sess:\n", " result = sess.run(tf.reduce_mean(c, axis = 1))\n", " print(result)\n", " print(sess.run(tf.argmax(c, axis = 1)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The functionality of `numpy.mean` and `tensorflow.reduce_mean` are the same. When axis argument parameter is 1, it computes mean across (3,4) and (5,6) and (6,7), so 1 defines across which axis the mean is computed (axis = 1, means the operation is along the column, so it will compute the mean for each row). When it is 0, the mean is computed across(3,5,6) and (4,6,7), and so on. The same can be applied to argmax which returns the index that contains the maximum value along an axis." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Linear Regression" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll start off by writing a simple linear regression model. To do so, we first need to understand the difference between `tf.Variable` and `tf.placeholder`. \n", "\n", "> [Stackoverflow](http://stackoverflow.com/questions/36693740/whats-the-difference-between-tf-placeholder-and-tf-variable). The difference is that with `tf.Variable` you have to provide an initial value when you declare it. With `tf.placeholder` you don't have to provide an initial value and you can specify it at run time with the `feed_dict` argument inside `Session.run`.\n", "> In short, we will use `tf.Variable` for trainable variables such as weights (W) and biases (B) for our model. On the other hand, `tf.placeholder` is used to feed actual training examples.\n", "\n", "Also note that, constants are automatically initialized when we call `tf.constant`, and their value can never change. By contrast, variables are not initialized when we call `tf.Variable`. To initialize all the variables in a TensorFlow program, we must explicitly call a special operation called `tf.global_variables_initializer()`. Things will become clearer with the example below." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "# Parameters\n", "learning_rate = 0.01 # learning rate for the optimizer (gradient descent)\n", "n_epochs = 1000 # number of iterations to train the model\n", "display_epoch = 100 # display the cost for every display_step iteration" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "# make up some trainig data\n", "X_train = np.asarray([3.3, 4.4, 5.5, 6.71, 6.93, 4.168, 9.779, 6.182, 7.59, \n", " 2.167, 7.042, 10.791, 5.313, 7.997, 5.654, 9.27, 3.1], dtype = np.float32)\n", "y_train = np.asarray([1.7, 2.76, 2.09, 3.19, 1.694, 1.573, 3.366, 2.596, 2.53, \n", " 1.221, 2.827, 3.465, 1.65, 2.904, 2.42, 2.94, 1.3], dtype = np.float32)\n", "\n", "# placeholder for the input data\n", "X = tf.placeholder(tf.float32)\n", "Y = tf.placeholder(tf.float32)\n", "\n", "# give the model's parameter a randomized initial value\n", "W = tf.Variable(np.random.randn(), tf.float32, name = 'weight')\n", "b = tf.Variable(np.random.randn(), tf.float32, name = 'bias')\n", "\n", "# Construct the formula for the linear model\n", "# we can also do\n", "# pred = tf.add(tf.multiply(X, W), b)\n", "pred = W * X + b\n", "\n", "# we then define the loss function that the model is going to optimize on,\n", "# here we use the standard mean squared error, which is sums the squares of the\n", "# prediction and the true y divided by the number of observations, note\n", "# that we're computing the difference between the prediction and the y label\n", "# from the placeholder\n", "cost = tf.reduce_mean(tf.pow(pred - Y, 2))\n", "\n", "# after defining the model structure and the function to optimize on,\n", "# tensorflow provides several optimizers that can do optimization task\n", "# for us, the simplest one being gradient descent\n", "optimizer = tf.train.GradientDescentOptimizer(learning_rate)\n", "train = optimizer.minimize(cost)\n", "\n", "# initializing the variables\n", "init = tf.global_variables_initializer()" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch: 100, cost: 0.58211350440979\n", "Epoch: 200, cost: 0.41723793745040894\n", "Epoch: 300, cost: 0.3158383071422577\n", "Epoch: 400, cost: 0.2534768581390381\n", "Epoch: 500, cost: 0.21512417495250702\n", "Epoch: 600, cost: 0.19153699278831482\n", "Epoch: 700, cost: 0.17703068256378174\n", "Epoch: 800, cost: 0.1681092530488968\n", "Epoch: 900, cost: 0.16262249648571014\n", "Epoch: 1000, cost: 0.15924812853336334\n", "Optimization Finished!\n", "Training cost: 0.15924812853336334, W: 0.2810680568218231, b: 0.5901336073875427\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAALSCAYAAADa01JpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl4VOXd//HPmYQkBAiEVRAlICIQikoUEDFBh926PBVFcUMiqSturUWUAEGRam21qK2kgFgVfj7FavsgsgxCZBEFFCGyKAjKIjsMSyDLnN8fk0RwziSTyeRMJnm/rmuuwPmec+ab0bR+ct/nvg3TNAUAAAAAAMrnCHcDAAAAAABECkI0AAAAAAABIkQDAAAAABAgQjQAAAAAAAEiRAMAAAAAECBCNAAAAAAAASJEAwAAAAAQIEI0AAAAAAABIkQDAAAAABAgQjQAAAAAAAEiRAMAAAAAECBCNAAAAAAAASJEAwAAAAAQIEI0AAAAAAABIkQDAAAAABCg6HA3UF0YhvG9pARJ28PcCgAAAAAg9JIkuU3TbFuZmxCif5ZQt27dxp06dWoc7kYAAAAAAKG1ceNG5eXlVfo+hOifbe/UqVPjNWvWhLsPAAAAAECIpaSkaO3atdsrex+eiQYAAAAAIECEaAAAAAAAAkSIBgAAAAAgQIRoAAAAAAACRIgGAAAAACBAhGgAAAAAAAJEiAYAAAAAIECEaAAAAAAAAkSIBgAAAAAgQNHhbiDSnTp1Sm63W8eOHVNBQYFM0wx3SwD8MAxDderUUYMGDZSQkKC4uLhwtwQAAIAIQ4iuhOPHj2vnzp0EZyBCmKap/Px8HTx4UIcOHVLr1q1Vv379cLcFAACACEKIDtKpU6dKA3RCQoISExMVFxcnh4MZ8kB15fF4dOrUKR0+fFhut1s7d+5U27ZtFRsbG+7WAAAAECEI0UFyu92lAbpVq1YyDCPcLQEoh8PhUHx8vOrWrSvJ+3N89OhRNW/ePMydAQAAIFIwbBqkY8eOSZISExMJ0ECEMQxDiYmJkn7+WQYAAAACQYgOUkFBgSSxMBEQoUp+dkt+lgEAAIBAEKKDVLKYGM9AA5GpZAYJCwMCAACgIkiAAGolHsMAAABAMAjRAAAAAAAEiBANAAAAAECACNEAAAAAAASIEA0AAAAAQIAI0ag1kpKSZBiGlixZEtL7GoYhwzC0ffv2kN43WOPHj5dhGBo+fHhE3RsAAACIBIRohN2hQ4f0/PPP66qrrtI555yjmJgYtWjRQr1799akSZN08ODBcLeIKrR9+3aNHz9eL7/8crhbAQAAAMoVHe4GUAm5uZLLJbndUkKC5HRKycnh7qpC3n33XT344IM6cuSIJO++2w0bNtSBAwe0b98+LV++XC+++KJee+01DRs2rFLvdcEFFyguLk7x8fGhaL3URRddJEmqU6dOSO9bW2zfvl0TJkxQmzZt9Oijj4a7HQAAAKBMhOhI5HJJWVlSTo5vLTVVysz0Bupq7o033tD9998v0zSVkpKiiRMnyul0KiYmRgUFBVq8eLHGjh2rL774QnfccYeOHTum3/72t0G/n8vlCmH3P9u0aVOV3BcAAABA9cN07kgzbZrUv791gJa8x/v3l6ZPt7evCvryyy81atQomaapG264QStXrtSgQYMUExMjyTuqO2DAAK1YsUI33HCDTNPUqFGj9NVXX4W5cwAAAAC1GSE6krhcUkaG5PGUfZ7HI40c6T2/mnrmmWeUn5+vVq1a6a233vI7FTo6OlozZ85Uy5YtlZ+fr7Fjx/qcc+aCYbt27dIDDzygdu3aKTY2VpdcconleVa++eYbDR06VM2bN1fdunXVsWNHjRs3TqdOnSpzQS1/C4v98pqZM2eqR48eatCggRISEnT11Vdr4cKFfj+jnJwcPfLII+rRo4datWqlmJgYNW/eXAMHDtS//vUvv9dV1ubNm3Xbbbed9TlMmDBBp0+fLvO6LVu2KCsrS9dcc43atm2ruLg4NWrUSD179tRLL72kvLw8n2uSkpJ09dVXS5J27NhR+lmWvN58883Scw8cOKDXX39dN9xwgzp27KgGDRqoXr166ty5sx5//HHt3r07pJ8DAAAAAlNQ5NE/P9uhf362QwVF5WSVGoDp3JEkK6v8AF3C45EmTqyW07p37typefPmSZIeeughJSQklHl+w4YN9dBDD+npp5/W3LlztXPnTrVu3drnvC1btujmm2/WgQMHFB8fX6FnlBctWqTrrrtOp06dkiQlJCTo+++/V1ZWlhYsWKA+ffoE/g1auPfeezVt2jRFRUWpXr16crvdWrJkiXJycvTee+/ppptuOuv848ePKy0trfTvDRo0UN26dbV//37Nnz9f8+fPV0ZGht54441K9fVLOTk5GjRokE6ePCnp589h/Pjxmj9/fpmfw7Bhw7RmzRpJUlxcnOrVq6fDhw9r1apVWrVqlWbPnq3FixerQYMGpdc0a9ZMbrdbhw8flsPhULNmzc66Z926dUv/PHnyZL300kuSvL9cSUhI0NGjR7Vx40Zt3LhRb7/9thYtWqSuXbuG6uMAAABAOdbvPKrrXl1W+vf8Qo/Se7cNY0dVj5HoSJGb638Ktz9Ll3qvq2aWLl0q0zQlSTfeeGNA15ScZ5qmcvx8Dk888YRatmyp5cuX68SJEzp+/HhAI7YHDhzQrbfeqlOnTql79+5av369jh49quPHj+udd97Rhg0b9Pe//z3A787Xhx9+qHfeeUd/+9vf5Ha7dfToUW3btk2pqanyeDx6+OGHVVhYeNY1DodDQ4YM0b///W8dPHiw9LrDhw/r1VdfVf369TV16lT97//+b9B9/dLhw4d188036+TJk+rWrZu++uqr0s9h5syZWrdunV5//XW/1/fo0UP/+Mc/tH37duXl5engwYPKy8vTf/7zH3Xo0EGrV6/W6NGjz7rmiy++0Pvvvy9JOu+88/TTTz+d9Ro6dGjpueeff74mTZqkr7/+uvT+p0+f1urVqzVgwADt379fw4YNK/13CwAAAFVr7AcbzgrQktQgtuaP09b877CmCHZqtstV7Vbs/uabbyRJsbGxpStbl6djx46KiYlRfn6+Nm7caHlOdHS0Fi5cqBYtWpQea9++fbn3njJlig4ePKjmzZtr/vz5atSokSTvc9nDhg1TdHT0WWGuoo4cOaK3335bt99+e+mxtm3batasWWrbtq327NmjFStWKDU1tbQeHx9vGZAbNWqkBx98UAkJCbrrrrv0+uuv6+abbw66tzO9+uqr2rdvn5o0aaL58+eradOmkryfw1133SWHw6E777zT7/Wvvfaaz7HY2Fhdd9116tKlizp06KA333xTL774YlArpI8aNcrnWFRUlFJSUvThhx+qW7duys3NVU5Ozlmj+AAAAAit46cL1WXcfMvakBTfGaM1DSPRkcLttve6KnTo0CFJUmJiohyOwP4VdDgcSkxMlCS/+0bfddddZwXoQJWMhGZkZJQG6DPdcsstateuXYXvW+L888+33J6rVatW6t69uyRpw4YNFbrnddddJ0n67LPPVFRUFHRvZyoZtR85cmRpgD7T7bffrjZt2gR177Zt2yo5OVknT56sksXhYmNj1a9fP0nS8uXLQ35/AAAAeC3etNcyQD/a90Jtn3ytHA4jDF3ZixAdKcp5bjjk10WgK664osLXnD59unRkvHfv3n7PK6tWnssuu0yGYf0/Jueee64k71TqXyosLNS0adM0cOBAtWzZUrGxsaULbpX8QuHUqVOW11ZUfn6+coun/vsbxTUM46zRcisLFy7UbbfdpgsuuEDx8fFnLRK2bt06SarUAmCbNm3SQw89pK5duyohIUEOh6P0/q+88kql7w8AAAD/7vjHKo14c7XPcdcTaXq0b4cwdBQeTOeOFMEuEFYNFxZr3LixJG9w9Hg8AY1Gezye0rBYcv0v/XJRqkCU9CBJLVu29Hteq1atKnzvEmcupPVLcXFxkqSCgoKzjh8/frx0i68SdevWVbNmzUo/r71790qSTpw4YTlyXBGHDh0qHdEu63stCf1WRo0apSlTppT+vU6dOmrcuHHpAm+HDh1SQUGBTpw4EVSPs2fP1l133VX6WTkcDjVs2FCxsbGSvJ/ZiRMngr4/AAAArO11n1KPSb6PlyY1idfiJ/rUitHnMzESHSmSk6VyRgF9pKVVu+ehJalTp06SvKPAmzdvDuiaTZs2KT8/X5LUuXNny3OioqJC02A1MHHiRK1YsUJNmzbVzJkztXfvXp08eVL79u3TTz/9pF27dpWeWx0W0po3b56mTJmiqKgojR8/Xt99951Onz6tgwcPli4S1qNHD0nB9bt//36NHDlSBQUFGjp0qFavXl06Cl9y/8ceeyzo+wMAAMDau6t+sAzQLwzpqiW/v7rWBWiJkejIkpkp9e8f2DZXDodksadyddCnTx8ZhiHTNPXBBx+UhuqyfPDBB5ICm1JcESXPZXs8Hu3Zs8fv9kh79uwJ2XsGomRRsSlTpujWW2/1qZeMQodK48aNFRUVpaKiIu3evdvv5+BvqnRJv/fee6/GjRtneU5lep43b56OHz+uzp07691337WcvRDqzwQAAKA2K/KY6v3Hxdpz9JRP7Yun+6pZg9gwdFU9MBIdSZxOaepUb0Aui8MhZWdXy6ncktS6dWsNGjRIkndFaHc5i5+53W69+uqrkqTBgwdb7hEdrNjY2NKR7WXLlvk979NPPw3ZewZi586dkqRLL73Usr5o0aKQvl9MTIySi2ct+NtCrKztxcrrd8eOHfruu+8sayWBuKwR5JL7d+3a1TJAm6apxYsX+70eAAAAgft27zFdMOYjnwB99UXNtH3ytbU6QEuE6MiTni4tWOCdqm0lLc1bHzHC3r4qKCsrS3Xq1NHu3bvPes71lwoLC3X33Xdrz549qlOnjrKyskLey//8z/9IkrKzs3X06FGf+pw5c7Rt27aQv29ZGjZsKElav369T+348eN67rnnQv6eJVtlZWdnl66gfqbZs2dr+/btlteW1a8kjRkzxm9ITihe/M7qs//l/Tds2GB5n+zsbG3dutXv9QAAAAjMn+ZvVr+/+A6cvDWiu2bc0z0MHVU/hOhI5HRKS5ZIGzZIr7wiTZzo/bphg/d4NR2BPlNKSor+8pe/SJI+/PBD9erVSx9//HFpmC4sLNSCBQt05ZVXlk7lfvnll9WtW7eQ9/Lwww8rMTFRe/fu1aBBg0pXqS4sLNTs2bN1zz33WG59VZVKtmt6/PHHtXTp0tLg+MUXX8jpdPrd5qsyHnzwQTVv3lwHDhzQgAED9PXXX0vyLnr29ttva+TIkaVh1l+/b7zxhqZPn176/PoPP/ygu+++W7NmzSpdUfyXLrzwQtWpU0dHjx7VnDlzLM/p27evDMPQhg0bNGrUKB05ckSSd5bCiy++qAcffFBNmjSp1PcPAABQm50qKFLS6Ll69RPf2YO5EwYotUPFF/GtqQjRkSw5WRo1SnrmGe/XariIWFkefPBBvfXWW2rYsKFWr16tQYMGKS4uTk2aNFFcXJwGDBigzz//XAkJCXrrrbf0wAMPVEkfzZo106xZsxQbG6uVK1eqS5cuatSokerXr6/bbrtNXbt21X333SdJpStBV7Vnn31WTZs21Y8//qg+ffooPj5e9evXV/fu3bV+/Xq9++67IX/PxMREvffee6pbt65Wr16tiy++WI0aNVKDBg105513qmvXrrr//vstrx0+fLh69uypwsJCpaenKz4+XomJiWrTpo3eeustTZgwwe9z1vXq1dNtt90mSRoyZIgaNWqkpKQkJSUlle5dfdFFF+nRRx+V5H0EIDExsfT15JNPyul0lv4zAgAAQMV8tu2gOo792Of4iCvbavvka1UvlqW0zkSIRljdeeed2rp1q5577jldeeWVatKkiY4dO6bGjRurV69emjhxorZu3ao777yzSvsYMGCAVq9erSFDhqhJkyY6ffq02rZtqwkTJsjlcikvL0+SbBuRbteunT7//HPdcccdat68uYqKitSoUSPdfvvt+uKLL9S/f/8qed+0tDR9+eWXGjp0qJo1a6bTp08rKSlJ48eP1+LFi/3+EiEmJkaLFi3S6NGj1a5dOzkcDkVHR6tfv37673//q7HlLHL397//XU899ZQ6duyo06dPa8eOHdqxY4eOHz9ees6f//xnTZ06VZdeeqliY2NVVFSkSy+9VC+//LLmzp2r6Gj+xx0AAKCiHnx3rW6d+pnP8f97uLcyr7PeFae2M9gOxsswjDXdunXrtmbNmoDO37hxoyQFtLI0It9VV12lZcuWacaMGRo+fHi420GI8HMMAABqq8Mn8nXpxIU+xxvWraM1z/RVdFTNG29NSUnR2rVr15qmmVKZ+zB0A5Rj5cqVWrZsmRwOh5wR8Lw5AAAAUJb/rNutUbO+9Dme+evOGtG7bRg6iiyEaEDS1KlTdeDAAQ0dOlRJSUmKiorS8ePH9f777+uxxx6TJN1yyy0677zzwtwpAAAAEBzTNHXtX5fpmz2+W8wu+8PVap0YH4auIg8hGpB3FennnntOTz/9tKKiotSwYUMdOXJEHo9HknTJJZdoypQpYe4SAAAACM4PB08q9cVPfI5fen4jvX9/LxmGEYauIhMhGpB06623Ki8vT0uXLtXOnTt16NAhJSQkqHPnzhoyZIjuu+8+1a1bN9xtAgAAABX2xtKten7eJp/jf7u9mwb9qmUYOopshGhAUpcuXfTSSy+Fuw0AAAAgZAqKPOoybr5OF3p8ausy+6thfJ0wdBX5CNEAAAAAUMOs+/GIbnhtuc/x33Q7V3++5ZIwdFRzEKIBAAAAoAZ5+t/r9c6qH3yO/+u+K3RZUuMwdFSzEKIBAAAAoAY4dqpAvxq/wLK2+dmBio2OsrmjmokQDQAAAAARzrVxr9JnrvY5/ljfDnqk74Vh6KjmCkmINgzjMkk3SLpcUntJzSTFSTogabWkGaZpfhDEfYdLmlHOaSdM06xf0XsDAAAAQE1w+z8+0/LvDvocdz2RpguaEZVCLVQj0fdK+u0Zfz8uySOplaTrJV1vGMYcSbeZplkQxP0LJB3yUzsRxP0AAAAAIKLtdZ9Sj0kun+Ntm9aT6/E0ORzs/VwVQhWiV0raJClH0hbTNI9LkmEY50l6WNLvJd0kabSkiUHcf4Vpmn1C0yoAAAAARLZ3V/2gMf9e73P8xSFddfNl54Who9ojJCHaNM2Zfo7/KOlJwzBaSrpD0nAFF6IBAAAAoNYr8pjq/cfF2nP0lE/ti6f7qlmD2DB0Vbs4bHqfL4q/trLp/QAAAACgRtmy95guGPORT4Du26m5tk++lgBtE7tW5+5V/PV7m94PAAAAAGqMFz7epNeXbPU5/taI7krt0CwMHdVeVTYSbRhGfcMwuhqG8ZqkocWHXw3ydsmGYeQahpFnGMYxwzA2GIbxF8Mw2oaoXUSI1q1byzAMLVu2LNytBK13794yDENvv/12ha674447ZBiGnn322bOOFxYWyjAMGYahnTt3hrJVAAAAhNmpgiIljZ5rGaC/yRpAgA6DkIZowzBaG4ZhGoZhSjomaZ2kBySdkjTWNM3Xg7x1U0mdJJ2Ud+usZEmPSso1DGNYBXtcY/WS1DHI3hCE4cOHlwa/sl4vv/xyQPfbtm2bxo8fr7/+9a9lnjd9+nSNHz9eX3/9dSi+DQAAAKDKrNx6UB3HfuxzPL13W22ffK3iY+yaWIwzhfpTL5K0t/jPiZJiJBVKel7Sa0Hcb7ekcZLmSPrWNM18wzBiJTklvSips6SZhmHsNE0zp7LNw3516tRR48aN/dbr1at31t/bt2+v+vXrKz4+/qzj27Zt04QJE3TBBRdo1KhRfu83ffp0LV++XO3bt1fXrl0r13w1YRiGLrroIknezxMAAACR78F312ru13t8js8d1VvJrRqGoSOUCGmINk1zj6RzJMkwDIek9pL+IGmCpHTDMAabpplbgfstkLTgF8dOS/rIMIzlklYXv8dk/fzcdXn3TLE6Xjwa3S3Q3hAavXr10pIlSwI+vyLn1hZRUVHatGlTuNsAAABACBw+ka9LJy70Od4ovo5WP91X0VF2rQ0Nf6ps/N80TY+kLfKG5yOSHpf0T8MwLiuuVfb+Rw3DmCRpuqSehmE0NU3zQGXvCwAAAADh8OFXu/TI7K98jo+7rrPuubIaLweVmyu5XJLbLSUkSE6nlJwc7q6qjF2/xphS/PXS4leorCr+akiqxv9WIVSsFhZr3bq1+vXrJ0naunWrz3PVb7/9tv7xj3/IMAwtX75cknTnnXeedU779u193is/P19//etf1bt3bzVu3FixsbFKSkpSenq6Nm/eXGafc+fO1dVXX62EhAQlJCSoV69eeuedd0L4SfysrIXFnnnmGRmGoXvvvVeSNGPGDHXv3l3169dXw4YN5XQ65XK5yrx/ZT4HAAAAlM80TQ18OccyQC8ffU31DdAul5SWJnXpIj3yiDR2rPdrly7e4+X8d2aksutJ9F1n/PkCSWtsel/UAs2bN9fJkyd1+PBhRUVFqWnTpmfV69atK4fDoRYtWujQoUMqKChQw4YNFRcXV3pOs2Znr2q4a9cuDRo0SOvXr5fknTIdHx+vHTt2aPr06Zo1a5Zmz56t66+/3qef559/XmPGjJHkfV65YcOGWrVqlVauXBnWBc2GDx+umTNnKjo6WnXr1pXb7dbixYu1ZMkSvf/++7rhhht8rqnM5wAAAIDy/XDwpFJf/MTneEqbRP3rvitkGEYYugrAtGlSRobk8TPJOCdH6t9fys6WRoywt7cqZtdI9Jm/Ojkewvv2OOPP20N4X0SQtWvX6r333pMkJSUl6aeffjrrddNNN2nYsGH66aef1L17d0nSq6++etY5K1euLL1ffn6+rr/+eq1fv179+vXTypUrlZeXJ7fbrV27dmnUqFHKy8vTsGHDtH379rN6WbJkSWmAvuuuu7Rnzx4dPnxYBw4c0BNPPKEXXnihNJDaac6cOXrvvff0xhtv6OjRo3K73dq6dat69+4tj8ejhx56SEVFRWddU5nPAQAAAOX725KtlgH673d005z7e1XfAO1ylR2gS3g80siRNW5EutIh2jCMKKP8f7q/L/5aKGllWSeecd8y72kYRoKk0cV//dw0zf2B3BfVy4oVK3TOOedYvu65556w9DR9+nStXbtWffr00UcffaSePXuWrnrdqlUrvfLKK0pPT9eJEyd8tuAaN26cJKlv375688031aJFC0lSYmKi/vSnP+nuu++W2+229xuSdOTIEc2YMUMZGRmlK5u3a9dOs2bNUp06dbRz506tWrXqrGsq8zkAAADAv4Iijzo8M09//Nh3cdh14/prYJeWYeiqArKyyg/QJTweaeLEqu3HZqGYzn2epDmGYbwmaYFpmjul0tW5u8oboEv2cp5imubhkgsNw0iS9H3xX+8xTfPNM+7bxjCM2ZKyJS00TfOH4mtiJF0j7xZXHSR5JD0Vgu+jSiSNnhvuFkJm++RrQ37PgoIC7d2717J2+PBhy+NVbebMmZKkRx99VNHR1j8it99+u6ZNm6aFC39eOXHfvn3KyfHutDZ69GjL3xw+/fTTpfe3U7t27TR06FCf461bt1ZKSoo+++wzbdiwQb16/bzIfbCfAwAAAPz76scjuvG15T7Hh6S01p9uvjgMHVVQbq53qnZFLF3qva6GLDYWqmeiu0maJkmGYZySd8p2A0mxZ5zzpqQnK3jfHsWvkvuekJQgqWQz3JOS7jNNc3GwjSO80tLSqtW2Vfn5+VqzxvvI/r333qvf/va3lueVTH3+8ccfS499+eWXkrzPDV955ZWW11144YVq2bKl9uzx3fOvKl122WV+a+eee66ks39pUZnPAQAAANbG/Hu93l31g8/xOfdfoZQ2jcPQURCCnZrtchGiz7Bb0lBJTkndJbWU1ETSKUlb5Z2+PcM0Td9ft5Rtr6RRknpLulhSM0kN5Q3S30pySfqbaZo7QvA9AJKkAwcOqKCgoPTP5Tl58mTpn/fv9z5R0Lhx47MWLfulc8891/YQ3aBBA7+1kl5Lvm+pcp8DAAAAznbsVIF+NX6BZW3zswMVGx1lc0eVEOyjiWF4pLGqVDpEm6aZL+m94ldFr90u7/ZUVrU8ebfGmmJVjxRVMQUaVcdzxrMd69evV5cuXcLYTfjwOQAAAISGa+Nepc9c7XP88X4dNMp5YRg6qqSEBHuvq4bsWp0biAhNmzaVw+H9sfjhB9+pNmUp2Sbr0KFDOn36tN/zdu/eHXyDNqnM5wAAAADv3s/Dsj+zDNCLn0iLzAAtSU6nvddVQ4Ro1Aglgc80zUqdFxcXp0svvVSSNG/evAr1UHJdUVGRli+3fnrhu+++i4gQXZnPAQAAoLb76egptX3qI63YevCs4xc0q6dtkwarXbP6YeosBJKTpdTUil2TllZjnoeWCNGoIRKKp4ccPXo0oPOOHDni95zhw4dLkqZNm6YNGzaUeb8zF+Nq3ry50tLSJEkvvPCCZVB//vnny7xfdRLs5wAAAFCbvf3ZDvV83nfxrT/dfLFcT/SRw1FN936uiMxMyRFglHQ4pLFjq7YfmxGiUSN06NBB0dHROnjwoD788EO/5yUX/wZszpw5fgN3RkaGLr/8cuXl5alPnz6aNm3aWXs779mzR2+//bauuuoqvfbaa2ddO378eEnS/PnzlZ6ern379knyhvY//OEPmj59emmQr+4q8zkAAADUNkUeUz0mLdIzH/gOPqx+pq+GpLQOQ1dVxOmUpk4tP0g7HFJ2do2ayi0RolFDJCQk6JZbbpEk3XjjjWrUqJGSkpKUlJSkDz74oPS8u+66S3Xq1NHSpUvVtGlTtW7dWklJSerTp0/pOTExMfrvf/+rnj176uDBg7r33nuVmJioJk2aqH79+mrVqpXuvPNOLVu2zGcv6D59+mjSpEmSpBkzZuicc85R48aN1aRJE73wwgt68skn9atf/arqP5AQqMznAAAAUJts/umYLhjzkfa6z14Xp2+n5to++Vo1rR/r58oIlp4uLVjgnaptJS3NWx8xwt6+bBCqfaKBsMvOztb4diaKAAAgAElEQVR5552nDz74QNu3b9eOHd7dz44fP156TnJyshYsWKDJkydr9erV2rNnjzwej6Kjz/5RaNGihZYtW6ZZs2bp3Xff1Zo1a3To0CHFxsaqU6dO6t69u37961/r+uuv9+njqaeeUteuXfXiiy9q7dq1KiwsVPfu3fXQQw/p9ttvV+/evav2gwihynwOAAAAtcEfP96kvy3Z6nP8n+ndddWFzcLQkY2cTu8rN9e7D7Tb7V2F2+msUc9A/5JR3kJMtYVhGGu6devWbc2aNQGdv3HjRklSp06dqrItAFWIn2MAABCsUwVF6jj2Y8vaN1kDFB/DeGV1k5KSorVr1641TTOlMvdhOjcAAAAAVMCKrQcsA/S9vdtq++RrCdA1HP90AQAAACBAD7yzRh+t/8nn+EejrlLnVpGxgCwqhxANAAAAAOU4dCJf3SYu9DneuF6MPh/jVHQUk3xrC0I0AAAAAJThw6926ZHZX/kcH39dZw2/sm0YOkI4EaIBAAAAwIJpmhr0yqfa9NMxn9qK0deoVaO6YegK4UaIBgAAAIBf2HHwhNJeXOJz/PKkRL332ytkGIb9TaFaIEQDAAAAwBleX/KdXvh4s8/xv9+RooFdzglDR6hOCNEAAAAAICm/0KMu4+Yrv8jjU1s3rr8a1q0Thq5Q3RCiAdRKpmmGuwUAABBKubmSyyW53VJCguR0SsnJAV/+1Y9HdONry32O33JZa70w5OJQdooIR4gOkmEYMk1TRUVFioqKCnc7ACrI4/H+hpnnmQAAiHAul5SVJeXk+NZSU6XMTG+gLsNT76/XrM9/8Dk+5/5eSmmTGKpOUUMQooMUFxenvLw8ud1uJSbygwVEGrfbLcn7swwAACLUtGlSRobk8Z1+LckbrPv3l7KzpREjfMrHThXoV+MX+Bw3DGnTxIGKjWawDL4I0UFKTExUXl6e9u7dq8LCQjVo0EAxMTEyDIORLaAaMk1TpmkqPz9fx44d04EDBySJX4IBABCpXK6yA3QJj0caOVJq0+asEelF3+zVvW+t9jn9iX4d9LDzwlB3ixqEEB2khIQEnTp1SocOHdKBAwdK/4McQORo3LixEhISwt0GAAAIRlZW+QG6hMcjTZwoOZ0yTVO3ZX+mz7Yd8jntk9/1Udum9ULcKGoaQnSQDMNQixYtVK9ePbndbp08eVKFhYUsVgRUY4ZhKDo6WvHx8UpISFD9+vXD3RIAAAhGbq71M9BlWbpUP32xTj3n7PQptW9eXwsfS2VGKQJCiK6k+vXr8x/iAAAAgJ1crgpf8s9LBmmsRYB+6eaLdVNK61B0hVqCEA0AAAAgshQvEBqIIsOhng+8qf31G/vUVj/TV03rx4ayM9QChGgAAAAAkSXANU02NW2jgemv+Rzv17mFsu+6LNRdoZYgRAMAAACILOXs+yxJz6cN1xs9h/gcfzu9h3pf2LQqukItQYgGAAAAEFmSk6XUVMvFxfKiY9XpiTmWl23MGqi6Mez9jMpxhLsBAAAAAKiwzEzJcXacWXF+V8sAnfH5+9reL44AjZBgJBoAAABA5HE6palTpYwMyeNR0h/+z/K0eW+OUqdJTwc0BRwIBCEaAAAAQGRKT9d3Tc9X35X5PqWmJw5r1fppinpnKgEaIUWIBgAAABCR7p7+uZZu8Q3QYxIOKOPOy6TkT8LQFWo6QjQAAACAiOLxmGo35iPL2pz7eymlTaLNHaE2IUQDAAAAiBhLt+zX3dM/t6x9//xgGYZhc0eobQjRAAAAACJCl3Hzdfx0oc/xISmt9aebLw5DR6iNCNEAAAAAqrUTpwuVPG6+ZW356Gt0bqO6NneE2owQDQAAAKDaenP59xr/328sa9snX2tzNwAhGgAAAEA1lTR6ruXxsb/urPTebW3uBvAiRAMAAACoVnYfyVOvyYsta7kTBqheLDEG4cO/fQAAAACqjSf/tU7vrd7pczw+JkrfZA0MQ0fA2QjRAAAAAMLONE21fcp67+cZ91yuqy9qbnNHgDVCNAAAAICwWvvDYf3m9RWWtW2TBsvhYO9nVB+EaAAAAABh0+/PS/XtvuM+x6+6sKn+md4jDB0BZSNEAwAAALBdfqFHHZ6ZZ1lb+FiqLmzRwOaOgMAQogEAAADY6sOvdumR2V9Z1tj7GdUdIRoAAACAbfzt/fzg1Rfo9wM62twNUHGEaAAAAABV7tCJfHWbuNCytnZsPzWuF2NzR0BwCNEAAAAAqtSL8zfptU+2WtaYvo1IQ4gGAAAAUGX8Td9+5dZLdMMl59rcDVB5hGgAAAAAIfft3mPq95ccy9qWZwcpJtphc0dAaBCiAQAAAITUndNW6dNvD/gcv7B5fS18PC0MHQGhQ4gGAAAAEBIej6l2Yz6yrL3/QC91Oz/R5o6A0CNEAwAAAKi0JZv3afiMLyxr3z8/WIZh2NwRUDUI0QAAAAAqpXPmxzqZX+Rz/JbLWuuFIReHoSOg6hCiAQAAAATlxOlCJY+bb1lbMfoatWpU1+aOgKpHiAYAAABQYdOWfa+J//eNZY29n1GTEaIBAAAAVIi/vZ/HXddZ91zZ1uZuAHsRogEAAAAEZNeRPF05ebFlLXfCANWLJV6g5uPfcgAAAADl+t3/rtO/1uz0OV4/NlobJgwIQ0dAeBCiAQAAAPhlmqbaPmW99/PMEd2V1qGZzR0B4UWIBgAAoZGbK7lcktstJSRITqeUnBzurgBUwpodh3XT31ZY1rZNGiyHg72fUfsQogEAQOW4XFJWlpST41tLTZUyM72BGkBEcb60RFv3n/A53ueiZnrznu5h6AioHgjRAAAgeNOmSRkZksdjXc/Jkfr3l7KzpREj7O0NQFDyCz3q8Mw8y9qix1PVvnkDmzsCqhdHuBsAAAARyuUqO0CX8HikkSO95wOo1v795U6/AXr75GsJ0IAYiQYAAMHKyio/QJfweKSJE5nWDVRj/vZ+fujq9vrdgIts7gaovgjRAACg4nJzrZ+BLsvSpd7rWGwMqFZ2Hj6p3n/8xLL25dh+SqwXY3NHQPVGiAYAABUX7NRsl4sQDVQj17+6TF/vPGpZ2z75Wpu7ASIDIRoAAFSc223vdQBCzt/07QevvkC/H9DR5m6AyEGIBgAAFZeQYO91AEJm5daDui37M8vapokDFVcnyuaOgMhCiAYAABUX7AJhLCwGhJW/0WeJ6dtAoNjiCgAAVFxyspSaWrFr0tJ4HhoIk8Iij98A/fc7UgjQQAUQogEAQHAyMyVHgP8p4XBIY8dWbT8ALE1f9r3aP2299/P3zw/WwC7n2NwRENmYzg0AAILjdEpTp0oZGWXvF+1wSNnZTOUGwsDf6HOzBrH64um+NncD1AyMRAMAgOClp0sLFninaltJS/PWR4ywty+gljt8It9vgF7wWCoBGqgERqIBAEDlOJ3eV26udx9ot9u7CrfTyTPQQBg8+M5azV2/x7LGs89A5RGiAQBAaCQnE5qBMPM3+vybS8/Vn4deYnM3QM1EiAYAAAAi3De73Rr8108ta+vG9VfDunVs7giouQjRAAAAQATrMm6+jp8utKwxfRsIPUI0AAAAEIFM01Tbpz6yrD17Yxfd0bONzR0BtQMhGgAAAIgw/123Ww/P+tKytnXSYEU5DJs7AmoPQjQAAAAQQfwtHiYxfRuwAyEaAAAAiAB5+UXqlPmxZe1f912hy5Ia29wRUDsRogEAAIBq7rm53yj70+8ta4w+A/YiRAMAAADVmL/p292TGuu9+66wuRsAhGgAAACgGtp5+KR6//ETy9pnTzl1TsM4mzsCIBGiAQAAgGpn8Cuf6ps9bssa07eB8CJEAwAAANWIv+nbo65pr8f7X2RzNwB+iRANAAAAVAMrvjugYf9YZVnbNHGg4upE2dwRACuEaAAAACDM2PsZiByEaAAAACBMCos8av/0PMvaG3emaEDyOTZ3BKA8hGgAAAAgDP7x6TY9O3ejZe375wfLMAybOwIQCEI0AAAAYDN/07dbJMRq1Zi+NncDoCII0QAAAIBNDp3IV7eJCy1rix5PVfvmDWzuCEBFEaIBAAAAG9z3zzX6OPcnyxqLhwGRgxANAAAAVDF/07eHpLTWn26+2OZuAFQGIRoAAACoIrm7j+ravy6zrH09vr8S4urY3BGAyiJEAwAAAFWg09iPlVdQZFlj+jYQuQjRAAAAQAiZpqm2T31kWZv0P7/SsB7n29wRgFAiRAMAAAAh8uFXu/TI7K8sa9smDZbDwd7PQKQjRAMAAAAh4G/xMInp20BNQogGAAAAKiEvv0idMj+2rM25v5dS2iTa3BGAqkSIBgAAAIKU9d9vNH3595Y1Rp+BmokQDQAAAATB3/Ttnu0aa3bGFTZ3A8AuhGgAAACgAn48dFJXvfCJZW3VGKdaJMTZ3BEAOxGiAQAAgAANfDlHm346Zllj+jZQOxCiAQAAgAD4m779iPNCPdavg83dAAiXkIRowzAuk3SDpMsltZfUTFKcpAOSVkuaYZrmB5W4/zmSnpL0a0nnSjoq6XNJL5um6apc9wAAAIB/y749oDumrbKsbX52oGKjo2zuCEA4hWok+l5Jvz3j78cleSS1knS9pOsNw5gj6TbTNAsqcmPDMLpKWiypSfEht6Sm8gbqaw3DGGOa5uRK9g8AAAD4YO9nAL/kCNF9Vkp6TFKKpAamaTYwTbOupPMlvVh8zk2SRlfkpoZh1JX0H3kD9JeSupim2VBSoqSXJBmSJhmG0T8k3wUAAAAgqbDI4zdAZ991GQEaqMVCMhJtmuZMP8d/lPSkYRgtJd0habikiRW49W8ltZF3ZPs60zR3Fd/XLel3hmFcIOlGSc9LWhD0NwAAAAAUm5qzVZM+2mRZIzwDsGthsS/kDdGtKnjd7cVf3y0J0L/worwhupthGBeZprm5Ej0CAACglvM3+nxuo7paPvoam7sBUB3ZFaJ7FX/9PtALDMNoIO/0cEma7+e0z+RdZKyhJKckQjQAAAAq7ODx00p5dpFlbdHjaWrfvL7NHQGorqosRBuGUV9SO3mnZA8tPvxqBW7RSd5nniUp1+oE0zQ9hmFsltRdUucA+1rjp9SxAr0BAACghsh4a7UWfLPXssb0bQC/FNIQbRhGa0k/WpROSXrONM3XK3C7lmf8eXcZ55XUWpZxDgAAAODD3/TtoZedpz8O6WpzNwAiQahHoosklfwaL1FSjKRCeRf+eq2C96p3xp/zyjjvZPHXgObYmKaZYnW8eIS6W2CtAQAAIJJt2HVUv56yzLK2fnx/NYirY3NHACJFSEO0aZp7JJ0jSYZhOCS1l/QHSRMkpRuGMdg0Tcup2QAAAIAdOjwzT/mFHssa07cBlKfKnok2TdMjaYu84fmIpMcl/dMwjMuKa+U5ccaf60o65ue8+OKvx4NuFgAAADWeaZpq+9RHlrXJv/mVbu1+vs0dAYhEDpveZ0rx10uLX4E48znosrbGKqntqWhTAAAAqB0++HKX3wC9bdJgAjSAgNm1xdWZezxfIMnfCtln2iTJlHeF7mRZbF9VPGX8ouK/flPJHgEAAFAD+Vs8TGL6NoCKs2skuu0Zfw5o2rVpmsckrS7+az8/p/WQd49oSXIF1xoAAABqopP5hX4D9Jz7exGgAQSl0iHaMIwowzCMck77ffHXQkkrK3D7d4u/3m4YhtUWVr8r/rrGNE2fkWoAAADUTuP/k6vOmfMta9snX6uUNok2dwSgpgjFdO7zJM0xDOM1SQtM09wplU617ipvgB5WfO4U0zQPl1xoGEaSpO+L/3qPaZpv/uLeb0h6VFIbSf9nGMadpml+YxhGA0ljJf2m+LwxIfg+AAAAUAP4G32+sn0TvXNvT5u7iSC5uZLLJbndUkKC5HRKycnh7gqodkL1THQ3SdMkyTCMU/JO2W4gKfaMc96U9GRFbmqaZp5hGDfIO1W7m6RcwzDc8u4J7ZD3mekxpmkuqOw3AAAAgMj2w8GTSn3xE8va52Ocap4QZ3NHEcLlkrKypJwc31pqqpSZ6Q3UACSFJkTvljRUklNSd0ktJTWRdErSVnmnb88wTXN5MDc3TXOdYRhdJD0l6deSzpV0UNLnkv5imibPQgMAANRy/f+yVFv2Wi+9w7PPZZg2TcrIkDx+dqDNyZH695eys6URI+ztDaimKh2iTdPMl/Re8aui126Xd/Xt8s77SdIjxS8AAACglL/p24/366BRzgtt7iaCuFxlB+gSHo80cqTUpg0j0oDs2+IKAAAACKmcLft11/TPLWtbnh2kmGi7NqKJUFlZ5QfoEh6PNHEiIRoQIRoAAAARiL2fKyk31/oZ6LIsXeq9jsXGUMsRogEAABAxCoo8uvDpeZa16cMv0zUdW9jcUYRyBbmskMtFiEatR4gGAABARHhj6VY9P2+TZY3R5wpyu+29DqhBCNEAAACo9vxN3z6vcV19+uQ1NndTAyQk2HsdUIMQogEAABCY3FzvdF632xumnM4qn9p74PhpXfbsIsva4ifS1K5Z/Sp9/xor2AXCWFgMIEQDAACgHC6XdyVnq4WoUlOlzMwqCVfpb34h16Z9ljWmb1dScrL3n11FFhdLS+N5aEAS6/4DAADAv2nTpP79/YetnBxvffr0kL5t0ui5lgH6tu7nEaBDJTNTcgQYBxwOaezYqu0HiBCEaAAAAFhzuaSMjPL3EvZ4pJEjg1/x+Qzrdx71+/zz+vH99fxvulb6PVDM6ZSmTi0/SDscUnY2U7mBYoRoAAAAWMvKKj9Al/B4pIkTK/V2F4z5SNe9usyytn3ytWoQV6dS94eF9HRpwQLvVG0raWne+ogR9vYFVGM8Ew0AAABfubkVe15WkpYu9V5XwedmTdNU26c+sqy9cFNX3XL5eRXrAxXjdHpfYVg4DohEhGgAAAD4CnZqtstVoeD1/tqdevy9dZa1bZMGy+EwgusDFZecTGgGAkCIBgAAgC+3u8qv8/fss8Tq2wCqL0I0AAAAfCUkVNl1J04XKnncfMvavx/opUvPTwzuvQHABoRoAAAA+Ap2JeZyrsv8cIPeWrnDssboM4BIQIgGAACAr+RkKTW1YouLpaWV+Uytv+nbV13YVP9M71HRDgEgLAjRAAAAsJaZKfXvH9g2Vw6HNHasZWnHwRNKe3GJZe3zp51q3iCuEk0CgL0I0QAAALDmdEpTp0oZGWUHaYdDys62nMrtfGmJtu4/YXkZ07cBRCJHuBsAAABANZaeLi1Y4J2qbSUtzVsfMcKnlDR6rmWA/l3/DgRoABGLkWgAAACUzen0vnJzvftAu93eVbidTstnoJdu2a+7p39ueastzw5STDTjOAAiFyEaAAAAgUlOLnPhMIm9nwHUfIRoAAAAVFpBkUcXPj3PsjZj+OW6umNzmzsCgKpBiAYAAEClvL7kO73w8WbLGqPPAGoaQjQAAACC5m/6dpsm8Vr6+6tt7gYAqh4hGgAARL4AF7xC6Ow/dlqXP7fIsvbJ7/qobdN6NncEAPYgRAMAgMjlcklZWVJOjm8tNVXKzLTcuxiVc8+Mz/XJ5v2WNaZvA6jp2F8AAABEpmnTpP79rQO05D3ev780fbq9fdVwSaPnWgboYT3OJ0ADqBUYiQYAAJHH5ZIyMiSPp+zzPB5p5EipTRtGpCspZ8t+3eVn7+cNEwaofiz/WQmgduB/7QAAQOTJyio/QJfweKSJEwnRlcDezwDwM6ZzAwCAyJKb638Ktz9Ll3qvQ4V4PKbfAD3xhmQCNIBaiZFoAAAQWVyu4K9jxe6Avbxoi15e9K1lbdukwXI4DJs7AoDqgRANAAAii9tt73W1ENO3AcA/QjQAAIgsCQn2XleLHD1ZoIuzFljWZo7orrQOzWzuCACqH0I0AACILMEuEMbCYmW68bXl+urHI5Y1Rp8B4GcsLAYAACJLcrKUmlqxa9LSeB66DEmj51oG6HoxUQRoAPgFRqIBAEDkycyU+vcPbJsrh0MaO7bqe4pAG3Yd1a+nLLOsLfvD1WqdGG9zRwBQ/RGiAQBA5HE6palTpYyMsoO0wyFlZzOV2wKLhwFAcJjODQAAIlN6urRggXeqtpW0NG99xAh7+4oA/gL0by49lwANAOVgJBoAAEQup9P7ys317gPtdntX4XY6eQbawntf/Kgn53xtWdv87EDFRkfZ3BEARB5CNAAAiHzJyYTmcjB9GwBCgxANoPZgpApALXSqoEgdx35sWXthSFfdctl5NncEAJGNEA2g5nO5pKwsKSfHt5aa6l3ll0WHANRAj/2/r/TvL3dZ1hh9BoDgEKIB1GzTppW9em9OjnebnOxsFh8CUKMwfRsAqgYhGkDN5XKVv/2N5K2PHCm1acOINICIt/PwSfX+4yeWtbmjeiu5VUObOwKAmoUQDaDmysoqP0CX8HikiRMJ0QAiWufMj3Uyv8iyxugzAIQG+0QDqJlyc62fgS7L0qXe6wAgAiWNnmsZoC89vxEBGgBCiJFoADWTyxX8dazYDSCCLNm8T8NnfGFZW5fZXw3j69jcEQDUbIRoADWT223vdQAQBiweBgD2I0QDqJkSEuy9DgBs5PGYajfmI8vaY3076JG+F9rcEQDUHoRoADVTsAuEsbAYgGruzwu36K+uby1r2yYNlsNh2NwRANQuhGgANVNyspSaWrHFxdLSeB4aQLXG9G0ACD9W5wZQc2VmSo4A/2fO4ZDGjq3afgAgSEdO5vsN0G+N6E6ABgAbMRINoOZyOqWpU6WMjLL3i3Y4pOxspnIDqJaum7JM63cdtawRngHAfoxEA6jZ0tOlBQu8U7WtpKV56yNG2NsXAAQgafRcywDdIC6aAA0AYcJINICaz+n0vnJzvftAu93eVbidTp6BBlAtrdp2UEOnfmZZWz76Gp3bqK7NHQEAShCiAdQeycmEZgDVHouHAUD1xnRuAACAasJfgB6S0poADQDVBCPRAAAAYTbF9a1eWrjFsrb52YGKjY6yuSMAgD+EaAAAgDBi+jYARBZCNAAAQBjk5RepU+bHlrXRgzrqvrQLbO4IABAIQjQAAIDNBr/yqb7Z47asMfoMANUbIRoAAMBGTN8GgMhGiAYAALDBd/uOqe+fcyxrc+6/QiltGtvcEQAgGIRoAACAKsboMwDUHOwTDQAAUIX8Bej4mCgCNABEIEaiAQAAqsC/v9ypx/7fOsval2P7KbFejM0dAQBCgRANAAAQYkzfBoCai+ncAAAAIVLkMf0G6Nu6n0eABoAagJFoAACAEHhk9pf68KvdlrVtkwbL4TBs7ggAUBUI0QAAAJXE9G0AqD0I0QAAAEHaf+y0Ln9ukWXt9du7afCvWtrcEQCgqhGiAQAAgsDoMwDUTiwsBgAAUEEEaACovRiJBgAACNCKrQc0LHuVZW3p7/uoTZN6NncEALAbIRoAACAAjD4DACSmcwMAAJTLX4DuntSYAA0AtQwj0QAAAH68vGiLXl70rWVt08SBiqsTZXNHAIBwI0QDAABYYPo2AMAKIRoAAOAMJ/ML1TlzvmXt6cGdNDK1nc0dAQCqE0I0AABAsQF/ydHmvccsa4w+AwAkQjQAAOGXmyu5XJLbLSUkSE6nlJwc7q5qHaZvAwACQYgGACBcXC4pK0vKyfGtpaZKmZneQI0q9e3eY+r3F4t/BpLef6CXup2faHNHAIDqjBANAEA4TJsmZWRIHo91PSdH6t9fys6WRoywt7dahNFnAEBFsU80AAB2c7nKDtAlPB5p5Ejv+Qg5fwG6QVw0ARoA4Bcj0QAA2C0rq/wAXcLjkSZOZFp3CP1rzU797n/XWda+yuynRvExNncEAIgkhGgAAOyUm2v9DHRZli71XsdiY5XG9G0AQGUxnRsAADsFOzWbKd2VUuQx/QboO3qeT4AGAASMkWgAAOzkdtt7HfTwrC/133W7LWvfPz9YhmHY3BEAIJIRogEAsFNCgr3X1XJM3wYAhBohGgAAOwW7QBgLi1XIvmOn1P056ynwf78jRQO7nGNzRwCAmoIQDQCAnZKTpdTUii0ulpbGomIVwOgzAKAqsbAYAAB2y8yUHAH+X7DDIY0dW7X91CAEaABAVSNEAwBgN6dTmjq1/CDtcEjZ2UzlDsDy7w74DdA5v7+aAA0ACBmmcwMAEA7p6VJSkjRxoncf6F9KS/OOQBOgy8XoMwDAToRoAADCxen0vnJzvftAu93eVbidTp6BDoBpmmr71EeWtZ7tGmt2xhU2dwQAqA0I0QAAhFtyMqG5gl5asFlTFn9nWdv87EDFRkfZ3BEAoLYgRAMAgIhSK6ZvMzsBAKotQjQAAIgIJ04XKnncfMva2F93VnrvtjZ3VAVcLikry3oLtNRU78ruPCcPAGFFiAYAANVe3z8v1Xf7jlvWaszo87RpUkaG5PFY13NypP79vSu2jxhhb28AgFKEaAAAUK3ViunbLlfZAbqExyONHCm1acOINACECftEAwCAamndj0f8BugPHryy5gRoyTuFu7wAXcLj8W6NBgAIC0aiAQBAtVMrRp9L5OZaPwNdlqVLvdex2BgA2I6RaAAAUK34C9CJ8XVqXoCWvFO57bwOAFApIRmJNgzjfEm/keSUdLGkFpLyJW2TNE/SK6Zp7gnivsMlzSjntBOmadav6L0BAED18sbSrXp+3ibL2tqx/dS4XozNHdnE7bb3OgBApVQ6RBuGcZ6k7ZKMMw67JdWT1LX4lWEYxk2maX4S5NsUSDrkp3YiyHsCAIBqolZN3/6lhAR7rwMAVEoopnNHFX+dK+lmSY1N02woKV7SYEnfS0qU9IFhGOcE+R4rTNM8x8/rgkp/BwAAICwKi/5/e3ceH1V59n/8e01YBaKyuNSFgCJgVFxQcUvAwQjB1la7PN0VBPvUPo/2V7UikkpAoNVqW+0G4tra1aX1YRGNCq6oSFGCKCpR3BBUCCD73L8/zqSmzJkwmcycM8vn/XrNazLnOufkooU439z33HcsaYA+68j9Cz9AS+mvskIp9M0AACAASURBVM3q3AAQikxM5/5E0nHOuaXNDzrntkuaa2bVkpZIKpV0saRJGfieAAAgz4361ROqf89/SvKqadUyM99awSkvlyoqWre4WGUli4oBQEjaPBLtnNuwe4Derb5C0rPxlye09fsBAID8V3bV7KQBumH6qOIJ0E1qaqRIim/LIhFp4sTs9gMASCqo1bk/ij+XtHgWAAAoaO988mnS6ds3fGVQcUzf9hONSjNm7DlIRyLSzJlM5QaAEGV9n2gzayfptPjLZWneptzM6iX1lbRT0luSHpb0K+fcqrZ3CQAAsq2oFw9LxZgxUlmZNHmytw/07iorvRFoAjQAhCrrIVrSJZIOkBSTdGea9+gpqYe8z1+XSiqPPy42s4ucc/ekeiMzW5ykNCDN3gAAwB4QoFMUjXqP+npvH+jGRm8V7miUz0ADQI7Iaog2s2MkTYu/vMU5t7yVt3hP0k8k3StppXNuu5l1lLcf9fWSjpR0p5m945xrxWocAAAgCHNffl///ccXfWt1P6rUYb26BtxRnigvJzQDQI7KWog2swMlPSCps6TFkn7c2ns45+ZLmr/bsW2S5pjZU5JekHS4pOmSTk3xnr6Lm8VHqI9vbY8AAMAfo88AgEKUlYXFzKy7vPDbR9JKSaOcc1sz+T2ccxskTY2/HGJmPTN5fwAAkB7nXNIAfdA+nQnQAIC8lvGRaDPbW9JDko6S9Lak4c65NZn+PnGLmr6tvMC+LkvfBwAApOBHf12qe198x7f26pQR6tiOjToAAPktoyHazLpImiNpsKQP5AXotzP5PQAAQG5i+jYAoBhkbDq3mXWW9KC8zyZ/JC9Ar8zU/ZM4udnXDVn+XgAAwEfj1h1JA/T/nHk4ARoAUFAyMhJtZh0k3SdpmKT1kqqcc/VtvKc551wL9VJJV8VfPuecW9uW7wcAAFqP0WcAQLFp80i0mZVIukfSCEkbJY10zvnvZZF4bZmZufjjgt3Kvc3sWTMbY2aHNrumg5mNkPSUpCPk7T89vq1/DgAA0DoEaABAMcrESPRpks6Pf91e0gNmluzc1c65E1tx75PjD5nZVkmbJZXGv48kfSrpe865R1vbNAAASM+Stz/Rl37ztG/tL+OG6OS+PQLuCACA4GQiRDcfze4UfyTTmm2u1kj6X0mnSxokqZekveUF6ZWS6iT91jn3Vqu6BQAAaWP0GQBQ7Nocop1zj8vbYiqdaxuSXeuc2yLp5vgDAACEjAANAEAW9okGAACF5TePv66fzXvVt7Zk4lnat0uHgDsCACA8hGgAQH6rr5fq6qTGRqm0VIpGpfLysLsqGIw+AwDwnwjRAID8VFcn1dZKCxcm1ioqpJoaL1AjLTt2xdRvwlzf2sijDtBvv3VCwB0BAJAbCNEAgPwza5Y0bpwUi/nXFy6UqqqkmTOl0aOD7a0AnH3TQr26ZqNvbdW0arWwCwcAAAWPEA0AyC91dS0H6CaxmDR2rNS7NyPSrcD0bQAAWhbZ8ykAAOSQ2to9B+gmsZg0eXJ2+ykQqz/+NGmAvulrgwjQAADEMRINAMgf9fX+n4FuyYIF3nUsNpYUo88AAKSOkWgAQP6oqwv2uiJAgAYAoHUYiQYA5I/GxmCvK2D/99J7+sE9S3xrj18+VGU9uwTcEQAA+YEQDQDIH6WlwV5XoBh9BgAgfUznBgDkj3RX2WZ1bkmScy5pgO7dYy8CNAAAKWAkGgCQP8rLpYqK1i0uVlnJomKSfviXf+n+Je/61l6bMlId2vF7dQAAUkGIBgDkl5oaqaoqtW2uIhFp4sTs95TjmL4NAEDm8GtnAEB+iUalGTO8gNySSESaObOop3Jv2LIjaYC+bHg/AjQAAGlgJBoAkH/GjJHKyqTJk719oHdXWemNQBdxgGb0GQCA7CBEAwDyUzTqPerrvX2gGxu9Vbij0aL/DDQBGgCA7CFEAwDyW3l50YfmJovf+kTn//Zp39rfvneKTizrHnBHAAAUHkI0AAAFgNFnAACCwcJiAADkOQI0AADBYSQaAIA8dXPdSv384dd8a/+qOUv77NUh4I4AACh8hGgAAPIQo88AAISD6dwAAOSR7TtjSQP0qGMOJEADAJBljEQDAJAnht+4QK9/uMm3tmpatcws4I4AACg+hGgAAPIA07cBAMgNhGgAAHLY2x99qorrH/Ot/fK/jtW5xx4UcEcAABQ3QjQAADmK0WcAAHIPC4sBAJCDCNAAAOQmRqIBAMgh/1z6nv73T0t8awuvGKZDe+wVcEcAAKA5QjQAADmC0WcAAHIf07kBAAiZcy5pgD6sVxcCNAAAOYSRaAAAQnTd7OWa+cQq39prU0aqQzt+3w0AQC4hRAMAEBKmbwMAkH8I0QAABKxx6w4dc+1839rEc47UmNP7BNwRAABIFSEaAIAADZlapw8at/rWGH0GACD3EaIBAAgI07cBAMh/hGgAALJs2bsbdM7NT/rWZv/v6Sr/3N4BdwQAANJFiAYAIIsYfQYAoLCwbwYAAFmSLEAftE9nAjQAAHmKkWgAADLs7mcaNPEf9b61l6+tUrdO7YNtCAAAZAwhGgCADGL6NgAAhY3p3AAAZMD2nbGkAfriyr4EaAAACgQj0QAAtNF3b3tOC15b61tbNa1aZhZwRwAAIFsI0QAAtAHTtwEAKC6EaAAA0vDu+i06bfqjvrU7LjxRQ/vvF3BHAAAgCIRoAABaidFnAACKFwuLAQDQCgRoAACKGyPRAACk4JHla3TRXS/41p4Zf6YO3LtzwB0BAIAwEKIBANgDRp8BAEATpnMDAJCEcy5pgD7ryP0J0AAAFCFGogEA8DHpwXrd/lSDb+3160aqXQm/hwYAoBgRogEA2A3TtwEAQDKEaAAA4jZs2aFBk+b71qadd7S+ftKhAXcEAAByDSEaAABJg6c8onWbtvnWGH0GAABNCNEAgKLH9G0AAJAqQjQAoGi99M56feGWp3xrcy89QwMPLA24IwAAkOsI0QCAosToMwAASAf7cwAAik6yAF3WYy8CNAAAaBEj0QCAonHHU6t07YPLfWvLJp2trh35zyIAAGgZ7xYAAEWB6dsAACATmM4NACho23fGkgbo7w89jAANAABahZFoAEDB+tati/Tk6+t8a6umVcvMAu4IAADkO0I0AKAgMX0bAABkAyEaAFBQVn/8qc742WO+tbtGn6SKI3oF3BEAACgkhGgAQMFg9BkAAGQbC4sBAAoCARoAAASBkWgAQF57ePkajb3rBd/aoquj2r+0U8AdAQCAQkaIBgDkLUafAQBA0JjODQDIO865pAF6RPkBBGgAAJA1jEQDAPLK7xe8oWlzV/jWXr9upNqV8PthAACQPYRoAEDeYPo2AAAIGyEaAJDzNm7doaOvne9bm/mdwTrryP0D7ggAABQrQjQAIKd989Zn9dTrH/nWGH0GAABBI0QDAHIW07cBAECuIUQDAHLOa2s2quqmhb61BVcMVe8eXQLuCAAAwEOIBgDkFEafAQBALmMfEABAzkgWoKuPZu9nAACQGxiJBgCE7h//eleX/vlfvrVXakeoc4eSgDsCAADwR4gGAISK6dsAACCfEKIBAKHYsSumfhPm+tYmf/EofXtI74A7AgAA2DNCNAAgcFff/7LuWfS2b23VtGqZWcAdAQAApIYQDQAIFNO3AQBAPiNEAwAC8cGGrRoyrc639o9LTtOgQ/YJuCMAAIDWI0QDALLuxOse0dqN23xrDdNHSfX10q/ukhobpdJSKRqVyssD7hIAAGDPCNEAgKxKNn17wAHdNO/oHVJlpbRwYeIJFRVSTY0XqAEAAHJEJOwGAACF6ek31iUN0EsmnqV53VZKVVX+AVryjldVSbfdlsUuAQAAWoeRaABAxu1x8bC6OmncOCkWa/lGsZg0dqzUuzcj0gAAICcwEg0AyBjnXNIA/f2hh322+nZt7Z4DdJNYTJo8OUMdAgAAtA0j0QCAjPjt42/op/NW+NbemFqtkkh87+f6+uRTuJNZsMC7jsXGAABAyAjRAIA2a9Xez3X+21ztUV0dIRoAAISOEA0ASNun23fqyJqHfGuzvjtY0YH7JxYaG9P7ZuleBwAAkEGEaABAWiY9WK/bn2rwrSWMPjdXWpreN0z3OgAAgAwiRAMAWi3Z9O2SiOmNqdUtX5zuKtuszg0AAHIAIRoAkLK3P/pUFdc/5ltbdHVU+5d22vNNysuliorWLS5WWcnnoQEAQE4gRAMAUjLiFwu14oONvrUWp2/7qamRqqpS2+YqEpEmTmzd/QEAALKEfaIBAHtUdtVs3wD9w+FHtD5AS97U7BkzvIDckkhEmjmTqdwAACBnEKIBAEk9sXJt0s8/vzplhC4d3i/9m48ZI82f703V9lNZ6dVHj07/ewAAAGQY07kBAL5atfdzuqJR71Ff7+0D3djorcIdjfIZaAAAkJMI0QCA/7BjV0z9Jsz1rSXd+7mtyssJzQAAIC8QogEA/zZj4RuaOmeFby1jo88AAAB5jBANAJCUfPr2Qft01lNXnRlwNwAAALmJEA0ARe6jTdt0wpRHfGt1P6rUYb26BtwRAABA7iJEA0ARG3vXC3p4+RrfGtO3AQAAEhGiAaBIJZu+/V8nHqLp5x8TcDcAAAD5ISMh2swOlXSepKikQZL2l7Rd0puS5kr6pXPu/Tbc/wBJ4yWdI+kgSRskPSfpF865urZ1DwDFZdm7G3TOzU/61l6+tkrdOrUPuCMAAID80eYQbWaHSGqQZM0ON0rqIumY+GOcmZ3vnHssjfsfI+lRST2a3bunvEA9ysyuds5NT/9PAADFo9+EOdqxy/nWmL4NAACwZ5EM3KMk/jxb0lckdXfO7S1pL0nVklZJ2lfSA/ER5ZSZWWdJ/5QXoJdIOip+730l/VxecJ9qZlUZ+HMAQMFyzqnsqtm+Afqn5x9NgAYAAEhRJqZzfyLpOOfc0uYHnXPbJc01s2p5AbhU0sWSJrXi3hdL6i1pk6TPO+fejd+7UdLlZnaYpC9KmiZpflv/IABQiO5f8o5++JelvrU3p1YrEjHfGgAAABK1OUQ75zZI8n935tVXmNmzkoZKOqGVt/9m/PmepgC9m+vlhejjzay/c+7VVt4fAApassXDJKZvAwAApCOo1bk/ij+XtHhWM2bWTZ+F7oeSnPasvEXG9pa3qBkhGgAkbd62U+U/8f/Red/3T9Xxh+4bcEcAAACFIesh2szaSTot/nJZKy4dqM8WK6v3O8E5FzOzVyWdJOnItJsEgAJy7T/rdcfTDb41Rp8BAADaJoiR6EskHSApJunOVlx3YLOv32vhvKbagS2c829mtjhJaUAq1wNALks2ffuMfj1195iTA+4GAACg8GQ1RMe3p5oWf3mLc255Ky7v0uzrLS2c92n8uWtregOAQvLWR5tVef3jvrXnJkS1X7dOwTYEAABQoLIWos3sQEkPSOosabGkH2fre7WGc853cbP4CPXxAbcDAG121o0LtPLDTb41pm8DAABkVlZCtJl1l7flVB9JKyWNcs5tbeVtNjf7urOkjUnO2yv+7P8OEgAKWLLp2z866wj9T7RfwN0AAAAUvoyHaDPbW95q2kdJelvScOfcmjRu1fxz0J9T8pW3Pxd/fj+N7wEAeWnha2v1ndue8629NmWkOrSLBNwRAABAcchoiDazLpLmSBos6QN5AfrtNG+3QpKTt0J3uXxCtJlFJPWPv2zN560BIG+x9zMAAEB4MjZUYWadJT0o6VR5+0IPd86tTPd+zrmNkl6IvzwryWkny9sjWpLq0v1eAJAPduyKJQ3Qt10wmAANAAAQgIyEaDPrIOk+ScMkrZdU5Zzz3du5le6JP38zvlDZ7i6PPy92ziWb7g0Aee93C95QvwlzfWsN00fpzAH7B9wRAABAcWpziDazEnlhd4S8xb9GOudeTPHaMjNz8ccFPqf8XtJbkrpJ+j8zOzJ+XTcz+5mk8+LnXd3GPwYA5Kyyq2Zr+twVCccP7b4Xo88AAAABy8Rnok+TdH786/aSHjCzZOeuds6dmOqNnXNbzOxceVO1j5dUb2aN8vaEjsj7zPTVzrn56TYPALlq3aZtGjzlEd/aoz+qVN9eXQPuCAAAAJkI0c1HszvFH8m0dpsrOeeWmtlRksZLOkfSQfI+c/2cpJucc3wWGkDBGX3H83p0xYe+NUafAQAAwtPmEO2ce1zeCtrpXNuQyrXOuQ8kXRp/AEBBS7Z42NdPOlTTzjs64G4AAADQXMb3iQYApOeld9brC7c85VtbNulsde3Ij2wAAICw8Y4MAHJA3/GzFXP+NaZvAwAA5A5CNACEyDmnPuPn+NZ+9uVj9NXBhwTcEQAAAFpCiAaAkNy7+B396G9LfWtvTq1WJJLWchMAAADIIkI0AIQg2eJhEtO3AQAAchkhGgACtHnbTpX/5CHf2gOXnKZjD9kn4I4AAADQGoRoAAjIxAeW6e5n3/KtMfoMAACQHwjRABCAZNO3K4/opTtHnxRwNwAAAEgXIRoAsqhh3WYNveFx39pzE6Lar1unYBsCAABAmxCiASBLzvz543pz7WbfGtO3AQAA8hMhGgCyINn07SvO7q9Lhh0ecDcAAADIFEI0AGTQ469+qAtuf9639tqUkerQLhJwRwAAAMgkQjQAZAh7PwMAABQ+QjQAtNGOXTH1mzDXt3b7hSdqWP/9Au4IAAAA2UKIBoA2+PVjr+v6h171rTH6DAAAUHgI0QCQpmTTt/v07KLHLh8abDMAAAAIBCEaAFpp7cZtOvG6R3xrj10+VH16dgm4IwAAAASFEA0ArXDB7c/p8VfX+taYvg0AAFD4CNEAkKJk07e/NeRQTfni0QF3AwAAgDAQogFgD5auXq9zf/2Ub23ZpLPVtSM/SgEAAIoF7/wAoAXs/QwAAIDmCNEA4CMWc+p79Rzf2s+/Mkjnn3BwwB0BAAAgFxCiAWA3f3thta74+0u+tVXTqmVmAXcEAACAXEGIBoBmkk3fNpNWTWP6NgAAQLEjRAOApE3bduqonzzkW/vnD07TMQfvE3BHAAAAyEWEaABFb8L9L+uPi972rbF4GAAAAJojRAMoasmmbw/r30u3X3hSwN0AAAAg1xGiARSlVes2a9gNj/vWnp8wXL26dQy2IQAAAOQFQjSAolN5/WN666NPfWtM3wYAAEBLCNEAikqy6dtXjuiv7w89POBuAAAAkG8I0QCKwmMrPtSFdzzvW1t53Ui1L4kE3BEAAADyESEaQMFLNvosMX0bAAAArUOIBlCwtu+M6Yhr5vrW7hx9kiqP6BVwRwAAAMh3hGgABemWR1fqhvmv+dYYfQYAAEC6CNEACk6y6duH79dVj/y/yoC7AQAAQCEhRAMoGB9u3KqTrqvzrS24Yqh69+gScEcAAAAoNIRoAAXh27MW6YmV63xrTN8GAABAphCiAeS9ZNO3v3tKb00696iAuwEAAEAhI0QDyFtL3v5EX/rN0761+klnq0tHfsQBAAAgs3iHCSAvsfczAAAAwkCIBpBXYjGnvlfP8a3d+NVBOu/4gwPuCAAAAMWEEA0gb/z1+dW68t6XfGurplXLzALuCAAAAMWGEA0gLySbvt0uYnp9anXA3QAAAKBYEaIB5LSNW3fo6Gvn+9Ye/MHpOvrgvQPuCAAAAMWMEA0gZ42/7yX96bnVvjUWDwMAAEAYCNEAclKy6dvDB+6nW797YsDdAAAAAB5CNICcsqZxq06eWudbW3zNcPXo2jHgjgAAAIDPEKIB5IyafyzTXc+85Vtj+jYAAAByASEaQE5INn17/MgBurjysIC7AQAAAPwRogGEatm7G3TOzU/61t6YWq2SCHs/AwAAIHcQogGE5txbntTSdzYkHD+prLv++r1TQugIAAAAaBkhGkDgdu6K6fAJc31rcy89QwMPLA24IwAAACA1hGgAgZr78vv67z++6Ftj8TAAAADkOkI0gMAkWzzsotP76Jpzjgy4GwAAAKD1CNEAsm7Dpzs0qHa+b+35CcPVqxt7PwMAACA/EKKBTKqvl+rqpMZGqbRUikal8vKwuwrVr+pW6saHX/OtMX0bAAAA+YYQDWRCXZ1UWystXJhYq6iQamq8QF1kkk3f/tmXj9FXBx8ScDcAAABA2xGigbaaNUsaN06KxfzrCxdKVVXSzJnS6NHB9haSVes2a9gNj/vWVkweoU7tS4JtCED+YoYPACDHEKKBtqirazlAN4nFpLFjpd69C35EetxdL2j+8jUJxw/et7Oe/PGZIXQEIC8xwwcAkKMiYTcA5LXa2j0H6CaxmDR5cnb7CVEs5lR21WzfAP3Xi08hQANI3axZ3gwevwAtfTbD57bbgu0LAAARooH01dcnf4OXzIIF3nUF5qnX16nv1XN8a6umVeukPt0D7ghA3mrtDJ+6umD6AgAgjhANpCvdN24F9oZv8JSH9c1bFyUcP/fYz6lh+iiZWQhdAchbzPABAOQ4PhMNpKuxMdjrcsyW7bs0sGaeb+2JK4fpkO57BdwRgLzXlhk+LDYGAAgII9FAukpLg70uh/zh2beSBuiG6aMI0ADSwwwfAEAeYCQaSFe6q8Lm+WqyyfZ+Hj9ygC6uPCzgbgAUlCKf4QMAyA+EaCBd5eXeNiutmXpYWZm3Uw4/2LBVQ6b5j/a8fG2VunVqH3BHAApOEc/wAQDkD6ZzA21RUyNFUvxnFIlIEydmt58sueaBl30DdEnE1DB9FAEaQGYU6QwfAEB+IUQDbRGNSjNm7DlIRyLSzJl5+Uav7KrZ+sOzbyccv/U7g/XG1OoQOgJQsJpm+LRGHs/wAQDkJ6ZzA201ZoxUVuZts7JgQWK9stIbgc6zAP3yOxv0+Vue9K29MbVaJRG2rkJcfb23sFNjozetNhol1CB9NTVSVVVq21zl8QwfAED+IkQDmRCNeo8CCROfv/lJvfzuhoTjQ/p215/HnRJCR8hJdXXenr5+6wJUVHhhKM9+eYQc0DTDZ9y4loN0Hs/wAQDkN0I0kEnl5XkZmpvs2BVTvwlzfWtzLz1DAw9k8R7EzZrVcshZuNAbTZw5Uxo9OtjekP8KdIYPAKAwEKIBSJJmv/S+LrnnRd9aw/RRAXeDnFZXt+dRQsmrjx0r9e5N2EHrFdgMHwBA4SBEA0i69/O4ir66unpgwN0g59XWpvZ5Vck7b/JkQjTSl+czfAAAhYcQDRSxDZ/u0KDa+b61F64Zrp5dOwbcEXJefX3r9kaXvOm49fUEIQAAUBDY4gooUr945LWkAbph+igCNPzVJe4XntXrAAAAcgwj0UARSjZ9+4avDNKXTzg44G6QVxobg70OAAAgxxCigSLyxtpNiv7cZ6VbSa9OGaGO7UoC7gh5pzTNFdrTvQ4AACDHEKKBInHRnS/okVfWJBzv3WMvLbhiWAgdIS+lu0AYC4sBAIACQYgGClws5tT36jm+tb9/7xQNLusecEfIa+XlUkVF6xYXq6xkUTEAAFAwWFgMKGBPrlyXNECvmlZNgEZ6amqkSIr/+YhEpIkTs9sPAABAgAjRQIE6rna+vjVrUcLxLx13kBqmj5KZhdAVCkI0Ks2YsecgHYlIM2cylRsAABQUQjRQYLZs36Wyq2brk093JNSeuHKYbvrasSF0hYIzZow0f743VdtPZaVXHz062L4AAACyjM9EAwXk7mff0sQHlvnWGqaPCrgbFLxo1HvU13v7QDc2eqtwR6N8BhoAABQsQjRQIJLt/TyheqDGVvQNuBsUlfJyQjMAACgahGggz32wYauGTKvzrS2bdLa6duSfOQAAAJApvLsG8tjV97+sexa9nXC8Q0lEr103MoSOAAAAgMJGiAbykHNOfcb7b10167uDFR24f8AdAQAAAMWBEA3kmaWr1+vcXz/lW3tjarVKIgWwdRULVQEAACBHEaKBPFL9yye0/P3GhOOn9O2hP40bEkJHGVZXJ9XWSgsXJtYqKqSaGvYcBgAAQKgI0UAe2LErpn4T5vrWHrqsQv0P6BZwR1kwa5Y0bpwUi/nXFy6UqqqkmTPZexgAAAChiYTdAICW/d9L7yUN0A3TRxVGgK6razlAN4nFpLFjvfMBAACAEDASDeSwZHs/X1zZV+NHDgy4myyqrd1zgG4Si0mTJzOtGwAAAKEgRAM5aP2n23Vs7cO+tcXXDFePrh0D7iiL6uv9PwPdkgULvOtYbAwAAAABYzo3kGNuevi1pAG6YfqowgrQUvpTs5nSDQAAgBAwEg3kkGTTt2/86iCdd/zBAXcTkMbE1cazeh0AAADQBoRoIAe8sXaToj9f4Ft7dcoIdWxXEnBHASotDfY6AAAAoA0I0UDIRt/xvB5d8WHC8b49u+jRy4cG31DQ0l0gjIXFAAAAEAJCNBCSWMyp79VzfGv3/vcpOqF394A7Ckl5uVRR0brFxSorWVQMAAAAoWBhMSAET6xcmzRAr5pWXTwBuklNjRRJ8cdRJCJNnJjdfgAAAIAkCNFAwAZNmq9vz3ou4fh5xx+khumjZGYhdBWyaFSaMWPPQToSkWbOZCo3AAAAQkOIBgLy6fadKrtqtjZs2ZFQe/LHw3TjV48NoascMmaMNH++N1XbT2WlVx89Oti+AAAAgGb4TDQQgLueaVDNP+p9aw3TRwXbTC6LRr1Hfb23D3Rjo7cKdzTKZ6ABAACQEwjRQJYl2/v5mlEDddEZfQPuJk+UlxOaAQAAkJMyEqLNrJukYZJOlDQ4/twjXh7onFuR5n2HSnoshVN7OefWpfM9gGx5f8MWnTLtUd/asklnq2tHfocFAAAA5JtMvYuPSro/Q/fyE5O0dg91IGeMv+8l/em51QnHO7aL6NUpI0PoCAAAAEAmZHIo7ENJL0h6XtK7kmZk8N6rnXNlGbwfkBXOOfUZ77911e0XnKhhA/YLuCMAAAAAmZSpEP2gc+6BphdmVpah+wJ54/UPN2r4jQt9a29MrVZJpAi3rgIAAAAKTEZCtHNu8lAA9AAAFfJJREFUVybuA+SrK/++VH994Z2E46cf3lN/uOjkEDoCAAAAkA2sbAS0wc5dMR0+Ya5vbf4PK3TE/t0C7ggAAABANkXCbiBFvczsRTPbHH+8ZmYzzOzosBtD8Xr69XVJA3TD9FEEaAAAAKAA5ctI9F6SjpP0iaQukvrFH6PN7Crn3A2p3sjMFicpDWhzlygan7/5Sb387oaE41O/dLS+cfKhIXQEAAAAIAi5HqLXS7pe0l8k1TvntppZiaTTJE2TdKqk683sPefcPSH2iSKxcesOHX3tfN/a0p9Uae/O7QPuCAAAAECQcjpEO+f+Jelfux3bJWmhmQ2T9Ki8QP1TM/uzc26P+0U7507wOx4foT6+7V2jUP31+dW68t6XEo4POKCb5l1WEUJHAAAAAIKW0yG6Jc657WY2UV6QPljedO9kU7WBNim7arbv8bvHnKQz+vUKuBsAAAAAYcnbEB23qNnXfUWIRoa9u36LTpv+qG9t5XUj1b4kX9bmAwAAAJAJ+R6igayZPneFfrfgjYTj5x1/kG786rEhdAQAAAAgbPkeok9u9vWq0LpAQYnFnPpePce3xt7PAAAAQHHL6RBtZuacc0lq7SXVxl++L+nFwBpDwVq6er3O/fVTvrVV06plZgF3BAAAACCXZOwDnWbWs+khad9mpX2a18wsstt1Lv641ue2y8zsf8ysn8XTi5mVmNnpkuoknR4/b3wqK3MDLbnw9ud8A/SVI/qrYfooAjQAAACAjI5Er01y/JndXveR1JDiPY+U9Kv419vMbKOkUkkd4sd2SrrGOXdnK/oE/sPWHbs0YOI839rzE4arV7eOAXcEAAAAIFfl9HRuSRfL2wf6BEn7yRvh3iLpVUkLJP3WObc8vPaQ75a8/Ym+9JunE45379JBL048K4SOAAAAAOSyjIVo51xac11bus45N0PSjLSbAlpw1b0v6c/Pr044/ptvHq/qow8MoSMAAAAAuS7XR6KBjGvcukPHXDvft7Zi8gh1al8ScEcAAAAA8gUhGkXl4eVrNPauFxKOX151hH5wZr8QOgIAAACQTwjRKArOOX1txrN6btXHCbXHLh+qPj27hNAVAAAAgHxDiEbB+2DDVg2ZVpdwvN9+XTX/hxVsXQUAAAAgZYRoFLS7n2nQxH/UJxy/8auDdN7xBwffEAAAAIC8RohGQdoVczp5ap3WbdqWUFt8zXD16MrezwAAAABajxCNgrPig0aN+MUTCcfPLt9fv//24BA6AgAAAFAoCNEoKNPmvKLfL3wz4fgfLzpZpx3eM4SOAAAAABQSQjQKwpbtuzSwZp5v7ZXaEercgb2fAQAAALQdIRp57+nX1+kbty5KOH5xRV+Nrx4YQkcAAAAAChUhGnnte3cv1rz6DxKOz730DA08sDSEjgAAAAAUMkI08tJHm7bphCmPJBzv2bWjFl0dVUmEvZ8BAAAAZB4hGnnn/iXv6Id/WZpwvPbccn3nlLLgGwIAAABQNAjRyBvOOVXdtFArP9yUUHtm/Jk6cO/OIXQFAAAAoJgQopEXGtZt1tAbHk84fnKf7vrzuCEyY/o2AAAAgOwjRCPn/fqx13X9Q68mHJ/x7RNUVX5ACB0BAAAAKFaEaOSs7TtjGjBxrmIusfbStVUq7dQ++KYAAAAAFLVI2A0Afl58+xMdcU1igP6vEw9Rw/RRBGgAAAAAoWAkGjnnx39/SX95YXXC8fu/f6qOO3TfEDoCAAAAAA8hGjmjcesOHXPt/ITj7SKm5bUj1KEdEycAAAAAhIsQjZzwUP0HuvjuxQnHrzi7vy4ZdngIHQEAAABAIkI0QuWc09d+/6yea/g4ofb45UNV1rNLCF0BAAAAgD9CNELz3votOnX6ownH++/fTfMuO4O9nwEAAADkHEI0QnHn0w36yT/rE47/4mvH6ovHHRRCRwAAAACwZ4RoBGpXzOnkqY9o3abtCbXF1wxXj64dQ+gKAAAAAFJDiEZgXnm/USN/+UTC8RHlB+h33z4hhI4AAAAAoHUI0QjE1DmvaMbCNxOO33PRyTr18J4hdAQAAAAArUeIRlZt2b5LA2vm+dZeqR2hzh1KAu4IAAAAANJHiEbWPPX6On3z1kUJxy+u7KvxIweG0BEAAAAAtA0hGlkx7q4XNH/5moTj8y47QwMOKA2hIwAAAABoO0I0Mmrdpm0aPOWRhOP7deuoZ8ZHVRJh72cAAAAA+YsQjYy578V39P/+ujTh+OQvHqVvD+kdQkcpqq+X6uqkxkaptFSKRqXy8rC7AgAAAJCDCNFoM+echt+4QG+s3ZxQe2b8mTpw784hdJWCujqptlZauDCxVlEh1dR4gRoAAAAA4iJhN4D8tmrdZvUZPychQA/p212rplXnboCeNUuqqvIP0JJ3vKpKuu22YPsCAAAAkNMYiUbabnl0pW6Y/1rC8ZnfGayzjtw/hI5SVFcnjRsnxWItnxeLSWPHSr17MyINAAAAQBIhGmnYvjOm/hPnyrnE2kvXVqm0U/vgm2qN2to9B+gmsZg0eTIhGgAAAIAkpnOjlRa/9YmOuCYxQH/9pEPUMH1U7gfo+vrkU7iTWbDAuw4AAABA0WMkGim74m9L9bfF7yQcf+CS03TsIfuE0FEa6urSv44VuwEAAICiR4jGHm3YskODJs1PON6hJKJlk85Wh3Z5NKGhsTHY6wAAAAAUFEI0WjRv2Qf63h8WJxy/ckR/fX/o4SF01EalpcFeBwAAAKCgEKLhyzmnr/7+GT3f8ElCbcEVQ9W7R5cQusqAdBcIY2ExAAAAACJEw8d767fo1OmPJhwfcEA3zb30DJlZCF1lSHm5VFHRusXFKiv5PDQAAAAASazOjd3c+XSDb4D+xdeO1bzLKvI7QDepqZEiKf7Vj0SkiROz2w8AAACAvMFINCRJu2JOJ173iD7evD2h9uLEs9S9S4cQusqSaFSaMUMaN67l/aIjEWnmTKZyAwAAAPg3RqKh5e816rCr5yQE6JFHHaCG6aMKK0A3GTNGmj/fm6rtp7LSq48eHWxfAAAAAHIaI9FF7rrZyzXziVUJx+8Ze7JOPaxnCB0FKBr1HvX13j7QjY3eKtzRKJ+BBgAAAOCLEF2kPt2+U0fWPORbWzF5hDq1Lwm4oxCVlxOaAQAAAKSEEF2Enly5Tt+atSjh+PcqD9NVIweE0BEAAAAA5AdCdJG56M4X9MgraxKOP3RZhfof0C2EjgAAAAAgfxCii8S6Tds0eMojCcf3L+2op6+KqiRSAFtXAQAAAECWEaKLwN8Xv6PL/7Y04fiULx6lbw3pHUJHAAAAAJCfCNH5IM3Vo2Mxp+E3LdCbazcn1J4dH9UBe3fKRrcAAAAAULAI0bmsrk6qrZUWLkysVVRINTVeoPbx5tpNOvPnCxKOn3pYD/3xopNlxvRtAAAAAGitSNgNIIlZs6SqKv8ALXnHq6qk225LKP2qbqVvgL71O4N1z9ghBGgAAAAASBMj0bmork4aN06KxVo+LxaTxo6VeveWolFt27lL/a+Z53vqy9dWqVun9lloFgAAAACKByPRuai2ds8BukksJk2erMVvfewboL9x8qFqmD6KAA0AAAAAGcBIdK6pr08+hTuJK/Y6Vn/77TMJx/9xyWkadMg+meoMAAAAAIoeITrX1NWlfOqGjl006LK/JBzv2C6iZZPOVvsSJhoAAAAAQCaRsnJNY2NKp83rd4pvgL5q5AC9OmUkARoAAAAAsoCR6FxTWtpi2Un68jd/psUHH5lQW3jFMB3aY68sNQYAAAAAIETnmiT7PkvSu9166bTv355wfOCaNzXnR8NkBGgAAAAAyCrm/Oaa8nKpoiLh8Jz+p/kG6F/+83rNXXWv7KijgugOAAAAAIoaI9G5qKZGqqr6j22upg29MOG0Jb/8uvbdvlm6eX6Q3QEAAABA0WIkOhdFo9KMGVLks/979tmy8d9fj1rxhBp+eo4XoGfObHEKOAAAAAAgcxiJzlVjxkhlZdLkydKCBfrDX67Rgr4n6Ph3X9HBjWulykpp4kQCNAAAAAAEiBCdy6JR71Ffr73r6vSFxkap9HzvWHl52N0BAAAAQNEhROeD8nJCMwAAAADkAD4TDQAAAABAigjRAAAAAACkiBANAAAAAECKCNEAAAAAAKSIEA0AAAAAQIoI0QAAAAAApIgQDQAAAABAigjRAAAAAACkiBANAAAAAECKCNEAAAAAAKSIEA0AAAAAQIoI0QAAAAAApIgQDQAAAABAigjRAAAAAACkiBANAAAAAECKCNEAAAAAAKSIEA0AAAAAQIoI0QAAAAAApIgQDQAAAABAigjRAAAAAACkiBANAAAAAECKCNEAAAAAAKSIEA0AAAAAQIoI0QAAAAAApIgQDQAAAABAisw5F3YPOcHMPurcuXP3gQMHht0KAAAAACDDXnnlFW3ZsuVj51yPttyHEB1nZqsklUpqyOK3GRB/XpHF7wHkCv6+o5jw9x3FhL/vKDb8nS8cZZIanXN92nITQnSAzGyxJDnnTgi7FyDb+PuOYsLfdxQT/r6j2PB3HrvjM9EAAAAAAKSIEA0AAAAAQIoI0QAAAAAApIgQDQAAAABAigjRAAAAAACkiNW5AQAAAABIESPRAAAAAACkiBANAAAAAECKCNEAAAAAAKSIEA0AAAAAQIoI0QAAAAAApIgQDQAAAABAigjRAAAAAACkiBCdRWZ2qJldZmYPmtnbZrbNzDaa2VIzm25mB4bdI5AtZtbVzFabmYs/Lgi7JyAbzKy/md1sZq+a2WYz22Bmr5jZbWZWGXZ/QCaYWcTMLjSzR8xsrZntMLP1ZrbIzCaYWbewewRSZWbdzOwLZjbZzOaa2bpm71cGpHB9xMzGmdkz8X8HG81siZldYWYdgvgzIFzmnAu7h4JkZodIekuSNTvcKKmLpJL4608kne+ceyzg9oCsM7NfSLq02aELnXN3hNQOkBVm9r+SrpfU9KZpk6R2kjrFX89yzl0URm9AppjZXpIelHRms8MbJJXqs/c5b0k60zn3ZsDtAa1mZl+UdH+S8kDn3IoWrm0v6QFJ1fFD2yXtktQ5/vp5ef8WNmWoXeQgRqKzpykoz5b0FUndnXN7S9pL3j+6VZL2lfSAmR0QTotAdpjZ8ZJ+IGlR2L0A2WJmF0v6pbzQ/FNJvZ1z3ZxznSUdKOk7kp4OsUUgUybKC9BO0nhJ+zjn9pH3y6KvS1ovqbekW0PrEGi9DyXNkTRJ0rhWXDdF3nv5rZIukPfevoukz0v6WNKJkn6fyUaRexiJzhIz21tSmXNuaZL6AElL5P0H6Frn3KQg+wOyxcwi8sLzcfL+Q/JivMRINAqGmZVJqpf35mmcc25mqA0BWWRmb0k6VNJtzrkxPvULJN0ef9ndOfdJgO0BrWZmJc65Xc1el8kb4JJaGImOD3w1SOoo6VLn3K92q58rb5TaSTrWOfdSxptHTmAkOkuccxuSBeh4fYWkZ+MvTwimKyAQ/yNpsKTfOueWhN0MkCWXygvQiwjQKAL7x5+T/Uxf3OzrvbLcC9BmzQN0K50vL0BvkDTD577/kPSavI85fCPtBpHzCNHh+ij+XNLiWUCeMLODJE2WtEbSNSG3A2RT05ujP4XaBRCMhvjzcUnqTYMBa5xz72a/HSA0w+LPC51zW5OcMz/+fGaSOgoAITokZtZO0mnxl8vC7AXIoJsldZN0uXNuQ9jNANlgZodJ2i/+comZDYnvwvCRmW0xsxVmdr2Z7dfSfYA80jTb4kIzuyr+kTWZWQcz+5qkm+RNX708rAaBgBwZf65v4Zzl8eeBZmYtnIc8RogOzyWSDpAUk3RnyL0AbWZmn5f0JUmPO+f+EHY/QBb1a/b1UElPSjpHUnt5QaK/vDDxLzMrD7w7IPN+IenX8qaoTpO03szWS9oi6c+SVkj6Aj/7UQSatqd9r4Vzmmpd4w8UIEJ0CMzsGHn/EZKkW5xzy1s6H8h1ZtZF0i2Sdsj7BRFQyPZp9vVP5H3+bYhzrlTeG6Zqeau+Hijp3vjMIyBvxT8/epmkH0naGT+8tz57H9lNUq8QWgOC1iX+vKWFcz5t9jUhukARogNmZgfKW7Wvs7yFOH4cbkdARtTKW7n1Jn4phCLQ/L+dTtKXnHOLJMk5F3POzZU0Ol7vL+m8gPsDMiq+IvFTkn4u6Y+SBskLB/3kbXnVV9JtZjYt6U0AoIAQogNkZt3lLTbQR9JKSaNaWJQAyAtmdqy8lYpXywvTQKHb1Ozrec65V3c/wTk3W94ItSRFA+kKyJ67JJ0kaZZz7gLn3EvOuc3Oudedc9MlXRw/70o+woACtzn+3LmFc5qvUL8p6VnIa4TogMQX4XhI0lGS3pY03Dm3JtyugIz4pbwV5idIMjPr2vzR7LyO8WNsf4J81/yzcAkB2qd2SBZ7AbLKzI6UdFb85U1+5zjn7pa340hE0ucDag0IQ9PP/8+1cE5TbZNzbmOW+0FICNEBiH9edI68vXM/kBeg3w63KyBjesef75K00efR5Hfx10z3Rr5bLm9RyFS5bDUCBGBgs69XtXDem/Hnsuy1AoSu6T1MSzMumlbwfiXLvSBEhOgsM7POkh6UdKq839IOd86tDLcrAEC6nHOfSnom/rJ/C6c21Rqy2hCQXc1/YXRoC+c1/UKVkTcUssfiz2eYWack5zTN3KgLoB+EhBCdRWbWQdJ98jZmXy+pyjnX0r5yQN5xzpU55yzZo9mpF8aPlYXVK5BBd8WfR5hZQpA2s1GSjoi/nBNYV0DmLW329Vi/E+JbHDbti74o6x0B4blP0jZ5uzRctHsx/m+hv7wZSH8KtjUEiRCdJWZWIukeSSPk/VZ2pHPuxXC7AgBkyG3ypvWVSLrPzE6SJDOLmNkISbPi5z0rQjTymHPuTXmLokrSZWY2zcz2k6T4OhcXSLojXm+Q9M+gewTSYWY9mx6S9m1W2qd5zcz+nZeccx/IWwtGkn5mZt+Ov+eXmVVLuj1e+5Nz7qUg/hwIhznHR7WywcwqJC2Iv9wqaUMLp692zp2Y/a6A4JlZ0w+ZC51zd4TZC5BJZtZX0uP6bOGwjfJCddPiecvlzUB6N/jugMyJb89Zp//8fPRGeftDN1kjb8BgSZC9Aelq9v5kT/o45xqaXdde3na11fFD2yTt0mc/+5+XFGVRscLGSHT2NP/ftpOk/Vt49Aq8OwBAm8RH6I6WdJ28wNxO3hS+F+XtnXsSARqFwDn3vqQTJF0maaGkj+UFhkZ5f98nSzqaAI1i4JzbIW8V+u/Jm220Td7P/n9J+rGk0wnQhY+RaAAAAAAAUsRINAAAAAAAKSJEAwAAAACQIkI0AAAAAAApIkQDAAAAAJAiQjQAAAAAACkiRAMAAAAAkCJCNAAAAAAAKSJEAwAAAACQIkI0AAAAAAApIkQDAAAAAJAiQjQAAAAAACkiRAMAAAAAkCJCNAAAAAAAKSJEAwAAAACQIkI0AAAAAAApIkQDAAAAAJAiQjQAAAAAACkiRAMAAAAAkKL/DxD0nvf9rg34AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "image/png": { "height": 361, "width": 488 } }, "output_type": "display_data" } ], "source": [ "# change default figure and font size\n", "plt.rcParams['figure.figsize'] = 8, 6 \n", "plt.rcParams['font.size'] = 12\n", "\n", "# Launch the graph\n", "with tf.Session() as sess:\n", " sess.run(init)\n", "\n", " # Fit on all the training data\n", " feed_dict = {X: X_train, Y: y_train}\n", " for epoch in range(n_epochs):\n", " sess.run(train, feed_dict = feed_dict)\n", "\n", " # Display logs per epoch step\n", " if (epoch + 1) % display_epoch == 0:\n", " # run the cost to obtain the value for the cost function at each step\n", " c = sess.run(cost, feed_dict = feed_dict)\n", " print(\"Epoch: {}, cost: {}\".format(epoch + 1, c))\n", "\n", " print(\"Optimization Finished!\")\n", " c = sess.run(cost, feed_dict = feed_dict)\n", " weight = sess.run(W)\n", " bias = sess.run(b)\n", " print(\"Training cost: {}, W: {}, b: {}\".format(c, weight, bias))\n", "\n", " # graphic display\n", " plt.plot(X_train, y_train, 'ro', label = 'Original data')\n", " plt.plot(X_train, weight * X_train + bias, label = 'Fitted line')\n", " plt.legend()\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## MNIST Using Softmax" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "MNIST is a simple computer vision dataset. It consists of images of handwritten digits like these:\n", "\n", "\n", "\n", "Each image is 28 pixels by 28 pixels, which is essentially a $28 \\times 28$ array of numbers. To use it in a context of a machine learning problem, we can flatten this array into a vector of $28 \\times 28 = 784$, this will be the number of features for each image. It doesn't matter how we flatten the array, as long as we're consistent between images. Note that, flattening the data throws away information about the 2D structure of the image. Isn't that bad? Well, the best computer vision methods do exploit this structure. But the simple method we will be using here, a softmax regression (defined below), won't.\n", "\n", "The dataset also includes labels for each image, telling us the each image's label. For example, the labels for the above images are 5, 0, 4, and 1. Here we're going to train a softmax model to look at images and predict what digits they are. The possible label values in the MNIST dataset are numbers between 0 and 9, hence this will be a 10-class classification problem." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "60000 train samples\n", "10000 test samples\n" ] } ], "source": [ "n_class = 10\n", "n_features = 784 # mnist is a 28 * 28 image\n", "\n", "# load the dataset and some preprocessing step that can be skipped\n", "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n", "X_train = X_train.reshape(60000, n_features)\n", "X_test = X_test.reshape(10000, n_features)\n", "X_train = X_train.astype('float32')\n", "X_test = X_test.astype('float32')\n", "\n", "# images takes values between 0 - 255, we can normalize it\n", "# by dividing every number by 255\n", "X_train /= 255\n", "X_test /= 255\n", "\n", "print(X_train.shape[0], 'train samples')\n", "print(X_test.shape[0], 'test samples')" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "# convert class vectors to binary class matrices (one-hot encoding)\n", "# note: you HAVE to to this step\n", "Y_train = np_utils.to_categorical(y_train, n_class)\n", "Y_test = np_utils.to_categorical(y_test , n_class)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the following code chunk, we define the overall computational graph/structure for the softmax classifier using the cross entropy cost function as the objective. Recall that the formula for this function can be denoted as:\n", "\n", "$$L = -\\sum_i y'_i \\log(y_i)$$\n", "\n", "Where y is our predicted probability distribution, and y′ is the true distribution." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "# define some global variables\n", "learning_rate = 0.1 \n", "n_iterations = 400\n", "\n", "# define the input and output \n", "# here None means that a dimension can be of any length,\n", "# which is what we want, since the number of observations\n", "# we have can vary;\n", "# note that the shape argument to placeholder is optional, \n", "# but it allows TensorFlow to automatically catch bugs stemming \n", "# from inconsistent tensor shapes\n", "X = tf.placeholder(tf.float32, [None, n_features])\n", "y = tf.placeholder(tf.float32, [None, n_class])\n", "\n", "# initialize both W and b as tensors full of zeros. \n", "# these are parameters that the model is later going to learn,\n", "# Notice that W has a shape of [784, 10] because we want to multiply \n", "# the 784-dimensional image vectors by it to produce 10-dimensional \n", "# vectors of evidence for the difference classes. b has a shape of [10] \n", "# so we can add it to the output.\n", "W = tf.Variable(tf.zeros([n_features, n_class]))\n", "b = tf.Variable(tf.zeros([n_class]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```python\n", "# to define the softmax classifier and cross entropy cost\n", "# we can do the following\n", "\n", "# matrix multiplication using the .matmul command\n", "# and add the softmax output\n", "output = tf.nn.softmax(tf.matmul(X, W) + b)\n", "\n", "# cost function: cross entropy, the reduce mean is simply the average of the\n", "# cost function across all observations\n", "cross_entropy = tf.reduce_mean(-tf.reduce_sum(y * tf.log(output), axis = 1))\n", "\n", "```" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:From :4: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.\n", "Instructions for updating:\n", "\n", "Future major versions of TensorFlow will allow gradients to flow\n", "into the labels input on backprop by default.\n", "\n", "See tf.nn.softmax_cross_entropy_with_logits_v2.\n", "\n" ] } ], "source": [ "# but for numerical stability reason, the tensorflow documentation\n", "# suggests using the following function\n", "output = tf.matmul(X, W) + b\n", "cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels = y, logits = output))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we defined the structure of our model, we'll:\n", "\n", "1. Define a optimization algorithm the train it. In this case, we ask TensorFlow to minimize our defined cross_entropy cost using the gradient descent algorithm with a learning rate of 0.5. There are also other off the shelf [optimizers](https://www.tensorflow.org/api_guides/python/train#optimizers) that we can use that are faster for more complex models.\n", "2. We'll also add an operation to initialize the variables we created\n", "3. Define helper \"function\" to evaluate the prediction accuracy" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(cross_entropy)\n", "init = tf.global_variables_initializer()\n", "\n", "# here we're return the predicted class of each observation using argmax\n", "# and see if the ouput (prediction) is equal to the target variable (y)\n", "# since equal is a boolean type tensor, we cast it to a float type to compute\n", "# the actual accuracy\n", "correct_prediction = tf.equal(tf.argmax(y, axis = 1), tf.argmax(output, axis = 1))\n", "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now it's time to run it. During each step of the loop, we get a \"batch\" of one hundred random data points (defined by `batch_size`) from our training set. We run train_step feeding in the batches data to replace the placeholders.\n", "\n", "Using small batches of random data is called stochastic training -- in this case, stochastic gradient descent. Ideally, we'd like to use all our data for every step of training because that would give us a better sense of what we should be doing, but that's expensive. So, instead, we use a different subset every time. Doing this is cheap and has much of the same benefit." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.09871667\n", "0.84145\n", "0.8613333\n", "0.87055\n", "0.87721664\n", "0.88168335\n", "0.88545\n", "0.8884\n", "test: 0.8906\n" ] } ], "source": [ "with tf.Session() as sess: \n", " # initialize the variable, train the \"batch\" gradient descent\n", " # for a specified number of iterations and evaluate on accuracy score\n", " # remember the key to the feed_dict dictionary must match the variable we use\n", " # as the placeholder for the data in the beginning\n", " sess.run(init)\n", " for i in range(n_iterations):\n", " # X_batch, y_batch = mnist.train.next_batch(batch_size)\n", " _, acc = sess.run([train_step, accuracy], feed_dict = {X: X_train, y: Y_train})\n", " \n", " # simply prints the training data's accuracy for every n iteration\n", " if i % 50 == 0:\n", " print(acc)\n", " \n", " # after training evaluate the accuracy on the testing data\n", " acc = sess.run(accuracy, feed_dict = {X: X_train, y: Y_train})\n", " print('test:', acc)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that we did not have to worry about computing the gradient to update the model, the nice thing about Tensorflow is that, once we've defined the structure of our model it has the capability to automatically differentiate mathematical expressions. This means we no longer need to compute the gradients ourselves! In this example, our softmax classifier obtained pretty nice result around 90%. But we can certainly do better with more advanced techniques such as convolutional deep learning." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Reference" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- [Blog: What is a TensorFlow Session?](http://danijar.com/what-is-a-tensorflow-session/)\n", "- [Github: Tensorflow Examples - Linear Regression](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/2_BasicModels/linear_regression.ipynb)\n", "- [Tensorflow Documentation: Getting Started With TensorFlow](https://www.tensorflow.org/get_started/get_started)\n", "- [TensorFlow Documentation: MNIST For ML Beginners](https://www.tensorflow.org/get_started/mnist/beginners)" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" }, "toc": { "nav_menu": { "height": "169px", "width": "252px" }, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": "block", "toc_window_display": true }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 1 }