{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "In a previous [post](http://www.danvatterott.com/blog/2016/04/29/an-introduction-to-neural-networks-part-1/), I described how to do [backpropogation](https://en.wikipedia.org/wiki/Backpropagation) with a 2-layer [neural network](https://en.wikipedia.org/wiki/Artificial_neural_network). I've written this post assuming some familiarity with the previous post. \n", "\n", "When first created, 2-layer neural networks [brought about quite a bit of excitement](https://en.wikipedia.org/wiki/Perceptron), but this excitement quickly dissipated when researchers realized that 2-layer [neural networks could only solve a limited set of problems](https://en.wikipedia.org/wiki/Perceptrons_%28book%29). \n", "\n", "Researchers knew that adding an extra layer to the neural networks enabled neural networks to solve much more complex problems, but they didn't know how to train these more complex networks.\n", "\n", "In the previous post, I described \"backpropogation,\" but this wasn't the portion of backpropogation that really changed the history of neural networks. What really changed neural networks is backpropogation with an extra layer. This extra layer enabled researchers to train more complex networks. The extra layer(s) is(are) called the *hidden layer(s)*. In this post, I will describe backpropogation with a hidden layer. \n", "\n", "To describe backpropogation with a hidden layer, I will demonstrate how neural networks can solve the [XOR problem](https://en.wikipedia.org/wiki/Exclusive_or).\n", "\n", "In this example of the XOR problem there are four items. Each item is defined by two values. If these two values are the same, then the item belongs to one group (blue here). If the two values are different, then the item belongs to another group (red here).\n", "\n", "Below, I have depicted the XOR problem. The goal is to find a model that can distinguish between the blue and red groups based on an item's values. \n", "\n", "This code is also available as a jupyter notebook on [my github](https://github.com/dvatterott/jupyter_notebooks). " ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEcxJREFUeJzt3X+MZWddx/H3Z1wacNXyuzJDu9kMP7QaAwilxCijZLUl\n2hKsUoTgojGExEZjYlDDZHYzRsWoibSYplpYGltqEwItwQaWwFAhWaiUgo0trZvp0s5oMUIJLJAs\nzNc/7p3ldpifz96Zc2f3/Upu9t5znnvOd06emc+e5/xKVSFJ0laNdV2AJGl3MkAkSU0MEElSEwNE\nktTEAJEkNTFAJElNOg+QJDcmeSzJF9eY/8okjye5p/96+07XKEn6QXu6LgB4D3AtcNM6be6qqit2\nqB5J0iZ0vgdSVZ8CvrZBs+xELZKkzes8QDbpFUnuTfLhJBd3XYwkaTSGsDbyOeCiqvpWksuBDwIv\n6LgmSTrnjXyAVNU3B97fmeQfkjy9qr66sm0Sb+wlSVtUVU2HCUZlCCuscZwjyQUD7y8Bslp4LKsq\nX0N4zczMdF7D2fRye7o9R/V1JjrfA0lyCzAFPCPJl4EZ4DygquoG4KokbwVOAd8GXtdVrZKk7+s8\nQKrqtzaY/y7gXTtUjiRpk0ZlCEsjZmpqqusSzipuz+Fye46GnOkY2ChJUmfTzyNJ2y0JtcsPokuS\ndhkDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFAJElNDBBJ\nUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFAJElNDBBJ\nUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFAJElNDBBJ\nUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSk84DJMmNSR5L8sV12rwzyUNJ\n7k3yop2sT5K0uj1dFwC8B7gWuGm1mUkuByar6vlJXg5cD1y6g/WdU07Mz3NkepqlhQXGJiY4ODvL\nvv37uy5LAmB+/gTT00dYWFhiYmKM2dmD7N+/r+uyzlmdB0hVfSrJej3gSvrhUlWfSXJ+kguq6rGd\nqfDccWJ+nmsPHODw8ePsBU4CM8eOcc3Ro4aIOjc/f4IDB67l+PHD0O+hx47NcPToNYZIRzofwtqE\nCeCRgc8L/WkasiPT06fDA3q/ooePH+fI9HSXZUkATE8fGQgPgL0cP36Y6ekjHVZ1but8D2TYDh06\ndPr91NQUU1NTndWy2ywtLJz+1Vy2F1haXOyiHOkJFhaWYJUeuri41EU5u9bc3Bxzc3NDWdZuCJAF\n4MKBz8/tT1vVYIBoa8YmJjjJE39FTwJj4+MdVSR938TEGKzSQ8fHd8NAyuhY+R/rw4cPNy9rVLZ8\n+q/V3AG8CSDJpcDjHv/YHgdnZ5mZnORk//NJYGZykoOzs12WJQEwO3uQyckZGOihk5MzzM4e7Kym\nc12qqtsCkluAKeAZwGPADHAeUFV1Q7/NdcBl9HrOm6vqnjWWVV3/PLvd6bOwFhcZGx/3LCyNlOWz\nsBYXlxgf9yysYUhCVa31H/j1v3s2/cE1QCRpa84kQEZlCEuStMsYIJKkJgaIJKmJASJJamKASJKa\nGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKa\nGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKa\nGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqcmaAZLkwiS3Jvm3JH+W5EkD8z64M+VJkkbVensg\n7wbmgGuA5wCfTPKM/rx921yXJGnE7Vln3rOq6vr++2uSvBG4K8kVQG1/aZKkUbZegDwpyZOr6jsA\nVfXPSf4H+Aiwd0eqkySNrPWGsP4JePnghKr6GPAbwH3bWZQkafSl6uwZjUpSZ9PPI0nbLQlVlZbv\nehqvJKmJASJJamKASJKabBggSS5IcmOSO/ufL07yu9tfmiRplG1mD+QIvVN3x/ufHwT+cLsKkiTt\nDpsJkGdW1W3AEkBVfRf43rZWJUkaeZsJkJP9W5gUQJJLga8Pq4AklyV5IMmDSd62yvxXJnk8yT39\n19uHtW5JUrv1rkRf9kfAHcBkkk8DzwKuGsbKk4wB1wGvAhaBu5PcXlUPrGh6V1VdMYx1SpKGY8MA\nqap7krwSeCEQ4EtVdWpI678EeKiqTgAkuRW4ElgZIE0XuUiSts+GAZLkTSsmvaR/5eJNQ1j/BPDI\nwOdH6YXKSq9Ici+wAPxxVf3nENYtSToDmxnCetnA+yfTG266BxhGgGzG54CLqupbSS4HPgi8YK3G\nhw4dOv1+amqKqamp7a5PknaNubk55ubmhrKsLd8LK8lTgVur6rIzXnnvgPyh5WUl+ROgquod63xn\nHvjZqvrqKvO8F5YkbcFO3wvrJLC/ZWWruBt4XpJ9Sc4DrqZ3wP60JBcMvL+EXuj9QHhIknbWZo6B\nfIjvP0BqDLgYuG0YK6+q7yX5feCj/WXfWFX3J3lLb3bdAFyV5K3AKeDbwOuGsW5J0pnZcAirfwbW\nsu8CJ6rq0W2tqpFDWJK0NWcyhOXzQCTpHHYmAbLmEFaSb7D6s89Db3jpx1pWKEk6O6wZIFX1oztZ\niCRpd9nMdSAAJHk2vetAAKiqL29LRZKkXWEzzwO5IslDwDzwSeBh4M5trkuSNOI2cx3ILHAp8GBV\n7ad3Jfqxba1KkjTyNhMgp6rq/4CxJGNV9QngpdtclyRpxG3mGMjjSX4EuAu4OclX6F2NLkk6h23m\nQsK9wHfonb77BuB84Ob+XslI8ToQSdqabbmQMMm7gFuq6tNnUtxOMkAkaWu262aKDwJ/k+ThJH+d\n5MVt5UmSzkabGcLaR+8uuVcDTwHeB7yvqh7c/vK2xj0QSdqaHbsXVn8v5N3Az1TVD7WscDsZIJK0\nNdv6PJAke5L8WpKb6V1A+CXgtS0rkySdPdY7iH4AeD3wauCzwK3A7VU1sqfwugciSVuzXWdhfRy4\nBXh/VX3tDOrbMQaIJG2NzwPpM0AkaWt2+pnokiQZIJKkNgaIJKmJASJJamKASJKaGCCSpCYGiCSp\niQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSp\niQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCadB0iS\ny5I8kOTBJG9bo807kzyU5N4kL9rpGiVJP2hPlytPMgZcB7wKWATuTnJ7VT0w0OZyYLKqnp/k5cD1\nwKWdFHwOmJ8/wfT0ERYWlpiYGGN29iD79+/ruiwJgBPz8xyZnmZpYYGxiQkOzs6yb//+rss6Z3Ua\nIMAlwENVdQIgya3AlcADA22uBG4CqKrPJDk/yQVV9diOV3uWm58/wYED13L8+GFgL3CSY8dmOHr0\nGkNEnTsxP8+1Bw5w+Pjxfu+EmWPHuOboUUOkI10PYU0Ajwx8frQ/bb02C6u00RBMTx8ZCA+AvRw/\nfpjp6SMdViX1HJmePh0e0Oulh48f58j0dJdlndO63gMZukOHDp1+PzU1xdTUVGe17DYLC0t8PzyW\n7WVxcamLcqQnWFpYWKV3wtLiYhfl7Fpzc3PMzc0NZVldB8gCcNHA5+f2p61sc+EGbU4bDBBtzcTE\nGL2BgcFf05OMj3e9oyrB2MTEKr0TxsbHO6pod1r5H+vDhw83L6vrvwx3A89Lsi/JecDVwB0r2twB\nvAkgyaXA4x7/2B6zsweZnJyh92sJcJLJyRlmZw92VpO07ODsLDOTkwO9E2YmJzk4O9tlWee0VFW3\nBSSXAX9PL8xurKq/SvIWoKrqhn6b64DL6PWZN1fVPWssq7r+eXa75bOwFheXGB/3LCyNltNnYS0u\nMjY+7llYQ5CEqkrTd8+mP7gGiCRtzZkESNdDWJKkXcoAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFA\nJElNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFA\nJElNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFA\nJElNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFA\nJElNDBBJUhMDRJLUxACRJDXZ09WKkzwN+BdgH/Aw8JtV9fVV2j0MfB1YAk5V1SU7WKYkaQ1d7oH8\nCfCxqnoh8HHgT9dotwRMVdWLDY+dMzc313UJZxW353C5PUdDlwFyJfDe/vv3Aq9Zo11wqG3H+Qs6\nXG7P4XJ7joYu/zA/u6oeA6iq/wGevUa7Ao4muTvJ7+1YdZKkdW3rMZAkR4ELBifRC4S3r9K81ljM\nz1XVfyd5Fr0gub+qPjXkUiVJW5Sqtf5ub/OKk/vpHdt4LMmPA5+oqp/c4DszwDeq6u/WmN/NDyNJ\nu1hVpeV7nZ2FBdwBHATeAfw2cPvKBkl+GBirqm8m2Qv8MnB4rQW2bgRJ0tZ1uQfydOA24ELgBL3T\neB9P8hzgH6vqV5PsBz5Ab3hrD3BzVf1VJwVLkp6gswCRJO1uu/b02CRPS/LRJF9K8pEk56/R7uEk\nX0jy+SSf3ek6R12Sy5I8kOTBJG9bo807kzyU5N4kL9rpGneTjbZnklcmeTzJPf3XaieUCEhyY5LH\nknxxnTb2zU3aaHu29M1dGyB4IeIZSzIGXAf8CvBTwOuT/MSKNpcDk1X1fOAtwPU7XugusZnt2XdX\nVb2k//rzHS1yd3kPvW25Kvvmlq27Pfu21Dd3c4B4IeKZuwR4qKpOVNUp4FZ623XQlcBNAFX1GeD8\nJBeg1Wxme0KvT2oD/dP1v7ZOE/vmFmxie8IW++Zu/sPqhYhnbgJ4ZODzo/1p67VZWKWNejazPQFe\n0R9y+XCSi3emtLOSfXP4ttQ3uzyNd0NeiKiz0OeAi6rqW/0hmA8CL+i4Jgka+uZIB0hVHVhrXv9g\n0AUDFyJ+ZY1l/Hf/3/9N8gF6wwwGSM8CcNHA5+f2p61sc+EGbdSz4fasqm8OvL8zyT8keXpVfXWH\najyb2DeHqKVv7uYhrOULEWGdCxGT/Ej//fKFiPftVIG7wN3A85LsS3IecDW97TroDuBNAEkuBR5f\nHjrUD9hwew6O0Se5hN6p9IbH2sLa4/L2za1bc3u29M2R3gPZwDuA25L8Dv0LEQEGL0SkN/z1gf4t\nTpYvRPxoVwWPmqr6XpLfBz5K7z8TN1bV/Une0ptdN1TVvyZ5dZL/Ak4Cb+6y5lG2me0JXJXkrcAp\n4NvA67qreLQluQWYAp6R5MvADHAe9s0mG21PGvqmFxJKkprs5iEsSVKHDBBJUhMDRJLUxACRJDUx\nQCRJTQwQSVITA0RaQ5KPJzmwYtofJHnXBt/7xpDW//NJPpfkVJLXDmOZ0jAZINLabgFev2La1f3p\n6xnWxVUn6N1l4eYhLU8aKgNEWtv7gVcn2QOQZB/wnKr6dJK9ST6W5N/7Dyy7YuWX+w/o+dDA52uT\nLN964yVJ5vp3ib5ztduQV9WXq+o+hhdI0lAZINIaquprwGeBy/uTrgZu67//DvCaqnop8EvA3661\nmJUT+oF0LfDrVfUyeg/6+Yshli7tiN18LyxpJ9xKLzg+1P/3d/rTA/xlkl+g99TL8STPrqpV7wq9\nwguBn6b3eIHlB54tDr1yaZsZINL6bgf+LsmLgadU1ef7098APBN4cVUtJZkHnrziu9/liXv5y/MD\n3FdVP7eNdUvbziEsaR1VdRKYA94NvG9g1vnAV/rh8YvAvoF5y7fLPgFcnORJSZ4KvKo//UvAs/q3\nICfJnk08/c3H4GrkGCDSxt4H/AxPDJCbgZcl+QLwRuD+gXkFUFWP0jtmch+9obB7+tNPAVcB70hy\nL/B54BUrV5rkpUke6be9Psl/DPnnks6It3OXJDVxD0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEk\nNTFAJElNDBBJUpP/ByQyL1jyZx4eAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np #import important libraries. \n", "from matplotlib import pyplot as plt\n", "import pandas as pd\n", "%matplotlib inline\n", "\n", "plt.plot([0,1],[0,1],'bo')\n", "plt.plot([0,1],[1,0],'ro')\n", "plt.ylabel('Value 2')\n", "plt.xlabel('Value 1')\n", "plt.axis([-0.5,1.5,-0.5,1.5]);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again, each item has two values. An item's first value is represented on the x-axis. An items second value is represented on the y-axis. The red items belong to one category and the blue items belong to another.\n", "\n", "This is a non-linear problem because no linear function can segregate the groups. For instance, a horizontal line could segregate the upper and lower items and a vertical line could segregate the left and right items, but no single linear function can segregate the red and blue items. \n", "\n", "We need a non-linear function to seperate the groups, and neural networks can emulate a non-linear function that segregates them. \n", "\n", "While this problem may seem relatively simple, it gave the initial neural networks quite a hard time. In fact, this is the problem that depleted much of the original enthusiasm for neural networks.\n", "\n", "Neural networks can easily solve this problem, but they require an extra layer. Below I depict a network with an extra layer (a 3-layer network). To depict the network, I use a repository available on my [github](https://github.com/dvatterott/visualise_neural_network). " ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAHMCAYAAAB2sntRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVPX+P/AXAgqiqOwz7G4giYKDEnrdpVQEtdQ0dzHX\nXO639FqZeVNbNCuv96qI3kxLTFyB3EDLnRBQsRQjZVEYEpFFdmb4/P7oyi+SnXPmLPN+Ph48Hskw\nn8/7BJ83H15z5hwDxhgIIYToRiuhCyCEEH1CTZcQQnSImi4hhOgQNV1CCNEharqEEKJDRvU9aGBg\nQKc2EEJIMzDGDGr7fL1N939P5L4aQgiRMQODWvstAIoXCCFEp6jpEkKIDlHTJYQQHaKmSwghOkRN\nlxBCdIiaLiGE6BA1XUII0SFquoQQokPUdAkhRIeo6RJCiA5R0yWEEB1q8NoLhIhFeXk5cnJyUFFR\nAY1GA0NDQxgbG8PS0hJmZmZCl0dIo1DTJaLCGMP9+/cRHx+PhIQEJCUlITMzE2q1GoWFhbCysoKJ\niQkMDQ1RVVWFiooK5OTkoE2bNlAoFFAqlfDw8IBKpYJKpYKHhweMjOjHnIiHQX1XETMwMGB0lTHC\nt7y8PJw8eRIRERE4c+YMzMzMqpuml5cXHB0doVQqYWVlhVatnk/EGGPIz89HVlYWMjMz8fPPPyMh\nIQHx8fF4+PAhBg4ciKCgIAQGBsLR0VGAIyT6xsDAoM5LO1LTJYIoLy/H4cOHsXv3bly7dg1Dhw5F\nYGAgRo8eDaVSydk8BQUFiI6ORkREBE6cOAEnJyfMnDkTM2fORMeOHTmbh5A/o6ZLRCMjIwMhISHY\nvXs3evbsiQULFmD06NFo27Yt73NrNBpcuHABoaGhOHXqFCZOnIhFixbBy8uL97mJfqmv6dLZC0Qn\nfv/9dyxZsgTe3t4oLi7Gjz/+iJiYGEyYMEEnDRcAjIyMMGzYMISFheHOnTtwdnbGmDFj8Morr+DO\nnTs6qYEQarqEV0+fPsWaNWvg4eEBQ0NDJCcn48svv4S7u7ugddnZ2eG9995DSkoK/Pz8MGjQIAQH\nB+Phw4eC1kXkj5ou4U1MTAw8PT1x//59JCQk4Msvv4S1tbXQZdVgamqKFStWICUlBTY2NvD29kZo\naCjdporwhjJdwrnCwkKsWLECJ0+exM6dOzFy5EihS2q0n3/+GbNmzYKlpSVCQ0Ph5OQkdElEgijT\nJTpz69YteHt7Q6vV4tatW5JquADQs2dPXL16FYMHD4aPjw+ioqKELonIDO10CWeOHTuGN954A19+\n+SWmTp0qdDktFhsbiwkTJmDJkiVYuXJlvXd4JeTP6JQxwivGGDZs2ICQkBAcOXIEffv2FbokzmRm\nZmLcuHHo3r07du/eDRMTE6FLIhJA8QLhDWMMy5cvx9GjRxEXFyerhgsA9vb2uHDhAiorKxEUFISS\nkhKhSyISR02XNBtjDG+++Sbi4uJw9uxZKBQKoUvihampKcLCwqBQKBAQEECNl7QIxQukWRhjePvt\nt3Hp0iVER0fD3Nxc6JJ4p9VqMWvWLDx69AgRERFo06aN0CURkaJ4gXBu+/btOHXqFE6dOqUXDRcA\nDA0N8dVXX8HU1BSLFi2ic3lJs9BOlzTZDz/8gMmTJ+PKlSvo0qWL0OXoXFFREfr374+5c+di6dKl\nQpdDRIjOXiCcSU1NhZ+fH7799lsMHz5c6HIEk5aWBj8/P+zbtw8jRowQuhwiMtR0CSe0Wi0GDBiA\nyZMnY/ny5UKXI7gff/wRU6ZMQVJSkuje3kyERU2XcGLjxo04ffo0oqOja72YuD56++238eDBA3z3\n3XdCl0JEhJouabE7d+5g4MCBuHbtGlxdXYUuRzRKS0vh7e2N9evXY8KECUKXQ0SCmi5pEcYYBgwY\ngOnTp2PhwoVClyM6V69erb4mL92NggB0yhhpoSNHjqCsrAzz588XuhRR8vPzw6hRo7Bp0yahSyES\nQDtdUi+NRoOePXtiy5YtePnll4UuR7QyMjLg7e2Nn3/+WbbvzCONRztd0mx79uyBQqHASy+9JHQp\noubk5IRZs2Zh3bp1QpdCRI52uqROWq0W3bp1wzfffIP+/fsLXY7o5ebmomvXrrhz5w7s7OyELocI\niHa6pFlOnz4NS0tLariNZGlpiUmTJmHXrl1Cl0JEjHa6pE5jxozBq6++itmzZwtdimTcuHEDgYGB\nSE1NhZGRkdDlEIHQTpc0WWpqKmJjY/Haa68JXYqkeHl5wcnJCZGRkUKXQkSKmi6p1cGDBzFp0iS0\nbdtW6FIkZ86cOThw4IDQZRCRoqZLahUREYFx48YJXYYkjRkzBmfOnEFFRYXQpRARoqZLnvPo0SP8\n8ssvGDx4sNClSJKtrS3c3d1x/vx5oUshIkRNlzzn+++/h7+/P90ZoQWCgoJw/PhxocsgIkRNlzzn\n4sWLen2tXC6MGDECFy9eFLoMIkLUdMlzEhIS4OPjI3QZkubp6YmUlBSUlpYKXQoRGWq6pIbS0lKk\npKTA09NT6FIkzcTEBG5ubkhKShK6FCIy1HRJDUlJSXB3d6c8lwMqlQrx8fFCl0FEhpouqSElJQXu\n7u5ClyELPXr0wL1794Qug4gMNV1SQ1ZWFpRKJW/j79mzB7169YKZmRmUSiUWLVqEgoKCRj3X1dUV\n586d46wWrsf7K4VCgaysLN7GJ9JETZfUoFarebse7ObNm/HOO+9g8+bNKCwsRGxsLNLT0+Hv7w+N\nRsPLnEJSKBRQq9VCl0FEhpouqYGvne7Tp0+xdu1a/Pvf/4a/vz8MDQ3h5OSEgwcPIj09Hfv27cPs\n2bOxZs2a6uecP38ejo6OAIAZM2YgIyMDgYGBMDc3x2effYb09HS0atUKoaGhsLe3h729PTZv3lz9\n/KaOxzWlUkk7XfIcugwSqSE/Px+dOnXifNwrV66gvLwc48ePr/F5MzMzjBo1CtHR0TAxMXnueQYG\nf1yoae/evbh48SL++9//YujQoQCA9PR0AH/cCv3evXv47bffMGzYMHh7e2PYsGG11lHfeFzr1KkT\n8vLyeBmbSBftdEkNlZWVMDY25nzcx48fw8rKqtZbtysUCuTm5jZqnNouNbp27VqYmJigZ8+emD17\nNsLCwhpdF5+XLjU2NpZlbEJahpouqYExVr0b5JKVlRUeP36Mqqqq5x5Tq9WwsrJq1rgGBgZwcHCo\n/rezs7No/qRv1apVrcdL9Bs1XVKDkZERtFot5+P6+fmhTZs2OHLkSI3PFxUV4eTJkxg+fDjatm2L\nkpKS6sf++iJUbb8MGGN48OBB9b8zMjKqM2kzM7Mmj8cljUbDy18NRNqo6ZIaTE1NUVxczPm45ubm\nWLNmDZYsWYLTp09Do9EgLS0Nr732GpycnDB9+nR4eXnhxIkTyMvLQ3Z2NrZs2VJjDDs7O9y/f/+5\nsdetW4fS0lL88ssv+OqrrzB58mQAaPZ4XCkuLq41pyb6jZouqcHOzo6305xWrFiBjz76CG+//TY6\ndOgAPz8/ODs7IyYmBsbGxpg+fTp69eoFFxcXjBw5srp5PrNq1SqsW7cOFhYW+Pzzz6s/P3jwYHTt\n2hX+/v5YuXJl9cV6mjseV/g8/Y5IF90jjdSwdu1aVFVV4cMPPxS6lAalp6ejc+fOqKysrPUFOqEd\nO3YM//3vfxERESF0KUTH6B5ppNGkdkK/mDcFtNMltaGmS2pwcnJCWlqa0GU0Gt8vhrVEWlpa9Zsx\nCHmGmi6pwcvLC4mJiaLeQT7j7OwMrVYrymgBABITE+Hl5SV0GURkxPnTSgSjUChgYmJS/W4v0jyM\nMSQkJEClUgldChEZarrkOSqVCgkJCUKXIWlpaWkwNTWlTJc8h5oueU6/fv1w5coVocuQtCtXrqBf\nv35Cl0FEiJouec7o0aMRFRUldBmSFhUVhYCAAKHLICJE5+mS5zDG4OjoiLNnz8LNzU3ociSnoqIC\ntra2uHPnDuzs7IQuhwiAztMlTWJgYICgoCA6qb+ZLl68CDc3N2q4pFbUdEmtxo8fj7CwMEmcOiY2\nBw4ceO66wYQ8Q/ECqVVVVRW6deuG/fv3w9fXV+hyJCM/Px+urq5ITk6Gra2t0OUQgVC8QJqsVatW\nWLhwIbZt2yZ0KZLy9ddfY9SoUdRwSZ1op0vqlJubi65duyIlJaXZFxnXJ1VVVfDw8MCuXbvwt7/9\nTehyiIBop0uaxdLSEpMmTeLlpo1ydOjQIbRv3x4DBgwQuhQiYrTTJfXKzMxEr169kJSUBHt7e6HL\nEa3Kykp4eHhg+/btGDFihNDlEIHRTpc0m729PebOnYt//vOfQpciart374aLiws1XNIg2umSBj15\n8gRubm44d+4cPD09hS5HdAoKCtCjRw9ERETAx8dH6HKICNS306WmSxolNDQUISEhiI2NhZGRkdDl\niEpwcDCMjY2xY8cOoUshIkHxAmmxuXPnwsLCAhs3bhS6FFE5efIkzp49i02bNgldCpEI2umSRsvI\nyIBKpaKY4X/y8/Ph6emJPXv2VN8MkxCAdrqEI05OTti8eTNeffVV5OXlCV2OoLRaLV5//XW88sor\n1HBJk9BOlzTZ8uXLcfv2bZw4cUJv892VK1ciPj4ep0+fhrGxsdDlEJGhnS7h1GeffQbGGFasWCF0\nKYLYt28fDh8+jPDwcGq4pMn0c5tCWsTIyAjfffcd+vfvD3t7e7z99ttCl6QzJ0+exFtvvYVz587B\n0tJS6HKIBFHTJc1iYWGBmJgYDB48GMbGxli2bJnQJfEuOjoaM2fOxPHjx9GzZ0+hyyESRU2XNJuD\ngwPOnTuH4cOHo7y8HCtXrhS6JN5ERUVhzpw5OHLkCPz8/IQuh0gYvZBGWiwzMxMvvfQSBg0ahC1b\ntqB169ZCl8QZxhi2bt2Kjz76CMePH6drC5NGoXekEd4VFhZi6tSpKCwsxKFDh2BtbS10SS1WXl6O\nxYsXIy4uDsePH4erq6vQJRGJoLMXCO/Mzc1x7NgxDBgwQBa3cE9NTcWwYcPw5MkTXLlyhRou4Qw1\nXcIZQ0NDfPTRR9i8eTPGjx+PoUOHori4WOiymqSqqgqzZs1Cnz59MG7cOBw6dAjt2rUTuiwiIxQv\nEF68+eabOH78OMzMzBAaGoqBAwcKXVKDUlJSMH/+fPz2229wcHDA5cuXYWBQ61+IhNSL4gWiU7Gx\nsTh06BASEhKwYcMGTJs2DUFBQfj555+FLq1WarUaCxYsgJ+fHwICApCSkgKNRoOdO3cKXRqRIWq6\nhFNlZWWYPXs2tm7dChsbG7z66qu4e/cuhg0bhuHDh2PmzJn45ZdfhC4TAJCVlYV33nkHPXv2RPv2\n7XH37l289dZbaNOmDfbs2YPVq1cjPT1d6DKJzFDTJZxas2YNevbsiYkTJ1Z/zsTEBMuXL0dKSgq6\ndOmCESNGYOjQoQgPD0dlZaVO62OM4YcffsDEiRPxwgsvoKCgADdu3MCmTZtqvMPMw8MDb731FubO\nnQuK2AiXKNMlnImNjcW4ceOQlJQEGxubOr+uoqICx44dw7Zt25CcnIygoCAEBQVh+PDhMDU15bwu\njUaDq1evIjIyEseOHYOxsTEWL16MadOmwdzcvN7n9e/fH8HBwZg/fz7ndRH5ovN0Ce/Kysrg7e2N\nDz/8sMYutyG//fYbIiIiEBERgcTERAwaNAi+vr5QqVRQqVSwtbVtci2FhYVITExEQkIC4uPjER0d\nDUdHRwQGBiIoKAgqlarRL5Ddvn0bgwcPRnx8PJydnZtcC9FP1HQJ71auXInU1FSEh4c3e4wnT57g\n7NmziI+PR0JCAhITE9GmTRs4OjpCoVBAqVTC1tYWbdq0gZGREbRaLSorK5GTk4OsrCyo1WpkZWUh\nJycHvXr1qm7cw4YNg5OTU7Pr+uSTT3D27FmcOXOGzmYgjUJNl/CqsbFCUzHG8PDhQ2RmZkKtVkOt\nViM7OxsVFRXQaDQwNDSEsbExrK2toVAoqj9cXFw4vc4vxQykqajpEt40N1aQGooZSFNQ0yW84SJW\nkAqKGUhjUdMlvOArVhArihlIY1HTJZzTl1jhryhmII1BTZdwTp9ihb+imIE0hJou4ZS+xQp/RTED\naQg1XcIZfY0V/opiBlIfarqEM/ocK/wVxQykLtR0CSf0PVb4K4oZSF2o6ZIWo1ihdhQzkNpQ0yUt\nRrFC3ShmIH9FTZe0CMUK9aOYgfwVNV3SbBQrNA7FDOTPqOmSZqNYofEoZiDPUNMlzUKxQtNQzECe\noaZLmoxiheahmIEA1HRJM1Cs0HwUMxBquqRJKFZoGYoZCDVd0mgUK3CDYgb9Rk2XNBrFCtyhmEF/\nUdMljUKxArcoZtBf1HRJgyhW4AfFDPqJmi5pEMUK/KGYQf9Q0yX1oliBXxQz6B9quqROFCvoBsUM\n+oWaLqkTxQq6QzGD/qCmS2pFsYJuUcygP6jpkudQrCAMihn0AzVd8hyKFYRDMYP8UdMlNVCsICyK\nGeSPmi6pRrGCOFDMIG/UdEk1ihXEg2IG+aKmSwBQrCA2FDPIFzVdQrGCSFHMIE/UdAnFCiJGMYP8\nUNPVcxQriBvFDPJDTVePUawgDRQzyAs1XT1GsYJ0UMwgH9R09RTFCtJCMYN8UNPVQxQrSBPFDPJA\nTVcPUawgXRQzSB81XT1DsYK0UcwgfdR09QjFCvJAMYO0UdPVIxQryAfFDNJFTVdPUKwgLxQzSBc1\nXT1AsYI8UcwgTdR09QDFCvJFMYP0UNOVOYoV5I1iBumhpitjFCvoB4oZpIWaroxRrKA/KGaQDmq6\nMkWxgn6hmEE6qOnKEMUK+oliBmmgpitDFCvoL4oZxI+arsxQrKDfKGYQP2q6MkKxAgEoZhA7aroy\nQrECeYZiBvGipisTFCuQP6OYQbyo6coAxQqkNhQziBM1XRmgWIHUhWIG8aGmK3EUK5D6UMwgPtR0\nJYxiBdIYFDOICzVdCaNYgTQWxQziQU1XoihWIE1BMYN4UNOVIIoVSHNQzCAO1HQliGIF0lwUMwiP\nmq7EUKxAWoJiBuFR05UQihUIFyhmEBY1XQmhWIFwhWIG4VDTlQiKFQiXKGYQDjVdCaBYgfCBYgZh\nUNOVAIoVCF8oZtA9aroiR7EC4RPFDLpHTVfEKFYgukAxg25R0xUxihWIrlDMoDvUdEWKYgWiSxQz\n6A41XRGiWIEIgWIG3aCmK0IUKxChUMzAP2q6IkOxAhESxQz8o6YrIhQrEDGgmIFf1HRFhGIFIhYU\nM/CHmq5IUKxAxIRiBv5Q0xUBihWIGFHMwA+9bbr5+fm4fv060tLSkJWVhaysLKjVahQVFaGyshIA\nYGxsjHbt2kGhUECpVEKpVMLFxQXe3t7o2LEjZ7VQrEDEio+YobS0FElJSUhJSaled1lZWcjPz0dl\nZSUYYzAyMoKpqSns7OygVCqhUCjg5OQELy8vKBQKTuoQit403Xv37iEqKgpXr15FQkICsrOz0bt3\nb3Tt2hUKhaL6o0OHDjAyMgLwx59YBQUFUKvV1R+//fYbbt68CTs7O6hUKvj5+WHMmDHo0qVLs+qi\nWIGIGRcxw6NHj/D999/j4sWLSEhIQEpKCtzd3eHu7l7dUJVKJTp16gRjY2MYGBhAq9WiuLi4xtpL\nS0tDYmIiTExMoFKp0K9fP4wePRre3t6Syp1l3XQTExMRHh6OiIgI5ObmYsyYMRg8eDBUKhXc3Nxg\naGjYrHG1Wi3u3r2LhIQEnD9/HlFRUbC0tERQUBAmTpyIPn36NGocihWIFDQnZkhNTcXBgwcRERGB\nX375Bf7+/hg+fDh8fHzg6emJNm3aNKsWxhjS09ORkJCAK1euIDIyEqWlpQgMDMT48eMxfPhwtGrV\nqllj60p9TReMsTo//nhYfEpKStiePXtYv379mLOzM3vnnXdYbGws02q1vM2p1WpZbGwse+edd5iz\nszPr168f27NnDyspKan3eStWrGATJkzgrS5CuPLxxx+zESNGsKqqqjq/RqPRsO+//54FBAQwS0tL\ntnDhQnb69GlWVlbGa23Jycls48aNzNvbm3Xu3Jlt2rSJPX78mNc5W+J/vbP2vlrXA0yETbewsJCt\nXbuWWVlZsVGjRrHIyEim0Wh0XodGo2GRkZFs1KhRzMrKiq1du5YVFhY+93VXr15ltra27Pfff9d5\njYQ0VWVlJevbty/bsWNHrY+FhoYyV1dXplKp2H//+19WXFys8xqrqqpYbGwsmzFjBuvYsSObN28e\ne/jwoc7raIjkm25ZWRnbsmULs7W1ZdOmTWMpKSlCl1Tt119/ZVOnTmW2trZsy5Yt1b/xS0tLmbu7\nOzt48KDAFRLSeL/88guzsrJiaWlpjLE/mtyhQ4eYm5sbGzJkCLt8+bLAFf5/OTk5bOXKlczCwoKt\nXLmSPXnyROiSqkm66UZHRzNXV1c2evRoduPGDaHLqdONGzfY6NGjmaurK4uOjqZYgUjWs5jhl19+\nYX5+fszLy4udOnWq3thBSA8fPmRvvPEGs7KyYjt37hRFnZJsuoWFhWz+/PnM0dGRnTx5UrA6murk\nyZPMxsaGmZqasnv37gldDiFNVlZWxhwcHFi7du3Ytm3beH2thEtJSUlMpVKxl156iaWnpwtaS31N\nV5QvAV66dAmenp7QaDS4desWRo4cKXRJjTZy5Ehcv34d/v7+GDZsGC5duiR0SYQ0WmpqKgYPHgwH\nBwdER0dj4cKFoj9T4BlPT09cvXoVgwYNgkqlwt69e4UuqXZ1dWMm0E53586dzMbGhkVFRel8bq5F\nRUUxGxsbtnPnTqFLIaRB586dY7a2tuyLL76QzO62LklJSaxr165s+fLlrLKyUufzQwrxQkVFBVu8\neDFzc3Njd+/e1dm8fLt79y5zc3Njb775JquoqBC6HEKeU1VVxf7zn/8wW1tbFhMTI3Q5nHny5Anz\n9/dn/v7+LDc3V6dz19d0RfHmiJKSErzyyito1aoVwsLC0KFDB97n1KWCggJMmTIFjDEcPnwYbdu2\nFbokQgD8sel6++23cerUKURERDT7XZdipdFosGLFCpw8eRIxMTFwcHDQybz1vTlC8LCmuLgYAQEB\nsLa2RkREhOwaLgB06NABERERsLS0REBAAIqLi4UuiRAwxvDmm2/i0qVLuHTpkuwaLgAYGRnhiy++\nQHBwMAYPHoz09HShSxJ2p1taWooxY8bAyckJu3btavZbdqVCq9UiODgYDx8+RGRkJExNTYUuiegp\nxhiWL1+OuLg4nD59Gubm5kKXxLstW7bgX//6Fy5cuAB7e3te5xLltReqqqowYcIEmJiYYN++fbJv\nuM9otVpMmzYNFRUVCA8Pl8wrw0Re1q9fj6NHj+Ls2bOcXk1P7D799FPs3bsXV69e5fUXjSib7gcf\nfICzZ8/i3LlzaN26NS9ziFV5eTmGDRsGf39/rF27VuhyiJ45evQoli5diri4OMlfQrE5FixYgKys\nLBw7doy3TY/oLnhz8OBB5uTkxLKzs3kZXwqys7OZk5MTCw8PF7oUokeSkpKYlZUVi4uLE7oUwZSX\nl7NBgwaxd999l7c5IKZTxm7fvs2srKxYYmIi52NLTUJCArOysmK3b98WuhSiBwoKCljnzp3ZN998\nI3Qpgnv06BFzcXFhR44c4WX8+pquTuMFjUaDAQMGYNasWVi4cCFn40rZtm3bsHfvXly+fFlvcm0i\njAULFkCj0WDXrl1ClyIKV65cwauvvoqkpCRYW1tzOrZoThn7/PPPYWZmRjfB+5MFCxbA1NQUn3/+\nudClEBmLiYnBiRMnsHnzZqFLEY3+/ftj6tSpWLJkiU7n1dlO986dOxg4cCCuXbsGV1dXTsaUi9TU\nVPTt2xeXLl2Cu7u70OUQmXn69Ck8PT0REhKCl19+WehyRKW0tBReXl746KOP8Oqrr3I2rijOXggI\nCMCIESPw97//nZPx5OaLL77A2bNnERUVJXQpRGbWrFmD+/fv45tvvhG6FFG6cOECpk+fjrt378LE\nxISTMQVvuhcuXMDMmTORnJzc7PsmyV15eTnc3Nywb98+DBw4UOhyiEz8/vvv8PDwQEJCAlxcXIQu\nR7SCgoIwbNgwLF++nJPxBG26jDEMGDAACxcuxPTp01s0ltzt3bsXISEhuHTpkqTufErEa8mSJTA0\nNMSXX34pdCmiduvWLYwYMQIpKSmcvGlC0BfSTpw4gaKiIrz++ut8TyV5U6dORWFhIU6ePCl0KUQG\nMjIysH//frz33ntClyJ6np6eGDlypE5+OfHedLdu3YoVK1bQ6VCNYGhoiBUrVmDr1q1Cl0JkICQk\nBNOnT+f8dCi5WrlyJXbs2IHKykpe5+E1Xvjtt9/Qv39/ZGRkcBZQy11ZWRmcnJxw9epVWV71iehG\neXk5nJ2dcf78ebi5uQldjmQMGTIEixcvxsSJE1s0jmDxwo4dOzBnzhxquE1gYmKC2bNnY8eOHUKX\nQiTsyJEj6NmzJzXcJlq0aBG2bdvG6xy87XQrKyuhUCjovNxmuH//Pnx9faFWq2FkZCR0OUSChg8f\njoULF2LChAlClyIpFRUVcHJywqVLl9C1a9dmjyPITvfy5ctwcXGhhtsMnTt3hpOTEy5fvix0KUSC\n8vLycO3aNYwePVroUiSndevWCAoKQkREBG9z8NZ0IyIiEBQUxNfwssf3N57I18mTJzFkyBC6LVQz\nSbLpMsYQERGBwMBAPobXC4GBgYiIiACXFxwi+oE2PC0zfPhwJCYm4smTJ7yMz0vTTU1NRUlJCby8\nvPgYXi94e3ujuLgYqampQpdCJIQxhjNnzlC00AKmpqYYNGgQzp49y8v4vDTd+Ph4+Pr60ruqWsDA\nwAD9+vVDfHy80KUQCbl//z7MzMygVCqFLkXSfH19eVt7vDVdlUrFx9B6RaVSISEhQegyiITQ2uMG\nn2uPl6abkJBA33gOUNMlTUVrjxsqlQqJiYm8vKbCS9NNSkqiPJcDXl5eSEpKEroMIiG09rhha2uL\nNm3a4OHDh5yPzXnTLS8vR0FBAWxtbbkeuoaePXviwoULtT52/vx5ODo61vnc2bNnY82aNXyVxhk7\nOzvk5+ceKIeqAAAgAElEQVSjoqJC6FKIRGRmZtb7s88FfVh7AODo6IjMzEzOx+W86WZnZ8PW1rZF\ntzZ2dXXFuXPnanzu66+/rnGd2Z9//hmDBg2qcwwhXsTLzs7G2LFjYW9vj1atWiEjI6NF47Vq1Qo2\nNjbIzs7mqEIid2q1ukUvokl17Z04cQIDBw5Ep06doFQqMW/ePBQXF7doTIVCAbVazVGF/x/nTbel\n3/T6iP1siFatWmHUqFE4cuQIZ7UqlUpevvFEfsrLy1FYWAgrKyvOxxb72issLMT7778PtVqNO3fu\n4OHDh1ixYkWLxuRr7XHedH///XfY2NhwPexz/vwbuaysDLNmzYKFhQV69uyJa9eu1fja69evQ6VS\noUOHDpg8eTLKyspqPB4VFQVvb2906tQJf/vb33Dr1q0a82zevBm9e/dGp06dMGXKlDr/3LexscGC\nBQvg4+PDWQBva2tLO13SKDk5ObCysmrRX5mNIca1N3nyZLz00kswMTFBhw4d8MYbb7T4bfR8rT1e\nMl0+ripWXxNbu3YtUlNTkZqaitOnT+Prr7+ufqyyshLjx4/HzJkz8eTJE0ycOBGHDx+ufvz69esI\nDg5GaGgonjx5gvnz5yMoKKjGNTXDw8Nx5swZpKam4ubNm9izZw/nx1eXNm3aUKZLGqWiooLW3v+c\nP38eL7zwQtMP9k/4WnucN12NRsPJlbHGjRsHCwuL6o/FixfX+bXh4eFYvXo1OnToAHt7eyxdurT6\nsatXr0Kj0WDp0qUwNDTEq6++ir59+1Y/HhoaWr07NTAwwPTp09GmTRvExsZWf82yZctga2uLjh07\nIjAwEDdu3Gjx8TWWkZERNBqNzuYj0qXRaDi5WYDU1150dDT27duHdevWNfP/wB/4WnucN11DQ0No\ntdoWj3P8+HE8efKk+qO+a1xmZWXBwcGh+t/Ozs7V/61Wq2Fvb1/j6//8eHp6OjZv3lz9A9apUyc8\nfPgQWVlZ1V/z5zMx2rZti6KiohYdW1NotVq66wZpFENDQ1RVVbV4HCmvvdjYWEydOhWHDx9u8U0A\n+Fp7nDfd1q1bc7Ilb0omqlAo8ODBg+p/p6en13jsr6d9/PmsAkdHR7z33nvVP2B5eXkoKirCa6+9\n1oLquVNZWQljY2OhyyASYGxsrNdr7/r16xg3bhz27NmDIUOGNGuMP+Nr7XHedK2srPD48WOuh63X\npEmT8PHHHyM/Px8PHz7Ev//97+rH/Pz8YGRkhK1bt0Kj0eDIkSOIi4urfvyNN97Ajh07qj9XXFyM\nEydONPt0k/Ly8uoXC8rKylBeXt6CI/vjxRG6xxVpDEtLS+Tk5Oj0ynRiWXs///wzRo0aha1bt3J2\nsR++1h7nTVehUNT486A5GnN6yp+/5oMPPoCTkxNcXV0xcuRIzJgxo/oxY2NjHDlyBF999RUsLS0R\nHh6OV199tfpxlUqF0NBQvPnmm7CwsED37t1rvBjQ1FNlTE1NYW5uDgMDA7i7u7f4mqZZWVlQKBQt\nGoPoBzMzM7Rp0wb5+fnNHkOqa+/zzz/H48ePERwcjPbt26N9+/bw9PRs9PNrw9fa4/x2PcXFxbCy\nskJJSYnoz+0TO8YY2rZti9zcXLogNWkUd3d3HD58uMWv3BNgwIAB+OSTT2q8MaSxdHq7HjMzM7Ru\n3Rp5eXlcD6138vLy0Lp1a2q4pNGUSiUvb13VR3ztdHk5i9rDwwM///wzH0PrlVu3btGOhTQJrT1u\nFBYWIicnBy4uLpyPzUvTValUdPFtDtC1UUlT0eVAuZGYmIhevXrxcjdu3poufeNbjq6NSpqK1h43\n+Fx7vDRdHx+f596DTZqOdrqkqTw8PPDgwQMUFBQIXYqk8bn2eMt0CwoKcP/+fT6G1wv37t1DYWEh\nZbqkSYyMjDBw4EBER0cLXYpkaTQaREdHY9iwYbyMz0vTNTQ0xJgxYxAZGcnH8HohMjISY8aM4f2K\nUUR+goKCEBERIXQZknX16lU4OjrCycmJl/F5W9GBgYHUdFsgMjISgYGBQpdBJCgwMBAnTpygCyU1\nE99rj7em6+/vj7i4OOTm5vI1hWzl5ubi2rVrGDFihNClEAl6tkur65Y6pG6MMRw7dgxBQUG8zcFb\n0zUzM6u++ARpmq+++grjx4+HmZmZ0KUQiZo5cyZCQ0OFLkNyfvzxRxgbG/P6AjbnbwP+s9jYWEyb\nNg2//vorZZONVFVVhW7dumH//v3w9fUVuhwiUfn5+XBxcUFycjLs7OyELkcyJk6ciCFDhtR7DeHG\n0OnbgP/M19cX5ubm9EpqE5w5cwYdO3ZEv379hC6FSFjHjh0xceJE7N69W+hSJCMrKwsxMTGYPn06\nr/Pw2nQNDAywePFibN68mc9pZGXz5s1YvHgxXSyItNjixYuxfft2lJaWCl2KJGzduhVTpkyBubk5\nr/PwGi8Af9y3qUePHggNDeXtvDe5OHv2LObPn487d+7QhcsJJ8aPH4/+/fu3+M64cpeVlQVPT0/c\nuHEDjo6OLR6vvniB96YLAGFhYfjiiy/w008/0Q6uDowx9OvXD2+99RYmT54sdDlEJu7cuYNBgwYh\nJSUFHTt2FLoc0Vq4cCHatWuHTZs2cTKeYJnuM6+99hoqKytr3AmU1HT48GFotVpMmjRJ6FKIjPTo\n0QNBQUH49NNPhS5FtFJSUhAeHo5Vq1bpZD6d7HQB4Ny5c5g1axZu3bqFDh06cDKmXOTn58PT0xN7\n9+7F0KFDhS6HyMzDhw/h7e2NH374AT179hS6HFGpqqrCiBEjEBAQgLfeeouzcQWPF55ZsGABNBoN\ndu3axdmYchAcHIzWrVtj+/btQpdCZCo0NBQhISGIjY3l5XKFUrVt2zbs27cPly5d4vTOv6Jpuk+f\nPoWnpyd27NiBkSNHcjaulJ08eRKLFi1CUlIS2rdvL3Q5RKYYY3j55ZcxZMgQvPvuu0KXIwqpqano\n27cvLl26BHd3d07Hrq/pgjFW58cfD3MrOjqa2dvbs6ysLM7Hlppff/2VtW7dmoWEhAhdCtED165d\nY8bGxmzfvn1ClyK4srIy1r9/f7Zx40Zexv9f76y1r+p0p/vMunXr8P333+PHH3+EiYkJ5+NLQVlZ\nGYYMGQJHR0f8+OOP2LBhA9544w06u4Pw4uzZs5g+fTqGDBmCH3/8EdeuXYO9vb3QZQmCMYa5c+ci\nLy8Phw4d4uXdsqLa6TLGWFVVFZs4cSKbOXMmq6qq4mUOMauqqmIzZsxgEydOZFVVVSw5OZn16tWL\nTZkyhRUWFgpdHpERjUbD1qxZw5RKJYuJiWGMMfbxxx8zHx8fVlJSInB1wtiyZQvz9PRkT58+5W0O\n1LPTFaTpMsZYUVER8/LyYhs2bOBtDrHasGED8/b2ZkVFRdWfKykpYfPmzWPdunVjN27cELA6IhdZ\nWVls6NChbNiwYUytVld/vqqqik2ZMoVNnDiRaTQaASvUvaioKGZra8vu37/P6zyibLqMMZaZmcm6\ndu3KvvjiC17nEZMvvviCde3alWVmZtb6+LfffsusrKxYSEiIXv4VQLgRExPDFAoFW7t2ba2NtbS0\nlPn7+7OZM2cyrVYrQIW6d+bMGWZtbc1iY2N5n0u0TZcxxtLT05mrqyv77LPPeJ9LaJs2bWKurq4s\nPT293q+juIE0V21xQl2Ki4vZkCFD2PTp01llZaWOKhTGiRMnmLW1Nbt48aJO5hN102WMsQcPHrDu\n3buzVatWyfK3rlarZatWrWLdunVjDx48aNRzKG4gTVVXnFCf4uJi9tJLL7GxY8fymnEKae/evcza\n2ppduXJFZ3OKvukyxtijR4/YoEGDWGBgICsoKNDZvHwrKChggYGBbNCgQezRo0dNfj7FDaQxGooT\n6lNWVsbmzJnDPD09WWpqKj8FCkCj0bAVK1awzp07s1u3bul0bkk0XcYYKy8vZ/Pnz2cvvPACS0lJ\n0encfEhJSWEeHh5swYIFrLy8vNnjUNxA6tKUOKE+VVVVbMuWLczOzo798MMP3BUokLy8PDZq1Cg2\ndOhQ9vjxY53PL5mm+8y2bduYlZUV27ZtmyTjBq1Wy/7zn/8wS0tLtm3bNk7GpLiB/FVz4oSGREdH\nMzs7O/b2229L9pSyEydOMAcHB7Z06VJWUVEhSA2Sa7qMMXb79m3m6+vLhg4dyvvpHVy6f/8+Gzp0\nKPP19WV37tzhfHyKGwhjLYsTGvLo0SM2adIk5ubmptMctKXy8vLYnDlzmLOzc4t2/VyQZNNljLHK\nykr26aefMgsLC7Z69WpRZ735+fls9erVzMLCgn366ae8nv9IcYP+4ipOaIzw8HBmZ2fHZs+ezTIy\nMnidqyUqKirY9u3bmUKhYPPnzxfFmpBs030mNTWVzZw5k9nY2LDPP/+clZaWCl1StdLSUrZ582Zm\nY2PDZs6cydLS0nQyL8UN+oePOKEheXl57N1332UWFhbsrbfeEiQfrYtWq2Xfffcd69atGxsxYgS7\ndu2a0CVVk3zTfebWrVssMDCQ2drasvfee6/B8135lJ6ezt577z1ma2vLAgMDdf7q6DMUN+gHPuOE\nxsjKymILFy5kHTt2ZPPmzRP0F31eXh778ssvmZubG/Px8WHR0dGC1VIX2TTdZ27fvs2WLl3KLCws\n2NixY9mhQ4d0co5hYWEhO3ToEBs7diyzsLBgS5cu5SW3bSqKG+RLl3FCY6jVarZu3Trm4ODA+vfv\nz3bt2sWys7N5n7eiooLFxMSwuXPnso4dO7IpU6awixcvinajUV/TFeQqY1wpKipCWFgYDh06hKtX\nr2LAgAEICgrCoEGD4O7u3uKLEmu1WiQnJ+PChQuIiIjA5cuX4efnhwkTJuD111+HmZkZR0fScqWl\npVi+fDl++OEHhIeHo3fv3kKXRFpIrVZj6tSpMDAwwLfffgs7OzuhS6qm0WgQGRmJsLAwnDlzBh4e\nHggMDMSIESPg6enZ4qsHMsaQlpaGK1euICoqCqdOnYKbmxvGjx+PWbNmwdbWlqMj4YdoLmLOp8LC\nQpw+fRqRkZG4evUq1Go1evfuDZVKha5du0KhUEChUECpVMLc3Lz66vkajQaFhYXIysqCWq2GWq3G\nb7/9hoSEBNy8eRMKhQJ+fn4IDAzEyy+/zPvtmVtq//79WLZsGV0qUuKeXYpx/vz5WL16Nad3NeBa\neXk5zp8/j4iICFy8eBEpKSlwc3ODSqVCjx49aqy9Tp06wdjYGK1atYJGo0FxcXH1ulOr1UhLS0Ni\nYiISExNhYmKCfv36ISAgAAEBAVAoFEIfaqPpRdP9q/z8/OpvXnp6OrKysqoba1FRESorK1FYWAhL\nS0u0a9eu+odCqVTC2dkZffr0QZ8+fSR5B9W7d+9i0qRJeOGFFxASEkJ3pJAQrVaLDz/8ELt27cLe\nvXsxfPhwoUtqstLSUiQlJSE+Ph737t2rXndZWVnIy8uDRqNBQUEBLCwsYGJiUr32FAoFHB0d4eXl\nBZVKJakm+1d62XQb43//Y4QugxcUN0iPmOMErsl57QEiuAU70T1TU1OEhIRg7dq1GDFiBHbu3Cnr\nH3KpO3v2LFQqFQYPHowzZ87IuuHqO9rpyvj4nqG4QbzkECc0h9zXHu109ZybmxtiY2PRvn17qFQq\n3Lx5U+iSCP6IE/z9/XHp0iUkJCToTcPVd9R09QTFDeJCcYL+onhBxsdXF4obhKOvccJfyX3tUbxA\naqC4QRgUJxCAmq7eorhBtyhOIM9QvCDj42ssihv4Q3FC7eS+9iheIPWiuIEfFCeQ2lDTJQAobuAa\nxQmkLhQvyPj4movihuajOKFx5L72KF4gTUJxQ/NQnEAag5ouqRXFDU1DcQJpLIoXZHx8XKG4oW4U\nJzSP3NcexQukRShuqB3FCaQ5qOmSRqG4oSaKE0hzUbwg4+Pjiz7HDRQncEPua4/iBcIpfY0bKE4g\nXKCmS5pF3+IGihMIVyhekPHx6Yqc4waKE/gh97VH8QLhlVzjBooTCB+o6RJOyC1uoDiB8IXiBRkf\nn1CkHDdQnKAbcl97FC8QnZJq3EBxAtEFarqEF1KLGyhOILpC8YKMj08sxBw3UJwgDLmvPYoXiKDE\nGjdQnECEQE2X6ITY4gaKE4hQKF6Q8fGJlZBxA8UJ4iD3tUfxAhEVoeIGihOIGFDTJYL4a9wQEhLC\n686H4gQiFhQvyPj4pILPuIHiBHGS+9qjeIGIGl9xA8UJRIyo6RJR4DpuoDiBiBXFCzI+PqlqSdxA\ncYI0yH3tUbxAJKW5cQPFCUQKqOkSUWpq3EBxApEKihdkfHxyUV/cQHGCNMl97VG8QCStrriB4gQi\nRbTTlfHxydH+/fuxbNkyTJ06FQcPHsT8+fOxevVqGBoaCl0aaQK5r736drrUdGV8fHKk1WqxdOlS\n7Ny5EwMHDsTx48dFdalI0jhyX3sULxBZeBYnJCcn49dff0W3bt1EdalIQhqDmi6RhJiYmBpnJ7i6\nuur02g2EcIXiBRkfnxw8OzshNDQU+/btq/XFMjHfmYLUTu5rj+IFIkl/PjshMTGxzrMTxHpnCkJq\nQ02XiNJf44SG3uyg60tFEtJcFC/I+PikqDFxQkMobhA/ua89iheIJDQ2TmgIxQ1EzKjpElFoapzQ\nEIobiFhRvCDj45MCLuKEhlDcID5yX3sULxBR4ipOaAjFDURMqOkSQXAdJzSE4gYiFhQvyPj4xEgX\ncUJDnsUNHh4eCAkJgbm5uc5r0HdyX3sULxBR0FWc0JBncYO5uTl8fHxw48YNQeog+omaLtEJXccJ\nDflz3ODv709xA9EZihdkfHxiIIY4oSEUN+ie3NcexQtEEGKJExpCcQPRJWq6hBdiixMaQnED0RWK\nF2R8fEKQQpzQEIob+Cf3tUfxAtEJqcQJDaG4gfCJmi7hhNTihIZQ3ED4QvGCjI9PF+QQJzSE4gbu\nyX3tUbxAeCGXOKEhFDcQLlHTJc0itzihIRQ3EK5QvCDj4+ODPsQJDaG4oeXkvvYoXiCc0Jc4oSEU\nN5CWoKZLGkXf4oSGUNxAmoviBRkfHxcoTmgYxQ1NJ/e1R/ECaRaKExqH4gbSFNR0Sa0oTmgaihtI\nY1G8IOPjaw6KE1qO4oaGyX3tUbxAGoXiBG5Q3EDqQ02XAKA4gWsUN5C6ULwg4+NrDIoT+Edxw/Pk\nvvYoXiC1ojhBNyhuIH9GTVdPUZygWxQ3kGcoXpDx8dWG4gThUdwg/7VH8QIBQHGCWFDcoN+o6eoJ\nihPEheIG/UXxgoyPD6A4QQr0MW6Q+9qjeEFPUZwgDRQ36BcjoQvQhaqqKuTm5kKtVkOtVuPp06eo\nrKwEABw6dAjt27eHQqGAQqGApaUlWrWS/u+imJgYzJgxA/Pnz8fq1athaGgodEmkHs/ihv3798Pf\n3x/r16/HvHnzYGBQ62ZJMhhjKCgoqF57eXl51WsvPDwcbdu2hZ2dHRQKBWxsbGBkJP+WJLsjLCws\nRHx8PBISEqo/MjIy0K5du+rG2qFDh+pv7oEDB2r8UBQVFcHJyQkqlar6w8fHRzJ/8lGcIG2vv/46\nVCoVJk2ahB9//FFScUN5eTmuX79eY+2lpKTAyMgICoUCSqUSnTp1grGxMQDg4MGDKC4url57T548\nga2tLby8vODj4wOVSoW+ffvC1tZW4CPjliwy3fv37yMyMhIRERGIi4tD7969q5ulSqVC586dYWJi\n0qixysrKcP/+fSQkJFQ375s3b8LX1xeBgYEICgqCq6srz0fUPGq1GlOnToWBgQG+/fZberFMwkpL\nS7F8+XL88MMPOHjwILy8vIQuqVY5OTk4ceIEIiIiEBMTg86dO8PHx6d67bm7u6Ndu3aNGkuj0eDB\ngwdITEysXn/Xrl1Dly5dEBQUhKCgIPTu3VsSu//6Ml0wxur8+ONhcSouLma7d+9mPj4+zMbGhs2Z\nM4cdPXqUFRUVcT5XUVERO3r0KJszZw6zsbFhPj4+bPfu3ay4uJjzuZorOjqaKRQKtnbtWqbRaIQu\nh3Dk22+/ZVZWVmzHjh2sqqpK6HIYY4xpNBoWERHBRo4cyczNzdkrr7zC9uzZwx49esT5XBUVFeyH\nH35gf//731mXLl2Yq6sr++STT3iZi0v/652199W6HmAibbppaWns73//O7O0tGSBgYHs5MmTOm0y\nGo2GnTx5kgUGBjJLS0v297//naWlpels/trqWbNmDVMoFCwmJkawOgh/kpOTWa9evdjkyZNZQUGB\nYHXk5eWxjz76iDk5OTFfX1/29ddfs5KSEp3NX1VVxa5du8bmzJnDOnbsyKZNm8bi4+N1Nn9TyKLp\nPnr0iC1btoxZWFiwlStXstTUVKFLYqmpqWzlypXMwsKCLVu2TOe/fbOystjQoUPZsGHDmFqt1unc\nRLdKSkrYvHnzWLdu3dj169d1PvfGjRuZtbU1mzFjhigaXW5uLvvss8+YUqlkEyZMYHfv3hW6pBok\n3XRLS0vZhx9+yCwtLdmbb77JsrOzhS7pOdnZ2ezNN99klpaW7MMPP2SlpaW8z0lxgn7SZdxQVVXF\nvvrqK+bg4MDGjx/Pbt++zet8zVFcXMw+/vhjZmVlxebNmyea2EGyTfenn35iHh4ebNy4cezevXuC\n1tIY9+7dY+PGjWMeHh4sLi6OlzkoTiC6iBvS09OZv78/U6lU7OrVq7zMwaXc3Fy2fPlyZmdnxw4d\nOiR0OdJrumVlZWzVqlXMxsaGhYWFieYFhMaoqqpi+/fvZzY2NmzVqlWsrKyMs7EpTiDP8BU3VFVV\nsZ07dzIrKyu2YcMGVllZydnYunD58mXWvXt3NmnSJEF3vZJqupmZmaxfv35s7NixoowSGis7O5uN\nHTuW9evXj2VmZrZ4PIoTSG24jBuKi4vZ5MmTmbe3N7t16xZHFepeSUkJ+7//+z/m6OgoWP4smab7\n008/MXt7e7ZhwwZJ7W7rUlVVxdavX88cHByaHTdQnEAawkXc8ODBA6ZSqdjUqVN1ekYCnw4dOsSs\nrKxYWFiYzueWRNM9cOAAs7a2ZsePH9fZnLpy7NgxZm1tzQ4cONCk51GcQBqrJXFDXFwcUyqV7NNP\nP5XFZufPbty4wVxcXNjq1at1emyib7q7du1i9vb2LCkpSSfzCSEpKYkplUq2e/fuRn09xQmkOZoa\nN1y8eFG2m51nHj16xFQqFVuyZAnTarU6mVPUTXfXrl3M0dGR/frrr7zPJbS7d+8yR0dHtmvXrjq/\nhuIE0lKNjRsuX77MrK2t2enTp3VYnTDy8vLYiy++yBYvXqyTHa9om+6BAweYvb29XjTcZ+7evcuU\nSmWtUQPFCYQrDcUNiYmJetNwn8nPz2c+Pj7sH//4B+9zibLpxsXFMWtra1lHCnVJSkpi1tbWNV5c\noziB8KG2uCE7O5s5OTmx8PBwgavTvdzcXNa1a1f21Vdf8TqP6JpuZmYmc3BwkHWO1JBjx44xBwcH\n9uDBA4oTCK/+HDc8evSI9e/fn61du1bosgRz+/ZtZm1tza5cucLbHKJqumVlZaxfv35sw4YNnI8t\nNe+//z5r3749GzJkCMUJhFfP4gZzc3P28ssv6+wFJbGKiopiSqWSPXz4kJfx62u6Or9Fwtq1a6FQ\nKPDOO+/oemrRWbt2Lbp37w5fX1+69i3hlampKfz9/dGuXTvs2bNHFndHaYmAgADMnz8fs2fPfrbB\n1BmdXsQ8Li4OgYGBSEpKkt3V4JsrOzsbvXv3RlRUFPr27St0OUSmcnJy0KtXLxw5cgR+fn5ClyMK\nGo0GL774IhYsWIC5c+dyOnZ9FzHXWdMtKytDnz59sGbNGkyePJmTMeUiLCwM69evR0JCQqPvcEFI\nU0yaNAnOzs7YtGmT0KWIyq1btzBs2DAkJCTAycmJs3FFcTfgTZs2wc3NDa+99pquppSMyZMno3v3\n7rQgCC9OnDiBmzdv4sMPPxS6FNHx9PTEsmXLsGTJEp3NqZOdbk5ODnr06IG4uDh07ty5xePJ0b17\n9+Dr64s7d+7A2tpa6HKITGi1Wnh7e2PdunUYO3as0OWIUllZGdzc3LB//34MGDCAkzEF3+lu2LAB\nU6ZMoYZbjy5dumDy5Mn46KOPhC6FyMj+/fvRrl07BAUFCV2KaJmYmOCf//wnVq1apZMX1Xjf6aal\npUGlUuH27dv04lkDsrOz8cILLyAxMRHOzs5Cl0Mkrry8HO7u7vj6668xaNAgocsRNa1Wi169emHj\nxo0ICAho8XiC7nT/9a9/Ye7cudRwG8HOzg7BwcH417/+JXQpRAYOHTqELl26UMNtBENDQ3zwwQfY\nuHEj73PxutMtKSmBk5MT4uPj4eLi0uxx9Elqair69u2LjIwMtG3bVuhyiIQNGDAAK1aswLhx44Qu\nRRIqKyvh4uKC06dPo2fPni0aS7Cd7oEDB9C/f39quE3g6uoKPz8/fPfdd0KXQiTsxo0byMjIwJgx\nY4QuRTKMjY3xxhtvYPv27bzOw2vT3b59OxYtWsTnFLK0aNEibNu2TegyiITt2LED8+fPh5GRkdCl\nSMobb7yBsLAwFBUV8TYHb033/v37yMjIgL+/P19TyNZLL72EjIwMpKamCl0KkSCtVovDhw9j2rRp\nQpciOfb29ujbty9Onz7N2xy8Nd3IyEiMGTMGhoaGfE0hW4aGhggICEBkZKTQpRAJio2NhVKppFiv\nmYKCghAREcHb+Lw23cDAQL6Gl73AwEBquqRZIiIi6LzcFhgzZgxOnDgBrVbLy/i8NN3CwkL89NNP\nFC20gL+/P3766ScUFhYKXQqRGNrwtIyzszPs7e0RGxvLy/i8NN34+Hj07t0bZmZmfAyvF9q1awdP\nT08kJCQIXQqRkPz8fDx48AAqlUroUiRt0KBBuHr1Ki9j89J0ExIS6JvOAZVKRU2XNEliYiJ69+5N\nr6W0EJ9rj7em6+Pjw8fQesXHxwfx8fFCl0EkhDY83OBz7dFOV8RUKhUSExOFLoNICK09bri7u0Ot\nVq3dIhcAAA81SURBVKOgoIDzsTlvulVVVcjIyECXLl24HlrvdOnSBenp6Tq/nQiRrrS0NHTt2lXo\nMiTP0NAQLi4uyMjI4Hxszptubm4u2rVrhzZt2nA9NFxdXXHu3DnOx/2rf/7zn5gxY0a9X/Of//wH\nffv2hYmJCebMmcNLHSYmJjAzM0Nubi4v4xP5ycrKgkKh4Hxcsay9iooKzJ07Fy4uLujQoQP69OmD\nU6dO8VKLQqFAVlYW5+Ny3nTVajUv33Sxsbe3x/vvv4/g4GBe51EoFFCr1bzOQeSBMYbff/9d1utP\no9HAyckJFy9eREFBAdatW4dJkybxsiPla+1Jtul+/fXXGDhwIFasWAELCwt06dKlxm+8oUOH4t13\n34Wvry86dOiA8ePHIz8/HwBw/vx5ODo61hjv2W/y06dP46OPPsJ3332H9u3bw9vbu9b5x40bh6Cg\nIFhYWPB3kACUSiUvv22J/OTm5sLMzIz3++wJufbatm2LNWvWVI8REBAAV1dXXs404Gvtcd50nz59\nig4dOnA9bK3i4uLQo0cP5ObmYsWKFc/tOvft24c9e/YgOzsbhoaGNe6DZGBQ61XX8PLLL+Pdd9/F\na6+9hqdPn+L69eu8HkNDzM3Neb34BpEPfVx7v//+O1JSUvDCCy+07IBqwdfa47zpVlZW6uzKRs7O\nzpgzZw4MDAwwc+ZMqNVqPHr0qPrx6dOno0ePHjA1NcW6desQHh4uuReljIyMUFlZKXQZRAI0Go3O\nzs8Vw9rTaDSYNm0aZs2ahe7du3M6NsDf2tPZ3YD5YGdnV/3fpqamAFDjN9Of/4xxdnZGZWUlHj9+\nrLsCCZEpodceYwzTpk1DmzZtsHXrVs7G1QXOt6TGxsbQaDRcD9ssDx48qP7v9PR0GBsbw8rKCmZm\nZigpKal+TKvVIicnp/rfdf35IwSNRgNjY2OhyyASYGRkxNtFWpqK77UXHByMx48f48SJE7zt7vla\ne5zvdNu3b8/LCcXN8c033yA5ORklJSX44IMPMHHiRBgYGKB79+4oKyvDyZMnodFosH79elRUVFQ/\nz9bWFmlpafX+OaTValFWVgatVguNRoPy8nJefuALCwvRrl07zscl8qMva2/BggVITk5GREQEWrdu\nzdsx8LX2OG+6fJ7i1NBvwb8+Pn36dMycORNKpRIVFRXYsmULgD8C8m3btiE4OBgODg5o3749HBwc\nqp83ceJEMMZgaWlZ59uZ169fj7Zt2+LTTz/Ft99+i7Zt22LDhg0tPMLnZWVlQalUcj4ukR9LS0sU\nFxejrKyM87HFsvYyMjKwc+dO3LhxA7a2tmjfvj3Mzc0RFhbGwVHWxNvaY4zV+fHHw03z6NEjZmFh\n0eTncW3IkCFs9+7dQpfRYp06dWI5OTlCl0EkwtHRkd2/f1/QGuSy9kaMGMFOnTrVrOf+r3fW2lc5\n3+laWlqiqKgI5eXlXA+td8rKylBcXAxLS0uhSyESoVQq6c00HFGr1bzsdDlvuq1atYKTkxPu3bvH\n9dBNIqYXw5rr3r17cHZ2lsWxEN1wcXHBb7/9JmgNcvh51Wq1SEtLg5OTE+dj83JC7bNrUXp4ePAx\nfKPo4n3ifEtISECfPn2ELoNIyLO119C1Q/gkh7WXnJwMhULBy5tNeDlPV6VS0XVgORAfH0/XJSZN\nQhe+5wafa4+3pkvf+Jaja6OSpurTpw9u3rwpmvN1pYrPtcdL0/Xx8cHNmzdRXFzMx/B6oaioCLdu\n3aKmS5qkY8eOcHR0pE1PC124cAF+fn68jM1L0zU3N4evry+io6P5GF4vREdHw9fXF+bm5kKXQiQm\nMDAQkZGRQpchWenp6cjMzMSLL77Iy/i8XXuBvvEtQ7fRJs0VFBSEiIgIocuQrKioKIwePZq3txfz\n2nSjoqIoW2oGrVaLqKgoarqkWV588UWo1WqkpaUJXYokRUREICgoiLfxeWu6nTt3hpOTE0UMzXDm\nzBk4OzvD1dVV6FKIBBkaGuKVV17BN998I3QpkpOZmYlr167h5Zdf5m0OXi/tuHDhQmzbto3PKWRp\n27ZtWLRokdBlEAlbsGABQkJCRHPFP6kIDQ3FlClTeL3IlAGr52o+BgYGrL7HG1JSUgInJyfEx8fD\nxcWl2ePok9TUVPTt2xcZGRlo27at0OUQCRswYABWrFiBcePGCV2KJFRWVsLFxQWnT59Gz549WzSW\ngYEBGGO1vjWP151u27ZtMWPGDGzfvp3PaWRl+/btmDlzJjVc0mKLFi3Cv//9b6HLkIyjR4+ia9eu\nLW64DeF1pwv8cfpFnz59cPv2bdja2rZoLLnLzs7GCy+8gMTERDg7OwtdDpG4iooKuLm54euvv8ag\nQYOELkfUtFotevXqhY0bNyIgIKDF4wm20wX+uFXH9OnTsX79er6nkrz169djxowZ1HAJJ1q3bo0P\nP/wQq1atkty9AXVt3759sLS0xOjRo3mfi/edLgDk5OSgR48eiIuLQ+fOnVs8nhzdu3cPvr6+SE5O\nhpWVldDlEJnQarXw9vbGunXrMHbsWKHLEaWysjK4ubkhLCwM/fv352RMQXe6AGBtbY1ly5bhrbfe\not+4tWCM4e2338ayZcuo4RJOGRoa4pNPPsGKFStQWloqdDmi9Nlnn8HLy4uzhtsQnex0gT9+m6hU\nKrz//vuYPHkyJ2PKRVhYGNavX4/ExES0adNG6HKIDE2aNAnOzs7YtGmT0KWIyq1btzBs2DAkJCRw\neu3c+na6Omu6ABAXF4fAwEAkJSXRi2r/k52djd69eyMqKgp9+/YVuhwiUzk5OejVqxeOHDnC24Vc\npKayshIvvvgiFi5ciLlz53I6tuDxwjP9+vXDnDlzMH/+fIoZ8EessGDBAgQHB1PDJbyytrbG1q1b\nMXv2bBQVFQldjih8/PHHsLKyQnBwsE7n1elOFwDKy8sxaNAgjB07Fu+++y6nY0vNhg0bEBkZifPn\nz1OsQHQiODgYBQUFOHjwIFq10umeS1S+//57zJs3D3FxcbC3t+d8/Pp2upzfDbgxMjMzmYODAzt2\n7Bgv40vBsWPHmIODA8vKyhK6FKJHysrKWP/+/dnatWuFLkUwt2/fZtbW1uzKlSu8zYF67gYsSNNl\njLG4uDhmbW3NkpKSeJtDrJKSkpi1tTWLi4sTuhSih7Kzs5mTkxMLDw8XuhSde/z4MevatSvbs2cP\nr/OIsukyxtiBAweYUqlkv/76K6/ziMndu3eZUqlkBw4cELoUoscSExOZtbU1O3XqlNCl6Ex+fj5T\nqVTsH//4B+9zibbpMsbYrl27mKOjI7t79y7vcwnt7t27zMHBge3evVvoUghhly9fZtbW1uz06dNC\nl8K7vLw89uKLL7LFixezqqoq3ucTddNl7I/Gq1QqZR01JCUlMaVSSQ2XiMqlS5eYtbW1rF9fefTo\nEVOpVGzJkiU6abiMSaDpMvZH1CDXb/6xY8eYtbU1RQpElOLi4phSqWSffPKJzpqSrty4cYO5uLiw\n999/X6fHJommy9gf33wHBwe2fv16WXzzq6qq2Lp165iDgwO9aEZE7cGDB0ylUrHXX3+dlZSUCF0O\nJw4dOsSsrKwE2ezU13R1fp5uQ7KysjB+/HgoFAqEhIRI9p1r2dnZWLBgAbKzs3H06FEoFAqhSyKk\nXqWlpQgODkZycjL27t3L+3Vl+VJaWorVq1cjPDwcR48ehUql0nkNonlHWmMolUpcuHABHh4e6NWr\nF8LCwiT17jXGGMLCwtC7d294eHj8v/buHKSVNQwD8OuJAwZFEIkaC8UdI4pmIYWFiFjpBEXEUrQU\nxUbsUoqVWClYWlgoVhNRxEpwAcmMWxFwIQTRUdAIwZUkzq1OONtVL9czk+j71CF8w+R/5+Nn8v1Y\nX19n4FJKMJvNmJubw8DAAJqbmzE2NpZyx/1sbW2hvr4e5+fnUBTFkMB907+1wJoB2wu/2tnZ0Ww2\nm9bR0aGdnJwYWst7nJycaB0dHZrNZuN2AqW0UCiktba2ag6HQ9ve3ja6nDfd3Nxow8PDWkFBgba4\nuGh0Oa9uLyRdp/sjl8sFWZZht9vhdrsxODiIq6sro8v6zeXlJQYHB+F2u2G326EoCmcpUEorKirC\n6uoqhoaG0N3djc7OTgQCAaPL+s3DwwPGx8dRVVWFx8dHHBwcoKury+iyXpXUoQsAGRkZ8Hq9CAQC\nEAQBNpsNo6OjCAaDRpeGYDCI0dFR1NTUQBAEBAIBeL1ezlGgTyEtLQ29vb04OjpCY2Mjmpqa0Nvb\nC1mWjS4N4XAYExMTqKiowO7uLjY3NzEzMwOLxWJ0aW9K+tD9zmKxYHJyEoqiIB6Pw+VyQRRFrKys\nIB6P61ZHPB7HysoKRFGEy+VCPB6HoiiYnJxMiRtO9F+ZzWaMjIzg6OgI1dXV6OrqgtvtxuzsrK6D\n0TVNg9/vR39/P8rKyrC/vw9JkrCwsIDKykrd6vi/ku7thfd6eHjA/Pw8pqenEQqF0N7eDlEU0dra\n+uFn1t/d3WFtbQ0+nw9LS0soLi7GwMAAenp6eGovfTnfG4+pqSlsbW2hpaUFHo8HbW1tH954RKNR\nbGxsQJIkSJKUGIfa19eX1E1O0gwx/1uCwSB8Ph8kScLOzg5qa2vhcDjgdDrhcDhQVlaGjIyMd33X\n09MTTk9PIcsy/H4/ZFnG4eEh3G43RFGEKIooKSn5y1dElBqur6+xvLwMSZKwtraG0tLSn9ZedXX1\nu5ugaDSKs7MzKIqSWH9+vx/l5eXweDzweDyoq6tDWtqfJyYmk08fuj+KRCKQZTlx0xRFQSgUQmZm\nJqxWKwoLC5GdnY309HQAQCwWQyQSwcXFBVRVxf39PYqLi2G32xM/HIfDgezsbIOvjCi5PT8/Y29v\n76f1d3x8DJPJlFh7OTk5EAQB3759QywWw/39PVRVhaqquL29RX5+PhoaGhLB7XQ6U/Jd/S8Vun/y\n8vKCcDgMVVVxcXGBu7s7RKNRAIAgCMjKykJhYSGsVityc3NT4klKlAo0TUMkEkkEazgcRiwWw8vL\nCwRBgNlshtVqhdVqRV5eHkwmk9Elf4gvH7pERHpKqX+kERF9ZgxdIiIdMXSJiHTE0CUi0hFDl4hI\nRwxdIiIdMXSJiHTE0CUi0hFDl4hIRwxdIiIdMXSJiHTE0CUi0lH6Wx/gxC0ioo/z6pQxIiL6WNxe\nICLSEUOXiEhHDF0iIh0xdImIdMTQJSLS0T/nX/DFV68YZQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from visualise_neural_network import NeuralNetwork\n", "\n", "network = NeuralNetwork() #create neural network object\n", "network.add_layer(2,['Input 1','Input 2']) #input layer with names\n", "network.add_layer(2,['Hidden 1','Hidden 2']) #hidden layer with names\n", "network.add_layer(1,['Output']) #output layer with name\n", "network.draw()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that this network now has 5 total neurons. The two units at the bottom are the *input layer*. The activity of input units is the value of the inputs (same as the inputs in my previous post). The two units in the middle are the *hidden layer*. The activity of hidden units are calculated in the same manner as the output units from my previous post. The unit at the top is the *output layer*. The activity of this unit is found in the same manner as in my previous post, but the activity of the hidden units replaces the input units. \n", "\n", "Thus, when the neural network makes its guess, the only difference is we have to compute an extra layer's activity. \n", "\n", "The goal of this network is for the output unit to have an activity of 0 when presented with an item from the blue group (inputs are same) and to have an activity of 1 when presented with an item from the red group (inputs are different).\n", "\n", "One additional aspect of neural networks that I haven't discussed is each non-input unit can have a *bias*. You can think about bias as a propensity for the unit to become active or not to become active. For instance, a unit with a postitive bias is more likely to be active than a unit with no bias. \n", "\n", "I will implement bias as an extra line feeding into each unit. The weight of this line is the bias, and the bias line is always active, meaning this bias is always present. \n", "\n", "Below, I seed this 3-layer neural network with a random set of weights." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAHMCAYAAAB2sntRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlcVOXbP/APm0LCgLIMw6amBpIIOKhfJXfNFEEtNbfc\nyz3tKX0y/ZqVZmply5O7ZZoragbmikuuJEKKCyoqoAKKySabzHL9/ijnF8nOnDlnZq7368Ufw5k5\n93WU++LmM2fOsSAiMMYYMwxLsQtgjDFzwk2XMcYMiJsuY4wZEDddxhgzIG66jDFmQNaVbbSwsOBT\nGxhjrBaIyKK871fadP9+of6rYYwxE2ZhUW6/BcDxAmOMGRQ3XcYYMyBuuowxZkDcdBljzIC46TLG\nmAFx02WMMQPipssYYwbETZcxxgyImy5jjBkQN13GGDMgbrqMMWZAVV57gTGpePLkCR4+fIjS0lKo\n1WpYWVnBxsYGzs7OaNCggdjlMVYt3HSZpBARbt++jfPnzyM+Ph6JiYlIT09HZmYm8vPz4eLiAltb\nW1hZWUGr1aK0tBQPHz5E/fr1oVAo4OHhAX9/fyiVSiiVSvj7+8Pamn/MmXRYVHYVMQsLC+KrjDGh\n5eTkYP/+/YiKisKhQ4fQoEEDXdMMCgqCt7c3PDw84OLiAkvLZxMxIkJubi4yMjKQnp6Oy5cvIz4+\nHufPn8e9e/fQqVMnREREIDw8HN7e3iIcITM3FhYWFV7akZsuE8WTJ0+wa9curF+/HnFxcejWrRvC\nw8PRt29feHh46G2cvLw8HD58GFFRUdi3bx98fHwwevRojB49Gk5OTnobh7F/4qbLJOPOnTtYvXo1\n1q9fj1atWmHSpEno27cvnnvuOcHHVqvVOHHiBNauXYsDBw5g8ODBmDJlCoKCggQfm5mXypoun73A\nDOLBgweYPn06goODUVhYiOPHjyMmJgaDBg0ySMMFAGtra3Tv3h1bt25FUlISGjdujH79+uHVV19F\nUlKSQWpgjJsuE9Tjx48xf/58+Pv7w8rKCteuXcNXX30FPz8/Uetyd3fH3LlzkZycjA4dOqBz584Y\nP3487t27J2pdzPRx02WCiYmJQUBAAG7fvo34+Hh89dVXcHV1FbusMuzs7DBr1iwkJyfDzc0NwcHB\nWLt2Ld+migmGM12md/n5+Zg1axb279+PNWvW4JVXXhG7pGq7fPkyxowZA2dnZ6xduxY+Pj5il8SM\nEGe6zGAuXbqE4OBgaDQaXLp0yagaLgC0atUKZ8+eRZcuXRASEoK9e/eKXRIzMbzSZXqzZ88evPnm\nm/jqq68wYsQIscups9jYWAwaNAjTp0/H7NmzK73DK2P/xKeMMUERERYtWoTVq1dj9+7daNu2rdgl\n6U16ejoGDBiAF154AevXr4etra3YJTEjwPECEwwRYebMmfj5559x7tw5k2q4AODp6YkTJ05ApVIh\nIiICRUVFYpfEjBw3XVZrRIRp06bh3LlzOHLkCBQKhdglCcLOzg5bt26FQqFAWFgYN15WJxwvsFoh\nIrz33ns4deoUDh8+DJlMJnZJgtNoNBgzZgyysrIQFRWF+vXri10SkyiOF5jerVy5EgcOHMCBAwfM\nouECgJWVFX744QfY2dlhypQpfC4vqxVe6bIaO3bsGIYOHYozZ86gWbNmYpdjcAUFBejYsSMmTJiA\nt99+W+xymATx2QtMb1JSUtChQwds3rwZPXr0ELsc0aSmpqJDhw7YtGkTevbsKXY5TGK46TK90Gg0\nCA0NxdChQzFz5kyxyxHd8ePHMWzYMCQmJkru481MXNx0mV4sXboUBw8exOHDh8u9mLg5eu+993D3\n7l1s375d7FKYhHDTZXWWlJSETp06IS4uDk2bNhW7HMkoLi5GcHAwFi5ciEGDBoldDpMIbrqsTogI\noaGheOONNzB58mSxy5Gcs2fP6q7Jy3ejYACfMsbqaPfu3SgpKcHEiRPFLkWSOnTogD59+mDZsmVi\nl8KMAK90WaXUajVatWqFr7/+Gr179xa7HMm6c+cOgoODcfnyZZP9ZB6rPl7pslrbsGEDFAoFXn75\nZbFLkTQfHx+MGTMGn3zyidilMInjlS6rkEajQYsWLfDTTz+hY8eOYpcjeY8ePULz5s2RlJQEd3d3\nscthIuKVLquVgwcPwtnZmRtuNTk7O2PIkCFYt26d2KUwCeOVLqtQv3798Nprr2Hs2LFil2I0Lly4\ngPDwcKSkpMDa2lrscphIeKXLaiwlJQWxsbF4/fXXxS7FqAQFBcHHxwfR0dFil8IkipsuK9eOHTsw\nZMgQPPfcc2KXYnTGjRuHbdu2iV0GkyhuuqxcUVFRGDBggNhlGKV+/frh0KFDKC0tFbsUJkHcdNkz\nsrKycOXKFXTp0kXsUoySXC6Hn58ffvvtN7FLYRLETZc949dff0WvXr34zgh1EBERgV9++UXsMpgE\ncdNlzzh58qRZXytXH3r27ImTJ0+KXQaTIG667Bnx8fEICQkRuwyjFhAQgOTkZBQXF4tdCpMYbrqs\njOLiYiQnJyMgIEDsUoyara0tfH19kZiYKHYpTGK46bIyEhMT4efnx3muHiiVSpw/f17sMpjEcNNl\nZSQnJ8PPz0/sMkxCy5YtcevWLbHLYBLDTZeVkZGRAQ8PD1FrWL58ORQKBZycnDBhwgSoVKpyn/fo\n0SO89NJLcHFxQcOGDREaGoozZ87otk+ePBkODg6QyWSQyWSwtbWFo6OjoQ4DCoUCGRkZBhuPGQdu\nuqyMzMxMUa8He/DgQSxduhTHjh1DWloabt26hQ8//LDc59rb22P9+vXIyspCTk4OZs+ejfDwcGi1\nWgDAypUr8fjxY+Tn5yM/Px/Dhg3D4MGDDXYsCoUCmZmZBhuPGQduuqwMsVe6GzduxPjx4+Hn5wdH\nR0fMnz8fP/zwQ7nPrV+/Pnx9fWFpaQkigqWlJXJzc5Gdnf3McwsLC7Fr1y6MGTNG4CP4/zw8PHil\ny57BTZeVkZubi4YNG4o2/pUrVxAYGKh7HBgYqFvJViQwMBC2trYYMGAA3nzzTbi4uDzznF27dsHN\nzQ0vvfSSIHWXp2HDhpXWzcwTX3uOlaFSqWBjYyPa+AUFBWVyV5lMBiLC48ePK/xlcPHiRZSWluLn\nn3+u8HoHGzduxKhRowSpuSI2NjZQq9UGHZNJH690WRlEBAuLci8DKogtW7bo3uwKCwuDg4MD8vPz\nddvz8vJgYWEBBweHSvdTr149vP7661i8eDEuXbpUZtudO3dw/PhxgzddS0tLXb7M2FPcdFkZ1tbW\n0Gg0Bhtv+PDhuje7fv31V/j7++PixYu67RcuXIBcLq925KFSqXD79u0y3/vpp5/w0ksvoUmTJvos\nvUpqtVrUvxqYNHHTZWXY2dmhsLBQtPFHjRqF9evXIykpCTk5OVi4cGGFd674/fffcfr0aahUKpSU\nlGDJkiXIyspC+/btyzxv48aNotz9orCwELa2tgYfl0kbN11Whru7u6inOfXu3RuzZ89Gt27d0LRp\nUzRr1gwLFizQbe/bty8+++wzAMCTJ08wdepUuLi4wMvLCwcOHMC+ffvK3BQyNjYW6enpGDRokKEP\nRfTT75g08T3SWBkLFiyAVqvFxx9/LHYpRm/Pnj34/vvvERUVJXYpzMD4Hmms2viEfv3hlS4rDzdd\nVoaPjw9SU1PFLsMkpKamwtvbW+wymMRw02VlBAUFISEhARwr1V1CQgKCgoLELoNJDDddVoZCoYCt\nrS3S0tLELsWoERHi4+OhVCrFLoVJDDdd9gylUon4+HixyzBqqampsLOz40yXPYObLntGu3btylwi\nkdXcmTNn0K5dO7HLYBLETZc9o2/fvti7d6/YZRi1vXv3IiwsTOwymATxebrsGUQEb29vHDlyBL6+\nvmKXY3RKS0shl8uRlJRU5oMazHzwebqsRiwsLBAREcEn9dfSyZMn4evryw2XlYubLivXwIEDsXXr\nVj51rBa2bduGgQMHil0GkyiOF1i5tFotWrRogS1btjxzARlWsdzcXDRt2hTXrl2DXC4XuxwmEo4X\nWI1ZWlpi8uTJWLFihdilGJUff/wRffr04YbLKsQrXVahR48eoXnz5khOTi73FjisLK1WC39/f6xb\nt86gtwVi0sMrXVYrzs7OGDJkCD7//HOxSzEKO3fuhIODA0JDQ8UuhUkYr3RZpdLT09G6dWskJibC\n09NT7HIkS6VSwd/fHytXrkTPnj3FLoeJjFe6rNY8PT0xYcIEfPTRR2KXImnr169HkyZNuOGyKvFK\nl1UpOzsbvr6+OHr0KAICAsQuR3Ly8vLQsmVLREVFISQkROxymARUttLlpsuqZe3atVi9ejViY2Nh\nbW0tdjmSMn78eNjY2GDVqlVil8IkguMFVmcTJkxAo0aNsHTpUrFLkZT9+/fjyJEjWLZsmdilMCPB\nK11WbXfu3IFSqeSY4W+5ubkICAjAhg0b0KNHD7HLYRLCK12mFz4+Pvjiiy/w2muvIScnR+xyRKXR\naDB8+HC8+uqr3HBZjfBKl9XYzJkzcfXqVezbt89s893Zs2fj/PnzOHjwIGxsbMQuh0kMr3SZXn3+\n+ecgIsyaNUvsUkSxadMm7Nq1C5GRkdxwWY2Z5zKF1Ym1tTW2b9+Ojh07wtPTE++9957YJRnM/v37\n8e677+Lo0aNwdnYWuxxmhLjpslpp1KgRYmJi0KVLF9jY2GDGjBlilyS4w4cPY/To0fjll1/QqlUr\nscthRoqbLqs1Ly8vHD16FD169MCTJ08we/ZssUsSzN69ezFu3Djs3r0bHTp0ELscZsT4jTRWZ+np\n6Xj55ZfRuXNnfP3116hXr57YJekNEeHbb7/Fp59+il9++YWvLcyqhT+RxgSXn5+PESNGID8/Hzt3\n7oSrq6vYJdXZkydPMHXqVJw7dw6//PILmjZtKnZJzEjw2QtMcDKZDHv27EFoaKhJ3MI9JSUF3bt3\nR3Z2Ns6cOcMNl+kNN12mN1ZWVvj000/xxRdfYODAgejWrRsKCwvFLqtGtFotxowZgzZt2mDAgAHY\nuXMn7O3txS6LmRCOF5ggpk2bhl9++QUNGjTA2rVr0alTJ7FLqlJycjImTpyImzdvwsvLC6dPn4aF\nRbl/ITJWKY4XmEHFxsZi586diI+Px6JFizBy5EhERETg8uXLYpdWrszMTEyaNAkdOnRAWFgYkpOT\noVarsWbNGrFLYyaImy7Tq5KSEowdOxbffvst3Nzc8Nprr+H69evo3r07evTogdGjR+PKlStilwkA\nyMjIwJw5c9CqVSs4ODjg+vXrePfdd1G/fn1s2LAB8+bNQ1pamthlMhPDTZfp1fz589GqVSsMHjxY\n9z1bW1vMnDkTycnJaNasGXr27Ilu3bohMjISKpXKoPUREY4dO4bBgwfjxRdfRF5eHi5cuIBly5aV\n+YSZv78/3n33XUyYMAEcsTF94kyX6U1sbCwGDBiAxMREuLm5Vfi80tJS7NmzBytWrMC1a9cQERGB\niIgI9OjRA3Z2dnqvS61W4+zZs4iOjsaePXtgY2ODqVOnYuTIkZDJZJW+rmPHjhg/fjwmTpyo97qY\n6eLzdJngSkpKEBwcjI8//rjMKrcqN2/eRFRUFKKiopCQkIDOnTujffv2UCqVUCqVkMvlNa4lPz8f\nCQkJiI+Px/nz53H48GF4e3sjPDwcERERUCqV1X6D7OrVq+jSpQvOnz+Pxo0b17gWZp646TLBzZ49\nGykpKYiMjKz1PrKzs3HkyBGcP38e8fHxSEhIQP369eHt7Q2FQgEPDw/I5XLUr18f1tbW0Gg0UKlU\nePjwITIyMpCZmYmMjAw8fPgQrVu31jXu7t27w8fHp9Z1ffbZZzhy5AgOHTrEZzOwauGmywRV3Vih\npogI9+7dQ3p6OjIzM5GZmYn79++jtLQUarUaVlZWsLGxgaurKxQKhe6rSZMmer3OL8cMrKa46TLB\n1DZWMDYcM7Ca4KbLBKOPWMFYcMzAqoubLhOEULGCVHHMwKqLmy7TO3OJFf6NYwZWHdx0md6ZU6zw\nbxwzsKpw02V6ZW6xwr9xzMCqwk2X6Y25xgr/xjEDqww3XaY35hwr/BvHDKwi3HSZXph7rPBvHDOw\ninDTZXXGsUL5OGZg5eGmy+qMY4WKcczA/o2bLqsTjhUqxzED+zduuqzWOFaoHo4Z2D9x02W1xrFC\n9XHMwJ7ipstqhWOFmuGYgT3FTZfVGMcKtcMxAwO46bJa4Fih9jhmYNx0WY1wrFA3HDMwbrqs2jhW\n0A+OGcwbN11WbRwr6A/HDOaLmy6rFo4V9ItjBvPFTZdViWMFYXDMYJ646bIqcawgHI4ZzA83XVYp\njhWExTGD+eGmyyrEsYJhcMxgXrjpsgpxrGA4HDOYD266rFwcKxgWxwzmg5suewbHCuLgmME8cNNl\nz+BYQTwcM5g+brqsDI4VxMUxg+njpst0OFaQBo4ZTBs3XabDsYJ0cMxgurjpMgAcK0gNxwymq7Km\na2noYhiwfPlyKBQKODk5YcKECVCpVOU+Lzk5GQMGDICbmxtcXFzQp08f3LhxQ7d9+/bt8PPzg6Oj\nI9zd3TF27FgUFBSUu6+SkhKMHTsW3377LTdcibC2tsaGDRswb948pKWlVes11f3ZAYDo6GgEBARA\nJpPhpZdeQlJSUpnt8+bNg5eXFxo2bIju3bvj6tWrdToeVk1EVOHXX5uZPh04cIDc3d0pKSmJcnNz\nqWvXrjRnzpxyn3vu3Dn6/vvvKScnh9RqNf33v/8lPz8/3fa7d+/SgwcPiIiosLCQRowYQTNmzCh3\nX7NmzaJBgwbp/4BYnS1evJh69uxJWq220ufV5GcnOTmZZDIZnTlzhjQaDS1evJiaN29OGo2GiIi2\nb99Onp6elJqaSlqtlubMmUNt2rTR+7GZq797Z/l9taINxE1XEMOHD6e5c+fqHh89epTc3d2r9drs\n7GyysLCg7OzsZ7Y9fvyYRo0aRWFhYc9sO3v2LMnlcl2DZtKiUqmobdu2tGrVqkqfV5Ofnf/7v/8r\n87Og1WrJzs6Ojh49SkRES5Ysoddff123/cqVK2RnZ1eXw2D/UFnT5XjBwK5cuYLAwEDd48DAQGRl\nZSEnJ6fK1/72229QKBRo2LCh7nunT5+Gk5MTZDIZdu/ejXfeeafMazhWkL7qxgx1+dnRarUgIly+\nfBkAMHToUNy6dQvJyclQqVTYsGED+vTpU/eDYVXipmtgBQUFcHR01D2WyWQgIjx+/LjS1927dw/T\npk3D8uXLy3w/NDQUubm5SE9Px6xZs+Dj41Nm+/z589GqVSs+PUzi/P398e6772LChAlP/8p8Rk1+\ndnr27InffvsNJ06cgEqlwqeffgqVSoWioiIAgEKhQGhoKHx9fdGgQQPs2rULX375pTAHx8rgpiuw\nLVu2wMHBATKZDGFhYXBwcEB+fr5ue15eHiwsLODg4FDhPh4+fIjevXtj2rRpGDJkSLnPUSgU6N27\nN4YOHar7XmxsLDZu3IjvvvtOfwfEBPPee+8hLy8Pa9asAVC3nx1fX1/8+OOPmDp1Kjw8PJCdnQ1/\nf394eXkBAD766CPExcUhPT0dJSUlmD9/Prp164aSkhLDHKw5qyh3IM50BTF8+HCaN2+e7nFMTAwp\nFIoKn5+Tk0PBwcH0wQcfVLnvkydPkpOTExERFRcXk5+fH+3YsaPuRTODuXLlCrm4uFBqauoz22r6\ns/NPubm5ZG9vTzdu3CAion79+tE333xT5jlOTk4UHx9fh+rZU+A30qTjwIEDpFAo6OrVq5SdnU1d\nu3atsKHm5+dT27Ztafr06eVu37x5M925c4eIiFJTU6lLly66MxT4bAXjVdHZDDX52SEiio+PJ41G\nQ1lZWTRkyBAaOXKkbttHH31EnTp1ogcPHpBWq6WNGzeSvb095eXlCXZc5oSbrsQsX76c5HI5OTo6\n0vjx46m0tFS3rU+fPrR48WIiIvrxxx/J0tKS7O3tdV8ODg509+5dIiKaO3cueXl5kb29PXl7e9Ok\nSZMoOzubz1YwcpWdzVDdnx0iopdeeokcHBzI2dmZJk+eTEVFRbptJSUlNG3aNFIoFOTo6EhKpZIO\nHTok7IGZkcqaLn8izcTwtRVMA1+bwbjxx4DNCF9bwXTwtRmMFzddM8HXVjAtfG0G48VN1wxwrGCa\nOGYwTtx0zQDHCqaLYwbjw03XxHGsYNo4ZjA+3HRNGMcK5oFjBuPCTdeEcaxgPjhmMB7cdE0Uxwrm\nhWMG48FN1wRxrGCeOGYwDtx0TRDHCuaLYwbp46ZrYjhWMG8cM0gfN10TwrECAzhmkDpuuiaEYwX2\nFMcM0sVN10RwrMD+iWMG6eKmawI4VmDl4ZhBmrjpmgCOFVhFOGaQHm66Ro5jBVYZjhmkh5uuEeNY\ngVUHxwzSwk3XiHGswKqLYwbp4KZrpDhWYDXBMYN0cNM1QhwrsNrgmEEauOkaIY4VWG1xzCA+brpG\nhmMFVhccM4iPm64R4ViB6QPHDOLipmtEOFZg+sIxg3i46RoJjhWYPnHMIB5uukaAYwUmBI4ZxMFN\n1whwrMCEwjGD4XHTlTiOFZiQOGYwPG66EsaxAjMEjhkMi5uuhHGswAyFYwbD4aYrURwrMEPimMFw\nuOlKEMcKTAwcMxgGN10J4liBiYVjBuFx05UYjhWYmDhmEB43XQnhWIFJAccMwuKmKyEcKzCp4JhB\nONx0JYJjBSYlHDMIh5uuBHCswKSIYwZhmG3Tzc3NxR9//IHU1FRkZGQgIyMDmZmZKCgogEqlAgDY\n2NjA3t4eCoUCHh4e8PDwQJMmTRAcHAwnJye91cKxApMqIWKG4uJiJCYmIjk5WTfvMjIykJubC5VK\nBSKCtbU17Ozs4O7uDg8PDygUCvj4+CAoKAgKhUIvdYjFbJrurVu3sHfvXpw9exbx8fG4f/8+AgMD\n0bx5cygUCt2Xo6MjrK2tAfz1J1ZeXh4yMzN1Xzdv3sTFixfh7u4OpVKJDh06oF+/fmjWrFmt6uJY\ngUmZPmKGrKws/Prrrzh58iTi4+ORnJwMPz8/+Pn56Rqqh4cHGjZsCBsbG1hYWECj0aCwsLDM3EtN\nTUVCQgJsbW2hVCrRrl079O3bF8HBwUaVO5t0001ISEBkZCSioqLw6NEj9OvXD126dIFSqYSvry+s\nrKxqtV+NRoPr168jPj4ev/32G/bu3QtnZ2dERERg8ODBaNOmTbX2w7ECMwa1iRlSUlKwY8cOREVF\n4cqVK+jVqxd69OiBkJAQBAQEoH79+rWqhYiQlpaG+Ph4nDlzBtHR0SguLkZ4eDgGDhyIHj16wNLS\nslb7NpTKmi6IqMKvvzZLT1FREW3YsIHatWtHjRs3pjlz5lBsbCxpNBrBxtRoNBQbG0tz5syhxo0b\nU7t27WjDhg1UVFRU6etmzZpFgwYNEqwuxvRl8eLF1LNnT9JqtRU+R61W06+//kphYWHk7OxMkydP\npoMHD1JJSYmgtV27do2WLl1KwcHB9Pzzz9OyZcvozz//FHTMuvi7d5bfVyvaQBJsuvn5+bRgwQJy\ncXGhPn36UHR0NKnVaoPXoVarKTo6mvr06UMuLi60YMECys/Pf+Z5Z8+eJblcTg8ePDB4jYzVlEql\norZt29KqVavK3bZ27Vpq2rQpKZVK+v7776mwsNDgNWq1WoqNjaVRo0aRk5MTvfXWW3Tv3j2D11EV\no2+6JSUl9PXXX5NcLqeRI0dScnKy2CXp3Lhxg0aMGEFyuZy+/vpr3W/84uJi8vPzox07dohcIWPV\nd+XKFXJxcaHU1FQi+qvJ7dy5k3x9falr1650+vRpkSv8/x4+fEizZ8+mRo0a0ezZsyk7O1vsknSM\nuukePnyYmjZtSn379qULFy6IXU6FLly4QH379qWmTZvS4cOHOVZgRutpzHDlyhXq0KEDBQUF0YED\nByqNHcR07949evPNN8nFxYXWrFkjiTqNsunm5+fTxIkTydvbm/bv3y9aHTW1f/9+cnNzIzs7O7p1\n65bY5TBWYyUlJeTl5UX29va0YsUKQd8r0afExERSKpX08ssvU1pamqi1VNZ0JfkW4KlTpxAQEAC1\nWo1Lly7hlVdeEbukanvllVfwxx9/oFevXujevTtOnToldkmMVVtKSgq6dOkCLy8vHD58GJMnT5b8\nmQJPBQQE4OzZs+jcuTOUSiU2btwodknlq6gbk0gr3TVr1pCbmxvt3bvX4GPr2969e8nNzY3WrFkj\ndimMVeno0aMkl8tp+fLlRrO6rUhiYiI1b96cZs6cSSqVyuDjwxjihdLSUpo6dSr5+vrS9evXDTau\n0K5fv06+vr40bdo0Ki0tFbscxp6h1Wrpu+++I7lcTjExMWKXozfZ2dnUq1cv6tWrFz169MigY1fW\ndCXx4YiioiK8+uqrsLS0xNatW+Ho6Cj4mIaUl5eHYcOGgYiwa9cuPPfcc2KXxBiAvxZd7733Hg4c\nOICoqKhaf+pSqtRqNWbNmoX9+/cjJiYGXl5eBhm3sg9HiB7WFBYWIiwsDK6uroiKijK5hgsAjo6O\niIqKgrOzM8LCwlBYWCh2SYyBiDBt2jScOnUKp06dMrmGCwDW1tZYvnw5xo8fjy5duiAtLU3sksRd\n6RYXF6Nfv37w8fHBunXrav2RXWOh0Wgwfvx43Lt3D9HR0bCzsxO7JGamiAgzZ87EuXPncPDgQchk\nMrFLEtzXX3+Nb775BidOnICnp6egY0ny2gtarRaDBg2Cra0tNm3aZPIN9ymNRoORI0eitLQUkZGR\nRvPOMDMtCxcuxM8//4wjR47o9Wp6UrdkyRJs3LgRZ8+eFfQXjSSb7ocffogjR47g6NGjqFevniBj\nSNWTJ0/QvXt39OrVCwsWLBC7HGZmfv75Z7z99ts4d+6c0V9CsTYmTZqEjIwM7NmzR7BFj+QueLNj\nxw7y8fGh+/fvC7J/Y3D//n3y8fGhyMhIsUthZiQxMZFcXFzo3LlzYpcimidPnlDnzp3pgw8+EGwM\nSOmUsatXr5KLiwslJCTofd/GJj4+nlxcXOjq1atil8LMQF5eHj3//PP0008/iV2K6LKysqhJkya0\ne/duQfZaoOU5AAAgAElEQVRfWdM1aLygVqsRGhqKMWPGYPLkyXrbrzFbsWIFNm7ciNOnT5tNrs3E\nMWnSJKjVaqxbt07sUiThzJkzeO2115CYmAhXV1e97lsyp4x9+eWXaNCgAd8E7x8mTZoEOzs7fPnl\nl2KXwkxYTEwM9u3bhy+++ELsUiSjY8eOGDFiBKZPn27QcQ220k1KSkKnTp0QFxeHpk2b6mWfpiIl\nJQVt27bFqVOn4OfnJ3Y5zMQ8fvwYAQEBWL16NXr37i12OZJSXFyMoKAgfPrpp3jttdf0tl9JnL0Q\nFhaGnj174p133tHL/kzN8uXLceTIEezdu1fsUpiJmT9/Pm7fvo2ffvpJ7FIk6cSJE3jjjTdw/fp1\n2Nra6mWfojfdEydOYPTo0bh27Vqt75tk6p48eQJfX19s2rQJnTp1ErscZiIePHgAf39/xMfHo0mT\nJmKXI1kRERHo3r07Zs6cqZf9idp0iQihoaGYPHky3njjjTrty9Rt3LgRq1evxqlTp4zqzqdMuqZP\nnw4rKyt89dVXYpciaZcuXULPnj2RnJyslw9NiPpG2r59+1BQUIDhw4cLPZTRGzFiBPLz87F//36x\nS2Em4M6dO9iyZQvmzp0rdimSFxAQgFdeecUgv5wEb7rffvstZs2axadDVYOVlRVmzZqFb7/9VuxS\nmAlYvXo13njjDb2fDmWqZs+ejVWrVkGlUgk6jqDxws2bN9GxY0fcuXNHbwG1qSspKYGPjw/Onj1r\nkld9Yobx5MkTNG7cGL/99ht8fX3FLsdodO3aFVOnTsXgwYPrtB/R4oVVq1Zh3Lhx3HBrwNbWFmPH\njsWqVavELoUZsd27d6NVq1bccGtoypQpWLFihaBjCLbSValUUCgUfF5uLdy+fRvt27dHZmYmrK2t\nxS6HGaEePXpg8uTJGDRokNilGJXS0lL4+Pjg1KlTaN68ea33I8pK9/Tp02jSpAk33Fp4/vnn4ePj\ng9OnT4tdCjNCOTk5iIuLQ9++fcUuxejUq1cPERERiIqKEmwMwZpuVFQUIiIihNq9yRP6P56Zrv37\n96Nr1658W6haMsqmS0SIiopCeHi4ELs3C+Hh4YiKioI+LzjEzAMveOqmR48eSEhIQHZ2tiD7F6Tp\npqSkoKioCEFBQULs3iwEBwejsLAQKSkpYpfCjAgR4dChQxwt1IGdnR06d+6MI0eOCLJ/QZru+fPn\n0b59e/5UVR1YWFigXbt2OH/+vNilMCNy+/ZtNGjQAB4eHmKXYtTat28v2NwTrOkqlUohdm1WlEol\n4uPjxS6DGRGee/oh5NwTpOnGx8fzf7wecNNlNcVzTz+USiUSEhIEeU9FkKabmJjIea4eBAUFITEx\nUewymBHhuacfcrkc9evXx7179/S+b7033SdPniAvLw9yuVzfuzY77u7uyM3NRWlpqdilMCORnp4O\nb29vscswCd7e3khPT9f7fvXedO/fvw+5XC7YrY2rY/ny5VAoFHBycsKECRMqvIDFqVOn4ODgAJlM\nBplMBgcHB1haWuLnn3/WPWfevHnw8vJCw4YN0b17d1y9etVQhwFLS0u4ubnh/v37BhuTGbfMzExR\n30Sr7twDgKNHj0KpVMLR0RHNmzfH2rVrddu2b98OPz8/ODo6wt3dHWPHjkVBQYEhDkFHoVAgMzNT\n/zuu6I6VVMu7AZ89e5batWtX49fpy4EDB8jd3Z2SkpIoNzeXunbtSnPmzKnWa48fP04ymYyKioqI\niGj79u3k6elJqamppNVqac6cOdSmTRshy39G27ZtKTY21qBjMuNUUlJCNjY2pNFoRBm/JnNPpVKR\no6MjrV27loiI4uLiyN7enhITE4mI6O7du/TgwQMiIiosLKQRI0bQjBkzDHMgf5s0aRJ99913tXot\nKrkbsN6Xow8ePICbm5u+d1ttGzduxPjx43W/JefPn48ffvihWq/dsGEDBg0aBDs7OwBAamoqXnrp\nJTRu3BgWFhYYOXIkkpKShCz/GXK5nFe6rFoePnwIFxcX0f7KrMncy87OxuPHjzFy5EgAQEhICFq2\nbKn7S9LLy0vXR7RaLaysrHDz5k3DHMjfhJp7gmS6Yl5V7MqVKwgMDNQ9DgwMRFZWFnJycip9XVFR\nEXbt2oUxY8bovjd06FDcunULycnJUKlU2LBhA/r06SNU6eWqX78+Z7qsWkpLS41m7rm5uWHYsGH4\n/vvvodVqcfbsWdy5cwcvvfSS7jmnT5+Gk5MTZDIZdu/ebfD7Kwo19/R+CSu1Wi3qlbEKCgrg6Oio\neyyTyUBEePz4MRo2bFjh63bt2gVXV9cy9ydTKBQIDQ2Fr68vrK2t4e3tjaNHjwpa/79ZW1tDrVYb\ndExmnNRqtag3C6jp3Bs6dCgmTJiAGTNmwMLCAitXroSnp6due2hoKHJzc5GZmYm1a9fCx8fHIMfx\nlFBzT+8rXSsrK2g0Gn3vtkJbtmzRvRkWFhYGBwcH5Ofn67bn5eXBwsICDg4Ole5n48aNGDVqVJnv\nffTRR4iLi0N6ejpKSkowf/58dOvWDSUlJYIcS3k0Gg3fdYNVi5WVFbRarcHGq8vcu379Ol5//XX8\n9NNPUKlUuHLlCpYsWVLuraoUCgV69+6NoUOHCno8/ybU3NN7061Xr55B/xwePnw4Hj9+jPz8fPz6\n66/w9/fHxYsXddsvXLgAuVxe6Sr33r17OH78+DNN9+LFixg6dCgUCgUsLS0xevRo5OTkGPQMBpVK\nBRsbG4ONx4yXjY2N0cy9y5cvw8/PDz179gQAtGjRAmFhYRXeH1ClUuH27dvCHEgFhJp7em+6Li4u\n+PPPP/W922obNWoU1q9fj6SkJOTk5GDhwoUYO3Zspa/ZuHEjQkNDn7n2b9u2bREZGYmsrCwQETZt\n2gS1Wl2nixvX1MOHD/keV6xanJ2d8fDhQ9GuTFeTuRccHIybN2/i2LFjAIBbt25h7969ukx4y5Yt\nuHv3LgAgLS0N8+bN0zVoQxFs7lV0WgPV8pSx5ORkatq0aa1Os9CX5cuXk1wuJ0dHRxo/fjyVlpbq\ntvXp04cWL15c5vktW7akH3744Zn9lJSU0LRp00ihUJCjoyMplUo6dOiQ0OWX0aRJE7p586ZBx2TG\nSyaTUXZ2tmjj12TuRUZGUqtWrUgmk5G3t3eZ08vmzp1LXl5eZG9vT97e3jRp0iSDH9drr71G27dv\nr9VrUckpY3q/XU9hYSFcXFxQVFTEVxmrIyLCc889h0ePHvEFqVm1+Pn5YdeuXXjxxRfFLsXohYaG\n4rPPPivz5np1GfR2PQ0aNEC9evWqPEWLVS0nJwf16tXjhsuqzcPDQ5CPrpqjjIwMKBQKve9XkLOo\n/f39cfnyZSF2bVYuXbrEKxZWIzz39CM/Px8PHz5EkyZN9L5vQZquUqnki2/rAV8bldUUXw5UPxIS\nEtC6dWtBPnMgWNPl//i642ujspriuacfQs49QZpuSEgI4uLihNi1WeGVLqspf39/3L17F3l5eWKX\nYtSEnHuCZbp5eXkGP5nZlNy6dQv5+fmc6bIasba2RqdOnXD48GGxSzFaarUahw8fRvfu3QXZvyBN\n18rKCv369UN0dLQQuzcL0dHR6Nevn6jXJWbGKSIiAlFRUWKXYbTOnj0Lb29vwa71INiMDg8P56Zb\nB9HR0QgPDxe7DGaEwsPDsW/fPr5QUi0JPfcEa7q9evXCuXPn8OjRI6GGMFmPHj1CXFycwT/2yEzD\n01XaiRMnxC7F6BAR9uzZg4iICMHGEKzpNmjQAAMGDMCGDRuEGsJk/fDDDxg4cCAaNGggdinMSI0e\nPbrM7W9Y9Rw/fhw2NjaCvoGt948B/1NsbCxGjhyJGzducDZZTVqtFi1atMCWLVvQvn17scthRio3\nNxdNmjTBtWvX4O7uLnY5RmPw4MHo2rUrpk6dWqf9GPRjwP/Uvn17yGQyfie1Bg4dOgQnJye0a9dO\n7FKYEXNycsLgwYOxfv16sUsxGhkZGYiJicEbb7wh6DiCNl0LCwtMnToVX3zxhZDDmJQvvvgCU6dO\n5YsFsTqbOnUqVq5cieLiYrFLMQrffvsthg0bBplMJug4gsYLwF/3bWrZsiXWrl0r2HlvpuLIkSOY\nOHEikpKS+MLlTC8GDhyIjh07YtasWWKXImkZGRkICAjAhQsX4O3tXef9VRYvCN50AWDr1q1Yvnw5\nfv/9d17BVYCI0K5dO7z77rsGvy0JM11JSUno3LkzkpOT4eTkJHY5kjV58mTY29tj2bJletmfaJnu\nU6+//jpUKhV27dpliOGM0q5du6DRaDBkyBCxS2EmpGXLloiIiMCSJUvELkWykpOTERkZiffff98g\n4xlkpQsAR48exZgxY3Dp0qUydwxlf73THBAQgI0bN6Jbt25il8NMzL179xAcHIxjx46hVatWYpcj\nKVqtFj179kRYWBjeffddve1X9HjhqUmTJkGtVmPdunV626cpGD9+POrVq4eVK1eKXQozUWvXrsXq\n1asRGxsryOUKjdWKFSuwadMmnDp1Sq93/pVM0338+DECAgKwatUqvPLKK3rbrzHbv38/pkyZgsTE\nxCpvE89YbRERevfuja5du+KDDz4QuxxJSElJQdu2bXHq1Cn4+fnpdd+VNV2935iyKocPHyZPT0/K\nyMjQ+76NzY0bN6hevXq0evVqsUthZiAuLo5sbGxo06ZNYpciupKSEurYsSMtXbpUkP3DkDemrI5P\nPvkEv/76K44fPw5bW1u9798YlJSUoGvXrvD29sbx48exaNEivPnmm3x2BxPEkSNH8MYbb6Br1644\nfvw44uLi4OnpKXZZoiAiTJgwATk5Odi5c6cgn5aV1EqXiEir1dLgwYNp9OjRpNVqBRlDyrRaLY0a\nNYoGDx5MWq2Wrl27Rq1bt6Zhw4ZRfn6+2OUxE6JWq2n+/Pnk4eFBMTExRES0ePFiCgkJoaKiIpGr\nE8fXX39NAQEB9PjxY8HGQCUrXVGaLhFRQUEBBQUF0aJFiwQbQ6oWLVpEwcHBVFBQoPteUVERvfXW\nW9SiRQu6cOGCiNUxU5GRkUHdunWj7t27U2Zmpu77Wq2Whg0bRoMHDya1Wi1ihYa3d+9eksvldPv2\nbUHHkWTTJSJKT0+n5s2b0/LlywUdR0qWL19OzZs3p/T09HK3b968mVxcXGj16tVm+VcA04+YmBhS\nKBS0YMGCchtrcXEx9erVi0aPHk0ajUaECg3v0KFD5OrqSrGxsYKPJdmmS0SUlpZGTZs2pc8//1zw\nscS2bNkyatq0KaWlpVX6PI4bWG2VFydUpLCwkLp27UpvvPEGqVQqA1Uojn379pGrqyudPHnSIONJ\nuukSEd29e5deeOEFev/9903yt65Go6H333+fWrRoQXfv3q3WazhuYDVVUZxQmcLCQnr55Zepf//+\ngmacYtq4cSO5urrSmTNnDDam5JsuEVFWVhZ17tyZwsPDKS8vz2DjCi0vL4/Cw8Opc+fOlJWVVePX\nc9zAqqOqOKEyJSUlNG7cOAoICKCUlBRhChSBWq2mWbNm0fPPP0+XLl0y6NhG0XSJiJ48eUITJ06k\nF198kZKTkw06thCSk5PJ39+fJk2aRE+ePKn1fjhuYBWpSZxQGa1WS19//TW5u7vTsWPH9FegSHJy\ncqhPnz7UrVs3+vPPPw0+vtE03adWrFhBLi4utGLFCqOMGzQaDX333Xfk7OxMK1as0Ms+OW5g/1ab\nOKEqhw8fJnd3d3rvvfeM9pSyffv2kZeXF7399ttUWloqSg1G13SJiK5evUrt27enbt26CX56hz7d\nvn2bunXrRu3bt6ekpCS975/jBkZUtzihKllZWTRkyBDy9fU1aA5aVzk5OTRu3Dhq3LhxnVb9+mCU\nTZeISKVS0ZIlS6hRo0Y0b948SWe9ubm5NG/ePGrUqBEtWbJE0PMfOW4wX/qKE6ojMjKS3N3daezY\nsXTnzh1Bx6qL0tJSWrlyJSkUCpo4caIk5oTRNt2nUlJSaPTo0eTm5kZffvklFRcXi12STnFxMX3x\nxRfk5uZGo0ePptTUVIOMy3GD+REiTqhKTk4OffDBB9SoUSN69913RclHK6LRaGj79u3UokUL6tmz\nJ8XFxYldko7RN92nLl26ROHh4SSXy2nu3LlVnu8qpLS0NJo7dy7J5XIKDw83+LujT3HcYB6EjBOq\nIyMjgyZPnkxOTk701ltvifqLPicnh7766ivy9fWlkJAQOnz4sGi1VMRkmu5TV69epbfffpsaNWpE\n/fv3p507dxrkHMP8/HzauXMn9e/fnxo1akRvv/22ILltTXHcYLoMGSdUR2ZmJn3yySfk5eVFHTt2\npHXr1tH9+/cFH7e0tJRiYmJowoQJ5OTkRMOGDaOTJ09KdqFRWdMV5Spj+lJQUICtW7di586dOHv2\nLEJDQxEREYHOnTvDz8+vzhcl1mg0uHbtGk6cOIGoqCicPn0aHTp0wKBBgzB8+HA0aNBAT0dSd8XF\nxZg5cyaOHTuGyMhIBAYGil0Sq6PMzEyMGDECFhYW2Lx5M9zd3cUuSUetViM6Ohpbt27FoUOH4O/v\nj/DwcPTs2RMBAQF1vnogESE1NRVnzpzB3r17ceDAAfj6+mLgwIEYM2YM5HK5no5EGJK5iLmQ8vPz\ncfDgQURHR+Ps2bPIzMxEYGAglEolmjdvDoVCAYVCAQ8PD8hkMt3V89VqNfLz85GRkYHMzExkZmbi\n5s2biI+Px8WLF6FQKNChQweEh4ejd+/egt+eua62bNmCGTNm8KUijdzTSzFOnDgR8+bN0+tdDfTt\nyZMn+O233xAVFYWTJ08iOTkZvr6+UCqVaNmyZZm517BhQ9jY2MDS0hJqtRqFhYW6eZeZmYnU1FQk\nJCQgISEBtra2aNeuHcLCwhAWFgaFQiH2oVabWTTdf8vNzdX956WlpSEjI0PXWAsKCqBSqZCfnw9n\nZ2fY29vrfig8PDzQuHFjtGnTBm3atDHKO6hev34dQ4YMwYsvvojVq1fzHSmMiEajwccff4x169Zh\n48aN6NGjh9gl1VhxcTESExNx/vx53Lp1SzfvMjIykJOTA7Vajby8PDRq1Ai2tra6uadQKODt7Y2g\noCAolUqjarL/ZpZNtzr+/ocRuwxBcNxgfKQcJ+ibKc89QAK3YGeGZ2dnh9WrV2PBggXo2bMn1qxZ\nY9I/5MbuyJEjUCqV6NKlCw4dOmTSDdfc8UrXhI/vKY4bpMsU4oTaMPW5xytdM+fr64vY2Fg4ODhA\nqVTi4sWLYpfE8Fec0KtXL5w6dQrx8fFm03DNHTddM8Fxg7RwnGC+OF4w4eOrCMcN4jHXOOHfTH3u\ncbzAyuC4QRwcJzCAm67Z4rjBsDhOYE9xvGDCx1ddHDcIh+OE8pn63ON4gVWK4wZhcJzAysNNlwHg\nuEHfOE5gFeF4wYSPr7Y4bqg9jhOqx9TnHscLrEY4bqgdjhNYdXDTZeXiuKFmOE5g1cXxggkfn75w\n3FAxjhNqx9TnHscLrE44bigfxwmsNrjpsmrhuKEsjhNYbXG8YMLHJxRzjhs4TtAPU597HC8wvTLX\nuIHjBKYP3HRZrZhb3MBxAtMXjhdM+PgMxZTjBo4ThGHqc4/jBSYoU40bOE5gQuCmy/TC1OIGjhOY\nUDheMOHjE4sxxw0cJxiGqc89jheYQRlr3MBxAjMEbrpMEMYWN3CcwAyF4wUTPj6pkHLcwHGCOEx9\n7nG8wEQl1biB4wQmBm66zCCkFjdwnMDEwvGCCR+fVIkZN3CcIA2mPvc4XmCSIlbcwHECkwJuukwU\n/44bVq9eLejKh+MEJhUcL5jw8RkLIeMGjhOkydTnnlHGC8uXL4dCoYCTkxMmTJgAlUpV7vOSk5Mx\nYMAAuLm5wcXFBX369MGNGzd023/88UdYW1tDJpPBwcEBMpkMJ06cMNRhsGoQKm7gOMH46Gve/1OP\nHj1gaWkJrVYrZOnVJsmme/DgQSxduhTHjh1DWloabt26hQ8//LDc5+bm5qJ///64ceMGHjx4gLZt\n26J///5lntOxY0fk5+fj8ePHyM/PR+fOnQ1xGKwG9B03cJxgfPQ97wFgy5YtUKvVsLAod9EpDiKq\n8OuvzYY3fPhwmjt3ru7x0aNHyd3dvVqvzc7OJgsLC8rOziYiog0bNlCnTp3Kfa5Yx8cqd+3aNWrd\nujUNGzaM8vPza/RatVpN8+fPJw8PD4qJiRGoQlZX5c09fc57IqK8vDzy9fWl33//nSwtLUmj0dS9\n8Gr6+/jK7auSXOleuXIFgYGBuseBgYHIyspCTk5Ola/97bffoFAo0LBhQ933/vjjD7i5ucHPzw8L\nFy6UzJ8ZrHy1jRs4TjBu+p73H3zwAaZMmQK5XC5IvbVWUTcmEVe6zZo1o4MHD+oeq1QqsrCwoLS0\ntEpfd/fuXfL09KTt27frvpeSkkKpqalERHT58mXy9/enzz77jIh4pWsMNm/eTC4uLrRq1SrSarUV\nPi8mJoYUCgUtWLCA1Gq1AStktVHe3NPnvI+Li6Pg4GDSarWUmpoqqZWuJJru5s2byd7enhwcHKhv\n374UFBREkZGRuu1//vknWVpalvnT4d+ysrLI39+fFi9eXOlY27Zto5CQECLipmssKosbOE4wTgAE\nm/darZbatWtHJ06cIKK/Fl7cdKswfPhwmjdvnu7x01VMRXJycig4OJg++OCDKve9bds2UiqVRMRN\n15gUFRXRW2+9RS1atKALFy4QEVFGRgZ169aNunfvTpmZmSJXyGqivLmnr3mfm5tLVlZWpFAoyN3d\nnVxdXcnCwoIUCgWdOnVKfwdRCaNrugcOHCCFQkFXr16l7Oxs6tq1a4UNNT8/n9q2bUvTp08vd/v+\n/fvpwYMHRESUlJRErVq1ok8++YSIuOkao6dxw4wZMzhOMGLlzT19zvsHDx7ovuLi4sjCwoIyMzNJ\npVLp9TgqYnRNl4ho+fLlJJfLydHRkcaPH0+lpaW6bX369NH9OfHjjz+SpaUl2dvb674cHBzo7t27\nRET03nvvkVwuJ3t7e2rWrFmZScpN1/io1WqaMmUKWVtbU7du3Wp8dgOThormnr7m/T9JLdPlT6SZ\n8PGZmszMTIwYMQIWFhZYt24dPvvsMxw7dgyRkZFl3vVm0mfqc88oP5HG2D/FxMSU+bBD06ZNDXrt\nBsb0hVe6Jnx8puDptRPWrl2LTZs2lXvurZTvTMHKZ+pzj1e6zCj988MOCQkJFX7YQap3pmCsPNx0\nmST9O06o6toJhr5UJGO1xfGCCR+fMapOnFAVjhukz9TnHscLzChUN06oCscNTMq46TJJqGmcUBWO\nG5hUcbxgwsdnDPQRJ1SF4wbpMfW5x/ECkyR9xQlV4biBSQk3XSYKfccJVeG4gUkFxwsmfHxSZIg4\noSpP4wZ/f3+sXr0aMpnM4DWYO1OfexwvMEkwVJxQladxg0wmQ0hICC5cuCBKHcw8cdNlBmHoOKEq\n/4wbevXqxXEDMxiOF0z4+KRACnFCVThuMDxTn3scLzBRSCVOqArHDcyQuOkyQUgtTqgKxw3MUDhe\nMOHjE4MxxAlV4bhBeKY+9zheYAZhLHFCVThuYELipsv0wtjihKpw3MCEwvGCCR+fIZhCnFAVjhv0\nz9TnHscLTBCmEidUheMGpk/cdFmtmFqcUBWOG5i+cLxgwscnBHOIE6rCcUPdmfrc43iB6YW5xAlV\n4biB1QU3XVYt5hYnVIXjBlZbHC+Y8PHpA8cJVeO4oeZMfe5xvMBqheOE6uG4gdUEN11WLo4Taobj\nBlZdHC+Y8PHVBscJdcdxQ9VMfe5xvMCqheME/eC4gVWGmy4DwHGCvnHcwCrC8YIJH191cJwgPI4b\nnmXqc4/jBVYujhMMg+MG9k/cdM0UxwmGxXEDe4rjBRM+vvJwnCA+jhtMf+5xvMAAcJwgFRw3mDdu\numaC4wRp4bjBfHG8YMLHB3CcYAzMMW4w9bnH8YKZ4jjBOHDcYF6sxS7AELRaLR49eoTMzExkZmbi\n8ePHUKlUAICdO3fCwcEBCoUCCoUCzs7OsLQ0/t9FMTExGDVqFCZOnIh58+bByspK7JJYJZ7GDVu2\nbEGvXr2wcOFCvPXWW7CwKHexZDSICHl5ebq5l5OTo5t7kZGReO655+Du7g6FQgE3NzdYW5t+SzK5\nI8zPz8f58+cRHx+v+7pz5w7s7e11jdXR0VH3n7tt27YyPxQFBQXw8fGBUqnUfYWEhBjNn3wcJxi3\n4cOHQ6lUYsiQITh+/LhRxQ1PnjzBH3/8UWbuJScnw9raGgqFAh4eHmjYsCFsbGwAADt27EBhYaFu\n7mVnZ0MulyMoKAghISFQKpVo27Yt5HK5yEemXyaR6d6+fRvR0dGIiorCuXPnEBgYqGuWSqUSzz//\nPGxtbau1r5KSEty+fRvx8fG65n3x4kW0b98e4eHhiIiIQNOmTQU+otrJzMzEiBEjYGFhgc2bN/Ob\nZUasuLgYM2fOxLFjx7Bjxw4EBQWJXVK5Hj58iH379iEqKgoxMTF4/vnnERISopt7fn5+sLe3r9a+\n1Go17t69i4SEBN38i4uLQ7NmzRAREYGIiAgEBgYaxeq/skwXRFTh11+bpamwsJDWr19PISEh5Obm\nRuPGjaOff/6ZCgoK9D5WQUEB/fzzzzRu3Dhyc3OjkJAQWr9+PRUWFup9rNo6fPgwKRQKWrBgAanV\narHLYXqyefNmcnFxoVWrVpFWqxW7HCIiUqvVFBUVRa+88grJZDJ69dVXacOGDZSVlaX3sUpLS+nY\nsWP0zjvvULNmzahp06b02WefCTKWPv3dO8vvqxVtIIk23dTUVHrnnXfI2dmZwsPDaf/+/QZtMmq1\nmvbv30/h4eHk7OxM77zzDqWmphps/PLqmT9/PikUCoqJiRGtDiaca9euUevWrWno0KGUl5cnWh05\nOTn06aefko+PD7Vv355+/PFHKioqMtj4Wq2W4uLiaNy4ceTk5EQjR46k8+fPG2z8mjCJppuVlUUz\nZiCB+DkAABlRSURBVMygRo0a0ezZsyklJUXskiglJYVmz55NjRo1ohkzZhj8t29GRgZ169aNunfv\nTpmZmQYdmxlWUVERvfXWW9SiRQv6448/DD720qVLydXVlUaNGiWJRvfo0SP6/PPPycPDgwYNGkTX\nr18Xu6QyjLrpFhcX08cff0zOzs40bdo0un//vtglPeP+/fs0bdo0cnZ2po8//piKi4sFH5PjBPNk\nyLhBq9XSDz/8QF5eXjRw4EC6evWqoOPVRmFhIS1evJhcXFzorbfekkzsYLRN9/fffyd/f38aMGAA\n3bp1S9RaquPWrVs0YMAA8vf3p3PnzgkyRl3jhC+//JLc3d3J0dGRxo8fT6WlpRU+98iRI9SmTRuS\nyWTUrFkzWrNmTV1KN0k1+fe0sLAge3t7sre3JwcHB3rzzTdrNaYh4oa0tDTq1asXKZVKOnv2rCBj\n6NOjR49o5syZ5O7uTjt37hS7HONruiUlJfT++++Tm5sbbd26VTJvIFSHVqulLVu2kJubG73//vtU\nUlKit33XNU44cOAAubu7U1JSEuXm5lLXrl1pzpw55T5XpVKRo6MjrV27loiI4uLiyN7enhITE+t0\nDKakJv+eRESWlpZ0+/ZtvYwtVNyg1WppzZo15OLiQosWLSKVSqW3fRvC6dOn6YUXXqAhQ4aIuuo1\nqqabnp5O7dq1o/79+0sySqiu+/fvU//+/aldu3aUnp5e5/3pI04YPnw4zZ07V/f46NGj5O7uXu5z\nHzx4QJaWlmWikrZt29K2bdtqNbYpqsm/J9FfK92bN2/qtQZ9xg2FhYU0dOhQCg4OpkuXLumpQsMr\nKiqi//mf/yFvb2/R8mejabq///47eXp60qJFi4xqdVsRrVZLCxcuJC8vr1rHDfo8OyEwMJB27Nih\ne/zo0SOytLSk7Ozscp8/YsQI+u6770ij0dCZM2dILpfTvXv36lSDKanpv6eFhQV5enqSQqGg1157\nTW9nvegjbrh79y4plUoaMWKEQc9IENLOnTvJxcWFtm7davCxjaLpbtu2jVxdXemXX34x2JiGsmfP\nHnJ1da3xKlHfZyc0a9aMDh48qHusUqnIwsKC0tLSyn1+dHQ0yeVysra2JhsbG1q3bl2dazAlNf33\nPHnyJKlUKsrLy6Np06ZRq1atSKPR6KWWusQN586dIw8PD1qyZIlJLHb+6cKFC9SkSROaN2+eQY9N\n8k133bp15OnpadJ5YWJiInl4eND69eur9Xx9xAmbN2/WvWnTt29fCgoKosjISN32P//8s8KV2bVr\n1+i5556jw4cPExHRjRs3qEWLFrRv375a1WIK6vLv+W8ajYbs7e3p8uXLeq+xJnHDyZMnTXax81RW\nVhYplUqaPn263n7JVUXSTXfdunXk7e1NN27cEHwssV2/fp28vb0rXTEK+WGH4cOH07x583SPY2Ji\nSKFQlPvcnTt3Ups2bcp8b+bMmTR9+nS91mTMavLv+W9qtZrs7e0FyU6rGzecPn2aXF1dy6zWTVVO\nTg795z//oalTpxpkxSvZprtt2zby9PQ0i4b71PXr18nDw6PcqEHoDzscOHCAFAoFXb16lbKzs6lr\n1670wQcflPvcW7dukUwmo6NHjxIR0c2bN6l58+YcMfxDTf49r1y5QhcuXCCNRkOPHz+mt99+m/z8\n/AQ7x7qquCEhIcFsGu5Tubm5FBISQv/7v/8r+FiSbLrnzp0jV1dXk44UKpKYmEiurq5l3lwz1Icd\nli9fTnK5vNzzSvv06UOLFy/WPY6MjKRWrVqRTCYjb2/vSk+HMlfV/fc8evQo+fr6kr29Pcnlcho4\ncKDez2QoT3lxw/3798nHx6dMNGIuHj16RM2bN6cffvhB0HEk13TT09PJy8vLpHOkquzZs4e8vLzo\n7t27fO0EJqh/xg1ZWVnUsWNHWrBggdhliebq1avk6upKZ86cEWwMSTXdkpISateuHS1atEjv+zY2\n//3vf8nBwYG6du3K105ggnoaN8hkMurdu7fB3lCSqr1795KHh4dgp0BW1nQNfouEBQsWQKFQYM6c\nOYYeWnIWLFiAF154Ae3bt+dr3zJB2dnZoVevXrC3t8eGDRtM4u4odREWFoaJEydi7NixTxeYBmPQ\ni5ifO3cO4eHhSExMNLmrwdfW/fv3ERgYiL1796Jt27Zil8NM1MOHD9G6dWvs3r0bHTp0ELscSVCr\n1fjPf/6DSZMmYcKECXrdd2UXMTdY0y0pKUGbNm0wf/58DB06VC/7NBVbt27FwoULER8fX+07XDBW\nE0OGDEHjxo2xbNkysUuRlEuXLqF79+6Ij4+Hj4+P3vYribsBL1u2DL6+vnj99dcNNaTRGDp0KF54\n4QWeEEwQ+/btw8WLF/Hxxx+LXYrkBAQEYMaMGZg+fbrBxjTISvfhw4do2bIlzp07h+eff77O+zNF\nt27dQvv27ZGUlARXV1exy2EmQqPRIDg4GJ988gn69+8vdjmSVFJSAl9fX2zZsgWhoaF62afoK91F\nixZh2LBh3HAr0axZMwwdOhSffvqp2KUwE7JlyxbY29sjIiJC7FIky9bWFh999BHef/99g7ypJvhK\nNzU1FUqlElevXuU3z6pw//59vPjii0hISEDjxo3FLocZuSdPnsDPzw8//vgjOnfuLHY5kqbRaNC6\ndWssXboUYWFhdd6fqCvdb775BhMmTOCGWw3u7u4YP348vvnmG7FLYSZg586daNasGTfcarCyssKH\nH36IpUuXCj6WoCvdoqIi+Pj44Pz582jSpEmt92NOUlJS0LZtW9y5cwfPPfec2OUwIxYaGopZs2Zh\nwIABYpdiFFQqFZo0aYKDBw+iVatWddqXaCvdbdu2oWPHjtxwa6Bp06bo0KEDtm/fLnYpzIhduHAB\nd+7cQb9+/cQuxWjY2NjgzTffxMqVKwUdR9Cmu3LlSkyZMkXIIUzSlClTsGLFCrHLYEZs1apVmDhx\nIqytrcUuxai8+eab2Lp1KwoKCgQbQ7Cme/v2bdy5cwe9evUSagiT9fLLL+POnTtISUkRuxRmhDQa\nDXbt2oWRI0eKXYrR8fT0RNu2bXHw4EHBxhCs6UZHR6Nfv36wsrISagiTZWVlhbCwMERHR4tdCjNC\nsbGx8PDw4FivliIiIhAVFSXY/gVtuuHh4ULt3uSFh4dz02W1EhUVxefl1kG/fv2wb98+aDQaQfYv\nSNPNz8/H77//ztFCHfTq1Qu///478vPzxS6FGRle8NRN48aN4enpidjYWEH2L0jTPX/+PAIDA9Gg\nQQMhdm8W7O3/X3v3HtPU+cYB/ItQ5e4UOmwtLQjTgRO1B1ammZvbjImuiDPqtqhMajZkGv+YuMWM\n3cDpbjGLE+M2E50avGWawlRGskTnJeloVUwUL4xbaKuAF+RSaMv7+8OfZE4HKufSU55PYoIpPOc5\nObzfvpye855wTJgwAVarVepWiIzcunULDQ0N4DhO6lZkbdq0aTh9+rQgtQUJXavVSgedBxzHUeiS\nx2Kz2TBx4kT6LGWAhBx7goVuamqqEKUHldTUVFRUVEjdBpERmvDwQ8ixRzNdH8ZxHGw2m9RtEBmh\nscePZ599Fg6HA7dv3+a9Nu+h29PTg/r6eiQkJPBdetBJSEhAXV2d6I8TIfJVW1uLxMREqduQvcDA\nQMTFxaG+vp732ryHbktLC8LDwzFs2DC+SyM+Ph5//PEH73X/7fPPP8eSJUv6/J7NmzcjLS0NwcHB\nyM7OFqSP4OBghIWFoaWlRZD6xP/Y7XaoVCre6/rK2Ovu7sayZcsQFxeH4cOHQ6/X4+jRo4L0olKp\nYLfbea/Le+g6HA5BDrqvGT16NPLz82EymQTdjkqlgsPhEHQbxD8wxnDt2jW/Hn8ejwdarRZ//vkn\nbt++jYKCAixYsECQGalQY0+2obtjxw68+OKLyMvLw8iRI5GQkHDfO9706dOxdu1aGAwGDB8+HHPn\nzsWtW7cAAMeOHUNsbOx99e69k5eVleHLL7/E3r17ERERgcmTJz90+5mZmcjIyMDIkSOF20kAarVa\nkHdb4n9aWloQFhYm+HP2pBx7oaGh+OSTT3przJ49G/Hx8YJcaSDU2OM9dO/cuYPhw4fzXfahLBYL\nkpKS0NLSgry8vAdmnTt37sT27dvhdDoRGBh433OQAgIeuuoaZs6cibVr12LhwoW4c+cOzpw5I+g+\n9CcyMlLQxTeI/xiMY+/atWu4cuUKxo8fP7Adegihxh7voet2u0Vb2Uin0yE7OxsBAQHIysqCw+HA\n9evXe19fvHgxkpKSEBISgoKCAuzfv192H0oFBQXB7XZL3QaRAY/HI9r1ub4w9jweDxYtWoR33nkH\nY8eO5bU2INzYE+1pwEIYNWpU79chISEAcN870z//jNHpdHC73WhubhavQUL8lNRjjzGGRYsWYdiw\nYdi0aRNvdcXA+5RUoVDA4/HwXfaJNDQ09H5dV1cHhUKB6OhohIWFoaOjo/c1r9eLpqam3v//158/\nUvB4PFAoFFK3QWQgKChIsEVaHpfQY89kMqG5uRmHDx8WbHYv1NjjfaYbEREhyAXFT2LXrl2oqqpC\nR0cHPv30U8yfPx8BAQEYO3YsXC4Xjhw5Ao/Hg8LCQnR3d/f+XExMDGpra/v8c8jr9cLlcsHr9cLj\n8aCrq0uQX/jW1laEh4fzXpf4n8Ey9nJyclBVVQWz2YyhQ4cKtg9CjT3eQ1fIS5z6exf89+uLFy9G\nVlYW1Go1uru78f333wO4e4K8qKgIJpMJGo0GERER0Gg0vT83f/58MMYQFRX1n7czFxYWIjQ0FF99\n9RV2796N0NBQrFu3boB7+CC73Q61Ws17XeJ/oqKi0N7eDpfLxXttXxl79fX1+PHHH3H27FnExMQg\nIiICkZGRKC4u5mEv7yfY2GOM/ee/uy8/nuvXr7ORI0c+9s/x7eWXX2bbtm2Tuo0BGzFiBGtqapK6\nDSITsbGx7O+//5a0B38Ze6+99ho7evToE/3s/7PzobnK+0w3KioKbW1t6Orq4rv0oONyudDe3o6o\nqCipWyEyoVar6WYanjgcDkFmuryH7pAhQ6DValFdXc136cfiSx+GPanq6mrodDq/2Bcijri4OFy9\nelXSHvzh99Xr9aK2thZarZb32oJcUHtvLcrk5GQhyj8SMe4TF5rVaoVer5e6DSIj98Zef2uHCMkf\nxl5VVRVUKpUgN5sIcp0ux3G0DiwPKioqaF1i8lho4Xt+CDn2BAtdOvADR2ujksel1+tx7tw5n7le\nV66EHHuChG5qairOnTuH9vZ2IcoPCm1tbTh//jyFLnksTz31FGJjY2nSM0DHjx/HCy+8IEhtQUI3\nMjISBoMB5eXlQpQfFMrLy2EwGBAZGSl1K0RmjEYjSkpKpG5Dturq6tDY2Ij09HRB6gu29gId+IGh\nx2iTJ5WRkQGz2Sx1G7JVWlqKWbNmCXZ7saChW1paSueWnoDX60VpaSmFLnki6enpcDgcqK2tlboV\nWTKbzcjIyBCsvmChO2bMGGi1WjrF8AR+//136HQ6xMfHS90KkaHAwEC88cYb2LVrl9StyE5jYyP+\n+usvzJw5U7BtCLq04/Lly1FUVCTkJvxSUVERcnNzpW6DyFhOTg62bt3qMyv+ycVPP/2Et956S9BF\npgJYH6v5BAQEsL5e709HRwe0Wi0qKioQFxf3xHUGk5qaGqSlpaG+vh6hoaFSt0NkbOrUqcjLy0Nm\nZqbUrciC2+1GXFwcysrK8Nxzzw2oVkBAABhjD701T9CZbmhoKJYsWYItW7YIuRm/smXLFmRlZVHg\nkgHLzc3FDz/8IHUbsnHw4EEkJiYOOHD7I+hMF7h7+YVer8eFCxcQExMzoFr+zul0Yvz48bDZbNDp\ndFK3Q2Suu7sb48aNw44dOzBt2jSp2/FpXq8XKSkp+PrrrzF79uwB15NspgvcfVTH4sWLUVhYKPSm\nZK+wsBBLliyhwCW8GDp0KL744gt89NFHsns2oNh27tyJqKgozJo1S/BtCT7TBYCmpiYkJSXBYrFg\nzJgxA67nj6qrq2EwGFBVVYXo6Gip2yF+wuv1YvLkySgoKMCcOXOkbscnuVwujBs3DsXFxZgyZQov\nNSWd6QKAUqnEqlWr8MEHH9A77kMwxrB69WqsWrWKApfwKjAwEBs2bEBeXh46Ozulbscnffvtt5g0\naRJvgdsfUWa6wN13E47jkJ+fjzfffJOXmv6iuLgYhYWFsNlsGDZsmNTtED+0YMEC6HQ6fPPNN1K3\n4lPOnz+PV155BVarlde1c/ua6YoWugBgsVhgNBpRWVlJH6r9n9PpxMSJE1FaWoq0tDSp2yF+qqmp\nCSkpKfj1118FW8hFbtxuN9LT07F8+XIsW7aM19qSn1645/nnn0d2djbee+89Os2Au6cVcnJyYDKZ\nKHCJoJRKJTZt2oSlS5eira1N6nZ8wvr16xEdHQ2TySTqdkWd6QJAV1cXpk2bhjlz5mDt2rW81pab\ndevWoaSkBMeOHaPTCkQUJpMJt2/fxr59+zBkiKhzLp/y22+/4d1334XFYsHo0aN5r9/XTJf3pwE/\nisbGRqbRaNihQ4cEqS8Hhw4dYhqNhtntdqlbIYOIy+ViU6ZMYZ999pnUrUjmwoULTKlUslOnTgm2\nDfTxNGBJQpcxxiwWC1MqlayyslKwbfiqyspKplQqmcVikboVMgg5nU6m1WrZ/v37pW5FdM3NzSwx\nMZFt375d0O34ZOgyxtiePXuYWq1mly9fFnQ7vuTSpUtMrVazPXv2SN0KGcRsNhtTKpXs6NGjUrci\nmlu3bjGO49iHH34o+LZ8NnQZY+znn39msbGx7NKlS4JvS2qXLl1iGo2Gbdu2TepWCGEnT55kSqWS\nlZWVSd2K4G7evMnS09PZ+++/z3p6egTfnk+HLmN3g1etVvv1qYbKykqmVqspcIlPOXHiBFMqlX79\n+cr169cZx3Fs5cqVogQuYzIIXcbunmrw14N/6NAhplQq6ZQC8UkWi4Wp1Wq2YcMG0UJJLGfPnmVx\ncXEsPz9f1H2TRegydvfgazQaVlhY6BcHv6enhxUUFDCNRkMfmhGf1tDQwDiOY2+//Tbr6OiQuh1e\nHDhwgEVHR0sy2ekrdEW/Trc/drsdc+fOhUqlwtatW2V755rT6UROTg6cTicOHjwIlUoldUuE9Kmz\nsxMmkwlVVVX45ZdfBF9XViidnZ34+OOPsX//fhw8eBAcx4neg8/ckfYo1Go1jh8/juTkZKSkpKC4\nuFhWd68xxlBcXIyJEyciOTkZx44do8AlshASEoLdu3cjNzcX06dPx7p162T3uJ9Tp05h0qRJaGxs\nhM1mkyRw+/VfU2AmwemFf7NYLCw5OZllZmayq1evStrLo7h69SrLzMxkycnJdDqByFpdXR2bMWMG\n4ziOnT59Wup2+tXS0sJWrVrFRo0axQ4cOCB1O32eXvC5me4/paWlwWq1Qq/Xw2AwYMWKFbh27ZrU\nbT3A6XRixYoVMBgM0Ov1sNlstJYCkTWtVouysjKsXLkS8+fPx9y5c3Hx4kWp23pAR0cH1q9fj3Hj\nxqGzsxOVlZWYN2+e1G31yadDFwCCg4ORn5+PixcvQqFQIDk5GWvWrEFNTY3UraGmpgZr1qzB+PHj\noVAocPHiReTn59M6CsQvBAQEICsrC5cvX8bUqVPx0ksvISsrC1arVerWcOPGDXz33Xd45plncObM\nGZw8eRJbt26FUqmUurV++Xzo3qNUKrFx40bYbDZ4vV6kpaXBaDTiyJEj8Hq9ovXh9Xpx5MgRGI1G\npKWlwev1wmazYePGjbI44IQ8rpCQEKxevRqXL19GUlIS5s2bB4PBgB07doi6MDpjDBUVFcjOzkZC\nQgLOnTsHs9mMffv2YezYsaL1MVA+d/XCo+ro6MDevXtRVFSEuro6vP766zAajZgxYwbvz6xva2tD\neXk5SkpKUFpaCp1Oh9zcXCxcuJCe2ksGnXsTj82bN+PUqVN49dVXkZGRgdmzZ/M+8XC73Thx4gTM\nZjPMZnPvcqhLly716UmOzyxiLpSamhqUlJTAbDbDYrFgwoQJ4DgOqamp4DgOCQkJCA4OfqRaLpcL\n1dXVsFqtqKiogNVqxfnz52EwGGA0GmE0GhEfHy/wHhEiD83NzTh8+DDMZjPKy8sxZsyY+8ZeUlLS\nI0+C3G43GhoaYLPZesdfRUUFEhMTkZGRgYyMDKSkpCAg4OErJvoSvw/df2ptbYXVau09aDabDXV1\ndQgLC4NKpYJarUZkZCSCgoIAAB6PB62trbDb7XA4HGhvb4dOp4Ner+/9xeE4DpGRkRLvGSG+raur\nC2fPnr1v/F25cgWBgYG9Y2/EiBFQKBQYMmQIPB4P2tvb4XA44HA4cPPmTcTExGDy5Mm9wZ2amirL\na/UHVeg+TE9PD27cuAGHwwG73Y62tja43W4AgEKhQHh4ONRqNVQqFaKiomTxTkqIHDDG0Nra2hus\nN27cgMfjQU9PDxQKBUJCQqBSqaBSqfD0008jMDBQ6pZ5MehDlxBCxCSrO9IIIcSfUegSQoiIKHQJ\nIUREFLqEECIiCl1CCBERhS4hhIiIQpcQQkREoUsIISKi0CWEEBFR6BJCiIgodAkhREQUuoQQIqKg\n/r6BVtwihBD+9LnKGCGEEH7R6QVCCBERhS4hhIiIQpcQQkREoUsIISKi0CWEEBH9DzdwmLU8k+Xd\nAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "np.random.seed(seed=10) #seed random number generator for reproducibility\n", "\n", "Weights_2 = np.random.rand(1,3)-0.5*2 #connections between hidden and output\n", "Weights_1 = np.random.rand(2,3)-0.5*2 #connections between input and hidden\n", "\n", "Weight_Dict = {'Weights_1':Weights_1,'Weights_2':Weights_2} #place weights in a dictionary\n", "\n", "Train_Set = [[1.0,1.0],[0.0,0.0],[0.0,1.0],[1.0,0.0]] #train set\n", "\n", "network = NeuralNetwork()\n", "network.add_layer(2,['Input 1','Input 2'],\n", " [[round(x,2) for x in Weight_Dict['Weights_1'][0][:2]],\n", " [round(x,2) for x in Weight_Dict['Weights_1'][1][:2]]])\n", "#add input layer with names and weights leaving the input neurons\n", "network.add_layer(2,[round(Weight_Dict['Weights_1'][0][2],2),round(Weight_Dict['Weights_1'][1][2],2)],\n", " [round(x,2) for x in Weight_Dict['Weights_2'][0][:2]])\n", "#add hidden layer with names (each units' bias) and weights leaving the hidden units\n", "network.add_layer(1,[round(Weight_Dict['Weights_2'][0][2],2)])\n", "#add output layer with name (the output unit's bias)\n", "network.draw()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Above we have out network. The depiction of $Weight_{Input_{1}\\to.Hidden_{2}}$ and $Weight_{Input_{2}\\to.Hidden_{1}}$ are confusing. -0.8 belongs to $Weight_{Input_{1}\\to.Hidden_{2}}$. -0.5 belongs to $Weight_{Input_{2}\\to.Hidden_{1}}$. \n", "\n", "Lets go through one example of our network receiving an input and making a guess. Lets say the input is [0 1].\n", "This means $Input_{1} = 0$ and $Input_{2} = 1$. The correct answer in this case is 1. \n", "\n", "First, we have to calculate $Hidden _{1}$'s input. Remember we can write input as\n", "\n", "$net = \\displaystyle\\sum_{i=1}^{Inputs}Input_i * Weight_i$\n", "\n", "with the a bias we can rewrite it as\n", "\n", "$net = Bias + \\displaystyle\\sum_{i=1}^{Inputs}Input_i * Weight_i$\n", "\n", "Specifically for $Hidden_{1}$\n", "\n", "$net_{Hidden_{1}} = -0.78 + -0.25*0 + -0.5*1 = -1.28$\n", "\n", "Remember the first term in the equation above is the bias term. Lets see what this looks like in code. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[-1.27669634 -1.07035845]\n" ] } ], "source": [ "Input = np.array([0,1])\n", "net_Hidden = np.dot(np.append(Input,1.0),Weights_1.T) #append the bias input\n", "print net_Hidden" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that by using np.dot, I can calculate both hidden unit's input in a single line of code. \n", "\n", "Next, we have to find the activity of units in the hidden layer. \n", "\n", "I will translate input into activity with a logistic function, as I did in the previous post. \n", "\n", "$Logistic = \\frac{1}{1+e^{-x}}$\n", "\n", "Lets see what this looks like in code." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[ 0.2181131 0.25533492]\n" ] } ], "source": [ "def logistic(x): #each neuron has a logistic activation function\n", " return 1.0/(1+np.exp(-x))\n", "\n", "Hidden_Units = logistic(net_Hidden)\n", "print Hidden_Units" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So far so good, the logistic function has transformed the negative inputs into values near 0. \n", "\n", "Now we have to compute the output unit's acitivity. \n", "\n", "$net_{Output} = Bias + Hidden_{1}*Weight_{Hidden_{1}\\to.Output} + Hidden_{2}*Weight_{Hidden_{2}\\to.Output}$\n", "\n", "plugging in the numbers\n", "\n", "$net_{Output} = -0.37 + 0.22*-0.23 + 0.26*-0.98 = -0.67$\n", "\n", "Now the code for computing $net_{Output}$ and the Output unit's activity. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "net_Output\n", "[-0.66626595]\n", "Output\n", "[ 0.33933346]\n" ] } ], "source": [ "net_Output = np.dot(np.append(Hidden_Units,1.0),Weights_2.T)\n", "print 'net_Output'\n", "print net_Output\n", "Output = logistic(net_Output)\n", "print 'Output'\n", "print Output" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Okay, thats the network's guess for one input.... no where near the correct answer (1). Let's look at what the network predicts for the other input patterns. Below I create a feedfoward, 2-layer neural network and plot the neural nets' guesses to the four input patterns. " ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWkAAAEPCAYAAACA1jUJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEC5JREFUeJzt3X+MXNV5xvHn2VIrJRCaX3JawxpsEohBjpukFvmhMihR\nMJFStwlKbOSKWi2yoliNKrWCVqowUaUqVSs1EY0SVxYB88OpRR2sCIiJlK1rJQSrwXZtbOEE1oBj\nWYkwiQMhsb1v/5hrMx52Zu7s7J15Z/z9SKO999xz57wjWY+PzpyZcUQIAJDT2KALAAC0RkgDQGKE\nNAAkRkgDQGKENAAkRkgDQGKENAB0wfYG20dt72nT58u2D9reZXtJQ/sy2wdsP2371jLjEdIA0J27\nJF3f6qLtGyQtjIh3Sloj6atF+5ikO4t7r5K00vaVnQYjpAGgCxGxQ9KxNl2WS7qn6PsDSRfZnitp\nqaSDEXEoIk5I2lT0bYuQBoDZNU/S8w3nLxRtrdrbIqQBoFru5ebzZquKXtjmC0QAlBIRPYXepXYc\nKt/9aES8o8shDku6pOH84qJtjqTxadrbShHSkvTww+S0JN177zqtWrVu0GUM3IsvDrqCPB58cJ0+\n9al1gy4jhVWrespnSdIhSTFWbhHBU1NzW11S6xnyVkmfk/QN29dIeikijtr+maTLbc+XdETSCkkr\nO9WQJqQBoG9KhrSmpl7XZPt+STVJb7X9nKTbVZ8lR0Ssj4iHbX/c9o8kvSxpteoXT9leK2mb6kvN\nGyJif6cSCGkA556yIT2NiLipRJ+1LdoflXRFN+MR0sksXlwbdAlI5t3vrg26hNFzXsno+81vqq2j\nBEI6GUIazRYtqg26hNHTw0y63whpAOceQhoAEiOkASAxQhoAEiOkASAxQhoAEiu7BS+B4akUAGYL\nM2kASIyQBoDECGkASIyQBoDECGkASIyQBoDE2IIHAIkxkwaAxAhpAEiMkAaAxAhpAEiMkAaAxAhp\nAEiMLXgAkBgzaQBIjJAGgMQIaQBIjJAGgMQIaQBIjN0dAJAYM2kASIyQBoDECGkASIyQBoDECGkA\nSIyQBoDE2IIHAIkxkwaAxAhpAEiMkAaAxAhpAEiMkAaAxAhpAEiMLXgAkBgzaQBIjJAGgMQIaQBI\njJAGgMSGKKQrrdT2BttHbe+pchwA6MrYWLlHC7aX2T5g+2nbt05z/W9sP2n7h7b/z/ZJ279bXJu0\nvbu4/kTHUnt6oZ3dJen6iscAgO6cd165xzRsj0m6U/Vsu0rSSttXNvaJiH+JiD+IiPdK+jtJExHx\nUnF5SlKtuL60U6mVhnRE7JB0rMoxAKBrvc2kl0o6GBGHIuKEpE2SlrcZbaWkBxrOrS6yd3gWZgBg\ntvQW0vMkPd9w/kLR9jq2f0fSMkkPNjSHpMds77R9S6dS07xxeO+9684cL15c0+LFtUGVAiCJp56a\n0P79E7P/xC0CeGJyUhOTk7M50ick7WhY6pCkD0XEEdtvVz2s9xerDtNKE9KrVq0bdAkAklm0qKZF\ni2pnzrdsuWN2nrhFSNcWLFBtwYIz53ds3z5dt8OSxhvOLy7aprNCZy91KCKOFH9/anuL6ssnLUO6\nH8sdLh4AkENvyx07JV1ue77tOaoH8dbmTrYvknStpIca2s63fUFx/EZJH5O0t12plc6kbd8vqSbp\nrbafk3R7RNxV5ZgA0FEP+6Qj4pTttZK2qT7R3RAR+22vqV+O9UXXP5H07Yj4VcPtcyVtsR2q5+99\nEbGt3XiVhnRE3FTl8wPAjPT4LXgR8aikK5ravtZ0freku5vanpW0pJux0qxJA0DfDNEnDglpAOce\nQhoAEiOkASAxQhoAEiOkASAxfuMQABJjJg0AiRHSAJAYIQ0AiRHSAJAYIQ0AiRHSAJAYW/AAIDFm\n0gCQGCENAIkR0gCQGCENAIkR0gCQGCENAImxBQ8AEmMmDQCJEdIAkBghDQCJEdIAkBghDQCJEdIA\nkBhb8AAgMWbSAJAYIQ0AiRHSAJAYIQ0AiRHSAJAYIQ0AibEFDwASG6KZdNtKbb/J9sJp2hdXVxIA\nVGxsrNwjgZZV2P60pAOSHrS9z/YfNlz+etWFAUBlRiGkJf29pPdFxBJJqyVttP2nxTVXXhkAVGWI\nQrrdmvRvRcQRSYqIJ2xfJ+lbti+RFH2pDgCqkCSAy2gX0sdtL4yIH0tSRByxXZP0TUlX9aM4AKjE\niOzu+KyaljUi4rjtZZI+XWlVAFClUZhJR8TuFu0nJN1XWUUAULVRCGkAGFmENAAkNkQh3bFS258v\n0wYAQ2OItuCVqeLmadr+fJbrAID+GaKQbrncYXulpJskXWZ7a8OlCyW9WHVhAFCZEdmC9z1JRyS9\nTdK/NrQfl7SnyqIAoFI9zpKLrcj/pvpqxIaI+GLT9WslPSTpmaLpvyLiH8vc26zdFrxDkg5J+sAM\nXwcA5NRDSNsek3SnpI9I+omknbYfiogDTV23R8Qfz/De10otUdBx278oHq/aPmX7F12+LgDIo7c1\n6aWSDkbEoeJzI5skLZ+m33TfcVT23jM6LsxExIVnRrRdPOE1ne4DgLR6W+6YJ+n5hvMXVA/fZh+w\nvUvSYUl/GxFPdXHvGV2tnkdESPqm7dsl3dbNvQCQRouQnpiY0MTExGyM8L+SxiPiFds3qP6dR++a\nyRN1DGnbn2w4HZP0fkmvzmSwdubMme1nxDB7wxsGXQFGWbT4tuVra9fp2tp1Z87v+MIXput2WNJ4\nw/nFRdtrzx/xy4bjR2x/xfZbytzbrMxM+hMNxyclTarDGgoAZHbyZE+375R0ue35qu+AWyFpZWMH\n23Mj4mhxvFSSI+JF2x3vbVZmTXr1jF4GACQ1NTXzeyPilO21krbptW10+22vqV+O9ZJutP1ZSSck\n/UrSZ9rd224815eZ23SwF0j6kupvFoak70v664h4pu2NXbAd3/kOvyOA17z00qArQEY33mhFRE+/\nDGU7XnmlXN6cf37v4/WqzFuc90v6T0m/J+n3JW2W9ECVRQFAlaamyj0yKBPS50fExog4WTzulcTb\nOgCG1jCFdJk3Dh+xfZvqm65D9bWVh4t3KhURfI8HgKGSJYDLKBPSp38qa01T+wrVQ3vBrFYEABUb\nqZCOiMv6UQgA9EuPW/D6qtQnDm1/UNKljf0j4p6KagKASo3UTNr2RkkLJe2SdKpoDkmENIChNFIh\nrfrHwBdFpw3VADAkRi2k90p6h+ofYQSAoTdqIf02SU/ZfkLSr083Nn+ZNQAMi1EL6XVVFwEA/TRS\nIR0R/92PQgCgX0ZiC57t46rv4njdJdW/6elNlVUFABUaiZl0489mAcAoGYmQBoBRRUgDQGKENAAk\nRkgDQGIjsbsDAEYVM2kASIyQBoDECGkASIyQBoDECGkASIyQBoDE2IIHAIkxkwaAxAhpAEiMkAaA\nxAhpAEiMkAaAxAhpAEiMLXgAkBgzaQBIjJAGgMQIaQBIjJAGgMQIaQBIjJAGgMTYggcAiTGTBoDE\nCGkASIyQBoDECGkASIyQBoDECGkASIwteACQ2DDNpMeqHsD2MtsHbD9t+9aqxwOATqamyj1a6ZRr\ntm+yvbt47LC9uOHaZNH+pO0nOtVa6Uza9pikOyV9RNJPJO20/VBEHKhyXABop5eZdMlce0bSH0XE\nz20vk7Re0jWnh5dUi4hjZcareia9VNLBiDgUESckbZK0vOIxAaCtHmfSHXMtIh6PiJ8Xp49Lmtdw\n2eoie6sO6XmSnm84f0FnFwsAfddjSHeba38p6ZGG85D0mO2dtm/pVGuaNw7vvnvdmeP3vKemJUtq\nA6sFQA57905o376JWX/eVrs7JicnNDk5e+PZvk7Sakkfbmj+UEQcsf121cN6f0TsaPUcVYf0YUnj\nDecXF22vc/PN6youBcCwufrqmq6+unbmfPPmO2bleVvNksfHaxoff2287dunHa9UrhVvFq6XtKxx\n/TkijhR/f2p7i+rLJy1Duurljp2SLrc93/YcSSskba14TABoq8fljo65Zntc0oOS/iwiftzQfr7t\nC4rjN0r6mKS97WqtdCYdEadsr5W0TfX/EDZExP4qxwSATnrZ3dEq12yvqV+O9ZL+QdJbJH3FtiWd\niIilkuZK2mI7VM/f+yJiW7vxKl+TjohHJV1R9TgAUFavH2aZLtci4msNx7dIet2bghHxrKQl3YyV\n5o1DAOiXYfrEISEN4JxDSANAYnzBEgAkxkwaABIjpAEgMUIaABIjpAEgMUIaABIjpAEgMbbgAUBi\nzKQBIDFCGgASI6QBIDFCGgASI6QBIDFCGgASYwseACTGTBoAEiOkASAxQhoAEiOkASAxQhoAEiOk\nASAxtuABQGLMpAEgMUIaABIjpAEgMUIaABIjpAEgMUIaABJjCx4AJMZMGgASI6QBIDFCGgASI6QB\nIDFCGgASY3cHACTGTBoAEiOkASAxQhoAEiOkASAxQhoAEiOkASAxtuABQGLMpAEgsWEK6bFBFwAA\n/TY1Ve7Riu1ltg/Yftr2rS36fNn2Qdu7bC/p5t5GhHQyu3ZNDLoEJLN378SgSxg5vYS07TFJd0q6\nXtJVklbavrKpzw2SFkbEOyWtkfTVsvc2I6ST2b17YtAlIJl9+yYGXcLI6XEmvVTSwYg4FBEnJG2S\ntLypz3JJ90hSRPxA0kW255a89yyENIBzTo8hPU/S8w3nLxRtZfqUufcsvHEI4JwzgC14numNaUL6\nox+d8WsYORs33jHoEpDM5s38m5hFhyTPL9n36DRthyWNN5xfXLQ197lkmj5zStx7lhQhHREkNIC+\niIhLe3yKnZIutz1f0hFJKyStbOqzVdLnJH3D9jWSXoqIo7Z/VuLes6QIaQAYFhFxyvZaSdtUf19v\nQ0Tst72mfjnWR8TDtj9u+0eSXpa0ut297cZzRFT6ggAAM8fujiS63eCO0Wd7g+2jtvcMuhYMDiGd\nwEw2uOOccJfq/yZwDiOkc+h6gztGX0TskHRs0HVgsAjpHLre4A7g3EBIA0BihHQOZTbHAzgHEdI5\nnNkcb3uO6hvctw64JuRg9fCRYgw/QjqBiDgl6fQG932SNnXa4I7RZ/t+Sd+T9C7bz9lePeia0H98\nmAUAEmMmDQCJEdIAkBghDQCJEdIAkBghDQCJEdIAkBghjUrYPl7Bc8633fJXLGw/YvuYbT4IhJFB\nSKMqVWzAv0zSTW2u/7OkVRWMCwwMIY1K2b7W9ndtb7a93/bGhmvP2v6i7T22H7e9oGi/y/YnG/qd\nnpX/k6QP2/6h7c83jxUR35X0y4pfEtBXhDT6YYmkv5K0SNJC2x9suHYsIhZL+ndJX2px/+lZ+W2S\n/ici3hsRrfoCI4WQRj88ERFHov4dBLskXdpwbVPx9wFJ1/S7MCA7Qhr98OuG41M6+1fqY5rjkyr+\nbdq2pDmVVgckRkijKmW/XvMzxd8Vkr5fHE9Ken9xvFzSbxfHxyVdWGJcvtoTI+O8zl2AGWm1u6O5\n/c22d0t6VdLp7XX/Iekh209K+rakl4v2PZKmivavN69L294u6QpJF9h+TtJfRMRjvb8UYHD4qlIM\njO1nJb0vIl4cdC1AVix3YJCYIQAdMJMGgMSYSQNAYoQ0ACRGSANAYoQ0ACRGSANAYoQ0ACT2/8bk\njhxt0kBmAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def layer_InputOutput(Inputs,Weights): #find a layers input and activity\n", " Inputs_with_bias = np.append(Inputs,1.0) #input 1 for each unit's bias\n", " return logistic(np.dot(Inputs_with_bias,Weights.T))\n", "\n", "def neural_net(Input,Weights_1,Weights_2,Training=False): #this function creates and runs the neural net \n", " \n", " target = 1 #set target value\n", " if np.array(Input[0])==np.array([Input[1]]): target = 0 #change target value if needed\n", " \n", " #forward pass\n", " Hidden_Units = layer_InputOutput(Input,Weights_1) #find hidden unit activity\n", " Output = layer_InputOutput(Hidden_Units,Weights_2) #find Output layer actiity\n", " \n", " return {'output':Output,'target':target,'input':Input} #record trial output\n", "\n", "Train_Set = [[1.0,1.0],[0.0,1.0],[1.0,0.0],[0.0,0.0]] #the four input patterns\n", "tempdict = {'output':[],'target':[],'input':[]} #data dictionary\n", "temp = [neural_net(Input,Weights_1,Weights_2) for Input in Train_Set] #get the data\n", "[tempdict[key].append([temp[x][key] for x in range(len(temp))]) for key in tempdict] #combine all the output dictionaries\n", "\n", "plotter = np.ones((2,2))*np.reshape(np.array(tempdict['output']),(2,2))\n", "plt.pcolor(plotter,vmin=0,vmax=1,cmap=plt.cm.bwr)\n", "plt.colorbar(ticks=[0,0.25,0.5,0.75,1]);\n", "plt.xlabel('Input 1')\n", "plt.ylabel('Input 2')\n", "plt.xticks([0.5,1.5], ['0','1'])\n", "plt.yticks([0.5,1.5], ['0','1']);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the plot above, I have Input 1 on the x-axis and Input 2 on the y-axis. So if the Input is [0,0], the network produces the activity depicted in the lower left square. If the Input is [1,0], the network produces the activity depicted in the lower right square. If the network produces an output of 0, then the square will be blue. If the network produces an output of 1, then the square will be red. As you can see, the network produces all output between 0.25 and 0.5... no where near the correct answers.\n", "\n", "So how do we update the weights in order to reduce the error between our guess and the correct answer?\n", "\n", "First, we will do backpropogation between the output and hidden layers. This is exactly the same as backpropogation in the previous post. \n", "\n", "In the previous post I described how our goal was to decrease error by changing the weights between units. This is the equation we used to describe changes in error with changes in the weights. The equation below expresses changes in error with changes to weights between the $Hidden_{1}$ and the Output unit. \n", "\n", "$\\frac{\\partial Error}{\\partial Weight_{Hidden_{1}\\to.Output}} = \\frac{\\partial Error}{\\partial Output} * \\frac{\\partial Output}{\\partial net_{Output}} * \\frac{\\partial net_{Output}}{\\partial Weight_{Hidden_{1}\\to.Output}}$\n", "\n", "$\n", "\\begin{multline}\n", "\\frac{\\partial Error}{\\partial Weight_{Hidden_{1}\\to.Output}} = -(target-Output) * Output(1-Output) * Hidden_{1} \\\\= -(1-0.34) * 0.34(1-0.34) * 0.22 = -0.03\n", "\\end{multline}\n", "$\n", "\n", "Now multiply this weight adjustment by the learning rate.\n", "\n", "$\\Delta Weight_{Input_{1}\\to.Output} = \\alpha * \\frac{\\partial Error}{\\partial Weight_{Input_{1}\\to.Output}}$\n", "\n", "Finally, we apply the weight adjustment to $Weight_{Hidden_{1}\\to.Output}$.\n", "\n", "$Weight_{Hidden_{1}\\to.Output}^{\\prime} = Weight_{Hidden_{1}\\to.Output} - 0.5 * -0.03 = -0.23 - 0.5 * -0.03 = -0.21$\n", "\n", "Now lets do the same thing, but for both the weights and in the code." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[-0.21252673 -0.96033892 -0.29229558]]\n" ] } ], "source": [ "alpha = 0.5 #learning rate\n", "target = 1 #target outpu\n", "\n", "error = target - Output #amount of error\n", "delta_out = np.atleast_2d(error*(Output*(1-Output))) #first two terms of error by weight derivative\n", "\n", "Hidden_Units = np.append(Hidden_Units,1.0) #add an input of 1 for the bias\n", "print Weights_2 + alpha*np.outer(delta_out,Hidden_Units) #apply weight change" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The hidden layer changes things when we do backpropogation. Above, we computed the new weights using the output unit's error. Now, we want to find how adjusting a weight changes the error, but this weight connects an input to the hidden layer rather than connecting to the output layer. This means we have to propogate the error backwards to the hidden layer. \n", "\n", "We will describe backpropogation for the line connecting $Input_{1}$ and $Hidden_{1}$ as \n", "\n", "$\\frac{\\partial Error}{\\partial Weight_{Input_{1}\\to.Hidden_{1}}} = \\frac{\\partial Error}{\\partial Hidden_{1}} * \\frac{\\partial Hidden_{1}}{\\partial net_{Hidden_{1}}} * \\frac{\\partial net_{Hidden_{1}}}{\\partial Weight_{Input_{1}\\to.Hidden_{1}}}$\n", "\n", "Pretty similar. We just replaced Output with $Hidden_{1}$. The interpretation (starting with the final term and moving left) is that changing the $Weight_{Input_{1}\\to.Hidden_{1}}$ changes $Hidden_{1}$'s input. Changing $Hidden_{1}$'s input changes $Hidden_{1}$'s activity. Changing $Hidden_{1}$'s activity changes the error. This last assertion (the first term) is where things get complicated. Lets take a closer look at this first term\n", "\n", "$\\frac{\\partial Error}{\\partial Hidden_{1}} = \\frac{\\partial Error}{\\partial net_{Output}} * \\frac{\\partial net_{Output}}{\\partial Hidden_{1}}$\n", "\n", "Changing $Hidden_{1}$'s activity changes changes the input to the Output unit. Changing the output unit's input changes the error. hmmmm still not quite there yet. Lets look at how changes to the output unit's input changes the error. \n", "\n", "$\\frac{\\partial Error}{\\partial net_{Output}} = \\frac{\\partial Error}{\\partial Output} * \\frac{\\partial Output}{\\partial net_{Output}}$\n", "\n", "You can probably see where this is going. Changing the output unit's input changes the output unit's activity. Changing the output unit's activity changes error. There we go.\n", "\n", "Okay, this got a bit heavy, but here comes some good news. Compare the two terms of the equation above to the first two terms of our original backpropogation equation. They're the same! Now lets look at $\\frac{\\partial net_{Output}}{\\partial Hidden_{1}}$ (the second term from the first equation after our new backpropogation equation).\n", "\n", "$\\frac{\\partial net_{Output}}{\\partial Hidden_{1}} = Weight_{Hidden_{1}\\to Output}$\n", "\n", "Again, I am glossing over how to derive these partial derivatives. For a more complete explantion, I recommend [Chapter 8 of Rumelhart and McClelland's PDP book](http://www-psych.stanford.edu/~jlm/papers/PDP/Volume%201/Chap8_PDP86.pdf). Nonetheless, this means we can take the output of our function *delta_output* multiplied by $Weight_{Hidden_{1}\\to Output}$ and we have the first term of our backpropogation equation! We want $Weight_{Hidden_{1}\\to Output}$ to be the weight used in the forward pass. Not the updated weight. \n", "\n", "The second two terms from our backpropogation equation are the same as in our original backpropogation equation.\n", "\n", "$\\frac{\\partial Hidden_{1}}{\\partial net_{Hidden_{1}}} = Hidden_{1}(1-Hidden_{1})$ - this is specific to logistic activation functions.\n", "\n", "and\n", "\n", "$\\frac{\\partial net_{Hidden_{1}}}{\\partial Weight_{1}} = Input_{1}$\n", "\n", "Lets try and write this out. \n", "\n", "$\\frac{\\partial Error}{\\partial Weight_{Input_{1}\\to.Hidden_{1}}} = -(target-Output) * Output(1-Output) * Weight_{Hidden_{1}\\to Output} * Hidden_{1}(1-Hidden_{1}) * Input_{1}$\n", "\n", "It's not short, but its doable. Let's plug in the numbers.\n", "\n", "$\\frac{\\partial Error}{\\partial Weight_{Input_{1}\\to.Hidden_{1}}} = -(1-0.34)*0.34(1-0.34)*-0.23*0.22(1-0.22)*0 = 0$\n", "\n", "Not too bad. Now lets see the code." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[-0.25119612 -0.50149299 -0.77809147]\n", " [-0.80193714 -0.23946929 -0.84467792]]\n" ] } ], "source": [ "delta_hidden = delta_out.dot(Weights_2)*(Hidden_Units*(1-Hidden_Units)) #find delta portion of weight update\n", " \n", "delta_hidden = np.delete(delta_hidden,2) #remove the bias input\n", "print Weights_1 + alpha*np.outer(delta_hidden,np.append(Input,1.0)) #append bias input and multiply input by delta portion " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Alright! Lets implement all of this into a single model and train the model on the XOR problem. Below I create a neural network that includes both a forward pass and an optional backpropogation pass." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def neural_net(Input,Weights_1,Weights_2,Training=False): #this function creates and runs the neural net \n", " \n", " target = 1 #set target value\n", " if np.array(Input[0])==np.array([Input[1]]): target = 0 #change target value if needed\n", " \n", " #forward pass\n", " Hidden_Units = layer_InputOutput(Input,Weights_1) #find hidden unit activity\n", " Output = layer_InputOutput(Hidden_Units,Weights_2) #find Output layer actiity\n", " \n", " if Training == True:\n", " alpha = 0.5 #learning rate\n", " \n", " Weights_2 = np.atleast_2d(Weights_2) #make sure this weight vector is 2d.\n", " \n", " error = target - Output #error\n", " delta_out = np.atleast_2d(error*(Output*(1-Output))) #delta between output and hidden\n", " \n", " Hidden_Units = np.append(Hidden_Units,1.0) #append an input for the bias\n", " delta_hidden = delta_out.dot(np.atleast_2d(Weights_2))*(Hidden_Units*(1-Hidden_Units)) #delta between hidden and input\n", " \n", " Weights_2 += alpha*np.outer(delta_out,Hidden_Units) #update weights\n", " \n", " delta_hidden = np.delete(delta_hidden,2) #remove bias activity\n", " Weights_1 += alpha*np.outer(delta_hidden,np.append(Input,1.0)) #update weights\n", " \n", " if Training == False: \n", " return {'output':Output,'target':target,'input':Input} #record trial output\n", " elif Training == True:\n", " return {'Weights_1':Weights_1,'Weights_2':Weights_2,'target':target,'output':Output,'error':error}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Okay, thats the network. Below, I train the network until its answers are very close to the correct answer. " ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [], "source": [ "from random import choice\n", "np.random.seed(seed=10) #seed random number generator for reproducibility\n", "\n", "Weights_2 = np.random.rand(1,3)-0.5*2 #connections between hidden and output\n", "Weights_1 = np.random.rand(2,3)-0.5*2 #connections between input and hidden\n", " \n", "Weight_Dict = {'Weights_1':Weights_1,'Weights_2':Weights_2}\n", "\n", "Train_Set = [[1.0,1.0],[0.0,0.0],[0.0,1.0],[1.0,0.0]] #train set\n", "\n", "Error = []\n", "while True: #train the neural net\n", " Train_Dict = neural_net(choice(Train_Set),Weight_Dict['Weights_1'],Weight_Dict['Weights_2'],Training=True)\n", " \n", " Error.append(abs(Train_Dict['error']))\n", " if len(Error) > 6 and np.mean(Error[-10:]) < 0.025: break #tell the code to stop iterating when recent mean error is small" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lets see how error changed across training" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUFOW57/HvA0jEG9GoGEERJUrUbRS3iBGTUU90NBrc\nJiokOx412bA8ITuJMYKJ2Y4nbi9JTIyaqHjXlWwSo1E8RMWo410hioA6CHjhIuAdRQUZhuf88VY7\nNT09Mz09XV1dPb/PWr26qvrtqqeLoZ+uem/m7oiIiHRXn7QDEBGRbFICERGRkiiBiIhISZRARESk\nJEogIiJSEiUQEREpSeIJxMzqzWyBmS00s8kFXt/KzKab2bNmNt/MTkk6JhER6TlLsh+ImfUBFgKH\nAyuA2cA4d18QK3M2sJW7n21m2wIvAoPcfUNigYmISI8lfQUyCljk7kvcvRmYBozNK+PAltHylsDb\nSh4iItUv6QQyGFgWW18ebYu7AtjTzFYAc4EfJByTiIiUQTVUoh8JzHH3HYH9gN+b2RYpxyQiIl3o\nl/D+XwN2jq0PibbFnQpcCODuL5nZK8AI4J/xQmamQbtERErg7pbEfpO+ApkNDDezoWbWHxgHTM8r\nswT4XwBmNgjYHXi50M7cXY8yPc4999zUY6ilh86nzmW1PpKU6BWIu7eY2SRgJiFZXefuTWY2Mbzs\nU4HzgRvNbF70trPc/Z0k4xIRkZ5L+hYW7n4PsEfetqtjyysJ9SAiIpIh1VCJLimoq6tLO4SaovNZ\nPjqX2ZFoR8JyMjPPSqwiItXCzPCMVqKLiEiNUgKpsBdfTDsCEZHyUAKpgJ12gqujZgMjRsDSpenG\nIyJSDkogFbB8Odx/f+v6b34D69alF4+ISDkogVRIczPk2gD87ndwyimphiMi0mNKIBXS3Ay//W3r\n+mv5A7qIiGSMEkiFNDe3rUBXi2QRyTolkAp57LG26xs3phOHiEi5KIFUyIcftl1ftSqdOEREykUJ\nJCWvvJJ2BCIiPaMEIiIiJVECSdDKlWlHICKSHCWQBO24I7yjmU1EpEYpgSRMCUREapUSSMIWL25d\nfvfd9OIQESk3JZCEWWwU/ltvTS8OEZFyUwIREZGSJJ5AzKzezBaY2UIzm1zg9TPNbI6ZPWNm881s\ng5l9Oum4krZsWXi2ROYBExFJX6IJxMz6AFcARwJ7AePNbES8jLv/2t33c/eRwNlAo7uvTjKuSth5\n57QjEBFJVtJXIKOARe6+xN2bgWnA2E7Kjwf+J+GYuvTBB/Dxx6W//+23W5d7sh8RkWqWdAIZDCyL\nrS+PtrVjZgOAeuC2hGPq0pAhcNJJpb8/PlDiLbf0PB4RkWpUTZXoxwKPVsPtq/feg7lzC7/Wrx+s\nWNHxey+5pO17NWy7iNSqfgnv/zUgXhswJNpWyDi6uH3V0NDwyXJdXR11dXU9i66Alpbw3NGtp5aW\nMBDijju2bjvssDBlrRmceSZst13ra0ogIlJJjY2NNDY2VuRY5gl+w5lZX+BF4HBgJTALGO/uTXnl\nBgIvA0PcfW0H+/IkY81ZvRq23ho+8xl4661CccCjj8LBB7fd1tICffq0b3V1/PFw++2Fj6XkIiJJ\nMzPcPZH2oInewnL3FmASMBN4Hpjm7k1mNtHMJsSKHgfc21HyqKQ+0RlZv771aqQYDz6YTDwiItUq\n6VtYuPs9wB55267OW78JuCnpWIqRSyBr1oT6jvhVwvr1Hb9vzRpYWyD96SpDRGpVNVWip8os1G30\n6eSMvPlmx6/dfTesW1f+uEREqpUSSMzSpTBvXtflCg2KOHVq4V7nugIRkVrVKxLIunXFf5EvXdp1\nmZs6uNlWKIE8+WRxxxURyZpekUAGDIC//KV8+/vgg9bleGL61a/al121qvN93X9/aPElIpI1vSKB\nALz6atv1J55oX6azq5S1a6G+vvNj/Pd/dzssHn9ck06JSDb1mgQye3bb9S9+sf0X9/nnd/z+5cvh\nuefabmtqKlxWRKQ3yFQC6UmF9Hvvdb2/++9vvy03bEm8fuOjj+CPf4Q99yw9Hgi93TtrGiwiUs0S\n7weSBeec07o8Y0bb13bbLYyu+/77rdsefjg8emrUqOJafYmIVKNenUBWrAgV2PG6i9V5QzmuWweb\nb57M8ZU8RCTLMnULqycK3f5avrz9Ns0gKCJSnEwlkHJ3ytu4MfQ+j1uzprzHKFZzs5KXiGRLphJI\nKXJJp1DycW8/4VN3B0UsV1LrzsCNIiLVoOYSyLp1MGtW63p+nUace9vZA9P0m9+kHYGISPfUXAL5\n/e/hwAPhH/9oO2dHIU8/XZmYinHRReFZY2eJSFZkKoEU8+Wa61cxY0bo5d2Z887reb1DuW9hVcsV\nkYhIVzKVQOJaWtoPTxJ36aXtyxdSbb/4lUBEJCsylUDiVxQ//zkMG9b1e3KJ46GHkompX6/uSSMi\nvVmmEkh8QqcLLwzPEya0LhcyZ06yMYmI9FaZSiCFbu9ccw1cfHHrEOuLF7d9fcOGzvd53nnlia2n\nPvoo7QhERLon8QRiZvVmtsDMFprZ5A7K1JnZHDN7zsw67InRUX3Fe+/BmDFh+frr2772zW+2Li9Z\n0r3Y0/Dcc3D77WlHISLSNfMEa5HNrA+wEDgcWAHMBsa5+4JYmYHA48AR7v6amW3r7m8V2JdPnOj8\n/OcweHD71lP9+4fRbbtqVZX7uNXa6/ugg8JcJdVWuS8i2WRmuHsi33hJX4GMAha5+xJ3bwamAWPz\nynwTuM3dXwMolDxyrr4ahgwJo+PmK7b10hZbFFdOREQ6l3QCGQwsi60vj7bF7Q5sY2YPmtlsM/t2\nVztdtKj9to0b4Utf6jqgDz/suoyIiHStGhqh9gNGAocBmwNPmNkT7r64fdEGAK69FqAuegQbN8Ij\njxR3wK4q1tOUhXoaEalejY2NNDY2VuRYSdeBjAYa3L0+Wp8CuLtfHCszGdjU3c+L1q8F7nb32/L2\n5RBivfNOGJt/I6wbbr4ZTj659PdXgupARKQcslwHMhsYbmZDzaw/MA6YnlfmTmCMmfU1s82AA4FO\nZxvvSfKAMOBitcufw11EpNokmkDcvQWYBMwEngemuXuTmU00swlRmQXAvcA84Elgqru/kGRcWbhN\nVI4pc0VEkpToLaxyit/C6g2mTOm8h72ISDGSvIWlBFLFMvJPIyJVLMt1ICIiUqOUQEREpCRKICIi\nUhIlEBERKYkSiIiIlEQJRERESqIEIiIiJVECERGRkiiBiIhISZRAqtjLL6cdgYhIx5RAqtgLiQ4p\nKSLSM0ogVaylJe0IREQ6pgRSxXLzvLvDxx+nG4uISD4lkCqWSyA33wybbppuLCIi+ZRAqlgugSwu\nMDu8iEjalECq2Iknwtq1MHdu2pGIiLSnBFLlHnkE7rorLE+YkG4sIiJxSiBVLncbC+CWW9KLQ0Qk\nX+IJxMzqzWyBmS00s8kFXv+yma02s2eixzlJx5Ql8QSybl16cYiI5OuX5M7NrA9wBXA4sAKYbWZ3\nuvuCvKIPu/vXkowlq/L7griDJTK7sYhI9yR9BTIKWOTuS9y9GZgGjC1QTl+JHYhfgQD85S/pxCEi\nki/pBDIYWBZbXx5ty3eQmT1rZjPMbM+EY8qU445ru67xsUSkWiR6C6tITwM7u/tHZnYUcAewe+Gi\nDbHluughIiI5jY2NNDY2VuRY5u7J7dxsNNDg7vXR+hTA3f3iTt7zCrC/u7+Tt90huViz4oIL4Oyz\n045CRLLCzHD3RKoJkr6FNRsYbmZDzaw/MA6YHi9gZoNiy6MISe0dRESkqiV6C8vdW8xsEjCTkKyu\nc/cmM5sYXvapwDfM7HSgGVgLnJRkTCIiUh6J14G4+z3AHnnbro4t/x74fdJxiIhIeaknuoiIlEQJ\nRERESqIEkjEJNpoTEekWJRARESmJEoiIiJRECUREREqiBCIiIiVRAhERkZIogYiISEmUQDJm5cq0\nIxARCRIdjbecNBpvq4z8k4lIFcjyaLwiIlKjlEBERKQkXSYQM+trZr+uRDDFOuCAtCMQEZEuE4i7\ntwBjKhBL0b7ylbQjEBGRoirRzexKYDBwK/Bhbru7355caO1i+KQS/dFHYUxVpbTKUiW6iBQryUr0\nYieU2hR4Gzgsts2BiiWQnB/+EAYN6rqciIgkq6gE4u6nJh1Id1giuVRERLqjqFZYZjbEzP5mZm9E\nj9vMbEjSwYmISPUqthnvDcB0YMfocVe0rUtmVm9mC8xsoZlN7qTcAWbWbGbHd7XPgQOLjFpERBJT\nbALZzt1vcPcN0eNGYLuu3mRmfYArgCOBvYDxZjaig3IXAfd2tU932Hbbttt2311DfIiIVFqxCeRt\nM/v3qE9IXzP7d0KleldGAYvcfYm7NwPTgLEFyn0f+CvwRlc7LNQCyR122KGIaEREpGyKTSCnAScC\nq4CVwDeAYirWBwPLYuvLo22fMLMdgePc/UqgpOpxNWsVEam8LlthmVlf4Hh3/1pCMVwKxOtGOkki\nDTz5JDQ0ANRFD9hkk/YlDzwQnnqqTBGKiGREY2MjjY2NFTlWsR0JZ7n7qG7v3Gw00ODu9dH6FMDd\n/eJYmZdzi8C2hI6KE9x9et6+HJzvfx8uu6y1Ke9zz8FWW8FOO7Vt3lvLCURXXCJSrGroSPiYmV0B\n/Jm2PdGf6eJ9s4HhZjaUcOtrHDA+XsDdd80tm9kNwF35yaMze+1VePsxx9RuAhERqQbF1oHsS2hF\n9X+BS6JHlwMsRuNoTQJmAs8D09y9ycwmmtmEQm/pep/FBXz00fD661BfX1z5LHm7mOYLIiIJK2Y0\n3j7Ale5+aN7jsK7eC+Du97j7Hu7+OXe/KNp2tbtPLVD2tK7G19pii2KOGmy/PVx/fVj+0Y86Lndv\nl42Hq8sdd6QdgYhIcaPxbgTOqkAsXfrxj+G//qu4srkrlc9+Njz3799x2SOO6FlclfbOO/DSS2lH\nISK9XbG3sP5hZmea2U5mtk3ukWhkBey/PwwYUNp7W1rKG0uazjoLhg9POwoR6e2KrUQ/KXr+Xmyb\nA7sWKJuYoUOLL9sv75Nt3FjeWEREertiR+MdlnQg5bbPPm3Xa+kKRESkGnR6C8vMzootn5D32gVJ\nBdWR/BZY48e3XV++vHU5f8j3DRuSiUlEpLfqqg5kXGz57LzXUm8ge/rpbdcHDy5cDtpfgUwvuqeJ\niIgU0lUCsQ6WC61X1HXXhd7mxdp557brxx5b3nhERHqbrupAvIPlQusVddpp3SufPwS8iIj0TFdX\nIF8ws/fNbA2wT7ScW/+XCsTXxmabdV1m4sT22+bPh1OralLe8jn7bDjggLSjEJHeqNMrEHfvW6lA\nirHffl2X2a7ANFd77912/cIL25d55BE45JDS4krTRRelHYGI9FbFdiSsKVOmtN82Zkzl4xARybKa\nSyD5zXfjdq1ot0cRkdpWcwmkM6UOg1KtDj447QhEpDfrVQmks6uTLHr88bQjEJHerOYSSK0lCRGR\nalVzCSQ3fHshSi4iIuVTcwlkwgR44420oxARqX01l0D69CncFwR0BSIiUk7FzgdSE447DrbaKu0o\nRERqQ+JXIGZWb2YLzGyhmU0u8PrXzGyumc0xs1lmlljj1PPOCz3Oa9GqVeEK65ln0o5ERHqLRBOI\nmfUBrgCOBPYCxpvZiLxi/3D3L7j7fsB3gGuTjKlWTY5S87Jl6cYhIr1H0lcgo4BF7r7E3ZuBacDY\neAF3/yi2ugWgyWdLcPPNaUcgIr1N0glkMBD/Tbw82taGmR1nZk3AXUA3B2ovXWcTUGVV/qyNIiJJ\nqYpKdHe/A7jDzMYA5wNfKVSuoaHhk+W6ujrq6up6dNw+HaTPr34VZszo0a5FRFLR2NhIY2NjRY5l\nnuBPVjMbDTS4e320PgVwd7+4k/e8BBzg7u/kbfdyx3rOOdDUBLfdljtGeJ46NfQnyaK//S20NhMR\nATAz3D2RTgxJX4HMBoab2VBgJWGO9fHxAma2m7u/FC2PBPrnJ4+knH9+4e1Zvg20YkXaEYhIb5Fo\nAnH3FjObBMwk1Ldc5+5NZjYxvOxTga+b2cnAemAtcGKSMdW6q64Kc8WPHKmOkyKSrMTrQNz9HmCP\nvG1Xx5Z/Cfwy6Th6C3f413+FWbM01a2IJKvmhjKRoLk57QhEpNYpgRSQ5TqQnFr4DCJS3ZRAaszG\nqBumEoiIJE0JpMa89lraEYhIb6EEUmPeey886wpERJKmBFJALXz5vv8+rF+fdhQiUsuUQGrUMcfA\nd7+bdhQiUsuUQAqohSsQgJdeSjsCEallSiAxuZ7btTJrYa0kQhGpTkogMd/7XngeOTLdOEREskAJ\npIY98UTaEYhILVMCKWCXXdKOQESk+imBFDBgQNoRiIhUPyUQEREpSaIzEpZTEjMS5nvzTfjnP+Go\no2pnLo2M/POKSEKSnJFQCaTD41XsUInKyD+viCQkyQSiW1id+MUv0o5ARKR66QqkAy+/DMOGQZ+M\np9iVK2GHHdKOQkTSoltYVD6BtB634ocsqxNOgOXL4fHH045ERNKQ6QRiZvXApYTbZde5+8V5r38T\nmBytrgFOd/f5BfajBNIDGfmdICJlltkEYmZ9gIXA4cAKYDYwzt0XxMqMBprc/b0o2TS4++gC+1IC\n6QElEJHeKcuV6KOARe6+xN2bgWnA2HgBd3/S3aNpkHgSGJxwTD12++1pRyAikr6kE8hgYFlsfTmd\nJ4jvAncnGlEZbL112hF03403woMPph2FiNSSfmkHkGNmhwKnAmM6KtPQ0PDJcl1dHXV1dYnHtXo1\nTJ4MV1/duu2gg+DKK+H00xM/fNmcemp41q0skdrW2NhIY2NjRY6VdB3IaEKdRn20PgXwAhXp+wC3\nAfXuXnAapLTqQHI22wzWrg3LuTCyWD+iBCLSu2S5DmQ2MNzMhppZf2AcMD1ewMx2JiSPb3eUPKrB\n4sVw8slpRyEiUj0STSDu3gJMAmYCzwPT3L3JzCaa2YSo2M+BbYA/mNkcM5uVZEyl2nFHGDUq7ShE\nRKqHOhJ2w8aN8OGHsOWWuZhSDackGfnnFpEyyWw/kHKqhgSSTwlERKpdlutARESkRimB9DIVat0n\nIr2AEkgvc+ihcPfdsGED/Nu/ZfM2nIhUB9WB9ED+l+8RR8DMmenE0l2bbx4aBAB89JHmgRepVapE\nJxsJxL112y67wKuvVjqi0mjOEJHapUr0Kjc2Njzk8ceH54ceSieWUlx1VdoRiEgWKYH00IAB8J3v\ntN/et2/lYynVeeelHYGIZJESSA/tv3/h7f2qZpjK7nvssdZxv0REOpLhr7nqscsu7bdldS71Z5+F\nMWNC/C0taUcjItVMCaQMCiWQbbeteBg9Yga33hqa+EIYtkVEpDNKIAnJYv+KE05ou/7SS7DbbunE\nIiLVL6M3WqQSvvSltCMQkWqmBNIDu+wSvmQHDIDhw8O2rNZ9FLJiRdoRiEg1U0fCMnv9dVi4EA45\npPU21sEHh1ZZDz0UOuytWlXcvrbZBt55B/74R/jWt5KLuTPxU/6nP4XP96MfpROLiHSfOhJmyKBB\nIXnEHXVU63JubvJiPPVUeB48uOdx9YRZeHzrW3DGGSGRiIgogVTAZz7TunzsscW/b9iw0Elx333h\n0kvLH1cx3n+//ba0roZEpLoogVTAsGGtt4JGjuy6/NChsG5d6M1+7bUwcCB84QvJxtiRgQM7fu3y\ny6GpCe67D+65p3IxiUh1UB1IgiZMgGuuCR3y6uvDF+26dbDppp2/b++9Yf789turqWlwS0v74Vqa\nmmDEiHTiEZHCMl0HYmb1ZrbAzBaa2eQCr+9hZo+b2TozOyPpeCpp6lSYMSO0zMrd9ikmB3aUKH74\nw/LF1lOFxvr6/OcrH4eIpCfRBGJmfYArgCOBvYDxZpb/G/Vt4PvAr5KMJS1HHx2eP/3p8NzV1QdA\n//6Ft//2t+WJSUSkHJK+AhkFLHL3Je7eDEwDxsYLuPtb7v40sCHhWFK15ZbFl+0ogeQsXdp2fdq0\ntuvXXw8PP1z88crp9dfTOa6IVF7SQ5kMBpbF1pcTkkqvc+ih8MorYXnbbeGttzoue8opHb+WP3vg\nmjWtLaU23TTUsXSnqXC57bAD/PrXqlgX6Q0yNRZWQ0PDJ8t1dXXU1dWlFkt3mbUOujh1auvEU4Xs\nvHPHr+WSx4cfwgMPwBZbhId7SCTLlnX83ko588zwbAZXXgkHHNDxsPciUl6NjY00NjZW5FiJtsIy\ns9FAg7vXR+tTAHf3iwuUPRdY4+6/6WBfmWuF1RH31iFPNmxoP3fI3XeHVls9VU2ttjZuDFdPzc2t\n9UEikrwkW2ElfQUyGxhuZkOBlcA4YHwn5avoKy858S/2vn3bzqVeqwqNEfbxx6G+Z+PG2hpDTKS3\nSPS/rbu3AJOAmcDzwDR3bzKziWY2AcDMBpnZMuBHwM/MbKmZbZFkXNWuXBdaozqobfrud1uXO+so\nmLRPfSokzr594Re/SC8OESmNOhKmZOpUmDixNVmYwfbbwxtvwOrV5fliX7cu3A7L1bdcdlmYbXC/\n/VrLrFkDW23V82OVw+OPw0EHpR2FSG1J8haWEkhKcvUg8QSy445hCPUPPoDNNy/fsXK3xzo7fT/5\nSWg9lbYvfhG+8Y3QadIsDAOz//7w17+GZNed5tAiogQC1F4CyWcGe+4Zmvtedll56wSefjokps4G\ncmxpgXffDU2MzzoLflXl3TqfeabtlZSIFKYEQu0nkKVLQz+O7bdPO5Lg0EOhQi0BS/azn8HcuXDX\nXSHhPfVU6K2/005pRyZSPZRAqP0EUo2y2jJs1iwYMgQ++1m49VY48UR4+eVwO0ykt8n0YIqSXe6h\nw6J7mBnRHe6/v/X1JUtCj/qbb04vxkJGjQr1SWYheQDsumvrxFi5x9ChrctTpsC3vx2Wf/rTdOMX\nyQpdgUjZbNgAm2wSlpuasj867403hvlYbropJCCRLNItLJRAsiyrt8JKNWNGmMa4t31uqU66hSWZ\ntnEjPPpo2lFUzle/GlrR5W6PbbIJnHsuLF4cbgWuXZt2hCLloSsQqagHHoDDDw9JJfcLff36UB+x\nalW6sVW7gw4KvfYvuCDU8zz/fLhNGB+dWSSfbmGhBNLbrF8ffrnnOlzmBp1cuTL0l1m9Ou0Isy0+\npcAVV4QGBFtu2fFtt3iHV8kWJRCUQKSwjRvDo1+/8Fxoql2prMsvD4n+ggtC67ZTToE99kg7qt5L\nCQQlECmee+hUeMghYT6S+OCRxfj730M9htSG006DY46B0aNh0KBwFZUbATt3ZdvZlVXWR4tWAkEJ\nRKrXxx+Hx1Zbwfz5sM8+sPfe8NxzaUcmWfCpT4UE16dPmBDujDNCQrvkEjjnnDAqxGablb5/JRCU\nQKR2vfEGXHVV6JD50kvw9a+HaYlnzQrTAs+alXaEklXhSksJRAlEpEjuYSj/5uZQMb5yZfh1u3hx\naEJ8333VP1imlIcSSEQJRCS7WlrgzTdDi69Bg2D48JDMXnghJLljj4XBg+Htt+Hhh8MgmZtsAg89\nFNalNEogESUQEUlb/vTTH30Uphbo2zc0PTeDG24Ic9hst12o31i2LCTNefNCXcaee4b6sb59wyCf\n224LL74YKuvvvLN8sV5+OUyapAQCKIGIiJQi00OZmFm9mS0ws4VmNrmDMpeZ2SIze9bM9k06JhER\n6blEE4iZ9QGuAI4E9gLGm9mIvDJHAbu5++eAicBVScYkQWO1zxaVMTqf5aNzmR1JX4GMAha5+xJ3\nbwamAWPzyowFbgZw96eAgWY2KOG4ej39Jy0vnc/y0bnMjqQTyGBgWWx9ebStszKvFSgjIiJVJsMd\n9EVEJE2JtsIys9FAg7vXR+tTAHf3i2NlrgIedPc/R+sLgC+7++t5+1ITLBGREiTVCqtfEjuNmQ0M\nN7OhwEpgHDA+r8x04HvAn6OEszo/eUByJ0BEREqTaAJx9xYzmwTMJNwuu87dm8xsYnjZp7r7383s\naDNbDHwInJpkTCIiUh6Z6UgoIiLVJROV6MV0RhQws1fNbK6ZzTGzWdG2rc1sppm9aGb3mtnAWPmz\now6cTWZ2RGz7SDObF53vS9P4LGkws+vM7HUzmxfbVrbzZ2b9zWxa9J4nzGznyn26yurgXJ5rZsvN\n7JnoUR97TeeyE2Y2xMweMLPnzWy+mf1ntD3dv093r+oHIcktBoYCmwDPAiPSjqsaH8DLwNZ52y4G\nzoqWJwMXRct7AnMItzF3ic5x7or0KeCAaPnvwJFpf7YKnb8xwL7AvCTOH3A68Ido+SRgWtqfucLn\n8lzgjAJlP69z2eX53AHYN1reAngRGJH232cWrkCK6YwogdH+qnIscFO0fBNwXLT8NcIfyAZ3fxVY\nBIwysx2ALd19dlTu5th7apq7Pwq8m7e5nOcvvq+/AoeX/UNUiQ7OJYS/0Xxj0bnslLuvcvdno+UP\ngCZgCCn/fWYhgRTTGVECB+4zs9lmlpvIdZBHrdrcfRWwfbS9ow6cgwnnOKe3n+/ty3j+PnmPu7cA\nq81sm+RCr0qTojHvro3dbtG57AYz24Vwdfck5f3/3e1zmoUEIsU72N1HAkcD3zOzQwhJJU6tJnqm\nnOevtzVN/wOwq7vvC6wCLinjvnvFuTSzLQhXBz+IrkSS/P/d5TnNQgJ5DYhX5gyJtkked18ZPb8J\n3EG4/fd6bmyx6PL1jaj4a8BOsbfnzmtH23urcp6/T14zs77AVu7+TnKhVxd3f9OjG+zANYS/T9C5\nLIqZ9SMkj1vcPTdzSKp/n1lIIJ90RjSz/oTOiNNTjqnqmNlm0a8TzGxz4AhgPuFcnRIV+99A7g9v\nOjAuankxDBgOzIoug98zs1FmZsDJsff0BkbbX17lPH/To30AnAA8kNinqA5tzmX0BZdzPPBctKxz\nWZzrgRfc/Xexben+fabduqDIFgj1hFYHi4ApacdTjQ9gGKGF2hxC4pgSbd8G+Ed0/mYCn46952xC\n64wm4IjY9v2jfSwCfpf2Z6vgOfwTsAL4GFhK6NS6dbnOH/Ap4C/R9ieBXdL+zBU+lzcD86K/0zsI\n9+91LoutZkEqAAACu0lEQVQ7nwcDLbH/489E34tl+/9dyjlVR0IRESlJFm5hiYhIFVICERGRkiiB\niIhISZRARESkJEogIiJSEiUQEREpiRKI1DQzWxM9DzWz/Nkwe7rvs/PWHy3z/nc3sxsseLyc+xYp\nByUQqXW5jk7DgG92543RcA6d+WmbA7mP6c7+i3AI8DDwL4SOXyJVRQlEeosLgTHRREY/MLM+ZvZL\nM3sqGh32PwDM7Mtm9rCZ3Qk8H237WzTC8fzcKMdmdiEwINrfLdG2NbmDmdmvovJzzezE2L4fNLNb\no0l+bikUqJmNMbM5wC+BM4EZwJEWTRImUi3UE11qmpm97+5bmdmXgR+7+9ei7f8BbOfuF0RjrD0G\nfIMw+c7/A/Zy96VR2U+7+2oz25QwNtuX3P3d3L4LHOvrwAR3P9LMto/eM4owAdAdhMl+VkXHPNPd\nC96eMrPH3P1gM7se+KW7Lyj/GRIpna5ApLc6Ajg5+qX/FGFMoc9Fr83KJY/ID83sWcL4QENi5Tpy\nMPA/AO7+BtAIHBDb90oPv9yeJSSsdsxsM8I4UkTHW1T0JxOpkH5pByCSEgO+7+73tdkYrlQ+zFs/\nDDjQ3T82sweBTWP7KPZYOR/Hllso8H8wun02AhhoZnMJ0znPNrML3f3WIo8pkjhdgUity315rwG2\njG2/F/g/0RwLmNnnol/9+QYC70bJYwQwOvba+tz78471CHBSVM+yHaEyvOj6C3cfS5gv43TgP4Gr\n3H2kkodUGyUQqXW5Sr55wEYzm2NmP3D3a4AXgGfMbD5wFVCo1dU9wCZm9jxwAfBE7LWpwLxYZbgD\nuPvfouPNJQy1/ZPoVlZHsRVyCPBo9PxQ1x9TpPJUiS4iIiXRFYiIiJRECUREREqiBCIiIiVRAhER\nkZIogYiISEmUQEREpCRKICIiUhIlEBERKcn/B7UlExjUJoWEAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "Error_vec = np.array(Error)[:,0]\n", "plt.plot(Error_vec)\n", "plt.ylabel('Error')\n", "plt.xlabel('Iteration #');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Really cool. The network start with volatile error - sometimes being nearly correct ans sometimes being completely incorrect. Then After about 5000 iterations, the network starts down the slow path of perfecting an answer scheme. Below, I create a plot depicting the networks' activity for the different input patterns. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWkAAAEPCAYAAACA1jUJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEHJJREFUeJzt3X+MXFd5xvHnWW/sJNgkBJBpnawTO0BEpCgFaqWA6o2Q\niINEXVAFdtSqtVpkIayiqi1JK1WxUaVC1EoFpQi5soKSJjGgNMSqEnD+YJtaEGKVJG7AVgzJOo6x\nLFAc2M0Psz/e/jHXzni9M3PHs3fmndnvRxrt3HPPnfOOtHl8cubcWUeEAAA5DfW6AABAY4Q0ACRG\nSANAYoQ0ACRGSANAYoQ0ACRGSANAG2zvsn3C9oEmfb5i+7Dtp2xfX9e+wfYh28/avrXMeIQ0ALTn\nLkk3NTpp+2ZJayPinZK2Svpa0T4k6c7i2mslbbZ9TavBCGkAaENE7JN0skmXjZLuLvr+UNIltldK\nWifpcEQciYgpSbuLvk0R0gCwsFZJOlp3/GLR1qi9KUIaAKrlTi4eXqgqOmGbLxABUEpEdBR6V9px\npHz3ExHxjjaHOCbpirrjy4u2pZJG5mlvKkVIS1IsX97rElLYfuqUti9b1usyem6FJnpdQhqnTm3X\nsmXbe11GCpOTHeWzJOmIpBgqt4jg2dmVjU6p8Qx5j6TPSvqG7RskvRwRJ2z/UtLVtldLOi5pk6TN\nrWpIE9IA0DUlQ1qzs+c02b5P0qikt9p+QdLtqs2SIyJ2RsTDtj9q+6eSXpG0RbWTM7a3Sdqr2lLz\nrog42KoEQhrA4lM2pOcREbeU6LOtQft3JL27nfEI6WRGlyzpdQlIZsmS0V6XMHiGS0bfb35TbR0l\nOMOX/tsO1qRRjzVpzGdy0h1/cNhO3nhysuPxOsVMGsDi08FyR7cR0gAWH0IaABIjpAEgMUIaABIj\npAEgsbJb8BLon0oBYKEwkwaAxAhpAEiMkAaAxAhpAEiMkAaAxAhpAEiMLXgAkBgzaQBIjJAGgMQI\naQBIjJAGgMQIaQBIjJAGgMTYggcAiTGTBoDECGkASIyQBoDECGkASIyQBoDE2N0BAIkxkwaAxAhp\nAEiMkAaAxAhpAEiMkAaAxAhpAEiMLXgAkBgzaQBIjJAGgMQIaQBIjJAGgMQIaQBIjJAGgMTYggcA\niTGTBoDECGkASIyQBoDECGkASKyPQrrSSm3vsn3C9oEqxwGAtgwNlXs0YHuD7UO2n7V96zzn/8b2\nk7Z/ZPv/bE/bvrQ4N2776eL8Ey1L7eiNtnaXpJsqHgMA2jM8XO4xD9tDku5ULduulbTZ9jX1fSLi\nnyPidyLivZL+TtJYRLxcnJ6VNFqcX9eq1EpDOiL2STpZ5RgA0LbOZtLrJB2OiCMRMSVpt6SNTUbb\nLOn+umOrjeztn4UZAFgonYX0KklH645fLNrOYfsiSRskPVDXHJIetb3f9qdblZrmg8Ptp06deT66\nZIlG++iOIADVmJ4e08zM2MK/cIMAHhsf19j4+EKO9DFJ++qWOiTpgxFx3PbbVQvrg8Wqw7zSJOH2\nZct6XQKAZIaHRzU8PHrmeGpqx8K8cIOQHl2zRqNr1pw53vHYY/N1OyZppO748qJtPpt09lKHIuJ4\n8fMXth9UbfmkYUh3Y7nDxQMAcuhsuWO/pKttr7a9VLUg3jO3k+1LJK2X9FBd28W2lxfP3yTpI5Ke\naVZqpTNp2/dJGpX0VtsvSLo9Iu6qckwAaKmDfdIRMWN7m6S9qk10d0XEQdtba6djZ9H1DyV9NyJe\nq7t8paQHbYdq+XtvROxtNp4j4ryLXSi2I5Yv73UZSGSFJnpdAhKanLQioqP/M7cd8cUvlut7220d\nj9epNGvSANA1fXTHISENYPEhpAEgMUIaABIjpAEgMUIaABLrozua+6dSAFgozKQBIDFCGgASI6QB\nIDFCGgASI6QBIDFCGgASYwseACTGTBoAEiOkASAxQhoAEiOkASAxQhoAEiOkASAxtuABQGLMpAEg\nMUIaABIjpAEgMUIaABIjpAEgMUIaABJjCx4AJMZMGgASI6QBIDFCGgASI6QBIDFCGgASI6QBIDG2\n4AFAYn00k25aqe032147T/t11ZUEABUbGir3SKBhFbY/KemQpAds/9j279ad/nrVhQFAZQYhpCX9\nvaT3RcT1krZIusf2x4tzrrwyAKhKH4V0szXpJRFxXJIi4gnbN0r6L9tXSIquVAcAVUgSwGU0C+kJ\n22sj4meSFBHHbY9K+raka7tRHABUYkB2d3xGc5Y1ImLC9gZJn6y0KgCo0iDMpCPi6QbtU5Lurawi\nAKjaIIQ0AAwsQhoAEuujkG5Zqe3PlWkDgL7RR1vwylTxp/O0/dkC1wEA3dNHId1wucP2Zkm3SLrK\n9p66UyskvVR1YQBQmQHZgvd9ScclvU3Sv9S1T0g6UGVRAFCpDmfJxVbkf1VtNWJXRHxpzvn1kh6S\n9FzR9J8R8Y9lrj1nrIje3zxoO2L58l6XgURWaKLXJSChyUkrIjr6WgrbEa+/Xq7vhReeM57tIUnP\nSvqwpJ9L2i9pU0QcquuzXtJfR8QftHvtXGU+OJyw/evi8brtGdu/LvUOASCjztak10k6HBFHivtG\ndkvaOE+/+f4xKXvtGS0XZiJixZkRbRcveEOr6wAgrc6WO1ZJOlp3/KJq4TvX79l+StIxSX8bET9p\n49oz2lo9j9rayLdt3y7ptnauBYA0GoT02NiYxsbGFmKE/5U0EhGv2r5Zte88etf5vFDLkLb9ibrD\nIUnvl1RuQacNbxlmDRJvmJhe0boTFp2F+o7kaPBK60dv1PrRG88c7/jCF+brdkzSSN3x5UXbG68f\nMVn3/BHbX7V9WZlr5yozk/5Y3fNpSeNqsYYCAJlNT3d0+X5JV9terdoOuE2SNtd3sL0yIk4Uz9ep\ntknjJdstr52rzJr0lvN6GwCQ1Ozs+V8bETO2t0naqze20R20vbV2OnZK+iPbn5E0Jek1SZ9qdm2z\n8VpuwbO9RtKXVfuwMCT9QNJfRcRzTS9sg+249NLebwVEHidZ7sA8PDm5IFvwXn21XN5cfHHnW/46\nVeYjzvskfVPSb0n6bUnfknR/lUUBQJVmZ8s9MigT0hdHxD0RMV08/kPShVUXBgBV6aeQLvPB4SO2\nb1Nt03WotrbycPFJpSKC7/EA0FeyBHAZZdakn29yOiJiTcdFsCaNOViTxnwWak36pZfK5c1ll/V+\nTbrM7o6rulEIAHRLh1vwuqrUHYe2PyDpyvr+EXF3RTUBQKX6abmjzB2H90haK+kpSTNFc0gipAH0\npYEKadVuA39PZPhOUwBYAIMW0s9IeodqtzACQN8btJB+m6Sf2H5C0qnTjXO/zBoA+sWghfT2qosA\ngG4aqJCOiP/uRiEA0C0DsQXP9oRquzjOOaXaTSxvrqwqAKjQQMyk6/9sFgAMkoEIaQAYVIQ0ACRG\nSANAYoQ0ACQ2ELs7AGBQMZMGgMQIaQBIjJAGgMQIaQBIjJAGgMQIaQBIjC14AJAYM2kASIyQBoDE\nCGkASIyQBoDECGkASIyQBoDE2IIHAIkxkwaAxAhpAEiMkAaAxAhpAEiMkAaAxAhpAEiMLXgAkBgz\naQBIjJAGgMQIaQBIjJAGgMQIaQBIjJAGgMTYggcAifXTTHqo6gFsb7B9yPaztm+tejwAaGV2ttyj\nkVa5ZvsW208Xj322r6s7N160P2n7iVa1VjqTtj0k6U5JH5b0c0n7bT8UEYeqHBcAmulkJl0y156T\n9PsR8SvbGyTtlHTD6eEljUbEyTLjVT2TXifpcEQciYgpSbslbax4TABoqsOZdMtci4jHI+JXxeHj\nklbVnbbayN6qQ3qVpKN1xy/q7GIBoOs6DOl2c+0vJD1SdxySHrW93/anW9Wa5oPD117bfub58PCo\nLrhgtGe1AMhhbHpaYzMzC/66jXZ3jI+PaXx8bMHGsX2jpC2SPlTX/MGIOG777aqF9cGI2NfoNaoO\n6WOSRuqOLy/aznHRRdsrLgVAvxkdHtbo8BsxtWNqakFet9EseWRkVCMjo2eOH3tsx3zdSuVa8WHh\nTkkb6tefI+J48fMXth9UbfmkYUhXvdyxX9LVtlfbXippk6Q9FY8JAE11uNzRMtdsj0h6QNKfRMTP\n6tovtr28eP4mSR+R9EyzWiudSUfEjO1tkvaq9g/Crog4WOWYANBKJ7s7GuWa7a2107FT0j9IukzS\nV21b0lRErJO0UtKDtkO1/L03IvY2G88Rcf7VLhDbcemlva8DeZycXtHrEpCQJycVEe7oNez4/OfL\n5c0dd7jj8TqV5oNDAOiWfrrjkJAGsOgQ0gCQGF+wBACJMZMGgMQIaQBIjJAGgMQIaQBIjJAGgMQI\naQBIjC14AJAYM2kASIyQBoDECGkASIyQBoDECGkASIyQBoDE2IIHAIkxkwaAxAhpAEiMkAaAxAhp\nAEiMkAaAxAhpAEiMLXgAkBgzaQBIjJAGgMQIaQBIjJAGgMQIaQBIjJAGgMTYggcAiTGTBoDECGkA\nSIyQBoDECGkASIyQBoDE2N0BAIkxkwaAxAhpAEiMkAaAxAhpAEiMkAaAxAhpAEiMLXgAkBgzaQBI\nrJ9CeqjXBQBAt83Olns0YnuD7UO2n7V9a4M+X7F92PZTtq9v59p6hHQyU1NjvS4ByYz10wJqn+gk\npG0PSbpT0k2SrpW02fY1c/rcLGltRLxT0lZJXyt77VyEdDLT02O9LgHJjM3M9LqEgdPhTHqdpMMR\ncSQipiTtlrRxTp+Nku6WpIj4oaRLbK8see1ZCGkAi06HIb1K0tG64xeLtjJ9ylx7Fj44BLDo9GAF\nyed7YZqQfvnl834PA+fUqR29LqHn+G04246pqV6XMEiOSF5dsu+JedqOSRqpO768aJvb54p5+iwt\nce1ZUoR0RPDfJICuiIgrO3yJ/ZKutr1a0nFJmyRtntNnj6TPSvqG7RskvRwRJ2z/ssS1Z0kR0gDQ\nLyJixvY2SXtV+1xvV0QctL21djp2RsTDtj9q+6eSXpG0pdm1zcZzRFT6hgAA54/dHUm0u8Edg8/2\nLtsnbB/odS3oHUI6gfPZ4I5F4S7VfiewiBHSObS9wR2DLyL2STrZ6zrQW4R0Dm1vcAewOBDSAJAY\nIZ1Dmc3xABYhQjqHM5vjbS9VbYP7nh7XhBwsbsBc1AjpBCJiRtLpDe4/lrS71QZ3DD7b90n6vqR3\n2X7B9pZe14Tu42YWAEiMmTQAJEZIA0BihDQAJEZIA0BihDQAJEZIA0BihDQqYXuigtdcbbvhX7Gw\n/Yjtk7a5EQgDg5BGVarYgH+VpFuanL9D0h9XMC7QM4Q0KmV7ve3v2f6W7YO276k797ztL9k+YPtx\n22uK9rtsf6Ku3+lZ+T9J+pDtH9n+3NyxIuJ7kiYrfktAVxHS6IbrJf2lpPdIWmv7A3XnTkbEdZL+\nTdKXG1x/elZ+m6T/iYj3RkSjvsBAIaTRDU9ExPGofQfBU5KurDu3u/h5v6Qbul0YkB0hjW44Vfd8\nRmf/lfqY5/m0it9N25a0tNLqgMQIaVSl7Ndrfqr4uUnSD4rn45LeXzzfKOmC4vmEpBUlxuWrPTEw\nhlt3Ac5Lo90dc9vfYvtpSa9LOr297t8lPWT7SUnflfRK0X5A0mzR/vW569K2H5P0bknLbb8g6c8j\n4tHO3wrQO3xVKXrG9vOS3hcRL/W6FiArljvQS8wQgBaYSQNAYsykASAxQhoAEiOkASAxQhoAEiOk\nASAxQhoAEvt/pJPBRDe9yq8AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "Weights_1 = Weight_Dict['Weights_1']\n", "Weights_2 = Weight_Dict['Weights_2']\n", "\n", "Train_Set = [[1.0,1.0],[0.0,1.0],[1.0,0.0],[0.0,0.0]] #train set\n", "\n", "tempdict = {'output':[],'target':[],'input':[]} #data dictionary\n", "temp = [neural_net(Input,Weights_1,Weights_2) for Input in Train_Set] #get the data\n", "[tempdict[key].append([temp[x][key] for x in range(len(temp))]) for key in tempdict] #combine all the output dictionaries\n", "\n", "plotter = np.ones((2,2))*np.reshape(np.array(tempdict['output']),(2,2))\n", "plt.pcolor(plotter,vmin=0,vmax=1,cmap=plt.cm.bwr)\n", "plt.colorbar(ticks=[0,0.25,0.5,0.75,1]);\n", "plt.xlabel('Input 1')\n", "plt.ylabel('Input 2')\n", "plt.xticks([0.5,1.5], ['0','1'])\n", "plt.yticks([0.5,1.5], ['0','1']);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again, the Input 1 value is on the x-axis and the Input 2 value is on the y-axis. As you can see, the network guesses 1 when the inputs are different and it guesses 0 when the inputs are the same. Perfect! Below I depict the network with these correct weights." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAHMCAYAAAB2sntRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XlYVHX7P/A3m2KyyT5s7kIkCg3g4y4gpiig5b5v5Z72\npH7NzKw0Sy0yr9wtl9xS09BcwX0hBDI0N1QQERBSFmWfmfv3R4/8InaYM+fMcL+ui+tKzpnP5z7p\n5+bwnjPn6BERGGOMaYa+2AUwxlhDwk2XMcY0iJsuY4xpEDddxhjTIG66jDGmQYZVbdTT0+NLGxhj\nrA6ISK+i71fZdP/3QvVXwxhjOkxPr8J+C4DjBcYY0yhuuowxpkHcdBljTIO46TLGmAZx02WMMQ3i\npssYYxrETZcxxjSImy5jjGkQN13GGNMgbrqMMaZB3HQZY0yDqr33AmNSUVRUhMzMTBQXF0OhUMDA\nwABGRkawsrJC06ZNxS6PsRrhpsskhYjw4MEDxMTEIDY2FvHx8Xj8+DHS0tKQm5sLa2trGBsbw8DA\nACqVCsXFxcjMzETjxo0hk8ng4OAAd3d3yOVyyOVyuLu7w9CQ/5kz6dCr6i5ienp6xHcZY0LLysrC\nsWPHEB4ejpMnT6Jp06alTdPT0xPOzs5wcHCAtbU19PXLJ2JEhOzsbKSmpuLx48e4ceMGYmNjERMT\ng5SUFHTv3h0hISEIDg6Gs7OzCEfIGho9Pb1Kb+3ITZeJoqioCAcOHMCWLVtw9epV+Pn5ITg4GEFB\nQXBwcFDbPDk5OTh16hTCw8Nx9OhRuLi4YNy4cRg3bhwsLCzUNg9j/8RNl0lGcnIyNmzYgC1btqB9\n+/aYOnUqgoKC8Morrwg+t0KhwPnz57Fp0yYcP34cQ4YMwfTp0+Hp6Sn43Kxhqarp8tULTCOePHmC\nWbNmwcvLC3l5eTh79iwiIiIwePBgjTRcADA0NIS/vz92796NW7duoXnz5hgwYADefPNN3Lp1SyM1\nMMZNlwnq+fPnWLx4Mdzd3WFgYIDbt2/jm2++gZubm6h12dvb48MPP0RCQgI6d+6MHj16YNKkSUhJ\nSRG1Lqb7uOkywURERMDDwwMPHjxAbGwsvvnmG9jY2IhdVhlNmjTBvHnzkJCQAFtbW3h5eWHTpk38\nmComGM50mdrl5uZi3rx5OHbsGDZu3Ii+ffuKXVKN3bhxA+PHj4eVlRU2bdoEFxcXsUtiWogzXaYx\n169fh5eXF5RKJa5fv65VDRcA2rdvjytXrqBnz57w9vbGkSNHxC6J6Rg+02Vqc+jQIbz99tv45ptv\nMGrUKLHLqbeoqCgMHjwYs2bNwvz586t8witj/8SXjDFBERGWLVuGDRs24Oeff4aPj4/YJanN48eP\nMXDgQLRr1w5btmyBsbGx2CUxLcDxAhMMEWHOnDk4ePAgoqOjdarhAoCjoyPOnz+PkpIShISEID8/\nX+ySmJbjpsvqjIgwc+ZMREdHIzIyEjKZTOySBNGkSRPs3r0bMpkM/fv358bL6oXjBVYnRIS5c+fi\n4sWLOHXqFMzMzMQuSXBKpRLjx49HRkYGwsPD0bhxY7FLYhLF8QJTu3Xr1uH48eM4fvx4g2i4AGBg\nYIAffvgBTZo0wfTp0/laXlYnfKbLau3MmTMYPnw4Ll++jNatW4tdjsa9ePECXbp0weTJk/Huu++K\nXQ6TIL56galNYmIiOnfujJ07dyIgIEDsckSTlJSEzp07Y8eOHejdu7fY5TCJ4abL1EKpVKJr164Y\nPnw45syZI3Y5ojt79ixGjBiB+Ph4yX28mYmLmy5TixUrVuDEiRM4depUhTcTb4jmzp2LR48eYe/e\nvWKXwiSEmy6rt1u3bqF79+64evUqWrZsKXY5klFQUAAvLy8sXboUgwcPFrscJhHcdFm9EBG6du2K\nMWPGYNq0aWKXIzlXrlwpvScvP42CAXzJGKunn3/+GYWFhZgyZYrYpUhS586d0a9fP6xcuVLsUpgW\n4DNdViWFQoH27dtj9erVeOONN8QuR7KSk5Ph5eWFGzdu6Own81jN8Zkuq7OtW7dCJpOhT58+Ypci\naS4uLhg/fjw+++wzsUthEsdnuqxSSqUSbdu2xY8//oguXbqIXY7kPX36FG3atMGtW7dgb28vdjlM\nRHymy+rkxIkTsLKy4oZbQ1ZWVhg6dCg2b94sdilMwvhMl1VqwIABeOuttzBhwgSxS9Ea165dQ3Bw\nMBITE2FoaCh2OUwkfKbLai0xMRFRUVEYNmyY2KVoFU9PT7i4uODw4cNil8Ikipsuq9BPP/2EoUOH\n4pVXXhG7FK0zceJE7NmzR+wymERx02UVCg8Px8CBA8UuQysNGDAAJ0+eRHFxsdilMAnipsvKycjI\nwJ9//omePXuKXYpWsrOzg5ubG86dOyd2KUyCuOmycn799VcEBgbykxHqISQkBL/88ovYZTAJ4qbL\nyrlw4UKDvleuOvTu3RsXLlwQuwwmQdx0WTmxsbHw9vYWuwyt5uHhgYSEBBQUFIhdCpMYbrqsjIKC\nAiQkJMDDw0PsUrSasbExXF1dER8fL3YpTGK46bIy4uPj4ebmxnmuGsjlcsTExIhdBpMYbrqsjISE\nBLi5uYldhk549dVXcf/+fbHLYBLDTZeVkZqaCgcHB7HLAAAEBARAX18fKpWqwu0JCQkYOHAgbG1t\nYW1tjX79+uHu3bul27dv3w5vb2+Ym5vDxcUF//d//1fpWEKQyWRITU3V2HxMO3DTZWWkpaVJ4n6w\nu3btgkKhgJ5ehR9fBwBkZ2cjNDQUd+/exZMnT+Dj44PQ0NDS7QUFBVi9ejWePn2K3377DZGRkVi1\napUmygfwd9NNS0vT2HxMO/ANb1gZw4YNw8CBAzFixAjRasjNzYWvry+2b9+Ozp07o6SkpEYPwszK\nyoKVlRWePn2KZs2aldseFhaGs2fPauz62Tt37mDAgAFISEjQyHxMOviGN6zGsrOzK2xYmrRw4UJM\nnz4ddnZ2tXrduXPnIJPJKq3//PnzeO2119RRYo00a9YMWVlZGpuPaQduuqyMkpISGBkZiTZ/TEwM\nLl++jFmzZtXqdSkpKZg5cybCwsIq3P79998jNjYWc+fOVUeZNWJkZASFQqGx+Zh24Bt+sjKIqMoc\nVd127dqFKVOmQE9PD926dcPTp0+xevXql7+e1WiMzMxMvPHGG5g5cyaGDh1abvuhQ4fw4YcfIjIy\nEpaWluo+hEpV9SYga7i46bIyDA0NoVQqNTbfyJEjMXLkSABATk4OrKysMGzYMBARlEoliAhOTk7Y\nt28funbtWu712dnZeOONNzBw4EAsWLCg3Pbjx49jypQpOHr0KNzd3QU/nn9SKBSi/tbApImbLiuj\nSZMmyMvLE2Vuc3PzMpdYJScnw9fXF3FxcbC2ti63//Pnz9GnTx9069YNy5YtK7f99OnTGD16NA4d\nOgS5XC5o7RXJy8uDsbGxxudl0saZLivD3t5e1MucbG1tS79sbGygp6cHW1vb0kffBAUF4YsvvgAA\nHDx4ELGxsfjhhx9gamoKU1NTmJmZISUlBQCwdOlS5ObmIigoqHRb//79NXYsUrn8jkkLn+myMhwc\nHCRzbWnz5s3LRR1Hjx4t/e+xY8di7Nixlb7+9OnTgtVWE2lpaZL5oAmTDj7TZWXwBf3qw2e6rCLc\ndFkZLi4uSEpKErsMnZCUlARnZ2exy2ASw02XleHp6Ym4uLgaX67FKhcXFwdPT0+xy2ASw02XlSGT\nyWBsbIyHDx+KXYpWIyLExsaKctUEkzZuuqwcuVyO2NhYscvQaklJSWjSpAlnuqwcbrqsHF9fX1y+\nfFnsMrTa5cuX4evrK3YZTIK46bJygoKCcOTIEbHL0GpHjhzR6DXBTHvwrR1ZOUQEZ2dnREZGwtXV\nVexytE5xcTHs7Oxw69Yt2Nvbi10OEwHf2pHVip6eHkJCQhAeHi52KVrpwoULcHV15YbLKsRNl1Vo\n0KBB2L17N186Vgd79uzBoEGDxC6DSRTHC6xCKpUKbdu2xa5du9CpUyexy9Ea2dnZaNmyJW7fvl3r\nm7Az3cHxAqs1fX19TJs2DWvXrhW7FK2ybds29OvXjxsuqxSf6bJKPX36FG3atEFCQkKFt1ZkZalU\nKri7u2Pz5s3o1q2b2OUwEfGZLqsTKysrDB06VKNP0NVm+/fvh6mpaYU3W2fsJT7TZVV6/PgxOnTo\ngPj4eDg6OopdjmSVlJTA3d0d69atQ+/evcUuh4mMz3RZnTk6OmLy5Mn45JNPxC5F0rZs2YIWLVpw\nw2XV4jNdVq1nz57B1dUVp0+fhoeHh9jlSE5OTg5effVVhIeHw9vbW+xymARUdabLTZfVyKZNm7Bh\nwwZERUWVPjqH/W3SpEkwMjLC+vXrxS6FSQTHC6zeJk+eDEtLS6xYsULsUiTl2LFjiIyMxMqVK8Uu\nhWkJPtNlNZacnAy5XM4xw/9kZ2fDw8MDW7duRUBAgNjlMAnhM12mFi4uLvjqq6/w1ltvISsrS+xy\nRKVUKjFy5Ei8+eab3HBZrfCZLqu1OXPm4ObNmzh69GiDzXfnz5+PmJgYnDhxAkZGRmKXwySGz3SZ\nWq1atQpEhHnz5oldiih27NiBAwcOYN++fdxwWa01zNMUVi+GhobYu3cvunTpAkdHR8ydO1fskjTm\n2LFjeP/993H69GlYWVmJXQ7TQtx0WZ1YWloiIiICPXv2hJGREWbPni12SYI7deoUxo0bh19++QXt\n27cXuxympbjpsjpzcnLC6dOnERAQgKKiIsyfP1/skgRz5MgRTJw4ET///DM6d+4sdjlMi/Ebaaze\nHj9+jD59+qBHjx5YvXo1GjVqJHZJakNEWLNmDT7//HP88ssvfG9hViP8iTQmuNzcXIwaNQq5ubnY\nv38/bGxsxC6p3oqKijBjxgxER0fjl19+QcuWLcUuiWkJvnqBCc7MzAyHDh1C165ddeIR7omJifD3\n98ezZ89w+fJlbrhMbbjpMrUxMDDA559/jq+++gqDBg2Cn58f8vLyxC6rVlQqFcaPH4/XX38dAwcO\nxP79+2FiYiJ2WUyHcLzABDFz5kz88ssvaNq0KTZt2oTu3buLXVK1EhISMGXKFNy7dw9OTk64dOkS\n9PQq/A2RsSpxvMA0KioqCvv370dsbCyWLVuG0aNHIyQkBDdu3BC7tAqlpaVh6tSp6Ny5M/r374+E\nhAQoFAps3LhR7NKYDuKmy9SqsLAQEyZMwJo1a2Bra4u33noLd+7cgb+/PwICAjBu3Dj8+eefYpcJ\nAEhNTcUHH3yA9u3bw9TUFHfu3MH777+Pxo0bY+vWrVi0aBEePnwodplMx3DTZWq1ePFitG/fHkOG\nDCn9nrGxMebMmYOEhAS0bt0avXv3hp+fH/bt24eSkhKN1kdEOHPmDIYMGYLXXnsNOTk5uHbtGlau\nXFnmE2bu7u54//33MXnyZHDExtSJM12mNlFRURg4cCDi4+Nha2tb6X7FxcU4dOgQ1q5di9u3byMk\nJAQhISEICAhAkyZN1F6XQqHAlStXcPjwYRw6dAhGRkaYMWMGRo8eDTMzsypf16VLF0yaNAlTpkxR\ne11Md/F1ukxwhYWF8PLywqefflrmLLc69+7dQ3h4OMLDwxEXF4cePXqgU6dOkMvlkMvlsLOzq3Ut\nubm5iIuLQ2xsLGJiYnDq1Ck4OzsjODgYISEhkMvlNX6D7ObNm+jZsydiYmLQvHnzWtfCGiZuukxw\n8+fPR2JiIvbt21fnMZ49e4bIyEjExMQgNjYWcXFxaNy4MZydnSGTyeDg4AA7Ozs0btwYhoaGUCqV\nKCkpQWZmJlJTU5GWlobU1FRkZmaiQ4cOpY3b398fLi4uda7riy++QGRkJE6ePMlXM7Aa4abLBFXT\nWKG2iAgpKSl4/Pgx0tLSkJaWhvT0dBQXF0OhUMDAwABGRkawsbGBTCYr/WrRooVa7/PLMQOrLW66\nTDB1jRW0DccMrDa46TLBqCNW0BYcM7Ca4qbLBCFUrCBVHDOwmuKmy9SuocQK/8YxA6sJbrpM7RpS\nrPBvHDOw6nDTZWrV0GKFf+OYgVWHmy5Tm4YaK/wbxwysKtx0mdo05Fjh3zhmYJXhpsvUoqHHCv/G\nMQOrDDddVm8cK1SMYwZWEW66rN44Vqgcxwzs37jpsnrhWKFqHDOwf+Omy+qMY4Wa4ZiB/RM3XVZn\nHCvUHMcM7CVuuqxOOFaoHY4Z2EvcdFmtcaxQNxwzMICbLqsDjhXqjmMGxk2X1QrHCvXDMQPjpstq\njGMF9eCYoWHjpstqjGMF9eGYoeHipstqhGMF9eKYoeHipsuqxbGCMDhmaJi46bJqcawgHI4ZGh5u\nuqxKHCsIi2OGhoebLqsUxwqawTFDw8JNl1WKYwXN4Zih4eCmyyrEsYJmcczQcHDTZeVwrCAOjhka\nBm66rByOFcTDMYPu46bLyuBYQVwcM+g+brqsFMcK0sAxg27jpstKcawgHRwz6C5uugwAxwpSwzGD\n7qqq6epruhj2t9u3byMgIAAWFhZo164dDh06VOm+YWFhkMlksLCwwOTJk1FSUlK67bvvvoOPjw+M\njY0xceLESscoLCzEhAkTsGbNGm64EmFoaIitW7di0aJFePjwYbX7FxcX47333oOjoyOsrKwwc+ZM\nKJXKSvefMmUK3NzcYGBggO3bt5fZNm3aNJiamsLMzAxmZmYwNjaGubl5vY+JVY+brgiUSiVCQ0MR\nEhKCrKwsbNiwAaNHj8a9e/fK7XvixAmsWLECZ86cwcOHD3H//n18/PHHpdsdHR3x0UcfYdKkSVXO\nuXjxYrRv355zXIlxd3fH+++/j8mTJ6O63yqXL1+OuLg43Lx5E3fv3kVsbCyWLl1a6f6enp5Yt24d\n5HJ5uW3r1q3D8+fPkZubi9zcXIwYMYL/bWgKEVX69fdmpm43btwgU1PTMt/r06cPLV68uNy+I0eO\npA8//LD0z6dPnyZ7e/ty+y1atIgmTJhQ4XxXrlwhOzs7evLkST0rZ0IoKSkhHx8fWr9+fZX7eXt7\n0759+0r/vGvXLnJxcal2/G7dutG2bdsq3f7ixQsyNTWlCxcu1LxoVqX/9c4K+yqf6UoEEeHGjRvl\nvv/nn3+iY8eOpX/u2LEjMjIykJWVVaNxOVaQvtrGDC+pVCqkpKTg+fPn9Zr/wIEDsLW1Rbdu3eo1\nDqsZbroicHV1ha2tLVatWgWFQoGTJ0/i3LlzyM/PL7fvixcvymRtZmZmIKIaLzSOFbRDTWKGvn37\nYvXq1fjrr7+Qnp6ONWvWAECF/25qY/v27Rg7dmy9xmA1x01XBIaGhjh06BCOHDkCmUyGsLAwDBs2\nDE5OTuX2NTExQW5ubumfc3JyoKenB1NT02rniYqKwvbt2/Hdd9+ptX4mjLlz5yInJwcbN24EAOza\ntav0za7+/ftj0aJF8PT0hKenJ7p164ZBgwbByMgIdnZ2dZ4zOTkZZ8+e5aarQdx0RdK+fXucPXsW\nmZmZOHbsGO7fvw9fX99y+7322mv4448/Sv987do12NnZoVmzZlWOz7GC9vl3zDBy5MjSN7t+/fVX\nNG7cGGvWrEFKSgru3buHZs2aVfgmWW38+OOP6NatG1q0aKGeg2DV4qYrkuvXr6OoqAj5+flYtWoV\n0tPTMX78+HL7jR07Flu2bMGtW7eQlZWFpUuXYsKECaXblUolCgsLoVQqoVAoUFRUBKVSybGClqoq\nZkhNTUVaWhqAv3+LWbp0KT799NNKxyopKUFhYSGICMXFxSgqKio35vbt28v8e2IaUNk7bMRXLwhq\n3rx51KxZMzI1NaWgoCC6f/8+ERElJyeTqakpPXr0qHTfsLAwsrOzI3Nzc5o0aRIVFxeXbluyZAnp\n6emRvr5+6dfkyZP5agUtVtnVDOfPn6cWLVpQ06ZNyc3NjXbv3l1me79+/Wj58uWlf+7Vq1e5fxvn\nzp0r3X7lyhUyMTGhFy9eCHtADRCquHqBP5GmY/jeCrqB782g3fhjwA0I31tBd/C9GbQXN90Ggu+t\noFv43gzai5tuA8Cxgm7imEE7cdNtADhW0F0cM2gfbro6jmMF3cYxg/bhpqvDOFZoGDhm0C7cdHUY\nxwoNB8cM2oObro7iWKFh4ZhBe3DT1UEcKzRMHDNoB266OohjhYaLYwbp46arYzhWaNg4ZpA+bro6\nhGMFBnDMIHXcdHUIxwrsJY4ZpIubro7gWIH9E8cM0sVNVwdwrMAqwjGDNHHT1QEcK7DKcMwgPdx0\ntRzHCqwqHDNIDzddLcaxAqsJjhmkhZuuFuNYgdUUxwzSwU1XS3GswGqDYwbp4KarhThWYHXBMYM0\ncNPVQhwrsLrimEF83HS1DMcKrD44ZhAfN10twrECUweOGcTFTVeLcKzA1IVjBvFw09USHCswdeKY\nQTzcdLUAxwpMCBwziIObrhbgWIEJhWMGzeOmK3EcKzAhccygedx0JYxjBaYJHDNoFjddCeNYgWkK\nxwyaw01XojhWYJrEMYPmcNOVII4VmBg4ZtAMbroSxLECEwvHDMLjpisxHCswMXHMIDxuuhLCsQKT\nAo4ZhMVNV0I4VmBSwTGDcLjpSgTHCkxKOGYQDjddCeBYgUkRxwzCaLBNNzs7G7///juSkpKQmpqK\n1NRUpKWl4cWLFygpKQEAGBkZwcTEBDKZDA4ODnBwcECLFi3g5eUFCwsLtdXCsQKTKiFihoKCAsTH\nxyMhIaF03aWmpiI7OxslJSUgIhgaGqJJkyawt7eHg4MDZDIZXFxc4OnpCZlMppY6xNJgmu79+/dx\n5MgRXLlyBbGxsUhPT0fHjh3Rpk0byGSy0i9zc3MYGhoC+PtXrJycHKSlpZV+3bt3D3/88Qfs7e0h\nl8vRuXNnDBgwAK1bt65TXRwrMClTR8yQkZGBX3/9FRcuXEBsbCwSEhLg5uYGNze30obq4OCAZs2a\nwcjICHp6elAqlcjLyyuz9pKSkhAXFwdjY2PI5XL4+voiKCgIXl5eWpU763TTjYuLw759+xAeHo6n\nT59iwIAB6NmzJ+RyOVxdXWFgYFCncZVKJe7cuYPY2FicO3cOR44cgZWVFUJCQjBkyBC8/vrrNRqH\nYwWmDeoSMyQmJuKnn35CeHg4/vzzTwQGBiIgIADe3t7w8PBA48aN61QLEeHhw4eIjY3F5cuXcfjw\nYRQUFCA4OBiDBg1CQEAA9PX16zS2plTVdEFElX79vVl68vPzaevWreTr60vNmzenDz74gKKiokip\nVAo2p1KppKioKPrggw+oefPm5OvrS1u3bqX8/PwqXzdv3jwaPHiwYHUxpi7Lly+n3r17k0qlqnQf\nhUJBv/76K/Xv35+srKxo2rRpdOLECSosLBS0ttu3b9OKFSvIy8uLWrVqRStXrqS//vpL0Dnr43+9\ns+K+WtkGkmDTzc3NpSVLlpC1tTX169ePDh8+TAqFQuN1KBQKOnz4MPXr14+sra1pyZIllJubW26/\nK1eukJ2dHT158kTjNTJWWyUlJeTj40Pr16+vcNumTZuoZcuWJJfL6fvvv6e8vDyN16hSqSgqKorG\njh1LFhYW9M4771BKSorG66iO1jfdwsJCWr16NdnZ2dHo0aMpISFB7JJK3b17l0aNGkV2dna0evXq\n0p/4BQUF5ObmRj/99JPIFTJWc3/++SdZW1tTUlISEf3d5Pbv30+urq7Uq1cvunTpksgV/n+ZmZk0\nf/58srS0pPnz59OzZ8/ELqmUVjfdU6dOUcuWLSkoKIiuXbsmdjmVunbtGgUFBVHLli3p1KlTHCsw\nrfUyZvjzzz+pc+fO5OnpScePH68ydhBTSkoKvf3222RtbU0bN26URJ1a2XRzc3NpypQp5OzsTMeO\nHROtjto6duwY2draUpMmTej+/ftil8NYrRUWFpKTkxOZmJjQ2rVrBX2vRJ3i4+NJLpdTnz596OHD\nh6LWUlXTleRbgBcvXoSHhwcUCgWuX7+Ovn37il1SjfXt2xe///47AgMD4e/vj4sXL4pdEmM1lpiY\niJ49e8LJyQmnTp3CtGnTJH+lwEseHh64cuUKevToAblcju3bt4tdUsUq68Yk0pnuxo0bydbWlo4c\nOaLxudXtyJEjZGtrSxs3bhS7FMaqdfr0abKzs6OwsDCtObutTHx8PLVp04bmzJlDJSUlGp8f2hAv\nFBcX04wZM8jV1ZXu3LmjsXmFdufOHXJ1daWZM2dScXGx2OUwVo5KpaLvvvuO7OzsKCIiQuxy1ObZ\ns2cUGBhIgYGB9PTpU43OXVXTlcSHI/Lz8/Hmm29CX18fu3fvhrm5ueBzalJOTg5GjBgBIsKBAwfw\nyiuviF0SYwD+PumaO3cujh8/jvDw8Dp/6lKqFAoF5s2bh2PHjiEiIgJOTk4ambeqD0eIHtbk5eWh\nf//+sLGxQXh4uM41XAAwNzdHeHg4rKys0L9/f+Tl5YldEmMgIsycORMXL17ExYsXda7hAoChoSHC\nwsIwadIk9OzZEw8fPhS7JHHPdAsKCjBgwAC4uLhg8+bNdf7IrrZQKpWYNGkSUlJScPjwYTRp0kTs\nklgDRUSYM2cOoqOjceLECZiZmYldkuBWr16Nb7/9FufPn4ejo6Ogc0ny3gsqlQqDBw+GsbExduzY\nofMN9yWlUonRo0ejuLgY+/bt05p3hpluWbp0KQ4ePIjIyEi13k1P6r788kts374dV65cEfQHjSSb\n7scff4zIyEicPn0ajRo1EmQOqSoqKoK/vz8CAwOxZMkSscthDczBgwfx7rvvIjo6WutvoVgXU6dO\nRWpqKg4dOiTYSY/kbnjz008/kYuLC6WnpwsyvjZIT08nFxcX2rdvn9ilsAYkPj6erK2tKTo6WuxS\nRFNUVEQ9evSghQsXCjYHpHTJ2M2bN8na2pri4uLUPra2iY2NJWtra7p586bYpbAGICcnh1q1akU/\n/vij2KWILiMjg1q0aEE///yzIONX1XQ1Gi8oFAp07doV48ePx7Rp09Q2rjZbu3Yttm/fjkuXLjWY\nXJuJY+rUqVAoFNi8ebPYpUjC5cuX8dZbbyE+Ph42NjZqHVsyl4x9/fXXaNq0KT8E7x+mTp2KJk2a\n4OuvvxbCd+1PAAAgAElEQVS7FKbDIiIicPToUXz11VdilyIZXbp0wahRozBr1iyNzquxM91bt26h\ne/fuuHr1Klq2bKmWMXVFYmIifHx8cPHiRbi5uYldDtMxz58/h4eHBzZs2IA33nhD7HIkpaCgAJ6e\nnvj888/x1ltvqW1cSVy90L9/f/Tu3RvvvfeeWsbTNWFhYYiMjMSRI0fELoXpmMWLF+PBgwf48ccf\nxS5Fks6fP48xY8bgzp07MDY2VsuYojfd8+fPY9y4cbh9+3adn5uk64qKiuDq6oodO3age/fuYpfD\ndMSTJ0/g7u6O2NhYtGjRQuxyJCskJAT+/v6YM2eOWsYTtekSEbp27Ypp06ZhzJgx9RpL123fvh0b\nNmzAxYsXterJp0y6Zs2aBQMDA3zzzTdilyJp169fR+/evZGQkKCWD02I+kba0aNH8eLFC4wcOVLo\nqbTeqFGjkJubi2PHjoldCtMBycnJ2LVrFz788EOxS5E8Dw8P9O3bVyM/nARvumvWrMG8efP4cqga\nMDAwwLx587BmzRqxS2E6YMOGDRgzZozaL4fSVfPnz8f69etRUlIi6DyCxgv37t1Dly5dkJycrLaA\nWtcVFhbCxcUFV65c0cm7PjHNKCoqQvPmzXHu3Dm4urqKXY7W6NWrF2bMmIEhQ4bUaxzR4oX169dj\n4sSJ3HBrwdjYGBMmTMD69evFLoVpsZ9//hnt27fnhltL06dPx9q1awWdQ7Az3ZKSEshkMr4utw4e\nPHiATp06IS0tDYaGhmKXw7RQQEAApk2bhsGDB4tdilYpLi6Gi4sLLl68iDZt2tR5HFHOdC9duoQW\nLVpww62DVq1awcXFBZcuXRK7FKaFsrKycPXqVQQFBYlditZp1KgRQkJCEB4eLtgcgjXd8PBwhISE\nCDW8zhP6L57prmPHjqFXr178WKg60sqmS0QIDw9HcHCwEMM3CMHBwQgPD4c6bzjEGgY+4amfgIAA\nxMXF4dmzZ4KML0jTTUxMRH5+Pjw9PYUYvkHw8vJCXl4eEhMTxS6FaREiwsmTJzlaqIcmTZqgR48e\niIyMFGR8QZpuTEwMOnXqxJ+qqgc9PT34+voiJiZG7FKYFnnw4AGaNm0KBwcHsUvRap06dRJs7QnW\ndOVyuRBDNyhyuRyxsbFil8G0CK899RBy7QnSdGNjY/kvXg246bLa4rWnHnK5HHFxcYK8pyJI042P\nj+c8Vw08PT0RHx8vdhlMi/DaUw87Ozs0btwYKSkpah9b7U23qKgIOTk5sLOzU/fQDY69vT2ys7NR\nXFwsdilMSzx+/BjOzs5il6ETnJ2d8fjxY7WPq/amm56eDjs7O8EebVwTe/bsgbu7O0xMTNC2bdtK\nP2SQmJiI4OBgmJmZwdbWFgsWLCjd9t1338HHxwfGxsaYOHGipkovQ19fH7a2tkhPTxdlfqZ90tLS\nRHkTrbi4GJMnT0aLFi1gbm6O119/HcePH6/2dQEBAdDX14dKpSr93u3btxEQEAALCwu0a9cOhw4d\nErL0SslkMqSlpal9XLV3RrH+0l86deoUPvjgA2zbtg0vXrzA+fPn0apVq3L7lZSUIDAwEL1790ZG\nRgZSUlIwevTo0u2Ojo746KOPMGnSJE2WX46Dg4Mgf/FM9xQVFSE3NxfW1tYan1uhUMDFxQUXLlxA\nTk4OPvvsMwwdOhTJycmVvmbXrl1QKBRlrnJSKpUIDQ1FSEgIsrKysGHDBowePRr37t3TxGGUIdTa\nU3vTffLkCWxtbdU9bI0tWbIEixcvho+PD4C/f1rJZLJy+23duhWOjo6YPXs2jI2N0ahRI7Rv3750\n+8CBAxESEgJLS0uN1V4ROzs7PtNlNZKZmQlra2tRfst85ZVXsHjx4tJoo3///mjZsmWlbwTn5ubi\n008/xcqVK8t8//bt20hLS8Ps2bOhp6cHPz8/dO3aFTt27BD8GP5NqLUnSKYr1l3FVCoVYmJikJGR\ngbZt28LFxQWzZs1CUVFRuX2joqLQvHlzBAUFwcbGBv7+/rhx44YIVVetcePGnOmyGikuLpbMHf2e\nPHmChIQEvPbaaxVuX7hwIaZPn16j936ISJS1KdTaU3vTVSgUot0Z68mTJygpKcGBAwdw6dIlXLt2\nDb///juWLl1abt+UlBTs3bsXc+bMQVpaGoKCghAaGgqFQiFC5ZUzNDSUXE1MmhQKhSQeFqBQKDB6\n9GiMHz8e7dq1K7c9JiYGly9frvDR566urrC1tcWqVaugUChw8uRJnDt3Dvn5+ZoovQyh1p7am66B\ngQGUSqW6h62RJk2aAADeffdd2NrawtLSEv/9739x9OjRCvft1q0b+vTpA0NDQ8ydOxdPnz7FrVu3\nNF12lZRKpSQWEpM+AwODMm9IiYGIMHr0aDRu3LjCJ6AQEWbMmIHVq1e/vP1hme2GhoY4dOgQjhw5\nAplMhrCwMAwbNgxOTk6aOoRSQq09tTfdRo0aifbrsIWFRbm/nMo+ityhQwet+JhySUkJjIyMxC6D\naQEjIyPRo6hJkybhr7/+ws8//1xhw8rNzUVsbCyGDRsGmUwGX19fEBGcnJxKrzJq3749zp49i8zM\nTBw7dgz379+Hr6+vpg9FsLWn9qZrbW2Nv/76S93D1tiECROwZs0aZGZmIisrC2FhYRXe7Wz06NGI\niorC6dOnoVKpEBYWBhsbG7z66qsA/v4pV1hYCKVSCYVCgaKiIlHO4DMzM/kZV6xGrKyskJmZKdqd\n6aZOnYrbt28jPDwcjRo1qnAfc3NzpKam4tq1a/jjjz9KfwuNi4tDp06dAPz9ZN6ioiLk5+dj1apV\nSE9Px/jx4zV1GKUEW3tEVOnX35trJyEhgVq2bFnr16lLSUkJTZ8+nSwsLEgmk9GcOXOoqKiIkpOT\nydTUlB49elS678GDB6lNmzZkbm5Ofn5+dPPmzdJtS5YsIT09PdLX1y/9+uSTTzR+PC1atKB79+5p\nfF6mnczMzOjZs2can/fhw4ekp6dHTZo0IRMTEzIxMSFTU1PatWtXhWvvpaSkJNLX1yelUln6vXnz\n5lGzZs3I1NSUgoKC6P79+5o8lFJvvfUW7d27t06v/V/vrLCvqv1xPXl5ebC2tkZ+fr5W/PouZUSE\nV155BU+fPuUbUrMacXNzw4EDByq9aoDVXNeuXfHFF1+ge/futX6tRh/X07RpUzRq1AhZWVnqHrrB\nycrKQqNGjbjhshpzcHAQ5KOrDVFqamqF1/jXlyBXUbu7u0vymldtc/36dT5jYbXCa089cnNzkZmZ\niRYtWqh9bEGarlwu55tvqwHfG5XVFt8OVD3i4uLQoUMHQT5zIFjT5b/4+uN7o7La4rWnHkKuPUGa\nrre3N65evSrE0A0Kn+my2nJ3d8ejR4+Qk5MjdilaTci1J1imm5OTgwcPHggxfINw//595ObmcqbL\nasXQ0BDdu3fHqVOnxC5FaykUCpw6dQr+/v6CjC9I0zUwMMCAAQNw+PBhIYZvEA4fPowBAwaIel9i\npp1CQkIQHh4udhla68qVK3B2doaLi4sg4wu2ooODg7np1sPhw4cr/CQdY9UJDg7G0aNH+UZJdST0\n2hOs6QYGBiI6OhpPnz4Vagqd9fTpU1y9ehW9e/cWuxSmhV6epZ0/f17sUrQOEeHQoUMICQkRbA7B\nmm7Tpk0xcOBAbN26VagpdNYPP/yAQYMGoWnTpmKXwrTUuHHjsGnTJrHL0Dpnz56FkZGRoG9gq/1j\nwP8UFRWF0aNH4+7du5xN1pBKpULbtm2xa9eu0huAMFZb2dnZaNGiBW7fvg17e3uxy9EaQ4YMQa9e\nvTBjxox6jaPRjwH/U6dOnWBmZsbvpNbCyZMnYWFhIcqt7JjusLCwwJAhQ7BlyxaxS9EaqampiIiI\nwJgxYwSdR9Cmq6enhxkzZuCrr74Schqd8tVXX2HGjBl8syBWbzNmzMC6detQUFAgdilaYc2aNRgx\nYgTMzMwEnUfQeAH4+7lNr776KjZt2iTYdW+6IjIyElOmTMGtW7f4xuVMLQYNGoQuXbpg3rx5Ypci\naampqfDw8MC1a9dKH65ZH1XFC4I3XQDYvXs3wsLC8Ntvv/EZXCWICL6+vnj//fcxfPhwscthOuLW\nrVvo0aMHEhISYGFhIXY5kjVt2jSYmJiUezpxXYmW6b40bNiw0gdGsoodOHAASqUSQ4cOFbsUpkNe\nffVVhISE4MsvvxS7FMlKSEjAvn37sGDBAo3Mp5EzXQA4ffo0xo8fj+vXr8Pc3FwtY+qK7OxseHh4\nYPv27fDz8xO7HKZjUlJS4OXlhTNnzqB9+/ZilyMpKpUKvXv3Rv/+/fH++++rbVzR44WXpk6dCoVC\ngc2bN6ttTF0wadIkNGrUCOvWrRO7FKajNm3ahA0bNiAqKkqQ2xVqq7Vr12LHjh24ePGiWp/8K5mm\n+/z5c3h4eGD9+vXo27ev2sbVZseOHcP06dMRHx8PU1NTscthOoqI8MYbb6BXr15YuHCh2OVIQmJi\nInx8fHDx4kW4ubmpdeyqmq7aH0xZnVOnTpGjoyOlpqaqfWxtc/fuXWrUqBFt2LBB7FJYA3D16lUy\nMjKiHTt2iF2K6AoLC6lLly60YsUKQcaHJh9MWROfffYZfv31V5w9exbGxsZqH18bFBYWolevXnB2\ndsbZs2exbNkyvP3223x1BxNEZGQkxowZg169euHs2bO4evUqHB0dxS5LFESEyZMnIysrC/v37xfk\n07KSOtMlIlKpVDRkyBAaN24cqVQqQeaQMpVKRWPHjqUhQ4aQSqWi27dvU4cOHWjEiBGUm5srdnlM\nhygUClq8eDE5ODhQREQEEREtX76cvL29KT8/X+TqxLF69Wry8PCg58+fCzYHqjjTFaXpEhG9ePGC\nPD09admyZYLNIVXLli0jLy8vevHiRen38vPz6Z133qG2bdvStWvXRKyO6YrU1FTy8/Mjf39/SktL\nK/2+SqWiESNG0JAhQ0ihUIhYoeYdOXKE7Ozs6MGDB4LOI8mmS0T0+PFjatOmDYWFhQk6j5SEhYVR\nmzZt6PHjxxVu37lzJ1lbW9OGDRsa5G8BTD0iIiJIJpPRkiVLKmysBQUFFBgYSOPGjSOlUilChZp3\n8uRJsrGxoaioKMHnkmzTJSJ6+PAhtWzZklatWiX4XGJbuXIltWzZkh4+fFjlfhw3sLqqKE6oTF5e\nHvXq1YvGjBlDJSUlGqpQHEePHiUbGxu6cOGCRuaTdNMlInr06BG1a9eOFixYoJM/dZVKJS1YsIDa\ntm1Ljx49qtFrOG5gtVVZnFCVvLw86tOnD4WGhgqacYpp+/btZGNjQ5cvX9bYnJJvukREGRkZ1KNH\nDwoODqacnByNzSu0nJwcCg4Oph49elBGRkatX89xA6uJ6uKEqhQWFtLEiRPJw8ODEhMThSlQBAqF\ngubNm0etWrWi69eva3RurWi6RERFRUU0ZcoUeu211yghIUGjcwshISGB3N3daerUqVRUVFTncThu\nYJWpTZxQFZVKRatXryZ7e3s6c+aM+goUSVZWFvXr14/8/Pzor7/+0vj8WtN0X1q7di1ZW1vT2rVr\ntTJuUCqV9N1335GVlRWtXbtWLWNy3MD+rS5xQnVOnTpF9vb2NHfuXK29pOzo0aPk5ORE7777LhUX\nF4tSg9Y1XSKimzdvUqdOncjPz0/wyzvU6cGDB+Tn50edOnWiW7duqX18jhsYUf3ihOpkZGTQ0KFD\nydXVVaM5aH1lZWXRxIkTqXnz5vU661cHrWy6REQlJSX05ZdfkqWlJS1atEjSWW92djYtWrSILC0t\n6csvvxT0+keOGxoudcUJNbFv3z6yt7enCRMmUHJysqBz1UdxcTGtW7eOZDIZTZkyRRJrQmub7kuJ\niYk0btw4srW1pa+//poKCgrELqlUQUEBffXVV2Rra0vjxo2jpKQkjczLcUPDI0ScUJ2srCxauHAh\nWVpa0vvvvy9KPloZpVJJe/fupbZt21Lv3r3p6tWrYpdUSuub7kvXr1+n4OBgsrOzow8//LDa612F\n9PDhQ/rwww/Jzs6OgoODNf7u6EscNzQMQsYJNZGamkrTpk0jCwsLeuedd0T9QZ+VlUXffPMNubq6\nkre3N506dUq0WiqjM033pZs3b9K7775LlpaWFBoaSvv379fINYa5ubm0f/9+Cg0NJUtLS3r33XcF\nyW1ri+MG3aXJOKEm0tLS6LPPPiMnJyfq0qULbd68mdLT0wWft7i4mCIiImjy5MlkYWFBI0aMoAsX\nLkj2RKOqpivKXcbU5cWLF9i9ezf279+PK1euoGvXrggJCUGPHj3g5uZW75sSK5VK3L59G+fPn0d4\neDguXbqEzp07Y/DgwRg5ciSaNm2qpiOpv4KCAsyZMwdnzpzBvn370LFjR7FLYvWUlpaGUaNGQU9P\nDzt37oS9vb3YJZVSKBQ4fPgwdu/ejZMnT8Ld3R3BwcHo3bs3PDw86n33QCJCUlISLl++jCNHjuD4\n8eNwdXXFoEGDMH78eNjZ2anpSIQhmZuYCyk3NxcnTpzA4cOHceXKFaSlpaFjx46Qy+Vo06YNZDIZ\nZDIZHBwcYGZmVnr3fIVCgdzcXKSmpiItLQ1paWm4d+8eYmNj8ccff0Amk6Fz584IDg7GG2+8Ifjj\nmetr165dmD17Nt8qUsu9vBXjlClTsGjRIrU+1UDdioqKcO7cOYSHh+PChQtISEiAq6sr5HI5Xn31\n1TJrr1mzZjAyMoK+vj4UCgXy8vJK111aWhqSkpIQFxeHuLg4GBsbw9fXF/3790f//v0hk8nEPtQa\naxBN99+ys7NL//IePnyI1NTU0sb64sULlJSUIDc3F1ZWVjAxMSn9R+Hg4IDmzZvj9ddfx+uvv66V\nT1C9c+cOhg4ditdeew0bNmzgJ1JoEaVSiU8//RSbN2/G9u3bERAQIHZJtVZQUID4+HjExMTg/v37\npesuNTUVWVlZUCgUyMnJgaWlJYyNjUvXnkwmg7OzMzw9PSGXy7Wqyf5bg2y6NfG//zFilyEIjhu0\nj5TjBHXT5bUHSOAR7EzzmjRpgg0bNmDJkiXo3bs3Nm7cqNP/yLVdZGQk5HI5evbsiZMnT+p0w23o\n+ExXh4/vJY4bpEsX4oS60PW1x2e6DZyrqyuioqJgamoKuVyOP/74Q+ySGP6OEwIDA3Hx4kXExsY2\nmIbb0HHTbSA4bpAWjhMaLo4XdPj4KsNxg3gaapzwb7q+9jheYGVw3CAOjhMYwE23weK4QbM4TmAv\ncbygw8dXUxw3CIfjhIrp+trjeIFVieMGYXCcwCrCTZcB4LhB3ThOYJXheEGHj6+uOG6oO44TakbX\n1x7HC6xWOG6oG44TWE1w02UV4rihdjhOYDXF8YIOH5+6cNxQOY4T6kbX1x7HC6xeOG6oGMcJrC64\n6bIa4bihLI4TWF1xvKDDxyeUhhw3cJygHrq+9jheYGrVUOMGjhOYOnDTZXXS0OIGjhOYunC8oMPH\npym6HDdwnCAMXV97HC8wQelq3MBxAhMCN12mFroWN3CcwITC8YIOH59YtDlu4DhBM3R97XG8wDRK\nW+MGjhOYJnDTZYLQtriB4wSmKRwv6PDxSYWU4waOE8Sh62uP4wUmKqnGDRwnMDFw02UaIbW4geME\nJhaOF3T4+KRKzLiB4wRp0PW1x/ECkxSx4gaOE5gUcNNlovh33LBhwwZBz3w4TmBSwfGCDh+fthAy\nbuA4QZp0fe1pdbwQEBAAfX19qFSqavfdvn079PX18f3339d7LKY5QsUNHCdon8TERAQHB8PMzAy2\ntrZYsGBBhfslJCRg4MCBsLW1hbW1Nfr164e7d+/WaSxNk3TT3bVrFxQKBfT0KvyBUUZ2djaWL1+O\n9u3b13sspnnqjhs4TtA+JSUlCAwMRO/evZGRkYGUlBSMHj26wn2zs7MRGhqKu3fv4smTJ/Dx8UFo\naGidxtI4Iqr06+/N4sjJySFXV1f67bffSF9fn5RKZZX7T506ldatW0e9evWiLVu21GgsMY+PVe72\n7dvUoUMHGjFiBOXm5tbqtQqFghYvXkwODg4UEREhUIWsvipaexs3bqQePXrUabxnz56Rnp4ePXv2\nrN5jqcP/jq/CvirZM92FCxdi+vTpsLOzq3bf6OhoxMbGYurUqfUei4mvrnEDxwnaLSoqCs2bN0dQ\nUBBsbGzg7++PGzdu1Oi1586dg0wmQ7Nmzeo9luAq68Yk4pnu1atXycvLi1QqFSUlJVV5pqtUKsnb\n25uio6OJiMqd6VY1lljHx2pu586dZG1tTevXryeVSlXpfhERESSTyWjJkiWkUCg0WCGri4rWXp8+\nfahRo0Z04sQJKikpoZUrV1KrVq2opKSkyrEePXpEjo6OtHfv3nqPpS6o4kxXEk13586dZGJiQqam\nptSvXz/y9fWl8+fPExFRYmJilU3322+/pUmTJpX++Z9NV6VSVTkWN13tUFXcwHGCdgJQZt0HBQVR\naGgo+fv7l9nP3Nyc4uPjKx0nIyOD3N3dafny5WW+X5ex1EnyTfefsrOzycDAgGQyGdnb25ONjQ3p\n6emRTCajixcvltt/4MCBZGlpSfb29mRvb0+NGjUiCwsLmjVrFmVnZ5O+vn6lY3HT1R75+fn0zjvv\nUNu2benatWtERJSamkp+fn7k7+9PaWlpIlfIaqOitffRRx9RQEBAme9V1SizsrLIy8uLFi5cWO+x\n1E2rmi4R0ZMnT0q/rl69Snp6epSWllbhrwY5OTll9u/SpQuFhYWVnhFVNRY3Xe3zMm6YPXs2xwla\nrKK1d+fOHWratClFRkaSUqmkr7/+mtq0aVPhus/NzSUfHx+aNWtWhePXZiwhaF3T/aeKMt1+/fqV\n+3XiJT8/v3JXL1Q2lhSOj9WOQqGg6dOnk6GhIfn5+dX66gYmDZWtvYMHD1KbNm3I3Nyc/Pz86ObN\nm6Xb/rnut23bRvr6+mRiYlL6ZWpqSo8eParRWEKrqunyJ9J0+Ph0TVpaGkaNGgU9PT1s3rwZX3zx\nBc6cOYN9+/ahY8eOYpfHakHX155WfyKNMQCIiIgo82GHli1bavTeDYypC5/p6vDx6YKX907YtGkT\nduzYUeG1t1J+MgWrmK6vPT7TZVrpnx92iIuLq/TDDlJ9MgVjFeGmyyTp33FCdfdO0PStIhmrK44X\ndPj4tFFN4oTqcNwgfbq+9jheYFqhpnFCdThuYFLGTZdJQm3jhOpw3MCkiuMFHT4+baCOOKE6HDdI\nj66vPY4XmCSpK06oDscNTEq46TJRqDtOqA7HDUwqOF7Q4eOTIk3ECdV5GTe4u7tjw4YNMDMz03gN\nDZ2urz2OF5gkaCpOqM7LuMHMzAze3t64du2aKHWwhombLtMITccJ1fln3BAYGMhxA9MYjhd0+Pik\nQApxQnU4btA8XV97HC8wUUglTqgOxw1Mk7jpMkFILU6oDscNTFM4XtDh4xODNsQJ1eG4QXi6vvY4\nXmAaoS1xQnU4bmBC4qbL1ELb4oTqcNzAhMLxgg4fnyboQpxQHY4b1E/X1x7HC0wQuhInVIfjBqZO\n3HRZnehanFAdjhuYunC8oMPHJ4SGECdUh+OG+tP1tcfxAlOLhhInVIfjBlYf3HRZjTS0OKE6HDew\nuuJ4QYePTx04Tqgexw21p+trj+MFViccJ9QMxw2sNrjpsgpxnFA7HDewmuJ4QYePry44Tqg/jhuq\np+trj+MFViMcJ6gHxw2sKtx0GQCOE9SN4wZWGY4XdPj4aoLjBOFx3FCerq89jhdYhThO0AyOG9g/\ncdNtoDhO0CyOG9hLHC/o8PFVhOME8XHcoPtrj+MFBoDjBKnguKFh46bbQHCcIC0cNzRcHC/o8PEB\nHCdog4YYN+j62uN4oYHiOEE7cNzQsBiKXYAmqFQqPH36FGlpaUhLS8Pz589RUlICANi/fz9MTU0h\nk8kgk8lgZWUFfX3t/1kUERGBsWPHYsqUKVi0aBEMDAzELolV4WXcsGvXLgQGBmLp0qV45513oKdX\n4cmS1iAi5OTklK69rKys0rW3b98+vPLKK7C3t4dMJoOtrS0MDXW/JencEebm5iImJgaxsbGlX8nJ\nyTAxMSltrObm5qV/uXv27Cnzj+LFixdwcXGBXC4v/fL29taaX/k4TtBuI0eOhFwux9ChQ3H27Fmt\nihuKiorw+++/l1l7CQkJMDQ0hEwmg4ODA5o1awYjIyMAwE8//YS8vLzStffs2TPY2dnB09MT3t7e\nkMvl8PHxgZ2dnchHpl46kek+ePAAhw8fRnh4OKKjo9GxY8fSZimXy9GqVSsYGxvXaKzCwkI8ePAA\nsbGxpc37jz/+QKdOnRAcHIyQkBC0bNlS4COqm7S0NIwaNQp6enrYuXMnv1mmxQoKCjBnzhycOXMG\nP/30Ezw9PcUuqUKZmZk4evQowsPDERERgVatWsHb27t07bm5ucHExKRGYykUCjx69AhxcXGl6+/q\n1ato3bo1QkJCEBISgo4dO2rF2X9VmS6IqNKvvzdLU15eHm3ZsoW8vb3J1taWJk6cSAcPHqQXL16o\nfa4XL17QwYMHaeLEiWRra0ve3t60ZcsWysvLU/tcdXXq1CmSyWS0ZMkSUigUYpfD1GTnzp1kbW1N\n69evJ5VKJXY5RESkUCgoPDyc+vbtS2ZmZvTmm2/S1q1bKSMjQ+1zFRcX05kzZ+i9996j1q1bU8uW\nLemLL74QZC51+l/vrLivVraBJNp0k5KS6L333iMrKysKDg6mY8eOabTJKBQKOnbsGAUHB5OVlRW9\n9957lJSUpLH5K6pn8eLFJJPJKCIiQrQ6mHBu375NHTp0oOHDh1NOTo5odWRlZdHnn39OLi4u1KlT\nJ9q2bRvl5+drbH6VSkVXr16liRMnkoWFBY0ePZpiYmI0Nn9t6ETTzcjIoNmzZ5OlpSXNnz+fEhMT\nxS6JEhMTaf78+WRpaUmzZ8/W+E/f1NRU8vPzI39/f0pLS9Po3Eyz8vPz6Z133qG2bdvS77//rvG5\nV0XGvikAABozSURBVKxYQTY2NjR27FhJNLqnT5/SqlWryMHBgQYPHkx37twRu6QytLrpFhQU0Kef\nfkpWVlY0c+ZMSk9PF7ukctLT02nmzJlkZWVFn376KRUUFAg+J8cJDZMm4waVSkU//PADOTk50aBB\ng+jmzZuCzlcXeXl5tHz5crK2tqZ33nlHMrGD1jbd3377jdzd3WngwIF0//59UWupifv379PAgQPJ\n3d2doqOjBZmjtnHCgwcPaMCAAWRqako2Njb0f//3fxXud/fuXQoNDSUbGxuysrKivn37ljl72LNn\nD7m6upKZmRnZ2dnR+PHj6fnz52o7Lm3j7+9Penp6pFQqq91327ZtpKenR1u2bCn93tatW8nAwIBM\nTU3JxMSETE1N6dy5czWaWxNxw8OHDykwMJDkcjlduXJFkDnU6enTpzRnzhyyt7en/fv3i12O9jXd\nwsJCWrBgAdna2tLu3bsl8wZCTahUKtq1axfZ2trSggULqLCwUG1j1zZOKC4uptatW9M333xDBQUF\nVFRURNevX69w3+joaPr+++8pKyuLFAoFffTRR+Tm5la6/dGjR/TkyRMi+vvsYtSoUTR79mz1HJiW\n2blzJ/Xo0YP09fWrbbpZWVnk5uZGHh4e5Zpu9+7d61yDUHGDSqWijRs3krW1NS1btoxKSkrUNrYm\nXLp0idq1a0dDhw4V9axXq5ru48ePydfXl0JDQyUZJdRUeno6hYaGkq+vLz1+/Lje49UlTti4cSP1\n6NGjTvM9e/aM9PT06NmzZ+W2PX/+nMaOHUv9+/ev09jaLCcnh1xdXem3336rUdOdOnUqrVu3jnr1\n6qXWpvuSOuOGvLw8Gj58OHl5eVX6w1kb5Ofn03//+19ydnYWLX/Wmqb722+/kaOjIy1btkyrzm4r\no1KpaOnSpeTk5FTnuKE+VydMnDiRxowZQ/369SNra2vy8/Or8WI6ePAgOTg4lPnexYsXydzcnPT0\n9MjExKRBXi0xY8YMWr16NSUlJVXbdH/77Tfy8fEhIqqw6ZqYmJCNjQ25urrSZ599VqOooiLqiBse\nPXpEcrmcRo0apdErEoS0f/9+sra2pt27d2t8bq1ounv27CEbGxv65ZdfNDanphw6dIhsbGxoz549\ntXpdfa9O6NOnDzVq1IhOnDhBJSUltHLlSmrVqlW1vzI+evSIHB0dae/evZXW9cknn9Ddu3drXZM2\nu3r1Knl5eZFKpaq26SqVSvL29i79YfvvppuYmFh6qeGNGzfI3d2dvvjiizrXVp+4ITo6mhwcHOjL\nL7/UiZOdf7p27Rq1aNGCFi1apNFjk3zT3bx5Mzk6OlJ8fLxG5hNDfHw8OTg4lFl4ValLnLBz587S\nN2WCgoIoNDSU/P39y+xjbm5e5f/njIwMcnd3p+XLl1c5V1RUFL3++us1qktb/fP/Z79+/cjX15fO\nnz9PRH83zaqa7rfffkuTJk0q/fO/m+6/7dmzh7y9vdVSc23ihgsXLujsyc5LGRkZJJfLadasWXX+\nbaK2JN10N2/eTM7Ozg3irOnOnTvk7OxMmzdvrnQfdX7Y4aOPPqKAgIAy36uq6WZlZZGXlxctXLiw\n2rEvXLhAFhYW9apPm2RnZ5OBgQHJZDKyt7cnGxsb0tPTI5lMRhcvXiy3/8CBA8nS0pLs7e3J3t6e\nGjVqRBYWFjRr1qwKx9+zZw/J5XK11FrTuOHSpUtkY2NDJ06cUMu8UpaVlUX/+c9/aMaMGRo545Vs\n092zZw85Ojo2iIb70p07d8jBwaHCqEHdH3a4c+cONW3alCIjI0mpVNLXX39Nbdq0qTBeyM3NJR8f\nn0qbws6dOyk5OZmI/v5UYM+ePWnw4MH1rlGbPHnypPTr6tWrpKenR2lpaRX+/8zJySmzf5cuXSgs\nLIxyc3OJiOjYsWOlV4PcunWL2rdvT5999pnaaq0uboiLi2swDfel7Oxs8vb2rvSySXWSZNONjo4m\nGxsbnY4UKhMfH082NjZl3lwT6sMOBw8epDZt2pC5uTn5+fmVucC9X79+pTHCtm3bSF9fn0xMTEq/\nTE1N6dGjR0RE9OGHH5KTkxOZmJiQs7MzTZ06tcIrGxqKijLdf/7//Dc/P78y8cLcuXPJzs6OTExM\nqHXr1oJ9yKWiuCE9PZ1cXFxo3759ap9P6p4+fUpt2rShH374QdB5JNd0Hz9+TE5OTjqdI1Xn0KFD\n5OTkRI8ePeJ7JzBB/TNuyMjIoC5dutCSJUvELks0N2/eJBsbG7p8+bJgc0iq6RYWFpKvry8tW7ZM\n7WNrm48++ohMTU2pV69efO8EJqiXcYOZmRm98cYbGntDSaqOHDlCDg4OlJKSIsj4VTVdjT8iYcmS\nJZDJZPjggw80PbXkLFmyBO3atUOnTp343rdMUE2aNEFgYCBMTEywdetWnXg6Sn30798fU6ZMwYQJ\nE16eYGqMRm9iHh0djeDgYMTHx+vc3eDrKj09HR07dsSRI0fg4+MjdjlMR2VmZqJDhw74+eef0blz\nZ7HLkQSFQoH//Oc/mDp1KiZPnqzWsau6ibnGmm5hYSFef/11LF68GMOHD1fLmLpi9+7dWLp0KWJj\nY2v8hAvGamPo0KFo3rw5Vq5cKXYpknL9+nX4+/sjNjYWLi4uahtXEk8DXrlyJVxdXTFs2DBNTak1\nhg8fjnbt2vGCYII4evQo/vjjD3z66adilyI5Hh4emD17NmbNmqWxOTVyppuZmYlXX30V0dHRaNWq\nVb3H00X3799Hp06dcOvWLdjY2IhdDtMRSqUSXl5e+OyzzxAaGip2OZJUWFgIV1dX7Nq1C127dlXL\nmKKf6S5btgwjRozghluF1q1bY/jw4fj888/FLoXpkF27dsHExAQhISFilyJZxsbG+OSTT7BgwQKN\nvKkm+JluUlIS5HI5bt68yW+eVSM9PR2vvfYa4uLi0Lx5c7HLYVquqKgIbm5u2LZtG3r06CF2OZKm\nVCrRoUMHrFixAv3796/3eKKe6X777beYPHkyN9wasLe3x6RJk/Dtt9+KXQrTAfv370fr1q254daA\ngYEBPv74Y6xYsULwuQQ9083Pz4eLiwtiYmLQokWLOo/TkCQmJsLHxwfJycl45ZVXxC6HabGuXbti\n3rx5GDhwoNilaIWSkhK0aNECJ06cQPv27es1lmhnunv27EGXLl244dZCy5Yt0blzZ+zdu1fsUpgW\nu3btGpKTkzFgwACxS9EaRkZGePvtt7Fu3TpB5xG06a5btw7Tp08XcgqdNH36dKxdu1bsMpgWW79+\nPaZMmQJDQ0OxS9Eqb7/9Nnbv3o0XL14INodgTffBgwdITk5GYGCgUFPorD59+iA5ORmJiYlil8K0\nkFKpxIEDBzB69GixS9E6jo6O8PH5f+3de0xT5xsH8C/WKnen0GFroSBMB05QCivTzM1txkQs4oy6\nLSoTzIZM4x8Tt5ixGzjdLWZxYtxmolODt0xTmMpIlui8dm1VTBQvjFugKOAFubfl/f3hTzI3Bwrn\nPaenPJ/EBFN4znNyeL99OT3nPYkoLi7mtg1uoVtYWIjZs2dDoVDw2oTHUigUSE5ORmFhodStEBk6\nc+YMNBoNndbrp5SUFJhMJm71uYau0WjkVd7jGY1GCl3SLyaTia7LHYDZs2fj8OHDcLlcXOpzCd3m\n5macPXuWTi0MwIwZM3D27Fk0NzdL3QqRGZrwDIxOp8OYMWNw5swZLvW5hK7FYkFcXBz8/Px4lB8U\n/P39MXHiRFitVqlbITJy584d1NTUQK/XS92KrE2bNg2nT5/mUptL6FqtVjroAtDr9RS65InYbDbE\nxcXRZykDxHPscQvdhIQEHqUHlYSEBFgsFqnbIDJCEx5h8Bx7NNN1Y3q9HjabTeo2iIzQ2BPGs88+\nC7vdjrt37wpeW/DQ7e7uRnV1NSIjI4UuPehERkaiqqpK9MeJEPmqrKxEVFSU1G3InkKhQHh4OKqr\nqwWvLXjoNjU1wd/fH8OHDxe6NCIiIvD7778LXvefPvvsMyxZsqTX79m8eTMSExPh7e2N9PR0Ln14\ne3vDz88PTU1NXOoTz1NXVwe1Wi14XXcZe11dXVi2bBnCw8MxYsQIxMfH4+jRo1x6UavVqKurE7yu\n4KFrt9u5HHR3M2bMGOTk5CAjI4PrdtRqNex2O9dtEM/AGMONGzc8evw5nU6EhYXhjz/+wN27d5Gb\nm4sFCxZwmZHyGnuyDd0dO3bgxRdfRHZ2NkaNGoXIyMiH3vGmT5+OtWvXwmAwYMSIEZg7dy7u3LkD\nADh27BhCQ0Mfqvfgnby4uBhffPEF9u7di4CAAEyePPmR209NTUVKSgpGjRrFbycBaDQaLu+2xPM0\nNTXBz8+P+3P2pBx7vr6++Pjjj3tqJCcnIyIigsuVBrzGnuChe+/ePYwYMULoso9kNpsRHR2NpqYm\nZGdn/2vWuXPnTmzfvh319fVQKBQPPQfJy+uRq65h5syZWLt2LRYuXIh79+7h3LlzXPehL4GBgVwX\n3yCeYzCOvRs3buDatWuYMGHCwHboEXiNPcFD1+FwiLaykU6nQ3p6Ory8vJCWlga73Y6bN2/2vL54\n8WJER0fDx8cHubm52L9/v+w+lBo6dCgcDofUbRAZcDqdol2f6w5jz+l0YtGiRXj77bcxbtw4QWsD\n/MaeaE8D5mH06NE9X/v4+ADAQ+9Mf/8zRqfTweFwoLGxUbwGCfFQUo89xhgWLVqE4cOHY9OmTYLV\nFYPgU1KlUgmn0yl02X6pqanp+bqqqgpKpRLBwcHw8/NDW1tbz2sulwsNDQ09//+vP3+k4HQ6oVQq\npW6DyMDQoUO5LdLypHiPvYyMDDQ2NuLw4cPcZve8xp7gM92AgAAuFxT3x65du1BWVoa2tjZ88skn\nmD9/Pry8vDBu3Dh0dHTgyJEjcDqdyMvLQ1dXV8/PhYSEoLKystc/h1wuFzo6OuByueB0OtHZ2cnl\nF765uRn+/v6C1yWeZ7CMvczMTJSVlcFkMmHYsGHc9oHX2BM8dHle4tTXu+A/X1+8eDHS0tKg0WjQ\n1dWF7777DsD9E+T5+fnIyMiAVqtFQEAAtFptz8/Nnz8fjDEEBQX95+3MeXl58PX1xZdffondu3fD\n19cX69atG+Ae/ltdXR00Go3gdYnnCQoKQmtrKzo6OgSv7S5jr7q6Gj/88APOnz+PkJAQBAQEIDAw\nEAUFBQLs5cO4jT3G2H/+u//yk7l58yYbNWrUE/+c0F5++WW2bds2qdsYsJEjR7KGhgap2yAyERoa\nyv766y9Je/CUsffaa6+xo0eP9utn/5+dj8xVwWe6QUFBaGlpQWdnp9ClB52Ojg60trYiKChI6laI\nTGg0GrqZRiB2u53LTFfw0B0yZAjCwsJQXl4udOkn4k4fhvVXeXk5dDqdR+wLEUd4eDiuX78uaQ+e\n8PvqcrlQWVmJsLAwwWtzuaD2wVqUMTExPMo/FjHuE+fNarUiPj5e6jaIjDwYe32tHcKTJ4y9srIy\nqNVqLjebcLlOV6/X0zqwArBYLLQuMXkitPC9MHiOPW6hSwd+4GhtVPKk4uPjceHCBbe5XleueI49\nLqGbkJCACxcuoLW1lUf5QaGlpQUXL16k0CVP5KmnnkJoaChNegbo+PHjeOGFF7jU5hK6gYGBMBgM\nKCkp4VF+UCgpKYHBYEBgYKDUrRCZMRqNKCwslLoN2aqqqkJtbS2SkpK41Oe29gId+IGhx2iT/kpJ\nSYHJZJK6DdkqKirCrFmzuN1ezDV0i4qK6NxSP7hcLhQVFVHokn5JSkqC3W5HZWWl1K3IkslkQkpK\nCrf63EJ37NixCAsLo1MM/fDbb79Bp9MhIiJC6laIDCkUCrz++uvYtWuX1K3ITm1tLf7880/MnDmT\n2za4Lu24fPly5Ofn89yER8rPz0dWVpbUbRAZy8zMxNatW91mxT+5+PHHH/Hmm29yXWTKi/Wymo+X\nlxfr7fW+tLW1ISwsDBaLBeHh4f2uM5hUVFQgMTER1dXV8PX1lbodImNTp05FdnY2UlNTpW5FFhwO\nB8LDw1FcXIznnntuQLW8vLzAGHvkrXlcZ7q+vr5YsmQJtmzZwnMzHmXLli1IS0ujwCUDlpWVhe+/\n/17qNmTj4MGDiIqKGnDg9oXrTBe4f/lFfHw8Ll26hJCQkAHV8nT19fWYMGECbDYbdDqd1O0Qmevq\n6sL48eOxY8cOTJs2Tep23JrL5UJsbCy++uorJCcnD7ieZDNd4P6jOhYvXoy8vDzem5K9vLw8LFmy\nhAKXCGLYsGH4/PPP8eGHH8ru2YBi27lzJ4KCgjBr1izu2+I+0wWAhoYGREdHw2w2Y+zYsQOu54nK\ny8thMBhQVlaG4OBgqdshHsLlcmHy5MnIzc3FnDlzpG7HLXV0dGD8+PEoKCjAlClTBKkp6UwXAFQq\nFVatWoX333+f3nEfgTGG1atXY9WqVRS4RFAKhQIbNmxAdnY22tvbpW7HLX3zzTeYNGmSYIHbF1Fm\nusD9dxO9Xo+cnBy88cYbgtT0FAUFBcjLy4PNZsPw4cOlbod4oAULFkCn0+Hrr7+WuhW3cvHiRbzy\nyiuwWq2Crp3b20xXtNAFALPZDKPRiNLSUvpQ7f/q6+sRFxeHoqIiJCYmSt0O8VANDQ2IjY3FL7/8\nwm0hF7lxOBxISkrC8uXLsWzZMkFrS3564YHnn38e6enpePfdd+k0A+6fVsjMzERGRgYFLuFKpVJh\n06ZNWLp0KVpaWqRuxy2sX78ewcHByMjIEHW7os50AaCzsxPTpk3DnDlzsHbtWkFry826detQWFiI\nY8eO0WkFIoqMjAzcvXsX+/btw5Ahos653Mqvv/6Kd955B2azGWPGjBG8fm8zXcGfBvw4amtrmVar\nZYcOHeJSXw4OHTrEtFotq6urk7oVMoh0dHSwKVOmsE8//VTqViRz6dIlplKp2KlTp7htA708DViS\n0GWMMbPZzFQqFSstLeW2DXdVWlrKVCoVM5vNUrdCBqH6+noWFhbG9u/fL3UromtsbGRRUVFs+/bt\nXLfjlqHLGGN79uxhGo2GXb16let23MmVK1eYRqNhe/bskboVMojZbDamUqnY0aNHpW5FNHfu3GF6\nvZ598MEH3LfltqHLGGM//fQTCw0NZVeuXOG+LalduXKFabVatm3bNqlbIYSdPHmSqVQqVlxcLHUr\n3N2+fZslJSWx9957j3V3d3PfnluHLmP3g1ej0Xj0qYbS0lKm0WgocIlbOXHiBFOpVB79+crNmzeZ\nXq9nK1euFCVwGZNB6DJ2/1SDpx78Q4cOMZVKRacUiFsym81Mo9GwDRs2iBZKYjl//jwLDw9nOTk5\nou6bLEKXsfsHX6vVsry8PI84+N3d3Sw3N5dptVr60Iy4tZqaGqbX69lbb73F2trapG5HEAcOHGDB\nwcGSTHZ6C13Rr9PtS11dHebOnQu1Wo2tW7fK9s61+vp6ZGZmor6+HgcPHoRarZa6JUJ61d7ejoyM\nDJSVleHnn3/mvq4sL+3t7fjoo4+wf/9+HDx4EHq9XvQe3OaOtMeh0Whw/PhxxMTEIDY2FgUFBbK6\ne40xhoKCAsTFxSEmJgbHjh2jwCWy4OPjg927dyMrKwvTp0/HunXrZPe4n1OnTmHSpEmora2FzWaT\nJHD79F9TYCbB6YV/MpvNLCYmhqWmprLr169L2svjuH79OktNTWUxMTF0OoHIWlVVFZsxYwbT6/Xs\n9OnTUrfTp6amJrZq1So2evRoduDAAanb6fX0gtvNdP8uMTERVqsV8fHxMBgMWLFiBW7cuCF1W/9S\nX1+PFStWwGAwID4+HjabjdZSILIWFhaG4uJirFy5EvPnz8fcuXNx+fJlqdv6l7a2Nqxfvx7jx49H\ne3s7SktLMW/ePKnb6pVbhy4AeHt7IycnB5cvX4ZSqURMTAzWrFmDiooKqVtDRUUF1qxZgwkTJkCp\nVOLy5cvIycmhdRSIR/Dy8kJaWhquXr2KqVOn4qWXXkJaWhqsVqvUreHWrVv49ttv8cwzz+DcuXM4\nefIktm7dCpVKJXVrfXL70H1ApVJh48aNsNlscLlcSExMhNFoxJEjR+ByuUTrw+Vy4ciRIzAajUhM\nTITL5YLNZsPGjRtlccAJeVI+Pj5YvXo1rl69iujoaMybNw8GgwE7duwQdWF0xhgsFgvS09MRGRmJ\nCxcuwGQyYd++fRg3bpxofQyU21298Lja2tqwd+9e5Ofno6qqCrNnz4bRaMSMGTMEf2Z9S0sLSkpK\nUFhYiKKiIuh0OmRlZWHhwoX01F4y6DyYeGzevBmnTp3Cq6++ipSUFCQnJws+8XA4HDhx4gRMJhNM\nJlPPcqhLly5160mO2yxizktFRQUKCwthMplgNpsxceJE6PV6JCQkQK/XIzIyEt7e3o9Vq6OjA+Xl\n5bBarbBYLLBarbh48SIMBgOMRiOMRiMiIiI47xEh8tDY2IjDhw/DZDKhpKQEY8eOfWjsRUdHP/Yk\nyOFwoKamBjabrWf8WSwWREVFISUlBSkpKYiNjYWX16NXTHQnHh+6f9fc3Ayr1dpz0Gw2G6qqquDn\n5we1Wg2NRoPAwEAMHToUAOB0OtHc3Iy6ujrY7Xa0trZCp9MhPj6+5xdHr9cjMDBQ4j0jxL11dnbi\n/PnzD42/a9euQaFQ9Iy9kSNHQqlUYsiQIXA6nWhtbYXdbofdbsft27cREhKCyZMn9wR3QkKCLK/V\nH1Sh+yjd3d24desW7HY76urq0NLSAofDAQBQKpXw9/eHRqOBWq1GUFCQLN5JCZEDxhiam5t7gvXW\nrVtwOp3o7u6GUqmEj48P1Go11Go1nn76aSgUCqlbFsSgD11CCBGTrO5II4QQT0ahSwghIqLQJYQQ\nEVHoEkKIiCh0CSFERBS6hBAiIgpdQggREYUuIYSIiEKXEEJERKFLCCEiotAlhBARUegSQoiIhvb1\nDbTiFiGECKfXVcYIIYQIi04vEEKIiCh0CSFERBS6hBAiIgpdQggREYUuIYSI6H9laN1q2/+9nAAA\nAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "Weight_Dict = {'Weights_1':Weights_1,'Weights_2':Weights_2}\n", "\n", "network = NeuralNetwork()\n", "network.add_layer(2,['Input 1','Input 2'],\n", " [[round(x,2) for x in Weight_Dict['Weights_1'][0][:2]],\n", " [round(x,2) for x in Weight_Dict['Weights_1'][1][:2]]])\n", "network.add_layer(2,[round(Weight_Dict['Weights_1'][0][2],2),round(Weight_Dict['Weights_1'][1][2],2)],\n", " [round(x,2) for x in Weight_Dict['Weights_2'][:2][0]])\n", "network.add_layer(1,[round(Weight_Dict['Weights_2'][0][2],2)])\n", "network.draw()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The network finds a pretty cool solution. Both hidden units are relatively active, but one hidden unit sends a strong postitive signal and the other sends a strong negative signal. The output unit has a negative bias, so if neither input is on, it will have an activity around 0. If both Input units are on, then the hidden unit that sends a postitive signal will be inhibited, and the output unit will have activity near 0. Otherwise, the hidden unit with a positive signal gives the output unit an acitivty near 1. \n", "\n", "This is all well and good, but if you try to train this network with random weights you might find that it produces an incorrect set of weights sometimes. This is because the network runs into a [local minima](https://en.wikipedia.org/wiki/Maxima_and_minima). A local minima is an instance when any change in the weights would increase the error, so the network is left with a sub-optimal set of weights. \n", "\n", "Below I hand-pick of set of weights that produce a local optima. " ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAHMCAYAAAB2sntRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3WlYVeX6P/AvAgrKoDJuBAQnEEXFjXrU4wBKqQhpOc9T\nDmllJ/E0eMwsK7Uy85+zZZpTahmaOOCQohACIc6hMijgECKTDHu4/y9KfpHMrLXX2nvfn+vihezF\n89wrem4evqy9lgkRgTHGmG40kLoAxhgzJtx0GWNMh7jpMsaYDnHTZYwxHeKmyxhjOmRW1YsmJiZ8\naQNjjNUBEZlU9Pkqm+5fXyh8NYwxZsBMTCrstwA4XmCMMZ3ipssYYzrETZcxxnSImy5jjOkQN13G\nGNMhbrqMMaZD3HQZY0yHuOkyxpgOcdNljDEd4qbLGGM6xE2XMcZ0qNp7LzAmFyUlJXj48CFKS0uh\nVqthamoKc3Nz2NnZoUmTJlKXx1iNcNNlskJEuH37NuLi4hAfH4+kpCRkZGQgKysLeXl5sLe3h4WF\nBUxNTaHValFaWoqHDx+iUaNGUCgUcHFxgY+PD5RKJZRKJXx8fGBmxv+bM/kwqeouYiYmJsR3GWNi\ny8nJQUREBMLDw3Hs2DE0adKkrGl26dIFbm5ucHFxgb29PRo0eDYRIyI8fvwYmZmZyMjIwOXLlxEf\nH4+4uDjcvXsXffr0QWhoKEJCQuDm5ibBGTJjY2JiUumtHbnpMkmUlJRg//792LJlCy5cuICAgACE\nhIRgyJAhcHFxEWye3NxcHD9+HOHh4Th8+DDc3d0xefJkTJ48GU2bNhVsHsb+jpsuk4309HRs2LAB\nW7ZsQceOHTF79mwMGTIEjRs3Fn1utVqNM2fOYNOmTThy5AhGjhyJV155BV26dBF9bmZcqmq6fPUC\n04n79+/j1VdfhZ+fHwoLC3H69GlERkZixIgROmm4AGBmZobAwEDs2rUL165dQ8uWLTF06FC8+OKL\nuHbtmk5qYIybLhNVfn4+Fi9eDB8fH5iamuL69ev44osv4O3tLWldzs7OePfdd5GcnIyePXuib9++\nmD59Ou7evStpXczwcdNloomMjISvry9u376N+Ph4fPHFF3BwcJC6rHIsLS0RFhaG5ORkODo6ws/P\nD5s2beLHVDHRcKbLBJeXl4ewsDBERERg48aNGDRokNQl1djly5cxZcoU2NnZYdOmTXB3d5e6JKaH\nONNlOnPp0iX4+flBo9Hg0qVLetVwAaBjx46Ijo5Gv3794O/vj0OHDkldEjMwvNNlgjlw4ABefvll\nfPHFFxg/frzU5dRbTEwMRowYgVdffRULFy6s8gmvjP0dXzLGREVEWLZsGTZs2IAffvgB3bp1k7ok\nwWRkZGDYsGFo164dtmzZAgsLC6lLYnqA4wUmGiLC/Pnz8eOPPyI2NtagGi4AtGjRAmfOnIFKpUJo\naCiePHkidUlMz3HTZXVGRJg3bx5iY2Nx4sQJKBQKqUsShaWlJXbt2gWFQoHg4GBuvKxeOF5gdUJE\nWLBgAaKionD8+HHY2NhIXZLoNBoNpkyZggcPHiA8PByNGjWSuiQmUxwvMMGtW7cOR44cwZEjR4yi\n4QKAqakpvvnmG1haWuKVV17ha3lZnfBOl9XaqVOnMGbMGJw/fx6tW7eWuhydKygoQK9evTBjxgy8\n9tprUpfDZIivXmCCSUlJQc+ePbFjxw4MGDBA6nIkk5qaip49e2L79u0YOHCg1OUwmeGmywSh0WjQ\nu3dvjBkzBvPnz5e6HMmdPn0aY8eORVJSkuze3sykxU2XCWLFihU4evQojh8/XuHNxI3RggULcOfO\nHezZs0fqUpiMcNNl9Xbt2jX06dMHFy5cgKenp9TlyEZRURH8/Pzw4YcfYsSIEVKXw2SCmy6rFyJC\n7969MXHiRMyZM0fqcmQnOjq67J68/DQKBvAlY6yefvjhBxQXF2PWrFlSlyJLPXv2xODBg7Fy5Uqp\nS2F6gHe6rEpqtRodO3bE6tWr8fzzz0tdjmylp6fDz88Ply9fNth35rGa450uq7OtW7dCoVDgueee\nk7oUWXN3d8eUKVPwwQcfSF0Kkzne6bJKaTQatG3bFt999x169eoldTmyl52djTZt2uDatWtwdnaW\nuhwmId7psjo5evQo7OzsuOHWkJ2dHUaNGoXNmzdLXQqTMd7pskoNHToUL730EqZOnSp1KXojMTER\nISEhSElJgZmZmdTlMInwTpfVWkpKCmJiYjB69GipS9ErXbp0gbu7Ow4ePCh1KUymuOmyCn3//fcY\nNWoUGjduLHUpemfatGnYvXu31GUwmeKmyyoUHh6OYcOGSV2GXho6dCiOHTuG0tJSqUthMsRNlz3j\nwYMHuHLlCvr16yd1KXrJyckJ3t7e+OWXX6QuhckQN132jJ9//hlBQUH8ZIR6CA0NxU8//SR1GUyG\nuOmyZ5w9e9ao75UrhIEDB+Ls2bNSl8FkiJsue0Z8fDz8/f2lLkOv+fr6Ijk5GUVFRVKXwmSGmy4r\np6ioCMnJyfD19ZW6FL1mYWEBLy8vJCUlSV0KkxluuqycpKQkeHt7c54rAKVSibi4OKnLYDLDTZeV\nk5ycDG9vb6nLMAjt27fHrVu3pC6DyQw3XVZOZmYmXFxcpC6jnFWrVkGhUKBp06aYMWMGVCpVpcee\nPHkSSqUStra2aNOmDTZt2qTDSstTKBTIzMyUbH4mT9x0WTlZWVmyuh/s0aNHsWLFCpw6dQppaWm4\ndesW3nvvvQqPVavVePHFFzFnzhzk5uZi9+7d+M9//oNLly7puOo/KRQKZGVlSTI3ky9uuqwcue10\nt23bhunTp8Pb2xu2trZYvHgxvvnmmwqPffToEfLz8zFhwgQAgL+/P9q3b4+rV6/qsuQyLi4uvNNl\nz+Cmy8p5/PgxmjVrJnUZZa5cuYLOnTuX/btz58548OABcnJynjnW0dERY8eOxddffw2tVovo6Gik\np6fj3//+ty5LLtOsWbMK62TGjZsuK0elUsHc3FzqMsoUFBTA1ta27N82NjYgIuTn51d4/JgxY7B0\n6VI0atQI/fr1w7Jly9CiRQtdlVuOubk51Gq1JHMz+eKmy8ohIpiYVHgbUJ3YuXMnrK2tYWNjg+Dg\nYFhbWyMvL6/s9dzcXJiYmMDa2vqZr71x4wZGjx6N7777DiqVCleuXMHy5csRERGhy1Mo06BBA2i1\nWknmZvLFTZeVY2ZmBo1GI9n848aNQ35+PvLy8vDzzz/Dx8cHFy9eLHs9MTERTk5OFUYgly9fhre3\nNwYOHAgAaNu2LYKDgyVrumq1Wla/NTB54KbLyrG0tERhYaHUZZSZNGkStmzZgmvXriEnJwcffvhh\npU+y8PPzw82bN3Hq1CkAwK1bt3Do0KFymbAuFRYWwsLCQpK5mXxx02XlODs7y+oyp+effx4LFy5E\nQEAAPD090bp1ayxZsqTs9SFDhuCTTz4BALRq1QpbtmzBa6+9BltbWwQEBGDkyJGYPn26JLXL7fI7\nJg/8jDRWzpIlS6DVarF06VKpS9F7Bw4cwNdff43w8HCpS2E6xs9IYzXGF/QLh3e6rCLcdFk57u7u\nSE1NlboMg5Camgo3Nzepy2Ayw02XldOlSxckJCSAY6X6S0hIQJcuXaQug8kMN11WjkKhgIWFBdLS\n0qQuRa8REeLj46FUKqUuhckMN132DKVSifj4eKnL0GupqamwtLTkTJc9g5sue0b37t1x/vx5qcvQ\na+fPn0f37t2lLoPJEDdd9owhQ4bg0KFDUpeh1w4dOoTg4GCpy2AyxNfpsmcQEdzc3HDixAl4eXlJ\nXY7eKS0thZOTE65duwZnZ2epy2ES4Ot0Wa2YmJggNDSUL+qvo7Nnz8LLy4sbLqsQN11WoeHDh2PX\nrl186Vgd7N69G8OHD5e6DCZTHC+wCmm1WrRt2xY7d+5Ejx49pC5Hbzx+/Bienp64fv06nJycpC6H\nSYTjBVZrDRo0wJw5c7B27VqpS9Er3377LQYPHswNl1WKd7qsUtnZ2WjTpg2Sk5Nhb28vdTmyp9Vq\n4ePjg82bN0v2iCAmD7zTZXViZ2eHUaNG4dNPP5W6FL2wb98+WFtbo3fv3lKXwmSMd7qsShkZGejU\nqROSkpIke9aYPlCpVPDx8cG6devKnlzBjBfvdFmdtWjRAjNmzMD7778vdSmytmXLFnh4eHDDZdXi\nnS6r1qNHj+Dl5YWTJ0/C19dX6nJkJzc3F+3bt0d4eDj8/f2lLofJQFU7XW66rEY2bdqEDRs2ICYm\nBmZmZlKXIyvTp0+Hubk51q9fL3UpTCY4XmD1NmPGDDRv3hwrVqyQuhRZiYiIwIkTJ7By5UqpS2F6\ngne6rMbS09OhVCo5ZvjL48eP4evri61bt2LAgAFSl8NkhHe6TBDu7u747LPP8NJLLyEnJ0fqciSl\n0Wgwbtw4vPjii9xwWa3wTpfV2vz583H16lUcPnzYaPPdhQsXIi4uDkePHoW5ubnU5TCZ4Z0uE9Sn\nn34KIkJYWJjUpUhi+/bt2L9/P/bu3csNl9WacW5TWL2YmZlhz5496NWrF1q0aIEFCxZIXZLORERE\n4M0338TJkydhZ2cndTlMD3HTZXXSvHlzREZGol+/fjA3N8frr78udUmiO378OCZPnoyffvoJHTt2\nlLocpqe46bI6c3V1xcmTJzFgwACUlJRg4cKFUpckmkOHDmHatGn44Ycf0LNnT6nLYXqM/5DG6i0j\nIwPPPfcc+vbti9WrV6Nhw4ZSlyQYIsKaNWvw0Ucf4aeffuJ7C7Ma4XekMdHl5eVh/PjxyMvLw759\n++Dg4CB1SfVWUlKCuXPnIjY2Fj/99BM8PT2lLonpCb56gYnOxsYGBw4cQO/evQ3iEe4pKSkIDAzE\no0ePcP78eW64TDDcdJlgTE1N8dFHH+Gzzz7D8OHDERAQgMLCQqnLqhWtVospU6aga9euGDZsGPbt\n2wcrKyupy2IGhOMFJop58+bhp59+QpMmTbBp0yb06dNH6pKqlZycjFmzZuHmzZtwdXXFuXPnYGJS\n4W+IjFWJ4wWmUzExMdi3bx/i4+OxbNkyTJgwAaGhobh8+bLUpVUoKysLs2fPRs+ePREcHIzk5GSo\n1Wps3LhR6tKYAeKmywRVXFyMqVOnYs2aNXB0dMRLL72EGzduIDAwEAMGDMDkyZNx5coVqcsEAGRm\nZuLtt99Gx44dYW1tjRs3buDNN99Eo0aNsHXrVixatAhpaWlSl8kMDDddJqjFixejY8eOGDlyZNnn\nLCwsMH/+fCQnJ6N169YYOHAgAgICsHfvXqhUKp3WR0Q4deoURo4ciQ4dOiA3NxeJiYlYuXJluXeY\n+fj44M0338SMGTPAERsTEme6TDAxMTEYNmwYkpKS4OjoWOlxpaWlOHDgANauXYvr168jNDQUoaGh\nGDBgACwtLQWvS61WIzo6GgcPHsSBAwdgbm6OuXPnYsKECbCxsany63r16oXp06dj1qxZgtfFDBdf\np8tEV1xcDD8/PyxdurTcLrc6N2/eRHh4OMLDw5GQkIC+ffuiR48eUCqVUCqVcHJyqnUteXl5SEhI\nQHx8POLi4nD8+HG4ubkhJCQEoaGhUCqVNf4D2dWrV9GvXz/ExcWhZcuWta6FGSduukx0CxcuREpK\nCvbu3VvnMR49eoQTJ04gLi4O8fHxSEhIQKNGjeDm5gaFQgEXFxc4OTmhUaNGMDMzg0ajgUqlwsOH\nD5GZmYmsrCxkZmbi4cOH6NSpU1njDgwMhLu7e53r+uSTT3DixAkcO3aMr2ZgNcJNl4mqprFCbRER\n7t69i4yMDGRlZSErKwv37t1DaWkp1Go1TE1NYW5uDgcHBygUirIPDw8PQe/zyzEDqy1uukw0dY0V\n9A3HDKw2uOky0QgRK+gLjhlYTXHTZaIQK1aQK44ZWE1x02WCM5ZY4Z84ZmA1wU2XCc6YYoV/4piB\nVYebLhOUscUK/8QxA6sON10mGGONFf6JYwZWFW66TDDGHCv8E8cMrDLcdJkgjD1W+CeOGVhluOmy\neuNYoWIcM7CKcNNl9caxQuU4ZmD/xE2X1QvHClXjmIH9EzddVmccK9QMxwzs77jpsjrjWKHmOGZg\nT3HTZXXCsULtcMzAnuKmy2qNY4W64ZiBAdx0WR1wrFB3HDMwbrqsVjhWqB+OGRg3XVZjHCsIg2MG\n48ZNl9UYxwrC4ZjBeHHTZTXCsYKwOGYwXtx0WbU4VhAHxwzGiZsuqxbHCuLhmMH4cNNlVeJYQVwc\nMxgfbrqsUhwr6AbHDMaFmy6rFMcKusMxg/HgpssqxLGCbnHMYDy46bJncKwgDY4ZjAM3XfYMjhWk\nwzGD4eOmy8rhWEFaHDMYPm66rAzHCvLAMYNh46bLynCsIB8cMxgubroMAMcKcsMxg+Gqquk20HUx\nrOYGDBiABg0aQKvVVnpMgwYNYG1tDWtra9jY2GDmzJkVHldcXIypU6dizZo13HBlwszMDFu3bsWi\nRYuQlpZW4TH9+/eHpaUlbGxsYG1tjfbt21d43J49e+Dt7Q1bW1s4Oztj6tSpKCgoELN8VkfcdGVq\n586dUKvV1f7aaWJigqSkJOTn5yMvLw8bN26s8LjFixejY8eOnOPKjI+PD958803MmDEDFf1WaWJi\ngrVr1yIvLw/5+fm4du1aheP07t0bZ86cQW5uLm7fvg2VSoVFixaJXT6rA266MpSXl4elS5di5cqV\n1R5LRFXuhIE/Y4Vt27bhq6++EqpEJqAFCxYgNze30h+YNYn4XF1dy36D0Wq1MDU1xc2bNwWtkwmE\niCr9+PNlpmtz586l1atXU2pqKjVo0IA0Gk2lx5qYmFCLFi1IoVDQSy+9RKmpqeVeLyoqIm9vb/r+\n++/FLpvVw5UrV8je3v6Z71///v3J0dGRHBwc6N///jedPn260jGioqLI1taWTExMyMrKiiIjI8Uu\nm1Xir95ZcV+t7AXipiuJCxcukJ+fH2m12ho13bNnz5JKpaLc3FyaN28edezYsdzxYWFhNGLECF2U\nzurp448/poEDB5JWqy37XGxsLBUUFFBpaSl9++23ZG1tTbdv365ynMzMTHr//ffp999/F7tkVglu\nujK2Y8cOsrKyImtraxo8eDB1796dzpw5Q0REKSkp1Tbdv9NoNGRlZUWXL18mIqLo6GhycnKi+/fv\ni1Y/E45KpaJu3brR+vXrKz1m0KBB9P/+3/+rdqyYmBjq2rWrkOWxWqiq6ZpJlmswAMC4ceMwbtw4\nAEBubi7s7OwwevRoEBE0Gg2ICK6urti7dy969+5d5Vj0V/ZHRHy1gh56ejVDv379MGjQoArfNPHX\npUjVjqVSqXD79m0xymT1VVk3Jt7pSuL+/ftlHxcuXCATExPKysoilUr1zLFXrlyhxMRE0mg0lJ+f\nT6+99hp5e3uTWq3mWEGPPY0ZcnJy6OjRo1RcXExqtZq+++47srKyouTk5Ge+ZseOHZSenk5ERKmp\nqdSvXz/+/ksIHC/op4oy3cGDB9PHH39MREQnT54kLy8vsrKyIicnJxo+fDjdvHmTYwU99zRm+PTT\nT6lbt25kY2NDzZo1o549e9KJEyeIiCg9PZ2sra3pzp07RET07rvvkqurK1lZWZGbmxvNnj2bHj16\nJOVpGLWqmi6/I83A8L0VDAPfm0G/8duAjQjfW8Fw8L0Z9Bc3XSPB91YwLHxvBv3FTdcIcKxgmDhm\n0E/cdI0AxwqGi2MG/cNN18BxrGDYOGbQP9x0DRjHCsaBYwb9wk3XgHGsYDw4ZtAf3HQNFMcKxoVj\nBv3BTdcAcaxgnDhm0A/cdA0QxwrGi2MG+eOma2A4VjBuHDPIHzddA8KxAgM4ZpA7broGhGMF9hTH\nDPLFTddAcKzA/o5jBvnipmsAOFZgFeGYQZ646RoAjhVYZThmkB9uunqOYwVWFY4Z5Iebrh7jWIHV\nBMcM8sJNV49xrMBqimMG+eCmq6c4VmC1wTGDfHDT1UMcK7C64JhBHrjp6iGOFVhdccwgPW66eoZj\nBVYfHDNIj5uuHuFYgQmBYwZpcdPVIxwrMKFwzCAdbrp6gmMFJiSOGaTDTVcPcKzAxMAxgzS46eoB\njhWYWDhm0D1uujLHsQITE8cMusdNV8Y4VmC6wDGDbnHTlTGOFZiucMygO9x0ZYpjBaZLHDPoDjdd\nGeJYgUmBYwbd4KYrQxwrMKlwzCA+broyw7ECkxLHDOLjpisjHCswOeCYQVzcdGWEYwUmFxwziIeb\nrkxwrMDkhGMG8XDTlQGOFZgcccwgDqNtuo8fP8Zvv/2G1NRUZGZmIjMzE1lZWSgoKIBKpQIAmJub\nw8rKCgqFAi4uLnBxcYGHhwf8/PzQtGlTwWrhWIHJlRgxQ1FREZKSkpCcnFy27jIzM/H48WOoVCoQ\nEczMzGBpaQlnZ2e4uLhAoVDA3d0dXbp0gUKhEKQOqRhN07116xYOHTqE6OhoxMfH4969e+jcuTPa\ntGkDhUJR9mFrawszMzMAf/6KlZubi6ysrLKPmzdv4uLFi3B2doZSqUTPnj0xdOhQtG7duk51cazA\n5EyImOHBgwf4+eefcfbsWcTHxyM5ORne3t7w9vYua6guLi5o1qwZzM3NYWJiAo1Gg8LCwnJrLzU1\nFQkJCbCwsIBSqUT37t0xZMgQ+Pn56VXubNBNNyEhAXv37kV4eDiys7MxdOhQ9OvXD0qlEl5eXjA1\nNa3TuBqNBjdu3EB8fDx++eUXHDp0CHZ2dggNDcXIkSPRtWvXGo3DsQLTB3WJGVJSUvD9998jPDwc\nV65cQVBQEAYMGAB/f3/4+vqiUaNGdaqFiJCWlob4+HicP38eBw8eRFFREUJCQjB8+HAMGDAADRo0\nqNPYulJV0wURVfrx58vy8+TJE9q6dSt1796dWrZsSW+//TbFxMSQRqMRbU6NRkMxMTH09ttvU8uW\nLal79+60detWevLkSZVfFxYWRiNGjBCtLsaE8vHHH9PAgQNJq9VWeoxaraaff/6ZgoODyc7OjubM\nmUNHjx6l4uJiUWu7fv06rVixgvz8/KhVq1a0cuVK+uOPP0Sdsz7+6p0V99XKXiAZNt28vDxasmQJ\n2dvb0+DBg+ngwYOkVqt1XodaraaDBw/S4MGDyd7enpYsWUJ5eXnPHBcdHU1OTk50//59ndfIWG2p\nVCrq1q0brV+/vsLXNm3aRJ6enqRUKunrr7+mwsJCndeo1WopJiaGJk2aRE2bNqWZM2fS3bt3dV5H\ndfS+6RYXF9Pq1avJycmJJkyYQMnJyVKXVOb333+n8ePHk5OTE61evbrsJ35RURF5e3vT999/L3GF\njNXclStXyN7enlJTU4nozya3b98+8vLyov79+9O5c+ckrvD/PHz4kBYuXEjNmzenhQsX0qNHj6Qu\nqYxeN93jx4+Tp6cnDRkyhBITE6Uup1KJiYk0ZMgQ8vT0pOPHj3OswPTW05jhypUr1LNnT+rSpQsd\nOXKkythBSnfv3qWXX36Z7O3taePGjbKoUy+bbl5eHs2aNYvc3NwoIiJCsjpqKyIighwdHcnS0pJu\n3boldTmM1VpxcTG5urqSlZUVrV27VtS/lQgpKSmJlEolPffcc5SWliZpLVU1XVn+CTAqKgq+vr5Q\nq9W4dOkSBg0aJHVJNTZo0CD89ttvCAoKQmBgIKKioqQuibEaS0lJQb9+/eDq6orjx49jzpw5sr9S\n4ClfX19ER0ejb9++UCqV2LZtm9QlVayybkwS7XQ3btxIjo6OdOjQIZ3PLbRDhw6Ro6Mjbdy4UepS\nGKvWyZMnycnJiVatWqU3u9vKJCUlUZs2bWj+/PmkUql0Pj/0IV4oLS2luXPnkpeXF924cUNn84rt\nxo0b5OXlRfPmzaPS0lKpy2HsGVqtlr766itycnKiyMhIqcsRzKNHjygoKIiCgoIoOztbp3NX1XRl\n8eaIJ0+e4MUXX0SDBg2wa9cu2Nraij6nLuXm5mLs2LEgIuzfvx+NGzeWuiTGAPy56VqwYAGOHDmC\n8PDwOr/rUq7UajXCwsIQERGByMhIuLq66mTeqt4cIXlYU1hYiODgYDg4OCA8PNzgGi4A2NraIjw8\nHHZ2dggODkZhYaHUJTEGIsK8efMQFRWFqKgog2u4AGBmZoZVq1Zh+vTp6NevH9LS0qQuSdqdblFR\nEYYOHQp3d3ds3ry5zm/Z1RcajQbTp0/H3bt3cfDgQVhaWkpdEjNSRIT58+cjNjYWR48ehY2NjdQl\niW716tX48ssvcebMGbRo0ULUuWR57wWtVosRI0bAwsIC27dvN/iG+5RGo8GECRNQWlqKvXv36s1f\nhplh+fDDD/Hjjz/ixIkTgt5NT+6WL1+Obdu2ITo6WtQfNLJsuu+99x5OnDiBkydPomHDhqLMIVcl\nJSUIDAxEUFAQlixZInU5zMj8+OOPeO211xAbG6v3t1Csi9mzZyMzMxMHDhwQbdMjuxvefP/99+Tu\n7k737t0TZXx9cO/ePXJ3d6e9e/dKXQozIklJSWRvb0+xsbFSlyKZkpIS6tu3L73zzjuizQE5XTJ2\n9epVsre3p4SEBMHH1jfx8fFkb29PV69elboUZgRyc3OpVatW9N1330ldiuQePHhAHh4e9MMPP4gy\nflVNV6fxglqtRu/evTFlyhTMmTNHsHH12dq1a7Ft2zacO3fOaHJtJo3Zs2dDrVZj8+bNUpciC+fP\nn8dLL72EpKQkODg4CDq2bC4Z+/zzz9GkSRN+CN7fzJ49G5aWlvj888+lLoUZsMjISBw+fBifffaZ\n1KXIRq9evTB+/Hi8+uqrOp1XZzvda9euoU+fPrhw4QI8PT0FGdNQpKSkoFu3boiKioK3t7fU5TAD\nk5+fD19fX2zYsAHPP/+81OXISlFREbp06YKPPvoIL730kmDjyuLqheDgYAwcOBBvvPGGIOMZmlWr\nVuHEiRM4dOiQ1KUwA7N48WLcvn0b3333ndSlyNKZM2cwceJE3LhxAxYWFoKMKXnTPXPmDCZPnozr\n16/X+blJhq6kpAReXl7Yvn07+vTpI3U5zEDcv38fPj4+iI+Ph4eHh9TlyFZoaCgCAwMxf/58QcaT\ntOkSEXpom1/DAAAgAElEQVT37o05c+Zg4sSJ9RrL0G3btg0bNmxAVFSUXj35lMnXq6++ClNTU3zx\nxRdSlyJrly5dwsCBA5GcnCzImyYk/UPa4cOHUVBQgHHjxok9ld4bP3488vLyEBERIXUpzACkp6dj\n586dePfdd6UuRfZ8fX0xaNAgnfxwEr3prlmzBmFhYXw5VA2YmpoiLCwMa9askboUZgA2bNiAiRMn\nCn45lKFauHAh1q9fD5VKJeo8osYLN2/eRK9evZCeni5YQG3oiouL4e7ujujoaIO86xPTjZKSErRs\n2RK//PILvLy8pC5Hb/Tv3x9z587FyJEj6zWOZPHC+vXrMW3aNG64tWBhYYGpU6di/fr1UpfC9NgP\nP/yAjh07csOtpVdeeQVr164VdQ7RdroqlQoKhYKvy62D27dvo0ePHsjKyoKZmZnU5TA9NGDAAMyZ\nMwcjRoyQuhS9UlpaCnd3d0RFRaFNmzZ1HkeSne65c+fg4eHBDbcOWrVqBXd3d5w7d07qUpgeysnJ\nwYULFzBkyBCpS9E7DRs2RGhoKMLDw0WbQ7SmGx4ejtDQULGGN3hif+OZ4YqIiED//v35sVB1pJdN\nl4gQHh6OkJAQMYY3CiEhIQgPD4eQNxxixoE3PPUzYMAAJCQk4NGjR6KML0rTTUlJwZMnT9ClSxcx\nhjcKfn5+KCwsREpKitSlMD1CRDh27BhHC/VgaWmJvn374sSJE6KML0rTjYuLQ48ePfhdVfVgYmKC\n7t27Iy4uTupSmB65ffs2mjRpAhcXF6lL0Ws9evQQbe2J1nSVSqUYQxsVpVKJ+Ph4qctgeoTXnjDE\nXHuiNN34+Hj+xguAmy6rLV57wlAqlUhISBDlbyqiNN2kpCTOcwXQpUsXJCUlSV0G0yO89oTh5OSE\nRo0a4e7du4KPLXjTLSkpQW5uLpycnIQe2ug4Ozvj8ePHKC0tlboUpicyMjLg5uYmdRkGwc3NDRkZ\nGYKPK3jTvXfvHpycnER7tHFNlZaWYsaMGfDw8ICtrS26du2KI0eOVHr8qlWroFAo0LRpU8yYMUP0\nm17URIMGDeDo6Ih79+5JXQrTE1lZWbL6I1pt1tWsWbPg7e0NU1NTbNu2TYdVVkyhUCArK0vwcQXv\njHL5pqvVari7u+Ps2bPIzc3FBx98gFGjRiE9Pf2ZY48ePYoVK1bg1KlTSEtLw61bt/Dee+9JUPWz\nXFxcRPnGM8NTUlKCvLw82NvbS10KgNqvqy5dumDdunWyyaTFWnuCN9379+/D0dFR6GFrrXHjxli8\neHHZr1rBwcHw9PSs8A9T27Ztw/Tp0+Ht7Q1bW1ssXrwY33zzja5LrpCTkxPvdFmNPHz4EPb29pL/\nlvlUbdfVnDlzEBAQIJuny4i19kTJdOV4V7H79+8jOTkZHTp0eOa1K1euoHPnzmX/7ty5Mx48eICc\nnBxdllihRo0acabLaqS0tFRWa0/O66omxFp7gjddtVotuztjqdVqTJgwAVOmTEG7du2eeb2goAC2\ntrZl/7axsQERIT8/X5dlVsjMzAxqtVrqMpgeUKvVsnpYgJzXVU2ItfYEb7qmpqbQaDRCD1tnRIQJ\nEyagUaNGlT6RwcrKCnl5eWX/zs3NhYmJCaytrXVVZqU0Go2sFhKTL1NTU2i1Wsnm37lzJ6ytrWFj\nY4Pg4GBYW1vLdl3VhFhrT/Cm27BhQ1n9Ojx9+nT88ccf+OGHHyr9D9ihQwdcvHix7N+JiYlwcnJC\ns2bNdFVmpVQqFczNzaUug+kBc3NzSdfeuHHjkJ+fj7y8PPz888/w8fGR7bqqCbHWnuBN197eHn/8\n8YfQw9bJ7Nmzcf36dYSHh6Nhw4aVHjdp0iRs2bIF165dQ05ODj788ENMnTpVh5VW7uHDh/yMK1Yj\ndnZ2ePjwoWzuTFfbdaVSqVBcXAwiQmlpKUpKSiQ9F9HWHhFV+vHny7WTnJxMnp6etf46oaWlpZGJ\niQlZWlqSlZUVWVlZkbW1Ne3cuZPS09PJ2tqa7ty5U3b8qlWryMnJiWxtbWn69OlUWloqYfX/x8PD\ng27evCl1GUxP2NjY0KNHj6Quo0xV62rw4MH08ccfl/27f//+ZGJiQg0aNCj7+OWXX6Qom4iIXnrp\nJdqzZ0+dvvav3llhXxX8cT2FhYWwt7fHkydP+C5j9UREaNy4MbKzs/mG1KxGvL29sX///gqv0mG1\n07t3b3zyySfo06dPrb9Wp4/radKkCRo2bKg3l4XIWU5ODho2bMgNl9WYi4uLKG9dNUaZmZlQKBSC\njyvKVdQ+Pj64fPmyGEMblUuXLvGOhdUKrz1h5OXl4eHDh/Dw8BB8bFGarlKp5JtvC4Dvjcpqi28H\nKoyEhAR06tRJlPcciNZ0+Rtff3xvVFZbvPaEIebaE6Xp+vv748KFC2IMbVR4p8tqy8fHB3fu3EFu\nbq7Upeg1MdeeaJlubm4ubt++LcbwRuHWrVvIy8vjTJfVipmZGfr06YPjx49LXYreUqvVOH78OAID\nA0UZX5Sma2pqiqFDh+LgwYNiDG8UDh48iKFDh8rmjlFMf4SGhiI8PFzqMvRWdHQ03Nzc4O7uLsr4\noq3okJAQbrr1cPDgQYSEhEhdBtNDISEhOHz4MN8oqY7EXnuiNd2goCDExsYiOztbrCkMVnZ2Ni5c\nuICBAwdKXQrTQ093aWfOnJG6FL1DRDhw4ABCQ0NFm0O0ptukSRMMGzYMW7duFWsKg/XNN99g+PDh\naNKkidSlMD01efJkbNq0Seoy9M7p06dhbm4u6h+wBX8b8N/FxMRgwoQJ+P333zmbrCGtVou2bdti\n586d6NGjh9TlMD31+PFjeHh44Pr163B2dpa6HL0xcuRI9O/fH3Pnzq3XODp9G/Df9ejRAzY2NvyX\n1Fo4duwYmjZtiu7du0tdCtNjTZs2xciRI7FlyxapS9EbmZmZiIyMxMSJE0WdR9Sma2Jigrlz5+Kz\nzz4TcxqD8tlnn2Hu3Ll8syBWb3PnzsW6detQVFQkdSl6Yc2aNRg7dixsbGxEnUfUeAH487lN7du3\nx6ZNm0S77s1QnDhxArNmzcK1a9f4xuVMEMOHD0evXr0QFhYmdSmylpmZCV9fXyQmJpY9zLY+qooX\nRG+6ALBr1y6sWrUKv/76K+/gKkFE6N69O958802MGTNG6nKYgbh27Rr69u2L5ORkNG3aVOpyZGvO\nnDmwsrLCypUrBRlPskz3qdGjR0OlUmH//v26mE4v7d+/HxqNBqNGjZK6FGZA2rdvj9DQUCxfvlzq\nUmQrOTkZe/fuxVtvvaWT+XSy0wWAkydPYsqUKbh06VK5J4SyP//S7Ovri23btiEgIEDqcpiBuXv3\nLvz8/HDq1Cl07NhR6nJkRavVYuDAgQgODsabb74p2LiSxwtPzZ49G2q1Gps3bxZsTEMwffp0NGzY\nEOvWrZO6FGagNm3ahA0bNiAmJkaU2xXqq7Vr12L79u2IiooS9Mm/smm6+fn58PX1xfr16zFo0CDB\nxtVnEREReOWVV5CUlKQ3j6Zm+oeI8Pzzz6N///545513pC5HFlJSUtCtWzdERUXB29tb0LGrarqC\nP5iyOsePH6cWLVpQZmam4GPrm99//50aNmxIGzZskLoUZgQuXLhA5ubmtH37dqlLkVxxcTH16tWL\nVqxYIcr40OWDKWvigw8+wM8//4zTp0/DwsJC8PH1QXFxMfr37w83NzecPn0ay5Ytw8svv8xXdzBR\nnDhxAhMnTkT//v1x+vRpXLhwAS1atJC6LEkQEWbMmIGcnBzs27dPlHfLymqnS0Sk1Wpp5MiRNHny\nZNJqtaLMIWdarZYmTZpEI0eOJK1WS9evX6dOnTrR2LFjKS8vT+rymAFRq9W0ePFicnFxocjISCIi\n+vjjj8nf35+ePHkicXXSWL16Nfn6+lJ+fr5oc6CKna4kTZeIqKCggLp06ULLli0TbQ65WrZsGfn5\n+VFBQUHZ5548eUIzZ86ktm3bUmJiooTVMUORmZlJAQEBFBgYSFlZWWWf12q1NHbsWBo5ciSp1WoJ\nK9S9Q4cOkZOTE92+fVvUeWTZdImIMjIyqE2bNrRq1SpR55GTVatWUZs2bSgjI6PC13fs2EH29va0\nYcMGo/wtgAkjMjKSFAoFLVmypMLGWlRUREFBQTR58mTSaDQSVKh7x44dIwcHB4qJiRF9Ltk2XSKi\ntLQ08vT0pE8//VT0uaS2cuVK8vT0pLS0tCqP47iB1VVFcUJlCgsLqX///jRx4kRSqVQ6qlAahw8f\nJgcHBzp79qxO5pN10yUiunPnDrVr147eeustg/ypq9Fo6K233qK2bdvSnTt3avQ1HDew2qosTqhK\nYWEhPffcc/TCCy+ImnFKadu2beTg4EDnz5/X2Zyyb7pERA8ePKC+fftSSEgI5ebm6mxeseXm5lJI\nSAj17duXHjx4UOuv57iB1UR1cUJViouLadq0aeTr60spKSniFCgBtVpNYWFh1KpVK7p06ZJO59aL\npktEVFJSQrNmzaIOHTpQcnKyTucWQ3JyMvn4+NDs2bOppKSkzuNw3MAqU5s4oSparZZWr15Nzs7O\ndOrUKeEKlEhOTg4NHjyYAgIC6I8//tD5/HrTdJ9au3Yt2dvb09q1a/UybtBoNPTVV1+RnZ0drV27\nVpAxOW5g/1SXOKE6x48fJ2dnZ1qwYIHeXlJ2+PBhcnV1pddee41KS0slqUHvmi4R0dWrV6lHjx4U\nEBAg+uUdQrp9+zYFBARQjx496Nq1a4KPz3EDI6pfnFCdBw8e0KhRo8jLy0unOWh95eTk0LRp06hl\ny5b12vULQS+bLhGRSqWi5cuXU/PmzWnRokWyznofP35MixYtoubNm9Py5ctFvf6R4wbjJVScUBN7\n9+4lZ2dnmjp1KqWnp4s6V32UlpbSunXrSKFQ0KxZs2SxJvS26T6VkpJCkydPJkdHR/r888+pqKhI\n6pLKFBUV0WeffUaOjo40efJkSk1N1cm8HDcYHzHihOrk5OTQO++8Q82bN6c333xTkny0MhqNhvbs\n2UNt27algQMH0oULF6QuqYzeN92nLl26RCEhIeTk5ETvvvtutde7iiktLY3effddcnJyopCQEJ3/\ndfQpjhuMg5hxQk1kZmbSnDlzqGnTpjRz5kxJf9Dn5OTQF198QV5eXuTv70/Hjx+XrJbKGEzTferq\n1av02muvUfPmzemFF16gffv26eQaw7y8PNq3bx+98MIL1Lx5c3rttddEyW1ri+MGw6XLOKEmsrKy\n6IMPPiBXV1fq1asXbd68me7duyf6vKWlpRQZGUkzZsygpk2b0tixY+ns2bOy3WhU1XQlucuYUAoK\nCrBr1y7s27cP0dHR6N27N0JDQ9G3b194e3vX+6bEGo0G169fx5kzZxAeHo5z586hZ8+eGDFiBMaN\nG4cmTZoIdCb1V1RUhPnz5+PUqVPYu3cvOnfuLHVJrJ6ysrIwfvx4mJiYYMeOHXB2dpa6pDJqtRoH\nDx7Erl27cOzYMfj4+CAkJAQDBw6Er69vve8eSERITU3F+fPncejQIRw5cgReXl4YPnw4pkyZAicn\nJ4HORByyuYm5mPLy8nD06FEcPHgQ0dHRyMrKQufOnaFUKtGmTRsoFAooFAq4uLjAxsam7O75arUa\neXl5yMzMRFZWFrKysnDz5k3Ex8fj4sWLUCgU6NmzJ0JCQvD888+L/njm+tq5cydef/11vlWknnt6\nK8ZZs2Zh0aJFgj7VQGglJSX45ZdfEB4ejrNnzyI5ORleXl5QKpVo3759ubXXrFkzmJubo0GDBlCr\n1SgsLCxbd1lZWUhNTUVCQgISEhJgYWGB7t27Izg4GMHBwVAoFFKfao0ZRdP9p8ePH5d989LS0pCZ\nmVnWWAsKCqBSqZCXlwc7OztYWVmV/U/h4uKCli1bomvXrujatatePkH1xo0bGDVqFDp06IANGzbw\nEyn0iEajwdKlS7F582Zs27YNAwYMkLqkWisqKkJSUhLi4uJw69atsnWXmZmJnJwcqNVq5Obmonnz\n5rCwsChbewqFAm5ubujSpQuUSqVeNdl/MsqmWxN//YeRugxRcNygf+QcJwjNkNceIINHsDPds7S0\nxIYNG7BkyRIMHDgQGzduNOj/yfXdiRMnoFQq0a9fPxw7dsygG66x452uAZ/fUxw3yJchxAl1Yehr\nj3e6Rs7LywsxMTGwtraGUqnExYsXpS6J4c84ISgoCFFRUYiPjzeahmvsuOkaCY4b5IXjBOPF8YIB\nn19lOG6QjrHGCf9k6GuP4wVWDscN0uA4gQHcdI0Wxw26xXECe4rjBQM+v5riuEE8HCdUzNDXHscL\nrEocN4iD4wRWEW66DADHDULjOIFVhuMFAz6/uuK4oe44TqgZQ197HC+wWuG4oW44TmA1wU2XVYjj\nhtrhOIHVFMcLBnx+QuG4oXIcJ9SNoa89jhdYvXDcUDGOE1hdcNNlNcJxQ3kcJ7C64njBgM9PLMYc\nN3CcIAxDX3scLzBBGWvcwHECEwI3XVYnxhY3cJzAhMLxggGfn64YctzAcYI4DH3tcbzARGWocQPH\nCUwM3HSZIAwtbuA4gYmF4wUDPj+p6HPcwHGCbhj62uN4gemUvsYNHCcwXeCmy0Shb3EDxwlMVzhe\nMODzkws5xw0cJ0jD0NcexwtMUnKNGzhOYFLgpst0Qm5xA8cJTCocLxjw+cmVlHEDxwnyYOhrj+MF\nJitSxQ0cJzA54KbLJPHPuGHDhg2i7nw4TmBywfGCAZ+fvhAzbuA4QZ4Mfe0ZVLywbds2+Pv7w9bW\nFu7u7vjvf/8LrVZb6fGJiYnw9/dHkyZN0K1bN9n85Zz9H7HiBo4T5C8tLQ3BwcFo3rw5XFxc8Oqr\nr1a6nhctWgRXV1c0a9YMgYGBuHr1qo6rFQgRVfrx58vysn79eoqKiiKVSkWZmZmkVCpp+fLlFR5b\nWlpKLVu2pNWrV1NpaSl9+eWX1LJlS1KpVEREJMfzM3Y7duwge3t7Wr9+PWm12jqPExkZSQqFgpYs\nWUJqtVrACpkQnq69IUOG0JQpU6i0tJTu379Pvr6+tGbNmmeO37NnD7Vo0YJSU1NJq9XS22+/TV27\ndtV12TX21/lV3Fcre4Fk2nT/6fPPP6fQ0NAKXzt27Bi5urqW+5y7uzsdPXqUiLjpytX169epU6dO\nNHbsWMrLy6vV16rValq8eDG5uLhQZGSkSBWy+nq69nx8fCgiIqLs82FhYTR79uxnjl++fDmNHj26\n7N9XrlwhS0tL8Quto6qart7FC/905swZdOjQocLXrly5gk6dOpX7XOfOnXHlyhVdlMbqqK5xA8cJ\n+mf+/PnYvXs3ioqKkJGRgYiICAwePPiZ48aMGYNbt24hOTkZKpUKW7durfA4faDXTffrr79GfHw8\nFixYUOHrBQUFsLW1Lfc5Gxsb5Ofn66I8Vg+1vbqBr07QT3369MHly5dhY2MDd3d3dOvWDaGhoc8c\np1Ao0Lt3b3h5eaFJkybYv38/Pv/8cwkqrj/ZN92dO3fC2toaNjY2CA4OLvv8gQMH8O677+LIkSNo\n3rx5hV9rZWWFvLy8cp/Lzc2V1Xv/WdXGjRuHqKgorF27FuPHj3/mB6ZGo8F7772HSZMmYfv27Xjv\nvfdgamoqUbWsNogIgwYNwogRI/DkyRP88ccfePToEf773/8+c+z777+PCxcuICMjA8XFxVi8eDEC\nAgJQXFwsQeX1VFnuQDLOdCMiIsjR0ZHi4uKqPO7YsWPk5uZW7nOc6eqnJ0+e0MyZM6lt27aUmJhI\nRESZmZkUEBBAgYGBlJWVJXGFrDYA0B9//EENGjQol9sfOHCAfH19nzl+6NCh9OWXX5b7XNOmTSk+\nPl70WusChpTpnjx5EhMmTMD+/fuhVCqrPLZ///4wNTXFmjVrUFpaii+//BINGjRAYGCgjqplQvln\n3DB//nyOE/ScnZ0dPD09sX79emg0Gjx+/BjffvstOnfu/Myx3bp1w969e/HgwQMQEbZv3w61Wo02\nbdpIUHk9VdaNSaY73YCAADI3Nydra2uysrIia2trGjJkSNnrgwcPpo8//rjs34mJiaRUKqlx48ak\nVCrp4sWLZa/J8fxY1dRqNb3yyitkZmZGAQEBtb66gcnD07V38eJF6t+/PzVr1owcHBxo9OjR9ODB\nA0pPTydra2u6c+cOEREVFxfTvHnzSKFQkK2tLSmVSjp27JiUp1AlVLHT5XekGfD5GZqsrCyMHz8e\nJiYm2Lx5Mz755BOcOnUKe/furXB3xOTL0NeeQb0jjRmnyMjIcnGCp6enTu/dwJhQeKdrwOdnCJ7e\nO2HTpk3Yvn17hdfeyvnJFKxihr72eKfL9NLf3+yQkJBQ6Zsd5PpkCsYqwk2XydI/44Tqrk7Q9a0i\nGasrjhcM+Pz0UU3ihOpw3CB/hr72OF5geqGmcUJ1OG5gcsZNl8lCbeOE6nDcwOSK4wUDPj99IESc\nUB2OG+TH0NcexwtMloSKE6rDcQOTE266TBJCxwnV4biByQXHCwZ8fnKkizihOk/jBh8fH2zYsAE2\nNjY6r8HYGfra43iByYKu4oTqPI0bbGxs4O/vj8TEREnqYMaJmy7TCV3HCdX5e9wQFBTEcQPTGY4X\nDPj85EAOcUJ1OG7QPUNfexwvMEnIJU6oDscNTJe46TJRyC1OqA7HDUxXOF4w4POTgj7ECdXhuEF8\nhr72OF5gOqEvcUJ1OG5gYuKmywShb3FCdThuYGLheMGAz08XDCFOqA7HDcIz9LXH8QIThaHECdXh\nuIEJiZsuqxNDixOqw3EDEwrHCwZ8fmIwhjihOhw31J+hrz2OF5ggjCVOqA7HDaw+uOmyGjG2OKE6\nHDewuuJ4wYDPTwgcJ1SP44baM/S1x/ECqxOOE2qG4wZWG9x0WYU4TqgdjhtYTXG8YMDnVxccJ9Qf\nxw3VM/S1x/ECqxGOE4TBcQOrCjddBoDjBKFx3MAqw/GCAZ9fTXCcID6OG55l6GuP4wVWIY4TdIPj\nBvZ33HSNFMcJusVxA3uK4wUDPr+KcJwgPY4bDH/tcbzAAHCcIBccNxg3brpGguMEeeG4wXhxvGDA\n5wdwnKAPjDFuMPS1x/GCkeI4QT9w3GBczKQuQBe0Wi2ys7ORlZWFrKws5OfnQ6VSAQD27dsHa2tr\nKBQKKBQK2NnZoUED/f9ZFBkZiUmTJmHWrFlYtGgRTE1NpS6JVeFp3LBz504EBQXhww8/xMyZM2Fi\nUuFmSW8QEXJzc8vWXk5OTtna27t3Lxo3bgxnZ2coFAo4OjrCzMzwW5LBnWFeXh7i4uIQHx9f9pGe\nng4rK6uyxmpra1v2zd29e3e5/ykKCgrg7u4OpVJZ9uHv7683v/JxnKDfxo0bB6VSiVGjRuH06dN6\nFTeUlJTgt99+K7f2kpOTYWZmBoVCARcXFzRr1gzm5uYAgO+//x6FhYVla+/Ro0dwcnJCly5d4O/v\nD6VSiW7dusHJyUniMxOWQWS6t2/fxsGDBxEeHo7Y2Fh07ty5rFkqlUq0atUKFhYWNRqruLgYt2/f\nRnx8fFnzvnjxInr06IGQkBCEhobC09NT5DOqm6ysLIwfPx4mJibYsWMH/7FMjxUVFWH+/Pk4deoU\nvv/+e3Tp0kXqkir08OFDHD58GOHh4YiMjESrVq3g7+9ftva8vb1hZWVVo7HUajXu3LmDhISEsvV3\n4cIFtG7dGqGhoQgNDUXnzp31YvdfVaYLIqr048+X5amwsJC2bNlC/v7+5OjoSNOmTaMff/yRCgoK\nBJ+roKCAfvzxR5o2bRo5OjqSv78/bdmyhQoLCwWfq66OHz9OCoWClixZQmq1WupymEB27NhB9vb2\ntH79etJqtVKXQ0REarWawsPDadCgQWRjY0Mvvvgibd26lR48eCD4XKWlpXTq1Cl64403qHXr1uTp\n6UmffPKJKHMJ6a/eWXFfrewFkmnTTU1NpTfeeIPs7OwoJCSEIiIidNpk1Go1RUREUEhICNnZ2dEb\nb7xBqampOpu/onoWL15MCoWCIiMjJauDief69evUqVMnGjNmDOXm5kpWR05ODn300Ufk7u5OPXr0\noG+//ZaePHmis/m1Wi1duHCBpk2bRk2bNqUJEyZQXFyczuavDYNoug8ePKDXX3+dmjdvTgsXLqSU\nlBSpS6KUlBRauHAhNW/enF5//XWd//TNzMykgIAACgwMpKysLJ3OzXTryZMnNHPmTGrbti399ttv\nOp97xYoV5ODgQJMmTZJFo8vOzqZPP/2UXFxcaMSIEXTjxg2pSypHr5tuUVERLV26lOzs7GjevHl0\n7949qUt6xr1792jevHlkZ2dHS5cupaKiItHn5DjBOOkybtBqtfTNN9+Qq6srDR8+nK5evSrqfHVR\nWFhIH3/8Mdnb29PMmTNlEzvobdP99ddfycfHh4YNG0a3bt2StJaauHXrFg0bNox8fHwoNjZWlDnq\nGyf8/vvvZGFhQRMnTqzw9d27d5OXlxfZ2NiQk5MTTZkyhfLz8+tbtsG5du0aBQYGkq2tLbVt25Z+\n/PHHCo/79ttvSalUko2NDbm5udHChQtJo9HUa25dxA1paWkUFBRESqWSoqOjRZlDSNnZ2TR//nxy\ndnamffv2SV2O/jXd4uJieuutt8jR0ZF27dolmz8g1IRWq6WdO3eSo6MjvfXWW1RcXCzY2ELECc89\n9xz17du30qZ7584dun//PhH9uYsYP348vf7663Wu2RCp1Wpq164dffHFF6TVaunkyZPUpEkTSk5O\nfubY9evXU1RUFKlUKsrMzCSlUknLly+vdw1ixQ1arZY2btxI9vb2tGzZMlKpVIKNrQvnzp2jdu3a\n0ahRoyTd9epV083IyKDu3bvTCy+8IMsooabu3btHL7zwAnXv3p0yMjLqPZ4QccKuXbto9OjR9P77\n71fadP8uPz+fJk2aRMHBwXWaz1BdvnyZrK2ty33uueeeo8WLF1f7tZ9//jmFhoYKVouQcUNhYSGN\nGSg8hQUAABcMSURBVDOG/Pz86NKlSwJVqHtPnjyh//znP+Tm5iZZ/qw3TffXX3+lFi1a0LJly/Rq\nd1sZrVZLH374Ibm6utY5bhDq6oTc3Fxq164dZWRk0JIlS6psulFRUWRra0smJiZkZWXFV0X8Q0VN\nNygoiF588cVqv3bYsGH09ttvC1qPEHHDnTt3SKlU0vjx43V6RYKY9u3bR/b29rRr1y6dz60XTXf3\n7t3k4OBAP/30k87m1JUDBw6Qg4MD7d69u1ZfJ+TVCa+//jqtXLmSiKjapvv3+d9//336/fff6zW3\noVGpVNS6dWtauXIlqVQqOnr0KDVs2JAGDRpU5ddt2bKF3NzcKDs7W/Ca6hM3xMbGkouLCy1fvtwg\nNjt/l5iYSB4eHrRo0SKdnpvsm+7mzZupRYsWlJSUpJP5pJCUlEQuLi60ZcuWGh0v5NUJv/32G3Xo\n0KEsn6tp0yUiiomJoa5du9ZrfkN06dIl6tevH9nb29OgQYNo4sSJNGPGjEqP//HHH8nZ2ZmuXLki\nal21jRvOnj1rsJudpx48eEBKpZJeffXVev8Rs6Zk3XQ3b95Mbm5uRrGbunHjBrm5udHmzZsrPUaM\nNzt88cUXZGVlRQqFgpydncnKyoosLS1JqVRW+7Vnz56lpk2bClKHIevVqxdt3LixwtciIiLI0dFR\nZ/liTeOGc+fOkYODAx09elQndUkpJyeH/vWvf9HcuXN1suOVbdPdvXs3tWjRwiga7lM3btwgFxeX\nCqMGsd7sUFRURPfv3y/7WLBgAY0cObLCX3N37NhB6enpRPTnu//69etHI0aMEKwWQ5GUlETFxcVU\nWFhIK1eupFatWlFpaekzx504cYLs7Ozo7NmzOq2vurghISHBaBruU48fPyZ/f3/673//K/pcsmy6\nsbGx5ODgYNCRQmWSkpLIwcGh3B/XdPlmh7/HC+np6WRtbU137twhIqJ3332XXF1dycrKitzc3Gj2\n7Nn06NEjUevRR2FhYdSsWTOytramIUOGlF1H/s//ngEBAWRubk7W1tZkZWVVdryuVBQ33Lt3j9zd\n3Wnv3r06q0MusrOzqU2bNvTNN9+IOo/smm5GRga5uroadI5UnQMHDpCrqyvduXOH753ARPX3uOHB\ngwfUq1cvWrJkidRlSebq1avk4OBA58+fF20OWTXd4uJi6t69Oy1btkzwsfXN//73P7K2tqb+/fvz\nvROYqJ7GDTY2NvT888/r7A9KcnXo0CFycXGhu3fvijJ+VU1X549IWLJkCRQKBd5++21dTy07S5Ys\nQbt27dCjRw++9y0TlaWlJYKCgmBlZYWtW7caxNNR6iM4OBizZs3C1KlTn24wdUanNzGPjY1FSEgI\nkpKSDO5u8HV17949dO7cGYcOHUK3bt2kLocZqIcPH6JTp0744Ycf0LNnT6nLkQW1Wo1//etfmD17\nNmbMmCHo2FXdxFxnTbe4uBhdu3bF4sWLMWbMGEHGNBS7du3Chx9+iPj4+Bo/4YKx2hg1ahRatmyJ\nlStXSl2KrFy6dAmBgYGIj4+Hu7u7YOPK4mnAK1euhJeXF0aPHq2rKfXGmDFj0K5dO14QTBSHDx/G\nxYsXsXTpUqlLkR1fX1+8/vrrePXVV3U2p052ug8fPkT79u0RGxuLVq1a1Xs8Q3Tr1i306NED165d\ng4ODg9TlMAOh0Wjg5+eHDz74AC+88ILU5chScXExvLy8sHPnTvTu3VuQMSXf6S5btgxjx47lhluF\n1q1bY8yYMfjoo4+kLoUZkJ07d8LKygqhoaFSlyJbFhYWeP/99/HWW2/p5I9qou90U1NToVQqcfXq\nVf7jWTXu3buHDh06ICEhAS1btpS6HKbnSkpK4O3tjW+//RZ9+/aVuhxZ02g06NSpE1asWIHg4OB6\njyfpTvfLL7/EjBkzuOHWgLOzM6ZPn44vv/xS6lKYAdi3bx9at27NDbcGTE1N8d5772HFihWizyXq\nTvfJkydwd3dHXFwcPDw86jyOMUlJSUG3bt2Qnp6Oxo0bS10O02O9e/dGWFgYhg0bJnUpekGlUsHD\nwwNHjx5Fx44d6zWWZDvd3bt3o1evXtxwa8HT0xM9e/bEnj17pC6F6bHExESkp6dj6NChUpeiN8zN\nzfHyyy9j3bp1os4jatNdt24dXnnlFTGnMEivvPIK1q5dK3UZTI+tX78es2bNgpmZmdSl6JWXX34Z\nu3btQkFBgWhziNZ0b9++jfT0dAQFBYk1hcF67rnnkJ6ejpSUFKlLYXpIo9Fg//79mDBhgtSl6J0W\nLVqgW7duOHr0qGhziNZ0Dx48iKFDh8LU1FSsKQyWqakpgoODcfDgQalLYXooJiYGLi4uHOvVUWho\nKMLDw0UbX9SmGxISItbwBi8kJISbLquT8PBwvi63HoYOHYrDhw9Do9GIMr4oTTcvLw+//vorRwv1\nEBQUhF9//RV5eXlSl8L0DG946qdly5Zo0aIFYmJiRBlflKYbFxeHzp07o0mTJmIMbxSsrKzg6+uL\n+Ph4qUtheuTx48e4c+cOlEql1KXotb59+yI6OlqUsUVpuvHx8fxNF4BSqeSmy2olISEBnTt35r+l\n1JOYa0+0puvv7y/G0EbF398fcXFxUpfB9AhveIQh5trjna6MKZVKJCQkSF0G0yO89oTh7e2NrKws\n5ObmCj624E1Xq9UiPT0drVu3Fnpoo9O6dWukpaXp/HEiTH+lpqaiTZs2Upeh90xNTeHh4YH09HTB\nxxa86WZnZ8PKygqNGjUSemh4enri5MmTgo/7T++//z4mTZpU5TFfffUVunXrBgsLC0ybNk2UOiws\nLNCkSRNkZ2eLMj4zPJmZmVAoFIKPK5e1V1paihkzZsDDwwO2trbo2rUrjhw5IkotCoUCmZmZgo8r\neNPNysoS5ZsuNy1atMD//vc/TJ8+XdR5FAoFsrKyRJ2DGQYiwv379w16/anVari7u+Ps2bPIzc3F\nBx98gFGjRomyIxVr7elt0/3222/Rp08fhIWFoXnz5mjdunW5n3gBAQF455130KNHD9ja2mL48OF4\n/PgxAOCXX36Bm5tbufGe/iQ/evQoPvroI+zZswfW1tbw8/OrcP5hw4YhNDQUzZs3F+8kAbi4uIjy\n05YZnuzsbDRp0kT05+xJufYaN26MxYsXl40RHBwMT09PUa40EGvtCd508/PzYWtrK/SwFYqNjUX7\n9u2RnZ2NsLCwZ3ad27dvx9atW3Hv3j2YmpqWew6SiUmFd13D888/j3feeQejR49Gfn4+fvvtN1HP\noTo2Njai3nyDGQ5jXHv3799HcnIyOnToUL8TqoBYa0/wpqtSqXR2Z6OWLVti2rRpMDExweTJk/9/\ne3cb09T1xwH8i1AVFDJXOmxXWkQGoW6oLaxME/doluiKOMOeIjLtsiHT+GLqlmVkyyjRPcUsbhi2\nmcxNgxMzSGEi45VGWdLRjodEHgYDJNAOGG4ESqEt5/+CSeZ/DhTuQ2/5fRITTe05v5vL+fZweu+5\ncDqd6O/vn349OzsbycnJCA8PR0FBAUpLSyX3pVRYWBi8Xq/YZRAJ8Pl8gl2fGwhjz+fzYefOnXj5\n5ZeRmJjIadsAf2NPsKcB82HlypXTfw8PDweAWz6Z/vlrjFarhdfrxeDgoHAFEhKkxB57jDHs3LkT\nS5YswfHjxzlrVwicT0llMhl8Ph/Xzc5JT0/P9N+7u7shk8kQHR2NZcuWwe12T7/m9/sxMDAw/e//\n+vVHDD6fDzKZTOwyiASEhYXxtknL3eJ77JnNZgwODuLChQu8ze75Gnucz3QjIyN5uaB4Lk6fPo2W\nlha43W68++67yMrKQkhICBITE+HxeFBVVQWfzweLxYKJiYnp98XExKCrq2vGX4f8fj88Hg/8fj98\nPh/Gx8d5+YEfHh7G8uXLOW+XBJ+FMvZyc3PR0tICq9WKxYsX83YMfI09zkOXz0ucZvsU/P/Xs7Oz\nkZOTA5VKhYmJCXz66acAphbIi4qKYDaboVarERkZCbVaPf2+rKwsMMYgl8v/83Zmi8WCiIgIfPDB\nBzhz5gwiIiJQWFg4zyP8t76+PqhUKs7bJcFHLpdjdHQUHo+H87YDZexdv34dX3zxBerr6xETE4PI\nyEhERUWhpKSEg6O8FW9jjzH2n3+mXr47/f397N57773r93HtscceYydPnhS7jHlbsWIFGxgYELsM\nIhGxsbHst99+E7WGYBl7Tz31FLt48eKc3vt3dt42Vzmf6crlcoyMjGB8fJzrphccj8eD0dFRyOVy\nsUshEqFSqehmGo44nU5eZrqch+6iRYug0WjQ0dHBddN3JZC+DJurjo4OaLXaoDgWIoy4uDi0t7eL\nWkMw/Lz6/X50dXVBo9Fw3jYvF9Te3ItSp9Px0fwdEeI+cb7Z7Xbo9XqxyyAScnPszbZ3CJ+CYey1\ntLRAqVTycrMJL9fpGgwG2geWA3V1dbQvMbkrtPE9N/gce7yFLp34+aO9Ucnd0uv1aGhoCJjrdaWK\nz7HHS+impqaioaEBo6OjfDS/IIyMjKCpqYlCl9yVe+65B7GxsTTpmafLly/jkUce4aVtXkI3KioK\nRqMRNTU1fDS/INTU1MBoNCIqKkrsUojEmEwmVFRUiF2GZHV3d6O3txfp6em8tM/b3gt04ueHHqNN\n5iojIwNWq1XsMiSrsrISW7Zs4e32Yl5Dt7KyktaW5sDv96OyspJCl8xJeno6nE4nurq6xC5FkqxW\nKzIyMnhrn7fQjY+Ph0ajoSWGOfjxxx+h1WqxatUqsUshEhQaGopnn30Wp0+fFrsUyent7cXPP/+M\np59+mrc+eN3ace/evSgqKuKzi6BUVFSEvLw8scsgEpabm4vi4uKA2fFPKr788ku8+OKLvG4yFcJm\n2M0nJCSEzfT6bNxuNzQaDerq6hAXFzfndhaSzs5OpKWl4fr164iIiBC7HCJhGzduxKFDh5CZmSl2\nKZLg9XoRFxeH6upqPPjgg/NqKyQkBIyx296ax+tMNyIiArt27cKJEyf47CaonDhxAjk5ORS4ZN7y\n8vLw2WefiV2GZJSVlSEhIWHegTsbXme6wNTlF3q9HteuXUNMTMy82gp2LpcLa9asgcPhgFarFbsc\nInETExNISkrCqVOnsGnTJrHLCWh+vx8pKSn48MMPsXXr1nm3J9pMF5h6VEd2djYsFgvfXUmexWLB\nrl27KHAJJxYvXoz3338fb731luSeDSi0b7/9FnK5HFu2bOG9L95nugAwMDCA5ORk2Gw2xMfHz7u9\nYNTR0QGj0YiWlhZER0eLXQ4JEn6/H+vXr0dBQQG2bdsmdjkByePxICkpCSUlJdiwYQMnbYo60wUA\nhUKBAwcO4I033qBP3NtgjOHgwYM4cOAABS7hVGhoKI4ePYpDhw5hbGxM7HIC0scff4x169ZxFriz\nEWSmC0x9mhgMBuTn5+OFF17gpM1gUVJSAovFAofDgSVLlohdDglCzz33HLRaLT766COxSwkoTU1N\neOKJJ2C32zndO3emma5goQsANpsNJpMJjY2N9KXa31wuF9auXYvKykqkpaWJXQ4JUgMDA0hJScH3\n33/P20YuUuP1epGeno69e/filVde4bRt0ZcXbnr44YexZ88evPbaa7TMgKllhdzcXJjNZgpcwiuF\nQoHjx49j9+7dGBkZEbucgHDkyBFER0fDbDYL2q+gM10AGB8fx6ZNm7Bt2za8/fbbnLYtNYWFhaio\nqMClS5doWYEIwmw246+//sK5c+ewaJGgc66A8sMPP+DVV1+FzWbD/fffz3n7M810OX8a8J3o7e1l\narWalZeX89K+FJSXlzO1Ws36+vrELoUsIB6Ph23YsIG99957YpcimmvXrjGFQsFqa2t56wMzPA1Y\nlNBljDGbzcYUCgVrbGzkrY9A1djYyBQKBbPZbGKXQhYgl8vFNBoNKy0tFbsUwQ0ODrKEhAT29ddf\n89pPQIYuY4ydPXuWqVQq1tbWxms/gaS1tZWpVCp29uxZsUshC5jD4WAKhYJdvHhR7FIE8+effzKD\nwcDefPNN3vsK2NBljLGvvvqKxcbGstbWVt77EltraytTq9Xs5MmTYpdCCLt69SpTKBSsurpa7FJ4\nd+PGDZaens5ef/11Njk5yXt/AR26jE0Fr0qlCuqlhsbGRqZSqShwSUC5cuUKUygUQf39Sn9/PzMY\nDGz//v2CBC5jEghdxqaWGoL15JeXlzOFQkFLCiQg2Ww2plKp2NGjRwULJaHU19ezuLg4lp+fL+ix\nSSJ0GZs6+Wq1mlkslqA4+ZOTk6ygoICp1Wr60owEtJ6eHmYwGNhLL73E3G632OVw4vz58yw6OlqU\nyc5MoSv4dbqz6evrw/bt26FUKlFcXCzZO9dcLhdyc3PhcrlQVlYGpVIpdkmEzGhsbAxmsxktLS34\n5ptveN9Xli9jY2N45513UFpairKyMhgMBsFrCJg70u6ESqXC5cuXodPpkJKSgpKSEkndvcYYQ0lJ\nCdauXQudTodLly5R4BJJCA8Px5kzZ5CXl4fHH38chYWFknvcT21tLdatW4fe3l44HA5RAndW/zUF\nZiIsL/w/m83GdDody8zMZO3t7aLWcifa29tZZmYm0+l0tJxAJK27u5tt3ryZGQwG9tNPP4ldzqz+\n+OMPduDAAbZy5Up2/vx5scuZcXkh4Ga6/5SWlga73Q69Xg+j0Yh9+/bh999/F7usf3G5XNi3bx+M\nRiP0ej0cDgftpUAkTaPRoLq6Gvv370dWVha2b9+O5uZmscv6F7fbjSNHjiApKQljY2NobGzEjh07\nxC5rRgEdugCwdOlS5Ofno7m5GTKZDDqdDocPH0ZnZ6fYpaGzsxOHDx/GmjVrIJPJ0NzcjPz8fNpH\ngQSFkJAQ5OTkoK2tDRs3bsSjjz6KnJwc2O12sUvD0NAQPvnkEzzwwAP45ZdfcPXqVRQXF0OhUIhd\n2qwCPnRvUigUOHbsGBwOB/x+P9LS0mAymVBVVQW/3y9YHX6/H1VVVTCZTEhLS4Pf74fD4cCxY8ck\nccIJuVvh4eE4ePAg2trakJycjB07dsBoNOLUqVOCbozOGENdXR327NmD1atXo6GhAVarFefOnUNi\nYqJgdcxXwF29cKfcbje+++47FBUVobu7G8888wxMJhM2b97M+TPrR0ZGUFNTg4qKClRWVkKr1SIv\nLw/PP/88PbWXLDg3Jx6ff/45amtr8eSTTyIjIwNbt27lfOLh9Xpx5coVWK1WWK3W6e1Qd+/eHdCT\nnIDZxJwvnZ2dqKiogNVqhc1mw0MPPQSDwYDU1FQYDAasXr0aS5cuvaO2PB4POjo6YLfbUVdXB7vd\njqamJhiNRphMJphMJqxatYrnIyJEGgYHB3HhwgVYrVbU1NQgPj7+lrGXnJx8x5Mgr9eLnp4eOByO\n6fFXV1eHhIQEZGRkICMjAykpKQgJuf2OiYEk6EP3n4aHh2G326dPmsPhQHd3N5YtWwalUgmVSoWo\nqCiEhYUBAHw+H4aHh9HX1wen04nR0VFotVro9frpHxyDwYCoqCiRj4yQwDY+Po76+vpbxt+vv/6K\n0NDQ6bG3YsUKyGQyLFq0CD6fD6Ojo3A6nXA6nbhx4wZiYmKwfv366eBOTU2V5LX6Cyp0b2dychJD\nQ0NwOp3o6+vDyMgIvF4vAEAmk2H58uVQqVRQKpWQy+WS+CQlRAoYYxgeHp4O1qGhIfh8PkxOTkIm\nkyE8PBxKpRJKpRL33XcfQkNDxS6ZEws+dAkhREiSuiONEEKCGYUuIYQIiEKXEEIERKFLCCECotAl\nhBABUegSQoiAKHQJIURAFLqEECIgCl1CCBEQhS4hhAiIQpcQQgREoUsIIQIKm+0/0I5bhBDCnRl3\nGSOEEMItWl4ghBABUegSQoiAKHQJIURAFLqEECIgCl1CCBHQ/wDCgDfDcd3GRgAAAABJRU5ErkJg\ngg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "Weights_2 = np.array([-4.5,5.3,-0.8]) #connections between hidden and output\n", "Weights_1 = np.array([[-2.0,9.2,2.0],\n", " [4.3,8.8,-0.1]])#connections between input and hidden\n", "\n", "Weight_Dict = {'Weights_1':Weights_1,'Weights_2':Weights_2}\n", "\n", "network = NeuralNetwork()\n", "network.add_layer(2,['Input 1','Input 2'],\n", " [[round(x,2) for x in Weight_Dict['Weights_1'][0][:2]],\n", " [round(x,2) for x in Weight_Dict['Weights_1'][1][:2]]])\n", "network.add_layer(2,[round(Weight_Dict['Weights_1'][0][2],2),round(Weight_Dict['Weights_1'][1][2],2)],\n", " [round(x,2) for x in Weight_Dict['Weights_2'][:2]])\n", "network.add_layer(1,[round(Weight_Dict['Weights_2'][2],2)])\n", "network.draw()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using these weights as the start of the training set, lets see what the network will do with training. " ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEPCAYAAABRHfM8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8HeO5B/DfLyEIih63IyEUdasi54hLODZRoj3FoSSh\nR9HWXVsOTaiynbaKo8elaUvqeihxJxRBZQuRSJAbsiVxiVwkLnFL4pLs/Zw/3lnW7Nmz1nrXWjNr\nzdr79/181mfPzJr1zrPXZZ6Z933nHZoZRERESulR7wBERKQxKGGIiIgXJQwREfGihCEiIl6UMERE\nxIsShoiIeEk9YZAcTLKV5GySw2OeX5/kfSSnk5xEcse0YxIRkfKlmjBI9gAwEsDBAHYCMIzk9pHV\nzgcw1cx2AfAjANekGZOIiFQm7TOMAQDmmNk8M1sJYDSAwyLr7AjgKQAws9cAbElyo5TjEhGRMqWd\nMPoAmB+aXxAsC5sO4AgAIDkAwBYA+qYcl4iIlCkLjd6XAtiA5EsATgcwFUBbfUMSEZGo1VIufyHc\nGUNO32DZV8zsUwAn5uZJvgngjWhBJDXolYhIBcyMSZST9hnGFADbkOxHsheAoQDGhFcguR7J1YPp\nnwJ42syWxRVmZpl/XHTRRXWPQXEqzkaNUXEm/0hSqmcYZtZG8gwAj8MlpxvMbBbJk93TNgrADgBu\nIdkO4BUAP04zJhERqUzaVVIws8cAbBdZdl1oelL0eRERyZ4sNHp3KU1NTfUOwYviTFYjxNkIMQKK\nM8uYdB1XWkhao8QqIpIVJGEN0ugtIiJdhBKGiIh4UcIQEREvShgiIuJFCUNERLwoYYiIiBclDBER\n8aKEISIiXpQwRETEixKGiIh4UcIQEREvShgiIuJFCUNq6tlngT32qHcUIlIJJQypqbFjgcmT6x2F\niFQi9YRBcjDJVpKzSQ6Pef5rJMeQnEZyJsnj045JRETKl2rCINkDwEgABwPYCcAwkttHVjsdwCtm\ntiuA/QH8gWTqdwIU6Y7a2wHdVkYqlfYZxgAAc8xsnpmtBDAawGGRdQzAusH0ugA+MLNVKccl0i1t\ntRVw0kn1jkIaVdoJow+A+aH5BcGysJEAdiS5CMB0AD9POSaRbuvtt4FJk+obQ3s78Pzz9Y1BKpOF\nqp+DAUw1swNIbg3gCZLfNrNl0RWbm5u/mm5qauqW99QVaXQPPwwcdpiqxtLS0tKClpaWVMpO9Z7e\nJPcE0Gxmg4P5EQDMzC4LrfMwgN+b2YRg/h8AhpvZC5GyMn9P7+eeAwYMAFbLQBp+7z1go43qHUVn\nZ50FXHWVdhb1QgLf+hYwc2Zlr58/H9h88+pjAPQdqJVGuqf3FADbkOxHsheAoQDGRNaZB+BAACC5\nCYBvAngj5bhSMXAg8OCD9Y7C2XhjV/2QNcs6nTdKI9liC2DhwnpHIfWS6rGwmbWRPAPA43DJ6QYz\nm0XyZPe0jQLwWwA3k5wRvOyXZrY0zbjStCpDzfWffVbvCDrroSt/Gt6XX9Y7AqmX1CtPzOwxANtF\nll0Xmn4Hrh1DugEmcmIskp5Vq4DVV1eVWRwd70nNfP45sHx5vaOQ996rdwTZ1tZW7wiyKwPNs9Jd\nHHWU6yEj9bVkSb0jkEalM4yE3XprvSPIrtbWekcgItVo+ITxzDPAP/5R7yjynn223hHkqQ5Wsqxe\njedvvw18+GF+ftWqbHYQyaKGTxiDBwMHHljvKPK0ky5s7tx6RyBZUq+2gn79gCOPzM+fdRbQu3d9\nYmk0DZ8wsiZLCeOUU+odQd4XX9Q7guqsWAHcfnu9o8iG+fNLr5N14TOMV1+tXxyNRgkjYVlKGE8/\nXe8I8u64o94RVOfOO4Fjj613FNnw5JP1jqByWfp9NiIljITpCxlv0aJ6R1CdE0+sdwTFvf56ttry\nsmrChHpH0NgaLmFcdx1w1131jqIwJQyphxNOyFZbXlaVcx3QW291nH/tNb/fN9l1r+VouIRxyinA\naafVOwqJMlP//noqd8iVs8+ubnsrVwJTp1ZXRj1Nm1b4udxoBO+803H59tsDEyf6la+EkSHhLL9i\nRWVlpFUP213PMO64A9h003pHUXsffFDvCJxyE8aVV1a3vRtvBPr3r66MRuS7v+mq+4GGvNK72g+j\nrQ34zneS+VCXLnW9RnbZxc23t1dfZiPqjsNNvPwysPPOyXyPWlqAtdYC9tijstfXclDHG25o/Dap\nSvl+1l01YTT8GUY1Pv3U/f3+94GRIysr44wzgF13TSaeNIwbV5vtNMKggmSyQ3N//HFyZe2/v7um\nqFLFEkbS/7dPsujTp/RQ/414cKWE0YCS+jC+9jX39+GHK+9jn/XrCw44oDbbaYSEAQAffZRcWUn/\nz9XEVuoMY8GCysuuxKJFwPjxxdd58cXaxFJKOclUCaMBpfFhdNUPuFZK7Tyz8v7+5jfAT35S+euv\nuip/cDEmeiuwOkqqSurjjzv3DkpLOd+JtKo8P/7Y9X4Ke+CBzuvl2qqUMBqQEkZ6Kj1jyvoZRq4H\n1513ujr4Sp11lruAb/lyV1ZWJJUwfvhDYKut/Nffbz/gvPMq21Y5v7mNNy59xlKJuBueDRnSedmG\nG7q/4Zjb2twBSFgu2XTV/UnqCYPkYJKtJGeTHB7z/Dkkp5J8ieRMkqtIrl+szGo/jLidW67M++5L\n54tZKI4sdb8bOxZYc83KXlsqYUSfr/UP6swzky1v3rxsJcmkEsbSMu91OX48cP31lQ26We53YL/9\n6v+eh2NevBi48MKOz+fGqFLCqADJHgBGwt1RbycAw0huH17HzK4ws93MrD+A8wC0mFmCNc1+ch/w\nkUe6o6xS6+XMm9dxvtwj9Czd0rWae4CXmzBqLekfsFnH/+mzz+rbiNuzZzLlVPI5vf8+sO++nZeX\nGo220s+kVjvjuO34brsRG/R9pH2GMQDAHDObZ2YrAYwGcFiR9YcBKDnqUNpVUoU+7HHj3JFceOCy\nJBruSOCnP62+nHpKOiG8+KKr/olauTLZ7ZSyYkX8WaBZx+W9ewOXXlq8rA8+SP59OuQQ1xupVLmX\nXeZXXpLxlep5WI+j8HL+v08+6bxMbRjp6gMgPLblgmBZJyTXAjAYwL2lCl2+HHjzzUTii1UoYbzx\nhvubxo2Arr8+P33//cl+4WrVtTZJ11/vGpijDbC9etX2Pgprrw2MGNF5uVnns8to42nOW2+56ou4\nHVC1HnsMuOee0jvC++/3K6/eZ4I+yv1tLFtW2e8pLuHFlXP55eWXDQAnnwwMHFjZa+slS43e3wfw\nbLHqqObmZgDucdNNLYlu/IUX8tOlvlx779152IA//CG5WI44onOvkBdeKD6cQTF/+lP1MZWSq0Nf\nvNivMXjbbf3K/da3Oi+rpN2nmh3hrFmdl8V9RwodaGy1FfDP/1z59tP07rsd2y2SThjPPFP4uVpV\nSa27LjBqVDLbiVs2PNQym/sO+LyPjzwCPPdc+XGV0tLSgubm5q8eSUr7Su+FALYIzfcNlsUZihLV\nUc3Nzbj4Yje9004JRFeAT/1jtK3inHMq29bo0fHLo1+43XcH1lgD+Pzz/LK77nIXHa61VvFt+P7A\nqtlZ5F6b2zFGe5pEY3j9db9yyxksrphqzti6cjXENtsAm22WP2tOOmEUGzql0nr+St7nSroKx8XX\nCJ9xU1MTmpqavpq/OLfTTEDaZxhTAGxDsh/JXnBJoVPvdZLrAdgPQIlrQ4tbuDCZ8Yx8vshJfXGO\nP770OrnBFqPbHDIknWsB0hhUrpY/tHnzgP/4D1eVVO5OacWKztWdjzzSeb1yzjCy7NNPO165nUTC\nCCf5enSBX7ECGDq0+u2UShiVvldZ6uhSrlQThpm1ATgDwOMAXgEw2sxmkTyZ5EmhVQ8HMNbMqrqz\nbmtr4RFTV650XwCf9gffhJFr00jbX/7i/tbqiOzddyvbTlqeftodBft66inXH/6yy8o/Qzn7bOAb\n3yi9Xtx76nuTqGre37gu39Xu5JPYCQL5s4n3348vu9h2y1GqPebNNwtXi5azzbg7C1abAB97DFh9\n9erKqKfU2zDM7DEz287MtjWzS4Nl15nZqNA6t5jZMeWUW+4Xu1cv4NxzgR12yI8hVYhvwvCtVklK\nrRJGFq5gDn9Gzz7bsc3o//6v+Gvjvhu+35dwD7hiqtlx/OAHlb92v/1cO1Exa65ZXt14Ugkjd3Hb\nllvGl11su+UYMgT44x/jn/vyS9cGmIRCbVf/+Afwq1+VX95NN+XPVh95pPbDtSQhS43eZZkxo/zX\n/O//ur+ldrw+O+Z99km3R0lc3W81jYTHHQccfbTf+n/+c2XbSdLf/pafjr7Pp5xS+Mzht7+tLpH7\nfqbVNFZ+VtV5dPHvwd/+5trXyununStv9uz8dz+pgRXLTRj33ed3v5tCdxf84AP3f0TFnXH4/J6i\n74MZcMUVwCWXdF53//2Ll3XiiflEV0nCyYKGTRhp9vxpb3dVWMVuy7l4sf/OZY01yr//wA47dF4W\n9wX3icEMuPVW4O673fyxxwJPPFFZWcW2UUuFtvfrX8cP/ZF0w/8ZZ/itV4xZ8u9bsYtOi8UBANtt\n54ZZB2rTHhP3v//2t/kq2GLK7doe1w3f52wy2l4cvWAzLPfe+ai0x2O9NWzCSPPovr3d1cPedFMy\n5X35JTBlSjJlVSL6w7z9duC222obQxI9sMKK7WhLVdnkvPuuO1oMVw3U8jqEQYOAgw7quCw6xPk9\n97g7vRUzZYrfjZwKXb+SZrL3PcP47DPX5uTb4SKJmOM+6+idCIs1UBf7rjRCb6pKNGzCqEapD7O9\n3e9ooZydSz170NTryzt+fP49KvZeNTUV70BQbsKIq5OPK+OWW1y71uabFy6rWtOnd16Wu8Zm3LjO\nd34cO7bj/BNPdL4gMPq/t7a6jgGlxA2qB7iu2sWuoP+3f/O/6DB61O77OU2aFH+BZC2Zda4JiMYf\nPsP48Y8Ll7XBBh3nG+GCSB8NmzA+/LBw9i/Ve6nUlc9mwDEeTfCVJIy5c5PdgQ8ZUrrxLG57cYPF\nVTPsd9zZ2Jw57u/VVxd/7dNPF0/Q5SaMauS25dv4XUrcjjZ6hbhPPOHphx4q/bq4OvtXXy28fq9e\nHednzXJXSAPu4jvfHoHDI8OLpvU5FSq31PaK/WZfecVvu7kyHn209PrlGDiwcGN+VjRswgAKj5G/\n9dbFq4CSatAr9uXbYIOO8bW3u53HttsCzz8f/xqfutuc8P9Q7GpaoOOPKHe0Gt4BfPklsNtu/tuO\nM2lSx/m2tvz784tflL5q9sc/LjxwYzUJo9wdVm5bX/96ea+rRKVHnbkdeTETJlRWds7AgfFjeZUS\nd0Tus26570Uaiejxx0tvp1gbRqWWL3dnic89V/ouhfXW0AkjKvxBVnOFcBINpB991HGnbObG9AcK\n30j+tNP8t71+0QHgOwpfeBZXH75oUfmNcHvtFV/dkrPaah3PLOJ6hUyc2HG+UMKoZtTQhQvd7UJ9\nhT/TW2/1f10cs/iL/uK2Vc468+aVd8+KSoXPjnzf79UiY0dUmjBKDf+SRsL4r/8qvZ1CCaOai/FG\njMh3csn6xZ9dKmGE7b9/vpok2mum3C/bxImVHVWE64Xb2/M7xGLbj9Zh18Luu5f/mkmT8l0bC9Wf\nl+r6/MUXrgdZTqEfXXhgxqi4vvJhr77qdw/qnPDnXG1PlqVLi49g63MPi7jv3bRpbqiLauv85871\nX9f3NxO9KM2nahfo/H+WurgxGk81Q/OXs51CCaOabtbhM0YljDrKdYuN1s2HP5SZMzu/LvwlGTo0\nv+OLfnlKJZFowvCRVHVZOcJX5lai0juu/eUvHXvuFGp4jduxLVvmEsGOOxZvbzj22PjlhcYWCn+m\naXYWuPJKv/KLVcf5DlleyKmn+q9b6RlGMcuX58dSK9Rj6eWXi8eTu4C2X7/4e3OXirvQ96OQNKqk\nouVnWZdKGL475ZNCg5J8+9vF173zzsKnx4W+OLmqlvARczi2Yl+K8E6z3K64t9/ujtrj6mKrZebG\nZ4ouAzpXLfnKXReSU+peF8cdl58eNChf1VSs+iI3WGP0wOB3vysdX7nXzkQV27GcfXY+7kJ3uXv/\n/fzwFOH3xnenUqpzR7SHVjG+2yznzn/33AMMG+am4+7IeOWVha/qz8XT0uIGUAQ6dxtuby99G4RK\nzujLaVPbccfO7XtRN9+cn9YZRg1Vm51zV+BGyzn99Pj1C+0Q9t7b/Q0njIcf9oshvGMYMMDvNTnH\nHuvq3Q8+uLzXFdPWlj/Cz92vOMcsvt63UqV+LOE2hVwPrFwcpfhe6JXk0ePkyX7rRXs0LVrk7gWy\n0Ub5HlGVdEo44ICO87UYsbfQ+zd1aunqw7BwL75iBxLRDgDhOHv2LH72HL0GJo5vlVSh92fWLL8u\nz6XKyYqGThhpXSVbqC49+kWpdOfie4ZRiaSPUG6+ueO9K8Kxm+WHW6mnSr8H1Yw5VUjuiPb88zt3\nu4y7t0ecd94Borcx8OnymSbf97hQx4X+/d1wOoXKjL7vuWtGzOKv3F+yxL0mPO5ZXIzFqkvD98Ap\nJDqMS9o7dJ1hNJBSDWfRo5VKdy7Fhl6u5AuZZp1q9DqCaMJI0rx57n+55ZZky41z/fWuSiRp3/iG\n+x9+/3vXUy6sUO+4ONHrIsLKed9vvDE/Xe795oH8EbzvNstpD/M5ODKLH3srtyzcISKuarTaO3PG\nJYw0rwtSwkhR+EIksrKbpJSj3KP/QjvypUurb2gOCw+2l9QNhwDXLTg65ESaCePAA93fcurWq4kj\n7nqYJBN2Ne9PUkNgh69GrqQn0brrJhNHnFw17fTpyRz0xFX9lNOmEqfaKinAHTxUur2saaiEEf2g\nvvOdjvO+YwjVSrF7/W60UfzyQvfzKOaCC/IX/ETHwqnG1lvnG4dzP7wZM/JVdkkP5ZArt9qLzrqC\nYgmjHjuVpAdvDFu+PJmkG9dJpNpEdM89HbtlV3KFeTk9HydPLn0hbj01VMLImlJfxnHjyv9xX3RR\nZbEk2fhcTP/+HXt1JCn3fpZbjZDkDjTJ6j3fC//iknyxI+N6VFtU8x6X+m4W66paznanTUt+kE+z\njmcItUjWxYZwqbfUEwbJwSRbSc4mObzAOk0kp5J8mWSJzoD1E+2m6LNzueaadGKpp2rv51BIGp0I\nfOV6NKWVDIsJ36cdKP0+1OMM48UXC18TERYXW6mOEUn+P9GbU9Vq0L+sVyUlpYzLbMpHsgeAkQAG\nAVgEYArJB82sNbTOegD+BOAgM1tIcsPKt1dtxMU99VTH+egPPU7czVzSkPXGsqwr52rwWsjC6Kbh\nrq1nngmss071ZcZ1FS81fEg5O+PoNTnVtmHExVPO8kr4DFVfL2mfYQwAMMfM5pnZSgCjARwWWecY\nAPea2UIAMLMEm4PTFXfXrajw0BdpqqQ7Lln4/gOF7p2QRUn8WP/6V+B736u+nKQUSxi+13dUK40k\nGldllGRHjajw+7hwYfwozaWMHJmfrkXCyPLd+FI9wwDQB0D4VuoL4JJI2DcBrB5URa0D4Bozq3LY\nt9rwqZop1j0ySZV+YQsNtV2rRBdWTZXUihV+CbyQYoME1tq//Eu9I6itQw4pPFpAtUNxhM8w+vat\nvJxwPPfeG7+8O0g7YfhYDUB/AAcAWBvARJITzSxmBKHm0HRT8Mir9Wm8z/aSPiUupNIvbJau+6im\n3Bdf9BvuQxqLWXW98Wq1Tyg1um4ttbS0oKWc+8WWIe2EsRDAFqH5vsGysAUA3jezzwF8TnI8gF0A\nlEgYnWUxYdRKV2jDqPT97Ns3+ZvZSGc+9+EoptCYWcXMnFld9WjSv9FC5cXd8rhe+4empiY0NTV9\nNX9x9MbkVUj7+HcKgG1I9iPZC8BQAGMi6zwIYB+SPUn2BrAHgDJGnclLq/dOIT73Hy73IrRKVZow\njjgi2TiqUc2wKL/4RXJxSOUqPdMtdBX8iy9WHksaCt2WNc12mCyhpVz5RnIwgKvhktMNZnYpyZMB\nmJmNCtY5B8AJANoA/NXMOt2okKQB5cea9nDEaYvGn/u46vk/rbNO9UebSdtkk8ouepRkDRkSf3vY\nSmXxu1YLyV5bRJhZInuM1NswzOwxANtFll0Xmb8CwBVpx9JV1Hsguu74AxY/SR9/6ruWLV3+Su9i\nN9dpBNGqJrK84ZK7C51dZEN36S3UXaVeJZWUSqukREQaTVarpLr8GYaIiCRDCUNERLwoYYiIiBcl\nDBER8aKEISIiXpQwRETEixKGiIh4UcIQEREvShgiIuJFCUNERLwoYYiIiBclDBER8aKEISIiXlJP\nGCQHk2wlOZvk8Jjn9yP5EcmXgscFacckIiLlS/UGSiR7ABgJYBCARQCmkHzQzFojq443s0PTjEVE\nRKqT9hnGAABzzGyema0EMBrAYTHrNfBNVEVEuoe0E0YfAPND8wuCZVF7kZxG8u8kd0w5JhERqUDq\n9/T28CKALcxsBclDADwA4Jt1jklERCLSThgLAWwRmu8bLPuKmS0LTT9K8s8kv25mSzsX1xyabgoe\nIiKS09LSgpaWllTKTvWe3iR7AngNrtH7HQCTAQwzs1mhdTYxsyXB9AAAd5nZljFl6Z7eItItNOw9\nvUn2JHlFJYWbWRuAMwA8DuAVAKPNbBbJk0meFKz2A5Ivk5wK4CoAQyrZloiIpMvrDIPkJDPbswbx\nFItBZxgi0i1k9QzDtw1jKskxAO4GsDy30MzuSyIIERHJPt+EsSaADwAcEFpmAJQwRES6iVQbvZOk\nKikR6S6yWiXldeEeyb4k7yf5bvC4l2TfJAIQEZHG4Hul900AxgDYLHg8FCwTEZFuwreX1DQz27XU\nsjSpSkpEuouGrpIC8AHJHwbXZPQk+UO4RnAREekmfBPGiQCOBrAY7ortHwA4Ia2gREQke0p2qw2G\n9zhC96sQEeneSp5hBMN7DKtBLCIikmG+jd5XAlgdwJ3oeKX3S+mF1ikGNXqLSLeQ1UZv34QxLmax\nmdkBMctToYQhIt1FwyaM4L7cPzCzu5LYYKWUMESku8hqwvBpw2gH8MskNiYiIo3Lt0rqUgDvo3Mb\nRsxd8dKhMwwR6S6yeobhmzDejFlsZvaNJILwoYQhIt1FVhOG14V7ZrZVzMMrWZAcTLKV5GySw4us\ntzvJlSSP8A1eRERqp2jCIPnL0PRRkecuKVV40GA+EsDBAHYCMIzk9gXWuxTAWL+wRUSk1kqdYQwN\nTZ8XeW6wR/kDAMwxs3lmthLAaACHxax3JoB7ALzrUaaIiNRBqYTBAtNx83H6AJgfml8QLMsXQm4G\n4HAz+4tnmSIiUgelEoYVmI6br9RVAMJtG0oaIiIZVGrwwV1IfgK3E18rmEYwv6ZH+QsBbBGa7xss\nC/tXAKNJEsCGAA4hudLMxnQurjk03RQ8REQkp6WlBS0tLamUneo9vYORbl8DMAhuWPTJAIaZ2awC\n698E4CEzuy/mOXWrFZFuIavdaksOb14NM2sjeQaAx+Gqv24ws1kkT3ZP26joS9KMR0REKpfqGUaS\ndIYhIt1FVs8wfO+4JyIi3ZwShoiIeFHCEBERL0oYIiLiRQlDRES8KGGIiIgXJQwREfGihCEiIl6U\nMERExIsShoiIeFHCEBERL0oYIiLiRQlDRES8KGGIiIgXJQwREfGihCEiIl5STxgkB5NsJTmb5PCY\n5w8lOZ3kVJKTSQ5MOyYRESlf2vf07gFgNtw9vRcBmAJgqJm1htbpbWYrgumdAdxlZjvElKU77olI\nt9Bd77g3AMAcM5tnZisBjAZwWHiFXLIIrAOgPeWYRESkAmknjD4A5ofmFwTLOiB5OMlZAB4CcGLK\nMYmISAVWq3cAAGBmDwB4gOQ+AH4L4DvxazaHppuCh4iI5LS0tKClpSWVstNuw9gTQLOZDQ7mRwAw\nM7usyGteB7C7mS2NLFcbhoh0C921DWMKgG1I9iPZC8BQAGPCK5DcOjTdH0CvaLIQEZH6S7VKysza\nSJ4B4HG45HSDmc0iebJ72kYBOJLkcQC+BPAZgKPTjElERCqTapVUklQlJSLdRXetkhIRkS5CCUNE\nRLwoYYiIiBclDBER8aKEISIiXpQwRETEixKGiIh4UcIQEREvShgiIuJFCUNERLwoYYiIiBclDBER\n8aKEISIiXpQwRETEixKGiIh4UcIQEREvqScMkoNJtpKcTXJ4zPPHkJwePJ4luXPaMYmISPlSveMe\nyR4AZgMYBGAR3D2+h5pZa2idPQHMMrOPSQ4G0Gxme8aUpTvuiUi30F3vuDcAwBwzm2dmKwGMBnBY\neAUzm2RmHwezkwD0STkmERGpQNoJow+A+aH5BSieEH4C4NFUIxIRkYqsVu8AckjuD+AEAPsUXqs5\nNN0UPEREJKelpQUtLS2plJ12G8aecG0Sg4P5EQDMzC6LrPdtAPcCGGxmrxcoS20YItItdNc2jCkA\ntiHZj2QvAEMBjAmvQHILuGTxn4WShYiI1F+qVVJm1kbyDACPwyWnG8xsFsmT3dM2CsCvAXwdwJ9J\nEsBKMxuQZlwiIlK+VKukkqQqKRHpLrprlZSIiHQRShgiIuJFCUNERLwoYYiIiBclDBER8aKEISIi\nXpQwRETEixKGiIh4UcIQEREvShgiIuJFCUNERLwoYYiIiBclDBER8aKEISIiXpQwRETEixKGiIh4\nST1hkBxMspXkbJLDY57fjuRzJD8neXba8YiISGVSvUUryR4ARgIYBGARgCkkHzSz1tBqHwA4E8Dh\nacYiIiLVSfsMYwCAOWY2z8xWAhgN4LDwCmb2vpm9CGBVyrGIiEgV0k4YfQDMD80vCJaJiEiDSbVK\nKnnNoemm4CEiIjktLS1oaWlJpWyaWSoFAwDJPQE0m9ngYH4EADOzy2LWvQjAp2b2vwXKMiC9WEVE\nsiLJ3TJJmBmTKCvtKqkpALYh2Y9kLwBDAYwpsn4i/5SIiCQv1TMMwHWrBXA1XHK6wcwuJXky3JnG\nKJKbAHgBwLoA2gEsA7CjmS2LlKMzDBHpFrJ6hpF6wkiKEoaIdBdZTRi60ltERLwoYYiIiBclDBER\n8dJQCWPq3oM1AAANxElEQVTixPz0+efXLw4Rke6ooRq9zQwMmm7MgIceAg49tL5xiYgkTY3eKejd\nu94RiIh0Hw2dMA44ID99yy2dn3/ppdrFIiLS1TV0wmDoJOu444DDgwHSf/1r4MYbgd12A9ragKlT\n6xOfiEhX0mCDDwJ77QUceWTh5x54APjv/84v69ED2HXX2sQmItKVNVzCeO65ws+tsUZ1ZZ94ojsz\nERGRzhq6SirqqKNcdVScJ5/MT99/f/w611yTfEwiIl1Fl0gYud5Sm23WsToqbNAg4M9/dtPbb+/+\n9uwJXHxxfp011yxvu7feWt76IiKNrOETxh//6B4+Tj3V9W/eYAM3v2oVcOGF+ed79sxPX3GF+/v3\nvwOTJ7sG9W23zT9/7rkdG93DJkwArr4aWL4c+OST/PLzzweeeaZwtdrhhwNnndVxWd++fv9bJX7y\nk/TKFpEuyMwa4uFCTcayZWbh4vbay+zBB930tGlmCxbEv+7mm93r7rjDbNUqs9dfd/OA2W9+4/5e\ncEHn151zjis3LPe6F15wf8eOdctXrHDzPXu6+ba2/LqA2dFHm82Y0XFZ7vH22/HLCz3MzHbZpbzX\nNDeb7bOP2ZIl5b1ODz308H8sWlT5/i0q2HciiUcihdTikWTCqFR7u9nHH8c/N3asezfjEkaczz83\nmznTb92ePfNfpJzc/D/9k9lDD+WXP/VU5y/f7rvnp88+2+ysszqWtXSp2fPPd3zNypVmTU1mzzxj\ndvrpLsm+807HuFasMBs2LP4Lv+mm+enrrvP7kTz3XGU/roceMrvvvnR/wHroUcvHpZf67Rt8NFTC\nADAYQCuA2QCGF1jnGgBzAEwDsGuBdZJ7B1OwapXZdtsVTijVOuUUs2uvzc8vXGj25Zfx60a/fLfd\nZnbUUS6ZmLnE9957nV938cVu/XKPbpYtM+vb1+y00/LbNDO7/XazjTeOj2mTTcw+/NAlIcCsV698\neW++2XHdjz5yy1tb88tuvNFs8WKzUaPyr/v8c/fc0qX59Y4+uvCP8qWX8tMrV7r3s6nJzS9YYDZ3\nrt+PO3wWOG6ciwkwGzmy8+eQm37ggfx0e7t7nwCz73/f7MQT888984z73z77rPN2v/vd4nE1N5ud\neWbp+MMHJNFH7qx18GC/96Kax6uvdl52553pbzeLj7lzy/sNFtMwCQOujWQugH4AVg8SwvaRdQ4B\n8Pdgeg8AkwqUldw7mKJx48bVOwQ75hj3yU6YUHiduDjb280+/TSdmEaM6PiDKGXePLPf/c7srrvG\nlbWdVavily9danbggWbHHmv285/nl3/yiUu+OS+/bHbvvfn5K690Z2W5uHNJcdEis/79zd54w623\n++7jbPz4+DjGjs0n6OOPz38ub73lkp6Z2cSJLgnmzJ3rEmdY7nXt7WYffOCmn3/e7P338/G9846L\nf+VK9/ykSW753Xeb7bvvuA6fwf/8j9n8+W7dxx4zO//8/HOTJrn3Jiz82kMOid/R3Xab2c475+dP\nOsnsllvMtt7anX2H133rLfc/zZ6d38a115rtsMM4u+22/P/43nudt/OjH7kz3Ecece8H4JLrY491\nXK+tzezUU83Gj3eP8HPz53ecj74WcLEtXpyfnzMnf1AB5N/PL77o/Foz95274AKz733PnTUAZlts\n0XG9vffOT++zj/t/ktRICWNPAI+G5kdEzzIAXAtgSGh+FoBNYspK9E1My0UXXVTvEMys9BFKPeKc\nOtW1mVx4of9rsvJ+vvdex6q/qHrH+emn8WeNZmbTp7ud0K9+ddFXO6bjjy9/G7nX3nVXftlnn8Un\n6d12M7vkErfDDsu1HwLujDBO3Hs5fLjbcb/9ttmUKYXPrnMKHThcfrnb9oABbv6TT9z86NH5dXJn\nUytW5Je1tpoNGWK2fLk7M543z+zCCy+yCRNcEjEze/JJ9zn4nB3cf7/Zz37mys355JPCcVcjyYSR\n9oV7fQDMD80vADCgxDoLg2VL0g2ta9t663pH0NmuuwLTptU7ispsuCHw7/9e7ygKW2cd94jz7W+7\nv6ut5nbVlTID3noL2HLL/LJCXdELjeO29trAlCnA0qXlXWh76aX56c03L71+uMdj2LnnukfOuut2\nfk8efbTz67bbDhg92k337g2sv77rJbn33vl1Bg1yfwt9DmGHH54fyigcS9Y13JXeIlI/4WRRqX/9\n1+rLkPpI9X4YJPcE0Gxmg4P5EXCnR5eF1rkWwDgzuzOYbwWwn5ktiZSVXqAiIl2YJXQ/jLTPMKYA\n2IZkPwDvABgKYFhknTEATgdwZ5BgPoomCyC5f1hERCqTasIwszaSZwB4HK7H1A1mNovkye5pG2Vm\nj5D8Lsm5AJYDOCHNmEREpDINc4tWERGpr4YYS4rkYJKtJGeTHF7jbd9AcgnJGaFlG5B8nORrJMeS\nXC/03Hkk55CcRfKg0PL+JGcE/8NVKcTZl+RTJF8hOZPkz7IYK8k1SD5PcmoQ50VZjDMovwfJl0iO\nyWqMwTbeIjk9eE8nZzFWkuuRvDvY5isk98hgjN8M3sOXgr8fk/xZ1uIMyj+L5MvBNv5GsldN4kyq\nf25aD3hc/Jfy9vcBsCuAGaFllwH4ZTA9HMClwfSOAKbCVfVtGcSdO4t7HsDuwfQjAA5OOM5NEVwl\nD2AdAK8B2D6jsfYO/vYEMAmuq3UW4zwLwG0AxmT1cw/KfQPABpFlmYoVwM0ATgimVwOwXtZijMTb\nA8AiAJtnLU4AmwWfea9g/k4AP6pFnIm/0Sl8cCUv/qtBDP3QMWG0Iri4EG5H3RoXG4BH4a5e3xTA\nq6HlQwH8JeWYHwBwYJZjBdAbwAsAds9anAD6AngCQBPyCSNTMYbKfRPAP0WWZSZWAF8D8HrM8szE\nGBPbQQCeyWKccAljHoAN4JLAmFr91huhSiru4r8+dYolZ2MLenKZ2WIAGwfLC12E2Acu7pxU/weS\nW8KdFU2C+wJlKtagqmcqgMUAnjCzKRmM80oA5wIIN/JlLcYcA/AEySkkc4PWZynWrQC8T/KmoLpn\nFMneGYsxagiA24PpTMVpZosA/AHA28E2PzazJ2sRZyMkjEaQmZ4DJNcBcA+An5vZMnSOre6xmlm7\nme0GdxQ/gOROyFCcJL8HYImZTQNQrDt33d/LwEAz6w/guwBOJ7kvMvR+wh0F9wfwpyDO5XBHvVmK\n8SskVwdwKIC7g0WZipPk+gAOg6v52AzA2iSPjYkr8TgbIWEsBLBFaL5vsKyelpDcBABIbgrg3WD5\nQrg6z5xcrIWWJ4rkanDJ4lYzezDLsQKAmX0CoAVuROMsxTkQwKEk3wBwB4ADSN4KYHGGYvyKmb0T\n/H0PripyALL1fi4AMN/MXgjm74VLIFmKMewQAC+a2fvBfNbiPBDAG2a21MzaANwPYO9axNkICeOr\ni/9I9oKrZxtT4xiIjkeaYwAcH0z/CMCDoeVDgx4LWwHYBsDk4PTwY5IDSBLAcaHXJOlGuDrJq7Ma\nK8kNc703SK4F4DtwA05mJk4zO9/MtjCzb8B9354ys/8E8FBWYswh2Ts4qwTJteHq3mciW+/nEgDz\nSX4zWDQIwCtZijFiGNyBQk7W4nwbwJ4k1wzKHwTg1ZrEmUaDUQoNUIPhev3MATCixtu+Ha63xBfB\nB3UCXGPTk0FMjwNYP7T+eXC9EGYBOCi0/F/gfshzAFydQpwDAbTB9SKbCuCl4H37epZiBbBzENs0\nADMA/CpYnqk4Q9vYD/lG78zFCNc+kPvMZ+Z+H1mLFcAucAd/0wDcB9dLKlMxBuX3BvAegHVDy7IY\n50XBNmcAuAWuB2nqcerCPRER8dIIVVIiIpIBShgiIuJFCUNERLwoYYiIiBclDBER8aKEISIiXpQw\npEsj+Wnwtx/J6N0eqy37vMj8swmX/81g/CWSfC7JskUqoYQhXV3uQqOtABxTzgtJ9iyxyvkdNmS2\nTznle9gXwHi4ix1nJly2SNmUMKS7+D2AfYLRUn8ejJh7Od3NnKaR/CkAkNyP5HiSD8INXwGS9wcj\nwc7MjQZL8vcA1grKuzVY9mluYyT/J1h/OsmjQ2WPY/5GQrfGBUpyn2A038sBnAPg7wAOZnBzJJF6\n0ZXe0qWR/MTMvkZyPwD/ZWaHBst/CmAjM7skGKNsAoAfwN1g5mEAO5nZ28G665vZRyTXhBve4t/M\n7MNc2THbOhLASWZ2MMmNg9cMgLuh1QNwN7RZHGzzHDOLrW4iOcHMBpK8EcDlZtaa/Dsk4k9nGNJd\nHQTguOBI/nm4cXi2DZ6bnEsWgV+QnAZ3f5G+ofUKGYhg8DozexduRN7dQ2W/Y+5IbRpcguqE7n4R\nXwSz28KN9SNSV6vVOwCROiGAM83siQ4L3ZnI8sj8AQD2MLMvSI4DsGaoDN9t5XwRmm5DzG8wqA7b\nHsB6JKfD3fdgCsnfm9nd0fVFakVnGNLV5XbWnwJYN7R8LIDT6O4hApLbBkf1UesB+DBIFtvD3TI4\n58vc6yPbegbAkKCdZCO4xmvv9gczOwzAXwGcCuBnAK41s/5KFlJvShjS1eUa6WYAaCc5leTPzeyv\ncPcQeInkTADXAojrFfUYgNVJvgLgEgATQ8+NAjAj1HhtAGBm9wfbmw433PS5QdVUodji7Avg2eDv\n06X/TZH0qdFbRES86AxDRES8KGGIiIgXJQwREfGihCEiIl6UMERExIsShoiIeFHCEBERL0oYIiLi\n5f8BNDh9I1nFSEEAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "Train_Set = [[1.0,1.0],[0.0,0.0],[0.0,1.0],[1.0,0.0]] #train set\n", "\n", "Error = []\n", "while True:\n", " Train_Dict = neural_net(choice(Train_Set),Weight_Dict['Weights_1'],Weight_Dict['Weights_2'],Training=True)\n", " \n", " Error.append(abs(Train_Dict['error']))\n", " if len(Error) > 6 and np.mean(Error[-10:]) < 0.025: break\n", " \n", "Error_vec = np.array(Error)[:]\n", "plt.plot(Error_vec)\n", "plt.ylabel('Error')\n", "plt.xlabel('Iteration #');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see the network never reduces error. Let's see how the network answers to the different input patterns." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWkAAAEPCAYAAACA1jUJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEDNJREFUeJzt3X+MXNV5xvHn2QULNhBakthpDRhskqAgIRpSy4mjslGk\nYCKlbqMq2KhVa7WRFcVqVLUNbpUKIyE1IFo1Ea0iVxYRFHAaUYJVQWP+yJbaCbHVAC6JLTuBNT9i\nmUaYxJhsbO+8/WOu3fF6Z+aOZ+7Mu7PfjzTaueeeO+cdaXl8OPfMrCNCAICcRgZdAACgOUIaABIj\npAEgMUIaABIjpAEgMUIaABIjpAGgA7a32D5se0+LPl+xfcD2s7avb2hfZXuf7f22byszHiENAJ25\nT9JNzU7avlnSsoh4j6T1kr5atI9Iure49lpJa21f024wQhoAOhAROyQdadFltaT7i77fk3SJ7UWS\nlks6EBEHI+KEpK1F35YIaQDorcWSXm44fqVoa9beEiENANVyNxef16squmGbLxABUEpEdBV6V9px\nsHz3wxHx7g6HeFXS5Q3HlxVtCyRdMUt7SylCWpJibGzQJaSw6fhxbVqwYNBlDNyFtWODLiGNEyc2\n6fzzNw26jBSmprrKZ0nSQUkxUm4RwbXaoman1HyGvE3S5yR93fYKSW9ExGHbP5V0te0lkg5JWiNp\nbbsa0oQ0APRNyZBWrXZWk+2HJI1LeoftlyTdrvosOSJic0Q8bvsTtn8k6ZikdaqfnLa9QdJ21Zea\nt0TE3nYlENIA5p+yIT2LiLi1RJ8NTdr/Q9L7OhmPkE5mfHR00CUgmZGR8UGXMHzOKxl9x49XW0cJ\nzvCl/7aDNWk0Yk0as5mactc3Dm1HXHRRub5vvtn1eN1iJg1g/uliuaPfCGkA8w8hDQCJEdIAkBgh\nDQCJEdIAkFjZLXgJzJ1KAaBXmEkDQGKENAAkRkgDQGKENAAkRkgDQGKENAAkxhY8AEiMmTQAJEZI\nA0BihDQAJEZIA0BihDQAJEZIA0BibMEDgMSYSQNAYoQ0ACRGSANAYoQ0ACRGSANAYuzuAIDEmEkD\nQGKENAAkRkgDQGKENAAkRkgDQGKENAAkxhY8AEiMmTQAJEZIA0BihDQAJEZIA0BihDQAJEZIA0Bi\nbMEDgMSYSQNAYoQ0ACRGSANAYoQ0ACQ2h0K60kptb7F92PaeKscBgI6MjJR7NGF7le19tvfbvm2W\n839h+xnb37f9P7ZP2v6V4tyk7eeK87valtrVG23vPkk3VTwGAHTmvPPKPWZhe0TSvapn27WS1tq+\nprFPRNwTEb8RER+Q9FeSJiLijeJ0TdJ4cX55u1IrDemI2CHpSJVjAEDHuptJL5d0ICIORsQJSVsl\nrW4x2lpJDzccWx1k79xZmAGAXukupBdLernh+JWi7Sy2L5S0StIjDc0h6Unbu21/pl2paW4cbjp+\n/PTz8dFRjY+ODrAaABlMT0+oVpvo/Qs3CeCJyUlNTE72cqRPStrRsNQhSSsj4pDtd6ke1nuLVYdZ\n5QnpBQsGXQKAZEZHxzU6On76eHr6jt68cJOQHl+6VONLl54+vuOpp2br9qqkKxqOLyvaZrNGZy51\nKCIOFT//1/ajqi+fNA3pfix3uHgAQA7dLXfslnS17SW2F6gexNtmdrJ9iaQbJT3W0DZm+6Li+dsk\nfVzS861KrXQmbfshSeOS3mH7JUm3R8R9VY4JAG11sU86IqZtb5C0XfWJ7paI2Gt7ff10bC66/o6k\nb0XELxouXyTpUduhev4+GBHbW43niDjnYnvFdsTY2KDLQCIX1o4NugQkNDVlRURX/2duO+JLXyrX\nd+PGrsfrVpo1aQDomzn0iUNCGsD8Q0gDQGKENAAkRkgDQGKENAAkxt84BIDEmEkDQGKENAAkRkgD\nQGKENAAkRkgDQGKENAAkxhY8AEiMmTQAJEZIA0BihDQAJEZIA0BihDQAJEZIA0BibMEDgMSYSQNA\nYoQ0ACRGSANAYoQ0ACRGSANAYoQ0ACTGFjwASIyZNAAkRkgDQGKENAAkRkgDQGKENAAkRkgDQGJs\nwQOAxObQTLplpbbfbnvZLO3XVVcSAFRsZKTcI4GmVdj+tKR9kh6x/QPbv9lw+mtVFwYAlRmGkJb0\n15JuiIjrJa2T9IDt3y3OufLKAKAqcyikW61Jj0bEIUmKiF22Pyrp321fLin6Uh0AVCFJAJfRKqSP\n2l4WET+WpIg4ZHtc0jclXduP4gCgEkOyu+OzmrGsERFHba+S9OlKqwKAKg3DTDoinmvSfkLSg5VV\nBABVG4aQBoChRUgDQGJzKKTbVmr782XaAGDOmENb8MpU8YeztP1Rj+sAgP6ZQyHddLnD9lpJt0q6\nyva2hlMXS3q96sIAoDJDsgXvO5IOSXqnpL9raD8qaU+VRQFApbqcJRdbkf9B9dWILRFx14zzN0p6\nTNILRdO/RcSdZa49a6yIwX940HbE2Nigy0AiF9aODboEJDQ1ZUVEV19LYTtiaqpc3wsuOGs82yOS\n9kv6mKSfSNotaU1E7Gvoc6OkP4+I3+702pnK3Dg8avvnxWPK9rTtn5d6hwCQUXdr0sslHYiIg8Xn\nRrZKWj1Lv9n+MSl77WltF2Yi4uLTI9ouXnBFu+sAIK3uljsWS3q54fgV1cN3pg/ZflbSq5L+MiJ+\n2MG1p3W0eh71tZFv2r5d0sZOrgWANJqE9MTEhCYmJnoxwn9LuiIi3rJ9s+rfefTec3mhtmvStj/V\ncDgi6YOSboyID53LgE3GiNdeG/zaOIDcFi7szZp0rVYub0ZGzh7P9gpJmyJiVXG8UfU5bNMbgLZf\nlHSD6kHd0bVlZtKfbHh+UtKk2qyhAEBmJ092dfluSVfbXqL6Drg1ktY2drC9KCIOF8+Xqz4hft12\n22tnKrMmve6c3gYAJFWrnfu1ETFte4Ok7fr/bXR7ba+vn47Nkn7P9mclnZD0C0m3tLq21XhlljuW\nSvqy6jcLQ9J3Jf1ZRLzQ8sIOsNwBoIxeLXe89Va5vBkb6368bpW5xfmQpH+V9GuSfl3SNyQ9XGVR\nAFClWq3cI4MyIT0WEQ9ExMni8S+SLqi6MACoylwK6TI3Dp8o7kBuVX254xZJj9u+VJIigu/xADCn\nZAngMsqsSb/Y4nRExNKui2BNGkAJvVqTfv31cnlz6aWDX5Mus7vjqn4UAgD90uUWvL4q9YlD2x+W\ndGVj/4i4v6KaAKBSc2m5o21I235A0jJJz0qaLppDEiENYE4aqpBW/WPg748M32kKAD0wbCH9vKR3\nq/4RRgCY84YtpN8p6Ye2d0n65anGmV9mDQBzxbCF9KaqiwCAfhqqkI6I/+xHIQDQL0OxBc/2UdV3\ncZx1SvUPsby9sqoAoEJDMZNu/LNZADBMhiKkAWBYEdIAkBghDQCJEdIAkNhQ7O4AgGHFTBoAEiOk\nASAxQhoAEiOkASAxQhoAEiOkASAxtuABQGLMpAEgMUIaABIjpAEgMUIaABIjpAEgMUIaABJjCx4A\nJMZMGgASI6QBIDFCGgASI6QBIDFCGgASI6QBIDG24AFAYsykASAxQhoAEiOkASAxQhoAEiOkASAx\nQhoAEmMLHgAkNpdm0iNVD2B7le19tvfbvq3q8QCgnVqt3KOZdrlm+1bbzxWPHbavazg3WbQ/Y3tX\nu1ornUnbHpF0r6SPSfqJpN22H4uIfVWOCwCtdDOTLplrL0j6rYj4me1VkjZLWnFqeEnjEXGkzHhV\nz6SXSzoQEQcj4oSkrZJWVzwmALTU5Uy6ba5FxNMR8bPi8GlJixtOWx1kb9UhvVjSyw3Hr+jMYgGg\n77oM6U5z7U8kPdFwHJKetL3b9mfa1ZrmxuHdd286/XzlynGtXDk+sFoA5LBz54R27pzo+es2290x\nOTmhycnejWf7o5LWSfpIQ/PKiDhk+12qh/XeiNjR9DUiomcFzVLgCkmbImJVcbxRUkTEXTP6xWuv\nVVcHgOGwcKEVEe7mNWzHF79YLm/uvPPs8TrIteskPSJpVUT8uEktt0s6GhF/36yGqpc7dku62vYS\n2wskrZG0reIxAaClLpc72uaa7StUD+g/aAxo22O2Lyqev03SxyU936rWSpc7ImLa9gZJ21X/B2FL\nROytckwAaKeb3R3Ncs32+vrp2CzpbyRdKumfbFvSiYhYLmmRpEdth+r5+2BEbG81XqXLHWWx3AGg\njF4td3zhC+Xy5u67ux+vW2luHAJAv8ylTxwS0gDmHUIaABLjC5YAIDFm0gCQGCENAIkR0gCQGCEN\nAIkR0gCQGCENAImxBQ8AEmMmDQCJEdIAkBghDQCJEdIAkBghDQCJEdIAkBhb8AAgMWbSAJAYIQ0A\niRHSAJAYIQ0AiRHSAJAYIQ0AibEFDwASYyYNAIkR0gCQGCENAIkR0gCQGCENAIkR0gCQGFvwACAx\nZtIAkBghDQCJEdIAkBghDQCJEdIAkBi7OwAgMWbSAJAYIQ0AiRHSAJAYIQ0AiRHSAJAYIQ0AibEF\nDwASYyYNAInNpZAeGXQBANBvtVq5RzO2V9neZ3u/7dua9PmK7QO2n7V9fSfXNiKkk9m5c2LQJSAZ\nfid6r5uQtj0i6V5JN0m6VtJa29fM6HOzpGUR8R5J6yV9tey1MxHSyfAfJGbid6L3upxJL5d0ICIO\nRsQJSVslrZ7RZ7Wk+yUpIr4n6RLbi0peewZCGsC802VIL5b0csPxK0VbmT5lrj0DNw4BzDsD2ILn\nc70wTUgvXHjO72Ho3HPPHYMuAcnwO9FTByUvKdn38Cxtr0q6ouH4sqJtZp/LZ+mzoMS1Z0gR0hFB\nQgPoi4i4ssuX2C3pattLJB2StEbS2hl9tkn6nKSv214h6Y2IOGz7pyWuPUOKkAaAuSIipm1vkLRd\n9ft6WyJir+319dOxOSIet/0J2z+SdEzSulbXthrPEVHpGwIAnDt2dyTR6QZ3DD/bW2wftr1n0LVg\ncAjpBM5lgzvmhftU/53APEZI59DxBncMv4jYIenIoOvAYBHSOXS8wR3A/EBIA0BihHQOZTbHA5iH\nCOkcTm+Ot71A9Q3u2wZcE3KwuvhIMeY+QjqBiJiWdGqD+w8kbW23wR3Dz/ZDkr4j6b22X7K9btA1\nof/4MAsAJMZMGgASI6QBIDFCGgASI6QBIDFCGgASI6QBIDFCGpWwfbSC11xiu+lfsbD9hO0jtvkg\nEIYGIY2qVLEB/ypJt7Y4f7ek369gXGBgCGlUyvaNtr9t+xu299p+oOHci7bvsr3H9tO2lxbt99n+\nVEO/U7Pyv5X0Edvft/35mWNFxLclvVnxWwL6ipBGP1wv6U8lvV/SMtsfbjh3JCKuk/SPkr7c5PpT\ns/KNkv4rIj4QEc36AkOFkEY/7IqIQ1H/DoJnJV3ZcG5r8fNhSSv6XRiQHSGNfvhlw/NpnflX6mOW\n5ydV/G7atqQFlVYHJEZIoyplv17zluLnGknfLZ5PSvpg8Xy1pPOL50clXVxiXL7aE0PjvPZdgHPS\nbHfHzPZftf2cpClJp7bX/bOkx2w/I+lbko4V7Xsk1Yr2r81cl7b9lKT3SbrI9kuS/jginuz+rQCD\nw1eVYmBsvyjphoh4fdC1AFmx3IFBYoYAtMFMGgASYyYNAIkR0gCQGCENAIkR0gCQGCENAIkR0gCQ\n2P8B+WfePJyqM0UAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "Weights_1 = Weight_Dict['Weights_1']\n", "Weights_2 = Weight_Dict['Weights_2']\n", "\n", "Train_Set = [[1.0,1.0],[0.0,1.0],[1.0,0.0],[0.0,0.0]] #train set\n", "\n", "tempdict = {'output':[],'target':[],'input':[]} #data dictionary\n", "temp = [neural_net(Input,Weights_1,Weights_2) for Input in Train_Set] #get the data\n", "[tempdict[key].append([temp[x][key] for x in range(len(temp))]) for key in tempdict] #combine all the output dictionaries\n", "\n", "plotter = np.ones((2,2))*np.reshape(np.array(tempdict['output']),(2,2))\n", "plt.pcolor(plotter,vmin=0,vmax=1,cmap=plt.cm.bwr)\n", "plt.colorbar(ticks=[0,0.25,0.5,0.75,1]);\n", "plt.xlabel('Input 1')\n", "plt.ylabel('Input 2')\n", "plt.xticks([0.5,1.5], ['0','1'])\n", "plt.yticks([0.5,1.5], ['0','1']);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Looks like the network produces the correct answer in some cases but not others. The network is particularly confused when Inputs 2 is 0. Below I depict the weights after \"training.\" As you can see, they have not changed too much from where the weights started before training." ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAHMCAYAAAB2sntRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XdYFOf6N/AvTSFSpS4CdiFELFnQI0bEXhDUxN5bLFGj\nXlFPEv0Zkmg0mkQ9nhhrYoklscQgsYK9EBGiWFBRQRRQiJQVpC17v3/kuG8IHXZ2Znfvz3XxB8zs\n89yjPrfDd2dmjYgIjDHGtMNY7AIYY8yQcNNljDEt4qbLGGNaxE2XMca0iJsuY4xpkWllG42MjPjS\nBsYYqwUiMirv55U23f+9UPPVMMaYHjMyKrffAuB4gTHGtIqbLmOMaRE3XcYY0yJuuowxpkXcdBlj\nTIu46TLGmBZx02WMMS3ipssYY1rETZcxxrSImy5jjGkRN13GGNOiKp+9wJhUFBYWIiMjA0VFRVAq\nlTAxMYGZmRns7e3RoEEDsctjrFq46TJJISI8fPgQV69eRUxMDOLi4pCSkoK0tDQoFAo4ODjA3Nwc\nJiYmUKlUKCoqQkZGBurXrw+ZTAZXV1d4e3tDLpdDLpfD29sbpqb8z5xJh1FlTxEzMjIifsoYE1pW\nVhaOHj2KsLAwnDhxAg0aNFA3zXbt2sHd3R2urq5wcHCAsXHZRIyIkJ2djdTUVKSkpODmzZuIiYnB\n1atX8eTJE3Tp0gUhISEIDg6Gu7u7CEfIDI2RkVGFj3bkpstEUVhYiAMHDmDr1q2Ijo5Gt27dEBwc\njP79+8PV1VVj8+Tk5ODkyZMICwvDkSNH4OHhgfHjx2P8+PGwtbXV2DyM/R03XSYZycnJ2LhxI7Zu\n3YrWrVtj+vTp6N+/P1577TXB51YqlTh37hw2b96MY8eOYejQoXjvvffQrl07wedmhqWypstXLzCt\nePbsGWbPno327dsjLy8PZ86cQUREBIYMGaKVhgsApqam6N69O/bs2YP4+Hg0btwYAwYMwNtvv434\n+Hit1MAYN10mqBcvXmDJkiXw9vaGiYkJ7ty5gzVr1sDLy0vUulxcXLBo0SIkJCSgU6dOCAgIwOTJ\nk/HkyRNR62L6j5suE0xERAR8fHzw8OFDxMTEYM2aNXB0dBS7rFIsLCywYMECJCQkwMnJCe3bt8fm\nzZv5Y6qYYDjTZRqnUCiwYMECHD16FJs2bULfvn3FLqnabt68iQkTJsDe3h6bN2+Gh4eH2CUxHcSZ\nLtOaGzduoH379igpKcGNGzd0quECQOvWrXH58mV07doVvr6+CA8PF7skpmf4TJdpzKFDh/Duu+9i\nzZo1GD16tNjl1FlUVBSGDBmC2bNnY+HChZV+witjf8eXjDFBERGWLVuGjRs34uDBg/Dz8xO7JI1J\nSUnBoEGD0KpVK2zduhXm5uZil8R0AMcLTDBEhLlz5+KXX37BlStX9KrhAkCjRo1w7tw5FBcXIyQk\nBC9fvhS7JKbjuOmyWiMizJo1C1euXEFkZCRkMpnYJQnCwsICe/bsgUwmQ1BQEDdeViccL7BaISLM\nnz8fFy5cwMmTJ2FtbS12SYIrKSnBhAkTkJ6ejrCwMNSvX1/skphEcbzANO67777DsWPHcOzYMYNo\nuABgYmKCH374ARYWFnjvvff4Wl5WK3ymy2rs9OnTGDFiBC5duoTmzZuLXY7W5ebmwt/fH1OmTMH7\n778vdjlMgvjqBaYxiYmJ6NSpE3bt2oUePXqIXY5okpKS0KlTJ+zcuRM9e/YUuxwmMdx0mUaUlJSg\nc+fOGDFiBObOnSt2OaI7c+YMRo4cibi4OMnd3szExU2XacTKlStx/PhxnDx5styHiRui+fPn4/Hj\nx/jpp5/ELoVJCDddVmfx8fHo0qULoqOj0bRpU7HLkYz8/Hy0b98eS5cuxZAhQ8Quh0kEN11WJ0SE\nzp07Y+zYsZgxY4bY5UjO5cuX1c/k5U+jYABfMsbq6ODBgygoKMC0adPELkWSOnXqhH79+mHVqlVi\nl8J0AJ/pskoplUq0bt0aa9euRZ8+fcQuR7KSk5PRvn173Lx5U2/vzGPVx2e6rNa2bdsGmUyG3r17\ni12KpHl4eGDChAn4/PPPxS6FSRyf6bIKlZSUoGXLlvjxxx/h7+8vdjmS9/z5c7Ro0QLx8fFwcXER\nuxwmIj7TZbVy/Phx2Nvbc8OtJnt7ewwbNgxbtmwRuxQmYXymyyo0YMAAvPPOO5g4caLYpeiMa9eu\nITg4GImJiTA1NRW7HCYSPtNlNZaYmIioqCgMHz5c7FJ0Srt27eDh4YHDhw+LXQqTKG66rFw///wz\nhg0bhtdee03sUnTOpEmTsHfvXrHLYBLFTZeVKywsDIMGDRK7DJ00YMAAnDhxAkVFRWKXwiSImy4r\nIz09Hbdu3ULXrl3FLkUnOTs7w8vLC2fPnhW7FCZB3HRZGb/99ht69erFn4xQByEhIfj111/FLoNJ\nEDddVsb58+cN+lm5mtCzZ0+cP39e7DKYBHHTZWXExMTA19dX7DJ0mo+PDxISEpCfny92KUxiuOmy\nUvLz85GQkAAfHx+xS9Fp5ubm8PT0RFxcnNilMInhpstKiYuLg5eXF+e5GiCXy3H16lWxy2ASw02X\nlZKQkAAvLy+xy9ALr7/+Oh48eCB2GUxiuOmyUlJTU+Hq6ipqDatXr4ZMJoOtrS2mTJmC4uLiCvc9\nfPgwfHx8YG1tjbfeegvx8fHqbbdu3ULfvn3h6OgIExMTbZReikwmQ2pqqtbnZdLGTZeVkpaWJurz\nYI8fP46VK1fi9OnTePToER48eIBPPvmk3H3v37+PMWPGYNOmTcjOzsaAAQMQEhIClUoFADAzM8Pw\n4cPx/fffa/MQ1GQyGdLS0kSZm0kXN11Withnujt27MDkyZPh5eUFGxsbLFmyBD/88EO5+x4/fhxd\nunRBp06dYGxsjH//+99ISUlR35TQqlUrTJw4Ed7e3to8BDVXV1c+02VlcNNlpWRnZ8POzk60+W/d\nuoW2bduqv2/bti3S09ORlZVV5WtVKhWICDdv3hSyxGqzs7OrVt3MsHDTZaUUFxfDzMxMtPlzc3Nh\nY2Oj/t7a2hpEhBcvXpTZt2fPnjh79izOnTuH4uJifPHFFyguLsbLly+1WXKFzMzMoFQqxS6DSQw3\nXVYKEcHIqNzHgApi9+7dsLKygrW1NYKCgmBlZQWFQqHenpOTAyMjI1hZWZV5raenJ7Zv346ZM2fC\n1dUVmZmZ8Pb2hpubm9bqr4yxsbE6X2bsFW66rBRTU1OUlJRobb5Ro0bhxYsXUCgU+O233+Dt7Y3r\n16+rt1+7dg3Ozs4VRh5vv/02bty4gYyMDISGhiIxMRF+fn7aKr9SSqVS1N8amDRx02WlWFhYIC8v\nT7T5x40bh61btyI+Ph5ZWVlYunRppZ9cERsbC5VKhYyMDEydOhWDBg1Cq1at1NsLCwtRWFgIIkJh\nYaFWH7eYl5cHc3Nzrc3HdAM3XVaKi4uLqJc59enTBwsXLkS3bt3QtGlTNG/eHKGhoert/fv3x4oV\nK9Tfz5kzB7a2tnj99ddhb2+PTZs2qbc9evQIFhYW8PHxgZGRESwsLLR644fYl98xaeIPcWKluLq6\nin5t6dy5czF37txytx05cqTU95U9yatx48aiZqppaWmi32jCpIfPdFkpfEG/5vCZLisPN11WioeH\nB5KSksQuQy8kJSXB3d1d7DKYxHDTZaW0a9cOsbGxICKxS9F5sbGxaNeundhlMInhpstKkclkMDc3\nx6NHj8QuRacREWJiYiCXy8UuhUkMN11WhlwuR0xMjNhl6LSkpCRYWFhwpsvK4KbLyujQoQMuXbok\ndhk67dKlS+jQoYPYZTAJ4qbLyujfvz/Cw8PFLkOnhYeHIygoSOwymAQZVfaGiZGREfEbKoaHiODu\n7o7IyEh4enqKXY7OKSoqgrOzM+Lj4+Hi4iJ2OUwERkZGIKJyH2LCZ7qsDCMjI4SEhCAsLEzsUnTS\n+fPn4enpyQ2XlYubLivX4MGDsWfPHr50rBb27t2LwYMHi10GkyiOF1i5VCoVWrZsid27d6Njx45i\nl6MzsrOz0bRpU9y5cwfOzs5il8NEwvECqzFjY2PMmDED69evF7sUnbJ9+3b069ePGy6rEJ/psgo9\nf/4cLVq0QEJCAhwcHMQuR/JUKhW8vb2xZcsWvPXWW2KXw0TEZ7qsVuzt7TFs2DB89dVXYpeiE/bv\n3w8rKyt07txZ7FKYhPGZLqtUSkoK2rRpg7i4ODRq1EjsciSruLgY3t7e+O6779CzZ0+xy2Ei4zNd\nVmuNGjXClClT8Omnn4pdiqRt3boVTZo04YbLqsRnuqxKmZmZ8PT0xKlTp+Dj4yN2OZKTk5OD119/\nHWFhYfD19RW7HCYBlZ3pctNl1bJ582Zs3LgRUVFRMDXlDxz5u8mTJ8PMzAwbNmwQuxQmERwvsDqb\nMmUKGjZsiJUrV4pdiqQcPXoUkZGRWLVqldilMB3BZ7qs2pKTkyGXyzlm+J/s7Gz4+Phg27Zt6NGj\nh9jlMAnhM12mER4eHvj666/xzjvvICsrS+xyRFVSUoJRo0bh7bff5obLaoTPdFmNzZ07F7dv38aR\nI0cMNt9duHAhrl69iuPHj8PMzEzscpjE8Jku06ivvvoKRIQFCxaIXYoodu7ciQMHDmDfvn3ccFmN\nGeZpCqsTU1NT/PTTT/D390ejRo0wf/58sUvSmqNHj+KDDz7AqVOnYG9vL3Y5TAdx02W10rBhQ0RE\nRKBr164wMzPDnDlzxC5JcCdPnsT48ePx66+/onXr1mKXw3QUN11Wa25ubjh16hR69OiBwsJCLFy4\nUOySBBMeHo5Jkybh4MGD6NSpk9jlMB3Gb6SxOktJSUHv3r0REBCAtWvXol69emKXpDFEhHXr1uGL\nL77Ar7/+ys8WZtXCd6QxwSkUCowePRoKhQL79++Ho6Oj2CXVWWFhIWbOnIkrV67g119/RdOmTcUu\niekIvnqBCc7a2hqHDh1C586d9eIj3BMTE9G9e3dkZmbi0qVL3HCZxnDTZRpjYmKCL774Al9//TUG\nDx6Mbt26IS8vT+yyakSlUmHChAl48803MWjQIOzfvx+WlpZil8X0CMcLTBCzZs3Cr7/+igYNGmDz\n5s3o0qWL2CVVKSEhAdOmTcP9+/fh5uaGixcvwsio3N8QGasUxwtMq6KiorB//37ExMRg2bJlGDNm\nDEJCQnDz5k2xSytXWloapk+fjk6dOiEoKAgJCQlQKpXYtGmT2KUxPcRNl2lUQUEBJk6ciHXr1sHJ\nyQnvvPMO7t69i+7du6NHjx4YP348bt26JXaZAIDU1FR89NFHaN26NaysrHD37l188MEHqF+/PrZt\n24bFixfj0aNHYpfJ9Aw3XaZRS5YsQevWrTF06FD1z8zNzTF37lwkJCSgefPm6NmzJ7p164Z9+/ah\nuLhYq/UREU6fPo2hQ4fijTfeQE5ODq5du4ZVq1aVusPM29sbH3zwAaZMmQKO2JgmcabLNCYqKgqD\nBg1CXFwcnJycKtyvqKgIhw4dwvr163Hnzh2EhIQgJCQEPXr0gIWFhcbrUiqVuHz5Mg4fPoxDhw7B\nzMwMM2fOxJgxY2BtbV3p6/z9/TF58mRMmzZN43Ux/cXX6TLBFRQUoH379vjss89KneVW5f79+wgL\nC0NYWBhiY2MREBCAjh07Qi6XQy6Xw9nZuca1KBQKxMbGIiYmBlevXsXJkyfh7u6O4OBghISEQC6X\nV/sNstu3b6Nr1664evUqGjduXONamGHipssEt3DhQiQmJmLfvn21HiMzMxORkZG4evUqYmJiEBsb\ni/r168Pd3R0ymQyurq5wdnZG/fr1YWpqipKSEhQXFyMjIwOpqalIS0tDamoqMjIy0KZNG3Xj7t69\nOzw8PGpd14oVKxAZGYkTJ07w1QysWrjpMkFVN1aoKSLCkydPkJKSgrS0NKSlpeHp06coKiqCUqmE\niYkJzMzM4OjoCJlMpv5q0qSJRp/zyzEDqyluukwwtY0VdA3HDKwmuOkywWgiVtAVHDOw6uKmywQh\nVKwgVRwzsOripss0zlBihX/imIFVBzddpnGGFCv8E8cMrCrcdJlGGVqs8E8cM7CqcNNlGmOoscI/\ncczAKsNNl2mMIccK/8QxA6sIN12mEYYeK/wTxwysItx0WZ1xrFA+jhlYebjpsjrjWKFiHDOwf+Km\ny+qEY4XKcczA/ombLqs1jhWqh2MG9nfcdFmtcaxQfRwzsFe46bJa4VihZjhmYK9w02U1xrFC7XDM\nwABuuqwWOFaoPY4ZGDddViMcK9QNxwyMmy6rNo4VNINjBsPGTZdVG8cKmsMxg+HipsuqhWMFzeKY\nwXBx02VV4lhBGBwzGCZuuqxKHCsIh2MGw8NNl1WKYwVhccxgeLjpsgpxrKAdHDMYFm66rEIcK2gP\nxwyGg5suKxfHCtrFMYPh4KbLyuBYQRwcMxgGbrqsDI4VxMMxg/7jpstK4VhBXBwz6D9uukyNYwVp\n4JhBv3HTZWocK0gHxwz6i5suA8CxgtRwzKC/Kmu6xtouxtBt374dpqamsLa2hpWVFaytrXHu3Lly\n901ISMCgQYPg5OQEBwcH9OvXD/fu3VNv/+mnn+Dl5QUbGxu4uLhg4sSJyM3NLXesgoICTJw4EevW\nreOGKxGmpqbYtm0bFi9ejEePHpW7T2BgICwsLNT/Xl5//fVy9ysqKsK8efPQqFEj2NvbY9asWSgp\nKVFvf/ToEYKCgtCwYUO4urpi9uzZUKlUghwXqxw3XRH4+/tDoVDgxYsXUCgUCAgIKHe/7OxsDBw4\nEPfu3cOzZ8/g5+eHgQMHqrd37twZ586dQ05ODh4+fIji4mIsXry43LGWLFmC1q1bc44rMd7e3vjg\ngw8wZcoUlPdbpZGREdavX6/+9xIfH1/uOMuXL0dsbCxu376Ne/fuISYmBkuXLlVvf++99+Dk5IRn\nz57h2rVrOHv2LNavXy/YcbGKcdOVMD8/P0ycOBG2trYwMTHBvHnzcPfuXWRlZQEA3Nzc1GetKpUK\nJiYmuH//fplxoqKisGPHDnz77bdarZ9Vz/z585GTk4NNmzaVu706EV94eDhmz54NGxsb2Nvb4/33\n38f333+v3p6UlIThw4fDzMwMTk5O6Nu3L27duqWxY2DVx01XBH/88QecnJzg5eWFpUuXVvvXvLNn\nz0Imk8HOzk79s4sXL8LW1hbW1tY4ePAg5s2bV+o1HCtIX1Uxw0cffQQnJyd06dIFZ8+erdaYKpUK\nT548wYsXLwAAc+fOxd69e5Gfn4+UlBQcPXoU/fr10+hxsGoiogq//trMNCkxMZGSkpKIiOjmzZvk\n7e1NK1asqPJ1jx8/pkaNGtFPP/1U7vbU1FT69NNP6d69e6V+vmDBAhoyZEjdC2eCW758OfXs2ZNU\nKpX6Z1euXKHc3FwqKiqi7du3k5WVFT18+LDMaxcvXkxvvfUWZWRkUFpaGnXs2JGMjY3p6dOnREQU\nHx9PcrmcTE1NydjYmCZOnKi14zJE/+ud5ffVijYQN12N2LVrF1laWpKVlRX179+/zPa9e/eSr69v\npWOkp6eTt7c3LV++vNL9oqKi6M0331R/f/nyZXJ2dqZnz57VrnimVcXFxeTn50cbNmyocJ++ffvS\nf//73zI/z8/Pp9mzZ1OjRo2oefPmtGLFCqpfvz4REalUKmrcuDEtX76cioqKKDMzkwYOHEgLFy4U\n7FgMHTddCdu7dy/J5fIKt2dlZVH79u3p448/rnKs8+fPk62tLRH9tQi9vLzo559/1litTHi3bt0i\nBwcH9W9D/9SvXz9at25dleNs3LiR/P39iYjozz//JGNjY1IoFOrthw4dIh8fH80UzcrgpishR48e\nVZ95xsfHU+vWrenzzz8vd1+FQkF+fn40e/bscrfv2rWLkpOTiYgoKSmJunbtqo4SOFbQXa9ihqys\nLDp+/DgVFBSQUqmkH3/8kSwtLSkhIaHMa1JSUig1NZWI/voNx93dnSIiItTbmzdvTitXriSlUklZ\nWVk0ePBgGjNmjNaOydBw05WQ+fPnk7OzM1laWlLz5s0pNDSUlEqlenu/fv3UMcL27dvJ2NiYLC0t\n1V9WVlb0+PFjIiJatGgRubm5kaWlJbm7u9P06dMpMzOTYwUd9ypm+Oqrr8jPz4+sra3Jzs6OOnXq\nRJGRkURElJycXOrfwrlz56hJkybUoEED8vLyoj179pQa8/r16xQYGEh2dnbk6OhIw4cPp/T0dK0f\nm6GorOnyHWl6hp+toB/42Qy6jW8DNiD8bAX9wc9m0F3cdA0EP1tBv/CzGXQXN10DwLGCfuKYQTdx\n0zUAHCvoL44ZdA83XT3HsYJ+45hB93DT1WMcKxgGjhl0CzddPcaxguHgmEF3cNPVUxwrGBaOGXQH\nN109xLGCYeKYQTdw09VDHCsYLo4ZpI+brp7hWMGwccwgfdx09QjHCgzgmEHquOnqEY4V2CscM0gX\nN109wbEC+zuOGaSLm64e4FiBlYdjBmnipqsHOFZgFeGYQXq46eo4jhVYZThmkB5uujqMYwVWHRwz\nSAs3XR3GsQKrLo4ZpIObro7iWIHVBMcM0sFNVwdxrMBqg2MGaeCmq4M4VmC1xTGD+Ljp6hiOFVhd\ncMwgPm66OoRjBaYJHDOIi5uuDuFYgWkKxwzi4aarIzhWYJrEMYN4uOnqAI4VmBA4ZhAHN10dwLEC\nEwrHDNrHTVfiOFZgQuKYQfu46UoYxwpMGzhm0C5uuhLGsQLTFo4ZtIebrkRxrMC0iWMG7eGmK0Ec\nKzAxcMygHdx0JYhjBSYWjhmEx01XYjhWYGLimEF43HQlhGMFJgUcMwiLm66EcKzApIJjBuFw05UI\njhWYlHDMIBxuuhLAsQKTIo4ZhGGwTTc7Oxt//PEHkpKSkJqaitTUVKSlpSE3NxfFxcUAADMzM1ha\nWkImk8HV1RWurq5o0qQJ2rdvD1tbW43VwrECkyohYob8/HzExcUhISFBve5SU1ORnZ2N4uJiEBFM\nTU1hYWEBFxcXuLq6QiaTwcPDA+3atYNMJtNIHWIxmKb74MEDhIeH4/Lly4iJicHTp0/Rtm1btGjR\nAjKZTP1lY2MDU1NTAH/9ipWTk4O0tDT11/3793H9+nW4uLhALpejU6dOGDBgAJo3b16rujhWYFKm\niZghPT0dv/32G86fP4+YmBgkJCTAy8sLXl5e6obq6uoKOzs7mJmZwcjICCUlJcjLyyu19pKSkhAb\nGwtzc3PI5XJ06NAB/fv3R/v27XUqd9brphsbG4t9+/YhLCwMz58/x4ABA9C1a1fI5XJ4enrCxMSk\nVuOWlJTg7t27iImJwdmzZxEeHg57e3uEhIRg6NChePPNN6s1DscKTBfUJmZITEzEzz//jLCwMNy6\ndQu9evVCjx494OvrCx8fH9SvX79WtRARHj16hJiYGFy6dAmHDx9Gfn4+goODMXjwYPTo0QPGxsa1\nGltbKmu6IKIKv/7aLD0vX76kbdu2UYcOHahx48b00UcfUVRUFJWUlAg2Z0lJCUVFRdFHH31EjRs3\npg4dOtC2bdvo5cuXlb5uwYIFNGTIEMHqYkxTli9fTj179iSVSlXhPkqlkn777TcKCgoie3t7mjFj\nBh0/fpwKCgoEre3OnTu0cuVKat++PTVr1oxWrVpFf/75p6Bz1sX/emf5fbWiDSTBpqtQKCg0NJQc\nHByoX79+dPjwYVIqlVqvQ6lU0uHDh6lfv37k4OBAoaGhpFAoyux3+fJlcnZ2pmfPnmm9RsZqqri4\nmPz8/GjDhg3lbtu8eTM1bdqU5HI5ff/995SXl6f1GlUqFUVFRdG4cePI1taWpk6dSk+ePNF6HVXR\n+aZbUFBAa9euJWdnZxozZgwlJCSIXZLavXv3aPTo0eTs7Exr165V/4+fn59PXl5e9PPPP4tcIWPV\nd+vWLXJwcKCkpCQi+qvJ7d+/nzw9PSkwMJAuXrwocoX/X0ZGBi1cuJAaNmxICxcupMzMTLFLUtPp\npnvy5Elq2rQp9e/fn65duyZ2ORW6du0a9e/fn5o2bUonT57kWIHprFcxw61bt6hTp07Url07Onbs\nWKWxg5iePHlC7777Ljk4ONCmTZskUadONl2FQkHTpk0jd3d3Onr0qGh11NTRo0fJycmJLCws6MGD\nB2KXw1iNFRQUkJubG1laWtL69esFfa9Ek+Li4kgul1Pv3r3p0aNHotZSWdOV5FuAFy5cgI+PD5RK\nJW7cuIG+ffuKXVK19e3bF3/88Qd69eqF7t2748KFC2KXxFi1JSYmomvXrnBzc8PJkycxY8YMyV8p\n8IqPjw8uX76MgIAAyOVy7NixQ+ySyldRNyaRznQ3bdpETk5OFB4ervW5NS08PJycnJxo06ZNYpfC\nWJVOnTpFzs7OtHr1ap05u61IXFwctWjRgubOnUvFxcVanx+6EC8UFRXRzJkzydPTk+7evau1eYV2\n9+5d8vT0pFmzZlFRUZHY5TBWhkqlom+//ZacnZ0pIiJC7HI0JjMzk3r16kW9evWi58+fa3Xuypqu\nJG6OePnyJd5++20YGxtjz549sLGxEXxObcrJycHIkSNBRDhw4ABee+01sUtiDMBfJ13z58/HsWPH\nEBYWVuu7LqVKqVRiwYIFOHr0KCIiIuDm5qaVeSu7OUL0sCYvLw9BQUFwdHREWFiY3jVcALCxsUFY\nWBjs7e0RFBSEvLw8sUtiDESEWbNm4cKFC7hw4YLeNVwAMDU1xerVqzF58mR07doVjx49Erskcc90\n8/PzMWDAAHh4eGDLli21vmVXV5SUlGDy5Ml48uQJDh8+DAsLC7FLYgaKiDB37lxcuXIFx48fh7W1\ntdglCW7t2rX4z3/+g3PnzqFRo0aCziXJZy+oVCoMGTIE5ubm2Llzp9433FdKSkowZswYFBUVYd++\nfTrzzjDTL0uXLsUvv/yCyMhIjT5NT+q+/PJL7NixA5cvXxb0PxpJNt1PPvkEkZGROHXqFOrVqyfI\nHFJVWFiI7t27o1evXggNDRW7HGZgfvnlF7z//vu4cuWKzj9CsTamT5+O1NRUHDp0SLCTHsk98Obn\nn38mDw/K+JtjAAAgAElEQVQPevr0qSDj64KnT5+Sh4cH7du3T+xSmAGJi4sjBwcHunLlitiliKaw\nsJACAgLo448/FmwOSOmSsdu3b5ODgwPFxsZqfGxdExMTQw4ODnT79m2xS2EGICcnh5o1a0Y//vij\n2KWILj09nZo0aUIHDx4UZPzKmq5W4wWlUonOnTtjwoQJmDFjhsbG1WXr16/Hjh07cPHiRYPJtZk4\npk+fDqVSiS1btohdiiRcunQJ77zzDuLi4uDo6KjRsSVzydg333yDBg0a8Ifg/c306dNhYWGBb775\nRuxSmB6LiIjAkSNH8PXXX4tdimT4+/tj9OjRmD17tlbn1dqZbnx8PLp06YLo6Gg0bdpUI2Pqi8TE\nRPj5+eHChQvw8vISuxymZ168eAEfHx9s3LgRffr0EbscScnPz0e7du3wxRdf4J133tHYuJK4eiEo\nKAg9e/bEvHnzNDKevlm9ejUiIyMRHh4udilMzyxZsgQPHz7Ejz/+KHYpknTu3DmMHTsWd+/ehbm5\nuUbGFL3pnjt3DuPHj8edO3dq/blJ+q6wsBCenp7YuXMnunTpInY5TE88e/YM3t7eiImJQZMmTcQu\nR7JCQkLQvXt3zJ07VyPjidp0iQidO3fGjBkzMHbs2DqNpe927NiBjRs34sKFCzr1yadMumbPng0T\nExOsWbNG7FIk7caNG+jZsycSEhI0ctOEqG+kHTlyBLm5uRg1apTQU+m80aNHQ6FQ4OjRo2KXwvRA\ncnIydu/ejUWLFoldiuT5+Pigb9++WvnPSfCmu27dOixYsIAvh6oGExMTLFiwAOvWrRO7FKYHNm7c\niLFjx2r8cih9tXDhQmzYsAHFxcWCziNovHD//n34+/sjOTlZYwG1visoKICHhwcuX76sl099YtpR\nWFiIxo0b4+zZs/D09BS7HJ0RGBiImTNnYujQoXUaR7R4YcOGDZg0aRI33BowNzfHxIkTsWHDBrFL\nYTrs4MGDaN26NTfcGnrvvfewfv16QecQ7Ey3uLgYMpmMr8uthYcPH6Jjx45IS0uDqamp2OUwHdSj\nRw/MmDEDQ4YMEbsUnVJUVAQPDw9cuHABLVq0qPU4opzpXrx4EU2aNOGGWwvNmjWDh4cHLl68KHYp\nTAdlZWUhOjoa/fv3F7sUnVOvXj2EhIQgLCxMsDkEa7phYWEICQkRani9J/RfPNNfR48eRWBgIH8s\nVC3pZNMlIoSFhSE4OFiI4Q1CcHAwwsLCoMkHDjHDwCc8ddOjRw/ExsYiMzNTkPEFabqJiYl4+fIl\n2rVrJ8TwBqF9+/bIy8tDYmKi2KUwHUJEOHHiBEcLdWBhYYGAgABERkYKMr4gTffq1avo2LEj31VV\nB0ZGRujQoQOuXr0qdilMhzx8+BANGjSAq6ur2KXotI4dOwq29gRrunK5XIihDYpcLkdMTIzYZTAd\nwmtPM4Rce4I03ZiYGP6L1wBuuqymeO1phlwuR2xsrCDvqQjSdOPi4jjP1YB27dohLi5O7DKYDuG1\npxnOzs6oX78+njx5ovGxNd50CwsLkZOTA2dnZ00PbXBcXFyQnZ2NoqIisUthOiIlJQXu7u5il6EX\n3N3dkZKSovFxNd50nz59CmdnZ8E+2rgmvv32W/j5+cHc3ByTJk2qcL8ZM2bAysoK1tbWsLa2hrm5\nOWxsbLRYafmMjY3h5OSEp0+fil0K0xFpaWmivYl269Yt9O3bF46OjlU+4CohIQGDBg2Ck5MTHBwc\n0K9fP9y7d0+9fceOHfD19YWNjQ08PDzw73//GyqVSuhDKEUmkyEtLU3j42q8M4r5l/5PjRo1wv/9\n3/9h8uTJle733Xff4cWLF1AoFFAoFBg5cmSdH3ihKa6uroL8xTP9U1hYCIVCAQcHB1HmNzMzw/Dh\nw/H9999XuW92djYGDhyIe/fu4dmzZ/Dz88PAgQPV2/Pz87F27Vo8f/4cv//+OyIjI/HVV18JWX4Z\nQq09jd/Y/+zZMzg5OWl62FoZNGgQACA6Orravybk5eXhwIEDOHLkiJClVZuzszOf6bJqycjIgIOD\ng2i/ZbZq1QqtWrXCgwcPqtzXz88Pfn5+6u/nzZuHpUuXIisrC3Z2dqU+vFYmk2H06NE4c+aMEGVX\nSKi1J0imq8tPFTtw4ACcnJzw1ltviV0KAKB+/fqc6bJqKSoq0tm1d/bsWchkMtjZ2ZW7/dy5c3jj\njTe0WpNQa0/jZ7pKpVKnn4y1Y8cOjBs3Tuwy1ExNTaFUKsUug+kApVKpkx8W8OTJE8yaNQurV68u\nd/v333+PmJgYbN26Vat1CbX2NH6ma2JigpKSEk0PqxXJyck4c+aMpJpuSUmJTi4kpn0mJiZafbNp\n9+7d6jegg4KCajVGRkYG+vTpg1mzZmHYsGFlth86dAiLFi3CsWPH0LBhw7qWXCNCrT2Nn5LWq1dP\nZ38d/vHHH/HWW29J6lNTi4uLYWZmJnYZTAeYmZlpde2NGjWqTp99mJ2djT59+mDQoEH48MMPy2w/\nduwYpk2bhiNHjsDb27supdaKUGtP42e6Dg4O+PPPPzU9bK2UlJSgoKAAJSUlUCqVKCwsrPQsfMeO\nHZg4caIWK6xaRkYGf8YVqxZ7e3tkZGSI+mS6wsJCFBYWgohQWFhY4X8CL168QO/evfHWW29h2bJl\nZbafOnUKY8aMwYEDB0S7w06wtUdEFX79tblmEhISqGnTpjV+nRBCQ0PJyMiIjI2N1V+ffvopJScn\nk6WlJT1+/Fi97+XLl8nS0pJyc3NFrLisJk2a0P3798Uug+kIa2tryszMFGXupKSkUuvNyMioVC/o\n168fLV++nIiItm/fTsbGxmRpaan+srKyUq/Jbt26kZmZGVlZWam39e/fX6vH884779BPP/1Uq9f+\nr3eW21c1/nE9eXl5cHBwwMuXL/kpY3VERHjttdfw/PlzfiA1qxYvLy8cOHBA6+/066POnTtjxYoV\n6NKlS41fq9WP62nQoAHq1auHrKwsTQ9tcLKyslCvXj1uuKzaXF1dBbl11RClpqZCJpNpfFxBrqL2\n9vbGzZs3hRjaoNy4cYPPWFiN8NrTDIVCgYyMDEHeVBek6crlcn74tgbws1FZTfHjQDUjNjYWbdq0\nEeSeA8GaLv/F1x0/G5XVFK89zRBy7QnSdH19fREdHS3E0AaFz3RZTXl7e+Px48fIyckRuxSdJuTa\nEyzTzcnJwcOHD4UY3iA8ePAACoWCM11WI6ampujSpQtOnjwpdik6S6lU4uTJk+jevbsg4wvSdE1M\nTDBgwAAcPnxYiOENwuHDhzFgwABJPJeY6ZaQkBCEhYWJXYbOunz5Mtzd3eHh4SHI+IKt6ODgYG66\ndXD48GEEBweLXQbTQcHBwThy5Ag/KKmWhF57gjXdXr164cqVK3j+/LlQU+it58+fIzo6Gj179hS7\nFKaDXp2lnTt3TuxSdA4R4dChQwgJCRFsDsGaboMGDTBo0CBs27ZNqCn01g8//IDBgwejQYMGYpfC\ndNT48eOxefNmscvQOWfOnIGZmZmgb2Br/Dbgv4uKisKYMWNw7949ziarSaVSoWXLlti9ezc6duwo\ndjlMR2VnZ6NJkya4c+cOXFxcxC5HZwwdOhSBgYGYOXNmncbR6m3Af9exY0dYW1vzO6k1cOLECdja\n2qJDhw5il8J0mK2tLYYOHar1B3/rstTUVERERGDs2LGCziNo0zUyMsLMmTPx9ddfCzmNXvn6668x\nc+ZMflgQq7OZM2fiu+++Q35+vtil6IR169Zh5MiRsLa2FnQeQeMF4K/PbXr99dexefNmwa570xeR\nkZGYNm0a4uPj+cHlTCMGDx4Mf39/LFiwQOxSJC01NRU+Pj64du0a3N3d6zxeZfGC4E0XAPbs2YPV\nq1fj999/5zO4ChAROnTogA8++AAjRowQuxymJ+Lj4xEQEICEhATY2tqKXY5kzZgxA5aWlli1apVG\nxhMt031l+PDhKC4uxoEDB7QxnU46cOAASkpKyv2cKMZq6/XXX0dISAi+/PJLsUuRrISEBOzbt6/c\njwwSglbOdIG/Pn5jwoQJuHHjBmxsbDQypr7Izs6Gj48PduzYgW7duoldDtMzT548Qfv27XH69Gm0\nbt1a7HIkRaVSoWfPnggKCsIHH3ygsXFFjxdemT59OpRKJbZs2aKxMfXB5MmTUa9ePXz33Xdil8L0\n1ObNm7Fx40ZERUUJ8rhCXbV+/Xrs3LkTFy5c0Ogn/0qm6b548QI+Pj7YsGED+vbtq7FxddnRo0fx\n3nvvIS4uDlZWVmKXw/QUEaFPnz4IDAzExx9/LHY5kpCYmAg/Pz9cuHABXl5eGh27sqar8Q+mrMrJ\nkyepUaNGlJqaqvGxdc29e/eoXr16tHHjRrFLYQYgOjqazMzMaOfOnWKXIrqCggLy9/enlStXCjI+\ntPnBlNXx+eef47fffsOZM2dgbm6u8fF1QUFBAQIDA+Hu7o4zZ85g2bJlePfdd/nqDiaIyMhIjB07\nFoGBgThz5gyio6PRqFEjscsSBRFhypQpyMrKwv79+wW5W1ZSZ7pERCqVioYOHUrjx48nlUolyBxS\nplKpaNy4cTR06FBSqVR0584datOmDY0cOZIUCoXY5TE9olQqacmSJeTq6koRERFERLR8+XLy9fWl\nly9filydONauXUs+Pj704sULweZAJWe6ojRdIqLc3Fxq164dLVu2TLA5pGrZsmXUvn17ys3NVf/s\n5cuXNHXqVGrZsiVdu3ZNxOqYvkhNTaVu3bpR9+7dKS0tTf1zlUpFI0eOpKFDh5JSqRSxQu0LDw8n\nZ2dnevjwoaDzSLLpEhGlpKRQixYtaPXq1YLOIyWrV6+mFi1aUEpKSrnbd+3aRQ4ODrRx40aD/C2A\naUZERATJZDIKDQ0tt7Hm5+dTr169aPz48VRSUiJChdp34sQJcnR0pKioKMHnkmzTJSJ69OgRNW3a\nlL766ivB5xLbqlWrqGnTpvTo0aNK9+O4gdVWeXFCRfLy8igwMJDGjh1LxcXFWqpQHEeOHCFHR0c6\nf/68VuaTdNMlInr8+DG1atWKPvzwQ738X7ekpIQ+/PBDatmyJT1+/Lhar+G4gdVURXFCZfLy8qh3\n7940cOBAQTNOMe3YsYMcHR3p0qVLWptT8k2XiCg9PZ0CAgIoODiYcnJytDav0HJycig4OJgCAgIo\nPT29xq/nuIFVR1VxQmUKCgpo0qRJ5OPjQ4mJicIUKAKlUkkLFiygZs2a0Y0bN7Q6t040XSKiwsJC\nmjZtGr3xxhuUkJCg1bmFkJCQQN7e3jR9+nQqLCys9TgcN7CK1CROqIxKpaK1a9eSi4sLnT59WnMF\niiQrK4v69etH3bp1oz///FPr8+tM031l/fr15ODgQOvXr9fJuKGkpIS+/fZbsre3p/Xr12tkTI4b\n2D/VJk6oysmTJ8nFxYXmz5+vs5eUHTlyhNzc3Oj999+noqIiUWrQuaZLRHT79m3q2LEjdevWTfDL\nOzTp4cOH1K1bN+rYsSPFx8drfHyOGxhR3eKEqqSnp9OwYcPI09NTqzloXWVlZdGkSZOocePGdTrr\n1wSdbLpERMXFxfTll19Sw4YNafHixZLOerOzs2nx4sXUsGFD+vLLLwW9/pHjBsOlqTihOvbt20cu\nLi40ceJESk5OFnSuuigqKqLvvvuOZDIZTZs2TRJrQmeb7iuJiYk0fvx4cnJyom+++Yby8/PFLkkt\nPz+fvv76a3JycqLx48dTUlKSVubluMHwCBEnVCUrK4s+/vhjatiwIX3wwQei5KMVKSkpoZ9++ola\ntmxJPXv2pOjoaLFLUtP5pvvKjRs3KDg4mJydnWnRokVVXu8qpEePHtGiRYvI2dmZgoODtf7u6Csc\nNxgGIeOE6khNTaUZM2aQra0tTZ06VdT/6LOysmjNmjXk6elJvr6+dPLkSdFqqYjeNN1Xbt++Te+/\n/z41bNiQBg4cSPv379fKNYYKhYL2799PAwcOpIYNG9L7778vSG5bUxw36C9txgnVkZaWRp9//jm5\nubmRv78/bdmyhZ4+fSr4vEVFRRQREUFTpkwhW1tbGjlyJJ0/f16yJxqVNV1RnjKmKbm5udizZw/2\n79+Py5cvo3PnzggJCUFAQAC8vLzq/FDikpIS3LlzB+fOnUNYWBguXryITp06YciQIRg1ahQaNGig\noSOpu/z8fMydOxenT5/Gvn370LZtW7FLYnWUlpaG0aNHw8jICLt27YKLi4vYJakplUocPnwYe/bs\nwYkTJ+Dt7Y3g4GD07NkTPj4+dX56IBEhKSkJly5dQnh4OI4dOwZPT08MHjwYEyZMgLOzs4aORBiS\neYi5kBQKBY4fP47Dhw/j8uXLSEtLQ9u2bSGXy9GiRQvIZDLIZDK4urrC2tpa/fR8pVIJhUKB1NRU\npKWlIS0tDffv30dMTAyuX78OmUyGTp06ITg4GH369BH845nravfu3ZgzZw4/KlLHvXoU47Rp07B4\n8WKNfqqBphUWFuLs2bMICwvD+fPnkZCQAE9PT8jlcrz++uul1p6dnR3MzMxgbGwMpVKJvLw89bpL\nS0tDUlISYmNjERsbC3Nzc3To0AFBQUEICgqCTCYT+1CrzSCa7j9lZ2er//IePXqE1NRUdWPNzc1F\ncXExFAoF7O3tYWlpqf5H4erqisaNG+PNN9/Em2++qZOfoHr37l0MGzYMb7zxBjZu3MifSKFDSkpK\n8Nlnn2HLli3YsWMHevToIXZJNZafn4+4uDhcvXoVDx48UK+71NRUZGVlQalUIicnBw0bNoS5ubl6\n7clkMri7u6Ndu3aQy+U61WT/ySCbbnX87w9G7DIEwXGD7pFynKBp+rz2AAl8BDvTPgsLC2zcuBGh\noaHo2bMnNm3apNf/yHVdZGQk5HI5unbtihMnTuh1wzV0fKarx8f3CscN0qUPcUJt6Pva4zNdA+fp\n6YmoqChYWVlBLpfj+vXrYpfE8Fec0KtXL1y4cAExMTEG03ANHTddA8Fxg7RwnGC4OF7Q4+OrCMcN\n4jHUOOGf9H3tcbzASuG4QRwcJzCAm67B4rhBuzhOYK9wvKDHx1ddHDcIh+OE8un72uN4gVWK4wZh\ncJzAysNNlwHguEHTOE5gFeF4QY+Pr7Y4bqg9jhOqR9/XHscLrEY4bqgdjhNYdXDTZeXiuKFmOE5g\n1cXxgh4fn6Zw3FAxjhNqR9/XHscLrE44bigfxwmsNrjpsmrhuKE0jhNYbXG8oMfHJxRDjhs4TtAM\nfV97HC8wjTLUuIHjBKYJ3HRZrRha3MBxAtMUjhf0+Pi0RZ/jBo4ThKHva4/jBSYofY0bOE5gQuCm\nyzRC3+IGjhOYUDhe0OPjE4suxw0cJ2iHvq89jheYVulq3MBxAtMGbrpMELoWN3CcwLSF4wU9Pj6p\nkHLcwHGCOPR97XG8wEQl1biB4wQmBm66TCukFjdwnMDEwvGCHh+fVIkZN3CcIA36vvY4XmCSIlbc\nwHECkwJuukwU/4wbNm7cKOiZD8cJTCo4XtDj49MVQsYNHCdIk76vPZ2LF3766Sd4eXnBxsYGLi4u\nmDhxInJzcyvc/9q1a/D19UWDBg3g5+dX4a+rPXr0gLGxMVQqlVCls1oQKm7gOEH67ty5gx49esDW\n1hatWrXCoUOHKtx39erVkMlksLW1xZQpU1BcXFxmn4SEBFhYWGDcuHFCll03RFTh11+bte/x48f0\n7NkzIiLKy8uj0aNH05w5c8rdt6ioiBo3bkxr166loqIi+s9//kONGzem4uLiUvvt2rWLAgICyNjY\nmEpKSoiISKzjYxXbtWsXOTg40IYNG0ilUtV6nIiICJLJZBQaGkpKpVKDFTJNAEBKpZJatWpFa9as\nIZVKRadOnaIGDRpQQkJCmf2PHTtGLi4uFB8fT9nZ2RQYGEgfffRRmf169+5NAQEBNHbsWG0cRoX+\n11vK76sVbSARm+7fvXjxgsaNG0dBQUHlbj9x4gS5ubmV+pmHhwcdP35c/X1OTg55enrS77//zk1X\nB9y5c4fatGlDI0eOJIVCUaPXKpVKWrJkCbm6ulJERIRAFbK6AkA3b94kKyurUj/v3bs3LVmypMz+\no0aNokWLFqm/P3XqFLm4uJTaZ8+ePTR8+HD69NNPJd10JRkvAMDFixdha2sLa2trHDx4EPPmzSt3\nv1u3bqFNmzalfta2bVvcunVL/f3HH3+M9957D87OzoLWzDSjtnEDxwm6j4hw8+bNMj+/desW2rZt\nq/6+bdu2SE9PR1ZWFgBAoVDgk08+wTfffCP5rFiyTbdz587Izs5GSkoKFixYAA8Pj3L3y83NhY2N\nTamfWVtb48WLFwCAq1ev4tKlS5g9e7bgNTPNqenVDXx1gu7x9PSEk5MTvvrqKyiVSpw4cQJnz57F\ny5cvy+z7z3VubW0NIlKv8yVLluDdd9+Fq6ur1uqvLUk03d27d8PKygrW1tYICgoqtU0mk6FPnz4Y\nMWJEua+1tLSEQqEo9bOcnBxYWVmBiDBz5kysXbtW798t1VejRo3ChQsXsH79eowePVq9yF4pKSnB\nJ598gnHjxmHnzp345JNPYGJiIlK1rCZMTU1x6NAhhIeHQyaTYfXq1Rg+fDjc3NzK7PvPdZ6TkwMj\nIyNYWVnh2rVriIiIwNy5c7VZfu1VlDuQRDJdIqLz58+Tra1tudtOnDhB7u7upX7m4eFBJ06coOzs\nbDI2NiaZTEYuLi7k6OhIRkZGJJPJ6MKFC5zp6pCXL1/S1KlTqWXLlnTt2jUiIkpNTaVu3bpR9+7d\nKS0tTeQKWU1UtPb8/f1p06ZNZX4+atQoWrx4sfr7V2+UEhGtWbOGLC0t1evc0tKSLCwsSC6XC1N8\nNUDX3kjbtWsXJScnExFRUlISde3alYYMGVLuvkVFRdSkSRP6z3/+Q4WFhbR27Vpq0qSJ+uqFZ8+e\nqb+io6PJyMiI0tLSqLi4mJuuDnp1dcOcOXP46gQd9mrtxcXFUUFBAeXl5dGqVauoWbNmVFRUVGb/\nY8eOkUwmo9u3b1NmZiYFBgbSxx9/TERE+fn5pdb5/PnzaejQofT8+XOtHtPf6VzTXbRoEbm5uZGl\npSW5u7vT9OnTKTMzU729X79+tHz5cvX3165dI7lcTq+99hrJ5XK6fv16ueMmJSXx1Qs6TqlU0nvv\nvUempqbUrVu3Gl/dwKTh1dpbsGAB2dnZkZWVFfXv358ePHhARETJyclkZWVFjx8/Vr9m9erV5Ozs\nTDY2NjR58uRymzMRUWhoqKSvXuA70vT4+PRNWloaRo8eDSMjI2zZsgUrVqzA6dOnsW/fvlLvbDPp\n0/e1p3N3pDH2TxEREaWuTmjatKlWn93AmKbwma4eH58+ePXshM2bN2Pnzp3lXnsr5U+mYOXT97XH\nZ7pMJ/39ZofY2NgKb3aQ6idTMFYebrpMkv4ZJ1R1s4O2HxXJWG1xvKDHx6eLqhMnVIXjBunT97XH\n8QLTCdWNE6rCcQOTMm66TBJqGidUheMGJlUcL+jx8ekCTcQJVeG4QXr0fe1xvMAkSVNxQlU4bmBS\nwk2XiULTcUJVOG5gUsHxgh4fnxRpI06oyqu4wdvbGxs3boS1tbXWazB0+r72OF5gkqCtOKEqr+IG\na2tr+Pr64tq1a6LUwQwTN12mFdqOE6ry97ihV69eHDcwreF4QY+PTwqkECdUheMG7dP3tcfxAhOF\nVOKEqnDcwLSJmy4ThNTihKpw3MC0heMFPT4+MehCnFAVjhuEp+9rj+MFphW6EidUheMGJiRuukwj\ndC1OqArHDUwoHC/o8fFpgz7ECVXhuEHz9H3tcbzABKEvcUJVOG5gmsRNl9WKvsUJVeG4gWkKxwt6\nfHxCMIQ4oSocN9Sdvq89jheYRhhKnFAVjhtYXXDTZdViaHFCVThuYLXF8YIeH58mcJxQNY4bak7f\n1x7HC6xWOE6oHo4bWE1w02Xl4jihZjhuYNXF8YIeH19tcJxQdxw3VE3f1x7HC6xaOE7QDI4bWGW4\n6TIAHCdoGscNrCIcL+jx8VUHxwnC47ihLH1fexwvsHJxnKAdHDewv+Oma6A4TtAujhvYKxwv6PHx\nlYfjBPFx3KD/a4/jBQaA4wSp4LjBsHHTNRAcJ0gLxw2Gi+MFPT4+gOMEXWCIcYO+rz2OFwwUxwm6\ngeMGw2IqdgHaoFKp8Pz5c6SlpSEtLQ0vXrxAcXExAGD//v2wsrKCTCaDTCaDvb09jI11//+iiIgI\njBs3DtOmTcPixYthYmIidkmsEq/iht27d6NXr15YunQppk6dCiOjck+WdAYRIScnR732srKy1Gtv\n3759eO211+Di4gKZTAYnJyeYmup/S9K7I1QoFLh69SpiYmLUX8nJybC0tFQ3VhsbG/Vf7t69e0v9\no8jNzYWHhwfkcrn6y9fXV2d+5eM4QbeNGjUKcrkcw4YNw5kzZ3QqbigsLMQff/xRau0lJCTA1NQU\nMpkMrq6usLOzg5mZGQDg559/Rl5ennrtZWZmwtnZGe3atYOvry/kcjn8/Pzg7Ows8pFpll5kug8f\nPsThw4cRFhaGK1euoG3btupmKZfL0axZM5ibm1drrIKCAjx8+BAxMTHq5n39+nV07NgRwcHBCAkJ\nQdOmTQU+otpJS0vD6NGjYWRkhF27dvGbZTosPz8fc+fOxenTp/Hzzz+jXbt2YpdUroyMDBw5cgRh\nYWGIiIhAs2bN4Ovrq157Xl5esLS0rNZYSqUSjx8/RmxsrHr9RUdHo3nz5ggJCUFISAjatm2rE2f/\nlWW6IKIKv/7aLE15eXm0detW8vX1JScnJ5o0aRL98ssvlJubq/G5cnNz6ZdffqFJkyaRk5MT+fr6\n0tatWykvL0/jc9XWyZMnSSaTUWhoKCmVSrHLYRqya9cucnBwoA0bNpBKpRK7HCIiUiqVFBYWRn37\n9iVra2t6++23adu2bZSenq7xuYqKiuj06dM0b948at68OTVt2pRWrFghyFya9L/eWX5frWgDSbTp\nJjvLTZkAABuISURBVCUl0bx588je3p6Cg4Pp6NGjWm0ySqWSjh49SsHBwWRvb0/z5s2jpKQkrc1f\nXj1LliwhmUxGERERotXBhHPnzh1q06YNjRgxgnJyckSrIysri7744gvy8PCgjh070vbt2+nly5da\nm1+lUlF0dDRNmjSJbG1tacyYMXT16lWtzV8TetF009PTac6cOdSwYUNauHAhJSYmil0SJSYm0sKF\nC6lhw4Y0Z84crf/vm5qaSt26daPu3btTWlqaVudm2vXy5UuaOnUqtWzZkv744w+tz71y5UpydHSk\ncePGSaLRPX/+nL766itydXWlIUOG0N27d8UuqRSdbrr5+fn02Wefkb29Pc2aNYuePn0qdkllPH36\nlGbNmkX29vb02WefUX5+vuBzcpxgmLQZN6hUKvrhhx/Izc2NBg8eTLdv3xZ0vtrIy8uj5cuXk4OD\nA02dOlUysYPONt3ff/+dvL29adCgQfTgwQNRa6mOBw8e0KBBg8jb25uuXLkiyBw1jRPu3btH5ubm\nNHbs2Cr37d69OxkZGVFJSYn6Z0lJSdS/f3+ys7MjmUxGs2bNKrXdUMTHx1P37t3JxsaGWrZsSb/8\n8ku5++3du5c8PT3J2tqanJ2dacKECfTixQv19q5du5K5uTlZWVmRpaUleXl51bgWbcQNjx49ol69\nepFcLqfLly8LMocmPX/+nObOnUsuLi60f/9+scvRvaZbUFBAH374ITk5OdGePXsk8wZCdahUKtq9\nezc5OTnRhx9+SAUFBRobuzZxQu/evSkgIKDKprtr1y4KCAggY2PjUk21f//+NGHCBCoqKqJnz56R\nj48PrVu3rk7HoWuUSiW1atWK1qxZQyqVik6dOkUNGjSghISEMvs+fvyYnj17RkR/nYWNHj2a3n//\nffX2wMBA+v777+tck1Bxg0qlok2bNpGDgwMtW7aMiouLNTa2Nly8eJFatWpFw4YNE/Wst7KmK7m7\nAFJTUxEQEID4+HjExcVhxIgROnGJyCtGRkYYOXIk4uLiEB8fj4CAAKSmptZ53No8O2Hv3r2ws7Or\n8lpdhUKBzz77DKtWrSqzLSkpCcOHD4eZmRmcnJzQt29f3Lp1q9bHoYvu3LmDtLQ0zJkzB0ZGRujW\nrRs6d+6MnTt3ltnXzc0NTk5OAP66KcfExAQPHjwotQ9p4DJMIZ7d8PLlS4waNQrfffcdTp8+jY8/\n/ljnblbw9/fHtWvX4ObmBrlcjpiYGLFLKquibkwinOn+/vvv1KhRI1q2bJlOnd1WRKVS0dKlS8nN\nza3WcUNtr07IycmhVq1aUUpKCoWGhlZ6pjtz5kxau3YtJSUllTnT3bRpE40fP55evnxJT548odat\nW9Ovv/5aq2PRVTdv3iQrK6tSP+vVqxe9/fbb5e5/4cIFsrGxISMjI7K0tCz19xYYGEhOTk7k6OhI\nb731Fp05c6bO9Wkibnj8+DHJ5XIaPXq0Vq9IENL+/fvJwcGB9uzZo/W5oQvxwt69e8nR0VEvF/Sh\nQ4fI0dGR9u7dW6PX1eXqhDlz5tCqVauIiCptutHR0dS+fXtSqVTlNt34+HiSy+VkampKxsbGNHHi\nxBrVoQ+Ki4upefPmtGrVKiouLqbjx49TvXr1qG/fvpW+LjU1lT799FO6d++e+mdXrlyh3NxcKioq\nou3bt5OVlRU9fPiwzjXWJW64cuUKubq60pdffqkXJzt/d+3aNWrSpAktXrxYq8cm+aa7ZcsWatSo\nEcXFxWllPjHExcWRq6srbd26tVr71+XqhD/++IPeeOMNdR5XUdNVqVTUoUMHOnfuHBH9dQmcsbGx\nej6VSkWNGzem5cuXU1FREWVmZtLAgQNp4cKFNapHH9y4cYO6du1KDg4O1LdvXxo7dixNmTKlytdF\nRUXRm2++WeH2vn370n//+1+N1VnTqxvOnz+vtyc7r6Snp5NcLqfZs2dr7U1gSTfdLVu2kLu7e6mz\nAX119+5dcnd3py1btlS4jyZudlizZg1ZWlqSTCYjFxcXsrS0JAsLC5LL5aX2y87OJhMTE/V+jo6O\nZGRkRDKZjC5cuEB//vknGRsbk0KhUL/m0KFD5OPjU6u69Im/vz9t2rSpyv3Onz9Ptra2FW7v16+f\nxt+YrG7ccPHiRXJ0dKTjx49rdH4pysrKon/96180c+ZMrZzxSrbp7t27lxo1amQQDfeVu3fvkqur\na7lRg6ZudsjPz6dnz56pv+bPn09Dhw6l58+fl9n37/tFR0eTkZERpaWlqc+SmzdvTitXriSlUklZ\nWVk0ePBgGjNmTK1r01VxcXFUUFBAeXl5tGrVKmrWrBkVFRWV2W/Xrl2UnJxMRH9dbte1a1caMmQI\nEf31n9zx48epoKCAlEol/fjjj2RpaVnuVRB1VVXcEBsbazAN95Xs7Gzy9fWlf//734LPJcmme+XK\nFXJ0dNTrSKEicXFx5OjoWOrNNSFvdvh7vJCcnExWVlb0+PHjMvuVl+lev36dAgMDyc7OjhwdHWn4\n8OGSuQBdmxYsWEB2dnZkZWVF/fv3V183/s8/z0WLFpGbmxtZWlqSu7s7TZ8+nTIzM4mIKCMjg/z8\n/Mja2prs7OyoU6dOFBkZKWjd5cUNT58+JQ8PD9q3b5+gc0vR8+fPqUWLFvTDDz8IOo/kmm5KSgq5\nubnpdY5UlUOHDpGbmxs9fvyYn53ABPX3uCE9PZ38/f0pNDRU7LJEc/v2bXJ0dKRLly4JNoekmm5B\nQQF16NCBli1bpvGxdc3//d//kZWVFQUGBvKzE5igXsUN1tbW1KdPH4O8q/DvwsPDydXVlZ48eSLI\n+JU1Xa3fHBEaGgqZTIaPPvpI21NLTmhoKFq1aoWOHTvys2+ZoCwsLNCrVy9YWlpi27ZtevHpKHUR\nFBSEadOmYeLEiRq5WaUmtPoQ8ytXriA4OBhxcXF69zT42nr69Cnatm2L8PBw+Pn5iV0O01MZGRlo\n06YNDh48iE6dOoldjiQolUr861//wvTp0zFlyhSNjl3ZQ8y11nQLCgrw5ptvYsmSJRgxYoRGxtQX\ne/bswdKlSxETE1PtT7hgrCaGDRuGxo0bl3urtyG7ceMGunfvjpiYGHh4eGhsXEl8GvCqVavg6emJ\n4cOHa2tKnTFixAi0atWKFwQTxJEjR3D9+nV89tlnYpciOT4+Pvh/7d17TFPnGwfwL9Yqd6fQYStS\nLk4FFJXCqpK5uY2ZiEWc8baoTNGJTOMfE2fMmHOgbm6LmU6Y20xkXvC2aQpTGckSndeOVsVE8YJc\njBQFvCD3try/P/xJ5uZQ4bw9PeX5JCYa5Hm/J+V9+nJ6znuWLVuGpUuX2m1Mu6x0q6urERoaCoPB\ngODg4C7Xc0YlJSXQarW4fPkyFAqF2HGIk7DZbBg1ahTS09MxefJkseM4pObmZgwZMgS7d+9GTEyM\nIDVFX+muXbsWs2bNoobbgZCQEMycORPr1q0TOwpxIrt374anpyfi4+PFjuKwXF1dsWbNGqxcudIu\nH6pxX+mWlZVBo9Hg0qVL9OHZM1RVVSE8PBwmkwlqtVrsOETiWlpaMHToUGRnZ2PcuHFix3FoNpsN\nERER2LBhA+Li4rpcT9SV7qZNm7BgwQJquM+hf//+SEpKwqZNm8SOQpzAgQMHEBISQg33OchkMqxe\nvRobNmzgPhbXlW5jYyMCAgJQWFiIwMDATtfpTkpLSxEdHY2Kigq4u7uLHYdIWExMDFJTU5GQkCB2\nFEmwWCwIDAxEfn4+hg0b1qVaoq109+zZg7Fjx1LDfQFBQUEYM2YM9u7dK3YUImHnz59HRUUFJk2a\nJHYUyZDL5Vi4cCGysrK4jsO16WZlZSElJYXnEE4pJSUFmZmZYscgEvb9999j0aJFknvcjtgWLlyI\nnJwc1NfXcxuDW9O9ceMGKioqEBsby2sIp/XOO++goqICpaWlYkchEmSz2fDLL79g9uzZYkeRnAED\nBiA6Ohr5+fncxuDWdHNzczFp0iTIZDJeQzgtmUyGuLg45Obmih2FSNCZM2egUqnotF4nxcfHQ6/X\nc6vPtenqdDpe5Z2eTqejpks6Ra/X03W5XTBp0iQcPnwYNpuNS30uTbeurg5nz56lUwtdEBsbi7Nn\nz6Kurk7sKERiaMHTNWq1GgMGDMCZM2e41OfSdAsLCzFixAh4eHjwKN8teHp6Yvjw4TAajWJHIRJy\n//593Lx5ExqNRuwokjZu3DicPn2aS20uTddoNNKLLgCNRkNNl7wQk8mEESNG0GcpXcRz7nFrulFR\nUTxKdytRUVEoLCwUOwaREFrwCIPn3KOVrgPTaDQwmUxixyASQnNPGEOHDoXZbMaDBw8Ery14021r\na0NFRQVCQkKELt3thISEoLy83O6PEyHSVVZWhkGDBokdQ/JkMhkCAwNRUVEheG3Bm25tbS08PT3R\nu3dvoUsjKCgIf/zxh+B1/2nNmjWYO3duh/9ny5YtiI6OhqurK+bPn88lh6urKzw8PFBbW8ulPnE+\nlZWVUCqVgtd1lLnX2tqKBQsWIDAwEH369EFkZCSOHj3KJYtSqURlZaXgdQVvumazmcuL7mgGDBiA\ntLQ0JCUlcR1HqVTCbDZzHYM4B8YYbt++7dTzz2q1IiAgAH/++ScePHiA9PR0TJ8+ncuKlNfck2zT\nzc7OxmuvvYbU1FT069cPISEhT7zjjR8/HqtWrYJWq0WfPn0wZcoU3L9/HwBw7NgxDBw48Il6j9/J\n8/PzsW7dOuzduxdeXl4YNWrUU8dPSEhAfHw8+vXrx+8gAahUKi7vtsT51NbWwsPDg/tz9sSce+7u\n7vj000/ba8TFxSEoKIjLlQa85p7gTffhw4fo06eP0GWfymAwIDQ0FLW1tUhNTf3XqnPHjh3Yvn07\nqqqqIJPJnngOkovLU3ddw4QJE7Bq1SrMmDEDDx8+xLlz57gew7N4e3tz3XyDOI/uOPdu376Na9eu\nITw8vGsH9BS85p7gTddisdhtZyO1Wo358+fDxcUFiYmJMJvNuHPnTvvX58yZg9DQULi5uSE9PR37\n9++X3IdSPXv2hMViETsGkQCr1Wq363MdYe5ZrVbMnj0b77//PgYPHixobYDf3LPb04B56N+/f/vf\n3dzcAOCJd6a//xqjVqthsVhQU1Njv4CEOCmx5x5jDLNnz0bv3r2xefNmwerag+BLUrlcDqvVKnTZ\nTrl582b738vLyyGXy+Hr6wsPDw80Nja2f81ms6G6urr93//1648YrFYr5HK52DGIBPTs2ZPbJi0v\nivfcS0pKQk1NDQ4fPsxtdc9r7gm+0vXy8uJyQXFn7Ny5E8XFxWhsbMTq1asxbdo0uLi4YPDgwWhu\nbsaRI0dgtVqRkZGB1tbW9u/z8/NDWVlZh78O2Ww2NDc3w2azwWq1oqWlhcsPfF1dHTw9PQWvS5xP\nd5l7ycnJKC4uhl6vR69evbgdA6+5J3jT5XmJ07PeBf/59Tlz5iAxMREqlQqtra349ttvATw6QZ6Z\nmYmkpCT4+/vDy8sL/v7+7d83bdo0MMbg4+Pzn7czZ2RkwN3dHV9++SV27doFd3d3rF27totH+G+V\nlZVQqVSC1yXOx8fHBw0NDWhubha8tqPMvYqKCvzwww84f/48/Pz84OXlBW9vb+Tk5AhwlE/iNvcY\nY//559GXX8ydO3dYv379Xvj7hPbGG2+wbdu2iR2jy/r27cuqq6vFjkEkYuDAgezGjRuiZnCWuff2\n22+zo0ePdup7/987n9pXBV/p+vj4oL6+Hi0tLUKX7naam5vR0NAAHx8fsaMQiVCpVHQzjUDMZjOX\nla7gTbdHjx4ICAhASUmJ0KVfiCN9GNZZJSUlUKvVTnEsxD4CAwNx/fp1UTM4w8+rzWZDWVkZAgIC\nBK/N5YLax3tRhoWF8Sj/XOxxnzhvRqMRkZGRYscgEvJ47j1r7xCenGHuFRcXQ6lUcrnZhMt1uhqN\nhvaBFUBhYSHtS0xeCG18Lwyec49b06UXvutob1TyoiIjI3HhwgWHuV5XqnjOPS5NNyoqChcuXEBD\nQwOP8t1CfX09Ll68SE2XvJCXXnoJAwcOpEVPFx0/fhxjxozhUptL0/X29oZWq0VBQQGP8t1CQUEB\ntFotvL29xY5CJEan0yE3N1fsGJJVXl6OW7duYfTo0Vzqc9t7gV74rqHHaJPOio+Ph16vFzuGZOXl\n5WHixIncbi/m2nTz8vLo3FIn2Gw25OXlUdMlnTJ69GiYzWaUlZWJHUWS9Ho94uPjudXn1nSDg4MR\nEBBApxg64ffff4darUZQUJDYUYgEyWQyvPvuu9i5c6fYUSTn1q1b+OuvvzBhwgRuY3Dd2nHx4sXI\nzMzkOYRTyszMREpKitgxiIQlJydj69atDrPjn1T8+OOPmDVrFtdNplxYB7v5uLi4sI6+/iyNjY0I\nCAhAYWEhAgMDO12nOyktLUV0dDQqKirg7u4udhwiYTExMUhNTUVCQoLYUSTBYrEgMDAQ+fn5GDZs\nWJdqubi4gDH21FvzuK503d3dMXfuXGRlZfEcxqlkZWUhMTGRGi7pspSUFHz33Xdix5CMgwcPYtCg\nQV1uuM/CdaULPLr8IjIyEpcuXYKfn1+Xajm7qqoqhIeHw2QyQa1Wix2HSFxrayuGDBmC7OxsjBs3\nTuw4Ds1msyEiIgIbNmxAXFxcl+uJttIFHj2qY86cOcjIyOA9lORlZGRg7ty51HCJIHr16oXPP/8c\nK1eulNyzAe1tx44d8PHxwcSJE7mPxX2lCwDV1dUIDQ2FwWBAcHBwl+s5o5KSEmi1WhQXF8PX11fs\nOMRJ2Gw2jBo1Cunp6Zg8ebLYcRxSc3MzhgwZgpycHIwdO1aQmqKudAFAoVBg2bJl+Oijj+gd9ykY\nY1i+fDmWLVtGDZcISiaT4YsvvkBqaiqamprEjuOQvv76a4wcOVKwhvssdlnpAo/eTTQaDdLS0jBz\n5kxBajqLnJwcZGRkwGQyoXfv3mLHIU5o+vTpUKvV+Oqrr8SO4lAuXryIN998E0ajUdC9czta6dqt\n6QKAwWCATqdDUVERfaj2f1VVVRgxYgTy8vIQHR0tdhzipKqrqxEREYFff/2V20YuUmOxWDB69Ggs\nXrwYCxYsELS26KcXHnv11Vcxf/58LFq0iE4z4NFpheTkZCQlJVHDJVwpFAps3rwZ8+bNQ319vdhx\nHML69evh6+uLpKQku45r15UuALS0tGDcuHGYPHkyVq1aJWhtqVm7di1yc3Nx7NgxOq1A7CIpKQkP\nHjzAvn370KOHXddcDuW3337DBx98AIPBgAEDBghev6OVruBPA34et27dYv7+/uzQoUNc6kvBoUOH\nmL+/P6usrBQ7CulGmpub2dixY9lnn30mdhTRXLp0iSkUCnbq1CluY6CDpwGL0nQZY8xgMDCFQsGK\nioq4jeGoioqKmEKhYAaDQewopBuqqqpiAQEBbP/+/WJHsbuamho2aNAgtn37dq7jOGTTZYyxPXv2\nMJVKxa5evcp1HEdy5coVplKp2J49e8SOQroxk8nEFAoFO3r0qNhR7Ob+/ftMo9Gwjz/+mPtYDtt0\nGWPsp59+YgMHDmRXrlzhPpbYrly5wvz9/dm2bdvEjkIIO3nyJFMoFCw/P1/sKNzdu3ePjR49mn34\n4Yesra2N+3gO3XQZe9R4VSqVU59qKCoqYiqVihoucSgnTpxgCoXCqT9fuXPnDtNoNGzp0qV2abiM\nSaDpMvboVIOzvviHDh1iCoWCTikQh2QwGJhKpWJffPGF3ZqSvZw/f54FBgaytLQ0ux6bJJouY49e\nfH9/f5aRkeEUL35bWxtLT09n/v7+9KEZcWg3b95kGo2Gvffee6yxsVHsOII4cOAA8/X1FWWx01HT\ntft1us9SWVmJKVOmQKlUYuvWrZK9c62qqgrJycmoqqrCwYMHoVQqxY5ESIeampqQlJSE4uJi/Pzz\nz9z3leWlqakJn3zyCfbv34+DBw9Co9HYPYPD3JH2PFQqFY4fP46wsDBEREQgJydHUnevMcaQk5OD\nESNGICwsDMeOHaOGSyTBzc0Nu3btQkpKCsaPH4+1a9dK7nE/p06dwsiRI3Hr1i2YTCZRGu4z/dcS\nmIlweuGfDAYDCwsLYwkJCez69euiZnke169fZwkJCSwsLIxOJxBJKy8vZ7GxsUyj0bDTp0+LHeeZ\namtr2bJly1j//v3ZgQMHxI7T4ekFh1vp/l10dDSMRiMiIyOh1WqxZMkS3L59W+xY/1JVVYUlS5ZA\nq9UiMjISJpOJ9lIgkhYQEID8/HwsXboU06ZNw5QpU3D58mWxY/1LY2Mj1q9fjyFDhqCpqQlFRUWY\nOnWq2LE65NBNFwBcXV2RlpaGy5cvQy6XIywsDCtWrEBpaanY0VBaWooVK1YgPDwccrkcly9fRlpa\nGu2jQJyCi4sLEhMTcfXqVcTExOD1119HYmIijEaj2NFw9+5dfPPNN3jllVdw7tw5nDx5Elu3boVC\noRA72jM5fNN9TKFQYOPGjTCZTLDZbIiOjoZOp8ORI0dgs9nslsNms+HIkSPQ6XSIjo6GzWaDyWTC\nxo0bJfGCE/Ki3NzcsHz5cly9ehWhoaGYOnUqtFotsrOz7boxOmMMhYWFmD9/PkJCQnDhwgXo9Xrs\n27cPgwcPtluOrnK4qxeeV2NjI/bu3YvMzEyUl5dj0qRJ0Ol0iI2NFfyZ9fX19SgoKEBubi7y8vKg\nVquRkpKCGTNm0FN7SbfzeOGxZcsWnDp1Cm+99Rbi4+MRFxcn+MLDYrHgxIkT0Ov10Ov17duhzps3\nz6EXOQ6ziTkvpaWlyM3NhV6vh8FgwPDhw6HRaBAVFQWNRoOQkBC4uro+V63m5maUlJTAaDSisLAQ\nRqMRFy9ehFarhU6ng06nQ1BQEOcjIkQaampqcPjwYej1ehQUFCA4OPiJuRcaGvrciyCLxYKbN2/C\nZDK1z7/CwkIMGjQI8fHxiI+PR0REBFxcnr5joiNx+qb7d3V1dTAaje0vmslkQnl5OTw8PKBUKqFS\nqeDt7Y2ePXsCAKxWK+rq6lBZWQmz2YyGhgao1WpERka2/+BoNBp4e3uLfGSEOLaWlhacP3/+ifl3\n7do1yGSy9rnXt29fyOVy9OjRA1arFQ0NDTCbzTCbzbh37x78/PwwatSo9sYdFRUlyWv1u1XTfZq2\ntjbcvXsXZrMZlZWVqK+vh8ViAQDI5XJ4enpCpVJBqVTCx8dHEu+khEgBYwx1dXXtjfXu3buwWq1o\na2uDXC6Hm5sblEollEolXn75ZchkMrEjC6LbN11CCLEnSd2RRgghzoyaLiGE2BE1XUIIsSNquoQQ\nYkfUdAkhxI6o6RJCiB1R0yWEEDuipksIIXZETZcQQuyImi4hhNgRNV1CCLEjarqEEGJHPZ/1H2jH\nLUIIEU6Hu4wRQggRFp1eIIQQO6KmSwghdkRNlxBC7IiaLiGE2BE1XUIIsaP/AeoqtglX8xa6AAAA\nAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "Weights_1 = Weight_Dict['Weights_1']\n", "Weights_2 = Weight_Dict['Weights_2']\n", "\n", "Weight_Dict = {'Weights_1':Weights_1,'Weights_2':Weights_2}\n", "\n", "network = NeuralNetwork()\n", "network.add_layer(2,['Input 1','Input 2'],\n", " [[round(x,2) for x in Weight_Dict['Weights_1'][0][:2]],\n", " [round(x,2) for x in Weight_Dict['Weights_1'][1][:2]]])\n", "network.add_layer(2,[round(Weight_Dict['Weights_1'][0][2],2),round(Weight_Dict['Weights_1'][1][2],2)],\n", " [round(x,2) for x in Weight_Dict['Weights_2'][:2]])\n", "network.add_layer(1,[round(Weight_Dict['Weights_2'][2],2)])\n", "network.draw()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This network was unable to push itself out of the local optima. While local optima are a problem, they're are a couple things we can do to avoid them. First, we should always train a network multiple times with different random weights in order to test for local optima. If the network continually finds local optima, then we can increase the learning rate. By increasing the learning rate, the network can escape local optima in some cases. This should be done with care though as too big of a learning rate can also prevent finding the global minima. \n", "\n", "Alright, that's it. Obviously the neural network behind [alpha go](https://en.wikipedia.org/wiki/AlphaGo) is much more complex than this one, but I would guess that while alpha go is much larger the basic computations underlying it are similar. \n", "\n", "Hopefully these posts have given you an idea for how neural networks function and why they're so cool!" ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.11" } }, "nbformat": 4, "nbformat_minor": 0 }