\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" \n",
" \n",
" \n",
" Introduction "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Self-organizing maps is a discretized representation of the multidimensional input space of the training samples in a 2D map of nodes and is therefore a method to do dimensionality reduction. \n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"An example of the use of SOM in art:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import HTML\n",
"\n",
"HTML('')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this algorithm, the map space is defined beforehand, usually as a finite two-dimensional region where nodes are arranged in a regular hexagonal or rectangular grid. There are no lateral connections between nodes within the lattice.\n",
"\n",
"Each node has a specific topological position (an x, y coordinate in the lattice) and contains a vector of weights of the same dimension as the input vectors. The scheme of SOM is:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![SOM 3](SOM3.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" \n",
" \n",
" \n",
" Algorithm "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* Initialize each node’s weights randomly\n",
"* Choose a random vector from training data and present it to the SOM\n",
"* Calculate the distance between the input vector and the weights of each node, according to "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\\begin{equation*}\n",
"D = \\sqrt {\\sum_{i=1}^n{(x_i-W_i)}^2} \n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compare the distances among all nodes, the lowest value distance is defined as the Best Matching Unit (BMU)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* Calculate the neighborhood radius around BMU by "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\\begin{equation*}\n",
"\\sigma (t) = \\sigma_0 \\exp \\left(\\frac {-t}{\\lambda}\\right) \n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\\begin{equation*}\n",
"\\lambda = \\frac {n}{ln \\sigma_0} \n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Where $\\sigma$ $(t)$ is the radius of neighborhood at step $t$, $t$ is the iteration step, $\\sigma_0$ is the initial radius of the complete array nodes, $\\lambda$ is a normalization factor, and $n$ is total number of iterations. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![SOM 4](SOM4.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In each iteration the neighborhood changes of size."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![SOM 5a](SOM5a.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* Change the weights of each node in the BMU’s neighborhood in order to become more like the BMU. Nodes closest to the BMU are altered more than the nodes furthest away in the neighborhood according to"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\\begin{equation*}\n",
" W (t+1) = W(t) + \\Theta(t) L(t) \\left ( x - W(t) \\right) \n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\\begin{equation*}\n",
"L(t) = L_0 \\exp \\left ( \\frac{-t}{\\lambda} \\right)\n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\\begin{equation*}\n",
"\\Theta (t) = \\exp \\Biggl( \\frac{-d^2}{2 \\sigma^2(t)} \\Biggr) \n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Where $W(t+1)$ is a new weight, $W(t)$ the old weight, $L_0$ is called learning factor, $d$ is the distance between a neighbor node and BMU, and $\\Theta$ is a factor that takes into account the neighborhood."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* Repeat steps for all vectors over enough iterations for convergence"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The node obtains a color according to with the factor $D$ and the chosen vector. If a node is close to a BMU it is similar but it is far away, then it is different. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![SOM6a](SOM6a.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![SOM6b](SOM6b.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" \n",
" \n",
" \n",
" Strengths/Weaknesses "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" \n",
" \n",
" \n",
" Summary "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Unsupervised learning algorithm.\n",
"\n",
"Used for clustering.\n",
"\n",
"Dimension reduction."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" \n",
" \n",
" \n",
" Example "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Imagine that you are in a valley where the hight is determined by a fractal. The group of people is spread across the land and everyone wants to stay in sight of the others at similar hight.\n",
"In this example, the people is our nodes. We will use the SOM algorithm to make the people gather in clusters.\n",
"\n",
"You will first find the person with similar hight, and call that person the BMU. Now everyone that is close enought to be seen, will be your local nbh and they will approach the BMU. This local nbh will shrink with time. Notice that when there are abrupt changes on hight the BMU may not be in your local nbh.\n",
"\n",
"The following algorithm implements the previous stragety, we will use the nbh radius as before but the weights will be updated with the rule $ W(t+1) = W(t) + c \\left ( x - W(t) \\right) $."
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "80ea9d977cb24a9f81fc3b689bed18af",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"HBox(children=(IntProgress(value=0, max=500), HTML(value='')))"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAVIAAAFDCAYAAABoX4bRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuUG9d9H/DvvQNgn3yTkmjRu9rVriiTFmmJNJfmSqIpmiSAjWMltqLETmvFTSzZafNw0xO3zWmaNqcnPs2rOW0sJU4dN09HjmNHWQArWm+RImmuFEmRLJoUKa4pyxIpiqTIfWLm9o87Mxi8gZ1dAAN8P+fwcAEMMBfAzA937v3de4VSCkRENH+y3gUgIgo6BlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp9C9S4AAOyRd3GcKhHVxX7rQeH3NVgjJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMgnBlIiIp8YSImIfGIgJSLyiYGUiMinUL0L0LCEKHy/UrUtB81Poe+P3x0tEtZIC/GehEJ/RMIwMo8VC7JUf57vR0Qi9n0y+zF+f7TAGEhzFTrJhERq4mj57ai+cr8T0wSEhAgXuPDi90cLqHkv7UudKLmXeAW2lW1tUOk0RCiE5MlDiPZsBYSCCIegZmczz+PlYmPwfIfCMKAsBQiJsTNHAADRnq36PgBQVvZzKjgeXPy+qYDmrJEKoWsioTBEJAIRChd4vPRlnjUzA2Up9/IwNXEUQopMEPW+FtVXznegTBMiHELq9JHMfZbSAdQJornPL3Y8CJn5V2BfRAAgVAP8wu6Rdy1sIexAOnZmHACwb92WwidQ2deREFKfOHm1mUIa4LNsOaUCW+73N59jAPrqJHnyUPZxxO+6aey3HvT969h8l/aeEyvasxVy2VIAF+f/eoah29qo8VRaOxQSwgBUen6BVKXTumkHurlgvgGZmlfzBVKHsqBMwHzHDqJC6raz9Fxlz7e3r0qxNrdKnlOJoNWCKn1vi/F5CU+rlRRV/xjKtjZYs/pYkd1dMC9equr51Fqaq420yEkmpICMhGFcvUYHR1HZ21amqU9Aw8j0/Hrby0qVo9J/1fDz3FqZT/kW8POSHR1uu7bs6tTflWm6nU/lyyLdS3kRDsFYuRzmpcuFy0xka65AWoTs60Hy5CGMHhmFsWa1vrNQQMztWIDdtmb/k+1tbptbQ2ikk7lBgrs1PQNYCsaKZVDTM/kbFPiOAWT/wEr9f+rUYSReeGSxi0xNoHkCaYGTWITCEFJgqn+le9/oeApCCgjDQGjt1VknloyE819DCqROH0Hq9BEkTxx0e+/zMgEW6m2EwpCdnVknujCM4vurdwArsX9hGJAdHdm329oyifILXRTDgJACxjVX4ZX/coP7vUFIhK5ek9lOCsj2trygKrs69Q/lYG/W67rb5u2w/j8c1BiC30Za4mBWdrtY2+MvZt3vTa735heqdBrG9b1IPPFNRHu36U6FAu2kqYmj+nGfHQ9uvqPnNZRpAjMWZCTsttEpS0FGJFS61It5PodatKVWEkSE1OlibuqQhDIt93txNwuF9X1+OnHsfeQNnAAguzpgnjsPYRhuSlS0b8jzVIHUxFHEb7oDyWNPAci8RmxwGNb0dIn9zqNdnJpOMGuklbSZ2akvsr0Nsq2t6GZGzzpdi1mxDKmJo0g88U39wPsHde2zxD6q7ozKKZ9buzUMjL3+XFbASZ48BGNpN8bOjEN2tCN58pBbexKGUXrfi1lLLfbaTrkikaxaXmriqG4ScfI6naaRnPcxdma84rbroop8Jsq0INrbsvJKU6cOQ0bCugx2rTnx4qN5z00eP4CxM+Plm3Qave2aFlUw80grrA3JSFgHoDLiu+9C4pEHqyrCyLYRmGfP5SfoV0pIGN1dcD9/y4Ka0W16hWpVsf7tbg0V0IGnovzYhfp+y33mdt6ut4bvrQF66VQi+2mRCOSK5UDIwOgzD2HftTfPs3wSxrKlSLz0WN5D8c17kHh+f8GnxTfsROLlJ8q+fLRna15NuqQGOK+oMguRRxqsQJo7mUixIOJMNCIFxIYBJFN/67eIWWKDw1Czc5kTa57J/vo/YdfiBJLHD5R8SrRvCDBNiFAIKp2uPsm82u+6qp536V4i6+G0EpACqVOHiz4l2jek04wmJysb8FBi3/o/AdnZicQrT1b/GmVkDTEFyh57ehRV/c8tKm8hAmkgL+3HXn8u/05Px4GQwr0UW5Qgmk4DynIvDf1ckoq2NiRPHCwbRAF9OWr0rNO1WMPQnSBV7ayKNKMqL1GNVSsBIXWwNwwY69aWDKIAIJcuReKVJyE2DFT3Pryv0dYG2dGe+SykQLRvCLte+ti8X7MYpxnGvpHf++8ZTadv8zK/VQQykA598XOZnnNPuyFgX05OHHX/LbTk8QNuO6Do6ACUpYP2fINplYnio09/S7fvtbXpVJ8GYb59Hso0dbC/di1GD3y77HOcy235ToE8zUoICXnNVVBOk4eQSLz8BKyt78NjG8vvvxrO8WSsXum2mQopdH6xkAj1vhfCMBAb2LGg+6VgCF4gFRLL//KIe1kr7GT51OkjCF21GrK7a9GLkDz2FCAFrMlJAMi+5KuEN081nc7qQa7UvNtma2D06W9VvG2sfzvMN36UuaPKH6T0xOv6OAiF3EETDz/451W9RjVGx1PYe9c9mc7CUAhCCqQnXoeIhBvqx41qJ7htpIBbEy3UobHYnE6KaN8Q1Jydl1Sufa9AkAhdtRrWlUmkPzCA/V//akX7ztqns996jgHPyXmt5vuIDQ4DUsK6Mpn/YIWfp9M2Wy+xwWFYU9OFy9sA5xeV1rJtpA4hRSadBsAHf+NzNdu309Ob1Q5YoM0st+khqw0NgHnubaipKYR/5GNiFSF1O2FbWyZ5328qURGyu1vvr7sbr/7VZhjLlmYXxahuvyqdhpqayrrP+YxkW37SfGZHntt+0tAWgDCMvO8VAINoCwl0IFWm6dbM4rs+ge/+9pfrXCJbzolv3bYJIhRC6KrViK2/DSIcyuoQAwBr4vWKXjrau0333OfkNarZOVz66GYY696D1Okjuge7rciInHmQbW0QoTCSx56C0d2F5LGncMN/vwLr8hV3GyEFVDqNWP92bP7S58u+5sgH43r4rf1cp81xZMePw1i5HMmTh7JzOIsFVdNE/KY7EN9z94K810rtveseAIA1OZmftsUg2lKCFUgLHJxCCuxbtwXWq6/VvDixwWEd2IoYOzMOEQ5h/998VY/1f3ZMt6962lTFhgEdGCyF+O67KtuxkJDLlmaCjxQQhsSB//UARg/+IwDdOaLSpYZCVUFIIBx2L9md9KLJvuX2w3pSGNgZDMq0sPb+cez4wn0lX9Y89zYAQC5ZAuEMmrAUVDiUPca9xExcTpC1Ll9BYv/X5/0W50Me/Z7O4qi2jZyaTrACaQFOLqeIRHDTH5avBS0kpz2w2KiXkVv2FU4BkgIiFEJq4ijmVnW6o24qGRTgjB+f29CrRw0N9gFC4uR/zk9kF6EQjBXL/NVK7WaDmQ+tz3vo8a/8qdvpkjx5CKlThzGzezN++EtbkTp1GH/xpd8t+dKysxOvfm0DEi895mZDyP6ezOgyz3sWhiz6OUPIRRu/X0rq1OHibaPUUoLV2QQUH55oB6dKRjItpNjgMJS9LEmmOJlL0csfuxlP/9ED7mM77/0snnjgTxa+HAM7kDxxsOD91vTM/E/2Eh16ez/+aTz891+b3+tW4X0PfB69/+Oo+x4Kfdb16GyK33i7HkxQKIWtAc4rqkzrjWwCiic5e5aVqNVJFd+8B9alS9m5oEK6QSfWvx1T/3gNHn9/5elAC1q+TbthXbgIuWSJnpi4mmBq12KNwT5Yr74GceMAkg8v7OCGSsQ37oJoi8C6cNH9kRzZEoV59lx2cSORgj8kiyHau638qLYGOK+oMq3Xa19mpIhTU5lPXua8XLUy7y7ZlZk2DoaBM8+vrU1ZcsQGdgArlgHQ7YfGwHWFO2qKcNseT07o2z86i2jfEOIbdy1KeYtJvPQYRp8dg/CM4hodT0EuX5a1nejsyH3qoogN7OClPOUJ/jR6HkIKwDDKDk1cKCpsZDqOhISxeiUsz5IUethn+aGfi8FbO9tz988hZeeo6toUSgcDu93TERvYUfcJjr0Ti8TW3waxdjlw8V03h1a9Z02JZy+c5ImDiN90R9YSNkVXJmWttGUEp0ZaZso8GdG5kyJUu98GcfoN/YdhQGwcwOh4qmaXl9XwJvq7I78KTlSsa6zGwHVZdzfae0oeewpvD10FsXHAfR/q2Kma7f/yrQN6GZLurtLzHXCsfcsITiAtQ6XT9nyXtXtLamrKneFInvORUF8j8c17svI+8wYP2KyTp2tYqvk5/KUvI5n466w5Tod+vTYDMp788p/g+O/cDKUUh4QSgKAE0koWPbNrWnqG89pxmhFGj4zWdL/zkXh+P0Q4BLX9/Zk7Pe2mQgqMnRmv63DL+TBWroAwDKz85gs12+erd99fctJvF2ulLaHx20g9B6KIRLIn6/DUosxLlxHt3QYRCVc0JZ1fscFhHP+zGwDUNt3KL32ZfjB/fk0Axtpr6lMon0bHUwDscfs1sPtf/RuEn3wRUPb8AN45SG2FlpGh5hWMGqnDUu7CcLKjI2u0i7G0W/9Rgwb++IadSB4/gBO7KptkpBHl1jqFFBg9/E91Ks3CSB4/gPiGnYu+n/aTOvVKLlni3ifCocxyMXnzArBW2uyCE0iFhLFurR4B4xnh4qz8aV3RE1/IpUsKPn0hVbI0RRA01NLSCyTx8hOIb9q9qPuw3nhT/z85mTdngrNOmDvkdZEmj6HG0vgJ+SIzSsiZYcdd4RP25f5cui5T6TUDZ/2koLWL1svIB+MY/W4CgB4skHjpMfd4VJbKm9LPncykAc4zKqylEvJFOIRoz1Z86Nf0RBjKUroNipdNvlz6xFbWmqrgBFFADxaI9g0hdfpIZklv+//Y4DCiPVthrF5dl3JSbQXmDFKzs1CWwrIHn81aA70Wk0bEP/JTi/r69XTw9+8HxmqTzB5ksb0/XfgBS2VWc7X/RfuGYE1NQ5kmzHPnCj+PmkpgAimg20O9QbRcAI3fvHdB9pv4zt8tyOs0qtSNjZ+6VW/i7PmKt1Vz6ez2Z17WN73GD6RKuQeiNVNgFiO7bcpp64utvw2Abr/CymV6fPjOn5zXrmvRA0y1Md/vMr55DwDAOn8h7zXiN91RZOYnq/D91LQav7PJUa4t1LtOvL3uu0N2d1Xc097/jXux/osvQi5dgtFnxyp6DgVDbHAYUKrskNf45j3uCqdATudmKARl6kApDCP7CqmQBji/qLTWm0avgmDqHtzQM9RH+4byJjGJ9g3BWHu1O5t8fNNuWO9ehuxohzU1DeuWG2syzybVhzM7mGxrww8++36s++N/RvLEQT2zk2VlzWkbi38SyhAQ35/IXqCvXLt8A5xXVJnWC6RA5b30drqUsxSIMCTEurVQb7wFSAnZ3QXr4iVMfXgj2vc/n3mePXaemle0byhruRdAZ4WIjnao2TnI7i6Mjqeyl5Gxm5Ccv8tqgPOKKtOagRSoOJiKUDj7BPA+lpuMbjcNMB+1NXgv13PlHi8iHILs7oJ5/p3KXrwBzimqXEvlkWap8EBV6bmijf5uHqqH7Huv76JRMKROH8nKn3WOh0I/ump2lkGUSgpmIAWqO2CdHL8i5PXXwVi3FonH/34BCkZBMPKhj0Le0KdXPi2mzHFD5AhuIF1Ib78D682z+NC/L718MDWP0Wcegnr1dGai64XA2mjLaq1AWqB2Id5/AxIvPAK5cgWe+b3761AoqpfkyUNIvPho4XkGqq2JMoi2tGAH0moPXm+uqf1PvvMugGBMzEyLxz0eSi0dQlREsAPpPKUmjrprO6npGXc0FLWekVvvRKx/OwB9XHBEEs1HSwbSkVv2IXnyEFITR/V66TVenoQax+jT38IPfuUWYPN6AMhMFs4ZsagKwT9aqry8N5YtzRr6eWXkAwtdIgqYnj/+F6Qe+isAeikWY9nSOpeIgib4gbQKwjCyxuADwFP/+4E6lYYaReKVJ7Nvv/QYRLjxlzOjxtE6gVRIKNOE4vK5VEZ8z91Qc+nyGxLZgh9Iq5ghX3a0w7iakxhTaeq1M3pZG7aTUoVa4/rFPiGsqWkI4109zpqTk5BH/Oa9sC5cBCwFYZh61ntAHzvenNLc20RohhppDhGJ6MlKnGVxc2oV5uUrgLIge67FyLaROpWSGk3iuYfx7sduhuzvyWtH9x5LMhJmTZXyNNURIdvaANOEMIq/LSEFYBhIPPFNjkahLAf+8H6ImdmS26h0mpf9lCfYR0NO+6gyLfd/IYWe/qyjHUIKjJ0Zd6fOc5LxvStCEgGAevcyALjHjzPayTmOgMwS1kSOYAfSHCo9p4OkFDCHb0Lq1GEkjx9wA6fRs0733s/O1bmk1Kisy1cAIZG+dRNEKAQRCiF54qB7HIlIRG+Y207KZcFbWjAndnYUOHidkSly2VIkXny04NOifUMwViznmkyUJX7THbAuXynYCbnjV+/D0m8+CygLIhTSCzHmaoBziarXuhM7l+BMzFssiAKAEAIXh6+rUYkoKBIvPgohBLb81ufyHjv4B/dDLlsCALB4RUM5mib9SUQiFSdR68XNDpXdjlrQ+j6M/+aXCz6UeOERex0nM9PZxFQoQpPVSEU4hLEz44XnlyQqY2RLFMnk35TcJnX6CIxVKyGkgLFmFXvvCUAzBVLThPGea+pdCgqyjvaKNks8v1+3k56/wBopAWiiQKpME+nTZ9w1y4mqNfr0tyre1potvrAitZ6mCaREfo0Mf6yy7bZEF7kkFDRNE0hDPesgpIC5fUO9i0IBNXrg25VtN55ylyYhApookI4+8xDkkiUIXZjGRz75mXoXhwJq7yc+XdmGpZZxppbTNOlP+669GRCXgBcvwQAQ7d2G1OkjBbeN77kbIw8exC8u/0FtC0kNLb5xF4ypY0Ufj/Zu87SLprM7mpiM39KCXSPNPXiVlTm4lYVY/3bEb7wdO76QWa8+NrAD1vdP4R9+cU8NC0pBIDo7IDo63MUQb3giUzuN9g3lBE721lNGsAMpULAmIKSA7OwEpIRYsQwHf/9+xDfs1DWKWT27T/jo8VqXlALAunwFamoKAND/r7+H+I23u4+lJo7qtlFe1lOO4AfSAlITR/UkzqEQzB++ifiNt+sTxL4sE+EQxNWr61xKajTH/udVEJEwAGDfui36eOlod1Pqor3bCi/ZzMv6lhfsSUscOZOXCMOAspT+v0Cun+zq5BLMVJQ7O76HjIRhzc7BWLMK5ltnsx9sgHOI5m8hJi1pjs4mpbKCqRM8VTpniQjn8akpxG+6A2p6BsnjB2pWTGpcd9zz8whdScNsMxDCP2cesNtCndme8oIoEZolkFZIGEbmxLh0uWivPrUeYzINeeRlSGUBUkC0tcGanCz/RNZGCc3URlrJAW0nUMtVKyG7uxAbHF7kQlFQhI6+AmP1SoiODgDglQpVpXVqpMpC6tRhxDftRuK5h+tdGmowemrFDL2cSJmx9KyNkq25AmlOW2mufeu2QMiLNSwQBVFsYAeUVWAGfC8GUfJonkt7R4kDXIRDBecq5bLM5JU8cRCyva3exaAAab5AWoKaSxdcAdI6/04dSkONKtqzFdZ0mRopkUdzXdqXoyyIUBuiPVthXHM11NQU1PQMwEl8WtKT08Dv/NhPAW+eQ+LFRxHfsNNdRRSqxLpMvKynHM1ZIy1xoFszM1CWgnn2HM59dD3UzAzUzAxi/dtrWEBqBL+zZRes46/BunhJj683DBjvvRYqzcXtqDrNGUjLURbUXBor//K7etVRIQEp3ckqqDUkXnrM/du6Mgk1NQ3zB6+XfhJro1RAawZSQAdTewSUsXolkicOInnsKWz7T/lL8VLzEuGQ/jFVFqypKS4fQvPSuoHUoSyk38wM+1v9rZfrWBiqhdjAjswNy6p8SjzWRqmI5g2k1Rz0ynInqrAuX1mkAlGjUKaJ2MAO7Fu3BdYs20PJv9bqtS9DB1PWOlqBm97ECZppATCQOpwTSkh7BJSAXLWSw0mbSHzTboiuTqi5H+o7GERpgTCQFqFTpN5GtGcr0rdtwnf+6v/Wu0g0T7H1t+mcYesCcP6CvpNBlBYQA2kuZWXPXWophJ56AdHebQDAqfcCIto3BNi98cCU7pknWiTN29lERFQjrJHmEjLnpuClfQClTh12/46tvw2YYq2UFg8DaRHZnU35M0ZRcDjrczmdTekf/NAeT892UloYDKQOT0107Mx4HQtCiyXxwiMAdPupmkszmNKCYRupx9iZcQjJqaBagTvfqOApQP4171FUYqb8/G2lWwuV3V2LVCBqFMIwkDxxEGNnxiHtdeyJ/GjeQFopIRG6eo1789ydG+pYGKqF5ImDmRtSVl4rrebHmVpK67aRCgkhhU68P3deT2RhGDhy7Mv1LhnVkJpL28eBhGxvg5qd5QxQVLXWrJEKCREO4fzPflC3iSoLsCy3d5daQ3zjLvdv2dUJ0dEO473Xln4Sa6VUQHMG0hIHu2xrg5ACxprVWP3QMYi2Noi2trzleKn5fXH8McjB6yCXLdU/oqYJ8wevQ4TYbkrVaa1LeyGh0um8lUSz5qeklnF7O3D7d/7OvZ14+QkA9pr2pVKjhODcpJSlOWukRRRbjlmuXFGH0lCjSk0c5XLMVJXmC6QlLuuLLcc8emR0MUtEARMb2MHlmKkqzRVIy3QEjJ0Zh1y+rEaFoaBKnjhYfmAGO53Io3XaSIXUU6uZFxG/eS8wOwc1O4vk8QP1Lhk1gFj/dsgVy2FdvgI1NYXUxFHsu/bm0k9iWynZmqdGWkkNwZ79x3r7PKzLVxhEyZXeeiPMc+ehpqYAALHB4TqXiIKkeQJpBZRpulOpyaXdPFnIZXaGYG3bgLnbN0NZCtbUdGUjnniJT2iWS/ucg1kYBpSl9P8FRqmIjg4kXny0VqWjAHj0z7/i/u2sKAvADaYyEoY1OwdjzSqYb53NfTq1uKaskaYmjkIYBmRHO4RhwOjucjsPhBSAaSJ++0/UuZTUaAYevwexweGsjiZjzSqIcAhK6R/mxHMPc8YoytOUR0S0ZytkRztUOg3jPVcj8cqTkN1dEIYBQKdBqTfP1bmU1GjW/4e3oOx17sfOjOvjZWranW0/dfoIoj1b3ePIxcv7lhf8QFrgIFaWgjU5CVgW1DsXseML9yHx8hNInT4CEYkAAOa2Dta6pBQAsrsLoqMDAHDy/70PiVeedB+L9myFshQnNaE8wW4jzQ2i3ksuIT3j5zMnQ/LEQcT33I2f+D/7F798FChqcgpqaso9br6/82vuY6lTh+2VZO0gytn1yUOoBsiD2yPvml8hPIF07PXnEN+4C6rnGsyt6MB3/pqL1VH19n7i03j4G18ru53OSTaza6cNcC5R9fZbD/pumwn+pb1t5EMfhfXuu0gvb2cQpXmrJIgCAHh5Tx5NE0jTE2egLAXj0Mv1LgoF1MjwxyrbbktUt5VyeWeyNU0gJfJr9MC3K9tuPLXIJaGgaZpAKgwDod51bqoKUbVGbr2z4m1lJJyfBkUtq2kCKQwD5g9/VO9SUJBNTVe0WXzzHqh0GnLlcibnE4BmCqTQifb71m0pOOcoUTmj4ynEYj9Tcpto7zaYb5/XiyaefZspUASgiQKpmp2t+KCO9W/Hrf/u3kUuEQXSsVPY8lufK/hQfNPuzDGmLAZRcjVNIHU446TjN91RdBulFJYdeK1GJaKgiN90B5RSGP/N/CW5d/zqfbAuvgtAt48SeTVfIA2FAMPA3Pt73fti/dsB6M6EaO82wFIYfXasXkWkBuXMCBbt3YaPfPIziA0Ou8fOwT+4H8KQEJEIVDpdz2JSAwr2ENEcIhTWB7mQMA68iGjfEEQoBJXW6++YE2f0dm1c2IwKk91dsC5eQujpF6AMAzBNvcqsEO5xBCB/iChHNbW0YNdIcw5eYUj3f2UpqLk0rKlpKEth37otbgK1U6MY+WC8tuWlhieWdAOAe/woS8GannGPIwAFV6Kl1hbsQJrDmpkBDAPKLN4JoCyl5yPd+ZOc/oyyDP/KfVBtkZLbiFBIT/zMjibyaKpLe8Duvc/lyfUzurtgTU3Dmnidyfvkit+8F0suPAfLUro5yJ6XFEBW0LRm5xhEKU/TBdKClAUIqWfMX7oEKc8ck0QA9Mz3ttjgMMbOHCpc82QQpQKCH0iVqvgS3ZqahprlejtUmrhuHS/fqSpN1UZakrIgDAOinT32VFpi/9chwsGvY1DttE4ghV6OWYSyT5Db/i1HOLW6+I23Z9/euAtqjrmiVLngB9Iqe97Ni5cwcss+93bX6D8vdIkoYCY+/35EP/opAEBsYAfMi5fqXCIKmpa8fhl9dgyx/u16Bp9VyxBbfxuSx56qd7GoDkZuvRPv/eGzbm6xu3QI20epCsFeswmorkZqp0FlrVt+7VqMPvPQvHdPzcGZMUxEIrCmZ6oPpA1wHtH8cM2mahPq7ZPDWSZCWQrWiiUAgJFtIwtdOgoQ93iYnim/MVGOYNdIqw2kBSbhFVJALl8GNTmFCx/bhGd+7/55FYWCJ9a/HaKrE9bFS4XXX2KttCWwRrpQVq2AvHoNg2gLGfnQRyGu74V1+crCvSiHHLes4AbSattGSywJYb36GswzbyD+4Y8vQMEoCEafeQjW90+VXlZRX+HiAAAQpElEQVS5zHFD5AjmUVJhEBWh4guUCSmyOp0AwDr1A99Fo2CI9m7LunR3jofcYwLQHVDGyhWVvTBrpS0peIG00gNVSKROHwGEhDAMyEgYRn8PZEc7ZFcnjDWrISIRTO+9OavWEe0bWqSCU6Nwv2Onxin0hM1y2VKIjg6Erl6DsTPjeiScYQCWgnXxUuU1VAbTlhOsQFruABUSIhTOPtilQOr0ESRPHkLiyX9A8vgBPUNUWwTJEwfx+Ff+FHLZEkAKyK4OAMDej396Ed8ENQQpIDva8fovb4UIh5A8cRBqahqYm3PXrU+dPgKxcQDYNAjR0ZF5Li/3KUdweu0rCKL6P2HPip8Z4ie7u5B4+YmKytL/jXux/osvQi5dwuVImkxscBhQCskTB0tuF9+8B4nn97u3vc0AIhSCMi093NgwdAJ/qd79Bji/qDT22juEhJBCX451dCB58hDkkiWQg32AYQBrVlX8Uic/8QCSJw5i9NkxxDfsXMRCUy3FN+xE8viBskEUQFYQjW/e4zYRye4uJE8eQur0EYydGYdcqmfTL1lD5WV+S2j8GqnnQJRtbfkT69pBtNDyD/Gb92bNM0k0X8WOpWjvtvxaqX1MZoab1v8co+JarkZqzc7pxn+nBlCmrWqhgmj8Iz+1IK/TqKKvcFRXOWrNyoq3FeFQdoI/a6VNLzCBVEQiEFLg4l23ZAVT2dG+6I3/ie/83aK+fj3t+MJ9wD5Odl1O8uG/LfyA3aTkzQBInTqsV2MwDBirV9e2oFQXgQmkai6N1MRRPPO7evSRm/PHyyZfln7jKGc6qoJ35dn4xl1InTqMaO82N//U+T95/ABSE0dhnjtXl3JSbQWnjVRIhHrXYfTAtxHt2QoRiUDNpSEMmbnkB2CsWsHe9gpFe7a6l6DF2pmDKL5pNxIvPLJorx/r365XqpUiMzLKMHRQtY9NCAFralo/piz+4Dew1mojVRbMM28gNjic1f5k2as9Ojmg1qV3F70ozdKbX3CijoCLb9i5qEEUAOTaq/X/nZ3urFEOZwYpNTPj3LGoZaHGEJxACgBSwJqcBJQFa2oq0ysKwLx0Wf9Rg4b9xMtPIDY4jIHHfm7R97VYnPk3HcpSGBn6sTqVZmHEBocrzhf2Y7pft3ta72Z+tNVcGvuuvdmufeauPNp8P1iUrfFnyPesEpq3Zr1zwAoJY2l3TU4iR/L4AcT6TeBkzXa5IGIDO2B+YBDC+pe8x8w3flSHEvk3siVqD+Gszf4e+Ys/A6DXejIvXylY61SlJkOhphOMGmkFv+jOdGix9bctdmmyOOO2gzAxdHzzHt2ufMgTRD01KGUp7Fu3Ja+22ujM8+9AmSbO/+Smmu3z+q/fh4r6F1gbbQnBCKQVEKGQHspn1a5NSnR0AJZCtG8I1uplNdvvfCWe3w/Z3ZW5w1uT8vwt+3trWKr5Gfr1zyEW/6T9netgdfhLX67Jvm//3Gcx+MXnIISA5PLehCAF0lK/7MpyRzx5x9gvepF61+o/TBPqpRMY2RJFbGBHzfZfqT13Z9py3YmMC3WC2LVT88RrWXc32nuKrb8Nqw6/BfXSicwY+PV9Ndt/99MnoObSMC9fKb00CWujLaPx20iroCwFARPRviGkTh1e9P2JORNKCsCEDkBnz2XV+GKDwzj+XzfhxKdqU1Pyig3sgLj2GlinJmCIlxD/8MftAFlmkg0AUJabGiUMA1CziG/aDevdy5CdnUi89Fgt3kKW+Iadbht48thTiG/anXkfyoL4YW0GFcQGdkDNepZrLvZZMoi2lODUSIGyB6eTDF2LIAoAeOt83l3WlanMDdOE1VGf9JfkiYPAOxcB6NmvzBOvFe5RLsJJ6ZH9Pfr2NWuQOnW45kE0vnEXRm7ZB+Wp+Y1sicK6cDFrOzU5lfvURZE8cZDT6FGe5jgi7EkiRChU06TyxPP7IUK6Uu+uTGqaiPZs1TU608J7Hs9+zs57P7soZSl0+Z144RGISATmxUvzWMjNvsw/fgoQsuAQyVrM25p46TGcvPd6KNNyP9f0m2fz8jcrmdVpoTizPxndXUVXYKDW0hyBFHos/plfuqWm+4zfeDuSxw/k3e+c5MaqFXj6jx7IeuyJB/4E0b4hxPq3AwA+8qnPzCvTwGn3jO++C9HebTj5HzfnbRPr3w7R0V71a+cSkTDuuOfn8+5/+O+/pjt7PHZ95hew6fc+DwB4de5yydeNb9yFwcfvyZR3YAfiuz6Rt9337v1jCEMfqgUHEQhZ82wNAEi88mRTDmqg6jX+ENFcOQn3wjB0W14dhjjGBoehZueK5gyOnRkv2F6bNVHwhgGo7+lkVDnQi8QjD5bcpxO45NJunTtpE6EQkicPZW/rDAH1O7pGSMiuTiSPPZV194d//hfQ/vBz7v6VUm4POqTApZ+4GQd/v/jKrNG+IcBSkN1dULOzejSQkJDXrUPiiW9mtrPfc6HP2V1jyR6iWUvRviGIUEgPBWUSfmC11hBRoOCoJWUpPcnu9dfVvDjJ4wf0pL9F7Fu3BWoujT0/83OI9W/HyC37dM3Js8CaetnueZaibBDNPMly12LPNClYGP7lezGy48cB6CDqNDv4pixgbs4NaPEbbwcAdJ66YD+sdNaEPS+nMCTeuG9LySAKAMZqPeG29e67mSGVUkDMpXVnkmf/xX6s3Lbc7i7E99w977c4H9bW9yF5/EDBBfOotQSrRlqgNuqtiXzwNz6H7/527XvI963bkn2HZ8SVQ0jhBn3v9s5JKPt6smphpUT7hqDmstO8ZEe7TsNKpwEhyy+BMU+yuxvWlUnIrk4cf+B63PD5U5nhuQBkJJxXMy4l2jeky+25RHY+IxkJu3MpAMibPNn9MxyqeW3UK37j7Ui88qQeIurVAOcWldd6NdIcyvJcSgI1DaLOxCVZq47m9oo7tz01qtyga6xeBdHRgblrfCT0KwvW1DSsmRko04RKzy1KEAUA6/Jlvb/Ll3H9p57XHVneopjV7VeEQtkLyyHzGVkzM1mfYfaOPLfrPBxTmWb+jynACZ1bSPACqXcJXbs25w7TvGUf4ht31aQY1tS0W5uqWIGAYJ57G2p2Fvu//tWKXyZ16jBkJJx5Te//9eDZd6mmjoJME2oqJ3WpijQtR/zG22s2cGDvXfe4zRyxwWG3WUJ2dla+ZDM1leB948rChZ+1J9K1L2HVXBrR3m1Iv3UuM3JnEcXW36Y7STo7AaD6NjJPkBCh+V2Wikik6ufUysitd1a8bfLkIRhrr8ncUWUADfVcq4+DdNpt7th71z1VvUY1RrZE8fCDf54ZtJBOQ1kKoZ5roWbnOGS0RQWyjXTs9ef0pVSxNjNPYFvonvzY4LBuhzTNrKV5q64RepZKKZRCVczIrXfCfP0N/RKGASu3NlcnxurVOkne/uyN91yD0QPfLvkcZ9njWPSnYf3LcX1nlZ+jbGvTK8UqBTU7C9ndBWtqGjP/dA0e21h6/9VwJ3Jx2p+9PG3ibhu4e6VQ//OLSluINtJABlL9tyx+0nnXuN8wgGSqyHo785SX9jSfy2pvGe0Z1csFVKcpQYRCbk2oqn1X+11X08bnWc012rNVvz8pSta2o31DemXYyclMZ5PPz1J2diLxypPVv0YZ3tUEAJQ99jgrfnC0XmeTUpmDs9QJ5+RohkIVBdH47ruqKkby+AEYV6+BCId8tU0a3V26o8UeHeOM3Ckk1r8dai7tphpVXNN2PrP5nNRVPtcJos4ILxRJVnffp51lYFx9FULvfY9eRG4+7O9ALllSMIjGN+8p+tSqVjso1vFVcBsG0VYSrEBaJZVO69Eydt5jISO33gnr+CnEb7oj6/7oyKcAwB2BlMt882zRQFGpxCtPQk1NQU1NIXn8gJsT6uZr2id5bHA4K6VISKEDUdnJRxbwZK4goEZ7tro/Cl6xgR1uh2C0byhreKeanYX51jmMPvNQ4Z7vKliTk4WLPj1dcORTrH87rCtTCz8qikG05QTr0r6QUpefziVfiTzD3AXgnHxOd/RRkREz0d5tvnM1nVFZuW29zrwB3hxKGQnrdKBK1OI7reCyX4TCOvXL+XztIJv7uYlQ2H/eq6dpIVd8w053Mhknq8Cbi+s8L37THUi8+GjWc2ODw9mL2BXTAOcRzU/rtZGWUuDEdk7kmd0fwGNf/Yp7v9OGZ1y1Guk33nTvz0sAR+HVNZ3nq3T2tgtBhMIQkXDWsENhGOX3V6/vsURAFYYBEYm4HWLCMNyhpHnLxixEUZyVZNdeg+/92jqcvEuPrIr2boOxeiXSb+qp9px2ae9cokIKiI4OqKkpiI2DSCb/xn0sNrBDb1twDtf6nz/kz0IE0uaZj9SztpN7V3oOEBIdJzPT3Y1siUJZZwGYOoh6To7cIAog61JbhEN6TLilALXwQdQpc27A1J1aRfJV630iO/svNHw3J0dUmeairmWkTBMQEuaP3sKN/20S0V+zJ1RRlhtE9U2lp+XLCowGrCu6aUAcP531ugyiVE7zBNISrFMTiPVvh1y9CuZZuwZabIZ4h6cn2EnpKTkbej000olcIqDWkmxvgzJNmO9chOzqzJrHFEDRy3M3wAvpLlcT7RuCXNIN4MIilpiaQXMF0gK1UsCugczOQb15tqoakTAMHUS9Y8EracerJsBVE3gaKXAW4y1jpe9tAT8vN69WSFhXJvUPoWFAoMLasLJgzczYWRJzMM9fgLG0O28obCC+C6qZ5gqkXnbng1y2FOY7F3WnR7qanEsLygSErOIjmm+KUbNajPdWaaBWVian01L235X/iHo79qzLVzIdg85rE3k0X/qT50RLTRzN64WtGtcnb1yVBuoS0/BVImvlBQZRKqB5a6SwJwSWAsJQ1dVGPbwjZfTyIQU2auZaZaMr0pwjIpGstLV556gKCWt2jsM+qaTmq5ECdvK4pXvAZ2fz04a8I3aKnBSyrU3PIWqn6birauZOFsKTqv5yvgNhGO5ENu599iQ3BWdmKnU85I5m4vdNBTRPHulCya3d2Cfe2JlxTtzb6IotQ2OnreXh90doxbH2tVDo5LKnTCu7HdVX7ndiGPrKJGc1gYLbEvnQ1G2k8+Ztd7Mv6TIzPfEEbGiefFa3FsrLclpkDKTF8KQLNn5/VEO8tCci8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJ6GUqncZiIgCjTVSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8omBlIjIJwZSIiKfGEiJiHxiICUi8un/A9Nas2Ru0rVbAAAAAElFTkSuQmCC\n",
"text/plain": [
"