{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "#Maximum Margin Classifiers - The Support Vector Machine" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<p>Given a labeled scatter plot, such as the one below, the best way to classify points of different classes is to find a hyper-plane (straight line in the 2-d case) that separates the two classes.</p>" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "[<matplotlib.lines.Line2D at 0x108b50450>]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAHfCAYAAACWMTteAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGBFJREFUeJzt3W+sZGd9H/Dvdnf9AlFYrJVMYi9aCULjVDKBNMaUIAah\nSmvUxlUipZVJosi88IsmEKlpbCdSfd+0SXkREEKiMZSIqBK2RKqIJCsSE7hKhWKsSHgxNHZsK452\ngbiBhiopkWIr0xfnLHt9PXPv3Dt/nvmd8/lII52ZOTvn93i9853nOc95TgIAAAAAAAAAAAAAAADA\nSHw8yXNJHp/z/ruTXEry5SRfSHLLhuoCAGZ4W5I3Zn5wvyXJK/vtC0ke2URRAMB85zM/uPd6VZIr\n6y0FAIbrH234eO9JcnHDxwSAwTi1wWO9I8ldSd465/2nk7x2c+UAQFPPJHldq4Ofz8FD5bekC+aD\nCpyusqCCdloX0NBO6wIa22ldQGM7rQtobKd1AQ3ttC6gsWPl3iaGyl+T5H8k+cl04Q0AHNMqhso/\nmeTtSc4muZzk/iSn+/d+Pcl/TDcp7SP9a88nuXUFxwUAGhr7UPmkdQENTVoX0NikdQGNTVoX0Nik\ndQENTVoX0Fj53CvfAAA4gq09xw0ArIjgBoBCBDcAFCK4AaAQwQ0AhQhuAChEcANAIYIbAAoR3ABQ\niOAGgEIENwAUIrgBoBDBDQCFCG4AKERwA0AhghsAChHcAFCI4AaAQgQ3ABQiuAGgEMENAIUIbgAo\nRHADQCGCGwAKEdwAUIjgBoBCBDcAFCK4AaAQwQ0AhQhuAChEcANAIYIbAAoR3ABQiOAGgEIENwAU\nIrgBoBDBDQCFCG4AKERwA0AhghsGazppXQGweoIbhmvSugBg9QQ3ABRyqnUBwCpNJ7nW074/mV59\nYzc5sbv5eoAhmx6+C7C46U7rCoADHSv3DJUDQCGCG4Zrt3UBwHb6eJLnkjx+wD4fSvJUkktJ3jhn\nH0PlAIxJs9x7W7ownhfc70pysd9+c5JH5uwnuAEYk6a5dz7zg/u/Jvk3e54/keSGGfsJbgDGZGsn\np92Y5PKe51eS3LSB4wLA4GzqOu4T+57P+5Wxs2d7NybXADAck2zRiobnc/BQ+b/d89xQOQBs8VD5\np5P8dL99W5Jvp5uFDgA08MkkX0/y9+nOZd+V5O7+cdWHkzyd7nKwN835HD1uAMakfO6VbwAAHMHW\nDpUDACsiuAGgEMENAIUIbgAoRHADQCGCGwAKEdwAUIjgBoBCBDcAFCK4gZGYTlpXAKsguIGxmLQu\nAFZBcAMMhlGFMTjVugCA9ZlOcq2nff+eezrsJid2N1/P2k2S7DaugTUT3MCAndjNd4NsmuTETrNS\nYEUEN0BpoxtVYIu4HzewRpXP/y5a+3RnnVWwcu7HDTBf6d7npHUBbA/BDbCUrerJ77YugPVzjhtg\nOZOsJTCPc+669KgCCxLcAFvJjHhmE9wAR2YmNyRmlQMlbWIm91adR2d1zCoHGCa9eK4R3ADL2W1d\nALRiqByAMTFUDgBDJ7gBoBDBDQCFCG4AKERwA0AhghsAChHcAFCI4AZgQIa/PKzgBmBIJq0LWDfB\nDQCFuK0nAMW5zWor1ioHYEmbuM3qylirHACGTnADMCS7rQsYE0PlAIyJoXIAGDrBDcCGDH9xlE0Q\n3AAbNerwmrQuYAgENzDDqMNl3SatC6A2C7AAs0xidi4rYXGUVRPcAGs35vA6sZvv/gicJjmx06yU\ngRDcQG/M4bJuwovtciHJE0meSnLPjPfPJvlMkseSfCXJz8z5HNdxw9YotWxkMdX/2y4z/8HciX2a\n5N7JJE8nOZ/kdLpwvnnfPjtJfqXfPpvkW5nd0xfcsDWqh8s2qx5e/t9YoSYLsNyaLrifTfJ8kgeT\n3LFvn28keUW//Yp0wf3CkscF1mu3dQHD5bQDy1n2HPeNSS7veX4lyZv37fPRJJ9L8vUk/zjJTyx5\nTGDthAt7mf+wTZYN7kW6+b+Ubgh9kuS1SR5O8oYkfzNj350927vxqx9gC5hctyKTbMF1/Lelm3h2\n1X156QS1i0neuuf5Hyb5ZzM+yzlugK3nHPcKNcm9U0meSTc57brMnpz2a0nu77dvSDecfv2MzxLc\nAFuv+uS6rdIs925P8mS6SWr39a/d3T+Sbib57yS5lOTxJHfO+RzBDcCYlM+98g0AgCNwP26Agxnm\npT7BDYzJpHUBsCzBDQCFuMkIMHAWD4F1MTkNWDPXILNVTE4DgKET3MCY7LYuAIbEUDmwh0u3GDxD\n5cCgTFoXANtIcAMjpldPPS4HA7bIxi/dmsR5bzg257iBPZa5dGvRnrTLw2jqWLmnxw0M0SRze9IW\nZIFV0eMG9ljm/POiPel19ridP+dQetzAkBy197t1PelJnD9nDQQ3MBAndvPdoJwmObGzwB/aXVMx\nsDaCGxixVffEt67XD2vlHDewIttwftmMdQ5l5TSAjt4twyW4AdZjt3UBsG6GygEYE0PlADB0ghsA\nChHcAFCI4AYoZxsud6MVwQ1Qz6R1AbQjuAGgEEueApRgOVW2j+u4gZFb9Ny15VQHwnXcAMVNWhfA\n9hPcAPXsti6AdpzjBmjqOOeundNmOzjHDYycc9cj4xw3AAyd4AbYHrutC4CjMFQOwJgYKgeAoRPc\nAFCI4AaAQgQ3ABQiuAGgEMENAIUIbgAoRHADQCGCG+DIFr1vNqye4AY4uknrAhgvwQ0AhaziftwX\nknwwyckkH0vyX2bsM0nygSSnk3wzfq0C5RznvtmwfU4meTrJ+XSh/FiSm/ftcybJV5Pc1D8/O+ez\n3GQEKMJ9s1mJJjcZuTVdcD+b5PkkDya5Y98+dyb5rSRX+uffXPKYADBaywb3jUku73l+pX9tr+9L\ncn2Szyf5kyQ/teQxAVrbbV0A47XsOe5Fuvmnk7wpyTuTvCzJHyd5JMlTM/bd2bO9G/84gK3knDbH\nMskK5ngtG9xfS3Juz/NzuTYkftXldMPjf9c//ijJG3J4cAPAkOzmxR3S+1sUcSrJM+kmp12X2ZPT\nvj/JZ9NNZHtZkseT/MCMzzI5DYAxaZZ7tyd5Mt0ktfv61+7uH1f9QrqZ5Y8nee+czxHcAIxJ+dwr\n3wBg6Cx1yko1uRwMYEwmrQsAwQ0AhaxiyVOAAbPUKczjHDew5Sx1yko5xw0AQye4ARa327oA2CaG\nygEYE0PlADB0ghsAChHcAFCI4AaAQgQ3ABQiuAGgEMENAIUIbqAIt9SERHADdUxaFwDbQHADQCFu\n6wlsMbfUhG1mrXLgAG6pyeBYqxwAhk5wA1Xsti4AeDFD5QCMiaFyABg6wQ0AhQhuAChEcANAIYIb\nAAoR3ABQiOAGgEKsVc7WmCYPJHl9ku8kufNE8u3GJQFwAAuwjNw02Z0m0/7xUOt6ANasfO6VbwDL\nmSYX+9B+dJqcaXD8B/ofDxdbHB8YnfK5V74BLGeanJkmD7UKTT1+YMPK5175BlSkl3lN6x4/MDrl\nc698AyrSy7ymdY8fGJ3yuVe+ARXpZQI0Uz73yjegIr1MgGbK5175BgDAEbgfNwAMnZXTgJWw8h2M\nz+iGyl2KxZC4QgGOrHzulW/AUY3ti84PlWFzhQIcWfncK9+AoxrbF93YfqiMjSsU4MjK5175BhzV\n2L7oxvZDBeAQ5XOvfAM42Nh+qAAconzulW8AABxBs+u4LyR5IslTSe45YL8fTvJCkh9bwTEBgGM4\nmeTpJOeTnE7yWJKb5+z3uSS/m+TH53yWHjcAY9Kkx31ruuB+NsnzSR5McseM/X4uyaeS/NWSxwOA\nUVs2uG9McnnP8yv9a/v3uSPJR/rnetYAcEzLBvciIfzBJPf2+57oHwDAMSy7VvnXkpzb8/xcul73\nXj+Ubgg9Sc4muT3dsPqnZ3zezp7t3f4BAEMw6R9NnUryTLrJaddl/uS0q34j82eVG0IHYEyOlXvL\n9rhfSPKzSX4/3czx/5bkT5Pc3b//60t+PgCwpfS4D+AGHQCDUz73yjdgnVZ9gw4/BACaK5975Ruw\nTqu+QYc7dQE0Vz73yjdgnVZ9gw536gJornzulW9AJe7UBdBc+dwr3wAAOIJmdwcDADZEcANAIYIb\nAAoR3ABQiOAGgEIENwAUIrgBoBDBDQCFCG4AKERwA0AhghsAChHcAFCI4AaAQgQ3ABQiuAGgEMEN\nAIWcal0AcHzT5IEkr0/ynSR3nki+3bgkYESmrQuAaqbJ7jSZ9o+HWtcDHEn53CvfANi0aXKxD+1H\np8mZ1vUAR1I+98o3ADZtmpyZJg8JbSipfO6VbwAAHMGxcs+scgAoRHADQCGCGwAKEdwAUIjgBoBC\nrJzG2lndC2CYXA42UFb3ApipfO6VbwCzWd0LYKbyuVe+AcxmdS+AmcrnXvkGAMARWDkNAIZOcANA\nIYIbAAoR3ABQiOAGgEIENwAUYslT2HKWjAW2leu4YQZLxsJglc+98g2AdbBkLAxW+dwr3wBYB0vG\nwmCVz73yDQCAI2i25OmFJE8keSrJPTPef3eSS0m+nOQLSW5ZwTEBgGM4meTpJOeTnE7yWJKb9+3z\nliSv7LcvJHlkzmfpcQMwJk1y7y1JPrPn+b39Y55XJbky5z3BDcCYNBkqvzHJ5T3Pr/SvzfOeJBeX\nPCYAjNayC7Ac5dfCO5LcleStB+yzs2d7t3/AkVm0BNhCk/7R1G158VD5fZk9Qe2WdOfCX3fAZxkq\nZ2UsWgIU0CT3TiV5Jt3ktOsye3Laa9KF9m2HfJbgZmUsWgIU0Cz3bk/yZLpwvq9/7e7+kSQfS/Kt\nJF/qH4/O+RzBzcpYtAQooHzulW8AABzBsXLP3cEYHRPXAFZDj5uNMHEN2BLlc698A9gO0+SBPpwv\nzjrHbeIasCXK5175BrAdDutRm7gGbInyuVe+AWwHPWqgiPK5V74BbAc9aqCI8rlXvgEAcATN7scN\nAGyI4AaAQizAwuhZkAXgeJzjpgkLsgCNlM+98g2gpmUuHztssReAA5TPvfINoKZlLh/TWweWUD73\nyjeA8bHYC7CE8rlXvgGMj8VegCWUz73yDQCAI7AACwAMneAGgEIENwAUIrgBoBDBDQCFCG4AKERw\nA0AhghsAChHcAFCI+3HDMbmPNzB2ljylFHcGA5ZUPvfKN4BxaXVnMPcAh8Eon3vlG8C4tLozmJ4+\nDEb53CvfANgE9wCHwSife+UbAJvgHuAwGOVzr3wDAOAI3I8bAIZOcANAIYIbAAoR3ABQiOAGgEIE\nNwAUIrgBoBDBDQCFCG4AKERwA0AhghsAChHcAFCI4AaAQgQ3ABQiuAGgkFUE94UkTyR5Ksk9c/b5\nUP/+pSRvXMExAYBjOJnk6STnk5xO8liSm/ft864kF/vtNyd5ZM5nHeuG4gBQ1LFyb9ke963pgvvZ\nJM8neTDJHfv2+dEkn+i3v5jkTJIbljwuAIzSssF9Y5LLe55f6V87bJ+bljwuAIzSqSX//KLd/BML\n/rmdPdu7/QMAhmDSP5aybHB/Lcm5Pc/PpetRH7TPTf1rs+wsWQ8AbKvdvLhDen+LIk4leSbd5LTr\ncvjktNtichoAJA1z7/YkT6abpHZf/9rd/eOqD/fvX0rypjmfI7gBGJPyuVe+AQBwBE0uBwMANkhw\nA0AhghsAChHcAFCI4AaAQgQ3ABQiuAGgEMENAIUIbgAoRHADQCGCGwAKEdwAUIjgBoBCBDcAFCK4\nAaAQwQ0AhQhuAChEcANAIYIbAAoR3ABQiOAGgEIENwAUcqp1AcBLTZMHkrw+yXeS3Hki+XbjkgBe\nYtq6ANgW02R3mkz7x0Ot6wHWonzulW8ArMo0udiH9qPT5EzreoC1KJ975RsAqzJNzkyTh4Q2DFr5\n3CvfAAA4gmPlnlnlAFCI4AaAQgQ3ABQiuAGgEMENAIUIbgAoRHADQCGCGwAKEdwAUIjgBoBCBDcA\nFCK4AaAQwQ0AhQhuAChEcANAIYIbAAoR3ABQiOAGgEIENwAUIrgBoJBlg/v6JA8n+bMkf5DkzIx9\nziX5fJKvJvlKkvcueUwA4Jjen+QX++17kvzqjH1eneQH++2XJ3kyyc0z9puuvDoA2F5Ncu+JJDf0\n26/unx/mt5O8c8brghuAMWmSe3+9Z/vEvueznE/yF+l63vsJbgDG5Fi5d2qBfR5O15ve75dnFHBQ\nES9P8qkk70vyt3P22dmzvds/AGAIJv2jqSdyLdS/J/OHyk8n+f0kP3/AZ+lxj9w0eWCa7E6Ti9PZ\nEx0BhqRJ7r0/3aS0JLk3syennUjym0k+cMhnCe6R60N72j8eal0PwJo1yb3rk3w2L70c7HuT/F6/\n/SNJ/iHJY0m+1D8uzPgswT1yfU97Ok0e1eMGRqB87pVvAMuZJmemyUNCGxiJ8rlXvgEAcATHyj1L\nngJAIYIbAAoR3ABQiOAGgEIENwAUIrgBoBDBDQCFCG4AKERwA0AhghsAChHcAFCI4AaAQgQ3ABQi\nuAGgEMENAIUIbgAoRHADQCGCGwAKEdwAUIjgBoBCBDcAFCK4AaAQwQ0AhQhuAChEcANAIYIbAAoR\n3ABQiOAGgEIENwAUIrgBoBDBDQCFCG4AKERwA0AhghsAChHcAFCI4AaAQgQ3ABQiuAGgEMENAIUI\nbgAoRHADQCGCGwAKEdwAUIjgBoBCBDcAFCK4AaCQZYL7+iQPJ/mzJH+Q5MwB+55M8qUkv7PE8YZu\n0rqAhiatC2hs0rqAxiatC2hs0rqAhiatC6homeC+N11wvz7JH/bP53lfkv+VZLrE8YZu0rqAhiat\nC2hs0rqAxiatC2hs0rqAhiatC6homeD+0SSf6Lc/keRfz9nvpiTvSvKxJCeWOB4AjN4ywX1Dkuf6\n7ef657N8IMl/SPIPSxwLAMjhPeCHk7x6xuu/nK6X/ao9r/2fdOe99/qXSW5P8u/SDYn8+yT/as6x\nnk7y2kPqAYCheCbJ6zZ5wCdyLdS/p3++339OcjnJnyf5RpL/l+Q3N1IdAPAi709yT799b5JfPWT/\nt8escgBo5vokn81LLwf73iS/N2P/tyf59GZKAwAAgBFbZAGXc0k+n+SrSb6S5L0bq249LqSbC/BU\nrp1m2O9D/fuXkrxxQ3VtymHtf3e6dn85yReS3LK50jZikb//JPnhJC8k+bFNFLVBi7R/km6xpq8k\n2d1IVZtxWNvPJvlMksfStf1nNlbZ+n083ZVHjx+wz5C/9w5rf6nvvfcn+cV++57MPkf+6iQ/2G+/\nPMmTSW5ef2lrcTLdzPnzSU6n+we6vy3vSnKx335zkkc2VdwGLNL+tyR5Zb99IeNr/9X9Ppfkd5P8\n+KaK24BF2n8m3Y/0m/rnZzdV3Jot0vadJL/Sb59N8q0kpzZT3tq9LV0YzwuuIX/vJYe3/8jfey3X\nKl9kAZe/TPc/eZL8bZI/TXcOvaJb0/3jfTbJ80keTHLHvn32/jf5YrovsnnXx1ezSPv/OMn/7be/\nmGtf4EOwSPuT5OeSfCrJX22sss1YpP13JvmtJFf659/cVHFrtkjbv5HkFf32K9IF9wsbqm/d/meS\nvz7g/SF/7yWHt//I33stg3vRBVyuOp/uV8sX11jTOt2Y7tK4q670rx22z1DCa5H27/WeXPsVPgSL\n/v3fkeQj/fMhLRG8SPu/L90ptM8n+ZMkP7WZ0tZukbZ/NMk/TfL1dMOm79tMaVthyN97R7XQ9966\nh2IOWsBlr2kO/pJ6ebpeyPvS9bwrWvRLeP+iOEP58j5KO96R5K4kb11TLS0s0v4Ppru0cpru/4Mh\nLRG8SPtPJ3lTkncmeVm6nsgj6c59VrZI238p3ejiJN1CVA8neUOSv1lfWVtlqN97R7Hw9966g/tf\nHPDec+lC/S/TLeDyv+fsdzrd8Nl/T/LbK61us76WbrLdVedybUhw3j439a8NwSLtT7qJGR9Nd67n\noOGlahZp/w+lG0ZNuvOct6cbWh3CZZSLtP9yuuHxv+sff5QuvKoH9yJt/+dJ/lO//Uy6Rav+SbqR\nh6Eb8vfeosp87y2ygMuJdCutfWBTRa3RqXT/IM8nuS6HT067LcOapLFI+1+T7lzgbRutbDMWaf9e\nv5FhzSpfpP3fn25tiJPpetyPJ/mBzZW4Nou0/deS3N9v35Au2PcvIV3Z+Sw2OW1o33tXnc/89pf6\n3ltkAZcfSXdzksfSXSLypXS/SKq6Pd3M+KeT3Ne/dnf/uOrD/fuX0g0bDslh7f9Yukk5V/+uH910\ngWu2yN//VUML7mSx9v9Cupnlj6f+5Z97Hdb2s+lWlryUru13brrANfpkunP3f59uVOWujOt777D2\nD/17DwAAAAAAAAAAAAAAAAAAYOv9f1j8x/hntmr+AAAAAElFTkSuQmCC\n", "text/plain": [ "<matplotlib.figure.Figure at 0x108a31490>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import course_utils as bd\n", "import pandas as pd\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import warnings\n", "warnings.filterwarnings('ignore')\n", "reload(bd)\n", "%matplotlib inline\n", "\n", "#generate data\n", "m = [[0.25,0.25], [0.75,0.75]]; s = 0.025; n=20\n", "X1 = pd.DataFrame(np.random.multivariate_normal([m[0][0], m[0][1]], [[s,0],[0,s]], n), columns=['x1','x2'])\n", "X2 = pd.DataFrame(np.random.multivariate_normal([m[1][0], m[1][1]], [[s,0],[0,s]], n), columns=['x1','x2'])\n", "\n", "plt.figure(figsize = (8, 8))\n", "\n", "#Plot data and any hyper-plane that separates the data\n", "plt.plot(X1['x1'], X1['x2'],'r.')\n", "plt.plot(X2['x1'], X2['x2'],'b+')\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<p>The issue with looking for just any separating line is that an infinite number of lines might qualify, even if the space between classes is quite small. The following shows the same data above along with many qualifying lines.</p>" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "(0, 1)" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAHfCAYAAACF5nuqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XnYJFV9L/DvOxszAwwDDgw7I/sOyg4aXncgBgxiXEBE\nEEjMEyWJyiKRN3ku5npvrhAM0SCJMQmRiRqX4IRojK8riCgQMMEALmExXPRqVEBFrPtHddnnPe85\nVaeqzvI7Vd/P8/Qz83ZXn/Ot6u7z61NdXQ0QEREREREREREREREREREREREREREREREREY3GXwB4\nGMCdNctcBeAeAHcAeFqMUERERGP1TJTF1laYTwKwafL/owDcHCMUERHRmG2AvTC/C8BLlb/vBrA+\ndCAiIqIhWuKhjZ0A3K/8/QCAnT20S0RENDrLPLUzo/1dGJa5F8AenvojIiKS7j4Ae4ZqfAPqd2W/\nTPnbtivbVKyp3pMot5vrtpsLF8WrNusk0VzqACMwlzrACMylDjACncY5H7uyPwrgzMn/jwbwfZRH\ncVN/S5X/51zIbIa4TkREvbjsyn4fgOMBrEP5WfJlAJZPbvszlEdkn4RyV/WjAF7tP+aozWBawAos\n/tggR+o6ERFRIhyI+ynQvAt4Nk4UL6p1qft+vFSzqQOMwGzqACMwmzrACIive+IDZqAqZpenDuJJ\n7p81ExHVET++iQ+Ygb0wLWa+jqhPqVqXC1IHISIKQHzdEx8wEz/CcGaaSzCcdSEi0okf28QHzIjL\n5825qNbjgNRBiIg8Ez9Giw+YmaEU550xjPUgItKJH9fEB8zQUIpztQ6bpQ5CROSR+LFZfMBMVUXt\nr1MH6eElGMYbDCIilfgxTXzATO2HaVFbkThLHyzMRDQ04sc08QEzNoRd2puQ/zoQEanEj2fiA2Zu\nCMU59/xERKpkP2LRxgsi9zcm6jm0cy9uuecnIspCNRv6zdRBBi73mXPO2YmIVOLHsgLANZN/r06c\nZeiq4rYpdZAOcn9jQURUET+OVQFfN/n/zQmzDN0+mBa3lYmzdMHCTERDIH4cUwOehOngO4TfF5Yo\n55lnztmJiCrixzA94EGYDr6r4scZhZwLXK65iYgq4scwU8D1mA7A6+PGGY1ci3OuuYmIKuLHL1vA\nVZgOwAfFizMquRa5HDMTEVXEj191AWcwHYRPihNndKrt+9nUQVrI9Q0FERGQwdjlEvCmyXKvD5xl\njPZGnp/pszATUa7Ej12uAa/G9DvP5FeOM9AcM9PgFLOpE1CWxI9bbQK+drL8VwJlGbMcC11ueWlw\nirnUCShL4settgFfgOmAHPuc3kOXW3HOLS8NDgszddJpzFrmO4VH/wTgAABfBfAkgM0BPJY00XBU\nB9sBeZzkRc1LFEkxC2B28sdlylNwHpiZj58nd8Ust5s8XQfWbTGdLe3gLw5hul3/NXUQB5w1U0Kc\nMfc3ym2Yxc8+dvEIpkcRPwTg0IRZhmavyb8HodwjIZn0WT0RUXb6znRmAPx00s7J/ePQRE4z0Srn\ntamD0NjwqOxuitlyplzMAUWh/H82ZaqIxI+rvgJ+ZtLW73pqj/IszkSUFe7KlshnwCsn7b3HY5tj\nl0txrjJyrwlRVliYJfId8Dzkc+BSLnIozptDfkaigeqzC3o0u69V4sepEAGfA37X2bdqe96TOkiN\nKuN2qYMQjcsoZ719DPao7DqfBLDf5P/Vd52pn90n/+4JYE3KIDWOm/z7cNIURESZCzmlXwd+9uhT\nDru0q3z8GhUNXOpdwKM/sroPyWMogPABt8B0sH5n4L7GQHpxvgqy8xF5Imn3saQsWRjlrmzVjzBd\nn1+f/E3dqTNRicXvdakDEBGFIPlc2V1UuzYLTI/e5a7O7nI5p7bkbEQdiD1Pd8q+KYDYsy7pu2Jz\nUm3HH6QOYsDHmAaOu48zNvpd2Trpu2JzUh2pvSWAtSmD1OBjTETUUqqB81OYzqpWJsowBJL3QEjN\nReQBj37OmPhxKWXA12M6eO/VsCzZSS3OUnMR0biJH5NSBzwc08H7txJnyZnUIigxExGNm/gxSULA\nrTAdwO9KnCVnEouzxExENG7ixyMpAauvAHEQ70fiNpSWh4jGjUdlO9K/88qBvJunKv+X9mMSfEyJ\niBxIHCwlzvpyInH7SctDROPFGXMHnDn3I3n7SctDRORk7IUZWFxcnpIqSKakFWeempOIyJGEQbvO\nbZjuBn1Z4iw5krRbW1IWIhov8WOQ+IAAzsV0QL8lcZYcSSqIUnIQ0XiJH4PEB5zYGbIKTE52w3S7\n7ZA4Cx9DIkpN/PgjPqCC33XuTtJ2k5KDiMaJR2V7xO86dydxuz2YOgARkURSBum2JM0AcyJlu0nI\nQETjxBlzIPoMkF/HcSNt5nxh6gBERNJIGJz7UGeAT21YlqZSz5yXJu6fiMZL/LgjPqADtchcljhL\nLtSj3HdKlKHq/4BE/RPROImve+IDOroY6WeBuUm9vfZM3D8RjZP4MacaGIfwufauSF9scpN6e1V9\nL0/UPxGNj/j6UCD94Ozb0NYntJTb6/yEfRPROIkfbwoAq7FwcP5B0kR+sDi3k3J78XEiopjEjzdq\nwOdh4QD9+iSJ/GFxbifV9rorUb9ENE7ixxpTQHWgLFB+dpsrdT34Xed6O2K6rTZE7puFmYhiET/W\n1AXUP3/OtbCp63Bg4izSpZo1c+8GEcUifpxpCqj/cIT4FbJQ81+bOIt0qYszEVFI4scZ14B7YOGA\nfXOwROG8Gfm/wYglxXbiY0NEMYgfY9oG1E/kcar3RGHtBBYAVymLMxFRKOLHmK4B9d3ba70lioPF\n2U3s7cTHhYhCEz++9A2Y8+fPueaOaT2m22iPSH3yMSGikMSPLz4CrkC+BTrHzLFx1kxEQyJ+bPEZ\n8CgsHFS/6LHtkNTMQzhneAipijMRkW/ix5YQAd+NhQP5iQH68E3Ne2ziLFLFLM6cNRNRKOLHlZAB\n9d3bqwP25YOa9ZOJs0iVojgTDVQxmzrBsFm3b6dxZSi7U2ew8Gxhj0L2QDsD4JLJ/58N2VlTUR/P\nmLu1iYZoNnWAgZv12dhQCnNlBsB2yt+SZ0J/CGAH5W+pOVPaVvn/vgH7yfUUsEQ0QMtSBwjgEZQD\n7ZsAvG1yXQHgUyhnp5L8F6anIsXkXxaJqe8o//93xNk2I3sMillgZj51CgqhmMV0JneZ8t5/no+5\nD8PYvqlmhPrnz8ckytGEByHZxdo2I9z+xVzqBBQDH+ewrNt31J8x19E/f/4CZH5dKcVnqrmIvW24\n/YkomSHuyrapCvTPJ38/qVwvhb5bewWAJ9LFESXGLn+1jwEbxi44amU+dYCBm08doCtJA95+kH0G\nMTXXrybOIk3ox6xq+/2B2heGuziJApJWWxZRA3588vfTEmWpvAcLB/or0sZZQM3148RZJNka0+1y\nYKA+JL5ZC4SFmSigYOPICQDuBnAPgAsNt68DcCOA2wHcBeAsSzsFgOtQ7i5ci+ng9wWk/7xXnz3v\nkzbOL8xB7qw+pViz5hcFal8QnniCxibqcz7IGLUUwL0ANgBYjrL47qctM4fyO7lAWaS/C/Nn1+pg\nWg14ZyrXvcBf7M4k7t5Wf3VJSiYJQm4T9Y0jEQ1K1L1EQY7KPhJlYf4myoOQrgdwirbMtwGsmfx/\nDcrC/DNLe6sBfAvAh1AG/gSAlZM2bgTwfyd/pzKD8oCrioTB+WHwiG2TkNvk+8r/13lum4iol9NQ\n/lBE5QwA79CWWYLyiLSHAPwQ9h+SUAfPQ7FwxrMEwPOUv8/umduHcyFvBi0pixShtskJgdolouiK\n2XKmXMwBRaH8fzZ0xyEafTGaC/OlAK6c/H8PAF8HsKWhLVPAR7C4QH9a+XunrsE90ovzDWnjsDgb\nhNom3M5EgyN/V3bT95gfBLCL8vcuAB7QljkWwOWT/98H4BsoD5661dDenPL/eZTnQt4K012HTwJ4\nKYDXA7hN6Svld42rvqsN/MuT/5u2Raw86vd5NwfwWIIckqzF9Dl0KMpjIXy4HsDLEO5700Q0LLOI\n8IMhy1AW2w0oP3s1Hfz1dgCXTf6/HmWx2sbQVtM7h0uxcOazQvv7jNbpw5Cye1vNcE7CHFJw1kxE\nDvI/KhsoPzP+GsqDwC6eXHf+5AKUB8f8A4A7ANwJ4BU9A+qF72WQUQhVO0BGgZaQQZIQ24Lbl4i6\nEj9utAm4JRYOiJdrf1/lPV03H0T64nhJ4v6lCVmciYjaED9udAl4JRYOtMdqfy/3lq4fvThfErl/\n9Xu34p8IEfjeFty2RNSF+DGjT0C98KWepdrouVYl7H/sQhVnIiJXncaM1KfCdDUDYAvtupuV/xcA\n9owXx0r/icnHEHcw54lIprZS/n+0x3bHvl2JaEB8DWi/hzxmz8chXTaJ2yMFzpqJKCXx40Wor7GY\nLud67quPVG8e1P7WRupTIp/bnW94iKgN8WNFiIArkcfsGVic7eWR+7w0Qn9ShSjONEr8NS5qRfxY\nETLgH8NenP8xYL9d6PlCn1FK8puVmHxtA27LUePvV1Mr4seJGAFzmT0vRdx8vxOxL8l8F2caHRZm\nakX8OBEroKnoSS1Kr0e8fFtE6kcydRs8s0c7Y9+OI5Psl4kof+LHiNgBr4O9OK+OnKVJzDcQYy8q\nnDVTD5wxUyudxohcvsfcxemwf377KGQNqvr3n4Ey39MD9aX2MTa+13+M25CIBiLlAFb9VKLpcljC\nXDYxZs+cOfdf/zFvv5Hi7mtqRfz4UKD8Deam34AOyfSjE1IH16cgfEa17W0DtC9d320r+flDROmJ\nHxvUQewNgrKolzemDGXxRYQt0Gq77/Tcdg58FWciIp34saFA+Zn2xzAdzC5Imiif2TOwOON/BGx7\nTFZjut7P7nD/6r4/8xmKiAZB/HiqBtwOCwvBwUkSlf4I5uL81YSZ6ug5t/fU7msx3uLMWTMRhSB+\nXDAF1H+QYmnURAvlNHsGwuRcFaDNXPRZ7+p+b/GaiIhyJ34crQuoDoo/jhPHKYv0IvUMhMkpfb1D\n6breyzvej3rhEdIkXqcxQcr3mGcArJn8fzOUK/M7CbOYPvsuAGweOUuTz8H8/ee+BWKs33Xuut5P\nKP8/0FMWajabOgBR7lwHul/GwplLyoEup9kzsDjn1R7bG5Mu631gh/tQLzwLF4knfjxoG/D7WDhA\nhv4Vpjqm4izxxCQVPWuf746r7WzonSwP6s+JntDifj62N9WSet7q1P2TUIMrzOr9JMzcTjNkkbzR\nTWc760ptY2P/aFnost3UgxkpOEkzZklZSBDxY0GfgLth4UD5Xi+JujEVZ4knJqnoR77n/gYppi7r\nO6btk5ikYigpCwkifizwEfADWDhY7u6hza5ymj0DfvKe0fP+OWq7vmPbPgn53n3ctj2pu9VJEPHj\ngM+AUgriLxmy3JUwjws977kt769+NUj8k86TrsWZstJn1ssZMxl1GgekfF2qrRksPBlJqoHwM1h8\nUNoBibK40n9i8hq0y/sExvd1qq7rO4ZtQ0QZCzVIHY+FM5o3B+rHRW67t9egX95c1tOHFZiu6684\nLD+W7ZI5X7ujU+6+5q5zwcSPAaEDfg0LC8U2gfuzORSLi520E5Povo7uBXpMxbnNuo5puwxErruj\nc809CqPalW2yDxbucvwu0gyKt2Px7u0fJcriand0P4OYvpt3f1+hBGqzSzvl9+6JKGNDPBHCDMrT\nelbn3C6U62PnUPuv/n8sgJsiZ3Fly3wQ6g9qq74zDZS/yvUlAEd6TyeDuq4F3J5XrstRWvOpA7gr\nZjE9Jellykt2HpiZj5+HcpVixng6Fu5SfEWCDACwh5ZD8uxZ1TZzjuvYlet6jmFbUFLclS2Y+Nd+\nyoB6wVglJMeHE+Vo42C0K7intlg2dy7rOZZtQcmwMAsm/nUvIaCUGZ2UHG20OeJ8ieNyuVO/0/3i\nmuWGvh0oKR6VLZj4172UgOshozBuJyRHW3rmexyXHarHwFkzEZmJf81LC/iXWDhgnpcoh17ozkiU\noy0995YOyw1Vm13aRDQe4l/zBcrZxV6pg2j0ApPqK2Q5zp4Bt9w5rldb3AZEpBP/etcH8MvSxllE\nQmH8EyE52jofzbnV2w6PFy0q1+JMROMg/vVeBXwaFg5gj0DOdzyfARmFUc9weaIcbTUdIKZef3/c\naNG4vjkhouET/1rXAx6IxYN4qtNo6vRcv5ogw3MMOXJRt3ck13VytQz1xwsMdb2JaDHxr3VbwLVY\nPFg/L1aoBhKKSNMsVDI9d7Vn5CTkuT6ufgzOmokog9e5S0B9IL81aCI3+ndyU2zoLQ0ZPp4gRxf6\n7zdX22/GcN2QuOzSJqJhE/86bxNQ4izxYqTPJHG7uPo7mLPnuC6umj5rvyV6IiKKSfy41iXgP2Dx\nYL7SZ6gO9DwHCsgg/sFXmLLnuB6umoozEQ2X+Nd4n4DPxuIB/AQfoXpIXVBMRe2RBDm6qivOUo7S\n98X0PKn+fmmSREQUw6ALc2U1Fg/i3/XQbldbGPLElvus01acT0wZyrOlmK7XOZPr1FPDEtEwiX99\n+w4oqSB9QcvxaOT+PwxZ26Ot3TD8Xdvfg33WvC5JIiJv+EMaFuLHsFABJQ3meo4dEvefenu0lXv+\nJvo6vRLDW0caJf70pEWn13aq80L7NDO5qL9yVA12+yXKUnkIcQfdGZTfEdYVyOOx1rdfZSiFS123\nAsBfpwpCRATEG1yPw+IZ1z9G6lt1kCFHTEOYferZd0kbxxt1ne5Eno8NjV4xW86UizmgKJT/z6ZM\nJYz413XsgPoJLKQcnPVPEfveytB/jkUg5+w2uT8mRAruyrYY7a5smwLmXaOxB0E9w/Mn/S+P0Pd/\nw/7VowLlUe45SP0YhrDUcF3u60TZ42xXgiEXZlVdgd4sUYafIt5AbPvs9tGIGfqyffacS37dzwF8\nO3UIIs1sx/vNe8wwemMpzJWqQP2pcl31gwOviZjh5crfMYtL3ew5hwJXl/8zMYN4sqPhuhweByLN\nzHzqBNSNxAFHPclDis/69H7fmLDv6rI+YoaubNljfUTgGz9rpoR4EFdA4l/P0gNKKtCx3GHoO5fi\n8AjqC3Rucs9Pg8CDuDzjwV89pTxQbAYLH4tY/R6CfHdvbwvgMO26v1X+Lz2/bmjnByeiDOQ0SALm\nWViMX5K6ytBvDGca+q0uB0TK0JW+rXKdQQ/9N6pJPO6+9kz861h8QItXY/Eg/1CEfvU+XxyhT1O/\nuRQKU049/+8myNXWvchjexNRM/GvY/EBG6i/EBSzWKUojk8x9JtDgbZlzCV/JaesRGQn/jUsPmAL\nsYuV6QxeMdQV5+dGytCWmnGZcv02yKtA55KTiOzEv37FB+zAVLCW1d7Df38x5DZ7VvP9es1tuayD\n1IxEVE/8a7cAcHzqEIH8EIsH0pDfSdb72itgX7Y+pReOpnxDWAcikk3861YdYO5C/J9kjOFgxB3w\npc2ez46UwZX+PW0TfR0OjxPNiZ7t0rRxiKilLArz5ihPh6kONpsAbJ8wVyixCrTpzUBoPzD0KXVm\ntz/cskldB6m5iKiZ+NesHnA9gBuweODZLnKu0GIVrhQFMqfd2y65ngF56yB9uxKRnfjXa13AI7B4\n8Pkx4v3yUwymATbEZ8N6H6F/2vFcQ5/q5Q8D99+Ga3GTVgyl5SEiN+Jfq64B34zFg9AnMZxTFprO\n7PVogH44ezZrk0lfhycD5mqTQ9o2JSIz8a/TLgE/hMWD0S0+QyW0CuEH27MCt6/bVuvrR5H7d6Xm\nWdly+QLAmnDRGjPoeYhILvGv0T4Bl8FcxD7iIZcEoQu03va7PLff1J9++Wjg/l2oeeY63Cf2C67q\n8w+w8Jzavxc5BxG5G3RhVu0I82D/F57aT8m0Xj5/AYy7txfqkuctHe/X1wqtv9sj909E7Yl/fYYI\neBrMA/7JAfqKybROF3tqW/8Fo9BPHFNfkgr05ztmSfFGo+rnAO3v1NuQiMzEvzZDB7wb5sFy18D9\nhnQywhWAvwnUro3e10+0v+8P3H+d3dB9O+jr9Zdeky10FBZnZHEmkkv86zJmQNuu06URM/gWqkDr\nbb7WU7sm+sFg+sFiqZ/EfXLo6xHqWwRV+8sM16XefkS0kPjXZKqAtiKdq1gFOiTJu7f7ZNgM4dfj\nXZa2JWw7IlpI/OsxdcDVGFaRDrHbfnNDm6Gcp/VzHoA7I/Zfp2//30LY9bC1WV1/uef+iKgb8fVF\nUsBZDKdIq0fnVpfv9Gwz5nZxmT37PDK9S66tPLQRau+Gat5zP0TUj/jXotSA18I8gEo6laSLdfBf\nCPS2juzZno3ts+bUb5rUvt/rqZ0CwKn9ozXOmqW+3qhRMZs6AXkj/nUoPiDss+inpQzVQegCHYqp\nn+si9u+aSVpbbW4j8Yq51AnIG/GvQfEBNbYi7XIKRyl8FuiYPy/pMnsO/eMcqk8Z8nS1D/xtx7r7\nszhni4V5QDq9/mL+MEQRuT+fbBs3l/Ux5T8L3XbPmtoKsR30fmYarg9tBwAPeezXx3ZU2zDdt+l2\nEqOYRXnsCwBcBuD3J/+fB2bm4+chT8TXvSG8c18J+0w6B/qR0H2y622s9RHQoR+g/Pw/1fb33Wff\nx6PpPtXtb++UjhLgjHlAgo1NJ6A8q9Y9AC60LDML4DYAd6E8MtQkl+Ll6pnIu0iHKtAh6H3saLl+\nfaD+6/KEaLMAsMFTlhsbbidxWJgHJMjrbimAe1EOEstRfjVnP22ZtQC+CmDnyd/rLG0NeWAwHaBU\nALgjZShHPgr0uR7acGHrI8Ubo1D9dVkX11nzkF+DA8KjsgckyGvuGJTvuCsXTS6q16L8KbomYxkU\nbLPo01OGcmDKbHuT5drGf/oMODGn9XHp5PpXGPoPTe1re4/ttn2j47IcizNRfEFeb6cBeLfy9xkA\n3qEtcwWAP0F55OqtAF5paWuMA4KtSIf6PNaHx7E4710t29DvH+rgMJfZ89MD9G3L8bmAbbsWXtf2\niCi8Tq+1ZQ23uzS6HOXg9xyUX2G5CcDNKD+T1s0p/5+H/fPooVALkrotv2dZRoJVk3/3xPQxPADT\n/C559SOof97ivq5mUH7W/KDWl973lwP0reeo+joOft+ImI5ELwB8YdKXSV3/atZQb5iIxmwW06Pr\ngzkaC3dlX4zFB4BdiIUF91qUM20d36WXlsE+k5aqT9Yv9LivK1v7z4rQd+XvI/Sjr4v+xtq172q5\nq72mIyJdkLFgGYD7UB78tQLmg7/2BfDPKA8UW43yhwj2jxUwc9UMK5ci3Senfr//4z2d++7tFwXo\nGwC2sfTvUzXrbVrPOh92XI6I+gn2GjsRwNdQHp198eS68yeXyhtQHpl9J4DXxQ44EDcgnyJtyuh6\n/udUs+cYfZv6CeX9MK9P21mzxOcX0VCIf32JDyiIrUD/ft2dEjCd6MPlcV7V4T5t2drfO0Lfev8h\n2Z4rLM5E6Yl/bYkPKJRt0O3728s+LUH/whCiQOht71Bzm21Pj6/+Q+u6LVmcicIR/7oSHzADOezq\n9lGgnx84k+ttvvvePUD7df2xOBOlJf41JT5gRkwHAEkbWE3Zmn4RKuS6zGltv1m5bevAfavtft1z\n2yZPQfv1qZb784C5iMZG0phsJD5gpkw/xyipSJty/VPN8gcYlg+Zp+62ayL1G0Lb58R1DstQMDwN\n50CJfz2JDzgAfwW5RfoItMsVch12amg7VL/vDNSuSd1eFVvfkp4vI8Mfrhgo8a8l8QEHxjYgfyBl\nqIk+BXrnmmX75lCtaJGxjc0DtGlT9XGy9nd1OaTmPny9RsXCPFCdXkcxT8lXRO6PpmxPjsMAfCVm\nEI0pl+05oi/r87lU17Z+25cBHO65z1CvC/WUpXXrpPcfIxuVu69nJ39chunXIeeBmfn4eSgA8XWP\n78BlsM2kUz55THmeaVjuAsNyoTK0vb1vn6FU7es/nGL6PfHY2egXOGMeKPGvH/EBR0ja59E3OuYJ\nmbmu3brTYfroL4SzGtqv25bV338TKBv9AgvzQImve+IDjpjpbFgpi/Rmjln025d66l9vd8eG2/tu\np6bPfftyyaivz3dQfnUq9Zu1keBR2QMl/rUjPiABKH8dTFKR7lKgQ/Udsl+1ncd7tqW7G+4ZbY89\nX79E7Yl/3YgPSIvYBujbhGRRf/bws4bbffifWpum85X76jdkIWzbJoszUX/iXzPiA1It20D9bAE5\n3lZz+6ZA/eoedVjGxR94aMOkS5uXw7y9iciN+NeL+IDkzFakl9XdybPnWzLYMvqwq0O7+u1djnbX\nv0PtS9f2WJyJuhH/WhEfkDqRsMvT1n+ok4S4tOmj35C5+96/AHC1p1xEQyW+7okPSL1sj/RF2tZ3\nqExNbX7OQ7+hMnf1p0j3+BLlRvzrQ3xA8uYypC3Spn73NVx3VoC+XJZZ2aOPvny0lfoNGFEuxL8u\nxAekIGyDeIznwxcb+s9p97Z6v2d1zKi35aMNFmgiO/GvB/EBKTjbQH5G4H63quk71Gy0QPkrVror\nevTrK6+vdY7xhocoZ+JfB+IDUlS2wXzzRP2GLNAuy2zrse027fSlZ9HzvdBDH0S5El/3xAekZFLM\nuJqK854929dnxn/kmMPFZR3uU9dvX1U7n7K0z9c+jZX45774gJTclohfpCXMnl/dod+lLZevy9bX\n1Za29ke8x5FIIvHPefEBSZTXIG6RPqOmv759bnBsT19mf4e2+2QM9Qak7jYWaBqTTs/1mL/BW0Tu\nj4aj7skd4jkVqj+9XVtbrsuZlm+br7qvj+3YlMO0XTkm0JB1qntLAgQh8m1GueiqGdjvBejPpEB5\nRjEf7dpmjzMAnuuwnLq8umwXPmaxTTlM27XA4p/VJKJIuPuKfLPtdnY9wrlvPz7bdF3u+Y7LvrxD\nFl9ctxF3b9PQiX9eiw9IWYvxefQjAfroWqD7Lme7jy9Ve/MNy5k+2ycaCvHPZ/EBaRBMP1zhe8Df\n0XP7ejumE5MA5tOKurTXJoMv6u9Yt+m/x/YsZtvfhygo8XVPfEAanF9F+CIdskC7Lvc6wzIXtMxU\nLffjFnn3YPyrAAAgAElEQVRd22yzTfR1u7HFXeda9EMUg/i6Jz4gDZqtQPt4XurfKe7T7pVaG2+t\nWbapv5mWeULsSu66PfR1W+pwl7mWfRCFJr7u9Q14GIDjAWzhIQuNm61AXxeo3dM9tGWzncOyrsWx\nWuZNbcM6ttt2DFiCxnUrZsuCXMwBRaH8f7ZzWiJ/Bl+Yn4B94HscwDcn/38M5S5MIhe259ReAdps\n+xrYrcX99eU+WHO7zaqOOV302ZPwd3DaDpwxkziDL8yV5QCeDuA3ALwHwL+hfiC0XW5EeQpIoorv\nXd2+CnSb+9Ytq15f9z3tAsA+LTO6qNq+uef9LevHwkzijKYwu5gBcCa6FWz98uyIuUkG0y7UrkV6\np5q2QhXolTXLqtddYLjvMztmc/EWpe2+Z1BTL2dPrp7tlY7IPxbmDo6Dn+L9PfDUgkNV9xxpo+k5\ntLZjO22WfdCxjer6EGcG9LEnwtSWxPGFSPzzUnxAi+3hp3gXAPaLnJ38uhX9i3TTc+SOju20Xbbu\n/u9xbLcrn8X0Kej+WBCFJv75KD5gDzMAHoCf4v2VyNmpG9vj97me929TYPTld65ZVv/qVIHyaHFb\nf6GLnO9C+hjabz+i0MQ/D8UHjOBS+Jt9L4ucnexsj9GRDfc7pea+fQp0l6z6fWMUuBB9dNl+RKGI\nf/6JDyjEAfBXvM+LnJ3sj0XdMQiuj2edd2nLXtUxp6k4h1T1cXugdqvLvp7bJ3Ihvu6JD5gZX8Wb\nj0s4bbe3vtwqw3UFgBe2aKNPxhjPkYuUPkL9tjaf65SK+Oec+IAD9C34K96bR84+JIeiXZE2LfNX\nLe6/h+Nytv6qi3qq0ZBCF85j0W57EPki/rkmPuBI7Q9/xbtp9ykBn4Bbkb7Wcvsyh/tWfM2eh1Cc\n9T5YoCkG8c8x8QGplq/izefBlMs20q//94b7u/TTNVdoKd4EFABuCtwfjZf48U58QOqt7nu+bS/b\nRc6emm07vNhye9N9VzYs0+S7lnZDS9VXgfIzfSKfxNc98QEpig3wV7z/OW70aGzra9qN3XS/TzUs\n0zaLXvBDqPp6IEJfan+x3hDQeIh/PokPSKL4Kt65P++6rOOzG5bRr9+1Q46Q3qT0E+tUt7+NYT1v\nSAbxzyPxASk7H4W/4r1H5OxtbUDzOlxiuF+bAt0k5pueVAVyaG/sKC3xzx/xAWmQtoG/4v2NyNlt\nNqL9HgLbcu/TrntXTb+2/kIdD5CyOOrraPolLqIm4uue+IA0ar6Kt4QZXtsCbfoFLV/99ZV65srZ\nM/Uh/jkjPiBRgz+Gv+J9dIB8tr4uNiz7Zcecrn2ELF6pC+NmCL+ONEzinyviAw5JAVxTAPMFsKlw\n/61f6m81ZMy+be1tqS23eYcMLrkP7ZHdpGr3p57bbeNj8Pf40DiIf46IDzgkk6JcTC4bU+ehRWIV\nb58Ft0t2X96gtLnEY7tdxNhLQMMg/rkhPuCQTGbKRQHcwhlzti6E3wLuWkDbFGcAOEZbRv+dZ1+v\nfWmFUF/HY9LGIYGkPFetxAcckgJYWwAbWZQHz3bubB9Fus/ypvu/yMP6SivOAGfPZCf++SA+IKVT\n8DPx0FLMtm19+1wXKXYHCzQtJv55ID4gpVPwM/EU9EJypuE6X0X76Jrb+maXpOmNC42L+MdffMAK\nZ2/xFfxMPJUr0H9Xdp+L6atcriQXP309H0obhxKR+NxcQHzACmdv8RX8TDw1vZD8pOH2uss+LZdv\nuqywZH6dsszS7qselL4u69LGocjE1z3xASucvdGI6YVE/xGJVYZl2lxUp/Vsq65taXLKSv6If6zF\nB6xw9kbkVEguMCzXpYDqt7+/4fZci/cpkJWHwhP/GIsPSEQLPA63QtLloDGTPkXLZ/HWz47mm7Q3\nDBSO+MdWfEAiMtKLyBU1y5iWb1OcTacI7ZLT5JAW2ZouV7fI5ZK3AHCVhzZJFvF1T3xAIqpVVzC/\nYrm+qcC9ybGvezpk7Mrn7LtLf6lPOUr+iK974gMSUaOtYC8+tmLUp4i1LXTnKssuc1i+q8/Dfb2a\nLtuhLMZd9xSQXOIfR/EBiciZjyLret+2RUtKcdsZ7uvcdPlA5OzkR+rnYCPxAYmotba7cK+vuU9T\nG236kVKcXfgq3jms69iIf0zEBySiztoWiT4FZ4jF2cZn0a4uT426BuMm/rknPiARdfcx4Da0K857\nG5ZvuvxAub9LX0MozoDbum5tWK7r5fpA6zE24p934gMSUXfF5Ix58Dd7BoA9am5/lmNfQynO28DP\nrmufs2+qJ34biQ8oWcEf1iDhioVnzDMN4ts2N1E78LcpGOqpRF+lXL+84+pJ8u8IXyB/w9BH18uY\nd52Lr3viA0pW8Ic1KC9dZ1ltZ8CuM7ohzvL0dX0wcv+bGTJ0vfxN5OyxiH++iQ8oWcEf1qD8qIXw\nLVg8GLvct27ZtgV6iMUZWLyuu6SNs8j/hb8CnhvxmcUHlKzgD2tQfppmrk2Drb7c0Zbl3mppV7+s\ncOw3VzkXsefCX/E+MHL2OuIfB/EBicg7W5HQB1P95yXrlrUxnYhkaDOwJgdjuOvY5fG1Xa6LlFn8\n9hcfkIi8ayoQXWfPf9qi37EVZ2A862nyUfgr4HVvGF2I3+7iAxJREE2F4ZtwKyK3Oi5n6tvlMoQj\ntnX6Ov512jhiHAR/xXtDTT/i6574gEQUhGsh1Qc805G66wzL+S7QQxyr9PXbLG2cbPgo3KKJD0hE\nwbQZpFwGtq4D4OGW+2Y7sLY05HVL5ZXI+PkjPiARBdN2oFqN5gGu70Bou/9XPbQt2dkY5npJ1Gnb\n9v1gu40icn9EJEs1SLUZB0wD20zD7T76qNqoG1hzH8+ati3116nuLXFY5gQAdwO4B8CFNcsdAeBn\nAE5tG4KIRuWmFsvOYPHAVgA4WbndpM1M0NRH1UbT7QXKk6fkyLZtX50gC7WwFMC9KI86Ww7gdgD7\nWZb7FwA3AHixpS3uLiGivrtObbtg6z7j29mx7VMa2qnLUV22aLtCgnD3tn+dtmPTjPlIlIX5mwCe\nQPlTYKcYlvstAB8A8EiXEEQ0Omd2vN8Myq+6VKoiUre78H64DZAf0f7Wj1xWC9YMzDPOHyLfwjYD\nYKXyd67rkb2mwrwTyid15YHJdfoypwB45+RvPpBEZLN68u97e7RxF8y7YIFygqB6v7bM2xvaVtv9\nyeTvZxj6Use5qkDvYFkupzGxWue/Va7LbR2yt6zhdpcH40oAF2H6rrXuneuc8v/5yYWIxuNx5f87\no3yz31U11qjj1GkA9kS5pw8AXoKFB3H99uRSN06py6vjmml3NlCeg/sJAP+ltHstgHMMy6q5JTt9\nctE/LgDyyJ/K7OQS1NEAblT+vhiLDwD7OoBvTC4/BPAwpgdmqPiOi4gA4CiEmYU1fT68p+V6l/Zc\n+rm+Y64c6LkPql+cJoI8xssA3Ify4K8VsB/8VXkP7Edl5/QkJKKwXD4b7uI1aC6CPorzbjX91LVp\nW75pF7sUub6xSCXYNjoRwNdQ7hq6eHLd+ZOLjoWZiFxcjrCDe5cjq01eoNy+2nB7XXFuU/TVy7YN\n90ttV7BAuxK/bcQHJKKoYgzqpsJ3VM3tTW206adN4cpxV/fDyCtvCuK3ifiARBTV44g3oPedPbsU\nny0t/aiXQxtymk5FKr3o6Tn/Pm0cUSQ/bgAyCEhE0cUsOjeivtjpt+lfDe26i/pxw3Uu63y+5X5S\nx1I945Zp44gg9bH6hVYBC+CaApgvgE0FsDZUKCJKKkWx0QvIxobbbbe17WeN4TrX9e6zmzw26fli\nEr/+bQvzfAEUk4v+wgmCbwaIkkgxgK9AfQF5v3bbO5Tb2hSdpZZ++hRYW4H+UIs2QjsOLNBABuvd\ntjBvmhTlW2IVyRRvBogo6eDdVCBNtz1L+dv13Ni2fkzXt/1lLNNl9xZthJTDDD8k8evbtjCvLYCN\nMWeuKd4MEBGAtIN20y7ivWuWaZvZdl9Tu2/0vB4p6ZnOTRsnGinb3yqHgNHfDBARgPSFxFbUXuKw\nTJfMtvuf6qF90+5zKUW6z96BHEnY5rXEBySipFIXD9fCG6o4u/TTlnqCFClFeomgLKGJXzfxAYko\nKQkDdV1xbtr13DV3Uxu++nnE0laq7f2bQnKEJH6dxAckouQkDNBNxbmpQPvq1yWX/l3rrn1Vl7+t\nu1MgeoYnE2QIJfVzuZH4gESUnJTZ0yWGLHoB2WC5fs8e/boU+n8zLHOvYbk+fRYA9u3Rpo8ch0Tu\nP4TUz+NG4gMSkQgSCjOw+LPQiuvsuQ+9rWWGZbYzLOe73+oS8yCtIe3eFp9ffEAiEkHaoGzKYzp6\nWl+2ADDnse+67eG7QM9Y2oz1mJjOO54j8bnFByQiMaQNxrYCYStePovKx1u0F6KQ7mNpN8bjc0WC\nPn0Sn1d8QCISQ+JAXJcpRoHW26o734Kp71/r2T8AXGVpO/TjpPd1deD+fJH0/DUSH5CIRJFWmIGF\nxeFlDbeHKmBt2ntNoAymHNXlOk/tu/S5TcC+fJD2/F1EfEAiEqUafO9OHUTTVORMu1+3drhfG29r\n2Z7tM2NfbEX66R77qOtPKsnZAGQQkIjEkTrwuhQGfZl3Od6vT46ndriP7+1rK9JLPPezwdCHNBIz\nLSA+IBGJUw24v5s6iMGvw60omIpHyt3btvsUKH8z2qdQu/NVnwjcfh+SshiJD0hE4qi7YCXSdxHb\n2AqUetmuZ5bTDW26+Jbhfh/smUW33tBH6N3pJ3tsuyupz9tfEB+QiESqBtoDUwep4as4h5g9H+14\nv90C5dFdYOnHV196m0s9tds1i2jiAxKRSNtD9qy50qc4b9L+vshznrbbLkaBtvVTALghQNspSH/O\nyg9IRGJVg+tmqYM0UAuB6TSWRyi3r4O9MPkqJkf2bDNWgbb1VQA4tkebhxnai0l83RMfkIjEOg15\nzJqBhUXgNxpuB+znvA65e9v0Hew29y9QvskIxbYdlnds71atHd+foduIf76KD0hEouVSmIHmwmq6\nLXRxdsnV5BxDG094ymbjc3vobezhI2BDf6KJD0hEon0Ewy/Opvv5LtC7eWhT/+WtGI/LFpY+u/Qb\nc/e8aOIDEpF4ORVmAHghuhVn/bYYs+ff9tROjMfnVyz9tul7aY/7uhL/XBUfkIjEizn4+9S1OB8D\newGSsnvb1k6B7p8Jt3GTpe+/crz/rOG+voh/nooPSERZyLEwA/aB/2nK9esd7qtftvWQzbSbuKsv\nGdo6r29AR7Zt9EsO971Zu89vesojmviARJSFXGfNgD276zrFnj1f2aOtDYb2Yj5mtu20quX9tu6Z\nQTTxAYkoG7kWZmDhoL/Ucn2bNnzP8kzt+24v9mPX5c2Mj7zin6PiAxJRNnKeNQP2YtpmvUKf2jLE\nwVGpC/RySwZbji0dl7MR//wUH5CIspJzYQbsA37bIhB79/a/BGizQLnrO6anW3KYtturtNv/w7EP\n8c9P8QGJKCu5z5oBf8XZ1JbvbROiXf2nMwsAH/fUdhtzhhwFgKu15fTbT2poV/xzU3xAIspO7oUZ\nWPyVKADYW/l7+xZt6T9DmUuBtu1iTsH0M5gFgOOUZfTbbF8LE//cFB+QiLKTehD3SV+XPuumf/XH\n9zZ6XGv3+x7bllKgbVkKACstt5vuL5r4gESUpdSDt08+i7N+/1xmz7a2C5SnAk3Fti23N1yn3kc0\n8QGJUimAawpgvgA2FcDa1HkyI2Fm5ZPv4qy3UV3W9WivqX2fvmpo/zTPfbRh+7jgR9rf/xsZPCfF\nByRKZVKUi8llY+o8GRpSYQYWDvAr4KfobQVzQfHlcwHbBoADDe2nfszXwpxJSr5G4gMSpTKZKRcF\ncAtnzJ1kMxC2oK7Tm+FvHU3F41U926xrP8TuZ4kFUD21qqRctcQHJEqlANYWwEYW5V6yGAhbss3C\ndgrUtk8xCpTUQvgayMpjJT4gEWVN2uDsy4MIW4D0dr/jse3LDe2HYNo+Pn7coy/xz0XxAYkoe0Ms\nzACwP8IW50MCtg1D2yvrF+/sQkNffxSoLxfin4viAxJR9qrB+O9TBwkk9u7nHHdvA8BqQ18papD4\nuic+IBENwlBnzZUYBUdvf3OPbZ9haD+klAVa/PNQfEAiGoRq8H1R6iABxSo2MWfPoT8TNm2zmQh9\niiY+IBENgrobc8j0IrNroH5+YOjLp9iz2XlDn4cH6kv8c1B8QCIajGrA3S51kMBS7hL+sMe2n2to\nP7SDDX1+zHMf4uue+IBENBhHYByzZiBuQTvP0J/P3cF627t5bLtNv762o/jnn/iARDQosT5HlOAz\niDvbHMrR20399u1bfN0TH5CIBuWtGM+sGQB2Qdxith5hC/QeAdtuov+sZYFu370W/9wTH5CIBmdM\nhbkSu5CZivO5Ads/0GPbTV5o6P8FLe4v/rknPiARDc4TGF9x3hnD2A3c1H5Mqwz93+pwP/HPO/EB\npSn4G71EPoytMAOyPqP12f82Adt21Wb9xD/vxAeUpuBv9BL5kHIQT0kvHlsn7Nv39tfbPd5j210z\nmNZP/HNOfEBpCv5GL5EvYyzMwOLCcW3Evv/D0H+B8vvDvqSePQPAVYYc6ye3iX/OiQ8oTcHf6CXy\nJfXgnVLoz3679O8zg+nz3xRMR5GLf76JD0hEg5bFQBlI6oLxKksGnzn+n9buyR7bbmMGLMxERE6y\nGSwD2AHpizMsGQoAPw/YR0qp+28kPiARDZ6EwTqV0DNWV9vVZNnSUx9LDG2nIP65Jj4gEQ1e6oE6\nNSnFuS6Lzzwf0do932PbLjqtS8xzyBaR+yMiMqkGy7GOR7ZikWp71BUvX5n0PmKta6e6tyRAECKi\nHIx11mwrFOrXfGKqK1wFgPd56kOtd2Pea7IANwIRScGB2b4b+QaBmXzucf1jrd3/4aldE/HPMfEB\niWg0Un++KsG2mG6D/bC4EKYyb8gSIleM9RX//BIfkIhGJXUBkkAvTO8wXJdKXXHOpUCn3oaNxAck\nolGRUoBS07fDGsN1qZyG+uL8Kk/9nK+1e6WndnlUNhFRS2M/QruiFpCZmutSaSpwoY7eXmK4rm17\nPCqbiKiDb6UOkJhaPExvVlLPnGcAPEW77hrl/75m9zNYuN4/99SuWINeOSLKloRdtlKYdmFL2a1d\n0Xdn32S4zofnaW1eU7+4kZRtZiU+IJFvBXBNUf6u9qaCvxImVTXwvjF1EAG2wXR7HKRcrxao3RLk\nMmk6OOzAQP0sbXlf0cQHJPJtUpSLyWVj6jxkpJ5XmewzT/X6T8cOZWGaKUs6elv8c0p8QCLfJjPl\nogBu4YxZtGqwPSB1ECFcirOkMV3Ptb3hOh8O1NpsOjmJpG1kJD4gkW8FsLYANrIoi7cb5BWb1GxF\n7ZKa21L6cyzMtRqLi/PDnvr6utbuZpblJG0fI/EBiWjUqkF2ReoggtgK8Mqa21Jz2b29JmBf+u2i\niQ9IRKN2JmQWmtSaCo/EbaYfUX3E5PoQu7d31NpUd29L2y6LiA9IRKMnscikthWm2+UQw+1SizNg\nLsRHWK7v67Nam9WudNHEBySi0fs05BaZlJqKmOTivBbmGa1enH39ypTUA+SMxAckIkImA2oCbYrz\nvrFCtWArmPr1Ps6IqZ5vXDTxAYmIkNFsJ4E2xfkrsUK1pGb8V8v1vh77oM+hEwDcDeAeABcabj8d\nwB0oV/LzAA42LMMnORHlgoXZrk1xlroNn4Q54yvgN3+w9V8K4F4AGwAsB3A7yh/VVh2D8gABoCzi\nNxvakfoAERHppBeW1Jq2zzkOy0jgunv71B7tB3EMgBuVvy+aXGy2BvCA4XrJDw4RkU56UUlpS0y3\nz2GWZZYhj+KsnzBlpXJb39lzp/V2+ZB7JwD3K38/MLnO5hwAm7qEISISSHJRSeWHyv9vtSzzM8j6\n6Uibt2JhzscxzTqD8g1GJcqbDJfC3CbEswCcDfPn0DRQBX9BiYap9Q/cj4xr0c2hOANlzqOUvwuU\nH9s+ObntCu22YOuyrHkRPAhgF+XvXWDeVX0wgHej/Iz5e5a25pT/z08ulL+9ARw/+f+fAXhpwixE\nIRRgoTaZwbRA1W0j1+VSuwULs/7b5N8ZAL8zueifRR8E4K7J37OTS3DLANyH8uCvFTAf/LUrygPE\njq5pR/I7Jeqh4C8o0bBJ/4xUAtfPYdXlDmpYNjX1c/QCwGu1210+fw76vDkRwNdQFt+LJ9edP7kA\nwLUAvgvgtsnlltgBKZ2Cv6BEw5bDAUwSdCnOXw4dyoO6AqyfVUy/XfxzRnxAIiILFuZmW2C6nY5p\nWDaH7zrr1Lyf0277pHb7vHIf0cQHJCKyyK2IpNJmO53UcnkJ7kN9Zv0NR6f1ivkBvOQP/ImImqhf\noSE7tRg1basZAD9vsbwUesHVc7fZBov4OFk3EdGY5DK7S6XN16P0CVsu23YGwKXK3/qJSWaw8NtM\nrRuPhTNmIsodZ83u2s4ae80yE6qbPXeqe5wxUxQ8CQkNTC4zu5TazoRznDkDZe4jlb+rE5NkIacN\nTZ5NinIxuWxMnYeoh5wOVkptFabb6xmO91EPnDo8UK5QvBz8FZP4gNROm1kwT0JCA1INuH+eOkgm\nuhQp9T53NSwrjX5iEtHEB6R22syCeRISGpgsBl1BuhSqb3a8nxRZ5BYfkNrhLJhGrBp0n586SEa6\nFNljOt5PCvGZxQekdjgLphHbHPkWi5S6Ftlci7P4vOIDEhG1UBWKbVIHycyYirP4rOIDEhG1oO5m\nJXebYbrdjmt539yKs/ic4gMSEbVUFYmcToghwdfRvcCqxVn6dhdf98QHJCJq6W3IawYnSZ/Zr3rf\nWY+ZfBP/vBAfkIioAxbm7nwV5+/4DOWR+OeF+IBERB38N1ic++hTnL/Y8/6hScy0gPiAREQdSS0M\nuehTXA/uef+QpOVZRHxAiq/gj1vQMEgtDLlYju5HalckPgaSshiJD0jxtTmtJ5Fw0opCbr6E/ttQ\nWnGWksNKfECKj6f1pAGRVhRy5GMbSnocJGSoJT4gxcfTetLASCkIOfNdnJf4CNUjh2jiA9Iw8HNr\nSkjSbC1nvovzC3yE6phBNPEBaRj4uTUlxsLsh+/i/AMfoTr0L5r4gDQM/NyaEuOs2Y9lmG7H43u0\ncz3SPSbinwPiA9Iw8HNrEoCF2Y9Pw8+23B1pinOnvmKeALyI3B8RUSrqgMxxrx+f2zL249Kp7qU8\nWo2IaKhYjP1Rt2Xf2a7PtoJhYSYiCktsAcjIqIozCzMRURicNfsVsjgv69meVyzMREThiZyZZUgt\noLM921KL8xMATujZXpb4xCSiMeIR2n59CH63qXq09rc8tam2LZr4gEREAVSD/k2pgwyI768+/X6A\nNuG5rSDEByQiCoSzZv98F9L1AdoU/5iLD0hEFEg12J+TOsjAhJrl+mpTfN0TH5CIKJCl4Kw5FMnF\nudP9eVQ2EVF4Tyr/3zNZimFaqvz/eZ7aFP9dZ18GvXJERA12A2fNoWxEmG2rzpxXdLy/aOIDEhEF\nVg3yok5oMRChfqRCbbftjFx83RMfkIgosF8DZ80hhSrOP1Xava1lHtHEByQiioCFOaxQxfmsDm2L\nf5zFByQiiuAjYHEOLVRxXtOybfGPsfiARESRsDCHtQTdPxd24VqcxT/G4gMSEUUSakZHU3+GsNvY\n5THs1HfMnyUrIvdHI1QA1wDYG8BjAF4xA3w/cSQim2rQ5rgYjloYQ2znpvbF1z2+M6TgCmC+AIrJ\nZWPqPEQ1OGuOI/R2VttfabhNNPEBKX8FsGlSlG8pgLWp8xA1YGGOI2ZxPlG7XjTxASl/BbC2ADay\nKFMmOGuOJ/S2vl1p/0tKn6KJD0gUSwFcM9ntvolvIkaPhTmOGUy39UmB+vhlLHwDIP5xFR+QKBZ+\nFk6KbAbxAXgHwm/rVcjoMRUfkCgWfhZOmiwG8YGIVTSzeEzFBySKhZ+FkyabGdZAxCzOookPSESU\nEAtzXDGKc6e2l/hOQUREvbA4x6Ge+EPUNmdhJiKSQfQZogZKrYG/lCxFQqLekRARCcTPmuO7COG2\nufjHUXxAIiIBWJjjC/WGiJ8xExENyE9SBxgRsZ83hzaqlSUi6oGz5jR8z5w5YyYiGpj/lTrAyIxu\n5jyKlSQi8oSz5jTUc2of1bMt8Y+f+IBERIJUxeG41EFG6OXw88ZIfN0TH5CISJAtwVlzSj4+bxb/\n2IkPSEQkTFUY1qQOMlJ9i7P4uic+IBGRMMeCs+bU+hRn8Y+b+IBERAKxMKfXtTjz61JERAP0tsm/\nLM7pqLVyUAfj8UlFRNQNZ83pvQTtHwfxj5n4gEREQj0OFmcJ2u7SFv94iQ9IRCQYC7MMbYozP2Mm\nIhoBFue0gp+2k4WZiCgPM82LUCTqY3FrshQe8F0eEVE/oX43mLqpHosDa24XTXxAIqIMsDDL0XQC\nGPGPk/iAREQZ4KxZlm/D/niIf4zEByQiygQLsyy2N0s8KpuIaGRYnGXweqQ2CzMRUX54hLY86mNy\ne7IULfGdHRGRP/ysWabqMTkYGTw24gMSEWWGhVmeo5DRmybxAYmIMpNNARiZ/0Qmj4v4gEREGcqi\nAIxQ58fF5eCvEwDcDeAeABdalrlqcvsdAJ7WJQh5MZs6wEjMpg4wArOpA2SobRGYDRGCfiHYAXpL\nAdwLYAOA5SiPNNtPW+YkAJsm/z8KwM2WtviOLry51AFGYi51gBGYSx0gM11mZ3MBctBCQWbMR6Is\nzN8E8ASA6wGcoi1zMoD3Tv7/RQBrAazvEoaIiHrZmDoA9ddUmHcCcL/y9wOT65qW2bl/NCIiclTt\nNv21pCnIi2UNt7tOw/V96ab73deiPerustQBRoLbOTxu427ajLPcxmHd1+VOTYX5QQC7KH/vgnJG\nXLfMzpPrdHu2TkdEREQLLENZ8TcAWIHmg7+Ohv3gLyIiIvLgRABfQ3kQ2MWT686fXCp/Mrn9DgBP\nj0cdb1oAAAKFSURBVJqOiIiIiIiIKBc8IUl4Tdv4dJTb9l8BfB7lydSpHZfnMQAcAeBnAE6NEWpg\nXLbxLIDbANwFYD5KquFp2s7rANyI8qPKuwCcFS3ZMPwFgIcB3FmzTNKa5/OEJGTmso2PAbDV5P8n\ngNu4LZdtXC33LwBuAPDiWOEGwmUbrwXwVUy/frkuVrgBcdnOcwD+cPL/dQC+i+YDg2nqmSiLra0w\nt655vn+PmSckCc9lG98E4L8n//8i+L3ytly2MQD8FoAPAHgkWrLhcNnGrwDwQUy/CfKdWOEGxGU7\nfxvAmsn/16AszD+LlG8IPgvgezW3t655vgszT0gSnss2Vp2D6bs1cuP6PD4FwDsnf/M7+u24bOO9\nAGwD4FMAbgXwyjjRBsVlO78bwAEAHkK5q/X1caKNRuua53t3hc8TkpBZm231LABnAzguUJahctnG\nVwK4aLLsDAKesH6gXLbxcpTf8ngOgNUo9wTdjPKzOnLjsp0vQbmLexbAHgA+AeAQAD8MF2t0WtU8\n34XZ5wlJyMxlGwPlAV/vRvkZc91uFlrMZRsfhnK3IFB+Lnciyl2FHw2ebhhctvH9KHdfPz65fAZl\nwWBhdueynY8FcPnk//cB+AaAfVDupaD+ktc8npAkPJdtvCvKz5WOjppsOFy2seo94FHZbbls430B\n/DPKA5hWozy4Zv94EQfBZTu/HdNTc65HWbi3iZRvKDbA7eCvZDWPJyQJr2kbX4vyAI7bJpdbYgcc\nAJfncYWFuRuXbfwGlEdm3wngdVHTDUfTdl4H4B9Qjsd3ojzojty9D+Xn8z9FuZfnbLDmERERERER\nERERERERERERERERERERERERERERERGRMP8fsf2WVjez+kAAAAAASUVORK5CYII=\n", "text/plain": [ "<matplotlib.figure.Figure at 0x108e9f4d0>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize = (8, 8))\n", "ax = fig.add_subplot(111)\n", "\n", "#Plot data and any hyper-plane that separates the data\n", "plt.plot(X1['x1'], X1['x2'],'r.')\n", "plt.plot(X2['x1'], X2['x2'],'b+')\n", "\n", "#We'll use a brute force search for this, but starting from an intelligent starting point\n", "success = 0; ws = np.array([1, 1]); t = 1\n", "while (success < 10):\n", " #Randomly perturbate the starting point, check to see if it seprates the points\n", " w_i = ws + np.random.normal(0, 1, 2); t_i = t + np.random.normal(0, 1, 1)\n", " if not ((sum((X1.dot(w_i) - t_i)>0)) or (sum((X2.dot(w_i) - t_i)<0))):\n", " bd.plot_dec_line(X1['x1'].min(), X2['x1'].max(), w_i[0], w_i[1],-1*t_i, 'k-', 't')\n", " success += 1\n", "\n", "ax.set_xlim(0, 1)\n", "ax.set_ylim(0, 1)\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##Getting the Maximum Margin\n", "<p>With so many hyperplane options, we need a principled way to choose an optimal separating line. Intuitively, we want a line that is simultaneously as far away from the positive instances as it is from the negative instances. In a sense, we want it to be in the middle of the empty region separating the classes. We can formalize this by searching for a hyperplane that maximizes the margin between the positive and negative classes.<br><br>\n", "\n", "<b>Some Definitions</b>\n", "<ul>\n", " <li>$W=<W^1,...W^k>$ is a weight vector, $X=<X^1,...X^k>$ is a vector of features </li>\n", " <li>The separating hyperplane is defined by the line: $W \\cdot X+t=0$</li>\n", " <li>The vector $W$ is called the \"normal vector\" to the line, and is perpendicular to it.\n", " <li>The distance between any point $X_i$ and the line is given by: $\\frac{W \\cdot X_i+t}{\\|W\\|_2}$</li>\n", "</ul>\n", "<br>\n", "<i>(How did we get that formula for the line, you ask??</i><br><br>\n", "Let $Q$ be a point on the line $W \\cdot X+t=0$, and let $W$ originate from $Q$. The distance from $X_i$ to the line is equal to the projection of the vector $\\vec{QX_i}$ onto $W$. The distance $d$ is given then by the projection formula:<br>\n", "<center>$d = \\frac{W \\cdot \\vec{QX_i}}{\\|W\\|_2} = \\frac{W \\cdot X_i - W \\cdot Q}{\\|W\\|_2} = \\frac{W \\cdot X_i+t}{\\|W\\|_2}$ </center><br>\n", "\n", "The last bit comes from the fact that $W \\cdot Q = t$ since $Q$ is on the line.<br><br>\n", "\n", "<b>Constraining the Problem</b><br><br>\n", "By adding contraints to the problem we make it easier to solve. Let $M^+$ and $M^-$ be the distances from the closest positive and negative point to the line. We are free to rescale $t$, $\\|W\\|_2$, $M^+$ and $M^-$ , so we do the following:<br><br>\n", "<center>$M^+ = M^- = m = 1$</center><br><br>\n", "With this constraint, the margin then becomes $\\frac{2m}{\\|W\\|_2}$. Maximizing the margin then is equivalent to minimizing $\\|W\\|_2$, or more conveniently, $\\frac{1}{2}\\|W\\|_2$.\n", "</p>\n", "\n", "##Defining the Optimization Problem\n", "\n", "<p>\n", "\n", "With perfectly separable data, we then define the following objective function.<br><br>\n", "\n", "<center>$W^*,t^* = \\underset{W,t} {\\mathrm{argmin}}\\frac{1}{2}\\|W\\|_2^2$ $\\,\\,\\,\\,\\,$ subject to $y_i(W\\cdot X_i + t) \\geq 1$</center>\n", "<br>\n", "<br>\n", "We can transform this using the method of Langrange multipliers. For each constraint we introduce a multipler $\\alpha_i$, which results in the following Lagrange function:<br><br>\n", "<center>$\\Lambda(w,t,\\alpha_i,...\\alpha_n)=\\frac{1}{2}\\|W\\|_2^2-\\sum\\limits_{i=1}^n\\alpha_i(y_i(W\\cdot X_i+t)-1)$</center>\n", "<br>\n", "<br>\n", "The optimal hyper-plane is a saddle point of $\\Lambda(w,t,\\alpha_i,...\\alpha_n)$. An interesting consequence of this formula is that there is a \"dual-form,\" where we eliminate $W$ and $t$ and formulate the problem completely in terms of the Lagrange multipliers $\\alpha_i$. The dual-form is:<br><br>\n", "\n", "<center>$\\underset{\\alpha_1,...,\\alpha_n} {\\mathrm{argmax}}\\sum\\limits_{i=1}^n \\alpha_i - \\frac{1}{2} \\sum\\limits_{i=1}^n \\sum\\limits_{j=1}^n \\alpha_i \\alpha_j y_i y_j X_i\\cdot X_j$\n", "</center>\n", "<center>subject to $\\alpha_i\\geq 0$ and $\\sum\\limits_{i=1}^n \\alpha_i y_i = 0$</center>\n", "<br>\n", "<br>\n", "The significance of the dual form is that searching for the optimal plane is equivalent to searching for the <i>support vectors</i>, which are the points that lie on the margin planes, and mathematically, are the points with non-zero Langrange multipliers. The decision boundary can be found using:<br><br>\n", "<center>$W=\\sum\\limits_{X \\in SV} \\alpha_i y_i X_i$</center>\n", "<br>\n", "<br>\n", "\n", "Now let's stop beating around the bush and actually build one! (note one important tip, with SVM's we generally encode a binary outcome as $[-1, 1]$ and not $[0, 1]$).\n", "\n", "</p>" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD7CAYAAAChScXIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtYVOW+B/DvIKLgHa8JGBqkoBAYiGnKCKGiAZKQwiy1\n1FNbzzbLau+Otr1s0+rJ01a3j26LkwVrcRe8oaSkA2YiuiFDEUV0VEAgNqLcB2bW+cOc1EC5zbxz\n+X2epyfEYdZ3WPibl9/7vmtJRFEUQQghxCCYsQ5ACCGk7ahoE0KIAaGiTQghBoSKNiGEGBAq2oQQ\nYkCoaBNCiAEx1/YBpFIp0tPTtX0YQggxKt7e3pDL5X/4vNZH2unp6RBF0eD/W7duHfMM9Prp9dPr\nN53X39pgl9ojhBBiQKhoE0KIAaGi3UZSqZR1BKbo9UtZR2CKXr+UdQQNiSiKWr32iEQigZYPQQgh\nRqe12kkjbUIIMSBUtAkhxIBQ0SaEEAOi10V79erVSElJQVNTE+sohBCiF/S2aIuiCDs7O2zevBk2\nNjZYsWIFzpw5Q5OahBCTZhCrR65du4bo6GhERUXh2WefxdGjR7soHSGE6KfWaqdBFO0HRFFEWVkZ\nhg0b1iXPRwgh+sooivaTJCQkoLm5GUFBQbCystL68QghRJuMfp22lZUVIiMjYWNjg0WLFuHYsWNQ\nqVSsYxFCSJcympH2A6WlpYiLiwPP8ygqKsLPP/+MoUOH6uz4hBDSFYy+PdKSwsJCPPfcc0yOTQgh\nnWH07ZGWtFawCwoK8NVXX6GyslLHiQghpHOMumi3RqlUIi0tDSNHjkRwcDD27t2LhoYG1rEIIeSp\nOtUeaWhogLe3NxobG6FUKhEUFIRPP/300QPo8VX+7t69i6SkJPA8j5ycHHz77bcIDAxkHYsQQrTX\n066rq4OVlRWam5vx8ssvY8uWLXj55ZefemB9U1xcDAsLCwwePJh1FEII0V5P+8GaaKVSCZVKBWtr\n684+JRM2NjYtFmxRFLFr1y4UFRUxSEUIIY/qdNFWq9Vwc3PD0KFDMW3aNDg7O3dFLr1RV1eH7Oxs\nuLq6wtfXF3v27MG9e/dYxyKEmKguW/J39+5dzJgxA5999tkjt+aRSCRYt26d5s9SqVSvbt3TVg0N\nDUhJSUFUVBROnDiBP//5z9i0aRPrWIQQIyGXyyGXyzV/3rBhg/bXaW/cuBGWlpb44IMPfj+AgfS0\n26OyshJFRUVwdXVlHYUQYqS00tOuqKhAVVUVAKC+vh7Hjh2Du7t7Z57SIFhbW7dasL///nvk5+fr\nOBEhxFR0qmjfvn0bPj4+cHNzg5eXFwICAuDr69tV2QxSTk4Opk2bBk9PT2zbtg1lZWWsIxFCjIhR\nb2NnRaVS4fjx4+B5Hvv378ekSZOwb98+WFhYsI5GCDEQJnntEX1QW1uLH3/8ETNmzGAdhRBiQKho\n66GrV6/izp078PDwgEQiYR2HEKJHTPKCUfruypUrCA8Px5gxY7Bx40Zcu3aNdSRCiJ6jos3QrFmz\ncOXKFURGRqK8vBwTJ07EpEmT8Msvv7CORgjRU9Qe0SNNTU04evQoJkyYQNdAIcTEUU/bwKlUKmRk\nZGDq1Kno1q0b6ziEEC2jnraBKykpwapVqzBixAh8+OGHOH/+PL0ZEmKCqGgbCDs7O+Tk5ODo0aPo\n3r07goKC4OLigtjYWNbRCCE6RO0RA6VWq3Hq1ClIJJJHrl9OCDEO1NM2MT///DOcnJzQo0cP1lEI\nIR1APW0Ts2HDBtjY2OBPf/oTTp48CbVazToSIaQLUNE2UsnJycjOzsbIkSOxbNkyjBo1CuvWraPf\neggxcNQeMQGiKOKXX37BuXPnsGTJEtZxCCFtQD1t0qri4mL07dsXffr0YR2FEPIb6mmTVsXFxcHO\nzg5hYWFISUlBU1MT60iEkFbQSJsAuH8Xovj4eAiCgIKCAsybNw8ff/wxhg4dyjoaISaJ2iOkzQoL\nCxEdHY2VK1eib9++rOMQYpKoaJMu0djYiLt372LIkCGsoxBi1KinTbrE+fPn8fzzz2P27NmIiYlB\nXV0d60iEmBQq2qRdJkyYgKKiIoSHhyMyMhI2NjZYtGgRsrOzWUcjxCRQe4R0SmlpKeLi4jBu3Dj4\n+vqyjkOI0aCeNmGisrIS1tbWrGMQYnCop010rrm5GW5ubpg6dSq++uorVFZWso5EiMGjok20xtzc\nHFevXsUHH3yAtLQ0jBw5EsHBwTh48CDraIQYLGqPEJ25e/cu9u7di/Lycnz00Ues4xCi16inTfRe\nXV0drKysWMcgRC9QT5vovWXLlsHNzQ1ffPEFioqKWMchRC9R0SZ6Y8+ePdi6dSsuX74MV1dX+Pr6\n4ptvvkFDQwPraIToDb0t2omJibTrzsSYmZlBKpUiIiICJSUlWL58OdLS0iCRSFhHI0Rv6G1Pu6am\nBvv37wfP88jMzERgYCA4joOPjw+6deumhaTEkDQ3N6Nbt25U0InRMriedu/evSGTyXDkyBFcunQJ\n48ePx+rVq2FnZ4dVq1YhOzubJjhNWGJiIhwcHLB27VpcvnyZdRxCdEZvR9qtyc/PhyAI4HkelpaW\n4DgO4eHhsLe377JjEP0niiKys7MhCAJiYmJgZ2cHmUyG8PBwDB48mHU8QjrN6Jb8iaKI06dPg+d5\nJCQkwMnJCRzHISQkhLZNm5jm5mYcP34cgiDgjTfewLRp01hHIqTTjK5oP0ypVCI1NRWCICA1NRU+\nPj7gOA6zZ89Gz549tXpsYhjUajXMzPS2G0jIHxh10X7Y3bt3kZSUBJ7nkZOTg9deew0cx2Hq1Kn0\nj9ZEVVZWwtXVFXPnzgXHcfDw8KAJTKL3TKZoP6yoqAixsbGIiopCZWUlZDIZOI7DuHHjmOQh7BQU\nFGjmQrp16waO4yCTyTBq1CjW0YiRkssBqbTjX2+SRfthubm5EAQBgiBg4MCBkMlkCAsLg62tLeto\nRIdEUcSZM2cgCAL69euHTz75hHUkomWdLZ4dtX79/f86yuSL9gNqtRoZGRngeR5JSUlwd3cHx3F4\n7bXX0K9fP9bxiB4QRZHaJ0aks8WT1XGpaLegoaEBKSkp4Hkex48fx4wZM8BxHGbOnAkLCwvW8Qgj\ngYGBGDRoEDiOg7e3N23mMnC6LNpy+f3/AGDDBmDduvsfS6XtH+1rpWjfunULCxcuRHl5OSQSCd56\n6y288847bTqwvqmsrERCQgIEQcClS5cQEhKCBQsW4KWXXqJRl4kpKSlBbGwseJ5HeXk5wsLCwHEc\nXF1d6WfBQHRl8ewovRxpl5aWorS0FG5ubqipqcGLL76Iffv2wcnJ6akH1mcKhQLR0dGIiopCY2Oj\nZgJz9OjRrKMRHcvLy4MgCJDL5Th58iStQDJA2hxpP6lfrq2i3amfwGHDhsHNzQ3A/W3nTk5OKCkp\n6cxT6gV7e3usXr0aeXl5SExMRG1tLaRSKTw9PbFt2zaUlZWxjkh0xNnZGZs2bcKpU6eoYJM/eDCa\nb4m2RvRd9lOoUCiQk5MDLy+vrnpK5iQSCcaPH48vv/wSRUVF2Lx5M/79739j9OjR8Pf3B8/zqKmp\nYR2TMLRz506EhIRg3759aGxsZB2HtIDFyhFtHrdLJiJramoglUrx8ccfY86cOY8eQCLBugcNJQBS\nqRRSVt/FLlJbW4sDBw5AEAT8+OOPePXVVyGTyeDn5wdzc3PW8YgO3blzB3v37gXP88jNzUVISAg4\njsPkyZNpZG6ktNUvl8vlkD80dN+wYYN2Vo80NTXh1Vdfhb+/P959990/HsAAe9rtUV5ejvj4ePA8\nD4VCgXnz5tGuOxN18+ZNREdHg+d57NmzB56enqwjkS72eA9bm/1yrUxEiqKIRYsWYeDAgfjHP/7R\nrgMbo4KCAs0/WjMzM9p1Z6Ie/LzTm7b+a+/Gm8eLNIui3anf306dOgWe53HixAm4u7vD3d0dqamp\nnXlKg+bo6Ih169bhypUriIyMRHl5OSZOnIjJkydj586dqKioYB2R6IBEImmxYBcWFmL69OmIjIxE\ndXU1g2TkcU+aSGwLFp1ek95cowtNTU04evQoeJ7HkSNHMHXqVHAch4CAAFhaWrKOR3Sovr4eBw8e\nBM/zyMjIgL+/PziOw/Tp09G9e3fW8bTi4ZEsq+3kT9KWkTKrNd+0I1IPVFdXIzk5GTzP4+zZs5gz\nZw44joNUKqVddyamoqJCMxfi5+eHDRs2sI6kFQ8XRVbbyR/XmSKsy9fQWu2kpQ461KdPHyxcuBAL\nFy5ESUkJ4uLi8Je//AWlpaUIDw+nXXcmZNCgQVi+fDmWL18OlUrFOo5JeDDSf7w468MbSXtQ0WZk\n+PDheO+99/Dee+9pdt0FBQWhd+/emluojRgxgnVMogOt/Za1cOFCTJgwAfPmzTOoW6g9PpJ98HF6\n+u+P0XZroaVWTFe0Z/SivSNqmQ4OYTRUKpWYnp4uvvXWW6K1tbXo7e0tfv311+KdO3dYRyM6plar\nxcOHD4symUzs16+fOHv2bDEmJkasra1lHa1d1q1r+WNdHvdJnztxQstBOqG12kkjbT1iZmaGqVOn\nYurUqdi+fTsOHz4Mnufx/vvvw8/PDxzHwd/fHz169GAdlWiZRCKBv78//P39UVNTg3379uHbb7/F\ntm3bcPr0adbxDMbjo/4HWmqTGAqaiDQAD3bdRUVF4eLFi5rbZtGuO9OjVCo7fNlgtVqNzMxMlJaW\n4plnnsHEiRO1Pn/y+OoRoO2Fsr3tjKdNMOrLRGhbtVo7WQ3xScfcuHFD/PTTT8WxY8eKzz77rLh6\n9Wrx4sWLrGMRxv71r3+JmzZtEq9fv97i3ycmJoqOjo7iuHHjxDlz5ojOzs7i6NGjxeTkZJ3mbE+L\npDPtlLa2R/RZa7WThmkGZsSIEfjoo4+Qm5uL/fv3Q6lUws/PT3Nhq9u3b7OOSBjw8PDArVu34OHh\ngalTp+Krr75CZWUlAEAQBLz77ruIiIjAL7/8guTkZFy4cAG7du3Cn//8Z8TFxTFOrxuG2AppCbVH\njIBKpUJ6ejp4nkdycjI8PT3BcRyCg4PRp08f1vGIDimVSqSmpoLneXz//ff4+eefMXHiRBw9ehQv\nvPACgEfbDufOnUNAQABu3Lihtbs1tWdddFdtZNH2ph5dbBSi9oiJqKurE+Pi4sSAgACxX79+4vz5\n88VDhw6JSqWSdTSiY9XV1WJcXJzo5+f3yOcfbxN4e3uLSUlJOsmkq/aINp5H28/5uNZqJ7VHjIyl\npSVef/11HDhwAFevXsXUqVOxefNm2NjYYMWKFThz5gz95mMievfujWvXrmluVPJAeXk5PvroI1y4\ncAEA4O7ujuvXr2slQ2ev7dHac2rjeQ0FLfkzYoMGDcKyZcuwbNkyFBYWIjo6GgsXLoRKpdJcgdDR\n0ZF1TKJFAwYMwMWLFx8pdLt2DcGkSb7YufMohgzZil69zmLBggVaOf7jbYT2tBRae2xbVqE8balf\nR2jjOTuE1RCfsKFWq8WsrCxx5cqV4tChQ0UvLy/xn//8p1heXs46GtGCsrIysX///uJ//vMfzece\n/GqvUqnE5ORk0cLCQuzfv7/I83yXH19brQkWbRZtP+fjWqudNNI2MRKJBJ6envD09MSWLVuQlpYG\nnufx8ccfY/LkyeA4DkFBQbCysmIdlXSBIUOGYNGiRZg/fz6SkpLQu3dvzd/V1tZi+/btWLFiBT75\n5BM0NTV1yTG1Ncr99ltAofh9O7xcDtjbA2+8YTwrQ9qCVo8QANDsuhMEAZmZmQgMDATHcfDx8aEr\nEBq45uZm/Pd//zeSk5OxYMECiKI3ADmioqIQGhqK7du3t3qbPLVajTVr1iAgIAAvvfRSuzfjaGND\ny8NXDWwLY1s9QhORBAA0F6o6cuQILl26hPHjx2P16tWwtbXFqlWrkJ2dTW++Bsrc3By7d+/GqVOn\n0Lt3b9TWpqBv377IzMzEzp07n3hfU6VSiT59+mDJkiVwcHDA3/72N1y+fLlTeR6eRNTFhKI2iivL\nkT2NtMkTXb58GYIggOd59OzZEzKZDDKZDPb29qyjER0SRRE5OTngeR4xMTGYNWsW/u///u+pX9fS\niLSz19hu73Z4Q0U3QSCdIooiTp8+DZ7nER8fDycnJ3Ach9DQUFhbW7OOR3RIpVKhtLQUNjY2Hfp6\nfbwxgj6iok26zOO77nx8fMBxHGbPno2ePXuyjkcYEgQBAwcOxCuvvPJI2+XxyUlv7/sfp6e3bdej\nPt6qTNtoRyTRiqqqKvGbb74RfXx8xAEDBohLly4V5XK5qFKpWEcjDHz33Xeil5eXOHToUHHlypVi\nVlaWqFarH3lMR66xbWgXe+oKrdVOmogkndKvXz+8+eab+OGHH/DLL7/g+eefxzvvvAN7e/tHdt0R\n07Bw4UJkZmbi5MmT6N+/P8LDwzF27Fg0NDSwjmY0qD1CtCI3NxeCIGh+XeY4DmFhYR3ugxLDJIoi\n8vLyMHbsWM3n2noxJ1Z3QdcX1NMmTKjVamRkZIDneSQlJcHd3R0cx+G1115Dv379WMcjDJ0/fx6X\nL19GQEAALC0tn/hYU5ywpHXahAkzMzNIpVJERESgpKQEy5Ytw4EDBzBixAjMmzcPBw8ehFKpZB2T\nMFBbW4uIiAgMHz4cixcvxvHjx+nO9G1AI23CRGVlJRISEsDzPPLz8xEaGgqO4zq0644YtpKSEsTG\nxoLneZSXlyMhIQEvvfTSI4+h1SMPfZ6KNmFNoVAgOjoaUVFRUCqVmg08o0ePZh2N6NjFixdhY2OD\n/v37s47CHBVtovfEx3bd2draguM4zJ8/H0OHDmUdjzDU1NQEnucRHBxsMgXd4Hraoihi4cKF2L17\nt+Zed8S4SSQSzb0ui4qKsHnzZmRnZ2PMmDHw9/eHIAiora1lHZMwcOfOHaSkpODZZ59FSEgI9u3b\nh8bGRtaxmNDbog0AoaGhOH78OEaOHIng4GDs3buX1nuaiG7dusHPzw/fffcdioqKsHDhQkRHR8PG\nxgYcxyE1NRXNzc2sYxIdGTJkCBITE6FQKDBz5kxs3boVNjY22LFjB+toOmcQ7ZF79+4hKSkJPM+j\nsbERJ0+e7KJ0xNCUl5cjPj4ePM9DoVBg/vz5kMlk8PDwoAlME3Pz5k3cvXsXLi4urKNohdH0tBsa\nGuj6FgQAUFBQoLkCYbdu3TS3UBs1ahTraISx77//Hq6urnjmmWdYR+kwg+tpt6a1gv2vf/0LW7Zs\nQXFxsY4TEVYcHR2xfv16FBQU4LvvvkN5eTkmTpyISZMmYefOnaioqGAdkTCSkpICZ2dnTJ8+HZGR\nkaiurmYdqcsYXNFujZubG/Lz8+Hi4gJfX1/s2bMHd+/eZR2L6IBEIsHEiRPxz3/+E8XFxVizZg1+\n/PFHODg4IDAwEPHx8aivr2cdk+jQ9u3bUVJSgqVLlyIxMRF2dnZ44403jGIlm8G1R56moaEBKSkp\n4HkeJ06cwJUrVzBkyBCdHZ/oj+rqaiQnJ4PneZw9exbBwcGQyWSQSqV0CzUTU1FRgaysLMyaNYt1\nlDYzmp52e9y7dw99+/ZlcmyiX0pKShATEwNBEFBWVobw8HBwHAdXV1eawDRxly5dQvfu3eHg4MA6\nyiOMpqfdHq0V7PPnz2Pt2rWdvtcdMRzDhw/H+++/j+zsbBw7dgwWFhYICgqCi4sLPvvsM9y8eZN1\nRMJIVlYWJk+ejIkTJ2LHjh349ddfWUd6IqMu2q3p27cvampqIJVK4enpiW3btqGsrIx1LKIjzs7O\n2LRpE65du4Zdu3ZBoVDA3d1dc2Grqqoq1hGJDi1atAjFxcVYv349Tp8+DUdHR7z66qu4ceMG62gt\nMur2yNM0NzfjxIkT4HkeBw4cwO7du/H666+zjkUYaGxsxJEjR8DzPI4dOwY/Pz9wHAd/f3/06NGD\ndTyiQzU1Ndi3bx+Cg4PRq1cvZjm01tNevHgxUlJSMGTIEOTm5rb5wPqmrq4OKpUKffr0YR2FMFZV\nVYXExETwPI/c3FyEhISA4zhMnjwZZmYm+csp+U1dXR3y8/Ph7u6u9bkQrfW033zzTaSmpnb2aZiz\nsrJqsWCr1WqsWbMGZ8+eNYg3H9J5/fv3x9KlSyGXy5GTk4NRo0Zh2bJlGDVqFNasWYNLly6xjkgY\nuXr1KubOnYuxY8di8+bNUCgUOs/Q6aI9ZcoUDBgwoCuy6KXGxkZYWFggPDwcY8aMwcaNG3Ht2jXW\nsYiOjBgxAn/961+Rm5uL/fv3Q6lU4pVXXsGLL76IL7/8Erdv32YdkeiQq6srrl27hoiICBQVFcHD\nwwNTpkxBWlqazjJ0SU9boVAgICDAoNsjTyOKIrKyshAVFYX4+HiEhIRg586drGMRBlQqFU6cOAFB\nELBv3z54enqC4zgEBwdTe83EKJVKpKamYtiwYZgwYUKXPrdW12mbQtF+WFNTE0pLS2FnZ8c6CmGs\nvr4eBw8eBM/zyMjIgL+/PziOw/Tp09G9e3fW8YgBY1q01z24jTIAqVQKqRHfN0gQBAwbNox23Zmg\niooKxMfHQxAEFBQUYN68eeA4DhMmTKANPOSp5HI55A9uPw9gw4YNNNLWhYiICOzatQulpaW0686E\nFRYWIjo6GjzPQ61WQyaTgeM4vdt1R/SX1kbaYWFhSE9Px3/+8x8MGTIEf//73/Hmm28+9cDGLi8v\nD4IgQBAE9O/fH1lZWbCwsGAdi+iYKIo4d+4ceJ5HbGwsRo4cCY7jMG/ePAwePJh1PKLHTPLaI/pA\nrVbjwoULcHV1ZR2FMNbc3Iy0tDTwPI9Dhw5h8uTJ4DgOQUFBsLKyYh2P6Bkq2nro559/hkKhoF13\nJujBrjtBEHD69GkEBQWB4zj4+PjQXAgBYKIXjNJ39+7dw7Zt22BjY4O3334bJ0+ehFqtZh2L6EDv\n3r3BcRyOHDmC/Px8jB8/HqtXr4atrS1WrVqF7OxsGuyQFtFIWw/cvHkTMTExiIqKQnV1NRITE+Hp\n6ck6FmEgPz9fcws1S0tLcByH8PBw2Nvbs45GdIzaIwZAFEWcP38eI0eORL9+/VjHIQyJooiffvoJ\ngiAgPj4eTk5O4DgOoaGhsLa2Zh2P6AAVbQOnVCoRFxeHOXPm0K47E/Ng150gCEhNTYWPjw84jsPs\n2bPpJtdGjHraBu7Bxg1bW1uEhYXh0KFDaGpqYh2L6ICFhQUCAwMRFxeHmzdvIiAgADt37sTw4cPx\nX//1X5DL5TQXYkJopG1gHt91t3HjRrz99tusYxEGioqKEBMTA57ncefOHc1mrnHjxrGORroAtUeM\nUGFhIerq6uDi4tKur6usrIQgCCgsLES/fv3w+uuvY+zYsVpKSXQhNzcXPM8jOjoaAwcOBMdxCAsL\ng42NDetopIOoPWKEnnvuuVYL9tGjR1u819327dsxatQonDlzBiNGjEB9fT38/PwQEhKC2tpabUcm\nWuLi4oLPP/8cN27cwNatW5Gfnw8XFxe88sor2LNnD+7du8c6IukiNNI2Un/6058QGxuLl19+GTKZ\nDEFBQUhISMAnn3yCY8eOPbKETKlUYvHixZoNH8Q4NDQ04NChQ+B5HidOnMDMmTPBcRxmzJhBl1Qw\nANQeMUEP77rLzMzUbKP28vL6w2Obmprg4OCApKQkvPjiiwzSEm2qrKxEQkICeJ5Hfn4+QkNDwXEc\nXnrpJbqYmZ6iom3iDhw4gJUrV+L69eu/f/Ktt4ArVwArKyA6Guu3bkVNTQ22bNnCLijROoVCgejo\naERFRaGxsVFzBcLRo0ezjkYeQj1tEyeRSODs7PzoJ69cAdLTgSNHUCuTwdbWFnfu3GETkOiMvb09\nVq9ejby8PCQkJKC2thZSqRSenp7Ytm0bysrKWEckT0BF20TY29vj4sWLUKlUv3/ytyvL/WpvD9fM\nTKxduxbl5eWorKxklJLokkQi0dzr8tatW9i0aROys7MxevRo+Pv7QxAEmpzWQ1S0TYSLiwuGDBny\n6ERjdDQQGorBOTn48cIFVFdXQxRFjBw5EsHBwSgqKmIXmOiUubk5pk+fju+++w7FxcVYsGABoqOj\nYWNjA47jkJqaiubmZtYxCainbVJOnjyJuXPnIiIiAgEBAZoJqKtXryIsLAy+vr747LPPcPfuXSQl\nJWH+/PmwtLRknJqwVF5ejri4OAiCAIVCobmFmoeHB01gahlNRBIAQHp6OlasWIHGxkaMHz8et2/f\nRl5eHj744AN8+OGHT/2HWFNTg+vXr7d7Qw8xfAUFBeB5HoIgoFu3buA4DjKZDKNGjWIdzShR0SYa\noigiKytLsyPS19e3zRceys7Oxpw5czBgwADNrjtbW1stJyb6RBRFnDlzBoIgIC4uDo6OjuA4Dq+/\n/joGDhzIOp7RoKJtbB5brof+/XV2aLVajZMnT4Lneezduxfu7u7429/+BqlUqrMMRD80NTXh6NGj\niIqKwpEjR+Dt7Q2O4xAQEECttU6ioq2vOlp8pdL7y/UAIDQUiI/XWsQ/eChzwzffIOXUKYwYMYJu\n3GDi7t27h+TkZAiCgLNnzyI4OBgymQxSqZRuodYBVLT1VUeL76xZwJEjgIcHcOyYTkfa7cmcn5+P\n0aNH06SViSkpKUFsbCx4nkdZWZnmCoSurq70s9BGtLlGXz24C7eHB/DVV23/ut+W63W6YL/11v0i\nPGsWUFXVtq9pY+bm5mbMmzcPDg4OWLt2LS5fvtzxnMSgDB8+XHOvy2PHjsHCwgJBQUFwdXXF559/\njlu3brGOaLhELdPBIQzbnTuiGBp6//8seHuLInD/v9DQtn1NOzKr1Wrx3//+t/jee++Jw4YNEz08\nPMSvv/66c5mJQVKpVGJGRob41ltvidbW1qJUKhUjIiLEO6x+9vVca7WT2iOmTodtFpVKhR9++AFF\nRUVYvHix1o5D9F9jYyMOHz4MQRBw7Ngx+Pn5geM4+Pv7o0ePHqzj6QXqaZOWVVXdb5F89ZVu++It\nuH37NgYPHgxzc3OmOYhu3blzB3v37gXP87hw4QJCQkLAcRwmTZoEMzPT7eBS0TYmHVlxwnCJYFut\nWLECCQlmQ4s1AAAOH0lEQVQJtOvOhN24cQPR0dHgeR61tbWaKxA6OTmxjqZzVLSNSUdWnLBcItgO\nBQUFEAQBPM+jW7dukMlkeP/999GrVy/W0YgOiaKI8+fPQxAEREdHY+jQoZrNXM888wzreDpBq0f0\nWXtXcHRkxUlHV6nomKOjI9avX4+CggJERkaipqaGepwmSCKRwM3NDV988QVu3ryJL774Arm5uXB2\ndsb06dMRGRmJ6upq1jGZoJF2V9LVRpmO9KH1qHfdFWpra2FmZka77kxMfX09Dh48CJ7nkZGRgVmz\nZoHjOPj5+aF79+6s43Upao/ogiFulDFQSUlJWLJkCYKDg8FxHLy9vWnXnYmpqKhAfHw8oqKiUFhY\nqJkLmTBhglHMhVDR1oWOFl9djYINYDKyPR7edVdeXo6wsDAsX74cI0eOZB2N6FhhYaFmAlOtVkMm\nk0Emk8HR0ZF1tA6joq0L+t6CMJDJyI7Iy8uDIAiYM2cOXQPFhImiiHPnzoHnecTGxmLkyJHgOA7z\n5s3D4MGDWcdrF9Mo2kY2kuxyJtyGqa2tpRUoJqa5uRlpaWngeR6HDh3Cyy+/DJlMhqCgIFg9mJjX\nY6ZRtI14JPkHHXmD0vffBLSksrISzz33HHx9fSGTyTBr1ixakWJiampqsG/fPvA8j8zMTAQFBYHj\nOPj4+OjtXIhpFG3WI0ldjvRN6Q2qC1RVVSEhIQGCICA3NxchISFYvHgxvLy8WEcjOlZaWorY2FgI\ngoDi4mLMnz8fHMfB3d1dryYwTaNosx5J6rKQsniDMpL2082bNxEdHQ1LS0usXLmSdRzCUH5+vmYz\nl6WlJTiOQ3h4OOzt7VlHM5GizZouCymLNygTGd03NTUZ3Zpf8mSiKOL06dOIiopCQkICnJ2dIZPJ\nEBoaCmtrayaZjLtod3QE2NUjR9YjfW1j3X7SEV9fX82Na4ODg9GnTx/WkYgOKZVKpKamgud5fP/9\n9/Dx8QHHcZg9e3ab76XaFVqtnR25zmt76OAQHbsmdGe+zlSxvva3jtTV1YlxcXFiQECA2LdvX3H+\n/PnioUOHRJVKxToa0bGqqirxm2++EadNmyYOGDBAXLp0qXjixAmd/Cy0Vjs7PdJOTU3Fu+++C5VK\nhaVLl+Kvf/1r294tusKDkfLFi0BFRftHgCYyciQdV1FRgYSEBMjlcsTGxurVRBXRraKiIsTExEAQ\nBFRWVmpuoTZu3DitHE8r7RGVSoXRo0cjLS0NNjY28PT0RExMzCOXUdRq0X64x2prC+Tmtq/wGns7\ng2idWq026Ws+m6rc3FwIggBBEGBtba25AqGtrW2XHUMrV/nLysqCg4MD7O3t0b17d8yfPx/79+/v\nzFO2z8NXrmtvwQbuPz4+vv1f15H7KhKjtGXLFkycOBE7duzAr7/+yjoO0REXFxd89tlnuHHjBrZv\n344rV67A1dUVvr6+2LNnD+7evau1Y3eqaBcXF8POzk7zZ1tbWxQXF3c6VJt11c1t2+vKlfsj/CNH\n7hdw0nEG/ga4atUqrF+/HpmZmXB0dMTs2bMRExODuro61tGIDpiZmcHb2xtff/01SkpKsHz5chw4\ncAAjRozA9u3btXLMTt3Xqa39vfXr12s+lkqlkEqlnTns7x6MlHXNQK5NbRAevAEC9wu4gS0jNDc3\nx8yZMzFz5kzNrrvIyEiMGzcOLi4urOMRHerZsyfmzp2LuXPnorKyst3X+5bL5ZDL5U99XKd62pmZ\nmVi/fj1SU1MBAJ9++inMzMwemYw0ynXa1AvvOiY2GSyKIk1mkjbRSk/bw8MDBQUFUCgUUCqViIuL\nQ2BgYGee8nf6/GtzR3vh5I9YtbgYyMvLw9ixY7F582YoFArWcYiB6lTRNjc3x44dOzBjxgw4Oztj\n3rx5XXcDTuobmwYTegN0cnJCREQEioqK4OHhgSlTpmD37t2orKxkHY0YEP3dEWlivzYT0/Jg150g\nCHjxxRfxl7/8hXUkomcMbxs79Y0JISbM8Io2ISZMFEW88sor8PT01OquO6K/tDIRSQjRDolEgq1b\ntwIA/P394ebmhi1btuh2HwTRSzTSJkTPqdVqZGRkgOd5KBQKpKWlsY5EdIDaI4QYAVrnbTqoPUKI\nEWitYG/atAnLly/HTz/9RIMkI0dFmxAjwHEcbGxssGTJEjg4OGDt2rW4fPky61hEC6g9QogREUUR\nOTk54HkesbGxOHfuHIYPH846FukA6mkTYmLoWt+GjXrahJiY1gr2mTNnwHEcUlNT0dzcrONUpLOo\naBNiYhwcHODl5YX169fD1tYWK1euxNmzZ+k3YgNB7RFCTFhBQYHmtln/8z//g8WLF7OORH5DPW1C\nSKtEUURzczO6d+/OOgr5DfW0CSGtkkgkLRbs5uZmLFiwAPHx8aivr2eQjDyOijYhpFVqtRp+fn6I\niIjA8OHD8eabb+KHH36ASqViHc1kUXuEENImJSUliI2NhSAIGDVqFBISElhHMmrU0yaEdJna2lr0\n6tWLdQyjRkWbEKJ1//jHP9CnTx+EhISgP928pFNoIpIQonVjxoxBamoqnn32WYSEhCA5ORmNjY2s\nYxkVGmkTQrrcnTt3sHfvXvA8j8uXL+P69evo2bMn61gGhdojhBAmKioqMGjQINYxDA61RwghTLRW\nsE+dOoUvv/wSt2/f1nEiw0ZFmxDCRJ8+fXDhwgU4Oztj+vTpiIyMRHV1NetYeo/aI4QQpurr63Hw\n4EHwPI+MjAzs378f3t7erGMxRz1tQojeq6iogJWVFaysrFhHYY562oQQvTdo0KAWC3Z9fT02btyI\ngoICBqn0CxVtQojeq6urQ0VFBaZMmYKJEydix44d+PXXX1nHYoLaI4QQg9Hc3Iy0tDTwPI9Dhw7h\nww8/xJo1a1jH0grqaRNCjEpNTQ0qKysxYsQI1lG0gnrahBCj0rt371YLdmRkJLKzs41ywEhFmxBi\ndIqKihASEoKxY8di8+bNUCgUrCN1GWqPEEKMkiiKOH36NHieR3x8PDw9PXH48GFIJBLW0dqEetqE\nEJOlVCpx4cIFjB8/nnWUNqOiTQghLThz5gzq6+sxdepUmJnpT8eYJiIJIaQFJSUlWLlyJezt7fHR\nRx/hwoULrCM9ERVtQohJCw4Oxvnz55GSkgJRFOHv7w83NzcUFhayjtYiao8QQshD1Go10tPTMWnS\nJPTo0YNZDuppE0JIJ1VVVSEjIwMzZ86EhYWFVo9FPW1CCOmk0tJSfPHFF7CxscHy5cvx008/6XxQ\n2uGinZCQgLFjx6Jbt27Izs7uykyEEKKXxowZg5MnT+Ls2bOwtbXF0qVL4eDggKSkJJ1l6HB7JD8/\nH2ZmZnj77bfxv//7v62uf6T2CCHEWImiiJycHFhaWsLJyalLn7u12mne0SccM2ZMpwIRQoihk0gk\nOt+wQz1tQggxIE8cafv5+aG0tPQPn9+8eTMCAgLafJD169drPpZKpZBKpW3+WkIIMQVyuRxyufyp\nj+v0kr9p06ZRT5sQQrqYVpf8UVEmhBDd6HDRTk5Ohp2dHTIzMzF79mz4+/t3ZS5CCCEtoB2RhBCi\nh2hHZCe1ZYLAmNHrl7OOwBS9fjnrCBpUtNtIn04aC/T65awjMEWvX846ggYVbUIIMSBUtAkhxIBo\nfSJSKpUiPT1dm4cghBCj4+3t3WJbRutFmxBCSNeh9gghhBgQKtqEEGJAqGg/JDU1FWPGjIGjoyM+\n//zzFh/zzjvvwNHRES+88AJycnJ0nFD7nvY9kMvl6NevH9zd3eHu7o5PPvmEQUrtWLx4MYYOHQoX\nF5dWH2PM5/9pr9+Yz/2tW7cwbdo0jB07FuPGjcP27dtbfJxenH+RiKIois3NzeJzzz0nXr9+XVQq\nleILL7wg5uXlPfKYlJQU0d/fXxRFUczMzBS9vLxYRNWatnwPTpw4IQYEBDBKqF0ZGRlidna2OG7c\nuBb/3tjP/9NevzGf+9u3b4s5OTmiKIpidXW1+Pzzz+vtv38aaf8mKysLDg4OsLe3R/fu3TF//nzs\n37//kcccOHAAixYtAgB4eXmhqqoKZWVlLOJqRVu+B4DxXiBsypQpGDBgQKt/b+zn/2mvHzDecz9s\n2DC4ubkBAHr37g0nJyeUlJQ88hh9Of9UtH9TXFwMOzs7zZ9tbW1RXFz81McUFRXpLKO2teV7IJFI\n8NNPP+GFF17ArFmzkJeXp+uYzBj7+X8aUzn3CoUCOTk58PLyeuTz+nL+O3y7MWMjkUja9LjHRxpt\n/TpD0JbXMn78eNy6dQtWVlY4cuQI5syZgytXruggnX4w5vP/NKZw7mtqahASEoJt27ahd+/ef/h7\nfTj/NNL+jY2NDW7duqX5861bt2Bra/vExxQVFcHGxkZnGbWtLd+DPn36wMrKCgDg7++PpqYmVFZW\n6jQnK8Z+/p/G2M99U1MT5s6dC47jMGfOnD/8vb6cfyrav/Hw8EBBQQEUCgWUSiXi4uIQGBj4yGMC\nAwMRGRkJAMjMzET//v0xdOhQFnG1oi3fg7KyMs1oIysrC6IowtramkVcnTP28/80xnzuRVHEkiVL\n4OzsjHfffbfFx+jL+af2yG/Mzc2xY8cOzJgxAyqVCkuWLIGTkxN2794NAHj77bcxa9YsHD58GA4O\nDujVqxf27NnDOHXXasv3IDExEbt27YK5uTmsrKwQGxvLOHXXCQsLQ3p6OioqKmBnZ4cNGzagqakJ\ngGmc/6e9fmM+96dOnQLP83B1dYW7uzuA+/fCvXnzJgD9Ov+0jZ0QQgwItUcIIcSAUNEmhBADQkWb\nEEIMCBVtQggxIFS0CSHEgFDRJoQQA0JFmxBCDAgVbUIIMSD/D6sl1dNxruxgAAAAAElFTkSuQmCC\n", "text/plain": [ "<matplotlib.figure.Figure at 0x1090e1c10>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from sklearn import svm\n", "import course_utils as bd\n", "import pandas as pd\n", "reload(bd)\n", "\n", "m = [[0.25,0.25],[1.75,1.75]]; s = 0.05; n=20\n", "X1=pd.DataFrame(np.random.multivariate_normal([m[0][0],m[0][1]],[[s,0],[0,s]],n), columns=['x1','x2'])\n", "X2=pd.DataFrame(np.random.multivariate_normal([m[1][0],m[1][1]],[[s,0],[0,s]],n), columns=['x1','x2'])\n", "X1['Y'] = -1 * np.ones(X1.shape[0]); X2['Y'] = 1 * np.ones(X2.shape[0])\n", "dat = X1.append(X2, ignore_index=True)\n", "\n", "\n", "my_svm = svm.SVC(kernel='linear')\n", "my_svm.fit(dat[['x1','x2']], dat['Y'])\n", "\n", "bd.plotSVM(dat[['x1','x2']], dat['Y'], my_svm)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "array([-1.00000005, 1.00000008])" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "my_svm.support_vectors_.dot(my_svm.coef_[0])+my_svm.intercept_" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<p>In the above we plot the optimal linear hyper-plane $W$ and also the hyper-planes that define the margin. Circled are the support vectors.<br><br>\n", "\n", "Once we have found $W$ and $t$, classification of a point $X$ is defined as:<br><br>\n", "<center>$f(X) = sign(W\\cdot X + t)$</center><br><br>\n", "which is just the sign of the distance between $X$ and the optimal hyper-plane.<br><br>\n", "\n", "In the above scenarios we identified an optimal separating hyper-plane for data that is cleanly separated by a decision boundary. This is seldom the case in real data, so we need to adapt our formulation to allow for such cases.\n", "</p>\n", "\n", "##The Soft Margin Classifier\n", "\n", "<p>Let $\\xi_i = [\\xi_i]_+ = [1- y_i(W\\cdot X_i+t)]_+$ be the distance between a wrongly classified point $X$ and the optimal margin plane $y_i(W\\cdot X_i+t)=1$, where $[1-z]_+ = max(0,1-z)$. This last function is also known as the <i>hinge function</i>. We can then redefine our optimization problem to allow for some misclassification of our training points. Specifically,:<br><br>\n", "\n", "<center>$W^*,t^* = \\underset{W,t} {\\mathrm{argmin}}\\frac{1}{2}\\|W\\|_2^2 + C\\sum\\limits_{i=1}^n\\xi_i$</center> \n", "<center>subject to $y_i(W\\cdot X_i + t) \\geq 1 - \\xi_i$</center><br><br>\n", "\n", "It is often useful to write this as:\n", "<center>$W^*,t^* = \\underset{W,t} {\\mathrm{argmin}}\\;\\lambda\\|W\\|_2^2 + \\sum\\limits_{i=1}^n [1- y_i(W\\cdot X_i+t)]_+$</center><br><br>\n", "where $\\lambda = \\frac{1}{2C}$. When we view the optimization problem in the above light, we can connect this back to ERM. The second term defines the training error, or the <i>hinge loss</i> of the problem. Our goal is to minimize this loss function while minimizing $W$, which in turn maximizes the margin.\n", "\n", "\n", "</p>\n", "\n", "##Example Soft Margin\n", "\n", "<p>When we presented the soft margin version of the SVM, we also introduced a new term $C$ which controls the amount of error we can tolerate (or conversely, the size of the margin). $C$ is thus a free parameter that we must choose to get the right fit. One way to think about it is as $C$ increases we increase the size of the 2nd term in our optimization function. This means we become <i>less</i> tolerant of training errors (and we thus decrease the size of the margin).\n", "\n", "\n", "</p>" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD8CAYAAACSCdTiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXdYVMfXx8+KxELvRUFQRJSmiBUFxBbUGI2RgEZRbGgS\nTUxUNEZQYywJsfdoxNglgrG/FlCwEDVSBLuCgoqFDgu7sN/3D36sLLCwu+yylPk8zz6y986cOff6\nvWfnzj13hgMAxGAwGIxGSTNlO8BgMBgMxcGCPIPBYDRiWJBnMBiMRgwL8gwGg9GIYUGewWAwGjEs\nyDMYDEYjhgX5Kti6dSsZGRmRpqYmZWZmKtsdBqNawsLCyMzMjDQ0NCguLk7Z7jDqGRIF+QMHDpCz\nszNpaGiQqakpDRs2jK5evSpVQ3fv3qWhQ4eSgYEBNWtWudmMjAwaPXo0qaurk4WFBR08eFBk/8WL\nF8nGxobU1NTIw8ODnj9/LrJ/wYIFpK+vT/r6+hQQECCVb+Xh8/n0/fff08WLFyknJ4d0dHRktiWO\nn376iezt7UlVVZWWLl1abdmgoCBSVVUlDQ0N0tDQIE1NTUpOTq6y7I0bN2jw4MGkp6dHhoaG5OXl\nRa9fv67Rn0ePHlHLli1pwoQJshyOCBs3biR7e3vi8/nCbevWrSMnJycSCAQy2RwwYAAZGhqSpqYm\nde7cmXbu3Flt+f/++49cXV1JQ0ODjI2NacOGDTK1WxV1cS1Iyw8//EBbtmyh3NxccnR0rLW9imza\ntImcnZ2pZcuWNHny5GrLSnts06dPJxsbG1JRUaGQkJAay1+4cIGcnJxIXV2dzMzM6OjRoxIfR1Wc\nOHGCTExMRDpzx48fp7Zt21Jubq7U9ng8Hk2ZMoUsLCxIU1OTunXrRmfPnhVbPiQkhJydnUlLS4vM\nzMxowYIFVFJSItOxiAU1EBwcDENDQ4SFhaGgoADFxcU4efIk5s+fX1NVER48eIDdu3fj+PHj4HA4\nlfZ7e3vD29sb+fn5iI6OhpaWFhITEwEAb9++hZaWFkJDQ1FUVIR58+ahd+/ewrrbtm1Dp06dkJaW\nhrS0NHTp0gXbtm2Tyr8ynj9/Dg6Hg+LiYpnqiyM9PV34d0hICM6cOYNPP/0US5curbZeUFAQJkyY\nIFEbZ86cQWhoKHJzc1FQUAA/Pz98/PHHNdYbPHgw+vfvL9JOUVERsrKyJGq3PAKBAP369UNgYCAA\n4MmTJ9DS0sKdO3ektlVGfHw8eDweACAmJgYtWrTA/fv3qyz79u1bGBoa4sCBA+DxeMjLy8O9e/dk\nbrs8dXUtSINAIEDz5s3x+PHjWtmpyOvXr4V/Hzt2DOHh4Zg5cyYmTZpUbT1pj23z5s24ePEinJ2d\nERISUm3ZxMREGBoa4uzZsygpKUFGRgaePHlSpc/SMG7cOPj6+gIAMjMzYWpqihMnTshkKz8/H0FB\nQUhJSQEAnDx5EhoaGkhOTq6y/NatWxEdHQ0+n4+0tDR0794dq1atkqltcVQb5LOysqCuro7Q0FC5\nNfjo0aNK//l5eXn46KOP8OjRI+G2iRMnIiAgAACwfft2uLi4CPfl5+ejVatWePDgAQCgT58+2Llz\np3D/7t27RX4EJOXBgwdQU1MDh8OBuro6Bg4cKLWN8rx69Qpr1qxBly5dMGvWrEr7v/zySwQFBVVr\nIzAwEF9++aVM7d++fRsaGhrVljl48CC8vLwQFBQk0k5qaio0NTUxfvx4XLhwASUlJRK3++DBA2hq\naiIhIQEeHh5YtGiRTP5XRUxMDPT09PDy5csq9y9cuBATJ06UW3tl1NW1IA2FhYVCvaqpqcHKyqpW\n/mRmZmLLli3o2bMnhg0bVmn/4sWLawzyZUh7bP369asxyPv4+GDJkiVi92toaODTTz9FeHi4sFMg\nCe/evYOhoSHOnTuHSZMmYdy4cRLXlQQHBwccO3ZMorK///47PvnkE7m2X+391PXr16mwsJBGjx4t\ntsyBAwdIR0enyo+uri6lpqbWeDfx8OFDat68OVlZWQm3OTo6UmJiIhERJSYmityGtm7dmqysrIT7\nk5KSRPY7ODgI91WFo6MjHTp0qNJ2a2trYb3s7Gy6cOGC0J64Y/z6669FbPD5fDp27Bh98sknZGNj\nQ3fv3qVNmzbR5s2bazwPVcHhcOjEiROkp6dHdnZ2tG3bNonrXrlyhezs7MTuz8nJocDAQFq7di2h\nwuwWbdq0oYcPH1K3bt3ou+++o/bt21NgYCA9e/asxnatra1p4cKF5O7uTi9fvqTAwECJfRbHiBEj\nqFWrVuTu7k67d+8mExOTKsvFxMSQjo4Oubi4kJGREY0cOZJevHhR6/br6lqoik8++YTWrFlTaXuL\nFi0oLy+PiIji4+Pp0aNHRFR6rsT5MXLkSBEbAoGA/u///o98fHzIwsKCLly4QIsXL6Z//vmnUnsV\nNVLXxMTEEABycHAgU1NTmjBhgsgwy4sXL8jT05NWr15NZmZm9P3339Pdu3drtKunp0fr16+ncePG\n0enTp+U6vJeenk4PHz4kW1tbicpfvny52mtWJqr7Bdi3bx+MjY3l+qtS1S/8lStXKrWzY8cOuLu7\nAwD8/PyEvfoyXFxchL/8Kioqwl49ADx8+FDmHtKzZ8/A4XCk6rmWsXjxYhgYGMDV1RV//vkn8vLy\nqi0vSU8+KSkJr169gkAgwLVr12BiYoKDBw/W6EtcXBx0dXURHR0ttszs2bOxZs0aAKjUk6/I7du3\nMXv2bBgYGMDd3R1xcXHVth8VFQUOh4PFixfX6KukFBcX4+jRo9DR0RHeDlekY8eO0NbWxq1bt1BY\nWIjZs2eL3AXKSl1dC7LA4XBEhi0kZePGjTAzM4OTkxM2btyI9+/fV1te2T15VVVVWFpa4tGjR8jL\ny8OYMWMwfvz4Kss+fPgQixYtQtu2beHs7IxLly5Va/vp06dQVVWV+a65Kng8HgYOHAh/f3+Jyu/a\ntQtmZmY1/j9IS7U9eT09PXr37p3MD8wkRV1dnXJyckS2ZWdnk6amJhERaWhoVLlfQ0OjyvrZ2dmk\nrq6uUJ+r4uHDh1RSUkLdunUje3t7UlNTq7XNzp07k7GxMXE4HOrTpw/NmTOHQkNDq63z+PFjGjZs\nGG3YsIFcXFyqLBMbG0sXL16kb7/9lohq7qVZWVmRg4MDdezYkR48eEDZ2dliy/J4PJoxYwbNnj2b\nNm7cWG3v39PTU/hQueLD9oqoqKjQ559/Tr169aKwsLAqy7Ru3Zo+++wz6t69O7Vo0YICAwPp2rVr\nMj1EK09dXQt1SXJyMmVnZ1O3bt2Ed6vVUZNGFE3r1q1p8uTJZGVlRWpqarRo0SI6ffp0lWXNzMzI\nwcGB7O3t6cmTJ/T27dtqbU+fPp0mTpxIp06dohs3bogt5+/vL9TrqlWrxJYTCAQ0YcIEatmyJW3a\ntKnGYwsPD6dFixbRmTNnSFdXt8by0lBtkO/Tpw+1aNFC7AVFRLR//37hQVf8aGpqSnSLam1tTcXF\nxfT48WPhtri4OOEtjq2trUhqWH5+Pj158kRkf2xsrEhded7y2Nraij3GWbNmCcsdPnyY7ty5Q3p6\nevTFF1+QnZ0drVmzhtLS0sTa5nA4cvOTiCglJYUGDx5MS5YsofHjx4std/nyZUpOTiZzc3MyMTGh\n4OBg+vvvv8nZ2VlYpqSkhM6cOUM+Pj7Url07OnPmDC1atIhSU1Opf//+Ym0vX76cjI2Nad26deTv\n708zZswQW/bMmTOUm5tLubm55OPjI9Ex8vl8sT+gDg4OEtmQlrq6FuRB+R/Oip/hw4cLy/3222/0\n+PFjsrW1pW+++Ybat29PS5YsEbkOyyNvrUpLTf+3ACgqKoqmTZtGbdq0oT///JN8fX3p9evX5OXl\nJbberl27KC0tjbZu3Uq//PILTZ06VSQ7rDzbtm0T6lVcFh8AmjJlCr19+5b+/vtvUlFRqdbvs2fP\n0vTp0+nkyZMSD+tIRU1d/eDgYBgZGSE8PBz5+fng8Xg4ffq01BkFAMDlcpGYmAgOh4PCwkIUFhYK\n93l7e8PHxwf5+fmIioqClpYWkpKSAHzIrvn777/B5XIxb9489OnTR1h327Zt6Ny5M9LS0pCamoou\nXbpg+/btUvsH1G64piKXL1/G5MmToaWlJTIsw+fzweVy4ePjg8WLF4PL5YptLzw8HBkZGRAIBIiJ\niYGpqSn27t0r3O/m5ia0nZqaivbt2+O3336r0beCggKkp6cjPT0dr1+/xg8//IDPP/8c7969A1Ca\nDWRsbAwnJyds2LBB4lvI2NhYaGlpCYcPuFwuOnbsiD///FOi+hW5f/8+Tp8+jYKCAvB4PPz111/Q\n0tISO1xz6dIl6OjoIDY2FjweD99++y1cXV1larsidXUtSIuswzUVuX37Nr755hvo6+vDz89PuL24\nuBhcLhcBAQGYMGECCgsLq80+q+7YfH19RYZ8eDweuFwu+vbti507d4LL5UIgEFRpd/fu3bC0tMTT\np0+Rn5+PsWPHijxkt7S0hLW1NX755RekpaVJdMxpaWmVhjVdXV2F2WGyMGPGDPTu3bvG4VoAuHjx\nInR1dREVFSVzezVRY5AHgP3798PZ2RlqamowNjbGiBEjcP36dakaKgueHA4HzZo1A4fDgaWlpXB/\nRkYGRo0aBTU1NbRr167SuPOFCxdgY2ODVq1aYcCAAZUu8vnz50NXVxe6urpYsGBBtb7Y2triwIED\nYv1s1qyZXIJ8GQUFBYiNjRV+9/X1FZ6Lsk/ZeOSVK1egrq4uLOvj4wM9PT2oq6vDxsYGGzduFLHd\noUMHXLhwAUDpuHpZZlDZp3x2zYoVK+Dp6VmljxVTNfPy8hAfHy/VcRYXF8PZ2Rm//vqryPbIyEjo\n6+vjzZs3UtkDgHv37qFXr17Q0NCArq4u3NzcRC7IiucLKE1La9OmDXR0dDBy5EikpqZK3a446uJa\nqIinpydWrlwpdn+zZs3kEuTL4PF4+Pfff4XfAwMDK+m1LPU3JSUF6urqePHiBYCaj23gwIH4448/\nhN/d3NxEynI4HFy+fBlA6XMQW1tbEd8CAwNhYGAAAwMDTJw4USTN9+rVq1If66hRo/DVV1+JbHvw\n4IFIJ1MakpOTweFw0KpVK5HrsCzeVDxfAwYMgKqqqkjZqjKbagMHYIuGNFRSU1PJ29uboqOjle0K\ng1EjPB6PunXrRvHx8TUOYTDkBwvyDAaD0Yhhc9cwGAxGI4YFeQaDwWjEsCDPYDAYjRm5Psb9H25u\nbiAi9mEfhXzc3NwUIVuma/ZR6kdRulbIg1cOh6P0t+MYjRdl6YvpmqFIFKUvNlzDYDAYjRgW5Osp\nkZHK9oDBkD9M13UPC/L1FHYxMBojTNd1DwvyjRx2UTEaI0zXktNc2Q4wPhAZ+UG85Zd+dXcv/chq\nU9a6DIY8YLpWLizI1yMqij4oSHobTPyM+oY8dE3EtC0rLMg3MvbsKf1X3j0nBkOZREaWapvpWnpY\nkK8HVNVDkVW0ycny6zkxGLWlorZrMzxjYSGqZaZryWBBvh5Q2yBffszz8uUP4me9m4YNj8cjFRWV\nBj0tb22DfJm2IyNLtV1GcnItHWtCsOyaRg4L9A2X7du304ABA+jp06fKdkUpVOy8uLmV/u3uTjRp\nknJ8aoiwnrySkGfGQfk6kZHsNraxMGvWLOLz+dSrVy9atmwZ+fv7K32dVUmQl7arKs+0LT0NPsi/\nf/+eJkyYQKtWrVLYIs6KQFHj5hYW8rHDUD4qKio0d+5c8vT0JF9fXzp27Bjt3r2bzMzMlO1atShK\n2+yuVDYa/HCNjo4OffbZZzRo0CBatGgRcblcZbukVNhtbOOjc+fOdO3aNXJ3d6fly5cr2x2lwDJo\nZKfRzEL5+vVrmj17Nt25c4e2b99OHh4eddp+bWD5v9LRlGehBNAghmzKYNqWHEXpq9EE+TJOnDhB\ns2fPpkuXLpGlpaVSfGAolqYc5BmNFxbkpYDP55OqqqrS2mcoFhbkRXn27Bmpq6uTgYGBsl1h1AI2\nn7wUsADPaEqcPXuWHBwcKDw8XNmuMOohjbInL47z58/TgAEDqHnzBp9U1KRhPfnKXL16lSZNmkS9\ne/emDRs2kI6OjrJdYkgJ68nXEj6fT6tXr6bevXvTnTt3lO0OgyFXXFxcKDY2lrS1tcnBwYHOnTun\nbJcY9YQmE+RVVVXp/Pnz9NVXX9HHH39M8+fPp4KCAmW7xWDIDTU1Ndq4cSOFhITQmzdvlO0Oo57Q\npIZrynjz5g19++23FBMTQ2FhYQ3qJSoGG65hNE5Ydo0YwsPDKTMzkyZNmiR1/vDZs2epV69ebPyy\ngcGCPKMxwsbkxWBhYUGbN2+mgQMH0qNHj6Sq+/HHH7MAz2gyhIWF0dWrV5XtBqOOafBBvmvXrnTj\nxg0aMWIE9enTh3755Rfi8/m1sikQCOTkHYMhO7t27aKMjAy52WvevDl9/vnnNH/+fCosLJSbXUb9\npsEHeaJS8c6dO5du3bpFUVFR1L17d4qJiZHZ3pAhQ2jNmjVUXFwsRy8ZDMkRCAQUHx9P9vb2dOrU\nKbnY/OSTTyg+Pp6ePn1K3bt3p1u3bsnFLqOeAwWgILMSIRAIcODAARgbG+Obb75BTk6O1DaePHmC\nwYMHo2vXrrh586YCvFQcERHK9kDxKEtfymg3IiICFhYW8PPzQ1ZWllxsll0jhoaG2LZtm1xsKhqm\na9lpFD358nA4HPLx8aG7d+9SXl4e2dra0okTJ6Sy0b59ezp37hx99913NGLECJo7dy7l5eUpyGP5\nUjaPt6z7Fd0+Qzrc3d0pPj6emjdvTg4ODnThwoVa2yy7Ru7cuUOurq5y8FLxKFvXddWGImh0Qb4M\nPT092r17N+3Zs4fmzp1LXl5e9OrVK4nrczgcmjhxIt29e5fevXtH//zzjwK9VQxViZIF+YaHhoYG\nbd++nXbs2EGTJ0+mr7/+mvLz82tt19TUlDp37iwHD+ueijpjQV48jf79fg8PD4qPj6fly5eTo6Mj\nrVixgqZMmULNmkn2+6avr0979+5VsJe1Q9xKPGWLejMaB0OHDqWEhASaM2cOOTo60p49e6hfv37K\ndkthVLfCFJvCWHIafZAnImrVqhX98ssv5O3tTdOmTaN9+/bRjh07qFOnTsp2TS6IW4mn7F95LjVY\nFYq2z/iAtrY2hYSE0PHjx8nLy4t8fHzo559/platWsmtjblz55KZmRnNmTNH4s6QIqhuhanyC3wT\nKU53jULbihjoV5BZuVBcXIz169dDT08Py5YtQ1FRkcy2wsPD8fDhQzl6V3t8fYHAwNIP0Ye/yx5c\nBQZKZkfWB12S2q8NytJXXbfL5/MhEAjE7n/79i28vLxgY2ODmJgYubX7+PFj9O/fH/3798fjx4/l\nZrc2lGlYnLal0V191bai9NUognxxcbHUdVJSUjBixAh06dIFV69elandLVu2QE9PDytWrACPx5PJ\nhrwpL2Bf38r7JRWqrIJmQV5+bNiwAZ6enkhNTa223KFDh2BoaIhFixbVqtNSnuLiYgQHB0NPTw9b\ntmyp9semLqgYmCtqWxrd1VdtsyAvhrdv36JDhw44evSo1EIUCAQ4cuQITExMMGvWLJlS1J49ewZP\nT0/Y29vjxo0bUtdXJG5ulbdJ2ouRVdB1kerWVII8j8dDYGAgDAwM8Ndff1Wr71evXmHkyJFwcHBA\nbGys3Hy4d+8eevbsibVr18rNpjyoqG1pdFdftc2CfDVERUWhc+fOGDlyJJ4/fy51/YyMDEybNg1t\n27ZFWFiY1PXL5+Zv3LhR6vqKoqqefHVUdztcn2gqQb6M27dvw9bWFqNHj0Z6errYcgKBAHv27IGB\ngQGWL18OPp8vl/b5fD7y8/PlYkteNEZtsyBfA4WFhQgKCoKenh42btwo0xBOZGQkrK2tMXr0aKSl\npUld//3793j69KnE5RUhMHmJuS6GXWSlqQV5oFTfCxYsgK8E0e3FixcYMmQInJ2dkZiYqHjnKqCo\nwNnYtc2CvIQkJSVhyJAhSE5Olqk+l8vFTz/9BH19fWzZsgUlJSVy9vADihZbbezX1wsBaJpBvgxJ\n9SgQCLBt2zbo6+vj119/lanTUxMZGRlVDiHVhXYao7YVpa9G9zJU586d6dy5c9SuXTuZ6rds2ZKW\nLVtGkZGR9Ndff5GrqyslJSXVyqf8/PwGN+lZg0kPa2JImtLI4XBoxowZFBMTQydOnCA3Nzd6/Pix\nXH2ZOnUqeXl50du3b0W28/l82r17N02bNo38/f3p6NGjtZ40UJ40OW0r4pdDQWbrnJKSEmzevBn6\n+vpYsmQJCgsLZbKzePFiuLq64t69e1Ldctb2trc+jTfKE2Xpq77qOjU1tdqkgZKSEqxduxZ6enrY\ntGmT3O5OuVwufvjhBxgbG2PZsisIDAQmTnwKIqBjx/0YPvwmZs8+Bjc3N7Rr1w537twR1mXaroyi\n9NVkgrxAIMC6devw/v17qeumpqZi1KhR6NSpE65cuSJ1fXG5+WW3jeIEW19vK5UNC/KirFu3Dubm\n5jh//ny15e7fv49evXrBw8ND5uHMqoiKikKHDh0wYsQI6Orqwtf3mXBfmbYPHz4MY2NjvHr1CgDT\ndlUoSl+NbrhGHMXFxfT48WOys7Ojw4cPS7UCS5s2bSgsLIxWrlxJPj4+NGPGDMrKypK4voqKCs2e\nPZvu3LlD//77L3Xr1o2uXbsm3K+IOTEa6jwbDOmZM2eOcF6bWbNmiZ1Mr1OnThQdHU2DBw8mZ2dn\n+uOPP+SyElG/fv0oLi6Onjx5QkOHDiULCwvhvjIdenl50ahRo2j79u21aovpWgYU8cuhILNy4dq1\na7Czs8OwYcNk6s1kZWXB398fpqamtcrNX7FihchbqHw+HyEhIejSZRZUVVegdes1IAL8/V83qgwC\neaAsfdVnXQNAZmYmJk6ciPbt2+Py5cvVlk1ISEC3bt3g6ekpUyZZRQQCAVq3bo03b96IaDUwsPRO\nOCAgADo6o0EUBH39TSACAgIKpdY207UMdhVitJ5fDEVFRfj555+hp6eH69evy2QjOjq6Vrn5Fcfm\nO3U6iLZt/8CKFVeRlZWFlJQUeHhcgYGBAY4dOya1fXYxNJ52peX48eMSzRMvzctWkthq1qwZBAJB\nJW2rqf2Knj1P47ffbsHY2BhXr15F586HYWtrizdv3kjVDtO19DT4hbxrw6NHj8jc3JxatGhR5f7n\nz5/TlStXCAD16dOHrKysRPYXFRXR6tWracOGDRQYGEizZs0iFRUVqf0YODCKVFVX0IkTJ0hVVZWI\nPkyMNHLkfzR48GC6e/cumZiYVGun4mRKgYGlfzeoyZQkgC3kXTsAUFRUFD158oS0tbXJwMCA/P39\nydramrZt20aGhoYy2TU3N6cTJ06Qo6MjERGVlJSQoeEW8vd/TYsWLaILFy7Q4sUXKCFhIwUGggoL\nA+jRo0d07Nixau0yXdcSRfxyKMhsnfHu3Tt8/vnn0NXVhZeXF3x8fGBgYCB2HpF79+6hf//+6NWr\nF+Lj46Vqi8/nQ0MjGHFxcYiMjBTm5pe/jZ0+fTqWLl0qlV3W42k87cqT8+fPo1OnTrCzs8PEiRMx\nZMgQ6OrqIjAwEPPnz4exsTFCQ0Nlsr1s2TKMHz9e+P3UqVMwMdmOadOmwcrKCo6Ojhg1qnTKhYgI\nIC8vD7q6ulINmzJdy2BXIUYb8MWQl5cHOzs7fPfdd8jLyxNu53K5WLZsGTp06IB3795VqldSUoLt\n27dDX18fixYtApfLlai9Bw8ewNjYG0Dpi1x9+/ZF3759MWvWh9vY06dPY9CgQVIdB7sYGk+78mLV\nqlXQ0dHBmTNnRIZmnj17ht69e2Pu3Lm4du0aOnbsCB8fH6kz0bKysmBra4u5c+ciMzMTCxYsgJ/f\nXrx69Qr9+/fHRx99hN69z4pcG59//jn2798vcRtM19LTZLJrJOXPP/+kly9f0vv374nL5Qq3t2zZ\nkn766SdydXWljRs3VqrXrFkzmj59OsXHx9PDhw/JwcGBIiIiamwPALVu/S8REaWnd6ZBg6JJW3sd\nbdliQG5uEbR4cTElJOiJ3MZJkmHQmG5jGbUHAG3bto1KSkro5s2bIovUW1hY0OnTpykkJISMjY0p\nNjaWDAwMpF5EXEtLiyIjI+n169dkaWlJhw8fpoiIQLKymkpc7gKaOTOdbtwYSmZmu2jmzHSKjCwd\noihPTdpmupYBRfxyKMhsneDk5ISTJ09izpw5MDIyqvRAKi4uDmZmZjXaOX78OMzMzODn51dtj4jP\n58PU1LTSMM9332Xj008/hYeHB2bMmCEyXNOYezOSoCx9NWRdx8bGwtLSEikpKRg6dCi6d++Ou3fv\nipSZM2cOAsuJqzaLiL9+/RqBgYGwtrZGZmamcPuSJaWT+T148KDK4Zqy5vPz85GWloaCggKpj7Wh\noih9sSBfAX19fbx+/RoAEBMTg6O6uojV1kaemxuQmYni4mJwOByJMhGys7Px9ddfw9jYGAcPHhRb\nZ/ny5fD09BSZk75M7OfPn4eurq5Impsig3xDeJOQBXnp+eeffzBs2DAApemOO3bsQEiLFnhpbQ14\negKZmdi5cyf8/PxE6uXk5GD69OkSvWxVkeLiYlhZWeHAgQPCbeW1GxAQgFGjRonU8fd/DS8vL6ip\nqcHIyAgaGhqYPHmyXBYvqe/aZkG+jrCxscG///4r/F7Sv39pHhgRMHYsnj17Bj09PalsXr9+HXZ2\ndvD09KzyIROPx8Mnn3wCFxcXnDx5Erm5uThy5A1WrVoFQ0NDhIaG1tlUqQ3hLoEFeekpez+kfEej\noGdPEW3/9NNPmDdvXpX1z549i7Zt22LWrFnIzc2VuN3Y2FgYGxvju+++w71793D6dAFu3LiBL774\nAl26dEF6erpQ25Mnp4AIGDLkGhYs4CIiAkhPT0dQUBCMjIyQkJBQq3NQ37XNgnwdsWzZMkyZMuXD\nBk/P0ovA2RnIzERAQABmz54ttd2ioiKsWLECenp6WLt2baVZAfl8Pv7880/06tULrVu3hq6uLvz8\n/KpcAGLkrCu9AAAgAElEQVTx4mK8ePFC+F2egb6+XwgAC/KyUFJSgvbt24uuglZO20Xp6WjTpo3I\n/DIVKf+ylTTTe7x48QILFixA27Zt0apVK3Tq1Alr1qxBdna2sAyfz4empiZGjLgl3FZe17t27YKz\ns7PEbVZFfdc2C/J1xJs3b2BmZobt27eX9noyM4GxY4HMTBw5cgRGRkZSzRlfkQcPHsDd3R3Ozs7V\nXlDVMWlSMvT09LBhwwYUFxfXWrwNYUGF8rAgLxv79u2DpaUlHj16VLrhf9ouePkSXl5eGDNmjER2\nwsPDYWJigrlz58ptzPz48eOwtLREq1arERwcXEnXxcXFsLS0FLnLloSGpG0W5OuQ+/fvw9bWFvb2\n9ggMDMSyZcvQo0cPtG/fHrdv3xaW+/3333H27Fmp7QsEAuzatQsGBgaYP3++1KvuRESUplv269cP\nvXr1gr//a6l9EEd97+0ALMjXhq1bt0JbWxuff/45Vq5cidmzZ8PAwAA+Pj7CgL1w4ULs2LGjThcR\nDwwMxE8//YR9+1LRr18/9O/fH998I5qwMHXqVGzfvr0WbdTSSQWjKH01jRTK6dNLc6+GDSOSYGKx\nTp06UUJCAv3+++9UXFxMXC6XlixZQg8fPiQnJydhuS5dutDMmTNp/Pjx9ObNG4nd4XA45OfnRwkJ\nCfT8+XOyt7en8+fPS1zf3b003dLD4woZGGymbduMqH//i7RkiaDKFDQ2qVMjRUpdExH5+/tTcnIy\nDRo0iDIzM8nIyIiuXr1KBw4coFatWhER0bhx42jbtm00bNgwSktLq9KOvr4+HT58mIKCgmjkyJH0\n448/UlFRkcyH8tFHH1FBQQG1adOGBgy4TK1araaNG3VpypQXFBRUquH8/HzhG+FlMG1LgCJ+ORRk\nVnbc3EQeMFXLtGml5f+XcVATeXl5+OGHH2BoaIg9e/bINP/HqVOn0K5dO0ycOBFv376Vuv7cuTn4\n+eefxbYtTQ+mPt7GVkRZ+mpKuubxeAgKCpJqEXF7e3uZhyDv3LkDMzMzkWdVX331VvjiVE5ODnR0\ndESeRQGNS9uK0lfTCPIVHp5WS/kLx9hYogsCKF1s2cnJCcuWLZPJxdzcXHz77bdV5ubXRE1Cr++3\nqdLCgvz/qCNd29raYv369dWWk8ci4gMGDMCiRYuE2i/TbUlJCaZMmYJx48ZVqtOYtK0ofTWNCcqy\nskpvbXfsINLWrr7ssGFEZ858+D52LNGRIxI1U1xcTPn5+aSlpSWzqzdv3qRp06aRkZERbd26ldq3\nb19jncjIym8CNuZJndgEZf+jjnRdVFRERUVFpKmpWWPZ1NRUmjJlCmVkZFBISAh16dJFojaIiNLT\n02nIkCFkZGRE/v7+9P69Pamp3aRNmzZR8+bN6dSpU6ShoUEREaDLl0vflG1M2mYTlNUVmZmlPR1J\ne0gKgMfjYfXq1dDT08OaNWtk6hUBQEpKCsaMGYM5c+r+GBSJsvTVlHT98uVL7N69G5s3b0ZUVJRU\nd5a1WUScy+UiJCQEgwYNgoODAzw9PXH06FHhi4J8Ph89e/YUvlzIevIS2FWI0YZ8MQAiaZPy4MWL\nF1K9QFLG48ePMWjQIHTr1g23bt2quUIFynLzW7Vajd9//12qi60+w4K8jEig6/z8fPj5+UFbWxve\n3t6YPn06bGxsYGdnVyl9sfwEflXx5MkTuLq6wsXF5UPaphz4999/0blzZ3z++ef44YfqfWhIsCAv\nL6R8ACUP1qxZA3Nzc5w8eVLqugKBACEhITA0NMTcuXNrvLCqYu/e57XOza9PsCBfBXLQdUlJCTw9\nPeHj4yMyV41AIMDRo0dhYGAgMsfS6NGjMXnyZIkXEd+4caNcFxGfN28edHRGy7SoTn2EBXl5IU1G\nghw5f/482rdvDy8vL+FixtLw5s0bfPnll7CwsMCZM2ekrl+Wm29mZibTYub1CRbkq0AOuj537hzs\n7e3FDg+uX78en332mfB7bm4u/P39YW5ujv/7v/+r1vb9+/fRu3dvuS8iHh0dDSsrK9Z5qc6uQozW\n54tBmowEQK49//z8fAQEBMDAwAB79+6Vyca5c+dgaWmJcePGIT09Xer6hYWFMrVbn2BBvgrkoGsf\nHx9s2bJFbJWcnBxoaWlVWk/h3LlzMDMzw8yZM6sdluTz+Vi5ciX09fWxc+fOWi03WJ6ioiK52FE2\nitJX03gZqozp04lycoiMjYlCQ2vOSCAieviQ6PLl0syE6dPF25XgpZTWrVvTypUr6fz58/TRRx/J\ndAhDhgyhhIQEMjU1JXt7e9qzZ49UT+TFLXXIaOAYGBDp60umaaIqdZ2WlkY2NjYfylTQtYaGBhkb\nG1N6erqIqSFDhlB8fDwVFBRQeHi42CabN29OAQEBFBERQVu2bKHhw4eLfdlKGmS9lpoMivjlUJDZ\n2iPLLa0kPSQlDQGV5eZ7eHjU+sFWZGSk3HpWikZZ+qq3ugak12AVuh4zZgx2794t1iaXy4W2trZM\nd5AVKb+I+L59+xSivZSUFLnbVCSK0lfTCvLS3tICkmXayGJXTvD5fAQHB0NPTw8rV64UmZNeUrKz\ns+Ho6IghQ4bgyZMnCvBSvrAgXwXSarAKXYeHh6Nnz54fHo5WsLlr1y54enrK1e1bt27B1tYWo0eP\nlsuPRxkFBQWwsLDA119/LVOygjJgQV4eyDk1UsjEiYC+PjBoUK1sHzp0CF999ZXIFKyS8vTpUwwd\nOhQODg4yTRhVPjf/119/lTk3vy5gQb4K5KBtPp+Pvn374uuvvy4d587MBCwtARcXvOvZEx309HD9\n+nWp7cbExFT7LKiwsBALFiyo1SLiVfH+/XuMHz8eVlZWiI6OlptdRcGCfH1GTsM179+/x5QpU9C2\nbVuEh4dLXV8gEGD//v0wNjbGnDlzZM7NHzhwILp164Znz55JXb8uYEFecWRkZGDEiBEwNjbGN998\ng6fm5kJtv+rfXyab48ePh729Pf77779qy127dg3W1tYyLSJeHceOHYOxsTHmzZsnsoh4fYMF+fqA\nuEwbOQ/XREREoGPHjhgzZgxevnwpdf13797B19cX5ubmOHHihNT1BQIBDh48KPUUyHUFC/IKoIK2\nExMTsWrVKjy0sgKIUNK9u8zaLnvXw8DAAMuWLat2SDE/Px+zZ8+GqampTO+ViOPNmzfw9fVFamqq\n3GzKGxbk6wPieuwKGAbicrn48ccfMWDAAJltXLhwAVZWVjLn5tdXWJBXAHWg7RcvXmDo0KFwdnbG\n/fv3qy1bm0XEGyosyNcHpOmxyym/XpYHqeUpKCgQ5ubLIze5PmTgsCCvAOpI2wKBANu3b5dovdba\nLCLeEGFBvi4RJ2JpejVKSqsUR2xsLHr06AE3N7cae1HiKC4uRp8+fWQaApInLMjLSHXBuR5rW9ZF\nxCWhpKQEe/furReJBorSV9N6GUpSyr8oYm1d+jKIry/RqFFEeXmS2WjduvRfZ+fSqWDlSHZ2Nl29\nelWqOo6OjnT9+nUaNWoUubi40M8//0w8Hk8qGyoqKrRp0yZq166dVPUY9YTyunZyKn3RycyMqF8/\nonHjJJuymEih2q6KoUOHUkJCAuXl5ZGjoyNFRUXJzXZubi7t3buXXFxc6P79+3KzW69QxC+HgsxW\njzwnHiu7dVVX/9BjMTCQrveiqHRNlOYWm5qaYsaMGciswv6DBw8QGhqKU6dOVdnzSUlJwfDhw2Fr\na4tr167J3T9FoxR9KatdReja2RlwcfmgZ2l75QrS9urVq6vN6OLz+Vi+fDl0dHQwbtw4uSUGCAQC\nbNmyBQYGBjIlOsgLRemr8QR5ed5Clol40KAPF0X5v5Uwx3xlFzMxffp0mJqaIjQ0FAKBAPfv34eH\nhweMjY0xatQoDBgwALq6uli4cGGl21GBQIBDhw7B2NgYs2bNkik3X1k0qSCvCF1nZn4I+Fpa9ULX\nAoEAq1evFjuvza5du9C2bVs4Oztj2LBhUFdXR4sWLWpcsUoalD1xHwvyNaGIt07LXxQK7JnXhitX\nrsDGxgaDBw+GkZERNm7cKDJh0/PnzzFkyBCMGzeuyoem79+/x9SpU2XOzVcGTSrIK+pt6jI9JyfX\nK10nJCTAyckJH3/8sTDdcd26dejYsSNu374tLCcQCLBgwQJwOBx4e3s3iknKWJAXR9nt7KBBwKhR\n9UasUlOL2/LCwkK4urri559/rnJ/QUEBOnbsWO1bfxEREbC2tsaYMWOQlpYmVft1TZMJ8tOmlQ6r\nGBuXBuOGiAy6Lr+I+P/93/9BW1tb7DDOoUOHoKamBgcHhwY/3TAL8uKoZ1ksMlOL48jKyoKWlhYy\nMjLEllm7di18fX2rtcPlcrF48WLo6+tj69atclvgQd40mSDfGLRdi2O4c+cOfv/9d3h7e4stIxAI\nYGdnh4ULF0JfX7/Gl63qM4rSV8PPrqnjJ/0KoxbH8fLlSzIyMiIdHZ0PG/83TSw8PYmysqhr1670\n7Nmzau20bNmSli9fThERERQSEkKurq507949aY+EIS8ag7ZrcQxdu3ally9fUteuXT9srDD9MYfD\noW7dulGnTp3ov//+o+joaOrbty8lJSXJ7xgaOA0/yB84ULry/Pnzks+lXR+pxXFoa2vT27dvic/n\nf9j4v3Q5ztmz9Gb0aEpLSyNtCe3a2dlRdHQ0eXt7U//+/SkoKIiKioqk8okhBxqDtmt5DDo6OvTy\n5csPG8TMg6+trU1mZmZ09uxZmjp1Krm5udFvv/1GJSUl8jqShosibg8UZJZRDW5ubjh06NCHDf97\nYJdhZYUupqYwMTHBH3/8IbXdFy9e4NNPP4WNjQ2ioqLk6LHsKEtfTNd1z6NHj6Cvry9MlywZOhQg\nQmbHjkBmJh4/fgx9fX0UFBSI1Hv69KlCFhFXJIrSFwvyjYTz58/DxMQEcXFxpRv+lz0hyMjAwoUL\noaurCxMTExw+fFjqqQkEAgH+/vtvmJqawt/fv8rc/LqEBfmmxZdffonRo0eXziCZmYl0d3c4mJvD\nx8cHXbt2xerVq6usp6hFxBWFovTF+Z9xucLhcEgBZhk1cPDgQfrqq6/Iw8OD3N3dKTs7m/bv308a\nGhoUFhZGycnJtGDBAjp+/Djp6upKbT8rK4sCAgIoKiqK4uPjSUVFRQFHUTPK0hfTtXIoLCwkPz8/\nioiIoIkTJ5K5uTn9999/tG/fPmrZsiWFhobS4MGDxdZ/8OAB+fr6kpqaGu3evbvevrGtKH2xIN/I\nyMnJof3799Pdu3epZcuWNHLkSHJ1dSUOhyO3Nt69e0f6+vpysyctLMg3TZKSkujgwYP0/v17MjMz\no4kTJ1JiYiJNnTqV/vnnH9EHtBUoLi6m3377jYKDg2nVqlXk5+cn12tCHrAg39iYPr30IVLr1qUP\npxrqgzUlwIJ8PaeOtV1QUECty7J4aiAhIYF8fX3JxMSEdu7cSaampgr1TRoUpa+Gn13TUKkiS0BZ\nCAQC2rx5M3G5XKX6wWgk1LG2JQ3wRET29vYUExNDzs7O1LVrV9q/f3+j/+FmQV5Z1KMc6IKCArpy\n5QrZ29vTxYsXleoLoxFQT7T99u3bKrerqqrS0qVL6cyZM7Ry5UoaM2YMvXnzpo69qztYkK9Lyr/I\nsXVrvcmBVldXp8OHD9O6devIz8+PJk+eTO/fv1eqT4wGRIUXlOpDfj+XyyVnZ2datGiR2Hc8unfv\nTrdv3yZra2tydHSkv//+u469rCMUkbKjILMNnwbwmnpOTg5mz54NY2NjmRcXUTTK0hfTtRjqqa5f\nvXqFkSNHwt7evsZ5ba5du4aOHTsiODi4jryrjKL0xR681iXDhpWOUzo714sefHXExsaSvb290tIk\nq4M9eK1n1GNdA6C//vqLfvjhB/rmm28oICCAVFVVqyxbUFBABQUFSsscY9k1jYGsrNJbW0lX4GFU\nCQvy9YwGoOvU1FSaM2cOrV27lszNzZXtTpWwIM+oF+Tm5pKGhoZSfWBBntEYYSmUDKVTUlJCvXr1\noj179ijbFQaDISEsyDMkRkVFhSIjI2nIkCHKdoXBkAsAKDw8nAQCgbJdURhsuIbR4GDDNQx5kZWV\nRZ6entSqVSvavXs3WVhYKM0XNlzDYDAYckZbW5uio6Pp448/pp49e9KrV6+U7ZLcYT15RoOD9eQZ\niuDVq1dkYmKitPZZdg2D8T9YkGc0RthwDYPBYDCkhgV5BoPBaMSwIM9gMBiNmOaKMOrm5lbvVl1h\nNB7c3NyU1i7TNUNRKErXCnnwymAwGIz6ARuuYTAYjEYMC/IMBoPRiGFBnsFgMBoxLMgzGAxGI6ZJ\nB/n09HRydXUlTU1NmjdvnrLdYTDqjMmTJ5Ouri717t1b2a4wFEytgvyBAwfI2dmZNDQ0yNTUlIYN\nG0ZXr16V2s7atWvJxMSEtLS0aMqUKcTj8cSWnT59OtnY2JCKigqFhITUxn3asWMHGRoaUk5ODv36\n66+1siUOaY4tOjqaevToQVpaWtShQwfauXOn2LJFRUXk5+dHWlpaZGJiQmvXrhVb9pdffiENDQ3h\np3Xr1qSiokIZGRkyH1deXh5ZWlrSgQMHhNtyc3PJ3Nycjh07JpPNu3fv0tChQ8nAwICaNZNcmnv3\n7qVmzZrRrl27JK6jDO3GxsZS9+7dSU1NjZydnSkuLk5iW5s2bSJnZ2dq2bIlTZ48WWo/yxMVFUUX\nLlygly9f0o0bN2plSxwXL14kGxsbUlNTIw8PD3r+/LnYshkZGTR69GhSV1cnCwsLOnjwYLW25XnO\nZWHMmDE0ffp0kW2jR4+m2bNny2xTmvMltRZkXRw2ODgYhoaGCAsLQ0FBAYqLi3Hy5EnMnz9fKjtn\nz56FkZERkpKSkJmZCXd3dwQEBIgtv3nzZly8eBHOzs4ICQmR1X0AwJQpU7B48eJa2ahIVlYWioqK\nAEh3bMXFxdDX18eOHTsAADdv3oS6ujri4uKqLB8QEABXV1dkZWXh3r17MDY2xtmzZyXyMSgoCAMH\nDhR+T09Pl+YQhZw7dw4GBgZ4+/YtAMDf3x9jxoyRyRYAPHjwALt378bx48fB4XAkqpORkYFOnTrB\n3t4eu3btkqiOMrRbVFQEc3NzrFu3DjweDxs2bEC7du3A4/EksnXs2DGEh4dj5syZmDRpklR+VuSv\nv/5Cv379amWjIjweDxkZGQCAt2/fQktLC6GhoSgqKsK8efPQu3dvsXW9vb3h7e2N/Px8REdHQ0tL\nC4mJiVWWlec5z8jIEP4tDa9fv4aenh4iIiIAAIcOHYKFhQXy8/OltgVIf76k1YJMQT4rKwvq6uoI\nDQ2VpboIPj4++PHHH4XfL126BGNj4xrr9evXr1ZB3tfXF6qqqvjoo4+grq6OixcvymyrpKQE58+f\nh4+PDzQ0NPDy5UsA0h1bWloaOBwOuFyucFuPHj1w6NChKsubmpri/Pnzwu9LliyBt7d3jb4KBAJY\nWlpi7969wm0zZ85Ely5d8Ouvv+LVq1c12ijPpEmT4OPjg4iICOjp6cn8g1GeR48eSRzkZ8yYgS1b\ntsDd3V2iIK8s7Z47dw5t2rQR2WZubo5z585JZWvx4sW1CvJ//PEHWrZsCRUVFairqyMoKEhmWwCQ\nkJCAuXPnwsjICEeOHAEAbN++HS4uLsIy+fn5aNWqFR48eFCpfl5eHj766CM8evRIuG3ixIliA7c8\nznlZZ+jQoUMwMjLC999/j7t370p4xKXs2bMHVlZWSElJgZGRkfD/URakOV/lkVQLMg3XXL9+nQoL\nC2n06NFiyxw4cIB0dHTEflJTU4mIKCkpiRwdHYX1HBwcKD09nTIzM2VxrZIP5W2XZ8+ePTR+/Hha\nsGAB5ebmkoeHR7U+6+rqCn0u4+nTp7RkyRJq3749ff/999SjRw968uSJcLpSaY7N1NSUHBwcaPfu\n3VRSUkLXrl2jlJQU6tevX6WymZmZ9OrVq0q2ExMTazwnUVFR9PbtWxozZoxw2+bNm2nDhg0UHx9P\nNjY29Omnn1J4eDgVFxfXaG/t2rUUERFBY8eOpeDgYDI0NKyxjrz4999/6b///iN/f3+J6yhLu4mJ\nieTg4CCyzdHRUfh/JqktSPDu4vPnz0X8LM+UKVNo27Zt1KdPH8rNzaXAwEBheXGfQ4cOidjIzMyk\nLVu2UI8ePWjo0KHUvHlzoQbKjrX8sbRu3ZqsrKzo7t27lfx5+PAhNW/enKysrKo8LxWR5zn/4osv\n6OLFi9SsWTMaMmQI9ezZk7Zu3UpZWVlVtl0eX19f6tChAzk5OZGnp2etVkuT5nyVRxItEMk4Jv/+\n/XvS19evdtx03LhxlJmZKfbTtm1bIiod29XS0hLW09TUJKLS8d3aMm7cuBrH4MqfqOp8zsjIEPoc\nFxdH7u7u1KdPH8rJyaHw8HCKi4uj7777jgwMDIT2pD22HTt2UGBgILVs2ZLc3Nzol19+oTZt2lQq\nl5eXR0RUybYk5ywkJITGjh1LrVu3Fm7jcDg0cOBA2rt3L6WlpdGoUaNo7dq1ZGpqSkuWLKnWnra2\nNtna2hKXy602cMqbkpIS+uqrr2jTpk1STTWgLO1WLFtWvqyspLYkOVZzc3MRPytSMTiUlRf38fb2\nFvri7e1NlpaWdPnyZVq+fDmlpqbS6tWrqXPnzkJ7+fn5Qv/LH0+ZbsuTl5dXqayGhoZYLcvznBMR\n2dra0po1ayg1NZWCgoIoMjKSLCwsyMfHp8brqV+/fpSRkUFffvllteVqQprzVR5JdS9TkNfT06N3\n797JZV1EdXV1ysnJEX7Pzs4motL/6PpKdnY2PXjwgDp27EgODg7Uvn37KstJc2xpaWk0YsQIOnDg\nAPH5fEpMTKTVq1fT6dOnq7RLRJVs13TOCgoKKDQ0lHx9fcWWUVNTI3t7e+ratSsVFxfTw4cPq7W5\nb98+SklJoUGDBtGCBQvElouKihI++LW3t6/WpiRs2bKFHBwcqGfPnsJtkvRslKVdDQ0NkbJEpUvP\nlZWV1JakvTdFUKZLfX196tq1K9na2lYZaCoeC5F4fUpTtqry0p7z7OzsSgGVqDRg2tnZkaOjI+np\n6VFiYmK1d7KPHj2i4OBg+uqrr2ju3LnVllVXVycNDQ3S1NSs8u5K2nNQhkJ78n369KEWLVpQWFiY\n2DL79+8Xyego/yl/sLa2thQbGyusFxcXR0ZGRqSjoyOLa7VCUp9dXV0pNTWVFixYQKdOnaJ27drR\n+PHj6dy5c1RSUiK0J82xXbt2jdq2bUuDBw8mIiJra2saPnw4nTlzplJZHR0dMjExqWTbzs6u2uML\nCwsjPT29KidCSk1NpVWrVpGtrS35+PiQoaEhxcfHV7pVL8+bN29o7ty59Mcff9C2bdvoyJEjFB0d\nXWXZ/v37U25uLuXm5lJCQkK1fkrCpUuXKCwsjExMTMjExISuXbtG33//fY0ZDsrSrq2tLcXHx4ts\nS0hIIFtbW6lsKWKCtOfPn4s9Xg0NDWG2i66uLiUkJNChQ4coNTWVnJycaODAgRQSEiLS67S1tRW5\ng87Pz6cnT54Ij7U81tbWVFxcTI8fPxZuq07LtT3n8fHxIn7k5eXRnj17yMPDg7p3704vX76kI0eO\nUHx8vNgYBICmTp1K3333HW3YsIHU1NRo9erVVZYtayM3N5dycnKqvLuS5nyVR2ItyPqwIDg4GEZG\nRggPD0d+fj54PB5Onz4tU4aCsbExkpKSkJGRATc3NyxcuFBseR6PBy6Xi759+2Lnzp3gcrkQCAQy\nHYOvr69csmvevXuH9evXo2vXrjA1NRVmm0hzbElJSWjdujUuXboEgUCAx48fw8rKCjt37gQARERE\niDyMDAgIgJubGzIzM5GUlARjY+MaH/4MHjwYgYGBlbYHBgZCS0sLfn5+iIqKkvi4x44di+nTpwu/\n//HHH7CxsRFmF8kCl8tFYmIiOBwOCgsLUVhYWGW5rKwspKenIz09Ha9fv0bfvn2xdu1a5OTk1NiG\nMrTL4/HQrl07rF+/HoWFhVi/fj0sLCzA5/MlslVcXAwul4uAgABMmDABhYWFKC4ulsrfMv7880+5\nZNfweDwcPnwYnp6e0NTUFOrvzZs30NLSwt9//w0ul4t58+ahT58+Yu14e3vDx8cH+fn5iIqKgpaW\nFpKSkoT7ORwOLl++DEC+5/zMmTPQ1NTEsGHDcOTIEYkzbTZv3gw7OzuhnaSkJGhqauL+/fsS1a9I\nWXaNpOdLWi3IHOQBYP/+/XB2doaamhqMjY0xYsQIXL9+XWo7v//+O4yMjKCpqQk/Pz+Rk+3p6YmV\nK1cKv7u5uYHD4aBZs2bgcDgiAqjIvn37YGtrK7bdSZMm4aeffpLa3+qIj49HXl6e8Ls0xxYSEoLO\nnTtDQ0MDbdu2Fckw2Lt3r8iFWVRUBD8/P2hqasLIyAhr164V8UNdXR3R0dHC76mpqVBVVcWTJ08q\n+RwbG4uCggKpjjMsLAxt2rRBdna2yHYPDw+ZfzifPXsm/D8t+/+1tLQU7q94vsojaXZNGcrQ7p07\nd9C9e3e0atUK3bt3R2xsrMS2AgMDheem7LN06dIqfUpJSYG6ujpevHhR5f49e/agf//+Uh9rdbx6\n9QrPnj0Tfr9w4QJsbGzQqlUrDBgwACkpKcJ9K1asgKenp/B7RkYGRo0aBTU1NbRr1w4HDx4U7nv+\n/Dk0NTWF6ZmA/M75s2fPpM4mS0lJgba2NmJiYkS2L126FK6urlLZKo8050saLQAAm2q4gTBt2jTy\n8vISDucwGE2B/fv3U1JSEq1YsULZrjRYWJBnMBiMRkyTnruGwWAwGjssyDMYDEZjRuYnBdXg5uYG\nImIf9lHIx83NTRGyZbpmH6V+FKVrhYzJczgcpb60wWjcKEtfTNcMRaIofbHhGgaDwWjEsCDPYDAY\njZgGH+QBUExMjLLdYDAYjHpJgw/y79+/py+//JLGjh1Lr169UrY7DAaDUa9o8EFeX1+f4uPjqVOn\nTu9VcvwAACAASURBVOTg4EBbt26VywyDDIayycnJIS6Xq2w3GA2cBh/kiYhatWpFP//8M0VGRtK+\nffvIxcWlyik9GYyGRFhYGHXo0IHWrVvHgj1DZhpFkC/D1taWoqKiaObMmaSvr69sdxiMWuHr60un\nT5+mK1eusGDPkBmWJ89ocDTFPPnY2FhatmwZxcTEUFJSUqUVjxgNH0Xpq0kFeQAKWXSBUbc0xSBf\nRnJyMllYWCjVB4ZiYC9D1ZKSkhLq3bs37d69W+kXKoMhKyzAM6SlyQR5FRUV2r59O23dupXc3d3p\n/v37ynaJwZAbAQEBtGHDBjZmz6hEgw/yO3bsoB9//FEicXft2pVu3LhBY8aMoX79+lFQUBAVFRXV\ngZcMhmLx8vKiS5cukZWVFQv2DBEafJAfMWIEPXz4kBwcHOjixYs1lldRUaHZs2dTbGwsxcXF0ZUr\nV+rASwZDsTg5OVF4eDidOHGCLl68SFZWVrR161Zlu8WoBzSaB68nT56kr776itzd3em3334jAwOD\nOm1f3kRGErm7K9uL+klTfvAqKf/99x/dvHmTZsyYoWxXGBLCHrzWwIgRIygxMZH09fXJzs6O9uzZ\n02AuyKqIjFS2B4yGjJOTEwvwDCJqREGeiEhdXZ2Cg4PpzJkztGnTJvLw8KCHDx9Kbeevv/6iJ0+e\nKMDDuof9WDAqcvz4cSosLFS2G4w6olEF+TKcnJzoxo0bNGrUKOrbty8tW7ZMqgesGRkZ1KtXL1q5\nciXxeDwFeipKZCRRUFDpZ+nSD39LE6grlmVBnlGeoqIi2rVrF1lZWdGmTZtYsG8KKGK5KQWZlYmU\nlBR88skn6Ny5M65cuSJxvWfPnsHT0xO2tra4evWqAj0EIiIqbwsMlM2Wr6987NRnlKWv+qTr2nLr\n1i188sknaNOmDTZu3Agul6tsl5o8itJXc+X+xCgec3NzOn78OB07dox8fHzI09OTVq9eTbq6utXW\ns7CwoFOnTtHRo0dp7NixFBQURNOmTVOIj/J8yJqcXGqvrAe/dOmHfe7u7GEuo5Tu3bvTP//8Q7dv\n36agoCDKzc2lhQsXKtsthgJo9EGeqPSp9ZgxY2jQoEH0448/kq2tLQUHB5OPj0+10xxwOBzy8vKi\nIUOG1Hk+vTTBuHxQv3z5w99lNoKC5OUVo7HRvXt3OnHiBJueuxHTJIJ8GVpaWrRp0yb68ssvacaM\nGRQSEkJbt26l9u3bV1tPW1tb7r7U1NtmPW5GXdKsWeXHcwCIx+NRixYtlOARQ24oYgxIQWblCo/H\nw+rVq6Gnp4dVq1aBx+NJbePt27fg8/m19kWe4+ZubqLfqxrvb+goS18NQdfyJCYmBm3btsXmzZtR\nWFiobHcaPYrSV6PMrpEEVVVVmj9/Pv3777906dIlcnZ2phs3bkhlIzg4mHr27Em3bt1SkJfSU3H+\nKnZHwJCVnj170rFjx+jUqVNkZWVFmzdvZtk4DRFF/HIoyKzCEAgEOHDgAIyNjTFr1ixkZWVJXC8k\nJASGhoaYPXs2cnJyZGpfnr3txthzr4iy9NXQdC1P/v33XwwfPhxt27ZFfHy8st1plChKX022J18e\nDodDPj4+lJSURHw+n2xtbenvv/+u8Y1ZDodDEydOpMTERMrJySFbW1s6efKk1O3Ls7fNeu4MRdCj\nRw86efIkHTt2jDp27KhsdxhS0CjmruHz+aSqqio3e1FRUTRjxgzhCyPm5uYS1YuIiKDk5GSaPHmy\n3HxhVIbNXcNojLC5a8Tw7t076tixI+3fv19uJ6h///50584d6tGjBzk5OdHatWupuLi4xnoDBgxg\nAZ7RJAkLC6OtW7eyqbvrIQ0+yOvr69ORI0fo119/pY8//lhuc860aNGCfvrpJ7p27Rr9888/1KtX\nL/rvv//kYluRsGkMGMqgXbt2dOLECerYsSNt27aNBft6RIMP8kSlWQA3b96kQYMGCeec4fP5crFt\nbW1Nly5dom+++YY8PT3p+++/p7y8PKlshIaG0vz58yk/P18uPlVH+SBfVcBX9I8A+5Fpmjg5OdHp\n06fp6NGj9M8//wiDvSR3wAzF0iiCPFFpSuS8efPo5s2bdPPmTXr58qXcbHM4HJo0aRLdvXuX3r17\nR3Z2dlI9YO3fvz+lpqaSvb09nT17Vm5+1QQL8oy6plevXsJgHxcXJ/JGeXZ2Nr179469XVvHNIoH\nr3XNxYsXyd/fn7p27Urr168nU1NTieqdO3eOpk6dShoaGtSxY0dq06YNjRs3jlxcXKqdXqEmKr49\nGxhY+ndyMtGePaJly2a2lMSmLJk6ktqvDezBa8MiNDSUfv/9d4qPjydVVVXS1tammTNn0pw5c9jb\ntOVQlL6a1LQG8mLgwIEUHx9PK1asIEdHR1q6dCn5+/tX+Wp4GQKBgE6ePEk8Ho+0tLTo0aNH1K9f\nP5oyZQq1a9eOQkNDSVNTUyZ/yk+DkJz8YXtISOnLUWXbLCwkn7BMmiDPJkSTPwBo6NChNGLECJo+\nfTq1bNlS2S7JxNKlS+ngwYO0evVqGj58ODVv3pwOHDhAf/31F50/f55OnjwpcaCPi4ujdevWUVhY\nGBUUFJCtrS3NmDGD/Pz86KOPPlLwkTRgFJF8ryCztUIgECAoKAjJyclytXv37l24uLigd+/e1b4k\nEhQUBBcXF2RnZwMAioqKEBEBpKenw8vLC8OGDQMAlJSU4OzZsxg3bhw8PDwwbtw4nDt3DiUlJRL5\nU36KhIpTHFTcL6kdaaiLqY2Vpa+6bvf27dsYOXIkTE1NsWHDhgY1HTCfz0doaCiMjIyQlpYm3B4R\nAUyYMAHm5uaws7PDzz//LNaGQCDAxo0b0b59ezRr1gxEBGtraxw4cABcLhfnz5/HwIEDMWjQoAZ1\nbsShKH01mSBfXFyM5cuXQ09PD8HBwXKZc6aMkpISbNu2Dfr6+ggICEB+fr7I/ry8POjp6Yn8wFy8\neBGWliHQ1NRE27ZtweFwMHr0aPTv3x8ODg7YsmULzp8/j82bN8Pe3h6DBw9GXl6eWB/4fD72798P\nB4fZMDQ0hKWlJdq2fYxnz56JlKsuCEdElO4PDASIPvwtzVu0LMjLn4Y09zufz8fKlSvRtm1bqKur\nQ1dXF6ampli2bBmKioqE+rh+/Tp69+4NFRUVbN26FUVFRSJ2srOz0aFDB6ioqMDDwwMtW7ZE7969\noa6uDkNDQ/j57QVQel1/9tlnWLRoUR0fqfxhQV5OPHjwAAMGDEC3bt1w69Ytudp+9eoVvvjiC7Rv\n3x7nzp0Tbj9+/DgGDRok/L5v3z6YmJjgs8/ihRM/zZw5E7q6umjZsiUePnyIkpISYXDl8/mYMGEC\nxo8fX2W7RUVFGD58OPr06YOlSy/j++9zMXNmOoiA1q3XYNKkZKEtSQO2rMG6LqZVaGpBvoz6HuzL\nAu7gwYNx584d2NnZ4c6dO0hISMCwYcMwbNgwLF5cLFJHQ0MDAwYMwBdffCHcJhAI4Oj4/+2deXxM\n1/vHTyYZ2ZNJMpNJTDZJRISQkBAJEgRJLSGofa+opf1StBoiqKWxtmqpr6WxL7XzQ0sbWy0tpaVK\nqW9RFWvULjOZ9++PyJURNEgay32/Xl4vM3Pn3HMn537uc57znOepjEql4sKFC4wdO5b4+DQyMuDg\nwYNotVpsbMaxfft2AH777TdcXV1f+iRqssgXIXk5Z7RarYkYFxUbN27Ex8eHdu3aceHCBRYuXEjb\ntm0BWLPmGpaWYyURTk3NrebUuvUMzM3NpWmphYUF5cotkapZ3bhxAycnJ06fPl3gfMnJyTRp0sQk\nk2aeVb5lyxY0Go3kJiosL3JFqddV5PPYv38/jRs3fuHEftGiRVSvXl2yykNDQ5k69QgZGZCSkoOX\n11yTGeLWrQZsbGy4du2aySx127Zt2NjYsGTJEgAaNWpE69a/SmNy7dq1ODpOpmXLltJ3goKCOHTo\n0L9zocWELPLFwKVLlwpME/PIyclh48aNDBo0iIEDB7Jq1aqncvHcvHmTgQMHotFoGDJkCH5+fuTk\n5DBx4kQ6dOgAPBDS5ORs7OzsUKlUrFu3DktLS1xcXKhceTVubm7MnTsXgPbt2/Of//yHLVu2cOnS\nJQDu3LmDWq3mxIkTJufPu5EAWrRowdSpUwvdd3ixE5297iKfxw8//CCJ/dSpU0tc7GvVqsXq1aul\n18nJyVSvvkl6vWnTJtzdZ0qv16xZQ40aNQq00717dxQKBbdv38ZoNNK4cWPefPMonTvnjumUlByE\nAGvrNMmdWL58eX766afiu7h/AVnk/0V++eUXypUrR2hoKKNGjWLs2LFERUXh5eXFvn37nqqtgwcP\nEh4ejq2tLZMnT6ZTp06SaOeJcNWq61EoFPTp04eMDChTZh7Vqh1FCNDpZmFpmTtdtbS0xMfHh5iY\nGBwdHencuTNbtmyhcuXKBc6b35e+ePFiE6vnZUcWeVO+//77F0LsHR0duXLlivT69OnTWFunsWfP\nHgBu376NQjESgIsXLxIYGMjy5csLtJOQkIC5uTlffnkJR8fVqNVHECJ3nSg6Onfm6+S0FmdnZwCO\nHj2Km5vbYw22lwVZ5P8lMjMzcXV1Zc6cORiNRpPP1qxZg0aj4eTJk0/VpsFgoF+/fpiZmeHt7c34\n8ePJyIBOnYy0b38CIaBSpVWULv1fMjKgevXqdO78P1JScli7di2enp6UK1eOihUrsnnzZgAuX75M\nnz598PHxoUKFCsDjF06HDfuWxMTEJ17ziBEjKF++PFqtloiICGbPnv3C+jhlkX80+dMBl4TYu7m5\ncerUqQLj0No6jWrVNjJmzG6srOJIS0vDw8ODYcOGPbKdvn374uvry6JFi/juu++IiYlBiOFERW3h\n3r173L17F6VytOSibNSoEakvsn+xkMgi/y8xfPhwfH19qV69+iOnf8nJyfTt2/eZ2l63bh329vYI\nIahcuTJ+fn74+flhazseg8GAj48Pc+fORa1WM2SIntTUXD9nhQoVCA4OxtfX974b6TZDhw5Fq9Vi\naWkptbds2TLpXPnHfMeOHRk/fvwj+3T48GFKly5NUlIS33//PefOnWPDhg3Uq1ePqKioZ86RX5zI\nIv9k8ov9v1nVqWfPnnz00Ucm76Wm5lr0ycnJ6HQ6VCoVnTt3fuKM+MCBA2g0Gvz9/bl8+TIA9et/\nh0IxEpVKRZs2bbCwiOXjjz8mMjKSxo0bv/RWPMgiX+zs37+f7t27Y2lpib+/Pw0aNMDFxYW9lStj\nqFkT4uMhK4s//vhDmiY+C3q9Ho1Gg52dHQkJCezduxdHx8kApKXtQ6EYSUDAYoTInZZ6es6hWrX3\nUSqVZGRkcPPmTXS6WbRs2ZKjR4/y448/ShEKAQEBjByZOx0eNszIjh07aNeuHaVKlWLMmDFcvXq1\nQF98fX1ZsGCB9F6eiycnJ4du3brRrVu3Z77W4kIW+cLxb4v9L7/8gkaj4YcffpDeyzM2fvrpJ1xd\nXTlw4ECh2urUqRNlypTB29ubqVOnMnfuKSZM2I+Pjw9CCCwtLYmMjGT+/PlFGg5dksgiX4ykpqbi\n6upKxYoVEUKg0+mwtLTEwsKCH2xtkRyCrVphMBgwMzMr4Mp5Gk6cOIG3tze+vr6oVCpsbRsxYMAA\nypUrR8OGDYmLi8PK6mMqVKiAmZkZbm5u9OnTB4AhQ4ZQrtwSvLy8WLJkCTk5OSgUdYmIiKB+/fq4\nuLiwZcsWAgPfRqPRYGNjQ8+ePWnbti0qlYo5c+ZI/Vi9enWBha/8M4DLly+jUqmkRd4XhddF5PV6\n/XONszz27dv3r4n92rVrcXFxISkpiU2bNpGWto/evXvj7OxsMtP8J7Kzs2ndujXm5uYIIaR/FhYW\nREZG0q9fP6ZOnUqvXr3QaDSUKlWK8uXLM2nSpCfuJ3lazp07x6hRo+jYsSO9evXim2++KZK/yaOQ\nRb6YWLp0KWXKlEGtVjN+/HhCQkLYsiXX95eSksLG+wJ/PTAQsrI4dOgQXl5ez33eGzduMHPmTMLD\nw1EqlSiVSmbPni0NoAEDbvDTTz/h4eGBo6Mja9ZcY+hQA7a24xECunU7g0YzjSpV3sPc/CNu3brF\npEmTcHFxkW6GDh06cPDgQck6P3bsGJ6enqxduxaAd999lwkTJpj062HXZlxcnHT8i8LrIvIzZ84k\nIiKCzZs3F4mwfP/997zxxht4enoyffr0YhP78+fP89FHH1GvXj3q1q1Lamoqf/7551O1sWzZMrRa\nLVOmTCE9PZ3k5GRcXFxwd3fHxcWFzp07Y2FhgaOjI+vXr+fIkSN88MEHNG7cmPDw8EeW8DQYDGRm\nZposDj+JcePG4eTkRK9evUhPT2f8+PFUrFiRiIgILly48FTXUxhkkX8eevTIXZa/73LJT3h4OIGB\ngcyePRuAadOm0ahRI+mm6tikCSsUCm7eH6SdO3dmxIgRRdq9e/fuUbVqVczNzUlMTOT48ePMmnWC\ngQMHYm1tTZMmTYBc36ZOp5OEODs7m7i4OMzNPyItLY3s7Gw+/vhjbG1tTUQhv3CvX7+esLAwIHeB\n65NPPnniTtcmTZqYhMW9CLwuIm8wGFi6dCnly5cvUrHft28f8fHxeHh4FKvYPytXr15FpVJJa2L/\n+9//UKmaMXv2bEaMGEFAQABmZnUYMmQIixcvRqvVMnHiAWJjYylTpgy1a9ema9euUnt37txh1KhR\neHh4oFarsbOzo1q1ak+cWaSnpxMQEMDZs2el9zIycvfYfPDBB1SrVq3QqUYKiyzyz0N0tInLJY/L\nly9jZ2eHTqeT/HrZXbuy386OI15e/H36NNu2bUOlUjF9+nSGDh1KQECA5Ns+d+4cH3/8Mb1792bY\nsGEcO3bsmbtoNBqZNWsWGo0GpVKJp6cngwYNYu/evWi1WlauXMn58+dxdnZm2DAjGRm51ryNzTiE\ngMDApbz33nVCQvpRtmxZk7bzi7zBYMDd3Z2TJ0+ycOFC6tev/9hjr1+/jpOT01NbYcXN6yLyeRgM\nBpYsWSKJfWELzf8Te/fuJT4+vsgs+5s3b7Jx40ZWrlz5XPfC5MmTTXZ39+/fH1/f+Tg5OdG7d2+6\ndeuGEMOpUaMG/v7+dO7cmaioLQDs3LmT6OhoFAoFn3zyCdevXyc6OpqEhAQOHjwI5P6e69atIzAw\n8JERPjk5OZQtW5Zdu3aZvJ93bxiNRkJCQop8I6Us8s9DfHyuwIeFmVjyf/31Fw4ODiZbqvM/EM4L\nQaOoKKysrLCysqJp06acP38eo9FISkoKVlZWVKlShUmTJjFo0CC0Wi0dOnR4rpvFaDSybNky3N3d\nefvtt8nKyuLAgQN4eHgQGxuLp6cnLVp8RvPmzXFxcWHTpk0mwqzT6ejSpcsTrfO8lA53797Fzc2N\nr7/+Wvp+/rYGDx5MixYtnvlaiovXTeTzMBgMRWbN5ye/2M+YMeOpx69er2fIkCE4OzsTExNDQkIC\nbm5u1KlTh6NHjz51fzp06EB6err0WqVSYW8/UcrD1L17d+ztV7J161ZmzJiBh4cHDg6TTNqoUqUK\nYWFhDBw4kKZNm5pY3Xnuy4sXL6LT6UwWigF+/PFHypYtW+B3zn9vTJkypciDEmSRfx6ysnIt+Idc\nNQaDAWdnZ6Kioh68mfdAuP9vt6cn3t7e9O7dWzpk3LhxVKlShR07dhAcHIyVlRV+fn60bNmSqKgo\nOnbsWARdzqJnz56ULl2aZcuWcefOHRYuXEjt2rVxdHRk3Lhx3LhxA3gw+L7++musrKz48MMPTdp6\n2DrPywkCsGPHDjQaDSNHjuT8+fN8+62Rn376iU6dOhEYGFgsvsfn5XUV+eLmWcTeaDTSqVMnYmNj\nTVJuHDt2jG7duuHo6Mh3330HwB9//MHgwYMJDQ0lODiYLl268P333xdos2vXrkyfPl0yVIQYjhCQ\nmHidKlWuU6bMdoSAhg3PEx0N7u5fFTBk6tSpw8aNG/H09JQs+Dzy3w9jx441ce1AblqFmjVrAo/f\ne5KS8k2RbzCURb6oeMg/P3jwYJRK5QOXRFYWuLmBEGSHhFDe3R2dTifF9d6+fRu1Ws2GDRsoXbo0\n7dq1IzU1VdpE5Ofnh6WlJcePHy+S7u7atYugoCDeeOMNyZJJSUmhdOnSjB49mr179zJhwn66deuG\nRqNh1qxZeHh4cP36dXJychg6dCj9+j2Y3k+cOJHmzZubnOPYsWMkJSVhZ2eHQqHA09OTESNGFAi5\nfFGQRb4gn3/+OV999VWRWPlPI/Z79uzB19dXyrx67tw5IiIGo1aradmyJUFBQSiVSmJiYnB2dqZ/\n//7s2bOHgwcPkpaWhk6nk8J+81i+fDnR9/Nknz17FoVCgVo9FTc3NypVqoSVlRVCZHD48GEgd29L\nqVJjpO9fuHABlUrFsWPHUKvVBfo8cOBNKc/TgQMHCAkJMfn8zJkzuLi4cPv2bZP38z8cBg4cyODB\ng5/8Qz4lssgXFQ/552/fvo1Op0Or1UrTNsPly5yrWZPwsmVZq9Xyo4MDxrg4yMpi7dq1REdH4+np\nyeLFi3Pb7NEDfVQUv3h706d9e7RaLZUqVWLp0qWPTCj2tNy7d4/Ro0fj4uLC+PHj0ev17N+/n7fe\neouwsDCioqIYM2aMZHX36NGD2rVrc/z4cYYPH46DQ1MmTJjAjBkzcHV1fewU2mg0vhQxx7LIF+TL\nL78kMDCQyMhIvv766yIT+7i4OLy8vPj8888fueGoe/fu0ka7y5cv4+/vT3R0hiSQWVlZ2NnZoVQq\ncXBwoF69enTv3p3PPjuM0WgkMzOTsmXLsmrVKqnN7OxsypQpQ3p6OqNGjbpveMxh61YDQ4bo8fFJ\nRwiwtBxL794XqVChD05OnwK5/vSuXbuSlJTE1atXsbe3x2AwFLDIVapPaNr0RyZN+pHw8PAC1xUf\nH89nn31m8l6eyF+6dAm1Wv3UO9//CVnki4IePcDJKfevHBIiuW/+/vtvKlSogIWFBfb29tja2uLr\n60tAQECBOPkvvviCqKgoGjZs+KDdfA+O3R4eWFpaYm5uTrNmzXB2diYxMbFI3B4nTpygXr16hISE\nPHKam4fBYGDEiBGo1Wpq1apF3bp1USqV2NnZPTJXyMuGLPKPxmAwsGjRIknst2zZ8sjjLl26xKRJ\nk+jduzfJycn/mL1xz549jxX7OnXqSOdJSUnhrbfeMrF416y5hoWFBS4uLpQvX55+/foxfvx4nJw+\npU2bNmRnZ/Pll18SEhLCggULWLNmDbdu3eLo0aPodDosLCywtrbGza0N9evXx8urE61atSIuLhM7\nOzvs7e3RaDTUrTuSLVu20LBhQyIjIyVXZkREBOvWrTO5ntRU2L59O3Xr1sXBwYGmTZuaZHCF3I1d\nrq6uJgvSGRlw6NAhQkJCiiV/vSzyRUF+K75ZM5OPjEYjGRkZtGrVisjISBITE1m5ciU5DRuaLNpu\n27YNR0dHvvjiiwdf9vDIjaVXKBjetSstW7bE3d2dH374gZs3bzJ48GCCgoKKJCrCaDSyYMECtFot\n77777hPTDty+fZvNmzezatUqjh49yty5c3F1dWXbtm3P3Y+SRBb5J5Mn9oMGDSrw2cSJE1GpVHTp\n0oUpU6aQnJyMh4cHTZs2lYTxcTxK7BMTE5k3bx4ZGWBnN4HevS+a+K612k2YmZmxfv16/u///o/y\n5XsBuZlX4+Pjad26tbTpr1WrVtSrVw9nZ2dGjhzJqFGjCAsLw9XVFWdnZ3Q6nZSzJiBgMfb29igU\nChQKBba2toSGhjJ9+nQTN8vy5cvx9/dn6dKlfPXVV1y5ckV6CO3YsQNHR0ciIyMJDQ0tMPs5fPgw\nsbGxuLq6EhsbS0hICDqdjilTphTLhihZ5IuCx0TZPJGHFm1zcnKwtbUlJSXlwTFRUdLD4+Ybb+Do\n6EjVqlWlBaejR48SG5uba+Nx5OTkPNXAuXz5Ml27dsXDw+Op4tgvXrz4UrhknoQs8rkujZ07d/LV\nV18VuqTlnDlzCAgI4MyZMwXa6ty5M40aNSpUO3v27KFhw4Z4eXmRlJREVFQU9+7dQ6FQYDQaJRHd\nv38/1tZpWFtbc/jwYZYvv4hS+Z3kMmnT5hhCDCc5+Wvs7e3JysqSosu0Wi329vbExMSwefNmfHx8\nSElJoXfvi6xYsYIRI7azZcsWVCqVVDrzYa5du0bXrl2xtLTE0tISX19f7OzsiIgYzFtvvYVarZYi\nyx7+TfLz+++/s3nzZnbt2lWs944s8kXBY6JsCs39RdtDpUvjZmXFpk2bcoX5/sPjoo8PVf38GDly\nJI6OjtLC5TfffINGo8HBwcFkMVOv1/P5559TqVIlFAoFpUqVolmzZlLFm8Lw7bffEhAQQPPmzV+4\nePbi4nUWeaPRyIQJE3B3d6dKlSrExsbi4uJCfHz8Ixf7jUYje/fupVevXlhbWxMfH8+iRYsKGBR6\nvR5vb+8C4YRPYvfu3TRo0AClUkmjRo2wtbXlr7/+kvK+V626HiHAxyedBg12M23aL2g004DczyMj\nIwkJCWHMmDH4+flx48YNqlV7X8rBVKpUKaKionB1fZPAwKUEBS1HCPDwmI2DwyRsbN5ApVIxY8aM\nAn27efMmYWFh9OzZk0uXLrF/f25wQsWKFVGr1eh0ugI1GEoaWeT/TR63Qzafu2eFuTl+fn4EBgbS\nsUkTlgtBOa2W//73v/Tv35+MsmVN2rhw4QIKhQJ3d3eWLl3KvXv3aNy4MdHR0WzZsgWDwcDff//N\nzJkz0el0zJw58zGdK8idO3cYNmwYarWaDh06UL16dWxsbHBxcaFr166FqpiTV3LwZeB1Fvl+/fpR\nrVo1KbIEct1ykyZNws3NzUS4bt++TbNmzdBoNDg7O6NQKHB2dkYIgbW1NSfr1sVYu7Y0RocPH857\n77331H1av349jo6OmJmZUa5cOfr0+ZI333wTKysrKlT4kqlTj+DgMImKFVdIrpzw8Fuo1S1p+oks\nbgAAF1BJREFU3bo1wcHBTJgwgdatW1Op0oPiPDExw1Gr1dy6dYu4uDj+85//0Lbtcdzc3KhQoQI9\ne/bEysoKnU5H3bp1yczMlPo0efJkEhISTB5mefHxRqORuLg4pk+f/tTXWpzIIv9PPCF1wVOT33ev\n0eS22amTyaLtwqlT0el0DB8+nEWLFuHm5sbatWtJSkoiMDCQ7MhIkwXbkydPotVq2b17NxUrVqRS\npUo0bNjwkdO/kydPolarnyoMMzs7W1pICggIYNeuXZw9e5aPP/4YV1dXVqxY8cTvJyQkUKNGDRPx\neFF5XUX+8OHDuLu7k/XQ+D548CDvv/8+4eHhVKhQQXI9dOrUiYoVK0qWcXx8PJAbMuvi4kJGvv0g\nxlatmDVr1jNv8Ml106SiUCiwtramXbt2fPrppwQELMZoNNKoUSMsLCxo3vwnDAYDaWn7CAsLw9ra\nmho1akgZLIcMeXA/JCQcxMXFhdTUVC5cuICdnR3W1mls2LCBGzduULt2bbp0SUev15OcnEzlypWl\nHPpBQUHs3LnTpI/5F4S3bNlCaGjoM11rcSGL/D/xmNQFz0Se797OzlTsH1q03bJlC7Gxsdja2mJr\na4uVlRUDBgzITYD0kP///fffp3///kCu5a3Vap9YSHzw4MHS8YVh9OjRNGzYkLt370rpEQYNGsTN\nmzc5ePAgzs7OnDt37rHfz8nJYcaMGajVaj788MMCMcIvEq+ryL/77rsmxTFu3bpF8+bNKV26NO3b\nt+ftt9+mVKlSODo60rt3bxwdHdFoNJw/f57Dhw/j5eXF6dOnGTZsGFWqVOH/7o/nQ6VK8emIEfTt\n21fKy3T58mWWLVtGenp6odMDQ64LR6PRoFKpUKlUWFjEEhAQQGBgIJMmTUKnm4WtrS329vaYm5vj\n4eHB119nExu7i/Dw/zNZtA0JOUT37t2pXLkyUVFRODk5Ubnyfxg9ejSenp706NGDYcNyLXWj0Uj9\n+vWlnbKlSpV6Ypx7VlYWDg4Oz/JnKDZkkX8ceRa8Wv30i6qPI893Hxv7oM38/3+o/WvXrnHgwAHc\n3NxYuHChaRtZWaxcuRKtVistkG3atAkHBwemTZvGL7/8Ynod92ciu3btonr16oXqrl6vR6fTmRQ5\nyczMpF27dpQpU4ZNmzbRq1cvhg8f/o9t/fXXX7Rq1Qo/P78CltCLwusq8m+88Qbr16+XXn/l7c1e\nKyu+trAgvkYNYmJiMDc3p0GDBri7u+Pm5saQIUNyD+7Rg71WVnxlbs6A7t3Zvn078TVqsFwIymo0\nDBw4ECcnJ3777Tf69u2Lo6MjTZo0oX379nh5eREeHl7oQtl5YZFt2rRBo9GgUChISkri9u3bZGTA\nn3/+KcW/z5o1C4Bhw4aRmppaoI5r9+5nGTrUwNChWzEzMyMgIICePXtKBlJ+4V61ahX16tUDwNXV\n9ZFVqvL+P2/eaTw9PZ/9j1EMyCL/OPJb8B4ezy/w+cm/UFuIRduff/4ZPz8/wsPDGTNmDGPHjqVG\njRp4e3uzf/9+Tp48SWRkJKVLl8bKyoru3bvj5uZGbGwsdyMiTGYi27dvf2SR40dx4sQJfHx8TN+8\n/9C4GBZGZW9vYmJiTNM3/AMbNmx46nq2/xavq8i3adNGypZ68OBBtuVzt9CqFTk5Obi7u9O+fXs0\nGo0Uughwp3p16djbKhU5V67Qp08f3NzcSE9Px9LSkhYtWtCsWTMSEhKYMmUK1apVQ6FQoFQqqVy5\nMiqV6ply0UyePBmVSoVCoUCn0+Ho6EhCQgLjxo3Dx8eH3377jUWLFtGgQQNSU3M3/3Xq1Al//4WS\nT/3cuXNYWFjw7bffPla4Z806QXBwMJCbYXXo0KEm/Xh4x+qAAQOe+lqKE1nkH8ezhEUWIwaDgfXr\n10uDaPXq1ej1ejIzM/H09OSTTz5Br9cTGBjIjh07uHfvHqmpqWy7v+nqoFLJhoULeffddx9YYf/A\nI0U+38NPn5jIm2++iVKpZObMmS/NAuvjeF1FftWqVURFRWE0GklISOAbS8sHY79TJ64EB7PT3p6c\nK1eoUaMGCoVC2np/zNfXJCfTBltbSXjNzc2xsrLC0tIStVpNs2bNqFatGpMnT6ZmzZqsX7+e2bNn\no1KpqFSpEpmZmVIag6dhxYoVRERE4OnpyaxZs8jOzubzzz9HpVLRokULbG1tqV9/NG5ubjRv3pwP\nP3yw6Wr16tUoFArOnz9v0mZ+4V6yZAlxcXFA7j2h0WhMMkXmHbthwwZcXV2lNCEvCrLIP47nDYv8\nl/jwww9NkpxNnz6diIgIqYpN54QEjlWuzM716/H29kapVBbaktbr9Xh4eJjWpH3o4derVy/efvtt\nqlevTlRUFEeOHCnS6/s3eV1FXq/XU6lSJUaOHIm7uzt92reXxn5+S51Wrfjuu+8QQhAYGAhAZFAQ\nf983JLLKluVw1apsE4JNZmZsXLyY06dPo1Ao8PHxoVSpUvz444/o9Xrmz5+Pv78/1atXp169eggh\ncHBwwMbGhubNmz/TbG/Xrl3Exsbi4+PDrFmzuHTpEtOnT6dx48bY2dnx2WefSakIsrOzWbBgAWq1\nmvj4+AK1HPKEOycnh6ioKJMc8Tt37kSr1RIXF8eMGTPo128N9evXx93dnT179jzrn6HYkEX+Jcfd\n3Z1ff/1Vep2Tk0P37t2pUKEC8+bNY8WKFQQEBJCamoparebNN99ErVYzZcoUDAbDP7aft/Aqbc/O\n9/DLv/BqMBiYNm0aarWaoUOHStEIheWDDz5g0aJFT/WdouZ1EPnMzExGjRpF9erVCQkJoXPnzuzZ\ns4dz585RpUoVFAoFwcHBpKen06NHD86ameUKvIMD/PEHBoMBIQRmZmaEhoaiUChoXqcOO9zccLe2\nNomsWSYEQUFBUnK6Dz74gLJly5LTvTtER3MrOpoyTk73F1ItaNiwITdu3GD69OloNBo2bNjwTNe4\nc+dOqdDH7Nmzyc7OZtOmTYSHh+Ph4UFkZCRarZaYmBh2794tRah98cUX0mw0IyO3ylpSUhKRkZEF\n0hPcvn2befPm0b17d9566y0WLFjwwhVJyUMW+ZcYo9GIEMLUTdKjB8batcmsWpWW97dOKxQK+vTp\nIz0Mfv31V5KSkgoM3EeRnZ1NQkICkZGRrFu3jr///pszZ84wduzYR4ZQnjt3jhYtWlC2bFm++eab\nQl/LmTNnCr3Dsrh41UV+x44dODs7S7HcP/zwAxMmTMDLy4tBgwZJhV8sLCxITExkzJgx3AsPN7Hk\nFyxYgEKhICIiAjMzMxQKBTY2NlIKgLzImlNqNcGenjRv3hyFQoEQghs3bhAWFsblihWlNq/b2nLz\nzz+xsrLC19dX6uvevXtxcXF5ZF3Vy5cv8/nnnzNy5Ejmzp3L33///djrrVevnonYHzt2jB07dvD7\n77+bHPvzzz9TpUoVfH19SUpKon379jg5OdG6desiK6ZSUsgi/5Lj6urKtTfffBBBky8VQt70ukKF\nCs91Dr1ez7x584iIiMDW1hYXFxe6detm6sZ5iHXr1uHl5UWnTp1euILdj+NVFvnNmzejVCpxdnam\nfv36BAUF4efnx7x587hy5QqVKlXiiy++YMiQIdSsWTM3id4PP0juOWNYGOsXLMDW1paaNWtSunRp\nMjMzJVdLxYoVUSqV1AoO5mxkJHVCQ/nzzz+ZrVBw0NGR/xOC95OSGDt2LEd9fEz8+P+rVo2IiIgC\n47Rp06bSgjDkzlKHDBmCSqWiXbt2JCcnk5iYiJOTExMnTnxs+o78lv2cOXNMjBuj0cj27dtJSkoi\nMTGRNm3akJKSwuzZs4sk0+uLgCzyLzkDBw7khE734Ka5n7M+z2feoUMH0tLSSqRvN27coH///ri6\nupKenl5s1eiLildV5Hfv3o2NjQ21atWSNskZjUZ2796Nv78/n3zyCVu3biU4OJizZ8+i1Wrp27cv\nXl5e1AoOZoe7O5W8vAgICMDR0ZHOnTtLOZbq1atH06ZNcXNzIygoiGbNmmEwGHBzcyMmJoa9VlbS\n2FxuZsbIkSMZ/Pbb3FGpQAhulC+Pv1pNp06d2BUUZBLu+9lnuZXKMu5vKf3www+JiIgw2YEKubVa\ny5cvz5QpU574O+S37OfMmcOVK1eoX78+gYGBTJgwgWXLljFs2DBKly5NUlJSodyZLwOyyL/k/Pnn\nn1I0hDEsDP74A1q1wnD5MhMmTMDX17fQRTquXr1KUlJSgUiD52X//v1UqVKFOnXqFFnRk+LgVRR5\no9FI1apVCQoKYtOmTQ8+uB8Keys6Gi8HBy5evIhGo+HMmTN8//336HQ66tWrx4ABA+jTpw916tSR\nEm+1bduWhQtzwxBLlSpFdpcu7Lez4xtLS74Qgl0WFpwRguMaDfr7u7kzvbxwMjPDwsKCZs2a0att\nW1Yrlfi5uDBt2jQ0Gg03w8JMZqHjxo2jadOm+Pn5ERUVha2t7WNTax8/fhy1Wl2ozXZ56YBtbGyo\nWbNmgfWjGzduEBMTU+gotBed4hpfCiHzr6DT6YRu+3bxlaOjqP7332LQ1KniPQ8P4R8WJlauXCm+\n/fZb4eTkVKi2rKyshIuLi6hUqZKYOXOmMBqNRdLHqlWrin379okmTZqIyMhIMWrUKJGdnV0kbcs8\nmUOHDomrV68KlUolbGxsHnzw229CbN8ubLZvF8tVKrFw4UJhY2Mj9Hq9CA8PF7///rvo0qWLyMrK\nEjdv3hRt2rQR//vf/0T9+vWFVqsVv//+u9SU4uRJUfXmTVH33j3RTKkUUQaD8BRCBFy6JCyyssQ5\nhUK4HjokjA4Owmg0Cr1eL7JtbERbc3PRvGtXMXLkSDFhwgRhq9HkNhgWJpg5UyxZskR06dJF9OnT\nR/zyyy/i1q1bwsfHR3Tp0kXc7thRiJgYId54Q4hr10RAQIAIDQ0VGzdu/MffpHbt2uLTTz8V1tbW\nQqlUiqCgIDF37lyh1+uFEELY2dmJhQsXiunTp4sbN24U4V/jFaM4nhzF1OwrgdFoZMeOHYwdO5a0\ntLQnFv/4J37++WciIiKIjIws8pwzp0+fpkmTJpQvX/6F2/laUuOrOM+7bNkyWrRoQd++fR+ksc5f\n5CY0lBljx9K+fXtcXV0fWaXpYQ4cOICXlxd3796lbt26nKtcGYTghJMTd2vVAiG4dt8iz/TyonXD\nhrzzzjsIIUhLS6Nx48ZotVrs7Ozw9vZ+MFbvR24Zr17lnXfewcnJCbVaTdWqVenQoQMDBgxg+PDh\nuLu7s7tUqQLpRnr27Mm0adMK9buMGDFC2rS0fft21ru7s8fSkjPBwWRfvAhAXFwcK1eufMpf/MWj\nuMaXLPIvOTk5OVIoW1EvQBmNRlasWIFOp6NHjx4vTM3XV1HkN2/eTM2aNTly5Aiurq65eYby7+ZO\nSCA5OZmgoKCnqi3aqlUrmjdvzpIlSwgvW5abb7yBn4sLcyZOZK2lJe80bcpaKyschcDGxoYyZcrQ\no0cPkzbu3r1Lo0aNCA0NJT09nZ9++on09HQcHR1RKpX4+/sTExNDw4YNsbe3l+qzApy5H6FjzLdZ\nMSYmhjVr1hSq/4MGDTKtw5DvN9lga8vcuXNp166dlLPmZUYWeZkn8rjwtKLg2rVr9O7dG3d3dxYv\nXlziC7OvosjfvXsXV1dXjh8/TlpaGn5+fpwPDZUW53/esQNra2uCg4MfGa74OO7cuUO3bt1wdnYm\nODgYZ2dnvL29MTMzIy4ujhEjRuDp6UnHjh2pWbOmlOTuYQwGA2vXrpVmd1ZWViQmJvLzzz/j7Ows\n7YDduHEjZmZmbNy4EQDj1atsdnAg435hm59//hmNRlOomQjArFmzaJa/ilu+TX67NmygTp06KJVK\nhgwZIhfDeVy7xdKoLPKvJHv27CE4OJhGjRqVaETDqyjyAOPGjaNq1apcvnyZ1atX07B69dy6BS4u\n2NjYULZs2acS+PycOXOGTz/9lA4dOlC5cmVsbGwoVaoU1tbW2NvbU6NGDdLT0wu1J2P69Ok0b94c\ngPnz59O6dWuTzzd5erLT3Jxb0dGQlUVqairJyckcOXIEX19fKlSoQJ06dQpVHOf69es4OTk9SOT3\n0A73lStX4uvrS0xMDH5+fnzxxRcvrdjLIi/zTOzbt69ILe/s7Gy2bt1aZO09C6+qyBuNRj744ANc\nXFzo378/6enpDB06FG9vb2JjY4t1tvY01KpVS7LU582bx1ZfX5OQSn2+PSA73NyoVq0aPj4+uLq6\nSjlr5s6di6+vL3Xr1v1HsU9PT8fDw4PNmzdLGwrv3bvHvHnzUKvV7Nq1C4Bt27a91GIvi7zMU3Pv\n3j2qVatGTEwMx44dK+nuFBmvqsjncerUKVJSUujYsSPvvPMOu3fvLnEXWX4qVKggbbA7ceIEuyws\nTBdX77tU7gQHs2L2bDw9PUlOTi7gBsov9j179nziOVevXk2lSpXw8fGhVq1aaLVa6tSpw969ewsc\nm5GRIYl9enr6SyP2ssjLPBMGg4FPPvkEFxcXRowY8cLm7XgaXnWRf9Fp1KjRg7oJwPcP1XL488gR\nVimV3LtwgaVLl+Ln5/dE9152djanTp36x/MajUYOHz7Mtm3bCqQ7eBQZGRlER0e/NGJfXOPL7H7j\nRYqZmZkohmZlnoOzZ8+Kvn37it9++00sX75cBAcHl3SXnpmSGl/yuM5l9erV4uOPPxbfffedsLCw\nEOd//VUcDA8XG5o2FUnvvy/mz58vrly5IrRarZg/f77YvHmzCAkJKbH+btu2TYwYMUKcPXtWpKSk\niPbt2wsLC4sS68/jKLbxVRxPjmJqVuY5MRqNrFq16rG7EV8WSmp8yeM6F4PBQIMGDWjTpg0X78eq\nX7lyhWHDhuHk5IQQArVaTf/+/Z8rmd3Nmzdp0aIFO3bsKJJ+Z2RkULt2bfz9/Zk3b94LZ9kX1/iS\nLXmZlw7Zki957ty5IwYMGCCWLFkiIiMjhbW1tdi5c6cICQkRM2fOFD4+Ps99Dr1eLxYsWCBGjRol\nfH19xfDhw0XNmjWfu91t27aJ1NRU8ddff4mUlBTRrl27F8KyL67xJYu8jBBCCECYmZmVdDcKhSzy\nLw5Xr14V27dvF3q9XlSpUkX4+/sX+Tn0er2YP3++GDVqlPD39xfjxo0ToaGhz93uiyb2ssjLFCuN\nGzcWNWvWFAMGDBBKpbKku/NEZJF/PckT+/Lly4vIyMgiaROQxD4zM1N8+umnIj4+vkjaflpkkZcp\nVk6dOiV69+4t/vrrL/Hf//5XRERElHSXHoss8jJFTZ7YK5XKInEJPQuyyMsUO4BYunSpeO+990Ri\nYqIYM2aMcHR0LOluFUAWeZmHuXTpkjhx4kSRWfglQXGNLznVsIyEmZmZaNu2rTh69KjQ6/VixYoV\nJd0lGZlCcerUKdG+fXvRoEEDsXv37pLuzguFbMnLvHTIlrzMo8jOzhbz5s0To0ePFgEBAWL48OEv\nlWUvu2tkZO4ji7zMk8gT+7Fjx4qtW7cKX1/fku5SoZBFXuaFYN26dcLT07NIQtieFVnkZQpDTk6O\nMDc3L+luFBrZJy/zQnDnzh2RmZlZ0t2QkflHHifweeUDXxdkS17mpUO25GWeh/79+4tff/1VpKam\niho1apR0dyRkS15GRkamCEhLSxOJiYmibdu2Ii4uTuzZs6eku1SsyJa8zEuHbMnLFAXZ2dkiPT1d\njB49WoSEhIg1a9aUaGoPeeFVRuY+ssjLFCXZ2dli3759olatWiXaD1nkZWTuI4u8zKuI7JOXkZGR\nkXlqZJGXkZGReYWRRV5GRkbmFUYWeRkZGZlXGFnkZWRkZF5hZJGXkZGReYWRRV5GRkbmFUYWeRkZ\nGZlXGFnkZWRkZF5hZJGXkZGReYWxKI5Go6OjSzTRj8yrTXR0dImdVx7XMsVFcY3rYsldIyMjIyPz\nYiC7a2RkZGReYWSRl5GRkXmFkUVeRkZG5hVGFnkZGRmZVxhZ5GVkZGReYf4fWq6hC/V4legAAAAA\nSUVORK5CYII=\n", "text/plain": [ "<matplotlib.figure.Figure at 0x1095fe290>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from sklearn import svm\n", "import course_utils as bd\n", "import pandas as pd\n", "reload(bd)\n", "\n", "\n", "#Generate noisier data than the last example\n", "m = [[0.25,0.25],[1.25,1.25]]; s = 0.35; n=20\n", "X1=pd.DataFrame(np.random.multivariate_normal([m[0][0],m[0][1]],[[s,0],[0,s]],n), columns=['x1','x2'])\n", "X2=pd.DataFrame(np.random.multivariate_normal([m[1][0],m[1][1]],[[s,0],[0,s]],n), columns=['x1','x2'])\n", "X1['Y'] = -1 * np.ones(X1.shape[0]); X2['Y'] = 1 * np.ones(X2.shape[0])\n", "dat = X1.append(X2, ignore_index=True)\n", "\n", "\n", "#Now plot for different levels of C\n", "\n", "fig = plt.figure()\n", "c_s = [ 1000, 1, 10e-2, 10e-4 ] \n", "for i,c in enumerate(c_s):\n", " my_svm = svm.SVC(kernel='linear', C=c)\n", " my_svm.fit(dat[['x1','x2']], dat['Y'])\n", " w = my_svm.coef_[0]\n", " t = my_svm.intercept_\n", " if t<0:\n", " sign = '-'\n", " else:\n", " sign = '+'\n", " ax = fig.add_subplot(2, 2, i+1)\n", " plt.title('C={} : f=<{},{}> X {} {}'.format(c, round(w[0],1), round(w[1],1), sign,np.abs(round(t,1))))\n", " bd.plotSVM(dat[['x1','x2']], dat['Y'], my_svm)\n", " ax.axes.get_xaxis().set_visible(False)\n", " ax.axes.get_yaxis().set_visible(False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<p>As we increase $C$, we can see that the norm of $W$ increases and the number of support vectors identified shrinks. \n", "</p>" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#SVM vs. Logistic Regression\n", "\n", "<p>Both SVM and Logistic Regression finds linear separating hyper-planes of the form $class(X)=f(W\\cdot X+t)$.\n", "\n", "</p>" ] }, { "cell_type": "code", "execution_count": 56, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "<matplotlib.legend.Legend at 0x10b5f9c10>" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3WdUVNfXBvBnaBZABRQLqAgWelHsDQsWFBsGC3YFkdiN\nGo0FNYq9Cwr5R0WDmteOIlbGjg0VQSIWUNAYJWBB6Zz3ww0jI50pdwb2b61Zyczcsh1gczj3nr0F\njDEGQgghSkeF7wAIIYSUDyVwQghRUpTACSFESVECJ4QQJUUJnBBClBQlcEIIUVISJ3AfHx9YWFjA\nysoKI0eOREZGhjTiIoQQUgKJEnh8fDwCAgIQERGBR48eIScnBwcPHpRWbIQQQoqhJsnONWrUgLq6\nOr5+/QpVVVV8/foVBgYG0oqNEEJIMSQagevq6mLOnDlo1KgRGjRogFq1aqFnz57Sio0QQkgxJErg\nz58/x+bNmxEfH483b94gNTUVf/zxh7RiI4QQUgyJplDu3r2LDh06QE9PDwAwZMgQ3LhxA25ubqJt\nmjZtiufPn0sWJSGEVDImJiZ49uxZsdtINAI3NTVFeHg40tLSwBjDhQsXYG5uLrbN8+fPwRijB2NY\nunQp7zEoyoM+C/os6LMo/lGaga9ECdzGxgZjxoyBvb09rK2tAQAeHh6SHJIQQkgpSTSFAgDz5s3D\nvHnzpBELIYSQMqCVmHLk4ODAdwgKgz6Lb+iz+IY+i7IRMMZk2tBBIBBAxqcghJAKpzS5U+IpFEII\n/3R1dZGSksJ3GKQcdHR0kJycXK59aQROSAVAP2fKq6ivXWm+pjQHTgghSooSOCGEKClK4IQQoqQo\ngRNCKoQFCxZgy5YtfIchJiMjA2ZmZkhKSpLJ8SmBE0Jk6tq1a+jQoQNq1aoFPT09dOrUCdeuXYOW\nlha+fPlSYHs7Ozv4+voiPj4eKioqaNmypdj7SUlJ0NDQQJMmTUSvvX//Hvv27YOnp6fotYsXL8LU\n1BSampro3r07Xr16VWSMWlpa0NbWFj3U1NQwffp0AEB4eDgcHR2hp6cHfX19uLq64u3bt6J9vb29\noa6uLtq3Ro0aiI+PBwBUqVIFEyZMwOrVq8v12ZWE1wROV80Jqdg+ffqE/v37Y8aMGUhJScHr16/h\n7e2NmjVrwtDQEIcPHxbbPioqCjExMRgxYoTotbS0NERHR4ueBwUFwdjYGAKBQPTanj170K9fP1Sp\nUgUAl+RdXFywcuVKpKSkwN7eHsOGDSsyztTUVHz+/BmfP3/G27dvUa1aNbi6ugIAPnz4AE9PT7x8\n+RIvX76EtrY2xo8fL9pXIBBgxIgRov0/ffoEIyMj0fsjRozA3r17kZWVVb4PsRi8JfCc3Bx02t0J\n+yP3UyInpIKKjY2FQCDAsGHDIBAIULVqVfTs2RNWVlYYO3YsAgMDxbYPDAxEv379oKOjI3pt9OjR\n2Lt3r+j5vn37MGbMGLG8ERoaiq5du4qeHz16FJaWlnBxcYGGhga8vb3x8OFDxMbGlhjz4cOHUbdu\nXXTq1AkA0KdPH7i4uEBLSwvVqlXDjz/+iOvXr4u2zys+VRRDQ0Po6Ojg5s2bJZ67rHhL4Koqqtjc\nezPW31iP3vt743kylZwlpKJp0aIFVFVVMW7cOISGhootNho1ahSuXLmCxMREAEBubi4OHDiAsWPH\nih3Dzc0NBw8eBGMMjx8/RmpqKtq2bSu2zaNHj9CiRQvR8+joaNjY2IieV69eHU2bNkVUVFSJMe/d\nuxdjxowp8v0rV67A0tJS9FwgECA4OBh6enqwtLTEzp07C+xjZmaGhw8flnjusuJ1CqW1QWvccb8D\nR2NHtP2tLdZcW4OsHOn/mUFIZScQSOdRVtra2rh27RoEAgHc3d2hr6+PgQMH4t27d2jYsCEcHByw\nb98+ANycdUZGBvr16yd2DENDQ7Ro0QLnz59HYGBgocn1w4cP0NbWFj3/8uULatSoIbZNjRo1kJqa\nWmy8L1++xJUrVwr8EskTGRmJFStWYN26daLXXF1d8ddffyEpKQkBAQFYvnx5gd7A2tra+PDhQ7Hn\nLg/eL2Kqq6pjbse5uON+B2HxYbAPsMetxFt8h0VIhcKYdB7lYWpqit27dyMhIQFRUVF48+YNZs6c\nCQAYO3asKIHv27cPI0aMgKqqqtj+AoEAY8aMwe7du3Hw4EGMHj26wJSFjo4OPn/+LHqupaWFT58+\niW3z8eNHsSRfmH379qFz585o3LhxgfeePXsGJycnbN26FR07dhS9bmZmhnr16kEgEKB9+/aYMWNG\ngbn9z58/i00LSQvvCTxPE50mOON2BvM7zsegQ4MwLWQaPmV8KnlHQojSaNGiBcaOHSuayhg8eDAS\nExMRFhaGY8eOFTnyHTJkCEJCQmBiYgJDQ8MC71tbW+PJkyei5xYWFmJTFl++fMHz589hYWFRbHyB\ngYGFxvDy5Us4OjpiyZIlYh3HSismJkZsSkdqmIyV5xRJX5LYhOMTmOFGQ3Y85rgMoiKkYpHDj3K5\n/PXXX2zDhg0sMTGRMcbYq1evWIcOHZiHh4dom/Hjx7PGjRszS0tLsX3j4uKYQCBgOTk5jDHG7t27\nx168eMEYY+z8+fPMyMhItO3GjRvFjvn+/XtWs2ZNduTIEZaWlsbmzp3L2rdvX2ys169fZ5qamiw1\nNVXs9cTERGZsbMzWr19f6H7Hjx9nycnJLDc3l926dYs1aNCABQYGiu2vp6fHMjMzC92/qK9dab6m\nCpnA81x6cYk129qMDTk0hCV+TJRiVIRULIqawF+/fs1cXV2ZgYEB09TUZAYGBszT05N9/vxZtI1Q\nKGQCgYCtXbtWbN+4uDimoqIiSuD5nT9/njVp0kT0PCkpiRkaGrK0tDTRaxcuXGCmpqasWrVqrFu3\nbuzly5ei91auXMn69u0rdszJkyezMWPGFDiXt7c3EwgETEtLS/TQ1tYWvT9ixAimp6fHtLS0mKmp\nKdu2bZvY/mvXrmVz5swp8jOSJIErfDXC9Ox0rLq6Cn53/bDMYRk87T2hIlCYmR9CFAJVIwR++eUX\n6OvrY8aMGXyHIpKRkQFbW1tcvXoVtWvXLnQbSaoRKnwCz/P4/WN4BHsgl+XC39kflvqWJe9ESCVB\nCVx5VYpysuZ1zHFl/BWMtRmLbnu7YdGlRUjPTuc7LEII4Y3SJHAAUBGoYLL9ZER6RiL231hY+1kj\nLC6M77AIIYQXSjOFUpjgJ8GYemYqujfpjvWO66FXXU8m5yFE0dEUivKqFFMohXFu4YyoKVGoWaUm\nLP0s8UfkH/RNTAipNCRO4B8+fMDQoUNhZmYGc3NzhIeHl2q/3FygXz8gMBDIySn/+bWraGNzn804\nOfwk1t1Yhz5/9MGLlBflPyAhhCgJiRP4jBkz4OTkhJiYGERGRsLMzKx0J1YB5s8HfvsNsLQEDh3i\nknp55dVV6dmkJ9oEtMHa62uprgohpEKTaA7848ePsLOzw4sXRY94S5rHYQy4cAFYtAhITweWLwcG\nDChf4Zw8L1JeYMrpKfgn9R/4O/ujjUGb8h+MECVAc+DKi7c58Li4ONSpUwfjx49Hy5Yt4e7ujq9f\nv5bpGAIB4OgIhIcDv/4KLF0KtGkDhIaWv3iOsY4xQt1CMbfDXAw4MAAzzszA54zPJe9ICFEYI0aM\nwIkTJ/gOQ8w///wDc3NzZGZm8h0KAAkTeHZ2NiIiIuDl5YWIiAhoamoW2jrI29tb9BAKhYUeSyAA\nnJ2BiAhg7lxg1iygc2egiM1LJBAI4GbthmivaHzO/AwLXwucfHKyfAcjhJSLkZERLl68KPaaUCiE\nioqKqP1Y8+bN4e/vL7ZNZGQkIiMjMXDgQNFrQUFBaNy4MbS0tDB48GCx2uL5vXr1Sqw9mra2NlRU\nVLBp0yYAwOnTp9GpUyfo6Oigfv36cHd3FyszO27cOFSpUkWsRVreSLhu3bro1q1bgXilQSgUiuXK\nUilxsX0x/v77b7GCMlevXmX9+vUT26a8p8jOZiwwkDFjY8Z69GDs5k1JImXs4ouLrNnWZszlkAt7\n/em1ZAcjRMFI+KMsM0ZGRuzixYtir4WFhTFDQ0PR85CQEKaqqsqioqJEr3l5ebFVq1aJnkdFRTFt\nbW129epVlpqaykaOHMmGDx9eqhji4uKYqqqqqBZKUFAQO3v2LEtLS2MpKSmsb9++zNPTU7T9uHHj\n2OLFi4s83vXr1wsU3pJEUV+70nxNJRqB16tXDw0bNhS1Kbpw4UKJ5RpLS1UVGD0a+OsvYNgwwNUV\n6N8fuH+/fMfr3qQ7IqdEwqy2GWx22sDvjh9ymQRXTQkhUtG3b1/o6ekhJiZG9Nr3LdL++OMPDBgw\nAJ06dYKmpiZWrFiBo0ePFtoU+Xt79+5F165d0ahRIwDc1EyvXr1QtWpV1KpVC+7u7mIt0oDi+/W2\nadMGL168QEJCQln/qVIn8V0o27Ztg5ubG2xsbBAZGYmFCxdKIy4RdXXA3R14+hTo3Zu79XDoUCBf\nj9NSq6pWFSu6r0DY2DDsi9yHzrs7I/pdOQ5ECJGK3NxcnDx5UnRDBMDV7o6LixNrkfb48WOxetrG\nxsaoUqVKiT0uGWNF1vjOc/nyZbEWaQDg6+sLPT092Nvb4+jRo2LvqampoWnTpnjw4EGp/52yoibp\nAWxsbHDnzh1pxFKsKlWAadOAiRMBX1+ge3fu4ufSpUCzZmU7lqW+Ja5NuIZdd3fBYa8DPFt54pcu\nv6CqWlXZBE8IzwTLJLitKx+2VDp3urx58wY6OjpIS0tDVlYWDh48CBMTEwAQtR7L3z0nNTUVNWvW\nFDtGjRo1xLrwFObatWt49+4dhg4dWuj7eW3abt++LXpt+vTp2LhxI2rWrImzZ89i2LBhqFevHjp0\n6CDaRltbGx8/fizbP1oWpDaRUwRZneLTJ8ZWrGBMT4+xCRMYi4sr33Fef3rNXA65sGZbm7FLLy5J\nNUZC5EUOP8rlUtIceEZGBps1axaztbUV1f1OTU1lAoGAJSUlifYZOHBggXrhWlpaLCIiotjzT5w4\nkY0bN67Q927evMnq1KnDLl0q/ufe09OzQD1va2trFhwcXOx+pVXU1640X1OlXUqvrc3dO/70KVC/\nPtCqFfDjj8CbN2U7TgPtBjjsehjrHNdhzPExmHBiAv79+q9sgiaEiNHQ0MCaNWvw8eNHUW9MTU1N\nmJiYFNsi7fnz58jMzETz5s2LPHZaWhoOHz5c6PTJ/fv3MXDgQOzZswfdunUrU8zZ2dl49uyZbFqk\nlZHSJvA8Ojrc/eN//QVUrw5YWQFz5gDv3pXtOANNByLaKxpaGlqw9LNE0KMgWhhBiBRkZmYiPT1d\n9MjOzhZ7X11dHXPmzMHatWtFrzk5OeHy5cui525ubggODsa1a9fw5csXLF68GC4uLtDU1CzyvMeO\nHYOuri4cHBzEXo+KikKfPn2wfft2ODk5Fdjv8OHDSE1NRW5uLs6dOye6gJrn9u3bMDIyQsOGDcv6\nUUifVP4GKIYcTiHmzRvGpk1jTFeXsYULGUtOLvsxwhPCmbWfNeu1rxd7nvxc+kESImXy/jkrLSMj\nIyYQCMQenTp1Yg0bNhTb7uvXr6x27drsxIkTjDHutkELCwuxbYKCglijRo2YpqYmGzRoEEtJSRG9\n5+npKXYrIGOM9e7dmy1ZsqRATOPHj2eqqqpiLdLy3xbYuXNnVrNmTVajRg1ma2vLDh06JLa/l5dX\ngbZpkijqa1ear6lSl5MtzqtX3Mj86FFg+nRg5kygRo3S75+Vk4WNNzdi3Y11mN9xPma2mwl1VXXZ\nBUyIBCriUno3Nze4urqKLebh27t37+Dg4IAHDx5AQ0NDKsesFC3VyuvZM66+Smgo8NNP3Dx5MX91\nFfA8+Tk8T3vi/Zf3CHAOQGuD1rILlpBy4vvnjJRfpa0HXhpNm3Ila4VC4O5d7vmWLVzhrNIw0TXB\nuVHnMKf9HDgfcMbM0JlUV4UQohAqfALPY24O/PkncOYMcPEid+/4rl1AaWrSCAQCjLYZjSivKHxI\n/wBLP0sEPwmWfdCEEFKMCj+FUpRbt4AlS7jbEJcuBdzcALVSLmu6+OIiPE97wraeLbb22Yr62vVl\nGywhJVDUnzNSMppCKYe2bYGzZ4E9e4D//a9sTSV6GPdApGckWui1gPVOa+y6u4vqqhBC5K7SjsDz\ny99UIi0NWLGi9E0lot5FwSPYAyoCFfg7+8O8jrnsAybkO8rwc0YKR3ehSAljQHAwN7Wirs4l8t69\nS07kuSwXu+7uwhLhEkyxn4KFnRdSXRUiV7q6ukXWxyaKTUdHB8nJyQVepwReTrm5wJEjXCLX0+Pu\nJ/9uMVehXn96jemh0xH1Lgr+/f3R1ahryTsRQkghKIFLKCcHCAoCvL2BJk24EXn79iXvd+KvE5h6\nZip6GffCul7roFtNV+axEkIqFrqIKaHvm0oMG8Y1lYiIKH6/vLoq1dWrw8LXAgceHVDaX2KEEMVF\nI/AyyMgAAgKAVau4kfiyZdzdK8W5lXgL7sHuMKhhAF8nXzTRaSKfYAkhSo1G4FJWpQowdSq3PL99\ne6BHD+7+8adPi96nrWFb3PO4h66Nu6J1QGusv7Ee2bnZRe9ACCGlRCNwCXz+zC3L37KFu+1w8WLA\nyKjo7fPqqvz79V/4O/vDvoG93GIlhCgXGoHLWF5TidhYoEGDkptK5NVVmdVuFvoH9ces0FlIzUyV\nb9CEkAqDErgU6Ohwd6iUpqlE/roqKekpsPC1wOnY0/IPmhCi9GgKRQb+/hvw8QH++APw9OSSuW4R\ndxJefHERk09NRqsGrbClzxbU06on32AJIQqJplB4Ur8+sHUrcP8+8P490Lw5V5P806eC2/Yw7oFH\nUx7BRMcE1n7W8L/nT3VVCCGlQiNwOcjfVGLOHO5OlsKaSjz65xHcg92hpqJGdVUIqeTkNgLPycmB\nnZ0dnJ2dpXG4Cid/U4l794puKmFV1wrXJ1zHCMsR6LK7C5aGLUV6dik7TxBCKh2pJPAtW7bA3Nwc\ngtKU76vEStNUQlVFFT+2+REPPB8g8l0kbHfa4nL85aIPSgiptCRO4ImJiQgJCcGkSZMq/VRJadna\nAidPAocPc0WzTE2BvXuB7HzrewxrGOLYsGPw6eEDt6NumHRyEpLTClYsI4RUXhIn8FmzZmHdunVQ\nUaHroWXVti1w7px4U4mDB8WbSgw2G4zHPz5GVbWqsPC1wMGog/SLkhACAChlE7HCnTp1Cvr6+rCz\ns4NQKCxyO29vb9H/Ozg4wKE0tVkrkS5dgMuXgfPnudWcq1ZxFz0HDuRqkdeoUgPbnbZjlPUouAe7\nY+/DvfDr5wejWkZ8h04IkRKhUFhsHi2MRHehLFy4EPv27YOamhrS09Px6dMnuLi4IDAw8NsJ6C6U\nMmEMOHWKS+SFNZXIysnC+hvrseHmBizotAAz2s2AmopEv4cJIQpIrvXAL1++jPXr1yM4WLxbOyXw\n8impqcSz5GfwPOWJ5LRkBDgHoFWDVrzFSgiRPrkv5KG7UKRHRQX44QcgKgqYPBmYNAno2RO4eZN7\nv6luU5wffR4z282EU5ATZp+dTXVVCKlkaCGPksjK4u5UWb4csLbm/tuyJfde0tckzDk3B5fjL8O3\nny+cmjnxGywhRGLUUq0Cymsq4eMDtGvHJXILC+69Cy8uwPOUJ+wb2GNzn81UV4UQJUa1UCqgvKYS\nT58CHToA3bt/ayrR07gnIqdEokmtJrD2s0bAvQCqq0JIBUYjcCX3+TNXOGvzZvGmEpH/RMIj2AMa\nqhrwd/aHaW1TvkMlhJQBjcArAW1t4JdfuBF4/qYStXOscX3CdbhauKLz7s7wFnojIzuD73AJIVJE\nCbyCqFWrYFOJeXNV4Wo0Ffcn38eDtw9gu8sWV15e4TtUQoiUUAKvYOrUAdat424/zMoCzMwAvzWG\n+L3Xcazqvgojj4yER7AHUtJS+A6VECIhSuAVVGFNJR7932DcHB0NdRV1WPha4FDUIbo+QYgSowRe\nwTVqBPj7A+HhXGOJVhY10Th6B/YPOIJfr/6K/gf64+WHl3yHSQgpB0rglUT+phJ37wJuXdpjXMY9\ntK3fEa38W2HTzU3Izs0u8TiEEMVBtxFWUg8ecHVW7t8H3H9+ijBNT3zO/IgA5wDY1bfjOzxCKj1a\niUlKdOsWd+/402cMPWYHIjh9HkZbj8Yyh2XQ1CikcSchRC7oPnBSorymEnv3CBD751ho74vCreh/\nYOFrgZCnIXyHx6sylmYmRO4ogRMA35pK+K2vg8yD+yA4FYAJh6dh+OHh+Cf1H77D4wUlcKLoKIET\nEYEAcHTk7ljZOsMR+ocf4fKJxmixxQoB936juiqEKBiaAydFymsqMX/TQ7xv5w7jRtVwaNSuCl1X\nRSj8NvJetgxYupT7fwcH8YYahMgaXcQkUpGTA/wRlIPZh3bgk+1yjDGdhh3Df0YVtSp8hyZT3t7c\ngxA+0EVMIhWqqsCY0ar4+9h0/Gp4H39cuge9Rbb43/mrfIcmVcoy560scRLZowROSk1dHZjn2RAp\nvicwvM6v8Dg3Ak2me+B6BD91VaSdyL4/nqJOmVACJ3kogZMyq1pVgN/muCBxYTTq11VDl4MW6DT5\nT8TGyneqTJaJTChU3AROSB41vgMgyqu+Tk3c+MUXF56MgttBD1j6BGKw+g6sWdgYRkZ8R1c631+0\nzBMfr1gJvKg46eJq5UYJnEisZ4sOSFgUgeUX12HjjVY4NXERxphOw6KFqjAwkOzY34+EpZ3Ivt8v\n76Klol28LCpOUrlRAidSoaGqgV97/YKxrVwxwXAyjr3ajz+6B8C9vx3mzwf09ct33O8TuCwTWXz8\nt+PRKJcoA0rgRKqa6TXDlYkXsffhXszV6oOw5DHYbeWNKZM0MWcOoKvLd4RFGzdOOUa59MuE5JE4\ngSckJGDMmDF49+4dBAIBPDw8MH36dGnERpSUQCDAONtx6NesH2afm433dSwREe+H5s37YPp0YOZM\noEaN73by8ABiY4Hq1SH0+hPCu1oAih8JSzuRKUtiVJY4iexJvJDn7du3ePv2LWxtbZGamopWrVrh\n+PHjMDMz405AC3kqvXPPz8HzlCcsarZD1cubcPl0Xfz0E9d8WTOv4KGDA1eMBQB++AH4808AUlpM\nk++XA4KCuAaiJaC7UAjf5LKQp169erC1tQUAaGlpwczMDG/evJH0sKQC6WXSC1FeUTAzMMRlcytM\n2/0/3L7D0LQpsGULkJ4OLrkCgL0910JImmJjuV8OZ85wybwUKHkTZSDV+8Dj4+Nx//59tG3bVpqH\nJQqstPdiV1evjrWOa3Fu9DkEv9mFpH7d4PfnE1y8CDRrBuxyPIxMl+HA+fNiI2SpJFJZ/nIghEdS\nu4iZmpqKoUOHYsuWLdDS0hJ7zzvf38AODg5woOFNhVHWqQbbera4OfEmdtzZgUmXO2LGnBmYpzEf\ny5dWx5pnB7DkODBqFKD233emVL5VgoK4kbe/f6mmTwjhg1AohLCMq9OkUswqKysL/fv3R9++fTFz\n5kzxE9AceIUmyRz1q4+v8GPIj3ie/Bz+zv7Ije+ERYuAd++4Y7q6Aiq0VphUUnKpRsgYw9ixY6Gn\np4dNmzaVKwiiXKRZcpUxhiMxRzAjdAacmzvDp8dq3L1WC4sWAWlpwPLlwMCBXK1yQioTuSTwa9eu\noUuXLrC2tobgv58yHx8f9OnTp9RBEOUlrZKrH9I/YMGFBTgZexKbe2+Gi9lQhIQIsGgRV0RrxQqg\nd29K5KTyoHrgROakXTP7+qvr8DjlAWMdY+xw2gFD7UY4ehRYsoRbBPTrr3SHCKkcqB44kTlpJ9OO\njTri/uT7aGvQFi13tcTW25sxeEgOHj0CPD2BSZOAnj2Bmzele15ClBGNwInCiv03FpNPTUZqZir8\n+/vDrr4dsrKAvXu5uXFra+6/LVvyHak4WgREpEHxR+AeHtx3upMT8OEDr6EQxdNcrzkujbmEKfZT\n0Ht/b8w7Pw+Z7AsmTQKePgX69AH69wdcXICoKL6j/YYaLhB54TeBl2OFHKlcBAIBJthNQJRXFF5/\nfg0rPyucfXYWVaoAU6cCz54BHToAPXoAbm7ctxQhlQW/UyhOTlzytrcvsAKPkMKEPgvFlNNT0P6N\nKjZF6KOuei0gKAifVWth61Zg0yZgwADuoqc8m0pQN3sibYp/F8qHD7RCjpTZl8wvWDbFFHv0ErH6\nAjC+6VAI/vw/AEBKCrBxI+DrCwwbBvzyCyRuKlFW1M2eSIPiz4HXqsVVnaPkTcpAU0MTa/+2wtl9\ngG/X6uje+y1i/+XmTnR0uHvGnzzhKh1aWQGzZ3OrO5UBzZ+TsqDbCIlyCgqCXecfcGvxKwyyGIoO\n/+uAX6/8isycTABA7drAunVAdDSQlQWYmgILFwLJybIPTZIpE0rgpCwogRPl9N9fb6q6epjRbgYi\nJkcgPDEcdrvscP3VddFm9esD27YBDx4ASUlA8+bcHPWnT7ILjea8ibzQfeCkwmCM4fDjw5gROgMD\nWwyET08f1KoqPj337Bl373hoKDBnDncni6ipBE/oAigpjOJfxCREBj6kf8DPF35GcGwwtvbZiiFm\nQ0R1evI8fsxdaLx6Ffj5Z2DyZKBqVX7izY8ugJI8in8RkxAZqFW1Fnb234lDQw9hcdhiDDw4EAkf\nE8S2MTfnrp+fOQNRU4mdO4HMTJ6CJqQcKIGTCqtTo064P/k+WjdoDbtddth6aytycnPEtrG1BU6e\nBA4fBo4eBVq0APbsAbKz+YmZpkxIWdAUClE4sqgl8iTpCSafmoyvWV8R4BwAm3o2hW535QqwaBHw\nzz/cfDQ1lSB8oSkUopRkcStdi9otEDY2DJ72nnDc54j55+fja9bXAtt16cJVd9i2jVvVaWMDHD8O\n0BiEKCJK4EShyPI+6Ly6Ko+mPELCpwRY+Vnh3PNzhWwH9OoFhIcDPj7cRcXWrbn5ckrkRJHQFAop\nPQ8PrlpU9epco2AprqDNu5VOKORGwPK4lS6vrkrHhh2xsfdG6GvqF7pdbi5ETSV0dLimEt26fbeR\nDD8bZUAMBN+8AAAXeUlEQVQldKWvVLmTyZgcTkHkpWtXxrhBKGM//CCTUyxdyj3kJTUjlf109iem\nv06f7b6/m+Xm5ha5bXY2Y/v3M2Ziwlj37oxdv57vTTl8NopMnl+zyqI0uZOmUEjpVa/O/dfenitA\nJiVC4bf7n5ct+/ZcHsvKNTU0sa7XOpxxO4Ptt7ejR2APPP33aaHbqqpyJWtjYoARI4Dhw4F+/YB7\n9yCzz4aQ4tAUCik9OVSP9PbON20i52mJ7NxsbL+9Hb9e+RWz2s3C3I5zoaGqUeT2GRlAQACwahXQ\nrlUmlmcugOWhxZVm+oRWkMoWrcQkSkdsJaKDAzchDgA//MCtvJGDlx9e4seQHxH/IR4BzgFo37B9\nsdt//Qr4+QFr13KNJby9uZorlQmtIJU+uo2QKB2xkRtP0xKNazVG8IhgLOm6BC5/usDrtBc+pn8s\ncvvq1bm6Ks+eARYWXIegCROA+Hi5hUwqKUrgRKGIJfCgIG7kff48hA/kOy0hEAjgauGKaK9o5OTm\nwMLXAkceHyl2RKStzTWQePYMMDQEWrUCvLyA169Ld05lLiVLUyb8kDiBh4aGwtTUFM2aNcOaNWuk\nERMhnHwNP/hKbjrVdLDLeRcOuBzAorBFGHRoUIG6Kt+rVYurePjkCaClVfqmEpTASVlJlMBzcnIw\ndepUhIaG4vHjxzhw4ABiYmKkFRshCqNz4854MPkBWtZrCbtddth2a1uBuirfq12bmxePjuZqq5iZ\nya+pBKkc1CTZ+fbt22jatCmM/useO3z4cJw4cQJmZmbSiI3wRUEWpXx/l0Mevu5yqKJWBUsdlmKY\n5TB4BHtg/6P9CHAOgHVd62L3q18f2LoV+OknbhFQ8+bAtGnArFlARIRi/RuJcpEogb9+/RoNGzYU\nPTc0NMStW7ckDorwLDb2290fHh5yu/vje98nMUW5y8G0timE44T4/f7v6BnYExPsJmBJ1yWorl69\n2P0aNQJGjgTmz+emWJo25S5+zp37ramEovwbiXKQKIF/XyS/KN75visdHBzgQEML6ShppFzekTQt\nSimRikAFk1pOQv/m/TEzdCas/Kzg188PvUx6Fbtf3iKlvXu5BUHe3lwi//ln/krYlhUtm5cNoVAI\nYVkvhEiy1PPmzZusd+/eouerVq1iq1evLvNyUFJOJS3fLu/y7pQUbvuUFGlFKrGwMJ4DcHfnPs++\nfQv9XE7HnmaNNzVmo46OYu9S3xV5mMKWnD94wNiAAYzVqcOYnx9jGRnSC1sWaNm8fJQmd0qUXbOy\nspixsTGLi4tjGRkZzMbGhj1+/LjMQZBy6tuXS8729oUn25LeJ6VXil+GnzM+s9mhs5n+On225/4e\nUV2VsLBvNV6Ab////S+lW7cY692bMSMjxnbvZiwrS1b/GMlQApcPmSdwxhgLCQlhzZs3ZyYmJmzV\nqlXlCoKUU/6RcmEjxEJG0ryPZJVVGX4Z3n19l7Xc1ZJ139udxSbFir1XmuR35QpjXbow1rw5YwcO\nMJaTI0HcUlLaX0JEeuSSwKURBJGCUk6X0OipnMo4rZSVk8U23tjI9NbosZVXVrKMbG5epLSff24u\nY+fOMdamDWOWlowdO8a9pgjoe0g+SpM7aSVmRcHXhUcPD+6KlpMTV+xKEUkjxnyLikpDTUUNs9rP\nwj2Pe7iecB2t/FvhZsLNUl/8EwgAR0dqKkGKR8WsKopiKgXKtGocTwWnyoTnGBlj+L/H/4eZoTMx\n2HQwVvVYhZpVa5bpGLm5wJEj3NeuyKYSckJ3ocgHNXQgBUj9z19luFCqIDEmf01m7ifdmcEGA3b0\n8dFyHSM7m7F9+4poKkEqlNLkTppCIZLJV3BKYetgK0iMOtV04O/sjwMuB7Dw0kIMOjgIiZ8Sy3QM\nVVVg1CjuHvKRI7nGEk5O3IpOUvnQFEolQ3/+KoaM7Az4XPPBjjs7sLTrUkyxnwJVFdWyHycD+O23\n/5pKtOOmyCwtZRAwkTtq6ECIgot5H4PJpyYjMycT/s7+JdZVKUpamnhTiaVLK19TiYqGGjoQouDM\n6phBOE6IiXYT0TOwJxZcWIC0rLQyH6daNa5k7dOnXFOJjh2pqURlQAmcEJ6pCFTg3sodkVMiEfch\nDlZ+Vrjw4kK5jqWtzZWsffq0fE0liHKhKRRCFMzp2NPwCvGCg5EDNvTagNrVa5f7WElJ3LTKb78B\nY8dyRbPq1pVisERmaAqFECXUr3k/RHtFQ6+aHix9LRH4MLDcg6D8TSVycrimEgsWUFOJioJG4IQo\nsHtv7sE92B261XSxs/9ONNVtKtHxXr3iFgEdPco1lZg5E6hZtjVFRE5oBK4klLkXIpGtVg1a4bb7\nbfRt2hftfmuH1ddWIysnq9zHa9SIW6x76xbw4gXQrBmwejXw5YsUgyZyQwlcAVACJ3kK+15QU1HD\nnA5zcMf9Di6/vIxW/q0Qnhgu0XlMTLimEpcvA/fvc883bwbS0yU6LJEzSuCEKJDifpk30WmCkJEh\nWNBpAQYfGoypIVPxKeOTROczMwMOHQLOngXCwrgR+c6dQGamRIclckIJnCd5rbW8vbnVc3n/T6Nx\nUhyBQIARViMQ7RWN9Ox0WPha4PhfxyU+ro0NcOIEVzDr+HGgRQtgzx7lafNWWdFFTAWQl7xJ5SRJ\ntcjL8ZfhccoD5nXMsb3vdhjUMJBKTFevAosWAW/fcjG5ugIqNNyTK1pKryQogZM85fleSM9Oh89V\nrq7KModl8LT3LFddle8xBly4wCXyr1+B5cuBQYO4WuVE9uguFCVBxaWIJKqqVcWybstwZfwVHIg6\ngE67O+HRP48kPm7+phKrVnEjcWoqoVhoBE6IApG0WmQuy8VvEb/hl0u/wKOlBxZ1WYRq6tWkEltu\nLnf/+JIl/DeVqAxoCoXIl4cHEBvLtXcLClLc+uCVwN+f/8aM0Bm4//Y+dvbbiR7GPaR27Jwc4OBB\nbq6+cWMukbdvL7XDk/9QAifypQzt1SqZU7Gn8GPIj+hm1A3re62XqK7K97KygMBAYMUKrgLiihVA\ny5ZSO3ylR3PgRL74aqxMitS/eX9Ee0VDp6oOLH0tse/hPqkNqNTVgYkTgSdPgH79AGdnwMUFiIqS\nyuFJKdAIvLKQx/RGMY2VCf/uvrkL92B31K5eGzv77YSJrolUj5+/qUT37tzdNNRUovxkPgKfO3cu\nzMzMYGNjgyFDhuDjx4+SHI7IUmwsN71x5gyXZGWhVi1u2oSSt0Kyb2CPO+530NukN9r+1lbiuirf\ny99UwtKSmkrIg0QJvFevXoiOjsbDhw/RvHlz+Pj4SCsuIm00vUHA1VX5qcNPoroq9gH2uJV4S6rn\noKYS8iNRAnd0dITKf8uz2rZti8TEsnXYJnKkIJ3ZiWLIq6vyc8efMejQIEw/M13iuirfq1WLW/zz\n5AmgpQVYWQGzZgH//CPV01RqUruI+fvvv8PJyUlahyPSRtMb5Dv566p8zfoKC18LnPjrhNTPk9dU\n4vFj7l5yMzOuM9C//0r9VJVOiRcxHR0d8fbt2wKvr1q1Cs7OzgCAlStXIiIiAkeOHCl4AoEAS/OK\nOwBwcHCAAy09JEThCOOFmHxqMiz1LbG1z1ap1VX5Xl5TiSNHuKYSs2ZRUwkAEAqFEOarZrds2TLZ\n3we+Z88eBAQE4OLFi6hatWrBE9BdKIQojfTsdKy6ugp+d/1EdVVUBLK52/j5c26K5cwZ7uLntGmA\npqZMTqWUZL6QJzQ0FHPmzMHly5dRu3bhCwQogROifB6/fwyPYA/kslz4O/vDUt9SZueKieFuObxy\nBZg/H/D0BAoZC1Y6Mk/gzZo1Q2ZmJnR1dQEA7du3h6+vb5mDIIQonlyWi4B7AVgcthgerbi6KlXV\nZJdZHz7k6qzcu8dVQJwwAdDQkNnpFB4tpSckD9VpKbe/P/+N6aHT8fDtQ+zqvwvdmsi2gtXt21wi\nf/KEq7cyahSgpibTUyokSuCE5KE6LRILfhKMqWemonuT7ljvuB561fVker78TSW8vYFhwypXUwmq\nhUJIHlrIJDHnFs6ImhKFmlVqwsLXAvsj98t0cNa5M1ded/t2YMsWru3bsWNUizw/GoHzgf6clz+q\n0yJVd17fgXuwO/Q19eHXz0/qdVW+xxhw+jSweDGgqspVPuzTp2J3B6IpFD4Vl6Tpz3lSAWTlZGFT\n+Casvb4WczvMxez2s6Guqi7Tc+bmcqPwJUu4H6mK3FSCplD4VFzxKPpznlQA6qrqmNdxHm6738al\n+EuwD7DH7de3ZXpOFRWuZG1kJFdfxcMD6NEDuHFDpqdVWJTAZaW4JE11SUgFYqxjjFC3UMzrMA8D\nDgzA9DPT8Tnjs0zPqaoKuLlx95C7uQEjR3I1ySMiZHpahUNTKLJCc66kEvr367+Ye34uLry4gO1O\n2zGgxQC5nDcjA/jf/4CVK4F27bgGzJayW3skFzQHTgjhRVhcGCafmgyrulbY1ncbGmg3kMt5K1JT\nCZoDJ4TwoluTboicEgnz2uaw2WkDvzt+yGW5Mj9vXlOJZ8++NZUYPx6Ii5P5qXlBI3BCiExFvYuC\nR7AHBAIB/Pv7w0LfQm7n/vAB2LgR2LEDcHUFfvmFazKhDGgETgjhnaW+Ja5NuIZRVqPgsNcBiy8t\nRnp2ulzOnb+phLY2YG1dsZpKUAInhMicikAFU1pPwUPPh4hJioG1nzXC4sLkdv7vm0qYm1eMphI0\nhUIIkbuTT05iashU9DTuiXWO62ReV+V7CQncIqDDhxW3qQRNoRBCFNKAFgMQ7RUNbQ1tWPpZIuhR\nkFwHeg0bArt2cZUP4+KAZs2A1auBL1/kFoJU0AicEMKr269vwyPYA3W16sKvnx+MdYzlHkNeU4nL\nl7mpFUVoKkEjcEKIwmtj0AZ33O+gu1F3tAlog7XX1yIrJ0uuMZiZAYcOAWfPAmFhQNOmwM6dQGam\nXMMoMxqBE0IUxouUF/A85Yl3X94hwDkArQ1a8xJH/qYSS5YAo0fLv6kErcRUBlRalhAxjDEEPQrC\nnHNzMNxyOFZ0WwHtKtq8xMJnUwmaQlEGxVUtJKQSEggEcLN2Q7RXND5mfISFrwWCnwTzEkteU4kd\nOxSzqQSNwPnm5MQlb3t7qk5ISCEuxV2C5ylP2NSzwdY+W1Ffuz4vcXzfVGLNGq6UrazQCFwZUGlZ\nQorVvUl3PPR8iBZ6LWC90xq77u6SS12V7wkEQP/+wL17wIIFwLt3cg+hYEw0AieEKIuod1FwD3aH\nqkAV/s7+MK9jzndIMkMjcEJIhWKpb4nrE65jpNVIdN3TFUvClsitrooikjiBb9iwASoqKkhOTpZG\nPIQQUiwVgQq8WnvhweQHiH4fDZudNhDGC/kOixcSJfCEhAScP38ejRs3llY8hBBSKgY1DHDE9QjW\n9FyD0cdGY+KJiUhOq1wDSYkS+OzZs7F27VppxUIIIWU2yHQQor2ioamhCQtfC7nXVeFTuRP4iRMn\nYGhoCGtra2nGQwghZVajSg1s7bsVx4cdx+prq9H3j76IS6mgbXjyKXZxqKOjI96+fVvg9ZUrV8LH\nxwfnzp0TvVbcbzxvb2/R/zs4OMDBwaHskRJCSAnaGrbFPY972HBzA1oHtMbPnX7GzHYzoaYi53Xw\n5SAUCiEUCsu0T7luI4yKikKPHj1QvXp1AEBiYiIMDAxw+/Zt6Ovri5+AbiMkhPDgefJzeJ72RNLX\nJAQ4B8C+gT3fIZWJ3GqhNGnSBPfu3YOurm65giCEEFlgjGF/5H7MPT8XIyxHYEX3FdDS0OI7rFKR\n233gAoFAGochhBCpEggEGG0zGlFeUUhJT4GFrwVOx57mOyypoZWYhJBK4+KLi5h8ajJa1m+JLX22\n8FZXpTRoJSYhhOTTw7gHHk15hKa6TWG90xr+9/x5qasiLTQCJ4RUSo/+eQT3YHeoqagpZF0VGoET\nQkgRrOpa4fqE6xhhOQJddnfB0rClSldXhUbghJBKL/FTIqadmYaY9zHY1X8Xuhp15TskaqlGCCFl\ncSzmGKaHTkdvk95Y67gWutUK3hotLzSFQgghZTDYbDCivaJRTa0aLHwtcDDqoEIPQGkETgghhQhP\nDIdHsAcMahjAr58fjGoZyfX8NAInhJByamfYDvc87qFr466w97fHhhsbkJ2bzXdYYmgETgghJXiW\n/AyepzyRnJaMAOcAtGrQSubnpIuYhBAiJYwx7Ivch7nn58LNyg3Luy2XaV0VSuCEECJlSV+TMOfc\nHOhW1cWmPptkdh5K4IQQIiPZudkyrTNOFzEJIURGFKFJBCVwQghRUpTACSFESVECJ4QQJUUJnBBC\nlBQlcEIIUVKUwAkhRElRAieEECVFCZwQQpQUJXBCCFFSEiXwbdu2wczMDJaWlpg/f760YiKEEFIK\n5U7gYWFhOHnyJCIjIxEVFYWffvpJmnFVSEKhkO8QFAZ9Ft/QZ/ENfRZlU+4E7ufnhwULFkBdXR0A\nUKdOHakFVVHRN+c39Fl8Q5/FN/RZlE25E/jTp09x5coVtGvXDg4ODrh796404yKEEFKCYstpOTo6\n4u3btwVeX7lyJbKzs5GSkoLw8HDcuXMHrq6uePHihcwCJYQQ8h1WTn369GFCoVD03MTEhCUlJRXY\nzsTEhAGgBz3oQQ96lOFhYmJSYh4ud0HbQYMG4dKlS+jatStiY2ORmZkJPT29Ats9e/asvKcghBBS\njHJ35MnKysKECRPw4MEDaGhoYMOGDXBwcJByeIQQQooi85ZqhBBCZEOuKzE3bNgAFRUVJCcny/O0\nCmXu3LkwMzODjY0NhgwZgo8fP/IdktyFhobC1NQUzZo1w5o1a/gOhzcJCQno1q0bLCwsYGlpia1b\nt/IdEq9ycnJgZ2cHZ2dnvkPh3YcPHzB06FCYmZnB3Nwc4eHhhW4ntwSekJCA8+fPo3HjxvI6pULq\n1asXoqOj8fDhQzRv3hw+Pj58hyRXOTk5mDp1KkJDQ/H48WMcOHAAMTExfIfFC3V1dWzatAnR0dEI\nDw/Hjh07Ku1nAQBbtmyBubk5BAIB36HwbsaMGXByckJMTAwiIyNhZmZW6HZyS+CzZ8/G2rVr5XU6\nheXo6AgVFe5jb9u2LRITE3mOSL5u376Npk2bwsjICOrq6hg+fDhOnDjBd1i8qFevHmxtbQEAWlpa\nMDMzw5s3b3iOih+JiYkICQnBpEmTSuzEXtF9/PgRV69exYQJEwAAampqqFmzZqHbyiWBnzhxAoaG\nhrC2tpbH6ZTG77//DicnJ77DkKvXr1+jYcOGoueGhoZ4/fo1jxEphvj4eNy/fx9t27blOxRezJo1\nC+vWrRMNbiqzuLg41KlTB+PHj0fLli3h7u6Or1+/Frqt1D4tR0dHWFlZFXicPHkSPj4+WLZsmWjb\niv4btqjPIjg4WLTNypUroaGhgZEjR/IYqfzRn8cFpaamYujQodiyZQu0tLT4DkfuTp06BX19fdjZ\n2VX43FAa2dnZiIiIgJeXFyIiIqCpqYnVq1cXum257wP/3vnz5wt9PSoqCnFxcbCxsQHA/anUqlUr\n3L59G/r6+tI6vUIp6rPIs2fPHoSEhODixYtyikhxGBgYICEhQfQ8ISEBhoaGPEbEr6ysLLi4uGDU\nqFEYNGgQ3+Hw4saNGzh58iRCQkKQnp6OT58+YcyYMQgMDOQ7NF4YGhrC0NAQrVu3BgAMHTq0yARe\n7pWY5WVkZMT+/fdfeZ9WYZw5c4aZm5uz9+/f8x0KL7KyspixsTGLi4tjGRkZzMbGhj1+/JjvsHiR\nm5vLRo8ezWbOnMl3KApDKBSy/v378x0G7zp37syePHnCGGNs6dKlbN68eYVuJ7UReGlV9j+hp02b\nhszMTDg6OgIA2rdvD19fX56jkh81NTVs374dvXv3Rk5ODiZOnFjkFfaK7vr169i/fz+sra1hZ2cH\nAPDx8UGfPn14joxflT1HAFyvBTc3N2RmZsLExAS7d+8udDtayEMIIUqKLvkSQoiSogROCCFKihI4\nIYQoKUrghBCipCiBE0KIkqIETgghSooSOCGEKClK4IQQoqT+H+SGN2/ToTJeAAAAAElFTkSuQmCC\n", "text/plain": [ "<matplotlib.figure.Figure at 0x10b2d85d0>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from sklearn import svm, linear_model\n", "import course_utils as bd\n", "from sklearn.metrics import confusion_matrix\n", "import pandas as pd\n", "reload(bd)\n", "\n", "#Generate training and testing data from the same distribution\n", "m = [[0.25,0.25],[1.25,1.25]]; s = 2; n=20\n", "\n", "def makeData(m, s, n):\n", " X1=pd.DataFrame(np.random.multivariate_normal([m[0][0],m[0][1]],[[s,0],[0,s]],n), columns=['x1','x2'])\n", " X2=pd.DataFrame(np.random.multivariate_normal([m[1][0],m[1][1]],[[s,0],[0,s]],n), columns=['x1','x2'])\n", " X1['Y'] = -1 * np.ones(X1.shape[0]) \n", " X2['Y'] = 1 * np.ones(X2.shape[0])\n", " return X1.append(X2, ignore_index=True)\n", "\n", "train = makeData(m, s, n)\n", "test = makeData(m, s, n)\n", "\n", "#Fit both types of models\n", "logreg = linear_model.LogisticRegression(C=1e30)\n", "logreg.fit(train[['x1','x2']], (train['Y']+1)/2)\n", "\n", "my_svm = svm.SVC(kernel='linear')\n", "my_svm.fit(train[['x1','x2']], train['Y'])\n", "\n", "#Get accuracy\n", "def getACC(X_t, Y_t, mod):\n", " cm=confusion_matrix(mod.predict(X_t),Y_t)\n", " return (cm[0][0]+cm[1][1])/float(sum(cm))\n", "\n", "acc_svm = getACC(test[['x1','x2']], test['Y'], my_svm)\n", "acc_lr = getACC(test[['x1','x2']], (test['Y']+1)/2, logreg)\n", "\n", "#Plot them\n", "w_svm = my_svm.coef_[0]; a_svm = -w_svm[0] / w_svm[1]; t_svm = my_svm.intercept_[0];\n", "w_lr = logreg.coef_[0]; a_lr = -w_lr[0] / w_lr[1]; t_lr = logreg.intercept_[0];\n", "\n", "xx = np.linspace(test[['x1','x2']].iloc[:,0].min(), test[['x1','x2']].iloc[:,1].max())\n", "\n", "yy_svm = a_svm * xx - (t_svm) / w_svm[1]\n", "yy_lr = a_lr * xx - (t_lr) / w_lr[1]\n", "\n", "\n", "plt.plot(test[(test['Y']==-1)].iloc[:,0], test[(test['Y']==-1)].iloc[:,1],'r.')\n", "plt.plot(test[(test['Y']==1)].iloc[:,0], test[(test['Y']==1)].iloc[:,1],'b+')\n", "plt.plot(xx, yy_svm, label='SVM({})'.format(acc_svm))\n", "plt.plot(xx, yy_lr, label='LR({})'.format(acc_svm))\n", "plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<p>We can see that the two classifiers behave very similarly when just looking for a linear decision boundary. The choice of which classifier to use is usually determined by comfort and familiarity as well as the need to exploit the specific properties of each.<br><br>\n", "\n", "<ul><b>SVM</b>\n", " <li>Intuitive geometric interpretation</li>\n", " <li>Extensibility to non-linear decision surfaces</li>\n", " <li>Strong learning guarantees</li>\n", "</ul>\n", "<ul><b>Logistic Regression</b>\n", " <li>Designed for estimating $P(Y|X)$</li>\n", " <li>Strong statistical properties</li>\n", " <li>Suited for ranking</li>\n", "</ul>\n", "</p>" ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.9" } }, "nbformat": 4, "nbformat_minor": 0 }