{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# Projections of Probability Distributions, and the Reparameterization Trick" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For approximate inference and learning, it's often necessary to approximate a target distribution $p$ with another distribution $q$ from a (restricted) set $Q$ of distributions. Here I demonstrate the two types of projections based on KL divergence, detailed in section 8.5 Daphne Koller's PGM textbook. I also demonstrate the score function trick (REINFORCE) and parameterization trick for variational inference in finding I-projections." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline\n", "import numpy as np\n", "np.random.seed(0)\n", "import matplotlib.pyplot as plt\n", "from scipy.stats import norm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let $p(x)$ be our target distribution, in this case a mixture of two scalar Gaussians, \n", "$$ p(x) = \\pi_1 \\mathcal{N}(x; \\mu_1, \\sigma^2_1) + \\pi_2 \\mathcal{N}(x; \\mu_2, \\sigma^2_2) $$\n", "\n", "Let's plot its pdf:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl83PV95/HXR6NblmVZl2Udvm3Z4AvkAxvwBYQjwbSE\nYI5k0yahJKFJN5vdZptst7t97G7aJJukLQkl2TRNIaGkBOyGy5gANtgG37dlSz4lyzpty5JsXfPd\nPyQTYWQ0kmfmN8f7+XjooZnf/H4zn7HgPd/5/r6/79ecc4iISPxI8LoAEREJLwW/iEicUfCLiMQZ\nBb+ISJxR8IuIxBkFv4hInFHwi4jEmYCC38xuN7MKM6s0s28M8PhKM9ttZjvNbKuZ3RjosSIiEl42\n2AVcZuYDDgG3AtXAFuAB59z+fvuMANqcc87MZgHPOufKAjlWRETCKzGAfeYDlc65IwBm9gywEng/\nvJ1zrf32zwBcoMcOJDc3140fPz7AtyAiItu2bWt0zuUFsm8gwV8EnOx3vxpYcPlOZvYHwP8B8oG7\nhnJs3/GPAI8AlJaWsnXr1gBKExERADM7Hui+QTu565x73jlXBtwD/PUwjn/SOVfunCvPywvoQ0tE\nRIYhkOCvAUr63S/u2zYg59x6YKKZ5Q71WBERCb1Agn8LMMXMJphZMrAKWNN/BzObbGbWd/s6IAVo\nCuRYEREJr0H7+J1z3Wb2GPAq4AN+5pzbZ2aP9j3+BHAv8Bkz6wIuAPe73uFCAx4bovciIiIBGHQ4\npxfKy8udTu6KiATOzLY558oD2VdX7oqIxBkFv4hInFHwy7B1dPfwwo4aNlY2el2KiAxBIBdwiXxA\nw/kOnn73OE9tPk5jaydm8PXbpvGlpZPoG9wlIhFMwS8B23+qhX965yird56is8fP8rJ8PnPDOJ7f\nUcN3Xq3gQG0L3/nkbNKSfV6XKiIfQcEvg+rq8fPFp7az7kAdaUk+Vs0v4bOLxjMxbwQAS6bmMb1w\nJH/zykGONrbx5GfKKRqV5nHVInIlCn4Z1D9vPMa6A3V8ZflkPnfjRLLSkz7wuJnx6JJJTCvI5CvP\n7ODuv3+bHz98PfMnjPaoYhH5KDq5Kx+pruUiP1h3mGXT8viPt079UOj3t6wsnxe+vJistCQe/Mlm\n/nXLiTBWKiKBUvDLR/rfLx2gs8fPX919TUAnbifljeD5Ly/mhkk5/MXzeznW2BaGKkVkKBT8ckWb\nqppYvfMUjy6ZxLicjICPy0pL4nufmk2Sz/i71w+HsEIRGQ4Fvwyoq8fPX67eS3F2Gl9aOmnIx+dn\npvIfbhjP8ztrOFx3PgQVishwKfhlQP/0zlEO17fyV5+4htSk4Q3P/JMlk0hP8vGDdWr1i0QSBb98\nSO25C/xg3WFWlOVzy4yCYT/P6IxkPnfjBF7cU8u+U+eCWKGIXA0Fv3zI/3rxAN1+x3//xDVX/Vyf\nu2kiI1MT+f5rh4JQmYgEg4JfPuCdykZ+u7uWLy2dRGlO+lU/X1ZaEn+yZBLrDtSz8+TZIFQoIldL\nwS/v6+zuPaFbOjqdR5cM/YTulXx20XhGZyTzvbUVQXtOERk+Bb+8b+3+01Q1tPHNu6YP+4TuQDJS\nEvnikklsONzIu0eagva8IjI8Cn5533PbqinMSuWW6cM/oXslDy8cR35mCt977RCRuOqbSDxR8AsA\n9ecvsv5wI/fMLcKXEPypldOSfTy2fDLvHW3mnUq1+kW8pOAXANbsPEWP33HvdUUhe43755UwNiuV\n766tUKtfxEMKfgHgue01zC7OYnJ+ZsheIyXRx2PLp7Dz5Fm2HT8TstcRkY+m4Bf2n2rhQG0L915f\nHPLXunvOWFKTEnhhZ03IX0tEBqbgF57bXk2Sz/jErLEhf60RKYncOmMML+6upavHH/LXE5EPU/DH\nue4eP6t31rC8LJ/sjOSwvOY9c8Zypr2L9YcawvJ6IvJBCv44t/5wA42tndx7Xei7eS65eWoe2elJ\nvLDzVNheU0R+T8Ef557bXkN2ehJLp+WH7TWTfAncNauQ1/afprWjO2yvKyK9Agp+M7vdzCrMrNLM\nvjHA4w+Z2W4z22NmG81sdr/HjvVt32lmW4NZvFydc+1dvLa/jpVzikhODG8b4J45RVzs8rN23+mw\nvq6IBBD8ZuYDHgfuAGYAD5jZjMt2Owoscc7NBP4aePKyx5c55+Y458qDULMEyYt7auns9oe1m+eS\n60qzKRqVpu4eEQ8E0sybD1Q654445zqBZ4CV/Xdwzm10zl0amL0ZCH+SyJA9t72aKfkjuLZoZNhf\nOyHBWDlnLG8fbqDhfEfYX18kngUS/EXAyX73q/u2XcnngJf73XfAOjPbZmaPXOkgM3vEzLaa2daG\nBo32CLWjjW1sO36Ge68vDmgR9VC4Z24Rfge/3a1Wv0g4BbVj18yW0Rv8f95v843OuTn0dhV92cxu\nHuhY59yTzrly51x5Xl5eMMuSATy/vZoE6+1r98rUgkymF45Ud49ImAUS/DVASb/7xX3bPsDMZgE/\nBVY6596fhcs5V9P3ux54nt6uI/GQ3+94bnsNiyfnMiYr1dNa7pkzll0nz3Kssc3TOkTiSSDBvwWY\nYmYTzCwZWAWs6b+DmZUCvwE+7Zw71G97hpllXroN3AbsDVbxMjzvHWum5uwFT07qXu7uOWMxg9Vq\n9YuEzaDB75zrBh4DXgUOAM865/aZ2aNm9mjfbn8J5AA/umzYZgHwtpntAt4DXnTOvRL0dyFD8sre\n06QkJnDbNcGfd3+oCrPSWDBhNKt31mjGTpEwSQxkJ+fcS8BLl217ot/tzwOfH+C4I8Dsy7eLt96s\nqGfRpBzSkwP684fcPXOK+MZv9rCn5hyzikd5XY5IzNOVu3HmaGMbx5raWVYWvit1B3PHzEKSfQm8\nsEPdPSLhoOCPM28crAdg6dTICf6stCSWleXx77t7F4MRkdBS8MeZNyrqmZSXQWlOutelfMA9c4po\nON/BpiotyygSagr+ONLe2c27R5pZFsYJ2QK1rCyftCQfr+3X3D0ioabgjyMbK5vo7PFHVP/+JalJ\nPhZPzmXdgXqN7hEJMQV/HHmjop6MZB/l47O9LmVAK6bnU3P2AofqWr0uRSSmKfjjhHOONysaWDw5\nl5REn9flDGh53zeRdQfqPK5EJLYp+OPE4fpWas5eiMhunksKRqYysyiL3/WNPBKR0FDwx4n3h3FO\ni+wJ8JaX5bP9xBma2zq9LkUkZin448QbFfWUjcmkMCvN61I+0i3TC3Du9x9UIhJ8Cv440HKxi63H\nzkR0N88l14wdSX5mirp7REJIwR8H3jncSLffReT4/cslJBgrpufz1qEGOrv9XpcjEpMU/HHgjYp6\nMlMTua40OiZAW15WQGtHN1uONXtdikhMUvDHOOccb1Q0cPPUPBJ90fHnXjw5h+TEBA3rFAmR6EgC\nGbZ9p1poON8RFd08l6QnJ7J4Ug6v6ypekZBQ8Me4Nyt6T5IumRrZwzgvt3x6ASea26lq0JKMIsGm\n4I9xb1Q0MKs4i7zMFK9LGZIVfSOQXld3j0jQKfhj2Jm2TnacOMPSKOrmuWTsqDSmF47kdQ3rFAk6\nBX8MW3+4Ab+DZRF+te6VrCjLZ9vxM5xt11W8IsGk4I9hb1U0kJ2eFLXr2K6Ynk+P3/HWoQavSxGJ\nKQr+GOWc452qRhZNzsWXYF6XMyyzi0eROyKZdQfU3SMSTAr+GHWksY26lg4WTcrxupRhS0gwlk3L\n562Kerp6dBWvSLAo+GPUxr61axdPyvW4kquzYno+LRe72Xb8jNeliMQMBX+M2lTVyNisVMZF2KLq\nQ3XjlDySfQka1ikSRAr+GOT3OzZVNXHDpFzMorN//5IRKYnMm5CtE7wiQRRQ8JvZ7WZWYWaVZvaN\nAR5/yMx2m9keM9toZrMDPVaC78DpFs60d7F4cvT27/e3dGo+h+paOXX2gteliMSEQYPfzHzA48Ad\nwAzgATObcdluR4ElzrmZwF8DTw7hWAmyTX39+zdE8Ynd/i6tGvZmhVr9IsEQSIt/PlDpnDvinOsE\nngFW9t/BObfROXfp7NtmoDjQYyX4NlY1MTE3I+JX2wrU5PwRjM1K5a1DGtYpEgyBBH8RcLLf/eq+\nbVfyOeDlYR4rV6mrx8+7R5piprUPYGYsmZbPO5VNWpxFJAiCenLXzJbRG/x/PoxjHzGzrWa2taFB\nX+mHa3f1Odo6e1g8ObqHcV5u6bQ8Wju62X5CwzpFrlYgwV8DlPS7X9y37QPMbBbwU2Clc65pKMcC\nOOeedM6VO+fK8/Kic26ZSLCpqhGAhRNjp8UPsGhSDokJpn5+kSAIJPi3AFPMbIKZJQOrgDX9dzCz\nUuA3wKedc4eGcqwE18aqJqYXjmR0RrLXpQRVZmoS5eM1rFMkGAYNfudcN/AY8CpwAHjWObfPzB41\ns0f7dvtLIAf4kZntNLOtH3VsCN6HABe7eth6/AyLY6h/v7+l0/I5UNtCXctFr0sRiWqJgezknHsJ\neOmybU/0u/154POBHiuhsf34GTq7/SyKkfH7l1syNY9vv3yQtyoa+NS8ksEPEJEB6crdGLKxqglf\ngjFv/GivSwmJsjGZFIxMUXePyFVS8MeQjVWNzC7OIjM1yetSQsLMWDI1jw2HG+jWbJ0iw6bgjxHn\nL3axq/oci6J8Ns7BLJ3WO1vnzpNnvS5FJGop+GPElmPN9PhdVM+/H4jFfQvLaFinyPAp+GPExsom\nkhMTuG5cttelhFRWWhLXlY5SP7/IVVDwx4iNVU2Uj8smNcnndSkht3RaPntqztFwvsPrUkSikoI/\nBjS3dbK/tiXmu3kuWTK198ru9Wr1iwyLgj8GbD7SO0PGohibn+dKZhSOJHeEhnWKDJeCPwZsrGpk\nREois4qyvC4lLBISeod1rj/cQI/feV2OSNRR8MeAjVVNzBufTaIvfv6cS6flcba9i93VGtYpMlTx\nkxQxqq7lIkca2mJuGubB3DQllwTTqlwiw6Hgj3KXllmMtWmYBzMqPZk5JaN4s0KrcokMlYI/ym2s\naiQrLYkZhSO9LiXslpfls6tawzpFhkrBH+U2HWli4cTRJCSY16WE3fKyAgC1+kWGSMEfxU42t3Oy\n+QI3xFk3zyXTCzMpzErldwcV/CJDoeCPYpv6xu/fEOMTs12JmbGsLJ8Nhxu1CLvIECj4o9jmqiZy\nMpKZWjDC61I8s3xaPq0d3Ww51ux1KSJRQ8EfpZxzvf37k3Iwi7/+/UsWT84lJTGB1w+ou0ckUAr+\nKHWsqZ3acxfjZn6eK0lL9rFoUg6/O1jndSkiUUPBH6Uujd+P1xO7/S0vy+dYUztHGlq9LkUkKij4\no9TGqkYKRqYwITfD61I8t6wsH0Cje0QCpOCPQs45Nh9pZtGk3Lju37+kODudsjGZ6ucXCZCCPwpV\n1rfS2Nqhbp5+lpXls+VYMy0Xu7wuRSTiKfij0MZL/ftxfmK3vxVl+XT7HRsONXpdikjEU/BHoU1V\nTRRnp1EyOt3rUiLG3NJsRqUn8bpG94gMSsEfZfx+x+ajTermuYwvwVg6NY83K7Q4i8hgAgp+M7vd\nzCrMrNLMvjHA42VmtsnMOszs65c9dszM9pjZTjPbGqzC49WB0y2cbe9SN88Alk8voLmtk11anEXk\nIw0a/GbmAx4H7gBmAA+Y2YzLdmsGvgJ89wpPs8w5N8c5V341xUq/8fsK/g9ZMiUPX4LxO43uEflI\ngbT45wOVzrkjzrlO4BlgZf8dnHP1zrktgIZUhNimqiYm5GZQmJXmdSkRJys9ievHZfO6xvOLfKRA\ngr8IONnvfnXftkA5YJ2ZbTOzR4ZSnHxQd4+f9442x91qW0OxoiyfA7Ut1J674HUpIhErHCd3b3TO\nzaG3q+jLZnbzQDuZ2SNmttXMtjY0aB3Vgew71cL5ju64n5/no6yYrqt4RQYTSPDXACX97hf3bQuI\nc66m73c98Dy9XUcD7fekc67cOVeel5cX6NPHlY1xur7uUEzKG0HJ6DT184t8hECCfwswxcwmmFky\nsApYE8iTm1mGmWVeug3cBuwdbrHxbtORJqYWjCAvM8XrUiKWmbGirIB3qhpp7+z2uhyRiDRo8Dvn\nuoHHgFeBA8Czzrl9ZvaomT0KYGZjzKwa+BrwLTOrNrORQAHwtpntAt4DXnTOvRKqNxPLOrv9bDna\nrPH7AfjYNWO42OVXd4/IFSQGspNz7iXgpcu2PdHv9ml6u4Au1wLMvpoCpdeOE2e40NUTt8ssDsX8\nCaPJHZHCS3tq+fissV6XIxJxdOVulNhwuBFfgrFoslr8g/ElGHdcO4bfHaxXd4/IABT8UWJDZSNz\nSkYxMjXJ61Kiwl2zCtXdI3IFCv4ocLa9k93VZ7lpirp5AjVv/O+7e0TkgxT8UeCdyiacg5umaJhr\noHwJxp0z1d0jMhAFfxR4u7KBzNREZhdneV1KVLlzprp7RAai4I9wzjnWH2pk0aQcEn36cw3FvPGj\nyctM4cXd6u4R6U9JEuGONrZRc/aCunmG4dLonjcq6mnrUHePyCUK/gj3dmXvUoI6sTs86u4R+TAF\nf4Rbf6iR0tHpjMvJ8LqUqHSpu0eje0R+T8Efwbp6/Gw+0qTW/lVQd4/Ihyn4I9jOk2dp7ehW8F+l\nu9TdI/IBCv4ItuFQAwmG5ue5SuXq7hH5AAV/BFt/uJHZJaPIStM0DVfDl2Dc2Td3j7p7RBT8Eetc\ne1ffNA0axhkMd84spKNb3T0ioOCPWBurGvE7uFn9+0FRPn40+bqYSwRQ8Ees9YcbGZGSyOySUV6X\nEhM0ukfk9xT8Ecg5x4bDDdwwKYckTdMQNHfPGUtHt1+tfol7SpUIdLypneozF9TNE2TXlWYztWAE\nT7973OtSRDyl4I9AGw43AHCjTuwGlZnx0IJx7Ko+x57qc16XI+IZBX8E2nC4keLsNMbnpHtdSsz5\ng+uKSEvy8cv31OqX+KXgjzBdPX42VTVx05Q8zMzrcmLOyNQk7p49ltU7T9FyscvrckQ8oeCPMDtP\nnuW8pmkIqYcWltLe2cPqHTVelyLiCQV/hFm3v44kn3Gjgj9kZhWPYmZRFk9tPoFzzutyRMJOwR9B\nnHOs3V/Hwok5jEzVNA2h9NCCUirqzrPt+BmvSxEJOwV/BKlqaOVoYxu3zSjwupSYd/ecsWSmJPL0\nuye8LkUk7BT8EWTt/joAblHwh1x6ciJ/eF0RL+6ppbmt0+tyRMIqoOA3s9vNrMLMKs3sGwM8XmZm\nm8ysw8y+PpRj5fde21/HrOIsCrPSvC4lLjy4YByd3X6e21btdSkiYTVo8JuZD3gcuAOYATxgZjMu\n260Z+Arw3WEcK0B9y0V2nDjLrdPV2g+XaWMymTc+m1++dwK/Xyd5JX4E0uKfD1Q654445zqBZ4CV\n/XdwztU757YAlw+MHvRY6bXuQO90wbdeo+APp4cWjONoYxubjjR5XYpI2AQS/EXAyX73q/u2BeJq\njo0rr+0/TenodKYVZHpdSly5/doxZKcnaf4eiSsRc3LXzB4xs61mtrWhocHrcsKqtaObdyqbuHVG\nga7WDbPUJB/3lZewdl8d9S0XvS5HJCwCCf4aoKTf/eK+bYEI+Fjn3JPOuXLnXHleXnxNTrb+UAOd\nPX5u1WgeTzwwv5Qe5/jZO8e8LkUkLAIJ/i3AFDObYGbJwCpgTYDPfzXHxo21+06TnZ5E+bhsr0uJ\nSxNyM7h79lh+vvEodWr1SxwYNPidc93AY8CrwAHgWefcPjN71MweBTCzMWZWDXwN+JaZVZvZyCsd\nG6o3E426enrXgV1eVkCiFl3xzNdunUp3j+OHrx/2uhSRkEsMZCfn3EvAS5dte6Lf7dP0duMEdKz8\n3pajzbRc7FY3j8fG5WTw4IJSnn73BF+4aSITcjO8LkkkZNTE9Nja/XWkJCZw81RNyua1x5ZPJtmX\nwPfWVnhdikhIKfg95Jzjtf113DQll/TkgL58SQjlZ6by+Zsm8NvdtVqhS2Kagt9D+2tbqDl7Qd08\nEeQLN08kOz2Jv331oNeliISMgt9Da/fVYQYrNE1DxBiZmsSXl01mw+FGNlY2el2OSEgo+D302v46\nri/NJndEitelSD8PLxzH2KxU/uaVg1qoRWKSgt8j1Wfa2V/bom6eCJSa5OPPbp3KrupzvLL3tNfl\niASdgt8jr+7rnXv/tmvGeFyJDOQP5xYxOX8E31lbQXeP3+tyRIJKwe+R53dUM6NwpMaLR6hEXwL/\n+WPTONLQxrNbNV+/xBYFvwcO1Lawt6aF+8oHvOZNIsRtMwqYP2E0/+vF/VQ1tHpdjkjQKPg98Ny2\napJ8xso5mqE6kpkZP1w1h5QkH196ajsXOnu8LkkkKBT8YdbV4+eFnTUsL8tndEay1+XIIAqz0vjB\n/XM4VH+ev1y91+tyRIJCwR9mb1U00NjaySevLxl8Z4kIN0/N40+XTebX26p5duvJwQ8QiXAK/jD7\nt23V5GQks3RafK05EO2+estUFk3K4b+9sJcDtS1elyNyVRT8YdTc1snrB+u4Z24RSZqCOar4Eowf\nrppLVloSX356O+cvXr68tEj0UPqE0ZqdNXT1OD55vUbzRKO8zBT+/oG5HGtq47/+Zo+u6pWopeAP\no+e213DN2JFMLxzpdSkyTAsm5vD1j03jt7tr+fnGY16XIzIsCv4wOXi6hT0159TajwGP3jyJW6bn\n8z/+fT/ff+2QWv4SdRT8YaKx+7EjIcF4/KHr+OT1xfzw9cM89qsdGuMvUUWrf4RBV4+f53ec0tj9\nGJKS6OM7n5zFlPwRfPuVg5xsbufJT5czJivV69JEBqUWfxisP9RAY2uHxu7HGDPjT5ZM4iefLqeq\nvpWVj7/N7uqzXpclMigFfxho7H5su2VGAc99aRGJCQl86h83sXpnjfr9JaKpqyfEzrR1su5AHZ+5\nYbzG7sewsjEjWf3YYh79l2189Zmd/GTDEb5w00TumllIYgT/3bt6/LR1dHOhq4cLnT0f+O13kOQz\nknwJfT+9t9OSfOSMSNY60VFMf7kQW7PrlMbux4ncESn88gsLeW57NT/ZcISvPrOTv32lgj++cQL3\nzythREr4/ndzztHQ2kH1mQtUn7lAzZkL1J+/SGNrJ43nO2ho7aCxtYOz7cO/EO3SB0DuiBRyRyST\nl5nCuJwMJuZmMDEvg9LRGSQnRu6HXjyzSPxKWl5e7rZu3ep1GVfN73fc+v23SEv28ds/vcnrciSM\n/H7H6wfr+cn6I7x3rJnM1EQemF/KDRNzKCvMZMzIVMxs2M/vnOPchS5ONLdzsvkCJ8+0c6K5vS/o\n26k5c4GO7g8uIJOR7CMvM4XcESnv/84dkUJmaiJpyT7SknykJvnev51g0NXj6Pb76erx997ucbR1\ndtPc1vsB0tTWSWNrB02tndS1XKSprfP91/MlGCXZaUzMG8G1Y0cyu2QUs0tGaanREDGzbc658kD2\nVYs/hF7dd5qqhjb+7oG5XpciYZaQYNw6o4BbZxSw48QZfrrhKD/dcIQn1x8BYFR6EmVjMpleOJKy\nMZlkpiZhQO9ngWEGBrR2dNNwvqP3p6+V3nC+g9pzFzl/sfsDr5mdnkRxdjplYzK5ZXoBxdlpfT/p\nFI1KIyMM3zjOXejiaGMbRxpa+363UVnfypsV9fj72pjF2WnMLhnF3JJRzBs/mmuLsvAlDP9DUIZO\nLf4Qcc7x8b9/m/bOHtZ9bYn+wxbOXeii4vR5Dp5u4UBtCwdqz1Nx+jwXuga/BiA1KYG8zBTy+lrr\nBSNTKR2dTnF2OqWj0ykZnUZmalIY3sXwtHd2s7emhV0nz7Kz76fm7AUAstKSWDQph8WTc7lpSi6l\no9Ov6ttQvAp6i9/Mbgd+CPiAnzrnvn3Z49b3+J1AO/BZ59z2vseOAeeBHqA70MKi3ZuHGth3qoW/\nvXeWQl+A3oCbP2E08yeMfn9bj99RfaadC109OEfvD45L7bGMlETyMlPISPZFdRimJyd+6L3Xn7/I\npqom3qls5O3Djbzct7B9cXYaS6bmcds1Y7hhYo7OE4TAoMFvZj7gceBWoBrYYmZrnHP7++12BzCl\n72cB8OO+35csc841Bq3qKPCjNyoZm5XKPXN1pa5cmS/BGJcTn+su52emsnJOESvnFOGc42hjG29X\nNrLhcCPP76jh6XdPkJmSyLKyfD52zRiWTMsL6wnyWBbIv+J8oNI5dwTAzJ4BVgL9g38l8AvX22+0\n2cxGmVmhc6426BVHgXePNLHl2Bn+6hMz1FoRCYCZMTFvBBPzRvCZG8ZzsauHtw83snb/adYdqGfN\nrlMkJyZw0+Rc7p4zlltnFGg46VUI5F+uCOi/7FA1H2zNX2mfIqAWcMA6M+sB/tE59+RAL2JmjwCP\nAJSWlgZUfKT6hzcqyR2RzKr50f0+RLySmuTjlhkF3DKjgO4eP1uPn2Htvjpe2lPL6wfrSU/2cduM\nAlbOLeLGybm6RmaIwvGReaNzrsbM8oHXzOygc2795Tv1fSA8Cb0nd8NQV0jsOnmWDYcb+fPby0hN\n8nldjkjUS/QlsHBiDgsn5vCtu6bz3rFmVu+s4cXdtbyw8xQ5GcncNauQ+64v4dqikVF9LiRcAgn+\nGqD/JDPFfdsC2sc5d+l3vZk9T2/X0YeCP1b86M1KRqYm8vBCtfZFgi0hwd7/EPiru69h/aFGXthZ\nw79uOckvNh2nbEwmnyov4Z65RZoQ8SMEEvxbgClmNoHeMF8FPHjZPmuAx/r6/xcA55xztWaWASQ4\n58733b4N+J/BKz+yHKo7z6v76vjK8skRPbROJBakJPrev1bi3IUu/n3XKX699ST/87f7+T8vH+DW\nGQXcV17CzVPyNLLuMoMGv3Ou28weA16ldzjnz5xz+8zs0b7HnwBeoncoZyW9wzn/qO/wAuD5vq9e\nicAvnXOvBP1dRIgfvVFJerKPP1o8wetSROJKVloSDy8cx8MLx3HwdAu/3lrN8ztqeGnPaYpGpXH/\nvBLun1dCwUhNmw26gCtoTjS1s/S7b/C5GyfwzbtmeF2OSNzr7Paz7kAdv3rvBBsON+JLMFaU5fPg\nglJunpIaEch6AAAI6UlEQVRHQox9C9CUDR748VtVJCYk8IWbJnpdiogAyYkJ3DmzkDtnFnKssY1f\nbTnBv22tZu3+Ooqz03hgfin3zyuJy7mD1OIPgr0151j5+Ds8OL+Uv77nWq/LEZEr6OjuYe2+Op5+\n9zibjzST5DPunFnIwwvHUT4uO6pHBKnFH0bdPX7+/LndjM5I5uu3TfO6HBH5CCmJPj4xeyyfmD2W\nyvrzPLX5BM9tq2b1zlOUjcnk4YXjuGduUcxfIawW/1X68ZtV/M0rB/nxQ9dxx8xCr8sRkSFq7+xm\nzc5T/GLTcfbXtjAiJZE/vK6IhxeOY2pBptflBWwoLX4F/1U40tDK7T/cwPJp+Tzx6eu9LkdEroJz\njh0nz/LU5uP8dnctnd1+FkwYzadvGMdtM8ZE/PQrCv4w8Psdq36ymYO1Laz72hLyNUxMJGY0t3Xy\n660neerd45xsvkBeZgqr5pWwan4pRaPSvC5vQAr+MHhq83G+9cJe/vbeWXxqXsngB4hI1PH7HW8d\nbuCpTcf5XUU9Biwvy+ehBeO4eWpkXRimk7shVnvuAt9++SCLJ+dwX7nW0hWJVQkJxrJp+Syblk/1\nmXaeee8kz2w5yboDWygalcaDC0q5r7yY/Mzo+savFv8QOef43D9vZVNVE6/+2c2U5qR7XZKIhFFn\nt5/X9vcOCd1Y1URi3zKb988r4SYPp4dQiz+E1uw6xe8O1vOtu6Yr9EXiUHJiAnfNKuSuWYVUNbTy\nq3dP8JsdNby8t3d6iPvKi7mvvCRizwWAWvxDUn2mnbv/4R1KRqfzmy8uiqj+PRHxTkd3D+v21/PM\nlt7pIczg5il5fKq8hBXT88MyRbta/CHQ1NrBZ/7fe3T1+PnuJ7WOroj8Xkqi7/1vASeb2/n11pM8\nu7WaL/9yOyNTE/nE7LHce30xc0tGRcTVwWrxB6C1o5sHf7KZitPn+ZfPLfjAgtEiIgPp8Ts2VjXy\nb9uqeXXfaS52+ZmYl8G91xXzB3OLGBvkriAN5wyiju4e/vjnW9h8pJknP309K6YXeF2SiESZ8xe7\neGlPLc9tq+G9Y80AzBufzd2zx3LHzMKgTBSn4A+SHr/jT3+1nZf2nOZ7983m3us1dFNErs6JpnbW\n7Kphza5THKprxZdgLJqUw92zx/Kxa8cwcpiLOCn4g8A5xzdf2Msv3z3Bt+6azuc13bKIBFnF6fPv\nfwicbL7AqPQktnzzlmEtHq+Tu0Hw/dcO8ct3T/DFpZMU+iISEtPGZPKfx5Tx9dumsav6HJX1rcMK\n/aFS8F/mYlcP3375ID/feIz7y0v4Lx/TVMsiElpmxpySUcwpGRWW11Pw91PV0Mpjv9zBgdoW/njx\nBP7izrKIGHolIhJMCn56+/N/va2a/756H2nJPn722XKWl2n0jojEprgP/vMXu/jWC3tZvfMUN0zM\n4Qer5lCgKZZFJIbFbfA759hwuJH/tnov1Wcu8PXbpvLFpZN1Ra6IxLy4C/4ev+OVvaf58VuV7K1p\noWhUGs88spB543U1rojEh7gJ/o7uHn6zvYZ/fKuKY03tTMjN4G/unck9c4tISQz9BEoiIpEipoO/\nx+/YXX2WNw7W88yWk9Sf72BmURY/fug6brtmjLp1RCQuxVzwN7d1sv5QA29W1PPWoQbOtHdhBjdO\nzuX7989h0aQcDdEUkbgWUPCb2e3ADwEf8FPn3Lcve9z6Hr8TaAc+65zbHsixwXKxq4dVT25mV/VZ\nnIOcjGSWTctnybQ8bp6SR3ZGciheVkQk6gwa/GbmAx4HbgWqgS1mtsY5t7/fbncAU/p+FgA/BhYE\neGxQpCb5GJ+TzrJp+SydlsfMoiwS1JUjIvIhgbT45wOVzrkjAGb2DLAS6B/eK4FfuN4Z3zab2Sgz\nKwTGB3Bs0Pxg1dxQPK2ISEwJZDagIuBkv/vVfdsC2SeQYwEws0fMbKuZbW1oaAigLBERGY7QTwMX\nIOfck865cudceV5entfliIjErEC6emqAkn73i/u2BbJPUgDHiohIGAXS4t8CTDGzCWaWDKwC1ly2\nzxrgM9ZrIXDOOVcb4LEiIhJGg7b4nXPdZvYY8Cq9QzJ/5pzbZ2aP9j3+BPASvUM5K+kdzvlHH3Vs\nSN6JiIgEREsviojEgKEsvRgxJ3dFRCQ8FPwiInEmIrt6zKwBOO51HUOUCzR6XUSY6T3HB73n6DDO\nORfQWPiIDP5oZGZbA+1fixV6z/FB7zn2qKtHRCTOKPhFROKMgj94nvS6AA/oPccHvecYoz5+EZE4\noxa/iEicUfCHgJn9JzNzZpbrdS2hZmbfMbODZrbbzJ43s1Fe1xQKZna7mVWYWaWZfcPrekLNzErM\n7A0z229m+8zsq17XFC5m5jOzHWb2W69rCRUFf5CZWQlwG3DC61rC5DXgWufcLOAQ8F89rifo+q0k\ndwcwA3jAzGZ4W1XIdQP/yTk3A1gIfDkO3vMlXwUOeF1EKCn4g+/7wH8B4uLkiXNurXOuu+/uZnqn\n3o41769C55zrBC6tJBeznHO1l9bNds6dpzcIB1xEKZaYWTFwF/BTr2sJJQV/EJnZSqDGObfL61o8\n8sfAy14XEQIBryQXi8xsPDAXeNfbSsLiB/Q23PxeFxJKgSzEIv2Y2TpgzAAPfRP4C3q7eWLKR71n\n59zqvn2+SW/3wNPhrE1Cy8xGAM8Bf+aca/G6nlAys48D9c65bWa21Ot6QknBP0TOuVsG2m5mM4EJ\nwC4zg94uj+1mNt85dzqMJQbdld7zJWb2WeDjwAoXm+ODA1mFLuaYWRK9of+0c+43XtcTBouBu83s\nTiAVGGlmTznnHva4rqDTOP4QMbNjQLlzLtomehoSM7sd+L/AEudcg9f1hIKZJdJ74noFvYG/BXgw\nlhcVst7Wyz8Dzc65P/O6nnDra/F/3Tn3ca9rCQX18cvV+gcgE3jNzHaa2RNeFxRsfSevL60kdwB4\nNpZDv89i4NPA8r6/686+lrDEALX4RUTijFr8IiJxRsEvIhJnFPwiInFGwS8iEmcU/CIicUbBLyIS\nZxT8IiJxRsEvIhJn/j+CW3rEy7CQlAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# let p(x) be the target distribution; example from Koller PGM text ex. 8.13, section 8.5.1, p. 275\n", "pi1, mu1, sig1 = .7, -2, 1\n", "pi2, mu2, sig2 = .3, 2, 2\n", "def p_x(x):\n", " c1 = norm(mu1, sig1)\n", " c2 = norm(mu2, sig2)\n", " return pi1 * c1.pdf(x) + pi2 * c2.pdf(x)\n", "\n", "# plot p(x)\n", "xs = np.linspace(-5, 5)\n", "plt.figure()\n", "plt.plot(xs, p_x(xs))\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We want to project $p$ onto the set of univariate normal distributions, $Q = \\{\\mathcal{N(\\mu, \\sigma^2)} \\}$. Recall the commonly used distance measure, KL divergence (i.e., relative entropy) is not symmetric in its two arguments. Therefore we have two types of projections." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "The **M-projection** is defined as \n", "$$ q^M = \\arg \\min_{q \\in Q} D(p \\| q), D(p \\| q) = E_p[ \\log p(X) - \\log q(X)] = -H[p] - E_p[\\log q(X)] $$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If $q(x)$ has some nice structure, then this optimization problem can be made easier. Indeed this is the case for exponential family distributions (so that $\\log q(x)$ is a linear function of parameters). \n", "Theorem 8.6 says that if $Q$ is an exponential family with sufficient statistics $\\psi(x)$ and parameterized by $\\phi$, then the M-projection $q^M$, is equal to $q_\\phi$ if $\\mathbb{E}_{q_\\phi}[\\psi(X)] = \\mathbb{E}_{p}[\\psi(X)]$. In words, $q_\\phi \\in Q$ is the M-projection if it matches the expected sufficient statistics induced by $p$.\n", "\n", "In our case, the univeriate Gaussian exponential family is defined by the vector of sufficient statisitics $[x, x^2]$, and for convenience let's parameterize $Q$ in mean parameters $\\mathbb{E}[X] = \\mu, \\mathbb{E}[X^2] = \\sigma^2 + \\mu^2 $ (instead of natural parameters); it's easy to show that $q^M$ must have $[\\hat \\mu, \\hat \\sigma^2]$ as its mean parameters, where \n", "$$ \\hat \\mu = \\mathbb{E}_p[X] $$\n", "and\n", "$$ \\hat \\sigma^2 = \\mathbb{E}_p[X^2] - (\\mathbb{E}_p[X])^2 = \\mathbb{E}_p[X^2] - \\hat \\mu^2 $$\n", "i.e., the mean and variance of the Gaussian M-projection must match those of the target distribution. \n", "\n", "In the example, we have direct access to $p$, so we could use numerical integration or sampling to find its first and second moments and thus $\\hat \\mu$ and $ \\hat \\sigma^2$. \n", "Note that in the general setting of learning a probabilistic model from data, we are often provided a set of samples $\\mathcal{S}$ from the unknwon distribution $p$ and attempt to learn an approximation $q$ by maximum likelihood--and this corresponds precisely to finding the M-projection of the empirical distribution $\\hat p$: $$q^{MLE} = \\arg \\max_{q\\in Q} \\sum_{x \\in \\mathcal{S}} \\log q(x) = \\arg \\min_{q\\in Q} -\\mathbb{E}_{\\hat p} [\\log q(X)] = \\arg \\min_{q\\in Q} D(\\hat p \\| q)$$" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "D(p||q) = 0.161106854028\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8VfX5wPHPk52QkEAGI4MNYSOGjWwQtAKiCIh7z7aO\nbqu1u7/Wqm0dReuqIg5UUBQBRfbeG8JOCGQB2fv7++Pc0BADXJK787xfr/vKvfes59wkzz3nO8UY\ng1JKqcbDz90BKKWUci1N/Eop1cho4ldKqUZGE79SSjUymviVUqqR0cSvlFKNjCZ+5RVEpEBE2rs7\nDk8lIjNFZJG37Fe5lyZ+hYh8JyKnRSTY3bFciDEm3BhzyNH7FZE7RGRlA7c3IvJ8rfcn2d5/6yLb\nficiJbYvtWwR+UREWtUnDmPMe8aYcfXZtkY8bW0xBzhyv8rzaOJv5ESkLXAVYICJTjxOwKXX8loH\ngZtqnePtwH47tn3EGBMOdAaigOfrWklE/BscpVI2mvjVbcBa4C2sZHWOiLwlIq+KyGIRyReRZSLS\npsZyIyI/FJFDtivWv4qIn23ZHSKySkSeF5Ec4Dci4iciT4nIURHJFJF3RCTStv40ETksIk1tryeI\nyEkRia1xrI414npZRL6yXS2vEpGWIvKC7c5lr4hcUSPOn4vIQds57BaR623vdwVeBQbZ9nPG9n6w\niPxNRI6JyCnbZxB6kc/wJLADuNq2fXNgMDDf3l+CMSYXmAv0qHGOr4jIlyJSCIwUkUjbZ5Zl+wyf\nqvV5n7tzEZFk2+8tV0T2ichNNZaFishztn2cFZGVtvNbblvljO3zGFTHfgeLyAbbdhtEZHCNZd+J\nyO9sv498EVkkIjH2fgbKdTTxq9uA92yPq0WkRa3lM4HfATHAVtt6NV0PpAB9gUnAXTWWDQAOAS2A\nPwB32B4jgfZAOPAvAGPMB8Bq4B8iEg38B7jHGJN1gbhvAp6yxVUKrAE2215/DPy9xroHse5qIoFn\ngXdFpJUxZg/wALDGVpQUZVv/z1hX4H2AjkA88PQF4qj2DtZnCTAdmGeLyy62BHkDsKXG2zdjfW4R\nwErgn7ZzaA8Mtx3vzjr21QRYDMwG4mzxvCwi3Wyr/A24EuvLqTnwU6AKGGZbHmX7PNbU2m9zYAHw\nDyAa6zNeYPt91Yz5Tttxg4An7f0MlAsZY/TRSB/AUKAciLG93gs8VmP5W8CcGq/DgUog0fbaAONr\nLH8I+Mb2/A7gWK3jfQM8VON1F9vxA2yvo4BjWFfP/661rQE61ojrtRrLHgX21HjdEzhzkfPeCkyq\nEefKGssEKAQ61HhvEHD4Avu6AysphwKnsBLzWmAI8HvgrYvE8R1QBJwB0rG+VGNrnOM7Ndb1B8qA\nbjXeux/4rvZ5ANOAFbWO9W/gGayLvWKgdx3xtLV9zgG1z8/2/FZgfa1t1gB31Difp2r9PSx099+5\nPr7/0Cv+xu12YJExJtv2eja1inuA49VPjDEFQC7Quq7lwNGLLMO27Git9QOw7ggwxpwBPsIq7nju\nErGfqvG8uI7X4dUvROQ2EdkqImdsxTk9sO4M6hILhAGbaqy/0Pb+BRljirGuhp8Coo0xq2outxUX\nFdgev6yx6IfGmChjTLwxZqY5/w6n5ucXAwTy/c8vvo5w2gADquO3ncNMoKVtPyFYd0GXq/bvr64Y\nTtZ4XkSN34PyHL5c4aYuwlamexPgLyLV/6zBQJSI9DbGbLO9l1hjm3CsooETNXaVCOyyPU+qtaz2\n0K8nsJJStSSgAlvSFpE+WEVF72MVJ4yv18nVYKuTeA0YjVWkUykiW7Gu7OuKMRvri6O7MSb9Mg/3\nDvAtVnHSeYwxD2AVK12OmrFlY90dtQF2295LwrpTqO04sMwYM7b2AludQAnQAdhWa/Glhuqt/fur\njmHhJbZTHkav+BuvyVjFNt2wyrL7AF2BFfyvrBrgGhEZKiJBWGX9a40xNa9EfyIizUQkEfgR8MFF\njvk+8JiItLN9ifwR+MAYUyEiIcC7wC+xyojjReQhB5xnE6yElgUgIndiq0C1OQUk2M4PY0wV1hfF\n8yISZ9smXkSutuNYy4CxWGXxDmWMqQQ+BP4gIhG2L7THsT6z2r4AOovIrSISaHv0E5GutvN7A/i7\niLQWEX9bJW4w1mdUhVWHUJcvbfu9WUQCRGQa1t/PFw4+XeVkmvgbr9uBN40xx4wxJ6sfWJWtM+V/\nTRNnY5UN52JVCN5Saz/zgE1Y5eYLsCplL+QN4L9YrUcOY115Pmpb9ifguDHmFWNMqe04vxeRTg05\nSWPMbqxiozVYSb4nULMY5lusO5aTIlJd5PUzIBVYKyJ5wBKs+ohLHcsYY74xVgsdZ3gUq/7hEFa9\nwmysz7R2HPnAOKxK3RNYxS9/wbqjA6vCdQewAev3+hfAzxhThFWZvMpWRDSw1n5zgB8ATwA5WJXC\nP6hRVKi8hBijE7GouonV+SjNGPPUBZYboJMxJtWlganvEZG7gFuMMaPcHYvyfHrFr5Rv6I51F6XU\nJWnlrlJeTkQ+AzoBU90di/IOdhX1iMh44EWstsSvG2P+XGv5JKyKvyqsVho/NsastGdbpZRSrnXJ\nxC/WGCH7sVorpGFVCM2wVZpVrxMOFBpjjIj0Aj40xiTbs61SSinXsqeopz+QamwjI4rIHKyu+eeS\nt61jT7Xq5nN2bVuXmJgY07ZtWztPQSml1KZNm7KNMRftaFjNnsQfz/k9CNOwxmA5j23gqz9hjdFx\n7eVsa9v+PuA+gKSkJDZu3GhHaEoppQBEpHav6gtyWKseY8ynxphkrI5Bv6vH9rOMMSnGmJTYWLu+\ntJRSStWDPYk/nRrd9oEE6u4mDoAxZjnQ3jba4GVtq5RSyvnsSfwbgE62bvZBWL0BzxtnXEQ6iojY\nnvfF6iGYY8+2SimlXOuSZfy2cVQeAb7GapL5hjFml4g8YFv+KtY44reJSDnWAFfTjNVcqM5tnXQu\nSilFeXk5aWlplJSUuDsUpwgJCSEhIYHAwMB678Mjh2xISUkxWrmrlKqPw4cPExERQXR0NLaCCJ9h\njCEnJ4f8/HzatWt33jIR2WSMSbFnPzpkg1LKp5SUlPhk0gcQEaKjoxt8N6OJXynlc3wx6VdzxLlp\n4lf1VlpRyWdb0lmdqqPyKuVNdJA2ddmy8kt5b91R3l17lOyCMkTgyXFdeGhEB5++0lLKV2jiV3bb\nfSKPN1cdZt7WE5RVVjEqOY7bBrXh0y3p/PXrfezJyOOvN/YmNMjf3aEqpS5CE7+6pPLKKh58dzNL\n9pwiNNCf6f0TuWNwW9rHWvNoD+8cS9dWTfnLwr0czi5k1m0pxEeFujlqpTxTcXEx48eP59tvv8Xf\nv+6LpLKyMsaMGcO3335LQIDj07SW8atLenv1EZbsOcUPR3Vk7S9G89tJPc4lfbAqmx4Y3oE3bu/H\nsdwiJv5zJesPO2v2QaW82xtvvMGUKVMumPQBgoKCGD16NB98cLEprOtPE7+6qFN5Jbyw5AAju8Ty\n2NjORIZduNPIyOQ4Pnt4CJGhgdz82lo+2HDMhZEq5VlmzJjBtGnT6N+/P23atGHBggUAvPfee0ya\nNOnceiNHjmTx4sUAPPXUUzz6qDUN9eTJk3nvvfecEpsW9aiL+uOXeyirrOI3E7vbVXHbITacTx8e\nwiOzN/PLT3cyoF00bWOauCBSpb7v2c93sftEnkP32a11U565rvsl19u2bRuTJk3igw8+YOXKlTz+\n+OOMHTuWQ4cOUXPY+WeffZann36azMxMtmzZwvz51qg2PXr0YMOGDQ6NvZpe8asLWnMwh3lbT/DA\n8A60ibY/eUeGBvLcTb0J9Bf+8c0BJ0aolGcqKSkhKyuLZ555BoBu3bpx+vRpsrOziYqKOm/dYcOG\nYYzh73//O3PmzDlXBOTv709QUBD5+fkOj0+v+FWdyiureHreThKahfLQiA6XvX1cRAi3D2rLrBWH\neHBEBzq1iHBClEpdnD1X5s6wc+dOOnXqREhICACbN2+md+/ehIaGfq/X7Y4dO8jIyCA6OpqIiPP/\nT0pLS8/tw5H0il/V6c1VhzmQWcBvrutOSGD9mmfeP7wDYYH+vLBEr/pV47Jt2zaOHTtGSUkJhYWF\nPPPMMzz22GM0a9aMysrKc8k/IyODmTNnMm/ePMLDw1m4cOG5feTk5BATE9OgwdguRBO/+p6Ms8W8\nsOQAo5PjGNOtRb3307xJEHcPbceCHRnsOnHWgREq5dm2bdvGlClTGDBgAP369ePBBx9kyJAhAIwb\nN46VK1dSVFTElClTeO655+jatSu//vWvefbZZ8/tY+nSpVx77bUXOkSDaOJX3/OHBXuoqDIOuU2+\n+6r2NA0J4PnF+x0QmVLeYdu2bTz88MNs27aN3bt3c8stt5xb9vDDD/P2228TFhbGmjVrGDt2LGCV\n9a9Zs+bcerNnz+b+++93Snya+NV5VqVm88X2DB4a0YGk6LAG7y8yNJD7h3dgyZ5Mth4/44AIlfJ8\nBw8epFOnTnUu69u3LyNHjqSysvKC25eVlTF58mQ6d+7slPg08atzyiqsCt2k5mE8MPzyK3Qv5I7B\nbWneJIjnFu1z2D6V8mRpaWn4+V04vd51112X7MB12223OSM0QBO/qmHR7pMczCrkV9d2rXeFbl2a\nBAfw4PAOrDiQzbpDOQ7br1KqfjTxq3PmbkqjVWQIY7rWv0L3Qm4Z2Ia4iGCeW7wfT5z1TanGRBO/\nAiAzv4TlB7KZfEU8/n6OH1o5NMifR0Z1ZP3hXFal6lW/Uu6kiV8BMH/rCSqrDDf0jXfaMab1S6R1\nZAh/W7RPr/qVciNN/AqAuZvT6Z0QScc45/WwDQ7w55FRndh6/Aybjp522nGUUheniV+x+0QeezLy\nuOHKBKcfa2Kf1oQE+vHZ1nSnH0spVTdN/Iq5m9MI9Beu69Xa6ccKDw5gbLeWLNieQXllldOPp5T6\nPk38jVxFZRXztqYzKjmOZk2CXHLMyX1ac7qonOX7s1xyPKXU+TTxN3LLD2SRXVDGDX2dX8xTbVjn\nWJqFBfLZ1hMuO6ZSnqK4uJjhw4dfsufusGHDqKiocEoMmvgbubmb02kWFsiILnEuO2agvx/X9mrF\n4t0nKSh1zh+2Up7Ka6ZeFJHxIrJPRFJF5Od1LJ8pIttFZIeIrBaR3jWWHbG9v1VENjoyeNUwZ4vK\nWbz7FJP6xBMU4NprgMl94ikpr2LRrpMuPa5SrnLgwAFGjBhBjx49eOKJJ+jQwRoGxSumXhQRf+Al\nYCyQBmwQkfnGmN01VjsMDDfGnBaRCcAsYECN5SONMdkOjFs5wIIdGZRVVLm0mKda36RmxEeF8tnW\nE0xxw/FVI/HVz+HkDsfus2VPmPDni65SWVnJbbfdxksvvUTfvn159NFH6d69O2VlZV4z9WJ/INUY\nc8gYUwbMASbVXMEYs9oYU90wey2g/8leYO7mNDrFhdMjvqnLj+3nJ0zq05qVB7LIyi91+fGVcqbP\nPvuMbt260bdvXwC6du1K7969vWrqxXjgeI3XaZx/NV/b3cBXNV4bYImIVAL/NsbMqmsjEbkPuA8g\nKSnJjrBUQxzOLmTT0dP8fEKyXZOoO8PkK+J5+buDfLH9BHcOaeeWGJSPu8SVubNs2bKFPn36nHu9\nbds2xowZ45tTL4rISKzE/7Mabw81xvQBJgAPi8iwurY1xswyxqQYY1JiY2MdGZaqw6eb0/ATq6zd\nXTq3iKBrq6baukf5nOjoaPbu3QvAunXreOedd+jdu7dXTb2YDiTWeJ1ge+88ItILeB2YZIw5NwqX\nMSbd9jMT+BSr6Ei5UVWVYe7mdIZ0jKFlpOOvJi7H5D6t2Xb8DEeyC90ah1KOdOutt7Jx40Z69uzJ\nJ598QnR0NB07dgS8Z+rFDUAnEWknIkHAdGB+zRVEJAn4BLjVGLO/xvtNRCSi+jkwDtjpqOBV/aw/\nkkv6mWK3VOrWNrFPa0Rgnl71Kx8SExPDunXr2LFjB4888gjx8fHnJmbxiqkXjTEVwCPA18Ae4ENj\nzC4ReUBEHrCt9jQQDbxcq9lmC2CliGwD1gMLjDELUW61cOdJggP8GNfd8ePuX65WkaEMaNeceVvT\ndcRO5ZO2bdtGr169zr32hKkXxRP/2VJSUszGjdrk31lG/HUp7WKa8OadnlHqNmf9MX7+yQ7mPzKE\nXglRl95AqYvYs2cPXbt2dXcYTlXXOYrIJmNMij3ba8/dRuZwdiFHcooYmey6nrqXMqFnK4L8/fhs\nixb3KOUKmvgbmaV7MwEY0dlzEn9kaCAjk2P5fLs1GYxSyrk08TcyS/dl0iG2CUnRYe4O5TyT+8ST\nlV/KmoM6LaNqOE8swnYUR5ybJv5GpKisgnWHchnpwgHZ7DUyOY7QQH8W79axe1TDhISEkJOT45PJ\n3xhDTk5Ogzt12dNzV/mI1ak5lFVWeVT5frWQQH+GdIxhyZ5MfjPRuK03sfJ+CQkJpKWlkZXlm/M9\nhISEkJDQsKbYmvgbkaX7MmkS5E9K22buDqVOo7vGsWTPKfafKqBLS+fN/at8W2BgIO3a6RAgF6NF\nPY2EMYbv9mUxpGMMwQEXHgfcnUbZ7kSW7Dnl5kiU8m2a+BuJA5kFpJ8p9shinmotmobQMz6Sb20t\nj5RSzqGJv5E414yzi2cPgDcqOY7Nx06TW1jm7lCU8lma+BuJpfsySW4ZQavIUHeHclFjurbAmP99\nUSmlHE8TfyOQV1LOxiOnPbqYp1r31k2JiwjW4h6lnEgTfyOw6kA2FVXGI9vv1+bnJ4zuGsey/VmU\nVVS5OxylfJIm/kZg6b5MIkIC6JvkHQOgjUpuQUFpBRuO5Lo7FKV8kiZ+H2eMYem+LIZ1jiXA3zt+\n3UM6RhMU4KfNOpVyEu/IBKredp3IIyu/1CuKeaqFBQUwpEM03+zJ9Mlu90q5myZ+H/fdPquSdHhn\nz27GWduori04llvEwSydklEpR9PE7+OW7suiV0IksRHB7g7lsoy2tUD6Rot7lHI4Tfw+7HRhGVuO\nnWaEFxXzVGsdFUrXVk35Rpt1KuVwmvh92PIDWVQZGOnhvXUvZHRyHJuOnuZMkfbiVcqRNPH7sGX7\nsmgWFui189iO7hpHZZVh2X7fHF5XKXfRxO+jjDGsOpjN4I4x+Pt559j2vROiiAkPYskeLe5RypE0\n8fuoQ9mFnMorZXCHaHeHUm9+fsLILnEs25dJeaX24lXKUTTx+6jVtrlrh3SIcXMkDTO6axx5JRVs\nOnra3aEo5TM08fuoNQezaR0ZQhsPm1T9cg3tFEuQv58261TKgTTx+6CqKsOagzkM6hDj9XPXhgcH\n0K9dM63gVcqBNPH7oD0n8zhdVM6Qjt5bvl/TiM5x7D9VwIkzxe4ORSmfYFfiF5HxIrJPRFJF5Od1\nLJ8pIttFZIeIrBaR3vZuqxxvja18f5AXV+zWVD1r2Hf79KpfKUe4ZOIXEX/gJWAC0A2YISLdaq12\nGBhujOkJ/A6YdRnbKgdbfTCH9jFNPH62LXt1jAundWQIy/Zrs06lHMGeK/7+QKox5pAxpgyYA0yq\nuYIxZrUxprrZxVogwd5tlWOVV1ax7lCOz1ztA4gIw7vEsSo1RydnUcoB7En88cDxGq/TbO9dyN3A\nV5e7rYjcJyIbRWRjVpbe0tfX9rSzFJZVMqSjdzfjrG1El1gKSivYfEybdSrVUA6t3BWRkViJ/2eX\nu60xZpYxJsUYkxIb651jy3iCNQezARjY3neu+AEGd4gmwE+0nF8pB7An8acDiTVeJ9jeO4+I9AJe\nByYZY3IuZ1vlOKsP5tC1VVOaNwlydygOFRESSEpbbdaplCPYk/g3AJ1EpJ2IBAHTgfk1VxCRJOAT\n4FZjzP7L2VY5Tkl5JRuPnmaID5Xv1zSiSxx7MvI4lVfi7lCU8mqXTPzGmArgEeBrYA/woTFml4g8\nICIP2FZ7GogGXhaRrSKy8WLbOuE8FLD56GnKKqoY7CPt92urnkVsmRb3KNUgAfasZIz5Eviy1nuv\n1nh+D3CPvdsq51h9MAd/P6Ff2+buDsVxjIHM3ZB9gOS8dP4Qtpb2y16D7cWQdwJMJYS3gIiW1s/w\nFhDRApq3h6RBEOBdM48p5Qp2JX7lHVYfzKZ3QiQRIYHuDqVhKivg2GrYu8B6nLUahgkwVYI5UdCc\nqpiO+LUZDOIHBSfh9FE4vh6Ksv+3n6Bw6DASOk+ATuMgXBsNKAWa+H1Gfkk529LO8uDwDu4OpX6q\nKmH/QtjzufWz+DQEhECHUTD8Z9D6CmjamiWpJTw0ewsfjxhESl13NpXlUJAJp3Za+9ln2ycCCSnQ\n5Rroezs08c3iMKXsoYnfR2w4kktllfG+8ferKmHXp7DsL5C9H0KbWVfoyddYST+oyXmrD+lUjr+t\nWWedid8/ECLjrUfnq+Hav8PJ7dYXwP6F8M2zsPxv0P9eGPwoNPGt/g5K2UMTv49YnZpDUIAffds0\nc3co9qmqgt2fWQk/ay/EdoWpb0PyD8D/wn+WkaGB9E2KYtn+LJ68usuljyMCrXpbjxE/g8y9sPyv\nsOpFWD8L+t0Ng38I4d43Ib1S9aWjc/qI1QdzSGnTjJBAf3eHcnHGwO558OoQ+PhO670b34QHV0P3\nyRdN+tVGdIljR/pZsvJLL//4cclw43/g4fXQ9TpY8xK80AsW/hKKci9/f0p5IU38PiC3sIzdGXme\nX8xzNh3+ez18eJtVFn/Df6yE32MK+Nn/p1jdrHN5QzpzxXaGKbPg4Q3Q/XpY9yq8PBD2fXXpbZXy\ncpr4fcDaQ1ZH6cGeOj6PMbDtA3h5EBxfB9f8DR5eBz1vBL/Lv0Pp1qopMeHBjunFG9MRrn8F7l8G\nTWLh/enw6YNQfKbh+1bKQ2ni9wGrD2YTHhxAr/hId4fyfYXZ8OGt8Ol9VjHLAyutitV6JPxqfn7C\n8M6xLD+QRWWVcUycLXvCvUth2E9gu+1LKnWJY/atlIfRxO8DVh/MoV/bZgT4e9ivc++XVvHJ/q9h\nzLNw51cQ7ZjmpiO6xHKmqJztaQ68Mg8IglFPwT2LITgC3r0B5v8QSvMddwylPICHZQp1uU7llXAo\nq9CzhmGurIAFT8KcGRDeEu77Dob+uEFX+bVd1SkGP3HSrFzxV8L9y63WPpvfgddGQ+4hxx9HKTfR\nxO/lqqdZ9JhhmEvOwuypsOE1GPgw3PsttOju8MNEhQXRJzGK7/Y5aVauwBAY9zu4fT4UZsJro+DI\nSuccSykX08Tv5VYfzCYyNJBurZq6OxTIPQz/GQeHl8PEf8L4P1rFJ04yKjmObWn1bNZpr3bD4J5v\nrIrfdyZbdwBKeTlN/F5uzaEcBrZvjp+fuDeQY2vh9dGQfxJu/RT63ub0Q45KbgHgvKv+atEd4O7F\n0O4qmP8ofP0rq8exUl5KE78XO55bxPHcYga5u5hn+4fw9nUQEmldHbcb5pLDdm0VQavIEL7d64JJ\n2EOj4OaPoP99sOZf8P4MKMlz/nGVcgJN/F5sja39/qAObqrYNQa++wt8ci8k9LeSfkxHlx1eRBiZ\nHMeKA9mumYTdPwCu+avVDyF1Cbx5jdVcVSkvo4nfi609mEN0kyA6twh3/cGNsQY8++6P0HuGVbwT\n5vp5AEZ1iaOgtIINR1w43EL/e2Hmh5BzAN76gTUaqFJeRBO/lzLGWOX7HaIRcXH5vjGw+New8nlI\nuQsmvezUStyLGdIxhuAAP77Z4+Lk23EMzPwIzhyFt66FvAzXHl+pBtDE76WO5BSRcbbE9ePzGANf\n/xJW/9Mq777275c1zo6jhQb5M7hDNN/uPeX6g7cbBrfMtWYCe+saOJvm+hiUqgdN/F6quv2+Syt2\njYGvfgprX4YBD8KE/7OGPXazUclxHMkp4lBWgesP3mawVcxVmG2V+Z8+6voYlLpMmvi91OqD2bRo\nGky7mCaXXtkRqqpgwePWGPaDH4Xxf/KIpA8wMtkaS98lrXvqktgfbvsMSs5YxT7ay1d5OE38XsgY\nw9pDuQzuEOOa8n1j4Isfw8Y3YOhjMPZ3HpP0ARKahZHcMsL15fw1xV8Jt38OZQXw5rV65a88miZ+\nL5SaWUB2QanrinkWPw2b34arnoTRz3hU0q82MjmODUdyySspd18QrXrD7V9AeaE170CBE8YRUsoB\nNPF7odXV5fuuqNhd9Q9Y/Q/od681cqUHJn2A0clxVFQZVux3c7v6lj3g5g+tCt/3btBOXsojaeL3\nQmsO5pDQLJTE5mHOPdDW2Vazze5TPKYi90KuSGpGVFgg37ijdU9tSQPhprfh5E74YCaUl7g7IqXO\no4nfy1RVGdYeznF+Mc++hTDvEWg/Aq5/1a1NNu3h7yeM6BzLd/scODlLQ3S+Gia/Yg1Y98k9OraP\n8iie/d+svmfPyTzOFJU7t5jn2Fr46A5o1QumvQsBwc47lgON6tqC3MIytjlycpaG6D0Nrv4T7Pkc\nvnjMqiRXygPYlfhFZLyI7BORVBH5eR3Lk0VkjYiUisiTtZYdEZEdIrJVRDY6KvDGao2zy/dP7YbZ\nN0FkPMz82JqJyksM7xSLv5/wrTtb99Q26CEY+rhVOf7t79wdjVKAHYlfRPyBl4AJQDdghoh0q7Va\nLvBD4G8X2M1IY0wfY0xKQ4JVVuJvF9OEVpGhjt/52TR4dwoEhsEtn0ATD5rVyw6RYYFc2aYZ37ir\nPf+FjH4a+t4OK56DDa+7Oxql7Lri7w+kGmMOGWPKgDnApJorGGMyjTEbADe2pfN9FZVVrD+c65zZ\ntkoLYPZ0KCu0hiFo1sbxx3CB0clx7MnII+NssbtD+R8R+MHz0Olq+PKnOom7cjt7En88cLzG6zTb\ne/YywBIR2SQi911oJRG5T0Q2isjGrCxt/1yXXSfyyC+tcPz4PFWVMPceyNwNU990ylSJrjK6q5t7\n8V6Inz/c+B+ITYaP7oTMPe6OSDVirqjcHWqM6YNVVPSwiNQ5S4cxZpYxJsUYkxIbG+uCsLzPamfN\nr7v4adj/FUz4izXqpBfrEBtOYvNQzyrnrxYcATd/AAEhVj2KdvBSbmJP4k8HEmu8TrC9ZxdjTLrt\nZybwKVbOLvg6AAAgAElEQVTRkaqHNYdy6NwinNgIB7ay2fimNaNU//utcea9nIgwOrkFqw5mU1RW\n4e5wvi8qEW6eYyX9OTdrG3/lFvYk/g1AJxFpJyJBwHRgvj07F5EmIhJR/RwYB+ysb7CNWVlFFRsO\n5zq2/f7BpbDgCeg4Fq7+o+P262ZXd29JSXmV5xX3VIu/0uobkbYe5j2szTyVy10y8RtjKoBHgK+B\nPcCHxphdIvKAiDwAICItRSQNeBx4SkTSRKQp0AJYKSLbgPXAAmPMQmedjC/bcuw0xeWVjptmMWs/\nfHg7xHaBG9+wphX0Ef3bNScmPJgvd3jw5CjdJ1utfXZ+DN/92d3RqEbGrv92Y8yXwJe13nu1xvOT\nWEVAteUBvRsSoLKsOJCNv58wuKMDrvgLc6wy5oAgq8w5pGnD9+lB/P2ECT1a8tGm4xSVVRAW5KFf\nakMfh5yDsOzPENMJet7o7ohUI6E9d73EitRs+iRG0TQksGE7qiyHj263BhGb/j5EJTkmQA9zba9W\nnl3cA7Zmni9A0mCryOfEVndHpBoJTfxe4ExRGdvTznBVJwcU8yz6NRxZAde9CIn9Gr4/D9WvrRcU\n94B113XTOxAWA3Nmaksf5RKa+L3AqtQcjIGrOjWwmevW2bDuFRj4EPSZ4ZjgPJS/n3BNz5Z8uzfT\nM1v31BQeC9PfhaJs626sUvtBKufSxO8FVqZmERESQO+EyPrvJG0TfP5ja4LwsY1jzJhrenpBcU+1\n1lfAxH/B0VWw8HvDYSnlUJr4PZwxhuX7sxncIZoA/3r+uvJPwQe3QEQLuPEtn2rBczH92jYnNiKY\nBds9vLinWq+p1nzGG16HTW+7OxrlwzTxe7jD2YWknymufzFPRSl8eKs1Efj02dDERdM1eoDq1j1L\n92VSWOrhxT3VxjwLHUZZ/SuOrXN3NMpHaeL3cCtTrakE612x+9VP4fg6mPQStOzpwMi8g1cV94Bt\nTJ83IDLB+sLOO+HuiJQP0sTv4ZbvzyapeRhtoptc/sYb34BNb1ntxXtMcXhs3qC6uMfjW/fUFNoM\nZrxvjZT6wa3WXZtSDqSJ34OVV1ax9lBO/a72j6+3hgDuOMaaJL2R8sriHoC4rtbUjekb4aufuTsa\n5WM08XuwrcfPUFBacfmJvyATPrzNmkVrymtW8UEjdq23FfdU6zYRhvwYNr0Jm//r7miUD9HE78FW\n7M/CT7i88Xkqy635covPWPPlhjV3WnzeIsUbi3uqjfq1NeH9gicgfbO7o1E+QhO/B1t+IJveiVFE\nhl7GMA2Ln7Hagk/8R6OszK2Lv59wTQ+rM5dXFfeA1fT2hjcgPM4q7y/MdndEygdo4vdQZ4vKbcM0\nXEYzzh0fw9qXYMAD0Osm5wXnha7p2YrSCi8s7gGrCe60/0JhFnx8F1R62ZeX8jia+D3U6oPZVBkY\nZm/5/qldMP9Ra8Cvcb93bnBeKKVtc+K8qTNXba2vsObtPbwMvv2tu6NRXk4Tv4dafiCb8OAAeidG\nXXrl4jPWAF/BTWHqW+DfwBE8fZDXtu6p6YqZkHIXrHoRds9zdzTKi2ni90DGGFYcyGJQh2gCLzVM\nQ1UVfHo/nE2zRnmMaOGaIL3QxD6tKa2o8t6rfoDxf4aEfvDZQ5C1z93RKC+lid8DHc0pIu10sX3F\nPMv/CvsXwvg/QdIA5wfnxfomNaNzi3DeW3fU3aHUX0Cw9QUfGGrd5ZXkuTsi5YU08XugFQesMdmH\nXqpi98Bi+O5P0Gs69LvHBZF5NxFh5oA2bEs7y460s+4Op/6atraK9HIPwbyHdM5eddk08XugFQey\nSWgWStvosAuvlHsY5t4NLXpYlX4irgvQi13fN57QQH9mr/fiq36AtkNh7G9hz+ew6gV3R6O8jCZ+\nD1NeWcWagzlc1SkWuVAyLyuy2nQjVjO/oIt8QajzNA0JZGLv1szbeoK8Ei+f8GTQw9B9CnzzWzj0\nnbujUV5EE7+H2Xr8DPkXG6bBGPjiMTi1E254HZq3c22APmDmwCSKyiqZtyXd3aE0jAhM/CfEdLHa\n95857u6IlJfQxO9hluw+RaC/MPRCiX/D67B9Doz4BXQa69rgfESvhCh6xkfy7tpjGG8vHw8Ot4bm\nqCy3hnEuL3F3RMoLaOL3IMYYFu0+xcD20TQNqaMt/rF11rR8ncfDsJ+4PkAfMnNAEvtO5bPp6Gl3\nh9JwMR3h+lfhxBb4Sv8u1KVp4vcgB7MKOJxdyLhudbTFzz9lTcQdmQjX/xv89FfXEBP7tCYiOID3\n1h1zdyiOkXwtXPUEbH7HmoNBqYvQ7OFBFu0+BcCY2om/osxK+iVnrdv6UDt686qLCgsKYErfeBbs\nyCC3sMzd4TjGyF9Bh9Hw5U/g+AZ3R6M8mF2JX0TGi8g+EUkVkZ/XsTxZRNaISKmIPHk526r/Wbz7\nFL0SImkVGXr+gq9/CcfWwKR/Qcse7gnOB908oA1lFVXM3ZTm7lAcw8/fqvBv2toq788/5e6IlIe6\nZOIXEX/gJWAC0A2YISLdaq2WC/wQ+Fs9tlVAZl4JW46dYWzXWlf7W96FDa/B4B9CjxvcE5yP6tIy\ngn5tmzF7/TGqqry8krdaWHOY9p51d/jR7dbdolK12HPF3x9INcYcMsaUAXOASTVXMMZkGmM2ALUb\nRl9yW2VZsscaLnhs9xqJP30TfPG4NRHH6GfcEpevmzmgDYezC1lzKMfdoThOyx5WM89ja2DRr9wd\njfJA9iT+eKBmA+E023v2aMi2jcri3SdJah5GlxYR1hsFWVYnrYgWcOOb1oQcyuHG92hJs7BA7x6/\npy49b4RBj8D6WbDlPXdHozyMx1Tuish9IrJRRDZmZWW5OxyXKiitYFVqDmO7tbB661ZPn1iUa922\n6/SJThMS6M/UlEQW7TpFZp6PtYEf8yy0G2Z1+NNpG1UN9iT+dCCxxusE23v2sHtbY8wsY0yKMSYl\nNvYyZp3yAcv3Z1FWWcXY6tY8i34NR1dat+uterk3uEZgRv8kKo3hjVVH3B2KY/kHwI1v/W/axoLG\ndUGlLsyexL8B6CQi7UQkCJgOzLdz/w3ZttFYtOskzcICSWnTDLbOhnWvwMCHoddUd4fWKLSLacLE\n3q15a/VhTvnaVX+TaKsJcFE2fHibVvYqwI7Eb4ypAB4Bvgb2AB8aY3aJyAMi8gCAiLQUkTTgceAp\nEUkTkaYX2tZZJ+ONyiuteWBHJbcgIH0DfP4j6/Z87LPuDq1ReXxsZyoqDS9+c8DdoThe6z4w6SU4\nthq+fFKHcVbYVWNojPkS+LLWe6/WeH4SqxjHrm3V/2w4nEteSQXXta2CD26BpvEw9W2dPtHF2kQ3\n4eYBSby37hj3XtWedjFN3B2SY/W8ETJ3w4rnrKG8B9zn7oiUG3lM5W5jtWj3KSIDyhm26YdQXgwz\n5mhlrps8MqojQf5+PLfIR6c0HPkUdJ5gjfekwzg3apr43cgYw+JdJ3k98g38Tu2AG/8DccnuDqvR\niosI4Z6r2vHF9gzvnqHrQvz8YMosiOkMH95uzeClGiVN/G60OyOPKQWz6Ve4DMb8Bjpf7e6QGr17\nh7WnWVgg//f1XneH4hwhTWHGbGss//dn6Jy9jZQmfjc6uHwOTwR+TEm3qTDkR+4OR2HN0PXwyI6s\nOJDN6tRsd4fjHM3bW/VI2Qfgk3uhqtLdESkX08TvLid3MHbv0xwI6ELI9f/SOXM9yC0D29A6MoS/\nLNzr/RO1XEj74TDhL7B/ISz5jbujUS6mid8d8jKoePcmzppQ1vb/BwSGuDsiVUNIoD8/HtuZbWln\nWbjzpLvDcZ5+91iP1f+AjW+6OxrlQpr4Xa20AGbfRFXxGe4q+wlD+/Z0d0SqDlOuiKdjXDh/XbSP\nisoqd4fjHCIw/i/QcSwseAJSl7g7IuUimvhdqbLCmhT71C7+EPZTaNnL99qL+4gAfz9+cnUXDmUV\n8uFGHxmvvy7+ATD1TYjrCh/eAae0f2VjoInfVYyx2k8f+JqMob/j7azOTE2ps8+b8hDjurWgf7vm\n/GHBbg5mFbg7HOcJjoCbP7Qmbn/vJsjLcHdEysk08bvK2pdtE6o8yn+KRxLoL0zqoyNUezIR4cXp\nfQgO9OehdzdTXObDrV8i4+HmD6D4NLw/DcoK3R2RciJN/K6w53P4+lfQdSLlo37DZ1vTGZUcR/Mm\nQe6OTF1Cq8hQXpjWh/2Z+Tw9b6e7w3GuVr3hxjfg5A74+G5t5unDNPE7W9ommHsvxF8JU2axbH8O\n2QVl3Hhl4qW3VR5hWOdYHh3ZkY82pfHhxuOX3sCbdRlvVfju/8oqmvTV5qyNnCZ+Z8o+ALOnWuOh\nz5gDgaF8vCmN6CZBjOjSuOYc8HY/GtOZwR2i+fVnO9mT4eO9XQfcZw0Lvn4WrPjbpddXXkcTv7Oc\nTYd3JoP4wa2fQngsuYVlfLP3FJOviCfQXz96b+LvJ7w4/QoiQwN5+L3N5JfUnl7ax4z7PfSaBt/+\nHja+4e5olINp9nGGolz47/VQmge3zIXoDgDM35pOeaXhxiu1NY83io0I5p8zruBITiG/+GSH7/bq\nBWtAt0kvQadx8MXjsOszd0ekHEgTv6OVFsB7N8LpIzDjfavCzGbu5nS6t25K11ZN3RefapAB7aN5\n8uoufLE9g7dWH3F3OM7lH2iN6ZPY3xrT59Ayd0ekHEQTvyNVlFqTqZzYClPfgrZDzy3aezKPHeln\n9WrfBzwwrANjusbx7Oe7eX7xft++8g8Ks5p5RneEOTfrpO0+QhO/o1RVwqf3w6Gl1iTpydect3ju\npjRtu+8j/PyEl2b25cYrE3jxmwM88v4W327jH9oMbvkEQptbd7PZqe6OSDWQJn5HMMYa62TXp1al\n2BUzz1tcXlnFp1tOaNt9HxIc4M9fb+zFLyYk8+WODKbNWsPJsz42UXtNTVvBbZ8BAv+dDKePujsi\n1QCa+BvKGPjqp7DpTRj6GAx+9HurLN+fRXZBqbbd9zEiwv3DO/DarSkczCxg0ksr2Z52xt1hOU90\nB7j1EyjNh7d+AGeOuTsiVU+a+BvCGPjqZ1Z750GPwOhn6lxN2+77tjHdWjD3ocEE+Plx07/XMG9r\nuu+W+7fqbV35l561JX8f79Dmo8QT/0BTUlLMxo0b3R3GxVUPurbuVSvpj/t9nZOpnC4so/8fl3Db\noLb8+gfd3BCocpXsglIe+O8mNh49TY/4ptx7VXuu7dmKAA/us1FeWUVhaQXF5ZUUl1We97PKQKC/\nEOjvZ3tYz0MD/YnN20nI+zdCWDO440trrB/lViKyyRiTYte6mvjrwRhY+AtY9woMfAiu/uMFZ9B6\ne/URnpm/i69+dJU242wEyiqqmLs5jddWHOJQViHxUaHcNbQd0/olEh4c4LI4jDFkFZSSdrqYtNPF\npJ8uJjO/hOyCMrLzS8kqKCW7oJQzRfXviNY/8DBv+P+BfL9Inot/nsBm8bSJbkL7mCa0j21CUvMm\nBAV47peer9HE70zGWAOurX0JBjwI4/90waRfVWUY+/wyQoP8+eLRq1wcqHKnqirDN3szeW35IdYf\nySUiJIAZ/ZMY1D6a5FYRtGwagjRguk1jDGeLyzmWW8Tx3GKOny7iWG6RLdEXkX66mNKK8yeQaRLk\nT2xEMDHhwed+xoQHExESQGiQP6GB/oQE+p977idQXmmoqKqivLLKel5pKCyrILfQ+gIJz97C/Uef\nJFeiuIvfsK8o/Nzx/P2ExGahtI8Np0frpvROjKJ3YhQx4cH1Pm91YZr4ncUYWPQUrPkXDHgAxv/5\nonPlfrUjgwff28w/ZlzBxN6tXRio8iRbjp3m9RWH+WpnBlW2f7eosECSW0bQtVVTkltGEBESiFD9\n5ySIgAAFpRVk5ZdaD9tVelZ+KRlnS8gvqTjvOM3CAkloFkZi81ASmoWR0CzU9ggjPiqUJs664zi2\nDt6dAhEtybvpEw6VRXIoq4DD2YUcyiokNbOAA5n55849oVkovROjuCIxin5tm9MjPhJ/P51zuqE0\n8TtDVSUseBw2vQX974MJ/3fRpG+M4Qf/XElRWSVLHh+uf9iKs8Xl7DuZz96TeezJyGNPRj77TuZT\nXH7pPgAhgX7ERgQTa7tab9E0hKTmYSQ0CyOpuZXsI0ICXXAWF3BsLbw3FUIi4dbPIKbjeYuLyirY\nmZ7HtuNn2Gp7pJ8pBiAyNJDBHaIZ0jGGqzrFkNQ8rEF3Q42VwxO/iIwHXgT8gdeNMX+utVxsy68B\nioA7jDGbbcuOAPlAJVBhT2Ael/grSmHuPbBnPgx9HEY/fdGkD7B0XyZ3vrmB/7uhFzf102acqm6V\nVYa000UUl1dijHVTaTDnRkNuEhxAbEQwTYL8PT8ZZmyD/06xnt8yF1r3uejqmfklrDmYw6rUbFYe\nyOaErR9EQrNQhneOZVz3lgxqH631BHZyaOIXEX9gPzAWSAM2ADOMMbtrrHMN8ChW4h8AvGiMGWBb\ndgRIMcZk23sCHpX4S/OtruqHl1uVuIMetmuzqa+uJv10Md/9ZKT+4arGIzvV6uBVfMYaq6qdfXVb\nxhgOZxeyMjWbFQeyWZWaTVFZJRHBAYxMjuPq7i0Z3iXWpRXk3uZyEr89n2J/INUYc8i28znAJGB3\njXUmAe8Y61tkrYhEiUgrY4x3T95ZkGV1UT+5A67/N/Sebtdm6w7lsOHIaX5zXTdN+qpxiekId31t\nlfm/e4M1o1fXH1xyMxGhfWw47WPDuW1QW0rKK1l5IJtFu0+yZE8m87edICjAj6s6xjCxT2vGdmtB\nWJB+CdSXPZ9cPFCzl0Ya1lX9pdaJBzIAAywRkUrg38aYWXUdRETuA+4DSEpKsit4pzp91PrjPZtu\nXbl0vtruTf+1NJWY8CCm9/eA81DK1SLj4c6vrDL/D2+Fif/63jAmlxIS6M+Ybi0Y060FFZVVbDx6\nmkW7TvHljgy+2ZtJWJA/47q1YNIV8QztGKPzW1wmV3xlDjXGpItIHLBYRPYaY5bXXsn2hTALrKIe\nF8R1YSd3WH+05UVWL8WkgXZvuu34GVYcyOZn45MJCfR3YpBKebCw5nDbPGu02nkPQd4JGPbkJevG\n6hLg78fA9tEMbB/NU9d2Zf2RXOZtTWfB9gw+23qC6CZBXNurFVOvTKRHfFPPrwvxAPYk/nSgZu1k\ngu09u9YxxlT/zBSRT7GKjr6X+D3Gns/hk/sgJMq6amnR/bI2f/m7VJqGBHDLQL3aV41ccLg1pPP8\nR2Hp7yFrL0z6FwSG1nuXfn5y7kvgNxO7s3x/Np9tTeeDDcd5Z81RkltGcFNKIpOviNcBES/CnsS/\nAegkIu2wkvl04OZa68wHHrGV/w8AzhpjMkSkCeBnjMm3PR8H/NZx4TuQMbD8r7D0D9bE6NNnQ0TL\ny9rF/lP5fL3rFD8c1dG9TeuU8hQBwVb9WGwyfPNbyD1k/W81bdXgXQcH+DO2WwvGdmvB2eJyPt92\ngo82Hue3X+zmT1/tYWy3FkxNSWRYp1htTl3LJRO/MaZCRB4BvsZqzvmGMWaXiDxgW/4q8CVWi55U\nrOacd9o2bwF8arv1CgBmG2MWOvwsGqqsCOY9DLs+seYZve4fEBhy2bt5eWkqYUH+3DmknROCVMpL\nicBVj0NsF5h7L7w2CmbMhtZXOOwQkaGB3DKwDbcMbMPek3l8tDGNT7ek8+WOk8RHhTKtXyLT+iXS\nounl/1/7Iu3AlXcC3p9htUEe8wwM+XG9yiGP5RQx4m9LuXtoO351rQ7GplSdTu6E96dDYTZc/wp0\nv95phyqrqGLJnlO8v/4YKw5k4+8njE6O4+YBSQzrFIufj90FOLo5p+86vt6qfCortFrudJlQ7129\nsuwgAX5+3HtVewcGqJSPadkD7l1q/d99dAdkbIeRvwJ/x6eioAA/runZimt6tuJIdiHvbzjGxxvT\nWLT7FAnNQpnRP4lp/RIb5dhBjfOKv6oKVr0A3/4eohJh+vvQov5X6TvTzzLppVXc3D+J303u4cBA\nlfJRFaXw5ZOw+R1IGgQ3vA6Rzp+PurSikkW7TvHeuqOsPZRLoL9wTc9W3DKwDSltmnl1iyAdq+di\n8k/Bp/fBoe+s28zrXrTGF6mnisoqJr20isz8UpY8NpzIMK3UVcpu2z+CL34M/oEw+ZUG3XVfrtTM\nfN5de4y5m9LIL60guWUEtwxsw+Qr4r2yh7Am/gtJ/caaEL20ACb8BfreVq/y/Jpe+e4gf1m4l1dm\n9mVCz4a3VFCq0ck5aBX7nNwOAx+GMb+BANc1xSwqq2D+1hO8s+YouzPyCA8OYErfeG4Z2IbOLSJc\nFkdDaeKvrbLcKtZZ9QLEdoWpb0Jc1wbv9lBWAeNfXMGoLnG8euuVDghUqUaqohQW/RrW/9tq7XPj\nG9DctfVlxhi2HD/Du2uP8sX2DMoqqhjQrjm3DmrDuG4tPX74FU38NZ3cCfMfgRNb4Mo7rYHWgsIa\nvNuqKsP019ayNyOPJY8PJ06biSnVcHu+sHr6VlZYo+D2vxf8XN8DPrewjI82HufddUc5nltMbEQw\n0/slMr1/EvFR9e+A5kya+AHKS6wOWategNBmcO1z0G2SYwIE3l17lKc+26nDLivlaGfT4PMfQ+pi\nSBwAE/9p9QFwg6oqw7IDWby75ijf7stEgFHJccwc0IZhnT2rY5gm/qNr4PMfQvZ+6D3DusoPa+6w\n+DLOFjP278vpnRjJu3cP8OqWAEp5JGNg+4ew8GdWc+vhP7X62Pi7r/FE2uki5qw/zpwNx8kuKCU+\nKpSbByQxNSWBuAj33/E33sRfkgffPAsbXofIJLjueeg4xqGxGWO4++2NrDmYw9c/HkZSdMOLjZRS\nF1CQBV/9BHZ9Ci16WGP9OLDHb32UVVSxeLfVJHT1wRwC/ISx3VowrV8iV7lxeIjGmfiLT8MrQyEv\n3ZoPd9RT1iBRDjZvazo/mrOVp67tyj3aWUsp19jzBSx4AgpOQd9bYeRTENHC3VFxMKuA99cd45Mt\n6eQWlhEfFcrUlASmpiS6vC6gcSZ+gKV/sq7wE/s5PiisW72J/1pFYvMwPnlwsEeV7ynl84rPwLL/\ng/WzrMHfhj5mzYjXgNE+HaW0opIluzOZs8EaHkIEhnWK5aaUREZ3jXPJEO2NN/E7UU5BKVNfXUNW\nQSmfPDiYTl7Uvlcpn5JzEBY/DXu/gKYJVrv/njc2uE+OoxzPLeKjjcf5cGMaJ/NKaBoSwHW9W3PD\nlQlckRjltDpBTfwOVlBawc2vrWXfyXz+e/cA+rdzXEWxUqqeDq+Ar39pdfyKvxJG/hI6jPaYL4DK\nKsPqg9l8vCmNr3edpKS8ivaxTbihbwLXXxFPawcXBWnid6DSikruemsDaw/lMuvWKxnd1f3likop\nm6pK2DbHmkcjL92q+B32E+g8Afw8p8NVfkk5X+7IYO6mdNYfyQWgX9tmTOzdmgk9WzlkoDhN/A5S\nWWV49P3NfLnjJM9N7c0NVzp/ECmlVD1UlMG292Hl3+H0EYjrBlc9YY3H5YYOYBdzLKeI+dvSmb/t\nBPtPFeDvJwzuEM3E3q25ukdLmtZzEidN/A5gjOFXn+1k9rpj2oJHKW9RWQE758KK5yB7H0R3tFr5\n9ZoGIU3dHd337DuZf+5L4HhuMVFhgWz41Zh6TR6vid8B/r5oH//4NpUHR3TgZ+OT3RqLUuoyVVXB\n3s9hxd8hYysENoFeN0HKXdCql7uj+x5jDNvSzpKaWcCN9SxZ0MTfACXllfz5q728tfoI01IS+fMN\nPbVnrlLeyhhI3wwb34CdH0NFCST0g5S7oftkj2gK6iia+OvpYFYBj8zewp6MPO4a0o5fXpNMQD1u\nuZRSHqj4NGx93/oSyDkAQeGQfC30uAHaj3TpUNDOoIn/Mhlj+GhTGs/M20VokD9/m9qLUcnaekcp\nn2QMHFkBOz6C3fOh5AyEREG3idaXQNurPK5C2B6a+C9Dfkk5T322k3lbTzCofTQvTO9DCx1iWanG\noaIMDi21KoT3LoCyAghtDh1HW6MAdBgN4bHujtIuOtm6HYwxrDiQza/n7STtdDFPjuvMgyM66jAM\nSjUmAUHQ+WrrUV4MBxbBvq8gdYl1R4BA6z7Wl0DHMVY/gQDvn5y90V3xV1YZFu48ySvLUtmZnkd8\nVCgvTO9Dv7baG1cpZVNVBSe3wYEl1rwAaRvAVIF/sNVLOGmg9Ujsb8334QG0qKcOpRWVfLI5nX8v\nO8iRnCLaxTThgeHtmXxFPMEB3leep5RyoeLT1hARx9fBsbVWE9GqCmtZbFfrrqBlT2jZC1r2cMuX\ngSZ+m8oqw/a0Myzdm8mcDcfJzC+lZ3wkD43owLjuLbVYRylVP2VFcGIzHFsDx9dDxnYoOPm/5ZFJ\n1hdBbBeI6WR1JIvu6NAJoWpr1GX8uYVlLN+fxXf7Mlm2P4vTReWIwNCOMTw/rQ+DO0Rru3ylVMME\nhUHbodajWkEmnNxhDRp3cof1OPD1/+4MwKo4jukEzdpBVCJEJkBkIkQlWc9d1K/ArsQvIuOBFwF/\n4HVjzJ9rLRfb8muAIuAOY8xme7Z1lJLySqbPWsu2tDMYA9FNghjZJY7hXWIZ1imWZk28u42uUsrD\nhcfZWgON/t97leVw+qjVbyAnFbJtP4+shPwTVr1BTc3awo+2OT3USyZ+EfEHXgLGAmnABhGZb4zZ\nXWO1CUAn22MA8AowwM5tHSIk0J+20WGM7BLHiC6x9IyPxE+LcpRS7uQfCDEdrUdtlRVW8j9z3Jpg\n/uwx64vCBey54u8PpBpjDgGIyBxgElAzeU8C3jFWhcFaEYkSkVZAWzu2dZgXprt3Lk6llLKbf4BV\nxBOV5PJD2zMeQTxwvMbrNNt79qxjz7YAiMh9IrJRRDZmZWXZEZZSSqn68JiBaIwxs4wxKcaYlNhY\n7+gpp5RS3sieop50ILHG6wTbe/asE2jHtkoppVzIniv+DUAnEWknIkHAdGB+rXXmA7eJZSBw1hiT\nYUbKaaAAAAMXSURBVOe2SimlXOiSV/zGmAoReQT4GqtJ5hvGmF0i8oBt+avAl1hNOVOxmnPeebFt\nnXImSiml7OLTPXeVUqqxuJyeux5TuauUUso1NPErpVQj45FFPSKSBRx1dxyXKQbIdncQLqbn3Djo\nOXuHNsYYu9rCe2Ti90YistHe8jVfoefcOOg5+x4t6lFKqUZGE79SSjUymvgdZ5a7A3ADPefGQc/Z\nx2gZv1JKNTJ6xa+UUo2MJn6llGpkNPE7gYg8ISJGRGLcHYuzichfRWSviGwXkU9FJMrdMTmDiIwX\nkX0ikioiP3d3PM4mIokislREdovILhH5kbtjchUR8ReRLSLyhbtjcRZN/A4mIonAOOCYu2NxkcVA\nD2NML2A/8As3x+NwNaYQnQB0A2aISDf3RuV0FcATxphuwEDg4UZwztV+BOxxdxDOpInf8Z4Hfgo0\nilpzY8wiY0yF7eVarDkXfM256UeNMWVA9RSiPssYk2GM2Wx7no+VCOucPc+XiEgCcC3wurtjcSZN\n/A4kIpOAdGPMNnfH4iZ3AV+5OwgnsHsKUV8kIm2BK4B17o3EJV7AunCrcncgzmTPDFyqBhFZArSs\nY9GvgF9iFfP4lIudszFmnm2dX2EVD7znytiUc4lIODAX+LExJs/d8TiTiPwAyDTGbBKREe6Ox5k0\n8f9/e3eMEjEQRnH8/0DBRu/gAezEZgvBtRBZPIAoeAQLz2FlYStoY+EFPIIga2VrKQhe4Vkk2ym4\nmGQw835lqhcCj48ZJrMk2/vfPZe0BWwCL5KgWfJ4lrRj+33AiJ376Z0XJJ0BM2DqcR4M+c31o6Mj\naZWm9O9sP5TOM4AJcCTpEFgDNiTd2j4pnKtzOcDVE0lvwLbt//aHv6VIOgAugV3bH6Xz9EHSCs3G\n9ZSm8J+A4zHfJqdmerkBPm2fl84ztHbiv7A9K52lD1njj7+6AtaBR0lzSdelA3Wt3bxeXCH6CtyP\nufRbE+AU2Gu/67ydhGMEMvFHRFQmE39ERGVS/BERlUnxR0RUJsUfEVGZFH9ERGVS/BERlUnxR0RU\n5gtAZbJ5ZFfIDQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# find M-projection by numerical integration; may fail in high dimensions (use smart sampling methods instead)\n", "xs = np.linspace(-5, 5, 50)\n", "pxs = p_x(xs)\n", "mu_hat = np.trapz(pxs * xs, xs) # first moment\n", "var_hat = np.trapz(pxs * xs**2, xs) - mu_hat**2 # second moment - (first moment)^2\n", "sig_hat = var_hat ** 0.5\n", "qxs = norm.pdf(xs, loc=mu_hat, scale=sig_hat)\n", "obj = np.trapz(pxs * (np.log(pxs)-np.log(qxs)), xs) # approximate D_KL(p||q)\n", "print('D(p||q) =', obj)\n", "\n", "# plot M-projection; by comparison I-projection avoids regions where p(x) is close to zero and appears \"sharper\"\n", "plt.figure()\n", "plt.plot(xs, pxs, label='$p(x)$')\n", "plt.plot(xs, norm.pdf(xs, loc=mu_hat, scale=sig_hat), label='$q(x)$')\n", "plt.legend(loc='best')\n", "plt.title('Approximate M-Projection')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The **I-projection** is defined as \n", "$$ q^I = \\arg \\min_{q \\in Q} D(q \\| p), D(q \\| p) = E_q[ \\log q(X) - \\log p(X)] = -H[q] - E_q[\\log p(X)] $$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "Unlike before, we no longer need to marginalize/integrate with respect to $p$, but we still need to deal with expectations (w.r.t. $q$). Generally, these expectations cannot be computed analytically, and some approximation is needed, even just to evaluate the objective.\n", "Note that for a generic $p(x) = \\tilde p(x) / Z$ where we don't necessarily know the partition function $Z$, e.g., in a Markov network, it's customary to maximize the evidence lower bound (ELBO): $ELBO = \\log Z - D(q \\| p) = \\mathbb{E}_q[\\log \\tilde p(X)] + H[q]$, instead of minimizing $D(q\\|p)$; these two approaches are equivalent. \n", "\n", "Here, ideas from general-purpose variational inference are often used for applying stochastic optimization to this problem, particularly \"tricks\" for computing derivatives of $E_q[ \\log q(X) - \\log p(X)]$ w.r.t. the parameters $\\phi$ of $q$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In our example, we know the entropy of Gaussian in closed form,\n", "$$H[q] = \\frac{1}{2} \\log 2 \\pi e \\sigma^2 $$\n", "so we only need to worry about finding the gradient $ \\nabla_\\phi \\mathbb{E}_{q_\\phi} [\\log p(X)]$ w.r.t. parameters $\\phi = (\\mu, \\sigma^2)$. Under most circumstances in practice we can exchange integration and differentiation:\n", "\n", "$$ \\nabla_\\phi \\mathbb{E}_{q_\\phi} [ f(X) ] = \\int_x f(x) \\nabla_\\phi q_\\phi(x) dx $$\n", "\n", "Often this integral is intractable; one idea for approximating it is to turn it into an expectation and use sampling." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Perhaps the first thing to try is multiplying and dividing the integrand by $q_\\phi(x)$ (somewhat reminiscent of importance sampling):\n", "$$ \\nabla_\\phi \\mathbb{E}_{q_\\phi} [ f(X) ] = \\int_x q_\\phi(x) f(x) \\frac{\\nabla_\\phi q_\\phi(x)}{q_\\phi(x)} dx = \\int_x q_\\phi(x) f(x) [\\nabla_\\phi \\log q_\\phi(x)] dx = \\mathbb{E}_{q_\\phi} [ f(X) \\nabla_\\phi \\log q_\\phi(X)] $$\n", "where we used the simple identity $\\frac{\\nabla_\\phi q_\\phi(x)}{q_\\phi(x)} = \\nabla_\\phi \\log q_\\phi(x)$.\n", "Now this expression is an expectation w.r.t. $q_\\phi(x)$, so we can draw samples from $q_\\phi(x)$ and approximate it. This technique is known as the ``score function trick'' or REINFORCE.\n", "\n", "In our example, $f(x) = \\log p(x)$ is the log pdf of the Gaussian mixture, $\\log q(x)$ is the log pdf of our approximating Gaussian, and by simple algebra we have $$\\nabla_\\phi \\log q_\\phi(x)= [\\frac{\\partial}{\\partial\\mu} \\log q_\\phi(x), \\frac{\\partial}{\\partial\\sigma} \\log q_\\phi(x) ] = [ \\frac{x-\\mu}{\\sigma^2}, -\\frac{1}{\\sigma} + \\frac{(x-\\mu)^2}{\\sigma^3} ] $$" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# score function estimator: grad E_q[-log P(x)] = E_q[-log p(x) * grad log q(x)]\n", "# we have \\partial log q / \\partial mu = (x-mu)/(sigma**2), \n", "# \\partial log q / \\partial sigma = - 1/sigma + (x-mu)**2 * sigma**(-3)\n", "\n", "def grad_log_q(x, mu, sigma):\n", " # x may be a length N vector of data points\n", " partial_mu = (x-mu)/(sigma**2)\n", " partial_sigma = - 1/sigma + (x-mu)**2 * sigma**(-3)\n", " return np.vstack([partial_mu, partial_sigma]).T # N x 2 array for convenience\n", "\n", "\n", "def est_grad(mu, sigma, num=1000):\n", " xs = np.random.normal(mu, sigma, num) # samples from q\n", " neg_log_p_x = -np.log(p_x(xs))\n", " grad_h_q = [0, 1 / sigma] # grad H[q]; 0 wrt mu\n", " grad = np.mean(neg_log_p_x[:, np.newaxis] * grad_log_q(xs, mu, sigma), axis=0) - grad_h_q # approx grad of obj\n", "\n", " # estimate the optimization objective (KL_q_p)\n", " h_q = 0.5 * np.log(2 * np.pi * sigma ** 2)\n", " obj = -h_q + np.mean(neg_log_p_x)\n", " return grad, obj" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "t = 0 D(q||p) = 1.78176312345\n", "t = 20 D(q||p) = 0.944366751202\n", "t = 40 D(q||p) = 0.730673826087\n", "t = 60 D(q||p) = 0.720010453942\n", "t = 80 D(q||p) = 0.735620602677\n", "t = 100 D(q||p) = 0.681848886175\n", "t = 120 D(q||p) = 0.763892916983\n", "t = 140 D(q||p) = 0.703759502188\n", "t = 160 D(q||p) = 0.719445890011\n", "t = 180 D(q||p) = 0.691542639593\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAskAAAEYCAYAAACuv2v6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4nNWV+PHv0TRVq7sX2ZZxx8YNGxuD6TUGpxKCQwuB\nANm0Ddn8SAibTZbsUpJsSOgQEmoILaETijGuchG2cbdc5KZqWV0azf398Y7kkawykmbmnZHO53n8\noJm3nRnjV2fOnHuvGGNQSimllFJKnRBndwBKKaWUUkpFG02SlVJKKaWUakOTZKWUUkoppdrQJFkp\npZRSSqk2NElWSimllFKqDU2SlVJKKaWUakOTZBVVRKRKRMbYHYedRGSk/31wxMJ5lVJKqb5Ik+R+\nREQ+EpFyEfHYHUtHjDHJxpg9oT6viFwrIsvDdbyI/EJEGv1J6DERWSEi83pyLWPMfv/70NTTeP0x\n7RWR80J9XqWUUqo/0CS5nxCRHOBMwABfCON1nOE6dwx4wRiTDGQDy4GXRUTa7tTP3yOllFIqJmiS\n3H8sBVYBTwHfDNwgIk+JyEMi8p6IVIrIxyIyKmC7EZHvisgeESkRkf8VkTj/tmtF5FMReUBESoFf\niEiciNwpIvtEpEhEnhaRVP/+XxWRAhEZ4H98sYgcEZHsgGvlBsT1RxF5y1+h/VREBovIb/0V8W0i\nclpAnD8Rkd3+1/C5iFzpf34i8BAwr7nS63/eIyL3ish+ETnqfw8SevtGG2MagT8Dg4HMHrxHOf73\nwel/nCoij4vIYRE5KCL/FdgyISLfEpGtAa97hoj8BRgJ/MP/mn/cznmHisjrIlImIrtE5FsB5/yF\niLzoj6tSRLaIyKzevjdKKaVUrNAkuf9YCjzj/3OhiAxqs/1q4JdAFrDRv1+gK4FZwAxgMXB9wLbT\ngT3AIOBXwLX+P4uAMUAy8AcAY8wLwArg9yKSCTwO3GiMKe4g7q8Ad/rjqgdWAuv9j18C7g/YdzdW\ntTwVuBv4q4gMMcZsBW4GVvrbDdL8+98DnAJMB3KBYcDPO4gjaP52lmuBA8aYEv/TQb9H7XgK8Ppj\nPA24ALjRf60vA7/A+vsdgPUtQakx5hpgP3C5/zX/TzvnfR4oBIYCXwJ+LSLnBGz/gn+fNOD1TuJT\nSiml+hxNkvsBEVkAjAJeNMasw0omv95mtzeMMcuMMfXA/8Oquo4I2P4bY0yZMWY/8FvgqoBth4wx\n/2eM8RpjarES7vuNMXuMMVXAfwBfC2gzuBU4B/gI+Icx5p+dhP+KMWadMaYOeAWoM8Y87e+rfQEr\naQTAGPM3Y8whY4zPn4zvBOZ08J4IcBPwff/rqgR+DXytk1i68hV/lfoAMBPrg0Wz7r5HzXEOAi4B\nvmeMqTbGFAEPBMR5I/A/xpi1xrLLGLOvq0D9f7fzgTuMMXXGmI3AY1jJdrPlxpg3/e/1X4Bp3X5H\nlFJKqRilSXL/8E3g3YCq5rO0abnASuwA8CdtZVgVxpO2A/s62YZ/W2Citg9wYlVRMcYcA/4GTAHu\n6yL2owE/17bzOLn5gYgsFZGN/oFzx/znz+rgvNlAIrAuYP+3/c93SkSu9rcwVInIWwGbXjTGpBlj\nBhpjzvF/IGnWrfcowCjABRwOiPNhYKB/+wisDz3dNRRo/nAQGMOwgMdHAn6uAeK1n1oppVR/ob/w\n+jh/j+1XAIeINCc9HiBNRKYZY/L9z40IOCYZyAAOBZxqBLDF//PINttMm8sewkrumo3Eahc46j//\ndKx2jeeA3wMX9ejFBfD3UD8KnIvVVtEkIhuB5oFzbWMswUqyJxtjDnbnWsaY5raVbh3W5nFn79Hw\ngOcPYLWZZBljvO2c9wAwNshrtr1+hoikBCTKI4FuvRdKKaVUX6WV5L7vCqAJmITVezsdmAh8Quuv\n1i8RkQUi4sbqTV5ljAmsfv67iKT7v6b/N6xWh448B3xfREb7E+5fY8384BWReOCvwE+B64BhIvKd\nELzOJKyksBhARK7DqiQ3OwoM978+jDE+rKT6AREZ6D9mmIhcGIJYgtHhexS4kzHmMPAucJ+IDPAP\n+BsrImf5d3kM+JGIzBRLrpwYdHkUq9/5JP6/2xXAf4tIvIicCtyA9XejlFJK9XuaJPd93wSe9M+R\ne6T5D9YgrKsDvj5/FrgLq81iJvCNNud5DViHNajvDawBdx15AquHdRlQANQBt/u3/TfWgLY/+fuf\nvwH8l4iM682LNMZ8jtW6sRIrOZwKfBqwywdYlfAjItLcdnIHsAtYJSLHgfeB8b2Joxs6e4/aWgq4\ngc+BcqwBi0PA6sPGGgj4LFAJvIr1LQBY7/Wd/jaNH7Vz3quAHKyq8ivAXcaY93v7wpRSSqm+QIzp\n7BtZ1R+IyFNAoTHmzg62G2CcMWZXRAPrp8RacXAH4DL6D1QppZSyhVaSlYo+U4B9miArpZRS9tEk\nWakoIiI/AB4BfmJ3LEoppVR/FlS7hYhcBPwOcACPGWPuabN9MdZgLx/WCP3vGWOWB3OsUkoppZRS\n0abLJNm//O0O4Hys1bnWAlf5B0o175MMVBtjjH+U/IvGmAnBHKuUUkoppVS0CabdYg6wy78yWAPW\nMrWLA3cwxlQF9E82T8UV1LFKKaWUUkpFm2AWExlG69XCCoHT2+4kIldiTTk1ELi0O8e2lZWVZXJy\ncoIITSmlYte6detKjDFdrvIYKXrvVUr1B8Hee0O24p4x5hXgFRFZiNWffF53jheRm4CbAEaOHEle\nXl6oQlNKqagkIvu63itycnJy9N6rlOrzgr33BtNucZCAJYuxlsztcOlaY8wyYIyIZHXnWGPMI8aY\nWcaYWdnZUVNYUUoppZRS/VAwSfJaYJx/+Vw38DXg9cAd/Evhiv/nGYAHKA3mWKWUUt0jIk+ISJGI\nbO5ge6qI/ENE8kVki3+ZdqWUUt3QZbuFMcYrIrcB72BN4/aEMWaLiNzs3/4Q8EVgqYg0ArXAV/0D\n+do9NkyvRSml+ounsJaWf7qD7bcCnxtjLheRbGC7iDzjH0CtlFIqCEH1JBtj3gTebPPcQwE//wb4\nTbDHKqVUs8bGRgoLC6mrq7M7lLCJj49n+PDhuFyukJzPGLNMRHI62wVI8X/DlwyUYc1hr5TqY/rD\nPbSnenvvDdnAPaWU6onCwkJSUlLIycnB37XVpxhjKC0tpbCwkNGjR0fqsn/Aam07BKRgfbvni9TF\nlVKR09fvoT0VinuvLkutlLJVXV0dmZmZffbmLiJkZmZGuspzIbARGApMB/4gIgM6iO8mEckTkbzi\n4uJIxqiUCoG+fg/tqVDcezVJVkrZrq/f3G14fdcBLxvLLqAAmNDejjqzkFKxr6/fQ3uqt++LJsmq\nS3uKq7jv3e3sL62xOxSlVHD2A+cCiMggYDywJxwXWr6zhP94eRP13qZwnF4ppWyjPcmqQ7uKqvjD\nBzt5Pf8QPgPPrz3A09fPYeKQdr+1VUpFiIg8B5wNZIlIIXAX4IKWQdW/BJ4SkU2AAHcYY0rCEcv2\no5U8t2Y/P7loAh6nIxyXUEopW2glWZ1k59FKvvvcBs5/4GPe2XKUG88cw/M3zcUZJ3zl4ZWsKSiz\nO0SlIqa2tpazzjqLpqaOK6UNDQ0sXLgQrzcyE0gYY64yxgwxxriMMcONMY8bYx5qnnXIGHPIGHOB\nMWaqMWaKMeav4Yol3mX9GqnTSrJS/drDDz/MkCFDmD59OtOmTePLX/4yBQUFQHD3UYCzzz6bvXv3\ndvi4PeG8/2qSrFr4fIYfv5TPBb9dxvtbj/LthWP55I5F/PSSicwdk8lLt5xBdoqHax5fzfufH7U7\nXKUi4oknnmDJkiU4HB1XSd1uN+eeey4vvPBCBCOLDs3V4/pGnTxDqf5s06ZN/Od//icbN24kPz+f\nc889lyVLlmCMCeo+2lPhvP9qkqxavLX5CC/mFXLN3FEsv+McfnLxBLKSPS3bh6Ul8Ldvz2P84BS+\n/dd1/H1doY3RKhVaV111FV/96leZM2cOo0aN4o033gDgmWeeYfHixS37LVq0iPfeew+AO++8k9tv\nvx2AK664gmeeeSbygdtMK8lKKYDPPvuMKVOmtDy++eabOXLkCAcOHDjpPrpz507OPvtspkyZwg9/\n+EPGjh3b5fk7ukdD+O6/2pOsAGhs8nHvu9s5ZVAyd10+GUdc+yNCM5M9PPutuXz7L3n88G/5lNc0\ncOOZYyIcreqr7v7HFj4/dDyk55w0dAB3XT65y/3y8/NZvHgxL7zwAsuXL+cHP/gB559/Pnv27CEn\nJ+dEjHffzc9//nOKiorYsGEDr7/+OgBTpkxh7dq1IY09FmglWanoYec9dPPmzUye3Hq/hIQEysvL\nW91Hm5qaWLp0KQ8++CAzZszg9ttvP+m49rR3j7700kuB8N1/tZKsAHgx7wAFJdX8+4UTOkyQmyV7\nnDxx7WwumTqY/3pjK5/uCst4IKUipq6ujuLiYu666y4AJk2aRHl5OSUlJaSlpbXad+HChRhjuP/+\n+3n++edbvj50OBy43W4qKysjHr+dtJKslDpw4ADJyckMGHBiYH9jYyOHDx8mLS2t1X301VdfZdKk\nScyYMQOAiRMnMm3atE7P39E9ulm47r9aSVbUNjTxu/d3MmtUOudNHBjUMR6ng/u/Mp01BR/y0Me7\nmZ+bFeYoVX8QTLUiHDZv3sy4ceOIj48HYP369UybNo2EhISTJqLftGkThw8fJjMzk5SUlFbb6uvr\nW87RX2glWanoYdc9dNOmTa1aLQCefPJJzjnnHAYMGNDqPrphwwamT5/e8jg/P5/zzjuv0/N3dI8O\nFI77r1aSFU+uKKCosp47Lp7QrYm3410Orpufwyc7S0L+9Y5SkZSfn8/+/fupq6ujurqau+66i+9/\n//ukp6fT1NTUcoM/fPgwV199Na+99hrJycm8/fbbLecoLS0lKysLl8tl18uwRUsluVEryUr1V237\nkd99913++7//m3vvvfek+2hmZibbtm0DYPXq1Tz99NNdVpI7ukc3C9f9V5Pkfu5YTQN/+mg350wY\nyOycjG4f/43TR5HodvDYJ2FZp0CpiMjPz2fJkiWcfvrpzJ49m1tuuYX58+cDcMEFF7B8+XJqampY\nsmQJ9913HxMnTuRnP/sZd999d8s5Pvzww5b+uP6kpZLs1UqyUv3Vpk2beOaZZ5g5cyYzZszgz3/+\nM2+//TYTJ04ETtxHAa655hry8vKYOnUqL7/8MpmZmeTm5nZ6/s7u0RC++6+2W/Rzf/p4N1X1Xn58\n0fgeHZ+a6OKrs0fwl5X7+NGF4xmalhDiCJUKv/z8fB555BF+//vfn7Tt1ltv5YEHHuC8885j5cqV\nLc8vXLiw1eNnn32We+65JyLxRpPmSrKuuKdU/9XVzBKB99GsrCxWr14NWL3MH330EXFxnddsO7tH\nQ/juv1pJ7scOV9Ty1Kd7uXL6MCYM7vkqejcsGI0Bnvy0IHTBKRVBu3fvZty4ce1umzFjBosWLepy\nMZErrriCU045JVwhRq3mSnKd9iQrpTrQ0X00Pz+fU089tcvjO7tHh/P+q0lyP/a793fiM4bvn9+7\n/7GGpydy6dQhPLfmAMfrGkMUnVKRU1hY2Gkl4/rrr+9yMZGlS5eGI7Sop5VkpVQw2ruPXnbZZTz6\n6KMtj6+99tpWM2E0P+7sHh3O+68myf3UrqIqXsw7wNWnj2JERmKvz3fTwjFU1Xt5dvX+EESnlIoV\nWklWSoVKR0myXTRJ7qceeH8HCS4Ht53TebN8sKYMS+WMsZk8+WkBDTqAR6l+w+PUSrJSqm/SJLkf\nqqhp5J3NR7hqzshWy0731k0Lx3D0eD2vbTwYsnMqpaJbXJzgdsRpJVkp1edoktwPvb/1KF6f4bJp\nQ0N63rNOyWbC4BQe/WQPxpiQnlspFb08rjitJCtlI/2d277evi+aJPdDb20+zNDUeKYNTw3peUWE\nb505hh1Hq/hoR3FIz62Uil4ep0MryUrZJD4+ntLSUk2U2zDGUFpa2qtV+HSe5H6msq6RZTtKuGbe\nqG6trhesy6cN5X/f2c4jH+9h0fjglrhWSsW2eK0kK2Wb4cOHU1hYSHGxFqfaio+PZ/jw4T0+XpPk\nfuaDbUU0NPm4ZOrgsJzf7Yzj2vk53PPWNvaWVJOTlRSW6yiloofHGUe9VpKVsoXL5WL06NF2h9En\nabtFP/PmpsMMGuDhtBHpYbvGpVOHAPDu50fCdg2lVPSIdzm0kqyU6nM0Se5Hquu9fLS9mIunDCEu\nLvStFs1GZCQyacgA3tlyNGzXUCpSamtrOeuss7pccW/hwoV4vd4IRhY9PE6d3UIp1fdoktyPfLi9\niHqvj4unhKfVItCFkwezfn85RZV1Yb+WUuH0xBNPsGTJki5X3Dv33HN54YUXIhhZ9PA4tZKslOp7\nNEnuR97adISsZDezcjLCfq0LpwzCGHj/86KwX0upUNi5cydnn302U6ZM4Yc//CFjx44F4JlnnmHx\n4sUt+y1atIj33nsPgDvvvJPbb78dgCuuuIJnnnkm8oFHgXiXVUl+//Oj/PDFfF1QSCnVJ+jAvX6i\ntqGJD7YVsWTGMBxhbLVoNn5QCqMyE3lnyxG+fvrIsF9P9RFv/QSObArtOQdPhYvv6XSXpqYmli5d\nyoMPPsiMGTO4/fbbmTx5Mg0NDezZs4ecnJyWfe+++25+/vOfU1RUxIYNG3j99dcBmDJlCmvXrg1t\n7DGiuZL8/taj/H19ISnxTn7xhcl2h6WUUr0SVCVZRC4Ske0isktEftLO9qtF5DMR2SQiK0RkWsC2\nvf7nN4pIXiiDV8H7eEcRtY1NXOIfVBduIsIFkwaxYncJx+saI3JNpXrq1VdfZdKkScyYMQOAiRMn\nMm3aNEpKSkhLS2u178KFCzHGcP/99/P888+3tGE4HA7cbjeVlZURj99uzZXkYzXWv/WnVuxlxe4S\nm6NSSqne6bKSLCIO4EHgfKAQWCsirxtjPg/YrQA4yxhTLiIXA48ApwdsX2SM0Tumjd7cdIT0RBen\njw5/q0WzCycP5tFPCvhwWxGLpw+Dugo4th+SB0NiJsRpt49qo4uKb7hs2LCB6dOntzzOz8/nvPPO\nIyEhgbq61n31mzZt4vDhw2RmZpKSktJqW319fa8mro9VzZXkY7UN5GQmsre0ht3F1ZwxNsvu0JRS\nqseCyVLmALuMMXuMMQ3A88DiwB2MMSuMMeX+h6uAns/crEKurtFqtbhw8mCcjsglpqeNTCcr2cPm\nDSvhH9+D+ybAQwvg3lz4r4HwwFR47Hx4cSmsfRx8OvBH2SMzM5Nt27YBsHr1ap5++mmmTZtGeno6\nTU1NLYny4cOHufrqq3nttddITk7m7bffbjlHaWkpWVlZuFwuW16DnQIrycPSEwBrNh2llIplwfQk\nDwMOBDwupHWVuK0bgLcCHhvgfRFpAh42xjzS7ShVryzfWUJVvZeLI9RqAUBTI46t/+BFz28Zsy8f\nczAemfIlyD0Hqkug8jAcPwyVh+DwZ/D5a7DhL3DZb2Ho9K7Pr1QIXXPNNVx66aVMnTqVSy65hMzM\nTHJzcwG44IILWL58OWeccQZLlizhvvvuY+LEifzsZz/jjjvu4KKLLgLgww8/5NJLL7XzZdjG458n\nuaK2kclDUxGBGk2SlVIxLqQD90RkEVaSvCDg6QXGmIMiMhB4T0S2GWOWtXPsTcBNACNH6kCvUHpz\n82FSE1ycMTYzMhfc9ia88QOoPMyQpOH8uvEqFlz5PRZOm9D+/sbAlpfh7f+ARxfB6TfDop+CJ6X9\n/ZUKsaysLFavXg3AgQMH+Oijj4jztwPdeuutPPDAA5x33nmsXLmy5ZiFCxe2evzss89yzz32tIvY\nLd4/T3J5TQMZSS4SXQ6q6vWbIaVUbAvmu/eDwIiAx8P9z7UiIqcCjwGLjTGlzc8bYw76/1sEvILV\nvnESY8wjxphZxphZ2dnZwb8C1akGr4/3Pj/K+ZMG4YpEq8WGv8ILV0PyQLjqBeL+bQPPOq/kzV0N\nHR8jAlO+CLeugZnXwao/wYOnw9Z/hj9epdrIz8/n1FNPbXk8Y8YMFi1a1OViIldccQWnnHJKJEKM\nOh6XNXixrtFHWqKbJI+TmgatJCulYlswWdNaYJyIjBYRN/A14PXAHURkJPAycI0xZkfA80kiktL8\nM3ABsDlUwauufbq7hMo6L5dMDf8CInz6e3jtVhhzNlz7Joy/CI/bzaIJA3nv86M0+UznxyekwWX3\nww3vQUK6lWy/8SOr0qxUhFx22WU8+uijrZ67/vrru1xMZOnSpeEOrYWIPCEiRSLS4f1URM72zyq0\nRUQ+Dmc8HueJXyWpCS6SPU6qtN1CKRXjukySjTFe4DbgHWAr8KIxZouI3CwiN/t3+zmQCfyxzVRv\ng4DlIpIPrAHeMMa8jYqYtzcdIcXjZH5uGEeZGwPv3QXv/QwmXwlXvQCe5JbNF0waRGl1A+v2lXdy\nkgAjZsNNH8HcW2Hto/Dx/4QlbKVi2FPARR1tFJE04I/AF4wxk4EvhzOY5koyWElyosehA/eUUjEv\nqJ5kY8ybwJttnnso4OcbgRvbOW4PMK3t8yoyjDEs31XCgnFZeJwdV8F6xdcE//werH/aapW49D6I\na32ts8dn43bE8e6WI8wJdgo6hwsu/BXUlsNHv7baN2ZdF4YXoFTsMcYsE5GcTnb5OvCyMWa/f/+w\nLn0ZH1BJTkt0keR2Ut2gPclKqdimE9X2YfvLajh4rDZ8A/a8DfDSdVaCvPDf4bIHTkqQAVLiXczP\nzeSdz49gutM6IQJf+D3knm8NBNz2RgiDV6pPOwVIF5GPRGSdiIS1FySwkpyWYPUkayVZKRXrNEnu\nw1bstsZPzgvXhP7v/dyauu3CX8M5d1pJbQcumDyYA2W1bD3czdXIHC74yp9hyHR46XrYv6qXQato\n1K0PTzHIhtfnBGYClwIXAj8TkXZHFYrITSKSJyJ5xcXFPbrYSZVkj5MarSQrpWKcJsl92MrdpWSn\neBibnRT6k+94F1b/CeZ8G+bd2uXu500chAi8s+VI96/lToKr/wYDhsGzX4WirT0IWEWr+Ph4SktL\n+2yibIyhtLQ00ivxFQLvGGOq/audLqOD1rdQzCzUqic50UWyx6ED95RSMS+k8ySr6GGMYcXuUubn\nZiKdVHh7pPIIvHoLDJoC5/9nUIdkp3iYNSqddz8/yvfP78E0WUlZcM3L8PgF8Ncvwo3vw4Ch3T+P\nijrDhw+nsLCQnlYxY0F8fDzDh0d0IdLXgD+IiBNwYy0A9UC4LtZcSXbECSkeJ4lupy4mopSKeZok\n91G7iqooqaoPfT+yzwev3AwN1fDFx8EVfHXsnAmD+M3b2yiqrGNgSg+qauk5cPVL8MRF8Npt8I2/\nd9rioWKDy+Vi9OjRdocRU0TkOeBsIEtECoG7ABdYg6qNMVtF5G3gM8AHPGaMCdv0m82V5NQEFyJi\n9SQ3NOHzGeLi9N+oUio2aZLcR63c4+9HHhPifuSVf4A9H1qD9AZ2sIJeB+bnWgn7yt2lLJ4+rGfX\nH3IqnPcLeOvfIf95mH5Vz86jVAwzxnT5P74x5n+B/41AOMS7rEpyaoILgCS3lTTXNDaR7NFfM0qp\n2KQ9yX3Uil2lDEtLYERGQuhOemgD/Os/YeLl1nRv3TR5aCqpCS6W7yzpXRyzb4QRp8M7/wFVYZ3Z\nSikVhOYpJluSZH9irC0XSqlYpklyH+TzGVYVlHLG2BD2I9dXwUs3WPMVX/77HrU5OOKEM8Zm8umu\nkt4N0oqLgy/8n9Xy8daPe34epVRINFeS0xKbk2QradbBe0qpWKZJch+09chxjtU0Mi+U/chv/RjK\n9sCSRyAxyAVB2jE/N4tDFXXsLa3pXTzZ42Hhj2HLKzp/slI2a64kp7W0W/gryToNnFIqhmmS3Aet\nbJkfOURJ8q5/wcZnYOGPIGdBr061wL889vJdvWy5AJj/b9YMG2/8EOoqen8+pVSPnKgkuwFa+pC1\nkqyUimWaJPdBK3aXMiYriSGpIehHbvLCO/8P0kdbq+r10qjMRIalJfBpb/uSAZxuq+2i6qi1sIlS\nyhYep4N4VxwDB3gASGzuSW7QJFkpFbs0Se5jvE0+1hSUMTdUVeQNf4HirdZ8yE5Pr08nIszPzWTF\n7hKafCFYPGLYDGsxk3VPQcEnvT+fUqrbHHHCy7fMZ+m8HACSW3qStd1CKRW7NEnuYzYdrKCq3hua\n+ZHrjsOHv4KRZ1gzWoTI/Nwsjtd52XIoRC0SZ//UqnT/47vQ0MteZ6VUj0waOqClzSLR35Ncre0W\nSqkYpklyH9M8P/LcMSFIkpffD9XFcOGvQrpoxxljQ9iXDOBOhMt/Zw0sXP1QaM6plOqx5ingNElW\nSsUyTZL7mJW7Sxk/KIWs5F62RpTvg5V/hFO/ZrU0hFB2iocJg1P4NFRJMsCYs+CUi+DT30LtsdCd\nVynVbc2LiVRru4VSKoZpktyH1HubWLu3LDSzWvzrbpA4ODc8A+IW5Gaxdm85dY0h/CV6zp3WLBcr\n/i9051RKdZvTEYfHGacD95RSMU2T5D4k/0AFdY2+3vcjH1gDm/8O878LqT1cProL88dl0eD1kbe3\nPHQnHTwVpnwRVv1JV+JTymbJHqdOAaeUimmaJPchK3aXIAKnj+5FkmwMvPNTSB4MZ3w3dMG1MScn\nA2ec8OnuELZcgDWIz1sHn9wf2vMqpbol0ePQxUSUUjFNk+Q+ZMXuUqYMTSXVvzRsj2z+OxSuhXN/\nBp7k0AXXRpLHyYyR6aHtSwbIyoXTroa8x+HYgdCeWykVtCS3VpKVUrFNk+Q+orahiY37j/Wu1cLb\nAO/fbbUtTLsqdMF1YH5uFpsOVnCspiG0Jz7rDuu/H/8mtOdVSgUt2ePU2S2UUjFNk+Q+Yt2+chqa\nfL1bROSz56FiP5z7C4hzhCy2jiwYl4kxJ5bRDpnU4TD7Rtj4LJTsDO25lVJBSfQ4qdZ2C6VUDNMk\nuY9YuacER5wwOyejZyfwNcHyB2DIdMg9N7TBdeDU4Wkke5yhmy850IIfgDMePvx16M+tlOpSVpKb\noxV1doetTap6AAAgAElEQVShlFI9pklyH7F2bzlTAla86rYtr1iLcZz5w5AuHNIZlyOOuWMyWBHq\nSjJAcjbM+w5seRkOfxb68yulOjV2YDJHjtdRWddodyhKKdUjmiT3AY1NPvIPHGPmqJ5WkX3WbBBZ\n42HCZaENrgvzc7MoKKmmsDwMy0nPuw3i0+CDX4b+3EqpTo0baA383VVUZXMkSinVM5ok9wFbDh2n\n3utj5qj0np1g5ztQtAXO/AHERfZ/ifm51hLVK3aFoZqckAYLvgc734WD60J/fqVUh3I1SVZKxThN\nkvuAvL1lAMzK6UGSbAwsuxfSRsKUL4U4sq6NG5hMdoonPH3JYA3g86TCp78Pz/mVUu0amZGI2xGn\nSbJSKmZpktwHrNtXzvD0BAYNiO/+wQXL4GAezP8eOHrYz9wLIsL8sZms3FOKMSb0F/CkwKzrYOvr\nUFYQ+vMrpdrldMQxJjtJk2SlVMzSJDnGGWPI21fOrJ62Wnxyr7W63vSrQxtYN8wdk0lxZT27i6vD\nc4HTbwZxwMoHw3N+pVS7xg5MZqcmyUqpGBVUkiwiF4nIdhHZJSI/aWf71SLymYhsEpEVIjIt2GNV\n7xSW11JcWc/Mnkz9dmCtVUk+43Zw9aAKHSJzx1hzO6/aE4a+ZIABQ+DUr8KGv0J1mK6hlDrJuIHJ\nHCivoa5R50tWSsWeLpNkEXEADwIXA5OAq0RkUpvdCoCzjDFTgV8Cj3TjWNULefusfuSZI3tQSf7k\nPkhIh5nXhjaobhqVmciQ1PjwJclgfRDw1sLax8J3DaVUK7kDkzFGB+8ppWJTMJXkOcAuY8weY0wD\n8DywOHAHY8wKY0y5/+EqYHiwx6reydtbTorHyfjBKd078Mhm2PEWzP0OeJLDE1yQRIS5YzJZtacs\nPH3JAAMnwLgLYc3D0BCG6eaUUic5zf/hPWwDc5VSKoyCSZKHAQcCHhf6n+vIDcBbPTxWddO6feVM\nH5mGI66bC4Asvx/cKTDnW+EJrJvmjcmkpKqe3cVhrDjN/zeoKYX8Z8N3DaVUi2FpCZw6PJW3Nx+x\nOxSllOq2kA7cE5FFWEnyHT049iYRyRORvOLi4lCG1Wcdr2tk+9FKZnV3EZGKQtjyKsy61mq3iALN\nfckrw7H6XrNRZ8CwmbDiD9Yy3EqpsLtw8mA2HjjG4Ypau0NRSqluCSZJPgiMCHg83P9cKyJyKvAY\nsNgYU9qdYwGMMY8YY2YZY2ZlZ2cHE3u/t2H/MYzpwfzIax4FDMy5KSxx9cSIjASGpsazak9Z+C4i\nAmd8F8oLYNs/w3cdpVSLi6YMBuDdLUdtjkQppbonmCR5LTBOREaLiBv4GvB64A4iMhJ4GbjGGLOj\nO8eqnlu3t4w4gWkj0oI/qKEG1j1lLT+dNjJssXWXiDB3bCarwjVfcrOJl0P6aGtxkXBeR6kwEpEn\nRKRIRDZ3sd9sEfGKSORXCvIbm53M6Kwk7UtWSsWcLpNkY4wXuA14B9gKvGiM2SIiN4vIzf7dfg5k\nAn8UkY0iktfZsWF4Hf1S3r5yJg4ZQLKnG4uAbHoR6o7B3FvCF1gPzR2TSWl1Q3jnVY1zwLxbrQVU\n9q8M33WUCq+ngIs628E/u9BvgHcjEVBnhqcnUFJV3/J4+c4SbvnruvB+IFZKqV4KqifZGPOmMeYU\nY8xYY8yv/M89ZIx5yP/zjcaYdGPMdP+fWZ0dq3rP2+Rj44Fj3VtExBhY9RAMPhVGzgtfcD00L9zz\nJTebfjUkZupS1SpmGWOWAV31Jt0O/B0oCn9EnUtPdFNW3dDyePmuEt7afIR6r8/GqJRSqnO64l6M\n2nakkpqGpu4tIlLwMRRv9a9A183ZMCJgREYiw9ISwjt4D8CdCLNvtKbAK90d3mspZQMRGQZcCfwp\niH3DPmg6I8lNWdWJJPl4XSMADU2aJCulopcmyTEqb69VROpWJXnVQ5CYBVO+GKaoem/umExWF5Th\n84X5a9hZ10OcE/KeCO91lLLHb4E7jDFdZqGRGDSdkeSmst5Lg79yfLzWnyRrJVkpFcU0SY5RefvK\nGZIaz9C0hOAOKNsDO962kkMbl6DuytwxGZRVN7CjqDK8F0oZDJMWw4a/QEN1eK+lVOTNAp4Xkb3A\nl7DGi1xhVzAZSW4AymusavLxOi+gSbJSKrppkhyj1u0rZ2Z3qshrHrUqp7NvCF9QIdA8X/KqcLdc\ngDUFXl0FbPpb+K+lVAQZY0YbY3KMMTnAS8B3jDGv2hVPpj9Jbu5LrvBXkrUnWSkVzTRJjkEHj9Vy\nuKIu+FaL+krY8FeYfKVVQY1iIzISGZ6eEN75klsudjoMnmp9gNBR9iqGiMhzwEpgvIgUisgNbWYc\niirpbZLkSm23UErFgG7MHaaixbp95QDMCnbQ3sZnof64NWAvBswbk8l7W4/i8xniurvcdneIwOxv\nwT++a00HN+qM8F1LqRAyxlzVjX2vDWMoQWlbSW4ZuKdJslIqimklOQat21tGotvBhMEpXe/s88Hq\nh2H4bBg+M/zBhcDcMZkcq7GW3A67qV+G+FRY80j4r6VUPxVYSTbGcLzW35PcpMvDK6WilybJMShv\nXznTR6ThdATx17frfSjbHTNVZIC5Y62+5LBPBQfWdHCnXQNb/wHHD4X/ekr1Q+mJbkSsJLmu0dcy\n9Vt9o1aSlVLRS5PkGFNV72Xr4ePB9yOveRhShlgzOcSIYWkJjMxIDP+iIs1m3wC+Jmu5bqVUyDni\nhLQEF2XVDS2tFgD1Ok+yUiqKaZIcYzbuP4bPENwiImV7rEryzOvA4Qp/cCE0d0xGZOZLBsgYA+Mu\ngLwnwdvQ9f5KqW5LT7JW3WueIxm0J1kpFd00SY4x6/aVIwKnjUzreue8J0EcMGNp+AMLsbljMqmo\nbWTrkeORueCcm6C6CLa+HpnrKdXPZPqT5ApNkpVSMUKT5Bizbn854welMCC+i8pwY5017duES2DA\nkMgEF0It8yVHYio4gLHnWBXlNY9G5npK9TPpie6T2y00SVZKRTFNkmOIz2fYsK+cGcH0I299HWrL\nYFZ0Lx7SkaFpCYzKTGTl7pLIXDAuDmbfCAdWweH8yFxTqX4kM9lNWU1Dy8wWoJVkpVR00yQ5huws\nqqKy3svMkUEkyWsfh4yxMPqs8AcWJmeMzWLVnjK8kRrcM/3r4ErU6eCUCoP0RDflJ7Vb6BRwSqno\npUlyDGleRKTL5aiPbrEqorOutyqkMWpBbhZV9V7yCysic8GEdGve5E1/h9pjkbmmUv1ERpIbr89Q\nWF7T8lyDzm6hlIpisZtB9UPr9pWTmeRmVGZi5zuufRyc8VZlNIbN88+XvGJXhFouwPpg4a2Fz16I\n3DWV6gcGDYgHYMuh47j9c7zXN/p4fs1+fvDCRjtDU0qpdmmSHEPW77f6kUU6Waq5vtJK8CYvgcQg\nl62OUhlJbiYNGcCnkepLBhg6HYbOgLwnwERg+jml+onpI6wZedbuLSM9yYWIVUletaeUf20rsjk6\npZQ6mSbJMaK0qp6CkuquWy0+exEaqqyKaB8wPzeT9fuOUdsQwd7FWddD8TbYvzJy11SqjxuensDg\nAfE0NhlSE1x4nHE0eH3UNDRRVe/F6IdSpVSU0SQ5Rqzfb/XIdpokG2NVQAdPheGzIhRZeJ2Rm0VD\nk4+8fRGaCg5gyhLwpFrvpVIqJESEWTnW/WtAvAu3I456r4/axiaafIY6XaJaKRVlNEmOEev2leNy\nCFOHpXa804E1cHSzNe1bZy0ZMWROTgbOOOHTXRFaohrAnQTTvgqfvwbVEbyuUn3cbP9KoQMSXLid\nDur9lWSAyvrGzg5VSqmI0yQ5RqzfV87koanEuxwd75T3OLhTrBka+ogkj5PTRqaxIpJ9yWAt5d3U\nABufiex1lerDmivJge0Wza1UVXXezg5VSqmI0yQ5BjR4feQXHuu81aK6FLa8CtO+Bp7kyAUXAfNz\ns9h0sIJjNQ2Ru+igSTByHqx7Enz6NbBSoTBh8AAyktwMTo23kuQmq90CoFKTZKVUlNEkOQZ8fvg4\n9V5f50nyxmegqb7PDNgLND83C2Ng1Z4Itz7Muh7K9kDBx5G9rlJ9lCNO+MftC7htUS5uZxwN3iZq\nGqzkuKpek2SlVHTRJDkGdLmIiM9nVTxHzrMqoH3MtOFpJLodke1LBpj4BUjIsN5bpVRIDEtLIMnj\nxO2Ma92TrJVkpVSU0SQ5BqzfV86wtISWyfhPsneZVfGceV1kA4sQtzOOOaMzIjtfMoDLvyDLtjeg\n8khkr61UH+d2tOlJ1kqyUirKaJIc5Ywx5O0r67zVIu9Ja0nlSYsjF1iEzR+bxZ7iao5U1EX2wjOv\nA58XNvwlstdVqo/zuOKobmjC67PmR66q09ktlFLRRZPkKHeooo6jx+s7TpKrimDbP2Ha163KZx91\nRq61RPWnkVyiGiArF0afBev+DL4ILmiiVB/ndsS1GoyrlWSlVLTRJDnKddmPvOEvVqVzVt9stWg2\n0T8qPuJJMlgD+CoOwK73I39tpfootzOOYzUnqseVmiQrpaJMUEmyiFwkIttFZJeI/KSd7RNEZKWI\n1IvIj9ps2ysim0Rko4jkhSrw/mL9vnISXA4mDE45eaPPZ1U4c86ErHGRDy6C4uKEeWMz+XR3SeSX\nr51wKSQN1BX4lAoht9PB8YAWCx24p5SKNl0mySLiAB4ELgYmAVeJSNspFMqA7wL3dnCaRcaY6caY\nvrFWcgSt21fO9BFpOB3t/FXt+QCO7YOZ10Y8LjvMH5vF0eP17C6ujuyFHS6YcQ3sfBcqCiN7baX6\nKI8zjsDPu7qYiFIq2gRTSZ4D7DLG7DHGNADPA61GiBljiowxawEdeRFCNQ1ePj98vONWi7wnITET\nJl4e2cBsMt/flxzx1fcAZnwTjIH1T0f+2kr1QW5n618/2pOslIo2wSTJw4ADAY8L/c8FywDvi8g6\nEbmpO8H1d/kHKmjymfaT5OOHYftbcNo3wOmJfHA2GJmRyLC0BHv6ktNHQe55VntLk34WVKq33AHf\njrmdcVpJVkpFnUgM3FtgjJmO1a5xq4gsbG8nEblJRPJEJK+4uDgCYUW/9futQXunjUw7eeOGv4Bp\nsiqc/YSIMD83k5W7S2nyRbgvGawBfFVHYMfbkb+2Un2MJ6CSnJ3s0YF7SqmoE0ySfBAYEfB4uP+5\noBhjDvr/WwS8gtW+0d5+jxhjZhljZmVnZwd7+j5t3b5ycgcmk5bobr3B12RVNMecDZlj7QjNNvNz\nszhe52XLoYrIX3zcBZAy1GpzUUr1SmCSPHCAh6p6/YZGKRVdgkmS1wLjRGS0iLiBrwGvB3NyEUkS\nkZTmn4ELgM09DbY/8fkM6/eXM3NkO60Wu96H44V9doW9zpwxNguAT3ba0HLhcMLMb8Luf0FZQeSv\nr5SfiDwhIkUi0u79VESuFpHP/DMLrRCRaZGOsSuBPckDUzzabqGUijpdJsnGGC9wG/AOsBV40Riz\nRURuFpGbAURksIgUAj8A7hSRQhEZAAwClotIPrAGeMMYo99VB2FPSTXHahrb70fOe8KakmzCpZEP\nzGbZKR6mDkvlg21F9gRw2jUgcbD+z/ZcXynLU8BFnWwvAM4yxkwFfgk8EomguqN1khxPZZ038tM7\nKqVUJ5zB7GSMeRN4s81zDwX8fASrDaOt40DUVTBiwdq9ZQDMzGmTJFcUWlORLfi+NTVZP3TOhIH8\n/oOdlFU3kJHk7vqAUEodBqdcDBv+Cmf/FJwRvr5SgDFmmYjkdLJ9RcDDVbR/f7ZV4MC97BQPXp+h\n3usj3uWwMSqllDpBV9yLUmsKyshK9jAmK6n1hnV/tqYim7HUnsCiwLkTB2IMfLTdpmryrOuhutha\nDlyp6HcD8FZHG+0aNO3xJ8NuRxxpidYHfl1QRCkVTTRJjlJrCsqYMzodETnxZFOjNU9v7nmQnmNb\nbHabMjSV7BSPfS0XY8+BtJG6Ap+KeiKyCCtJvqOjfewaNN1cSU5wO0iJt77U1LmSlVLRRJPkKFRY\nXsPBY7XMyclovWHbG9YUZLNvsCewKBEXJ5wzfiAf7yimsclnRwDWKod7P4GSnZG/vlJBEJFTgceA\nxcaYUrvjaau5JznR7SDZY1WSdfCeUiqaaJIchZr7keeMzmy9Ie9xSB1hTUXWzy2aMJDKOi95e8vt\nCWD6NyDOCeuesuf6SnVCREYCLwPXGGN22B1Pe5qT5AS3g2SPVUmu1GnglFJRRJPkKLSmoIwB8U7G\nD0458WTJTihYZk1BFqcDWxaMy8LtiOODbUftCSBlEEy4DDY+A4119sSg+i0ReQ5YCYz3zyZ0Q+CM\nQ8DPgUzgjyKyUUTybAu2A56ASnJzu4X2JCulookmyVFodUEZs3MycMQF9CPnPQFxLjit/w7YC5Ts\ncXL6mAz+ZVdfMlgD+GrL4fNX7YtB9UvGmKuMMUOMMS5jzHBjzOPGmIeaZx0yxtxojEk3xkz3/5ll\nd8xttbRbuJykJljtFhW1WklWSkUPTZKjTHFlPXuKq5k9OqAfuaHGqlhOvNyqYCoAzp0wkD3F1ewt\nqbYngNELITMX1j5uz/WVimHNleR4t6NldotjNQ12hqSUUq1okhxlTvQjByTJW16Guop+P2CvrXMm\nWB8YbJvlQgRm3QCFa+Bwvj0xKBWj3A6rbSzRZfUkO+OEsmqtJCuloocmyVFmTUEZCS4HU4amnnhy\n7eOQPQFGzbcvsCg0MjOR3IHJ9iXJANO/Ds4EWPuYfTEoFYM8rhM9ySJCepK7w0ry5f+3nBfXHohk\neEoppUlytFlTUMaMUWknlmw9tAEOrbf6XwPnTFaA1XKxuqCUyjqbKlAJaXDql+Gzv0HtMXtiUCoG\nBc6TDJCe6KK8nSTZGMOmgxVsOVQR0fiUUkqT5ChSUdvI1iPHmZMTMPXb2sfBlQjTvmZfYFHsnAkD\naWwyLN9ZYl8Qs28Eby1sfNa+GJSKMYHzJAOkJbopb6fdosE/F7rOfKGUijRNkqPIun1lGAOzR6db\nT9Qeg00vwdQvQXxq5wf3UzNHpTMg3mnvLBdDpsHwOVbLhc+GxU2UikEn5km2pn/LSHS3W0mu91r/\npo5rkqyUijBNkqPI6oIyXA7htBH+JDn/eatCOUsH7HXE6Yjj7PED+Wh7ET6fsS+Q2TdC2W4o+Mi+\nGJSKIQkuByIwwD9HcnqSi/KakyvJ9Y3NlWQd1KeUiixNkqPI2oIyTh2eZvXoGWPNjTxsJgydbndo\nUe2cCQMpqWrgs4M29ixOvgISM3U6OKWClORx8tR1c/jyzBGA1W5xrKYBY1p/2K1rbAK03UIpFXma\nJEeJ2oYmPiusODH1295PoGS7VpGDcNYp2cQJfLDVptX3AJwemLEUtr8Jx3QUvlLBOOuUbFL9cyRn\nJLrx+gyV9a2T4eZ2C12yWikVaZokR4kN+8vx+syJJHn1w5CQAVOW2BtYDEhPcjNzVLq9fclgzUBi\nDKx7yt44lIpBLQuKtBm8V+/VSrJSyh6aJEeJ1QVliFgD0SjfZ1UkZ14LrgS7Q4sJ50wYxJZDxzlS\nUWdfEGkj4ZSLYP2fwVtvXxxKxaD0RDfASYP3mivJVXXek1oxlFIqnDRJjhJrCsqYNGQAA+Jd/oUp\nRFfY64bzJw0E4K3Nh+0NZM6NUF0Mn79ubxxKxZj0JCtJLmubJPsH7nl9hrpGnT1GKRU5miRHgQav\njw0Hyq1Wi4ZqqxI58XJIHW53aDEjd2AKk4cO4NUNB+0NZMw5kDFGV+BTqpvSm9stTqokN7X8rDNc\nKKUiSZPkKLDpYAV1jT5OH50Bn70IdRVw+s12hxVzrjxtGPmFFewurrIviLg4a7DlgVVw+DP74lAq\nxrS0W5zUk3yieqxzJSulIkmT5CiwpqAMgNmj0q0Be4NPhZFzbY4q9lw+bShxgv3V5Olft1ZJXP2w\nvXEoFUMGJLiIk/YqySeSZK0kK6UiSZPkKLCmoJSx2UlkFq+G4q1WFVnE7rBizqAB8czPzeKVDQft\nHeCTmAHTroJNL0KVzTNuKBUjHHFCaoKrnZ7kwHYLrSQrpSJHk2SbNfkMeXvLmTM606o8JmbClC/a\nHVbMuvK0YRSW17JuX7m9gcy9BZoadHERpbohPdHdsurezqOVHKmoo65VJVmTZKVU5GiSbLPNByuo\nrPdyzqAa/7Rv14Er3u6wYtaFkweT4HLwst0tF1njYNyFkPc4NNo4LZ1SMSQ9yd3SbnHrs+v533e2\nt6kka7uFUipyNEm22bIdxQAsKH8FJE6nfeulJI+TCyYP4o3PDrcaFW+LubdY08FtfsneOJSKEakJ\nLo7XWtXi0qoGjtU0tOpJrqrXSrJSKnI0SbbZJztLmDnUTcLmZ2HSF2DAULtDinlXnjaMitpGPtpe\nbG8gY86GgZNh5R+tlfiUUp1K9jhbEuHKei/VDd6WJFlEZ7dQSkWWJsk2qqxrZP3+cm5KXavTvoXQ\ngtwsspLd9s9yIWJVk4u2QMHH9saiVAxIjndSWddIg9dHg9dHTUMT9d4m3M44kj1ObbdQSkWUJsk2\nWrm7FK/Px5llL8OQaTDidLtD6hOcjjgunzaUf20toqLG5l+qU78MSdlWNVkp1akUj5PKOi/V/mpy\ndb2X+kYfHmdcyzallIqUoJJkEblIRLaLyC4R+Uk72yeIyEoRqReRH3Xn2P7sk50lXOjeRGLFTp32\nLcSWnDachiYfb9q9TLUr3lpcZOc7ULLT3liUinLJHif1Xl/LNHBWJdmHx+kgJd6llWSlVER1mSSL\niAN4ELgYmARcJSKT2uxWBnwXuLcHx/Zbn+ws5vuJb8GAYTDlS3aH06dMGTaAsdlJvGJ3ywVYgzEd\nblj1J7sjUSqqJcc7AThaYc0IU13vpd7bZFWS47WSrJSKrGAqyXOAXcaYPcaYBuB5YHHgDsaYImPM\nWqDtx/wuj+2v9pfWkFr2GRPq8mHud8DptjukPkVEuPK0YawpKONAWY29wSQPhKlfgfznoKbM3liU\nimLJHitJPnLcSpJrGpqsdguXJslKqcgLJkkeBhwIeFzofy4YQR8rIjeJSJ6I5BUX2zwrQQQs21nM\nt53/pMmTCjO/aXc4fdLi6db/aq/nH7I5EmDed6CxBtY9ZXckSkWtFH8l+bC/kuz1GSrrvS3tFjoF\nnFIqkqJm4J4x5hFjzCxjzKzs7Gy7wwm77Vs2cJFjLXGzbwRPit3h9EkjMhKZk5PBy+sL7V2mGmDQ\nZBh9Fqx5FJq0r1Kp9qTEuwA4UnFiAZ6y6nriWyrJ+m9HKRU5wSTJB4ERAY+H+58LRm+O7bMam3xM\n3f8XmsSFnP5tu8Pp066cMYzdxdWs3WvzMtUA826FykOwSRcXUb0jIk+ISJGIbO5gu4jI7/0Dpj8T\nkRmRjrEnmtstDgckyeXVjXiccaQmuKiobaTJp3OOK6UiI5gkeS0wTkRGi4gb+BrwepDn782xfdaW\n7TtZzMcczrkCUgbZHU6fdsX0YWQmufnjR7vsDgXGXQCDpsDy+8Hn63p/pTr2FHBRJ9svBsb5/9wE\nxMSo0eaBe0eO17Y8V1bdgMfpYEx2Mo1Nhn2l1XaFp5TqZ7pMko0xXuA24B1gK/CiMWaLiNwsIjcD\niMhgESkEfgDcKSKFIjKgo2PD9WJiRf2nf8SFl/Rzf2h3KH1egtvBdfNz+Gh7MVsOVdgbjAgs+D6U\n7IBt/7A3FhXTjDHLsGYV6shi4GljWQWkiciQyETXcynNA/cCKsm1jdbsFqcMSgZgx9FKW2JTSvU/\nQfUkG2PeNMacYowZa4z5lf+5h4wxD/l/PmKMGW6MGWCMSfP/fLyjY/u1+komH3qJ1fFnkDJ8gt3R\n9AvXzMsh2ePkTx/ttjsUmHwlZIyFZffqUtUqnHoz4No2zZXkkqqGVs97XA5yByYjAtuPVLXadvR4\nHUWVdSilVKhFzcC9/qJ21RMkmyoKxn/L7lD6jdQEF1fPHcmbmw6zt8Tmr2rjHFY1+chnsOt9e2NR\niuiaWSjB5SCunTWVPM44Et1ORmYksqOodSX5e89v5D/+vilCESql+hNNkiPJ2wCr/sjKpkmMn3mW\n3dH0KzcsGI3TEcfDy6KgmnzqV2HAcK0mq3AKetB0NM0sJCItg/ecAdmyx2n9qho3MIUdR1onyfvL\najh4rBallAo1TZIjafNLJNQe4WnHYqYNT7M7mn5lYEo8X545nL+vO8jR4zZ/Net0w/zvwoFVsG+F\nvbGovup1YKl/lou5QIUxxuY12oPTPA1cdoqn5TmP0wHA+MHJFJRUU+9tAsAYQ3FlPaXVDSefSCml\nekmT5Ejx+TCf/o5dMgoz5jycDn3rI+3bC8fi9fl47JM9docCM5ZCUjZ8cm/X+yrVhog8B6wExvsH\nSt8QOJgaeBPYA+wCHgW+Y1Oo3dZcSR44IL7lOY/Lul+eMigFr89Q4G+bqqhtpKHJR3l1Az6dGk4p\nFWJOuwPoNz5/BSnexu8abmPh+IF2R9MvjcxM5AvThvLM6v3cuiiXtEQblwJ3JVjzJr//Czi4DobN\ntC8WFXOMMVd1sd0At0YonJBqHrw3sFUl2UqSxw+2Fl7afqSSCYMHUFRZD1gr8x2va7T337RSqs/R\ncmYk+Jrgo3soTxrLP31zOXNclt0R9Vu3nJ1LTUMTf16xz+5QYNYNEJ8Kn9xvdyRKRY3mpalTE1zE\n+yvI8S6r3WJsdjLJHierC6zZ74r9STKgLRdKqZDTJDkSNv0NSnbwbOLV5GSlMCIj0e6I+q3xg1M4\nb+JAnlpRQE2D195g4gfAnG/Dtn9C0VZ7Y1EqSjS3WyR7nCS5rZ+bK8kuRxxzx2SybEcxxphWU7+V\naZKslAoxTZLDrckLH92Db9BU/nBkAgu1imy7W87OpbymkefWHOh653Cbewu4krSarJRfcyU52eMk\n0WNVkJsH7gGcdUoWheW17C2toeh4QCW5qh6llAolTZLDLf85KC9g3ZhbqG2Ei6ZE/aJXfd7MUemc\nPuUQ/8EAACAASURBVDqDR5btprKu0d5gEjNg1nWw+SUo3mFvLEpFgZZKcvzJlWSAM8dZ09R9srNY\n2y2UUmGlSXI4eRvg4/+BoTN4qngCWcke5ozOsDsqBdxx8QSKK+v59Zvb7A4F5n8PXInwwS/tjkQp\n2yV7XP7/Oklw+yvJrhO/qnKykhiZkciyHcUUVdYzNNWaBaOsSpNkpVRoaZIcThv/ChX7qTvzJ3yw\nvZiLpwzG0d5yUiriZoxM51tnjuG5Nfv5eIe9q4yRnA3zboOtr1szXSjVjzXPbpHSqpLsaLXPmeOy\nWLm7lEPHahmWnkBKvFMryUqpkNMkOVwa66wV1Uaczr8aplLb2MQlU7XVIpp8//xTyB2YzB0vfUZF\nrc1tF2fcBolZ1pRwugqf6sdSAgbuJTZXkp2tf1UtPCWb6oYm1u8vJzvFQ2aSW5NkpVTIaZIcLuv/\nDMcPwqKf8ubmI9pqEYXiXQ7u+/I0iqvq+eU/P7c3GE8KLPx3KFgGuz+wNxalbNRcSU7yOEnynNyT\nDDBvbCaOOMFnrNU0M5LclFV3PXCvoKSaqXe9w17/YiRKKdUZTZLDobEWPrkPRi2gdtgCPthWxEVT\nBmmrRRSaNiKNW84ay0vrCvnX1qP2BjPrOkgbaVWTfT57Y1HKJtNGpDF3TAYTBqecqCS7WrdbDIh3\nMWNkGmAtX52Z7KE0iJ7kXUVVVNZ7KSjVJFkp1TVNksNh7eNQdRQW/ZQPdxRrq0WUu/3cXCYMTuEn\nL2/iWI2NX9n+//bOO7yt6vzjn6NpW/LeI3Ycj+ydkB2yE8IIe5VZRinwg5ZCgdJJS4EWKBTKLmVv\nwkxYCWSRvR1nx3G8956ypfP740ryTpzEthz7fJ5Hj+U7pFdHV9L3vvd73tdghtkPQd5u2Pup5+JQ\nKDxIdIA37986hQAfkzuT7GVs+1PlqnLRGbtFQWUdDoekwmmrqrXZuyFyhULR11AiuaupKYG1T8Cg\n2TBwGstScgmxmpgUH+zpyBQdYDboeeKy0ZRW2/jTF6meDWbkZRA2HH74G9g97JNWKDxMkydZ32bd\nguHhGHSC5HBfgiwmSqttyHb8/IcLqpj66A+s2JdPhbPkY3W9hxsJKRSKMwIlkruaVY9BXTks+Bu1\nNjs/7Ctg4XBV1aK3MyLanzvnJPL5zhy+2ZPruUB0epj3JyhJ03ztCkU/pr06yS6GRPix608LGDMg\ngCCLiUaHpKK2rfj9bEc2jQ5JZmmte31tg8okKxSKE6NEcldSsB+2vArjb4SIEaw6UEBtg51zRymr\nxZnAHbMTGRHtx70f7WZjWrHnAklaALFTtRrbNuWdVPRfkiN8ifT3ItDH1O56lx0jxGoGoLBV1z0p\nJV/sygGgvLbB3Tyoul6JZIVCcWKUSO4qpIRvHgCzVfOVAl8pq8UZhVGv49XrJhLh78X1r23mx/0F\nnglECJj3Z83XvvEFz8SgUPQCzk4OZcODc91NRToi0tlQJLe8tsXynZllZJTUAFBeY3PbLWpsym6h\nUChOjBLJXcXBbyDtR5j1IFiCldXiDCXC34sPbp1MUriVW97cypfOLFSPEzsJBp8L656GCg/aPxSK\nM4CYIB8AMkuaRPKLq49w61vbMBt0hFjNlNU2uO0WNWrinkKh6ARKJHcFjfXw7e8gJBkm3gzQZLVQ\nVS3OOIKtZt69ZTLjYgO56/0dvLc5wzOBLPgr2G3w3e898/wKxRlChJ8XBp0gq1TLGmeX1fLY1/uJ\nD7Hw+o1nERXgRVlNg8okKxSKk0KJ5K5g00vaRKuFj4LeCMCylFyCLSbVQOQMxc/LyBs/P4uZSaE8\nuDSFV9ak9XwQwQkw4x7Y8zGkrer551cozhD0OkFUgDeZpVomeWt6CQB/PG8YUxKC8fc2UlbbQGWd\nyiQrFIrOo0Ty6VJVoE2wSloISfMAqGuw88P+AhaOiMCgV0N8puJt0vPKdRM4d2Qkjyzfx4NLUyio\nqOvZIKb9CgLjYdm92hULhULRLgOCvMl0+o+3ppdiNRsYEuELQICPqYUnubreTo2tkeUpysqkUCg6\nxuDpAM54Vj4MjbWw8O/uRasOFFBjU1aLvoDJoOPfV40lwt+L19ens3R7FtdNieO2sxMIds6o71aM\nXrD4CXjnElj/LMy8173K1uigoLKO/Ip68ivqyK+oo6K2kZqGRmrq7dTY7NQ2NFJjs6MTApNeh9Gg\nw6gXmA06THodwVYzYb5mQn3NhPl6EeanNWZQJ3eKM42YAB9WOifbbkkvYWxsgPs4DnBmkl2zQ2ob\nGvlqdy6//Xg3q++bRVywxUNRKxSK3owSyadDzk7Y8TZMuQNCEt2Lv9ytWS0mKatFn0CvE/zhvGFc\nOzmOf/9wiP+uO8o7mzK4YepAbp05iIAOylOdLlJKiqpsHDOOIyJyAeGr/sEjR4exrcKPnLLaDjuM\nmQw6fEx6LCYD3iY9PiY9DilpaJQ02B3Y7A4a7A7qGhyU17ZtWGLQCeJDLCRH+JIc5svgCCvJ4b7E\nBVvUJFRFr2VAkDdFVfUUVNRxIL+yRZfTAB8jFbUNCKEdv9X1dkqcn5/CynolkhUKRbsokXyq2Bvh\nq1+DTzDMvM+9uLCynu9S8/jZpDiVjetjDAyx8NTlY7h9ViLPrDzEC6uP8NaGY8weEsaoGH9GxQQw\nPMrPXbu1MzgckqLqenLK6kgvqiatqJr0omqOOm9Vzs5gEZzPSvNqFhx7kiNRjzAiOpxwPy8i/LwI\n99MywOF+XgR4G0/quLM1Oih0CouCynoKKuvJKavlUH4VKVnlLE/JxdXEzNuoZ1xcABMHBnHWwCDG\nxgaesDSXQtFTxARqFS6+2JWDlDBhYKB7nb+3EYcE18FcY2t0nyAWVXmwFb1CoejVKJF8qqx/BnK2\nw6X/A+8A9+L3NmfQYJdcNyXOg8EpupPEMCvPXjWWO2Yn8OKqI2w+WuJuWCAEJIZaGRHtj8WsRyDQ\nCdwZLIeUFFTUk19ZR365JkwbHU2tdIWAmEBv4kOsjIsNYGCIRbsFWzDvL2Pqij8wdXoxDJncJa/F\nZNARHeBNdIB3u+trbI0cLqjiYH4Ve7LL2Xy0hGdWHkJKLeM8ItqfqQnBzB0axpgBgSrTrPAYA4K0\nY/i/647iY9IzZkDT93Lrqz01NjtlNZpILungioxCoVAokXwqFOzT2k8PvQCGX+Re3GB38M6mY8xM\nDmVQqNWDASp6giERfjx95VgACirr2JNdzu6sclKyytmUVkxdowMpJRItYywBnRCE+pqJ8PNickIw\nEX5eRPhrGeFBoRYGBPlgNnSQnZ3yS9j9Hnx9PwyaBabuv0TsYzIwKiaAUTEBXDo+BtA6l20/Vsrm\n9BK2HC3hpTVpPL/qCMEWE7MGhzF/WBgzkkJPKqOuUJwurkxybnkdd8xOwMfUdPwFeBvd9wN9jNTY\n7FQ4M8nFVWpCrEKhaJ9O/YoJIRYBzwB64FUp5WOt1gvn+sVADXCDlHK7c106UAnYgUYp5YQui94T\n2Bvhs1+C2RfOfUpL/Tn5Zk8e+RX1PHqxyiL3N8J8vZgzxIs5Q8K770n0Ru2Y+98iWP04zH+4+57r\nOPh7G5k9JIzZQ8IATTSvPljIyn35fL83j0+2Z2HS65iRFMIFY6KYPyy8hWBRKLqDUKsZk0GH2aDj\n1hkJLdYF+DSJ5Ah/b44WVVFWq2WQO/L2KxQKxQl/uYQQeuA/wHwgC9gihPhCSrm32WbnAEnO2yTg\nBedfF7OllEVdFrUnWf8M5OzQbBbW0Bar3lifTlywD7OSwzwUnKLPEzcFxl6rVbpIPkf738P4exu5\nYHQUF4yOotHuYOuxUr7fm8/ylFxW7i/Ax6Rn4fAIloyJYnpiiPLqK7oFnU5w7eQ4hkb64d9MFEMr\nkexnZl9uBSXVzkyyEskKhaIDOpPeOQs4LKVMAxBCvA8sAZqL5CXAm1JKCWwUQgQIISKllH2rCKXL\nZjFsCYy4uMWqPdnlbD1Wyu/PHYpO+TIV3cmiRyF9LSy9FX65Drz8PR2RG4Nex+RBwUweFMxDi4ey\n6WgJn+/MZnlKLp/uyCbYYuKisdFceVYsiWHKkqToWv5w3rB2l/t7N3mSI/y9AMgr1xqPlFQru4VC\noWifzqR0ooHMZv9nOZd1dhsJrBBCbBNC3NrRkwghbhVCbBVCbC0sLOxEWD1Mc5vF4ifbrH5jfTre\nRj2XTRjggeAU/QqzL1z8KlRka01Geik6nWBKQjCPXTKKLb+fx4vXjGfiwCBeX5/OvKdWc/lLG/h8\nZzZ1Dar7maJ78W/mSQ7300RyaY3Lk6xlkvPK65jz5CpW7M3v+QAVCkWvpCeMgtOllNlCiDDgeyHE\nfinlmtYbSSlfBl4GmDBhgmy93uO4bBaXvd7GZlFabePzXTlcNj6mxZexQtFtDJgIsx6AHx+BpPkw\n6nJPR3RczAY9i0ZEsGhEBIWV9Xy8LYv3Nmdw9/s7CfAxcsm4GK6ZHEd8iKpXq+h6TAYdFpOeapud\nCKdIdlFcbUNKyX0f7yKtsJptGaXMG9aNcwsUCsUZQ2cyydlA8/RojHNZp7aRUrr+FgCfotk3zizy\n9zptFhe2qGbh4v0tmdgaHVw3ZWDPx6bov0y/BwZMhmW/gdJjno6m04T6mvnlrARW3TuLd26exLTE\nEN7ckM7sJ1Zx4/82s+ZgIVL2vvPk3oYQYpEQ4oAQ4rAQ4oF21vsLIb4UQuwSQqQKIW70RJy9hQAf\nEyaDjkBLk/XCYtJTUm1j6fZs1h4qQggto6xQKBTQOZG8BUgSQsQLIUzAlcAXrbb5ArhOaEwGyqWU\nuUIIixDCF0AIYQEWAHu6MP7up64CPrpe830ufqLN6ka7g7c3HmPKoGAGR/h6IEBFv0VvgItf1u4v\nvVWzBJ1B6HSCaYkh/Ofqcfz0wBx+NS+JlOwKrnttM/OeWs1bG49RXX9mvaaeotmE6nOAYcBVQojW\nhtw7gL1SytHALOBJ53d4v8TP24iflxGfZg1w4kMt2B2Sz3ZmE+XvxfjYQHLKaj0YpUKh6E2cUCRL\nKRuBO4FvgX3Ah1LKVCHEbUKI25ybLQfSgMPAK8DtzuXhwDohxC5gM7BMSvlNF7+G7kNKzYdcfKRd\nmwXAin0FZJfVcv3UgT0enkJBYByc+yRkboR1//J0NKdMmK8Xv5qXzE8PzOapy0fjYzLwh8/2MOXR\nlTz29X7yK1R2rxXuCdVSShvgmlDdHAn4Okt0WoESoN+edQR4G/HzNrQoRzgoRJs8ujGtmLFxgUQG\neJPX6ljLLqvl+tc2U17TtoW7QqHo23TKkyylXI4mhJsve7HZfYmWtWi9Xxow+jRj9Bzr/gX7v4KF\nf4eB09vd5I316UQHeDNvqCr7pvAQoy6HQ9/Bqke1JiMDJno6olPGbNBz8bgYLhobzfaMUv677igv\nrznCf9elsWRMNLfMGKSu2Gi0N1l6UqttnkO7ypcD+AJXSCkdPRNe72Ph8HCKq21YzE2Z5EGhmge+\nwS4ZOyCAgsp6vk2tQ0rp7pK55mAhqw8Wsju7jBlJbRMlCoWi76Iq/HfE4ZXww19hxCUw+fZ2NzmQ\nV8mGtGLuXzRE1X5VeJbFT0DGJvjwWrjlB/CL8nREp4UQgvFxQYyPCyKjuIb/rkvjw61ZfLwti7OT\nQ/nFzEFMSQh2CxlFuywEdgJzgAS0idNrpZQVzTdyVh26FSA2NrbHg+wpbpgWD0BGcY17WfPOqGNj\nA9mVWYat0UFpTQNBTu/y0aJqQOvkp1Ao+hdK2bVH6TH45CYIHQIXPNuiq15zXlmbhtmg48qJquyb\nwsN4B8BV70F9Jbx3JdiqPR1RlxEb7MNfloxg/QNzuHdBMqk55Vz96iYueO4nlu3Oxe7ol5P8OjOh\n+kZgqdQ4DBwFhrR+ICnly1LKCVLKCaGhfT9T6tM8k+yspmLUC4ZH+RHprKGcW97kS04r1D5L+b1Q\nJGeW1PDU9wf762dAoeh2lEhuTUOtlo1zOOCKt8HUfkmqPdnlfLI9i+umxLWYLa1QeIyIEVonyLwU\nbSKfo29dWQ+0mLhzThLr7p/D3y8aSWVdA3e8u525T67i3U0Z/a3ecmcmVGcAcwGEEOHAYLS5I/2a\n5hP3XHaLYVH+eBn1zRqNNAnio0VVAOR2whdf0gPd+3LLa7n5jS1U1DXw5e4c/r3yEHuyy7v9eRWK\n/ogSyc2REr66B3J3aVUDghM62Ezy8Jd7CfIx8X9zk3o4SIXiOCQvgIWPal76lX/2dDTdgpdRz9WT\nYln5m1k8/7Nx+Hkb+d2nKUx//EeeX3WYirq+P8GqkxOq/wpMFUKkACuB+6WURZ6JuPfgZdAjhCaW\nfUwGYoN8mJ4YDECkvzfQZK2wOyQZJZo9o7lwvuPd7Xy/Nx8pJa//dJSiqnr25lQw/m/f8+OBgm6N\nf2t6KSv2FbAnu5yiSk2Ub0gr7tbnVHSORvvpJSayu7mySm55LQVqEvRJoTzJzVn3FOx6F85+AAYv\n6nCz5Sl5bE4v4e8XjcTPSzUPUfQyJv0Cig/BT89AcCKMu87TEXULep1g8chIzhkRwYYjxbyw+gj/\n+OYAz/94hJ9NjuWmafGEtWoc0ZfoxITqHLSym4pm6HQCH6MeP2fjp6/umo63Ucsuh/qa0euEWxBn\nl9bSYNesDK5ldQ12lu3OxaTXER/iw5+/3EtpTQN+3kakhLc2HGP24O6byO06CSysrKeoSmupveFI\nMbed3X5SR9EzVNU3MuXRlTy8ZDgXjY056f1Tsso5/7l1fHnndEbG+J/0/uU1Daw+VMj5oyI7nKtx\n57s78Pc28toNZ+7k7p5GZZJdbHoJVj4MIy+Hs+/vcLO6Bjt/X76PIRG+XKG8yIreiBCw6HFImAtf\n/RqOtmlw2acQQjA1MYS3bprEV/83nVmDQ3llTRrTH/+RB5fudk+8UihceJsM7u6ofl5GjM6J13qd\nIMzX7M4kpzmtFsnhVndpuNIaLXt7uKCKA3na+i3pJezIKAVg1YGCFp7mrqaiVqviV1hZT3G1JpK3\nppfQcJpZzN5IVmkNF/7npy4vAbnhSDHLU3K79DGPFVdTWdfIWxtOrbHTwfxK7XFKTu376pPtWdz1\n3g725la0u97ukOzJLj+tOuAFFXX9zdamRDIAO96Gr38LQ86DC18AXcfD8uraNLLLavnj+cPQ69TM\nekUvRW+Ay/6nZZI/uBYKD3g6oh5hRLQ/z109jh/vncXlE2NYuj2bOU+u4pdvb3OLGIXCYta7RXJr\nIvy93CLXdYI1NSGEkmobdQ12t+/4SGEVB5zCZntGKduOlTI6xh+HhH9+e4D9ee2LFRcr9+Xz+c7W\ncy1PjDuTXFVPUaUNs0FHtc1OSg/6kh0OyXubOzcPYFdm2SkL+J2ZZezMLGP1wcJT2r8j/rXiIH/8\nfE+XdvbMKdOE/PaMMtJP4cTcJV6Lq07N1+7af92h9h1Vx4qrqW90nLJvXkrJec+u46nvD57S/mcq\nSiTvWQpf/J+Wdbv0NU1cdEB+RR3PrzrCwuHhTE0I6cEgFYpTwMsfrv4A9CZ4/Two2OfpiHqMuGAL\nf7twJOvun8PtsxL46XARFz2/nstf2sDKffk4VDWAfk10gDdxwT7trovy93Znko8WVePrZWBYlB8A\nBRX1bpFRY7O7xVtdg4Pc8jrOHx3FxWOjWbo9m0VPr+Xu93dQ7LRENEdKyR8/T+Wf3578yWtFrVMk\nV2h2C5e1Y1NaiXubg/mVbEkvaXf/zrI/r4JfvLW1XSG8LaOUB5em8MWunOM+Rk5ZLUv+8xOvrD21\n+aKusd6RUeZe9uOBgtPy7kopOZRfSVGVjfyKtu/NqdI8Q7t0x8mf/OQ4j7n2jpfO4Dpm1x1uXyQf\nyNNO6EprbKd0clBUZaOgsp5tx3o22bDtWKlHr5L0b5F84BtYegsMmKxVsjCYj7v549/sp9EueWhx\n6+6vCkUvJXAg3PAVCB28fi7k7vZ0RD1KqK+Z+xYOYf2Dc/nDecPILq3lpje2suDpNXy4JZP6xv51\n6VCh8fJ1E3h4yYh21w0M8SGjpIb6RjtHCqsYFGJpURqueSZuV2YZY2MD3P+PjQ3gyctH89MDc7hr\nbhLLU3K54Lmf2JvTMqu8K6uc7LJasstqqbWd3DFYUdfojKWOkhobyeFWAnyMbpEmpeT2d7bzq/d3\nntTjtmb57ly+Tc3nUH5Vm3VHCrRlJ6qq4crEf7Al85ROTF1ZVddVoMLKem56fQuvrDn1Ii3F1TZK\nnd0Tu7IqSE5ZLSaDjsmDgvh2T95J7++6elF0iple1/6bjpa0e2Kz3ymSG+zSfQx1RKPd0eYx0ou1\n93JfbkWXlhzckl7CxEdWtGtRSskq55IX1rN0e1aXPd/J0n9Fctoq+PA6iBilZdtM7WcVXOzMLGPp\n9mx+Pj2e2A4yEApFryR0MNy4HAze8Mb5kL3N0xH1OFazgZumx7Pqvlk8fcUYjHodv/1kN9Mf/5F/\nrzx0ytkbxZmJ1WzAy6hvd92QCD/sDsmh/CpScyoYGtlUPzmvoq7N5eqpCcHEh1gw6ATDo/wRQhAd\n4M0985P59PZp2B2SK17eQHltU9UVlx9WSs22cTJUOu0WB/MrkRJCfM0EW0xuf/Kqg4UcLqgiu6yW\nqvq2YqjG1thGtLeHy9t6rKSa8toGNqUVuzOQLvHb3OJxuKDKPZHQRaazMsix4ho2Hj35Chwu//fB\n/Eqq6xv5JjUPh2xZBUJK2SkB/vG2LK797ya397d1/KdLdlktUf5eDInwOyXfb5Pd4tS+i/LK6wjz\nNWNrdLR7FcGVSQYoPYEQ/+0nu5n75GqOFTfZRo4664XX2OxuwdwVfJ2SR2FlPd+l5rdZ902q9jnZ\nfNRzVrn+KZIPfgfvXa35Na/5BLz8jru5VvItlRCrmTvnJPZQkApFFxKcoAllL39480KtO18/xKjX\nceHYaJbfNZ23bjqLYZF+PPX9QaY+9gMPLt3NoWY/oIr+ydBIre35in35lNU0MCLanwhnabi88jpK\nq20IAQE+mqc5KcyXS8fHcMHoqDbCe0S0Py9eO57Kuka+dgtjybLducQGacmWkxXJLrtFsVPohFjN\nBFvNFDmzrv9de9S9rSvj25w3NxzjvGfXklVa02ad3SH5LjUPh0O6hfSx4hqeX3WYK17eyJ3v7qCq\nvpE0p0jem1PhLnt2/WubufPd7S0eL7O0Br1O4Odl4P3NmZwsrtfokLA7q5yvnPaO5lnHh7/ay4XP\n/4SUkhpbIzW29rOkm9KKWXuoiC93ae9DkMVEak7HIjmzpAZbY+cv8+eW1xHp702wxURlfeNJT3DL\nLXPZLU4+k2x3SPIr6zl/dBQGnWhhvXFxIL8SX7NmJy0+gUjem1NBdlktV7y00V0y7mgzYZx6gpOs\ntzce48f9nSuF6CpfuLKd7V3C2ZPzSfqfSN70Erx3BYQkwnWfgU/QCXf5aGsW2zPK+O3CwVjNqmqe\n4gwlME4TypZQeOsiSF/n6Yg8hhCCGUmhvPHzs/j+1zO5eJw2yW/+v9Zw3Wub+XF/gfIt91MGBlsw\nGXR8vE27xDsy2h+r2YDVbHBbHAJ9TCSHa2I6KdzKHbMTeeqKMe0+3ugYfwaFWtw+1a3HSskuq+WX\nsxLQCS0D25z6Rvtxj73Wl8pDrGZCrCaKq+opqKxj3eEiLh4bDcChdkRyWmEVDgmfteObXX2wgFvf\n2sb7WzLdHtn0ompSsyvw9zayLCWXN9ank1ZYhUmvo77RwaGCKirrGsguq2Vjmlbl44tdORRU1JFZ\nUktUgBfnj45ixb78kxaOJVU2EsO01uGfbM9ic3oJBp1wT5JzOCRf7sphd1Y5B/OruP61zVz/2mak\nlFz32mZeWn3E/Vgui8Un27Pw9TIwMynEnUnemFbMmIe/o6BSe9zc8lrmPrmaNzekdzrWnLJaogK8\nCfHVbJsnEqJFVfXsztK81hV1DVQ6s/7N9+vsd1BhZT12h2RQqIUIf682J0C1zuzvWfGa3jne5D0p\ntdrgM5JCKK6u518rtIl66UXVDAjyxqTXHffkQkrJ41/v5y9fpnbofXY4JF/syiG/oo79eRVYTHo2\nHinmWHE1649onuqjRdUcKqgiOsCbtKLqE2a/u4v+I5LtjbD8Pq2KRfIiuPFrsJ64luXurDJ+//ke\nJsUHcen4k699qFD0KvxjNKHsHwNvXwqpn3k6Io+TFO7LoxePZMODc/nN/GT25VZw4+tbmPXEKl5Z\nk0ZZjWe+nBWewaDXMTjcl6zSWgw6weAITQxHBXiRVap5kgN9jCSFWdHrBAmh1uM+nhCCi8dGs/lo\nCZklNby3KQOr2cCSMVHEBVs4XFDFo1/v4+2Nx5BSMv+pNTz7w+EOH6+itqFFZaUQq4lgi5niahtZ\npVqG9ZyRkZj0Og4VtL0y4trmk+3ZbUSMS7A/+8MhQGvXfay4hv15FSwcHs7wKD9WHSggo6SG2UO0\nFuYp2eUtyixe/9pm7npvB//+4RCZpTUMCPRh0YgIamz2DisvdERJtY1BIRaSw618vC0LKeHcUZHu\nSiO7s8vdGfR/fnuALemlbEkv5cnvDrLmYGGLMm8u64at0UFyuC8jov3Jr9BOLLaml1BW08Dmo1oG\n9qOtWdjsDvf/J6LB7iC/oo7oAC9CrE6RfALbxHM/HObKlzfSYHe4s8ghVpPbsiKlZNEza7jnw53U\n2Bp5ZU1ahzYOV2Y90t+LKH9v9wmOi0MFmjVnSkKwc1zrOzwZK662UWOzM2dIGD+bFMeHW7M4UljF\n0aJqksJ8SQq3Hteuk1teR2V9I+nFNWxJbz8DvO5wEXe9t8N5QgO3nZ2Aze5g4dNr+NmrmyisrOf7\nvZqv+575yYB2crk7q6xLK5J0hv4hkusr4f2rYPPLMOXO47abbk5BZR23vrmNUKuZ5382Dp0q+abo\nC/hGwA3LtDbWH10P3/1eO4ns5wRZtA6aP90/h2evGku4n5lHlu9j0t9X8tuPd6nWv/2IIU5hbCse\nAAAAIABJREFUnBzu67ZQDAy2kF5cTUm1jWCLmdvOTuA/V4/r0NvcnAvHRiME/OXLVJal5HLh2Ch8\nTAYSQq1sTCvmpdVpvL8lg7yKOjJKali5X+vm9+HWTDKKW2YFK+oa3FYNcHqSrSbKahrcAjgm0JtB\noRYOtzPpLqu0Fh+TnqNF1Zz37Dpuen2Le12a03fqqpQwLTGEPTmaEB0a6ce0xBC2pJfSYJfMGRKG\n1WxgT3a52zKyeGQEFXWNBPoY2ZpeSmZJLQMCfZgUH4yvl4Hv9p7chLbiahvBVhNv3zSJd26exGd3\nTGNGUqg7xh/25aMT2vu1Yl8+XkYdfl4GnvtRO8nYl1vptkyUNjvZTQ63MiJaa9ixN6eCY84x3n6s\nDIdD8sEWzRrS3LO8bHcuH2zJaDfO/Io6HBKiArwJtpoA2vizW5NVWkONzc7B/Eq3+B0R7U9lXSO2\nRgfZZbUczK9i6fZspj32A48s38czKw61eIwdGaWM++v3rD+iWRYi/LyJDGgqYfj93nyq6hvdk/Zc\nVblKqhtY8txP3Pzm1jZdAl0dJmODfLhzTiJeBh3/+GY/6cXVDAy2MDzKj5Ts8g4nnB5oZln7aGv7\nFpvtTvvE/rxKvI16bp4xyG1fklKbHLgprYSEUAvnjIxArxPc/8luLnjuJz7elsXXKblMfXRlj7SB\n7/siuTwLXlsEh1fCef+ChY+A7sRfavWNdm57axvltQ28fN14gq3Hr3yhUJxRWEPhhuUw8RZY/yy8\ndSFUdW873TMFk0HH+aOj+Oi2qSy/awYXj4vhy125nPfsOpY8t453N2W4J08p+iZDIrV5KiOjmzqf\nxYdayCiuoajKRqDFyIAgLUPaGWICffjdOUNZsa+A+kYHV58VB0BimNVtAziYX8WebC1Dtye7nA1p\nxfz2491c9tJ69wSq+kY7dQ0OEkK1JI/JoMPXbCDYogkzV4Yv0t+LxDCr225RUFnHI8v2UtdgJ6es\nlkvHxxBiNZFeVM3K/QVuH29aUTUWU1P3wfGxgdQ4xdCQCE0ku0gItTIi2o/tGaUcKajGoBM8dfkY\nVtxzNtdPHciB/EqKquqJDfbBZNAxd0gYK/YVdLp1s8MhKXVaW8L8vJiWGMKYAQFEBTgrjZTVsnJ/\nAeNiA7l8gtbY64LRUVw1KRaAKYOCsdkd7nrVpdU2hjnf1+RwX7dd5nBBFcecwnB7Rik/HSkiu6yW\nsbEB5JbXuS0Yz/5wiIc+3UNaOx5y10lFVIA3oU6t4GoZvj+vgvOeXdvmZMe1z+6scnKconaU83gr\nqba5y97NGxqOXicYMyCA5XtyW1hW3t6YQUm1jf/9lO58fi8i/L3IL68no7iGW97cypsb0jmQV4mX\nUcfgCF+8jDoySmrYn1fJD/sLePirvS3iymwmkkOsZm6dmcC3qfnUNTiID7Vw7qgoymsbuO3tbe1W\nBzroFOTnjIhgWUpui3buLnZmlhEfYiExzMr0pBC8TXo+vX0a39w9E9A8z7uyyhk9IAAfk4Ghkb7u\nKzhPrzjEn75IJae8jrWHurZ+dnv0bZF84Gt4eTaUZcDPPoIJP+/UblJK/vhZKtszynjistEMjzr5\nFpEKRa/HYIJzn4CLXoKsrfDSTMjc7OmoehXDovx49OKRbPzdXP50/jDqGhz87tMUznpkJfd9tItt\nx0p6/PKfovtxTd4b0aw98KAQCza7g7TCKoKcovRkuGXmIP5xySh+cfYgd91ll9h1VSVwTe5zSHhk\n2T4MOoGt0cFd7+0AoNLpR3ZZPEKtZoQQ7iROak45XkYd/t5GksJ8ySytodZm59s9ebyy9ihf7sqh\n0SEZEuHH5t/N45+XjQaaMshphdUsGhFJiNXEiCg/4kKarrgOjfRl4sBATM7uhINCrUxLCCE1p4LN\n6SXEBvvgZdSTGGZlQlwQro9FTKA26XHB8AhKqm3uiVonoqKuAbtDthnrKOckyl1Z5aTmVDB7SBjn\nj45iUnwQt84cxC/PTuC+hYP520Uj3NvZHZKy2gbmDQ3jictGc8n4GIIsJoItJg4XVLmFYWpOOS+s\nOkKQxcS9CwYDsDuznPpGO4cLqmh0SB7/Zn+bWF2Z4KgAr6ZMsqvayIFC9mRX8McvWjYvcYnHXZll\n5JbVodcJhjpFfFFVPTsyyvAy6njhmnFseWgev56fTGVdI6sOaMKwxtbI13ty3du73vcof29sdgcb\nneO8/Vgp+/MqSA73Ra8TBFvMbHJWGhkV48+bG47x/d6myhKusYgJ1K5W3DwjnhDna4oPtnB2ciiP\nXTyS1QcLef7HJs+3iwN5lUT4eXHvwsEI4FZnrW0pJW9tPEZBZR07M8uYODCQL+6cxrNXjdUeO8TC\nwBALMYHerNyXT1FVPaNjtPKKDy0exjNXjuHpK8eSXVZLQWU93kY9aw6enH3nVOibIrmuHD67Hd67\nUvMd3/QdJM7t9O5vrE/ng62Z/N+cRM4dFdmNgSoUvYDRV8LNK8DgBf9bDJteBiX8WuDvbeTGafF8\n86sZfHbHNC4cG8XylFwueWED8/+1pkVZKcWZz8SBQfx6XjIXjIpyL4sP0YSpQ0Kgz8mLZIDLJw7g\nwXOGuv+fmhjC2NgA/nahJui+Sc0jxGrCoBOk5lQwLTGES8bFcNBpm3CJ5EFOce0SL65M8p7sciL9\nvRFCkBRudZeYc11C/2q3JqpiAr3R6QRJzklxhwoqqahroKiqnqRwK2/8/Cz+csEI4py2jkh/LwJ8\nTPiYDIyNDcDf20igj5EZyaFICZuPlrTwZo8e4I/LnTjA+RhzhoQRYjXz6tqjHCms4s9fpB63Trlr\nAptLdLqIcJbj+2ibdin/7ORQQn3NfPCLKSSG+RLgY+KO2YkMCrEQZDGxO7OMitoGpIRAi4lLx8fg\n56Vd2k8Is5KaU0FeRR1DI/1osEvWHynmpunxjI0NQCdgd3a5WyCPjPbn29T8Ng01dmWWO8fJGx+T\nAR+T3p1JdlWCWHWgkG+c9ZPrG+3u17crq5zM0hoi/LwI82ua9Lcjs5RR0QEY9TqEEExLCCbEauLp\nFQf57ce7eGTZPmpsdhYMC3c/txDCXa5wrbOpyPaMMg7kVTLYmTkPtBjdJ0VPXT6aIRG+/P6zFHcn\nx4ySGkJ9zXg7ryhYzAbumT8Yk17n9udfMTGWBcPCeX19OtWtygweyK8kOcKXhFArT185lpTscv7z\n42G2Z5Tyh8/2cNtb2yiraWDMgEB8TG1LMQ6L9GOrc3xHOU9SpyQEs2RMNDOTQrh4bDR3zk5kztAw\n1h4q7PYkRd8TyUd+gOenwq73Yca9cMuPEDb0xPs5WX+4iL8u28e8oWH8el5yNwaqUPQiIkbArasg\ncR58fZ9WT7mftLI+GYTQLns+evEoNj80j39cOooIPy93tkzRNzDqddw9Lwl/n6bW1fHNsqqnkklu\nj+gAbz69fRqzh4Rh0uuosdkZFRPg9ssuHhlBqK+Z2gY71fWN7vJvob5m/L2N7klirkxyaU0DEX6a\nSEoObxLALpHs6sbmEq5xwRb0OsHhgip3Hdz4EAvDo/yJDfZhYLD2ml0ZToAHzhnCIxeNQAjByGh/\nt5fUJdwBfL2MDI7Q9hngzEh6GfXcOG0gqw8WcvUrG3l9fbpbXLp4dW0a1/53E1JKt980yNLS6uhl\n1BNsMZFWWE2gj9FtoWiNEIJRMf7sziqnxOlHbn1ykxRmJSW7HCnhwjHaCZGv2cA1k+PwMRlICvNl\nd1aZ28by6MUjCbaYeGblIQor6/nPj4e5+Y0tvPbTUeYNDcPirH4VYjW761bvzSlnzpAwhkT4OhuS\nOShwdvqL8PPiYH4lX6fkMXlQsPu15pbVkppd0aJRjUGv42eT4jhaVM13e/N5Z1MGA4K8+f25WnMz\nlziOCtC+i35yvtcl1TaKqmxuget6DrNBR3yIlccvGUVhZT0vrtKywhklNS087wBXT4pl+x/nE+rb\n9F7cNiuB8toG3tvc5NO2OySHCqoY7Dz25g8LZ2ZSKEu3Z/N1inaCsN1pI2n+2prjuspiaJZZb/6e\nPnXFGO5dOJiZSSEUVNa7TyC7i74jkuur4Kt7tNJWJh+46XuY+wftknInOZBXye3vbic+xMK/rhij\nJuop+hfeAXDlu3D+M5CXAi9Mg5UPg61tPVWFlmG5fMIA3r55Ej4mVRqyrxNiNbnrzHaVSHZh1OtI\ncgqLIRG+TEsMxmTQsWBYhFsIF1bWu7N9fl5GzhsVyewhYe7YXLjEUlyw1uDkUH4VGSWaHcDVKc3l\n6zUZdAwM9tFEsrNCRUIzsevvY2TMgABmDw51LxsbG8h5zgy7XifcPuXWVT6mDAomwMfYIrZrJsXh\nY9K7awHvbVZKTErJ/35KZ+2hIlJzKtzbBLcz1pHO+Kcmhhz3d3pUTACHCpomxgVa2opkFxMGBjEz\nOZS75ibh760J/zEDAtiWXsq2Y6V4G/UMjfTjlpmDWHOwkHP/vZZ/fnuAXVnl3D03iRevGe9+rGBn\nlYoam1ZTemS0P/cuGEx6cQ1Lt2e7/cgLh4djd0h8zHoeXDzEnTVfc6gQm93RRkj+en4y+/+6iO2/\nn89Ht03hfzdMJDbYh9mDQ93l3VzvvzbBtOn1DnGetLiWJTqrs4weEMC8oeG87+xAmllS20YkA23K\n346LDeSs+CDe2njMvexYcbW7eoiLJWOiyC6r5Z1NGYyPC8TXS8u0N9+mOa6TniGRvsedFOuawNnd\nvuS+8c3eaINXZkPRIa16xZzfg/HkMjsr9uZz9/s78DEbeOW6Cfh6GU+8k0LR19DpYPwNMPhc+P4P\nsPZJSPkYFj8ByQs8HZ1C4TGEEMSHWtidVd5GbHUFQyP9SM2pYHCELwuGRXDR2BgCLSZ39q6wqp6K\nWu3Stq+XkUcuGune18/LiEEnaHRIt4A06nXEh1g4mK95bi0mPdU2O+F+ZsyGJvGRGGblcEEVaYVV\n6ERTltnFZ3dMO27cZyeFsmx3bgvBCfCbBcncMHUgQjSJWH8fI8//bBxmg57b39nGvtwmm9JuZ6tu\ngC935xAXpIn19k5IIv292ZNdwYxmEwnbIzncikPCTmf2MqhVJjkxrEmoxQX78ObPz2qx/rIJMXyw\nNZOPtmUxKsYfvU5w7eQ4XlmTht0h+fyOaYwe0DYjGmI1k+mcHCclDI/yY+7QMEYPCOCZlYf4zQLt\nKvW5o6JYlpLLH84bRojVjJQSk17Ht6n5mAw6Jgxs28dBCIEQmiXIxf9ubIo7yGLCZNBha3Qwf1g4\nX+7Kodpmb5ZJNjnHpum1Xzslju/25vPZjmxyymvbHAMdsWBYOH9bto+CijrC/LzY6Gxi4soGg+ZF\nNxtSqG2wc9n4GHzMBvLKa1uUMWyOa99RMe1nml1EBXgzIymkw8fpKvqGSDaYYNJtmq0ibupJ7Sql\n5MXVafzj2/0Mj/LjlesmEOmvLp0q+jnWULjoRRjzM1h2D7x7GQw5D2Y9ABEjT7y/QtEHiQ/RRHJ7\n2c3TZXiUHx9v0zJp3ia9u4mGWyQ3zyR7t/zp1ukEQRYTBZX17u6AoDU62XCkmKr6Ri4aG82nO7Ld\n9gcXiWFWVuwrYHtGGQOCfFoI6M5w0bhoApwZ5+ZYzAa3/aA5swZr2e9hUX7sy6sgp6yW7/fmc6Sw\nCoMzs/nVrlyunKhVrGhPJEc7LQXTTiCSXXYRV8mxAJ+WyS9X9t5i0rf7no6PC3RbNlwZTovZwGd3\nTMPLqG9hP2hOiNXMjoxSt01jWJQfQghumh7PXe/tYMU+baLc0Ehftjw0z30ioU3CNJFbXsf9iwa7\nryKcDC5f8rHiGpLDfRk9IIADeZXuWNsTydMSQogPsXD/JykAjIg6fhdiF+PiAgHNQrFweDhvbkhn\naKRfCwuM1Wxg3rBwvk7JZd6w8BO+pugAb26ZEc/5o6OOux3AWzdN6lScp0PfEMkAE2866V3qGuw8\nuDSFT3dkc96oSP556Wi3WV2hUADxM+C2n2D9v2Hd07D/K0haCDN+A7Hd/wWlUPQmXKLrVCfuHY8r\nJg4g3M+LpFaXoV3ipqiq3l2b1q+dK53BVjMFlfVEOj3JoGVKlzu9oAuHh7MsJbeNfz4xzIrdIVl3\nuIi75yaddNxGvY4FwztXCq85QyP8eGvjMR5Zvo9lzgmFZyeHcsHoKH7z0S6+Sc3DYtK3e8n9sgkx\nhPmZT5jxdPnId2Q6M8mthHCYrxlfLwMxgT4tMt4uhBD8fFo8v/pgZwt/7ImeN8RqoqTaxp7scvy8\nDG5RP9XZzGPFvgKsZkO7V6wTQq0MDLZw0/RBx32O4+ESyQlhViYODKKwqqkMW5NIbsr863SCB84Z\nwhc7c1gyJor5zsmAJ2J4lB8mvY4dmaUE+hjZn1fJ45eMbDOWDy0eyqXjYjol+oUQPOT0WfcG+o5I\nPkkKKuq49a1t7Mws4zfzk7lzTmK7HxKFot9jMMHMe7UT0c2vwsbn4bUFEDcdZtwDCXNAfXYU/YDz\nR0dSWFXvnhzVlfiYDCwe2baaUqCPCZ3QMskOKdHrBD7tJHNc3l9X9Qdo6blNCLXy9BVjWkyw07bR\nRPmMpBDuOgWRfKoMjfSjvtHBst25nJ0cSl2DnZtnxDMuNpBnfzhEak5FhxNih0f5d6o0q8VsIMxX\nO3kw6XVtxk0IwazBYYR1kBEGOG9UJBV1DSwZc+LMposQqxmHhO/25jMqJsCtLUKsZgaH+3Igv7Jd\n3y/AazdMRAhOy0bgKpOXGGZ1CvSmsRobG8DgcF/Gxga22Gfh8AgWnuTJjtmgZ1iUHzuOlXGkoBp/\nbyMXjI5uG0+Ad7d8ZnqCfieS7Q7J5zuzefyb/VTWNfLiNeM7XRBeoejXeAfC2ffBlNth2xtaE5K3\nL4bwEVoZuZGXad38FIo+SmKYL3+/qGftRnqdVge5sLIek0HrKNdeQsdlF4hsLpKbZQsHBPm0yVKD\nlg188rLRzBsW3u3+zuY0z8zev2hICx/ru7dM5vKXNrSxhpwKA0MsFFTWE2gxtjturjq9HWHQ67hu\nysCTek7XBLySahu3z0posW5KQjAH8itbvE/NMRlOv57C8Gh/Nh0taXFVwcWQCD++/fXM034OF2Nj\nA3hjfToOCfctHNznrsb3neoWJ0BKyXepeSx+Zi33fLiLEKuZj2+bqgSyQnGymCyaUL57J1zwLOhN\nWmvrp4Zq1WV2faBVm1EoFF1CqFMkV9Q2dDipPDLAG6vZ0MJSEB9iQScg3M/cYaUAIQSXjI9xV3To\nKRLDrBj1Wpm2Ya08sFEB3nx99wxeuGbcaT/PoJDus8h0hMtWsHhkBFNb+aanOC0X4e0I2K7ixqkD\n+fHeWT1SoWtcbCAOqVVFuWXGqVtEeiv9IpO8/kgR//z2ADsyyhgUYuG5q8eyeESkKvGmUJwOBjOM\nu067FR2C3R9ot09vBaNFa+CTMBsGzYageE9Hq1CcsYT6mimqqkfSdtKei1/MHMQFo6NaZEvNBj0D\ngy1tGnL0BkwGHX86fzhDItovBdZVFaYGekAkjxkQwA1TB/LLVllkgMnxwRh0okO7RVeg0wlMPaRv\npiYEMyzSj78sGd4lWfDeRp8VycVV9Xy/N5/PdmazMa2ESH8vHrt4JJeOj8Gg73tvpELhUUKStNKL\ns34HmZs0sXzoO9j3hbY+cKAmlhNmw4DJ4Nu5iSEKhULLTB7Mr6S42tZh84wAHxMB7QjBvywZjvdx\n6s16kmsmx3X7c7gmW3Z1bevj4WXU8+cLhre7zt/HyGd3TCMuuPtEck8SbDWz/O4Zng6j2+hTIjmv\nvI5vU/P4ek8um4+W4JAwIMibhxYP5dopccctTK1QKLoAnQ7ipmg3KbUMc9qPcORHSPkItv1P284a\nAVFjIHI0RDr/+kWpCYAKRTuE+prdDShuO7ttdvJ4uJou9FdcExVbl3/zJK6OioreT6dEshBiEfAM\noAdelVI+1mq9cK5fDNQAN0gpt3dm365ASsnVr2xiQ1oxoHmd7pidyKIREQyL9FNVKxQKTyAEhCZr\nt0m/AHsDZG/Tbrm7tNuh70A6tO1NVs2WETSo6RYYr4ln3wjNC61Q9EOa1+OdNbh/i96TJTbIB5NB\n527XrVCcDCcUyUIIPfAfYD6QBWwRQnwhpdzbbLNzgCTnbRLwAjCpk/ueNkIIRsb4My0xmEUjIlp0\n0VEoFL0EvRFiJ2s3F7ZqyNsDebuh+DCUpEF+KuxfBo7GlvubfDWx7BsB1nCt2oZ3oNZO23Xfy18T\n00aL9td106mrSIozF1d5t8QwKzFdUPGhP+Fl1LP0l1OJ7SP2BkXP0plM8lnAYSllGoAQ4n1gCdBc\n6C4B3pRSSmCjECJACBEJDOzEvl3C7xYP7eqHVCgU3Y3JojUlad2YxN4I5ZlQmg6VeVCZC1X52t/K\nfC0bXVsKdeWAPPHz6M1g8NImGzb/qzdq1Tn0RtAZmt3Xg9A3+2vQrCRCp/0vdC1v03+tdSlUKLoB\nVyZ5tsoinxLK3qA4VTojkqOBzGb/Z6Fli0+0TXQn9wVACHErcCtAbGxsJ8JSKBR9Fr3Bab04QVUM\nhwPqy6G2rEk0N9RoGWpblfa3vgoaa6GxHhrrmv421IGjAew2TZQ31jXdl3Ytk+2wO+87tP+lo9VN\nan8n3qREsqLbGBLhR1KYlQvHtm3UoFAouo9eM3FPSvky8DLAhAkTOpEaUigU/R6drslqgSozp+ib\nBFlMfH/P2Z4OQ6Hod3SmFlo2MKDZ/zHOZZ3ZpjP7KhQKheIkEUIsEkIcEEIcFkI80ME2s4QQO4UQ\nqUKI1T0do0KhUJzJdEYkbwGShBDxQggTcCXwRattvgCuExqTgXIpZW4n91UoFArFSdBsUvQ5wDDg\nKiHEsFbbBADPAxdIKYcDl/V4oAqFQnEGc0K7hZSyUQhxJ/AtWhm316SUqUKI25zrXwSWo5V/O4xW\nAu7G4+3bLa9EoVAo+g+dmVB9NbBUSpkBIKUs6PEoFQqF4gymU55kKeVyNCHcfNmLze5L4I7O7qtQ\nKBSK06Izk6KTAaMQYhXgCzwjpXyz9QOpSdMKhULRPqo/s0KhUPRNDMB44FxgIfAHIURy642klC9L\nKSdIKSeEhqoKHQqFQuGi11S3UCgUCkWn6cyk6CygWEpZDVQLIdYAo4GDPROiQqFQnNmoTLJCoVCc\neXRmUvTnwHQhhEEI4YNmx9jXw3EqFArFGYvKJCsUCsUZRmcmVEsp9wkhvgF2Aw7gVSnlHs9FrVAo\nFGcWSiQrFArFGciJJlQ7//8n8M+ejEuhUCj6CkIrTNG7EEIUAsc8HUcnCQGKPB1EN6NeY9+gP7xG\nOLNeZ5yUstfMljuN797eNuYqnuOj4jk+Kp7j0xfi6dR3b68UyWcSQoitUsoJno6jO1GvsW/QH14j\n9J/X2ZvobWOu4jk+Kp7jo+I5Pv0pHjVxT6FQKBQKhUKhaIUSyQqFQqFQKBQKRSuUSD59XvZ0AD2A\neo19g/7wGqH/vM7eRG8bcxXP8VHxHB8Vz/HpN/EoT7JCoVAoFAqFQtEKlUlWKBQKhUKhUChaoURy\nFyGE+I0QQgohQjwdS3cghPinEGK/EGK3EOJTIUSAp2PqKoQQi4QQB4QQh4UQD3g6nq5GCDFACPGj\nEGKvECJVCHG3p2PqLoQQeiHEDiHEV56Opb/g6c9PR8e3EOLPQohsIcRO521xD8aULoRIcT7vVuey\nICHE90KIQ86/gT0Uy+BmY7BTCFEhhPhVT46PEOI1IUSBEGJPs2UdjocQ4kHn8XRACLGwh+Jp9zdO\nCDFQCFHbbJxe7PiRuzSeDt8fD43PB81iSRdC7HQu79bxOc7nu2eOHymlup3mDRiA1vnqGBDi6Xi6\n6TUuAAzO+48Dj3s6pi56XXrgCDAIMAG7gGGejquLX2MkMM553xc42NdeY7PXeg/wLvCVp2PpD7fe\n8Pnp6PgG/gzc66FxSW/9WwD8A3jAef8BT3yHOt+vPCCuJ8cHmAmMA/acaDyc790uwAzEO48vfQ/E\n0+5vHDCw+XY9OD7tvj+eGp9W658E/tgT43Ocz3ePHD8qk9w1/Av4LdBnDd5Syu+klI3OfzcCMZ6M\npws5CzgspUyTUtqA94ElHo6pS5FS5koptzvvVwL7gGjPRtX1CCFigHOBVz0dSz/C45+fM+j4XgK8\n4bz/BnChB2KYCxyRUvZosy4p5RqgpNXijsZjCfC+lLJeSnkUOIx2nHVrPJ78jetgfDrCI+PjQggh\ngMuB97ryOY8TS0ef7x45fpRIPk2EEEuAbCnlLk/H0oP8HPja00F0EdFAZrP/s+idP7BdghBiIDAW\n2OTZSLqFp9FOVh2eDqQf0as+P+0c3//nvHz+Wk/ZG5xIYIUQYpsQ4lbnsnApZa7zfh4Q3oPxuLiS\nluLGU+MDHY9HbzimWv/GxTutBKuFEDN6MI723h9Pj88MIF9KeajZsh4Zn1af7x45fpRI7gRCiBVC\niD3t3JYAvwP+6OkYu4ITvE7XNg8BjcA7notUcSoIIazAJ8CvpJQVno6nKxFCnAcUSCm3eToWhWdo\n5/h+Ac0GMgbIRbtE3FNMl1KOAc4B7hBCzGy+UmrXhXv0yqMQwgRcAHzkXOTJ8WmBJ8ajI9r5jcsF\nYp3v5z3Au0IIvx4Ipde8P624ipYnWj0yPsf7/erO48fQHQ/a15BSzmtvuRBiJJrnZZd2BYIYYLsQ\n4iwpZV4PhtgldPQ6XQghbgDOA+Y6D8q+QDaap9xFjHNZn0IIYUT7gnlHSrnU0/F0A9OAC5yTW7wA\nPyHE21LKazwcV1+nV3x+2ju+pZT5zda/AvTYZE4pZbbzb4EQ4lO0y735QohIKWWuECISKOipeJyc\nA2x3jYsnx8dJR+PhsWOqvd84KWU9UO+8v00IcQRIBrZ2ZyzHeX88OT4G4GJgvGtZT4xPB79fPXL8\nqEzyaSClTJFShkkpB0opB6Kl9cediQL5RAghFqFdyr5ASlnj6Xi6kC1AkhAi3plpuRJTalhoAAAB\nz0lEQVT4wsMxdSlOD9l/gX1Syqc8HU93IKV8UEoZ4/wcXgn8oARyj+Dxz09Hx7fzh9PFRcCe1vt2\nUzwWIYSv6z7ahLA9aONyvXOz64HPeyKeZrTIAHpqfJrR0Xh8AVwphDALIeKBJGBzdwfT0W+cECJU\nCKF33h/kjCetB+Lp6P3xyPg4mQfsl1JmNYuzW8fnOL9fPXL8qEyyorM8hzZb9Htn1nyjlPI2z4Z0\n+kgpG4UQd6JVJ9EDr0kpUz0cVlczDbgWSBHOsj3A76SUyz0Yk6IP0Es+P+0e38BVQogxaJdh04Ff\n9FA84cCnzu9JA/CulPIbIcQW4EMhxE1olZAu76F4XGJ9Pi3H4B89NT5CiPeAWUCIECIL+BPwGO2M\nh5QyVQjxIbAXzfZwh5TS3gPxPEj7v3EzgYeFEA1o8x1uk1J2dpLd6cQzq733x1PjI6X8L2097dD9\n49PR57tHjh/VcU+hUCgUCoVCoWiFslsoFAqFQqFQKBStUCJZoVAoFAqFQqFohRLJCoVCoVAoFApF\nK5RIVigUCoVCoVAoWqFEskKhUCgUCoVC0QolkhUKhUKhUCgUilYokaxQKBQKhUKhULRCiWSFQqFQ\nKBQKhaIV/w9vIF2q5bdKLgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# let's do sgd\n", "# intentionally bad initialization\n", "mu = -2\n", "sigma = 3\n", "T = 200\n", "\n", "eta = 0.05 # step size\n", "objs = [] # keep a record\n", "for t in range(T):\n", " grad, obj = est_grad(mu, sigma)\n", " if t % 20 == 0:\n", " print('t =', t, 'D(q||p) =', obj)\n", " objs.append(obj)\n", " mu, sigma = [mu, sigma] - eta * grad\n", "\n", "# plot results\n", "xs = np.linspace(-5, 5)\n", "plt.figure(figsize=(10, 4))\n", "plt.subplot(1, 2, 1)\n", "\n", "plt.plot(xs, p_x(xs), label='$p(x)$') # target p(x)\n", "plt.plot(xs, norm.pdf(xs, mu, sigma), label='$q(x)$') # I-projection q(x)\n", "plt.legend(loc='best')\n", "plt.title('Approximate I-Projection')\n", "\n", "plt.subplot(1, 2, 2)\n", "plt.plot(objs, label='$D(q||p)$')\n", "plt.legend(loc='best')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Another approach to finding $ \\nabla_\\phi \\mathbb{E}_{q_\\phi} [\\log p(X)]$ is based on the observation that\n", "$$X \\sim q_\\phi(X) = \\mathcal{N}(\\mu, \\sigma^2)$$\n", "is equivalent to\n", "$$X = g_\\phi(\\epsilon) = \\mu + \\sigma \\epsilon, \\text{where } \\epsilon \\sim p(\\epsilon) = \\mathcal{N}(0, 1)$$\n", "i.e., the process of generating $X$ from $q_\\phi(X)$ is equivalent to first generating an underlying source of randomness $\\epsilon \\sim \\mathcal{N}(0, 1)$, then transforming $\\epsilon$ by a deterministic function $g$ governed by the parameters $\\phi$. Now it makes sense to take derivative of the value of a random sample $x=g_\\phi(\\epsilon)$ w.r.t. parameters $\\phi$: $\\nabla_\\phi x = \\nabla_\\phi g_\\phi(\\epsilon)$ measures the rate of change in $x$ from adjusting $\\phi$, if we repeated the sampling process using the same noise source $\\epsilon$. Formally, we make the following change of variable in the expectation $\\mathbb{E}_{q_\\phi} [\\log p(X)]$ (a rigorous justification of this step can be found [here](https://math.stackexchange.com/a/4007827)) :\n", "$$ \\mathbb{E}_{X} [f(X)] = \\int_x f(x) q_\\phi(x) dx = \\int_\\epsilon f(g_\\phi(\\epsilon)) p(\\epsilon) d \\epsilon = \\mathbb{E}_{\\epsilon} [f(g(\\epsilon))] $$\n", "where $f(X) = \log p(X)$. Note crucially that $\\phi$ and $\\epsilon$ have no functional dependence on each other. We can finally differentiate w.r.t. $\\phi$ as usual:\n", "$$ \\nabla_\\phi \\mathbb{E}_{q_\\phi} [\\log p(X)] = \\int_x f(x) \\nabla_\\phi q_\\phi(x) dx = \\int_\\epsilon \\nabla_\\phi f(x) p(\\epsilon) d \\epsilon = \\int_\\epsilon \\nabla_\\phi f(g_\\phi(\\epsilon)) p(\\epsilon) d \\epsilon = \\int_\\epsilon f'(g_\\phi(\\epsilon)) [\\nabla_\\phi g_\\phi(\\epsilon)] p(\\epsilon) d \\epsilon $$\n", "\n", "which now has the form of an expectation w.r.t. $p(\\epsilon)$. This technique is called the Reparameterization Trick, popularized by the [Variational Autoencoder paper](https://arxiv.org/abs/1312.6114)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After some algebra, in our example we have\n", "\\begin{align}\n", "f'(x) &= \\frac{d}{d x} \\log p(x) = -\\frac{1}{p(x)} \\{ \\pi_1 \\mathcal{N}(x; \\mu_1, \\sigma_1) \\frac{x-\\mu_1}{\\sigma_1^2} + \\pi_2 \\mathcal{N}(x; \\mu_2, \\sigma_2) \\frac{x-\\mu_2}{\\sigma_2^2} \\} \\\\\n", "\\nabla_\\phi g_\\phi(\\epsilon) &= [1, \\epsilon]\n", "\\end{align}" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "t = 0 D(q||p) = 1.79964369771\n", "t = 20 D(q||p) = 1.05578290301\n", "t = 40 D(q||p) = 0.733509820609\n", "t = 60 D(q||p) = 0.709207215267\n", "t = 80 D(q||p) = 0.674809679004\n", "t = 100 D(q||p) = 0.699469241391\n", "t = 120 D(q||p) = 0.698766894145\n", "t = 140 D(q||p) = 0.69433893559\n", "t = 160 D(q||p) = 0.649860176551\n", "t = 180 D(q||p) = 0.714982255485\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAskAAAEYCAYAAACuv2v6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4nNWV+PHv0RSNqiVLsuUuN2y54N4dg8F0JwY2JCEE\nQyiGQMim7cL+loSQbDZkQ8lmIaFDCD2ElgCmBIxx78K9N7mp2pLVNXN/f7wjeSyrWjPzzozO53n8\noJm3nRnjV2funHuuGGNQSimllFJKnRJndwBKKaWUUkpFGk2SlVJKKaWUakKTZKWUUkoppZrQJFkp\npZRSSqkmNElWSimllFKqCU2SlVJKKaWUakKTZBVRROSkiAyyOw47iUh///vgiIbzKqWUUrFIk+Qu\nREQWiUipiMTbHUtLjDHJxpg9wT6viNwoIktCdbyI/EJE6vxJ6HERWSYi087mWsaYA/73wXu28fpj\n2icic4J9XqWUUqor0CS5ixCRHOArgAG+FsLrOEN17ijwmjEmGcgClgBviog03amLv0dKKaVUVNAk\nueuYD6wAngduCNwgIs+LyOMi8rGIlIvI5yIyIGC7EZEfiMgeESkSkd+JSJx/240islREHhGRYuAX\nIhInIveKyH4RKRCRF0Skm3//b4rIXhFJ9T++TESOikhWwLWGBMT1RxH5wD9Cu1REskXk9/4R8W0i\nMi4gzntEZLf/NWwRkav8z+cCjwPTGkZ6/c/Hi8iDInJARI7534OEzr7Rxpg64M9ANpBxFu9Rjv99\ncPofdxORZ0TkiIgcEpH/CiyZEJFbRWRrwOseLyJ/AfoDf/e/5n9v5ry9ReRdESkRkV0icmvAOX8h\nIq/74yoXkc0iMrGz741SSikVLTRJ7jrmAy/5/1wiIj2bbL8O+BWQCWzw7xfoKmAiMB6YB9wUsG0K\nsAfoCfwauNH/ZzYwCEgGHgUwxrwGLAP+ICIZwDPALcaYwhbi/gZwrz+uGmA5sM7/+A3g4YB9d2ON\nlncD7gdeFJFexpitwO3Acn+5QZp//weAc4CxwBCgD/DzFuJoN385y43AQWNMkf/pdr9HzXgeqPfH\nOA64GLjFf61rgF9g/f2mYn1LUGyMuR44AHzV/5r/p5nzvgrkA72BrwP/LSIXBGz/mn+fNODdVuJT\nSimlYo4myV2AiMwEBgCvG2PWYiWT326y23vGmMXGmBrgP7FGXfsFbP+tMabEGHMA+D1wbcC2w8aY\n/zPG1BtjqrAS7oeNMXuMMSeB/wC+FVBmcCdwAbAI+Lsx5h+thP+WMWatMaYaeAuoNsa84K+rfQ0r\naQTAGPNXY8xhY4zPn4zvBCa38J4IsAD4kf91lQP/DXyrlVja8g3/KPVBYALWB4sGHX2PGuLsCVwO\n/NAYU2GMKQAeCYjzFuB/jDGrjWWXMWZ/W4H6/25nAHcbY6qNMRuAp7GS7QZLjDHv+9/rvwBjOvyO\nKKWUUlFKk+Su4Qbgo4BRzZdpUnKBldgB4E/aSrBGGM/YDuxvZRv+bYGJ2n7AiTWKijHmOPBXYBTw\nUBuxHwv4uaqZx8kND0Rkvohs8E+cO+4/f2YL580CEoG1Afsv9D/fKhG5zl/CcFJEPgjY9LoxJs0Y\n08MYc4H/A0mDDr1HAQYALuBIQJxPAD382/thfejpqN5Aw4eDwBj6BDw+GvBzJeDRemqllFJdhf7C\ni3H+GttvAA4RaUh64oE0ERljjMnzP9cv4JhkoDtwOOBU/YDN/p/7N9lmmlz2MFZy16A/VrnAMf/5\nx2KVa7wC/AG49KxeXAB/DfVTwIVYZRVeEdkANEycaxpjEVaSPdIYc6gj1zLGNJStdOiwJo9be4/6\nBjx/EKvMJNMYU9/MeQ8Cg9t5zabX7y4iKQGJcn+gQ++FUkopFat0JDn2XQl4gRFYtbdjgVzgC07/\nav1yEZkpIm6s2uQVxpjA0c9/E5F0/9f0/4pV6tCSV4AfichAf8L931idH+pFxAO8CPw/4LtAHxG5\nIwivMwkrKSwEEJHvYo0kNzgG9PW/PowxPqyk+hER6eE/po+IXBKEWNqjxfcocCdjzBHgI+AhEUn1\nT/gbLCLn+Xd5GvipiEwQyxA5NenyGFa98xn8f7fLgN+IiEdEzgVuxvq7UUoppbo8TZJj3w3Ac/4e\nuUcb/mBNwrou4Ovzl4H7sMosJgDfaXKed4C1WJP63sOacNeSZ7FqWBcDe4Fq4C7/tt9gTWj7k7/+\n+TvAf4nI0M68SGPMFqzSjeVYyeFoYGnALp9ijYQfFZGGspO7gV3AChEpAz4BhnUmjg5o7T1qaj7g\nBrYApVgTFnuBVYeNNRHwZaAceBvrWwCw3ut7/WUaP23mvNcCOVijym8B9xljPunsC1NKKaVigRjT\n2jeyqisQkeeBfGPMvS1sN8BQY8yusAbWRYm14uAOwGX0H6hSSillCx1JViryjAL2a4KslFJK2UeT\nZKUiiIj8GHgSuMfuWJRSSqmurF3lFiJyKfC/gAN42hjzQJPt87Ame/mwZuj/0BizpD3HKqWUUkop\nFWnaTJL9y9/uAC7CWp1rNXCtf6JUwz7JQIUxxvhnyb9ujBnenmOVUkoppZSKNO0pt5gM7PKvDFaL\ntUztvMAdjDEnA+onG1pxtetYpZRSSimlIk17FhPpw+mrheUDU5ruJCJXYbWc6gFc0ZFjm8rMzDQ5\nOTntCE0ppaLX2rVri4wxba7yGC5671VKdQXtvfcGbcU9Y8xbwFsiMgurPnlOR44XkQXAAoD+/fuz\nZs2aYIWmlFIRSUT2t71X+OTk5Oi9VykV89p7721PucUhApYsxloyt8Wla40xi4FBIpLZkWONMU8a\nYyYaYyZmZUXMwIpSSimllOqC2pMkrwaG+pfPdQPfAt4N3MG/FK74fx4PxAPF7TlWKaWUUkqpSNNm\nuYUxpl5Evg98iNXG7VljzGYRud2//XHgX4D5IlIHVAHf9E/ka/bYEL0WpZRSSimlgqJdNcnGmPeB\n95s893jAz78FftveY5VSqkFdXR35+flUV1fbHUrIeDwe+vbti8vlsjsUpVSM6Qr30LPV2Xtv0Cbu\nKaXU2cjPzyclJYWcnBz8VVsxxRhDcXEx+fn5DBw40O5wlFIxJtbvoWcrGPdeXZZaKWWr6upqMjIy\nYvbmLiJkZGQEdZRHRJ4VkQIR2dTC9m4i8ncRyRORzSLy3aBdXCkVUWL9Hnq2gnHv1SRZKWW7WL+5\nh+D1PQ9c2sr2O4EtxpgxwPnAQ/7J00qpGBTr99Cz1dn3RZNk1aY9hSd56KPtHCiutDsUpRSNrTZL\nWtsFSPF3HUr271sfiliW7CziP978kjqvLxSnV0op22iSrFq0q+AkP3x1PXMe/pz/+3QX//L4MrYe\nKbM7LKVU2x4FcoHDwEbgX40xzWaxIrJARNaIyJrCwsIOX2hnQTmvrDpIRU1IcnCllLKNJsnqDDuP\nlfODV9Zz0SOf8+HmY9zylUG8umAqzjjhG08sZ9Xe1gawlIotVVVVnHfeeXi93hb3qa2tZdasWdTX\nR0yieAmwAegNjAUeFZHU5nbs7EJOyfHW/O/y6oh57UopGzzxxBP06tWLsWPHMmbMGK655hr27t0L\ntO8+CnD++eezb9++Fh83J5T3X02SVSOfz/Dvb+Rx8e8X88nWY9w2azBf3D2b/3d5LlMHZfDG96aT\nlRLP9c+s5JMtx+wOV6mwePbZZ7n66qtxOBwt7uN2u7nwwgt57bXXwhhZq74LvGksu4C9wPBQXCjF\nYyXJJ3UkWakubePGjfzyl79kw4YN5OXlceGFF3L11VdjjGnXffRshfL+q0myavTBpqO8viaf66cO\nYMndF3DPZcPJTI5v3N4nLYG/3jaNYdkp3PbiWv62Nt/GaJUKrmuvvZZvfvObTJ48mQEDBvDee+8B\n8NJLLzFv3rzG/WbPns3HH38MwL333stdd90FwJVXXslLL70U/sCbdwC4EEBEegLDgD2huFByvNV/\nVJNkpbq2L7/8klGjRjU+vv322zl69CgHDx484z66c+dOzj//fEaNGsVPfvITBg8e3Ob5W7pHQ+ju\nv9onWQFQ5/Xx4EfbOadnMvd9dSSOuOZnhGYkx/PyrVO57S9r+Mlf8yitrOWWrwwKc7QqVt3/981s\nORzcuvcRvVO576sj29wvLy+PefPm8dprr7FkyRJ+/OMfc9FFF7Fnzx5ycnJOxXj//fz85z+noKCA\n9evX8+677wIwatQoVq9eHdTYWyIir2B1rcgUkXzgPsAFjQs9/Qp4XkQ2AgLcbYwpCkUsyQ0jyVpu\noZTt7LyHbtq0iZEjT98vISGB0tLS0+6jXq+X+fPn89hjjzF+/HjuuuuuM45rTnP36CuuuAII3f1X\nk2QFwF/X5LO3qIKn5k9sMUFukBzv5NkbJ/Gj1zbwX+9tJbdXKjOGZIYpUqWCr7q6msLCQu677z4A\nRowYQWlpKUVFRaSlpZ2276xZszDG8PDDD7No0aLGrw8dDgdut5vy8nJSUlJCGq8x5to2th8GLg5p\nEH6NNck6kqxUl3Xw4EGSk5NJTT019aGuro4jR46QlpZ22n307bffZsSIEYwfPx6A3NzcM+6zTbV0\nj24QqvuvJsmKqlovv/9kBxMHpDMnt0e7jol3Onj4G2NZtfczHv98tybJKijaM1oRCps2bWLo0KF4\nPB4A1q1bx5gxY0hISDijEf3GjRs5cuQIGRkZZ9yMa2pqGs/RVaToSLJSEcOue+jGjRtPK7UAeO65\n57jgggtITU097T66fv16xo4d2/g4Ly+POXPmtHr+lu7RgUJx/9WaZMVzy/ZSUF7D3ZcN71DjbY/L\nwXdn5PDFziJtDaeiWl5eHgcOHKC6upqKigruu+8+fvSjH5Geno7X6228wR85coTrrruOd955h+Tk\nZBYuXNh4juLiYjIzM3G5XHa9DFs0jCSfrKnj2SV7ufKxpTZHpJQKt6b1yB999BG/+c1vePDBB8+4\nj2ZkZLBt2zYAVq5cyQsvvHBGwttUS/foBqG6/2qS3MWdqKzj8UW7uWB4DybldO/w8d+ZMoBEt4On\nFodkTpBSYZGXl8fVV1/NlClTmDRpEt/73veYMWMGABdffDFLliyhsrKSq6++moceeojc3Fx+9rOf\ncf/99zee47PPPmusj+tKEt0ORKyR5E2HTugHZqW6oI0bN/LSSy8xYcIExo8fz5///GcWLlxIbm4u\ncOo+CnD99dezZs0aRo8ezZtvvklGRgZDhgxp9fyt3aMhdPdfLbfo4v74+S7Ka+r5t0uGndXx3RJd\nfHNSP/6yfD8/vWQYvdMSghyhUqGXl5fHk08+yR/+8Icztt1555088sgjzJkzh+XLlzc+P2vWrNMe\nv/zyyzzwwANhiTeSiAjJ8U7Ka+opraylpt6Hz2eIa2Nug1IqdrTVWSLwPpqZmcnKlSsBq5Z50aJF\nxMW1Pmbb2j0aQnf/1ZHkLuzIiSqeX7qPK8f2IbdXs+sMtMtNMwZigOeW7g1ecEqF0e7duxk6dGiz\n28aPH8/s2bPbXEzkyiuv5JxzzglViBEtJd7Jyep6SirrAKip1yWqlVKntHQfzcvL49xzz23z+Nbu\n0aG8/2qS3IX94Z878RnDjy/q3P9Y/boncvnoXryy6iBl1XVBik6p8MnPz291JOOmm25qczGR+fPn\nhyK0qJDscXKypp7jlbUAVNW1vqqWUqrrae4+OnfuXJ566qnGxzfeeONpnS4aHrd2jw7l/VeT5C5q\nd+FJXl+Tz3VTBtCve2Knz3fbrEGcrKnn5ZUHghCdUiqaJMdbSXJJhSbJSqmz11KSbBdNkruohz/e\ngccZx/cvaL1Yvr1G9enG9MEZPLd0L7X6VatSXUqyx0VpZS3l/jZwVbWaJCulop8myV3Qico6Ptx0\nlGsn9z9t2enOWjBrEMfKang373DQzqmUinwp8U7yS6saH1frSLJSKgZoktwFfbL1GPU+w9wxvYN6\n3vPOyWJ4dgpPLd6DMSao51ZKRa6keAfHK0/NR9ByC6XCS3/nNq+z74smyV3QB5uO0LubhzF9uwX1\nvCLCrV8ZxPZj5SzaURjUcyulIldy/OkN/LXcQqnw8Xg8FBcXa6LchDGG4uLiTq3Cp32Su5jy6joW\n7yji+mkDOrS6Xnt9dUxvfvfhdp78fA+zh7VviWulVHRL9pz+q0TLLZQKn759+5Kfn09hoQ5ONeXx\neOjbt+9ZH69Jchfz6bYCar0+Lh+dHZLzu51x3DA9h98u3Ma+ogpyMpNCch2lVORIiT/9V4mWWygV\nPi6Xi4EDB9odRkzScosu5v2NR+iZGs+4fukhu8bcc3sB8NGWoyG7hlIqcuhIslIqFmmS3IVU1NSz\naHshl47MDumSsf26JzKiVyofbj4WsmsoFS5VVVWcd955ba64N2vWLOrr68MYWeRIbjqSrDXJSqkY\noElyF/LZ9gJq6n1cNrpXyK91ychs1h0opaC8OuTXUiqUnn32Wa6++uo2V9y78MILee2118IYWeRo\nGEnunuQGoKpOe6UrpaKfJsldyAcbj5KZ7GZSTveQX+vikT0xBj7ZUhDyaykVDDt37uT8889n1KhR\n/OQnP2Hw4MEAvPTSS8ybN69xv9mzZ/Pxxx8DcO+993LXXXcBcOWVV/LSSy+FP/AI0FCT3DPVmkWu\nNclKqVigE/e6iKpaL59uK+Dq8X1whLDUosHw7BT6d0/kw81H+faU/iG/nooRH9wDRzcG95zZo+Gy\nB1rdxev1Mn/+fB577DHGjx/PXXfdxciRI6mtrWXPnj3k5OQ07nv//ffz85//nIKCAtavX8+7774L\nwKhRo1i9enVwY48SDSPJGUluPK44rUlWSsWEdo0ki8ilIrJdRHaJyD3NbL9ORL4UkY0iskxExgRs\n2+d/foOIrAlm8Kr9Pt9RSFWdl8vDUGoBVs/kS0b2ZNnuIsqq69o+QCkbvf3224wYMYLx48cDkJub\ny5gxYygqKiItLe20fWfNmoUxhocffphXX321sQzD4XDgdrspLy8Pe/x2a6hJTk9yk+ByaJKslIoJ\nbY4ki4gDeAy4CMgHVovIu8aYLQG77QXOM8aUishlwJPAlIDts40xRUGMW3XQB5uOkJ7oYsrA0Jda\nNLhkZDZPfbGXRdsL+VqQV/dTMaqNEd9QWb9+PWPHjm18nJeXx5w5c0hISKC6+vS6+o0bN3LkyBEy\nMjJISUk5bVtNTU2nGtdHqxT/YiLpiS4SXA6duKeUigntGUmeDOwyxuwxxtQCrwLzAncwxiwzxpT6\nH64Azr5zswq66jov/9xawCUjs3E6wleGPq5/OpnJbj7crK3gVGTLyMhg27ZtAKxcuZIXXniBMWPG\nkJ6ejtfrbUyUjxw5wnXXXcc777xDcnIyCxcubDxHcXExmZmZuFyuZq8Ry5LiHSS6HfROS8DjdmhN\nslIqJrQnY+oDHAx4nO9/riU3Ax8EPDbAJyKyVkQWdDxE1VlLdhZxsqY+LF0tAjnihItG9GTRtgL9\n+lVFtOuvv541a9YwevRo3nzzTTIyMhgyZAgAF198MUuWLKGyspKrr76ahx56iNzcXH72s59x//33\nN57js88+44orrrDrJdjK6Yjj73fN5IZpOXicWm6hlIoNQZ24JyKzsZLkmQFPzzTGHBKRHsDHIrLN\nGLO4mWMXAAsA+vfXiV7B9P6mI3RLcDF9cEbYr33xyGxeWXWQ5buLmd0vDop2gicVPGmQkAauRAjB\n8thKdURmZiYrV64E4ODBgyxatIi4OGsM4c477+SRRx5hzpw5LF++vPGYWbNmnfb45Zdf5oEH7CkX\niQSDs5IBSNCRZKVUjGhPknwI6BfwuK//udOIyLnA08BlxpjihueNMYf8/y0QkbewyjfOSJKNMU9i\n1TIzceJE04HXoFpRW+/j4y3HuGRkNq4wllo0mD44gwnx+aR8+Bco+xS8NafvEOeykuV+U+DCn0PW\nsLDHqFSgvLw8zj333MbH48ePZ/bs2Xi93hZ7JdfW1nLllVdyzjnnhCvMiJXgclBZ2zUXVVFKxZb2\nJMmrgaEiMhArOf4W8O3AHUSkP/AmcL0xZkfA80lAnDGm3P/zxcAvgxW8atvS3UWUV9dz+ejs8F7Y\n54Vt7xG/8nH+JkupKo3HN/E7xA27DGpPQtVxqD4B1cehogi2vAN/nAaTbobz7oGk8I96KwUwd+5c\n5s6de9pzN910U6vHuN1u5s+fH8qwoobH5aCkotbuMJRSqtPaTJKNMfUi8n3gQ8ABPGuM2Swit/u3\nPw78HMgA/ijWV+f1xpiJQE/gLf9zTuBlY8zCZi6jQmThxqOkxDuZMSQzfBfd+Qm89yM4fgC69Wfz\nqH/j2jVDeWbURS0vZHLhfbDoN7D6ach7Dc77N5i8AJzx4YtbKdVpCW6tSVZKxYZ21SQbY94H3m/y\n3OMBP98C3NLMcXuAMU2fV+FhjGHJriJmDs0k3tnykrpBlfcqvH0HZA2Hb74I51xG/zpD9fpP+HDT\n0ZaT5OQsmPswTL4VPvoZfHQvrH4G5j0KOTObP0YpFXESXHFak6yUigm6LHUMO1BSyaHjVeGbsLfs\nUXjrNsiZATcthNyvgsNJisfF9CEZfLjlKMa0UW7eIxe+8wZ8528Q54AXvw57vwhP/EqpTvO4Tk3c\n8/kMH285hs+n00yUUtFHk+QYtny3NX9y2uAQl1oYAx/fBx/9J4yYB9e9YXWwCHDJyGwOllSx9Ug7\nVyMbMgdu+hDSB8DL34QDK0MQuIoUbX54inLBfn0i8qyIFIjIplb2Od+/0ulmEfk8qAG0InAxkbUH\nSrn1hTWsPVDaxlFKKRV5NEmOYct2F5OVEs/grKTQXcRbD+9+H5b+HibeBF9/rtk64jm5PRGBj7Z0\nYGGRpEyY/w6kZMNLX4dD64IYuIoUHo+H4uLimE2UjTEUFxcHeyW+54FLW9ooImnAH4GvGWNGAtcE\n8+Kt8bgc1NT78PkMJyqtJelPVmu3C6VU9Alqn2QVOYwxLNtdzIwhGUio+hDXVcMb34Xt71sdKc6/\np8Wex1kp8UwckM6Hm4/xwzkdaJOVkg03/B2euwz+chXc+A/IHh2kF6AiQd++fcnPz6ewsNDuUELG\n4/HQt2/wFiI1xiwWkZxWdvk28KYx5oB//4KgXbwNCW5r/kN1vZdKf9lFTb0vXJdXSqmg0SQ5Ru0u\nPEnRyZrQ1iN/+B9WgnzZ72BK24spzh7eg/9ZuJ2C8mp6pHRgVK1bH3+ifDm8MA9ufB96DO9E4CqS\nuFwuBg4caHcYseYcwCUii4AU4H+NMS80t2OwF3JKcPmT5DofVf5+ybVeTZKVUtFHyy1i1LKGeuRB\nIapH3vwWrHkWpv+gXQkywEx/G7qGWukOSR8AN7xrLT7ywtegZE/Hz6FU1+EEJgBXAJcAPxORZr/C\nMcY8aYyZaIyZmJWV1ekLNyTJVXVeKv21ybU6kqyUikKaJMeoZbuK6ZOWQL/uCcE/ecleePcH0HeS\ntUpeO43s3Y1uCS6W7io6u+tmDLYSZW8t/O0Wqx5aKdWcfOBDY0yFMaYIa5XTsLTjjHdZv1aqajVJ\nVkpFN02SY5DPZ1ixt5jpg0NQj1xfa9Uhi8C/PAMOV7sPdcQJ0wdnsGRn0dlP0soaBnMfgUNrYckj\nZ3cOpWLfO8BMEXGKSCIwBdgajgufKrfwNna5qK3XvslKqeijSXIM2nq0jOOVdUwLRT3yJ7+Aw+th\n3mNWCUQHzRiSyeET1ewrrjz7GEZeBaOvgc8fgMMbzv48SkUpEXkFWA4ME5F8EblZRG4PWAl1K7AQ\n+BJYBTxtjGmxXVwwNUzcCyy30Il7SqlopBP3YtCp/shBTpK3fwArHrOWi8796lmdomF57CW7ihiY\n2YnWdJf/DvYtsRYvWfA5uILaXkupiGaMubYd+/wO+F0YwjlNY01yrZeqOv/EPU2SlVJRSEeSY9Dy\n3cUMykyiV7cg1iOfyIe3vwfZ58JFvzrr0+RkJNInLYFlZ1uX3CAh3RrNLtwGn559PEqp4PI0N3FP\nu1sopaKQJskxpt7rY+XeEqYGcxTZWw9v3AzeOrjm+U6N2ooIM4ZksGx3Md7OLlU75EKYdAssf0yX\nrlYqQjT2ST6tJlmTZKVU9NEkOcZsOlzGyZr64PZHXvscHFwBVzxsdZjopBlDMjlRVcfmwyc6H9tF\nv4TuA+HtO6C6rPPnU0p1SqI/Sa6o8VKli4kopaKYJskxZtluq4xh6qAgJcmVJfDZryHnK3DuN4Jy\nyumDT9Uld5o7Ca56AsryrcVNlFK2SvVYHW/Kqut04p5SKqppkhxjlu8uZljPFDKT44NzwkW/geoT\ncOkDLS453VFZKfEMz045+37JTfWbDDN/BOtfhF3/DM45lVJnJdHtwBknlFXVaZ9kpVRU0yQ5htTW\n+1i9ryR4XS2ObYHVz8DEmyB7VHDO6TdjSCar95VSXRek/qnn3Q3pOfDRz8CnPVmVsouIkJrgoqy6\nTpelVkpFNU2SY8iGg8eprvMFJ0k2BhbeDfEpMPs/O3++JmYOyaS23sfa/aXBOaEzHi68Dwo2w4aX\ng3NOpdRZSfU4KauqDxhJ1g+uSqnoo0lyDFm2uwgRmDowCEnytn/A3sVWgpzYvfPna2LywO444yQ4\ndckNRl4FfSZaNdS1FcE7r1KqQ06NJGu5hVIqemmSHEOW7y5mVO9udEts/1LRzaqrhg//E3qMsEot\nQiAp3sn4/unBq0sGq2b6kl9D+RGrLZxSyhapHhcnquqo1O4WSqkopklyjKiq9bL+wPHglFosfxSO\n74dLfwOO0C3KOGNIJhsPneB4ZW3wTtp/qrUa4JLfQ/mx4J1XKdVuqQlOik7WNPZC15FkpVQ00iQ5\nRqzdX0qtNwj1yGWH4YuHYfhcGHR+MEJr0cyhGRgDK/YUB/fEc+4Hb43VmUMpFXbdElwcK6tpfKwT\n95RS0UiT5BixYk8xjjhhUk4n64c/+QX46q2yhRA7t28aSW5HcOuSwVrwZOLNsO4FKNgW3HMrpdqU\n6nGdNnqsI8lKqWikSXKMWL2vhFG9U0mO70R5xNFN8OVrMO1Oq51aiLkccUwdlMHSXUEeSQarJZw7\nCT65L/gxFfQjAAAgAElEQVTnVkq1KjXh1LwIEU2SlVLRSZPkGFDn9ZGXf5zxA9I7d6LF/wPxqTDj\nB8EJrB1mDMlkb1EF+aWVwT1xUoa1wMiOhVaXDqVU2KR6nAE/u3TinlIqKmmSHAM2Hy6jus7HxAGd\nKLU4tgW2vANTboOETibbHTBzqLVE9bJQjCZP/R6k9oWP7gWf/pJWKlwCR5LTEjVJVkpFJ02SY0DD\nghwTczqR3C7+HbiTYeodQYqqfYb2SCYrJT74dckArgS44F44kgfb3wv++ZVSzUr1BCbJbl1MRCkV\nlTRJjgFr95fQNz2BnqmesztB4XbY/BZMXhCShUNaIyJMH5zB8j3FGGOCf4HR10D6QFj8oLWKoFIq\n5E4bSU5waXcLpVRU0iQ5yhljWLOvlImdqUde/DtwJcK07wcvsA6YNiiDwvIa9hSFYJU8hxNm/hCO\nbIDdnwb//EqpM3RLOFWTnJZodboIyYdgpZQKoXYlySJyqYhsF5FdInJPM9uvE5EvRWSjiCwTkTHt\nPVZ1Tn5pFQXlNUw42yS5aCds+htMvsWa7GaDqYOs6y7fHYK6ZIAx10JKb/jiodCcXyl1mtPKLRJc\n+AzU+zRJVkpFlzaTZBFxAI8BlwEjgGtFZEST3fYC5xljRgO/Ap7swLGqE9bsLwFgwtlO2lv8IDg9\nMD18HS2aGpCRSHaqJ/iLijRwxlsdO/Yvhf3LQ3MNpVSj0yfuuQFtA6eUij7tGUmeDOwyxuwxxtQC\nrwLzAncwxiwzxpT6H64A+rb3WNU5a/eXkhLvZFh2SscPLt4NG1+HSTdDUmbwg2snEWHa4AxW7CkJ\n3Vey4+dDYoaOJisVBvHOONwO69dLQ8KsSbJSKtq0J0nuAxwMeJzvf64lNwMfnOWxqoPW7CtlbP80\nHHHS8YMXPwiOeFtHkRtMHdSdopM17C48GZoLuJOszh27PobDG0JzDaUUYH3wTU1w4XHF4XFZv2Z0\n8p5SKtoEdeKeiMzGSpLvPotjF4jIGhFZU1hYGMywYlZZdR3bj5WfXX/kkj3W6noTb4LkHsEProMa\n65L3lITuIpNvtRZL0dFkpUIuNcFJottJvNMB6EiyUir6tCdJPgT0C3jc1//caUTkXOBpYJ4xprgj\nxwIYY540xkw0xkzMyspqT+xd3voDxzGGs5u098XD4HCFdXW91vTvnkjvbh5WhGryHoCnm5Uob/27\n1fZOKRUyqR4XCS4Hbqf1a0YXFFFKRZv2JMmrgaEiMlBE3MC3gHcDdxCR/sCbwPXGmB0dOVadvbX7\nSogTGNs/rWMHlh2BvFetOt2U7NAE10EiwtRBGawIVb/kBlPvsBYZWfJI6K6hlCI1wUWi29FYm1yj\nC4oopaJMm0myMaYe+D7wIbAVeN0Ys1lEbheR2/27/RzIAP4oIhtEZE1rx4bgdXRJaw+UktsrleR4\nZ9s7B1r1JBhv2FfXa8vUQRkUV9SyqyBEdclgTVCccCN8+TqU7gvddZTq4q6b0p+bZg4k3j+SrOUW\nSqlo067syhjzPvB+k+ceD/j5FuCW9h6rOq/e62P9geNcM6Fv2zsHqq2ANc/C8LnQfWBogjtL0wY3\n1CUXM7TnWXTraK/pd8Gqp2DpH2Duw6G7jlJd2CUjrW+plvqXnNckWSkVbXTFvSi17Wg5lbVexne0\nHnnDy1B93LbV9VrTNz2BPmkJoeuX3CC1N4z9Nqx/EU7qJFEVfUTkWREpEJFNbew3SUTqReTr4Yqt\nqcaRZO1uoZSKMpokR6k1+6wuEBNzOtDZwueF5Y9B30nQf0qIIjt7p+qSS/CFenWuad8Hb401qq5U\n9HkeuLS1HfyLOf0W+CgcAbXEreUWSqkopUlylFqzv5Re3Tz0SUto/0HbP4DSvTDtztAF1klTB3Wn\npKKWnaGsSwbIOgeGXASrn4b6mtBeS6kgM8YsBtrql3gX8DegIPQRtUy7WyilopUmyVFq3f7Sjrd+\nW/4odOsPw78amqCCoKFfcshLLgCm3QEVBbDpb6G/llJhJCJ9gKuAP9kdS0N3Cx1JVkpFG02So9Dh\n41UcPlHNxI4kyflr4cBymPo9cHSwG0YY9eueSN/0BJaHsl9yg0GzISsXVvwRQtl2Tqnw+z1wtzGm\nzcw01As5xbt0MRGlVHTSJDkKrdlfCsCEjqy0t/xRa7W5cd8JUVTBM3VQBiv3Foe+LlnE+tBwdCPs\nWxLaaykVXhOBV0VkH/B1rPacVza3Y6gXcmrsk6wT95RSUUaT5Ci0dl8JiW4Hub3a2Sbt+AHY8g5M\nuAE8qaENLgimDsqgtNJacjvkzv0GJHSHFbZ/K61U0BhjBhpjcowxOcAbwB3GmLftiEUn7imlopUm\nyVFozf5SxvZLw+lo51/fyies/06+LXRBBdHUQdYIeVjqkl0JMPEm2P4+lOwJ/fWUCgIReQVYDgwT\nkXwRubnJAk8RI96pK+4ppaKTJslRpqKmnq1Hytpfj1x9Atb+GUZeBWn9QhtckPRNT6Rf9zDVJQNM\nugXinKc+TCgV4Ywx1xpjehljXMaYvsaYZ4wxjwcu8hSw743GmDfsiBN04p5SKnppkhxlNhw8js/A\nhPb2R173F6gtj+i2b82ZNiiDlXvD0C8ZILUXjLraWlyk+kTor6dUFxIXJ7gcokmyUirqaJIcZdbu\nL0UExvVPa3tnnxdWPQH9p0Of8aEPLoimDsrgRFUd246GoS4ZrAl8tSetDxVKqaByO+Iak+QvdhZy\nsKTS5oiUUqptmiRHmbX7SxnWM4VUj6vtnXd+ZE3am7Ig9IEFWUO/5OXhqEsG6D3O+jCx8gnw1ofn\nmkp1EW5nHLVeH4XlNXz3udU8/YXW/yulIp8myVHE5zOsO1DK+PbWI698AlJ6w/C5oQ0sBHqnJTAg\nI5Hlu4vCd9Fpd8CJA9YkPqVU0LidcdTU+Xh7/SHqfYbjVXV2h6SUUm3SJDmK7Co8SXl1PRP6tyNJ\nLtwBez6zOjc42jHqHIGmD85k5Z4S6sPVX3XY5ZA2wFpcRCkVNPFOB7VeH39dexCAMk2SlVJRQJPk\nKLK2cRGRdiTJq58Gh9vqjRylZgzJoLymni8PhWkyXZwDJt9qrUx4dGN4rqlUF+B2xrFiTzE7jp1E\nBMqrtaRJKRX5NEmOImv3l5KR5GZARmLrO9aUw4aXrbZvyT3CE1wITPPXJS/bFcaSi7HXgTMBVj0V\nvmsqFeMyk90cOVHNpJx0Zg7JpKxaR5KVUpFPk+Qosm6/VY8sIq3vmPeq1fZtcvRN2AuUkRxPbq9U\nlu4K0+Q9gMTuMPrrsPGvUFUavusqFcMe/84EVv6/C/nr7dPJTvXoSLJSKipokhwlSipq2VNU0Xap\nhTGw6kmrW0OfCeEJLoRmDM5g7f5SqmrDuFrX5FuhrtIajVdKdVpaopueqR4AUhNcWpOslIoKmiRH\niXXtrUfeswiKdlhLULc14hwFZgzNpNbrY83+kvBdtNcY6DfFquv26QIISgVTisdJRa03fBNylVLq\nLGmSHCXWHijF5RBG9+nW+o6rnoLEDKseOQZMzumOM07CW3IBVqlKyR7Y/Wl4r6tUjGvo8X6yRksu\nlFKRTZPkKLF2fykje3fD43K0vFPpftjxAUy4EVyesMUWSknxTsb1T2NZOPslA+R+DZJ6wGqdwKdU\nMKV4nIB2uFBKRT5NkqNAnddH3sHjbZdarHnW+u/Em0IfVBhNH5zJxkMnOFEZxjpGp9v6sLHjQyjd\nF77rKhXjUhOskeQTWpeslIpwmiRHgS2Hy6ip97WeJNdVwbo/w/AroFvf8AUXBjOGZGJMGJeobjDx\nuyBxsPqZ8F5XqRimI8lKqWihSXIUaNciIpv+ZrUsm3xbmKIKn7H90khwOcJfcpHaG3Lnwvq/WB9C\nlFKd1lCTrL2SlVKRTpPkKLD2QCl90hIaWyidoaHtW1Yu5MwMb3Bh4HbGMXlgd5aGc1GRBpNutT58\nbPpb+K+tVAxqSJJ1JFkpFek0SY4C6/aXtj6KfGgtHMmDSTfHRNu35swcksnuwgqOnqgO74VzZlof\nPlY9aX0YUUp1SmqCVW6hvZKVUpFOk+QId/h4FUdOVLeeJK9+GtzJMOZb4QsszKYPsZaoDvtosghM\nvsX6EJK/JrzXVioGJcdrTbJSKjpokhzh2qxHriiGTW9aCXJ8ShgjC6/c7FS6J7lZGu66ZIBzvwXx\nqdZoslKqU5yOOJLcDq1JVkpFvHYlySJyqYhsF5FdInJPM9uHi8hyEakRkZ822bZPRDaKyAYR0aG4\nDlq7v5QEl4Ph2S0kwOv/At4amHRLeAMLs7g4YdqgDJbtKsaEu+whPhnGXAtb3oaTheG9tlIxKMXj\nolyTZKVUhGszSRYRB/AYcBkwArhWREY02a0E+AHwYAunmW2MGWuMmdiZYLuidQdKGdsvDaejmb8q\nnxfWPAMDZkKP3PAHF2bTh2RwtKyaPUUV4b/4pJvBWwvrXwj/tZWKMakJTsqqtNxCKRXZ2jOSPBnY\nZYzZY4ypBV4F5gXuYIwpMMasBnRoIIgqa+vZfLis5VKLXZ/A8QNWzWwXMGNwJgDL7OhykTUMBs6C\nNc9ZH06UUmctxeOivEZ/XSilIlt7kuQ+wMGAx/n+59rLAJ+IyFoRWdCR4Lq6L/NP4PWZlpPk1U9D\ncjYMnxvewGwyICORPmkJLN0V5kVFGky6FU4ctFbhU0qdtVSPjiQrpSJfOCbuzTTGjMUq17hTRGY1\nt5OILBCRNSKyprBQ6z7h1KS9cf3TztxYshd2fmwtnexwhTcwm4gIM4ZksGx3EV6fDe3Yhl0OKb1h\n9VPhv7ZSMaS5muTNh0/gs+PftVJKtaA9SfIhoF/A477+59rFGHPI/98C4C2s8o3m9nvSGDPRGDMx\nKyurvaePaev2lzKkRzJpie4zN6551loyecIN4Q/MRjOGZFJWXc/mwyfCf3GH01qqevenULw7/NdX\nKkakJjgpC2gBt7+4giv+sIRFOwpsjEoppU7XniR5NTBURAaKiBv4FvBue04uIkkiktLwM3AxsOls\ng+1KjDGsPVDKhP7NlFrUVVldLXLnWksndyHTBlv9kr/YaUNdMsD4+RDnhNXP2HN9pWJAw0hyQ6ea\nopO1ABwrq7EzLKWUOk2bSbIxph74PvAhsBV43RizWURuF5HbAUQkW0TygR8D94pIvoikAj2BJSKS\nB6wC3jPGLAzVi4kluwsrOF5Z13w98ua3rKWSY7ztW3N6pHgY1SeVz7bZNOKUkg25X4MNL0JtpT0x\nKBXluiW4qPMaKmutSbCVtdaosraFU0pFEmd7djLGvA+83+S5xwN+PopVhtFUGTCmMwF2VWv2lQAw\nvrkkedVTkDkMcr4S5qgiwwXDe/Lopzspqaile1IzpSihNvlW2PwmbHrDGllWSnVIWoI1j+J4VR1J\n8U4qaqwk+aSuwqeUiiC64l6EWrW3hMxkN4Ozkk7fcGgtHF5njSKL2BOczS4c3gOfgc/tql/sPw16\njLA+rIR7YROlABF5VkQKRKTZ8jURuU5EvvQv5LRMRCJqsKJhnsXxSqvMoqLGGlEu0yRZKRVBNEmO\nUCv3ljB5YHekaSK86mlwJ1vLUHdRo/t0IzM5nk+32dQFRcT6kHL0S8jXRSSVLZ4HLm1l+17gPGPM\naOBXQEStqZ6W6B9JrrTKKxrKLU7WaJKslIocmiRHoPzSSg4dr2JSTvfTN1QUwaa/WQmyJ9We4CJA\nXJxwwfAsPt9eQJ3XZ08Q534D3CnaDk7ZwhizGGul05a2LzPGlPofrqD5cjjbpDeOJFtJcoW/Nllr\nkpVSkUST5Ai02l+PPHlgkyR53QvgrbEWtejiLhjeg7Lq+sZe0mEXnwJjr7UmUZ7Uvt4qot0MfNDS\nRjt61DeMJJf6yy0qaxom7ulIslIqcmiSHIFW7S0lxeNkeHbAaLG33uqNPHAW9BhuX3ARYubQLFwO\n4VO7ulyAVXLhrYX1L9gXg1KtEJHZWEny3S3tY0eP+m7+iXsnqqyR45P+mmQtt1BKRRJNkiPQqr3F\nTMrpjiMuoB55x0JrSeTJt9kXWARJjncydVAG/9x6zL4gsoZZH1pWP2t9iFEqgojIucDTwDxjjE1r\nuTfP43KQ4HJQWuEfSa7VkWSlVOTRJDnCFJ2sYXdhxZmlFquehG794JzW5up0LRcM78Huwgr2F1fY\nF8Tk26AsH7a/Z18MSjUhIv2BN4HrjTE77I6nOemJLo5XNa1J1iRZKRU5NEmOMKv3WvXIp03aK9wO\nez+HiTdZSyMrwEqSAXtLLoZdBt36w8qIah6gYpyIvAIsB4b5F2+6OXCBJ+DnQAbwRxHZICIR14al\nW6K7sQXcqZpknbinlIocmnFFmJV7S/C44hjdp9upJ1c9BY54XbiiiQEZSQzOSuLTbQV8d8ZAe4KI\nc8Ckm+GT++DoJsgeZU8cqksxxlzbxvZbgIhekjM90UVpZUNNspUk19T7qK334Xbq+I1Syn56J4ow\nq/eVML5/+qlfEtVlkPcKjPoXSMq0N7gIdGFuT1bsKbZ3ws/4+eD0WCUxSql2SUt0nRpJ9pdbgE7e\nU0pFDk2SI0hZdR1bjpSdXo+c9yrUnrSWQlZnuGB4D+q8hiU7bWzDltgdRl8DX74OlS22rlVKBUhL\ndAf0ST6VGGvJhVIqUmiSHEHW7ivFmID+yMZYo5N9JkKf8fYGF6EmDEgn1eO0ty4ZYMptUF8F61+0\nNw6lokRagjVxzxhDZY2XdH/vZJ28p5SKFJokR5CVe0twOYRx/dKtJ/YsguKdMHmBrXFFMpcjjvOG\n9eDTbYX4fMa+QLJHQ//p1gp8Pm/b+yvVxaUnuvH6DOU19VTU1NMz1QNokqyUihyaJEeQVXuLGd2n\nGwluh/+JJyExE0ZeaW9gEe7C4T0oOlnDxkMn7A1kygI4fgB2fGhvHEpFgW7+kePjFXVU1NaT3a0h\nSdZyC6VUZNAkOUJU1XrZeOgEkwdmWE+U7oftH8CEG8EZb2tske68c7KIE/in3SUXw+dCSm9Y9YS9\ncSgVBdIT3QAcLavGZyDbP5KsE/eUUpFCk+QIsf5gKXVew5SGeuRVT1rtxSbeZG9gUSA9yc34/ul8\nus3G1fcAHC6rHdyeRVZva6VUi9L8I8mHj1cBaLmFUiriaJIcIVbtLUEEJuSkQ005rHsBRlwJ3frY\nHVpUuCC3B5sOlXGsrNreQCbcaPW01nZwSrWqYaLeoTOSZC23UEpFBk2SI8SqvSXkZqeS6nFZHRJq\nymDaHXaHFTUuyu0JwAcbj9gbSFKm1dN6wytQbXONtFIRrFuCVW6RX1oJWEmz2xlHuZZbKKUihCbJ\nEaC23se6A6VW6zefF1b8CfpNhT4T7A4tagztmcKIXqm8teGw3aFYE/jqKrQdnFKt6J7kJt4Zx7aj\n5QAkxTtJ9Ti13EIpFTE0SY4Amw6foLrOZ9Ujb38fju/XUeSzcNW4PuQdPM6ewpP2BtJ7nNUObsXj\n4NVf+Eo1xxEnDOmRzOZDZQAkxTtIjndyUpNkpVSE0CQ5Aqzaa63SNmlgd1j+R0jrb3VKUB3ytbG9\nEYG3I2E0efr34cQB2Pqu3ZEoFbHO6ZlCrdcHQKLbSYrH1ViTXOf1YYyNvc+VUl2eJskRYNXeEgZn\nJZFZtgUOLIMpt1udLVSH9Ez1MGNwJm+vP2T/L9dzLoXug2D5o9bKiUqpMwztmdz4c5LbSbcEF/ml\nVXh9hq89upRfv7fVxuiUUl2dJsk28/oMq/eVWPXIy/8I7mQY9x27w4paV43rw4GSStYdKLU3kDgH\nTL0DDq2FgyvtjUWpCHVOj5TGn5PiHVw2OpudBSf5z7c2svVIGTsLbC6dUkp1aZok22zz4ROUV9dz\nXrYXNr8J464HTze7w4pal4zKxuOK4631h+wOBcZ+GxLSYdn/2R2JUhFpWHZgkuzk6xP60jM1nldX\nHwSgtLLWrtCUUkqTZLst3lEIwKwTb1udLabcZnNE0S053snFI7L5x5dHqK332RuMO8laDGbbe1C8\n295YlIpAfdISSHA5iBOId8YR73SwYNZgwFpspKRCk2SllH00SbbZ4p1FjO/lJvHLP8PwK6D7QLtD\ninpXjevD8co6Fm23eZlqgMkLrJX4Vj5udyRKRZy4OGFoz2SS4p2ICAA3TBvAczdO4qpxfSjVJFkp\nZSNNkm10sqaedftLWZC2GqpKYdqddocUE74yNJOMJDdvb4iAkouUbBh9jdUzubLE7miUijjn9u1G\nj5T4xsdORxyzh/cgI8lNRa2X6jqvjdEppboyTZJttHx3MV6fl1nFb0CvsdB/mt0hxQSnI46vjunN\nJ1sLOFEVAUvcTrsT6iph7XN2R6JUxPmPy3J55dapZzyfnmStyHe8so7rn1nJ3/MioLWjUqpLaVeS\nLCKXish2EdklIvc0s324iCwXkRoR+WlHju3KvthZyOXuPBLLdluJlP/rRtV5V43rQ229z/5lqgF6\njoRBs2Hlk1CvXx8rFSgp3kmPVM8Zz3dPtJLkAyWVfLGzyP6ONUqpLqfNJFlEHMBjwGXACOBaERnR\nZLcS4AfAg2dxbJf1xY5CfpTwD0gbACOvtjucmHJu324MykyKjC4XYC0ucvIobPqb3ZEoFRUaRpI3\nHToBQFXtqbKLf3x5mMc+22VLXEqprqM9I8mTgV3GmD3GmFrgVWBe4A7GmAJjzGqg6XfbbR7bVR0s\nqaRn6RqG1GyFGT8Ah9PukGKKiHDVuD6s3FvCoeNVdocDgy+ErFxdXESpduruT5I3H7aWra4MSJLf\n2XCYl1bstyUupVTX0Z4kuQ9wMOBxvv+59mj3sSKyQETWiMiawsLCdp4+ei3eWcgdjneoT8iCsbp4\nSCjMG2v9r/ZOJEzgE7FKao5tgt3/tDsapSJeemJDkmyNJAcmyWVVdZRX19sSl1Kq64iYiXvGmCeN\nMRONMROzsrLsDifk9m1cyizHRhzT7wTXmfV4qvP6ZyQycUA6b62LgGWqAc79BqT2gcUP2R2JUhEv\nLdEF0LjqXlXdqaS4vLqe8pp6vL4I+HetlIpZ7UmSDwH9Ah739T/XHp05NmbVe31MOvg8VXHJyKSb\n7Q4npl09vi87C06yZn8ETPpxxsOMf4UDy2DfUrujUSqiuRxxpHicjYnwaSPJ1VZl38kaHU1WSoVO\ne5Lk1cBQERkoIm7gW8C77Tx/Z46NWds2rWUOqzg09DrwpNodTky7clxv0hNd/GlRhKx4N34+JGXB\n4t/ZHYlSEa+hLhlOn7hX5m/tWBYJLR6VUjGrzSTZGFMPfB/4ENgKvG6M2Swit4vI7QAiki0i+cCP\ngXtFJF9EUls6NlQvJmos/V9qcdJjzg/tjiTmJbqdfHfGQD7dVsDWI2V2hwOuBJh+F+z5DPLX2h2N\nUhGtoS4ZTo0k+3yGcv8IstYlK6VCqV01ycaY940x5xhjBhtjfu1/7nFjzOP+n48aY/oaY1KNMWn+\nn8taOrZLO5HP8IL3+WfipaRm9bY7mi7hhmk5JLkdkTOaPPEmSEiHLx5se1+lurDAkeSGJLmitr6x\nQUxD2YVSSoVCxEzc6ypqFv8BY+DwiFvtDqXL6Jbo4rqpA/jHl4fZX1xhdzgQnwJTvgfb34ejG+2O\nRkUhEXlWRApEZFML20VE/uBfxOlLERkf7hiDoWEkOd4ZR1WtNWpcVn36BD6llAoVTZLDqaIYx4YX\neNc3nXGjR9sdTZdy88yBOOPieGLxHrtDsUxZAO4U+EI7Xaiz8jxwaSvbLwOG+v8sAP4UhpiCrnuS\n1eFiUFYylXVejDGn1SFrTbJSKpQ0SQ6nlY8T563mz3FXMbZfmt3RdCk9Uz38y4S+vLEmn4KyarvD\nscotJt8Km9+Gwh12R6OijDFmMdZKpy2ZB7xgLCuANBHpFZ7ogifNP5I8pEcyxkBNve+0xLhcyy2U\nUiGkSXK4VB3HrHqCL+Im02vIGJwOfevD7fbzBlHv8/HMkr12h2KZdic4PbDkYbsjUbEnJhZyGtsv\njUFZSeT2SgGsuuTAEosyLbdQSoWQZmrhsuz/kOoT/LZqHrPOif3FUiLRgIwk5p7bmxdX7OdEZQSM\nQCVlWpP4vnwdSiIkcVddTiQv5DRjSCaf/uR8MvwT+Cpr60+brKcjyUqpUNIkORxOFsKKP7G358Vs\nMTnMGhpZv4i6ku+dP5iKWi8vLN9ndyiW6XdBnAOW/q/dkajYElMLOSW4nYDVK7mh3MLtjKOsSkeS\nlVKho0lyOCx5BOqreNrxLXIyEunXPdHuiLqs3F6pXDC8B88t20dlbQT8gk3tBeO+A+tfhOMH7I5G\nxY53gfn+LhdTgRPGmCN2B3W2El0OwCq3aCix6JOWoC3glFIhpUlyqJ04BKufpn70N3nzYJKWWkSA\nO84fTElFLa+uOtj2zuHwlZ+AxMFnv7E7EhUlROQVYDkwzL94082BCzwB7wN7gF3AU8AdNoUaFInu\nU0lyeXUdCS4H6YkubQGnlAopp90BxLzFvwPj44veN1O1qoBLR2XbHVGXNzGnO5NzuvPk4j18c1I/\nkuJt/mfQra/VEm7Zo1b5Rc8R9sajIp4x5to2thvgzjCFE3IJ/iS5qq6esqp6UhOcpHhclFbW2hyZ\nUiqW6UhyKJXshfV/gQk38MYeB5nJbqYMzLA7KgX8+6XDOFZezW8+2Gp3KJaZP4b4VPjnL+2ORKmI\nk9hYk+yjrLqOVI+LFI9TR5KVUiGlSXIoLXoA4pxUTf0Rn24t4JKR2TjixO6oFNZo8s0zBvLiigMs\n2VlkdziQ2B1m/ivs+AD2L7c7GqUiyqlyC6u7RYrHSWqCSxcTUUqFlCbJoVKwFb58DSbfyqLDDqrq\nvFwxOup6+ce0n14yjEFZSfz7G3mRMQFoyvcgORs++QUYY3c0SkWMxMZyC6tPcmrCqZFko/9WlFIh\noklyqHz2a3Anw4wf8d7GI2QkuZk8sLvdUakAHpeDh64Zw9Gyan79jwgou3Anwvl3w8EVsGOh3dEo\nFR8H9SYAACAASURBVDEayi0q/S3gUj0uUj0uar0+aup9NkenlIpVmiSHwuH1sPXvMO0Oqlxp/HOr\nNWFPV9mLPOP6p3PbeYN5bc1BPttWYHc4MO566D4YPrkffF67o1EqInhccYicagGXmuAk1WMlzhHx\nLZBSKiZp1hZsxliTrzxpMO1OFm0v0FKLCPfDOUM5p2cy97z5pf0r8TlccOHPoHCrtRKfUgoRIcHl\noKq2nrKqOlI8LlITXAC6oIhSKmQ0SQ62HQth96dw3r+Dp5uWWkSBeKeDh64ZS9HJWu7/+2a7w4ER\nV0LvcfDZf0N9jd3RKBUREt0OiitqqfeZxu4WoEtTK6VCR5PkYKqrhoX3QOYwmLyA6jovn24r4BIt\ntYh4o/t2487zB/Pm+kN8tPmovcGIwJxfwIkDsPoZe2NRKkIkuB0cKK4EoHuSVZMMaK9kpVTIaOYW\nTCseg9J9cNkD4HCxaHsBlbVaahEtvn/BUHJ7pfKT1/NYva/E3mAGnQ+DZsPi/4FKm2NRKgIkupxs\nPHQCgKE9UzgnO4U4gQ0HjtscmVIqVmmSHCxlh2HxQzB8Lgy+AID3Nh6le5KbKVpqERXczjieuWEi\nWSnxXP/MSj7fUWhvQJf8N1SXWS3hlOriEtwOaup9iMCwnimkelyM7pvGst3FdoemlIpRmiQHy8f3\nga8eLv4vAKrrvPxz6zEuGamlFtGkd1oCr98+jUGZydzy59V8sPGIfcH0HAFTv8f/b++849uqzj7+\nPZIsW5ZteW87thM7ezs7gQwICXuVvaGUFigtb0tDB6Wb2Rf6slfLng07CSNABiF7OjtxnHjvvSWd\n9497pcjbSWzLsc/389FH8l167rlXvr/73Of8Dltfg5zN3otDoegHuLySh4T6u4eSnzk0jO3ZFdQ2\nqs57CoWi51HqrSc4th52vQcz74bQZAB3qcX541SpxelGeIAvb98+nXHxwdz51lbe35ztvWDmLoHA\naPj8XmUJpxjUuETyyJgg97SZQ8OwO6X3y6MUCsWARInkU8XpgOX3QWAszLnXPVmVWpze2Cw+vH7r\nVGYNC+fXH+zk398f8U4gvoFa2UX+Dtj8indiUCj6ARZ9QJER0cdFcvqQUHyMgh+6WXJhdzh5c8NR\n7A41AIlCoegaJZJPlW2vawJm4V/AbAVUqcVAwd9s4qUb01k0Opo/fbqHBz7OoKTGC5Zsoy/ROvKt\n/AvU9IMBTxQKL+Dv48okB7qnWcxG0oeE8vXeQqSU/GPZXlZkdOxOs+FIGb/7MIM1h0p6PV6FQnH6\nY/J2AKc19RXawCGJM2HMZe7J3+0vVq4WAwRfk5GnrpnIXz/fy2s/ZPHBlhxunJnE7XNSCLGa+yYI\nIeDcx+CZGfDVA3DJcwA4nJKSmkYKqxooqGygsLqRqvpm6psc1DbZ9XcH9U12hBCYTQbMRgM+Rtdn\nI2EBZiIDfYkM8iMiwJfIIF9C/c0YDKJv9k2h6CaWdsotAC4YH8tvP9zFO5uyeX51JovHRLNoTHS7\n26is1zyVj5bUwvDejVehUJz+KJF8KnzzV6gvh8UPa0JG5/Nd+YRazUxPUaUWAwGT0cCDF47mhhlD\neHLlQZ5bdZjXfzjKLbOTuXV2MjZ95K+eRkpJRV0zWaW1HC31Jzb2Wqbu+A+/PTqRlXVDKa5uxCnb\nrmcQYDWbsJiN+JuNWMwmpJQ0OZw0O5w02yXNDicNzZqIbo3ZaCAlwsrw6EDSorTX8KhA4kMsSjwr\nvEZaVCDDIgOID7G0mH7e2Bge/GQ3f/xYGwiooKqhw23UNGgd/LJ0v2WFQqHoDCWST5Yja2DTizDt\npxAzzj25pKaRLzIKuHpqgiq1GGCkRATw5FUTuXPeMJ74+gD/WnmQ/3x/hLNGRTE+Ppix8TZGxQTh\npz8W7g5SSsrrmsmrqOdISS1HSmrJKqklU//synwB+Isz+Mb3c+6oeQZH2itEBgcQFeSnv3yJCvLD\nZvHB12RAiO6J2fomB0XVDRRVN1Jc3UhRVQN5lQ0cLKxmc1Y5H2/Pcy8b4Gti0pAQpiaFMDU5jHHx\nthPaV4XiVLhmWiLXTEtsM93m78OCkZEs18ssCis7FslV+uh8x8qUSFYoFF2jRPLJ0FgDH/8MQlNg\nwQMtZr2z8RhNDifXz0jyTmyKXictKpBnrp3M7rxKnv3uMKsPlLB0ay4ARoMgLSqQMbFBWH1NCAEG\nIRBoDxscTiiqbtBKJKoaKKxqpMneshNRjM2P5HAr54+LISnMSlK4laQwfxJC/fE75AvvXsvD8T/A\nzLtOeV8sZiNDwqwMCbO2O7+6oZmDRTUcKKhmV24lm7LKeOxLzT/abDQwPsHGjKHhLBgRydg4m8o0\nK7zCddOHsPZQCXNSw/lydyFOp2z3XKx2Z5Jr+zpEhUJxGqJE8snw1QNQkQ23rACzv3tys8PJG+uP\nMSc1nGGRAV4MUNEXjI618dQ1k5BSUlDVwM6cSnblVLIzt5LvDhTT0OwACRJwSomUWilERKCW9Z2U\nGEK0ngmOsfnpYtjqrr1slxHnQdoirdQndSFEpPXqPgb6+TApMYRJiSFcpU8rr21iU1YZm7LK2JhV\nzlPfHORfKw8SGejL/BGRnDUyilnDwjvfD4WiB5k1LJxdD57Daz9ksWxXAaW1TUQE+rZZrkb3U84p\nq8fhlBjVTZ1CoeiEbolkIcQi4EnACLwkpXyo1Xyhzz8XqANuklJu1edlAdWAA7BLKdN7LHpvcPhb\n2PwyzLgLEqe3mPXl7kIKqhr468VjvBScwhsIIYixWYixWThndPsdhnrwy+CCJ7VOfEt/DLd+BaY+\n6kCoE2I1s3B0NAv1fS2rbeK7/UWs3FvEZzvzeWdTNr4mA3OHR3DRhDjmj4hUZRmKPiEqyA+AwqqG\ndkVytV5u0eRwUlDVQFywpc0yCoVC4aJLkSyEMAJPA2cDOcAmIcQnUso9HostBlL11zTgWf3dxTwp\n5envudNQBZ/cDWGpMP/3bWa/ui6LhFAL80ZEeiE4xaAhMFoTyu9dD6sehgV/8Go4oVYzl06K59JJ\n8TTZnWw4UsrXewpZllHAF7sLCfQ1sWhMNBdPjGN6SpjK3il6jWhdJBdUNjAmztZmfo3HyHxHS2qV\nSFYoFJ3SnUzyVOCQlDITQAjxDnAR4CmSLwJek1JKYL0QIlgIESOl9OKYvr3Al7+Dqlwte+fT8p/r\nnrwqNmaV8btzRyoRoOh9Rl0IE66Dtf+E1LPbPNXwFmaTgTmpEcxJjeCBC0az7nAJH2/PY3lGAe9v\nySEy0JdLJ8Vz1ZQEksLbr4NWKE6WaJsukjtwuKhusBMZ6EtRdSNHy+qY2ZfBKRSK047u2C/EAZ7j\n8ubo07q7jAS+FkJsEULc3tGXCCFuF0JsFkJsLi4u7kZYfczBr2HrazDrHohvWzHy6rosLD5GrkhP\n8EJwikHJ4ofAlgBLb9eecvQzjAbBnNQIHvvReDb//iyevmYS4+KDeXFNJnMf+45rX1rPZzvz2nRc\nVChOlvAAX4wGQV5FPT95fTNrD7Z8gFndYCctKhAfo1Cd9xQKRZf0Rce92VLKXCFEJPCVEGKflHJ1\n64WklC8ALwCkp6e34/7qReortDKLiJEw9/42syvqmvhoey6XTorH5t87nrkKRRt8A+HSF+Dfi2HF\n/XDx096OqEP8fIycNy6G88bFUFjVwHubsnlnUzZ3vbWNMKuZyyfHc+20ISSG+Xe9MYWiA4wGQUSA\nL8szCjhSUktCiD+zU8Pd86sbmokLtpAQ6s+RYiWSFQpF53Qnk5wLeKZH4/Vp3VpGSul6LwI+RCvf\nOH2QUhPINYVw8TNgatsZ5N1N2TTandw4c4gXAlQMahKnw+x7YfsbsOcTb0fTLaKC/Lh7QSqr75vH\nf26eQnpSCC+tPcKZj33Lj1/bzLpDJWiVWwrFiRNl8+NIiSaAC6u1YeTXZ5bSZHdS3WAnwNfE+Phg\nth4rV+eZQqHolO6I5E1AqhAiWQhhBq4CWl+NPwFuEBrTgUopZb4QwiqECAQQQliBhUBGD8bf+3z/\nBOz9BM7+M8RNajPb4ZS8vv4o05JDGREd1M4GFIpeZu4SiJkAn94D1QXejqbbGA2CucMjef76dL7/\nzXzunDuMLUfLuealDSx+cg3vbDym2egpFCdAdNDxREZhVQNZJbVc9YJW2lPTaCfQz8SUpFBKaprc\nYlqhUCjao0uRLKW0A3cBXwB7gfeklLuFEHcIIe7QF1sGZAKHgBeBn+nTo4C1QogdwEbgcynlih7e\nh97j8Dew8s8w+lKYcWe7i6zcW0hOeT03zUzq29gUChdGH7j0RWiuhw9uBXuTtyM6YaJtfvzqnOGs\nWzKfRy7TRrBcsnQX0/+xkse/3E+xnhFUKLrC5XBhMgiK9Q56AFkltdQ1OQjwMzE1ORSATVllXotT\noVD0f7pVkyylXIYmhD2nPefxWQJtVKTuiDH+FGP0DuVHNcERMQIuekrzp22HV3/IItbmx9mjovo2\nPoXCk4g0uPBfmnfy8l/D+U90eM72Z/x8jFwxJYEfpcez4UgZL689wlPfHuL51ZlcNimOW2enqIF6\nFJ2SGGbFIODsUVGsOlBMbnk9AIeKawBtgJyhEVbCrGY2Hinnyilth7pWKBQK6F65xeCjuR7evQ6c\nDrjyDTC3b1V1sLCa7w+Vcu30IZiMqikVXmbcFTD7l7DlP7DxRW9Hc0oIIZieEsaLN6Sz8t4zuXxy\nPEu35nLWP1dx26ub2HikbNDXkwohFgkh9gshDgkhlrQz3yaE+FQIsUMIsVsIcbM34uxrrpmayKd3\nz2ZCQjB1TQ4OFFYDcLDQJZJNCCFITwphY1Zpu9uQUrJybyGNdlXuo1AMZpSya42U8NkvoWCn5hwQ\nNrTDRV9ck4nZZODqqSoToegnzH8A0hbDiiXa6JADgJSIAP5+yVi+XzKfexaksuVoOVc8/wOXPruO\nL3YX4HQOPrHsMcjTYmAUcLUQYlSrxe4E9kgpxwNzgcf1fiUDGovZyOhYm3v0va3HygHc9ceBvtoD\n1ClJoWSX1VNU3dZTeU9+Fbe+upl/fnWgj6JWKBT9ESWSW7PpJdjxtmb1NnxRh4vtzqvk/S05XDdt\nCKHWAX/dUZwuGAxw2YsQMRzevxFKD3s7oh4jPMCXX56dxrolC/jzRaMprm7kJ69v4ez/XcV7m7IH\nm9+ye5AnKWUT4BrkyRMJBAohBBAAlAF2BgmR+rDUe/I0D3G7fjMV6KfZdA6N0Mp2cvRyDE+yy7Rp\nr6w9wmG9TKM1ZbVNfLOvkCp9qGuFQjHwUCLZk8xVWgYubRGccV+Hi0kp+fOnewi2+HDPgtQ+DFCh\n6Aa+gXD12yCM8NaVms/3AMJiNnLDjCS++9VcnrxqAmaTkfv+u5M5j3zDi6szWww9PIDpziBPTwEj\ngTxgF3CPlLLNnUS/H8jpJInUM8n2Vk8aAvy0TLJ7dL7KtpnkvApNJBsNgn9+2X42+R/L9nLLfzYz\n+S9fsebgwGk3hUJxHCWSXWRvgrevhrBUuOR5LSPXAV/sLmDDkTLuXThcDR6i6J+EJGn19OVH4INb\nwDHwhKPJaOCiCXEs+/lsXr1lKinhAfxt2V5m/mMlj32xn5KaQe+IcQ6wHYgFJgBPCSHa+FRKKV+Q\nUqZLKdMjIiL6OsZeI9LDCi4u2OL+HKiL5BhdJLsEsSf5lfX4mgycOyaGzUfbOmA02h2s2F3AGWkR\nSKn5MAP93ns5u6xOZb4VihNAiWSA/J3w5mUQGAU3fASW4A4XbWh28LdlexkeFcjVU9QQ1Ip+TNIs\nOO+fcHil5qHsHJjlCEIIzkyL4O3bp/PRnbOYOTScp787xKyHvuEPH2VwrLTO2yH2Bt0Z5OlmYKnU\nOAQcAUb0UXxeJ9DXhMXHCEB6UkiL6QA2iw8WH2MHmeQGYmx+pEUHUljVSGV9S2G59mAJ1Q12bp6Z\nRGKYP4eLatlytIxLn1nHdwf6Z1a5ptHOef9awz+W7euxbX61p5Bv9hX22PYUiv6GEsnFB+D1S8Ac\nCDd8DIHRnS7+yvdHyC6r54ELRilHC0X/Z/KNcOYSbUS+T+8esELZxYSEYJ67fjJf33sml0yM491N\n2cx97FvufnsbGbmV3g6vJ+nOIE/HgAUAQogoYDian/2gQAjhzianD/EQyXpNshCCGJsf+VXtiOTK\nemKDLaRFaXXLB3WHDBef78zHZvFh1rBwhkYEcLi4hm3HtLImVw10f2Pp1hyqGuw95g1tdzi5f+nO\nAdW58aZ/b+TNDUe9HYaiHzG4VV55Frx2keYne8PHENy5S0VRVQNPf3OIs0ZGMWtYeN/EqFCcKnOX\naDX2296AT38+4IUyaJ2yHrpsHGt+M48fz0nh231FnP9/a7n2pfWsOlDcrx+Jd4duDvL0F2CmEGIX\nsBL4jZSyxDsRe4eoQK2kYpIuko0GgZ/P8ctetM2PgsoGlu/K58rnf3CfF/kVDcTYLKRGBgJwoPB4\n5z0pJV/vLeTsUVGYTQaGRQaQVVrLLv0mrLWg7g9IKXl1XRYAh4pq2mTGT4aNR8ooqWkiv6LtTcbp\nSHF1I9/tL+a7/af+JMDhlHyyI2+wdSYekAxekVyVrwnk5jq4/iMIH9blKo9+sZ8mh5PfnTeyDwJU\nKHoIIWDeb3Wh/PqgEcoAUUF+3H/uSNbdP5/7F4/gUFENN76ykcVPrmHp1hyaHadvO0gpl0kp06SU\nQ6WUf9OnPeca6ElKmSelXCilHCulHCOlfMO7Efc9rkzy0IgAbBYft0eyC5dI/mxXPhuOlFHVYKfZ\n4aSwuoG4YD/igi34m41ur2WA+mYHVQ12tzvG0IgAmh2Sb/YVAXCwqH03DBdOpyZY+7IM6IfMUg4X\n13LZpHgAdmSfemfeT3fmA1Ba23TCw8fXNdl5b3N2v7pZdbXJ0dJTH6r8gy3Z/PztbazqZunNy2uP\n8NDyniuDUfQcg1MkV+XD6xdDbQlc91+IHtPlKrtyKvlgaw43z0omObz9wUUUin6LWyj/WhPKnw3c\nGuX2CPLz4SdnDmXNffN59PJxOKXk3vd2MOfhb3n2u8NU1qnOTAOR9CEhTEkKwc/HSIzNjwDfloPM\nxtj8KKhqYLteKlFR10RhVQNSQkywBYNBkBoZwMGi4yK5Qj9XQvxdVnLa9aC6wY7RIDhUVIOjE+/u\nHTkV/PGT3Vz09Fq2tNMpsDdYdaAYH6PgN4uHIwRsP0WR3OxwsiIjH3+zVvPdXufHzvh8Zz73fbCz\nRYbe2+zIcYnkulPyXrc7nDz7nWa9WVzdvc7D72/O5qNtrbsUKDricHENP397W58M9jP4RHLBLnhp\nAVRkw9XvQHx6l6tIKfnTp7sJ9Tdz1/yuM84KRb9ECJj3O5jzK9j6Gnz2i0EllAHMJgM/Sk9gxT1n\n8MpN6QyNtPLwin1M/8dK/vBRBpkdeOIqTk9umpXM+3fMBCAh1J+wAN8W86NtFhxOSa4u8spqm8jT\nywdidUeM1KjAFmKuvK4JgGB/zR8/JeL4MOlzUsNptDvJKe84S7w3XxPcZpOBn725tU8eyW/ILGN8\nfDCRgX6kRgawTR9g5WTZkV1BeV0zP5qsZabz2+n82BmuzpKF7dSDnwjLd+X3SOkIHL9xaLRrTxJO\nhOqGZnct+ue78snSnxKU1XYtkhvtDg4V1VBY3dDrT7Y2Hik75WPfH/hydyGf7MjjcNGpZ/27YnCJ\n5INfwyuLQDrhlhWQPKdbqy3dmsvmo+X86pzhBPkpyzfFaYwQMP/3ulB+Fd67Hhr7Xw1lb2MwCOaP\niOLN26az/J45nD8uhnc3ZTP/8VXc+p9NrDl4+tctK1rypwtH88SVE1pMi9Vt4FyU1zWRX1nfYl5a\nVADF1Y2U12ri2JVJDtYzyTaLDxH6wCUXT9CsqjvLkO4rqCLA18RDl42jsKqRT3fknequdUpto51d\nuZVMSwkFtM6t27Mrun1+v7D6cBth5RKUF+r7m9tFJjmnvI79Bcf/z7hEaFEnmdaiqoZOt1tS08hP\n39zK6z9kdbjM94dKuOSZ77ssB3E6JTuyKxgS5g9AVkmd+zs84+6Ix788wKXPfk+zw8myXfnEBVsI\n8DVRWtuE3eHk+0MddwU4WFiD3SmR8uRvGv7y2R7e2nCs02WklPzy3e38fdnek/qOrli+K5/P9RKc\n3sZVEnOiTzBOhsEjkje/Am9dASHJcNtKiBnXrdUyciv53Ue7mJIUwhXpyvJNMQBwCeVFD8P+5fDS\n2VA2aEwP2jAyJohHfzSe75fM5+cLUtmeXcH1L29kwT9X8e/vjyhf2QFCbLClTalcdCuRXFbb7M4k\nx+iZ5DFxNgBW6jXHrkxyiP/xkVaHRlgJ8DUxf2QkQIvyjNbsza9ieHQgc9MiGB4VyItrMlsI1ia7\ns0dv0LYcLcfhlExLDgNgRHQQ5XXNlOmivzMKqxr4+7J9vLTmSIvp27IriAu2MCYuCCG6Fit/+nQP\nd7211WO7mjhub0hwF0uW7uKO17d0OL+0Rot/Z07HrjWbs8rZdqyC3V04jmSV1lLVYOei8bHuvwH+\nsWwf17y4vsvj8e3+IhqaneSU13OkpJaRMUGEBZgprWniqz2FXPvShg5dTzyn551EJ8j6Jgevrsvi\nvc3Z7c7fnVfJN/sKOVJSS25F/Ul9R1c0O5z8/qMMHlrROwK8Na7jk1epRPKp43TCl3+Az34JQ+fD\nLcvB1npgqvYpqWnk9tc2E+Jv5plrJ2M0iK5XUihOB4SA6XfA9UuhOh9enA+Z33k7Kq8SEejLvWen\nse7++fzvleMJ8vPhT5/uYfrfV/LbD3exr6B/WnspTp4YmyaEo/QOfhV1TeRV1BPkZ3LXL89ICWNU\nTBBPfXMQu8PZpiYZ4MYZSdyzIJUgPx9ibX4c7CCTLKVkX341I2MCEUJw25xk9hVU879fH0RKSX2T\ng3mPfcezq7o3nHxZbRMfbsvpVMRtOFKK0SCYrDt8uG4MWmdxf/fhLn72ZktRulrveLb5aFmL79iR\nXcGEhGB8TUYiAny7FMmHi2o4Vlbn3kaRnjEtqjoewzsbj7XYzqGiGvbkV3WYBa7Qb1Y6s3Z03dDs\nzOm8BnvNQS3Tu3hsDD5G4RZhO3IqKK1t6rS2OKuklqN6eUVmcQ1HS+tIDvcn1GqmrLaJbL305lCr\nUq6c8jpWHyhmT76nSD5x0bctuxy7U7I3vwp7O+UaT31ziDve2OqueS6saui0Zv5kWHuohNLaJrLL\n6tu01Y7sCirrmnE6JS+tyey0FKm7uDL9XT3B6AkGtkiur4D3b4B1/4L0W7UaZN/Abq3aZHfysze2\nUlrbxAvXp7sfpykUA4qUuXD7txAQDa9fCuufg0FeZuBrMnLJxHg+unMWn941m/PGxvDfLTksemIN\nlz27jvc2Z1PXNPBGMByMhPhrA4rMHBqOySAoq9XKLWI9RugTQnDPWalkldbx8fY8tzjzHG118dgY\nfnxGCgBJ4VaOlbUvBHIr6qlutDMiWhv48NJJ8Vw+OZ5/rTzIwyv28/bGY+RW1Hfba/m/W3L45bs7\n2NdJScCmI+WMibNh1UV/pH4ta/1o/4fDpWw92lJMrtbFY2FVIznlmiApqdE+T0jQBt2KDbZ0mp20\nO5wcK6uj0e6kXL/BcAl0l6DKq6hnydJdvPpDFqBZqOVV1ONwyhYi0hPXtvIqGzocXdOVLe8s29zs\ncPLimkwmJgYzIjqQhFB/jpbUUdto57AubPfr7ibltU388eMM9zmgtdFxB4sfDpfSaHeSFG4lzGqm\ntLbJXa+dVdKyfvbvy/Zy4783siKjgFEx2vlwMqJvc5ZWCtNod3K4uG2NbklNI012J8/onQntTtnj\no5F+vC0Xl2mMZ2lOaU0jlz27jse/2s/O3Er++vlebnhlY4uO0lJK901Td6hvclCgL98bWfHWDFyR\nnPkdPDsT9i2DhX+D8x4Ho6nL1Vw8+OluNmaV8cjl4xgbb+u9OBUKbxOaArd9BWmLYMVv4OO7oKn3\nO0ScDoyNt/Hoj8az/v4F/O7ckVTUNXHfBzuZ+reV3L90FztOoLZT0f8QQvDMdZO49+w0Qqxmyuua\nyK1oaCGSARaOiiIh1MKXewoor2vG32zE12Rsd5sxNkuLUfwyciu5993t7MqpdHfaG6mLIqNB8Mhl\n47hmWiLPrTrMY1/uB2h3FMD2cAmwtQfbr3m1O5zszK1gcuLxwVSigvRMskcWt8nu5GhZXYvOYw6n\nZO3BYreAcw3P7bJKG6+L5LhgS6cZ0Jzyeux65jKvoh6nU7pFsqvcwpUNdrVPfuXxdTrKFFfWHxeq\nLo/qP326mz9+nOGe7sok7+gkk/zpjjxyyuu5c+4whBAkhVnJKq1lb36VO1/gqjH/Zl8Rr/5wtEVd\n7+oDxSSG+hPoZ+Kb/VpJTnKYlVCrmdKaRvex9BTJ9U0Ovt1XjJRQUNXA5CEhhPj7dJlJrm9ytCmT\n2ZRV5h5qfXee1g4vrcnk/P9bAxwvS7E7pXt49hPpaHnve9t5eEXH9nR1TXa+2F3IpRPj8TEKtnk4\np6zYXYDdKflufzFr9ZuJ7LI67n1vu3uZ7/YXM+OhbzhSUktNo53VXfjYe96A5vZAVrorBp5Ibq6H\n5Us0D2Qff+3iP/MuEN0vlXhj/VHe2nCMO84cykUTuleaoVCc1vgGwpVvaF7K29+AZ2YM+vILT0Ks\nZn58Rgpf33smH9wxg0VjovlwWw4XPf09i59c0y8HkFB0j3nDI0kI9SfU3+zOJMe0qlUWQpASHkBu\nRT0Vdc0t6pFbE23zbfFI++PtuSzdlssFT61lyX93AjA8+vgTTYNB8OcLRzMnNZy6JgdxwZY2Iqaw\nqoHz/29NG8HoygavPljM5zvzueutrS1ssfYXVtPQ7GR8wvFEj+upaFF1Ax9uy+Hed7dzrKwWh955\nzCXqduVWUl7XzI/PSCbQ1+TOWG7PrsBoEIyJ08RzbLAfeZX1bmFjdzi57qUN7g6JRzx8h/MrksrK\nqgAAIABJREFUGyitbXK3jUssHxfJWtbYUwjt6iALXO6RjczQl/lqTyFvb8qmplF70uMSlJnFtfzz\ny/1c//KGNgLszQ3HSIsKYP4IrZ58SJg/R0vr3J0TfU0G9+/b5Zf93uYcNh4pw+mU/HC4lDmp4aSE\nW8nUM7lJ4VbCAnwpr2sizyWSPdph9cFi6psd3Do7GYBx8TZi2znurfn7sr1c/PT3Ldp669FyLhwf\ni5+PgYzcKgqrGnj8ywNk5FZR22intLaJ4VHa+Xbd9CEA5HczYy2l5IuMAt7fnN2hLd6O7Erqmx2c\nPz6GUbE2th49nkn+bIfWke9YWR3vbc5hVEwQ9yxIZeW+IneWfk9+FQ6nZENmKc99d5gbXtnIra9u\n7tCW09WOw6MCVSb5hMndCs+fARueham3w09WQ9zkE9rEhsxSHvxkN/OGR/Drc4b3UqAKRT/EYID5\nv4Obl4PBpN1ofnyXVrakADSxlJ4UymM/Gs/G353F3y4ZQ5DFx93JS3H6EmL1Ia+igYq65jaZZDhe\nVlBR14TN0rHLUbTNgt0pKdUfaWcW15ISbuXX5wxnVGwQl0+Ob+PXbDIaeO66ybx12zQunBBLYVVD\nC1Hy/uZsMnKreGVtyw50LpG88UgZv/9oF5/tzOfRFfvd83dka+JxYsLxTLKfj5Fgfx8Kqxr5fGcB\nS7flsj7zuF+zK5v57qZszEYDZ6ZFMiExmC26+Pl2fxFj4mz4m7V9iLFZaGg+Xkqx8UgZaw+V8Pxq\n7fG+ZwY1v7LeHXNCqIWiqkaklGTo5SXF1Y1aOUeZFkNKuNWdJW5odrAhs9S9rYq6ZnyMgmR9mYZm\nB7kV9TTZne6BXcprm9z15v/65hBrDpaQ2ars4XBxDVOSQjHofY5mDg2nvtnBs98dJiLQl/EJwW5x\nvL+wmpQIK1FBvry4JpNjZXXUNjkYF28jSe8U6msyEB3kR5jVTLNDclgfXCbLY/CYLzIKCPb3Ycni\nEaz4xRwunhinn1+di9cfMks5VlbnLjfYk19FbZODaXrdfEZuJY+s2E+9XsedX1lPZX0zi8dGs27J\nfK6ckqBP7564zK2op7bJQUlNE3s76JNxSO+kOjI6iIkJwezMqcTucFJU3cCGI6VcNEHrDHmsrI45\nqeFcOSURk0HwzkbNjcM1qM7WY+V8f7iE8ABfVh0odp8/rXGdTzOGhvWJbd7AEMkOO6x6BF4+Gxpr\n4PoP4dxHwex/Qps5VFTDz97cSmKYP09ePVF11FMMTobMhJ9+D7N/Cdvfgqenwb7PvR1VvyPIz4dr\npw3hvZ/MaCN6FKcfoVaz2+4rNtivzfz4EIvmo1zZQIi1Y5Eco5czuIRIZkktw6MDuXPeMF6/dRqP\n/Wh8u+tZfU3MHBZOrM1PqxvVPXallHywJQfQPHg9M2wFVQ2EB5hptDupqG9m3vAIXlp7xC0md2RX\nEOLvQ0JoS9EfGahluzNLNAH37qbjzgh5lfXkVtTzwZZsrpySQKjVzMyh4ewrqGbZrnwycqu4UHeB\nAEjWB1N5c/1RAD7bpWUPM3Kr2F9QTVZJLVazER+jIK+iwV1iMTbORn2zg5pGOxm5le5SgH351WSX\n12E0CBaOjuZgUQ0NzQ7e3HCMK19Yz4qMAkDruBfsb2ZsnI2M3Eqyy+rc5RErMrQYyuqamJumZYhd\nln6epSlVDc1U1DWTGHpcK5w1MpIJCcGU1jYxJjaItKgADhbWIKVkf0E14+ODmZ4SRkZupbtWeXh0\nkNs5JSnMisEgCLVqTxtqGu34m42U1TZRWd9MZV0zX+0t5KyRUfgYDYyIDsLHaCDW5tdpTXJVQ7M7\n++oqeVmRUYDRIJg5NIwxcTY2ZpXx3605jNdLRPcXaMuHWc3EBlsI8ffB12Rw2xx2hWcHVFe7rcgo\n4OKnv+eJrw9Q3+TgYFENgb4mooJ8mTQkhPpmB5uyylm6NRenhLvmDXO376xh4UQE+rJwdBQfbMmh\n0e5wPzVYd7iUnTmVXDklnvHxNtZ73BDVNNrd531WaR2hVjMjYwJbPPnoLQaGSBYCDq2EURfDz9Zp\nLhYnyKoDxVzyzPcAvHhDuvJDVgxufCxw1oPw45VgjYB3roF3r4eSg96OTKHoFYL9zTTpWSmX64Un\nLuF8qKjaPZBIe7jcI/IrG2iya53WUiK6P0prtP7dh4tqmfXQN1zz4gaySuu4aWYSjXYnH23XXAq0\nDk+NnDs2Bn+zkUsmxvHsdZMJDzDz4hrN0nFHTgXjE4JbDMUNWl1yXmW9O4u3K7fS7daRV9HAc3on\nrzvmDgXgqikJWM1G7n1vO0LABeNi3Ns6MzWCiyfE8vhXB3hu1WFWZBQwa1gYJoNg6dYcjpTWkRRu\nJSrIj/zKencttMtaLyO3iqLqRi6bpJU27s2vIrusjhibHxMTg3E4JbtyK1mnew0/8HGGW9wGW3wY\nG2cjr7LBnekeF2/jm31FlNY00tDsJDnCypNXTeDdn8wgIdTCWg/P4mxdoCV4iGQhBEsWjwA0IZ8W\nFUh1o50DhTXkVzaQFhXImFgb+ZUN/HC4FCE0L223SA7XtuUSyQDpSZpH9dHSWh5asZe6Jgc3z0pq\ncUxigy1UN9g7tJzcmV3pvgnYkVOJlJKPt+cxe1g44QG+zBsRSajVzIMXjOKhyzSLW5cjT6jV171v\nMTa/Fplku8PJpc98z7lPruHPn+6h2uP7XRn0uGALaw6W4HBKHlq+l0NFNTzx9UGeW3WYA4XVDIsK\nQAjBWSMjiQj05ZEv9vHcqsOckRZBalQg80dE4m82MkVvhyvSEyiva2btwRK3SM4p1zpqzkgJZ2py\nGDtzKqlv0jLi932wgx89vw4pJVkltQwJ83c/7eltr+SBIZINRs3K6vKXwRLS9fIeSKnZktz8743E\nh/jz8V2zGOoxgpJCMaiJnai5X8z/PRz6Gp6eCh/+dFD7KisGJqEewjeuvXILXbw2OyTBnZRbuOqZ\nC6sayC6vw+GUpIR3/5riWv+rPYXkVtTzQ2YpFh8jvzpnOGPjbLy6LsttRdfkcJIUZmX5PXP4+yVj\n8fMxctWURFbuK+JAYTUHCrXMZ2siA/3Yl1+N3Snd3XVGx9oItZrJKa/n0515nDc2xt0OIVYzN85M\noqHZycyhYUQGHc+0GwyCR380nsVjonlo+T7Kapu4fvoQ5g6P5P0tOezNryIp3EqszUJ+RYPbI9nV\nIfBbvbPbnLQIooJ82ZtfxbGyOhJC/JmeEoaPUfBFRgEbj5QxKTGY4ppG/r02i/K6JkL8zW6x/Zk+\nkMXNs7Q4XbZuof5mLpoQR0KoP7OHRbD+cCn1TQ4a7Q6y9bKOhJCWT52np4Tx8o3p3DwrmTS9nvdD\n3UJteHQAo/V67E925JEY6o+/2URSmEska+/hHqM7zkjRPKrfWH+Utzdmc+vsZEbHtjQEcIm+fI86\n2wc/2c1XewoB2J6t3QQkh1vZkV3B1mPl5FbUu8sZ5g2PZOsfzuamWclE68fH1RHSU7DH2LTa56Ol\ntVTUNbE3v5qtxyqQwH/WHWHRE2vcNw/7C6uJCvJl8ZhoNmaV8cy3h8gqrePRy8cxISGYNQeLOVRU\nQ1qk1kb+ZhN3zx/GtmMVVNQ18+uFWsnqr84Zzqd3z8aiD2M+NTkUo0GwMauM/Mp60nV7Qh+jZlU4\nLSUUu1OyVXfK2JlTyYHCGv67NZfNR7XRI90iuZe9kgeGSAYwd/9O3UWj3cF9H+zkr5/vZeGoaD64\nYwbxISdWoqFQDHiMPnDGr+GeHTD9Z7B7KfxfulavXH7U29EpFD1CiC4khDjuAOFJXMhx4dxZx71Q\nqxmz0UB+ZYO7I9eJZZK17/5yj1ZW8NZt03jjtqkE+Jq4a/4wMktqWbo1112XGm3zY0iYFT8fTYBc\nPS0RAVz9wnqcUhMkrYkM8nW7R8weFg5oA6LE2PxYfaCYirpmZqdGtFjntjkpJIX5c/30pDbb8zEa\neObaSfzh/FHMGx7B3OGR/M/CNJrtToqrG0kOsxKjd/ArrG4gzGp2C/CPt+diMghGxgQxMiaI7TkV\nHCurJyHUgs3iw8yh4by54RjVjXZunJlEcpiV/YVVVNY3Y/P3cQvWdYdLCPH3cddfu6zIgj2s+mYP\nC6e60c6kv3zFdS9tcItBz3ILFwtGRhFiNTMu3kZ4gC8vr9USA2lRgW6BW+bRKS41KoDEUH+3IPYU\npq5j8N7mHFIjA/jFWaltvm+E3pnz851ah8eCygb+sy7LXbu7PbuCoRFWZg3Tsqxvrj+Gr8nAwtHR\nbbYV7O+Dj1G4M8lhAZ4i2Y/M4hrO/9da/vCx5uIF8MpN6bx/x0wKqhrc5TcHC2tIiwrk8vR4/M1G\nHv/qACkRVs4ZHc2sYWFsz66gpKaJ1KjjN4FXTUkkJdzKBeNj3c5gAb6mFslHf7OJtKhAlu8qwCnh\ngvGxGIRWO28xG0kfEoJBwIYjZdQ3OdxlKPcv1Tq+3jYn2X3T2tud9waOSD5BiqsbuebFDby/JYef\nL0jlmWsnuX0kFQpFOwREwjl/08Ty1B/Dznfh/ybBR3dCzuZB76+sOL0J1euMIwJ8MZvaXhqjgvxw\ndVPxFF6tEUIQZfOloLKeTL2GNOUEnk6G+msiO6e8nlibHzOHhTN5iCayFo6KYnxCME98fcAt8Fwd\n01zEBVs4b1wszQ4nf79kLDOHhrXdFw/f/6umJLpjjA22uAXJtFbiOtRq5rtfz2PRmLaizLXft85O\n5t83T8XPx8jImCBevDGdQF8Tk4Zomb/CqgYKKhuIDPIjMtCVcW/khhlJBPiaOH9cLJnFtZTUNLqF\n66Ix0e6OaDNSwjQf49I6PZPsQ5CfD8nhVpxSy+LGh1gwGQRbj1W443YxOzWc0bFBxAb7sfloOXvy\nqwj0M7XwvG6Nv9nEfYuG0+yQWM1G4oI18e6Kz+VU4m82sfq+ecwdHtnme5PDrQyNsDI6Noh3bp/u\n7vToSWpUIOePi+GFNZkUVDaw7rCWCd+Vq5VWbM+uYEJCCOPjg6lptLN0Wy7XThvSbn8IIQQRAb5u\nb+swz0xysB/ldc1UN9r5ek8hqw8UEx9iIcZmYfKQECYmBLP6YDFOp+RgUTWpkYGMiA5i5b1ncvsZ\nKfzt4rEYDIJZw8Jx9S1NjTru1mI2GVh2zxz+eUX7tfcuJiTY3KUWI2OC+MmZQ7lFd/sI9PNhdKyN\nDZmlHC6uQUpICvOn2SG5fHI88SH+WMxGwqxm93DxvcWgE8lOp+STHXlc9NRadudV8tQ1E7n37DR3\nz1aFQtEFgdGw+GH4+XaYfJOWWX5pgeZLvv5ZqCvrchMKRX/DlR3uyKnEx2hwZ5g7q0kGiAmyuDPJ\n4QHmTt0wWmMwaCIbaOPRL4TgFwtSyats4L3NWmc+l9j05HHdfeWaaYlt6pHheKY8xN+HhaOj+MVZ\nqZw/7nh5RVywpUWd7skyPSWM7X9cyPwRUcTa/Gh2aFZfCSEWgiwmzCYDoVYz9yzQMquXT47nN4u0\neuBhkdqNxdmjojAILRsfGeTHkDB/jpXWaTXJ+nFwlVwkh1kxGQ3Eh1jcdnIhHgLRZvHh85/P4S8X\nj0FK+GJ3QZtSi/a4fFI8ExODmZB4vL57dKyWwfa08/PEz8eI1WzEbDIQ4u/D0p/O4sOfzSIsoOOB\nyX6zaAROJ/zzq/18f0jruFZU3cgPh0spqWkiPSmEaclh+JoM3DQzid+fN7LDbUXox1iIluerq+Z9\nRHQg9c0OVh0obvG0YU5qBLtyK9mRU0FDs5Ph0dpxCAvw5bfnjmSGftM1KTEEX/1mMjWy5U2gn48R\nH2Pn8tKzDCgx1J/fLBrR4gZsWnIo245VuN1N/njhaOaPiOTu+cez8Ot/u4Dfnz+q0+85VQZN6lRK\nyXcHinl0xX725FcxPCqQ569PVwOFKBQniy1OG6RnwR81obz1NVixBL56AEZeAGOvgOQzTthlRqHw\nBi6RHNeOs4ULl5dtSCeZR9BKIHbkVOCUJ1aP7CImyEJ2WT3j2qknnjUsnABfk7uWNzKorehqLxPu\niWudlIgAfIwGfnFWGnC8c+K0lLYlGieLyyXK1RnSYjayZPEIhBDcOXcYY+KCWmRyfzp3KOeMjmJI\n2PHa3htmJDEkTPs/khjqT7Xug+zK6I+Ls/HpjjyPznNWt+VaaDs3NBMTQvAxCuqaHO2WWrTGYBC8\nddt0JMeflo2Js7E8o8BdJtEeYQG+CKHd3HSWrXaREOrPjTOH8NLaIwT4moi1+ZFX2cCTK7UO02em\nRRAbbGHHHxe6y2s6wjWyYoi/uYVT17TkUCYlBvPkVRO58Km1lNc1MzXp+PE+Iy2c//36AP/z/g6E\ngImJ7ffz8vMxMlUXsq19xbuDazAas8ngjtWTM9I0p5Y3Nxx1O3jM07P0LroS4j3BoBDJm7PKeGTF\nfjZmlZEY6s8TV07ggvGxyuJNoegJ/IK0jPLkm6AgQxPLO9+BjP+CyU8TyqkLIe0cCE70drQKRbu4\nHo+352zhIi7Ywpaj5Z2WW2jb8GN5Rj0FlQ1ckZ5wwrHE6GJ1XDtJHLPJwOxh4azYXUCo1dzhyH+d\n4co+p4S3rJV2dYaanty2RONUmZgYzIIRkdy7MM1dfnJPO7W50LY85cELR7s/u8QzQLBFO2audnKt\np3WiK8YgIKidLL7FbGRMnI1txyra2ON1hKvTmYurpyZis/h02tE/2uaH+QSF3F3zUnlvcw6V9c38\nfH4qf1++lw1HyhgRHeg+Pl0JZDgukj3LPkCrqV76s1kALBwVzbubs5nikUkeFx+MzeJDZnEtPzkj\nxd1xsT1+s2gE2WV17T6t6IrUyAAsPkbiQiztPsmfmhyKr0kbICUl3HpS53lPMGBFcmVdM1/vLeSj\n7bmsOVhCRKAvf7loNFdOSezyLluhUJwk0WPg3Edg4V8gay0c/BIOrNDel/0KIkdB0hyInwLxkyEk\n+YRGw1QoeouIQF8C/Uxu14X2cImUrsotovXSgohAM3fPH3bCsbiE+pjY9p90zhsRwYrdBe12MOwO\nkUG+BPv7MCGxZaZ6ekoY542L4exRUSe13c4IC/Dl5ZumnPJ2XBllOJ5JnpocyrPXTuIsPe4kfZng\nVllUT6Ymheoi+eSedIVaze4R7DrikcvGYTjB/282fx/uPTuNv3y2h0Vjonln0zEOF9cyb0Rk1yt7\n4BpZMcza8bl61/xhpEYFtLhZMhoEF4yPYW9+Nf+zsPMB1cbE2dylLieKyWjgzLSIDkuR/HyMTE8J\nY9WBYoZGes9xbECJ5OLqRr7aU8jyjHx+OFyK3SmJsflx36Lh3DQzqd1ieYVC0QuYfGHYAu216CHN\nX/ngF5pY3vY6bHxeW84/TBPMcekQNQrCh0NIEhjVb1XRt/j5GFl//wIsnWTpRscGYTUb23087MmY\nOBuBfiaevXZSC7u07nLDjCGMi7e1qKf1xNU5rHWnve7iazKy9jfz8W+1r+EBvjx9zaST2mZf4Vke\n4RLJQggWjz3u3TxEF32dlcVMSwnl+dWZ7hKN3iDpJLd9w4whnDcuhvAAX8bFB2siefiJiWTX0wJP\nZ4vWJIT6c9uclDbT/3rxWKSUJ5UhPhGevW5Sp99xZloEqw4Ut6l57ku6dSUSQiwCngSMwEtSyoda\nzRf6/HOBOuAmKeXW7qzbE0gpueGVjXx/qETr4RqmHfhFY6IZH2/r9QOtUCg6QQiISNNeM+/WRsgs\n3gs5mzRXjJxNWrbZhcEHwoZCeBqEp4ItAWzxEBQLQXHgZ1PZZ0Wv0JXD0fnjYpg7PILALgabmpIU\nyo4HFp50h/DYYEu7Q2O7iAry48LxsUxIaFuz3F1O11Ei/XyMRAX5UljV6C63aE2yXpLRutTAk3nD\nI/n3TVOYNTS8V+I8FYQQbp/lc8fGUFjVwKTEEzvWHZVbnEgMvU1X3zF/RCR/X7a33dr8vqLLX4kQ\nwgg8DZwN5ACbhBCfSCn3eCy2GEjVX9OAZ4Fp3Vz3lBFCkBoZyKTEEBaNiWZEdKASxgpFf8Voguix\n2iv9Fm1aQ6WWbS45oL2KD0DRXm04bOloub45QBPM/uHgH6plo63h2rslBHwDPV5B2rvZCiaLylAr\nTgkhRJcC2UVvOyb96+qJvbr9/syQUCuFVY0dDg8eF2LBaBCd+lkLIU64hMEbnD0q6qTKX1ydM12j\n7Z2OJIVbWbdkvrt0xBt054oxFTgkpcwEEEK8A1wEeArdi4DXpJQSWC+ECBZCxABJ3Vi3R3jggt61\nAVEoFL2Inw3i07WXJw471BRAVR5U5kBVLlTmQnWeZjVXlgnZG6GutK2Ybg+Djzbktutl9AWjGUxm\n7d3oo70bfLSRPA2mVi8DCAMIozZfGLW/Z/9C85FWKBS9TmKYPxuzyjrMJPsYDcwcGuZ2UBiMxNgs\nCAGxJ+E80Z84mXKlnqQ7IjkOyPb4OwctW9zVMnHdXBcAIcTtwO0AiYmqB7xCoUDL/NritVfC1I6X\nk1LLRteXQWMNNFZ7vCqhqQ7sDdBcr73srvdGcDSDowkc+ufGGnDawenQ3z1e0qlNlw79XWqf029R\nIlmh6CPmDY8kv7K+jeOEJ6/f2q7UGDREBPry35/O7LQjqqJr+s2zRynlC8ALAOnp6WroLoVC0X2E\nAEuw9lIoFAOa88bFcN64mK4XHORM6sDjWNF9uiOScwFPo8d4fVp3lvHpxroKhUKhUCgUCkW/ojuG\nwZuAVCFEshDCDFwFfNJqmU+AG4TGdKBSSpnfzXUVCoVCoVAoFIp+RZeZZCmlXQhxF/AFmo3bK1LK\n3UKIO/T5zwHL0OzfDqFZwN3c2bq9sicKhUKhUCgUCkUP0a2aZCnlMjQh7DntOY/PErizu+sqFAqF\n4tTojge9EGIu8ARa6VuJlPLMPg1SoVAoTmP6Tcc9hUKhUHSP7njQCyGCgWeARVLKY0IIZb+hUCgU\nJ0B3apIVCoVC0b9w+9dLKZsAlwe9J9cAS6WUxwCklEV9HKNCoVCc1iiRrFAoFKcfHXnTe5IGhAgh\nvhNCbBFC3NBn0SkUCsUAQJVbKBQKxcDEBEwGFgAW4AchxHop5QHPhdRATgqFQtE+KpOsUCgUpx/d\n8a/PAb6QUtZKKUuA1cD41huSUr4gpUyXUqZHRET0WsAKhUJxuiE0Y4r+hRCiGDjq7Ti6SThQ4u0g\nehm1jwODwbCPcHrt5xAp5QkrUyGECTiAliXORfOkv8bTYlMIMRJ4CjgHMAMbgauklBmdbPdk//f2\ntzZX8XSOiqdzVDydMxDi6db/3n5ZbnEyFw1vIYTYLKVM93YcvYnax4HBYNhHGBz72R3/einlXiHE\nCmAn4ESzietQIOvrndT/3v7W5iqezlHxdI6Kp3MGUzz9UiQrFAqFonO68q/X/34UeLQv41IoFIqB\ngqpJVigUCoVCoVAoWqFE8qnzgrcD6APUPg4MBsM+wuDZz/5Ef2tzFU/nqHg6R8XTOYMmnn7ZcU+h\nUCgUCoVCofAmKpOsUCgUCoVCoVC0QonkHkII8T9CCCmECPd2LL2BEOJRIcQ+IcROIcSHQohgb8fU\nUwghFgkh9gshDgkhlng7np5GCJEghPhWCLFHCLFbCHGPt2PqLYQQRiHENiHEZ96OZbDg7d9PR+e3\nEOJBIUSuEGK7/jq3D2PKEkLs0r93sz4tVAjxlRDioP4e0kexDPdog+1CiCohxC/6sn2EEK8IIYqE\nEBke0zpsDyHE/fr5tF8IcU4fxdPuNU4IkSSEqPdop+c63nKPxtPh8fFS+7zrEUuWEGK7Pr1X26eT\n33ffnD9SSvU6xReaqf8XaP6i4d6Op5f2cSFg0j8/DDzs7Zh6aL+MwGEgBc1Ldgcwyttx9fA+xgCT\n9M+BaP66A2ofPfb1XuAt4DNvxzIYXv3h99PR+Q08CPzKS+2S1fpaADwCLNE/L/HG/1D9eBUAQ/qy\nfYAzgElARlftoR+7HYAvkKyfX8Y+iKfdaxyQ5LlcH7ZPu8fHW+3Tav7jwAN90T6d/L775PxRmeSe\n4X+B+4ABW+AtpfxSSmnX/1yPNsLXQGAqcEhKmSmlbALeAS7yckw9ipQyX0q5Vf9cDewF4rwbVc8j\nhIgHzgNe8nYsgwiv/35Oo/P7IuBV/fOrwMVeiGEBcFhK2aeDdUkpVwNlrSZ31B4XAe9IKRullEeA\nQ2jnWa/G481rXAft0xFeaR8XQggBXAG83ZPf2UksHf2+++T8USL5FBFCXATkSil3eDuWPuQWYLm3\ng+gh4oBsj79z6J8X2B5BCJEETAQ2eDeSXuEJtJtVp7cDGUT0q99PO+f33frj81f6qrxBRwJfCyG2\nCCFu16dFSSnz9c8FQFQfxuPiKlqKG2+1D3TcHv3hnGp9jUvWSwlWCSHm9GEc7R0fb7fPHKBQSnnQ\nY1qftE+r33efnD9KJHcDIcTXQoiMdl4XAb8FHvB2jD1BF/vpWuZ3gB1403uRKk4GIUQA8F/gF1LK\nKm/H05MIIc4HiqSUW7wdi8I7tHN+P4tWBjIByEd7RNxXzJZSTgAWA3cKIc7wnCm158J9+uRRCGEG\nLgTe1yd5s31a4I326Ih2rnH5QKJ+PO8F3hJCBPVBKP3m+LTialreaPVJ+3R2/erN80eNuNcNpJRn\ntTddCDEWreZlh/YEgnhgqxBiqpSyoA9D7BE62k8XQoibgPOBBfpJORDIRaspdxGvTxtQCCF80P7B\nvCmlXOrteHqBWcCFeucWPyBICPGGlPI6L8c10OkXv5/2zm8pZaHH/BeBPuvMKaXM1d+LhBAfoj3u\nLRRCxEgp84UQMUBRX8WjsxjY6moXb7aPTkft4bVzqr1rnJSyEWjUP28RQhwG0oDNvRlLJ8fHm+1j\nAi4FJrum9UX7dHD96pPzR2WSTwEp5S4pZaSUMklKmYSW1p90OgrkrhBCLEJ7lH2hlLLO2/H0IJuA\nVCFEsp5puQr4xMsx9Sh6DdnLwF4p5T+9HU9vIKW8X0oZr/8OrwK+UQK5T/D676ej81vbKHU4AAAB\no0lEQVS/cLq4BMhovW4vxWMVQgS6PqN1CMtAa5cb9cVuBD7ui3g8aJEB9Fb7eNBRe3wCXCWE8BVC\nJAOpwMbeDqaja5wQIkIIYdQ/p+jxZPZBPB0dH6+0j85ZwD4pZY5HnL3aPp1cv/rk/FGZZEV3eQqt\nt+hXetZ8vZTyDu+GdOpIKe1CiLvQ3EmMwCtSyt1eDqunmQVcD+wSum0P8Fsp5TIvxqQYAPST30+7\n5zdwtRBiAtpj2CzgJ30UTxTwof5/0gS8JaVcIYTYBLwnhLgVzQnpij6KxyXWz6ZlGzzSV+0jhHgb\nmAuECyFygD8CD9FOe0gpdwsh3gP2oJU93CmldPRBPPfT/jXuDODPQohmtP4Od0gpu9vJ7lTimdve\n8fFW+0gpX6ZtTTv0fvt09Pvuk/NHjbinUCgUCoVCoVC0QpVbKBQKhUKhUCgUrVAiWaFQKBQKhUKh\naIUSyQqFQqFQKBQKRSuUSFYoFAqFQqFQKFqhRLJCoVAoFAqFQtEKJZIVCoVCoVAoFIpWKJGsUCgU\nCoVCoVC0QolkhUKhUCgUCoWiFf8PCqulZBwhdEkAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# SGD using the Reparameterization Trick for estimating cross entropy gradient\n", "def dfdx(x):\n", " # x is allowed to be a length N vec of data points\n", " c1 = norm(mu1, sig1)\n", " c2 = norm(mu2, sig2)\n", " px = p_x(x)\n", " return -1/ px * (pi1 * c1.pdf(x) * (x-mu1) * sig1**(-2) + pi2 * c2.pdf(x) * (x-mu2) * sig2**(-2))\n", "\n", "def g(eps, mu, sigma):\n", " return mu + sigma * eps\n", "\n", "def est_grad_reparam(mu, sigma, num=1000):\n", " es = np.random.normal(0, 1, num) # sample \"underlying randomness\" epsilon\n", " xs = g(es, mu, sigma) # the corresponding x as the output of sampling process\n", " grad_g = np.vstack([np.ones(num), es]) # 2 by num\n", " df = dfdx(xs)\n", " grad_h_q = [0, 1 / sigma] # grad H[q]; 0 wrt mu\n", " grad = -np.mean(df * grad_g, axis=1) - grad_h_q # approx grad of obj\n", "\n", " # estimate the optimization objective (KL_q_p); this is in the original parameterization, same code as before\n", " # in REINFORCE\n", " h_q = 0.5 * np.log(2 * np.pi * sigma ** 2)\n", " neg_log_p_x = -np.log(p_x(xs))\n", " obj = -h_q + np.mean(neg_log_p_x)\n", " return grad, obj\n", "\n", "\n", "# let's do sgd\n", "# intentionally bad initialization\n", "mu = -2\n", "sigma = 3\n", "T = 200\n", "\n", "eta = 0.05 # step size\n", "objs = [] # keep a record\n", "for t in range(T):\n", " grad, obj = est_grad_reparam(mu, sigma)\n", " if t % 20 == 0:\n", " print('t =', t, 'D(q||p) =', obj)\n", " objs.append(obj)\n", " mu, sigma = [mu, sigma] - eta * grad\n", "\n", "# plot results\n", "xs = np.linspace(-5, 5)\n", "plt.figure(figsize=(10, 4))\n", "plt.subplot(1, 2, 1)\n", "\n", "plt.plot(xs, p_x(xs), label='$p(x)$') # target p(x)\n", "plt.plot(xs, norm.pdf(xs, mu, sigma), label='$q(x)$') # I-projection q(x)\n", "plt.legend(loc='best')\n", "plt.title('Approximate I-Projection')\n", "\n", "plt.subplot(1, 2, 2)\n", "plt.plot(objs, label='$D(q||p)$')\n", "plt.legend(loc='best')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On this low-dimensional toy problem, vanilla reparameterization trick and REINFORCE (score function trick) seem to perform similarly (although generally REINFORCE is known to have higher variance and much work has gone into variance reduction for these methods; see Appendix D of [the other variational autoencoder paper](https://arxiv.org/pdf/1401.4082.pdf) for a discussion). [This blog post](https://gabrielhuang.gitbooks.io/machine-learning/content/reparametrization-trick.html) gives a concise summary of methods related to reparameterization trick." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.3" } }, "nbformat": 4, "nbformat_minor": 0 }