{ "metadata": { "name": "ch7" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "code", "collapsed": false, "input": [ "%pylab inline" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "\n", "Welcome to pylab, a matplotlib-based Python environment [backend: module://IPython.zmq.pylab.backend_inline].\n", "For more information, type 'help(pylab)'.\n" ] } ], "prompt_number": 1 }, { "cell_type": "code", "collapsed": false, "input": [ "import numpy as np\n", "from pandas import *\n", "import matplotlib.pyplot as plt\n", "from pandas import *\n", "import scipy.optimize as opt\n", "import statsmodels.api as sm\n", "from statsmodels.formula.api import ols\n", "import os\n", "import random" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 10 }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Ridge regression by explicit SSE minimization." ] }, { "cell_type": "code", "collapsed": false, "input": [ "heights_weights = read_csv('data/01_heights_weights_genders.csv')" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 11 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The OLS estimate" ] }, { "cell_type": "code", "collapsed": false, "input": [ "ols_fit = ols('Weight ~ Height', df = heights_weights).fit()\n", "ols_sse = ols_fit.mse_resid * (ols_fit.df_resid) \n", "\n", "print np.round(ols_fit.params, 3)\n", "print 'MSE %i' % round(ols_sse)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Intercept -350.737\n", "Height 7.717\n", "MSE 1492935\n" ] } ], "prompt_number": 24 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Set up the ridge regression. \n", "\n", "We'll explicitly set up the ridge SSE function to minimize. We'll also provide explicit functions for the gradient and hessian of the SSE, since they're pretty straightforward.\n", "\n", "We'll try a couple of different optimization algorithms, some use the gradient and hessians, other (e.g., Nelder-Mead) don't use either." ] }, { "cell_type": "code", "collapsed": false, "input": [ "y = heights_weights['Weight'].values\n", "Xmat = sm.add_constant(heights_weights['Height'], prepend = True)\n", "\n", "def ridge_error(params, y, Xmat, lam):\n", " '''\n", " Compute SSE of the ridge regression.\n", " This is the normal regression SSE, plus the\n", " L2 cost of the parameters.\n", " '''\n", " predicted = np.dot(Xmat, params)\n", " sse = ((y - predicted) ** 2).sum()\n", " sse += lam * (params ** 2).sum()\n", " \n", " return sse\n", "\n", "def ridge_grad(params, y, Xmat, lam):\n", " '''\n", " The gradiant of the ridge regression SSE.\n", " '''\n", " grad = np.dot(np.dot(Xmat.T, Xmat), params) - np.dot(Xmat.T, y)\n", " grad += lam * params\n", " grad *= 2\n", " return grad\n", "\n", "def ridge_hess(params, y, Xmat, lam):\n", " ''' \n", " The hessian of the ridge regression SSE.\n", " '''\n", " return np.dot(Xmat.T, Xmat) + np.eye(2) * lam" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 25 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Minimize the ridge SSE using different algorithms" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Set the starting parameters (constant, coefficient)." ] }, { "cell_type": "code", "collapsed": false, "input": [ "params0 = np.array([0.0, 0.0])" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 26 }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Nelder-Mead Simplex:** no gradient required. This is the default of `optim` in R." ] }, { "cell_type": "code", "collapsed": false, "input": [ "\n", "ridge_fit = opt.fmin(ridge_error, params0, args = (y, Xmat, 1))\n", "print 'Solution: a = %8.3f, b = %8.3f ' % tuple(ridge_fit)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Optimization terminated successfully.\n", " Current function value: 1612442.197636\n", " Iterations: 117\n", " Function evaluations: 221\n", "Solution: a = -340.565, b = 7.565 \n" ] } ], "prompt_number": 28 }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Newton conjugate gradient:** requires gradient, hessian optional." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First without the hessian." ] }, { "cell_type": "code", "collapsed": false, "input": [ "ridge_fit = opt.fmin_ncg(ridge_error, params0, fprime = ridge_grad, args = (y, Xmat, 1))\n", "print 'Solution: a = %8.3f, b = %8.3f ' % tuple(ridge_fit)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Optimization terminated successfully.\n", " Current function value: 1612442.197636\n", " Iterations: 3\n", " Function evaluations: 4\n", " Gradient evaluations: 11\n", " Hessian evaluations: 0\n", "Solution: a = -340.565, b = 7.565 \n" ] } ], "prompt_number": 34 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, with the hessian. This shaves a little bit of time off." ] }, { "cell_type": "code", "collapsed": false, "input": [ "ridge_fit = opt.fmin_ncg(ridge_error, params0, fprime = ridge_grad, \n", " fhess = ridge_hess, args = (y, Xmat, 1))\n", "print 'Solution: a = %8.3f, b = %8.3f ' % tuple(ridge_fit)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Optimization terminated successfully.\n", " Current function value: 1612442.197636\n", " Iterations: 3\n", " Function evaluations: 7\n", " Gradient evaluations: 3\n", " Hessian evaluations: 3\n", "Solution: a = -340.565, b = 7.565 \n" ] } ], "prompt_number": 35 }, { "cell_type": "markdown", "metadata": {}, "source": [ "**BFGS:** Uses the gradient, no hessian" ] }, { "cell_type": "code", "collapsed": false, "input": [ "ridge_fit = opt.fmin_bfgs(ridge_error, params0, fprime = ridge_grad, \n", " args = (y, Xmat, 1))\n", "print 'Solution: ', ridge_fit" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Optimization terminated successfully.\n", " Current function value: 1612442.197636\n", " Iterations: 15\n", " Function evaluations: 290\n", " Gradient evaluations: 243\n", "Solution: [-340.565481 7.5645375]\n" ] } ], "prompt_number": 36 }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Deciphering text with the Metropolis-Hastings Algorithm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The lexical database is a collection of words and their frequency on Wikipedia." ] }, { "cell_type": "code", "collapsed": false, "input": [ "lexical_database = read_csv('data/lexical_database.csv', index_col = 0, \n", " header = None, skiprows = 1, squeeze = True)\n", "lexical_database.index.name = 'word'" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 37 }, { "cell_type": "code", "collapsed": false, "input": [ "letters = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', \n", " 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', \n", " 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', \n", " 'y', 'z']" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 38 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The \"Ceasar Cipher\" is a cipher that encrypts text by translating each letter in the text to the next letter in the alphabet (with Z going back to A). We'll represent the cipher, and its key, as mappings in a dictionary." ] }, { "cell_type": "code", "collapsed": false, "input": [ "ceasar_cipher = {i: j for i, j in zip(letters, letters[1:] + letters[:1])}\n", "inverse_ceasar_cipher = {ceasar_cipher[k]: k for k in ceasar_cipher}" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 39 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Functions that cipher (encrypt) and decipher a test string. `decipher_text` doesn't need an explicit deciphering key, it will invert the cipher dictionary used by `cipher_text`." ] }, { "cell_type": "code", "collapsed": false, "input": [ "def cipher_text(text, cipher_dict = ceasar_cipher):\n", " # Split the string into a list of characters to apply\n", " # the decoder over.\n", " strlist = list(text)\n", " \n", " ciphered = ''.join([cipher_dict.get(x) or x for x in strlist])\n", " return ciphered\n", "\n", "def decipher_text(text, cipher_dict = ceasar_cipher):\n", " # Split the string into a list of characters to apply\n", " # the decoder over.\n", " strlist = list(text)\n", " \n", " # Invert the cipher dictionary (k, v) -> (v, k)\n", " decipher_dict = {cipher_dict[k]: k for k in cipher_dict}\n", "\n", " deciphered = ''.join([decipher_dict.get(x) or x for x in strlist]) \n", " return deciphered" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 40 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following functions are used to generate proposal ciphers for the Metropolis algorithm.\n", "The idea is to randomly generate ciphers and see what text they result in. If the text resulting from a proposed cipher is more likely (according to the lexical database) than the current cipher, we accept the proposal. If it's not, we accept it wil a probability that is lower the less likely the resulting text is.\n", "\n", "The method of generating new proposals is important. The authors use a method that chooses a key (letter) at random from the current cipher, and swaps its with some other letter. For example, if we start with the Ceasar Cipher, our proposal might randomly choose to re-map A to N (instead of B). The proposal would then be the same a the Ceasar Cipher, but with A $\\rightarrow$ N and M $\\rightarrow$ B (since A originally mapped to B and M originally mapped to N). This proposal-generating mechanism is encapsulated in `propose_modified_cipher_from_cipher`. \n", "\n", "This is inefficient in a few ways. First, the letter chosen to modify in the cipher may not even appear in the text, so the proposed cipher won't modify the text at all and you end up wasting cycles generating a lot of useless proposals. Second, we may end up picking a letter that occurs in a highly likely word, which will increase the probability of generating an inferior proposal.\n", "\n", "We'll suggest another mechanism that, instead of selecting a letter from the current cipher to re-map, will choose a letter amongst the non-words in the current deciphered text. For example, if our current deciphered text is \"hello wqrld\", we will only select amongst {w, q, r, l, d} to modify at random. The minimizes the chances that a modified cipher will turn real words into gibberish and produce less likely text. The function `propose_modified_cipher_from_text` performs this proposal mechanism.\n", "\n", "One way to think of this is that it's analogous to tuning the variance of the proposal distribution in the typical Metropolis algorithm. If the variance is too low, our algorithm won't efficiently explore the target distribution. If it's too high, we'll end up generating lots of lousy proposals. Our cipher proposal rules can suffer from similar problems." ] }, { "cell_type": "code", "collapsed": false, "input": [ "def generate_random_cipher():\n", " '''\n", " Randomly generate a cipher dictionary (a one-to-one letter -> letter map).\n", " Used to generate the starting cipher of the algorithm.\n", " '''\n", " cipher = []\n", "\n", " input = letters\n", " output = letters[:]\n", " random.shuffle(output)\n", " \n", " cipher_dict = {k: v for (k, v) in zip(input, output)}\n", " \n", " return cipher_dict\n", "\n", "def modify_cipher(cipher_dict, input, new_output):\n", " '''\n", " Swap a single key in a cipher dictionary.\n", "\n", " Old: a -> b, ..., m -> n, ...\n", " New: a -> n, ..., m -> b, ...\n", " '''\n", " decipher_dict = {cipher_dict[k]: k for k in cipher_dict}\n", " old_output = cipher_dict[input]\n", " \n", " new_cipher = cipher_dict.copy()\n", " new_cipher[input] = new_output\n", " new_cipher[decipher_dict[new_output]] = old_output\n", " \n", " return new_cipher\n", " \n", "def propose_modified_cipher_from_cipher(text, cipher_dict, \n", " lexical_db = lexical_database):\n", " '''\n", " Generates a new cipher by choosing and swapping a key in the\n", " current cipher.\n", " '''\n", " _ = text # Unused\n", " input = random.sample(cipher_dict.keys(), 1)[0]\n", " new_output = random.sample(letters, 1)[0]\n", " return modify_cipher(cipher_dict, input, new_output)\n", "\n", "def propose_modified_cipher_from_text(text, cipher_dict, \n", " lexical_db = lexical_database):\n", " \n", " '''\n", " Generates a new cipher by choosing a swapping a key in the current\n", " cipher, but only chooses keys that are letters that appear in the\n", " gibberish words in the current text.\n", " '''\n", " deciphered = decipher_text(text, cipher_dict).split()\n", " letters_to_sample = ''.join([t for t in deciphered \n", " if lexical_db.get(t) is None])\n", " letters_to_sample = letters_to_sample or ''.join(set(deciphered))\n", " \n", " input = random.sample(letters_to_sample, 1)[0]\n", " new_output = random.sample(letters, 1)[0]\n", " return modify_cipher(cipher_dict, input, new_output)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 60 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following functions compute the likelihood of text generated by proposed ciphers. To find a word's probability, we just look it up in the lexical database. To compute the log probability of some text, we just sum up the log probabilities of the words in it." ] }, { "cell_type": "code", "collapsed": false, "input": [ "def one_gram_prob(one_gram, lexical_db = lexical_database):\n", " return lexical_db.get(one_gram) or np.finfo(float).eps \n", "\n", "def text_logp(text, cipher_dict, lexical_db = lexical_database):\n", " deciphered = decipher_text(text, cipher_dict).split()\n", " logp = np.array([math.log(one_gram_prob(w)) for w in deciphered]).sum()\n", " return logp" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 43 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now the Metropolis algorithm step. We generate a proposal cipher, and compute the log probability of the its decoded text. If this is greater than the log probability of the decoded text of the current cipher, we accept the proposal for sure. Otherwise, we only accept it with a probability given by its likelihood relative to that of the current cipher, so the less likely its decoded text is, the less likely we accept it." ] }, { "cell_type": "code", "collapsed": false, "input": [ "def metropolis_step(text, cipher_dict, proposal_rule, lexical_db = lexical_database):\n", " proposed_cipher = proposal_rule(text, cipher_dict)\n", " lp1 = text_logp(text, cipher_dict)\n", " lp2 = text_logp(text, proposed_cipher)\n", " \n", " if lp2 > lp1:\n", " return proposed_cipher\n", " else:\n", " a = math.exp(lp2 - lp1)\n", " x = random.random()\n", " if x < a:\n", " return proposed_cipher\n", " else:\n", " return cipher_dict" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 44 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create a message and decode it with the Ceaser Cipher. Then set up the Metropolis algorithm which generates proposal ciphers (we can provide different proposal mechanisms), and calls the Metropolis step defined above. This runs for a fixed number of iterations. " ] }, { "cell_type": "code", "collapsed": false, "input": [ "message = 'here is some sample text'\n", "ciphered_text = cipher_text(message, ceasar_cipher)\n", "niter = 250000\n", "\n", "def metropolis_decipher(ciphered_text, proposal_rule, niter, seed = 4):\n", " random.seed(seed)\n", " cipher = generate_random_cipher()\n", "\n", " deciphered_text_list = []\n", " logp_list = []\n", "\n", " for i in xrange(niter):\n", " logp = text_logp(ciphered_text, cipher)\n", " current_deciphered_text = decipher_text(ciphered_text, cipher)\n", "\n", " deciphered_text_list.append(current_deciphered_text)\n", " logp_list.append(logp)\n", " \n", " cipher = metropolis_step(ciphered_text, cipher, proposal_rule)\n", " \n", " results = DataFrame({'deciphered_text': deciphered_text_list, 'logp': logp_list})\n", " results.index = np.arange(1, niter + 1)\n", " return results" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 45 }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, let's try the author's proposal rule." ] }, { "cell_type": "code", "collapsed": false, "input": [ "results0 = metropolis_decipher(ciphered_text, \n", " propose_modified_cipher_from_cipher, niter)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 46 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Looking at the deciphered text of every 10,000th entry, we find we're not even close after 250,000 iteration, and we seem to be locked in a pocket of the target distribution that's the proposal rule has trouble escaping from." ] }, { "cell_type": "code", "collapsed": false, "input": [ "results0.ix[10000::10000]" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
deciphered_textlogp
10000 kudu of feru fyrvbu hush-86.585205
20000 wudu of feru fbrkxu hush-87.124919
30000 kudu of feru fnrbau hush-86.585205
40000 wudu of feru fmrjiu hush-87.124919
50000 kudu of feru fyrnbu hush-86.585205
60000 kudu of feru fxrnvu hush-86.585205
70000 pudu of feru fvrnlu hush-87.561022
80000 kudu of feru fvrxgu hush-86.585205
90000 kudu of feru fbrvtu hush-86.585205
100000 kudu of feru fjrnlu hush-86.585205
110000 kudu of feru fprbju hush-86.585205
120000 kudu of feru fnrjcu hush-86.585205
130000 kudu of feru flrvpu hush-86.585205
140000 puku of feru flrvxu hush-88.028362
150000 kudu of feru fxrviu hush-86.585205
160000 pulu of feru ftrdzu hush-88.323162
170000 wuzu of feru flrxdu hush-89.575925
180000 kudu of feru firamu hush-86.585205
190000 wudu of feru fyrzqu hush-87.124919
200000 wudu of feru fnraxu hush-87.124919
210000 puku of feru fjrnyu hush-88.028362
220000 puku of feru firyau hush-88.028362
230000 pudu of feru fkrcvu hush-87.561022
240000 kudu of feru ftrwzu hush-86.585205
250000 kudu of feru fprxzu hush-86.585205
\n", "
" ], "output_type": "pyout", "prompt_number": 47, "text": [ " deciphered_text logp\n", "10000 kudu of feru fyrvbu hush -86.585205\n", "20000 wudu of feru fbrkxu hush -87.124919\n", "30000 kudu of feru fnrbau hush -86.585205\n", "40000 wudu of feru fmrjiu hush -87.124919\n", "50000 kudu of feru fyrnbu hush -86.585205\n", "60000 kudu of feru fxrnvu hush -86.585205\n", "70000 pudu of feru fvrnlu hush -87.561022\n", "80000 kudu of feru fvrxgu hush -86.585205\n", "90000 kudu of feru fbrvtu hush -86.585205\n", "100000 kudu of feru fjrnlu hush -86.585205\n", "110000 kudu of feru fprbju hush -86.585205\n", "120000 kudu of feru fnrjcu hush -86.585205\n", "130000 kudu of feru flrvpu hush -86.585205\n", "140000 puku of feru flrvxu hush -88.028362\n", "150000 kudu of feru fxrviu hush -86.585205\n", "160000 pulu of feru ftrdzu hush -88.323162\n", "170000 wuzu of feru flrxdu hush -89.575925\n", "180000 kudu of feru firamu hush -86.585205\n", "190000 wudu of feru fyrzqu hush -87.124919\n", "200000 wudu of feru fnraxu hush -87.124919\n", "210000 puku of feru fjrnyu hush -88.028362\n", "220000 puku of feru firyau hush -88.028362\n", "230000 pudu of feru fkrcvu hush -87.561022\n", "240000 kudu of feru ftrwzu hush -86.585205\n", "250000 kudu of feru fprxzu hush -86.585205" ] } ], "prompt_number": 47 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's try the alternative proposal rule, which only chooses letters from gibberish words when it modifies the current cipher to propose a new one." ] }, { "cell_type": "code", "collapsed": false, "input": [ "results1 = metropolis_decipher(ciphered_text, \n", " propose_modified_cipher_from_text, niter)" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 48 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The algorithm doesn't find the actual message, but it actually finds a more likely message (according the the lexical database) within 20,000 iterations." ] }, { "cell_type": "code", "collapsed": false, "input": [ "results1.ix[10000::10000]" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
deciphered_textlogp
10000 were mi isle izlkde text-68.946850
20000 were as some simple text-35.784429
30000 were as some simple text-35.784429
40000 were as some simple text-35.784429
50000 were as some simple text-35.784429
60000 were as some simple text-35.784429
70000 were us some simple text-38.176725
80000 were as some simple text-35.784429
90000 were as some simple text-35.784429
100000 were as some simple text-35.784429
110000 were as some simple text-35.784429
120000 were as some simple text-35.784429
130000 were as some simple text-35.784429
140000 were as some simple text-35.784429
150000 were us some simple text-38.176725
160000 were as some simple text-35.784429
170000 were is some sample text-37.012894
180000 were as some simple text-35.784429
190000 were as some simple text-35.784429
200000 were as some simple text-35.784429
210000 were as some simple text-35.784429
220000 were as some simple text-35.784429
230000 were as some simple text-35.784429
240000 were as some simple text-35.784429
250000 were is some sample text-37.012894
\n", "
" ], "output_type": "pyout", "prompt_number": 49, "text": [ " deciphered_text logp\n", "10000 were mi isle izlkde text -68.946850\n", "20000 were as some simple text -35.784429\n", "30000 were as some simple text -35.784429\n", "40000 were as some simple text -35.784429\n", "50000 were as some simple text -35.784429\n", "60000 were as some simple text -35.784429\n", "70000 were us some simple text -38.176725\n", "80000 were as some simple text -35.784429\n", "90000 were as some simple text -35.784429\n", "100000 were as some simple text -35.784429\n", "110000 were as some simple text -35.784429\n", "120000 were as some simple text -35.784429\n", "130000 were as some simple text -35.784429\n", "140000 were as some simple text -35.784429\n", "150000 were us some simple text -38.176725\n", "160000 were as some simple text -35.784429\n", "170000 were is some sample text -37.012894\n", "180000 were as some simple text -35.784429\n", "190000 were as some simple text -35.784429\n", "200000 were as some simple text -35.784429\n", "210000 were as some simple text -35.784429\n", "220000 were as some simple text -35.784429\n", "230000 were as some simple text -35.784429\n", "240000 were as some simple text -35.784429\n", "250000 were is some sample text -37.012894" ] } ], "prompt_number": 49 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's take a look at the likelihood paths of the algorithm with the two proposal rules. We can compute the probability of the original message, which is somewhat less than the message the \"propose from text\" rule converges to." ] }, { "cell_type": "code", "collapsed": false, "input": [ "plt.figure(figsize = (10, 7))\n", "results0.logp.plot(style = 'r-', label = 'propose from cipher')\n", "results1.logp.plot(style = 'k-', label = 'propose from text')\n", "plt.xlabel('Iteration')\n", "plt.ylabel('Log Probability')\n", "\n", "plt.axhline(text_logp(ciphered_text, ceasar_cipher), label = 'log prob of true text')\n", "plt.legend(loc = 'best')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 50, "text": [ "" ] }, { "output_type": "display_data", "png": "iVBORw0KGgoAAAANSUhEUgAAAqYAAAHXCAYAAABu7ZhmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4FNXeB/Dv1mwKSSAxJPTQpVdFQJEmGmoUeEFREbgo\nckEEAUEUC+0i6kVREREUERBFxYaAAiISRSlK74IUEwgJyYb0nPePc8/szu6mAEl2E76f59lnd6ee\nmXPmzG/OnJ01CCEEiIiIiIi8zOjtBBARERERAQxMiYiIiMhHMDAlIiIiIp/AwJSIiIiIfAIDUyIi\nIiLyCQxMiYiIiMgnmL2dgPIiMzMTU6dOhcViwY4dOzB27Fj069cPFy5cwIQJE1C9enWkpKRg/vz5\nMBp5PUBERETkihFSMdm6dSv27NmDOXPmYOHChRg+fDgAYOLEiYiNjcXMmTMREBCApUuXejmlRERE\nRL7JwAfsF5/Lly8jJCQEly5dQsOGDZGQkIAqVapg165diIyMxNq1a7Fs2TKsWbNGm+fKlSs4ffo0\nzGY2Xpc16tAxGAxeTgldC+Zf2cb8K7uYd2VbYGAgwsLCYLVaS2T5jIaKqHv37h6Hb9y4UfscEhIC\nAJg9ezZmzJgBAEhMTESFChUAAEFBQYiPj9fNf/r0afzzzz+oVq1aSSSbSlBaWhoAma9U9jD/yjbm\nX9nFvCvbkpOTYTAYEBkZWSLLZ2BaRM4BaEHefvttVKhQASNHjgQAREREwG63IzAwEKmpqW4ZaTab\nUa1aNdStW7fY00wlKzk5GQAQGhrq5ZTQtWD+lW3Mv7KLeVe2JSQklOjy2ce0mAgh8NJLL8FqteK5\n557Dhg0bcPLkScTExGD79u0AgLi4OMTExHg5pURERES+iS2mxeStt97CvHnzUKVKFcybNw9JSUnY\nsGEDZs2ahdGjRyMuLg5paWkYOnSot5NKRERE5JP44ycvO3bsGADwVn4ZxNtRZRvzr2xj/pVdzLuy\nTd3Kj4iIKJHl81Y+EREREfkEBqZERERE5BMYmBIRERGRT2BgSkREREQ+gYEpEREREfkEBqZERERE\n5BMYmBIRERGRT2BgSkREREQ+gYEpEREREfkEBqZERERE5BMYmBIRERGRT2BgSkREREQ+gYEpERER\nEfkEBqZERERE5BMYmBIRERGRT2BgSkREREQ+gYEpEREREfkEBqZERERE5BMYmBIRERGRT2BgSkRE\nREQ+gYEpEREREfkEBqZERERE5BMYmBIRERGRT2BgSkREREQ+gYEpEREREfkEBqZERERE5BMYmBIR\nERGRT2BgSkREREQ+weztBFDZkpeXhytXrsBqtSIrKwuBgYEwGAzIy8tDWloaAMBgMGjTGwwGCCFg\nNpuRl5cHIQRsNhuEEMjJyYHNZkN6ejpyc3O1ecxmM2w2G9LS0iCE0K3fYDDAaJTXU0II5OXlwc/P\nD5mZmbrp/Pz8YDabkZOTA4PBgIyMDBiNRpjNZhiNRmRlZcFgMMBiscBsNiM9PR1CCAQEBCA3Nxfp\n6elamv39/ZGRkaGl0d/fH9nZ2UhLS4PBYIDZbNat12g0wmAw4MqVK9pwo9GIvLw8bX8IIWCxWLTl\nq31nMpm0/aS20WAwwGQywWq1wmQyAQAyMzORnZ2tjVfLt1qtsFqtyMjIQE5OjrZ+lU+uhBBIS0vT\n9nlWVhaysrJgNBphtVq1bcvKykJOTg7y8vK0/aqWm5WVhezsbG2ZVqtVS5fVatXyV227cxkJCAhw\nS1d6ejosFgvS09O1cWp5/v7+yMvLQ2ZmJvz8/JCbm4vc3FyYzWZdGlzLhMlkgsFgQE5ODvz9/ZGV\nlaXb52o7hBDw8/PT5Z3ivN3O+yc7O1tbl/M2CiFgNBq18mCz2ZCZmamVyby8PG05qkxkZWVpaVLp\nVPvdarVqeW40GhEQEOCWRgDa/nE+pmw2GzIyMmCxWGAwGJCVlQWLxaLbZ4rzNnhisVjg5+eHnJwc\nZGRk6KY3m83aeq1Wq7ZfVP655r/BYNCG+/n5afmUkZGh5W92djb8/f0hhEBubi4yMzMRGBgIANq+\nV9OaTCZtmNFohBACVqsVRqMR6enpuu1QaVJ5qNLhnLbAwEBcuXIFubm5sFqtsFgsWj6p7VTHnlqX\nShcgj4+0tDT4+fkBgFbGnPMFAHJzc7Vp1DEJyOMjOztbG6f2uZpO1Reu1HxqXpXuK1euaPsOgLaf\n1bbn5eUhPT0dNpsNOTk5yMnJ0cqiqnNUPaTqyJycHF09FBAQoO0ftY9c6zW1fap+UfWWovJd7R/n\n7Xbl7+8Pk8kEu90Ok8mE3Nxc+Pv7a+tXx5lzuVR1XEBAgFY/BQQEQAiBjIwMXfkymUy4cuUKLBaL\nVg7yS0tQUJDH4a6c6wxA5mVgYKB2bgNkPehc56v8ysnJ0eVpenq6Vj5UHSeE0B3bqu40GAxIT0/X\n6nF1LlSc81btS7PZrNVPnspaiRDkVUePHhVHjx71djKKbOLEicJsNgsAAoB45ZVXhBBC+65egYGB\n2mebzaYbN3r0aO3zH3/8oZsnMDBQmM1m8f777wuTyaQNc16e80st22w2a9PZbDbRtWtX0bJlSwFA\n9OnTRzdP5cqVtc933323OHr0qDCZTMJkMolff/1VjBw5UlitVm2aChUqCAAiICBAWK1W8eCDDwqL\nxSJCQ0NFaGiotl41T0xMjHjyySeF1Wr1mHbX72p5zsNdP5tMJgFA7NmzR+Tk5HjcHwEBAaJmzZri\n4sWL2vLU/nznnXc85ufrr78uzGazsNlsIjEx0W2ZeXl5uvz19/cXAISfn5+wWCza54CAABEYGCj8\n/f1FvXr1tOnffffdAsvGvHnzdOn5888/3aY1GAwCgC5P1LoBaOPV9qp9ZbPZ3Pa/n5+f6NSpkzCZ\nTCIqKkqEhoYKAOKpp57Stk2ty7nsOY9zfVWrVk033lOe+/v7i4yMDN0wPz8/AUD07t1b1KxZU1eO\nTSaTOHHihKhfv75uHnVMmEwmcfDgQY956rr/1H6zWCwiIiJCNGvWTBiNRq3MeDrGnIc5v1Sad+7c\nKfr06aN9d81X5zxwLp/57SM1/ZNPPilWrFjhcT8PHTpU9/1f//qXiI6OFgBE586dRfXq1UVWVpbH\neUeMGOHxeHStm1TanNPq/HrsscdEly5ddMeC8yskJERbrtlsFh07dhRGo1G0b99etG7dWrcO530X\nEhKi5d9bb70lzGazsFgsom7dugKAyMnJEVFRUfnWD84vq9Uqxo4dq9WFHTp0EEIIsWDBAgFAfPbZ\nZ0IIIdLS0rRjVAghnnnmGW2ZPXr0EH5+flr98c8//2j7pWrVqmLMmDG649E5H+Pi4vLNZ39/fxEY\nGCiioqJEVFSUbtzWrVu1ffDwww8LAOLChQtCCCF69OihO55dl7tmzRqtHjCbzaJ9+/Ye81XVGzab\nTZhMJvH5558Lk8kkLBaLeO6558QjjzwiAIjvv/9eCCFExYoVBQDteFF55SktJpNJrFq1qtBz6OnT\np7V6y3kbpk2bpk1z9uxZrbwpycnJwmq1CpvNJjp37iyEEPkeK5UrV9Yd2yaTSdhsNtG7d2+tDrFY\nLKJKlSpanqh577jjDo95q/ahyWQS8fHxIj4+vtBtvVa8lU9XJSkpCZMmTQIgr66Sk5PdpmnSpAns\ndjs6deoEAPj111914xMSErTPZ86cAQDUr18fdrsddrsdkZGROHfuHAYNGqQNs9vtCA8P1y0nLCwM\nmzZtAgA8//zz2nS//PILEhISsHv3bgDApUuX8MYbb2jzxcfH6z5fuHABbdq0QZcuXZCUlIRLly7h\nww8/1KZJTU0FANjtdixatAinT59GZGSkdrWv1vvOO+8AADZt2oTk5GS89dZb2jhnrt/379+Pl156\nSRter1492O123H777dr0rVu3BgBcvnxZa0Vr2rQpAKBOnToAgLNnzyIpKQmpqamIiorS1j1+/Hhc\nunTJLZ8AIDExEVOmTIHFYtGmiYqKQoUKFQBAdzUNAEeOHAEArF69GnPmzAEgW2/Vus6cOYO///5b\nm/7UqVMAgEqVKgEAmjVrBrvdru0757IAABcuXHDbV3v27AEAXZ7Mnj1b+xwTE6NNa7fb8dhjjwEA\nfvvtN22YagH59NNPcfLkSTRo0AAHDhzAe++9BwA4ceIEbDYbatWqBQBYsGCBruydPn1aW1/jxo11\naTxz5gz+/PNP3HXXXbp0OPP393fLgzVr1gAAUlJS8M8//wAApk2bBrvdjsaNG+Py5cvaPJUrVwYA\n3H333Vp58HTsOWvQoAHsdjtWrlwJAHjppZeQkpKCpKQkdOnSRZdW9XrxxRc9Dlev77//HgC042TD\nhg0AoJU35eGHHwYA7Ny5E0OGDAHgOI6cl1+lShUAwMCBAwEA27dvx9mzZz1uT1xcnO77zz//jKSk\nJADA5s2b8ffff2stcYMGDdJNe+nSJXz00UfaemfNmgUA2vYAQI8ePbS0nTt3zmMaLly4gMOHDwMA\n9u7di44dO+rWd/nyZQDAgQMHMH78eBw7dgx333034uPjtXpnxYoVsNvt+Oqrr7TlqvkAuW+feuop\nvP766zh27BgA2Xp2/vx5AMDKlSt1+9o1j95991389ddfMBqN2LNnj3aMqfWr7ykpKbrval80b94c\nR48eRWhoKOx2O6pXr47ExERkZ2fj0qVLSEhIQHx8PD744APdvunWrRu6du2q2xbXY2HXrl2w2+04\ncOAADhw4ALvdrpUVlZcqvwBHmbl06RK2bNnitq3Vq1fXtk0dfxMnTsS+ffsAAOHh4WjTpo2WFnUe\n2L17N+68804cP34cjRo1wrx583Dp0iUkJibCZrNp+0Sl6d5779Xl1aZNm9zSMmrUKN25JT9qv8+a\nNQt2ux0PPvigbpvVOtR2Oe/LSpUqYceOHW556ioxMRFJSUla2gYMGICMjAx89dVXaNu2Lex2O2rU\nqIFz587h+PHjsNvtaNu2LQDg6NGjbnkLALVq1dK2s6TxVj5dldzcXISEhAAAgoODdbeLFddhzreO\nAGgVO+C4peN8C8NoNCI7O1s3DJC3SS5evKh9V7faXNcRGBioO2DV7SdPzp8/j88++wyBgYHardL8\nple3/dQtFvVZsVgsAKDd8nBNvydWqxWnTp3Spd/TfOp2Tl5eHtavX691AwActwPVrdns7GwtLWod\nnm7ZAtBuk1qtVi0vgoKCtBPCqVOnUL9+fW165/2t1qFuD6vhzre51C3xwMBAXLp0Cf7+/rr1nzhx\nQvddnXydqe1z3kfOt7GFy21nNc55ehVgq+1UgbJK9/79+xEYGKjtZ9d0qmUZDAaPt9Bdy7in8YsX\nL9YNU+s+ffq0dltPLce5LDoPV+krKE8V5+4WgOzekJeXp93ydk6Da5ryo7Zd3f5WZSA4ONjjdIGB\ngVo59VSu1S1IlcYDBw64Xci6Tqt4uq24dOlSj+tyPabVhZFzvjkfM863T52dOHFCCwqcjwHXMpGT\nk6OVtdDQUFy8eNGtXnStY1SdkZOTA7PZrEubuoAC3Mumq8DAQBw4cACBgYEIDAzExYsXsWTJEuzY\nsQMAsGXLFlgsFq0ujYuLw5IlS7B3714AwE033YQjR44gKipKS2daWpp2uz0vLw+HDh1yK/NZWVkI\nCgrSdSFx5dztyZVzeVafnd891ckVK1bE33//jR9//BE33XSTln4V/HnaN+rdbDZj+/btWjcpdbyF\nhIRgw4YNurrdNX89bUdgYCC2bt2K2rVro1evXvlup2s5UGXV0/Zv2bIFycnJCA0N1faBOr8tWbIE\nP/30U77rcC7Pnsq567lTbdP58+c9ln/nOr6kMTClq5KTk4Pw8HCMHj0aiYmJuoNs5MiR6N69u9YK\nolStWhWjR49Go0aN4Ofnh7i4OFy6dAlnzpzxGJiqfmKuJ5dly5bh/Pnz2LlzJxo2bIigoCBER0fj\nX//6F+68805tumrVquHuu+/GsmXLAMjWhiZNmqB27dqoVasW0tPTERcXh6effhrJyclITEzEQw89\nhC+++ELrY6kOXrPZjK5du2oHrdls1vobvfLKK7rtd64InE/+gGyBq1GjhhYgDR48GCtXrsTQoUOR\nnZ2NO+64AwCwePFiREREuO33kSNHIi4uDhs3bsSsWbNQsWJF7cT85ptvIj4+Pt/A1GKxuLV8KtnZ\n2QgODobFYtGCyCVLluD06dN44IEHsHXrVi0wXbx4MUJCQvDEE0+gcePG2onrmWee0ZZntVoxYcIE\n/PPPP/joo49w5coVhIWF4f3330dycjKqVq2qW79ri6lz386uXbsCAKKjozFp0iS0aNFCG+ccwI0a\nNQq33XabNq5nz55ISUnRlcO33npL62OZlpam7Z+2bdvigQcegJ+fH+rXr4/w8HCMGDEC7dq106XL\nZrOhUaNGmD9/PkJDQ7XWBQB4/PHHER4ejrlz5+KBBx7Qhj/77LNo2LAhsrKykJCQgEOHDuHRRx9F\ndnY2OnfujLZt22L8+PGw2+1aP0XVkqnyLCsrC0uWLEFERAR69eqllcP88tQ5WHv//fcBALVr19a2\nITc3Fzk5OXj44Ydx6623us0/dOhQtzsTzurWrQvA0QfQYrHg22+/RWhoqG66vn37Ijs7G1FRUW4B\npPNJftGiRUhKSkJQUBD27duHqKgorZ9ecHAwMjIy4O/vj8uXLyMyMhJ//vknpk2bhhkzZqBZs2Za\nS3OrVq2QkpKCvXv3Ijw8HFOmTEH9+vXRuHFjDBgwQHdMA9BacWvXro0xY8YgODgYw4YNwy233ALA\nUR9Vr14dQ4cOhd1uxz///IOAgAA0a9YMgGyNmz17Ng4fPozbbrsNbdq0QXx8PF544QXUrFlTK2tV\nq1bFkCFDYDQaER4errXMOqdH7VPnwLRdu3YYNmwYzp07hz/++EObTt2JqlGjhq4lX2nTpg06deqE\nOnXqoHLlyrj//vuxbds2VKlSBS+//DIOHDiAbdu2AZAt1YGBgdi2bRs6d+6MM2fOYOzYsahdu7Z2\n3DsfMwaDAZMnT0Z8fDxatGiB2bNnIzo6GjNmzMCrr76KF154QQuqnO9SzZo1C1OnTvUY8Dz66KP4\n4IMPdIGZKtvO7677C5B10j333IOAgAAMHjwYERER6N27N/bv368L5pV27dph1KhRqFy5MkwmE77+\n+msMGjRIO56ys7MxaNAgpKSkaPsIcJRZf39/rW+tq5iYGCxevBgDBw702EddUdupWmGnTJmCw4cP\n647nrKwsREVFISEhARs2bMDAgQO1fVC1alUMGDAA27ZtQ0hICMLCwpCYmAjA0afZbDbrLjJjY2Px\n3nvvoWrVqtp+HDlyJI4ePaodb3PmzNHKlnM9ExERgcceewyNGjXStrPElVgnASqSstbHdPDgwWL5\n8uVCCCHmzZsnxo8fL4QQonXr1uK3337TTdupUydRUBFr2rSp1u+pcePG2vA6deqIyZMni2HDhl1z\nOs+fP6/1i6lbt674448/tHH79+8XAITdbtfNM3DgQLFq1SrRrVs3sWHDBgFAtG7dWjfNp59+KurU\nqSOaNWsmkpKSRFJSkjbus88+E4Dsn+i8nzz566+/BADx008/5TtNx44ddfvvjjvuEGPGjBEARERE\nhGjQoIFufF5engBkP9SmTZtqw2fPni0mT57scR1PPPGEePXVV0WNGjXExo0bRZ06dbRxI0eOFG+/\n/bYQQgij0ShycnLyTasn7du3Fw899JAYMWKE2zjVT7lTp0664W+++aaWb7t373abT4375JNPBAAx\nadKkq0rTtm3bBADRoUMHt/y7GiodJaVTp05i8+bNwmw2i6ysLHH58mUBQAwYMEAIIfvcrVu3zm0+\n1Y/Vz89PG6b6Ei5ZskQYDAZRqVIlcfHixWtOW/fu3cV3330nmjZtKvbs2aMbB0B0795dN2zgwIHa\nvgIggoODr2p9av677rpLvP7660IIIWbOnCmmTJmi9RE+efKkx3kzMjKE1WoVXbp00foNFoXajxkZ\nGVeVVmezZ88WAMT06dM9jv/tt990/ffS09OFEEJMnTpVzJgxw216AOLZZ5/Vvs+dO7dEy6DSqlUr\n8fXXX4vKlSsXOm1sbKxYvXq1MBqNuuFqf54+fVoIIdyOvSFDhogPPvhA+3777bcLAGLXrl1CCCHq\n1asnDh8+fFXpBiDCw8NFmzZtPO6nPn36CKPRKNasWSOWLVsmhgwZIu644w6xefNmbZr58+cLAGL8\n+PECgNbn9M8///S4zszMTGGxWApMV1xcnLj11lt1w5YvXy7uv/9+7fv27dtFu3btdOeRffv2iZtv\nvtlteQMGDNDK0Jdffqn15XU1c+ZM0aJFC9GjR4980/bEE08IAGLt2rUCgOjatavH6djHlHyKupoH\n9L9SVr/edNa+fXvtKsuToKAgTJ8+HUDRWkyvhvPV9aVLl9xaEF2nUekZO3Ystm3bprXIud52MZvN\nOH78uK6/nOtys7KysHLlygLT76kLQmFMJpN2yzchIQEtWrTQbl8B0H69v3HjRt22BQUFYeHChTAY\nDIiJiUG1atVgt9uRkJCA+fPnIzAwEEFBQbj//vt1vxRVLbDiKromOLNYLDh+/LjHlg51Zf7jjz+i\nXr16WLx4Mbp164bRo0dr0xS0b1QLXX63XPOjWj5cf6Htay5fvozOnTtrT5JwPuYA2X1h/Pjxunme\nfvpp7XhzzUfA8bSE7OzsAm+pFsZisWDbtm1ISkrymLfVqlXTfW/QoIH2uVGjRlqfv6JSebZhwwZt\nfUFBQfjoo490acovrVlZWdi0adNVH2tA4V0bCqJaovJbr+svuJ3r0vzyRxTwtISSsmvXLvTq1cut\n/7cnZrMZGRkZbul3Lb+uAgIC8PDDD6Np06aoV6+e1u1gw4YNuHjxIo4ePXrVt5BtNhtuuukmNGjQ\nQOt+5pqmvLw8mEwmBAUFYfny5di6davH296q+4R6z287nJ/woJ76YjAYdH2ZPeVvUFAQVqxYgWbN\nmsFgMGDAgAFaV5Fvv/0We/bswZAhQzzuA+djXXVR8DSdxWLBhQsX8j1WnLf3euqH4sBb+SXg/vvv\nR4MGDTB9+nRcuHABEyZMQPXq1ZGSkoL58+dfV8Dlbc4Hlclk0ipT9VgJZy+99JLuNq+rr7/+GomJ\niahfv77HPqZXG3Q4UwdYxYoV0blzZ91t3aioKDz44INuB+j8+fPx9NNPw2QyITo6GhMnTtRuXSpq\nG/v06eO2zkaNGiEmJgZDhgzB/fffX2D6w8PDcf78ee1HLUWhKn1l+fLlbv25hg0bhq1bt+K+++7T\nhj366KPo0aMH6tevj3Xr1gGQHdwvX76M6tWrY9iwYWjXrh2aN2+Oli1bavOpwFR1S7jak3S/fv2w\nadMm7fa0sy+//BK//PILunbtijNnzmDv3r26/lJ//fUXatSo4XG5r7/+Ou644w707dtX+/FJUanu\nAOrHL9fKU5/B4tSnTx/s2bMHf/31l3bBATgCpgULFqB58+a6H5ps27YNL774IuLi4nTBoSqz9erV\nK5Zjq1evXli3bh1uvfVWt64ZiYmJbv3xpk+fjilTpgCQQc7V1n8LFiyAzWbDwoULtQBj1KhRuP32\n25GTk4O5c+ciLCzM47xqXQMHDvTYdSE/xRGYquOuZs2aHsc3aNAABw8ehM1mQ/PmzQu8yFecb/f2\n7t270B/AFafHH3+80GnMZjMyMzPd0q/yIb+8f/XVV/Huu+9i3759OHLkCMxmM9577z38/fffOHLk\nCGrXru1W1gpz9uxZGI1G7VFpntKq3u+55x4AMrBzvnBq27Yt+vTpg5iYGOzduxepqak4d+5cvvnj\nun0qT3///Xet+09SUpJbuerduzcaNmyo9fOdOXMm+vbti0GDBmH9+vXYvXs3Dh8+7PHiQF2k9+vX\nD/Xr18epU6c8pu+2227DTz/9VOCteHVOvJ76oTgwMC1mr776Ko4fP46GDRsCkL8QjI2NRWxsLCZP\nnoylS5di+PDhXk7l1fnxxx/xyiuvAAB27NiBoUOHApAtMF9//TWqVKni8SpQPR8yP5UqVdJ+hFKU\nHz9dDXWALV26FH379tWNCwoK0vqfug6vV6+e9n3u3Llu00RGRiI0NFT7lbyz2rVr45tvvtF+lV7Y\nwR0ZGVngeNfWEZPJpAtEnFvSlEWLFrktx2Kx6LYLgNYnt0+fPjCbzVq/QueTtwpMCzpRFmTcuHEY\nN26cx3EBAQFakBEWFoYrV67oniWZ38kckH3rrFYrvvjii6tOkzohuPaJvFr5BULF5c4778SaNWu0\nfryuLU41a9ZEeHi4LnhX/RKd+7kCjm1WZcBTi9bVGDVqVL6/zFXHszP1HEzA/QdMRREYGIi3334b\nCxcu1C4ULRYLoqOjAQCffPJJgfM3adIEd91111UFmc7Pz71Wno4713Wo84TVasV9992HzMxMnDt3\nDk888YTHeZz3b8OGDTFz5sxrTt/VUv1vC2IymfCvf/3Lbbjan/nVI4GBgQgLC8PFixe1fda6dWvc\ne++9ePPNNzF48OCrTq/zvlI/oHSmfkwl/vdc4B49eqBhw4a6ctKmTRusXbsWALB27VoMGzYMQP4t\n9PmZPXu29iSRlJQUtxZco9GITp064dChQwBkEBkaGqoF1Ha7HVar1ePxdcstt2DTpk34/PPPC0xD\nx44dC72Qd61nXH8vUloMwhv3BsqpH374AVu2bNE6Hj/33HOoUqUKdu3ahcjISKxduxbLli3THhMD\nAFWqHEVgYBouXqzlvYTTNQkOlr/8TEkJLmRK8kXMv7KN+Vd2Me/KtsOHZcu9px/qFge2mBZR9+7d\nPQ7fuHEjAHnrcfHixfjoo4+0ZwEC8taWeiZkUFCQ23PHfvwRSEsD/vf4RJ80e/ZshISEeLyV89df\nf+HBBx+EEAIrV67Unit3NaKjo9GsWTPtyjQmJgaNGjVChQoVtD6o12Lr1q1o27ZtoY9XuVbqiSQu\nT8pBUlISWrVqhUWLFuVbboqif//+2LlzJ06ePAkAOHjwIA4fPozExETccccdBbbGXK2UlBQ0b94c\nI0aM0LpBcFOhAAAgAElEQVRfnDt3TuvrValSJe3JAcUlJycH69evx80334y9e/eiUqVKWn/E/Cq8\n6OhovPvuu+jWrds1r/f3339Hq1atoB6v6Jp/vmrz5s2oWbOm9iv7q7F37140bdoU0dHR8PPz01pm\nypJdu3ahZcuWWotWfsdfWdWxY0ecPXtWO95dHTx4EHXr1r3q1rrrFR0djUaNGuGbb74p8vRPPvkk\nxo4dm+80nvJu9+7dyMzM1D0RIzo6GsHBwbqnEhSn6Oho/Oc//9Geo1tcywRkQ5W/v79218N5fIcO\nHbB8+fJCl3X69Gns27cP48ePh81m057pXFLi4+OxY8cOdO/e3WMrs5LPQ16KDQPTIlIBaH5Wr16N\nxMRE3HfffTh06BAMBgO6deuGiIgI7QHfqampbrdvDQYDgoKCrvvWYklZv3495s2bh2XLlnlMY61a\ntXDmzBnk5eWhSpUq17QdycnJyMzM1ObNzMxEeno6KlSocF37xVM/0JLgmkaj0Yjk5GQEBARcV/rT\n09O1Z9gB8vaO82ORipPFYkFycjLMZrO2vtDQ0AJ/vFYc1G3nVq1aFWn65ORkNG7c+Lr2qwpqVVcR\nXz32XMXGxl7zvOrPGpKTk326vimIp77KQNnJv8Kox1Dltz0ldewXJjk5GRkZGUXez+np6ahXr16R\npneepnPnzh7XLYQosTyuWLHiddcnrlS/3/zqNNfzXUFCQ0NRuXJlxMfHo3LlyiVe1kNDQ3U/VsyP\n62P+ihsD02IyadIk7R+RXnjhBRgMBnTo0AExMTHYvn07YmNjERcXVzrPACtGly9fRv/+/d36rSmh\noaG6f+y4Vq59TM+dO1doH0xf5e/vD39/f6/1z7kWNptNe5anL2PPo+vn7V/ckmcrVqzwdhI86t+/\n/1XdLcnvf+R9kesffBSH3r176/7V63oFBwejQYMG2r/S3QhYQ5WwWbNmYfTo0YiLi0NaWpr2w6Gy\nojQCgalTp+o61vfo0QM//fTTVf2K1pc4P6z+ekyePBm//PJLMaSocCaTqUydUOjalcY/t1D5UdiP\ny0qap3/48mVF6WpxNT/s9ff3L5Ndb64HA9MS4NwvMiwsDKtWrfJiaq6PEOK6HplSFK6/LJ0xY0aJ\nrq+s6NOnT6l1R6Abw7vvvuvxL1WJfNFrr71W5i6knnvuOQwYMCDf8QsWLNC6bZBnDEyJiG4QI0aM\n8HYSiIosv8fN+bLmzZujefPm+Y53/hMR8qzsPumdSkVptJgSERERAQxMqRAMTImIiKi0MDAlIiIi\nIp/AwJQKxBZTIiIiKi0MTKlADEyJiIiotDAwJSIiIiKfwMCUCsQWUyIiIiotDEyJiIiIyCcwMKUC\nscWUiIiISgsDUyoQA1MiIiIqLQxMiYiIiMgnMDClArHFlIiIiEoLA1MqEANTIiIiKi0MTImIiIjI\nJzAwpQKxxZSIiIhKCwNTKpAQwttJICIiohsEA1MqFFtMiYiIqDQwMKUC8VY+ERERlRYGpkRERETk\nExiYUoHYYkpERESlhYEpFYiBKREREZUWBqZERERE5BMYmFKB2GJKREREpYWBKRWIzzElIiKi0sLA\nlArFFlMiIiIqDQxMqUC8lU9ERESlhYEpFYi38omIiKi0MDClQrHFlIiIiEoDA1MqEG/lExERUWlh\nYEpEREREPoGBKRWILaZERERUWhiYUoH44yciIiIqLQxMqVBsMSUiIqLSwMCUCsRb+URERFRaGJhS\ngXgrn4iIiEqL2dsJKE8SEhLw/PPPIysrC0OHDkXHjh1x4cIFTJgwAdWrV0dKSgrmz58Po7FsXQ+w\nxZSIiIhKQ9mKkHxYTk4OBgwYgFGjRmHx4sXo2LEjAGDixImIjY3FzJkzERAQgKVLl3o5pVeHt/KJ\niIiotLDFtJh89NFHyMrKwltvvYXTp09j1KhR6NWrFzZs2IA5c+YAANq3b49ly5Zh+PDh2nxCCKSl\npSE5OdlbSS9Qbm4ujEajz6bPm1JSUrydBLoOzL+yjflXdjHvqCAMTIuoe/fuHodv3LgRALBjxw50\n7twZs2bNwqlTp9C6dWscPXoUiYmJqFChAgAgKCgI8fHxpZbm4sIWUyIiIioNDEyLSAWg+bHZbKhY\nsSIAoGbNmqhSpQpOnjyJiIgI2O12BAYGIjU1FZGRkbr5DAYDgoKCEBoaWmJpvx4GgwF5eXk+mz5f\nwH1TtjH/yjbmX9nFvCubEhISSnT57GNaTNq1a4edO3cCAK5cuYLU1FQ0bNgQMTEx2L59OwAgLi4O\nMTEx3kzmNWGLKREREZUGtpgWk/79+2Pz5s0YN24ckpOTsWjRIgQEBGDWrFkYPXo04uLikJaWhqFD\nh3o7qVeFP34iIiKi0sLAtJgYDAa89dZbbsPDwsKwatUqL6SoePA5pkRERFRaeCufCsUWUyIiIioN\nDEypQLyVT0RERKWFgSkViLfyiYiIqLQwMKVCscWUiIiISgMDUyoQW0yJiIiotDAwpQKxjykRERGV\nFj4uijSZmZmYNm0afv/9d5w7dw69e/dGWFgYA1MiIiIqFQxMSXP69GnMmzdP+/7KK68AACZPnuyt\nJBEREdENhLfySXPlyhXd9xUrVgDgj5+IiIiodDAwJc3evXt13ytUqACAgSkRERGVDgamhEGDBsFg\nMGD//v0ICwvThgcEBHgxVURERHSjYWBKOHbsGAAgOzsbw4cPx2OPPQYAqFKlCgAgPj7ea2kjIiKi\nGwcDU0KlSpUAAJcvX0aVKlXw9ttvIy8vDw0bNgQAZGVleTN5REREdIPgr/LLqdTUVKSnpwOQzyL1\n9/dHZmam7oH5YWFhMJlMsFqtAIBz586hZcuWAPT9Slu0aFGKKSciIqIbFQPTcio6OhoGgwEGgwEX\nLlwAAFitVoSEhAAA0tLSMHXqVDzzzDPIzc0FIG/pWywWt2WZTKbSSzgRERHdsBiYlgNCCPz999/I\ny8sDAKSnpyMpKQk5OTkwGAyoWrUqzp07hwEDBmD58uUAgJdffhkJCQlISEhAdnY2AODIkSNuy779\n9tvRrVu30tsYIiIiumExMC0HNm3ahF69eqFy5coAgFOnTqFJkyba7fioqCicO3cOzZs31+bJzs7G\nvHnztAfqh4SE4PLly4iMjNQte+vWraW0FURERHSjY2BaDqSkpOCuu+7C2rVrAcj+oYsXL9bGf/fd\nd7hy5Qpq1KihDfvnn38AAM2bN8cff/yBy5cvA3D8Ep+IiIiotDEwLaOysrLwySefIDs7G2lpabDZ\nbNq4Tz/9FG3atNG+h4eHu80/fvx4dOvWDT///DP++OMPbXirVq1KNuFERERE+WBgWkbt27cPY8eO\nRXh4OG666SbUrl1bG3ffffcVOn+tWrVQq1YtXVBauXJl/ssTEREReQ2fY1pGZWVloW7duujVqxeO\nHTsGPz+/a1rOmDFjAAAjR47EkiVLijOJRERERFeFLaZlVHZ2NiwWC+69916kp6ejb9++17Sc0NBQ\nJCUlISQkhK2lRERE5FUMTMsoFZh26NABHTp0uK5lhYaGFlOqiIiIiK4db+WXUSowJSIiIiov2GJa\nBn322WdYsGABAgICvJ0UIiIiomLDwLQM+uSTT9CwYUOMHDnS20khIiIiKjYMTMuYo0ePYtWqVfjm\nm2/QokULbyeHiIiIqNiwj2kZs3fvXlSvXh2dO3f2dlKIiIiIihUD0zImOzsb7dq1g7+/v7eTQkRE\nRFSsGJiWMfw1PhEREZVXDEzLGAamREREVF4xMC1DPv/8c0yfPh3p6eneTgoRERFRsWNgWobce++9\n+Pvvv5Gdne3tpBAREREVOz4uqgyJiorChg0bUKdOHW8nhYiIiKjYMTD1Udu2bUOzZs0QHByMb7/9\nFgaDAXl5eQgLC+Mv8omIiKhcYmBaTHJzczFq1CgEBwfj/PnzGDRoEHr37o0LFy5gwoQJqF69OlJS\nUjB//nwYjfn3oNi8eTO6dOmifa9duzZOnDiBoKAg2Gw2mEym0tgcIiIiolLHPqbFZP369Th//jzm\nzZuHN954A48//jgAYOLEiYiNjcXMmTMREBCApUuXFricQ4cO6b6fOHECn376KTIyMpCbm8vAlIiI\niMottpgWk6ioKOzYsQM7duyAwWDAbbfdBgDYsGED5syZAwBo3749li1bhuHDh2vzCSGQlpaG8+fP\nY926ddi1axdCQ0N1y65WrRqCgoIghMCVK1cYnPqIlJQUbyeBrgPzr2xj/pVdzDsqCAPTIurevbvH\n4Rs3bgQANGvWDL169cLEiRNx4MABrFy5EgCQmJiIChUqAACCgoIQHx/vcTm7du3ChAkTdMPatm2L\nWrVqoVq1amjRogX27NlTYDcAIiIiorKMgWkRqQA0Py+//DI6d+6M9957D7/88gvuuece7Nu3DxER\nEbDb7QgMDERqaioiIyN18xkMBgQFBQEAUlNTkZubq43bsGGD9nnatGno1q0bKlWqxB8/+RjXFm4q\nW5h/ZRvzr+xi3pVNCQkJJbp8Nr8VE7vdrgWVt9xyC0JCQpCVlYWYmBhs374dABAXF4eYmBiP82dk\nZCAkJET7brPZdONzcnIAgLfxiYiIqNxii2kxeeqppzB27Fj8+eefiI+Px5QpUxAdHY1Zs2Zh9OjR\niIuLQ1paGoYOHeo27+rVq/HMM89o3zt16oRnn31WN83JkycBgH9HSkREROUWA9NiEhoaimXLlrkN\nDwsLw6pVqwqcNykpCQMGDMDgwYPx1VdfYdiwYejYsaPHaQ0GQ7Gkl4iIiMjXMDD1ES1atEBsbCxi\nY2O9nRQiIiIir2Af0zLilltuQaVKlbydDCIiIqISw8DUBwghCr1F36pVKyQmJpZSioiIiIhKHwNT\nIiIiIvIJDEx9QFFaTImIiIjKOwamREREROQTGJj6ALaYEhERETEwJSIiIiIfwcDUR7DFlIiIiG50\nDEzLstxcwGAA6tcHWraUn0eNkp+tVvn9zBng3Xfl5yNH9PP/8gsQGirHGQzA5s3yvWVLx8tgAG6/\nHViwQH6+9Vbgvfcc8wQEAEFB8nPr1sD27UCHDkCnTsDGjY7pDAbglVfketPTgY4d5fKjouTnr74C\nQkLk8tS6W7cG7rlHznPXXcDOnZ73w8KFQLNmcp7mzeXrtdfkuPHj5bqrVJHvf/4pp4+IkN/r1AEq\nVwYaNAA+/NCR1pYtgaVL5TIefFB+nzPHfd0LFsh1q/nq1gW6dwd+/dUxzcMPy3FffAH07w8cPuy+\nnDFj5DQ9egAtWsjPAwcCycn6fdi+vXxv184x72uvyfQ1aSLHNW0K/PWXHLd7txwWGwv8/jvQqpUs\nI0895Uivc54HBsrvly87hjdurC8TzmVD7UM1rHNnIDtbpmH5cpmGjAyZthUrgAce8JyHK1YA9eoB\nqan64Tk5siyptLiWT+d0qJcqqwYD8PLLwJ13OvLR+Rj4+WeZ3sWLgalTZT59+SXw3Xf6ZY8eDQwb\nJucZMkQO69MHuO02IDpa7tPwcMc6b75ZznvTTXJ4pUr6fS0EcPCgLN8tW8oyrsr5vn2O/B8/Xo73\n85PfVTl0PmYNBpkGgwGw2+WxZDAADz3kvp+mTAEqVJDpadVKTrdzp5z2vvuAW26Rw15/Xb7bbDL9\nmzfLfbhqlRz+8MMyrQYD0LChTLtKy969ctwrr8h1btzoOb+VhQuBNm2ARYuARo3021WliiyPzZs7\nhtWvD9SoATz7rNxute1PP60vC61aAcHBsj5ZuNBx3PTpAxw9ql/P2rUyLdOmOfL50CGZP+qYUvmX\nkCC3/9ix/LepSxfHMfHGG3JYzZrymHKmjsGWLYFq1WQZdM0zlZ8NG8r6ac0a4JFH5PxJSbLeOnxY\nlpXOneW0M2cCu3YBRiNgNgNLlshjqHJl4MknZd488ohc/jffAL/9JvfXnXc66vTUVFm+1bbn5cnt\nmjJFlpMOHWRaWreW40eOlO81a8r36tWBceMc9diQIXJ7W7WS6733XllHtmolpwPk+Uitz2jU59Gz\nzwJt2+qHrVzp2JfO9WSrVo56z2CQ62rZUtZJBoOjPnjpJbkdV67o82XTJv16VF588IEcr45ngwGY\nONEx36+/ymH9+8vvv/3mvhzn7wBw6pTcrv/8x70cqTrFuY6eMMFzmZszRx6HISH6uqtlS5nnANC3\nr2PcgAHy/cMP5biePeX0FSs6ziH//a8ct2aNnLZZM6BfP8/rL26CvOro0aPiySefFHPnzi36TC++\nKAQgRHS0fH/oISF27ZKfASEeftjxefNmx+evv9Yv5+mnHeMAIR54QL5/9ZWcVg03Gh2fmzUTwmJx\nfK9eXYiYGP1yatUSIiBAiFGj9MMBIVavFmLgQCFCQ4Xo0sUxvFcv+V6/vhDbt8vP3boJYTAIkZcn\nvy9d6r4vVq50LGPYMMfn2rXleNf1v/yyEA8+6D4cEKJGDcfnF14Qom5dIdauld/nzJHbqSQmiqTQ\nUJEUGipE9+7uy3rxRce0ruNiY+Vy/f3dpzGZ8k+T6+uTT4T45Rf5+bXX9OO2bJHLXbXKMWzZMiGa\nNtVP9+9/y/ddu+SrZUvHd+fpVqxwTKNezuPfflsOCwwU4q+/5LCICFk+2rfXT5uWps/DiRP12/T4\n40IkJMi8XbPGMW7GDPn+6qv5pyO/lyrbjz6qrTbp1Vdl/lWqpJ+2c2dZliZM0A93zct69Tyva/Fi\nIRYtch9+663y/dw5WWa6dhVi9GjH+Icecuw3QIgKFYQYOVK/jCZN3Jf7zDPyvWpV/fDJkx376Mkn\nPac1PLxow1xfZ87I94oV3cvJwYPyc3CwEM712ty5Qpw4IcTvvwsxfHj+aSriSzv+ACHCwvTjW7Vy\nfG7WTIjISMd3q1U/7UMPyfKr9t+RI0J8840QHTrkv/7bbxdi/XpZFoUQIjlZDh8xwjHN00/Lcvf4\n4/K7qj9+/dVRHgF92gYO1JdtdfybTEIEBenL4oED8nPPnkI0bqxP3223OT67HvPq1bOnELNmCbFk\niaP+dc7Hguqea30NHizErl0iqUkTmXd16ggREiLE2LFCPPXU1S1r3DiZBw88IETz5o7h3bvr6z1P\nr4ceEsLPT36OiBBi/ny5T3fscJSdkBD5vnSpzMvHH3evAzp0cJTvt9+WwypUkN+XLNFP+/PP+u+H\nDwuxbZv83KuXYzkvvSTEP/+4p7lDB5nmIUNk/TJunDzH2u366R55RG6H2k61jWazEDt36s9/lSo5\njlfXV//+Mj1Tp+rP5TVqiPj4eBEfH+9+Pi4mKLElU5EcPXpUjBs37uoCU9cCdPGiHL54sRCvvy7E\nqVNC7Nkjx73yinw3GOT7wYMyaPrsMyFuvtl9WTVqyEBQBYOAEH36OD7/9psQH37oqODPnhUiJaXg\nA9DTa8IEIe691334qVNyW776Sla8Vquj4r7vPvd98X//56hgXnpJvyx1En/sMRloV6jgOIk/9ph8\nV8sODXVs19y5QiQlCTF0qKxIAUeAf+GCEO++KwScTowrVwqxdas+qHzySVmBOFdOqqJzfl2+LE9U\n6vuGDUKsWyfEd9/Jk2PDhu7z3HmnDGhq13YM27JFP83ChUKMGSO3qVo1mf8LF+rzEhAiPV1WYsrf\nf8vh6mT222/yffduz+XQZpPlLj1dDrNahejbV46rX99z3p844VhGaqrnaX76yfFZXRht2ybEF1/I\nithZbGzh5a1ePVmxG43yeJk9WySNHOkIbFxfAwbI4FiVCUCI8+f1+1ddFACyDHz2mQy68vLk/vj4\nYyE++kie2BYtEmL6dDmtKid79jiW366d44TYubMsg6tXCxEXJ4epAF3ly9ix8n3JEiGysjxvw4UL\njn0UH+95GudjUJX15cvlCWzrViHuv7/g/Tpvnvuw55+X76rOOXBAiI4d5eeuXfVBGeB+Yev8+uor\nmY7Ro4X44QfH8AUL9IGpcyCyZo0MLp2X06OH+zESHCxPvq7rHDJELiM2VohNm9yPGZWPKlB8/HFH\nfji/VqyQ784XPkLIi2PnQFrVw5GRsl50tm2bDNhc67axY2XgarUKERWlr3uaN5fl6ttvZT0CyAtG\nNd5slu9PPCGPh/Bw/QUSIETNmkK0bu0InAAh7r7bPc3qtWSJvKjcubPg8vK/Yz/phx9k3vXsKcu5\nxSJElSqOC+WICLmfXOf/+mtZPnv3dgwbMcJRB6xZIxtmnnvOsZxly/K/iHReR61acrv695fnMBUc\n7t4tl+HnJ8QHH+jnv+02R16tWiXrWkCeZ/79bxkkv/WWEBs3yml27JBBXlSUrMt+/NGxrIQER72q\n6s7x4/XbCQjRqJFjWFCQrGPU9wYNhDh92rEclf/qsxCy0cV5G5wbDpzzuHp1Of2TT8o4Yt06IWbO\nFAJgYFreqcD05ZdfLvpMquD8+KM8EeZn8mRHhaVapTydaF54wdHC45yOixflOj7+WI57/33HuGPH\nhMjIcE/T0aP672++KYPX/fsdV+CPPCIDkvh4Odz55JjftuY3vn9/RyWlgvDWrfXzqKDp//7PMeyp\np4TYt08G6idPylYsIeT3vDz5WQVyERGOVlyn1uKk0FCRFBcnAwMhZKWrlu/aslVQy546aXhqEb54\nUQYw77wjK+SPPpIVz7ffOipTQIi9e+VJIS5OiDvu0LegDBsmK9UmTeTJaMsWWW4OHvRcbt5/X86n\nLgQOHPA83alT7kFiVJScd8wY2foDyJP78eMysKhd21FGhJDb4mmfzJqlP9GqE4QnGRmy5erLLz0v\n69FHZXlNT9cN1wU2gAxUVAB4zz2O5XtaZmKiY9yIEZ7T5Sonx70sq2DrzBkZTL38suMEJYQsi+rC\nQZ0IAXmS2LJFiNxcfRovXHC0/Lnas0fm5Q8/yH158qScfsMGOf2BA7IVx1lenhB//ilPuq+/Lter\nAk9AiOxsWQ7OnhVi0iTH8I4d3Vuc1cu1hTo9XdYDe/fK708/LcSUKZ63Qd0lEkIkbdsmkjZtkukX\nQuaJurAVQm6LqtcWLpTD+vWT33/9Va53zhz39EVFCVG5sjxJCyH38aFDsg5QeTFggPt806bpj3dV\nlnbskPtbbc/MmTIgU9N+/72jXn7gAc9lZ8ECOX7qVMd8P/wgL2h27ZL5AsgW7MuX9fOGhjou3qOj\n5fH30Uf6oGz6dJnHa9fKY+WLL4T44w85/2uvyYuDjAy5X1T9qYLruDj9+r7+Wl7Qzpsn83TXLlnX\nqwYUIURSUpJI+uUXRxCl7hIkJ8v0qW04cMARHE+f7ljH3LmOtG/eLMvhkSNy3OOPy+VZLPI8I4QQ\nly7J8a6B86FD+u/33ac//nbskPnv2viyYoW8EL3lFse0K1bozzF+fkK88Ybn/OzXT5aPTZtkYKj2\nq6r3Vq+W+1wIWe/Pny+P1XXr9HWmusiw2eTFiNpe57QnJsogW4mPl/n82WfuZfjSJbn/16+X3995\nR979fPttx/znzzMwLe+OHj0qnnjiiWsLTAtz6ZLjtr7zlZOnwqiW+9tv7ss5flyOS0nJf11VqsjK\nStmyRQZaKigUQojZs+VBmJmpn3frVrn8sDD35d5+uyOdkye7j7/3XsfV/t698rbd/PmyAvrXv+SV\np7J4sbw9D8hKoTBLlsjuBNWqyROvSkeLFjKw+c9/RFJSkn4e19tRAQGOE+edd+rHObcAAY7guCjU\nrUPVKuNcSYwZ4zhJqZPkjz/KbVYngoLk5cmTi/NJvqiGD5etE0lJjsDUWb16+uDn5EkhbrpJTtem\njX5/3HGHo0z++KMM7Apy7Jhj3v79HS2cx487punUSZsmKTRUJKmLpYoVZSvRiRNyHyUnO+ZxTpMq\na6oMA0I8+2zR988ff8h56tSR31XLsPMJJT8qMK1Wzf1iYfduR369+qr+1mBRADI4LIqMDHnx4twi\nK4T+luLChTLgAmRrpc3maJH94gtZX3iqxw4flhd6qk5wtWCBXJ74X3Djevy52rfPUTcIIVtgg4Md\n411bwL79Vt5ZAWRZzs/Zs/LE3ratY97sbNnarbrxbNsmW42FcNyBysuTgf2zzzq28cwZGaAA+i5A\nzi5fli30mZmO9fn7CzFokGMaQLZIu9qxQ2434OjiJITMvxkz5HKcgsYCqQBqwgS5HXfdVbT5XLjl\n3fbtMsDMz++/67sApafLC61vv3WvF/76S9Z1x455XlavXo4W7kuX5Ltq3f/55/zToC72J01yXDC2\naeMYv3y54wJDnbec7w45u+8+GXxu3CjzTNULhZU7ZfBg2dikLlhOnix8HlfqTkuXLnJZDz/sGHfl\niiM9kZGOGOF/GJiWcyownTdvXtFnKmpgWp48/bRsaXDVt688QZSEDz+UgUB0tMfRRTox3sg8BaYN\nGuiDquPH5S0jQLZuXQ91AVVEWv799JO8YKpaVXZlcAXIFiZPANm6d61UdwDnuw/5AWQLWUkAHC1k\npcH51uo1KrbjLzi4dOpTi0UGls88I2/NFzdAXkgXNN45ML0WquuTa5eDq+RTdScgzyNX6/ffZZ98\n5YMPZDeQohg4UHb/+e67aw7uvamkA1P+Kt8HCCG8nQTfZzLJpxC4ys2V40qCxQKcOyff6ep5egSa\nySR/4avk5sonSADyl7jeYLHIX1vb7fJXzJ4UdIzm5Fzfup3fvSk7u/TW5QvbqziXx5JksQCffgrs\n319y21/YueR6H0vorWO0pF3LfnE9J+XlFf1cpObNySm581cZlk8tTKXtqp5jWq2afBTKjcQbgWmL\nFvJxKM2alczyy7tJk+TjbpwZjZ4r82HD5GODvKF2bccj1ipVch/vaTuU115zPOrpWtSsCQwf7v0T\n/pAh8tFDpWXwYGD1avnYGm8rrcB0xAgZmBoM8jFFpe3tt+Ujo67H/Pm+kWfF6fXX5SObrpbRqC87\neXlFP47V+awkz19lGAPTsujuu+Wz5G4kZjOQleU+vCQP7AYN5DPc6Nq0aiVfzjy1mBqN8tm43nLT\nTcNhOb0AACAASURBVDJIyo+nZwwq6hmM1yosTD5H1dvU8wxLy7Rp8uULSuuO1fz5Jb8Ouz3/cY89\ndv3LV8/oLE93+caMubb5XBtLVF1W1HkTE4GzZ8vXviwmDEx9BP/5qRAmk3wYeI8e+gN540b54GYq\nG/JrMS0ON8IxxJNY8SutFtPS4PzHHlSyXC+yr/ZW/vjx8vPzzxd70so6BqY+gH1Mi0AFpk2aAJMn\ny2G//ioDU2/fBqWic739dTWtDIUJDS2e5dCNpbzUv5MnAz/+WPLrsVpvvDt2nni6yC5qXeY83fTp\nxZuucoCBqY9gi2kh8vLkD1SqVZN/T6qGAeyjU5ao219btsi/BizOFtOKFctPkEGlp7y0mHr6y+SS\nkJlZOuvxdSaTvBWvXM1Ftipz335b/OkqBxiY+gC2mBZBixby3flX0+qXrQxMy44dO+R/cBP5iut5\nqgLduKpUAdLTZRei224D4uKA6tWBN98sfN6aNR3LIDcMTH0EW0wLoa5EnYNQFaQyMC076tcHjhyR\nnwcMAD75xLvpKUueew64+WZvp4KIAMDfHxg0CGjaFHjmGTmsb9+izfvww0C9ekDz5iWXvjKMgakP\nYItpEajg0zkIPXRIvoeHl3566NocPixbGQYNkr+Eb9NG9h2mwr3wgrdTUH5FRno7BVQWrVwp31Vg\n+vjjRZuvZk1Hqym5YWDqI9hiWgjVYurpVn7t2qWfHrp2/v7A2rXy8zvvAB9/7N300I3tjTcYmNL1\n+e9/gQULGGwWEwamPoAtpkXg6Vb+kCFAcnL+/9ZDvq91a/ki8pZ//9vbKaCy7okn5IuKBZ+zQ2WD\np1v5VqvjWXBERERU5pXrwDQxMdHbSSgy3sovhGoxrVbNu+kgIiKiElOuA9MBAwbg559/9nYyqDio\nwLSoncuJiIiozCnXgemzzz6Lf/75B+PHj8e7776LK1eueDtJHgkh2GJaGBWYqh88ERERUblTrgPT\nOnXqoGvXrqhVqxZmzJiBe++9FzNnzkRSUpK3k0ZXS/1AjH8/SkREVG6V658z33XXXcjMzMTAgQPx\n/fffo169ekhOTsbgwYOxbt06bydPhy2mhXD+T2IiIiIql8p1YNq/f388//zzMDs9Tshms6Fy5cpe\nTJU7Pi6qCHgLn4iIqNwr1/dFK1asqAtKX3nlFdhsNrz//vveS1Q+2GJaiDZtgAMHvJ0KIiIiKkHl\ntsV0xowZ+OGHH2C32wEAWVlZ+OSTTzBhwgQvp4yuidHI/wknIiIq58ptYNqvXz/s378ftWrVghAC\nRqMRK1asuO7lnjt3DvPnz8eXX36JefPmoWfPngCACxcuYMKECahevTpSUlIwf/58GI1GfPXVV1i7\ndi1MJhPuvvtuxMbGelwuW0yJiIjoRlduA9MmTZpg5cqVumEnT5687uWGhYVh5syZ2LFjhy6YnDhx\nImJjYxEbG4vJkydj6dKlGDBgAMaNG4fDhw/DYDCgUaNG6NKlC0JCQnTLZB9TIiIionIamLZu3Rqr\nVq3Cf/7zH+Q6/Zr74MGD+OWXX65r2X5+fh6Hb9iwAXPmzAEAtG/fHsuWLUP16tVRt25drZ9ro0aN\nsHXrVvTu3VubT7Xm5uXlITk5uWiJMJmAnBz5P/HkNSkpKd5OAl0H5l/Zxvwru5h3VJByGZi+8847\nqF27NipVqoSePXtqLZLffPNNofN2797d4/CNGzcWOF9iYiIqVKgAAAgKCkJ8fLxuGABUqFAB8fHx\nbvOyxZSIiIionAambdq0AQDMnTtXN9w5SMxPYQFofiIiImC32xEYGIjU1FRERkZqw5TU1FRERUXp\n5jMYDMjLy4PZbEZoaGjRVpabC5jNQFGnpxJV5Hwjn8T8K9uYf2UX865sSkhIKNHll8vAtEePHsjK\nynIbfubMGRw9erTY1uPc0hkTE4Pt27cjNjYWcXFxiImJQfv27XHixAlkZ2fDYDDgwIED6NSpU4HL\nISIiIrpRlcvANCYmBv369XML+D7//PNiWf7cuXNx8uRJrF69GhEREWjbti1mzZqF0aNHIy4uDmlp\naRg6dCiMRiNee+01DB8+HEajEbNnz0ZQUJDHZfJX+URERHSjK5eB6ZgxY2D08J/qXbp0KZblT5o0\nCZMmTdINCwsLw6pVq9ym7dmzp/ZIqfywxZSIiIionAam1apVww8//IBx48bpbukX96384sQWUyIi\nIrrRlcvAdNeuXYiMjESvXr3Qp08fALJV8uuvv/ZyyjxjiykRERFROQ1MIyMjAQCPPvooVqxYgdOn\nT6NDhw7497//7eWU5Y8tpkRERHSjc++IWY488MAD+Pjjj5Geno7//ve/eO2117ydJCIiIiLKR7ls\nMVUyMzOxbt067fuECRO8mJqCscWUiIiIbnTlMjA9ffo0hBBo2rQpTp8+DUD24/TV4I99TImIiIjK\naWDas2dPhIeHAwC2b9/u5dQUja8GzURERESlpVwGpitXrkSTJk3chl+4cMELqSEiIiKioiiXgakK\nSvfs2YNvv/0WOTk5EEJg69at+OGHH7ycOs/YYkpEREQ3unL9q/ypU6ciLCwMZ86cQc2aNdG8eXNv\nJ8kj9jElIiIiKueBadeuXfHoo4+iRo0aGDp0KKKjo72dpMIdPQrcfTcwfTowebL8fOKEt1NFRERE\nVOLK5a185fDhw5gzZw6Cg4PRs2dPJCQkYMyYMd5OlrtLl2B4+WUgMBAICAB+/x1Yv16Oi4gAjhwB\natcG0tMBi8W7aSUiIiIqIeU6MF20aBEyMzNhsVhQr149372Vn5oKHDwIbNsGdO4MhIcDiYlyZMWK\nQHa2/BwSAjzyiPcSSkRERFSCyvWt/PXr16NBgwbw8/PDrFmzkJGR4e0kefR3UhIMgAxAs7OBChUc\nI202wG4H8vLkuEWLvJVMIiIiohJVrgPTKVOm4OOPP0Zqaipee+01zJgxw9tJ8mj/+fOoZTQCCxcC\n//d/8va98scfwP33AyYTEBzsvUQSERERlbByfSu/S5cuuPXWWwEAbdq0Qe3atb2cIs9C/P1xs9UK\nqBZd58DUWUqKfF+8OP9piIiIiMqochmYDh8+HHl5eTh8+DAeceqT+eeff2LatGleTFn+DGanrOjT\nBxg5EmjfXj/R778D/foBZ84An34KzJxZuokkIiIiKkHlMjCtWrUqunbtCiFEmXhwvRDC0Ze0WjUg\nNtbxg6fmzeXtfACoVw+IiQE++wxYtsx7CSYiIiIqAeWyj+mLL76ITp064c4770RmZiZ+++03ZGdn\no1OnTt5OmkdCCBh69AC++kr+Oh+QfUoBwOiURcHBwDvvABcuAP/rokBERERUXpTLFlNl2rRp2Llz\nJ+rXr4/vv/8eW7Zs8dkfQBlsNqBXL8cAoxEIDQUaNwZ27/ZewoiIiIhKSbkOTAFg3bp12uepU6d6\nMSX5y/cPSZOS5OviReC770ozSURERESlrlzeylcuXryInJwcAEBWVhYuXrzo5RTlQwj8f3t3HlZV\nufd//LMBJwQ1UhzIKYkUjDpPWkqDpik4YGKlXWWG85jiHJ5OaVooamYOWc5anbRHcQjHHJ9SUztJ\nmkkaqKWIqSiIE7rv3x/83EcEx9S1N7xf18UVbNa693fvr/vqw73Wutc1z4S9777sc0sBAADyuXwd\nTBs1aqQqVaro0UcfVeXKldWwYUOrS7qm616kZa45pwoAAJBv5OtD+Z6envrhhx906NAh+fv7y8fH\nx+qSbg/BFAAAFAD5esY0KipKvr6+euKJJ+Tj46OUlBSrS8qT0Q1mTO32e1YLAACAVfL1jGlYWJhG\njBihatWqyRijNWvWaI6Trv953WB65Mi9KwQAAMAi+XrGdMeOHXJzc1NycrKSk5N1/Phxq0vKk7nR\noXpnvWgLAADgDsq3M6ZnzpzR3LlzValSJcdjJ06csLCi67vujOmlS/euEAAAAIvkyxnTWbNmqWzZ\nsqpdu7aCgoL0xx9/SJLTXvx0w0ubPvkk+zakAAAA+Vi+DKbffPONDh06pNTUVM2aNUsffvih1SVd\n1w0vfgoOliIi7lk9AAAAVsiXwTQoKEglSpSQJNWuXVuFCxeWJM2bN8/Ksq7rOrEUAACgQMiXwXTM\nmDEqU6aM42vixIkqU6aMOnToYHVpeTKSdL0ZUwAAgAIgX1781KNHD/Xq1SvX1e6ff/65RRVdnzHm\n+ofyAQAACoB8GUxHjx6d5+Nvv/32Pa7k5hFLAQBAQZcvD+W7HGM4lA8AAAo8gqkTuOFV+QAAAAVA\nvg6ms2bNkv2K+8wvWLBAC//meqCHDx/W4MGDVaNGDcXHxzse/+mnn9SrVy/17NlTYWFhSk1NlSQt\nXbpUnTp1UteuXRUXF3fNcYmlAACgoMvXwbRnz56qUqWKNmzYoO3bt2v06NE6fvy4Pvnkk9se8/77\n79f777+vcuXK5ZjlHDt2rJ588klNmjRJ//M//6NRo0YpPT1dUVFRmjJliiZPnqy33npLp06dyj3o\njW5JCgAAUADky4ufLouKitKIESMcF0P17t1br776qiZOnHjbYxYpUiTPxz/77DPHeqmlS5dWSkqK\ntmzZIn9/f3l4ZL/NgYGB2rhxo8LDwx37GWPkWby40t3dde7kyduuC/deenq61SXgb6B/ro3+uS56\nh+vJ18H02LFjWrp0qTZt2qSkpCT16NFDmZmZWrdunXr16pXnPo0aNcrz8dWrV1/3uTw9PSVJGRkZ\nmj9/vhYuXKj169fL29vbsY23t7fjEH8unGMKAAAKuHwdTMPDwzV9+nTFxsYqICBAo0aNUo0aNfTW\nW29dc58bBdDrOX36tLp06aJJkyapQoUKKlu2rE6fPu34fUZGhsqXL59jH5vNptOnT6uU3a7CpUrd\n9nPDOqXom0ujf66N/rkueueajh49elfHz9fBtHnz5vL399fOnTslSYMHD9bgwYPv2PhXLuB/5MgR\nDRgwQMOHD5e/v78++eQTRUZGKikpSVlZWbLZbNq9e7fq1auXexxxVT4AAEC+DqaTJk3S5MmT5e/v\nr71796pXr17q0aPH3x43NjZWycnJmj9/vsqWLatatWqpRYsWSk1NdZw/WqpUKXXv3l3jxo1Tx44d\n5ebmppiYGHl5eeU5JsEUAAAUdPk6mP7+++/65ZdfHD/36dPnjow7aNAgDRo0KMdjW7duzXPbZs2a\nqVmzZtcd7+pbpwIAABRE+Xq5KFfCjCkAACjo8nUwrVatmh555BG98MILCgoKUtWqVa0uKU/MlwIA\nAOTzQ/k9e/ZUgwYNtHPnTlWvXl2VK1e2uqRrYsYUAAAUdPl6xlSSatSoodatW+uLL77Qs88+a3U5\n10QwBQAABV2+njG90qhRo+Tr62t1GQAAALiGfDljOnXqVJ05cybHV2ZmprKysqwuLZfLV+QzYwoA\nAAq6fBlMu3btKi8vrxxf3t7e+uc//2l1addGMAUAAAVcvgymM2bMkN1uz/U1ffp0q0sDAADANeTL\nYBoZGXlLj1uJxfUBAACy5ctg6mpsEofyAQBAgUcwtRgzpgAAANkIphYzxnBFPgAAgAimToFD+QAA\nAARTy3EoHwAAIBvB1AlcIpwCAAAQTK126dKl7G84lA8AAAo4gqnFjDEq4uFhdRkAAACWI5g6Aa7K\nBwAAIJhaznHxE+EUAAAUcARTJ0AkBQAAIJhajuWiAAAAshFMnQAzpgAAAART58E5pgAAoIAjmDoB\nIikAAADB1HKcYwoAAJCNYOoEbDYbh/IBAECBRzC1GDOmAAAA2QimAAAAcAoEU4tx5ycAAIBsBFMA\nAAA4BYKpxTjHFAAAIBvB1AnYJA7lAwCAAo9gCgAAAKdAMHUCNmZLAQAACKZW46p8AACAbATTW3T4\n8GENHjxYNWrUUHx8fK7f//nnnypbtqwOHjwoSVq6dKk6deqkrl27Ki4uLs8xiaQAAACSh9UFuJr7\n779f77//vrZu3ZrrEPy5c+fUv39/x8/p6emKiopSYmKibDabAgMD1aBBA5UsWdKxDVflAwAAZCOY\n3qIiRYpc83f9+vVTdHS0tm7dKknavHmz/P395eGR/TYHBgZq48aNCg8Pd+xjt9tV3MtLJ2026eTJ\nu1s87qj09HSrS8DfQP9cG/1zXfQO10MwvUqjRo3yfHz16tXX3W/ChAmqU6eOHnvsMUnZM6HHjx+X\nt7e3Yxtvb2+lpqbeuWIBAADyEYLpVW4UQK9l3rx5KlOmjOLi4nT06FG9+eab6tOnj06fPu3YJiMj\nQ+XLl8+xn81m09nMTJWSpFKl/kblsEop+ubS6J9ro3+ui965pqNHj97V8Qmmf8OV54d+9913ju+r\nVq2qiRMnqnTp0kpKSlJWVpZsNpt2796tevXq5RqHBfYBAAAIprclNjZWycnJmj9/vnx9fVW7du08\nt/P09NS4cePUsWNHubm5KSYmRl5eXjm24eInAACAbATT2zBo0CANGjTomr9PTk52fN+sWTM1a9bs\nuuOxwD4AAADrmFqOBfYBAACyEUwBAADgFAimFuMcUwAAgGwEU2dgs3EoHwAAFHgEU4sxYwoAAJCN\nYOoEmCsFAAAgmDoPDuUDAIACjmDqBIikAAAABFPLcY4pAABANoKpE+DOTwAAAART50E4BQAABRzB\n1AkQSQEAAAimluMcUwAAgGwEUydg485PAAAABFOrGWOkCxckZk4BAEABRzB1ArYLF6Sff7a6DAAA\nAEsRTC3mOMe0Zk1rCwEAALAYwdRZFClidQUAAACWIphazDFjWrSotYUAAABYjGBqNbs9ex1TDw+r\nKwEAALAUwdRiJjMz+xuWiwIAAAUcwdRi5sKF7G/eeMPaQgAAACxGMHUCNjc3ydPT6jIAAAAsRTC1\nGgvrAwAASCKYWs8YcXYpAAAAwdQ5cOETAAAAwdQZEEsBAAAIppY7f/68Mux2q8sAAACwHMHUajab\nvNxoAwAAAInIana7inCOKQAAAMHUGRBLAQAACKaWM6xjCgAAIIlg6hSYMQUAACCYWo4ZUwAAgGwE\nU2fAxU8AAAAEU6sZ1jAFAACQRDC9ZYcPH9bgwYNVo0YNxcfH5/jdmjVr1LZtW3Xt2lWXLl2SJC1d\nulSdOnVS165dFRcXl+eYzJcCAABIHlYX4Gruv/9+vf/++9q6datsVxyC37p1qz788EMtXLhQRYoU\nkSSlp6crKipKiYmJstlsCgwMVIMGDVSyZEnHfpxhCgAAkI1geosuh86rDR48WFWrVlWnTp1kjNHo\n0aP1888/y9/fXx4e2W9zYGCgNm7cqPDwcMd+dmNUtHhxnTx58p7UjzsnPT3d6hLwN9A/10b/XBe9\nw/UQTK/SqFGjPB9fvXr1dffbtm2bpkyZoocfflixsbHq27evXnjhBXl7ezu28fb2Vmpqas4dOccU\nAABAEsE0lxsF0GspUqSI7r//fklSw4YN9fnnn6tz5846ffq0Y5uMjAyVL18+x342m00XMjNVqlSp\n2y8alqJ3ro3+uTb657ronWs6evToXR2fi5/+hivXIK1Tp462b98uSdqzZ49CQkIUEhKipKQkZWVl\n6eLFi9q9e7fq1at3zTEAAAAKMmZMb0NsbKySk5M1f/58+fr6qnbt2vroo4/01ltvaf369Tp69KjG\njBmjYsWKady4cerYsaPc3NwUExMjLy+vnIMRTAEAACRJNsOUnaWWTJumj957T2sPHrS6FNyiyxes\ncTjKNdE/10b/XBe9c22XD+X7+vrelfE5lG8x/i4AAADIRjB1AiywDwAAQDC1HjOmAAAAkgimTuHK\nO0gBAAAUVARTi3GOKQAAQDaCKQAAAJwCwdRihluSAgAASCKYOgXOMAUAACCYWo4ZUwAAgGwEUyfA\njCkAAADB1HJckw8AAJCNYOoEmDEFAAAgmFqOdUwBAACyEUydADOmAAAABFPrGSNxS1IAAACCKQAA\nAJwDwdRinGMKAACQjWBqMWO3c44pAACACKZOwcY5pgAAAARTy3EoHwAAQBLBFAAAAE6CYGoxLn4C\nAADIRjB1ApxjCgAAQDC1HPOl99bChQvVtm1bVaxYUWfOnLGsjrCwMI0cOfK62+zcuVMNGzZUhw4d\n7lFVAABYy8PqAsAtSe+VI0eOaNiwYUpISNBvv/0mT09Py2qJiopSpUqVrrvNI488omeffVb79++/\nN0UBAGAxgqnFjN1udQkFxoYNG+Tj4yNJCggIsLSWsLCwm9qOc5ABAAUJh/KdgKvNmE6ZMkUVK1ZU\nZGSk6tatKz8/P82fP1+S1K5dO3l7e2vhwoXy8fFRfHy8jhw5otdee00DBgzQyy+/rBkzZtxwnKys\nLA0aNEjdu3dX586d1a9fP2VlZenixYuKjIxUz5491ahRI33zzTeSpBkzZuj1119XmzZt1Ldv31w1\np6ena+bMmdqzZ486dOig7du3KyQkRO3atVNkZKTKli0rSVqxYoVat26t/v37KyIiQomJiTp//rwi\nIyPl5+engQMHqnbt2goODtZnn32m7t27y8/PT9OmTcvzvTp8+LDatGmjwYMHq379+lq1apWWLVum\n6tWra/bs2Tp37pxat26typUrq3v37qpUqZKeffZZ/fnnn44xfv/9d73xxhvy9fVVy5YtHWF1z549\nevXVV9WtWzc1adJEBw4c0Lfffqvg4GANHDhQTZo00RNPPHGHug4AwD1gYKkvYmNN86pVb27j7FVP\n7+7XTapSpYqJj483xhgzffp04+npaY4ePWp2795tbDabmTt3rtm1a5c5ePCgCQ0NNWPHjjXGGJOZ\nmWlKly5ttmzZcs1xUlNTTUxMjGnevLnj+cLCwkxMTIxZt26d8ff3N8YYk5CQYNasWWO+++47U6tW\nLce2/v7+Zu3atblqnjVrlqlfv74xxpiLFy+aIUOGmMDAQLNz506zZs0ak5ycbEqUKGFSUlKMMcZ8\n9dVXJjAw0NjtdrNu3TpTsWJFY4wxdrvdBAQEmIkTJ5q0tDQzd+5cU7NmzTzfp0aNGpnx48cbY4zp\n06eP4/v69eub2bNnG2OMWb9+vfHz8zNZWVnmwoULpmHDhubFF180xhjz7rvvOr7/66+/jKenp9mx\nY4e5ePGiqV69utm5c6cxxpjhw4ebdu3amaysLNO2bVvzzDPPmH379pn169ffbEsLnLS0NJOWlmZ1\nGbhN9M910TvXlpqaalJTU+/a+MyYWs2Ym78q/15E01vg5eUlSYqMjJS7u7u2bNmiYsWKSZLatm2r\noKAg+fj4aPXq1QoNDZUkeXp66plnntGiRYuuO05cXFyOw91NmjTR4sWLFRQUJLvdriZNmigrK0sN\nGjTQkiVLZIxRdHS0oqOjVb16daWnp+fx9v339bm7u6tQoUJ68sknVbNmTTVo0EDLli1TQECAypUr\nJyn7cPuvv/6qvXv3OvaRsldRqFChguPnBx54QKdPn871fGfOnNHatWvVqFEjSdJHH32k3r1751lX\noUKF5OHhoUKFCqlbt25au3at4/fe3t6SpNKlS8vX11enTp3Sb7/9pqSkJH355ZeKjo5WUlKSihYt\nKg8PD3l4eOj5559XtWrVVK9evet0EAAA58I5pvjb3Nzc5OHhkefFRHa7Pdd5ksaYPM+dvHKcq/cz\nxshut6tMmTL65ZdfNGnSJDVs2FBz5syRMUbBwcGKiYm5pbptNluu58irLvttngd86dIl2e12ZWRk\n3NJ+hQoVUvHixa+7jd1ul81m0/Dhwx0BGQAAV8eMqdWMcblzTC+7HOLWrl2rokWLqk6dOrm28fb2\nVv369bVixQpJUmZmpr7//nu1bNnymuPUrVtX4eHhWrlypWObFStWqFWrVlqwYIG2bdum/v37q337\n9tq1a5dCQ0O1ePFiHTx4UFL2TOW+fftuuv7LQkNDlZiYqJSUFEnS8uXLFRQUpIcffjjPffMKsVe/\n9meeeUYfffSRY9uTJ0/ecP8FCxbkeH/yeu7q1aurfPnymjx5suPxhISEm64NAABnxIypxVw5QIwb\nN05fffWV9u/fr6VLl6p48eJ65513ZLPZ9N5772nIkCHy8PDQ559/rj59+qh3795KSUnRmDFjcoTY\nvMZ56623NGjQIHXo0EE2m001a9ZU//79tWbNGg0dOlSPPfaYDh8+rH/961/y8fHRu+++q9DQUFWo\nUEFly5bVO++8k6PWw4cPKy4uTklJSZo3b54ee+wxrV27Vunp6YqLi1NERIT8/f31xRdfqFu3bqpa\ntaoOHjyouLg4nT17VjNnztSJEye0ZMkSlSpVSklJSYqPj1dYWJhmzpyptLQ0ff3113r55ZdzPO/c\nuXPVpUsXBQUFKSAgQC+++KLKlCmjpKQkLVq0SM8995wk6dixY3rzzTd1/PhxlShRQiNHjlRiYqLW\nrVunU6dOKSEhQcnJyTp27JhmzZqlp556SnFxcerZs6fmzJkjPz8/vfDCCzp37px+/PFH7d+/X888\n84waNGhw9/8hAABwh9iMKyejfOCLmBjNnz5di29ihs+ZVK1aVbNnz9azzz7rFONY4fLsZ6lSpf7W\nOOvXr1f79u2VnJx8J8rCTbpT/YM16J/roneu7ejRo5IkX1/fuzI+h/It5sp/Fdypv2n42wgAAEgE\nU6fgaueYfvzxx467KB04cMDycVzZmTNn9MEHH+jIkSN6++23rS4HAABLcSj/Fh0+fFjjx4/XkiVL\nNGbMGDVr1kyS9Msvv2jYsGHy9/fX77//rokTJ6pMmTJaunSpFi9eLHd3d4WFhSkiIiLHeHNjYrRg\n+nQtcrFD+eBwlKujf66N/rkueufa7vahfC5+ukX333+/3n//fW3dujXH+qPDhg3TK6+8olatWunD\nDz/Uxx9/rIEDByoqKkqJiYmy2WwKDAxUgwYNVLJkyf8OaIx0s+uYAgAA5GME01tUpEiRPB9/4IEH\nNHPmTD333HPav3+/GjdurM2bN8vf318eHtlvc2BgoDZu3Kjw8HDHfsYYFfb0dPwFCdeR1yL+l20Z\nkAAAGpVJREFUcB30z7XRP9dF73A9BNOrXL5Lz9VWr1593f26d++u7t27q06dOipTpozeffddrVy5\n0nHXHil7XcvU1NQc+/2VnKxLly79/cIBAABcHMH0KjcKoNfy6quvasOGDSpSpIiio6P1xhtvqG/f\nvjluVZmRkaHy5cvn2K9uixaqERzMuTYujN65Nvrn2uif66J3runyOaZ3C1fl/w1XXjd2+baT7u7u\naty4sTIzMxUSEqKkpCRlZWXp4sWL2r17d657l/sGBiqgadN7WjcAAIAzYsb0NsTGxio5OVnz58+X\nr6+vateurYkTJ6pz584qX7689u/fr8mTJ6tYsWIaN26cOnbsKDc3N8XExMjLy8vq8gu0hQsXauHC\nhdqwYYMSExPl6elpdUkAAOD/Y7koi12+p7u/v7/FleR/R44cUWhoqBISEvTbb78pICDgb413s0ue\nLFq0SC1btrzt57l8y1TcWSxZ49ron+uid66NOz8Bd8iGDRvk4+MjSX87lN6sH374QR9//PFt7//H\nH38oOjr6DlYEAIDzIpjilk2ZMkUVK1ZUZGSk6tatKz8/P82fP1+S1K5dO3l7e2vhwoXy8fFRfHy8\njhw5otdee00DBgzQyy+/rBkzZtxwnKysLA0aNEjdu3dX586d1a9fP8e5upGRkerZs6caNWqkb775\nRpI0Y8YMvf7662rTpo369u2bq+b09HTNnDlTe/bsUYcOHbR9+3aFhISoXbt2ioyMVNmyZSVJK1as\nUOvWrdW/f39FREQoMTFR58+fV2RkpPz8/DRw4EDVrl1bwcHB+uyzz9S9e3f5+flp2rRpeT7nP//5\nT+3Zs0edO3fWmjVrlJ6eri5duqhHjx5q3Lixvv32WyUmJiokJERlypRRYmKioqKiVKtWLe3evVv9\n+/fX4cOH1blzZ8d7AwBAvmVgqb1795q9e/fe1LaS7vrXzapSpYqJj483xhgzffp04+npaY4ePWp2\n795tbDabmTt3rtm1a5c5ePCgCQ0NNWPHjjXGGJOZmWlKly5ttmzZcs1xUlNTTUxMjGnevLnj+cLC\nwkxMTIxZt26d8ff3N8YYk5CQYNasWWO+++47U6tWLce2/v7+Zu3atblqnjVrlqlfv74xxpiLFy+a\nIUOGmMDAQLNz506zZs0ak5ycbEqUKGFSUlKMMcZ89dVXJjAw0NjtdrNu3TpTsWJFY4wxdrvdBAQE\nmIkTJ5q0tDQzd+5cU7NmzTzfpyuf0xhjOnXqZD7++GNjjDEbN240lSpVMsYYk5qaakqXLm2WL19u\n2rRpY1JTU40xxqxfv95UqVLlpvuCm5eWlmbS0tKsLgO3if65Lnrn2lJTUx3/j7obmDF1IcaYu/51\nKy5fyBUZGSl3d3dt2bJFxYoVkyS1bdtWQUFB8vHx0erVqxUaGipJ8vT01DPPPKNFixZdd5y4uDiF\nhYU5tmnSpIkWL16soKAg2e12NWnSRFlZWWrQoIGWLFkiY4yio6MVHR2t6tWr57mA85Wvz93dXYUK\nFdKTTz6pmjVrqkGDBlq2bJkCAgJUrlw5SVJYWJh+/fVX7d2717GPJNlsNlWoUMHx8wMPPJBjWbBr\nPackLVmyRD/99JOio6O1cOFCVa5cWRcuXJCvr6/Gjh2rF154Qe3atXOcu3OrPQEAwJVxVT7+Njc3\nN3l4eOR5hbvdbs8Vrq4Vgq8c5+r9jDGy2+0qU6aMfvnlF02aNEkNGzbUnDlzZIxRcHCwYmJibqlu\nm82W6znyqstut9/SuNdjjFH79u31zDPP5Ppd6dKldd999yk+Pl5NWUIMAFAAMWOK23Y5xK1du1ZF\nixZVnTp1cm3j7e2t+vXra8WKFZKkzMxMff/99zmuUr96nLp16yo8PFwrV650bLNixQq1atVKCxYs\n0LZt29S/f3+1b99eu3btUmhoqBYvXqyDBw9Kks6cOeNY7eBm6r8sNDRUiYmJSklJkSQtX75cQUFB\nevjhh/Pc92ZmM93c3HTixIkczzFhwgTHvnv27NGFCxd0+PBhTZs2Tdu2bdO///1v/d///Z9j/1On\nTjFzCgAoEAimuG3jxo1T9+7dNXr0aC1dulTFixfXhAkTZLPZ9N577+nixYuSpM8//1xbtmxR7969\nFRkZqTFjxuQIsXmN89Zbb6latWrq0KGDOnbsqJo1a6p///7y8vLS0KFDHRcFdevWTQ0bNtS7776r\n0NBQNWzYUJ06dXI892WHDx9WXFyckpKSNG/ePCUmJmrt2rX66aefFBcXJyl7ya4vvvhC3bp1U1RU\nlObPn6+4uDidPXtWM2fO1IkTJ7RkyRJt3LhRSUlJio+PV1pammbOnKm0tDR9/fXXud6j2rVrKyUl\nRS+//LLWrl2r8ePHS5IeeeQRtWjRQjNmzNDOnTvVokULlS1bVmXKlFHlypUVGRmp9evXKygoSCVK\nlFBYWJgWLFhwt1oJAIBTYB1Ti7nqOqZVq1bV7Nmz9eyzzzrFOFZgLT7XRv9cG/1zXfTOtbGOKZzW\nnfqbhr+NAACARDDFbfj444915MgRDRs2TAcOHLB8HAAAkD9wKN9irnooHxyOcnX0z7XRP9dF71wb\nh/IBAABQIBBMAQAA4BQIpgAAAHAKBFMAAAA4BYIpAAAAnALBFLfsf//3f1W9enXNnj3b6lJyeO+9\n9+Tj4+O4Nent+u6779S+fXuVLVtW+/fvvzPFAQCAGyKY4pa99NJLKl++vGw2m9Wl5PDOO+84liG5\nXefPn1dkZKSmTp2qTZs2qVKlSjl+v2PHjru+5ur+/fuVkJBw2/ufPHlSGzZsuIMVAQBwbxBMke/8\nnaV5d+zYoUuXLsnDw0PVqlWTm9t/PyJ2u11vv/32XZ9FjY2N1Y4dO257/08//VTr16+/cwUBAHCP\nEEzxt2VkZKhLly6KiorS66+/rg8++MDxu5EjR6pXr16KiIiQm5ubunTpokOHDuXYf+jQoSpTpox6\n9eql4OBg+fv7a8OGDTp9+rQiIiJUvXp1TZw4UV5eXtq1a5f27t2rNm3aaMCAAYqIiFB8fHyO8UaO\nHKnnnntOwcHB+vHHH/Osec6cOXr11VfVr18/tW7dWikpKZKk8ePH69ixY2rfvr22bNmSY5/58+dr\n48aNGj16tPr06aMFCxbo8ccf16hRo/TEE0/o5Zdf1tixYx2nE6SkpKhVq1Z6/vnnJUkXLlzQgAED\n1K1bNzVp0kRffPFFrrpWr16tRYsWaebMmercubPOnTunLVu26JVXXlHnzp3VsmVLpaWlac6cOSpV\nqpTCw8N15MgRBQYGqkePHtqxY4dmzJihJUuWqHPnzvrzzz9vvaEAAFjFwFJ79+41e/fuvaltpbv/\ndbPq169vZs+ebYwxpmvXrubNN980xhhz6dIlExQUZP7973+bPXv2GB8fH8fj9913n/nPf/6T53g2\nm8388ssvxhhj/vWvf5kKFSqY8+fPm+XLl5sSJUqYpUuXmm3btpnjx4+bwMBAs2DBAmOMMX/88Yfx\n8vIyBw4ccIxz+fvo6GhTo0aNXM+1adMmU65cOXP27FljjDEjR440oaGhxhhj1q9fb6pUqXLN112l\nShWzYcMGY4wxqamppm7duuall14yBw4cMN9//32uGmbNmmXq169vjDFmxIgRpl+/fsYYYw4ePGg8\nPT1NZmbmdd/bkydPmooVK5q//vrLGGNMx44dzTvvvGOMMWbq1KkmICDAbNmyxfTo0cOxf2RkpBk2\nbNg1XwOypaWlmbS0NKvLwG2if66L3rm21NRUk5qaetfGZ8bUhdyLaHo74uLiFBoaKklyc3NT48aN\ntXjxYhUtWlQXL17UqVOnlJ6ersKFC6tatWrXHMfLy0uS9OabbyolJUWJiYkqWrSofHx81Lx5c9Wq\nVUtHjhzRr7/+6ni+Bx54QA899JBWrFiRa7w2bdpoz549SktLy1Xv008/raJFi0qSwsLC9O233+rc\nuXO3dBpA4cKF5e7urvDwcFWqVEkhISG5trlyvCVLlig5OVnR0dGaPHmyHn/8cR07duy6z7Fp0yad\nO3dOY8eOVXR0tM6ePes4t7dTp07y8/NT27ZtNXbs2Gs+LwAArsLD6gLg+ux2e44gZIyRMUaVK1fW\nF198oZYtW+rpp5/WypUrVaJEiRuOV7hwYUmSp6enjh8/nuu5rmaMuebj7u7u8vb2zvX41cEtr8fu\nNGOMIiIi9Prrr9/0Pna7XV5eXoqJicnz95UqVdKOHTuUkJCgJ5980vG4s12YBgDAzWDGFLflyiAX\nHh6ulStXSpIuXbqkb7/9VhEREZKkYsWKqXjx4goLC9NDDz10wzElacGCBXrkkUfynF19+OGH5e/v\n75gh/fPPP/X777+rSZMmucZZsmSJXnzxRXl45Pz7q3nz5vruu+909uxZSdLy5cvVuHFjFStW7Iav\n283NLUdYzivQent76/Dhw7n2DQ0N1aeffqrz5887aj9x4sR1n6NOnTpKS0vT0qVLHc+3c+dOSdJn\nn32mkJAQDR8+XB06dNCFCxfyrBEAAFfhPnTo0KFWF1GQXQ4mPj4+Fldy8+Li4vTll1/q1KlTevLJ\nJxUREaHFixdr1apV+uqrr9S4cWN17dpVUvYV4sePH9fXX3+t6Oho+fn56dFHH8015rBhw3Ts2DGt\nXLlS27dv18yZM1WsWDG9//772rFjh4oWLao6derI3d1dzz//vD788ENt3rxZX331lUaNGqVatWo5\nxjlx4oQWLFigjIwMTZgwwXHI/rIqVaqoVKlSio2N1aZNm5SUlKRPP/1UNptNI0aM0E8//SQfHx/H\nmFf69ddfNWXKFKWkpOjMmTOaN2+eMjIy9NBDD6lChQqSsmc5hw4dqgMHDmjv3r3avn27HnvsMb36\n6qv68ccfFR0dreXLl2vnzp0KDQ1VkSJFcjzHoUOH9NFHH+nQoUOqV6+e6tWrpyFDhmj27NlatmyZ\nqlSpovXr1+tf//qXunTpIj8/P3344YdKSEhQ06ZNdf78eY0ePVoHDx5U9erVdd99992Rvuc3586d\nk6Rc/z7gGuif66J3ri0zM1OSVLx48bsyvs1wMpql9u3bJ0ny9/e3uJI778svv9Ty5cs1d+5cSdlX\nwo8ZM0Y///xzrm3d3Ny0f//+XOuGOrPLa6aWKlXK4kpwO+ifa6N/roveubajR49Kknx9fe/K+BzK\nx12Tnp7u+AcsSWfOnFGLFi2uuT1/IwEAULARTHHXvPHGG/Ly8tILL7ygIUOGKD09XcOHD8+1XXR0\ntGw2m/r06aPTp09bUCkAAHAGHMq3WH4+lJ/fcTjKtdE/10b/XBe9c20cygcAAECBQDAFAACAUyCY\nAgAAwCkQTAEAAOAUCKYAAABwCgRTAAAAOAWPG2+CK50/f15DhgxRoUKFtHXrVvXu3VstW7bUX3/9\npf79+6tixYpKT0/X+PHj5ebmpqVLl2rx4sVyd3dXWFiY4x7yAAAAyIlgeos2btyoHTt2aM2aNfrt\nt99Ut25dtWzZUgMHDlRERIQiIiI0ePBgzZw5Uy+//LKioqKUmJgom82mwMBANWjQQCVLlrT6ZQAA\nADgdguktatSokZ544glJUunSpeXu7i5JWrVqlUaOHClJCgkJ0Zw5c1SxYkX5+/vLwyP7bQ4MDNTG\njRsVHh7uGO/ixYs6cuTIPX4VuBMu36XKy8vL4kpwO+ifa6N/roveubasrKy7enMEgulVGjVqlOfj\nq1evdnx/ecYzJiZGI0aMkCQdP35c3t7ekrI/bKmpqTkekyRvb2+lpqbmGPfBBx9UyZIlHQEXrqNE\niRJWl4C/gf65Nvrnuuida7Pb7fLx8blr4xNMr3JlAL2eTz75RN7e3urSpYuk7FtznT59WsWLF1dG\nRobKlSvneOyyjIwMlS9fPsc4hQsXzvUYAABAQcRV+bfIGKPhw4ercOHCeuedd7Rq1SolJyeradOm\n2rRpkyRp8+bNatq0qUJCQpSUlKSsrCxdvHhRu3fvVr169Sx+BQAAAM7JZowxVhfhSiZNmqQhQ4ao\nQoUKkqS0tDStWrVKfn5+6tmzpypVqqTMzExNmDBBbm5uio+P17x58+Tm5qYWLVqoVatWFr8CAAAA\n50QwBQAAgFPgHFMLDR8+XKdOndLRo0c1ZMgQVa9e3eqSCrT9+/crODhYZcuWlSQ1bNhQI0aMUN++\nfVWpUqUbrk9rt9sVFRUlLy8v/fnnnxo9erTKli2rvXv36r333lP58uVVrFgxDRs2zOJXmn8cPnxY\n48eP15IlSzRmzBg1a9bsltYUvtWeTZ06VQkJCcrMzFSnTp301FNPWfwOuK68ejd06FBNmzZNxYoV\nk5R9hKpx48b0zgnltab3U089xWfPBeTVux07djjPZ8/AEuvWrTPh4eHGGGN2795tnn76aYsrQnJy\nsomMjMzx2BtvvGEWLlxojDFm0KBBZtq0aebUqVPmwQcfNFlZWebixYsmICDAnDx50sycOdP07t3b\nGGPMsmXLTNu2bY0xxtSrV8/85z//McYY07p1a/Ptt9/ew1eVv507d85kZWWZ+vXrm/j4eGPM3evZ\nvn37zGOPPWaMMSYtLc0EBATc65ebr+TVu6FDh5oNGzbk2I7eOadVq1aZBg0aGGOMSUxMNPfddx+f\nPReRV+/effdds379+hzbWdU7Ln6yyMqVKxUSEiJJqlGjhn755RdlZGRYXBV27Nihzp07Kzw8XN9/\n/71WrVqlunXrSspen3bZsmXasmWLY31ad3d3x/q0V25bt25dLVu2TGfPntX27dv1j3/8I8cYuDOK\nFCniWCf4srvVszVr1jjWMC5VqpQ8PT21c+fOe/hq85e8eidJn332mV588UV1795dp06d0ubNm+md\nE2rUqJEWLlwo6b9revPZcw159c5ms2nq1KlO8dnjUL5FTpw4IT8/P8fPXl5eOnr0aI51T3FvPfDA\nA/rhhx9UuHBhrV69Wi1atNDJkydven3a48ePOxaM9vb2Vlpamv766y8VLVo017a4e25lTeFb6Vnp\n0qVzjXH06NF79KoKhv79+8vLy0vGGL3xxht6++23FRISQu+c1OU1vUeOHKkRI0aod+/efPZcxJXr\nsb///vt67bXXVLx4caf47DFjapGr1zg9ffq049xGWMPDw0OFCxeWlP0Xpc1mk5+fn2Mm+3rr0179\neEZGhnx8fFS+fHmdPXvWsW16ejrr1t5lV/fhTvTsemPgzrn8PzubzaY2bdpo165d9M7JffLJJ/Ly\n8lLXrl357LmYK9djL168uCTn+OwRTC3StGlTbd68WZL066+/qmbNmtyezWKffvqpJkyYIEk6cOCA\nihcvriZNmjj6dL31aevXr5+jp5e3LVSokJ5++mn99NNPkqQtW7aoSZMm1rzAfM78/wVGbnZN4Vvp\nWdOmTRUaGqpt27ZJkk6ePKkzZ84oKCjIglea/1zuXcOGDR1HFHbs2KE6derQOydl8ljTOzAwkM+e\nC8ird8702WO5KAu99957On78uFJSUjRixAgFBARYXVKBtmfPHvXt21dPPPGEkpOT1adPH1WpUuWm\n16c1xqhPnz4qUqSIDh48qAkTJsjX11dJSUl666235OfnJy8vLw0fPtzql5qvxMbGavLkyapXr556\n9eqlBx988K71bNq0adq2bZvS09P15ptvOs4Tx+25unfbtm3Txo0b5e/vr7S0NI0ZM0bFihWjd04o\nrzW9ly1bptjYWD57Tu7q3p04cUJRUVFKSEhwis8ewRQAAABOgUP5AAAAcAoEUwAAADgFgikAAACc\nAsEUAAAAToFgCgBOZOHChSpRooT69et3x8fevn27Hn/88Ts+LgDcKVyVDwBOpmrVqoqPj9fs2bPV\no0cPVa5c+bbH+uSTTxQYGKh69erp0qVLSkpK0kMPPXQHqwWAO4dbkgKAEzLG6Ouvv9aFCxfUvHlz\nlStXTqNGjVKZMmXk6empIUOG6JVXXtH999+vn3/+WZ06dVJKSooKFy6s7du3q2vXrnr00Uc1bdo0\nBQcH69ixY9q6dauOHz+uadOm6dChQxoyZIiqV6+uPXv26IMPPlBCQoI6dOigzp07a82aNapRo4am\nT59u9VsBoAAhmAKAk6pcubL69u2rSpUqKSQkRFOmTFFwcLAeffRRde/eXa1atdLatWsdi1dPmzZN\nXbt21ffff68FCxYoLCxMwcHBat++vZ599lmVLl1as2bNkiQNGDBAL774ol566SXNmzdPAwcO1Jdf\nfqlixYpp+PDhGjp0KLfPBXDPEUwBwAXs3LlTa9as0bp16/TII4/o1KlTkqQHH3xQNptNJUuWVNmy\nZRUbG6sLFy7o4sWLuca48sytn3/+WYMHD5YkPfTQQ/r5559zbOvu7u64fzYA3CsEUwBwUjabTefP\nn5ckVa9eXY0bN3bcZ9put+uHH35wbLt7926NHDlSu3bt0vr16zV79uxcY1ypZs2aSkxM1GOPPeb4\nLwBYjWAKAE5k0aJFOnHihGbOnKnHH39cUVFR6t+/v6ZOnaq+ffuqfPnyqlChgrp3765169bp2LFj\n2rt3r6pUqSJvb2/Fxsbq0qVLSkhI0L59+/SPf/xDw4YN0+nTp7Vp0yb9+uuvSk5O1ujRozVkyBDt\n27dPe/bs0ahRo/Tdd9/p1KlTWr16tUqUKKGTJ09q9erVatSokdVvC4ACgqvyAQAA4BRYxxQAAABO\ngWAKAAAAp0AwBQAAgFMgmAIAAMApEEwBAADgFAimAAAAcAoEUwAAADgFgikAAACcAsEUAAAAToFg\nCgAAAKdAMAUAAIBTIJgCAADAKRBMAQAA4BQIpgAAAHAKBFMAAAA4BYIpAAAAnALBFAAAAE6BYAoA\nAACnQDAFAACAUyCYAgAAwCkQTAEAAOAUCKYAAABwCgRTAAAAOAWCKQAAAJwCwRQAAABOgWAKAAAA\np0AwBQAAgFMgmAIAAMApEEwBAADgFAimAAAAcAoEUwAAADgFgikAAACcAsEUAAAAToFgCgAAAKdA\nMAUAAIBTIJgCAADAKRBMAQAA4BQIpgAAAHAKBFMAAAA4BYIpAAAAnALBFAAAAE6BYAoAAACnQDAF\nAACAUyCYAgAAwCkQTAEAAOAUCKYAAABwCv8PPwKOoEnKKscAAAAASUVORK5CYII=\n" } ], "prompt_number": 50 }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Metropolis algorithm is kind of pointless for this application. It's really just jumping around looking for the most likely phrase. But since the likelihood of a message is just the sum of the log probabilities of the log probabilities of its component words, we just need to look for the most likely words of the lengths of the words of the ciphered message.\n", "\n", "If the message at some point is \"fgk tp hpdt\", then, if run long enough, the algorithm should just find the most likely three-letter word, the most likely two-letter word, and the most likely four-letter word. But we can look these up directly.\n", "\n", "For example, the message we encrypted is 'here is some sample text', which has word lengths 4, 2, 4, 6, 4. What's the most likely message with these word lengths?" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def maxprob_message(word_lens = (4, 2, 4, 6, 4), lexical_db = lexical_database):\n", " db_word_series = Series(lexical_db.index)\n", " db_word_len = db_word_series.str.len() \n", " max_prob_wordlist = []\n", " logp = 0.0\n", " for i in word_lens:\n", " db_words_i = list(db_word_series[db_word_len == i])\n", " db_max_prob_word = lexical_db[db_words_i].idxmax()\n", " logp += math.log(lexical_db[db_words_i].max())\n", " max_prob_wordlist.append(db_max_prob_word)\n", " return max_prob_wordlist, logp" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 64 }, { "cell_type": "code", "collapsed": false, "input": [ "maxprob_message()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 65, "text": [ "(['with', 'of', 'with', 'united', 'with'], -25.642396806584493)" ] } ], "prompt_number": 65 } ], "metadata": {} } ] }