{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "version": "0.3.2",
      "views": {},
      "default_view": {},
      "name": "Fitting MaxEnt models via persistent contrastive divergence",
      "provenance": [
        {
          "file_id": "0B99p0KvCr0FTX2hYdFRzQ2tTOUU",
          "timestamp": 1457490307613
        }
      ]
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6dHNH184MplP",
        "colab_type": "text"
      },
      "source": [
	    "Authored by: Patrick Mineault",
		"\n\n",
		"http://xcorr.net/",
		"\n\n",
        "Our aim is to fit a maximum entropy model for strings. We could use this to:\n",
        "\n",
        "  - Model spelling / create a spell-checker\n",
        "  - Generate fake words / RhymeMaster 3000\n",
        "  - Do computational linguistics\n",
        "\n",
        "Maximum entropy models are generative models of exponential form:\n",
        "\n",
        "$p(x|\\theta) =  \\frac {1}{Z(\\theta)} \\exp(\\phi(x)^T \\theta )$\n",
        "\n",
        "Where $Z(\\theta)$ is a normalization constant:\n",
        "\n",
        "$Z(\\theta) = \\int \\exp( \\phi(x)^T \\theta ) dx$\n",
        "\n",
        "The gradient of the log-likelihood will be:\n",
        "\n",
        "$\\nabla L \\equiv \\frac{\\partial \\log p(x|\\theta)}{\\partial \\theta} =  \\phi(x)^T \\theta - \\frac{ \\partial \\log(Z(\\theta)) } { \\partial \\theta}$\n",
        "\n",
        "Taking the partial derivative in the second expression, we find:\n",
        "\n",
        "$\\nabla L = \\phi(x)^T \\theta - \\mathbb{E} (\\phi(x))$\n",
        "\n",
        "The expected value on the right hand side is tricky to evaluate. The idea behind contrastive divergence and variants is to compute the expectation in this expression via several independent Markov chains. *Persistent* constrastive divergence means that we won't reset our Markov chains between gradient descent iterations -- although the distribution we need to sample from changes between GD iterations, it changes very slowly. One can prove mathematically that sampling this way converges to the correct ML estimate.\n",
        "\n",
        "For illustrative purposes, we'll limit our attention to sampling from fixed length strings -- sampling from non-fixed length strings is more annoying, but not necessarily more informative."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p83zKdZHg0KP",
        "colab_type": "text"
      },
      "source": [
        "# Sampling from fixed-length strings\n",
        "\n",
        "Let's start by creating a sampler over fixed-length strings. We will use a Metropolis-Hastings (MH) proposal within a Gibbs sampler. What this means:\n",
        "\n",
        "  - Gibbs: resample one character at a time at each iteration\n",
        "  - MH: sample a tokeb from a proposal distribution and adjust based on the density function\n",
        "    - Important life lesson: code the MH sampler on a log scale otherwise numerical problems ensue.\n",
        "  \n",
        "Let's do this a dummy density function and verify that we're sampling from the right distribution -- it's very easy to mess up and sample from the wrong distro. Don't do it!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7LLOV3ngxxFu",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": []
        },
        "cellView": "both",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1457681318876,
          "user_tz": 480,
          "elapsed": 843,
          "user": {
            "sessionId": "ab1f0339c96f28e",
            "userId": "103444865736961301998",
            "permissionId": "00759584880725894106",
            "displayName": "Patrick Mineault",
            "color": "#1FA15D",
            "isMe": true,
            "isAnonymous": false,
            "photoUrl": "//lh3.googleusercontent.com/-CdPTZM1TPbU/AAAAAAAAAAI/AAAAAAAAABQ/MOmbIGKtlQA/s50-c-k-no/photo.jpg"
          }
        },
        "outputId": "9c98dd8c-7f6c-4f8e-8f6d-ca9b0e3cbd56"
      },
      "source": [
        "import itertools\n",
        "import numpy as np\n",
        "import collections\n",
        "\n",
        "class FixedLengthGMHMC:\n",
        "  \"\"\"Fixed-length string Gibbs-sampling Metropolis-Hastings Markov chain. \n",
        "  This class samples from a distribution over string variables of length str_len. \n",
        "  It only works on ASCII (or ISO-8859-1).\n",
        "  It does that by Gibbs sampling, one character at a time. It uses Metropolis \n",
        "  Hastings to sample from the marginal distribution of the next character using \n",
        "  a proposal distribution log_proposal_dist. The Q function is density_fun. \n",
        "  \"\"\"\n",
        "  def __init__(self, \n",
        "               str_len,\n",
        "               proposal_dist,  # For individual letter\n",
        "               log_density_fun):\n",
        "    self.proposal_dist = proposal_dist\n",
        "    self.log_density_fun = log_density_fun\n",
        "    self.str_len = str_len\n",
        "    \n",
        "    self.num_accepted = 0\n",
        "    self.num_tried = 0\n",
        "    \n",
        "  def take_first_sample(self):\n",
        "    # Take a sample from the proposal distribution to start with\n",
        "    chars = []\n",
        "    for i in range(self.str_len):\n",
        "      chars.append(chr(\n",
        "        np.where(np.random.multinomial(1, self.proposal_dist))[0]))\n",
        "    \n",
        "    chars = ''.join(chars)\n",
        "    self.last_sample = chars\n",
        "    self.last_sample_log_density = self.log_density_fun(chars)\n",
        "  \n",
        "  def sample(self):\n",
        "    # Take a sample from the conditional distribution of the \n",
        "    # str_pos character via MH\n",
        "    # Pick which character to resample at random\n",
        "    str_pos = np.int(np.random.rand()*self.str_len)\n",
        "    \n",
        "    # Sample a char from the proposal distribution\n",
        "    new_char = chr(np.where(np.random.multinomial(1, self.proposal_dist))[0])\n",
        "    \n",
        "    # Splice it in\n",
        "    new_sample = (self.last_sample[:str_pos] + \n",
        "                  new_char + \n",
        "                  self.last_sample[str_pos+1:])\n",
        "    \n",
        "    new_sample_log_density = self.log_density_fun(new_sample)\n",
        "    \n",
        "    # We're doing everything on the log scale for numerical stability\n",
        "    alpha1 = np.exp(new_sample_log_density - self.last_sample_log_density)\n",
        "    \n",
        "    # Q(x | x') / Q(x' | x)\n",
        "    old_char = self.last_sample[str_pos]\n",
        "    alpha2 = (self.proposal_dist[ord(old_char)] / \n",
        "              self.proposal_dist[ord(new_char)])\n",
        "    \n",
        "    # Acceptance ratio\n",
        "    alpha = min(alpha1*alpha2, 1.0)\n",
        "    \n",
        "    # Draw a binomial sample to decide whether to accept\n",
        "    accept = np.random.rand() < alpha\n",
        "    \n",
        "    if accept:\n",
        "      self.last_sample = new_sample\n",
        "      self.last_sample_log_density = new_sample_log_density\n",
        "      self.num_accepted += 1\n",
        "    \n",
        "    self.num_tried += 1"
      ],
      "outputs": [],
      "execution_count": 0
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lqZnawsf1HjY",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            }
          ]
        },
        "cellView": "both",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1457681363139,
          "user_tz": 480,
          "elapsed": 1014,
          "user": {
            "sessionId": "ab1f0339c96f28e",
            "userId": "103444865736961301998",
            "permissionId": "00759584880725894106",
            "displayName": "Patrick Mineault",
            "color": "#1FA15D",
            "isMe": true,
            "isAnonymous": false,
            "photoUrl": "//lh3.googleusercontent.com/-CdPTZM1TPbU/AAAAAAAAAAI/AAAAAAAAABQ/MOmbIGKtlQA/s50-c-k-no/photo.jpg"
          }
        },
        "outputId": "e837aeea-d2e5-43b7-f105-f3085d606332"
      },
      "source": [
        "# Test out the sampler on an example density on the letters \n",
        "# ABab\n",
        "def sample_log_density(arr):\n",
        "  # Is the initial letter an uppercase letter?\n",
        "  phi0 = (arr[0] == 'A' or \n",
        "          arr[0] == 'B')\n",
        "  \n",
        "  # Does the second letter follow the first letter? e.g. 'ab'\n",
        "  phi1 = ord(arr[1]) - ord(arr[0]) == 1\n",
        "  \n",
        "  return phi0 * 1.0 + phi1 * 1.5\n",
        "\n",
        "# Compute the probability of every length 2 string containing the characters\n",
        "# abAB\n",
        "charset = 'abAB'\n",
        "str_len = 2\n",
        "\n",
        "ps = {}\n",
        "for the_str in itertools.product(charset, repeat=str_len):\n",
        "  the_str = the_str[0] + the_str[1]\n",
        "  ps[the_str] = np.exp(sample_log_density(the_str))\n",
        "    \n",
        "# Create a non-uniform proposal distribution, because subtle bugs can lurk \n",
        "# in the way we've coded up Q(x|x') / Q(x'|x)\n",
        "proposal_distro = [(chr(x) in charset) / (1.0 + float(x)) for x in range(128)]\n",
        "proposal_distro = [x / sum(proposal_distro) for x in proposal_distro]\n",
        "sampler = FixedLengthGMHMC(str_len, proposal_distro, sample_log_density)\n",
        "sampler.take_first_sample()\n",
        "\n",
        "samples = []\n",
        "# Take some samples\n",
        "for i in range(10000):\n",
        "  sampler.sample()\n",
        "  samples.append(sampler.last_sample)\n",
        "\n",
        "# Look at empirical distribution of samples\n",
        "cnt = collections.Counter(samples)\n",
        "\n",
        "# Concatenate the results\n",
        "distros = np.array([[y, cnt[x]] for x, y in ps.iteritems()])\n",
        "\n",
        "# Normalize\n",
        "distros = distros / distros.sum(0).reshape((1,-1))\n",
        "\n",
        "print 'String | Exact p(x)*1000 | Sample p(x)*1000'\n",
        "print '\\n'.join(['%s        % 8d         % 8d' % (x, y[0], y[1]) for x, y in \n",
        "                 zip(ps.keys(), np.round(distros*1000))])\n",
        "\n",
        "max_rel_disc = max(abs(.5 * (distros[:,0] - distros[:,1]) / \n",
        "                       (distros[:,0] + distros[:,1])))\n",
        "\n",
        "print 'Max relative discrepancy: %.3f' % max_rel_disc\n",
        "\n",
        "# relative discrepancy should not be much higher than 6%\n",
        "assert( max_rel_disc < .06)\n",
        "\n",
        "print ('Acceptance probability: %.3f' % (sampler.num_accepted / \n",
        "  float(sampler.num_tried)))\n",
        "\n",
        "# That looks good."
      ],
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "String | Exact p(x)*1000 | Sample p(x)*1000\n",
            "Ba              64               60\n",
            "BA              64               64\n",
            "bA              23               22\n",
            "aa              23               22\n",
            "bB              23               24\n",
            "Ab              64               63\n",
            "AB             285              287\n",
            "ba              23               22\n",
            "bb              23               25\n",
            "Aa              64               59\n",
            "AA              64               65\n",
            "BB              64               67\n",
            "ab             105              106\n",
            "aA              23               24\n",
            "Bb              64               65\n",
            "aB              23               24\n",
            "Max relative discrepancy: 0.020\n",
            "Acceptance probability: 0.728\n"
          ],
          "name": "stdout"
        }
      ],
      "execution_count": 0
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ypf_jz8s15Rx",
        "colab_type": "text"
      },
      "source": [
        "#Persistent contrastive divergence to estimate parameter values \n",
        "\n",
        "PCD is an algorithm for maximum likelihood estimation of MaxEnt models. It's essentially gradient descent with a twist: the gradient of a MaxEnt model includes a term corresponding to an expected value wrt. the distribution to be fitted. This value is approximated via parallel Markov chains. The Marjov chains are not reset every gradient descent iteration, hence the *persistence*."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "341Wc9X4e4hv",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": []
        },
        "cellView": "both",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1457678926649,
          "user_tz": 480,
          "elapsed": 678,
          "user": {
            "sessionId": "ab1f0339c96f28e",
            "userId": "103444865736961301998",
            "permissionId": "00759584880725894106",
            "displayName": "Patrick Mineault",
            "color": "#1FA15D",
            "isMe": true,
            "isAnonymous": false,
            "photoUrl": "//lh3.googleusercontent.com/-CdPTZM1TPbU/AAAAAAAAAAI/AAAAAAAAABQ/MOmbIGKtlQA/s50-c-k-no/photo.jpg"
          }
        },
        "outputId": "c46706a2-8ea8-476e-db70-cbff8d75c21e"
      },
      "source": [
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt"
      ],
      "outputs": [],
      "execution_count": 0
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fHG5Ls_30x--",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "both"
      },
      "source": [
        "class FixedLengthPCD:\n",
        "  \"\"\"Stores a run of persistent contrastive divergence. Some features of this \n",
        "  version of PCD:\n",
        "    - Uses a sequence of etas to gradient descent for better convergence.\n",
        "    - Uses RMSProp update rule\n",
        "  \"\"\"\n",
        "  \n",
        "  def __init__(self, \n",
        "               words, \n",
        "               feature_fun,\n",
        "               S_chains = 10):\n",
        "    \"\"\"Initializes PCD on a given vocabulary.\n",
        "    \n",
        "    Args:\n",
        "      words: a list of strings, all of the same size\n",
        "      feature_fun: a function which takes a string and returns an array of \n",
        "        features\n",
        "      S_chains: the number of parallel chains to use for PCD\n",
        "    \"\"\"\n",
        "    self.words = words\n",
        "    self.feature_fun = feature_fun\n",
        "    self.S_chains = S_chains\n",
        "    \n",
        "    self.init_measurements()\n",
        "    self.init_params()\n",
        "    self.init_chains()\n",
        "  \n",
        "  def init_measurements(self):\n",
        "    \"\"\"Make some measurements on the vocabulary\"\"\"\n",
        "    max_str_len = max(len(w) for w in self.words)\n",
        "    min_str_len = min(len(w) for w in self.words)\n",
        "    if min_str_len != max_str_len:\n",
        "      raise Error('This class only works for fixed-length strings')\n",
        "      \n",
        "    # Compute the empirical distribution of tokens\n",
        "    all_chars = ''.join(self.words)\n",
        "    proposal_dist,_ = np.histogram(np.array([ord(x) for x in all_chars]),\n",
        "                                   bins=range(128))\n",
        "    proposal_dist = proposal_dist / np.float(proposal_dist.sum())  \n",
        "    \n",
        "    # Moment matching\n",
        "    X = np.array([self.feature_fun(w) for w in self.words])\n",
        "    phi_c = X.mean(0)\n",
        "    \n",
        "    self.proposal_dist = proposal_dist\n",
        "    self.str_len = min_str_len\n",
        "    self.phi_c = phi_c\n",
        "    self.n_features = phi_c.size\n",
        "  \n",
        "  def log_density_fun(self, word):\n",
        "    \"\"\"Returns the log density of a word, i.e. the inner product of the features\n",
        "    and the parameters\"\"\"\n",
        "    phi = self.feature_fun(word)\n",
        "    return phi.dot(self.theta)\n",
        "    \n",
        "  def get_new_sampler(self):\n",
        "    \"\"\"Create a sampler from current set of parameters\"\"\"\n",
        "    s = FixedLengthGMHMC(self.str_len,\n",
        "                         self.proposal_dist,\n",
        "                         self.log_density_fun)\n",
        "    s.take_first_sample()\n",
        "    return s\n",
        "    \n",
        "  def init_chains(self):\n",
        "    # Initialize MCMC chains\n",
        "    chains = []\n",
        "    for i in range(self.S_chains):\n",
        "      s = self.get_new_sampler()\n",
        "      chains.append(s)\n",
        "    self.chains = chains\n",
        "      \n",
        "  def init_params(self):\n",
        "    # Initialize parameters\n",
        "    sd = .1\n",
        "    # Alternatively, we could use a crude estimate as an initializer\n",
        "    self.theta = sd * (np.random.randn(self.n_features) / \n",
        "                       np.sqrt(self.n_features))\n",
        "\n",
        "  def run(self, \n",
        "          etas = [0.1], \n",
        "          k_steps_per = 5,\n",
        "          T = 100,\n",
        "          rms_prop_alpha = .9,\n",
        "          display_every = 1):\n",
        "    \"\"\"Run a number of persistent contrastive divergence iterations.\n",
        "    \n",
        "    Args:\n",
        "      etas: a list of learning rates to use\n",
        "      k_steps_per: the number of MCMC steps to take per gradient descent \n",
        "        iteration\n",
        "      T: the number of gradient descent iterations\n",
        "      rms_prop_alpha: the decay rate of RMSProp being used\n",
        "      \n",
        "    Returns:\n",
        "      (thetas, gnorm, likelihoods): sequences of (parameters, \n",
        "                      norm of the gradient, likelihoods)\n",
        "    \"\"\"\n",
        "    \n",
        "    # Fudge factor for RMSProp\n",
        "    ff = 0.001\n",
        "    \n",
        "    thetas = np.zeros((T, self.n_features))\n",
        "    gnorms = np.zeros((T))\n",
        "    gnorms_adjusted = np.zeros((T))\n",
        "    likes = np.zeros((T))\n",
        "    running_g = np.ones((self.n_features))\n",
        "\n",
        "    for t in range(T):\n",
        "      which_eta = (t * len(etas)) / T\n",
        "      eta = etas[which_eta]\n",
        "\n",
        "      # Sample for a few steps\n",
        "      for s in self.chains:\n",
        "        for m in range(k_steps_per):\n",
        "          s.sample()\n",
        "\n",
        "      X_ = np.array([self.feature_fun(s.last_sample) for s in self.chains])\n",
        "      \n",
        "      phi_t = X_.mean(0)\n",
        "      \n",
        "      # Gradient\n",
        "      g = self.phi_c - phi_t\n",
        "      \n",
        "      # Gradient ascent\n",
        "      # Use an RMSGrad update for more consistent convergence\n",
        "      gg = eta* (g / (ff + np.sqrt(running_g)))\n",
        "      self.theta = self.theta + gg\n",
        "      thetas[t,:] = self.theta\n",
        "      running_g = running_g*rms_prop_alpha + g**2\n",
        "      \n",
        "\n",
        "      # Monitor convergence\n",
        "      gnorms[t] = np.sqrt((g**2).sum())\n",
        "      gnorms_adjusted[t] = np.sqrt((gg**2).sum())\n",
        "                          \n",
        "      # Evaluate model likelihood\n",
        "      like = sum(np.exp(np.array(\n",
        "        [self.log_density_fun(w) for w in self.words]))) / float(len(self.words))\n",
        "      likes[t] = like\n",
        "      \n",
        "      if t % display_every == 0:      \n",
        "        print \"Iteration %04d, log likelihood = %.8f\" % (t, like)\n",
        "\n",
        "    return thetas, gnorms, gnorms_adjusted, likes   "
      ],
      "outputs": [],
      "execution_count": 0
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TgSrXTSCnaQt",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "cellView": "both"
      },
      "source": [
        "def sample_feature_fun(arr):\n",
        "  # Is the initial letter an uppercase letter?\n",
        "  phi0 = (arr[0] == 'A' or \n",
        "          arr[0] == 'B')\n",
        "  \n",
        "  # Does the second letter follow the first letter? e.g. 'ab'\n",
        "  phi1 = ord(arr[1]) - ord(arr[0]) == 1\n",
        "  \n",
        "  return np.array([phi0*1.0, phi1*1.0])\n",
        "\n",
        "# Draw some words\n",
        "nsamples = 1000\n",
        "keys = np.random.multinomial(nsamples, distros[:,0])\n",
        "words = ps.keys()\n",
        "vocab = list(itertools.chain(*[[words[i]]*k for i, k in enumerate(keys)]))\n",
        "\n",
        "pcd = FixedLengthPCD(vocab, \n",
        "                     sample_feature_fun,\n",
        "                     S_chains = 100)\n",
        "\n",
        "convergence = pcd.run(T = 600, \n",
        "                      etas = [.1, 0.03, .01],\n",
        "                      k_steps_per = 1,\n",
        "                      rms_prop_alpha = 0.9,\n",
        "                      display_every = 100)\n",
        "\n",
        "# Plot some indices of convergence\n",
        "plt.subplot(411)\n",
        "plt.plot(convergence[0])\n",
        "plt.title('theta')\n",
        "plt.subplot(412)\n",
        "plt.plot(convergence[1])\n",
        "plt.xlabel('Iteration')\n",
        "plt.title('||g||')\n",
        "\n",
        "plt.subplot(413)\n",
        "plt.plot(convergence[2])\n",
        "plt.xlabel('Iteration')\n",
        "plt.title('||g_a||')\n",
        "\n",
        "plt.subplot(414)\n",
        "plt.plot(convergence[3])\n",
        "plt.title('likelihood')\n",
        "\n",
        "print \"Actual parameters\"\n",
        "print np.array([1.0, 1.5])\n",
        "print \"Estimated parameters\"\n",
        "print pcd.theta\n",
        "\n",
        "samples = []\n",
        "sampler = pcd.get_new_sampler()\n",
        "    \n",
        "sampler.proposal_dist = proposal_distro\n",
        "for i in range(10000):\n",
        "  sampler.sample()\n",
        "  samples.append(sampler.last_sample)\n",
        "\n",
        "# Look at empirical distribution of samples\n",
        "cnt = collections.Counter(samples)\n",
        "\n",
        "# Concatenate the results\n",
        "distros_ = np.array([[y, cnt[x]] for x, y in ps.iteritems()])\n",
        "\n",
        "# Normalize\n",
        "distros_ = distros_ / distros_.sum(0).reshape((1,-1))\n",
        "\n",
        "print 'String | Exact p(x)*1000 | Sample p(x)*1000'\n",
        "print '\\n'.join(['%s        % 8d         % 8d' % (x, y[0], y[1]) for x, y in \n",
        "                 zip(ps.keys(), np.round(distros_*1000))])\n",
        "\n",
        "max_rel_disc = max(abs(.5 * (distros_[:,0] - distros_[:,1]) / \n",
        "                       (distros_[:,0] + distros_[:,1])))\n",
        "\n",
        "print 'Max relative discrepancy: %.3f' % max_rel_disc"
      ],
      "outputs": [],
      "execution_count": 0
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q0J83HaPnhkc",
        "colab_type": "text"
      },
      "source": [
        "# Run on real data\n",
        "\n",
        "Let's fit a toy spelling model with features like the number of each letter, whether a vowel is followed by a consonant, etc. for all the 7-letter words in the Wikipedia article on machine learning. We'll add a global feature that determines whether the word is \"machine\"."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IhaHvfDgngvr",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": []
        },
        "cellView": "both",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1457682301818,
          "user_tz": 480,
          "elapsed": 613,
          "user": {
            "sessionId": "ab1f0339c96f28e",
            "userId": "103444865736961301998",
            "permissionId": "00759584880725894106",
            "displayName": "Patrick Mineault",
            "color": "#1FA15D",
            "isMe": true,
            "isAnonymous": false,
            "photoUrl": "//lh3.googleusercontent.com/-CdPTZM1TPbU/AAAAAAAAAAI/AAAAAAAAABQ/MOmbIGKtlQA/s50-c-k-no/photo.jpg"
          }
        },
        "outputId": "6e382b17-e015-4429-a074-36e9c2670a1f"
      },
      "source": [
        "words = ['Machine', 'evolved', 'pattern', 'defined', 'machine', 'ability', 'without', 'Machine', 'operate', 'example', 'program', 'Machine', 'closely', 'related', 'focuses', 'through', 'domains', 'Machine', 'Example', 'include', 'optical', 'engines', 'vision.', 'Machine', 'focuses', 'quoted,', 'program', 'respect', 'measure', 'notable', 'machine', 'think?\"', 'do?\"[9]', 'Machine', 'system.', 'are[10]', 'example', 'desired', 'general', 'leaving', 'towards', 'program', 'dynamic', 'perform', 'certain', 'driving', 'without', 'teacher', 'telling', 'whether', 'Another', 'example', 'playing', 'against', 'Between', 'teacher', 'signal:', 'outputs', 'special', 'problem', 'targets', 'support', 'machine', 'divides', 'learned', 'machine', 'acquire', 'through', 'Another', 'machine', 'desired', 'divided', 'learner', 'produce', 'assigns', 'tackled', 'example', 'classes', 'outputs', 'divided', 'groups.', 'Density', 'mapping', 'related', 'program', 'similar', 'topics.', 'History', 'machine', 'Already', 'problem', 'various', '\"neural', 'medical', 'between', 'machine', 'systems', 'plagued', 'systems', 'leading', 'outside', 'proper,', 'pattern', 'science', 'outside', 'Hinton.', 'success', 'Machine', 'started', 'changed', 'nature.', 'shifted', 'methods', 'Machine', 'methods', 'overlap', 'roughly', 'Machine', 'focuses', 'learned', 'focuses', 'unknown', 'overlap', 'machine', 'machine', 'employs', 'methods', 'improve', 'learner', 'between', 'machine', 'usually', 'respect', 'ability', 'unknown', 'respect', 'typical', 'methods', 'Machine', 'express', 'between', 'trained', 'problem', 'trained', 'predict', 'between', 'machine', 'Machine', 'closely', 'related', 'fields.', 'Michael', 'Jordan,', 'machine', 'science', 'overall', 'Breiman', 'wherein', 'machine', 'forest.', 'adopted', 'methods', 'machine', 'leading', 'learner', 'context', 'ability', 'machine', 'perform', 'unknown', 'learner', 'general', 'enables', 'produce', 'machine', 'science', 'theory.', 'Because', 'usually', 'common.', 'trained', 'unknown', 'complex', 'complex', 'minimum', 'bounds,', 'theory,', 'results', 'certain', 'learned', 'results', 'certain', 'classes', 'learned', 'between', 'machine', 'machine', 'between', 'network', 'network', 'usually', '\"neural', 'aspects', 'usually', 'complex', 'between', 'capture', 'unknown', 'between', 'uniform', 'logical', 'program', 'entails', 'related', 'Support', 'Support', 'Support', 'related', 'methods', 'whether', 'example', 'Cluster', 'Cluster', 'subsets', '(called', 'cluster', 'similar', 'defined', 'example', 'between', 'members', 'between', 'methods', 'density', 'network', 'network', 'acyclic', 'acyclic', 'network', 'between', 'network', 'compute', 'various', 'perform', 'actions', 'reward.', 'attempt', 'actions', 'states.', 'differs', 'problem', 'correct', 'actions', 'Several', 'include', 'cluster', 'attempt', 'useful,', 'unknown', 'attempt', 'learned', 'attempt', 'learned', 'zeros).', 'without', 'defined', 'machine', 'factors', 'explain', 'machine', 'similar', 'similar', 'predict', 'objects', 'method,', 'assumed', 'sparse.', 'matrix,', 'solving', 'sparse.', 'assumed', 'freedom', 'NP-hard', 'popular', 'applied', 'several', 'problem', 'classes', 'belongs', 'Suppose', 'already', 'applied', 'Genetic', 'Genetic', 'genetic', 'process', 'natural', 'methods', 'finding', 'machine', 'genetic', 'machine', 'improve', 'genetic', 'machine', 'finance', 'vision,', 'Machine', 'Medical', 'Natural', 'systems', 'engines', 'opinion', 'mining)', 'pattern', 'Finance', 'company', 'Netflix', 'program', 'predict', 'improve', 'Shortly', 'Netflix', 'ratings', 'viewing', 'changed', 'Journal', 'machine', 'predict', 'article', 'machine', 'applied', 'History', 'between']"
      ],
      "outputs": [],
      "execution_count": 0
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MhZd2CNuoM2Y",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 11
            },
            {
              "item_id": 12
            }
          ]
        },
        "cellView": "both",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1457683062754,
          "user_tz": 480,
          "elapsed": 65860,
          "user": {
            "sessionId": "ab1f0339c96f28e",
            "userId": "103444865736961301998",
            "permissionId": "00759584880725894106",
            "displayName": "Patrick Mineault",
            "color": "#1FA15D",
            "isMe": true,
            "isAnonymous": false,
            "photoUrl": "//lh3.googleusercontent.com/-CdPTZM1TPbU/AAAAAAAAAAI/AAAAAAAAABQ/MOmbIGKtlQA/s50-c-k-no/photo.jpg"
          }
        },
        "outputId": "b07e7071-4f46-4f99-ee06-243c833b911e"
      },
      "source": [
        "def compute_features(word):\n",
        "  \n",
        "  phis = []\n",
        "  # Check whether the first letter is uppercase\n",
        "  phis.append(1*(word[0] >= 'A' and word[0] <= 'Z'))\n",
        "  \n",
        "  # Check whether the other letters are uppercase\n",
        "  phis.append(sum(l >= ord('A') and l <= ord('Z') for l in word[1:]))\n",
        "  \n",
        "  # Check whether vowels are followed by consonants\n",
        "  vowels = 'aeiou'\n",
        "  \n",
        "  letters = ('abcdefghijklmnopqrstuvwxyz'\n",
        "             'ABCDEFGHIJKLMNOPQRSTUVWXYZ')\n",
        "    \n",
        "  phis.append(sum(x in letters and x not in vowels and y in vowels for x,y in \n",
        "                  zip(word[:-1], word[1:])))\n",
        "\n",
        "  # Check the number of symbols\n",
        "  phis.append(sum(x not in letters for x in word))\n",
        "  \n",
        "  # The number of each letter \n",
        "  phis += [sum(x == y for x in word) for y in letters]\n",
        "  \n",
        "  # And whether the word is machine (to add a global feature)\n",
        "  phis.append(word == 'machine' or word == 'Machine')\n",
        "  \n",
        "  return np.array(phis)\n",
        "\n",
        "# Draw some words\n",
        "pcd = FixedLengthPCD(words, \n",
        "                     compute_features,\n",
        "                     S_chains = 100)\n",
        "\n",
        "convergence = pcd.run(T = 1000, \n",
        "                      etas = [.1, 0.03, .01],\n",
        "                      k_steps_per = 1,\n",
        "                      rms_prop_alpha = 0.9,\n",
        "                      display_every = 100)\n",
        "\n",
        "# Plot some indices of convergence\n",
        "plt.subplot(411)\n",
        "plt.plot(convergence[0])\n",
        "plt.title('theta')\n",
        "plt.subplot(412)\n",
        "plt.plot(convergence[1])\n",
        "plt.xlabel('Iteration')\n",
        "plt.title('||g||')\n",
        "\n",
        "plt.subplot(413)\n",
        "plt.plot(convergence[2])\n",
        "plt.xlabel('Iteration')\n",
        "plt.title('||g_a||')\n",
        "\n",
        "plt.subplot(414)\n",
        "plt.plot(convergence[3])\n",
        "plt.title('likelihood')\n",
        "\n",
        "print \"Estimated parameters\"\n",
        "print pcd.theta\n"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Iteration 0000, log likelihood = 1.20084818\n",
            "Iteration 0100, log likelihood = 12494850.38926714\n",
            "Iteration 0200, log likelihood = 2762440748.43619156\n",
            "Iteration 0300, log likelihood = 50845303416.27217102\n",
            "Iteration 0400, log likelihood = 2190501833230.38696289\n",
            "Iteration 0500, log likelihood = 4276988176134.80468750\n",
            "Iteration 0600, log likelihood = 11247602689282.73437500\n",
            "Iteration 0700, log likelihood = 25981700574598.57031250\n",
            "Iteration 0800, log likelihood = 24467063671332.46093750\n",
            "Iteration 0900, log likelihood = 32817339298285.48437500\n",
            "Estimated parameters\n",
            "[  3.35791760e+00   1.83633295e-02   1.95713117e+00  -1.02220839e+00\n",
            "   1.99813306e+00   6.06049274e-01   2.19943471e+00   2.00876385e+00\n",
            "   2.60486341e+00   4.57186511e-01   1.10098610e+00   1.58116821e+00\n",
            "   1.86626637e+00  -2.49000326e+00   1.21867005e-01   2.11907471e+00\n",
            "   1.95922081e+00   2.51286906e+00   1.52986826e+00   1.74849790e+00\n",
            "  -1.62980114e+00   2.44229185e+00   2.21451871e+00   2.44444045e+00\n",
            "   1.04837968e+00   2.80821176e-01   8.65999568e-01   1.55036043e-01\n",
            "   9.38534493e-01  -2.08448326e+00  -3.64286502e+00  -3.31481477e+00\n",
            "  -3.26019284e+00  -4.66082320e+00  -3.71480812e+00  -3.18516343e+00\n",
            "  -4.16325253e+00  -3.86595672e+00  -1.40366120e-03  -3.42084374e+00\n",
            "  -2.48984540e-03   1.15546348e-02  -2.01891457e+00  -3.06088033e+00\n",
            "  -2.34202229e-03  -4.01397783e+00   1.22163070e-02  -1.67549149e-02\n",
            "  -2.01315696e+00  -7.38599497e-03   1.47584718e-02   1.64015462e-02\n",
            "   1.04174271e-02  -3.67685204e-03  -2.68521826e-03   6.47515500e-04\n",
            "   1.41110097e+01]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "metadata": {},
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAENCAYAAAAPAhLDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXmcHkWd+P+u6uM55p5JMkmYBDCIqFkMynIoggSCYhS+\n/ggCP2QXVC4FicEDXVEUFV0VFfGnsLjsfkXXA8EFgqwCa1RuDwi3JJBMJpO5z+eZ53m6u6p+f3Q/\n12QmmUxmSCD9nldPV1dXV1XX0/351PGpamGMMcTExMTE7PPIPZ2BmJiYmJi9g1ghxMTExMQAsUKI\niYmJiYmIFUJMTExMDBArhJiYmJiYiFghxMTExMQAsUKI2Yfp6OjgkEMOQWu9p7MSE7NXECuEmH2K\n5cuX89BDD+12PLEyiXk1EiuEmJjdIJ7XGfNqIlYIMfsMn/zkJ9m2bRsXXXQRhx12GPfccw8Ad9xx\nB8cffzxHHXUUP/zhD0vhjTHceOONrFixgiOPPJLVq1czPDwMwAc+8AEADj/8cA477DCeeOIJ2tvb\n+ad/+ieOPPJIjjrqKD7xiU8wOjr68t9oTMx0MTEx+xDHH3+8efDBB40xxnR0dJjXve515sorrzSF\nQsE8++yzZunSpWbjxo3GGGNuvvlmc8YZZ5iuri7jeZ658sorzZo1a6quVUqV4t68ebN58MEHjed5\npr+/35x99tnmK1/5yst/kzEx0yRuIcTss5iou+ejH/0orutyyCGHcMghh/Dcc88B8POf/5zLLruM\n1tZWHMfhkksu4X/+53/QWk/YVbR48WKOPvpoHMehubmZc889l8cee+xlvaeYmN3B3tMZiInZ08yd\nO7fkTiaTjI2NAdDZ2ckll1yClOV6k2VZ9PX1TRhPf38/X/7yl/nLX/5CNptFa01jY+PsZj4mZgaJ\nFULMPoUQYsphFyxYwDXXXMNhhx223bmtW7du5/etb30LKSV33nknDQ0N3HvvvVx99dW7ld+YmJeT\nuMsoZp9izpw5bNmyZUphzzzzTK699lo6OzsBGBgY4L777gOgubkZKSXt7e2l8GNjY6TTaerq6uju\n7uamm26a+RuIiZlFYoUQs09xwQUX8IMf/IAjjjiC3/72tztsMfzzP/8zJ5xwAh/84Ad585vfzBln\nnMH69esBSKVSXHTRRZx11lkcccQRrF+/nksuuYSnn36at7zlLVx00UW8853v3KUWSUzMnkaYiUbH\ndpHPfOYzrFu3jpaWFu68804Avve97/HLX/6S5uZmANasWcOxxx67u0nFxMTExMwSM6IQ/vznP5NO\np/n0pz9dUgjXX389NTU1nHfeebudyZiYmJiY2WdGuowOP/xw6uvrt/OfAV0TExMTE/MyMatjCLfc\ncgunnHIKn/3sZxkZGZnNpGJiYmJidpNZUwhnnXUW9913H//93//N3Llz+drXvjZbScXExMTEzACz\nNg+hpaWl5D799NO5+OKLd3qNMSa2yoiJiXlFYrTGKBVugcLoaK8CjFLoQJXPK4UJgoowFdepAKM0\nRgXjron8g6AqHmHb7H/2WTNyD7OmEHp6epg3bx4A9957LwcffPBOrxFC0NsbLwYGMHduXVwWEXFZ\nlHmll4UxBiqEGVqXBCZKV/hXC8/SNbrsrqtxGRnMhtdHwpiiIC5eEwnc0K1LgnWytFAV+akQ8JV5\nrsxD5Tn24JjpXqUQ1qxZw6OPPsrQ0BDHHXccl156KY8++ijPPvssQgja2tr40pe+NBNJxcTETAOj\nNTqfR+dz6Fy0z+fRuVzklyuf97xIyOlIeFYI02CcAK2o4VIpcLWu8K9WADPFthmLaQKkRFgWSAth\nhRtW6CfdBEJKsCrPRW4ZuW0LUYwj8hd2hbt0jURYdpSW3D4+a6JrKs5JiUylZ+y2Z8TsdCZ5Jdd+\nZpJXek1wJtlXy8IYg/G8KkFel4DBroEKv1zVeTWBn87nMIXCzGdQiImFYZUA3V7YTX5NUcja1cJ0\nIsFoW9TVp8nmfIRll4T19kK8WngW81Mp4IVlVwtjKUOB/wpi7ty6GYknXssoJmaGMUEQCuRCVAPP\nbS+81QRCeyJBPt0atXAcZDKFTKWwGxqQySQylSr5yWRyEr8UMuFOWjuuEp57WGjuqxWF2WRGFMJE\nM5WHhob4+Mc/TmdnJ/vttx/f+c53JpyrEBOzN2CMwRQK4wRzPhTcuVyFcK/oahnvF3XDGM+bXiak\nLAllu6kJmVywndCua2kgpwQylcIaL8hT0T6ZRNhxXS9m15m1mcr/+q//SlNTE+effz433ngjIyMj\nfOITn9hpXLHGD4lrP2V2VBYmCNC53KRdJRP6VfWdl/2mOygoEokKoZyqEOKhnzVpjbwYNgwnXHen\nVnbxc1EmLosye1WX0eGHH05HR0eV3/33388tt9wCwPve9z7OOeecKSmEmFc/Rmt0oTDxgOYEgnzA\nBOSGRifuGw+C6WXCskq1bLtlTlloV9W4pyDIE8mwSyUm5lXArLUr+/v7mTNnDhB+gGRgYGC2korZ\ngxit8Xt7KGzZgt/bM6HQVrkcJl8c8MxjCvlpp1cUzlZdHc7cudt1lZQFd+RORPtxglzYTjznJSZm\nHHtdR+NMNX1eDextZRFksmQ3b2Zs0yaymzaTfWkzY+3t6J1YsAjHwU6ncFJprOYmrFQKK53CSqWx\n0ymsdDr0i/ztquN0KayVTOzxgcy9gb3tudiTxGUxs8zqTOXe3l7mzp1LT09PaRnsnRH3CYbsyf5R\nozV+TzeFji0UtmwJ9x1bCPr7qwNaFomFC3HbFpFoW4TbOh8ZCfPKLpddHeDU0eZHx3NborLITrN7\n6FVE3G9eJi6LMnvVGMJELF++nNtvv50LLriAX//615x44omzlVTMbqCyWQpbOyhsaS8pAK9z63aW\nMlZDA+k3LiURCf/EokW48xfE1iwxMa8iZmWm8sc+9jEuuOACVq9eza9+9SsWLlzId7/73ZlIKmaa\nGK3xu7sodFQI/44tBOPGdoRt4y5YWBb6kQKwY5PhmJhXPfFM5b2U3WkOq0wmqvWXu3u8rR0Y368K\nZzU0kmhrI7FocWnvts7f62r9cddAmbgsysRlUWav7zKKmX2MUnjd3XgdZcFf2LKFYHCCWv/C/UKh\n37Y4qvm3YdfFtf6YmJgys64Qli9fTk1NDZZlYds2t95662wn+apEZTJloV/Z1z++1t/YSHrpP5S6\nfBJti3FbW/e6Wn9MTMzex8siJX784x/T2Nj4ciT1iies9XfR+2wfvc/8vVT7DwYHq8KVa/1FwR9u\nVl1shhcTEzM9XhaFsJcNU+w1qEymyrqn0BHV+sfNvrWbmkgvPbQs+BeFJp57coasMYZMzmdgpIA2\nBikEtiVIOBauY0V7GU/+iol5BTHrCkEIwYc+9CEAzjzzTN7//vfPdpJ7HSYI8LqLdv3toaVPRztq\naKgqnLBt3P3aSLQtouX1B+E3ziOxX9seq/UHStM/kqd3KEfvYI7eodDdM5SjdyhH3lM7vN62BHVp\nl9qUQ23KIZ20qUnapJNO9T5hk6rY0gk7ViYxMXuAWbcyKk5OGxgY4LzzzuPKK6/k8MMPn80k9yj+\nyAjZl8KZvMUZvWPtW7ar9bstLdQcsD81Bx5Aev9wn1q44GWv9WdzPtv6snQNZNnWl6V7YCw6HqNv\ncAw9wdORcC0WtNTQ2pxmXnMaSwq0MfiBpuApCp4i5wWMZj1Gsh4j2QK5wo6Vx3gsKUiXlIZDOmVT\nk3RIJWySCZuka4Vu1yaVsCK/0F0Mk3KjfcLCtmIFExOzM15Ws9Prr7+edDrNBz/4wUnDvFLMyMJa\nf1dFjT/s9lHD42r9jhPV+tuwF7bhN89HzF+AXVuHEIKcFzCWD1g4p4balFO6bmcmdblCwHDWo+Ap\n6mtcGmpd5E4EntaGFzqGeL59iPaeDO3do/QNT7yuUGOty9zGFPMaU8wtbk3hvj696+sABUozlg/I\n5v1oHzCW9xkrhO5cobyNldyKXCEg7wW7rFDGY8mwOyvhRt1ZtsRxJK5t4dgyPLYljh12daVcO1Is\nVuS2IoVjk3KtklKyrdlfSsMYQ6AMeS8gXZtkW9cIeU9R8BWB0ihtwr0yVceBMji2pL4mbKUlXYtk\ndP9J1ybhSqwZWgpEG4PWBqUMSptSN6IlBZYlkFLs9PncVWKz0zKvCLPTXC6HUora2lrGxsb405/+\nxCWXXDKbSc4KwehIaNVT2de/rXP7vv7mZlJLD2WscR6D6Ra2OY1sU2n6sx79IwVG/uoB/dFWjQAO\nWFDHGw9s4bVtDbwl5ZbO5QoBL24bob17lM6+LFu6M2zpyVCpyROuRducGl7b1sjS1zRz4IJ6Ugkb\nP9D8fcsQf3m+h7/+vZeRsbJVUl3a4Y0HNLGgpaZK4M9tSOI6M9tSsa1QMNXXuDsPPI65c+vo7h4h\n5wUUPFUShvmoNZL3g1LLJF/pH4UreAF5v+yXzfsMBhrf1+jdrA+5tiSZsEk4EtsKBaxtCWwr3FtW\nqGySbqhckm6oYCwpMITCXmvDSNZnKFMg5wXki3mvcKuJmmozgGPLkpJIRL95sUx0mEGMAUOYz0BX\nC/5w01NaOVwIsKTEsgS2LCoLGe63c4swbOS2I7cQYTe0EJBKunieXzqWQiCEQFaEqT43wTEVx3Lc\ntWwfF9FvFpZJ+N7KiuuknDh+Ij8o7wOtS+VYVOBKR/vouKjgVcV5baLnJtq7tuSzHzxqRp6HWVUI\nfX19JQWglOK9730vxxxzzGwmuVuYIMDr2lY1oavQsQU1PFwVTrgu9sI29LwF5BpbGa5podtt5O99\nPhu3DqO6im9HDshhW4LmuiQLFzfSVJfAsiQ6qkWlXJuEa/Fi5wgbtw7z0rZijecJGmtdapIO2/qy\nVH43y7Ykr1vcyLymFI5tMZwpsG1gjE1do2zsHOGeR9uxkhappiTKGKy8wsr6CFvyjmULOfSgORww\nv46GSDgbpTAmj9aKQNvkMh5ZFaaYTDskU2GLwBhDTmkyvsKVgnrXRhTXPdq6FeN7CNcNvznruuGq\npDW1WLW1O1zr3xiDrw2e1vjaYAlBwpK4UpSuEQLSCUlN0pkwjkpUFFdBaTxtEBAKICFwopdXG3BM\nuAy3l83j5fJ4uRxBwceXNp6wyQmHnLHI+5qcF7VWCmF3WNhyCf1yFcJbKVP1ogujSakCDgGWBGlL\ngqSLshw8LQmUwCiDVoZi/1xxcD7p2jTWJUg6VlS7t2moT4LWJN1wnMWxJFZR+chQEZUEqCXwA83o\nmM/omFehIBX5QkCQG8PkcgSFHF5WEQQKCQgMQhiMtEACCBChQEZKpCWQjkRYEmlZCGljWSL67KVE\nWuEnKJUBFQktpQ2BpqRUjDZopdEqQGsNniZQmoLWBBqCKLwJc1NWOEUpLARCRg+GKArdiudLhPeB\niA62f+qqduO9i3GUE5wgiCkHE8Yg0EhjkBiE1ggM0mhk8e2tijtSLkKiRVg5CG/IhHEZg9QKYUAY\nXfILb8lE6YbHWlrAzCiEfWKmslGKYGQENTRIMDRIMDSE19WF19uDHh0lyGTQ2Sw6N7bdR1J0Oo2p\nTaASFoXAIp81PJNoY0vdfqTTYKctAi0YzcBgweHgBsHS5BgNwSB+TZJBO81wIkHeSWKUhT2kYEjh\nBxbCsbBqLIK6GvLNCTJpgac1SgqUlAijSWgPIwQFO0lNYYyFw2M0UofWFl7BZ1gXGDGKAAuVtqHG\nwjiCpOfRlBmiOT9EQgcUPIdcwWWkpY5Mc4oxN4FtAprkMAe4XSyw+6ljDGkUw8N19PQ20zfYyJCq\nI5dI4tc4qIRFkLLxUxbatXDzPqmRUSzb4Ne4aDusYRpE+EAbjYweXiMExojo7QFk+DIoKdFChueB\n4uNuobEJcIWPQ0BC+Dj42CbAMgpLawJthRs2OZLkZBJf2ESvZinOcB8KjvC4opvEGCQay2hsHaWF\nj4uHNNFLrSOxFL2UQpfdqCh+KRCWwTYKrQRKWWgl0Tq8b4MAA1IYNFAQLp7l4FsOgbTQQoayTYbC\nGAlIEcYtRCQ0otIUII3GRmGbAEdEZWI0UmssHZaP0aCwUEaiTFgeFPMqDBYaKTSW0EihwnQJ0yv+\njpW/SdW+9HuVhXblNSb6oQ0CIyoE+zg/Kvz0dm5ZchcFqIieKUEkJCtjMOUUzGT3IURVHsfnnUkq\nLXsFpnyfEN23AUsrrj/16BlJYtYVwh/+8Ae++tWvorVm1apVXHDBBTsMv6sKwWiN39+H391FMBAK\nfK+nm2BwEH90lE6RZkSmEAYS3hg1hRFqxoZx/eolmwPLZqhpDiONzWSbGhBzkojWBMPJZoZMHUPU\n4+Fi49MUjJDOjJHIeKTIUV+TgaRgSNYxLOoZkI1krJpJHy5b+zQwSos1jEQxSi2del61oIpw8XDx\nSeAxSD2aXevKEWhcfGyCKKYd17Ad4+MKH4XEw0WzfZ6kUdgoPFHd/WPjlxSAQqKwGF+7qnzti0Lf\nRmGhofRqgjISH5tQRNull3tHSBQJPCRFoWEmcGscESDRKCwCEyqUAAsPlwLOLpfxzGOwUciofMJy\nUuWaJqCReDh4ODv9TfccpkKsmwkFd+UmK/ZS6NJvJ9Fhi8WET48u/ZLVMVQeF8X8ROmUz+moblKp\nHkCIalU4Xh2Wn9JKyn4TCdTKa8anX5mvcqmV79Ug0KZcIai+Z4mF4qvvXrF7P1XErHYZKaW4+uqr\nufnmm2ltbWXVqlWccMIJLFmyZJfj0p6H392Nt62TfPtmCu2byXdtIzeWZyxdQzZdSy5Rg+8kSYiA\ndN6jLptlzkgn88bpvNG6BroWLmagpZVMSxNecx35+hpGZT05ktUJR022RFCg2QygbJteuxkaW2CS\nuXYSzXz6mCMGqNNZ0jKPL2xGTC1D1DEk6xmgkX5TXhK8kUEO1ptoMcMkggKWUqAVfZkk3Z5EpDMs\ncCzcmlaEkySPyxhpbBQuHmmyWGoY/FEC5TNg1TPMHEQ2RSbdQCFRg02WxmCI5sIwc9QQlvZpz81l\nGw3k6iFwLQq5JGpEoDIBzmiWhPHRCYu6REB9ncZtSjKWbiAQDgvoplYN4ah+CPrJaQ+DAakxwhAI\njYdBI0ovmTHgYfBNsVtA4ikLHdiowCIwAowsFb5lJEJqhAWWZSGlixQuSAspfbRRCBQWeSR5bAyO\nBBeBjQRduVmgJUJLMBaWFiS0hQ0IWzFmNFmjGBOQtwTGkiDLXRMi6h9GCqSwEEIikYDG1x4BBoNF\nQtq4SFwhsYSFQCKNBVgYbSGRuKZAQhdwfIUIFBZhV5OnDQqFxqC0QAU2RkvCHjwBRqAjwWFpiavA\nMgIcwDUYS2CkjcFGIHAsH9fycR0PIT209FEmoGAMBWUTBC6B7xIELtq3IRovCEvfRO5wM8IA4W+L\nUBgZ7dEgNUJojFAgNEYqhNBEfR7FFylqIYb+QoQN8qlWysW4rdhjVHpVx0liA2gTCdNow0gw0UWl\nfIFFmB9ZrH0LE44LALL4/AoRph02MyO3oPhXzGPxOkw0nhD9bjJqJVK8InIrI9BaoAFlwq5ZXSx1\nsf19F+MsKhYjA+AVoBDWr1/P4sWLaWtrA2DlypXcd999O1UIKpcjv+klxp55mvzGF8h09zDoJOlr\naqWzpQ3PrSXZsIR6p4XG0X4aR/po7dmKM27JZt926Ju3kMHmeQy0zGOwuRW/Ockc0UcyN0KQzTLs\nd6JyOZpRHCgktbZFyhEgDP2+xZaMzXMDSUYyEisQeCLsGqlrFNTUWdjpBAW3Hs8boznfj5v38HKa\nmsIYUipywibvGFLJQWqcMXTaQeXrsYabcVU9JnAYU4qcKPB8wiPjJ8nkmhhT0FCbobUuy/51gxxU\n79GcEEC5BTWYc9kyVM9LA7VsDAyW8Fiox1gkshxIL/XJF7GaXUTSwSgY7jBs2pBiS6GJF9NzGHbq\nsMdypAr9NOox0kGOGuXR5I9y4FgnNSq0QNIIRu003XXN5Bc1YbUkEDUS6Qoc28O2fGwhUV4CrQi7\nSTSYAHQA2hfgG2RgsCwQCYFMCKyEwHIIha0FwglfPBuDJTQJS5NwVKhE8gLlK4w3hgmyiECHXVJG\nYwmDtEE6YR89ApQW+FpQ8ARjBYsxz0EZQSQmEMUWSdTPbxmDY4eb7RhsB6Q04YtnDMYHfIX0AqSv\nEEpjAgNaYxQYBZ628H2B8gVKEXZnKYmnbAIddftICbZEOALH1iSED4HBVxZBQHitDl912ygcrcJu\nLCvAtRWurXEdjePoMJ8u4Eq0G3Y7aSNRUe1SIfEDCAKBp8APBAUt8AOQgcEVeVw0lgTl2CjLRkTd\nd5WKUMpIWGIiQWmiAVNTFpzCIGVUuxZh15iQ5cFViuNBUVcYAoQUJakuZHE4IBTQoth7U9xHA9wl\n/YSp6sOv0gZmXB3emEiIm/CHqoindJmuiLZ4rhhtNIiMMRV+UTTFCETYDVZZbtGATHTPpjzOIYvl\nUd4LO1JEsnzvIrq+6FcqC0mpLIUIb2mmmFWF0N3dzYIFC0rHra2trF+/ftLwV/zLN0gEClf5uFqR\n1BpXS9LpRtKFHPttfZHXbngSy+iq67QQjNY3MbKgmVxTLZnaBMO1isGswBoUJLyARG4bc4a2kBgR\nJB1BOmmoSWnePEdRkwqQE9RS2lLwpnp4z0IwmQCTV5i8xuQVqgBWv0KMWag6F1nvYDdGfZHaYAY8\nVEcOPyPx0jXgJnDSCdKNPkIUYME2YNuUy9L4Br01j8lrtK8RaYvGVkXTAo9DF/SNC52KNtCBoTCs\ncFxoWmLTtMRjWbANk+2AbIDxDMIVYIVPmw4gn3UZLhxM1pYk0h6pRIGmtGZeDQiZATJTzvd0MBow\nBu2D8QABqXqQ9vgfSUZb5bUGNIiqsEG07Spi3N5m91+Z4ud/iozvkjOhFUukTKUtkU4CSOxmulOl\nJHGnQLHOOoOpVwpqI6pr/UWFUXRXZqPo3IuHACbDaKLWCxgtxpVBuNdGhONV4/zRoGbwu1Ev+4pn\nO7Jff+9TD096zgDZpEVPk81wjc1AncNAvcNAncNQnYOWgvAbWxVr/tQCrTvIjCascE9p2EIAdtg0\nd4DxZr8jGkYqPWxYUBmoAB7QM5W0JsF2w3sqMgQwleqBCOXhWGVYCZZb1Bsh4Wgu1Gug2rKKsWjb\nU+zJtGNi9mYEvGOGoppVhTB//ny2bSvXgru6upg3b96k4R9fUkfBkeRdScGVJfdo2iKTsiOhHxMT\nExMzG8yqQli6dCmbN2+mo6ODefPmcffdd3PttddOGv6j1/5HlZWR7xUYHuine3MXm595nt7RAXoc\nGLBcRk2CgrZBOxC46EIKo7e/HZuwse1GewdQGAIEHmGbIs/U6tlFXMIGgiM00oKEFtRoQcrWOCoD\nmX7qxnqRJmDYrqHPbSRrpRizkgw5tRNaEzkGaqOxQQMkUw5z6pM0CkGuP4vyd80YrDhgNxEpF5qS\nATUmxxw3R01CUnBqUDU1WKkagm1bGO3somvEMOQ2o6wUykohhF3VwlPGoDEEGCQBtggHR4UxGK3D\nQVERDqaGfaAyHIwF5ARlUERKge1IbMfCjiZ++fk8ppBH5QuYQh5LB1jaDweqRdR/jozsui0EoRmp\npX1s7WFrf7tjS3sIrVHSRgs7vE6YshWHiGxchIVvJ/BlEt9yCaSDkg5KhM+bNCo0UTUaaRSWCbB0\nMC69ik15uCqPG+RwjDflThcDBMLCFzaBtFDCIhASJSyUkAQi9POFTcFKUrCSgMQtWrBE94MQ6KjM\nlLAwiOgeVMnWS4uwm8KIyHon6seXUV+FjAZjRZQxAaV/ZXPQ4lbs0y8OxBbHIMIBWiki80lCs1ch\nKkxSS6ajMhoLLtvolEulbAFU9JPRCyCqBgpMhV/xmupjUQpD1bnwuSqaMofPRGgOXJy5UZ0/I8r5\nr7R3osLsNfw9KIetvG4Ct6l4xsNnU5T2nDPFh2gnzLrZ6bp166rMTi+88MIdht+Z2anWhsG+LD1b\nhxjY0M7gpg5Gh4cZVQU6G1z60ykKMhEWnLYgcDB+ssJyZSqYyMRP4MrQPtwWAmGHk31MTiG0IRR1\nYCGwgDyGASZWLuEEJY+GIMv+uW00e2GXTG+iif7aefQ4jWTMxPpZRukIiHIFjUKwX9Kh0ZZkM96k\n3b6WLUgkbIQlyXsKrxCEcwQqxJDB4GMoCIMxknSUpo7uRUX3liXsedKUx8sEkCZUkpReo8qSLG/F\n+AIMinAelsRglSb1REOhWiGMiiw+dLSvtCEv2uWHex2ds43BBpzIxsYxGtdoHMLBWQuBjoR5UahP\npJxnChPa35TKUEVlUCxDRVmpFrcwnEGZKIwoDhCH97ovUJxTYUWKtvh8WKY48UuX5rjI8X6mwoS1\n0ry15KYUZjsTWFOa6VHyR4xXQRVuEVklRT9L6drSvjzmURqNqhr7MCVVWRLsUeDSHBojwvmKIrSW\nwhRHoIo1PkNSB1z2oy/OSNm/aiam5XM+w4M5hgdzZEcLZEdy5AZHGB0eoV1tZdAdRtmSgk7gm0hZ\nKBsRJLC8BNJPILSNJuyunmiFHwElJVBpQOACNQgcwodCuwFIRa1foKEwSsPYAOQyZC2XvJUiaRQJ\nwBYW2Cnydh1jTh0ZO1k13JgBMsJQkKBdHxJ57HQB3x7Ft0cRiRxkGqlXbTSKBhoci7QxBBlDfsRH\n+qJK8EMobDIYhqJ7NJQFVXFTTH1Y8ZVG6TcU4EiBY4WzfRPRekbFJRwcx8KxBbZtYVnR7FwrfEl1\nEKCDgMAP8AJNwVfkfM2Yp8j7hnyg8RR4gZl2ORafsYk2MYG7uAeQEmwLbFuCjIbTK8x2pCwvt1Cs\n3ovieRkuLyErlpAQlogmyYVK3AgTmkoaUEZjtEFFFjjhOkbR2kaRdY7WlNxGRyaV2qA1pWUgtDal\nAVOjixY+hqL9SJVFT4VFUEzInd86dUbiedUohJ1hjCE35jOW8fAKASMjY2zs7GDrYA/D+QE8xggs\nhUYitAX5NIV8LXlloQhthUN74YqJIkbusOUhgAZCZSEJ7e+L3VMBOqotCiwMNULRaI3S4G5BuoNo\nO08+GTDTSnL4AAAgAElEQVRYLxhLSZS149qh8R10tgGdbcDJNpDKNiD9sLMgQOPbHgVLoZQFQXL8\n1WAFICpsxxGgJUY5YMZP1DKknIC045O0AwqBzXA+ga+LomlKvwiOpbClwrYDhFRRza7YrSAjM0dA\nhBOVhDQIqQnt2KO5DgZ8A9pofGEIhAlt95ULyg7zX5yHUPV7zUZtu+JVKtrfCxPNPNah6aE0YIV7\nYZvQ3NYyCJvIDSCRRiJ1ZNbqA76NCSyMkqBEaI0SmfdGwULBOsWcCgxS6mgCWNQCMyK07KFo3VPs\nVKm0PXq1tVJM1S1NZLk03i3GBxxvjFbpjrraiklNli5iXKBiy6I0jyPyi94HEc2hMICQhtv/5f/d\nwT1OnVlTCN/73vf45S9/SXNzOPlqzZo1HHvssTu9bm9YvbCQDxXGtoE+uvq20TXQSW++l1GZDWtb\n2kYoB6ls0BLtJ/C8cGkKhSAQmkBqvLFalJ/aYVou5e4ECH/vlOWTSI/g1Azj2B6WdrB9B8tLoI2N\n7zsEXhKtHPz0KPn0CH5qFOMWEJYCy0fWDGP8BDrTiB5twnjJUECiEY6HsMNF7oxywE9g8mmMl2K7\nF94uYDX0I9KjyEQWJznG4kSBBQVJelTjjASkkganJTQCyAibjBGMaknWQM5AXmh8DEpEXSMinORk\nrHBC08uNMUTKwsb4DiYIx6HwE5jAwfhuWC7KwSgLtA3KwkQT2kLlUtlLPmlK0X62hKgBSyGkivZB\nqNilKp+vSj4SLtoK71k5mMAO7680iUyXhVDlxLIp+Veco0IZVkwAK+UBqoWgKOdXTORf2pfTrpr4\nJnXZDwNyB2Gr7nWWfpri7RQnrhWfl+gYI6IJfmHFxpTKbXr84owfzEiWZ00hXH/99dTU1HDeeeft\n0nV7g0KYClobegYG2Lz1Jbb1dJMdCiiMCoKsixpLgBH4VoHh2iFG3DxeYiwS2B6OV4OTTyFsH2n7\n4fLGuTrUaDO5fJrcONv06uGzqSFtD9nUhagZCQVcvhY92ojx0hNf4OSRySwilQ33xS2Rq35pdmVq\naSWmaGtdrq0bZSH9cJyHwC4NzBU3IYvCQyJEeI0odpaY4l6U/IROgIpaPzIA6YPlQ6QkhR3ukT5G\n+ICPKZkWTAMjwNgIYyG0XVIWQsvwXrWI1jLSURcIlGripeujGmaVu5hANBRr+WD7oRK3gpICmBGB\nZoByuywSVlUnESbK2vgHcHw+i34leR7Fp8uzdIvCsSggy5Xe8jHR+fDa4jXhby11WaCGfsUsRLOq\nq5RfWIcuFWnp2Qov0qIcxpSuN6WwlG+j+rYrlFe5CIrH1XkozYUs1uajki7mX2gZtoa1hdASGR1X\nnYveG6EFwljRcXR9tP/W53a8JNBUmVUro72sN2pGkVIwf04L8+e0bHdOBZrB/lF6Ozvp765joD/H\n8JAhm7HKwmAStDRkXZ9RI8kaO1z6IarZ2FLjADVAWoH0rLAbC9AYnGSBRKKALw2bR9PkehdDbzlu\nW/o0pnqpdTzSCUMqaUhYgsA21LYIGuYlqE8vpD49l7pEI42JeiwheWHoJdpHOnhppJ3+3ADKKKQI\nhYg2GmVUuMKniZYxmAhRrPlFwozovSk1oBykcBDCReCG+2ijdDzJ+ZLf+ElqCmNUaY8JMMW+FSFL\nbm0UaB9tChhdwJADCoR9NR6IoGozQkVLNCiM0Bh8jFVgMpW9s3bEhBgqlJ4sKwwjQdngOeC7mMDG\nBA7KdzDKLi3PIZWFNBaWEViyuIidCWd1C4NFOMPYMiLqngq7qSwdCieprVDgQlk4jxPkTOhfXsbh\nlYyhWHM3JWVTcld21xRr/5UtgapyIaqwvDKY1RbCbbfdRm1tLUuXLuWKK66gvr5+p9e9UloI00Ep\nzehwnpGhHIVcgXy2l2zOI5dX+JYmJwzZpE3e9slLQYYEY6TwmOQbAsaQHCrQ0J/FHfZhRJU6kQ2G\nUcAX4AhwdWh2u6OXVUpBy7xa5s6vpWVetM2twU3YFJSmv+DTn/foz/v0F3z68h6DBZ8Rv2xXFdaG\nPTAexvgY42GocBsPSseFivMV1+Dzyh4xrBz2lYSrKlnYxsLRkoSWJLUkFQhSgSTt2bjawlUCBwfL\nhItrBCY029XGwggLYyzAIfBDowijZLiiqhIoJUpus0sWddtTXMFo+66dsruyb7vYn11S+hV93FAx\nBhE1PkrfV6g8V5FucdMVte7QT0TXRnsTDnQbHbUWik0AXe6eKVb3hSm2YqraQqVfa2cbk7jNBBuT\nuCuf6PHnd7QvlkHlvrIbzhj4/775/zAT7JZCOO+88+jrG79sAqxevZply5aVxg++853v0Nvby1e/\n+tXp53QfRAUFcplt5L082QCyJkFW2WSUxbBn6Mt59IwV6M0WyPgKtMHN+DgZHycbYGd8ZKARGrQt\n0LbEOBLpSJQt8C2JiZZ4qBlTpEd99FAhXP6hMh8pi0KNg18bbXUOQcpCCEFzymVO2mVOKkFLyiXt\nWLiWxLVk+E2DCfauFYaxJ5loaIzBUz5jfo4xP0emkGWkMMpIIcOolyVTyJLxsmSj8zk/Tz7Ikw88\nfOVHFi661HpRWkVf8JLROjuh8JAiFAuOZZOwEri2gysdHMvGljaWtLClHV5nBFJbmECALxDKYk6y\nhVpRS1qmsU0CVHhOBaB9Q+BpPE/hewrPC/AKAV6h6A73u6v3LFviuhZuwg63orvCzyn52biJ8nnH\nsZAlYwUdKnNj0EqhtcZoXd4rjTYqtBjSCoxGq2hFTi3QKhTQoUVRtI+WWtDRUtxa6/A7INFmKtzV\nfjpKR1eFmzC8Cb8lURm3icyUKz9qU+xaE4JoGfBQKYSfeAjXWRr/IZyiu7gGkYiuCYVwmB+lyumq\nYr6ij9pgTPitC2OwpAw/wFOxVR9Hz6YlkSK0aCt9cKcynBWGlaUv0UkSSZuVqw7dvQcp4mWxMuro\n6ODiiy/mzjvv3GnYV3MLYVfY1c8D5gLFQCGsuQ/kfQYKPpYU1Dk2dY5V2jclHFKWRBnD5kyeLZk8\nW8fybM0WGPIC0AYnG4RKJeOTyPi4GR/hVXcFWbaksTlNU0u0zUlT35jCdiyc4qQyJ/x4y+5+y3h8\nWWhtUIHC9zWBr1CBJgg0SumyOwhfVNuWWLYk8DW+H34EJvAit6+ivS65fa+8D3xdOlbB7g9+O5EQ\nLu5t19rOzxnvN849r7We0Uw+8pPIGfoE5iuR+BOaZfb6T2j29PSUlqm49957Ofjgg2crqRggZVvs\nZ1vsVzPepHRibCFYUp9mSX15kDnjBwwWAgpaU+/Y1DsWiUigj2U9+nsy0ZZloDfL0MAY/T07X+jO\ndmQoAO1oZmb0tThTVdML/YthHdfCdqxSzdArBKECCFT4dbFZRFqilIdU2qHeTYaKbhoCvHidbe++\nYgSYM7e2NCgaEzPTzJpC+OY3v8mzzz6LEIK2tja+9KUvzVZSMTNErWNT60z8SKRrXNIHNrPowPI3\nHIwxjA7nGewfY6h/jNGRfFjbDipr3eE+CMK9EAJph5+yLDaFy9+iFQRBWDsv5AMyowWklLgJC8uW\nJFJOuKSFbY3bh62AcB8KX8sKm9VBEH7W0rbLrRbHsca1ZEJ3UYBb1r5b647Zt9lnJqa90oibw2Xi\nsigTl0WZuCzKzFSXUVwVitmn+dGPbuDf//3G7dxTYdWq99LV1TXpcUzMK43dVgi/+c1vWLlyJa9/\n/et5+umnq87dcMMNnHTSSbzrXe/iT3/60+4mFRMz4xStS4ruXb12R8cxMa80dlshHHzwwVx//fUc\nfvjhVf4bNmzg7rvvZu3atdx000188YtfROuXf5mCmJidsZf1msbE7DF2e1B5su8j33fffaxcuRLH\ncWhra2Px4sWsX7+eZcuW7W6SMTEvC88//xxf+9qX2Lp1K0ceeTRCCBYtWsz551+8p7MWEzMrzNoY\nQk9PD/Pnzy8dz58/n+7u7tlKLiZmRvF9n89+9hOsXHkKv/nN/Zx44jv54x9/H3cLxbyqmVILYbIZ\nyR//+MdZvnz5hNdM1AyfyiSamRotfzUQl0WZ2SqLmppEKf5K92OPPQYYLr74fABWrTqF2277Gem0\nW8qLlIKWlppJj2eL+LkoE5fFzDIlhXDzzTfvcsTz58+vsrjY2feUY2L2Jnp6emhtba3yW7BgQTze\nEPOqZka7jCpfluXLl7N27Vo8z2PLli1s3ryZQw+dmfU2YmJmm7lz527XxdnZ2Rl3GcW8qtlthfC7\n3/2O4447jieeeIILL7yQD3/4wwAcdNBBnHzyyaxcuZLzzz+fL3zhC/HLFPOKYdmyZViWxS233EIQ\nBNx77708+eSTezpbMTGzym5bGa1YsYIVK1ZMeO6iiy7ioosu2t0kYmJedlzX5Xvf+x6f+9zn+Na3\nvsWxxx7L8ccfj+M4ezprMTGzxrQVwmc+8xnWrVtHS0vLhKuY3nHHHdx0000ApNNprrrqKg455JDp\n5zQm5mVm6dKl/PrXvy4dn3766ZMaUcTEvBqYdpfRaaedVhL4E7Fo0SJ+8pOfcMcdd/CRj3yEz3/+\n89NNKiZmj/DYY4/R29tLEATcfvvtvPDCC7z97W/f09mKiZk1pt1COPzww+no6Jj0/GGHHVZyv+lN\nb4rXeInZKzniiCNKY1tHHHFE1bmXXnqJ1atXMzY2xuLFi/nud7/LnDlzSufPPfdc6urqJj2OiXnF\nYXaDLVu2mPe85z07DXfTTTeZz33uczsMs27dOvPOd77TrFixwtxwww27k629ns7OTvOBD3zAnHzy\nyWblypXmP//zP40xxgwODppzzz3XnHTSSea8884zw8PDpWuuvvpqs2LFCvPe977XPP3003sq67NG\nEATm1FNPNRdeeKExxpj29nazatUqc9JJJ5nVq1cbz/OMMcYUCgVz2WWXmRUrVpjTTz/ddHR07Mls\nzzjDw8Pm0ksvNe9617vMySefbB5//PF99rm4+eabzcqVK8173vMes2bNGlMoFPaZ5+KKK64wRx99\ndJV8nc5zcNttt5mTTjrJnHTSSeb222/fabqzrhAeeughc/LJJ5uhoaFJwwRBYE488USzZcsW43me\nOeWUU8yGDRt2J2t7NT09PeaZZ54xxhiTyWTMSSedZDZs2GC+/vWvmxtvvNEYY8wNN9xgvvGNbxhj\njPn9739vPvzhDxtjjHn88cfN6aefvmcyPov8+7//u1mzZk1JIXzsYx8za9euNcYY8/nPf9789Kc/\nNcYYc8stt5gvfOELxhhj1q5da1avXr1H8jtbfOpTnzK//OUvjTHG+L5vRkZG9snnoquryyxfvtwU\nCgVjjDGXXXaZue222/aZ5+Kxxx4zTz/9dJV83dXnYHBw0JxwwglmeHjYDA8Pl9w7YlYVwrPPPmtO\nPPFEs2nTph3G89e//tV88IMfNFrr3clOTExMzD5JUUkaY8w73/lO09PTY+68807z+c9/vuR/5ZVX\nmrvuumuH8czaF9M6Ozu59NJL+cY3vsH++++/w7Dd3d0sWLAAIUTpgxd/3zLEcNbjHw/ZN2c3xx//\nKBOXRZm4LMrEZVFmonXjprOe3LQVwpo1a3j00UcZGhriuOOO49JLLyUIAgDOPPNMvv/97zMyMsJV\nV10VJmTb3HrrrTuNd8PWYf7n0Xb+8nwvAK+79Bjqa9zpZjMmJibmVY8Zt6SKEGLCZVZ2Njl42grh\n2muv3eH5r3zlK3zlK1+ZUlzz589n27ZtANzy2+dp7y5/uP3ZzYMc+YbWyS6NiYmJ2ecZv25ca2sr\n8+fP55FHHqnyP+qoo3YYz7TnIXzmM5/hrW99K+9973snDfPlL3+Zk046iVNOOYVnnnlm0nBLly5l\n8+bNAIzlg6pz7d1xkzAmJiZmRxQnUD7++OPU19czZ84c3va2t/HAAw8wMjLC8PAwDzzwAMccc8wO\n45l2C+G0007jnHPO4dOf/vSE59etW8fmzZv57W9/yxNPPMFVV13FL37xi4kzYdtceeWVAOQKsUKI\niYmJ2RUWLVrEihUrSKVSXHPNNQA0NjbykY98hFWrVgFwySWXUF9fv8N4Zm1i2n333cf73vc+IJyY\nNjIyQl9fX9XEnkqOO+44jDGMRQrhbUvn8/yWITZ3Z9DGIOOF8WJiYmImZLKVIE477TROO+20Kcfz\nsn4xbWezlXOFAGPg0CUtfOg9b+B1ixvJ5Hxe6hyZrWzGxMTExETMmtmpCec4VPnt7ItpAyN5AJob\nUsydW8ebXz+fB57s4vu/foof/csKHFvuU0tox1+DKhOXRZm4LMrEZTGzzJpCaG1t3eUvpv3Pw+HA\n8gGttfT2jmITKpSh0QKnXXEXK4/en9OOWzJbWd6riG2sy8RlUSYuizJxWZSZKcU4a11GJ5xwwoQj\n3zviwSe3UZO0OfqNYVdTbap67fm1D21mOFOYnQzHxMTE7OPM2sS04447jnXr1m038r0jegbGeMMB\nTTh2qKcWzasl4VgUfFVO9/sP8K2Pvo3G2sR0sx4TExMTMwHTbiFce+21fPWrX6WtrY1EIsHAwABn\nnnkmZ555ZinMhz/8YebPn49lWaUP6uyM1uZ0yW1bkh9cfhz/evHRvPuocPkLY2DN9Q+gtJ5u1mNi\nYmJiJmDaCkEpxdVXX81NN93E2rVrWbt2LRs3bqwK84Mf/ICVK1dy++238+1vf5svfvGLO413opr/\nnIYUq96xhEXzakt+3QO56WY9JiYmJmYCpq0Q1q9fz+LFi2lra8NxHFauXMl9991XFUYIwehoOOgz\nMjJCa+vOl6Cw5eRWRCf946KS+/M/epRnNw1MM/cxMTExMeOZtkIorlBapLW1dbuV9C699FLuuOMO\njjvuOC688MLSbOQd0VbRChjPW5fO56sXhGtxaGP4xs8en2buY2JiYmLGM6Nmp+PnCNx1112cdtpp\nnHvuuTz++ON88pOfZO3atZNef+3qY3ntoqYdpjFvXj3fuuxYLv/uH4BwkPnrHz2GhXMnVySvVGIb\n6zJxWZSJy6JMXBYzy7QVQuUKpTDxPINf/epX/OhHPwJg2bJlFAoFBgYGaG5unjDO1y5qmpJdcVPK\n5pS3HcAdD2xiaLTA7fe/wPuXHzTdW9kriW2sy8RlUSYuizJxWZTZ4/MQiiuUdnR04Hked999Nyec\ncEJVmAULFvDggw8CsHHjRgqFwqTKYFc5cEF5kaYXO4cZiucnxMTExOwW01YIxRVKP/ShD7Fy5Ure\n/e53s2TJEq677jruv/9+AK644gp+8YtfcOqpp3L55Zfz9a9/fcYyfkCFQvh7xzBf+8lfZyzumJiY\nmH2RaSuEP/zhD1xzzTUYYzj99NO58MILAfjYxz7G8uXLAXj++ecZGRlBKcWSJUt461vfOjO5Bhpq\nXL5y/pG8blEjAD2DsRlqTExMzO4wrTGE4hyEm2++mdbWVlatWsUJJ5zAkiXldYY2bdrEv/3bv/Gz\nn/2Muro6BgZm3kR0QUsNa854Exd+cx2pxKwtyxQTExOzTzCtFsJU5iD84he/4Oyzz6auLhzsmKmx\ng/E4tsWblrSQKwSl1VJjYmJiYnadaSmEqcxB2Lx5My+99BJnnXUWZ5xxBn/84x93L6c74B+WtADw\ntxf6Zi2NmJiYmFc7M9bPMn4OglKK9vZ2brnlFrZt28bZZ5/NXXfdVWoxTMZ0zKeWH3EAt/z27/z5\n+V7OfNfrd/n6vZXYxrpMXBZl4rIoE5fFzDIthTCVOQitra0sW7YMy7Joa2vjwAMPZPPmzSxdunSH\nce+OXfHz7YN84Au/4UMrX8/SA1umHc/eQGxjXSYuizJxWZSJy6LMHp2HMJU5CCeeeCKPPPIIAAMD\nA2zatIlFixZNFN2MMpzxuONPm2Y9nZiYmJhXG9NqIVTOQdBas2rVqtIchKVLl7J8+XLe/va388AD\nD7By5UqklHzqU5+ioaFhpvNf4poLjuIP6zv5zcPtdA2MzVo6MTExMa9Wpj2GIISo2iCcg1DJFVdc\nwbJly1i9ejX777//7uV0J7Q2pzn9HQfR2ZvliY39ZHJ+1RfXtA4/xyl3sJpqTExMzL7MtLqMpvIt\nBIBMJsOPf/xjli1bttsZnSoL5tQA0NmXLfkFSnP59x/gkz94kEDFH9aJiYmJmYhZm4cA8N3vfpfz\nzz8fx3Ewxux2ZqfCgpbwi2svbRthOFNgc9coXf1jDGc9BkcLPPx0905i2HMMZQrc+vuNdPRm9nRW\nYmJi9kFmbR7CM888Q3d3N+94xzuA7c1SZ4vF88LR9p/fv4GPX/8AX/yPx7j/rx2l8xu2Dr8s+dgR\nv/jfDfxq3fYtqvv+0sHdD2/mulvXz1raz7cP8vRLMzNrvOApnm8fnPRzpn1DOR54cht+MPutMj/Q\n3HTXM/z5uZ5ZT2tvxA9UXJHYCUrrUsV049ZhXto2sodztPcxK/MQtNZcc801fO1rXyv5TbWFsLvm\nU41NNTTUugxnvJLf7x/vLLk7+7NsHcxx8OIm0klnoihmlW19We55pB2AD7/vUGxLkisEXP2jR3hy\nYzixrm84jx+o7coiXwh47Nlu/vENrSTdqf903/nZX3lyQx+f+ecj+PpP/4YU8NOr301Navfu/7qf\n/43fPdrOaccfxLnveWPVOaUNl3//AQZHC/znPc/xnY+/g/0rFiTcVXLKcP+ft/CBk1+PbW1fj3no\nyU4efKqLB5/q4r+uPpnatLvLaTy/eYCegRxvP2y/aefz5WD8c/Hc5gE+eV048fO6y9/BgQurjTc2\nbBni1vtf4Pi3tHHk0gXMBMYYcoVgt94hP1A4tgXA/X9uRylD0rV3qfynKi/yXsCa76yjtbmGy89+\nC1/58f2lcxefdijvfuuBu5b5HWCMoXcwx7yK78O/UpiVeQjZbJYXXniBc845B4C+vj4uvvhifvjD\nH/LGN75xu/gqmQm74m9fcgz/+9cOHnuuh+fah0r+8xpTbOgY5sobHqKpLsHXLjwax64WLlobfKVJ\nONak8W/tzfD0pkFWHN425ZZPoDSWFNzz0OaS3xPPdrG4tY4Hn9pWUgZFntzYz6LmVOk4Vwj46LfD\njwId+YZWLjxlx+VYyX2PbQHgv9dtAEAbeOypTt54QHk5kZExj7se3MThr5vHwdGCgTvj8b+HtfGH\nn9rGyiMXV5276a5nGBwNlyQPlOGSb/4vn/nAm/np717g/ccv4fVR2mN5n5vvfo6Tj9qf1yycWGHM\nnVvHR7/xvwDUJW2OfdPC7cL88LZyq+qWu5/htOPK62r1DI6x7vFOTj3mQNwd/K7/8sMHKXiKx5/v\nZtU7lkyoeHYFrQ0vdAyxZL+GSeMaHC3QUOsip/gcjbe9v/vhzdz6+3Jr8+kXeql1wrT++EQn/3Xf\nC9iWJJPzeWB9J6tPPxQ/MAyM5nn02W4+8n/+gaa67b9jPp6BkTwJ16ImGXb/Xn/bk/zthT7+5Z/e\nwpKFU7cefPjpLn567wtkcj4AH1t1KI21Lt/+r7+VwrTUOsxtTE0WxaRlsSOefmmALd0ZtnRn+L93\nPVV17ge/Wk9PX4aTj9p/yr/DZCitufnu53jwqS4+edZhvH7/HX/wa6aYqXkI01IIlfMQ5s2bx913\n3821115bOl9XV8fDDz9cOj7nnHO44oordqoMZpLj39zG8W9uY3C0wL1/2cLSA5oZGC3wo7XPAuGL\n+PyWQbQ2PPXSAO8//iByhYDLrvsTAFec/eYJBWOgNFf+6FEAHn+hl8vPXIYlJxccBU9x/W3reXrT\n4Hbnbl23kTXvX8YLHdt3Y33hxof41FmHsai1lpqkw9be8iD5I890s+q4JbQ0JHdYBi92jvDl//vn\n0nGl0nlx6zBLFtaT9xR3/Okl/vx8L5mcz7ObBllzxjJu/f1G2ubWcMJb2nAdCz9Q/Oc9z7NkvwaO\nP2w/+oZz9A6Fa0eNX0PKDxQPPtW1XX6uuSVcovwbP3uc9739QI5/cxsf+25Ys92wdZhvX3rMDu8H\n4D9+8xytTSlet7j8om3sHGZgpPw9jBc7y10BgdJccUP4LC5qreWoN8yfMN6nXuyn4CkAfvvYFoyB\ns0587U7zMx5jDEIIBkcLXP79BwCor3FZsrCeM5YfxLymcq3xby/08r1fPck573wdx0+jVfLQ011V\nygAoCVqAux9pJ+8pQJX8vvPL6u7Iy7//AKtPP5RDl8yZNJ3eoRz/8m8PY1mSy89Yxm8fbS8tE3P/\nX7by3398iXNPPoTm+iSB0nz9p3/ljQc083/e/pqqeHKFgBvvfKbK77pb13PK2w6o8tu4dXhKCqGI\nH2gyOZ+mugQdPRn+tqGP4w/br2Rl2DuU49qflz+3+5uH27eL41frXuSQxU28ZmE9T77Yz2sWNlRZ\nKU6Ve//cUXr2H3q6q6QQjDFs6ckwMFpg2UGTlzXAQ091saFzmHxB8aH3vL5KSd3zSDuPPNPNZacf\nSmPtzhX5rmJdddVVV+3qRVJK9t9/fz75yU/yk5/8hFNPPZUVK1Zw3XXXkc1mOfDA6ubX7bffzjHH\nHLPdbOaJGBvzdhpmV0glbN54QDNzG1Msbq3j2DctZHTMp6M3w0NPd/PwM9282DlCXdrhxjufKQmF\nPz25jZP+cRGOLfEDxR1/2sTCljQPPtXFky+GffB9w3n+4cAWmuuTbOwc5rFne3hm0wD/cc/zvLBl\niKUHNvORb/+hJDjH0zOY45S3HcDdD7czlPG49tJjOOHN+/G7P4djHg881cVvHm7n4LYG2nsyPPXi\nAAta0mRyPoHWvGncS/yrdRu555F23vK6uViW5Js/+1uVgMgVVFX4ex5p57Y/vMimrlG8qJ8/k/MJ\nlOEPT3TyzKZB7npoM48918PahzazYesw6zf28+6j9ufRZ3t48sV+IGwBHDC/jvlRE7mzb4zf/23r\nDn+X59qH+HvHUKkVUfAVxhhes7AeKUVVy2s0H3DnH18sHb+0LawVGgP/+tO/8puoC+7/Z+88o6Oo\n2mY+/lQAACAASURBVAD8bEmvpPcEQiC0UKS3SIBQAggKiAUVROxgVyygWBFFUVDhEwRFUWkCAoJK\nUXqvAUIo6b1sym62zvdjk91sOpAFIvOcw2EzO3fmzt2Z+8596119m5OnKCO3sIxBXYP483CqRZ0M\ng0Gga6Q3BoPA1+tP8836M8hlEloFu5uERgWX0ovoHOGFWqtnxuL9/HUklahwT1xqUEVpdQbOXsnH\n2cGWN/53gFyFiqx8pWl1qtbqycxXkldURo+2voBx1fLBD8a+FZaoubMWgZCSXUKZRmeanJyc7FAq\nNSjLtMz9+Th6vVEV27OtL6k5pfi4OxAV7onBIPDr9kQaoqnNLlRVW3Xp9Aa0OgNZBSoOxmcRn1SA\nXi+wPz6L1EovJ6k5JWQXqkjOKqZPB38y8pWs2XWJ8ymFRIa44+XmUP5bCZxIzONQuY3HxdEGjdZ4\nz51PKbQ4d5lGh0QiYdbSgyjLdLQL8+Dg2Wx0egPuznZcTFeg0RpIy1OSm6/kryOpfP3baUJ8nJnz\n0zHOJRWw5UAyUeGe2NnIeGHBnlqvvVc7Xzxc7ckqUBEZ2oy0nFIWrjvNlgPJFv2/mKbgk5+PczFd\nQdswj2qaBTCudueutKzzXvG7/m9jPCv+TOBAfBaCIOBgJ+eN/+0nJbuELq29Tfe7olTDByuOcCWj\nmNScEtqENjP1AeDzVSfIVZSx9WAKxxJy0BsEAr2dcHWp++WwoUiE63D/+eeff/jggw9MwWlTp061\n+P67775j9erVyGQyPDw8+OCDDwgIqL7cr8yNCEXPLyrjpa/21rvfqD5htAltxo5jaRw8m02YnwvK\nMh3ZhebaCz7uDgzpHswP2xKqtY8K9+Tkxbxq2796oT9PzfvHdI6tB1PwdLPnvSk9AJj80fZqbSp4\nanR7vll/Bn8vR5RlOgqK1bQIcOXpMR1Mb6ThAa5cTK/dYBbi60xyVnUDpKerPXlFZdjKpSYBURPD\ne4Zy7EIOGXlKxg9oyaodiQT7OvP2pO4AHDybxTfrz3DfwAjUWj3uznZGo2duKaG+Lmzcc4W8OjLT\n+ns6Eh7oxh2tvElIKTRN+G7Otmi0BlRqXY3tJg2LJLtQxaZKarmqDO4aTGSoO1+uOWXaNvORrsxe\nZlxJff5sX77fep6jCTnV2o69M5zhPc3xNIIgsHHPFTbvT0KjM1j83oFeTqTlliKVSDBUesQq1AgL\n157iSPk5OoZ7Mn1cR8Bo9D+akEtUuCfr/r1kWu0sfc1YY8Tb24Ws7CKe/fwfVGo9fdr78eiItmh1\nep6a9w96g8CnT/dBbzDwytf78Gnm0KBaIW1Cm3FX3+a0CnanqFTD64v3o6w0zhIgIsiNhPLVbFyv\nULYfTav1twBwd7YlpksQg7sGs/d0hukZuSe6BXG9wjiXVMDHlVRFtRHdKYBd5XbAF+7tyLxfTtTb\nBsDJXk50p0A2769+P3z8ZC/TRHvyYh6fr6r5mItfvhO5TMoXq09yPDHX1J+Hh0ai1ur5/o9zyKRS\nMvOVFg4rIT7OJGeXMGl4JL/8nWgxllV5ZFgk/TsGoCzT8sznlklA/T0deW9KD5PAePqzXdVe7AA2\nfnpXPaPRMK5phQDGWISpU6eydOlSpk6dyvvvv0/37t0t0lxrNBqmT5/OxIkTUalUrFq1iqFDh9Z5\n3MZeIdSEg52ctmHN6NvBHxu5lOb+rlzJNAqiMf2a8/TdHdiyP5nzKYXsOZVJWnlMQ2GJhtIyHX07\n+DNtbBTbDqVQWqarcdIHyCp/EJv7uzDjwTsI83OhW6QvYX6ubD+aikZr4HxKIXqDQFQLT7q08gag\nexsfStV60qp4jbQJbcaoPs05kpBNeq6yXB1gVH9tK7cTVPxdmbcndUOl0ZOWU8q4O8NJyylFUWo5\nzg/GtqJ1iDsnEvPQGwQ8Xe1ND3uXVt5EBLnTq50vpy/ncyFVQYlKy4DOgdwTHc7ZpAIupRfh7W7P\noXM5JKQqSM8tJbZbMH2jAgjxdaG5vysdw70I9XMhtlswnVp6mR7yQXcEcamSx0eJSktyVgkH4rMs\nHjJ/D0fGD2iJRAIZeUoEIDLEHTtbGW1CmzGidxgRQW7VJoD/vXInG/dcAYxv85cziy1UTBX9aBXk\nxsCuwXRt7cPfR1KreUe5OdlSWqYjp1CFp6s9v+xIZMuBZPTlQY9ZlSbeYqWWqHBP3p3Snbv6NGdD\n+fn3ns4kKtyTn7cnEurnQplaj1ZvwNnBhl93JrLun8tcSi9i35lMi9/R292e3SczCfZzYd3ORM4m\nGd+qH41rg5uzHTKplNOX8skvViOTSricXkximoKBdwQxpn8LRvdrjqOdnDB/V9OYvvHQHfx7wmgL\nzFWUUVBcRpC3M68t2o+2SrxOu+Ye9Ongb1IVtQ5xRwK1rn4ByjR6ziYVoNbq2X0ywzQpDrwjCH9P\nJ7zcHVi/+zJgvOdfn3gHlzOKyFVYHjMp0/ySuK8BbuNymYQOLTxJzSk1qWNfvLcTd/UN468jxtX3\nhIERpknWRi61eH4qs/VgMiN6h3H4fDYZeUpTf8IDXPl1RyJHzueQXK4KquDxUe3w83Dk1KV8jl/I\nNY2lXCbhwdjWnKgyX5y+nE+fDn68/PU+UwBtBSUqLVq9wWTv23owpcaXtfuHRNY7Lg3hmr2MKsci\nAKZYhMpFcnr06GH63LFjRzZs2HAdXW1cIoKM9oFWwe7o9AbikwqwlUvp1zEAp3o8J9qGNcPD1Z5n\n7+7Al2uNb5pxvUK5nFFE/JUCizeaEB9n3nq4G4CFXvSF8Z14Z9kh09+VS4L6ezrRr1MgB86Y9fAD\nOgcycUhrAMID3Uw3Z1Wa+7uYVCpebvbcG9OSEF8XHh/VzmSI9vN0tHhDBgj1dSHIx5nv/zgPQKC3\nE8/c3QFXJ1uT0VEQBJRqnWlyG9YzpHws3UhIKeTb389aHNPPs3Yvi1A/F+5o7Y2rky2DugWz70wm\nnSK82HOquu0BQC6T8uTo9ni7O9A10gf1cD2X0hQm43RlHhvZlsUbjLrquF6hyKRSOrX04nhiLinZ\nJeXj5Mr9gyOY98sJk+DzcDUuu6VSCY+NbGvStw/rEcLO4+nsO5PVoAmpAn9PR5N9adyd4awq1/e/\nu9y4GunexofL6UUcPp9TTbcuk0pMggYwje2fh40Tl5O9nDcf6opvJU+W8TEt+eCHI6YVFUCLAFeT\nLWxU3+ZkFSj540AyPdr6Eh7gxh2tvTly3rhSOXOlgCu1vLFHBLnRp4M/VzKL+ftIKlEtvIgIcjfZ\nxpzs5ZSWGcexdbA7wb7O/FWu+jxzOd9iRRgVbk48eXf/FhxNyOHRuLbYyKVMGxvFk5/uasDowsNx\nbfF3t8ff05EipZb4y/kMquTo8ev2RP44aByLYF9nXB1teWF8R2zkUguVpIerPaG+LiRlGZ+bOU/0\nIjmrhIXrTqHRGUjPLTWN0UNDWvP91vPM+7X6isKnmQOj+zWnR1tfLqRaqsFG9A5jdN/mSKUSfJs5\nmNLrnEsu5NC5bD7+6ZjFC8inT/chIaWQRRvOsGV/Mn8drv6C8mBsK/QGwaSqbQyuWSDUFItw8mTt\n/vOrV68mOjr6Wk9nVeQyKe9P6WGR1uLJ0e35+rfTONjJqi3RKia6zq28mT+tL1KpxCRE0nNL8Wnm\nwJFyI+3dlbxdKhPq50L3Nj4cPGvUqYb5W3oJtGthfmikEgnRncyqtl7t/Nh90vhm9/UL0azYdp49\n5YasEb3DTJP9R0/0qtFronOEN8/e3YGsAhWZ+UqUZVqaB7gilUjo1c6XfWeyCPRyItTPsk8SiYTY\nbiEmgVCx5G4b5lGjmqZigq2Np8d0MH3+8rn+ADw4uDVJWcUUlqj5Zv0ZALq09uGxuDbY2Zo9hOxs\nZDUKA4Cebf2IauHFqUt5dI4w2lmmjY0yqeICvY2V9pzsbZg9uTsvf70XOxsZg7uZky9GhXvx7SsD\nSExTEBHkho1carruqjw/viNebvYs/+M8Pu4O7D5l/G0CyqPmAYb1DCUytJlJGAAMuiOYMx75HD5v\nqZ56aUInQnxduJBaiFQiYX6VuBQnezmvPtDFQhiAUeVXlbAqrr6+zRz5/Nm+ODkYH/2K32D2skNc\nySw2TepvTLwDJwejR9H2o2nElo/NA4NbMapPGC6OtugNBqI7BRAV7knnCG+Ss4rxcLXH2cEGg0Eg\nPMCNlX8lWLy8PDW6vYXH1YjeYYzoHWb6285GxufT+rJ6x0V6tvPlnxPpCAL07xjAp5UMw8+P70hM\njzCTitnF0ZbASuMNENMl0CQQXMttP+1b1JwFedakbny17hSCYHxxq2ywffNbY5JOicS4Wv5+63nT\ndz3b+tImrBl92vtbzB+BXs6mz+MHtGRoJS+8NmEepnu3cyujQK5QQ4/p38JkEO/Y0tzXysIgtlsw\nw3uFmq6pMWnUupO1uWCuX7+e+Ph4ZsyYUe8xbpX85sO9XRjezziZn0jIITmrmMW/naJFoBtd2voj\nK7+pvau0q+j/Z89Hc+RcNjE9Qmsdl/YtvU0CoVuHQGRV8iz98v5w7GzlqMq0Fn713t4uZBSoKFJq\nCAp057VJPTiRYBRAd0T68Nvuy0R3CcLXp3a//9haxvn5B7oSufcKw3qH1VqW9Pn7OuPTzNF0rV5e\nzmzYe4XzSQW4O9tRWKIm0Nu5zvPXRVCgOxqtnqScUsID3RnS89ryYIUG1+zyd1f/cMKCjQ+kt7dL\nnfpXX1/jNTwwvC0laj3ODjacS8onIbmQO7sEMbhHCFEtjXfBp5FGD6aBZ7OIv5zHyOiWJj/7inM9\ndY+Go+ezufvOCAL83Qjwd+NMcgGnEnO5Kzqcji29TfEazUOMfTRIpXz5q3kyvCcmgs5tq8cTeHg4\nVdsWHlp9AvSuetMCT9zTkdcWGj3sYroG07NTkOm7qEhLz6zKzV+a2M3i+iozwteV1i08eXH+P7g4\n2vLZc9EN8s33Bl59xNjv6G7m37535yBOX8xj/5kM+ncNqfGcFsfxdmHq6A44O9o0aF6ZNdWy5nuL\nQDculavXfD0cWfDyAOxt5Wz4ZBTnrhTg6W6Pt7tDva7no+5siVstHkHe3i7MndaPF+cbbYoDu4da\nxOssnzWEh9/ZatHmvqFt8LoKL6yr4ZqNysePH+fLL79kyZIlACxatAiJRFLNsLx3717ee+89VqxY\n0aAymrdyfvPMfCVebvbX7Z9eQYVRLdTPhVmPdLP4rqnlehcEgVOX8okMcefQuWyTCqoxaKyxOBCf\nxYH4LB4ZHnldb1cqtQ6VWlfvCqixycgr5VJWCT0jvWt1dX7ik51odAYiQ9yZMDCCEN+Gv2DlKcr4\n92Q6g7oGX5PLZW1cySzCzcmuQfEOV4O1n5GsAiWf/XqCXu38GNk77KoTY/59JJXsAlWD3JdPXswj\nLbeEYT2qv/wkZRbj6mSLva0Me1tZjQKosV6kr1kg6HQ6hg4dyrJly/Dx8WHcuHHMmzfPwoYQHx/P\n9OnTWbJkCSEhIXUczUxTmgQbg6MJObQKdq/2ADY1gWBNxLEwU99YnE0qIKdQRb8o/xuWLuZmId4X\nZm5qYBrUXROhQ4cODBgwgLlz56JSqUxpsQMCAvjqq68apeP/FSo8i0REGoM2oc1uWHSsyH+P69J9\n1FYTYcCAAYBRjdS1a1eUSiW2tra88cYb199jEZGrZMmSRSxdurjaZ2uxefNGPvjgnVr/FhG5Vblm\ngdCQmgirVq3C3d2dbdu28cgjj/DJJ59cd4dFRK6Wyi8sN0KNUvUc/3XVjch/h2sWCA2pibB9+3bG\njBkDQGxsLPv27bu+3oqIXCM3qh6HiEhTxqpxCNnZ2fj5GV3W5HI5Li4uFBYW4u7esGyaIiLW5vz5\nc3z00WzS0tLo0aMXEomE4OAQHnvsyVrbFBcX8+67bxEffwa9Xk+HDh15+eUZeHvXn6tLRORWxqpx\nCDW9ldW3fL5V4hBuBcSxMHM9Y+HkZGc6RuXPGo2Gt956hcmTJ3P//fezfft2nn/+eR577LE6z2dj\no+f++yfQv39/dDodr7/+OgsXzmPhwoUAuLjYY29v9n2v+vf1It4XZsSxaFyuWWVUX00EMK4aKvbR\n6XQUFxfj5tbw3OkiItbkxIkT6PV6Jk6ciEwmY/DgwURFRdXbzt3dncGDB2NnZ4eTkxNPPPEEhw4d\nqrediMitzjWvEOqriQAQExPDunXr6NSpE1u3bqVXr17X3WERkcYiOzsbX19fi23+/v712htUKhUf\nfvghu3fvRqEwRrIqlUpTLQQRkabKNa8QKschxMXFMXz4cFMcwvbtxpwx48aNo7CwkNjYWJYvX86L\nL77YaB0XEblevL29q9UCT09Pr3dSX7p0KZcvX2bVqlUcOXKEFStWIAiCaLgWafJclw0hOjq6WsK6\niiA0AFtbW+bPn389pxARsRqdOnVCJpOxYsUKJkyYwM6dOzl16hQ9e/ass51SqcTe3t7kJLFgwYIb\n1GMREevSOEl5RESaILa2tnz55ZesXr2abt26sXHjRgYMGICNTd15fB5++GHKysro0aMHEyYYjcui\nqkjkv0C9K4T6qqKtXLmSn376CZlMhqOjI++++64pn9GiRYtYs2YNUqmUN998k75966+ZKyJyI2nf\nvj2//fab6e9x48YRExNTZxsfHx9++OEHi2333nuv6XNV1ZGoShJpKtS5QmhINPLIkSPZuHEjv/32\nG1OmTOHDDz8EIDExkc2bN7Np0ya+/fZb3nnnHQyG2ssyiojcDA4dOkROTg46nY5169Zx4cIF+vXr\nd13HFCOVRZoqdQqEhkQjOzubUxwrlUqk5Wl5//77b+Li4rCxsSEoKIiQkJA6C+j8888/DB06lNjY\nWBYvtm6umZtNRkYGEydOZPjw4YwYMYLvv/8egMLCQiZNmsSQIUOYPHkyRUXmspLvvfcesbGxjBo1\nivj4+NoO3WTR6/WMHj2aJ554AoCUlBTGjRvHkCFDeP7559FqtYCxLOtzzz1HbGws48ePJy0trd5j\nd+/e3VS9r3v37nTv3t303eXLlxk9ejTdunVj2bJlzJ8/Hy8vL7755hs6d+5c7V/VFXJNtGnThkGD\nBtX6d10UFRUxbdo0hg0bxvDhwzlx4sRte18sW7aMESNGMHLkSF588UU0Gk2j3he3MjNmzKB3796M\nHDnStO1a7oN169YxZMgQhgwZYrESrhWhDrZs2SK88cYbpr9/++03Yfbs2dX2W7FihTBo0CAhOjpa\nSEpKEgRBEGbPni2sX7/etM/rr78u/PHHHzWeR6fTCYMGDRJSUlIEjUYjjBo1SkhMTKyra02a7Oxs\nIT4+XhAEQSgpKRFiY2OFxMREYc6cOcLixYsFQRCERYsWCXPnzhUEQRB27twpTJkyRRAEQTh+/Lgw\nbty4m9NxK7J06VLhhRdeEB5//HFBEARh2rRpwqZNmwRBEISZM2cKP/30kyAIxntt1qxZgiAIwqZN\nm4TnnnvupvTXWrzyyivCqlWrBEEQBK1WKxQVFd2W90VmZqYQExMjqNVqQRAEYfr06cLatWtvm/vi\n0KFDwpkzZ4QRI0aYtl3tfVBQUCAMHDhQUCgUgkKhMH2ui6sWCO+++26t+2/cuFF49dVXBUEQhHfe\neaeaQNi2bVuN7Y4ePSpMnjxZMBgMdXb2dsFgMAjFpepq23ceSRHW/3NjBGVOgVI4fTH3hpxLRETk\n+qgQkoIgCEOGDBGys7OFjRs3CjNnzjRtf+utt4Tff/+9zuPUaVRuSDRyZYYPH87bb79tapuZaS6Y\nXlfbirxIEolELHgBrN55kc37k3jjoTsIDzBHdn/y4xEAekVaP2dORf3hBc/1w9G+8apnXQtiIRQz\n4liYEcfCTEXOuIrPWVlZFrnkKm+vizptCJWjkTUaDZs3b2bgwIEW+yQlmYur79y5k7CwMMAYpbxp\n0yaT3i8pKalBaQFEYPN+45ievpR/k3sCaq3oCCAicqsjVPFkk0gk15RLrs4VQl1V0dq3b09MTAwr\nVqxg3759yOVy3NzcmDNnDgAtW7Zk2LBhxMXFIZPJmDVrVq2dqboSETFS0w8KoDcYaq2pKyIicvtR\nVRvj6+uLn58fBw4csNheX9BlvbNKbVXRKny1AwICEAQBg8GARCLBwcHB1Hb+/Pk4OTlhb2/PypUr\naz1HxUpExEiF3DTUIhB0uhvn124wiD70IiK3OhUeRMePH8fV1RUvLy/69OnDnj17KCoqQqFQsGfP\nnnpjwepcIVTEIXz33Xf4+voyduxYBg4caAo8A2jbti1r167Fzs6OlStXMnfuXD777DMA7O3tG+Tq\nVLESETEilUjQCwK1hW1odHrsbGU3pC/68k4YBIHXvtlHh3BPJsa2viHnFhERaRjBwcEMHjwYBwcH\nUyyYu7s7Tz31FGPHjgXgmWeewdXVtc7j1CkQKschAKY4hMoCocK/G6Bjx45s2LDhmi6oak6k2w2D\nQUAiMa7IpFIJekPtydK0uhun19fpjX0oLtWQqyhjx9E0USCIiNxizJw5s8bt99xzD/fcc0+Dj1On\nyqimqmh1WalXr15tMbFrNBruuece7r33Xv76668Gd+p2o6BYzZSPd/DrjkReX7zfNOFXlgcard70\nWauvWyAcTchh8kfbuZJZVOd+DUFfrjLKK1Jf97FERERuba4622lthuH169cTHx/PjBkzTNt27tyJ\nt7c3KSkpPPzww7Ru3Zrg4OA6j387VkD6ZacxHcjWgykW2+3Kq2zlKVQ8Ue4GCuDs4lDnOP26eD8A\n/57KpFuHQNP2c1fymb3kAO890ZsWgWZ31uwCJV+vOcnTYzvi5e5gcSxXV+O5EtLN7n034ze6He+L\n2hDHwow4Fo1Lo8Qh7N27l0WLFrFixQqLTJHe3t6AUb/Vo0cP4uPj6xUIt5tfcWKqgq37azaolyrV\n5OQUc+R8jsX27JxinOR1uI+Vv9WrVFqL8fzy12MUKzUsXnuCFyd0Nm3/ZOUxziYVMH/lUaaNjSJX\noTJ9l5tXgpu9jLRMhfn82UU3ND+P6G9uRhwLM+JYmGkswXjdcQjx8fHMmjWLb775Bg8PD9P2oqIi\nNBoNAPn5+Rw9epSWLVs2SqdvBeb9cpxlW85edTu1Vs/afy6Rkl0CQF5RWa37CibNkKUtoT4bQk1e\nSgXFapKzjOdUafQW+5dpdMbjlquiXvl6n+m7vKIyLmcUUazSVtrf2F6l1pFdqEJEROS/wTXHIXTo\n0IEBAwYwd+5cVCqVqTBOQEAAX331FYmJicyaNQupVIrBYGDq1KkWxuimhKJEzcJ1p7l/cARhfq58\n9dtpTl82Bo09MqxNg49z+nIe8345AcDve6+w9LUYbOW1y2RDuSDQV3H9TM0poVWwe63tpFKjRKho\ndvJiHgvWmhMLVhZCqTklXM4wvmWduZxfTdh8s/4MAD3amktNFhSrcbCT8/mqE1xIVfDp031o5mJX\na39ERESaBtcchzBgwAAA+vfvj5ubGwaDAXd3d958800AunTpwuTJkykrK0Oj0SCT3Rg3SWuwYe8V\nEtMUzF99Ep3ewOFz2RbfC4JAsVLDd5vPkpZbWq29skzHhdRC1u++bLHdIAiotfpq+1dw5lI+3/4e\nz/4zlob8vw6ncjmjiHeXH+a1RftQl7+xn7mSz55TGabfSRAEEtMUfL7qhMlbCEBRojG1OXkxz+LY\nB8/W7DRwIN68PVdhFCgXUo1qpB+2nkdZpq2xnYiISNPhuushVMQhbNiwgSFDhjB37lzAmKp14cKF\nrFq1ilWrVrFgwQKLdK03i6x8ZZ0FS0pUWi6mKyw3lu+u1wsoSjQWX6nUOqbO3cn0L3bz78kM5q48\nVu2YP2+/wIcrjnIxzfL6i5Vak/qlJrILVew9ncnxxFyL7Zn5St5dfpjLGUVkF6h4ct4usgtVfPrz\ncZZsOotcZhQIaTmlfPDDEYu2nSO8ACgoqdlraNXOizVur3r+yhxPzOXjGq5bRESkaXHd9RB69OiB\nnZ1RXdCxY0dTCPXu3bvp06cPrq6uuLq60rt3b/79918rXUbD2H0ygxmL97PjWO250mcs2sf73x/h\n699OU1BsnDQrdPKCIHD4vOXqYM+pDAuVTlGpBp3ewN9HUiksn3TPXimo8Vw7jqaaBMLovs2v+boA\nlm85Z/pcYSuoOnEDBHob61cUll9bVYFUVKqp1qaCds2NNqLE1EJ0VVxfK84pIiLSdLFaHMK1ZNqz\nJoIg8OOfCQCs2JbA91vPW3xfotKSq1BRWmY0sB46l83Pf18AzK62eoPAL9sTLdqdTao+2a/eeZEf\n/0zgt38vUVCsrtVwvGHPFZNBNyLIjdF9mzNlRMNtEpVRqXX17hPdKQBPV6Pwzi5UsT8+k9/3XgHg\n6TEd6m3fKsgNW7mUYxdymTp35zX1U0RE5NbFanEI15JpDxrfr/hyuoJXF+xGrzegqWQw3XksjSfH\ndsTBTs7eUxl8+etxSlVV9OBSCd7eLkjKE8nVpN5JrcFmsO2QMZ4gr1hDcg3fV+bPw6kABPi50b9b\nKCVKDd/+Xt17ycleTmmZjifujqKguIxfyoVbn6gA9pxMr9NbCeCViV3p1cGfKxlFLP/jPBkFKgt3\n18hwrzrbA3Ru48fu05nkFNTsWeTl5Ww1d1TR39yMOBZmxLFoXKwWh3Atmfbg+uIQ/jiQTHN/F1qH\nNDNt+23HhVrfns9cyCYzX8niDZalB58a3Z6vfjvNofgsRr64vlq7uF6h7D+TSV6R2jQ5+jZzIKvK\nRHnmUh5nLlkabe+NaUl4oBt/H0nlQHyWqW+q8piD2hLaPTG6Pe5OtiaVj71MgqJUw6g+zcktUHI+\npdBi/5ZBbiSmmm0hkYGuFOSX4iSXYCuXEl+lX7oyDZ881Zsv154iKdPyNxjQJZA7WnkT5OGAo53l\nLSMrT7MBkJJWiIPdVb9j1Ivob25GHAsz4liYueXjEPr27XvVmfauh4JiNb/uSGTOT8c4cj6HXQ/A\nqgAAIABJREFUdf9cQhCEOtNEz152mO82n6u2vWukDxFBbjW0MOLubMfofi0str0/tSftwoyCqOqk\nCTDjwS68N6UHQ7qH0DLQjWbOlm6a9uXJ6qSV3rBnPNjF9Lm5n4tJGABEdwpkVB+j3aF3e7Nqbuqo\nttzVtzmThkXW2He5TEqYvyupOZY6fztbGR6u9rg4GgV6c38X4nqF8unTfZgY25q2YcbftuJ7gLlP\n9mbOE73wdLUH6o6pEBERufWxWhyCm5vbVWfaux7yK01GC9edAmBjuX68Lqr63dvaGAXI5OFtmFGe\nAqIq9rYyC/UTGCdyWxvjpO5gJ+eRYZF89dtp0/f+nk44O5gnU/cqfvsOttV/ioggd0b0DqWoVFtn\n1bJQP/PbQZC3Mz3bGgXE46PakV2ooluVCmu92vmSUGlFMapPGPbl579/UCv+t/EMU0a0xd/Tqdq5\nolp4mgr3eLoZBcHgrkH8vD2RjDwlQZWEloiISNOi3vV9bXEIFTz11FN88MEHJCQkMG/ePIYMGWL6\n7s0336R1a2NmzD///JMxY8Y0dv/R6Q289/1h8utIvubv6ciYfi1ISClkQJdAMvKULFh7ymKfibGt\nOHQumwkDIwDw9XDki+n9mDa/umeUg52c9s1d+aGKYfru/i04l1zA+JiWdI30YcmrA3h0zg7AaAOo\nTNVArgpBBPDR4z1xdXMsP2b9wXx+Ho6mzy6OtqbPlYPJKtMuzLySaxXsbrHa8fNw5K2Hu9V6roF3\nBFGk1BDiYxZCFfmPCsQVgohIk+a66yEEBATw0UcfsXTp0mrtG1oP4XrIzFPW6fLYJrQZE4e0xs/D\nka7lb8r+nk5MGxvFF6uN0bs92voyoEsQA7oEWbR1drDhvSk9eGfZIfp08Gdnubuqg60MN2c7lrw6\ngE9+Po6fp3FCDvR2ZuHz5myvEomE8ABXbOTSasZWD1dLgVD5e59mjlelH61YmRj7XL8O36P8zR6q\nC6r6kEgk1YRURdyDvo74DhERkVuf666HEBhozKYpvYElHUtUWl79Zh8eLnYmX/8KfNwd6NHWFwc7\nOSG+zibdd1U6tfTi21cGkJ5XavGGXZUALye+eTEaAUwCwb7cRiCRSHj5vs61tgV446GuNW5v7u/K\n+AEt+XVHYo3fXy0PDWlNrqKsQaU1K9sp6kqB0VAqUmXo9aJAEBFpytQpEGqKQzh58mQdLSypqIcg\nl8t57LHHGDRo0LX3tBLbDiWjUutIq+Q9NDG2FQO6BCEIQoNdH6VSSYN03hKJhMpHlMuuX/hJJRKG\n9gihQ7hnnZHTDeXOzoH171QDUeGe131uWfl4i+U2RUSaNo0Wh1AT1qiHIAgCpy9XDwYbOzjS9KZq\nLT54sg97T6XTqY1fo52rruu1po/1rCk9uZyuoENr3+uOHcgqt9/YOdhYrc+iv7kZcSzMiGPRuDRK\nHEIFVSeWxq6HIAgCq3Ze5EpGEZ1aevHA4FZs2HOZbpE+5OVZP3WCn5sdd/dtfkPOZW0f61AvR0K9\nHMnNvf5rKS4uK/9fbZU+i/7mZsSxMCOOhZlbJg6hAkGwrAFsjXoIF1IV/HEgGWcHGx6MbYWnmz2T\nhrehfYvrV3uIXDsyqagyEhH5L3DNcQjt27cnJiaGkydP8uyzz1JUVMSOHTtYsGABGzdubPR6CMVK\nDZ/9aqwlMHl4Gzxc7etpIXKjkFbK9SQiItJ0ueZ6CDExMQCo1Wo8PDzQaDTMnj2bjRs3Ao1bD0Gt\n0fPxT8dMtQPahDWrp4XIjURcIYiI/DewWhxCRT2EtWvXAnD33XcTExNz1dHKgiDwzfrTpsIzz97T\nATubplts57+IrCIOwVB3aU8REZFbm+uuhxAYGEjr1q2rxSE0Vj2E1JxSTpRX9eob5U/nCO+rPoaI\ndTHFIYgrBBGRJk2j1kOoTGPUQ1CUqJm19CAADw9tzeTh11YrQMS61BSHkF9Uxokqld5ERERubawW\nh3C99RBUah3z15iD4Ib1DcfJofYEb/9FmoqPtbQ8MZ7cRm7q8/Qv/qVYqWXBywMI9bv+pIZNZSxu\nBOJYmBHHonGxWhzCtdZDOJuYzbELuRw8m2WqQfzB1J4oS8pQltw+ydOako+1orzsplKlMfW5WGks\nNpSUUoCj7PoC35rSWFgbcSzMiGNh5paPQ7iWegiz/rePL9ecYuVfF0zCYPrYqDpzDYncfKp6GVVO\nKa7WNR1Dc9V7uC4OncsmNbuENbsu8vbSg9VqTN9oLqYr0GirV/QTEbkarBaHcC31EI6eMxewbxno\nZqouJnJrUxGHcCQhhyPns8lTmFdyBcVqNFo9EokEG7nx/WPZlrO4ONpyd/8WCFgm27tWBEGgTKO3\nqNgWfyWff06kc3f/Fvg0q/+l4n+/x5OUWcy7j/aoMzVJiUrL15VqXYCxbOqQ7sENSi7Y2JxNKmDu\nymPY2cj4+kVztt2L6QpOXcxjaI8QpBIJcrm0UcZa5L/LdddDiIyMpHPnzsTHx+Pu7s5nn30GQGpq\nKu+88w4tWhhz7Z84caLB9RDuiW5BXK+wq70WkZuErJJKaPHGeIb1CDH9vXrnRX7Yep5QXxcGdwuy\nqBe9aV8SjnZy7urbHFcnW4v6DclZxdjZyvBt5kh2gZJjCTl0jvBCIpFw6lIeG/de4dm7O5jqP+w8\nns4PW8/z9qRuhPi6UFii5pOfjwPGCfylCdWz0iakFFJYoqZ7G18Ons1i/xmj00NimqLOLLDFSk21\nbat3XmT1zot8Mb0f9rYy5DLpVSVaBGNtj9/3XsHVyZbMPCWDugXjU15rQqPVs2lfEoN7hXElpQCd\nQSCqhSe5ChUnLxqN92qtnovpCsID3BAEgfmrTlKi0rJhzxXA+FzFdgvGRi7j7JV8CkrU9GrnZ7U6\n2CJNj+uOQ1i1ahXu7u5s27aNzZs388knn5iEQmho6FXVQ2gZ7M7gO4Lo2lp0LW1KyCq9TWt1BjRa\ns/qkomZ0UlaxhTCoQKnWsfLvC4CxNkP7Fp6cSMxl/uqT2NnKGNIt2DShgbGwUEGxMZnee98fJtTX\nBQc7Of+eNNq6Nu65wlNj2vPCgj2mNvFXCnjms3946+Gu+DRzoFilxdXRlo9+PApAZr6Svw6nmvaf\n89NRFjzXn1U7EjmfUohMKiGmSxClZVq83Bz492R6rWMxbf6/9Gzny8AuQcz56Rgersb+anUGvNzs\nee2BLhQrtXyx5iQarZ4QXxf0egM2NjIupRdZ1P8+eTEPf09HTlzMI8jbidSc0nqrAGblKwkPcCMt\np5QSldbiuzW7LrFm1yUeG9GW77eeR63VU6LS4dvMgQ7hnuLqQQSJUIfS9NixYyxYsIAlS5YAsHjx\nYgCmTp1q2ufRRx9l2rRpdOzYEZ1OR9++fdm/fz+pqak8+eSTpsjlhiIaiYw0JYOZQRCYUl4ZDiAy\nxJ1zyYV8OLVnjWVIXRxtTEbnqtwT3YLdJzPIKlBdc3+83e3JKTSqrdqGNSP+ijE77ujylcj3W8/j\n4WpXrcpem9BmXExTVCuP2tQI83PhSqbx3rknugVrdl2qt02/KH8mNTG37qb0jFibxjIqX3c9hMrx\nBnK5HBcXFwoLjfV6U1NTGTNmDM7OzkyfPp2uXWsuFiPStKn6ZnkuuRA7GxnezRyY9Ug3dh1PQyKV\n4OVmz9DuRnXSX4dT2XIgiZfv68ypi3n8tvsyZRp9rZNX5wgvcgrLSM0pYXS/5vx9JJVipZYurbzJ\nVajw83Dkzk6BfPrLcXIKy3C0kxPm78Jz4zqiUut45Zt9/Lb7sul4NZVcHdA5kEFdg/hyjbG8qkQC\n70zuzswlB037yKQSUwBeoJcTD8a2ws3ZDhdHG46cz2HZlnM19v+e6Bb8cSCZ0jLzCmBE71B+35tk\nsV/HcE9TIGZzf1cuZxRVO9aQ7sHEdgth+9FUosI92bQviQGdA1mz6xKpOSUmYQAQ0yWIvh38cbS3\nITNfaYrrsbeV0czFjow8JQB7TmU2OYEg0vhcdxxCbfEGPj4+7Ny5Ezc3N86cOcPTTz/N77//jrNz\n3QVpRL9iM01pLNo290CrM3AhxfgyEOrvgq+PK74+rnTtEFBt//uHt+X+4W0BiIr044G4dny28ijb\nD6dgZyvjvcd7k5ZTQmSYB25OtjhXqhUNENcvnB1HUhk3MMKiYFGLUA+y8pV0bOmFrNL2J++O4vOf\nj1kcY2ivMCaNaItBME70FQbpqNa+7DqaSr9OgQR4O/PL+8P5YfNZOrT0ondUACVKDQXFaoJ9LX+f\nsGAP7h7Yine+3c+Rc9l8+8ZgMnNLaeZqR4ifK4+M6sCx89nkKVT0bO+Ps6MtQX5unL6Yy5S72uPp\nZrQX/PpXAh6udgzsFoJaoyc1u4RjCdnkF5Wx62gqDw5vh7uLHa1aeAHQu7MxpXx0txAemb2NolKz\njSMkyJz3K8DfjZ/fG87Pf56nb8cAfD2cOHAmkwWrjmMQBDw8nS3Uf02BpvSMNAXqVBkdP36cL7/8\n0qQyWrRoERKJpJrK6Nlnn6VTp04WKqOqTJw4kddee4127drV2SFxCWikqS2HKwyoWQVKth5MIbpj\nAKF+V/ew6vQGchVl1dyMG2ssPv7pKOeSC3n1/s60DmmaCRI9PZ3rrMehLNNy8lIexxJyiekS2KDr\n/GL1SY4n5rLguf44XmWN7ZtJU3tGrMkNURlVjkPw8fFh8+bNzJs3z2KfmJgY1q1bR6dOndi6dSu9\nevUCjDUQ3NzckMlkpKSkkJSUVG9xHJGmS8XK0beZIw8NaX1Nx5DLpFaNOZk+tiMFJeomHddSX6U+\nR3sberb1o2dbvzr3q4y9nTFZZJlGd8MFgsEgWL3SoUjDue44hHHjxvHyyy8TGxuLu7u7SWAcPnyY\nL774AplMhkwmY/bs2Ved6VREpDGxs5U1aWFgLezLU4+o1DpKVFq0OgPNXOxq3b9YqcHRXl5rzIXe\nYEAmlaI3GDh1KR8/D0ey8pU4OdjgaCfH3dkOR3s5CSmFfPTjUZ4f35FWwe5IJaDTCxaxJGBcfSak\nFNLc3xXbejId6/QGtDpDtWMA5BaqkMuluDvXfm1gjLw/fC6bHm19cS5Pl6PTG9h6MBlFqYYx/VrU\nePwKtDoDiamFhPm7cupSHvlFaob2CEGj1SOVSuqsya5S69Dpjf3PKlDh4mCDq5NtrfuXlmmxlTde\n9ufrjkOo2Kfq59jYWC5fvsyaNWuQSqXI5U1nKSrSNBg7diSvvfYWZ8+eIT09jVdffZOMjHTGj7+L\nXbsOVMvAWxdV27300jQGDRrC0KFxbN68kd9/X89XX31rxasx0q9fN37+eR2BgUFWP1cFDrbGCWXm\n0oNUViAHeTvj7mKLRqMnJaeUyBB37Gxk7I/PIsTHmWAfZ9xd7Nh7OtPkCmw6pp0MZwcbk7dXZZwd\nbGjX3IMD8ca4j4rCVwC2cil9ovxRa/QkZ5VQWqY1HdvJXk4zF3tKy7TGc7vacyVdgbrczTnQy4nT\nl/LQ6gz4NHPAy82eYF8XcgtVJKQqTLaV1sHupOWW4upki7ODDXq9gXbNPbijtQ9rd100GfV//DOB\nScMi0RsEvt963tTHM5fz6dLKG4lEwqGzWcT1CqO5vwufrzpBXg3OCgA7j6WRV1SGg52ce2NaklOo\nIjNfSWGJBldHG9qGeXDoXDZnkyzrxcukEnq18+NKZhGpOaW0DHKjqFRD9zY+nEjMIyW7BB93B5a8\nFVv/D90ArBaHkJiYyObNm9m0aRNZWVlMmjSJrVu3XtVDKiJSFxUvKRMnTmr0Y3/yyReNfsxblQAv\nJwCqWhNTc0pIzTH/feyCOXttcnYJydm12zJUaj0qdc2pNEpVWpMwqIpGZ2DH0bSa25XpKC0znrOq\nAAJjDEblfbMKCjhzpaDafudTjF5w6eU1VgAuphdZxLtU8F0NXmMZeUo27TN7hy3dbBlf4+xgUy0G\nJLtQhb2tDGWZjiWbqsfjHD6fU20bGFPK7z5lzieXmKoAsPBOyy68dhftqtQpECrXQwBM9RAqC4Tt\n27ebVgyxsbHMnj0bgL///pu4uDhsbGwICgoiJCSEkydP0qlTp0brvIiIyPXTp4PRtTwlu4SukT40\nc7bD1kbKsQu5BHg6odLoaBPajOwCFVkFSopKNdjbyrmSWYSNXEavdsYIc0WJhryiMloEuJJfpOZS\nuoKzSQV4uNozuGuwycmgSKkhT1GGRqvH0d6GtJwS2jb3wMFWjk5vYM+pDIqVWtqGNcPWRkZYebuc\nQhVSqYSUrBJsbWU4O9vjZCMhMVVBsUrLsYQcerbzo3sbH+xt5WTklbL7ZAbBPs50jfRBrdWTkavE\n0V6Ov6cjmflKbOUyyjQ6vttyjkvpRTT3d+WV+zqDBF5auIfSMh2D7ggiItgdZ3s5kaHNOHUpz6Ra\nU5RoyC8uo6hUS4ivM3G9QrGRyzAYBH7dkYheLzCmfwsMgoC9rYzMfCUnEnPRGwS6tPLGw8WOjHwl\nF9OKkMsk9O8YQLFSS3aBEm93B3IVZWj1Bvw9HNlzOhMbmRSZTEJBsZpWQe60CWuGTR0qqKvFKnEI\nBQUFZGdn07FjR9N+11IPQUSkPgRBYMmSRaSnp/LWW+9W+37nzr9ZuHA+H3/8OWFhzVmxYjm///4b\nJSXF3HFHd156aUaNtq1nnpnK0KHDGTFitGnbwoXz+f339Tg7u/Dii6/Ss2dvAHJzc5g79wNOnTqJ\nq6srDzzwMCNHGttpNBq+/vpLduz4C4CYmEE8+eQ0bGyMuumffvqeX375CalUyqOPPt7o49NQKoRC\nZfp3tHQXDvByMq0mAItUIwD+nk4Wn9s192Bkn+bVjuvqaItrJTfiYB+zK7qNXMqgrjU7n1Tko/Iq\nd8+t8DKq+HtwlXb+nk6MG9DS9LdcJqVlkJvF9xW8+ZAxRqpyupEvn+uPVmcw5eCqICrcq8b+VUYq\nlTBhYES17UHezgR5W7rehwe4ER5g7lczFzuTDady7fiRvcPqPe/1ctWipaFxCDVtr09dNGPGDEaO\njOWhh+6ttx/Hjx9l8uQHiI7uwc6d5ipumZkZPProRCZNup+JE8fz229r6j2WSNOmplw8giCwadMG\nvvlmAZ9//hXNm7dg1aqV7NnzDwsX/o/167fi4uLCvHlz6jim+bjx8acJDQ1l8+a/eeCBiXz0kVn4\nvP32G/j6+rF+/R+8++4cFi1ayJEjhwD4/vulnD17hmXLVrJs2UrOnj3D8uVGN+79+/fy888/8vnn\nX7Fy5VoOHz6IyM2l6r1UVRj817nuegi+vr5kZGTg6+uLTqejuLgYd3d3/Pz8yMzMrLNtVe655x4m\nTpzIq6++Wq9fbdu2Lfnkk7ksXboUV1cH0/7u7vasWbMKGxsblEolI0aMYPToOLy9m15+JDHoxkxN\nYyGVSnB3d8TJyQ47Oxu8vV1Qq41vfZs2rWHt2rX89NOP+Pr6lm9bz1tvvUWbNsaEiy+99DwxMTF4\nejqZ2nl7uyCVSrGxkeHiYo+3twsuLvYEBgYyadJEAB58cAKffjoHqVSDRqPh1KkTfPfdEhwdHQkI\n8ODee8eza9efDB0aw99/b2XmzJlERBjfXp97bjozZ85kxoyX2bt3J+PGjaV7d+NK+uWXX+Dvv7fh\n6elc528v3hdmxLFoXK67HkJFHAJgEYcQExPDpk2b0Gg0pjiEqKioOjvTtWvXasv35ORkpkyZwt13\n380DDzzApUvG1Aa11XK2sbExLcfLysoanN9e5L/Fd999x/33328SBgBpaWk888wzdOvWjW7duhEX\nF4dMJiM3t/5Sn15eZjWBg4NRRaFUKsnOzsbNzQ1HR7M7q7+/P9nZxlTuOTk5BAYG1vpd5TKzAQHV\nI7pFRG4kVotDaNmyJcOGDTM9dLNmzbqmNLtvvfUWs2fPJjQ0lBMnTvDOO++wfPnyOttkZmby2GOP\nkZKSwiuvvNIkVwci18eSJUt49NFH8fb2JjbW6JLn7+/Phx9+SOfO1VNhp6amVtvWEHx8fFAoFJSW\nluLkZFxlZGRkmFbDPj4+pKammhwxKlbTAN7e3hYr8PT02rOoiojcCOoNDoiOjiY6OtpiW+U4BFtb\nW+bPn19j2yeeeIInnnjimjtXWlrK8ePHmT59ummbVltzlszK+Pn5sXHjRrKzs3n66acZMmQInp6e\n19wPkVubmlaBERERfPvtt0yZMgW5XE5MTAwTJkxg3rx5zJkzh4CAAPLz8zl27FitVQAbgr+/P507\nd2bevHm8+uqrXLp0iTVr1vDJJ58ARs+8r7/+mg4dOgCwcOFCRo0aBcCwYcN4/fXXGT16NAEBASxY\nsOCa+yEi0hjc0tFigiDg4uJSb02F2lYePj4+tGzZksOHDzNkyBBrdFHkFqBy0GTF32As3vTNN9/w\n+OOPY2Njw8MPPwzA5MmTyc7OxtPTk+HDh5sEQm33UdXjV9133rx5zJo1i379+uHq6sq0adNMqtOn\nnnqKkpISCyHw5JNPAtC/f38eeughHn74YaRSKdOnT2fTpk2NMSQiItdEncntGoPly5ezevVqBEFg\n3LhxpoeyNqrWUZgwYQKPPPIIQ4cORRAEzp8/T2RkpGn/1157jQEDBpgm/KysLNzc3LC3t0ehUDB+\n/HgWLFhARER1FzARERERETNWFQgJCQm8+OKLrF69GrlczpQpU3j77bcJDQ2tcf8XXniBgwcPUlhY\niKenJ9OmTaNHjx68/fbb5OTkoNPpiIuL46mnnrKo5Wxra4uPjw8bN25kz549zJkzx+T6OnHiRMaN\nG2etSxQRERH57yBYkS1btghvvPGG6e+FCxcK//vf/2rcd9euXcKQIUOEwYMHC4sWLbJmt2466enp\nwoMPPigMGzZMiIuLE5YvXy4IgiAUFBQIjzzyiBAbGytMmjRJUCgUpjbvvvuuMHjwYGHkyJHCmTNn\nblbXrYZOpxPuuusu4fHHHxcEQRCSk5OFsWPHCrGxscJzzz0naDQaQRAEQa1WC9OnTxcGDx4sjBs3\nTkhNTb2Z3W50FAqF8OyzzwpDhw4Vhg0bJhw/fvy2vS++++47IS4uThgxYoTwwgsvCGq1+ra5L157\n7TWhV69ewogRI0zbruU+WLt2rRAbGyvExsYK69atq/e8VhUIiYmJQmxsrFBQUCAolUph/Pjxwrvv\nvlttP51OJwwaNEhISUkRNBqNMGrUKCExMdGaXbupZGdnC/Hx8YIgCEJJSYkQGxsrJCYmCnPmzBEW\nL14sCIIgLFq0SJg7d64gCIKwc+dOYcqUKYIgCMLx48eFcePG3ZyOW5GlS5cKL7zwgkkgTJs2Tdi0\naZMgCIIwc+ZM4aeffhIEQRBWrFghzJo1SxAEQdi0aZPw3HPP3ZT+WotXXnlFWLVqlSAIgqDVaoWi\noqLb8r7IzMwUYmJiBLVaLQiCIEyfPl1Yu3btbXNfHDp0SDhz5oyFQLja+6CgoEAYOHCgoFAoBIVC\nYfpcF1a3IaxevZqffvoJR0dHwsPDsbe3Z8aMGRb7VNRu/vbbb6/JNVVERETkdkOj1fPlr8fJVajo\nF1bE8OHDARg6dCg//PADBw4c4NChQ7zzzjsAzJw5kx49ehAXF1frMa3uZVRSUoJGo0Gr1XL48GHu\nu+++avtU5EySSCRiBaRyxGpQZsSxMCOOhZnbfSw27r3CzqPG+JlxPc15nCryxlXOM1d5e11YVSBk\nZWWxbNkytm3bRm5uLiNGjEAma7xiDiIiIiK3I2qtnj8OJOPsYMNHj/ciN/uSxfe15ZOrTwNj9RVC\nXl4eo0aNwtbWloiIiBo9jKrmTBIRERERqZ0Tibmo1DrieoXiaC+vljfO19cXPz8/Dhw4YLG9Z8+e\ndR7Xqqn8fH19eemll8jKyiInJ4eQkBB69+5dbb+KnEkiIiIiIvVTUWCoRxtjGpSK4N3jx4/j6uqK\nl5cXffr0Yc+ePRQVFaFQKNizZw99+/at87hWXSEoFAq2b9/O9u3bcXFxYfr06WzYsMEUtWnqRHnO\nJBERERGRulGWaTl1KY9ALyeCymtJBAcHM3jwYBwcHPjwww8BcHd356mnnmLs2LEAPPPMM/XWtbeq\nl9GWLVvYvXs377//PmCUYidOnGDWrFnWOqWIiIjIf5o/DyTxxa/HeXBYJPcOat2ox7a6DWHjxo2c\nOnUKiURCYmJivYnEbmevgcrc7h4UlRHHwow4FmZu17HYtv8KAFGhzUzX31h1IaxqQxg2bBiPPfYY\nWq0WvV6PTCbj5ZdftuYpRURERP6z5CnKOJdcSKsgN7zcHRr9+FavD/fss8+yZcsWXnvtNdq1a0dw\ncM31UkVERERE6ubAWaMxuVd7v3r2vDZuWMHQTZs2MWLEiBt1OhEREZH/HAfis5BJJXSNrLsc8bVy\nQ+ohaDQaduzY0SB1kVgj1Yw4FmbEsTAjjoWZ22ksTlzIISW7hJ7t/QgL9rDKOawuEIqKipg6dSpl\nZWU8+OCDfPDBB3Tq1KnW/W9HI1FN3K4Gs5oQx8KMOBZmboex0OoMgIBcJuW7DacBiO0aVO26G0sw\nWl0gvP/+++h0OmbNmsXIkSNRqVTWPqWIiIhIk+dyRhGf/XoCiQTaN/fgYnoRXVp5E+ZXdyzB9WBV\ngVBSUsLBgwdRKpUMHjwYuVyOi8vts8QTERERqY3krGL+PJRCjqIMvd6An4cj9w9uhYOdnH9OpPP9\nH+cxlIeJ7TuThZO9nLF3hlu1T1YVCCkpKXh5edGyZUsmTpxIu3bteOONN3BwaHx3KREREZGmQGGJ\nmh+3JXAkIcdi+8X0Ik5fzsfZwYa03FKc7OU8PDSSiGB3kjKLaRHgirODjVX7ZtVI5VOnTjFhwgRW\nrlxJVFQU77//Ps7OzkyfPt1apxQRERG5ZTl6LptPfjxMsVJLoLczD8e1pU2YBy6ONny28hi7jqUi\nkUConytPj+1IZJh1jMe1YVWBkJOTQ3R0NOHh4chkMtRqNSEhISxatKiONv9tI1FDuR0oW2K3AAAU\nA0lEQVQMZg1FHAsz4liYaWpjcSwhhwVrTyGTSbk3piUDugQirZKOukyjQyqRYGtzdWUCmoRR2dvb\nG5lMxvvvv09UVBRffvklZWVl1jyliIiIyC2HWqNnxZ8JyGQSXr2/M+GBbjXuZ297QyIBasXqZ3d3\nd2fmzJkYDAZCQkJMmfhEREREbhfW77lMQbGauF6htQqDWwGrCwRbW1ukUilSqZT+/fuLXkYiIiK3\nFfvOZPLHgWSaudgxvGf1AmG3ElYXCD///DPe3t7k5+czadIkWrRoQdeuXa19WhEREZGbTmp2Ccu3\nnMPeVsZLEzrhYHdzVUL1YVWjclUWLFiAo6MjkydPvlGnFBEREbkplKl1vDB/FylZJbz+SDd6dQi4\n2V2qF6uKK5VKhV6vx8HBgTFjxpCRkcFnn31WZ5um5DVgTZqaB4U1EcfCjDgWZm71sdiw+zIpWSUM\nuiOIln7W7WuT8DLKzc3lmWeeIT8/H5VKhbe3d701PUVERESaOrmFKrYcTMbJXs7d0S1udncajFXT\nXwcHB7No0SLCw8NZsGABoaG3tkFFRERE5HrRGwws3hiPWqNnwsCIm+5KejVYvR7CBx98wCuvvIJU\nesNKL4iIiIg0KmUaHQZDw8yt24+mkZimoFukD72tVMjGWlhVdO3YsQNPT0/atm3LgQMHGtTmdspv\nXh/iWJgRx8KMOBZmrDUWJSotJxJyKCgu4+j5bA7FZ9E6tBnvPt67Tk8hnd7AjmNp2MqlTL+vC27O\ndlbpn7WwqpfRxx9/zPfffw+AwWBAIpEQFxfHxx9/XGubW9lIdCO51Q1mNxJxLMyIY2HGWmOh1emZ\nufQQWflK0zaJBAQB/D0dmTy8TY3BZYIgsPyP8/xzIp07Owfy0JDWjd632mgSRuVXXnmFZ599FgcH\nB/bt28e0adN44IEHrHlKERGRa0RvMHDoXDYFRWpCfF1o1/zGJla7Vdh2KIWsfCXtW3jQt4M/AZ5O\n+Hs5suT3s+yPz+KjH48yYWAEPdr64mQvRyKRIAgCa3Zd4p8T6YT4OjN+gHXTVFsLq1s7KlJd63Q6\nBEFAUiWZk4iIyM3n1KU8ftmeSHpuqWnbqD5h3NW3+W31zBYUq/l9bxLODjY8MaodjvbmdNNTR7Wj\nX5Q/X68/w49/JvDjnwk42skREHCwk5NfpMa3mQMvjO/UpAzJlbF6rw0GA2PGjCE5OZkHH3yQqKgo\na59SRETkKli1I5EtB5KRSKBPBz8igtzZtO8KG/ZcoVSl475BEUilt7ZQUGv1JKQU4u5sh7+nI2qt\nnsISDW5OtjjYyZA1wKklNaeExRviUWv13BvT0kIYVNAmzIO3Hu7Kxj1XKFFpycxXotUZKFJqaN/C\ng0nD2uDqZGuNS7whWF0gSKVS1q9fT3FxMU8//TQXLlwgIiLC2qcVERFpAP+cSGfLgWT8PBx54q52\nhPgaddEdWngy79fj/H00lcR0BWOjw29ZFdK5pAKWbj5LrqLmTMq2cimdIry4b2BErUbe+Cv5LFx3\nCpVaT3SnAPp3rD2q2NvdgclxbSy2/Ve0H2LqChGR25TsfCVPzPkbOxsZnz0fjZ+nk8X3pSotX685\nya5jqQB0ifTh7uiWtA5thv0tkJNHpdbx7frTbDuQhFQCg3uEotbqyVeUYWcrw8PVnjxFGRm5JaTl\nlOLhasdd/cOxkcsI9XehuFTLycQczl7J53J6EXKZlOcmdCa6S9DNvrSbhlUFwtmzZ5k9ezYKhQIA\njUbDW2+9RXR0dK1tRA8KI6I3iRlxLMw05lgs3XSW3acymDQ8kn5Rtb8RJ2UW8+uORM4mFZi2BXk7\n07OdL15u9tjKZVzOKEJvEP7f3t3HRlXuCRz/TjvTdvoyfZlOZwq1iK3WttyCK1eBIhEKbcWCsAGj\niSSirggxFcuKYfE9BGJw2ai7GyFZYrzGu7teLa6BlSxUClQUQShIC1guL22Zmba089JO5/3ZPwam\nvLXcYqfTMs8nIcycc2bOc57+5vzOc55znsM4QxKT83T0uny0WRx4vQKX10dGihpdytA9OveC2c4X\nu3/j9AULY9ITWDr3fnLG3HxYaSEEOw8285c9Z4LPKL6aMjqK3LEanpg+nrzs1CEr43AaFVcZWa1W\nLBYLKpUKn8+H3W4nKytys68kjQTdvR721l9k/3Ej2RmJTC0c+OapcYYk/vGpSZy6YOFoUwcXzHZ+\na7Hylz1nbrr8lUs0rzcpN52/u0/HH/MziB3kE8EgsGM/2tRB7dGLHD9zCQFMLTSwdO79KKP77yNQ\nKBSUP5zNhPFpNLd1A9Da0YM6Npq8u1K5OzNpwM9HkmE9ZbRixQqWLFnC1KlT+11GHgkGyKPiPrIu\n+vzeujh5vot/qz5Oj9NLbEw0a5c8SJYucdDfY3e4OXGuE4fTS6/LS6Y2AXVMNLX1F2m39JKojiEj\nRY1CATGqaE5d6OLMRRsAiWoV992VwqOTxlA4Pu1vOvfe2t7N1h0nOWsMfMfdhiSemz+BrLSha3WM\nZqOihXC1lpYWGhsb5VVGkhQme+sv8qedpwCYMTGTOZPvYuxtJAOApPgYphTc2LLI7+eh8EIIzpvt\nHDrZTu3RVn453c4vp9tJiFNytyGJnLHJTM7LICvj2vL4hWD/MSNf/N9p3F4/D+bpeKJ4PFkZifJA\nIQSGpYXQ09PDkiVLWLFiBbNnzw716iRJus6f/reR/951mqR4Ff/07ENMyEkPW1mEEJxpsbLjh7P8\neuYSxkt99z4U5aaTm5WCrcdNW5eDljY7nTYXCXFKXnnqgVHxTIHRLOQJ4fXXX2f79u2kpKSwf//+\nWy4vM36APPrpE0l10WlzcuaijXsyNWiT426Yfzt1UXfcyH9sb0SfqmblkxPRp8YPVXGHRHevh5Pn\nu9h1qJnTLdZr5mkSYpiUq6Vi2t2kJ197eiiS4uJWRsUpIyEEHR0dlJeXc+rUqVCuSpJGBI/Xx3/u\nbqLN0svjU8Zh0MajiY/p98Yuh9PD4dPttFuc2Hpc7D9mwi8EKmUU/1BRQF52Cknxt3+j03mTnc92\nnkIdqxyRyQACfQqT789g8v0Z2B1ujJccaBJi0GriUCllZ+9wCmlCOHz4MHV1ddxzzz20tLSwYMEC\nqqqqmDFjRihXK0lh8+ddv7Hn6EUATpztBEAdG80TxeOZ88e7rulAPXGuk63bG+myu675jqmFen5q\naOPft/1KlELBjEljyBmjIT05jiZTNx2d3TyUryfqFp2xdoebf/36OF6vnxULJozIZHC9pPiY35UA\npd9nWPoQWlpaWL58Od9+++0tl5VNwADZHO4zWuriyqmZLF0iCx8Zz/7jRgBON1vocXrJH5eKPi2e\n+qYOXG4fDpeXKIWCx6Zkk3dXSnD8m9ysZJparBw+3cbhU+03vQO3eIKBpY/n95sUzF0OPvnmBOdN\ndhZMH8/86eNDt+FhMlriYjiMilNGt0OO9d5H1kWfkVYXPb0ezhlt9PR60CTEYHO4+WznKRLUKt58\n4WHGpCdSWhx4dGKXzcm//PkXjpxup/F8F0nxMWSkxaNNjuOZx/LJzUq54ft1uiSmPpCFx+vnpxNG\n7A4Pf221EqOM4ucGM3W/mlDFKHluXiGpmr6+Bmu3i//Z91e+/r4Jr8/PnIeyWfrEH0b8WES3a6TF\nxWgX8hbC3r17effddzGbzVRWVvLiiy8OuLzM+AHy6KfPSKoLS7eLr2rPcLCxDY/Xf808BfDK4okU\n5Whv+JwQgtaOHnpdXsZnam77RiidLolzzZ1s+q/64DX5iWoVBm08Pp/ggtmOzy9ITYrlqZJ7mZyn\nuyPG2LmZkRQX4TZUiTGkCcHn81FeXs6GDRt45513iI6OZtOmTeTk9D9WuPwDB8hg7xOqurA53Jgu\nObhkc2KxuzBo45kwXntDR6bxUg/Nbd2cM9qprW+l1+VDn6rmgft0JMQFhj12e3xM+0Mm+eNCO/TB\nlbpweXzsOdLKqQsWjJ0O2rt6USgCQ0pMm2DgkYmZo3YI5r+V/I30GRUJ4ciRI7z88ssoFAosFgux\nsbEUFxfz0Ucf9fsZ+QcOiORgd3t82Hrc2Bwe/EKgTUug2+4kLiaauJhohAiMPxMfd+sdns/vx3TJ\nwQVzN83t3XRYnXQ73LRZeum0uW5YPkYVhSY+BpUyCr9f4Pb6r+n0TVSr+PsZ9zBj0phbduqGQn9x\n4fUFWiuRNARDJP9Grjcq+hDMZjMzZ85k3bp1AHzzzTccO3as3+Wt3S7sDjcAN2QpMeDbGwZPuX7+\nQGnvVjnx+tligMLcsN5bfFl/2+lBQWen45r5gy/n7a37ZvNutm6fXwT++fz4L7/2+gU+n8AvBEII\nlNFRREcpcHv9OF1enB4fTrcPp9sb+N8VeN3T68Hq8GDrcdHr8g24nVeoY5UkxClJiFMRH6ckRhmF\nXwTubvX7BU63l9b2HtzXndoBSE6MoShHy1hdAunJajTxMTS1Wmg410V3rwenw0N0lAJltIJJuenk\nj0slI1VNwd2pqJSDH4cn1CIpEUihM+xtyoHOZz7z9nfDWBJpJFEASfEqtBo1yQkqNAmxaBJUREUp\niI1VYbM7LycSHwpF4IEoFruLHqcXY2cPbs+NO/3oKAVj0xPINiSRnZFItj4JfaqaBLXqpjvQB/N0\nw7ClkjRyhTQhGAwGjEZj8L3JZCIjI6Pf5b/95ydCWRxJuiPIK2v6yLoYWiFtZ06YMIHz58/T0tKC\n2+1mx44dlJSUhHKVkiRJ0m0KaQtBqVTy5ptv8vzzz+P3+1m0aNGAVxhJkiRJ4TOsz0OQJEmSRi55\naYIkSZIEyIQgSZIkXSYTgiRJkgSMoISwd+9eysvLKS0tZcuWLeEuTkgZjUaWLFnC3Llzqaio4LPP\nPgPAYrGwdOlSysrKeO6557DZbMHPrFu3jtLSUubPn09DQ0O4ih4yPp+PBQsW8NJLLwHQ3NzM4sWL\nKSsr49VXX8Xj8QDgdrtZuXIlpaWlPPnkk7S2toaz2EPOZrNRWVnJY489xty5c6mvr4/YuPj000+p\nqKhg3rx5rFq1CrfbHTFxsWbNGqZNm8a8efOC024nDqqrqykrK6OsrIxt27bdesViBPB6vWL27Nmi\nublZuN1uMX/+fNHU1BTuYoVMW1ubaGhoEEII0d3dLUpLS0VTU5N4//33xZYtW4QQQmzevFls3LhR\nCCHEnj17xAsvvCCEEOLo0aNi8eLF4Sl4CG3dulVUVVWJZcuWCSGEqKysFNu3bxdCCPHWW2+JL774\nQgghxOeffy7efvttIYQQ27dvFytXrgxLeUNl9erV4ssvvxRCCOHxeITNZovIuDCZTGLWrFnC5XIJ\nIYR45ZVXxNdffx0xcfHzzz+LEydOiIqKiuC0wcZBV1eXKCkpEVarVVit1uDrgYyIFsKxY8fIzs4m\nKysLlUrF448/zu7du8NdrJDR6XTk5+cDkJCQQE5ODmazmZqaGhYuXAjAwoUL2bVrFwC7d+8OTp84\ncSI2m42Ojo7wFD4ETCYTtbW1LF68ODjtp59+ory8HLi2Lq6uo9LSUg4cODD8BQ6R7u5uDh06xKJF\ni4DAZdtJSUkRGxc+n4/e3l68Xi9OpxOdThcxcTF58mQ0Gs010wYTB+3t7ezfv5/i4mI0Gg0ajYZp\n06axb9++Adc7IhKC2WwmMzMz+F6v12M2m8NYouHT0tJCY2MjRUVFXLp0ifT0wMPPdTodnZ2BJ261\ntbVhMBiCnzEYDJhMprCUNxTWr1/P6tWriYoKhGNXVxcajSb4Xq/X09bWBlxbF1d2mBaLJTwFH2LN\nzc2kpaWxZs0aFi5cyBtvvIHD4YjIuNDr9SxdupRHH32URx55hKSkJAoLCyMyLq4YTByYzeZ+pw9k\nRCSEm7lTx3C/Wk9PD5WVlaxdu5bExMR+lxOXB4q72pUfxWj3/fffo9VqKSgoCG7j9dt6tZvNu1Ni\nxev10tDQwNNPP011dTVqtXrA/rQ7OS6sVis1NTXU1NSwb98+HA4HtbW1/S5/J8fFrdwsDhQKxW3V\nyYiInsGOeXQn8Hg8VFZWMn/+fGbPng2AVqulvb0dCGT9tLQ0IHAkdPWR351UP0eOHKGmpoZZs2ax\natUqfvzxR9avX4/NZsPvDwxYZzKZ0Ov1QKAursSK1+vFbreTnJwctvIPJYPBgF6vp6ioCICysjIa\nGhpIT0+PuLj44YcfyMrKIjU1FaVSyZw5czhy5EhExsUVg9k/6PX6m+5Xr9RXf0ZEQoi0MY+EEKxd\nu5acnByeffbZ4PRZs2ZRXV0NwLZt24KJoqSkJHiFwNGjR9FoNMGm42hXVVVFbW0tNTU1bNq0iSlT\npvDBBx/w8MMP8913gdFvq6urg/FwdR3t3LmTqVOnhq3sQ02n05GZmcnZs2cBOHDgALm5ucycOTPi\n4mLs2LHU19fjdDoRQnDgwAHuvffeiIyLKwa7fyguLqaurg6bzYbVaqWuro7p06cPuI4RM3RFbW0t\n69evD455tGzZsnAXKWQOHTrEM888Q15eXrAJV1VVRVFREStXrsRoNDJmzBg+/PDDYMfSe++9x759\n+1Cr1WzYsIHCwsJwbkJIHDx4kK1bt/LJJ5/Q3NxMVVUVVquVgoICNm7ciEqlwu1289prr9HY2EhK\nSgqbNm0iKysr3EUfMidPnmTt2rV4PB6ys7PZsGEDPp8vIuPi448/ZseOHSiVSgoKCli3bh0mkyki\n4qKqqoqDBw9isVjQarVUVlZSUlIy6Dj46quv2Lx5MwDLly8Pdj73Z8QkBEmSJCm8RsQpI0mSJCn8\nZEKQJEmSAJkQJEmSpMtkQpAkSZIAmRAkSZKky2RCkCRJkgCZECRJkqTLZEKQJEmSAPh/cabPhM8u\nKAgAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x979e4d0>"
            ]
          }
        }
      ],
      "execution_count": 0
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mCQCiqgcq6RC",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            },
            {
              "item_id": 2
            }
          ]
        },
        "cellView": "both",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1457683148314,
          "user_tz": 480,
          "elapsed": 2245,
          "user": {
            "sessionId": "ab1f0339c96f28e",
            "userId": "103444865736961301998",
            "permissionId": "00759584880725894106",
            "displayName": "Patrick Mineault",
            "color": "#1FA15D",
            "isMe": true,
            "isAnonymous": false,
            "photoUrl": "//lh3.googleusercontent.com/-CdPTZM1TPbU/AAAAAAAAAAI/AAAAAAAAABQ/MOmbIGKtlQA/s50-c-k-no/photo.jpg"
          }
        },
        "outputId": "78717327-bc04-4597-c2a7-27b4ef3812e4"
      },
      "source": [
        "samples = []\n",
        "sampler = pcd.get_new_sampler()\n",
        "    \n",
        "for i in range(10000):\n",
        "  sampler.sample()\n",
        "  samples.append(sampler.last_sample)\n",
        "\n",
        "print \"Some samples:\"\n",
        "samples[::100]"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Some samples:\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "execution_count": 191,
          "metadata": {},
          "data": {
            "text/plain": [
              "['gotrobt',\n",
              " 'ntrrirs',\n",
              " 'aldarac',\n",
              " 'rntHcbr',\n",
              " 'sumenCs',\n",
              " 'umorego',\n",
              " 'elihgah',\n",
              " 'catccin',\n",
              " 'rirucab',\n",
              " 'gnilvlm',\n",
              " 'hodeehb',\n",
              " 'osupiag',\n",
              " 'tedralh',\n",
              " 'rorwemr',\n",
              " 'Mhmnsru',\n",
              " 'ricdpin',\n",
              " 'lereeno',\n",
              " 'N,mbndo',\n",
              " 'rNanivu',\n",
              " 'nitpitn',\n",
              " 'tarmNau',\n",
              " 'ensgfen',\n",
              " 'ramsake',\n",
              " 'Stpanri',\n",
              " 'rrsenor',\n",
              " 'cntaett',\n",
              " 'Abenecd',\n",
              " 'papiloc',\n",
              " 'fuiyeob',\n",
              " 'pfrmnet',\n",
              " 'tresehi',\n",
              " 'ealutsr',\n",
              " 'naeqsnn',\n",
              " 'eeemcnn',\n",
              " 'titrole',\n",
              " 'Sedulam',\n",
              " 'irorige',\n",
              " 'lccsapi',\n",
              " 'fbisere',\n",
              " 'dlnlexa',\n",
              " 'gedwena',\n",
              " 'holttad',\n",
              " 'scoinos',\n",
              " 'stnyepl',\n",
              " '(anisdu',\n",
              " 'bbneye(',\n",
              " 'uohsgcn',\n",
              " 'mineftl',\n",
              " 'yaintso',\n",
              " 'SenatAt',\n",
              " 'hlatunl',\n",
              " 'ylica:i',\n",
              " ')diliyi',\n",
              " 'ricaene',\n",
              " 'egeceme',\n",
              " 'ceciesa',\n",
              " 'nudrtch',\n",
              " 'lchinhl',\n",
              " 'lradiar',\n",
              " 'nelsmef',\n",
              " '[legate',\n",
              " 'yinoroe',\n",
              " '0ieceti',\n",
              " 'nexatis',\n",
              " 'damita,',\n",
              " 'legnmae',\n",
              " 'nlihhrb',\n",
              " 'mucrnin',\n",
              " 'cmnpena',\n",
              " 'wetdebe',\n",
              " 'celcekm',\n",
              " 'dertime',\n",
              " 'pirlhif',\n",
              " 'cilhini',\n",
              " 'ghtrcir',\n",
              " 'Helrons',\n",
              " 'gritmda',\n",
              " 'thdemer',\n",
              " 'busathi',\n",
              " 'epemcei',\n",
              " 'ierokrs',\n",
              " 'Ghmipac',\n",
              " 'sairoau',\n",
              " 'xfe\"dib',\n",
              " 'nibedei',\n",
              " 'pirsgia',\n",
              " 'toudtey',\n",
              " 'r,cpenn',\n",
              " '-geltou',\n",
              " 'Svqlour',\n",
              " 'nlpruun',\n",
              " 'Pirecec',\n",
              " 'tibenam',\n",
              " 'eshohoi',\n",
              " 'teeripc',\n",
              " 'tnesitf',\n",
              " 'lutedar',\n",
              " 'lesuflt',\n",
              " 'flye?te',\n",
              " 'gceyese']"
            ]
          }
        }
      ],
      "execution_count": 0
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lb4NjJ7Er5Yq",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            }
          ]
        },
        "cellView": "both",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1457683466941,
          "user_tz": 480,
          "elapsed": 628,
          "user": {
            "sessionId": "ab1f0339c96f28e",
            "userId": "103444865736961301998",
            "permissionId": "00759584880725894106",
            "displayName": "Patrick Mineault",
            "color": "#1FA15D",
            "isMe": true,
            "isAnonymous": false,
            "photoUrl": "//lh3.googleusercontent.com/-CdPTZM1TPbU/AAAAAAAAAAI/AAAAAAAAABQ/MOmbIGKtlQA/s50-c-k-no/photo.jpg"
          }
        },
        "outputId": "5d45a1ae-a58e-446b-ae3c-c9681c7cf310"
      },
      "source": [
        "print 'Most likely word'\n",
        "print max([(sampler.log_density_fun(x),x) for x in words])\n",
        "print 'log-likelihod of machine versus machina:'\n",
        "print (sampler.log_density_fun('machine'),\n",
        "       sampler.log_density_fun('machina'))\n",
        "print 'Number of machine samples out of 10000:'\n",
        "print(sum(x == 'machine' for x in samples))\n",
        "print \"Weight for word 'machine'\"\n",
        "print pcd.theta[-1]"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Most likely word\n",
            "(34.704358830698958, 'machine')\n",
            "log-likelihod of machine versus machina:\n",
            "(34.704358830698958, 19.98661878108004)\n",
            "Number of machine samples out of 10000:\n",
            "0\n",
            "Weight for word 'machine'\n",
            "14.1110096977\n"
          ],
          "name": "stdout"
        }
      ],
      "execution_count": 0
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "343ONLiisUZl",
        "colab_type": "text"
      },
      "source": [
        "What's interesting is that the model has learned that machine is by far the most likely word -- it's given a huge weight of 14 to the corresponding feature -- but the sampler didn't find it once in 10k samples! If the chain ever stumbled onto the word machine, it would have a 1 in exp(14) chance of getting unstuck! The weight for the word machine is much too large, and this is because the chains don't mix well enough to find such a global feature. We have to be careful about blindly implementing global features then!"
      ]
    }
  ]
}