{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "# %cd .. \n",
    "import sys\n",
    "sys.path.append(\"..\")\n",
    "import statnlpbook.util as util\n",
    "import matplotlib\n",
    "matplotlib.rcParams['figure.figsize'] = (10.0, 6.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!---\n",
    "Latex Macros\n",
    "-->\n",
    "$$\n",
    "\\newcommand{\\bar}{\\,|\\,}\n",
    "\\newcommand{\\Xs}{\\mathcal{X}}\n",
    "\\newcommand{\\Ys}{\\mathcal{Y}}\n",
    "\\newcommand{\\y}{\\mathbf{y}}\n",
    "\\newcommand{\\weights}{\\mathbf{w}}\n",
    "\\newcommand{\\balpha}{\\boldsymbol{\\alpha}}\n",
    "\\newcommand{\\bbeta}{\\boldsymbol{\\beta}}\n",
    "\\newcommand{\\aligns}{\\mathbf{a}}\n",
    "\\newcommand{\\align}{a}\n",
    "\\newcommand{\\source}{\\mathbf{s}}\n",
    "\\newcommand{\\target}{\\mathbf{t}}\n",
    "\\newcommand{\\ssource}{s}\n",
    "\\newcommand{\\starget}{t}\n",
    "\\newcommand{\\repr}{\\mathbf{f}}\n",
    "\\newcommand{\\repry}{\\mathbf{g}}\n",
    "\\newcommand{\\x}{\\mathbf{x}}\n",
    "\\newcommand{\\prob}{p}\n",
    "\\newcommand{\\vocab}{V}\n",
    "\\newcommand{\\params}{\\boldsymbol{\\theta}}\n",
    "\\newcommand{\\param}{\\theta}\n",
    "\\DeclareMathOperator{\\perplexity}{PP}\n",
    "\\DeclareMathOperator{\\argmax}{argmax}\n",
    "\\DeclareMathOperator{\\argmin}{argmin}\n",
    "\\newcommand{\\train}{\\mathcal{D}}\n",
    "\\newcommand{\\counts}[2]{\\#_{#1}(#2) }\n",
    "\\newcommand{\\length}[1]{\\text{length}(#1) }\n",
    "\\newcommand{\\indi}{\\mathbb{I}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Classification \n",
    "\n",
    "In many applications we need to automatically classify some input text with respect to a set of classes or labels. For example,\n",
    "\n",
    "* for information retrieval it is useful to classify documents into a set of topics, such as \"sport\" or \"business\",\n",
    "* for sentiment analysis we classify tweets into being \"positive\" or \"negative\" and \n",
    "* for spam filters we need to distinguish between Ham and Spam.\n",
    "\n",
    "<!-- TODO: Load Web Corpus, 4 Universities, something were Maxent works -->\n",
    "\n",
    "## Text Classification as Structured Prediction\n",
    "We can formalize text classification as the simplest instance of [structured prediction](/template/statnlpbook/02_methods/00_structuredprediction) where the input space $\\Xs$ are sequences of words, and the output space $\\Ys$ is a set of labels such as $\\Ys=\\{ \\text{sports},\\text{business}\\}$ in document classification or $\\Ys=\\{ \\text{positive},\\text{negative}, \\text{neutral}\\} $ in sentiment prediction. On a high level, our goal is to define a model $s_{\\params}(\\x,y)$ that assigns high *scores* to the label $y$ that fits the text $\\x$, and lower scores otherwise. The model will be parametrized by $\\params$, and these parameters we will learn from some training set $\\train$ of $(\\x,y)$ pairs. When we need to classify a text $\\x$ we have to solve the trivial (if the number of classes is low) maximization problem $\\argmax_{y \\in \\Ys} s_{\\params}(\\x,y)$. \n",
    "\n",
    "<!-- TODO: Show a silly classifier example? --> \n",
    "\n",
    "In the following we will present two typical approaches to text classifiers. The first one is the standard generative model for this problem: Naive Bayes. The second one is a discriminative linear classifier. This chapter hence constitutes a shift from generative to discriminative learning. We will focus particularly on the fact that both models can use the same model structure and parameterisation, and differ only in how model parameters are trained."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sentiment Analysis as Text Classification\n",
    "Before we go on to classification methods let us focus on a specific task: sentiment analysis. We will load data for this task from the [Movie Review dataset](https://www.cs.cornell.edu/people/pabo/movie-review-data/). We will load the data into a very simple format of $(\\x,y)$ Python pairs, where $\\x$ corresponds to the input text document, and $y\\in\\{\\text{pos},\\text{neg}\\}$ to the sentiment label. We notice that the text has already been [tokenized](tokenization.ipynb), and the whitespace is sufficient to determine token boundaries. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['a',\n",
       "  'movie',\n",
       "  \"that's\",\n",
       "  'been',\n",
       "  'as',\n",
       "  'highly',\n",
       "  'built',\n",
       "  'up',\n",
       "  'as',\n",
       "  'the',\n",
       "  'truman',\n",
       "  'show',\n",
       "  ',',\n",
       "  'with',\n",
       "  'reviews',\n",
       "  'boasting',\n",
       "  ',',\n",
       "  '\"',\n",
       "  'the',\n",
       "  'film'],\n",
       " 'pos')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from os import listdir\n",
    "from os.path import isfile, join\n",
    "def load_from_dir(directory,label):\n",
    "    \"\"\"\n",
    "    Load documents from a directory, and give them all the same label `label`.\n",
    "    Params:\n",
    "        directory: the directory to load the documents from.\n",
    "        label: the label to assign to each document.\n",
    "    Returns:\n",
    "        a list of (x,y) pairs where x is a tokenised document (list of words), and y is the label `label`.\n",
    "    \"\"\"\n",
    "    result = []\n",
    "    for file in listdir(directory):\n",
    "        with open(directory + file, 'r') as f:\n",
    "            text = f.read()\n",
    "            tokens = [t.strip() for t in text.split()]\n",
    "            result.append((tokens,label))\n",
    "    return result\n",
    "    \n",
    "data = load_from_dir('../data/rt-2k/txt_sentoken/pos/', 'pos') + \\\n",
    "       load_from_dir('../data/rt-2k/txt_sentoken/neg/', 'neg')\n",
    "data[3][0][:20], data[3][1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we devide the data into a training, development and test set. The training dataset will be used for estimating parameters of our models. The development set will allow us to test different configurations and optimise hyperparameters (or model choices) without overfitting on the test set. This problem of fitting-the-test-set is quite widespread in ML and NLP research. It has led to impressive F1 scores on certain datasets, but poor generalisation of the findings to other slightly different dataset. We will shuffle the data and then allocate 200 instances to both dev and test set, the remainder to the training set. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "815"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import random \n",
    "random.seed(0)\n",
    "shuffled = list(data)\n",
    "random.shuffle(shuffled)\n",
    "train, dev, test = shuffled[:1600], shuffled[1600:1800], shuffled[1800:]\n",
    "len([(x,y) for (x,y) in train if y == 'pos']) # check balance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Naive Bayes\n",
    "One of the most widely used approaches to text classification relies on the so-called Naive Bayes (NB) model. In NB we use a distribution $p^{\\mbox{NB}}_{\\params}$ for $s_\\params$. In particular, we use the *a posteriori* probability of a label $y$ given the input text $\\x$ as a score for that label given the text.   \n",
    "\n",
    "\\begin{equation}\n",
    "  s_{\\params}(\\x,\\y)\\ = p^{\\text{NB}}_{\\params}(y|\\x)\n",
    "\\end{equation}\n",
    "\n",
    "By Bayes Law we get\n",
    "\n",
    "\\begin{equation}\n",
    "    p^{\\text{NB}}_{\\params}(y|\\x) =\n",
    "  \\frac{p^{\\text{NB}}_{\\params}(\\x|y) p^\\text{NB}_{\\params}(y)}{p^{\\text{NB}}_{\\params}(x)}  \n",
    "\\end{equation}\n",
    "\n",
    "and when an input $\\x$ is fixed we can focus on \n",
    "\n",
    "\\begin{equation}\\label{eq:NB}\n",
    "\\prob^{\\text{NB}}_{\\params}(\\x,y)= p^{\\text{NB}}_{\\params}(\\x|y) p^\\text{NB}_{\\params}(y)\n",
    "\\end{equation}\n",
    "\n",
    "because in this case  $p^{\\text{NB}}_{\\params}(x)$ is a constant factor. In the above $p^{\\text{NB}}_{\\params}(\\x|y)$ is the *likelihood*, and $p^\\text{NB}_{\\params}(y) $ is the *prior*.\n",
    "\n",
    "<!--Let us assume that we have a number $K(\\x)$ of feature functions $f_k(\\x)$ that represent the input $\\x$. For example, in document classification this set could be used to represent the text $\\x = (x_1,\\ldots,x_n)$ as a bag of words by setting $f_k(\\x) = x_k$ and $K(\\x) = n$. We could also use bigrams instead, setting $f_k(\\x) = (x_k,x_{k+1})$ and $K(\\x) = n-1$, or any other representation that is effective for distinguishing between classes of text.-->\n",
    "\n",
    "The \"naivity\" of NB stems from a certain conditional independence assumption we make for the likelihood $p^{\\mbox{NB}}_{\\params}(\\x|y)$. Note that conditional independence of two events $a$ and $b$ given a third event $c$ requires that $p(a,b|c) = p(a|c) p(b|c)$. In particular, for the likelihood in NB we have:\n",
    "\n",
    "\\begin{equation}\n",
    "  p^{\\text{NB}}_{\\params}(\\x|y) = \n",
    "  \\prod_i^{\\text{length}(\\x)} p^{\\text{NB}}_{\\params}(x_i|y)\n",
    "\\end{equation}\n",
    "\n",
    "That is, NB makes the assumption that the observed words are independent of each other when *conditioned on the label* $y$. \n",
    "\n",
    "### Parametrization\n",
    "The NB model has the parameters $\\params=(\\balpha,\\bbeta)$ where \n",
    "\n",
    "\\begin{split}\n",
    "  p^{\\text{NB}}_{\\params}(f|y) & = \\alpha_{f,y} \\\\\\\\\n",
    "  p^{\\text{NB}}_{\\params}(y) & = \\beta_{y}.\n",
    "\\end{split}\n",
    "\n",
    "That is, $\\balpha$ captures the per-class feature weights, and $\\bbeta$ the class priors. \n",
    "\n",
    "### Training the Naive Bayes Model\n",
    "\n",
    "The NB model again can be trained using Maximum Likelihood estimation. This amounts to setting \n",
    "\n",
    "\\begin{split}\n",
    "  \\alpha_{x,y} & = \\frac{\\counts{\\train}{x,y}}{\\sum_{x'}\\counts{\\train}{x',y}}\\\\\\\\\n",
    "  \\beta_{y} & = \\frac{\\counts{\\train}{y}}{\\left| \\train \\right|}\n",
    "\\end{split}\n",
    "\n",
    "### Laplace Smoothing\n",
    "Just as it was the case for [language models](language_models.ipynb), the MLE can produce zero probabilities for unseen words in the test set. We can distinguish between two types of events at test: \n",
    "\n",
    "1. We encounter a word that appears in the training set, but never for a given class.\n",
    "2. We encounter a word that does not appear in the training set at all.\n",
    "\n",
    "We address the first problem by using Laplace Smoothing and pseudo-counts. Let $V$ be the full training set vocabulary, and $\\gamma$ a pseudo-count, then\n",
    "\n",
    "\\begin{split}\n",
    "  \\alpha^\\gamma_{x,y} & = \\frac{\\counts{\\train}{x,y} + \\gamma}{|V|\\gamma + \\sum_{x'}\\counts{\\train}{x',y}}.\\\\\\\\\n",
    "\\end{split}\n",
    "\n",
    "Let us implement the MLE training algorithm with Laplace Smoothing for an NB model in Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "\n",
    "def train_nb(data, pseudo_count=0.0):\n",
    "    \"\"\"\n",
    "    Trains a NB model using MLE.\n",
    "    Params:\n",
    "        data: a list of (x,y) pairs where x is a list of words, and y is a label.\n",
    "    Returns:\n",
    "        The pair (alpha,beta) where alpha are the per class word emission probabilities as a dictionary \n",
    "        from pairs of word and label to probability, and beta the class priors as a dictionary from \n",
    "        class name to probability.\n",
    "    \"\"\"\n",
    "    alpha = defaultdict(float)\n",
    "    beta = defaultdict(float)\n",
    "    vocab = set(w for x,_ in data for w in x)\n",
    "    labels = set(y for _,y in data)\n",
    "    norm = 0\n",
    "    for x,y in data:\n",
    "        for w in x:\n",
    "            beta[y] += 1.0\n",
    "            alpha[w,y] += 1\n",
    "            norm += 1\n",
    "    for y in labels:\n",
    "        for w in vocab:\n",
    "            alpha[w,y] = (alpha[w,y]+pseudo_count) / (beta[y] + len(vocab) * pseudo_count)\n",
    "\n",
    "    for y in list(beta.keys()):\n",
    "        beta[y] = beta[y] / norm\n",
    "    return (alpha, beta)\n",
    "    \n",
    "theta = (alpha, beta) = train_nb(train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is useful to inspect the learned parameters of the NB model, in particular because they are easy to interpret. The class prior looks sensible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defaultdict(float, {'neg': 0.46393907123093997, 'pos': 0.53606092876906})"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "beta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us look at the $\\balpha$ parameters that correspond to the probability of generating a word given a class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl0AAAFzCAYAAAAEzLzwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAF5FJREFUeJzt3X+w5Xdd3/HXm10CCLWxskAggU3HFbvVAnGN8QfWXzC7\nibJUBYmWYIpdIskAgi2LFB3aoQ0dayEYs42SYhCNzBTtQnZEq6JjRzTLr0CAdHbSWJJJZfmRAMYS\nAu/+cb7rXC8b7tnszefcc/fxmLmz95zv9+y+v7P33vO83/P9fk91dwAAeGA9aNEDAACcCkQXAMAA\nogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAbYueoDjeeQjH9nbt29f9BgAAGt6z3ve\n84nu3rbWehsyurZv357Dhw8vegwAgDVV1V/Os56XFwEABhBdAAADiC4AgAFEFwDAAKILAGAA0QUA\nMIDoAgAYQHQBAAwgugAABhBdAAADiC4AgAFEFwDAAKILAGAA0QUAMMDWRQ+wKNv3X7/oEdZ06+UX\nLHoEAGCd2NMFADCA6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACn7HW6NhvXHQOAjc2eLgCA\nAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAeaKrqraXVU3V9WRqtp/nOVV\nVVdMy2+sqnNWLLu1qj5YVe+vqsPrOTwAwLJY822AqmpLkiuTPC3JbUluqKqD3f3hFavtSbJj+vjW\nJFdNfx7zPd39iXWbGgBgycyzp+vcJEe6+5buvifJdUn2rlpnb5Jre+bdSU6vqjPWeVYAgKU1T3Q9\nLsnHVty+bbpv3nU6yf+oqvdU1b77OygAwDJb8+XFdfCd3X17VT0qye9X1Ue7+09WrzQF2b4kefzj\nHz9gLACAcebZ03V7krNW3D5zum+udbr72J8fT/Lbmb1c+WW6++ru3tXdu7Zt2zbf9AAAS2Ke6Loh\nyY6qOruqTkvynCQHV61zMMlF01mM5yW5q7vvqKqHV9XfS5KqeniSpyf50DrODwCwFNZ8ebG7762q\ny5K8M8mWJNd0901Vdcm0/ECSQ0nOT3Ikyd1JLp4e/ugkv11Vx/6t3+ju3133rQAA2ODmOqaruw9l\nFlYr7zuw4vNOculxHndLkied5IwAAEvPFekBAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCA6AIA\nGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMILoAAAYQXQAAA4gu\nAIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCA\n6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwABbFz0ArLZ9//WLHmEut15+waJHAGCJ2NMFADCA\n6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMILoAAAaYK7qq\nandV3VxVR6pq/3GWV1VdMS2/sarOWbV8S1W9r6resV6DAwAskzWjq6q2JLkyyZ4kO5NcWFU7V622\nJ8mO6WNfkqtWLX9xko+c9LQAAEtqnj1d5yY50t23dPc9Sa5LsnfVOnuTXNsz705yelWdkSRVdWaS\nC5L86jrODQCwVOaJrscl+diK27dN9827zuuS/OskX7qfMwIALL0H9ED6qvqBJB/v7vfMse6+qjpc\nVYePHj36QI4FADDcPNF1e5KzVtw+c7pvnnW+I8kzqurWzF6W/N6q+vXj/SPdfXV37+ruXdu2bZtz\nfACA5TBPdN2QZEdVnV1VpyV5TpKDq9Y5mOSi6SzG85Lc1d13dPcruvvM7t4+Pe4Pu/ufr+cGAAAs\ng61rrdDd91bVZUnemWRLkmu6+6aqumRafiDJoSTnJzmS5O4kFz9wIwMALJ81oytJuvtQZmG18r4D\nKz7vJJeu8Xe8K8m7TnhCAIBNwBXpAQAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEA\nDCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQX\nAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhA\ndAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCA\nAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGmCu6qmp3Vd1cVUeqav9xlldVXTEtv7Gqzpnu\nf2hV/UVVfaCqbqqqV6/3BgAALIM1o6uqtiS5MsmeJDuTXFhVO1ettifJjuljX5Krpvs/n+R7u/tJ\nSZ6cZHdVnbdOswMALI159nSdm+RId9/S3fckuS7J3lXr7E1ybc+8O8npVXXGdPtz0zoPnj56vYYH\nAFgW80TX45J8bMXt26b75lqnqrZU1fuTfDzJ73f3nx/vH6mqfVV1uKoOHz16dN75AQCWwgN+IH13\nf7G7n5zkzCTnVtU33sd6V3f3ru7etW3btgd6LACAoeaJrtuTnLXi9pnTfSe0TnffmeSPkuw+8TEB\nAJbbPNF1Q5IdVXV2VZ2W5DlJDq5a52CSi6azGM9Lcld331FV26rq9CSpqocleVqSj67j/AAAS2Hr\nWit0971VdVmSdybZkuSa7r6pqi6Zlh9IcijJ+UmOJLk7ycXTw89I8mvTGZAPSvLW7n7H+m8GAMDG\ntmZ0JUl3H8osrFbed2DF553k0uM87sYkTznJGQEAlp4r0gMADCC6AAAGEF0AAAOILgCAAUQXAMAA\nogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEA\nDCC6AAAGEF0AAAOILgCAAbYuegDY7Lbvv37RI8zl1ssvWPQIAJuaPV0AAAPY0wWckGXYc2evHbAR\n2dMFADCA6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBgANEFADCAK9IDpzRX2AdGsacL\nAGAA0QUAMIDoAgAYQHQBAAwgugAABhBdAAADiC4AgAFEFwDAAKILAGAA0QUAMIDoAgAYQHQBAAwg\nugAABti66AEAWD/b91+/6BHWdOvlFyx6BFgIe7oAAAaYK7qqandV3VxVR6pq/3GWV1VdMS2/sarO\nme4/q6r+qKo+XFU3VdWL13sDAACWwZrRVVVbklyZZE+SnUkurKqdq1bbk2TH9LEvyVXT/fcmeVl3\n70xyXpJLj/NYAIBNb549XecmOdLdt3T3PUmuS7J31Tp7k1zbM+9OcnpVndHdd3T3e5Okuz+b5CNJ\nHreO8wMALIV5outxST624vZt+fJwWnOdqtqe5ClJ/vxEhwQAWHZDDqSvqkck+W9JXtLdn7mPdfZV\n1eGqOnz06NERYwEADDNPdN2e5KwVt8+c7ptrnap6cGbB9Zbuftt9/SPdfXV37+ruXdu2bZtndgCA\npTFPdN2QZEdVnV1VpyV5TpKDq9Y5mOSi6SzG85Lc1d13VFUleWOSj3T3L67r5AAAS2TNi6N2971V\ndVmSdybZkuSa7r6pqi6Zlh9IcijJ+UmOJLk7ycXTw78jyXOTfLCq3j/d97PdfWh9NwMAYGOb64r0\nUyQdWnXfgRWfd5JLj/O4P01SJzkjAMDSc0V6AIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMILoA\nAAaY6zpdADDa9v3XL3qEudx6+QWLHoElYU8XAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0A\nAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAWxc9AACc\nCrbvv37RI6zp1ssvWPQIm5o9XQAAA4guAIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMILoAAAYQ\nXQAAA4guAIABRBcAwACiCwBgANEFADCA6AIAGEB0AQAMILoAAAYQXQAAA4guAIABRBcAwACiCwBg\nANEFADCA6AIAGGDrogcAAJbP9v3XL3qENd16+QWLHuHvmGtPV1Xtrqqbq+pIVe0/zvKqqium5TdW\n1Tkrll1TVR+vqg+t5+AAAMtkzeiqqi1JrkyyJ8nOJBdW1c5Vq+1JsmP62JfkqhXL3pRk93oMCwCw\nrObZ03VukiPdfUt335PkuiR7V62zN8m1PfPuJKdX1RlJ0t1/kuRT6zk0AMCymSe6HpfkYytu3zbd\nd6LrAACcsjbM2YtVta+qDlfV4aNHjy56HACAdTVPdN2e5KwVt8+c7jvRdb6i7r66u3d1965t27ad\nyEMBADa8eaLrhiQ7qursqjotyXOSHFy1zsEkF01nMZ6X5K7uvmOdZwUAWFprRld335vksiTvTPKR\nJG/t7puq6pKqumRa7VCSW5IcSfIrSV547PFV9ZtJ/izJE6vqtqp6/jpvAwDAhjfXxVG7+1BmYbXy\nvgMrPu8kl97HYy88mQEBADaDDXMgPQDAZia6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgC\nABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOI\nLgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAw\ngOgCABhAdAEADCC6AAAGEF0AAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADCC6AAAGEF0A\nAAOILgCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEADDBXdFXV7qq6uaqOVNX+4yyvqrpiWn5jVZ0z\n72MBAE4Fa0ZXVW1JcmWSPUl2JrmwqnauWm1Pkh3Tx74kV53AYwEANr159nSdm+RId9/S3fckuS7J\n3lXr7E1ybc+8O8npVXXGnI8FANj0qru/8gpVP5Jkd3f/5HT7uUm+tbsvW7HOO5Jc3t1/Ot3+gyQv\nT7J9rceu+Dv2ZbaXLEmemOTmk9u0hXhkkk8seoh1spm2JbE9G91m2p7NtC2J7dnobM/G8ITu3rbW\nSltHTDKP7r46ydWLnuNkVNXh7t616DnWw2balsT2bHSbaXs207Yktmejsz3LZZ7ouj3JWStunznd\nN886D57jsQAAm948x3TdkGRHVZ1dVacleU6Sg6vWOZjkouksxvOS3NXdd8z5WACATW/NPV3dfW9V\nXZbknUm2JLmmu2+qqkum5QeSHEpyfpIjSe5OcvFXeuwDsiUbw1K/PLrKZtqWxPZsdJtpezbTtiS2\nZ6OzPUtkzQPpAQA4ea5IDwAwgOgCABhAdAEbQlXVyj+XUVV9e1U9cdFzABuT6AI2in+UJN3dyxhe\n05nbb0ryhap6yILHWVfL+P+x0rLPz+Yhuvgy9/UDqqp8vWwAVfWwRc+wnqZLzWxN8t+r6s3J8oXX\n9L3xdUneltk7cbxg2qaldiwee4nPuKqqOjZ/VX19VT180TOdrGX63lhLVT2vqq5b9ByjOHuRv2PV\nD6gfT/KlJKd1968tdjKSZLoEyxOTfC6zt966a8EjnbSq2tLdX5w+P5Lkd7r7Z6bbtdGf8I/NOD2Z\n35LZpXie2N3L+FYmf6uqXpzkSUnOSPJvk3yku+9c7FTzOxYmK36evTTJ7iTPm64juZRW/Yw+P0kl\n+bPu/tRiJztxVfULSX41yfOSnN7dP7XgkR5w9lyso6p68KJnOFkrvplfkuRfJrk3yc9W1Y8tdDBS\nVS9M8qwklyf5F0neUFU7FjvVyVsRXLuTvCPJC6vqimnZht7jVVUPWhGFpyf5vSQfT/LsxU118qYn\n84szi63DmT0p/tNp2Yb9/1hly6pfIJ+V5FndfUdVPaaqHrPY8U7MtEf4QSu26eIk/zGz9zn++ar6\nroUOeP/8YWYvyb8xyX9Y7ChjiK51Mr288Oaq+ulFz3KyqurvJ/mW7v7uJDsye/Px39psL2stk6r6\n6iTnZPauDj+c5H3Tois2Q3hV1bOSXJnkl5PsSfL0qjqQbOzw6u4vJUlVvSDJ/syC61VJXl5VL1vk\nbCeiqk6rqp3T59+V5OlJ3t7dt3b3q5K8N8n+qvqqjb7nMUmqaluSt634utmSWdCfX1X/JsnvJHlN\nVX3Doma8H7as+Hq7IMkPJfmmJN+X5LNJfrCqnrrA+U5Ydx9K8sokD+3u/7PoeUYQXetk+mZ4TZIf\nq6pzFj3PiTjOsVpbkjy8qn4lya4kPzrtjfjRqlraNyKd3opqKXX3Z5JcmuRRSf5Zd+/ObO/DtyR5\n7jJv2+RLSd7U3f+ru/84s70qz6yqq5KNfUxRVf1wkhdl9tv6aUken+SaJD9RVcvy2/vjk7yuqt6S\n5MWZRdZjj0VJd1+d5K+SPGFxI86vu49m9gvK06rqHyT5iySPTbIvyQeSvCyzUNmQMb/ayoicjhXc\nldn3yLnd/YUkr0/y+SQXVtW3L3DUE9bdf9DdH1r0HKOIrnXU3R/MbFfvXy56lhOx4rennVX1kOnY\ngHcleWaS/d39N1V1UZKXJvm/i5v0/quqhyZ50zKfVdbdn8/sbba2VtU3JbkgyR8k+dXuvmehw52A\n+9hr9TdJnn0sHrv7r5L8ZpLdVfXojbqna/LEJP+1u9+f2ZP53Um+JsmFSZ5aVY9c5HDz6O4jSW5M\n8oNJfq+7r80shH+kqp497Yn8+iSfXOCYJ6S7707ysMwC8vbuvjTJBd399iRfm1m03L3AEee2IiK/\nP8kjuvvVSd6Q2d7HfzItf0NmP5+PLG5S1uJA+nW2DAf+HlNVT0ny1O6+Yjpe6NIktyf5L5n9Nrg3\nyQuTXJ/kO5M8d5nfO7OqHt7df73oOU7GFI0vyeyH72MzO0blw4udan6rDgL+qcyeyN+T5C1J/n1m\noX9JkqckOTfJi6cnlA2rqp6Z2fFPrzj2f1FVf5zZyz+fPvZLzUZXVV+X5Nsy++XqlZntHfqBzI6F\nOprkF7v7xsVNeP9U1Z4kv5RkV3d/uqouzOyl4B9ftj0s09faLyX5x0k+k+QVSb45yWu6+73TMV9L\n8fV2qhJdp6hpz8H3J/mZzH4T/IdJXpDZD9inZPYD9zeSPDmzg+k/2d1LtQdvs5pO2HhMki919+2L\nnuf+qKrvziyyDiY5K8n/S/KvkhwLsa/LLGI2/JN8VZ2e2fdRZbaH+GFJfi7Jno0ejMdTVc/I7FCJ\nFyV5SGYh9rru/vRCBzsJU3j95yTnJXlokq3dfdtip7p/phNO3pDZS4yfSfLvkpyd2ck19yzLL/2n\nKtF1CqqqRyX5mu6+eTrm5PuS3NbdPzQtf25me7bel+Sty3gqMhvX9PX100ku7u4PTMcJPjuzwx1e\nM+2NOG3JXjJ9bGZ7tp6R2eU8Xt3dH1jsVPffFCmvTfLFJBd290cXPNJJm/YSvTKz46CW+olvOrv0\nPyX5tu6+s6q+truX5qXfU5noOgVNZ7v9cpLbMjsw9o2ZPQle291XTOv8ZJJvTPLzm+FaUCzO6pfc\nq+qszI4fenN3v2i675uT/ERme7xekeSLy/jEWFVfldnP1aV+GTv524O3s4x76+5LVT2iuz+36DnW\nwxSRr8rsZdOl+145VYmuU9R0Ubp9SV7e3VdNv9m+IMkfdffrp3W+ejprDu6XVcdwXZZZyH8ws+O4\nDiV5bXe/dlr+5MwOeN40T/LwQNpMEXmqWPq3qeB+O5DZwfIvrapPdfdvVdXHk/xyVX2yu39dcHGy\nVgTXsQu7/nhme7kOJHl+kl+aTnD4uensP2BOgmv5iK5T1HSK+JGqujOziwTemdkBpvck+Z8LHY5N\nZdWFXZ+V5IbM3p/w0ZmdiXl5Vb0+yae8TAJsZqLrFNfdb6+qLyT5hSR/neT53f2/FzwWm0h3f6aq\nLk3yDZld2PV7prNn78zsLWae3N2fXeiQAAOILtLdv1tV75196nga1l93f76qVl7Y9QlJfjfJIcEF\nnCocSA8MsewXdgU4WaILGGYzXNgV4P4SXQAAA3jDawCAAUQXAMAAogsAYADRBQAwgOgCABhAdAEA\nDCC6AAAG+P/Q8SqIhllbHQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e49af7fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_top_k(alpha, label='pos', k=10):\n",
    "    positive_words = [w for (w,y) in alpha.keys() if y == label]\n",
    "    sorted_positive_words = sorted(positive_words, key=lambda w:-alpha[w,label])[:k]\n",
    "    util.plot_bar_graph([alpha[w,label] for w in sorted_positive_words],sorted_positive_words,rotation=45)\n",
    "plot_top_k(alpha)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is actually fairly uninformative. What we see are words that appear with high probability in positive documents. Clearly *any* document will contain these words with high probability, and so they don't serve well to discriminate between document classes. It makes sense to remove such words apriori, using a so-called *stop-word* list.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmMAAAGLCAYAAACLN5UPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcXFWd///Xh4SAgOxBEYhBCbKoCEbEBXEBTAANKiq4\ngAjEIJGvO3EfN8RlRkdFGGZEwA2Z+aJmhBlg+DJuIwouoCBoRBxAVNzw56Ai4/n98Tlll2UnXZ2+\n3SfdeT0fj350LbfuObfq1r3ve865t6KUgiRJktrYoHUFJEmS1meGMUmSpIYMY5IkSQ0ZxiRJkhoy\njEmSJDVkGJMkSWrIMCZJktSQYUySJKkhw5gkSVJDs1tXYDy23XbbMn/+/NbVkCRJGtPXv/71n5dS\n5o413bQKY/Pnz+fqq69uXQ1JkqQxRcSPhpnObkpJkqSGDGOSJEkNGcYkSZIaMoxJkiQ1ZBiTJElq\nyDAmSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxjkiRJDRnGJEmSGjKMSZIkNWQYkyRJasgwJkmS1NDs\n1hVY18xfcVHn87z5tEM7n6ckSZoZbBmTJElqyDAmSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxjkiRJ\nDRnGJEmSGjKMSZIkNWQYkyRJasgwJkmS1JBhTJIkqSHDmCRJUkOGMUmSpIaGCmMRsSgiboyIVRGx\nYpTnIyLeX5+/NiL26Xvu7Ij4WUR8Z+A1746IG+r0n46ILSe+OJIkSdPLmGEsImYBpwOLgT2AoyJi\nj4HJFgML6t9S4Iy+584BFo0y68uAB5dSHgp8D3jNeCsvSZI03Q3TMrYvsKqUclMp5W7gfGDJwDRL\ngPNKuhLYMiK2ByilfAH45eBMSymXllLuqXevBHZc24WQJEmaroYJYzsAt/Tdv7U+Nt5p1uSFwL+N\n9kRELI2IqyPi6jvuuGMcs5QkSVr3NR/AHxGvA+4BPj7a86WUs0opC0spC+fOnTu1lZMkSZpks4eY\n5jZgp777O9bHxjvNX4mIFwCHAU8qpZQh6iJJkjSjDNMydhWwICJ2jog5wJHAyoFpVgJH17Mq9wPu\nLKXcvqaZRsQi4NXAU0spd61F3SVJkqa9McNYHWS/HLgE+C5wQSnluohYFhHL6mQXAzcBq4B/BF7c\ne31EfBL4CvCgiLg1Io6rT30QuDdwWUR8KyLO7GqhJEmSpothuikppVxMBq7+x87su12Ak1bz2qNW\n8/guw1dTkiRpZmo+gF+SJGl9ZhiTJElqyDAmSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxjkiRJDRnG\nJEmSGjKMSZIkNWQYkyRJasgwJkmS1JBhTJIkqSHDmCRJUkOGMUmSpIYMY5IkSQ0ZxiRJkhoyjEmS\nJDVkGJMkSWrIMCZJktSQYUySJKkhw5gkSVJDhjFJkqSGDGOSJEkNGcYkSZIaMoxJkiQ1ZBiTJElq\nyDAmSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxjkiRJDRnGJEmSGjKMSZIkNWQYkyRJasgwJkmS1JBh\nTJIkqSHDmCRJUkOGMUmSpIYMY5IkSQ0ZxiRJkhoaKoxFxKKIuDEiVkXEilGej4h4f33+2ojYp++5\nsyPiZxHxnYHXbB0Rl0XE9+v/rSa+OJIkSdPLmGEsImYBpwOLgT2AoyJij4HJFgML6t9S4Iy+584B\nFo0y6xXA5aWUBcDl9b4kSdJ6ZZiWsX2BVaWUm0opdwPnA0sGplkCnFfSlcCWEbE9QCnlC8AvR5nv\nEuDcevtc4PC1WQBJkqTpbJgwtgNwS9/9W+tj451m0H1KKbfX2z8B7jPaRBGxNCKujoir77jjjiGq\nK0mSNH2sEwP4SykFKKt57qxSysJSysK5c+dOcc0kSZIm1zBh7DZgp777O9bHxjvNoJ/2ujLr/58N\nURdJkqQZZZgwdhWwICJ2jog5wJHAyoFpVgJH17Mq9wPu7OuCXJ2VwDH19jHAZ8dRb0mSpBlhzDBW\nSrkHWA5cAnwXuKCUcl1ELIuIZXWyi4GbgFXAPwIv7r0+Ij4JfAV4UETcGhHH1adOAw6KiO8DB9b7\nkiRJ65XZw0xUSrmYDFz9j53Zd7sAJ63mtUet5vFfAE8auqaSJEkz0DoxgF+SJGl9ZRiTJElqyDAm\nSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxjkiRJDRnGJEmSGjKMSZIkNWQYkyRJasgwJkmS1JBhTJIk\nqSHDmCRJUkOGMUmSpIYMY5IkSQ0ZxiRJkhoyjEmSJDVkGJMkSWrIMCZJktSQYUySJKkhw5gkSVJD\nhjFJkqSGDGOSJEkNGcYkSZIaMoxJkiQ1ZBiTJElqyDAmSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxj\nkiRJDRnGJEmSGjKMSZIkNWQYkyRJasgwJkmS1JBhTJIkqSHDmCRJUkOGMUmSpIYMY5IkSQ0ZxiRJ\nkhoaKoxFxKKIuDEiVkXEilGej4h4f33+2ojYZ6zXRsTDIuLKiPhWRFwdEft2s0iSJEnTx5hhLCJm\nAacDi4E9gKMiYo+ByRYDC+rfUuCMIV77LuDNpZSHAW+s9yVJktYrw7SM7QusKqXcVEq5GzgfWDIw\nzRLgvJKuBLaMiO3HeG0BNq+3twB+PMFlkSRJmnZmDzHNDsAtffdvBR45xDQ7jPHalwKXRMR7yFD4\n6NEKj4ilZGsb8+bNG6K6kiRJ00fLAfwnAi8rpewEvAz48GgTlVLOKqUsLKUsnDt37pRWUJIkabIN\nE8ZuA3bqu79jfWyYadb02mOAC+vtfya7NCVJktYrw4Sxq4AFEbFzRMwBjgRWDkyzEji6nlW5H3Bn\nKeX2MV77Y+CAevuJwPcnuCySJEnTzphjxkop90TEcuASYBZwdinluohYVp8/E7gYOARYBdwFHLum\n19ZZnwD8fUTMBn5PHRcmSZK0PhlmAD+llIvJwNX/2Jl9twtw0rCvrY9/CXj4eCorSZI003gFfkmS\npIaGahlT9+avuKjT+d182qGdzk+SJE0NW8YkSZIaMoxJkiQ1ZBiTJElqyDAmSZLUkGFMkiSpIcOY\nJElSQ4YxSZKkhgxjkiRJDRnGJEmSGjKMSZIkNWQYkyRJasgwJkmS1JBhTJIkqSHDmCRJUkOGMUmS\npIYMY5IkSQ0ZxiRJkhoyjEmSJDVkGJMkSWrIMCZJktSQYUySJKkhw5gkSVJDhjFJkqSGDGOSJEkN\nGcYkSZIaMoxJkiQ1ZBiTJElqyDAmSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxjkiRJDc1uXQFNrvkr\nLup0fjefdmin85MkaX1ny5gkSVJDhjFJkqSGDGOSJEkNGcYkSZIaGiqMRcSiiLgxIlZFxIpRno+I\neH99/tqI2GeY10bESyLihoi4LiLeNfHFkSRJml7GPJsyImYBpwMHAbcCV0XEylLK9X2TLQYW1L9H\nAmcAj1zTayPiCcASYK9Syh8iYrsuF0ySJGk6GKZlbF9gVSnlplLK3cD5ZIjqtwQ4r6QrgS0jYvsx\nXnsicFop5Q8ApZSfdbA8kiRJ08owYWwH4Ja++7fWx4aZZk2v3RXYPyK+GhGfj4hHjKfikiRJM0HL\ni77OBrYG9gMeAVwQEQ8opZT+iSJiKbAUYN68eVNeSUmSpMk0TMvYbcBOffd3rI8NM82aXnsrcGHt\n2vwa8Cdg28HCSylnlVIWllIWzp07d4jqSpIkTR/DhLGrgAURsXNEzAGOBFYOTLMSOLqeVbkfcGcp\n5fYxXvsZ4AkAEbErMAf4+YSXSJIkaRoZs5uylHJPRCwHLgFmAWeXUq6LiGX1+TOBi4FDgFXAXcCx\na3ptnfXZwNkR8R3gbuCYwS5KSZKkmW6oMWOllIvJwNX/2Jl9twtw0rCvrY/fDTxvPJWVJEmaabwC\nvyRJUkOGMUmSpIYMY5IkSQ0ZxiRJkhoyjEmSJDVkGJMkSWrIMCZJktSQYUySJKkhw5gkSVJDhjFJ\nkqSGDGOSJEkNGcYkSZIaMoxJkiQ1ZBiTJElqyDAmSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxjkiRJ\nDRnGJEmSGprdugKa/uavuKjzed582qGdz1OSpHWRLWOSJEkNGcYkSZIaMoxJkiQ1ZBiTJElqyDAm\nSZLUkGFMkiSpIcOYJElSQ15nTNNG19cz81pmkqR1gS1jkiRJDRnGJEmSGjKMSZIkNWQYkyRJasgw\nJkmS1JBhTJIkqSHDmCRJUkOGMUmSpIYMY5IkSQ0ZxiRJkhoyjEmSJDU0VBiLiEURcWNErIqIFaM8\nHxHx/vr8tRGxzzhe+4qIKBGx7cQWRZIkafoZM4xFxCzgdGAxsAdwVETsMTDZYmBB/VsKnDHMayNi\nJ+Bg4L8nvCSSJEnT0DAtY/sCq0opN5VS7gbOB5YMTLMEOK+kK4EtI2L7IV77XuDVQJnogkiSJE1H\nw4SxHYBb+u7fWh8bZprVvjYilgC3lVKuGWedJUmSZozZLQqNiE2A15JdlGNNu5Ts+mTevHmTXDNJ\nkqSpNUzL2G3ATn33d6yPDTPN6h5/ILAzcE1E3Fwf/0ZE3Hew8FLKWaWUhaWUhXPnzh2iupIkSdPH\nMGHsKmBBROwcEXOAI4GVA9OsBI6uZ1XuB9xZSrl9da8tpXy7lLJdKWV+KWU+2X25TynlJ10tmCRJ\n0nQwZjdlKeWeiFgOXALMAs4upVwXEcvq82cCFwOHAKuAu4Bj1/TaSVkSSZKkaWioMWOllIvJwNX/\n2Jl9twtw0rCvHWWa+cPUQ5IkaabxCvySJEkNGcYkSZIaMoxJkiQ1ZBiTJElqyDAmSZLUkGFMkiSp\nIcOYJElSQ4YxSZKkhgxjkiRJDRnGJEmSGjKMSZIkNWQYkyRJamioHwqX1hfzV1zU+TxvPu3Qzucp\nSZo5DGNSA4Y+SVKP3ZSSJEkNGcYkSZIasptSmsG67g61K1SSumcYkzQhUzX+zWApaaaym1KSJKkh\nw5gkSVJDhjFJkqSGDGOSJEkNGcYkSZIaMoxJkiQ1ZBiTJElqyDAmSZLUkBd9laQ+XlxW0lQzjEnS\nFJuqXy2QND0YxiRphpqKVr6Z9nNYtoyqBcOYJElTyJZRDXIAvyRJUkO2jEmSNAPNpG7qmc6WMUmS\npIZsGZMkSeu0md4CZ8uYJElSQ4YxSZKkhgxjkiRJDRnGJEmSGjKMSZIkNWQYkyRJamioMBYRiyLi\nxohYFRErRnk+IuL99flrI2KfsV4bEe+OiBvq9J+OiC27WSRJkqTpY8wwFhGzgNOBxcAewFERscfA\nZIuBBfVvKXDGEK+9DHhwKeWhwPeA10x4aSRJkqaZYVrG9gVWlVJuKqXcDZwPLBmYZglwXklXAltG\nxPZrem0p5dJSyj319VcCO3awPJIkSdPKMGFsB+CWvvu31seGmWaY1wK8EPi30QqPiKURcXVEXH3H\nHXcMUV1JkqTpo/kA/oh4HXAP8PHRni+lnFVKWVhKWTh37typrZwkSdIkG+a3KW8Dduq7v2N9bJhp\nNlzTayPiBcBhwJNKKWXoWkuSJM0Qw7SMXQUsiIidI2IOcCSwcmCalcDR9azK/YA7Sym3r+m1EbEI\neDXw1FLKXR0tjyRJ0rQyZstYKeWeiFgOXALMAs4upVwXEcvq82cCFwOHAKuAu4Bj1/TaOusPAhsB\nl0UEwJWllGVdLpwkSdK6bphuSkopF5OBq/+xM/tuF+CkYV9bH99lXDWVJEmagZoP4JckSVqfGcYk\nSZIaMoxJkiQ1ZBiTJElqyDAmSZLUkGFMkiSpIcOYJElSQ4YxSZKkhgxjkiRJDRnGJEmSGjKMSZIk\nNWQYkyRJasgwJkmS1JBhTJIkqSHDmCRJUkOGMUmSpIYMY5IkSQ0ZxiRJkhoyjEmSJDVkGJMkSWrI\nMCZJktSQYUySJKkhw5gkSVJDhjFJkqSGDGOSJEkNGcYkSZIaMoxJkiQ1ZBiTJElqyDAmSZLUkGFM\nkiSpIcOYJElSQ4YxSZKkhgxjkiRJDRnGJEmSGjKMSZIkNWQYkyRJasgwJkmS1JBhTJIkqSHDmCRJ\nUkOGMUmSpIaGCmMRsSgiboyIVRGxYpTnIyLeX5+/NiL2Geu1EbF1RFwWEd+v/7fqZpEkSZKmjzHD\nWETMAk4HFgN7AEdFxB4Dky0GFtS/pcAZQ7x2BXB5KWUBcHm9L0mStF4ZpmVsX2BVKeWmUsrdwPnA\nkoFplgDnlXQlsGVEbD/Ga5cA59bb5wKHT3BZJEmSpp0opax5gogjgEWllOPr/ecDjyylLO+b5nPA\naaWUL9X7lwOnAPNX99qI+HUpZcv6eAC/6t0fKH8p2doG8CDgxgksb9e2BX4+A8qYqnJm0rJMVTkz\naVmmqhyXZf0ux2VZv8uZqmUZ1v1LKXPHmmj2VNRkLKWUEhGjpsJSylnAWVNcpaFExNWllIXTvYyp\nKmcmLctUlTOTlmWqynFZ1u9yXJb1u5ypWpauDdNNeRuwU9/9Hetjw0yzptf+tHZlUv//bPhqS5Ik\nzQzDhLGrgAURsXNEzAGOBFYOTLMSOLqeVbkfcGcp5fYxXrsSOKbePgb47ASXRZIkadoZs5uylHJP\nRCwHLgFmAWeXUq6LiGX1+TOBi4FDgFXAXcCxa3ptnfVpwAURcRzwI+BZnS7Z1JiK7tOp6qJ1WdbN\ncmbSskxVOS7L+l2Oy7J+l7NODmsay5gD+CVJkjR5vAK/JElSQ4YxSZKkhgxjkv5CRMyu/90+TGP1\n+o3C90LrPje2mjLrwwZxui9jRGwDXBQRDyil/GkyAtl0f4/WVf3va0RsXMYxIHg9+Ew2gW4PMCJi\ns77b23Q139WUtWn9v858TutSXWYCw9hamukr4uqWb22XOyKit3OIiHtNpG7DlDXKY1O1rt+v6xlO\n8br2K/KSNOdGxI5dB7KB9WCDiNiwd7urMsYqfyrKaaHvfV0OfGjY9zQiNuh77ZyI2HgSqzlY9qR/\nHhExH/hyRDy0q/W5vkdPiYjDI+I5wAkRsdFE5ztKOREROwNXR8RD6gXSm6/DA9/jucO8py3qHRH7\nRsQhU13u2jCMraW+FfGAqfiw6wr/Vz8XNUll9X/RnhMRz4j8KSvGc7S9mvm9HDhrMjZco5S1KCIO\njogHl1L+NBnlDZS9DXB6RDy0w3n2L8/eEbFTROzQ1fxHKetPwCeBO4GPdh3I+pblZOBM4LyIeMJk\nfj71fds6IjapO7NOt3sRsU1fqHxuRCyPiIPr/SndAUXEQcBTgVcM+572pqufyYfJIP7MSazj4+tn\nsmCyw0Vdp28GPkZudx7U0fr8R+C7wLuAdwAfKqX8ISJmTXC+g+5VSvkh+bvOH42IPdaFQNb3PX4F\n8BZgq3G85oiIeH3dr8yfrDrW92hP8jeymYTPplOGsXHq/xLUI9D3A2+NiLMjYqfVv3JCZZ4MnAN8\nMCLeMRll9Bs4wn4x8DvgzIhYqx9z75vfScDTgTfWDVfngayvrBcDbwJ2Ba6JiD26Lms1bgV6v7k6\n4e9X3/K8FDgd+ABwakQsmei8RysrIp4CfBT4AvAb4F8i4v5dBrKIOIEMDG8GtgNe0MV8V1PWi8kL\nTP8t8IaImNvxsjwAOBV4dEQcA7yR7BL7l4h4zlTuOCNiO+AwYA9g6/rYasuOiAdFxH3q7ePIz+RV\nwL2AIyapjieR79cTgS9GxLy1OcAbsqzom/fVwC/IoLnb2q4DvfezlPK/dX6/Bm4AntL3eCci4kHA\nx2oAezMZyC5YVwJZRLwIWAL8TSnlFxGxSeTF3dfUs/Iicrv8W/Ji78dHxCMmo371s/9v4MCI2K7L\nz2YyGMbGYaCVYg5wb2C/UsrDgTnAiojYseMyjyRX+OcB/wM8pMv5D5S1U9QuxIiYCzwOeBKwG3AF\n8K/jCVB1Y/83fQ/NIzf229Sd5GUR8eyI2KirDUuk3cmLEB8E/C/weeCGXhldb8R6R3ellF+QXXwf\niIhtu2rtqUHy+cAi4KXAp4FjI+LhXcx/wCHAO0op7yJ/MeMS4JyI2KnD1qtNyeV5FvAH4LjI7rHt\nOpo/ABHxVGAH4LFki08hA9m2dWc84fWglHIT+aPERwGPBp5V37vDyQOYIyczbPTdnlNK+Rnwd2T4\nXF5D9Kg77Yg4kGyZ7N8HHE9+5hsCz42I2RExr8P67kUGvgNrGdcAt3YVjAcNHMi8E/i/wPXAJyJi\nz/EGsoHt/3Jy3TqYDBfPrNs0ImJhZNfiROt/I/Bj4I01QJ5GtvA1CWQD69ts8qDzHOB+kT0enyC/\nX5uNts7XlqlHAy8opbwPeHV96jEd1/PBEfEWgFLK5cC/Aa+qdV5nGcaGFH85ruKVwEXA0cChdZIT\ngM3JVosuu5F+B5xSy5oPPK3WYe8Oy6DW+Uhgdl1pf0X+msI7yZ3Zs+qRxVAhoH5xNwHmRsRb68O/\nB15HNuv/kfySHAJr1/05mjqfHwP/VctaAhxSg8QJEbF9lzvHuoN5R0RcGhH7AJcCZ5PN42vVOraa\nDexvSim/qV0uXyR/47WzHWUtdwPyoOLB9aG7yeC3LbkD2Hi8G//VTL8DGZD3LaUcUkq5BziOXLcm\n3JUQORZtK7Il4VGllB8BXyGX5bfAu2ogW+v1oIb+XivJ68guqz2B/evO6P+RgewTETEprUwDXf+n\nR8SFwGbAueSB2/LIEzH+vJwxMk7vcLL1c7uIeAywC/mZPKqUsrjvM3lhnb4LvyHX3ZOBxwNL6vfy\nyIjYuqMyiIhdI+KAvof2BF5aSjmb3E7/X7LLcvfxHGD0vd8vI1tzf1NK+TXwDeAMYHH9DP6O3Nat\nbf2jt90opSwnv+unDgSyT0SOgZuyq7YPDDE4i/w96acA7yP3FZ8l17+/EhH7kvuDnwPPizzB5Abg\nP4DDop6g0JFNgb0j4oKIeA3wS3J/87+1Ls3H3I2qlOLfOP7IFP85stXoteRR6MH1uU2AfwTu22F5\nzyObwj/X99jxwAfJ8QRdLtu9gYcCy+v9t5Er8Cb1/nOArwM7jjGfOX239wVOgj//2sPDgC3q7YPJ\nHcA2E6jzVn23jwTeTobiy4AfDTz3X8D9OnifesuyL7mT3x54DfBW4EvAtcCHJzLvevspZGAB+Hfg\ntL7n/g44paPl2AvYhwx388jWgxPrc4+uZe3dwXr8XLILbdP6WXywPncccB3woI7W4zn1/wJy4/+K\nvuceTbZk3Gei71u9fQJwfL19CvAhsuXnXvWx/btartXU5fl1XZ8NfA94T318H+Dv6/dhdt/0W9b/\nO9b35qd9z/0n8HGym3Jp/Ux266COc8mWsDnAlcDtA+vFfwBzO3o/Nia7v/8eeFx97PyB787DgW8B\nl9c6xRjz7P+8tya3/zuR25kjgGV1vd6NHEO2e0fr1s59t/+GPJjYrd5/S/0OzVnbstayfgfV9W3j\nev9+wGb19uFkd/C2A6/Zmhxesah+H/4WWFqfeyrwGWDTDuq2O/CwvvuPBV5U1+P/AV45le/VuOvf\nugLr+l/9gj2j3n4EGUbeUu9vVTfGnwUO67DMY+uXb/96/63AN+vK9n/Inf2ek7Csm5A7zA/XjeQc\nMvR9ndzJXAU8eIx5bE522WwLPJkMcHMGppldN2DXAg+ZQH3nA/8EPKnePwZ4Ub29LfB9MhyfVTcS\na13WKGUfSDbLH9L32Abkzv6MuqF8ygTmf3L9zHet9/cEPkK2vL2UDEy7dLAch9Zyzqsb2ZeSrSTf\nr+Xd2r+M45hv/07l8Frff6jr1rOBbeqyfAz4f8AeHX0uLyS7304GdmYkkL28b5qNOyrrZOCr/esV\n8HpybN8hdHywVOc/a+D+S8kDxJPJluaN+tbFBcB2vc8DuD8ZTDYlQ/ePyB3Vc+s0m5C/M3wO2T09\n4c+EPBC7mNwZv4RsFb2h3j+N3KZ08r1k5OBiAfAG4D31e7NT3dasqM8/p5Y95oHZwHr8PLJF7FN1\nm/LP5Pbl89QD2A4/5xfXz+BdwGvrY++pZe9Z76/1Qexa1mk7MuTeCizoe3zT+t58l4H9EjWYkg0I\nZ9TbTydPEroc+BqwVwd1W0Juxz5Vtyn7AhvU5/Yg96mnkw0Oawzfrf6aV2Bd/6sbuzOAQ+v9d9aN\n3gPq/S3qRuZTdaWc0AddV6qv1Y3V2XUjuzG5kf+nuhKv9ZHXasoM4ABypxj1y/Ih4Jj6/GPJkDF/\njPnMrv+fXze41zPSShED0x7DBMJE/VLdG1hB7nwfRXblntA3zRZ1WZ7Z+7w6fM+eC/yJkfA3u+/L\nf2/gFcBx45jfA/pu70uGx94R52PIgLtVXR9fTDc7ynuTO5LH1vu7kF1XT6tl7Ta4cR12feq7vQPw\nMkaO6J9F7sCe1l+Pjj6TpbX+jyNbPt5YH9+9flYv6fDz36x+F+fV+xv3Pddrmdmk43VuC+CJ9faT\ngL3r+nAF2fW2YX3uDcDbB17b+25uAzyh7/PYixy7taxv2lld1J1sjf5PshXuY8D59fH7kAeVy6gH\nGx2/T71tw5uorbrAA8lA9klg1XjXa/Lg6z/r7ceQoax3oHQCeTAzZivbkGUdAnyZbFG8EDi377kP\n1rI27KKsMerR/z3ubcfvT+4b3gvcvz52L/Kg7gEDr9+fbK09tb43l/Z9JzcgQ9K2HdTz4WTgv0/9\nXG6vdXwcI9vkzclw+8DJfM8mtBytK7Cu/gH7kTujzcmw9SHgoPrc+8jm/Af2fdBbdlDmErIZf/t6\n/+m1rOV9X4bZEy2nzif6/9fb5wNH19tHk2eKnkgNBWPMby7w2Xr7QOCndeW/b31sg67qTbaI9Tbs\n9wVeSYbXC8nxMnvVOjyJvm7Mjt6v7fo+iyPIsSGP6X02fdO9nezO+PNja1iejcmjyrfWx7YiW5HO\nJ3fql5A72xO6WJa+sjclxz7O73vsKPq6dNZinhv03X4F2QX1Y0ZC65ZkIDufkbDfyU6F7Fbfngz6\nva673me1Kx3u+Mmd4RXULsq+x/eq/7fu8rOq85wPvLy+p9+sj+1GtggsAx5AHiRcQ19YJ1uJb+7V\niTyAuYE8+QjyzMargJd1WNfNyO3Xw8mQfCkjYbHTA6OBch9CdnvtS4bXN5HdYrvVz2xramvhGPPZ\nD1hYbz9bewLNAAAStUlEQVSmfk9OH2W6Y4Hv0F3L7qPJg8eDye77S/vW4V3q/zHr3/F7+nLyAOpC\nsqXxkfW79m5G9oGDB9u9Af4ryZa0V5P7ki9Rh150VLcFZMBeSDYofJ0c83oB2bjwZHIbu1etx7yp\nfO/G8+cA/lHUAbf/Sg5gnk02b14PHB4RB5VSXgr8BHhvROxccmD1rydY5iZ1nrtRT/UvpVxIHlk+\nDFhaB3V2cnpuqWsyeTTR88/kmDFKKeeRG5n7k+/BWPO7AzgqIh5PdtHtSoaRf4h6na+I2D0mfjmL\nWSUHsS+PiCeQLS+nk6cw70AeDS0md04vZ4jr3wyjlD9f9uE84LMR8Vryy35kvX9AyUHPvcGhvyG7\nF+7pe69HE6WU35Pjww6LPPv0TjKs/IzcCB5CtvhMaFl6A1cj4n4RsVEp5X/IbrZPxciFPgvwgHp2\n47gHupaRa1YdQg7SfgbZ5f6yiHhs/Z5cRm4sL6mvWdP7M8xyHVAH7P+C/L48p5RyUP08Toi8xMT3\nSinfm0g5taxnR8QppZQ/kp/N/HpmIhFxFPC+iNimlPLLiZbVV2ZvMPfNZKDYnxwIT8lB0C8hW6/f\nRHbBPbeUcn3v9aWUn9dp/isitio5CPwD5HfzkSVPNng98LSI2HKiA5zrWYUnkNuWK8iTfw4upfwx\n8rImS6OjC8uOUte7yO/lcnJH/T5yzO3LgUeUUn5Z8qzTNc2zt/2/I/LSH18nt//bR8TCvul2IVvk\nn9X/fq9t/SNic/Ig+ACyJfH4+r7dHREnAifHyFmzUyLy+pKLySD1YODYUspXyWA2hzzBYzawYdTL\nB0VeY+9ksjXyBLJL94/ktvHRwDPW5sSm1diEPPC/q877o6WU75D7nt8DPyjpGuCRpZT/7qjc7rVO\ng+vaH3kUeRF5hLU1+aU4jGy677WQ9cYonUo3A8KXkeM03kyOebmBXOl7zx/GBAYcD5TVa7nZgOy+\nuZ1sWVrMyOnmz+ubfotxzv9pZDDaor5nr63v55vJsLfWLQb89RH+q+t79dBa95eRXZYLJ2G92LWW\ntTfZJP9KRsZAHEcOEN2SgTE9w3wWffd3qO//ioHHjyG7LbvomlxEnnRwHtm6O4+8NtZ36zLdACxa\ni/k+ipETP+aTYevyvudPrGX0utm6ag3bun4nDyCPfj9H7Y4kg8m36XAAPdnicj25k3kA2fr3FXKY\nwl+NmemgvP6W6/3JVvjDyYB7KnXcUF0/gzW0Ytfv+A+orcXk9uxqRlp2u+iafBE5zGKHev+dZLCZ\nR3avX9P1e1TLWdJ3ez65Tf04GSC2Jk+uGKZFbLTt/2JgI7LL893APn3TdzIusPfdrtuSN5EnBH2C\nbLU8jjxjc43jdTuqx9qOSdyFbMX7KHmQtbxuA/6BbFzofe+f3MX3kWyd67W2v4ccm/g44Ifkduyb\nwAH1+U56kyb9vW9dgXXtjxxr8O9ky8Qn6wbkn8id2Kx6/zzg8R2V9wyyBephZLfgy8md41fpOwus\no7L6N+y9s2F2IZvav0AeHa+oyz6R0LSIHADeO2tyad3ATXgjTLYg3cDIDmU5OT5oL3Lswhup43Xo\naIdfy9mdvzyjdff6Pj2h3t9pAp/FS8hxiSeTJ4l8l5FBu3sD/0IHg5zJ8PA9cqe+Kxlmv0x2KT2T\n7HY9YC3nPY/sIpxf7x9M7oT7z2R8GdnKcK+OP5tTgY/U208nd6CX12XrZAdGds9sW2/3zsY7gQxA\nO5LdJNt3tUyjlH9i/ezuV+8/iex+e3Ndf97HECcm8NeB7NVkK9vGE/1M6uf6GXKHuy0ZiN4K3EJu\nMy/oYhtQy/rzQWX9/zXgy33P71LLu5Q8WBtq2Rh9+/9h6hmyZBg7gwmeXTxQ5qPIkyleVL+XXyCH\nOSwhu/nO6Wo9HqMeaz0msT7+HrJHoHd28X3Jg6Pvka1XnXw/yAPvD9WyjmKkAeAI8kD5HGDxZL9f\nnb//rSuwLv6RFyb9LSM7xOPJwfRPJgPZCXR0+Qqy5eiV9facugF7Lzlm4QqytaXTgZrkTn8l2aVz\nVH3s3mQY/AxwBxMcA0d2rX2XkVasjTqs/+AO5WTyCH8f8gh2QuN1yB3sM8kj48eTR/eb1Y3k8X3T\nvY+R1qBZvdeOs6wXk4PodyS72d5JBs5rgDfVacYcszdEOduTR/sfqvd7O7HTqWfTreV8t+vtKMgW\nm58zcrbxYvLIuP9MxgmPrazzWUi9pEy9/+/AM/s+v/sCm09g/v1j3x5I7oBfwkhL1CPI7uTXd7Ve\nr6Eue9f1e6eBxw8gx+58k3HsrOvnckPfd7OTcZV1Xktrff6VPBNwWV2n70V34137P5st+25/mjrI\nvt7/GzI8jav3gtG3/x8hA8om9T3vZNwWuc3fkRza8Q0ygL2bPNmgN0ask/G2Q9RlPmsxJrHv9buQ\nJ299kzr2uD5+IhmQJjxWkDwzdm6ty1Vk62dvrPUl5Da7t21bJ8+aXO2yta7AuvhHjpM6sG6wltUv\n4AvJbrYDOy7rcPLSGP0Dbq+oK3Ynp+APlPd0stVtV7IF6wfAi+tzs8hukAl3vdb5LSFbEDboeoPC\nXweyU8gj/E5CH3mq9i/4y4HOTyPDxXvJJvEbyYtkrm0Zm5OtrtuTgfISsmviI7WsrzGxFspe68FD\nye7bV5MXkOzvAn8LE2iBJbsqet0TF5Jjl65g5KypJ9cN5vL+Ok3ws9mnLsv15I5zd7J198Qu17Fa\n1mHktZXeU8tcRr0mFhlkr2ScXflrUYeHM3LCSjDSQtH7P+7QycilADbocqdFtrA9gpGg95y6PnR6\nZmmd99K6TT6TehkZMpB9njzI/TZrMWCb1W///4WOekRqOY8iWzb3JMPFSrJLbyl59u8bGMewhwnU\noz/cnkL+Ksb7+x57LNnafC7ZwzHW5Y16B5NLyO3kW+nmgPJeZBA+t9bpoWRwfRDZo3MbfZfcmG5/\nzSuwLv/Vjf73yT773k+4dNoVQbZ8vY1slu79yO9VdHDKb53/BgP3n03+3E3v/iPIboR9uihvlPIn\n/CVcw7w7P8Jn5KjqfnXD/kNGuoa2IbtDzyObyZ/aQXkb1XleUe8H2eJyEh1c8qFuGD9PhoaPkq15\nt9Sd1dPJsPz4CZbR657ojdXak+wifEO9fxDdtSSfRHY/vaZ+Z84lrxn1HfJsqUdOcP793cdHkidR\nvJkMxh8hw987yHEpF9Dx2VkD5ffG52xBdu+e2Pfci4C/HXzNOMuazO/mBnW7+W0moYuNkeEKDydD\n8vsYuZDoKWSQmVC5TPL2n2wRO55s9Xxh/UyfXp87jim4DAMdjkkcmO8isnXvG3R4Kab6XTiQ7Pp8\nJ3lyQO9aZuvsmZJDLVvrCqzrf+SO8hfU0/AnqYz7kWOfLiVbFyZ8EbxRyjicHMdzKtkd+edxO2Sw\n6OTU7AafT+dH+PW9+ibZPfUSsrv1YfW5HQem7aKlZwHZqvcQsiXmU9Rr+ExwvvchTyXvXVPqJLIl\n7G1kC9zbmcCFafvK6XVPfIORC4jOJU+26Ky1qn4uXyTPKv1yXY7NyK7SN5KBZa2PjAd2TPPIA5fe\nqftH1J3Lu8lWuMvpOGQMlH98/Z6eQHa9PJYcQH0mOfau04sYd/1HtiYd2+WOuG/evQHlJ/aVdVhd\np3u/FtLVtmAqtv97kdfJ+iJwQ6PPq5MxiQPznEtHv6wwyrx3JXsofgp8rT426a2Ik/oZtK7AdPgj\nz8iZ8NXOhyhnEzr4WYg6r8Ej/Nvrjvir5Kne7yQHO76I7O4Z1wD0demPDo/wyRMprqHvjB+ym+0r\ndQf5YzocvFvnvxF5NH8ZeUX0rq5ZtBU5FqV3ttyGZLfop4EjRltXJljeU2pgOYwcz/SJiYSjUeZ/\nDNma9wLywKV3Esr8+n+tWxIHvi+9K+tfXz/zXjnPJFvf9mESz9Aiu8a+TAawW8mxVwvJgHga2SWz\nzh88dbhe9X82m5MHjx+vn9FD+567tOvvZp3vpG//yQOKo8gW7PlT/Dl1OiZxCuu9Cdmz87jWdeni\nr9cyohkkIqLUDzYi5pFjE64upfygXivrbeQ1WD5ONvmuKGt5nZyZJiJ2J4PRV8iWpf3JADaXbHn5\nYinl0kkod0Ny0PmfSim3dTjfV5AbrU+XUr4TEU8mw8bPyauu/66rsmp5i8jwcDfw/FLKdzuc9wHk\niTQ/LqXsXx87mRzjc0rJ64pNtIzDycD3NjKIbUSG1y+VUu6JiCPJI/GbJlpWX5n939cdyIOmV5EH\nUc8lW2Yhhxf8oKtyp5uI2KOUcn1EHEf+1NUd5MHTJ8l1/DXkJS5+0rCaExIRG5a8ht1Ulvlw4FWl\nlCPrdc9ml7wm3Ib1/+allN9MZZ3Gq/87NF0ZxmaYgQ37yeTG/N7kNXI+Vkr5fd3hfIC8EvpVXezE\nZoqI2IxseXkOORbqBjKQ/ZwMNH9qV7vxi4gdydbPhWQ34hFkl+LryTFd10xCmduR13K9o+P5bkYG\nlT+Rp8zPI39W55iSF3qc6Px3IEP4ZaWU4+qFSV/HyJXEr+j6uzLwfV1Ojn37Fhk23lVKOSgidiW7\nsP4W+EDXAXo6iIhHkZebOJW8qOuHydbDq8mWxN+QZ/F2vj7PJAPr20allD9ExBbke/pPpZQz6nMv\nIn+x4hUzIehMB16Bf4bp+6IdTu6An0/uuB4C7BcRs0spnyGPvH9qEPtLpZTfllI+SA5qv5AcuPsS\n4NfTLYgBlFJuJcc5/T25wzqCkZ+UmpQWhFLKz7oOYnW+vyWX5fvk+nsg8IIuglid/23kdZUWR8RR\nJX8Z4c3k1cOfTF6GoFMD39cnAjeV/JWCjYD7RMQscuziV8iri6+PQWwOedLJbeSBxR5kF+Wh5JCC\ng8lfXTCIrcFAEDseeHf9RYRZ5EHNUyPizIh4GTlW8RyY+C9kaDi2jM1ALY7wZ5q6E3wYOT7l1FLK\nZxtXqRP1J6TeQV69etruvGq3LpPRpRMRh5Lv0TtKKZ+sP/ey1WQEzFre9uT39aullGfXda+Qg6b3\nJQ8IjiylXDcZ5a/LaovYIvLM1d+R70nvp3jOJM/6e3sppZOfiVsfRMQy8iD9FLK18RPk+/sz8rqH\nvwZWOnRlahnGZqiIeDrwQfIaUr0dyrvILp43llLualrBaSAiNiUv7vjDmdJUX3f8c0opP2pdl3VZ\nRCwmT5t/eSnlnzue91+tSxFxGHlyxYpSyjn1sQ3Jayn9pMtxhNNJ7WZfRHZFfog8CeWOUsqFdezY\nf67P4+iG4ZjE6cEwNoNN9RG+NJNExEHkDw1P1mD9p5GXmLm+lPKtWt67gff1AplSROxFbsvuTV4u\nYbfGVZoWHJM4fThmbAYrpVxENkW/KyKeWUq5xyAmDaeUclmXQazOs7djPIm8RMUWwBcj4tBSymXk\nz9G8OSKe02W5013tUn8B2Tr264iY37I+04VjEqcPW8bWA5NxhC9p7UTEI8hWiEOBo8kLbm5CdlFe\nUC/hcYvf19G1uPzDdOaYxOnBMCZJk2g1Y8S2Bx4NnFRKeWJEvJy8lMrBpZT/aFFPzQyOSZyeZreu\ngCTNVBGxcb1EBhGxN/nbp18vpdxer8f27TrpreTFZVc1qqpmgDWMSfxcRDyfvJwFpZRzauvi11vW\nVyMMY5I0CSKid22/j5E/BP1/gNsj4lellMPJa2QdEBGfAnYjrx5/c7MKa9obGJN4NHmtsC9GxJGl\nlItqC+xHIuLuUsonGlZVAwxjkjQ57g8sJseDPQrYt5Ty64j4akScX/LnZ/4/8jcA32QQUxfqmMRn\nkxdFPhr4EfCBiNi0jkk8mryIrtYhjhmTpA5FxAa9X2uI/G3QPYEdyPFhq+rjVwK/KqUsbldTzQSO\nSZwZvLSFJHWoL4gtA/YB/oP8Kar9I2KnOs1+wJzefWlt1DGJva7JvSN/9JtSyu2AYxKnEbspJalj\nEfFU4CTg0FLKf0fEb8iuo4iIK0opPyylPKltLTWdOSZxZjGMSVL37gd8sgax2fVstv8ld5q/i4hb\ngP+dCT+xpWYckziD2E0pSd37EfC4iHhQKeWe+tgGwC+AK+qvYRjENG4RsQFAKeVzwJeBvYCtgG3r\n448E5kfEv5VSvl1K+WAp5YZmFdZQDGOS1L0vA1cBL4iIwyLiecCbyN8D/Enbqmk6c0zizOTZlJI0\nCeoZbUuApwJ3Au8opVzbtlaaCeqYxLczMibxMHJM4hVky+sPm1ZQ4+aYMUmaBPWMtjMj4ux6/+7G\nVdLM4ZjEGcZuSkmaRKWUuw1i6phjEmcYuyklSZpGImJz4FVk79aXgS2Bk4EjSyk3tayb1o5hTJKk\nacYxiTOLYUySpGkqIuaAYxKnO8OYJElSQw7glyRJasgwJkmS1JBhTJIkqSHDmCRJUkOGMUmSpIYM\nY5IkSQ0ZxiRJkhr6/wFqOl6/pu+qKAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e4a0dc748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import string\n",
    "\n",
    "stop_words = set(['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', 'your', 'yours',\n",
    "'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', 'her', 'hers',\n",
    "'herself', 'it', 'its', 'itself', 'they', 'them', 'their', 'theirs', 'themselves',\n",
    "'what', 'which', 'who', 'whom', 'this', 'that', 'these', 'those', 'am', 'is', 'are',\n",
    "'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does',\n",
    "'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until',\n",
    "'while', 'of', 'at', 'by', 'for', 'with', 'about', 'against', 'between', 'into',\n",
    "'through', 'during', 'before', 'after', 'above', 'below', 'to', 'from', 'up', 'down',\n",
    "'in', 'out', 'on', 'off', 'over', 'under', 'again', 'further', 'then', 'once', 'here',\n",
    "'there', 'when', 'where', 'why', 'how', 'all', 'any', 'both', 'each', 'few', 'more',\n",
    "'most', 'other', 'some', 'such', 'no', 'nor', 'not', 'only', 'own', 'same', 'so',\n",
    "'than', 'too', 'very', 's', 't', 'can', 'will', 'just', 'don', 'should', 'now', '\\n', 'the'] + list(string.punctuation))\n",
    "\n",
    "def filter_dataset(data):\n",
    "    \"\"\"\n",
    "    Removes stop words from a dataset of (x,y) pairs.\n",
    "    \"\"\"\n",
    "    return [([w for w in x if w not in stop_words],y) for x,y in data]\n",
    "\n",
    "train_filtered = filter_dataset(train)\n",
    "dev_filtered = filter_dataset(dev)\n",
    "test_filtered = filter_dataset(test)\n",
    "\n",
    "theta_filtered = (alpha, beta) = train_nb(train_filtered,pseudo_count=0.01)\n",
    "plot_top_k(alpha,'pos',k=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point we see some words that may indicate positive movie reviews (\"good\", \"like\"), but also words that are generally likely to appear in movie reviews, regardless of the review's sentiment. Let us also look at the conditional probabilities for negative sentiment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmMAAAGLCAYAAACLN5UPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8HXV5+PHPQ0JAQFmDIltQg4gLihHRqqgIErbgDi4g\nCggSLYpLqi1uVVBstSiS0opKqyJWLFGwQHGpGwpqoYIgkWIBsaJV/LVakfr8/ni+xxyuN7nn3nPu\nndzk83697uuemTMz3+8sZ+b5LjMTmYkkSZK6sUHXGZAkSVqfGYxJkiR1yGBMkiSpQwZjkiRJHTIY\nkyRJ6pDBmCRJUocMxiRJkjpkMCZJktQhgzFJkqQOze06A5OxzTbb5IIFC7rOhiRJ0oS+9a1v/TQz\n50803awKxhYsWMBVV13VdTYkSZImFBE/HGQ6myklSZI6ZDAmSZLUIYMxSZKkDhmMSZIkdchgTJIk\nqUMGY5IkSR0aKBiLiAMi4oaIWBkRy8b5PiLijPb9NRGxZ99350TETyLiu2PmOT0irm/Tfzoithh+\ndSRJkmaXCYOxiJgDnAksBnYHjoiI3cdMthhY2P6OA87q++7DwAHjLPoy4GGZ+Qjg+8CfTDbzkiRJ\ns90gNWN7ASsz86bMvAs4D1gyZpolwLlZrgC2iIjtADLzX4D/GrvQzLw0M+9ug1cAO0x1JSRJkmar\nQYKx7YFb+oZvbeMmO82avAT43CSmlyRJWid03oE/It4I3A18dDXfHxcRV0XEVXfcccfMZk6SJGma\nDRKM3Qbs2De8Qxs32Wn+QES8GDgYeEFm5njTZObZmbkoMxfNnz/huzYlSZJmlUGCsSuBhRGxS0TM\nAw4HVoyZZgVwZLurcm/gzsy8fU0LjYgDgNcBh2bmr6aQd0mSpFlvwmCsdbJfClwCfA84PzOvjYjj\nI+L4NtnFwE3ASuBvgJf35o+IjwNfBx4cEbdGxEvbV+8H7g1cFhH/GhHLR7VSkiRJs0WspnVwrbRo\n0aK86qqrpjWNBcsuGvkybz7toJEvU5Ikrd0i4luZuWii6TrvwC9JkrQ+MxiTJEnqkMGYJElShwzG\nJEmSOmQwJkmS1CGDMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQw\nJkmS1CGDMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGD\nMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGDMUmSpA4Z\njEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGDMUmSpA4ZjEmSJHXI\nYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOjRQMBYRB0TEDRGxMiKWjfN9RMQZ7ftrImLP\nvu/OiYifRMR3x8yzVURcFhE3tv9bDr86kiRJs8uEwVhEzAHOBBYDuwNHRMTuYyZbDCxsf8cBZ/V9\n92HggHEWvQy4PDMXApe3YUmSpPXKIDVjewErM/OmzLwLOA9YMmaaJcC5Wa4AtoiI7QAy81+A/xpn\nuUuAj7TPHwEOm8oKSJIkzWaDBGPbA7f0Dd/axk12mrHum5m3t88/Bu473kQRcVxEXBURV91xxx0D\nZFeSJGn2WCs68GdmArma787OzEWZuWj+/PkznDNJkqTpNUgwdhuwY9/wDm3cZKcZ6z97TZnt/08G\nyIskSdI6ZZBg7EpgYUTsEhHzgMOBFWOmWQEc2e6q3Bu4s68JcnVWAEe1z0cBF04i35IkSeuECYOx\nzLwbWApcAnwPOD8zr42I4yPi+DbZxcBNwErgb4CX9+aPiI8DXwceHBG3RsRL21enAftFxI3A09qw\nJEnSemXuIBNl5sVUwNU/bnnf5wROXM28R6xm/M+AfQfOqSRJ0jporejAL0mStL4yGJMkSeqQwZgk\nSVKHDMYkSZI6ZDAmSZLUIYMxSZKkDhmMSZIkdchgTJIkqUMGY5IkSR0yGJMkSeqQwZgkSVKHDMYk\nSZI6ZDAmSZLUIYMxSZKkDhmMSZIkdchgTJIkqUNzu87A+mrBsotGurybTztopMuTJEkzw5oxSZKk\nDhmMSZIkdchgTJIkqUMGY5IkSR0yGJMkSeqQwZgkSVKHDMYkSZI6ZDAmSZLUIYMxSZKkDhmMSZIk\ndchgTJIkqUMGY5IkSR0yGJMkSeqQwZgkSVKHDMYkSZI6ZDAmSZLUIYMxSZKkDhmMSZIkdchgTJIk\nqUMGY5IkSR0yGJMkSeqQwZgkSVKHDMYkSZI6ZDAmSZLUIYMxSZKkDhmMSZIkdWigYCwiDoiIGyJi\nZUQsG+f7iIgz2vfXRMSeE80bEY+MiCsi4l8j4qqI2Gs0qyRJkjR7TBiMRcQc4ExgMbA7cERE7D5m\nssXAwvZ3HHDWAPO+C3hLZj4SOKUNS5IkrVcGqRnbC1iZmTdl5l3AecCSMdMsAc7NcgWwRURsN8G8\nCdynfd4c+NGQ6yJJkjTrzB1gmu2BW/qGbwUeO8A0208w70nAJRHxbioofPx4iUfEcVRtGzvttNMA\n2ZUkSZo9uuzAfwLwqszcEXgV8MHxJsrMszNzUWYumj9//oxmUJIkaboNEozdBuzYN7xDGzfINGua\n9yjggvb5k1STpiRJ0nplkGDsSmBhROwSEfOAw4EVY6ZZARzZ7qrcG7gzM2+fYN4fAfu0z08Fbhxy\nXSRJkmadCfuMZebdEbEUuASYA5yTmddGxPHt++XAxcCBwErgV8DRa5q3LfpY4K8iYi7wv7R+YZIk\nSeuTQTrwk5kXUwFX/7jlfZ8TOHHQedv4rwCPnkxmJUmS1jU+gV+SJKlDA9WMafZasOyikS7v5tMO\nGunyJEla31kzJkmS1CGDMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmS\nOmQwJkmS1CGDMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS\n1CGDMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGDMUmS\npA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGDMUmSpA4ZjEmS\nJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGDMUmSpA4NFIxFxAERcUNE\nrIyIZeN8HxFxRvv+mojYc5B5I+IVEXF9RFwbEe8afnUkSZJml7kTTRARc4Azgf2AW4ErI2JFZl7X\nN9liYGH7eyxwFvDYNc0bEU8BlgB7ZOZvImLbUa6YJEnSbDBIzdhewMrMvCkz7wLOo4KofkuAc7Nc\nAWwREdtNMO8JwGmZ+RuAzPzJCNZHkiRpVhkkGNseuKVv+NY2bpBp1jTvrsATI+IbEfGliHjMeIlH\nxHERcVVEXHXHHXcMkF1JkqTZo8sO/HOBrYC9gdcC50dEjJ0oM8/OzEWZuWj+/PkznUdJkqRpNWGf\nMeA2YMe+4R3auEGm2XAN894KXJCZCXwzIn4HbANY/SVJktYbg9SMXQksjIhdImIecDiwYsw0K4Aj\n212VewN3ZubtE8z7j8BTACJiV2Ae8NOh10iSJGkWmbBmLDPvjoilwCXAHOCczLw2Io5v3y8HLgYO\nBFYCvwKOXtO8bdHnAOdExHeBu4CjWi2ZJEnSemOQZkoy82Iq4Ooft7zvcwInDjpvG38X8MLJZFaS\nJGld4xP4JUmSOmQwJkmS1CGDMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzG\nJEmSOmQwJkmS1KGBXockrcmCZReNfJk3n3bQyJcpSdLayGBMs8aogz4DPknS2sBmSkmSpA4ZjEmS\nJHXIYEySJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGDMUmSpA4ZjEmSJHXIYEyS\nJKlDBmOSJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGDMUmSpA4ZjEmSJHXIYEySJKlDBmOS\nJEkdMhiTJEnqkMGYJElShwzGJEmSOmQwJkmS1CGDMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdMhiT\nJEnqkMGYJElShwzGJEmSOjRQMBYRB0TEDRGxMiKWjfN9RMQZ7ftrImLPScx7ckRkRGwz3KpIkiTN\nPhMGYxExBzgTWAzsDhwREbuPmWwxsLD9HQecNci8EbEjsD/wH0OviSRJ0iw0SM3YXsDKzLwpM+8C\nzgOWjJlmCXBuliuALSJiuwHmfQ/wOiCHXRFJkqTZaO4A02wP3NI3fCvw2AGm2X5N80bEEuC2zLw6\nIlabeEQcR9W2sdNOOw2QXWnqFiy7aOTLvPm0g0a+TEnSuqOTDvwRsQnwBuCUiabNzLMzc1FmLpo/\nf/70Z06SJGkGDRKM3Qbs2De8Qxs3yDSrG/9AYBfg6oi4uY3/dkTcbzKZlyRJmu0GCcauBBZGxC4R\nMQ84HFgxZpoVwJHtrsq9gTsz8/bVzZuZ/5aZ22bmgsxcQDVf7pmZPx7VikmSJM0GE/YZy8y7I2Ip\ncAkwBzgnM6+NiOPb98uBi4EDgZXAr4Cj1zTvtKyJJEnSLDRIB34y82Iq4Ooft7zvcwInDjrvONMs\nGCQfkiRJ6xqfwC9JktQhgzFJkqQOGYxJkiR1yGBMkiSpQwZjkiRJHTIYkyRJ6pDBmCRJUocMxiRJ\nkjpkMCZJktShgZ7AL2m0Fiy7aOTLvPm0g0a+TEnS9LNmTJIkqUMGY5IkSR0yGJMkSeqQfcakddio\n+6bZL02SRs+aMUmSpA4ZjEmSJHXIYEySJKlDBmOSJEkdsgO/pKHM1ANsvRlB0rrKmjFJkqQOWTMm\nSX2sgZM00wzGJGmG+W5SSf1sppQkSeqQwZgkSVKHDMYkSZI6ZDAmSZLUIYMxSZKkDhmMSZIkdchH\nW0jSOmomnpnmYzqk4VkzJkmS1CFrxiRJaz3fjKB1mTVjkiRJHTIYkyRJ6pDNlJIkNTaHqgsGY5Ik\nzSDvQNVYBmOSJK2DrOWbPQzGJEnSlMxULd+6XptoB35JkqQOGYxJkiR1yGBMkiSpQwZjkiRJHTIY\nkyRJ6pDBmCRJUocGCsYi4oCIuCEiVkbEsnG+j4g4o31/TUTsOdG8EXF6RFzfpv90RGwxmlWSJEma\nPSYMxiJiDnAmsBjYHTgiInYfM9liYGH7Ow44a4B5LwMelpmPAL4P/MnQayNJkjTLDFIzthewMjNv\nysy7gPOAJWOmWQKcm+UKYIuI2G5N82bmpZl5d5v/CmCHEayPJEnSrDJIMLY9cEvf8K1t3CDTDDIv\nwEuAzw2QF0mSpHVK5x34I+KNwN3AR1fz/XERcVVEXHXHHXfMbOYkSZKm2SDB2G3Ajn3DO7Rxg0yz\nxnkj4sXAwcALMjPHSzwzz87MRZm5aP78+QNkV5IkafYYJBi7ElgYEbtExDzgcGDFmGlWAEe2uyr3\nBu7MzNvXNG9EHAC8Djg0M381ovWRJEmaVeZONEFm3h0RS4FLgDnAOZl5bUQc375fDlwMHAisBH4F\nHL2medui3w9sBFwWEQBXZObxo1w5SZKktd2EwRhAZl5MBVz945b3fU7gxEHnbeMfNKmcSpIkrYM6\n78AvSZK0PjMYkyRJ6pDBmCRJUocMxiRJkjpkMCZJktQhgzFJkqQOGYxJkiR1yGBMkiSpQwZjkiRJ\nHTIYkyRJ6pDBmCRJUocMxiRJkjpkMCZJktQhgzFJkqQOGYxJkiR1yGBMkiSpQwZjkiRJHTIYkyRJ\n6pDBmCRJUocMxiRJkjpkMCZJktQhgzFJkqQOGYxJkiR1yGBMkiSpQwZjkiRJHTIYkyRJ6pDBmCRJ\nUocMxiRJkjpkMCZJktQhgzFJkqQOGYxJkiR1yGBMkiSpQwZjkiRJHTIYkyRJ6pDBmCRJUocMxiRJ\nkjpkMCZJktQhgzFJkqQOGYxJkiR1yGBMkiSpQwZjkiRJHTIYkyRJ6pDBmCRJUocMxiRJkjo0UDAW\nEQdExA0RsTIilo3zfUTEGe37ayJiz4nmjYitIuKyiLix/d9yNKskSZI0e0wYjEXEHOBMYDGwO3BE\nROw+ZrLFwML2dxxw1gDzLgMuz8yFwOVtWJIkab0ySM3YXsDKzLwpM+8CzgOWjJlmCXBuliuALSJi\nuwnmXQJ8pH3+CHDYkOsiSZI060RmrnmCiGcDB2TmMW34RcBjM3Np3zSfBU7LzK+04cuB1wMLVjdv\nRPwiM7do4wP4eW94TPrHUbVtAA8GbhhifUdtG+Cn60AaM5XOurQuM5XOurQuM5WO67J+p+O6rN/p\nzNS6DGrnzJw/0URzZyInE8nMjIhxo8LMPBs4e4azNJCIuCozF832NGYqnXVpXWYqnXVpXWYqHddl\n/U7HdVm/05mpdRm1QZopbwN27BveoY0bZJo1zfufrSmT9v8ng2dbkiRp3TBIMHYlsDAidomIecDh\nwIox06wAjmx3Ve4N3JmZt08w7wrgqPb5KODCIddFkiRp1pmwmTIz746IpcAlwBzgnMy8NiKOb98v\nBy4GDgRWAr8Cjl7TvG3RpwHnR8RLgR8Czx3pms2MmWg+nakmWtdl7UxnXVqXmUrHdVm/03Fd1u90\n1spuTROZsAO/JEmSpo9P4JckSeqQwZgkSVKHDMakWSQi5rb//nYnoT3LcNYsV/fkdta6zhO61jnr\n6ok7IrYGLoqIB2Tm70YdkK2r263ZBKYliH3AiJfXmfb6ut7nTp9B2X8sRsTGOWTn5nX82L6Hqa5r\nRGzW93nr0eVIgzAYm6KuftyrS3c2nGzGy+N0BBS9E3dE3GuUy55sPqZhsT+nHhfzkYjYYZQB2Zjt\ntkFEbNj7PIrl99KIiI3b5xnbNxGxAPhqRDxixNtsM+DvIuKdo1hel9r+PjYidoqIPYC39wdnM63v\nWFwKfGAE++z+w+dqlbX1fDvmdzx/0O3WfpeHRMRhEfF86ljYaLrzOp3LX02aO890moPybsohRcQ+\nwKaZefEMpNX/Q3s+8Btgk8z8uxGmMR/4bWb+YlTLbMvtz/sBwO+AH2Xmd6cpjVcDjwKOyczfjCqN\nCdJ/FPWYlv/NzF9FxAaZ+bsRLTvamyoeCrwT2BR4UWbeOuJ0Xgk8DLg3cHZmfmFEyw3gycBvgZ2B\nJwAnTfe+6dturwGeDRyVmTcMu81680fEA4C/A1Zk5rQGZa224peZ+duIeAGwJfD9zLy0/9gfYvn7\nA/8E/Dv1Grsbh8/1UPnZD3gt8LzM/PkQy9ka+CBwSmZeM6r8tWU/G9gN+B7wrcy8eZTLn6qIOBl4\nEPCnmfmzAaafAzwcOB/YCHh4Zv4yIuZk5v9NQ/76z9XPbKPvzMzLR51WX5pbA28H/qLrY3s81oxN\n0pjq86XAGcDbIuKciNhx9XMOb0xp8eXAr4HlETGSl6y3C/GHgfdHxKmjWGZPX95fDrwJ2BW4OiJ2\nn4Y0TgSeSZ18fzPdJbyW5supBxn/BfBnETF/lLUwLaA4hLrw/wvwS+AfImLnUaUTEccChwJvAbYF\nXjzsMnvavtkAOB34c+CSmQrE2uBVwM+oWsXdht1mfYHcHsA1wAkR8Ybhcrx6Leh7B/D4iDgKOIVq\nev2HiHh+Oz6m2jwVLbi8lDq2tqWeC9mrMZtxEbEtcDCwO7BVL59DLPJWoPcu5FHVjL6MOpf9N/Xg\n8mMi4jGjWPYwWr6WAG/OzJ9FxCZRD13/g23YG24B18+AXwDXA4f0jR+5vnP1CcBrgHnAZRHxR6NM\nZ8z6BnVsr5WvSjIYm4Qx0fw8qvZg78x8NHUwLYuIHaYh3R17zTqt5upJwL5UiewLwGeGDTgi4nDq\nB/xC4H+oUtLItBP+Q6iHA+8H/B/wJeD63g9miIvJgyPizX2jdqJK1Fu3IOmyiHheRGw0HVXjEXEo\nsD1V2/NBIKmAbJt20R9VmgcCp2bmu6i3WVwCfDgidhxRzdimwIuoBzD/BnhpRMxrF8Yp6zvhXw5c\nR9W8/HbY5U6k77d6ElWb+KmW/sci4qHDBmQRcWRb7keANwOLxxyHI5OZN1EvPz4CeDzw3HYcHEYV\nyA6fSs1Y3zlt67af9geeB3wjIvZptXC7RcSmo1ub8fPR93leZv4E+EuqgLO0FTomFXBGNU/Taoau\nBN7X+02OIL9zqP3w4sx8L/C69tVIg4kB89K/7eZSQeeHgftHtRB8jDofbdZ/jIy5ni2lzmH7UwHm\nc9q5k4hYFBG7jDKvUV0hdm7pLQY2Bz4PXDGiguW83vpFNb1vk5k/Bd4PHB8RDxo2jVEzGBtQKzn2\nDtzXABcBRwIHtUmOBe4DvCMith9huttTF9657Yf2c+otB++kLv7PbaWXoyPi0UMk9Wvg9dQ6LQCe\n0dJ/1BDL/L227X4EfA14IxX4HdhOjMdGxHZTvZhQNQTzI+JtbfT/tjROpZrFPkcFMkwljTWkvUFE\nbAmcBzwuM38IfB34NFVaflc7CQydZjtBzaOaEAHuaulsQ73JYuNJXqjGm3Z7KkDeKzMPzMy7gZdS\nx9aU+g/1nRB3i4idqP3yJ1TQv3+b5gERMbI+PRGxa1T3gZ6HUk2i51C/008BZ0fEQ4a8MN8LeGdm\nXkHVWL6auoidMsQy76EVYnrB7Bup5rCHAk9sF9fPUwHZx6KazCa17L4a148Bn6FqKa6kanoubhfp\n84GFo1qn8YzpXnBmRFwAbEYFuv9DBWQPGPS3FNXv7dSIuDQi9gQuBc6htt1QtWMRsRd1zvkp8MKo\nGwyuB/4ZOHi6A9ex+rbdK6mnz/+Eqtl6L3WtuJDalqub71VULfgvs7qnfBs4iypcXEAFxf87yrxS\n57Lbge9StbyHAIe0a9mJEfHgqaYREVtRheJFUe+9PhH4p4h4EvCf1BuDRlpLOhKZ6d8k/qiSz2ep\n2qk3UCW3/dt3mwB/A9xvxGneG3gEsLQN/zlVs7RJG34+8C1ghyHSeCFVRf3ZvnHHUCWJe01heVv2\nfT6caqu/D3AZ8MMx330NuP8U0pjX93kv6kfX6wf5SGDz9nl/KsjYesT7ZV77v5A6MZ/c993jqRLm\nfae47N567AHsSdX27UTV7JzQl8ZfAo8acr+/gGoO2rTti/e3714KXAs8eMjttBj4N+DPqCaQe1O1\nL+dSzZY/pQLAUeyTjalm1r8CntTGnQec1jfNo4F/BS6nLgox6P4YM+5lbfv0joMNgb8HvjqKY60/\nTSqIPKZ9fj3wAeBpvd8m8MRB9xOwQd/nXds+eRRVYHkN8IH23WHt97//KPbNAPl6UTs/zAW+D7y7\njd+z7c+3A3Mn2l7tXPB1YDsq8H8b8BWqOfmDQ+ZxK+B9wAFtm/8FcFz77lDgH6k+xNO+vcbka7+2\n7TZuw/cHNuvbj1cB24xzXG1FXc92pM7PzwaOb+eD3YB3AQ8ZQf627ft8GPDXVGXQx4Cf9n33XOCb\nwI5TTGej9v9U4JPtuL5XO7bOoIK0n7b9NGem99Ma8951Btb2v3ZAPqt9fgwV9Ly1DW/ZTpIXAgdP\nYx42oS6YH6QunvPaSfJb7aR8JfCwKSz3aKp55Ylt+G3Ad4CHAH/cTl4PncJyFwB/C+zbho8CXtY+\nbwPcSAWtZ7eTxMOnkMZ9qCabbYCnUwHpvDHTzG0nlmumksYE6b8EWA68EtiFVQHZq/um2XjINA5q\n++PcdqI9ieqUeyPwIaofzIGTXGb/ifgwKrj763ZsPQ/YmqpF+Huq2WD3IddhQTu5PrDto+8CW7Tv\n9m7H4L4j2ie9i/FCKvB7N1UTsmM7Bpa1759PvRt3oALAmG32bKqQskcbfjfwjbZfjgY+Dswf8bH2\nypbGw/vG/SlwJhVADVxYomo//4R2caT6z/QXwB5CBa9PbsMbjXJdxuRlzpjhk6jC7iup2uzehXWD\ntk+3HWCZT6Mu8Af2jduAKricRRU2Dplifh/S/h8DnNU+P7Pt88vbcb7HdG2vNeRrWypYvRVY2Dd+\nU+p68T3aeXzMsfxCqkbsE9S5+JPUeflLtIL/iPK3A3U9eG4bPhR4Yy8/VOD8Kapp9ZtM8VxNnbtO\nBx7fhk+hrs2PasObU+eh5VQw1rs+TVgYm5H92HUG1va/doI4CzioDb+znSge0LeDX9EO6E1HuWPb\ngboPdVGM9sP/AHVHGFQz5eOBBVNY9pJ24L+Pqr5/JVWr8Kfth/NxplAiomo97g0sawf946imz2P7\nptm8rctzettxkmnMbf9fRJXqr2NV7USMmfYo4EEjPiaOozo6P4mqYTmljX8IdZfoK0aQxr3bSfEJ\nbfhBLc1nUIWA3ZhkoDzmRLw98Cpgtzb83HYifkZ/HoZch+2oQPV4VpV4H9S+ezqt5D5e/kaw7e5N\n1Uz+JVU6fiAVkH0cWDnZbdeWexLw5bbcL7T12oQqxJzf9tdIL8ZU89LHgZ3a8MZ93/VqADeZ5D75\nMvBWqqlm07Yu/b/P9wInjnKfjJOPzYGnts/7tn10UsvLp4AN23d/Brx9Est9QfsN9gp/c2k1ge2Y\nOBl46RTy+0Sqtu4dVGH40r7f/QZUTdI207GtxslL/++4d97bmbo2vAfYuY27F1Wg+4NzLBW0frF9\n/iMqKNu1DR9LFQAHqjUeIL/3o1ot/rrlZwnwyjHT7NfytGCIdDZs6/9+Wk071S3iM+34mtO3Xd4K\nvHYm9tfA+e86A2vrH1Vq342qgXlFO9D3a9+9F/go8MA2fB9aaX8E6Ub///b5PODI9vlIqrr1BMZc\nzCaRxhLgCmC7NvzMtk5L+37cq20OWF2+qVqQ89rw/agmj/cBF1B9P/ZoP7h96WvGnGQ684EL2+en\nUX0ALqE1DdPXBDONx8afUxe1o1jVrNLbbrv2TmpDprEp1S9xQd+4I+hrbpvk8vqbpk6m+rf8iFUX\nrS2ogOk8VgX7kz4R9x2/j2gn39cCPwD+g1VB9N7URfeB07BvHk6VeveiLvhvopqSdmsn660YoIZl\nnG32aKpTXh/XAAAUr0lEQVTAtUE7rr/d1u9lfes8VE3oavKwYdtWx4wZ36uZ22oSy+pt/23aNnoT\ncF8qwF/ezgFPAm6g+kBO529oAdXH7p+B77Rxu1E1wcdTD9N9AXA1a6id7dv22/b9Bp9N9XH6o956\n9033dqpZ7vfjBtlu7fexgqp9eh11/v0KI2pen+I2fDVVgLqAqgF+LHVuOp1V16beeu8NLGqf/4g6\nt5w5zjKPpmqvh6oR70+7fb4fVYh9PxUcXUpVNLyAuv4sGDKtXsA9r+3j5dwzILuAqgXuBWQn02pf\nBz0Opvtv7em8thZpHWE/Q3WSnks1B1wHHBYR+2XmScCPgfdExC6Z2ev4OLRsRwp1kuz5JHVxIzPP\npX4sO7e8TUpEbELlfTfaowsy8wLgi1Q/q+Nap8bJ3tI8J+sZO0sj4ilUzcuZ1EV4e+okv5g60b6a\nqt2ZtMy8AzgiIp5MNTnsSp1c/zoiHpZ1h9xDYhoeZxER+7QO+z+jttfzM3O/rI7ux0Y9YuD7mfn9\nKSy7d5fR/SNio8z8H6pp6hPRHpRK3aX5gN6dQpNZfraO6hFxIPW8r2dRTdSviogntOP3MqqG55I2\nT467sDWn0+sQ/j4q+N6DqunYGDipdQg/C3hvZv5gsssfa5zt8CuqJnkp1bT1Xqov5KuBx2Tmf2Xd\nqTfIuvS22fbUHaBvpGqjD6GCvdupgtrL229mZI/qiLr79/WZ+VvqgrsgIp7WvjsCeG9EbJ2Z/zXo\nMjPz7nZjw97UPtmPuhBeS+2Trana6tdm5tdHtS79eh2m27liQ6rG6ctt3PXU9nwCFSg+H3hBZl63\nhnXqHW/nAhdGPV7k81Rf1Auj7gi9myosQj0S5g2Zeffqju/2+9q9fd6fajV4IFVjdDZ1U1BQrRLP\n6qITeES8iDqfnkDd1HN0Zn6DCjrmAS+JVa9O613P7oiI+1LdW64DtouIRX3LfBDVkvHcNW3zAfPX\nf6fmxpn5Y6pA/m9Ube8CquD0VKowMOVHqLS0fhcRm2bmXdR57cfU3eB7ZebbqWvmW6hCLtQ54fWZ\n+ZupnOemRdfR4Nr2R5UaL6IOlK2ovjMHU8/d6dWQ9dqa38EUOp6vJt1eCWYDqrnrdqoEvpg6UK8G\nXtg3/eZTSON4ql3+LVSfp+upH3Hv+4OZQofzts1uppXSqZLj9VQAuSHVHLacVjIb0fZ6BhXobd72\nzRvafnsLFbwOXGMwYHpbtX2/DxVgfJbWHEldNP6N4Tu6H0D1nziXqnndier38L12LFxPPYxzMst8\nHKtu/FhABVuX931/Qlt+r8loqFIiVYj4CquaP5e2ffI26oL/Glbd8DLKJv0lfZ8XtGP9o9SFaiuq\n0/ugNWKPBw5vn19BNUV/iOof9mbqYk77DZ096HInuT57URfMY6laopPbsfEJ+voADbis3rnl8W3e\nv6dqT75ENR2fRGuSZlXNwchrC7hnTckTqRaFw9o2fQftpgeqgBUMUPPPPW9AOKgdX73+XC+l7sTc\ngkl01qa6BFxK3SF7STuGz6dqQl/c91t5OkP+5ieRpyn1r2P869liqkboL9txsGffcid9s9YE+X4l\n1cz+QWCfNu446nowqXPZBMf206nz5olUk/EG1LnzA7RaXqahJn6k26rrDKxtf1S/gn+imms+Tj1c\n9W+pC+WcNnwurYPriNLsP0n17obpdQj+F6of17KWpykFGVRNyHep2q8zqJqCU6jal5NHsA6HtJPi\nlm14KXUR24Nqoz+F1r+FEZ3o2z65kVV3TR7XTjyT7g80YHrvAD7UPj+zndgup+6em/QNFGOW/QCq\nT8oTqQvM69pyN6NqK57dO5lNcrk7UU2qC9rw/lQpuf/Oz1dRpeV7DbtvqBrPr7GqiWjD9vu5kHYj\nzIj2xe8LL+3/N4Gv9n3/IOoCeilVKBh4vaiL+r9TAeTHqFqRfamA7stUn6SPtGNvtxEfYw9l1V1v\nvbs+j6WCkx2oppbtprDcvajmzr37ts8fU0Hml6m+sJPqmjDEOp7QjvX7t+F9qabkt1DB73sZsMmX\nKriOdwPCU9rwVO/KezdVi9a7g/V+VAHs+1TXiEnvgyG215T71zH+9eyDtDtxqWDsLIa4I3sN+T6R\nakHYhWoWvxk4tH33x1Tt+b1HcM7Zj6rdfWo7lj/Zxm1AnbP/lilUXsz0X+cZWBv/qH4u/82qEvAx\nVCf3p1MB2bGM+PEVLZ1XUv0Svggc0cbdmwqe/hG4gyn2TaNqjl7TPs+jag7ew6r+O1uM4EexmOof\ntGXf+lxF3Zq+ESOurWppHEiV9nu1ciO9+4u6+O3fN/xPwHPa52gn6fsMmcZ2VMm190iBXoBxJtVM\nM5VlbksLEKnah5+y6i7gxVQpv//Oz5H0eWzLOrldGHrpP526kH2EEZS8uWdfri36Pn+a1im5Db+Z\nuthM5bEp+1GFl4/2jivqQv/2tj2PYgQ3hoxZlwdSF8ZXsKqW6DHAndRrbYZJp/eg5d45bUPqrrZT\nqW4Eew6z/Enk41HtnLDjmPH7UP2dvsNqCjbt9/YcqobnyVQAuRlVYD2mb7r3sqo2eEo1fVSw+qKW\nnyP7xp9AtS5M+sajIbbZAoboX8f417MPUYHdJm27D127O+ZY3qjlbet2PriAulv7JlbdDDfUOacd\nD/dpx8HuVIB5NRXUn9+OkQ1G8Tudkf3cdQbWxj+qP9bTqJqe3h1TL6Ei7qdNU5rPpGqpdqVqfH4A\nvLx9N6cddFNuEqUuIBf2/1ipIOxBjLDjMX8YkPVqE6bzFvklVA3CBoywAz8VRL6Oai56LXUxPpr2\nnK8hl93f0X15S+c27tls/FamWGtJNVX0mlouoPrhfIFVd4A9nWrGW9qfnxFttx2oWqXPUcHLDVTN\nzApGeLchVRP6ybb9DmnjPk01v72Bajreacjj6ufUuxF741YwDY+xoboI7EfVyLyunXfmt+/OpG64\nGap039ZnJasKevtQNyKMLBAfIA+PZtVNPsGqWp3e/zUWbKjHMfyMOjf3avmeQRUu3sOIb0Cgavyv\nbtvuSe24ntKNU1NIuz+4eT3VJ/GMvnFPoGrnP0K1CKwuiF3d9ewfGGELz5jjbH+qln8jqnKhdyxf\n3n6XU34W29hzFVXDd7+27C2pvtTXUn0tZ+QO11H8TboD+Pog60nqP4x6GfcnqA6b57X/144ijfjD\nFxVvCHw+q/P396NeT3RBRFyRmd+mqsx/OUSSX6RqeV4QEV+kDuDNgF9k5kiergyQmZ9rnbS/HhGP\nz8x3RsTZOY3vIczMCyPi8hzRy7KB3vst96FKn5+m+h5tTV00t4iIb2d1mJ2SzN93PH4NdcLalAos\n3tqeGn09VXNx0hSXf2NEXEMFLG/MzK+0/XJ261v7toj4HXViJNtZbRSyXl5+OtVHaQ+qiXUTqoT/\n41Gk0bbdq6kagX2BfaPe4vCMiHg9Vft7RGb+x1TTaMfVi4Azol7l9a9Uk8v3RpD//g7Oh1O132dR\nF/xrqQLAqyPiZ9QdxM/NzDuHSbOtz++Aj0bEs6jm1jfniG4+GmvMOm7UzgErgYURcUJmnkW9Futl\nVCH0ZOD/rWZZvfPl56masEdSfUZpwze1+Q9nhDcgZOZnIuK31MNP76Zqqv97FMtek16n9Pb5idSx\ncQPwyIh4B/Wy669ExE+o5vJNV5evCa5nN4wor/3H8nup2sN9qT5b36RuFlhCBbbvzLpBacppRcRT\nqZqvG6ma1v+kHnS7MVVzehvwnqxXIM0OXUeDa/sfdTH5Ge12/2lY/mFUKeId1An59/12qAN56FuM\n+9K6P9WX61KqtmTaHlBIlY6+Q9VWrRW3Dk9yn3yZKmV9larG34xq+juF6l+1cMg0xnZ0P5GqCftz\nqp/S25niwyn70ug1tXyb1txJXdhvZgS1e5PIx1Oomp2RHG+s6rjcexPBJlSQ/DFWvZVilDV9h1FN\nfBcyguYp7tlHdCeq+ab3KIJnU89DO52qhb2cIfsjjpP+oVRg+dpefkb9Gx2zjse0c9ux1IXyCVSt\n6XKqv+JAD35u++E7VHPuK6ig+JHtux1Wl/6I1mc+I36Y74Dpjqx/XZt/pNezCY7l57dj+Xvt3PYD\n4BEjSPOgdk57IXUt672p4S1UIeY6+p6XOFv+Os/AbPijakVG0u485uA9nLpr8q1UE+UvqPbvg6jn\nF13HFDugTpCHTZiBV3YwQ9X505Dvo6hm4xe3H3vvpooF7f9QD0Nty1hdR/dPA88e73gZIq1D2knx\nYKq272MMGUxOMv3taA+inOL8/b+Z+1CFlI+238wj+r67lGnoiNyWvQ9DPgtpnHXpPVn/Oipg6R1n\nz6GeZ7Un09SpnioA3go8c5r3/fFUgeYJLb13UTX0O1FvQVjGAAVOqibsavruXmzHwNfbtvvRdO37\nrv4Yon/dBMsdyfVswGP5UOAWqil5FOfNHakC8Q5Us/6V/duHqmFdODZ/s+HPZsoBZOZ3R7GcMdW5\nO1HPjXpCZv4gIq6kfmBPpqpYn0ZdlG8ZRdr9MvNXo17matKZ9ur8aXIzdcPGjzLzifD7l/Du3J79\nNG5TymRk5s8j4lPAUyPizsz8bkR8kjqpHRIRF2Xmr3vHy5Bp9Te13AW8KDNvHHa5k0j/9iHn7/1m\nds/M6yLiW1Rz4Tep56R9nCpgbE4VbkYuM780ouX01uUwKih5EXXxejiwd0R8JTM/GfVi9l9kPSNr\n5DLz0og4mqqtGJkx57jtqRsQDqEKnj+kasaOB07NzGWTWPRvqODjyRHxPKo/0o+oPn07Ay/OzO+M\nbEXWDhsAKzPzlvY8vblZz537WmZ+KSLelZmT7royquvZgMfyivacxKunet7sa5p8AtV/8jtUP7mN\nqDu0b4mIg6jtc+HY/M0aXUeD68sfg5UiDqNKEY9jhm4z92/cfbUZ9Qyed1PB8ZFU0+Som4tmpKN7\nX3rb0kFTy4jy/jjqYv4y6uaEf6HuBHwW9bDaT03HNpumddme6u/0wTa8cTsO3keV9mflb3/MOW5p\n++1sQdXwXNbG70r173kdk3un5mZtmV+jaq13b8fCs5iBt27M8LbrPTNs83beOaHvu5dR/cXuMU+H\n+Z6RY5l6u8A/t/PA/agaw1e1755A9R17ctfbY5g/n8A/Q7L367lnKeKzrCpFzM3Mf6Tu2vvPnKYS\nsSaWVaN3OvUDfy1VS/niHFGJsi+dW1s6f0XdnPFsVr1WaiQd3cek95OsNxjMKhExjyqk3EZdjHan\nCjMHUU1X+1NvQ7i6s0xOQmbeRt2YsTgijsi6geYtVIfqp1M3H8w6Y85xTwVuyro5YCPgvq2274FU\n0+LfZeavJ7Hs/87M91MX3AuoG15eQdUejuzGnS6MqU08Bjg9Io6l7qL/Y+DQiFgeEa+i+t19GNaO\nmp8ZPJY3p/qe/lHW0/zfDTyntS6cRQVmXxxRWp2ItWB/rjdatf3XqVLiS1v17RtZ9d6zLxiErV0i\nYkOArOaB6U7rKVRtz8tmS2Ax3SLicdSjXs4Hfk11WO698mU59Syxt2fmZF/f1bnWtHIq1WT38fb6\nmi1nY8Dc0+4E/jrwjcx8XgvAktpve1FB1OGZOaW70tvyHkn1G3xH9jVLzXYRcTxVSH89dbfjx6jj\n/ifUw1p/AazIIV9VNB1m4lhud2OeTt14cmFEbEG1Lvw6q6tPrA0B6lQZjM2wiHgm9bLUk/sO2ndR\nt5mfkjPUn0trn3Yhm5d1K7qAiNiBCsaOpy7AGwJ3ZOYFEfFS6iGvI+33NJMiYjH1SqVXZ+Ynu87P\nZI13AYyIg6mbUZZl5ofbuA2pZ+r9uNWmDJPmptRDSv99Nl+Ax+lf91aqJv5w6pEtvUeonDobjvGZ\nOJZb0PdW6nlrH5mONLpiMNaBdbFELE2niNiD+s3cm+r3tlvHWRqZiNgP+EFm3tR1XiZjTDDxDOqx\nPNdl5r+2dTqdeiH8hzvM5lppzLZbSr3tofccu3dl5n4RsSv1iJ2/AN43mWbdrszEsdxqyE6lHvPx\n49kajI/l3ZQdyMyL2sMXz46Iu1spwkBMWo3MvDoiXkydgP84IhZk5s3d5mo0MvOyrvMwFX3BxInU\nTS4fBr4cEYe3c9yrgQ9FxF2Z+bEOs7rWGad/3YrM/EVEDN2/rkszcSy3JsqvrWuVF9aMdWi2loil\nLkXEhjPRh08Ti4jHUDU3B1EB2QnUY0aWZeb5EbEPcIvnuD803f3rNLsYjEmSBrKaPmLbUa++OjEz\nn9pqxN4N7J+Z/9xFPtdGXfSv0+xhM6UkaUIRsXF7dAER8Sjq+V7fyszbI2Jb2ntOqSftf5p6D6VY\nY/+6z0a9//T0iCAzP9xqfb/VZX418wzGJElrFBG95yH+PfAS6vlXt0fEzzPzMOp5b/tExCeA3YAl\n60qfvlGwf50mYjAmSZrIzsBiqj/Y44C9Wofzb0TEeZl5eET8P+q9iW8yEPtDrX/d86iHSB9JvVHi\nfRGxaetfdyT1cGOth+wzJkkaV0Rs0HvCfUScDDyUegXOiZm5so2/Avh5Zi7uLqdrH/vXaTJ8HZIk\naVx9gdjxwJ7U+wF/CTwxInZs0+wNzOsN6/f963pNk4+KiEcDZObt1Dti7V+ne7CZUpK0WhFxKHAi\ncFBm/kdE/JJqbouI+EJm/ntm7tttLtce9q/TVBiMSZLW5P7Ax1sgNrfdAfh/VKDx64i4Bfi/deVJ\n6CNg/zpNms2UkqQ1+SHwpIh4cGbe3cZtAPwM+EJm3m0gVv3rADLzs8BXgT2ALYFt2vjHAgsi4nOZ\n+W+Z+f7MvL6zDGutYjAmSVqTrwJXAi+OiIMj4oXAm6h3KP6426ytPexfp2F4N6UkaY3aXYBLgEOB\nO4FTM/OabnO19mn9697Oqv51B1P9675A1SL+e6cZ1FrLPmOSpDVqdwEuj4hz2vBdHWdpbWX/Ok2J\nzZSSpIFk5l0GYmtk/zpNic2UkiSNQETcB3gt1er0VWAL4JXA4Zl5U5d509rNYEySpBGxf52mwmBM\nkqQRi4h5YP86DcZgTJIkqUN24JckSeqQwZgkSVKHDMYkSZI6ZDAmSZLUIYMxSZKkDhmMSZIkdchg\nTJIkqUP/HzKocl1XGKN4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e746dc2b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_top_k(alpha,'neg', 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly, the negative reviews look fairly similar when considering the head of the distribution, and even mention \"good\" just as often as the positive reviews. However, note some subtle differences: in negative reviews the word \"movie\" is used more often than in positive reviews; for the word \"film\" this effect is reversed. \n",
    "\n",
    "Notably \"bad\" appears with high probability only in the negative reviews. Let's look at some occurences of \"good\" in negative reviews to check whether this actually makes sense.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['is',\n",
       " 'neither',\n",
       " 'bad',\n",
       " 'enough',\n",
       " 'nor',\n",
       " 'good',\n",
       " 'enough',\n",
       " 'to',\n",
       " 'be',\n",
       " 'enjoyable']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs_with_good = [x for x,y in train if 'good' in x and y=='neg']\n",
    "good_index = docs_with_good[1].index(\"good\")\n",
    "docs_with_good[1][good_index-5:good_index+5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generally, looking at the most likely words in each class is not that helpful in order to understand the NB classification model, as discriminative features or words may be rare when compared to general words in the domain (such as \"film\", \"movie\", \"story\" in movie reviews). We can instead look at probability *ratios* or *differences*: which words have much higher probability in one class than the other.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnIAAAGICAYAAADFzw13AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecXWWd+PHPlxQgiBSBGDpKEIIISkTEgiItIAQ7iFKW\nFRQillWMYkEFiatrQQUMLhJQRNYCKFGaYMECsYvIjyKsFAXdYlsL+Pz++D7XORlnMjP33snkTD7v\n1+u+5p4z95zn9PN92jlRSkGSJEnts8ZEL4AkSZK6YyAnSZLUUgZykiRJLWUgJ0mS1FIGcpIkSS1l\nICdJktRSfQnkImL/iLglIm6LiIVD/D8i4oz6/x9FxBNGmjYiNoyIqyLi1vp3g8b/3lh/f0tE7NeP\ndZAkSWqbngO5iJgCfASYB8wBDouIOYN+Ng+YXT/HAmeNYtqFwDWllNnANXWY+v9DgR2B/YEz63wk\nSZJWK/0okdsNuK2Uckcp5S/ARcD8Qb+ZD5xf0reB9SNi1gjTzgeW1O9LgEMa4y8qpfy5lPJz4LY6\nH0mSpNVKPwK5zYBfNIbvruNG85sVTTuzlHJf/f5LYOYY0pMkSZr0pk70AoxGKaVExJjfJRYRx5JV\nuayzzjq7br/99n1ftqYf3/O/fZ/nTputN+7prIw0VlY6Q6WxstKZTNusreuystKZ7OuystKZTNts\nMq3Lykpnsm2zfvvud7/761LKxiP9rh+B3D3AFo3hzeu40fxm2gqm/VVEzCql3FerYe8fQ3oAlFIW\nA4sB5s6dW5YtWzbaderK1gsv7/s8ly06cNzTWRlprKx0hkpjZaUzmbZZW9dlZaUz2ddlZaUzmbbZ\nZFqXlZXOZNtm/RYRd43md/2oWr0RmB0R20TEdLIjwmWDfnMZcETtvbo78L+12nRF014GHFm/Hwlc\n2hh/aESsGRHbkB0obujDekiSJLVKzyVypZQHI2IBcAUwBTi3lHJTRLy8/v9sYClwANkx4Y/A0Sua\nts56EXBxRBwD3AW8sE5zU0RcDPwUeBA4oZTyUK/rIUmS1DZ9aSNXSllKBmvNcWc3vhfghNFOW8f/\nBnjWMNOcBpzWwyJLkiS1nm92kCRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElqKQM5SZKkljKQ\nkyRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMkSWop\nAzlJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElqKQM5SZKk\nljKQkyRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMk\nSWopAzlJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElqKQM5\nSZKkljKQkyRJaikDOUmSpJYykJMkSWqpqRO9AJPJnYsOnOhFkCRJq5GeArmI2BD4NLA1cCfwwlLK\nfw/xu/2BDwJTgI+VUhaNNH1EvBE4BngIOLGUckUdfx0wC/i/Ovt9Syn397IebbMyAkaDUkmSVn29\nlsgtBK4ppSyKiIV1+A3NH0TEFOAjwD7A3cCNEXFZKeWnw00fEXOAQ4EdgU2BqyNiu1LKQ3W2h5dS\nlvW47FoFGDBKktS9XtvIzQeW1O9LgEOG+M1uwG2llDtKKX8BLqrTrWj6+cBFpZQ/l1J+DtxW5yNJ\nkqSq10BuZinlvvr9l8DMIX6zGfCLxvDdddyKpl/RNABLIuIHEfGWiIheVkCSJKmtRqxajYirgUcO\n8a+TmwOllBIRpdsFGcP0h5dS7omIdYHPAi8Fzh/qhxFxLHAswJZbbtntomkSsApXkjQZjRjIlVL2\nHu5/EfGriJhVSrkvImYBQ3U6uAfYojG8eR0HMNz0w05TSun8/V1EXEhWuQ4ZyJVSFgOLAebOndt1\nkClJkrQq6rWzw2XAkcCi+vfSIX5zIzA7IrYhg7FDgRePMP1lwIUR8T6ys8Ns4IaImAqsX0r5dURM\nA54NXN3jOkh9YamfJGll6zWQWwRcHBHHAHcBLwSIiE3Jx4wcUEp5MCIWAFeQjx85t5Ry04qmL6Xc\nFBEXAz8FHgROKKU8FBHrAFfUIG4KGcSd0+M6SK1iwChJ6ugpkCul/AZ41hDj7wUOaAwvBZaOdvr6\nv9OA0waN+wOway/LLGlkBouS1A6+2UHShDFglKTe+K5VSZKklrJETtKkZ8mfpMnKEjlJkqSWMpCT\nJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMkSWopHz8iSX2wsh5xMtnSkdQbS+QkSZJaykBOkiSp\npQzkJEmSWspATpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSppQzkJEmSWso3O0iSJsRkekuF\nb8LQRLFETpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSppQzkJEmSWspATpIkqaUM5CRJklrK\nQE6SJKmlDOQkSZJaykBOkiSppQzkJEmSWspATpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSp\npQzkJEmSWspATpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSppaZO9AJIkqTRuXPRgRO9CFrF\nWCInSZLUUgZykiRJLdVTIBcRG0bEVRFxa/27wTC/2z8ibomI2yJi4UjTR8QjIuLaiPh9RHx40Lx2\njYgf13mdERHRyzpIkiS1Va8lcguBa0ops4Fr6vByImIK8BFgHjAHOCwi5oww/Z+AtwCvGyLNs4CX\nAbPrZ/8e10GSJKmVeg3k5gNL6vclwCFD/GY34LZSyh2llL8AF9Xphp2+lPKHUso3yIDu7yJiFvDw\nUsq3SykFOH+YNCVJkia9XgO5maWU++r3XwIzh/jNZsAvGsN313GjnX7wvO4eZl7/ICKOjYhlEbHs\ngQceGGHWkiRJ7TLi40ci4mrgkUP86+TmQCmlRETpdkF6nX6YeS4GFgPMnTu3r/OWJEmaaCMGcqWU\nvYf7X0T8KiJmlVLuq9We9w/xs3uALRrDm9dxAKOZfvC8Nh9mXpIkSauVXqtWLwOOrN+PBC4d4jc3\nArMjYpuImA4cWqcb7fR/V6thfxsRu9feqkeMNI0kSdJk1WsgtwjYJyJuBfauw0TEphGxFKCU8iCw\nALgCuBm4uJRy04qmr/O4E3gfcFRE3N3o6Xo88DHgNuB24Es9roMkSVIr9fSKrlLKb4BnDTH+XuCA\nxvBSYOlop6//23qY8cuAx3a3xJIkSZOHb3aQJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMkSWop\nAzlJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElqKQM5SZKk\nljKQkyRJaikDOUmSpJYykJMkSWqpqRO9AJIkadVy56IDJ3oRNEqWyEmSJLWUgZwkSVJLGchJkiS1\nlIGcJElSSxnISZIktZSBnCRJUksZyEmSJLWUgZwkSVJLGchJkiS1lIGcJElSSxnISZIktZSBnCRJ\nUksZyEmSJLWUgZwkSVJLGchJkiS1lIGcJElSSxnISZIktZSBnCRJUksZyEmSJLWUgZwkSVJLGchJ\nkiS1lIGcJElSSxnISZIktZSBnCRJUksZyEmSJLWUgZwkSVJLGchJkiS1lIGcJElSS/UUyEXEhhFx\nVUTcWv9uMMzv9o+IWyLitohYONL0EfGIiLg2In4fER8eNK/r6rx+UD+b9LIOkiRJbdVridxC4JpS\nymzgmjq8nIiYAnwEmAfMAQ6LiDkjTP8n4C3A64ZJ9/BSyi71c3+P6yBJktRKvQZy84El9fsS4JAh\nfrMbcFsp5Y5Syl+Ai+p0w05fSvlDKeUbZEAnSZKkIfQayM0spdxXv/8SmDnEbzYDftEYvruOG+30\nQ1lSq1XfEhEx3I8i4tiIWBYRyx544IFRzlqSJKkdpo70g4i4GnjkEP86uTlQSikRUbpdkDFMf3gp\n5Z6IWBf4LPBS4Pxh5rkYWAwwd+7crpdNkiRpVTRiIFdK2Xu4/0XEryJiVinlvoiYBQzVXu0eYIvG\n8OZ1HMBoph+8PPfUv7+LiAvJqtshAzlJkqTJrNeq1cuAI+v3I4FLh/jNjcDsiNgmIqYDh9bpRjv9\n30XE1IjYqH6fBjwb+ElPayBJktRSI5bIjWARcHFEHAPcBbwQICI2BT5WSjmglPJgRCwArgCmAOeW\nUm5a0fR1HncCDwemR8QhwL71N1fUIG4KcDVwTo/rIEmS1Eo9BXKllN8Azxpi/L3AAY3hpcDS0U5f\n/7f1MMnu2s2ySpIkTTa+2UGSJKmlDOQkSZJaykBOkiSppQzkJEmSWspATpIkqaUM5CRJklrKQE6S\nJKmlDOQkSZJaykBOkiSppQzkJEmSWspATpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSppQzk\nJEmSWspATpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSppQzkJEmSWspATpIkqaUM5CRJklrK\nQE6SJKmlDOQkSZJaykBOkiSppQzkJEmSWspATpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSp\npQzkJEmSWspATpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSppQzkJEmSWspATpIkqaUM5CRJ\nklrKQE6SJKmlDOQkSZJaykBOkiSppQzkJEmSWqqnQC4iNoyIqyLi1vp3g2F+t39E3BIRt0XEwpGm\nj4h9IuK7EfHj+nevxjS71vG3RcQZERG9rIMkSVJb9VoitxC4ppQyG7imDi8nIqYAHwHmAXOAwyJi\nzgjT/xo4qJSyE3AkcEFjlmcBLwNm18/+Pa6DJElSK/UayM0HltTvS4BDhvjNbsBtpZQ7Sil/AS6q\n0w07fSnl+6WUe+v4m4C1I2LNiJgFPLyU8u1SSgHOHyZNSZKkSa/XQG5mKeW++v2XwMwhfrMZ8IvG\n8N113Ginfx7wvVLKn+t0dw8zr38QEcdGxLKIWPbAAw+MuDKSJEltMnWkH0TE1cAjh/jXyc2BUkqJ\niNLtggw1fUTsCLwb2LfLeS4GFgPMnTu362WTJElaFY0YyJVS9h7ufxHxq4iYVUq5r1Z73j/Ez+4B\ntmgMb17HAQw7fURsDnweOKKUcntjXpsPMy9JkqTVSq9Vq5eRnRGofy8d4jc3ArMjYpuImA4cWqcb\ndvqIWB+4HFhYSrm+M6NaDfvbiNi99lY9Ypg0JUmSJr1eA7lFwD4RcSuwdx0mIjaNiKUApZQHgQXA\nFcDNwMWllJtWNH39/bbAWyPiB/WzSf3f8cDHgNuA24Ev9bgOkiRJrTRi1eqKlFJ+AzxriPH3Agc0\nhpcCS8cw/anAqcOkuQx4bPdLLUmSNDn4ZgdJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmSpJYy\nkJMkSWopAzlJkqSWMpCTJElqKQM5SZKklurpzQ6SJEnduHPRgRO9CJOCJXKSJEktZSAnSZLUUgZy\nkiRJLWUgJ0mS1FIGcpIkSS1lICdJktRSBnKSJEktZSAnSZLUUgZykiRJLWUgJ0mS1FIGcpIkSS1l\nICdJktRSBnKSJEktZSAnSZLUUgZykiRJLWUgJ0mS1FIGcpIkSS1lICdJktRSBnKSJEktZSAnSZLU\nUgZykiRJLWUgJ0mS1FIGcpIkSS1lICdJktRSBnKSJEktZSAnSZLUUgZykiRJLWUgJ0mS1FIGcpIk\nSS1lICdJktRSBnKSJEktZSAnSZLUUgZykiRJLWUgJ0mS1FIGcpIkSS3VUyAXERtGxFURcWv9u8Ew\nv9s/Im6JiNsiYuFI00fEPhHx3Yj4cf27V2Oa6+q8flA/m/SyDpIkSW3Va4ncQuCaUsps4Jo6vJyI\nmAJ8BJgHzAEOi4g5I0z/a+CgUspOwJHABYNme3gpZZf6ub/HdZAkSWqlXgO5+cCS+n0JcMgQv9kN\nuK2Uckcp5S/ARXW6YacvpXy/lHJvHX8TsHZErNnjskqSJE0qvQZyM0sp99XvvwRmDvGbzYBfNIbv\nruNGO/3zgO+VUv7cGLekVqu+JSKi+8WXJElqr6kj/SAirgYeOcS/Tm4OlFJKRJRuF2So6SNiR+Dd\nwL6N0YeXUu6JiHWBzwIvBc4fZtmPBY4F2HLLLbtdNEmSpFXSiIFcKWXv4f4XEb+KiFmllPsiYhYw\nVHu1e4AtGsOb13EAw04fEZsDnweOKKXc3liee+rf30XEhWTV7ZCBXCllMbAYYO7cuV0HmZIkSaui\nXqtWLyM7I1D/XjrEb24EZkfENhExHTi0Tjfs9BGxPnA5sLCUcn1nRhExNSI2qt+nAc8GftLjOkiS\nJLVSr4HcImCfiLgV2LsOExGbRsRSgFLKg8AC4ArgZuDiUspNK5q+/n5b4K2DHjOyJnBFRPwI+AFZ\nsndOj+sgSZLUSiNWra5IKeU3wLOGGH8vcEBjeCmwdAzTnwqcOkyyu3a7vJIkSZOJb3aQJElqKQM5\nSZKkljKQkyRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmSpJYy\nkJMkSWopAzlJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElq\nKQM5SZKkljKQkyRJaikDOUmSpJYykJMkSWopAzlJkqSWMpCTJElqKQM5SZKkljKQkyRJaikDOUmS\npJYykJMkSWopAzlJkqSWMpCTJElqqakTvQCSJEnj5c5FB070IowrS+QkSZJaykBOkiSppQzkJEmS\nWspATpIkqaUM5CRJklrKQE6SJKmlDOQkSZJaykBOkiSppQzkJEmSWspATpIkqaUM5CRJklrKQE6S\nJKmlDOQkSZJaykBOkiSppQzkJEmSWipKKRO9DCtFRDwA3DXRy1FtBPzadFa5NFZWOpNpXVZWOpNp\nXVZWOq7L6p2O67LqpjNaW5VSNh7pR6tNILcqiYhlpZS5prNqpbGy0plM67Ky0plM67Ky0nFdVu90\nXJdVN51+s2pVkiSppQzkJEmSWspAbmIsNp1VMo2Vlc5kWpeVlc5kWpeVlY7rsnqn47qsuun0lW3k\nJEmSWsoSOUmSpJYykGuRiIiJXoZVidtDklYfXvOHZiDXAhExE6CUUjyQlzMDICLG9TheWds8IkZ8\nXlAbRMR6K2ufTKbzobkuk2m9pD561EQvwKrIQG4VVy/ol0TExdD/YK7P81onIqbV7xv0a77DpLU1\ncH1EPK6U8rfxChwiIkptSBoRL4iIPcYpnTWAT0bEeeMx/5UlIh4FXATsERFTximNdYDOvNcbjzSG\nSHPcgqyImBERU+u5vRXked7PNEZIv2/bMCKmRsSs+n1Ov46BSGvV72v3Y57DpTPEuJV2n1xZGZRx\nOIanNL5P7ee8G/N9GHBBRLx7PObfZgZyq7BOEFFKeTLwuIh4P/QvmBsUpGzX47zWBPYC9ouIk4Dj\nO0Fdv9XlvhP4BLA4Ih4zXsFcY/vsAbwc+Em/06jr87dSyr7AnIg4td9p1HSG3D79vKiXUu4AvgIs\nAJ7Yr/l2RMR04IXAARGxEDg/IqaM942vnnNzI+LR41Ayvgfwvog4CPjXiNisj/MeUiNg2AVYWP/2\nw17AYRHxPuA8oOdrQF3WZwBzI+Jwclut2et8h0qncb7vHxH7RsRjSyl/63daI6UPPGIc5v/4iNgw\nImbUY7gv18t6nX9ZRGwZETsDp/U7ExcRa5RSfg+8BHhqRLyhn/NfQbpDnuerXIl5KcXPKv4BdgQW\nAX8EPtQYH32a/wLgU8Aju5k3sE39uyfwA+A2YMc6bo0+b4tofH8GcDnwbWD78UivznNf8vVur6/D\nU8djnYBdyJvfH4Fzx/F4Ohp4JXB4v44lIBrrcTBwI3AnGaT05ThtpPU44D+BnwNzxms7Ddo3TwC+\nC9wPzO7HNhuUztXA74C9xuMYGybNecAVwK+ATwK79mGeawFfBv4bmN/HZX0W8K26zw8Z5+1yfE1r\nAfDQeB9jg4+lem5+CVinj9f444FfAB8HTgc2ruP7cr2s18i/Abd3zo9x2k7PAc6q15Y3rcR98mLg\necBLx/tY6OZjidwqLiKeTF5svwQ8H9gnIs6B/pTMRcRhwBHAa0opv4yIjcYy74hYF3htRLyRDOL+\nH3A98NiI2LD0OTdbOmdYxKuBdwOfBX4KXBgRO5Y+lMwNXu9SypXAtcBzImLNUsqDvcy/kc6UOv8S\nEXOBzwEfAZ5Glj58tB/pDErzecBCsjT+NRHxjsYydH0slSoijgbeCBxF3tBPAXbvw3I3l+0+4KP1\n784xqBq/zyWMJSL2BT5GrsvVwJciYrtet9mgaW8ErgEWRMTa/TrGVpD2Y4B/BU4AHk8GkS+OiJ16\nnPVM4L3Ap4HHR8QejdK/MZ+XnWlLKdeQ5/nPgb9GxCY9LueQaUXEDsABwD5kEPdV4GfjWeUZEdMa\n17XjgMOBE0opfwDW7cP8DwY2A54K/DtQgLdExEb1etnTMVxLy64EvgZsQm320O8amYg4grzmLyHP\nxXkRcUo/02hq7JMFZCD8f8DZEXHIeKXZtYmOJP2s+EPmRD/QGH44cDdwZpfz65QwrFE/pwKvBx5D\n3oC/DXx+DPObAswlA5AjgalkwPkh4GX1N7sAW/e4HbYD9mwMnwM8ubEMJ5MB5A49ptPMhb2IDHI7\npSSfAi4B1uzDft0YeBcwrQ4/BVjS+P8M4F76WDIHzK/76Yl1+NFk8H1KD/PcAziqMfxe4KTG8Clk\nLv3pwJQ+7JOXAW+s3+eSAfYrGuvX03E2TPqnAyc3hk8D7gEeVYfHXKrROA93quuxYR3+ILC0fn8U\n41T6RJbyfwVYrw5vAnwHuJBaut3FPE8Azq/H7jrA++tn+3oevbDLbbQ9sCXwSOBJ9Tx8SWMbbdrH\n7bIe8Ka6z78MrFXHHwvMGof98GjyujuzDr+TzPjsSpbM/YyB6+iYSufI6/sGZAn/V+q4KWSTh3cB\n5wIb9bDsnf2zMVkiP50Mgv+Xeq2u+26dPm2r44BjBq3HTcBb+7xPtgDWbqzbxcCawGuBL9a0e74H\n9PNjidwqZojc0W+BvSIbelJK+S1wAfDSyMbEo85NDWqDMaNkadkF5EXqfcADwKHAmhGx7WiWs5Ty\nEBkMnAF8sZTyYCnlM8ANZKncZ8kT4S+jXc4h0lqLzKU+NyKeXkevS964O8vwZfLm8eGImN5tLrOz\nfSLiteSFYx3gPRExr5RyWF2PL0e21erFVLKUZ1ZEbEoGbZtGduKglPJH4MNkrnOF+2I4Q5QibE+2\nYXpcREwvpdxOVhccFREnd7keM4A3R8RRdfgmYGZEPKKuxynAH8jq3K5y6I19chzwCrIzBaWUZeTN\n7gURsZgsbeh1vwx1Dv6CPA46y3Myub8ujIgNyhhLnTvnYUTsB3yGDH4XR8QxZBOKeyPie8ClwP/0\nsCrLpVn/Tq8lwTeTTSD2qSUz95OlnJuR59pY538YuY8X1mP3T2QG8Q/AG8gM401jmWfdRvOA/yAz\nideRpXKXAPtGxHvI68zmY13euswbNL4fGhGnkaVVzwReXErZv5Typ4g4lCxhHo92UbPIY+uYyE4c\nvyGrDk8Bfg+8A3h+RMxsXLtHa2op5b+Bnclz/l9KKQ+VUm4kA5K7GOg0NCaNY/ggMvj/AvA6smT5\nSGBpLcm6GJjdzfyH+ddr67Wrc9/5Ptkuuy9tCiPbpx4KTI3stPHfZCD8brJE84U17aMjYtd+pNkX\nEx1J+hn4MJDDOQB4D3nwPAJ4O3nhfSJwEHlyPKaHdI4jb4an1LRmMFAyNI88QR4x0nLW79sAmzSG\n12p83w04hh5KyRrbZDbwFvKmtyOZa/oReeOAbMOwiC5z5yxfUvkI4MI6fDJ50Vuz8dvzgC36sL+n\n1X18CbA2eSH8GvBc8mJyOfD4Xtanft+eDNwhb9JfBp5MbYdV9+Gjxjj/NRrfnwX8kCzB3KAu96vI\n0rrn1+N1616OAbKk5JK63BuQmY9PAfuRueZ51Laa/fjUdJ4J7EDebL8P/HM9NnYng57/AN4+hnk2\nj6E5wFXU9lf1+H0P8NQ6fBiwe7/Wp87zYLKD0PlkRugAsmT738iSzhvrb65untOjmO9Mspp2AbAt\nGbjdCJxd/78Fjfa3Y5jv1mSg9ui6fX4CrF//tzsZOD6ry22xNZmRelYdPhI4rn7fCLi1bpvFwDJg\npz7vi+b5+TSyI1Wn9O+xDJQIPYO8Joyp5Az4J+Bs4MR6fs8Gfg28tvGbtbpY7uZ5vx1ZYvj4eiy9\njlpTBBxCZkT37XHbPL+edzvX4feSJcfb1v3/KWp7vz7um3XJdrgL6vCpZDV75xr6YrK97Ob9TLen\nZZ7oBfAzaIfkjemGeqG6Bvh0Hf82ssThWuA5jd+Ptbj9aDJnu1O9YJ/RmQ8ZdN28oovWoJPstWQu\n+zLg1Mb4aeOwXdatn7eRpYePrxf4H9WT+e8dLLqY9/qN79vWv5eQpZWXUm/A9WK/VY/rEYOGNyar\ncZaQxfcvIW+sS4GD+7Ddjieryz9JtltajyxduJys7uypUT31Bg3sD/y4XtA3q/vo02R195j3y1DH\nNXlT+k+yFOtdZIbkM8D0Ph1ja9S/TyHb351Dtk/dh6zCu5wMhG4nmwsczSgbXJPB54X1GJ5GBjsP\nAC+o/59S9/vifqzLEOnvVK8rBwFn1vN8HTJoOIlsBP8EMoC9Hnj4KOf7MrKq+SlkyeUXyCB7Nnl9\n2brL5Z1FBiAvJ3sp39A4N/cDHjbS8bKCeXeuJQvJYOfJZNXvyxq/WY/MUL2AMWZyujm2h/nda4Dv\nAY8b4/yPJYO/p5OZ8rfW8TuQHRJe2eVyb0ZWA29Sh+eStTA05n8R8Iw63FP1I/Bq4OvkNf/aeizM\nIKufLybbL+7cz31T051BZnj/nbweTyeD0u/Wc+dG4LH9TrenZZ7oBfAzaIdkCdAOZLXhNwZfRKjt\nDcZy4RqPBqhYAAAa9klEQVQ0/avJAOifgCsZKJXZuqa77Sjn8yTyprYN2b7uBuBfG//vqj3UMGnt\nRAZWu9UL7NvIm9725E1xQ8ZQgjBo3kHm+N5WLxRfreP/hZrbrMNHkyUCPefCgGeTgc7ZZNukR5GB\nycepJaEM5MjHGqg3g9I9yEB3S2CrehH+ChkwLqwXwzHnyuu812jcGJ5Wx3WCuYM7xwCwQTf7pPF9\nf7J0qtOG6GmNbXQIjXZMPeyPtVk+iHs7sEcdPpC8iexfhzcg22rtR95kRx2k1v0wux7PU+oxdx7w\nzPr/fchMSc+9Fesy7la/71jTOa3x/w+RmbBO27wpZGncDxnlzZEMfj4AbFeHZ1KD6rrfvteZ/1j2\nO1ka8lGyavZ2MnjvXKd2r/vj0d0cV+R17qLGNnpd3RafIzNTOwN7k6XMYz52x7g8J5Il8hcwROBc\n98d2Xcz3VDIQPpIs9Z3a2C/bdTPPOu0sMrB6B7B+PU6vZfkA+ANkR42/788xzL9Z2rcrmRFco+6j\n79Vj4rjGcdLTeT/M8bEneY0MMpA/Eziy/v+p5DV16/E8Lrpa9olegNX90zgoOxfUd5E9VK9m4LEe\nh5BVVdMYQ8PqemHtBCKH1xPxBLK04crG715e5z/ivOsBvjMZIJxLDdjIm9S3gY/0a5s0hh9dL3rn\nk7nA9chq1sXUG26X6TQvHLeQ7ZE2r8OPIUtNbiE7CPyYLkv8BqX5BPIG+hwGSsl2IQO6M8jqurXo\nIhCuF+kFjYv2rsA59fu0+vl3anUHK6g+H0Oax5JVNp0qwf3IkpmX9GHer6zH1OlkCdK+DJSOvoIs\nbeipyossET2JgSq79wP/xUCV25pkte13gH/ubDfypjLagKdzjm9Sz8PbySB4Rj3GfkxmTL5Df0ph\np5JV89uTpQkbkUHKZ5rLXM+fuxloVrEbo+jowEDQezXZhvcRjXTXqMfETd3sG7LE8Kt1v3+CrLK9\nn7yZLyCruLt6rAkDweBGZLX5rmQQ//q67X9OZnA+Q5a+9rUkbtCyHEfWuGxMtrdcNHj7djHPPcmM\nxmvI69YVjf+dQLb76/qYamy7S8hMyEzyOnY2GcA9vab75B63zWZk5nzbOs+v1mPrFDIzfUI9znp+\nNEvj3GxmHi8CjqjfjyCvy69gUCnwqvSZ8AVYnT+Ng2ge2VttBhm03EnNPTdOjr26mP8WwDfJYOEG\nMge6MZk7/wB5Yz+SvJEM+6ykoU6YeoB/nSzB6NwItiFzaJv06SSb3/i+NRlwfpKsDtqQvAl2WxL3\nCGDL+n2/epP4NvC+Qb97PNk2ccs+rM9Oddu/vTHu7cDX6vfN6TK33Nj+G5JBwh51He9g+Rzz2XTZ\nC64xj6eQN41OEH8U2SC4U4q1Fz3eBOv8ryUDkQVkm6Ur67myST1ue36+Vz0HtqnnxpPquPeSVVOd\n4G5Nssr4Sc3pxpjOPvVc3IDMlHyfbCc3lSzh+AwDPfL6ce6sWff/x8nS8w3IquK30giwRrsNWf5G\n1yz1/ULn+K3DU+s5M+ZniZGBwTcYeCbkgnp+vJPsAPA6BjIhYy3t2Yi8rnYyzCeRJe6Pq8fAa+q5\nMbfXbT/S9qvDp5Cl5K8im1FMrcd6t0HchmTp0Z5kRvuL1CpUsk3Xj+mhXXWdz55kbcJL6356BZl5\n3JksVfwQXWREyGvVofX7K8kM2sfreXgKtfkCWYu0mC6v+SMsQ/MZqs8D3tsYPpZsf71+v9Pt2/JP\n9AKs7h/yprdcLqaeHD8g29TcCBw4xnk2L7qvIJ8R9YY6vBYZmJxDFrtfzSjr+8nShFPIdlcPJ6u8\nriKruzrBXNdtrhgIbDs5/huA6xv/35asDryyXoC7vuGR7WLOJQPaHzKQ47we+Gj9/mJqtVcf9vMM\nsgr1UuDzNDqAkNWDXT3yoTHvzrbblCzdeR8ZoOxcj69TyQDiu4yy+nyo/ULe9D5M3jT2aOz395PV\nrE/oZd8PGjeL7EBxVR1+L1mC9Ez6UHXP8qXJHySD7F3ruLPIKpYNm8vXzTHHQMeG3RrjTqzH9+Pq\n/ns1mUnpuoMDWVreLGXeiixhWkwGV5vU9Tqd2u6qca6Ntt3WsWSp8UeBg+q4Szr7qMf9sQEZ7D6l\nDk8jOyRcCjyvD/M/iAzeNqjDC8jr7M5kydxbGchQ9/Nhz4MfLLtfPV+uIK/xnU4O/0KtluwynXcB\nH6/fn0uWaF5Dfa5nL8tez/Wb6zzfQ5aS3VCP23Xrb6Y0pxlDGgeSpaHvrNvj0WTV9hvIwoK/kSXK\nt9LDdXKY9eo0EbmPzCjMq8fdD2nUKlAf07OqfiZ8AVbXTz2AppA3xVfU7y8hqw9fWf+/ET00ridL\ngOaTPZ9uoT5vq5N+/TtjBdNvykBbrU4116vqxe5bZC7wJXV8r8XpzRtQM8f/eeC6xvAp9ULS87Oj\nyJvR/wHPb4ybQd5MPkNWgfWj1Gcr8iY9hwwaLiBLBJ5B3kTupPt2K+vUi88zyODgpXWe7yBzkY8h\nA8hF5IV+TNXDLH8T6jRynkpWrXyQgSrVQ8ggu5s2Pc00nlI/neq6E6mlpORN8At00QNyuDTJTMhF\ndR+9nQx05tb/fZy8CY45c9KY/5R67twD/NOg37yOLCmZTt5MFtCHZ5XV/f8YskRubbK06VyyCn8m\nWb3eTYlZJxDatR6/HwCOrf+7AbikD8v+L2SzicfW4f3I0qUl1GtRj/OfV8/rTjB3Itkr9QlkKeao\n2/R1kfbTGWiD+ySyKcdzG8f2j8d6/pBNTfZtDH+ZgQ40QZY0j6rjygrS2I0sHd+9Dm9L3gc+TgZa\n7+7mHBmUxj5ktekn6/Ca9Zw4rV5bjmSMGdAVpNW83nSC6E4v2K8BbyYzQBeN5/HQ12Nrohdgdfvw\nj0Xs+5PPRvpqPWhfUi9cXTeqryfw5mRQeFIdN5fs2fkSBnr7TRu8PI15bEYWlR9Xbwb/xvJVS28C\nPla/n0Afqh7rvDo5/rMZyPF/vm6fN9WLXVdpDbHtdyNvnl8kS3k6J/VaZIldX7qXkw3c31gvfNvV\ni8aF5M3vIuCAoZZvlPOeSvbqW0beoB5dx+9IlsL9vfSlx3U4vl7k/gN4Tx33FjJAuJgs2disx/3x\nerIU7HzyjR07kkHdl2oa36OLRu4rSP9pZCnfc+rwmnWbfZiBatYxl2QwEMTtQzbO35i88Z1L7TTR\n+O3Wje9d9fYmm1D8W/3+DPKVW0vI3uQ71fU6sR5zu3aTTt0PJzLw8OUZZDXbhQyUxGzVh32yOVky\n8yXyenhLPU8vo089FPnHYK5T8jNuD3klS5iuB05vjDuYzASfTwZKYzrWyODzJPL+8Xoy8DmaRoa9\nT8veectFp4pzWl3208n7RFel8EOkM598btuLGuMuA549TvvkxDr/64DD6rh1yTZxl5C9ylfZ6tTl\n1mWiF2B1+jQu8E8lSzReRFZ5bNW5CNYL7zL6kzPfhyx1eh2Z659bD9ClI10UyWDwKPJG989kTq/5\nxP7H0ef3gbLiHP8baOTUe0znWDLX9WYyEHpZvXHsWrfVO/u0Pk9ofN+WLBm5gGzvtwVZTbGQLjod\nsHyu8pFkAHQeWU3Red7RdmQV65sZY2nGoPnPIzu37FCX/ZvUx2SQVS6voIsqDwaqs6eQpUWX1uF3\nAZfV7xvW/fIGemzjM3jdyFLL33Uu4nXc9HrML6aHkoy6zX4G7FeHNyczPGdRMyhDbesu01qbrPY6\nj7yuPJkMtE6ox8VOZObkX+giqCebUZxJlip/pzkPsplDX9uV1fT2r/t8J7L06kfUnst9SqOzfzpV\n533toTp4n5LB9FlkJ4qtGKgReSTwsLFeA+q+vZjMIJ5GBu6LyFKtu2lkuvu0PvPJgoBOwLMnmbHq\na6BDZg7uIGteDiEz7n3LvDXSeW49lrerx9rtwPH1f1PqMdi3N4aM92fCF2B1+5DB1R1k6dL59YbR\nKZHZj8xddd1zjWxzd3Zj+Jn1Av9mshpuOiO8MoWBG90/kUHfUvLm+gsGeu4dXm8S6/d6I6rzW1GO\nf0ZzuXpM50SyXeCeZIPzTi7zVWSQdQPdP4R3yqDha2m0HSKruz5JBtOzyeDl82Rp16hLSVg+yDqB\n2qOKDLzPoPZOI3PLT2eMD8wcNP9HkcHaOwf95uvUx450ua0GNz6fQ7Ybel893jq9U7t64OsIx/XG\njXGvJksTt2yMm05vnU7Wquuw16B0tySfvdjTq5Gax1tjO80gS5Z/SK02JW9Gx5M3rJ27OX8YeGDx\nMWRpZadKbV/yRvsd+lDVvYL0n0mWWo3H88Lm12tAX3pADj7O6vfDyeYO8+vwuWQv+K5rMOp2/zrZ\npvD6ul8eRraBfCvZDrbvL64nM9q/JWtzLqYPPaxXsH4PkW0j+9JzmEGdSMhClGbp6BPJ+1tfShdX\n9mfCF2B1+tQLxlsYaMOwBVkUvoisfpkP7FP/N9rGx4Nzfk8gg59mr5sXk69jeQWjbMtQL0DfJ4ON\nD5E5/feTz3Q6k8z5df04jkEXu5Fy/F0FVp1t3kyPbF83jSydWErmlDsN9tenyxwmefN/KVmC9Ezq\nuznJ0qvPN373hnox36UO70r3b6M4kWyr2Hnq+dpkYPcBsqfyj+jtXYqvINukHUWWXsxs/O+j1Ad/\n9jD/g8iqs04V1yfJG1TnmXHHkp19eg56GmnuX4+pT5Bt4tYiS39vpIc3QzSOrzlkafXV1A4tDFTZ\nr0fefLfqw3pMJ294W5MPrT257v/rqJ116u8eXo+TMZfQkCV7d5HNK2aTVeunk736riKrv/seYA1a\nhln92F4rmP+4PVKCzCRcV/fP7WTP57XIjPUFjLEpQmO+R5IlSkfVY7lzfG1d/647jut0MJnxeX0d\nDvoYBDfS2ZPxeW/yIWQm5F1kpnftxrl7Jn1oEz0RnwlfgNXtU28elzMQPMwhexaN+WbF8sHQHAZy\n4jvUi8UH6vC+ZA5q1Dlnstqpc7JOrxelT9eb3tbdLO8w6ayUHD/Zg3MGWQK2tP7t7IPj6MPLycn2\nh78hg549GuO/TeYun08GV52ekb30up1OPjy286zBTsnM2mQp3JvoLdA+uC5r5xEtbyd7lnWeafh9\n+pBbJqu4biVLFPYieyl+gqwuuqmXdRgirR3J6qGnkTfVtwEX1P+9p96gun7IKBmYfp/sJPQOsuS1\nU+L49Hoc9PzsvkZ6LyJL8G9hoFR/BllKc2bjd908k3A6WR38TbIKbX7dRj8imwkEq9iLw1elDxm0\ndxruv4lsi9U5R6eTQUNX1zUyyLkd+Hpj3IlkO+aeOh2MMv19yerb5070dh7FsjbvkYeSvVPfQd5X\n/ofsqHFgvQf8lD68enFC1nOiF2B1+JBt015QL4Db15vi2+r/OjndMRW1s3wvz9fUg/A71AfyktV4\nX6w3j5sY4/tO6w37kuaNlCy1eDc99oJqzG/ccvz847OJvk+WKF4O/JL6OAMyV3szPbTDYKDUb1My\nQPw5g0rZatpn0/3DTJsXpE5P4uuBowb97ol92jcvZ6DaeUpj3NvJEsV+BlgH1H0wg6wOPposjeu5\nlxqNIIYsrT6rs8/IQOU8Bt5M0cs7gXchA8HOM9B2IJtN3FjPz5voU1UUAyUIU8kS02VkRq7z1pcZ\nZPXax7uc/5Prft6RrFq/rJ4nx5KPgngrfXxzy2T48I9VdxvW6+c55FsjOufsP/d6XJOZnveRbTmf\nQT7T87usxNdGUV9bN9HbfYRlbF4ztyQzPp3OYAeRzRC+w0Cnh1aWxpVSmIrGVUQ8k8x9/T8yV/tL\n8uDZNyKuJy+6p5ZS/nMM85xWSvlr/b47eeF9MvAg8L2IOLOUcjzw7IjYG7i5lHLPGBf9OrLdwIsj\n4itkSc//ku9m/e0Y5zXUOkwn2yTcQwZyne1yIPXtA+QbCv7cZRIbAKdHxI7kc4meX//+J7nNPxwR\nB5KNqZ9XSrm923UppfwtIg4hS3ieTwYmV0fEYaWUH0bErFLKazq/j4go9WoyGs3fR8QCYIeI+H/U\nN1tExAOllMsj4nDgtRExr5Ryf7frU90FHBIRny2l3FLH3Q/cXUp5W4/zXk4pZWlEBBn0PK2U8vFe\n5xkR65ZSfldKeSginkY2MJ8GvCAivlBKWQrcHRH/R2auvk6WbHXrz2Qg94yIeC5Zung3+daLW8kX\nsn9jrPt+iPWKUkqJiM3IEoUjyFdKnUH29uw8BX9PMrjrxi/qZwl57boc+G0p5XMR8RD5OKCHul2H\nyaiU8jeAiJgD3F5K+a+IuI6s8n5KKeX/IuIIMmi4ose0fh8R7yFLzV9P1gIcVUr5SS/zHeMyXLWy\n0urGoGvmiWRToXWB90XEPaWUL0TEFLLZ0I1kCfaDE7fEPZroSHIyf8i2Mtcy0IbpaeTNvvME9x0Y\neCXUaNvEbU9eDKaQN6cryB6ls+r/H0bm/i/qw/JvSj6e4yvUh/D2abuslBw/wz+b6F/JjhQbMsaO\nAMOkswuZu3tMY9wnyfZr/0y+gqfrdn6NeR5P3qg3JwPfy8i2MreQwe9P6VOukmxb9U6yhPTZ5IXw\nRsahEXUjzb40PicD9a+RJbvbM/Ag038jqwm/TWYenlr3W0/PQKxpPqyeK9+s+2ROTeOwXuc9RFr7\nk5me8+p6bUQ+nPtK8sb+e/rTu3tnshnC14Gfjdd+b/OHfOrAifX78fXae2U99rYl2+LeTgbaY3o3\n7yjTn0aXj61ZHT5kzdL5ZO/UfyWfffkMBnrMH8oqXrI4mk+niF59VqP9F5Ftfk4qpXy4jjuOrP46\nusv57kKWKs0kS012J28gFwPXllJ+FRHrksHXwcAvS487OSI6L/H+fS/zacxvc/Jm9HIyxz8NeKBk\njv8YMsffdQnZoLTmkze8l5dSPl3HXUq2jfpMn9LYgezE8C1yvzyNDN42Jqs8vl5KubLHNB5OVqe8\nhXxu3DzyOUdTyV5kt5Lb8L5e0hmU5iwyuDqYLI09vZTyo37Nf5g0H9aP4ywinkM+2uUPwJtLKd+M\niEeR67IHGcTfBXyhlHJJr+k10p1eSvlLRDyRbOv5qlLKNX2c/3ZkAP8yMph/LrlO+5KlgDsAP+j1\neGuktwn5DLRXkU0V7uzHfCeDWoq8NwMvdX8UeX1/ARng3UB2PNuFrC35TSnlrolZ2tVPLbX+Fvnk\ngGMiYi2yhHR98hy6trS5FK5poiPJyfRhoO3Kegw8MuNoag6tDu9Jtv/agDGUOjDozQdkkfB7yNKH\nA8gSoBczUDLX955E47C9VkqOn3F+NhHDl8Y8jy7fnThMOmvWbXZtZx+TL3h/K+OYKycbZ0+f6OOl\ni+XemwxA31yHOw8yfe+g86mfj56YQvZE/g59bhNXv28FnFO/d9pmfgQ4vH7vlDT0tTfheB5fbfyQ\nj/p4TP1+Ohm0fa7x/5eSPbtfTkveDjAZP/V6fC8Dz7+bykD7wmHfatS2j23k+qiUUmoJ0KuAdSPi\nQ2RV0R+BD0bEQdQ3JpRS/nuM8+60wTgY+BPZ5m4KWULzDrJK8gTgrxHxuTq8SivZfuwoao4/IrYu\n45DjL6V8MSKmkp0nvkh2OLijj/P/PdnmbnEZKI15JVka07f9UEr5c0T8EZgaETuRN/WrgPNKbTM5\nHkopfxmveY+nUsrV9fh6T0TcXkr5VET8L5mZ2qi2LSylXuH7lOZDEfEzsvTq5722iavzLBGxJ1lN\nfBdwYEQcXQbaEv6GfEwHpZYw9HOd6vzG7fhqqfXIc/5u8jz8IPCaiDixlHJGKeWCiFgTeCz5TDRN\ngJK1PH8m20tTrwEnkY88+uNEL1+/WLXaR/Xmeib1lVVktdQPyWcGvZAscl9aSvnAaC/wgxptHkpe\nMM4hq1K+RJY4bEg+F+cpwE/L2Ds2TLhmB45xTGNP4K7xCBbr/DtvKDgTeFcp5dJxSGNN8lEwe5Nt\nGF9QSvlpv9OZTGoGaglZ8vsn4BOllC9M7FKNrNGx4UlkT+FbyHaQ65MlDR8kq9TfCry6lHLdRC3r\n6igi3ku2631DKeWsiJhHlsRfW0r5YP3Nw0sfOoepN3XfLAZeW0r5j4lenn4zkOuTWh9/GvkYkb3q\nuCeTxevHkW0oDiFLak4rpXxpFPNsBnFbkp0ElpVSbo+IF5Jva/gMmeObUkp5R//XTGNR2xNu0q/S\nmGHSmEa+2udvbQzaJ0LtSfoO4GWllG+N177pt4jYjVzuk0opP4qIl5BtsR5JtsG8GbihlPLFCVzM\n1VJEbEtek18LLCqlfDoidiUzch8qpXxiQhdQy4mIfcgexX2rjVlVWLXaBxGxTb1xXwe8JCKOJHuN\nfisiriTbUnwrIq4gqzxH7CY+iu7TF2dbW/6NvHCcNS4rpzEppfyBfI5c36u3Gmn8lXw8hEapVrFc\nV0r5rzq8ygdx1fpk6es+5MN4LyJL99ciS+M+UEvtWhGYTiallNuA2yLif4DT6t+1gL+Qz3jUKqSs\n4o9M6YWBXI9qD9EPRcSyUsopEbEG+ZLnXSJiKdng/TKAUsr/RMRnRnPBbQRxh5APFH4p+SiLnYDd\nI+IbNZh7kOyl9l/jsoLSJNHGc6SUcmUtTTw9Iu6tbXw+Xf/9g851wiBu4pR8JtlfyQb0fyAfL/Xz\nCV4srUasWu3CoNKyKWRX8zeRVRyLavXHAvKl4OeVUr4cEVPKGB+iuVp1n5Y0rIg4gHyu3xmllCUT\nvTz6R/VRLaWU8sBEL4tWL2tM9AK0Ua3K2CMiHleDs++TD7jdo/Za+gTZNu4eYL3akH/MPZdq+6dX\nA/Mi3xLwp5rOX4H9yMdCSJrkSr6J4u3AGyJi01ryr1VIKeV+gzhNBKtWx6DRi2wb8k0E+0bE/JKP\n0biZfLTFSbUx+hnkY0h2IXuXdtUjc3XpPi1pxUopl0XEtwwWJDUZyI1BDeIOJh8sO498sOynIuKF\npZSfRMSdwKXAN0spf42ID5MvS+6p+3nJ92j+DVgcEQ/W7tNezKXVjEGcpMFsIzcGka/HOo98SvTN\nddwF5DPjvgb8U/3f17ppEzeK9Cdt92lJkjR2BnJjEEO/U/M+oACXA78upXxl4pZQkiStTgzkxiAi\nHka2jXsx2dX8Z2Qw99tSyqcmcNEkSdJqyECuCxExvQy8U/Pj5Ds1r5no5ZIkSasXu7B356H6KpYP\nAycbxEmSpIlgiVyXVsY7NSVJklbEQE6SJKmlrFqVJElqKQM5SZKkljKQkyRJaikDOUmSpJYykJMk\nSWopAzlJkqSWMpCTJElqqf8PKC/XOiAgcoAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e4195cd30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def diff(alpha, w):\n",
    "    return alpha[w,'pos'] - alpha[w,'neg'] \n",
    "\n",
    "def plot_discriminative_features(alpha, threshold = 0.0001, reverse=False):\n",
    "    frequent_words = {w for ((w,y),p) in alpha.items() if p > 0.0001}\n",
    "    sorted_by_ratio = sorted(frequent_words, key=lambda w: diff(alpha, w),reverse=reverse)[-20:]\n",
    "    util.plot_bar_graph([diff(alpha,w) for w in sorted_by_ratio],sorted_by_ratio,rotation=45)\n",
    "\n",
    "plot_discriminative_features(alpha,reverse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many of these words seem to match our intuition for negative reviews."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Naive Bayes Prediction\n",
    "Given a trained NB model, how do we predict the class of a given text? As we have done in [MT](word_mt.ipynb) and [parsing](word_mt.ipynb), we search for the $y\\in\\Ys$ with maximum *a posteriori* probability:\n",
    "\n",
    "$$\n",
    "\\argmax_{y\\in\\Ys} \\prob_\\params(y|\\x) =  \\argmax_{y\\in\\Ys} \\frac{\\prob(\\x|y) \\prob(y) }{ \\prob(\\x) } = \\argmax_{y\\in\\Ys} \\prob(\\x|y) \\prob(y) \n",
    "$$\n",
    "where the last equation follows from the fact that $\\prob(\\x)$ is constant for each $y$ we plug in.\n",
    "\n",
    "In python we implement this decoder in two steps. First we introduce a function that returns $\\prob(\\x|y) \\prob(y)$, then we search for the label with the highest return value of this function. \n",
    "\n",
    "Notice how we only take words into account that appear in the parameters of $\\balpha$. This trick deals with the problem of words that have not been seen at all in the training set. The underlying assumption is that these words have the same probability under both models, and hence can be ignored when comparing the probabilities of the whole document. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('pos', 'pos')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from math import log, exp\n",
    "def log_prob_nb(theta, x, y):\n",
    "    \"\"\"\n",
    "    Calculates the log probability log(p_theta(x|y)).\n",
    "    \"\"\"\n",
    "    alpha, beta = theta\n",
    "    result = util.safe_log(beta[y])\n",
    "    for w in x:\n",
    "        if (w,y) in alpha:\n",
    "            result += util.safe_log(alpha[w,y])\n",
    "    return result\n",
    "\n",
    "def predict_nb(theta, x):\n",
    "    \"\"\"\n",
    "    Finds y^*=argmax_y p_theta(y|x)\n",
    "    \"\"\"\n",
    "    if log_prob_nb(theta, x, 'pos') > log_prob_nb(theta, x, 'neg'):\n",
    "        return 'pos'\n",
    "    else:\n",
    "        return 'neg'\n",
    "\n",
    "i = 5\n",
    "predict_nb(theta_filtered,train_filtered[i][0]), train_filtered[i][1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation\n",
    "We can evaluate a classification model by measuring its **accuracy**, the ratio of the number of correct predictions  and the number of instances. Let $\\y^*$ be a predicted sequence of labels, and $\\y$ the true labels. Then\n",
    "\n",
    "$$\n",
    "\\mathrm{Acc}(y_1,\\ldots,y_n, y^*_1,\\ldots,y^*_n, ) = \\frac{\\sum_i \\indi [ y_i = y^*_i]}{n}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.979375, 0.815)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def accuracy(data, guess):\n",
    "    correct = 0\n",
    "    for (x,y),y_guess in zip(data,guess):\n",
    "        if y_guess == y:\n",
    "            correct += 1\n",
    "    return correct / len(data)\n",
    "\n",
    "def batch_predict_nb(theta, data):\n",
    "    return [predict_nb(theta, x) for x,_ in data]\n",
    "\n",
    "def accuracy_nb(theta,data):\n",
    "    return accuracy(data, batch_predict_nb(theta,data))\n",
    "\n",
    "theta_filtered = (alpha, beta) = train_nb(train_filtered,pseudo_count=2.0)\n",
    "accuracy_nb(theta_filtered, train_filtered), accuracy_nb(theta_filtered, dev_filtered)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Accuracy is particularly suitable when the dataset is balanced. For unbalanced datasets other metrics are often preferred. For example, when a \"none of the above\" label exists that dominates the dataset, it is useful to measure precision, recall and the so called $\\mathrm{F}_1$ score, the geometric mean of precision and recall.   \n",
    "\n",
    "Notice that we can sometimes do a bit better by counting each word in the document only once. Here, however, this does not make a difference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.83"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def binarize_dataset(data):\n",
    "    return [(set(x),y) for x,y in data]\n",
    "\n",
    "train_binary = binarize_dataset(train_filtered)\n",
    "dev_binary = binarize_dataset(dev_filtered)\n",
    "test_binary = binarize_dataset(test_filtered)\n",
    "\n",
    "theta_binary = train_nb(train_binary, 2.0)\n",
    "accuracy_nb(theta_binary, dev_binary) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Broken Assumptions\n",
    "\n",
    "The Naive Bayes assumption can make sense, as some words a strongly correlated with the class labels, but not necessarily with each other once we know the class label. For example, if once you a review is positive, the probability of seeing \"great\" will be larger, and so will be the probability of seeing \"awesome\", but seeing one of these does not necessarily increase the probability of the other any further. \n",
    "\n",
    "$\\prob(\\text{... great ... awesome}\\bar +) \\approx \\ldots \\prob(\\text{great}\\bar +) \\ldots \\prob(\\text{awesome}\\bar +) \\ldots$\n",
    "\n",
    "Importantly, this does not mean that the event of seeing one word is independent of the event of seeing the other. They are only independent once we know the class.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, there are many cases where the NB assumption is violated. Let us consider the following dataset to illustrate this:\n",
    "\n",
    "---------------------------\n",
    "|$\\x$                 |$y$| \n",
    "|---------------------|---|\n",
    "|... Pulp Fiction ... | + |\n",
    "|... Pulp Fiction ... | + |\n",
    "|... Pulp Fiction ... | + |\n",
    "|... Fiction ...      | - |\n",
    "|... Fiction ...      | - |\n",
    "|... Pulp Fiction ... | - |\n",
    "\n",
    "Let us assume we train a NB model on this data. What will be the probability $p(\\text{Fiction}\\bar \\text{Pulp},+)$ according to the Naive Bayes Model? It will be $p(\\text{Fiction}\\bar+)$, the probability of seeing \"Fiction\" in positive documents, regardless of its preceeding word, or any other word seen in the document.  \n",
    "\n",
    "Now, what is $p(\\text{Fiction}\\bar \\text{Pulp},+)$ according to the data? This probability is 1: whenever we see \"Pulp\" in positive reviews we will always see \"Fiction\". \n",
    "\n",
    "NB will therefore **underestimate** $p(\\text{Pulp Fiction}\\bar +)$ which should be $p(\\text{Pulp}\\bar +)$ but is times $p(\\text{Fiction}\\bar+)$ smaller. This means that we may misclassify instances that mention \"Pulp Fiction\", or generally any instances that contain highly correlated words. Notice that $p(\\text{Pulp Fiction}\\bar -)$ would also be underestimated, but in this case, to a lesser extent. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bigram Representation\n",
    "We can address the above problem partially by using a *bigram* representation of the document. Instead of applying NB to a sequence of words and ignoring context, we can apply it to a sequence of document bigrams. So the document (a,good,job) would become ((a,good),(good,job)). Let us do this for our running example by transforming the datasets accordingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.775"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def bigram_dataset(data):\n",
    "    return [([tuple(x[i:i+2]) for i in range(0,len(x)-1)],y) for x,y in data]\n",
    "\n",
    "train_bigram = bigram_dataset(train_filtered)\n",
    "dev_bigram = bigram_dataset(dev_filtered)\n",
    "test_bigram = bigram_dataset(test_filtered)\n",
    "\n",
    "theta_bigram = (alpha_bigram, beta_bigram) = train_nb(train_bigram, 0.1)\n",
    "accuracy_nb(theta_bigram, dev_bigram) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The bigram representation improves the results substantially. We can look at the most discriminative features again to understand what aspect of the data the model picked up upon."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAGtCAYAAACbRTmIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xm8VXW9//HXRwZFEhE8EooGJg6oOZFZDpWYghOWZpgl\nGkmWNGhWkFlqWljmQCpcUwvtGpJWomJmWPfeBkusrkXFlZ9lag6EXSu9atjn98fnszjrbM+wzzl7\nH2Dxfj4e+7H3XnsN3zXuz/pOy9wdEREREVn/bbS2EyAiIiIijaHATkRERKQiFNiJiIiIVIQCOxER\nEZGKUGAnIiIiUhEK7EREREQqQoGdiIiISEUosBMRERGpCAV2IiIiIhWhwE5ERESkIvqv7QSsLVtu\nuaWPHj16bSdDREREpEv333//X9y9pavxNtjAbvTo0SxdunRtJ0NERESkS2b2cD3jqShWREREpCIU\n2ImIiIhUhAI7ERERkYpQYCciIiJSEQrsRERERCpCgZ2IiIhIRSiwExEREakIBXYiIiIiFaHATkRE\nRKQiFNiJiIiIVIQCOxEREZGKUGAnIiIiUhEK7EREREQqov/aToCIiIhIXxk9846Gz/OPs49o+Dx7\nSjl2IiIiIhWhwE5ERESkIhTYiYiIiFSE6tiJiIjIOqHR9d/WpbpvfUU5diIiIiIVocBOREREpCIU\n2ImIiIhUhAI7ERERkYpQYCciIiJSEQrsRERERCqirsDOzCaa2XIzW2FmM9v53cxsTv7+gJnt3dW0\nZjbMzO42swfzfYsc/hYzu9/Mfp3vB5em2SeHr8jlWQ7f2MxuyuE/M7PRPd8kIiIiIuunLgM7M+sH\nXAlMAsYBJ5jZuJrRJgFj8zUdmFvHtDOBJe4+FliS3wH+Ahzl7rsDU4EbSsuZC5xaWtbEHD4N+Ku7\n7wBcClxUz8qLiIiIVEk9OXb7Aivc/SF3fxFYAEyuGWcycL2He4GhZjayi2knA/Pz83zgGAB3/6W7\n/zmHLwMGZY7cSGCIu9/r7g5cX0xTM6+bgQlFbp6IiIjIhqKewG4b4JHS90dzWD3jdDbtCHd/PD8/\nAYxoZ9nHAr9w9xdyukc7mNea5bj7auAZYHhXKyYiIiJSJevEI8Xc3c3My8PMbFeiSPXQRi3HzKYT\nRcVst912jZqtiIiIyDqhnhy7x4BtS99H5bB6xuls2iezeJV8f6oYycxGAd8GTnL3/1daxqgO5rVm\nOWbWH9gcWFW7Iu5+tbuPd/fxLS0tnayyiIiIyPqnnsDuPmCsmY0xs4HAFGBRzTiLgJOydex+wDNZ\nzNrZtIuIxhHk+60AZjYUuAOY6e4/LhaQ8/ubme2X9edOKqapmddxwD1ZD09ERERkg9FlUay7rzaz\nGcBdQD/gOndfZman5e/zgMXA4cAK4DnglM6mzVnPBhaa2TTgYeD4HD4D2AH4tJl9Oocd6u5PAR8A\nvgYMAu7MF8C1wA1mtgJ4mgggRURERDYoddWxc/fFRPBWHjav9NmB0+udNoevAia0M/wC4IIO5rUU\n2K2d4c8Db+90JUREREQqTk+eEBEREakIBXYiIiIiFaHATkRERKQiFNiJiIiIVIQCOxEREZGKUGAn\nIiIiUhEK7EREREQqQoGdiIiISEUosBMRERGpCAV2IiIiIhWhwE5ERESkIhTYiYiIiFSEAjsRERGR\nilBgJyIiIlIRCuxEREREKkKBnYiIiEhFKLATERERqQgFdiIiIiIVocBOREREpCIU2ImIiIhUhAI7\nERERkYpQYCciIiJSEQrsRERERCpCgZ2IiIhIRSiwExEREakIBXYiIiIiFaHATkRERKQiFNiJiIiI\nVERdgZ2ZTTSz5Wa2wsxmtvO7mdmc/P0BM9u7q2nNbJiZ3W1mD+b7Fjl8uJn9wMz+YWZXlMbfzMx+\nVXr9xcwuy99ONrOVpd/e25uNIiIiIrI+6jKwM7N+wJXAJGAccIKZjasZbRIwNl/Tgbl1TDsTWOLu\nY4El+R3geeAc4KzyAtz97+6+Z/ECHga+VRrlptLv19S19iIiIiIVUk+O3b7ACnd/yN1fBBYAk2vG\nmQxc7+FeYKiZjexi2snA/Pw8HzgGwN2fdfcfEQFeu8xsR2Ar4L/qWUkRERGRDUE9gd02wCOl74/m\nsHrG6WzaEe7+eH5+AhhRZ5oBphA5dF4adqyZ/drMbjazbbsxLxEREZFKWCcaT2SA5l2O2GoK8I3S\n99uA0e6+O3A3rTmBbZjZdDNbamZLV65c2eP0ioiIiKyL6gnsHgPKOWCjclg943Q27ZNZXEu+P1VP\ngs1sD6C/u99fDHP3Ve7+Qn69BtinvWnd/Wp3H+/u41taWupZnIiIiMh6o57A7j5grJmNMbOBRG7Z\noppxFgEnZevY/YBnspi1s2kXAVPz81Tg1jrTfAJtc+uKwLBwNPC7OuclIiIiUhn9uxrB3Veb2Qzg\nLqAfcJ27LzOz0/L3ecBi4HBgBfAccEpn0+asZwMLzWwa0cL1+GKZZvZHYAgw0MyOAQ5199/mz8fn\nsso+ZGZHA6uBp4GTu7MRRERERKqgy8AOwN0XE8Fbedi80mcHTq932hy+CpjQwTSjO0nL9u0MmwXM\n6mgaERERkQ3BOtF4QkRERER6T4GdiIiISEUosBMRERGpCAV2IiIiIhWhwE5ERESkIhTYiYiIiFSE\nAjsRERGRilBgJyIiIlIRCuxEREREKkKBnYiIiEhFKLATERERqQgFdiIiIiIVocBOREREpCIU2ImI\niIhUhAI7ERERkYpQYCciIiJSEQrsRERERCpCgZ2IiIhIRSiwExEREakIBXYiIiIiFaHATkRERKQi\nFNiJiIiIVIQCOxEREZGKUGAnIiIiUhEK7EREREQqQoGdiIiISEUosBMRERGpCAV2IiIiIhVRV2Bn\nZhPNbLmZrTCzme38bmY2J39/wMz27mpaMxtmZneb2YP5vkUOH25mPzCzf5jZFTXL+WHO61f52iqH\nb2xmN+UyfmZmo3u2OURERETWX10GdmbWD7gSmASMA04ws3E1o00CxuZrOjC3jmlnAkvcfSywJL8D\nPA+cA5zVQZJOdPc98/VUDpsG/NXddwAuBS7qar1EREREqqaeHLt9gRXu/pC7vwgsACbXjDMZuN7D\nvcBQMxvZxbSTgfn5eT5wDIC7P+vuPyICvHqV53UzMMHMrBvTi4iIiKz36gnstgEeKX1/NIfVM05n\n045w98fz8xPAiDrTPD+LYc8pBW9rluPuq4FngOF1zk9ERESkEtaJxhPu7oDXMeqJ7r4rcGC+3t2d\n5ZjZdDNbamZLV65c2YOUioiIiKy76gnsHgO2LX0flcPqGaezaZ/M4lry/Sm64O6P5fvfgRuJot42\nyzez/sDmwKp2pr/a3ce7+/iWlpauFiciIiKyXqknsLsPGGtmY8xsIDAFWFQzziLgpGwdux/wTBaz\ndjbtImBqfp4K3NpZIsysv5ltmZ8HAEcCv2lnXscB92QuoIiIiMgGo39XI7j7ajObAdwF9AOuc/dl\nZnZa/j4PWAwcDqwAngNO6WzanPVsYKGZTQMeBo4vlmlmfwSGAAPN7Bjg0Bznrgzq+gHfB76Sk1wL\n3GBmK4CniQBSREREGmD0zDsaOr8/zj6iofOTVl0GdgDuvpgI3srD5pU+O3B6vdPm8FXAhA6mGd1B\nUvbpYPzngbd3MI2IiIjIBmGdaDwhIiIiIr2nwE5ERESkIhTYiYiIiFSEAjsRERGRilBgJyIiIlIR\nCuxEREREKkKBnYiIiEhFKLATERERqQgFdiIiIiIVocBOREREpCIU2ImIiIhUhAI7ERERkYpQYCci\nIiJSEQrsRERERCpCgZ2IiIhIRSiwExEREakIBXYiIiIiFaHATkRERKQiFNiJiIiIVIQCOxEREZGK\nUGAnIiIiUhEK7EREREQqQoGdiIiISEUosBMRERGpCAV2IiIiIhWhwE5ERESkIhTYiYiIiFSEAjsR\nERGRiqgrsDOziWa23MxWmNnMdn43M5uTvz9gZnt3Na2ZDTOzu83swXzfIocPN7MfmNk/zOyK0vib\nmtkdZvZ7M1tmZrNLv51sZivN7Ff5em9PN4iIiIjI+qrLwM7M+gFXApOAccAJZjauZrRJwNh8TQfm\n1jHtTGCJu48FluR3gOeBc4Cz2knOxe6+M7AXsL+ZTSr9dpO775mva7paLxEREZGqqSfHbl9ghbs/\n5O4vAguAyTXjTAau93AvMNTMRnYx7WRgfn6eDxwD4O7PuvuPiABvDXd/zt1/kJ9fBH4BjOre6oqI\niIhUVz2B3TbAI6Xvj+awesbpbNoR7v54fn4CGFFnmjGzocBRRE5f4Vgz+7WZ3Wxm29Y7LxEREZGq\nWCcaT7i7A17PuGbWH/gGMMfdH8rBtwGj3X134G5acwJrp51uZkvNbOnKlSsbkHIRERGRdUc9gd1j\nQDkHbFQOq2eczqZ9Motryfen6kzz1cCD7n5ZMcDdV7n7C/n1GmCf9iZ096vdfby7j29paalzcSIi\nIiLrh3oCu/uAsWY2xswGAlOARTXjLAJOytax+wHPZDFrZ9MuAqbm56nArV0lxMwuADYHPlIzfGTp\n69HA7+pYLxEREZFK6d/VCO6+2sxmAHcB/YDr3H2ZmZ2Wv88DFgOHAyuA54BTOps2Zz0bWGhm04CH\ngeOLZZrZH4EhwEAzOwY4FPgbcDbwe+AXZgZwRbaA/ZCZHQ2sBp4GTu7pBhERERFZX3UZ2AG4+2Ii\neCsPm1f67MDp9U6bw1cBEzqYZnQHSbEOxp8FzOpgGhEREZENwjrReEJEREREek+BnYiIiEhFKLAT\nERERqQgFdiIiIiIVocBOREREpCIU2ImIiIhUhAI7ERERkYpQYCciIiJSEQrsRERERCpCgZ2IiIhI\nRSiwExEREakIBXYiIiIiFaHATkRERKQiFNiJiIiIVET/tZ0AERER6ZnRM+9o+Dz/OPuIhs9T+o5y\n7EREREQqQoGdiIiISEUosBMRERGpCAV2IiIiIhWhxhMiIiJN0OiGDWrUIPVQjp2IiIhIRSiwExER\nEakIFcWKiMgGRX2/SZUpx05ERESkIhTYiYiIiFSEAjsRERGRilBgJyIiIlIRCuxEREREKqKuwM7M\nJprZcjNbYWYz2/ndzGxO/v6Ame3d1bRmNszM7jazB/N9ixw+3Mx+YGb/MLMrapazj5n9Ouc1x8ws\nh29sZjfl8J+Z2eiebQ4RERGR9VeXgZ2Z9QOuBCYB44ATzGxczWiTgLH5mg7MrWPamcASdx8LLMnv\nAM8D5wBntZOcucCppWVNzOHTgL+6+w7ApcBFXa2XiIiISNXUk2O3L7DC3R9y9xeBBcDkmnEmA9d7\nuBcYamYju5h2MjA/P88HjgFw92fd/UdEgLdGzm+Iu9/r7g5cX0xTM6+bgQlFbp6IiIjIhqKewG4b\n4JHS90dzWD3jdDbtCHd/PD8/AYyoIx2PdjCvNctx99XAM8DwLuYnIiIiUinrxJMn3N3NzJu9HDOb\nThQVs9122zV7cSIi0g16IoRI79WTY/cYsG3p+6gcVs84nU37ZBavFsWsT9WRjlEdzGvNcsysP7A5\nsKp2Bu5+tbuPd/fxLS0tXSxOREREZP1ST2B3HzDWzMaY2UBgCrCoZpxFwEnZOnY/4JksZu1s2kXA\n1Pw8Fbi1s0Tk/P5mZvtl/bmTStOU53UccE/WwxMRERHZYHRZFOvuq81sBnAX0A+4zt2Xmdlp+fs8\nYDFwOLACeA44pbNpc9azgYVmNg14GDi+WKaZ/REYAgw0s2OAQ939t8AHgK8Bg4A78wVwLXCDma0A\nniYCSBEREZENSl117Nx9MRG8lYfNK3124PR6p83hq4AJHUwzuoPhS4Hd2hn+PPD2DldAREREZAOg\nJ0+IiIiIVIQCOxEREZGKUGAnIiIiUhEK7EREREQqYp3ooFhERNZtje48WB0HizSHcuxEREREKkKB\nnYiIiEhFKLATERERqQjVsRMRWY81uu4bqP6byPpMOXYiIiIiFaHATkRERKQiVBQrItIk6iJERPqa\ncuxEREREKkKBnYiIiEhFKLATERERqQgFdiIiIiIVocBOREREpCIU2ImIiIhUhAI7ERERkYpQYCci\nIiJSEQrsRERERCpCgZ2IiIhIRSiwExEREakIBXYiIiIiFaHATkRERKQiFNiJiIiIVIQCOxEREZGK\nUGAnIiIiUhF1BXZmNtHMlpvZCjOb2c7vZmZz8vcHzGzvrqY1s2FmdreZPZjvW5R+m5XjLzezw3LY\nZmb2q9LrL2Z2Wf52spmtLP323t5sFBEREZH1UZeBnZn1A64EJgHjgBPMbFzNaJOAsfmaDsytY9qZ\nwBJ3Hwssye/k71OAXYGJwFVm1s/d/+7uexYv4GHgW6U03FT6/ZrubggRERGR9V09OXb7Aivc/SF3\nfxFYAEyuGWcycL2He4GhZjayi2knA/Pz83zgmNLwBe7+grv/AViR81nDzHYEtgL+qxvrKiIiIlJp\n9QR22wCPlL4/msPqGaezaUe4++P5+QlgRDeWN4XIofPSsGPN7NdmdrOZbdvlWomIiIhUzDrReCID\nNO9yxFZTgG+Uvt8GjHb33YG7ac0JbMPMppvZUjNbunLlyh6nV0RERGRdVE9g9xhQzgEblcPqGaez\naZ/M4lry/al6lmdmewD93f3+Ypi7r3L3F/LrNcA+7a2Iu1/t7uPdfXxLS0v7aysiIiKynupfxzj3\nAWPNbAwRYE0B3lkzziJghpktAF4HPOPuj5vZyk6mXQRMBWbn+62l4Tea2SXA1kSDjJ+XlnUCbXPr\nMLORpWLdo4Hf1bFeIrKBGj3zjobP84+zj2j4PEVEuqvLwM7dV5vZDOAuoB9wnbsvM7PT8vd5wGLg\ncKKhw3PAKZ1Nm7OeDSw0s2lEC9fjc5plZrYQ+C2wGjjd3V8qJen4XFbZh8zs6Bz/aeDkbm0FEVkn\nKOASEemdenLscPfFRPBWHjav9NmB0+udNoevAiZ0MM2FwIUd/LZ9O8NmAbM6XgMRERGR6lsnGk+I\niIiISO8psBMRERGpCAV2IiIiIhWhwE5ERESkIhTYiYiIiFSEAjsRERGRilBgJyIiIlIRdfVjJyLS\n6M6D1XGwiEjjKcdOREREpCIU2ImIiIhUhAI7ERERkYpQYCciIiJSEWo8IbKea3SjBlDDBhGR9ZVy\n7EREREQqQjl2Ik2kLkJERKQvKcdOREREpCKUYycbJOWkiYhIFSnHTkRERKQiFNiJiIiIVIQCOxER\nEZGKUB07WaeoTzYREZGeU46diIiISEUox07qppakIiIi6zYFdhWg4ksREREBFcWKiIiIVIYCOxER\nEZGKUFFsE6mIVERERPqScuxEREREKqKuwM7MJprZcjNbYWYz2/ndzGxO/v6Ame3d1bRmNszM7jaz\nB/N9i9Jvs3L85WZ2WGn4D3PYr/K1VQ7f2Mxuyml+Zmaje7Y5RERERNZfXQZ2ZtYPuBKYBIwDTjCz\ncTWjTQLG5ms6MLeOaWcCS9x9LLAkv5O/TwF2BSYCV+V8Cie6+575eiqHTQP+6u47AJcCF9W/CURE\nRESqoZ4cu32BFe7+kLu/CCwAJteMMxm43sO9wFAzG9nFtJOB+fl5PnBMafgCd3/B3f8ArMj5dKY8\nr5uBCWZmdaybiIiISGXUE9htAzxS+v5oDqtnnM6mHeHuj+fnJ4ARdS5vfhbDnlMK3tZM4+6rgWeA\n4XWsm4iIiEhlrBONJ9zdAa9j1BPdfVfgwHy9uzvLMbPpZrbUzJauXLmyBykVERERWXfVE9g9Bmxb\n+j4qh9UzTmfTPpnFteR7UV+uw2ncvXj/O3AjrUW0a6Yxs/7A5sCq2hVx96vdfby7j29pael0pUVE\nRETWN/UEdvcBY81sjJkNJBo2LKoZZxFwUraO3Q94JotZO5t2ETA1P08Fbi0Nn5ItXccQDTJ+bmb9\nzWxLADMbABwJ/KadeR0H3JO5gCIiIiIbjC47KHb31WY2A7gL6Adc5+7LzOy0/H0esBg4nGjo8Bxw\nSmfT5qxnAwvNbBrwMHB8TrPMzBYCvwVWA6e7+0tmNhi4K4O6fsD3ga/kvK4FbjCzFcDTRAApIiIi\nskGp68kT7r6YCN7Kw+aVPjtwer3T5vBVwIQOprkQuLBm2LPAPh2M/zzw9k5XQkRERKTi1onGEyIi\nIiLSewrsRERERCpCgZ2IiIhIRSiwExEREakIBXYiIiIiFaHATkRERKQiFNiJiIiIVIQCOxEREZGK\nUGAnIiIiUhEK7EREREQqQoGdiIiISEUosBMRERGpCAV2IiIiIhWhwE5ERESkIhTYiYiIiFSEAjsR\nERGRilBgJyIiIlIRCuxEREREKkKBnYiIiEhFKLATERERqQgFdiIiIiIVocBOREREpCIU2ImIiIhU\nhAI7ERERkYpQYCciIiJSEQrsRERERCpCgZ2IiIhIRSiwExEREamIugI7M5toZsvNbIWZzWzndzOz\nOfn7A2a2d1fTmtkwM7vbzB7M9y1Kv83K8Zeb2WE5bFMzu8PMfm9my8xsdmn8k81spZn9Kl/v7ekG\nEREREVlfdRnYmVk/4EpgEjAOOMHMxtWMNgkYm6/pwNw6pp0JLHH3scCS/E7+PgXYFZgIXJXzAbjY\n3XcG9gL2N7NJpTTc5O575uuabmwDERERkUqoJ8duX2CFuz/k7i8CC4DJNeNMBq73cC8w1MxGdjHt\nZGB+fp4PHFMavsDdX3D3PwArgH3d/Tl3/wFAzusXwKgerLOIiIhIJdUT2G0DPFL6/mgOq2eczqYd\n4e6P5+cngBH1Ls/MhgJHETl9hWPN7NdmdrOZbVvHeomIiIhUyjrReMLdHfB6xjWz/sA3gDnu/lAO\nvg0Y7e67A3fTmhNYO+10M1tqZktXrlzZgJSLiIiIrDvqCeweA8o5YKNyWD3jdDbtk1lcS74/Vefy\nrgYedPfLigHuvsrdX8iv1wD7tLci7n61u4939/EtLS3tjSIiIiKy3qonsLsPGGtmY8xsINGwYVHN\nOIuAk7J17H7AM1nM2tm0i4Cp+XkqcGtp+BQz29jMxhANMn4OYGYXAJsDHykvvAgQ09HA7+pYLxER\nEZFK6d/VCO6+2sxmAHcB/YDr3H2ZmZ2Wv88DFgOHEw0dngNO6WzanPVsYKGZTQMeBo7PaZaZ2ULg\nt8Bq4HR3f8nMRgFnA78HfmFmAFdkC9gPmdnROf7TwMm92ywiIiIi658uAzsAd19MBG/lYfNKnx04\nvd5pc/gqYEIH01wIXFgz7FHAOhh/FjCr05UQERERqbh1ovGEiIiIiPSeAjsRERGRilBgJyIiIlIR\nCuxEREREKkKBnYiIiEhFKLATERERqQgFdiIiIiIVocBOREREpCIU2ImIiIhUhAI7ERERkYpQYCci\nIiJSEQrsRERERCpCgZ2IiIhIRSiwExEREakIBXYiIiIiFaHATkRERKQiFNiJiIiIVIQCOxEREZGK\nUGAnIiIiUhEK7EREREQqQoGdiIiISEUosBMRERGpCAV2IiIiIhWhwE5ERESkIhTYiYiIiFSEAjsR\nERGRilBgJyIiIlIRCuxEREREKqKuwM7MJprZcjNbYWYz2/ndzGxO/v6Ame3d1bRmNszM7jazB/N9\ni9Jvs3L85WZ2WGn4Pmb26/xtjplZDt/YzG7K4T8zs9E92xwiIiIi668uAzsz6wdcCUwCxgEnmNm4\nmtEmAWPzNR2YW8e0M4El7j4WWJLfyd+nALsCE4Grcj7kfE8tLWtiDp8G/NXddwAuBS6qfxOIiIiI\nVEM9OXb7Aivc/SF3fxFYAEyuGWcycL2He4GhZjayi2knA/Pz83zgmNLwBe7+grv/AVgB7JvzG+Lu\n97q7A9fXTFPM62ZgQpGbJyIiIrKhqCew2wZ4pPT90RxWzzidTTvC3R/Pz08AI+qY16MdzGvNNO6+\nGngGGN71qomIiIhUR/+1nQAAd3cz82Yvx8ymE0XFAP8ws+XNXmY3bAn8pauRrHeFzHUto6+W08tl\n9NVyNrh16avlaJt1fzlV2mZVWpe+Wk6VtlmV1qUBy6nXq+oZqZ7A7jFg29L3UTmsnnEGdDLtk2Y2\n0t0fz2LWp7qY12P5ub15FdM8amb9gc2BVbUr4u5XA1d3uKZrkZktdffx6/syqrYcrcuGvZwqrUtf\nLUfrsmEvR+uy9tVTFHsfMNbMxpjZQKJhw6KacRYBJ2Xr2P2AZ7KYtbNpFwFT8/NU4NbS8CnZ0nUM\n0Uji5zm/v5nZfll/7qSaaYp5HQfck/XwRERERDYYXebYuftqM5sB3AX0A65z92Vmdlr+Pg9YDBxO\nNHR4Djils2lz1rOBhWY2DXgYOD6nWWZmC4HfAquB0939pZzmA8DXgEHAnfkCuBa4wcxWAE8TAaSI\niIjIBqWuOnbuvpgI3srD5pU+O3B6vdPm8FXAhA6muRC4sJ3hS4Hd2hn+PPD2Tldi3dcXRcR9VQxd\npeVoXTbs5VRpXfpqOVqXDXs5Wpe1zFRiKSIiIlINeqSYiIiISEUosJN11rrUybSZ6VypMO3f+qxL\n5+TaomOlY2a2cRPnre1eJ20oaSozG1DzvTt/DG0uEmvrT8XMtiRafR/bR8vrk/U0s0FmNrQH003K\n9z69fpjZoAbPb5iZHWZmw9z9X42cdzvLWivbrIO0dPv4MrOtYU196j5jZtt2PVbfMLOdzWxId48V\nM3vF2nx+eZ7nQ/pgOZsBXzSz48ysoQ8IyGvwu8zsqEbOt2b+TWNmm+R7n1zb1/pFRhrPzD5pZgea\n2U6lYX0eFOWJvtDMTjWzt0N9fwzZbc6mwC1mNt3Mjiym7ev1yOVtSrS2PtPMzs1goGHnjpmdbWbH\nmtnB0Dd/nnmh+QxwoZl9Mod1uW0zEHy/mV0EfNDMRnQ1TSOY2WBgdh4Pr2zQbPcFDgC+a2ZHmdkO\nDZpvG2trm7WTjo+Y2euBLbo53SRglplda2Yj89xsukzrV83sRDPbuy+W2Ula9gVOJo6VNxaBbh3T\nDQTeC1xmZh+rd7pGyZuhc4HPm9lHm7icfu7+d+A24lGfHzOzTzRo3gZsBjwLfCqvl4c0Yt45/7cA\nV5rZ2xpDMd4xAAAgAElEQVR4bSnP/wjgy/l//IpGz7/dZarxRPWY2VRge2A7YLm7z14LaRjg7v80\ns/2Jg/nTwE+As/O5wZ1N2z+7ynktsBMwEcDd35W/Wx8FP8OA9wAL3f1PZtZCXNy3Bv4IfLkROT1m\ndijxWLzjiP11Zm/n2cXyhgDPEzmimxLPWf4N8APge+7+zw6mO8jd/zOD2rcA+wFHAW/P5zo3K70D\n3f3FPB7eT3Rm/k93P6eH89sSeL2735bf3w68HvgXsMjd/7NBSV9r26yDtLyHCGSfAv7k7lfVMc02\n7v5Yfp4DGPAQ8TzvxzuduHdpHeLuf8uA6ijiEZFPufu5zVpmJ2kZnz0yYGYnAgcBK4lzpcNjJa8f\n/3L3/7XohP/zwJPA/xBdfzX1Gpbn+XPAYGAI8FVgKfDDTHtDcqnzxmUe8Cl3X2FmWxHX7dOJPm3f\n14t5DyP6qL3J3f9sZtsQfdiOAB509yt7mfZNgBeB/YmnUj0KPOTuX+nNfGuW0UJ053YA8HvgJ+5+\nd6Pm3y5316siL2AmMCA/DwV2zAPpS6VxrA/SMQy4Adi5NGwL4HvApcCQjtKS6f4JsEN+70d0y/N9\n4JY+Xo89gM8SOVuvymGbAG8FLgeO7eX8PwzsXvq+ObAEuALYqhnrSQRy5xJ9Qm5TGnZW7pvJwEbt\nTLcN8FfgfTXDP0UEuds1aR8MAb4FvLb0/bVE35ULejA/A44GvgxMLQ3fGfhoboPdGpT2rXObndaX\n26yddJwNvCI/v4r4E/sJMKeL6Q4GrgIOKg07kghQLgRampTew4EfA6Py++bAXkSH9Jf3xTYrpWUf\nYG7NsbJbHitXAOM7mG5j4Ezgc8CrS+txIjCn9phoQroH53k+HXhlafkz8xg/slHXljynZhHX9+1K\nw7cGvgl8ppfb/3PAOaXr1WCie7PLgcm9mPfRRLBbXGu3Ia7tNwAfa8B2ORE4v/R9z9wfNwNvbeb+\nV1FstewGLMkcr/919/8B3gAcamZfgL4p5nP3p4FHiPoWo81sI3f/K3As8GrgvPbSkjkz/0s8SeQO\nM9vW3V9y99XufgiwuZnd1Ifr8d+Zlr8C/8xhzxMdY/8P0NvioS2Az5nZLpkL+QxxwW0hgsmGr6e7\nPwfcD4wBDjCzjXPYl4E/AQcSxR5rZNoeI+44P2Zm44tiW3e/ALgOuMTMNm9kWnP+fyP+ML5gZru7\n+9/c/T7gY8BzZnZxN+fnwD3AfwHbFHVr3P33wO1EDsfu0Lv6cHkO/pnYZmea2Wv7apu1YwzwfTMb\n7O4Pu/uPiePsYDP7XCfTPUzk5B5uWUfM3W8n+iUdQKxbM6p53JnLuNrMNnP3Z9z9l8SN0EiLZ373\nleXETeU4M9sLwN1/A3yHyH17Hbz8WHH3F4jg+VngKDPbJM/vW3L4q61UVabR3P1Z4L+JJzcdmOf5\nM0RQ+Wci17EhxYJ5Ts0mApZNSsP/DHyBuG6P6+G87yeuwX8jctSLdbsD+H/07hq8nMh9/nSeG48R\nx911wI5mdnhPZ2zxaNOHiHUvruW/AhYSN6qTm1q9oJlRo15986KUw0JkiU/Lz/3zvQX4OfDuJqdj\nIDCo9H0qMLacRiJoWAq8p2baLYk7pb3z+3vJ3CygX2m8WyjdPTdpPTq8ky1+I+5+f1Bs627Ov3/p\n80eAd5XXk7g43gfMauJx8iby7rq0bzYl7rDPr52m9D6inW0xHDifDnIvGpT2KcBONcN2B74E7Nnd\n/ZrbeLN2hh9B/PFu09vjhwh+NgGGraVtVj7OPkvmEpWOs1cRuWCH10xXPt/GAK9rZ96nAjc0OL17\nkDn1xTFau12JYtnPAJs0a7u1sw1eAezSzjgHETnsO9SmM9+LnMZRNcM3I3JCZzYp7eXz/JDSeV4s\nfwjwbaJaTG+WU8yvf3vDS9vgYuDknsy7g3kWy92SuEF7VzfnXT4vtgYOrvn9FcAHgY/0cLsU15VN\niCoeb6n5fSSRaz+ldn817BhoxoGlV9+9Shfpfl38PgE4q5npIHLkDgNOIO6oimJhq0nLAcDH2zl5\nLwJuBPYoDduoZtojgQ81cT2stJwvAV8Btq0Zp0jTa4kizP7dmH+xHgOAfTv5fYdc/hYNXq+diJzC\nQfm9CP6LdRqW+2CHUlqMCAT3BAbXzjM/nwN8vgn7YWB7F/X8PIgoEjyji3kNKq3nm4F3Aie2N8/8\n/gngvT1M9xalbXYVpRuY2gt4o7dZB+npTxQJDajdvkTjuZPK61ra5xvlsbBlOf012/9GYEaD0jmX\nqPt1M1H/7GXnW37ehshZPaSJ22zNzQyRI1QuXqwNYj6c52k5WDBgWyKn71U18yyO662JnOi9Gpz2\nYv6jyWostfs130cC3wDG9HJ5RxGZCVd0so32JHLBunUtI+pWnwdMAzbtYD32J4rF67oGl6bvB3y4\nvW2Xn3ckbvAO6Gaax2R6diFyMcvVGMrH8cQ83rdqxjGsotj1WBZxvpTFAJ8ys5kWLVHX8Nbn7K4E\n9stKvA2Xy/kdkdV/MXClZyV8zyO5lJaniQv00FyP4tF2NxFFtV8zsz1ymn/VTPsgsFdWom3Geni2\nkvo0EdTtTrRo26g0TlHp+A9E7uNLL5tRO7JIs9hfdwOHlNZ9ze/59R9EPZ1utWDsSK7XocB/EH9E\nl5vZKz0aqWzk7v8ys35E0dE9wLOltP4nUSfveuCj2SCmmGdRDPc54MGsjNyo9B4NXEMcD0W3LP1K\n4/wfsY92zQrKL2NmWxD1aPYxswNyHcYAZ5vZFeX1KK3L7cCvu5tmM3sH8AszG5XH/P8Rx0hRzeBf\n+bk4lhq6zUrp6Ff6OpoIZnfO3wZArHOmZzkwpWgNWNrndxJ1yH5qrd38eG6rYv4XEVUSepveS4jK\n8JOIP8WNzezNxe+l8w2P4rIvEkWyDf//smjd+a88Fn5IXMsuMbMP5vJXm9lGpWPlm8AP3X11KY3u\n7o8QRaFnZrF8cR3zXMafie37dCPTn/OfRJQmXE38L2xe7Ndct42I6iU/IM73HjGzPYlc53uAV5rZ\n0kzD6uK6lsv8FXAJUZza1Twt33cjtr0Rrde/ksWlLxX7KCd5ELi/vP07Uzq+bwPG1fzmRRo8qjFd\nQOQ4dseTRMnHfwBbe6lxTc1x/F2i9Gn3bs6/Ps2IFvVq/ovWOzMjyu2vJoKFa4HXwMuLE4kWeQ3J\nAeogLQOJYpL7iVZAQ2h7F1S+Y9mRrNCd398C/ILIKfsOkeO3RwfLGUXewTVp255Ja+vFH9NalLFx\nO+MO6MH8bwDOzc+DiArCW5bXMT/vQS8rqJe22ebEn+b+xJ/9BcQde1GxurxvRtNa2f59wNz8vBeR\nQ3l2eb1pvXse2MB9sDtRHH00kSPwIDA0f6vNZdyFTu7YiSD9DqIe4dtz2KbAT4HLOphmUA/TfXGm\newgR+BzUwXgN32a1+x04Kj9/Avhq7TFR+v4Gsvgov/8b2bAiz4E/0E5l71zHXl1P8vj/FFnET+Qw\nfgU4tb3jOD8PppRz3KRtN40sKiUakSymlMNTk54hpc+vJIveifptV1KTW1cad1safB0jchgvInLY\nDyHquF0CbF4+7vLzqyldg7u5nN2Jm65ZpWELgZ91MH67JUodjHsg8O9kw4jcTnOAr7eXXuq4Btes\n94eAz+XnYbmvR7YzzcjyedHF/Ms5tqfntWUB7TSQKh0PPd7+XaanGTPVq+9eRGukhaXvlxN/hLvT\nhLL7dpZfBA5vIf7wBwPjibvB9+dvewGju5jPbNrW7foiEejt1Rfrkct8E9Eq9JNEBde7aW2d+3bi\nD6jbaamdhig+PJ8IIK/P5dxID4OJOpZ/WP7B3AXsSvxxbUcEd98pX9RqL8BEYHdr6fs+RG7W2Cbu\nh12JG5TPloZdQeQIvyyQ6Gif0Lae1Icy3Z+mNWjdjsiZquvi3Ul6a7dZEdz9FxGkfDGP74vJOqRN\n2m7lYONM4Jk81oYSuU+f6GC6IlAekOfvLDIAzuHHELlPDb0pJFogjiGC7E1pLSZ7H72s/9XLbXcq\n8AJZzEzknB+Yx8o5HU2b4/wo9/MFxI3uj2lQcXUX6d+IqLu3ClhS7FfiBvULRLWAoQ1a1og8d76V\n161y7we3A//dy/kfADxB9uaQ16tReS59s6PzvbNtk+/9iGoJFxI3L/9G3Oz9N/Ddnl4HSsdt/9zv\ng4k6tJ/IbbR1/n5o7bWiWS8Vxa5nysUPWRw5lCiaPDgHn0G04DyHqMfRVO5rivjmAv/p7s969Pn0\nEeAdZvZF4k9l+y5mtQx4RamV4seIu/mZxInSVNlC7QLiD3khkf1/p0e/TG8gArGfefd7nV9TtGPR\nc/3GRDb9q4iT/xJgBlH02jClopD9iG4P7ica0RxH5DD8ibjjXk5cqIu0vpRFTWdnMeY9wGNmdjys\naaX2RyK3plleIv5Mdzaz1+RyZxB1Xn5pZv3LLTHb2yelYu9tszhoDnFBP5Aolh1EVAdooVS8212l\nbWZmNtnMdnD3s4ji372A3xLF9SuAFe7+i54uqyte/AtG0e61RB2uyUTfi0uBw2pbYlprn5FG5OC+\nldj2h5ZGW0QE1Z32P9kdee3aEziFqMP6nLdWQ9gE2CrHu8mi77imyX3oxXePPsw+CUw3s609Wrj+\nlAjY/q92+rwGvokIps8lAoZx+fmvRKvi4eVjttHc/V8eHQQfAbzWzE7zKJ78GbH/XiByoHosz7vN\niGB1DyJnajVwhJntmOk4kjjeujPfovh1e4t+E39E5JIeY2aneHiUqG/3me5cg/M6UIw/l8ih+wxx\n07M057cH4EQpUrd5a/HurUSduWfdfRXRP+h9RFWSe4D9S8d4U6mD4vVI+U+EqNz+NNHR7Awiu3qB\nu/8oD7IT3f2GJqfHiIvwV4Eb3X2RmR1DNNT4NvGn9kbgUY8uFtZMlxfDfYng5lmibsLFRPb1UiL3\n4DzgYnf/aZPXY0fiYvx7d39/DnsDcTe3jMgy/7xHVw/dmW+xnhsRF9dVRNC9CLijOMnNbD7wkru/\npwHrMgZ40t2fM7PtiWLf77j7F81sLJGj+1Oic+X/teiG4fnS9MUF6k/ufnrWp5pG5ACPJboDWe3u\nx/c2raVlFttpN+DvRAe6mxF10B4Gvu3RxQRmtqu7L+tifkVdoolEDt0K4q7/Q8Qf3wyiI9L+wPXu\n/q1epr/YZo8ROU+r3P0MM7uAOP6Pc/cna9e3N8usWf6a+ZnZx4k/r3uI9Z5MbMNXErme73L3G3Pc\nzdz973kezyCqPrw39/l3ie5vfkT0K/esu5/cqDTn8ncn6tUtdfd7SsfBkbnM/kTc1OMObutIw5qb\nGaJ/t5eIYv9riEDteKIY+k/FcZXTlbf5TsSN33fc/Rulee9LdIcyA/iAuy9pcNqL7fV64sb5d+7+\nCzMbT5QCzHL3ebl/N/foSqo3yxvu7qvM7I3EOp1BZCycSRxrt7j78u4c36V1OIoIpv9C/K/9G/C/\nRI7Xpe7+b71M+xwio+MMd3+ktN+HA5cRx9lJvZj/u4nGcB+06BroHcC9xLm3L5GreX5v1qFbepLN\np9faexFZ7j8kDsbHiDppuxD1p+YBE2rG74uOfN9P5PzcStSFKDqrHNxRWoii24eIbhgeIVoqHktc\nUL9NFJsd0QdpLzqnvJgoPl5TX4vIyRpKTXcBdc63vK5ziQvhcCKAPbI0/9mUuo3o7f4i/pxWEgHG\nQKII5rfArvn79kSO4fm0FiG8tTT9h4Gv5OdRRI7KkUQOxDuIIKXhxxZRl+v+TP/VuR9G5DH9OVq7\nvumsK5pNSp+LzrkPJFoCf5IoGh1MdKVzL9l9Sk/Wo2b/fjK326ZEsc4pNfv+gkZvr9L8a4uBRxNF\np7cQxVbnAPvlb68pjXcs8Lb8/FoiN3QRrV1zbEy0XJ9J1gdtxjrQTqtAovjwX8TNRzGsqdUxiG5L\nLgFOI64FNxKB5WeIAHd4J9O+M4+ta2mnuJMIDhfQg/q4daR7EnHtPYMovjwlh+9F5Kb1uhiYuCna\nOed3BnHjfhqt3TTtT9R/q7uFLVEaU26h+ysiB7c4fm8nbqj3z+3fre6Hao8Xot/L+3PeA0rr9Sng\nqu4e3+2cd0cR9QLvIP6b7yRaeI/tLF1NO577YiF6NXCHxV33R/Pzn4B35OdXEUHSKU1efpHLu2de\nVIqKwofTGjy8hsgxaO8iZ8Sf6y20Vo59DVH/64T8Pozsrb3J6zKK+OM9Pr9fSuRwjW3ECUhr4HRe\nbp/byC5niKKmrWnbb1ePl1m+8OU6/Y7WLgI+Q9z5jsvvO9D6JIdX5rifLe3Hy4k/ukuJYHdu7UW7\nkRcooljn55mWWURQvyCHjyRyhHfoYh4tRN9TxVNNxgBfq9kPc2ntO6rH3TzQtjuMQURQdybxZ1Ts\n31cSjy1r5vFbTsc1RFHqGUT9p8FE/c0/EIH+moZMOX7xNISTctwDiCoIUyj1Vdisfd7Feu0JfKGZ\nyyWKmofn5y1pW095qzwH3p3f31kzbbnLksH5eQJxE3Uqrf2YFcfdFCIHraGBHXH9+mmezwcTVST+\ng8gdhKgPe2gDl/fN3C6X53JupPXGuO76e3mufozWBh1bAT8o/T6UuOEt+l3sMKjuYP7lbpoOLO2j\n9xEB1/6lcV7WCKwb89+I6Nrr2Pz+NqIudpExcCe9eDJGr/bV2lioXt3YQS9vSXU60VLwHvKxJ0Tu\nRgtNbClWk4bDidyQi4me6Q8pHcyTiD/mTg9o4u74vWQuC1Fk9SN6WZG9O9szLyAziFzGY3LYl4ig\ns0eNA4hgpLhrnk3kdH2Y+IM9uzTeXWRQ3t5+7uYyBxNFF+UcmWvJ4I7IefhULnPXdqbfOS/UZ+fF\n8G1EPZniD+x7wGHN2hdEjtGetLaMfh2Ra7eEaIjTactRoo7cMOIOfySR47MZUTz0/tJ4n6X1pqhH\n25tSC8e8cL+N+KN4hLZ9ed1BqbFCb/ZvHWm6Kc/FNxAB8fl5TPQjimI/Wpv+/DyO6F/sAiJAnUTk\nvLyLJvWv1dPt3eB5jiY7zCVyhQcTpQflPgc/SqnxTjvb7nAid/ZbRLH1ECI3aA5xjV4T3BG5o+Ma\nvA475Xk9Ks+RpbmsKUT1nJMbcezlMfKdPD5OJFrcbkrkpD9PtBbeqDv7icgN3oEIjA/IYTfXnD8f\npbXxRHf6CS3fvNxFBJ9fpzV38X25v95M28ZV3dpGOf/FxE3z9/JaUOQEbkXcZH2t2edHh+lbWwvW\nq46d0/bAKwKnmURO3aml3+4CPl763sw/kXFE7sp2xF3vk7n8icSd7zSyCLV0khXv2+UJbcSd7VW0\n5vLtReR4NKVlaE063gC8IT9vTgSYV5HBC1G3bo8eLmPjvHD8GbiwtLyvE4HTsXkR+0qD1mk3orXY\nSKKe5aLSb9fQNrg7D9in9tjK77sQxZPl4HMLImhoSFrb2Q/b1Az/NDAxP88igtVOnyqRafxk6Ti6\nMPff9rltfpfrfTRR3POmXqS7/Mc+n+xcmNae6u8kKswvavQ26yRNI/P8a8nv2+XyX9YZOe38QRK5\n5ZcQxd2b5Hl8Z3F+NCiNPX6KRxO2VzmH5gwiqB1C5HjdBkzP324CPtnBPHYgbkBen9+vzX3Qn8ix\nuZJSp+Y0vvh6MyKALIKit5HP0Sauo9+nFx0f16Y3z6ebiMD1XuKmrx+RW/Wyp5J0tu1pzT3blHjm\n8FeJwHQX4mbue7kNl1PzRIjupJu4YTkzl/MnIje6yMk8i14+C5a4Wf9kboe7adsVznHARc3a//W8\n1HhiHWVtO5P8JpHN3p8IDr5D1Bd4jMjp+rs3uGJzTVqKCq5FC7pdiPpil+XyP0483PhkorPOF2or\n0JrZEUS3Dw8RfyBn5fgtxN3PTsQd8rebkP41DQQsWt0eSuTevNPdf2Zmw4iAYFfiLvHWHiyjXJm6\neMj2Ynf/cA47iMjNG0tUrj8vh6+pkN2DZfYn6nX8jtgX/0dcGB939yk5zr8RuQs7eTwTtrbC+A7E\nhWd57tdrge+7+6fN7AQisPpEb9PaTtonEdUKvkfk/s4lugd4B3HzcjFwkrv/vIv57ET8EQwi/oCG\nEHWeBgFfI1pyziIa6Nzt3WwAU1rO5u7+jEUHv8OI7d7i0aKu6Ah5F6L46x/u/tUc3rBt1tH8zOwr\nxB/uLR4NYg4m/uw/4tlxa8315DbievKku5+fjVbeQxw/FxLFtN3uoLmD9B5D5Mbe0tE8G92YpJO0\n7EgcB08RXRv9k3js4a+JwGxLIifqAeCf7v6unG5jonHT6mwVOog4Pj/o8fxVzOxW4KfuPjtbdj7W\n5HW5jCjtOM3MXkWc/y8QgfoMd7+nh/MtrvUHENerJ4i6028iGlC9hbh5eJtHJ771zrc/UVz9DyLH\ndDci9+8s4sboW8AviWDsOeDXHp349ojFc15/kvNdDDxOXFeud/cv9mB+bc47M3srsR2OIq4rF5nZ\ntsSN4w0dTddn+jqS1Kv+F5GzdQtx8O9EVCgeSwRVZxB/WGeVxm9GsUUR/B9BZDsXdZhOBebl54OI\nysO7dTCP7fL3IrfofOJPdwQRWBxC6zNiG313a0QDgA8Sf7o3E7l07ydycIqK5W8l/qy7XWTCy3O/\nBhABxmJKOTfUFIP2Zn/Rtuh7MdF8v6gz9F3g5tK4XwXeWE4rEUzfQxQ9L6e1nuEuRHcGsxuV1naO\npWFEPZ03E0WFl9JareA8Igey7ropREB+Gq0V/3ckAvcv8PLny/akocS25HOMiXprx+U5+D3ij6Kj\n6Rp6PpKdSefnM/KY3oi4ibgqv4/JY/yCDs6Fj+e4h+RxUZzDuxG5HOX+63p9LuZ8z8/lblXzW8M7\nS+8kHUUO0xyiC5Arcvi+RO7rmURO2Ma07Th9ABHUvJPI9f08EeB8j2wEleO9hyY+sjGX8WpaSxo2\nI27wD8l12524pr25Acs5nLg2XkTk+v+Q1hzhyUSL8t17MN/DiBuQP9BaL21QbtPLiSoUPa0iUVxb\nLicfOZnb5N9L+/EeSs+V7e6y8vzZkqjCswtRd/WS0u+3AF/sq2O607Su7QToVbND2mYnb0X2yk4U\nj5yRw4e1M13TKjYTf8C/IYODHFY0qZ9L1DV5Y800bXpmJ+6cdi8N+zrx2LG+2q5PE83ndynSR+Qy\nPkXkgv4PPSimo22gtIAoPix6rN82t9H1RNDYkE5XicDozDw+9iX6ylpUs31vJ3IMX7ZPiJzf7xA5\nZJsQOWb/BN6Xv7+G6N+pGfvhTZn2b+T3gUQO6pw81gdSqsfWjfluUp6GzAHOP42XPZO3G/Mtngjy\nEeLG6uul31qIIGp+Hxy/04hi3rFEDsqNueyf57FcdEJ9G3B5abrX0HoztjB/L9ZpDJFDOy/nMaqB\n6S1XIxnHy1vrD8/93Rct34tzdDCRi7OMtvVRi+DufEpPesltYkSO4z1E9YpD8re3Ede9TxAB9W9o\nQj3UUloOIm6slxENDw4kguXjm7CsL5Et5XP9P01cv4pi1M27Ob/ifB6U5/n3iSL/4qk3A4jqAFfR\n/efJ1rZ+nUBUK3gVUYfvH7m9vkl0m7Jm39Y5/xmlz/9JXDd/Qlyz9iOKeS8jAv3rupP2ph7zazsB\nepV2RtvWPFvlxW9xXkDeXRrvRqKzw2alYzgZAOX3S2itn1DkFA3Ni8sFdNDyiqhUXjRKuJrIOSty\nlY6k9KSJJq1H8SdfFNP9iZdXiD6MyKLv8UPFaa1IexbRuvBfxboRd9aXk/XtGrRe++TF61PEXekB\nRI7B1eX1yAtokRP6TqIbmWKbTMjtsoSo93dQpnt6e9uwl+ktjpn9iSByJpHDWLR660fkElxJF61f\nu7ncnYjA7n09nP5oWjvXPogI3P9Sc260EDdd729EmjtJy+uIXO4v0raS+Y1Ey8ji2lEOTPYlbqiK\nOq9HEVUhphb7lgjuvk2DKtsX50PpvHglpXqzpd9eQdxQNfUJE6XtMoDIad2DCPgvpnQjR9xw1B77\nRVo3JhrDLCL6QiyeXHJgnvNfBN7SiG3XQdrHE3X6ts/X+URu9F+IBkK7NGh5BxBVa64v9kuemzsT\nNwAbd3cdab3eHEpcBwcR15sbyNbGxP/N9sCOvUj7N4kc2VcTwePROfz1mfbP9eT4Jhr03Uzkhn4u\nt8chxM30mzLdbya7DiofN2vztVYXrldpR7TN+bkReG9+n0E8PHn7vAjfRBPvDPIC+AHiru01OewC\nWrO3i5N7HKWcw9IJXLzvlSfUn4k/9LFELtLlxJ3nbykVZTRhPYp0FA8WH5on5R/I54MSF/leXRRz\nnrsQf1KbETl0n8r1u6Kd8Rty0hN3ix8kHweWF7QzibvelwXaRPHzHDJ3KdO9N1lkS+RmtCk6bkAa\nR5Q+75T7v9z31QpaK6v3I3OSGnwcfJioPzWwOxf00vSfznOueE7t6cQf6tj8/h7ayUFvYPrLz6Dc\ngchZ+j6lRiVEbsoTee2oPQ+Pz2l2zu9FcFfccBm9fB5xTXrLz7D+EREQXUepIjytAdM2RBDxip7s\nm26kqahXeFF+34TWZ6geQuTC7F9eh9J6HEnk7A/IY/YK8rFieU7t3IT0lq+rryFy6Y4vDRuQ58tp\nxDV2anm79nCZO+b+aiGqNqygteXw/sTNw9Y9nPehxA3dW0rDTsx9/wUiSOpWY4/y8UI0oLqHyEk7\nkrgBeoDW7p160qXJZFqLn78B/D9K3d4Q/yvfo+ZZr808jru1fdZ2AvRqczEscn7Orfn9lLxA3kzb\nYpamHERELtBZZEtbIjflL7S2wnotUWG/ozp1E4ne299N3M3+hsjx2CpP6M/QWuerGZ22FtvzjZmO\nA0vDtiQanfx7Xmze2IP570PrA9bPJf6gBhCdwV6ew99N9GJ/eCP3V80FrbYD6DFEbtg1xF1wbTHF\nEEpFX0RwfiMRnNxJVAZv2D7Ii/aC/L43kTN0O63B6OuJVtUfqGN+W9BOVy11HAOTgO17kP5yC9iD\na9D8G0EAACAASURBVH47gyjC/y6lm6xGH8u03uz1I+oeDs999jXiZqX8jM7TOjlOaut2Hk0Edyd0\nNE1v0pufDyVucLYkAqPbKRXHltat7q4sepGu84A5NcMG5/l6HaXi9Xa20y9L53p/IhD8MpFD9Bsa\n/Ozf3Nf/QWu/kltlGpaUxikHKsfTy6oARPD4G+C80rD9iQY2VxPXybqLzGlbF7QfkWte5KBtXPrt\nAKJHggndTO+Wpc/vJoLRNxIlW8cQAe+LRA770NK49Ra/voK4GTqP1uBuAXGNLI7b4XlMD2n28duj\nfbq2E7Chv4js4/3yBBhD3AUMJe4YrsyTqrjzKFfqbXZP7GNrTtB3EndB1xItco8u/TYcOLD0/RO0\n5jgW/Tj9vrsncA/SPIpS/3NEccuHSuko+hnanMh56VFxNvFn9VMi9+/c0vCT80K4NaVKvH1wDJX/\nxF9N93qAf19ewOa1N79epmsL8skR+f11RKD/eTI3L/9A3tTFfDbKY+o8St3Q1KazUekuL7fmezn3\n7GDa5qI06yZrI6JuT7nD3p2Ihg8zqbm5om1AWrt9+tEa8L6D7Ceskdsq0/sZ4o/w1Bw2lKgjeDs1\nAUIzthvRf2S5Dt2c0nVgYM245aeVtJBBPBHELSByrwYTxW3nEnW3diXqbTWlTl1eP35TSvNWxE3E\nNaVximvZW4lWvXXXeyNuumqP7a8TfeFtVhq2TR5ra+ol1zHvwURQtGNp2JeBuTXj7UNNfbo651/0\n4bo9rVUtfkzUZ/442XckUeLU42J+4hp/BW2rttxO/D+fQjTwurEZ+78hx9DaTsCG/iLK7S+ntS7U\nlUTT7IuIlqgfIAKU8p9Kn2X30rZ4Z4c8gdY8iilf7yf+sCfk8HNp25/alkSF758BBzUpnf2IjlVf\nQ9bpIYKB4kQvLoSvpYf9atE2R+KrxHMAP1YadiDRwvM+2j4mbK3XuWhvv3ZneC+WM4RSoJnb6PNE\nhePyjUO7xzTRcGgs8ef2WSIgH1QzTrs5x03abi9LZxO22ThaK6pPJ3ObiCL/M8gnvuQ51e3GB826\nfuS1YD4RzBcNNcaUjoMPUeosuUlpmEhrf5RFCcNb84+4HOx9lbZ/2hsRN69fBiblsG8RuaO35jX4\np9RUr2j0tqQ1R2ir8nGd58G3qAkmiIYddbfkJ59Gk59fT+T4FRkH1xIlRj3qS5RoAX0ZEZiOprXa\nx3ii2Lt46ss+ROvYHvW1R1SPWEhr9YjpRG7apflee33odufDtduq9Ns3iNawTb+p69VxtLYTsKG+\naJvLcjZtm87vWvr8VeBTazu9XazL5kTR0Mfy4r4RUQ/ouvz99UR9iovpYUX2OtMxgAgibyf+HN9A\n5C6+Lv9Y9iCy67sdCNC2DuTBxN3iaKJeyvml8fam7eO9etPr+1BgdA+n7Xaw0cwLFG1zkt5M3FHv\n1MU0A4lg4Mbc3sN4eafGe+UF/YAGpLFfb+fRgDS8gahyUVTGfwtRv2cBETBdRVSLGE0DW7E2KO2f\nJfpALL5flef9Dvl9kyYv/0qyWJWo8P8ikcPTkmm7kcjV+Tbt1CUlOnueVlxviRKAD9J6071rXlua\nVqey9jisOW9eSVTJ6fK5yR3MdwjR7dTr8/q4nKg/uoDMESaCu3u6u6/yXF1E3EwPJoozf0lkWgwm\ncrm+xf9v77zD5aqq/v9ZNwkhJCEJSeg1QggSaug99OgLQYogIBJApQmISJMizUhiAAGDBJDQeSEg\nSC/+4EXEQlGaiCiiIAqigoKIlPX747sOs+dkbnJn5sydey9nPc95Zk7d6+y9z9qrL60LT9BAqa08\nDcmdWwcJPR9QXW2mEPeX3JgcWuTzWzKH2o3AR3nrbPFF2qfhSPq4uIXtz7NUUxefkWnzhlLtLD80\nCMT3kZ/beOTcP63ZNjvDIf6PRBrDG5CUuwtynr8OaQx3auIdO1BwRBqVuDqS5Kchf50v18KrgTYH\nIh+lr1JdT3ae5kdU+3OFZttv9UYX0xog6f94ZOLOqkukzu1DUXBPw5ogZLpbJxvjHtA3+5DUx0Vm\n0/2ppC25hiTjP2JWPtsGPNOFdnDg+ApKLJ0dvwAxIzXrzxaMz/HIAnITYuZ2QkzxtkFPJwVtOCy5\nJ//9jKJGEA9iEJ8kcUFp09xoiGZTEUwPRSbXO5I5PxYxxVnmg5uzc/XMg6C1t6BgiMEoAvYhJHgZ\nYv7WInxeG6FP+e+TnI8msGwL+z7fVttpRae4thuBcksGo5pBGQjsmuwXbe4ZibQhE4rEO48rkoJH\nodDwpykoND/fNpJAJyCpdDDSIN6E/ESGIA3H2Fr4zuf52eJqyFcnC44Yisw+KyDJ/koKTk6JtDdT\nke/ewFp45Y4tgvybLgSW62x8it6SRWO+c7SruCTjug3SYP0CMc7r1Wh3SZKSPg3gfxASPtasF8+C\n+zH9buYqpRZzew6RdDU5vhVwHzXMQxQgvM1nzC2+8awY/N5ocU/rIO/b4n5bN343QkEtDyfnPoUi\nPOcS6LpKUxGD8l0iEKrouUGLNcVBF/6P0HYjk/PbwP6xPxAJD99t4NkjkVZzKFEFKehuJlwOQj6i\nVxTxLnXg1dWxHdJKPNq1tR2BcuvCILWmosQKSMt0HAUXqM7jjPzyzqSBbOVdaQMxWC/Fovc0YiKX\nQpG999C4L8d6yHSbZXs/BGnQLkOaiCeRqaFf7n2bSTuwIBVmcjiKwsubH7MghLnq2aIqH0eT0yyg\noIotWjDOIwKXZWK/yDxeKyONyxoxxsfHApsGUDS8KFLtt7ovKsGVnh9LBAF115bvv2SOD0CpSi5I\nz1Fh4LYApuTuXQaZxobUenazOFKpXnI2Su+TMVifQVq6PD6tCJRYP77Fycgq8B0kDJ1AxQdrMipT\ntW4T7QwoAt/keUuQRLM2M4+72N7pKCAjY8C/GN/WxNjfHplpF6kHF2TWnRY0Z9WgVwfGd5rV0l0I\n+dQ1Uq2iZf2CMjTMoQVpltq9tR2Bj8qWX5zbjEtGmJdHmq0lc+e3JZG4k+MDaTA9AQ065HbyrDSS\n7ePIjycrDXZgLCofR6r/o6nDrFCjrT2IkjooZcvX4pnDEPN4B9URYM2mjJiAGMhvoCCMuXKMIQ3V\n2VQSfM5V0iw/3sgs9QOSqOGCxmIZKlrCxQt6ZjY/NwTuTo6vHu9wbTOLdNpnKPrxTmozySuipNZ7\nNtNWKzYqSZ87qOHkHeeGowTUx7UIh9uQoDMYeBExTxmjcAAtDpSIdkYgRvJMKkEAE5HAdRIR5Uki\n3FGdMLlt7gooUOHmeZxvmqmhWuA8jOqAjCmoGs8lyP9thwbb2Bz5t2WR7qsghcH5ROYB6hR26Sb3\niMDxe8wdcDGMnJWkN21tR+CjsCGJ8UcUrLFqEqdVSTR1VGsGDglCmZqGR8cCvlGecehmvIeh/FPZ\nAjILSaKp6ecM4Pvxvwg/wrRUV6qtuAK4sOD3G4Ii+N4Gjs+PT7K/Drk6wajs1eezd07fHZlMTqUG\nA1MAzkvForFc7vhnqKOiR9K3WRLsgchZPXVWPj0IcdORsIjhPY9cNZK0v1EU79eLmEcF9neaWuQ6\nFIk4kySAJLlm2TjXdBJg5nYVmYy0Mfchk+cUFLCweTf3x/DcN5pVBzgbBemkudNGoDQxk5rtjybw\nTR3xZ5ETtlBS7SXTcSxivnQyjpkbwvr5c3XOh3yi3pXRejErxqdexq5l7hE1+mNSbn9E4L1hret7\nw5YR0hJaBGY2BTmTrom0Mfu7+6vJ+VHAf939n92Ai7m7m9k6KI3KYFTs/Re588ORFmyqu//EzEYi\nQrkVKgV1lbv/MvfsIe7+Zje8wzLAbihC7ELgHSQd/h0xc4+Z2UTgc8B+7v5BgW13uPsHZrYwckJe\n2t0PjnPmBXxMMR+WRKbBfyNH55tjXBZw9/8m49SRvZ+Z3YKYwf8gzdb67v6OmfV39/fimknAY+7+\nSrN4JviOB57Ov7uZDUCmjs1QSpi/dfF52yAt6avIfNMfCRMLoaSw30bj+nABuI9DgS/fd/f9zMwA\n0ncxs0GIsL/VbHu5to8Gnnf3OQ3eb8iX6YcoyvACZEq+K7mmn7u/b2YD3P3dJvHt5+7vx/+FM3oV\n39rB7r6bmS2ONKqPuvtBzbTXLJ5m1oESqw9z96uS84Op+ITd6e4/yt0/yN3fbiF+2bc71N3/lRxP\nv+UzkNVkq25aFxYriiakdNDMVgHecffn67g/pVf7InP6Ocn5sShl1sUN4pfND0MWgReAvwRdT8fg\nKyhgZpK7/7uRttoJHe1G4CMALyPfmJ8BZ+WYuoWQZmyn2LdWIhIEJSu58me0WH7RzDZIzvdz99dR\nlN1P4ta1YnsIeAulW/gQzOwTwK5mNrAb3uFFZDb4FfCuu7+EpPLFgdPMbEbs31QkUxdtfxC//wRm\nJ0xdR0FM3QgU8bipux+JmNWJwOZmtj6wdzB3HzJ1JhgL3OPun3b3fYBfAg+b2UB3f8/M+gfedxRB\nwLMxjnYvBubEQkrS1rtIk/M60qB05bkbIG3lbXFoA+RfdxnyPdwdpaJoiKkzs37pf3f/NdI6bWpm\nh3pA9i7xHm8XzdQF/Bo4xMx2SHDK+nWBTvDf3cyWjN2FUPWXS5FrwMXufpeZjTGz0YH7+/HbLFNn\nGbNkZrcB08zsB4HLv4B3zGwX5Fd1UbuYOtA7B74fALdlTF18J/1iLGeiVB9/SO81s62BA4L5axV+\nbmbbA9PN7Ggz2yzBN7vmaygA5WONtFEvDS5S0EvpoLs/UydT1y+jV2Z2J/CLlKkL+AA4ycz2bBC/\njOm/CWmYv4Hca8iYu/g/A1lkhjbSTtthfiq9cmtso1rdvjXwqWQ/VWFvgxbiQqNFO8FpASTZbxv7\nqyFt16V0ovLO/gPboWCEkTWe+ymU1mSZbuzfzNyY+UotF+92GQ3kSKr17q24fl73IzP4lshvL4tY\nOwL5wPyVSlRe5l9lSGPzCAoaSXMhzo57WlKyCTFE9yOz9/2IUHak+MX/9brwrGyO7Q2cFP8HI4fu\ni4nUKMmYN5ImIc1DeD5KmJolot0ACV4trxRCtc/TJ5D2MT2/LPJXypJsp2ldfohyQWYl2a5BGvRD\nkvuvpEWmUJQH7nBk1n8HaWP7BV04H7i01rwusP0ifM7Wo5MqKyhdx/kkZaha8A6bIaF0AvAYCvZY\nII8PDdSgRQFx26XPKRDvYTRYK7bOdlruHhHf0JdQOqyniMTJ+XZ689Z2BPrilltEBtGJE2byEW+V\nEetuwO3SHGHbKha1GeRyACX4rYiYjk4DIJDW4KCi8e3iO2V4roCYu6nUwSij/Ft7t3G+bEAlqnRA\nEK57iFqeQYA+nrunAwW+HBnnz0BJWNN6nF9oEb79ESOXLSIjken+f5OxyOd86kq5oO1QpHHqL3Un\nEZVcAN4dyGfvaOSL+DawVzIGfySEnhb1W0oXOqu0sTjyx/1SciyrmjISaUG/gxi9jRBzd0Kcm0OT\ndUPnhS+KSl4DCR5fiXO1cr61Iop/F2BXmvR1RIzoveT8qpLz5wG7tXAOHIkY+nVQcFT23c8lMMfx\nenzeNkFuP2m5x7w/WSOJywehQJTDSYT3PG714DqPtsbFO2TJ7a1GO4PI1cmezzNT5r1ffC9Zzr2v\nxvElKSD1V0/ZSlNswZCaLZAW61Tgf81s4/y1HjPK3X/o7n9uBS7xO87MNozD04D3zezQ2H8F+AuK\nkF02ubfD3d3MdkRak9U8fE9SU1XilzTN3S8o+h3SNjqDwNPc/ffIYXoQ0lZ1Ff4FbGNmu3bWZvrO\nRUHSxh7APWa2jMts9nOUVPkUMzvY3V9x91+Z2cFmtkbc8xmUZuQllynlSuDdeI9PArj7rBbh3g8R\n3Mxk9Ua0Px6YFWPxXnpDNtczSObmBmb25TBPvYw0Jgeb2UZmtjIK2nm9WYSjva1QPryZiEmYDXzT\nzD7v7j9FATl3N9tWJ+2nvl9XAdeb2Q5mtmhyTYe7/wUxnePNbHAcezfMs+ciE+KnUdTgK2i+r49c\nOl5x988l79sMvh0Jvhcgq8MYxEh+32WqArjazCYn91WZFYuAeJelUVWX7fLz2eTPOb9nDIQPzdO3\noDJ16TzMTHBfcvfrC8J7ITPbJP6vafI5exaZ/mYhy8KLQXf2rfWd5r+bebRl7v4gmtfHmtnwdCzC\nzIyH+0Y97xF0/yEUJLV51t8Zbma2fD245vBuqXtEPNOT/++jNC8XA0+5+/S4dBbSovYNaDdn2Vc3\npNH4MgqaeJYmzINN4rEd8ud5Ei0MqyOn0NuQ5Po7lK/rTBSJNICK1mVNpKoeF/sjCMmSTswi2b0F\n4L0Idaa0SPDuUtQu0nBl5rjJwC6588uR1IItcEwyPEclx76GTDOZBL8jChTYILnmu0grNi7B/1Ui\nJQparGZQI1VNQfiOS9r6n5g7WT3OLVB05vV0MV8eikx8DgkbV8f8PAAxNo8gbd0uTeC9HpXyS4cR\nKQyQGfbUOD4VRXJOSO5riSkGPkxyfTpKy3M5yie2eL5t5DaRperItAzXxv6iiL7MpIbmoij8A98T\ngDOTOflrpHHaBKWduaQVfVWjP/qhmqDb5s4vGX1ZM9UO0jaORfkop8ScWC7m7vjctXO5oDSJ+3Dk\nRnEzCtIZi9xfvo+0X8MQs/oE4WbR7HjF7+Dc/iDg98DXGu3/+L8Bc1t1NkRrR92uRLTYPYLq6PFZ\niHZuiFI0HY8Y/M/F+LSswlM7tv6UUAhYElET8BT6qK9HDsU3m5zN3/ACnVXnhYcpYvFQtIC+iMyw\nu8XvLohx+xMyYe6EfJyORZFtjyOz0O+A/mZ2PDIRrmVm67t7leNxBh5fUrPvAHwWWNwULPDjOF4V\neZrfz/57153FfwucaWZvuvvNNc4PAVY3s0nufkej75MH9w8dqI80s1cQc/NN4D0UiHA7sB8yxf7U\nzHYGfuzuB5rZVGCqmR3r7ieaghUeNbMN3P05M5tW5PzK+tgUrToLeNbMHkTarqMC36uR1nEymkPz\ndTiOMd4SmRzvDO3clsgl4TQzuxF4z93fyI9zHfA2cI2ZvY+Yojei7VeBIdF3C6C0Mo9mN3nBGqcE\njgK2dPfxCR7bAW5mt7v7S0nb2wNfNbNd3P1VM3sBGG2K2nzVzA5EfpWDzOxId/9HPLMpjVmi1QCl\nPZqMyvPh7j8ws6HAJ5G29g/ufkzcl6d/TUOq5QycZtW4bAXELD2GLA/ZveaCD4DfmNmRSOM4Cwkg\nzyB69lSGey1a0gy4++umIICLgNvd/TeB2/WIobsdadlPdPfbG5nnyfe5MbC2mf0SeB54K46bu79t\nZtsierOku79cx7M/MEWPvwH81t1fsyRyFWmNF0LuOs/Ug7tXNMI/QNUp/gHcaGYHuPtVZnY4cJ2Z\n/dob0KR7RTt5EbLgPA+cgoSra6goNR7w0EC3Yh63BdrNWfalDUm434z/c1AE6f7J+auA7VvY/mik\nYh6ApLTTkN/QhDg/CgUXfIfIOYYS+c5BxHEBpCWahoigUamxuh/y4zmTJBCkhe+yBJKqjiCnlUAq\n84YTHlMd2DIdMeGbxX4qoQ5ATrYNa406aX9VRFQ2Q5qvUwjfKBSI8qFmAi2s+yCNRabNm4oig1eO\n/TMQUzg8aaPIChAbotQy4+P/aYHDSKTF2xxpCzdCps6PdfZ9xG8WBHEh4UsT+5sjE3SX/We6ML53\noAU/688FgJ3RAv8kim7Ori26bF+KRxYY8wxKI5Qd3xkJfxNr3H8i8gkchIKsrkXR6Vmev6sQU1Ao\nvvHdb4BKmH0LaQbXTq5bgGrtVit86vK5+k5CAnKta48CZtSYZ5PQon4ZEcQT8/RoFIjyK1oQXJTr\nmxVRVoR7czguEeO6WP6eBvpoUrzLIUijejDVdMyQdnDDBp69HVIKXIY0i9laks7tLWkgGXXgtQ2i\nJ0Pi278g2vt8XFOTltTRxmHAQ8n+vkhj+ql8n7diHrdrazsCfWkL4n0zWuBWRg7wpyPV/xzg8ha3\n3y8I1xhkyhyFpJPzqRRQH4XMXqsk942mUnZoeBC+ZZPz2bl1gnA0lfV/Pu+QEsVFyRUPR0zROTQZ\n/YUWjHsQo3ojWuSzxT8tMr9UHodm3w35RF2cjNkyQTg3zV0/Czg6/h8Q12SmxakxpzLmbtcicKyB\n8wJoYX0hOZYxd2dTyfa/GjLv10yAnLz7eogRXTi+ke8Q5haUsf5emggkotq8s2r09TZIA71HMr6r\nUs2stKT+Z7S1BmH2C9pwHYlZjKQyCtJUX5DMjS2Sc8ci5u5CxJicWzT+0W83UzG/LhZjdEbaX63q\nt/yzEWN7bMybt0gCaaiOvh6du3cTlG1gJxSw8AIhVCdjcznwxaJxjt8tkECW0amxKPDlG0iYvpgw\ntTfQxmLJ/6HIRWP5+C5/SZil6STQZF5jluvTdRBjv2nsfx65fjScHJwWu0dQzXAuFGPwGIlLDWLu\nHqIPBUvM1Q/tRqAvbYixOw44OfZXRH5C5wHnJNcVvYgMSP4vhDJ+P4Z84pZEi/A5yQeVRdn1C0I+\nKQjh55AGK0u10C95r42QyfB/isS9k/dZL7YxybHU1+ZoYuFpoo3dgevj/2Bkon6c0NwVOU5U0pNk\n/bks0tilRdsvJCmWjpjaB5BJ7gBkupmONKqrxzVTUXHvJZL7CpM6iYz4SLvwE6q1a5tE+xljOQxY\nZD7P2xoxDW8jU+4YZNa7HbgbmRYb1gZTzdTdH/hl8+ZTSKvxGaSB2qcVfZbDp4NKipJfEVHK8S1d\nn5/DiJEZhVI5LFxrDka/7wAcWPQ8jWd9i4RWxbHFEQ27gJyPVQv6LBXsFkNa+4FICDs8jq+QH7v8\nGKIFfXqyvxvSBH2MCvN1KpFep+B3+ATSzO4ecz0TXJZDtXUfp0E6SoXxXik5dmLMp59l44M0/fNN\nNVRjnA9DjKehIK5nEVOafVsnA6c30TerIQvJ4ySl7hDzfjqKuJ9BUlWnjmenwtT6VKppfDrmb/rN\nbNLoO/SGre0I9NYNmdGWjv/nE7mj4uP4JZ2Y74peROJD2AtYF2kGTkVq7TPR4pYxd9OR5D0kIYaj\nY39zlOX/BTpZWOO6jKFoRY6qjNhugvz+rkc+ijul7xq/A6lT0iYX7IF8DG9O9ochZ9pXKSinIFqY\nFor/2yPfxj2jre0R0/9lJGk/TqWETbao7wO8CdwV+0OD6KVM+qeLwDU/R5Gm7qfA2cm8vpWkhBqV\nIuudpe5IzUGrI23vKsgv6nZk8h8Vc3gtwuzSyPyiOtDgSioapwGIKV4E+VTdD5xfdJ91gtP1aMEa\nGOP7X6JMHEo8/aUa94yI+6oYNzqhG50db+S7iHYuzuYU1TWZh9PilEBUL8zLxPfzCGKK90qu+y7V\n2pwFkYZsSMyzbRFN/F7M5ey5M6lomIcha0bTpely7zAKeDDm+TZIWHkO+Ho2XjSY75MKjeyPmK3v\nxv7uKHXKTrG/LmLINqvz+asif8pjon+ydzkzueZAEk1xg/OscPcIqk339wcNeB4xi6sg5m4WOZMx\nLdQ4t3NrOwK9cUNq7+8i1fQY5NvwEGKqdkZ+CV8n0kIk97VkEsWH/DKqJjE2ji2AFs570KK2FNXF\n6gciE8VyaPE7G0mTnwQWbVO/TkSmitWRqe5zKPv35OSaPIPWlfxo6YKxITJXZ9qUs5Prvg1MKehd\nBqCAiCsQ4/xjFPmaFU5fC0mVt6EFKCPKO8a1iyGp+wEkKGS1cYchrcplhGBR9Nyi4sO1NNIIfiP2\nF0PmpEvn12bMuSupmIUmIJN35l83AvgNMoc17C+Z9NmPsnmLIjn3j/l/CVosTs/6rxV9Fs/La402\nQX5U96CFZRzKnH/svOYwEtBeBPZsFa6576KDinZjP6SZ+1hy3cWEUNdCXNIazNcQghvyN30dadpG\nIjP2pWmfx7YvEpSej3m7YDZ3EcOyKWJ2xif3LljwO4xFwvKi8X0/Gsc3iHE/oqA+Ghrba8m3eToy\n098U832HRuYu0tbNRj6Nw+JdHkNawi8Cj1Jnhgda7B5BtVB3FBX6NBL5oZ4R+5+jIPre07e2I9Db\nNsLpEuXEmk74x8Sk/J8gJg+jCJ+luwmnzeLD+w2Kuss+lA7ku/Bjqn0nFonfjiCC05FGZiLyv9s7\nzo+he7KNZ0TlkiCAWTqPpZDWag5NBjDEu96CVPI3IfPFosjccCOS8i7L49Rkm+Ojbx8kkp4iE/P5\nyGS/RNoWFcJ9IpKcs+OfptoHcASRvLgFYzEOJT5eLvaXQCaec5P9dbr4rKlIyzcaCUNXIMY689nM\nIjsbNu0kbZ0Y45vVqD0XuSAsgwSfG0hMxRTP1KXCwxJUGNjVqZj8+yPGpGYQQO55WyNN1b61vpUC\n8e1AAt202F8V+dNdjtKD3EiipW31hlJSzMgdOwLR1OuBmcnxkTGnRgEbE1HlVBj8xeL8hUjw/mSR\nfZjDMdMAZgLYVsD/xv8JgVdD5j+SBPdBPy6M/4ugNCanxf4y0W6mlay3is7KyE1kOWRdOAkJ2CMR\nDbuPintGXT5vtMg9grmFuj1jHDJ3opFoXRxHtQa6T2rqPny/diPQm7b4cDIJoz9R6ih3zYIoKumM\nbsJpEyolaiaiyKXd49yKiHFLpe+FkKZueuxPQIzGKYH7J5CUMx1JhHX5adSJe8bIDEmOzYr3yc4t\njbQINZ3y62hjBsodNRotElPieD+kUZuUv6eJ90pNkMuhcP4HqZhlJwThPJEKk5N3mp+Ye+aeMbaT\na71fgWPySaShOYIKczeeqNHYhfsXBb6d4YZ8dsbE/lFo4T4sxvR+pJW8hQYcyWv02RbJudTvdHZ2\nXSs3tHg9hJjIB9GiMgQxc4egwJCvJNfPc+zi2/599FfTec6S53Zmtu6HFsCJaNGdRqJdLHqudTJ3\nbqFi4k8X4oXSOUKFOfgmEtQGI2b4METP1orzS8cYjG4BvnlN64mI+emPNIyXIgb5d1Si7utlv6RB\n4AAAGBRJREFUtoYjl4jMTWN9ggmP/REobVPDzHd8pwsjhv7b0WfLB/5fQwzkSCTgTavjud3iHkF1\n9Pi2SHO5ZjZ/EP1dtZVzt6dtbUegt2xUl2n5AjJXZFJClXN87r6WhVAj34GHgK1TPIOQnB7nUhPK\nIjHpd0DmqiPi+MbIvHcGMtGuiRberVqIe8ZwbY9MLxcRkZ3Rt7+gIu3VLMk2n+cPyu0fgzSqt1KJ\nNF2BHONa1HihxfGK+L8E0hrMosLcrZsSGzpxmocqU/7nCdNLC8ZhDNJwDAjczkXM3TAkIFxFLmp3\nHs+cSSUKO2+e3DPm3g1B3LdA5uaFGsB9nn0W588kSaJLizR18X9jKumOvo18qzKG4wQaiGJFbgMH\nx/e5bwH4zsts/T1kHjslP3ZFfRed9V1y7HJkVs00LgOQQJAGCOWFhyOJgApkxj4p5u8XSdwBCsQ7\n1fyuQKKJi+/8s/F/PRS0MVcqmzrbOwYJu+uj1CJn5c6PiLk2vplxQtr0U+KbWSbe7Voq/oHLxLc6\nX1ed+cyzpt0jmLdQdwxaU2YhQeHqouduT9/ajkBv2OKDmUpF5b1p7E9NJm63q3YDj9+SRCvG8U2C\nSM9V9xKZp+5EC+pSuXvORJqtUbl7WuUbuDbSSGyD/M6+RcW5fA7wdCPtI3PS7oihui0I1BRUuPzU\n5LqbgUNbNC6nAe8TKW6CKH4nFpq5mBg6cZrvpnmU5cG6Gi3sy6LIzfNQtOrzJCb+eTzHkLbidKpr\nnRpz+0b2Q4z20yTCR514z7fPqE7rU3TgUuqwfV6M75HJ+fNQdOTIWvc10N7CBeHdFbP1XNaIFvbd\ngcABsb9b4HNo4DOHxA82uX9ewsNqKP/kwxRc8Sf67Aak0VoIaQxvRMLoiGi37goPnbSVCg0HoGjS\nk6KtwxGN2zu+1Ubn1FqEABr768b3OwMJemOoVg7UrDjUxDxryD2CrkWPf5LEp66e5/f2re0I9PQt\nFp+foICIE6jk9MkYoQ+Zu27AJdVEZAzlRsi89bXctZkW0aiWcDuQBmAuXwlkkpxBpLBoAf7LIVV5\n9h47Jnj1RwzRtYRfH2FOqbONMYhpuBl4iWptzfHIrLEPUs8XXg4JSekvI+3NhsiHb06cWx5Jkat1\ncm+3OM3n2lwmcNw49k9AwRqLooVrQ8Kxvo5nrgL8gWqfmbneI/ro403iX7PPahD6ojV1lrQ1AzHF\nlyKBarvkusvJaVjasdGDzNYkPqUoOvpalJz27jg+GWl1biAx0WXjSufCQ37May76BYz5IBQocXC0\nOxwxoGcis/t/aDKJe9LWlsDB8f/zyHf7DqTFPCf6aaMm2hmFhLfZybFJSOCaQS7AZH592V3zjG6O\nHu9tW9sR6A0bcnS/A1gxd3zj+JgvoMFkkw3gshMKhngEqbU3RAzRlYQJpZP75pJwa32ktFBSR0Ee\nawchNCQtPkl17rgbiRxP9RJkxChmQQp7oAiuU5E5MXNmn4KCA76c3Fdk7reNyZncUBqZS7rSFi12\nms89sz/yP7qCJAUD0jSdW2//1xjrpwizVNofLXiPbuuzGm3fTWjMUQDScfFdTmp123Xi2XazdX4e\nIK3cd5JjdwB3JvupNievleuS8FA0zvG7KvKTfRk4Jjm/NnLTeZkQlJpsb1vkVjMxObYPYroaEr6T\ndxgb79CBNHNXU3EdGRd0uJH6r902z2iDINxbtrYj0FM3qiWP/lT7UeTLqRRadH0eOH0MaZxWiUVk\nBvKJWBRp224gV4KFeUi4+etaiHdH8mEviBzLpyD/o4OQf91uQTAfp/FAiZHxuzdi8gYiCXoGlcLY\nH8/jVtA7rhrEchwyv6WVO45GJucuJVWmRU7zuTbWQBHTC6ASOwcl53angOAf5Gf4YoxxqxPbtrzP\nop38d3Msqve5ZuwvixzOZ9HiFCF14t1Ws3X+mchceldsabu3A8/Oq8+T450JD630a14HpU0ZjPwf\nnyHcR5JrGi4Tlt2HtIKXEWlLqNZ2HRFzfREaKIlGxfXiZ0jDvBtaT+4K2vw7IoK4p88z2ijU9eSt\n7Qj01I3OJY9OJ0yriTcKR7+PCgMzGgVIZP4pnWrbaIOEG20MomLmm4TSP+yIcnvtHO+0O9JC3kgD\naU2oZrSXQDn5ZqCcTMMQwzsd+dx8r4j3yrU/BGVkvwrllzoEJVneCtgV+fntRg1foXk8s2in+Txx\nHYmibDeNfnoMaZqOR8x1Q4S9RrvjkUP5WSSZ5ls01wrts/m0dRoRtIT8Q1+jkmZiBZJgq56y0Saz\ndTwzTQczEgl4S8Y3czDVkftdrjtK9woPqyJt2R7JsbHxHZ2UHCvK9Ptt5H+Y0resNN1SdTwnZajH\nB+3N8p1+Ib6XVZL+bCqCtLvnGd0k1PWmre0I9NSNNjqz59tDvk4DkHbl/GAWRse5zxMla7rwvHZI\nuIMQo3UzCvTYPI5PRvmzMtPpQCpatXqcaFN/nVHxfxHEaJ0dxHgI0uId1Oz7dDYXqETjXRTj9Vnk\n/3JXnNspCGrdyXhp0mk+R9gHUvG/3JuooYiYuwORtmm+gRINzIGswklTyYi7q8/m1YexPzXoQ+Zz\neySqWrJa7roeZRqiDRoOqgMl7kJC1uPIH3UC8rc6jJx5sat9RzcJD1RSJT2QO74Kikod08SzM1q/\nKJUUSAcger9i7K+JUuasWMdz84nCT4l5ulHsD0W+oVN78zyjG4W63rC1HYGevNEDbPhIuzUnCOKm\nKJT7LOQDtR+SVLqcloTulXAzYrVOtHlN7GeMxeQglJ9t8PmpFuA+5O90DvLdG4YYrbPIpekoirgg\nRjn1ERofhHMmFcZ7AWSur8p6343zJyPsSyJNyR3Ix3AdZDZ+spkF6aOwpd88wfTG/+MQc7dJ7H89\nnQ89daNNGg6UguIryBryWyqJ0NdDaYjmiuKv49mFCw8J/Rob30sWiHYDEe2eXFt3up4a7XwCBTKd\nTSUVyDQUHDcHMcN1B2UgIeQWlKtupaCJH7oKIGvCLCT0FVnBpl3zrFChrjdu2YQqoRMws60REzXN\n3Wcnxzvc/YMWt70WKl12BGLqlkfJHP+GTJrLA7e4+711Pnc8Cst/C/iru08tDOlKG+bubmYLIuZm\nSaQNehVFCf4prtsceMfdf9poO8icsAhKVLo90gKcjaTok4FH3P2aJl+pVtuLIB+b69z9S3Hskyhg\n43HUx4ak12fd/ZmiceginlOR5nIXFN09GvmGHYeYkesQ8fdWz+neBmbWz93fN7P+aBz3AW5w9xvi\n/JmoXw+s9ztsJ5jZSijN0BjgqZS2FdjG7sAz7v5E7B+LmLuLUQTsdDP7OPBHFCjxx6JxaBbMbBJy\n6/gXMsPegdxGZiABcfe4zrzOxdTMFnT3/8T/zZF2bnck8H4FRdMfaGZj0Di96u5PdKUtM1sUZUo4\nPGjk4cAP3P15M1sNucFMRoLJrsDJ7n5rPfh38R1bPs9KqAHt5ix7w0YbJA/0IcwGrk2OfRqlXlk6\n9ut2nE2e1XLzGPqgb0RS52eQRHgtqt84BZmF647Cpdq0eAqqipBlmh+DVPJXolQwhb0b1cl8M3+q\nIcjk8J3Yz6pKrJK/rzs3aidxXT45vyZKFnwv8MPuxq83bFSb+a9Abg8HIO1Glky7I+bxfu0c7ybf\ns3ANR9CXbeL/BvE7G5X92iu5bk5uv91BJl3xR1sjvq/baC4H4zlUfLi3ifa2Q0ENqyBrxmVNvMu8\ncv2tjBQWNxHlCfPX9IZ5Vm61tw5KmC+4+4Mo9Pw9YEsz27cV7YRklcFbKDHsEma2Q+BxHWIw145r\n3m+0LXd/293fdPf/c/e3G31OZ2BmGyIJ9FpEqPZHuZf2RfmNtkRS4j/qfbaHVsnMRrj7ySjh8sw4\n9zyKrHsCEd2341rr5HH1tOtmthNKDTDTzC5C5th1gIlmdh3yJbze3Z/J2vSgat0J7v4qMMDMVo32\nz3H3F8ysIyT+X7r71e6+NfCOmR3W3Tj2ZIg+yrSXJyPN9kUo/+HDwM5mdhRiVua4+/eye9sx3s2A\nu/+zyOeZWf+gL/eY2Y7AoaGROghVLtjSzDY2sxuAf7v7VQkubeu70MBfbmaLx6HdEBM3KvavQe4M\nu8b3tZOHNrLOdjqC7p0DLGJm67r7Pcg8vRdKhP8Mci9Zz8zWrPP5FhrmvyM6i7t/EMczmvQs8gH+\nKbCNmY3zFmvri55nJXQO/duNQG8Bd38OmfYws4Vb1Iab2aaIkPzL3U81s3eBrcxsRRSKviHykejp\nC8giwJXBjGJmP0Gh9Tcik+xAd3+rXhNGYuI9HhhnZke5+yfM7G4z+6G7bxUMzPfc/bXsviL6ysxG\noQjIKcAriGjuhJjtdRDDfXpG7Ns1PkG8+1Eh7E9nhD0l3pmZEZn3R7QD154IZjbU3f8V/bg7coN4\nCcQwm9lNKMJ8T8TwfT3uq9sc19cg5tR70XcbIY3wx5Dprz9K+H52/P7G3Y+L+9red+7+dzN7EbjI\nzPZCWv+hwL5m9qbLDHo7sK2ZDQT+W28bZjYSOCfo1gtmdiSwg5kd4+4/N7N/A0sHQ/xxFF39XJ3v\n4cB7ZnYVcKeZveHulwfdtKyv3f1xM+uHLB6v1/suJfRcKDV2DUALJFyL302QVLg2cImZHY00UX9G\nmq5jgS+4+yNm1qPGroZGzIApwQzh7i+jFCAj3f09d38rjneJmIcEml5/IdJYfs3MFnX3bRExezKu\ne60TvBoCM1saLUwDgX+4+9+RRP1flIz23+7+YCMSfNEQRPs9lEriKDPbJzue9ofLd2w4Mivd2B5s\nexaEALWzyQ/1FBT0cyXwvpntHtqov7r73e6+r7t/Je7raDdj0hMg5lQHMvHt6O7/RlrNP6Ia1eu7\n+xfd/biEqWtr35nZomb27dg9HvghirB/DrlV/AmYbWbHIRp8s7u/0wjO7v43oiKOmY2I518PnGhm\n45DJf30UJDe7XqYu19YzKDr/aDP7bBzzoAP9Yv8x5LLxl0bbKaHnQY9iDj6qEB/aukj1f4i7nwhs\ngD7K/ahERv0WJUP90BzZUyDeYXsz+1aYju9Cvkj3mtmqZrYRYlj/U++zQ4Pynpn1N7MjzGztIJBH\nokivb5rZMHffDpVwqsKr0XfKMdy3A28iU9JRwUz+DeV/W9bM+hXFRBYFnRF2qLybu7+OnKyfbBui\nPQTMbAjSvq6E/KdWcvcfI9P7Y+ib3C0TMpL7rKd9j22GacDv3f0YgDA7fg8xd/ua2YT04nb3nc/t\ntnCuy6WD+C6uRdaS9YFvufut9QrWNZjHe1DAyD9Q8NI9KA3Ma+4+Bflyz2mWprj7AyiI6xtmdpCZ\nLRvHMwYcd3+nmTZK6HlQMnY9BzZGUUofM7PB7v5n5G+xfRCb2Yip29LMBrcPzdoQ0ubXAUeRqaej\nnG5XIIJ1KvKpe6TO5+4I3G6K8hqMapvua2arBVE8EqVwmWlmw919etxXlE/deoH7Ue7+JmKw3wFu\nMrMvoELXt7r7+z1RYzMPwu4lYa+AmU1GzNxQxIC8AfzJzFaK/pmFBKstUQHzD6Enjnt3Qqb9if9Z\n7dSHYn9B+JC5mwlc5e6PtgPPWhCWyZb7o9VgHs9z99/FudeQ8HAPMN3M1nP3N+Jc03PL3e9DyeHX\nBI4IzWPbGeoSWgelj12bwcxWQXmwzjH5000CfmRmjyJt1NDQRr1sZucD72ZmzHZD5qsRZsoJSNK9\n2hQ4sQPyRzvD3WeY2RB3fzO7p6ttuPsPzGwNxCTuijRyOwL7mdmF7v5rM7sfpRN5PbmvqMV2MIqK\n3hClaPiZmT2HFv+BwP7u/v8Kaqsl4O73mdI2fAlYycz+6u5TS8JeAXe/2eSkfgmKPn8A+BRwuJnN\ncPffm9lvgOfd/SftxLUngVXSwXQA68b38RAKJno0Y15MgUYz3f3u2G+7Tx10jz9aMIid+bx64PGa\nmV2DXDu2R/nsCgN3f8rMjgg8JpjZIG9B0FwJPQPKPHZtgvAr+cDkpDsR+LG7X2ryq9sF+DUKorjY\n3b/fUwhhHszsEyiY403gDXffKo6vD+yBfO2OQ7nqusxIhOlwI3c/KAjqZiF5EozeDogAOvCEux8S\n5xrup/ReU4TcB+7+ejBFZ6Fchpc28uyeAGY2iCDswM9Lwl5znm2eMerB6E0GlkN5AK9392/FuR75\nPXYn5Ji6e1G+yKPNbFUUWLIM8kPdAQWZfLGN6M4XzGwzpFU8092vSI5nQUaY2cBGNdwhxN8JnOju\nl8exqnlkZgciwXVHl59sCSXUDaXGrn0wGkVWXoekwI2C2ZtmZm8iM+wZ7n479BxzjyWJmU0O5oeg\nyNCXgAfMbLq7fzUk9w7gbw0yEHcgs/TC7v7P0MoBENLzH5BWZUWPVBNNMnWjUfTjjWa2PXAC0pYe\n6+53mNl/kS/fAHef1Ugb7YZkHP6vrYj0LMjPs/uyE+7+SzN7C5mw/pQxdXGuR3yP7QKrRA73Q8Xq\nH3b3Y2L/fcToDUea7p+7+zfjvh7LELv7A2b2JZTyZAhwm7v/MWNe3f2DZtwWXCmQPovcRtzdr0gE\nyf7ByD0O3FMydSU0A6XGrg0QTMT9iHG72swGIPPPXiir/SWmdB5booLYj7cP2wqEFutc5G/2FzM7\nBUXr7uKK1F0Mla75ubsf2mRbI5Bf0w/d/btxrNNFwZqoBBKmkv1QeoZHkQnuK0hLcxwq8H2TmW2H\nMs5vB7zcUxeoEroO3TnP+gqY/F6/ir77V83sBBS5vzKyMkwAbnL3k3OCYK/oO2txZR4zm4hcSr5B\nMI9FPr+EEkqNXTdB4o822t3/amYnoVQd/3X3OcBVZrYnsKmZ3eru3zAzR+XDegT43HmerkIF76fE\nezwRRP9uk4Pxr5to6x9mdjpwq5n905VM1ztbdJtg6jJz0qUoW/4qwJuutCVPmPweTwpN3fVm9ogr\nGraEPgDdNc/6EnjF7/USM9sZRRKvjwJOTgYWB441JRD/R3Jfr+i7VvujeSc+r0U9v4QSSo1dN0DC\n1O0AHA3s6e4vmiLxTkHpAX6BKjV82XtALrQUbN51B1dHfkijUN6lX5jZAu5ed/LOTtpuWa1eU8Te\nBqjvRyKT22jk43I1qgH7rpntAZwETHT3V5pps4SeCa2cZ30Favgjburu98e5Ae7+bvyfDbzt7ge1\nDdleAKXPawmtgpKxayHkzBBboIVjf3d/2JSy5N/Ir+tcVGT6bHfvkYlizWwmqof6dH6xMxWV3gNV\nLzgGlQlquNxZjbY3QWlTzgZ+m/kdNvnMLNXPPgj3VdCi9SczOxRly/85Khf1rpkt7mUSzz4NrZhn\nfQlMycYPBc4Kf8RMYM1+s8oso9x9/7inx/rUlVBCX4Uyj12LwOauO7gkyjI+wMwOAO4GzkDaoomo\n7uCNoRHrMWCC+eV5ehIFgZzl7v8qkqmL5xdaq9dU1ucyVPbsOWBrVP8z0zLOBp5FdWA/E8debabN\nEno+FD3P+iC8j/xO94RKJZOMcXPlY5udMHVlNY4SSmgDlBq7FoKZTQVWAz6H1O17IcJ4IRVt3cye\nZnqtBdZJqD50f4SgRQRjk8/4JjAOjcmGqC7jWOAiV9Ttciig5QeuBKUlfMSgiHnW1yB8624FjnH3\nq+PYXHSg1NSVUEL7oGTsCoYa/miHodqCL5jZUsj35O9mNhZpufb3HpSJfV5gned56hV+SDXG5svA\n913JZ9eg4iv4K+Rvd7q7v9Q+jEsooedB6Y9YQgk9G8qo2IIhwv+z0jFPm9l5GbEL/61+prQZ3wJO\n6C1MHcwzz1OvIOY1xuacZGweN7MPgC2A/VEqmpKpK6GEHLj7vaZyeleY2cKEP2JvoQMllNDXodTY\nFQihBeqHaqa+4u7nZcdzZopNUCWGh3ujycJanOepFTCvsYG5zEhZstpeNzYllNBdYGYrAdsAY4Cn\nUu1dCSWU0D4oGbsWQGf+aH2JSeitofofhbEpoYTuhtIfsYQSeg6UjF2LoLf7o/VlKMemhBJKKKGE\nvgolY9dCsLJ0TI+FcmxKKKGEEkroi1Aydi2G3uiP9lGBcmxKKKGEEkroa1Aydt0AvdUf7aMA5diU\nUEIJJZTQl6Bk7EoooYQSSiihhBL6CJQlxUoooYQSSiihhBL6CJSMXQkllFBCCSWUUEIfgZKxK6GE\nEkoooYQSSugjUDJ2JZRQQgkllFBCCX0ESsauhBJKKKGEEkoooY9AydiVUEIJJZRQQgkl9BEoGbsS\nSiihhBJKKKGEPgL/HweXhovZa9x8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e41853160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_discriminative_features(alpha_bigram, reverse=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly the model has learned to pick up on a completely different type of phenomenon: the presence of entities such as movies and actors. For example, the presence of \"pulp fiction\" seems to correlate with positive reviews. Often reviewers praise a good movie by its resemblance to \"pulp fiction\". We can see this by printing out a few such reviews."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "in essence , this is the science fiction equivolence of pulp fiction . the easiest way to write a review\n",
      "than anything actually shown--like most of the violence in \" pulp fiction \" . \" se7en \" is gory ,\n",
      ", the film is paced at half the speed of pulp fiction , which avary co-wrote with quentin tarantino .\n",
      "this statement , but you need look no further than pulp fiction for an example ) . and , while\n",
      "\n",
      "since this is a gritty crime comedy , flashbacks of pulp fiction should arise ) . at first , i\n",
      ", stock & two smoking barrels ( 8/10 ) - pulp fiction ( 8/10 ) - reservoir dogs ( 9/10\n",
      "incredibly fun to watch . it starts off as a pulp fiction-type crime story , with criminal brothers george clooney\n",
      "can be even more graphic ( e . g . pulp fiction ) . pulp fiction did it in a\n",
      "you couldn't find in any of the two dozen \" pulp fiction \" wannabes ; fortunately , the reappearance of\n",
      "but if most of those movies have their roots in pulp fiction , exploring a modern myth of the doomed\n",
      "the three years since the release of the groundbreaking success pulp fiction , the cinematic output from its creator ,\n",
      "first movie quentin tarantino has directed since the highly touted pulp fiction . to say he has been inactive in\n",
      "me give you an example . do you know in pulp fiction where jules and vincent go on brain detail\n"
     ]
    }
   ],
   "source": [
    "docs_with_good = [x for x,y in train if 'pulp' in x and 'fiction' in x and y=='pos']\n",
    "for doc_index in range(0,14):\n",
    "    good_index = docs_with_good[doc_index].index(\"pulp\")\n",
    "    print(\" \".join(docs_with_good[doc_index][good_index-10:good_index+10]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discriminative Text Classification\n",
    "\n",
    "The bigram representation can lead to improvements, but at the same time suffers from more sparsity issues. Is it possible to effectively capture features such as \"pulp fiction\" without moving to a sparser bigram representation?\n",
    "\n",
    "Recall that the problem is the fact that positive documents with \"Pulp Fiction\" may receive a too low probability under the positive class, and if this probability is lower than the probability of the negative class, the instance will be misclassified. Formally, we may have\n",
    "\n",
    "$$\n",
    "\\prob(\\text{...Pulp Fiction...}, +) < \\prob(\\text{...Pulp Fiction...}, -)\n",
    "$$\n",
    "\n",
    "for a positive instance. We can also phrase this in terms of the *a posteriori* probability of \"+\":\n",
    "\n",
    "$$\n",
    "\\prob(+ \\bar \\text{...Pulp Fiction...}) = \\frac{\\prob(+ , \\text{...Pulp Fiction...})}{\\prob(+, \\text{...Pulp Fiction...}) + \\prob(- , \\text{...Pulp Fiction...})} < \\frac{1}{2}\n",
    "$$\n",
    "\n",
    "Looking at the data table again, it is clear that we can't increase $\\prob(\\text{Fiction}\\bar +)$ as this would also increase the probability of \"+\" in three negative instances. However, we can also increase $\\prob(+ \\bar \\text{...Pulp Fiction...})$ by decreasing $\\prob(- , \\text{...Pulp Fiction...})$, the probability of the negative class in positive instances mentioning \"Pulp Fiction\". Now this in turn can be achieved by decreasing $\\prob(\\text{Pulp}\\bar -)$, and this will only move the probability of one negative instance (the 6th) in the wrong direction. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How can we operationalise this idea of penalising features in order to increase the conditional probability of the correct class? There is an intuitive alternative objective to the log-likelihood that achieves this: the *conditional* (log-)likelhood (CL) of the data:\n",
    "\n",
    "$$\n",
    "\\mathit{CL}(\\params) = \\sum_{(\\x,y) \\in \\train} \\log(\\prob_\\params(y \\bar \\x)) = \\sum_{(\\x,y) \\in \\train} \\log\\left(\\frac{\\prob_\\params(y,\\x)}{\\sum_y \\prob_\\params(y,\\x)}\\right)\n",
    "$$\n",
    "\n",
    "The CL objective can be increased both by assigning more probability to the training instance (as before in the joint objective), but also by assigning less probability to the other classes in sum of the denominator. It could hence down-weigh $\\prob(\\text{Pulp} \\bar -)$ whenever \"Pulp\" is seen in a positive review.  \n",
    "\n",
    "The CL objective also makes sense from another point of view: you should always train to optimise the actual prediction rule. In the case of classification we are looking for the class $y$ with highest *a posteriori* probability $\\prob(y \\bar \\x)$, and this is precisely the quantity the CL is encouraging to be large for each training instance.   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Log-Linear Models\n",
    "Before we discuss how to train classifiers using the CL objective we will introduce a *log-linear* representation for probabilistic models. This representation simplifies the math and enables a richer set of models later on. \n",
    "\n",
    "In log-linear form the joint NB distribution $p^{\\text{NB}}_{\\params}(\\x,y)$ can be written as:  \n",
    "\n",
    "\\begin{equation}\n",
    "  p^{\\text{NB}}_{\\weights}(\\x,y)= \\exp \\left( \\sum_{i \\in \\mathcal{I}} f_i(x) w_{y,i} \\right) = \\exp \\langle \\mathbf{f}(\\x), \\mathbf{w}_y \\rangle = \\exp s_\\weights(\\x,y)\n",
    "\\end{equation}\n",
    "\n",
    "where $s_\\weights(\\x,y) = \\langle \\mathbf{f}(\\x), \\mathbf{w}_y \\rangle$ is the linear *score* of $(\\x,y)$.\n",
    "\n",
    "Likewise, the conditional probability has the form:\n",
    "\n",
    "\\begin{equation}\n",
    "  p^{\\text{NB}}_{\\params}(y|\\x)= \\frac{1}{Z_\\x} \\exp \\langle \\mathbf{f}(\\x), \\mathbf{w}_y \\rangle = \\frac{1}{Z_\\x} \\exp s_\\weights(\\x,y)\n",
    "\\end{equation}\n",
    "\n",
    "where $Z_\\x=\\sum_{y\\in\\Ys} \\exp s_\\weights(\\x,y)$ is the *partition function*.\n",
    "\n",
    "Here the $f_i$ are so called *feature functions*. The index set $\\mathcal{I}$ contains an index for the *bias* feature $f_0$ and one for each word $x$ in the vocabulary. The corresponding feature functions are defined as follows:\n",
    "\n",
    "\\begin{split}\n",
    "  f_{0}(\\x) & = 1  \\\\\\\\\n",
    "  f_{x'}(\\x) & = \\sum_i^{\\text{length}(\\x)} \\delta(x',x_i) \n",
    "\\end{split}\n",
    "\n",
    "In words, the first feature function $f_0$ always returns 1 and hence serves as a *bias* feature. The second feature function returns the number of times the word $x'$ appears in $\\x$.\n",
    "\n",
    "If one now sets the weights according to\n",
    "\n",
    "\\begin{split}\n",
    "  w_{y'} & = \\log \\beta_{y'}\\\\\\\\\n",
    "  w_{x',y'} & = \\log \\alpha_{x',y'}\n",
    "\\end{split}\n",
    "\n",
    "it is easy to show that $\\ref{eq-loglinear}$ is equivalent to the original NB formulation in equation $\\ref{eq:NB}$. \n",
    "\n",
    "It is worthwhile to view the conditional log-likelihood in log-linear form:\n",
    "\n",
    "$$\n",
    "\\mathit{CL}(\\weights) = \\sum_{(\\x,y) \\in \\train} \\log(\\frac{1}{Z_\\x} \\exp s_\\weights(\\x,y) = \\sum_{(\\x,y) \\in \\train} s_\\weights(\\x,y) - \\log Z_\\x.\n",
    "$$\n",
    "\n",
    "This form should remind you of the log-loss defined in the [exercise solutions](http://localhost:8888/notebooks/exercises/structured_prediction_solutions.ipynb). It encourages high scores $\\langle \\mathbf{f}(\\x), \\mathbf{w}_y \\rangle$ of the gold solution, and low scores of the rest $\\log Z_\\x = \\log \\sum_{y\\in\\Ys} \\exp s_\\weights(\\x,y)$. \n",
    "\n",
    "Through simple changes to the above objective one can arrive at other standard loss/reward functions. For example, instead of subtracting the exponentiated scores of all solution, one can only subtract the score of the current predicted solution:\n",
    "\n",
    "$$\n",
    "\\mathit{PERCEPTRON}(\\weights) = \\sum_{(\\x,y) \\in \\train} s_\\weights(\\x,y) - s_\\weights(\\x,y^*(\\x))\n",
    "$$\n",
    "\n",
    "where $y^*(\\x)=\\argmax_{y\\in\\Ys} s_\\weights(\\x,y)$ as usual. When optimising this reward via gradient descent one arrives at the perceptron algorithm, and its natural extension to [structured prediction](todo:collins paper). Note that for a given instance this reward is maximal (zero) only if the score of the gold solution $y$ is at least as high as the score of any other solution. It hence directly encourages the gold $y$ to \"win\".   \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regularisation\n",
    "Estimating a log-linear model using the conditional log-likelihood can sometimes lead to *overfitting*. For example, for example, the reviews of movies may often contain the name of the movie to reviewed. This name is a false *killer feature* in the sense that it likely only appears in one training instance, and its weight can be set in way that we get perfect accuracy on this instance (if the instance is positive, we set this weight to be very high for $\\mathrm{pos}$ and low for $\\mathrm{neg}$, and vice versa for negative instances). This gives high training accuracy, but it will not generalise at all. \n",
    "\n",
    "To address this issue it is useful to *regularise* the model by penalising large weights. This is achieved by adding a regularisation penalty to the training objective. Most common are two types of terms, $||\\weights||_2$ and $||\\weights||_1$, the L2 and L1 norm of the weight vector. The training objective then becomes\n",
    "\n",
    "$$\n",
    "\\mathit{RCL}(\\weights) = \\sum_{(\\x,y) \\in \\train} \\log(\\prob_\\weights(y|\\x)) - \\frac{1}{C} ||\\weights||_2\n",
    "$$\n",
    "\n",
    "in the case of the L2 regulariser. Here the parameter $C$ controls the inverse strength of regularisation.\n",
    "\n",
    "Both L1 and L2 have their strength and weaknesses. L1 regularisation can lead to sparse vectors with zero weights, and this can improve the memory footprint of your model. L2 seems can lead to [better results](http://www.csie.ntu.edu.tw/~cjlin/liblinear/FAQ.html#l1_regularized_classification). Notice that learning with regularisation can also be understood as maximum-a-posteriori inference of the parameters under [specific priors](http://www.stat.columbia.edu/~gelman/research/published/priors11.pdf). For example, L2 regularisation assumes a Gaussian prior.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimising the Conditional Loglikelihood\n",
    "In contrast to the [joint loglikelihood objective](mle.ipynb) the conditional loglikelihood has no closed form solution. Instead the solution has to be found using iterative methods. Most commonly this is achieved through gradient algorithms such as L-BFGS or Stochastic Gradient Descent. \n",
    "\n",
    "In practice you will optimise the CL (and other training objectives) in one of the following ways:\n",
    "* You have to implement both optimisation code and gradient calculation.\n",
    "* You have an off-the-shelf gradient descent library that calls your client code to ask for the gradient of the objective at the current parameters. (Example: [factorie](http://factorie.cs.umass.edu/))\n",
    "* You have a back-propagation framework (such as Tensorflow) that requires you to declare the training objective *declaratively* and then provides gradients automatically\n",
    "* You have a library (such as scikit-learn) for a specific model class that provides tailor-made and built-in gradients and optimisation methods.\n",
    "\n",
    "Here we will consider the third case and work with a classifier toolkit that optimises the CL. However, in other chapters we will optimise the CL (and other variants of training objectives) using some of the other approaches. \n",
    "\n",
    "To prepare for these cases, and to give a better intuition for CL objective, it is useful to understand the CL gradient:\n",
    "\n",
    "\\begin{split}\n",
    "  \\nabla_{\\weights_{y'}} CL(\\params) =  \\sum_{(\\x,y) \\in \\train} \\mathbf{f}(\\x) \\delta(y,y') - p_\\params(y'|\\x) \\mathbf{f}(\\x) = \\sum_{(\\x,y) \\in \\train} \\mathbf{f}(\\x) \\delta(y,y') - \\sum_{(\\x,y) \\in \\train} p_\\params(y'|\\x) \\mathbf{f}(\\x).\n",
    "\\end{split}\n",
    "\n",
    "Because the conditional log-likelhood is a strictly concave function, we have found a solution $\\weights*$ when this gradient is zero. When will it be zero? When the empirical expectations $\\sum_{(\\x,y) \\in \\train} \\mathbf{f}(\\x) \\delta(y,y')$ for class $y'$ match the model expectations $\\sum_{(\\x,y) \\in \\train} p_\\params(y'|\\x) \\mathbf{f}(\\x)$. This gives rise to a *dual* view on maximising the conditional likelihood: it finds a distribution with the same moments as the empirical distribution. There are actually several such distributions, and maximimising the conditional likelihood corresponds to finding the distribution that matches moments, and has [maximum entropy](todo).\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression Toolkits\n",
    "A log-linear model trained by maximising the CL in fact corresponds to a *logistic regression* with maximum-likelihood estimation, as well as the so called *maximum entropy* approach. This means that we can use tools branded as logistic regression or maximum entropy software to train CL optimised models.  \n",
    "\n",
    "Here we use the [logistic regression](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) implementation of [scikit-learn](http://scikit-learn.org/stable/index.html). As it is common with such libraries, we have to first convert our data into a vector form. That is, every $\\x \\in \\Xs$ should be converted to a (sparse) feature vector $\\mathbf{f}(\\x)$, and every $y\\in\\Ys$ to a real number. `scikit-learn` comes with build-in support for this type of conversion, for example in the form of the `DictVectorizer` for inputs $\\x$ and the `LabelEncoder` for labels $y$.\n",
    "\n",
    "Let us first convert the input data into vectors. We do this by first providing a dictionary representation of the feature vector in `to_dict`, and then use the scikit-learn `DictVectorizer` to turn these dictionaries into vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "from sklearn.feature_extraction import DictVectorizer\n",
    "\n",
    "vectorizer = DictVectorizer()\n",
    "\n",
    "def to_dict(x):\n",
    "    result = defaultdict(float)\n",
    "    for w in x:\n",
    "        result[w] += 1.0\n",
    "    return result\n",
    "\n",
    "train_X = vectorizer.fit_transform([to_dict(x) for x,_ in train_filtered])\n",
    "dev_X = vectorizer.transform([to_dict(x) for x,_ in dev_filtered])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we convert the labels using the `LabelEncoder`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "label_encoder = LabelEncoder()\n",
    "\n",
    "train_Y = label_encoder.fit_transform([y for _,y in train_filtered])\n",
    "dev_Y = label_encoder.transform([y for _,y in dev_filtered])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can train the logistic regression model using a regularisation parameter $C=0.25$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=0.25, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "import numpy as np\n",
    "\n",
    "lr = LogisticRegression(C=0.25)\n",
    "lr.fit(train_X, train_Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us have a look at the weights the model learned. Note that the scikit-learn package only learns one set of weights, not one for each class. It is easy to see that this is equivalent, as you can always convert a model with two sets of weights to one with only one weight vector by subtracting the weights of the negative class from the weights of the positive class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl8AAAGWCAYAAABGsOb8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYJVWZ+PHvywwZJTMMQRBJogjiSJIgwpBhEBOgiKIC\nCmJCQV1RUBQVcwBHJIiY1lVBwQAouiZkEFAEd0GXXZURZnEN6/52Tef3x3uuXbTdM9Nd1dWB7+d5\n7tO37q2uU1W3wnvec6oqSilIkiSpHytM9gxIkiQ9mBh8SZIk9cjgS5IkqUcGX5IkST0y+JIkSeqR\nwZckSVKPOgm+IuLAiPiXiLgrIs4Y4fuIiPfW738YETt1Ua4kSdJ00zr4iohZwAeAg4DtgKMjYrth\nox0EbFVfJwDnty1XkiRpOuoi87UzcFcp5WellD8CnwQWDBtnAfDRkr4HrBURczsoW5IkaVqZ3cE0\nNgZ+3hj+BbDLcoyzMbB4+MQi4gQyO8bqq6/+uG233baDWRzdj375206nt/3Ga05aOV2X0Vc5M2lZ\n+ipnpi9LX+XMpHU2k5alr3Jm0jqbrsvSVzmjbWddu+mmm/6zlLL+ssbrIvjqVCllIbAQYN68eWXR\nokUTWt7mZ1zV6fQWnXvIpJXTdRl9lTOTlqWvcmb6svRVzkxaZzNpWfoqZyats+m6LH2VM9p21rWI\n+PflGa+LZsdfAps2hjepn411HEmSpBmvi+DrRmCriHh4RKwEHAVcOWycK4Fn16sedwV+W0r5uyZH\nSZKkma51s2Mp5c8RcQrwFWAWcFEp5ccRcVL9/gLgauBg4C7gf4Dnti1XkiRpOuqkz1cp5WoywGp+\ndkHjfQFO7qIsSZKk6cw73EuSJPXI4EuSJKlHBl+SJEk9MviSJEnqkcGXJElSjwy+JEmSemTwJUmS\n1CODL0mSpB4ZfEmSJPXI4EuSJKlHBl+SJEk9MviSJEnqkcGXJElSjwy+JEmSemTwJUmS1CODL0mS\npB4ZfEmSJPXI4EuSJKlHBl+SJEk9MviSJEnqkcGXJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmS\nJPXI4EuSJKlHBl+SJEk9MviSJEnqkcGXJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI4EuS\nJKlHBl+SJEk9MviSJEnqkcGXJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPWoVfAVEetExDUR\ncWf9u/Yo410UEfdFxG1typMkSZru2ma+zgCuK6VsBVxXh0dyCXBgy7IkSZKmvbbB1wLg0vr+UuCI\nkUYqpXwT+HXLsiRJkqa9tsHXnFLK4vr+V8CcltMjIk6IiEURsWjJkiVtJydJkjSlzF7WCBFxLbDh\nCF+9tjlQSikRUdrOUCllIbAQYN68ea2nJ0mSNJUsM/gqpew32ncRcW9EzC2lLI6IucB9nc6dJEnS\nDNO22fFK4Lj6/jjgipbTkyRJmtHaBl/nAvMj4k5gvzpMRGwUEVcPRoqITwDfBbaJiF9ExPNalitJ\nkjQtLbPZcWlKKfcD+47w+T3AwY3ho9uUI0mSNFN4h3tJkqQeGXxJkiT1yOBLkiSpRwZfkiRJPTL4\nkiRJ6pHBlyRJUo8MviRJknpk8CVJktQjgy9JkqQeGXxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHB\nlyRJUo8MviRJknpk8CVJktQjgy9JkqQeGXxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8M\nviRJknpk8CVJktQjgy9JkqQeGXxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJknpk\n8CVJktQjgy9JkqQeGXxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJknpk8CVJktQj\ngy9JkqQetQq+ImKdiLgmIu6sf9ceYZxNI+LrEXF7RPw4Il7SpkxJkqTprG3m6wzgulLKVsB1dXi4\nPwOvKKVsB+wKnBwR27UsV5IkaVpqG3wtAC6t7y8Fjhg+QillcSnlB/X974E7gI1blitJkjQttQ2+\n5pRSFtf3vwLmLG3kiNgceCxww1LGOSEiFkXEoiVLlrScPUmSpKll9rJGiIhrgQ1H+Oq1zYFSSomI\nspTprAH8E/DSUsrvRhuvlLIQWAgwb968UacnSZI0HS0z+Cql7DfadxFxb0TMLaUsjoi5wH2jjLci\nGXhdXkr57LjnVpIkaZpr2+x4JXBcfX8ccMXwESIigI8Ad5RS3tmyPEmSpGmtbfB1LjA/Iu4E9qvD\nRMRGEXF1HecJwLHAkyLilvo6uGW5kiRJ09Iymx2XppRyP7DvCJ/fAxxc338LiDblSJIkzRTe4V6S\nJKlHBl+SJEk9MviSJEnqkcGXJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI4EuSJKlHBl+S\nJEk9MviSJEnqkcGXJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI4EuSJKlHBl+SJEk9MviS\nJEnqkcGXJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPVo9mTPgCRJUtPd5x4y2bMwocx8SZIk\n9cjgS5IkqUcGX5IkST2yz5ckSdNcX32kZnpfrL4YfEmSNIEMWDScwZck6UHJoEiTxT5fkiRJPTLz\nJUmacsxKaSYz8yVJktQjM1+SpOVmRkpqz8yXJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI\n4EuSJKlHrYKviFgnIq6JiDvr37VHGGeViPh+RNwaET+OiLPalClJkjSdtc18nQFcV0rZCriuDg/3\nf8CTSik7ADsCB0bEri3LlSRJmpbaBl8LgEvr+0uBI4aPUNJ/18EV66u0LFeSJGlaaht8zSmlLK7v\nfwXMGWmkiJgVEbcA9wHXlFJuGG2CEXFCRCyKiEVLlixpOXuSJElTyzIfLxQR1wIbjvDVa5sDpZQS\nESNmtEopfwF2jIi1gM9FxKNLKbeNMu5CYCHAvHnzzJBJkqQZZZnBVyllv9G+i4h7I2JuKWVxRMwl\nM1tLm9ZvIuLrwIHAiMGXJEnSTNb2wdpXAscB59a/VwwfISLWB/5UA69VgfnAW1uWK0lq8IHX0vTR\nts/XucD8iLgT2K8OExEbRcTVdZy5wNcj4ofAjWSfry+2LFeSJGlaapX5KqXcD+w7wuf3AAfX9z8E\nHtumHEmSpJnCO9xLkiT1yOBLkiSpR2073EuSlsHO8JKazHxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ\n6pHBlyRJUo+82lHSg5ZXIUqaDGa+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI4EuSJKlHBl+SJEk9\nMviSJEnqkcGXJElSjwy+JEmSeuQd7iVNSd59XtJMZeZLkiSpR2a+JI2JGSlJasfMlyRJUo8MviRJ\nknpk8CVJktQjgy9JkqQe2eFemiHsCC9J04OZL0mSpB4ZfEmSJPXI4EuSJKlH9vmSemB/LEnSgJkv\nSZKkHpn50oOaGSlJUt/MfEmSJPXI4EuSJKlHBl+SJEk9ss+XpiT7YkmSZiozX5IkST0y+JIkSeqR\nwZckSVKPDL4kSZJ6ZPAlSZLUo1bBV0SsExHXRMSd9e/aSxl3VkTcHBFfbFOmJEnSdNY283UGcF0p\nZSvgujo8mpcAd7QsT5IkaVprG3wtAC6t7y8FjhhppIjYBDgEuLBleZIkSdNa2+BrTillcX3/K2DO\nKOO9G3gV8NdlTTAiToiIRRGxaMmSJS1nT5IkaWpZ5h3uI+JaYMMRvnptc6CUUiKijPD/hwL3lVJu\niognLqu8UspCYCHAvHnz/m56kiRJ09kyg69Syn6jfRcR90bE3FLK4oiYC9w3wmhPAA6PiIOBVYCH\nRsTHSinPGvdcS5IkTVNtn+14JXAccG79e8XwEUoprwZeDVAzX6cZeE1vPndRkqTxa9vn61xgfkTc\nCexXh4mIjSLi6rYzJ0mSNNO0ynyVUu4H9h3h83uAg0f4/Hrg+jZlSpIkTWfe4V6SJKlHBl+SJEk9\natvhXlOIHeElSZr6zHxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJknpk8CVJktQj\n7/PVE+/BJUmSwMyXJElSrwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI4EuSJKlHBl+SJEk9MviS\nJEnqkcGXJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI4EuSJKlHBl+SJEk9MviSJEnqkcGX\nJElSjwy+JEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI4EuSJKlHBl+SJEk9MviSJEnqkcGXJElSjwy+\nJEmSemTwJUmS1CODL0mSpB4ZfEmSJPXI4EuSJKlHs9v8c0SsA3wK2By4G3h6KeW/RhjvbuD3wF+A\nP5dS5rUpV5Ikabpqm/k6A7iulLIVcF0dHs0+pZQdDbwkSdKDWdvgawFwaX1/KXBEy+lJkiTNaG2D\nrzmllMX1/a+AOaOMV4BrI+KmiDhhaROMiBMiYlFELFqyZEnL2ZMkSZpaltnnKyKuBTYc4avXNgdK\nKSUiyiiT2aOU8suI2AC4JiJ+Ukr55kgjllIWAgsB5s2bN9r0JEmSpqVlBl+llP1G+y4i7o2IuaWU\nxRExF7hvlGn8sv69LyI+B+wMjBh8SZIkzWRtmx2vBI6r748Drhg+QkSsHhEPGbwH9gdua1muJEnS\ntNQ2+DoXmB8RdwL71WEiYqOIuLqOMwf4VkTcCnwfuKqU8uWW5UqSJE1Lre7zVUq5H9h3hM/vAQ6u\n738G7NCmHEmSpJnCO9xLkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJknpk8CVJktQj\ngy9JkqQeGXxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJknpk8CVJktQjgy9JkqQe\nGXxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJknpk8CVJktQjgy9JkqQeGXxJkiT1\nyOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJknpk8CVJktQjgy9JkqQeGXxJkiT1yOBLkiSp\nRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJknpk8CVJktQjgy9JkqQeGXxJkiT1qFXwFRHrRMQ1EXFn\n/bv2KOOtFRGfiYifRMQdEbFbm3IlSZKmq7aZrzOA60opWwHX1eGRvAf4cillW2AH4I6W5UqSJE1L\nbYOvBcCl9f2lwBHDR4iINYG9gI8AlFL+WEr5TctyJUmSpqW2wdecUsri+v5XwJwRxnk4sAS4OCJu\njogLI2L10SYYESdExKKIWLRkyZKWsydJkjS1LDP4iohrI+K2EV4LmuOVUgpQRpjEbGAn4PxSymOB\nPzB68ySllIWllHmllHnrr7/+2JZGkiRpipu9rBFKKfuN9l1E3BsRc0spiyNiLnDfCKP9AvhFKeWG\nOvwZlhJ8SZIkzWRtmx2vBI6r748Drhg+QinlV8DPI2Kb+tG+wO0ty5UkSZqW2gZf5wLzI+JOYL86\nTERsFBFXN8Z7MXB5RPwQ2BF4c8tyJUmSpqVlNjsuTSnlfjKTNfzze4CDG8O3APPalCVJkjQTeId7\nSZKkHhl8SZIk9cjgS5IkqUcGX5IkST0y+JIkSepRq6sdZ4K7zz1ksmdBkiQ9iJj5kiRJ6pHBlyRJ\nUo8MviRJknpk8CVJktQjgy9JkqQeGXxJkiT1yOBLkiSpRwZfkiRJPTL4kiRJ6pHBlyRJUo8MviRJ\nknpk8CVJktQjgy9JkqQeGXxJkiT1yOBLkiSpR1FKmex5GFVELAH+fbLno1oP+M8ZUs5MWpa+yplJ\ny9JXOS7Lg7scl+XBXc5MWpax2KyUsv6yRprSwddUEhGLSinzZkI5M2lZ+ipnJi1LX+W4LA/uclyW\nB3c5M2lZJoLNjpIkST0y+JIkSeqRwdfyWziDyplJy9JXOTNpWfoqx2V5cJfjsjy4y5lJy9I5+3xJ\nkiT1yMyXJElSjwy+xigiYrLnQWpym5Sk6cXga+y2mOwZkAAiYg5AKaUYgM1czd92In/niFjmvYmm\ng4hYMyIm7dwWEbMnq2xNHwZfYxARawCXRcRbJ3teuhIRsyNibn2/XUTM6nj6sxrvez0o9XXSGlbm\nmj2VE8DnI+LTMHEBWFfTNDgcu4hYLSJm1992M8jfeYLKWgG4PCIumYjpN8rZISK2nMDpbw58Eth9\nMgKwGsCeEhFbt5xONP/2ZbLKfTAy+FpOEbFCKeW/gWcBe0TE6ZM9T8sjIraPiO2WMsqTgKMj4p3A\nJcCKHZa9IvCCiHhYROwAnNN1cLc09aQ1LyIeMZHZocYBa0fgjPp3wkRElLQb8JiIeBd0G4ANgsgu\npjmY3/r+JRHx3C7mcTnLntX8OwHT32giplvtDrwzIg4D3hYRG09EIfX3+WspZX9gu4h40wSVsz3w\nEeAvEzF9gFLK3cB3gVcBu0xCALY18EjgyRHxiPFMoLm/ABt0NmdjK3fdiS4rInav7x8ZEXt0Pf36\n93ER8ZSI2DIiVu+yjLYMvpZTKeWv9e0OwA+BF0bEayZxlpYpIlYFngpsV4dH+r2/CewPPBc4p5Ty\nv12VX0r5E/Az4G7gs8CFpZQJO/AONHa8nYAPAd+NiK0mKgCr0z0IeCtwPPDKiHhc1+U0ywOIiEcB\nnwdOjIj3NealbbD0MuCjEXFFRKzeNtvSmN+XAs8AvtdmessjIl4cEe8GLoqI7Sdiu6vr6YyJaq4r\npVxL7rsfBz5USvnlRGaPa6XhduDlEXFRx9N+AnAu8OFSyr9NZJa2lHI2cC1wJj0HYKWUbwOfIIOm\np48nAGvsLyeT++FZEXFct3P6QMMqSC8mW3hWn8AM2NrAXhHxJXJ93dPlxOtx8DDgYuBx5D50UJdl\ntGXwNQYR8WzyBHsp8AbgoIh4w2TO09KUUv4f8GvgpEHtdoTR5gDnAZ8CHhsRuzeCl3FvH7Vms0Ip\n5atkgLcBMMhCdJZdG0nd8fYHLiR/p2uBL0XE1hMRgEXENsDbgJOBxwK/B46pNf0JERG7AV8BvkQG\n2PMj4sPQLgCLiEOAw4FXAP8LfHK82Z3mPETEQ4G9yczx/RFxTEScPRHrKCJOAhYA7wa2JX+Xrss4\nHngK8KZSypLIJsKJaKK9EbiObMpatZTy5y7KqOXMgqEMMVlB+gCwJzAvIj7UVVnAfWRWaM9BmR1O\nm8E0I2KfiFhQSnkv8GV6CsCav1kp5XrgM+Sx9RnjCcAi4jnAUcALgF2BYyPijE5m9u/LWrEReJ0I\nPBM4uZTyB+AhE1FmKeXXwC+APYC7Sik/q+V3kqGOiPXI5dgHuB5YCfhG/W5KxD1TYiamkVWBt5ZS\nvgdcBrwceFpEnDm5swURsU5ErFXfbxURRwOUUt4D/AQ4tn7XPCGeDLwR+A55sl0TeBqwTQ00nzrO\neRnUotat5e1PZjxuiIi9Syl/iohtJzgNvA/wT6WUL5RSjiGDy69HxBb1IN3ltj8bWAIsKaXcQx7w\n9wBeHRHbdlhO02rAZ0op3yilXA3sTFYGPgjjO7lFxC5k5u5rpZS7SinPABYDF0TEJuOd0cgmhVnA\n/wFnAR8mg9THkYFeKyMEPeuQJ64jyQfunhIRK0e3/fEeA1wOrBURrwQ+Cryn7XY12HciuwvMA95e\nSjkC+DnwT3WcLSLiiJblrA+8sVERWhn451LKjaWUm8gT/mFtM2ARsU3dB+4m94nHRMSr20xzhDKi\nsd53Bw6PiAPrse/LwGvIriITcr5r/GZ7Rzar7wv8gNwmNgCeGhFbLWsajWV5CLAKGdwvIPedDwJP\n6joAq4HhaVEv3gE2AV4KrF0zYN+PiBc057Flec1pfIGsjP1HRLy/BoF/iYgumjx/B/w7cDZ5zHly\nrSQdQC7jpDP4GsVSNrSXR8RKtRnjFuBm4ICONphxiWyKOB84vWYp9gJeFBEXRcThwK+obfiNGs7R\nZFPjGaWU/yGzHK8E/gCcDrwJ+PE45mVwIDqMTPV+ATiNrMEfB1wdEacAnwaWekAaa7nDPvo58Lfg\nrpTyWjK1/fGIWHuULOCYyoqIlWpN7Q7gLjL7tF4p5T6yuXNjsvbV2gjL9zvyYLwGQCnld2SF4NjI\nCyfGc6D8LVkb3TFqf4xSygnAfwPvWN5aaUTsGBHb1u1gLvCuUsp/AeeQzaQvKaW8kgxedoqIVcZ7\nYK/Bw9b1/b61xrsZmRWcBxxes0XPI9dN2+Bol7oergOeDlxABpWXkcfTcQd4jX3nADJzch6wMCKe\nRzbZ3RMRPwCuAH7TZjnICsOFwNx6zLgH2Ciywzr1mPB+MqAfVwf5euz5KPB2srVgV2A/sjnu9S3n\nn0HFph7TBvN9DnAbsCAiDq4B2LeBF5PZj87V3+xQMtM6i6zInkUeEy4GHk5mwFYdZTmafa2ilPL7\nUsoFdXhfYH4p5bN1+OEdn2vmksfJ59X5u588l7yB3O/PJoPHOV1kKwddNCK7BLycDFI/CRSyX+Me\nwFl1P14uzX06ItaLiA1LKX8ks607A/9Qm7qfALwXmBpX9ZZSfA17Ue/8X98/FXg+sEMdPg+4AdiS\nDF4+Aaw/BeZ5K+CL5I7/0PrZKcAZZA3gN8Ah9fM5ZDPZKXU5TieDowvq95sCG46x/BUa77cms22P\nBQ4mg68P1u+OIA/q+0/AOtiNzHg9kjyo3Fx/u3XJA/+HgH8EzuqgrMOBj5Enl4fU5fww8A6yqeDG\nOs61wAZdbI+1jLeTTd/rkgf4O4DHA4eRAe0245j+QWR28rHAQ4E3k4HSbo1x5i7ntFYiMw1fqdtB\nkJnVFZvLU3+XHwOParludqrzeyHwk/rZw4CfAq+tw88l+zJt3bKsFwO3AuuRF6ZsAKxav3sKeSJZ\nZxzTXbnxfjvgGmC7OnxM/c33qMNHA7t2tL+sWLelz5NZ/dPILgJHkpnDq4DHjnPa65J9+3YgM5H7\n1HIeTx5z7gS2aDHvB5CVyoeSwc11wFMb359BNjcdVofX7mKdNaa/PrBjfb8RGUBsSmaqbiObb88j\nA5vHDn7PZUzzhQwdQ+YDa9T1tGf9Ta6m5bGkUVbzHLcncBKwSh1+dGO7fmLdJtbrqNxd637/dLIS\nMVjW7cjj879Sz1PLOb0NgGMb28R36z56at1X3k8ep88nj5WHdrkdtFoXkz0DU/lFpl//GXg98PW6\nga5GNtV9mmxD3mGS57G5E21Vd9A38sAD+v5ksHUaGRicAzyBzA59ATih/u+1wObjmIeNgVcPDgxk\nxuGLje8fWQ9OT6zDK4+1jKWUvUL9+wSyiezD5Il/PnlPtqvqzvdTYEfyRPyalmVuD3yfDHg+WHfq\n1etB61VkbXcnMhj8NjUYblnmAbXMXckTzafq568nryD7Opla/7vtYhnTPQn4EZnpXEwGEXPr8HuA\nnce5PZxONpM9nryQ42/zRAZH7wQe2dE28Dbgf4ATGp89igy+LyVPHss8+S2jjMOBm4B16/A25El3\nFfIgfwfjCCTJjscfJwP4Fet6WwI8rX4/izxBLexoXcWw4fWBt9T1tDLZDPQO8jhy+HinD2wIfIuh\niuBDgNcBr6zDq7dZBrKS+TKy4nAc2a3iSuCIxnjfJwPXNbpYd43prlj3m0cwFLBsUY8vPyCDwcPJ\nCuj7gVnLMc2T6n69A3kcfn/9/Hn1GPIdarDX9TawlPFeVpfnMR2VuzWZIX5dHZ5dt4mPNcbZZIzz\n+HTgIjKL9vW63+9IHpdPJs89e5DdXuaNZdoT/Zr0GZhKLx6YvXkc2U9oBTJo+QEZmZ/YOMCsMsnz\nO5iPHckT/3r1YHpV3ajXaYy7E3mS/QA1A0BmwFaq7w9k/DX3uWSQejawFhmIfB14QWOcd5OdODvZ\n+MmaejPwOgvYvQ4fUss/sA6vTZ4MDqjLOKaTZP3fnev7R5G35Din8f37yNrcOnV4Vj343kpHwTnw\n2nogWUCe1LYY9v3qY1m3dbvemAxMHlk/24XMkh5A1uJfx3JmdYeXS2Y+Xl3X91/rOvoi+RDc04HV\nOlovjyczCy+u2/bTqFmOuj+sDKzVQTn71OV5Wd3WfkIG+nuQTWlbtpj2w8jKz/Z123l93cb2qd/P\nJzPsq3e07xxKBr8XkJmDLcjs4cUMBZeDzMeYyqORISGDuMsb0zqBPIbOAmaPc94Hx7zDyGD49sG+\nQAaOV5FZu4Pr9rZJF9vZCOWvXNfd24A962d7UYPk+v4jjBL0k4HI4xrDp5PHqVPIfmorkYF91N+9\n0wCylnkqmfm8jBEqiOQxrFW2eNj0diMr4V8Ftm98/j3GmWGt/38smdn6dmNb25asfB3Z9XrrbH1M\n9gxMxRd5UlqHTI/vRWa4ZpPt4LeREfUKXRwIO5jXw8ka3tlkc8WjyZrXF8jsxbp1XvcE/h95Jd7g\nADu7fncCGTxsP47yZ9e/65HNCq8ng7onkwf3d9d1+C80mrFaLvP6ZIZprTr8LvKqzn3r8MpkjfgG\n4Pn1s3XJA/+YgqG6jo6qO/NKdTkvJfvk7NAYbyHZX2rFOrwzsG2LZRwc5AcB3ZvJKxuvBR5ePzsC\neAlZE19hnOVcTAbvg9/xmeRtDWA5M5Q8MPu6N3ly34gM4E6p2+czajk709EBnQyEvgBsXIdfQGY5\nD67bxznjXS+NMjas29tGZBbl02Qgtln9zffr4DfeoK73n5IB9mrkyfhHZABzA+PIQo1S5k51X38y\nGRh9qv4uG5D9Yf6RPOkvM1szwrQPJE+sl5HZ993Ji09uIjPOd7ZZX8PKegzZbPVZsvK1Wt0PFpAn\n4esZx/FsGWWuwlBFZTOySe5tdTvbhTxn/KbuU78ADhplOiuTGbH3MpSNOY+8Lc/nGuOdVF+ttuFR\n5uFEMtO2Ptnf79zGd52U19i+dyCbFR9KdnZ/D3ku3Y2hLiqPGGcZjyQrLQeSQevJDB0zX00GmMEU\nOFf/3bxP9gxMhVc9SBxV37+Y7Eh/cd0h3kBtpiKvBFtIR+3uLec5yIzTN8ga04vJjMic+v1mZPPb\nlmQ2am2SFKlQAAAb80lEQVQyTf4F4JuN6cwmMwdbtZiXwQn32DoPL6w71Q71QPw+Ojp51PJWJAPM\nDYFd6mfnkVmcQUC2MnkS3qX5f+Msb2UyeLu4HmTXJrMeZ/LAGlyrpq3mb1v/HlQPVKvV3+5uhprw\nBgHtk8Yx/ecDL6/vF5I19DXr8IlkGn/MB6x6oPshGQB9g+yvtD3ZJPBVxtEfbSllHUzWmAcZv43q\ntnxY3d5uomXWkcx4f4HsR/Ki+rsPfpvDgUWM86TRKGM+2aS0dl1/N5MnqtlkheozwPOa20WLsrYn\nM2pnNT47a3A8IE+M4wqMyYzwXWQl7xDyxPcpMqg7pW5XXQVeTwC+Rp7MjyMrQ4cwlPV4CB009Y+y\n/k4lK3t31HK2qOvwPDKDuS6ZfR2xoslQtn5WPYa8jTxWzyGDh/Pq98eTWb1O9pnh2w55XtuMrLxd\nXbe3leg40CMrKr+ux5jra5lb1mPMj8hM5QHj3b7r+ruivj+q7vsXkX2172Icx8e+XpM+A1PhVXfc\nfyNrax8nT3T7krXPfyabTS4la27jzmZ0MJ/NTsurkEHBh+oB6FvUAIo8oK9YxzmBrM1+iKHOp58H\nrmk5L4OT0O71QPQxMjPwDTLT8VLgIXWcWc3/aVnuYFoPIwOTS6jpezL1/DWGaj4x3nLJ4KPZDL0Z\n2Yl3IRmsblDLewu1T0TjwNrFcj6JYdlCMqC9pW6jNzKGjqmNaZxGBhPNoPEisn/WpXW6jx7HdDcl\nM6/r1eGnD34bMvh/BR00ATV+0+fV9bA3mXn4Dpk5Wp1skm518iWzil+u7y8DPtco+ylkdqVVZoWh\nzvU7Nz47te4/jyGD7peSGapWnezrtLYgs0Wfo9HfjjzptzqukRm18+v7Fer2cBGwV9vffISyViWz\nRnvV4ZfX/fJIJqB5bljZ55FXhr+p8dk2ZMb/fTQqe8O3Wx6YIT6+rvdf1O1rHhnAXlP3xX+m48pc\nfX8M2a3gXWTl/OMM9Vt7BbVrSBflMXTrokF3kFfW48vmZOLgvWQFdrkv7uLvg8ggj1uDix+eRlbC\nP0lttp+qr0mfganyIgOW24DL6/DKZErznHogPo4W/To6mL9ZwHPIVO3eZPNH1APzrwbzRtY0biT7\nkBxGpnQfRzbDvJvaKbke4D/fcp52JvtW7VqHtyRrUhfXg8dbGWffjlHKG+zUe9adazOy1nk+Q+n7\ni8kTYyflktm7bcga7apkU9dFZFPNHLJGN+6s4QjlrVB/6/eTGcRZZF+Wj5LZzRXIps/NxjHt9cgT\n72wya3hMPQivUbeRg6hNmsv7WzSGVyKzRM0Oz28GPjLYfjtaP4MM3cpkJeJq8qS7AnnFY1dN2/Pr\ngfx15Elq0Jz88Lq+xnQ18Ajb8Kz6e/4SOH7YOKeRWYGVyGPQKSzn1aajlLlZPU5sR1ZaLqvHgyfW\n7ftuxp/xekLdPp9L3lPt4MZ35w9ftpa/yTPJIGs1MuB6XeO7fyBP5hPRN6oZvDyqbhNvIc8Jg8zr\njmTlfcRMVfN4VNfZLXUb2Jjsq/iOwT5df/eHTMBy7AV8o77fhWwiPbIOH1O3ua66BBxMBpffrr/b\noNJ8GllZ34Lsr3kBGfStNI5tbrc6fC5weuP7ZzOJSZLlXo7JnoGp9CL7C/wX8IzGZ1cyRS5PrTv4\nb8lO0YMO4LuTV3Z8lqxN3UY2iTyBrEW/sI63Gtk0+PHGjrBZy/mZTz6nbdAsu2It+y31oLLTBKyD\nPcna55Pr8Mpk37b3M9QEOebMTWP6mwLvqO+fCNxL1qyuJJsdVq7r9eNkwDKupswRyh0ezBxINjt8\ng6wAPIuWHYjJoOtrDNUM31SHPzTeeSVP7IMLOM4gg9PBtvkssmN3V31ITq6/xeuHb7tkNuqONuun\nTucI8urgs8jg7gqG+sOdSjaTjOtqXYYCr/nkVavrk5WVi6gXhzTG3bzxvtU2RlbEXk1WTLYmK0kf\nJytgn6QGTMO3weWYbjPr/Q7y4orvkU2Me5AXnHQSDDe2r/vJZvOXkZnO5hWuY75YaAxlP4kMMPer\nw8eRQdNTyAsuTqD2pR3hf9evx4/B779b3e8GV4JuSAZjn6dFx/NlzP++ZCD0lsZnh9ff66NkJXrc\nx81hZc0jK3lH1v317cDjh/2Og2zYroyxGw9DmbsvkZmzJ5NJhk6uyuzrNekzMNVeZIDyM7JN/Aiy\nNtCqX0dH8zVo0rqUrC0fVIdXITMap5Mnp33JvhAfJGu7NzQ3SrLvzbwO52sB2bZ+dB3eux6EW19h\nNqycwYHrbPKigaMb361EvSEl7ZubVq0nlEvIk/xuZOB6cj1gbl/X+Su62tkby7ZHLfMZZNPmZgzV\nhrcn+xiNOQNSTw6DTskrk0H65vW7A8js3TIDCv4+QHwlmeH8cZ3vZ5PB1mfIk/ttdNTpmcz6fofM\nPP2sbtv71t/iULKJttXJg+wzsrhuYzeQGaHzyJPIy8n+bG3vS3ZQPVEM+rlsUret86ndAkZb3+Mo\na6fG+y3JgOUystlnUzJoOoNRgoZlTHt41nsLson00+RFIR+hkQVtuRy71n1jjVrmOfW3v71uC7t3\nUc5o2ztZybqbbDn4HHnTTur2/sG6zSy1T2vd9w5g6AKoC8ngZ3Dx02vruhvzb7G0eW8Mr1y3savI\n48rgfLJhXa9dlbshea+u99bhDchA6W2MMxBv/A7zyOPi5nV4LbKp903kDWGfXT/vJMs+0a9Jn4Gp\n+CKDrr+Qtd5x3wiwo3kZbHhzGKqB70L2UXtWHX4UQ31tBjdnfF7dKAfNgPvX5bqBcTaZLGUeDyPv\nuP6ZegDpsnP9YPnXb3z2UrKm+LDGZyvRImVONgGsXN+vRmaZbmWoH91DyU7XN5BNNZ1ePUNmQ35G\n3qD0o2QgOchIHECeaMZz36VXkAHSh8kT7xMb372qrsflCpAa298K5Mn322Q2bXDT3leRJ/VHkQHL\nw8Y6v6OUe2g9gK9JZp+uI4OiK8iT8LrUKx5blPEwMuh9RB1eUH//68krzi6k5X3JyEDxamon4Ma2\n/TAyuLuIFjezZNhJhwxUrmkMb0MGrZ8ns2E7ksHEixhjdo3Rs97n8cC+kuPqb9l4P5cMvL9G9iPc\ntW7H69bf6Iu0aJJdjnnZq257g9/sMWQrw+AGvis3tpmlLmud38GTNw6v++SFZMXlRjqq5A9bf88k\nL4RaUIcvIjN2neybo5T/IrIJerDO1iGbF99F7TYwjmkeQlbyXk5elHJo/XxwocBLyWNkJ7ew6eM1\n6TMwVV9kBmfzyZ6POi8LyNrkp4Bn1s/2IgOws8ma155klubfybT/VmTz0lvI1PigI+eE3BS2Hkxu\nYegmip1d3svQJewfI5uDViFP9DeynH2UljH9lcjAdHOyn89ryQzY9TSa5MgA7FRG6VTbovwVyH4k\ngxtrbko2cZxLNlksIB8xsswD/LDpzqfe7JYMjm4ma+oHkcHme1jObBGZXb2boYsZdiGDoEEfrA3J\nYOWojtfNQ8mMw3PqNv2VxvZ1d90eWh1w6296Qz14P5+hDsiDZyo+vsW0BwHWduSJ+1qGblcwKGdN\n8mrHzVpuw8cydDf5QXDwHR54+4LTqX0W6/DjgI3GWeZIWe8byWzHuPZ9Hhg4nEI2Z/4DGSh+u/7e\nX2Xo5Dvum7Uu5/ycRGbaT6rDs8gs9JeAt41jegfV7Ww1Mhv5IjJg7eSGw8PKeil5DHsa2UR7MHns\nvIQMYFtVWIZt37vXdbVr3WePITPFg3vVrTveZSQrDTeR2dXjyIz6vwFPGTbep5nkm56PabkmewZ8\nLeMHeuAB7SIy+n9p/W6HuoPtXQ++m9SD7Q/qgfHtdQfYkjxZdXZn+VHmdX/y6p3ObmzHAy9hP5is\nJV5Wv3s7GfC1vtktmfW4nWy+GmScVqsH/A82xpuQlHY9qVzFUMfu7cjgpk0m5JFkM93x5El/EzI7\neS3juBKIzHD+C3mCHzwbsNl0ciZwTIfrZJDFPZ4MGrZi6LFVB5FZj7YZryPITOPWZID6HrKv3yDL\ndxQts991vd1M3g7jbDLzNAhi9yL73bRu9iH72N1f19Hujc+/R2YJn1qPB4Org7u4KndCst5kUPLN\nus3+mgxQ9iErJTeT3UFmdbEMw8odBBNbUIN6svJ6J0MX9cwiA+lx9Wklszi3MXQz4Im4j9eaDF08\n9hqyz9kgs78SWQnrpAWEPC7fTp6L7mDooq5jyWz+mG8x0vgdVq7bwPp1X7mVbCY9jQdeMPDouk1M\nWEav899osmfA1yg/zNDGdxzZxHMoWTs/ngzG/oHaFEdmvM4iA5Ut6o72HLIT6F/Jk2Iv7eDUx/q0\nnMasxvvhl7BvQtbcBneVbtsUNFjPs8mr9RaRgc/gKqbVyFrXxROwruaRtdItyZu4ngW8vn43yFyO\n+WBSt5c5DD3u6a0MPdfzTDLAmDPOeT6onohWJ0+8Hyb797ymHmg7ufKTB2ZxtyZvj3EcWaP+Dhl0\nt+3jtTHwHwxdkbkKecXa++p23PqKWTJjcwv16isyIF5Y9+GXkZWpVgELQ/13NiKbEf+NYdksssnn\nAmrzU8fbcadZbzJzciHZ5HgqebXp5WRz3/bk/bU27Xo5GuUfUrext5AZotlkMPhjOupfRgb9t9Rj\nWuvga/g0yArS5+v++VmG7oH2fDq4ap+h4+ZqZFLgYWT261ZqUFeX7bnUY/U4yngyeUx+L1n5Poh6\nVSNZKbqQocza2nT0/Mm+XpM+A76G/SCNjbrx2apk7XyQCbicrGU+rA5vUneqRWRwdiJDNYLnMQUu\nGFjOZX9I4/2eLP0S9hfU9+M+cDXW9cZkMLE2GQxdC+xdv3soWdMa8zMOl1H2PmQt8QqyOfW8WvaH\nyWzbzQxLqy/ndE8lr5B8M3l/t3XI5pvfkBeR3N52e2CopjtoOnkhmdHp6jL14Vncw8nmpx+SWa/1\nGGfwOEJZR5J3+B40nc0mLxo4jw76j5DB1iVkk8xr6rZ1CdlsdShDD8xu27n+iLrNPIK8hcXt1CYY\nhvWJalvWKOV3mvUmMx47AF8fzDOZAXsjY7gtwTjKHTyWZmOyz+z1DDWtn0JWMMbVb2mEsibithjb\nMZThein5nNDB1cjPrvvQuANXMqAarI9Bi8qrycdffY+hC4QO54EXfCzvY88Gx+S1yODxaPLJFbfU\n49n7yazXT6lXhk7E9tzHa9JnwNcIP0r2cbqczIQMLke+tL52JzMif9fvqB6sriY7WP9kspdjjMu8\nWl2up9QDYC+XsNd1fUM9IX6MPLEfTfYreSV5FU0nl2A3ynwM2Rl6cHLck2xOHdzJ/JGM8QGzddzB\nFWhB1go/xVBW5BnkVWJd3bhxQppOWHYW9/UTsO0dUk9KzQBsuZ5puRzTXoM8aX+HDPS2q9vx0V1M\nv5axY90ntml8djmZLXw+GVxOyC0Mhs1H66z3sOltVY9l25OB6qfouFlp+HZLBhSvq8vyPYY60w+u\n6pywjNs45/+xwKn1/SA799V6HN2SvODmp2T2aMzPtR1lW/tE3R+vqmUcV9fV4XWcnckrHvcYZxm7\nkBW65n3cnkq2QFxdt+nOLuqatN9usmfA17AfJDe879aT5dfIWvjm9UB0Sd3IR206IPuGHV3H23yy\nl2eMy/5kMhD6GkP3gZnIS9gHzxXbs67f08ms06BJ7VXA/h0v4yyy6ex/gFMan72IFk2b9f/PJIO4\nwcN5Bx26B1cddf3okE6bTuo0JyWLSzZp/Jx60cMETH/wAPvHk0Hrvh1Oe5BdO7FuA9eQFbWryUxR\np9twXy8y+3V6XZ4f01HFYdj0j6773z5kZvKhdZv+D4b6Mu5HBhqdZFs7nP8gg8SvkE2knyIzRi8g\n+3Q9h8wi70x239iso3IXks8JHlz89TAyU/xx8qKu2xh225TlWZb6d3eyX+lVZNP8Pgz1gz2azOiu\n2/yf6fqa9Bnw1fgxsnP054BX1+F1yNtEvI2hWx4M+nkt67LmTm7+OQnrYD/yRrKDe+l0dgn78P8j\n73fz4fp+kCH6QOOgMuh03bYPS/NxG4NOvM+l1lDr8N71JLP2WMsig6BPk7XgG4DbGt+dSDZtdn7H\n7Dr9CXmcC5OQxaXjzM2wac8iryzs7CHZzd+AkbNrT2ECOnP3+ar7/6Z0cGXeKNMfPGXgNoayW0eR\n2Z3TyKzorUxAX7mW870BNdNJBl7fBz7b+P5Y8pFyJ9Hy5rNk5WpwdeyuZPD17rq9DbrCrExelbgb\ntaVgHMexXeoxcPs6POh/uTdDAdiEbAeT8htO9gz4qj9EHmTm1A37Www1Sa1VT6zvZ4KvVpwqLzID\n1ukl7MOmv3c9OR1INsk8t/Hd2cBpE7BMC8iM3o1k34sdyezmL8isxTXjOSmTfVN+DnysDh9L3rX8\nLfXE8gM6utHpJGwH0zaLO8ryrE69NUoX2/EI05+w7NpMe/HADuPfIZvJBp3SN6hBxCfJlodDmv8z\nFV5kpv4asnL+NfJ+XouoTZB1nOfXIKlVHzWy0viduj6+1jg3vYYM+tYms1+tHiVF9hv8M/DyOrwi\n2QXhIqb4cxrHtbyTPQO+/rYj/ZC8umfNGgC8j6EHNq/FNHt0Qgfr5DCyg+0VZDp9TGnsEaY3ONju\nQjZhfJahxxLdQzZvHEk2OTyx42XZnszgPIbsu/JhMlOxJtlE8FWGbh8ynptSHgncR22OreV8gOwI\n22lTzSRtC9MyizsJ66mZXZtSmZqp9GocC9Yj+/cNHvx+KzXzybA+f1Mp8GrM03nkbT4Gj5A7iOyk\n/pLGOG2f+DFoEXgyeb+ziwbro667M8mK8h100LxNVlJ/TL1lTS3jzbTsqzYVX5M+A77qD5Edy2+k\n3iKA7LtzIdPopnETsE6OJGvwgweotr0ibGeyL9QgqH1WPXh8kLyS5mw6fo4nmZm6BPha47PdyGB7\nNzJd/wwy23lQi3IOrdOckD5LvqbHiwnOrs2UF5n1vo6hx4itSl5g8wOyA/l/0PGFNhOwDFuSme6b\nqc8jbgTfz+pg+s17nj2b7BLyS2q3mMZ4j6LD5/iSV1PfBDxnstfxRL5mo0kTEVsAvy2l3F9KeUVE\nvIkMDuaTzY8nk2nYB6VSymcj4vpSyq/rcGk5ybXIA8h8MlD5JPnIklXIe1e9u5RSIiI6KIuIeHgp\n5d8i4nrgWRFxHPDJUsp3I+KrZJ+N70bEV8gr+W4bb1mllC9GxF+AhRHx11LKP7Wdf00/pZQ/kPf5\n6mJ/mZEi4lFkxvu55D3DHg8sLKUcGxG/J/sbnlhKGff+2IdSyl3AXRHxG+Cc+ncV4I/khUNtp18i\n4nCyUvpT8vh0EnBRRPyOvDr9DPIq7f9tW16j3KsjYjZwbj1O3ltK+UtX058qwv2zP3WDilLKnyJi\nTTLb8j3ypP/rOs4nyQ6z+5KB2R8nbYZnoHoweQvwplLKJyJiFpl5uqWUcnuH5TyE7LS7qJTyhog4\nnryp6v+RHckXAseVUr5Zx+8q4JsP/LSU8rO205Jmiub+FRE7kfcJfGFErEDenPZNZJPaNyNilVLK\n/3a1T/YhIg4kn/jxBzIY+nEH09yV7P5yKFlpXUj2gVtEdob/DfCuUsrn2pY1Svnrl1KWTMS0pwKD\nr55ExErk4xEWk8HVWuQtJc4lOzN+qJRyf0Q8m0x7n1lKaV170d+LiIPJg8d7SymXdjjd5gF+Fnn1\n4WuA75dSzo2IZ5F9ve4GLimlfDkiZs3EWp001UTEE8grylckA5Vnl1Kurt+dT1aUPjKJs9hKRGxA\nJqw6CVgiYhOyH/LaZHB6DHkF5RKyT+nPSim/nE5B6lRis2NPSil/rDvHP5BXhpxaSrktIl5OXpGy\nRkT8J3mZ83O6zMLogYalta8BflVK+WsH0y0RsTvw36WUH0bEzeTVOm+MiFNLKe+NiBXJ55CtGREr\nllL+1LZcSSMbBAZ1v7yQ7Et0L9mn68yI2JT62CDyKSLTVinlvo6n9wvgFxFxDvmcyLsi4lLyKup7\nSim/rOMZeI2DwVcPImKFenK/hVzndwBLImKtUsq/RsTJ5B18twbOMfCaeKWUKyPiu13UEhsH+IeT\nNzbcPyIWlFJujYg7yAdAv6oGXu8FXkLeauJLgMGXNEHqfrkz+XSH55ZSvlf72v6cDLieRl6I8/pS\nyncncVansh8BJ9bj15Hkldk/neR5mvYMviZYPTH/taa8zyTvx3IoeYuBz5F38r0fuKCU8vvG/1ib\nmGBdpecbHVPfQF7u/SPgExHx9JrdvJu8ZcZ3an+/95P3FPpdF+VLWqo1yS4fTyL72P6cfEbjJsBR\ng6y3x91RXU1elX04mRywO0wH7PPVg1rzOp7s1H1B/eyV5P297iWvujlwql9do5FFxI7kJetHl1Lu\nqJ9dRjYvf5P87Y+unXnt4yX1LCIWkLfzeV290GZv8j5ZhwBLDLqWLSJml1L+bJDaDTNf/diMvKv6\n7yJijVLKf5dS3h4RTyc7gJ5g4DWt/R/ZpLxXRDyNfFbkYrJj/Y+AYwdXNRp4Sf0rpVwREX8FLo+I\np5C3dnlj1/2kZri/gH28umLmq2MREfC3pqiHA78upfy2Nju+max9XVNK+X+D8eu41iamqYhYg+zr\ndQxZmx48rPt3pZRPTOKsSWpo3Lfq8loB/tvxenLnTA82Bl8danSsJyIOInfyfyX7HJzG0J3rzwe+\nUEr5v8maV3UvIlaqV7U+nnzm2ktKKddN9nxJGhIR+5PPCzy1lPLZyZ4fPTgZfHUkIrYGnkherrwB\neSXb88mmp/3IZ4cdSj5b8CXAU015zyz13l47ko8renMp5YpJniVJI/BmxJpsBl8diIj1yYcjHwv8\nC3nH5HNLKUc3xnkTeW+UD0bExoN7pGhmiYjVgQ1KPlbIpmRJ0t9ZYbJnYIaYTXZGPBS4APhfYNuI\neGtjnN8Bm9b3i/udPfWllPKHUorP1pMkjcrgq6Wa3VhMZr5eDywupdxL3hNlfkRcHBFHA88CrgXo\n4m7qkiRpejL4aqmR3bgbeCtwcEQ8o5Tyc7Kv12+BRwCn2/lakiTZ52ucGreI2AH4M5nx+nW9z9OZ\nwFmllM+M9D+TMb+SJGlq8Car41QDr/nAZcBXgEdFxDNLKf8YEX8G3hkRNAMwAy9JkmTwNU4R8Ujg\nYOAppZRvR8TpwOcj4ohSyuciYjb56CBJkqS/Mfgah3pPrw8AAfxTbU58a0QU4NqImF9K+cc6rk2N\nkiTpb+xwP0YRMRc4CriVfKbfHsD6AKWUt5F3r58zGN/AS5IkNZn5Wg7Dslf3AT8nO9n/CTgQKBHx\n0VLK4lLKm0f4H0mSJMDM13Kpnet3rx3q/0I+Quhu8sap3wSeDBwfESs1/2dSZlaSJE1pBl/Lb23g\njRFxVA3A/hFYFdgJ+GfgulLKHydzBiVJ0tRns+NyKqVcFRF/Bc6NiBVKKR+PiGuArYHzfUCrJEla\nHgZfY1BK+VK9ovGjEbE7cBBwooGXJElaXt7hfhwi4tHA7sDtpZRvTfb8SJKk6cPgS5IkqUd2uJck\nSeqRwZckSVKPDL4kSZJ6ZPAlSZLUI4MvSZKkHhl8SZIk9cjgS5IkqUf/H7bBvjzG9yqaAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e2d8d5c88>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Exercise: why is there only one coefficient vector\n",
    "# lr.predict_proba(train_X[10])\n",
    "weights = vectorizer.inverse_transform(lr.coef_)[0]\n",
    "sorted_weights = sorted(weights.items(), key=lambda t: t[1])\n",
    "util.plot_bar_graph([w for _,w in sorted_weights[:20]],\n",
    "                    [f for f,_ in sorted_weights[:20]],rotation=45)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly, the features we see now are much more obvious **discriminative** features for the negative class. Intuitively, the conditional log-likelihood leads to such discriminative weights because it down-weighs a feature $f_i$ for a class $y$ if $f_i$ is active in an instance $\\x$, but the label of the instance is **not** $y$. Frequent but non-discriminative words hence get constantly pushed down.   \n",
    "\n",
    "You can change the order of weights to see the features indicative of the positive class. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl0AAAGRCAYAAABfQAZ5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xv8bfWc+PHXu3O6SREdoXtERRc6SkRCKqGSS3JXKkoa\n45IhPzNuhXEtEmJcG4NoiOQyLkOj06ASmSPNVBjHZRjXSu/fH+/P9l1955zO/n73+q7vubyej8f3\n8d177bXX+qy91/6s9+e6IjORJEnS3FpnvhMgSZK0NjDokiRJGoBBlyRJ0gAMuiRJkgZg0CVJkjQA\ngy5JkqQBGHRJkiQNwKBLkiRpAAZdkiRJA1g43wlYns022yy33Xbb+U6GJEnSSl1yySU/z8xFK1tv\nlQy6tt12W5YsWTLfyZAkSVqpiPjPcdazeVGSJGkABl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQ\nJUmSNICxgq6IODAiroyIpRFx8i2sd5+IuDEiHjPT90qSJK3JVhp0RcQC4AzgIGBn4AkRsfMK1jsN\n+NxM3ytJkrSmG6ema09gaWZelZnXA+cAhyxnvecAHwN+Nov3SpIkrdHGCbq2AK7pPL+2LfuLiNgC\nOAx4+0zfK0mStDboqyP9m4AXZeZNs91ARBwTEUsiYsmyZct6SpYkSdKqYZx7L14HbNV5vmVb1rUY\nOCciADYDHh4RN475XgAy8yzgLIDFixfnOImXJElaXYwTdF0M7BAR21EB0xHAkd0VMnO70eOIeC/w\nqcz8REQsXNl7JUmS1gYrDboy88aIOAG4AFgAnJ2Z342I49rrZ870vf0kXZIkafURmateS97ixYtz\nyZIl850MSZI0oG1P/nSv27v61IN73d6KRMQlmbl4Zes5I70kSdIADLokSZIGYNAlSZI0AIMuSZKk\nARh0SZIkDcCgS5IkaQAGXZIkSQMw6JIkSRqAQZckSdIADLokSZIGYNAlSZI0AIMuSZKkARh0SZIk\nDcCgS5IkaQAGXZIkSQMw6JIkSRqAQZckSdIADLokSZIGYNAlSZI0AIMuSZKkARh0SZIkDcCgS5Ik\naQAGXZIkSQMw6JIkSRqAQZckSdIADLokSZIGYNAlSZI0AIMuSZKkARh0SZIkDcCgS5IkaQAGXZIk\nSQMYK+iKiAMj4sqIWBoRJy/n9UMi4tKI+HZELImIfTqvXR0Rl41e6zPxkiRJq4uFK1shIhYAZwD7\nA9cCF0fEeZl5RWe1LwDnZWZGxK7AR4AdO6/vl5k/7zHdkiRJq5Vxarr2BJZm5lWZeT1wDnBId4XM\n/G1mZnu6EZBIkiTpL8YJurYAruk8v7Ytu5mIOCwivg98GnhG56UEPh8Rl0TEMSvaSUQc05omlyxb\ntmy81EuSJK0meutIn5nnZuaOwKHAKzov7ZOZuwMHAcdHxANX8P6zMnNxZi5etGhRX8mSJElaJYwT\ndF0HbNV5vmVbtlyZ+RVg+4jYrD2/rv3/GXAu1VwpSZK0Vhkn6LoY2CEitouI9YAjgPO6K0TEXSMi\n2uN7A+sDv4iIjSJi47Z8I+BhwOV9HoAkSdLqYKWjFzPzxog4AbgAWACcnZnfjYjj2utnAocDT4mI\nG4A/AI9vIxk3B85t8dhC4EOZ+dk5OhZJkqRV1kqDLoDMPB84f9qyMzuPTwNOW877rgJ2mzCNkiRJ\nq72xgi5JkrT22vbkT/e+zatPPbj3ba7qvA2QJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA3A\noEuSJGkABl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA3AoEuSJGkA\nBl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBrBwvhMgSZJmZ9uTP937Nq8+9eDet6liTZckSdIADLok\nSZIGYNAlSZI0AIMuSZKkARh0SZIkDcDRi5IkzYG+RxY6qnD1Z9AlSVqrOM2C5ovNi5IkSQMw6JIk\nSRqAQZckSdIAxgq6IuLAiLgyIpZGxMnLef2QiLg0Ir4dEUsiYp9x3ytJkrQ2WGlH+ohYAJwB7A9c\nC1wcEedl5hWd1b4AnJeZGRG7Ah8BdhzzvZIkAY7405ptnJquPYGlmXlVZl4PnAMc0l0hM3+bmdme\nbgTkuO+VJElaG4wTdG0BXNN5fm1bdjMRcVhEfB/4NPCMmby3vf+Y1jS5ZNmyZeOkXZIkabXRW0f6\nzDw3M3cEDgVeMYv3n5WZizNz8aJFi/pKliRJ0iphnMlRrwO26jzfsi1brsz8SkRsHxGbzfS9kqRV\nk32tpMmNU9N1MbBDRGwXEesBRwDndVeIiLtGRLTH9wbWB34xznslSZLWBiut6crMGyPiBOACYAFw\ndmZ+NyKOa6+fCRwOPCUibgD+ADy+daxf7nvn6FgkSZJWWWPdezEzzwfOn7bszM7j04DTxn2vJEnS\n2sYZ6SVJkgYwVk2XJGnV1HcHd7CTuzRXrOmSJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEY\ndEmSJA3AoEuSJGkABl2SJEkDcHJUSZojfU9c6qSl0urNoEvSWsdZ3CXNB5sXJUmSBmDQJUmSNACD\nLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkDMOiSJEkagPN0SRrLUBN9OqGopDWVQZe0mnOiT0laPRh0\nSXPIWhtJ0oh9uiRJkgZgTZfWSjbJSZKGZk2XJEnSAAy6JEmSBmDQJUmSNACDLkmSpAHYkV6rFDu4\nS5LWVNZ0SZIkDWCsoCsiDoyIKyNiaUScvJzXnxgRl0bEZRHx9YjYrfPa1W35tyNiSZ+JlyRJWl2s\ntHkxIhYAZwD7A9cCF0fEeZl5RWe1HwH7ZuavIuIg4Cxgr87r+2Xmz3tMtyRJ0mplnJquPYGlmXlV\nZl4PnAMc0l0hM7+emb9qTy8Ctuw3mZIkSau3cYKuLYBrOs+vbctW5CjgM53nCXw+Ii6JiGNmnkRJ\nkqTVX6+jFyNiPyro2qezeJ/MvC4i7gBcGBHfz8yvLOe9xwDHAGy99dZ9JkuSJGnejVPTdR2wVef5\nlm3ZzUTErsC7gEMy8xej5Zl5Xfv/M+Bcqrny/8jMszJzcWYuXrRo0fhHIEmStBoYJ+i6GNghIraL\niPWAI4DzuitExNbAx4EnZ+YPOss3ioiNR4+BhwGX95V4SZKk1cVKmxcz88aIOAG4AFgAnJ2Z342I\n49rrZwIvA24PvC0iAG7MzMXA5sC5bdlC4EOZ+dk5ORJJkqRV2Fh9ujLzfOD8acvO7Dw+Gjh6Oe+7\nCtht+nJJkqS1jTPSS5IkDcCgS5IkaQAGXZIkSQMw6JIkSRqAQZckSdIAep2RXvNj25M/3fs2rz71\n4Dnfz/L2IUnSmsqaLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQ\nJUmSNACDLkmSpAE4I/0ccxZ3SZIE1nRJkiQNYq2t6bIGSpIkDcmaLkmSpAEYdEmSJA3AoEuSJGkA\nBl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkD\nMOiSJEkawFhBV0QcGBFXRsTSiDh5Oa8/MSIujYjLIuLrEbHbuO+VJElaG6w06IqIBcAZwEHAzsAT\nImLnaav9CNg3M3cBXgGcNYP3SpIkrfHGqenaE1iamVdl5vXAOcAh3RUy8+uZ+av29CJgy3HfK0mS\ntDYYJ+jaArim8/zatmxFjgI+M9P3RsQxEbEkIpYsW7ZsjGRJkiStPnrtSB8R+1FB14tm+t7MPCsz\nF2fm4kWLFvWZLEmSpHm3cIx1rgO26jzfsi27mYjYFXgXcFBm/mIm75UkSVrTjVPTdTGwQ0RsFxHr\nAUcA53VXiIitgY8DT87MH8zkvZIkSWuDldZ0ZeaNEXECcAGwADg7M78bEce1188EXgbcHnhbRADc\n2JoKl/veOToWSZKkVdY4zYtk5vnA+dOWndl5fDRw9LjvlSRJWts4I70kSdIADLokSZIGYNAlSZI0\nAIMuSZKkARh0SZIkDcCgS5IkaQAGXZIkSQMw6JIkSRqAQZckSdIADLokSZIGYNAlSZI0AIMuSZKk\nARh0SZIkDcCgS5IkaQAGXZIkSQMw6JIkSRqAQZckSdIADLokSZIGYNAlSZI0AIMuSZKkARh0SZIk\nDcCgS5IkaQAGXZIkSQMw6JIkSRqAQZckSdIADLokSZIGYNAlSZI0AIMuSZKkARh0SZIkDcCgS5Ik\naQBjBV0RcWBEXBkRSyPi5OW8vmNEfCMi/hQRz5/22tURcVlEfDsilvSVcEmSpNXJwpWtEBELgDOA\n/YFrgYsj4rzMvKKz2i+BE4FDV7CZ/TLz55MmVpIkaXU1Tk3XnsDSzLwqM68HzgEO6a6QmT/LzIuB\nG+YgjZIkSau9cYKuLYBrOs+vbcvGlcDnI+KSiDhmRStFxDERsSQilixbtmwGm5ckSVr1DdGRfp/M\n3B04CDg+Ih64vJUy86zMXJyZixctWjRAsiRJkoYzTtB1HbBV5/mWbdlYMvO69v9nwLlUc6UkSdJa\nZZyg62Jgh4jYLiLWA44Azhtn4xGxUURsPHoMPAy4fLaJlSRJWl2tdPRiZt4YEScAFwALgLMz87sR\ncVx7/cyIuCOwBNgEuCkiTgJ2BjYDzo2I0b4+lJmfnZtDkSRJWnWtNOgCyMzzgfOnLTuz8/inVLPj\ndL8BdpskgZIkSWsCZ6SXJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA3AoEuSJGkA\nBl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkD\nMOiSJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkDMOiSJEka\ngEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA1grKArIg6MiCsjYmlEnLyc13eMiG9ExJ8i\n4vkzea8kSdLaYKVBV0QsAM4ADgJ2Bp4QETtPW+2XwInA62fxXkmSpDXeODVdewJLM/OqzLweOAc4\npLtCZv4sMy8GbpjpeyVJktYG4wRdWwDXdJ5f25aNY5L3SpIkrTFWmY70EXFMRCyJiCXLli2b7+RI\nkiT1apyg6zpgq87zLduycYz93sw8KzMXZ+biRYsWjbl5SZKk1cM4QdfFwA4RsV1ErAccAZw35vYn\nea8kSdIaY+HKVsjMGyPiBOACYAFwdmZ+NyKOa6+fGRF3BJYAmwA3RcRJwM6Z+ZvlvXeuDkaSJGlV\ntdKgCyAzzwfOn7bszM7jn1JNh2O9V5IkaW2zynSklyRJWpMZdEmSJA3AoEuSJGkABl2SJEkDMOiS\nJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkDMOiSJEkagEGX\nJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkDMOiSJEkagEGXJEnSAAy6\nJEmSBmDQJUmSNACDLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBjBW\n0BURB0bElRGxNCJOXs7rERFvaa9fGhH37rx2dURcFhHfjoglfSZekiRpdbFwZStExALgDGB/4Frg\n4og4LzOv6Kx2ELBD+9sLeHv7P7JfZv68t1RLkiStZsap6doTWJqZV2Xm9cA5wCHT1jkEeF+Wi4Db\nRsSdek6rJEnSamucoGsL4JrO82vbsnHXSeDzEXFJRByzop1ExDERsSQilixbtmyMZEmSJK0+huhI\nv09m7k41QR4fEQ9c3kqZeVZmLs7MxYsWLRogWZIkScMZJ+i6Dtiq83zLtmysdTJz9P9nwLlUc6Uk\nSdJaZZyg62Jgh4jYLiLWA44Azpu2znnAU9ooxvsCv87Mn0TERhGxMUBEbAQ8DLi8x/RLkiStFlY6\nejEzb4yIE4ALgAXA2Zn53Yg4rr1+JnA+8HBgKfB74Ont7ZsD50bEaF8fyszP9n4UkiRJq7iVBl0A\nmXk+FVh1l53ZeZzA8ct531XAbhOmUZIkabXnjPSSJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQJUmS\nNACDLkmSpAEYdEmSJA3AoEuSJGkABl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmS\npAEYdEmSJA3AoEuSJGkABl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAEYdEmS\nJA3AoEuSJGkABl2SJEkDMOiSJEkagEGXJEnSAAy6JEmSBmDQJUmSNACDLkmSpAGMFXRFxIERcWVE\nLI2Ik5fzekTEW9rrl0bEvcd9ryRJ0tpgpUFXRCwAzgAOAnYGnhARO09b7SBgh/Z3DPD2GbxXkiRp\njTdOTdeewNLMvCozrwfOAQ6Zts4hwPuyXATcNiLuNOZ7JUmS1niRmbe8QsRjgAMz8+j2/MnAXpl5\nQmedTwGnZubX2vMvAC8Ctl3ZezvbOIaqJQO4O3DlZIfWm82An68h+1mTjmWo/axJxzLUfjyWtXs/\nHsvavZ816VhmYpvMXLSylRYOkZJxZOZZwFnznY7pImJJZi5eE/azJh3LUPtZk45lqP14LGv3fjyW\ntXs/a9KxzIVxgq7rgK06z7dsy8ZZZ90x3itJkrTGG6dP18XADhGxXUSsBxwBnDdtnfOAp7RRjPcF\nfp2ZPxnzvZIkSWu8ldZ0ZeaNEXECcAGwADg7M78bEce1188EzgceDiwFfg88/ZbeOydHMneGavIc\nYj9r0rEMtZ816ViG2o/Hsnbvx2NZu/ezJh1L71bakV6SJEmTc0Z6SZKkARh0SZIkDcCgS5IkaQAG\nXVolRER0/2tKu53WYJ9N57swfxiQ576GEBEbtf+eb/PATHUC83nSRsQuEfGoOdx+TH88VxfhiIic\nGtGx01zs4xb2vXNE7DnD9yzoPJ7TcyAi7glcGBGbZGYOcc61/RwMvCMi1hk62FvbRMRoFPngk1VP\n+51vNum2IuKg9njXiHjcpOlb2f7mcvszERG3j4jt2uM9I2LTnrc/8bG272c7YElE7DIX+cmKtjdA\nPhkRsUV7vHXfn3+fVpkZ6Vc33UAhIo4H/tymzxhi3wuBfYEN2/N1MvOmHrcf7Qf5CODBwG0i4hWZ\neXVf++ia9jk+te13Wc7R0NrO8S0GngXcOiJek5nfHuO9xwN3jYjfZeZLRxnXXKQ1IhZk5uUR8XPg\nIxHx2Mz837naX2e/uwLPBV7c53m1kn12f087ADfM9Hxr6V4EfI86f27oMX2HAb8D1snMz/a0zTsD\nCzLzmhasHBERVwDfyszP9bGPley/+5k/B9g8It6Qmb+c5SYXALtExOuBPwOH95TU/2Na2u9F3Q7m\npswcfPLtFlBsDZwQEddT5+BRs91Wy1P2BW5PnR//1NPvfcPM/FFEnAO8PyKOzMwr+spPpn0njwZu\nAH6QmVfOZT7Z3AN4RNR8oPdllp//EKzpmqXOyfXXwJHAvw6x33bi3ghcDjytlVh6vTC2H8hDgZcB\n76Bqn944VzVdAK1U/DTgsMz8GXDnudpXJ6B8L/BT4FbAcVET+95SGh9F/ZgvpH7gH+5sr+8S427A\n6RFx28x8XEvnuRGx8VzWeEXEbYEnUfc//UlbNuf5ROf39ELgrcC7IuK0iLjjOO+PiEOAfwAOBd4P\nPKSvtEXNNfh84HbAxyLiAT1scwHwZKo28VHAKcBXgDsCh0Tdi3ZOdT7zJ1K/vTMnCLho+dIXgNsC\nf8zM/2jbX3e0Tl/nbSftJwFnUOfMq9t5MKiWlsuBjYDHAedm5q+jUyM+k21FxAHUMf0Z+MeIeNqk\naYyIuwMfiIidM/NvgXOogtzOfeUnne/kSODVVF55QkQcPnp9Dmu8fgzsCDwP+HybnH2Vqg0dMeia\nQKvCvD9wIPDziHhsRLw+IjbvcR93jIjt2+MdgBdHxEaZ+S/A6cBBUU1AE32XEbF5RHQvVPcBTgTu\nBiRwUmbeFBHrT7Kfzv6mN1/eDjgTuHtE/A3wrxHx9ojYsI/9Tdv3esDBwPMy8yXA3wD/CRwbEbuv\n4D37A48FXp2Z52fm7i2tH4SpDKdHPwS2AP4uIm6TmU8DrqUCr16bGkfbaQH9/1DB6JeB50fE5u17\nn/PMqwX6+2XmgcCPgB2A/x7jfXeiJmR+IPBF6lxaEs0E6YmI2AbYn6rx3YL6XL7eDSRmIzP/DPwj\n8DXgZOBjmflu4JXA54F7RcRKb547GxGxOCL26Cy6D/DWzLy2/Ta6TZ7jbG90/mwIXAocBHw+IkbN\n4jdE1er1+juJiJ2pwPVA4CTgXODp045tznTz3Far+nHg9cBjIuLA9h0TERuMu72IuBVwDJXX/Bb4\nd2DiWs/MvJIKTF4WETtm5qnAB+g58IqIxwOHAPei7kDzfeCBrear93xylOZWWPgi8CFgm4h42Gh/\nk/5W+2bQNQPLOSn/SJXq3k2VTPYA9qKClb48j6plujtV+7AN8JmIeCawLXVn85smuTC2EtnBwNHR\n+mQANwIvpo7lKZn5nxHxWOBFsynBTdtftxp6o7b4y8BxwAnAldSFbhtg50n2tTyZeT2wATDKCC4D\nvtn29eSI2Ho5b9uAqoW4b0SM7ie6B3C/iHh3n+mLalb8LVWDujnwmmmB10fa814ysJYxPRJ4Q0Sc\nTmX27wCuB54bEXeco+bTXSLi3M6iPwMXRMTLqXu2HtHSdu+VbOq31N0wXgy8ADg8M38O7Ec10cw6\nicAy6jN/GdWk/5h2MX1qRNxtVhudulBcTdXOXQYcHxHbZuYvgM9StctbTJD2Fe17O2AT4LpOUPcr\nqkls9NsAeHBEjNW/sn1HBwNnUzV2dwb+HvgBVUjYH3hb53cz27QvL3/7TWb+pn2WX6Xu7bu832+v\nWh52U3t8aMsbL8rM11BB0nNacLsfda4s91obERuMAlLgDlQB9zvAY4D/BzwxM38cEU+KiAfOJp2j\nfWfmCdTn8+ppgdeHImLXnn7j27e03zMz/wh8lDoPDo6e+yCPriMRsXv7LX4uM59NFaAfGxF7RFVU\nPHaVCrwy078x/miz97fHR1IlkYdRGdiTgW3ba4cB7wPW73HfpwPvBO7anh8EPAX4BvAH4IU97GNr\nKuh5J1V7twVwNXBye/3+VF+Z/Xs8rhOBjwD/BDwKWG/0OVNB1xLgzn19d1T1897t8U5UoHxCe35P\n4JPAx4B9Ou/dD9iNyhB3BT5I1aps0Vlnuzk4x27V/m9Eld7OBG7Tln2cKtVHT/t9UPust6eaSd5F\n9fe8L/AWqvS+Xl/fe2e/m7TP+yOd7+dLVNCxYVt2PNVkdevlvP+h7bewCfC3wLc73+++7XzdZZZp\n2wd4dnv8Hqq5bPTakcBFMz03qSBudC7eHVjc0r45FdR9kuqbsj1wBXCPnj/vRwDfogqKW1EFncVU\nrcSlwCOpgtxj2/Oxjo+qKbuCqpn8MpX/rUcV6k+jgqFH9PjbeCSwZ3v8WeDUzmtvAF7U97l6C+k6\niqqVfivVr2zntvyZ7Tf1PeBut/D++7d1X0gFa7emmub+RMtjOt/P3hN8Ztt1Hr+cyj92bM//Dvj6\nTH7jTMt7qNqtu3S2dwlw9/b8Tu0YN5+Dz//h7dw7Gfg3Ko9eSNV8/jNVU/6Qoc6HsdI83wlY3f6o\nmpivtS/7j8DDpr12GRXl973fN1MByvadZVtRgcvrqQ6+s9nuOp3HdwKeQ11k7kVdGL5JlYa+NmnG\nOW2/T6WaURZRQddfggiqc/u/A7v2uL8DqKruy6hAYlcq0Pt0S8cPqabU04Bntfcc3zKjF1JV85u0\nzOW9wLOBO/WYvm4GeXRL4zOp/mYbtu/gdGDTts7EwWhnfy9un8/DqEB+285ruwM79Xwud491Yyqo\nPKc9/2uq9uf57e8ylhN8tLReSTVHQl283tne+xrqYjfj85UKFBZQTTzvoJpINqSCwy+03+HFzDKY\na/t4BFX6/yTVj+vRwC7Uhfv3wGeAg3r+zNdv588hVJD0cCq/Og+4CxWkfpzKY74w7vFRhbUT2+e0\nd8svtm2vbdb+32769z7BcZxIBY53a8/vQeVXn6MutFfQCqdz/Uc1Z7+nc7wncfPA667Alit471ZU\nYW6z9rn/EnhO5/V/pJrL3k7lhYdMkM5nU/c/fi3wN23Z69s+7tGe336G27x19zul7oP41c5n8TdU\nXjL6LBbMwed/D6qgtR3wjPabuoqpgHxHYPEQ58KM0j3fCVjV/9qPY1TqXtQypfWpZr9PtQx6/ZYx\nv2N0kk24z9GJfD+q5myP9vzvqYvvDqOTmCpRfoMZBnrAup393Kfta6d2PKPAa7e23sbA1t209XCM\nT6WCnJOoi8xC6oJ3Jyojn6j2iJsHk/ekSj3btf28H3gFVaOwAVWjs1XLRK9smeVuLaO6FVWKurCz\nvUe0TOY2c3C+HUcNytiHatZ6c0v3BtQF8g2TfAd0als6y46igoqvMFVafRrw8rn4TXX2u177f6t2\nXr+/PX8kVfNzKq00Pu19G1LB8sHTlt+TaiY/iqkarxl9Vp3z/FbtHH0LcGRb9qi2/e1nuM2tgHe2\nx7ejgpp7t+fPoPKNewG3ace9Z8+f88L2/4VUDd2/UXnZhlTB4tO0wJoqWIx1AaZq6F5H1fx9kQqQ\nR7Uzh7fXFk6Y9m4hc0+q9mh0wb8/8ARgUyofeTY95L+3kJZRfrlO+z2+narReSxT+fFzgZu4hQJj\n+w0eShVoop3v7wdeAjyos94j2jHu0d3/DNP8cCo/WUQFd//Qee10qlZy3Zlsmwrc3zU6BzrL39DO\n7e3a81e282JG259BOraguoQ8kArEN6FqCX9GK4ytin/znoBV+a99qS+ggo6F7e+9wJvaCTxqAnoW\nVVqcVW3TCvb9KKpE/UKqBPG4tvwt1AXybu3Hf3fgGmCrGWz7jsAbqSazh1Cj1E6nRsgdTjVpHQ98\nmGkXtllodWz2AAAgAElEQVQey//5wVFB638DH+0sO5aqaZqoVNSO613tx74hFWD9Vyfz2oyqETmD\n6hNH+/F+lFbCpwKdl1IB12eBDdryI6jA9FY9fc/dWp8tqP6Bt6MuIF9rx/FuKhDdgAlquKbt6z7t\nb5u27cvaebxhW34Z/de2bM9UUPdsKth4c0vDZtQF4H2d9RcsL/1UIeefOt/VRqPPb8L03ZnqD3LQ\naLtUjdcFVHP+rAMIKogfnWvvBR7dee11wIfb4w16/szvAxzbHu9HFSo+BmzSlt2WCvS/Ctx/htte\nAHyi/W7e3n4rO7V9focJasapYGQDqsbyFW3Zpu2cOaedNxe0Y3lmn5/ZitLTeXwnpmpEX0HVIO3Z\nef1ZtKa1W9jeelR/w88CD2jn2qupgsY9qHx9oryXKkiPusEcRdUGjgo6o64qd5jhNm9PtQxsTwW8\nH+DmgeIZVBeF0e98RjVo43wHVD7ZDfZOAp7bHj+Vum4ZdK2uf1TAtStTfX9eSXX4HQVcR1LVv2MH\nPSvZ3zpUu/7HqJLvqP9Itw/RGcBuneczauZqmcXHqIv5m2lt3lTfnh9RpfkNqBqvWTejLGe/R1FB\n7APbcb6PquW6XcuoLqefmsJ1qCB4+5ZRb0YFq6czVZ2+GdWstVPnfZtSJcJ1W6Z4EfCTzutPovoc\nLerp8+hm5Ce0z/+2VK3HhW353ajg9MXAuhPs685M1STt3bZ5NpXp70WVus+jLqBfBh45PY0THuum\nVKD/MuoifzFVm/cRqoR83/Zb+3QnndNr5DbrPH4VVWOzbnt+P6pEf8dZpu+FVIHjiVSpudtt4EtU\nc8xtJ/yOLwS+C/wV1YS8e1u+T/tNz0UTzJ3a312ofOx27bfwPqZqJDal+imOlYdRF71Rf52t2/ZO\noDp+/wvVAnDIJOcPrQBL1Tp/i+qHtA4VOL6FqtVcQNUsTdyndQbpOrH9Zj7U0rQBlYe+hjGCVqYC\nh9H146T2u1vcvptXte/ml8BDJzjXNqH6gZ5O1fx8o/Pas9pnOON+mu03+lmqJeRKKth+JbBvZ52r\nqJq7iWo5V7D/g6naxbPb73JhOwc+QFUUfI9Wyzjbc2/Oz6H5TsCq/kc1MzyRClCeRF2MT29f/Nuo\ni8dEfbjolG5pEXz74b2SavIZlUoO5uYdImdcs8ZUNfgCqonssnZ867fljwc+OT1dszyubiZwQPvM\nXkf1vTmpfbZntWP9Z3puGqAu5pdQF5U7U6XSNzFVQzK6YI8+k+OB86m+Nc+hLi7fb89PZcK+PLeQ\nzkOpmtNR09Z9qY6zC6iO4p+gh/5jVDD1aerieL+27Ii2/L7t+Sa0i2/fmRZVq3oaVUt1zOgcoy40\nH2zPb81yavOoppYL2vlzWMtsX0sFMS9vn9es+r1QTTDndz7/I6kCwMFUjfP5fXz+bdsfBn7dzs0z\nqIvW92eb9pXsa3SB34bqgvCq9nyd9rt7N1N5y1jfNVUj80aq2egYqi/a2zvn08ZM2Idr+vva7/A7\ntEE9neVPpZob57JJcWHn8eOYmofsDOCCtvx27fN8BWPkmVQt+pVM9Ut7VjvH9mjfze7AfSZI86gf\n1VHtt/5iKkh8cFv270xwzaIKKL+j+lwuoK5Tp1I1aoe086GXwUXT9rsLVRDeirpm/bDlA+tRfUHf\nABw6V+dCb8cx3wlYVf+o6u19W+YSVGfXtwFPba/vQ5Wut+1hX4+lSr/7t8xxg3YSXcNUR+F9W+Z8\nr0mOqf3fuP1fhyrxvAfYoS3bnyp5zbpWpbuv9ng7KogZdXA8iM7IwbZs4maVzvF1+3O9nip9jwKv\n17V933raeke09bakSk2jjt2bUyWp47iFUUgTpPnO1CjR0Qi+BZ3v5SJW0JF8BttfyM2D+rOo6RVG\n38WGVKB9MZ0mrx6Pb/oFdJ/2+Z7LzWsZv7aijJqqAfwOVevxMaop7MT22sFU0HS/5e1vBdtbj6kL\n09Pa7+pT09Z5TPstfpEeBnRMO9c+2L7zvdp598Bx0z7Tz5wajPESqrnqPCowGPXx+hCVp82oRoLK\nn+5NdcR+CTWtxsX0UNs/Le3PoS7gJ1JNlt9jqiP4veh0B5iLP6qp9HimgshHUrXEz6UKAKNC2/ZU\nMDr26DwqGLqUqaD3WCr/eci09WbaJ3Fv2pyDVC35V6hCzSHt+38vk1cSbEONHP4+FcRtTl2/zqOn\n1orpx8/UiNsnUYWif6P196P1j5ztZzb037wnYFX663zB3R/+OdQ8VVD9Ot5Cu3VMD/vbjNYcAvwH\n8D+0Kmqqs/wpVE3N3zHL0VjLObYDqUz/JKqKPqhq6C+2Y/sKNc/RJMfVvcD8FRU8XAOc1pat19Lx\nD9QEpb39UKiL8BtG+2rL3kj1Q7gdVWq+27T33JoKqvegSu+f62aoc3GOTftODqBGRz6989q6LT2z\n7qdEBVyHUyXnxwNva8s/Qo2cGx3jhlT/jPvO4bHuSfW125AaVfQmqoS8d/vOvkO7uC1nO8dRF8CD\nqZqNZ1EXvZOYXZPfXdt3/A9UE8WT2+/sxGnr3YbWX6ynz6P7u/hn4LPLe63Hz/wp7XMeNQVuS9Wo\nvLzz3c+6Bq99Pneg8qmPMcvBCyvY9rOpGtgtgV9QNaSPbOfJ/2vrTJwHryQNj6BqA49rx3oYVUv5\nmc46R1E19yudIogqfN6x8/xF1GjLUTPvs5mshmu99nl9nTbikSpkdoO7Pvsd35u6bj29Pd+Anrpe\nTNvPw9pn/DCqZvvbTLXMPICqfdxmLs+FXo9nvhOwKv5N+2EcDry+8/wYqip1xpn9tH2s2zKru1GB\nz4vaD+XUaevtT5XkR80/k4xc258qiTywZWgfpy74QZV4v8TUKKY+Ms4D2sVlfaqGYylTNYXrtfRM\nPHcLU8HLvamaoSe1H+IXO+u8naohWTjtvc+mAsNnAb+hbiExeu2Z7bvutXNz2/aTqaDj8VQguG/L\nTJ7a834eTA2l/iGdjrlUk+XHOplXb5nxctJwAhV4v5q6iG1KBVHvpQoTH2M5NRbtnHlcy2y3aOfo\nNu21z1L9OmYVFFM1oL9hanqQUTPuc2azvRnstxt4fQJ4bY/b7gZcm1L9JX9NpzMzVUvxNVqNUY/7\nfglwVk/b2oQaQHInqpbrAqpW7j1U4PNNVhCg97T/nZjqmP1wqmb82JZPnkLlobu3fONSxqiJpvrU\n/QtVgO52An8PlS9OOlp7b2quutE8b+dRtbjHUKMpT2Fu+gzuRk2T8ew5+i52bnnGPu3506g+aodQ\nefZltP6nq8vfvCdgVfij0yzVfnA/odqrD6KCo+8AT+qs38tUAVTJYAsq4Bq173+dqRqJ+zLDEUW3\ncGwLmRoZ81DqAv8y6oL3kJahTDQZI1XlP+o3sjl1UfwOU0O8H0L1ZTi2p89vc6aqmHehRjb9Vef1\nC6lZikfPd5r2/mNbBj4a5n4aFSRuTQVj35n0M1lBuo+lLnyPpEZVjvo37Uc1DRzZ1zndHr+3fd8P\npI1aa8vPpy7Mfffd6u57NGR9I6pG9RKqo/VmVI3XqSwn8Kaa7r9H9fd7D1VAuJrqgH6P9vntNkEa\n70oFvt8CHt+W7UE1Wzyh7+982r5HncSfQdW29drhmCos/EN7/AXgS9Ne34r+Bv6M8pcjqOBow562\nuz51Qf/SaD9UAHk8rXvEHH03oykc3sPU4KmDqULbsVRh8YVUM/0HpucpK/hsdqOa+F5GFRZOZqqF\n4wntd/iACdO9ZfttLGnn1bG07gJUbdxd5vAzu+dcbJ8KHj9PNfPv3/ndPJnKq99CG2zQdx42l3/z\nnoD5/pt2gRhNC3BXajTPV5iaNuAceipdTdvnTlQp7hXUyLlbU7UC76ea5CYetk9dbEf9Ru5ANa2M\nJtm8jCpJTFwtTJVQ78jU1Ay7tIzpFbTSNtWs+C2qun6SWrv1WiYzmjrjni3zem83A6CC2K8s5/0b\nUjUNB1ABwHEtnddQF/qPMDcB14ZUyXmzlnl8nql+XOu072qiJs1OZr9T+z62oIKYL9M6mlJ9ydZn\ngsBljHQspgL8zdtF4PMtLf9MBQObsZwRVFRT5JeYqt3dnurf81GqNvgieup/Rl1gL23nwSFUbWjv\nnYBXsO8H0f/gkSOoAPVunWUX0mnKnIPjGAUqvU4KTXWx+GrLRx5B9SHbZoDvZX0q0DqTm/cdfFvL\nc0bTLqy05qh9Ll+mCncfomqcLqRquV/czufRiOo+Whd2a/ngV4Hvz/VnNRfnUuc4XkNNsvrPVE3q\nrEYmr2p/856AVeWPqsY+j6oCfkJbtjEVTX+C6iw6UZNi2+bopLonU235d6barF9JzYOyQbsA9DKb\nLtXx8CyqtmtbqgPkXdrfhZNm/N0LJ1PBzGjeoT2oIdV/x9QM1X3NcbURFUS+jSrp7cjU0OHuKM/l\nTjZJVb1/q/2oX0sFXqe1Y5iL4c73Ymqy1a/TRj+1154DPLjHfT2cqt16SdvXplRT3RepTry/X9Hn\n0tP+n0KN1LsdFVS+lRbgUX3s3skKaluoUu2fmeo0vW77PbyaKpTcoS3vqx/ggVTh4yLmINCey7/p\nn0H7vm8C9pq2/GLgE/Od3hke2/pUK8Boqo05n/i083yddh6exVTg9XCqMHxUe/0Wz7+WN32NqUEb\nJ1D96V7Q8oBXMwej7dp+n9DO523n+3ucRfpHgeq/tTzklJZvvYge7wIyb8c33wlYFf6oTtT/RtWa\nHEj1fxndd20BVYPT5z0AH0I1lYxKPHehaiTeTnV87LWqtl3sP8lUbdNLqQ6clwOHTbjtW1Nt67dv\nF8ajqRqMjwDvbevcm6pNe2nLrCbqP8TNawp3owKmN1N9QHajmmz+mpXUGFHB7X2YGp10JFXD0ktQ\nOG1fW1NNFtu0TOVLTM2FdQR10d+hp31tT5Wst2KqP9XoGPehgss5mzywHd87maqpWp9qxn4dFVz+\nGysptbZzaSlTBaB9qabJzeYozXdgDjoBz+XftN/BVkx1jn8JFXDfddr6W893mmdxjOu2Y5to4tsZ\nfI7HtLxjdM/ZA6lC3aip8WErO3c729q0/fYe0DmWd1HNsE9j6k4nvfe1Gu1vvr+/WaR5cypQHd0X\nchSojgqqpzAH94Ed9BjnOwHz9MWuM+3544HXdJ7fh2pmuvcc7HtvqvS0I9U35WVUp967UDVeZ/dx\n8W3H8HqmOkq/ol30R+3iOzI1a/BENQZUx/VlVBA3yvhvTzUHjW4XsTv9dprfhmpiXNgy5VdQ86fd\nkQryPsyYTRFUIHgUc3TfzM5+zgHe0h6/gBpF+jkqCOltv1Tw+XIq+LmIqZFLD2XaiK9Jv/tp38no\n3HoN1XfvKZ3zYdv2/ZxNmxR0jO0+kurs/lEqiH/UXH03q/MfdXeHT7Tf99FUIfHF7SL1f26j5N8K\nP8djqRqWnajawme0PObhVA36jPuitu/mZaPfN9WM/fGWNz+LnvrArSl/VKD6daZG8Y8C1Y9So5X3\nms/09XKM852Aef6CD6VKLq+mmhE37FxA3kY/s6NvS7vrPVVr9m5qCPRoRuLdqOj9DKqmrZconuqX\ndi4V4P09VWJ7M60vVw/b75YOd6cCrsvp1Ai2H9D5wJk9f28Pp2o9XtW+t1tTAevL2+d7J2bQ2ZZq\n8ns6Pd/YuW37Hkz1cbsNNTHofahg8XbtO++l9oYKRO/Rtv3Ndp6N5mSbk6HV086D7ZgKvE6gald3\npTMJLTMs1VOTk34beMFof6xGnWbn+q/lX19uj78KvL09XocazfZFVsMaj4E+u+4Aqg2p2tnNqUE0\n53PzbhMPZhZNW1S3h1dShatXUSOJH0YFdKcyB/dvXd3/qJrGU7h5oPqZdi1b7c/l0Um3VoiIyHbA\nEXEEU31LDqAmD3wH1Xl+S2oCvAMy85oJ9rcVdSG8Ebg2M6+NiI2oqP2PmXlYW+/eVHPKOZn5vUmO\nLSL2ojKOn2fm11sankBNwHoP4O8y89TZHlN3X+3xFtQtZTahLpAnUtMeXBYRu1A1YOtk5o8n2Wdn\n33tRgdVhVEf0x1L9PY5qaTgO+FBmfne2xzRh+tbJzJva43u0NO5INSd+iApGfpiZ75t0X20fo+/9\nflTQ+d9ULdrWVD+6y6ig5fnUDaw/2cd+l5OOZ1HHehXwR6oUfyLVgf9VwLdHn8sstv0wqobsxMz8\neD8pXj1FxKgZ9LsRsS81wvmnVFP5o6nawD9FxHaZ+aOIuH1m/mI+07wqmpaH3SYzfx0Rb6EKQutT\nczP+ISJeCHwnMy+YYF8bUy0c9wS+lZlfiogFVIHofyY/mjVLRGxJ1ToupgYaPIbKW14KnJKZ35nH\n5E1srQm6pv3ItqZ+BEsy84cR8UiqNPJHqsnnoVSb/hUT7G9rqpr0QZm5NCL+hQq8ntQCr3dRwcjj\n2/obZ+b/TnCIRMTB1AXuIirw+kNmHtleuwtVq3YkdTH8yaRBRkQ8jyoB/pxqIjuHqjE6hgowHkD1\nyfn5BPtYj6od+UNE3JZqOvwpVbN1atvXS6gakGdQx/ynWR9UTyLiWCrIvYmaMuH5VC3XIVQt5ENn\nGhjewr4OoAKsj1K3xziv7es3VDPTNdS5fkFfweW0/Y/uh3co1RxwNHWMh1H97e5ATYsx6+8lIvan\ngtWrJk/x6isidqBqxZdRNbTvo/q7/D4z92vrPI8qRJ6QmTfMV1pXBxFxDDXA4/hWcHgj1cJxVUQ8\njspbHp2ZP+xxnwsy8899bW9NFBGbUAW20WjMW1EVJPtn5n/PZ9omNt9VbUP8cfMmkBOpAOEK6uIw\nmibiUOritDc9jFyjOq9/gro/2IuokZBfpTW1USPvPkkbUdRN4wz2cXum5phaSF1092/Pb011KH9D\nZ/112zoT386GGkL9xfb4S1Qt4SiIfyJ1MZh0VOS6VAfqg6lg8ZVUk+VC6iI/OtbXUYFG733wZpnu\nw6nat92pUXsvoGpO96KCkB/S0y1TqBqOD9Lm9qKG2b+Xqoqfk47h089VapqLN3a+sw1bmkYzlN9h\nvr+TNemPqYldj2vP300F3Y+jOmh/iznsm7im/LX8/9/p9Huj+oZ+vZ2//+rnOP9/1PyFFzGH09sM\n+bcOa4Fs31xEHEpVWT6ZmqBxF+C+EbEwMz9BXRz/OzNv7GG3lwG/pTLIP2XVYh0I7BoRZ2Tm76jg\n5OXdNI4rIm5FBXQLI2K9luaFVIkAakqAd1MXwZG7U1XcfZR+bwt8uJUO/0Q1/WRE3DUzPwgclRPU\nFAJkldJvpKaAeC1wcWb+iuqDsSlwv9b09ECqZvLfJ9lfj+4OvCczv031T/g11Y/r4sx8ITVVw6yb\nrUey/JG6Fcc9WzPJf1C3QToQeFprxujNtBrjDdviHwOPi4hHZeYNmfkH6pzYpqXzZ32mQZxJ/SaO\njYhHUPnW5dSAlt2oprHL5zF9q6SIiGmP70HdgeD7LT8lM0+h8uVTgMf4Oa4Svk9NYLxaNyuOrBVB\nF/yl79FbgBsy8wfUiJLfULUS+7XA65zsr/niz9TtCj4FbBURe7dAa39gn4h4Z2b+tl2YZywzf0/1\nc7keeEFrens/8IaI2COr/8wGwE4RcZv2nsupqQJ+NJN9RcQ6ncfrtYc/pGa+fnxmHpjVj+SvgJe2\nIHCiwK6zz3WoPnbfAG4dEXfIzOupEXK7UH2HXj9pgNezK4AHRMTOmXl9Zp5Fde6/R3v9l7Pd8OjC\nERE7RMR9WzX8Eiq4fkB7/XfU6MFnUIMOetMJuJ4LfCoiTqZq3B4PvDkiToqIZ1Md6L/Z575VMnNp\nZr6fmm/tNVSt+q+oGq6XZuZl85m+VVXn3N2iPV4IPLHlV79vrz0KuDEzr8rMn8xjctVk5k8y8z/n\nOx19WWv6dAFExKOpYet/nZkfjoiFVA3KTcDLRj+8CbY/6tS8I3Xh+zPVsfkl1EX3PZn5zdana7fM\n/Pos99PtrP0Qap6nH1JNSgdStWvvp5pM/yozz++jH0FEHEV1Cr8sM98XEadTN+n+PnXRPxF4cl+l\nw4jYm/p+TqD6cB0E/CAzT4+IDaimtKVZ/b1676s0Wy0Afj4VjPwL1dx2CnV3gUn6t43Or0dSI1K/\nRzUjn04FOXem5ujagprB+zDgp63mcSLTzrndqAv++6jAdyuq9mU0zB7qPnzWEsyxiDiQ+o3cSDUx\nf3+ek7TKagWSBwAvzMxHtEE5h1N9Bd8REY+n8rAnZOZ/zWdateZaq4Iu+Etn89dQ83KNAq9NM3NZ\nT9s/iMoE/5Fq/tubqYlDdwLekZkXTbD90YV3e+DHmfnHiLgX1YH9p8CbqE7MG1O1erOubVhOcPdW\nqu/Im6iawk9Tw5/3o5ozT+8x4NqF6ge1NNtoy6gRp3tT/eEOoEZqfauP/fUtIu5MG01GNTP/7Wyr\nxyNi3VHNYURsS/VhOzUzL4mIo6nA573Aj6gbxF5HBaSnU5/RDyY8lm6T4r5UzUpm5pvbAI2DqWbr\nsyc5tzU7EbEIoK88bE3WCmvvouZ8+hNThZPbUwWYo60p1Fxa64Iu+EtgdBbwvMz8pwm3dQdqtOMn\nqSkLPklN0bAXNXJs38z8Zbs4HQmcO9vApBNwHUCN5PgydYPk11I1HEdRN+v+YJ/VsRFxH6oP1TqZ\n+dmoqRA+Drw1M09v6/wlMOhpnw+mRmX9jGoyubotvz/VTHd1Zn6ur/3NldZXJFrT8mzev5Bquvsh\nFbw9iRrBeUa26R8i4u+pOYRGI1W3oUaSPrPP2qYW4L2AGiSwB7Br1lD7bana1jtR90r7/apS66i1\n17TCwtOoAsn11Ln7qsz8WmfdzahmRadw0JxaK4Mu6GcIequuPpKaNuHL1OjEA6g+O8+nqvuXtiaA\nL1Gf9x8nTPeeVK3ZZ9qig6man5dQs9o/Hfgv6pZCf5jNxW9aZnUM1ZR0NdVk+qysaTZ2oo751Mx8\nwyTH1N1na7r6I9VsuRE1YvFfgPMz89oVpXNNFhG7M/V970/dyuc2wKey5mt6ABWYnZRtEEhEbJKZ\nv+kxDQ+kzoPHZOavIuJsKtA/rD3fGvjfrIEO0ryalofdkTpX708NIno5lb/8M1XD9Y85R/PXSdOt\nNR3pp8vMCyftNJ/lg1TH6b2ppqQXUH2q7tcCrvtSU0Zs2UPAFdT8Sw/KzK+1ktpHqRqQv6ealz5I\nZS4bzTYg6WRWj6ICub2oWZovAo6Pmnjxe9TFv5fMqgVcB1G323gEcCmVQb6bmq/l8KhJ8/5POtcC\n/8FUTdeoNnU74MUR8QbqM7owM28cdbQHJp3zrTvSa0NqVNx21LQEZOYzqHsjfikibpuZ/2XApVVF\nJw87gWpO3B/4XWa+iWoZ+CJ115HLqGkjpEGstUFXX1pT36OoPi17Uzf23QA4qf3g3w68KWc5uV5n\ntNpeVPDzIuBeUSMFycxLqIvwr6kJUZdR0zmsP8G+FrS+D6+imk7/t/VHOp+q7XpxRGybmT+Y7XFN\n328LqF5CfZb/RfVP+2NmfoHqr7Q3a+n5mlOjXp9OzYe2W2YeR9UA/paqUf1kt3Q/SUC6nFqC9TLz\nrdRktPeKiMe2fRxN3Zz2NrM+OGmORE0R9DhqiqD9qIID1K14fpmZ38vM12QP07dI41prmxf70Ppz\nfZyabfuKFmRtRo1auwNVQ3FZZl44SVNYRBxCdVy/kOo3cw11m4S/73Qy3yQzf9OCpY1n2ql22oV2\no8z8XdRUE5+gbl3xvPbaPsCDgHfmBDMDR8S61CS0f2hpXo/qOH8VNVrxqZn5g3bsF1CT2K71/S2i\n5mV6M1WjuQ81z1AvM9svZ1/Ppyan3YwaGPJJKgjfA/haZn5gLvYr9SEinkrV+G5CdQN5RGZeHzW3\n30lUP8Tf5ixvTyXNxsL5TsBq7gbqM9ysPX8HdYuOu1Cd2c8erThBwHVbKnPYj6rteFlmPjUiPgBc\n2kYYvroFXOu0JswZN2N2Aq6jgIMi4kdUTcrBwOcj4vWZ+fzM/FpELJmkqbR1Dn8w8LvW6Xsf4IXU\nSMh7U6NJr2+1eycDV6ZD4QHIzE9FxC+oQROnzWHAdSh1q6IDI+L9VJP2myLiXGoKjD0j4pPURcuS\nm1ZFV1NzGf44Mx8A0FoI1gcel5m/nce0aS1l0DWB1oH4I8CDIuKXmXl5RHyMmrBzVnNwLccN1CSu\nf0fNpv/o0e6pOWb+0EnPRCW2iDiSKgH+FTXlwFOpjqYHAN+KiD9k5imT9k1rfY9uoKY+uCPw/Mz8\n39ZsdQHwloj4AdUs8HIDrpvLzG9ExMWjPlx9BD3RmR6k+R3w7og4haq1fWRbvhHVZL5RTnivUGmO\njbpe3BQRD6JuAv8kqhbdgEvzwubFCbW+SMcBewIXU3dEPz4zP9/jPk6gOrI/tzVV7kuV4A7OuoXF\nrC6805oUN6Fuf/HrzPxQ1ASue1DNmE+mgq+NJx18MG2f76EywjdQTZg/joiNgedRN9H+XmZ+cW0Z\npbgqaDVcv6cGY+xGNZUf3gK8E6kA/PBJA29pCBFxJ6qf6KOAXwCvS+fh0jwy6OpBCxT2pjrTX5KZ\nX+55+5sDz6E60n+HGt3315n56Qm22Q1+nk3NKL8B1ffhIZn58xZ4/SPwgjZacSKdaSHuQs3UfxN1\nTM+kpoT4QNREj+tm5o8n3Z9Wbtp5cATwRmoOuAOogRkfpWptt6VupvyEuWrSlOZK60M6up+rNG9s\nXuxBa2b5XPubi+3/d0S8hmpe3JSaV+biSWqAOhfaY6mL6WGZeV0LtN7R+j7s1fY3cQf2TsB1ADWE\n+0tUn4vTqD4WT46ao2t0v0CDrjk2LeDammqy3idrHrZLmGrSXkpNLPn4PoJvaWgGW1pVWNO1Fmvz\nL32Y6qOzBHgsdXPpo4B/pW4l9ILMvLSn/S1vYtdbUTP335W6wP9nmyZCc2hawHUi1bS8MdXU+4Gs\n20sdSt366dGZefH8pVaS1gwGXWu5qBnnnwVcS03y+iNqEsxTqRnte+m70+YAuxq4NjPv35btQQ0G\n2MjMrZwAAAF+SURBVAw4pTsFhf24htECq0dTM/8fTdU6nktNCXFja3L85qR9+SRJBl1rvTZH1i7U\nLZF+2UYwPpPqpP/7Cbc9alLci+qQvS01AOAlmfnGts5eVOD1D/YVGlZEbAF8g5rN/qh2LryEmlz3\nPOBL2W4rJEmanEGXgJoygJrx/CSqs3QvN0qeycSufexPMxMRjwZOpwZmfLjNofZaapDDyyYNvCVJ\nU+xIr5ENqAvt4/rqLD2TiV372J9mLjM/HhF/Al4TEbTA64XUBLUGXJLUI4MuAZCZv4+I9/bcj2rs\niV01fzLz0xFxE3BWRNyYmf9E3cNTktSjtfIGwlq+vjuuZ92o+TLq9j5/26Yi2Je6cfZ/ZObnWwd7\nzbPM/Aw1Xccl850WSVpT2adLc2ouJnaVJGl1ZNClOdcmXB1N7HrdpBO7SpK0OjLokiRJGoB9uiRJ\nkgZg0CVJkjQAgy5JkqQBGHRJkiQNwKBLkiRpAAZdkiRJAzDokiRJGoBBlyRJ0gD+P7WP9pCBC4vW\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2e3db47128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "util.plot_bar_graph([w for _,w in sorted_weights[-20:]],\n",
    "                    [f for f,_ in sorted_weights[-20:]],rotation=45)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In terms of accuracy, the discriminative training regime leads to substantial improvements: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.785"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_guess = label_encoder.inverse_transform(lr.predict(dev_X))\n",
    "accuracy(dev_filtered, lr_guess)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is remarkable, as the underlying model is the unigram model we had at 0.80 accuracy when trained by maximising the joint likelihood. The same model, when trained discriminatively, leads to 3 points improvements without changing the representation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Background Material\n",
    "* [Introduction to Information Retrieval](http://nlp.stanford.edu/IR-book/html/htmledition/naive-bayes-text-classification-1.html), Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze\n",
    "* [Tackling the Poor Assumptions of Naive Bayes Text Classifiers](https://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf), Rennie et al, ICML 2003 \n",
    "* [Simple Sentiment Classification](http://nlp.stanford.edu/pubs/sidaw12_simple_sentiment.pdf), Sida Wang and Christopher D. Manning\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}