{ "cells": [ { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%%capture\n", "%load_ext autoreload\n", "%autoreload 2\n", "%matplotlib inline\n", "# %cd .. \n", "import sys\n", "sys.path.append(\"..\")\n", "import statnlpbook.util as util\n", "import statnlpbook.sequence as seq\n", "import matplotlib\n", "import warnings\n", "warnings.filterwarnings('ignore')\n", "matplotlib.rcParams['figure.figsize'] = (10.0, 6.0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "$$\n", "\\newcommand{\\Xs}{\\mathcal{X}}\n", "\\newcommand{\\Ys}{\\mathcal{Y}}\n", "\\newcommand{\\y}{\\mathbf{y}}\n", "\\newcommand{\\balpha}{\\boldsymbol{\\alpha}}\n", "\\newcommand{\\bbeta}{\\boldsymbol{\\beta}}\n", "\\newcommand{\\aligns}{\\mathbf{a}}\n", "\\newcommand{\\align}{a}\n", "\\newcommand{\\source}{\\mathbf{s}}\n", "\\newcommand{\\target}{\\mathbf{t}}\n", "\\newcommand{\\ssource}{s}\n", "\\newcommand{\\starget}{t}\n", "\\newcommand{\\repr}{\\mathbf{f}}\n", "\\newcommand{\\repry}{\\mathbf{g}}\n", "\\newcommand{\\x}{\\mathbf{x}}\n", "\\newcommand{\\prob}{p}\n", "\\newcommand{\\vocab}{V}\n", "\\newcommand{\\params}{\\boldsymbol{\\theta}}\n", "\\newcommand{\\param}{\\theta}\n", "\\DeclareMathOperator{\\perplexity}{PP}\n", "\\DeclareMathOperator{\\argmax}{argmax}\n", "\\DeclareMathOperator{\\argmin}{argmin}\n", "\\newcommand{\\train}{\\mathcal{D}}\n", "\\newcommand{\\counts}[2]{\\#_{#1}(#2) }\n", "\\newcommand{\\length}[1]{\\text{length}(#1) }\n", "\\newcommand{\\indi}{\\mathbb{I}}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Sequence Labelling \n", "\n", "Many real-world applications can be cast as *sequence labelling* problems that involve assigning labels to each element in a sequence. For example, in *Part-Of-Speech tagging* each token in a sentence as assigned a part-of-speech such as verb or determiner that indicates the syntactic type of the token. In *Named Entity Tagging* we assign each token with the type of entity the token refers to, such as \"Person\" or \"Organisation\", or \"None\" if the token does not refer to an entity. \n", "\n", "## Sequence Labelling as Structured Prediction\n", "\n", "The problem of sequence labelling is an obvious (and somewhat canonical) instance of structured prediction. Here the input space \\\\(\\Xs\\\\) are sequences of words and the output space $\\Ys$ are sequences of output labels. Our goal is again to define a model a model \\\\(s_{\\params}(\\x,\\y)\\\\) that assigns high *scores* to the sequence of label \\\\(\\y=y_1 \\ldots y_n\\\\) that fits the input text \\\\(\\x=x_1 \\ldots x_n\\\\), and lower scores otherwise. The model will be parametrized by \\\\(\\params\\\\), and these parameters we will learn from some training set \\\\(\\train\\\\) of \\\\((\\x,\\y)\\\\) pairs. In contrast to the classification scenario the prediction problem $\\argmax_\\y s_{\\params}(\\x,\\y)$ is now non-trivial in general, as we have to search through an exponentional number of label sequences. In practice this issue is overcome by making assumptions about the factorization structure of $s_{\\params}(\\x,\\y)$ and/or search approximations that sacrifice the ability to find the true optimum of the search problem for more expressiveness. \n", "\n", "In this chapter we will restrict ourselves to scoring functions $s_{\\params}(\\x,\\y)$ of the form $\\prob_\\params(\\y|\\x)$ that model probability distributions over label sequences $\\y$ conditioned on input sequences $\\x$. This is the sequence equivalent to conditional models we discussed in the [text classification](doc_classify.ipynb) chapter. Other choices are possible but omitted here they are either very similar (structured SVMs, cite) or less effective in *supervised* sequence labelling (HMMs, cite). \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Part-of-Speech Tagging as Sequence Labelling\n", "Part-of-Speech (PoS) tagging is an important task within NLP. It is a standard pre-processing step in many tasks. For example, most dependency parsers assume as input PoS tagged sentences. Likewise, [Reverb](reverb), one of the most effective relation extraction methods, defines relations in terms of PoS sequences.\n", "\n", "Traditionally, and based on the existence of corresponding annotated training sets, PoS tagging has been applied to quite restricted domains such newswire or biomedical texts. Recently there has been increasing interest in NLP in general, and PoS tagging in particular, for social media data. He we will focus on PoS tagging for tweets and use the [Tweebank dataset](http://www.cs.cmu.edu/~ark/TweetNLP/#pos) and the [\"october 27\" splits](https://github.com/brendano/ark-tweet-nlp/tree/master/data/twpos-data-v0.3/oct27.splits).\n", "\n", "Let us load the data and look at an example tagged sentence." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"I/O predict/V I/O won't/V win/V a/D single/A game/N I/O bet/V on/P ./, Got/V Cliff/^ Lee/^ today/N ,/, so/P if/P he/O loses/V its/L on/P me/O RT/~ @e_one/@ :/~ Texas/^ (/, cont/~ )/, http://tl.gd/6meogh/U\"" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train = seq.load_tweebank(\"../data/oct27.splits/oct27.train\")\n", "dev = seq.load_tweebank(\"../data/oct27.splits/oct27.dev\")\n", "test = seq.load_tweebank(\"../data/oct27.splits/oct27.test\")\n", "\" \".join([w + \"/\" + t for w,t in zip(train[0][0],train[0][1])])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have printed the tokens of tweet paired with their PoS tag. The tags (such as \"O\", \"V\" and \"^\") are described in the [Tweebank annotation guideline](http://www.cs.cmu.edu/~ark/TweetNLP/annot_guidelines.pdf). For example, \"O\" denotes pronouns, \"V\" verbs and \"^\" proper nouns. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# count tags here?`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Local Models / Classifiers\n", "We will tackle sequence labelling as a (discriminative) structured prediction problem. This means we will build a model $p_\\params(\\y|\\x)$ that computes the conditional probability of output label sequence $\\y$ given input sequence $\\x$. We will first consider the simplest type of model: a *fully factorised* or *local* model. In this model the probability of $\\x$ and $\\y$ is a product of *local* probabilities for the label $y_i$ of each token:\n", "\n", "$$\n", "p_\\params(\\y|\\x) = \\prod_{i=1}^n p_\\params(y_i|\\x,i)\n", "$$\n", "\n", "In this model all labels are independent of each other. This assumption has a crucial benefit: inference (and hence training) in this model is trivial. To find the most likely assignment of labels under this model you can find the most likely tag for each token independently. Notice that each local term conditions on the complete input $\\x$, not just on $x_i$. This is important, as the sentential context at position $i$ is often important to determine the tag at $i$. \n", "\n", "It is common to indicate this independence structure in a factor graph, a graphical representation of the model. In this representation each variable of the model (our per-token tag labels and the input sequence $\\x$) is drawn using a circle, and *observed* variables are shaded. Each factor in the model (terms in the product) is drawn as a box that connects the variables that appear in the corresponding term. For example, the term $p_\\params(y_3|\\x,3)$ would connect the variables $y_3$ and $\\x$. " ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "%3\n", "\n", "\n", "x\n", "\n", "x\n", "\n", "\n", "p(y1 | x, 1)\n", "\n", "p(y1 | x, 1)\n", "\n", "\n", "p(y1 | x, 1)--x\n", "\n", "\n", "\n", "y1\n", "\n", "y1\n", "\n", "\n", "y1--p(y1 | x, 1)\n", "\n", "\n", "\n", "p(y2 | x, 2)\n", "\n", "p(y2 | x, 2)\n", "\n", "\n", "p(y2 | x, 2)--x\n", "\n", "\n", "\n", "y2\n", "\n", "y2\n", "\n", "\n", "y2--p(y2 | x, 2)\n", "\n", "\n", "\n", "p(y3 | x, 3)\n", "\n", "p(y3 | x, 3)\n", "\n", "\n", "p(y3 | x, 3)--x\n", "\n", "\n", "\n", "y3\n", "\n", "y3\n", "\n", "\n", "y3--p(y3 | x, 3)\n", "\n", "\n", "\n", "p(y4 | x, 4)\n", "\n", "p(y4 | x, 4)\n", "\n", "\n", "p(y4 | x, 4)--x\n", "\n", "\n", "\n", "y4\n", "\n", "y4\n", "\n", "\n", "y4--p(y4 | x, 4)\n", "\n", "\n", "\n", "p(y5 | x, 5)\n", "\n", "p(y5 | x, 5)\n", "\n", "\n", "p(y5 | x, 5)--x\n", "\n", "\n", "\n", "y5\n", "\n", "y5\n", "\n", "\n", "y5--p(y5 | x, 5)\n", "\n", "\n", "\n", "p(y6 | x, 6)\n", "\n", "p(y6 | x, 6)\n", "\n", "\n", "p(y6 | x, 6)--x\n", "\n", "\n", "\n", "y6\n", "\n", "y6\n", "\n", "\n", "y6--p(y6 | x, 6)\n", "\n", "\n", "\n", "p(y7 | x, 7)\n", "\n", "p(y7 | x, 7)\n", "\n", "\n", "p(y7 | x, 7)--x\n", "\n", "\n", "\n", "y7\n", "\n", "y7\n", "\n", "\n", "y7--p(y7 | x, 7)\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seq.draw_local_fg(7)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The independence assumptions of this model allow us to think of it not as a single sequence model, but a sequence of classification models. We have a classifier $p_\\params(y|\\x,i)$ that predicts the best class/tag for a token based on its sentence $\\x$ and position $i$.\n", "\n", "$$\n", " p_\\params(y|\\x,i) = \\frac{1}{Z_\\x} \\exp \\langle \\repr(\\x,i),\\params_y \\rangle\n", "$$\n", "\n", "Let us define a simple version of this model on the PoS tagging task defined above. Here we use a feature function (template) that has one active feature corresponding to the word $x_i$ observed at index $i$. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "def feat_1(x,i):\n", " return {\n", " 'word':x[i]\n", " }\n", "local_1 = seq.LocalSequenceLabeler(feat_1, train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that in the above feature functions return a dictionary mapping a feature name (`word`) to a feature value (`x[i]`, the word at index `i`). This is not quite what we require `repr(\\x,i)` to be, namely a function from $\\x$ and $i$ to a real feature vector. However, internally this dictionary-based representation can be transformed into the desired form by defining a feature function template like so:\n", "\n", "$$\n", "f_{\\text{word},w}(\\x,i) = \\indi\\left[x_i = w\\right]\n", "$$\n", "\n", "This means that we have one feature function $f_{\\text{word},w}$ per word $w$. For example, the feature function $f_{\\text{word},\\text{\"the\"}}$ returns $1$ if $x_i = \\text{\"the\"}$ and $0$ otherwise. \n", "\n", "We can assess the accuracy of this model on the development set." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.6392286958324694" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seq.accuracy(dev, local_1.predict(dev))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is a good start, but a macro-level evaluation score does not really show us where the approach fails and can be improved. One alternative view on the system is a confusion matrix that shows, for each pair of labels $l_1$ and $l_2$, how often $l_1$ was classified as $l_2$. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAGoCAYAAABWs9xCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8JGV94P/P98yF24DOcIZZLoOgAoo3hCNe96dAFFAj\nGBccvOHPy6yK64qSCCYbMZHIriZm4wUzv3hBjSLxyv6SqARlo9GIA0tAQARFBIIwg5iIkYGZ890/\nqo60J3PO6eqqmtNd/XnPq17TXV311Le76/S3nqeeeioyE0mSNHomFjsASZI0GJO4JEkjyiQuSdKI\nMolLkjSiTOKSJI0ok7gkSSPKJC5J0ogyiUuSNKJM4pIkjailbW9g5arJ3Hft/rXK2GmpxxoaXnXH\nPIxGopCG1xVXXL45M1cvdhwAS/Z4SObWXzZSVv5y05cz87hGChtQ60l837X7c+Hffr1WGQ/da7eG\nopGaV3fo4gjTuLptl2Vx82LHMCO3/pKdDjm5kbLuvfL9k40UVEPrSVySpOEREN1p3TWJS5LGRwAd\nav3qzuGIJEljplZNPCLOBu7JzHc3E44kSS2zOV2SpBFlc7okSVpsrdTEI2I9sB5g733XtrEJSZIG\n0K3e6a28k8zckJlTmTm1as9Fv4xOkqQHRDQz9bWp+FFEXB0RV0bExnLeqoi4OCJuKP9f2bP8WRFx\nY0RcHxHHLlR+rSSemWfbqU2SpHkdlZmHZeZU+fxM4JLMPAi4pHxORBwKrAMeBRwHfCAilsxXcHfa\nFCRJWkhQNKc3MQ3uBOD88vH5wIk98y/IzC2ZeRNwI3DkfAXViiIiXhMRL6tThiRJO05DTen993BP\n4O8i4vKyvxjAmsy8vXz8E2BN+Xhf4JaedW8t582pVse2zPxgnfUlSRphkzPnuUsbMnPDrGWelpm3\nRcRewMUR8b3eFzMzI2LgGzB4nbgkabw01zt9c8957u3KzNvK/++MiM9TNI/fERF7Z+btEbE3cGe5\n+G1A7yVd+5Xz5uQ5cUnSeNlBzekRsVtE7D7zGHgW8F3gIuDUcrFTgS+Wjy8C1kXEThFxIHAQcNl8\n27AmLklSO9YAny9vN7wU+GRmfikivgNcGBGvBG4GTgbIzGsi4kLgWmArcFpmbptvA60n8SUTwYN3\nXVarjOnpevdrBpiY6M4wexou3g9cGiU7brCXzPwh8LjtzL8LOGaOdc4Bzul3G9bEJUnjw1uRSpKk\nYWBNXJI0Xjo0drpJXJI0Rrp1A5SBknhEvBP4CvAg4JGZ+c5Go5IkSQsa9HDkicA/Ak8H/r65cCRJ\natlENDMNgUo18Yh4F3AscCDwLeBhwDER8ZnM/IMW4pMkSXOolMQz87fLC9FfBrwJuDQznzp7uXKQ\n9/UA++63fxNxSpJU38xdzDpikHdyOPBPwCOA67a3QGZuyMypzJzac3KyTnySJDVrx97FrFV918Qj\n4jDgoxQDsm8Gdi1mx5XAkzPzl61EKEmStqvvmnhmXpmZhwHfBw4Fvgocm5mHmcAlSaOhvMSsiWkI\nVO3Ythq4OzOnI+IRmXltS3FJktSOIWkKb0LVjm2bgOeUj5/USkSSJKkvjtgmSRovQ9IU3gSTuCRp\nfAxRz/ImdOdwRJKkMbNDauJZc/2JBoa3+8W9W2utv9vONlpoeE1P1/0ra+bvTBoJNqdLkjSibE6X\nJEmLzZq4JGmMeD9xSZJGl83pEBEHRMTLG4xFkiRVMFBNPCJeC7wBWFEm8nWZ+ZMmA5MkqXEduxVp\n5SQeEbsDbweOAx4LXAr8otmwJElqQ7fOiQ/yTqYpLv1eBZCZP8rMn/cuEBHrI2JjRGy8a/PmBsKU\nJEmzVU7imfkL4NXAO4E/jIh3R8Sus5bZkJlTmTm15+RkQ6FKktSAmaFX605DYKA2hcy8CDgJ+B/A\nauDNTQYlSVJrxvV+4gARsQLYs3z6c+A6yqZ1SZK04wzSO30Z8OcUiXwS+DHwoiaDkiSpNUPSFN6E\nykk8M+8GjouIA4BnZOZHG45JkqR2hL3TZ/wMuLKpQCRJUjUDD7uamSZxSdLoGefm9MobmAj2XLG8\n7c0sqO79wH94Z/3xbB661261y5C2x3uBS/2LDiXx7pwYkCRpzHgXM0nS2Ai6VRM3iUuSxkeUU0fY\nnC5J0oiqVBOPiEngrygGerkXODoz72kjMEmSmhdj3Zz+WuDvM/NtEbEPcF8LMUmS1JpxTuL3AQcA\nZOY/Nx6NJEnqW9Vz4j8AfisiXtNGMJIktS0iGpmGQd9JPCL2Bc4CHg68KiJeUM6/KiIeNGvZ9RGx\nMSI2btq8qdGAJUmqo0tJvEpz+lOBqzPzroh4DnBJRKwBfpSZ/9K7YGZuADYAHHHEVDYWrSRJ+pUq\nzelXAUdFxD6ZeQdwOvB+4JOtRCZJUtOiwWkI9F0Tz8zvRcTvAl+OiPuBO4B1wLkRcUVmfr+tICVJ\nakKM8yVmmfkJ4BOzZn+6uXAkSVK/HHZVkjRWxrYmLknSqOtSEnfsdEmSRlTrNfEEtk3Xu8psycTi\nHzU9dK/dapex8sT31y7j7i+cVrsMSerXPfduXewQGtelmrjN6ZKk8TFEl4c1weZ0SZJGlDVxSdJY\nsTldkqQR1LXBXmxOlyRpRFkTlySNlS7VxE3ikqTx0p0c3k5zeu/9xDd7P3FJklrRShLPzA2ZOZWZ\nU5OTq9vYhCRJ1UXRnN7ENAxsTpckjZVhScBNqF0Tj4hLImLfJoKRJEn9q1UTj4gJ4OHAT5sJR5Kk\ndnWpJl63Of1Q4LOZ+csmgpEkqU1dG+ylVhLPzO8Cb2ooFkmSVIEd2yRJ46U7FXGTuCRpjITnxCu5\n/ef38s6v3lCrjN/7jYMbimZx3f2F02qXMT2dtcuYmOjODiypXSt2tq5XV0QsATYCt2XmcyNiFfBp\n4ADgR8DJmXl3uexZwCuBbcAbMvPL85XtDVAkSWNlEQZ7+a/AdT3PzwQuycyDgEvK50TEocA64FHA\nccAHygOAOZnEJUljZUcm8YjYD3gO8Bc9s08Azi8fnw+c2DP/gszckpk3ATcCR85XvklckqTBTM7c\nJ6Sc1m9nmT8FfgeY7pm3JjNvLx//BFhTPt4XuKVnuVvLeXPyZIckabw01y1oc2ZOzbmZiOcCd2bm\n5RHxjO0tk5kZEQN3duo7iUfEGuCtwFHAVuAK4O2Zecu8K0qSNER2YO/0pwLPi4hnAzsDe0TEJ4A7\nImLvzLw9IvYG7iyXvw1Y27P+fuW8OfXVnB4RDwO+BPwDMJWZhwOfAj5fviZJknpk5lmZuV9mHkDR\nYe2rmfkS4CLg1HKxU4Evlo8vAtZFxE4RcSBwEHDZfNvotyZ+HnBqZl7VE9wlEfES4I954KS8JElD\na0huI3oucGFEvBK4GTgZIDOviYgLgWspWrxPy8xt8xW0YBKPiIOBTZl5Vdm+/wfAD4HIzBdExHRE\nTGbm5p511gPrAfbYa5+B3qEkSW1YjCSemZcCl5aP7wKOmWO5c4Bz+i23n5r444B/LK9VextwNPAg\n4Lvl6zcABwK/SuKZuQHYALD3wY+uPzqJJEn6d/ptTt8GTAI/yMyfAT+LiGvL1/bigZPykiQNtSFo\nTm9MPx3bvgs8kaKm/bCIeFBE7A88MiIeA+yVmTe3GaQkSY2JhqYhsGBNPDOvK5P2IcA7gK9RnBO/\nCDgDeEWrEUqSpO3qtzn9dcBfAm8BjijnHQ7sk5l3tBGYJEltGLfmdDLzOuB5wAsoBnn5J+C1wFXz\nrSdJ0lCJRbkBSmv6HrEtM28FXtNiLJIkqQLHTpckjY0AhqQS3YjWk/iqXZZzymMc8KUpExP1974r\nbrq71vqHH7iydgxdMj1dbyiEJr7TJtR9HzA876WuzPqfxbA0t2q24WkKb4K3IpUkaUTZnC5JGisd\nqoibxCVJ48XmdEmStOgqJ/GIODEiMiIe0UZAkiS1Jorm9CamYTBITfwU4Bvl/5IkjYyguIqiiWkY\nVEriEbECeBrwSmBdKxFJkqS+VK2JnwB8KTO/D9wVEUdsb6GIWB8RGyNi4913bd7eIpIkLYpxbk4/\nBbigfHwBczSpZ+aGzJzKzKmVe07WiU+SpEaN5djpEbEKOBp4TEQksATIiPjtbGJ4I0mSVEmVmvh/\nAj6emQ/JzAMycy1wE/Af2wlNkqSGjXHv9FOAz8+a91nspS5JGhHFDVDGsDk9M4/azrw/azYcSZLU\nL4ddlSSNkeGpRTfBJC5JGisdyuHtJ/Gdlk7wkMld296MKqh7P/CXfeKK2jF87CWH1y5jWAzDyE3e\nC7xZXaqpqdusiUuSxkqXDtJM4pKk8TFEl4c1wVuRSpI0oqyJS5LGxsx14l1RZdjVbcDVwDJgK/Ax\n4D2ZOd1SbJIkNa5DObxSTfyXmXkYQETsBXwS2AN4WxuBSZKk+Q10Tjwz7wTWA6+PLrVLSJI6byyH\nXZ0tM38YEUuAvYA7mgtJkqT2DEn+bUQrvdMjYn1EbIyIjZs3bWpjE5Ikjb2Bk3hEPBTYBtw5+7XM\n3JCZU5k5Nbl6dZ34JElqTticTkSsBj4IvC8z64/3KEnSDlBcYrbYUTSnShLfJSKu5IFLzD4O/Ekr\nUUmSpAVVuZ/4kjYDkSSpfcPTFN4ER2yTJI2VDuVwx06XJGlUWROXJI0Vm9M1kCY68g/Dzvexlxxe\nu4xf3Lu1dhm77ezuO2NiYvH3C3WTFyANN38FJUnjo2P3EzeJS5LGRtduRWrHNkmSRpQ1cUnSWOlS\nTbxyEo+IbcDVPbMuyMxzmwtJkqT2dCiHD1QT/2VmHtZ4JJIkqRKb0yVJY6VLzemDdGzbJSKu7Jle\nOHsB7ycuSRpK5SVmTUzDoJXm9MzcAGwAOPyIKUcKkCSpBTanS5LGRngXM0mSRleHcvhASXyXiLiy\n5/mXMvPMpgKSJEn9qZzEM3NJG4FIkrQjTHSoKm5zuiRprHQohzt2uiRJo8qauCRpbBTXeHenKt56\nEp/OZMv907XKWLqkGw0G26brXzK/dEk3dr7ddq6/6923td5+BbB8aTf2LUn9m+jGzyhgc7okSSPL\n5nRJ0lixOV2SpBHVoRxuc7okSaOqUhKPiDUR8cmI+GFEXB4R34qI57cVnCRJTQrK8dMb+DcM+m5O\nj+IkwheA8zPzReW8hwDPayk2SZIa16Xe6VXOiR8N3JeZH5yZkZk3A+9tPCpJkrSgKs3pjwKu6GfB\niFgfERsjYuNdmzcPFpkkSU2L4lakTUwLbyp2jojLIuKfIuKaiHh7OX9VRFwcETeU/6/sWeesiLgx\nIq6PiGMX2sbAHdsi4v1lYN+Z/VpmbsjMqcyc2nNyctBNSJLUuGLUtvpTH7YAR2fm44DDgOMi4knA\nmcAlmXkQcEn5nIg4FFhHUWk+DvhARMx707EqSfwa4PCZJ5l5GnAMsLpCGZIkjYUs3FM+XVZOCZwA\nnF/OPx84sXx8AnBBZm7JzJuAG4Ej59tGlST+VWDniHhtz7xdK6wvSdKiCopbkTYxAZMzp47Laf2/\n217Ekoi4ErgTuDgzvw2syczby0V+AqwpH+8L3NKz+q3lvDn13bEtMzMiTgTeExG/A2wCfgG8pd8y\nJElabA0O9rI5M6fmWyAztwGHRcSDgc9HxKNnvZ4RMfCNNSqN2FYeOawbdGOSJI2jzPxZRHyN4lz3\nHRGxd2beHhF7U9TSAW4D1vastl85b06O2CZJGis7sHf66rIGTkTsAjwT+B5wEXBqudipwBfLxxcB\n6yJip4g4EDgIuGy+bTh2uiRpbFToWd6EvYHzyx7mE8CFmfn/R8S3gAsj4pXAzcDJAJl5TURcCFwL\nbAVOK5vj52QSlySpBZl5FfD47cy/i+Lqru2tcw5wTr/baD2JT0Sw0zJb7QGWLvFzaNLypfU/z80/\n31K7jMndd6pdhjSsunTbzhkTHXpP1sQlSWOlOyncjm2SJI0sa+KSpLHSpVMEJnFJ0tgoRmxb7Cia\nM1BzekTcs/BSkiSpTdbEJUnjo8+BWkaFSVySNFY6lMPb6Z0eEetn7uqyedOmNjYhSdLYayWJZ+aG\nzJzKzKnJ1d5uXJI0PHbU2Ok7gs3pkqSxYe90SZI0FAZN4rtGxK0905sajUqSpJaMfXN6ZlqDlySN\npOFIv80wGUuSNKLs2CZJGhsR3opUkqSR1aEc3n4S//4dP+c33vP1WmVcesbTG4pG+nWTu+9Uu4xX\nfOrKWut/+JTDaseQmbXLaMKwdPapa8v922qXsdOyJQ1EUk8T+0VXvtOusiYuSRorXTowMYlLksZK\nh3K4vdMlSRpV1sQlSWMjiE71Tq9UE4+IjIg/7nl+RkSc3XhUkiS1IYrm9CamYVC1OX0L8FsRMdlG\nMJIkqX9Vk/hWYANweguxSJLUui6NnT5Ix7b3Ay+OiAfNtUBErI+IjRGx8f5f/Mvg0UmS1LCJhqZh\nUDmOzPxX4GPAG+ZZZkNmTmXm1LLd5sz1kiSphkF7p/8pcAXwkQZjkSSpVUG3BnsZqEUgM38KXAi8\nstlwJElq10Q0Mw2DOs36fwzYS12SpEVSqTk9M1f0PL4D2LXxiCRJatGw1KKb4IhtkqSxUQzU0p0s\nPiy95CVJUkWt18QPXrM7l7zp/2l7M9qBtk3Xv0fxkg61Z9W9H/hJH/5O7Rg+deoRtctYusRj+hnD\ncC/wJnSpxtmkDv382JwuSRovXTq28dBbkqQRZU1ckjQ2Ajp1K1KTuCRprHSpCbrq/cT3i4gvRsQN\nEfGDiPifEbG8reAkSdLc+k7iUXRz/Bzwhcw8CDgYWAGc01JskiQ1rrhWvP40DKrUxI8G7s3MjwBk\n5jaK+4q/IiIcuU2SNPQigomGpmFQJYk/Cri8d0Z5W9IfAw9vMihJkrSwVjq2RcR6YD3A2v33b2MT\nkiQNZEgq0Y2oUhO/Fvi1YaEiYg9gf+DG3vmZuSEzpzJzanJydf0oJUlqyLjeivQSYNeIeBlARCyh\nuB3pRzPz39oITpIkza3vJJ6ZCTwfOCkibgC+D9wLvLWl2CRJatTMYC9d6dhW9X7itwC/2VIskiS1\nbkjybyO6NHCNJEljxWFXJUnjY4g6pTXBJC5JGitBd7L4DkniRZ+4OrrzgXfBki4dxg6Bv3rFE2qX\nsXb9p2uX8eM/P7l2GVHzZOP0dN3fCphw/9QYsSYuSRobRe/0xY6iOSZxSdJY6VISt3e6JEkjypq4\nJGms1O27MUwqJ/GI2AZcXa57HXCqw65KkkZB186JD9Kc/svMPCwzHw3cB7ym4ZgkSVIf6janfx14\nbBOBSJLUuujWsKsDJ/GIWAocD3xpO689cD/xtd5PXJI0PIbl5iVNGKQ5fZeIuBLYCPwY+NDsBX7t\nfuKrvZ+4JEltGKQm/svMPKzxSCRJalnXOrZ5iZkkaax0qDXdwV4kSRpVlWvimbmijUAkSWpfMNGh\nm2rZnC5JGhuBzemSJGkBEbE2Ir4WEddGxDUR8V/L+asi4uKIuKH8f2XPOmdFxI0RcX1EHLvQNkzi\nkqTxEUXv9CamPmwF3pyZhwJPAk6LiEOBM4FLMvMg4JLyOeVr64BHAccBH4iIJfNtoPXm9OlMttw/\nXauMpUs81mjSlvu31Vp/p2Xz7lNjZ3o6a60/0cD1Ljedd1LtMv7xBz+tXcaTH75nrfW71Myp4bWj\nBnvJzNuB28vHP4+I64B9gROAZ5SLnQ9cCrylnH9BZm4BboqIG4EjgW/NtQ2zoyRJg5mMiI090/q5\nFoyIA4DHA98G1pQJHuAnwJry8b7ALT2r3VrOm5Md2yRJY6Phjm2bM3NqwW1GrAA+C7wxM/+191ao\nmZkRMXBznklckjRWduTY6RGxjCKB/2Vmfq6cfUdE7J2Zt0fE3sCd5fzbgLU9q+9XzpuTzemSJLUg\niir3h4DrMvNPel66CDi1fHwq8MWe+esiYqeIOBA4CLhsvm1UqolHxDbg6nK9m4CXZubPqpQhSdJi\n2oEV8acCLwWuLm8cBvBW4Fzgwoh4JXAzcDJAZl4TERcC11L0bD8tM+ftiVy1Of1XNz+JiPOB04Bz\nKpYhSdKiCHZcE3RmfqPc5PYcM8c651Ahr9Z5L99igV5zkiSpPQN1bCsvPj+G7dxLvHx9PbAeYL+1\n+w8cnCRJjQqIDg1IULUmvkvZrj9zXdvF21soMzdk5lRmTu05OVk3RkmStB1Vk/jMOfGHULTzn9Z8\nSJIktScamobBQOfEM/PfgDcAb44IrzWXJI2EoLhOvIlpGAzcsS0z/w9wFXBKc+FIkqR+VapFZ+aK\nWc9/s9lwJElq13DUoZthU7gkaawMSUt4Ixx2VZKkEWVNXJI0RqJT14m3nsQnIthl+ZK2N6MKdlrm\n99GkiYnF/0FYuqR+o9qTH75nA5HU06UfVw2nHTns6o7QpfciSdJYsTldkjRWutTiYxKXJI2V7qRw\nm9MlSRpZAyXxiPjdiLgmIq6KiCsj4olNByZJUuPKu5g1MQ2Dys3pEfFk4LnA4Zm5JSImgeWNRyZJ\nUsO61jt9kHPiewObM3MLQGZubjYkSZLUj0EOSL4CrI2I70fEByLi6bMXiIj1EbExIjZu3rypfpSS\nJDWkS83plZN4Zt4DHAGsBzYBn46Il89aZkNmTmXm1OTk6kYClSSpCV26n/hAl5hl5jbgUuDSiLga\nOBX4aHNhSZKkhQzSse0QYDozbyhnHQbc3GhUkiS1ZEhawhsxSE18BfDeiHgwsBW4kaJpXZKkoVb0\nTu9OFq+cxDPzcuApLcQiSZIqcNhVSdJYGffmdEmSRlQQ49ycXlUwHPdblrpsejprlzEMf6e/uHdr\n7TJ229m6icaHe7skaazYnC5J0gjqWu/0Lo0DL0nSWLEmLkkaHzHGzekRsSdwSfn0PwDbKMZPBzgy\nM+9rMDZJkho3tkk8M++iGGaViDgbuCcz391CXJIkaQE2p0uSxorXiUuSNIICGIIhERrTSu/0iFgf\nERsjYuOmzZsWXkGSJFXWShLPzA2ZOZWZU6snV7exCUmSBhIN/RsGNqdLksZKl3qnO9iLJEkjauCa\neGae3WAckiTtEMPSFN4Em9MlSWPD3umSJGkoWBOXJI2R4elZ3oTWk/iWrdP88M5f1CrjoXvt1lA0\nUjdNdKR9cLedrVeoZR27AYrN6ZIkjSgPeyVJY6VDFXGTuCRpfBS907uTxm1OlyRpRFVK4hFxQER8\nd9a8syPijGbDkiSpHdHQNAxsTpckjZdhycANsDldkqQR1UpNPCLWA+sB9t53bRubkCRpIF0a7KVq\nTTz7md97P/FVe04OFpkkSS2IaGYaBlWT+F3AylnzVgGbmwlHkiT1q1ISz8x7gNsj4miAiFgFHAd8\no4XYJElq3Lj3Tn8Z8P6I+JPy+dsz8wcNxiRJUnuGJQM3oHISz8xrgaNaiEWSJFXgdeKSpLFRNIV3\npypuEpckjY8h6lneBAd7kSRpRLVeE//5fVv5+5s31SrjoXvt1lA0Apienuty//5MTHToMLYB92+d\nrrX+sqX1j6Xrfqfg99pr5RNeX7uMu7/zvgYiWXyZ9fetYdOlPd3mdEnSeOlQFrc5XZKkEWVNXJI0\nRsLe6ZIkjaqx7J0eEV+LiGNnzXtjRJzXfFiSJGkhVc6JfwpYN2veunK+JElDr6lx0/upzEfEhyPi\nzoj4bs+8VRFxcUTcUP6/sue1syLixoi4fnaleS5VkvhngOdExPJyYwcA+wBfr1CGJEmLa8fdAeWj\nFDcJ63UmcElmHgRcUj4nIg6lqBg/qlznAxGxZKEN9J3EM/OnwGXA8eWsdcCFuZ2LCCNifURsjIiN\n99x9V7+bkCSpMzLz74Gfzpp9AnB++fh84MSe+Rdk5pbMvAm4EThyoW1UvcSst0l9zqb0zNyQmVOZ\nObVi5Z4VNyFJUnuioX/A5EyFtZzW97H5NZl5e/n4J8Ca8vG+wC09y91azptX1d7pXwTeExGHA7tm\n5uUV15ckaVE12Dt9c2ZODbpyZmZE1BoSr1JNPDPvAb4GfBg7tEmSVNUdEbE3QPn/neX824C1Pcvt\nV86b1yAjtn0KeBwmcUnSCNpx/dq26yLg1PLxqRQt3DPz10XEThFxIHAQRT+0eVUe7CUzv0CnRp6V\nJI2Nmhm40qYiPgU8g+Lc+a3A24BzgQsj4pXAzcDJAJl5TURcCFwLbAVOy8xtC23DEdskSWpBZp4y\nx0vHzLH8OcA5VbZhEpckjRXHTpckaQQF3Ro7vfUkPrnrcl7+hAPa3owqmJjo0B48BJYtXfw7+jbx\nnW5n3KbKoiO/jnd/532LHcLQ6Mp32lXWxCVJY6VLhyUmcUnSeOlQFl/8dkBJkjQQa+KSpLHSpd7p\nlWriEbE2Im6KiFXl85Xl8wPaCE6SpKZFNDMNg6pjp98CnEcx4gzl/xsy80cNxyVJkhYwSHP6e4DL\nI+KNwNOA1zcbkiRJ7RmSSnQjBhk7/f6I+G3gS8CzMvP+2cuU91RdD7B2//1rBylJUmM6lMUH7Z1+\nPHA78OjtvZiZGzJzKjOnVk+uHjg4SZI0t8pJPCIOA54JPAk4fea+qJIkDbviJmbN/BsGVXunB0XH\ntjdm5o+BdwHvbiMwSZIa11DP9JHsnQ68GvhxZl5cPv8A8MiIeHqzYUmSpIVU6tiWmRuADT3PtwGH\nNx2UJEltGZJKdCMcsU2SNF46lMUdO12SpBFlTVySNEaGp2d5E1pP4gls3TZdq4ylS2wwkNoWw9Ld\nVmpZl3Z1s6MkSSPK5nRJ0tgIOtWvzSQuSRozHcriNqdLkjSiqg67+vyIuHLWNB0Rx7cVoCRJTerS\n2OlVR2z7PPD5meflLUdfDHy54bgkSWpFl3qnD3xOPCIOBn4feEpm1ruGTJIkVTbQOfGIWAZ8Enhz\neTez2a+vj4iNEbFx86ZNdWOUJKkx0dA0DAbt2PaHwDWZ+entvZiZGzJzKjOnJlevHjw6SZKa1LFb\nkVZuTo+IZwAvwLuXSZK0qCol8YhYCXwEeFFm/rydkCRJatOQVKMbULUm/hpgL+C8WeMsv3OupnVJ\nkoZFMDxN4U2oeonZO4F3thSLJEmqwGFXJUljpUMVcZO4JGm8jG1z+iBu/9d7ecff3VCrjLOPPaSh\naASQmbV2f6XFAAAOsUlEQVTW977Tv27rtnpjHS1d4i0M1I677rmvdhl7rljeQCRqizVxSdJYGZZx\nz5tgFUCSpBFlTVySNF66UxE3iUuSxkuHcnjl+4kvjYi/jojNEfHotoKSJEkLq3pO/Dzge8CJwKcj\nYr/mQ5IkqR1N3fxkWC7S6bs5PSLeBvxLZp5RPn8V8KmIeG5m/ktbAUqS1KQu9U7vO4ln5ttnPf8W\n8B+3t2xErAfWA+yxep868UmSpDm0colZ7/3Ed33QyjY2IUnSYKKhaQjYO12SNFaGJP82wsFeJEka\nUdbEJUljZVh6ljfBJC5JGiPRqd7pNqdLkjSirIlLksZG0K3mdGvikiSNqNZr4vvssTNnH3tI25sZ\nCffcu7V2GSt2rv+VRZcOQ4fA0iUeC2s47bliee0yLvvhTxuIRG2xOV2SNFa6VI8xiUuSxoq90yVJ\n0qKzJi5JGh9DdBvRJpjEJUljY4juXdIIm9MlSRpR1sQlSeOlQ1XxvpN4RJwOrAPuAz4CfB04AfiH\nzPzWrGXXA+sB1u6/f2PBSpJU17j2Tl8DPBV4FXAU8L+APYBvz14wMzdk5lRmTq2eXN1IoJIk6df1\nXRPPzDPLh9cDL20nHEmS2mXvdEmSRlSHcri90yVJGlUmcUnSeImGpn42FXFcRFwfETdGxJkLr1GN\nzemSpLGyo3qnR8QS4P3AM4Fbge9ExEWZeW1T27AmLklSO44EbszMH2bmfcAFFJdmN8aauCRpbAQ7\ntHf6vsAtPc9vBZ7Y5AZaT+JXXHH55l2Wxc0LLDYJbK6xmbrrd6mMYYihS2UMQwzDUsYwxDAsZQxD\nDKNUxkNqlt+YK664/Mu7LIvJhorbOSI29jzfkJkbGiq7L60n8cxccLSXiNiYmVODbqPu+l0qYxhi\n6FIZwxDDsJQxDDEMSxnDEEPXythRMvO4Hbi524C1Pc/3K+c1xnPikiS14zvAQRFxYEQspxi6/KIm\nN+A5cUmSWpCZWyPi9cCXgSXAhzPzmia3MSxJvO45hCbOQXSljGGIoUtlDEMMw1LGMMQwLGUMQwxd\nK6OTMvNvgL9pq/zIzLbKliRJLfKcuKSREBHe11iaxSQudVhEN+7XFBHPBi6JiH1rllPrFGJEY5cm\n1RIR/nYLGPEkHhGrFjsGDZ9h+YGLiP0jYrcGyqmTiJcs4rZ7y9m1xrrHAu8GXpqZtw36/UbEwcDv\nRcSeA67/EOBdEbHfIOuXZRwVEU8ZdP2yjN8AXlanDHXHovzYRcRDI2L3mmXsBbwuIpZHxAE1yxqK\nH/3FEhFrZj2v/XnU+KE9IiKeVGO7TwNeHhGPH2Ddxmqt5Wf6ZuC1gybyiNivTDgDJY2IeCbw8Yg4\nMyKeO0gZwPIB1+uN49nAH0XE2gUX/vfrPgv4GHAt8FOAzJwe8LtaCayi+E4GqQCsoBiBa68ytkH2\n8acDLx10/Yg4GvgCcFZXWllUzw5PXhGxM/B64M0RsWLAMvYDDqCoZfwP4G2DlBURj4yIFwGnR8Qu\nA6z/7Ig4o3y8aAcCEbEsIp4WEWdFxPOqJMGIeARwe0S8JyJeDcWPZPla3+8pIg6KiCdFxNERsXKQ\nH9qIOJ6il+u/VVmvZ/3jgPcCW4EHDVDEkrKcJq7a2ERxjeg+wCuqJvKIOAH4K+BDwGci4uzyOtN+\n1z8OOAf4JrAb8IKIqDTcY5lAL4iIt0XEb1VZt6eM5wLvBC7NzFsWWn7WuscA7wPeRPE+XlEepJGZ\nWXX/ysxvAx8H9gBeXzWRl5cGfQ34YETsMfN3UtE3y+1Tdf2eFonXAZdlQ72SPRgYcZm5wyfgQIrk\nexawouK6K4D/DhwMvAO4B3jigHH8PnAz8LoB1j0G+CeKwewneufv4M9yOcXgAWcAZwMnAV8q/9+9\nj/X3A74BvAX4CkWt53nAHhVieA5wBfB54GKKsYIfX74WfZZxHPAPwLPK5yuBAyvE8HTgxtn7AvCo\nPtefBH4ErCqfLx3w+zgIOGTmvQO/SXFg8YZ+93XgKOD7wBHAg8t9/R8pkvKSPtZfBUwDv1k+Xwt8\nGjixwvs4Dvg2cBrwduD/Ax5e8bP4DxRJ7wk9++qu5T63cx/rPwF4Svn4EOAPKQ4IntqzzLz7F/AU\nYN2seU8sf39+b6G/kfKzXNHzfDfgg8BR5fOJPt7HMRQVl6dS/Pb9A7DPrGXmLYfiLljXAU8un18N\nHN7PZzCrnKcBJwOvKf+v9J0u8F3tVCUWp2amRak9ZuZNFLdnWwW8tcr5ssy8h+IH6W+AZ1H8Ib4w\nIk6IiFP6LSciHgQ8ieIH6tqIOCQinhgRh/VZA3068NnM/GI+UHPdFfhARLyg3zi2E9eyCssuBS4E\nPp+Z787Ms4H/DbwYOIzi85lXZt4KXAYcDjyb4nN9BfDXEXFkRBy0QAzHAf8NOD0zn5+Zz6T4wb8o\nIh6XmbnQ51nWiP4GeFdmfiUiHkZxYFKlN/LjgfdmUduaKfddwP+OYrCFeWXmZuC/AN8sWxK2RsTS\nKrWUsun7euDrEXEa8J+Bv6ZIhnsAr+pzX38K8GeZeTlwb2Z+H3ghRWI9q4/38lOKg4dzyxrjLcD9\nwJr51/zV+5j5Pt6Rme+n+D6XUxzoVLGl3O69ZQvcWym+178EzluoJpyZ38nMb0bERGZeT3GAeT/w\n3JnzyllmkHmspGjKP6mn3G8Dn6FoJTl6rhUj4sHlcm8vW0bIzF9QNOu/qnzeT216OcXf1+9QfJaP\npGhVeF65nZivnPLv/BHAqzLzW1Hc3vJOYHUZQ/YsN1cZUe6ff0nx23AIcGgfsS8oIg4ETuyNRTvQ\nYh5BAA+jqKFUOnoDng/8EPho+fxdwO1Uq2nsTtG0dQbFEen1FD+459NHzRx4EcUR+dkUNdFnA4+h\nOMI9nSLJL6v4viYoblV3ep/LHwKcXT5eCvwJ8BPg1RQ/Xn/HPDXRmc+d4kfmAoqa0zPKz/aDFOfe\nzgN2m2P9mRrfc8vnO/e89raynAf1+V5mavOPpajNv7nP9Wbew3spks7M/OOBj1IkxJuBE/os73jg\nB8DKmc+1/P8oytaFBdY/uvxM/gvFqYG/Aj4C/DnFwedpwE4LvJfzer7XoKx9l5/NpRTnZBf8mynf\nyw0UTdKfo4/a76zv4xrKFpnyb+MbwJ9SNG9PLrR/l7G/mWK0qlvL7+NVFDXhjwLPq/L3UZZ5ULlv\n/Rl9tsCVn8NVwAtnfc6vpjgInrMWTPEb9VKK8a7PoagRLy3/tl40QPwHAp8E/hfwVYqDhG8CL5jv\nO+3ZDyfK//8AOKPn9ZOA9fO9l3K5TwMPqxr3POU9geLg7G/p82/dqdlp0QPo2RkeC+zax3K7lTvN\nYymS+TvKH6qPD7DN36BoEr+g/FE4ieKg4p19rLsG+H+B/0ORQL5I0Rz9FeB7wJXMajLrM6bHUJxL\nPbyPZf8z8Bfl4w9SnB54IkVz3akUByf7LlBGUCTxP6Q4Sv8e5cFQ+YO5coH1n0PRtLdn+Xynnte+\nCkxVeO/HUSTAM8vnS3rmH7XAusdQJP+ZJsZlwPLy8e8Bp1SIY3Yifx1FU/3+fa4/0/S5nKIp+9Ty\nR+4u4LsL/dj1vJcjyucT5fvZB/gscxxUzbOPTwN7lc+rJPKZg4D3lvE/nyJRfBv4C/o45UJx+uvJ\n5b7Yu298CHhJ1b+Pct1HULRIrK6wzrPLv9UX9sxbR3FwteCpE4pTGmdRHNj+bfn39u4K2+895XY6\n8JHy8d7l/vHQPsuZOQB5C/C58vGLy/3qEX2s91nK0z11J4oDgg+X06FNlOk0wPew2AGUO8MK4I3A\nZJ/LzxyN/jbF+dfjKM6V/f4A296d8rwY8GiKo+IP9RtH+QP35J55L6Y4ENilxudxNGVSXGC5ZwFv\nLB//GQ+cLzsY+OfyB+eFfW7zEIpa/H8bIN7ZSW9Z+f8XgUdXLOuZFAcSDy6fv5yiuX/e8+MUB3dn\nU5zrPLJn/ikUBzV9/UjOek9XUdQ6rwMOq7j+cyjOa8+cY19JUYM+oI91e9/LVM/8kynOMT94gPdy\nDWUir7juzEHAmln7fV9/q3OUeRKwkRo1Qiq2cpXrHEvRKvNWihaC71TZP3ngoPId5T61mT76nWyn\nnIcAnxj0vZdlPJaiVeQF5ftYMIlSnLP+40H2g+2U9TLgvPLx8rrlOdX4LhY7gF8FUuGPkqL2uIqi\nee/LwLnlD8vaGtvfm6Jj1meA/9RvPOUP68cpjupfRlFLmfOIuN/31+dyB1Ocu3wYRUL/NEVN/lCK\nJrsnlj/gfdW+yoR5Nn20iGxn3dmJfOazGCRxHE9Ru38t8HX675y2L0VrxKXAe4A/okikA9USKBLx\nNPC4Adc/vtz+ggdkc7yXt1H0cTiXovn02hqxnEBxumKi3/1r1vu4tu6Pf/k39kaKA4pKB3dNTRR9\nJ86l6AvzyIrrRs/jveg5sKlYzoPLz3OgDrllGfuX++b1Vd5HUwkXeDjlab+q+5NTs9PIjp1eduJ4\ncGZuLjswnZuZdw1Y1gRFZ5MTM/OcKIZ33JyZC17qVHZ+OYniiPinFE3xVw8SxyAi4mSKpv0vUNSm\nT6c4Z3c68FCKg4vXZubP+yjrERS1v3X9vPftrH98uf4HKM4jrs/M71YtpyzruRTncR+fFe76U14q\neDhFjf42ikubbhgkhrK8XQf5LHrWP4HiwOiIrH5J0S7AFEUNcjPwt1l08Bo0lhVZdAwdZN0TKA4q\npqq+j54ydqFoZbo+M28cpIzFVnZCq/WjWXaW/F2KO1r984BlLKPoA/O+OvtEHRGxU2ZuWYxt6wEj\nm8ThgT+oiFiSmdsaLLdyeeUfVWTmfU3F0ed2J4Hforhu/nMUl0ntQ1HreTdFQq6SBOsmrYGSbxux\nDIs6yXOYdOV9DIOIWJqZW2uWsSwz728qJo2mkU7iKpQDiRxJcXS/iWKwk38F3p+Z1y5CPJ1IvpI0\n7EziHVJei7uljdYJSdLwGesxwztoS8/5uoHOW0qSRoc1cUmSRpQ1cUmSRpRJXJKkEWUSlyRpRJnE\nJUkaUSZxSZJG1P8FP2+b/23mY28AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "seq.plot_confusion_matrix(dev, local_1.predict(dev))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This matrix shows a couple of interesting properties of our current model. First, we notice a relatively strong diagonal. This corresponds to correctly predicted labels and is a good sign. We also notice the 'N' (common nouns) column that receives a lot of counts. This means that many words that aren't common nouns are labelled as common nouns. This may not be surprising, as the large frequency of common nouns in the data (observe the very dark 'N'-'N' dot) could have 'N' the default class, chosen whenever there is too much uncertainty. \n", "\n", "The matrix enables us to spot systematic problems that, if fixed, may lead to substantial improvements. One problem is the fact that '@' (used for tokens that address twitter user ids) is never labelled correctly (its diagonal entry is missing). Why could this be? To investigate this further it is useful to look at the learned weight vector for this class. " ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAHBCAYAAACrLvV5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xe8XVWZ//HPk0pIgFQgpBBKQkjohN4RSEjoAoLSQaQq\nglKlDxCK9GboHQQE6UVABREQEFFwnGEcZ8RhlHFmnJ/jNGX9/niezVn35Nx7z71n3ZLwfb9e93XP\n2eecXdde+1llr20pJURERESkdQP6egVEREREFhcKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiI\nFKLASkRERKQQBVYiIiIihSiwEhERESlEgZWIiIhIIYP6asFjx45NU6ZM6avFi4iIiDTtjTfe+JeU\n0rjOvtdngdWUKVN4/fXX+2rxIiIiIk0zs39o5ntqChQREREpRIGViIiISCEKrEREREQKUWAlIiIi\nUogCKxEREZFCFFiJiIiIFNJpYGVmS5jZa2b2EzN7x8zObvAdM7Mrzew9M3vbzNbrmdUVERER6b+a\nGcfqf4BtU0p/NLPBwEtm9mRK6ZXsOzsCU+NvI+C6+C8iIiLyidFpjVVyf4y3g+Mv1X1tV+D2+O4r\nwEgzG192VUVERET6t6b6WJnZQDN7C/gd8GxK6dW6r0wAfp29fz+m1c/ncDN73cxe//DDD7u7ziIi\nIiL9UlOBVUrpLymldYCJwIZmtkZ3FpZSWpBSmpVSmjVuXKeP2xERERFZpHTprsCU0r8DLwBz6j76\nDTApez8xpomIiIh8YjRzV+A4MxsZr4cB2wN/Xfe1R4AD4u7AjYE/pJQ+KL62IiIiIv1YM3cFjgdu\nM7OBeCD2zZTSY2Z2BEBK6XrgCWAu8B7wJ+DgHlrfLpty8uNF5/er+fOKzk9EREQWH50GVimlt4F1\nG0y/PnudgKPLrpqIiIjIokUjr4uIiIgUosBKREREpBAFViIiIiKFKLASERERKUSBlYiIiEghCqxE\nREREClFgJSIiIlKIAisRERGRQhRYiYiIiBSiwEpERESkEAVWIiIiIoUosBIREREpRIGViIiISCEK\nrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKQQBVYiIiIihSiwEhERESlEgZWIiIhI\nIQqsRERERApRYCUiIiJSiAIrERERkUIUWImIiIgUosBKREREpBAFViIiIiKFKLASERERKUSBlYiI\niEghCqxEREREClFgJSIiIlKIAisRERGRQhRYiYiIiBSiwEpERESkEAVWIiIiIoUosBIREREpRIGV\niIiISCEKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKQQBVYiIiIihXQaWJnZJDN7\nwczeNbN3zOxLDb6ztZn9wczeir8zemZ1RURERPqvQU1858/ACSmlN81sKeANM3s2pfRu3fdeTCnt\nVH4VRURERBYNndZYpZQ+SCm9Ga//H/BzYEJPr5iIiIjIoqZLfazMbAqwLvBqg483NbO3zexJM5tZ\nYN1EREREFinNNAUCYGYjgAeB41JK/1H38ZvA5JTSH81sLvAwMLXBPA4HDgeYPHlyt1daREREpD9q\nqsbKzAbjQdVdKaVv1X+eUvqPlNIf4/UTwGAzG9vgewtSSrNSSrPGjRvX4qqLiIiI9C/N3BVowE3A\nz1NKl7bzneXje5jZhjHf35dcUREREZH+rpmmwM2A/YGfmtlbMe1UYDJASul6YE/gSDP7M/BfwD4p\npdQD6ysiIiLSb3UaWKWUXgKsk+9cDVxdaqVEREREFkUaeV1ERESkEAVWIiIiIoUosBIREREpRIGV\niIiISCEKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKQQBVYiIiIihSiwEhERESlE\ngZWIiIhIIQqsRERERApRYCUiIiJSiAIrERERkUIUWImIiIgUosBKREREpBAFViIiIiKFKLASERER\nKUSBlYiIiEghCqxEREREClFgJSIiIlKIAisRERGRQhRYiYiIiBSiwEpERESkEAVWIiIiIoUosBIR\nEREpRIGViIiISCEKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKQQBVYiIiIihQzq\n6xVYHEw5+fHi8/zV/Hk9vpzeWMbitpzFfVt6azmL0z5bnLalt5bTW/tMpC+oxkpERESkEAVWIiIi\nIoUosBIREREpRIGViIiISCEKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKSQTgMr\nM5tkZi+Y2btm9o6ZfanBd8zMrjSz98zsbTNbr2dWV0RERKT/auaRNn8GTkgpvWlmSwFvmNmzKaV3\ns+/sCEyNv42A6+K/iIiIyCdGpzVWKaUPUkpvxuv/B/wcmFD3tV2B25N7BRhpZuOLr62IiIhIP9al\nPlZmNgVYF3i17qMJwK+z9++zcPCFmR1uZq+b2esffvhh19ZUREREpJ9rOrAysxHAg8BxKaX/6M7C\nUkoLUkqzUkqzxo0b151ZiIiIiPRbTQVWZjYYD6ruSil9q8FXfgNMyt5PjGkiIiIinxjN3BVowE3A\nz1NKl7bztUeAA+LuwI2BP6SUPii4niIiIiL9XjN3BW4G7A/81MzeimmnApMBUkrXA08Ac4H3gD8B\nB5dfVREREZH+rdPAKqX0EmCdfCcBR5daKREREZFFkUZeFxERESlEgZWIiIhIIQqsRERERApRYCUi\nIiJSiAIrERERkUIUWImIiIgUosBKREREpBAFViIiIiKFKLASERERKUSBlYiIiEghCqxEREREClFg\nJSIiIlKIAisRERGRQhRYiYiIiBSiwEpERESkEAVWIiIiIoUosBIREREpRIGViIiISCEKrEREREQK\nUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKQQBVYiIiIihSiwEhERESlEgZWIiIhIIQqsRERE\nRApRYCUiIiJSiAIrERERkUIUWImIiIgUosBKREREpBAFViIiIiKFDOrrFRAREekJU05+vOj8fjV/\nXo8vY3FbTl9uS19RjZWIiIhIIQqsRERERApRYCUiIiJSiAIrERERkUIUWImIiIgUosBKREREpBAF\nViIiIiKFKLASERERKUSBlYiIiEghCqxERERECuk0sDKzm83sd2b2s3Y+39rM/mBmb8XfGeVXU0RE\nRKT/a+ZZgbcCVwO3d/CdF1NKOxVZIxEREZFFVKc1Viml7wP/2gvrIiIiIrJIK9XHalMze9vMnjSz\nmYXmKSIiIrJIaaYpsDNvApNTSn80s7nAw8DURl80s8OBwwEmT55cYNEiIiIi/UfLNVYppf9IKf0x\nXj8BDDazse18d0FKaVZKada4ceNaXbSIiIhIv9JyYGVmy5uZxesNY56/b3W+IiIiIouaTpsCzewe\nYGtgrJm9D5wJDAZIKV0P7AkcaWZ/Bv4L2CellHpsjUVERET6qU4Dq5TSvp18fjU+HIOIiIjIJ5pG\nXhcREREpRIGViIiISCEKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKQQBVYiIiIi\nhSiwEhERESlEgZWIiIhIIQqsRERERApRYCUiIiJSiAIrERERkUIUWImIiIgUosBKREREpBAFViIi\nIiKFKLASERERKUSBlYiIiEghCqxEREREClFgJSIiIlKIAisRERGRQhRYiYiIiBSiwEpERESkEAVW\nIiIiIoUosBIREREpRIGViIiISCEKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKQQ\nBVYiIiIihSiwEhERESlEgZWIiIhIIQqsRERERApRYCUiIiJSiAIrERERkUIUWImIiIgUosBKRERE\npBAFViIiIiKFKLASERERKUSBlYiIiEghCqxEREREClFgJSIiIlJIp4GVmd1sZr8zs5+187mZ2ZVm\n9p6ZvW1m65VfTREREZH+r5kaq1uBOR18viMwNf4OB65rfbVEREREFj2dBlYppe8D/9rBV3YFbk/u\nFWCkmY0vtYIiIiIii4oSfawmAL/O3r8f0xZiZoeb2etm9vqHH35YYNEiIiIi/Uevdl5PKS1IKc1K\nKc0aN25cby5aREREpMeVCKx+A0zK3k+MaSIiIiKfKCUCq0eAA+LuwI2BP6SUPigwXxEREZFFyqDO\nvmBm9wBbA2PN7H3gTGAwQErpeuAJYC7wHvAn4OCeWlkRERGR/qzTwCqltG8nnyfg6GJrJCIiIrKI\n0sjrIiIiIoUosBIREREpRIGViIiISCEKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRE\nRKQQBVYiIiIihSiwEhERESlEgZWIiIhIIQqsRERERApRYCUiIiJSiAIrERERkUIUWImIiIgUosBK\nREREpBAFViIiIiKFKLASERERKUSBlYiIiEghCqxEREREClFgJSIiIlKIAisRERGRQhRYiYiIiBSi\nwEpERESkEAVWIiIiIoUosBIREREpRIGViIiISCEKrEREREQKUWAlIiIiUogCKxEREZFCFFiJiIiI\nFKLASkRERKQQBVYiIiIihSiwEhERESlEgZWIiIhIIQqsRERERApRYCUiIiJSiAIrERERkUIUWImI\niIgUosBKREREpBAFViIiIiKFKLASERERKUSBlYiIiEghTQVWZjbHzH5hZu+Z2ckNPt/azP5gZm/F\n3xnlV1VERESkfxvU2RfMbCBwDbA98D7wIzN7JKX0bt1XX0wp7dQD6ygiIiKySGimxmpD4L2U0i9T\nSv8L3Avs2rOrJSIiIrLoaSawmgD8Onv/fkyrt6mZvW1mT5rZzEYzMrPDzex1M3v9ww8/7MbqioiI\niPRfpTqvvwlMTimtBVwFPNzoSymlBSmlWSmlWePGjSu0aBEREZH+oZnA6jfApOz9xJj2sZTSf6SU\n/hivnwAGm9nYYmspIiIisghoJrD6ETDVzFYysyHAPsAj+RfMbHkzs3i9Ycz396VXVkRERKQ/6/Su\nwJTSn83sGOBpYCBwc0rpHTM7Ij6/HtgTONLM/gz8F7BPSin14HqLiIiI9DudBlbwcfPeE3XTrs9e\nXw1cXXbVRERERBYtGnldREREpBAFViIiIiKFKLASERERKUSBlYiIiEghCqxEREREClFgJSIiIlKI\nAisRERGRQhRYiYiIiBSiwEpERESkEAVWIiIiIoUosBIREREpRIGViIiISCEKrEREREQKUWAlIiIi\nUogCKxEREZFCFFiJiIiIFKLASkRERKQQBVYiIiIihSiwEhERESlEgZWIiIhIIQqsRERERApRYCUi\nIiJSiAIrERERkUIUWImIiIgUosBKREREpBAFViIiIiKFKLASERERKUSBlYiIiEghCqxEREREClFg\nJSIiIlKIAisRERGRQhRYiYiIiBSiwEpERESkEAVWIiIiIoUosBIREREpRIGViIiISCEKrEREREQK\nUWAlIiIiUogCKxEREZFCFFiJiIiIFKLASkRERKQQBVYiIiIihSiwEhERESlEgZWIiIhIIU0FVmY2\nx8x+YWbvmdnJDT43M7syPn/bzNYrv6oiIiIi/VungZWZDQSuAXYEZgD7mtmMuq/tCEyNv8OB6wqv\np4iIiEi/10yN1YbAeymlX6aU/he4F9i17ju7Arcn9wow0szGF15XERERkX7NUkodf8FsT2BOSumw\neL8/sFFK6ZjsO48B81NKL8X754CTUkqv183rcLxGC2A14BelNqSAscC/LAbLWNyWszhtS28tR9vy\nyV6OtuWTvZzFaVt6cznNWDGlNK6zLw3qjTWppJQWAAt6c5nNMrPXU0qzFvVlLG7LWZy2pbeWo235\nZC9H2/LJXs7itC29uZySmmkK/A0wKXs/MaZ19TsiIiIii7VmAqsfAVPNbCUzGwLsAzxS951HgAPi\n7sCNgT+klD4ovK4iIiIi/VqnTYEppT+b2THA08BA4OaU0jtmdkR8fj3wBDAXeA/4E3Bwz61yj+mN\nJsreagZdnJazOG1Lby1H2/LJXo625ZO9nMVpW3pzOcV02nldRERERJqjkddFREREClFgJdLHzMz6\neh1ERKQMBVYifcTMlgRIKSUFVyIiiwcFViJ9wMxGAyeZ2TxQcNUTqv3Z3/drtp6j+npdpH8ysxXM\nbOlu/nakmU0ovU5NLLfo+Wdmi0y8ssisqHSPmQ2NYTJ6e7nVSdWry+7vF9HMEGAwsIWZbQ99H1wt\nShlXk6aA79fuziBLx0MLrdNC84/jvj1wlpkt0xPLaWY9CsxjWIl1aWH5Xd6G7PgOLL9GZZjZqsDf\nAad047dLAOcD+5nZpM6+X0oUHCdD7fzrbhozs7PNbMOU0keLSh61SKzkJ52ZnWJmXzGzXbqSAZjZ\nNOBqfIyx5XtuDRcWF4t5wDVmdmVPLad+f7R6EjezHDMbFP+7ff6klP4Zf7j5/wN26OvgyswGRMZl\nZja21Qt8Xwe4ZjYcuMfMZrcynzges4HrzOz4mG8xMf9ZwBzgoZTSH0rOv14WSMwys43MbP1qPVqc\n7wzgCTMb35vHPtuejYFPVedms7+N/b8D8DUzO8bMVuipde0OM5sO3IIPd/RRTGt6/6aU/hu4H5gO\n7G5mk3tiPXNmtgC4AXjOzK4ws8/GunQ3b/sI+IaZrdNbwZWZrVgVpsxsRleXqcCqnzOzq4CNgHeA\n44DzzGzlJn43A7gDeAt4Pi7kvSYuFucAjwNrmdndZjay8DIGpJT+YmYDzGy+mR1mZpua2cDSAUq2\nnCvwi+xaKaWPuri+UyzGfzOzQSml3wB3Av8OzO6r4CouMB/FMl/EM/Kvm9nmzf4+/q9mZhtC6xfq\n7sgzv5TSfwIPAkvXf9bFeW4MnIvvl72Ak81sSqvrGvMeGIHA5cAuwO9ieo8d+0hbuwDXAbOBC81s\n11bmaWarATcD96SUPujNYx/bMwe4Cx9D8S9d/O02+P7/DnAacFRXgrOeZGarA7cDVwKnArtFrWBT\n6aNK8ymlF/DjsyE9HFyZ2S3AssABwA7APwJbmtmxsS5Npw0zu8DMlkopnY3vh1t6I7gys08BawNf\nNrOXgS26mtcrsOrHzKtupwOHppSeBE4CDgE+11ENlJmNxRPidSmla1JKv4zp88xsg15Y76nAUcDD\nKaWHU0pb44PRXm0F+5FkJ9izwH8Bs4Az8Bq6QSUClLoasevw7fgF8KiZbdaF+QwGzgSuNbPzgaPN\nbEJK6R+Am4B/BeZG6bnXApMqCI23s4FngM8Db+J9wLbsbB6xn3fCA5ljzewpM1vLeql5xcyWNrOh\nkR7Wzj76KXCumU3qasYY810JOBm4M6V0Cx5YrQIcGp91d32rNLlESunP+H7/O+AY6Nljb2ZjgKOB\n7YAPgGHAy909VuY1m7cD349nwWJes75FFO56jLkxwJeBQ1JKL3fxtwOBHYEjgP/DH8N2fQyK3adN\ng7Ffz4j1uR/4PZ5H/F9WCOro91Vhac0IpH6GF3Q3wAO04s2CsZzlgN1TSv8Z153rge8DM81r37pi\nFeBHZjY8pXQZXgjt0eDKzG4DtgX+B09X/wvcG581fS1RYNW//QaP+LeOg/o28Apeg7VLB78bBLyV\nUro1q004GrgYr4mY07OrzdJ4qWpDM1sPIKW0NzASr9JtuUSYJfLDgSeiVLMG8M/AVsDeVXDVwjLy\nGrEtgPdSSkenlC4BLgBubrZWJ6X0f3ip8V08Ex8HPGxme+H76gLgt/ixXrvdGRVUt31fw0uZ/xi1\nm/cAjwEnVjVpHcxnFh40bg88hKfPU4E1e7rmLS5AJwMHmdeIHm1mz5vZnnht7WV4M1t3aq1G4U21\nu5vZtJTS+8BXgLWAL5j3X+myrHnxDjM7LdZvD2B9M7u0O/Psgj8D/wQciB/vg1JKHwLbWBM14fWi\n6fIRYLKZbWZmzwL74oXA46JGqEfEuf1veID4pzi+A+Hj2uGF+ndmtTgppfQXvJD0JbzLxKdTSu+b\n2f7A/j213k36b+CMlNLNACml3+K1cRvEeZvMa4inNvpxfD4XuA3YDy8w/SdeOFwX+EwP1Fz9G56X\nTTKvjR0YNccPAcsT52FnqqA2rhnfB96M4OrreHB1UxZcFctforC0DHAWMBzfdz8EDovzv+pm0mk+\nosCqHzKzbc2sqn58CdgJv9B9F7gbv5AcaGbLtHOQlwG2NbPpcYItAUwF9gQuwS9CIwqubxW8rRGl\n1A/wjPUXwE5VoJBS2gmYH6X07i6rOumqgOl+vGnuOuCBlNJB+El8INBU0NPecrIT9yl8vx1qZudE\nxnY9cCner2RaB/NZKnv7Q/w4/iyl9DXgW/jjGr6JByLfAcbipcoel23fbXhH0yWBXc1sckrp34D7\ngOeBNTuZ1d/hpf4ZeAfbGUDCmxR7elv+BPwSWBU/T74AXIuXdp8H9gZ2A9/ejmaUpePVIsj4KfBV\nvDBzhJlNTSn9E14be2/0X+ky81rAS+JvBrBfSulPeNPJdmZ2TXfm28ky1zKzTSMQ+hBv9jompfS3\nZrY1npabvtHEzJaNgJqU0nnAT/DmuLdSSvvix+HfgNUKb0d1jIbHsj/Cg8UdU0ofRW3TLOCv8IJc\n9buh1fejFmeTqO36JTAeuDCl9A+RV52E52F9wswGp5T+B/h9BCjVcflPYEpsw6bAo0RTd4N5TMZr\nvHbGj8NfgP9JKf0A+AawHoWu/2Z2iJmtiQeDSwK7pJT+EoW2YSml/8Lzuk5vAIl89y9ZPn84ni/+\nOAuu7gIej3yqWO1uSunv8RuKPgBmpZROjGVPAnY2f17yKUDnBZCUkv760R9edfo8HuVfHtNm4VXW\nO8T7EXj1++AGv68eUzQfL10vH++HxP+5eN+rEYXXezbwN7H+7wNb4he7i/DamHULLKPatgH4BW/j\nbPu+CewZr+/Bmwa6u5zh1fLwIPbEeL873h/jhGxdtulgPkvhF/mDs2lfwpvMxgOv4tXNq+B9eJbA\na37uxmsdrRfS23nA09n7BXhAtFK8H9boGMTrscBy2ftTgTOzffVDYHoPrvvA+L8dnnG/jNf8DI7p\n68exehs4ssm0NQev0bkHL+WPAFbEL9TXA9MKrPc+wDbAJvhD7ifH9BGRZjYtuI+q7Tobr1naGA92\n5wMvAF/E+2/u3IV5zsRrA58Dns2mb5+nl0jb8+vTTYFt2RF4Ms6VnfGaxe9Gur08jvfu2e+WwwP+\nVfHg9f04vu8CWwAnArfG8X4Z2LWn0mwn2zcqe71u5BNLZ9NOAj6L15i+AczrYF7j8Hy3Og+nxvTZ\neK3e0oXWeVe8c/lleOFsLTwwObzue48BX+lkXgOq/3hQeBywXky7Lo5XlTfvWPIcoZaXnIs/8/iW\n7PPZcb68jheoOp9nXyQg/bV7gL8EPBmvV40TY0CDRPAo8I1O5rU7fmfGcdQukpvgpe8dCq/3SOAH\nwLbxfjfg7/HOklOBr1cndgvLqDJVw4OopyPjOQ2YgAeMvwW+h3eibfO7LiznUGCjeL0e3tZ+Zbxf\nKjKSa4Cz6n43oMG8xuJ94m4FPpdNfywyo69m04bG/68Aa/VgGhuQvR6G1zT9FvhsNv1G4NvAuLp0\ntyywT7yfHWnpHeAzMW0efoE7G3gN2Lwnz5dY5lZ4QL8T3lftWuAgvP8S+EVkFyI4bvD7IdnrGXhH\n4U3j/ZWRrofjQzdcBMzoxjoOrHu/K35xfwsYHdPm4H1gBnZ1/p0sOz+GZ+CFqs2A0cBheDPRVs2e\nK5Gmv0etEPNco+Mcy/gpkScU3J7t8NqxTfGal5/iTZpD4zgflh2/Ks+YgNdynIfXzlafHxrpfEac\nCytTyyt7vFBTt12DIx2fHu+XBRbE6yrgOBIPWt6inaAKWAlYJ15/F78xZmS83wJvWlu54HpPBJ7A\nazxPx/P7DfEbMa6MNP0o3k+xmfkZXrFwMn79ehmYFJ9dF9szNPv+QvluF9e//tys0swzZEFUpPsZ\n9d9rd769mXj01+lBPhD4Urw+Db/g3Y13DM2/0yZwaC8zptb09/NIlO9QuDSGl1CWi/mvQy3yPwq4\nK16PbHEZA7PXM4Gz4/UmeEniLLwGaApZ0NidzJFaCelT8X8bvLlgl3g/DK9xOLCDeYwEBsXrEfjF\n607ggJg2u+4YDmrvGBY+VtWxsdiP1UVkX+BhvI9J9d1DG/x+fzx4ORYPbGfitQdvxz4ZidcY3QrM\n7eFtqTLAE4C/itdD8IvPU/jFdonsOy/jNYJ5jduyeJPVUngTxnN403uegV6BX8hHVPPrwjqOzF5v\nHuu2EV5oOj/204j47Gd0UAPRzX00Ce/Xt2c27dzYF5vRjYsSMAZvfl8z3r+D51HfxDv9DsLzhF/2\nwPYsgRcEpsc59BrwaTz4PaKTND8e70f1JrB/9vkZeEFnUE+m1ya3by28tvREvDvHgrrP18ADydnt\nnAub4U1XD+K1R+vggeP1+HXjLQrl/8CS2evL8NrPM/Ga3YnxdyheUDwq+267aQ7Pl2YD58X75/Dm\naqjVVHW7JaKDtDEAz5+vA75cfYbncXc3+F2n502fJiT9LXTAPhWZ+G14J+xVYvqDRMRP2xqHlasE\nnmcMtL14DMKbEtcEVq//vMX13Qy/EK2BX3AvzhLrXODmVpdH2+rh7wEPAL+uW4eL8BLT2PrfdWE5\nefC2BV5iOTDe74hXD+8e7wdn37W6+UzCA9nH8SBjVhyDffBmtr3xAOCdKtPopbSVZyIv4hfCf8QD\ni1mxXo+R1axV20ct4x6M1wbdCDyWfWc2fsHar/63PbAd9ft7Z/yu0JnZtB/Gvq5KugfRoKYJr6lY\nFS8YTIjz6UngeNo2wVwLbNLF9VwSb67/At4k+Q4eSN2Nl+T3wUv4r8b671xyn+FB7gC8OW4B2QUV\nb9K4ky4WeCItjMBrJB+P/XwbfhE6C29eG1Ht2xLb0+B4D8aDuyeo1cx8Gw8WV260PDzA2A4PzK7E\nC2Oz4rMt8Vq8PgusaJtfTwX+BW/a/lYcvyPxm3QOBdZuZ7/sgAeax+DNy/Px5sTl8LzxK9S6krR6\nTG6J9LNRfx5xAAAgAElEQVRrvF8Kr2HaH29lmZ+fj9nvGtXqH4kHfWvF+9Xxvp1vA8fFtKXxoG1Y\nR/Pq5rYMiLT0ldiHHwFfy9L7j2knaO9wvn2VmPT38YGtr4pcPjKCs7NpM4Er6r5nccL8jlo0v1Bw\n1epJ1MF6r4lXxR8e70fhJYxbgAvx0lHT/TaaWN78OGmH4lXFeS3eNkQTVYljENO2xi8UB8X7OXhH\n6fU7mdcykbH9O960+3P8QnRZZIwP4f1cNqdwM0kT2zkI7wd1dbzfCrgKH15hIJ55n9Dgd1VaWjH+\n74tfzA6mFtjPi22d0INprlqPjWPZ0yI9nAd8Da/BXBVvAlmng/nkgfGoSK+XxLk3FS/1f5kWalqp\nNU3djTe/VM1P6+P9A6uS8RhgqXz7CuyfGcBfRxozvPZ4AT5cxHT8gr1xC8uZGMu4Bdgsm/4c0axY\n+LjvhNcmHJmlwcfwsYY2xAueC3U1iG0fggfWD+H5xNJ4U/4TeA3WG8AePZFeu3jMZgMnxetpwD/g\nLRa74AXHBUQXhXa28RaiL2ccnxvxgHFi4fVdGg98/jbS2FF4DfYFsX/H4LWxVwGrdjKva+NcPQsP\n0kfhtcg34c2Ay8X3HgBuKrgNec396nit5VJ4YfocvEn24paW0VcJSn8L1SKchjfdbRkJ7G38wjcI\nr124qp15XBoJfFi8X6jkRcFmpiwj2B+vgr+aWgfyEXjNx8HAlvn3W1zm5/H+PPtl0x4DXmpv/bqx\nPQMigz4/MqRReG3IHdSCq3b7PuE1VV+I1yMjozgVDzS2x8dCeRa/O+c1YEyp/dPJ9t1K22bHk4jA\nKt5vg/eHW56sv1GD+czBA6fpeBB2GN5R+ABqwdXYntiGuvWo1vcG/MKzCbVg5VW85mL3Dn4/JM6x\ndfES6j7x/gK8GWN5PDh7GS/FdvvciWXNjvW9PJu+J36h74kavZ3wi/ALeO33NjF9/5j2M2CnQss6\nGK8hmYQHpD8jasULbs+Kkc+cHMfoPrx/2LFxPv2crKmznXlMiHR6V6Sf4Xih8B5qtV692qcqllnV\nxm+DBypbZp+tgvfDO6zJeZ2CtxhUffZWjnR3WkfndQvH5Mt4k/uRePPsH/A8bzge8HZ2TO4DbozX\nA/Ha3Vl4d4618X65r+A1o7dnv2u18DEgW+b9eBA7BA+ovh6ffQGvuep2t5JeTUj6a3yg8dLThXjk\n/lZMPyROtsdpe4fCCKKEm037PV5L0qjmqgreRgLbt7CeVQCyQjZtdmRWe1PwosrCtXjTItO4hqxJ\nBg9QTi+0zCfwvjibAb+i1sdq18hAtsiPWYPfr4U3hx0b78fhtXbnZd+ZCHwG2K6X0tZYPNj4qEpD\neKff68lqLPD+VSt1MJ/p+AVs82zaILwGbkGk1QGN9kvh7ZmGN0FsFe8PxWscqiB+DLUgv2FGiNck\n7Y33n/g1UeOC3yl3EV67uALRCbcb61idJ+OJjuN4s/jjWdpYDw8Wluvq/DtZ9iS8yXrTWP5n8T45\nW8fnwynbcflTeFPgo3iTSbsBbTf34TS8f+Ih8X4CnkfeGcd6MJEfUVdDjw/z8Atqd4iuQK3magu8\n5mWh5qre+IvtqNLpQLw27rDsfdU/cyY+VMI0GrdGzMRrb4fgBYUF+I1Dw/DA7Fmym0taXOf6PHly\nnCt347U9m8ey669NjZpml8QHO60CmaPxwOxivAVmu5i+CtGXL94Xy1/wvpN53nw4XoM2Fm9dOK6l\n+fdFwtJfmwO8O3F3GN73Je/oNwaYkL1fOg5+3vnyoUjgV0RG8nFwRS2oWibmvVWL6zoX71M1Hy+x\nVHdc3YaXiEe1Mv9YRl6Ld2Isc2W8r8AFeLNPl/q7NLHM4XiGPQBvBjo+pld93DpqVlqNWk3VOngz\nZdU3YBx+4V+oWrlRhtND6WvTyCh+Dnwzpp2MNxXcgFez39fJPGYBt1XrTW3ojgFx3Hv0AhXLGRRp\n7nW8b1JV8jwIDybmNDOf+D8ZDzgexGurqgvVelWGSwulfPwC8118eINz8RqwOXjN8vN4P65dCu6f\nPJj7VjZ9YGzPO0RBoZvzH9bBZyvG9s0ska6zbdkBz89ej325bLaN8/F8L+9zM4JoeiIKDXiN1JvU\nApWV8Fq7bxE1O739F+n4C3i/1Oo8OocYDoRaIFiNM7VkO/OZjQ8g/S28ZnIIfi25EW+SfQcPTI4h\nG+6lm+uc58nzqBVGxuA1S7fRZME6OwfH4jVqr+EBYBUg74OPs7Z8o3TRwjacEPtsWbyw/Ndkwz/g\nffCuwvPr2+vXt8vL64vE9Un+Y+HIfx7eTPQd4IsxbTgexS+Tfa/qFHpQnDx74zUpF2XfqcaQGp5N\nGxnz7tat71lGtznePDk1lvM63i9lMN60cS8wvtA+GoBffK6ITPQZvHp4LN5UdzVZ+31XTzrqxv8i\nHuuBdxo9MJt+K9mYQvXLoXY3yb/jzSKDWDi4Ghsn8dd7MY3Ny9ZveTzwXjbSQXWn5jp4/4jDs9+1\nV8szEc+8d86mzSXuYO3B7ajSXl5Y+HxkgPmdboeQ1Sh2Mq9tI2Mdj1/gLifuhsRrs+YAq7Wwzmvi\nfapG4H03Xo30NQgvhDxCFAy6mm472Ka8o/1TZHeT4YHvbXgQuWw3ljGDWsFhYP2yG3y/5W4HeFDx\nJB6wDcaDqAuIizde+7Rq3W9WinziArwQUd2oc0ek3cF4rfIDrRzfQul6CF7oqsaz2wUvsE6NdLI2\n3gLRcAw4vDB3E7BBvL8Rrx2vhmzZAK+93CH2RSvpOe8q8QSeJ9+HP2oH/PpyUaS7ZTqZVxWgVYHu\naLz5+Pa6732bQteSmF9VgNw80sF4/AaVb5HdYYl3/2h6SIUOl9mXCeyT9kfbO9wuwKtvh+B3u+UD\n7T0MXJa9Xx3vZ1WVag7Fo/xHGizjRmpNJcPxAKjDi0476zoGDwqqEtRe+EVjBzyq3wVvArgUvyB1\nOdPuYNkn4H2BBuIXppvwkuZMPFDYq4V5VwHqYGo1YgPxvjov4xfpUbG/72hifvvjA9ddiQdXA6gF\nV3mzYI+P6RTLOhVv+rsS7/swFO9/dE18/gOi5qpR2mwwvcpYD8Q7yJ6A9wl5i0J9dTpZbjUY5Fl4\nAG94oeNysrG38t90MM/d8CarHeP9SHyct8vxQsLr+MjWXVnPFYBLsvcbxf4+MNLTyjF9tUgbRfuh\nxfn4DN6cdFps06P4RftA4ukHeJ+iLtco44Hme2RdAOo+ry6WRfrxxLlyMP4ooa1j2nj8InhFo/2X\npZWj8XHnTq/7/E681utvgN16Ks02sW1VXjoa70t0Md63aEm8kPM0XkD9EQ2aVfF8agRe4GszHiHe\nDPhravnbBLyVYs0W1nd6tX/xlpJT8NaPl/FrwLfi8zF00txI21qvA/F+p0Njfn8PXBCf30FdoNXi\nPr+A7A7mbPoEvDbvehoMDUOr42P1VSL7pP5FIn0Gb56pEts6eMT/JN4XIy9xzohEvD9x+3hM/yw+\nRs0uNKjWjuWsRDcGm4xl/iBO9KpZbIk4EW6hVu1/B17d3lKHVRauxaue2fQgcGpMex4vfa2Yb2MX\nl/MZvPQ7Otb7YTyo/Rre3LVRZGq3ANe2txyiBiJeD4p53RcZXnWX5Np4UHh8L6evdfDS7914k9Zz\nkYn9gCgF4o+g+XIX57ss3qfmiciMig4P0CgtxHrng0G+Tdz6jDcLXkNdk0EH8x0d+2IinrmvH+fQ\nYDzguopuNM/hF8UZ1O5Wm4AHNj+i1jQ1N/Z/scJHzHc9PFiYiwe7d0aeMATviH8GXkuzGd7k0nSf\nLrIxu/Cmqi+RDb2RHys8mHuMdoKvLixzZbxANRavZX2MWq3M+HjfXi3OJLwpbD+8b+pedZ+Pzo5R\nbw/+mQ8Dsyaery4TaeWv8H6qS8U5tgbZUBV4AFb1MasK5jPxQOo02taw3EwEo/G+20/XwG+c+iG1\nmx/WjbT+JB7ALonnI0/X/a7dfRvn3ffx/sTPURuOZyw+7tl/EwXAUscJLzCtG6+H1n22Ct75/pu0\nEIA2XG5vJjD9JfC23Iey91VpayDe3yO/fXkofoFcaLDG+PzzeBCwD51Uw3Zh/VbHLwqfxjPu18g6\nNuPNSafFifYKsEah5Q7AS98bZtNuplb9v4C6cZK6OH/Daw0uwYODS2L6TLzG4mtk/dny9ap7PwW/\nmJ1PrWZwW7xGr2rWrTqirkuDW6R7KF0dTq1Gc328hm8ufqE5AW+ObLlzMVkn9RIZX928x+I1UxPj\n/VGRHvPBIF/K9u+KTc53aTwAfi2O/814LeiHxJg1eTrpZtq6m9pTE47Fg+1D49x8hx6o3cNrWS/O\njssIvGYnb77eDG9uabqAFennTDzIGYDfHXtd3fbm/Tefoxu14g2Wuy2e94zC877jieFJ4vOFasWo\nPRHgG9Sa37fHL9RzYp53NPptb/zhgfsj1J7eMIJsFHK8j9q58Z0pDX57J14DfQ5ew1LdELF6pOPT\nqAsKWj0/Y58ugV9fHiPy5NjPt8ZnhuebZ3Vhvp8HTonXb5Bd1/Barwvqt6HA/n+mWmY+b2pNw6vT\nwk1d7S63LxLbJ+mPhWtjZuI1JZOpdV5cmrpxZbLE/Si1KvGFxqbCbyO+myZL7p2s60g8Y74rm/Yz\n/CJxHl6jszJe+v4OLTTJxbwvotYs8zxeknsEeD6m3RLr8z2i83T99je5nKoK3vAS4dV4qXZUTF8T\nr7H4OvHctvaWgwdLr+MX+HfxwOV0PFhbAW/H/xbdGFSuxX15QGTCB8b7LfDg6nP4RWokdc0o9ekJ\nL9V31k+ix0r7+JhEVb+6MZEBNhoM8hWy2ttO5rkyXju8Kl6zdA21Ry9Vd0h2aUT1un1WBRhLEgOA\nxvu9I31fS7mBGetrTjfAH3GyfjbtGrLmLjxImtzsvOP7X8Nrd2/C86r98P6Hh9X9ZlSksZaaucnu\nJMObxy6KY78sHtw9jgdw7fbfwoPuBdRG6t4Bb/57mU5u/e/pP/yu2ufwZqmRZEFqfF49MHm9Br/d\nDq+pPyv2y6t4K8XI+LsHD66KPPuVtgNQn4cHJs/g4/oNxvPoE4mbmNpLm/XzivfVUCAv0XYct707\n+l13zpMsPe8S5+Cs+jSHB6YrdrQN3V6Hvkxwi/sfWcmBeIRIJM578aa9qqngftreoTCZ2l0Sl1Dr\njFxl4gOJZ33F+5aq4GMeU/AS9ol49fR+eGnlKryZ7Ct4lfWSeKmrev5UK6OqH4ZXJ59P2wFR76d2\nB9vutH2OXXdLYYPxgGoNvAr+2jixquBqPbK7LRv8fjy1xw1tgV98bsQ7QJ+Aj091Wny+Mx3cSVg4\njeVNZ3NpO6jpZnjAehjtlNizDGi3yDSfwpvIFsqos/Q3nMIDD2bL2BvvbH0efmEdQBODQXYwv7Vi\nXpfQttlkJ/zOwC7VJMV5V90tOgcPBM7H+/6NwYOrO7Lvlx5DaAc8eNoFD5j3jXNoJ/zi92O6+BDn\nLA3Mxm97f7TueHwOL4jkzTQD8Saddh9C3uSyp+P9NC/B88ktY39WTe3jaaffWxzbv6o7pjfh/eYM\nD8bG59vYV394x/QX8QLpq/hNSF/G89xd2jnfDK9pPZ3aUx/OxofXeRGv0d2Gws8WjeV+By9ozonl\nPx7Hag28+eys+vRTN498EM5V8ebptfH85dLse49QN/h1i+teX5ExFr+GnZ+nVfwa0+Hzdltaj75M\nbIvzX5YxGB6hPx4n1I542+4deG3U03UZ8Wp4dfjceP9FvBlnhbr5bxzzbHksnFjmT6jVHh0dJ9ZT\n2XfGx/K6/BDaBssbSK0WaQ5+J+PNdd95hLiAZdO6+piar9O2r9TD1AZSXS0+v5UYrDP7Xn3NwFC8\ntu5Jan3OPoVfVE/HL/47UVcq6sW0NhC/AFZ3teXB1aZ4reOWHfz+U3gz2Vi80/tvIvMckS8j/o/E\na+xKNQEPpVYAmRfrcStewr8I7892An6zxrs0WftA9KuI12vggfAleCFhWJx3XepTFb87B7+47UGM\nERTn8RV4wL0U3nz1aL7fWtxH+XAQL2fp9hw8oNsDr8l7sKvblC1je7x5Zg4enAyv+7x6LM/HDzgm\nuxuxO9uTpaeN8WD6xti3vyMbdqaD+Wwc250Xyg7DRy0/lj5q/qs7ZisQNYZ4cPU03jF/bzwwvYJO\nglO8UHsPtXHpvhxp+od0Mrp5F9Y3fxrBQNoO27EsXtv2DHVD3dAgT862vRq+5sbY7u3wgvvV1Fon\nbiux/vm6xHLviOXuircInYPnIY/H+drtFpCm1qWvEt4n4S8yn88D58b72XjpsqqBWom2A15Oxy8s\nB9fNZz4eXG2BN0Vtipe2W36gJj743K9pO8zAEnifnUuIatrIFN6mxQsqfvfSrfjF+Yt4p9NqUM7t\ns++9Un8Sd2NZE/Hn4VUD0T1PFojiNYPXELeTtzOPlSMjGI33afkmUbuI1xBcHxnd0FbWtZtpK3/4\n6i14n5TB1AZuPag6dp3Ma5/Ytl3wZt7PRjo9Bc/MqwxrmcicWu5PE/ObiQdyy+JBy7ep9efYGg8g\nzoptGk7dYJAdzHdgbP8z2bS1Y/63xXm2ZDPzajDv7fAL4u3AyTFtCN6UUw1lMZx4pluL+2dM9npN\nfEymavDErfBS+DnU+t183OTdxeUMxZtLqybSt6nVyi1HrTByFV7D0kotdZVmt8MD51Oq9BlpcF+8\nIHAbHnRZg98uS22AzQ0j7Vd57HS86bhPBv+s29ad8QLrx2Mm4bU3TxK12x3sn61o+3zHp/Fa8a9m\n07rchN3OMtfFrwNL4LW7g/DKgIuz7+yGB0kLPfKqg/lei9/0MAAfkqZKX9PwguiO2Xdbbv7LXn8D\nz9d3iuXuG9NH4wWHjUstt9316evEt7j94SXY9eP15/FblU/IPv8UftE6su53y+MlwqpGxPDqyupB\nlydFgvk+3jRSTW8lk1sdD+TejRMpT5zD8Jqr+XgG/mNafPYf3gfoYfyW6gPwDrKP4jV4O+AdTm/D\na4I6Heqgg+XkzWPL4oHjZfjFcO/I8LaI/x2ORI1f0O6P10PxICYPrrbEmxRPp+6uqR5MYwvVFOBB\n0WV47c5AvGa0fsT4+pq4/M6vpfAm6qqj8A1437YqmBkZ74sMG4Fn4s/hNWPDYt89R9Y3LT77OR7g\nd1j70Gjb4nh//OgY/CJ+O+3cWdbJ/PO+J5tHOn2WrKCB1yYUubsIDybvJ5rB8ELAL4Bv163HZXF+\njuhK2sv2yfLUNUPhd7gugddS3RfLrsZdavmRNdQKUvvi/UMvw/tIVeu0Dt53a6HnGeIX+FfwWv0r\nYh9sHGn92dhHs1tdxwLbuDpeO7JavH4zyzNWwwswM2hc41MNC5IPp7Az0X8v3g/qyvHuZF23xAud\nr1K7U296HPsrIi3cRwwf08F8TsSDmSroPRnPl57Mtn0FmhidvYVtOQ0fs6p6/uZGeNPpQjWgJZe7\n0Lz7OgEuTn+RAKvbU6vbhM/AM/dVqJX8d6Tt2DdL4KXcp/H+AeOJMaIaLGMkcWFtJWFERvnxnXZx\n4rxZ951heInjB7QeVK1D3DWVTVshtveOeL8b8L9kNXaNMp5OlpOPl3IsfoFaDg8eP8IDj7vxIOL0\n7Hf1F+YheICyOd4J9uPB7fALw33UguBt6KUSMh7UVQ9bvbZuG3bCg9cvxXrW3y00htpQGfPwJp+8\nqfRyvAZudmSGeW3q7mS3chfalgPwi/Vv8cBgZzxIrZrBN8Zrnjq8mFO7IM/Bm01OiPdVh/If4E0C\nP6YbDx/O5j8T75+3BH4+X4Gf3xviNbo/p+wjY5bAS/dnxPuJ+AUwf9bjlnRzAEj8ovcy3iRzM7V+\nk1fgtVNvEk2LcT61OwJ7s/sx/o6nVts3Aq8dv5G2gf4NZLUzMW0lPKhandrAlOfF62F4oLZByTTa\nze0cj9fM/yDbp2vihdhq+Jj2RlQfG8d4YuyrtWO7lsSDxkMKrmdeWLga+CeiIIYXIlfBa6nuoW13\nlUZ9qm7ECwKb4jXbhhdk/ppsyBk8+G14l3s3t6G+T9WJeOXDPtSukxvheX9L/QG7tF59nQgXlz/a\njmu0N56hV32WLorEOZ2F75RYLU7CoXFCfhuvuaq/c2Qdmri7p8l1HY23mc+rm34f8EbdtKHUbn9v\nJZBbi9qt1Hl7/jT8Lrpp8X5W9ll3O6oPwGs/LqXWjFGNgH5yE78fg7fRrx8n5WO0rQVbEm+2eIS6\nW3l7OI2th9csTsQ7qlfjbn0p+87leCCYV7NbHMcz8WasPakN8vocHqyNxgOEy/HBP1sKpJvcnn3x\n/iYP4AWLJfGL7I8jo/5Vvh2dzGsu3jy+OX4H2/XUmsYuxQsRXd4maoWhHYB/xmu87sNrelaP9fwF\n3vzUctN8g+VPxfsgVjdGTMKD3ptanO86+AVoDN4J+qfUbuS4NI5L9bzMVptp6i9+e+GFyOrmnaof\n6hrxfmm84JPfbDAUr31+gVqNyKjYhl4dK66dbawvmM3DA42jqTXVro03s3b0XM6xcW6eF2nrVrw5\n6wi8wNPtRxO1d2zwG6k2jvzgO2T99KirKW6UFvCbnRoNwrk+3i/rS/igoN9sNd02SleRfrandufw\nQXihcQ61AVOLFXiaWre+TpCLw1/dAb4OLx0fg5f6qtL3hZHIxme/m4ZfRA7Jpo2MjPvMLKPbCK9x\nabcDchfXdxbevHJJ/Ukey361B/bNRLwZcFI1nVpNwJ3AAXW/63Zmjt+plw8ZUV1gl8drxPbPlr1Q\nR/X4fz5eW7ILXprfAC9dL4ln8mPx2oKid+R0sl0j8Yvem8CtMW1TvAq/qqW5iXY6/uI1bRfEd/Ka\nrvvInvdF7bls5Tt1tm1unozXsn01zpUVY/o0vPayqZsB8Avxt/HgfQ5eq/FDvDasOs5D65ffyTzz\nh96ui/dl2iyO+9fwZp7xeB+8a6g9QqXUY2qmUevntCLe7+j07P3ztFBLitegHI4/aeCVKh/A+wAt\nT/SHaTW91qW9I/G8bBbeZHNSrMeqeJ/LfLy8IdnrVeL7o/H89TNZGj2QLg5221NpGq+5PpRaH9q5\neE3Qkdn61je7Vr+dgQfRS1MbAHT7+GyrmE+3+tC1t77xehO8i0k1ZtmeePC6O17Ddnyj39XN70Ji\nvL768yz2yXF4vnVa9ptS41QNwFt3vo1fX76DX4cPwfPt3cj6v5Zabqfr1ZcJcnH6w5ucjiKq6fGq\n6S/inXOrmqvPZN+fgXdsrPpKDcAj+yGRsT2CV6XuhF9IFxp2vxvrmF8sNsf7BV3GwgPTPQL8pPD+\nWQ5vYniThdvYH6eFGpL6Ez4yhcvjddVJeVQco6U6mM/0yGT2jPfn47URv4tM5nn8AvBDvMQ9rZfS\nVp4R3oUH2UcQnZvxkuGvIkO8t/53tK3yrzrjP07b8Y8exZusB9IDAVXd+myBD+dRPcx1Gl5Cv4xu\n3nWKBzxVn8GBeAD8f3jA1tXm5PF4M+XwOB/fiHS7XHw+Gr/Qfxdvzh7enXXuYPk74rXWb+A3RiyB\n11T9mBhegC52XK5LQwMjHXwv0vKEmL59pO+xjX7XxeWNwAtuR+AFkL/Bg6Jn8Iv4sXgfnFci3bZ7\ntyceWLyONzHthNdGXoMXoP6e6NDfl394M/abeBD1LF6AMbym8yZq3RIa1fjMi9/ehdcg75t9NhcP\nqlvO/ztY9zl44HMKfh3aBe/n2tTzTfGg5py6aQOpDcJZP+J5seAGb1bOB1y9hbibHa+cKNZ02qX1\n6usEuSj/4dH9rpHRbo93kp5PrVlwRJxQd5DVbMQJdyLwUTbtWdr2uxqPV/m/T12TXTfXdQjecX4t\n/EK8Hx5c/RWNg6t1W1ze1tnrU6nVqNyF9z04Cr+APAzc0sJy8ia6AbGd1W3peVPCvWQjt9ef3Hig\n+xpe0lklm34cfgHdIo7JFLzGoOgjEDrbPmpNUqvjJf4r8P5i1UVxJFmnbBYe/HPHSIeD8BqYiyLj\nyYclaOmYN7k91QXoNLyJ9VQ8cFgZz8yvxoOijh6NUW3TrNiuqrZlJh4cjsGbum6je8/JnB7pYVn8\nYr4cHoDkYyaNwe9YLNKnJ9umIXgwOCO24RY8EBkW6e6v6eIt9tm8d8YLejfjAeFReFPaHnhNy88o\nNEI8nvfth+ctz9B2UNYzqfUTXIFa0159ASl/uPSF1J4ntyUewHydwk1j3dzW8Xhz9mS8Nu1HeH5z\nOZ7X70g7tYtx3r4W+2UI3hz/o5jPUnjQWbxZHg9O85HOZ+MtByeS1TTGZx0GQvHb61h4KIYl4xzM\nb/BotbatPt8+gBhTK0vnD9DkUxl6LE30daJcVP8iIX0Xv1j9MhLRYXgwtBa1C+JStB1GYFL22WX4\nmCvPER1U65YxnAjICiTIEXGCv4AHa1XTxUb4IxXaBFetLA9vNruHts91y6uUj4hlXgecl01vpaP6\nw/iF+Sq8On0/vN/GN+KY3NnBfIbHdxo+MgcPlu/GA9KWxyXqwvbl47J8Ew9CrsJLglUp88v1mQie\nmee1VHPwmrdts2kr4xer+TQY8bmHtmcy3p9uBbyfzc/wfiRn4heVVWmyFhAvJPxT/P5fqm3AazLu\nxYfZqPoINd38R61QNBzvT3VunDvj8RrmfMykQc3Mtwv7Zxe8puN5ak3mm+CB0Bl4cNWtW+xjf72K\n1w6+SO0BukfhhavrqTU9Fbn44Z3N98Frm67PPt8Dry1rt6YP73t6WaTxgXjBZj51BaneSLdNbO8I\nvLlyPbx/4hS8purvyW4OafC7aZHmH6Lt6PNHUGv2rbqDtHpM6vu5bYjnj3nz3Hz8mrZzNq1RR/Wj\nI/+o+q9OiWM1H5iTfa/oIJy07XKzQpyTU/G+j1tm33uetkMq9PrgsH2eKBfFvzjZn8ve31ydQJFJ\nPYjXCgyq+90ovFnpKWrt5efg45PkfQo2wS8OYwqv97r4Rehx2t7xtQFe+r6QMo/GWRq/cC7An1R/\nKC8y0dEAACAASURBVNmtw9n38uaJbmWSeNDxFF57sy3e5PAk3pw6DR8v5zMdLScyxsep9Wlp9Oig\n6qGw7TYl9lBas9i+E/Fa0X+kNkjjHLxEOKfuN8vF/h8U++divK/DsJj+VOyX1SItNz2SeYF0MR0P\nUH+MX1QOwJs25zebAeK1dtdRu4Pp2EjX0/Cgc1O6OAYa3m9ua7xWaGf84r8R3oxzaqSR5fHhU87v\ngX2zNt70dwTeFP8CtVvGN8drE5rugItfdPKLzVdpO1ZZldYH5f8Lbs+qkZ8Mw2v1bwa+GJ/NwAOr\ncdn3l4pjMBlvARiE50u3xzE4G3+247HZb/p6NPUN8VrTal9uBVwUr7fHa9TWr/tNlbdsgAe4o/C8\n/pHsO4fgNZUDKDPAbB6QzIt0Pgyv4f0uUViI/XxME+n0I/wa9xS1Avp4vGvJk3hg8wjRD7TksYp9\n8jRe4/7TONd3wisLzo3PbujLdJGSAqvuHNhReG3M2dTuatmZtoOpnYNXsa9Y99vRkej+hDeJVcHV\nZXgfmWr4/7cp1KaenchV6WIC/oiKm6j171o2tuEyWrilnralyaH4Rfwa/HEY/4gHbzfjJZmZ9evY\nheV8nujYHJnDgXip9gW8Bue8OMHrhxuob2pYhtotuQ8QTWnULjYD8Ivpp+N9bwUgeUA3CjgzXj9H\n7UGz1WjOCw3YigdNUyO9LYn3RfkADwzPwJvh3ozPijxjrIvbNw+4MF7vgNcqdjrwLLWHp16OByEH\nUrtofBG/MWH9bq7TQLz25lmyfjt4LcSteH+gpfALSJEBUrNlz8QDiLOq9IfXTD5NPLuRuuaZJrbl\n03gQW6Xv4+P4P0stENgLvxgNpsULeINzawRxYw6er+2FB1MvxXmaP89wNbwm81r8gvwA3gRf5Y9b\nxe/fjv3U9L7owTS8GT5MSJWfzcZbI36FF2R+Szt9v2J7bqQ21M3A2OaXIg2+S4OCaIvrOwAPqO+M\n43IBXou5Kt68/AJwT3vHM5s+Gq9hOwTPV57E+6JWhZxhxA0K+bILbscdeCF6ND68SdWkvC6erxzU\n2Tb0Svro6wS6KP1RC1JWwqvOT42T5CUWHi19j3bmsQFeS/BiJMrqIn4J8D+RWJq6xbwL671rZKi3\nUHug8zF4cHUeXlszJN6f2s1l5M1Wz+AX92XwfmjVwKBrxfR9WtiWBZH5rhjLGoj30TmUWof1feOY\nnNTBfEbgJeov4hn/OXiH4SXi8+q4bBHr363Hd3Rj+/LS5Ur4xfwneJPxgdn3ricb3bs+E4l9fw1t\nn2+4crweH+lvfE9sQxPbuCbw5zgP3qeTi0h23lU1OMPxgODrxEjtMf24zubVSdqdiDeXPYTXilVp\nYV38zsnT8+NTcH+sil8w7if6BeLB1Y14MDKQLgY+cT6PjnN6B7zw9LfUgrct8Lxm+wLrn98UMwoY\nHa+3woMMi2O2T6xPVSgyvAntLdqOXbcEXgg7kay5EM8/XqCLz0IseJyqdDgaD+qrmuP98OE21sbz\npU/TtrZwIl5TWPWH3At/JuNpdfM/Dq/hL9JvLPbXkHj9WWKoA7zp7mjiIcp4AWtq/XZ2MN+94pgZ\nXlD/ET4I54VkgU2J86T+9/g4ddPwa8yJMW0GdbW59HEzcZ8teFH7wzuQXkqUtCJxLsBLURdm36sf\n92M63tF4VfxityReazARHwTvGWoX8bPJSnKF1nsqHlTtEZnBe9T6UuyBlxSrGpl76cao1NmyDC8V\nX5ZNWzJOxGuBo+u+39U+VecBD7fz2X54ADQcvwi2ezcItbFNPofXDBwY76/F+/1MxC9EG8Tx7fEx\nnar9V+0XvAR7aLzfEw+uPoVfdL4J3N7e77P3e+LB1ZHULnafw6vQGwb+pbelwfQqiFkb75PYVO0P\nXt3/g0gDO+CBw8Xxt2kzy+5kn2+BB+TLxfl5NbVCyAB8nJ8iQ2tky9wEDz7Wx0v6N+JDOXxcc0pr\nQyqMxwsON+C1CCviN3XciQfWJW6KGY3XsFTNsPfiNZBz8KDpGWr9RJemLpjHmyir4L9qtq46qZ9O\nXeEo5n1iT6bdJtLhQ5Ev7BPTRuBDuHyPhZvlp+MFtgfxvHfFmL4n3je35cC2nfW8lbYPO9471q/K\n+6bgecpGjdJmJ/MejAdRG+DXtqfiWJ8OHN4D25L3F70B7zpzQDbtAeoCur7+6/MVWBT+4mA+hFfT\nPkPtdvzJeM3BV2lwm3hkjFfibdJ34s1/E/DOujfGd+6KBJ8PmlmqPXom3pwwP5u2B94PaedqHbPP\nWnreHTGwZva+KvEvExlPt299xYO2S6g9cmUbvPTyELVS8cPx/v729mWsy4XA5+L9nnh/nYPxjP0i\nvCbxWbw5d7dG8+nh9HYX2a3OeLPqp/EO6HeRdQhtsH3b4qXRzeP9HGrB1dIxn3mNflto3RuOD1b3\nnQH136GDIBuvPboD7491DN4ctDu1O+guJ5rMurnOu+IXyqqz+xJ4cHUNfrH4W8oNzlvtn2pA04Pw\n2ost8ULQN/DgseVHx8RylsebbG7Bg6tBeBA3KV+fbs57On7n4iV4U/9yeIHk03hT7eH4ndJ30k7f\nxDiuO8Trx/GayDfxvqpLUXvO58BIv3dT4EHw3dzeDfEgYis83/+4IIoHVweS3SWKF6Z/Auwf76/F\nA91qwNBd8by46FAKcU7cl70fiQfZl8c6VkPQPEInj6jC+zDthhdm8kfsHA/8G/BsNm1Y9rrVmqqv\nkN0tTjaGVxyD+/GC5kMUfJBzsWPQ1yvQ3//wC2/+MNd98Dbqj0d0jczwcqKqt+73q+KB2Tl4jdft\neBvxb6g1zTxE4du24/UA/OLwbKxndUH7TGR4y9G26amrfZ3qb32diDft5HeGDMH7bw1otI5dXN5J\nsd8uxquib8QvGj+O/Wu0fXBto47qS+PjhV1JLUCugqsD4/1w/ILU8DbwHkhjbZqXIj1UaSO/qWEJ\n2gbgA+p+tw4efN2E9/04ldpYOjeT1Rj25Dbh/U1ui3Ol4eNWsm0e1Og4Zd+bBPw38LV4vzxeA3pb\n/B9Ca7Ws1WjeVaCxHrWg+7N40LBLd+dfl+6qi8Mo/OKwIn7Reo1aM9FKeBDUdH8+Ohn8FL+oHkHW\nr7LA9iwb52I1jMIx+F2aVf+taXggciteUBnXznwux/OhMbQdj+g7eKHwGtoOuzC4xPp3Y3vH4YHf\nA3Xrfje1EePzPG4wnsc8Qq1F4u2Yx4tETRxe0/4PFLpRCS80vp+9PxkP6AZHer4Sr/l9mKwA2s68\nronzbD6ezz6EVyJUgeH91Gobi94tjV+vnqFt60eeF16Fj7t1bjatX9wlmpICq2YO8ChqQ+UPjvfP\n0Pb22JVo20dgmbp5rIFf2A7FSykbR4JtamTpLqxr/mT0fYmHkcYJckOsZ/Wd5VpcVh6QzaXWkf8z\nkSHujJcy76PQOFXx/qjYlu2oBT6bkN08kO+L/JhQG1BzBN758jpgr5i2J978c1T98evh9JX3TbsJ\nr16/Hq9ZzDOS44iLfzvb9ym8iXDDeL9NZKinUBtLp9MO4gW2Z128BvacOE7nUzecQ5Z2RuIXp4YX\nFbyJbDhee/Gv1Jozx+JNmve0ko7xZupBeJBza+yv2/FahKq/3kK1a91YzjJ4oWC57Py7NNLai0Tw\niTfXTKALNcd4fnQXtZrc9oKrFfCL47UUGNgWD6wux/tBPR3TzsE7ba9a993byS6AdZ/dhF+sB0S6\nqZrXdozfHdgb6baddcsLqYPwwORdsmfdxbn6AA2GjsDz/Rvw5t0XiHEK8ebEh6nVkC5UIG9hnffA\nmx3n4rXwL9K2/9SYyCvmNdrObNoFZN0u4lwdiBdKvxrT9ort65H8Em8RerY6F+s+24G2gWy/CapS\nUmDV0UFdh3bG98Hvzhoer/eg7d1wM4C/q89I8I6EN+MXupaCmk7Weyd83JiD8WroQyLTWoCXrko+\nJHYAfrvuN/ABFI+O7d8j9sF9tPBsKLKLGt75c3Q733uIusCq7vMJkeG/gjchVaXsL+AX/ioz/ywe\nbE3q7jq3sK0XUxtV+wg8aDgA74v0INmDd9v5/Y54k3P1FPkl8AD7FhqMkdZD2zAD76NW3W26Id4J\n+TxqDyWvgqpl8IvNVu3MayB+oT4y3l+Fl+yr0c/H0cLQIHgz1rl4zcrK+AV+6/hsHbzvSMtDEFDr\ncD8ar9E9IN5fHceretTJhnjN6zpdmPeA+DslzoGPO4W38/1RePBdqnP0DXgz5hnZtDPx2vBVs2lf\niDyiOvZ5fjkx0v40/O7Ap/Bawp/gBcF98b54vXqHF7UAeIs4H+dQG/n9IdrefZYPRLwiHvRXx2L1\nOAefJQug8MJuNWhyiSEVToo8YOVY1xfxoX2Wrvve1rQNGNtLK5dm59o+ePPszfH+WrwJezLZo7EK\nbEOb8bGy/fkd4Kps2kt0MD5Yf/jr8xXoj394EPIofrdD3p9lKF7b8XSc9PcCV2afLxOJ4HK8Vqs+\nuKpKMGdTe+hwybuLhuIX5GXx0u/LZH0p8Kazljrg0jbYWUDtQv5L/M6YE6jdBTS8/nfdXM4TeHPJ\n03gpqaq52BjvLH9jE/N7DL+QHR/zuQ7P0M+OY1L1O2p5HK8mty+/uByK90k5Lpv2eXz4i/tp2wm1\nfkT1pakNpbEzPpTHLll62IZeKvHjNUmvAd/Ppq2PNyVcTDZ0AB5Udda/4yBiIMt4fxFec9VywQTv\nf3gDHghMyabPw5tsSjT/LRXHsBoo9yC8prrqH3YXXjA5E2/abnqZcY7fgAdWo+K8e5R2gitqzVF3\n59vb4vYdgQeg5xE3wMT08/FxxYbh+eWXiaFPqAVXA4g75/ACzfH4RX8o3t9sGh7U/JheGuakwfZt\njedr50Sa+Eqk553xwvUhdd+fjveb+wbeYX2nmF4FVyfjBYL14/MO038X1vNGvNZss9h/g/Aaq8eB\nz2bf+zbZ0wPamZfhwcwb+Pk8Gw/S1sfz4QvwAtQ+9b9rcRvy8bGejH1Z9QVbEb+eXorfbHVXK8vq\nlbTT1yvQ3/7wkvG34/VkPDqvf7bdw3gEnwdV1YVuJzxDnR4JslFwdRvZY1MKrXd1W+1tkfi/W2VI\neEbe8iNYqAU7A/HAcgs8mHwWL5VujdcMnUV28evuSRcn+WX8//bOO9ySqkr7v9XdhG5yN7SAkhSE\nNhAVGkFBCULTgE1QUII6kmYGBkmCJBFoFJDkAwgoCogINIogGWlEcAgjiug4hhnHMDA6oKOjHyjg\n+v54V3H2qT7n3nPvqXDu6VrPU8+9Vadqh6q991p7hXe1wmoPju/xPqTSficByJe2L1dG6px/My2z\nxfaIITyOhJEfUDAga4/924iWueuzJFFy8Z6ndXj/2VjbHQleX6XFpOag0Oe903tLavvLZmVg9fg/\nS4icJsF+UzIWl0QOy2k/p9PyzXk94ewb5zfQjtp/Hn0kCEZQDzOSui6M8frGGFNfoaVx65dZLIeE\nqYsJBod8alIT9L5Iw7v1WOtEDG4dWsmzOwpXybhZLn5fabzfPP5miZTfktR7Pon/Fu0aq5fXjewc\nrRmZn9EUFHRxXXzfLyHz2TeoKHVUh76ujzbOeyTj5lICQBM56KdYTashPKgMV+ko5OOYfZt1kCD8\nWeQH2XdEZpR7BvD1DtenIg3WAqTx+ywJ+v0oZS6d3RtlpNkGHkfKg7NI1qYC+tEJH+tMWhveNZGw\nd3t+XA3iUXsDBulAO780n9zpSK19He2qyHtojzybRAfnbFrC1ZlxvkYM2qkFtTerZxathfvvkHo+\ns99vHRO+L+f4pC6L93FinK+XG+wLyWF69VHn5igi65Lk2gfQjvBgOjhyJ+cZkvNawKuS698gknTG\n+WviHY2bWffRv+3RznUbtLM/HWl3FtnJsqgGYkekUV0N7fJ+RItZ7w48hwSeUhcfZKJ9FO1mz4tr\n05FA9NUuz6ya/L8kcqg9HQk6uyMz0HnIFPpuEsGq2/sYoX1rElrnpK4FtISrN8QcvSz+X2Ys5Y9Q\nbyZMzETC0+eSOXoAMgXuRw6epYdyX0W778sFiNll/TkaMagt4jzTVK2EwInfNt4+RTlzkRbp8Bi7\nO0TZRyHBI4NumUS7ySkVqm5D2p9lok1rZ22lhQ6/JiW6TPTQz/cjTLMraIGsboE0ix0BSmmByhoS\nnr6OTJsnog3o69DGtzDwT+Ssnn3rfLLjFdHm8+e0g3+OuiagDca+MZdvJRJDxzzZD60/hfmGRdl5\nfKxH0SZxfnyPEQOTBumovQGDdtCK3FkfObS+ATG9Rwg/F5LQa7RDzxj4ImakmEx3osX8+xQUKkxL\n0HkHEhaeQir1NVBI70/RruJJCkqsGvVdnZukSyI17bVII/TpfBvHUPYivgZI7b4QOCS59o+MjFOV\nR3K+gcjcHr/fBdzX7Z2WOLbywt+0WLzuR+aPFWjlUOxo/kj6cBQyhWYpSo6Nb5758HSMwiqgD2+g\nJSBsG2N6tfgmL2bfH2l/bibxGer2fpHD+xXIEXq1mE97IV+bXwD/Sx/MCPlMZYmaV0HAolfRim46\nFmnZCvM/TPr1cSRMvAdpDdIN0KWMgzmhtSgNcz8XOX5n2pETkKC7YpyvGGOsL9MTrcCdmUg4/DaB\nS4U2Mm2wMySRn9nYjfl4XIz9u2m5EtQL6NhaT9egJVzPRc7pB0fb14+5NiP3bNumOublhXE+G639\nO8f5tLS+Atr9NeDj+fYgIXUzJBh1BRLudj3G7slo0/4axGOuR1qkzOXkVUX0IalzJHysQ0brwyAd\ntTdgEA46OKnGxE/NMG8iwRaKa50Y+DEsmh38/SjdxryC2/0WtGt8M/IVug4x6kmIUb+VURxae6gj\nLwycgsLfU7yWdZDqPnViHZdQFW2/KCZThrW1Sywgh/VQTjck51ORg2cmmHybxBeowrFmSH2fCfDT\naIH3vTkWwj1zz6Tat1TbsyLaTWYRmXfEYjRzvN9hlLbPQMLb3tGP9WJezEEOpW+M369GZswRnb9p\n12i8GvmhnEp79OOOMbbPRkJ8zwyYdj+2+4FH4v+VkXC1EAk8jwBblfCtV0Ebmx3j++2LGPWB8fuY\nhCrazdrfABYm5+fFtUxYzATJKYhZbVNAf1ZEGotDkZYv8xPdFWkYUgFjZeSD+cqkHbcg7c00kvRM\nRY/TPvo3F+FxXUsrKnQvtBG7P8bL3OT+dF521a7F2P37+L9QARJppC4llxsTmQKvIoEi6VZ3bh4e\njnjHSkh4PglpkSchQW0m0jhu22e7a8HHqmws1d2AQTkQIzgyFolFosKQL0sKtDkSAz+alqPuWrFw\nzytqYNDaXR1GknAS2af/Ffkg9W3/pp0xvWxWQk6Y36NLBNN4Fw9aZoKTkMnvL7TMW7ug3eKIJjtG\nR3JOs7lvPp52jqdfufNvIYE8xTVagML8U98Uo7Pwfixa1A35K30UmVBuIcnqXkI/VkXakUORRmQF\nxFiupBVZ+VFkehjRhzAZwxsgwWk6Eq6uRMw3ZQivj373BENAd0H0dtqTp5+GwCsL8XdJyl0q+bb7\nE6bM6OMB0cee0wnR3ax9F3B/cn5JzJElcs/361O1cnLtIuSTmPnEbRVrwRtyz66JhNdrCJgJtNGb\nFm08Il9PHUfSx2lo4/U2pDX9CnB5/LZLfLNUEOy2qV4pV/5stPktJA0Pucg5hKB+PnIhSPEDbyQJ\nvBqt/0nZD8V4e1VceyPa1F6Ooqo/xRgTnHeoszZ8rMrGVd0NqLXzMrucE//fhFTTFyAtQLZwrB2T\n5srcs6Mx8OOSezONwphBOJMyVke7kFTY2RIxhlTVexPSXBUV/TcJ+UFcjmzeWZThPyHfjsKwuNCO\n/mTke3E3Ugc/T0u42qiHMkZFcqaAMPox9CnF+3pDcv7FWFCyoINPkzhtx7WRhPdjEaPeKcr6ERWk\n3kFapT/TzmSOj8XyQORX0lMi5Gj7bUhLcCbaDWdOvqfRQrXejgC07aHM0czAd5OYgWn5zxSS0wyZ\n/m9GkXCbxPkTtJJmr8jYhKpezNrp7r5vxHakfZoT/89B2t0vIC3Dq9Ea+WD0sWsEJYpIvR4JJZmZ\n8v0MiFCVtGEHtL7dQCtX43S0ln4xzvdDfnLvQ6DPI26qkVlrWwQ7U5Sjeho5d2f2rZEgeDLSWN8X\n4+QLI71jcrwIaaoWIreXj5D4ocbvk5EGtt95Ujs+ViVjqu4G1Np5LeLfR1EoJ8S1GbFw3YaEqlVJ\nTFDJgtYLAy9E7Yt2Uz9Hmo5P0QLGnI4Y8olIXbsZUrF/nj5AOZN6J9OCUFgaMdRHkgl9El2A/3os\nfycUsfie5NpUpNrOHP4vicVks+SekdKljAnJuaJxNgkJUfegSKPj4/pVcf0RkrQMtJj0SML7KbQL\nN2umz5bYl4OR1uI85Hw/CTn1noR2uz1BBiBctx8jYfM9tKLzVkImxpcjZ+Pauj2U2asZ+F7aNT1F\n+btsizQ4e6BIrB/H+H4SafLGJNCPoT//TEFm7fiee8a3PAIx6bcjB+JPR3+mIu34e2hFo+Z9dHZD\nwRXnx7z7EomJOqurzLHaY383Q0LiidHnw2ltHmegjcJr0Vr4PrS2jLapzgSEdWnhtxVhqegUOTef\niLCN77IF7dGKnSKlV0h/QwLyvUioOhrxs0zjWjSieuX4WLWMq7obUEun27U+qyLGloXhW0yok+L6\nism9qQ9BZQwc+Zach3ZMR6GF+kOIKa0YE/06pKrdGNmtLx7PpEDC5Frx/9JIC5GBOe5Hy6F43Mlh\no+zPI83ggij7e7QExtNoOYzOJ3Fc7KHcgUNyjnd2YYytrWMBycABtyYcW7Pxl/w/kvC+fIyJsgWp\nTMibjcwkW8b58cjfJoVNWDbfhw7lZYv5XODW5Po2Md/OivGW5ZnsWRhhbGbgrwHz+3w3qwMXJ+fn\n0I5ovSlixguBe8dRfi1mbbSR24MEIy7G7lEomnEeI5hlkT/VNbQAV9dHZp8raTHVQdBUvRaZ37OA\ngu2RQP+PtNbATv63veY3LFxwZNHIucdQ5NwnySUi7vSOkdB8J62gg9mIZy2HBKw7aAUbFCZUURM+\nVl3HJBYzMrPJ7v6SmU0ys8MRNME8YDUz+5iLnkVOpme5+/9mz7r735KilkP+Er8H1jCzfeL6pxCW\n06NI8u6b3P2vSAjZHu389kHRGvch58rH0AK+PRq8n0T29ZfGUo+ZXYkEsi+a2f7u/ry7X4UEtf9x\n9y8SmjikPs+eszHWczGavLu6+17u/naEI3WnmU1Fi91WCEF+DXe/LJ7rOF7NbHJyeioyKa2LhLP3\nm9m5aGE/FQURzB5rm8dCufaABON/dq0U30GmnY3NbAl3f9Dd74jnJsU9GT0LPGFmM4Dfu/vJ7r4p\nWgxvRH43y5XVDwB3dzPbDTHUrYCzzWwecsj9JXCQmW0X7/PP2TNpGWa2nJktZWZroW8DEqJeMLM9\n4plvIo3BmkjQfN7MzN1fHENzN0QmUdB8eR54p5mdgTZC05Nvcyua81PGUH6e/gJsYGafj/PJSLsG\ngLs/7u7Xxvh+ycz+YYzl99KfKVHXo330AzObambvjNN10By/AdjOzPaMdfE8Ihco2tB1KmdzJLDM\njPYD/Ayth5sDF5jZkvkxUjXFWvJrFGxxXMy9e5HJc1vgXWa2dJfxN9K8XAAsbWbL5/hFUXQz2ri/\nCUVXPove65/QOvMydZiHk5F28ztoHq+M5uGuaKO+G3IneDHjk0U1OtryG+Axd3+GcDFx9++4+xwk\naF0EbGRm07r1YaLQYidYZUIVMsusBbzg7k8hjcZcMzsz7nvG3W+GluBQBwPPhAl3X4i0MVsjYW5n\n5BewNZoUa7v7n5AfxAHu/v0x1nMxYgz7IZX/qma2VPz8c2Cyme2JGOq57n5B9uxYBr+ZHYpMq/vH\n+TJRxn4I5fhyd78W7f6OTO6zTgtVTlB+m7v/GuHszEW+B7sjc8beCAPoOOCbZU3YXHv2TBazI81s\nY3d/Di1mq0V7XqYO/atMeM/1YWoy5pdHpoftUNLdKcDD0db5iGn+JhjvIu/UzNZHWoDz0bi6yMyO\niXLuB7Yws5PNbBs0ln8M7G1mU8bxjUZleMh/DzSmzx2j4NZGsQF7NzDDzC5BjPoZM1vdzFYws7WC\neYG0VtNL6M+0kQoYI+1oZg8h4eJ7BF4dWtvmAbj7J1AQz2/yD5vZLMQcf4tgJnYNoewl4Glkcpsf\nG8XayMw2RRpzR5vqJ4BbYswtRJvq+939+S5FVDIvzewtZvYuM9vRzHYEcPcXoo67kT/YTu7+72gs\nXx7PLcJzEkHpv9EGaG20ls+MNekZYIcyhKqM4n3ONLN90Rq4afwPUg58Ac2TlTqXMIGoCLXXRDlo\nmTU+RiviY21k130r2oU9RS7cPe6rLBUDXcJ4kXbqbgT4eWRcW4ac78I46jsU+FtyfhHSIC1E/mbT\nkHPhl0l8txinmjb6cEdynpl9Nkeq6DzQXbcw4YFEcqaVQ3F+0sZDELP6INJYXTpSn+L/yvOoIeZ/\nES2n7mWQCfUU4AFagRg7k+C5dSmrm4/QafE+1kXa0C/GO9kMCVdfYRxRrVRgBu40N+OdXY98AR+O\n/mRpmF5Dy2w7prFXRX9y9W2HhLm7kmszYpzdB7x7hGc3RmtnBiuwPBJa/gNtAP+LxORd54E2kPch\nXLGpMSavQOtdR9Nz1fOSEiLnYhw9FG09AgmQ15Fg3vVa1hj6UTs+Vi1jrO4GVNLJ3GBBquwvIfX6\npfFhb40FcpG0JlTIwOkeXp+F115HEjKee7afiXw3UjMfghbxjVGo7dPAwfny+6krnr+HBIcnrm0Q\n73BUZPrcNxkIJGdagvunaflZrILU3FsgP6KTaI8Y7YZOXVsetXhXa9FKs/IRxDSz820QrMeIkaeM\n7CN0Ku3O90uhsPbH6SH6M//O4v9SGV6HuXljjLtlaeXuSyOe0qS7vUJFVNmfVEicjgSMt8QcrCcG\ncwAAF3VJREFUTP1G10M+VyNGe8bc+7fctSz11SZlzbtR2pRmZ3g1IYiiteFWJKAuGX2/ig5RzlXP\nSwqMnMu1/ZXAguS3tYlAJyIlVcHvvnJ8rEE5am9A6R1sD3ffC4WsborsyofHhJpKkh0+7u+U36pU\nBs7IUUDHxQIwC6lM16Zg50haiYpTgMk5yIyTLvhjBf9MweBS5947kFkuO/8KORDWLt9yoJCc83Uh\nVO0rEQO+Mt7rbSwKHJsGQ9SufUvasAbSKn0rxvWsGIOPRt3/Sg9o/ozufP8p2hffg0nwq8bQ3iq0\nyN3m5sdQJNWySDhZSORJjG83ltx/VfYnLyTehHD8lkIBPQuQ0PGm+PvK3PNpOq1tiHyq8Z3vr2ru\njdLHGUgwnRbf58IYexmkgiEh9Xa0pneKoKt8XlJQ5BztvO90tFH+Be1p245BWtZDC373leNjDdJR\newMq6aQmxQPI4fN+5NydpRaYQg6nipoYOKNHAR0TC9/d9KmpoLuwcxvtpoBbCeiDcdbTSaV9PK10\nG3fFYnQ1CaAdi6qQBxLJObd4bYt8p2ajCJsTEZNaGWlFO6q4GQDtGy1GuTkSqJZADP3WqHfZGC+7\n00q+O+I7psTIWSpmeKPMzVOS+lcBNh3k/jC6kLhMfO8bUABJRyEamfoeR5q0LxGwNGiOP1zmvOux\nnzORlmrNmJezkMbvxOzdoaCfn9JBoK96XlJC5FyUeQRwfpzPib4cGOfXkosm7POd14KPNWhH7Q0o\nrWPt2oCjaalQf5gMqploR5Dmt6uNgTP6Dj+bHH0BpzE2YefiPuoZSaWdmsMeIEnYS+ed48AiOdMO\nonpHLGRp+pFryeGKMYDaN+Rj8xUSLCokUNzCOFKiUJKPEDUIoj3Mza6a1kHrDz0KiXGeQaDkNzrL\no83CRnG+A/LN2yHOv0kkB676IAf7gbSvdyDhan0kXF0c7/le2i0Vtc5LJOB+JpknpyW/PR7tPYtR\nfBCTvp+JrBBbxfn0KPenyPXjxvwzfbS9NnysQTtqb0ApnWpNjiVigdqFVm6wLL3EKogxL5s8N4ka\nGTij7/AvoX+hqjBhp4e6RlNpH9Hhma71MEBIzsnCZWjhzoIJ/pMAPI1veTjt2lBjcLVvO8ci/JHc\n9Y8ihrJiD2WU4iNE/QyvUO1bnf2hNxPtSPNwFoLfeIyW391UpPE6r4qxOkLb1kcC6XnRj8nIt+hE\nZN5cPc4PQpuINPffQMzLaNe+SAi6Fdg3rl+GorZ3pEueSToILFHGY7lrywPTk/O+xhg14WMN6lF7\nAwrvUDvD+3osVFuhkPezk/tuJ8n9lyujFgZONVFNhQo73b4Bvam09+707bqUOTBIzsgcmwoQRyCz\n2Z20gD9nIga0cr59DIj2LZkry9MKjtgV4TPtmrt3nR7KK8VHiAFgeEXOzbr7wziERNrNOxvGHL4A\nRZZtEtd3Qf6f06qai7k2rodMnIeiNfwaWlqUFdAGaEE2lomAgmQe1DIv82Uxzsg5WuvLJKSN+yda\nGsVbaE/lVGQg0mQksJ4Z727leHdTkVC1kMVIqHIfMsGKJPErcrbN8jxNQbgqn44B+jWSXEq5Mipl\n4JS0w+9QT+HCzij1FaLSTr7fQCA5IwfYWxAGzNvj2rsQPMXpyX0vaxw6tY+atW+0mMnuKLrtq7SE\noTko7H4R2JHRxjEl+AhRH8MrS/tWq2DNOIVEFNl6SPx/DtJaHYDSgp2GtLW1QCrEODuF8BdC5qcn\nkV/P0SjwZyriCzcBr+1STqXzknYBZ9yRc7SEKkO5HS9F0X6X0zL73g48UfB7T+f9RxFPSdfl/VnM\nhCr3IRKsYkHIGPn6SG36MyIkHJkEN0E76L2S51JfrEoZOCXt8EeorzBhp8f6xq3STsrYHAl8d2WL\nG9oh7YGEmeuIRMYVjLGLYyzMRWaPs2gJrJcgYXw+ElI+N0I5A6F9i/f/GPI9uQmhfGcJr3cHnkOL\n+WhO6qX7CFE9wyt1btbVn/h/zEIicqu4BPhjvIfdUILhfZDA9T4iqKGug5aP6BQkPF4T7fxMzMlM\ns3Ic4XObe77qTXXhkXNIALsiGbtbI7yu9eLaCSW898rxsQb9qL0BBX3Yi2lPYjsJ7QTnI3X167oN\niOT/Shk4Je7wR6m3b2FnhLILBYOL5x5GGrat4j3tGb9tGRO4dMDPqC8PoroxEkZmxPmySAt4NEno\ncn4hZgC0b7R2t0chX4jdYmE8Fjm1HhC/rzJCGZX5CFGTFrmsuVlzf8YsJCL4jVVQhN1jKBjjXTE3\n76EEDKQx9O1VSMO6Ae0BI1sn/6+JgnUy099RaB2qZVNNgZFzuXKOQJGcv6I9af21wAe7PVfAmKoU\nH2siHLU3oO8OLMrwjiZ8qRBe1SkoWqWr/wMVM3Aq2OEndVWCfNthoegLDI4BRHJGwsLd2TeMNjyA\nkLY/mH275P68UFWb9o12MMhVk+srIgE7Q1S/A2kwUiyz2qAvqFeLXPjcrLk/YxYSkfnsDCSYbI38\nq86Jdh8fc2DdMsZsD32bhbSsVyNhcO8u970DmahWjfewPe2CY2XzkgIj51h0fdkpvuHHkNA+O67f\nRLGQCrXhY02Uo/YGFPSh70aq3kMRTtW6yW8bIRXl/l2erYSBU+EOP6mzEuRbSlBpx3ODiOT8DbRr\nfyDG2/bAexDez0EjPFeb9o3uaP7LxeJ4DfKP2DJ+nz1KeZX4CFERw6tqblbVnw796ktIRI7f26Pk\n6J+MeTwrGwtljNke+vYK5C96YJzvi6B02iLdkBb5CWBOl3Iqm5eUFDmHrA1HIuHsOmTB2Qf5vN0C\n3FDC+68UH2uiHbU3oMAP3YYaTpIWZbTJT8kMnBqigChJ2MnXQcEqbQYIyZnuIKrXA0/l7l12hHJq\n074xMpr/scgkvBPSuv2IXDTgCOWW6iNERQyvqrlZYX9KExKRqfB8ItFwWl/VR4zbD9Bu/lsArJ/7\ntguyuZv/dlXOSwqMnKN9zZ2OBPVfocwih6EE45sjDd1rk3v73rDTWqcrw8eaiEftDRjnx+3G8L5O\nwszJ7f5IBAEqZOBUGAVEgcLOKPUUDgbHACE50x1EdaX4/W6S8OXRvh81ad8YHQwyFRrW7GUMUrKP\nENUyvNLnZlX9oQIhMcp5M+GrVdcRYznTuGZr0D3ZeyR8e4BlRuprFfMSyomcQ4LU2sj37UFkst0H\n+B3SXK2Q3FuIT1XuWun4WBP1qL0B4/jAo6GG30kHhpcro3IGTgVRQFSEfEsJKm0GCMmZ3kFUH6dD\nlA0DpH1jdDDI88Yy7qjIR4gKBdGK5mYVDLxSGIciyhhjfamfYBpxlq01C5Bbw4bIJWTtTu2tY15S\ncORcfN+PR9vnIiXDETE/P0qC2VhE25M+VIaPNZGP2hswxg/cF8OL3ypn4FQQBURFyLeUAAbHgCE5\nMw4Q1Q5lDIT2jQIRwynJR4h6GV7hc7Pm/gxMdoKC+9XJT/AY2rUyJwNXIJPrvC7lVDYvodzIuSh3\nNvI1uy7m9Aq5e/rdENSCjzXRj9obMMaP3BfDowYGTgU7fCpCvqVAlTbtO5uBQHKmIBBVBkv7Vghi\nOCX7CFGPFrm0uVlTfwYCH62Efo3kJ3gULfP8pcjvJwPurW1eUmHkXKxZRwN/AE5OrhcZXVo5PtZE\nPiYxAchEa6Hd30tm9k7kkH0QsKqZnYUEpqc7PGfJpSWA/0Mq2f3MbBN3fw4tRtPNbJqZFfZOzGxz\n5PA5EwkPINDSR9Hu/wIzW9JjZI6zjsnu/hLw38Cf0U7obLSgPgc8gxaOF5N7x0Xu/lK8n28hZ8Ub\ngb9GP1Zx92t6rcfd3cy2MLND3P37iMlNQbuiq8zsNKRNud7d/5+7/2287e6V4jv8BvkNPEMkV3b3\n77j7HCRoXQRsambTcs8BYGazEH7aq5G2EOT/8DskzODu27j7I2X1w8wmJ6enovG3LnAw8H4zOxcJ\nEKei7zc7N0/y5W1MJOWO9/Ik2sicY2aXIxPM19z9yXG2d3m0UfqAu++EdsSzzGwHd58H/MXMthhP\n2SPUWdrcrKk/U4C9EfDlh5FZ6JfAJ8zsFXGPVTGPSqA9UBLzz5vZJDM7BzjV3U9DUY6Hxn2fQULy\nQqh3XsZaacjHddmYG4cB+5rZgXHbJgiw+TN91vULlBHide5+enK9H75iyf9HIP/cncxsM3f/m7s/\niDbub426zso/tzjThBCs+mF4dTHwmMgXAb9FtvBdzWzPEDieRpN6vrv/tZ96ihR2RujL5Pg7CaF0\nP+3ux7j7RYhBPw+cbWarZ23qocwlkLbkHDN7b7R/52j7QSi0+73ufsd42jxecvfngZlmti/wCBpT\n+8bPjyEN2kIEWwEssphUKrznKfvOwYDe5u6/Rhg/c9G32x2ZUvZG4IrHAd8caRF29+8B/4aYNe7+\nR3f/KrAd0vjMHe93qkMQLXNu1tSf0jdwNdOGKGIV5DD9PPBOMzsDablXjt9+4O4PQL2b6qTerH0L\n4vxhpAA4ycxuRmbzL+SeGRe5+4vu/l9RVl/9MLNJubHyE5S26HPo3c2O60sjDWHajok6xgqlCSFY\nwfgYHtTDwMve4UcdhQs73epJdl+nIeHtzWa2X5T7nwg/ZhYyRfRS5hrIP+5c4MfI5DcJ+B/g74Bf\nufu17v7t8bR5rNRhUTsdaXhegfBhPmRm1yPt340ozPvlZwZF+5YKVcgUtGX8dAMypRyCzJmHAu9G\nJvAD3P2nuXIs/s4ys23MbDl3fwfw72Z2f9Lvn7v7t9z9u2NsZ51a5MLnZs39qWQDVzM9CzxhZjOA\n37v7ye6+KfIvWgAsbWbLp2tcHZvqbE3OhAt3PxH51V0Q578LPrMZQkLfO57LCzJ9Ub/9yJ43s8vM\n7Eg0hl6PoscfAb5sZrcAL2WCYUM58gGwR3Y76BM1nBpTMVBiFBAVI99SIBgcA4bknI4xxgmiygDk\nUaMgMMikvFJ9hKgxoW8Zc7OO/jCA2QlK+l7jTRpd2bxkCCLncu2qFB9r2I7aG9DjRx4zw6NCBp61\nlWqxsUpHvk36VSgYHAOC5JwbY+MCUaVe4b0UMEhKdvKlWoZX+tyssj8d6h647AQF9avfpNGVzUuG\nLHKOCvGxhvWovQFdPmwhqOFUyMCpKAqIkoSdXB2VgMFRE5IzBYGoUqP2jZLAICk5cpYaBNEy52aV\n/aGGDVwdB+3a1/Ekja5lXjIEkXNUiI81zEftDch91FJQwymZgVNBGG+n9lKOsFOpSpuKkZwpGESV\nmrRvFAgGmfuOpUFfUAPDK3Nu1tSfgcBHK3Fc95U0Oimn9HmZmzdHIF/GXwGbJdevRf5UHZ8bxIMK\n8LGG/ai9AcmHKhU1nJIYOBVgY1GRsEPNKu2yJyslgqhSg/aNAsEgqchHqAqGl9RVxdyssj8Dg49W\nxkHBfoJxfynzktzmAuXbPCnG1vlEInPgJiZoMmIqwMca1iNTLddKZrYnitD7gLs/HeGcZ6Dd2YeQ\nOnJXLwCLKeoz76Pj6fNmtiGy8U9F4aeXuft3zWwXFNL+98DzPs5IjYgY+VtEHj2E/AqWBl4Afuju\nF5rZ7cgktNF4+5SrczPk9H5QRDC9BTHsT7r7T83sBA/ckolCEbHzCmRaXhMJUs+id3koWqj7wvsK\nqI/XowTgDxTW+M517YYCOR5EoKXPItPfb5N7JvUy7iJy9kJgP/Qu/oSYw4nIYXVd4OdeYJSmmb0W\nOcS+F/iFu29e9Nwue27m6i2lP0n5s9DY3QL4sLs/aGZTkSZ/eXc/qoh6qqYkknVKzL1JyJXhQSQU\n3wzc7e7n9jqeO9RR2rw0s8sQFMRnEer771Dk+SfQpuR5d393kXVWSSZ8tFd4QDk01CPVLdlREWp4\nCe2uNAqIEu33DKlKOx1jyXsrLAlqr++0hP4UhhhOvc73ZWmRa4k4LLo/uXk5ENkJCn5fpSeNHumd\nFvBNFqvIuWHoQ1VHrThWViFqeJFUBTZWio1jJSLfLg5gcF4BiGqHOkt5N0WCQYbG4xAUYbU6whB7\nCu3Ab0aasGldC+iTXNhBj7n7A/0CJGZUB25dRkX3x30w8NFKpGkI8ytDh38JCcLnI6H+a+5+AfRv\nZcio3zKy581sL2SaPQxB3WwM/B6Zhj8MPOfuP4l7JyrifRsNQx+qoloFqzoYXr9UBbhllcKODzEY\nnFUEoloVFQ0GGZuXc5D260oEC7AC2pl+AkXa/qz4nnRsS99Mc5CAZwvqz0BmJyiK3P2XwNeBJZE2\nboa7fwuB114f87QwoaooCiFwQyTUvg7Nxd8isNJz0Tj7Q3b/ILW9oWqoFsFqojK8qnb4VQg7OY3Y\ndKQtPBo5YT6AcLH+Azl4H+PhJ1AkanSZZCUgxtdJZSCGA7j7H9z9XhRavSTCh7sqfp4wfhV1a9+K\npkESEsui8BM8G32nVwIXmdlMd/9CIlQVikpeBLn7b5CT+ulobfkAWkOWcff57n4c9J+mpqGJS5Uz\nyYnM8Mre4Vcp7Ay7StsrTIJaBXmJ+fqivJ8g35a5KBprYDY0vdAgad/6pWETEjuRTfCk0eGO8TBK\nEP0vyLfuiOz3QdOyNVQt1RIVmDC8ddz9w2Y2B0W83ODuV5nZtcA9g2x6KjMKKISdf0GC79XA/cAP\nkIbiLhSx94e4d9wTOBawf0CO8ZciE8MGUc9xwIrZ7msiUfZOzOxM4ATgre7+UAirWyBz2g+BF7yV\nr2ugFsKkD7OQP9Xj7v5/ZnYb2hlvW0X9ZdZRFpU5N6siM1sBMetLEEbVSijLwo/MbM0wo01ICj/B\nLPL0Nne/KKwYuyOz2pMog8SEyG9oTeRcQzmqVLAaBoaXkpUQxlu1sBPars2Rv8BPkLlxz9RHYJC/\nQUqdmKeZ3Qqs6u5vTq4tj6IAfxfn4wrjLpvMbB6CVPgtEcbt7pea2VcRFtfsEQtYjKmMuVkHDYOQ\nmFJsFD6PNKPrI5PaZ9z9JjPbEvmUXTxWk/ag0KCuJQ1VS5UIVsPG8DpRkcJHHcKOma2FQoZPAc51\n99PLqKcsshbe1ySU+uIFlObjifBHW9bd3xH3plhHA9m/mAtXIGf0J8xsB2BX4FZ3v8fMvgkc5+6P\n1NrQCUCD+o17pSESEjdGAMNnuPslMca3QxkE7kV+ZB/qx6TdUEODQKULVsPG8KqkqoWdiarSthpA\nVMskG1IwyIb6p4m+LprZfQgbbYPk2joo0fKf3P27tTWuoYYKolIFq2FjeHVQXcLORNIWZmQTGDE+\nt6moDDG8oYbKorr9BBtqqC4qNSowWfg3RYLUYSjC5WrgjWa2nrvPAb5cZjsmMrn7i5lQZRVCHUwE\nph0Ce/Z/aSCqVVAwoGEGg2xoMaMY0/MQmv8JwGVmdpi77wL83swerreFDTVUDpXCqIeJ4Q0SNcy0\nRTZkiPE25GCQDS1+FD5U+6AcsDshp/VZZraDu88D/mJmW9TayIYaKoEKF6yGjeE1NJjkQ4QYb4sB\nGGRDixeF+W8+ykG5XFx+EEW37gzg7ts0wRcNDSMVLlgNE8NraPAopw2d8IjxthiAQTa0eFDO4rAE\nygP4ENpQb+ICcX0MmG5m0wZ1TjbUUL9U2MAeNobX0GBS4uA9FIjxPkSI4Q0t3tT4CTbUkKgwoWbY\nGF5Dg0s2ZElQfUjy9TW0eFPjJ9hQQ6JC4RZsSFOkNDR4VAeIahU0LGCQDS1eFH6CzyN/quvRnLwR\nOB6ZBA9096fqa2FDDVVHheNYDSvDa2gwqWoQ1SppGPrQ0PBT+AmeCKyBsgX8Edgf+CwwD+WF3aYx\naTe0uFBpAKHDzPAaGiyaqIjxDTU0LGRDnDS6oYbGSmUjrzcMr6FKaSIixjfU0LCQDVnS6IYaGg9V\nkoQZGobXUEMNNbQ4UOMn2NDiTpUJVg011FBDDS1e1Lh+NLQ4UiNYNdRQQw011FBDDRVEDThnQw01\n1FBDDTXUUEHUCFYNNdRQQw011FBDBVEjWDXUUEMNNdRQQw0VRI1g1VBDDTXUUEMNNVQQNYJVQw01\n1FBDDTXUUEHUCFYNNdRQQw011FBDBdH/BxGt3Y319vaGAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "local_1.plot_lr_weights([\"@\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see features for specific users such as \"word=@justinbieber\" corresponding to the feature function $f_{\\text{word},\\text{\"@justinbieber\"}}$. This is all the model can do right now: remembering words from the training set. This works for \"@justinbieber\", but for less popular twitter accounts that are unlikely to appear in the training set this means we cannot correctly classify them at test time. In other words, the model does not generalise well. \n", "\n", "In this particular case the problem can be easily solved. We notice that each of the \"@\" words starts (obviously) with a \"@\" symbol. If we turn this observation into a feature it should be simple to learn the class. We can do this by introducting a new feature `first_at` which returns `True` if $x_i \\text{ starts with @}$ and `False` otherwise. Internally this is again transformed into a proper real-valued feature template." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.6877462160481028" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def feat_2(x,i):\n", " return {\n", " **feat_1(x,i),\n", " 'first_at':x[i][0:1] == '@'\n", " }\n", "local_2 = seq.LocalSequenceLabeler(feat_2, train)\n", "seq.accuracy(dev, local_2.predict(dev))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This looks much better. Of course this particular aspect of the problem is so deterministic that we could simply preprocess all words starting with @ and label them as '@' right away. But is important to know how easy such observations can be incorporated into the probabilistic model. \n", "\n", "To confirm that these results actually from improved '@' prediction, let us look at the confusion matrix again." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAGoCAYAAABWs9xCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8JVV54P3fc/rGpUG6OU0Pl0ZQEUWjCEfwNq8CUUCN\nYBywveLrpUfFcURNBJMJmMjIO5qYN14w/cYLahSJV+ZNohKUiUYjNISAgAiCCAShGzERAw3d/cwf\nVUe2Z/qcs2tXVZ+9a/++/alP712natWz99lnP7VWrVorMhNJkjR6JhY6AEmSNBiTuCRJI8okLknS\niDKJS5I0okzikiSNKJO4JEkjyiQuSdKIMolLkjSiTOKSJI2oxW0fYMXKydx3zf61yli22HMNDa+6\nYx5GI1FIw+uKKy7flJmrFjoOgEW7Pzxzy32NlJX3bfxaZh7XSGEDaj2J77tmfy7422/VKuMRe+3a\nUDRS8+oOXRxhGle37bwkblnoGKbllvtYdvDJjZR1/5UfmmykoBpaT+KSJA2PgOhO665JXJI0PgLo\nUOtXd05HJEkaM7Vq4hFxFnBvZr6vmXAkSWqZzemSJI0om9MlSdJCa6UmHhHrgHUAe++7po1DSJI0\ngG71Tm/llWTm+sycysyplXsu+G10kiQ9JKKZpa9DxY8j4uqIuDIiNpTrVkbERRFxQ/n/ip7tz4iI\nGyPi+og4dr7yayXxzDzLTm2SJM3pqMw8NDOnyuenAxdn5kHAxeVzIuIQYC3wOOA44MMRsWiugrvT\npiBJ0nyCojm9iWVwJwDnlY/PA07sWX9+Zm7OzJuBG4Ej5iqoVhQR8fqIeGWdMiRJ2nEaakrvv4d7\nAn8XEZeX/cUAVmfmHeXjnwKry8f7Arf27HtbuW5WtTq2ZeZH6uwvSdIIm5y+zl1an5nrZ2zzjMy8\nPSL2Ai6KiB/0/jAzMyIGnoDB+8QlSeOlud7pm3quc29XZt5e/n9XRHyJonn8zojYOzPviIi9gbvK\nzW8Hem/p2q9cNyuviUuSxssOak6PiF0jYrfpx8BzgO8DFwKnlJudAnylfHwhsDYilkXEgcBBwKVz\nHcOauCRJ7VgNfKmcbngx8JnM/GpEXAZcEBGvAW4BTgbIzGsi4gLgWmALcGpmbp3rAK0n8UUTwR67\nLKlVxrZt9eZrBpiY6M4wexouzgcujZIdN9hLZt4EPHE76+8Gjplln7OBs/s9hjVxSdL4cCpSSZI0\nDKyJS5LGS4fGTjeJS5LGSLcmQBkoiUfEe4CvAw8DHpuZ72k0KkmSNK9BT0eOBP4ReCbw982FI0lS\nyyaimWUIVKqJR8R7gWOBA4HvAo8EjomIz2fmH7YQnyRJmkWlJJ6Zv1PeiP5K4K3AJZn59JnblYO8\nrwPYd7/9m4hTkqT6pmcx64hBXslhwD8DjwGu294Gmbk+M6cyc2rPyck68UmS1KwdO4tZq/quiUfE\nocAnKAZk3wTsUqyOK4GnZuZ9rUQoSZK2q++aeGZemZmHAj8EDgG+ARybmYeawCVJo6G8xayJZQhU\n7di2CrgnM7dFxGMy89qW4pIkqR1D0hTehKod2zYCzysfP6WViCRJUl8csU2SNF6GpCm8CSZxSdL4\nGKKe5U3ozumIJEljZofUxLPm/hMNDG/3y/u31Np/151stNDw2rat7l9ZM39n0kiwOV2SpBFlc7ok\nSVpo1sQlSWPE+cQlSRpdNqdDRBwQEa9qMBZJklTBQDXxiHgD8GZgeZnI12bmT5sMTJKkxnVsKtLK\nSTwidgPeBRwHPAG4BPhls2FJktSGbl0TH+SVbKO49XslQGb+ODN/0btBRKyLiA0RseHuTZsaCFOS\nJM1UOYln5i+B1wHvAf4oIt4XEbvM2GZ9Zk5l5tSek5MNhSpJUgOmh16tuwyBgdoUMvNC4CTgfwCr\ngLc1GZQkSa0Z1/nEASJiObBn+fQXwHWUTeuSJGnHGaR3+hLgzykS+STwE+ClTQYlSVJrhqQpvAmV\nk3hm3gMcFxEHAM/KzE80HJMkSe0Ie6dP+zlwZVOBSJKkagYedjUzTeKSpNEzzs3plQ8wEey5fGnb\nh5lX3fnAb7qr/ng2j9hr19plSNvjXOBS/6JDSbw7FwYkSRozzmImSRobQbdq4iZxSdL4iHLpCJvT\nJUkaUZVq4hExCfwVxUAv9wNHZ+a9bQQmSVLzYqyb098A/H1mnhkR+wAPtBCTJEmtGeck/gBwAEBm\n/kvj0UiSpL5VvSb+I+C3I+L1bQQjSVLbIqKRZRj0ncQjYl/gDOBRwGsj4kXl+qsi4mEztl0XERsi\nYsPGTRsbDViSpDq6lMSrNKc/Hbg6M++OiOcBF0fEauDHmfmvvRtm5npgPcDhh09lY9FKkqRfqdKc\nfhVwVETsk5l3AqcBHwI+00pkkiQ1LRpchkDfNfHM/EFE/B7wtYh4ELgTWAucExFXZOYP2wpSkqQm\nxDjfYpaZnwY+PWP155oLR5Ik9cthVyVJY2Vsa+KSJI26LiVxx06XJGlEtV4TT2Drtnp3mS2aWPiz\npkfstWvtMlac+KHaZdzz5VNrlyFJ/br3/i0LHULjulQTtzldkjQ+huj2sCbYnC5J0oiyJi5JGis2\np0uSNIK6NtiLzemSJI0oa+KSpLHSpZq4SVySNF66k8PbaU7vnU98k/OJS5LUilaSeGauz8ypzJya\nnFzVxiEkSaouiub0JpZhYHO6JGmsDEsCbkLtmnhEXBwR+zYRjCRJ6l+tmnhETACPAn7WTDiSJLWr\nSzXxus3phwBfyMz7mghGkqQ2dW2wl1pJPDO/D7y1oVgkSVIFdmyTJI2X7lTETeKSpDESXhOvJIBF\nE915w+q458un1i7jrK9dX7+MYw+uXYak8bB8J+t6dUXEImADcHtmPj8iVgKfAw4AfgycnJn3lNue\nAbwG2Aq8OTO/NlfZToAiSRorCzDYy38Frut5fjpwcWYeBFxcPiciDgHWAo8DjgM+XJ4AzMokLkka\nKzsyiUfEfsDzgL/oWX0CcF75+DzgxJ7152fm5sy8GbgROGKu8k3ikiQNZnJ6npByWbedbf4U+F1g\nW8+61Zl5R/n4p8Dq8vG+wK09291WrpuVFzskSeOluW5amzJzatbDRDwfuCszL4+IZ21vm8zMiMhB\nA+g7iUfEauCdwFHAFuAK4F2ZeeucO0qSNER2YO/0pwMviIjnAjsBu0fEp4E7I2LvzLwjIvYG7iq3\nvx1Y07P/fuW6WfXVnB4RjwS+CvwDMJWZhwGfBb5U/kySJPXIzDMyc7/MPICiw9o3MvPlwIXAKeVm\npwBfKR9fCKyNiGURcSBwEHDpXMfotyZ+LnBKZl7VE9zFEfFy4I956KK8JElDa0imET0HuCAiXgPc\nApwMkJnXRMQFwLUULd6nZubWuQqaN4lHxKOBjZl5Vdm+/4fATUBk5osiYltETGbmpp591gHrANbs\nv/9Ar1CSpDYsRBLPzEuAS8rHdwPHzLLd2cDZ/ZbbT038icA/lveqnQkcDTwM+H758xuAA4FfJfHM\nXA+sBzj88KmBL9hLkqTZ9ducvhWYBH6UmT8Hfh4R15Y/24uHLspLkjTUhqA5vTH9dGz7PnAkRU37\nkRHxsIjYH3hsRPwGsFdm3tJmkJIkNSYaWobAvDXxzLyuTNoHA+8GvklxTfxC4O3Aq1uNUJIkbVe/\nzelvBP4SeAdweLnuMGCfzLyzjcAkSWrDuDWnk5nXAS8AXkQxyMs/A28ArpprP0mShkosyAQorel7\nxLbMvA14fYuxSJKkChw7XZI0NgIYkkp0I1pP4vdv2caP7ry3VhmPXL28oWhG31nHHly7jH/68c9r\n7f+kA/aoHUOXbNtWbyiEiYnh+Eap+zpgeF5LXZn134thaW7VTMPTFN4EpyKVJGlE2ZwuSRorHaqI\nm8QlSePF5nRJkrTgKifxiDgxIjIiHtNGQJIktSaK5vQmlmEwSE38JcC3y/8lSRoZQXEXRRPLMKiU\nxCNiOfAM4DXA2lYikiRJfalaEz8B+Gpm/hC4OyIO395GEbEuIjZExIZ77t60vU0kSVoQ49yc/hLg\n/PLx+czSpJ6Z6zNzKjOnVuw5WSc+SZIaNZZjp0fESuBo4DciIoFFQEbE72QTwxtJkqRKqtTE/xPw\nqcx8eGYekJlrgJuB/9hOaJIkNWyMe6e/BPjSjHVfwF7qkqQRUUyAMobN6Zl51HbW/Vmz4UiSpH45\n7KokaYwMTy26CSZxSdJY6VAObz+JL1s8wcMnd2n7MKqg7nzgr/z0FbVj+OTLD6tdxrAYhpGbnAu8\nWV2qqanbrIlLksZKl07STOKSpPExRLeHNcGpSCVJGlHWxCVJY2P6PvGuqDLs6lbgamAJsAX4JPD+\nzNzWUmySJDWuQzm8Uk38vsw8FCAi9gI+A+wOnNlGYJIkaW4DXRPPzLuAdcCbokvtEpKkzhvLYVdn\nysybImIRsBdwZ3MhSZLUniHJv41opXd6RKyLiA0RsWHTxo1tHEKSpLE3cBKPiEcAW4G7Zv4sM9dn\n5lRmTk2uWlUnPkmSmhM2pxMRq4CPAB/MzPrjPUqStAMUt5gtdBTNqZLEd46IK3noFrNPAX/SSlSS\nJGleVeYTX9RmIJIktW94msKb4IhtkqSx0qEc7tjpkiSNKmvikqSxYnO6BtJER/5h+PB98uWH1S7j\nl/dvqV3Grjv58Z02MbHwnwt1kzcgDTe/BSVJ46Nj84mbxCVJY6NrU5HasU2SpBFlTVySNFa6VBOv\nnMQjYitwdc+q8zPznOZCkiSpPR3K4QPVxO/LzEMbj0SSJFVic7okaax0qTl9kI5tO0fElT3Li2du\n4HzikqShVN5i1sQyDFppTs/M9cB6gMMOn3KkAEmSWmBzuiRpbISzmEmSNLo6lMMHSuI7R8SVPc+/\nmpmnNxWQJEnqT+UknpmL2ghEkqQdYaJDVXGb0yVJY6VDOdyx0yVJGlXWxCVJY6O4x7s7VfHWk/i2\nTDY/uK1WGYsXdaPBYOu2+rfML17UjQ/frjvV/+g9sKXe5wpg6eJufLYk9W+iG1+jgM3pkiSNLJvT\nJUljxeZ0SZJGVIdyuM3pkiSNqkpJPCJWR8RnIuKmiLg8Ir4bES9sKzhJkpoUlOOnN/BvGPTdnB7F\nRYQvA+dl5kvLdQ8HXtBSbJIkNa5LvdOrXBM/GnggMz8yvSIzbwE+0HhUkiRpXlWa0x8HXNHPhhGx\nLiI2RMSGuzdtGiwySZKaFsVUpE0s8x8qdoqISyPinyPimoh4V7l+ZURcFBE3lP+v6NnnjIi4MSKu\nj4hj5zvGwB3bIuJDZWCXzfxZZq7PzKnMnNpzcnLQQ0iS1Lhi1Lb6Sx82A0dn5hOBQ4HjIuIpwOnA\nxZl5EHBx+ZyIOARYS1FpPg74cETMOelYlSR+DXDY9JPMPBU4BlhVoQxJksZCFu4tny4plwROAM4r\n158HnFg+PgE4PzM3Z+bNwI3AEXMdo0oS/wawU0S8oWfdLhX2lyRpQQXFVKRNLMDk9KXjcln3fxwv\nYlFEXAncBVyUmd8DVmfmHeUmPwVWl4/3BW7t2f22ct2s+u7YlpkZEScC74+I3wU2Ar8E3tFvGZIk\nLbQGB3vZlJlTc22QmVuBQyNiD+BLEfH4GT/PiBh4Yo1KI7aVZw5rBz2YJEnjKDN/HhHfpLjWfWdE\n7J2Zd0TE3hS1dIDbgTU9u+1XrpuVI7ZJksbKDuydvqqsgRMROwPPBn4AXAicUm52CvCV8vGFwNqI\nWBYRBwIHAZfOdQzHTpckjY0KPcubsDdwXtnDfAK4IDP//4j4LnBBRLwGuAU4GSAzr4mIC4BrgS3A\nqWVz/KxM4pIktSAzrwKetJ31d1Pc3bW9fc4Gzu73GK0n8YkIli2x1R5g8SLfhyYtXVz//dz0i821\ny5jcbVntMqRh1aVpO6dNdOg1WROXJI2V7qRwO7ZJkjSyrIlLksZKly4RmMQlSWOjGLFtoaNozkDN\n6RFx7/xbSZKkNlkTlySNjz4HahkVJnFJ0ljpUA5vp3d6RKybntVl08aNbRxCkqSx10oSz8z1mTmV\nmVOTq5xuXJI0PHbU2Ok7gs3pkqSxYe90SZI0FAZN4rtExG09y1sbjUqSpJaMfXN6ZlqDlySNpOFI\nv80wGUuSNKLs2CZJGhsRTkUqSdLI6lAObz+J//DOX/Cb7/9WrTIuefszG4pG+nWTuy2rXcarP3tl\nrf0/9pJDa8eQmbXLaMKwdPapa/ODW2uXsWzJogYiqaeJz0VXfqddZU1ckjRWunRiYhKXJI2VDuVw\ne6dLkjSqrIlLksZGEJ3qnV6pJh4RGRF/3PP87RFxVuNRSZLUhiia05tYhkHV5vTNwG9HxGQbwUiS\npP5VTeJbgPXAaS3EIklS67o0dvogHds+BLwsIh422wYRsS4iNkTEhgd/+a+DRydJUsMmGlqGQeU4\nMvPfgE8Cb55jm/WZOZWZU0t2nTXXS5KkGgbtnf6nwBXAxxuMRZKkVgXdGuxloBaBzPwZcAHwmmbD\nkSSpXRPRzDIM6jTr/zFgL3VJkhZIpeb0zFze8/hOYJfGI5IkqUXDUotugiO2SZLGRjFQS3ey+LD0\nkpckSRW1XhN/9OrduPit/1fbh9EOtHVb/TmKF3WoPavufOAnfeyy2jF89pTDa5exeJHn9NOGYS7w\nJnSpxtmkDn392JwuSRovXTq38dRbkqQRZU1ckjQ2Ajo1FalJXJI0VrrUBF11PvH9IuIrEXFDRPwo\nIv7fiFjaVnCSJGl2fSfxKLo5fhH4cmYeBDwaWA6c3VJskiQ1rrhXvP4yDKrUxI8G7s/MjwNk5laK\necVfHRGO3CZJGnoRwURDyzCoksQfB1zeu6KclvQnwKOaDEqSJM2vlY5tEbEOWAewZv/92ziEJEkD\nGZJKdCOq1MSvBX5tWKiI2B3YH7ixd31mrs/MqcycmpxcVT9KSZIaMq5TkV4M7BIRrwSIiEUU05F+\nIjP/vY3gJEnS7PpO4pmZwAuBkyLiBuCHwP3AO1uKTZKkRk0P9tKVjm1V5xO/FfitlmKRJKl1Q5J/\nG9GlgWskSRorDrsqSRofQ9QprQkmcUnSWAm6k8V3SBIv+sTV0Z03vAsWdek0dgj81aufXLuMNes+\nV7uMn/z5ybXLiJoXG7dtq/tdARN+PjVGrIlLksZG0Tt9oaNojklckjRWupTE7Z0uSdKIsiYuSRor\ndftuDJPKSTwitgJXl/teB5zisKuSpFHQtWvigzSn35eZh2bm44EHgNc3HJMkSepD3eb0bwFPaCIQ\nSZJaF90adnXgJB4Ri4Hjga9u52cPzSe+xvnEJUnDY1gmL2nCIM3pO0fElcAG4CfAR2du8Gvzia9y\nPnFJktowSE38vsw8tPFIJElqWdc6tnmLmSRprHSoNd3BXiRJGlWVa+KZubyNQCRJal8w0aFJtWxO\nlySNjcDmdEmSNI+IWBMR34yIayPimoj4r+X6lRFxUUTcUP6/omefMyLixoi4PiKOne8YJnFJ0viI\nond6E0sftgBvy8xDgKcAp0bEIcDpwMWZeRBwcfmc8mdrgccBxwEfjohFcx2g9eb0bZlsfnBbrTIW\nL/Jco0mbH9xaa/9lS+b8TI2dbduy1v4TDdzvcvO5J9Uu4x9/9LPaZTz1UXvW2r9LzZwaXjtqsJfM\nvAO4o3z8i4i4DtgXOAF4VrnZecAlwDvK9edn5mbg5oi4ETgC+O5sxzA7SpI0mMmI2NCzrJttw4g4\nAHgS8D1gdZngAX4KrC4f7wvc2rPbbeW6WdmxTZI0Nhru2LYpM6fmPWbEcuALwFsy8996p0LNzIyI\ngZvzTOKSpLGyI8dOj4glFAn8LzPzi+XqOyNi78y8IyL2Bu4q198OrOnZfb9y3axsTpckqQVRVLk/\nClyXmX/S86MLgVPKx6cAX+lZvzYilkXEgcBBwKVzHaNSTTwitgJXl/vdDLwiM39epQxJkhbSDqyI\nPx14BXB1OXEYwDuBc4ALIuI1wC3AyQCZeU1EXABcS9Gz/dTMnLMnctXm9F9NfhIR5wGnAmdXLEOS\npAUR7Lgm6Mz8dnnI7Tlmln3OpkJerfNavss8veYkSVJ7BurYVt58fgzbmUu8/Pk6YB3Afmv2Hzg4\nSZIaFRAdGpCgak1857Jdf/q+tou2t1Fmrs/Mqcyc2nNysm6MkiRpO6om8elr4g+naOc/tfmQJElq\nTzS0DIOBroln5r8DbwbeFhHeay5JGglBcZ94E8swGLhjW2b+E3AV8JLmwpEkSf2qVIvOzOUznv9W\ns+FIktSu4ahDN8OmcEnSWBmSlvBGOOyqJEkjypq4JGmMRKfuE289iU9EsPPSRW0fRhUsW+Lvo0kT\nEwv/hbB4Uf1Gtac+as8GIqmnS1+uGk47ctjVHaFLr0WSpLFic7okaax0qcXHJC5JGivdSeE2p0uS\nNLIGSuIR8XsRcU1EXBURV0bEkU0HJklS48pZzJpYhkHl5vSIeCrwfOCwzNwcEZPA0sYjkySpYV3r\nnT7INfG9gU2ZuRkgMzc1G5IkSerHICckXwfWRMQPI+LDEfHMmRtExLqI2BARGzZt2lg/SkmSGtKl\n5vTKSTwz7wUOB9YBG4HPRcSrZmyzPjOnMnNqcnJVI4FKktSELs0nPtAtZpm5FbgEuCQirgZOAT7R\nXFiSJGk+g3RsOxjYlpk3lKsOBW5pNCpJkloyJC3hjRikJr4c+EBE7AFsAW6kaFqXJGmoFb3Tu5PF\nKyfxzLwceFoLsUiSpAocdlWSNFbGvTldkqQRFcQ4N6dXFQzHfMtSl23blrXLaKJ2Uvfe2V9u3lI7\nhl2XWTfR+PDTLkkaKzanS5I0grrWO71L48BLkjRWrIlLksZHjHFzekTsCVxcPv0PwFaK8dMBjsjM\nBxqMTZKkxo1tEs/MuymGWSUizgLuzcz3tRCXJEmah83pkqSx4n3ikiSNoAC6NHRJK73TI2JdRGyI\niA0bN22cfwdJklRZK0k8M9dn5lRmTq2aXNXGISRJGkg09G8Y2JwuSRorXeqd7mAvkiSNqIFr4pl5\nVoNxSJK0QwxLU3gTbE6XJI0Ne6dLkqShYE1ckjRGhqdneRNaT+Kbt2zjprt+WauMR+y1a0PRSN00\n0ZH2wV2XWa9Qyzo2AYrN6ZIkjShPeyVJY6VDFXGTuCRpfBS907uTxm1OlyRpRFVK4hFxQER8f8a6\nsyLi7c2GJUlSO6KhZRjYnC5JGi/DkoEbYHO6JEkjqpWaeESsA9YB7L3vmjYOIUnSQLo02EvVmnj2\ns753PvGVe04OFpkkSS2IaGYZBlWT+N3AihnrVgKbmglHkiT1q1ISz8x7gTsi4miAiFgJHAd8u4XY\nJElq3Lj3Tn8l8KGI+JPy+bsy80cNxiRJUnuGJQM3oHISz8xrgaNaiEWSJFXgfeKSpLFRNIV3pypu\nEpckjY8h6lneBAd7kSRpRLVeE//FA1v4+1s21irjEXvt2lA0Ati2bbbb/fszMdGh09gGPLhlW639\nlyyufy5d93cK/l57rXjym2qXcc9lH2wgkoWXWf+zNWy69Em3OV2SNF46lMVtTpckaURZE5ckjZGw\nd7okSaNqLHunR8Q3I+LYGeveEhHnNh+WJEmaT5Vr4p8F1s5Yt7ZcL0nS0Gtq3PR+KvMR8bGIuCsi\nvt+zbmVEXBQRN5T/r+j52RkRcWNEXD+z0jybKkn888DzImJpebADgH2Ab1UoQ5KkhbXjZkD5BMUk\nYb1OBy7OzIOAi8vnRMQhFBXjx5X7fDgiFs13gL6TeGb+DLgUOL5ctRa4ILdzE2FErIuIDRGx4d57\n7u73EJIkdUZm/j3wsxmrTwDOKx+fB5zYs/78zNycmTcDNwJHzHeMqreY9Tapz9qUnpnrM3MqM6eW\nr9iz4iEkSWpPNPQPmJyusJbLuj4Ovzoz7ygf/xRYXT7eF7i1Z7vbynVzqto7/SvA+yPiMGCXzLy8\n4v6SJC2oBnunb8rMqUF3zsyMiFpD4lWqiWfmvcA3gY9hhzZJkqq6MyL2Bij/v6tcfzuwpme7/cp1\ncxpkxLbPAk/EJC5JGkE7rl/bdl0InFI+PoWihXt6/dqIWBYRBwIHUfRDm1PlwV4y88t0auRZSdLY\nqJmBKx0q4rPAsyiund8GnAmcA1wQEa8BbgFOBsjMayLiAuBaYAtwamZune8YjtgmSVILMvMls/zo\nmFm2Pxs4u8oxTOKSpLHi2OmSJI2goFtjp7eexCd3WcqrnnxA24dRBRMTHfoED4Elixd+Rt8mfqfb\nGbepsujIt+M9l31woUMYGl35nXaVNXFJ0ljp0mmJSVySNF46lMUXvh1QkiQNxJq4JGmsdKl3eqWa\neESsiYibI2Jl+XxF+fyANoKTJKlpEc0sw6Dq2Om3AudSjDhD+f/6zPxxw3FJkqR5DNKc/n7g8oh4\nC/AM4E3NhiRJUnuGpBLdiEHGTn8wIn4H+CrwnMx8cOY25Zyq6wDW7L9/7SAlSWpMh7L4oL3Tjwfu\nAB6/vR9m5vrMnMrMqVWTqwYOTpIkza5yEo+IQ4FnA08BTpueF1WSpGFXTGLWzL9hULV3elB0bHtL\nZv4EeC/wvjYCkySpcQ31TB/J3unA64CfZOZF5fMPA4+NiGc2G5YkSZpPpY5tmbkeWN/zfCtwWNNB\nSZLUliGpRDfCEdskSeOlQ1ncsdMlSRpR1sQlSWNkeHqWN6H1JJ7Alq3bapWxeJENBlLbYli620ot\n69JH3ewoSdKIsjldkjQ2gk71azOJS5LGTIeyuM3pkiSNqKrDrr4wIq6csWyLiOPbClCSpCZ1aez0\nqiO2fQn40vTzcsrRlwFfazguSZJa0aXe6QNfE4+IRwN/ADwtM+vdQyZJkiob6Jp4RCwBPgO8rZzN\nbObP10XEhojYsGnjxroxSpLUmGhoGQaDdmz7I+CazPzc9n6YmeszcyozpyZXrRo8OkmSmtSxqUgr\nN6dHxLOAF+HsZZIkLahKSTwiVgAfB16amb9oJyRJkto0JNXoBlStib8e2As4d8Y4y++ZrWldkqRh\nEQxPU3gTqt5i9h7gPS3FIkmSKnDYVUnSWOlQRdwkLkkaL2PbnD6IO/7tft79dzfUKuOsYw9uKBoB\nZGat/Z1XaJwxAAAOsklEQVR3+tdt2VpvrKPFi5zCQO24+94Hapex5/KlDUSitlgTlySNlWEZ97wJ\nVgEkSRpR1sQlSeOlOxVxk7gkabx0KIdXnk98cUT8dURsiojHtxWUJEmaX9Vr4ucCPwBOBD4XEfs1\nH5IkSe1oavKTYblJp+/m9Ig4E/jXzHx7+fy1wGcj4vmZ+a9tBShJUpO61Du97ySeme+a8fy7wH/c\n3rYRsQ5YB7D7qn3qxCdJkmbRyi1mvfOJ7/KwFW0cQpKkwURDyxCwd7okaawMSf5thIO9SJI0oqyJ\nS5LGyrD0LG+CSVySNEaiU73TbU6XJGlEWROXJI2NoFvN6dbEJUkaUa3XxPfZfSfOOvbgtg8zEu69\nf0vtMpbvVP9XFl06DR0Cixd5LqzhtOfypbXLuPSmnzUQidpic7okaax0qR5jEpckjRV7p0uSpAVn\nTVySND6GaBrRJpjEJUljY4jmLmmEzemSJI0oa+KSpPHSoap430k8Ik4D1gIPAB8HvgWcAPxDZn53\nxrbrgHUAa/bfv7FgJUmqa1x7p68Gng68FjgK+J/A7sD3Zm6Ymeszcyozp1ZNrmokUEmS9Ov6roln\n5unlw+uBV7QTjiRJ7bJ3uiRJI6pDOdze6ZIkjSqTuCRpvERDSz+HijguIq6PiBsj4vT596jG5nRJ\n0ljZUb3TI2IR8CHg2cBtwGURcWFmXtvUMayJS5LUjiOAGzPzpsx8ADif4tbsxlgTlySNjWCH9k7f\nF7i15/ltwJFNHqD1JH7FFZdv2nlJ3DLPZpPAphqHqbt/l8oYhhi6VMYwxDAsZQxDDMNSxjDEMEpl\nPLxm+Y254orLv7bzkphsqLidImJDz/P1mbm+obL70noSz8x5R3uJiA2ZOTXoMeru36UyhiGGLpUx\nDDEMSxnDEMOwlDEMMXStjB0lM4/bgYe7HVjT83y/cl1jvCYuSVI7LgMOiogDI2IpxdDlFzZ5AK+J\nS5LUgszcEhFvAr4GLAI+lpnXNHmMYUnida8hNHENoitlDEMMXSpjGGIYljKGIYZhKWMYYuhaGZ2U\nmX8D/E1b5UdmtlW2JElqkdfEJY2EiHBeY2kGk7jUYRHdmK8pIp4LXBwR+9Ysp9YlxIjGbk2qJSL8\n7hYw4kk8IlYudAwaPsPyBRcR+0fErg2UUycRL1rAY/eWs0uNfY8F3ge8IjNvH/T3GxGPBn4/IvYc\ncP+HA++NiP0G2b8s46iIeNqg+5dl/CbwyjplqDsW5MsuIh4REbvVLGMv4I0RsTQiDqhZ1lB86S+U\niFg943nt96PGF+3hEfGUGsd9BvCqiHjSAPs2Vmst39O3AW8YNJFHxH5lwhkoaUTEs4FPRcTpEfH8\nQcoAlg64X28czwX+e0SsmXfj/3Pf5wCfBK4FfgaQmdsG/F2tAFZS/E4GqQAspxiBa68ytkE+488E\nXjHo/hFxNPBl4IyutLKonh2evCJiJ+BNwNsiYvmAZewHHEBRy/gfwJmDlBURj42IlwKnRcTOA+z/\n3Ih4e/l4wU4EImJJRDwjIs6IiBdUSYIR8Rjgjoh4f0S8DoovyfJnfb+miDgoIp4SEUdHxIpBvmgj\n4niKXq7/XmW/nv2PAz4AbAEeNkARi8pymrhrYyPFPaL7AK+umsgj4gTgr4CPAp+PiLPK+0z73f84\n4GzgO8CuwIsiotJwj2UCPT8izoyI366yb08ZzwfeA1ySmbfOt/2MfY8BPgi8leJ1vLo8SSMzs+rn\nKzO/B3wK2B14U9VEXt4a9E3gIxGx+/TfSUXfKY9P1f17WiTeCFyaDfVK9mRgxGXmDl+AAymS7xnA\n8or7Lgf+H+DRwLuBe4EjB4zjD4BbgDcOsO8xwD9TDGY/0bt+B7+XSykGD3g7cBZwEvDV8v/d+th/\nP+DbwDuAr1PUel4A7F4hhucBVwBfAi6iGCv4SeXPos8yjgP+AXhO+XwFcGCFGJ4J3DjzswA8rs/9\nJ4EfAyvL54sH/H0cBBw8/dqB36I4sXhzv5914Cjgh8DhwB7lZ/0fKZLyoj72XwlsA36rfL4G+Bxw\nYoXXcRzwPeBU4F3A/wc8quJ78R8okt6Tez6ru5SfuZ362P/JwNPKxwcDf0RxQvD0nm3m/HwBTwPW\nzlh3ZPn98/vz/Y2U7+Xynue7Ah8BjiqfT/TxOo6hqLg8neK77x+AfWZsM2c5FLNgXQc8tXx+NXBY\nP+/BjHKeAZwMvL78v9LvdJ7f1bIqsbg0syxI7TEzb6aYnm0l8M4q18sy816KL6S/AZ5D8Yf44og4\nISJe0m85EfEw4CkUX1DXRsTBEXFkRBzaZw30mcAXMvMr+VDNdRfgwxHxon7j2E5cSypsuxi4APhS\nZr4vM88C/hfwMuBQivdnTpl5G3ApcBjwXIr39dXAX0fEERFx0DwxHAf8N+C0zHxhZj6b4gv/woh4\nYmbmfO9nWSP6G+C9mfn1iHgkxYlJld7ITwI+kEVta7rc9wL/K4rBFuaUmZuA/wJ8p2xJ2BIRi6vU\nUsqm7+uBb0XEqcB/Bv6aIhnuDry2z8/604A/y8zLgfsz84fAiykS6xl9vJafUZw8nFPWGG8FHgRW\nz73nr17H9O/j3Zn5IYrf51KKE50qNpfHvb9sgXsnxe/1L4Fz56sJZ+ZlmfmdiJjIzOspTjAfBJ4/\nfV05ywwyhxUUTfkn9ZT7PeDzFK0kR8+2Y0TsUW73rrJlhMz8JUWz/mvL5/3UppdS/H39LsV7+ViK\nVoUXlMeJucop/84fA7w2M78bxfSWdwGryhiyZ7vZyojy8/mXFN8NBwOH9BH7vCLiQODE3li0Ay3k\nGQTwSIoaSqWzN+CFwE3AJ8rn7wXuoFpNYzeKpq23U5yRXk/xhXsefdTMgZdSnJGfRVETfS7wGxRn\nuKdRJPklFV/XBMVUdaf1uf3BwFnl48XAnwA/BV5H8eX1d8xRE51+3ym+ZM6nqDk9q3xvP0Jx7e1c\nYNdZ9p+u8T2/fL5Tz8/OLMt5WJ+vZbo2/wSK2vzb+txv+jV8gCLpTK8/HvgERUK8BTihz/KOB34E\nrJh+X8v/j6JsXZhn/6PL9+S/UFwa+Cvg48CfU5x8ngosm+e1nNvzew3K2nf53lxCcU123r+Z8rXc\nQNEk/UX6qP3O+H1cQ9kiU/5tfBv4U4rm7cn5Pt9l7G+jGK3qtvL38VqKmvAngBdU+fsoyzyo/Gz9\nGX22wJXvw1XAi2e8z6+jOAmetRZM8R31Corxrs+mqBEvLv+2XjpA/AcCnwH+J/ANipOE7wAvmut3\n2vM5nCj//0Pg7T0/PwlYN9drKbf7HPDIqnHPUd6TKU7O/pY+/9Zdml0WPICeD8MTgF362G7X8kPz\nBIpk/u7yi+pTAxzzNymaxM8vvxROojipeE8f+64G/m/gnygSyFcomqO/DvwAuJIZTWZ9xvQbFNdS\nD+tj2/8M/EX5+CMUlweOpGiuO4Xi5GTfecoIiiT+RxRn6T+gPBkqvzBXzLP/8yia9vYsny/r+dk3\ngKkKr/04igR4evl8Uc/6o+bZ9xiK5D/dxLgEWFo+/n3gJRXimJnI30jRVL9/n/tPN30upWjKPqX8\nkrsb+P58X3Y9r+Xw8vlE+Xr2Ab7ALCdVc3zGtwF7lc+rJPLpk4APlPG/kCJRfA/4C/q45EJx+eup\n5Wex97PxUeDlVf8+yn0fQ9EisarCPs8t/1Zf3LNuLcXJ1byXTiguaZxBcWL7t+Xf2/sqHL/3kttp\nwMfLx3uXn49H9FnO9AnIO4Avlo9fVn6uHtPHfl+gvNxTd6E4IfhYuRzSRJkuA/weFjqA8sOwHHgL\nMNnn9tNno79Dcf31OIprZX8wwLF3o7wuBjye4qz4o/3GUX7BPbVn3csoTgR2rvF+HE2ZFOfZ7jnA\nW8rHf8ZD18seDfxL+YXz4j6PeTBFLf6/DRDvzKS3pPz/K8DjK5b1bIoTiT3K56+iaO6f8/o4xcnd\nWRTXOo/oWf8SipOavr4kZ7ymqyhqndcBh1bc/3kU17Wnr7GvoKhBH9DHvr2vZapn/ckU15j3GOC1\nXEOZyCvuO30SsHrG576vv9VZyjwJ2ECNGiEVW7nKfY6laJV5J0ULwWVVPp88dFL57vIztYk++p1s\np5yHA58e9LWXZTyBolXkReXrmDeJUlyz/uNBPgfbKeuVwLnl46V1y3Op8btY6AB+FUiFP0qK2uNK\niua9rwHnlF8sa2ocf2+KjlmfB/5Tv/GUX6yfojirfyVFLWXWM+J+X1+f2z2a4trlIykS+ucoavKH\nUDTZHVl+gfdV+yoT5ln00SKynX1nJvLp92KQxHE8Re3+DcC36L9z2r4UrRGXAO8H/jtFIh2olkCR\niLcBTxxw/+PL4897QjbLazmToo/DORTNp9fWiOUEissVE/1+vma8jmvrfvmXf2NvoTihqHRy19RC\n0XfiHIq+MI+tuG/0PN6LnhObiuXsUb6fA3XILcvYv/xsXl/ldTSVcIFHUV72q/p5cml2Gdmx08tO\nHHtk5qayA9M5mXn3gGVNUHQ2OTEzz45ieMdNmTnvrU5l55eTKM6If0bRFH/1IHEMIiJOpmja/zJF\nbfo0imt2pwGPoDi5eENm/qKPsh5DUftb289r387+x5f7f5jiOuK6zPx+1XLKsp5PcR33SVlh1p/y\nVsHDKGr0t1Pc2nTDIDGU5e0yyHvRs/8JFCdGh2f1W4p2BqYoapCbgL/NooPXoLEsz6Jj6CD7nkBx\nUjFV9XX0lLEzRSvT9Zl54yBlLLSyE1qtL82ys+TvUcxo9S8DlrGEog/MB+t8JuqIiGWZuXkhjq2H\njGwSh4f+oCJiUWZubbDcyuWVf1SRmQ80FUefx50EfpvivvkvUtwmtQ9Fred9FAm5ShKsm7QGSr5t\nxDIs6iTPYdKV1zEMImJxZm6pWcaSzHywqZg0mkY6iatQDiRyBMXZ/UaKwU7+DfhQZl67APF0IvlK\n0rAziXdIeS/u5jZaJyRJw2esxwzvoM091+sGum4pSRod1sQlSRpR1sQlSRpRJnFJkkaUSVySpBFl\nEpckaUSZxCVJGlH/G/L+nI5W+eW/AAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "seq.plot_confusion_matrix(dev, local_2.predict(dev))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Clearly, and unsurprisingly, we are now doing very well at predicting '@' labels. We can check that the model learned what we expected by also replotting its weight. (Exercise: figure out what happened to \"Deja_fckn_Vu\"). " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk8AAAHBCAYAAABwlV9bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XeYJVWd//H3mcQkYIZhgIEBBiTnMOSMwAwMQYIIShJc\nJIqCApKDBBGJkoacQVSQIEmCCQUBAcWwuq5xdxV3dfP6W9f6/fH5FnX6zu3uW911uxv4vJ6nn+57\n+96qU+mc7wl1KhVFgZmZmZl1ZtRwJ8DMzMzsrcTBk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOz\nGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIzMzOrYUw3Frr44osXs2bN6saizczMzBr10ksv/aEo\niumdfr4rwdOsWbN48cUXu7FoMzMzs0allH5Z5/PutjMzMzOrwcGTmZmZWQ0OnszMzMxqcPBkZmZm\nVoODJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZmNTh4MjMzM6vBwZOZmZlZDQ6ezMzMzGpw8GRm\nZmZWg4MnMzMzsxrGDHcCBmrWyY80urxfXDiv0eWZmZnZ25NbnszMzMxqcPBkZmZmVoODJzMzM7Ma\nHDyZmZmZ1eDgyczMzKwGB09mZmZmNTh4MjMzM6vBwZOZmZlZDQ6ezMzMzGpw8GRmZmZWg4MnMzMz\nsxocPJmZmZnV4ODJzMzMrAYHT2ZmZmY1OHgyMzMzq8HBk5mZmVkNDp7MzMzManDwZGZmZlaDgycz\nMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIzMzOrwcGTmZmZWQ0OnszMzMxqcPBkZmZmVoOD\nJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZmNTh4MjMzM6vBwZOZmZlZDWM6+VBK6RfAvwP/B/yl\nKIrZ3UyUmZmZ2UjVUfAUtiuK4g9dS4mZmZnZW4C77czMzMxq6DR4KoCvppReSikd3s0EmZmZmY1k\nnXbbbVkUxW9TSksAT6aUflwUxdfzD0RQdTjAcsst13AyzczMzEaGjlqeiqL4bfz+PXA/sHGbz8wv\nimJ2URSzp0+f3mwqzczMzEaIfoOnlNKklNLC5d/ATsAPup0wMzMzs5Gok267JYH7U0rl5+8qiuKx\nrqbKzMzMbITqN3gqiuLnwLpDkBYzMzOzEc9TFZiZmZnV4ODJzMzMrAYHT2ZmZmY1OHgyMzMzq8HB\nk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIzMzOr\nwcGTmZmZWQ0OnszMzMxqcPBkZmZmVoODJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZmNTh4MjMz\nM6vBwZOZmZlZDQ6ezMzMzGpw8GRmZmZWg4MnMzMzsxocPJmZmZnV4ODJzMzMrAYHT2ZmZmY1OHgy\nMzMzq8HBk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4\neDIzMzOrwcGTmZmZWQ0OnszMzMxqcPBkZmZmVoODJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZm\nNTh4MjMzM6vBwZOZmZlZDQ6ezMzMzGroOHhKKY1OKX0vpfRwNxNkZmZmNpLVaXk6DvhRtxJiZmZm\n9lbQUfCUUpoJzANu6G5yzMzMzEa2TlueLgNOBP7a2wdSSoenlF5MKb34xhtvNJI4MzMzs5Gm3+Ap\npbQr8PuiKF7q63NFUcwvimJ2URSzp0+f3lgCzczMzEaSTlqetgB2Tyn9ArgH2D6ldEdXU2VmZmY2\nQvUbPBVF8cmiKGYWRTEL2A94uiiKA7qeMjMzM7MRyPM8mZmZmdUwps6Hi6J4Fni2KykxMzMzewtw\ny5OZmZlZDQ6ezMzMzGpw8GRmZmZWg4MnMzMzsxocPJmZmZnV4ODJzMzMrAYHT2ZmZmY1OHgyMzMz\nq8HBk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIz\nMzOrwcGTmZmZWQ0OnszMzMxqcPBkZmZmVoODJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZmNTh4\nMjMzM6vBwZOZmZlZDQ6ezMzMzGpw8GRmZmZWg4MnMzMzsxocPJmZmZnV4ODJzMzMrAYHT2ZmZmY1\nOHgyMzMzq8HBk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZm\nZjU4eDIzMzOrwcGTmZmZWQ0OnszMzMxqcPBkZmZmVoODJzMzM7MaHDyZmZmZ1eDgyczMzKyGfoOn\nlNL4lNILKaVXU0qvp5TOHoqEmZmZmY1EYzr4zJ+B7Yui+I+U0ljgmymlR4ui+E6X02ZmZmY24vQb\nPBVFUQD/ES/Hxk/RzUSZmZmZjVQdjXlKKY1OKb0C/B54siiK59t85vCU0osppRffeOONptNpZmZm\nNiJ0FDwVRfF/RVGsB8wENk4prdXmM/OLophdFMXs6dOnN51OMzMzsxGh1t12RVH8CXgGmNud5JiZ\nmZmNbJ3cbTc9pTQl/p4A7Aj8uNsJMzMzMxuJOrnbbgZwa0ppNAq2Pl8UxcPdTZaZmZnZyNTJ3Xav\nAesPQVrMzMzMRjzPMG5mZmZWg4MnMzMzsxocPJmZmZnV4ODJzMzMrAYHT2ZmZmY1OHgyMzMzq8HB\nk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIzMzOr\nwcGTmZmZWQ0OnszMzMxqcPBkZmZmVoODJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZmNTh4MjMz\nM6vBwZOZmZlZDQ6ezMzMzGpw8GRmZmZWg4MnMzMzsxocPJmZmZnV4ODJzMzMrAYHT2ZmZmY1OHgy\nMzMzq8HBk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4\neDIzMzOrwcGTmZmZWQ0OnszMzMxqcPBkZmZmVoODJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZm\nNTh4MjMzM6uh3+AppbRsSumZlNIPU0qvp5SOG4qEmZmZmY1EYzr4zF+AE4qieDmltDDwUkrpyaIo\nftjltJmZmZmNOP22PBVF8Y9FUbwcf/878CNgmW4nzMzMzGwkqjXmKaU0C1gfeL7N/w5PKb2YUnrx\njTfeaCZ1ZmZmZiNMx8FTSmky8EXgo0VR/Fvr/4uimF8UxeyiKGZPnz69yTSamZmZjRgdBU8ppbEo\ncLqzKIovdTdJZmZmZiNXJ3fbJeBG4EdFUVzS/SSZmZmZjVydtDxtARwIbJ9SeiV+dulyuszMzMxG\npH6nKiiK4ptAGoK0mJmZmY14nmHczMzMrAYHT2ZmZmY1OHgyMzMzq8HBk5mZmVkNDp7MzMzManDw\nZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIzMzOrwcGTmZmZWQ0OnszMzMxq\ncPBkZmZmVsOY4U7ASDfr5EcaXd4vLpzX6PLMzMxsaLnlyczMzKwGB09mZmZmNTh4MjMzM6vBwZOZ\nmZlZDQ6ezMzMzGpw8GRmZmZWg4MnMzMzsxocPJmZmZnV4ODJzMzMrAYHT2ZmZmY1OHgyMzMzq8HB\nk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIzMzOr\nwcGTmZmZWQ0OnszMzMxqcPBkZmZmVoODJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZmNTh4MjMz\nM6vBwZOZmZlZDQ6ezMzMzGpw8GRmZmZWg4MnMzMzsxr6DZ5SSjellH6fUvrBUCTIzMzMbCTrpOXp\nFmBul9NhZmZm9pbQb/BUFMXXgX8ZgrSYmZmZjXiNjXlKKR2eUnoxpfTiG2+80dRizczMzEaUxoKn\noijmF0UxuyiK2dOnT29qsWZmZmYjiu+2MzMzM6vBwZOZmZlZDZ1MVXA38G1g1ZTSb1JKh3U/WWZm\nZmYj05j+PlAUxf5DkRAzMzOztwJ325mZmZnV4ODJzMzMrAYHT2ZmZmY1OHgyMzMzq8HBk5mZmVkN\nDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIzMzOrwcGTmZmZ\nWQ0OnszMzMxqcPBkZmZmVoODJzMzM7MaHDyZmZmZ1eDgyczMzKwGB09mZmZmNTh4MjMzM6vBwZOZ\nmZlZDQ6ezMzMzGpw8GRmZmZWg4MnMzMzsxocPJmZmZnV4ODJzMzMrAYHT2ZmZmY1OHgyMzMzq8HB\nk5mZmVkNDp7MzMzManDwZGZmZlaDgyczMzOzGhw8mZmZmdXg4MnMzMysBgdPZmZmZjU4eDIzMzOr\nwcGTmZmZWQ0OnszMzMxqcPBkZmZmVsOY4U6AwayTH2l8mb+4cF7jyzQzMzO3PJmZmZnV4uDJzMzM\nrAYHT2ZmZmY1OHgyMzMzq8HBk5mZmVkNvtvuHaTpu/ra3dE3VHcOvlXX83bflqFaz9tpn72dtmWo\n1vN222f21tNRy1NKaW5K6ScppZ+llE7udqLMzMzMRqp+W55SSqOBq4Adgd8A300pPVgUxQ+7nTgz\nM7O3s7dqK9pQrWekttZ10vK0MfCzoih+XhTF/wPuAfbobrLMzMzMRqZUFEXfH0hpH2BuURQfitcH\nApsURXFMy+cOBw6Pl6sCP2k+uQOyOPCHt8l63k7bMlTreTtty1Ctx9vyzl6Pt+WdvZ6307bUsXxR\nFNM7/XBjA8aLopgPzG9qeU1JKb1YFMXst8N63k7bMlTreTtty1Ctx9vyzl6Pt+WdvZ6307Z0Uyfd\ndr8Fls1ez4z3zMzMzN5xOgmevgusnFJaIaU0DtgPeLC7yTIzMzMbmfrttiuK4i8ppWOAx4HRwE1F\nUbze9ZQ1Z6i6EodiPW+nbRmq9bydtmWo1uNteWevx9vyzl7P22lbuqbfAeNmZmZmVvHjWczMzMxq\ncPBkZh1JKaXhToOZ2Ujg4MnM+pRSmghQFEXhAMrMzMGTa9NmfUgpLQaclFKaBw6guqHcnyN5v2Zp\nnDrcabGRK6W0dEppkQF+d0pKaZmm09TBegd0/b1jg6eU0kLwZmGw+HCn550upbRQTIUxlOssL5oh\nXW++7reAccBYYKuU0o4wvAFUSuntmGfNAu3XgS4gO5cXaihNPZYdx3xH4KyU0qJNr6PTdDS0nAlN\nLGeA6669DdmxHd18ipqTUloJ+DvgkwP47njgfOCAlNKy/X2+KVE5XA6q66/TY/R2zIj6FSfhNiml\nD6eUdgBOeavXqFJKU4Z5/Z9MKX08pbR73Ys8pbQK8DngoJTSUt1J4YKiQJgHXJVSuqKb62rdJ3Uv\n1LrrSCmNid+DusaLovgn9GDwfwd2Gs4AKqU0qiiKvyZZvIlCfLiD2JTSJODulNKcwSwnjscc4JqU\n0vGx3EbEsmcDc4H7i6L416aW3U4WLMxOKW2SUtqwTEcDy14D+EpKacZQHftsezYF3l1em51+N/b/\nTsBpKaVjUkpLdyutA5VSWg24GU1p9Nd4r+P9WxTF/wD3AasBe6aUlutGOnMppfnA9cBTKaXLU0rv\nj7R0lLe9I4MndHBfBw4E7gbuK4rijyM9su9NSmlF4IyU0nbDtP4rgU3QPv0ocF6kqZPvrgHcDrwC\nPB2F9ZCIAuEc4BFgnZTSXd0IQqPQ/7+U0qiU0oUppQ+llDZPKY1uMgjJ1nE5KkTXKYrirwNI76yU\n0hHx95iiKH4L3AH8CZgzHAFUFCJ/jfV9A2XUn00pbVlnGfF71ZTSxtBMgVxXHtAWRfGfwBeBRVr/\nV3OZmwLnon3zXuDklNKsBtI6Ogr7y4Ddgd/H+1077nFe7Q5cA8wBPp1SGvTD6FNKqwI3AXcXRfGP\nQ3XsY3vmAncC/wX8X83vbof2/1eBU4Gj6gRg3ZZSWh24DbgCOAV4T7TudXSOlOd8URTPoOOzMV0O\noFJKNwNLAAcBOwG/ArZOKR0baen33HhHBk+xY/4tfl4B3h3vd3xSD7eWi2c88B/AzimlrYc4Hcui\n2sJhRVE8CpwEHAp8oL9WpKTu0tuAa4qiuKooip/H+/NSSht1Od0rA0cBDxRF8UBRFNuiSWM/13Qr\nZBT6o4Angf8GZgNnoJa2MYMNQlqC/mvQdvwEeCiltEXNZY0FzgSuTimdDxydUlqmKIpfAjcC/wLs\nEjXhIQk+yiAzXs4BngD+BngZjcfq6JyP/bwrClaOTSk9llJaZ6gqTSmlRVJKC8X5sG72r+8D56aU\nlh1gsLsCcDJwR1EUN6Pg6V3AYfG/gaS1PB/HF0XxF7Tf/w44Brp73FNK04CjgR2AfwQmAM8N5jhF\nK+VtwNfjOawktZJvFRW4rkgyDfgYcGhRFM/V/O5oYGfgCOB/0aPRro3Jq4e9sh/79YxI033AP6M8\n4n+zyk5f3y8rRWtHsPQDVKHdCAVhjXfhxXqWBPYsiuI/o9y5Fvg6sGa0ovXrHRU8ZTXPxYui+Pei\nKHYBDgNWTyl9Ov63XLRIjEgppZkppSXj4tkwpTS2KIofoha0PwJ7dDvwaPFbFLVvG/v3NeA7qCVq\n936+OwZ4pSiKW7JjczTwGdSqMLd7yWYRVDPaOKW0AUBRFPsCU4DrmqrZZZnH4cBXiqI4G1gL+Cdg\nG2DfMoAa4PLzVq2tgJ8VRXF0URQXAxcAN9VpnSmK4n9R7e+HKLOeDjyQUnov2l8XAL9Dx3vdXhfU\nkJbtOw3VFH8VLZR3Aw8DJ5atYf0sazYKDHcE7kfn6CnA2t1uQYtC5mTgkKTWzaNTSk+nlPZBFbhL\nUbfYQFqfpqJu1T1TSqsURfEb4OPAOsCHk8aT1JJ1A96eUjo10rYXsGFK6ZK6y6vpL8A/AAej431I\nURRvANulDlu0W0VX44PAcimlLVJKTwL7o8reR1OXWu3juv4jCgL/K47taHizhXeB8ZZZS0wRFfqf\nAMehoQ17F0Xxm5TSgajnZLj9D3BGURQ3ARRF8TvUsrZRXLtFUkvvyu2+HP/fBbgVOABVjP4TVQLX\nB97XhRaoP6K8bNmkltXR0QJ8P7AUcR325x0VPMWB2g34ckrp7pTSmUVR/Aq4CJiVUvoy2oEjUtQ0\njgI+EoX7R4AnI4D6EcocVgSOi4K0m2nZPqW0VdSUvwnsigqzZ4G7UEFxcEpp0T4Kg0WB7VNKq8Wx\nGQ+sDOwDXIwKmskNpbcMztaKmuY/oozzJ8CuZSBQFMWuwIVR2x7M+kbH8sqg6D7UlXYN8IWiKA5B\nF+rBQMfBTes6strdY2ifHZZSOicyrmuBS9AYj1X6WdbC2ctvo2P5g6IoTgO+hB6l8HkUbHwVWBzV\nDrsq275b0cDOiaiCsFxRFH8E7gWeBtbuYHF/h2rwa6BBrWsABeoC7Pa2/Bfwc2AldK18GLgatRA9\nDewLvAe0zX0tKDuXV41g4vvAJ1Cl5YiU0spFUfwDyivuifEktSS15l0cP2sABxRF8V+oi2OHlNJV\ndZfZwTrXSSltHoHOG6iL6piiKH6aUtoWncu1bu5IKS1RVoaLojgPeBV1n71SFMX+6Dj8EVi1we0o\nj8+kWO9fUUC4c1EUf42K72zgU6iyVn6vvImpbInZLKnV6ufADODTRVH8MvKqk1AeNmyi3Pkz8M8R\nhJTH5j+BWbEdmwMPEd3SbZaxHGq52g0dh/8D/lwUxbeA64ANaChOSSkdmlJaGwV8E4Hdi6L4v6ic\nTSiK4r9RXtfZTRdFUbxjflD33CsowDgRjX26NP43HTgd2GG409nPNhwE3Bl/j0aF2iPAQvHex1Hz\n9GpdTMO1KMO/H7gs3puNmpd3iteTIx1je1lG+WigCyPNS8XrcfF7FzQWanKD6Z4D/G2k/zfA1qgw\nuwi1qKzf0HrKbRuFCrVNs+37PLBP/H03asofyDomletCgeqJ8XpPND7ihCwd2/WzrIVRQf7B7L3j\nUPfWDOB51O3wLjSmZjxqwbkLtR6mLp/z5wGPZ6/no4BnhXg9obdjEH8vDiyZvT4FODPbX9/u8vUy\nOn7vgDLn51Arzth4f8M4Xq8BR3Z4bs1FrTN3o9r6ZGB5VCBfC6wyyDTvB2wHbIYeDr9cvD85zpfN\nG9w/5TadjSqAm6Jg9kLgGVRJfB3YreZy10T5/VPAk9n7O+bnTJzbF7aeN4Pclp2BR+M62Q21Dj4b\n5+1lcaz3zL63JAroV0IB6m/i2P4Q2AqVV7fEsX4O2KNb52sH2zg1+3v9yCcWyd47CXg/avl8CZjX\nx7Kmo7y3vA5XjvfnoPJtkYbSvAdR3qNK2Doo+Dy85XMPAx/vaJnDdQCG6aDvCKwLzEP9m2ujPtrL\nWz7X1cKgge14ATgr/p6AIvQXUYvNq8BmXVz3ccCj8fdKceKPat1/qLZxXQfL2xPd8fBRqsJwM1SL\n3qnBdE8BvgVsH6/fA/w9Gpy4MvDZ8sId5HrKzDOhQOnxyFxOBZZBQeHvgK+hgau1zznU1bxJ/L0B\n8Gfgini9cGQUV5XnSPa9Ub0sb3E0Tu0W4APZ+w9HhvOJ7L08SF+nS+fYqOzvCai16HfA+7P3bwC+\nDExvs++XAPaLv+fEufQ68L54bx4qyM6Oa2nLbl0vWdq2QYH7rmjs2NXAIWhMEaig2J0Igtt8f1z2\n9xpocO7m8fqKOLcnoWkPLgLWqJm+0S2v90AF+CvAYvHeXDQeZXSdZXew7vwYnoEqTVsAiwEfQt05\n29S5TuKc/hpVReWpdsc51vN9Il9oaHt2QPnw5ihv/j6q9C4Ux/hD2bErz9llUIvYeaiVtfz/YXGe\nrxHXwopU+eSQl1No2pK/BU6P10sA8+PvUfH7SBSYvEIvgROwArBe/P0suhllSrzeCpXPKzaY7pnA\nV1Dr5ekoz98Y3QBxRZzXD6Fxg50tc6h3/nD+oCbf8aipf/d477NoEO+q9FK4DHOal0RB36TsvW3i\nolwqe+88VGDu2uX0HAwcF3+figq1u9BAzPwzCwQGvWW6VN10P0J93a/TYM0K1TKWjGWvR9UScBRV\nK96UBtYzOvt7TeDs+HszVIs+C7XkzCILDOtmgsAG8fvd8Xs71LRfntMTUMvBwf0sZwowJv6ejAqp\nO4CD4r05LcdxTG/HsMFjVR6bFPuwLCj2Bx5AYz7Kzx7WyzIORAHKsSh4XRO1BLwW+2UKavm5Bdil\ny9tTnvsnAJ+Kv8ehAuYxVKiOzz7zHMqj8pazJVD30sKou+Ep1FW+RvaZy1GBPblcXofpm5L9vWWk\naxNUMTo/9tHk+N8P6KMVYYD7Z1k0xm6f7L1zYz9swQDzZGAa6ipfO16/jvKpzwPbx7m8Tlw3jW1T\nHLsj0E00c1BwvjcKbo/o55yfgcY1vQwcmP3/DFSRGdPNc7XGNq6DWj1PREMv5rf8fy0UMM7p5VrY\nAnX9fxG1Aq2HAsRrUdnxCg3l/8DE7O9LUUvmmaiFdmb8HIYqg0dln+33vBv2AzFMB/9C1NKxd1xQ\ntWppQ5zW04B7UCS+Hmr+XThOvve2fLbs8upajQR1fb6Kakf/BLwr3v8iEbXTs+VgFKplTIzXY7L/\n5QXEGNT1tzawelPbERfqN+OCvhENRi8zq12Am5pYF1WtaxSq8X4B+HVLOi5CNZ/FW7/X4Try4Gwr\n1IR/cLzeGfgZ0RVA1l3abttQofUj1OW7V+z7MSi4mI/G4IxDhc4xQ3Sul8dlFOoe/Dy6GeHDkb59\nUSHygZbvpZbfY1Grzg3Aw9nn5qCC6YB23294W1LL693Q3ZZrZu99O/b1svH6ENrkRajVYSVUAVgG\ntT48ChxPz+6Sq6nR6owCsdti/24Yx/oWlCdeEefC6ajb9kmi26yp/YWC2FGo22w+WYGJWtLvYACV\nGhR4T0Yti4/Efr4Vte6dhbrDJpf7drDb1OZYj0XB21eoWle+jALCFXu5HtdDLVbjY99fCMyO/22N\nWuOGNXiiZ369MvAH1A39pTiGR6IbYw4D1u1l3+yEAspjUHfwhajrb0mUN36caujHYPPkm+Mc2iNe\nL4yGOByIekwuzK/H7Hsd5cnDdiCG+STYD7U4vUzPfucR110XF/zEuOgfjsxtJ1TwvwDMGIo0tLxe\nKi72s7P31qSX7s+4KH5PNU5ngQCqG/seBWLXEf3aKPB8Ki6qT6MaTq1xFB2s88K4MBdC48LyFrnt\niO6kwR6DeG9bVBAcEq/nooHJG3awvEUj8/oT6or9ESpsLo3M73409mRLGuzS6CBdY9CYpM/F622A\nK9HUBKMjcz6hl++W59Ly8Xt/VGh9kCp4nxfbuky3rvcsHZvGuleJ8+E8VBnaDAVDzxKFay/LyQPg\nqXHOXhzX38qoAvUxBthqStWNdBeqnJVdRRuisXofi9fTgIXzbWtg36wB/DjOr4RageejaRZWQwXy\npoNc18xYz83AFtn7TxHdgA0e811Ry/aR2fn3MBomsjGqXC4wLCC2fRwKnO9HecQiqBfhK6gl6iVg\nr26cqwM4bnOAk+LvVYBfot6H3VHlcD4xpKCX7byZGFsZx+cGFBjObDi9i6Aepp/GeXYUaom+IPbx\nNNSyeiWw0oDWMZwHpAsHeAIxDiMuwHf18dnRwKL5iTHSflrTFRnB3qipeT6K/Od0OQ15a8CpqItt\na5SZv4YKtzGoleDKPpZzSZzEE+L1ArUoGuoSyi70A1Fz+eeoBmxPRi0YHwS2bvL4owL+O2QtG5GB\nfrO/Y9vh9oyKTPj8yHCmohaN26kCqD7HIaEWpw/H31NQrfwUFEzsiFo5n0R3vbwATGtyH/WSplvo\n2T14EhE8xevt0Pi0pcjG/vSyrLkoOFotrvEPoQG6B1EFUIs3vQ1t0lGm+XpUuGxGFZQ8j1oh9uzj\n++PiOlsfVZb2i9cXoC6HpVAA9hyqrQ/o2on1zIm0Xpa9vw8qzLtRqdkV5V/PoFbs7eL9A+O9H9Dg\n8IO41o8ttmbgAAAgAElEQVSJc3/lWP7qDS5/+chnTo7jcy8ar3VsXEs/IuuW7GUZy8Q5emecO5NQ\nxe9uqtarYSmnqFrVt0PByNbZ/96FxsZ9qMNlfRK1/pfj6FaMc+/U/q7tAR6Xj6Hu8SNRd+q/ojxv\nEgps+zwufS5/OA5GFw/yLNSidB3qOlmhzWf6HH8zEn9aLxrUN/4BVFB3vVUAFdpfQTXfs9BtvqBB\nxj9FTeM35+kl7sppWc4/o9aOdi1QZZA2BdhxMPsJWDp7b05kSPvScKHZeg6hmthnUK1xs+z9F4gB\nloNc31fQuJgtgF9QjXnaIzKIrfJj1ssy1kEtrsfG6+moBe687DMzgfcxBHeeooG9z6OB6TfHe5uj\n8Q+bZp97oN313LKs1VBBtWX23hjUkjY/ztdRve2bBrdpFdRdsE28Pgy1HpTB+jSqYL5tgYhahfZF\nY7Z+TbScoLvQLkKthEsTA19rpq+8TmYQg7VRS/Yj2XmxAQoIlqyz7A7WvSzqXt481v9+ND5m2/j/\nJBocKBzLfDfqtnsI+B59BK0D2IeroLGCh8brZVAeeUcc57FEfkRLKzsaZ/sTqrsul6ZqgdoKtZ4s\n0K00VD+xLeV5Ohq1rH0oe12Ol1wTTTOwCu17FdZErbDjUGVgPrphZwIKvp4ku6FjkGluzZOXi2vl\nLtRtt2Wsu7Vsqh2YDstB6eLBHoVq0f8OXJK/324Hx8k5oqcmaLeNbbaj27eK70nccYXGouQD66YB\ny7Qcg/LW93zQ4/1xEl8eGcabAVS2HYvG8rcZRFp3QWOcLkS1jvIupltRzXbqQJfdyzk0Cg2c3AXV\nopZEtc/zaPCuR1SonBXr+ypwfLxfjjnrtfsn/r8qVYvTeqhL8aPxejoq3D/T5ntdr+2igvRSFPh8\nPt47GTXpX4/Gj93bwXJmA7eW6aYaAzgqjn1XC6JYz5g4715E44XKWvshKGiY28ly4vdyKLD4Imp1\nKgujDeI6Oo8B1tZRAfIsmhrgXNSKNRe1Dj+NxlTt3uC+yQO2L2Xvj45teZ2oDAxiHQtMWZH9b/nY\nxjUHe15n27ITystejH25RLaNF6I8L58SYTLRRURUDFDL0stUgcgKqPXtS0TrzHD8xHn8YTRWtLyO\nziGm0qAK+Mp5mCb2spw5aCLlL6FWxnGoPLkBdZ++jgKoY8imShlgmvM8eR5VhWMaalS5lQYr0MNy\nYLpwoMuTeVEU2X40Tsqjs8+UhXXewvE0Xbytf6i3v8HltUbv81B3zleBj5T7Ez1CYdE8HVQDMQ+J\nC2Rf1CpyUfa5cp6l/A7CKbH82reNZ8d/S9SVuHKs40U0RmQs6oa4hwbHiMVF+ijK/C9EA7jXRa0p\n56PuwpVa09nhsse2vJ6Aumj+QHYXHery2ryvdUQ6y2fTfTAyxtYAanFUcH52iM7ZeVnalkKB9RJx\nDpR3QK6Hxiocnn2v132IWsxeJRvHhoLa47q8LeX5l1cI/gaNp8jvIjuUrHWwn2Vtj1oUZ6BC7DLi\nTkPUKjUXWHWA6V0bjXGajO7kej7OrzGoovEgkS/WOWf72Z58YPtjZHdoocD2VhQkLjHA9axBVTkY\n3br+Np8fVM8DChoeRQHZWBQoXUAUzqgVaaWW76wQecQFqKJQ3hhze5y3Y1HL8BcGemwbPq/HoYpV\nOd/b7qhiunKcK+uinoS2c6ShCtuNwEbx+gbUyl1Od7IRaoncKfbHgLc5O8/KXpLLUffptfH+FNRi\n+xhZmTWo/TPcB6jBA70rimKXiAO7c5zQhwCro1pgOcZpCmoq7DMjG+L0lwd/3diWXi/uwV74/aQj\nv2vsAhSMjkPdoPlEcw8QE4xm762Oxj6VtZPDYj8/2GY9N1B1a0xCgU6t44FqFItT1YLeiwqGnVBL\nyu6oqf4SVOAMKGPuY/0noPE5o1EBdCOqNa6JAoL3DnC5ZQA6lqpVazQaN/McKoSnxr6+vcNlHogm\n3LsCBVCjqAKovAtvKOY8OgV1012BxiEshMYCXRX//xbRAtXu3OxlmeX1czAalHoCGqPxCl2cviNb\nbzkp4lkoUE+ocnEZ2fxU+Xf6WOZ7UPfSzvF6CqoQXoYqAy+iGZw7TePSwMXZ601ifx8c59OK8f6q\ncV403b29EwoarkFjW6bEdfnFSMNPUJ53HQNsGUbB5M/Iuuxb/l9Wmgc9riaukw+iHo5t470ZqHXl\n8nb7LztPjkbzsp3e8v87UOvV3wLv6db52uH2lfnpYmgozGfQXZkTUWXmcVQR/S5tukBRXjUZVex6\nzNeHuux+TZXHLYN6G9YeRHpXK/cx6vH4JGpIeQ6VA1+K/0+jga7BN9c7nAepwYO9eVyAZVPoxPjZ\nAUWhvySay+OgPs8ICpyy7dgxtuM5FPitS+9djouiDLvx8RtxEj6BulLK9a2HovZH0diI+S2fXyNO\n1AOJW6/jf+9H87jsTptm6PjuCtSccDHW9624kMsurPGoIL6Zqnn+dtQKOegBoizYIrdCHIcvAqfE\ne0+jWtTy+TbWWMf74tgvFul+AAWup6FuqU0i07oZuLqvdRCtCfH3mFjevZGplXcgrhvXw/FDeJ6v\nh2qwd6Gup6fi3P8W0TKIHqXysQEsewk0xuUrqPWx0dvr250LkfZ8UsTXiDl9UBfeVWRzsvWz3MVi\nf8xEgcyGcQ2NRUHVldTsTkN54RpUd4Etg4KX71J1I+0S+7/pCsYGKCDYBQWzd0R+MA4NfD8DtbZs\ngcYG1hpjRTanFepWOg7lKflt9Xlvw8P0EmB1uL4VUYVpcdRa+jBVy8qMeN1bS8yyqMvqADRWtHWq\nmcWyYzQcE2DmU6isjfLWReN8+RQaO7pwXGNrkU3zgIKsctxXWQFfEwVLp9JzTrKbiKAzXg/4KRLo\nhqVvU910sH6c74+iQHUiykseb/ne4KfBGeoD1OCBzmdmnYsi4k3RqPpXUbfJzDjYb84bhAqgdYc7\n/W22Z3UUlKwSry9BLRlvBlD0DJy+Q3b7bcNp2QG4v82+Ho3GXuS3/Y5CGeFd9D5p4d+gwn4/Gmgy\njX31XXTn4QYo010h+/9X44JdP/bTWg3um1GoJr1x9t5NVM3182mZR6jOOY1q/xejwv/ieH9N1Opw\nGtn4sjxNbd6bhQqt86la+LZHrXNlN2w5+HN92txe3IXz6nCqVskNUSvdLqgwOQF1Gw56MG92nMrr\npunAaXHUwjQzXh8V52Q+KeI3s/27fIfLXQQFuS/EOXBT5AFvAKe1nisDOLfuono6wLEomD4srsvX\n6UILHWot/Ux2TCajFpq8q3kLdAdc3QrUhmjCw5Ni2bsB17Rsc55nPsUgK81xDX0XtfwuhObaup+q\n4r5Ay1akYwkUWJfd5Duiu6bnxjJvb/fdofpBwfmDVE8qmEw22zYaM3ZufGZWm+/egVqTz0Hjl8ob\nEVaP8/hUWlqXBnt9xn4dj8qXh4k8Ofb1LfG/hPLOsxrfZ8N1sAZ5oMudvgPquvgAKmyeRrcmz0GF\nw9ZtvjtipiWgCkoWRs3yv6LnGImLI4Nbv00m0Fj3Cgu2qKyJWjyWoxosuAgt865k6R+ParLbtryf\n1/4OQpl3RzXwPtI6BWW+d2bv/SD203moZWZFVIteYCLRAa7zIqoulKdRjexB4Ol47+ZI09eIAct1\nzzWqpvKEanWfQ7XTqfH+2qjV4bPEc8b6WkecMy+iQvyHKEA5Pa6TpdH4sC/Ry6zHXTrfD0KZ7MHx\neisUQH0AFURTaOnyaDmHyvNqMfoJwrt5naN5e8pxbtNQwd1uUsTvkLXC9rPMFVFL70qolegqqkcJ\nlXcfdjxzeMv+KvOOicQkmPF63zi3r6a5iQlb7wzeCD2qY8PsvavIuqZQELRcneXHd05DLbU3ovzq\nADQm8EMt35ka59mA80yyu7NQN9ZFcdyXQMHbIyhv7mu4xd6ocnVMvN4JddU9xyBumW/wvF4NlS0X\nxLV4Tcv/y4f4btDmuzugFvezYt88j3obpsTP3SiAauRZpfS8ceo81FPyBJr7bizKo08kbh7q7fwc\nVBqG+4DV3GH5VOtroGi9vP13eWJQYmREL9PBZIHDtB15gfBuVHgsj+4wmk/2IFeUSZePGFgYFYhN\nBk5vRv/E4zDi5LsHdcGVzfr30fLAxLiYyttwL6YaBFxm1qOJ51PF6wE3l8f3Z6Ga8omoGfkAVOO4\nErUofhw1LU9ENafyWUmDLRA+hJp+z6fnxKD3Ud0dtic9n71W/9ZX7ffPoeBpGVSo3UQVQG1Adgdj\nL8uYQfX4nK1QAXMDGnR8Apq/6dT4/270c5deQ+dY3sW1Cz0n9tyCqtLTa82bqtB8D8oYH0PdWQtk\nxtn5N4mGJ9/L1rEvGuR8HipAR9HBpIh9LG+dWNbF9Ozi2BXdcddxq1Bcc+VdmHNRYX8+Goc3DQVQ\nt2efb3p+nZ1QgLQ7Cor3j+tnV1S4fY8BPFg4OwfmoPl6Hmo5Hh9AFY6r8nMBTbGy3SC2ZzXUE3Ax\nyie3jv1ZdonPoJcxaHFcP9VyPG9EleWEAq4Z+fYN5w8aDP4NVPF8Ho0Z/hjKd3fv5XpLqMX0dKon\nHJyNpqb5BmqZ3Y6Gn4UZ6/0qqlDOjfU/EsdrLdQLdVbr+dPY+of7YNXYUeugwKKc0v+zkakcRs8A\nYDcUOA3roLs+tmMJVOMvB68fQ1UTXylO1Gto81Bc1PqzQNQ/iLSUF39CEfojccHsjG4fvR21KD1O\ny8Bk1L30XeLZYOjJ5z+mJUBCXamPMMj5YmJ9r1K1AB0dF85j2WdmxLoaedwOynjLFqG56A7Bm1o+\n8yAtk7FSYxxanMf52KUHqCYSXTX+fwsxWWX2uXZjnBZCLW+PUo0DezcqPE+P62ZX4rEPQ3zej0YF\nXHmnWB5AbY5aDxdoKW5ZxrtRl9biaLD5byODnJyvJ35PQRWNRrpsI91lPjMv0nELqqlfhMaXnYBu\nkPghHbYkAOtnf6+Fgt2LUWVgQlx7HY9xiu+cgwqvvYj5c+I6vhwF1AujrqaH8n02yP2TT6PwXHbe\nnoOCtr1Qa9wX62xPm/XsiMZWzkVByKSW/5ePmXnzwbtkd/rV3Z7sXNoUBcs3xL79PdmULX0sZ9PY\n7rzS9SE0DvdYhrGrruW4LU20/qEA6nE0IH5fFHxeTj8BKKq83k01d9vH4pz+NgOcxbvNOvJZ90fT\nc9qLJVCr2RO03EVPN8YGD+eBq7HDlkez9B4cO2guGmdzPqoRbJB9djZV3+ewR/JttqWcov7xyMSO\nIxsYGyfxyWh+m2m0NL13IT0J9RmfG6/noFpi2Yq0Aj0nfByFCvUXaJmXA3Vj/Bhl0OujQvH7DPIh\nj2jytV/T8xb98WgMzcXAvvHeymiw7qALTBTA3oIK4I+gwZ7l5JQ7Zp/7TuuFWnM9M1F37Wfj9dNk\ngSZq3buKuA27j+WsiFqtFkPjTD5PtBSi2v61kZktNITn+psDd2Pf3YzGiIylmrz0kPLYdbC8/WLb\ndkfdsu+Pc/WTKMMug5tFafBuWtQiewXKeyagwrDMY7ZFgcJZsV2TaJkUsY/ljo598ET23rqx/FtR\nDXpiJ8tqWe4OqMC7DTg53huHulzKaSAm0cDYT7KgHnUtv0zMnYcG856PAqhyDMyb3dMDWNdCqGuz\n7M58jaqFbUmqSseVqFI94LE02X68KM6vleO9zVBL2g/iGE2Btl3LS1BNMLlxnPtlHrsa6uIdtgkw\nW7Z3N1Qx/TFVnrESqoSd2s8+2oaezyR8HLVufyJ7r1Z3cx/pXB+VBeNRK+0YVOn/TPaZ96BKddtH\nODW634b7wHWwwxIa5PppVIP5GIpud0AZ2SXxv64PeG1wmxZCg9geRrdtHxYX1EzU4rMVHQ4yHeD6\n30d0aaLA6Wf5yYZq+H9HDO5tORZLoZrd8dl791E9fPEkNDDy67F95fsDzchWR4HaD+NCyTOqCagF\n6kKUSX+PBp5Vh8blPIBuST4IDUp9KI7NTmig562oRaejqQLarCPvyloCBYeXogJv38jQtorf/c64\njAqu+7Lzawt6BlBboy7A02m5G6lL59gCtX0U9FyKWmhGoxbO1pnR27Wo5XdULYy6lMsButejsWZl\nwDIlXjfStY0y6qdQC9eE2HdPkY0Vi//9CAXy/T0+pnVM0Pg45m8+CgUV1rfRy11bfSw7HweyZZyj\nT5JVJlCLwIBvC29Z39i49mfF6+XQ3cJfbknHpXF9Tq573mX7ZClauozQnaPjUWvTvbH+cl6iQd1d\nS1VR2h+N1bwUjVkq07MeGke1wPP3UAH+HdQyf3nsg03jXH8y9lFXH6tVYzvLG5VWjb9fzvKMVVEl\nZQ3a35RSTqmRT0WwGzGmLl6PqXvM+0jr1qhy+TzVA91Xi2N/eZwL9xJTr3R93w33wetwp01Fgw7/\nhAqUQ1AkXwZQV6MgqpHJr7qQ/rwGvhGqyU5ENda/osDgStTc+GLdTLNmWsZT3dZZ3mJ7Bsq830VV\ne9+ZbG6Y7LuTUO3io6ib7CGy2dyzz06hGoM20MBpOtnda3FhvNzymQmo9e5bNBM4rUfckZS9t3Rs\n7+3x+j3A/yNreWuXufSxjnwm3GNRIbRknAd/RcHFXShIOD37XrvAYhwKRLZEg0/LZY9BBcC9VIHu\ndgxBbRcFbeXDP69u2YZdUXB6XKRxgYIctbiWU03MQ90zedfmZaglbQ6qHecto3uS3Qbd0PYchArk\n36EAYDcUiJZd1puiFqQ+C2yqPGAu6uI4IV6XA7m/hSbG/B41H4qbLXtNNF5uPLqeL0fX98aoZfZH\nNPj4k1jPKsAZ8XomKuDyZxNuzeAmQNwddQU+GPu9HMt4Oco3Xya6AuOa6nWm8U72Y/wcT9VqNxm1\nct9Az0D+erIWlnhvBRQ4rU41MeN58fcEFIxt1OT5OYhtnYFa2L+V7dO1UWW1nHqlt5nDF4/jPDP2\n17qxbRNRcHhog+nMKwWfA/6BqHChiuK7UGvT3fQcy9fdCuJwH8BOdx5q5vwVmoBuMuo3vj4yogn9\nZVzDmPa8paR8AOemVLdZng18PfvMgJ6Q3mFa8jl/9kUZdjmG6KI4+VajfS1j1bjQFoqL7suoBar1\njoz16PDOmX7Suhjqv57X8v69wEst7y1Eddv4YAeHr0N1K3Lev74KGqtWTiUxO/vfQLogRqEWjEuo\nuhvKWb5P7nAZ09C4tA3RWKeH6dmiNRF1MzwIfHKIzvcNUAvhTDQ4vJyX6rjsM5ehQG/n1n0Yx/JM\n1Jq8D9Vkp0+hoGwxFAhchibAHHTA3ME27Y/Gf3wBVR4mosL0e6hA/UW+Lf0saxfUlb0lujPsWqqu\nrEtQZaHWNlFVeHYC/gm1Wt2LWmtWjzT+BOWhg+pC72X9K6PxgOWNCMuioPbGBpa9HmrFnoYGHn+f\n6gaKS+K4lM94HPC4Fha84/i9qJJY3jBTjgtdK14vgio3+eD+hVAr8jNUXXZTI/1DNo9aP9vZ2vI5\nD7UeHk3Vtbou6hLt9VmSKK96CQWGN6Cy5E9o0ts5DPJRO+2OD7qBadPID75KNnaOlhbfwZwLHadp\nuA9mzR24PLqb4vDIwI6MjGLacKetl/SuhmowC0VG9i2quwPLwmJCnHxfQy0QY7uUljefg4eCoD3Q\nYPUrqWrQn0atXzNavrsKKigOzd6bEhn0mVlmtglqPelz4G+H6Z2NukEubr2IY73Pd2n/zERddsuW\n72fH6g7goJbvDegiRXe/5dMtlAXoUqhV68BsvW0Hh8fv81Grx+6oVr4RqlxMjPNucVTzb/ROlz62\nawoq1F4Gbon3NkdN7WVLy430MdgWtZhdEJ/LW63uJXs+FdWzxBqvYdKz0rMcajH7RFwvy8f7q6BW\nyI4G4KMC98soQJ+LWii+jVq13gweO90mej6EdX00tmiLOOanoe6YGWg83FVk8901sW9i+8sxR8uj\ncUCnZ6+fZpAtnagl5HA0q/53yrwAjclZikE+GJ2sshr77kiUj81Gt9afFGlYCfUK5PPJjcv+fld8\nfjGUv74vOz8PZgATvnbrnEYt0IdRjWvdBbXoHJmlubWLtPzuGihYXoRqEswd43/bxHIGPK6t3Trj\n783QcJByTq99UJC6J2opO77d97q6P4f7gA5gh66HCuhy9tCu3IrcQDqnxsW+HgqQpqLCYBoKYMpM\nconYjq63nKHg7CiiST3S9RE0GLZsgXpfy3fWQIMJy7FLo1CXy7jIvB5EYzR2RQXmLoNMY14gbInG\n6FzKghOzPQi82oV9tCTqDniZBZ+8/QgDbOlovaDjor8s/i4HBU+NY7RwP8taLTKSfeL1+ahl4feR\nkTyNMvpvo9rzKkNwbuUZ3Z1xjR5BVGxQ69gvIsO7p5fv5c3z5QD4R+g5R9BDqIt5NF3KJKkKiq3Q\ndBjlA0ZXQTXtSxngHZ0osCnH8Y2Oa/9/UVBWp+t3BupOnBTX4ktxzi4Z/18MFebPom7nSQNJbx/r\n3xm1PL+ExqGORy1O3yNuzWcAA4VbzofRcR58Lc7lZeL9HeP8Xrzd92qsazKqnB2BKhh/iwKfJ1Ah\nfSy6eec7cd72egclChxeRDcr7IpaFK9ClaS/Z4Q8gJ7qbvQj0firC1B5tBMqn8phBO16H+bFd+9E\nLcH7Z//bBQXPg8r/+0n7XFQx+yQqh3ZH406H5HmcC6RnuA/mAHfiBnFCdjQB3TClcdk4mY6Nk3Il\nNOfFJ7PPbIa6HvssLAeZjn1QK9Niken8Gg2wLrvwJkcabydrnaAqQE4E/pq9/yQ9n5M1AzXR/4aW\nLrYBpHUcGqy+DipsD0AB1KdoH0CtP5j1xTK2zf4+hap15E7UUngUKigeAG4e4DryrrRyRvbylu68\n2f8estnJe8nA1kAF76FkUySgMVnPogJ/BpoTa3kaGhzcyfZRdR+tjmrul6PxW2WhN4VsPB89C8r8\nGXG3o67l9VF38pn0vKV/0Me9g20qC5lTqW7sGI8K88+gwG4ifRTa2TbNju0qW03WRAHgNFS5upX6\nz3VcLc6FJVCBvSQKMPI5haahuwAbGWOTbc84FOytEem/GQUbE+Kc+zEDuDU9W/5uqEJ3Ewr8jkJd\nX3uhFpMf0MBs6CjvOwDlLU/Qc1LSM6nG7S1N1Q3XWgnKH3j8aeCC+HtrFKB8loa7sAaxvTNQ1/Ny\nqGXsuyjPuQwFUDvTS0thXLsvxL4Zh7rOvxvLWRgFl413oaMg9ILs9RzUA3AiLUNcGIKuuh7rG+4D\nOoid2rWAo8E0XgD8hWrwXTk24GrUrP4qDT2Oopf1X4MK1NvRHWIT0VixR1GAUhZ6C5Pdfh/vLZv9\n/1I0L8lTxMDQls9OIgKv1sylZnonxwX8TOynspthE/RogB4B1GDWFd/fCI3zyp9Dljf/HhHrvQY4\nL3t/oIPDH0AF75Wo2fsANI7iujgmd/SzrEnxubaPf0FB8V0o8Ozaw6Nb1pk/SPrzKMi4EtVey5ri\nx2i5e5SqoMxbm+aiFrTts/dWRIXShTQ4x1k/27QcGt+2NBr78gPUtX4mKjhWosPWPFQZ+If4/h/K\nbUCtEvegcZzlmJ2OuuqoKj6T0LCFc+PamYHylHxOoTGdbneH27M7yteepura3gwFOmegAGrAt6bH\n/noetfJ9g+qhrkehStS1VN1Eg8lryvN2BTQFxovAtdn/90ItXr222KFxoJfGOT4aVVwupKWyNBTn\nbIfbPBl1L26AxgvOQi1Of092Q0ab760S5/z99Jxp/Qiqbtpy6MZg8+TWsWcbozzy1Oy9C1G5tlv2\n3tA/C3C4D+ggdvKIm8OpNW1oGoDT0Gy45QNKl0WtBMfR8jiThtNwCfBU9vqm8gKJTOiLqGY/pjXt\nqPvo9+jW2rL/+hw0f0fez78ZKgAaG3MWafoD6q7J76LaCNWiP80gH/GSLXMRVDjOR09JP4z2k5O2\n7VqqsZ5RsS9PQM+x+lsUBC0VGdMOZN2lva0jMr9HqMaZtHsMTvmw0iGrXMQ58xiqDe6IAoJyksK5\nqGVlbpvvLRn7f0zso8+g8RgT4v3HYt+sGudzxzN2N3BerIaC0O+hguMg1BV5YafXK2qBu4bqzqBj\n49xeBQWXm1NjjjA0hm1b1LqzGyrgN0Et26fE+bEUmnrk/C7sl3VRN90RqNv8mfI8Qy3Ed1DzTj4U\n8G2dvf4EPefyKs/1MfnvhrZnpchPJqDW+ZuAj8T/1kDB0/Ts8wvHMVgOteSPQfnSbXEMzkbPITw2\n+86wl1MoAJmd7cttgIvi7x1R69iGLd/J7w7/BioT7gEezD5zKGp1HEUzE63m43Lnxbk+AbXUPktU\nCmJfHzPs+3W4E/BO+EFNm/9GNa6o23PsTEUtKmdT3S2yGz0nEzsHNYUv3+b7i6Ga5X+h7qsygLoU\njVtJkfG8RgN93NmFWt51tgx61MKNVGOtlohtuJRB3opOz5rhQqigvgrdjPArFKTdhO5CWbM1nR2u\n42+IgcRx8R+MaqfPoJaY81AA1fqwzHaDwxelmvbhC0TXF1WBMgoVmnvH664HGfQM2KYCZ8bfT1E9\n/LScsbjtpKUoMFo5zreJaHzIP6Lg7wzUZfZy/K+RZ2LV3MZ5wKfj751QC2G/E7DG8RiLukNeKo99\n/O8j6IaA2o+OivPnKNR1/vdUk1FugO52OhkV8DNoaILQbN1roiDhrPLcQy2Mj1M9LaHWncKxPXuj\nQLU8v4+P4/8kVWH/XtS6NpZBFNKt1xYKNr+HgoByPsGvoZaOZ+j5/L1VUWvk1Shw/AKqBJd54zbx\n/ddiP3Xtruma27wFmmKjzM/moAr8L1Bl5Xf0Mh4rtukGqqliRsd2fzPOwx/SprI5yPSOQkHzHXFs\nLkCtkSuh7uBngLt7O6ZDum+H++C+nX/iRCgDg73RHD4L1MAbXme5vhVQE/cpcRF8kwVnBN+rj+Vs\nhGr730CFfFlQXwz8Gc0X09Ht2R2me4/IMG+mapE7BgVQ56EWl3Hx+pTBHJPs2DyBCvBF0diwcnLM\ndT/61c4AACAASURBVOL9/Qa4jvmRwS4f6xmNxsscRjVIfP84Jif1s6zJqHb8kcjgz0EDdcfH/8vj\nslWkv/ajKAawfXkNcQVUYL+KunYPzj53Ldks1u0yutj3V9HzeXwrxt8z4vyb0fQ2dLida6Nu90tQ\nN3KfBUV27ZWtMZNQof9ZYkbyeP+j/S2rj/N2JurWuh+1bJXnwfrobsTT8+PT4L5YCXX/30eM00MB\nVHmn8GgGENjENb1YXNc7oUrST6mCtK1QXrPjINOf34gyFVgs/t4GBREpjtd+kZay4pNQV9cr9JzX\nbTyqZJ1I1rWH8o5nGMCz+xo8VvkDtA+magU+AE1XsS7Km/amZ6vfTNTiV45RfC/qNTm1ZfkfRS31\njYzlin1WPoD+/cQUF6hb8Wjiwb6oErVy63YO234ezpW/U36yk/m9dHFmWTRo8xKixhQn33xUG/p0\n9rnWOTESqv1dHpnkonGinhEX1PUo0CgL6rNp8NmBqPXhSdQNcTDqdijHNeyFanxlq8o9DHIS0dje\nh4FLs/cmxvG5Gji65fN1xjidBzzQy/8OQAHOJFTQ9TmRHNHaglrhrqR6BuLVaBzOTFTYbBTHeCjm\nPHpzrBKqhR4Wr/dBAdS7UcHyeeC2vpaRvd4HBVBHUhVqH0Bz+vQa4De5PW3eL4OVddE4wY5actCd\nVt+K82AnFBx8Jn4272TdfezzrVDQvWRcm5+jqmiMQnPgNDIlRbbOzVCAsSHVtCqnkbV+MvjpCGag\nysH1qBtyeXQzxR0oeB7sjSiLoVaSsrv0HtSKOBcFRk9QjdlchAWnavkEVXBfdi+XA8NPp6UCFMs+\nsZvnbYfn4f2RL+wX701GU6B8jZZKPMr/X0fDOX5GNR3HPmi87KCC1z7SeQvZJMto/sGvUeV9s1C+\nsknL94a/O3S4E/BW/2l3EFlw0FuPx2G0vm4oHdfHxXJUZAblbezLoRaAT9DL7dWRAV6BWsbuQF11\ny6ABsjfEZ+6MkzqfOHLQ24C6Ax4nahfx3l5oXFA5TiyvNQ762WzE5JLZ67L2vmhkLgOaHTeO68VU\njw7ZDs0kfT9V7faBeH1fP+fQoqjF6QPxeh80fuaDkYFfhFoEn0Tdr+/pbVldOu/vJLtFGHV/7o0G\nfN8JXNfP9m2PapVbxuu5VAHUIrGseb19v4H09zqHVvaZUa2foY9AGrUE3Y7GRx2Dum/2pLo77TIG\n+BQE1DL7GtXg8vEogLoKFeA/pYHJaVv2TTmh5yGoBWJrVNG5DgWHjU2vgrqdP4xanjdBedIEqkHp\nA31KwWrojsCLUZf8kqjCsTfqUj0c3YF8B72ME4xjulP8/QhqTXwZjR1dmOq5lKPj3L2Lhh5OPsBt\n3hiNF9wG5f1vVjhRAHUw2R2YqML8KnBgvL4aBbPlpJl7oPy40WkI4pq4N3s9BQXSl0UayylcHqSh\nxy01mv7hTsBb+SfLZObFBXUxLd0p2WcbH/CYLXsfej5cdD/UX1xG7ytGhncZ0STbZhkroQDsHNR6\ndRsa4Pxbqm6U+2ngtmdaCiNUADwZ6SwLrPdFprYkPbuJBjSbd8vrmagbZm723jg0pmpUu3TWWNdJ\nsc8+g5r6b0CFwvdi3yZ6Pky1t8Hhi6CbCq6gCoTLAOrgeD0JFTptb6Nu+Bzr0RUU50J5XuQ3EYyn\nZ4C9wP5EhdlPUPfITahruZxr5iaylr8ub9McNJh9P3p5fEi23WN6O1bx/2WB/wFOi9dLoZbMW+P3\nOAbYYko1a3UZSGxAFVS/H+U7uw9k2W3OuXIMz1RUAC+PJgJ9gao7ZwUU5NQaW0c/E4CigvMIsrGO\ng9yeJeJaLKcgOAbd+ViOpVoFBRq3oIrI9F6WcxnKh6aR3RGLZrl+HOVf+ZQFXZnouMNtno4CvC+0\npP8uqtnR82tyLMpjHqQqo16LZXyDaFVDrea/pKGbg1DF8DfZ65NR0DY2zukrUAvuA2QVzZH0M+wJ\neKv/oMDpJXQ3w/NorpVy4HN5MpYZcPnQ0rYX6SDSMBVYL/4eG6+foOdtpSuw4JinRVter4UKr8NQ\nbWNTlPl3NINyh2nNn8a9P9GNGRnQ9ZHO8jNLNrC+PPDahWoA/fsi49sN1RjvpYF5nOL1UbEtO1AF\nNpuRDdjP90XrMaGaWHIyGsx6DfDeeG8f1F1zVOvx6+I5no8TuxF1E16LWgjzwOmjZHOv9bJ970Zd\nehvH6+0i0/wk1Vwz/Q7KbmCb1o9r8Zw4VufTMhVCy3V7Gb0UHKhLaxJqifgXqq7HxVH3490DPZdR\nd/IYFMjcEvvqNtQSUI6fW6CFbADrWRQF/ktm198lcZ59gwguUbfKMtRsAUZ50p1UrbK9BVBLo8rG\n1QxyclcUPF2GxiU9Hu+dgwZJr9Ty2duAc3tZzo2o5X5UnDNlN9jO8b2Dh+Kc7WM788roGBR8/JDo\nUo/3ryUeL9Tm+2vFNXAaCtIvjvd3RcFL2drZtuI9wDTvhboId0Gt6d+g53imaZFXzGu3nSPhZ9gT\n8Fb7QYV7+RDKCWhM0VqodvYUqpH/mCqAKmtyU1BAs02DaVmPXua+ibRMir/3oucdZgndivt3rRkG\nGrx3EyrMBh289JK2XdG8Kh9EzcWHRsY0H9WQGntwaaxvFLrV9ToU3B4d279X7IN7GeBzuMgKLjTY\ncrFePnc/LcFTm88sExn7d1B3T1lj/jAq3MtM+/0ooBrSSWJRa1o5e/QRKCg4CI0L+iLZw2D7WMbO\nqHu4fHL7eBRI30ybOcS6tB1roDFj5Z2cG6PBv+dRPSy7DJwWRQXKNr0sazQqkI+M11eiGno50/d0\nBji1BupyOhe1kKyICvFt43/roTGKg27JphrgvhhqlT0oXn8ujlX5yI6NUQvqejWXPyp+PhnXwZuD\nsXv5/FQUYA96QDIKCv41P7fQcIRfkwVQcY1dlx33PL+cGef+Kuiuu8dQa9+rqDzYH42LG/q5hnqO\nhzsCdX+Xs5zfDxySn/fZ38ujwL48FqvHNfgkWZCEKrXlxMFNTEdwUuQBK0Zav4GmxVmk5XPb0jMo\nHFGBU1E4eBrIwZ8dmUg5KdhU1OX1QpbJvBGZzKjsM1+lwduHUaDxEJrlNR9fshBqsXg8Lux7gCta\nvrtopOcyFNC1BlBlTeRsqgfhNnLyRvruRrXCfdHA0DfHNaBurkEPeqVnUDOfqrD+Obrj5ASqO2wm\ntX5vAOv4SpwDj6NumrL1YVM0OP2GDpf5MCqwjo9lXYMy7rPjmJTjgBqZ66qftOQFyGGohfWj2Xt/\ng6aOuI+egz7bzRy+CFWFYjc0DUZZCVkItUANSe0dtQi9QM8Hcm+I5nD6DNmt9yhw6nO8BRoT9KXs\n9UWoBWpQlQ80HvB6VNjPyt6fh7pWmuiqWziOYTlR7CGotbkcq3Unqnicibqha60zrvPrUfA0Na67\nh+glgKJqrb8r3+ZBbN8RKMg8j7jpJN4/H825NQHllx8jpg2hCqBGUT2L9P1xTW4b5+vWKJjaCuX1\nQzIHWS/buC3K186J8+LjcT7vhirRh7Z8fjU0lu06NEh813i/DKBORkH/hvH/RsYbobz9C2j6hIVQ\nK9kuqIvw/dnnvkw2U/5I/Rn2BLwVf1CB+/0sw5kRGc4sNPHd+VQDYUfFxTuoB1i2rP9K4Mvx93Ko\nibv1OWwPoEGNV2Tv5YXarpFxroai/3YB1K1kjwBpIN3l7ai3ovk7ni0zHZRZN/IoEaqgZjQKILdC\nAeOTqIa5LWrhOYusgGMAASIKnC4l7q5BA1CvRrW6aWhczUWtaWuznHxQ/ANU3Qw7oIz/ZRRw/IAh\nfBB2bN+6VN1SN5BVAmIfT2y3fVSB0x4owLqfqjDaBfhnqu7Irj2nLkvHksDS8Xf5kN784cyzs/Nx\nHBoonG/rYlTjZdYkBtjG68/Tc3b6SxjgNY+mSJiWrefyOFfXjnPqS1StZoOd0XlhFDBdRRRgaHxL\n3lW8P2ql3XIg60QtfStQPdC5bQBFdd0uHP+fOpDjHb/Lh/tunq3zUrKxVPRseXozzyhfo/yiHPMz\nBt3kcHcc27tQN9dTDMEjkPrY3lVRBXmv7Ny5hphEEg2Mn519fgbqGSkfPXM8GnNYHpsVULB7AxqX\nOKg7HbP1fgp4uM37E1BL1BdQ690NZDO9j+SfYU/AW+WnNcNAheJ3qW7Vvh51d/2ebA4XonWjwXRM\npefzz85FTdB3A1dm7z9Jzzu6RuW/822iCqDOi9fLou6UCU3tN1SrKTPnw1BTetmXvmVc0I0NRo/9\nfjcxRwm6S+gr2eeeoWUM2ADXtzG60+nq7L0Polrd4fQyeDp7r5y1eHmyh1xHpvxY9vpdsZ8aC8I7\n3L4dUO1zG1RDPxe10CxQG229RuK9neI6mYG69n5EVSjvAfw3Cmq6+hgL1KX6AmohvCTeWwwFPff3\n8p2lsr/HoUGs56KAZg/UbXMJ6rrclyx46muftPnMckTrcbaeL1AFUGvF9Xld/D2p02X3s94yT1gC\nBUg3ZtfoQajb7gBapjbpcNkzyabsQK3cL2fbdAIKpjeJ12WL01Q0Qe/WA9mmWMauqDXo2Dh3d4zl\nHo8Ci3Lakzfn4YvXeeD0CGrBmRTpmVWmk2oG9OXo0tCGGtt6CBprez3VRKOboFbCthN1Uk2umlCA\n9DDqijwVVTLXQBXcxibARAPEy2O9UMv/pqDy9O/pOQHmiHm0TdttGu4EvBV+qArkLVDBuFVcYFtH\nBvqeeL0KDQ6u7iM95TiqVdEg0rVQwfY8Me6E7LblSFteSC/Q5RMXzGMo036NBm61zfbb9igY+AfU\n/L0suhX2p6gF6vs08KDPlnXf1nIhjkNdlHeilp0rW9PZ4XIX6PdHzePPAB/O3juG/udxap21+PPE\nE8Pj/48DT/e2X7t0brXemTgRFazPxvm+KNXz/nrtqsi24XjUdVk+buMTcdzLcTWN3jyRrX8tqkBg\n2zinZ8Rx+Ut5/FFLzgNk43h6279okPn1aADyjLie9kHjX34J/IkBFjhoDFP54ODpaGLNW6luF/8E\nailrejzg+qi7ZxS6ieIGelZyrmGAA4VRfvRk9vpiNOC6bOX4JApmp8TrKXGeDbibiOpmmSVQAPgc\nMW8TygN7TNlCdjdled7GtXhinPtPUHX5D3thTpWnLksVRO+KBoQfHulfNa61aS3f7VFxjuvy8ni9\nKcr7y6dgTMzX10C6vwyc05oeFIxuiCoyfU6oO9J+hj0Bb5UfFBn/BDUt/gtVs+f2KIDKZ5/txrw0\nCwwMjYs77zKZTTb3TpkW2hfSH2fBp1Ifgh4d0djDilE35uvoDq2zUWvQ/lTB51b0M4C0w/W0Fvpn\noFvH8/lMVkBN7fng0dqBU6T9CjS/TjkX1bzIII7scFm9zVp8JhpUWQYfz5GNzRmicz2hZvYySJ9I\nNXndRpHR7d3me20DdFQoPkR1p+OjKFBfYiDHoYP0T0MB2ntjW1aOa2MXNKv72vH/21C3Y5+DrunZ\nOrEiGhdyJj3vLNwpzu+LUKDeUUFLz3FlzwLPx9+LowDqGRTUPA9s0YVjPR1VXnaKY7c/KogPjv/X\nDpzo2QX9FPBM9vqSeK8MCsuAcQxqndhmkNszBbXQHYFa68oxm7uhGzLyAGJxNB5ymSwND6IWmIlk\njxpq+hwd5DbuiuasupPqjst9UGXr2Thnds0+n1+XvbaUxbl7VPzdaKCIys9raHmWI+q2u5VsGo+m\n19214zDcCXgr/KDC4lbUvbUxCpbywuHdqGui0edJtUnHaHQ7+G60udMKjSu5sOW9vgrpE6gGxy4f\nGfSe8XqwXQJlDelI4Prs/UPRbbQfIAv8Brtfsr/f7AJCAx9foZe7gwZykVI16Z+GWiH/TNUNNQ/V\n+PrtWqP/WYvzp4hvXDedA9multffQAF3Pu/PF9At8vlYkfI4twvQPxEZd0Ljh05BXR4PEresd2lb\nlkKtHEeglo1FUeFxE9Udi6egboI+x/TRs2v77MgLVoxlnUrPTH/N2O5+b+On90DzK/R8oPfZaALH\nRsaeZMtdKDu2BxJdjrF9B8X21XosDr13QT8OPJu9vjquk7Et3x/MGKfFs/euQOMDy/FpW0Q+sFbL\nd5dDAertxBQNqDI3MdL3kdb1DNdPtp0TUeVqa9T6+SVgfvxvXhy3PODrreI8tWX5m6JKbiOPlUF3\nNK9IdZPILDTm7EJ6zq93H9kNT2+ln2FPwEj8iUxgaksGcAIaE/ACVf/3fsQDPtEg7g90IS3nEre4\nozEjT6DxAw9lmcOsuChuavP9/grp/9/emYfbUZXp/vedBEwgCZgwCQJqoxC6m1EhCN6ggGJkMAIK\niqK2CF6vQaYIDYJcJhkExMukNhAGNRAUiRANCDLZQG6DNHpV9Dbd2kprN3K19RoH+PqPd1X2OsU+\n++yhalWdfep9nnrO2XtX1Vqraq31fesb3rU4OjezDPRLRrkpWknEysxuaPKPTbK3IAtUkVl1Iyg2\n4bPhHWUZfMegWItC3KloZf5xFAuxEgXzrqalQG3f5X3GZS2mBELVMeoSc2H9VfT5BhSXkgX6f4Yo\nSDq6vpOCfiISyPuG+32fNNvIXAP8jtGC5CQUGH0EivPoanPeUPfb0Wr/bOQSygJrz6DF3rwXgdh1\nnPuN565dSeSupRXLUtSCZlvkqjwWue22RQvCbCPn9eldcerGBR278AZiJ0dWpAXh/wXIQnstsnS/\nAs2RD4Q2jpmZiLI8lyKlI3MnvpcaKU5RPfZB89tNtPYXnI3m0xvC58ORnHoXygLvuHBGvIB7IsqW\nooLDt0cZw7cgC/Pc8P1L0Ny5As3VtwHX1u05d93OqitQtyNMJHeEl34v0oynIk36HwnWpTDp/AAF\n0mbbcgy059oY9Xl5KPcLwMnhuzlhYrodKU6bELmLGG2a7kZID2wmRSuip5DF4lO0yCFnI6F7Cgqy\n3RmZw6+hT1LKNmVPoUU/MA0JzYejQXsqYxDgdXHvfVEm4Dui76YjE3QWYH95mCx2js7pOBHQI2tx\ngn4/ghSlO1H2zknh+yXh+4eBJe3aR2cF/TRGKzBb5K8vqT0fRBaIi1DQ+wgKpD0VWRW6SrlHvGc/\nRErlO2hlvr0YuQPXZKSG77Ya537dumvvYrS1pqjYkz2RJeZtKAThh6F/P4GscT0r7D206e8pwAUd\n3uVB4T0uQkL49SjL+TOhPdORlfsdtDI880k/ByCPwcVhzH2ByJWclVVmP+2hzTsjZfCU0O6P0Fog\nzkGLgVehufBdaG4Zb+F8Yvh/K1r8ZkVsuTUbzRnvR9nNK8K7yWTn9DAW4yzAWjznntpZdQXqdKDV\n06rw0l+KVjffRu6K9cPEuQRp1I8RTcD9TDrj1CW23myChFeWvm5hwJwavl8/OjfvfkkipFGcx0Vo\n1XNcmIw/gITO+mEgfxERQO6AYiwuo0/iNaQ0bhn+n4asCRmh4eG0gnj73rQUKXjLw/u/BwmITCk8\ng1aA5jlEweJd3rtWrMXheX069K09kEKYkePtQQgkzfpf7tpOCvqs0C/KVpYyq8o85NLYLXw+CcXA\nxJQDM9q1I3e/TODvByyPvp8fxty5ob+13Y6pw317cdd+FThnwOeyKXBZ9PkCRrM274SE7T3AXX2W\nkdwFjRZrbyPiUAt99ziUJbiQDu5TtCC+nhbp6NbIpXQ1LYLTWlhCkFL0JK1A/r2RHPoftObAdjGx\n3e7JV7jiguINvxPeyWZIrj6D4tremzu3Fs+512OEBgCY2eaI9fpMd78a+Dd3/5m7vxZNQBe7+zFI\nyPwvlC10m5mNALj7nwusyxR3f87MRszsIyitfyHwEjP7hAvPoMDOc939/2XXuvvzudvNRDEMzwKb\nm9mh4ftPIa6jR5DvfyC4+x+RorE3WsEdiugJ7kbBjKvQJL03iok4D/m6n+u1LDO7GileN5jZu919\ntbsvQQrZv7v7DQSLGjJ1Z9dZD2Vchiak/d39YHd/PeJY+rqZTUeT2e6oz2zu7leF68YcU2Y2Jfp4\nOnL/bIWUsPea2YVoAj8dBe7P66XOvSBXF5Dy+/eu2ewfkAtmBzNby90fcPcV4bqRcE6MZ4DHzWwO\n8Ky7f9zdd0KKzM0oDmZmGe3I4O5uZgegsbk7cL6ZLURBsD8BjjSzvcLz/F12TXwPM5tpZi8ysy3R\nuwEpSn8ys7eFa+5FK/8tkEK52sysh/G/HXJdgsbLauBNZnYWWujMjt7NcjTmp/b2NEbhD8A2ZnZN\n+DwFWcgI7XnU3W8M/fs5M/twH2V006apobxH+mwHZjbdzN4UPr4cje+bgL3M7KAwL15E2LcSLdra\n3WcXpJBsFOoO8GM0F+4CXGJma7fp58kR5pN/RQkOi8P4uwu5KPcE3mpm08bof53G5TJgmpnNaiMz\nisCtaHH+apS5+Ax6tr9Fc80a1OE594Wqtbe6HGgl8hSjU9izVeVMNLCSsciiieGbSFnL0mg3Q8L6\n7Dbnv4CvJPxf+tYCjHYTnoaUpU1RYPhZaKDfQGsj2fPok1gOKU1LkAB4O1r1ZhuOvhpZBQ9Cq65j\n+izjaOD56HPMQP5l4Prw/yuJ+Gg6PUNqxFqcq8tBSKAeRrAKRn3+LkI21Dj3q8SKhsz/MYP5rcga\n9B7kIspS1C30yzHrwNiBtS9BrqHzkBVlPpoLTgt9rSeLMz1YglEMVd+W0+jec0KbLg/9bd8wPtdD\nC5ks1uck4ON93D+VdXs6WhQ9iBZjW9HaSHY5UZbwWP0WLegeQpl2uyM3dcb7tBtakFZGepmr605o\nrpuOFI4lyNKWyYO96RBjmWpcojizt6LFa8xxeBxatMexbtOj/yekxWlN/auuQB0OWsJkLeRuuj76\nbRpawXyTiDupxLpkwuATtLIoXoaCGF8X6vJz2qSK59pSqpBmjPRXpDitRPFgHw3frUsulqDPZ5NX\nai5F1qB7UAzYOihA/EtE8VT9DNLQhhVxPwh/d0E+/DzR25imb2rIWkxrv79zovodhUzt70eKxBXj\ntSn8n3zvLxRXcSktRWNdJBBOA+6jlfzw5vHGLWPH7JwRnslWSDDcEJ7LzsiV+WV6zBglgUBrNzbD\n81qKYvMeCm3JthT6C1ru1Z77XYo2RWXthawY34i+mxP62N3A2ztcuwOaO7N0/FnIov9PKBD7Z0Tu\n6aoPtKi5G3FvTQ998nNovmurtKcel7QWs58MY+UroS9kVBQ303LpDrw3Xp2OyitQhyPX4dZGAvm6\n6LvdkaApbFfpTnUIn/dFwnQ5Ck5eGv6fzdi7uycR0oydlp6lpX6RKN06d+2gGUMrkYXhqDBR74B4\ne54GPpgvY8CJ4U4ijprw3Tbh+XXFvk7NWItpKeefoRXzsCHiYdkVWVZOZXQWZj7GqRZWtPCstqS1\nbcjHkHDMPs9H1s+OWZ10jtk5ndEB7y9CKeGP0n1mZTKB1mZs3hz63Axa+8zFzN/xJrDjUiykbhOj\nFcHZSIF4bRiDsZXrlSgGqmMGZRh3P8h9l23htGNZ466LdsY7EbyCoGyiuWE5UkLXDu1fQpvs4dTj\nEsX9xX1pfaTwHUMrGP2Q0OfWq+rZlvbOqq5A1Ueuw2W72K+FFKhLkbB8jGhPpBLrYMhysz0y2e6P\nsiqmoZXHmh3JszqP0Y7ShDSdM2sWhwE+F7nqXkY5wYjZ5rkxyeIClDUTT+q9EGDGpuc4oHYFcG/0\n+cvkiEg7vM/asBbny0Hs0VcjIXt1eKa380Li1Px1lVvRojpsjqxD94d+PTf0w0dC2f+HLpjrGT/g\n/VOMVsg/SJeZtSQUaB3G5idQJuoMpIDcQ9jTL7y3XveqS2XdziuCtyCeuxehJJplSKl4dfi7We76\neGuo+YT9P8M7/laKcddlO+cg5XOd8I4+HfpeRkdgSBG9A83p7bZ5Sj4uw/0yy+ahYcxcHT5fjrwl\nW9CHK3giHJVXoNLGv7DDLablnlkLBQU/T4uyvtRtMZC74RJk5TqPFkX+VNrwOFGBkGb8zJoTwuS2\nsp8Js015Yyk1tzPadL+cNrFgXZbRzvR8Eq1tI74RJpvriAjd2vUHashazGjlfE8UxzMPceCcggTR\nBsi6+dIu7lOZFY2WQNwFKU1rIaG9PJQ7I/SXA2ltClsJbQSJBdo4Y/O0qPwNgZ0G7EtlW7fHUwTX\nDe/6JhQH2lZJRm65R5E17AsEShc0xh8qc9z10NaNkLVpizA25yLL3SnZs0PJNj+ijdKeelyieWRL\nlFiyAbJa34/c2Xcgi9S2BPdtfF3Vz7rQ91Z1BSpr+Ngd7pu0NP6p5OjkC65DbDk6npap83u0tkfY\nCGn1o/ZioyIhzfir9IvDbwObaelNqbmszzI6mZ5j19V9RBvIMoYySk1ZixlNIroCBUHH22jcSBve\nLeppRdsLWSRiqpATQ73m93G/wmN2qEDR7GJsdrSY1qlNdKkIhs8ZfUjevTwLLQi2D5/3Qd6EfcLn\newmb1VZxkKPMQFbUFUiB2hopUJeF53wXo70OlY5LpMheGY2TM6LfHg31PZeCdpGo41F5BZI2trsO\nd3x8bnRtoYIuqstaYQJ6C629rLKtEjZEgndGdF3GP1OJkGb8VfrlFKM4FarUdChnPNPzojbXdCyD\nmrAWR5OyhYk5C+D/ZwLpZ3iXHyGyakbX1c6KFsp5M7IIfyz3/d+GMdx2N/ncuYXH7FC9QCvcglZV\nm+jOldopSWMuoq1YRSsGbjqyXF2Uop+O076tkdJ5UWjLFJRNfQpyR24aPh+JFgrxXnW1GJehXoch\nV/By4LDw/VWIa++NlBgnXPVReQWSNbQmHS6+NxJqXwsT0e4ovfr86Lw7yO1Vl7tPciFNuvTXwpWa\n/DugO9PzIe3eXYf71oK1GLlOYwVhEXJvfZ3WAmEjJGQ2aFc3amJFi8bLLFpJCfsjDqP9c+d2Q61Q\neMwONZhfih6bVbaJPhTBuHzE33RuuM+VhGBwtEi9NtS/ElZrFNz+HZQ9/PowvrI9RtdDC51lZPLi\nigAAEv1JREFUtDZNzqhYsnFQ1cI5b9nbEYVqzEVu1m8ieXR2GENfpEMYwEQ/Kq9AsobWRxD8RfT/\n6bT2JJoK/E+UBbUUsQtf2+E+yYQ0abOFSlFqxiirUNMzNWEtRgGntyGCyNeH796KkiDOjM5bYzUY\nq25UbEWjJTAORJljX6Gl8CxAaettaTs69WUKjtmhOoFW2tisqk3hfn0pgihj9Kjw/wUEQmO0xdUZ\nyOpaGR1B6GenEVi2UdzWEyhT9HiUcDMdyYZbgFeNcZ+k45LRiulH0OLixeHZnhrG5wiapzdC1sI9\nq3rOSd5l1RVI2tjqBcF1tIT11si8+WNCKjVy3+2IVsEHR9flM56SCWkSZgtFZSbzp1OQ6RkFMG+L\nFNlF2bND6dPfRUJ57QR9/LLQD/ZDLopzaSmklyNl+xykhPzdOPeqixXtjaEeL0EC5fu0NmI+EPh9\nmLC7dauVErNDeoFW+thM2SYGVARRCMTlwG/CczgAbXh7KFKq3kVIIqjyoBWzORUpideHul4ZxuUU\n5LZbTIiDzV2fdFzmnvGHkeI8k2BVQlQxX0VWyoeQwl1arHBdjsorkKyhFQsCQvBzXBZazZ2DTMvb\njnFdXnFKJqRJnC2UK7sUf3p+oqcA0zM1YS3mhSSiOyBlY074PANZ8o4Hju7U76mBFY1WfN9xKDvw\ngDBxn4gyj94Tft+wiz5caswO6QVa6WMzZZsYUBFEtBUboqy1VSgB4q1hXN4JbFpmX+2ifS9FltJt\nGJ2osUf0/xYoQSZz0x2H5qLYlZ5y4Wy80OJ0D5pHPgZ8Pf8OwzsYqqy6MZ9P1RVI0siKBQEvFGrH\nE2KbEJ/TaSgLpGM8AgmFNBWkv+Y+F+5PbzMRDGx6pmasxUgZWJm9w1D+fYhR+v3Zu4vOb6c4VWZF\nYzQp4ibR9+sjJTpjDl+BrBEx11e+DyWJ2SHx/JJibKZsEwMqgsjNdRZSPPZA8U4XhDqfFMbAVmX0\n1y7bNxdZS69DSt8hY5z3BhSisEl4FnszWkFMuXDOYrCyBcyxyOI/A8mv2wnEnuSSqybLUXkFSm9g\nfdwpK5FJ9mjE47RV9Nv2yCT97g7Xly6kqS6zpnR/OiWanqkZazESLquQ0nR0mITfgfhwjhzn2sqs\naIzNXD8TrYKvRxl1u4Xf541zv9JjdlLNLynHZqo25do1kCKIAq33Rtx854VxPDfrB2X01y7btzGK\n3zwifD4MUdHMjs4ZQRbhx4EFY9wn5cL5ILQwyfaGnIcUp5lIiVoR9cNJqTi5D7nylLLDdVmfUczY\njN4kcdwBTolCmup4o0r1p1Ow6Tn7npqwFjM2iehS4Oe5c2eMc6/KrGh0Zq4/Eblv90UWtO+Ty7Lr\ncN/SYnZSzS8px2aKNlGiIojcehcDvwAeicur4gj99n2MdtUtA7bOvd9l2fjNv7+U45JWvNXZaNGx\nAZpDpyPF6R4axUnPquoKlNawit0pjC3UvkYksMmt4GgJ56RCmsSZNSTwp1OS6ZmasBYzNonoi8Pv\nK4G72z37DvesxIrG+KSIsXKwRTf9kBJjdlLOL6nGZoo2kUARDPd5DSF2qsoj9OXMcprNQ3fS2rVi\n0/B33U7tTTEuGW0F/FvkQozdtO+mUZxaz6vqCpTauOoEwXjM2F+njVBrc5+kQppEmTUk8KdTkumZ\nmrAW0z2J6KPAyWPcozZWNMYnRbyol75HgpidlPNLwrFZapuoYJFWVp/tUGYct7dh9H023yxDIQjb\noRCOl7WrcxXjEilOD6IwkkXI0vjFXDsmveLkPkTKU10EQRFCLfyeVEiTKLOGBP50SjI9UyPWYvog\nER3jPnWxohXGjk0JMTtVzi9ljc2q2kRNGPhLalu7uL0TiHZdQIkwn0Pu0YVj3CfZuGS0xWkzYFn0\n28uAa8JRacZi3Y7KK1BwJ6hcEBQh1EgspEmUWUMCfzoFm54Z7VqsnLWYAklEqYkVLZRVCDs2Jcbs\nVDG/lD02U7eJmnCHlfSuOsXtHUfLnX4Fin3NCGwr25OP1nxpyNr718C/AIdH55wQxtTRRZQ5LMcI\nQwIzm4WUlfe5+75IU55rZvu4+0LgD2a2a4nlm5ltiVZwz5nZm1AA9JHAJmZ2LlKIns5fF/8NWAv4\nT2Q+PdzMdnT336NJZ7aZrWNmhbw7M9sFBVluhJQDEHHnI2gFf4mZre1hFA1QzhR3fw74N+B3aEVz\nPpo4fw/8B5oc/hyd2zPc/bnwbO5HwZo3A38M7djQ3a/vpQx3dzPb1cyOcvd/RMJsKvBtYImZnYGs\nIkvd/f+7+/P91LtbhPfwC2CVu/8HYcNfd/8Hd1+AlKlLgZ3MbJ3cdWtgZnMRx9grkNUP4AHgV0hh\nwd3nu/vDZbXFzKZEH09HfXAr4IPAe83sQqQonI7e4bzcOMnfbwfCRtHh2TyBFiwXmNlnkbvkq+7+\nRB91TT6/lD02U7fJzKYChyDix2ORW+gnwCfNbONwjpU9hkrE29Dm2teY2YiZXQCc7u5noAzCo8N5\nVyJl+B4YPTZTj8swXxqKOZ0RxsaHgMPM7Ihw2o6ItPjKIsocFgyF8lQHQTCoUKtCSIfndinwS7Q1\nzP5mdlBQKp5Gz/Acd//joGUVrdS0acuU8HcEMVE/7e4nuPulSACvBs43s02z+nR537WQ1eMCM3tn\nqP+bQ92PRKnR73T3Fb3WuV+4+2pgIzM7DHgY9anDws+rkBXsHkT3sAZVKejtkL3nIGT+m7v/K+LA\n2Q+9vwOR2+MQRDC4GLi3k6Lg7t8BfoCEMu7+G3f/CrAXst7s1897qmJ+KXtspm5TqkVaxdgOZYKC\nOMlWA28ys7OQtXqD8Nt33f0+WLPormrhnJWb1W9Z+PwQWuifama3Ihf3tblrJj0mrPJUJ0GQoV+h\nBumFdJmr9Fw5pSg1+TKiFdQZSDl7jZkdHu75z4hbZS5yG3R7381RzNqFwA+Re24E+Hfgb4CfuvuN\n7v7tXuvcK9pMWmciK83GwEeBD5jZUmTBuxmlR4+6pi5WtFhxQq6b3cJPNyG3x1HI/Xg08Hbkrn6P\nu/8od5/MajvXzOab2Ux3fwPwf83sW1G7n3L3+939sR7qWNn8UtbYrFBIJ1ukVYxngMfNbA7wrLt/\n3N13QnGdy4BpZjYrnuM8IOW4zObkaOF+CopzuyR8/lWQMzsD73f3Q8J1IxNcuS0WXgPfYb8HNdgE\nkgKYsaloawHKz6xJ5k8PZSwCLg6fF6AJ4Yjw+UbCZpxd3q82rMVxH2MAElFqsPcXBZEiRvcrLWan\nyvmlrLGZuk3UjIG/zIP+NzNONi5pZTiPhLF3DK3YqtuIssBz886EDOAv9X1XXYEBOkEdBMHAQo1E\nQjqrK+kzawpVajq062wUhLl7+Dw7TFo/Am4Fbm733sa5d+Wsxbk+1jeJKBXu/UVJpIiUGFibcn5J\nNTZTtilXbq0Y+Atu26CbGScbl7QUJ0NWrStQnNtnaXGs3QE8XvVznQhH5RXosxNUvgkkBQm1cG4S\nIU3a9NfSlJpw7gsy5VCcwarcd7PIbYXQR1uSsxZTIIkoFVrRKIkUkRIzUquYX8oem6naREWLtCoO\nRltR+9nMuJJxiRbzn4vqvgfw+axudKDQaY7oOVZdgT5efKXuFEpkxqZEIU2i9Nd29aVgpYYKTM8k\nZC2mBBJRKrKiUSApYu5dlkIbUcX8UvbYTN0makAZU/bBaMWp582Mo/uUPi5z42YRii38KbBz9P2N\nKL6p7XXN0ea5Vl2BPjtDVYIgBTN24UKaRLxRJFBqqIHpucxJhZJJRKnGilYYKSIJYnZSzi8Jx2Yq\n63ZtuMPKOig4bi+cX8q4JLeAQPtDnhr618WEzbWBWxggfGIyHpmJdULCzF6FOCneCfyLu+/Sb6p7\nF2UdhLLe3ufuT5vZPLSaW4j82G9Em5UOxFOUK9O8zxcUX2tm2yF/+3RE2naVuz9mZm9BqeD/HVjt\nA2RxhEyM50NGz4PIzz8N+BPwPXf/tJndgdw32/dbTlTezijI/MiQFfRaJJDPc/cfmdnJ7n7uoOWk\nRMiC2Ri5gbdAytIz6DkejSbigbmwAlXGX6KNqe8rpPJjl3UASqB4AJF3PoPcdL+Mzhnppu+FjNRP\nA4ej5/FbJABOAZ5C2YdPeUHZj2XNL6nHZq7s0ubMkFX3YaTgHuvuD5jZdGSRn+Xuxw1aRlWIMkSn\nhvE3gsIOHkDK763ASne/sNv+3KaM0salmV2FaBQ+j9jNf4Uyuj+JFh6r3f3tRZY59Khaexv0IIE7\nhQm60zQVZAtRkj+dITc9UzAzej/PtYQ2FcaOTXUZqaXML1WMzTLalBuXlTPwl/S8St/MuNNzLeC9\nzEaUID8FDkbK81OIX2sT4FXRuRPu/VR1TFiepwwu3otV7n5fGQRelogZu2ik4o2Kn7mZLUKrzH3N\nbGd3f97dH0BK5usAPFiDenlXbfhFnkRC5u8QR8288P00FJy+BrnragsvmUS0Q7mlPB8rkBQxWC+O\nQplLmyKOrZ+jlfStyKK1zpg3GABlzC+pOd3yKLJN7vXgDisZ6yBOrIwJ/Tmk8F6MFPevuvslMJi3\nIMag98iuN7ODkSv1Q4gmZgfgWeTGPRb4vbs/Gc6dyOzuyTHhlacYZQiCqoTaIEhF7phKqckGtJld\nZWYfRabyv0SWiIeBL5nZbcBzHphwJwosAYloahRNihgWKRcgK9bVKKV+PbRK/iTKYv1x8S15QT0G\nnl/qRLwKg7epakUwBdz9J8DXgLWRZW2Ou9+PCFyXhrFamOJUFIKitx1SXrdFY/GXiLDzQtTXfp2d\nX6e6TwQMlfJUJCaqUEu5Si9bqclZtWYjq9/xKOjxPsQZ9U8oqPoEDz77Itmey4SVxIxeJcpix3b3\nX7v7XWj7lrURh9qS8PPPiqp/majSglYG6qYIloUQt3c+elebAZea2Ubufm2kONWOfdvdf4ECw89E\n88v70Dyyrruf4+6LodlypV9MCCGTGhNZqKVYpadSaobd9OxDuCmnl7S/XHT/J1GsyX4oy6k2C5fx\nUBcLWhEYNkVwLNgE38w4hE48hDYt/t8o1m1R9nvdrGUTCRM6265MRELt5e5+rJktQJkkN7n7EjO7\nEbizzm6iMjNrwv0PRgNyBG0/8C3gu8jS8A2UDffrcG5fgzRMUB9GgehXIHfANqGMxcD62QpqIiF7\nHmZ2NnAy8Dp3fzAoo7sit9f3gD95a2+p2k10UTvmovimR939P83sdrTC3TNF+WWWUQbKHpspYGbr\nIWF8OeJwejHaSeD7ZrZFcHdNWIS4vSyj83Z3vzR4JA5ELrAn0E4JE2JPvqAIbuzuE8JSW3c0ylMO\nwyLUMlhJ6a8plZpgsdoF+e6fRG7Bg2J/fZ3fQYx2AtLMlgObuPtrou9moey6X4XPfaU/p4CZLUR0\nBL8kpEC7+xVm9hXEVzWv4w0mKcoam6kxDIpgHmExcA2ycG6N3F9XuvstZrYbivO6rFf3c11Q5/lk\noqBRngKGUajlUbSCkVqpMbMtUartacCF7n5m0WWUCWtxYY2gLRz+hLaseDzEhs1w9zeEc2MuoNq2\nL4yHz6EA8MfNbB9gf2C5u99pZvcCi9394UorWnPU+R13g2FRBGFN3N4dwFnufnno43shpvy7UGzX\nBwZxPzeY+GiUJ4ZTqKVESqVmopqeLTGJaArYEJMiNugfwzAvmtndiDtsm+i7l6PNf3/r7o9VVrkG\ntcCkV56GUahVgSqUmolk9ctgE5wZPbd4SMqO3aBBGag6bq/BxMSkz7aLJvadkLL0IZQ5ch3w12b2\nSndfAHypqjpOBLj7nzPFyRJRBUwEoRyU8uz/UkhEUyIImWEnRWwwiRD69ELEWn8ycJWZfcjd3wI8\na2YPVVvDBnXEpFWehk2o1QmNwBRsCJnRbRKQIjaYXAgxTYeifUv3RYHic81sH3dfCPzBzHattJIN\naodJqTwNo1BrUD/4kDGj2yQhRWwweRBcdeegPRNnhq8fQFmjbwZw9/lNwkODPCal8jRsQq1BvZCz\nag4FM7pNElLEBsOPnPdgLbRv3YNo4byji8x0FTDbzNap87hsUB0mVacYRqHWoH6IAqqHhhndh4gd\nu8HkRhO316AITCqlYBiFWoN6woZwU04fgv3lGjRo4vYaFIFJR1VgQ7rdR4P6ITWJaEoMEylig8mD\nELe3GsU3LUXj8mbgJOS+O8Ldf15dDRtMFEw65QmGW6g1qB9SkohWgWFpR4PhRojbOwXYHLHi/wZ4\nN/B5YCHay3R+435u0A0mpfKUYdiFWoP6YKIyozdoMEywId/MuEE6TGrlCRqh1iA9JiIzeoMGwwQb\nws2MG6TFpFeeYjRCrUGDBg0mB5q4vQaDoFGeGjRo0KDBpEYTqtGgVzTKU4MGDRo0aNCgQQ+YVDxP\nDRo0aNCgQYMGg6JRnho0aNCgQYMGDXpAozw1aNCgQYMGDRr0gEZ5atCgQYMGDRo06AGN8tSgQYMG\nDRo0aNADGuWpQYMGDRo0aNCgB/wXRv44epBsjP4AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "local_2.plot_lr_weights(['@'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are other rows in the confusion matrix with empty diagonal entries, suggesting problems. But another approach to improving the system is to look for errors with high frequency. On such error corresponds to the problem of distinguishing proper nouns (label '^') such as \"McDonals\" or \"smash burger\" from common nouns (label 'N') such as \"wife\" or \"rain\". \n", "\n", "To better understand this problem it is useful to look at some actual predictions of the model where this error occurs. In particular, if we show for each error both the context of this error (say, the surrounding words) as well as the used feature representation, we can both spot potential bugs and get inspiration for additional features. In fact, **this type of debugging and sanity checking should be performed for any model you ever train**. Even if (or maybe: in particular when) your model is doing well it is worth to perform this type of micro analysis to spot potential bugs that lead to worse performance, or bugs that effectively mean you are cheating (e.g. because you are using gold data at test time). " ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Previous\n", "  \n", " Next\n", "
\n", " \n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "util.Carousel(local_2.errors(dev[:10], filter_guess=lambda y: y=='N',filter_gold=lambda y: y=='^'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By inspecting proper noun instances in this way we can notice that proper nouns tend to be capitalised. This suggests that the model could benefit from a feature representation that captures whether a word starts with a lower or upper case character. Indeed, adding such feature leads to better performance." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.7325316193240722" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def feat_3(x,i):\n", " return {\n", " **feat_2(x,i),\n", " 'is_lower':x[i].islower()\n", " }\n", "local_3 = seq.LocalSequenceLabeler(feat_3, train, C=10)\n", "seq.accuracy(dev, local_3.predict(dev))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This improvement indeed comes from being able to identify proper nouns when they are capitalised. This can be observed when inspecting the corresponding errors: some of previous errors are now fixed, and the specific error count on the first 10 development instances has reduced from 7 to 3. " ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Previous\n", "  \n", " Next\n", "
\n", "
playersandhiswifeownsmashburger
N&DNV^^
N&DNNNN
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTruesmash
-5.56-4.060.00
-6.67-1.090.00
1 / 3
\n", "
andhiswifeownsmashburger
&DNV^^
&DNNNN
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTrueburger
-5.56-4.060.00
-6.67-1.090.00
2 / 3
\n", "
meforblowingupyouryoutubecommentsection.
OPVTD^NN,
OPNTDNNN,
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTrueyoutube
-5.56-4.060.00
-6.67-1.090.00
3 / 3
\n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "util.Carousel(local_3.errors(dev[:10], filter_guess=lambda y: y=='N',filter_gold=lambda y: y=='^'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us look for further problems we could fix with more features, and consider the confusion matrix of the current model." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAGoCAYAAABWs9xCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8JGV94P/P9wwzw2VUZjjDhMsgBEFFowgn4G1fCqwC\nSgR11UGjuF5mVdysqFnBXMQokV80JqtR3NkYJSaKeGc3iUpQdr2tOBAigiIoIhAEBjSKgYGZ+f7+\nqDranp05p6urak531ec9r3pNd3XVU9/urtPfep566qnITCRJ0uSZWuwAJEnSaEzikiRNKJO4JEkT\nyiQuSdKEMolLkjShTOKSJE0ok7gkSRPKJC5J0oQyiUuSNKF2aXsDq/aazv3WHlCrjGVLPNbQ+NpW\nc9DDqWgmDmlcXXHF5Zsyc/VixwGw5IEPztxyTyNl5T13fC4zT2iksBG1nsT3W3sAn/78V+qVsWq3\nhqKRmnfv/Vtrrb/r0iUNRSKNp92Wxo2LHcOs3HIPyx/63EbKuvfK90w3UlANrSdxSZLGR0B0p3XX\nJC5J6o8AojvnsLpzOCJJUs/UqolHxNnA3Zn5jmbCkSSpZTanS5I0oWxOlyRJi62VmnhErAfWA+y7\n/9o2NiFJ0gi61Tu9lXeSmRsycyYzZ1btteiX0UmS9EsRzUxjoFZNPDPPbigOSZJUkR3bJEn9EXSq\nOb3uJWavAP4tM/+6oXgkSWrR+DSFN6Fuc/r7mgpEkiRVY3O6JKlfbE6XJGlCdag5vTuHI5IkjZmI\n+EFEXBURV0bExnLeqoi4OCKuK/9fObD8WRFxfURcGxHHL1R+6zXxJVPByj2W1ipj27asHcfUVHeO\nvCRJo1qUwV6OycxNA8/PBC7JzHMj4szy+Rsi4jBgHfAIYF/gHyPi0MzcuqOCrYlLkvpj9lakizvY\ny8nA+eXj84FTBuZfkJmbM/MG4HrgqPkKMolLktSepKhRX14OSQ6wJjNvLR//CFhTPt4PuGlg3ZvL\neTtkxzZJUr8015w+PXueu7QhMzfMWeaJmXlLROwNXBwR3xl8MTMzIkY+Z2wSlyT1SKPnxDdl5sx8\nC2TmLeX/t0fEpyiax2+LiH0y89aI2Ae4vVz8FmDwrmH7l/N2aKR3EhFvi4hjIuKUiDhrlDIkSeqy\niNgjIh4w+xh4KvAt4CLgtHKx04DPlI8vAtZFxPKIOAg4BLhsvm2MWhM/Gvgj4I+Bj49YhiRJO9/O\nu1ppDfCpKDrB7QJ8ODM/GxHfAC6MiJcCNwLPBcjMqyPiQuAaYAtw+nw902cLHVpEvB04HjgI+Bpw\nMHBcRHw8M/+o0luTJKnDMvP7wKO3M/9O4LgdrHMOcM6w26iUxDPzd8ujhBcBrwUuzcwnzF2u7IG3\nHmD/tQdU2YQkSe3p2F3MRnknRwD/DDwM+Pb2FsjMDZk5k5kze01P14lPkqRmLf514o0ZuiYeEYcD\nH6ToLbcJ2L2YHVcCj8vMe1qJUJIkbdfQNfHMvDIzDwe+CxwGfAE4PjMPN4FLkiZDeYlZE9MYqNqx\nbTXw48zcFhEPy8xrWopLkqR2jElTeBOqdmy7A3h6+fixrUQkSZKG4ohtkqR+GZOm8CaYxCVJ/TFG\nPcub0J3DEUmSeqb1mngm3LdlW60ydl9eP8yf37ul1vp77GqjhbZv2ZLFPxbetm3kmyD9wtTOG4pS\nWlw2p0uSNKFsTpckSYvNmrgkqUcavZ/4ojOJS5L6xeZ0iIgDI+LFDcYiSZIqGKkmHhGvBH4HWFEm\n8nWZ+aMmA5MkqXEduxVp5SQeEQ8A3gycADwKuBT4ebNhSZLUhm6dEx/lnWwDElgFkJk/yMyfDS4Q\nEesjYmNEbLzzzk0NhClJkuaqnMQz8+fAy4G3AW+JiHdExO5zltmQmTOZObPXXtMNhSpJUgNmh16t\nO42BkdoUMvMi4DnAnwCrgdc1GZQkSa3p6/3EASJiBbBX+fRnwLcpm9YlSdLOM0rv9KXAf6dI5NPA\nD4HnNxmUJEmtGZOm8CZUTuKZ+WPghIg4EHhyZn6w4ZgkSWpH2Dt91k+AK5sKRJIkVTPysKuZaRKX\nJE2ePjenV97AVLDnHsva3syC6t4P/Ibb649nc9Dee9QuQ+NnHO7DPQ4xSJMiOpTEu3NiQJKknvEu\nZpKk3gi6VRM3iUuS+iPKqSNsTpckaUJVqolHxDTwMYqBXu4Fjs3Mu9sITJKk5kWvm9NfCfyfzHxT\nROwL3NdCTJIktabPSfw+4ECAzPyXxqORJElDq3pO/HvAsyLiFW0EI0lS2yKikWkcDJ3EI2I/4Czg\nIcDLIuLZ5fxvRsSD5iy7PiI2RsTGOzbd0WjAkiTV0aUkXqU5/QnAVZl5Z0Q8HbgkItYAP8jMfx1c\nMDM3ABsAjjxyJhuLVpIk/UKV5vRvAsdExL6ZeRtwBvAe4MOtRCZJUtOiwWkMDF0Tz8zvRMTvAZ+L\niPuB24B1wLkRcUVmfretICVJakL0+RKzzPwb4G/mzP5oc+FIkqRhOeyqJKlXelsTlyRp0nUpiTt2\nuiRJE6r1mngCW7fVu8psydTiHzUdtPcetctY+Zz/UbuMH3/s5bXLkKRh3X3vlsUOoXFdqonbnC5J\n6o8xujysCTanS5I0oayJS5J6xeZ0SZImUNcGe7E5XZKkCWVNXJLUK12qiZvEJUn90p0c3k5z+uD9\nxDd5P3FJklrRShLPzA2ZOZOZM9PTq9vYhCRJ1UXRnN7ENA5sTpck9cq4JOAm1K6JR8QlEbFfE8FI\nkqTh1aqJR8QU8BDgrmbCkSSpXV2qiddtTj8M+ERm3tNEMJIktalrg73USuKZ+S3gtQ3FIkmSKrBj\nmySpX7pTETeJS5J6JDwnXkkAS6a684HV8eOPvbx2GW+5+Lu1y/iDpxxauwxJ/bBiV+t648xvR5LU\nK9bEJUmaUF1K4t6KVJKkCWVNXJLUL92piA+fxCNiDfBG4BhgC3AF8ObMvKml2CRJalzvmtMj4mDg\ns8BXgJnMPAL4CPCp8jVJkrQdEbEkIv4pIv5X+XxVRFwcEdeV/68cWPasiLg+Iq6NiOMXKnvYc+Ln\nAadl5oWZeR9AZl4C/Dbwp9XfkiRJO19TtyGtWJv/L8C3B56fCVySmYcAl5TPiYjDgHXAI4ATgPdG\nxJL5Cl4wiUfEocAdmfnNiDgpIq6IiI9HxCcy8zvAtoiYnrPO+ojYGBEb79h0R4X3KUlSu3ZmEo+I\n/YGnA385MPtk4Pzy8fnAKQPzL8jMzZl5A3A9cNR85Q9TE3808H/Lo4E3AccCrwOeWr5+HXDQ4AqZ\nuSEzZzJzZvX06iE2IUnSxJmerbCW0/rtLPPnwH8Ftg3MW5OZt5aPfwSsKR/vBwz2M7u5nLdDw3Zs\n2wpMA9/LzJ8AP4mIa8rX9gZuH7IcSZIWVYMd2zZl5sw82zkJuD0zL4+IJ29vmczMiMhRAximJv4t\n4GhgE3BwRDwoIg4AHh4RvwHsnZk3jhqAJEk7VTQ0LewJwDMi4gfABcCxEfE3wG0RsQ9A+f9sRfgW\nYO3A+vuX83ZowSSemd8GDgAeCrwV+CLwTuAi4PXAS4Z6K5Ik9UhmnpWZ+2fmgRQd1r6Qmb9NkT9P\nKxc7DfhM+fgiYF1ELI+Ig4BDgMvm28awzemvAv4WeANwZDnvCGDfzLxtyDIkSVp0Y3Cd+LnAhRHx\nUuBG4LkAmXl1RFwIXEMxHsvpmbl1voKGSuKZ+e2IeAbw+8CfAEsojg7eMvJbkCRpZ1ukW5Fm5qXA\npeXjO4HjdrDcOcA5w5Y79IhtmXkz8Iphl5ckSe1y7HRJUm8EsPit6c1pPYlvzeTn926pVcYe3pT+\nF/7gKYfWLuOfb/xJrfUf/eA9a8fQJbfcdU+t9fdbtVtDkdSzbdvIV7n8wtRUN34dtzbwWSzpyGfR\nPZVHWxtr3opUkqQJZRVXktQrHaqIm8QlSf1ic7okSVp0lZN4RJwSERkRD2sjIEmSWhNFc3oT0zgY\npSZ+KvDl8n9JkiZGUFxF0cQ0Diol8YhYATwReCnFOLCSJGmRVK2Jnwx8NjO/C9wZEUdub6GIWD97\nf9U7N22qHaQkSU3pc3P6qRS3U6P8f7tN6pm5ITNnMnNmr+npOvFJktSoiGhkGgdDX2IWEauAY4Hf\nKG9gvgTIiPjdzKw/vJEkSaqkSk38PwAfyswHZ+aBmbkWuAH4d+2EJklSw3rcO/1U4FNz5n0Ce6lL\nkiZEcQOUHjanZ+Yx25n3rmbDkSRJw3LYVUlSj4xPLboJJnFJUq90KIe3n8SnIli+1CHax0nd+4G/\n9IIra8fw/nWH1y6jCU3cQ3sc7gfuvcCb5b3ANSmsiUuSesXmdEmSJtEYXR7WBNu5JUmaUNbEJUm9\nMXudeFdUGXZ1K3AVsBTYAvw18GeZua2l2CRJalyHcnilmvg9mXk4QETsDXwYeCDwpjYCkyRJ8xvp\nnHhm3g6sB14dXWqXkCR1Xi+HXZ0rM78fEUuAvYHbmgtJkqT2jEn+bUQrvdMjYn1EbIyIjZvuuKON\nTUiS1HsjJ/GI+HVgK3D73Ncyc0NmzmTmzPTq1XXikySpOWFzOhGxGngf8BeZWX+8R0mSdoLiErPF\njqI5VZL4bhFxJb+8xOxDwDtbiUqSJC2oyv3El7QZiCRJ7RufpvAmOGKbJKlXOpTDHTtdkqRJZU1c\nktQrNqdrJE105B+Hne/96w6vXcbP791Su4w9dq2/+05NLf7n2YSuvA+NHy9AGm8mcUlSf3TsfuIm\ncUlSb3TtVqR2bJMkaUJZE5ck9UqXauKVk3hEbAWuGph1QWae21xIkiS1p0M5fKSa+D2ZWb97siRJ\nqsXmdElSr3SpOX2Ujm27RcSVA9Pz5i7g/cQlSWOpvMSsiWkctNKcnpkbgA0ARxw540gBkiS1wOZ0\nSVJvhHcxkyRpcnUoh4+UxHeLiCsHnn82M89sKiBJkjScykk8M5e0EYgkSTvDVIeq4janS5J6pUM5\n3LHTJUmaVNbEJUm9UVzj3Z2qeOtJPIFtXikOwNYGPohdlnRj59tj1/q73n1bttUuY9ku9Ruj7q8Z\nx9IGYpA0vKlu/IwCNqdLkjSxbE6XJPWKzemSJE2oDuVwm9MlSZpUlZJ4RKyJiA9HxPcj4vKI+FpE\nPLOt4CRJalJQjp/ewL9xMHRzehQnET4NnJ+Zzy/nPRh4RkuxSZLUuC71Tq9yTvxY4L7MfN/sjMy8\nEXh341FJkqQFVUnijwCuGGbBiFgPrAdYu/aAEcKSJKkF0a1bkY7csS0i3hMR/xwR35j7WmZuyMyZ\nzJzZa/XqehFKktSgYtS2+tM4qJLErwaOmH2SmacDxwFmaUmSFkGVJP4FYNeIeOXAvN0bjkeSpNYE\nxa1Im5jGwdBJPDMTOAV4UkTcEBGXAecDb2grOEmSmrazmtMjYteIuKw89Xx1RLy5nL8qIi6OiOvK\n/1cOrHNWRFwfEddGxPELbaPSiG2ZeSuwrso6kiT11Gbg2My8OyKWAl+OiH8AngVckpnnRsSZwJnA\nGyLiMIoc+whgX+AfI+LQzNy6ow04YpskqVei7KFed1pIFu4uny4tpwROpmjJpvz/lPLxycAFmbk5\nM28ArgeOmm8bJnFJUm801ZRe5vDpiNg4MK3/f7cXSyLiSuB24OLM/DqwpmzZBvgRsKZ8vB9w08Dq\nN5fzdsgboEiSNJpNmTkz3wJlU/jhEbEn8KmIeOSc1zMictQAWk/iASxdMh69+BbbLkts+GjSsl3q\nf56bfra5dhkr91hWuwxpXHVpYJRZi9GzPDN/EhFfBE4AbouIfTLz1ojYh6KWDnALsHZgtf3LeTtk\nVpEk9Uo0NC24nYjVZQ2ciNgNeArwHeAi4LRysdOAz5SPLwLWRcTyiDgIOAS4bL5t2JwuSVI79gHO\nj4glFJXmCzPzf0XE14ALI+KlwI3AcwEy8+qIuBC4BtgCnD5fz3QwiUuSemZnnSLIzG8Cj9nO/Dsp\nRjzd3jrnAOcMuw2TuCSpN4oR2xY7iuaMdE48Iu5eeClJktQma+KSpP7o2K1ITeKSpF7pUA5v5xKz\niFg/O4LNpk13tLEJSZJ6r5UknpkbMnMmM2emp73duCRpfOyssdN3BpvTJUm9Ye90SZI0FkZN4rtH\nxM0D02sbjUqSpJb0vjk9M63BS5Im0nik32aYjCVJmlB2bJMk9UbE4tyKtC0mcUlSr3Qoh7efxDff\nv43rflRvqPVD93lAQ9FIv2r6Actrl7HyN19da/0ff+MvaseQmbXLaMK4dPapa/P98979cSjLly5p\nIJJ6mtgvuvKddpU1cUlSr3TpwMQkLknqlQ7lcHunS5I0qayJS5J6I4hO9U6vVBOPiIyIPx14/vqI\nOLvxqCRJakMUzelNTOOganP6ZuBZETHdRjCSJGl4VZP4FmADcEYLsUiS1LoujZ0+Sse29wAviIgH\n7WiBiFgfERsjYuOP79o0enSSJDVsqqFpHFSOIzN/Cvw18DvzLLMhM2cyc2blKlveJUlqw6i90/8c\nuAL4QIOxSJLUqqBbg72M1CKQmXcBFwIvbTYcSZLaNRXNTOOgTrP+nwK2lUuStEgqNadn5oqBx7cB\nuzcekSRJLRqXWnQTHLFNktQbxUAt3cni49JLXpIkVdR6TXz50ikO+bUVCy+oibF1W/17FC/pUHvW\nnV9/d631T/3gxtoxfOiFR9QuY5clHtPPGod7gTehSzXOJnXo58fmdElSv3Tp2MZDb0mSJpQ1cUlS\nbwR06lakJnFJUq90qQm66v3E94+Iz0TEdRHxvYj4bxGxrK3gJEnSjg2dxKPo5vhJ4NOZeQhwKLAC\nOKel2CRJalxxrXj9aRxUqYkfC9ybmR8AyMytFPcVf0lEOHKbJGnsRQRTDU3joEoSfwRw+eCM8rak\nPwQe0mRQkiRpYa10bIuI9cB6gLUHHNDGJiRJGsmYVKIbUaUmfg1w5OCMiHggcABw/eD8zNyQmTOZ\nOTM9vbp+lJIkNaSvtyK9BNg9Il4EEBFLKG5H+sHM/Lc2gpMkSTs2dBLPzASeCTwnIq4DvgvcC7yx\npdgkSWrU7GAvXenYVvV+4jcBv9VSLJIktW5M8m8jujRwjSRJveKwq5Kk/hijTmlNMIlLknol6E4W\n3ylJPLPe+l06f9EFS7p0GDsGPvLimdplHHT6J2qX8b13P6t2GVM1941t22r+WDQQgzRJrIlLknqj\n6J2+2FE0xyQuSeqVLiVxe6dLkjShrIlLknolOtTRqnISj4itwFXlut8GTnPYVUnSJOjaOfFRmtPv\nyczDM/ORwH3AKxqOSZIkDaFuc/qXgEc1EYgkSa2Lbl22PHISj4hdgBOBz27nNe8nLkkaS+Ny85Im\njNKcvltEXAlsBH4IvH/uAt5PXJKk9o1SE78nMw9vPBJJklrWtY5tXmImSeqVDrWmO9iLJEmTqnJN\nPDNXtBGIJEntC6a8i5kkSZMnsDldkiSNAWvikqT+CHunV7Z1W9Zaf6pLn/gY2Hz/1lrrL1+6pKFI\nuuG+Ldtqrb/rsvqf53XvembtMi674a7aZTz24L1qrd+lZk6Nr74P9iJJkhYQEWsj4osRcU1EXB0R\n/6WcvyoiLo6I68r/Vw6sc1ZEXB8R10bE8QttwyQuSeqN2Y5tTUxD2AK8LjMPAx4LnB4RhwFnApdk\n5iHAJeVzytfWAY8ATgDeGxHzNtWZxCVJvTIV0ci0kMy8NTOvKB//jOL23fsBJwPnl4udD5xSPj4Z\nuCAzN2fmDcD1wFHzvpeRPgFJkjQdERsHpvU7WjAiDgQeA3wdWJOZt5Yv/QhYUz7eD7hpYLWby3k7\nVKljW0RsBa4q17sBeGFm/qRKGZIkLaYG+7VtysyZhbcXK4BPAK/JzJ/GQACZmRExcu/vqjXxezLz\n8Mx8JHAXcPqoG5YkaWcLisTXxDTU9iKWUiTwv83MT5azb4uIfcrX9wFuL+ffAqwdWH3/ct4O1WlO\n/xoLVPMlSeqrKKrc7we+nZnvHHjpIuC08vFpwGcG5q+LiOURcRBwCHDZfNsY6TrxsrfccWznXuLl\n6+uB9QBr1x4wyiYkSWpeQDTYnr6AJwAvBK6KiCvLeW8EzgUujIiXAjcCzwXIzKsj4kLgGoqe7adn\n5rwDe1RN4ruVgexH0cvu4u0tlJkbgA0ARxw5U2+kF0mSJlBmfhl2eLeV43awzjnAOcNuY6Rz4sCD\ny8A8Jy5JmijR0DQORjonnpn/BvwO8LqIcPx1SdJECHbedeI7w8gd2zLzn4BvAqc2F44kSRpWpVp0\nZq6Y8/y3mg1HkqR2jUcduhk2hUuSemVMWsIb4bCrkiRNKGvikqQeiZ15nXjrWk/iASzdxQr/OFm+\ndN4726mi5UsXf//eZUn9GB578F4NRFJPl35cNZ5mh13tii69F0mSesXmdElSr3SpxcckLknqle6k\ncJvTJUmaWCMl8Yj4vYi4OiK+GRFXRsTRTQcmSVLjyruYNTGNg8rN6RHxOOAk4IjM3BwR08CyxiOT\nJKlhXeudPso58X2ATZm5GSAzNzUbkiRJGsYoBySfB9ZGxHcj4r0R8aS5C0TE+ojYGBEb79h0R/0o\nJUlqSJea0ysn8cy8GzgSWA/cAXw0Il48Z5kNmTmTmTOrp1c3EqgkSU3o0v3ER7rELDO3ApcCl0bE\nVcBpwAebC0uSJC1klI5tDwW2ZeZ15azDgRsbjUqSpJaMSUt4I0apia8A3h0RewJbgOspmtYlSRpr\nRe/07mTxykk8My8HHt9CLJIkqQKHXZUk9Urfm9MlSZpQQfS5OV3SrxqH60W3bcvaZTTxNup+Fj/f\nvKV2DHss92dN/eHeLknqlTE47m6MSVyS1Btd653epXHgJUnqFWvikqT+iB43p0fEXsAl5dNfA7ZS\njJ8OcFRm3tdgbJIkNa63STwz76QYZpWIOBu4OzPf0UJckiRpATanS5J6xevEJUmaQAFMdSeHt9M7\nPSLWR8TGiNh4x6Y7Fl5BkiRV1koSz8wNmTmTmTOrp1e3sQlJkkYSDf0bBzanS5J6pUu90x3sRZKk\nCTVyTTwzz24wDkmSdopxaQpvgs3pkqTesHe6JEkaC9bEJUk9Mj49y5vQehK/f2ty+0831ypj7wcu\nbygaqXl337ul1vordq3/ZzjVkfbBPZZbr1DLOnYDFJvTJUmaUB72SpJ6pUMVcZO4JKk/it7p3Unj\nNqdLkjShKiXxiDgwIr41Z97ZEfH6ZsOSJKkd0dA0DmxOlyT1y7hk4AbYnC5J0oRqpSYeEeuB9QD7\n7b+2jU1IkjSSLg32UrUmnsPMH7yf+Kq9vJ+4JGl8RDQzjYOqSfxOYOWceauATc2EI0mShlUpiWfm\n3cCtEXEsQESsAk4AvtxCbJIkNa7vvdNfBLwnIt5ZPn9zZn6vwZgkSWrPuGTgBlRO4pl5DXBMC7FI\nkqQKvE5cktQbRVN4d6riJnFJUn+MUc/yJjjYiyRJE6r1mnhmsvn+rW1vRhVs27ajy/2HMzXVocPY\nBlx9y09rrX/0watqx1D3OwW/10ErH/fa2mX8+GvvXHihCZBZf98aN13a021OlyT1S4eyuM3pkiRN\nKGvikqQeCXunS5I0qXrZOz0ivhgRx8+Z95qIOK/5sCRJ0kKqnBP/CLBuzrx15XxJksZeU+Omj0tl\nvkoS/zjw9IhYBhARBwL7Al9qPixJklqyk7J4RPxVRNweEd8amLcqIi6OiOvK/1cOvHZWRFwfEdfO\nbfnekaGTeGbeBVwGnFjOWgdcmNu5iDAi1kfExojYeNdd3qVUktRLH6S40+egM4FLMvMQ4JLyORFx\nGEVefUS5znsjYslCG6h6idlgk/oOm9Izc0NmzmTmzKpV0xU3IUlSe6KhfwvJzP8D3DVn9snA+eXj\n84FTBuZfkJmbM/MG4HrgqIW2UTWJfwY4LiKOAHbPzMsrri9J0qKKaGYCpmdbnctp/RCbX5OZt5aP\nfwSsKR/vB9w0sNzN5bx5VbrELDPvjogvAn+FHdokSf22KTNnRl05MzMiao1rO8qIbR8BHo1JXJI0\ngRa5d/ptEbEPQPn/7eX8W4C1A8vtX86bV+UknpmfzszIzO9UXVeSpEW1+NeYXQScVj4+jeI09ez8\ndRGxPCIOAg6h6Ew+L0dskySpBRHxEeDJFOfObwbeBJwLXBgRLwVuBJ4LkJlXR8SFwDXAFuD0zFzw\nFqAmcUlSr+yssdMz89QdvHTcDpY/BzinyjZM4pKk3gi6NXZ660l82S5TrN1r97Y3owqmpjq0B4+B\now9etdghNPKdbmfcpsqiI7+OP/7aOxc7hLHRle+0q6yJS5J6pUuHJSZxSVK/dCiLj3KduCRJGgPW\nxCVJvbKzeqfvDJVq4hGxNiJuiIhV5fOV5fMD2whOkqSmNTh2+qKrlMQz8ybgPIqL1Sn/35CZP2g4\nLkmStIBRmtP/DLg8Il4DPBF4dbMhSZLUnjGpRDeichLPzPsj4neBzwJPzcz75y5T3o5tPcDaAw6o\nHaQkSY3pUBYftXf6icCtwCO392JmbsjMmcycWT29euTgJEnSjlVO4hFxOPAU4LHAGbO3VJMkadwV\nNyBr5t84qNo7PSg6tr0mM38IvB14RxuBSZLUuIZ6pk9k73Tg5cAPM/Pi8vl7gYdHxJOaDUuSJC2k\nUse2zNwAbBh4vhU4oumgJElqy5hUohvhiG2SpH7pUBZ37HRJkiaUNXFJUo+MT8/yJrSexBPYui1r\nlbFkqjsfuDSuYly620ot69KubnO6JEkTyuZ0SVJvBJ3q12YSlyT1TIeyuM3pkiRNqKrDrj4zIq6c\nM22LiBPbClCSpCZ1aez0qiO2fQr41Ozz8pajLwA+13BckiS1oku900c+Jx4RhwJ/CDw+M7c1F5Ik\nSRrGSOfEI2Ip8GHgdeXdzOa+vj4iNkbExk2b7qgboyRJjYmGpnEwase2twBXZ+ZHt/diZm7IzJnM\nnJmeXj16dJIkNaljtyKt3JweEU8Gno13L5MkaVFVSuIRsRL4APD8zPxZOyFJktSmMalGN6BqTfwV\nwN7AeXPGWX7bjprWJUkaF8H4NIU3oeolZm8D3tZSLJIkqQKHXZUk9UqHKuImcUlSv/S2OX0Ud/78\nPs7feGM8hPiXAAAOv0lEQVStMl5y1IHNBCMAMuvd3937Tv+qu+6+r9b6q1YsaygS6VfdWXPfBNjL\n/XOsWROXJPXKuIx73gTvYiZJ0oSyJi5J6pfuVMRN4pKkfulQDq98P/FdIuLvImJTRDyyraAkSdLC\nqp4TPw/4DnAK8NGI2L/5kCRJakdTNz8Zl4t0hm5Oj4g3Af+ama8vn78M+EhEnJSZ/9pWgJIkNalL\nvdOHTuKZ+eY5z78G/LvtLRsR64H1AKvW7FsnPkmStAOtXGI2eD/xFXvu1cYmJEkaTTQ0jQF7p0uS\nemVM8m8jHOxFkqQJZU1cktQr49KzvAkmcUlSj0SneqfbnC5J0oSyJi5J6o2gW83p1sQlSZpQrdfE\np/dYxkuOOrDtzUyEu+/dUruMFbvW/8qiS4ehY2DVimWLHYK0XXs1sG9u/P6PG4hEbbE5XZLUK12q\nx5jEJUm9Yu90SZK06KyJS5L6Y4xuI9oEk7gkqTfG6N4ljbA5XZKkCWVNXJLULx2qig+dxCPiDGAd\ncB/wAeBLwMnAVzLza3OWXQ+sB1h7wAGNBStJUl197Z2+BngC8DLgGOB/Ag8Evj53wczckJkzmTmz\nenp1I4FKkqRfNXRNPDPPLB9eC7ywnXAkSWqXvdMlSZpQHcrh9k6XJGlSmcQlSf0SDU3DbCrihIi4\nNiKuj4gzF16jGpvTJUm9srN6p0fEEuA9wFOAm4FvRMRFmXlNU9uwJi5JUjuOAq7PzO9n5n3ABRSX\nZjfGmrgkqTeCndo7fT/gpoHnNwNHN7mB1pP4FVdcvmm3pXHjAotNA5tqbKbu+l0qYxxi6FIZ4xDD\nuJQxDjGMSxnjEMMklfHgmuU35oorLv/cbktjuqHido2IjQPPN2TmhobKHkrrSTwzFxztJSI2ZubM\nqNuou36XyhiHGLpUxjjEMC5ljEMM41LGOMTQtTJ2lsw8YSdu7hZg7cDz/ct5jfGcuCRJ7fgGcEhE\nHBQRyyiGLr+oyQ14TlySpBZk5paIeDXwOWAJ8FeZeXWT2xiXJF73HEIT5yC6UsY4xNClMsYhhnEp\nYxxiGJcyxiGGrpXRSZn598Dft1V+ZGZbZUuSpBZ5TlzSRIgI72sszWESlzosohv3a4qIpwGXRMR+\nNcupdQoxorFLk2qJCH+7BUx4Eo+IVYsdg8bPuPzARcQBEbFHA+XUScRLFnHbg+XsXmPd44F3AC/M\nzFtG/X4j4lDg9yNirxHXfzDw9ojYf5T1yzKOiYjHj7p+Wca/B15Upwx1x6L82EXEr0fEA2qWsTfw\nqohYFhEH1ixrLH70F0tErJnzvPbnUeOH9siIeGyN7T4ReHFEPGaEdRurtZaf6euAV46ayCNi/zLh\njJQ0IuIpwIci4syIOGmUMoBlI643GMfTgD+OiLULLvz/rvtU4K+Ba4C7ADJz24jf1UpgFcV3MkoF\nYAXFCFx7l7GNso8/CXjhqOtHxLHAp4GzutLKonp2evKKiF2BVwOvi4gVI5axP3AgRS3jT4A3jVJW\nRDw8Ip4PnBERu42w/tMi4vXl40U7EIiIpRHxxIg4KyKeUSUJRsTDgFsj4s8i4uVQ/EiWrw39niLi\nkIh4bEQcGxErR/mhjYgTKXq5/luV9QbWPwF4N7AFeNAIRSwpy2niqo07KK4R3Rd4SdVEHhEnAx8D\n3g98PCLOLq8zHXb9E4BzgK8CewDPjohKwz2WCfSCiHhTRDyryroDZZwEvA24NDNvWmj5OeseB/wF\n8FqK9/GS8iCNzMyq+1dmfh34EPBA4NVVE3l5adAXgfdFxANn/04q+mq5faquP9Ai8SrgsmyoV7IH\nAxMuM3f6BBxEkXzPAlZUXHcF8P8BhwJvBe4Gjh4xjj8EbgReNcK6xwH/TDGY/dTg/J38WS6jGDzg\n9cDZwHOAz5b/P2CI9fcHvgy8Afg8Ra3nGcADK8TwdOAK4FPAxRRjBT+mfC2GLOME4CvAU8vnK4GD\nKsTwJOD6ufsC8Igh158GfgCsKp/vMuL3cQjw0Nn3DvwWxYHF7wy7rwPHAN8FjgT2LPf1/0uRlJcM\nsf4qYBvwW+XztcBHgVMqvI8TgK8DpwNvBv4H8JCKn8WvUSS93xzYV3cv97ldh1j/N4HHl48fCryF\n4oDgCQPLzLt/AY8H1s2Zd3T5+/P7C/2NlJ/lioHnewDvA44pn08N8T6Oo6i4PIHit+8rwL5zlpm3\nHIq7YH0beFz5/CrgiGE+gznlPBF4LvCK8v9K3+kC39XyKrE4NTMtSu0xM2+guD3bKuCNVc6XZebd\nFD9Ifw88leIP8XkRcXJEnDpsORHxIOCxFD9Q10TEQyPi6Ig4fMga6JOAT2TmZ/KXNdfdgfdGxLOH\njWM7cS2tsOwuwIXApzLzHZl5NvC/gRcAh1N8PvPKzJuBy4AjgKdRfK4vAf4uIo6KiEMWiOEE4A+A\nMzLzmZn5FIof/Isi4tGZmQt9nmWN6O+Bt2fm5yPiYIoDkyq9kR8DvDuL2tZsuW8H/ncUgy3MKzM3\nAf8Z+GrZkrAlInapUkspm76vBb4UEacD/wn4O4pk+EDgZUPu648H3pWZlwP3ZuZ3gedRJNazhngv\nd1EcPJxb1hhvAu4H1sy/5i/ex+z38dbMfA/F97mM4kCnis3ldu8tW+DeSPG9/i1w3kI14cz8RmZ+\nNSKmMvNaigPM+4GTZs8rZ5lB5rGSoin/OQPlfh34OEUrybE7WjEi9iyXe3PZMkJm/pyiWf9l5fNh\natPLKP6+/ivFZ/lwilaFZ5TbifnKKf/OHwa8LDO/FsXtLW8HVpcx5MByOyojyv3zbyl+Gx4KHDZE\n7AuKiIOAUwZj0U60mEcQwMEUNZRKR2/AM4HvAx8sn78duJVqNY0HUDRtvZ7iiPRaih/c8xmiZg48\nn+KI/GyKmujTgN+gOMI9gyLJL634vqYoblV3xpDLPxQ4u3y8C/BO4EfAyyl+vP6ReWqis587xY/M\nBRQ1pyeXn+37KM69nQfssYP1Z2t8J5XPdx147U1lOQ8a8r3M1uYfRVGbf92Q682+h3dTJJ3Z+ScC\nH6RIiDcCJw9Z3onA94CVs59r+f8xlK0LC6x/bPmZ/GeKUwMfAz4A/HeKg8/TgeULvJfzBr7XoKx9\nl5/NpRTnZBf8mynfy3UUTdKfZIja75zv42rKFpnyb+PLwJ9TNG9PL7R/l7G/jmK0qpvL7+NlFDXh\nDwLPqPL3UZZ5SLlvvYshW+DKz+GbwPPmfM4vpzgI3mEtmOI36oUU412fQ1Ej3qX823r+CPEfBHwY\n+J/AFygOEr4KPHu+73RgP5wq//8j4PUDrz8HWD/feymX+yhwcNW45ynvNykOzv6BIf/WnZqdFj2A\ngZ3hUcDuQyy3R7nTPIoimb+1/KH60Ajb/PcUTeIXlD8Kz6E4qHjbEOuuAf4j8E8UCeQzFM3Rnwe+\nA1zJnCazIWP6DYpzqUcMsex/Av6yfPw+itMDR1M0151GcXCy3wJlBEUSfwvFUfp3KA+Gyh/MlQus\n/3SKpr29yufLB177AjBT4b2fQJEAzyyfLxmYf8wC6x5HkfxnmxiXAsvKx78PnFohjrmJ/FUUTfUH\nDLn+bNPnMoqm7NPKH7k7gW8t9GM38F6OLJ9Ple9nX+AT7OCgap59fBuwd/m8SiKfPQh4dxn/MykS\nxdeBv2SIUy4Up78eV+6Lg/vG+4Hfrvr3Ua77MIoWidUV1nla+bf6vIF56ygOrhY8dUJxSuMsigPb\nfyj/3t5RYfuDp9zOAD5QPt6n3D9+fchyZg9A3gB8snz8gnK/etgQ632C8nRP3YnigOCvyumwJsp0\nGuF7WOwAyp1hBfAaYHrI5WePRn+X4vzrCRTnyv5whG0/gPK8GPBIiqPi9w8bR/kD97iBeS+gOBDY\nrcbncSxlUlxguacCrykfv4tfni87FPiX8gfneUNu86EUtfg/GCHeuUlvafn/Z4BHVizrKRQHEnuW\nz19M0dw/7/lxioO7synOdR41MP9UioOaoX4k57ynb1LUOr8NHF5x/adTnNeePce+kqIGfeAQ6w6+\nl5mB+c+lOMe85wjv5WrKRF5x3dmDgDVz9vuh/lZ3UOZzgI3UqBFSsZWrXOd4ilaZN1K0EHyjyv7J\nLw8q31ruU5sYot/Jdsp5MPA3o773soxHUbSKPLt8HwsmUYpz1n86yn6wnbJeBJxXPl5WtzynGt/F\nYgfwi0Aq/FFS1B5XUTTvfQ44t/xhWVtj+/tQdMz6OPAfho2n/GH9EMVR/Ysoaik7PCIe9v0Nudyh\nFOcuD6ZI6B+lqMkfRtFkd3T5Az5U7atMmGczRIvIdtadm8hnP4tREseJFLX7VwJfYvjOaftRtEZc\nCvwZ8McUiXSkWgJFIt4GPHrE9U8st7/gAdkO3subKPo4nEvRfHpNjVhOpjhdMTXs/jXnfVxT98e/\n/Bt7DcUBRaWDu6Ymir4T51L0hXl4xXVj4PHeDBzYVCxnz/LzHKlDblnGAeW+eW2V99FUwgUeQnna\nr+r+5NTsNLFjp5edOPbMzE1lB6ZzM/POEcuaouhsckpmnhPF8I6bMnPBS53Kzi/PoTgivouiKf6q\nUeIYRUQ8l6Jp/9MUtekzKM7ZnQH8OsXBxSsz82dDlPUwitrfumHe+3bWP7Fc/70U5xHXZ+a3qpZT\nlnUSxXncx2SFu/6UlwoeQVGjv4Xi0qbrRomhLG/3UT6LgfVPpjgwOjKrX1K0GzBDUYPcBPxDFh28\nRo1lRRYdQ0dZ92SKg4qZqu9joIzdKFqZrs3M60cpY7GVndBq/WiWnSV/j+KOVv8yYhlLKfrA/EWd\nfaKOiFiemZsXY9v6pYlN4vDLP6iIWJKZWxsst3J55R9VZOZ9TcUx5HangWdRXDf/SYrLpPalqPW8\ngyIhV0mCdZPWSMm3jVjGRZ3kOU668j7GQUTskplbapaxNDPvbyomTaaJTuIqlAOJHEVxdH8HxWAn\nPwXek5nXLEI8nUi+kjTuTOIdUl6Lu7mN1glJ0vjp9ZjhHbR54HzdSOctJUmTw5q4JEkTypq4JEkT\nyiQuSdKEMolLkjShTOKSJE0ok7gkSRPq/wcNX6Rf28As/wAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "seq.plot_confusion_matrix(dev, local_3.predict(dev))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Another high frequency error stems from misclassifying verbs ('V') as common nouns ('N'). We can understand these errors better by again inspecting examples. " ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Previous\n", "  \n", " Next\n", "
\n", "
theplayersandhiswifeownsmashburger
DN&DNV^^
DN&DNNNN
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTrueown
-6.60-1.690.00
-6.67-1.090.00
1 / 9
\n", "
RT@TheRealQuailman:CurrentlylaughingatLakerhaters.
~@~RVP^N,
~@~!NP^N,
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTruelaughing
-6.60-1.690.00
-6.67-1.090.00
2 / 9
\n", "
@ShiversTheNinjaforgivemeforblowingup
@VOPVT
@NOPNT
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTrueforgive
-6.60-1.690.00
-6.67-1.090.00
3 / 9
\n", "
@ShiversTheNinjaforgivemeforblowingupyouryoutubecomment
@VOPVTD^N
@NOPNTDNN
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTrueblowing
-6.60-1.690.00
-6.67-1.090.00
4 / 9
\n", "
Question:HowCANyoumendabrokenheart?
N,RVOVDAN,
!~RVONDVV,
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTruemend
-6.60-1.690.00
-6.67-1.090.00
5 / 9
\n", "
lastnight,butdidn'tbothercallingShawnbecauseI'd
AN,&VVV^PL
AN,&VNV!PL
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTruebother
-6.60-1.690.00
-6.67-1.090.00
6 / 9
\n", "
arein!Seewhopassedandwhomadethe
VP,VOV&OVD
VP,VON&OVD
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTruepassed
-6.60-1.690.00
-6.67-1.090.00
7 / 9
\n", "
andwatchthenewsandtuneoutoversomefresh
&VDN&VTPDA
&VDN&NTPDA
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTruetune
-6.60-1.690.00
-6.67-1.090.00
8 / 9
\n", "
that,regretfullyIwastiedup,Physedat
P,ROVVT,NP
P,NOVNT,!P
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerword
FalseTruetied
-6.60-1.690.00
-6.67-1.090.00
9 / 9
\n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "util.Carousel(local_3.errors(dev[:20], filter_guess=lambda y: y=='N',filter_gold=lambda y: y=='V'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We find a couple of errors where verbs like \"laughing\", \"blowing\" or \"passed\" are misclassified as common nouns. When looking at the features of those instances, and the corresponding weights, we see that for $f_{\\text{word},w}$ feature template weights are $0$. This usually suggests that the word has not appeared (or not appeared as a verb) in the training set. However, we can tell that these words may be verbs without having to have seen them before because they come with standard verb suffixes such as \"ing\" or \"ed\". We can easily test for such cases by incorporating features that look at length two and three suffixes of the target token." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.7760729836201534" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def feat_4(x,i):\n", " return {\n", " **feat_3(x,i),\n", " 'last_3': \"\".join(x[i][-3:]),\n", " 'last_2': \"\".join(x[i][-2:]),\n", " }\n", "local_4 = seq.LocalSequenceLabeler(feat_4, train)\n", "seq.accuracy(dev, local_4.predict(dev))" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Previous\n", "  \n", " Next\n", "
\n", "
theplayersandhiswifeownsmashburger
DN&DNV^^
DN&DNNVN
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerlast_2last_3word
FalseTruewnownown
-6.20-0.550.000.000.00
-6.89-0.090.000.000.00
1 / 4
\n", "
Question:HowCANyoumendabrokenheart?
N,RVOVDAN,
N~RVONDVV,
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerlast_2last_3word
FalseTruendendmend
-6.20-0.550.000.000.00
-6.89-0.090.000.000.00
2 / 4
\n", "
lastnight,butdidn'tbothercallingShawnbecauseI'd
AN,&VVV^PL
AN,&VNVNPL
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerlast_2last_3word
FalseTrueerherbother
-6.20-0.550.000.000.00
-6.89-0.090.000.000.00
3 / 4
\n", "
andwatchthenewsandtuneoutoversomefresh
&VDN&VTPDA
&VDN&NTPDA
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerlast_2last_3word
FalseTrueneunetune
-6.20-0.550.000.000.00
-6.89-0.090.000.000.00
4 / 4
\n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "util.Carousel(local_4.errors(dev[:20], filter_guess=lambda y: y=='N',filter_gold=lambda y: y=='V' ))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This indeed removed the earlier mistakes for unseen verbs.\n", "\n", "## Maximum Entropy Markov Models (MEMM)\n", "\n", "There is an observation we have not made use of. For PoS tagging as well as many other sequence labelling tasks we often have *dependencies* between consecutive labels. For example, after a non-possessive pronoun (\"O\") such as \"you\" a verb is more likely than a noun. This means that the current label $y_i$ does not only depend on the observation $\\x$ and index $i$, but also on the previous label $y_{i-1}$ (or even labels further in the past). Our *local* model above cannot capture this. \n", "\n", "One simple extension to the local model is the [Maximum Entropy Markov Model (MEMM)](http://www.ai.mit.edu/courses/6.891-nlp/READINGS/maxent.pdf). This model can again be understood as a product of local logistic regression (aka Maximum Entropy) classifiers $\\prob_\\params(y_i|\\x,y_{i-1},i)$, but now each of these classifiers can use the previous label as observed feature and hence makes a first-order Markov assumption. Let $y_0=\\text{PAD}$ be padding for the label sequence to operate properly $\\prob_\\params(y_1|\\x,y_{0},i)$, then the MEMM defines the following distribution over \n", "\n", "$$\n", "p_\\params(\\y|\\x) = \\prod_{i=1}^n p_\\params(y_i|\\x,y_{i-1},i)\n", "$$\n", "\n", "The individual terms are defined as log-linear models as before, this time with access for features to the previous label: \n", "\n", "$$\n", " p_\\params(y_i|\\x,y_{i-1},i) = \\frac{1}{Z_{\\x,y_{i-1},i}} \\exp \\langle \\repr(\\x,y_{i-1},i),\\params_{y_i} \\rangle\n", "$$\n", "\n", "where $Z_{\\x,y_{i-1},i}=\\sum_y \\exp \\langle \\repr(\\x,y_{i-1},i),\\params_{y_i} \\rangle $ is a *local* per-token normalisation factor.\n", "\n", "We show the factor graph of this model below. The observed \"P\" node corresponds to out setting of $y_0=\\text{PAD}$. " ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "\n", "\n", "\n", "%3\n", "\n", "\n", "x\n", "\n", "x\n", "\n", "\n", "y0\n", "\n", "P\n", "\n", "\n", "t1\n", "\n", "t1\n", "\n", "\n", "y0--t1\n", "\n", "\n", "\n", "y1\n", "\n", "y1\n", "\n", "\n", "y1--t1\n", "\n", "\n", "\n", "t2\n", "\n", "t2\n", "\n", "\n", "y1--t2\n", "\n", "\n", "\n", "y2\n", "\n", "y2\n", "\n", "\n", "y2--t2\n", "\n", "\n", "\n", "t3\n", "\n", "t3\n", "\n", "\n", "y2--t3\n", "\n", "\n", "\n", "y3\n", "\n", "y3\n", "\n", "\n", "y3--t3\n", "\n", "\n", "\n", "t4\n", "\n", "t4\n", "\n", "\n", "y3--t4\n", "\n", "\n", "\n", "y4\n", "\n", "y4\n", "\n", "\n", "y4--t4\n", "\n", "\n", "\n", "t5\n", "\n", "t5\n", "\n", "\n", "y4--t5\n", "\n", "\n", "\n", "y5\n", "\n", "y5\n", "\n", "\n", "y5--t5\n", "\n", "\n", "\n", "t6\n", "\n", "t6\n", "\n", "\n", "y5--t6\n", "\n", "\n", "\n", "y6\n", "\n", "y6\n", "\n", "\n", "y6--t6\n", "\n", "\n", "\n", "t1--x\n", "\n", "\n", "\n", "t2--x\n", "\n", "\n", "\n", "t3--x\n", "\n", "\n", "\n", "t4--x\n", "\n", "\n", "\n", "t5--x\n", "\n", "\n", "\n", "t6--x\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seq.draw_transition_fg(7)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Training MEMMs\n", "We can train MEMMs by optimising the conditional likelihood of the gold label sequences, $\\sum_{(\\x,\\y) \\in \\train} \\log \\prob_\\params(\\y|\\x)$. This objective can be formulated as follows:\n", "\n", "$$\n", "\\sum_{(\\x,\\y) \\in \\train} \\sum_{i=1}^{|\\x|} \\log \\prob_\\params(y_i|\\x,y_{i-1},i) \n", "$$\n", "\n", "The objective is equivalent to a logistic regression objective for a classifier that assigns labels based on previous gold labels. This makes MEMMs easy to train: one can simply use a logistic regression classifier library and prepare a list of classifier training instances by iterating over all sequences and then generating one training instance per token. \n", "\n", "### Prediction in MEMMs\n", "To predict the best label sequence we need to find a $\\y^*$ with maximal conditional probability given the observed word sequence $\\x$:\n", "\n", "$$\n", "\\y^* =\\argmax_\\y \\prob_\\params(\\y|\\x).\n", "$$\n", "\n", "Due to the label dependencies, we cannot simply choose each label in isolation in order find the maximum. One solution to this problem is a forward greedy method:\n", "\n", "1. set $y_0 \\leftarrow \\text{PAD}$\n", "1. for $i$ in $1 \\ldots |\\x|$:\n", " 1. $y_i \\leftarrow \\argmax_{y} \\prob_\\params(y|\\x,y_{i-1},i)$\n", " \n", "This is an approximation because you can choose a locally optimal $y_i$ but for choosing the next $y_{i+1}$ a different previous $y_i'$ could have led to a much higher aggregate probability. In other words, you may find that \n", "\n", "$$\n", "\\prob_\\params(y_i'|\\x,y_{i-1},i) \\prob_\\params(y_{i+1}'|\\x,y_i',i) > \\prob_\\params(y_i|\\x,y_{i-1},i)\\prob_\\params(y_{i+1}|\\x,y_i,i)\n", "$$ \n", "\n", "even though $\\prob_\\params(y_i|\\x,y_{i-1},i) > \\prob_\\params(y_i'|\\x,y_{i-1},i)$. We will address this issue later.\n", "\n", "We provide an implementation of MEMMs, wrapping around a Scikit-Learn logistic regression model, in `seq.MEMMSequenceLabeler`. Below we first implement the greedy prediction algorithm above using this class. The class provides a local `predict_next` method that chooses the best label conditioned on a previous label. With this function at hand the greedy prediction algorithm is easy:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "def memm_greedy_predict(memm: seq.MEMMSequenceLabeler, data, use_gold_history=False):\n", " result = []\n", " for x, y in data:\n", " y_guess = []\n", " for i in range(0, len(x)):\n", " prediction = memm.predict_next(x, i, y_guess if not use_gold_history else y)\n", " y_guess.append(prediction)\n", " result.append(y_guess)\n", " return result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we are ready to define a specific MEMM model. To specify the model we need to define its feature function. We use the feature function from before, assessing only $\\x$ and $i$, and append to it (see the `**feat_4(x,i)` notation) a new feature that captures the first item of the label history so far. Notice that we could assess labels further in the past (after increasing the `order` appropriately) but leave this for the reader to test. " ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8100767157370931" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def memm_feat_1(x,i,hist):\n", " return {\n", " **feat_4(x,i),\n", " 'prev_y': hist[0],\n", "# 'prev_nom': hist[0] in {'N','^','O','S','Z'}\n", " }\n", "\n", "memm_1 = seq.MEMMSequenceLabeler(memm_feat_1, train, order=1, C=10)\n", "seq.accuracy(dev,memm_greedy_predict(memm_1, dev))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have improved the predictions even further, and did avoid some of the verb mistakes we observed earlier. To illustrate this let us consider the remaining errors on the development subset used earlier. We see that instead of 5 errors we now encounter only 3. " ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Previous\n", "  \n", " Next\n", "
\n", "
theplayersandhiswifeownsmashburger
DN&DNV^^
DN&DNNVN
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerlast_2last_3prev_yword
FalseTruewnownNown
-5.010.860.000.000.000.00
-5.790.570.000.000.000.00
1 / 2
\n", "
andwatchthenewsandtuneoutoversomefresh
&VDN&VTPDA
&VDN&NPPDA
\n", " \n", " \n", " \n", " \n", " \n", "
first_atis_lowerlast_2last_3prev_yword
FalseTrueneune&tune
-5.010.860.000.000.000.00
-5.790.570.000.000.000.00
2 / 2
\n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "util.Carousel(seq.errors(dev[:20], memm_greedy_predict(memm_1, dev[:20]), 'V', 'N',model=memm_1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We should also inspect what transition weights the model learnt. For the case of verbs ('V') we observe a high weight for $f_{\\text{prev_y},\\text{O}}$, indicating that pronouns are often followed by verbs, as we expected earlier. " ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlUAAAGTCAYAAAABX+eGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYJWV59/HvPTMggiISENxGQUBwi8uIirsCiqBEfV1Q\nYtxC9HXURCMuvIqiiYNijAajEoGgErfEfV/iFhUVicYtMWJwN8EkaExUAtzvH08dOdPO2ueuOn1q\nvp/rOld3n+6uu6qeWn7nqS0yE0mSJM1m1bxHQJIkaQwMVZIkSQUMVZIkSQUMVZIkSQUMVZIkSQUM\nVZIkSQVKQlVE3Dsi/ikivhkRz6gYpiRJ0iKJWe9TFRGrgW8ARwDfAz4PHJeZX5t99CRJkhZDRU/V\nocA3M/NbmXkp8Ebg2ILhSpIkLYyKUHVd4LtTP3+ve0+SJGmHsWaoQhFxAnACwG677Xabgw8+uNd6\nX/7+T8qHefPrXqP3OpuqMVSdMc2zRZ2WoeqMfVqGqjOmeTamaRmqzpjm2aJOy+bqVPvCF77w48zc\ne2t/V3FO1R2A52bmvbqfnwmQmS/c3P+sW7cuzz///Jnqbs0Nn/Ge8mFetOHo3utsqsZQdcY0zxZ1\nWoaqM/ZpGarOmObZmKZlqDpjmmeLOi2bq1MtIr6Qmeu29ncVh/8+DxwYEftFxM7AQ4F3FgxXkiRp\nYcx8+C8zL4uI9cAHgNXAWZn51ZnHTJIkaYGUnFOVme8F3lsxLEmSpEXkHdUlSZIKGKokSZIKGKok\nSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIK\nGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKok\nSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIK\nGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKok\nSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKrJn3CPTlog1Hz3sUJEnSDsSeKkmSpAKGKkmSpAKG\nKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmS\npAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAIzhaqIeFBEfDUiroiIdVUjJUmStGhm7an6\nCvAA4BMF4yJJkrSw1szyz5n5dYCIqBkbSZKkBTXYOVURcUJEnB8R51988cVDlZUkSRrEVnuqIuLD\nwL6b+NVJmfmObS2UmWcAZwCsW7cut3kMJUmSFsBWQ1VmHj7EiEiSJC0yb6kgSZJUYNZbKtw/Ir4H\n3AF4T0R8oGa0JEmSFsusV/+9DXhb0bhIkiQtLA//SZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIk\nFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBU\nSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFVgz7xFYdBdtOHreoyBJklYA\ne6okSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKok\nSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIK\nGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKok\nSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIK\nGKokSZIKGKokSZIKrJnlnyPixcB9gUuBC4FHZeYlFSOmjV204eh5j4IkSdqCWXuqPgTcLDNvAXwD\neObsoyRJkrR4ZgpVmfnBzLys+/E84Hqzj5IkSdLiqTyn6tHA+wqHJ0mStDC2ek5VRHwY2HcTvzop\nM9/R/c1JwGXAuVsYzgnACQBr165d1shKkiStVFsNVZl5+JZ+HxGPBI4B7pmZuYXhnAGcAbBu3brN\n/p0kSdIimvXqv3sDJwJ3zcz/qRklSZKkxTPrOVWnA1cHPhQRX4yIVxWMkyRJ0sKZqacqMw+oGhFJ\nkqRF5h3VJUmSChiqJEmSChiqJEmSChiqJEmSChiqJEmSCsx09Z/G5aINR897FCRJWlj2VEmSJBUw\nVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmS\nJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUw\nVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmS\nJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUw\nVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmS\nJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBUwVEmSJBWYKVRFxPMj\n4h8i4osR8cGIuE7ViEmSJC2SWXuqXpyZt8jMWwLvBp5TME6SJEkLZ6ZQlZk/nfpxNyBnGx1JkqTF\ntGbWAUTEHwGPAH4C3H0Lf3cCcALA2rVrZy0rSZK0omy1pyoiPhwRX9nE61iAzDwpM68PnAus39xw\nMvOMzFyXmev23nvvuimQJElaAbbaU5WZh2/jsM4F3gucPNMYSZIkLaBZr/47cOrHY4F/nG10JEmS\nFtOs51RtiIgbA1cA3wYeN/soaewu2nD0KGpIkjRtplCVmQ+sGhFJkqRF5h3VJUmSChiqJEmSChiq\nJEmSChiqJEmSChiqJEmSChiqJEmSChiqJEmSChiqJEmSChiqJEmSChiqJEmSChiqJEmSCsz6QGVp\nh+aDmyVJE/ZUSZIkFTBUSZIkFTBUSZIkFTBUSZIkFfBEdWkBeEK8JK189lRJkiQVMFRJkiQVMFRJ\nkiQVMFRJkiQVMFRJkiQVMFRJkiQVMFRJkiQV8D5VkgDvhSVJs7KnSpIkqYChSpIkqYCH/yQNysOM\nksbKnipJkqQChipJkqQChipJkqQCnlMlaZQ8d0vS0OypkiRJKmCokiRJKuDhP0laJg8xSppmT5Uk\nSVIBQ5UkSVIBQ5UkSVIBQ5UkSVIBQ5UkSVIBQ5UkSVIBQ5UkSVIBQ5UkSVIBQ5UkSVIBQ5UkSVIB\nQ5UkSVIBQ5UkSVIBH6gsSSucD26WFoM9VZIkSQUMVZIkSQU8/CdJ8hCjVMBQJUkajOFNY+bhP0mS\npAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAKGKkmSpAIl\nj6mJiKcCpwF7Z+aPK4YpSdJy+TgczcPMoSoirg8cCXxn9tGRJGkxDBXcDIiLo+Lw30uBE4EsGJYk\nSdJCmilURcSxwPcz80tF4yNJkrSQtnr4LyI+DOy7iV+dBDyLduhvqyLiBOAEgLVr127HKEqSJK18\nWw1VmXn4pt6PiJsD+wFfigiA6wEXRMShmfmjTQznDOAMgHXr1nmoUJIkjcqyT1TPzC8D15r8HBEX\nAeu8+k+SJO2IvE+VJElSgZL7VAFk5g2rhiVJkrRo7KmSJEkqYKiSJEkqYKiSJEkqYKiSJEkqYKiS\nJEkqYKiSJEkqYKiSJEkqYKiSJEkqUHbzT0mStJgu2nD0vEdhFOypkiRJKmCokiRJKmCokiRJKmCo\nkiRJKmCokiRJKmCokiRJKmCokiRJKmCokiRJKmCokiRJKmCokiRJKmCokiRJKuCz/yRJ0iDG/oxB\ne6okSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKGKok\nSZIKGKokSZIKGKokSZIKGKokSZIKGKokSZIKRGYOXzTiYuDbgxfevL2AH4+gxlB1xjQtQ9UZ07QM\nVcdp2bHrOC07dp2hpmVb3SAz997aH80lVK00EXF+Zq5b9BpD1RnTtAxVZ0zTMlQdp2XHruO07Nh1\nhpqWah7+kyRJKmCokiRJKmCoas4YSY2h6oxpWoaqM6ZpGaqO07Jj13Faduw6Q01LKc+pkiRJKmBP\nlSRJUgFDlSRJUgFD1Q4uIlZ1X2Pe46LhRcTVI2KnAeq4fEkr0NQ+wDxQwJm4TEPsiPoWEdcCzoiI\nnYHednoRcZ2IuHv3fZ91Rrc89zy/dgfeCty8z1oRcSPgCRGxpo/hT9UZVXCLiKsMUOPaEbHVGxoW\n1Nm5+9prG/W9jA1tgPl1MPDhiLh6Zl7RZ62uXi/b6IjYo4/hLsfodkJDiIgbA2dGxG/2NPx9IuJm\nEbF/RFyzjxqd6wD7ZualPa9QTwQeBpA9XRnRtcnJEfHsvlawIXZyU7V2hf7mVzfsnwJfAp4bEbv2\nUatrlzcD/5aZl1UPf6rOQcDTIuLpEbFbX3WG0i3D74mIO/ZY42Dgo8BN+qrR1dkT+NuI+M3MzB7D\n+wHAH0fEET1vN5fWvVZEHBgR94yIXYoHv1fx8H6la/9zgJsCe/ZYJybr5GQ/U7kMdNvlD0bEH1YN\ncxaGquW5Ky0kHBsRh1YOOCIOAT4GnAS8EXhVRDy8uMakl+3LwDUj4i6Vw9+E1wJX6WqXL3PdxuEN\nwH8DxwAv6aHGnsALIuL21cPeRK2DgZdGxM16rLFz9+0rgH8FbtC9X9Y+XaD6IPDqzHxzjzvTg2nr\nSgAPAF7eV09yt4PofbuZmZfQtgOnVW9j4Ffbmb8ATsvMj/fZI5KZ/wH8LXBORNysp/B+CG0Z+DHw\nn5n5n9U1tlD3PcDTgdfRlr0Tioa9C/CciLhBxfCWDPvGtED1p7R19M7VNbo6twZOpbX9qyPiaRFx\nh8pwnZm/BNYDj46IJ1QMcxaGquU5H/gH4GbA3SPiNhUDjYjr0g7HvCgzjwMeArwJ+MOIOL6oxr7A\nSyLiccBdaMFq50383UwLfESsjXa+zm7Aj4DbRsQNqnvEImIv4O3ABzLzRbRp2i8ifmvJ3826rP8G\n8HPg+On2rt4ZRcT1gPcD5wFfn3r/fhFxbMHw10bEbpl5affWd4GrA78PV36SLKhzCG0n8zPgyK5m\neS/FkvY/lfaB5zbAY4uGH93XAyNi92yuiIg9IuK2FTWW1Ntjsqxm5gtoQeGVEXG7whp7AW8DzsvM\n10TEauCTFcvXkjq/OhSXmc+hTcubJh8WqsJpRFwbeAvwisx8UWae371/uy489KIL838J/GlmPha4\nNfD3wGER8fhlDnOyvO0OrAWuC5Qe0oyI/WkfdM/IzDcA36FtA0p1H0BfRptHj+pqfpz24fSuVeE6\nIlZn5ueA44H1EfG0iuEul6FqG0XEXl1vBZl5AXAm8EvgAOBBXSKf1fWAT2fm2V2df8nMtwLPAx7c\nha5ZXQf4InBH4DHAPYC/jIj7R8QDACLiqrMs8N1G+unAu2mh8IHA/9A2EtVBZDfaDmJNRNyu+9Ty\nZeBeEXFaRNwyIvaaNSxk5j8DfwV8D3jsVLCanOR5UNQcHrw28L7MPDszL596f1dg/1kG3M33E4EL\nIuJR3fy6DHgycGBEHD3L8KfqXL2r89LMvClwGfDXPQWrpe3/C+CdwH/NOuCIiG58j6HtHNZM9YAd\nBjwvIsoOnUU7t+mfgY9ExKkRsTYzXwb8CfBnVR/euHKe/bILa28APpeZ7yga/mRa1kc7LAtAZm6g\nBe03Tz5gFQWrfWgB8eyp+k+j9cQ9pqfwe81u+Odn5rkAmfkj2jbiM8CtI+I3tne43fJ2BK0H6cXA\ndzPzwroxB+Bg4NTMPLP7+SLa/uBXRzHiyp7sZemC+zOB/5OZXwOuCfw27UPW44AnR8TaWWp0dVZl\n5uXd1wto4e3Y7gP9XM6xNFRtg24D8Y+0jcFJ3dufBD4BbKAd2npQRMz68MfLgVtExD5L3j+PtiHc\n7pV0qcy8IDPPAh6RmccDL6cdT7858JSIeDvtxMWrLnehzMzLM/MJtE8OL+jG/efA0yNil+Ku32/T\nPgH9O/DQiDgVuBfwaeAQWrj7TERcbbk1Jp+4M/MfaTug79CC1aHdCn034AJqzk35MW2jcGpE/HFE\n/EFEPBi4D63H58iIOGo5A+56WdYDz6cFtNdFxFNpO6UP0p2/UbCj+znwvO5TMJn5UOASeghWm2j/\nFwP3pJ0rtGxTgeo+tA81TwbW0TbaAJ+lLWO7z1JniZ1oH0KuoC3DT4yI82jLxMXAqRU9VlPz7GfA\n67v3njL5fUQcHBEHzljmINr6d/9oFypMam+g9Vi9I+pOjg6mtpvdun4T4DnAfwD3jPoT2IN2ePbH\nEXGvyZuZ+RPaOYR3Be6+3QONuCVtn/Im4BmZ+eTu/bJ9dWa+t/uwPtm2/ZDWI0Zm/m+000H+JiKu\nOkOZg4GLM/Nfuw9rpwJJO63lOrT950yH6Lseqkkwf3i3bTkPuGdm/hftA+rgDFXbZifgXNo5O8dF\nxPOBw4EnAbcH/h9th/Tgrtt2uX4A/Avwqy7rbuP+I9qOfKaTCbseJGCjE6BfB5xNO+R4J+DxwGMz\n8+fL7a2a2oB9PzPPy8yXdcP+KfCWyY51+VOycZ3M/DptQ30JcBzw1Mx8HXBMdxj1mMz82TJrrM7M\nyyJidUQ8rNshvZV22OyBEbGe9jiFR2Tm3884Pasy819oO9QbAbeg9YbuT1v29gd+ixZaljP8nQAy\n8/WZ+Wzgd4BDgUcD/xc4JSIOKejVuywzL1ry3nG09nljz+3/EGBDZn5/ennfXlOB6pRumDeh7Rje\n2/3+34GvdDVLZOYPaB9y3k47B+nttB6/29Ha/B7Ae2OGCzGWzLPX0rZrP+x25kQ7Mf5dzBgWM/NT\ntF6bawEPWRKsTqGdQlF14ccPmdpuduv6CZn5dlrYXkvxIbRs54m9krZ+HrkkWP0nbTn5/jIGvQp4\nVWZ+oGujyXah6rD8RvOh66n+O+AX3e/vRDsn9azMXNZ2prM37cMAtPMc35SZj6fta46nnaax7E6C\nbrt8eReoPgBcDbi0C+q/jHbaySujh/PRtiozfW3Di7bCrqctcCcCRwDfAl7T/X4f4KbLHPbqqe/X\n07qP7wTs0b13R+CbwC1mGP/V3ddVtIX6Kt3PewGfB+5dNJ+m6/w2sMuS6XsHrVdkVVGd1bSLBoIW\nOv4fcBpw56m/jYJp+RAteKymhewb0Xa4l9C6uOnGYbm1Vi35ejdaL9vk50OB1wDXLZhfxwO7TrX/\nAcCrgAuB2xa2//G0Heeqqd/32f770T4Jvxg4bMbhH0k7TH4AcD9ab/ENut+tmWXYm6m3Zur7/WmH\nTl4NHDg1nUcBtyuaZw/v5tnBwLO6deZxtN63o2eclp2mvr8d8FLgGcAh3Xt36rY5ayuWge776e3m\nNbr37ti9d/eiNroh8ITp9qKdsvEs2n7hqO6929Cdc7sNw5w8Ku5AWpC9IbCuevla0v6b2gd8BHhK\n1y73mR63Zda6Ja23LWgf1B9F225evVuu/3q57bJkOt5N2x/vRvtgdfupv9u9j/m41fGbR9FFei3Z\n2N0IeBrw593Cvy/tctRlb2SnFpA1wEO670+kfVr8IO18iguB+xXUmISDx3cL+GTj8zzgoQXzalN1\n1gBXW/J3tyquMwk7O9N2SM+h9R4te6Xawor7JrodW1frxt33s2yAlgbRnWmh6h3d8raedvj5qOL2\n333J3y0rsG1D+199wPa/0SztD6zvvj6SFtJ+C/gcVwaq1bOM+1amZzXwsO77G9MC4unAHQtrbOoD\nwv7dNuCHk+3McpdnNt6eHdd9fxfaaQB/CzyXdgHGfYqmZ+l2853d9L2Mdn7afYvaaA1wFu3Q7Aba\nhR1ru9/tQwvBL+i+foFt2F5zZaA6Bngf3YfoPl5b2AZM9gFvpl1QdK+ietegBfXr0ULms2nh8zXd\nPNqLFrT33c7hrpqajvfSto+70kLhUzc1f4d+DV5wkV6b2dgdTOsNeQVwm6LhTxb09VO/2492TP4Y\nut6D5SwkbDnV36H73e8Ddymclk1+eqBgh7SFOtNh5xBg/xlqbPOKO/U/lb1hk/oPpgWEPweO6KFd\nJu0/U6/RWNqfFsreTdfL1Q3nXQwTqDYXdiYB8RpFy9jm5tmNgIOKl+UndN+vpp2ofDitx+IOxfNs\nert5A9r5TEfShfflTs8m6h5G64F6Hq2X+nzaoeHrdr9/Dq13875bq8uVgeo+tBB2IO2w/wlV47sN\n7T+9DXgS7TSJWdp/Mk0H0Hqofo92zt6etB7EWwF/2P3NGcCfzDBN59JC2a7Ah4EnLx2Peb3mVnil\nv7ZhY3dyt2DsvpxG3MKC/hZm6OJfUmO7w4F1Nqq3pRW3zyDyFqYOw82wkRtVu/Rdp1u3nwM8farO\n5BB834FqxXxAKJ6WN1dtz7ZSp2y7uaReTH9P+0D94O7nE2mH/z/ZvX/zSVttyzpLC1Tn00LIsbTb\nMczUWzxr+8+wrZnudfsAXS8x7dYmp9F6XB9JC6SnA2dub03aIcVzuu/vAFyVdkHK7886/qXzfN4j\nsBJffW/shtrQTdXrNRxsY52yhb3P6Rlqxd2eZWDWeiukXVZs+9MOs016065P27ndfer35RvqMW0D\nhpqWOcyzg2jb/qcDu3XvPYF2O4rr0s6z+wPaqSAfpztnbBuHvaVz9sq2yQO1/6Z63Y4CHt29vz+t\n9/BJtAuJDt+e2lPDvzbthqV7dj8/knZR1UZ/N+/X3Edgpb2GXHG3sqDPuiMdKhyMos68Vty+loGx\ntEvfdWi9zyd26/cf066I/F3gMZPfV7b3drb/QnxA2MZpWag6tNM8LqAFqs8CZ0/97u20c6tOnHpv\np20c7mDn7A3Z/my6122LFyJsa23gqt3X3YG/Yepwbx/L18zzYt4jsFJffW3s+l7QGSgcjLDOYCtu\nn8vACNtlqDoH0a4W+wjwNdoNEa82yzDn0f5DzrMhpmXoOt1w9qJdGPLC7uddaL1Kk0B0FHDu1N+v\n2ZbaDHTOXt/tv/T/2MZet+XUowXPz3XzfNeuxrvpzv9bia+5j8BKevW54g61oWOgcDCmOkOtuEMs\nA2NqlyHqbGIHsTPtcSNn0n2QqpiOodp/qLYZcFoG70Gmnez+QtqtOSane5wC/Hb3/TVpVzA+cTuH\nO8g5ewOum4P0utEubjiDdtL7abSrMO82mYdV861s/s97BFbCa4gVd6AN3VDhYFR1ulq9r7h9LwNj\na5c+62xqPrPxp+oH0W4iWjk9Y9oGjCJUb6HuIbQjFS+lhatPMXUSOe0+hffcxmENds7egO0/6JWy\nwB7dvDsb+BLt3K1dqtu94jUJEzu0aM+6+3l3N/SzgY9m5ulL/iZymTMrIvajneR+Mu2ExuvQnu30\nlMz8xmxj/2u1HkXrZduVdt+RPYHXZubHiu/MO6o6Xa09aDenO4XWW3EZ7R5BvygY9iDLwNjapbpO\nRLyc9mzF923m95M7Nb+aFrJ+d8ZJmAx3NNuAAZflwebZVM012e4yTkTckHaO3WNoN/18V3dH8ssn\n+4Kt7Re6u/o/lXa7hM8C/0QLJFdk5pmT5a14GnpfN7unMzwT+GVmntrd2Xz3zLxklmna3PycemzU\nTrS7tZ8EvD4zPzPLdPRhhw9VY9vZdbV6CwdjqDOvFXfAILKQ7TJEnYg4gXa/nGdke07b9O9+tVxE\nxN0y82NL35/FmLYBixqqt1JrEqhX0+4/9QbalX0Po/WUvC0zP73MYR9E26acQjsisivtjuvLenzW\nNtTrpf2jPfv2I5l5XkRcn3az1adk5ke738+0rkTE3pl58aaGs2T9fAPwqaWdHyvCvLvKVsKLgY7Z\nUtyFyWa6jbkyLO9EC4mvYIYb7o2wzt6bqzf9Hm2j+muHG1bKMjDCdum9Du1O5Wey5ETaJe1+PPAi\nWo9C6Tkbi7oNGGJa5lmHnu7Sv7R9KD5nb8j2p8crZbv5vjftmaqbvd9YN/+uRrsy/8bVy1rFywcq\nA5l5Nm1heSbtIY+3AV4SEbvkMj4JRURs5lc/yczv0u6aexTtKolbLW+sgXaVyq/Vy8zsUv3/ZntI\n6560adqh60TEqojYG7ggIm6X3Vq6iVo7R3vS/TVoG9jtNtAyMIp26btORDw/Is6OiEOAb9Ce2fnq\nbthXLPkEfBztcM05mXnpctb/TU3DlIXbBgw1LQPOs6V1px/OO3nMzTm0B0LfKjMvpN1cdENm/nQb\nhxmw0YPrJw9GvjQzLwDeT+ux2uhvlmGodZPMvDwzX0R7xM17gEfQ7tP17Ii4Wi7/kN+qzLwiMy+m\nHSW6fbQH2K9a8neru/n3M+AFmflPs0xPX3a4UDWGnd1Q4WBMdeaw4va2DIypXQaq83Lg32jPUHw9\n7S7fF0fEbSfD7sbjeFqgenhmfnV7p2OJUWwDOgsdqrek2y5MAtW7aaeAnE4LV5/LzM924/D1zPzW\nNgzv5RFx1Gba44pohxahPbLnN2YZ7wHbf1JzEhS/kZmfou0nj6f1XD1m+m+20wFT338RuAet1+uK\nyfCmgu+eEfEx2p3sV6ZcAd1lQ77o8dAPA3RhsvEVSk8Fnkzrll215O+mn+B+2DLm09jqHDT1/T1p\nDyveebrdufIQwJ7Ax4CbLGd6+lwGRtguvdWhXbn1DODk7uc9aI/J+DDtHJOnT/3tzWm9Bzfd3mkY\nsv2Hbpu+p2XIOlsZh5L7EtI+lL+S7kHFS343vX+526beX0ntv6Xxo+BK2a7dr0K7aehZwNHd+6cB\nr1o6Hd0PPGYAAAAId0lEQVT6+0HgrpVtX/3aYXqq+k72A/aEDJXqR1Gna/erAH8VEWdFxNGZ+RHa\nE+xfDr9q90mNPWgPGj05M7+2PRMy0DIwinbpu05E/AXt0VLvAZ4ZEcdk5iWZeTLtAeIPB14y+fvM\n/DLtPkTL7qEa0zZgqGmZQw/y9DBvGRHndD+eDryMtry8OzNf1v1N5PYdAv44Lfzt0f3/qqnhTPeI\n3qfb16za1L5oKwZZN/vqdZvqzdorM39Je9jyJ4GjIuITtFtO7B0R+3S1Lo92Zf7baG3/8e2dlkHN\nO9UN8WKAZE/PPSEMlOrHUmdqnl+r+7oL7YKE04FP0HaqfwPsM/U/u9Nu9nqXZS5nvS0DY2mXIerQ\nrnra0H1/DvAn3ffX3sTfrqHuPmSj2AYMMS1D11lSs/S+hMDzaSfRHwIE7arh929qOLRn3/09y+gR\nHbL9u//vrdeN9libz9BOoJ9+uPJ62vlrV9Cujpy8/zt09/pa6a+5j8AgE7nAOzsGCgdjq9P93yAr\nbp/LwNjape86tKuRbk87z+NTdIf+ut+9ELjz9rbvPNt/DuvMwofqbahdekNR2qHLU2nbmXNpDxR+\nHXDbJX93fDe92xUKh2z/JXV7uVKWdj7cBbRHQz0Z+CpTwQ241qTmIr7mPgK9TtwIdnbd/w0VDkZT\nZ4gVd6hlYEzt0mcdWk/DG7vvTwD+G9iv+/nFtHW78nEgo9kGDLgszyUgTA2r7I7jDHjO3oDrZi+9\nblPtflXgFsD9gWNoN0SdrKO/dssHtiPcrpTX3Eegl4ka0c6OgVL9GOrMY8XtexkYQ7sMUQd4Nm0D\nf/TUe08DvkK76OTtwJru/cp7z41pG7DQoXo76s98X0LgL7ph3Bz4BXDM1O9uRrt56Jol/7P3Mse3\n923m9DjSU68b7cHLLwCOpt3I9ctcecToLrRzWfdZznSspNfcR6C3CVvgnR0DhYOx1en+Z7AVt69l\nYGzt0ncdWm/DobSH3J6y5Hf70S7Vnxy6qOypWvhtwBDTMo862zAey76hKAOcszdE+7Nxr1OvvW60\nG57+Gd3hd+CJdFd7Ag8F/gG4b9/tPsRr7iPQy0Qt+M6u+59BwsGY6gyx4g61DIypXfqsA7yGdgXU\nn9Ku3LqYqXNk2PgilZl7qMa0DRhwWZ7boZ/NDWNqnLbrjuMMeM7egOtmL71uU/M4aPvj82gfgCbv\nr6fdYPW1wL2r2nzer7mPQNmEjGhnx0Cpfgx15rHi9r0MjKFdhqhD+2T90u77w4Dfo32yvpBl3Ddn\npbT/wG2z0KF6G+qW3ZeQAc/Z67P9l0x3r71utNslHEU75PrPwBOXjsv2DnOlv+Y+AqUTs8A7OwYK\nB2Or0/3fYCtuX8vA2Nql7zq0E2l/SnfBCe28yVvTzpG5F/CXy12e5tH+Q68zfU/LPOosqVl6Q1EG\nOGdviPaf/nt66nWbGt/DaIfkz6Wdp/VpuicbVLb1SnvNfQTKJmTBd3bd/w0SDsZQZ8gVd6hlYAzt\nMmQdWm/0D4AHTb33aeAWS9tuEdp/oHk2ilC9ldql9yVkwHP2Blw3H0mPvW7d/Poo3XnK3Tx8Am07\n/QPgeRVtvRJfcx+BGReMhd/ZMVA4GFudrsZgK25fy8DY2mXI9u/qHEP78PQS2uHADwE7Vdbos/3n\nNM9GEaq3ULfsvoQMcM7eEO0PG/VQDdHrdgRwOfCs7uedgGOBU2hB9E7V7b5SXnMfgYKFZaF3dl2N\nQcLBCOv0uuIOtQyMsF0G/ZRK67H6LvCGqffWFAx3NNuAAZfluR36ofi+hAx4zl6f7c/GgWrIXrdj\ngW8Cx3U/35XW+XGNpeM1ptfcR2CWhWSADcQowsFY63TD7nXFHWIZGFu7DNn+UzUPp10af1zxcEez\nDRhwWoYO1ZP9Qdl9CRn4nL2+2p+NA9WgV8p2w7lvNx//GngzcL/Ktl+Jr7mPwAyNNZqdXTfsQVL9\n2Op0w+ptxR1qGRhbuwzZ/lM1Hwj8UfEwR7MNGHBZnkeoLr8vIQOdszdE+zOnK2W7evejhbmnTaaj\nj/V/pbzmPgIzNNSodnbdsAZJ9WOr09XqbcUdahkYW7sM2f59vsa0DRhwWR5ynvV5E9ZBztnrs/2Z\n05WyS8bhSOB7wAP6rLMSXnMfgRkbalQ7u67WIKl+bHW6Yfe24g4YREbVLkO2f5+vMW0DBlyW++xB\nHuS+hN3/9HLO3pDtz8C9bpsZhyOA/fussRJecx+BgoYa1c6uG/YgqX5sdbpava24AwaRUbXLkO3f\n83SMZhsw4LLcZw/ykI+k6uWcvSHbn4F73XbU19xHoGQiRraz62oNkurHVmeA6RgqiIyqXWz/lTfP\nFjlUM58bipafszd0+zNwr9uO+Jp0oS68iDiSdhntkzLzrT3WOQK4MDO/1VcNrWwuAzu2MbX/UNNS\nUSciIjMzIoL2AORLgeO74WZErKedhH0Z8FeZ+f7J/1RMw1hExOG03qoNmfmGeY/P2IwmVMG4NnaS\npI1FxJ1oz6vbF3gW8PLM/LOp30+OUlwxp1FcCBHxQODWmXnSvMdlbEYVqiRJ4zLVQ3UYcCbtYqTv\nAXem3UrnlMw8fZ7jKE2smfcISJK0OV2gOhT4I+BRmXleRBwAfId2uO9ZEbF3Zp481xGVaLf1lyRp\nJbsG7Yq+e3Q/f5vWW3UhcEfalWzS3BmqJEkrWmZ+CHgA8OiIOC4z/xe4hHabgP/IzL/rzqeS5srD\nf5KkFS8z3xERVwDndidaXwE8NzN/0v3eE4Q1d/ZUSZIWQma+i3YbhQOAz2fmO6Mz51GTAHuqJEkL\npAtSvwDOiogL+7wvobS9vKWCJGnheF9CrUSGKkmSpAKeUyVJklTAUCVJklTAUCVJklTAUCVJklTA\nUCVJklTAUCVJklTg/wOwgulZP3OgRgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "memm_1.plot_lr_weights('V',feat_filter=lambda s: s.startswith(\"prev_\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Beam Search\n", "As we discussed, the greedy search approach is an approximate solution to the $\\argmax$ problem. One way to improve over greedy search is to maintain a *beam* of $k$-best solutions in each step. This enables initially weaker solutions to remain in the beam and move up the ranks in later steps in case they are more consistent with future observations. \n", "\n", "Technically a $k$-best beam search proceeds as follows. Let $L$ be the label set.\n", "\n", "1. Initialise a beam $B \\leftarrow \\left[(\\text{PAD}, 0) \\right]$ of partial solutions $(\\y,s)$ where $s$ is the partial log-score of $\\y$. \n", "1. **for** $i$ in $1 \\ldots |\\x|$:\n", " 1. Let $C\\leftarrow \\{\\}$ be the next beam candidates.\n", " 1. **for** $\\y, s$ in $B$ and $y$ in $L$: \n", " 1. $C \\leftarrow C \\cup \\{ (\\y \\| y, s + \\log \\prob_\\params(y|\\x,y_{i-1},i)) \\} $\n", " 1. Let $B\\leftarrow k\\text{-highest-scoring}(C)$ be the $k$ pairs $(\\y, s)$ with highest scores.\n", "1. **Return** $\\y$ with highest score in $B$. \n", "\n", "Note that a slightly faster version can use a priority queue. \n", "\n", "In Python we can implement this algorithm like so:" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8133941530167945" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def memm_beam_search(memm, x, width=2):\n", " beam = [([],0.)]\n", " history = [beam]\n", " for i in range(0, len(x)):\n", " # use priority queue \n", " candidates = []\n", " for (prev,score) in beam:\n", " scores = memm.predict_scores(x, i, prev)\n", " for label_index,label_score in enumerate(scores):\n", " candidates.append((prev + [memm.labels()[label_index]], score + label_score))\n", " beam = sorted(candidates, key=lambda x: -x[1])[:width]\n", " history.append(beam)\n", " return beam, history\n", " \n", "def batch_predict(data, beam_predictor):\n", " return [beam_predictor(x)[0][0][0] for x,y in data]\n", "\n", "seq.accuracy(dev, batch_predict(dev, lambda x: memm_beam_search(memm_1, x, 10)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With beam of size 10 accuracy improves only marginally. You can try other beam sizes (leading to longer runtimes) but likely will not see substantial improvements. Is this because we already finding solutions with highest probability, or because higher probability doesn't necessarily mean higher accuracy? \n", "\n", "We can test how many per-token predictions differ when comparing greedy search to a beam search of a given width, simply calculating their accuracies relative to each other:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.9691063653327804" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seq.accuracy(memm_greedy_predict(memm_1, dev), batch_predict(dev, lambda x: memm_beam_search(memm_1, x, 10)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We notice that about 4% of the tokens receive different labels, simply searching for higher scoring sequences. This suggest that we frequently find higher probability sequences, but that these are not necessarily more correct. We can also calculate the average log probability of the argmax sequence using different beam sizes. Again we see that there is a substantial difference between scores, they are just not reflected in task accuracy. " ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "-3.62528740936034" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sum([memm_beam_search(memm_1, x, 1)[0][0][1] for x,y in dev]) / len(dev)" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "-3.5424313113979289" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sum([memm_beam_search(memm_1, x, 5)[0][0][1] for x,y in dev]) / len(dev)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Beam search is a simple and often effective way to find sequences (or other structures) with higher probability. However, often it is also inefficient in the sense that it does not fully leverage the factorisation structure and conditional independences. To illustrate this problem recall that the conditional probability of a label $y_i$ only depends on the previous label $y_{i-1}$, any earlier labels have no impact on the term $\\prob(y_i|\\x,y_{i-1},i)$. With this in mind let us follow the beam for an example instance. " ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Previous\n", "  \n", " Next\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
0.00
\n", " 1 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
A-0.17
^-3.43
!-3.63
\n", " 2 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AN-0.85
AA-1.06
A^-3.36
\n", " 3 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AAN-1.32
ANN-1.67
ANV-2.13
\n", " 4 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANP-1.33
ANNP-1.68
ANVP-2.14
\n", " 5 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN-1.70
ANNPN-2.05
ANVPN-2.51
\n", " 6 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,-1.71
ANNPN,-2.06
ANVPN,-2.52
\n", " 7 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,O-1.88
ANNPN,O-2.23
ANVPN,O-2.69
\n", " 8 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OR-2.46
ANNPN,OR-2.81
AANPN,OA-2.93
\n", " 9 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OAN-2.96
AANPN,ORN-3.02
ANNPN,ORN-3.37
\n", " 10 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANP-2.97
AANPN,ORNP-3.02
ANNPN,ORNP-3.37
\n", " 11 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPN-3.56
AANPN,ORNPN-3.61
ANNPN,ORNPN-3.96
\n", " 12 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNP-3.61
AANPN,ORNPNP-3.66
ANNPN,ORNPNP-4.01
\n", " 13 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNPV-3.69
AANPN,ORNPNPV-3.75
ANNPN,ORNPNPV-4.10
\n", " 14 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNPVP-3.87
AANPN,ORNPNPVP-3.93
ANNPN,ORNPNPVP-4.28
\n", " 15 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNPVPP-3.88
AANPN,ORNPNPVPP-3.93
ANNPN,ORNPNPVPP-4.28
\n", " 16 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNPVPPL-4.72
AANPN,ORNPNPVPPL-4.78
AANPN,OANPNPVPPZ-5.02
\n", " 17 / 17
\n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "example = 56\n", "beam, history = memm_beam_search(memm_1, dev[example][0],3)\n", "seq.render_beam_history(history, dev[example], end=17)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice the search *frontier*, the most recent label in each of the hypotheses, often has very little diversity. Sometimes this makes sense: for the word \"of\" it is very certain that the label \"P\" for preposition should be assigned, and the frontier reflects that. However, for the adjective \"better\" of \"better way\" the frontier contains the label \"R\" (adverb) twice, and the gold label \"A\" (adjective) not at all. This leads to an error in this case. We can fix this error by simply increasing the beam size to 4. You can test this above. In this case \"A\" barely makes it into the beam, and becomes the winning label in the next step as it fits better to the noun \"way\". \n", "\n", "One can generally avoid search errors by increasing the width, but for many models this is sub-optimal because it ignores the factorization or dependency structure of the model. In this particular case labels only depend on the previous label. This means that it makes no sense to maintain more than one hypothesis with the same frontier label in the beam. One only needs to remember the highest scoring sequence with that frontier label. To prove this consider two partial sequences $\\y$ and $\\y'$ of length $l$ with the same last label $t=y_l=y'_l$. Assume that the log probability $s = \\sum_{i}^0 \\log \\prob(y_i|\\x,y_{i-1},i)$ of $\\y$ is larger than the log probability $s' = \\sum_{i}^0 \\log \\prob(y'_i|\\x,y'_{i-1},i)$ of $\\y'$. Further assume that the label $y_{l+1}$ maximises $\\prob(y_{l+1}|\\x,t,i+1)$. Then the log probability of $\\y \\| y_{l+1}$ is larger than the log probability of $\\y' \\| y_{l+1}$ and hence there is no need to carry around $\\y'$. \n", "\n", "### Viterbi \n", "\n", "The Viterbi (link/cite) algorithm leverages conditional independences of the model directly. It does so by maintaining a map $\\alpha_i(l)$ from label $l$ and token index $i$ to the score $\\log \\prob(\\y|\\x)$ of highest scoring sequence $\\y$ ending in label $l$ at token $i$. For each pair $(l,i)$ we also remember the sequence $\\y$ that yielded that score in a map $\\beta_i(l)$. The algorithm initialises $\\alpha_{1}(l) =\\log \\prob(l|\\x,\\text{PAD},1)$ and then updates the $\\alpha$ map via the following recursion:\n", "\n", "$$\n", "\\alpha_i(l) = \\max_y \\alpha_{i-1}(y) + \\log \\prob(l|\\x,y,i) \n", "$$\n", "\n", "and in $\\beta_i(l)$ we store the 'winning' $y$ from the $\\max$ term. Once we reached the sequence end the result sequence can be inferred by finding the label $l$ with highest $\\alpha_{|\\x|}(l)$ and then back-tracking using $\\beta$. It is easy to show that this algorithm returns the *optimal* solution to the prediction/search problem, assuming that labels only depend on the previous label. (Exercise: extend to $n$ previous labels) \n", "\n", "Below we implement a beam version of the viterbi algorithm. In this version we restrict the maximisation that defines $\\alpha_i(l)$ to only range over the top $k$ highest scoring previous labels $y$. " ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Previous\n", "  \n", " Next\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
0.00
\n", " 1 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
A-0.17
^-3.43
!-3.63
\n", " 2 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AN-0.85
AA-1.06
A^-3.36
\n", " 3 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AAN-1.32
ANV-2.13
ANP-2.66
\n", " 4 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANP-1.33
AANN-7.63
AANV-8.02
\n", " 5 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN-1.70
AANP^-2.75
AANPA-5.33
\n", " 6 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,-1.71
AANP^^-5.37
AANPNP-8.60
\n", " 7 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,O-1.88
AANPN,D-4.38
AANPN,#-5.34
\n", " 8 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OR-2.46
AANPN,OA-2.93
AANPN,OV-4.55
\n", " 9 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OAN-2.96
AANPN,ORR-3.99
AANPN,ORV-4.34
\n", " 10 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANP-2.97
AANPN,ORVT-8.52
AANPN,OANN-9.52
\n", " 11 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPN-3.56
AANPN,OANPV-4.25
AANPN,OANPA-5.25
\n", " 12 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNP-3.61
AANPN,OANPNV-7.12
AANPN,OANPNU-8.82
\n", " 13 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNPV-3.69
AANPN,OANPNPN-6.35
AANPN,OANPNPA-8.09
\n", " 14 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNPVP-3.87
AANPN,OANPNPVT-5.60
AANPN,OANPNPV&-9.22
\n", " 15 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNPVPP-3.88
AANPN,OANPNPVP^-10.16
AANPN,OANPNPVP$-10.55
\n", " 16 / 17
\n", "
\n", " \n", " \n", " \n", " \n", "
HappyInternationalYearofBiodiversity!WhatbetterwaytocelebratethantuningintoCropLife'sBiodiversity
AANPN,OANPVPVTPZ^
AANPN,OANPNPVPPL-4.72
AANPN,OANPNPVPPZ-5.02
AANPN,OANPNPVPPS-6.22
\n", " 17 / 17
\n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from collections import defaultdict\n", "import math\n", "def memm_viterbi_search(memm, x, width=2):\n", " labels = memm.labels()\n", " # initialise\n", " alpha = [{}]\n", " beta = [{}]\n", " for label_index, label_score in enumerate(memm.predict_scores_hist(x, 0, [\"PAD\"])):\n", " label = labels[label_index]\n", " alpha[0][label] = label_score\n", " beta[0][label] = \"PAD\"\n", " \n", " # prune\n", " seq.prune_alpha_beta(alpha[0], beta[0], width)\n", " \n", " # recursion \n", " for i in range(1, len(x)):\n", " alpha.append(defaultdict(lambda: -math.inf))\n", " beta.append({})\n", " for p in alpha[i-1].keys():\n", " for label_index, label_score in enumerate(memm.predict_scores_hist(x, i, [p])):\n", " label = labels[label_index]\n", " new_score = alpha[i-1][p] + label_score\n", " if new_score > alpha[i][label]:\n", " alpha[i][label] = new_score\n", " beta[i][label] = p\n", " # prune\n", " seq.prune_alpha_beta(alpha[i], beta[i], width)\n", " \n", " # convert to beam history to be used in the same way beam search was used. \n", " history = seq.convert_alpha_beta_to_history(x, alpha, beta)\n", " return history[-1], history\n", "\n", "beam, history = memm_viterbi_search(memm_1, dev[example][0],3)\n", "seq.render_beam_history(history, dev[example], 17)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Crucially, for the same beam size we now keep the correct labelling of \"better\" in the beam and reach a better solution, both in terms of log probability and actual accuracy. \n", "\n", "This improvement in log probabilities does not always lead to higher global accuracy:" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8131868131868132" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "seq.accuracy(dev, batch_predict(dev, lambda x: memm_viterbi_search(memm_1, x, 10)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Label Bias Problem\n", "\n", "MEMMs multiply several *locally* normalised transition probabilities to arrive at a sequence probability. That is, for each token $i$ and a given previous state $y_{i-1}$ the sum of transition scores into next states $\\sum_{y_i} \\prob_\\params(y_i|\\x,y_{i-1},i)$ equals 1. This local normalisation makes training easy (why?), but it also leads to a problem. Consider two simple sequences \"that works\" and \"that house\". The former is a pronoun (\"O\") followed by a verb (\"V\"), the latter is a determiner (\"D\") followed by a noun (\"N\"). Let us assume that \n", "\n", "$$\n", "\\prob_\\params(\\text{D}|\\x,\\text{PAD},0) =\\prob_\\params(\\text{O}|\\x,\\text{PAD},0) \\approx 0.5,\n", "$$\n", "\n", "meaning that at the beginning of a sentence both the determiner and pronoun label for \"that\" have roughly the same probability 0.5. Now assume that in the training set determiners are always followed by nouns, and pronouns always by verbs. This would mean that \n", "\n", "$$\n", "\\prob_\\params(\\text{N}|\\x,\\text{D},i) = \\prob_\\params(\\text{V}|\\x,\\text{O},i) \\approx 1\n", "$$ \n", "\n", "and hence transitions from these two states are completely independent of the observation. \n", "\n", "Now we have $\\prob_\\params(\\text{D N}|\\, \\text{that works}) \\approx 0.5$ and $\\prob_\\params(\\text{O V}|\\, \\text{that works}) \\approx 0.5$, and the same for the input \"that house\". This means that once we enter the \"D\" or \"O\" state, the following observations have no effect on the sequence probability. The reason is that MEMMs requires *all* incoming probability mass (0.5 in the above example) to a given state (such as \"D\" or \"O\") to be distributed among the outgoing states. If there is only one possible next state, then that next state will receive all the mass, regardless of the observation. In particular, the model cannot say \"in state \"D\" and for observation \"works\", *all* labels are impossible. More generally, states with few outgoing distributions effectively ignore observations, and this creates a bias towards such states. This problem is known as the *label bias problem*. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## CRFs\n", "\n", "[Conditional Random Fields (CRFs)](http://www.seas.upenn.edu/~strctlrn/bib/PDF/crf.pdf) have been developed to overcome the label bias. The core problem of MEMMs is local normalisation. CRFs replace this with *global* normalisation. That is instead of normalising across all possible next states $y_{i+1}$ given a current state $y_i$ and observation $\\x$, the CRF normalises across all possible *sequences* $\\y$ given observation $\\x$. Formally the CRF is hence defined as follows:\n", "\n", "$$\n", "p_\\params(\\y|\\x) = \\frac{1}{Z_{\\x}} \\prod_i^{|\\x|} \\exp \\langle \\repr(\\x,y_{i-1},i), \\params_{y_i} \\rangle\n", "$$\n", "\n", "where $Z_{\\x}=\\sum_\\y \\prod_i^{|\\x|} \\exp \\langle \\repr(\\x,y_{i-1},i), \\params_{y_i} \\rangle$ is the *partition function*, a *global* normalisation constant depending on $\\x$. Notably each term $\\exp \\langle \\repr(\\x,y_{i-1},i), \\params_{y_i} \\rangle$ in the product can now take on values in $[0,\\infty)$ as opposed to the MEMM terms in $[0,1]$. \n", "\n", "The name CRF stems from the fact they correspond to [Markov random fields](http://www.statslab.cam.ac.uk/~grg/books/hammfest/3-pdc.ps), globally conditioned on the observation $\\x$. While in this chapter we focus on cases where the dependency structure corresponds to a linear chain, CRFs are more general and encompass any graphical structure. \n", "\n", "### Training Linear Chain CRFs\n", "\n", "CRFs can be trained (as usual) by maximising the conditional log-likelihood of the data \n", "\n", "$$\n", "CL(\\params) = \\sum_{(\\x,\\y) \\in \\train} \\log \\prob_\\params(\\y|\\x).\n", "$$\n", "\n", "This is substantially harder than for MEMMs because the partition function makes it impossible to break up the objective into only per-token logistic regression terms. Instead the objective needs to be treated on a per-sequence basis. Conceptually this is not difficult: just as for logistic regression we need to calculate the gradient of the objective, and once we have this gradient, we choose a gradient descent/ascent method to optimise the function. The general CRF conditional log-likelihood is in fact a generalisation of the logistic regression objective, and hence the CRF gradient will look very similar to the gradient of logistic regression. In this chapter we will only briefly discuss the gradient and what is necessary to calculate it. \n", "\n", "\\begin{split}\n", " \\nabla_{y'} CL(\\params) = \\sum_{(\\x,\\y) \\in \\train} \\sum^{|\\x|}_i\\repr(\\x,y_{i-1},i) \\delta(y_i,y') - p_\\params(y',y_{i-1}|\\x) \\repr(\\x,y_{i-1},i) \n", "\\end{split}\n", "\n", "\n", "### Prediction in Linear Chain CRFs\n", "\n", "From the perspective of finding $\\argmax_\\y \\prob_\\params(\\y|\\x)$ we can treat the CRF just as the MEMM. They share the same factorisation/dependency structure and are just differently normalised. That is, we can again simply perform greedy search, use a beam or apply Viterbi. \n", "\n", "Below we train a CRF model using the same feature vectors as used for the MEMM model, and then do prediction via the Viterbi algorithm (this is the standard algorithm in most CRF libraries). " ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8216877462160481" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "crf_1 = seq.CRFSequenceLabeler(feat_4, train)\n", "seq.accuracy(dev, crf_1.predict(dev))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A notable 1% point improvement over the MEMM model that essentially comes from free in the sense that we are using exactly the same feature representation and are just changing from local to global normalisation." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Data\n", "Let us run the above models on the test data to see whether our findings generalise.\n" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
01
0word0.651147
1+ first @0.696029
2+ cap0.740492
3+ suffix0.790968
4MEMM0.812640
5CRF0.822707
\n", "
" ], "text/plain": [ " 0 1\n", "0 word 0.651147\n", "1 + first @ 0.696029\n", "2 + cap 0.740492\n", "3 + suffix 0.790968\n", "4 MEMM 0.812640\n", "5 CRF 0.822707" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "pd.DataFrame([\n", " [\"word\", seq.accuracy(test, local_1.predict(test))],\n", " [\"+ first @\", seq.accuracy(test, local_2.predict(test))],\n", " [\"+ cap\", seq.accuracy(test, local_3.predict(test))],\n", " [\"+ suffix\", seq.accuracy(test, local_4.predict(test))],\n", " [\"MEMM\", seq.accuracy(test, memm_1.predict(test))],\n", " [\"CRF\", seq.accuracy(test, crf_1.predict(test))] \n", " ])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Background Material\n", "* [Tackling the Poor Assumptions of Naive Bayes Text Classifiers](https://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf), Rennie et al, ICML 2003 " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.2" } }, "nbformat": 4, "nbformat_minor": 1 }