{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Supervised sentiment: Dense feature representations and neural networks" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "__author__ = \"Christopher Potts\"\n", "__version__ = \"CS224u, Stanford, Spring 2019\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Contents\n", "\n", "1. [Overview](#Overview)\n", "1. [Set-up](#Set-up)\n", "1. [Distributed representations as features](#Distributed-representations-as-features)\n", " 1. [GloVe inputs](#GloVe-inputs)\n", " 1. [IMDB representations](#IMDB-representations)\n", " 1. [Remarks on this approach](#Remarks-on-this-approach)\n", "1. [RNN classifiers](#RNN-classifiers)\n", " 1. [RNN dataset preparation](#RNN-dataset-preparation)\n", " 1. [Vocabulary for the embedding](#Vocabulary-for-the-embedding)\n", " 1. [Pure NumPy RNN implementation](#Pure-NumPy-RNN-implementation)\n", " 1. [PyTorch implementation](#PyTorch-implementation)\n", " 1. [TensorFlow implementation](#TensorFlow-implementation)\n", " 1. [Pretrained embeddings](#Pretrained-embeddings)\n", "1. [Tree-structured neural networks](#Tree-structured-neural-networks)\n", " 1. [TreeNN dataset preparation](#TreeNN-dataset-preparation)\n", " 1. [Pure NumPy TreeNN implementation](#Pure-NumPy-TreeNN-implementation)\n", " 1. [Torch TreeNN implementation](#Torch-TreeNN-implementation)\n", " 1. [Subtree supervision](#Subtree-supervision)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Overview\n", "\n", "This notebook defines and explores __recurrent neural network (RNN) classifiers__ and __tree-structured neural network (TreeNN) classifiers__ for the Stanford Sentiment Treebank. \n", "\n", "These approaches make their predictions based on comprehensive representations of the examples: \n", "\n", "* For the RNN, each word is modeled, as are its sequential relationships to the other words.\n", "* For the TreeNN, the entire parsed structure of the sentence is modeled.\n", "\n", "Both models contrast with the ones explored in [the previous notebook](sst_02_hand_built_features.ipynb), which make predictions based on more partial, potentially idiosyncratic information extracted from the examples." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Set-up\n", "\n", "See [the first notebook in this unit](sst_01_overview.ipynb#Set-up) for set-up instructions." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from collections import Counter\n", "import numpy as np\n", "import os\n", "import pandas as pd\n", "import random\n", "from np_rnn_classifier import RNNClassifier\n", "from np_tree_nn import TreeNN\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.metrics import classification_report\n", "import tensorflow as tf\n", "from tf_rnn_classifier import TfRNNClassifier\n", "import torch\n", "import torch.nn as nn\n", "from torch_rnn_classifier import TorchRNNClassifier\n", "from torch_tree_nn import TorchTreeNN\n", "from torch_subtree_nn import TorchSubtreeNN\n", "import sst\n", "import vsm\n", "import utils" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# This will limit the TensorFlow log messages to just those\n", "# that track traing progress.\n", "\n", "utils.tf_train_progress_logging()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "DATE_HOME = 'data'\n", "\n", "GLOVE_HOME = os.path.join(DATE_HOME, 'glove.6B')\n", "\n", "VSMDATA_HOME = os.path.join(DATE_HOME, 'vsmdata')\n", "\n", "SST_HOME = os.path.join(DATE_HOME, 'trees')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Distributed representations as features\n", "\n", "As a first step in the direction of neural networks for sentiment, we can connect with our previous unit on distributed representations. Arguably, more than any specific model architecture, this is the major innovation of deep learning: __rather than designing feature functions by hand, we use dense, distributed representations, often derived from unsupervised models__.\n", "\n", "\"distreps-as-features.png\"\n", "\n", "Our model will just be `LogisticRegression`, and we'll continue with the experiment framework from the previous notebook. Here is `fit_maxent_classifier` again:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def fit_maxent_classifier(X, y): \n", " mod = LogisticRegression(\n", " fit_intercept=True, \n", " solver='liblinear', \n", " multi_class='auto')\n", " mod.fit(X, y)\n", " return mod" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### GloVe inputs\n", "\n", "To illustrate this process, we'll use the general purpose GloVe representations released by the GloVe team, at 50d:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "glove_lookup = utils.glove2dict(\n", " os.path.join(GLOVE_HOME, 'glove.6B.300d.txt'))" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "def vsm_leaves_phi(tree, lookup, np_func=np.sum):\n", " \"\"\"Represent `tree` as a combination of the vector of its words.\n", " \n", " Parameters\n", " ----------\n", " tree : nltk.Tree \n", " lookup : dict\n", " From words to vectors.\n", " np_func : function (default: np.sum)\n", " A numpy matrix operation that can be applied columnwise, \n", " like `np.mean`, `np.sum`, or `np.prod`. The requirement is that \n", " the function take `axis=0` as one of its arguments (to ensure\n", " columnwise combination) and that it return a vector of a \n", " fixed length, no matter what the size of the tree is.\n", " \n", " Returns\n", " -------\n", " np.array, dimension `X.shape[1]`\n", " \n", " \"\"\" \n", " allvecs = np.array([lookup[w] for w in tree.leaves() if w in lookup]) \n", " if len(allvecs) == 0:\n", " dim = len(next(iter(lookup.values())))\n", " feats = np.zeros(dim)\n", " else: \n", " feats = np_func(allvecs, axis=0) \n", " return feats" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "def glove_leaves_phi(tree, np_func=np.sum):\n", " return vsm_leaves_phi(tree, glove_lookup, np_func=np_func)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " negative 0.612 0.715 0.660 991\n", " neutral 0.322 0.076 0.122 490\n", " positive 0.659 0.785 0.716 1083\n", "\n", " micro avg 0.622 0.622 0.622 2564\n", " macro avg 0.531 0.525 0.499 2564\n", "weighted avg 0.576 0.622 0.581 2564\n", "\n" ] } ], "source": [ "_ = sst.experiment(\n", " SST_HOME,\n", " glove_leaves_phi,\n", " fit_maxent_classifier,\n", " class_func=sst.ternary_class_func,\n", " vectorize=False) # Tell `experiment` that we already have our feature vectors." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### IMDB representations\n", "\n", "Our IMDB VSMs seems pretty well-attuned to the Stanford Sentiment Treebank, so we might think that they can do even better than the general-purpose GloVe inputs. Here are two quick assessments of that idea:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "imdb20 = pd.read_csv(\n", " os.path.join(VSMDATA_HOME, 'imdb_window20-flat.csv.gz'), index_col=0)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "imdb20_ppmi = vsm.pmi(imdb20, positive=False) " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "imdb20_ppmi_svd = vsm.lsa(imdb20_ppmi, k=50) " ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "imdb_lookup = dict(zip(imdb20_ppmi_svd.index, imdb20_ppmi_svd.values))" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "def imdb_phi(tree, np_func=np.sum):\n", " return vsm_leaves_phi(tree, imdb_lookup, np_func=np_func)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " negative 0.591 0.716 0.648 1018\n", " neutral 0.214 0.006 0.012 495\n", " positive 0.617 0.774 0.687 1051\n", "\n", " micro avg 0.603 0.603 0.603 2564\n", " macro avg 0.474 0.499 0.449 2564\n", "weighted avg 0.529 0.603 0.541 2564\n", "\n" ] } ], "source": [ "_ = sst.experiment(\n", " SST_HOME,\n", " imdb_phi,\n", " fit_maxent_classifier,\n", " class_func=sst.ternary_class_func,\n", " vectorize=False) # Tell `experiment` that we already have our feature vectors." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Remarks on this approach\n", "\n", "* Recall that our `unigrams_phi` created feature representations with over 16K dimensions and got about 0.77.\n", "\n", "* The above models have only 50 dimensions and come close in terms of performance. In many ways, it's striking that we can get a model that is competitive with so few dimensions.\n", "\n", "* The promise of the Mittens model of [Dingwall and Potts 2018](https://arxiv.org/abs/1803.09901) is that we can use GloVe itself to update the general purpose information in the 'glove.6B' vectors with specialized information from one of these IMDB count matrices. That might be worth trying; the `mittens` package already implements this!\n", "\n", "* That said, just summing up all the word representations is pretty unappealing linguistically. There's no doubt that we're losing a lot of valuable information in doing this. The models we turn to now can be seen as addressing this shortcoming while retaining the insight that our distributed representations are valuable for this task." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## RNN classifiers\n", "\n", "A recurrent neural network (RNN) is any deep learning model that process its inputs sequentially. There are many variations on this theme. The one that we use here is an __RNN classifier__.\n", "\n", "\n", "\n", "For a sequence of length $n$:\n", "\n", "$$\\begin{align*}\n", "h_{t} &= \\tanh(x_{t}W_{xh} + h_{t-1}W_{hh}) \\\\\n", "y &= \\textbf{softmax}(h_{n}W_{hy} + b)\n", "\\end{align*}$$\n", "\n", "where $1 \\leqslant t \\leqslant n$. As indicated in the above diagram, the sequence of hidden states is padded with an initial state $h_{0}$ In our implementations, this is always an all $0$ vector, but it can be initialized in more sophisticated ways (some of which we will explore in our unit on natural language inference).\n", "\n", "This is a potential gain over our sum-the-word-vectors baseline, in that it processes each word independently, and in the context of those that came before it. Thus, not only is this sensitive to word order, but the hidden representation give us the potential to encode how the preceding context for a word affects its interpretation.\n", "\n", "The downside of this, of course, is that this model is much more difficult to set up and optimize. Let's dive into those details." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### RNN dataset preparation\n", "\n", "SST contains trees, but the RNN processes just the sequence of leaf nodes. The function `sst.build_binary_rnn_dataset` creates datasets in this format:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "X_rnn_train, y_rnn_train = sst.build_rnn_dataset(\n", " SST_HOME, sst.train_reader, class_func=sst.ternary_class_func)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each member of `X_rnn_train` is a list of lists of words. Here's a look at the start of the first:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['The', 'Rock', 'is', 'destined', 'to', 'be']" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_rnn_train[0][: 6]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because this is a classifier, `y_rnn_train` is just a list of labels, one per example:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'positive'" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_rnn_train[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For experiments, let's build a `dev` dataset as well:" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "X_rnn_dev, y_rnn_dev = sst.build_rnn_dataset(\n", " SST_HOME, sst.dev_reader, class_func=sst.ternary_class_func)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vocabulary for the embedding\n", "\n", "The first delicate issue we need to address is the vocabulary for our model:\n", "\n", "* As indicated in the figure above, the first thing we do when processing an example is look up the words in an embedding (a VSM), which has to have a fixed dimensionality. \n", "\n", "* We can use our training data to specify the vocabulary for this embedding; at prediction time, though, we will inevitably encounter words we haven't seen before. \n", "\n", "* The convention we adopt here is to map them to an `$UNK` token that is in our pre-specified vocabulary.\n", "\n", "* At the same time, we might want to collapse infrequent tokens into `$UNK` to make optimization easier.\n", "\n", "In `utils`, the function `get_vocab` implements these strategies. Now we can extract the training vocab and use it for the model embedding, secure in the knowledge that we will be able to process tokens outside of this set (by mapping them to `$UNK`)." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "sst_full_train_vocab = utils.get_vocab(X_rnn_train)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "sst_full_train_vocab has 18,279 items\n" ] } ], "source": [ "print(\"sst_full_train_vocab has {:,} items\".format(len(sst_full_train_vocab)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This frankly seems too big relative to our dataset size. Let's restrict to just 10000 words:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "sst_train_vocab = utils.get_vocab(X_rnn_train, n_words=10000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Pure NumPy RNN implementation\n", "\n", "The first implementation we'll look at is a pure NumPy implementation of exactly the model depicted above. This implementation is a bit slow and might not be all that effective, but it is useful to have available in case one really wants to inspect the details of how these models process examples." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "rnn = RNNClassifier(\n", " sst_train_vocab,\n", " embedding=None, # Will be randomly initialized.\n", " embed_dim=50,\n", " hidden_dim=50,\n", " max_iter=50, \n", " eta=0.05) " ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Finished epoch 50 of 50; error is 15.908830618217432" ] }, { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 4min 7s, sys: 332 ms, total: 4min 7s\n", "Wall time: 4min 7s\n" ] } ], "source": [ "%time _ = rnn.fit(X_rnn_train, y_rnn_train)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "rnn_dev_predictions = rnn.predict(X_rnn_dev)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " negative 0.34 0.29 0.31 428\n", " neutral 0.25 0.34 0.29 229\n", " positive 0.41 0.40 0.40 444\n", "\n", " micro avg 0.34 0.34 0.34 1101\n", " macro avg 0.33 0.34 0.33 1101\n", "weighted avg 0.35 0.34 0.34 1101\n", "\n" ] } ], "source": [ "print(classification_report(y_rnn_dev, rnn_dev_predictions))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### PyTorch implementation\n", "\n", "The included PyTorch implementation is much faster and more configurable." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "torch_rnn = TorchRNNClassifier(\n", " sst_train_vocab,\n", " embed_dim=50,\n", " hidden_dim=50,\n", " max_iter=50,\n", " eta=0.05) " ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Finished epoch 50 of 50; error is 0.21702845860272646" ] }, { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 11min 30s, sys: 2min 33s, total: 14min 4s\n", "Wall time: 2min 26s\n" ] } ], "source": [ "%time _ = torch_rnn.fit(X_rnn_train, y_rnn_train)" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "torch_rnn_dev_predictions = torch_rnn.predict(X_rnn_dev)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " negative 0.58 0.61 0.59 428\n", " neutral 0.21 0.17 0.19 229\n", " positive 0.59 0.61 0.60 444\n", "\n", " micro avg 0.52 0.52 0.52 1101\n", " macro avg 0.46 0.46 0.46 1101\n", "weighted avg 0.51 0.52 0.51 1101\n", "\n" ] } ], "source": [ "print(classification_report(y_rnn_dev, torch_rnn_dev_predictions))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TensorFlow implementation\n", "\n", "This has a very similar interface to the above implementations. It's generally faster than both of them, but you might find TensorFlow to be more challenging when it comes to debugging new architectures." ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "tf_rnn = TfRNNClassifier(\n", " sst_train_vocab,\n", " embedding=None,\n", " embed_dim=50,\n", " hidden_dim=50,\n", " hidden_activation=tf.nn.tanh,\n", " cell_class=tf.nn.rnn_cell.LSTMCell,\n", " train_embedding=True,\n", " max_iter=50,\n", " eta=0.05)" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:From /Applications/anaconda3/envs/nlu/lib/python3.7/site-packages/tensorflow/python/ops/losses/losses_impl.py:209: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n", "Instructions for updating:\n", "Use tf.cast instead.\n", "INFO:tensorflow:loss = 1.0981584, step = 1\n", "INFO:tensorflow:loss = 0.02402336, step = 101 (27.944 sec)\n", "INFO:tensorflow:loss = 0.013917241, step = 201 (27.987 sec)\n", "INFO:tensorflow:loss = 0.0017507431, step = 301 (27.898 sec)\n", "INFO:tensorflow:loss = 0.00930609, step = 401 (27.060 sec)\n", "INFO:tensorflow:Loss for final step: 0.038237844.\n", "CPU times: user 4min 26s, sys: 1min 12s, total: 5min 39s\n", "Wall time: 1min 56s\n" ] } ], "source": [ "%time _ = tf_rnn.fit(X_rnn_train, y_rnn_train)" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "tf_rnn_dev_predictions = tf_rnn.predict(X_rnn_dev)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " negative 0.61 0.60 0.60 428\n", " neutral 0.25 0.23 0.24 229\n", " positive 0.62 0.65 0.63 444\n", "\n", " micro avg 0.54 0.54 0.54 1101\n", " macro avg 0.49 0.49 0.49 1101\n", "weighted avg 0.54 0.54 0.54 1101\n", "\n" ] } ], "source": [ "print(classification_report(y_rnn_dev, tf_rnn_dev_predictions))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Pretrained embeddings" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With `embedding=None`, `RNNClassifier`, `TorchRNNClassifier` and `TfRNNClassifier` create random embeddings in which the values are drawn from a uniform distribution with bounds `[-1, 1)`. You can also pass in an embedding, as long as you make sure it has the right vocabulary. The utility `utils.create_pretrained_embedding` will help with that:" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "glove_embedding, sst_glove_vocab = utils.create_pretrained_embedding(\n", " glove_lookup, sst_train_vocab)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here's an illustration using `TorchRNNClassifier`:" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "torch_rnn_glove = TorchRNNClassifier(\n", " sst_glove_vocab,\n", " embedding=glove_embedding,\n", " hidden_dim=50,\n", " max_iter=50,\n", " eta=0.05) " ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Finished epoch 50 of 50; error is 3.2841555774211884" ] }, { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 8min 52s, sys: 2min 27s, total: 11min 19s\n", "Wall time: 1min 58s\n" ] } ], "source": [ "%time _ = torch_rnn_glove.fit(X_rnn_train, y_rnn_train)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [], "source": [ "torch_rnn_imdb_dev_predictions = torch_rnn_glove.predict(X_rnn_dev)" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " negative 0.64 0.59 0.62 428\n", " neutral 0.29 0.22 0.25 229\n", " positive 0.62 0.74 0.67 444\n", "\n", " micro avg 0.57 0.57 0.57 1101\n", " macro avg 0.52 0.52 0.51 1101\n", "weighted avg 0.56 0.57 0.56 1101\n", "\n" ] } ], "source": [ "print(classification_report(y_rnn_dev, torch_rnn_imdb_dev_predictions))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tree-structured neural networks\n", "\n", "Tree-structured neural networks (TreeNNs) are close relatives of RNN classifiers. (If you tilt your head, you can see the above sequence model as a kind of tree.) The TreeNNs we explore here are the simplest possible and actually have many fewer parameters than RNNs. Here's a summary:\n", "\n", "\n", "\n", "The crucial property of these networks is the way they employ recursion: the representation of a parent node $p$ has the same dimensionality as the word representations, allowing seamless repeated application of the central combination function:\n", "\n", "$$p = \\tanh([x_{L};x_{R}]W_{wh} + b)$$\n", "\n", "Here, $[x_{L};x_{R}]$ is the concatenation of the left and right child representations, and $p$ is the resulting parent node, which can then be a child node in a higher subtree.\n", "\n", "When we reach the root node $h_{r}$ of the tree, we apply a softmax classifier using that top node's representation:\n", "\n", "$$y = \\textbf{softmax}(h_{r}W_{hy} + b)$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TreeNN dataset preparation\n", "\n", "This is the only model under consideration here that makes use of the tree structures in the SST:" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "X_tree_train, _ = sst.build_tree_dataset(SST_HOME, sst.train_reader)" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAACsIAAALRCAIAAABk8bP+AAAACXBIWXMAAA3XAAAN1wFCKJt4AAAAHXRFWHRTb2Z0d2FyZQBHUEwgR2hvc3RzY3JpcHQgOS4xNnO9PXQAACAASURBVHic7N09byNZnifqU7WFxbaEufdGY5VG4wIpxDiLkhlZ9owRcnrcptzGONQXmAFpjkli+wswnMa6yXKrHYXRZbfCWFyknIsOKLFrLFJABu4OpFmso2ucrCgWJVGkxFfpeVAoMBgvOkGJQWac3/mfr25vbwMAAAAAAAAAQAhfb7oBAAAAAAAAAMC2ECMAAAAAAAAAAL4QIwAAAAAAAAAAvhAjAAAAAAAAAAC+ECMAAFi3qqqaplnnjgAAAAAAMCcxAgCAdev3+1VVTT05Tz7g3h0BAAAAAGCJxAgAANat0+mkaTr15Onp6dN2BAAAAACAJfrq9vZ2020AACAcHx+fnZ1tuhUAAAAAALx2YgQAAD8rimI8Hg8Gg/F43E4f0Ov18jxvt6mqqiiKuq6TJEmSZDAYJEkyubbf74cQkiSJlQMGg0G7tp2VYDAYZFkWn6zr+vT0tKqq9pkkSd6/fz/ZsHt3DCGMx+OiKEIIWZbFH3RychLnRxiNRmmajsfj8XjcNE1sT6/Xm2wtAAAAAABMESMAAPiFsiyHw2EbHWiapt/vd7vd2HkfUwLv37+PnfFTiyGEd+/enZ2dTa69W2Og3+/neT4ZTQjzVSO4d8e6rofD4Wg0ap85OTmJKYTxeFyWZbuqqqrhcDgVUAAAAAAAgElfb7oBAABbJ8uytqs+1hsYDodxsSiKydBAlmWdTmc8Hrf7pmlalmW7drJ3f0XSNG2aJlYgCCGUZdmWKxiPx5MNiOc12VoAAAAAAJjyzaYbAACwdSZnDQghJEnSdtLHiQwm13Y6nTZkEEIYjUZFUfT7/TjrQbfbXUODu93ucDiMkxrESRni82VZHh8fT23c6XTW0CQAAAAAAHaUGAEAwLSyLCf72uu6TtM0Pm7zBK2qqtpgQVzb6/XaxePj4/Pz81U3OM/zGGWI2YW2PXmem8IAAAAAAICFmNQAAGBaXddtgYG6rk9PT9tkQJ7n/X6/3bJpmuFw2GYOqqoqiqJdO1W3YLY4N8HkkRdqc6fTKYpiOBxO1j+YqpQQQqjruq7rhY4MAAAAAMCr8tXt7e2m2wAAsEXKsgwh1HVdlmXs2u/1em01ghBCURRxVQihqqrBYNBOglCWZVEUbT2Auq7zPG/79cfjcQwZTNYMGI1G7aH6/X48VMwQjEajeXZsvXv3Ls/zdkaDaDgcVlUVt4yHHQwGC+UbAAAAAAB4VcQIAAB+IcYI8jyfsU3TNFVVPbTZ7LXzHDbLsuX29MeTWvphAQAAAAB4ecQIAAB+YZ4YAQAAAAAAvFTfbLoBAABbpCiKOH1AkiR3Zw0AAAAAAIAXTzUCAAAAAAAAAOAL1QgAAJ6uubmpLi8fXHt9XX38OGP3uO+//q//9Tf/4T/EZ7LDwxnb599+++S1AAAAAAAwD9UIAICXYCnd+Q/u+/DaJ8iPjqaemYwRrPPnThJfAAAAAAAgEiMAAJZpV7rzk/39GR3nyd5e+ubNjLXP6XRfil15nYP4AgAAAADArhEjAICXaVe6mVfXnZ/s72dv3z6vdUzblb+rIL4AAAAAAPBUYgQAsFq70u2qO59ttivvoyC+AAAAAADsPjECAF6LXemG1J0P22NXrhtBfAEAAAAAWB4xAgDWbVe65XTnA5uyK9fJIL4AAAAAAC+RGAHA61VfXdVXV09b21xfz16rOx9g54gvPLoWAAAAAF4DMQKAzVtdd359dVV/+vTc9v1Edz4AqyC+8OhaAAAAAFgnMQKAn+1Kd3765k16cPDQWt35ADAP8YVH1wIAAADwOokRANvoZXTnpwcHyf7+jLWz952xFgDYXeILj64FAAAAYLPECIBZdOfrzgcAXgbxhUfXAgAAABCJEcBu0J2vOx8AYBeJLzy6FgAAAGDbiBHAYnTn684HAGD7iS88uhYAAADgIWIE7Krq48fm+vqhtbrzAQCA9RNfeHQtAAAAsP3ECHiu2d351eVlc3Pz0Nr606cZa6vLyxlHXpTufAAA4KUSX3h0LQAAADA/MYKXQ3e+7nwAAIB1El94dC0AAADsIjGC5duV7vzs8HBGh/3seyjZ27dP7uwHAACA5xBfeHQtAAAAPMdLjhHozs8OD5O9vee0DQAAAIjEFx5dCwAAwMvwQmIEx3/4Q/nhwzMPojsfAAAA2DbbH1+4/eMfn38QAAAAtscLiREUP/5YX10F3fkAAAAAyzB/fGHwu9+tqU0AAACsxQuJEQAAAAAAAAAAz/f1phsAAAAAAAAAAGwLMQIAAAAA5lVVVdM069wRAACANdv5GMGK/v1ZFEW/3z89Pa2qahXHBwAAANhF/X7/3rslj96ieWhHAAAAts3OxwhOT09XcdhutzsYDJIkEZMHAAAAaHU6nTRN7z7/6C2ah3YEAABg23yz6QY8l25+AAAAgLXpdrv3Pv/oLZqHdgQAAGDbrCNGcHJykqZpmqbj8TiEkKZpHOjfbjAej8fjcdM0SZKkadrr9eLauq5PT0+zLBsMBnGzoijaxbi2qqrj4+N4nCRJ3r9/Hx/HtSGEs7OzePwQQp7n7T9Z+/1+XdftD43HBAAAANhdRVGMx+PBYDAej9sZBHq9Xp7n8XFVVUVR1HWdJEmSJFO3aKqq6vf7IYR4tySEMHnDpJ2VYDAYZFnWPj/7Fs2MHeOtnhBCe7fn5OQkxhFGo1FswEN3jQAAAFidr25vb9fwY379618PBoPYhR//vToajeKq8XhclmW7WFXVcDic/Kfm8fHx2dnZnItTjo+PsyxLkqTX64UQyrJs/9kc//0ZHw+HwyRJ7obi+/1+nuftLgAAAABbrizL4XDYRgeapun3+91uN8uymBJ4//59vCUytRhCePfu3dnZ2eTau3ddHrpbMvsWzUM71nU9HA7b+0IhhJOTk/a+0KN3jQAAAFiFr9fzY7Isazvpsyyr67pdNR6PJ/+tmGVZnuexeMBSxKB6fDz5L9UkSZqmKcuyruupJgEAAADsrnh3JT6OJQeGw2EIoSiKydBAlmWdTmfyJkyapmVZtmsn79isSJqmTdO0EyKUZTlVrmCld40AAAC41zomNZitLMu25F2r0+ks6/j3TrzXNM3p6Wks3xdCqKpq8t+oAAAAALtr6i5HHEoRH0zNCNDpdGLCIBqNRkVRxIkg7y3cuArdbnc4HLYzWk5Oo7Dqu0YAAADca/MxgjzP11+M7vT0tNfrtf+oLsuyzdoDAAAA7LSyLCf72uu6TtM0hNAO+m9VVdUGC+LatqZj0zTHx8fn5+erbm2e5zHKELMLk0GHjdw1AgAAYE2TGswwFXsPIdR1/dAUA3fL1sXad+3i3X8P3ytJkqkSefM2FwAAAGC71XXd3myp6zqOpggh5Hne7/fbzZqmGQ6HbeCgqqqiKNq1U3ULHvW0WzRRp9MpimI4HE7VP1jorhEAAADL8tXt7e1Kf0DTNCcnJ1VVdbvdWJWu3+8XRdEuhhCGw2FVVZO5+MFg0P5jtd/vN00T6++laRr/SdnuW1VVv9+PmYC4b5wzbzgclmU5OVvBYDBoH4/H47Is2xkN8jwviqLT6bQF9OI/mydT8KPRKLYQAAAAYGvFgot1XZdlGbv2e71ee0+jKIr4fAihqqrJuyVlWRZF0d4Jqes6z/O2X//RuyUP3aKZ8zbLu3fv8jyfnNEgmn3XCAAAgFVYeYxgfvFfuVmW3f2nYFVVTdPcuyqE0DRNVVUP7fuQuNdUWQIAAACAnRZvsOR5/tAG7Y2Ue7eZvXa2p92imceMu0YAAAAs3RbFCAAAAAB4pkdjBAAAADDbN5tuAAAAAADLURRFnEEgSRLzMwIAAPA0qhEAAAAAAAAAAF98vekGAAAAAAAAAADbYucnNWhuboZ/+tP/+z/+x3/8m7/p/fa36cHBplsEAAAAsHz11VV9dTXnxs31dfXx4wIH//SpublZoCWfPt27Ktnfzw4P5/+5C22cHhzMf+cn2d/P3r6d/+AAAAC0dnhSgxggKP785+b6+u//03/6r//tv4UQun//973f/jbZ29t06wAAAIAtVV5czL/xQp33IYTq8nKhjZvr6/m3X5b86Gj+jdODg2R/f8YG/9/Nzf8581bMTrwm2eHh7NOc9OhrMiX/9tvFWuLWFgAAsFG7GiPof/99DBDE3EB6cBBTBcMffkj294UJAAAAYNUW7V9frPN+kcHxzfX1Qh3Vy5K+ebNQWcTFhum/fTt/R/WrHXnf3Nws9Ktf6I+wub5erPzDJv4IV1r+wR8hAAC8WrsXIyh+/HH4pz/Vnz61AYLJtfXVVSxRIEwAAADArjA4frmD4ycle3uLdrK6k8AaeNd71wMAwDbbpRhBGyDIj44Gnc6MgPNUmGDwu9+ts50AAABsxErHJe/E4HjjkoGV2p4aJPXVVf3p0/wHX5aFapAke3vpmzfzH3yhy2x6cLBQNRQAAFjIbsQIJgMEvd/+ds755NowQfrmTe+3v+3+3d+tup0AAAAvQ/Xx4/yDU1/kMFmzpAO8JAsFGqrLy/kDDWHBz7Xyw4f5N16ihco/LBQyW7T8w0IfggAAbMq2xwjKi4vT//JfFg0QTKo+fuyPx+WHD8IEAADAmpm0e6WD4xcaiGlwPAA7Z6GgXnN9XX38uMDBd6H8w0KpvkXLPyx0q1n5BwDgtdneGEF5cTH805/KDx+eHCC492jCBAAAsIsMjjc4HgDYCYvGKBcq/7BQjDJsqPzD9sQog/IPAMBTbWOMYHVd/pNHHv3+975CAQDw2mzPXV2D4w2OBwDYKgvlVhct/yC3KrcKALtlu2IE66kZsNw6BwAAvE7bc5NRjVk1ZgEAeLXMoiUoDACrsC0xgvrqqj8ej//yl7VNOlD8+OPwT3+qP30SJgAA2KDtGRwflDxV8hQAAFgX05blR0fzb7xQ+Ydkb2/Rf2Yq/wDAlM3HCOqrq+Gf/lT8+c/J/v6g01lDgGCSMAEA8GIYHG9wPAAAACu1aKBhofIP9adPC+XmN1L+IX3zZqF/8C4WaHj7dqGZNfzTG2B1NhkjmAwQdP/+73u//e2m8m5tmKDz3XeDTscHDwC8cgv9I3+hwfFhwf74jQyODwsOiVjojsCiQyKkPAEAAOD53OtwrwNgIZuJEWxPgGBS8eOP/fG4ub6OTRImAIBnWiihv+jg+IUS+psaHL9QQn/RwfES+gAAAMDL5ubSSm8uqbwIzLDuGEFzcxMDBCGE7QkQtNrmCRMAsFUExgXGAQAAANgVq7ub11xfLzazxibu5iX7+4vNZ7HIxosGGtzNg6dZX4xgsoe+9w//sG0BgknCBAAvlfyywfEAAAAAwL2qjx+b6+s5N1709ulCY5k2dfs0Ozyc/w7nSss/ZIeHW9uRyiuxjhjBjvbKb3nhBIDNWihOu1DnfVjwC2V1eTn/V9slWmhwfHpwsNC3z0Wzuj6hAAAAAIDXqbm5Waze6iI3txcq/9BcXy/UkmXZnvIPyf5+9vbt/Adnm608RtD//vudCxBMimGC4Q8/JPv7wgTA0i3av75Y5/0ig+M39f1mocHxYcHvNwsNjvf9BgAAAACAtVmo/MOLHK23UPkHo/XWbIUxguLHH4d/+lP96dOOBggm1VdXsTKBMAHshEc/TV/kx63B8QAAAAAAwEoZHrnZ4ZFr64JZYYzg+A9/CCEMOp0XM7gzhgnKi4u/DoebbgswS//774c//PC0fVda/MfgeAAAAAAAgPXbnsmayw8f5t94ytk//3P+7bdP3n1+K4wRNDc3RqMCGzF1cTc4HgAAAAAAgJ2zqT6vFcYIAAAAAAAAAIDd8vWmGwAAAAAAAAAAbAsxAuAlqKqqaZp17ggAAAAAAAAv0r/7l3/5l+UesWmaX/3qV8s9ZlQUxXg8Ho/Hv/nNb37zm9+s4kfMtqJT2/h5wQvw+9//Pk3TNE0nn5znPXvvjgAAAAAAALA91txVvfxqBKenp0s/ZtTtdgeDQZIkmxo6vKJT2/h5wQvQ6XTuRgHmec/euyMAAAAAAABsjzV3VX+z9J/0gvvCX/Cpwa7rdrt3n5znPXvvjgAAAAAAALA91txV/c3x8XEIIU3TmDJoV8TaBU3TJEmSpmmv14tr67o+PT3NsmwwGMTNiqKIi3FVVVXxmCGEJEnev3/fHjNuEEI4OzuLxw8h5HneduP1+/26rtsfGn/Eok5OTmKJ8nj8qVN7wnm1a598aks5L9hRsRbKYDAYj8dVVcUne71enuftNlVVFUVR13WSJEmSTF2Oqqrq9/shhPgOCiFMvon6/X487GAwyLIsPvnoe/ahHcNPb/8QQnsFODk5iZfm0WgUG/DQlQQAAAAAAIDXaXY/dVhNV/WquuBvb29vb2/Pz8+73e7tT96/fz+5eH5+3ul0bifkef7Q4tSqu/I87/V6g8EgLp6dnbWrPn/+3D4eDAaj0eju7r1eb3KXeyVJ0u47eWrPOa+7i3c9dGrLOi/YUWdnZ3meT74jut3u+fl5XDw/P8/zvH2bTC3e3t5mWTa19u6PuPcd9Oh79qEd//rXv05eK25vbyevFY9eSQAAAAAAAHiFHuqnvl1xV/XSu+C/jmGCLMvqup7MQYxGo3Yxy7I8z2NyYSlitiI+nhyRHCddKMuyruupJi0ky7I2XjF5nFWfV3jg1JZ1XrC74tstPo71BobDYVwsiuL9+/dtFCvLsk6nM/nGTNO0LMt27eS7eEXSNG2api0OU5blVLmCVV9JAAAAAAAA2DkP9VOHXeuC/+beZ8uybEsitDqdzpMafI+HZjE/PT2NJc1DCFVVTfbbLcWqzyvcd2prOC/YflN/9vGC1T6emhGg0+m0IYMQwmg0KooiVlxJkuTeC8jSdbvd4XDYlo6ZLPCyhisJAAAAAAAAL8ludcHfHyPI83xqEvE1OD097fV6bbvLsmzHHy/LSz0v2H5lWU5eB+u6TtM0Pm7zBK2qqtpgQVzbhqeapjk+Pj4/P191g/M8j1GGmF2YDDps5EoCAAAAAADA7tqtruqv7312aihwCKGu64fqG0xVWojFwNvFux2ED0mSZKps+Jw7zu855xWeemprOC/YfnVdt+++uq7jNSsu5nne7/fbLZumGQ6HbeagqqqiKNq1U3ULZnvy5SjqdDpFUQyHw6n01kJXEgAAAAAAANitruqvbm9vQwj9fr8oim632xbuHg6HVVXF4cKxEYPBoO3A6/f7TdPEsuRpmsZutrhvVVX9fj+2Ju7YzvEwHA7LspwslTAYDNrH4/G4LMu2nEKe50VRdDqdtqh47EqcHBk8Go3aAc3t63VyclJVVdueqVN78nk9+dSWcl6w02Ksqa7rsizjNa7X603+kRdFEVeFEKqqmrwylGVZFEX77qjrOs/ztl9/9jtoxnt2zrfeu3fv8jyfnNEgmn0lAQAAAAAA4FV5tJ86rKarekVd8F9iBA+JnX9Zlt3tHquqqmmae1c1TVNV1UM7zn5xYzHzOadkeLKnnVd46qmt7bxgO8V3XJ7nM7Zp31z3bjZ77TyHXfRyNI8ZVxIAAAAAAAC4aye6qh+JEQA83zwxAgAAAAAAAGAbfLPpBgAvXFEUsRxKkiQm7AAAAAAAAIAtpxoBAAAAAAAAAPDF15tuAMDylRcX5cXFplsBAAAAAAAAu2dVkxrEPrzB7363ouNvUPHjj+nBQf7tt5tuCLxG1cePzfV1CKG+uqqvrkIIzfV1+6C6vLx3r+zwMNnfDyGkBwfxQbK3lx0ehhCS/f3s7ds1tR4AAAAAAACepL66Kn78cT1d8KuKEYQQhj/88CJjBOO//CU7PBQjgOVqYwHtgxBCGwsoP3y4d6/0zZv04CD8FAvIj47afEB7hObmJoRQf/rU3NzUV1f1xUX96dO9R8uPjuKD9gjpwcGX40sbAAAAAAAAsDn11dXauuBXGCMACCE0NzcxDdBcX1cfP8Yn23xAdXkZqwtMSfb327787PAwPm7jO9nhYbK3N89Pn5H4aRsWQmhnQIhpg+rycs6GfXnw9m1b7SDGDgAAAAAAAGDpyouLNYx4X22MoLm5mbOrD9hFbe/7VDd8iEUF5hj0H3viN9INn+zttRfZGVfbydkTpmIQ86QNkr299M2bqZ8yfwwCAAAAAAAA1my1MYLq8lLxf9hFbSygnRSgub7+uTf9p0H8U7LDw5gGaPMBL2BSgMlkQ+e77x7arPr4MeYJpiZliLUN4qQMwx9+mD74xKQMMW0wOSmD6ycAAAAAAADrZ1IDeF0e6u2OD2Jv911tb3d6cJAcHuZHR3q7p8wZkmhf/6l8RnNzU3748Gg+Iz04iA/a13938xkAAAAAAABsJzECeCFm1N6PD+apvR8fq72/Om2X/+zsxd1qEHG2iPrqqr64mGe2iPjgBVSDAAAAAAAAIISwtpnBgxgBbL9YFT8+bnuXY6dyiOmB+zqV23xAmJhiIHv7th3Uvs4LDYtqQwYz0gYz/jCqy8tHgyNhIm3gDwMAAAAAAGDLvYQYgY4omMfdQeexxH188FCJ+8lB57En2KDzVyjZ25snbTCjTMU8aYNYpmLqpyhTAQAAAAAA8IKJEcBKVB8/xt7ZthM3/NR3OyMfkB0etoPCk8PD/Ogo2dtrh4zPLoMP95osMND57ruHNnvoLzbWNig/fAghDH/4Yfrgb958ya/8lDbwFwsAAAAAALA6bT/OSpnUABYzY2x3CCH2tt412duaHR7GcgLGdrM95ixi0aYNpupnNDc35YcPc+Vj9vfDRNpA/QwAAAAAAID5vYQYQXV5aTQqu2LGTPMhdprOMdN8fGymeV6qtst/9rX97mwd8a1UX13VFxf1p0/37jU5W0d8YLYOAAAAAACA9VttjCD2HsHGPdSpGWJ1gTk6NWO/ZtupKR8AM7QhgxlpgxnBnerycs7gzpcHgjsAAAAAAADLY1IDdtvdCd1jifX4YJ4S618qqyuxDuuV7O3NkzaYMY3IPGmDZG8vffNm6qeYRgQAAAAAAGAGMQK21Iy+wxBC+eHDvXulb958KYG+t5cdHsZyAvN0VQLbabLAQOe77x7a7G6iKPxUfaS6vIxXjOEPP0wffOKKEdMGbaIouGIAAAAAAABbpi2mvmpiBKzbvZXM22fmrGQeHxtbDERzFhFp0wbt/Caxfklzc1N++DBP/ZL4QP0SAAAAAADgBVthjGBtUQi2RxsLmJrpPMQhwp8+3btX+6fS5gPMdA6sQtvlP7vSwGTC6csV7NOn5uamvrqqLy7muZTFB+0VTNoAAAAAAADYIaoRMJeHhvDGB/MM4Y39ajrVgO03z0wo9xZWiWmD6vJyzsIqXx4ITgEAAAAAAPN5qGd2ucQI+Fl5cRE7w37uG/vw4d4tJycUzw4P86MjE4oDr0qytzdP2qC+uvo5cfXxY3wyXmPnSRske3vpmzchhMHvfrfc9gMAAAAAADzkq9vb2xUdOvaXvLwR59XHj8ne3oscMNr//vviz3+e6rsKEz1k2eFhsre3qeYBvFRtxZc2dhB+mei6/eMfN9U2AAAAAABgS6ytC36FMQIAAAAAAAAAYLd8vekGAAAAAAAAAADbQowAAAAAAAAAAPhCjAAAtkhVVU3TrH9fAAAAAACA6Kvb29uFdri3iyJN0zRN67ouiiKEMBgMltbAVSrLsn0cT2FZR+73+3Vdv3//flkHXKmmaZIkWcWRi6Ko67ppmm63m2XZKn4EwAtzfHzc6/XyPJ96fp5r9UP7AgAAAAAAzO+bRXcoiiJ2Y1RVlaZp26UxGAzSNB0MBsfHx0tu42rUdV2W5Xg87nQ68ZmqqgaDwVJ6u3fodQghnJ6erijx0O12Qwj9ft/oWIA5dTqde2Nt81yrH9oXAAAAAABgfgvHCJIkicUG+v1+nudxyGO/319+01Yshh5idCA+0zTNycnJ2dnZZhu2fvr4AbZHDGDdNc+1+qF9AQAAAAAA5rdwjODeLoqpJ4uiKMuyaZrYVT9ZhHk8Ho/H41iZOU3TXq+3onL6T5AkyVQpgqqqYln+JElifmKytXVdD4fDuq7DT3Mi5Hl+t5jByclJPN9njvg/OTmJP2U8HoefYhDzvLZ1XZ+enmZZFgMT4/G4KIq4GFdVVdXWTphqZ9wghHB2dhaPH0LI87z9jcfpG9ofuivzWQA8TVEU4/F4MBiMx+OqquKTU/MIzP7sqKoqZu/iZTNMzATU7/fjMadK4zx6rZ6xb7zmhxDaT4H4qRRCGI1G8TNlaz+XAQAAAACAzbh9ql6vd3Z2dvf5JElGo1F8fH5+3u1221Xv37+fXDw/P+90Ok9uwFLked4+Pjs7m2zP+fl5nuefP3++d/Hz589Ti51OZ/IFiUeOz5+fny+ltc95bSfPdGpxatVdeZ73er3BYBAXJ0+zfQVub28Hg0HbvEkP/akA7KKzs7M8z9vL2ufPn7vdbnudn/3ZcXt7m2XZ1Nqp4z90zXz0Wv3Qvn/9618nPyBub2/bD4gt/FwGAAAAAAA27uul5xKyLGuHqmdZFgfrR+PxeDQaTW6Z53kc4L4pcXDn8fHx3/7t35ZlOTm4syiK9+/ft4MysyzrdDpta4uimByymSRJr9ebmpG6qqqTk5PRaHS3RMHTbPC1jUNU4+PJQbdJkjRNU5ZlXddTTQJ4qeI1Nj6O9QaGw2FcnP3ZEUJI07Qsy3bt5KV7RdI0bZqmnROhLMv2U2kLP5cBAAAAAICNW3hSg+coy7ItyNzqdDrrbMOULMvOzs5CCJPlqaNYjHrymU6n03YUVVXVdqu3h5o6eJw7YD3VoVf92t47mUXTNKenp+0LVVXVsgITANts6loXA1Xt4xmfHSGE0WhUFEWcESZJknuvrkvX7XaHw2E7tU07jcIWfi4DAAAAAAAbt9YYQZ7nU3M5b4/Ya9Lv99vOlbZPqFVVVds5lKZpXddT5QemjEajWO1gcmTqimzktT09Pe31em13WlmW7RBbgBesLMvJvvbJj4PZnx1xbZtCkdnbggAAIABJREFUa5rm+Pj4/Px81Q3O8zxGGWJ2oW3PNn8uAwAAAAAAm7L8SQ1mmBqRGUKo63p7yuB3Op3J9uR53u/327VN0wyHw7bfqNvtTq4NIVRVVRTF1DGzLBsMBicnJ3c7lpZrodd2qmB1rHfdLs7f1CRJJofkqoMNvBJ1XbeX3LquY6YqLs7+7Jj6pFgoYfbka3XU6XSKohgOh5P1D7b8cxkAAAAAAIjquv7bv/3buzWGV+Sr29vbRfcpimI8HrcjGgeDQexLbprm5OSkqqputxvH9B8fH1dV1el02qmXh8NhVVVx1GbsAhkMBusp+z8ldvzEOvztKdR1fXJy0uv1YpdPURRlWcbWVlXVbhZNrp06l5OTk7IsP3/+HI8Zf52dTqctdbCou69tv98viqJdDI+9tv1+v2maWHk7TdPYkxT3raqq3++3v8QQwuTvqyzLydkKJl+E8XhclmU7o0Ge50VRtKc5Ho9jb9nk4NfRaDS7hAPAlotlV+q6jh8BTdP0er3JK9uMz46yLIuiaC+JdV3neR779R+9Zs64Vs95vX337l2e51OfRNvzuQwAAAAAADykqqp3794lSRL7oFftKTGC54t9MFmWbX9HRdM0VVWFEPI8v3eDbTuXGe2pqqppmntXtae56InEHafKEgC8YPEy+9CHQjT7s+PRT5ZHD7v0D51t+ywDAAAAAACmxG7Z9YzZ3kyMAAB21DwxAgAAAAAAgN31zaYbAAA7oyiKOH1AkiRmaQEAAAAAAF4k1QgAAAAAAAAAgC9UIwCAbdHc3FSXl5PPlBcX/8evfvU//+3fsrdvk/399vlkfz97+3bd7QMAAAAAAF4BMQK+GP/lL8WPP5790z9tuiEAO6z6+LG5vm4Xm+vr6uPHX2zwy5RAdXk5uf29Dg8OLq+uHv3R+dHR5GJ6cPCL2MHeXnZ4+Ivtv/320WMCAAAAAACvkBgBXyT7++WHD5tuBcCGlRcXk4vV5WVzczP1zC+2n/vKOdXNHzv12679Obv5548p1FdX4eqqvrqqP32ap3npmzfpwUG7mOztpW/ezGhPenAwuT0AAAAAAPBifHV7e7vpNrAVyouL4//8n2//+MdNNwTg6eqrq/qXA/enYgH1p0+TsYAn97KHiQTAl8VdmHTg3kkTfrHB9fXkC9hcX09t/5Bkf3/6Bfnl4lTsYDtfHwAAAAAAIKhGwJT66sroUmCDpkbb340FPGFSgCg7PJzs5o81/ye7uqdG22eHh8ne3gJN3wXJ3t7UaS46tcF0LOPhX1B8MH+1hnt/Qe3i3WoNL/IXBAAAAAAA20CMgF8QIwCe4+5g96lJAZ482D08NinA3Rr7i3aQM49nvqpTsYO7kzK05SKWPinDVLkIkzIAAAAAAMBDxAgA+Nn8vbw/b/+8Xt42HKCX9zW4+2vtfPfd/LvPn1OJW65uUoYgpwIAAAAAwMslRgDwosxfc75dXMqkAGrOswbPn5Rh/lkz4oNnvkFmNNUbBAAAAACArSVGALBF7g62nooFPHlSgIcGWz80KUCyv5+9fTt/y2EnPPOv+m7sYDq48/CkDMMffphx5KlyHeFOOYSpch3eoQAAAAAArI4YAV9MdVcAT/PopABTvf5LnBRgaqyzSQFg6Z4/tcFU7OChSRnCT9eK8sOHOY/cXgqiqY/1u/VCTMoAAAAAAMBDxAj4QmlliGZ38oU7OYAnd/I9OimATj54eZY7KcOMoNIzJ2Vog0oPNdWkDAAAAAAAL9hXt7e3m24D2+Krf/zHs3/+Zz2X7LRHJwVoS45/2f7ZkwL8vKjkOLDdNniFNG0KAAAAAMAOESPgZ1/94z/2/uEfBr/73aYbwqs2Ndb27kzkU31axtoCrM2a67W0i+q1AAAAAACsk0kNgCWbf+bvL9s/b+ZvkwIArM0zJ2WYSobNmJShvroKV1f11VX96dM8R07fvPlFtYPHgmJT1REAAAAAAJikGgE/U42AcN/o/9klr5/cxxMemxRAHw8Akx6dlGEqqWZSBgAAAACAp1GNAF6atU0KECtOT3a0mBQAgNVJ9vaeWQ5h/o/I+OCZH5GTLb8bU/ARCQAAAABsLTECfjZ5v5tNuTvUcvakAM8fatk+eXf0v0kBAHhJnllC4NFJGdqCPc+clCEo2AMAAAAAbJRJDfjZ8R/+kB0emtTgmebvY/h5+yVN/KyPAQC21qJJwRBC+eHDnAfPj44mF2dPyhAkBQEAAACAmVQjgGlTEy0/OinAk2/xT00KoOIxALxguzspw92m+ooCAAAAAC+bGAG/MOe95hfg+A9/iA9WNylAsr//zOLJAACt5U7KEO5GJx+elGH4ww8zjjz5Hanz3Xfdv/u757QTAAAAANg4kxrwSh3/4Q93JwWYGmlnUgAAgNZU7GBqUob4TO+3vzVjAgAAAADsOjECAAAAAAAAAOCLrzfdAAAAAAAAAABgW4gRAAAAAAAAAABfiBG8XlVVNU2z6VYAAAAAAAAAsEW+ur293XQb2Izj4+Ner5fn+aYbslZN0yRJsvTDFkVR13XTNN1uN8uypR8fAGCbregrVvAtCwAAAAA2QTWC16vT6aRpuulWrNvp6ekqDtvtdgeDQZIkCjwAAK/Qir5iBd+yAAAAAGATvtl0A9iYbre76SZsgBvQAABL5ysWAAAAALwkYgSvUb/fr6oqhDAYDO7Whq2qqt/vhxCSJInlCgaDwZpbeHJykqZpmqbj8TiEkKZpHIXWbjAej8fjcSyfm6Zpr9dLkqSu69PT0yzLYoPH43FRFO1iXFtV1fHxcTxIkiTv379vjxk3CCGcnZ3F44cQ8jxv8xb9fj/W1I0/dP0vCwDAczztK1b46WvSvd+yfMUCAAAAgJfnq9vb2023gc3o9/t5nud5PvX8u3fvzs7O4i3jGCk4Oztbf/N+/etfDwaDeH+5qqqiKEajUVw1Ho/LsmwXq6oaDoft3erj4+PJBs9evOv4+DjLsiRJer1eCKEsy/Ylmpz0dzgcJkkyVdHhoZcUAGBLPPkrVpj5tWqlX7GCb1kAAAAAsF5fb7oBbJ00TcuyjI+zLGtvJa9ZlmXtHeQsy+q6bleNx+PJVmVZlud5HNm2FHHsXXw8ebc6TspblmVd11NNAgDYCb5iAQAAAACPMqkB00ajUVEUsbrsvaPBNq4sy7ZqbqvT6Szr+PeectM0p6enSZK0dRruzgcBALC7fMUCAAAAACIxAn6haZoQQjtQrGma4+Pj8/PzjTZqWp7nk/V11+P09LTX67X3tcuybGs2AAC8AL5iAQAAAACRSQ34hThFbrvYzlO7VTqdznA4nHymrut769/eLcObpmmMSkSTj2dLkmRybNwSC/wCAGyD+b9ihTvfhXzFAgAAAICX5Kvb29tNt4G1Go/HMSgQ5yyIQYHRaJSmaQihLMuiKNrn67rO83zN8xo0TXNyclJVVbfbHQwGIYR+v18URbsYQhgOh1VVxTbH+9SDwSC2ud/vN00TJ9lN03Q4HE7uWFVVv9+Pd6vjju0cwMPhsCzLyVK6g8GgfTwej8uybMvt5nleFEWn0xkMBrNfUgCAbfDMr1hh5resVXzFCo99cQUAAAAAVkSMgHs0TVNVVQghz/NNt2WWWPM2y7KpqglVVTVNc/f5qD27hzZ4SNxxaswcAMAL89BXrDDzW5avWAAAAADwYogRAAAAAAAAAABffL3pBgAAAAAAAAAA20KMAAAAAAAAAAD44ptNNwAAANhV5cVFCKG6vGxubprr6//5b//29j/+xxBC9vZtsr+fHhykBwebbiMAAAAAsJivbm9vN90GWLf+999Xl5dn//RPm24IAMAOmMoK1FdXzfV1dXk5uU2yv58dHn7z9ddX//qv964KIcT/x2xBsr+fvX27phMAAAAAABahGgEAALBAViCEkB8d5UdHs0sO3D1g+eHD1AHjoYKEAQAAAABsEzECAAB4LZaeFZgh//bb9v+PNkPCAAAAAAC2hxgBAAC8KOvMCjzNowmD+uqqvroKIVSXl9XlZfnhw/QRJAwAAAAAYGXECAAAYPdsf1bgae7NFkTVx4/xTCUMAAAAAGClxAgAAGBLvdSswNPMiAJIGAAAAADAEokRAADAJskKPJ+EAQAAAAAskRgBAACsnKzApkgYAAAAAMCixAgAAGA5ZAV2y/MTBtnhYfz1Jfv7yd7el9/st9+ustUAAAAAsHJiBAAAsABZgddgoYRBfXVV/vnP00eQMAAAAABgZ4kRAADANFkBHrJowqD6y1+a6+tfHEHCAAAAAIDtJkYAAMArJSvAcs1IGMRsQXN9XX38GCQMAAAAANhuYgQAALxksgJsg/bPqfPdd1OrJAwAAAAA2DZiBAAA7DxZAXaXhAEAAAAA20aMAACA3SArwGsjYQAAAADARogRAACwRWQFYB7LTRiEn7IF2eFhsre3pnMAAAAAYFt9dXt7u+k2wLr1v/++urw8+6d/2nRDAOCVWigr8OX/sgLwbHcTBvH/UwmD9M2b9OAg2dtL37wJEgYAAAAAr48YAa+RGAEArIGsAOwQCQMAAAAAWmIEvEbFjz/WV1eD3/1u0w0BgJep//33wx9+aBdlBWCnxYRB+CkbVH/61Nzc1FdX9adP7Tb50ZGQLgAAAMCLIUYAAMCSlRcXzfW1rAC8eM3NTaxbUF9ddf/u7zbdHAAAAACWQ4wAAAAAAAAAAPji6003AAAAAAAAAADYFmIEEKqqappm060AAIDd8OTvz754AwAAAOwEMQK2UVEUp6enVVU9/1Dz3Kbs9/tL+Vn3WuK5AMCWW13vYFEU/X7fRypsiYe+Pz96EVjpF28AAAAAlkWMgG3U7XaTJFlKV8Tp6emj23Q6nTRNn/+z7rXEcwGALTfPx+7TdLvdwWDgIxW2xEPfnx+9CKz0izcAAAAAy/LNphsAqzVPZ0O3211DSwDgxdPHD6/EQ9+fH70I+OINAAAAsBPECNgW4/F4PB43TZMkSafTmbE2TdNer5ckSbu2qqp+vx9CiGtDCIPBoK7rWPr4+Pg4bpYkyfv37ycP21ZVHQwGWZZNrjo5OUnTNE3T8XgcQkjTNA6CnKdJs88FADbryZ9x8bM1y7LBYBA3K4oiLj76sRs3CCGcnZ3F44cQ8jxv+xT7/X5d1+0PjT8CeIKiKMbj8WAwGI/H7QwCvV4vz/N2m6qqiqKo6zpJkiRJJi8C9361bnd86PvzoxeBh3aMV5IQQnttOTk5iXGE0WgUGzD73wIAAAAALN8tbIH37993u912cTQaZVl2dnZ279rz8/NOpzO5e5Zlnz9/btfmed6umnz8kF6v1/6sSUmSjEaj9rCTbZjRpNnnAgDb4GmfcdHUZ+tCH7t5nvd6vcFgEBcnPx/bj/Lb29vBYNA2b9JDH9nAlLOzszzP2/fL58+fu93u+fl5XIxfmKe+P7eLM75atx56Mz56Ebh3x7/+9a+Tl53b29vJy86jFyUAAAAAlu7rTccYIIQQxuPxaDRqF7vd7tSYyMm1WZbleR5HMUZpmpZl2a6d3Pg5sixrh0hmWVbX9TxNmn0uALANnvYZtxRxJHF8PDk2OkmSpmnKsqzreqpJwBPEN298HOsNDIfDuFgUxfv379vvqFmWdTqd9m2+oq/WM6Rp2jRNOyFCWZZT5QpWelECAAAA4C6TGrAV7s6iOnnrsCzLtjhqa3KygNFoVBRFLIacJMkaplyd0aTZ5wIAW+7Rj91nuvdjumma09PTWFw9hFBVlU9PeKapN1FM6rSPp3KunU6nDRms/6t1CKHb7Q6Hw3bClMlpFFZ9UQIAAADgLjECtsLd8fqTnfF5nk/Oq3rvlu24xqZpjo+Pz8/PV9DMn81o0uxzAYAtN/tjd0VOT097vV7b61mWZTsYGniasiwn+9rruk7TND6+++20qqr4JXYjX61DCHmexxxDzC5MfqPeyEUJAAAA4JUzqQFbIU3ToijaxaqqJuuUTo6Oiuq6bmsdV1U1ue9UL34skdouLqtHf0aTZp8LAGy52R+7U6Y+4578sZskyVQN83mbCzygruv2vVzXdQzrxMU8z/v9frtl0zTD4TBmDmZ/tX7Uc757dzqdoiiGw+FU/YOFLkoAAAAALMVXt7e3m24DhBBCv99vmmbyTmVZlr1eL97QHA6HVVXFEVTxduRgMIgbl2VZFEU7aKmu6zzP25uPVVX1+/3YMxF3bKdWHY/H8Sbp5Jin0WgU736enJxUVdXtdmNJ1X6/XxRFuzi7SbPPBQA26JmfcWHiY65pmjRNY59f3HfGx+5wOCzLcnK2gsFg0D4ej8dlWbYzGuR5XhRFp9NpK5w/9JG9jpcMdlCs51HXdVmW8cttr9ebfMsURRFXhRCqqmrfj7O/Wj/6ZnzoIjDnu/jdu3d5nk/OaBDNvigBAAAAsHRiBGyROK4oTdOHegXi/dAsy+6dOKCqqhBCnucPrbp3x2d6qEmPngsAbLkZH7tVVTVNM/sTedGP3bjjVFkC4Gni+/fuF+NJ83x/nn2E2Ydd+nfvGRclAAAAAJZLjAAAAOBFmSdGAAAAAAAP+WbTDQAAAGBpiqKIMwgkSWL6DwAAAACeQDUCAAAAAAAAAOCLrzfdAAAAAAAAAABgW5jUAACAJyovLkII9dVVfXUVQqguL+Pi/7W39+v9/RBCenCQ7O+HEPJvv42L6cHBBhsMLFFzc1N++FB9/FhdXv7rv/3b3/zqV9nhYf7tt9nhYbK3t+nWAQAAAPB0JjUAAOBB1cePzfV1c31dffwYQqg/fWpubuqrq/rTp6kts8PDZH8/2dtL37y5+d//e+/f//uYKqguL5vr66mN86OjuEsIIdnbiw90PcL2qz5+LD98qK+uyouLeB1I9vfzb7/9v3/96//nv//38sOHuFl2eJgfHaUHB/m33woPAQAAAOwcMQIAgFetrSXwpbTAp0/NzU1zfR1DAJPSN2/Sg4MYFAghZG/fJvv7yf5+9vbtoz+lubn5OVVwcxN+Kl3Qdjq2kv39NlUQJgoYxHoGwPqVFxflxUV1edmmgrLDw/jf3ZTAQzmDdvvNnAMAAAAAixAjAAB44dou/BgUaK6vv+QG7nThx6BAWHsX/lSUYdEWzhllAOZUX11Vl5cxENAmivKjo0XnLGiPU11etm/neJzs7dv86EgBEgAAAIDtJEYAALDzHhrrf3dCgamx/jsxocC9EyvcWy9hcmKFsGC9BHjl2tBA9fFjrCKQvnmTvX0bpydYypsoVjWoP30qLy7ipSl98ybWM1jWjwAAAABgKcQIAAB2w5dJB34auB870eurq9jhNyk/OgohxGH64adaAtscFHiOGDJ48svSVlyA1ybGj+JsBW2pgDY0kB0ervStEWc9iBMlxLdtDDktWvAAAAAAgFUQIwAA2Bb3Dru/t0d8ati9HvGH3Ju9uFukIfwUMtihIg3wBLM771c9fclDZgQa0oODWK5gIw0DAAAAeLXECAAA1qftz/7Svf1wff70zZv04EB9/tV5aCaIthezNTUTRBvX2FSfKyykzQ3sylQCcXqFmHiIIapkfz+WKIhxh003EAAAAODlEyMAAFimtnM6BgVivf1wX+d0DAoEndNbaSrwsejvUeCDDWpubsoPH6qPHycH9+dHR9nhYfb2bX50tEM1Nuqrq+ry8mWcCwAAAMAOESMAAFjMQ6PY75bKnxrFrlT+i3Hv9BP3VpWYmn5CVQlWJHa0x5IDkyP4Y+GBFxNOKi8u4gnuSmUFAAAAgB0lRgAAcI8vkw78NCQ9dg/XV1exf25SfnQUQogD0MNPtQQEBV6zGDJ48h9PW5cCZrvbpx7L/n/576X3qcdZD2J4Ir7LYnIrTnzgIgwAAADwHGIEAMArde+A8nv7eqcGlOvr5TnuTajcLWURfgoZKGVBS4X/h8QiMTFV0L4y2eFhfnSUHhzEcgWbbSEAAADAbhEjAABerKnp7WdUno/T26s8z2Y9NF9G2y3ampovow21vJja9bSqjx/LDx/iyPsYckrfvMnevm2H3W+6gdvo7osWp3jwogEAAADMSYwAANhhbbdrDArESvLhvm7XGBQIul3ZWVOxmEX/2sVidsXkwPq2TIWB9U+mhAMAAADAE4gRAABb7aHx2XeLwE+Nz1YEnlfl3kk67q29MTVJh9ob2+DLuPmrq/LDh/ZXFru64xh6F7FlKS8uYomC8uIifoikb97EcEZ+dORdAAAAABCJEQAAm3fvbPH11VUsRj0pzhYfh1aHn2oJ6GOD2WLI4MlvsbZ6B0sUC+/HgfLtbAX6s9cppjdi1Yf4pohxNOkNAAAAADECAGAd7h0qfW8v5tRQab2YsGr35njuFvwIP4UMFPx4mubmpvzwYaq6fpytIHv7Njs8dJXboMm5JKZ+O+aSAAAAAF4hMQIAYDmmJm6fUVM9Ttyupjpsv4dmFWn7WVtTs4q00Z+YBHq1Zo93f+UvzjaLtSK+TDbx6VMIIdnfjyUK4u9u0w0EAAAAWC0xAgBgAbGoQAwKxBrp4b4OxRgUCDoU4UWbCg8tek14qWUM2tkKyouLWNEhzlYQe6DlpXZOfXUV556YLFSQHx3F32bnu+822zwAAACAVRAjAAAW0P/+++EPP4Q7I4+VNwem3DuVyWSFkts//nGT7VuZeJ1su5nzoyNXxZekvLiIJQpiTOSl/hkDAAAAr5wYAQCwgPrqKtnf1yUGPFP18eNLHZdfX13F0gu8eM3NjQ9EAAAA4EUSIwAAAAAAAAAAvvh60w0AAAAAAAAAALaFGAEAAAA8oqqqpmnWvy8AAADA+v27f/mXf9l0GwCAHdA0za9+9aulH7YoivF4PB6Pf/Ob3/zmN79Z+vGBbfNSLyYrOq+wBadG9Pvf/z5N0zRNp56f51f/0L4AAAAA20k1AgBgLqenp6s4bLfbHQwGSZIYpgmvxEu9mKzovMIWnBpRp9O5Nwcwz6/+oX0BAAAAttM3m24AALAbdF8BS/FSLyYv9bxodbvde5+f51f/0L4AAAAA20mMAABejpOTk1gzeTwehxDSNI0DWNsNYlnspmmSJEnTtNfrxbV1XZ+enmZZNhgM4mZFUbSLcW1VVcfHx/E4SZK8f/8+Po5rQwhnZ2fx+CGEPM/bLpN+v1/XdftD4zGBLfe068lzLiZhLdeTVVwnt+G8iFM/DAaD8XhcVVV8stfr5XneblNVVVEUdV0nSZIkyeSvvqqqfr8fQoi/ghDC5G+h3+/HYw4GgyzL2ucf/dU/tG/8+wkhtH9RJycnMY4wGo1iAx76UwQAAABYh1sA4AVJkmQ0GsXH5+fn3W63XfX+/fvJxfPz806nM7lvnufzL07J87zX6w0Gg7h4dnbWrvr8+XP7eDAYtM2b1Ov1JncBtsGTryfPuZjcPu96Ms/FZEXXyZWe163r5BzOzs7yPG9fpc+fP3e73fPz87h4fn6e53n7Uk8tZlk2teru8R/6FTz6q79337/+9a+Tf2y3t7eTf2yP/ikCAAAArNTXm44xAADLlGVZO7w1y7K6rttV4/F4NBpNbpnneRwUuxRxrGR8PDn6M87nXZZlXddTTQK22Uu9nrzU8yL89CuLj2O9geFwGBeLonj//n07oD/Lsk6n0/5y0zQty7JdNflnsCJpmjZN006IUJblZJ2DVf8pAgAAAMxmUgMAeC3KsmyrLrc6nc6yjn/vxM9N05yensby0SGEqqomu0mAHfVSrycv9bxej6mXLuYz2sdTkwJ0Op02ZDAajYqiiLNLJEly729q6brd7nA4bKfJmJxGYdV/igAAAACziREAwGuR5/nUhM1rcHp62uv12n6dsizb4Z7A7nqp15OXel6vR1mWk33tdV2naRoft3mCVlVVMVgQV7WFIpqmOT4+Pj8/X3Vr8zyPOYaYXZhMOWzkTxEAAACgZVIDAHgtJoddRnVdP1Q6+27l5Fh+uV282x9zryRJpqo0z9tcYIvNfz1Z1sUkrOV68pzr5Daf1+tR13X7G6zrOkY04mKe5/1+v92yaZrhcBgzB1VVFUXRrpoqWvCoJ//qQwidTqcoiuFwOFX/YKE/RQAAAICl++r29nbTbQAAlqBpmpOTk6qqut1uLIzc7/eLomgXQwjD4bCqqjg0M/ZzDAaDtr+k3+83TRNLQKdpGns12n2rqur3+7GvK+4bp20eDodlWU5W4R4MBu3j8XhclmVbqTvP86IoOp1OW8M59txMDsQcjUbt4FHg/2fv7mEb2fr8zp++0+PxSMYYNTC18IM1SNQkuyJgYFAtYLPu4CiZTjZoKn18Ewqw074mM4+zou+NB2Al17HYqZyoAt3MwNXBRuL6WaALJHbhByNiu4AHluzZscEN/upzz+VLiaJYrGLx+0GjUXzVIatYL6d+9T+FeOH6ZL2ViXrB+mTFlUmu68k8PpdiPfkcUsUhSZI4juXUfqfTcb+oKIrkIaWUMcbOhTiOoyiyX2+SJFpre17/yVmQMetXmX1v3rzRWrsjGojsRREAAAAAACBXxAgAANg7cqIlCIL5sxHGmDRNFz6klErT1Biz7LXLyKtmLrcFUA3L1id5rEzUFtcn660ny/+5qk3mmtY64zl2Hs0/LeOhJ60965+UsSgCAAAAAADkhxgBAAAAAGDnrRIjAAAAAAAAwCpeF90AAAAAAABeJIoiGT7A8zwGfQAAAAAAAHghqhEAAAAAAAAAAAAAAIBH3xTdAAAAsDPMePzT735nxuOiGwIAQPF++t3vim4CAAAAAABALhjUAACA/ZVMJslkMj+d3t/baTMapff39iX/21/8xX/8/Nne9A4Pg0ZDpv1azTs8tNN+rTY/DQAiHg6TyaT99m3RDdmwqn4uCDMem9HI/vtf/uk//U+//33QaASNhl+rBY2GPj4uuo0AAAAAAAAbwKAGAABUSjwcykR6f++WDTCj0S/3f51eSDebdtpGBLyDA5n+4z/6o7//H/9D3jB9eJivX+c2AAAgAElEQVR5c6VUfHub8eb+0ZGNFNg3V0oF9bqNIASNhndwkPEmACqg++mTGY2uPn4suiEbVtXPtbckN5BMJmY0shs4/+hIHx/7tdr//Od//v98+SKP2k0hqQIAAAAAAFABVCMAAKDUzHhsiwGsd+Y+aDTcM/Tqa1DAPXO/xnmOVV5iMw0z09J4qXOQnWmYb/z8X+ckDQBgU5blBoJ6vfP+vT4+zsi6xcOhfW10fS132lSBbjaDen07nwIAAAAAAOCFqEYAAMBWrTGOwIyqjiOQPjz8qmTC11IK7jeTTCbJ3V3Gm9hSCt7BgX90ZKd/KatweMhZHKAMqnrVflU/V4U91hL4OlqBbH8lNyBVB9aukeOmCuwGTjebtlYB2yMAAAAAAFBaVCMAAOClNj6OgJzz3reT397BgVtXoHVykv38ZYGM5O4ufXiQUEJ2IMMdYaFKgQwAQIaFuQGJ6LXfvQvqdd1sbmRsHX18bLdrslWKh8Pk7m7w8882EkeqAAAAAAAAlBMxAgAAZpV2HAG4nnuC352t8yMsJJOJmkyeO1sfp53ZuvZFqwCAnNhT+MtyA3IiP9c2SFSOVAEAAAAAANgVxAgAANWXxzgCcpPL1neIez5mlQxHdpEJWWCeVWRi4V8nTQIAG+eepDfjsXuSfmu5gWzPShWUocEAAAAAAGDfECMAAOwkxhFA3p41woKcAZLphVEVGR5bzgz1Li/n38GNqngHB/7RkZ12IyycRgKAeRm5gdbJSfkv7s9IFUTX1275BFIFAAAAAABgO4gRAABKgXEEsNPkDNDqz3ejBu4Cn9zdpQ8PcgIpu0KGf3TkVsKwC7kbNWCEBQAVJoMUJJOJrDDlzl3JDWSbSRU8fsbx2IxGpAoAAAAAAMB2ECMAAOSCcQSADO6i+6wRFmam5cxZMpmoyeS5OZvHaXI2AHbEwtyAnEpvnZzoZnN3cwPZZJNh6+KQKgAAAAAAAFtAjAAAsCrGEQCK4p7gf/JkvzvCwsJfqyR4nvVrnf/r/FoB5E3OlNt/cqfNDQSNxn6Gn1ZMFejjY//oSB8fU5kGAAAAAACs4dV0Oi26DQCAYuQxjsDjNNc3A7tjldohyWRixxqf59YO8Q4O/KMjmaZ2CDJ0P30yo9HVx49FN2TDqvq5tkPOhct5cbsHIufCZZwCdiqe5KYK3O8wqNdJFQAAAAAAgNVRjQAAqoNxBACs4bk/6mUJpOTuLn14kFoI2Qkk/+jIXaX8kkZyEkic6AL2wbLcQFCvd96/55z3GmZqFbjfcO/ysnd5qUgVAAAAAACAFRAjAIBSYxwBAGXjrjFWuTLYrsdmpmXdlUwmqVOufPFfXFL45FljPQAoA/daeRttJDeQn6Bed1fapAoAAAAAAMCKGNQAALaKcQQAIIMUM3icXhKfyh5hQc3Fp2SC+FTZVLX4f1U/19oW5gak3FHQaAT1um42OW9dlIXVIGTWMIoEAAAAAAB7jmoEAPAijCMAABvkHRy4Z61sXe5lnlwJu2cuF//FuZWwnWYlDKzhcWST4XA+N9B+9y6o1+UUddHNhFKLahXEt7eSKoiurx+fQ6oAAAAAAIC9RIwAAGYxjgAA7IrnnuDPLgmTTCZqMskuCeMfHdm/uKwkDFXBsVdsbiC5uzPjsS0WoptNcgO7ZSZVIFkQUgUAAAAAAOwnYgQAqi+PcQQkKMA4AgCwW9wzZKust22wbGZatiBypXV2sGzZSDTuX2cLgt2SkRtonZzICWbikhWgj4/dtVN2qkA3m8x0AAAAAACq5NV0Oi26DQDwIslkEv30k3rZOAJ2mhLWAID1yLnVx2mnno27eUomE3vOdSFbz8Y7OPCPjtTcmbzK6H76ZEajq48fi27IhlX1cyml4uFw8PPPMlSB3KObTXthOqeQ942bKphZJNpv37IXDQAAAADAriNGAGDnxcPh6fffz4wjIBOMIwAAKK1kMvlVvMBO3925hXPa796FHz4U08Q8JZNJ+vBQvU1zVT+XUir66afo+ppLzzFvpkDF1cePxAgAAAAAANh1xAgAAAAAAAAAAAAAAMCjb4puAAAAAAAAAAAAAAAAKAtiBAAAAAAAAAAAAAAA4BExAgAAAAAA8GzGmDRNt/9aAAAAAACQN2IEAHZVTt2OURR1u93z83NjTB7vDwDAKtjMKaWiKCq8qQtnRJIk3W632+1u9m1Li6URy3S73YWzb5VlZtlrAQAAAABAGRAjALCrzs/P83jbdrsdhqHnebvVvw8AqBg2c0qpdrtdeFMXzgjf98MwfMkZ0Jzmb05YGrFMq9XyfX/+/lWWmWWvBQAAAAAAZfC66AYAwJrocQYAVBibuZLIaUbs1vzdrdZim9rt9sL7V1lmlr0WAAAAAACUATECAPk6Ozvzfd/3/cFgoL5eved5nn3CYDAYDAZpmnqe5/t+p9ORR5MkOT8/D4IgDEN5WhRFclMeMsacnp7Km3ied3FxYd9TnqCUurq6kvdXSmmtbWdlt9tNksT+UfkTAAA813qbuYxtnPq6FVt7M1eGbVxGG571jbVare033npyRiiloiiK4zhN0/kPsuxLWOVt15DHHteTrWWPq+SiKBoMBmEYDgYDWzyj0+lore1zjDFRFCVJ4nme53kzi40xRgbvkDmolLIz0Q5JEIZhEAT2Jass4cteK4ufUsougWdnZ5JI6Pf7sngvXIwBAAAAAMDmTQEgZ57n9ft9mb65uWm32/ahi4sL9+bNzU2r1XJfq7VednPmoXla606nE4ah3Ly6urIPffnyxU6HYWibZ3U6Hff5AAAss/ZmLmMbN39z3rLN3JPbuGn+m7nsNqz+jfX7/SAIit0iZ8yIjA8yfepLeHL+riGnPa4nW/uSPa4pO105u7q60lq764d2u31zcyM3b25utNZ2Ns3cnE6nQRDMPDrz/stm3ypL+MLXfv78eeanZJfVJxdjAAAAAACwQd8UHWMAUH1BENiL0oIgSJLEPjQYDPr9vvtMrbVcyrYRcpWSTLvXXckovHEcJ0ky0yQAAJ6lbJu5Mmzjstuw+jfWbrfLfKlxxgdRRcyIsi2KqhxLI2R2y7TUG+j1enIziqKLiwv7KwuCoNVquQuG7/txHNtH3aUoJ77vp2lqx0SI49iWK8h7MQYAAAAAAC4GNQBQpDiObb1Ta4MVjBcOuZqm6fn5uRRuVUoZY9xiqgAAbMr2N3Nl2Ma9pA3z46nv6Da6DDPCxR7XPpv52iXbYadnkjqtVsuGDJRS/X4/iiIZnMLzvIUzeuPa7Xav17OjbNhhFPJejAEAAAAAgIsYAYAiaa1fPhjwc52fn3c6HdujGsexvcoKAIAN2v5mrgzbuJe0Yb72wHywYCeUYUa42OPaZ3Ecu+fakyTxfV+m539fxhj7M5RHbZ2JNE1PT09vbm7ybrDWWqIMkl2w7SlkMQYAAAAAYG8xqAGAIs1c8KSUSpJkWcHbmZqlUvLU3lz9NIPnee5VWZRCBQDkZPXN3PzGaL3NXBm2cS9pg+/7URTZm8aYwjfTOc2ItXdj1vOSPS61y0sjlFJJkti5nySJxDvkpta62+3aZ6Zp2uv1bObAGOP+Hp81wsgLl/BWqxVFUa/Xc+sfPGsxBgAAAAAAL/RqOp0W3QYAlZWm6dnZmTGm3W5LPdJutxtFkb2plOr1esYYuShKehjDMLTdlN1uN01TKb7q+750JsprjTHdble6p+WFdrTUXq8Xx7FbOzcMQ3dc1TiObX1drXUURa1WKwzDwWAgvaXuxU/9ft9eswUAgPXCzVzGNk6tu5nL2MbJo1vYzGW0YZVvzH4t9g3jOO50OkWVLl84I+Y/yOnpqTGm1WrJbMqeEcvedu1G5rrHldHatfe41LaWRkgFiCRJ4jiWU/udTsf9kqMokoeUUsYYdw7GcRxFkZ07SZJoreW8/pOzL2MJX3HWv3nzRmttF0KRvRgDAAAAAIANIkYAoBSkizMIgvl+QGNMmqYLH0rT1Biz7IUZ5IUzF8kBAJCTZZu5jG2cWnczV4Zt3AvbIFcY+75fkjPKOc2ItXdjXmK9PS61y0vjnpM5rrXOeI6duQuflv3oKm+78SU8YzEGAAAAAACbQowAAAAAAIAKWiVGAAAAAAAAMO910Q0AAAAAAAAbFkWRDB/geR4DRgAAAAAAgGehGgEAAAAAAAAAAAAAAHhENQIAOyx9eIhvb5VSQaPh12pFNwcAAADYU+nDgxmNfvf73x/92Z/5R0dBvV50iwAAAAAAwPqIEQDYMRIdMONxfHtrRiN7v390pI+P/VpNN5v0WgIAqqH76ZNfq7Xfvi26Idh3ZjzuDgZXHz8W3RCUhYQGzGj0y8T9vVKq9md/NvnDH+Q5kvSVSAHBAgAAAAAAdgsxAgA7YD46IKGB9rt3+vjYOzy0j0bX12ow8A4P9fFx0GgQKQAA7DQ3MAcUKL2/lxJQ2E/LQgPqa1ag/e6dzQrMPLl3eWnfh2ABAAAAAAC74tV0Oi26DQCwwLLoQNBoSNWBFV9FpAAAsLtOf/ghaDTCDx+Kbgj2XTwcnn7//fTHH4tuCLYhIzTgHx35tVrQaDwrB5BMJslkEg+H6f19Mpm4kRQbLJA9fMYpAwAAAACgJIgRACiR9aIDT76bGY2ks5JIAQBgtxAjQEkQI6iwVUIDcoJfHx9v6o9mBAt0s+nXarLfTrAAAAAAAICiECMAULDNRgcyxMNhPBy6kYKg0ZC/ssEuUQAANogYAUqCGEFlFBIaWAXBAgAAAAAASoUYAYACbC06sMxMpEAppZtNIgUAgLIhRoCSIEawu+LhUM7Qlyo0sAobLEju7qT99iGCBQAAAAAA5I0YAYAtKTw6sAyRAgBAaREjQEkQI9gVbmggmUySuzu5v/yhgVWY8Ti5u5P/54MFQaPhHRxIvTHv4KC4ZgIAAAAAUAWvi24AgCpbFh1ov3tXbHTA5cYFJFKQ3N1F19e9y0tFpAAAAABllR0aaJ2c7HRoYF5Qrwf1euvkxN7jBgvsEYdyxi8jWAAAAAAAwHqIEQDYsJ2IDizjxgXsR7CRgqDR0M1mUK/rZpOOSAAAAGzTvoUGVkGwAAAAAACAnBAjALABOx0dWEY6JWXajRTIgLJECgAAAJAfQgPrWRgskO/QjEaDn3+23yTBAgAAAAAAMhAjALCmSkYHliFSAAAAgPwsCw3Iqe7WyQmnutfm7skL99t2gwUS0QgaDSIaAAAAAAAQIwDwDHsVHVhmJlJgRiPpf5wZ+ED6HwttKQAAAMqI0ECx5vMBBAsAAAAAAJhBjADAE4gOZHiMFLx9q5RKJpN4OHQjBfJFBY0GXxQAAMDeIjRQfgQLAAAAAACYQYwAwAJEB9bg12rtt29nIgXxcBhdXysiBQAAAPvBjMfJ3Z38Lyeh5X5CA7tlJh+QPjxIETKZsKObqV8HC4JGY2YABQAAAAAAdhQxAgCPiA5sEJECAACAfZAdGtDNZuvkhNBABXgHB/r42M0WZAQLHmsVHB0F9br8X1CrAQAAAABYHzECYK8RHdgCN1LgfuFupMCv1XSzSQ8jAABAmREagEWwAAAAAABQba+m02nRbQCwVcuiA1wcv2XzM8I7PHycEUQKAABKKaVOf/ghaDTCDx+Kbgj2XTwcnn7//fTHH4tuyFZlhwYkK0BoABlmggXx7a19iGABAAAAAKDkiBEAe4HoQMkRKQAALESMACWxDzGCZaEBpZRuNv1azZeR7wkN4AWSyUSGPEvv75PJZGGwQA7QOEYDAAAAABSLGAFQfW/+7b8lOrBDbKTAXrHkHR6GrVb77duimwYA2DZiBCiJCscIzHh8/u///XxoQDKdnM1F3jKCBbrZ7P/2tyyBAAAAAIBCECMAqq/76ZNfqxEd2FHxcBgPh0G93jo5KbotAAAAVZNMJr3/8B8IDaA83GBB/7e/Lbo5AAAAAIA9RYwAAAAAAAAAAAAAAAA8+qboBgAAAAAAAAAAAAAAgLIgRgAAAAAAAAAAAAAAAB4RIwAAAAAAACiYMSZN022+EAAAAACAZYgRAJWVU0dSFEXdbvf8/NwYk8f7w8VMBIC9kqbp6enp+fn5Go8CG5EkSbfb7Xa7RTdkG/I77cq+FtbT7XYXLjNPLqvLXggAAAAAwNqIEQCVldNphna7HYah53lc77IFzEQA2Cue58VxnCTJGo8CG+H7fhiGe3I+Mr9QDvtaWE+r1fJ9f/7+J5fVZS8EAAAAAGBtr4tuAIC80GtZAcxEANg3QRAEQbDeowCehR0tlE273V54/5PL6rIXAgAAAACwNmIEQJHOzs583/d9fzAYqK+Xf3meZ58wGAwGg0Gapp7n+b7f6XTk0SRJzs/PgyAIw1CeFkWRvSmPGmNOT0/lfTzPu7i4kGl5VCl1dXUl76+U0lrbvqdut5skif2j8p7IkD0f85iJ6qn5yEwEgB3l7gY891Fgg6IoiuM4TdMVd2zyk8cO8wt3tBT7WnspiqLBYBCG4WAwsBU7Op2O1lqmjTFRFCVJ4nme53kzC6oxRoYLkWVGKeUuNnZUgjAM3bjYk8vqshfKAq+Usj+Bs7MziSP0+31pwPZ/zgAAAACAXTIFUCjP8/r9vkzf3Ny022370MXFhXvz5uam1Wq5r9Var35zhta60+mEYSg3r66u7ENfvnyx02EY2ua5Op2O+xIsm4+5zsTp8vnITASAHZW9cmbVje1Ye8dmy+1ZpUkZO1dr72hN2dfaV1dXV1prd5e73W7f3NxMp9ObmxuttV0wZm5Op9MgCGYenX//ZcvMk8vqwhd+/vzZ/XVMp1P311HUzxkAAAAAsCu+KTrGAOy7IAjsVU1BELgDHg8Gg36/7z5Tay3XQm2EXHEi0/YaGqWUDOMqoy/PNAnLLJuPec9EtWQ+MhMBYEeFYehulJ/1KLApBe7YPKs9W2gSO8yYIcuYTEvJgV6vp5SKouji4sJezR8EQavVchdF3/fjOLaPusttTnzfT9PUDogQx/FMuYJCfs4AAAAAgF3BoAZAecVxbGtXWq1Wa1Pvv3AEzTRNz8/PpQ6nUsoYwxjML5H3TFSL5iMzEQAA5GELOzbPxQ4ztmxmXkugRCZmRgRotVqSMBD9fj+KIhkOw/O8hYvWxrXb7V6vZ8f1cIdRKOHPGQAAAABQKsQIgPLSWrvDXm7H+fl5p9OxvWNxHNuLZrAGZiIAAKiMQnZssrGvhS2L49g9154kie/7Sil70b9ljLHBAnnUVrZI0/T09PTm5ibv1mqtJcog2QU36FDCnzMAAAAAoFQY1AAor5nrV5RSSZIsq5g6X39Siljam/MdWwt5njdT63LV5mIRZiIAAKiMZ+3YbMdL9rXW29FS7GvttyRJ7CKXJIlkSpRSWutut2uflqZpr9ezgQNjTBRF9tGZugVPWntZVUq1Wq0oinq93kz9gxL+nAEAAAAApfJqOp0W3QZgT6VpenZ2Zoxpt9tSXrLb7UZRZG8qpXq9njHGvcAlDEPb69TtdtM0lUKavu9L35B9rTGm2+1KF6e8Vga/7PV6cRy7xVfDMLTTg8EgjmNboFVrHUVRq9WylTCl/8u9nKXf70sL99OT8zGPmagy5yMzEQAArGd+x+b09NQY02q17J5kxo7NFtqzwR3m9Xa0FPtae0zKTiRJEsexnNrvdDp2zkZRJPcrpYwx7jITx3EURXZ5SJJEa23P6z+5zCxbVldc2N68eaO1dkc0EFv+OQMAAAAAdgsxAmAHSHdVEATzfTrGmDRNFz6klErT1Biz7LXLyKtmrrLCCzETAQBAZWTs2BRlvX2t9Xa0FPta+0oWM631sifYJWrhc7Ifzbb2svqkEv6cAQAAAABlQIwAAAAAAADgCU/GCAAAAAAAqIzXRTcAAAAAAACg1KIokhEEPM9jlAoAAAAAQOVRjQAAAAAAAAAAAAAAADyiGgEAlJcZj+Pb2//0+9//T//4Hwf1um42vYODohsFAAAAIBfJZGL/pff3f/iv/7X+T/6JUkofH9v/AQAAAADYAqoRAFUWD4en339/9d139DftkHg4jIdDMxqZ0Si9v1dK/fN/9s/+7y9fZDpoNOSfPj72a7WiGwsAALDbup8+9S4vpz/+WHRDsF9m4gKP03d37nN0s/n6m2/+r7/92/n7lVJBo6GIFwAAAAAAckM1AgAoWPrwEN/emvHYjEbx7a3cqZvN9rt3+vjY9gkmk4nEC+LhMLq+Vkr5R0cSJtDNZlCvF9V+AAAAAAutGBdQSrVOTtTyWMD8+wx+/jm5u+tdXs68D/ECAAAAAMBGUI0AqLL04eHP/9W/ohpBCSWTiRmNZMwCMxoppbzDQ318bCsN5PpyAAAACKoRYCNWjwts9jR/UX8XAAAAAFB5xAiAinv17bfECErCLScgXXsbKSewrJiB5AmY9QAAABmIEeBZduW0/a60EwAAAABQWsQIgIojRlAsWzAgHg7T+3ullBQMkHP8fq228b8YD4cSVjCjkf2LElPQzaZ3cLDxvwgAALC7Bj//fPY3f0OMADOqehq+qp8LAAAAALBxxAiAinv17bdhq9X5q78quiF7pDwn8rcfYgAAANgt8XB4+v33xAj21pOn1b3Dw6DR8A4O/KMj7+AgaDS8w8O1C4mVFvECAAAAAMCM10U3AEDu0oeHoptQccuGFWi/e1fssAJBvW67ON0hFaLra7WhIRUAAACA8jPjcXp/b0aj9OEhubtLHx6WxQVaJycVjgss49dqC0PG8/GCwc8/J3d3vctL+xziBQAAAABQSVQjACru1bffdt6/Dz98KLohVZNMJmY0spf7K6W8w0N9fGyv9S+6gVl2uvEAAAAbJNUIGAWsMubjArZCmNiT6gJ5o3oBAAAAAFQe1QgAYFXuBf3SRyYX9LdOTnbrgn652Kh1cqI+fHBLKXQHA3mCbjYlT0BPHwAAAEpo9bhA+9074gIbR/UCAAAAAKg8YgRAxXmHh0U3YbfZS/bj4VD6JeX8evBXfyUjAhTdwJeSwq2tkxO5GQ+HEpWIrq+lsy9oNCQkoZtN7+Cg0MYCAABgvxAX2C3ECwAAAACgMogRABUnvTB4Fnsq3fZRBo1G+927fTiV7lYgsBGK6Prafg924IMKRCgAAABQEsQFqo14AQAAAADsHGIEAKDcwv7x7a3cqZvN9rt3+1zYP6jXbc+sO6BDdH2tvg7o4NdquzWgAwAAgIvaXVtGXAAu4gUAAAAAUFqvptNp0W0AkKPTH34IGo3ww4eiG1I6yWRiRiN7wb1Syjs81MfH9mr7ohtYXnx1AACgSl59++3Vd9+xD7NZT8YF/KMjOYXsHR4SF8CK5uMFyWSS3N25zyFeAAAAAAAbQYwAqLjTH35QSl19/Fh0Q0rBvaReOpu4pP6FlhVykDwBHXYAAKD8iBG8RDwcKqVWjAvIxLIL0IG1ES8AAAAAgI0jRgBUHDECe9F8PBxKh6ZcNC/nuenB3Kx4OJSghu0+DhoNiWjoZtM7OCi6gQAAALNefftt/1/8i/bbt0U3pNQkLmBDA0opGyEVxAVQQsQLAAAAAGBtr4tuAABs3sKT2e137ziZnTe3AoENcETX1wQ4AABAmSWTSdFNKIvV4wKd9++JC6Dkli2c8/GCwc8/J3d3vctL+xziBQAAAAD2HDECAFWwrLR++907SusXJajX7TgR7nAS0fW1YjgJAACAQhEXwD4jXgAAAAAAT2JQA6Diup8+mdGokoMaJJOJGY3sJe9KKe/wUB8f2+vdi24gFmPGAQCAUnn17bed9+/DDx+KbkguGIwAeDkGRwAAAACwh4gRABUX/fRTMplUr1c0mUz+4l//a8VF7TtuvoxE+927/m9/W3S7AADAHjn94YfWyUn77duiG7Jh3U+f3EuoiQsAG/dkvKDCESUAAAAAlUeMAMCuin76SQIERTcEGyPXyXHVDgAAwMvJqFLEBYDts9kCjlgBAAAA7C5iBAAAAAAAAAAAAAAA4NE3RTcAAAAAAAAAAAAAAACUBTECAAAAAAAA4EWMMWmabv+1AAAAAJAHYgRARexPj0NOnzSKom63e35+bozJ4/3xJOYsAADIW5qmp6en5+fnRTckX+xWAYXodrsLfx2r/CSXvRYAAAAAikKMAKiIyneGWjl90na7HYah53n7E8goG+YsAADIm+d5cRwnSVJ0Q/LFbhVQiFar5fv+/P2r/CSXvRYAAAAAivK66AYA2Iz96cvbn0+6b5izAABgC4IgCIKg6Fbki90qoBDtdnvh/av8JJe9FgAAAACKQowA2HlJkkhl0dPTU7nH87yLiwv3OcaYKIqSJPE8z/M8uYoo11adnZ35vu/7/mAwUEr5vj/zRweDwWAwSNPU8zzf9zudjud58lmCIAjDUJ4TRZG9+eQnlScopa6uruT9lVJaa9sj0+12kySxf1TeFqtbb7aqr7NmvTnLbAUAABu0bDfYGNPtduUJck1w3jsV7DAD2xRF0WAwCMNwMBjY4QM6nY7W2j4n+8A5Yy1hhyQIw9ANKq1ytL7wtfLTVkrZX/fZ2ZnEEfr9vvz1ZQdfAAAAALAZUwCVoLVe9tDNzY3W+suXLwtv5sfzvH6/b/9ou922D11cXLg3b25uWq2WvTnzWbJvztNadzqdMAzl5tXVlX3I/dRhGNrmWZ1Ox30+5q09W6cvm7Mvma1T5iwAAPhq2V5BEAQzO8xbaAw7zMA2XV1daa3tAvzly5d2u31zcyM3nzxwfnItsezXscr6ZP61nz9/dlcC0+nUXQk8efAFAAAAAC/0TdExBgC5i6Lo4uLCXpcQBEGr1ZILj3IVBIG9qikIAncM2sFg0O/33WdqrTfYJLkUQ6bdi0tkJFcZEHemSVgRsxUAAOy0MAzdHQnL9/04jmU6CAJ3ryY/7FkBWyY/JZmWegO9Xk9uPnngvOW1hO/7aZraARHiOHbrHOS9igAAAAAABjUAqk/qMbr3tFot21dSiDiObR0kj20AACAASURBVFFHq9Vqber9F44rmabp+fm5/TaMMZUfFnfLmK0AAGB39fv9KIqkpL/neYWPU86eFZCHmUVacjN2OvvAeftriXa73ev17Agm7iAjea8iAAAAAIAYAVB9tlvEMsYUO2ii1npmPMgtOD8/73Q6ttsojmN7KQk2gtkKAAB2lOww26vz0zQ9PT29ubkpsEnsWQF5iOPYPdeeJInv+zKdfeBcyFpCay05BgkuuEfxhawiAAAAAOwVBjUAKkIKHtqb7rTWutvtug/1er1iL1OYL4eQJMnCiqnzVRkzPmk2z/NmikCu2lysZvXZqjY3Z5mtAADg5YwxURTZm8UmbgU7zEAekiSxv6wkSSQ6IzezD5xfspZY+yeplGq1WlEU9Xq9meIHzzr4AgAAAIA1vJpOp0W3AcAGGGO63a70+kmvhDtQYhRFcRzLZRbGmDAMcy1Pmqbp2dmZMabdbkvdxW63G0WRvamU6vV6xhhpkjQ4DEPpi+l2u2maSnlJ3/elx8S+MOOT9nq9OI7d4qvuJx0MBnEc2wKtWusoilqtVhiGg8FAuoTcizz6/b69MAXqxbNVrTtn156t8ihzFgAAPCmO4yiK7N5CkiRa61wrlrPDDGyfVNdIkkSOjtM07XQ67jKcceCcsZZ48teR8ZNc5Zf15s0brbU7ooHIPvgCAAAAgBciRgBUR5qmxhilVBAE830H9lGtdQGNW0L6ceYbbIxJ03ThB1FPfdIM8sKZq6ywcctmq8pnzjJbAQDARrDDrNizQqXJryn7B569Hlh7LbH2T/JJGQdfAAAAAPASxAgAAAAAAABQcavECAAAAAAA4nXRDQAAAAAAAAByFEWRDB/geR7jcQAAAADAk6hGAAAAAAAAAAAAAAAAHn1TdAMAAAAAAAAAAAAAAEBZMKgBgJ1kxmOlVFCvF90QbED68BDf3prx2IxG/+W//bd/9A//YdBoBPV60Gj4tVrRrQMAANhJyWSSPjywwwzkKplMksnEnUjv7+3E/3t//82rV3JQ4x0c+EdHMhE0Gkop7/CQXygAAACA0mJQAwA76fSHH5RSVx8/Ft0QrCkeDs1oZEYjMx4nd3dKKe/wUB8f/8kf//Hf/f3fx8Nhen+vlPKPjoJ63T860sfH+vi46FYDAADsjO6nT2Y0YocZWJsZj+WoZD4ikEwmchQzwz86ktCAX6t5h4f/33//7//g9WullBmN5OUyMcM7PHwMFnyNGiil7OEPx0EAAAAACkE1AgDANiSTiYQGzGgU397KnUGj0To58Ws1fXw8U3ggmUzi4VBe1bu87F1eyvN1s+nXalKuoICPAQAAAGD32YiAGY3ShwelVHJ3JxNmNJKHZgSNhnd4qJSS4xGZsKGBZ9VRSx8efgkWjMePTRqN5H45XJIjIJduNm1LHu/5mjAIGg3v4GD1BgAAAADAk6hGAGAncXHVTlhWcmCN6gIL3ypoNIJGQx8f02sGAAAwgx1m7K14OJyZsDUAnowI2DP0Qb1u7ynqWKMyHwQAAADALqIaAQBgYxaWHNDNZuvkJKjXg0bjWdfouNzYQfrwEN/eyl+xhQokmuDXarrZpFABAAAAUD32In616My6PQCZ4V7ELyfXd+Ui/lXGNVhWVkFC2AujBjMjL6gXlFUAAAAAUGHECAAALxIPh/FwmNzdxcOh9FL5R0dBvR62WlIqYON/0Ts4aJ2ctE5O5KbkCeRfdH2tBgOllG42ZeCDl2QXAAAAAGzHsjr/j/d8zQq4pD6ZTM9HBPI4EikhG6HO+LzJZJJMJu5Een8v08lwKMXeZtiogXdw4B8dyYR8w97hIbltAAAAYB8QIwCwq5Zda4K8JZNJPBwmk0l8e2u783Sz2X73rpDT9kG9HtTr6u1b9bXzUWIN0fW1G2tYYyQFAAAAAC+37DS2WiEiIGevdbPJaey1rVJjwJ01bozjsRTcU/NIogZq/2IcAAAAQIW9mk6nRbcBAJ6t++lT7/Jy+uOPRTdkXywrOSAX/ZS2h8gmHtxBFqQX0q/VpFxBsS0EAADISffTJzMaXX38WHRDUHG2qP58RCCZTLKvdJ8vqk9EoMwyKkao1QaVeLxnRwaVAAAAAPYc1QgAAAvISAHLSg7IxUCFNnAlfq3WfvvW3oyHQxn7YPDzz9KhKRfQSBKCPiwAAADAZSMCclW6Uiq5u5MJMxrJQzOCRsMmA+S0sY0IrHJNPMrMOziwCQA7xty8eDicmZAjSjkWS+/ve5eXMy+xi80vo1TU6/YeDtMAAACAQhAjAAAo5QwHYDt3lFIyEEDr5EQ3mxW4Ksgd1OCxOOd4bEaj3uWl9GTJ5/VrtWp8XgAAAGCZZed61QoRAUniKs71YpFVxjVYllBxj0ZnZBSxIKECAAAA5IEYAYCdRB/BRshJdPtP7ty5kgPr8Q4OWicn9hoa96uIrq/VYKCU0s2mDHwQNBoscgAAANgJtvK8WhQRWKXyvEQEqDyP/NjQdkbUwA6TMTNeRjKZJMNh9ngZ3sGBf3QkE7I8M14GAAAA8FzECADsJM7prmdhyQHpKKxMyYH1BPV6UK+rt2+V8y0ld3fR9bUtzBDU61KuIKOrCwAAAMhPxuD06f29zQq4ZBgvmZ6PCLBni9JapcaAmzBwfxGPxecyfxE2aqD4RQAAAACLvJpOp0W3AQCeLR4OT7//fvrjj0U3ZAeY8Ti+vU0mE1tyQPpNgkZDHx9zadGTkskkHg7lC7QXbwWNhm42ZcDXvc1eAACAMut++mRGo6uPH4tuCFa17NprtUJEgGuvgWUywjdqtfocj/dQnwMAAAB7hhgBgJ0kMYLP/+7fUZZg3uOFF+PxzGnvX/7Rn/gCtpaDGY+lkCaxDAAAUELECErFjgQ/HxFIJpPs8uzzI8ETEQA2zo4AMj8UiK3kNyNoNOS3+UvBj3rd3sOBIQAAAHYdgxoA2GHJZEKMQNiSA/HXQSLl3Hbn/XvObW+WO6iBm9joXV72Li+VUjLwgV+r7fMgEQAAAPvDRgSklLpSKrm7k4knzz5KdSvlRARWKeQOYONWGddg2Y/dHTRwRkYeiB87AAAASo4YAQDspGUlB1onJ1Ta3xrv4KB1ctI6OZGbMjvkX3R9rQYDpZRuNmV2BI0GnUQAAAC75YUXKEtEgAuUgWqwR9kZUYNlo5Mkk0nyNfQ/w0YNGJ0EAAAApUKMAAB2xsJy+vr4WEoOZHRkYDuCej2o19Xbt+rrAJxSHCK6vpYuZv/oKKjXpVwB8wsAAKBAdrh0tSgisMpw6XKSj+HSAbhWqTHgJgzMeCx3SoWD+PbWrohcUm5QOVEDtVoFBQAAAGBtr6bTadFtAIBni4fD0++/v/ruu2ofLSeTiYQGZkoO6GbTr9Wkcn6xLcSKHsebmEwWzkqqRwAAgI3rfvpkRqOrjx+LbkgBbERg5hTd4z2Zp+iUM8w5p+gAFCVjPaZWizo93kPUCQAAAOsiRgBgV7369ttKxgjMeCzXH8yUHOAS9ipZWFhCrmljLgMAgI2ocIxAhiefKRiuVogIUDAcQMW8cOCVx3vqde/wcJU6CgAAANgrxAgA7Krup0/tt2+rd5Tb/fSpd3mpm025Qj1oNKr3GeF6LFz5teZE0Gjc/Jt/U3SjAADAzpNKSO23b4tuyObJDrNM2zHF/VpNTozZM2FEBABAfY1eqa/jJiilkrs7mXCjBp3378MPHwpsJwAAAMqGGAEAlEsymZAb2GcsAAAAANmk9gC7TACwKaxXAQAAMI8YAQAAAAAAAAAAAAAAePRN0Q0AAAAAAAAAAAAAAABlQYwAAAAAAICqMcakaVp0KwAAZbT2NoKNCwAAwP74o7/+678uug0AsFgcx1EUeZ73m9/8pui25C5N0z/90z/d+NtGUTQYDAaDwW9+85t9+Bp3FwsAAADYrN/+9re+7/u+X3RDNian/SXFLhOA/bNwG7HKarZ6GxcAAAAsQzUCAOWltdZa70nO/fz8PI+3bbfbYRh6nrcnX+PuYgEAAACb1Wq1KnaaJ6f9JcUuE4D9s3AbscpqtnobFwAAACzzuugGAACUUoouyz3HAgAAADar3W4X3YQNY38JADZl4TZildVs9TYuAAAAWIYYAYAd0O12jTFKKc/zOp1OEARyv5QeTdPU8zzf9zudjud5ubbk7OxMyvcNBgOllO/7ct2SfcLCJiVJcn5+HgRBGIbynCiK7E151Bhzenoqb+J53sXFhX1PeYJS6urqSt5fKaW1lqP3brebJIn9i/KeyMl6C4D6OhMXLgMvXAAUywAAABUllfbV133gwWCQJInsJBhjoihKksTzPM/zZnZI7M5zGIZ2z1k8uTOT8UdXxP4SALyErIfDMBwMBrIyV0p1Oh2ttX1O9lbAGNPtdpVSssZTSrkrvYXbiCdXs8teqL6urpVSdgV+dnYmiYR+vy8N2H7vDQAAADZgCgAldnV1pbW+urqSm1++fNFaf/78eTqdXlxctNtt+8ybm5tWq7WFJnme1+/37R9125DdJK21+z7ZN+dprTudThiGctP9TuxzwjC0bXN1Oh37fLzQ2gvANHOmr70ATFdYBlgAAADYORcXF51Ox97sdDp2N+Pm5kZrbXcAZm66L1m4A5C9M7Psjz7LLu4vTdllAlAa8z0h7Xb75uZGbj65FQiCYObR+T+xcI335Gp22Qs/f/48s71w1+1F9d4AAADghb4pOsYAAE8IgsCG7qVHstfrKaUGg0G/3595mlx4lHd77FVNQRAkSWIfyrtJktmXafc7SdM0juMkSWbagzyUbQFQLAMAAFRRFEXuxaOdTsdu4qMouri4sNdxBkHQarVW3+XI2JnJ+KPPwv4SALzQTE9IGIbSE6JW2Ar4vh/HsX3UXevmxPf9NE3tmAhxHM+UKyik9wYAAAAvxKAGAMpuphCr7/vS6xfHsa22Z7Vare21bE7eTZofgzBN0/PzcyljqJQyxsx8Xdim7S8AimUAAID9YAtTy/RMLehWq2VPL+X0RzeF/SUAWMXMakqyUHY6eyvQ7/ejKJLBXDzPW7hi3Lh2u93r9eyoNG4orYS9NwAAAFgFMQIAZWeMcQ8vZSw9pZTW+lmjtG7B9pt0fn7e6XRs/0Icx/aaA2xfIcskywAAAJU0c4ooSRJ7Ab09k2QZYzYyyHTGH90U9pcAYBVxHLs9IUmS2FxX9lZAHrV1WdI0PT09vbm5ybvBWmuJMkh2wd2glLD3BgAAAKtgUAMAZRfHsXuQ3O12JUo/f9FVHh2dz7J6k+bL90kNQHtzvl9gIc/zZkoFPqO52LRnLZMzM2u9BUCxDAAAUFHtdvv8/Fx2CdI07Xa7douvte52u/aZaZr2er2NXNaZ8Uc3hf0lAFhFkiR2bZkkicSh5Gb2VsAYE0WRffRZIbO1V7Oi1WpFUdTr9WbqH5Sw9wYAAACreDWdTotuAwAs1uv1BoNBp9MZDAaSu0+SRGttj0h7vZ4xRh6S49swDDdyJdZCaZqenZ0ZY9rtthTo63a7URTZm9lN6na7UkohTVPf9+XQ2r7QGGM7auWFduzAXq8Xx7FbfDUMQ5keDAZxHNvqrFrrKIparZYtJCjdB+7VAP1+f+PFaffECxcAlbkMrLcAqMxlgAUAAICdFsdxFEVyUqfT6URRZHc5oiiK41i26caYmX2DZTsAq+zMZPzRVezc/lL2N/ac2QUAGyMVU5IkkVW9rJDdlVLGVkBW43ZtNtOLkr3Gy1jNrriqfPPmjdZ6fsOx5d4bAAAAbAQxAgC7wRiTpmkQBPPHmXKAvfChoixrUsanUEqlaWqMWfjCDPKqmUusUKyMZTJjGVhvAVAsAwAA7Ifz83N7Okc5ew5a66390Q1ifwkAlpE1ZPbqPXsrsPY2Yu3V7CpK2HsDAACADMQIAAAAAAAotTiO4zh+VmGAHf2jAIBVYgQAAABA3l4X3QAAAAAAADBLBgiQ6SAItnM6v5A/CgCwoiiS4QM8z2OAFQAAABSIagQAAAAAAAAAAAAAAOAR1QgAACiLZDJRSvm1WtENAQAAAADshfThwYxGSqmg0fAODopuDgAAAMqCGAEAlEj0009+raaPj4tuCLbEjMfJ3Z0Zj81oZEaj9P5e7tfNpvTgBI0GywMAAIAVD4fxcBh++FB0QwCgvMx4bI8uzWiUPjzYafuc+PY24x2CRsM7PLTT9n57fOodHgb1+sZaDAAAgPJhUAMAKJHTH34IGg16RSssHg6TySSZTMxoZHttvMPDoNEIGo2gXv/j16//4+fPki1I7u7kCUGj4ddq/tGRPj7mAhEAALDPup8+9S4vpz/+WHRDAGB7bMEApVR6f2/GYzstNe2UUslkYg8h58lRp0z7tZqNCPi1mtTD+y9/93f/6E/+5OVv7h0c+EdHM28+Mw0AAICdQDUCACgX9+IA7Drp64mHw/T+XuoNyP3+0VFQr3fev9fHx/OdKf/7X/6lnY6HQzMaSfKgd3nZu7xUv44dyFtt8TMBAAAAAF7KLRgQD4e/3L9ywQDdbNppOUJ8vP9lBQNaJyfZT5DjU9taW+ogubtLHx4ej4JfVupgZhoAAACFIEYAAMDGSJkBO1SBW04gaDRaJyfPHaRAHx+7z5fhD+SvRNfXDIIAAAAAAGXgFgxwT7S71/S7I9nN84+O3Gv37SX+Qb3unnQvvDqdG4Vf5fDThiTcUgfqa2BCvhOZltz8PBuYcEsdyPHvfJMAAACwKcQIAABYn3te3+0S0s1m6+TEr9WkZsCm/lxQr7vvJp1T8XCY3N0Nfv6ZQRAAAEDlsWMDYGueLBhgz38vs7BggHv+e72CAbvFjRo8q9TBr6adUgerpzHcUgdlS2MAAACU36vpdFp0GwAAj05/+EEpdfXxY9ENwVIyysBMnUZ3lAE5hV9s8x7jBYuaxyAIAABgp8XD4en3309//LHohgDYPXkUDJBpTlEXxY13bCTqIRP7FvUAAABYiGoEAFAifq1mOy9QBo9xgeEwubuTkgNyv2QFOu/fl+1yfwZBAAAAALA/NngWWQrm72fBgN3lHs8+eWz7ZI7kWaUO3ByJO6pCqboIAAAAXoIYAQCUiHd4qIgRFOpxeIKvZ9/tMAFy3r11crJb590ZBAEAAFRSMpkwDDZQSdk17WX6uTXtJSjgFgzYoWM6bJB3cPCsWZ89qkUymSinCuBCQaPhlqmw99tmEFIBAABlxqAGAFAi3U+fzGjEoAbb5F6vb3uj7CgAfq0mYwEU3cy8MAgCAADYLTKowdV333EWENgVGy8YYKfteVn3WnCgEG6pg/T+3ozHdvpXyZivaf55ciQu08tKHbCoAwCAbaIaAQCUC4Ma5G3hiXO5YKX97l1Qr0t6oNhGbg2DIAAAAAB4FgoGAPNmSh20Tk6yn++WOjCjkf3tyO9oI6UOFL8jAADwMsQIAKBcMpLpWINcECBl/M147JbxDxqNzvv3lPF3MQgCAAAAsIdskQD3KmrlFAxY/SpqKRggN7mKGljGPfRe5WR/9o/UHZaxd3k5//L5H6mdpqoHAABYhhgBAKBSHocnGI/dQQqUUrrZbJ2ccD39s0gngvt1ubUcepeX0j3BIAgAAABAqbhFAuYvdJbpZ13obE80cqEzUIhnlTrILhkil1s8t2TI4zQlQwAA2CfECAAAu82tw287wuTEdvvdO79Wk9PbxTayMhgEAQAAFMs9mQHsFQoGAFjRc3/LdvUyM21LHaT393ZVs5CMSyLcVc0v04eH9MwAALBziBEAAHaJTc0/Vh34ehwrF8F33r8P6nUpuV9oM/cFgyAAAIAtY0cClZE9MrpMUzAAwBY8a6WxrNRBen8v088qdeDXar+sx5xSB3QdAABQBq+m02nRbQAAPIqHw9Pvv5/++GPRDSkR98y0GY/dM9NyclomOLwsJ3cQhJlaEQyCAAAA1vPq22+vvvuOk6MoG0k8P047BQPsqTX1nIIB7qk198Jijn0A7BY3MjVf6kDJCjOz1MFMZMreb/cEKHUAAEBOiBEAQIkQI1BKSVZAquW7AXapky89aPQa7yh3EIT5mcsgCAAAYBWvvv228/59+OFD0Q1B9S0rGOCe8XpWwQB7P2e/AGAeeSwAAEqFQQ0AAAWLh0O5Wt2MRjMXrLffveOC9SphEAQAAAAUKI8TVHKTE1QA8HLewYF7aUHr5CT7+dlhr2QyUU5lxIWeDHspRocBAOwxqhEAQIlINYJq12iVnrt4OJSydbYXT7ICcubY7YPDXmEQBAAA8CSqEcD1ZLls9VTBAN1s2mkKBgBAhdnNhJskU84mY/UkmXdw4B8dybTbi0WPFgCgSqhGAADYhuinn+LbWxmwQO6RE8OtkxPq2MPSx8fuwuAOghBdX88MgtB++5aDcwAAgGpLJpPop5/UrwsGuMNjzfOPjtzTOfasT1Cvu5ecUjAAAPbNs0odyBUOMu2WOkju7tKHB7lIZvXtkc0fkIMEAOwQqhEAQInIQUglu7TO/uZv0ocHKVYvl5UX3SLsHncQBDMeX/zLf8mCBADAHoqHQy712x9SsG1hwQDv4OCXaQoGAAAK5VbEWVgdR6pyTn/8ccsNAwBgbcQIAAAAAAAAAAAAAADAo2+KbgAAAAAAAAAAAAAAACgLYgQAAAAAAAAAAAAAAOARMQIAAAAAAAAUzxiTpun2XwsAAAAAmEGMAAC2ZGGXVpIk3W632+1uvz3bkVNHXhRF3W73/PzcGJPH+6PMWKgAANgTnBLeQ91ud+HO2CoLw7LXAgBQOLoyAAC7iBgBAGzJ+fn5/J2+74dhWOF9/YWf+uXa7XYYhp7n0bm8h1ioAADYEzlt9FFmrVbL9/35+1dZGJa9FgCAwtGVAQDYRa+LbgAA7Iv93KHfz0+NXLFQAQCwJ9jo76F2u73w/lUWhmWvBQCgcOzVAAB2ETECAMhdkiRSXuz09FTu8Tzv4uLCfU4URXEcp2kq9Qk8z7MPDQaDwWCQpqnneb7vdzod99GNODs7833f9/3BYKC+1kiwf2VZA+RzBUEQhqE8LYoie/PJTy1PUEpdXV3Jn1BKaa2l+6/b7SZJYv+ovCd2RfYSpfJZqLKXKMVCBQDA7lhl/9kYE0VRkiSe53meN7OzgTxEUTQYDMIwHAwGtqBap9PRWtvnZMwXY4yM5iY7Y0opd3/MDkkQhmEQBPb+VRaGha+VPUmllN2ZPDs7k7M4/X5fGrCFQy0AwA6hfwwAgF+ZAgC2Qmu97CHP8/r9vkzf3Ny022370MXFhXvz5uam1Wrl0bxlbXiyATOfK/vmPK11p9MJw1BuXl1dycSXL1/sc8IwtG1zdTod+3yUzUuW6pcsVMuWqCkLFQAAuyZjo39zc6O1thv3mZvIz9XVldba3Wlvt9s3NzdyM3u+BEEw89D8+y/bGXvysGLhaz9//uzudk6nU3e3c2uHWgCAHUL/GAAA1jdFxxgAACoIAnvBdBAESZLYhwaDQb/fd5+ptZZg8nbasIUGSIJbpu2VTDKuWxzHSZLMfCfYCQUu1QuXKMVCBQBAhURRdHFxYa8ODIKg1WrlsZOMebLzJtNSb6DX68nN7Pni+34cx/Yhd4cwJ77vp2lq60jHcezWOdjaoRYAYIfQPwYAgMWgBgBQanEc26JnVqvVqlID5gcxTdP0/Pxc6qAqpYwxbn8fdl3eC9XCYXFZqAAAqBK7TbdarZY9mY1czexEyekNO50xX/r9fhRFUpzZ87yF+2wb1263e72erTLt1oIu/FALALBD6B8DAOwhYgQAUGpa65mBP/ehAefn551Oxx4axXFsr1tCBbBQAQCAF7LnrS1jDKPab0ccx+5ZkyRJfN+X6Yz5Ig/ZiyzTND09Pb25ucm7tVpryTFIdsFdSAo/1AIA7BC6MgAAe4hBDQBgS6Sipr0538W20PxlVUmSbLOC2bMaMF/Mbb1P7XneTLnRVZuLXcBCBQAAVpGx0ddad7td96Fer8d15NuRJIndl0uSRM5wyM2M+WKMiaLIPvTczMd6e4Ci1WpFUdTr9Wau8iz8UAsAsEPoygAA7KFX0+m06DYAwF4wxnS7Xdn7l6OFfr+fpunZ2Zkxpt1uS4HN09NTY0yr1bIjrvV6PWOMXOIjLwzDcIPXWs23odvtRlFkb2Y3oNvtpmkqtUx935fuOVssdOGntp8rjmO3IFsYhjI9GAziOLYV27TWURS1Wi1bjFS6IN0rivr9vr0KCsV6colS+SxUGUuUYqECAGAHZexJKqWiKIrjWDbWxhh3u4/8yEWQSZLIl5+maafTcXeZls2XOI6jKLI7WkmSaK3tef0nd8YyFoZVduTevHmjtXZHNBB5H2oBAHYI/WMAAMwgRgAA25OmqTFGKRUEwXM7p6TDbo0XbkpGA4wxaZoua9t6n1peNRO7RsWwUAEAgGzZG337qNa6gMbtJdl/y/7CM+bLS2bZSw6mshV+qAUA2CF0ZQAA9gcxAgAAAAAAADxtlRgBAAAAAKACXhfdAAAAAAAAAJRdFEVSPNnzPGomAwAAAEC1UY0AAAAAAAAAAAAAAAA8ohoBAADYMfFw+H+Mx39Zr+vj46LbAgAAgFykDw9mNJK9vqDR8A4Oim4RAADPY8bj9P7+cXo0+ts//OEfvH7tdmXQrQEAKDOqEQAActf99Mmv1dpv3xbdEOwqMx6b0SiZTOLbWzMaKaX+19/85v/8z/9ZKRU0GrrZ9Gu1oNEI6vWCGwoAAIB1SW4gHg6TuzszHid3d8rZ6/OPjoJ63T860sfHpAoAAIWYiQWkDw8yndzd2WkzGtnnzGjUaqPJZNmb62bTTgeNxi/3EzsAABSEGAEAIHenP/wQNBrhhw9FNwQ7w3Yim9HIHoFLr7Ffq+lmM6jXzXgc394mk4n0NSulvMPDoNEIGg06lwEAAMpvYW5AKaWbzaDRsDlRmyg1o1F8eyvPkVSB3fcr7kMAAHbYC2MB6mtHxOP0wYF/dGSnl0UBlFLJXzu58AAAIABJREFUZJJ8zRO40+n9/a/u/7plXGhZ7CCo173DQ3s/fSMAgLURIwAA5I4YAVYhHcT2n9wpnchBva6bzYxD3/ThIb69lXewncvSrfz4j0IFAAAARVsxN5D9JvN1qtTXHT95B1IFALC3thML8A4Pt9zJsJHYQdBouPGCX+4ndgAAWIIYAQAgd8QIsNDCkgO2C1hKDqz3zrZQgU0kUKgAAACgEHZnb+3cwCrv7+74KVIFAFAV8XC4cNqu8JVS9lqChcoZC8ib9Lc8Tt/fm/HYPvSr+52vcZ4bO/BrNXfar9XmpwEA1UOMAACQO2IEsLZ/gj+/sAIAAADmZZzX30KZKFIFAFBmxALKaZXYgXpq1vhHR268gNgBAFQAMQIAQO6IEeyzsg038JKhEwAAADCvzGful7VNN5sbqYUAAHvu5bEAzj3vKBIhALAPiBEAAHJHjGDf2JIDMvCtKuuYAgsLFfhHR/r4mEIFAAAAy5Q5N5At7xEWAGCnuZekK2IB2KhVYge2Z2YhYgcAsH3ECAAAuSNGUHnLSg7s1mVe0v5kMolvbylUAAAAYO1ubiCDTZQmd3ekCgBU0iqV6tP7e/c87jxiASiEGY9tpMCMRunDg0wnd3d2Ojt2oJTSzaadtlEDpZS737KL+zAAsDXECAAAuSNGUEnz13J5h4f6+Fiu5q/AYVg8HEq3cjwc2kIFQb1uyyoU3UAAAIC8VDI3kI1UAYDy20gsIGg03CgAsQBUw7LYQXp/n0wmMp1MJnb7vtAqsYPylNgEgC0gRgAAyB0xgmp47EdeUnJABgIotoX5eRygYUmhAulZLrSBAAAAL7KHuYFsC1MFdqAu9gABbMTGYwHuic+gXnfv58Qn4Eomk1/FC75OPyt2wK8PQOURIwAA5I4Ywe6qfMmB9VCoAAAA7DpyA8/ijuFlSyiTKgAwY9mJSUUsANhxW/t1UwsEQKkQIwAA5I4YwQ5ZWHKAy+4z8I0BAICdILsrsuviVpYiN7AGdw+QVAFQYVyvDGA9G6k14h8duUONMAQJgO0jRgAAyB0xgpLj2voNon4DAAAoiYW5AdnNkz0TzlptypOpAt1s8lUDZbCRWMCy0dOJBQBYmxs7UErFw6Gd/tX9X/foFloWO/AODuzKyjs8DOr1jbQZwD4gRgAAyB0xgrJJJpN4OEwmk/j21h6NcAH9xi0sVBA0GrrZ9Gs1fXzM9wwAADaI3EBJLEwVMCOAPOQaC3BT4CTCAZSNGzVYL3YgkcfH6YMD/+jIThM7ACCIEQAAckeMoAwoOVA4ChUAAIDNIjewE5hNwOrMeCyHq0opMxqlDw8yndzd/TJNLAAAnm8marBwBWvjjwstix0oVrBAdREjAADkjhhBIWx/5cKSAxRWLVb68BDf3i4rVCDzqNgWAgCAEuKEdAUwE7FXVokFZJ+1UsQCAKAIrMABKGIEAIAtIEawHTKOmr3k3ZYckOL5utnkzHRpmfE4/v/Zu3vYyNE8z/NP1RTuUKndu+PMSoPNwa4CbOziTjrgDGbgjDMygaWcTMxYxXBzy2EAdWuM0d2kt9UeYzvN7gKCTqMxxgGi3ExHXCDTaGMhEXcHnOIWN0hCGmAWOxk3SQx2JIynNf6ZT7EjFIxXBl/i+4GMCDFenod8IvgE+ePzXF19mmzi80AFeqAIDiUDALCzOOXcegWbmGHDUEPrn1VaZAxtxVklAGiaUmeZsQ4Pjb09/X96v8DWECMAAJSOGEF59JSr8if/ZMiBRps1UMGPf8RBAABoL3IDO+7BEcWkEyhjVnFuFZu1nVgAU2sDACZsJHZgdTr5eMGP/yd2AGwIMQIAQOmIEZTBPzsL376Vozn6wCJDDrSPHqhAJ0WMvT332TM+UAAAtIx/djZ4/VpukxuAkJHG8l1BpZTV6Zx+9525v19p0dA8/tmZtCIdUXoQsQAAQA3JIKyfbt/eJjc3etHv/f/z7Qfp2EHgOOzIgEUQIwAAoJHCd+/S8ZgjyztFz1th7u+7T59WXRwAALBJ/tmZUoreHQroVMHw5cuqy4Lm8c/O0g8fiAUAAHZBceyAGAGwIGIEAAAAAAAAAAAAAADgky+rLgAAAAAAAAAAAAAAAKgLYgQAAAAAAAAAAAAAAOATYgQAAAAAAABATSVJkmXZNp8IAAAAAH/w/fffV10GAECTJEny53/+52EY/sVf/MU//MM/WJaVX5qmaRRFg8HAMAzTNBd/2SzLvv76600Xtp1KWldhGEZRFEXR48ePHz9+vPHXx0aw9QEAaB/27yj28uVL0zSnf17NbTmznoidwjcMAACCfSKwLEYjAAAsIcsy3/eHw+H5+fn5+XmWZVEU5ZeuECAQ/X5/oyVts5LWleu6QRAYhsEVS3XG1gcAoH3Yv6OY4zgP/sKa23JmPRE7hW8YAAAE+0RgWV9VXQAAQJOkaep5nmEYctfzvF6v5ziO3DUMYzgcKqWSJFn2lelmLY51tcvY+gAAtA/7dxRzXffB/89tObOeiJ3CNwwAAIJ9IrAsYgQAgCVMTGGQJMlSV7ckSeL7vlJKj1gQBEGapv1+P0mSk5MTeZhhGKenp5srdQV6vZ4MHyqjNZimKaFU/QAZ6irLMlkVOpwha8OyrCAI5GFhGOq7xetKliqlzs/P5fWVUrZt66OHvu+naarfVF4TG8fWBwCglYp38WXs39W8XTz79zqQYWyDIIiiSMepPc+zbVtuJ0kShmGapoZhGIYx0TN88CeSXur7vrxmEAT532JzW86sJ0oLVErpRtjr9eSQ+nA4lALMaswoFd8wAAAojqoBdXMPAMBKPn786Lrux48fpxd5nnd+fj79f8uy9OMvLy9t29aL8rfbQcZmkNuXl5eu6+pFp6en+buXl5eO4+SfO7E2iu9OsG3b87wgCORufkPkN1YQBLp4ebO2HZbC1gcAoJVm7eJL3b/fz97Fs3+vifPzc9u289vFdd3Ly8v7z796Jn4E5TdcwU8kbdZGnNtyHnzi+/fv8831/v4+31znNmaUh28YAADuOaoG1MmXVccYAACNlGVZr9dzXXepC1NM04zjWG5bliUzILSVZVk6smpZVpqmelEURfm6W5Zl27YEXTdCorhyW18FpZSSObriOE7TdKJI2Cy2PgAArTRrF1/2/l3N2MWzf68P2ehyW4YcGAwGSqkwDE9PT/WPJsuyHMfJt43t/0QyTTPLMj2obxzHE8MVlN2YMQvfMAAAKI6qAXXCpAYAgKVlWeb7/sTwmIsYDodhGMowUIZh7OxUnXEc6xG0NMdxNvX6D67YLMv6/b6MpKqUSpJk2c2HjWDrAwDQPmXv39VDu3j277UysfLlWLPcmAheO44jCQNRyU8k13UHg4Ee8jc/MO8WGjOWxTcMAACCo2rAlhEjAAAsJ01TOeS07ASZchxNJzqzLDs5Obm8vNx8EWvPtu38hJTb0e/3Pc/T3dw4jvVlT9gmtj4AAO3D/h1xHOcPYadpapqm+vwjKC9JEv1LqqqfSLZtS5RBsgv5X3aVNGYU4xsGAADBPhHYMiY1AAAsIUmSfr+fzxDkr6SZ+9wwDPXdiRSCDK2p704fbmuTiSuQlFJpms4aDmt6YK7V1pVhGBOjlS5aXGwUWx8AgPZh/w4JW+vbcrhZKWXbtu/7+mFZlg0GAx04KP6JNNc6v6EcxwnDcDAYTFx1t1RjxnbwDQMAgGCfCGzZF/f391WXAQDQGCcnJ1mWTRzeOj8/lxtZlvV6PZW7qMU0TT1hVRzHYRjqi13SNLVtWx+0SpLE933pk0kfbgvTgpZE1kOSJK7rygChvu+HYajvKqUGg0GSJPlLlPLhDN/3ZT1nWWaaphzd08+dta4Gg0Ecx/mRtfITT0RRFMexHn3Ltu0wDB3H0WOZyhHM/AVJw+FQSojFsfUBAGilubv4MvbvqnAXz/69PuSKtDRN4ziWw9Oe5+lVHYah/F8plSRJvpNW/BNp7kac1XIW3PpPnjyxbTs/o4EobswoA98wAAAojqoB9UOMAACwVVmWJUmilLJte9Yiy7J25CiVHHB8sL5JkmRZNmtVrLau5FkTEVpUha0PAED7sH/fTbLdp3/gaAU/guYuLVbeb6iCxoyq8A0DAIBgnwhsBzECAAAAAAAAYEVzYwQAAAAA0DhfVV0AAAAAAAAAoJHCMJSRbA3DYABbAAAAAK3BaAQAAAAAAAAAAAAAAOATRiMAAKB54tHo/7y5+e8fPXKfPq26LNiqdDyOLi7+n7/+6//5T/7E6XbN/f2qSwQAAIAtye7u4qur5Obmb/7u7/7Hf/pP7eNj6/Cw6kIBAADUXToep+Nxcn2d3d39dZb9iWEYjx5ZnY65v8+xNaAAoxEAANAk8Wg0ePMmvrr6nx4//n//038yDw68588JE7RedncXXVzEV1fRxYVSSra+UsrqdJxulzwBAAAt8MW3357/7Gf20VHVBUG96OhAfHWVXF8rpYy9PXN/X9+2j46sTodIAYp98e233osXwTffVF0QAADKld3dJdfXP+YGbm+l1yTMg4ODf/yPP/yX/5J++KD/aXU6xt6eThVYnY7x6FEFRQfqhxgBAADNoAME9vGx9/y5fXSk/0OYoK0m0gP50IAMSxBdXMhvIfIEAAA0HTECaA9GB6bjAvFoFI9GyfV1fHU16zGAIEYAAGileDSSxED64YP0oPQi8+BAMgFKKfvoyNjbm+ggJTc32e1tPBoppT4lD3LZAvv42Hj0SF7E3N+nl47dRIwAAIC6mw4QPLiUMEFrFKQHph9MngAAgHYgRrDjFowOzDIrUmB1OjQqKKWe/OIX9vExMQIAQHPFo1F2eyvn/vVIA7JIjyUgWQHpCNX5XYCmIEYAAEB9FQcIHnykeXDgdLve8+eMvtU4S6UHppEnAACg0YgR7KA1owOzTEQKlFL28bHkCWhgO+vk1Sur0yFGAABohLqNE7DOmAdAoxEjAACgjhYPEOQlNzfh27fh27fG3p777BlhgkZYMz0wjTwBAABNRIxgR5QUHZiFSAEEMQIAQD3J6fnk+jq7u5tODFidjh4DQM7W1+RQ54+l/TxigfTrhM4WGI8e6cJXV1hgdcQIAACol9UCBHnpeDx484YwQc1tPD0wjTwBAAANQoygxbYcHZiFSMEuO3n1ynj06PS776ouCABgd7X+1HtDIxFAAWIEAADUxfoBgjzCBPW0hfTANPIEAADUHzGClqlJdGAWIgW75uTVK6XU+U9/WnVBAAC7gokARN0maACWQowAAIDqbTZAkKfDBEopCRNw5rgqciJf0gPmwYHT7bpPn255c5AnAACgtogRtEDNowOzECnYBcQIAADliUej7PZWzpfrkQZkkb7+XrIC0jWqtrR1wBpDUxAjAACgSuUFCPKyuzsJE2S3t4QJtiy6uIhHo+jiIru9lfSA0+1WfhCZPAEAAHVDjKChGhodmEUiBemHD3J0WxEpaAtiBACAjeDa+vIwfgNqiBgBAADV2E6AII8wwTbVMz0wjTwBAAA1QYygQVoWHZhFV5BIQQv0fvghu7sjRgAAWJyc0k6ur7O7u+nEgNXp6Ovm5Qw3s6lu3I9r/vOIBdLzFDpbYDx6pDdEdYVFOxEjAABg27YfIMibCBO4z5615kBnHTQlPTCNPAEAANUiRlBzOxIdmGVWpMA6PJTrDqsuIObwz86S62tiBACAB3G6ukGId2CbiBEAALA91QYI8iRMEF1cpB8+VF6YFmhuemAaeQIAACpBjKCGdjw6MMt0pECvECIFtUWMAACgMXh++zDZBEpCjAAAgG2oT4BgQvju3eDNG8IEq2lTemAaeQIAALaJGEFNEB1YCpGCpvDPzuKrq8t/+2+rLggAYKtkBy3nmPVIA7JIX7MuWQHp8FRbWmwWWx/rI0YAAEC5ahsgyCNMsJR2pwemkScAAGALiBFUiOjARhApqDP/7Gzw+vX9b35TdUEAAGXhenQsgrEosBRiBAAAlKURAYK8fJjA6Xbdp0+rLlG97Fp6YBp5AgAAykOMYMuIDpSKSEHdECMAgDaR08DJ9XV2dzedGLA6HX2tuZwVZs+LAj+2os8jFkjfWOhsgfHokW5U1RUW20aMAACAzWtcgCAvfPcuuriIr67MgwPv+XPCBMnNjZw7Tz98MPb2nG7XPjpyut2qy1Ul8gQAAGwcMYItIDpQCSIFdUCMAAAailO8qARRFQhiBAAAbFKjAwR5uiI7GyYgPbAI8gQAAGwKMYKSEB2olXQ8jkcjiRTIwWgiBdtBjAAAGoEB51FnTJyxg4gRAACwGa0JEOTtYJiA9MBqyBMAALAmYgQbRHSgEQoiBXJxW9UFbJXo4qL3ww/ECACgPmSQHjkvq0cakEX6Om/JCkg3ptrSArPQktuNGAEAABuQ3Nw8+f77NgUI8nSYIHAc7/nzqotTIjm4RnpgHfk8gbG39/FXv6q6RAAANMPJq1eB43CGeyOe/OIX0hUhOtAU05GCj7/+NYMTbFByc+NH0flPf1p1QQAAKh2Pf/Lzn+u7XMON9ikYV+P9v/t3pEUbhBgBAACbkdzctPvoZHJzY+7vt/5YXnRxQXpgIySAzMoEAADbF49GjPTbXBIp2IWB0AAAOyt8944Z5bFTsrs7mQeBPl6zECMAAAAAAAAAAAAAAACffFl1AQAAAAAAAAAAAAAAQF0QIwAAADsqSZIsy7b/XAAAAGzQyh0zenQAAAAAMMsffP/991WXAQCABsuy7Ouvv974y4ZhGEVRFEWPHz9+/Pjxxl9/KW2t48uXL03TNE1z4v+L1HfWcyHa2mYAACib7/thGDqOU3VBmuTBjhk9uoaiG7mIB9dSmqaDwSCOY9u2KykVAGAudnPYWTT+hmI0AgAA1tLv98t4Wdd1gyAwDKMOF0i1tY6O4zx41HiR+s56LkRb2wwAAGULgoDd3LIe7JjRo2soupGLeHAtmaYZBEGSJNsvDwBgQezmsLNo/A31VdUFAACg2Xahj9LWOrqu++D/F6nvrOdCtLXNAACAGnqwY0aPrqHoRi6CtQQADcUXOHYWjb+hiBEAANqv1+vJaKVRFKnPV2kYhqEfIGMfZVlmGIZpmp7nydI0Tfv9vmVZQRDIw8Iw1HdlaZIkJycn8jqGYZyensptWaqUOj8/l9dXStm2rQ9W+r6fpql+U3nN8qpZRh3nVnOdOsqYVEEQRFGkL6nxPC8/RGeSJGEYpmlqGIZhGPn6Jkni+74UWC4yy7+77/vymkEQWJY1UZ2C+s56rqw3pZRedb1eT/rHw+FQCjBrE1RlRz4XAABsmfRAlFJJkpimaVmW7O/yPYcF93fSncjvSevWnZhrzR6dWqlTt3KPTjWzU1eJlv30aJBFmncYhnEcZ1k23cMvWEtzfx0AwO5Y7YBJqbs5tTN7OlSLxo9J9wAA7ADDMIbDody+vLx0XVcvOj09zd+9vLx0HCf/XNu2F787wbZtz/OCIJC75+fnetHHjx/17SAIdPHyPM/LP2WuWdUstY73s6u5Zh3Pz89t286/muu6l5eXuha2beu3mLhrWdbEosXfem59H3zu+/fv8yv5/v4+v5LnboJK7MjnAgCALbNtW3osnufJ/lS6MfoBxfs72Y1+/PjRcRzd87mva3dirnV6dPdrdOpW69HdN7NTV4mW/fRoloK1VNDDv5+3loqfCwA7ZeUDJiXt5u4X2NO1ZjeHatH4kfdl1TEGAAC2wbIsHWC0LCtNU70oiqLhcJh/pG3bEnvcCAlmyu38dVcyaVMcx2maThRpZbOqWXYd1Yxqrl9HKap+tSAIBoOB3A3D8PT0VOdhLctyHEdXyjTNOI71onz1S2KaZpZleoSuOI4nrmwrexOsYEc+FwAAbJllWbob4DiOUkqu0dEPmLu/S5Kk1+sNh8P6dycWsXKPTtGpq7H2/fRoh4Ievpq3loqfCwA7hQMm2Fk0fuQxqQEAYNfFcayHVNLkgO9GzJqrtd/vy8CtSqkkSfJHJzeu7Dqqh6q5kTpOPEV6jfr2xBibjuPoQ9LD4TAMQxnzyjCM7Ux867ruYDDQ43flh9jawibYrF34XAAAUIlF9ncyCuhEV6dx3Qlt5R6dolPXQM396dF6rCUA2AgOmGBn0fh3EDECAMCus217Yi7JLej3+57n6U5PHMf6KqsyNLeOcRznO6NpmsqUtEopffRZS5JEOpSySMdXsyw7OTm5vLxcvhLLsW1bjnrryX3zi7a/CdbR3DYDAEDNLbK/Gw6HMnto/kr9xnUntNV6dIpOXTPRjawt1hIAbAR7OuwsGv8OYlIDAMCum7jgSSmVpumswZGmh2mSUU/13ekjoQ8yDGNicNRFi7uS5tYxTVNd8jRNpeMod23b9n0/X6rBYCBHqJMkCcMwX5Kl3nS1+grHccIwHAwGE/nZpTZBHTS3zQAAUHML7u8sywqCoNfr6d1o47oT2mo9OrVep26dHp1qUadu++hGbgFrCQAqtPieblO7OcV3OOqBxr+Dvri/v6+6DAAAlCjLsl6vlySJ67oyHqnv+2EY6rtKqcFgkCSJXBQlnZggCPRhSt/3syyTwVdN05SDifq5SZL4vi+9GXmuzBE1GAziOM6PsxQEgb4dRVEcx3osJtu2wzB0HEcPnSoHTPPXPw2HQ33Z1grVLKOOxdVcv46SLU3TNI5j6Wt6npd/QBiGskjeQr91HMdhGOqXTdPUtm19CHjuWxfUd5FiP3nyxLbt/OC3el0VbIIt25HPBQAA2+f7fhRFsv/yfT+O49PTU9M0//AP/zAIAtd1i/d3vV4vjuOPHz8qpdI0lVFD9dJadScWtHKPTq3XqVuzR6ca0qmrRCt/ejTLg2tperucnJwkSeI4jqzDgrW0yK8DANgRax4wKWM3pwq/w9u3m0NVaPyYRowAAIBP5BDn9DS0SqkkSbIse3CRUirLsiRJZj13FnnWRKCybM2qo5TWtu25b/HgwwoWzbVafRdRsAnqqVltBgCARthIH6kp3Yk1e3Rzly7ysmWsq2ZthUrQjSwVawkAKjdrT1fGbk7xHY46ofHvDmIEAACgphY56AwAAIA6o0cHAAAAAE30VdUFAAAAeEAYhjIslWEYjEYFAADQRPToAAAAAKChGI0AAAAAAAAAAAAAAAB88mXVBQAAAAAAAAAAAAAAAHXBpAYAAKwru7tLrq/to6OqC1KueDSyOh3j0aOqC1KiXajjdqTj8eDNm///7//+X/zxH3vPn7NKAQDAdoTv3v0f/+E//A9ff/2//6t/1fr+eStld3eDN2/+77/6q3/+R3/kPX9u7u9XXSIAANaSjsc//n34IEcRlVJyAMo8ODD39/Vf1YUFShSPRr/+9/9eKUVHvUGY1AAAgHXFo9HJL395/5vfVF2Qcn3x7bfnP/tZuzt5u1DHssmR3/DtW6XU//LP/tnb//gfjb0999kzwgQAAKBU4bt3gzdv0g8f/rd/+S/V/f3v/vIv7eNj7/lzunZNobuR2e3t//qTn/x///k/Z7e30o3ktAoAoP4kHyBxgez2VucG9AOMvT2JDvzRP/pHSqm//fu/l6dkt7f6MTpVYOztyQ0ud0ELxKPR4M2b+Orqn/+Tf6Lu7//qb/+WjnpTMBoBAACbkdzcWIeHVZeiXLsw6AJWlj/y6714IbkBGZZg8Pp1+Pat++xZ8M03VRcTAAC0jQ4Q2MfHw5cvpb8qBytPfvlLjlHWX74bqXMD+p/SjSRMAACoCX3uP7m5UUp9un19nX+MfXyslHK6XaWUfXRk7O0VHzNMbm6y29t4NJIXTMfj+O3b/AOsTkdSCEop6/BQJxI2WjNg83SAwDw4GP7rf+0+fao+997pqDcCoxEAALAuGY2g9Vexf/Htt96LF+0+DcxoBKt58Mhv/gESJgjfvjUPDrznz+U3AwAAwJryAYIHD0HqA5cco6ynud3IuQ8AAKA8cl5fn91XSsVXV/kH5M/uSzdjs52NiQLMyiuUVwBgZcnNjR9FEiB48GBgvicfOE7rL89rKGIEAACsixhBaxAjWIF/drbggd25vx8AAAAWNDdAkEeYoIaWygfkH8xGBABsnAwGkFxfZ3d36YcP2d3dRFxA5hqoyWAA08MhTMyeoJSyj4+NR4/MgwPj0aNPWQfO0WJblrqaSPfqCYzWEzECAADWRYygNYgRLGW1jn5+NDPCBAAAYFlLBQjyCBPUxDoDDKy89QEAUEql4/GPfx8+6PPx+gESFzD39429PbkhfxWWeXH52mW3t7qa+gE6/aCr2aDaoRF0gMDY21tqbtPFL1LClhEjAABgXcnNzZPvv2/96ec//Df/pvVz2xMjWND6w4793tRon+cwBgAAKLCRU8iECSq0qRkKCBMAAIpJPmDBE+qtv15/eqyFueGJCsdaQENNBAi858+XbUJMZVVPxAgAANiAXTj9fPLqldXpECPYcZs9aMtxfAAAMFd2dxddXGz2tDGdkC0r47gwYQIAwPTw/nK+PP8Y+/hYKSXzEdhHRy2OCyxL4gXxaKSUkpU2MZXDp3RFPaZyQG1JN2/w+vXKAYLpV5NOo/fixZqvhvURIwAAYAN24fQzMYIdF49G/d/+towDtRwCBgAAD8ofRnS6XffZs832EwgTbEHZF5bRkwSAHSFnuxc85y27A3YKq5lY1XOTGYpVvat0N08ptX6AYDuvjGURIwAAYAO++Pbb0+++c7rdqgtSImIEO2s7R9g5BAwAALRtDmpKmKAk29yI9CQBoDWmR+CfiAvICPxcIr9l0wM/TMwToZSyj493ZJ4IbKebt/5ECVgfMQIAADbgi2+/9V68aPcp9pNXr5RS5z/9adUFKRExggn6qLp5cOA9f+4+fVr2O+pDwE63GzgOs6ABALBrqpoVlTDBBlW1EQkTAECDpOPxj38fPuiz1PoBEhcw9/eNvT25IX8VlhnT8tsxu73VG1Q/QOc89AZlOzba9rt5hAmqRYwAAIB/qI/NAAAgAElEQVQNIEbQDsQItO0HCPLCd+/8KNrycWcAAFCtqs495xEmWFMdNmI+TOB0u1vuxwIAJkg+YMHTzFzF3hrTo0rMjYkwqkT9VXi8Lh8mCByHDt7WECMAAGADiBG0AzECpVQ6HvtRFF1cVBIg0OpwDBoAAGxH3fb7hAlWULeNGL57F11cVBWKBYAdND3ovZxFzj/GPj5WSsl8BPbREXGBHSTxgng0UkpJ85iYtOJTjoRJK2qmJqOH1uSg5U4hRgAAwAYQI2iHHY8R1DDYW7fj0QAAYLPqvK8nTLCgRmxEjjUDwAbJOeAFzwTLDpTdKIpNNKq5GRRFo9qWGk4aRQdvm4gRAACwAU9+8Qv7+LjdMQL/7Cy5viZG0Eo1n2ZMH5tWStWweAAAYAV1PvecR5igQOM2IseaAWAp0+PST8QFZFx6LhxHSaaHuJiYEUMpZR8fMyNGSWoYIMijg7cdxAgAANiAk1evrE6HGEHT7WCMoOYBgjw5Tj14/br+RQUAAAWk+xFdXNT83HMeYYIJTQkQ5HGsGQBmScfjH/8Wm8Ze/iosM3ZZvsVmt7e66eoH6ESLbrq02KU0qOvboKI2FDECAAA2gBhBO+xUjKChl/g3KPcAAAAmTOzH3adPm3U8l8OUqpkBgjzCBAB2meQDFjz5yrXdaJzp8TPmBmIYP2NCvqc0fPmyKd3deDTq//a3tR04odGIEQAAsAEnr14Zjx6dfvdd1QUpkX92Fr59+/FXv6q6ICXakRhB/uCv9+JFE8/ET5yEaHeCBwCAFmhTEHBnwwRNDxDkESYA0G7TQ8HPnWmeuABaTOIF8WiklJIPwsT0HJ8SM7s9PUcLekc1n4WhoYgRAACwASevXiml2n2lvn92Nnj9+v43v6m6ICVqfYygTQd/Ve6ERHN/4QAA0HptChDk7VSYoGV9SK0Fh8sBQEYUWPD8qOyt2r3PAhYnHxz98ZmbtmnrzAjJzY0fRa3pEeXDBIHjEJBaEzECAAA2IHz3TinV9G5WsXg0ikejdl/27Z+dNW5w3WX9xPPso6PWHPxVn3/tqLbneAAAaKh4NOr98EObAgR5ch7a3N8fvnxZdVlKlN3d/eTnP3e63Tb1ITXZiOl4/H4wqLosALC06OKi98MP6vNo7Tt+OTWwvunBPPTcH6fffed0u1UXcPPkvHsLAgR5raxUJYgRAAAAoPGyuzuOjwAAUE/spluAjQgA9ZTd3WW3t+3LeAF1k47Hxt4e3SHsGmIEAAAAAAAAAAAAAADgky+rLgAAAAAAAAAAAAAAAKgLYgQAAAAAAADAQpIkybJs+8/FBrERAQAAgLn+4Pvvv6+6DAAAbEOWZb/73e/SNDUM4+uvv86y7Ouvv17/Ndd/kRrKsuzP/uzPfve73/3pn/5p1WXBZpTXVsMwjKIoiqLHjx8/fvy4jLcoVlLVKq8XAABN19bux8uXL03TNE1z4v+L1HfWc2urrR2tndqIANqqrV/RQN20tU+r2vs10uJNtn2MRgAA2AlhGPb7/SzL0jQdDAb9fn8wGKz/sv1+f/0XqSHDMOI4TtO06oJgY8prq67rBkFgGEZVV2WVVLXK6wUAQNO1tfvhOM6Dp5AXqe+s59ZWWztaO7URAbRVW7+igbppa59WtfdrpMWbbPu+qroAAACULoqiNE1PT0/1f3zf38grt7jHYFmWZVlVlwIb0+K22uKqAQDQaG3dR7uu++D/F6nvrOfWFhtx8ecCwJa19SsaqJsWf9baWrW21qsSxAgAAO0XhuH5+Xn+P67r6v6EjESUZZlhGKZpep5nGIYs6vV6Ml5lFEVKKdM0JW+olErTtN/vJ0lycnIiDzYMQycVZKllWUEQyFuEYajvylKl1Pn5uby7Usq2bSlVr9eTEjqOo5TyfT9JEv3crdErYUKSJBLCkNWllNpywVZQsB3FrDYgG04pdXp6Kv+J41jGsRgOh2VchLRyUQuaXHFbVYUNUh7g+36apvpNV9vixVVboV5q3sdwbtU2Ui8AAJquZd0PGWg0CIIoipIkkX96nmfbtn5MkiRhGMpkZ4ZhTNS3oLsrPXP5Tz5xO7e+Bc/VHU69Jnu9nvxUkQ5nwa8VrWUdrTU3YvEPlo1vxLlbsGATANgFLfuKBmqrZX3axavW3K+RFm+yVrkHAKDtbNuetej09NR1XX338vLScZz8AwzDGA6Hemn+wcWvPL10+q7neUEQyN3z83O58f79+4l3cRzn48ePBW9UBs/zdJHyLMvShbm8vCxeA/VRsB2L28DEtpDH65eqVVHvC5vc3C01q0He39/nm18QBA9Wf1aDyZtVtXXqNX132qyqbapeAAA0Xcu6H+fn57Zt5/f4ruteXl7qKti2PdGhzb/d3O7urPddpGP84HMf7P/LjbnrX2tZR2udjbjID5bNbsSCLXi/zEYE0FYt+4oGaqtlfdq8tn6NtHiTtcaXVccYAACoUhRFw+FQ37Usy7ZtySHq/+g0omVZaZpu8N0lRym39bU1csGKfqM4jk3T3P7VKkEQ5C/30UzTjONYbluWlV97dVawHYvbgK5vr9eTcQiSJHlwzVRe1PU92CCVUjLjVxzHaZqu8ymYVbWy66VmVG1T9QIAoOna1/2QcuqXCoJAOnJKqTAM9UBT8kjHcfI12n531zTNLMv0YGlxHOevdF9w/bevo7XyRqzVFlRb2QQAaq59X9FAPbWvT6u19WukxZusNZjUAACw0+I41mMcaTKbwBbMmlbT87zBYCBdpTAMa3WqfjgchmEo4zsZhtGCmUGL24Bt2zrJIUOYpmlaxnQGiyi7uT64NbMs6/f7MlqsUkqm2NjUO4otfAynq7aFegEA0AIN7X5MPF6OBurbEwldx3H0+WlVUXfXdd3BYKDHZdWjp66//pvb0Vp5I9ZqC6qqf3ICqLPmfkUDjdPQPu1cLf4aaesmaxxiBACA9pNDTg9e0G/b9sSEl3WgByTIsqySoQhmkSN3OqqZZdnJycnl5WWlhVpXcRuwLEuOSNq2nSSJHIvcYul+TyXNtd/ve56nO81xHOuruzalrfUCAKAFGrqbjuM4f5AxHwPVp6K1JEl0B6+q7q5t29LnlN6mLs/667+hW1CtuhHrtgVVXX9yAqiD5n5FA43T1o9bW+ulWl21ZmFSAwBA+wVB4Pt+/mCTXJ6ipq49Ukqlabr4aEUyfKW+O308S1t2zCUZkCCKIn0IrA6SJAnDUN+tT75hHXPbgGEYchDTcRzf9yus9VLNdaLJLd5WJxiGMTEo66LFXdg69VKrVm0L9QIAoAUa2v1I01QXO01TOSAod23blh8CulSDwUCfrl6nu7tyfYXjOGEYDgaD/KVRa/5aWfYVatXRWm0jrvmDZZ2N+OAWVJvYiADaqrlf0UDjNLRPO1eLv0bauska54v7+/uqywAAQOmSJPF9X/oBMjKBHmdyMBgkSSKXtkivIggCGcCg1+slSeK6rjzY9/0wDPXd6ZdVSuUnIJDsgryUaZpyREkm9YzjOD8sUhAE00MkyRhK+fEwKxfHcRiG+vKaNE1t2675vAaLbMdZbUCWynaULfuTn/xkOBzmZ8yqYVGnm5wqbKvFDTKKojiO9Vhetm2HYeg4jh6yVQ7U5q+7Gg6HE/M+zK3ayvVauWobqRcAAE3Xyu6HXDOUpqlMTZVlmed5+Z14GIaySF4//9YF3d2571tQ3wW7Fk+ePLFte6L/X7z+F9mITexorbwRi3+wlL0RH9yCi2xEAG3Vyq9ooIZa2addsGoN/Rpp8SZrGWIEAIAdkiRJlmWWZU0fspEDVQ8uKpZlWZIks55b8I5z+b7veV4Njy7pKpd0Nr0qK7eB7SsoakGTK26rBeSJE4HcMqxWL7Vq1bZWLwAAWqBB3Q8panFPtbhDu3J3d+X6zrV+T7VZHa01N+I6P1jqvBEBtFWzvqKBRmtQn3YpLf4aaesmawpiBAAA1FGapmEY1mooAgAAAKD+FjkDjZpjIwIAAACV+6rqAgAAgN/T6/WyLJPBkaIo0hO1AgAAACgWhqGMNWoYRuuHGG0rNiIAAABQB4xGAAAAAAAAAAAAAAAAPvmy6gIAAAAAAAAAAABgOf/XX/1VdndXdSkAAO3EpAYAAAC7Ih6N4tEo+Oabqguyef7ZmX10ZB8dVV0QAADwe+LRKB2P3adPqy5IiVrfD2l9BdPxOHz3rpWdZAAtI3vVdDxOrq/T8Tj98ME8OEg/fFBK2cfH5v6+sbdnHx0Ze3vW4WHVhQVapcW9hbb29Fq8ybaJGAEAAMCuiEejwevXrexAD16/Vkq17zcPAABNF49GyfV1u2MEyfW1anU/pPUVTMfjtnaSATRXdneXXF8n19c/3ri9lUXG3p7V6TjdrvHo0Z8Yxn/71VfJzU364UM6Hsdv38qvY6WU1emY+/vmwYG5v2/u77f4axzYghb3Ftp6SK3Fm2ybiBEAAAAAAAAAAABUI7m5yW5v49Eou71Nx+P46kovsjodY2/PffasIA3gdLv6tgxXEI9GSqnk+jp8+1bnDyRSYHU6xqNHVqcjN0quGYC6M/b2qi4C6osYAQAAAAAAAAAAwDZIXEAPISCDvgj7+Nh49Mh78cI6PDQPDlaYm2A6bSDjGejZEKKLC5kHQb8jsyEAu8zqdKouAuqLGAEAAMCuaPF1BkSnAQBAhfJngFqp9RUEgJJMnMJPx2N9Cl/mJrA6Hfv42D46ktP/ZZTBePRoegwDCTEwGwIAoAAxAgAAgF3R4nxxi6sGAAAAAGgEHReQ9EByfT0xoYDT7dZkQgHr8NA6PGQ2BABAAWIEAAAAAAAAAAAAS9AX9Ge3t+l4HF9d6UVyQb/77FmDLuhnNgRgZzHuFGYhRgAAALBb0vG4pJESq6WvkwAAAAAAYLPi0UhOqMssAPmzbnJC3Xvxwjo8NA8OWnNCndkQAGDHESMAAADYLW2NEaTjcdVFAAAAu8jc3293P6T1FQSACT9OSaAvx/98Fb6xt2d1OvbxsX18bB8dyfnyaku7ZcyGAAC7gxgBAAAAAAAAsCJjb0+1+ix76ysIYMdNnAVPrq8nzoI73S5nwQswGwIAtBUxAgAAAAAAAAAA0H75Mfmzu7v46kovkjH53WfPrMNDOcldYTkbjdkQgAaxOp38LC1AHjECAAAAAAAAYHWtP/ba+goCaKt4NNLXxGe3t/lvM7km3nvxgmvit4PZEACgcYgRAAAA7Aqr06m6CGUhOg0AACqkT3u0VesrCKAFZCD95Pr6xxH1Pw+kb+ztWZ2OfXzsdLufLnzf36+2tFDMhgAAtUeMAAAAYFcQ3gcAAAAAtIC+kD27vdWDDcgiuZDd6XblLDUXsjcIsyEAleDKHMxCjAAAAAAAAAAAANTUxInk/BkvOZHsPntmHR6aBwdcod4+zIYAlI1xpzALMQIAAIDdEo9GrUzox1dXVRcBAAAAALCueDTSw9rn5yZQStnHxzI9gX10xNwEO4vZEABgO4gRAAAAAAAAACuyj4706Mqt1PoKAqiQnP1Nrq9/vPH5olhjb8/qdJxul8vKMRezIQBAGYgRAAAAAAAAAACAciU3N9ntbTwaZbe36XicH1LO6nSMvT332TNO62JTmA0BWARNHQWIEQAAAAAAAABrye7u2n0QtvUVBFAS/+xMrgVPrq/1P+3jY+PRI+/FC+vw0Dw4YJB5bMdqsyFYnY779GkV5QW2wep0qi4C6uuL+/v7qssAAACALYlHo1bOHym/+blgBQCAuknH4+zurt3nh+QMRIsvW9yRCtKTBErS++GH7O5Oz0zfyh+kaJn8bAjZ3Z3V6QTffFN1oVCxFvcW2lq1ttZry4gRAAAAAAAAAAAAAACAT76sugAAAAAAAAAAAAAAAKAuiBEAAAAAAAAAAAAAAIBPiBEAAAC0UJqmWZatthQAAAAAAAAAsMu+qroAAAAA2Lwsy3zf9zzPsqxll1Yiy7IkSZRSlmUZhpFlmWEYVRcKAADg94RhKHFM13U30o9K0zQMQ6VUEATrvxo2ix4psI6SPkEb/x4GZqENo7bSNE3TVCklTSjfUFvWeymoaUn4hE744v7+vuoyAAAAYPOyLOv1esPh0DTNZZduWRiGcRw7jpNlmXTWDcPgYDoAAKgn3/dt27Zte1MveHJycn5+vqlXw6b0er3T09OqSwE0VamfoI1/DwPTaMOop/wxtDiO0zS9vLzUS9vUeymuaan4hGqMRgAAANBOhmEUHI8uXrpNURSlaZr/keP7foXlAQAAAJRSzAIGrINPEJqONox6iqJIH9BzHOfJkyf5pW1qt8U1xXYQIwAAAECVwjCcCDS4rqt/9iRJIuOJGYYhQxToEcx6vZ5pmqZpRlGklDJNM79UKeX7vh7bQJbqRXOfCwAA1lSwI65ckiQSW5SyKaWCIAjDMIqiIAiiKJK5lpRSnuctfh1ScZUffFO9VC63yrKs7G5JFEVRFOlCep5nGEYURTK3gmVZUqperyf9MT141YNPlNfcQs9Kto5SyjAMz/N0DjVN036/r5Q6Pz+XEiqlbNt2XbegvvplZ20yedkkSU5OTuQ/hmHkY68FLzu3SECtFH9+ZzV1aef6G0O+Q/TduZ+g4o9JnXcfqKG5+6AHmzFtGMVmHYyqfC+fpml+VNF8y1y591LPfmBBTYuro2b3G2Upn9Dl3AMAAADVsW171qLLy0vbtj9+/Pjg3fv7e8MwhsOhXuq6bv7p+UcGQaAfuchzAQDAmop3xNWyLGuigyG3z8/Pbds+Pz+Xux8/fnRd9/LycuLpnufpx+QVV3nWm95vsVtyenqaf/HLy0vHceT2+/fvJ95XLyp+oii1Cqenp57n6bue5028vm3bnucFQSB39daZW+ziTTarmzr3ZQuKBNTQrM/v3KY+8Rkpvjtt1sdkkd3HrO9h7KaCfVBxM6YN40FzD0ZVu5d3XTcIgvfv3z+4dLXeSz37gQU1LSjS3H4jn9ClfFl1jAEAAAB4WBiGp6enOk1sWZbjOBIo1v/RiW/LstI0zT/dMAw9fdr00uLnAgCANRXviKtlmmYcx3LbsqzhcKgXWZalhx+Qi88Gg8GCL1tc5eI33U63JIqi6crqS8eyLNMjQsVxbFnWIk/cQhXCMMxfKOZ53vTry1VocltvwbnFXq2Vzn3ZgiIBNTTr87tgU1/Hgx+TOu8+UE8F+6CymzFtuJXmHoxSle7lh8OhZVlhGPZ6PRl+YJFnFX8W6tkPLKhpQZHm9hv5hC6FSQ0AAABQUzJ2XP4/juMseCg/y7J+v69fIUmS/E8gAABQqprviIfDYRiGMqKpYRj5cWgnyinHGRd5zblVLnjTrYnjWI9zqzmOIzdc1x0MBnp45/wR2OInbpmeFSLvwfVZXOyVW+mCa4NZDNB0W/jgT39Mar77QOOU3Yxpw620yMGoavfytm1LdiHLsl6vlw89zDL3s1DPfuCsmi5epIl+I5/QZREjAAAAQJXk6PyDP3imj9onSbLg/Gr9ft/zPP1jII5jff0fAAAoW513xNLB0BeQZVl2cnJyeXkpd+M4zh+CnJiWtUBxlYvfdGts256YVnZiqRwi1zMBL/jEsk10/9I0XfC6seJir9xKq10bwNZU0tTrvPtAE22/GdOGW2Cdg1Fli6LIsizdOzUMw7KsJEnmjogw97NQt35gcU0LilTcb+QTuiwmNQAAAECVgiDwfT//I00u1FNK2bYtN0SWZYPBYMG8s/zA0Hc3O/YmAAAoVucdcZIkYRjqu9OHGvXVZmmayqHGRV62uMrFb7o109fSTRxadRwnDMPBYDBxjd3cJ5bKdd1+vy/dxSzLfN9f8Lqx4mLPbaUywK++q29XuzaArVmqqS/+CSpW590HmmjxZkwbhrbOwaiyJUky0ahkZH59d53eS636gcU1LShScb+RT+iyvri/v6+6DAAAANhpSZLobr2MTKAHTwvDMI5jSR8nSRIEgX5Yr9dLksR1XXmw7/thGOq7URTFcazHKLNtOwxDx3GCIJj7XAAAsKaCHXHVRVNxHIdhqK+yStPUtm05WipXI6VpKt2PLMs8z9NXQUVRJFGA/EVaw+FQHlBc5VlvOt0tOTk5SZLEcZz8hK8bNBgMkiSRMssB1iAI8rGGJ0+e2LY9vaVmPXE7PStZgXqj6FlvB4NBHMf5AWl1d3Fufee20ok+qlJKb5SCl51bJKA+5n5+i78xJA4u3wOmacqZpwU/QbM+JsUfzOLvYeygRfZBBc2YNoxZZh2MqnwvL/kGPahnHMeu6+bP+q/We9Hq0w+cW9OC6szqNyo+ocsjRgAAAIBaSJIkyzLLsiZ+w2RZliSJUmruEG0T5IkTQWMAALAdNd8RP9jBkBjBsl2OidcsqPLKvZqNk5pO97vKe+Jm9fv9pWIWs4q9+CZ7sMo1WRtA2Qqa+qwfcaL4EzRLzXcfaKhZzZg2jFnq022bVly28nov2+/5zN0KixRpot/IJ3QpxAgAAAAAAACAdWME2A6ZxbYOI1sAAACgzug3rumrqgsAAAAAAAAAVCwMQxnF1DCMHR+8tIZkyFy5bVkWx4IBAADwIPqNG8RoBAAAAAAAAAAAAAAA4BNGIwAAAAAAAAAAAJsUj0Z/+Td/8y/++I+tTsd49Kjq4gBLy+7ukuvr7PbWPj6mDQPYQcQIAAAAAAAAAADAWtLxOLm+Tm5ukuvr+OpKKdXZ378ej5VSVqfz49/hYdUlBYokNzfx1VV0cZFcX+t/Ot2u1ek43a65v19d0QBgq5jUAAAAAAAAAAAALE1OuKbjcTwapR8+KKWMvT376Mg8OLCPjuyjI0kVJNfX+QdInsA+OmKgAtRHdHGR3NxEFxfSUJ1u1z4+to+Osru7fKrAPDhwul1p3hWXGABKRowAAAAAAAAAAADMJ6dU80MOKKWsTsc+Pjb39+2jo4JrtWc9V6cKuM4bW5bd3UUXF/HVVTwaZbe3Ov7idLuzHpxcX0cXF9ntrSRm7ONjp9slDQOglYgRAAAAAAAAAACAh+kRBeRP/f6IAitfkz13JIMNVgHIm5i2QHIwTre7+Iwb8WgUj0bx1dXKrwAA9UeMAAAAAAAAAAAA/EjOkkpuILu9VblhA6xOZ+PnStcZ5ABY0KxpC9ZpXel4rIcoUErpEIx9fMwQBQCajhgBAAAAAAAAAAA77dOoAOOxvsBaKWUfH0toYMvnROVi8eT6Orm5YaACrGOpaQvWFF1cSP5GxxSsTsfpdgnBAGgoYgQAAAAAAAAAAOwcfcpTzrCqz9dSm/v79vFxTYZnT8djyRNMD1RgHR5anQ7naDFt/WkLNvju5sGB0+2SgAHQOMQIAAAAAAAAAABovwdPyeshBxpxSl5PtaAHKjAPDqTwVqfDadodV8a0BeuQsRBkyoPs9lYG1ZBAA1MeAKg/YgQAAAAAAAAAALTTrAkCWnDevQWpCKxvm9MWrEMG/9CThmx5gAQAWAExAgAAAAAAAAAAWiK7u4uvrh6cBcDc36/wyuyyPThHAwMVtFW10xasIx2P9RAF6vNMIjJKAUMUAKgVYgQAAAAAAAAAADSYhAaS62s5j64+Dzmgz1BWXcBtS8fjeDRKx2N98bfKDVTA+drmqtu0BWuKLi50/EUp5XS7VqfjdLsNrQ6AliFGAAAAAAAAAABAk2R3dxIakPSAXHwvl91/+qv9Ndnb9OBABXLu2T4+Zl3VXFOmLVjHxOAK5sGB0+3uZgYIQH0QIwAAAAAAAAAAoO4KrrC3j46sTocr7BfBQAVN0dxpC9YhmQmZ8iC7vZVhRaTitEwAW0aMAAAAAAAAAACAOuIy+lI9OKgDa7haLZu2YB3y8dd5lx0JUgCoD2IEAAAAAAAAAADUQjoeJ9fXch0218pvWXJzI3kC+VNKGXt7MkkE4z2UahemLVhHOh7rIQrU56SLjFJAmwRQHmIEAAAAAAAAAABURl8Nn9zcyBXY5sGBdXioT2BXXcBd9OBABbJFPv1xRfjadnPagjVFFxd6kBKllNPtWp2O0+3u4GgNAMpGjAAAAAAAAAAAgO3J7u7iqyu59j2+upJ/yjlUc39/N8dvrzk9UIE+fctABStj2oKNmAhhmAcHTrcroxRUXTQALUGMAAAAAAAAAACAcsk5v3Q8zp+Hto+O9PjkVRcQi5qVAtGpAk6HT2PagvLIupUpD7LbW/likXEdSLcAWAcxAgAAAAAAAAAAyuKfnYVv306Mis/J5tZ4MCASOI779GnVRauL3g8/RBcXimkLyifzHcRXVzJEgfvs2fDly6oLBaCpiBEAAAAAAAAAAFCW8N27dDxm6PtdoAcqsA4Puc5eC9+9U0oRndmmdDyOLi6MvT3iLABWRowAAAAAAAAAAAAAAAB88mXVBQAAAAAAAAAAAAAAAHVBjAAAAAAAAAAAAAAAAHxCjAAAAAAAAAAAgIolSZJlWdWlAAAAUEqpL+7v76suAwAAAAAAAAAAbZZlmWEYBQ84OTnxPM+27a0VCRs3dyuvJgzDNE2zLHNd17Ksjb9+SVgbVWHNA9gIRiMAAAAAAAAAAKBc/X6/+AGO45imuZ3CoCRzt/JqXNcNgsAwjGaNV8HaqAprHsBGfFV1AQAAAAAAAAAAaLm5J95c191OSVAeTq/msTaqwpoHsBHECAAAAAAAAAAAu6LX65mmaZpmFEVKKdM05fpaWRpFURRFMiS4aZqe5+XHBg/DUJ5lGIbneVEUpWl6enoqS33flxG/5blBEMj/0zTt9/tJkpycnMh/DMPQz5InJkmilAqCID9UeBRFYRgqpSzLklfr9XpygnA4HMrQBcUFxoOK24CasVZlO+ptIVtH3527leUBSqnz83N5faWUbds6PjKr/VS7NmY1sFLXRlWrYstohwDq7h4AAAAAAAAAgJ1hGMZwOJTbl5eXruvK7dPTU31bFjmOo++enp56nqfvep6Xf/D9/f3Hjx/17SAI9FsI27aLS+V53jRk/oUAABYvSURBVPn5+cQ/379/P/EuE0UqKDAKzGoD9/PW6sR2LL47zbZtz/OCIJC7+S1e3H7uZ7SQjVjtE3Ff2tqYuyruy1wb20Q7BFBnX1YdYwAAAAAAAAAAYHssy9KX3lqWlaap3I6iaDgc5h9m27ZcrauUCsMwf2Gu53n6iUKmDI/jOE3T/MuuwzTNLMv0EOVxHE8MV1BQYBSY1QZU+WtVLiuX27Zt6/+X0X4WtNonYiMeXBsVrootox0CqDMmNQAAAAAAAAAAQMVxrEcC1xzHefDBMuK3vptlWb/fNwxDxiRPkiR/vn8drusOBgM9gHk+yrBUgbGgsteqPm2cV177WccWGtj02qjnqtg+2iGAyhEjAAAAAAAAAABA2badn0R8Qn7OcqVUmqb563T7/b7nefqUWxzHcRxvqlSDwUDeUZ/eW6TAWE0la7W89rMOVkWFWPkAKsekBgAAAAAAAAAAKMdx5IS9ls8KuK7b7/dlfoEsy3zfz1+naxjGxHQDEy8u0xPou/nbixQsDMPBYDBxAXFxgbGaxdfqBrfy3PZTiaUa2KbWRj1XxfbRDgFU7ov7+/uqywAAAAAAAAAAQOmyLOv1ekmSuK4rswP4vh+Gob47GAySJJHZCuTcWxAEegCAOI7DMJRTdJ7nhWGopxiIoiiOYz0SuG3bYRg6jqMfkCSJTh7IK8vE51EUhWGofn+wgeFwmJ8xQSn15MkT27bzMxqI4gJj2tw2oArXqu/7WZbJ/PGmaUq2Y+5WlteM4zg/SnwQBPp2QftZsIWUtDaKG1gZa6P4o1Tq2tgm2iGA+iNGAAAAAAAAAADAj2Qcb8uyis/H9/t9fXJOKZVlWZIkE5fz5skDFnnlkgqMpcxaq0mSZFk2a22vvJXntp8KFTSwMtZGnVfF9tEOAVSFGAEAAAAAAAAAAMuRWcOnhwcAAABoga+qLgAAAAAAAAAAAA0g45DLbcuyyBAAAIC2YjQCAAAAAAAAAAAAAADwCaMRAAAAAAAAAACwecnNTXRxcX9//6TTcbrdqosDYFdkd3fh27dKKffZM+PRo6qLA6CRGI0AAAAAAAAAAICNkfRAdHGRfvhg7O39N1999Td/93fG3p7T7dpHR+QJWq//29+m4/H5T39adUFq4eTVK6vTCb75puqC7Irs7m7w5k349m12e/vfff31H3z5pfvsmff8OWECAMtiNAIAAAAAAAAAANaVjsfhu3c6PeB0u/Y330hoQAcLwrdvyRO0nrG3p8bjqkuBnZMPEHyKDuztyX/Ct28JEwBYFqMRAAAAAAAAAACwonQ8lohAcn2tlHK6Xfv42Ol2HzxdNzFQAXmCVvLPzpLra0YjEIxGsAXTAQJzf396qVKKMAGAxREjAAAAAAAAAABgOUulB6aRJ2gx/+wsfPv2469+VXVBaoEYQdnCd+8Gb96kHz5MBwjyCBMAWBYxAgAAAAAAAAAAFrJmemAaeYL28c/OBq9f3//mN1UXpBaIEZRHBwjs4+Phy5ezAgR5hAkALI4YAQAAAAAAAAAARTaeHpgmeQIZllzyBO6zZ9bh4aZeH1tDjCCPGEEZ8gEC7/lz++hoqacTJgCwCGIEAAAAAAAAAAA8YAvpgWnRxUU8GkUXF9ntrXlw4HS7TrdLnqBBiBHkESPYrOjiwj87WzlAkEeYAEAxYgQAAAAAAAAAAPyokvTANPIEDUWMII8YwabEo9HgzZv46mr9AEEeYQIAsxAjAAAAAAAAAACgLumBaeQJmiUejU5++UtiBIIYwfp0gMA8OAi++cbpdjf+FoQJAEwjRgAAAAAAAAAA2F3Z3Z2kB+KrK6WU1em4z57VIT0wjTxBIxAjyCNGsI58gMB7/tx9+rTUtyNMACCPGAEAAAAAAAAAYOdIeiC+uoouLpRSVqcjZ+XN/f2qizYfeYI6kxjBx1//mlOwihjBqtLxuP/b324tQJBHmACAIEYAAAAAAAAAANgVjU4PTCNPUEMSIzj/2c82NXt9oxEjWFY6HstZ/O0HCPIIEwAgRgAAAAAAAAAAaLmWpQemkSeoD2IEecQIFqcDBMbeXk3O3BMmAHYZMQIAAAAAAAAAQDu1Pj0wLXz3bqfqW0PECPKIESyihgGCPMIEwG4iRgAAAAAAAAAAaJUdTA9MYA1UiBhBHjGCYnKGfvD6dT0DBHmECYBdQ4wAAAAAAAAAANAGnDufxjrZvnQ8/snPf06MQBAjmEWflc9ub70XL5pyVp4wAbA7iBEAAAAAAAAAAJqNkfznIk+wTV98+y0xAkGMYFo+QCBn4hv3MSRMAOwCYgQAAAAAAAAAgEaKLi7i0Si6uMhub82DAzkvbh0eVl2uWiNPsAXECDRiBHktCBDkESYA2o0YAQAAAAAAAACgefq//W349i3pgZXl8wTG3t7HX/2q6hK1x8mrV+7Tp063W3VBquefnZn7++7Tp1UXpBYGb974UdSCAEGeDhPYR0en331XdXEAbAwxAgAAAAAAAABA8yQ3N0op0gPry+7ukutrLp0Hypbd3WW3t60JEOS1uGrAziJGAAAAAAAAAAAAAAAAPvmy6gIAAAAAAAAAAAAAAIC6IEYAAAAAAAAAAAAAAAA++arqAgAAAAAAAAAAsIQsywzD2PjLhmGYpmmWZa7rWpa18devJ1Zmi8VxnL9rWVYZ23pxWZYlSaJLotvedlpLGIZJklTSINv9KWt37YBdxmgEAAAAAAAAAIAm6ff7Zbys67pBEMjZzTJev55YmW2Vpmkcx3EcR1E0GAziOJZT+FUJw7Df72dZlqbpYDDo9/uDwUAWbae1uK5bVYNs96es3bUDdhmjEQAAAAAAAAAAmoSzShvEymwr0zSDIFBKSZhAblcliqI0TU9PT/V/fN+vsDxb1u5PWbtrB+wyYgQAAAAAAAAAgA3r9XqmaZqmGUWR+nxGMz/wdRRFURTJaNimaXqeJ0vTNO33+5ZlyVnPKIrCMNR3ZWmSJCcnJ/I6hmHoc5OyVCl1fn4ur6+Usm3bdV15gO/7Mkq2vGm1J1aXUrw+y1iZat76bO7KxKwGI2Zt2bkf6gJhGJ6fn+f/47rugqefZ5WnuHlP19RxnOI3KuNba81PWUH1l7Va7fgOAXbaPQAAAAAAAAAAm2YYxnA4lNuXl5eu6+pFp6en+buXl5eO4+Sfa9v24ncn2LbteV4QBHL3/PxcL/r48aO+HQSBLl6e53n5p9THrPVZ6sq8n70+G70yd8r5+bnnefru3AZTsGULPtTF5jaz+9mtpbilFbTniZoOh0PLsoobZEnfWit/yu4X+KAt/ilbuXZ8hwC7idEIAAAAAAAAAACbZ1mWvqDWsqw0TfWiKIryV6xalmXbdhRFcy8XXpBpmvqtbdvW/5dptpMkMU3Tsqw4jjfydtsxa32WvTLVjPXZ6JW5y+Y2mIItW/ChDsMwf1fYtp3/AK5m5ZY2UVPXdeVq+ALt/taqW+34DgFqjhgBAAAAAAAAAGCr4jjWg2BrGzztrc9X5WVZ1u/3DcOQcbyTJLEsa1PvWKGyV6Z6aH22dWW2Q6/Xy58SnlDcYFbesg9+6Na3TkubnjRhnVba7m+t7deO7xCg/ogRAAAAAAAAAAC2yrbtgtOcJen3+57n6TNVcRy34+JXVmatJEmSZZlMQl9VGaZPn+cVN5iStqxcdy4njJeyTnmm3654zRRr9wdt+7XjOwSovy+rLgAAAAAAAAAAYLc4jjMYDPL/SdN0ekR0MT0OuWma+dOBC54aNAwjf7Xr3OHNm4KVWSu9Xq/X661wvnyDDMNIkkTfjeM4f1l5cYMpacsGQeD7fr51hWHo+/7cJy5VnomlpmmGYajvJkmyTnXW+aCt9ilTW/ygLV47vkOA3fHF/f191WUAAAAAAAAAALRHlmW9Xi9JEtd1gyBQSvm+H4ahvquUGgwGMiW2+nzaKQgCffJVzjjKFcymaQ4Gg/xzkyTxfV9OQclzh8OhvGYcx/nBsYMg0LejKIrjWA+gbdt2GIaO48jLRlEkZxzTNNXjbA+HwwqvKdfmrs8yVqYqXJ/NXZmlSpLkyZMnnufp1VuJ/HDxaZratj0xpHxBg5m1ZT3Pm/uhLjbR0gzDWKS1FLc0Na9566W6GHEce543PVx/qd9aq33Kiqu/1KdszdrxHQLsJmIEAAAAAAAAAIBqyCjWlmVNX70tg8M/uEgplWWZXG896wEPkmdNXAXbGqzMyg0Gg8Fg8P79+2pHIxByNXnBNp3VYErdssVN8UFzy1P8mrIeNjjNxGoftNU+ZWrrH7RZteM7BNhBxAgAAAAAAAAAAADW1ev1pi/9BwCgiYgRAAAAAAAAAAAArCs/eDuA/9re3SM3bqQBAMW6HNkRXOVN7cJkTjG+ARRtDB0BPAJwBKh8AuIIxBGEZHIhdTYoT7pTJUR2qg1ag8VyLIjDkQiJ+16gAtn4+RpUd9JfdwOvmjQCAAAAAAAAAODeN2sHAAAAAAAAAAC8FNIIAAAAAAB4Zca//up+/33866+1AzkT/YcPw8ePa0cB52z4+LH/8GHtKJ6LPgTOjzQCAAAAAABen4vffuv/+GPtKM5E1bbNu3drRwHnrHn3rmrbtaN4LvoQOD/SCAAAAAAAeGXi775bOwQA/mv497/XDgF4StIIAAAAAAAAgOPZZQbOjDQCAAAAAABepe7339cOAQDgDEkjAAAAAAAAAADuSSMAAAAAAAAAjhR/993aIQBPTBoBAAAAAAAAcKTkn/9cOwTgiUkjAAAAAADg9Ym//37tEAAAzpM0AgAAAAAAXp/055/XDgGAe+Off64dAvCUpBEAAAAAAAAAx+v/+GPtEICnJI0AAAAAAIBXyeRXAIDnII0AAAAAAIBXafj4ce0QAADOkDQCAAAAAAAAAOCeNAIAAAAAAADgSOlPP60dAvDEpBEAAAAAAAAAR4q//37tEIAnJo0AAAAAAIDXJ/3557VDAAA4T9IIAAAAAAAAgK/Sf/iwdgjAk5FGAAAAAAAAAHyV8c8/1w4BeDLfrh0AAAAAAAB8seyXX9YO4Xzkv/6a/Pjj2lHAOct++eWMW1ny44/lv/51xhWE/0P/uLu7WzsGAAAAAAAAAOBFsKkBAAAAAAAAAHBPGgEAAAAAAAAAcE8aAQAAAAAAAABw79u1AwAAAAAA4MyN49j3fThO0zSO49M8t2maYRjGcSyKIk3ThdP6vl8+Z9kwDMMwRFEU7vBoBYdhaJomiqK6ro97IvBCzJt/aPuff/OSdV0XDuI4ProPBM6PNAIAAAAAAJ7XMAxhpKrv+yRJttvtaZ5bFEUURVVVjeO4fNqj5yxomqbrujzPx3EMiQs3NzfLlyRJUtf1xcXFcU8EXo6u65qmybIsiqLwt+u6YRj6vi/LMnzzYg3D0LZt9Cn5aUpvevnZD8Bzk0YAAAAAAMDzStM0TdOmadI0DUNW56Rt2+vr63Cc5/nbt2/XjQc4paIohmEoiiJJkumbkFjwwnMIoihKkiTP8+hTAkQURX3fV1V1smQv4MWSRgAAAAAAwCkMw1DXdZiyH9YJCF9uNpsoiq6vr9u2DUkGWZaFEy4vL5MkSZIkfB8m8c+nyfZ9HxYAiOM4juPDJ9GGZ43jGMdxGEU7ulLT2GEURXEc73a7eenV1VVY3jxUJMuy+bLhYSWDcRznVWvbNswJ3u124Zuu666urqIo2m634XFVVYX9GuI4DtdOT1x4n1EU7S2BkKbpfGOF+WtJkqQsy+l9Lv8WCxeGak4znsuybNt2GIbpRS2XPnTnR2sKJxPyBuZNqeu66X/4odYaHdDFnV7YiKHv+6mn+tv4q6oKpWGNmSlFrK7rNE2XS0PRcqcBrO8OAAAAAACe2fv378uyDMdFUeyVZllWlmVd1+Hj9fX1VBTH8Xa7Dcc3Nzfza29ubrIsu729/duPk7Is5ze8u7vb7Xbz+2y32zRN9845XFEUdV2/f/9+7/vb29t5PLe3t3meH1i1z1/RbrebTg53m47rup4X3S2+zyzL5sc3NzfzR+y93jzP57d9KODlC3e73fTT393dlWU5P/nR0uWQFmoKpzRvWTc3N9P/5N1jrXWhHziN6+vrvYZzfX19SPxTBzI129vb2yn+5dJHmzawum/WTmMAAAAAAOD8tW07n/QfJujPhdmo4Xi+EniaptP88jRN5xc2TTPN1w+leZ4fsmlC27bzJbuLoviaWbAhC6FpmsvLy81m0/f9FN58fm2Yaj9fumChakmSdF0XRdHl5WVYh6Dv+/lrieN4HMewBfvetdMd/vZ9ThWvqmo+Mzj67LWkaZpl2fx9PhTw8oV7s7TLstz7ERdKHw1poaZwSvPOZ77gSvRYa13oB1Y0juN0/FD8YbeacBy69ziODyw9pGkD67KpAQAAAAAAzy6s3h+Ox3Fs23Ya+g2OWIs+bGQw/ybP8zDovmw+QhbMR9OPMG2CPo7j5eVlSG7o+36vjoc/JcuyruuSJAn3if5394RxHDebzVT9+fLjk4feZ7jJZrOZD/IFXdftbXkQfRr/W/ZFF4YFzB+61V7pIXe2iwEvQZ7nVVXleR5W6Z+6pkNa6wv0Rb3NEY7ubYCTkUYAAAAAAMDz6vu+KIr5mPrFxcXeEPsRPs8G6Pv+kHUFPj/n81sdqG3bNE2nIbc4jtM0DSsHJEkyH/v/ImmahnyILMv6vh+GYR7zZrMpy3IazOu6bkrROETTNPMJ0JMsy6bd3L/I8oV7b3sYhvl86+XSo0OCEwv/ycMwdF03XxXjK1vrKuZpXs8Uv6YNL59NDQAAAAAAeF57S3xHn2bbf+Vtsyyrqmr6OI7j1dXVIfNZkyRpmmb62Pf90Ytpf35tWPc7iqKiKObhhZPnz10Wx3HXdXmeh1nO8+H2kKwwffyi4JumGYZh+jkuLy+nos/Xctgb1H/I8oVFUWw2m5CoMY5jVVXz4JdLjw4JTi/P87Dsyt7+I0e31lU0TTNfFOSZ4te04eX7x93d3doxAAAAAABwtqqqats2SZKiKMIYf1VVXdeN41iWZdhye75Qdl3X4ThsEBBWMqjrOlwYMhLCx+jTXglhxKvv++naKIratg1j9mEqfxiG32634eSqqsLa41OcXdeVZfmlq2qHRIHpVl3XFUUxDdLPwwsj5XVdh43G96p2cXHR932e59N+4SHC8PHNmzfb7XYam2zbtuu6aY3xLMuapsnzvK7rq6urh95neBVv3ryZj3H2fX97ezt9vLq66vv+kID3fouHLpzebdM0SZKEH71pmukXfLT0oTsv1xRW8fbt2zzP50utLLTWQ7q45zYMQ8glChGGBIL50x+KPxSFioQufbfbJUnyww8/1HU9DMNCaeghlzsNYHXSCAAAAAAAeMXGcez7Poqi+ej4IcLk1yRJjtt34PAYwroLaZo+4QhZeOLeROGncnTAB1642WymbIkDS5/jHcJpPGtrPYGX2dsAz00aAQAAAAAAcCJhb/WHJlsvlwIAp/Ht2gEAAAAAAADnLCzeHo7TNN3LElguBQBOz2oEAAAAAAAAAMC9b9YOAAAAAAAAAAB4KaQRAAAAAAAAAAD3pBEAAAAAAAAAAPekEQAAAAAAAAAA9/4DRqD8PRWHf2YAAAAASUVORK5CYII=", "text/plain": [ "Tree('positive', [Tree('neutral', [Tree('neutral', ['The']), Tree('neutral', ['Rock'])]), Tree('positive', [Tree('positive', [Tree('neutral', ['is']), Tree('positive', [Tree('neutral', ['destined']), Tree('neutral', [Tree('neutral', [Tree('neutral', [Tree('neutral', [Tree('neutral', ['to']), Tree('neutral', [Tree('neutral', ['be']), Tree('neutral', [Tree('neutral', ['the']), Tree('neutral', [Tree('neutral', ['21st']), Tree('neutral', [Tree('neutral', [Tree('neutral', ['Century']), Tree('neutral', [\"'s\"])]), Tree('neutral', [Tree('positive', ['new']), Tree('neutral', [Tree('neutral', ['``']), Tree('neutral', ['Conan'])])])])])])])]), Tree('neutral', [\"''\"])]), Tree('neutral', ['and'])]), Tree('positive', [Tree('neutral', ['that']), Tree('positive', [Tree('neutral', ['he']), Tree('positive', [Tree('neutral', [\"'s\"]), Tree('positive', [Tree('neutral', ['going']), Tree('positive', [Tree('neutral', ['to']), Tree('positive', [Tree('positive', [Tree('neutral', ['make']), Tree('positive', [Tree('positive', [Tree('neutral', ['a']), Tree('positive', ['splash'])]), Tree('neutral', [Tree('neutral', ['even']), Tree('positive', ['greater'])])])]), Tree('neutral', [Tree('neutral', ['than']), Tree('neutral', [Tree('neutral', [Tree('neutral', [Tree('neutral', [Tree('negative', [Tree('neutral', ['Arnold']), Tree('neutral', ['Schwarzenegger'])]), Tree('neutral', [','])]), Tree('neutral', [Tree('neutral', ['Jean-Claud']), Tree('neutral', [Tree('neutral', ['Van']), Tree('neutral', ['Damme'])])])]), Tree('neutral', ['or'])]), Tree('neutral', [Tree('neutral', ['Steven']), Tree('neutral', ['Segal'])])])])])])])])])])])])]), Tree('neutral', ['.'])])])" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_tree_train[0]" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [], "source": [ "X_tree_dev, y_tree_dev = sst.build_tree_dataset(\n", " SST_HOME, sst.dev_reader, class_func=sst.ternary_class_func)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Pure NumPy TreeNN implementation\n", "\n", "`TreeNN` is a pure NumPy implementation of this model. It should be regarded as a baseline for models of this form. The original SST paper includes evaluations of a wide range of models in this family." ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [], "source": [ "tree_nn_glove = TreeNN(\n", " sst_glove_vocab,\n", " embedding=glove_embedding,\n", " embed_dim=None, # Ignored when embedding is not `None`\n", " max_iter=10,\n", " eta=0.05) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `fit` method to this model is unusual in that it takes only a list of trees as its argument. It is assumed that the label on the root node of each tree (`tree.label()`) is its class label." ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Finished epoch 10 of 10; error is 9.031078368143927" ] }, { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 27min 40s, sys: 4.74 s, total: 27min 45s\n", "Wall time: 7min 9s\n" ] } ], "source": [ "%time _ = tree_nn_glove.fit(X_tree_train)" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [], "source": [ "tree_glove_dev_predictions = tree_nn_glove.predict(X_tree_dev)" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " negative 0.39 0.20 0.27 428\n", " neutral 0.20 0.39 0.27 229\n", " positive 0.40 0.39 0.39 444\n", "\n", " micro avg 0.32 0.32 0.32 1101\n", " macro avg 0.33 0.33 0.31 1101\n", "weighted avg 0.35 0.32 0.32 1101\n", "\n" ] } ], "source": [ "print(classification_report(y_tree_dev, tree_glove_dev_predictions))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Torch TreeNN implementation" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [], "source": [ "torch_tree_nn_glove = TorchTreeNN(\n", " sst_glove_vocab,\n", " embedding=glove_embedding,\n", " embed_dim=50,\n", " max_iter=10,\n", " eta=0.05)" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Finished epoch 10 of 10; error is 93838.61479759216" ] }, { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 1h 30s, sys: 11min 14s, total: 1h 11min 45s\n", "Wall time: 10min 32s\n" ] } ], "source": [ "%time _ = torch_tree_nn_glove.fit(X_tree_train)" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [], "source": [ "torch_tree_glove_dev_predictions = torch_tree_nn_glove.predict(X_tree_dev)" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " negative 0.36 0.25 0.30 428\n", " neutral 0.50 0.01 0.03 229\n", " positive 0.40 0.73 0.52 444\n", "\n", " micro avg 0.39 0.39 0.39 1101\n", " macro avg 0.42 0.33 0.28 1101\n", "weighted avg 0.41 0.39 0.33 1101\n", "\n" ] } ], "source": [ "print(classification_report(y_tree_dev, torch_tree_glove_dev_predictions))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Subtree supervision\n", "\n", "We've so far ignored one of the most exciting aspects of the SST: it has sentiment labels on every constituent from the root down to the lexical nodes. \n", "\n", "It is fairly easy to extend `TorchTreeNN` to learn from these additional labels. The key change is that the recursive interpretation function has to gather all of the node representations and their true labels and pass these to the loss function:\n", "\n", "\n", "\n", "This model is implemented in `torch_subtree_nn.py`, which uses `TorchTreeNN` and `TorchTreeNNModel` to create this variant. This version should also help pave the way to other subclasses of `TorchTreeNN` that you might want to build." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" }, "widgets": { "state": {}, "version": "1.1.2" } }, "nbformat": 4, "nbformat_minor": 1 }