{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# Artificial Intelligence Engineer Nanodegree - Probabilistic Models\n", "## Project: Sign Language Recognition System\n", "- [Introduction](#intro)\n", "- [Part 1 Feature Selection](#part1_tutorial)\n", " - [Tutorial](#part1_tutorial)\n", " - [Features Submission](#part1_submission)\n", " - [Features Unittest](#part1_test)\n", "- [Part 2 Train the models](#part2_tutorial)\n", " - [Tutorial](#part2_tutorial)\n", " - [Model Selection Score Submission](#part2_submission)\n", " - [Model Score Unittest](#part2_test)\n", "- [Part 3 Build a Recognizer](#part3_tutorial)\n", " - [Tutorial](#part3_tutorial)\n", " - [Recognizer Submission](#part3_submission)\n", " - [Recognizer Unittest](#part3_test)\n", "- [Part 4 (OPTIONAL) Improve the WER with Language Models](#part4_info)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Introduction\n", "The overall goal of this project is to build a word recognizer for American Sign Language video sequences, demonstrating the power of probabalistic models. In particular, this project employs [hidden Markov models (HMM's)](https://en.wikipedia.org/wiki/Hidden_Markov_model) to analyze a series of measurements taken from videos of American Sign Language (ASL) collected for research (see the [RWTH-BOSTON-104 Database](http://www-i6.informatik.rwth-aachen.de/~dreuw/database-rwth-boston-104.php)). In this video, the right-hand x and y locations are plotted as the speaker signs the sentence.\n", "[![ASLR demo](http://www-i6.informatik.rwth-aachen.de/~dreuw/images/demosample.png)](https://drive.google.com/open?id=0B_5qGuFe-wbhUXRuVnNZVnMtam8)\n", "\n", "The raw data, train, and test sets are pre-defined. You will derive a variety of feature sets (explored in Part 1), as well as implement three different model selection criterion to determine the optimal number of hidden states for each word model (explored in Part 2). Finally, in Part 3 you will implement the recognizer and compare the effects the different combinations of feature sets and model selection criteria. \n", "\n", "At the end of each Part, complete the submission cells with implementations, answer all questions, and pass the unit tests. Then submit the completed notebook for review!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## PART 1: Data\n", "\n", "### Features Tutorial\n", "##### Load the initial database\n", "A data handler designed for this database is provided in the student codebase as the `AslDb` class in the `asl_data` module. This handler creates the initial [pandas](http://pandas.pydata.org/pandas-docs/stable/) dataframe from the corpus of data included in the `data` directory as well as dictionaries suitable for extracting data in a format friendly to the [hmmlearn](https://hmmlearn.readthedocs.io/en/latest/) library. We'll use those to create models in Part 2.\n", "\n", "To start, let's set up the initial database and select an example set of features for the training set. At the end of Part 1, you will create additional feature sets for experimentation. " ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The autoreload extension is already loaded. To reload it, use:\n", " %reload_ext autoreload\n" ] } ], "source": [ "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
left-xleft-yright-xright-ynose-xnose-yspeaker
videoframe
98014918117017516162woman-1
114918117017516162woman-1
214918117017516162woman-1
314918117017516162woman-1
414918117017516162woman-1
\n", "
" ], "text/plain": [ " left-x left-y right-x right-y nose-x nose-y speaker\n", "video frame \n", "98 0 149 181 170 175 161 62 woman-1\n", " 1 149 181 170 175 161 62 woman-1\n", " 2 149 181 170 175 161 62 woman-1\n", " 3 149 181 170 175 161 62 woman-1\n", " 4 149 181 170 175 161 62 woman-1" ] }, "execution_count": 63, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "from asl_data import AslDb\n", "\n", "asl = AslDb() # initializes the database\n", "asl.df.head() # displays the first five rows of the asl database, indexed by video and frame" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "left-x 149\n", "left-y 181\n", "right-x 170\n", "right-y 175\n", "nose-x 161\n", "nose-y 62\n", "speaker woman-1\n", "Name: (98, 1), dtype: object" ] }, "execution_count": 64, "metadata": {}, "output_type": "execute_result" } ], "source": [ "asl.df.ix[98, 1] # look at the data available for an individual frame" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The frame represented by video 98, frame 1 is shown here:\n", "![Video 98](http://www-i6.informatik.rwth-aachen.de/~dreuw/database/rwth-boston-104/overview/images/orig/098-start.jpg)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Feature selection for training the model\n", "The objective of feature selection when training a model is to choose the most relevant variables while keeping the model as simple as possible, thus reducing training time. We can use the raw features already provided or derive our own and add columns to the pandas dataframe `asl.df` for selection. As an example, in the next cell a feature named `'grnd-ry'` is added. This feature is the difference between the right-hand y value and the nose y value, which serves as the \"ground\" right y value. " ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
left-xleft-yright-xright-ynose-xnose-yspeakergrnd-ry
videoframe
98014918117017516162woman-1113
114918117017516162woman-1113
214918117017516162woman-1113
314918117017516162woman-1113
414918117017516162woman-1113
\n", "
" ], "text/plain": [ " left-x left-y right-x right-y nose-x nose-y speaker \\\n", "video frame \n", "98 0 149 181 170 175 161 62 woman-1 \n", " 1 149 181 170 175 161 62 woman-1 \n", " 2 149 181 170 175 161 62 woman-1 \n", " 3 149 181 170 175 161 62 woman-1 \n", " 4 149 181 170 175 161 62 woman-1 \n", "\n", " grnd-ry \n", "video frame \n", "98 0 113 \n", " 1 113 \n", " 2 113 \n", " 3 113 \n", " 4 113 " ] }, "execution_count": 65, "metadata": {}, "output_type": "execute_result" } ], "source": [ "asl.df['grnd-ry'] = asl.df['right-y'] - asl.df['nose-y']\n", "asl.df.head() # the new feature 'grnd-ry' is now in the frames dictionary" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Try it!" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "asl.df sample\n" ] }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
left-xleft-yright-xright-ynose-xnose-yspeakergrnd-rygrnd-lygrnd-lxgrnd-rx
videoframe
98014918117017516162woman-1113119-129
114918117017516162woman-1113119-129
214918117017516162woman-1113119-129
314918117017516162woman-1113119-129
414918117017516162woman-1113119-129
\n", "
" ], "text/plain": [ " left-x left-y right-x right-y nose-x nose-y speaker \\\n", "video frame \n", "98 0 149 181 170 175 161 62 woman-1 \n", " 1 149 181 170 175 161 62 woman-1 \n", " 2 149 181 170 175 161 62 woman-1 \n", " 3 149 181 170 175 161 62 woman-1 \n", " 4 149 181 170 175 161 62 woman-1 \n", "\n", " grnd-ry grnd-ly grnd-lx grnd-rx \n", "video frame \n", "98 0 113 119 -12 9 \n", " 1 113 119 -12 9 \n", " 2 113 119 -12 9 \n", " 3 113 119 -12 9 \n", " 4 113 119 -12 9 " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "Correct!
" ], "text/plain": [ "" ] }, "execution_count": 66, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from asl_utils import test_features_tryit\n", "# TODO add df columns for 'grnd-rx', 'grnd-ly', 'grnd-lx' representing\n", "# differences between hand and nose locations\n", "\n", "asl.df['grnd-ly'] = asl.df['left-y'] - asl.df['nose-y']\n", "asl.df['grnd-lx'] = asl.df['left-x'] - asl.df['nose-x']\n", "asl.df['grnd-rx'] = asl.df['right-x'] - asl.df['nose-x']\n", "\n", "# test the code\n", "test_features_tryit(asl)" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[9, 113, -12, 119]" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# collect the features into a list\n", "features_ground = ['grnd-rx', 'grnd-ry', 'grnd-lx', 'grnd-ly']\n", "# show a single set of features for a given (video, frame) tuple\n", "[asl.df.ix[98, 1][v] for v in features_ground]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Build the training set\n", "Now that we have a feature list defined, we can pass that list to the `build_training` method to collect the features for all the words in the training set. Each word in the training set has multiple examples from various videos. Below we can see the unique words that have been loaded into the training set:" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training words: ['JOHN', 'WRITE', 'HOMEWORK', 'IX-1P', 'SEE', 'YESTERDAY', 'IX', 'LOVE', 'MARY', 'CAN', 'GO', 'GO1', 'FUTURE', 'GO2', 'PARTY', 'FUTURE1', 'HIT', 'BLAME', 'FRED', 'FISH', 'WONT', 'EAT', 'BUT', 'CHICKEN', 'VEGETABLE', 'CHINA', 'PEOPLE', 'PREFER', 'BROCCOLI', 'LIKE', 'LEAVE', 'SAY', 'BUY', 'HOUSE', 'KNOW', 'CORN', 'CORN1', 'THINK', 'NOT', 'PAST', 'LIVE', 'CHICAGO', 'CAR', 'SHOULD', 'DECIDE', 'VISIT', 'MOVIE', 'WANT', 'SELL', 'TOMORROW', 'NEXT-WEEK', 'NEW-YORK', 'LAST-WEEK', 'WILL', 'FINISH', 'ANN', 'READ', 'BOOK', 'CHOCOLATE', 'FIND', 'SOMETHING-ONE', 'POSS', 'BROTHER', 'ARRIVE', 'HERE', 'GIVE', 'MAN', 'NEW', 'COAT', 'WOMAN', 'GIVE1', 'HAVE', 'FRANK', 'BREAK-DOWN', 'SEARCH-FOR', 'WHO', 'WHAT', 'LEG', 'FRIEND', 'CANDY', 'BLUE', 'SUE', 'BUY1', 'STOLEN', 'OLD', 'STUDENT', 'VIDEOTAPE', 'BORROW', 'MOTHER', 'POTATO', 'TELL', 'BILL', 'THROW', 'APPLE', 'NAME', 'SHOOT', 'SAY-1P', 'SELF', 'GROUP', 'JANA', 'TOY1', 'MANY', 'TOY', 'ALL', 'BOY', 'TEACHER', 'GIRL', 'BOX', 'GIVE2', 'GIVE3', 'GET', 'PUTASIDE']\n" ] } ], "source": [ "training = asl.build_training(features_ground)\n", "print(\"Training words: {}\".format(training.words))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The training data in `training` is an object of class `WordsData` defined in the `asl_data` module. in addition to the `words` list, data can be accessed with the `get_all_sequences`, `get_all_Xlengths`, `get_word_sequences`, and `get_word_Xlengths` methods. We need the `get_word_Xlengths` method to train multiple sequences with the `hmmlearn` library. In the following example, notice that there are two lists; the first is a concatenation of all the sequences(the X portion) and the second is a list of the sequence lengths(the Lengths portion)." ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array([[-11, 48, 7, 120],\n", " [-11, 48, 8, 109],\n", " [ -8, 49, 11, 98],\n", " [ -7, 50, 7, 87],\n", " [ -4, 54, 7, 77],\n", " [ -4, 54, 6, 69],\n", " [ -4, 54, 6, 69],\n", " [-13, 52, 6, 69],\n", " [-13, 52, 6, 69],\n", " [ -8, 51, 6, 69],\n", " [ -8, 51, 6, 69],\n", " [ -8, 51, 6, 69],\n", " [ -8, 51, 6, 69],\n", " [ -8, 51, 6, 69],\n", " [-10, 59, 7, 71],\n", " [-15, 64, 9, 77],\n", " [-17, 75, 13, 81],\n", " [ -4, 48, -4, 113],\n", " [ -2, 53, -4, 113],\n", " [ -4, 55, 2, 98],\n", " [ -4, 58, 2, 98],\n", " [ -1, 59, 2, 89],\n", " [ -1, 59, -1, 84],\n", " [ -1, 59, -1, 84],\n", " [ -7, 63, -1, 84],\n", " [ -7, 63, -1, 84],\n", " [ -7, 63, 3, 83],\n", " [ -7, 63, 3, 83],\n", " [ -7, 63, 3, 83],\n", " [ -7, 63, 3, 83],\n", " [ -7, 63, 3, 83],\n", " [ -7, 63, 3, 83],\n", " [ -7, 63, 3, 83],\n", " [ -4, 70, 3, 83],\n", " [ -4, 70, 3, 83],\n", " [ -2, 73, 5, 90],\n", " [ -3, 79, -4, 96],\n", " [-15, 98, 13, 135],\n", " [ -6, 93, 12, 128],\n", " [ -2, 89, 14, 118],\n", " [ 5, 90, 10, 108],\n", " [ 4, 86, 7, 105],\n", " [ 4, 86, 7, 105],\n", " [ 4, 86, 13, 100],\n", " [ -3, 82, 14, 96],\n", " [ -3, 82, 14, 96],\n", " [ 6, 89, 16, 100],\n", " [ 6, 89, 16, 100],\n", " [ 7, 85, 17, 111]], dtype=int64), [17, 20, 12])" ] }, "execution_count": 69, "metadata": {}, "output_type": "execute_result" } ], "source": [ "training.get_word_Xlengths('CHOCOLATE')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### More feature sets\n", "So far we have a simple feature set that is enough to get started modeling. However, we might get better results if we manipulate the raw values a bit more, so we will go ahead and set up some other options now for experimentation later. For example, we could normalize each speaker's range of motion with grouped statistics using [Pandas stats](http://pandas.pydata.org/pandas-docs/stable/api.html#api-dataframe-stats) functions and [pandas groupby](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html). Below is an example for finding the means of all speaker subgroups." ] }, { "cell_type": "code", "execution_count": 70, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
left-xleft-yright-xright-ynose-xnose-ygrnd-rygrnd-lygrnd-lxgrnd-rx
speaker
man-1206.248203218.679449155.464350150.371031175.03175661.64260088.728430157.03684831.216447-19.567406
woman-1164.661438161.271242151.017865117.332462162.65512057.24509860.087364104.0261442.006318-11.637255
woman-2183.214509176.527232156.866295119.835714170.31897358.02209861.813616118.50513412.895536-13.452679
\n", "
" ], "text/plain": [ " left-x left-y right-x right-y nose-x \\\n", "speaker \n", "man-1 206.248203 218.679449 155.464350 150.371031 175.031756 \n", "woman-1 164.661438 161.271242 151.017865 117.332462 162.655120 \n", "woman-2 183.214509 176.527232 156.866295 119.835714 170.318973 \n", "\n", " nose-y grnd-ry grnd-ly grnd-lx grnd-rx \n", "speaker \n", "man-1 61.642600 88.728430 157.036848 31.216447 -19.567406 \n", "woman-1 57.245098 60.087364 104.026144 2.006318 -11.637255 \n", "woman-2 58.022098 61.813616 118.505134 12.895536 -13.452679 " ] }, "execution_count": 70, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_means = asl.df.groupby('speaker').mean()\n", "df_means" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To select a mean that matches by speaker, use the pandas [map](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.map.html) method:" ] }, { "cell_type": "code", "execution_count": 71, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
left-xleft-yright-xright-ynose-xnose-yspeakergrnd-rygrnd-lygrnd-lxgrnd-rxleft-x-mean
videoframe
98014918117017516162woman-1113119-129164.661438
114918117017516162woman-1113119-129164.661438
214918117017516162woman-1113119-129164.661438
314918117017516162woman-1113119-129164.661438
414918117017516162woman-1113119-129164.661438
\n", "
" ], "text/plain": [ " left-x left-y right-x right-y nose-x nose-y speaker \\\n", "video frame \n", "98 0 149 181 170 175 161 62 woman-1 \n", " 1 149 181 170 175 161 62 woman-1 \n", " 2 149 181 170 175 161 62 woman-1 \n", " 3 149 181 170 175 161 62 woman-1 \n", " 4 149 181 170 175 161 62 woman-1 \n", "\n", " grnd-ry grnd-ly grnd-lx grnd-rx left-x-mean \n", "video frame \n", "98 0 113 119 -12 9 164.661438 \n", " 1 113 119 -12 9 164.661438 \n", " 2 113 119 -12 9 164.661438 \n", " 3 113 119 -12 9 164.661438 \n", " 4 113 119 -12 9 164.661438 " ] }, "execution_count": 71, "metadata": {}, "output_type": "execute_result" } ], "source": [ "asl.df['left-x-mean'] = asl.df['speaker'].map(df_means['left-x'])\n", "asl.df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Try it!" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "df_std\n" ] }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
left-xleft-yright-xright-ynose-xnose-ygrnd-rygrnd-lygrnd-lxgrnd-rxleft-x-mean
speaker
man-115.15442536.32848518.90191754.9023406.6545735.52004553.48799936.57274915.08036020.2690320.0
woman-117.57344226.59452116.45994334.6677873.5493923.53833033.97266027.11739317.32894116.7647060.0
woman-215.38871128.82502514.89028839.6491114.0997603.41616739.12857229.32065515.05093816.1913240.0
\n", "
" ], "text/plain": [ " left-x left-y right-x right-y nose-x nose-y \\\n", "speaker \n", "man-1 15.154425 36.328485 18.901917 54.902340 6.654573 5.520045 \n", "woman-1 17.573442 26.594521 16.459943 34.667787 3.549392 3.538330 \n", "woman-2 15.388711 28.825025 14.890288 39.649111 4.099760 3.416167 \n", "\n", " grnd-ry grnd-ly grnd-lx grnd-rx left-x-mean \n", "speaker \n", "man-1 53.487999 36.572749 15.080360 20.269032 0.0 \n", "woman-1 33.972660 27.117393 17.328941 16.764706 0.0 \n", "woman-2 39.128572 29.320655 15.050938 16.191324 0.0 " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "Correct!
" ], "text/plain": [ "" ] }, "execution_count": 72, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from asl_utils import test_std_tryit\n", "# TODO Create a dataframe named `df_std` with standard deviations grouped\n", "# by speaker\n", "df_std = asl.df.groupby(\"speaker\").std()\n", "\n", "# test the code\n", "test_std_tryit(df_std)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Features Implementation Submission\n", "Implement four feature sets and answer the question that follows.\n", "- normalized Cartesian coordinates\n", " - use *mean* and *standard deviation* statistics and the [standard score](https://en.wikipedia.org/wiki/Standard_score) equation to account for speakers with different heights and arm length\n", " \n", "- polar coordinates\n", " - calculate polar coordinates with [Cartesian to polar equations](https://en.wikipedia.org/wiki/Polar_coordinate_system#Converting_between_polar_and_Cartesian_coordinates)\n", " - use the [np.arctan2](https://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.arctan2.html) function and *swap the x and y axes* to move the $0$ to $2\\pi$ discontinuity to 12 o'clock instead of 3 o'clock; in other words, the normal break in radians value from $0$ to $2\\pi$ occurs directly to the left of the speaker's nose, which may be in the signing area and interfere with results. By swapping the x and y axes, that discontinuity move to directly above the speaker's head, an area not generally used in signing.\n", "\n", "- delta difference\n", " - as described in Thad's lecture, use the difference in values between one frame and the next frames as features\n", " - pandas [diff method](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.diff.html) and [fillna method](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html) will be helpful for this one\n", "\n", "- custom features\n", " - These are your own design; combine techniques used above or come up with something else entirely. We look forward to seeing what you come up with! \n", " Some ideas to get you started:\n", " - normalize using a [feature scaling equation](https://en.wikipedia.org/wiki/Feature_scaling)\n", " - normalize the polar coordinates\n", " - adding additional deltas\n" ] }, { "cell_type": "code", "execution_count": 73, "metadata": { "collapsed": true, "scrolled": false }, "outputs": [], "source": [ "# TODO add features for normalized by speaker values of left, right, x, y\n", "# Name these 'norm-rx', 'norm-ry', 'norm-lx', and 'norm-ly'\n", "# using Z-score scaling (X-Xmean)/Xstd\n", "\n", "\n", "def normalize(df, label):\n", " numerator = df[label] - df['speaker'].map(df.groupby(\"speaker\").mean()[label])\n", " denominator = df['speaker'].map(df.groupby(\"speaker\").std()[label])\n", "\n", " return numerator / denominator\n", "\n", "\n", "asl.df['norm-rx'] = normalize(asl.df, 'right-x')\n", "asl.df['norm-ry'] = normalize(asl.df, 'right-y')\n", "asl.df['norm-lx'] = normalize(asl.df, 'left-x')\n", "asl.df['norm-ly'] = normalize(asl.df, 'left-y')\n", "features_norm = ['norm-rx', 'norm-ry', 'norm-lx', 'norm-ly']" ] }, { "cell_type": "code", "execution_count": 74, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# TODO add features for polar coordinate values where the nose is the origin\n", "# Name these 'polar-rr', 'polar-rtheta', 'polar-lr', and 'polar-ltheta'\n", "# Note that 'polar-rr' and 'polar-rtheta' refer to the radius and angle\n", "\n", "\n", "def cart2polar(x_series, y_series):\n", " r = np.sqrt(x_series**2 + y_series**2)\n", " theta = np.arctan2(x_series, y_series)\n", "\n", " return r, theta\n", "\n", "\n", "asl.df['polar-rr'], asl.df['polar-rtheta'] = cart2polar(\n", " asl.df['grnd-rx'], asl.df['grnd-ry'])\n", "asl.df['polar-lr'], asl.df['polar-ltheta'] = cart2polar(\n", " asl.df['grnd-lx'], asl.df['grnd-ly'])\n", "\n", "features_polar = ['polar-rr', 'polar-rtheta', 'polar-lr', 'polar-ltheta']" ] }, { "cell_type": "code", "execution_count": 75, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# TODO add features for left, right, x, y differences by one time step, i.e. the \"delta\" values discussed in the lecture\n", "# Name these 'delta-rx', 'delta-ry', 'delta-lx', and 'delta-ly'\n", "asl.df['delta-rx'] = asl.df['right-x'].diff()\n", "asl.df['delta-lx'] = asl.df['left-x'].diff()\n", "asl.df['delta-ry'] = asl.df['right-y'].diff()\n", "asl.df['delta-ly'] = asl.df['left-y'].diff()\n", "\n", "beg_idx = [(vid, frame) for vid, frame in asl.df.index.tolist() if frame == 0]\n", "asl.df.loc[beg_idx, ['delta-rx', 'delta-lx', 'delta-ry', 'delta-ly']] = 0\n", "\n", "features_delta = ['delta-rx', 'delta-ry', 'delta-lx', 'delta-ly']" ] }, { "cell_type": "code", "execution_count": 178, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# TODO add features of your own design, which may be a combination of the above or something else\n", "# Name these whatever you would like\n", "asl.df['delta-norm-rx'] = normalize(asl.df, 'delta-rx')\n", "asl.df['delta-norm-ry'] = normalize(asl.df, 'delta-ry')\n", "asl.df['delta-norm-lx'] = normalize(asl.df, 'delta-lx')\n", "asl.df['delta-norm-ly'] = normalize(asl.df, 'delta-ly')\n", "# TODO define a list named 'features_custom' for building the training set\n", "features_custom = ['delta-norm-rx', 'delta-norm-ry', 'delta-norm-lx', 'delta-norm-ly']" ] }, { "cell_type": "code", "execution_count": 182, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['delta-rx',\n", " 'delta-ry',\n", " 'delta-lx',\n", " 'delta-ly',\n", " 'grnd-rx',\n", " 'grnd-ry',\n", " 'grnd-lx',\n", " 'grnd-ly',\n", " 'norm-rx',\n", " 'norm-ry',\n", " 'norm-lx',\n", " 'norm-ly',\n", " 'polar-rr',\n", " 'polar-rtheta',\n", " 'polar-lr',\n", " 'polar-ltheta',\n", " 'delta-norm-rx',\n", " 'delta-norm-ry',\n", " 'delta-norm-lx',\n", " 'delta-norm-ly']" ] }, "execution_count": 182, "metadata": {}, "output_type": "execute_result" } ], "source": [ "features_all = [features_delta, features_ground, features_norm, features_polar, features_custom]\n", "features_all = sum(features_all, [])\n", "features_all" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Question 1:** What custom features did you choose for the features_custom set and why?\n", "\n", "**Answer 1:**\n", "\n", "I chose `delta-norm` features which is obtained by standardizing `delta` features.\n", "By standardizing, I mean\n", "\n", "$$ z = \\frac{X - \\mu}{\\sigma} $$\n", "\n", "Why did I choose these features?\n", "\n", "* Each person has a different body shape(arm width, height, etc.) Hence, it can result in significantly different signals\n", "* It also keeps numbers small such that it makes the prediction easier because predicting the range of -1 ~ 1 is easier than predicting -100 ~ 100." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Features Unit Testing\n", "Run the following unit tests as a sanity check on the defined \"ground\", \"norm\", \"polar\", and 'delta\"\n", "feature sets. The test simply looks for some valid values but is not exhaustive. However, the project should not be submitted if these tests don't pass." ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "....\n", "----------------------------------------------------------------------\n", "Ran 4 tests in 0.005s\n", "\n", "OK\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 77, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import unittest\n", "\n", "# import numpy as np\n", "\n", "\n", "class TestFeatures(unittest.TestCase):\n", " def test_features_ground(self):\n", " sample = (asl.df.ix[98, 1][features_ground]).tolist()\n", " self.assertEqual(sample, [9, 113, -12, 119])\n", "\n", " def test_features_norm(self):\n", " sample = (asl.df.ix[98, 1][features_norm]).tolist()\n", " np.testing.assert_almost_equal(sample, [1.153, 1.663, -0.891, 0.742],\n", " 3)\n", "\n", " def test_features_polar(self):\n", " sample = (asl.df.ix[98, 1][features_polar]).tolist()\n", " np.testing.assert_almost_equal(sample,\n", " [113.3578, 0.0794, 119.603, -0.1005], 3)\n", "\n", " def test_features_delta(self):\n", " sample = (asl.df.ix[98, 0][features_delta]).tolist()\n", " self.assertEqual(sample, [0, 0, 0, 0])\n", " sample = (asl.df.ix[98, 18][features_delta]).tolist()\n", " self.assertTrue(sample in [[-16, -5, -2, 4], [-14, -9, 0, 0]],\n", " \"Sample value found was {}\".format(sample))\n", "\n", "\n", "suite = unittest.TestLoader().loadTestsFromModule(TestFeatures())\n", "unittest.TextTestRunner().run(suite)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "\n", "## PART 2: Model Selection\n", "### Model Selection Tutorial\n", "The objective of Model Selection is to tune the number of states for each word HMM prior to testing on unseen data. In this section you will explore three methods: \n", "- Log likelihood using cross-validation folds (CV)\n", "- Bayesian Information Criterion (BIC)\n", "- Discriminative Information Criterion (DIC) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Train a single word\n", "Now that we have built a training set with sequence data, we can \"train\" models for each word. As a simple starting example, we train a single word using Gaussian hidden Markov models (HMM). By using the `fit` method during training, the [Baum-Welch Expectation-Maximization](https://en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm) (EM) algorithm is invoked iteratively to find the best estimate for the model *for the number of hidden states specified* from a group of sample seequences. For this example, we *assume* the correct number of hidden states is 3, but that is just a guess. How do we know what the \"best\" number of states for training is? We will need to find some model selection technique to choose the best parameter." ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of states trained in model for BOOK is 3\n", "logL = -2331.1138127433196\n" ] } ], "source": [ "import warnings\n", "from hmmlearn.hmm import GaussianHMM\n", "\n", "\n", "def train_a_word(word, num_hidden_states, features):\n", " warnings.filterwarnings(\"ignore\", category=DeprecationWarning)\n", "\n", " training = asl.build_training(features)\n", " X, lengths = training.get_word_Xlengths(word)\n", " model = GaussianHMM(n_components=num_hidden_states,\n", " n_iter=1000).fit(X, lengths)\n", " logL = model.score(X, lengths)\n", " return model, logL\n", "\n", "\n", "demoword = 'BOOK'\n", "model, logL = train_a_word(demoword, 3, features_ground)\n", "print(\"Number of states trained in model for {} is {}\".format(\n", " demoword, model.n_components))\n", "print(\"logL = {}\".format(logL))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The HMM model has been trained and information can be pulled from the model, including means and variances for each feature and hidden state. The [log likelihood](http://math.stackexchange.com/questions/892832/why-we-consider-log-likelihood-instead-of-likelihood-in-gaussian-distribution) for any individual sample or group of samples can also be calculated with the `score` method." ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of states trained in model for BOOK is 3\n", "hidden state #0\n", "mean = [ -3.46504869 50.66686933 14.02391587 52.04731066]\n", "variance = [ 49.12346305 43.04799144 39.35109609 47.24195772]\n", "\n", "hidden state #1\n", "mean = [ -11.45300909 94.109178 19.03512475 102.2030162 ]\n", "variance = [ 77.403668 203.35441965 26.68898447 156.12444034]\n", "\n", "hidden state #2\n", "mean = [ -1.12415027 69.44164191 17.02866283 77.7231196 ]\n", "variance = [ 19.70434594 16.83041492 30.51552305 11.03678246]\n", "\n" ] } ], "source": [ "def show_model_stats(word, model):\n", " print(\"Number of states trained in model for {} is {}\".format(\n", " word, model.n_components))\n", "\n", " variance = np.array([np.diag(model.covars_[i])\n", " for i in range(model.n_components)])\n", "\n", " for i in range(model.n_components): # for each hidden state\n", " print(\"hidden state #{}\".format(i))\n", " print(\"mean = \", model.means_[i])\n", " print(\"variance = \", variance[i])\n", " print()\n", "\n", "\n", "show_model_stats(demoword, model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Try it!\n", "Experiment by changing the feature set, word, and/or num_hidden_states values in the next cell to see changes in values. " ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of states trained in model for CHOCOLATE is 3\n", "hidden state #0\n", "mean = [ -9.30211403 55.32333876 6.92259936 71.24057775]\n", "variance = [ 16.16920957 46.50917372 3.81388185 15.79446427]\n", "\n", "hidden state #1\n", "mean = [ 0.58333333 87.91666667 12.75 108.5 ]\n", "variance = [ 39.41055556 18.74388889 9.855 144.4175 ]\n", "\n", "hidden state #2\n", "mean = [ -5.40587658 60.1652424 2.32479599 91.3095432 ]\n", "variance = [ 7.95073876 64.13103127 13.68077479 129.5912395 ]\n", "\n", "logL = -601.3291470028638\n" ] } ], "source": [ "my_testword = 'CHOCOLATE'\n", "\n", "# Experiment here with different parameters\n", "model, logL = train_a_word(my_testword, 3, features_ground)\n", "\n", "show_model_stats(my_testword, model)\n", "print(\"logL = {}\".format(logL))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Visualize the hidden states\n", "We can plot the means and variances for each state and feature. Try varying the number of states trained for the HMM model and examine the variances. Are there some models that are \"better\" than others? How can you tell? We would like to hear what you think in the classroom online." ] }, { "cell_type": "code", "execution_count": 81, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 82, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXl4lNXZ/z83WVgS9khYJSgoosgWwZ1YrQWX4gKouGuL\n9Fetb9W29tXrBX1ra32rrbtiW5e6olZFRVGoEVGQTZRNFiHs+x4gCUnu3x/nBIdxZjJJZjKT5P5c\n13PNM8/Z7nPmmfM923MeUVUMwzAMo1GiDTAMwzCSAxMEwzAMAzBBMAzDMDwmCIZhGAZggmAYhmF4\nTBAMwzAMwATBqGOISLaITBORvSLyYKLtiSciMk5EXozgvkhE8sK45YnIughhnxORP8TATKMeYYKQ\nQERklIjMEZFCEdkoIh+IyOneLWRlICIqIt0DvvcSkYkisttXkp+IyKlBYdJ9fMtFZJ+IFIjIP0Uk\nJ8DPBSIyy7tvF5GXRKRzgPt1IjK9kvw8JyKlItIh4NpTPn+FIlIiIgcDvn8gIjk+T4VBx2VhkhkN\nbANaqOrtEQu4EhJRKfr8fiIi+0XkWxE5p7pxqerxqpofQ/PiSvC9G4X/fBH5WYzSniAi54pIYxHZ\nFOTW2P8f9ojIJhG5LRZp1kVMEBKEv+n+BvwRyAaOBB4HflqFOI4GPgcWAN2AjsBbwEcickqA1zd8\nvKOAlkAfYA5wto9nOPCytycLOB4oBqaLSOsobckALgV2A1dVXFfVMaqaqaqZPq+vVXxX1aEBUbQK\nuJ6pqq+FSaorsFiT4IlKEUmtRrBXgK+AtsBdwBsickRMDTNCMQB3z58ILAxyGwf0wN1bZwG/FZEh\ntWpdsqCqdtTygauUC4EREfyMA14McV2B7v78X8CkEH6eBKb583OAA0CXMOkIsBr4bdD1Rrg/zr3+\n+3XA9Aj2XgOsBW4FFkabJyDH5yk1inJ7DjgIlPjyO8fbeSfwHbAdmAC0CQjzOrAJJ1TTgOP99dFB\ncb0bXL4Baf7Bn+cB64Df+Tj/5a9fAMwHdgFfACeGsf8YnNA2D7g2DRgTobwmAC8Ae4FFQG6AewFw\njj9v6m3dCSwGfgOsC/DbD5jn43kNeLUiX5XlwadzB/CNL8fXgCZhbO4OfOr9bcM1ACryqcA+X96X\nAa2B94Ct3u73gM7e/31AGVDk/T/mr/cEPgZ2AEuBkVHcN62Blf78F8ADQe4bgHMDvt8LvJqo+iGR\nR8INaIgHMAQoJUIlSHSCsAm4PoSfs/yfqSlwP/BphHR6+ji7hXC7B5jhz68jsiBMBR7A9XZKgQHR\n5IkqCIL3/1xQRXYrMBPoDDQGngZeCXC/AWju3f4GzA8XV3D5BvvBCUIp8GcfX1NcRbsFGASkANfi\nKtDGIWy/GFgSdO1R4NEI90ARcJ6P+0/AzAD3Ar4XhPuBz4A2QBecmK/zbuk40f81kAYMx4lhRb4i\n5sGfz8L1QNsASwgvYq/gej6NgCbA6RHKti2uV9nM/0avA28HuOcDPwv4noFrdFwPpHq7twG9wthy\nNk7gCnHCv8t/7vPng3FioUB2QLhLgQWJqh8SediQUWJoC2xT1dJK/I0UkV2BR5B7FrAxRLiNuD9k\nG59WKD+BcRDGz8YA97CIyJE4EXpZVTfjxOGaysIFsS0or8dFGW4McJeqrlPVYlwlOrxiOEdV/6mq\newPc+ohIyyraFkg5MFZVi1X1AK6n8bSqfqmqZar6PK4XcHKIsJm4lnMge3CVYTimq+okVS3D9Qj7\nhPE3ErhPVXeo6lrgkQC3k3FC8DdVPaiqbwCzA9yjycMjqrpBVXcA7wJ9w9hxEDf00lFVi1Q17LyT\nqm5X1TdVdb+q7sX1CgaH84/rxRSo6rOqWqqqXwFvAiPCxD9VVVsBb3s/nXDilqWqrVT1U9xvAof/\nLpX9JvUWE4TEsB3IimIMeoK/cQ8dQe7bgA4hwnXAVVw7fVqh/ATGQRg/HQLcI3E1ruU7339/CRgl\nImlRhK0gKyivS6IM1xV4K0Awl+B6R9kikiIi94vIdyKyB1cZQBQiF4GtqloUlP7tQaLdBdeaDqYQ\naBF0rSVuGCccgROg+4EmYe6bjrjWcwWrg9zWq2/+hnCPJg/BdmQSmt/ihiFn+VVQN4Txh4g0E5Gn\nRWS1/32mAa1EJCVMkK7AoCA7rwTah4l/nfdzBfA8rhfUFdgoIg95b4X+M/B3qew3qbeYICSGGbgW\n2EU1jGcKoVtHI3FDPfu9n4GBK4aCWIobFz8sHhFphOs6T43CjmuAo/wKjU3AQ7hK97yoclEz1gJD\ng8Skiaqux02iD8PNNbTEDU+Bq7DADRUEsx83hFFBcGUTHGYtrmUemH4zVX0lRNyLcOUU2Prs46/X\nlI24SryCI4PcOomIhHGvSh4ioqqbVPXnqtoRuAl4IsLKotuBY4FBqtoCONNfD/f7rMUNfwbamamq\nvwhjS2fc8OwU35gaD/zSh7vN+9mJK5/AnlesfpM6hwlCAlDV3cD/AI+LyEW+pZQmIkNF5IEqRHUP\ncKqI3CcibUSkuYjcgqugf+fTmoKbhHtLRAaISKr3N0ZEbvCtxjuAu8Utg20iIu2Bv+NaTX8NSE+8\ne+BxCnA0MBA3jNAXOAG3aqmqw0bV4SngPhHp6g08QkSGebfmOOHdjqvk/xgUdjNwVNC1+bjeTYpf\naRJpCAPgGWCMiAwSR4aInB9U6QOgqst8/GN92V0C9MYNe9SUCcDvRaS1F/9bAtxm4OY+fuXvs0tw\nv1eV81AZIjIioPGxE1epl/vvweXdHLfgYZeItAHGBkUX7P894BgRudrnI01ETqpkeHEAbjIdoD9u\npVEwL+Du/9Y+rp/j5o4aHCYICUJVHwRuA+7GrbJYC9yMG++MNo7lwOm4Fk0BrqVzKfATVf08wOtw\nYBJudchu3IRjLq73gLolnlfjJh2341apNAVOU9XtAfGcivsDBx43Au+o6gLfOtykqpuAh4EL/B89\nGnbJ4c8hRLsW/GFgIm6p7V7cBPMg7/YCbmhkvc/TzKCw/wB6+eGHinK/FbgQN+l4JZX8Hqo6B1eB\nPIarAFfgJuDDcTmu7HfiJomHq+rWSnNZOffg8roK+Ag331BhYwlwibdrB26Fz79rkIdInAR8KSKF\nuN/lVlVd6d3GAc/78h6Jm+RvihuWnAl8GBTXw7j5oJ0i8oifZzgXV4YbcMNYFRP84RgAzPO9o56E\nbvmPxa1SW42byH5AVYNtaRDI4cOKhmEYRkPFegiGYRgGYIJgGIZheEwQDMMwDMAEwTAMw/BUZ3Ou\nQ/hleQ/jHnf/u6reH+TeE3gWt9zrLlX9S7RhQ5GVlaU5OTk1MbnW2LdvHxkZGYk2I+mwcgmNlUto\nrFxCU9VymTt37jZVrXQTxWoLgn+a8HHgx7gHm2aLyERVXRzgbQfwK4IewIoy7A/IyclhzpxQy4iT\nj/z8fPLy8hJtRtJh5RIaK5fQWLmEpqrlIiKrK/dVsyGjgcAKVV3p1zm/insq9BCqukVVZ+P2N6lS\nWMOoDXauhDlPw+41ibbEMBJPTYaMOnH43inr+P6BoJiFFZHRuM23yM7OJj8/v8qGJoLCwsI6Y2tt\nkizlsm9VBmtePpIt/2kH5QKNlLYnb6fjxetpk7uz1u1JlnJJNqxcQhOvcqnRHEJtoKrjcXuQkJub\nq3Wl+2hd3dAkQ7nM+zt8+nNIy4BTboMTLodv3xLmPZPFgt9kce5DcMqva9emZCiXZMTKJTTxKpea\nCMJ6Dt9Mq7O/Fu+whlFtNsyBSb+Eo34Ml74Czdq66x0HwOD/gTcuh49/A+37QLcfJdZWw6htajKH\nMBvoISLdRCQdt7/IxFoIaxjV4sAOmDAcMrLh0pe/F4MKUtLhouch61h4fSTsKkiImYaRMKotCP7l\nLjcDk3F70E9Q1UV+F80xACLSXkTW4Tdx8/uTtwgXtqaZMYxwaDm8dQ3s3QAjXodmYd6I0Lg5XPY2\nlJfCaxfDwf21a6dhJJIazSGo6iTcLpqB154KON+EGw6KKqxhxIs5T8Hy92Hoo9C5kqUPbXvAJS/B\nKxfA5w9A3rhaMdEwEo49qWzUew7uh0/vha6D4aRfRhfmmPOh58Uw4yHYv71y/4ZRHzBBMOo9s5+A\nfZvhR3+Aw94ZVgln/S+UFMLnf46fbYaRTJggGPWa4r2uQj/6XDjy9KqFbXc8nHgVzHrUzT0YRn3H\nBMGo18x6FPZvc6396pA3zk0wT/tDTM0yjKTEBMGotxTthi/+AsdcAJ0GVu4/FK2Pgn4/g3nPuG0u\nDKM+Y4Jg1FtmPQpFOyHv3prFc+bdII3cBLNh1GdMEIx6SVmJm0zuPgQ69KtZXC06ue0tvn7e9ToM\no75igmDUS5b8Gwo3wsBbYhPfwFvciqP5z8UmPsNIRkwQjHrJrMeg9dGuhxALOuZC51Ng9mPuqWfD\nqI+YIBj1jo1fwdrP3UNoEsM7fOAtsGMFrPgwdnEaRjJhgmDUO2Y9CmnNoN/1sY2316WQ2cHFbxj1\nERMEo16xfzsseBlOvBqatIpt3CnpkPsL10PYviy2cRtGMmCCYNQrvvoHlBXDwJvjE/+A0U4YZj8R\nn/gNI5GYIBj1Bi2HuU+7TezanRCfNDKz4bhL4OsX4OCB+KRhGInCBMGoN6yc6p4mHnBTfNPpP9o9\n8Lb4jfimYxi1jQmCUW+Y+zQ0beta8PEkJw/a9IB54+ObjmHUNiYIRr2gcBMsfQf6XgepjeObloib\nS1gzHbbYe/6MeoQJglEv+OpZtyvpgNG1k17f69zk8lzrJRj1CBMEo86j5W430pw8aHtM7aTZLMsN\nTX1jk8tGPcIEwajzrJwKu1bFfzI5mAE3QdEuWPx67aZrGPGiRoIgIkNEZKmIrBCRO0O4i4g84t2/\nEZH+AW4FIrJAROaLyJya2GE0bOY+5SaTe15cu+l2HQxtj4U5T9ZuuoYRL6otCCKSAjwODAV6AVeI\nSK8gb0OBHv4YDQT/dc5S1b6qmltdO4yGzZ518O070O/G+E8mByPinlxeNxM2zqvdtA0jHtSkhzAQ\nWKGqK1W1BHgVGBbkZxjwgjpmAq1EpEMN0jSMw5g73s0h5I5JTPp9r4XUpjDbeglGPSC1BmE7AWsD\nvq8DBkXhpxOwEVBgioiUAU+rasj1GiIyGte7IDs7m/z8/BqYXHsUFhbWGVtrk1iWS/lBYeZjp9Bm\n0F6+Xr0AVsck2iqT9aNj+Ppf2TQbNoPUzNJqxWH3S2isXEITr3KpiSDUlNNVdb2ItAM+FpFvVXVa\nsCcvFOMBcnNzNS8vr5bNrB75+fnUFVtrk1iWy8JX4eBOGPI/bemRwLI+tgWMfx8yvzudk2+tXhx2\nv4TGyiU08SqXmgwZrQe6BHzv7K9F5UdVKz63AG/hhqAMI2pmP+FfgvOTxNrRoT90PhnmPGEvzzHq\nNjURhNlADxHpJiLpwOXAxCA/E4Fr/Gqjk4HdqrpRRDJEpDmAiGQA5wILa2CL0cDYvADWfOYmdWP5\nEpzqkvv/3JbYq/6TaEsMo/pU+6+kqqXAzcBkYAkwQVUXicgYEamY4psErARWAM8A/89fzwami8jX\nwCzgfVW191AZUTP7cUhtEvuX4FSX40e4h9W+fCTRlhhG9anRHIKqTsJV+oHXngo4V+CXIcKtBPrU\nJG2j4VK4Gb5+HnpfBU3bJNoaR2oT98rOT+9x+xu1Oz7RFhlG1UmCzrZhVI0vH4bSYjjtN4m25HAG\n3uyWoH7xf4m2xDCqhwmCUaco2u2Gi3pdWnv7FkVLsyzo/zNY8BLsXlu5f8NINkwQjDrFnKegeA+c\n9oONUpKDU24DVZj510RbYhhVxwTBqDOUFrmK9qgfQ8cBibYmNK1yoPcV7gnqAzsSbY1hVA0TBKPO\nMP852LcZTv99oi2JzKm/hYP7YNZjibbEMKqGCYJRJygtgun3Q6eB7r0HyUx2bzj2pzDjQdi3NdHW\nGEb0mCAYdYIZf4Xdq+Hs+90uo8nO2fdDyT63DNUw6gomCEbSs3cjTP8j9LwIup2VaGui44jj3At0\n5jwFW5ck2hrDiA4TBCPp+c/d7rmDH9ex9f154yA9Az5OsuclDCMcJghGUrNxHsx/FgbdCm26J9qa\nqpFxBJxxNyx/H1ZOSbQ1hlE5JghG0lJeBh/8yj3wdebdibamegz6FbTqBh/cAgf3J9oaw4iMCYKR\ntEz/E6z93A0VNWmZaGuqR2pjuHA8bFsKk29LtDWGERkTBCMpWf0Z5I+F3qOgzzWJtqZmHHUOnPob\nmPs0LPl3oq0xjPCYIBhJx/7t8O9R0PooOP+purHMtDJ+9L/QMRcm/sz2OTKSFxMEI6koL4W3r3Vb\nXA9/DRo3T7RFsSElHS59BcoPwhuXuWcUDCPZMEEwkoayg/DmKLcqZ8jf3Ksp6xNtusOw52D9l/DK\nBTbJbCQfJghGUlBWAm9eDotfhx//BU76f5WHqYv0uhQu/hesngavXGiiYCQXJghGwinaDROGuwnX\nn/wNTr090RbFl96j4KIXoCAfXjrPPYltGMlAjV6haRg15buPYOKNsHcDDH0MBv7ghav1kxOvdJPl\nE38GT54A3X51BOQl2iqjoWM9BCMh7F4D746GF38C6Zlw44yGIwYV9B4FN30FrY+GxeOO580rbN8j\nI7HUSBBEZIiILBWRFSLyg3dYieMR7/6NiPSPNqxR/9By2L2gBW9cBg8fBV/9A065HUbPc9taN0Sy\njoUbPoeu1xbw7dvwRC94+QK31UV5aaKtMxoa1R4yEpEU4HHgx8A6YLaITFTVxQHehgI9/DEIeBIY\nFGVYo45TvMc9obv5a1g1FVZOhf1b+9O4JZz8a/dS+lZdE21l4klJg5zrChjxfznMfgJmPwb/+jE0\nbgHdznYPtmX3cTuoNm2TaGuN+kxN5hAGAitUdSWAiLwKDAMCK/VhwAuqqsBMEWklIh2AnCjCxowN\nc2DHinjEHJ4ti9uxcFPtplkdVAO/BF1Tf66udV9eBlrmWq5lB93KoLJit6a+pBBK9sC+Le7YuwEK\nA/Kf2QG6D4GSzku4+L+PIz2zljJYh8g4AvLGwmm/heWT4LvJsOJD+PatAD/tXFlmZkNGthON9ExI\ny4DUJu55h5R0aJQCkgLSyB/iPvEP+Yl8f15BMj4AWFf+R7VB18HQvEN806iJIHQCAp+5XIfrBVTm\np1OUYQEQkdHAaIDs7Gzy8/OrbOjyv/VgwzudqhyuZvSioQwHS1o5KU3KSGlWRnqrEtJaHSSzbwlH\nHLmfZl3206zrfpp2PoAIFBYW8sWczYk2OekoLCw8/N5uC81HQd8roGhTE/YXNGP/mgz2r21KyY50\ntqxJp+TrdMoOpFB2IAU9WF+nAxvO/6gyev/5G9oMdC/q/sH9EiOSfpWRqo4HxgPk5uZqXl5elePI\n7QlF98fYsEqYNWsWAwfWkYHxgJbhoVZiUEtSGn3f6myU6oY5KlqjKemNcNNRaUCTiEnl5+dTnd+w\ngs2bNzNixAi++uorRo8ezYMPPljtuJKJUOUybtw4VqxYwYsvvhgyzPHHH8/jjz9OXl7e9z02f0z7\nLJ+f33IVi2asQ8tdT0/LfUCFX/7uOjq278x//9cfDl1LRurU/yjOtOh84qGedU3/R2FR1WodwCnA\n5IDvvwd+H+TnaeCKgO9LgQ7RhA11DBgwQOsKn3zySaV+XnrpJR0wYIBmZGRo+/btdciQIfrZZ5+p\nqurYsWP1yiuv/EEYQJcvX37o+6JFi/TCCy/UFi1aaGZmpubl5ennn39+WJji4mIdO3asdu/eXZs1\na6Zdu3bV66+/XletWnXIz7vvvqsnnXSSNmvWTNu0aaOjRo3StWvXHnJ/9tln9bTTTouYn2uvvVZT\nUlJ0w4YNh67ddNNNmpGRoRkZGZqWlqYpKSmHvg8ZMkRXrVqlwKFrFcerr74aMo17771XL774Yi0v\nL49oSzRce+21etddd9U4nqqwatUqzcvL06ZNm+qxxx6rH3/8saqGvl/C3QPR8Mknn2inTp3Cuici\n78H3bmUMHjxY77jjjpikPWLECJ08ebIWFRVpdnb2YW6vvfaannLKKdq0aVMdPHhwTNKLN9HUL4EA\nczSKel1Uq9c0EJFUYBlwNrAemA2MUtVFAX7OB24GzsMNCT2iqgOjCRsmza3A6moZXPtkAdsiuGcD\n7XH52YNro7UAmuOG0DoCjYFVQeEGAAuBYu9+HLAV2OTjyMINyS0DKnbM6Y5rvq8G9gMpQBvvfxvQ\nGjevsxrY6d07e1sWA2VAWx/30jD5aQT08XFuBEKNC3X0efw24Fo60BuYGybeYLoCB4ENUfqPRA5Q\nEqO4oqUnUIi771t6GxYCrfjh/RLuHoiG5kA34Jsw7jnUft4D791oONb7LYhB2icAS3Dl2Rn3/6ig\nOW60pAnu/gx3jycTldUvwXRV1SMq9RWNaoQ7cBX9MuA74C5/bQwwxp8LbjXRd8ACIDdS2Pp0EEGR\ncRVBITAigp9xwIshrivQ3Z//C5gUws+TwDR/fg5wAOgSJh3BCcFvg643wv157/XfrwOmR7D3Gty8\n0K3Awgh52h50LcfnKTWKMn0OJwYlvvzO8Xbe6e+j7cAEoE1AmNdxYrkbmAYc76+PDorr3eDyDUjz\nD/48DyfWv/Nx/stfvwCYD+wCvgBODGP/MbgKrnnAtWn+P/OD+8WX1wTgBWAvsCjoP1QAnOPPm3pb\nd+JE/DfAugC//YB5Pp7XgFcr8lVZHnw6d+DEZbcP3yRMHrsDn3p/24DXAvKpuEZKIXAZriHyHq5B\ns9Ofd/b+78M1RMq9/8f89Z7Ax8AOXMU9Mor7pjWw0p//AnggjL+fAfm1UT/U9Ah1v8Qk3kRnrL4e\nkX4wYAhQSoRKkOgEYRNwfQg/Z/k/U1PgfuDTCOn09HF2C+F2DzDDn19HZEGYCjyA6/mUAgPC5Kna\nguD9PxdUkd0KzMS1+hrjhilfCXC/AdcCbAz8DZgfLq7g8g32gxOEUuDPPr6muIp2C64HnAJci6tA\nG4ew/WJgSdC1R/0RThCKcI2nFOBPwMwA9wK+F4T7gc9wPb8uODFf593ScaL/a1xPcThODCvyFTEP\n/nwWrsfSBtfSHhPm93kFuAsn1E2A0yOUbVvgUqCZ/41eB94OcM8HCgK+Z+AaHdfjWvT9cKLTK4wt\nZ+MErhAn/Lv85z5/PjjIf4MXhPq6NCHZaQtsU9XKHj0aKSK7Ao8g9yzc8EwwG3F/yDY+rUi75WQF\nhAkVT1aI64chIkfiROhlVd2ME4eqvtZmW1Bej4sy3BhcD3OdqhbjKtHhflgSVf2nqu4NcOsjIjV5\n/1o5MFZVi1X1AK6n8bSqfqmqZar6PK4XcHKIsJm4lnMge3CVYTimq+okVS3D9Qj7hPE3ErhPVXeo\n6lrgkQC3k3FC8DdVPaiqb+CGaSuIJg+PqOoGVd0BvAv0DWPHQdywXkdVLVLV6eEypqrbVfVNVd2v\nqntxvYLB4fzjejEFqvqsqpaq6lfAm8CIMPFPVdVWwNveTyecuGWpaitV/TRCWg0SE4T4MT6C23Yg\nq6LSisAEf+MeOoLct+Em6YPpgKu4dvq0Iq1erhiHDBdPNOOUV+NavvP995eAUSKSFsLv8jBxZAXl\nNdrVhl2BtwIEcwmud5QtIikicr+IfCcie/h+LLpSkYvAVlUtCkr/9iDR7oJrTQdTiBujDqQlbhgn\n3P0SuAp/P9AkzH3TkcOXcq8OcluvvmkZwj2aPATbEe5Jkt/ihiFnicgiEbkhjD9EpJmIPC0iq/3v\nMw1o5R9crWBakJ2Dguy8EjcXFyr+dd7PFcDzuF5QV2CjiDwUzq46QqT6pdqYIMQJdctlwzED1wK7\nqIbJTCF062gkbqhnv/czUEQ6h4ljKW5c/LB4RKQRrjs/NQo7rgGOEpFNIrIJeAhX6Z4Xwm+sHxFc\nCwwNEpMmqroeGIV74PEcvp/Ahe8X2oZaUbEfN4RRQXBlExxmLa5lHph+M1V9JUTci3DlFNgj6AMs\nquR+iYaNuEq8giOD3DqJHPboWaB7VfIQEVXdpKo/V9WOwE3AEyLSPYz323ETx4NUtQVwpr8e+PsE\nCsJa3PBnoJ2ZqvqLMLZ0xg3PTvGNqfHAL324Ov2G6xjcLyExQUgAqrob+B/gcRG5yLeU0kRkqIg8\nUIWo7gFOFZH7RKSNiDQXkVtwFfTvfFpTcJNwb4nIABFJ9f7GiMgNvtV4B3C3iIwSkSYi0h74O641\n+9eA9MS7Bx6nAEfjnlzv648TgJep+rBRdXgKuE9EunoDjxCRYd6tOU54t+Mq+T8Ghd0MHBV0bT6u\nd5MiIkOIPIQB8AwwRkQG+b27MkTk/KBKHwBVXebjH+vL7hLcCqs3o85teCYAvxeR1l78bwlwm4Gb\n+/iVv88uwf1eVc5DZYjIiIDGx05cpV7xBERweTfHLXjYJSJtgLFB0QX7fw84RkSu9vlIE5GTKhle\nHICbTAfoD8wJYXOKiDTBzUs08r9NqN5tvccEIUGo6oPAbcDduFUWa3FLdN+uQhzLgdNxrcwCXEvw\nUuAnqvp5gNfhwCTc6pDduAnHXFzvAVV9DTfs82tc5bkYN2F6mqpuD4jnVNwfOPC4EXhHVRf41uEm\nVd0EPAxc4P/o0bBLRAoDjmhbcA8DE4GPRGQvboK54qn3F3BDI+t9nmYGhf0H0MsPP1SU+63AhbhJ\nxyup5PdQ1TnAz4HHcBXgCtwEfDgux5X9Ttwk8XBV3VppLivnHlxeVwEf4eYbKmwsAS7xdu3ArfD5\ndw3yEImTgC9FpBD3u9yqfosa3BzO8768R+Im+ZvihiVnAh8GxfUwbj5op4g84ucZzsWV4QbcMFbF\nBH84BgDzfO+oJ66XFszVuHv5SeAMf/5MlXJdT6j2cwiGYRhG/cJ6CIZhGAZggmAYhmF4TBAMwzAM\nwATBMAzD8CT99teBZGVlaU5OTqLNiIp9+/aRkZGRaDOSDiuX0Fi5hMbKJTRVLZe5c+du0yg2t6tT\ngpCTk8OcOT9YRpyUxG2/8jqOlUtorFxCY+USmqqWi4hEtUt0nRIEw6hvlFLGGnby3dHFrGcWTUmj\nGem0oikLT9AxAAAgAElEQVS96URGxCX2hhFbTBAMIwGUUMonLGMeazhIGdIJsilmB/vYTwnFlDKV\npZxIJ06mG0dE3P/OMGKDCYJh1DIr2ML7LGQ3BziRTvSiA2unL+bsM8845GcbhXzJKr5mHV+xljyO\n4Qy6I4HvOzWMGGOCYBi1hKJMZjGzKCCLTK7jFI7E7eyxofzwzV2zyOR8enMWx/Ihi8hnGRvYxUX0\npQkNcpsdoxawZaeGUQuUU847fM0sChhEN0Zz+iExiEQz0rmYvgyhFyvYyjNMZyf7a8FioyFigmAY\ncaaMct7kK75hPWdxDD+hF6mkVB7QIwgD6cY1nMwBDvIiX1JIUeUBDaOKmCAYRhxRlDf5iiVs4lx6\ncQY9qh3XkbRhFCdRSDEvMYsDHIyhpYZhgmAYcSWfZXzLJn7McZxMtxrH15nWXEYu29jHK8zmIGUx\nsNIwHCYIhhEnFrGBz1hBP7rERAwqOIosLqEv69jJ+yxAQ774zTCqjgmCYcSBDezmHb6mC605jxNi\nvlz0ODowmGP4hvV8ddirlA2j+kQlCCIyRESWisgKEbkzhLuIyCPe/RsR6R/gViAiC0RkvojMCbje\nRkQ+FpHl/rN1bLJkGImliIO8zlyakc5IBpASp3bXmXTnKLL4gEVsZHdc0jAaFpXeqSKSAjwODAV6\nAVeISK8gb0OBHv4YjXsVXSBnqWpfVc0NuHYnMFVVe+Be5P4DoTGMusgHLGQPRQynf1y3nhCEi+lL\nBum8wTyKbJLZqCHRNF0GAitUdaV/N+urwLAgP8OAF9QxE2glIh0qiXcY8Lw/fx64qAp2G0ZSspD1\nLGADZ9KdzsS/05tBYy6lP7s4wAchXxdsGNETzZPKneCwQcp1fP8S80h+OuFe+q7AFBEpA55W1fHe\nT7aqbvTnm4DsUImLyGhcr4Ps7Gzy8/OjMDnxFBYW1hlba5P6XC5FjcuZe9IBWuxrRPn89eTrhqjD\n1rRcuuSksiBnPWULt5G1rf5sQFCf75eaEK9yqY0753RVXS8i7YCPReRbVZ0W6EFVVURCLpXwAjIe\nIDc3V+vKVri2bW9o6mu5KMq/+JIUSri25Zm0HtysSuFrWi5llPMPPmf1CUWcx6k0I73acSUT9fV+\nqSnxKpdohozWA10Cvnf216Lyo6oVn1uAt3BDUACbK4aV/OeWqhpvGMnCPNZQwHbOpRetqZoYxIIU\nGjGMPhzgIJNYWOvpG/WDaARhNtBDRLqJSDpwOTAxyM9E4Bq/2uhkYLeqbhSRDBFpDiAiGcC5cOhu\nnQhc68+vBd6pYV4MIyHs4QAf8y3daEu/w9pFtUs2LRjMMSxmI4uIfrjKMCqodMhIVUtF5GZgMpAC\n/FNVF4nIGO/+FDAJOA9YAewHrvfBs4G3RKQirZdV9UPvdj8wQURuBFYDI2OWK8OoJRTlfRZSTjkX\ncGLCt6c+jaNYyiY+ZBFHcQRNbWdUowpENYegqpNwlX7gtacCzhX4ZYhwK4E+YeLcDpxdFWMNI9lY\nxAaWs4VzOS4hQ0XBNKIRF9CbZ/icKSzhQk5MtElGHcKeVDaMarKfEj5kMR1pxcAYbk1RU9rTklPo\nxlespYDtiTbHqEOYIBhGNZnCEoo4yIX0plGSvclsMMfQmma8zwJKbQM8I0pMEAyjGhSwnfms42SO\nIpsWiTbnB6SRwnmcwHb28RkrEm2OUUcwQTCMKlJKGe+zgFY0ZXAN3m8Qb47mCHrTic/5jq3sTbQ5\nRh3ABMEwqsjnrGQ7+xjKCaRV4c1nieBcjiOdVN5noW2TbVSKCYJhVIHtFDKdFfSiAz1ol2hzKiWD\nxpxDT9awg69Zl2hzjCTHBMEwoqTimYNUGjGE4A1/k5d+dKELrfmYJeyjONHmGEmMCYJhRMnXrKOA\n7ZxDTzJpkmhzokYQzqc3xZTyMUsSbY6RxJggGEYU7KOYj1lCF1rTnyMTbU6VaUdzTuVovmE937E1\n0eYYSYoJgmFEwccsoZhSzqd3wrenqC5n0p22ZDCJhRy0ZxOMEJggGEYlfMdWvmE9p3IU7WieaHOq\nTSopnE9vdrKffJYl2hwjCTFBMIwIFFPKeyygLRmcmcTPHERLjt+RdSar7D3Mxg8wQTCMCPyHb9nN\nAX7KiaQm+TMH0XIOx5FBOu/yDWWUJ9ocI4kwQTCMMKxmO7NZzSBy6EKbRJsTM5qSxvmcwCb22LYW\nxmFEJQgiMkRElorIChG5M4S7iMgj3v0bEenvr3cRkU9EZLGILBKRWwPCjBOR9SIy3x/nxS5bhlEz\nDlLGu3xDK5pyFscm2pyYcyzt6U0nprPCho6MQ1QqCCKSAjwODAV6AVeISPBTOUOBHv4YDTzpr5cC\nt6tqL+Bk4JdBYf+qqn39cdj7FgwjkXzMEnawnws5kfRaefV47TOE42lGOu/wte2IagDR9RAGAitU\ndaWqlgCvAsOC/AwDXlDHTKCViHRQ1Y2qOg9AVfcCS4BOMbTfMGLOcrYwh9WcTDe6kZVoc+JGU9K4\nkBPZwl4+ZXmizTGSgGiaPp2AtQHf1wGDovDTCdhYcUFEcoB+wJcB/m4RkWuAObiexM7gxEVkNK7X\nQXZ2Nvn5+VGYnHgKCwvrjK21SbKXS0maMvek/WSUCKnzNpNfvqVW0k1kubQ/JpXPO3zHnq/X03pX\nck2cJ/v9kijiVS610hcWkUzgTeC/VHWPv/wk8L+A+s8HgRuCw6rqeGA8QG5urubl5dWGyTUmPz+f\numJrbZLM5aIorzGHcoq4Kv00ss+svfccJLJcTqWUvzOdVX1LOZdTyKBxQuwIRTLfL4kkXuUSzZDR\neqBLwPfO/lpUfkQkDScGL6nqvys8qOpmVS1T1XLgGdzQlGEkjFkUsIwtnE3PpHzpTbxIJ5VL6c8B\nDvIOX9s22Q2YaARhNtBDRLqJSDpwOTAxyM9E4Bq/2uhkYLeqbhQRAf4BLFHVhwIDiEiHgK8XAwur\nnQvDqCGr2cHHLOFYshlETqLNqXWyacG5HMcKtjKDlYk2x0gQlQ4ZqWqpiNwMTAZSgH+q6iIRGePd\nnwImAecBK4D9wPU++GnA1cACEZnvr/23X1H0gIj0xQ0ZFQA3xSxXhlEF9lLEm8yjFc0YRp86u1dR\nTcmlK6vYzlSW0p6WHFWPJ9SN0EQ1h+Ar8ElB154KOFfglyHCTYfQ/y5VvbpKlhpGHCijnDeYRzGl\nXMUgmpCWaJMShiAMow/P8gVvMJcbOZ22ZCTaLKMWsSeVjQaLorzHAtaykwvpXac3rosVjUnlMnIR\nhNeYTREHE22SUYuYIBgNlk9Yxtes40x6cII9HnOI1jRjBAPYwX7e5Cvb76gBYYJgNEjmsJrprKAf\nXRhcD3YxjTU5tOV8TuA7tvIW8yk3UWgQ1M9n8g0jAovYwAcspAftOJ8TGuwkcmX040iK/Gs300jh\np5xoZVXPMUEwGhTzWcu7fEMX2nAp/WhkneSInMJRlFDGpywjjRSGcryJQj3GBMFoMMymgA9YxFFk\ncRm5pNWT9xvEmzPpzkFK+YKVHOAgw+rRuyGMwzFBMOo9ipLPMj5jBceSzaX0swqtCgjC2fSkKelM\n5VsKKeIychv0Et36ivWXjXpNEQd5hdl8xgr60pnh9DcxqAaCcBpHczF9WctOnuULtrI30WYZMcYE\nwai3bGIPf2c6K9nGeZzAhZxIit3yNaI3nbiSgeyjhGeYzjzW2N5H9Qj7dxj1jlLK+A/f8nemU0IZ\n13AyuXS1ydAY0Y0sbuIMutCa91jAm3xFIUWJNsuIATaHYNQbFGUl2/iQRWxnH33ozLkcR1PSE21a\nvaM5TbiKQXzOd+SzjBVsZTA9GEiO9cLqMCYIRr1gNdv5hGWsYQetaMqVDORojki0WfUaQTid7hxH\nez5iCR+zhHms4XS6cwIdTRjqICYIRp3lIGUsZiPzWMNadpJJY4ZyPP3oYhPHtUhbMrmCk1jOFqby\nLe/wNf9hKYPIoQ+dk+qFO0ZkTBCMOkUJpaxiO8vYzGI2UkwpbcngJ/SiP0faswUJpAft6M4RfMdW\nvmAlU/iWqSylG205gY70oJ2JQ5ITlSCIyBDgYdz7EP6uqvcHuYt3Pw/3PoTrVHVepLAi0gZ4DcjB\nvQ9hZKh3KhsNm30Us4HdrGMn69jFGnZQRjmNSeUYsulHF7rSxiaMkwRB6E47utOOLexlIetZyAYm\n8g0A7WhON7LoTCva04I2ZNhvl0RUKggikgI8DvwYWAfMFpGJqro4wNtQoIc/BuHelzyokrB3AlNV\n9X4RudN//13ssmYkM4pykDL2U8J+DrKfYvZSxB5/bKOQbRRywG+/LLi3euXSlWNox5G0sTHqJKcd\nzfkRPTmLY9nAblaxjVVsYw6r+ZJVAKSTQlsyaE0GrWlGS5qSSWMyaUwz0jmYqpSjNDLRqBWi6SEM\nBFao6koAEXkVGAYECsIw4AX/opyZItLKvyIzJ0LYYUCeD/88kE+cBGEB61nN9nhEHZYNxxRT6FtF\nyUykFeTB68s14LoG+FB/Vu5dyv1RRrk/lFLKKKWMwlMO8CWTKaE0bNqZNKYNGRxHB7LIoD0t6UhL\n0m2Es04iCJ1oRSdacTrdKaOcLexlE3vYzB62s49N7OZbNlEefFecDjOYRBoppJNKOimk0ohUUkih\nEak0ohHiD7czlbvi+h1yyAIO9UQkyLbK7U8OBtIt7u/siOYf1glYG/B9Ha4XUJmfTpWEzVbVjf58\nE5AdKnERGQ2MBsjOziY/Pz8Kkw9nVbcSNrcvrXK4mqBtytlRvK5W04wLAf/Pw/5IGnSuzl1UEHXX\npBwaqbvWqAxSyqF5cTlNGqWQUppGShmkHRRSDwppB4X0EqFxsdBIBSgGiiliOwW4McX6TGFhYbXu\n7bpOEyoqCkGlKSVpSkm6O0rTlH3lxTTKSKcsRSlLKaM8pZSyRlDcCMobKeUCKqCN/Ke/Ed05h27a\nH5wHfK8rlC7ZSqtdbo4sbveLqkY8gOG4sf+K71cDjwX5eQ84PeD7VCA3UlhgV1AcOyuzZcCAAVpX\n+OSTTxJtQlJS03LZtGmTnnHGGZqZmam33XZbbIxKAkKVy9ixY/XKK68MG6ZXr15hy/OTTz7RTp06\nhQ177bXX6l133VVVM2sd+x+FpqrlAszRSupXVY1qEHY90CXge2d/LRo/kcJu9sNK+M8tUdhSr3j5\n5ZfJzc0lMzOTDh06MHToUKZPnw7AuHHjuOqqq34QRkRYsWLFoe+LFy/mpz/9KS1btqR58+acddZZ\nfPHFF4eFKSkpYdy4cfTo0YOMjAxycnK44YYbKCgoOOTnvffeY+DAgWRkZNC2bVuuvPJK1q37vofz\n3HPPcfrpp0fMz3XXXUdqaiobN248dG3MmDFkZmaSmZlJeno655xzzqHvQ4cOpaCgABE5dK3ieO21\n10KmMX78eLKystizZw8PPvhgRHsq47rrruPuu++uURxVpaCggLPOOotmzZrRs2dPpkyZUu24Fi1a\nRF5eXuyMizPB925l5OXl8f7778ck7ZEjR/LRRx9RXFxM+/btD3O744476NGjB82bN6dnz5688MIL\nMUmzLiJOPCJ4EEkFlgFn4yrz2cAoVV0U4Od84GbcKqNBwCOqOjBSWBH5P2C7fj+p3EZVf1uJLVuB\n1dXLaq2TBWyL4J4NtMflZw+uJ9sCaI4bWusINAY/+/Y9A4CFuDGVxsBxwFbcsJv6dDvhyn2fD9Md\nSPNp7cet+Grj/W8DWuPme1YDO717Z2/LYqAMaOvjXhomP42APj7OjcDmEH46+jx+G3AtHegNzA0T\nbzBdgYPAhij9RyIHKIlRXNHSEyjE/R9aehsWAq344f0S7h6IhuZANwg7kZVD7ec98N6NhmO934IY\npH0CsARXnp1x/48KOgI7gCIgA7c4Zjnf/3+Skcrql2C6qmrlT2pG043AVfTLgO+Au/y1McAYfy64\n1UTfAQuA3Ehh/fW2uKGl5cAUnCBEZU9dOIjQRcNVBIXAiAh+xgEvhriuQHd//i9gUgg/TwLT/Pk5\nwAGgS5h0BCcEvw263gj3573Xf78OmB7B3mtw80W3Agsj5Gl70LUcn6fUKMr0OZwYlPjyO8fbeae/\nv7YDEwLvJeB1nFjuBqYBx/vro4Pieje4fAPS/IM/z8OJ9e98nP/y1y8A5gO7gC+AE8PYfwyugmse\ncG2a/y/94H7x5TUBeAHYCywK+m8VAOf486be1p04Ef8NsC7Abz9gno/nNeDVinxVlgefzh04cdnt\nwzcJk8fuwKfe3zbgtYB8Kq6SLQQuwzVE3sM1aHb6887e/324hki5918x1NwT+BhXgS/FLVev7L5p\nDaz0578AHqjE/0Tg9njWDzU9Qt0vMYk30Rmrr0ekHwwYApQSoRIkOkHYBFwfws9Z/s/UFLgf+DRC\nOj19nN1CuN0DzPDn1xFZEKYCD+B6PqXAgDB5qrYgeP/PBVVktwIzca2+xsDTwCsB7jfgWsuNgb8B\n88PFFVy+wX5wglAK/NnH1xRX0W7B9YxTgGtxFWjjELZfDCwJuvaoP8IJQhGuUZUC/AmYGeBewPeC\ncD/wGa7n1wUn5uu8WzpO9H+N6ykOx4lhRb4i5sGfz8K1pNvgWtpjwvw+rwB34YS6CYfPLQaXbVvg\nUqCZ/41eB94OcM8HCgK+Z+AaHdfjFsT0w4lOrzC2nI0TuEKc8O/yn/v8+eAQYZrierhD4lU3xOII\ndb/E4rCF3ImhLbBNVStb+jRSRHYFHkHuWbibN5iNuD9kG59WKD+BcRDGz8YA97CIyJE4EXpZVTfj\nxOGaysIFsS0or8dFGW4Mrue5TlWLcZXocD9ciar+U1X3Brj1EZGWVbQtkHJgrKoWq+oBXE/jaVX9\nUlXLVPV5XC/g5BBhM3Et50D2QMS1hNNVdZKqluF6hH3C+BsJ3KeqO1R1LfBIgNvJOCH4m6oeVNU3\ncMO3FUSTh0dUdYOq7gDeBfqGseMgblivo6oWqer0cBlT1e2q+qaq7lfVvbheweBw/nG9mAJVfVZV\nS1X1K+BNYESY+Keqaivgbe+nE07cslS1lap+GiLYU8DXwOQIdtRbTBDix/gIbtuBrIpKKwIT/I17\n6Ahy3wZ0CBGuA67i2unTCuUnMA7C+OlAdOOUV+NavvP995eAUSIS6pVay8PEkRWU1yVRpAuu8nkr\nQDCX4HpH2SKSIiL3i8h3IrKH78eiKxW5CGxV1cC9nrsCtweJdhdcazqYQtwcSiAtccM44e6XTQHn\n+4EmYe6bjhy+xHt1kNt69U3LEO7R5CHYjsww9v4WNww5S0QWicgNYfwhIs1E5GkRWe1/n2lAK/9A\nawXTguwcFGTnlbi5uFDxr/N+rsA967TFx7FRRB4K4f//cHMNI4PKKhmJVL9UGxOEOKGqkX6wGbgW\n2EU1TGYKoVtHI3FDPfu9n4Ei0jlMHEtx4+KHxSMijXDd+alR2HENcJSIbBKRTcBDuEr3vBB+o19m\nEh1rgaFBYtJEVdcDo3APQJ7D9xO48P0K9FB/+v24IYwKgiub4DBrcS3zwPSbqeorIeJehCunwB5B\nH2BRJfdLNGzk8BV9Rwa5dfJbzIRyr0oeIqKqm1T156raEbgJeEJEuofxfjtu4niQqrYAzvTXA3+f\nQEFYixv+DLQzU1V/EcaWzrjh2Sm+MTUe+KUPd1ugXxG5B7fjwrmquqeq+a5tYnC/hMQEIQGo6m7g\nf4DHReQi31JKE5GhIvJAFaK6BzhVRO4TkTYi0lxEbsFV0L/zaU3BTcK9JSIDRCTV+xsjIjf4ltAd\nwN0iMkpEmohIe+DvuNbsXwPSE+8eeJwCHI17or2vP04AXqbqw0bV4SngPhHp6g08QkSGebfmOOHd\njqvk/xgUdjNwVNC1+bjeTYrfhyvSEAbAM8AYERkkjgwROT+o0gdAVZf5+Mf6srsEt8LqzahzG54J\nwO9FpLUX/1sC3Gbg5j5+5e+zS3C/V5XzUBkiMiKg8bETV6mX++/B5d0ct+Bhl9/bbGxQdMH+3wOO\nEZGrfT7SROSkSoYXB+Am0wH6A3NC2Px7XOPhHFWt3S0NkgwThAShqg8CtwF341ZZrMUt3X27CnEs\nB07HtTILcC3BS4GfqOrnAV6HA5Nwq0N24yYcc3G9B1T1Ndywz69xledi3OTaaUF/kFNxf+DA40bg\nHVVd4FuHm1R1E25Dwwv8Hz0adolIYcBxW+VBwKczEfhIRPbiJpgrnoZ/ATc0st7naWZQ2H8Avfzw\nQ0W53wpciJt0vJJKfg9VnQP8HHgMVwGuwE3Ah+NyXNnvxE0SD1fVrZXmsnLuweV1FfARbr6hwsYS\n4BJv1w7cCp9/1yAPkTgJ+FJECnG/y63qt67BzeE878t7JG6SvyluWHIm8GFQXA/j5oN2isgjfp7h\nXFwZbsANY1VM8IdjADDP94564nppwfwR12NaEXD//XdVM14fqPQ5BMMwDKNhYD0EwzAMAzBBMAzD\nMDwmCIZhGAZggmAYhmF46tQbR7KysjQnJyfRZkTFvn37yMjISLQZSYeVS2isXEJj5RKaqpbL3Llz\nt2kUm9vVKUHIyclhzpwfLCNOSvLz8+vU1sS1hZVLaKxcQmPlEpqqlouIRLVLtA0ZGUZV2LkTunWD\njAzo0AF69YI33ki0VYYRE0wQDKMqjB0La9bAjTfChReCKtx0kxMKw6jjmCAYRrQsWABPPAG/+AU8\n8giMHw+vvurE4L77Em2dYdQYEwTDiAZVuOUWaNUK7r33++t9+sB118Gjj8LKlWGDG0ZdwATBMKLh\n9dfh009dT6BN0PZM//u/kJoKv/99YmwzjBhhgmAYlVFcDLffDv36wc9+9kP3Tp3gjjtgwgSYMaP2\n7TOMGGGCYBiVMXkyrFvnegIpKaH9/OY30K4d3H9/7dpmGDHEBMEwKuP116F1azj33PB+MjPhiiuc\neOxJ+verGEZITBAMIxJFRfDOO3DxxZAW6o2gAQwf7oaX3n+/dmwzjBhjgmAYkfjoI9i7F0aEfI/7\n4Zx6qntYzR5UM+ooJgiGEYkJE9xw0dlnV+63USPXk/jgA9i3L/62GUaMMUEwjHAUFcHEidENF1Uw\nfDgcOOBEwTDqGCYIhhGOyZPdcNHIkdGHOeMMOOIIGzYy6iQmCIYRjtdfdw+h/ehH0YdJTXU9ivfe\ncz0Fw6hDxE0QRGSIiCwVkRUicmcI954iMkNEikXkjnjZYRjVomK46KKLoh8uqmD4cDeHMHlyfGwz\njDgRF0EQkRTgcWAo0Au4QkR6BXnbAfwK+Es8bDCMGpGf74aLLr206mHz8lzP4s03Y22VYcSVePUQ\nBgIrVHWlqpYArwLDAj2o6hZVnQ0cjJMNhlF9PvwQmjSBs86qeti0NDjvPBdHeXnsbTOMOBGvN6Z1\nAtYGfF8HDKpORCIyGhgNkJ2dTX5+fo2Nqw0KCwvrjK21SV0pl5Peeovi3r355ssvqxW+3ZFH0mvb\nNuY+8wx7jz22Uv91pVxqGyuX0MSrXJL+FZqqOh4YD5Cbm6t15XV69uq/0NSJcikogDVryPj1r6tv\n6/HHw5/+xICtW90LdCqhTpRLArByCU28yiVeQ0brgS4B3zv7a4aR/FRMBg8ZUv04jjgC+vd3w0aG\nUUeIlyDMBnqISDcRSQcuBybGKS3DiC0ffghdu0IUQz0RGTIEZs6EXbtiY5dhxJm4CIKqlgI3A5OB\nJcAEVV0kImNEZAyAiLQXkXXAbcDdIrJORFrEwx7DiJqSEpg61VXmIjWLa8gQKCtz8RlGHSBucwiq\nOgmYFHTtqYDzTbihJMNIHmbMcMtNazJcVMHJJ0PLlq7HUZ3lq4ZRy9iTyoYRyIcfuqeNq/J0cjhS\nU92meB9+6N7JbBhJjgmCYQTy4Ydw2mnQIkajl0OGuLetLVkSm/gMI46YIBhGBZs2wfz58JOfxC7O\nirhstZFRBzBBMIwKKirtWMwfVHDkkdCrl22HbdQJTBAMo4L33oNOnaBv39jGO3QoTJvmJqsNI4kx\nQTAMcMtNP/rI7UFU0+WmwVxwgYt/ypTYxmsYMcYEwTAApk93Lfjzz4993Ked5pafvvde7OM2jBhi\ngmAYAO+/D+np0b07uaqkpbnJ5UmTbPdTI6kxQTAMcIKQlweZmfGJ/4IL3CqmefPiE79hxAATBMP4\n7jtYujQ+w0UVDB3q5iZs2MhIYkwQDOP9991nPAUhKwtOOcUEwUhqTBAM4/333c6mRx8d33TOPx/m\nzoWNG+ObjmFUExMEo2FTWOjen3zBBfFPqyKNSZMi+zOMBGGCYDRsJk92zwjEc7iogt69oUsXePfd\n+KdlGNXABMFo2Lz0EmRnwxlnxD8tEfjpT50I7dkT//QMo4qYIBgNlx073PzBqFFuq+ra4KqroKgI\n3nijdtIzjCpggmA0XN54ww0XXXVV7aU5aBD06AEvvFB7aRpGlJggGA2XF1+E446Dfv1qL00RuOYa\n+PRTWL269tI1jCgwQTAaJgUF8NlnrncQ683sKqOiR/Lii7WbrmFUggmC0TB56SX3eeWVtZ92Tg4M\nHuyGjezVmkYSYYJgNDxUXev8zDOha9fE2HDNNbBsGcyalZj0DSMEJghGw2PuXPj229qdTA5m+HBo\n0sQml42kwgTBaHj85S9uV9MRIxJnQ4sWcPHF8PLLsHt34uwwjABMEIyGxYIFMGEC3HortGqVWFvu\nuAN27YK//jWxdhiGxwTBaFjccw80bw633ZZoS6B/fzd09OCDpFkvwUgCTBCMhsP8+fDmm/DrX0Ob\nNom2xnHvvbB/P0e+/HKiLTEMEwSjATFunBsm+q//SrQl33PccXD11XR8+21Yvz7R1hgNHBMEo2Ew\neza88w7cfnvi5w6CGTsWKS+HP/wh0ZYYDRwTBKP+s2MHXHEFtG8Pv/pVoq35Id26sfH88+GZZ2DK\nlERbYzRgTBCM+k1pKVx+OaxZ4+YPWrRItEUhWfnzn7vhoxEj3PudDSMBmCAY9Zvf/Q4+/hiefBJO\nPf3y2aEAAAZ6SURBVDXR1oSlLCPDvTgnLc29WW379kSbZDRATBCM+kl5Ofz5z/DQQ3DzzXDjjYm2\nqHJycuDtt11v5pJLTBSMWscEwah/FBTAj34Ed94Jl17qRKGucOqp8Nxz8MUXcPzx8O9/J9oiowER\nN0EQkSEislREVojInSHcRUQe8e7fiEj/eNliNBCWL3fr+k88EebNg2efhddfd8MwdYkrroA5c6Bj\nRydow4fDf/7j5kMMI47E5b2BIpICPA78GFgHzBaRiaq6OMDbUKCHPwYBT/pPwwhPeTns3ev2/1mz\nxk3ALl3qVud89ZV7t8GQIW7OIFE7mcaCPn3gyy/hgQfgj390E+JZWXDhhU7wevz/9u4uxIo6jOP4\n97cv5qKktdUa7pJC3tgLFGKBN0JRpqJdhUXRG4qgUBBE6o2XQWARESEZGAUihChhlll7ae9J2ItJ\nkRmadRHprrUc99fFf7Yd9bjb2s6Zs87zgWFn/jN79n+ePTvPvOw8/zlw/fXQ2QnTpk28pBeaUlED\nyc4HDtv+AUDSNmA5kE8Iy4HXbRvYL2m6pGttHxv33mzcOFz/vkHmnz4NHR0N/ZkTwW39/fXjMjQu\ngD08DQ6mr7VaGupyYAD6+s4fQ2DSpFQGYtOm9F863d3Fv5FGaG+HDRvSg3R79qSksGNHOvM51+TJ\naWpvT1NrK7S0pCk/ANBIgwE1eqCg/yD+jnK2bEkl2wtUVEKYCfycWz7K+Uf/9baZCZyVECStAlYB\ndHV10dvbO+bOzDh9misafLRYq9U42aiB2yeQWq1G2wWOZn3Ojsst6Yqm29oYbG/Hra2c6eigNnUq\ntSlTGOjspL+nh79mzEg7QIDDh9M0wZw6dWrkz3ZnJ6xaBStX0v7HH3QcPUrHsWO0nTxJW18frf39\ntAwMoDNnaKnVYHAQDSXVjEYajKdJB+qJv6NhRw4doi/7fY76eblITR9p25uBzQDz5s3zwoULx/4i\nF/M9/1Nvby8X1ddLXMSlvohLfRGXYV25+aLiUtRN5V+Antxyd9Y21m1CCCE0iFzAqaKkNuAQcAdp\nJ/8J8IDtg7ltlgBrgcWky0kv2p4/yuv+Bvw07h0uxlXA72V3oglFXOqLuNQXcalvrHG5zvbVo21U\nyCUj2zVJa4F3gVbgNdsHJa3O1r8C7CYlg8NAP/Dof3jdUd9Qs5D0qe15Zfej2URc6ou41Bdxqa+o\nuBR2D8H2btJOP9/2Sm7ewJqifn4IIYSxiSeVQwghAJEQirS57A40qYhLfRGX+iIu9RUSl0JuKocQ\nQph44gwhhBACEAkhhBBCJhJCQSQ9JcmSrsq1rcuqu34n6e4y+9dokp6T9G1W2XaHpOm5dZWNC4xe\nGbgKJPVI+lDS15IOSnoia79S0l5J32dfryi7r2WQ1CrpC0lvZ8uFxCUSQgEk9QB3AUdybXOBFcAN\nwCLg5awqbFXsBW60fTPpocV1EHHJVQa+B5gL3J/FpGpqwFO25wK3A2uyODwD7LM9B9iXLVfRE8A3\nueVC4hIJoRjPA08D+Tv2y4Fttv+2/SPpgbwRn8y+lNh+z/ZQQf/9pFIlUPG4kKsMbHsAGKoMXCm2\nj9n+PJs/Sdr5zSTFYmu22Vbg3nJ6WB5J3cAS4NVccyFxiYQwziQtB36xfeCcVReq7lpFjwHvZPNV\nj0vV3/95JM0CbgE+ArpyJfGPc3aNt6p4gXSAOZhrKyQuTV/ttBlJeh+YUWfVBmA96XJR5YwUF9s7\ns202kC4PNHaAijAhSJoKvAU8aftP5Uqi27akSv2fvKSlwAnbn0laWG+b8YxLJISLYPvOeu2SbgJm\nAweyD3I38Lmk+VSguuuF4jJE0iPAUuAODz8Ac8nHZRRVf///ktROSgZv2h4aTPrXoYGzJF0LnCiv\nh6VYACyTtBiYDFwu6Q0KiktcMhpHtr+yfY3tWbZnkU7/b7V9HNgFrJB0maTZpKFDPy6xuw0laRHp\ntHeZ7f7cqkrHhVQJeI6k2ZImkW6w7yq5Tw2ndAS1BfjG9qbcql3Aw9n8w8DORvetTLbX2e7O9icr\ngA9sP0hBcYkzhAbJqr1uJw0jWgPW2D5Tcrca6SXgMmBvdva03/bqqsflQpWBS+5WGRYADwFfSfoy\na1sPPAtsl/Q4qfT9fSX1r9kUEpcoXRFCCAGIS0YhhBAykRBCCCEAkRBCCCFkIiGEEEIAIiGEEELI\nREIIIYQAREIIIYSQ+QeHJR3Sfj5rEQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEICAYAAABBBrPDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXecVNX5/9/PFmBZ6rK6VAEFJSiCgIglEaJRsISYKLGL\nRhGjibHEEkzERI0lRiX2EhWNYosGDd9o9MeKqKggBAVElia9LHVp257fH+csDOPM7NSd2d3n/Xrd\n19x7T/uce++c57R7j6gqhmEYhhGOrHQLMAzDMDIbMxSGYRhGRMxQGIZhGBExQ2EYhmFExAyFYRiG\nEREzFIZhGEZEzFAY9QoRKRKRqSKyTUTuS7eeVCIi40TkhQjuc0VkSBi3ISKyIkLYZ0Xk9iTINBoB\nZijSiIicKyIzRKRMRFaLyP+JyHHeLWQhISIqIj0CjnuLyCQR2eILzykickxQmCY+voUisl1ElorI\n30WkW4Cf00TkM+9eKiL/EJHOAe6jRGRaLfl5VkQqRaRDwLnHfP7KRKRcRCoCjv9PRLr5PJUFbT8P\nk8xoYAPQSlWvi3iBayEdhaWI/ElEvvTXaVwicanqoapanBxlqSf42Y3Cf7GIXJqktF8RkZNEpKmI\nrAlya+r/D1tFZI2IXJuMNBsSZijShH8YHwDuBIqAA4CHgR/HEMdBwEfAl0B3oCPwBvCuiBwd4PU1\nH++5QGugLzADOMHHcybwotdTCBwK7AamiUjbKLXkAz8DtgDn15xX1TGq2kJVW/i8vlxzrKrDA6Jo\nE3C+haq+HCaprsA8zYA3RUUkJ45gJcANwL+TLMeIzADcM3848FWQ2zigJ+7ZGgrcICLD6lRdpqOq\nttXxhiusy4CzIvgZB7wQ4rwCPfz+88DkEH4eBab6/ROBnUCXMOkIsAy4Ieh8Fu4P9Ud/PAqYFkHv\nhcBy4Grgq2jzBHTzecqJ4ro9C1QA5f76neh13gQsAkqBV4CCgDCvAmtwBmwqcKg/PzoorreCr29A\nmrf7/SHACuBGH+fz/vxpwGxgM/AxcHgUeXkBGFeLn3E+PxOAbcBcYGCA+1LgRL+f57VuAuYBvwVW\nBPg9AvjCx/MyMLEmX7XlwadzPTDHX8eXgWZhNPcAPvD+NuAqBvhrr8B2f71/DrQF3gbWe91vA529\n/zuAKmCX9/+QP98L+C+wEVgAjIziWrcFFvv9K4B7gtxXAScFHP8RmFiXZUKmb2kX0Bg3YBhQSYTC\nkegMxRrg4hB+hvo/WR5wF/BBhHR6+Ti7h3C7DfjE748isqF4H7gH1zqqBAZEkydiMBTe/7NBBdzV\nwHSgM9AUeBx4KcD9EqCld3sAmB0uruDrG+wHZygqgbt9fHm4AngdcBSQDVyEK1ib1pKPaA3FLuAU\nH/efgekB7kvZayjuAj4ECoAuOCO/wrs1wVUGrgFygTNxRrImXxHz4Pc/w7VYC4D5wJgwml8CxuIM\neDPguAjXth2uFdrc36NXgTcD3IuBSwOO83GVkYuBHK97A9A7jJYTcIavDFch2Ox/t/v943FGRIGi\ngHA/A76s63IhkzfrekoP7YANqlpZi7+RIrI5cAtyLwRWhwi3GvdHLfBphfITGAdh/KwOcA+LiByA\nM04vqupanNG4sLZwQWwIyuv3ogw3BhirqitUdTeucD2zpltIVf+uqtsC3PqKSOsYtQVSDdyqqrtV\ndSeuZfK4qn6qqlWq+hyu225wAmkEMk1VJ6tqFa4F2TeMv5HAHaq6UVWXA+MD3AbjDMQDqlqhqq8B\nnwe4R5OH8aq6SlU3Am8B/cLoqMB14XRU1V2qGnZcS1VLVfV1Vd2hqttwrYjjw/nHtXqWquozqlqp\nqrOA14GzwsT/vqq2Ad70fjrhjF6hqrZR1Q+AFt77loCgW3GGy/CYoUgPpUBhFH3cr/gHes8W5L4B\n6BAiXAdcgbbJpxXKT2AchPHTIcA9EhcA81V1tj/+B3CuiORGEbaGwqC8zo8yXFfgjQBDOh/XmioS\nkWwRuUtEFonIVlwhAVEYvwisV9VdQelfF2TMu+Bq38kgcOB1B9AszHPTEVfbrmFZkNtK9dXlEO7R\n5CFYRwtCcwOuO/MzPyvrkjD+EJHmIvK4iCzz92cq0EZEssME6QocFaTzPKB9mPhXeD/nAM/hWk1d\ngdUi8lfvrcz/tgoI2hrXRWd4zFCkh09wNbafJBjPe4SuTY3EdRnt8H4GBc5gCmIBrt99n3hEJAvX\nBH8/Ch0XAgf6GSNrgL/iCuNTospFYiwHhgcZmWaquhI3eD8CN5bRGtfNBa4gA9flEMwOXFdIDcGF\nUHCY5biafGD6zVX1pQTyFA+rcYV7DQcEuXUSEQnjnrQ8qOoaVb1MVTsClwOPRJjpdB1wCHCUqrYC\nfuDPh7s/y3HdqIE6W6jqFWG0dMZ1877nK1lPAFf6cNd6P5tw1yewpdYXNx5keMxQpAFV3QL8AXhY\nRH7ia1a5IjJcRO6JIarbgGNE5A4RKRCRliLyK1zBfaNP6z3c4N8bIjJARHK8vzEicomvZV4P3CJu\num4zEWkPPIWrZd0fkJ5498DtaOAgYBCuO6IfcBhuFlWs3U/x8Bhwh4h09QL3E5ER3q0lziCX4gr/\nO4PCrgUODDo3G9cayvYzXyJ1hQA8CYwRkaPEkS8ip4pIyK4Lf5+b4f57Of4ahqtBx8IrwM0i0tZX\nCn4V4PYJbmzl1z79n+LuV1x5iISInBVQKdmEK+yr/XHw9W6Jm2ixWUQKgFuDogv2/zZwsIhc4POR\nKyJH1tJNOQA3iA/QHzfzKZgJuOe/rY/rMtzYlOExQ5EmVPU+4FrgFtysj+XAVbj+1GjjWAgch6sB\nLcXVjH4GnKyqHwV4PROYjJutsgU30DkQ19pA3VTUC3CDnaW4WTN5wLGqWhoQzzG4P3bg9gvgX6r6\npa9NrlHVNcCDwGm+AIiGzbLvexTRzmV/EJiEmxK8DTewfZR3m4DrYlnp8zQ9KOzTQG/fjVFz3a8G\nTscNdp5HLfdDVWfgCpaHcAVjCW7gPxxP4q7bObhB3524a58ot+HyugR4FzeeUaOxHPip17URN+Po\nnwnkIRJHAp+KSBnuvlytqou92zjgOX+9R+ImF+ThujenA/8JiutB3HjTJhEZ78cxTgLOxs1UWsPe\niQXhGAB84VtTvQjdUrgVN2tuGW4A/R5VDdbSqJF9uy0NwzAMY1+sRWEYhmFExAyFYRiGEREzFIZh\nGEZEzFAYhmEYEYnno2Zpo7CwULt161bn6W7fvp38/Pw6T7c2TFfsZKo20xUbpis2Zs6cuUFV94s7\ngrr8Xkii24ABAzQdTJkyJS3p1obpip1M1Wa6YsN0xQYwQ+1bT0ZjZvknMOcF2Lkx3UoMo2FSr7qe\nDCOQyt0w5ffw8V8AhawcOPBHcOQv4eDT0q3OMBoO1qIw6iUbvoanB8PH98LAMXDJxzD4Wlg/DyaO\ngKXF6VZoGA0HMxRGvaN8O7wwDLaugLMnwamPQJej4Ud3wxVfQkEPeP0cKFubbqWG0TAwQ2HUOz68\nA7Ysg5+/AYecvq9b05Zw1muwazP88zyorkqPRsNoSJihMOoV6+e7MYl+o+CA40L7KeoDpzwMS96H\nqbfXqTzDaJCYoTDqDaow+ZfQpAWcWMvH2PtdDIefD1P/BJuX1ok8w2iwRGUoRGSYiCwQkRIRuSmE\nu4jIeO8+R0T6B7gtFZEvRWS2iMwIOF8gIv8VkYX+t21ysmQ0VL580Q1Sn/BnyK/l1SER50+y/Kwo\nwzDiplZD4RdVeRgYDvQGzhGR3kHehgM9/TYaeDTIfaiq9lPVgQHnbgLeV9WeuFXUvmOADKOGqgp4\n/yboeCT0vzS6MK06Q9+L4IunoGxN7f4NwwhNNC2KQUCJqi5WtwDKRNzykoGMACb4lwCn49a9jbRO\nc02Y5/z+cyS+LKjRgPn6DTfL6Qe/h6wY1oM79gaoroDpD6ROm2E0dGpduEhEzgSGqeql/vgC3Bq3\nVwX4eRu4S1Wn+eP3gRtVdYaILMGtqlYFPK6qT3g/m9WtY4tffWpTzXFQ+qNxrRSKiooGTJw4MdE8\nx0xZWRktWoRbSz59NCZds646gvJNTRg04VNiXTh03h97s/HTAga/PJ1dbG401ywZmK7YyFRdQ4cO\nnRnUoxMTdfFm9nGqulJE9gf+KyJfq+rUQA+qqiIS0mJ5w/IEwMCBA3XIkCEpFxxMcXEx6Ui3NhqL\nrlUz4IO5cPL9MPiE2OPt1QYePwKa/O84co5tHNcsWZiu2MhUXYkSTdfTSqBLwHFnfy4qP6pa87sO\neIO9i7qvreme8r/rYhVvNA4+He9mOvW7OL7w7ftBj+Hw6QNQXW4T/QwjVqL513wO9BSR7iLSBLew\n+aQgP5OAC/3sp8HAFlVdLSL5ItISQETycQujfxUQ5iK/fxHwrwTzYjRAytbAVxOh7yho1jr+eI6+\nFnZsgA0fFiZNm2E0FmrtelLVShG5CngHyAb+rqpzRWSMd38MmAycApQAO4Caul8R8IYbgiAHeFFV\n/+Pd7gJeEZFfAMuAkUnLldFgmPG4G4wedFXtfiPR/YfQphus/ncH+FNSpBlGoyGqMQpVnYwzBoHn\nHgvYV+DKEOEWA33DxFkKnBCLWKNxUV0JMx+DHsOg8JDE4pIs6HcJFP+hLZsWQ9sDk6PRMBoD1mFr\nZCwl77iupwGXJye+fqOALGXWM8mJzzAaC2YojIzlf89C80LoeUpy4mvdBQqO3MjsZ+xjgYYRC2Yo\njIxkRyksmAR9zofsJsmLt/3w1WxbCYveSV6chtHQMUNhZCRfTYSqct9dlETaHVNK8/1g1tPJjdcw\nGjJmKIyMZPYz7v2H9iGnQsRPVq7S90LXWtlub+4YRlSYoTAyjrVfwuqZ8b9gVxv9LnYzqr58KTXx\nG0ZDwwyFkXH87znIyoU+56Ym/v0PhQ4DXDqGYdSOGQojo6iqgDkvwMGnuRlPqaLvRbBmlmu9GIYR\nGTMURkax6B3Yvjb5g9jBHHY2ZOXA/yakNh3DaAiYoTAyiv89B833cx/xSyX5+0HPU+HLF9x4hWEY\n4TFDYWQMOzf6dyfOg+zc1KfX90L35vfi91KflmHUZ8xQGBlDqt6dCEfPUyGvwAa1DaM2zFAYGcPs\nZ6Gob/LfnQhHTlM49Gz4+k3YtaVu0jSM+ogZCiMjWD8PVn1ed62JGvqNgspd8JW9U2EYYTFDYWQE\ns59zs5BS9e5EODoOhKLD4Ysn6zZdw6hPRGUoRGSYiCwQkRIRuSmEu4jIeO8+R0T6+/NdRGSKiMwT\nkbkicnVAmHEislJEZvstSd8INeob1ZUw53n3ldj8/es2bRHofxms/sJthmF8l1oNhYhkAw8Dw4He\nwDki0jvI23Cgp99GA4/685XAdaraGxgMXBkU9n5V7ee3fRZGMhoP37wNZavdwkLpoM95kNMMvngq\nPekbRqYTTYtiEFCiqotVtRyYCIwI8jMCmKCO6UAbEemgqqtV9QsAVd0GzAc6JVG/0QCY8Si06gwH\nn5qe9PPaQu8z4ct/QMWO9GgwjEwmGkPRCVgecLyC7xb2tfoRkW7AEcCnAad/5buq/i4ibaPUbDQg\nNpbAoneh/2g3RpEu+l8Gu7fC3FfTp8EwMpU6+WuKSAvgdeA3qrrVn34Ut8y9+t/7gO90PojIaFx3\nFkVFRRQXF9eF5H0oKytLS7q10RB0LXr0ICS7E7t6T6e4uDy1wgivTRXyugxiyl8q2Nx1Vsp1RKsr\n3Ziu2MhUXQmjqhE34GjgnYDjm4Gbg/w8DpwTcLwA6OD3c4F3gGsjpNEN+Ko2LQMGDNB0MGXKlLSk\nWxv1XVf5DtW7C1RfOTO1egKJpG3aParjUF03t+701FDf72VdY7piA5ihtZSvkbZoup4+B3qKSHcR\naQKcDUwK8jMJuNDPfhoMbFHV1SIiwNPAfFX9a2AAEekQcHgG8FU0hs1oOMx71X22Y+Av063E0W+U\nG9T+5P50KzGMzKJWQ6GqlcBVuFbBfOAVVZ0rImNEZIz3NhlYDJQATwI1f/1jgQuAH4aYBnuPiHwp\nInOAocA1ScuVUS/4/BEo7AXdhqRbiSN/P+g7CuZMgG2r063GMDKHqMYo1E1dnRx07rGAfQWuDBFu\nGiBh4rwgJqVGg2LFdFj5KZz8gHuXIVM4+lqY+Th89jc44c50qzGMzMDezDbSwgd/hLx20P8X6Vay\nL+16wvfOcFN2d29LtxrDyAzMUBh1zsrPoeT/4OjroEmLdKv5LsfcALs2w6yn063EMDIDMxRGnTP1\nT+7z3oOuSreS0HQ+Cg74Pky/3y3NahiNHTMURp2y+gv45i0YfA00bZluNeE55rew5VtbKtUwwAyF\nUcdM/RM0bQ2DfpVuJZE5+DTofDT8v7HujW3DaMyYoTDqjBWfukWCBv8GmrVOt5rIiMCwB2H7Wph6\nR7rVGEZ6MUNh1AlV5fDWZdCyk5uCWh/odKR7CW/6/VC6MN1qDCN9mKEw6oSP7oV1X8Kpj0DTVulW\nEz0/vNMtmfrf69OtxDDShxkKI+Vs+Bqm/hEOHQmH/DjdamKjZQf4/i2wYBJ88+90qzGM9GCGwkgp\nWu26nHLzYdj4dKuJj8G/gf37wJsXwqYl6VZjGHWPGQojpfy/38O30+Ck+6BFUbrVxEdOU/j5P6G6\nCl75qS1uZDQ+zFAYKePzR2DanW5RoH6j0q0mMQp6wE//AWtmw7+vcOtXGEZjwQyFkRLm/xMmX+XG\nJE59JLM+/BcvB58Kx49zL+F9cJsZC6PxkMbFJ42GyvqphUy7EzoPhp+9lN4lTpPN8b+HzYudodiy\nHE57DLJz063KMFJLA/oLG+mmche8+1uY99BhdBoE57wFuc3TrSq5SBaMeBZaHQAf3g5bl8NZr2b+\nC4SGkQjW9WQkhZWfwdPHwOcPQeezlnPxh9C8XbpVpQYR+OGf4MdPw9Ip8NAhMPMJqK5MtzLDSA1R\nGQoRGSYiC0SkRERuCuEuIjLeu88Rkf61hRWRAhH5r4gs9L9tk5Mlo65QhZJ34Lmh8NRR7iN6Z0+C\ng365iOwm6VaXeo64BC752K1h8fbl8FhfmPOCfRvKaHjU2vUkItnAw8CPgBXA5yIySVXnBXgbDvT0\n21HAo8BRtYS9CXhfVe/yBuQm4MbkZc1INlXlsGmxW09i8buw+D0oW+M+y3HSfW52U9OWsLo43Urr\njk5Hwqip7htW790Ib1wA2U2hx8lw0DBo3w+K+mTmuhuGES3RjFEMAkpUdTGAiEwERgCBhmIEMMEv\niTpdRNqISAegW4SwI4AhPvxzQDEpMhSrZsDGkvjDr5u3P1+tSZ6eZBFKV9iZOOrdAn+r3VZdBdUV\nruukcrd7T6ByJ+zaAjs3wI5S1xe/aQlolYuu+X5w4InQ81Q49CwaRQsiHCJuVbxeI9yHD+e+AvNf\nc29zOw/QqjO07Oje9G6+HzRp6YxHk3x37VYu7cjMhW4MJCvb/SJ+tpgEzRqTfdNOJfXp2c8EUqmr\n6/Hu+UkH0RiKTsDygOMVuFZDbX461RK2SFVrlrBfA4R8HUtERgOjAYqKiiguLo5C8r4sfKAnq/7V\nKeZwe+nN/ARCp47U6ZLcanKaV5LbuoKc1hU07VROl6N30LzzTvIP3E7+gWVIFmwEPvx437BlZWVx\n3ae6oC60NRsB/X4Mu9c1paykBWUlLdi1Ko/tpU3YNLspFVtyqdqZTfWu7IBQB5NAXSaFNL5nPzFS\np6vP3XMoGLQxRbFHJiNmPamqikjIurCqPgE8ATBw4EAdMmRIzPEP7AW77opf32effcagQYPijyBF\nhNUVppYZWDuVrL211qycvVtOU8jJg6zsLKCJ32KjuLiYeO5TNKxdu5azzjqLWbNmMXr0aO67776M\n0RYrWg0VO11rbuqUjxh85LF7WnhaDXc/OI7Fy0p47N4X9oYJ+Jccd+qh3P2Hhzl20JDvxP3RZ8Vc\nccP5zCleETLtq24eRceizvzuN7dH1Fjvnv00k0pdrTofnr4uTFWNuAFHA+8EHN8M3Bzk53HgnIDj\nBUCHSGFr/Pj9DsCC2rQMGDBA08GUKVNSEu8//vEPHTBggObn52v79u112LBh+uGHH6qq6q233qrn\nnXfed8IAunDhwj265s6dq6effrq2atVKW7RooUOGDNGPPvponzC7d+/WW2+9VXv06KHNmzfXrl27\n6sUXX6xLlizZ4+ett97SI488Ups3b64FBQV67rnn6vLly/e4P/PMM3rsscdGzM9FF12k2dnZ+tpr\nr+05d/nll2t+fr7m5+drbm6u5uTk7DkeNmyYLlmyRIE952q2iRMnhkzjj3/8o55xxhlaXV0d+eKG\nIfBeXnTRRTp27Ni44omXW265RQ877DDNzs7WW2+9NaSuGsI9A9EwZcoU7dSpU1j3aPOezGc/8NmN\nhuOPP16ffPLJkG6x6jrrrLP0nXfe0V27dmlRUdE+bi+//LIeffTRmpeXp8cff3xM8Saqq64AZmgt\n5WukTbSW10tFJAf4BjgBWAl8DpyrqnMD/JwKXAWcgutaGq+qgyKFFZF7gVLdO5hdoKo31KJlPbAs\nouDUUAhsSHKcRUB7XH62Agq0Alriuug6Ak2B4M/QDQC+Anb78O2B9bjuO/VaO+Gu+3YfpgeQ69Pa\nAWQDBd7/BqAtbjxpGbDJu3f2WuYBVUA7H/eCMPnJAvr6OLcCi0P4CZWnJkAfYGaYeIPpClQAq6L0\nH0zgvewGlCcQVzy0w+nfD9gZkHaoZyzcMxANLYHuwJww7t2ILu/JfPYDn91oOAQoDZN+rLoOA+bj\nrmdn3P+jhpa43pVmuP9guGc8GlJRViSDQ1Q1/sWHo7EmOAPwDbAIGOvPjQHG+H3BzW5aBHwJDIwU\n1p9vB7wPLATewxmKuC1eKjcStMYh4msNlAFnRfAzDnghxHkFevj9UmByCD+PAlP9/om4AqlLmHQE\nZyBuCDqfhftT/9EfjwKmRdB7IW486mpgZ7R5whVYCuREcd2exRWy5f76neh13uSfr1LglcBnCXgV\nZ0S3AFOBr/z50UFxvRV8fQPSvN3vD8EZ8Rt9nM/786cBs4HNwMfA4VHk5QVgXKRnzF+vV4AJwDZg\nbtB/aylwot/P81o34Yz7b4EVAX6PAL7w8bwMTKzJV6Q8ADN8OtfjjM4WH75ZmHz1AD7w/jYAL/vz\nU/213e6v989xFZS3cRWdTX6/s/d/B66Cssv7f8if7wX8F6jEFegjo7jWbYHFfv8K4J4w/i4FihP8\nbye1rEjWlqiutGegPmzJvvnAMP+ghy0cic5QVAAXh/Az1P/J8oC7gA8ipNPLx9k9hNttwCd+fxSR\nDcX7wD24lpICA6LJEzEYCu//2aAC7mpgOq6W2BTXDfpSgPsluBpjU+ABYEe4uIKvb7AfnKGoBO72\n8eXhCuB1uJZ0NnARrmBtWks+ojUUu3CVrWzgz8D0APel7DUUdwEf4lqKXXBGfoV3a4KrDFyDa1me\n6Z+dmnyFzQN7DcVnuBZOAa5mPiZMvl4CxuIMeDPguAjXth3wM6C5v0evAm8GuBcDlwYc5+MqIxd7\nXUfgjFHvMFpOwBm+MlyFYLP/3e73jw/yb4YizGZvZqeHdsAGVa3tXd6RIrI5cAtyzwFWhwi3GvdH\nLfBphfJTQ2FAmFDxFIY4vw8icgDOOL2oqmtxXU8X1hYuiA1Bef1elOHG4FqqK1R1N65wPdN3e6Kq\nf1fVbQFueSKSyAc3qoFbVXW3qu7EtUweV9VPVbVKVZ/Dda0MTiCNQKap6mRVrQKex3XvhWIkcIeq\nblTV5UDg6h+DcQbiAVWtUNXXcN3ANUSTh/GqukpVNwJvAf3C6KjAdQ92VNVdqjotXMZUtVRVX1fV\nHaq6DdeKOD6cf1yrZ6mqPuPDzwJeB84KE//7qtoGeNP76YQzeoWq2kZVP4iQlhGAGYroeCLJ8ZUC\nhTWFWQRe8Q/0ni3IfStuIkAwHXAF2iafVqTZ1zX9qeHiiaa/9QJgvqrO9sevA+eKSCyfyysMymu0\nswy7Am8EGNL5uNZUkYhki8hdIrJIRLbiCgmIwvhFYL2q7gpK/7ogY94FV/uOhXDPWOCs/B1AszDP\nTUf2nYq+LMhtpfqqZQj3SHmo0RWsI9z8mxtw3ZmfichcEbkkjD9EpLmIPC4iy/z9mQq08S/qhqIr\n7kXezUBv/3sebpwuVPwrvJ9zcO9qrfNxrBaRv4bTlSDJLiuSRUK6zFBEgbopusnkE1yN7ScJxvMW\noWtTI3FdRjtw4z+DRKRzmDgW4Prd94lHRLJw3QLvR6HjQuBAEVkjImuAH+MK41OiykViLAeGBxmZ\nZqq6EjgX92LnibhxoW4+TM0E4lAzOXbgukJqCC6EgsMsx9XkA9NvrqovxZKJJDxjq3GFew0HBLl1\nEtnn9bxA97B5iFWXqq5R1ctUtSNwOfCIiPQI4/063ID1UaraCviBPx/u/izHdaPW6Gujqi1U9Yow\nWjrjunnf85WsJ4ArfbhrY8lXtKSgrEgKieoyQ5EGVHUL8AfgYRH5ia9Z5YrIcBG5J4aobgOOEZE7\n/LezWorIr3AF940+rfdwg39viMgAEcnx/saIyCW+lnk9cIuInCsizUSkPfAUbgbI/QHpiXcP3I4G\nDsK9wd/Pb4cBLxJ791M8PAbcISJdvcD9RGSEd2uJM8iluML/zqCwa4EDg87NxrWGskVkGJG7QgCe\nBMaIyFH+m2f5InKqiIScYeLvczPcfy/HX8NwNehYeAW4WUTa+krBrwLcPsGNrfzap/9T3P2KKw+R\nEJGzAiolm3CFfbU/Dr7eLXETLTaLSAFwa1B0wf7fBg4WkQt8PnJF5MhauikH4AbxAfrjxjaCNWf7\ne5IDZPl7Yh+PD8AMRZpQ1fuAa4FbcLM+luOmGL8ZQxwLgeNw/dZLcTXHnwEnq+pHAV7PBCbjZqts\nwQ10DsS1NlDVl3HdR9fgCtV5uIHaY1W1NCCeY3B/7MDtF8C/VPVLX5tco6prgAeB03wBEA2bRaQs\nYIu2xvcgMAl4V0S24Qa2a97+n4DrYlnp8zQ9KOzT+C4MEam57lcDp+MGO8+jlvuhqjOAy4CHcAVj\nCW7gPxxOy7K1AAAgAElEQVRP4q7bObhB3524a58ot+HyugR4FzeeUaOxHPip17URN+PonwnkIRJH\nAp+KSBnuvlyt/hM+uDGi5/z1HombXJCH696cDvwnKK4HceNNm0RkvB/HOAk4Gzetdw17JxaEYwDw\nhW9N9cLNHAvmAtx9eBT4vt9/MqZcN3BqfY/CMAzDaNxYi8IwDMOIiBkKwzAMIyJmKAzDMIyImKEw\nDMMwIpIRnxmPlsLCQu3WrVudp7t9+3by8/PrPN3aMF2xk6naTFdsmK7YmDlz5gZV3S/uCBL5/kdd\nbw3tM+OJYrpiJ1O1ma7YMF2xgX3ryTAaHzsp55/M4jOWsJOKdMsxGjj1quvJMAzHdJbwFav4ilW8\nx9f0pgMn05u8OFYkNIzaMENhGPWM3VTwOUvpRRHfpyez+JaZLKclzTiBXumWZzRArOvJMOoZM/mW\nXVRyLD3oQGtOoQ8HUciXrERDfufQMBLDDIVh1CMqqWI6S+hOIZ3Y+9X5vnRmK7tYSmmE0IYRH2Yo\nDKMeMZsVlLGb4zhon/MHU0RTcpjDyjQpMxoyZigMo55QTTUfs4iOtKEb7fZxyyWb79Ge+aymgqo0\nKTQaKmYoDKOesIgNbGYnx3EQgnzH/XA6U04VX++zGJ1hJI4ZCsOoJyxmAzlk0YPQL9h2pYDW5Fn3\nk5F0zFAYRj1hCRvoQltyCL0gniD0oSOLWU8Zu0L6MYx4MENhGPWAMnazjm10pzCiv8PpjALzWF03\nwoxGgRkKw6gHLGUDQK2GopAWtKW5TZM1kooZCsOoByyhlGbk0IHWtfo9gAKWsdFevjOShhkKw6gH\nLGEDXWlHVojZTsF0pYCdVLCesjpQZjQGzFAYRoaziR1sZmet3U41dPXvWCyz7icjSZihMIwMZ8me\n8Yl2tfh0tCGPVjRjGRtTKctoRJihMIwMZwkbaEFTCmkRlX9B6GrjFEYSMUNhGBmMoiyhlO4Uhnwb\nOxxdacd2drOR7SlUZzQWzFAYRgazjm3soDzqbqcaDqAAwLqfjKRghsIwMpia9yGiHciuoR355NPU\nBrSNpGCGwjAymOVsojV5tCYvpnA2TmEkEzMUhpGhKMq3bKQLbeMK35UCtrKLzexMsjKjsWGGwjAy\nlE3soIzde8YbYsXepzCShRkKw8hQlrMJIG5DsR8tyCOXb21A20iQhAyFiAwTkQUiUiIiN4VwFxEZ\n793niEj/ALelIvKliMwWkRmJ6DCMhsi3bKQZOewX5fsTwQjCARSYoTASJm5DISLZwMPAcKA3cI6I\n9A7yNhzo6bfRwKNB7kNVtZ+qDoxXh2E0VJazkS4UxPT+RDAHUMBGdrDN1qcwEiCRFsUgoERVF6tq\nOTARGBHkZwQwQR3TgTYi0iGBNA2jUbCd3Wxge9zdTjXUhLdWhZEIohrf1DkRORMYpqqX+uMLgKNU\n9aoAP28Dd6nqNH/8PnCjqs4QkSXAFqAKeFxVnwiTzmhca4SioqIBEydOjEtvIpSVldGiRXzN/1Ri\numInU7UF69pQWMm8w3bT94tmtN4aekW7aFBRPjpuB+3X5NBjYdOEdWUKpis2hg4dOjORnpucZIqJ\nkeNUdaWI7A/8V0S+VtWpwZ68AXkCYODAgTpkyJA6lgnFxcWkI93aMF2xk6nagnW9yzyyWcap/YeE\nXfo0WlbyKWWddjOk0w8S1pUpmK66JZGup5VAl4Djzv5cVH5UteZ3HfAGrivLMAzcjKeOtE7YSIDr\nflrHNnZSngRlRmMkEUPxOdBTRLqLSBPgbGBSkJ9JwIV+9tNgYIuqrhaRfBFpCSAi+cBJwFcJaDGM\nBkMFVaxmS8LjEzV03TNOsSkp8RmNj7i7nlS1UkSuAt4BsoG/q+pcERnj3R8DJgOnACXADuBiH7wI\neENEajS8qKr/iTsXhtGAWMlmqtGkGYpOtCGbLL5lI4dQlJQ4jcZFQmMUqjoZZwwCzz0WsK/AlSHC\nLQb6JpK2YTRUllKKQNyf7ggmh2w60tpmPhlxY29mG0aGsYj1dKQNzchNWpwHUMBqtlBOZdLiNBoP\nZigMI4PYSQWr2MxB7JfUeLtSQDXKCjYnNV6jcWCGwjAyiCVsQIGDYlx/oja60BbBPhBoxIcZCsPI\nIBaxnqbk0Ik2SY23Kbl0pA2LWJ/UeI3GgRkKw8gQFGUR6+lOIVkp+GseTBGr2GLffTJixgyFYWQI\npWxnK7uSPj5Rw8HsD8BC1qUkfqPhYobCMDKEmm6hZI9P1LA/LWlNHt+wNiXxGw0XMxSGkSEsYj3t\nyKcNzVMSvyAcQhGL2UAFVSlJw2iYmKEwjAygWpSllHJgiloTNfRkfyqpZjEbUpqO0bAwQ2EYGcCW\n1tVUUp2y8YkautGOJuRY95MRE2YoDCMDKC2sJJssutEupelkk0UP9mMh61DiW4vGaHyYoTCMNFNB\nFeuKKvke7WlSB0vEHMz+lLGbVWxJeVpGw8AMhWGkmXmspjIX+nNAnaTXg/0RYIF1PxlRYobCMNLM\nF3xL3g7Zs25EqmlOE7pRyP9YQRXVdZKmUb8xQ2EYaWQd21jOJjqsykWQOkt3MN3Zxi6+YlWdpWnU\nX8xQGEYamcW3ZJNF0dq6Xb6+B/uxPy35mEU2qG3UihkKw0gTFVTxP1bSi/bkVtRdawLcy3fHcCDr\nKbNPehi1YobCMNLEPFaziwr60yUt6R9KR1rRjI9ZnJb0jfqDGQrDSAM7Ked9vmZ/Wqb83YlwZJPF\nYA7kWzaynE1p0WDUD8xQGEYamMxcdlDOCPrW6SB2MP3pQjNymcICqm2swgiDGQrDqGO+YhVzWcXx\n9KQDrdOqpQk5nEgvllLKf5mfVi1G5lK3Uy0Mo5GzlV1M5ks60YZjOSjdcgD3ot96tvEpS2hHPgPp\nmm5JRoZhhsIw6ojVbOENZlGFcgb9UrKKXbz8iN6Usp3/Yy5taE6PFH+c0KhfJPSkisgwEVkgIiUi\nclMIdxGR8d59joj0jzasYTQUqlE+YhFP8xG7qeJsBlJAfrpl7UMWws84gv1owUt8xiT+Z0umGnuI\nu0UhItnAw8CPgBXA5yIySVXnBXgbDvT021HAo8BRUYY1jHpLJVWsYDPfsJZvWMdGtvM92nMafcij\nSbrlhaQpuVzE0XzIQj5nGXNZzX49hPasoRNtaEmzdEs00kQiXU+DgBJVXQwgIhOBEUBgYT8CmKCq\nCkwXkTYi0gHoFkXYpPElK1lGadzhVx28mzLmJFFRcjBdsROvNt1nX/ecU5Qqqqmgmkqq2EE529jF\ndsoBNwW1KwUMoSeH0jGtM5yiIY9cTqI3R9KNKSxgbsdVvMJMwH0jKp8m5NGEZuSSQxZZCNlk7clV\ncP5SldtMfcZSqWsQ3dmflimJuzYSMRSdgOUBxytwrYba/HSKMiwAIjIaGA1QVFREcXFxzEKXdC9n\nbfvKmMPVoAXVbNy9Iu7wqcJ0xU6ytEmN5VDIUsiqFrKqIbdCaLlbKNydS/72LNpsyianaicbWMgH\nLAwbX1lZWVzPdiopAPruUGifx9ZW1exoXk1l7k625O6gNAdUlOosUG8NtA5tYKY+Y6nUVTl/PW02\nZ6ck7lpR1bg24EzgqYDjC4CHgvy8DRwXcPw+MDCasKG2AQMGaDqYMmVKWtKtjcaoa82aNfr9739f\nW7Rooddee23M4evTNbv11lv1vPPOCxumd+/eYfMzZcoU7dSpU9iwF110kY4dOzYuXZmA6YoNYIbG\nWdarakKD2Sthn28PdPbnovETTdgGz4svvsjAgQNp0aIFHTp0YPjw4UybNg2AcePGcf75538njIhQ\nUlKy53jevHn8+Mc/pnXr1rRs2ZKhQ4fy8ccf7xOmvLyccePG0bNnT/Lz8+nWrRuXXHIJS5cu3ePn\n7bffZtCgQeTn59OuXTvOO+88VqzYWzN69tlnOe644yLmZ9SoUeTk5FBaurebb8yYMbRo0YIWLVrQ\npEkTcnNz9xwPHz6cpUuXIiJ7ztVsL7/8csg0nnjiCQoLC9m6dSv33XdfRD21MWrUKG655ZaE4oiV\n3//+9/Tp04ecnBzGjRuXUFxz585lyJAhSdFVFwQ/u7UxZMgQnnrqqaSkPXLkSN599112795N+/bt\n93G7/vrr6dmzJy1btqRXr15MmDAhKWk2JMQZmzgCiuQA3wAn4Ar5z4FzVXVugJ9TgauAU3BdS+NV\ndVA0YcOkuR5YFpfgxCiEpK9GXwS0x+VnK67LuxXQEtcV1xFoCiwJCjcA+ArY7cO3B9YDa3wchbiu\nvW+A7T5MDyDXp7UDyMb1LKjPV1vcuNEyYJN37+y1zAOqgHY+7gVh8pMF9PVxboWQHxAKlacmQB/w\nHeG10xWogLi/jx14L7sB5QnEFQ/tcPr3A3YGpB3qGQv3DERDS6A7hO0w70Z0eU/msx/47EbDIUBp\nmPRj1XUYMB93PTvj/h81dAQ2AruAfNzkm4Xs/f/EQirKimRwiKrGP8CRSHMEZwC+ARYBY/25McAY\nvy+42U2LgC+BgZHCZupGgs22EPG1BsqAsyL4GQe8EOK8Aj38fikwOYSfR4Gpfv9EXIHUJUw6gjMQ\nNwSdz8L9qf/oj0cB0yLovRA37nQ1sDPaPOEKLAVyorhuz+IK2XJ//U70Om/yz1Ep8ApQEBDmVZwR\n3QJMBb7y50cHxfVW8PUNSPN2vz8EZ8Rv9HE+78+fBswGNgMfA4dHkZcXgHGRnjF/vV4BJgDbgLlB\n/6GlwIl+P89r3YQz7r8FVgT4PQL4wsfzMjCxJl+R8gDM8OlcjzM6W3z4ZmHy1QP4wPvbALzsz0/1\n13a7v94/x1VQ3sZVdDb5/c7e/x24Csou7/8hf74X8F+gEldpGRnFtW4LLPb7VwD31OJ/EnBdnP/t\npJYVydoS1ZX2DNSHLdk3HxjmH/SwhSPRGYoK4OIQfob6P1kecBfwQYR0evk4u4dwuw34xO+PIrKh\neB+4B9dSUmBANHkiBkPh/T8bVMBdDUzH1RKbAo8DLwW4X4KrXTcFHgB2hIsr+PoG+8EZikrgbh9f\nHq4AXodrMWcDF+EK1qa15CNaQ7ELV6nKBv4MTA9wX8peQ3EX8CGupdgFZ+RXeLcmuMrANbiW5Zn+\n2anJV9g8sNdQfIareRfgauZjwuTrJWAszoA3Y98xyuBr2w74GdDc36NXgTcD3IuBSwOO83GVkYu9\nriNwxqh3GC0n4AxfGa5CsNn/bvf7x4cIkwesBobF+d9ukIYic14NbVy0Azaoam1TsUaKyObALcg9\nB/dQB7Ma90ct8GmF8lNDYUCYUPEUhji/DyJyAM44vaiqa3FdTxfWFi6IDUF5/V6U4cbgWqQrVHU3\nrnA903dvoqp/V9VtAW55IpLIB5aqgVtVdbeq7sS1TB5X1U9VtUpVn8N1rQxOII1ApqnqZFWtAp7H\nde+FYiRwh6puVNXlwPgAt8E4A/GAqlao6mu47t4aosnDeFVdpaobgbeAfmF0VOC6Bzuq6i5VnRYu\nY6paqqqvq+oOVd2Ga0UcH84/rtWzVFWf8eFnAa8DZ4WJ/31VbQO86f10whm9QlVto6ofhAj2GPA/\n4J0IOhodZiii44kkx1cKFNYUZhF4xT/Qe7Yg961AhxDhOuAKtE0+rVB+aqjpTw0XTzT9rRcA81V1\ntj9+HThXRHKjCFtDYVBeo/1CXVfgjQBDOh/XmioSkWwRuUtEFonIVlwhAVEYvwisV9XAV5a7AtcF\nGfMuuNp3LIR7xtYE7O8AmoV5bjqy75TzZUFuK9VXLUO4R8pDja5gHS3C6L0B1535mYjMFZFLwvhD\nRJqLyOMisszfn6lAG/9Cbii64l7Y3Qz09r/n4cbpQsW/wvs5B3gO12rqCqwWkb+G8H8vbixjZNC1\nioVklxXJIiFdZiiiQFWTffM/wdXYfpJgPG8RujY1EtdltAN4DxgkIp3DxLEA1+++TzwikoXrFng/\nCh0XAgeKyBoRWQP8GFcYnxJVLhJjOTA8yMg0U9WVwLm4FzlPxI0LdfNhamb8hyoMduC6QmoILoSC\nwyzH1eQD02+uqi/FkokkPGOr2Xcm4QFBbp1ERMK4h81DrLpUdY2qXqaqHYHLgUdEpEcY79fhBqyP\nUtVWwA/8+XD3ZzmuG7VGXxtVbaGqV4TR0hnXzfuer2Q9AVzpw10b6FdEbsN9SeIkVd0aS56D0sxI\nQ5GoLjMUaUBVtwB/AB4WkZ/4mlWuiAwXkXtiiOo24BgRuUNECkSkpYj8Cldw3+jTeg83+PeGiAwQ\nkRzvb4yIXOJrTtcDt4jIuSLSTETaA0/hZmHdH5CeePfA7WjgINyb+v38dhjwIrF3P8XDY8AdItLV\nC9xPREZ4t5Y4g1yKK/zvDAq7Fjgw6NxsXGsoW0SGEbkrBOBJYIyIHOW/bZYvIqeKSMgZJv4+N8P9\n93L8NUzGW1SvADeLSFtfKfhVgNsnuLGVX/v0f4q7X3HlIRIiclZApWQTrrCv9sfB17slbqLFZhEp\nAG4Nii7Y/9vAwSJygc9HrogcWUs35QDcID5Af9zYRrDmm3GVihNVNf5PODRgzFCkCVW9D7gWuAU3\n62M5birxmzHEsRA4DtdvvRRXc/wZcLKqfhTg9UxgMm62yhbcQOdAXGsDVX0Z1310Da5QnYcb1Ds2\n6I9zDO6PHbj9AviXqn7pa5NrVHUN8CBwmi8AomGziJQFbNfWHgR8OpOAd0VkG25gu+Yt/wm4LpaV\nPk/Tg8I+je/CEJGa6341cDpusPM8arkfqjoDuAx4CFcwluAG/sPxJO66nYMb9N2Ju/aJchsur0uA\nd3HjGTUay4Gfel0bcTOO/plAHiJxJPCpiJTh7svV6j/Vgxsjes5f75G4yQV5uO7N6cB/guJ6EDfe\ntElExvtxjJOAs3HTetewd2JBOAYAX/jWVC/czLFg7sS1sEoCnr/fxZrxhkzc71EYhmEYjQNrURiG\nYRgRMUNhGIZhRMQMhWEYhhERMxSGYRhGROrVmtmFhYXarVu3Ok93+/bt5Odn1tKVYLriIVO1ma7Y\nMF2xMXPmzA2qGv9C6Il8/6OuN1uPYl9MV+xkqjbTFRumKzawbz0ZRi3MmQM/+QkUFHD4b38Ld94J\nM6P9qrlhGGYojIbLihVw9tnQty9MmQKnnkqT0lIYOxYGDoTx42uPwzAMMxRGA2XLFjjpJHjrLfjd\n72DJEnj+eWb8/e+wfj389Kdw9dWQ4Cp5htEYqFeD2YYRFVVVcO65sHAhvPsuDB26r3thIUycCOed\nB9dfDxUVcNNN6dFqGPUAMxRGw+Pmm2HyZHjkke8aiRpyc+HFF93vzTdDr15uHMMwjO9gXU9Gw+Kl\nl+Dee+GKK9wWiZwcePZZOPxwuOoq2LatTiQaRn3DDIXRcNi6FX7zGxg8GB58MLowubnw+OOwahXc\ncktq9RlGPcUMhdFw+POfYd06N5spN4bF9QYPdq2Pv/0NPv+8dv+G0cgwQ2E0DJYuhfvvh/PPhyOP\njD38nXdC+/Zw+eVQWdtS5obRuEjIUIjIMBFZICIlIvKdaSN+tazx3n2OiPQPcs8WkVki8nYiOgyD\nm2+GrCxX4MdD69auu2rWLJgwIbnaDKOeE7eh8Ms3PoxbZ7Y3cI6I9A7yNhzo6bfRwKNB7lcD8+PV\nYBgAfPKJm+56/fXQpUvt/sNx5pkwYIAzNtaqMIw9JNKiGASUqOpidUstTsQtZB/ICGCC/9zIdKCN\niHQA8Ovqnopbm9kw4ud3v3PdRjfckFg8IvCHP8CiRW72lGEYQGLvUXTCrfNcwwr2rlUcyU8n3NrO\nDwA34BZYD4uIjMa1RigqKqK4uDgByfFRVlaWlnRrw3RBy3nzGFBcTMkVV7Bixoxa/deqrWVLBh50\nEFljx/JZx46QnZ08sYnoShOmKzYyVVfCxPs1QeBM4KmA4wuAh4L8vA0cF3D8PjAQOA14xJ8bArwd\nTZr29dh9MV2qesYZqm3bqm7dGpX3qLS99poqqL74YmLaYsDuZWyYrtggjV+PXQkEdgh39uei8XMs\n8GMRWYrrsvqhiLyQgBajMfL11/Dmm3DlldAyYsM0Ns44Aw49FG6/HaqrkxevYdRTEjEUnwM9RaS7\niDQBzgYmBfmZBFzoZz8NBrao6mpVvVlVO6tqNx/u/6nq+QloMRoj994LzZrBr3+d3HizsuD3v4d5\n85whMoxGTtyGQlUrgauAd3Azl15R1bkiMkZExnhvk4HFQAnwJPDLBPUahmPFCnj+efjFL2C/+Bfu\nCsuZZ0L37vDXvyY/bsOoZyT0UUBVnYwzBoHnHgvYV+DKWuIoBooT0WE0Qh54wHULXXddauLPznYt\nlWuucW9rx/MSn2E0EOzNbKP+sW0bPPkkjBwJqVxD/ZJL3NjH/fenLg3DqAeYoTDqH88+u/cDgKmk\nVSu49FJ49VXX1WUYjRQzFEb9orraffTv6KNh0KDUp/frX7s0H3oo9WkZRoZihsKoX0yeDCUlbhnT\nuqBbNzdd9oknYPv2uknTMDIMMxRG/eLBB6FTJ7fmdV1xzTWwaZN9LNBotJihMOoPc+fCe++51ehi\nWW8iUY45BgYOdF1e9gKe0QgxQ2HUHx58EPLy4LLL6jZdEdfV9fXX8O67dZu2YWQAZiiM+sGGDe4F\nu/PPh3bt6j79kSPdF2qjXWLVMBoQZiiM+sHjj8OuXamfEhuOJk3gl7+E//zHtSwMoxFhhsLIfHbv\ndtNTTz4ZegevjVWHXH45NG3qxioMoxFhhsLIfF5+GdascbOP0sn++8O558Jzz7lZUIbRSDBDYWQ2\nqu4TGr17w0knpVuNG9TescO9V2EYjQQzFEZm88EHMHu2G5sQSbca6NsXTjjBDWrv3p1uNYZRJ5ih\nMDKb+++HwkI32ylTuPFGWL0aXrC1tozGQUKGQkSGicgCESkRkZtCuIuIjPfuc0Skvz/fRUSmiMg8\nEZkrInX0PQajXjF3Lkya5GYb5eWlW81eTjwRjjjCLZxkL+AZjYC4DYWIZAMPA8OB3sA5IhI8JWU4\n0NNvo4FH/flK4DpV7Q0MBq4MEdZo7Nx5J+TnJ38Fu0QRca2KBQvgX/9KtxrDSDmJtCgGASWqulhV\ny3FrX48I8jMCmODX954OtBGRDn451C8AVHUbboW8TgloMRoaJSUwcSJccUV6XrCrjZ/9DA48EO6+\n2w24G0YDJhFD0QlYHnC8gu8W9rX6EZFuwBHApwloMRoad93lvueUqhXsEiUnB66/Hj79FKZOTbca\nw0gpCS2Fmigi0gJ4HfiNqm4N42c0rtuKoqIiiouL606gp6ysLC3p1kZD1dV07VqOevZZVp9+Ogu/\n/jqpb0In85plHXggg9u2Zfs11/C/++5LaFZWQ72XqcJ01TGqGtcGHA28E3B8M3BzkJ/HgXMCjhcA\nHfx+LvAOcG20aQ4YMEDTwZQpU9KSbm00WF1XXaWak6O6bFlS9ASS9Gv24IOqoPruuwlF02DvZYow\nXbEBzNA4y3pVTajr6XOgp4h0F5EmwNnApCA/k4AL/eynwcAWVV0tIgI8DcxX1b8moMFoaCxb5tbD\nvvBCOOCAdKupncsvh65d4Xe/s7EKo8ESt6FQ1UrgKlyrYD7wiqrOFZExIjLGe5sMLAZKgCeBX/rz\nxwIXAD8Ukdl+OyVeLUYD4ne/c104t96abiXR0bQp3HYbzJgB//xnutUYRkpIaIxCVSfjjEHguccC\n9hW4MkS4aUAGvGZrZBSffgovvghjx9aP1kQN558P99zjdI8Y4Qa6DaMBYW9mG5mBKlx7LRQVuXcU\n6hPZ2XDHHe69iueeS7caw0g6ZiiMzOC11+Djj+H226Fly3SriZ0RI9ySqTfd5BZZMowGhBkKI/3s\n2OFaEX36wMUXp1tNfIjAY4/B5s2Z++6HYcSJGQoj/dx4IyxZ4r7Imp2dbjXx06ePa1FMmGBraxsN\nCjMURnp59123et1vfgNDh6ZbTeKMHQuHHOKmzW7fnm41hpEUzFAY6WPjRtfV9L3vuQ8ANgSaNXPv\ngSxdCjffnG41hpEUzFAY6UEVrrwS1q1z6zpk0mfEE+X733dfvP3b3+D559OtxjASxgyFkR7+8hf3\nddhx46B//3SrST5/+QsMGQKXXQaffZZuNYaREGYojLrn+efhhhvg5z9vuN0zubnw6qvQoQOccYZb\nEc8w6ilmKIy65T//gUsucetOP/ccZDXgR7Cw0C1stGULnHIKrF2bbkWGERcN+F9qZBz//rdb8KdP\nH/ddpKZN060o9Rx+OLz+untr+7jjYPHidCsyjJgxQ2GkHlU3q+n0093U0cmToVWrdKuqO04+Gd5/\nH0pL4dhjYfbsdCsyjJgwQ2Gklk2b3FjE2LFwzjkwbRq0b59uVXXP0Ue7vOfkuP2774bKynSrMoyo\nMENhpIaKChg/Hnr0cF0v997rpsE2b55uZemjd283A2rYMPcG96BBMHNmulUZRq2YoTCSy8aNdHzz\nTTjsMLj6ajf1ddYst750AkuFNhg6dHDjM6+9BqtWwcCBcOKJtPv4Y6iuTrc6wwhJQoZCRIaJyAIR\nKRGRm0K4i4iM9+5zRKR/tGGNeoIqLFwIzzwDZ50FHTpw8IMPupbD22+7T3Qcfni6VWYWIm5Q/+uv\n4c9/hgUL6DN2LHTvDr/8pZsptW1bulUaxh7iXmFFRLKBh4EfASuAz0VkkqrOC/A2HOjpt6OAR4Gj\nogxrpJvycigrc98s2rwZ1q9324oVUFLitjlz3NvVAPvvD1dcwYzDDmPgpZemV3t9oE0b1wV13XXM\nvf12Dp01y31Q8NFHnTE58EA49FDXZdW5M3Ts6MZ32rSB1q3d59jz8myhJCPlJPKEDQJKVHUxgIhM\nBEYAgYX9CGCCX+luuoi0EZEOQLcowiaPcePgH/+IO/ignTsz8hMTe3SFW6s58HzNvuq+W3U1VFXt\n3SornYEoL4/cFdK2LfTs6frbjz3WfbbikEMgK4uy4uKk5bFRkJvL+qFD3ZKq5eVuXY6pU+Grr9w2\neWrOTDIAAAZ1SURBVHLkge/sbPeNqdxcZzSys92WleW2mi4/kdD7EThq586MHFfK+P9kKnj6afjB\nD1ITdy0kYig6AcsDjlfgWg21+ekUZVgARGQ0MBqgqKiI4jgKofY7d9K2a9eYw9VQWVnJtgyste2j\nK8yfXkOd94WE1vxmZaG+YKnOyUFzcqjOzaW6SROqmjWjKi+Pqvx8Ktq0obx1a8rbtaMyeHrr2rV7\nXigrKyuL6z7VBZmq7Tu6fvCDvYVCVRW5W7bQtLSUJhs3krN9O9llZeTs2EFWeTlZFRVklZcjVVVI\nZSVSVQWqSHU1UmPsayoGHglXuQiisqKCnEx/9jOIVOr69ptv2J6mcazMu9JBqOoTwBMAAwcO1CFD\nhsQeSTxhAiguLiaudFOM6YqdTNVmumKjMeoqSkms0ZGIoVgJdAk47uzPReMnN4qwhmEYRgYgGmUT\n9DsBRXKAb4ATcIX858C5qjo3wM+pwFXAKbiupfGqOiiasGHSXA8si0twYhQCmbgQsumKnUzVZrpi\nw3TFxiGqGvdi9HG3KFS1UkSuAt4BsoG/q+pcERnj3R8DJuOMRAmw4/+3d26hVlRhHP/90TS1SO1k\nFxQ8RQY+pVTYldLoKp4ehQTt8hIR3Sg0IegtLaq3JLSILEPMTKTAbthTmljHjqlZaKVpx4guFKjB\nv4e1Tm0O7umcs3Ovqb4fDHvNmj3sHzOz5tuzZs03wG1V6w7gN88Yqm8rSNpq+6ISv11FeA2eurqF\n1+AIr8EhaWsr67d0j8L2m6Rg0Fi3rKFs4O6BrhsEQRDUj3gyOwiCIKgkAsXAeK60QBPCa/DU1S28\nBkd4DY6WvIZ8MzsIgiD4fxBXFEEQBEElESiCIAiCSiJQ/A11yXIraZKk9yV9JmmHpHtz/XhJb0va\nkz/HFfIbJuljSRvq4pVzi62RtEvSTkmX1sTr/rwPeyStknRyCS9Jz0vqldTTUNfUQ9Ki3A52S7q+\nzV5P5P24XdLrksbWwath2YOSLKmjLl6S7snbbIekpS152Y6pyUR6xuNL4FxgBNANTC3kcjYwPZdP\nJT2wOBVYCizM9QuBJYX8HgBeATbk+eJewIvAnbk8Ahhb2ouU52wvMCrPrwYWlPACrgKmAz0Ndcf1\nyMdaNzAS6MztYlgbva4Dhufykrp45fpJpGfCvgI66uAFXAO8A4zM8xNa8Yorimr+zJBr+yjQl+W2\n7dg+aHtbLv8C7CSddLpIJ0Ty5y3tdpM0EbgZWN5QXdRL0mmkBrQCwPZR2z+W9soMB0blDAWjgW9L\neNn+APihX3Uzjy7gVdtHbO8lPUR7Sbu8bG+03ZdC90NS2p/iXpmngYeBxpFBpb3uAh63fSR/p7cV\nrwgU1TTLflsUSZOBacBm4EzbB/OiQ5TJHfYMqaE0prYs7dUJHAZeyF1iyyWNKe1l+wDwJPA1cBD4\nyfbG0l4NNPOoU1u4HXgrl4t6SeoCDtju7reo9PaaAlwpabOkTZIubsUrAsW/DEmnAK8B99n+uXGZ\n07VlW8c7S5oN9Npu+vLnEl6kf+3TgWdtTwN+JXWlFPXKff5dpEB2DjBG0rzSXsejLh6NSFoM/A4M\n/QUz/5zLaOAR4NHSLsdhODAemAE8BKyWhv4u4ggU1QwkQ27bkHQSKUi8bHttrv4uvwyK/NnbbP0T\nxOXAHEn7SF1zMyWtrIHXfmC/7c15fg0pcJT2uhbYa/uw7WPAWuCyGnj10cyjeFuQtACYDdyag1hp\nr/NIAb87H/8TgW2SzirsBen4X+vEFtLVfsdQvSJQVPMRcL6kTkkjgLnA+hIi+d/ACmCn7acaFq0H\n5ufyfOCNdnrZXmR7ou3JpO3znu15NfA6BHwj6YJcNYv0BsWiXqQupxmSRud9Oot0v6m0Vx/NPNYD\ncyWNlNRJer3xlnZJSbqB1L05x/Zv/XyLeNn+1PYE25Pz8b+fNODkUEmvzDrSDW0kTSEN5vh+yF4n\n4i78f2kiZb/9nDQ6YHFBjytI3QDbgU/ydBNwOvAusIc0ymF8Qcer+WvUU3Ev4EJga95m64BxNfF6\nDNgF9AAvkUagtN0LWEW6T3KMdJK7o8oDWJzbwW7gxjZ7fUHqW+879pfVwavf8n3kUU+lvUiBYWU+\nxrYBM1vxihQeQRAEQSXR9RQEQRBUEoEiCIIgqCQCRRAEQVBJBIogCIKgkggUQRAEQSURKIIgCIJK\nIlAEQRAElfwBm+ji+xOtRekAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd8VUXa+L9PCiSk0IKhSlcElRKkCAos6oJdKVYEfV3E\nXV3XspZXdy2vfddd9aerorurrgUUO6K4KkUEpIlKEQiCEiD0lkD68/tjJnBzuTf3Jrk39yZ3vvmc\nT845056Ze84zM8/MmRFVxeFwOByxQ1ykBXA4HA5H7eIUv8PhcMQYTvE7HA5HjOEUv8PhcMQYTvE7\nHA5HjOEUv8PhcMQYTvE7ohIRyRSRuSJyQESeiLQ84URE7hOR1ypxXykiQ/24DRWRnErCviwiD4ZA\nTEc9win+WkBELheRJSKSJyJbReQTERls3Xy+9CKiItLF47q7iHwoIvusMpwlIqd6hWlg41snIvki\nslFE/iUiHTz8nCsii6z7LhF5XUTaerhPEJF5AfLzsoiUiEgrj3vP2/zliUiRiBR7XH8iIh1snvK8\njkv8JDMR2Amkq+qtlRZwAGpb+YnIMSLypohssb/X1yLSv7rxqWoPVZ0dQhHDivezG4T/2SJybYjS\nfktEzhKRhiKS6+XW0L4P+0UkV0RuCUWadRGn+MOMfbieBB4GMoFjgWeB86sQR2fga+AHoCPQGngP\n+ExEBnp4nWbjvRxoDPQElgDDbTyjgTesPBlAD6AQmCciTYOUJQUYBewDriy/r6qTVDVVVVNtXqeW\nX6vqSI8omnjcT1XVqX6Sag+s0ij4wlBEEqoYJBVYDGQBzYBXgI9FJDXUsjmOIgvzzJ8MrPByuw/o\ninm2hgG3i8iIWpUuWlBVd4TpwCjfPGBMJX7uA17zcV+BLvb8P8AMH36eA+ba8zOAQ0A7P+kI8DNw\nu9f9OMwL8oC9ngDMq0Teq4BNwE3AimDzBHSweUoIotxeBoqBIlt+Z1g57wTWA7uAt4BmHmHeBnIx\nFdJcoIe9P9Erro+8y9cjzQft+VAgB7jDxvkfe/9cYDmwF5gPnFyFZ2E/kFVJeb0FvAocAFYCfT3c\nNwJn2PNkK+seYBXwRyDHw29vYJmNZyowpTxfgfJg07kN+N6W41QgyY/MXYA51t9OTEWPLXsF8m15\nXwI0BaYDO6zc04G21v9DQClQYP0/Y+93A/4L7AbWAGODKOOmwE/2/HrgcS/3LcBZHtcPAFNqUydE\nyxFxAerzAYwASqhE2RGc4s8FrvbhZ5h9aZKBR4E5laTTzcbZ0Yfb/cACez6ByhX/F8DjmN5LiS9l\n5itPVEHxW/8veymsm4CFQFugIfAC8KaH+zVAmnV7EljuLy7v8vX2g1H8JcBjNr5kjELdDvQH4oHx\nGEXZMIi89LKKrXElz0ABcLaN+xFgoYf7Ro4o/keBrzA9iXaYSjvHujXAVO43A4nAaEylV56vSvNg\nzxdhepTNgNXAJD8yvwncjamQk4DBlZRtc0wvsZH9jd4G3vdwnw1c63GdgmlcXA0kWLl3At39yDIc\nU5HlYSr4vfZ/vj0fgqkUFMj0CDcK+CESuiHShzP1hJfmwE5VLQngb6yI7PU8vNwzgK0+wm3FvHjN\nbFq+/HjGgR8/Wz3c/SIix2IqmzdUdRumErgqUDgvdnrl9YQgw00C7lbVHFUtxCjL0eVmGFX9l6oe\n8HDrKSKNqyibJ2XAvapaqKqHMD2HF1T1G1UtVdVXMGayAZVFIiLpmB7b/aq6rxKv81R1hqqWWv89\n/fgbCzykqrtVdRPwtIfbAIzCf1JVi1V1GsbkVE4weXhaVbeo6m7gI0yl5YtijMmktaoWqKrfcSFV\n3aWq76jqQVU9gGnlD/HnH9Mr2aiq/1bVElX9FngHGOMn/i9UtQnwvvXTBlOJZahqE1WdgzG/gemh\nlLMfUxHFHE7xh5ddQEYQNuK37AN6+PBy3wm08hGuFUZB7bFp+fLjGQd+/LTycK+MccBqVV1ur18H\nLheRxCDClpPhldfVQYZrD7znUTGuxvR2MkUkXkQeFZH1IrIf89JDEJVZJexQ1QKv9G/1qpzbYVrH\nPhGRZIzyXKiqjwRIz3Mg8iCQ5Oe5aY1pDZfzs5fbZrXNWR/uweTBWw5/4xK3Y8yHi+yso2v8+ENE\nGonICyLys/195gJNRCTeT5D2QH8vOa8AWvqJP8f6uQwznrLdxrFVRP5mveXZ/+keQRtjTGIxh1P8\n4WUBpkV1YQ3j+RzfrZ2xGBPNQeunn+cMHS/WYOzWFeIRkThMl/eLIOS4CuhkZ0TkAn/DKNezg8pF\nzdgEjPSqNJJUdTNmMPsCzFhAY4xZCYxiAtPF9+YgxvRQjrdS8Q6zCdPS9ky/kaq+6UtYEWmIaYHm\nANcFl8Wg2IpR1uUc6+XWRkTEj3uV8lAZqpqrqr9R1daY/P2jkpk8twLHA/1VNR043d739/tswpgt\nPeVMVdXr/cjSFmNW/dw2miYDv7PhbrF+9mDKx7Mn1RMznhJzOMUfRmzX/s/AsyJyoW35JIrISBF5\nvApR3Q+cKiIPiUgzEUkTkRsxivgOm9bnmMGw90QkS0QSrL9JInKNbQXeBtwjZnppkoi0BF7CtIL+\n7pGeWHfPYyDQGeiH6f73Ak7EzBKqqrmnOjwPPCQi7a2ALUTkAuuWhqlgd2GU+cNeYbcBnbzuLcf0\nVuLtzI7KTA8ALwKTRKS/GFJE5BwROcpUYHtA0zCD7eNVtSz4bAbkLeAuEWlqK/kbPdwWYMYmfm+f\ns4sxv1eV8xAIERnj0cjYg1He5fn0Lu80TFnsFZFmwL1e0Xn7nw4cJyLjbD4SReSUAGbBLMygNkAf\nzMweb17FPP9NbVy/wYztxBxO8YcZVX0CuAW4BzOrYRNwA6Y1GGwc64DBmBbKRkzLZRTwa1X92sPr\naGAGZjbGPszAX19MbwA1UyfHYQb/dmFmhSQDg1R1l0c8p2JeVM/jf4APVPUH29rLVdVc4CngXPtC\nB8NeqTiPP9i51E8BH2KmsB7ADPSWz41/FWPS2GzztNAr7D+B7tZsUF7uNwHnYQb/riDA76GqSzCK\n4hmMosvGDIT74lSMnfosKub3tOCyWin3Y/K6AfgMMx5QLmMRcLGVazdmRs271cxDIE4BvhGRPMzv\ncpOq/mTd7gNeseU9FjPYnowxJy4EPvWK6ynMeM0eEXnajgOcBVyKmYmTy5GBdn9kActsb6cbvlvy\n92Jmhf2MGVB+XFW9ZYkJpKI50OFwOBz1HdfidzgcjhjDKX6Hw+GIMZzidzgcjhjDKX6Hw+GIMaq6\n+FStkJGRoR06dAhL3Pn5+aSkpIQl7lDhZAwNdUFGqBtyOhlDQzhlXLp06U5VbRGU55qs94D5aGIN\nZlrYnT7cr8As+PQDZkGonsHEm5WVpeFi1qxZYYs7VDgZQ0NdkFG1bsjpZAwN4ZQRWKLhXqvHfm79\nLDAS6A5cJiLdvbxtAIao6knA/2G+qHM4opqyEtg4Gxb/A/b9EmlpHI7QUxNTTz8gW+1HGyIyBfPZ\n/KpyD6o638N/+cqKDkdUsjsb5j4Iaz+CQ7vNvRk3QMdfQe//gRMvhQqLITgcdZRqf8AlZlOPEap6\nrb0eh1mL4wY//m8DupX79+E+EbN6IJmZmVlTpkypllyByMvLIzU1uvfDcDKGhqrImL8hhe9u7UlZ\nYRwZg3bSfPBOGrU/yI7ZLdj2WUsKtiTT5qIcOt+YHXLlX9/KMlLEuozDhg1bqqp9g/IcrE3I+8As\nD/CSx/U47CYKPvwOw6ym2DyYuJ2Nf1akRQhIfZJxyzLVx5qrPtFadcfqo93LylQ/vUX1Psz/srLI\nyBlJnIyhIVps/DUx9Wym4iqBbe29CojIyZiFwEZqxfVgHI6Is2Up/OcMaJgOV30JzTof7UcEzvqr\nsf0v/BvEJ8LwR5zZx1F3qYniXwx0FZGOGIV/KWZ53MOI2bjjXWCcqq6tQVoOR8gpPgjvXAYN0mDC\nXGjS3r9fERjxJJQVw9ePQcYJ0Gt87cnqcISSas/qUbOr1A3ATIwZ5y1VXWmXAZ5kvf0ZszPUP0Rk\nuYj4WirV4YgIs/4Mu9fBhS9XrvTLEYGzn4F2g2DmzZC3LewiOhxhoUYfcKnqDMwywJ73nvc4vxbw\nOZjrcESSnIWw8O+QdZ2ZtRMsEgfnvQgv9IKZf4BRVd7CxOGIPG7JBkfMUVIAH1wDaW3gzKpsh2Np\ncQKcdg+smAJrp4dePocj3DjF74g5vnoEdq42LfeG6YH9+2LwHdCiB3x8PRTuD618Dke4cYrfEVPk\nbYMFf4Uel0CXX1c/nvgGcP5LsD8Hvv5L6ORzOGoDp/gdMcVXD0NJIQx7oOZxtR0A3cfAN0/CwZ01\nj8/hqC2c4nfEDPt+gaXPQ6+roflxoYlz6P1mWujX1RgrcDgihVP8jphhjm3lD/lz6OJscQKcdAUs\negYObA1dvA5HOHGK3xET7FoLy1+GvtdD43YBvVeJIX+G0iKY90ho43U4woVT/I6YYM79kJAEg+8K\nfdzNuhjz0dIX3DLOjrqBU/yOes+en8yc+77XQ2pmeNIY8idQdTN8HHUDp/gd9Z75T0BcAgy8OXxp\nND4WTr4Svv0n5O8IXzoORyhwit9Rr8nbBsv/BSdfBWmtw5vWqX+EkkNmoNfhiGac4nfUa7552szb\nH/TH8KfV4gQ4/gJY/AwU5YU/PYejujjF76i3lOTHs/hZ6D4qdPP2AzHoDrNt47KXaic9h6M6OMXv\nqLdsnd6awn1GGdcW7QbCsafBgr9BaXHtpetwVAWn+B31ktIiyJnWlo7DoXVwu5CGjEF3wP5NsMIt\n2eyIUpzid9RLfngDinY2ZNDttZ9217PhmBNh/l/MFE+HI9pwit9R79Ayo3RTOufR6czaT18EBt4G\n21dA9qe1n77DEQin+B31jnWfwI5V0O6STRHbEP2ky8xGL/PdB12OKMQpfke9Y/5fIL0dtBi2PWIy\nxDeAAX+AjbNgi9tp2hFlOMXvqFdsXgQ/zzFKNy4hsgb2rIlmhy/X6ndEGzVS/CIyQkTWiEi2iNzp\nw72biCwQkUIRua0maTkcwTD/L9CwMfT5TaQlMUo/axKsmmbWC3I4ooVqK34RiQeeBUYC3YHLRKS7\nl7fdwO+Bv1ZbQocjSHb+CKveMYuxNUyLtDSGATeBxJv1ghyOaKEmLf5+QLaq/qSqRcAU4AJPD6q6\nXVUXA+5TFkfYmfeoWXo5nIuxVZW01tBrglm8zW3U4ogWRKs50VhERgMjVPVaez0O6K+qN/jwex+Q\np6p+W/4iMhGYCJCZmZk1ZcqUaskViLy8PFJTU8MSd6hwMladgtwkvrmiP20u3EyXG7OB6JHx0OYk\nFl3Vn7ajc+h8/fqj3KNFzspwMoaGcMo4bNiwpaoa3OeKqlqtAxgNvORxPQ54xo/f+4Dbgo07KytL\nw8WsWbPCFneocDJWnem/VX0gUXXfpiP3oknGd69UfaiRav6Oo92iSU5/OBlDQzhlBJZokDq2Jqae\nzYDnJnZt7T2Ho1Y5sNWYUnpNgPS2kZbGN4PvguJDsPDJSEvicNTMxr8Y6CoiHUWkAXAp8GFoxHI4\ngmfB36CsuHYXY6sqLbqbVUIX/T8o2BtpaRyxTrUVv6qWADcAM4HVwFuqulJEJonIJAARaSkiOcAt\nwD0ikiMi6aEQ3OEAyMuFJf+AEy+DZp0jLU3lnHY3FO43ewQ4HJEkoSaBVXUGMMPr3vMe57kYE5DD\nERbmPGBW4hx6X6QlCUzLXmajlvl/NVNOU1pEWiJHrOK+3HXUWXZnw7IXoc9EaNYl0tIEx/CHoTgf\n5j4YaUkcsYxT/I46y5f3mDVxhvwp0pIET4vu0Pt/YMlz7mteR+Rwit9RJ9myFFZOhYG3QmrLSEtT\nNYbeB3EJ8OXdkZbEEas4xe+oc6jC53dAoww4tQ6uAJXWGgbeAiumuJU7HZHBKX5HnWPV27DhCzj9\nz2YhtLrIoNtNxfXpH8zGMQ5HbeIUv6NOcWgPfPJ7s4/uKb+NtDTVp2E6nPE4bPoatn7UOtLiOGIM\np/gddYrP74CDO+HcyRAXH2lpakavCdBxOPw0uRP7cyItjSOWcIrfUWf4ea6ZvjnwFmjVO9LS1BwR\nOG8yaKnw8W/dxuyO2sMpfkedoPggfDQRmnSEIfdGWprQ0bQTdLhmA2s/gpVvRVoaR6zgFL8j6lE1\nSn/XWtNCbpASaYlCS9tRm2l9Cnw8CXYfvWqzwxFynOJ3RD2LnoEfXodhD0CnMyItTeiReGX0FJA4\nmHohFOVFWiJHfccpfkdU88s8+OwWOO48OO1/Iy1N+GjaCUZPhR2r4IOrnb3fEV6c4ndELbvXw9tj\noEkHuOhV0yKuz3Q6A854zGzO/tXDkZbGUZ+p0eqcDke42LUOXhlmVt685D1IahJpiWqHgbdC7nKY\ndY9Z1mFwFO8x4Ki7OMXviDp2/giv/MpsrjJ+FhxzYqQlqj1E4IJ/m695v7jTrOQ59H5z3+EIFU7x\nO6KKn78y5h2A8bPhmB4RFScixCfCRf+BxEYw9/+gYB+c9Vdz3+EIBfXcauqoK5SVwKx74ZWh0CAV\nJsyOTaVfTly8mbo64GZY9DT8c6DpCTkcocApfkfE2boMXh4Ccx+AnlfBdd9CRrdISxV5JA5+/TcY\n+w7s3Qgv9DabtZcWRVoyR13HKX5HxMhdDlMuhMlZsGM1jHrT2LcbpkVasujihIvhtyugwzCYeTM8\n3QUW/wNKCiItmaOuUiPFLyIjRGSNiGSLyJ0+3EVEnrbu34tIn5qk56j7HNgCC5+Cf55qWrAbZ5vB\ny5s2wImXRlq66CW1JVz+MVzxCTRuBzN+B0+2hxk3mDIsK420hI66RLUHd0UkHngWOBPIARaLyIeq\nusrD20igqz36A8/Z/456jioc3GG2F9y+EnIWQs4C84ESCpk9Yfij0Pe62JmqWVNEoMsI6Pxr2DgL\nFj8L3/7L/E9uDm0HQJt+0PoUaN4VGh9rtqZ0OLypyayefkC2qv4EICJTgAsAT8V/AfCqqiqwUESa\niEgrVd1ag3T9svJt0AAtn+2rjmFFbjhSDx3hlrHCV6HqdV99/C8zLcqyElO+pcXwy+pjmT0HSg5B\nUT4U50HBXsjfYRT+gS0Vlx5IamoU04mXQffRkHF8+PJX3xGBjr8yR1E+ZH8C6z6GzYtg3QwO/6YS\nZ3b7SsmERs1N5dAwHRJTzHpHCUmmYohvaAaTJd7+jztyICa9bauP4Yct3oJUlCnS1Id3OyEJul0Y\nfjlEq/ltuIiMBkao6rX2ehzQX1Vv8PAzHXhUVefZ6y+AO1T1qA3nRGQiMBEgMzMza8qUKVWW6auR\np1FWUMcXaa9jSEIZ8cml5mhUSmKTIho0LqZB8yKSWh8iuXUBye0OktzmUK0rh7y8PFJTU2s30WoQ\nSjlL8uLJW59KwdZkCrYmUZCbRPG+RHPsT6T0UDylh+LdexKlJDYt4tR351cr7LBhw5aqat9g/EbN\nPH5VnQxMBujbt68OHTq0ynGcuJwKLVhfLFq0iH79+lVdwFqkVmT011qzLbzy/+Wtvrh48yWpxJtW\n4tcL5zLsjNOJi4/DDBWVTzIPzdKZ27ZtY8yYMXz77bdMnDiRJ554ospxzJ49m+o8R7XNhAkTKCkp\n4bXXXvPp3qNHD5599lmfeZk9ezZXXnklOTm+d3KZMGECbU9py4MPPljhvqr5QK60CEoKTU+urMT0\n7LQMUDtuYHt+3yz8hv79+1cIf+SiihkOE/Xh3Zb4BjTvOjT8gqhqtQ5gIDDT4/ou4C4vPy8Al3lc\nrwFaBYo7KytLw8WsWbPCFrc/Xn/9dc3KytKUlBRt2bKljhgxQr/66itVVb333nv1iiuuOEpGQNet\nW3f43sqVK/W8887T9PR0TU1N1aFDh+rXX39dIVxhYaHee++92qVLF23UqJG2b99er776at2wYcNh\nPx999JGecsop2qhRI23WrJlefvnlumnTpsPu//73v3XQoEGV5mf8+PEaFxenW7ZsOXzvuuuu05SU\nFE1JSdHExERNSEg4fD1ixAjdsGGDAofvlR9TpkzxmcYDDzygF110kZaVlVVeuJVQ/luPHz9e7777\n7mrHU1W2bduml156qbZq1UrT09P11FNP1YULF/r1P378+KOegWCZNWuWtmnTptK4Q5H3qrw33s9u\nIIYMGaIvvvhiNaSqyKxZs3TMmDE6c+ZMLSgo0MzMzAruU6dO1YEDB2pycrIOGTKkxulVV8ZwASzR\nIPV3TUw9CcBaYDiwGVgMXK6qKz38nAPcAJyNGdR9WlUDVskisgP4uVqCBSYD2BmmuH2RCbTE5Gc/\npn2UDqRhBsVbAw2BDV4ytgdWAIXW/QRgB5Br48gA2mB+g3wbrgum6f0zcBCIB5pZ/zuBpkAH677H\nure1sqwCSoHmNu41fvITB/S051uAbT78+MpTA+AkYKmfeL1pDxTbNKpL+W/dASiqYVxVoQGmrHdj\n8lD+W/0A+NpavQum7Df4cAtEGtAR+N6PewdCk/eqvDdZHHl2g+F4YFcV4vdHBuZdW415/tpi3o9y\n0jBWjiTMO+jvGQ8n4dQ/7VW1RVA+g60hfB0Yhb4WWA/cbe9NAibZc8HM/FmPeej71iS9UBxUoVYM\nQVqNgTxgTCV+7gNe85YRo6y72Ov/ADN8hH0OmGvPzwAOAe38pCMYhX+71/04zEv6gL2eAMyrRN6r\ngE3AL8CKKuSpg81TQhDl9jJGYRbZ8jvDynmnfZZ2AW8BzTzCvI2pFPcBc4EethwnesX1kfV/uHw9\n0nzQng/FVMp32Dj/Y++fCywH9gLzgZOr8CzsB7L8uG2x+XkVOACs9HxXgI3AGfY82cq6B1NZ/xHI\n8fDbG1hm45kKTCnPV6A82HRuw1Qi+2z4JF/vDaaymmP97QSm2vtzbdnm2/K+BFMJTsc0XPbY87bW\n/0OYSq/A+n/G3u8G/BdTea4BxgZRxt8CP9nz64HH/fi7FphdW3rA+92ORLpHyRFpAepzwQMjgBIq\nUXYEp/hzgat9hB1mX5pk4FFgTiXpdLNxdvThdj+wwJ5PoHLF/wXwuFUeJb6UmZ88dSBIxW/9v+yl\nsG4CFmJacQ0xZsQ3PdyvwbToGgJPWvmW+IrL3guk+EuAx2x8yRiFuh3Tc40HxmMUZcMg8tLLKrbG\nfty3WPezbdyPAAs93DdyRPE/CnyF6cm1w1TaOdatAaZyvxnT8xuNqfTK81VpHuz5IkyPrRmm5Vze\niPNW/G8Cd2Mq5CRgcCVl2xwYBTSyv9HbwPse7rOBaz2uUzCNi6sxLfTemMqlu5/yG46pyEoxFfxe\n+z/fng/x8h/zit99uRtemgM7VbUkgL+xIrK3/MAoCk8yAF9TYLdiXrxmNq3KpslmeITxFU+Gj/sV\nEJFjMZXNGxjF+AWmB1AVdnrmVUROCDLcJEyvMkdVCzGVy2hrckRV/6WqBzzcemKUW3UpA+5V1UJV\nPYTpObygqt+oaqmqvoIxZQyoLBIRScf02O5X1X2VeJ2nqjNUtdT67+nH31jgIVXdraqbgKc93AZg\nFP6TqlqsqtMwJthygsnD06q6RVV3Ax9x9LNYTjHGHNdaVQvUztzzharuUtV3VPWgqh7AtPKH+POP\n6ZVsVNV/q2qJqn4LvAOM8RP/F6raBKPkx2DMahuBDFVtoqpzKkkrJolFxT+5FtPaBWSUK6dKeMs+\noE3sA/xbL/edQCsf4VphFNQem5YvP55x4MdPK4KzO44DVqvqckw5vg5cLiJVWTcywzOvqro6yHDt\ngfc8KsfVmBZepojEi8ijIrJeRPZjXnqAaVWQy5sdquq5KEJ74FavCrodpnXsExFJxijPhar6SCVp\nLcX06so5CCT5eW5aY1rD5fzs5bZZbdPSh3swefCWo3yeqfd7czvGfLhIRFaKyDW+swYi0khEXhCR\nn+3vMxdoYj8C9UV7oL+XnFdg7Pe+4s+xfpoCr2B6Ne2BrSLyN39yRYja1D9+iTnFr2baaG2xANOi\nqtInGT5k/BzfrZ2xGBPNQeunn4i09RPtGozdukI8IhKH6YZ/EYRoVwGdRCQXeAD4G6ancHYQYWvK\nJmCkV6WRpKqbgcsxHwuegRlX6WDDTLX/fc1gOIgxPZTjrVS8w2zCtLQ902+kqm/6ElZEGgLvY8r8\nugB5C3bAG0zvrJ3H9bFebm1EKkzQ9XSvUh488X4mVTVXVX+jqq0x+fuHiHTxE/xWzABuf1VNB063\n98vl9FXWc7zkTFXV6/3I1hZjVv2vbThNBn5nw90SKG+1SS3rH7/EnOKvTWzX/s/AsyJyoW35JIrI\nSBF5vApR3Q+cKiIPiUgzEUkTkRsxivgOm9bnmMGw90QkS0QSrL9JInKNbQXeBtwjIpeLSJKItARe\nwsxw+LtHemLdPY+BQGfMF9u97HEixuxTVXNPdXgeeEhE2lsBW4jIBdYtDVPB7sIoc++NC7cBnbzu\nLcf0VuJFZASVmx4AXgQmiUh/uwZVioicIyJHLSlne0DTMIPt41XV10ye6vIWcJeINLWV/I0ebgsw\nJrjf2+fsYszvVeU8BEJExng0MvZglHd5Pr3LOw1TFntFpBlwr1d03v6nA8eJyDibj0QROSWAWTAL\nM6gN0AczTuYtc7yIJGHGDeLscx2Tuxw4xR9mVPUJ4BbgHsyshk2YKa7vVyGOdcBgjN13I6ZlNwr4\ntap+7eF1NDAD09Ldhxn464vpDaCqUzHmmpsxSnIVZuBykKru8ojnVMyL6nn8D/CBqv5gW3u5qpoL\nPAWca1/oYNgrInkeR7AtsqeAD4HPROQAZqC3/IuiVzEmjc02Twu9wv4T6G7NBuXlfhNwHsYufAUB\nfg81X5v/BngGo+iyMQPhvjgVY6c+i4r5PS24rFbK/Zi8bgA+w4wHlMtYBFxs5dqNmVHzbjXzEIhT\ngG9EJA/zu9ykdvkWzBjLK7a8x2IG25Mx5sSFwKdecT2FGa/ZIyJP23GAs4BLMQPfuRwZaPdHFrDM\n9na6YWZGeTMO8yw/B5xmz1+sUq7rCdWex+9wOByOuolr8TscDkeM4RS/w+FwxBhO8TscDkeM4RS/\nw+FwxBjy0FqBAAAgAElEQVRRsyyzJxkZGdqhQ4ewxJ2fn09KSmiWDg4XTsbQUBdkhLohp5MxNIRT\nxqVLl+7U2likLVxHfVuWuao4GUNDXZBRtW7I6WQMDdGyLHNUtvgdDkfVUJRtHGA9O8hhDyWUofaD\n2FY0pgPNaUdTGrhX3kGUmnocDkdwFFPKN2xgMRs5YJe/b04KDUkkDiiljAX8xNesJw6hJ205jS40\nqbBahSPWcIrf4aiDlFHGt2xiDuvIo5DOtGAYrehEC9JJquC3iBJ+YQ9ryeVbcviOHHrTjqEcR0ql\nH8M66itO8TscdYwDFPAO3/ILu2lLU0bRh/b4XzGjAQl0oQVdaMFgujCPbJaxiTVs4yJ60THwityO\neoZT/A5HHWIDO3mX5RRRwgX05GTaIEjggJZ0kjmbk8iiPdNYxn/4htPpyul0Ja4K8TjqNm4ev8NR\nR1jMRl7jG5JJ4FoG0ZO2VVL6nmSSzm8YzMm0YS7reIullFAaYokd0YpT/A5HHWA+6/mElXQlk2sZ\nTAuqvJLyUTQggQvpxQi6s5ZtvMkSigi0WZyjPuAUv8MRxSjKXNbxOT/Sg9aMoU/Ip2T2oyPnczIb\n2clrLKKA4pDG74g+nOJ3OKKYuWQzm7X0pC0X0Yv4ML2yvWjHKPqwhb28wSKKndmnXuMUv8MRpSzj\nF+ZYpX8+J4d98LU7rRhFb3LYyzt8Sxmh3DjMEU0EpfhFZISIrBGRbBG504d7NxFZICKFInJbVcI6\nHI6jWcd2PmYFnWnBuZxU7UHcqnICrRhBD9ayjRmsPPz1r6N+EdBYKCLxwLPAmZiNoxeLyIequsrD\n227g93htKh5kWIfD4cEW9jKNZbQkjTH0CZt5xx/96EAeBcxjPekkcTpdazV9R/gJ5onqB2Sr6k9q\n9vScAlzg6UFVt6vqYjhqVChgWIfDcYQ8CpjKElJowGWcErG1dYZxPCfThtmsZQ25EZHBET6Ceara\nYDYILyeHI5tchyysiEwEJgJkZmYye/bsIJOoGnl5eWGLO1Q4GUNDXZARjshZJsr3vQrITy2j97Jk\nluR77xlfu6THKWm94pjWaCnHoVFflnXh944WGaPmy11VnQxMBujbt68OHTo0LOnMnj2bcMUdKpyM\noaEuyAhH5JzBD+znF0bRmx6ntI60WAD05RAv8jUb+xVxXqNBJJEYaZH8Uhd+72iRMRhTz2agncd1\nW3svGGoS1uGIGb5lE0v4hVPpRA+iQ+mDWeJhDH0oSFLeY7kb7K0nBKP4FwNdRaSjiDQALgU+DDL+\nmoR1OGKCvNRSZrCCjmTwK7pFWpyjOJZmdM5uwDq2M5+fIi2OIwQENPWoaomI3ADMBOKBf6nqShGZ\nZN2fF5GWwBIgHSgTkT8A3VV1v6+w4cqMw1HXKKSY1d0LaUQDLqZX1C6U1mpLAg2Oa86XrKEtTStd\nDdQR/QRl41fVGcAMr3vPe5znYsw4QYV1OBxmOYbp/MChZGU8faJ6bXxBOI+TyGU/77KMiZwW1fI6\nKsd9uetwRIil/MJKttJhQ2KdaEE3JJHR9OEgxc7eX8dxit/hiAC57Gcmq+hMC9r9Er0zZbxpSToj\n6MFP7HT2/jqMU/wORy1TRAnvsIxkErmQnrW2HEOo6EM7utOKL1nDJvZEWhxHNXCK3+GoZT5lJbvI\n5yJ61Uk7uSCcy0k0Jol3+dYt41wHcYrf4ahFVrCZ5eRwGl3q9F63SSRyMb05QAEf8b2z99cxnOJ3\nOGqJ3eQznRW0oylD6sHCZ21pyjCOZzW5LOOXSIvjqAJO8TsctUApZbzDt8QhXExv4urJq3cqnehE\nBjNZxTb2R1ocR5DUj6fP4YhyPudHtrKP8zmZxiRHWpyQIQgX0oskEnmHb92evXUEp/gdjjCzlm18\nwwZOoT3daBlpcUJOKg25kF7sJI+ZuK026gJO8TscYWQfh/iA72hJOmdyQqTFCRudyGAwnfmWTXxP\nTqTFcQTAKX6HI0wYu/4ySlFG0ZsE4iMtUlgZynEcSzM+ZgU7OBBpcRyV4BS/wxEmvuBHctjLeZxE\nc1IjLU7YiSOOUfQmkXimsczZ+6MYp/gdjjDwI7kstHb9aFpfP9ykkcTF9GYHecxghZvfH6U4xe9w\nhJhd5PEB39GaxvXaru+PTmRwOl35ns0sdfP7o5KgFL+IjBCRNSKSLSJ3+nAXEXnaun8vIn083DaK\nyA8islxEloRSeIcj2iikmKksJQ5hNH3qvV3fH6fTlS604FNW8gu7Iy2Ow4uAil9E4oFngZFAd+Ay\nEenu5W0k0NUeE4HnvNyHqWovVe1bc5EdjuhEUd7nO3aRz2j60IRGkRYpYpR/qNaEZKaxjAMURFok\nhwfBtPj7Admq+pOqFgFTgAu8/FwAvKqGhUATEWkVYlkdjqjmK7JZwzbOpFudXocnVCSRyFj6UkgJ\nb7GUEkojLZLDIqqVD76IyGhghKpea6/HAf1V9QYPP9OBR1V1nr3+ArhDVZeIyAZgH1AKvKCqk/2k\nMxHTWyAzMzNrypQpNc6cL/Ly8khNje4ZFk7G0FCbMu7IKGF1j0KO2R7P8asbVmmp5fpeljtamLJp\nsS2eblUsm6pQ38sxEMOGDVsarFUlqK0Xa8hgVd0sIscA/xWRH1V1rrcnWyFMBujbt68OHTo0LMLM\nnj2bcMUdKpyMoaG2ZMxhD/NZSFuaMC5zAImZVbPrx0JZfkU2szLXcEJmG4ZyXOgE8yAWyjFUBGPq\n2Qy087hua+8F5UdVy/9vB97DmI4cjnrBbvKZwhLSSOIS+pIYo4O5gRhMZ3rRlrms4zv3ZW/ECUbx\nLwa6ikhHEWkAXAp86OXnQ+AqO7tnALBPVbeKSIqIpAGISApwFrAihPI7HBHjIEW8yWIU5XJOqZOb\nqtQWgnAOJ9GB5nzE96xnR6RFimkCKn5VLQFuAGYCq4G3VHWliEwSkUnW2wzgJyAbeBH4rb2fCcwT\nke+ARcDHqvppiPPgcNQ6BRTzGt+wj0NcQt+Y+DK3psQTx1iyaEEaU1nipnlGkKBs/Ko6A6PcPe89\n73GuwO98hPsJ6FlDGR2OqKKIEt5gEds5wKX05ViaRVqkOkMSiVxBP15hAW+ymKsYQCsaR1qsmMN9\nuetwVIFiSpnCEjazj9H0oQvHRFqkOkcqDbmS/iSRyOsschu4RACn+B2OICmgmNdZxEZ2cQEn18u1\n9WuLxiQzjv4kEMcrLCSHPZEWKaZwit/hCIJ8CnnVKqhR9OZk2kZapDpPM1KYwECSSeQ/fMMGdkZa\npJjBKX6HIwB7OMjLLGAneVxK35habTPcNKERExhIE5J5g8WsYmukRYoJnOJ3OCphAzt5iXnkU8iV\n9Hc2/TCQRhLjGUgrGjONZcxmrVvOOcw4xe9w+EBRFrGB11hEKg25lsFu9k4YaUQDrqL/4Y+83mYp\nhW4jl7BRG0s2OBx1ioMUMYMVrGIrx5PJhfSioXtVwk4C8ZzHyRxDGv9lNZP5iovoRVuaRlq0eod7\nmh0OD7LZzod8z0GKGE43TqVT2BYVcxyNIAygE61ozPt8x79ZwOl05TQ6E+cMFCHDKX6HAzhAAV+y\nhu/IoQWpXM4ptHQfFkWM9jTnOk7jE1Ywh7X8SC4j6eHMbSHCKX5HTFNCKYvYyFzWUYoyiM4MoWvM\n7pwVTSSRyEX0phstmckqXmYBJ9GG4RxPOsmRFq9O4xS/IyYpppRl/MICfmI/BRzHMZxFd5qREmnR\nHF6cQCs604J5rGcBP7GKrfSmLYPoQmNXAVQLp/gdMcU+DrGcTSzhZ/Ip4liacT496eR2zIpqGpDA\nrziePrTja9azjE0sYxMn0po+HEs7NwBcJZzid9R7Cigmm+18z2ay7XLAnWnBYLrQ3tmM6xRNaMQ5\nnMRgujCf9XzHZr5nMy1IJbVdEbvJd722IHCK31HvUJRtHGBzG7O2zgZ2UoaSRkNOowu9aEfTGN4I\nvT7QmGRGciLD6cZKtrKMX9jQOY9nmE0maXQlkw40px1N3eY4PghK8YvICOApIB54SVUf9XIX6342\ncBCYoKrLggnrcNSEUsrYy0G2cYBc9pHLfnLYQwEl0BWacZD+dOR4MmlLU+Lc1Mx6RQMS6E07etOO\nTxd+SZMBHfiRbXzNeuaRTRxCKxrbI51M0mlOCg1JjLToESWg4heReOBZ4EwgB1gsIh+q6ioPbyOB\nrvboDzwH9A8yrMNRAUUpopRCiimghIMUcYgi8iniAAUcoID9FLCHg+zl0OHP+wWhBamcQCva04wt\nC9YyYuDQyGbGUWskFcQxgE4MoBOFlLCJ3WxkFzns5Xs2s4SfD/tNoSHNaEQ6yaSTRDpJNKLB4SOJ\nRJJIoCEJ9fL7gWBa/P2AbLupCiIyBbgA8FTeFwCv2g1ZFopIExFpBXQIImzI+IQVlFJWqZ8txxWS\nx/fhSD5kREpGf6uj+Fo3Zevxhezjuwruas/Lr8rsuR4+N/9LKTv8v5QySuz/Ikoptoc/BLOeexpJ\ntKYxPWhNc1JoQSrHkFZhGubuwuzqFIOjHtCQBLpwzOG1lRRlNwfZzn52c5Dd5LObfLawlx8pqFRv\nxBNHIvE0IJ4E4kggnnjiiEeI8/gfhyAIcWDPyv84/D+325H3BqjQ/xSEhiRwFt3DUiaeBKP42wCb\nPK5zMK36QH7aBBkWABGZCEwEyMzMZPbs2UGIVpEf+h+kLEDlrM3K2F0Y3Zs9R6WMHrpfgLImZewt\n2FzBXTz/A6Ll13Lk3B5xClImxCnElUF8mZBcav7HlSaSUCLEl0JCiZBQLCQWC4nF0KBYEBWgBNgP\n7GcPsAdY6yVyXl5etZ6j2qYuyFnfZEyzBwhKEiUJUNxAKU40R0mCUhrP4f9l8UppXAllcUpZHJTE\nQZkoGgcqXod9CdS+CGqSMY2g9DL2eb43Hu4AicVCgyXba14YgVDVSg9gNMY2X349DnjGy890YLDH\n9RdA32DC+jqysrI0XMyaNStscYcKJ6Nqbm6unnbaaZqamqq33HJLteKoC+Woqjp+/Hi94oor/Lp3\n797db15mzZqlbdq0qTTuu+++u6Yi1omyjHUZgSUaQLeWH8EYrzYD7Tyu29p7wfgJJmy954033qBv\n376kpqbSqlUrRo4cybx58wC47777uPLKK48KIyJkZx8xVaxatYrzzz+fxo0bk5aWxrBhw5g/f36F\nMEVFRdx333107dqVlJQUOnTowDXXXMPGjRsP+5k+fTr9+vUjJSWF5s2bc8UVV5CTc6R38fLLLzN4\n8OBK8zNhwgSGDx/O1q1H1k6fNGkSqamppKam0qBBAxITEw9fjxw5ko0bNyIih++VH1OnTvWZxuTJ\nk8nIyGD//v088cQTlcoTiAkTJnDPPffUKI6qsH37di677DJat25N48aNGTRoEN98802141u5ciVD\nhw4NnYBhxvvZDcTQoUN56aWXQpL22LFj+eyzzygsLKRly4o7pN1222107dqVtLQ0unXrxquvvhqS\nNOsiYiqKSjyIJGB60cMxSnsxcLmqrvTwcw5wA2ZWT3/gaVXtF0xYP2nuAI+RmNCSAbW61U8m0BKT\nn/2Ynl06pqeZA7QGGgIbvGRsD6wACq37CcAOINfGkYExpa0F8m24LkCiTesgZiZVM+t/J9AUM+7y\nM8Y6Eo+pjNMw4y6lQHMb9xo/+YkDetrzLcA2H3585akBcBKw1E+83rQHim0a1aX8t+4AFNUwrqrQ\nAFPWuzF5KP+tfgCfxuQumLLf4MMtEGlAR/A7KNSB0OS9Ku9NFkee3WA4HthVhfj9kYF511Zjnr+2\nVLQAtsb8JgVACmYyyjqOvD+1QTj1T3tVbRGUz2C6BRiFvhZYD9xt700CJtlzwczeWY95uPtWFjaS\nB1XoDoUgrcZAHjCmEj/3Aa95y4hR1l3s9X+AGT7CPgfMtednAIeAdn7SEYzCv93rfhzmJX3AXk8A\n5lUi71WYcZtfgBVVyFMHm6eEIMrtZYzCLLLld4aV8077HO0C3gKaeYR5G1Mp7gPmAj1sOU70iusj\n6/9w+Xqk+aA9H4qplO+wcf7H3j8XWA7sBeYDJ1fhWdgPZPlx22Lz8ypwAFjp9Q5tBM6w58lW1j2Y\nyvqPQI6H397AMhvPVGBKeb4C5cGmcxumEtlnwyf5em8wldUc628nMNXen2vLNt+W9yWYSnA6puGy\nx563tf4fwlR6Bdb/M/Z+N+C/GEW9BhgbRBl/C/xkz68HHg/g/0Pg1trSB77KMVJHxAWo9QzXruIf\ngRmF9KvsCE7x5wJX+wg7zL40ycCjwJxK0ulm4+zow+1+YIE9n0Dliv8L4HGrPEp8KTM/eepAkIrf\n+n/ZS2HdBCzEtOIaAi8Ab3q4X4Np/TYEnrTyLfEVl70XSPGXAI/Z+JIxCnU7pkcbD4zHKMqGQeSl\nl1Vsjf24b7HuZ9u4HwEWerhv5IjifxT4CtOTa4eptHOsWwNM5X4zpuc3GlPpleer0jzY80WYlnEz\nTMu5vHHnrfjfBO7GVMhJVBzj8y7b5sAooJH9jd4G3vdwnw1c63GdgmlcXI2ZgNIbU7l091N+wzEV\nWSmmgt9r/+fb8yE+wiQDW4ERtaUPfJVjpI76N0E1umgO7FTVQFsJjRWRveUHRlF4kgE+NyPdinnx\nmtm0KtuwtHwxGn/xBFysRkSOxVQ2b2AU4xeYHkBV2OmZVxE5IchwkzA9xhxVLcRULqOtORFV/Zeq\nHvBw6wk1+mSzDLhXVQtV9RCm5/CCqn6jqqWq+grGlDGgskhEJB3TY7tfVfdV4nWeqs5Q1VLrv6cf\nf2OBh1R1t6puAp72cBuAUfhPqmqxqk7DmFfLCSYPT6vqFlXdDXzE0c9iOcUYc1xrVS1Q1Xn+Mqaq\nu1T1HVU9qKoHMK38If78Y3olG1X136paoqrfAu8AY/zE/4WqNsEo+TEYs9pGIENVm6jqHB/Bnge+\nA2ZWIke9JRYV/+RaTGsXkFGunCrhLfuANrEP8G+93HcCrXyEa4VRUHtsWr78eMaBHz+tCM7uOA5Y\nrarLMeX4OnC5iFTlM8gMz7yq6uogw7UH3vOoHFdjWniZIhIvIo+KyHoR2Y956QGmVUEub3aoaoFX\n+rd6VdDtwP/O6yKSjFGeC1X1kUrSWorp1ZVzEEjy89y0puIU6Z+93DarbVr6cA8mD95ypNpz7/fm\ndoz5cJGIrBSRa3xnDUSkkYi8ICI/299nLtDEfuDpi/aYD0A95bwCY7/3FX+O9dMUeAXTq2kPbBWR\nv/nw/xfgRIz5qPJBztBTm/rHLzGn+FW1Ngt+AaZFdWFVAvmQ8XN8t3bGYkw0B62ffiLS1k+0azB2\n6wrxiEgcphv+RRCiXQV0EpFc4AHgb5iewtlBhK0pm4CRXpVGkqpuBi7HfBh4BmZcpYMNUz5lyNfL\nfRAqLNjjrVS8w2zCtLQ902+kqm/6ElZEGgLvY8r8ugB5C3bAG0zvzHOm3LFebm3sEiq+3KuUB0+8\nn0lVzVXV36hqa0z+/iEiXfwEvxUzgNtfVdOB0+39cjl9lfUcLzlTVfV6P7K1xZhV/2sbTpOB39lw\nt3j6FZH7MSsNnKWq+wPlO9TUsv7xS8wp/trEdu3/DDwrIhfalk+iiIwUkcerENX9wKki8pCINBOR\nNBG5EaOI77BpfY4ZDHtPRLJEJMH6myQi19iWzW3APSJyuYgkiUhL4CXMLKO/e6Qn1t3zGAh0xnzJ\n3cseJ2LMPlU191SH54GHRKS9FbCFiFxg3dIwFewujDJ/2CvsNqCT173lmN5KvF1PqjLTA8CLwCQR\n6S+GFBE5R0TSvD3aHtA0zGD7eFWt/HPyqvEWcJeINLWV/I0ebgswJrjf2+fsYszvVeU8BEJExng0\nMvZglHd5Pr3LOw1TFntFpBlwr1d03v6nA8eJyDibj0QROSWAWTALM6gN0AczTuYt812YRsIZqror\nYCbrMU7xhxlVfQK4BbgHM6thE2bq6/tViGMdMBhj992IadmNAn6tql97eB0NzMC0dPdhBv76YnoD\nqOpUjLnmZoySXIUZ5Brk9SKcinlRPY//AT5Q1R9say9XVXMxC/Cda1/oYNgrInkexy2Bg4BN50Pg\nMxE5gBnoLf8K/FWMSWOzzdNCr7D/BLpbs0F5ud8EnIexC19BgN9DVZcAvwGewSi6bMxAuC9Oxdip\nz6Jifk8LLquVcj8mrxuAzzDjAeUyFgEXW7l2Y2bUvFvNPATiFOAbEcnD/C43qV2aBTPG8oot77GY\nwfZkjDlxIfCpV1xPYcZr9ojI03Yc4CzgUszAdy5HBtr9kQUss72dbpiZUd48jOkBZXv8Jv9b1YzX\nBwLO43c4HA5H/cK1+B0OhyPGcIrf4XA4Ygyn+B0OhyPGcIrf4XA4Yoyo3HM3IyNDO3ToEJa48/Pz\nSUmJ7s2YnYyhoS7ICHVDTidjaAinjEuXLt2poVykrbYPtx7/rEiLEBAnY+ioC3I6GUNDtKzHH5Ut\nfkc9paAAZs2CBQtg4UL44Qc4eNDcV4XOnaF7d+jRA847D/r2hQofoTocjlDgFL8j/KxZA5Mnw8sv\nw+7dEBcHJ58MI0dCejokJRnFv24drFwJH3wA//d/0KULXH45TJoErSpbhsjhcFSFoBS//aT9Kcxq\nhy+p6qNe7t2Af2M+lb5bVf8abFhHPWbjRvjjH2HaNEhIgIsugmuugcGDITXVf7i9e+Hdd+GNN+DB\nB+Evf4GbboLbb4emTWtNfIejvhJwVo9dQe9ZzMJG3YHLRMR7G/jdwO+Bv1YjrKO+kZ8Pf/oTdOsG\nH39szjdtgrfeghEjKlf6AE2amAri889h7Vq4+GJ47DHo1Amef970DhwOR7UJZjpnPyBbVX9SsxbI\nFMxKiIdR1e2quhizRneVwjrqGcuXQ58+pqV+8cXGzPPAA9DS54q6gencGV57zcSblQXXXw9nnw1b\namsXRYej/hHMnrujMbvUXGuvx2GWV73Bh9/7gLxyU08Vw07EbBRBZmZm1pQpU2qSL7/k5eWRGqjF\nGWHqpIyqtHn/fTo/9xzFjRuz+q672NunT2gTVaX1++/T+YUXKGvQgB/vuINdgwYFL2OUUhfkdDKG\nhnDKOGzYsKWq2jcoz4Gm/WBWfHzJ43ocdl9MH37vA26rTljPw03nnBVpEQJSQcaDB1XHjlUF1XPO\nUd2+PbyJr1mjmpWlKqL6yCOqZWWBZYxi6oKcTsbQEC3TOYMx9Wym4sYPbe29YKhJWEddYMcOGD4c\n3n7b2OE/+ghaBPcNSbU57jj46iu49FK46y646iozJdThcARFMLN6FgNdRaQjRmlfitnMIBhqEtYR\n7axbZ6Zkbt5sFP+oUbWXdnIyvP66mfN/zz3wyy+m0klPrz0ZHI46SkDFr6olInIDZlPieOBfqrpS\nRCZZ9+ftTk5LMDs5lYnIH4DuqrrfV9hwZcZRe6Rs2ACXXAJlZfDllzBwYO0LIQJ3320GgMeNgzPO\ngE8+gebNa18Wh6MOEdQ8flWdgdnZyfPe8x7nuRgzTlBhHXWc776j1x/+YKZlfvklHH98ZOW59FJI\nSYExY2DoUPjsM/fBl8NRCW51TkfVWLYMfvUrSpOSYM6cyCv9cs47z3wzsGEDDBsG27ZFWiKHI2px\nit8RPCtWGHNKWhrLn3zSLKkQTQwfDjNmGHv/mWeSsH9/pCVyOKISp/gdwZGdDWeeaQZVZ82iIFpN\nKaefbtb6WbuWk2+/Hfbti7REDkfU4RS/IzA5OaalX1wM//0vdOwYaYkq58wzYdo0UrOz4dxz4dCh\nSEvkcEQVTvE7Kmf3bjjrLPN/5kyzbHJd4NxzWf2//wtff21W+CwpibREDkfU4BS/wz+HDsH558P6\n9WaOfFZWpCWqEjt+9St46il4/3347W/d4m4Oh8Wtx+/wTWmpaSnPn29W1RwyJNISVY8bb4TcXHj4\nYTPF8/77Iy2RwxFxnOJ3HI0q/P73pqX89NMwenSkJaoZDz5olP8DD0D79mbJZ4cjhnGK33E0f/0r\n/OMfZuOTG2+MtDQ1R8Ss45+TAxMnQps28OtfR1oqhyNiOBu/oyJvv20U/iWXwCOPRFqa0JGYaPJ2\n4ommB7N8eaQlcjgihlP8jiPMn2/WvBk0yOyPG1fPHo/0dPN1b5MmcM45pgfgcMQg9ezNdlSb7Gwz\ng+fYY41tPykp0hKFhzZtzNe9Bw6YOf4HDkRaIoej1nGK3wG7dpntDMEoxYyMyMoTbk46yWwAv2IF\njB3r5vg7Yg6n+GOdggK48EKzvs0HH0Tf+jvh4qyz4Lnn4NNPzQC2m+PviCHcrJ5YRtVMbZw3D6ZM\nMbb9WOI3vzEfpz32GHTqBH/8Y6QlcjhqBaf4Y5m774Y33zQfN11ySaSliQwPPwwbN5qZTO3bG9OP\nw1HPcYo/VnnhBTNdc+JEuPPOSEsTOeLizAymzZvNjKbWrWHw4EhL5XCEFWfjj0VmzDBr15x9Njz7\nrPnAKZZJSjIzmTp0gAsugB9/jLREDkdYCUrxi8gIEVkjItkiclTzUAxPW/fvRaSPh9tGEflBRJaL\nyJJQCu+oBosXG3NG794wdSokuE4fYPbpnTHDlMeIEbBlS6QlcjjCRkDFLyLxwLPASKA7cJmIeK/N\nOxLoao+JwHNe7sNUtZeq9q25yI5qs3ataeUfcwxMn272zHUcoXNno/x37jTl5DZxcdRTgmnx9wOy\nVfUnVS0CpgAXePm5AHhVDQuBJiISpVs0xShbtpgpjCJmM/KWLSMtUXSSlQXvvgsrV8JFF0FhYaQl\ncjhCjmiA+csiMhoYoarX2utxQH9VvcHDz3TgUVWdZ6+/AO5Q1SUisgHYB5QCL6jqZD/pTMT0FsjM\nzMyaMmVKjTPni7y8PFKjvKUbahkT8vLoddNNJG3dyvK//528EGyQXt/LMfOzzzjhkUfYcdpprLr3\nXjQ+PsTSHaG+l2VtEesyDhs2bGnQVhVVrfQARgMveVyPA57x8jMdGOxx/QXQ1563sf+PAb4DTg+U\nZpJj25AAAAwUSURBVFZWloaLWbNmhS3uUBFSGQ8cUB0wQDUxUfWzz0IWbUyU45NPqoLq+PGqpaWh\nEMknMVGWtUCsywgs0QC6tfwIZmRvM9DO47qtvReUH1Ut/79dRN7DmI7mBlUrOWpG+Q5aixeblSnP\nPDPSEtUtbrrJ2Pnvvdcs7Pb3v7sZUI56QTA2/sVAVxHpKCINgEuBD738fAhcZWf3DAD2qepWEUkR\nkTQAEUkBzgJWhFB+hz+KimDMGJg928xTv+iiSEtUN/nTn+Dmm80Wjnfd5ZZ2cNQLArb4VbVERG4A\nZgLxwL9UdaWITLLuzwMzgLOBbOAgcLUNngm8J6aVlAC8oaqfhjwXjooUFhql//HHZgOSK6+MtER1\nFxGzMU1BgVnaQcR87eta/o46TFCTuFV1Bka5e9573uNcgd/5CPcT0LOGMjqqQkEBjBplpiU+8wxc\nd12kJar7xMWZslSFRx81Sv+hh5zyd9RZ3Nc79YlDh4xJZ+ZMsyTDxIn/v71zj5GqvuL458y+WHdN\nkUCWlYogWRVoyoorIBGziluskFCNVawx0prYJopttEbdGkuiRGLqo6kN1mdofRATSyUVNWpcFXzx\nEGXZBSUIVp6C2cLycHdmTv84d91h2cdAZ+fe2Tmf5GTu3Mfsd3+5c+b3+93zOydsRQOHWMxWOata\nqovDh+HBBwdesRonL3DHP1BoabEHuStWwFNPeUHx/iAWs1rEgwbBI49Ymz/xhK9+dnIOv2MHAjt2\nWJqBjRst22a+ZtrMBrGYRfcMGWLRPi0t1uYDtWKZMyBxx5/rbNxoTn/fPpvXv+SSsBUNfETgnnvg\nlFPglltg+nRL8jZsWNjKHCctfIIyl3ntNZgyxeab33nHnX62mTfP1kesXQuTJ0NTU9iKHCct3PHn\nIqrw0EMwc6alEl61CiZO7PMypx+48kr70T10CKZOtVGX40Qcd/y5xoEDFpd/220WwbNyJYwcGbaq\n/GbSJPjoI/sRnjkT7r4bEomwVTlOj7jjzyXWrrWe/ZIlcN998OKLUFYWtioHrGzjBx/ADTdYjH9d\nHezaFbYqx+kWd/y5QCJhq0enTLEFWg0NVi/XY8ijRWkpPPkkPPOM/QiMH28/zo4TMdxzRJ3GRps7\nvv12Kw6ybh1Mmxa2Kqc35s610dmYMRZae/XVVtzFcSKCO/6ocvCghQxOnAhbtlis+NKlViLQiT5j\nx8L779u0z9KlcOaZsGiRz/07kcAdf9RIJql4/XVzFPfea/Vxm5pgzhzPDZNrFBZCfT188glUV1uB\n+5oaiwJynBBxxx8Vkkkr+XfuuYxduBBGjLCInWef9YVBuc748fDWW1bcfu9eqK21h78ffhi2MidP\ncccfNm1t8PzzMGGCZdVsbaW5vt6cwtSpYatzMoWIjd4+/9ySu336KZx/PsyYwZCPP/Y8/05Wcccf\nFl99ZUU+Ro6Ea6+FeNx6983N7K6r84idgUppKdx6qz23uf9+WL+eH99xh40KHn3UUm84Tj/j3iWb\n7Ntn6ZJra22xz4IFcN558OqrsGGD/QB4psf8oLwc7rwTtm6lqb7e1mPMmweVlXDFFfDSS9DaGrZK\nZ4DiXqY/SSbhs8/MsS9fblEeySScdRbMnw/XXQejR4et0gmT4mL21NUxbsECm/5ZvBiee84igUpK\nLP/SrFlw8cVQVeUP+J2M4I4/k3zzjUVwrFljefFXrrRi3WBhmXfdZfP41dX+BXaOZcIEy8H0wAPw\n3nvw8stmr7xix089FS680FJE1NTAOefYyMFxjpO0HL+IXAr8Gau5+6SqLuxyXILjl2E1d+eq6tp0\nrs059u+HbdvMvvwSNm0ya26G7ds7zxs71h7mTZtmvbbKyvA0O7lFYSFcdJHZww/D5s3w9ttmK1ZY\nyg6wzsOoUTBunNmYMTaCHD3aosJOOinUf8OJLn06fhEpAP4K1AFfA6tEZJmqpuag/SlQFdhkYBEw\nOc1r+w9VWzATj0N7O7S1Ubx3rznsI0fMDh0yO3jQ5lT377deekuLzcnv22cheLt2wc6ddl4qJ58M\nZ59tQ/EJE6xnX11tudod5/9FxKZ4qqo6S2nu2gWrV9vq4KYmszfesAixVAYPtlHCsGGdNniw2Q9+\nYPdueblZaWmnlZR0WnExFBXZFKUzYEinxz8J2BwUTkdElgCzgVTnPRv4e1B0/UMRGSwilcCoNK7N\nHBUVlr0ykei0LqQdIFlcbKtkO6ymBoYPt5776aebjRplf9OnbZxsMny4zfvPmtW5L5m0SmxbtljH\nZscOG4Hu2GEdl8ZGe21psY7QcVILdp8XFBxtsVjnayxm5/RlHZ/VQTrbaTD58GH74TrB67PBpK4a\nuzJ0qOV56mfScfwjgP+kvP8a69X3dc6INK8FQERuBG4EqKiooKGhIQ1pR3PG9On2BYjF0FgMLSj4\n3pJFRWhhIYcTCYrLy0mWlJAsLiZRXEyytJTEoEEkSkuJl5WRKCsjWVTU+41z5IhVv9q48bh19kVr\na+sJ/f/ZxDVmjozr7OiYdIcqsSNHKGxtpeDw4U5rayP23Xdm7e3E2tuRtjZiiQQSj9N+6BAlBQVI\nMokkEkgyCcmkvU8mbXSt2rkNSLDvqPeBhlQ93SEnsK6hPR6nqCMqLqLrIuLxOAd6idyLl5XxRRbu\n2cg83FXVx4HHAWpqarS2tvb4PySNaxoaGjihz84irjEz5IJGyA2drjEzpKNxRBZ0pOP4twOnpbz/\nYbAvnXOK0rjWcRzHySLpOP5VQJWIjMac9hzgF13OWQbcHMzhTwb+q6o7ReSbNK49hjVr1uwVkW3H\n8X8cD0OBqOfIdY2ZIRc0Qm7odI2ZoT819jC/dyx9On5VjYvIzcDrWEjm06q6QUR+Exx/DFiOhXJu\nxsI5f9nbtWn8zX7LSiYiq1W1pr8+PxO4xsyQCxohN3S6xswQFY1pzfGr6nLMuafueyxlW4Gb0r3W\ncRzHCQ/P1eM4jpNn5KPjfzxsAWngGjNDLmiE3NDpGjNDJDSKRjTe1XEcx+kf8rHH7ziOk9e443cc\nx8kz8sbxi8jPRWSDiCRFpKbLsbtEZLOIbBKRGWFpTEVE5ovIdhFZF9hlYWvqQEQuDdpqs4jcGbae\n7hCRrSKyPmi71WHrARCRp0Vkj4g0puwbIiJviMgXwWuo2f160Bipe1FEThORt0WkKfhO/zbYH5m2\n7EVjJNoyb+b4RWQskAT+BvxeVVcH+8cBL2DJ6E4F3gTOVNVjM7xlERGZD7Sq6p/C1NGVIOPq56Rk\nXAWuyVrG1TQRka1AjapGZkGPiFwItGIJDX8U7HsA+FZVFwY/oqeo6h0R0zifCN2LQQLISlVdKyIn\nA2uAnwFziUhb9qLxKiLQlnnT41fVZlXd1M2h2cASVf1OVb/EFqFNyq66nOL7bK2q2gZ0ZFx1+kBV\n3wW+7bJ7NrA42F6MOYfQ6EFjpFDVnR31PlT1ANCMpbiJTFv2ojES5I3j74WeMotGgXki8lkw/I5K\ngv8ot1cqCrwpImuCzK9RpUJVdwbbu4CKMMX0QhTvRURkFHAO8BERbcsuGiECbTmgHL+IvCkijd1Y\nJHukfehdBJwBVAM7gQdDFZt7XKCq1ViRoJuCKYxIE6yAj+LcayTvRREpB14Cfqeq+1OPRaUtu9EY\nibaMTFrmTKCql5zAZelkH+0X0tUrIk8A/+5nOekSWnsdD6q6PXjdIyJLsSmqd8NV1S27RaQySGpY\nCewJW1BXVHV3x3ZU7kURKcIc6nOq+s9gd6TasjuNUWnLAdXjP0GWAXNEpCTIIloFfByypo6HQx1c\nDjT2dG6W+T5bq4gUYxlXl4Ws6ShEpCx4oIaIlAE/ITrt15VlwPXB9vXAyyFq6Zao3YsiIsBTQLOq\nPpRyKDJt2ZPGqLRlPkX1XA78BRgGtADrVHVGcOwPwK+AODYkezU0oQEi8g9sOKjAVuDXKfOXoRKE\noD1CZ8bVBSFLOgoROQNYGrwtBJ6PgkYReQGrYjgU2A38EfgX8CIwEtgGXKWqoT1c7UFjLRG6F0Xk\nAuA9YD0WqQdQj82hR6Ite9F4DRFoy7xx/I7jOI7hUz2O4zh5hjt+x3GcPMMdv+M4Tp7hjt9xHCfP\ncMfvOI6TZ7jjdxzHyTPc8TuO4+QZ/wMBXr6rHUbrvgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYFNW5/z/vLGzDDNvgsDMqKOKCCoILiZi4ADGSX1Ri\nNG5JJCSaGJck5pobTa4mxly90atR0SRKYhT3oJe4hlFREUERBQIiAgPMsDMw7MO8vz/OaSja7p7q\nme7pnun38zz1dHWdpb6nuvpbp845dUpUFcMwDCN3yMu0AMMwDKN5MeM3DMPIMcz4DcMwcgwzfsMw\njBzDjN8wDCPHMOM3DMPIMcz4jaxERMpE5A0R2Soid2RaTzoRkZtF5G8JwueLyKg4YaNEZGWCtA+L\nyC0pkGm0Isz4mwERuVBEZotIrYhUicg/RWSkD4v5pxcRFZEBge+DRWSqiNR4M5wuIidHpWnj8/tE\nRLaJyDIR+bOIlAfinC0is3z4BhF5VET6BMIvE5EZDZTnYRGpE5GegW33+/LVishuEdkT+P5PESn3\nZaqNWr4RZzcTgPVAiapel/AAN0AmzM//PutEZIuIfCgi4xqbl6oeqaoVKZSXVqLP3RDxK0Tkuyna\n9xMicqaItBWR6qiwtv7/sEVEqkXk2lTssyVixp9m/Mn1B+A3QBnQD7gXOCeJPA4F3gI+Ag4GegHP\nAi+LyEmBqE/5fC8EOgFDgNnAl30+5wF/93pKgSOBXcAMEekSUksRcC5QA3wrsl1VJ6pqR1Xt6Ms6\nJfJdVccEsugc2N5RVafE2VV/YIFmwROGIlLQiGQ/BvqoagnuIva34IXSSBtDcef8McDHUWE3AwNx\n59ZpwE9FZHSzqssWVNWWNC04860Fzk8Q52bgbzG2KzDAr/8VmBYjzn3AG379dGAH0DfOfgRYDvw0\nanse7g/ya//9MmBGAr2XAJXA1cDHYcsElPsyFYQ4bg8De4Dd/vid7nXeAHwKbACeALoG0jwJVOMu\nSG8AR/rtE6Lyej76+Ab2eYtfHwWsBH7m8/yr3342MBfYDLwNHBPyPBgO7ASGJzheTwCTga3AfGBY\nIHwZcLpfb++1bgIWAD8BVgbiHge87/OZAjweKVdDZfD7uR6Y54/jFKBdHM0DgNd9vPW4Cz3+2Cuw\nzR/vbwBdgBeAdV73C7iLIsCtwF5/fGqBe/z2QcArwEZgETA+xHHuAiz1698Hbo8KXw2cGfj+a+Dx\n5vaFbFgyLqA1L8BooI4EZkc4468GLo8R5zT/p2kP3Aa8nmA/g3yeB8cI+xXwjl+/jMTG/xpwO+7u\npQ4YGqZMJGH8Pv7DUYZ1NTAT6AO0BR4AHguEfxso9mF/AObGyyv6+EbHwRl/HfA7n197nKGuBUYA\n+cClOKNsm6AML3hDU+BFIC/BObATGOvz/i0wMxC+jP3GfxvwJtAV6Iu7aK/0YW1wF/drgELgPNxF\nL1KuhGXw67Nwd5RdgYXAxDiaHwNuxF2Q2wEjExzbbri7xA7+N3oSeC4QXgF8N/C9CFe5uBwo8LrX\nA4PjaPky7kJWi7vAb/af2/z6qbiLggJlgXTnAh9l0iMytVhTT3rpBqxX1boG4o0Xkc3BJSq8FKiK\nka4K98fr6vcVK04wD+LEqQqEx0VE+uEuNn9X1TW4i8AlDaWLYn1UWY8ImW4icKOqrlTVXTizPC/S\nDKOqf1bVrYGwISLSKUltQeqBm1R1l6ruwN05PKCq76rqXlV9BNdMdmK8DFT1bJzRjQVeVtX6BPub\noarTVHUv7g5vSJx444FbVXWjqlYCdwfCTsQZ/h9UdY+qPgW8FwgPU4a7VXW1qm4EngeOjaNjD67J\npJeq7lTVuP1CqrpBVZ9W1e2quhVXyz81XnzcXckyVf2Lqtap6gfA08D5cfJ/TVU7A8/5OL1xF7FS\nVe2sqq8DHX30mkDSLbjfJ+cw408vG4DSEG3ET/gTdN8SFb4eiNU+3BNnUJv8vhK1Ia8PpImVz/oY\n26O5GFioqnP990eBC0WkMETaCKVRZV0YMl1/4NnAhXEh7m6nTETyReQ2EflURLbg/vQQ4mKWgHWq\nujNq/9dFXZz74mrHcfEG/E/gTBFJ1K8T7IjcDrSLc970wtWGIyyPClulvjobIzxMGaJ1dCQ2P8U1\nH87yo46+HSceItJBRB4QkeX+93kD6Cwi+XGS9AdGROm8COgRJ/+VPs43gUdwdzX9gSoRudNHq/Wf\nJYGknXBNYjmHGX96eQdXo/paE/N5ldi1nfG4JprtPs7w4AidKBbh2q0PyEdE8nC3vK+F0HEJcIgf\nEVEN3Ikz17GhStE0KoExUReNdqq6CteZPQ7XF9AJ16wEzpjA3eJHsx3X9BAh2lSi01TiatrB/XdQ\n1cdC6i8ADg0ZNxFVOLOO0C8qrLeISJzwppZhH6parapXqGov4HvAHxOM5LkOOBwYoa6z+4t+e7zf\npxLXbBnU2VFVvx9HSx9cs+qrvtI0CbjSp7vWx9mEOz7BO6khuP6UnMOMP42oag3wS+BeEfmar/kU\nisgYEbk9iax+BZwsIreKSFcRKRaRH+KM+Gd+X6/iOsOeFZGhIlLg400UkW/7WuD1wC/EDS9tJyI9\ngIdwtaD/CexPfHhwOQlnXMNxt//HAkfhRgkl29zTGO4HbhWR/l5g98AQyWLcBXYDzsx/E5V2DXBI\n1La5uLuVfD+yI1HTA8CDwEQRGSGOIhH5ioh8rqlARAb537i9/72/hTO715MobzyeAH4uIl38Rf6H\ngbB3cH0TP/L7/Tru90q6DA0hIucHKhmbcOYdacqKPt7FuIEHm0WkK3BTVHbR8V8ADhORi305CkXk\nhAaaBYfiOrUBjseN7IlmMu787+LzugLXt5NzmPGnGVW9A7gW+AVuVEMlcBWuPTJsHp8AI3E1lGW4\nmsu5wFmq+lYg6nnANNxojBpcx98w3N0A6oZOXozr/NuAGxXSHjhFVTcE8jkZ90cNLt8B/qGqH/na\nXrWqVgN3AWf7P3QYNsuB4/jDjqW+C5iKG8K6FdfRO8KHTcY1aazyZZoZlfZPwGDfbBA57lcDX8V1\n/l1EA7+Hqs7GGcU9OKNbgusIj4Xg+hnW4n7zq4FvqOr7ceInw69wZf0MeBnXHxDRuBv4ute1ETei\n5plGlqEhTgDeFZFa3O9ytaou9WE3A4/44z0e19neHtecOBPX0R3kLlx/zSYRudv3A5wJXIAbiVPN\n/o72eAwF3vd3O4OIXZO/CTcqbDmuQ/l2VY3WkhPIgc2BhmEYRmvHavyGYRg5hhm/YRhGjmHGbxiG\nkWOY8RuGYeQYoSaf8sPd7sI95v2Qqt4WFT4I+AtuGNWNqvrfYdPGorS0VMvLy8OWIe1s27aNoqKi\nTMtoFKY9M5j25qel6obUaJ8zZ856Ve0eKnJDczrgDPtT3DjbNsCHRM2ZARyEG951K3B9MmljLUOH\nDtVsYvr06ZmW0GhMe2Yw7c1PS9WtmhrtwGxN4Vw9w4ElqrpU3Tjhx3FPSQYvHmtV9T3c/B1JpTWM\n5mDtx1D5dqZVGEZ2EKappzcHzg2ykv0PzqQsrYhMwE0iRVlZGRUVFSF3kX5qa2uzSk8ymHbH3GuG\nsGttO0Y8+m5K8msIO+7NT0vVDc2vvTEvmEgLqjoJN8cGw4YN01GjRmVWUICKigqySU8ymHbQenhn\nCezeBqcMH0Vhh4bTNBU77s1PS9UNza89TFPPKg6cFKqP3xaGpqQ1jJSwfhHsrgUU1v8702oMI/OE\nMf73gIEicrCItMHNnzE1ZP5NSWsYKWF1YLqudQsyp8MwsoUGm3pUtU5ErgJewo3S+bOqzheRiT78\nfj/L42zcLI/1IvJj3OidLbHSpqswhhGL1e9BYQfYuwfW2tlnGOHa+FV1Gm7Wx+C2+wPr1bhmnFBp\nDaM5WT0beg6FHRthvdX4DcOe3DVaN/V1UP0B9BoG3Qdbjd8wwIzfaOWsnQ91O6HXCdD9SNi0FPbs\nyLQqw8gsZvxGqybSsRup8dvIHsMw4zdaOavfg7adoOuhcNCRbpuN7DFyHTN+o1Wzerar7UsedB0A\neQWwztr5jRzHjN9otdTtgjXznPED5LeBbodZjd8wzPiNVsuaeVC/Z7/xg2vntxq/keuY8Rutln0d\nuyfs32YjewzDjN9oxax+DzqUQqd++7d1H+wmbduwKHO6DCPTmPEbrZa1H0OPY0Fk/7buNrLHMMz4\njdZLzQro1P/Abd0GupE99gSvkcuY8RutkrpdsG0NlPQ9cHt+G+g60ObsMXIbM36jVbLVv/Uh2L4f\nwebsMXIdM36jVVKzwn126vv5sK4DoGa56+Q1jFzEjN9oldT4Nz1HN/WAuwvYuxtq1zSvJsPIFsz4\njVbJFm/8sWr8keafyF2BYeQaZvxGq6SmEtp3JeaL1SMjfcz4jVzFjN9olWypjN2xC4Ea//Lm02MY\n2YQZv9EqqVkRu30foF0naFtiNX4jdzHjN1olWyrjGz+4Wr8Zv5GrhDJ+ERktIotEZImI3BAjXETk\nbh8+T0SOD4QtE5GPRGSuiMxOpXjDiMXuWti5OXbHboRO/a2px8hdChqKICL5wL3AGcBK4D0Rmaqq\nwWcfxwAD/TICuM9/RjhNVdenTLVhJCDRUM4InfrByneaR49hZBthavzDgSWqulRVdwOPA+Oi4owD\nJqtjJtBZRHqmWKthhCLRUM4InfrBjo3u7sAwco0Ga/xAb6Ay8H0lB9bm48XpDVQBCrwqInuBB1R1\nUqydiMgEYAJAWVkZFRUVYfQ3C7W1tVmlJxlyUXvVKz2Bw1m4eiafVeyMGWfttoOAwbzy9CyK+m9v\nks5Y5OJxzzQtVTc0v/Ywxt9URqrqKhE5CHhFRP6tqm9ER/IXhEkAw4YN01GjRjWDtHBUVFSQTXqS\nIRe1T58OiwXOOPdE8gtjx1lRCAtvgcPLhjMg+V00SC4e90zTUnVD82sP09SzCgjeNPfx20LFUdXI\n51rgWVzTkWGkjS2V0LEHcU0f7OldI7cJY/zvAQNF5GARaQNcAEyNijMVuMSP7jkRqFHVKhEpEpFi\nABEpAs4EPk6hfsP4HFsqE7fvAxT3BMk34zdykwabelS1TkSuAl4C8oE/q+p8EZnow+8HpgFjgSXA\nduByn7wMeFbcK5AKgL+r6ospL4VhBKiphIOOShwnrwBK+tiQTiM3CdXGr6rTcOYe3HZ/YF2BK2Ok\nWwoMaaJGwwiNqqvFDxzbcFx7iMvIVezJXaNVsWMj1O1IPIY/ghm/kauY8RutijBj+CN06g9bVkL9\n3vRqMoxsw4zfaFWEeWo3Qqd+UF8HtVXp1WQY2YYZv9GqSKrGb0M6jRzFjN9oVdSsgLxCN46/ISLG\nv9lG9hg5hhm/0arYUgklvUFCnNlW4zdyFTN+o1VR08A8/EHaFkO7Lmb8Ru5hxm+0GlRh/ULoemj4\nNJ36wRYzfiPHMOM3Wg0bFsP29dD3lPBpOve3Nn4j9zDjN1oNK2a4z34jw6cp6Qebl4HWp0WSYWQl\nZvxGq6FyBrTvBt0OD5+m93DYvRWqP0yfLsPINsz4jVbDihmutu/mBAzHIae7z09fTo8mw8hGzPiN\nVkFtNWxcklwzD7jpmcuOgaVm/EYOYcZvtApWvOU+kzV+gEPOdHcLu7elVpNhZCtm/EarYMUMKGgH\nPY9PPu2hZ8Le3bD8cy8ENYzWiRm/0SqonAG9R0B+m+TT9hvpLhpLX0m9LsPIRsz4jRbP7lqo+qBx\nzTwAhe2h3xesg9fIHcz4jRbPyndB9zbe+ME196ybD1tWpU6XYWQrZvxGi2fFDECgz0mNz+PQM92n\nNfcYuYAZv9HiWfGmG5LZrlPj8zjoaCgqs+YeIzcIZfwiMlpEFonIEhG5IUa4iMjdPnyeiBwfNq1h\nNBZVeOt2+Ow1GDC6aXmJuFr/py9bc4/R+mnQ+EUkH7gXGAMMBr4pIoOjoo0BBvplAnBfEmkNI2m0\nHl66Fl79GRz1TTjt103Pc+gE96L2+46GBU81PT/DyFYKQsQZDixR1aUAIvI4MA5YEIgzDpisqgrM\nFJHOItITKA+RNmXMf9J18qWatQsO4uPq1OfbHLR07R9VAepq97oXdmx0T+munu1q+iN+DGfdEe7F\nKw3RbyR8by48cxE8eT4ccS70ONbN2d+uk9+Hnw6ioWkhWvpxb4naW6pu2K+9oB0M+lr69xfG+HsD\nlYHvK4ERIeL0DpkWABGZgLtboKysjIqKihDSDuTNS75A/c78pNM1zGAWpiHX5qH1aZeCetp02c2h\nP1hJ23NW8nqKH7w69DdCweT+LH66DwufDvMXiUXrO+7ZT0vVDRHthV12c3Lnt9O+t8ae1SlHVScB\nkwCGDRumo0aNSjqPo+YCmlpdALNmzWL48OGpz7gZaKna165fw/jvjGbZyiVcMn4C//UfdyAC7btC\nuy55iLQDBvglDZwOTHZP9O7YBLu2BKZuDnGOJXPcb7/nZj5bsYT7bv9bzPCRZx/J7355L6cMH/W5\nsLdmVfD9n36LeRUrY6a96ueX0ausD//x41tCaUlWezbRUnXDfu2S34ZuA0elf4eqmnABTgJeCnz/\nOfDzqDgPAN8MfF8E9AyTNtYydOhQzSamT5/epPSPPvqoDh06VIuKirRHjx46evRoffPNN1VV9aab\nbtKLLrroc2kA/eSTT/Z9nz9/vn71q1/VkpIS7dixo44aNUrfeuutA9Ls2rVLb7rpJh0wYIB26NBB\n+/fvr6NHj9bPPvtsX5znn39eTzjhBO3QoYN27dpVL7zwQq2srNwX/pe//EVPOeWUhOW59NJLNT8/\nX1evXr1v2/e+9z0tKirSoqIiLSws1IKCgn3fIxqAfdsiy+OPPx5zH7/+9a/1C1/4gtbX1yfUEoZL\nL71Ub7zxxibnkwzHHnuslpaWanFxsR5zzDH63HPPxY0b7xwIw/Tp07V3795xwxtT9qae79HnbkOc\neuqp+uCDDzZpn6pO9/nnn68vvfSS7ty5U8vKyg4InzJlip500knavn17PfXUU5u8v1TS1GOuqgrM\n1ga8NbKIix8fESkAFgNfBlYB7wEXqur8QJyvAFcBY3FNOXer6vAwaePscx2QTe9FKgXWNzJtGdAD\nV54tuPpiCVCMa/rqBbQFPotKNxT4GNjlw48A1gHVPo9SXFPaYiAyvdgAoNDvazuQD/QDtnr9XXD9\nLsuBTT68j9eyANgLdPN5L4pTnjxgiNdQBayJESdWmdoARwNz4uQbTX9fliUh4yeiHNgNrE5BXmHp\nyf7fqgg4DPd77okRN945EIZi4GBgXpzwcpIve1POdzjw3A3D4cCGJu4TnO4ewELc8eyD+39EKMa1\ncrTD/QfjneOZoKnHHKC/qnYPFTPM1QFn6IuBT4Eb/baJwES/LrjRO58CHwHDEqVtaQtJXEmj0nUC\naoHzE8S5GfhbjO0KDPDrfwWmxYhzH/CGXz8d2AH0jaXd/0bLgZ9Ghefh/qS/9t8vA2Yk0HsJrt/m\nauDjsGXCGZACBSGO28M4g6z3x+90r/MGfx5tAJ4AugbSPIkz2hrgDeBIv32Cz2u3z+v56OMb2Oct\nfn0U7qL8M5/nX/32s4G5wGbgbeCYMOcMboDETmB4guP1BDAZd5GeH/UfWgac7tfbe62bcBfrnwAr\nA3GPA973+UwBHo+Uq6Ey+P1cj6s01Pj07eJoHgC87uOtB6b47W/4Y7vNH+9v4CocL+AqLpv8eh8f\n/1ZchWOnj3+P3z4IeAXYiDPo8SHOmw+ApX79+8DtceJ9F6hIhTekaqGRHtPo/WW6wC1haeyPAowG\n6khgdoQz/mrg8hhxTvN/mvbAbcDr8bT7P5ICB8eI8yvgHb9+GYmN/zXgdtydTB0wNEyZSML4ffyH\ngdWB71cDM3G1uLa45sXHAuHfxtXo2gJ/AOZG5XVLVP4NGX8d8DufX3ucoa7F3dHmA5fijLJtvHPG\nG9xOv68XgbwE58BOXCUpH/gtMDMQvoz9xn8b8CbQFeiLu2iv9GFtcBf3a3B3S+fhLnqRciUsg1+f\nBXzo81+Ir9zF0PwYcCPugtwOGJng2HYDzgU6+N/oSeC5QHgF8N3A9yJc5eJyXA39ONzFZXAcLV/G\nXcj24i7wm/3nNr9+alT8nDd+e3I3vXQD1qtqXQPxxovI5uASFV6Ka1aJpgr3x+vq9xUrTjAP4sSp\nCoTHRUT64S42f1fVNbiLwCUNpYtifVRZjwiZbiLujnGlqu7CmeV5vjkRVf2zqm4NhA0RkSY8y0s9\ncJOq7lLVHbg7hwdU9V1V3auqj+CaMk6Ml4Gqno0zurHAy6oJ3+w7Q1Wnqepe3B3ekDjxxgO3qupG\nVa0E7g6EnYgz/D+o6h5VfQrXvBohTBnuBvao6kbgeeDYODr24JrjeqnqTlWdEa9gqrpBVZ9W1e2q\nuhVXyz81XnzcXckyVf2Lqtap6gfA08D5cfJ/TVU740z+fFwT6DKgVFU7q+rrCfaVk5jxh2NSI9Nt\nAEoj5pSAJ/wJum+JCl+PazOOpifOoDb5fcWKE9G+PpAmVj5h2hcvBhaq6lz//VHgQhEpDJE2QmlU\nWRONwPsgsN4feDZwYVyIq+GViUi+iNwmIp+KyBbcnx5CXMwSsE5Vd0bt/7qoi3NfXPt8LCYBeAP+\nJ3CmiJyTYH/BEejbgXZxzpteHDhEenlU2Cr1VcgY4WHKUM3+c2Y70DGO3p/img9nich8Efl2nHiI\nSAcReUBElvvf5w2gs3/AMxb9gRFROi/Ctd/Hyn+lj9MFeAR3V9MfqBKRO+PpyjIa6zGNwow/BOqG\nmjaGd3A1qqY+kvEqsWs743FNNNt9nOEi0icYIaB9Ea7d+oB8RCQPdxv+WggdlwCHiEi1iFQDd+LM\ndWwSZUmGoPFXAmOiLhrtVHUVcCHuwcDTcf0q5T5N5DGrWCMYtuOaHiJEm0p0mkpcTTu4/w6q+lgs\n4THOmQLg0Fhxk6QKZ9YR+kWF9RY54PGyYHioMoQ531W1WlWvUNVewPeAP4pIvLG11+E6cEeoagnw\nRb893u9TiWu2DOrsqKrfj6OlD65Z9RVfaZoEXOnTXdtQWbKBJnhMozDjTyOqWgP8ErhXRL7maz6F\nIjJGRG5PIqtfASeLyK0i0lVEikXkhzgj/pnf16u4zrBnRWSoiBT4eBNF5Nu+Fng98AsRuVBE2olI\nD+Ah3AiH/wnsT3x4cDkJZ1zDcbf/xwJHAX8n+eaexnA/cKuI9PcCu4vIOB9WjLvAbsCZ+W+i0q4B\nDonaNhd3t5IvIqNJ3PQA8CAwUURG+LmpikTkKyJSHB1RRAb537i9/72/hTO7VDQ5PAH8XES6+Iv8\nDwNh7+D6Jn7k9/t13O+VdBkaQkTOD1QyNuHMO9KUFX28i3EDDzaLSFfgpqjsouO/ABwmIhf7chSK\nyAkNNAsOxXVqAxyP62OJ1pwv7gGQAiDPn9fJ3K22Gsz404yq3gFcC/wCN6qhEjf09bkk8vgEGIlr\n912Gq9mdC5ylqm8Fop4HTMONxqjBdfwNw90NoKpTcM011+BMcgGu4/IUVd0QyOdk3B81uHwH+Ieq\nfuRre9WqWg3cBZzt/9Bh2CwitYElbI3sLmAq8LKIbMV19EaeAp+Ma9JY5cs0Myrtn4DBvtkgctyv\nBr6Kaxe+iAZ+D1WdDVwB3IMzuiW4jvBYCK6fYS3uN78a+Iaqvh8nfjL8ClfWz4CXcf0BEY27ga97\nXRtxI2qeaWQZGuIE4F0RqcX9Llern5oFV/ZH/PEej+tsb49rTpyJ6+gOcheuv2aTiNzt+wHOBC7A\nDUOtZn9HezyGAu/7u51BuJFR0VyMO5fvA77g1x9MqtSthAbH8RuGYRitC6vxG4Zh5Bhm/IZhGDmG\nGb9hGEaOYcZvGIaRY2TNtMxBSktLtby8PNMy9rFt2zaKiooyLaNRmPbMYNqbn5aqG1Kjfc6cOes1\n5CRtWWn85eXlzJ79uWG4GaOiooLGvB8gGzDtmcG0Nz8tVTekRruIhJ7ROCuN3zBaI/Uor7KQPexl\nIAdxMKUUko43xhlGYsz4DaMZUJRpfMz7rKCAPOb4zy8wgC8wMNPyjBzDjN8wmoF/sYj3WcEpHMqp\nDGQ5G5nFMqazmEPoTm+i5+UzjPRho3oMI828w1Le4lOOpx9f4nAKyOdQuvN1jqWYtvwfH1FPohmb\nDSO1mPEbRhqpYQev8W8GUcZYjkLYP3FmWwo5iyOpZguz9s0kbRjpx4zfMNLILJahKGcymDzkc+FH\n0IMBdGc6i6lhRwYUGrmIGb9hpIld7OF9VnAEPel8wNT/+xGEsRyForxConfSGEbqMOM3jDTxAZXs\noo6TPvcqgAPpTAdOoJyFVLOVnQnjGkYqMOM3jDSgorzLMvrRNdSInePoi6LMY1UzqDNyHTN+w0gD\n60r3UsMOTuLgUPFL6UhfujCXSjTmmyINI3WY8RtGGljVdw9dKeIwykKnOY6+bGAbK9iURmWGYcZv\nGClnLVvZWlLPcPofMHyzIQbTkzYUMJfKNKozDDN+w0g581kNCoPplVS6NhRwFD1ZQBW72JMmdYZh\nxm8YKUVR5lNF5815dEz4bvDYHEtf9rCX+VSlQZ1hOMz4DSOFVLOFjWyj+9rGTYPVm850pyMfWHOP\nkUbM+A0jhcxnNXkIpesaZ/yCcDS9WcVme5LXSBtm/IaRIiLNPIdQSmFd+E7daI6gJwALqU6VNMM4\nADN+w0gRkVr6kUl26kbTjSLKKGahtfMbacKM3zBSxHyqyCePw5MYux+PQfSkkk02hYORFsz4DSMF\nKMoCVjOA7rSjsMn5DaYHAP+25h4jDZjxG0YKcLXzXRzp2+ebSneKKaXI2vmNtBDK+EVktIgsEpEl\nInJDjHARkbt9+DwROd5v7ysi00VkgYjMF5GrU10Aw8gG/k01+eQxkINSlucgerKcDWxjV8ryNAwI\nYfwikg/cC4wBBgPfFJHBUdHGAAP9MgG4z2+vA65T1cHAicCVMdIaRotGUf5NNYdQStsUNPNEGEwP\nFFjEmpTlaRgQrsY/HFiiqktVdTfwODAuKs44YLI6ZgKdRaSnqlap6vsAqroVWAj0TqF+w8g41Wxh\nMzsY5NskpUDdAAAfnklEQVTlU0UZJXShgzX3GClHVBNPASsi5wGjVfW7/vvFwAhVvSoQ5wXgNlWd\n4b+/BvxMVWcH4pQDbwBHqeqWGPuZgLtboKysbOjjjz/etJKlkNraWjp27JhpGY3CtKefZeW7WdF/\nDye93YHCPW78fqq0Lz1kN6v67OHEtzs06dmAZGgpxz2alqobUqP9tNNOm6Oqw8LEbdzjhUkiIh2B\np4EfxzJ9AFWdBEwCGDZsmI4aNao5pIWioqKCbNKTDKY9/dzH65RTwhmnnLhvW6q0D2Qzf+ItDho5\nkCH0aXJ+YWgpxz2alqobml97mKaeVUDfwPc+fluoOCJSiDP9R1X1mcZLNYzsYz21rKM25c08EXrR\niRLa2cNcRkoJY/zvAQNF5GARaQNcAEyNijMVuMSP7jkRqFHVKhER4E/AQlW9M6XKDSMLiIyzH5SC\nh7ZiIQhH0JNPWW9TNRspo0HjV9U64CrgJVzn7BOqOl9EJorIRB9tGrAUWAI8CPzAbz8FuBj4kojM\n9cvYVBfCMDLFv6mmN50poX3a9nEEPdhLPYtZm7Z9GLlFqDZ+VZ2GM/fgtvsD6wpcGSPdDEjiFUSG\n0YLYzHZWU8OXGZTW/fSlCx1py0KqOdoGxRkpwJ7cNYxGEnlZyuAUPa0bD9fc04MlrGU3dWndl5Eb\nmPEbRiOZz2p605kudEj7vo6gJ3XUs4R1ad+X0fox4zeMRrCOrVSzhaOaOAVzWPrRlQ60YYGN7jFS\ngBm/YTSCj1mNQMomZWuIPIRB9OATa+4xUoAZv2EkiXvT1mrK6UZH2jXbfo+mF3vYa1M1G03GjN8w\nkqSKGjayvdmaeSL0oytd6MBcVjbrfo3Whxm/YSTJx/6F6oOaqZkngiAcQx+WsYHNbG/WfRutCzN+\nw0iCet/MM4CDaJ/CKZjDMsSP45/3uVlTDCM8ZvyGkQTL2MBWdjV7M0+EznSgnG58yEqUxDPrGkY8\nzPgNIwneYxkdaJO2uXnCMIQ+bGI7lWzKmAajZWPGbxgh2cR2FrGG4+lHAfkZ03EEPWhDPh9aJ6/R\nSMz4DSMks1mOIAyjX0Z1tKGAwfRkPlXssjH9RiMw4zeMEOxhLx9QyRH0SOtMnGEZSn92U8cclmda\nitECMeM3jBB8xCp2socTKM+0FAB605mD6cZMPqOOvZmWY7QwzPgNowEUZRbLKKOEfnTJtJx9jGQA\nteziAyozLcVoYZjxG0YDLGU9a9nKcPojWfR6iXK60YfOvM1S9lKfaTlGC8KM3zASUE89r7CQzrTP\nupegCMJIBlDDDj6yB7qMJDDjN4wEzGEFa9nKGRyR0SGc8RjIQfSghLf4lHp7oMsIiRm/YcRhB7up\nYDHldGMQPTItJyaC8EUGsoFtvMtnmZZjtBDM+A0jDhUsZid7OIvBWdW2H83hlHE4ZfyLRaxla6bl\nGC0AM37DiEE1W5jNCobSnzJKMi0nIYLwFY6mLQX8gw+to9doEDN+w4iilp1MYTZFtGEUh2VaTig6\n0pavcBRV1DCDJZmWY2Q5ZvyGEWA3dTzGbLazm29yAh1ok2lJoTmCnhxNL95kCZ+wNtNyjCzGjN8w\nPPUoT/MB1dRwLsfRk06ZlpQ0ozmKgyhmCrPtxexGXMz4DQPXvPM47/EJaxnNkRyWwWmXm0J7CrmE\nE+lNZ57mfebaU71GDMz4jZxnEdXcz5ssYwNjOSpr5uNpLO0o5CKGczClTGUeU/mQLezMtCwjiwhl\n/CIyWkQWicgSEbkhRriIyN0+fJ6IHB82rWFkgr3U82+qeZRZTGEOJbTjCkYyjP6ZlpYS2lDABQzj\nJA7hI1ZzD9OZziK7ABgAFDQUQUTygXuBM4CVwHsiMlVVFwSijQEG+mUEcB8wImRaw0grirKLOjaw\njSpqWM1mPmEd29hFMW05jcM4mUPJb2U3wAXkcwZHMIz+/ItFvMkS3mQJPSjhMMroRSe6UURnOrS6\nshuJadD4geHAElVdCiAijwPjgKB5jwMmq6oCM0Wks4j0BMpDpE0Z/+TjtIxhXn3YLmqZl/J8m4PW\npD04IUHkfbPq1xVlL0o9Sh172eOXHexhKzupC5wX7SmkP105lr4MoDt5rdz0utCBczmOUxnIItaw\nmDW8ySf7jqcgFNGGdhTSjkK2HbOTat4jnzzyyUMQ8vwDbBJIEyQbHm9rDed6Gwo4k8Fp318Y4+8N\nB/QQrcTV6huK0ztkWgBEZAIwAaCsrIyKiooQ0g7koxHbqU/Df1i71rNxV8t8zV2r1q4BI9L9S169\nkL8X8vYKBXVQtjuPNrsLaLdDKN6aR9tdglDLahaymoVp0V5bW9uoc7g5OBjoW9CB7R3q2dG+nu0d\nlD2Fe6krqKO2cDt7ZC+rt66nPk+pF0BA/YFWyA6Xj0FrONcL9whtZjfDUFxVTbgA5wEPBb5fDNwT\nFecFYGTg+2vAsDBpYy1Dhw7VbGL69OmZltBoWqr26upqPeaYY7Rjx4567bXXZlpO0iRz3G+66Sa9\n6KKL4oYPHjw4bn7Tp0/X3r17x0176aWX6o033hhaSyTPlkhL1a2aGu3AbG3AWyNLmPrxKqBv4Hsf\nvy1MnDBpWz1///vfGTZsGB07dqRnz56MGTOGGTNmAHDzzTfzrW9963NpRIQlS/Y/gblgwQLOOecc\nOnXqRHFxMaeddhpvv/32AWl2797NzTffzMCBAykqKqK8vJzf/e53LFu2bF+cF154geHDh1NUVES3\nbt246KKLWLlyfy3p4YcfZuTIkQnLc9lll1FQUEBV1f5x4hMnTqRjx4507NiRNm3aUFhYuO/7mDFj\nWLZsGSKyb1tkmTJlSsx9TJo0iU6dOrFlyxbuuOOOhHoa4rLLLuMXv/hFk/JIlmuuuYbu3btTUlLC\nkCFD+Mc//tHovObPn8+oUaNSJy7NRJ+7DTFq1CgeeuihlOx7/PjxvPzyy+zatYsePQ6cWO/6669n\n4MCBFBcXM2jQICZPnpySfbZExF0oEkQQKQAWA1/GmfZ7wIWqOj8Q5yvAVcBYXFPO3ao6PEzaOPtc\nB1n1MtFSYH0j05YBPXDl2YK7Wy4BinFNX72AtvC5qRWHAh8Du3z4EcA6oNrnUYprSlsMbPNpBgCF\nfl/bgXygH7DV6++C63dZDmzy4X28lgXAXqCbz3tRnPLkAUO8hipgTYw4scrUBjgamBMn32j6+7Kk\nYv6BcmA3sDoFeYWlJ/t/qyLgMNzvuSdG3HjnQBiKca038Rq3y0m+7E053+HAczcMhwMbmrhPcLp7\nAAtxx7MP7v8RoRewEdiJ+00GAp+w//+TSZp6zAH6q2r3UDHD3BbgDH0x8Clwo982EZjo1wU3eudT\n4CNgWKK0LW0hiVuoqHSdgFrg/ARxbgb+FmO7AgP8+l+BaTHi3Ae84ddPB3YAfWNp97/RcuCnUeF5\nuD/pr/33y4AZCfReguu3uRr4OGyZcAakQEGI4/YwziDr/fE73eu8wZ9HG4AngK6BNE/ijLYGeAM4\n0m+f4PPa7fN6Pvr4BvZ5i18fhbso/8zn+Ve//WxgLrAZeBs4Jsw5gxsgsRMYnuB4PQFMxl2k50f9\nh5YBp/v19l7rJtzF+ifAykDc44D3fT5TgMcj5WqoDH4/1+MqDTU+fbs4mgcAr/t464Epfvsb/thu\n88f7G7gKxwu4issmv97Hx78VV+HY6ePf47cPAl7BGfUiYHyI8+YDYKlf/z5wewPxpwLXNcUbUrXQ\nSI9p9P4yXeCWsDT2RwFGA3UkMDvCGX81cHmMOKf5P0174Dbg9Xja/R9JgYNjxPkV8I5fv4zExv8a\ncDvuTqYOGBqmTCRh/D7+w8DqwPergZm4Wlxb4AHgsUD4t3G137bAH4C5UXndEpV/Q8ZfB/zO59ce\nZ6hrcXe0+cClOKNsG++c8Qa30+/rRSAvwTmwE1dJygd+C8wMhC9jv/HfBrwJdMU1o36MN37cXdVy\n4Brc3dJ5uItepFwJy+DXZwEf+vwX4it3MTQ/BtyIuyC348A+vuhj2w04F+jgf6MngecC4RXAdwPf\ni3CVi8txA1COw11cBsfR8mXchWwv7gK/2X9u8+unxkjTHnfHOjoVHtHUhWY2/tY9ji3zdAPWq2pd\nA/HGi8jm4BIVXgoxJ16pwv3xuvp9JZqcpTSQJlY+pTG2H4CI9MNdbP6uqmtwF4FLGkoXxfqosh4R\nMt1E3B3jSlXdhTPL83xzIqr6Z1XdGggbIiJNmWynHrhJVXep6g7cncMDqvququ5V1UdwTRknxstA\nVc/GGd1Y4GVVTTTWeIaqTlPVvbg7vCFx4o0HblXVjapaCdwdCDsRZ/h/UNU9qvoUrnk1Qpgy3A3s\nUdWNwPPAsXF07ME1x/VS1Z2qOiNewVR1g6o+rarbVXUrrpZ/arz4uLuSZar6F1WtU9UPgKeB8+Pk\n/5qqdsaZ/Pm4JtBlQKmqdlbV12Mkux93gXspgY5Wixl/OCY1Mt0GoDRiTgl4wp+g+5ao8PW4NuNo\neuIMapPfV6w4Ee3rA2li5ROmffFiYKGqzvXfHwUuFJHCEGkjlEaVNdF4yg8C6/2BZwMXxoW4Gl6Z\niOSLyG0i8qmIbMH96SHExSwB61Q1+Jhrf+C6qItzX1y7cSwmAXgD/idwpoick2B/1YH17UC7OOdN\nLw4cIr08KmyV+ipkjPAwZahm/zmzHegYR+9Pcc2Hs0Rkvoh8O048RKSDiDwgIsv97/MG0Nk/4BmL\n/rgHQIM6L4LYr0ETkZU+ThfgEdxdTX+gSkTujBH/98BRuOajbHlfZWM9plGY8YdAVRv7o7yDq1F9\nrYkSXiV2bWc8rolmu48zXET6BCMEtC/CtVsfkI+I5OFuw18LoeMS4BARqRaRauBOnLmOTaIsyRA0\n/kpgTNRFo52qrgIuxD0YeDquX6Xcp4mMOI/1596Oa3qIEG0q0WkqcTXt4P47qOpjsYTHOGcKgENj\nxU2SKg4cKdcvKqy3iEic8FBlCHO+q2q1ql6hqr2A7wF/FJEBcaJfh+vAHaGqJcAX/fZ4v08lrtky\nqLOjqn4/jpY+uGbVV3ylaRJwpU93bTCuiPwKN9PAmaq6paFyNhdN8JhGYcafRlS1BvglcK+IfM3X\nfApFZIyI3J5EVr8CThaRW0Wkq4gUi8gPcUb8M7+vV3GdYc+KyFARKfDxJorIt33N5nrgFyJyoYi0\nE5EewEO4UUb/E9if+PDgchLOuIbjbv+PxdWa/k7yzT2N4X7gVhHp7wV2F5FxPqwYd4HdgDPz30Sl\nXQMcErVtLu5uJV9ERpO46QHgQWCiiIzwc1MVichXRKQ4OqKIDPK/cXv/e38LZ3axmhyS5Qng5yLS\nxV/kfxgIewfXN/Ejv9+v436vpMvQECJyfqCSsQln3pGmrOjjXYwbeLBZRLoCN0VlFx3/BeAwEbnY\nl6NQRE5ooFlwKK5TG+B4XB9LtOaf4yoJp6vqhgYL2Yox408zqnoHcC3wC9yohkrc0NfnksjjE2Ak\nrt13Ga5mdy5wlqq+FYh6HjANNxqjBtfxNwx3N4CqTsE111yDM8kFuE6uU6L+CCfj/qjB5TvAP1T1\nI1/bq1bVauAu4Gz/hw7DZhGpDSzXNpwE/H6mAi+LyFZcR2/kKfDJuCaNVb5MM6PS/gkY7JsNIsf9\nauCruHbhi2jg91DV2cAVwD04o1uC6wiPheD6GdbifvOrgW+o6vtx4ifDr3Bl/Qx4GdcfENG4G/i6\n17URN6LmmUaWoSFOAN4VkVrc73K1+qlZcGV/xB/v8bjO9va45sSZuI7uIHfh+ms2icjdvh/gTOAC\n3DDUavZ3tMdjKPC+v9sZhBsZFc1vcHdASwLn338kW/DWQIPj+A3DMIzWhdX4DcMwcgwzfsMwjBzD\njN8wDCPHMOM3DMPIMcLMx9/slJaWanl5eaZl7GPbtm0UFRVlWkajMO2ZwbQ3Py1VN6RG+5w5c9Zr\nyEnastL4y8vLmT37c8NwM0ZFRUWLmhY3iGnPDKa9+WmpuiE12kUk9IzG1tRjGMnw8MNw0EHwm99A\nXUNTMBlGdmLGbxhhqK+Hn/8cLr8c2rWDG2+EL34RPv0008oMI2nM+A2jIXbuhPPPh9tugwkTnNk/\n+igsXAhDhkAWNUsaRhjM+A2jIf73f+GZZ+COO+D++6GwEC68EObNgw4d4D//M9MKDSMpzPgNIxG7\nd8Ndd8Fpp8G110Jw4su+fd22F1+0Wr/RojDjN4xEPPYYrFoFP/lJ7PAf/AA6d4Zbb21eXYbRBMz4\nDSMeqvDf/w1HHQWjR8eOU1ICV18Nzz0HH33UvPoMo5GY8RtGPF58ET7+2NX2D3i3SRQ/+hF07Ai/\n/W3zaTOMJmDGbxjx+P3voXdvuOCCxPG6doUrr4QpU2Dx4ubRZhhNwIzfMGIxZw5Mnw4//jG0adNw\n/Guugbw8eOih9GszjCZixm8YsZg8Gdq2hSuuCBe/rAzOOAOefNL1DRhGFmPGbxjR1NfD00/DmDHQ\nqVP4dOPHw7JlNrTTyHrM+A0jmpkz3RDO885LLt24ce7hrieeSI8uw0gRZvyGEc1TT7l2/a9+Nbl0\nXbrAmWc647fmHiOLMeM3jCD19c74zzrLjdFPlvHjYcUKShYuTL02w0gRZvyGEeS996CyMvlmngjn\nnANt2tB9+vTU6jKMFGLGbxhBnnrKtdOfc07j0nfuDGedRffXX3d3D4aRhZjxG0YEVTcc84wznIE3\nlvHjabduneskNowsxIzfMCLMmQPLlze+mSfCOedQX1jo7h4MIwsx4zeMCM88A/n5blhmUygpYdNx\nx8HUqTa6x8hKzPgNI8LUqXDqqW7unSay4eST3Zu6/v3vFAgzjNRixm8Y4Ex6/vzGd+pGseHkk93K\n1Kkpyc8wUokZv2EAPP+8+0z2oa047OreHY4/3ozfyEpCGb+IjBaRRSKyRERuiBEuInK3D58nIsf7\n7X1FZLqILBCR+SJydaoLYBgpYepU98KVQw5JXZ7nnAPvvANr16YuT8NIAQ0av4jkA/cCY4DBwDdF\nZHBUtDHAQL9MAO7z2+uA61R1MHAicGWMtIaRWTZtgjfeSFkzzz7OOcd17k6bltp8DaOJhKnxDweW\nqOpSVd0NPA5ED3sYB0xWx0ygs4j0VNUqVX0fQFW3AguB3inUbxhN55//hL17U9bMs49jj4U+fay5\nx8g6CkLE6Q1UBr6vBEaEiNMbqIpsEJFy4Djg3Vg7EZEJuLsFysrKqKioCCGteaitrc0qPclg2htm\n8EMP0blLF97evh1StL/a2loqXn+dgUOH0uOf/+Stl1+mPswLXbKAlnrOtFTdkAHtqppwAc4DHgp8\nvxi4JyrOC8DIwPfXgGGB7x2BOcDXG9qfqjJ06FDNJqZPn55pCY3GtDfArl2qJSWq3/lOSrPdp/3F\nF1VB9f/+L6X5p5OWes60VN2qqdEOzNYQ/qqqoZp6VgF9A9/7+G2h4ohIIfA08KiqPhP6imQYzcEb\nb8CWLalv348wapR7Eftzz6Unf8NoBGGM/z1goIgcLCJtgAuA6EbLqcAlfnTPiUCNqlaJiAB/Ahaq\n6p0pVW4YqeC556BdOzj99PTk37YtnH22209dXXr2YRhJ0qDxq2odcBXwEq5z9glVnS8iE0Vkoo82\nDVgKLAEeBH7gt5+Caxr6kojM9cvYVBfCMBrF3r3uFYtjx0KHDunbz3nnwbp18Oab6duHYSRBmM5d\nVHUaztyD2+4PrCtwZYx0MwBpokbDSA9vvQXV1XD++endz5gx7sLy1FNw2mnp3ZdhhMCe3DVylyef\ndM08Z5+d3v106ODM/5lnbI5+Iysw4zdyk/p618wzZozrfE03553n7i7efjv9+zKMBjDjN3KTt96C\nqqr0N/NE+MpXXEevzdFvZAFm/EZu8uST+0fcNAfFxTB6tLvLsOYeI8OY8Ru5R7CZp7i4+fZ77rmw\nciXMmtV8+zSMGJjxG7nH22/D6tXN18wT4atfdS9yf/LJ5t2vYURhxm/kHlOmuGaeVE/K1hCdO7um\npb/+FXbvbt59G0YAM34jt9i2zRnv17/evM08ESZMcA9zPfts8+/bMDxm/EZu8dhjUFMDP/hBw3HT\nwZlnQnk5PPBAZvZvGJjxG7mEKvzxj3D00XDKKZnRkJcHV1wB06fD4sWZ0WDkPGb8Ru4waxZ88IGr\n7UsGZxK5/HIoKIAHH8ycBiOnMeM3coc//tE9pXvRRZnV0bOnmwb64Ydh167MajFyEjN+IzdYv96N\n5rnkksx06kbzve85Tc/YKyqM5seM38gN/vIXV7v+/vczrcRx+ulwyCFw5532JK/R7JjxG62fmhr4\n/e/hS1+Co47KtBpHXh788pcwezY88USm1Rg5hhm/0fr5r/9yzSq//32mlRzIt74Fxx4LN9wAO3dm\nWo2RQ5jxG62bRYvgrrvgO9+B44/PtJoDyc+H//5vWL4c7rkn02qMHMKM32jdXHcdtG8Pt9ySaSWx\n+fKX3asfb7kFNmzItBojRzDjN1ovL74I//d/ri29rCzTauJz++2wdSv8539mWomRI5jxG62TVavc\nvDgDBsCPfpRpNYk58kj44Q/hvvvc2H7DSDOhXrZuGC2KzZvdS082b4bXX4c2bTKtqGF+/3uYP99d\nrMrLYdSoTCsyWjFW4zdaFzt3wte+5jp1n3kGjjsu04rCEZmnf8AAN3OozeNjpBEzfqP1sGaNM83X\nX3dNJqefnmlFydG5s+uTKCiA006DV1/NtCKjlWLGb7R8VOFvf4PBg+G119ycPBdemGlVjePgg+GV\nV6CkBM44w/VPbN+eaVVGKyOU8YvIaBFZJCJLROSGGOEiInf78HkicnzYtIbRaGpqYPJkVzu++GI4\n/HCYOzd7pmVoLEOGwPvvO9P/3/+FQYPgpptgyZJMKzNaCQ127opIPnAvcAawEnhPRKaq6oJAtDHA\nQL+MAO4DRoRMaxjxqa93Br9xo3tz1eLFsGABzJsH//qXm3+nf3+4+2433XJ+fqYVp4b27d2DZ+PG\nwW23uaePf/1rGDbMPYh2zDFwxBFumGr37tC1q2siMowQhDlThgNLVHUpgIg8DowDguY9DpisqgrM\nFJHOItITKA+RNnUccwzs2JHybIfv2OH+iC2QrNCuGn898l0V9u7dv+zaxRe3b4c9ez6fX2EhHHaY\nm+Hym9+EESMyO79+OvnSl9yycqVrznrxRdcJPGnS5+MWFkK7dm7Jz9+/iBy4RIiznhXnTCNoqboh\noL20FN55J+37C2P8vYHKwPeVuFp9Q3F6h0wLgIhMACYAlJWVUVFREULagRzeuzd5sYyiidTV1bG1\nhdamskZ7wFg02qT9d83LQ/PzIS+P+sJCdopQ0KEDdcXF7Ckpoa6khB29e7OjVy80UqadO11nbpZR\nW1vbqHM4ISee6BZV2q5fT/vKSgo3b6awpobCLVvI27173yJ79yL19Uh9/YEXWI/EuxiTRedMkrRU\n3bBfe11REZ+k+ryJQdYcJVWdBEwCGDZsmI5qzDjmNI19rqiooFF6sgDTnhlMe/PTUnXDgdp7N8P+\nwhj/KqBv4Hsfvy1MnMIQaQ3DMIxmJIzxvwcMFJGDcaZ9ARA9Vm4qcJVvwx8B1KhqlYisC5H2c8yZ\nM2e9iCxPohzpphRYn2kRjcS0ZwbT3vy0VN2QGu39w0Zs0PhVtU5ErgJeAvKBP6vqfBGZ6MPvB6YB\nY4ElwHbg8kRpQ+yze9gCNAciMltVh2VaR2Mw7ZnBtDc/LVU3NL/2UG38qjoNZ+7BbfcH1hW4Mmxa\nwzAMI3PYk7uGYRg5hhl/OGIMmm4xmPbMYNqbn5aqG5pZu2jUGF7DMAyjdWM1fsMwjBzDjN8wDCPH\nMONPgIjcLCKrRGSuX8YGwn7uZxxdJCJnZVJnPFrSzKgiskxEPvLHebbf1lVEXhGRT/xnl0zrBBCR\nP4vIWhH5OLAtrtZsOlfiaG8R57mI9BWR6SKyQETmi8jVfntWH/sEujN33FXVljgLcDNwfYztg4EP\ngbbAwcCnQH6m9UZpzPe6DgHaeL2DM60rgd5lQGnUttuBG/z6DcDvMq3Ta/kicDzwcUNas+1ciaO9\nRZznQE/geL9eDCz2GrP62CfQnbHjbjX+xjEOeFxVd6nqZ7gH14ZnWFM0+2ZVVdXdQGRm1JbEOOAR\nv/4I8LUMatmHqr4BbIzaHE9rVp0rcbTHI9u0V6nq+359K7AQN7VNVh/7BLrjkXbdZvwN80P/cpk/\nB24h481Gmk20BI1BFHhVROb4mVoBylS1yq9XA2WZkRaKeFpbyu/Qos5zESkHjgPepQUd+yjdkKHj\nnvPGLyKvisjHMZZxuBfKHAIcC1QBd2RUbOtmpKoei3upz5Ui8sVgoLp74BYx9rglafW0qPNcRDoC\nTwM/VtUtwbBsPvYxdGfsuGfNtMyZQlVDvZFbRB4EXvBfw8xYmmlagsZ9qOoq/7lWRJ7F3dquEZGe\n6ib86wmszajIxMTTmvW/g6quiaxn+3kuIoU483xUVZ/xm7P+2MfSncnjnvM1/kT4kyjC/wMiIyGm\nAheISFs/8+hAYFZz62uAfbOqikgb3MyoUzOsKSYiUiQixZF14EzcsZ4KXOqjXQr8IzMKQxFPa9af\nKy3lPBcRAf4ELFTVOwNBWX3s4+nO6HHPRO98S1mAvwIfAfP8j9EzEHYjrrd9ETAm01rj6B+LG0Hw\nKXBjpvUk0HkIbhTDh8D8iFagG/Aa8AnwKtA101q9rsdwt+Z7cO2v30mkNZvOlTjaW8R5DozENePM\nA+b6ZWy2H/sEujN23G3KBsMwjBzDmnoMwzByDDN+wzCMHMOM3zAMI8cw4zcMw8gxzPgNwzByDDN+\nwzCMHMOM3zAMI8f4/9JTPkbW4t0uAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import math\n", "from matplotlib import (cm, pyplot as plt, mlab)\n", "\n", "\n", "def visualize(word, model):\n", " \"\"\" visualize the input model for a particular word \"\"\"\n", "\n", " variance = np.array([np.diag(model.covars_[i])\n", " for i in range(model.n_components)])\n", " figures = []\n", "\n", " for parm_idx in range(len(model.means_[0])):\n", "\n", " xmin = int(min(model.means_[:, parm_idx]) - max(variance[:, parm_idx]))\n", " xmax = int(max(model.means_[:, parm_idx]) + max(variance[:, parm_idx]))\n", "\n", " fig, axs = plt.subplots(model.n_components, sharex=True, sharey=False)\n", " colours = cm.rainbow(np.linspace(0, 1, model.n_components))\n", "\n", " for i, (ax, colour) in enumerate(zip(axs, colours)):\n", " x = np.linspace(xmin, xmax, 100)\n", " mu = model.means_[i, parm_idx]\n", " sigma = math.sqrt(np.diag(model.covars_[i])[parm_idx])\n", " ax.plot(x, mlab.normpdf(x, mu, sigma), c=colour)\n", " ax.set_title(\n", " \"{} feature {} hidden state #{}\".format(word, parm_idx, i))\n", " ax.grid(True)\n", "\n", " figures.append(plt)\n", "\n", " for p in figures:\n", " p.show()\n", "\n", "\n", "visualize(my_testword, model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### ModelSelector class\n", "Review the `SelectorModel` class from the codebase found in the `my_model_selectors.py` module. It is designed to be a strategy pattern for choosing different model selectors. For the project submission in this section, subclass `SelectorModel` to implement the following model selectors. In other words, you will write your own classes/functions in the `my_model_selectors.py` module and run them from this notebook:\n", "\n", "- `SelectorCV `: Log likelihood with CV\n", "- `SelectorBIC`: BIC \n", "- `SelectorDIC`: DIC\n", "\n", "You will train each word in the training set with a range of values for the number of hidden states, and then score these alternatives with the model selector, choosing the \"best\" according to each strategy. The simple case of training with a constant value for `n_components` can be called using the provided `SelectorConstant` subclass as follow:" ] }, { "cell_type": "code", "execution_count": 83, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of states trained in model for VEGETABLE is 3\n" ] } ], "source": [ "from my_model_selectors import SelectorConstant\n", "\n", "# Experiment here with different feature sets defined in part 1\n", "training = asl.build_training(features_ground)\n", "\n", "word = 'VEGETABLE' # Experiment here with different words\n", "\n", "model = SelectorConstant(training.get_all_sequences(),\n", " training.get_all_Xlengths(),\n", " word, n_constant=3).select()\n", "\n", "print(\"Number of states trained in model for {} is {}\".format(\n", " word, model.n_components))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Cross-validation folds\n", "If we simply score the model with the Log Likelihood calculated from the feature sequences it has been trained on, we should expect that more complex models will have higher likelihoods. However, that doesn't tell us which would have a better likelihood score on unseen data. The model will likely be overfit as complexity is added. To estimate which topology model is better using only the training data, we can compare scores using cross-validation. One technique for cross-validation is to break the training set into \"folds\" and rotate which fold is left out of training. The \"left out\" fold scored. This gives us a proxy method of finding the best model to use on \"unseen data\". In the following example, a set of word sequences is broken into three folds using the [scikit-learn Kfold](http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html) class object. When you implement `SelectorCV`, you will use this technique." ] }, { "cell_type": "code", "execution_count": 84, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train fold indices:[2 3 4 5] Test fold indices:[0 1]\n", "Train fold indices:[0 1 4 5] Test fold indices:[2 3]\n", "Train fold indices:[0 1 2 3] Test fold indices:[4 5]\n" ] } ], "source": [ "from sklearn.model_selection import KFold\n", "from asl_utils import combine_sequences\n", "\n", "# Experiment here with different feature sets\n", "training = asl.build_training(features_ground)\n", "\n", "word = 'VEGETABLE' # Experiment here with different words\n", "word_sequences = training.get_word_sequences(word)\n", "split_method = KFold()\n", "\n", "for cv_train_idx, cv_test_idx in split_method.split(word_sequences):\n", " print(\"Train fold indices:{} Test fold indices:{}\".format(\n", " cv_train_idx, cv_test_idx)) # view indices of the folds\n", " X, y = combine_sequences(cv_train_idx, word_sequences)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tip:** In order to run `hmmlearn` training using the X,lengths tuples on the new folds, subsets must be combined based on the indices given for the folds. A helper utility has been provided in the `asl_utils` module named `combine_sequences` for this purpose." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Scoring models with other criterion\n", "Scoring model topologies with **BIC** balances fit and complexity within the training set for each word. In the BIC equation, a penalty term penalizes complexity to avoid overfitting, so that it is not necessary to also use cross-validation in the selection process. There are a number of references on the internet for this criterion. These [slides](http://www2.imm.dtu.dk/courses/02433/doc/ch6_slides.pdf) include a formula you may find helpful for your implementation.\n", "\n", "The advantages of scoring model topologies with **DIC** over BIC are presented by Alain Biem in this [reference](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.58.6208&rep=rep1&type=pdf) (also found [here](https://pdfs.semanticscholar.org/ed3d/7c4a5f607201f3848d4c02dd9ba17c791fc2.pdf)). DIC scores the discriminant ability of a training set for one word against competing words. Instead of a penalty term for complexity, it provides a penalty if model liklihoods for non-matching words are too similar to model likelihoods for the correct word in the word set." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Model Selection Implementation Submission\n", "Implement `SelectorCV`, `SelectorBIC`, and `SelectorDIC` classes in the `my_model_selectors.py` module. Run the selectors on the following five words. Then answer the questions about your results.\n", "\n", "**Tip:** The `hmmlearn` library may not be able to train or score all models. Implement try/except contructs as necessary to eliminate non-viable models from consideration." ] }, { "cell_type": "code", "execution_count": 85, "metadata": { "collapsed": true }, "outputs": [], "source": [ "words_to_train = ['FISH', 'BOOK', 'VEGETABLE', 'FUTURE', 'JOHN']\n", "import timeit" ] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training complete for FISH with 11 states with time 0.23446067998884246 seconds\n", "Training complete for BOOK with 6 states with time 2.337669961998472 seconds\n", "Training complete for VEGETABLE with 2 states with time 0.9285565570171457 seconds\n", "Training complete for FUTURE with 2 states with time 2.1497781430080067 seconds\n", "Training complete for JOHN with 12 states with time 24.004929588001687 seconds\n" ] } ], "source": [ "# TODO: Implement SelectorCV in my_model_selector.py\n", "from my_model_selectors import SelectorCV\n", "\n", "# Experiment here with different feature sets defined in part 1\n", "training = asl.build_training(features_ground)\n", "sequences = training.get_all_sequences()\n", "Xlengths = training.get_all_Xlengths()\n", "\n", "cv_model_list = []\n", "\n", "for word in words_to_train:\n", "\n", " start = timeit.default_timer()\n", " model = SelectorCV(sequences,\n", " Xlengths,\n", " word,\n", " min_n_components=2,\n", " max_n_components=15,\n", " random_state=14).select()\n", " end = timeit.default_timer() - start\n", "\n", " if model is not None:\n", " print(\"Training complete for {} with {} states with time {} seconds\".\n", " format(word, model.n_components, end))\n", " \n", " cv_model_list.append(model)\n", " \n", " else:\n", " print(\"Training failed for {}\".format(word))\n", " " ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training complete for FISH with 5 states with time 0.1925643530266825 seconds\n", "Training complete for BOOK with 8 states with time 1.1851199230004568 seconds\n", "Training complete for VEGETABLE with 9 states with time 0.39815885000280105 seconds\n", "Training complete for FUTURE with 9 states with time 1.2749047129764222 seconds\n", "Training complete for JOHN with 13 states with time 12.217969883000478 seconds\n" ] } ], "source": [ "# TODO: Implement SelectorBIC in module my_model_selectors.py\n", "from my_model_selectors import SelectorBIC\n", "\n", "# Experiment here with different feature sets defined in part 1\n", "training = asl.build_training(features_ground)\n", "sequences = training.get_all_sequences()\n", "Xlengths = training.get_all_Xlengths()\n", "\n", "\n", "bic_model_list = []\n", "\n", "for word in words_to_train:\n", " start = timeit.default_timer()\n", "\n", " model = SelectorBIC(sequences,\n", " Xlengths,\n", " word,\n", " min_n_components=2,\n", " max_n_components=15,\n", " random_state=14, verbose=False).select()\n", "\n", " end = timeit.default_timer() - start\n", "\n", " if model is not None:\n", " print(\"Training complete for {} with {} states with time {} seconds\".format(\n", " word, model.n_components, end))\n", " \n", " bic_model_list.append(model)\n", "\n", " else:\n", " print(\"Training failed for {}\".format(word))" ] }, { "cell_type": "code", "execution_count": 88, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training complete for FISH with 3 states with time 0.4397394340194296 seconds\n", "Training complete for BOOK with 15 states with time 2.3636560129816644 seconds\n", "Training complete for VEGETABLE with 15 states with time 1.5425242250203155 seconds\n", "Training complete for FUTURE with 15 states with time 2.397094385989476 seconds\n", "Training complete for JOHN with 15 states with time 13.0246704040037 seconds\n" ] } ], "source": [ "# TODO: Implement SelectorDIC in module my_model_selectors.py\n", "from my_model_selectors import SelectorDIC\n", "\n", "# Experiment here with different feature sets defined in part 1\n", "training = asl.build_training(features_ground)\n", "sequences = training.get_all_sequences()\n", "Xlengths = training.get_all_Xlengths()\n", "\n", "dic_model_list = []\n", "\n", "for word in words_to_train:\n", " start = timeit.default_timer()\n", " model = SelectorDIC(sequences,\n", " Xlengths,\n", " word,\n", " min_n_components=2,\n", " max_n_components=15,\n", " random_state=14).select()\n", "\n", " end = timeit.default_timer() - start\n", "\n", " if model is not None:\n", " print(\"Training complete for {} with {} states with time {} seconds\".format(\n", " word, model.n_components, end))\n", " \n", " dic_model_list.append(model)\n", "\n", " else:\n", " print(\"Training failed for {}\".format(word))" ] }, { "cell_type": "code", "execution_count": 92, "metadata": {}, "outputs": [], "source": [ "from io import StringIO" ] }, { "cell_type": "code", "execution_count": 105, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAERCAYAAABrWly6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFllJREFUeJzt3X+UXGWd5/H3lxBoIDHLEqYNE5ZkVyQQkCS0gJtBkuWH\n/NBFPLjIQUdGxuaAAq4rI46sMLMw6ygHMi6cHbOrB9bVBA3RVRBEIB1+BSEdkkASI4QfayBACAuk\nwQ4kfvePLmIISbq7qrqq+8n7dU5Oqm49Vc+36nZ9+vZz731uZCaSpKFvl2YXIEmqDwNdkgphoEtS\nIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVIhdG9nZ6NGjc9y4cY3ssqFef/119tprr2aXoSq4\n7oa20tdfZ2fnS5m5b2/tGhro48aNY+HChY3ssqE6OjqYNm1as8tQFVx3Q1vp6y8inulLO4dcJKkQ\nBrokFcJAl6RCNHQMfVveeustVq9eTXd3d7NLqdmoUaNYsWLFgPfT0tLC2LFjGT58+ID3JWnoaHqg\nr169mpEjRzJu3Dgiotnl1GT9+vWMHDlyQPvITNatW8fq1asZP378gPYlaWhp+pBLd3c3++yzz5AP\n80aJCPbZZ58i/qKRVF9ND3TAMO8nPy9J2zIoAl2SVLumj6Fvbdylt9b19Z7+5ql9anfVVVfxox/9\niGHDhrHLLrvw3e9+l6OOOqqutexIR0cHV199NbfcckvD+pSKsWYxXHFa4/q74tXG9dUPgy7Qm2HB\nggXccsstLFq0iN13352XXnqJN998s9llSVK/OOQCrFmzhtGjR7P77rsDMHr0aPbbbz86Ozs59thj\nOeKII/jIRz7CmjVrAHjiiSc4/vjjOfzww5kyZQqrVq0iM7nssss49NBDOeyww7jpppuAP52SfMYZ\nZzBhwgTOPvtsMhOA22+/nQkTJjBlyhTmzp27uZ758+czadIkJk2axOTJk1m/fn2DPxFJQ5GBDpx4\n4on8/ve/5/3vfz8XXHAB8+fP56233uLCCy9kzpw5dHZ28rnPfY6vf/3rAJx99tl84QtfYMmSJTzw\nwAOMGTOGuXPn8uijj7JkyRLuvPNOLrnkks2/AB555BFmzJjB8uXLefLJJ7n//vvp7u7m85//PL/4\nxS/o7Ozk+eef31zP1VdfzfXXX8/ixYu599572WOPPZryuUgaWhxyAUaMGEFnZyf33nsv8+bN48wz\nz+Syyy7jscce44QTTgBg06ZNjBkzhvXr1/Pss89y+umnAz0n+QDcd999nHHGGQwbNozW1laOPfZY\nHn74Yd7znvdw5JFHMnbsWAAmTZrE008/zYgRIxg/fjwHHnggAJ/+9KeZOXMmAFOnTuXLX/4yZ599\nNp/4xCc2P1eSdsRArxg2bBjTpk1j2rRpHHbYYVx//fVMnDiRBQsWvKNdNcMfbw/lvN3Pxo0bd9j+\n0ksv5dRTT+WXv/wlU6dO5Ve/+hUTJkzod7+Sdi4OuQArV67k8ccf33x/8eLFHHzwwaxdu3ZzoL/1\n1lssW7aMkSNHMnbsWH72s58BsGHDBt544w2OOeYYbr75ZjZt2sTatWu55557OPLII7fb54QJE3j6\n6adZtWoVALNmzdr82KpVqzjssMP46le/ygc/+EF++9vfDsTbllSYXrfQI+L7wEeBFzPz0MqybwMf\nA94EVgF/lZmv1KOgvh5mWE9dXV1ceOGFvPLKK+y66668733vY+bMmbS3t3PRRRfx6quvsnHjRr70\npS8xceJEfvCDH3DeeefxjW98g+HDh/OTn/yE008/nfnz53P44YcTEXzrW9/ive9973bDuKWlhZkz\nZ3Lqqaey5557cswxx2ze+p8xYwbz5s1jl112YeLEiZx88smN/DgkDVHx9hEX220Q8WGgC/hfWwT6\nicDdmbkxIv4RIDO/2ltnbW1tufUFLlasWMHBBx9cZfmDSyPmcnlbSZ/bYFD6BRJK1zFrBtNWXt64\nDht8HHpEdGZmW2/teh1yycx7gJe3WnZHZr49EPwg4F47SWqyeuwU/Rxw0/YejIh2oB2gtbWVjo6O\ndzw+atSoYo6z3rRpU8PeS3d397s+S1Wvq6vLz3MI69p9PzoO+rvGdThIf1ZqCvSI+DqwEfjh9tpk\n5kxgJvQMuWz9Z+2KFSsaNkwx0Bo55NLS0sLkyZMb0tfOwCGXoa3hQy5nFXbqf0ScQ8/O0uOyt4F4\nSdKAqyrQI+Ik4G+AYzPzjfqWJEmqRq87RSNiFrAAOCgiVkfEucB1wEjg1xGxOCL+eYDrlCT1otct\n9Mw8axuLvzcAtfS4YlSdX69vY13bmj53wYIFtLe3s+eee+7wuTNmzKC9vb0e1UpS1TxTlHdOn7t0\n6VLuvPNO9t9/f2bMmMEbb/Q+otTXdpI0kAx0tj197pw5c3juueeYPn0606dPB+D888+nra2NiRMn\ncvnlPXvUv/Od72xud+qpPWe53nHHHXzoQx9iypQpfPKTn6SrqwvomaPlkEMO4QMf+ABf+cpXmvBO\nJZXMQGfb0+dedNFF7LfffsybN4958+YBPcMyCxcuZOnSpcyfP5+lS5e+o92tt97KSy+9xJVXXsmd\nd97JokWLaGtr45prrmHdunX89Kc/ZdmyZSxdupTLLrusye9aUmkMdP40fe7MmTPZd999OfPMM7nh\nhhve1e7HP/4xU6ZMYfLkySxbtozly5e/q82DDz7I8uXLmTp1KpMmTeLGG2/kmWeeYdSoUbS0tHDu\nuecyd+7cXsflJam/nD63Yuvpc2+88cZ3PP7UU09x9dVX8/DDD7P33ntzzjnn0N3d/a7XyUxOOOGE\nd8ye+LaHHnqIu+66izlz5nDddddx9913D9j7kbTzcQudbU+fe8ABBzBy5MjNp/K/9tpr7LXXXowa\nNYoXXniB2267bXP7LdsdffTR3H///TzxxBMAvP766/zud7+jq6uLV199lVNOOYVrr72WJUuWNPAd\nStoZDL4t9CZcTXt70+fOmjWLk046afMY+eTJk5kwYQL7778/U6dO3fz89vZ2TjrpJFpbW7nnnnu4\n4YYbOOuss9iwYQMAV155JSNHjuS0006ju7ubzOSaa65p+PuUVLZep8+tJ6fPrZ+SPrfBwLlchjan\nz+3hkIskFcJAl6RCDIpAd7LG/vHzkrQtTQ/0lpYW1q1bZ0j1UWaybt06Wlpaml2KpEGm6Ue5jB07\nltWrV7N27dpml1Kz7u7uhgRtS0sLY8d61T9J79T0QB8+fDjjx49vdhl10dHR4VWEJDVN04dcJEn1\nYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1Iheg30iPh+RLwYEY9tsexf\nRsSvI+Lxyv97D2yZkqTe9GUL/QbgpK2WXQrclZkHAndV7kuSmqjXQM/Me4CXt1p8GnBj5faNwMfr\nXJckqZ+qnW2xNTPXVG4/D7Rur2FEtAPtAK2trXR0dFTZ5eDX1dVV9PsrmetuaOvafT86Dvq7xnU4\nSH9Wap4+NzMzIrZ7dYrMnAnMhJ6LRJd8IV4vNDx0ue6GtoZfJPqsxl4kuq+qPcrlhYgYA1D5/8X6\nlSRJqka1gf5z4LOV258F/k99ypEkVasvhy3OAhYAB0XE6og4F/gmcEJEPA4cX7kvSWqiXsfQM/Os\n7Tx0XJ1rkSTVwDNFJakQBrokFcJAl6RCGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXC\nQJekQhjoklQIA12SCmGgS1Ihar4EnbawZjFccVrj+rticF4GS1JzuIUuSYUw0CWpEAa6JBXCQJek\nQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFqCnQI+I/RsSyiHgsImZFREu9CpMk9U/VgR4Rfw5cBLRl\n5qHAMOBT9SpMktQ/tQ657ArsERG7AnsCz9VekiSpGpGZ1T854mLgKuAPwB2ZefY22rQD7QCtra1H\nzJ49u+r+Bruul19kxIYG/k4bM6lxfRWuq6uLESNGNLsMVan079706dM7M7Ott3ZVB3pE7A3cDJwJ\nvAL8BJiTmf97e89pa2vLhQsXVtXfUNAxawbTVl7euA6dbbFuOjo6mDZtWrPLUJVK/+5FRJ8CvZYh\nl+OBpzJzbWa+BcwF/m0NrydJqkEtgf5/gaMjYs+ICOA4YEV9ypIk9VfVgZ6ZvwHmAIuARyuvNbNO\ndUmS+qmmKxZl5uVAAweuJEnb45miklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEIY\n6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVIiarlgkFWPNYrjitMb11+Cr\nxmvn4Ba6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKkRNgR4R/yIi5kTEbyNiRUR8qF6FSZL6p9bj\n0P8JuD0zz4iI3YA961CTJKkKVQd6RIwCPgycA5CZbwJv1qcsSVJ/RWZW98SIScBMYDlwONAJXJyZ\nr2/Vrh1oB2htbT1i9uzZNRU8mHW9/CIjNjzXuA7HTGpcX4Vz3Q1tpa+/6dOnd2ZmW2/tagn0NuBB\nYGpm/iYi/gl4LTP/8/ae09bWlgsXLqyqv6GgY9YMpq28vHEdFnz6+LhLb21ofzcc/rjrbggr/bsX\nEX0K9Fp2iq4GVmfmbyr35wBTang9SVINqg70zHwe+H1EHFRZdBw9wy+SpCao9SiXC4EfVo5weRL4\nq9pLkiRVo6ZAz8zFQK/jOpKkgeeZopJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RC\nGOiSVAgDXZIKYaBLUiEMdEkqhIEuSYUw0CWpEAa6JBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSB\nLkmFMNAlqRAGuiQVouZAj4hhEfFIRNxSj4IkSdWpxxb6xcCKOryOJKkGNQV6RIwFTgX+Z33KkSRV\nq9Yt9BnA3wB/rEMtkqQaRGZW98SIjwKnZOYFETEN+EpmfnQb7dqBdoDW1tYjZs+eXUO5g1vXyy8y\nYsNzjetwzKTG9dVgjz77akP7G7/HBtddHbn+6mv69OmdmdnWW7taAv2/Ap8BNgItwHuAuZn56e09\np62tLRcuXFhVf0NBx6wZTFt5eeM6vKKxX5pGGnfprQ3t74bDH3fd1ZHrr74iok+BXvWQS2Z+LTPH\nZuY44FPA3TsKc0nSwPI4dEkqxK71eJHM7AA66vFakqTquIUuSYUw0CWpEAa6JBXCQJekQhjoklQI\nA12SCmGgS1IhDHRJKoSBLkmFMNAlqRB1OfV/sGr8jG8N7U6S3sEtdEkqhIEuSYUw0CWpEAa6JBXC\nQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAGuiQVwkCXpEJUHegRsX9EzIuI5RGxLCIu\nrmdhkqT+qWX63I3Af8rMRRExEuiMiF9n5vI61SZJ6oeqt9Azc01mLqrcXg+sAP68XoVJkvonMrP2\nF4kYB9wDHJqZr231WDvQDtDa2nrE7Nmza+6vrx599tWG9QUwfo8NjNjwXOM6HDOpcX01mOtuaHP9\n1df06dM7M7Ott3Y1B3pEjADmA1dl5twdtW1ra8uFCxfW1F9/NP6KRY8zbeXljevwisZ+aRrJdTe0\nuf7qKyL6FOg1HeUSEcOBm4Ef9hbmkqSBVctRLgF8D1iRmdfUryRJUjVq2UKfCnwG+HcRsbjy75Q6\n1SVJ6qeqD1vMzPuAqGMtkqQaeKaoJBXCQJekQhjoklQIA12SCmGgS1IhDHRJKoSBLkmFMNAlqRAG\nuiQVwkCXpEIY6JJUCANdkgphoEtSIQx0SSqEgS5JhTDQJakQBrokFcJAl6RCGOiSVAgDXZIKYaBL\nUiEMdEkqhIEuSYUw0CWpEDUFekScFBErI+KJiLi0XkVJkvqv6kCPiGHA9cDJwCHAWRFxSL0KkyT1\nTy1b6EcCT2Tmk5n5JjAbOK0+ZUmS+isys7onRpwBnJSZf125/xngqMz84lbt2oH2yt2DgJXVlzvo\njQZeanYRqorrbmgrff0dkJn79tZo14GuIjNnAjMHup/BICIWZmZbs+tQ/7nuhjbXX49ahlyeBfbf\n4v7YyjJJUhPUEugPAwdGxPiI2A34FPDz+pQlSeqvqodcMnNjRHwR+BUwDPh+Zi6rW2VD004xtFQo\n193Q5vqjhp2ikqTBxTNFJakQBrokFcJAl6RCGOiSVAgDvUoR0RIR7zpzKyL2jYiWZtSkvomISyJi\nbLPrkOrNQK/ed4BjtrH8L4BrG1yL+mc/YEFE3BsRF2zrF7MGr4j4dkSct43l50XEN5tR02DhYYtV\niojOzDxiO48ty8yJja5JfRcRAXyYnhPiPg4sAWYBczNzfTNr045FRCfQlluFV0TsAizNzEObU1nz\nuYVevT138Jif6yCXPeZn5vn0TFtxLfAl4IXmVqY+2H3rMAfIzD8C0YR6Bg2Dp3ovRsSRWy+MiA8C\na5tQj6oQEYcBf0/P3P4bgK81tyL1wR8i4sCtF1aW/aEJ9QwaAz7bYsEuAX4cETcAnZVlbcBf0vNn\nvAapyhf/U5V/m+iZy//EzHyyqYWpr74B3BYRV/LO797X6Pkra6flGHoNIqIVuAB4e8xuGXBdZr7Y\nvKrUm4jYBHQAF2fmY1ssnwo8n5mrmlWb+iYiDqVno2rL7963M/PR5lXVfAa6djoRcStw6dZf/srw\nyz9k5seaU5lUG4dcqhQRjwLb+m0Y9Oxz+0CDS1Lf/dm2tuQy89GIGNf4ctQfEfELtv3dAyAz/30D\nyxlUDPTqfbTZBahqe+/gsT0aVoWqdXWzCxisDPQqZeYzWy+LiNHAum0dUqVB5eGI+Hxm/o8tF0bE\nX/OnnWwapDJz/tu33z4pLDM9sgzH0KsWEUcD3wReBv4L8AN6LlS7C/CXmXl7E8vTDlR2Zv8UeJN3\nHiWxG3B6Zj7frNrUNxFxOXAhPd+3ADYC/y0z/76phTWZgV6liFgI/C0wip6rpZycmQ9GxARgVmZO\nbmqB6lVETGeLoyQy8+5m1qO+iYgvAycD7Zn5VGXZvwb+O3B7Zu60U28Y6FWKiMWZOalye0VmHrzF\nY48Y6NLAiIhHgBMy86Wtlu8L3LEzf/c8U7R6f9zi9tZnp/lbUho4w7cOc9g8jj68CfUMGu4Urd7h\nEfEaPeN3e1RuU7nv9LnSwHmzyseK55CLpCGlcqbv69t6CGjJzJ12K91Al6RCOIYuSYUw0CWpEO4U\nVZEiYh/grsrd99IzTe7bZxMemZk79c4zlckxdBUvIq4AujLTOUBUNIdctFOJiH+IiC9ucf8fI+IL\nEXF8RMyLiNsiYmVEXF+57igRcXJELIiIRRFxU0Ts1bx3IG2fga6dzfeBzwJExDDgk8CPKo8dBZwP\nHAIcDJwWEX8GXAocl5lTgKXAxY0uWuoLx9C1U8nMJyJifeViFgcAD2Xm/6tsjD+YmU8DRMRs4C8q\nTzsEeKDSZjfgvoYXLvWBga6d0feAc4BxwHe3WL71DqWk52SV2zPzMw2pTKqBQy7aGd0MfAyYBNy5\nxfKjI+JfVYZi/gM9W+IPAMdWZvMjIvba1hXnpcHALXTtdDKzOyLuoeeC0FtOsvYQ8M/Av6En6H+e\nmRkR5wI3RcRulXZ/Czze0KKlPvCwRe10ImIXYDHw8cx8srLseOCLmfnxphYn1cAhF+1UKjtDV9Ez\nLv5ks+uR6sktdEkqhFvoklQIA12SCmGgS1IhDHRJKoSBLkmF+P/fKtKoMbj0WgAAAABJRU5ErkJg\ngg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "table = StringIO(\"\"\"Type,Word,States,Seconds\n", "CV,FISH,11,0.23446068\n", "CV,BOOK,6,2.337669962\n", "CV,VEGETABLE,2,0.928556557\n", "CV,FUTURE,2,2.149778143\n", "CV,JOHN,12,24.004929588\n", "BIC,FISH,5,0.192564353\n", "BIC,BOOK,8,1.185119923\n", "BIC,VEGETABLE,9,0.39815885\n", "BIC,FUTURE,9,1.274904713\n", "BIC,JOHN,13,12.217969883\n", "DIC,FISH,3,0.439739434\n", "DIC,BOOK,15,2.363656013\n", "DIC,VEGETABLE,15,1.542524225\n", "DIC,FUTURE,15,2.397094386\n", "DIC,JOHN,15,13.024670404\n", "\"\"\")\n", "df = pd.read_csv(table, sep=\",\")\n", "df = df.groupby(\"Type\").mean()[['Seconds', \"States\"]]\n", "\n", "df.plot(kind='bar', grid=True);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Question 2:** Compare and contrast the possible advantages and disadvantages of the various model selectors implemented.\n", "\n", "**Answer 2:**\n", "\n", "| Type | Pros | Cons |\n", "|:----:|:--------------------------------------------------------:|:---------------------------------------------------------------------------:|\n", "| BIC | Fast and good perforamce by utilizing a penalty term | As the number of features increases, it's difficult to find a right answer |\n", "| CV | The least number of states implies it's probably generalized well or at least not overfitting | Slow because of cross validations |\n", "| DIC | Its performance is likely to be better than BIC because of high number of parameters | Too many hidden states implies possible overfitting and more complex model |" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Model Selector Unit Testing\n", "Run the following unit tests as a sanity check on the implemented model selectors. The test simply looks for valid interfaces but is not exhaustive. However, the project should not be submitted if these tests don't pass." ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "....\n", "----------------------------------------------------------------------\n", "Ran 4 tests in 28.669s\n", "\n", "OK\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from asl_test_model_selectors import TestSelectors\n", "suite = unittest.TestLoader().loadTestsFromModule(TestSelectors())\n", "unittest.TextTestRunner().run(suite)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## PART 3: Recognizer\n", "The objective of this section is to \"put it all together\". Using the four feature sets created and the three model selectors, you will experiment with the models and present your results. Instead of training only five specific words as in the previous section, train the entire set with a feature set and model selector strategy. \n", "### Recognizer Tutorial\n", "##### Train the full training set\n", "The following example trains the entire set with the example `features_ground` and `SelectorConstant` features and model selector. Use this pattern for you experimentation and final submission cells.\n", "\n" ] }, { "cell_type": "code", "execution_count": 112, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The autoreload extension is already loaded. To reload it, use:\n", " %reload_ext autoreload\n", "Number of word models returned = 112\n" ] } ], "source": [ "# autoreload for automatically reloading changes made in my_model_selectors and my_recognizer\n", "%load_ext autoreload\n", "%autoreload 2\n", "\n", "from my_model_selectors import SelectorConstant\n", "\n", "def train_all_words(features, model_selector):\n", " training = asl.build_training(features) # Experiment here with different feature sets defined in part 1\n", " sequences = training.get_all_sequences()\n", " Xlengths = training.get_all_Xlengths()\n", " model_dict = {}\n", " for word in training.words:\n", " model = model_selector(sequences, Xlengths, word, \n", " n_constant=3).select()\n", " model_dict[word]=model\n", " return model_dict\n", "\n", "models = train_all_words(features_ground, SelectorConstant)\n", "print(\"Number of word models returned = {}\".format(len(models)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Load the test set\n", "The `build_test` method in `ASLdb` is similar to the `build_training` method already presented, but there are a few differences:\n", "- the object is type `SinglesData` \n", "- the internal dictionary keys are the index of the test word rather than the word itself\n", "- the getter methods are `get_all_sequences`, `get_all_Xlengths`, `get_item_sequences` and `get_item_Xlengths`" ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of test set items: 178\n", "Number of test set sentences: 40\n" ] } ], "source": [ "test_set = asl.build_test(features_ground)\n", "print(\"Number of test set items: {}\".format(test_set.num_items))\n", "print(\"Number of test set sentences: {}\".format(len(test_set.sentences_index)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Recognizer Implementation Submission\n", "For the final project submission, students must implement a recognizer following guidance in the `my_recognizer.py` module. Experiment with the four feature sets and the three model selection methods (that's 12 possible combinations). You can add and remove cells for experimentation or run the recognizers locally in some other way during your experiments, but retain the results for your discussion. For submission, you will provide code cells of **only three** interesting combinations for your discussion (see questions below). At least one of these should produce a word error rate of less than 60%, i.e. WER < 0.60 . \n", "\n", "**Tip:** The hmmlearn library may not be able to train or score all models. Implement try/except contructs as necessary to eliminate non-viable models from consideration." ] }, { "cell_type": "code", "execution_count": 202, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# TODO implement the recognize method in my_recognizer\n", "from my_recognizer import recognize\n", "from asl_utils import show_errors\n", "from itertools import combinations\n", "from tqdm import tqdm" ] }, { "cell_type": "code", "execution_count": 200, "metadata": {}, "outputs": [], "source": [ "def get_WER(guesses, test_set):\n", " \"\"\" Get WER \"\"\"\n", " S = 0\n", " N = len(test_set.wordlist)\n", " num_test_words = len(test_set.wordlist)\n", " \n", " for word_id in range(num_test_words):\n", " if guesses[word_id] != test_set.wordlist[word_id]:\n", " S += 1\n", " \n", " return S / N\n", "\n", "def run_train(feat, selector):\n", " \n", " models = train_all_words(feat, selector)\n", " test_set = asl.build_test(feat)\n", " \n", " probabilities, guesses = recognize(models, test_set)\n", " \n", " return get_WER(guesses, test_set)" ] }, { "cell_type": "code", "execution_count": 201, "metadata": {}, "outputs": [], "source": [ "features_list = [\n", " \"features_ground\",\n", " \"features_delta\",\n", " \"features_all\",\n", " \"features_custom\",\n", " \"features_ground\",\n", " \"features_norm\",\n", " \"features_polar\" \n", "]\n", "model_selector_list = [\"SelectorCV\", \"SelectorDIC\", \"SelectorBIC\"]" ] }, { "cell_type": "code", "execution_count": 205, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\n", " 0%| | 0/7 [00:00\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
featureselectorWER
0features_allSelectorBIC0.438202
1features_allSelectorDIC0.494382
2features_allSelectorCV0.516854
3features_groundSelectorCV0.528090
4features_groundSelectorCV0.528090
\n", "" ], "text/plain": [ " feature selector WER\n", "0 features_all SelectorBIC 0.438202\n", "1 features_all SelectorDIC 0.494382\n", "2 features_all SelectorCV 0.516854\n", "3 features_ground SelectorCV 0.528090\n", "4 features_ground SelectorCV 0.528090" ] }, "execution_count": 221, "metadata": {}, "output_type": "execute_result" } ], "source": [ "result_frame = pd.DataFrame(sorted(result_list, key=lambda x: x[-1]), columns=[\"feature\", \"selector\", \"WER\"])\n", "result_frame.head()" ] }, { "cell_type": "code", "execution_count": 222, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "**** WER = 0.43820224719101125\n", "Total correct: 100 out of 178\n", "Video Recognized Correct\n", "=====================================================================================================\n", " 2: JOHN WRITE *ARRIVE JOHN WRITE HOMEWORK\n", " 7: JOHN *PEOPLE GO CAN JOHN CAN GO CAN\n", " 12: JOHN CAN *WHAT CAN JOHN CAN GO CAN\n", " 21: JOHN *ARRIVE *JOHN *JOHN *CAR *CAR *FUTURE *MARY JOHN FISH WONT EAT BUT CAN EAT CHICKEN\n", " 25: JOHN LIKE *LOVE IX IX JOHN LIKE IX IX IX\n", " 28: JOHN LIKE IX *LIKE IX JOHN LIKE IX IX IX\n", " 30: JOHN LIKE *MARY IX IX JOHN LIKE IX IX IX\n", " 36: MARY *JOHN *IX IX *MARY *MARY MARY VEGETABLE KNOW IX LIKE CORN1\n", " 40: JOHN IX *CORN MARY *MARY JOHN IX THINK MARY LOVE\n", " 43: JOHN *JOHN BUY HOUSE JOHN MUST BUY HOUSE\n", " 50: *SOMETHING-ONE JOHN BUY CAR *JOHN FUTURE JOHN BUY CAR SHOULD\n", " 54: JOHN *FUTURE *FUTURE BUY HOUSE JOHN SHOULD NOT BUY HOUSE\n", " 57: JOHN *JOHN *IX MARY JOHN DECIDE VISIT MARY\n", " 67: JOHN FUTURE *MARY BUY HOUSE JOHN FUTURE NOT BUY HOUSE\n", " 71: JOHN *FUTURE VISIT MARY JOHN WILL VISIT MARY\n", " 74: *IX *MARY *MARY MARY JOHN NOT VISIT MARY\n", " 77: *JOHN BLAME MARY ANN BLAME MARY\n", " 84: *JOHN *JOHN *JOHN BOOK IX-1P FIND SOMETHING-ONE BOOK\n", " 89: *FUTURE IX *IX *IX IX NEW COAT JOHN IX GIVE MAN IX NEW COAT\n", " 90: JOHN *IX IX *IX *MARY BOOK JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n", " 92: JOHN *IX IX *WOMAN WOMAN BOOK JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n", " 100: POSS NEW CAR BREAK-DOWN POSS NEW CAR BREAK-DOWN\n", " 105: JOHN *SEE JOHN LEG\n", " 107: JOHN *IX *JOHN *MARY *MARY JOHN POSS FRIEND HAVE CANDY\n", " 108: *JOHN ARRIVE WOMAN ARRIVE\n", " 113: IX CAR *IX *JOHN *BOX IX CAR BLUE SUE BUY\n", " 119: *MARY *BUY1 IX CAR *IX SUE BUY IX CAR BLUE\n", " 122: JOHN *GIVE1 BOOK JOHN READ BOOK\n", " 139: JOHN *BUY1 WHAT *GIVE1 BOOK JOHN BUY WHAT YESTERDAY BOOK\n", " 142: JOHN BUY YESTERDAY WHAT BOOK JOHN BUY YESTERDAY WHAT BOOK\n", " 158: LOVE JOHN WHO LOVE JOHN WHO\n", " 167: JOHN IX *IX LOVE MARY JOHN IX SAY LOVE MARY\n", " 171: JOHN *JOHN BLAME JOHN MARY BLAME\n", " 174: *JOHN *GIVE1 GIVE1 *IX *BLAME PEOPLE GROUP GIVE1 JANA TOY\n", " 181: JOHN *BOX JOHN ARRIVE\n", " 184: *IX *IX *GIVE1 TEACHER APPLE ALL BOY GIVE TEACHER APPLE\n", " 189: JOHN *IX *JOHN BOX JOHN GIVE GIRL BOX\n", " 193: JOHN *IX *IX BOX JOHN GIVE GIRL BOX\n", " 199: *JOHN *ARRIVE *JOHN LIKE CHOCOLATE WHO\n", " 201: JOHN *MAN *IX *WOMAN BUY HOUSE JOHN TELL MARY IX-1P BUY HOUSE\n" ] } ], "source": [ "# TODO Choose a feature set and model selector\n", "features = features_all # change as needed\n", "model_selector = SelectorBIC # change as needed\n", "\n", "# TODO Recognize the test set and display the result with the show_errors method\n", "models = train_all_words(features, model_selector)\n", "test_set = asl.build_test(features)\n", "probabilities, guesses = recognize(models, test_set)\n", "show_errors(guesses, test_set)" ] }, { "cell_type": "code", "execution_count": 223, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "**** WER = 0.4943820224719101\n", "Total correct: 90 out of 178\n", "Video Recognized Correct\n", "=====================================================================================================\n", " 2: JOHN *ARRIVE *ARRIVE JOHN WRITE HOMEWORK\n", " 7: JOHN CAN *IX CAN JOHN CAN GO CAN\n", " 12: JOHN CAN *WHAT CAN JOHN CAN GO CAN\n", " 21: JOHN *JOHN *JOHN *JOHN *CAR *CAR *FUTURE *MARY JOHN FISH WONT EAT BUT CAN EAT CHICKEN\n", " 25: JOHN *JOHN *LOVE IX *JOHN JOHN LIKE IX IX IX\n", " 28: JOHN *MARY IX *JOHN IX JOHN LIKE IX IX IX\n", " 30: JOHN *MARY *MARY IX IX JOHN LIKE IX IX IX\n", " 36: MARY *JOHN *IX IX *MARY *MARY MARY VEGETABLE KNOW IX LIKE CORN1\n", " 40: JOHN IX *CORN MARY *MARY JOHN IX THINK MARY LOVE\n", " 43: JOHN *JOHN BUY HOUSE JOHN MUST BUY HOUSE\n", " 50: *JOHN JOHN BUY CAR *JOHN FUTURE JOHN BUY CAR SHOULD\n", " 54: JOHN *FUTURE *MARY BUY HOUSE JOHN SHOULD NOT BUY HOUSE\n", " 57: JOHN *JOHN *IX MARY JOHN DECIDE VISIT MARY\n", " 67: JOHN FUTURE *MARY BUY HOUSE JOHN FUTURE NOT BUY HOUSE\n", " 71: JOHN *FUTURE VISIT MARY JOHN WILL VISIT MARY\n", " 74: *IX *MARY *MARY MARY JOHN NOT VISIT MARY\n", " 77: *JOHN BLAME MARY ANN BLAME MARY\n", " 84: *JOHN *JOHN *JOHN BOOK IX-1P FIND SOMETHING-ONE BOOK\n", " 89: *MARY IX *IX *IX IX *ARRIVE *ARRIVE JOHN IX GIVE MAN IX NEW COAT\n", " 90: JOHN *IX IX *IX *MARY BOOK JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n", " 92: JOHN *IX IX *IX *JOHN BOOK JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n", " 100: *IX NEW CAR BREAK-DOWN POSS NEW CAR BREAK-DOWN\n", " 105: JOHN *POSS JOHN LEG\n", " 107: JOHN *IX *JOHN *MARY *MARY JOHN POSS FRIEND HAVE CANDY\n", " 108: *JOHN ARRIVE WOMAN ARRIVE\n", " 113: IX CAR *IX *JOHN *JOHN IX CAR BLUE SUE BUY\n", " 119: *MARY *BUY1 IX *JOHN *IX SUE BUY IX CAR BLUE\n", " 122: JOHN *GIVE1 BOOK JOHN READ BOOK\n", " 139: JOHN *ARRIVE WHAT *JOHN BOOK JOHN BUY WHAT YESTERDAY BOOK\n", " 142: JOHN BUY YESTERDAY WHAT BOOK JOHN BUY YESTERDAY WHAT BOOK\n", " 158: LOVE JOHN WHO LOVE JOHN WHO\n", " 167: JOHN IX *IX LOVE MARY JOHN IX SAY LOVE MARY\n", " 171: JOHN *JOHN BLAME JOHN MARY BLAME\n", " 174: *JOHN *GIVE1 GIVE1 *IX *JOHN PEOPLE GROUP GIVE1 JANA TOY\n", " 181: JOHN *JOHN JOHN ARRIVE\n", " 184: *IX BOY *GIVE1 TEACHER *IX ALL BOY GIVE TEACHER APPLE\n", " 189: JOHN *MARY *JOHN BOX JOHN GIVE GIRL BOX\n", " 193: JOHN *IX *IX BOX JOHN GIVE GIRL BOX\n", " 199: *JOHN *ARRIVE *JOHN LIKE CHOCOLATE WHO\n", " 201: JOHN *MAN *IX *JOHN BUY HOUSE JOHN TELL MARY IX-1P BUY HOUSE\n" ] } ], "source": [ "# TODO Choose a feature set and model selector\n", "features = features_all # change as needed\n", "model_selector = SelectorDIC # change as needed\n", "\n", "# TODO Recognize the test set and display the result with the show_errors method\n", "models = train_all_words(features, model_selector)\n", "test_set = asl.build_test(features)\n", "probabilities, guesses = recognize(models, test_set)\n", "show_errors(guesses, test_set)" ] }, { "cell_type": "code", "execution_count": 224, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "**** WER = 0.5168539325842697\n", "Total correct: 86 out of 178\n", "Video Recognized Correct\n", "=====================================================================================================\n", " 2: JOHN *BOOK *ARRIVE JOHN WRITE HOMEWORK\n", " 7: JOHN *PEOPLE *HAVE *HAVE JOHN CAN GO CAN\n", " 12: JOHN CAN *GO1 CAN JOHN CAN GO CAN\n", " 21: JOHN *VIDEOTAPE *VISIT *MARY *CAR *CAR *FUTURE *MARY JOHN FISH WONT EAT BUT CAN EAT CHICKEN\n", " 25: JOHN *MARY *MARY *JOHN IX JOHN LIKE IX IX IX\n", " 28: JOHN *MARY IX *JOHN IX JOHN LIKE IX IX IX\n", " 30: JOHN LIKE *MARY *MARY *MARY JOHN LIKE IX IX IX\n", " 36: MARY *JOHN *IX *GIVE *MARY *MARY MARY VEGETABLE KNOW IX LIKE CORN1\n", " 40: JOHN *MARY *CORN MARY *MARY JOHN IX THINK MARY LOVE\n", " 43: JOHN *FUTURE BUY HOUSE JOHN MUST BUY HOUSE\n", " 50: *JOHN *FRANK BUY CAR *JOHN FUTURE JOHN BUY CAR SHOULD\n", " 54: JOHN SHOULD *MARY BUY HOUSE JOHN SHOULD NOT BUY HOUSE\n", " 57: *MARY *JOHN *IX MARY JOHN DECIDE VISIT MARY\n", " 67: JOHN FUTURE *MARY BUY HOUSE JOHN FUTURE NOT BUY HOUSE\n", " 71: JOHN *FUTURE *BLAME MARY JOHN WILL VISIT MARY\n", " 74: *MARY *MARY *MARY MARY JOHN NOT VISIT MARY\n", " 77: *JOHN BLAME MARY ANN BLAME MARY\n", " 84: *JOHN *ARRIVE *FUTURE BOOK IX-1P FIND SOMETHING-ONE BOOK\n", " 89: *MARY IX *IX *IX IX NEW COAT JOHN IX GIVE MAN IX NEW COAT\n", " 90: JOHN *IX IX *ALL *MARY BOOK JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n", " 92: JOHN *IX IX *IX *IX BOOK JOHN GIVE IX SOMETHING-ONE WOMAN BOOK\n", " 100: *IX NEW CAR BREAK-DOWN POSS NEW CAR BREAK-DOWN\n", " 105: JOHN *WHO JOHN LEG\n", " 107: JOHN *JOHN *HAVE *MARY *JOHN JOHN POSS FRIEND HAVE CANDY\n", " 108: *MARY ARRIVE WOMAN ARRIVE\n", " 113: *JOHN CAR BLUE *JOHN *ARRIVE IX CAR BLUE SUE BUY\n", " 119: *PREFER *BUY1 IX CAR *JOHN SUE BUY IX CAR BLUE\n", " 122: JOHN *HOUSE BOOK JOHN READ BOOK\n", " 139: JOHN *BUY1 WHAT YESTERDAY BOOK JOHN BUY WHAT YESTERDAY BOOK\n", " 142: JOHN BUY YESTERDAY WHAT BOOK JOHN BUY YESTERDAY WHAT BOOK\n", " 158: LOVE *MARY WHO LOVE JOHN WHO\n", " 167: JOHN *SOMETHING-ONE *MARY LOVE MARY JOHN IX SAY LOVE MARY\n", " 171: *MARY *JOHN BLAME JOHN MARY BLAME\n", " 174: *GIVE1 *GIVE1 GIVE1 *JOHN *CAN PEOPLE GROUP GIVE1 JANA TOY\n", " 181: JOHN ARRIVE JOHN ARRIVE\n", " 184: *IX BOY *GIVE1 TEACHER APPLE ALL BOY GIVE TEACHER APPLE\n", " 189: JOHN *JOHN *JOHN *ARRIVE JOHN GIVE GIRL BOX\n", " 193: JOHN *SOMETHING-ONE *IX BOX JOHN GIVE GIRL BOX\n", " 199: *JOHN CHOCOLATE *MARY LIKE CHOCOLATE WHO\n", " 201: JOHN *SHOULD *IX *IX BUY HOUSE JOHN TELL MARY IX-1P BUY HOUSE\n" ] } ], "source": [ "# TODO Choose a feature set and model selector\n", "features = features_all # change as needed\n", "model_selector = SelectorCV # change as needed\n", "\n", "# TODO Recognize the test set and display the result with the show_errors method\n", "models = train_all_words(features, model_selector)\n", "test_set = asl.build_test(features)\n", "probabilities, guesses = recognize(models, test_set)\n", "show_errors(guesses, test_set)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Question 3:** Summarize the error results from three combinations of features and model selectors. What was the \"best\" combination and why? What additional information might we use to improve our WER? For more insight on improving WER, take a look at the introduction to Part 4.\n", "\n", "**Answer 3:**\n", "\n", "For the best performance, \n", "\n", "I have to use every features including my custom features and SelectorBIC.\n", "The reason for such high performance is the standardization of features because each person has different body shapes hence different signals.\n", "\n", "However, the best score is still low and I believe it can improve further by standardizing every other features which I didn't because of the time limit." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Recognizer Unit Tests\n", "Run the following unit tests as a sanity check on the defined recognizer. The test simply looks for some valid values but is not exhaustive. However, the project should not be submitted if these tests don't pass." ] }, { "cell_type": "code", "execution_count": 176, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "..\n", "----------------------------------------------------------------------\n", "Ran 2 tests in 16.490s\n", "\n", "OK\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 176, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from asl_test_recognizer import TestRecognize\n", "suite = unittest.TestLoader().loadTestsFromModule(TestRecognize())\n", "unittest.TextTestRunner().run(suite)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## PART 4: (OPTIONAL) Improve the WER with Language Models\n", "We've squeezed just about as much as we can out of the model and still only get about 50% of the words right! Surely we can do better than that. Probability to the rescue again in the form of [statistical language models (SLM)](https://en.wikipedia.org/wiki/Language_model). The basic idea is that each word has some probability of occurrence within the set, and some probability that it is adjacent to specific other words. We can use that additional information to make better choices.\n", "\n", "##### Additional reading and resources\n", "- [Introduction to N-grams (Stanford Jurafsky slides)](https://web.stanford.edu/class/cs124/lec/languagemodeling.pdf)\n", "- [Speech Recognition Techniques for a Sign Language Recognition System, Philippe Dreuw et al](https://www-i6.informatik.rwth-aachen.de/publications/download/154/Dreuw--2007.pdf) see the improved results of applying LM on *this* data!\n", "- [SLM data for *this* ASL dataset](ftp://wasserstoff.informatik.rwth-aachen.de/pub/rwth-boston-104/lm/)\n", "\n", "##### Optional challenge\n", "The recognizer you implemented in Part 3 is equivalent to a \"0-gram\" SLM. Improve the WER with the SLM data provided with the data set in the link above using \"1-gram\", \"2-gram\", and/or \"3-gram\" statistics. The `probabilities` data you've already calculated will be useful and can be turned into a pandas DataFrame if desired (see next cell). \n", "Good luck! Share your results with the class!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# create a DataFrame of log likelihoods for the test word items\n", "df_probs = pd.DataFrame(data=probabilities)\n", "df_probs.head()" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.1" }, "nbpresent": { "slides": { "0a2d4faf-9fb8-4cee-853b-ed68b90f3f8a": { "id": "0a2d4faf-9fb8-4cee-853b-ed68b90f3f8a", "prev": null, "regions": { "3fb9ce83-fbb2-4995-832a-f8f400734ad3": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "1dbb9346-179b-4835-b430-6369d88f1a1b", "part": "whole" }, "id": "3fb9ce83-fbb2-4995-832a-f8f400734ad3" } } }, "1519a4fa-1588-4644-98de-9c43bf0aceb5": { "id": "1519a4fa-1588-4644-98de-9c43bf0aceb5", "prev": "8a712017-49b7-449f-8264-43a032ace902", "regions": { "29546121-ed11-44b7-8144-0c44e874098f": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "365590a4-6963-4812-a1cf-688f7b6bb9ff", "part": "whole" }, "id": "29546121-ed11-44b7-8144-0c44e874098f" } } }, "176eaccb-15dd-455d-bf07-504213e7aa01": { "id": "176eaccb-15dd-455d-bf07-504213e7aa01", "prev": "de6b30f4-2463-4901-92ed-aabad78e5e0f", "regions": { "1542aa9e-dc55-4b90-adef-bf5181872b42": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "5c242050-c1f7-4b3b-8103-2ea9d71a40dc", "part": "whole" }, "id": "1542aa9e-dc55-4b90-adef-bf5181872b42" } } }, "19091b36-b0e7-49b1-b501-ec05937e0da9": { "id": "19091b36-b0e7-49b1-b501-ec05937e0da9", "prev": "1983c02e-fb99-4c05-a728-e0c0ad7c06d8", "regions": { "6529a31c-8d45-425c-b1d7-d0ac6fca6a32": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "e766909d-9421-4aaf-9fb1-bc90d27e49e3", "part": "whole" }, "id": "6529a31c-8d45-425c-b1d7-d0ac6fca6a32" } } }, "1983c02e-fb99-4c05-a728-e0c0ad7c06d8": { "id": "1983c02e-fb99-4c05-a728-e0c0ad7c06d8", "prev": "176eaccb-15dd-455d-bf07-504213e7aa01", "regions": { "1c4e605d-7f22-4f30-b3fb-74b2937e7a4a": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "4d217204-e5c0-4568-bd30-12c2e41b681d", "part": "whole" }, "id": "1c4e605d-7f22-4f30-b3fb-74b2937e7a4a" } } }, "212b111f-4527-459c-8297-1db5580ee5c9": { "id": "212b111f-4527-459c-8297-1db5580ee5c9", "prev": "76898529-e49e-4663-8d02-8261dfe1d94b", "regions": { "2e4bd280-3cd6-47d0-9c81-17737b24053b": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "0c316996-9933-4b3d-82ec-259518dc8bc9", "part": "whole" }, "id": "2e4bd280-3cd6-47d0-9c81-17737b24053b" } } }, "23a7337f-a0cf-4ed4-baa9-ec06bfdc0579": { "id": "23a7337f-a0cf-4ed4-baa9-ec06bfdc0579", "prev": "e76e9a02-54c1-4ec9-80fb-c611ed398122", "regions": { "b5721d20-d6f8-4ddb-a5aa-eb16f0cc8893": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "313015a2-b5a9-4136-a8ea-5d011e47d840", "part": "whole" }, "id": "b5721d20-d6f8-4ddb-a5aa-eb16f0cc8893" } } }, "732f1952-ee54-46fb-8067-099512824296": { "id": "732f1952-ee54-46fb-8067-099512824296", "prev": "0a2d4faf-9fb8-4cee-853b-ed68b90f3f8a", "regions": { "f31d4597-08ad-4c46-ad52-4bd2d775c624": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "aadfec52-27ca-4541-8920-fa9253d51827", "part": "whole" }, "id": "f31d4597-08ad-4c46-ad52-4bd2d775c624" } } }, "76898529-e49e-4663-8d02-8261dfe1d94b": { "id": "76898529-e49e-4663-8d02-8261dfe1d94b", "prev": "19091b36-b0e7-49b1-b501-ec05937e0da9", "regions": { "ec1746fc-aec9-4a7c-8225-9e9ac8d45889": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "b3e539be-84e2-49ce-a183-31cfc5c7ce7c", "part": "whole" }, "id": "ec1746fc-aec9-4a7c-8225-9e9ac8d45889" } } }, "8a712017-49b7-449f-8264-43a032ace902": { "id": "8a712017-49b7-449f-8264-43a032ace902", "prev": "bed9e696-630e-4747-be1c-bc3737ba992f", "regions": { "1faab517-cd16-4c63-bb01-a67246749d7a": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "3f14ddf0-4145-4687-9c33-712c3c32520f", "part": "whole" }, "id": "1faab517-cd16-4c63-bb01-a67246749d7a" } } }, "90af992d-eb6d-4496-b2d2-6aa9a95b6a61": { "id": "90af992d-eb6d-4496-b2d2-6aa9a95b6a61", "prev": "732f1952-ee54-46fb-8067-099512824296", "regions": { "4f448bec-5be9-4553-88ae-e35ed7612f25": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "c445fbfb-b8ab-4e9a-8d13-12231a1c588f", "part": "whole" }, "id": "4f448bec-5be9-4553-88ae-e35ed7612f25" } } }, "bed9e696-630e-4747-be1c-bc3737ba992f": { "id": "bed9e696-630e-4747-be1c-bc3737ba992f", "prev": "23a7337f-a0cf-4ed4-baa9-ec06bfdc0579", "regions": { "ac1513f0-404f-492b-8b42-0313e9a753b0": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "18dd2eee-8b6c-4a5e-9539-132d00a7c7e1", "part": "whole" }, "id": "ac1513f0-404f-492b-8b42-0313e9a753b0" } } }, "de6b30f4-2463-4901-92ed-aabad78e5e0f": { "id": "de6b30f4-2463-4901-92ed-aabad78e5e0f", "prev": "e36b4639-be8c-46f7-a8c9-bcfb134f9fd0", "regions": { "55ec36e0-362f-4fd3-8060-7cee056039aa": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "c3cf461e-4c9e-4dec-99d2-07bfa79cbe23", "part": "whole" }, "id": "55ec36e0-362f-4fd3-8060-7cee056039aa" } } }, "e36b4639-be8c-46f7-a8c9-bcfb134f9fd0": { "id": "e36b4639-be8c-46f7-a8c9-bcfb134f9fd0", "prev": "1519a4fa-1588-4644-98de-9c43bf0aceb5", "regions": { "4c1e9714-9ba0-45fd-8a2f-ef80a5c85c2e": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "6534d4dc-125f-47e6-a022-cf1e0d277174", "part": "whole" }, "id": "4c1e9714-9ba0-45fd-8a2f-ef80a5c85c2e" } } }, "e76e9a02-54c1-4ec9-80fb-c611ed398122": { "id": "e76e9a02-54c1-4ec9-80fb-c611ed398122", "prev": "90af992d-eb6d-4496-b2d2-6aa9a95b6a61", "regions": { "9491b84d-193b-40ff-9321-d21eb1ba88d4": { "attrs": { "height": 0.8, "width": 0.8, "x": 0.1, "y": 0.1 }, "content": { "cell": "b64ec10e-fa9d-4f3f-907f-6799611ed6b1", "part": "whole" }, "id": "9491b84d-193b-40ff-9321-d21eb1ba88d4" } } } }, "themes": {} } }, "nbformat": 4, "nbformat_minor": 1 }