{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Structured & Time Series Data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook walks through an implementation of a deep learning model for structured time series data using Keras. We’ll use the dataset from Kaggle’s [Rossmann Store Sales competition](https://www.kaggle.com/c/rossmann-store-sales). The steps outlined below are inspired by (and partially based on) lesson 3 of Jeremy Howard’s [fast.ai course](http://course.fast.ai) where he builds a model for the Rossman dataset using PyTorch and the fast.ai library.\n", "\n", "The focus here is on implementing a deep learning model for structured data. I’ve skipped a bunch of pre-processing steps that are specific to this particular dataset but don’t reflect general principles about applying deep learning to tabular datasets. If you’re interested, you’ll find complete step-by-step instructions on creating the “joined” dataset in [this notebook](https://github.com/fastai/fastai/blob/master/courses/dl1/lesson3-rossman.ipynb). With that, let’s get started!\n", "\n", "First we need to get a few imports out of the way. All of these should come standard with an Anaconda install. I’m also specifying the path where I’ve pre-saved the “joined” dataset that we’ll use as a starting point (created from running the first few sections of the above-referenced notebook).\n", "\n", "(As an aside, I’m using [Paperspace](https://www.paperspace.com) to run this notebook. If you’re not familiar with it, Paperspace is a cloud service that lets you rent GPU instances much cheaper than AWS. It’s a great way to get started if you don’t have your own hardware.)\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import datetime\n", "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "from sklearn.decomposition import PCA\n", "from sklearn.preprocessing import LabelEncoder, StandardScaler\n", "\n", "PATH = '/home/paperspace/data/rossmann/'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Read the data file into a pandas dataframe and take a peek at the data to see what we’re working with." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(844338, 93)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data = pd.read_feather(f'{PATH}joined')\n", "data.shape" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
01234
index01234
Store12345
DayOfWeek55555
Date2015-07-31 00:00:002015-07-31 00:00:002015-07-31 00:00:002015-07-31 00:00:002015-07-31 00:00:00
Sales526360648314139954822
Customers5556258211498559
Open11111
Promo11111
StateHolidayFalseFalseFalseFalseFalse
SchoolHoliday11111
Year20152015201520152015
Month77777
Week3131313131
Day3131313131
Dayofweek44444
Dayofyear212212212212212
Is_month_endTrueTrueTrueTrueTrue
Is_month_startFalseFalseFalseFalseFalse
Is_quarter_endFalseFalseFalseFalseFalse
Is_quarter_startFalseFalseFalseFalseFalse
Is_year_endFalseFalseFalseFalseFalse
Is_year_startFalseFalseFalseFalseFalse
Elapsed14383008001438300800143830080014383008001438300800
StoreTypecaaca
Assortmentaaaca
CompetitionDistance12705701413062029910
CompetitionOpenSinceMonth9111294
CompetitionOpenSinceYear20082007200620092015
Promo201100
Promo2SinceWeek1131411
..................
Min_Sea_Level_PressurehPa10151017101710141016
Max_VisibilityKm3110311010
Mean_VisibilityKm1510141010
Min_VisibilitykM1010101010
Max_Wind_SpeedKm_h2414142314
Mean_Wind_SpeedKm_h111151611
Max_Gust_SpeedKm_hNaNNaNNaNNaNNaN
Precipitationmm00000
CloudCover14264
EventsFogFogFogNoneNone
WindDirDegrees13309354282290
StateNameHessenThueringenNordrheinWestfalenBerlinSachsen
CompetitionOpenSince2008-09-15 00:00:002007-11-15 00:00:002006-12-15 00:00:002009-09-15 00:00:002015-04-15 00:00:00
CompetitionDaysOpen2510281531502145107
CompetitionMonthsOpen242424243
Promo2Since1900-01-01 00:00:002010-03-29 00:00:002011-04-04 00:00:001900-01-01 00:00:001900-01-01 00:00:00
Promo2Days01950157900
Promo2Weeks0252500
AfterSchoolHoliday00000
BeforeSchoolHoliday00000
AfterStateHoliday5767576757
BeforeStateHoliday00000
AfterPromo00000
BeforePromo00000
SchoolHoliday_bw55555
StateHoliday_bw00000
Promo_bw55555
SchoolHoliday_fw11111
StateHoliday_fw00000
Promo_fw11111
\n", "

93 rows × 5 columns

\n", "
" ], "text/plain": [ " 0 1 \\\n", "index 0 1 \n", "Store 1 2 \n", "DayOfWeek 5 5 \n", "Date 2015-07-31 00:00:00 2015-07-31 00:00:00 \n", "Sales 5263 6064 \n", "Customers 555 625 \n", "Open 1 1 \n", "Promo 1 1 \n", "StateHoliday False False \n", "SchoolHoliday 1 1 \n", "Year 2015 2015 \n", "Month 7 7 \n", "Week 31 31 \n", "Day 31 31 \n", "Dayofweek 4 4 \n", "Dayofyear 212 212 \n", "Is_month_end True True \n", "Is_month_start False False \n", "Is_quarter_end False False \n", "Is_quarter_start False False \n", "Is_year_end False False \n", "Is_year_start False False \n", "Elapsed 1438300800 1438300800 \n", "StoreType c a \n", "Assortment a a \n", "CompetitionDistance 1270 570 \n", "CompetitionOpenSinceMonth 9 11 \n", "CompetitionOpenSinceYear 2008 2007 \n", "Promo2 0 1 \n", "Promo2SinceWeek 1 13 \n", "... ... ... \n", "Min_Sea_Level_PressurehPa 1015 1017 \n", "Max_VisibilityKm 31 10 \n", "Mean_VisibilityKm 15 10 \n", "Min_VisibilitykM 10 10 \n", "Max_Wind_SpeedKm_h 24 14 \n", "Mean_Wind_SpeedKm_h 11 11 \n", "Max_Gust_SpeedKm_h NaN NaN \n", "Precipitationmm 0 0 \n", "CloudCover 1 4 \n", "Events Fog Fog \n", "WindDirDegrees 13 309 \n", "StateName Hessen Thueringen \n", "CompetitionOpenSince 2008-09-15 00:00:00 2007-11-15 00:00:00 \n", "CompetitionDaysOpen 2510 2815 \n", "CompetitionMonthsOpen 24 24 \n", "Promo2Since 1900-01-01 00:00:00 2010-03-29 00:00:00 \n", "Promo2Days 0 1950 \n", "Promo2Weeks 0 25 \n", "AfterSchoolHoliday 0 0 \n", "BeforeSchoolHoliday 0 0 \n", "AfterStateHoliday 57 67 \n", "BeforeStateHoliday 0 0 \n", "AfterPromo 0 0 \n", "BeforePromo 0 0 \n", "SchoolHoliday_bw 5 5 \n", "StateHoliday_bw 0 0 \n", "Promo_bw 5 5 \n", "SchoolHoliday_fw 1 1 \n", "StateHoliday_fw 0 0 \n", "Promo_fw 1 1 \n", "\n", " 2 3 \\\n", "index 2 3 \n", "Store 3 4 \n", "DayOfWeek 5 5 \n", "Date 2015-07-31 00:00:00 2015-07-31 00:00:00 \n", "Sales 8314 13995 \n", "Customers 821 1498 \n", "Open 1 1 \n", "Promo 1 1 \n", "StateHoliday False False \n", "SchoolHoliday 1 1 \n", "Year 2015 2015 \n", "Month 7 7 \n", "Week 31 31 \n", "Day 31 31 \n", "Dayofweek 4 4 \n", "Dayofyear 212 212 \n", "Is_month_end True True \n", "Is_month_start False False \n", "Is_quarter_end False False \n", "Is_quarter_start False False \n", "Is_year_end False False \n", "Is_year_start False False \n", "Elapsed 1438300800 1438300800 \n", "StoreType a c \n", "Assortment a c \n", "CompetitionDistance 14130 620 \n", "CompetitionOpenSinceMonth 12 9 \n", "CompetitionOpenSinceYear 2006 2009 \n", "Promo2 1 0 \n", "Promo2SinceWeek 14 1 \n", "... ... ... \n", "Min_Sea_Level_PressurehPa 1017 1014 \n", "Max_VisibilityKm 31 10 \n", "Mean_VisibilityKm 14 10 \n", "Min_VisibilitykM 10 10 \n", "Max_Wind_SpeedKm_h 14 23 \n", "Mean_Wind_SpeedKm_h 5 16 \n", "Max_Gust_SpeedKm_h NaN NaN \n", "Precipitationmm 0 0 \n", "CloudCover 2 6 \n", "Events Fog None \n", "WindDirDegrees 354 282 \n", "StateName NordrheinWestfalen Berlin \n", "CompetitionOpenSince 2006-12-15 00:00:00 2009-09-15 00:00:00 \n", "CompetitionDaysOpen 3150 2145 \n", "CompetitionMonthsOpen 24 24 \n", "Promo2Since 2011-04-04 00:00:00 1900-01-01 00:00:00 \n", "Promo2Days 1579 0 \n", "Promo2Weeks 25 0 \n", "AfterSchoolHoliday 0 0 \n", "BeforeSchoolHoliday 0 0 \n", "AfterStateHoliday 57 67 \n", "BeforeStateHoliday 0 0 \n", "AfterPromo 0 0 \n", "BeforePromo 0 0 \n", "SchoolHoliday_bw 5 5 \n", "StateHoliday_bw 0 0 \n", "Promo_bw 5 5 \n", "SchoolHoliday_fw 1 1 \n", "StateHoliday_fw 0 0 \n", "Promo_fw 1 1 \n", "\n", " 4 \n", "index 4 \n", "Store 5 \n", "DayOfWeek 5 \n", "Date 2015-07-31 00:00:00 \n", "Sales 4822 \n", "Customers 559 \n", "Open 1 \n", "Promo 1 \n", "StateHoliday False \n", "SchoolHoliday 1 \n", "Year 2015 \n", "Month 7 \n", "Week 31 \n", "Day 31 \n", "Dayofweek 4 \n", "Dayofyear 212 \n", "Is_month_end True \n", "Is_month_start False \n", "Is_quarter_end False \n", "Is_quarter_start False \n", "Is_year_end False \n", "Is_year_start False \n", "Elapsed 1438300800 \n", "StoreType a \n", "Assortment a \n", "CompetitionDistance 29910 \n", "CompetitionOpenSinceMonth 4 \n", "CompetitionOpenSinceYear 2015 \n", "Promo2 0 \n", "Promo2SinceWeek 1 \n", "... ... \n", "Min_Sea_Level_PressurehPa 1016 \n", "Max_VisibilityKm 10 \n", "Mean_VisibilityKm 10 \n", "Min_VisibilitykM 10 \n", "Max_Wind_SpeedKm_h 14 \n", "Mean_Wind_SpeedKm_h 11 \n", "Max_Gust_SpeedKm_h NaN \n", "Precipitationmm 0 \n", "CloudCover 4 \n", "Events None \n", "WindDirDegrees 290 \n", "StateName Sachsen \n", "CompetitionOpenSince 2015-04-15 00:00:00 \n", "CompetitionDaysOpen 107 \n", "CompetitionMonthsOpen 3 \n", "Promo2Since 1900-01-01 00:00:00 \n", "Promo2Days 0 \n", "Promo2Weeks 0 \n", "AfterSchoolHoliday 0 \n", "BeforeSchoolHoliday 0 \n", "AfterStateHoliday 57 \n", "BeforeStateHoliday 0 \n", "AfterPromo 0 \n", "BeforePromo 0 \n", "SchoolHoliday_bw 5 \n", "StateHoliday_bw 0 \n", "Promo_bw 5 \n", "SchoolHoliday_fw 1 \n", "StateHoliday_fw 0 \n", "Promo_fw 1 \n", "\n", "[93 rows x 5 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.head().T.head(93)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The data consists of ~800,000 records with a variety of features used to predict sales at a given store on a given day. As mentioned before, we’re skipping over details about where these features came from as it’s not the focus of this notebook, but you can find more info through the links above. Next we’ll define variables that group our features into continuous and categorical buckets. This is very important as neural networks (really anything other than tree models) do not natively handle categorical data well." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "target = 'Sales'\n", "cat_vars = ['Store', 'DayOfWeek', 'Year', 'Month', 'Day', 'StateHoliday', 'CompetitionMonthsOpen',\n", " 'Promo2Weeks', 'StoreType', 'Assortment', 'PromoInterval', 'CompetitionOpenSinceYear', 'Promo2SinceYear',\n", " 'State', 'Week', 'Events', 'Promo_fw', 'Promo_bw', 'StateHoliday_fw', 'StateHoliday_bw',\n", " 'SchoolHoliday_fw', 'SchoolHoliday_bw']\n", "cont_vars = ['CompetitionDistance', 'Max_TemperatureC', 'Mean_TemperatureC', 'Min_TemperatureC',\n", " 'Max_Humidity', 'Mean_Humidity', 'Min_Humidity', 'Max_Wind_SpeedKm_h', \n", " 'Mean_Wind_SpeedKm_h', 'CloudCover', 'trend', 'trend_DE',\n", " 'AfterStateHoliday', 'BeforeStateHoliday', 'Promo', 'SchoolHoliday']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Set some reasonable default values for missing information so our pre-processing steps won’t fail." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "data = data.set_index('Date')\n", "data[cat_vars] = data[cat_vars].fillna(value='')\n", "data[cont_vars] = data[cont_vars].fillna(value=0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can do something with the categorical variables. The simplest first step is to use scikit-learn’s LabelEncoder class to transform the raw category values (many of which are plain text) into unique integers, where each integer maps to a distinct value in that category. The code block below saves the fitted encoders (we’ll need them later) and prints out the unique labels that each encoder found." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Store: [ 1 2 3 ... 1113 1114 1115]\n", "DayOfWeek: [1 2 3 4 5 6 7]\n", "Year: [2013 2014 2015]\n", "Month: [ 1 2 3 4 5 6 7 8 9 10 11 12]\n", "Day: [ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24\n", " 25 26 27 28 29 30 31]\n", "StateHoliday: [False True]\n", "CompetitionMonthsOpen: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23\n", " 24]\n", "Promo2Weeks: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23\n", " 24 25]\n", "StoreType: ['a' 'b' 'c' 'd']\n", "Assortment: ['a' 'b' 'c']\n", "PromoInterval: ['' 'Feb,May,Aug,Nov' 'Jan,Apr,Jul,Oct' 'Mar,Jun,Sept,Dec']\n", "CompetitionOpenSinceYear: [1900 1961 1990 1994 1995 1998 1999 2000 2001 2002 2003 2004 2005 2006\n", " 2007 2008 2009 2010 2011 2012 2013 2014 2015]\n", "Promo2SinceYear: [1900 2009 2010 2011 2012 2013 2014 2015]\n", "State: ['BE' 'BW' 'BY' 'HB,NI' 'HE' 'HH' 'NW' 'RP' 'SH' 'SN' 'ST' 'TH']\n", "Week: [ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24\n", " 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48\n", " 49 50 51 52]\n", "Events: ['' 'Fog' 'Fog-Rain' 'Fog-Rain-Hail' 'Fog-Rain-Hail-Thunderstorm'\n", " 'Fog-Rain-Snow' 'Fog-Rain-Snow-Hail' 'Fog-Rain-Thunderstorm' 'Fog-Snow'\n", " 'Fog-Snow-Hail' 'Fog-Thunderstorm' 'Rain' 'Rain-Hail'\n", " 'Rain-Hail-Thunderstorm' 'Rain-Snow' 'Rain-Snow-Hail'\n", " 'Rain-Snow-Hail-Thunderstorm' 'Rain-Snow-Thunderstorm'\n", " 'Rain-Thunderstorm' 'Snow' 'Snow-Hail' 'Thunderstorm']\n", "Promo_fw: [0. 1. 2. 3. 4. 5.]\n", "Promo_bw: [0. 1. 2. 3. 4. 5.]\n", "StateHoliday_fw: [0. 1. 2.]\n", "StateHoliday_bw: [0. 1. 2.]\n", "SchoolHoliday_fw: [0. 1. 2. 3. 4. 5. 6. 7.]\n", "SchoolHoliday_bw: [0. 1. 2. 3. 4. 5. 6. 7.]\n" ] } ], "source": [ "encoders = {}\n", "for v in cat_vars:\n", " le = LabelEncoder()\n", " le.fit(data[v].values)\n", " encoders[v] = le\n", " data.loc[:, v] = le.transform(data[v].values)\n", " print('{0}: {1}'.format(v, le.classes_))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Split the data set into training and validation sets. To preserve the temporal nature of the data and make sure that we don’t have any information leaks, we’ll just take everything past a certain date and use that as our validation set." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "train = data[data.index < datetime.datetime(2015, 7, 1)]\n", "val = data[data.index >= datetime.datetime(2015, 7, 1)]\n", "\n", "X = train[cat_vars + cont_vars].copy()\n", "X_val = val[cat_vars + cont_vars].copy()\n", "y = train[target].copy()\n", "y_val = val[target].copy()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we can apply scaling to our continuous variables. We can once again leverage scikit-learn and use the StandardScaler class for this. The proper way to apply scaling is to “fit” the scaler on the training data and then apply the same transformation to both the training and validation data (this is why we had to split the data set in the last step)." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "scaler = StandardScaler()\n", "X.loc[:, cont_vars] = scaler.fit_transform(X[cont_vars].values)\n", "X_val.loc[:, cont_vars] = scaler.transform(X_val[cont_vars].values)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Normalize the data types that each variable is stored as. This is not strictly necessary but helps save storage space (and potentially processing time, although I’m not sure about that one)." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "for v in cat_vars:\n", " X[v] = X[v].astype('int').astype('category').cat.as_ordered()\n", " X_val[v] = X_val[v].astype('int').astype('category').cat.as_ordered()\n", "for v in cont_vars:\n", " X[v] = X[v].astype('float32')\n", " X_val[v] = X_val[v].astype('float32')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let’s take a look at where we’re at. The data should basically be ready to move into the modeling phase." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "((814150, 38), (30188, 38), (814150,), (30188,))" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X.shape, X_val.shape, y.shape, y_val.shape" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreDayOfWeekYearMonthDayStateHolidayCompetitionMonthsOpenPromo2WeeksStoreTypeAssortment...Min_HumidityMax_Wind_SpeedKm_hMean_Wind_SpeedKm_hCloudCovertrendtrend_DEAfterStateHolidayBeforeStateHolidayPromoSchoolHoliday
Date
2015-06-30012529024020...-1.9640090.047353-0.310342-2.3142231.0086590.885609-0.3810791.1591281.116479-0.476624
2015-06-301125290242500...-1.147185-1.065656-0.646876-0.5020291.0086590.885609-0.0634891.1591281.116479-0.476624
2015-06-302125290242500...-1.453494-0.397851-1.151678-1.8611751.5449900.885609-0.3810791.1591281.1164792.098092
2015-06-30312529024022...-1.453494-0.175249-0.310342-0.5020290.0253840.885609-0.0634891.1591281.116479-0.476624
2015-06-3041252902000...-1.096133-0.954355-0.310342-0.048980-0.4215590.885609-0.3810791.1591281.116479-0.476624
\n", "

5 rows × 38 columns

\n", "
" ], "text/plain": [ " Store DayOfWeek Year Month Day StateHoliday CompetitionMonthsOpen \\\n", "Date \n", "2015-06-30 0 1 2 5 29 0 24 \n", "2015-06-30 1 1 2 5 29 0 24 \n", "2015-06-30 2 1 2 5 29 0 24 \n", "2015-06-30 3 1 2 5 29 0 24 \n", "2015-06-30 4 1 2 5 29 0 2 \n", "\n", " Promo2Weeks StoreType Assortment ... Min_Humidity \\\n", "Date ... \n", "2015-06-30 0 2 0 ... -1.964009 \n", "2015-06-30 25 0 0 ... -1.147185 \n", "2015-06-30 25 0 0 ... -1.453494 \n", "2015-06-30 0 2 2 ... -1.453494 \n", "2015-06-30 0 0 0 ... -1.096133 \n", "\n", " Max_Wind_SpeedKm_h Mean_Wind_SpeedKm_h CloudCover trend \\\n", "Date \n", "2015-06-30 0.047353 -0.310342 -2.314223 1.008659 \n", "2015-06-30 -1.065656 -0.646876 -0.502029 1.008659 \n", "2015-06-30 -0.397851 -1.151678 -1.861175 1.544990 \n", "2015-06-30 -0.175249 -0.310342 -0.502029 0.025384 \n", "2015-06-30 -0.954355 -0.310342 -0.048980 -0.421559 \n", "\n", " trend_DE AfterStateHoliday BeforeStateHoliday Promo \\\n", "Date \n", "2015-06-30 0.885609 -0.381079 1.159128 1.116479 \n", "2015-06-30 0.885609 -0.063489 1.159128 1.116479 \n", "2015-06-30 0.885609 -0.381079 1.159128 1.116479 \n", "2015-06-30 0.885609 -0.063489 1.159128 1.116479 \n", "2015-06-30 0.885609 -0.381079 1.159128 1.116479 \n", "\n", " SchoolHoliday \n", "Date \n", "2015-06-30 -0.476624 \n", "2015-06-30 -0.476624 \n", "2015-06-30 2.098092 \n", "2015-06-30 -0.476624 \n", "2015-06-30 -0.476624 \n", "\n", "[5 rows x 38 columns]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X.head()" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Store category\n", "DayOfWeek category\n", "Year category\n", "Month category\n", "Day category\n", "StateHoliday category\n", "CompetitionMonthsOpen category\n", "Promo2Weeks category\n", "StoreType category\n", "Assortment category\n", "PromoInterval category\n", "CompetitionOpenSinceYear category\n", "Promo2SinceYear category\n", "State category\n", "Week category\n", "Events category\n", "Promo_fw category\n", "Promo_bw category\n", "StateHoliday_fw category\n", "StateHoliday_bw category\n", "SchoolHoliday_fw category\n", "SchoolHoliday_bw category\n", "CompetitionDistance float32\n", "Max_TemperatureC float32\n", "Mean_TemperatureC float32\n", "Min_TemperatureC float32\n", "Max_Humidity float32\n", "Mean_Humidity float32\n", "Min_Humidity float32\n", "Max_Wind_SpeedKm_h float32\n", "Mean_Wind_SpeedKm_h float32\n", "CloudCover float32\n", "trend float32\n", "trend_DE float32\n", "AfterStateHoliday float32\n", "BeforeStateHoliday float32\n", "Promo float32\n", "SchoolHoliday float32\n", "dtype: object" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X.dtypes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We now basically have two options when it comes to handling of categorical variables. The first option, which is the “traditional” way of handling categories, is to do a one-hot encoding for each category. This approach would create a binary variable for each unique value in each category, with the value being a 1 for the “correct” category and 0 for everything else. One-hot encoding works fairly well and is quite easy to do (there’s even a scikit-learn class for it), however it’s not perfect. It’s particularly challenging with high-cardinality variables because it creates a very large, very sparse array that’s hard to learn from.\n", "\n", "Fortunately there’s a better way, which is something called entity embeddings or category embeddings (I don’t think there’s a standard name for this yet). Jeremy covers it extensively in the class (also [this blog post](https://towardsdatascience.com/deep-learning-structured-data-8d6a278f3088) explains it very well). The basic idea is to create a distributed representation of the category using a vector of continuous numbers, where the length of the vector is lower than the cardinality of the category. The key insight is that this vector is learned by the network. It’s part of the optimization graph. This allows the network to model complex, non-linear interactions between categories and other features in your input. It’s quite useful, and as we’ll see at the end, these embeddings can be used in interesting ways outside of the neural network itself.\n", "\n", "In order to build a model using embeddings, we need to do some more prep work on our categories. First, let’s create a list of category names along with their cardinality.\n" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('Store', 1115),\n", " ('DayOfWeek', 7),\n", " ('Year', 3),\n", " ('Month', 12),\n", " ('Day', 31),\n", " ('StateHoliday', 2),\n", " ('CompetitionMonthsOpen', 25),\n", " ('Promo2Weeks', 26),\n", " ('StoreType', 4),\n", " ('Assortment', 3),\n", " ('PromoInterval', 4),\n", " ('CompetitionOpenSinceYear', 23),\n", " ('Promo2SinceYear', 8),\n", " ('State', 12),\n", " ('Week', 52),\n", " ('Events', 22),\n", " ('Promo_fw', 6),\n", " ('Promo_bw', 6),\n", " ('StateHoliday_fw', 3),\n", " ('StateHoliday_bw', 3),\n", " ('SchoolHoliday_fw', 8),\n", " ('SchoolHoliday_bw', 8)]" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cat_sizes = [(c, len(X[c].cat.categories)) for c in cat_vars]\n", "cat_sizes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we need to decide on the length of each embedding vector. Jeremy proposed using a simple formula: cardinality / 2, with a max of 50." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[(1115, 50),\n", " (7, 4),\n", " (3, 2),\n", " (12, 6),\n", " (31, 16),\n", " (2, 1),\n", " (25, 13),\n", " (26, 13),\n", " (4, 2),\n", " (3, 2),\n", " (4, 2),\n", " (23, 12),\n", " (8, 4),\n", " (12, 6),\n", " (52, 26),\n", " (22, 11),\n", " (6, 3),\n", " (6, 3),\n", " (3, 2),\n", " (3, 2),\n", " (8, 4),\n", " (8, 4)]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "embedding_sizes = [(c, min(50, (c + 1) // 2)) for _, c in cat_sizes]\n", "embedding_sizes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One last pre-processing step. Keras requires that each “input” into the model be fed in as a separate array, and since each embedding has its own input, we need to do some transformations to get the data in the right format." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(23, 23)" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_array = []\n", "X_val_array = []\n", "\n", "for i, v in enumerate(cat_vars):\n", " X_array.append(X.iloc[:, i])\n", " X_val_array.append(X_val.iloc[:, i])\n", "\n", "X_array.append(X.iloc[:, len(cat_vars):])\n", "X_val_array.append(X_val.iloc[:, len(cat_vars):])\n", "\n", "len(X_array), len(X_val_array)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Okay! We’re finally ready to get to the modeling part. Let’s get some imports out of the way. I’ve also defined a custom metric to calculate root mean squared percentage error, which was originally used by the Kaggle competition to score this data set." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "from keras import backend as K\n", "from keras import regularizers\n", "from keras.models import Sequential\n", "from keras.models import Model\n", "from keras.layers import Activation, BatchNormalization, Concatenate\n", "from keras.layers import Dropout, Dense, Input, Reshape\n", "from keras.layers.embeddings import Embedding\n", "from keras.optimizers import Adam\n", "from keras.callbacks import ModelCheckpoint, ReduceLROnPlateau\n", "\n", "def rmspe(y_true, y_pred):\n", " pct_var = (y_true - y_pred) / y_true\n", " return K.sqrt(K.mean(K.square(pct_var)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now for the model itself. I tried to make this a similar to Jeremy’s model as I could, although there are some slight differences. The “for” section at the top shows how to add embeddings. They then get concatenated together and we apply dropout to the unified embedding layer. Next we concatenate the output of that layer with our continuous inputs and feed the whole thing into a dense layer. From here on it’s pretty standard stuff. The only notable design choice is I omitted batch normalization because it seemed to hurt performance no matter what I did. I also increased dropout a bit from what Jeremy had in his PyTorch architecture for this data. Finally, note the inclusion of the “rmspe” function as a metric during the compile step (this will show up later during training)." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "def EmbeddingNet(cat_vars, cont_vars, embedding_sizes):\n", " inputs = []\n", " embed_layers = []\n", " for (c, (in_size, out_size)) in zip(cat_vars, embedding_sizes):\n", " i = Input(shape=(1,))\n", " o = Embedding(in_size, out_size, name=c)(i)\n", " o = Reshape(target_shape=(out_size,))(o)\n", " inputs.append(i)\n", " embed_layers.append(o)\n", "\n", " embed = Concatenate()(embed_layers)\n", " embed = Dropout(0.04)(embed)\n", "\n", " cont_input = Input(shape=(len(cont_vars),))\n", " inputs.append(cont_input)\n", "\n", " x = Concatenate()([embed, cont_input])\n", "\n", " x = Dense(1000, kernel_initializer='he_normal')(x)\n", " x = Activation('relu')(x)\n", " x = Dropout(0.1)(x)\n", "\n", " x = Dense(500, kernel_initializer='he_normal')(x)\n", " x = Activation('relu')(x)\n", " x = Dropout(0.1)(x)\n", "\n", " x = Dense(1, kernel_initializer='he_normal')(x)\n", " x = Activation('linear')(x)\n", "\n", " model = Model(inputs=inputs, outputs=x)\n", " opt = Adam(lr=0.001)\n", " model.compile(loss='mean_absolute_error', optimizer=opt, metrics=[rmspe])\n", "\n", " return model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One of the cool tricks Jeremy introduced in the class was the concept of a learning rate finder. The idea is to start with a very small learning rate and slowly increase it throughout the epoch, and monitor the loss along the way. It should end up as a curve that gives a good indication of where to set the learning rate for training. To accomplish this with Keras, I found a script on Github that implements learning rate cycling and includes a class that’s supposed to mimic Jeremy’s LR finder. We can just download a copy to the local directory." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--2018-10-04 20:20:17-- https://raw.githubusercontent.com/titu1994/keras-one-cycle/master/clr.py\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.200.133\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.200.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 22310 (22K) [text/plain]\n", "Saving to: ‘clr.py’\n", "\n", "clr.py 100%[===================>] 21.79K --.-KB/s in 0.009s \n", "\n", "2018-10-04 20:20:17 (2.35 MB/s) - ‘clr.py’ saved [22310/22310]\n", "\n" ] } ], "source": [ "!wget \"https://raw.githubusercontent.com/titu1994/keras-one-cycle/master/clr.py\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let’s set up and train the model for one epoch using the LRFinder class as a callback. It will slowly but exponentially increase the learning rate each batch and track the loss so we can plot the results." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train on 814150 samples, validate on 30188 samples\n", "Epoch 1/1\n", "814150/814150 [==============================] - 73s 90us/step - loss: 2521.7429 - rmspe: 0.4402 - val_loss: 3441.1762 - val_rmspe: 0.5088\n" ] } ], "source": [ "from clr import LRFinder\n", "\n", "lr_finder = LRFinder(num_samples=X.shape[0], batch_size=128, minimum_lr=1e-5, maximum_lr=10,\n", " lr_scale='exp', loss_smoothing_beta=0.995, verbose=False)\n", "model = EmbeddingNet(cat_vars, cont_vars, embedding_sizes)\n", "history = model.fit(x=X_array, y=y, batch_size=128, epochs=1, verbose=1, callbacks=[lr_finder],\n", " validation_data=(X_val_array, y_val), shuffle=False)" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAETCAYAAAAh/OHhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3XlcVXX+x/HXvSCIrC5cwCXNpdyVcgcxwQuKkag4OdNYmv0sWxxanLSZzFyrn2Na/qaRsXXGckGBElMUTSSXLCXCrUxJUbkoyC7b5fz+uHLkwmUzEJTP8/HoMdzvPefc770y9835rhpFURSEEEKIWtI2dgWEEELcWSQ4hBBC1IkEhxBCiDqR4BBCCFEnEhxCCCHqRIJDCCFEnUhwCAE89dRTRERENHY1hLgjSHCIRuXr68uBAwcauxqsW7eOiRMnNnY1AJg2bRqbN29u1Dps3bqVP/7xj41aB9F0SXCIu15JSUljV0HVlOoixK2S4BBN1t69e5kwYQKDBg1i6tSpnDp1Sn0uLCyMMWPG4OnpSWBgILt27VKf27p1K1OnTmXZsmUMGTKE999/X/0L+u2332bw4MH4+vqyb98+9Zzyf+XXdOyFCxd47LHH8PT0ZPr06bz55pu88sorFt/D4cOH8fHxISwsDC8vL+bPn09WVhZPP/00w4YNY/DgwTz99NOkpqYC8O677/L999+zaNEiPD09WbRoEQC//vorM2bMYMiQIQQEBLB9+3aLrxcdHc2kSZPMyj755BOeeeYZAPbt20dgYCCenp6MHDmSDz/8sNb/HmUMBgPPPPMMQ4YMQa/Xs2nTJvW5xMREJk2axAMPPMCIESNYvnw5AIWFhbzyyisMHTqUQYMGMXnyZK5evVrn1xZNhCJEIxo9erTy7bffVipPSkpShg0bpiQkJCglJSXK1q1bldGjRyuFhYWKoijK9u3bldTUVMVoNCrR0dHKgAEDFIPBoCiKomzZskXp1auX8tlnnynFxcXK9evXlS1btii9e/dWNm7cqJSUlCjr169XvLy8lNLSUkVRFOXPf/6zsmnTJvX86o79wx/+oLz11ltKYWGhcuTIEcXT01N5+eWXLb6/Q4cOKb169VLeeecdpbCwULl+/bqSkZGh7NixQ8nPz1dycnKUF154QZk9e7Z6Tvm6KIqi5OXlKT4+Pkp4eLhSXFysJCUlKUOGDFF+/vnnSq+Xn5+vDBw4UDl37pxaNmnSJGXbtm2KoiiKl5eXcuTIEUVRFCUzM1NJSkqyWO8tW7YoU6dOtfjcY489przxxhtKQUGBcuLECWXo0KHKgQMH1M8mIiJCURRFyc3NVY4dO6YoiqJ88cUXytNPP63k5+crJSUlyk8//aTk5ORYvL5o+uSOQzRJmzZt4tFHH2XAgAFYWVkxceJEWrRoQUJCAgDjxo3Dzc0NrVZLYGAgnTt3JjExUT1fp9Mxbdo0rK2tadmyJQDt27fnD3/4g3q9K1euVPlXb1XHXrp0iZ9++ok5c+ZgY2PDoEGD8PX1rfa9aLVa9fiWLVvSunVrAgICsLOzw8HBgdmzZ3PkyJEqz//mm2/o0KEDkydPxtramj59+hAQEMDOnTsrHWtnZ4efnx/btm0DIDk5mbNnz6p1tLa25syZM+Tm5uLs7EyfPn2qrXtFly9f5ocffuCVV17B1taWXr16MWXKFKKiotTrnz9/noyMDOzt7Rk4cKBanpmZyW+//YaVlRV9+/bFwcGhTq8tmg4JDtEkXbp0iY8//phBgwap/6WmppKWlgZAZGSk2ow1aNAgfvnlF65du6ae7+7uXuma7dq1U3+2s7MDID8/3+LrV3VsWloazs7OahmAh4dHte+ldevW2Nraqo+vX7/OggULGD16NA888ACPPfYY2dnZGI1Gi+dfvHiRxMREs8/iq6++4sqVKxaPDwoKIjo6GoBt27YxZswYtb7vvfce+/btY/To0fz5z3/m2LFj1da9orL3X/5Lv3379hgMBgCWLl1KcnIy48aNY/LkyezduxeACRMm4O3tzUsvvYS3tzfvvPMOxcXFdXpt0XRYN3YFhLDEw8ODZ555htmzZ1d67uLFi/z973/nk08+wdPTEysrKyZMmGB2jEajaZB6ubq6kpWVxfXr19Uv48uXL1d7TsW6fPTRR5w7d45Nmzbh6urKyZMnCQ4ORqlioWoPDw8GDx7Mxx9/XKs6enl5ce3aNU6ePMm2bduYP3+++lz//v354IMPKC4uZv369YSGhpr139REp9ORlZVFbm6uGh6XL1/Gzc0NgC5durBy5UpKS0uJiYlhzpw5HD58mFatWvH888/z/PPPk5KSwqxZs7j33nuZMmVKrV9bNB1yxyEaXXFxMYWFhep/JSUlTJkyhQ0bNvDjjz+iKAr5+fl888035Obmcv36dTQaDW3atAFgy5Yt/PLLL7elrh06dKBv3768//77FBUVcezYMfWv6trKy8vD1tYWJycnMjMzWbNmjdnz7dq148KFC+rjhx56iOTkZCIjIykuLqa4uJjExER+/fVXi9e3trYmICCAd955h6ysLLy8vAAoKiriyy+/JCcnhxYtWmBvb4+VlVWV9VQUxezfpbCwEA8PDzw9PVm5ciWFhYWcOnWK8PBwgoKCAIiKiiIjIwOtVouTkxMAVlZWHDp0iNOnT2M0GnFwcMDa2rra1xZNmwSHaHSzZs2if//+6n/vv/8+/fr1Y/HixSxatIjBgwfj7+/P1q1bAejevTtPPvkkU6dOZcSIEfz888888MADt62+K1asICEhgaFDh7Jq1SoCAwOxsbGp9flPPPEEhYWFDBs2jEcffZSRI0eaPf/444+zc+dOBg8ezJIlS3BwcODDDz9k+/btjBw5Em9vb1asWEFRUVGVrxEUFMSBAwcYO3Ys1tY3GxaioqLw9fXlgQceYMOGDbzzzjtVXuPYsWNm/y79+/enpKSElStXcvHiRUaOHMnzzz/PCy+8oIbT/v37GT9+PJ6enixdupR3330XW1tbrl69ypw5c3jwwQcJDAxkyJAhPPLII7X+zETTolGquj8WQtRKaGgoXbt2Zc6cOY1dFSFuC7njEKKOEhMTOX/+PKWlpcTFxREbG8uYMWMau1pC3DbSOS5EHV29epUXXniBzMxM3N3dWbhwIb17927saglx20hTlRBCiDqRpiohhBB1clc3VRUUFJCUlISrq6sM/RNCiFoyGo1cuXKFvn37qisvlHdXB0dSUhKPPfZYY1dDCCHuSOvXr2fQoEGVyu/q4HB1dQVMb97SEhRCCCEqS01N5bHHHlO/Qyu6q4OjrHnK3d2djh07NnJthBDizlJVE790jgshhKgTCQ4hhBB1IsEhhBCiTiQ4hBBC1IkEhxBCiDqR4BBCCFEnEhxV+OrHS3SZF012gWxvKYQQ5UlwVOH/9p4BIDWroJFrIoQQTUuDBkd2djZz5sxh7NixjBs3jmPHjpGZmcmMGTPw9/dnxowZZGVlAaZtKpcsWYJerycoKIjjx4+r14mIiMDf3x9/f38iIiLU8qSkJIKCgtDr9SxZsqTKPZtvxbi+HgA42N7VcySFEKLOGjQ4li5dysiRI9mxYwdRUVF069aNsLAwhg8fTkxMDMOHDycsLAyAuLg4kpOTiYmJYfHixSxcuBBA3ZN506ZNbN68mTVr1qhhs3DhQhYtWkRMTAzJycnExcXVW93dnW0BkDXnhRDCXIMFR25uLkeOHCEkJAQAGxsbnJyciI2NJTg4GIDg4GB2794NoJZrNBoGDhxIdnY2aWlpxMfH4+XlhYuLC87Oznh5ebF//37S0tLIzc3F09MTjUZDcHAwsbGx9VZ/jUYDUK93MUIIcTdosHaYCxcu0KZNG+bPn8+pU6fo06cPf/vb30hPT0en0wGg0+nIyMgAwGAwmC1E6O7ujsFgqFTu5uZmsbzs+PqiufG/khtCCGGuwe44SkpKOHHiBH/84x+JjIzEzs5ObZayxNJf9hqNps7l9eXmHUe9XVIIIe4KDRYc7u7uuLu7M2DAAADGjh3LiRMnaNu2LWlpaQCkpaXRpk0b9fjU1FT1/NTUVHQ6XaVyg8Fgsbzs+Pqi3nFIL4cQQphpsOBwdXXF3d2ds2fPAnDw4EG6deuGr68vkZGRAERGRuLn5wegliuKQkJCAo6Ojuh0Ory9vYmPjycrK4usrCzi4+Px9vZGp9Nhb29PQkICiqKYXas+lN28yB2HEEKYa9Cxpq+//jqvvPIKxcXFdOrUieXLl1NaWkpoaCjh4eF4eHiwevVqAEaNGsW+ffvQ6/XY2dmxbNkyAFxcXHj22WfVTvbnnnsOFxcXwDSqav78+RQUFODj44OPj0+91V0Njnq7ohBC3B00yl08bCglJQU/Pz9iY2PrvJFTVMJF/rIhgT0vj6Krq0MD1VAIIZqemr47ZeZ4DUrv2lgVQohbI8FRhZsjtCQ5hBCiPAmOKsg8DiGEsEyCowrSOS6EEJZJcFRBg0wAFEIISyQ4qqC9ccdRKskhhBBmJDiqIBMAhRDCMgmOKt1oqpJeDiGEMCPBUQW54xBCCMskOKpQf+vsCiHE3UWCowpaWVZdCCEskuCogkZGVQkhhEUSHFWQCYBCCGGZBEcVbk4AlOgQQojyJDiqInccQghhkQRHFWSRQyGEsEyCowpaWVZdCCEskuCows1RVY1bDyGEaGokOKogq+MKIYRlEhxVuLnkiCSHEEKUJ8FRBenhEEIIyyQ4qqCRJUeEEMIiCY4qSFOVEEJYJsFRBWmqEkIIyxo0OHx9fQkKCmLChAlMmjQJgMzMTGbMmIG/vz8zZswgKysLMP1lv2TJEvR6PUFBQRw/fly9TkREBP7+/vj7+xMREaGWJyUlERQUhF6vZ8mSJfV6dyBNVUIIYVmD33F8+umnREVFsXXrVgDCwsIYPnw4MTExDB8+nLCwMADi4uJITk4mJiaGxYsXs3DhQsAUNGvWrGHTpk1s3ryZNWvWqGGzcOFCFi1aRExMDMnJycTFxdVbvW8ucijJIYQQ5d32pqrY2FiCg4MBCA4OZvfu3WblGo2GgQMHkp2dTVpaGvHx8Xh5eeHi4oKzszNeXl7s37+ftLQ0cnNz8fT0RKPREBwcTGxsbL3VU5YcEUIIyxo8OGbOnMmkSZPYuHEjAOnp6eh0OgB0Oh0ZGRkAGAwG3N3d1fPc3d0xGAyVyt3c3CyWlx1fX9Smqnq7ohBC3B2sG/LiX3zxBW5ubqSnpzNjxgy6du1a5bGW+ic0Gk2dy+uLbOQkhBCWNegdh5ubGwBt27ZFr9eTmJhI27ZtSUtLAyAtLY02bdoApjuG1NRU9dzU1FR0Ol2lcoPBYLG87Pj6okaQ5IYQQphpsODIz88nNzdX/fnbb7+lR48e+Pr6EhkZCUBkZCR+fn4AarmiKCQkJODo6IhOp8Pb25v4+HiysrLIysoiPj4eb29vdDod9vb2JCQkoCiK2bXqw82mKkkOIYQor8GaqtLT03nuuecAMBqNPPzww/j4+NCvXz9CQ0MJDw/Hw8OD1atXAzBq1Cj27duHXq/Hzs6OZcuWAeDi4sKzzz5LSEgIAM899xwuLi6AaVTV/PnzKSgowMfHBx8fn3qrv7asqaq03i4phBB3BY1yF0+NTklJwc/Pj9jYWDp27Finc39KySJoTTz/fnwQ+t5uDVRDIYRoemr67pSZ41WQJUeEEMIyCY4qlO0AKBs5CSGEOQmOKsgdhxBCWCbBUQWtTAAUQgiLJDiqoJUJgEIIYZEERxVuzhxv3HoIIURTI8FRhZvLqktyCCFEeRIcVSjr48grNDZyTYQQommR4KhCWd/GjuOpNRwphBDNiwRHFdo72wEw9N42jVwTIYRoWiQ4qmBjbfpoSozSxyGEEOVJcFShbDiuUTrHhRDCjARHFcpGVZ24lN3INRFCiKZFgqMGu0/W33a0QghxN5DgEEIIUScSHEIIIepEgkMIIUSdNNjWsXeDXh5OdGxt19jVEEKIJkXuOKqh1UCprHIohBBmJDiqodVoZFl1IYSoQIKjGlqtRpZVF0KICiQ4qqHVyEZOQghRkQRHNaykqUoIISpp8OAwGo0EBwfz9NNPA3DhwgWmTJmCv78/oaGhFBUVAVBUVERoaCh6vZ4pU6aQkpKiXmPt2rXo9XoCAgLYv3+/Wh4XF0dAQAB6vZ6wsLB6r7tWo6G0tN4vK4QQd7QGD47PPvuMbt26qY9XrFjB9OnTiYmJwcnJifDwcAA2b96Mk5MTu3btYvr06axYsQKAM2fOEB0dTXR0NOvWrePNN9/EaDRiNBpZtGgR69atIzo6mm3btnHmzJl6rbtGI4scCiFERQ0aHKmpqXzzzTeEhIQApm1YDx06REBAAAATJ04kNjYWgD179jBx4kQAAgICOHjwIIqiEBsby/jx47GxsaFTp0507tyZxMREEhMT6dy5M506dcLGxobx48er16ovVlqNbB0rhBAVNGhwLFu2jLlz56LVml7m2rVrODk5YW1tmnfo7u6OwWBaRNBgMODh4QGAtbU1jo6OXLt2DYPBgLu7u3pNNzc3DAZDleX1yTQct14vKYQQd7wGC469e/fSpk0b+vbtW+1xZcuXW/rLXqOx/Bd/deX1SaMBoySHEEKYabAlR44ePcqePXuIi4ujsLCQ3Nxcli5dSnZ2NiUlJVhbW5OamopOpwNMdx+XL1/G3d2dkpIScnJycHFxwd3dndTUm/t+GwwG9ZyqyuuLNFUJIURlDXbH8fLLLxMXF8eePXtYuXIlw4YN4x//+AdDhw5l586dAERERODr6wuAr68vERERAOzcuZNhw4ah0Wjw9fUlOjqaoqIiLly4QHJyMv3796dfv34kJydz4cIFioqKiI6OVq9VX7QajXSOCyFEBbd9kcO5c+fy4osvsmrVKnr16sWUKVMACAkJYe7cuej1epydnXn33XcB6NGjB+PGjSMwMBArKysWLFiAlZUVAAsWLOCpp57CaDQyefJkevToUa91Na1VVa+XFEKIO55GuYvbYlJSUvDz8yM2NpaOHTvW+fxZn33P+Yx8doT6NEDthBCiaarpu1NmjldDFjkUQojKJDiqYSWLHAohRCUSHNXQyCKHQghRiQRHNUxrVUlwCCFEeRIc1bDSynBcIYSoSIKjGnY2VmRfL2nsagghRJMiwVGNltZW0lQlhBAVSHBUQ3YAFEKIyiQ4qiF7jgshRGUSHNXQIHccQghRkQRHNTQaDRIbQghhToKjGlqN5X1ChBCiOZPgqIbsACiEEJVJcFRDRlUJIURlEhzVMG1RK81VQghRngRHNbTqfuiNXBEhhGhCJDiq8V1yOgBZ14sbuSZCCNF01Co4Pv30U3Jzc1EUhddee42JEycSHx/f0HVrdN+eMQVHQkpmI9dECCGajloFx5YtW3BwcCA+Pp6MjAyWL1/OP/7xj4auW5NRVCIbjwshRJlaBUdZ5/C+ffuYPHkyPXv2bFYdxsVGCQ4hhChTq+Do27cvTz75JHFxcXh7e5Obm4tW23y6RyQ4hBDiJuvaHLR06VJOnjxJp06dsLOzIzMzk2XLljV03ZqM4pLmc3clhBA1qdVtw7Fjx7j33ntxcnIiKiqKDz74AEdHx4auW6PzvMcFgJY2Vo1cEyGEaDpqFRwLFy7Ezs6OU6dOsW7dOtq3b8+rr75a7TmFhYWEhITwyCOPMH78eN577z0ALly4wJQpU/D39yc0NJSioiIAioqKCA0NRa/XM2XKFFJSUtRrrV27Fr1eT0BAAPv371fL4+LiCAgIQK/XExYWVuc3X5M3gvoAYNdCgkMIIcrUKjisra3RaDTs3r2bxx9/nCeeeIK8vLxqz7GxseHTTz/lyy+/JDIykv3795OQkMCKFSuYPn06MTExODk5ER4eDsDmzZtxcnJi165dTJ8+nRUrVgBw5swZoqOjiY6OZt26dbz55psYjUaMRiOLFi1i3bp1REdHs23bNs6cOfM7Pw5zZYEho6qEEOKmWgWHvb09a9eu5csvv+Shhx7CaDRSUlL9XtwajQZ7e3sASkpKKCkpQaPRcOjQIQICAgCYOHEisbGxAOzZs4eJEycCEBAQwMGDB1EUhdjYWMaPH4+NjQ2dOnWic+fOJCYmkpiYSOfOnenUqRM2NjaMHz9evVZ9aWFlmjkuneNCCHFTrYLj3XffxcbGhmXLluHq6orBYGDmzJk1nmc0GpkwYQIjRoxgxIgRdOrUCScnJ6ytTX3y7u7uGAwGAAwGAx4eHoDpDsfR0ZFr165hMBhwd3dXr+nm5obBYKiyvD7ZWJs+niIJDiGEUNUqOFxdXQkKCiInJ4e9e/dia2tLcHBwjedZWVkRFRXFvn37SExM5OzZs5WO0ajrQVUeuWRaZLBu5fXJxupGcEhTlRBCqGoVHNu3b2fKlCns2LGDr7/+Wv25tpycnBg6dCgJCQlkZ2erzVypqanodDrAdPdx+fJlwNS0lZOTg4uLC+7u7qSmpqrXMhgM6HS6KsvrU9kdhzRVCSHETbUKjn/961+Eh4fz9ttv88477xAeHs4///nPas/JyMggOzsbgIKCAg4cOEC3bt0YOnQoO3fuBCAiIgJfX18AfH19iYiIAGDnzp0MGzYMjUaDr68v0dHRFBUVceHCBZKTk+nfvz/9+vUjOTmZCxcuUFRURHR0tHqt+tJC7jiEEKKSWk0AVBSFtm3bqo9dXFxqXHIkLS2NefPmYTQaURSFsWPHMnr0aLp3786LL77IqlWr6NWrF1OmTAEgJCSEuXPnotfrcXZ25t133wWgR48ejBs3jsDAQKysrFiwYAFWVqbRTgsWLOCpp57CaDQyefJkevTocUsfQlXKgkPuOIQQ4qZaBYe3tzczZ85k/PjxgKnpysfHp9pzevbsSWRkZKXyTp06qUNwy7O1tVXnelQ0e/ZsZs+eXal81KhRjBo1qjZv4ZaUjaoqMsrMcSGEKFOr4Hj11VfZuXMnR48eRVEUHn30UfR6fUPXrdFpNBpsrLTSVCWEEOXUKjjANLeibP5Fc2JjrZWmKiGEKKfa4PD09LQ4xFVRFDQaDUePHm2wijUVuYUl/HhBNnISQogy1QbHsWPHblc9mrTvf7vW2FUQQogmo/lsqiGEEKJe1LqPo7lybGmNrbXkqxBClJFvxBq0d7YjI6+osashhBBNhtxx1OC0IaexqyCEEE2K3HEIIYSoEwkOIYQQdSLBIYQQok4kOGrJWCrrVQkhBEhw1GjeuJ6A+dLqNa0MLIQQdzMJjhqU7QJYWGIEIPakgXvnb+fsldzGrJYQQjQaCY4a7Pv5CgCfHfwNgG2Jpl0Kj56X9auEEM2TBEcNDNkFAPx0MQuAM2mmO43/HExupBoJIUTjkuCoQSsb026D+UWmfdLLmqh+TMlqtDoJIURjkuCogb2taXL9t2fS6TIvGqN0jAshmjkJjhoEDWhv9rhU9nQSQjRzEhw18O/tZva4bB/yzm1bNUZ1hBCi0Ulw1KBlCyuzxzqnlgA8PrxLI9RGCCEanwRHDcrmcZQpm0FuVXlHXSGEaBYkOGqg1ZonxPmMfIvlQgjRXEhw3KJSWbtKCNFMNVhwXL58mWnTpjFu3DjGjx/Pp59+CkBmZiYzZszA39+fGTNmkJVlmg+hKApLlixBr9cTFBTE8ePH1WtFRETg7++Pv78/ERERanlSUhJBQUHo9XqWLFnSYGtIeXdvV6nMKLkhhGimGiw4rKysmDdvHl9//TUbN27k888/58yZM4SFhTF8+HBiYmIYPnw4YWFhAMTFxZGcnExMTAyLFy9m4cKFgClo1qxZw6ZNm9i8eTNr1qxRw2bhwoUsWrSImJgYkpOTiYuLa5D3En/maqUyWehQCNFcNVhw6HQ6+vTpA4CDgwNdu3bFYDAQGxtLcHAwAMHBwezevRtALddoNAwcOJDs7GzS0tKIj4/Hy8sLFxcXnJ2d8fLyYv/+/aSlpZGbm4unpycajYbg4GBiY2Mb6u1UIsusCyGaq9vSx5GSksLJkycZMGAA6enp6HQ6wBQuGRkZABgMBtzd3dVz3N3dMRgMlcrd3Nwslpcd3xAWPNy7UpnkhhCiuWrw4MjLy2POnDm89tprODg4VHmcpaYfjUZT5/KGYOmypdJUJYRopho0OIqLi5kzZw5BQUH4+/sD0LZtW9LS0gBIS0ujTZs2gOmOITU1VT03NTUVnU5XqdxgMFgsLzu+IdhaW1Uqk1FVQojmqsGCQ1EU/va3v9G1a1dmzJihlvv6+hIZGQlAZGQkfn5+ZuWKopCQkICjoyM6nQ5vb2/i4+PJysoiKyuL+Ph4vL290el02Nvbk5CQgKIoZteqb5ambEhuCCGaK+uGuvAPP/xAVFQU9913HxMmTADgpZdeYtasWYSGhhIeHo6HhwerV68GYNSoUezbtw+9Xo+dnR3Lli0DwMXFhWeffZaQkBAAnnvuOVxcXADTqKr58+dTUFCAj48PPj4+DfJeLK2IK6vkCiGaqwYLjkGDBnH69GmLz5XN6ShPo9HwxhtvWDw+JCREDY7y+vXrx7Zt235fRWvBUrOUDMcVQjRXMnO8FgL7eag/Tx/RBZDhuEKI5kuCoxbaOtiqPy98pA821lrp4xBCNFsSHLdAq5HhuEKI5kuC4xZosDyPRAghmgMJjlug0YCl3Fi3/yxJF7Nuf4WEEOI2arBRVXebj6cPxqVVCwA0gKX7jSXRJwFIfmv87auYEELcZhIctTS6581Z6aYlT8yfl6YrIURzIU1Vt8B0x2EeFMWyQYcQopmQ4LgVFvo48otKGqcuQghxm0lw3AJLa/BeLzbe9noIIURjkD6OW1BxWffQDce4lFnQiDUSQojbR4LjFmg05qOqIhMuNVpdhBDidpOmqlugwfI8DiGEaA4kOG6BRqOpNKpKCCGaCwmOW5CRV8SuEw2zv7kQQjR1Ehy3yJBd2NhVEEKIRiHBIYQQok4kOH6HEmOp2ePR97vS1t6mkWojhBC3hwTH7/Dw+/HqfI7QMT3o2LqVxS7zgHfjeHb9D7e3ckII0UAkOH6HU6k57DmVBoBWo6lyg6fThhy2/5Rq8Rphcb/Se8GOOr921vVizl3Nq/N5Qgjxe8kEwN9p5qffA2Cl1aCvibfTAAAasklEQVTRaCitw56yoRuOqZMHi0pKsbGufY7/4V8HOW3I4dzyQDQaS4ugCCFEw5A7jlvgbNeiUplGU3lGeU3KzzjPvF5U6/OOnr/GaUMOAJt/SKnDKwohxO8nwXELNswaVqlMq9FgyC4gp6CEvm/srPM13/zqRJUr7GbkFal9KTuSUpn0zwPqc6cu59T5tYQQ4vdosOCYP38+w4cP5+GHH1bLMjMzmTFjBv7+/syYMYOsLNM2q4qisGTJEvR6PUFBQRw/flw9JyIiAn9/f/z9/YmIiFDLk5KSCAoKQq/Xs2TJktu6kVIvDyda2ViZlVlpNGo/Rm6h5QCorhkrOvEyvRdUDpyUa/k8sHgX987fDsAbXyaZPd+yhWS/EOL2arBvnUmTJrFu3TqzsrCwMIYPH05MTAzDhw8nLCwMgLi4OJKTk4mJiWHx4sUsXLgQMAXNmjVr2LRpE5s3b2bNmjVq2CxcuJBFixYRExNDcnIycXFxDfVWLDr0mp/ZY0vdDPlFJZy4lK0+LqowfLc2fknLNXtcceLh8G5tK51zPj2fzPzaN30JIURdNFhwDB48GGdnZ7Oy2NhYgoODAQgODmb37t1m5RqNhoEDB5KdnU1aWhrx8fF4eXnh4uKCs7MzXl5e7N+/n7S0NHJzc/H09ESj0RAcHExsbGxDvRWLnFqa93OcuJxt9riopJTeC3YS+N5+tax8E1Nt2d7oMC+bH2JboQPd0o2Wz//uZeCiXZXKFUXh0Nl02eZWCPG73NZ2jvT0dHQ6097dOp2OjIwMAAwGA+7u7upx7u7uGAyGSuVubm4Wy8uOv90OzPNVf75eZL6R031//7rS8RXDpTYKi013KZ3atAIgdMx9Zs9bGv5blS1HLzI17BAbjlyocz2EEKJMk2ggt/QXcMXNkmpTfru1d7FTf9ZqNYzr617N0Sb5RSX86d+H+GPYoVq9RsGNnQWttab3t+l705f+son9gLot737w13QA5m/9qfYnCSFEBbc1ONq2bUtammnCXFpaGm3atAFMdwypqTcnyKWmpqLT6SqVGwwGi+VlxzeGd0L6A6amqUcHd6rx+NErvuHAr+kcPJteq+vn37iTsboRHGWT/rrrHIDKdxzVNUMN6tIagA7lAk8IIerqtgaHr68vkZGRAERGRuLn52dWrigKCQkJODo6otPp8Pb2Jj4+nqysLLKysoiPj8fb2xudToe9vT0JCQkoimJ2rdutS1t7ANq0sqG3h1ONx9d2Vd1TqdnM/u8PvLz5RwCsrczvqMoeVxyoVVhSdQd8Wf+IBIcQ4vdosJnjL730Et999x3Xrl3Dx8eHF154gVmzZhEaGkp4eDgeHh6sXr0agFGjRrFv3z70ej12dnYsW7YMABcXF5599llCQkIAeO6553BxcQFMo6rmz59PQUEBPj4++Pj4NNRbqdbgLq35x5QBjO3rjr1t5Y/zwycGkZyez+JtJ6q9juc9LmYjsF4NT+THlCz1sZXWPOPLmq4q3nFUFxwFN/pLWljLTHMhxK1rsOBYuXKlxfJPP/20UplGo+GNN96weHxISIgaHOX169ePbdu2/b5K1gONRsPkBzuqj9dOe5Cn/2O+oGFtvqZHdGtLYrmgcGhp/k9jX2HeiPWNIKnYNJWYklnla6w//BsANlZNomtLCHGHkm+QehbQx50xvW72txQbFYtzPJ4Z1Y3kt8bzxPDOvPlIH6y1WoylihoEFe8wio3mAVH2dPmmqm9OpzHtw++qrNvxG3c0dVkTSwghKpJvkAaw7onB6s+KonD4bEalY7ILigF4c0JfnhjRRf0yLwuIgR3N58CUlJYS9/MV9bH2RhqV3XCcu5rH9I+PmJ1TFkLfnrlKl3nRankLC3ccaTkFHDt/rXZvUAjRrElwNJDVUwcC0NPDiQO/Xq30fNn8jDJlfRbFN2aXF1aYZV5iVPgw/hwAvT2cuHG42sdRbGFWetndyGsR5sNvLY27GrI0lon/PCAzzoUQNZLgaCATBnbg1OKx3NvOHicLq+mWlFYIjht3ASU37jh2nzBNaHx1bE96eThRbCxVw8GlVQt13krZxMOKQQQ3Q8WlwutHJ14mr4r1tM7KHh9CiBpIcDSgli1MHdqfP1V5Nd3543qZPd553DQvpezu5Ncrpi/w2Q91w66FltTsAhIumDq+357cX+1w/+uWRK4XGSkyms9ch5vBMaCTS6Xn+txYwbfPgh1mzVj2NrJFixCievItcRvc07YVX/zPMLq5mu4+FAXsKoySOnmj43r2+qN0amM+z+LoefORUp3atOIXw83l1L9LzuCJj0yd4htmDePAmau8t+cM/z10npne97L/l6s4tbQmcWGAWUgA5FVYKsVSk5cQQpQnwXGbWFrFtjwX+xbk3Gg+upBxvcbrlZQbTlUWGmAaMRX1o2mDqMXbTvD54d/qtMVsdfNAhBACpKmqyXh7cv9aHffAPaZmpx43lhypyMZKy2/p+erjsiaviirOCylTJMEhhKiBBEcTMaJbOxZP6KM+7tPeiV0vmmbDvxbYE4Dlk/qxZfYIwNSZ/q8/P1jpOjkFJcwNuL/K10lc6A+YmqiOXzJNOBzTS8faaaZrSVOVEKIm0lTVhEwb3oVpw7uQfDWPLu3s1fL/GdmVKQ92ovWNPTnKjO3rTkAfN3Yev7mkfKc2dgzu0prwH1LMmqjcnVoC5vuIjH8vHoCfDbm0dzb1q/yWnge41vt7E0LcPSQ4mqDyoQGmZU0qhkaZtdMGcez8NXp5OGEsVdT1sva+8hAFxUbC4s7ygm/3apedv3AtX52A+HrUcUb2cKVLO3u+OZ3G0Hvbsv7wb+icWvLIgPb19A6FEHcyCY67gOc9rS2Wt2xhxRy/HpXK418djffbe9XHDrbWZoslHj1/jVJFYfrHR5jo2YGIYxcBJDiEEID0cTRLHVu34rMnh6iP1z81FJdWN5uwtBqNupJuWWjUVYmxlMKSynNLALrMi1aHBZ+7mseOpMu39BpCiMYhdxzNlM99rvyydBzGUkWdqFgmdGMCz4/u/ruuP+3D7zh4Np3kt8ablZdfzddYqjB6xTcAnF0WiFYry70LcSeQ4GjGWlhpKZ8Z5Zul1uw9U+n43MISHCzsOWJJxR0OI46l4NSyhXonA6bVfMsUGUtpqbU8RFgI0bRIU5VQrfzDgGqfH7J0d6WywhIj+pX7iP+l8kKOAOm5hfzw2zVe3PgjMz/9nl/Sbs54f+GLY+WuI8OAhbhTSHAIlUajYfqILlU+n19kRFEUsz6Ks1fy+CUtlz9/eNjiOcH//JbJHxxQH6/a/Yv6c9kQYZD5I0LcSSQ4hJmFj/Thu9eq3r89KuGS2ePrxTc7wMsCpbTccijVLZ9SfiXelzb9SJd50eRWsWqvEKLpkOAQleicWrJ4Qh9a2ViR9GaA2XPv7fnF7PFrW833+gB4JfzHOr9m2SZVK3aeptfrO3j8o6p3Mvz6p8ts+O482xIvVXmMEKLhSOe4sKhsFjuARnNzp8Gz5da+ivv5CqdScyqdu/VozUN4y1+zvM8OJlOqmK5tyC7ArVxzVpnZ64+qP3fXOdDT3YnvzmWQfDWPPwzuZHassVRBq6HaCZBCiLqR4BA1OvyaH0OWxlYqr+6uoCbzxvZk+denKpWX30P91yu5fBh/jnYONszy6cbnh8/zYGfzyY5jV+1nzZ88WRZ9kktZBUzwbE+JUeHc1Tw6tW7FgEUxAOx60Ye9p9OY5dONnIJirLXaSkvblzlw5iqd29nTweXm8vanU3NoY2+Dq6OtWqYoCtnXS3C+MQcmM78IZ7ubm2ydvJyNsVTB2kpDT3cn9bzkq3mk5xXRp72T2VDorOvFXMsrqrRyQGmpwhdHzjOiWzvurfBcVaaGHUTf252pgzupqwmAaX5NVMIlJj3QwSxMFUWhyFiKrXXlzyQzvwhFMTVLtr/xmRiyC8gpKKF7ucU2S0sVdp000M3VHseWLdh1wsB35zJYPXUgeUVGWlpryS4ooaikFHfnlmQXFPPfQ78xybMjl7Ku062dA2eu5HAxs4BOre2IPHYRK62W50Z3QwFijhsI9mxP9vUS3J1bkpVfzPmMfPp2cGLz9ykEe3Ygp6CYH1My6dvBmR1JqZy4lM3L/vfj6miLoihk5BXRxt6G16OSOJ9xnZf09xFzPJW/ju1p9p6NpQoJFzLV37eIYymkZRcyYWAH2jrYEHHsIj3dHdn+Uyqd27bCr5eO5dtP4dtTx/fJGfTt4Ey/js70dHfim9NpfHvmKsO6tuX1yCQWB/elnYMtX/54iX0/X8G3p47urg5sPZZCZn4xG2cNJ/xoCslX87icdZ2gAe0Z3q0tb20/RfyZq9zbzp6ki1noe7sxx68H4T+k4NtTx6AubTh0Nh1bay0XM6/zYOfWeDibb9FQXzSKYunvvrtDSkoKfn5+xMbG0rFjx8auzh0tr7BE3fzp93jS616eGNGZ68VGxq7aX+2xjw29h/WHzwOmEV8vbaq5Cewl/X0cPpfOt2fS2TJ7hFnHPMAPfx/Dg0tMo8OS3xrP/K0/0a+DM38aeg9pOQW0tbel22vbAVj16EDmb/2Jva88xLDlpuA8tXgsV3IK+ejbc2TkFRGVcInAfu7493YndGMCIQ92ZJJnB05czmZJ9En1dc8tDyT4nwcIebAjr0cmAaaFLJ8Y3oXjl7KYNrwzY1bGAXDsdT0/G3Jo62BDd52j2R4qq6cOZES3dvySlsOf/n1YfR8vbkwg4thF/jNzCFqNhsfW3RyssP+vo7lwLZ9XtySqfU6PDGjP/MCejHrnGw6/5ofn4l0A7H7Jh6zrJbyy+UdiXxrFB/t+5X93njb7DHeG+hCwKs6sbJJnBwpKjGz/KbXGfyOAacM6859Dv9XqWEvuc3PgZ0MuAIM6t+b7367d8rUAFjzcm3/vP8sD97Rm3riejHxnb5XHajXmf+A0ZZ/MGMxD9+vqfF5N350SHKLWvk/OYPG2E0Q+58VfNiTw5Y83+xjKJvoNWxZLanYBAM+P7s6avWdY8ydPnv/8mNlxiqJw73zTF7SjrbW6F0l9Gt/Pg+ifzGelb5g1jKlhhwB4I6g3b351AoCD830ZvnwPvT2cOHE52+wcuxZW6iCAj2cMZsbHR+pcl9iXR+H3j321OnZH6Eg1VH9eMo77/v51tcd7d29H/BnTcOhurvYWl9If0NGZH1Oy1Mf3tGnFiG5t2XDkAg621uqghEmeHdh6Yy5P8MD2RCY0v36kv469n3d2nK75wDtA8MD2rJrqWefz7vrgiIuLY+nSpZSWljJlyhRmzZqlPifB0XDKf/Gv+ZMnD/c3X8fKWKpgdWMmuKIoTPnXQZZM7GvWZKMuO7I8kGv5xdi1sGL4W7Fk5hffpndx06anh/OHtQdv++uK5slKq2Gm972ExZ0FYElwX/5+4060OrEvj+LAr+ncp3Pg0Rt/AFXnxKIAWt3CdtB3dXAYjUYCAgL4+OOPcXNzIyQkhJUrV9K9u2m5DAmOO9f2ny7zbLlO8Ip6ujtiY60lsdxf0XezYV3bcOhsRp3OeWRAe7O7wjtJBxc7lk3qR0trrfoFOce3O+/tqbyiwa1aMWUAr2yu+wjAijzvceHYje2dy76ooxIu4tjSmt4ezqzcdZq3JvUn83oxrVu1qPNAjROXspm9/ge++J9hah9TmWt5RdjbWnMmLZdLmdfx7tGu0hJCt6Km7847unM8MTGRzp0706mTaSTN+PHjiY2NVYND3LkC+3kQ/sxw9pxKY6b3vVzOKqCbqwMnLmfRua097RxsyS8q4VRqDpP+eYBFE/rwybfJZnNDypxaPBYbKy36d/dVasYp349SppWNFd/9bQw/Xsg06yt4SX8fK3f9rD52tLXmkycHczo1lzV7fuGVgPvN+mFsrbVM9OzADK97CVgVh2NLa/p3dOZPQzpzv7uD2qfx4pj78LzHhYH3uPDSxgR2nzQtxfK/If2ZG56Ib08dH00fzMPv7yfp4s1mtPnjKg8waN2qBQqmAQ221la0sbfhkwPJ6vMJC/SE/5Bi1v9irdXw/h89+TEli3/t+7XS53dPm1acz7i5q6RfTx1GRWH5pH4MX77H7NiKzYOrpw7kLxsSeC2wJ5cyC4g5nkq/js7Y21jzcsD9OLa0xsZKS/wvV8krKuGRAe0rfbGWX+/sJf+bm5QVG0sxliq8uDGBF3x70Lv9zbvZHUmXuZJTqI4MLJOeW4izXQusrUwzEUIevPmlWP4uGky/Ny1bWHEp8zpf/XiJJ73vpcWN8xTF1Hnep70zNtZaikpK1a0JACYM7KD+/E6IaUWGNlVsjVCT3u2d2Dd3tMXnyrZb6N3eyez9N7Q7+o5jx44d7N+/n6VLlwIQGRlJYmIiCxYsAOSOQ1imKEqlL6es/GJ1dJSwrOyrQoY23/3u6jsOS5knv9SiJpZ+RyQ0aib/3xJl7uiZ4+7u7qSm3hz+ZzAY0OnqPvRMCCFE7d3RwdGvXz+Sk5O5cOECRUVFREdH4+vr29jVEkKIu9od3VRlbW3NggULeOqppzAajUyePJkePSpvlSqEEKL+3NHBATBq1ChGjRrV2NUQQohm445uqhJCCHH7SXAIIYSokzu+qao6RqNpfaHyI6+EEEJUr+w7s+w7tKK7OjiuXDFtDvTYY481ck2EEOLOc+XKFTp37lyp/I6eOV6TgoICkpKScHV1xcrq96/fIoQQzYHRaOTKlSv07duXli0rb6Z2VweHEEKI+ied40IIIepEgqMa77//PiNHjmTChAlMmDCBfftqtxHP3ezDDz/k/vvvJyOjbkt8321WrVpFUFAQEyZM4Mknn8RgMDR2lRrN22+/zdixYwkKCuK5554jOzu75pPuYl9//TXjx4+nZ8+e/PTTT41dnQYhwVGD6dOnExUVRVRUVLOfaHj58mUOHDhA+/btaz74LvfUU0/x1VdfERUVxUMPPcT//d//NXaVGo2Xlxfbtm3jq6++okuXLqxdu7axq9So7rvvPt5//30GDx7c2FVpMBIcotaWL1/O3LlzZZVUwMHBQf35+vXrzfoz8fb2xtraNEBz4MCBzX74e7du3ejatWtjV6NB3dXDcevD+vXriYyMpG/fvsybNw9nZ+fGrlKjiI2NRafT0bNnz8auSpPx7rvvEhkZiaOjI5999lljV6dJ2LJlC+PGjWvsaogG1uxHVU2fPp2rV69WKg8NDWXgwIG0bt0ajUbD6tWrSUtLY/ny5Y1Qy9ujus9i7dq1fPTRRzg6OuLr60t4eDht2rRphFrePtV9HmPGjFEfr127lsLCQubMmXM7q3db1eaz+OCDD0hKSmLNmjV3/R1YbT6PadOm8de//pV+/frd7uo1uGYfHLWVkpLCM888w7Zt2xq7Krfd6dOnmT59OnZ2pv2OU1NT0el0bN68GVdX10auXeO7ePEiTz/9dLP83SgTERHBhg0b+OSTT9Tfk+bubg4OaaqqRlpamrox1O7du5vtku33338/Bw8eVB83lzuO6iQnJ9OlSxcA9uzZc9e3aVcnLi6Of//73/z3v/+V0Ggm5I6jGnPnzuXUqVMAdOjQgUWLFskOg0hwALzwwgucO3cOjUZDhw4dePPNN3Fzc2vsajUKvV5PUVERLi4uAAwYMIBFixY1cq0az65du1i8eDEZGRk4OTnRq1cvPvzww8auVr2S4BBCCFEnMhxXCCFEnUhwCCGEqBMJDiGEEHUiwSGEEKJOJDiEEELUiQSHaNY8PT0b/DViY2MJCwtr8Ncp7/Dhwxw9evS2vqZoPmQCoBD1wGg0VrnLpJ+fH35+fvX+miUlJerighV99913tGrVigceeKDeX1cICQ4hbli3bh1ff/01RUVF6PV6de2pZ599ltTUVAoLC3n88cd59NFHAdPdyvTp04mPj+fVV1/lr3/9K8HBwezdu5eSkhJWrVpFt27d2Lp1K0lJSSxYsIB58+bh4OBAUlISV65cYe7cuYwdO5bS0lIWLVrEkSNH6NixI6WlpUyePJmxY8ea1XHatGl4enpy9OhRfH196dKlCx988AHFxcW4uLiwYsUKCgoK2LBhA1qtli+//JLXX3+drl278sYbb3Dp0iUAXnvtNR588MHb+wGLu4YEhxBAfHw8v/32G+Hh4SiKwuzZszly5AiDBw9m2bJluLi4UFBQQEhICP7+/rRu3Zr8/Hx69OjBX/7yF/U6rVu3JiIigvXr1/PRRx+xdOnSSq+VlpbG559/ztmzZ5k9ezZjx44lJiaGixcv8tVXX5Genk5gYCCTJ0+2WNfs7Gz++9//ApCVlcWmTZvQaDRs3ryZdevWMW/ePKZOnUqrVq2YOXMmAC+//DJPPPEEgwYN4tKlS8ycOZOvv/66AT5J0RxIcAgBfPvtt3z77bcEBwcDkJ+fT3JyMoMHD+Y///kPu3btAkybWf3222+0bt0aKysrAgICzK7j7+8PQN++fdVzKhozZgxarZbu3burK6z+8MMPjB07Fq1Wi6urK0OHDq2yroGBgerPqampvPjii1y5coWioiI6duxo8ZwDBw5w5swZ9XFubi65ublm+4oIUVsSHEIAiqIwa9Yspk6dalZ++PBhDhw4wMaNG7Gzs2PatGkUFhYCYGtrW6lfo0WLFgBotVqMRqPF17KxsbH4+rVVfiHBJUuWMH36dPz8/Dh8+DBr1qyxeE5paSkbN26kZcuWtX4dIaoio6qEwLSL3ZYtW8jLywPAYDCQnp5OTk4Ozs7O2NnZ8euvv5KQkNAgr//ggw8SExNDaWkpV69e5bvvvqvVeTk5OeriipGRkWq5vb29+l7A9P7KmrcATp48WU81F82RBIcQmL5YH374YaZOnUpQUBBz5swhLy8PHx8fSkpKCAoKYvXq1QwcOLBBXj8gIAA3NzcefvhhFixYQP/+/XF0dKzxvOeff56//OUv/OlPf1JXpwUYPXo0u3btYsKECXz//ff87W9/IykpiaCgIAIDA/niiy8a5H2I5kFWxxWiicjLy8Pe3p5r164xZcoUvvjiC9koSzRJ0schRBPxzDPPkJ2dTXFxMc8++6yEhmiy5I5DCCFEnUgfhxBCiDqR4BBCCFEnEhxCCCHqRIJDCCFEnUhwCCGEqBMJDiGEEHXy/x41yIqdzkBmAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "lr_finder.plot_schedule(clip_beginning=20)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It doesn’t look as good as the plot Jeremy used in the class. The PyTorch version seemed to make it much more apparent where the loss started to level off. I haven’t dug into this too closely but I’m guessing there are some \"tricks\" in that version that we aren't using. If I had to eyeball this I’d say it’s recommending 1e-4 for the learning rate, but Jeremy used 1e-3 so we’ll go with that instead.\n", "\n", "We’re now ready to train the model. I’ve included two new callbacks (both built into Keras) to demonstrate how they work. The first one automatically reduces the learning rate as we progress through training if the validation error stops improving. The second one will save a copy of the model weights to a file every time we reach a new low in validation error." ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()\n", " if d.decorator_argspec is not None), _inspect.getargspec(target))\n", "/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py:560: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead\n", " return np.fromstring(tensor.tensor_content, dtype=dtype).reshape(shape)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Train on 814150 samples, validate on 30188 samples\n", "Epoch 1/20\n", "814150/814150 [==============================] - 68s 83us/step - loss: 1138.6056 - rmspe: 0.2421 - val_loss: 1923.3162 - val_rmspe: 0.3177\n", "Epoch 2/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 962.1155 - rmspe: 0.2140 - val_loss: 1895.0041 - val_rmspe: 0.3015\n", "Epoch 3/20\n", "814150/814150 [==============================] - 66s 80us/step - loss: 850.5718 - rmspe: 0.1899 - val_loss: 1551.5644 - val_rmspe: 0.2554\n", "Epoch 4/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 760.7246 - rmspe: 0.1607 - val_loss: 1589.6841 - val_rmspe: 0.2556\n", "Epoch 5/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 723.1884 - rmspe: 0.1522 - val_loss: 2032.6661 - val_rmspe: 0.3093\n", "Epoch 6/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 701.6135 - rmspe: 0.1470 - val_loss: 1559.3813 - val_rmspe: 0.2455\n", "\n", "Epoch 00006: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026.\n", "Epoch 7/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 759.7100 - rmspe: 0.1551 - val_loss: 1363.9912 - val_rmspe: 0.2134\n", "Epoch 8/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 687.3188 - rmspe: 0.1445 - val_loss: 1238.6456 - val_rmspe: 0.1987\n", "Epoch 9/20\n", "814150/814150 [==============================] - 66s 82us/step - loss: 664.9696 - rmspe: 0.1411 - val_loss: 1156.7629 - val_rmspe: 0.1894\n", "Epoch 10/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 648.3002 - rmspe: 0.1383 - val_loss: 1085.9985 - val_rmspe: 0.1804\n", "Epoch 11/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 634.7324 - rmspe: 0.1358 - val_loss: 1046.5626 - val_rmspe: 0.1764\n", "Epoch 12/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 620.5305 - rmspe: 0.1331 - val_loss: 998.0284 - val_rmspe: 0.1702\n", "Epoch 13/20\n", "814150/814150 [==============================] - 66s 80us/step - loss: 608.7635 - rmspe: 0.1308 - val_loss: 972.2079 - val_rmspe: 0.1672\n", "Epoch 14/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 596.7082 - rmspe: 0.1287 - val_loss: 944.8604 - val_rmspe: 0.1627\n", "Epoch 15/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 585.2907 - rmspe: 0.1265 - val_loss: 902.0995 - val_rmspe: 0.1568\n", "Epoch 16/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 575.5892 - rmspe: 0.1246 - val_loss: 854.3993 - val_rmspe: 0.1492\n", "Epoch 17/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 566.3440 - rmspe: 0.1228 - val_loss: 817.1876 - val_rmspe: 0.1438\n", "Epoch 18/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 558.5853 - rmspe: 0.1214 - val_loss: 767.2299 - val_rmspe: 0.1369\n", "Epoch 19/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 550.4629 - rmspe: 0.1200 - val_loss: 730.3196 - val_rmspe: 0.1317\n", "Epoch 20/20\n", "814150/814150 [==============================] - 66s 81us/step - loss: 542.9558 - rmspe: 0.1188 - val_loss: 698.6143 - val_rmspe: 0.1278\n" ] } ], "source": [ "model = EmbeddingNet(cat_vars, cont_vars, embedding_sizes)\n", "lr_reducer = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=3, verbose=1, mode='auto',\n", " min_delta=10, cooldown=0, min_lr=0.0001)\n", "checkpoint = ModelCheckpoint('best_model_weights.hdf5', monitor='val_loss', save_best_only=True)\n", "history = model.fit(x=X_array, y=y, batch_size=128, epochs=20, verbose=1, callbacks=[lr_reducer, checkpoint],\n", " validation_data=(X_val_array, y_val), shuffle=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By the end it’s doing pretty good, and it looks like the model is still improving. We can quickly get a snapshot of its performance using the “history” object that Keras’s \"fit\" method returns." ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "min training loss = 542.9558401937004\n", "min val loss = 698.6142525395542\n", "min val epoch = 20\n" ] } ], "source": [ "loss_history = history.history['loss']\n", "val_loss_history = history.history['val_loss']\n", "min_val_epoch = val_loss_history.index(min(val_loss_history)) + 1\n", "\n", "print('min training loss = {0}'.format(min(loss_history)))\n", "print('min val loss = {0}'.format(min(val_loss_history)))\n", "print('min val epoch = {0}'.format(min_val_epoch))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "I also like to make plots to visually see what’s going on. Let’s create a function that plots the training and validation loss history." ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGhCAYAAACQ4eUqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzs3Xl4lOWh/vHv+74zk2SyQHYIkICIyiaCG27gXvfWuuCGoNVq1db22NbW03NOW3t+drHVVj3WWtfiRrEuuBZlUxZBRED2LUBCQvZ1Mvv7+yMSAQkkMMls9+e65rrIPLM8T2aYufOsxrhTL7ARERERiTNmtCsgIiIicigUYkRERCQuKcSIiIhIXFKIERERkbjkiHYFosmdnkEg4I92NUREROQAnE4XntaWr12ftCHGnZ7B1ZO/F+1qiIiISBdM/8fjXwsySRtidvfATP/H4xHujTHIyOpLS1MDkOir19XWxKS2Jp5kaSeorYnH6XRx9eTv7fe7OmlDzG6BgJ+AP7IhJhgIfPmYifumaqe2Jia1NfEkSztBbU0umtgrIiIicUkhRkREROKSQoyIiIjEJYUYERERiUsKMSIiIhKXFGJEREQkLinEiIiISFxSiBEREZG4pBAjIiIicUkhRkREROKSQoyIiEiEFBTk89wzfyMlxdWl2992681MuvrKHq5V4kr6s5NERCQ5PffM3zr+7XQ6sW2bYDAIwNp16/nt7/7Y7cesqqpmyk3f7fLtn3jy6W4/R3f836MP8/ennuWz5Z/36PNEi0KMiIgkpT3Dxo//425Kt21nxquvdXp70zQJh8O9UTXpIoUYSQgp7jADjgyyZWXXunBFJEqsFDCdPfgEBrYZmc+BwsIC/vLwgzz+1ye5/PLLcLvd3PrdO7n0kos45+wz6du3D01Nzbz3/izeeff9ve5z49Rb8fl83HXnbXi9PtLdbo47bgxNzU08/czzrFixCoC77ryNhoZGpr3wcsd9//LI41x91bfJyspk+ecrePyvfycQCAAwcuRwpt54AwUF+axes47Gpmbcaak89PAj3W5fv36F3Dz1RoYOHUJrq4cPZ8/lzZlvY9s2GRkZ3P7d7zB8+NGYpkV1TTUPPfwoFRWVjBkzmuuvm0RBfj7BYIjlyz/nscf/dvAn7AEKMZIQBh4VZNTpPratcRIKGtGujojsl4E59m4MR2qPPktbyAdL/wDYEXm8448fy33/+T/4/e1Borq6mvv/93fU1tZy9NFHcd/PfsyOsjJWrVq93/ufduopPPjHh/nLo49z0YXf4I7bb+X2O+7Gtvdfv3HjjuNn9/0XLpeLX//yvzj7rIm8/+8PyMzM4Cf3/JBnn3+B+fM/ZuSI4fz4nh/y+YqV3W6TZVn87Kf3sPTTZfzhjw9TUJDPz+69h5aWFj6cPZdvXnYxGAbfu/OHBAIBBg4cQGtrKwB3fu82nv/HC3y8YBEul4shQwZ3+/kjRSFGEkJeUQjThOzCEDXleluLxCab8PI/93hPTEZGOi0RCjAA06e/Smurp+PnxZ8s7fj3+vUbWPrpMkaOGN5piPnss+WsXrMWgNlz5nHj5OvIzc2hpqZ2v7f/54x/0dbmpa3Ny7LPlnPEkMEAHD9uLFVV1cydOx+AVV+sYdUXaw6pTUcddSSZmZm8/MoMQqEQ5eU7efvt95g48Qw+nD2XYDBIZkYG/fsVsm37DnbsKOu4bzAYpF+/QrKyMmlqamb9+g2HVIdI0Ke9JACb3KIQ4RDk9FeIEYlpIV/7pccYGOHIhqTqfcLG6aefysUXfoOCgnwMw8DlcjH/owWd3r++oaHj3z5fe9vTUjvvjaqv3/v2ubk5AGRn9/1a8Kmtq6dvn8yuN+ZLuTk51NXVEQqFOq6rqqomN6f9uV5/4y0sy+LuH9xJZmYGixYv4YUXX8Hn8/GHBx/i8m9dxp8e/B319fW8MfNtPv54YbfrEAn6tJe4l5EdJsVts22Ng5x+oYPfQUSkG/Yc9snPz+OO22/l/z3wB9asXUc4HOauO2+jNwax6+sbyMvL3eu63JxsQqFgtx+rtq6OnJwcLMvqCDIFBfnU1tUB7eHpxZem8+JL08nNyeGee+7mkosv4NV/vcHW0m386eFHMAyDMceO5t6f/gfr122guqbm8BvZTb0WYhwOBzffdCOjRo6gT58s6usb+PesDzsmQ5mmyeQbrmXCGadhGAaffLKUp555vmO52+GWS+LKLQrRXG9QscXB2HN8tI+Da16MiERe6pc9KE3NzYTDYUaPHsnx48axePEnPf7cyz5bztQpNzBxwul89PFChg8/htGjhvP5l5OEO2NZFk7nV71T4XCYDRs20dLSwtVXXcE/Z/yLgoJ8Lr7oAt548y2gfR7QzvIKKnftwuvzEgqGCIfDOJ1OTj75RJYv/5zWVg8ejwfbtqO2aqvXQoxlmTQ0NPK/D/yeqqpqiosHcd/Pf0J9fQOLFn/C5d+6lBHDj+HHP72PYDDET3/8I6695mr+Me1FgMMul8SVVxSidqeDukqLlDSb9D42rY0KMSISeTt2lPHa6zP5r1/8DNMw+XzFSpYsWXrwO0ZAc3MLf/jjw9w0ZTI333Qjq9esY8mnyzEPMv/nx/fcvdfPGzdu4hf//Wt++/s/ctPUyTzx+CN4PO2rkz6cPReA/v0KmTL5evr0ycLr8/HZsuW89fZ7AJx+2ilMvfEGnE4HtbV1PP7XJzt6cHqbMe7UCyI3+6mbbr/tFrxeL88+N43HHnmI5//xIp98+WYYc+xofvD9O7jlu3dg2/Zhl+/L6XJx/c1388LTfyHg90e0XZl9smlurI/oY8aqWGjr+VOaWfdJCtvXuTh/SjNrF6ewY33kl1rHQlt7i9qaeJKlnZBcbb33p//B9u07eOnlf0a7Kj2m/fv6B7zw9J+/9n0dtTkxpmly9NFHMXPm27jdbvLycikt3dZRvmVrKRkZ6eTm5uDxtB1WeWczwAEysvoS/HL9fSRl9smO+GPGqmi2NcUdJL1PE97mHDL7OGmuDVJYYtJQ2TN10uuamJKlrcnSTkjcto4ccQzbtu3A09bGsaNHcuzokbz1zqyEbS+Aw9n5RO2ohZibpk6mzeNh3vyP6dunDwCetq+WsHk87f9OS03FDtuHVX4gLU0N6ok5DNFua1ZBgLYWg6ryZsBg1/YwJSO8NDdG/liwaLe1N6mtiSdZ2gmJ3db+hXlMnXwNaWmp1NTU8szzL7H6iwPPiYl3TlfnPetRCTGTb7iWo48exv2/+S2hUIg2rxcAd5qb5uaW9n+73QC0eb2HXX5gNpHaEKndnnMxojZS10ui39bcoiC1O62OOtRVmoyeEMZyhCO86V3029p71NbEkyzthERv62uvv8lrr7/55U/GHj0widfWr3Tetl4/xXrKjddz7OhR3P+b33UEDo/HQ01NLSWDiztuN2RwCS0trdTW1h12uSSu9km9VsfPjTUmdrh90zsREUlsvRpipk65gdGjRvLr3/yW5ubmvcpmz5nLt791GdnZfcnMzOTKKy9n7ryPOiblHm65JB6nyyYrL0zNHiHGDhvU77LI6a8QIyKS6HptOCkvL5cLLzgfv9/Po3/56njz3cedv/b6TDIzM3nw9w9gmgaLFy/h5Ve+mm19uOWSeHL6hwj6oal27yxeV2lq0zsRkSTQayGmpqaWSdfe2Gl5OBzm2eem8exz03qkXBJP7oAQtRUW2HvPfamrsCgZHkSb3omIJLZenxMjEin7zofZra7SIsXdvumdiIgkLoUYiUumZdO3MERt+ddDjM9j0tpoaEhJRHrcpKuv4Ec/vAsAwzB47pm/MaCoqNPb/9+jDzNu7HGH/Hy33Xozk66+8pDvn2gUYiQu7V59VF/19RAD7b0xmtwrIgfys3vv4aapk792vdPp5Om/P86JJxzfrcezbZspN32X8p07I1K/u+68jRuuv2av65548mlemT4jIo+/r9GjRvLk3x7rkcfuKQoxEpdyi0LUV1qEQ/uf81JXaaknRkQOaPaceZx+2ql7HY4IcPLJJxIIBvls+edRqpl0VdR27BU5HLmdzIfZra7CYvQZPiyHHeFN70QkUSxbtpzgzUFOOvEEFixc1HH9WWdOYP78jwmFQqSlpXLXnbcz7MgjcTodbN9exrPPT2Pr1tKvPZ5pmrz0wrP8+Cc/Z0dZOYZhcOUVl3PuOWcC8ObMd/a6fX5eHt/97s0MLinGNC02bd7M008/z66qKi6+6AJOGX8yAOedezZ1dfX86J57uevO22hoaGTaCy8DMLhkEFd9+04GDiyioaGRN2e+zdx5HwFw9lkTOffcs1m27DPOP/9cDAzeefd9Xn9j5iH9viaccRrf+ualZGf3paxsJ8//4wU2btoMwJgxo7n+ukkU5OcTDIZYvvxzHnv8bwBcf901TDjjVFJTU2lqauaFF19m8SeROTRTIUbij2GT2z/ElhWdb0W956Z3NeV6m4vECqfLxnL25KR7A4cz3KVbhkIh5s9fwFlnTegIMfn5eYwYfgx/f+rZ9kczTD76eCF/eeRxQqEQk66+gv/40fe5+4c/IRw+8POcdeYEJk44nV/d/wDV1TVMnXIDffv2+aqmpsFbb7/LmjXrsCyLW74zhTvv+C7//cvf8PY77zFkSMlegWVf6enp/OgHtzP9n/9i1gezGXrEEH527z3U1dWzctUXAJQUD2LhwkXccecPGTK4hF/98hcs+2w5O3aUdel3tNvIEcO5+aYb+d3v/8SGjZs4c+IZ/PxnP+buH/2U5uZm7vzebTz/jxf4eMEiXC4XQ4YMBuC4447llPEn8bP7/pv6+gZycrJJS0vr1nMfiD7dJe70yQ3jcLX3tnSmY9O7fgoxIrHCMGy+cVMLzpSefZ6gv5W3nsigK3udzp4zlz9d/Fvy8/Oorq7hzIkTWL9hIxUVlUD7jvKLFy/puP0r01/lkosvpLCggIrKygM+9umnn8q77/2bnTsrAJj2wkucfdbEjvKqqmqqqqoBCAQC/HPGazz8p9/jdDoJdOFg4uOPP46Gxibee/8DwGbDxk3MmfsREyee0RFiGhubeOvt9wDYtHkLO8rKGTJkcLdDzIQzTmP+RwtYu249AB/Onst5557NSScez4ez5xIMBunXr5CsrEyamppZv34DAKFgCJfLxcABA2hqaqaurh6I3LlW+nSXuJNbFKKxxiTgP/AwUV2lSU7/rv1FJiI9z7YN3n8mo8d7YtLSsrHtRrpynlBFRSXrN2zkzIkTmPHqa0yccDr/nPGvjnKXy8XkG67luDHHkpGRgW2HMU2TrKzMg4aYnOxsqmtqOn5ua/PS0tra8XNWViZTJl/PMcOPxp3mBmxM0yQjI4P6+oN/0efm5FBTU7vXdVVVVQw9YnDHzw2NjXuV+7y+gx6MvN+25OawYsXKva7bVVVFTk4OAH948CEu/9Zl/OnB31FfX88bM9/m448XsuqL1cx49TUmXX0FAwcO4IvVa/nHtBfZtauq23XYH4UYiTu5Aw48H2a39k3vfGjTO5HYEfAbB/0D5PAYOB3dW7Mye848Jl11BRs3biI9PZ1Fe/S8XHbpRQwuKeF/fvUb6urqcTqdPPfM38A4eBvq6uvJz8vr+DktLZWM9PSOn6+/dhIul4uf3/ffNDU1U1hYwF8efrDjoe3wgUNYbV0debm5e11XUJBPbV3kT/Cuq60jPz9/7+fKz2flyvYen62l2/jTw49gGAZjjh3NvT/9D9av20B1TQ3/nvUh/571IampqUyZfB3fvfVm7v/NbyNSL61Oijgby6G//nuO3ekmd/vSpnci0hWLFn1CWloq37l5CgsXLsLv93eUpaWl4Q/4aW1tJSXFxXXXXo3RhQADsGDBIi74xnn079cPp9PJdddO2us8v7S0NHx+H62tHtLT07lmn/1fGhqb6FdY2Onjf/bZ5/Tt24fzzzsH0zQZduRQzpzYPin5cDidzr0uhmEw/+MFTDjjdI4++ihM0+TssybSr18hSz9dhtPp5PTTTyU93Y1t23g8HmzbJhwOc+TQIxg27EgsyyIQCODz+w86l6g71BMTYf0GBxl/6Q4aqkyqtjuo2mFRV9H5UmDpnvQ+NqnpdpdCjM9j0trUvulda6PyuojsXyAQYMHCxZx/3jnMnjNvr7KZb73DXXfezt/++ijNzS3867U39go5BzJ7zjzy8vL45S//E2ybmW+9Q0PDV8M7r/zzVe64/Vae/vtfqauv5403ZnLqqeO/uv/sudz9gzt46snHqW+o58c/uW+vx29paeXPjz7BFZdfwrXXXElDQyMvvPgyK1auOuTfRVZmJtOef2qv65586hk++GAOzz0/jdtuvZns7L6U76zggd8+SFNTc3uIOe0Upt54A06ng9raOh7/65PU1tVRNKA/N1x3DYWFBYRCITZv3toxaToSjHGnXpCUf6Y6XS6uv/luXnj6zwS6+Ibsqv6DM0jPrqOgOEjegBC2DTXlFtU7HFRtt748sDARQo1BZp9smhvr6crYcyQUDw9w9Ek+Zj2X0aXbn/CNNgI+gxVzuz8GvLfeb2v0qK2JJ1naCWpr4jnQ97V6YiLOoKXeRUVpCpuWuzCt9uXABcUhBh4dYNTpPnweg6odFlXbHVTvsPC2qpegq3KL9n/UQGfqKi1Khh98lr+IiMQfhZgeFg4ZVJc5qC5zwMIUXKlh8ge1h5oR4324z7dpqjWp2m5RtcNBTblFKJAIvTQ9I68oyIZlne8Psy9teicikrgUYnqZ32tSvtGkfKMTsMnoa5NfHKRgUIgTv9GG5WjvPaja3t5TU19lgq0vX4CUtDAZ2Ta1O7v+tt296V37YZF6u4uIJBJ9qkeVQUuDQUuDi60r2zeCyi4MU1AcpLAkxDEn+wn6obqsfS5N9Q5HUk9QzS0K4fW0/866avemd7n9FGJERBKNPtVjiG0b1FVa1FVarFsCDpdN3oAQBcVBjjwuwNizfbQ0GCyamUZLfdfnhSSKr85L6l7PlDa9ExFJTAoxMSzoN6jc6qBya/vLlJYR5sQL2igZHmT1wiQMMQNC7FjnPPgN91FfqU3vREQSUfKOTcShthaTHRuc9D8iGO2q9DqH06ZvXrhL+8Psq7ZCm96JiCQihZg4U7nFQWZOmIy+yTU8ktM/RDAIjdXdf8vuuemdiIgkDoWYONPWYtJQZdJvSHL1xuQWhairtLAPcaVWXYVFtkKMiEhCUYiJQxVbHUk3pNTdTe72VVdpkdNfIUZEJJEoxMShii0OcvuHcKUmx5CSYdrk9AtRW3EYIabCok9eGMuheTEiIolCISYONVabtLUa9BucHD0L2QVhDKN9ldGh2nPTOxERSQwKMXGpfel1vyQZUsotCtJQZR7WsQF22KChqn3TOxERSQwKMXGqYouDwuIgppX4wyO5A0LUHMLS6n3VVWjTOxGRRKIQE6dqyi1sIH9govcstJ8C3p3zkjpTV2l9ucw68YOfiEgyUIiJU+GQwa5tib9KKTMnjCu1fWLu4dq96Z07SyFGRCQRKMTEsYotji/3i0ncL+W8ASGaak383sM/LqBj0zsttRYRSQgKMXFsV6mDFLdN34LEneeRWxSi5jD2h9lXXYWlnXtFRBKEQkwcC/gMandaCT2klFt0ePvD7Eub3omIJA6FmDhXscVB/wQ9giAtM4w70z6snXr3VV+pTe9ERBKFQkycq9zqoE9+GHdW4g0p5RWF8DQZtLVE7m3aUK1N70REEoVCTJxrbTRpqk3MAyFziyKzP8yetOmdiEjiUIhJAIk6pJRbFKI2wiEG2je9y+6XeD1XIiLJ5vB3EOuG8887h4kTTqe4eBAbN23m1/c/0FE2oKiIKVOuZ+gRRxAKhVixYiVPP/sP2traAEhJSeHWW6Zy/LhxBIMBZs+Zz0svT++4/8HKE1nFFgfDxvlxumwC/sNfihwLXKk2WbnhngkxlRaDhvtoX5qeGL8vEZFk1Ks9MQ0Njbzx5tu88+77Xyv7wfe/R1VVNbd97/v86J6fkpObw3XXXt1RftPUyWRlZnHXD37Efb/4JeNPPpELvnFel8sTWf2u9n1UCgcnTm9MblEQfxs010X+LVpbYZGqTe9EROJer4aYJUs/ZcnST2lsbPpaWUFBAR99tIBgMEhrq4dPPllK8aBBALhcLk47dTwvT59Ba6uH6uoaZr71DmefNbFL5QdmRPjSk4/d2cWkYuvuje966zl7tq25RWFqKxy0v0Uj+9g+j/XlpnfhmGhr7F3U1sS7JEs71dbEvexfrw4nHcjMt95hwoTT2Vq6jZSUFMaPP4nln38OQP/+/XA4HJSWbuu4/datpQwcOADDMA5abtud/8WdkdWXYCAQ8fZk9smO+GMeSOOuFEadUUNWdl/scOcveE/oibYWDKqgalsGmX36RPyxAZprg/QrsWis7F7de/t1jSa1NfEkSztBbU0kDqez87JerMcBrVixkttvu4Vnn34Cy7JYsWIVM996F4C01FT8fj/h8FeTMVs9HizLwuVyHbTc5/N1+rwtTQ0E/P6ItiWzTzbNjfURfcyDaW2xGTXBJiWjhuodvfey9kRbLYdNZo6f5XMcPfZ7rNoeYtBwL82NXQ980Xhdo0VtTTzJ0k5QWxON0+XqtCwmQkx6uptf/Oe9TP/nv/j3rA9JTU3hpqmT+f5dt/Pwnx+jzevF5XJhmmZHUEl3uwmFQvj9/oOWH5hNZM8e2vNLsffmXIRDULXdQf8hAap3RH4y7P71TFuz+wUJh6Ghyozo4+6prtJi1Bk+LEeYULArQSY6r2t0qK2JJ1naCWprIuq8bTGxxLqwsBCXy8W77/2bUChEa6uHDz6Yw9jjxgBQUVFJMBikpKS44z6DB5dQVl6ObdsHLU8WFVt2n2od323OKwpRX2n16LBYx6Z3BdovRkQkXvVqiDFNE6fTiWmamIaB0+nEsizKy3fi9Xo5/7xzME2TtLRUzj33LLZ+OcfF7/ezYOFiJl19BW63m/y8PC655CJmz57XpfJkUVlqkZZpk5UX33ug9NT+MHvavemdzlESEYlfvTqc9O3Lv8lVV17e8fO0559i9Zq1/Pr+B/j9Hx7iumsncc2kKwmHbdZv2MD/Pf63jts++9w0bvnOFB575CGCoSCzZ8/jvfdndbk8GfjbTOoq2g+EbKrprSGlyDIMm5x+ITYs63wMNFLqKkxytOmdiEjc6tUQM+PV15jx6mv7LVu/YSP/86vfdHpfr9fLo489ccjlyaJii8WAYUHWL0mJdlUOSZ/8MKajfc5KT9OmdyIi8S0m5sRI5FRscZJdGCY1Iz57GPIGhGisNgkFej5U1FVq0zsRkXimEJNgWhpMmuuNuD1LKbd/z8+H2c3bauJpMjQvRkQkTinEJKDKLY44PdXa7pVJvXuqq7TI0YnWIiJxSSEmAVVscZA/KITDGV/DJBnZYVLcNjW9GWIqFGJEROKVQkwCqq20CPoNCkriqzcmryhEc52Jv6333pZ1lRZ98sJYjvgKfCIiohCTmGyDyq1W3M2LyS0KUVvRu0vDG6pNbFub3omIxCOFmARVscVBv8FBDCN+ehhyi0LUlvduiNGmdyIi8UshJkFV7XBgOSGnKD6+nFPTw6T36d35MLvVVWrTOxGReKQQk6BCAYPqHfEzpJRbFKKtxcDT1PubztVV7O6JiZ9eKxERUYhJaPF0IORXS6ujEGK06Z2ISFxSiElglaUOMvraZObE/lBJXi/vD7Onjk3vtNRaRCSuKMQkMG+rSV2lGfNDSk5X+8nb0ZgPs1tdpSb3iojEG4WYBFexxUG/I2I7xOQUhQj4oak2em9HbXonIhJ/FGISXOVWBzn9wqSkxe6QUm5RiLoKC+zonSStTe9EROKPQkyCa6ptn+8Ry2cpRXM+zG7a9E5EJP4oxCQ848tVSrH55WxaNn0Le3+Tu321b3pnal6MiEgcUYhJAhVbHeQPCsbkUEl2YQhsqK+KboiB3Sdax+6wm4iI7E0hJgnU7rQIhyB/UOz1MuQWhajfZREORW8+zG5fTe6NvbAnIiJfpxCTBOywwa7S3RvfxZbcGJgPs1tdpUVquja9ExGJFwoxSaJiq6N9cm8sHQhp2OT2j50Qo03vRETii0JMkti1zYErxSanMHbmfPTJC+NwQW1FbIQY0KZ3IiLxRCEmSQT9BtXlVkxtfJfbP0RjjUnQH/35MLtp0zsRkfihEJNEKrfE1ryY3AGxM5S0mza9ExGJHwoxSaRii4OsnDDpfWJhSMmOiU3u9qVN70RE4odCTBJpazFpqDZjojcmvY9NarodcyGmY9M7DSmJiMQ8hZgkUxEjQ0q5RSFaGg28rbH3Fmyf3BsLvVUiInIgsfcNIj2qYouD3P4hXKnRnfORWxT9owY6o03vRETig0JMkmmsNmlrNSgcHN3emLyiYMwNJe2mTe9EROKDQkzSMajcGt0hpRR3mIxsm9qdjqjV4UC06Z2ISHxQiElCFVscFBYHMa3o9DTk9g/h9Ri0NMTO/jD70qZ3IiKxTyEmCdWUW9hA3sDofEl/dV5SjIcY9cSIiMQ0hZgkFA4Z7NrmoP+Q6AwpxeImd/uqq2jf9C5avVUiInJwCjFJ6qvde3v3S9rhsumbF47ZlUm77d70LrtQvTEiIrFKISZJVZY6SHHb9C3o3f1QcvqFCAahsSa233ra9E5EJPbF9jeJ9JiAz6B2p9XrQ0q5RSHqKi1sO3bnw+ymTe9ERGJbr65xPf+8c5g44XSKiwexcdNmfn3/A3uVn3nmBC675CJyc3NpaWnhpZen8/GCRQCkpKRw6y1TOX7cOILBALPnzOell6d33Pdg5fJ1FVscFA8PsPaTlF57ztyiENU7Ynsoabe6CotBR/toH3KL/dAlIpJsejXENDQ08sabbzN06BCGDTtyr7Jzzj6TSy6+kEce+ytbt5aSkZFBerq7o/ymqZPJyszirh/8CLfbzS/uu5f6+nree39Wl8rl6yq3Ojh2gg93ZhhPc893ypmWTU6/EOs+cfX4c0XCnpveeZqiXRsREdlXrw4nLVn6KUuWfkpj497fCIZhcPVV3+a5519gy5at2LZNc3MzlZW7AHC5XJx26nhenj6D1lYP1dVAT1ngAAAgAElEQVQ1zHzrHc4+a2KXyg/MiPClJx87spfWRoumWpN+Q0K90ta++WEMA+p3OaLe9q5cvK0WnmaDnH77DilFv249e1FbE++SLO1UWxP3sn8xsWVqUVF/+vbtS0FBPn95+EGcTgervljDc8+/QGtrK/3798PhcFBauq3jPlu3ljJw4AAMwzhouW13vgInI6svwUAg4m3K7JMd8cfsCbXlMPAoP9XbDr2+XW1r0RGNNNUGcafnHPJz9bammiCFJRaNu9rbGC+vaySorYknWdoJamsicTidnZf1Yj06lZGRAcBJJ53AL/77V4RCIe783m1895abeOjPj5KWmorf7ycc/uov4laPB8uycLlcBy33+XydPndLUwMBvz+i7cnsk01zY31EH7OnbFsXZMIVXrxtdQT8nafdznS1rQ6XTd9+Hqp2WHHzuwGo2hFi0NFemhuNuHpdD5famniSpZ2gtiYap6vzKQgxEWK8bW0AvPHGWzQ1NQMw41+v86v/+U8Mw6DN68XlcmGaZkdQSXe7CYVC+P3+g5YfmE1k90rZMwjE/kZp9ZUmfq9BQUmA8o2dp939O3BbDdOmoDjEoKMDFA0N4vcarJqfst/bxqq6CotRp/n22fQufup/aOLrPXx4kqWtydJOUFsTUedti4kQs7OiEr/f3+mwT0VFJcFgkJKSYrZuLQVg8OASysrLsW37oOVyIF8dCNn9ELM/7XvPDDomwKCjglgOm52bHSyamUZ1mQVxsLR6T3tueudvjXZtRERkT706sdc0TZxOJ6ZpYhoGTqcTy7IIBALMm/8x3/rmpaSnp5OWlsq3v3UZn332ObZt4/f7WbBwMZOuvgK3201+Xh6XXHIRs2fPAzhouRxYxRYHhYODGOahB760zDBHneDj3Bs8nHm1h4y+YVZ+lMI7f89g2aw0qnc44i7AgDa9ExGJZb3aE/Pty7/JVVde3vHztOefYvWatfz6/gd47vkXuHnqZB7584MEgyFWrFjJs8+/0HHbZ5+bxi3fmcJjjzxEMBRk9ux5ey2fPli5dK5qh4VpQl5RiOqyrr8lHC6bomHN5Be3kj8wREOVydYvnJRtcODzJM4+irtPtK7cHO2aiIjInno1xMx49TVmvPrafssCgQBPPPk0Tzz59H7LvV4vjz72RKePfbBy6Vw4ZFC13UG/I4IHDTGGaVNYEmLQMQH6DwkS8FlsX2uxYm4KzXXxsYldd9VV7rnpnYiIxIqYmBMj0VexxcExJ/tYNX9/u9PaZBe2z3MZeFQQ07LZudHBwjfd+JrzaG5sIJG/4Osq2je9S00P0dwY7dqIiMhuCjECQGWpxbhzbbJywzTVtveouLPCDDo6wKBjAqT3sana1t7jUrnVQSjYvgFRZp/4m+fSXd5WE0+zQd9CL9U7o10bERHZTSFGAPC3mdRVmBQfE6ClIcSgY4LkDQhRv8tk60pX+zyXtsSZ59Jd5RudFI9oZuPy3jtnSkREDkwhRjpUbHEy6nQfniaDHeudfD47heb6xJzn0l0bPnUxZFQrA4+yKNug/zYiIrFAn8bSYfMKJ9VlFg1VJgc6qyIZ+b0mpV/0YfgpjZRvSscO6/cjIhJtyTs+IF8TDhk0VFkowOzf9rWZWBYMGRX5s7ZERKT7FGJEuigcNFm3JIVjTvLjcCbuaiwRkXihECPSDdvWOPH74MhxkT00VEREuk8hRqQb7LDBmoUpHDnWT0pa+OB3EBGRHqMQI9JNOzc7aK4zOfok9caIiESTQoxItxmsXpDCkFEB0vuoN0ZEJFoUYkQOQU25g6odFsPH+6JdFRGRpKUQI3KIVi9MYeCwIH3yQ9GuiohIUlKIETlETTUWOzY4GHmaemNERKJBIUbkMKxdlELegBD5g4LRroqISNJRiBE5DJ5mk60rnV/2xmgDPBGR3qQQI3KY1i91kdE3zIBh6o0REelNCjEih8nvNdm4zMWIU3wYpnpjRER6i0KMSARsWu7C4YTBOhxSRKTXKMSIREAoaLD2ExfHnOTH0uGQIiK9QiFGJEK2rXES9MGRY3UcgYhIb1CIEYkQO2ywelEKw8b5celwSBGRHqcQIxJBOze1Hw55zInqjRER6WkKMSIRZbB6YQpDRgdwZ6k3RkSkJynEiERYTZmD6jKLETocUkSkRynEiPSA1QtSGHiUDocUEelJCjEiPaBx9+GQp6o3RkSkpyjEiPSQtYtTyB8YIn+gjiMQEekJCjEiPcTTZLJllQ6HFBHpKQoxIj1o/VIXGdk6HFJEpCcoxIj0IH+bDocUEekpCjEiPWzT518eDjlSh0OKiESSQoxIDwsFDNYt0eGQIiKRphAj0gtKVzsJBuDI43QcgYhIpCjEiPQCO2ywZlEKw47X4ZAiIpHi6M0nO/+8c5g44XSKiwexcdNmfn3/A1+7TZ8+Wfzpwd9SU1PLvT//r47rU1JSuPWWqRw/bhzBYIDZc+bz0svTu1wuEm3lGx0MG2dy9Al+Vn2UGu3qiIjEvV4NMQ0Njbzx5tsMHTqEYcOO3O9tbp56I9u27yDd7d7r+pumTiYrM4u7fvAj3G43v7jvXurr63nv/VldKheJPoPVC1I49ZttbF7hwtOkjlARkcPRq5+iS5Z+ypKln9LY2LTf8uOPH0tmZgbz5n201/Uul4vTTh3Py9Nn0Nrqobq6hplvvcPZZ03sUvmBGRG+9ORjx9pFbe3upbrMSXWZxfDx/hhok17X5GlrsrRTbU3cy/71ak/MgaSlpXLj5Ov47e/+yFH79NL0798Ph8NBaem2juu2bi1l4MABGIZx0HLb7nxFSEZWX4KByC99zeyTHfHHjFVqa/eUrkznpIsr2Lkhj5Z6VwRq1TP0uiaeZGknqK2JxOF0dl7Wi/U4oOuuncRHHy2goqLyayEmLTUVv99POPzVhMhWjwfLsnC5XAct9/k6P4SvpamBgD+yK0Yy+2TT3Fgf0ceMVWpr9zU3QtlGB4OPrWLRm+kRqFnk6XVNPMnSTlBbE43T1fkfezERYo4+ahgjhh/DT3/2i/2Wt3m9uFwuTNPsCCrpbjehUAi/33/Q8gOziey5Nnt2eyX6niBq66FasyiF8ya3kjcwQE1ZTPw33INe18STLO0EtTURdd62mJhZOHr0KPLycvm/Rx/iicf/wtQpNzBw4ACeePwv5ObkUFFRSTAYpKSkuOM+gweXUFZejm3bBy0XiTWeJpOtq5yMPFWHQ4qIHKpe/RPQNE0sy8I0TUzDwOl0Eg6HmfnWO8z6YHbH7U495WTOOftM7v/f39HU1IRt2yxYuJhJV1/BXx55nHS3m0suuYj3v1x55Pf7D1guEovWLXVx/pRWio4MsnNT52O+IiKyf70aYr59+Te56srLO36e9vxTrF6zll/f/8Be81Y8Hg+hUIjGxsaO6559bhq3fGcKjz3yEMFQkNmz5+21fPpg5SKxxt9msukzFyNP8VGxxYEd7nwGvoiIfF2vhpgZr77GjFdfO+jt5s3/mHnzP97rOq/Xy6OPPdHpfQ5WLhKLNi13MeTYACUjApR+EbsrlUREYlFMzIkRSVbBLw+HHD7eT4pbxxGIiHSHQoxIlJWuctJUYzL+kjZMS5N8RUS6SiFGJMps22DJu2k4XXD8eV60WklEpGsUYkRiQMBnsGhmGgXFQY45ObKbL4qIJCqFGJEY0dpo8snbaRx9gp8BwyJ/FIaISKJRiBGJITXlDj6fm8Lx53nJLgxFuzoiIjHtsENMSkpKJOohIl/attrFlpVOxl/SRlqGViyJiHSmWyHmkosv4JTxJ3f8fOf3vsuzTz/BI3/+IwOKiiJeOZFk9cWCFBqqTMZf2obl1ERfEZH96VaIOe/cc2hobADgmKOP4qSTTuAvjzzO5i1buP66ST1SQZGkZBssfT8Nw4ATzteKJRGR/elWiMnO7ktVVQ0AY8cex+JPlrJo8SfMePV1hg0b2iMVFElWQb/B4plp5PYPMfJUrVgSEdlXt0KMz+fD7U4DYOSIY1i9Zi3QfgCjy6W5MSKR5mk2Wfx2KkPH+ikerhVLIiJ76tbZSWvWrmPyDdeyfv0GBg8uYcWKlQAU9e9PbW1tj1RQJNnVVThY/kEqY8/10tpoULuzV488ExGJWd3qiXn2uRcIBAKcdOIJPPn3Z2hsbAJg7NgxrPpidY9UUERgx3onmz5zcfLFXtxZWrEkIgLd7Impr6/nDw8+/LXrn3n2HxGrkIjs35pFLjKyw5xyaRvz/ukm6DeiXSURkajqVk+Mw+HA4fgq92RnZ3P+eecwYvgxEa+YiOzLYNm/UwmF4MQL2jAMrVgSkeTWrRDz43vu5txzzgLaN7n7f7/5HyZdfSX/ed9PmTjh9B6poIh8JRRsX7HUJy/MqDN80a6OiEhUdSvEHDFkCGvWrgPgxBOPp63Ny3dvv4snn3qWiy++oEcqKCJ787aaLH4rjSGjAgwepaXXIpK8uhVi0tJSaWlpAWD0yBEs/fQzQqEQq1atprCgoEcqKCJf11Bl8em/Uxkz0Uf+wGC0qyMiEhXdCjF19fUUFw/CMAyOPXYUq1evASA93U0goA9Skd60c5OTdUtcnHRRGxl9tWJJRJJPt1YnzZkzn7u/fwf19Q0EAsGOze6OHHoEO3fu7JEKikjn1i91kZkT5pRLPcydnk7ApxVLIpI8uhViXn9jJjt3VpCXl8uixZ8QCoUACNs2M996t0cqKCIHYvDZB6mccYWHky5qY+EbadhhBRkRSQ7d3vpzydJPv3bd3LnzI1IZEem+cMhg8VtpnDnJw5iJPj6fkwIoyIhI4ut2iOnfrx+XXXoRAwcOxMamrKycN2e+TWXlrp6on4h0gc9jsmhmGhOv9NBcb7L5c1e0qyQi0uO6NbF39OiR/OH3/8vgwSVs3LSJzZu3MGRwCX/43f8yauSInqqjiHRBU43F0vfTGHWaj8ISTbQXkcTXrZ6YayddxawPZvPc8y/sdf2UG6/n2muu4j//61cRrZyIdE/lVgdrFqVw4oVtzJvuprnOinaVRER6TLd6YgYNGsi/Z334tev/PetDiosHRaxSInLoNn7mZOdGB6dc2oYrTUuvRSRxdSvEtLV5ycvN/dr1+Xl5eNraIlYpETkcBsvnpNLWYjL+Yi+mpTOWRCQxdSvELF36KbfechNjjh2Ny+XC5XIxZsxobvnOVJYs+fqqJRGJDjts8MnbqaSkhxl7thdQkBGRxNOtOTHPT3uJO26/lZ/de89e1y9evIRpL7wc0YqJyOHxe00Wz0xj4lUeRp/h44uPU7BtLb0WkcTRrRDj8/l46M+PUlhQwICBRQCUlZVjWRb/7ze/5J6f/Lwn6igih6i5zmLhG25OuqiNPnlhlr6Xiq+tWx2wIiIx65A+zXZVVfHZZ5/z2WefU1VVjcvppKiof6TrJiIRUFdpMedlN4YJZ13rIadfKNpVEhGJCP1JJpIEfB6Tj19Lo2yDgzOu8HDEsX40T0ZE4l23d+wVkfhkhw2++DiVukqLced6ye4X4vPZqYSCmicjIvFJPTEiSWbnJifzXnHTtyDMxKs9pPfRXjIiEp+61BNz389/csDy1NTULj3Z+eedw8QJp1NcPIiNmzbz6/sfACArK5Mpk6/nmOFHk+52U1NTy2uvz2TBwkUd901JSeHWW6Zy/LhxBIMBZs+Zz0svT+9yuYh8pbneYt4rbsae6+Wsa1pZNiuVii3OaFdLRKRbuhRi6uvqD3qbip0VB71NQ0Mjb7z5NkOHDmHYsCM7rk9NTaV023ZefGk6tXV1jBw5nJ/++EdUVVezceMmAG6aOpmszCzu+sGPcLvd/OK+e6mvr+e992d1qVxE9hYMGCx9N5WhxwU46UIvG5eHWbvIpWXYIhI3uhRiHn/i7xF5siVL2zfEy8vbe9ffqqpqZr71TsfPq1evZdPmLRw17Eg2btyEy+XitFPH8z+/+l9aWz20tnqY+dY7nH/eObz3/qyDlh+Y8eWlJyTTl4HaGp8MNn+eQkOVxUkXtpFdEGLp+2l7lSePZGlrsrQT1NZE0XnbYnJib1paGoNLSnj99ZkA9O/fD4fDQWnpto7bbN1aysCBAzAM46Dltt35KoyMrL4EA4GItyGzT3bEHzNWqa3xz98KS94OMnpiDedc18bKuT4gMdu6P4n6uu4rWdoJamsicTg7H+qOuRBjmiZ33Xkba9etZ9UXqwFIS03F7/cTDn81AbHV48GyLFwu10HLfT5fp8/X0tRAwO+PaBsy+2TT3HjwIbhEoLYmkEaY908Xo06zOeGCSlbOT2XrKieJ/RdeEryuX0qWdoLammicLlenZTEVYkzT5Pt33k5KSgq/+/2fOq5v83pxuVyYptkRVNLdbkKhEH6//6DlB2YT2f0y9vzAT/R9ONTWRGOHYdVHqbQ1ZTHqtBpy+gX5fE4iL8NOjtc1edoJamsi6rxtMbPE2jRN7v7+HWT1yeL3f3iIwB5DPBUVlQSDQUpKijuuGzy4hLLycmzbPmi5iHTPrtJ05k5PJ7swzMSrtAxbRGJTr4YY0zRxOp2YpolpGDidTizLwrIsfviDO8nMyuR3v//T13pP/H4/CxYuZtLVV+B2u8nPy+OSSy5i9ux5XSoXke5rrrOYO91NS6PJmde00m9IMNpVEhHZS68OJ3378m9y1ZWXd/w87fmnWL1mLf+c8S9OPvlE/H4/Tz7xaEf5Rx8v5O9PPQvAs89N45bvTOGxRx4iGAoye/a8vVYeHaxcRLov6DdY8k4qR44NcPJFbWxY5mLtJy7QMmwRiQG9GmJmvPoaM159bb9lk6698YD39Xq9PPrYE4dcLiKHymDTchf1VSYnXdh+XMGn76Xi98bMaLSIJCl9ColIl9SWO5jzkhuHw+asazz0LdBp2CISXQoxItJl3laTj/7lZucWBxOu8jB4pE7DFpHoiakl1iIS++ywwar5qdRXWow9x0tO/xAr56USDGiejIj0LvXEiMghKdvgZO4rbrILw5x9XSt5A7R6SUR6l0KMiByy5jqLOS+7Kd/o5LRvtXHsBC+WQ8NLItI7FGJE5LCEQwarF6Yw/1U3BcUhzr6uldz+6pURkZ6nEBNpznRCqYXRroVIr6uvtJj9kpuKLQ5Ov6KNUad7MS31yohIz1GIiTR3Ib6Sb4IrM9o1Eel14ZDBFx+n8vG/0uh/RJCzr/WQ3U9LsUWkZyjERFrjFsy2Kozic6NdE5Goqd3pYPaL6VRtt5hwhYeRp/rUKyMiEacQ0wNclfMxcoZD1uBoV0UkakJBg5XzU1nwehoDhgW0QZ6IRJxCTA8w/XXYu5ZiDr4ADP2KJbnVlLf3ytSUW0y8ysPw8T4MU70yInL49A3bQ+yyeeBwYxSeGO2qiERdMGCwYm4qC99Mo3h4gLMmeeiTp14ZETk8CjE9JeTH3j4LY+CZ4EyPdm1EYkL1DgcfvpBOfZXFmZM8HH2SemVE5NApxPQgu2YVeHZpkq/IHoJ+g+UfprL4rTSGjAow8WoPmTnqlRGR7lOI6WHh0ncx8kZD5qBoV0Ukpuza1t4r01xrcta1Ho46wYdhqFdGRLpOIaaneXZh7/oUc/CFgA7IE9lTwGewbFYaS99NZeiYABOu8pCZrV4ZEekahZheYO+YC65MjMLjo10VkZhUscXJhy+k09rY3itz5Fg/qFdGRA5CIaY3hLzY2z/EGHQWONzRro1ITPJ7DT59P41P30/lqBP8TLjCQ3qfcLSrJSIxTCGml9jVn0NbLUbx2dGuikhM27nZyQfT3Hg9Jmdf18qIU3wKMyKyXwoxvShc+i5G/nGQMSDaVRGJaf42kyXvpLJsVir5A4OcP6WVM67wUDIigMOpYSYRaeeIdgWSSmsFdtVyzMEXEv7iKUAfxiKdM9i5ycnOTU4yskMUDw8yfLyPYyd62bnJwba1TmrKLDRhXiR5qSeml9k7ZkNKX4yCsdGuikjcaKm3WLMwhfeeSeeTt9MwTDj1sjbOn9rKMSdruEkkWaknprcF27B3zMEYdDZ23VoItkW7RiLxwzao2u6garsDZ4rNgGEBSoYHGH6yn5pyi21rnOzc5CAYUO+MSDJQiIkCu+ozjIKxGIPOwt76TrSrIxKXAj6D0i9clH7h6hhuGnGKjzFnarhJJFkoxESFTbj0PcyRU7F3fQaeymhXSCSutQ83WaxZ5KJgUIji4QFOvawNr8dg+1on29c68TRp9Fwk0SjEREtLGXb1CswhFxJe/Uy0ayOSGPY33DRCw00iiUohJorsHbMxxtyJkT8Gu3pFtKsjklD2HG7K7HS4SR+BIvFM/4OjKdCKXTYXY9A52HXrIOSLdo1EElJzvcXqhRarF7koKA5Rsnu4qdWgbL2D9ctsQoFo11JEukshJsrsyqUY+WMxBp6Jve39aFdHJLHZBlXbHFRtax9uGnRMgKOOb2LIsSG2rHSxeYUTf5vmzojEC4WYqLMJl76LOeJG7Orl4KmKdoVEkkLAZ7BlRQo12wrJKqhm2PE+ho3zs22Nk03LXbQ2KsyIxDr9L40Fzduxa77AHHxhtGsiknRs26Bsg5M5L7lZNDONjL5hzpvcykkXttG3IBTt6onIASjExAh7+weQ3g8jd1S0qyKSpAyqdzhY8Lqbua+4sW0482oPp1/uoaAkiI4JEYk9Gk6KFYEW7LJ5GCXnYTdsgJA/2jUSSVoN1RZL30tjdVaYI8f6GX9xGy0NJhuXuSjb6MAOa4m2SCxQT0wMsSuXQNCLMWBCtKsiIoCnyWTlvFTeeyadnZsdHDvBy/k3tjJ0jB9Lp2mLRJ1CTCyxw4RL38PodzKk5UW7NiLyJX+bybpPUnjvmQw2fuZi6Fg/F9zUwvDxPlxpOnxSJFp6dTjp/PPOYeKE0ykuHsTGTZv59f0PdJSlpKRw6y1TOX7cOILBALPnzOell6dHrDxuNG3Frl+HOfgCwmunRbs2IrKHUNBgy0oXW1c5GTAsyLDj/VrRJBJFvRpiGhoaeePNtxk6dAjDhh25V9lNUyeTlZnFXT/4EW63m1/cdy/19fW89/6siJTHE3vbLIwxd0DOcKhbG+3qiMg+dq9oKtvgoKA4xLBxfs6b3Er5Zgcbl7loqLKiXUWRpNCrfzYsWfopS5Z+SmNj017Xu1wuTjt1PC9Pn0Frq4fq6hpmvvUOZ581MSLlB2ZE+BKBx/Y3Y5d/jFlyPpiuHqhjDLU1bi5qa2JeDretJlXbnSx4PZ25r6TDXiuaQofxuLHWzni6qK2Jedm/mFid1L9/PxwOB6Wl2zqu27q1lIEDB2AYxmGX23bnE/AysvoSDER+v/HMPtmHdX/bsw4v43AecS6u6sURqlXPONy2xhO1NTFFoq0hP6xbBNtWBSge2cT4i1vxe03qK1Np2JVK/a4U2podHOgDuafpNU1Mid5Wh9PZeVkv1qNTaamp+P1+wuGvJsi1ejxYloXL5Trscp+v8zOJWpoaCPgju5w5s082zY31h/9AW94hfPTV+Mo+AW/d4T9eD4hYW+OA2pqYIt3W5kaoKjdxfZROYUmQvAF+ike2MeK0MG0tBjVlFjXlDmp2WrTUm/RWqNFrmpiSoa1Ol6vTspgIMW1eLy6XC9M0O4JIuttNKBTC7/cfdvmB2UR2E6s9P5AO83EbN0HDJszB3yC87sXDe6weEcG2xjy1NTH1XFv9bQY71jnZsa79r8jU9DB5A0LkDQhx5Fg/Y88J4201qNlpUVNuUVNm0VzXU6FGr2liSpa2dt62mAgxFRWVBINBSkqK2bq1FIDBg0soKy/Htu3DLo9n4W3vY465A7KPgvoN0a6OiBwib6tJ2QaTsg3toSYlLUzul6FmyKgAx53pw9dmUFv+Zagpt2isNcGO3vCTSKzr1Ym9pmnidDoxTRPTMHA6nViWhd/vZ8HCxUy6+grcbjf5eXlccslFzJ49D+Cwy+OarxG7fAFmyTfAiInMKSIR4Gsz2bnJycp5qcx+MZ23/5bB8g9T8DQbFA8PcNa1Hi6+tYXxl3o4cqyfvgUhDCO+/ygTibRe/Vb89uXf5KorL+/4edrzT7F6zVp+ff8DPPvcNG75zhQee+QhgqEgs2fP22t59OGWxzN750KM/DEYA07DLkuAYCYiX+P3GlRscVKxpb2nxumyyS0KkTcgyIBhAUae5iMUhNqd1pe9NQ7qq0wdgSBJrVdDzIxXX2PGq6/tt8zr9fLoY090et/DLY9rdrB9WGnYldjVK8DXEO0aiUgPC/gNKksdVJa2f0w7nDY5/duHn/oNCTJ8vJ9QCGrLLarLHFTvsGis6b2JwiKxQOMT8aJ+AzRuxSz5BuENr0S7NiLSy4IBg6rtDqq2O4AULEd7T03+wBADjwow6nQffq9BdZlF9Q6L6h0OWhsPvMeGSLxTiIkj4dL3MMd8DyPvWOyaldGujohEUSi4d6hxptjkDQiSPyjE0OMCjD3bh6fZaA80ZQ7aGoM0N0a71iKRpRATT3z12FtmYhxxGXbIB/Xro10jEYkRAd/ec2pS08PkDQxRMDDIiPE+3FnlNNeZX/XUlDkI+NRLI/FNISbO2DWrwErBHHYF4fUvQ+OWaFdJRGKQt9WkbL1J2fr2UFMwMBN3nzryBwU57iwfrjQvjdUm1Tssqsoc1O60CAUUaiS+KMTEIXvXp+1B5qir20+6bimLdpVEJKYZtDU7qSpzUbraCdhk5YbJHxQif1CQwaPbsCyoq7Q6emrqKi2tfJKYpxATp+ydC8BKxTzmOsJrngPPrmhXSUTihkFTrUVTrcXmz10Yhk3fwjD5A9vn1Bx1vJ9QALatdbJ1lYvWxl7dUkykyxRi4pi940OwXJjDbyC8+lnw1ka7SiISh2zboL7Sor7SYsOnYFo2/Y8IMmR0gPNubKV6h8XWL5xUbHGod0ZiikJMnLNL320fWtodZPxafiAihyccMijf6KR8o5PM7BCDRwUYe7aXMRMNSlc7Kf3CSVuLejEUh8IAACAASURBVGck+vQuTAD2ljehtQJz+A3gTI92dUQkgTTXW6z6KJX3ns5g9cIUCoqDfGNqK+Mv9VA4OAg6CkGiSCEmEdhhwhtfBX9je5CxUqNdIxFJMKGgwfa1TuZNT2fOK268LSYnXdDGN6a0ctQJPlLc4WhXUZKQQkyisEOE178CIT/m8OvBdEW7RiKSoBqrLT6fk8q7T2ewYZmLgUcFueCmVk68sI28gUFAvTPSOxRiEkk4QHjdS2BYmEdfo1OvRaRHBf0GW1e5mP2im49edRMOwqmXtXHu5FaOHOvHlaowIz1LISbRhLyE170ArgzMo64EQy+xiPQ0g7pKi2Wz0njv6QxKv3AxZJSfC25u4fjz2sjpF0K9M9IT9A2XiAKt7ZvguQswhn4LHQAnIr3F7zXYtNzFrH+ks/DNNEwHnHGFh7Ov9TBktB+HS2FGIkfjDYnK30R47TTMEVNhyMXYW9+Kdo1EJKkY1JQ5qClzkOIOUzIiwLDj/Yw6zceO9U42r3DSXGdFu5IS5xRiEpm3jvC6aZgjpkDIh719VrRrJCJJyOcx2fBpChuWuSgsDnHEsX7Oud7DrlKLjctd1JRZqMdYDoVCTKLzVBFe9yLm8MkQ8mKXfxTtGolIsrINdm1zsGubg8ycEEeODXDqN9toqjXZ9JmL8o0ObFthRrpOc2KSQUs54fUvYww4A6PfSdGujYgIzXUWyz9M5f1n0qna5mDMmV7On9LK0OP8OJyaNyNd8//bu+/4uKo77+Ofc6dJo2ZZMu69N4rpxRUCZJcADr04QIAsCQkJGx7yJLubJ5vsBgKk7S5sNhtCb45DMxAI4EKzjSEuuElWtaxiq2tG0+ee5487Hkm2imXLmqLf+/Wa14zvObo6h2Ojr8499x6ZiRkq2iowi1dhzLgGoiF0/dZEt0gIIQj6DHZtcFH0mZOJs8NMOy3ErLODVHzhpHSbg0C7/K4teiYhZihpKUaXvoqaeiU6GoSm3YlukRBCABANK8q2Oyn/wsHoqRGmLwgx7bQQVUV2SrY4aWuURcDiSBJihhjduNPaMHLacsyiELSWJrpJQggRp7WipsRBTYmdgjFRpi8IsexGHwf32dj7uZN6WQQsOpEQMwTpg3+zgsyMa60H43n2JbpJQghxGEVjjZ3GGjvZ+dYi4HMv9+NpNth7aBGwKWFmqJOLjUOUrt2Art2AMfMGyBqd6OYIIUSPvM02tq6xFgHXldk5ZZG1CHjaafLwvKFOZmKGML1/nTUjM+smzF1Pgr8h0U0SQogeBf0GuzdZz5uZMCe2CPisIOU7HJRucxLwyu/lQ42EmCFOV75jBZnZKzB3PgHBlkQ3SQghehWNKMpji4DHTIkw/fQQ005tZ3+xnb1bXOhwolsoBovEVoEuWw2eKuuBeI6cRDdHCCGOjlbUlDpYv9LNRy9nYnfCshvaWXBxHRNmy6WmoUBCjAA0ZskrEGjEmPM1cOUnukFCCNEPisZaO5vezOS9Z7Joa3Qx59wgf3eHl7P+zs/oKWEMmwSadCSXk4RFRzGLVmJMuxJj3u2YRS+BtyrRrRJCiH7xttgo+TyfLWs0hWMjjJsZYcFFAVBQU+KgqshOQ7UNZHuDtCAhRnTQEcy9q1Djl2HMWYEufR3duCPRrRJCiGOgaKi201BtZ/t6FyMnWoHmvMv9hAKK/cV2qooctNYbyHNnUpeEGHEEXbUGAk2oqVdAxnB09QeJbpIQQhwzM6qoLXNQW+bA7tSMmRJh3KwwS0/14W0xqCqys7/YQXurrLBINRJiRLd0/VZ0sMXaaykjH132BuhoopslhBDHJRJS7NvjYN8eBy63ybjpEcbNDDPn3BBNtQZVRQ7277UT8kugSQUSYkTP2iowd/wRY9aNqNk3YxavhIg/0a0SQogBEfQZlG5zUrrNSfYwk3Ezwkw9JcT8RUHqq2xUFTmoLbUTCcvlpmQlUVP0LtCIueNxUAbG3K9DxvBEt0gIIQact8Vgz6cu3n0mi/Ur3XiaDOadH+TLd3g581I/oyZHUIbc4ZRsZCZG9C3iw9z1NGrqFRhzv27NyMh+S0KItKRoOWij5aCNLz7SjBgXZfzMMGdc4kdHobrEQdl2h+yqnSQkxIijo6Pokpdh3BKM2Tejy95AN3yR6FYJIcSJoxX1VXbqq+xsXasZNTnChNlhlt3oo67CRvFmF011EmYSKalCTH7+MG67ZQWz58xCodhTVMwfn3iKpqZmDMNgxc03sGjh+Sil2LRpM48/8TSRSASgz3IxMPT+ddadS1Mus+5c8mxPdJOEEOKEM6OKmhIHNSUOcoZHmXF6iIVX+2issVH8mZOD+2zIrdqDL6nWxNx+2y3Y7Da+c8/3+da3v0cwGOQbd34dgOVXfoU5s2dx3/0/4rv33s+4ceO44fpr41/bV7kYOLphO+bu51CjziQ05mJQ8puIEGLo8DTZ+PzdTN59OgtPk8E5l/lZcp2PMVPDgKybGUxJFWJOGjmCDRs2EQgECIVCfPTxJ0wYPx6AZUuX8PIrr9Pc3ILH42HVn19hyeKFKKWOqrxnaoBfJ/LcSfTy7MPc8UfMzJMw5qwAe1bi23RCX0NkXKWvafoaKv0c3L762mxsW5fJO09mU19lZ8GXAlx0s48Js8IoYzD+Ww9eXxP/6l5SXU568823Oeecs/nblm2YpsmiC85ny5atuN1uCgsLqKiojNctK68gOzuLgoLh+Hz+XssbGhp7/J7ZucOIhAd+y9OcvKGw/5BGV6wiOO7LGCffgatqNUYovXfBHhrjapG+pp+h0k8Y/L7u2wk1e6OMn+Vh/iIPc84LU7kzl5q92ZjREztfkO7janc4ei4bxHb0qah4L0uXLubx/30MgMrKffzs3x8kMyMDAJ/fF6/r81mfMzMy0Kbutbw33rYWwqHQwHUC6y+Up7V5QM+ZrHLy8gl/8RRqymX4J16FWbwK2ioS3awTYqiNq/Q1vQyVfkJi+9pcDzs3ZDFpbojpp7cwaX4LpVuclH3hJBLqeUbhWA2FcXU4nT2WJU2IUUrxTz+6n40bP+XnDzwMwDVXL+eHP7iPnz/4CADuTDcej9f67HYD4A8E8AcCvZb3TjOw1zA7/yVN92ujsb7qKLr0VRi7CGPWjejyN9D12xLbtAE3BMcVkL6mi6HST0iGvkYjULrNSfkOB+NnhplxRogZZwQp2+6kZKtjAJ8GnPi+Do6e+5Y0a2Kys7M4acQI/vL2XwmFQoRCId5++12mT5+GzWbQ0NDIxEkT4vUnT5qI19tOY2MTPp+v13IxOHT1B+jS11CT/x41flmimyOEEAllRhWVu5y8+0wWW9ZkMHJShEtubefkRQEys81ENy8tJE2I8Xi81NbWccnFF+FwOHA4HFx66ZdoaGzE4/GyZu06vnrl5eTnDyMnJ4err17OuvUforWV0PoqF4NDN+7A3P0M6qQFqOlXgUqayT4hhEgMraje62DtC24+fSuTYSeZXHxLOwsu9JM9TMLM8UiqnzAP//I33LLiJv770d+iFFRU7uPhR34DwCuvriYnJ4dHHnoAw1Bs3PgpL770p/jX9lUuBpGnCnPH4xizbkDN+Rpm8UsQbk90q4QQIsEUByrtHKi0UzAmwowzQlx0czs1pXaKPnPSWi+Pq+ivpAox1dU1/PzBh7stM02TJ596liefevaYysUgCzZbm0fOuNbaqqDoRfDXJ7pVQgiRFBpr7Gx43U7eiCgzzgix9DofB6tsFH/upGG/PDjvaCXN5SSRhqIBzD3PotsqMebeBnlTEt0iIYRIKq31Njb/JZP3ns3C7zU47wo/l9zWzrzzA+SNiJLeC3aPX1LNxIg0pE102esQaMKYeQO6Ybu151Ka3oZ9TLLHovKmoOs2Q7Svu+mEEOnI22Kw5f0MdnzoYvTUCONnhpl2mg9vi0FVkZ39xQ7aW2Xe4XASYsSg0DUfob3VGCNPR826EcI+dOMOK9D4DiS6eYnhGoaacCFq+GwItqJOOh2z7HVoLUt0y4QQCRIOKfbtdrBvtwOX22Tc9AjjZoaZc26IpjqD/UUO9u+1E/RJoAEJMWIwtZVjtpWDzYUaPhtVOB81+lzwH0Q3xAJNqC3RrTzxbBmocYtQI8+E1jLM7f8DgUbU2IXWbNXBv6H3vQfmwD9JWgiROoI+g9JtTkq3OcnKMxk3I8zk+WHmLwxSv99GVZEDT/3QvrtJQowYfNEgun4run4rOHJQhXNRhfMxJlyIbqtEN3yBbtyVfpdWlA016izU2Asg2Iq553loK48X6/3r0c17MaZdiZr/DczSV8FbncAGCyGSRXurQdFmF0WbneQVmoybGWb2OUEy3FXUldupKrZTV27HjA6tBcESYkRihT3o2o3o2o2QOcKanRl7AWrSpdBSgtnwBTQXg44muqXHRRXMtR4AqGzoinesWafuFuy112B+8XvU+Asx5tyKrvkEXb0e9ND+bUsIcYiitcFGa4ONnR/D+BluCsY2cdrSAOoiqCmx1s/UV9nQOv0DjYQYkTz89eiqNeiqNZAz3pqdmXIZoNBNu1NzQXDOBIyJX4KMQnTNx+i6jWBGev8aM4KufAfdXIwx9XJU/jTMklflFnUhxGEULQcyqCrOZNt6FyMnRBk3M8zZf+8nElZUF9upKnbQXGeQrrdsS4gRyclThfZUoSvehmHTrECTSguCMwowJlwIw6ajD36O3vMCRHx9f11nbeWY23+HmnQpxvw7rYBXuwm55VIIcThtKuoq7NRV2LHZNaOnWAuCF13lw+9V7C92UFVkx9OUXg/UkxAjkps2obkY3VyMtrlQw2d1WhBcb62fSaYFwXY3atxi1MjTobkYc/vvINB47OeLBtGlr6GbizAmX4bKn2mtlQm2DlybhRBpJRqxQsv+YgfODM2YaWHGz4ww88wQniaD2nIbdeV2mmpT/5KThBiROqJBdP02a4fs7hYEN+5Ce/dbMzSDvYbEsKNGnYMacz4EGjB3PQ2efQN3/qY9mJ4qjClfwZh/l3W5qX7rwJ1fCJGWQgFFxQ4nFTucZGSbjJ4cYdTkCFNP9RMNQV2ltSD4QKWdSCj1Ao2EGJGauiwILrRmZ0YuQE26xAowvgNobw2011jv/gZOzGUYZX3v8ctAR9Hlb6Abd56A7wOE2zGLXkSNOBU16RJrVqb8DdmXSghxVAJeg/IvnJR/4cTm0Jw0PsLoKRFOWRzE4QrQUG2jttwKNb621HgOjYQYkfr8DeiqteiqtWA4IGsUKmsMZI9BjbkAI7MAHQ1Bex26vQa8NdZ7oOn4vm/eFIwJF4ErD139ofXE3UG4i0rXb0W3VWBMvQLj5Lswy96A5qIT/n2FEOkjGlbUljmoLXOA0gwfaTJqcoTJ88KcsjhIW6NBbZkVaJoOGJCkl50kxIj0Yobji4IhNvdic0HWGFT2aCvcTJiF4RqGjgSgvRZ9KNR4ayB0FGtNMk/CmHgR5E5GH9iM3v/B4D/TJtiCuetp1OhzMKZfhW7caS2CjgYHtx1CiNSnFU11NprqbOza4MKdawWa0ZMjTF8QIhxU1FVY62gO7LMTDSdPoJEQI9JfNAht5ei28o4LSna3NVOTNQaVPQY14hSUMxsdbo/P1GhvLbRXxy/XmPYs1JTzUSNOsW753vYYBJsT1i3Q6NoN6JYS6wF5J9+FWfpa6t2GLoRIKr42g7JtTsq2ObE7NSMnWutoTlsWwOaE+ior0NSV2/F7E3vZSUKMGJoiPmgpQbeUdAQbZ05sxmYMKnuc9XRdeyY62Aa+AwTyJqG8tZg7n0iuJ+n66zF3PI4auxhj1k3W7NC+NaD7eB6NEEL0IRJSVO91UL3XgVKa4aOj1sLgU8KcujRIS71BXbmd2jI7LQcH/3k0EmKEOCTkgVARurmoI9i4hqGyx4B7NK72vfj3f05SPqdFm+j9a9EtezGmXoEaNtV6QF57baJbJoRIE1orGmvsNNbY2fkxZOWZjJ5izdLMOCNEQ7WNj19xD2qbJMQI0ZtgCzrYAo27seflJ7o1ffPut7YtmHARxtyvWwuOaz6SbQuEEAOuvdWgZIuTki1OHC6NM2Pwf8GTECNEujHD6Iq/xLYt+Aoqfzrm/g/Aux8i/kS3TgiRhsJBRTg4+At+JcQIka5aSzG3/Q418WKMactR9gx0oAntrbYWL3urob1O1s4IIVKWhBgh0lk0gC57HV32OmQUoLLHWndlFc5DTbjIquM7EAs0NWhP9fFtkyCEEINIQowQQ0WgER1ohIbt1tJkZbMeDJg9FrLHdjwYMBIgEKxHtVTGZm2qIexNdOuFEOIIEmKEGKp0FLzVVlAhds+VPROyxuIsnErUPRJ10mkoRxY62Bq/BKXbq8FbC2Yooc0XQggJMUKIDhE/tJbioIlAazOgY7eZx2Zrhs9EjV9szeL46619qbw1aH+9tT9VxJfoHgghhhAJMUKI3sVvM98ZuwxlQOZJHetrRp6ByixAGXZ02GftZeVvsN4D1jvBlkT3QgiRhiTECCH6R5vgq0P76uDg57FH/ylwDbN2FM8stN4LZqMyC62nHpth8DfGw0086AQaB2XTTCFEepIQI4QYANraRyrYjG7Ze+iIxZFlhZqMWLjJGQ8nnYbhykNr05ql6Tx7cyjoDPammkKIlCMhRghxYoXbIdyObqsEOoUbw9Exc5MRex82HZUxHGXY0CEvBBrQ/qZYQIpd1go0y9obIQQgIUYIkShmGNpr0bH9neLhRhngyofMAlTmCMjIR+VNsRYYu/JQykBHQ9YMTrAFHegUcIItVuCJyp1TQgwFEmKEEMlFm9ZamUAjurnYOhQvVODKBVc+yjXMCjsZ1iadyjUM5cyx6od9HSEnFmx0oCV+TNbhCJEeJMQIIVKIhmArBFu77CXeMYtjtxYYZwyLhxzlyoe8yVbIsWda9UNtELACTogAylGLDjRBsMm6/CWESAkSYoQQ6UNHINBgraU5dKhzuc1lzd64hqEyrJBjZo9EZU+1Qo5hi12qao6FnGYINMfem6wAJbM4QiQNCTFCiKEjGgRfnXWLOACKjLx8PK3NVrkzBzKGW7M3GdZL5Yy3FhvbM9FaQ6g1FnCarEXGwWZrXU6gWe6oEmKQSYgRQggANITaINSGpqLzUYstwwo1hwKOK7bgOCMf5cy1FhxH/J1mbmILjmPnJNgmIUeIASYhRgghjkY00OVuKjj8jqrYGpxDASezEIZNRTnzUA63VT8aigcaHWqFkAdCrehgWzxAEQ0Oft+ESFESYoQQ4nhp01ozE2hCt8YOdS5XduuuKmcuymm948pFZY2C/BnWreOHFh1Hgh0zQvFZnFZ0LPAQbJPNN4WISboQs2DBqVx79VcZPXoUfn+AN996m9VvvIVhGKy4+QYWLTwfpRSbNm3m8SeeJhKJAPRZLoQQCaMjHSGn8+HOdQyHFW6cuahY4MGZi8oaA8NnWTM69ozY6QIdQSfY2inoWO+E2mQBshgSkirEnHLKfO68/VYefez37Nq9B5fLSWFBAQDLr/wKc2bP4r77f0QkEuX+++7lhuuv5Zlnnz+qciGESGpmuOP5OJ0Odw06zo5ZHGcuuPKsoJMzAQrzrLU5Nqf1dSFvl6ATNiIoW13sMlYrhLyHn12IlJNUIea6a67i5VdeZ8fOXQD4/QGq9lcDsGzpEp5+5nmam63dcFf9+RXu+c63ePa5F9Ba91neMxV7nQgn6rzJSPqanqSvSeVogo49s9OMTl489ETdBahhc1GOHOtWcm12Wp/TdthsTpsVdCL+Qe7gQEuBMR0w6dzXnvuWNCHG5XIyefIkPtmwkV8+/ADZOdkUF+/lyaeew+/3U1hYQEVFZbx+WXkF2dlZFBQMx+frvbyhobHH75udO4xIODzg/cnJyx/wcyYr6Wt6kr6mshDohtgzc7qWaBTa7kbbs9GObLQjx3rPHonOn4rpyAZ7llXZjKAiXlS4PfbuRUU6PhsRL0T8qCSc0Um/Me1ZuvfV7nD0XDaI7ehVVlYWhmGwaNEFPPiLX9La1sYtK27k+/d+h1/+6j8A8Pk7Nn3z+azPmRkZaFP3Wt4bb1sL4dDALpLL6fzciTQnfU1P0tf007WfTb1XVrbYbE5ObCFyTsfn7FHWnx3Z1m3lZhTCHgh5YouPD83keGIzPB6rXJsnvI+HDJUxhaHRV4fT2WNZ0oQYv9/6deHtt9+lvqEBgBdXruIPv38MM/aX353pxuPxWp/d1i2L/kAAfyDQa3nvNAN7XbjztFfy/XYysKSv6Un6mn762U8dsbZgCPayEBkFzuyOS1fOnK6LkWOhx7p0pSF8aI3OoXDjhbAXHW63ysJea8uH4w47Q2VMYej0tee+JVGI8XOwvr7L+pXOS1kaGhqZOGkCBw4eBGDypIl4ve00Njahte61XAghxEDTsefceIDqXtbouDstRu4UdHILwJGFcmRb78r6gazDPivMhL3oQ8Em7IVQ58DTHtvjKp1/cIujkTQhBuC999by5S9fzPbtO2jzeLju2qsoLS2jubmFNWvX8dUrL2fv3hIikShXX72cdes/jIeevsqFEEIkQMRnveJbPViOmNVxuMGRbV2mcmRZn52xgOMeGQ888QcHat0RZroEnnYidg0c7AhAYd8R31Gkh6QKMa+vfpOsLDcPPvBTlDLYU1TML3/9nwC88upqcnJyeOShBzAMxcaNn/LiS3+Kf21f5UIIIZJVp0DCgSPiRtcHBxpwKOR0DjyOLHDlobLHEnblYozM7HiujtZWkAq3Q9jXJfAQaY/N8HR6mQN/s4c4MZIqxGitef6FlTz/wsojykzT5MmnnuXJp57t9mv7KhdCCJEGtNnpMtaR8ysaRdahxa7xwJMVm8mJfbbH/pxZ2OW4UoZ1jmioS6jRh2Zzwt5Y6PHHZphi76Y8VDVRkirECCGEEANGRzv2pKK7wHMYe2Z8Vkd1Cj/Ys1Duk8AxOVbmRtlcHecxw1agCfvjwUZHOj4T8R8WfGIvucR13CTECCGEENARLvz1fQceZVihx+6OvWei7G5wZHb8ObOwa5k9E2XYOs7ZOdCEDwUfX6c/d3yOry0axFvVU4GEGCGEEKK/tNlpHU/s0OFVuvs6wxkLOrFQc1gQil/msrutRcz2zK6zPpFgR6AJ+wgaUdSwlnjQiQehePDxp3XwkRAjhBBCDBYzBMGQtVEn3QedI2d9bFbQcbitcGPP7PiclW8tcM4cYQWeQ8dtHQ+I05FAl0tZOhz7HA3Ewo4f3elzvCwFSIgRQgghkpmOPRU5fPhiZoUzL59gazP68Oij7J1mfNzxcBOf8bFnojKGd7rclQn2jI7FzVp3Xb8T8Xe9/NXdsbBn0Bc5S4gRQggh0o2O9HoXV0/HsGV0DTqdPmPPsAJRD+FHe/Zj7vzjiezVESTECCGEEMISDVivoLUf09GHH1dC1t5IiBFCCCHE8YkGE/JtjYR8VyGEEEKI4yQhRgghhBApSUKMEEIIIVKShBghhBBCpCQJMUIIIYRISRJihBBCCJGSJMQIIYQQIiVJiBFCCCFESpIQI4QQQoiUJCFGCCGEEClJQowQQgghUpKEGCGEEEKkJAkxQgghhEhJQ34Xa4fDOcBnVNgdDhxOJz1sWJ5GpK/pSfqafoZKP0H6mn56+zk9ZEPMof8o1674ZoJbIoQQQoi+OBxOwqFQl2NqwXmXpm9864M7K5twONR3RSGEEEIkjMPhxNfuPeL4kJ2JAbr9DyKEEEKI5HL4DMwhsrBXCCGEEClJQowQQgghUpKEGCGEEEKkJAkxQgghhEhJEmKEEEIIkZKG9N1Jx8owDFbcfAOLFp6PUopNmzbz+BNPE4lEjqtusrHb7Xz9tq8xb+4c8vJyaW5u4a/vvs9bf3mn2/rfvOtOLjj/3C59++nPHqC0rHywmnzM+tv2VB7Xp574fZc/2+12qmtquP8H/9xt/VQa14u/dCGLF13AhAnj2VtSyk9/9kC8zOVycecdt3L6ggVEImHWrP2AF15c2eO5+lt/sPXU19zcHG5ZcROzZs8ky+2moaGRV15dzcefbOjxXD/+lx8yY/o0otFo/Nh3vvt92to8J7wfR6O3ce1v25N5XHvqZ0FBAb965IEudR0OB1u2buPhR37T7bmSfUwHioSYY7D8yq8wZ/Ys7rv/R0QiUe6/715uuP5annn2+eOqm2xsNoOWllb+/YGHOHiwngkTxvOjH/4fmptb2LBxU7df89d33+epp58b5JYOjP60PZXH9ZbbvtHlzw/94t/45JPux/OQVBnXlpZWXnv9TaZOncz06dO6lN126wpyc3L59j334na7+ecf/YDm5mbefufdbs/V3/qDrae+ZmRkUFG5j+dfWEljUxNz587m/vvu5WB9PXv3lvR4vudfWNnjLyiJ1tu4Qv/anszj2lM/Gxsbu/y7tdls/O6x3/b57zaZx3SgyOWkY7Bs6RJefuV1mptb8Hg8rPrzKyxZvBCl1HHVTTbBYIiVf/ozBw4cRGtNZeU+tmzZxsyZ0xPdtIRL5XHtbOrUKYwbO5b1H3yY6KYMiE83f8anmz+jtbWty3Gn08n5553DiytX0d7uo76+gdVvvMWypYu7PU9/6ydCT309eLCe1W+8RWNTEwA7d+6mpLSMGd388E8VPfW1v5J9XI+2n2eeeTqGYfDp5s8GqWXJS2Zi+sntdlNYWEBFRWX8WFl5BdnZWRQUDKehofGY6qYCwzCYOXMGq1e/2WOdRQvPZ9HC82luaWHdug9486130Do1Hgp9tG1Pp3FdtmQRW7dup7m5pdd6qTyuAKNHj8Jut3cZs/LyCsaNG4tS6oi+9Ld+MsvMzGTSxIm8+urqXustX345V331CuobGnnrrbf54MOPB6mFx+9o254u47psyWI++ngD4XC413qpPKZHS0JMP2VmZADg8/vix3w+X5eyY6mbCm67dQV+n4/1H3zUbfnbcIeF9AAAB3lJREFUb/+VZ597Ea/Xy7RpU/nePXdjmjolpjP70/Z0GVen08l5553Do4/9vtd6qTyuh2RmZBAKhTBNM36s3efDZrPhdDoJBoPHVT9ZGYbBt+/+B3bvKeKLHTt7rPfCCyvZX11NKBRm3rw5fO+eu/H7A2z+7PNBbO2x6U/b02FcCwsLmD9/Ls89/2Kv9VJ5TPtDLif1kz8QAMCd6Y4fc7vdXcqOpW6yW3HzDcycOZ0HfvFIl4VinZVXVOLxeNBas3dvCa+9/gbnnXv2ILf02PSn7ekyrueecxbBYIi/bdnaa71UHtdD/IEATqcTw+j4X16W2000GiXUzePM+1s/GRmGwXfuvguXy8Vv/+PRXuvuLSnF7w8QjUbZtu0L3nt/bcqMcX/ang7jumTxIsorKqncV9VrvVQe0/6QENNPPp+PhoZGJk6aED82edJEvN52GhubjrluMrvlazdx8vx5/OzffoHHc/T7TZna7LtSkuqt7ekyrsuWLmH9Bx91+a30aKTiuNbW1hGJRJg4sWPMJk2ayP7q6m4vIfS3frIxDIPvfudb5Obl8tDDv+7zssPhTDP5+9iT3tqe6uOqlGLJ4oWsWbuu31+bymPaGwkxx2DN2nV89crLyc8fRk5ODldfvZx16z/s9h9Bf+omo1tvuZn58+by0397EI+n91vzzjnnLDIzrcspU6ZM5orLL2PTp6mx8Ky/bU/1cR09ehQzZkxj3boP+qybSuNqGAYOhwPDMDCUwuFwYLPZCIVCfPzJRq679ircbjcjCgu57LK/Y82a9d2ep7/1E6GnvtpsNr53z93k5Obwi4d+1ecMg9vt5tRTT8bpdKKUYt7cOVx04VI2fbp5kHrSt5762t+2J/u49tTPQ06eP4+cnBw+/nhjr+dJhTEdKGrBeZemxv91k4hhGHxtxY0svOB8DEOxceOn/PHJZwiHw9xx+60A/OHxJ/usm+wKCwt49D9/fcQ15N17injwF788oq8/+fGPmDBhPDabjaamZtauW8/qN/6SEj/Y+2p7Oo0rwE03Xse0aVP515/+/IiyVB7Xq69azjVXL+9ybOeu3fz0Zw+QkZHBHbffYj0fJBphzZr1XZ4P8n9/8H327Cnm1desBbB91U+0nvr6p1Uv85Mf/9MR/24//OiT+Jh27mtOTg4/uP8fGTtmDAD1DfW89Ze/HlXAHSw99fXXv/mvPtueSuPa299fgO99925CoTCP/feR69hSbUwHioQYIYQQQqQkuZwkhBBCiJQkIUYIIYQQKUlCjBBCCCFSkoQYIYQQQqQkCTFCCCGESEkSYoQQQgiRkiTECCGGtBGFhbz0wtPMnDkj0U0RQvSTbAAphEiYb951J0sWLzzieCAQ4JbbvpGAFgkhUomEGCFEQu3evYdf/7brBoU6BfdnEkIMPgkxQoiEikSitLa2dlv243/5IQcP1tPa2sayZYux2+xs3LipyxYPNpuNa6+5ikULzyM3N5e6ugO8/MrrfPzJhvh5XC4X1193NWeddQZ5ubk0t7Tw/vvr4o+iB8jPH8b9993LvHlzaGlp5U+rXubDjz45sZ0XQhwXCTFCiKR29llnsmHjJn7yk39n5KiTuOsbdxAMhXjyqWcBuP66a1i6ZBF/ePwJKiurOPvsM/n23f9Aa2srO3buAuAH9/8jhQUFPPHkM+zbV0XB8OGMHjO6y/e58fpref7FlTz1zHMsW7qEb951J3tLSqmrOzDofRZCHB0JMUKIhJozZxZPPdF1Q7udO3fz0CO/BsDb7uV///AEWmuqa2p4aeUqbrt1BS+8uBKt4cuXfomnn3mejZusHXpffW01U6dOZvnyy9mxcxfz5s5h7pzZ/PCf/h9lZeUAHDxYz+49RV2+5zt/fY+NGz8F4KWVq7j0kouYN3eOhBghkpiEGCFEQpWUlPLoYbvyhkKh+OfS0rIuO2YXFe/F4XAwcuRIABwOB7t37+ny9bt3F3HFFZcBMGXKJLxebzzA9KSisjL+2TRNWlpbycvLPbZOCSEGhYQYIURChUJhDhw4eNT1VTfHdDfH6BR8dLcVuopEokecVCl5CoUQyUz+hQohktrUKVNQqiO6TJ8xnXA4zIEDB6irO0AoFGLO7Fldvmb27JlU7a8GoKysgpycbKZMmTyo7RZCnHgSYoQQCWW328jLyzvidUh2dja3f/0Wxo4Zw2mnncJ113yV99esIxgMEQqFePudd7n2mqs45+wzGTVqJFde8RXOOH0Br75q3Xm0Y+cudu/ew3fv+RZnnL6AESMKmTljOsuWLk5Ul4UQA0QuJwkhEmr27Fn8/nf/ecTxO77xLQA2fboZvz/Av/7kn7HbbWzctJnnnn8pXu/Fl1ZhmppbvnZT/Bbr/3r0f+J3JgE8+NCvuOH6a7jj9lvJycmmqamZ995fe8L7JoQ4sdSC8y49iqvFQggx+H78Lz/kQN0B/ud//5jopgghkpBcThJCCCFESpIQI4QQQoiUJJeThBBCCJGSZCZGCCGEEClJQowQQgghUpKEGCGEEEKkJAkxQgghhEhJEmKEEEIIkZL+P0Q3SRxEuSfjAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from jupyterthemes import jtplot\n", "jtplot.style()\n", "\n", "def plot_loss_history(history, n_epochs):\n", " fig, ax = plt.subplots(figsize=(8, 8 * 3 / 4))\n", " ax.plot(list(range(n_epochs)), history.history['loss'], label='Training Loss')\n", " ax.plot(list(range(n_epochs)), history.history['val_loss'], label='Validation Loss')\n", " ax.set_xlabel('Epoch')\n", " ax.set_ylabel('Loss')\n", " ax.legend(loc='upper right')\n", " fig.tight_layout()\n", "\n", "plot_loss_history(history, 20)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The validation loss was pretty unstable early on but was really starting to converge toward the end of training. We can do something similar for the learning rate history." ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGhCAYAAACQ4eUqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xl8VNX9//H3nY1kyBBiwmpkUZDFtfxaF5BF3FsXQEQREFxqa62i1WJLrXVpi2utttb2q4iiKCiKigvWFllcWGy1KnUBIahsISGBSWYmyZ3c3x+BKZEsdzJLZnk9H4/7CNxzcud87pHk4znn3mMMGXqmJQAAgDTjaO8GAAAAtAVJDAAASEskMQAAIC2RxAAAgLTkau8GpBNvxzzV1dW2dzMAAMgqbrdHgeqqA86TxNjk7ZinCVOuau9mAACQlZ598uEDEhmSGJv2jcA8++TDcR6NMZTXqbOq9lRKytSn3Ykxc2RDnNkQo5QdcRJjJnC7PZow5aomf/eSxESprq5WdbXxTWLMurq918zM/wCJMZNkQ5zZEKOUHXESY6ZjYS8AAEhLJDEAACAtkcQAAIC0RBIDAADSEkkMAABISyQxAAAgLZHEAACAtEQSAwAA0hJJDAAASEskMQAAIC2RxAAAgLSU1L2THA6HpkyeqBHDh8kwDK1evVaz58yVaZpR122t/PTTTtHIESepV69DtH7Dl7r9jlmNrt+hQwf98Ipp+n9Dhsg067T0rRV6Zv6zib8JAAAgLpI6EjN2zDkaPGigbpwxU9Ovn6Hi4mJNvGhCm+q2Vl5ZuVsvvfyqXnv9jSavf+m0Kerk66SfXnu9Zt58q044/ns684zT4hswAABImKSOxIw+eZTmPvm0KioqJUkLn1+ka6/5iZ6a94wsy4qqbmvla9a+L0kqKio8oB0ej0fDhp6g39z2O1VXB1RdHdDiV17T6aedoiVvvNlKFMbeIz6MnsNU06mbjMJ47oydemo8noyPsdYRlna/rnj+95HasiHObIhRyo44iTF9NR9X0pIYr9eroqJClZRsjpzbuKlEeXkdVVh4kMrKym3XDQSCtq/VlB49usvlcjX6/k2bSlRcfLAMwzggodpfXqfOMuvqooq9JXUdOsiy6uVxJzWfTL4Mj9FyeGTmD1Be+b9k1Mfvv49U5csvaO8mJFw2xChlR5zEmN5cbnfzZclqRG5OjiQpEAxEzgUCgUZlduta9ZbtazXXltraWtXX10fOVQcCcjqd8ng8qqmpafZ7q/ZUqq42jiMKu5fKl18g/+6K+F0zBWV8jC6vnN8doKrqGql2d3u3JqEyvi+VHTFK2REnMaY/t8fTbFnSkphgKCRJ8uZ65fdXNfzZ621UZrduNNdqri0ej0cOhyOSyHT0ehUOh1XbaoJi7T3iZf9hsnheN5VkQYxmsOGrK1eqrWzftiRUFvRlVsQoZUecxJgZmo8raQt7A4GAysrK1btPr8i5vn16q6qqWuXlu6KqG821mrJt23aZpqnevf/3/X369NY3W7a0OJUENM+SwjWSq/WRQABAfCT16aSlby3TuDHnqqCgs3w+n8aPH6tly1c2mTi0Vre1cofDIbfbLYfDIYdhyO12y+l0SpJqa2v1zrurdOGE8+X1etWlqEhnn/19LV26PHk3AxnHCNc0jMQAAJIiqSstF724WD6fT/fePUsOh6FVq9Zo/oLnJElXXD5NkvTo7MdbrWunfNzY83TB+LGRvz81d7bW/ffTyPtiHn/iKV1x+VQ99Kf7ZYZNLV263MaTSUAL6kMyXDkZO6ALAKnGGDL0TH7m2uD2eDTpsuma99gD8V3YK2O/RVmZ2hXZEaP7qGkyy7+QtfWd9m5MAmVHX2Z+jFJ2xEmMmaCl379sOwDECdNJAJBcJDFAnBjhkORkYS8AJAtJDBAv4RoZjMQAQNKQxABxYoRDPGINAElEEgPEiVHPmhgASCaSGCBeeNkdACQVSQwQJw3TSYzEAECykMQAcWKEa2Q4O0gG/6wAIBn4aQvES3jv5qM8Zg0ASUESA8SJEa5p+ANTSgCQFCQxQLxYdbLqwyQxAJAkJDFAnBiSZAZJYgAgSUhigHgKN+xkDQBIPJIYIJ4YiQGApCGJAeKJJAYAkoYkBogjy2QnawBIFpIYIJ4YiQGApCGJAeLJZGEvACQLSQwQT2FGYgAgWUhigHgy2QQSAJKFJAaII8sMsrAXAJKEJAaIJxb2AkDSkMQA8WSGZDicksPT3i0BgIxHEgPEkxls+MoTSgCQcCQxQDyFQw1fmVICgIQjiQHiyaqXZdaQxABAEpDEAPEWDjKdBABJQBIDxJsZlMFIDAAkHEkMEG9mSHKSxABAopHEAPHGu2IAIClIYoA4s0zWxABAMpDEAPEWDrEmBgCSwJXMD3M4HJoyeaJGDB8mwzC0evVazZ4zV6ZpRl031vJuXbtq2rTJOrx/P5nhsJYtW6H5CxbKsqzk3RBkJjModezZ3q0AgIyX1JGYsWPO0eBBA3XjjJmafv0MFRcXa+JFE9pUN5ZywzD0859fp6+//kY/uupazZz5G33n2GN0ztnfT+wNQHZgOgkAkiKpSczok0fphUUvq6KiUn6/XwufX6RRI4fLMIyo68ZS3rNnD/Xs0UPPPveCTNNU+a5devW1JTrt1NE2ojDifCTy2qlyZFeM/3vZXXu3ib4kxmyPkxgz52ha0qaTvF6viooKVVKyOXJu46YS5eV1VGHhQSorK7ddNxAIxlTeVNJkGIa6du2i3NwcBYOhZuPI69RZZl1dm+9Dc3z5BXG/ZqrJlhjDHZyqcXkzOt5Mjm2fbIhRyo44iTG9udzu5suS1YjcnIbh9UAwEDkXCAQaldmta9VbMZVv2bpNO3aU6sIJ5+vZ515Q5/x8nXXm6Q3lubktJjFVeypVV1trO247fPkF8u+uiOs1U01WxVjnkbO3R/49uyWrvr2bFXdZ1ZcZLhviJMb05/Z4mi1LWhITDDUkBt5cr/z+qoY/e72NyuzWjbW8vr5ed99zv6ZeMkkP/el+VVVV6a1lK9Sr1yGqrq5uJRJr7xEv+48KZeqi4iyLcd9O1s4OkhlounrayrK+zNgYpeyIkxgzQ/NxJS2JCQQCKisrV+8+vbSjtFSS1LdPb1VVVau8fFdUdS3LiqlckrZt3647774v8pmnn3aKvty4STU18R1lQRbal8S4cjMwiQGA1JHUhb1L31qmcWPOVUFBZ/l8Po0fP1bLlq9s8rHm1urGWt6r1yHq0KGDHA6HjjrqCI0be54WPLsweTcDmau+VpZVzxNKAJBgSX1PzKIXF8vn8+neu2fJ4TC0atUazV/wnCTpisunSZIenf14q3XjUX7C8cfp9NNOkdvt1tatW/W3R2br44/XJfweIEuw9QAAJJwxZOiZmTqJFlduj0eTLpuueY89EOeFvcZ+i7IytSuyL0bHMT+R9c0KWeWftHfD4iz7+jJzZUOcxJgJWvr9y7YDQCKYIUZiACDBSGKARGA6CQASjiQGSAB2sgaAxCOJARKBkRgASDiSGCARwiEZJDEAkFAkMUAiMBIDAAlHEgMkAmtiACDhSGKABLDMkORkJAYAEokkBkgEppMAIOFIYoBEMIMyHE7J4W7vlgBAxiKJARIhHGr4ymgMACQMSQyQCGaw4StJDAAkDEkMkAhWvaxwDUkMACQQSQyQKGZIcvKYNQAkCkkMkChmkLf2AkACkcQAicIL7wAgoUhigEQxQ6yJAYAEIokBEsTihXcAkFAkMUCihENMJwFAApHEAInCwl4ASCiSGCBRmE4CgIQiiQESxDKDvCcGABKIJAZIFJ5OAoCEIokBEsUMynDlSDLauyUAkJFIYoBECe/bBJIpJQBIBJIYIFHYyRoAEookBkiUcK0sq54kBgAShCQGSCQeswaAhCGJARLJDMrgMWsASAiSGCCReMwaABKGJAZIJKaTACBhXMn8MIfDoSmTJ2rE8GEyDEOrV6/V7DlzZZpm1HVjLS8o6KxLp07RoMEDZcjQZ59/ocfmPKFduyqSd0OQ8Rp2smY6CQASIakjMWPHnKPBgwbqxhkzNf36GSouLtbEiya0qW6s5ZdfOlVOl1PXXHuDfvLT61RTU6Mrf3hZ4oJHdgoznQQAiZLUJGb0yaP0wqKXVVFRKb/fr4XPL9KokcNlGAe+0bS1urGWd+3WRe+9t1qhUEi1tbV6+5131euQQ2xEYcT5SOS1U+XI4hjN0N6drNu7ffQlMWZjnMSYOUfTkjad5PV6VVRUqJKSzZFzGzeVKC+vowoLD1JZWbntuoFAMKbysrJyvfrqEp1wwvH69wf/UX19vUacNEwffPBhq3Hkdeoss64u1ttxAF9+QdyvmWqyMcY6l6Fwhzx1zLDYs7EvM1U2xEmM6c3ldjdflqxG5OY0rAsIBAORc4FAoFGZ3bpWvRVTuSR9/sV6nXzySM1+5C+SpM2bv9Idv7uz1Tiq9lSqrra21XrR8OUXyL87s9fiZGuMhnuXjE4DMyr2bO3LTJQNcRJj+nN7PM2WJW06KRgKSZK8ud7IOa/X26jMbt1Yyw3D0K9mztD69Rs09dIrNfXSK/XxJ+v0y5tutBGJFecjkddOlSN7Y7TMwN41Me3dPvqSGLMxTmLMnKNpSUtiAoGAysrK1btPr8i5vn16q6qqWuXlu6KqG2t5Xl5Hde3SRa8v+btqa2tVW1urJUveVP/+/eTz5SXwLiDrmCGeTgKABEnqwt6lby3TuDHnqqCgs3w+n8aPH6tly1fKsg7MslqrG0u531+lbdu264zTT5Xb7Zbb7daZZ56msvJy+f1VybwlyHRmUIbDJTman9MFALRNUt8Ts+jFxfL5fLr37llyOAytWrVG8xc8J0m64vJpkqRHZz/eat14lN9z3x81dcokPfzQAzIMqWTzV7rn3j8m/B4gy4T37WSdI9XGf0E4AGQzY8jQM5ufbEKE2+PRpMuma95jD8R5Ya+x36KsTO2KLI7RcMh5/M0Kf/RXKVDabq2Lnyzuy4yTDXESYyZo6fcv2w4AiWTVywrX8sI7AEgAkhgg0cygxE7WABB3JDFAopnBvW/tBQDEE0kMkGgm+ycBQCKQxACJFg6SxABAApDEAAlmmUFeeAcACUASAyQa00kAkBAkMUCimUEZTpIYAIg3khgg0ZhOAoCEIIkBEo3pJABICJIYIMEaFvaSxABAvJHEAIlmBmW4ciQZ7d0SAMgoJDFAooVDDV9ZFwMAcUUSAySaGWz4ypQSAMQVSQyQaOEaWVY9SQwAxBlJDJAMZoidrAEgzkhigGRgJ2sAiDuSGCAZwiEW9gJAnJHEAMnAu2IAIO5IYoAk4IV3ABB/JDFAMrB/EgDEHUkMkAxmiJ2sASDOSGKAZGA6CQDijiQGSAamkwAg7khigCSwzBAjMQAQZyQxQDKEmU4CgHgjiQGSwQzKcLgkh6u9WwIAGYMkBkgGdrIGgLgjiQGSwQw1fOUxawCIG9tJzKiRw3XnrDs0Z/bf1KVLkSTpnLO/r+OO+27CGgdkDCssK1zLSAwAxJGtJOaU0aM0ZfJErVnzvlwupwzDkCT5/X6defqpiWwfkDlMNoEEgHiylcScecZpeuTROXph0UsKh8OR8xs3lai4uDhhjQMyihmUwUgMAMSNrUclunfvpg1fbjzgfE2oRrm59v/P0uFwaMrkiRoxfJgMw9Dq1Ws1e85cmaYZdd1Yy5+Y83+Nb4TLpS1bt2rGTTfbjgeICo9ZA0Bc2RqJqaioVPdu3Q443//wfiot3Wn7w8aOOUeDBw3UjTNmavr1M1RcXKyJF01oU91Yy6deemWjY8vWrXr33dW2YwGixnQSAMSVrSRmxcp3NGXyRPXo3l2WJXk8Hh177NGaNHGC3lq2wvaHjT55lF5Y9LIqKirl9/u18PlFGjVyeGSNTTR1Yy3f32GHHarigw/W8hUrbURhxPlI5LVT5SBGyZAV2T+pvdtKXxJjtsRJjJlzNM3WdNLzL7yoLl2K9If77pQk3XPX7yRJy5at0Cuvvm7nEvJ6vSoqKlRJyebIuY2bSpSX11GFhQeprKzcdt1AIBhT+f6fJUmjR43Qhx9+pIqKylbjyOvUWWZdna2Yo+HLL4j7NVNNtsdY67RkufPVIQPuQ7b3ZSbJhjiJMb253O7my+xcwLIsPfzXR7Rw4SIdemhfGQ5DmzaWaEdpqe1G5OY0DKMHgoHIuUAg0KjMbl2r3oqpfH8ej0dDh56gh/7SeI1Mc6r2VKquttZWXbt8+QXy766I6zVTDTFKRsdKGZ3yVZvm94G+zBzZECcxpj+3x9Nsma0k5vxx52nxK69rZ1mZdpaV/e/CbrfOPef7ev6Fl1q9RjDU8LIvb65Xfn9Vw5+93kZlduvGWr6/E084TjU1tfr3Bx+2fiMkSdbeI172HyaL53VTCTFK2ruTdW7z5WmBvswc2RAnMWaG5uOytSZm/PljlZPT4YDzHTp00Pjzx9pqQiAQUFlZuXr36RU517dPb1VVVau8fFdUdWMt39/ok0dp+Yq3VV9fbysOoM14OgkA4sr2G3utJhKhHt27qbo6cGBBM5a+tUzjxpyrgoLO8vl8Gj9+rJYtXymriYu3VjfWcknq0aO7Dj+8n5ZFsTgZaCuLp5MAIK5anE760wP3RX7pz/rdbY1GKxwOhzp3zteq1Wtsf9iiFxfL5/Pp3rtnyeEwtGrVGs1f8Jwk6YrLp0mSHp39eKt141EuSaNPHqnPPv9C27Zvtx0D0GaRl90ZytxhXwBIHmPI0DOb/Wk6dsw5MgxDF4wfp5cXv6rQfutJTDOs0tKdWvv+v7JiKsbt8WjSZdM177EH4ryw19hvUVam/mIjRklSh85yfudahdfeLYVDTddJefRl5siGOIkxE7T0+7fFkZhFLy6WJJWV79J7761WXQIeLQayhhls+OrKTeMkBgBSh72X3a14mwQGiFW4RpZVz+JeAIgTW49YO51OjR1zjoYNPVFFRYVyuRp/28RJ0xLRNiDzmCGSGACIE9uPWJ8yepTe/MdSSdKCZxdq6VvLVVVVpTmPP5nI9gGZxQzK4AklAIgLWyMxQ4cer0cefVz//uBDTbhgnN57b412lJbqm2+26PDD++nvb/4z0e0EMkOYkRgAiBdbIzEFnTtr81dfS5JqamqU6234Ifyvf3+gId85NnGtAzKNyQvvACBebCUxFRWVys/vJEnaubNMgwYOkCQdckixwlnweDUQLxZJDADEja3ppE/W/Vf/b8h3tHHjJi1dtkKXTZuioSeeoF69irVi5TuJbiOQOcyQ5GRNDADEg60k5pFH50T+vHTpMgWqqzVw4ACtfPsd/eOfbyWscUDGMYMyvJ0y9JVUAJBctpKYb1u1eq1WrV4b77YAmY/pJACIG9sbQDalf/9++s0tM+PVFiDzhdkEEgDipdWRmKOOPEJHH32UTNPU0qXLtLOsTD26d9fkyRdpyHeO1ccfr0tGO4GMYEU2gQQAxKrFJOakk4bq6quuVFV1tfI6dtTJo0bosTlzddWPf6h///sD/WLmLdq8+atktRVIf7yxFwDipsUk5gdnnaFnn3tei15crKEnHq9rfnqVxp8/Rjffcpu2bNmarDYCmcMMynC4JMMlWWZ7twYA0lqLa2K6d++md95dJUl6b9Ua1dfXa+6TT5PAAG21/07WAICYtJjE5OTkKBQKSZIsy1JdXZ1Kd+5MSsOAjBRJYljcCwCxanVh72GHHarqqmpJkmEY6tunjzrn5zeq88X6DYlpHZBprLCscB0jMQAQB60mMTNuvL7R36+bfvUBdSZOmha3BgEZj3fFAEBctJjEXDP9hmS1A8ge4aAMVw5v7QWAGLWYxJSVlSerHUD24DFrAIiLmN7YC6ANmE4CgLggiQGSzDKD7GQNAHFAEgMkG9NJABAXJDFAsrF/EgDEBUkMkGzhIC+7A4A4aPU9MZL0wysubbrAslRbV6dt27brvVVr5Pf749k2IDOxsBcA4sJWEtOjezf16dNbTqdL27ZtazjXo7tMM6xt27dr5IiTdOGE83XLrb9lXyWgFZYZYjoJAOLAVhKzavVa1dTW6s8P/VXV1QFJUseOXl191Y/0wYf/0YqV7+j66T/VlEkTdefd9yW0wUDaizydZEi88g4A2szWmphzzvm+npn/XCSBkaTq6oAWPLtQ5517tmpqarTw+UXqe2ifRLUTyBxmSIZhSM4O7d0SAEhrtpKYzvn5cjqdB5x3ulzq1MknSdq9e49yOvBDGWhVZCdrppQAIBa2kphPP/tcl027RF26FEXOde3aRdMumaTPPvtCknTwwT21s6wsMa0EMkk4JMuySGIAIEa21sQ88sgc3XjDdD34x3u1x++XLEudOnXS5q++1oN/fliS5HA69PzzLyW0sUDGCId4zBoAYmQridlZVqabfvlrHXXUETq4Z09J0pYtW/XxJ+sidf797w9bvY7D4dCUyRM1YvgwGYah1avXavacuTJNM+q6sZZL0pAhx2rC+HHq0aO7gsGQXn1tiRa/8pqdWwLEZu8L71jWCwBtZyuJ2efjj9fp44/XtV6xGWPHnKPBgwbqxhkzZZphzbjxek28aIKefOrpqOvGWn7MMUfph5dP00N/+T/999PP1KGDR0WFhW2ODYgK74oBgJjZfmNvt65dNXr0KI0be67OH3deo8Ou0SeP0guLXlZFRaX8fr8WPr9Io0YOb3hSI8q6sZZfeMH5emHRy/pk3X9VX1+vYDCkr7/ZYiMKI85HIq+dKgcxHnCYIcmZmwLtpi+zM8ZsiZMYM+domq2RmJNOGqqrfnSF6urqtHv3noZFiXtZkp5/ofW1MF6vV0VFhSop2Rw5t3FTifLyOqqw8CCVlZXbrhsIBGMq9/v96tu3j959b5Xuu2eW8nx5+uKL9Xr8iXkqL/9fO5qS16mzzLq6VuONli+/IO7XTDXE+D81RlhGx87ypOk9oS8zRzbESYzpzeV2N19m5wIXjB+rV159XfMXLGyUwEQjN6dhEWMg+L93zQQCgUZlduta9VZM5fX19XI4HBox4iTdedd92r1nj6ZOuVg3XH+NZt58a4txVO2pVF1trc2o7fHlF8i/uyKu10w1xNiYUbBHcrhUk4b3hL7MHNkQJzGmP7fH02yZrSSmc35n/XPpsjYnMJIUDIUkSd5cr/z+qoY/e72NyuzWjbk82FC+ZMmbkcfC5z+7UI/+319UUNBZFRWVLURiKb5vWd1/mCxTl3kS4wHMoAxvF1lpdz/oy8yRDXESY2ZoPi5ba2I+WbdOffr0jqkJgUBAZWXl6t2nV+Rc3z69VVVVrfLyXVHVjbU8GAyqdOfOxtNimdr3SE0s7AWAmNkaiVm58h1NmnihCg86SJs3f3XAI9FfrN9g68OWvrVM48acq/XrN8g0wxo/fqyWLV/Z5AhPa3VjLf/HP97SWWedro8++kR7/H5dOOF8ffnlxlZGYYA4CQd5TwwAxMhWEjP92qslSZdMubjJ8omTptn6sEUvLpbP59O9d8+Sw2Fo1ao1mr/gOUnSFZc3XOPR2Y+3Wjce5S8vflUdO3p156zbZRgOffb5F7rv/j/ZigOIlbX3PTEAgLYzhgw9s9WJlKKilt+fsv+TRZnK7fFo0mXTNe+xB+K8sNfYb1FWps5pEeMBfL3kGDRJ9WtmJbxl8UVfZo5siJMYM0FLv39tjcRkQ5ICJJUZlOFwS4ZLsg58YzUAoHXNJjGH9++n9Ru+lGVZOrx/vxYvYndNDIC9IjtZ50h1Ve3bFgBIU80mMbfderN+dNU12rPHr9tuvbnFi9hdEwNgL3PvawVcuSQxANBGzSYx10y/QXv2+CN/BhBHlimrvo7HrAEgBs0mMfuvg2FNDJAAJo9ZA0AsbO9ibRiGunXrqvz8fDm+tWHjp599HveGARnPDMlw5Wbo8wQAkHi2kpi+fXpr+rVXq1u3rk2WsyYGaAMzuHcnawBAW9hKYq64fJpKS3fqzw/9VbsqKnhFPxAPZog1MQAQA1tJTHFxsX7xy19r2/btiW4PkDUs1sQAQExsbQC5detW+Xx5iW4LkF3YBBIAYmIriZnz+JO66KIL1LvXIYluD5A9wiH2TwKAGNiaTvrNLTPlcDh056w7ZFnWAbtOT5pyWUIaB2Q0ppMAICa2kpi/PfJYotsBZB+mkwAgJq0mMU6nUzkdOmjt+/9SRUVlMtoEZAXLDMlwMhIDAG3V6pqYcDisyZMuktPpTEZ7gOzBSAwAxMTWwt6Nm0rU6xAW9QJxZQZlGIbEaAwAtImtNTEvvrRYUyZNVG5urr7cuFE1NTWNyplmAtrADDZ8deVI4VD7tgUA0pCtJOamn/9MkvTTq3/UZDnbDgBtEK5peNLPlSvV8D8CABAtW0nM7b+dleh2AFnIahiBYV0MALSJrSTm00/ZpRpICDMow5XDTtYA0Aa2kph9OnfOV5eiIrlcjb/t089IcoA24QklAGgzW0lMfn6+pl9zlQYNGthkOWtigDYyQ5KTJAYA2sLWI9ZTL7lYTqdLv/zVb1RbW6vf/v4u/fmhv2nbtu2adee9iW4jkLEsRmIAoM1sjcQMHjRI9973R5WUbFZ9vaXy8l1at+5T1dTWaNzYc/XRx58kup1AZgqH2D8JANrI1khMTk4HVe7eLUkKBALy+XySpM0lX6lv376Jax2Q6cwgO1kDQBvZSmK27yhVt25dJUnfbNmiEcOHyeVy6aSTTpTf709oA4GMxnQSALSZrSRm2bIVOqS4WJL00kuvaMTwk/TkE4/qgvHj9NLLryS0gUBGM5lOAoC2srUmZskbb0b+/N9PP9PPbrhJhx12qLZt366vv/4mYY0DMp1lBmXwdBIAtElU74nZp3zXLpXv2hXvtgDZh+kkAGgzW9NJkjRq5HDdOesOzZn9N3XpUiRJOufs7+u4476bsMYBGc8MyXC6JcPZ3i0BgLRjK4k5ZfQoTZk8UWvWvC+XyynDMCRJfr9fZ55+aiLbB2S2yE7WjMYAQLRsTSedecZpeuTROVq1eq3OPef7kfMbN5Xo4okX2v4wh8OhKZMnasTwYTIMQ6tXr9XsOXNlmmbUdWMtv+rHP9RJw05s9Nm33zFLX27cZDseIGY7RyiwAAAgAElEQVTh/ZKYuqr2bQsApBlbIzHdu3fThi83HnC+JlSj3Fz7T1aMHXOOBg8aqBtnzNT062eouLhYEy+a0Ka6sZZL0t/f/KemXnpl5CCBQdLVm7LqTZ5QAoA2sJXEVFRUqnu3bgec7394P5WW7rT9YaNPHqUXFr2siopK+f1+LXx+kUaNHB6ZnoqmbqzlbWfE+UjktVPlIMYWDzMoubwpEAN9mT0xZkucxJg5R9NsTSetWPmOpkyeqD8+8JAsS/J4PDr22KM1aeIEvfraG3YuIa/Xq6KiQpWUbI6c27ipRHl5HVVYeJDKyspt1w0EgjGV7/usEcOHacTwYaqorNSyZSv06mtvyLKsFuPI69RZZl2drZij4csviPs1Uw0xNi1o1aqDr1Cu+vS5P/Rl5siGOIkxvbnc7ubL7Fzg+RdeVJcuRfrDfXdKku6563eStPcX/xJbjcjNaRguDwQDkXOBQKBRmd26Vr0VU7kkLVnydz01b76qqqrUr99huu7aq1Vfb+m111tOyqr2VKquttZWzHb58gvk310R12umGmJsnqOmWqHaellpcn/oy8yRDXESY/pzezzNltlKYizL0sN/fUQLFy7SoYf2leEwtGljiXaUltpuRDAUkiR5c73y+xsWMHq93kZlduvGWi5Jm/YbpVm/foNeevkVjRg+rNUkRrL2HvGy/zBZPK+bSoixRWZw75qYdLg39GXmyIY4iTEzNB+X7ffESNLOsjKtXrNWq1at0Y7SUvXo0V333TPL1vcGAgGVlZWrd59ekXN9+/RWVVW1yst3RVU31vKm1Fv1tuIA4s0KB1nYCwBtEFUS820et1s9e/awXX/pW8s0bsy5KijoLJ/Pp/Hjx2rZ8pVNrkNprW6s5SeccFzkyapDD+2r8849W6vXvB/L7QDaxgzxnhgAaIM2bTvQVoteXCyfz6d7754lh8PQqlVrNH/Bc5KkKy6fJkl6dPbjrdaNR/mZp5+qK6+4VE6nU7t2VejNN/+pV159PeH3ADiAGZSRW5SxA8EAkChJTWLq6+v1+BNP6fEnnjqgbF/yYqduPMpvvf330TUeSBT2TwKANolpOglAHJhBycmaGACIVosjMTN/+fMWvzknhx+8QKwsMySDkRgAiFqLSUzFrtafO9+2dVvcGgNkJZOnkwCgLVpMYh7+26PJageQvcygDMMhOTtI4Zr2bg0ApA3WxADtLbz3ZY9MKQFAVEhigPZmksQAQFuQxADtzpLFC+8AIGokMUAqMIMyeMwaAKJCEgOkAl54BwBRI4kBUgHTSQAQNZIYIAWwkzUARI8kBkgFTCcBQNRIYoBUwNYDABA1khggFbD1AABEjSQGSAVmUHIyEgMA0SCJAVKAxUgMAESNJAZIBTxiDQBRI4kBUkE4KMPpkQxne7cEANIGSQyQCsxgw1emlADANpIYIBWwkzUARI0kBkgF9XWy6k2SGACIAkkMkCp4zBoAokISA6QKMySDNTEAYBtJDJAq2D8JAKJCEgOkCpIYAIgKSQyQIqxwiEesASAKJDFAqmAkBgCiQhIDpAozJIMkBgBsI4kBUgUjMQAQFZIYIFWYQcnJmhgAsIskBkgRDQt7GYkBALtIYoBUYQZ5OgkAouBK5oc5HA5NmTxRI4YPk2EYWr16rWbPmSvTNKOuG2v5Pm63W/fe/Xv5fHm67IqrEn8TgOaYQRmGQ3J2kMI17d0aAEh5SR2JGTvmHA0eNFA3zpip6dfPUHFxsSZeNKFNdWMt32fCBeerfNeu+AcLRMsMNnxlSgkAbElqEjP65FF6YdHLqqiolN/v18LnF2nUyOEyDCPqurGWS1LfPr31ne8crZdeeiWKKIw4H4m8dqocxGjrMPeOvjhzUyAe+jKzY8yWOIkxc46mJW06yev1qqioUCUlmyPnNm4qUV5eRxUWHqSysnLbdQOBYEzlZWXlcjgcuvKHl+mxx+ZGFUdep84y6+racgta5MsviPs1Uw0xti4QrpE3v4uc7tSeTqIvM0c2xEmM6c3ldjdflqxG5OY0LFgMBAORc4FAoFGZ3bpWvRVTuSSd/YOz9NVXX+u/n36mwYMG2o6jak+l6mprbde3w5dfIP/uirheM9UQoz2OuoCCNWFZKXyv6MvMkQ1xEmP6c3s8zZYlLYkJhkKSJG+uV35/VcOfvd5GZXbrxlrerWtXnXH6qbrpl79uQyTW3iNe9h8mi+d1Uwkx2mYGJVeH2K6RUPRl5siGOIkxMzQfV9KSmEAgoLKycvXu00s7SkslNaxJqaqqVnn5rqjqWpYVU/mI4cPUqZNP993z+4ab4HIpNzdXf3v4Qd33hwf1xfoNybotQGO8KwYAbEvqI9ZL31qmcWPO1fr1G2SaYY0fP1bLlq+UZR2YZbVWN5byd99brQ//83HkswYc3l9X/fgKzfjFr1VVVZWcmwE0wWLrAQCwLalJzKIXF8vn8+neu2fJ4TC0atUazV/wnCTpisunSZIenf14q3VjLa+rq9Pu3bsjdauqqmRZVqNzQLvghXcAYJsxZOiZmTqJFlduj0eTLpuueY89EOeFvcZ+i7IytSuI0fZVDhktI7dQ9V8813rldkFfZo5siJMYM0FLv3/ZdgBIJWZw73tiAACtIYkBUokZYjoJAGwiiQFSCAt7AcA+khgglYRJYgDALpIYIJWYIRlOj2TwTxMAWsNPSiCVsJM1ANhGEgOkkn1JDE8oAUCrSGKAVFJfJ6s+zEgMANhAEgOkGt7aCwC2kMQAqSYclMFIDAC0iiQGSDUmO1kDgB0kMUCqYToJAGwhiQFSDG/tBQB7SGKAVGMGJScjMQDQGpIYINWYIRb2AoANJDFAqmE6CQBsIYkBUg0LewHAFpIYIMVYYR6xBgA7SGKAVMN0EgDYQhIDpBozKMNwSE5Pe7cEAFIaSQyQasxQw1dGYwCgRSQxQKoxgw1fnSQxANASkhgg5Viy2D8JAFpFEgOkonCIx6wBoBUkMUAqMoO8tRcAWkESA6QippMAoFUkMUAq4l0xANAqkhggBVnsZA0ArSKJAVIRO1kDQKtIYoBUxCaQANAqkhggFYVZEwMArSGJAVKQxcJeAGiVK5kf5nA4NGXyRI0YPkyGYWj16rWaPWeuTNOMum6s5ZdfNlVDhhwrb65XoVBQq1av1VPz5iscDifvhgDNMXnZHQC0JqkjMWPHnKPBgwbqxhkzNf36GSouLtbEiya0qW6s5UveeFM/u+EmXXr5jzTjF79W716H6Lxzz05c8EA0zKAMZwfJYLAUAJqT1J+Qo08epRcWvayKikr5/X4tfH6RRo0cLsMwoq4ba/mWLVtVU1Mb+TxLUvfu3WxEYcT5SOS1U+UgxqiPyE7W3hSIjb7MzCMb4iTGzDmalrTpJK/Xq6KiQpWUbI6c27ipRHl5HVVYeJDKyspt1w0EgjGV7/us8849W2PHnKPc3Fzt8fs17+kFrcaR16mzzLq6mO5FU3z5BXG/ZqohRvssw62gpI6du8pRWxmXa8YTfZk5siFOYkxvLre7+bJkNSI3p2F+PxAMRM4FAoFGZXbrWvVWTOX7vPTyK3rp5Vd0cM+eGj58qCorW/9lUbWnUnW1ta3Wi4Yvv0D+3RVxvWaqIcboOerDqg7WSVWpdd/oy8yRDXESY/pzezzNliVtOikYahge9+Z6I+e8Xm+jMrt1Yy3/ti1bt6pk81f6yY+vtBGJFecjkddOlYMY23RE3hXT3rHRl5l5ZEOcxJg5R9OSlsQEAgGVlZWrd59ekXN9+/RWVVW1yst3RVU31vKmOBwOm2tigCQJh2TwhBIANCupC3uXvrVM48acq4KCzvL5fBo/fqyWLV8pyzowy2qtbizlHTp00KiRwyOjM4ccUqzzx56n/3z0cfJuBtAa3hUDAC1K6ntiFr24WD6fT/fePUsOh6FVq9Zo/oLnJElXXD5NkvTo7MdbrRt7uaVhw07UlMkT5XK5tHv3Hq1e876eW/hCEu4CYBNJDAC0yBgy9MzmJ5sQ4fZ4NOmy6Zr32ANxXthr7LcoK1O7ghjbdMXDxkhmUNbmN+JyvfigLzNHNsRJjJmgpd+/vEkLSFWMxABAi0higFRlBmWQxABAs0higFQVDjESAwAtIIkBUlXkPTEAgKaQxAApyjIZiQGAlpDEAKmKhb0A0CKSGCBVmUEZhkNyNL9vCABkM5IYIFWZwYavjMYAQJNIYoBUFd67WSlJDAA0iSQGSFVWvSyzhieUAKAZJDFAKguzuBcAmkMSA6QyMyiDkRgAaBJJDJDKeFcMADSLJAZIZbwrBgCaRRIDpDDLDEpOppMAoCkkMUAqC4fYyRoAmkESA6QyppMAoFkkMUAqYydrAGgWSQyQwtjJGgCaRxIDpDIzKDlJYgCgKSQxQCozQzJcHSSDf6oA8G38ZARS2b6drHnMGgAOQBIDpLJ9SQzrYgDgACQxQCqrr5Vl1ZPEAEATSGKAVMe7YgCgSSQxQKpjJ2sAaBJJDJDqeFcMADSJJAZIdWwCCQBNIokBUpzFmhgAaBJJDJDqSGIAoEkkMUCqC4dY2AsATXAl88McDoemTJ6oEcOHyTAMrV69VrPnzJVpmlHXjaXc5XLpsksv0ZFHDFZ+fidVVFTq72/+U6+9/kYybwdgDyMxANCkpI7EjB1zjgYPGqgbZ8zU9OtnqLi4WBMvmtCmurGUO50OVVbu1u9m3a1pl/1I9z/wZ5133tk68YTjE3sDgLYgiQGAJiV1JGb0yaM098mnVVFRKUla+PwiXXvNT/TUvGdkWVZUdWMpr6mp1bPPPR/5rM2bv9IHH/xHAwb013urVifjVgC2WWZIhis3RRIZQ5YzZ29brFZrp6dsiFHKjjiJMen2bZWSJElLYrxer4qKClVSsjlybuOmEuXldVRh4UEqKyu3XTcQCMZUvv9nSQ1TTwMGHK7Fi1+1EYmx90iERF03lRBj1Gr9Mtwd5fzuz+N73TYKSnK2dyMSLBtilLIjTmJMrvAXC6Vdn8b5qs3/TE1aEpOb07AwMRAMRM4FAoFGZXbrWvVWTOXfdum0KQoGAlq+4u1W48jr1FlmXV2r9aLlyy+I+zVTDTG2Va2s9Y/JMlLlxxQANMGSjLBfRpx/Drrc7ubL4vpJLQiGQpIkb65Xfn9Vw5+93kZlduvGWr6/KZMnasCA/rrjt3cqHA63GkfVnkrV1dbajtsOX36B/Lsr4nrNVEOMsUqde0dfZo5siJMY05/b42m2LGlJTCAQUFlZuXr36aUdpaWSpL59equqqlrl5buiqmtZVkzl+0y9ZJKOPGKwbv/tnZFkp3WW4jvvuP8wWQrMZyYEMWaObIgzG2KUsiNOYswMzceV1KeTlr61TOPGnKuCgs7y+XwaP36sli1fecCiXjt1Yy2fNnWyjjryiL0JjD95NwEAAMRFUp9OWvTiYvl8Pt179yw5HIZWrVqj+QuekyRdcfk0SdKjsx9vtW6s5UVFhTrrzNNVW1urPz94X+R7Pv3sc9151//+DgAAUpcxZOiZmTr+FFduj0eTLpuueY89EOc1McZ+85mZ2hXEmDmyIc5siFHKjjiJMRO09PuXbQcAAEBaIokBAABpiSQGAACkJZIYAACQlkhiAABAWiKJAQAAaYkkBgAApCWSGAAAkJZIYgAAQFpK6rYDmcDtbn43zbYx5HK79+7SmZlvWyTGTJINcWZDjFJ2xEmMmaCl37skMTbtu4kTplzVzi0BACD7uN2eA7YdYO+kKHg75qmuLp77JgEAgNa43R4FqqsOOM9ITBSauoEAACCxmtt4mYW9AAAgLZHEAACAtEQSAwAA0hJJDAAASEskMQAAIC3xdFKCORwOTZk8USOGD5NhGFq9eq1mz5kr0zRjqptKXC6XLrv0Eh15xGDl53dSRUWl/v7mP/Xa6280Wf+qH/9QJw07sVFct98xS19u3JSsJkct2jana18+Mef/Gv3d5XJpy9atmnHTzU3WT5e+PP20UzRyxEnq1esQrd/wpW6/Y1akrEOHDvrhFdP0/4YMkWnWaelbK/TM/GebvVa09ZOluRg7dfJp6pRJGjhogDp6vSorK9eiFxfrnXffa/Zat/z6lzq8fz+Fw+HIuWum36A9e/wJj6MlLfVjtG1O1X6Umo+zsLBQf7h3VqO6brdbH3z4H91z7x+bvFaq9mW8kMQk2Ngx52jwoIG6ccZMmWZYM268XhMvmqAnn3o6prqpxOl0qLJyt343626Vlu5Ur16HaOYvf66Kikq9t2p1k9/z9zf/qSfmzktyS2MTTZvTtS+nXnplo7/ffddv9e67TffhPunQl5WVu/XSy6/qsMP6qn//fo3KLp02RZ18nfTTa6+X1+vVzTNvUkVFhZa88WaT14q2frI0F2NOTo5KNn+lp595VuW7dumIIwZpxo3Xq3TnTq1fv6HZ6z39zLPN/o9Ie2mpH6Xo2pyq/Sg1H2d5eXmjf6NOp1N//csDrf4bTcW+jBemkxJs9Mmj9MKil1VRUSm/36+Fzy/SqJHDZRhGTHVTSU1NrZ597nnt2FEqy7K0efNX+uCD/2jAgP7t3bR2k659ub/DDjtUxQcfrOUrVrZ3U2K2Zu37WrP2fe3evafReY/Ho2FDT9D8ZxequjqgnTvLtPiV1zT65JFNXifa+snUXIylpTu1+JXXVL5rlyRp3bpPteHLjTq8iSQg1TUXY7RSuR8l+3F+73v/Tw6HQ2vWvp+klqUeRmISyOv1qqioUCUlmyPnNm4qUV5eRxUWHqSysvI21U11DodDAwYcrsWLX222zojhwzRi+DBVVFZq2bIVevW1N2RZqf3yaLttzpS+HD1qhD788CNVVFS2WC8d+3KfHj26y+VyNeqrTZtKVFx8sAzDOCCOaOunotzcXPXp3Vsvvri4xXpjx56r88edp51l5XrttSVasfKdJLWw7ey2ORP6UZJGjxqpt995T3V1dS3WS8e+tIskJoFyc3IkSYFgIHIuEAg0KmtL3VR36bQpCgYCWr7i7SbLlyz5u56aN19VVVXq1+8wXXft1aqvt1J6uDOaNmdCX3o8Hg0deoIe+sv/tVgvHftyf7k5OaqtrVV9fX3kXHUgIKfTKY/Ho5qampjqpxqHw6GfXv0jffrZ5/r4k3XN1nvmmWf1zZYtqq2t05FHDtZ1116tYDCkte//K4mtjU40bU73fpSkoqJCHXXUEZr39PwW66VjX0aD6aQECoZCkiRvrjdyzuv1NiprS91UNmXyRA0Y0F+z7rq30UKy/W0q2Sy/3y/LsrR+/Qa99PIrGnri8UluaXSiaXMm9OWJJxynmppa/fuDD1usl459ub9gKCSPxyOH438/Cjt6vQqHw6pt4jXn0dZPJQ6HQ9dc/WN16NBBDzz4UIt112/4UsFgSOFwWP/5z8f6xz/fSvl+jabN6dyP+4waOUKbSjZr81dft1gvHfsyGiQxCRQIBFRWVq7efXpFzvXt01tVVdUqL9/V5rqpauolk3T0UUfqjt/eJb/f/j5T9VZ965VSTEttzoS+HH3yKC1f8Xaj/1O1I936ctu27TJNU717/6+v+vTprW+2bGlySiHa+qnC4XBo+jU/Uaf8Trr7nvtbnX74tvr61I2tOS21OV37cR/DMDRq5HAtfWtZ1N+bjn3ZEpKYBFv61jKNG3OuCgo6y+fzafz4sVq2fGWT/1CiqZtqpk2drKOOPEK3//ZO+f0tP7p3wgnHKTe3YVrl0EP76rxzz9bqNam9MC3aNqdzX/bo0V2HH95Py5ataLVuuvSlw+GQ2+2Ww+GQwzDkdrvldDpVW1urd95dpQsnnC+v16suRUU6++zva+nS5U1eJ9r6ydRcjE6nU9dde7V8nXy66+4/tDrS4PV6deyxR8vj8cgwDB15xGCdesrJWr1mbZIiaV5zMUbb5lTuR6n5OPc5+qgj5fP59M47q1q8Tir3ZbwYQ4aemfo/VdOYw+HQJVMu1vCThsnhMLRq1Ro99viTqqur0xWXT5MkPTr78VbrprKiokI99Kf7D5hj/vSzz3XnXfcdEOett8xUr16HyOl0ateuCr21bLkWv/J6Sv+Cb63NmdKXkjTp4gvVr99huu323x9Qlq59Of78sbpg/NhG59b991Pdfscs5eTk6IrLpza8LyRsaunS5Y3eF/KLm27QZ599oRdfalgI21r99tJcjM8tfEG33vKrA/59rnz73Ug/7h+jz+fTTTN+poN79pQk7Szbqdde/7utpDbRmovx/j/+udU2p0s/Si3/9ypJ102/WrW1dfrLwweuWUuXvowXkhgAAJCWmE4CAABpiSQGAACkJZIYAACQlkhiAABAWiKJAQAAaYkkBgAApCWSGAAx6dmzhxY8M1eHHto3qu/728MP6uwfnJWgVqUHt9utBc/M1fHHfa+9mwKkJd4TA2S4Bc/MbbG8dOdOXXPtDW2+vmEY6tSpk/x+f1TbFPh8PtXU1CRlr5q/PfygFr/yul559XVJ0l/+/Ee98fd/6KWXX0n4Z+9z229u1tfffBN5wdw++fn5qq6ulmmaSWsLkCnYxRrIcFf++JrIn/v1O1QzbrxeM3/1G5Xt3cepucTD6XQ2u4nn/izL0u7du6NuV2vbU6QLu/epOW25dwAakMQAGW7/X5JVVdWSpD17/Af88vzbww/qzX8sVUFBgU44/nv6+pstuvW23+nsH5ylESOGqVvXbgoEA1q37r+a++TT2rOnIQnp2bOH7r/vLv3yV7/Rxo2bIn+/7w8PavTokRo8aKAqKio1f8FCvbdqdaPP23905G8PP6glb7ypgoICnTTsRNXV1WnZ8pWav2BhZBuDDh08unTaJTr+uO+pvj6slW+/q3A4rGOOPko3zphp63789vZbVFh4kC6eOEEXT5wgqSHR2717t3r27KGLJ16oIwYPVH19vTZ8uVFzn3xaW7ZslSSdeurJmjJpov5w/5806eILdfDBPXXnXfdpZ1mZJl98kfr3P0y5ubnavn2HFr24OBLv9Gt+ooEDD9fAgYfrtFNHS5Ju/vVtKtn8lZ6aO1t/uP9Pkf1sDjqoQFOnTNLRRx8pp9OpL9Zv0JNPPh3ZrfiYY47SzF/8XLffMUsXTjhfffv20fbtO/T4E09p3X8/jcQ5/vwxGjlyuAo6d1YgENTGTZt09z33R72pJ5DKSGIARPzg+2fp5cWv6Fe/vk1OR8OGc5Zl6Ym581RaulMHFRTokikX66c/+bF+f+c9LV7r4okT9PQzz2rO40/qjNNP1dU/uVLrN2xQWVl5s99z9g/O0qIXX9bMm29V/36H6SdXXanNX32td99t2OjukimTdMzRR+qBPz2k0tKdOmX0KI0+eWRUu4PPuus+3XvP77Vs2QoteeMfkqQ9e/aooKBAt916s95++13d8pvnFK6v1w/OOkO3/PqX+tkNN6m6OiCpYR3LBReM05wnntSu8gpVBwLqUlSoDz/6SAuee161NbX67neH6NprrlJ5ebm+WL9B//foHHXp2kVbt27TvKcXSJKqqqrkcDRelmgYhm76+c8Urq/XrDvvVaimRheMH6ubf3WTpl8/Q4FAIFJ38qSL9PT8Z1W2s1wTLhin6677qX56zc9UU1OjYUNP1FlnnqE/PfSwvv56i3y+PB0xeJDtewSkCxb2Aoj47PPPtejFxdq+fYe2bG0YfXj1tSVat+5T7dxZps+/WK/H587TMcccpby8vBav9drrb2jN2ve1Y0dp5Bf34FZ+kX78yTq98uoSbd++Qyvffleffva5jjnqSElSx45ejRo5XPOeXqAPP/xIW7du05NPPaMdpTujirG6ulpWvaVQqEa7d+/W7t27ZVmWzjzjNH399Td6Yu48ff3NFm3duk2PPvaEwmZYJ554fOT7nU6n5syZq08//Vw7SktVVVWlTSWb9Y9/vKWvv/5GO0pL9eprS/Txx+s0bNiJkqRgMKiwGVZtbW3kM5uagjr22KPVq9cheuDBh/TF+g366quv9ac//1UyDJ16yqhGdRc897w+/nidtm3frqefeVadfD4demgfSVKXLoWqqKjQRx99ovLycpWUbNarry1hFAYZh5EYABFffrnxgHNHHXmEzj33B+rZs4c6er0yjIb/9+lSVKiqqqpmr1VSsjny53A4LL/fr875nVr8/JKSrxr9vWJXhfLz8yVJPbp3l8vl0vr1Xzaqs379Bg0aOKDlwGw47LC+GnB4fz0xp/HOwB6PRz26d4/8PRwOa+OmkkZ1OnTooPHnj9WQ7xyjzp07y+VyyuVyqc6MbtfyQ4oPVkVFpXbsKI2cq62t1caNm1RcXNyo7ubN/7tXuyoqJClyr955Z5VOO+0U/fnBP+ijjz/Rxx+v0/v/+pdqahK/iBpIJpIYABGhUE2jv3fr1lUzfn69lr61XM8994L8VVXq1q2rfnnTjXK5Wv7xYX5rpMGyFEmAmv2ebz2hY8mS4TAOOJcIDsPQBx9+pCefevqAsv2ncerq6iJrdPaZNnWyjjxysJ56ar62b9+uUE2NLrv0klbvUVOais+QccBn7n+v9pU5jIZ7tbOsTNddP0NHHDFIRx4xWBMuGKeLJ07Qr359qyorWUiMzMF0EoBm9e93mFwul56YO09frN+gbdu2q3Pnzu3Slm3bt8s0TR3ev1/jNn7r73aYpinHt5KjLzeW6JBDDlZZWbl27ChtdPj9zY84SdKggQO0YsXbWr1mrTZ/9bV27ixT9+7dGn9m2DxgDcy3ff3NFh1UUKBu3bpGznk8HvXt21vffLMlqhjr6ur04Ycf6al583XjjJny+fI0ZMh3oroGkOpIYgA0a9u27XI4HPrB989Uly5FOu6472rMuWe3S1uqqwNatnylLp54oY495mj16NFdky6+SF27FB0wStGa0p07NXDgAB10UIF8voa1Pa+9/oZycnJ0w8+ma8Dh/dWlqEgDBxyuiRdNaPVFflu3bddx3/uu+vbto+Lig3XVjwyBprgAAAGzSURBVK5QJ5+v8WeW7tRhh/ZV165d5PPlNZnQfPjhR/rqq681/ZqfqH//fjrkkGJd89MfS5L+uXSZ7fhOPeVknTxqhHr1OkRFRYUaOeIkud3uyFNWQKZgOglAs77cuElPzJ2ns39wliZcME7rN3ypuU8+rZtm/Kxd2jP3yXlyOZ26bvrVCofrtfLtd/TOu6t0WJRvC17w7PO64rKpeuD+e+TxeHTlj69RRUWFfn3L7Zp44QW68cbrlJuTo4rKSn326efa3coUzJzH5+pHV16uW2/5lQLBgN5885/6178/kG+/RObll1/VVT++Qvfc9Tvl5OREHrHen2VZuuueP2jqJZM08xc3yul0av2GL/Xb39/daEqrNdXVAf3g+2doyuSJcrlc2r59hx7+6yP6/PMvorpPQKrjjb0A0trtt/1aO0t36k8P/bW9mwIgyRiJAZA2+vTpreLig7Vhw5dyu906edQIDTi8vxYsWNjeTQPQDkhiAKSVs844TT0vnSpJ2rJ1q2bddW+jN9UCyB5MJwEAgLTE00kAACAtkcQAAIC0RBIDAADSEkkMAABISyQxAAAgLf1/2D2gRGpSt4QAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plot_learning_rate(history):\n", " fig, ax = plt.subplots(figsize=(8, 8 * 3 / 4))\n", " ax.set_xlabel('Training Iterations')\n", " ax.set_ylabel('Learning Rate')\n", " ax.plot(history.history['lr'])\n", " fig.tight_layout()\n", "\n", "plot_learning_rate(history)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One other innovation Jeremy introduced in the class is the idea of using learning rate cycles to help prevent the model from settling in a bad local minimum. This is based on research by Leslie Smith that showed using this type of learning rate policy can lead to quicker convergence and better accuracy (this is also where the learning rate finder idea came from). Fortunately the file we downloaded earlier includes support for cyclical learning rates in Keras, so we can try this out ourselves. The policy Jeremy is currently recommending is called a “one-cycle” policy so that’s what we’ll try.\n", "\n", "(As an aside, Jeremy [wrote a blog post](http://www.fast.ai/2018/04/30/dawnbench-fastai/) about this if you'd like to dig into its origins a bit more. His results applying it to ImageNet were quite impressive.)" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py:560: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead\n", " return np.fromstring(tensor.tensor_content, dtype=dtype).reshape(shape)\n", "/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()\n", " if d.decorator_argspec is not None), _inspect.getargspec(target))\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Train on 814150 samples, validate on 30188 samples\n", "Epoch 1/10\n", "814150/814150 [==============================] - 76s 93us/step - loss: 1115.8234 - rmspe: 0.2384 - val_loss: 1625.4826 - val_rmspe: 0.2847\n", "Epoch 2/10\n", "814150/814150 [==============================] - 74s 90us/step - loss: 853.5083 - rmspe: 0.1828 - val_loss: 1308.4618 - val_rmspe: 0.2416\n", "Epoch 3/10\n", "814150/814150 [==============================] - 73s 90us/step - loss: 800.1833 - rmspe: 0.1622 - val_loss: 1379.4527 - val_rmspe: 0.2425\n", "Epoch 4/10\n", "814150/814150 [==============================] - 74s 91us/step - loss: 820.6853 - rmspe: 0.1627 - val_loss: 1353.2198 - val_rmspe: 0.2386\n", "Epoch 5/10\n", "814150/814150 [==============================] - 73s 90us/step - loss: 823.7708 - rmspe: 0.1641 - val_loss: 1423.9368 - val_rmspe: 0.2440\n", "Epoch 6/10\n", "814150/814150 [==============================] - 74s 90us/step - loss: 778.9107 - rmspe: 0.1548 - val_loss: 1425.7734 - val_rmspe: 0.2449\n", "Epoch 7/10\n", "814150/814150 [==============================] - 73s 90us/step - loss: 760.5194 - rmspe: 0.1508 - val_loss: 1324.7112 - val_rmspe: 0.2273\n", "Epoch 8/10\n", "814150/814150 [==============================] - 74s 91us/step - loss: 734.5933 - rmspe: 0.1464 - val_loss: 1449.1921 - val_rmspe: 0.2401\n", "Epoch 9/10\n", "814150/814150 [==============================] - 74s 91us/step - loss: 750.8221 - rmspe: 0.1491 - val_loss: 2127.6987 - val_rmspe: 0.3179\n", "Epoch 10/10\n", "814150/814150 [==============================] - 74s 91us/step - loss: 750.6736 - rmspe: 0.1500 - val_loss: 1375.3424 - val_rmspe: 0.2121\n" ] } ], "source": [ "from clr import OneCycleLR\n", "\n", "model2 = EmbeddingNet(cat_vars, cont_vars, embedding_sizes)\n", "batch_size = 128\n", "n_epochs = 10\n", "lr_manager = OneCycleLR(num_samples=X.shape[0] + batch_size, num_epochs=n_epochs, batch_size=batch_size, max_lr=0.01,\n", " end_percentage=0.1, scale_percentage=None, maximum_momentum=None,\n", " minimum_momentum=None, verbose=False)\n", "history = model2.fit(x=X_array, y=y, batch_size=batch_size, epochs=n_epochs, verbose=1,\n", " callbacks=[checkpoint, lr_manager], validation_data=(X_val_array, y_val), shuffle=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can probably tell from the model error, I didn’t have a lot of success with this strategy. I tried a few different configurations and nothing really worked, but I wouldn’t say it’s an indictment of the technique so much as it just didn’t happen to do well within the narrow scope that I attempted to apply it. Nevertheless, I’m definitely adding it to my toolbox for future reference.\n", "\n", "If my earlier description wasn’t clear, this is how the learning is supposed to evolve over time. It forms a triangle from the starting point, coming back to the original learning rate towards the end and then decaying further as training wraps up." ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGhCAYAAACQ4eUqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzs3Xl4W+WZNvD7PbZkW7a8O17i3WSFsIQ9e8IWIAkEQkoKKQECLdOFYehHv+Gb6bSlLbSFmWmnHUohQGlp2UP2BUhCAmSjQBKyJ7bsLE6827JlW5bP+/0hxYlj2ZZtSa+OdP+uS9eV6hwf3XqqxA96znuOGD9hpgQRERGRwWiqAxARERENBpsYIiIiMiQ2MURERGRIbGKIiIjIkKJVBwgllvgEdHQ4VccgIiKic5hMZjhamns8zybGwxKfgPkLH1Edg4iIiLx46y/P92hk2MR4nPkG5q2/PO/nb2MEEhKT0dzUAICr2c9iXXpiTbxjXbxjXbxjXXoydk1MJjPmL3zE6+9mNjHn6ehwosPp3ybG1dHhOabxPjyBw7r0xJp4x7p4x7p4x7r0FL414Ym9REREZEhsYoiIiMiQ2MQQERGRIbGJISIiIkNiE0NERESGxCaGiIiIDIlNDBERERkSmxgiIiIyJDYxREREZEhsYoiIiMiQ2MQQERGRIQX13kmapmHhvQswZfJECCGwfftOLHnlNbhcrgHve+MN12HqlEnIz8/D4SNH8bOnnu728zExMXho8SJcPn48XK4ObNi4GX9/462gvE8iIiIKvKB+EzP39tkYO2Y0fvjEk3j0sSeQm5uLBXfPH9S+DQ2NWLZ8FVavWef15+9ftBCJ1kR87weP4cl/+wmuufpKzLzphoC8LyIiIgq+oDYxM6ZPw3tLl6O+vgF2ux3vvLsU06ZOhhBiwPvu2Pk5duz8HI2NTT1+1mw2Y+KEa/DGW++gpcWB6uoarFi5GjOmT/UhpfDzI5DHNvKDdYn4mogoiJxJQFQc6zKoB+vCukRSTbwL2jjJYrEgPT0NNlt513OlZTYkJMQjLS0VNTW1g9rXm+zsLERHR3f7+bIyG3Jzh0MIASl7vxV5QmIyXB0dg3mLfbImpfj9mOGAdekpUmrSkX4lOjKuRlRiDmIqP+x3/0ipy0CxLt6xLj0ZtSbRJlPv24IVIi42FgDgaHV0PedwOLptG8y+vb2W0+mErutdz7U4HIiKioLZbEZ7e3uvP9vc1IAOp7Pf1xgIa1IK7I31fj1mOGBdeoqYmlgyoaVdAVn+IWTedDhPfQk0HOl194ipywCxLt6xLj0ZuSYms7nXbUFrYlrb2gAAljgL7PZm958tlm7bBrNvb69lNpuhaVpXIxNvsaCzsxPOfhsU6Xn4y7lfg/nzuEbHuvQUITURGrSS2ZA1uyErPwO0KGjFt0Lf9TzQ6e0/MCKkLgPGunjHuvRk9Jr0njlo58Q4HA7U1NSioDC/67miwgI0N7egtrZu0Pt6U1l5Ci6XCwUFZ3++sLAAx0+c6HOURESBJ3ImAqZ4yPL1AAB58lOgowWi4EbFyYjIaIJ6Yu+GjZtwx+1zkJKSDKvVinnz5mLTx1u8Nhb97atpGkwmEzRNgyYETCYToqKiAABOpxOffrYN35h/JywWCzLS0zFr1i3YsOHjYL5dIjqfZRjE8CnQS1ed/dZF6tCPLodIvxhIvkBtPiIylKBeJ2bp+ytgtVrx7K+fhqYJbNu2A2+8+TYAYPGDiwAALy15td99AeCOubfhrnlzu/73X19bgr379nddL+bVP/8Vix+8D3/4n/+Cq9OFDRs+xtp1HwTlfRKRF0KDVjwHsnYP0HC4+zbHacgTW6AVzYK+u7exEhFRd2L8hJmcr8B94tA9DzyK11/+rZ9P7BXnnFDFUp/FuvQU3jUROZMgsq70nPvi5dw2oUG7aDFky0nI0pXnbgjrugwe6+Id69KTsWvS1+9n3naAiAIvLgMid6pnjNTLyflSh350GUT6JUBSSXDzEZEhsYkhogAT0Epug6z9Gmg41PeujtOQJz+BVjwLiIoJTjwiMiw2MUQUUCJnAmC2Qtq83yLkfPLEFsDVBpF/fYCTEZHRsYkhosA5M0Yq62OMdL4zY6WMS4Gk4sDmIyJDYxNDRAEioJXMgazdB9T3M0Y6n+MU5MlPPWOl3q/WSUSRjU0MEQWEyLkWMCdB2tYO6ufliS1AZzvHSkTUKzYxROR/cekQudOgl630fYx0PtnpvghexqXotOT6Nx8RhQU2MUTkZ57VSHX7Bz5GOl9LJWTlVjizZwAax0pE1B2bGCLyK5E9tDHS+eTxzYDeAZF/nV+OR0Thg00MEflPbDpE3jT3aiRXq3+OKTthrvwIYthlQGKhf45JRGGBTQwR+YlnNVLdAaD+oF+PHNVWBVm5FVrxbEAz+fXYRGRcbGKIyC9E9jVAbAqkbU1Ajn92rMTVSkTkxiaGiIYuNg0ib7p/x0jnO7Naadh4jpWICACbGCIaMs8Yqf4gUHcgsC/VchKychvHSkQEgE0MEQ2RyLoaiE2FLAvMGOl88vgmQLq4WomI2MQQ0RDEpkLkT4dethpwOYLzmmfGSplXANaC4LwmEYUkNjFENEgCWvEcoP4wULc/uC/dfMI9VirhWIkokrGJIaJBEVlXAXHp7m9hFJDHNgFSh8iboeT1iUg9NjFENHCxqRB5MyBtQRwjnU+63GOlrCsBa76aDESkFJsYIhowrXgO0HAEsnaf2iDNxyErt0MrmQNo0WqzEFHQsYkhogHpGiPZ1IyRziePbeRYiShCsYkhIt/FpEDkXee+Km9Hi+o0btIF/egKd3NlzVOdhoiCiE0MEflMK5kDNB6FrN2rOkp3zccgT+1wj7k4ViKKGGxiiMgnIvNKwDLMfWuBECSPbQAAiNzpipMQUbCwiSGi/sUkQ+Rf574qb6iMkc6nu6CXroDIvhpIyFWdhoiCgE0MEfVLK54DNJZB1n6tOkrf7BXusVLJHEBwrEQU7tjEEFGfROYVQHxmyI6RziePbQSEBpE3TXUUIgowNjFE1LuYZIj86yFta4GOZtVpfKN3uC+Cl30Nx0pEYY5NDBH1SiueDTTZIGv2qI4yMPYKyFM7OVYiCnNsYojIKzHsciA+C3rpStVRBkUe2+AZK01VHYWIAoRNDBH1FJMEUXA9pG2dccZI59M7PBfBuwZIGK46DREFAJsYIurBPUYqh6zZrTrK0NjLIav+4V5dJaJUpyEiP2MTQ0TdiGHjgfgc6KXGWI3UH1nxEaBFQ+RyrEQUbtjEENFZ5iSIghsgy9cBHXbVafxD7/BcBO9aID5HdRoi8iM2MUTUxT1GqoCs3qU6in812SCrvvCsVuJYiShcsIkhIgCAGHYZkJADvcyYq5H6Iys+BDQTx0pEYYRNDBEB5kSI/Bshy9cDzjAZI52PYyWisMMmhojcY6TmY5DVX6mOElhNNsjqLzlWIgoTbGKIIpzIuBRIyDXsRe0GSpZ/CESZIYZPUR2FiIaITQxRJDMnQhScGSM1qU4THLrTfRG8nAlAfLbqNEQ0BGxiiCKYVjwLaD4BWf2l6ijB1VQGWf2VZ6zEfwaJjIp/e4kilMi4BEjIg166QnUUJWTFB0BULMdKRAbGJoYoEpmtEAU3uX+RR8oY6XydTvdqpZyJgCVLdRoiGgQ2MUQRSCuaBTSfhKz6QnUUtRpLIat3caxEZFD8W0sUYUT6xUBiPvSyyBwjnU9WfABEx0EMn6w6ChENEJsYokhiskIUznQvM25vVJ0mNHS2Qy9dCZEzCbBkqk5DRAPAJoYogmjFtwItlZBV/1AdJbQ0HoWs2Q2t5DaOlYgMhH9biSKESB8HJBZG7Gqk/sjy9UC0xf2NDBEZApsYokhgSoAonOm+CWJ7g+o0oamzHXrZSve5MRwrERkCmxiiCKAV3Qq0nIY8/bnqKKGt4QhkzR6uViIyCP4tJQpzIu0iIKkIeuly1VEMQZavA0zx7uvHEFFIYxNDFM5M8RBFN0NWfMQxkq8626GXrnJfyTdumOo0RNQHNjFEYUwruhVwVEGe3qk6irE0HIas5ViJKNTxbydRmBJpFwJJJdCPcow0GNK23n17huwJqqMQUS/YxBCFI1M8ROHNkMc+AtrrVacxps4291gpdwoQl6E6DRF5wSaGKAxpRbcArdWQp3aojmJsDYcga/e6x0oQqtMQ0XnYxBCFGfcY6QKOkfxE2tYB5kSIHI6ViEJNdDBfTNM0LLx3AaZMngghBLZv34klr7wGl8s14H37256Skoz771uIMWNHQ0DgwMFDePmVP6Oujl+tUxjjGMn/Otugl62CNmIeZP1BoLVGdSIi8gjqNzFzb5+NsWNG44dPPIlHH3sCubm5WHD3/EHt29/2B++/D1HRUfj+Dx7HP33vn9He3o6HH3og4O+RSCWt8BagtYZjJH+rPwRZu899byWOlYhCRlCbmBnTp+G9pctRX98Au92Od95dimlTJ0OInv8o9Ldvf9uHZWZg69btaGtrg9PpxCeffob8vDwfUgo/PwJ5bCM/WBd/10SkXgikXOC5N5Lq9xI6dfHXQ5avB8xJnrGS6pqETl1C78G6hGdNvAvaOMlisSA9PQ02W3nXc6VlNiQkxCMtLRU1NbU+7+twtPZ7rFWr1uKaa67GF1/ugq7rmDJpIr788qt+cyYkJsPV0eGnd32WNSnF78cMB6xLT4OpiYyKQ2vxLTBVb4MpRgIx4VfXUPisuE5/DGfuTYjtOAXNGRrjulCoSyhiXXoyak2iTabetwUrRFxsLADA0eroes7hcHTb5uu+Upf9HuvgocOYPn0qlrz4vwCA8vIKPPWLZ/rN2dzUgA6ncwDvrH/WpBTYG0PjH7xQwrr0NNiaiBEzIFpr0Fb2MdogA5BMrZD5rDTWQ8QVoDVzGvSvXwEU1zpk6hJiWJeejFwTk9nc67agjZNa29oAAJY4S9dzFoul2zZf9+1vuxAC/+/JJ3D48BHcd//DuO/+h7Hn67341x/90Iek0s+PQB7byA/WxW81SR0NkTLCsxpJD4H3ESJ1CdBD2ta4x0rZV7MuIflgXcKzJt4FrYlxOByoqalFQWF+13NFhQVobm5BbW3dgPbtb3tCQjyGZWRgzdr1cDqdcDqdWLv2A4wYcQGs1oTAv1miYIm2QCu6BfLYRqCttv/9aehcrdDLVkPkTQdi01SnIYpoQT2xd8PGTbjj9jlISUmG1WrFvHlzsenjLZCyZ5fV3759bbfbm1FZeQo33Xg9TCYTTCYTZs68ATW1tbDbm4P5lokCShTOBNrqISu3q44SWeoPQNYd4GolIsWCep2Ype+vgNVqxbO/fhqaJrBt2w688ebbAIDFDy4CALy05NV+9/Vl+2+e+2/ct/AePP+H30IIwFZegd88+9/BeqtEgZc6GiJ1NPTdL6Cvr1spMKRtLcQlj0BkXw1ZuU11HKKIJMZPmMl//eA+ceieBx7F6y//1s8n9opzTqhiqc9iXXoaQE2i46Bd/Ahk5VbIyq1BSadOCH9WUsdAu+B26Lv/pGCcF8J1UYp16cnYNenr9zNvO0BkQKJwJtDewG8AVKvbD1l/iPdWIlKETQyR0aSMgkgd41mNZLz/qgo3smwNEJsKkXWV6ihEEYdNDJGRRMdBK7oV8vgmoI338AkJLgdk2RqI/BlAbKrqNEQRhU0MkYGIgpsAZyPkyXA/D8ZYZN0+oP4wtGKOlYiCiU0MkVGkjIRIG8sxUojSy1YDcekQWVeqjkIUMdjEEBlBVKxnjPQx0FqtOg1543JA2lZD5F0XlveuIgpFbGKIDEAU3gQ47ZAnP1Mdhfoga/cBDUc8q5WIKNDYxBCFuuSREGkXQj+6DBwjhT7dthqIy4DI5FiJKNDYxBCFsqhYaMW3Qh7fzDGSUXS0QNrWQORzrEQUaGxiiEKYKLzRPUaq5BjJSGTtXqCxFFrJbNVRiMIamxiiUJU8AiJtHPTS5YDUVaehAdLLVgGWTIjMK1RHIQpbbGKIQlFUjHuMdGIz4KhSnYYGo6PFcxG864GYZNVpiMISmxiiECQKbnT/Ejz5qeooNASy9mugscxzETwi8jc2MUShJrkEIv1i92okjpEMTy9bBcRnQmRerjoKUdhhE0MUQqRmhlY0C/LEFo6RwkVHM6RtLUT+DUBMkuo0RGGFTQxRCHFmTnJf+fXkJ6qjkB/Jmj1Ak41jJSI/YxNDFCqSStCZNMp9bySOkcKOXroSiM+CGMaxEpG/sIkhCgWe1UjRNf8AHKdVp6FA6GiGtK2DKLieYyUiP2ETQxQCRP4NgKsNpprPVUehAJI1u4GmCmjFvAgekT+wiSFSLakYIuMS6EeXQ4BjpHDnHivlQAwbrzoKkeGxiSFSKcoMrXi2+3owjlOq01AwdNghy9dBFNwAmDlWIhoKNjFECp0ZI8kTm1VHoSCS1bsA+zFoxbNURyEyNDYxRKokFkFkXMp7I0UovXQlkDAcIuMy1VGIDItNDJEKmhlayWzIk58BLZWq05AKzibI8vXuW0yYE1WnITIkNjFECoiC64FOJ8dIEU5WfwU0c6xENFhsYoiCLbEIIuMyz72ROlWnIcXcY6U8iIxLVUchMhw2MUTBdGaMVLmVYyRy6zZWsqpOQ2QobGKIgkjkX+ceIx3/WHUUCiGy+kug+QS0Io6ViAaCTQxRsCQWQgwb77k3EsdI1J1eugKw5kNkXKI6CpFhsIkhCgbN5L6oXeU2oOWk6jQUipxNkBUfQBTcBJg4ViLyBZsYoiAQ+dcB0gV5fJPqKBTCZNUXQPNJaMW3qo5CZAhsYogCzVoAkXkFx0jkE71sBZBYAJF+seooRCGPTQxRIGkmz2qkbUDzCdVpyAjaGyHLP4QovAkwJahOQxTS2MQQBZDImwFIHfLYJtVRyEBk1T+AllO8CB5RP9jEEAWKNR8i60rPGMmlOg0ZjF66wr2iLX2c6ihEIYtNDFEgaNHQSuZAVm4Hmo+rTkNG1N4AWfEhROFMjpWIesEmhigAzo6RNqqOQgYmT38OtJyGVsTVSkTesIkh8jdrHkTWVdCPruAYiYZML10OJBVBpF2kOgpRyGETQ+RPWjS04jNjpGOq01A4aG+ArPgI7rFSvOo0RCGFTQyRH4m8GQAAeZxjJPIfeXon0FrNsRLRedjEEPlLgmeMVLoc0DlGIv/Sjy4Hkkog0i5UHYUoZLCJIfIH4VmNdGoHYOcYiQKgvR7y2EcQhTdzrETkwSaGyA9E3nRACMhjG1RHoTAmT+3wjJVuUR2FKCSwiSEaqoRciOyr3V/3c4xEAaaXrgCSLoBIG6s6CpFybGKIhqJrjLQTsFeoTkORoK0O8tgGiMKbIaPiVKchUsrnJmba1Ml45umn8MqSF5CRkQ4AmD3rFlx11RUBC0cU6kTeNEBoHCNRULnHSjVwZk1VHYVIKZ+amOtmTMPCexdgx47PER0dBSEEAMBut2PmjdcHMh9R6EoYDpF9jfuidnqH6jQUUST00hXoTCgEUseoDkOkjE9NzMybbsCLL72C95YuQ2dnZ9fzpWU25ObmBiwcUcgSUe4x0unPAXu56jQUidrqYKreCq3oZiDaojoNkRI+NTFZWZk4crS0x/Ptbe2Ii4v1eyiiUCdypwEiGrLiI9VRKIJF1+0G2uogim5WHYVICZ+amPr6BmRlZvZ4fsTIC1BVVe33UEQhLT7HPUYq5RiJ1BKQ0I+ugEgZybESRSSfmpjNWz7FwnsXIDsrC1ICZrMZl156Me5ZMB8bN20OdEai0HFmjFT1BdBkU52GCGirhTy20X3tmGiuVqLIEu3LTu++9z4yMtLxn889AwD4za9+AQDYtGkzVq5aE7h0RCFG5E4FNBNkxYeqoxB1kZXbIVLHQhTeDHnkPdVxiILGpyZGSonn//gi3nlnKYqLiyA0gbJSG05XVQU6H1HoiM+ByL4W+oHXOUaiECOhH10G7eJvQ6aMBuoPqA5EFBQ+jZPuvOM2mM1mVNfUYPuOndi2bQdOV1XBZDLhzjtuC3RGIvXOjJGqv+QYiUITx0oUgXxqYubdORexsTE9no+JicG8O+f6PRRRqBHDpwBRZshyjpEodMnKbUB7A0ThTNVRiILC5yv2StnzueysTLS0OPyZhyj0xGdD5EzwXNTOqToNUR8k9KPLIVLHACmjVIchCrg+z4n5n98+B+npXp7+xU+h63rXNk3TkJychG3bd/j8YpqmYeG9CzBl8kQIIbB9+04seeU1uFw9b5rX376+HGv8+Esxf94dyM7OQmtrG1atXosVK1f7nJcIQvOMkb4CmspUpyHqX1sN5PFN0Ipuhd5UDnS2qU5EFDB9NjEbNm6CEAJ3zbsDn362FW1tZ/8yuFydqKqqxs7P/+Hzi829fTbGjhmNHz7xJFyuTjzxw8ew4O75+Mtf/zbgffvbfskl4/DQg4vwh//9E/btP4CYGDPS09J8zkoEnBkjxUJWfKA6CpHP5MmtEKljIApnQh59X3UcooDps4lZ+v4KAEBNbR22bt2Ojo6hrciYMX0aXvvL31Bf3wAAeOfdpfjB9/8Jf339713f+Pi6b3/bv3HXnXhv6XJ8vXcfAKC1tQ3Hjp8YUn6KMJYsiJyJ0A/+HejkGImMxD1W0sY9BFm3D6g/pDoQUUD4tMR68+ZPhvxCFosF6elpsNnO3memtMyGhIR4pKWloqam1ud9HY7WPrfb7XYUFRXis63b8NxvnkaCNQGHDh3Gq39+HbW1Z1/HO+F5BEKgjmt0IVgXoUEruQ2yehfQWIbgZwzBmoQE1sU7L3VprYE8vtkzVjoWoWMlfl56MmJNes/sUxMTFRWFubfPxsQJ1yI9PQ3R0d1/bME9i/o9Rlys+x5LjtazJwI7HI5u23zdV+qyz+26rkPTNEyZMgnP/Oo5NDY14b6F38Tjj30fT/7bT/rMmZCYDNcQv3HyxpqU4vdjhoNQrYsz/Sp0mi2IbfgcIsgZQ7UmqrEu3vVVF9myH+2dFyF6xGzEVEbWyjp+Xnoyak2iTabet/lygHl3zsW0qZOwYuUaLLj7Lrz51jvIyMjAVVdejrffWepTiFbP+TSWOAvs9mb3ny2Wbtt83bff7a3u7WvXfoDqmhoAwBtvvYOX/vS/SElJ7hpBedPc1IAOp39HB9akFNgb6/16zHAQsnWxZEJLuxz6wTfR3Hg6qC8dsjVRjHXxzqe6HHoP2rjFcJ76Cmg4HJxgivHz0pORa2Iym3vd5lMTM2HC1XjxpVfxxZdfYf5dd2DrVvfF7o4fP4GRIy/A+g/6v5Ovw+FATU0tCgrzu670W1RYgObmFtTW1g1oXyllv9urqqu7nWfjbYm4d9Lz8Jdzvwbz53GNLkTrcmY1Us1uoPFIsF/8nD+HUE2UY12887EurVXusVLxLdB3/TECxkr8vPRk9Jr0ntmn68SkJCejvOIYAKC9vR1xFvfVIP/xxZcYf9mlPsfYsHET7rh9DlJSkmG1WjFv3lxs+nhLj5N6fdm3v+0ffrgRN998I9JSU2EymfCN+Xfi6NHSPr+FIRI5k4DoOMjy9aqjEPmNrPwMcDZDFNyoOgqRX/n0TUx9fQOSkhJRW1uL6uoajBk9CjZbOfLyctF5zrVj+rP0/RWwWq149tdPQ9MEtm3bgTfefBsAsPjBRQCAl5a82u++vmxfvmIV4uMteObpn0EIDQcOHsJz//U/PmelCGTJhBg+GfqhN4HOdtVpiPxH6tBLl0O76CHIuv0RM1ai8CfGT5jZ73dLDy2+Hw0NjXj7nfcwY8Y0PLBoIcrKypGfn4vNWz7Fkpf/HPikAWYym3HPA4/i9Zd/6+dzYsQ5s0gjfo0XKCFWF6FBu+hByJZTkKUrVIUIrZqEDNbFu4HXRQyfDJF5OfRdz4dxo87PS0/Grklfv599+ibmxZde6frzhg2b4GhpwejRo7Dlk0/x4Ucb/ZuWSAGRMxEwxXOMRGFNnvwUInU0RMGNCpt1Iv/xqYk537btO7Ft+05/ZyFSwzIMYvgU6IfeCuP/OiWCe6x0dNk5Y6Vgn7xO5F8+3wDSmxEjLsB//PhJf2UhCj6hQSueA1m7h+cJUGRwVEGe2AKtaBYQFaM6DdGQ9PtNzLiLLsTFF4+Dy+XChg2bUF1Tg+ysLNx7790Yf9ml2LNnbzByEgWEyJ4AmK2Q+/+qOgpR0MiTn0CkjoIouAGydKXqOESD1mcTM2nSBHz3kYfR3NKChPh4TJ82BS+/8hoe+c5D+OKLL/F/n/wxyssrgpWVyL/iMiByp0I/9HYEXDuD6BxSd99b6aLFkLX7gcajqhMRDUqfTcytN9+Et95+F0vfX4EJ116N73/vEcy783b8249/ihMnTgYrI1EACPe9kWq/Bhp4czyKQI7TkCc/gVY8C/ruP/J8MDKkPs+JycrKxKefbQMAbN22A7qu47W//I0NDBmeyPGMkWzrVEchUkae2AK4WiHyr1cdhWhQ+mxiYmNj0ea5T5GUEh0dHaiqrg5KMKKAOTNGKlvFMRJFNs9YSWRcCiQVq05DNGD9nthbUlKMluYWAIAQAkWFhUhOSuq2z6HDXKZHRiHc90aq3QfUc4xEBMcpyJOfnjNW8u8NcIkCqd8m5okfPtbtf//zo9/tsc+Cexb5LRBRIImcawFzEuSBv6mOQhQy5InNECmjIPKvhyxbrToOkc/6bGK+/+jjwcpBFHhx6RC506AffhdwtapOQxQ6ztxb6cIH3KuVmspUJyLySZ9NTE1NbbByEAWYZzVS3X6g/qDqMEShp6US8uRn0EpmQ9/1R0DnWIlC35Cu2EtkFCL7GvcYybZWdRSikCVPbAY6nRD516mOQuQTNjEU/mLTIPKmQy9bzTESUV9kJ/SjyyCGjQcSC1WnIeoXmxgKc2fGSAeA+gOqwxCFvpZKyMqt0IpnA5pJdRqiPrGJobAmsq8GYlM4RiIaAHn8Y0Dv4FiJQh6bGApf3cZIDtVpiIxDdrovgjfsciCxQHUaol71e50YAHho8f3eN0gJZ0cHKitPYeu2HbDb7f7MRjQEnova1R8C6varDkNkPC0nISu3QSue474Int6hOhFRDz41MdlZmSgsLEBUVDQqKyvdz2VnweXqROWpU5iCr/9BAAAgAElEQVQ6ZRK+Mf9O/PgnP+d9lSgkiKyrgNhUyINvqo5CZFjy+CaI1JEQeTMgy3mfMQo9PjUx27bvRLvTid//4Y9oaXF/LR8fb8F3H/k2vvxqFzZv+RSPPfo9LLxnAZ759XMBDUzUr9hUiPwZkEeWcYxENBSesZI2dpH75Hh7uepERN34dE7M7Nm34O9vvN3VwABAS4sDb771Dm6bMwvt7e14592lKCouDFROIh8JaMVzgPrDkHX7VIchMr7mE5CntkMr4WolCj0+NTHJSUmIiorq8XxUdDQSE60AgMbGJsTGxPg3HdEAiawrgbh06LY1qqMQhQ15bBMgdYi86aqjEHXjUxOz/8BBPLDoW8jISO96btiwDCz61j04cMB9J+Dhw3NQXVMTmJREvohJgci7DtK2BuhoUZ2GKHxIl/sieJlXAtZ81WmIuvh0TsyLL76CHz7+KH7338+iyW4HpERiYiLKK47hd79/HgCgRWl4991lAQ1L1BetZA7QcASydq/qKEThp2usdGa1kkt1IiLfmpjqmhr86F//HePGXYjhOTkAgBMnTmLP12d/WXzxxVeBSUjkA5F1FRCXAf3w86qjEIUteWwTRMqZ1UrrVcch8q2JOWPPnr3Ys4f/lUsh5swYqXQ5x0hEgSRd7tVKFy5y3xHefkx1IopwPjcxmcOG4cKLxiI5KRFCiG7b3n2PYyRSRyuZAzQe5RiJKBiaj0NWbndfBG/PCxwrkVI+NTGTJk3AI99ejI6ODjQ2NkFK2bVNgk0MqSMyrwQsw6Dv+l/VUYgihjy+0T1Wyp0OWfGB6jgUwXxqYu6aNxcrV63BG2++062BIVIqJhki/zrI0pUcIxEFk+6CXroc2tj73GOl5uOqE1GE8vE6Mcn4aMMmNjAUUrTiOUBjGWTt16qjEEUe+zHIUzvc41wxoNMrifzGpybm6717UVjIO5lS6BCZVwDxmdDLVqmOQhSx5LENgNAg8qapjkIRyqf2ecuWT3HPgm8gLTUV5eUVcLm6n8h16PCRgIQj8iomGSL/esiyVUBHs+o0RJFL96xWGrPQfW8ljpUoyHxqYh79wXcBAN9a+E2v2xfcs8hvgYj6oxXPBppskDV7VEchInsF5OnPPRfB+xMguVqJgsenJub7jz4e6BxEPhHDLgfis6Dv4kXtiEKFPLYBImUERN5UyIqPVMehCOJTE1NTUxvoHET9i0mCKLgesmwNx0hEoUTvgH50BbQx93rGSidUJ6II0WsTM3LEBTh85CiklBg54oI+D8JzYigY3GOkCsia3aqjENH57OXnjZU6VSeiCNBrE/PTn/wbvv3I99HUZMdPf/JvfR6E58RQoIlh44H4HI6RiELY2bHSNI6VKCh6bWK+/+jjaGqyd/2ZSBlzEkTBDZC2tUCHXXUaIuqN3gG9dAW00RwrUXD02sScex4Mz4khlbTiWe4La1XvUh2FiPrTVA5Z9Q/PvZU4VqLA8vkyi0IIZGYOQ1JSErTzbgC5/8BBvwcjAgCRcRmQMBz67j+qjkJEPpIVH0Fc/B2I3GmQxzhWosDxqYkpKizAoz/4LjIzh3ndznNiKCDMiRAFN0KWrwOcTarTEJGv9A73vZVG3+u+t1LLSdWJKEz51MQsfnARqqqq8fs//BF19fXgLZQoGLTiWUDzccjqr1RHIaKBOjNWKpkDfc+LHCtRQPh076Tc3Fy88upfcORoKerq6lFf3/1B5G8i41IgIQ966QrVUYhokGTFR4BmgsidqjoKhSmfmpiTJ0/Cak0IdBYiN7PVPUaqWM8xEpGReVYriexrgfgc1WkoDPnUxLzy6l9w9913oSA/L9B5iKAVzQKaT0JWfak6ChENVZMNsuoLaCVzABGlOg2FGZ/OifmPHz8JTdPwzNNPQUoJed5JMfcsfCAg4SjyiIxLgMR86Lu4GokoXMiKDz2rlaZCHtugOg6FEZ+amBdefDnQOYgAkxWi4CbI8g8AZ6PqNETkL10XwbvHfRE8rlYiP+m3iYmKikJsTAx2fv4P1Nc3BCMTRSit+Fag5SRk1ReqoxCRv50zVuJqJfKXfs+J6ezsxL333I2oKM4yKXBE+sVAYgFXIxGFMVnxERBlhsidojoKhQmfTuwtLbMhP48n9VKAmBIgCm+CLP8QaOcYiShs6U7oR5dDZE8A4rNVp6Ew4NM5Me8vW4GF9yxAXFwcjpaWor29vdt2jploKLTiWUDLKciqf6iOQkSB1mSDrP4SWsltHCvRkPnUxPzo//wLAOB73/221+287QANlkgfByQW8t5IRBFEln8Iccl3IIZPgTy+UXUcMjCfmpif/fzpQOegCCSjLRCFN0FWfAi089s8ooihO6EfXQFt9Dch6w8ALZWqE5FB+dTE7N/Pu1ST/zmzpgMtpyFPf646ChEFW1OZZ6w0B/qelzhWokHxqYk5Izk5CRnp6YiO7v5j+w+wyaGBEWkXoTM+lxe1I4pgXRfBGz4Z8vgm1XHIgHxqYpKSkvDo9x/BmDGjvW7nOTE0IKZ4iKKZMFV9hk6OkYgiV+c5Y6W6A4DjlOpEZDA+LbG+71vfRFRUNP71//0HnE4nfv7LX+H3f3gBlZWn8PQzzwY6I4UZrehWwFGF6Po9qqMQkWpNZZDVX0EruQ0QPv1KIuri0zcxY8eMwbPP/TdstnLoukRtbR327t2Pdmc77pg7B7v3fO3Ti2mahoX3LsCUyRMhhMD27Tux5JXX4HK5Bryvr8cymUx49te/hNWagAcWP+JTTgockXYhkFQCffcLELGq0xBRKJAVH3jGSlM4VqIB8antjY2NQUOj+yJkDocDVqsVAFBuq0BRUZHPLzb39tkYO2Y0fvjEk3j0sSeQm5uLBXfPH9S+vh5r/l13orauzueMFECmeIjCmyGPfQS016tOQ0ShotMJvXQlRM5EwJKlOg0ZiE9NzKnTVcjMHAYAOH7iBKZMnojo6GhMmnQt7Ha7zy82Y/o0vLd0OerrG2C32/HOu0sxbepkCCEGvK8vxyoqLMBll12MZctW+pwREH5+BPLYxnpoRbcCrdWQp3ayLl4frAnrEsF1aSyDrN7lGStFsS5+fYRDTbzzaZy0adNm5OXmYu/e/Vi2bCV+9MTjuP666ZBS4pVX/+LLIWCxWJCengabrbzrudIyGxIS4pGWloqamlqf93U4Wvs9lqZpePihB/Dyy6/5lO+MhMRkuDo6BvQzvrAmpfj9mEbisl4AZ3IJYkvfgHZOLSK9Lt6wJt6xLt6FU11kw060pY6AqfgGmGt2DOlY4VQXfzFqTaJNpt63+XKAtes+6Przvv0H8C+P/wglJcWoPHUKx44d9ylEXKz7BAhHq6PrOYfD0W2br/tKXfZ7rFm33oyKimPYt/8Axvayqsqb5qYGdDidPu/vC2tSCuyNETw+ibZAu2AKZMUGtNTYup6O+Lp4wZp4x7p4F5Z1ObIc+qi70V65a9CrlcKyLkNk5JqYzOZetw3oOjFn1NbVDfg8k9a2NgCAJc4Cu73Z/WeLpds2X/ftb3vmsGG46cbr8aN//fcBvzdAeh7+cu7XYP48rnFoRTcDrTWQp3bgbA1Yl55YE+9YF+/CtC6NRz1jpdnQv34JkPoADxCmdRkSo9ek98w+r2ebNnUynnn6Kbyy5AVkZKQDAGbPugVXXXWFTz/vcDhQU1OLgsL8rueKCgvQ3NyC2tq6Ae3b3/bRo0ciMdGK537zS7zw/O/w+L/8AHFxcXjh+d9h5IgLfH3L5A+pY4CUEdBLl8OYf3mIKNhkxXogOg5i+GTVUSjE+fRNzHUzpuGbC+Zj1ep1mHv77K6TZ+12O2beeD127PDtsvEbNm7CHbfPweHDR+BydWLevLnY9PEWSNnzl1t/+/a1/bOt2/HVrrPXIBk1cgQe+c5iPPF//x3Nzc0+ZSU/iLZAK7oFsmIj0MYVYkTkI89qJW3U3Z6L4J1WnYhClE9NzMybbsCLL72Cbdt3Ys7sW7qeLy2z4ZsLvuHziy19fwWsViue/fXT0DSBbdt24I033wYALH5wEQDgpSWv9rtvf9s7OjrQ6FkSDgDNzc2QUnZ7jgJPFN0MtNVBntquOgoRGU3jUcga92qlwY2VKBL41MRkZWXiyNHSHs+3t7UjLs73K5bpuo5X//xXvPrnv/bYdqZ58WVfX7afa9/+A7zQXbCljoFIGQV99wvgGImIBkOWey6ClzMJ8sRm1XEoBPl0Tkx9fQOyMjN7PD9i5AWoqqr2eygyuOg49xjp2Eagrbb//YmIvOlsd18Eb/hkwNLzdxCRT03M5i2fYuG9C5CdlQUpAbPZjEsvvRj3LJiPjZvYHVN3ovBmoK0esnKb6ihEZHSNRyFrdvPeSuSVT+Okd997HxkZ6fjP554BAPzmV78A4L4I3qrVawOXjownZTRE6mjou/8EjpGIyB9k+XqOlcgrn5oYKSWe/+OLeOedpSguLoLQBMpKbThdVRXofGQkXWOkTUBbjeo0RBQuOtuhl62CNvIbkPUHuVqJugzou7nqmhps37ET27btwOmqKmRnZ+G53zwdqGxkMKJwJuBshKzcqjoKEYWbhiOesdIcjpWoy5A+CWaTCTk52f7KQkaWMgoidQz0o8vAMRIRBYIsXw9Ex0PkTFIdhUIE21kauqhYaEW3Qh7fBLRyjEREAdLZDr2Mq5XoLDYxNGRdY6STHCMRUYA1HIGs2cOxEgFgE0NDlTISIm0s9KO8NxIRBYcsXweY4iFyJqqOQor1uTrpyX/9P33+cGys71frpTDUNUb6GGjlRQ+JKEg626GXroI2cr5ntRJXykaqPpuY+rr6fg9QebLSb2HIWEThTYDTDnnyM9VRiCjSNByGrN0DrXgO9L0v895KEarPJub5F14KVg4ymuSREGkXQt/zIjhGIiIVpG09xCXfgciZCHlii+o4pADPiaGBi4qFVnwr5PHNHCMRkTqdbdBLV0EMnwLEDVOdhhRgE0MDJgpvdI+RKjlGIiLFzoyVuFopIvH/cRqY5BEQaeOgly7nDJqIQoK0rQfMVojsCaqjUJCxiSHfRcW4x0gnNnM1ABGFjs426KUrIXKnAHEZqtNQELGJIZ+JghuBjhbIk5+qjkJE1F3DYcjavdBK5kDyV1vE4P/T5JvkCyDSL3Zf1I5jJCIKQdK2DjBb4Uq7THUUChI2MdS/qBhoRbPcSxgdp1WnISLyzrNaqSP9Ko6VIgSbGOqXKLgBcDkgT36iOgoRUd8aDiOq6bB7tRKE6jQUYGxiqG9JJRDpl0A/uoxjJCIyBPPpLe7VSjlcrRTu2MRQ76JioBXPcn8DwzESERmE0Nuhl62GyJ3KsVKYYxNDvRL51wOuNl7Om4iMp/4QZO0+jpXCHJsY8i6pGCLjUo6RiMiwpG0tYE7iWCmMsYmhnqLMnjHSp4DjlOo0RESD09kGvWwlx0phjE0M9SDyrwc62zlGIiLj41gprLGJoe4SiyAyLvNc1K5TdRoioiGT5es8Y6VrVUchP2MTQ2dpZmgls913p26pVJ2GiMg/XK3Qy1ZB5E4D4tJVpyE/YhNDXUT+dUCnE/L4ZtVRiIj8q/4gZN1+aCW3gWOl8MEmhtwSCyGGjecYiYjCVtdqpWyOlcIFmxgCNBO04tmQlVuBlpOq0xARBcaZsVLeNCCWY6VwwCaG3GMkvQPy+MeqoxARBdaZsdIFHCuFAzYxkS6xEGLY5RwjEVHEODtWukZ1FBoiNjGRrGuMtI1jJCKKHF1jpekcKxkcm5gIJvKvA6QL8vgm1VGIiIKr/iBk3QFeBM/g2MREKmsBROYVHCMRUcSStjVATDLHSgbGJiYSaSbPRe22Ac0nVKchIlLD1Qq9bLVnrJSmOg0NApuYCCTyZgBShzy2SXUUIiK16g94xkpcrWREbGIijTUfIutKzxjJpToNEZFy0rYWiE3hWMmA2MREEi0aWskcyMrtQPNx1WmIiEKDy3HOaiWOlYyETUwEOTtG2qg6ChFRaKk7AFl/kKuVDIZNTKSw5kFkXQX96AqOkYiIvJBla4DYVIjsq1VHIR+xiYkEWjS04jmQp3YAzcdUpyEiCk0uB1crGQybmAggcqcDAOSxDYqTEBGFuLr9kPWHOFYyCDYx4S4hFyL7auilKwCdYyQiov50jZWyOFYKdWxiwpnwrEY6tQOwV6hOQ0RkDGfGSvnTgdhU1WmoD2xiwpjImwYIjauRiIgGqm4/UH+YF8ELcWxiwlVCLkT2Ne6L2ukdqtMQERmOXrbaM1a6SnUU6gWbmHDUNUbayTESEdFguRyQZWsg8mdwrBSi2MSEIZE31TNG4mokIqKhkHX73GOlYq5WCkVsYsJNwnCI7Gs9q5E4RiIiGiq9bDUQl86xUghiExNORJT7onanPweaylWnISIKDy4HpG21+9YtHCuFFDYxYUTkTgW0aMiKj1RHISIKK7J2H9BwxDNWolDBJiZcxOdwjEREFEC6jWOlUMMmJhyIKPdqpKovgCab6jREROGpowXStgYi7zogJkV1GgKbmLAgcqcAUWaOkYiIAkzW7gUaj3jurUSqsYkxuvhsiOwJnovaOVWnISIKe+7VShkcK4WA6GC+mKZpWHjvAkyZPBFCCGzfvhNLXnkNLlfPGxP2t29f26Ojo/HA/d/CRReORVJSIurrG7D+g4+wes26YL7dwBNR0Epug6z+kmMkIqJgOTNWKp4DWX8YaK9XnShiBfWbmLm3z8bYMaPxwyeexKOPPYHc3FwsuHv+oPbta3tUlIaGhkb84ulfY9ED38Z//fb3uO22Wbj2mvC6I6kY7hkjlX+oOgoRUURxj5WOcqykWFCbmBnTp+G9pctRX98Au92Od95dimlTJ0OInldB7G/fvra3tzvx1tvv4vTpKkgpUV5egS+/3IVRo0b4kFL4+RGgY8dnQ+RMgH50pWc1kr9zB/oRyJob9cGasC6si5Hq0jVWyrwqBN53aNQkOO/hrKCNkywWC9LT02CzlXc9V1pmQ0JCPNLSUlFTU+vzvg5Hq8/HAtyjp1GjRmLFilX95kxITIarw/9LlK1J/juTXUJDW9EdiGrcD7NoAPx47GDzZ13CBWviHeviHeviXbDq4qr6BM6C6xDbWQWtoykorzlYRv2sRJtMvW8LVoi42FgAgKPV0fWcw+Hots3XfaUufT4WANy/aCFaHQ58vPmTfnM2NzWgw+nfE2StSSmwN/pvZipyp0KIaLiOrEJ7p3FP5vV3XcIBa+Id6+Id6+JdUOvSuB1aXD5ah02Fvu+14LzmIBj5s2Iym3vdFrQmprWtDQBgibPAbm92/9li6bbN130HcqyF9y7AqFEj8NTPn0FnZ6cPSaXn4S/nfg3mh+NasiByJkI/+Hegs33ox1PGz3UJC6yJd6yLd6yLd8Gvi162Ctol/wSReQXk6Z1Bec2BMfpnpffMQTsnxuFwoKamFgWF+V3PFRUWoLm5BbW1dQPa19dj3fete3DxuIvw1M9/1dXsGJrQ3Be1q94FNJaqTkNERIB7tVLZGoh8XgQv2IJ6Yu+GjZtwx+1zkJKSDKvVinnz5mLTx1sgZc8uq799+9u+6L57Me6iC/Gznz8Du90ezLcZMGL4ZCA6DrLiA9VRiIjoHLL2a6CxFFrJbNVRIkpQrxOz9P0VsFqtePbXT0PTBLZt24E33nwbALD4wUUAgJeWvNrvvv1tT09Pw80zb4TT6cTvf/dc18/sP3AQz/zq7P82FEsmRM4k6AffMPgYiYgoPOllq6Fd8ohnrPS56jgRQYyfMNOIAzK/M5nNuOeBR/H6y7/184m94pwTqgZZaqFBu2gxZEslZOkKP2ZTyQ91CTusiXesi3esi3dq6yLSx0EU3Qp99x+B9oagv753xv6s9PX7mbcdMACRMwmItkCWr1cdhYiI+iBr9gCNZdCKeRG8YGATE+osmRDDJ0MvW8kxEhGRAehlq4D4TIjMK1RHCXtsYkLZmdVINXuAhiOq0xARkS86miFtayHyrwdiklWnCWtsYkKYyJkImOI5RiIiMhiOlYKDTUyoihsGMXwK9NJVQGdb//sTEVFI4Vgp8NjEhKIzY6Tar4GGw6rTEBHRYHQ0Q9rWcawUQGxiQpDIngCYrZC2daqjEBHREMia3UCTDVoxL4IXCGxiQk1cBkQux0hEROFCL10JxGdBZF6uOkrYYRMTUoRnjLQPaDikOgwREflD11jpBiAmSXWasMImJoSInAmAOQnStlZ1FCIi8qOzYyWuVvInNjGhIi4dIneq56J2HCMREYUbvXQVEJ8NMYxjJX9hExMSBLSS2yDr9gP1HCMREYWlDrv7IngF13Os5CdsYkKAyL6WYyQiogjgHitVcLWSn7CJUS02HSJvmvuiSK5W1WmIiCjA3KuVciCGjVcdxfDYxCjlWY1Utx+oP6g6DBERBUOHHbJ8HUTBDYCZY6WhYBOjkMi+BohJ5hiJiCjCyOpdHCv5AZsYVWLTIPKmQy9bzTESEVEE0stWAgkcKw0FmxglzqxGOgDUH1AdhoiIVHByrDRUbGIUENlXA7EpHCMREUU4Wb0LsB+DVjxLdRRDYhMTbN3GSA7VaYiISDG9dAWQMBxi2GWqoxgOm5ig8qxGqj8E1O1XHYaIiEKB0w5Zvh4i/0bAnKg6jaGwiQkikXUVEJsKWbZGdRQiIgohsvoroPkYVysNEJuYYIlNhcif4W5gOEYiIqLz6KUr3WOlDI6VfMUmJggkhLu7rj8MWbdPdRwiIgpFzib3WKmAYyVfsYkJAlfKOCAuHbqNYyQiIuqde6x0nGMlH7GJCbSYFHQMu9a9nLqjRXUaIiIKce7VSrkQGZeqjhLy2MQEmEi/CFH2MsjavaqjEBGREXSNlW4CkkpUpwlp0aoDhDt54hOYm5PhVB2EiIgMQ1Z/CZjioY26G7JyG+TxjYDUVccKOWxiAk5CqI5ARESGI09+AmmvgDbiDojEfOiH3wWcTapjhRSOk4iIiEKVvQL67j8BrjZo4x4GkkeqThRS2MQQERGFMpcD+sG/Q578DNrIuyDybwAEf30DHCcREREZgqz8zDNeuvPseKm9QXUspdjKERERGUXzcei7XwCcze7xUsoo1YmUYhNDRERkJJ1t0A+9CXn8Y2gj5rmXYoso1amU4DiJiIjIgOSp7ZD2Y+7xkjXPM16qVx0rqPhNDBERkVG1nIS+509AeyO0cQ9DpI5VnSio2MQQEREZWWc79MNvQx7bAHHB7RBFt0TMeIlNDBERURiQp3dC//pliKRiaBc9CMSmqo4UcGxiiIiIwoXjFPQ9f4JsrXGPl9IuUp0ooHhiLxERUTjpdEIeeQ8YNh6iZA6QVARZt111qoDgNzFERERhSFZ9Af3rJRDWfLQV3gXEpauO5HdsYoiIiMKV4zT0PS9Ca6+GdtGDEBmXqE7kVxwnERERhTO9A+aTH6LDnA9ReAuQWAhZthrQO1QnGzJ+E0NERBTmBABZvQv61y9BxGdDG7cYiBumOtaQsYkhIiKKFK3V0L9eAmk/Dm3cgxAZl6lONCQcJxEREUUSvQOydAXQZIMouhVIKoQsXQXoTtXJBozfxBAREUUgWbMH+p4XIeKGQRv3EGDJVB1pwNjEEBERRaq2Wvd4qcnmXr2UeYXqRAPCcRIREVEkky7IslXu8VLxLIjEQuilK4DOdtXJ+sVvYoiIiAiydi/0PS8CsSnu8VJ8tupI/WITQ0RERG5tddC/fhmy4Qi0C++HyLpKdaI+cZxEREREZ8lOSNtayKZyaMWz3eOlo8uBzjbVyXrgNzFERETUU91+6Hv+BJit0C5+GEgYrjpRD2xiiIiIyLv2Buh7X4GsOwBt7CKI7GtUJ+qG4yQiIiLqndQhy9e7l2GX3OYZLy0DXK2qk/GbGCIiIvJB/SHou/8ERFugjXsYSMhTnYhNDBEREfnI2Qh936uQtXuhjf0WRM5EpXGCOk7SNA0L712AKZMnQgiB7dt3Yskrr8Hlcg1436FuJyIiokGQOmTFh+7VSyW3QSQWQD/yPuByBD1KUL+JmXv7bIwdMxo/fOJJPPrYE8jNzcWCu+cPat+hbiciIqIhaDgMfc8LgGaGdvG3AWtB0CMEtYmZMX0a3lu6HPX1DbDb7Xjn3aWYNnUyhBAD3neo23sn/PwI5LGN/GBdWBPWhXVhXQxfE2cz9H2vQVbvgjbqG4CICsJ7OCto4ySLxYL09DTYbOVdz5WW2ZCQEI+0tFTU1NT6vK/D0Tqk7ee+1vkSEpPh6ujw19vuYk1K8fsxwwHr0hNr4h3r4h3r4h3r0lNAa9L0FaR9N0Rikt8PHW0y9b7N76/Wi7jYWACAo/XszMzhcHTb5uu+UpdD2t6X5qYGdDidvr4tn1iTUmBvrPfrMcMB69ITa+Id6+Id6+Id69KTkWtiMpt73Ra0Jqa1zX25YkucBXZ7s/vPFku3bb7uO9TtfZOeh7+c+zWYP49rdKxLT6yJd6yLd6yLd6xLT0avSe+Zg3ZOjMPhQE1NLQoK87ueKyosQHNzC2pr6wa071C3ExERkfEF9cTeDRs34Y7b5yAlJRlWqxXz5s3Fpo+3QMqeXVZ/+w51OxERERlbUK8Ts/T9FbBarXj2109D0wS2bduBN958GwCw+MFFAICXlrza777+2E5ERETGJsZPmMmvJuA+ceieBx7F6y//1s8n9opzTqhiqc9iXXpiTbxjXbxjXbxjXXoydk36+v3M2w4QERGRIbGJISIiIkNiE0NERESGxCaGiIiIDIlNDBERERkSmxgiIiIyJDYxREREZEhsYoiIiMiQgnrFXiMwmXq/W+bgCESbTJ67cBrvIkOBw7r0xJp4x7p4x7p4x7r0ZOya9PV7mU2Mx5kizV/4iOIkREREdD6Tydzjir287cA5LPEJ6Ojw5y0HiIiIaKhMJjMcLc09nuc3MefwViAiIiJSq7d7GvLEXiIiIjIkNjFERERkSGxiiIiIyNAsxnkAABAQSURBVJDYxBAREZEhsYkhIiIiQ+LqpADSNA0L712AKZMnQgiB7dt3Yskrr8HlcqmONmg33nAdpk6ZhPz8PBw+chQ/e+rprm0xMTF4aPEiXD5+PFyuDmzYuBl/f+OtoG1XJTo6Gg/c/y1cdOFYJCUlor6+Aes/+Air16wD0P/nINDbVXrwgfswfvylsMRZ0NbWim3bd+Kvr7+Bzs7OiK7LGSaTCc/++pewWhPwwGL3NaoitS6PfOchTJp4bbccP3vqaRwtLQMQuXUBgPHjL8X8eXcgOzsLra1tWLV6LVasXB3RNTkjKjvvgp+oDhGu7pg7B5decjF+/JOfY/Wa9bh55k3IycnG7t17VEcbtLS0NBw8eBi1tbVIS0vFx5s/6dr20OL7kZyUjP/46VPYvOVTfHPBfADAkaOlQdmuiskUjcKCAvzl9b/j9b+9if0HDuKhxfejtqYOx4+f6PdzEOjtKlVVVWPp+8vx7nvLsHnLp5h50/VITU3F/gMHI7ouZyy4ez5i42KRnJyEZctXAQj8+w7Vulx5xeU4cPAQfvKzX+L9ZSvw/rIVqK9v6NoeqXW55JJx+PZDD2DJy3/GS0texYcfbUBTUxOamuwRW5NzcZwUQDOmT8N7S5ejvr4Bdrsd77y7FNOmToYQQnW0Qdux83Ps2Pk5Ghubuj1vNpsxccI1eOOtd9DS4kB1dQ1WrFyNGdOnBmW7Su3tTrz19rs4fboKUkqUl1fgyy93YdSoEQD6/xwEertKJ06cRHv72es7SABZWZkAIrsuAFBUWIDLLrsYy5at7PZ8pNelN5Fal2/cdSfeW7ocX+/dB13X0drahmPHT/iUOVxrci6OkwLEYrEgPT0NNlt513OlZTYkJMQjLS0VNTW1CtP5X3Z2FqKjo7u937IyG3Jzh0MIEfDtUobOhac1TcOoUSOxYsWqfj8HDkdrQLeHwufstjmzMPf22YiLi0OT3Y7X//ZmxNdF0zQ8/NADePnl17o9H+l1mTJ5IqZMnoj6hgZs2rQZq1avg5QyYusSE2NGUVEhPtu6Dc/95mkkWBNw6NBhvPrn19HaGtj3HKo1OR+bmACJi40FADhaHV3PORyObtvCSVxsLJxOJ3Rd73quxeFAVFQUzGZzwLe3t7cH54364P5FC9HqcODjzZ8gOSkJQO+fA6nLgG4PBcuWr8Sy5SsxPCcHkydPQENDQ79/P8K9LrNuvRkVFcewb/8BjB0zuuv5SK7L2rXr8dfX30BzczMuuKAE//yD70LXJVavWRexdYmPj4emaZgyZRKe+dVzaGxqwn0Lv4nHH/s+nvvP37lzRlhNzsdxUoC0trUBACxxlq7nLBZLt23hpLWtDWazGZp29iMVb7Ggs7MTTqcz4NtDxcJ7F2DUqBF4+lfPorOzs9/PQaC3h5ITJ0/CVl6Bf/rOwxFdl8xhw3DTjdfjL6+/0WNbJNelzFYOu90OKSUOHz6CZctXYsK1V3fLFml1aW11v/batR+guqYGTqcTb7z1DkpKiqFL93/QRVpNzscmJkAcDgdqampRUJjf9VxRYQGam1tQW1unMFlgVFaegsvlQkHB2fdbWFiA4ydOQEoZ8O2h4L5v3YOLx12Ep37+K9jt7vtw9fc5CPT2UKNpGrKyMiO6LqNHj0RiohXP/eaXeOH53+Hxf/kB4uLi8MLzv0Pu8JyIrcv5zvySBiL371Frayuqqqu7/Rt37j93kViT87GJCaANGzfhjtvnICUlGf+/vTuPrunaAzj+vVMS6pJEIkSiEkKMaS1DPTVTIpbWkJCYOliKUlMaiig6oIpX6qm+tkgkBB2e2aOGGorqqxdNiQwy5yY3cUUGmW7y/kh7n2uKWEvjJr/PWlnrnrPPsM9v3ZXzu3vvs49Wq2XUqOGcOHnqqbnpPg6lUolGo0GpVKJUKNBoNKhUKoqLizlz9hyj/UZSt25dHB0cGDp0CMeOnQR44uXV7dWJ4+jQvh3LPlhBbm6uWVll34MnXV5drK2t6dO7p+nXm6urCyOHv8x//3iyobbG5exP55n+9lyC5gcTND+YTV98TWFhIUHzg4mLv15r4/LCC12pU6eim8Ld3Y2Xhw3l/IWLpvLaGpejR4/j7f0SDe3t0Wg0jPYbSVxcPAbDzVobkzspOv1t8NNTmxpGqVQyYXwAPV/sgVKp4Ny5C3y9JZSSkpLqrtpjGzVyOL6jhputi/r9CsveX46NjQ2T3phYMY+LsZRjx06azePypMuri4NDQzasX3vPmJ0rV6NZsXJ1pd+DJ11eXaytrQicOwt3t+ao1Wpycm5x/sJFdu3+luLi4lobl7u1beNJ4NyZZvPE1Ma4LFm8gGbNXFGpVNy4YeD4iZPs3XfQdMOsrXFRKBT4j/Glb59eKBRKrkZfY/OWULKzs2ttTO4kSYwQQgghLJJ0JwkhhBDCIkkSI4QQQgiLJEmMEEIIISySJDFCCCGEsEiSxAghhBDCIkkSI4QQQgiLJEmMEKJKnJ2bELE9BHd3tyrtt2njOob6eD+hWlkGjUZDxPYQunXtUt1VEaJGkHlihKhhIraHPLQ8U69nxttzH/v4CoWC+vXrk5ubaza5X2W0Wi1FRUV/ybuuNm1cx959B9m3/yAA//js7xz+91H+tWffEz/3n5a+t4jklBS+/GqL2foGDRqQn59PaWnpX1YXIWoqeYu1EDXM5CkzTJ9btnQnKHA2Cxa+R9Yf7zt5UOKhUqkwGo2VHr+8vJycnJwq1+vu1zFYqkeN04M8TuyEEPcnSYwQNcydN8m8vHwAbt3KvefmuWnjOo4cPYadnR0vdOtCckoqS5Z+yFAfb3r16oFTIycKbhcQFfU7IaHh3LpVkYQ4Ozdh7eqVvLvwPeLjr5uWV69ZR79+vWnbxhOD4SY7Inbz07nzZue7s3Vk08Z1HDp8BDs7O17s0Z2SkhJOnDzFjojdpqnmra2teO3VCXTr2oWyMiOnTp/FaDTi1bEDgUELHikeHyxbTMOG9gT4+xHg7wdUJHo5OTk4OzchwH807dp6UlZWRmxcPCGh4aSmpgEwYEBfxo/1Z83a9YwNGE3Tps6sWLkafVYW4wLG4OHRgjp16qDTZfDd93tN1ztzxjQ8PVvh6dmKgQP6AbAoeCkJiUlsC/mKNWvXc/7CzwDY29sxcfxYOnZsj0ql4lpMLKGh4SQmJQPg5dWBBfPfYdn7yxntNxI3t+bodBls2bqNqN+vmK5z1MhX6N27J3a2thQU3Cb++nU+XrW2Sq1lQlgaSWKEqMV8hnizZ+8+FgYvRaVUARUtLVtDwsjM1GNvZ8eE8QFMnzaFj1aseuixAvz9CN++k81bQhn00gDemjaZmNhYsrKyH7jPUB9vvvt+DwsWLcGjZQumTZ1MYlIyZ8+eA2DC+LF4dWzPp+s3kJmpp3+/PvTr27tKb9FdvnI1n6z6iBMnfuTQ4aMA3Lp1Czs7O5YuWcTp02dZ/N4ujGVl+HgPYnHwu8yZO4/8/AKgYhyLr+8INm8N5Ua2gfyCAhwdGnIpMpKIXd9QXFRM586deHvGVLKzs7kWE8sXX27GsZEjaWnphIVHAJCXl4dSaT4MUaFQMO+dORjLyli+4hMKi4rwHTWcRQvnMXN2EAUFBaZtx40dQ/iOnWTps/HzHcGsWdOZPmMORUVF9Phbd7wHD2L9ho0kJ6ei1dajXds2jxwjISyVDOwVoha7Gh3Nd9/vRafLIDWtovVh/4FDREVdQa/PIvpaDFtCwvDy6kC9evUeeqwDBw9z4eeLZGRkmm7cbSu5kV7+LYp9+w+h02Vw6vRZrlyNxqtDewCeeaYufXr3JCw8gkuXIklLSyd023YyMvVVusb8/HzKy8opLCwiJyeHnJwcysvLGTxoIMnJKWwNCSM5JZW0tHS+/HorxlIj3bt3M+2vUqnYvDmEK1eiycjMJC8vj+sJiRw9epzk5BQyMjPZf+AQly9H0aNHdwBu376NsdRIcXGx6Zz364J67rmONGvmyqfrNnAtJpakpGTWf/Y5KBQM6N/HbNuIXd9w+XIU6Tod4dt3Ul+rxd29OQCOjg0xGAxERv5GdnY2CQmJ7D9wSFphRI0nLTFC1GJxcfH3rOvQvh3Dhvng7NyEZ+rWRaGo+K3j6NCQvLy8Bx4rISHR9NloNJKbm4ttg/oPPX9CQpLZsuGGgQYNGgDQpHFj1Go1MTFxZtvExMTSxrP1wy/sEbRo4UbrVh5s3fyF2XorKyuaNG5sWjYajcRfTzDbxtramlEjh9PpeS9sbW1Rq1Wo1WpKSqv2dl9Xl6YYDDfJyMg0rSsuLiY+/jouLi5m2yYm/j9WNwwGAFOszpw5x8CB/fls3RoiL//G5ctRXPzlF4qKnvwgaiGqkyQxQtRihYVFZstOTo0Iemc2x46fZNeub8nNy8PJqRHvzgtErX74v4vSu1oayssxJUAP3OeuJ3TKKUehVNyz7klQKhT8eimS0G3h95Td2Y1TUlJiGqPzp1cnjqN9+7Zs27YDnU5HYVERr782odIY3c/9rk+B4p5z3hmrP8uUiopY6bOymDU7iHbt2tC+XVv8fEcQ4O/HwuAl3LwpA4lFzSXdSUIIE4+WLVCr1WwNCeNaTCzp6TpsbW2rpS7pOh2lpaW08mhpXse7lh9FaWkpyruSo7j4BFxdm5KVlU1GRqbZX27ug1ucANp4tubHH09z/sLPJCYlo9dn0bixk/k5jaX3jIG5W3JKKvZ2djg5NTKts7Kyws3tWVJSUqt0jSUlJVy6FMm2sB0EBi1Aq61Hp07PV+kYQlgaSWKEECbp6TqUSiU+Qwbj6OhA166deWXY0GqpS35+ASdOniLAfzTPeXWkSZPGjA0YQyNHh3taKSqTqdfj6dkae3s7tNqKsT0HDh7GxsaGuXNm0rqVB44ODni2boX/GL9KJ/JLS9fRtUtn3Nya4+LSlKlvTqK+Vmt+zkw9LdzdaNTIEa223n0TmkuXIklKSmbmjGl4eLTE1dWFGdOnAPDDsROPfH0D+velb59eNGvmioNDQ3r3ehGNRmN6ykqImkq6k4QQJnHx19kaEsZQH2/8fEcQExtHSGg484LmVEt9QkLDUKtUzJr5FkZjGadOn+HM2XO0qOJswRE7v2HS6xP5dO0qrKysmDxlBgaDgeDFy/Af7Utg4Czq2NhguHmTq1eiyamkC2bzlhDenPwGSxYvpOB2AUeO/MAv//kV7R2JzJ49+5k6ZRKrVn6IjY2N6RHrO5WXl7Ny1RomThjLgvmBqFQqYmLj+OCjj826tCqTn1+Az5BBjB/nj1qtRqfLYOPn/yQ6+lqV4iSEpZEZe4UQFmXZ0mD0mXrWb/i8uqsihKhm0hIjhHhqNW/+LC4uTYmNjUOj0dC3Ty9at/IgImJ3dVdNCPEUkCRGCPFU8x40EOfXJgKQmpbG8pWfmM1UK4SovaQ7SQghhBAWSZ5OEkIIIYRFkiRGCCGEEBZJkhghhBBCWCRJYoQQQghhkSSJEUIIIYRF+h9ylMFyTF7bwAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_learning_rate(lr_manager)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One last trick worth discussing is what we can do with the embeddings that our network learned. Similar to word embeddings, these vectors contain potentially interesting information about how the values in each category relate to each other. One really simple way to see this visually is to do a PCA transform on the learned embedding weights and plot the first two dimensions. Let’s create a function to do just that." ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "def plot_embedding(model, encoders, category):\n", " embedding_layer = model.get_layer(category)\n", " weights = embedding_layer.get_weights()[0]\n", " pca = PCA(n_components=2)\n", " weights = pca.fit_transform(weights)\n", " weights_t = weights.T\n", " fig, ax = plt.subplots(figsize=(8, 8 * 3 / 4))\n", " ax.scatter(weights_t[0], weights_t[1])\n", " for i, day in enumerate(encoders[category].classes_):\n", " ax.annotate(day, (weights_t[0, i], weights_t[1, i]))\n", " fig.tight_layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now plot any categorical variable in the model and get a sense of which categories are more or less similar to each other. For instance, if we examine \"day of week\", it seems to have picked up that Sunday (7 on the chart) is quite different than every other day for store sales. And if we look at \"state\" (this data is for a German company BTW) there’s probably some regional similarity to the cluster in the bottom left. It’s a really cool technique that potentially has a wide range of uses." ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGhCAYAAACQ4eUqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xt4FOX99/HP7mbDZpMQQhAJZxDw2FptfxQ5idR6aD2AUiwKRQSt1latIlbUXy36gFKhrdWnD61UREDxhBw9AAoeYgK0iuKJU4IKxJCQhCSbZJPsPn8AKyuzEGSyk3vzfl3XXmTnnrnnO19yyceZ2VnX2f0vCgsAAMAwbqcLAAAA+C4IMQAAwEiEGAAAYCRCDAAAMFKS0wXYyZ+aprq6oNNlAAAAG3m9yQpUVR62PGFCjD81TSPH3OR0GQAAoAk89/Q/DgsyCRNiDp6Bee7pf8T5bIxLaa3bqHJfmSQ+rR6N3lijL9boS2z0xhp9sZZYffF6kzVyzE2W/7YnTIg5qK4uqLpgfENMfV3dgX2a/8tiL3pjjb5Yoy+x0Rtr9MVay+kLN/YCAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIyUcM+JQeK56cbrNeTcQVHLnpo7Xyteec2higAAzQEhBkbIeS9Pc56aF3lfXV3tYDUAgOaAEAMjBINBlZeXO10GAKAZIcTACD/64Vn616zHVFpWppycXC1ZukKhUMjpsgAADiLEoNl7/4ONynkvV3v3lqpnj+4afc0v5fP59OzCF5wuDQDgIEIMmr3c3HWRn7/88iuFQiFNGH8tIQYAWjg+Yo3mK7WDdMJZ+/88RH7BDvl8PqWnpztUGACgOeBMDJolV9fz5erwY7ncHoVDDQoX5in8xSpJUpfOnVVTU6uKigqHqwQAOIkQg+YnNTsSYCRpzLknKufzc1UW3qkeHVL1qzGjtGr1mw4XCQBwGiEGzY+/QyTASFKntl5NuryzUn5xp4r3FGvV6je1eMlyBwsEADQHhBg0P4HdCocaIkHm4cX734c2zZYChQ4XBwBoLrixF81PVeH+e2BCDZK0/56Y3bkEGABAFFvPxFzw05/o3MED1bVrF23Zuk1THpgWc91WrVrp+gnX6odnn636+jq98eZbeubZ5xo9jsQW/mKVwiWbJH+2FNgtVRFgAADRbA0xZWXlWrxkuU46qYd69+51xHXHXTtGrdNb67e3/F5+v1/3Tr5LpaWlevW1lY0aRwtQVUh4AQDEZOvlpHXrN2jd+g0qL993xPWSk5M1oH8/PfvcC6qqCmjPnmItXbZCQ887t1HjR+aK88vJfTf3F72hL/SF3tAX+mL3MX3DkRt7s7M7KCkpSQUFOyLL8vML1LlzJ7lcrqOOh8PhmHOntW6j+rq6Jq3fSnpGZtz3aQp6Y42+WKMvsdEba/TFWqL0JcnrjT0WxzoiUnw+BYPBqC/wqwoE5PF4lJycfNTx2tramHNX7itTXTDYpPV/W3pGpirKS+O6T1PQG2v0xRp9iY3eWKMv1hKpL97k5JhjjoSY6poaJScny+12R4JKqt+vhoYGBYPBo44fWfjAK14OPc0Vz/2agN5Yoy/W6Ets9MYafbGWaH2JfQyOfMR69+5C1dfXq1u3rpFl3bt301c7dyocDh91HAAAwNYQ43a75fV65Xa75Xa55PV65fF4DlsvGAzq3ZxcXTXySvn9fp3Qrp0uueRneuONtY0aBwAAsPVy0hXDL9cvRgyPvJ83d7Y+/uRTTXlgmv5w1x367LPNennxUknSnKfmacL4sXr8739RfUO93nhjbdTHp482DgAAWjZbQ8wLLy7SCy8ushx76OEZUe9ramr02OOzYs51tHEAANCy8bUDAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABgpye4J3W63xowepcGDBsjlcikvb71mPzlX9fX1h6371JP/jC4mKUk7d+3SpLvulSTddOP1GjjgnKhtpzwwTdu259tdNgAAMIztIWb4sEt12qmnaOKkyaqvb9Ckib/XqF+O1NPzFhy27thxN0S9n/7wg8rJyYta9vrK1Xpq7ny7ywQAAIazPcQMPW+I5j69QKWlZZKkF15cpFt+9xvNm/+MwuFwzO1OOqmnOnfqpLVvvX2cFbgOvJzg1H5NQG+s0Rdr9CU2emONvlhLhL7EPgZbQ4zf71e7dlkqKNgRWbY9v0BpaanKymqr4uKSmNsOHTJYH3zwYST8HDR40AANHjRApWVlWrPmLS1f8doRw1Ba6zaqr6s7/oM5RukZmXHfpynojTX6Yo2+xEZvrNEXa4nSlySvN/aYnTtK8fkkSYHqQGRZIBCIGrOSnJys/v376fH/G32PzKuvvq55859VZWWlevU6SbfdcrNCobBWvPJazLkq95WpLhg8nsM4ZukZmaooL43rPk1Bb6zRF2v0JTZ6Y42+WEukvniTk2OO2RpiqmtqJEn+FL8qKir3/+z3R41ZOadfX9XWBvXf9z+IWp5/yBmdLVu2avGSZRo8aMARQ4wUPvCKl0NPc8VzvyagN9boizX6Ehu9sUZfrCVaX2Ifg60fsQ4EAiouLlG37l0jy3p076bKyiqVlOyNud3Q84Zo7VvvKBQKHXH+UPjI4wAAoOWw/Tkxb7y5RlcMu0yZmW2Unp6uESOGa83at2Pex5Kd3UF9+vTSmjVvHTbWr19fpaTsvwzVs2cPXX7ZJcpbt8HukgEAgIFs/3TSopeXKj09XY9Mnya326Xc3HV6duHzkqQJ46+VJD0xe05k/aHnnavPPt+s3YWFh8110QXn64YJ4+TxeLR3b6lWrlytZctfsbtkAABgINtDTCgU0pyn5mnOU/MOGzs0vBw0f8HCmHPdP2WqnaUBAIAEwtcOAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGCkJLsndLvdGjN6lAYPGiCXy6W8vPWa/eRc1dfXH7buTTder4EDzokam/LANG3bnn/McwEAgJbF9hAzfNilOu3UUzRx0mTV1zdo0sTfa9QvR+rpeQss13995Wo9NXe+LXMBAICWw/bLSUPPG6KXFi1RaWmZKioq9MKLizTk3EFyuVxxmssV55eT+27uL3pDX+gLvaEv9MXuY/qGrWdi/H6/2rXLUkHBjsiy7fkFSktLVVZWWxUXlxy2zeBBAzR40ACVlpVpzZq3tHzFawqHw99pLklKa91G9XV1dh5Wo6RnZMZ9n6agN9boizX6Ehu9sUZfrCVKX5K83thjdu4oxeeTJAWqA5FlgUAgauxQr776uubNf1aVlZXq1esk3XbLzQqFwlrxymvHPNdBlfvKVBcMHv/BHIP0jExVlJfGdZ+moDfW6Is1+hIbvbFGX6wlUl+8yckxx2wNMdU1NZIkf4pfFRWV+3/2+6PGDpV/yFmWLVu2avGSZRo8aIBWvPLaMc/1jfCBV7wcepornvs1Ab2xRl+s0ZfY6I01+mIt0foS+xhsvScmEAiouLhE3bp3jSzr0b2bKiurVFKy96jbh8Ih2+YCAACJzfYbe994c42uGHaZMjPbKD09XSNGDNeatW8rHD48SfXr11cpKfsvDfXs2UOXX3aJ8tZt+E5zAQCAlsX2j1gvenmp0tPT9cj0aXK7XcrNXadnFz4vSZow/lpJ0hOz50iSLrrgfN0wYZw8Ho/27i3VypWrtWz5K42aCwAAtGy2h5hQKKQ5T83TnKfmHTZ2MLwcdP+Uqd95LgAA0LLxtQMAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMj3HH7LVr4zFydduopTpcCAGgmCDFo9gYN7K9WrVo5XQYAoJkhxKBZy8zM1FUjR2jWP//tdCkAgGaGEINm7dc3XKeXFy9VSUmJ06UAAJoZQgyarfN/cp6SPElatfpNp0sBADRDhBg0S1lZWbryymGa9c/ZTpcCAGimkpwuAIiS2kHyZ6tHn7Zqk5Ghv/31z1HD9937B61Z+zbhBgBAiEHz4ep6vlwdfiyX26NPFNKkvy6Tdr8XGX/kz9M065+ztfHDTQ5WCQBoLmwPMW63W2NGj9LgQQPkcrmUl7des5+cq/r6+ugdJyXpunG/0hmnn6aMjNYqLS3T6ytXa8Urr0XWuenG6zVwwDlR2055YJq2bc+3u2w4LTU7EmAkqbbBrZ2ukxXa+44UKIysVlS0R6WlpU5VCQBoRmwPMcOHXarTTj1FEydNVn19gyZN/L1G/XKknp63IGo9j8etsrJy/Z9p01VUtEddu3bR5LvvVGlpmd7LzYus9/rK1Xpq7ny7y0Rz4+8QCTAHudweKTU7KsQAAHCQ7SFm6HlDNPfpBSotLZMkvfDiIt3yu99o3vxnFA6HI+vV1gb13PMvRt7v2PGF3n9/o04+uXdUiDl2rgMvJzi1XxMcpTeBQoVDDVFBJhxqkKoKI9teNWps4+YySiIdi53oS2z0xhp9sZYIfYl9DLaGGL/fr3btslRQsCOybHt+gdLSUpWV1VbFxbGf9eF2u3XyyX20dOnyqOWDBw3Q4EEDVFpWpjVr3tLyFa9FhaFvS2vdRvV1dcd/MMcoPSMz7vs0ReN6U6vavRvVkHWm5PJI4QYl7d2oVt5aKUF7y++MNfoSG72xRl+sJUpfkrze2GN27ijF55MkBaoDkWWBQCBqLJZx145RdSCgtW+9E1n26quva978Z1VZWalevU7SbbfcrFAoHHXfzLdV7itTXTB4PIdxzNIzMlVRzn0aVo6pN+XLpd3/kfzZUmC3GqoKFd+/yfjhd8YafYmN3lijL9YSqS/e5OSYY7aGmOqaGkmSP8WviorK/T/7/VFjVsaMHqWTT+6tBx58SA0NDZHl+Yec0dmyZasWL1mmwYMGHDHESOEDr3g59DRXPPdrgu/Qm6rd+18Jjd8Za/QlNnpjjb5YS7S+xD4GWx92FwgEVFxcom7du0aW9ejeTZWVVSop2Wu5zdhfXaPvf+8MPfDgw5HgE0soHLKzXAAAYDDbn9j7xptrdMWwy5SZ2Ubp6ekaMWK41qx92/I+lmvHjtb3zjhdUx58SBUVFYeN9+vXVykp+y9D9ezZQ5dfdony1m2wu2QAAGAg2z+dtOjlpUpPT9cj06fJ7XYpN3ednl34vCRpwvhrJUlPzJ6jdu2ydPFFFygYDOqxR2dEtv/0s8/10MP73190wfm6YcI4eTwe7d1bqpUrV2vZ8lfsLhkAABjI9hATCoU056l5mvPUvMPGnpg9J/JzcXGJrhr1qyPOdf+UqXaXBwAAEgRfAAkAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMlOV0AAAAw398fnaH2J5xw2PLJ996vbdu2N8k+CTEAAOC4Tb7nfrnd31zgufCC8zXk3EHavj2/yfZJiAEAAMetoqIi6v0Pz/6B3nk3R+FwuMn2yT0xAADAVl06d1L37t301tvvNul+CDEAAMBWgwcPVH5+gb76ameT7ocQAwAAbONyuTSg/zlNfhZGIsQAAIDjkdpBOuGs/X9KOuOM05SR0Vrv5rzX5Lvmxl4AAPCduLqeL1eHH8vl9igcalC4ME+DB52kDz/apPLyfU2+f0LMMer7Pz/SBRf8RCf17CG/36+rRo11uiQAAOIvNTsSYCTJ5faoVed+6vs/3TTrn0/EpQTbQ4zb7daY0aM0eNAAuVwu5eWt1+wn56q+vv6Y1z2WueIluVWyPv74E23a9LFG/XKkY3UAAOAof4dIgDmob58MhcLS+g3/jUsJtt8TM3zYpTrt1FM0cdJk3fr7SercuXPMf+yPtu6xzBUv77yTo0UvL9XmzVsdrQMAAEcFdiscaoha9PYnZRr32/tUV1cXlxJsDzFDzxuilxYtUWlpmSoqKvTCi4s05NxBcrlcx7zuscz1DVccX07u24QXvaEv9IXe0JeE7UvV1woX5kWCTDjUoPDuPCnwdRMf0zdsvZzk9/vVrl2WCgp2RJZtzy9QWlqqsrLaqri4pNHrBgLVjZ7rUGmt26g+DgnQn5YuSUrPyIz6E4ejN9boizX6Ehu9sUZfrMWlL+X/VUPNlwr5TpC7Zo88tXskm/eb5PXGHrNzRyk+nyQpUB2ILAsEAlFjjV03HAo3eq5DVe4rU10w+J2PobEClfsfr1xRXqr0jExVlJc2+T5NRG+s0Rdr9CU2emONvliLb19KJW1ustm9yckxx2wNMdU1NZIkf4pfFRWV+3/2+6PGGrvuscwVLXzgZbPUDpI/WwrslqoKY+yj6b4fwkyHngKkN9+gL9boS2z0xhp9sZZofYl9DLaGmEAgoOLiEnXr3lVfFxVJknp076bKyiqVlOw9pnXD4XCj52pqVp+Dl76Kaw0AACCa7Tf2vvHmGl0x7DJlZrZRenq6RowYrjVr37b8FsujrXssczWZb30OPi3Fq+7fG6wOXftIkrp166ounTvJ4/EcaRYAAGAz258Ts+jlpUpPT9cj06fJ7XYpN3ednl34vCRpwvhrJUlPzJ5z1HUbMx4X3/oc/A97purGC06U1F2SNP2hByRJv/3dHdpTvCe+tQEA0IK5zu5/USJcMJM3OVnXXHer5v/7b/be2JvaQe7Tx0cFmXCoQaFNs6VAoSTXITdQJUQrbURvrNEXa/QlNnpjjb5YS6y+HOnfd74A8miqCi0+B597IMAAAACn8N1JjRD+YpXCJZu+9ekkAADgJEJMY1UVEl4AAGhGuJwEAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJH4AkgAcdWjezddc80v1ad3L9XV1eujTZv017897nRZAAxEiAEQN506dtR99/5BK155TXOemqdwKKxOnTo6XRYAQxFiAMTNVSOvVF7eer3w4suRZTt37XKwIgAmI8QAiAuXy6Uzz/y+Fi9Zpv+972516pitHV98qaefXqAvv9rpdHkADMSNvQDionXr1vL5WumyS3+mnJxcPfTwDO3dW6p777lLPp/P6fIAGIgQAyAu3G6XJCkvb71WrX5T+QU79K8nnpTL5dbZZ//A4eoAmIjLSQCaVmoHyZ+tfbVFamho0K7dhZGhhoYGFRUVqV1WWwcLBGAqQgyAJuPqer5cHX4sl9ujcKhBBbtK1eHE9pFxt9utE05op+LiEgerBGAqQgyAppGaHQkwkuRye7Tio6B+ff4AffzJp9q2LV8XXfRThUIh/ee/HzhcLAATEWIANA1/h0iAOei9LQG1rnlXo0aNVKrfr63btuvBqdNVW1sryeVMnQCMRYgB0DQCuxUONUQFmXCoQSuWr9CK5//tYGEAEgWfTgLQNKoKFS7MUzjUIGl/gAnvzpUChUfZEAAahzMxAJpM+ItVCpdskvzZUmC3VEWAAWAfQgyAplVVSHgB0CS4nAQAAIxEiAEAAEYixAAAACNxTwxahF+MGK7+/c9Ru6y2ClRXa9NHH2vuvGdUXl7udGkAgO+IEIMWYeeu3XriiSdVVLRHrTNaa8zoUbr5phs09aE/O10aAOA7IsSgRcjJyY38vKe4WEuWLtetv/uNgxUBAI4X98SgxUlJSdHAAefo88+3OF0KAOA42HompnfvXhp/3Vh16pitr4uKNGfOPG36+BPLdc8660xddunP1bVLF4XCIW3ZvFVz5y1QYeHXkqQT2rXTY3+fqZqamsg2H360STNmPmpnyWhBBg44R9dPGCefz6ctW7Zq2sMznC4JAHAcbAsxfr9fd915u55d+LzWrH1b/fr11cQ7btVtt09SWdnhN0/6U/xaumyFPvnkMzU0NOjqUSM1aeLvdfvEP0Std9PNtykQCNhVJlqwDf95X1u3bVdWVlv9YsQVumHCOP3lb485XRYA4DuyLcT07fsjlZWVadXqNyVJ77yTowt/er76n9NPK1557bD13815L+r9suWv6mcXX6jU1FRVVVUdRyUuOfdtuHwLb2wO9Ca1Q9Tj7mtqalVYWKTCwiLt2vUP/b//+1d1eqGTdu7cFf/aIvidsUZfYqM31uiLtUToS+xjsC3EdOvaRfkFO6KW5RcUqFvXLo3a/vTTT1VJyd7DAswj06fK43Fr67btmr9goXbt2n3EedJat1F9Xd2xFW+D9IzMuO/TFE70pvaE/mrIOlNyeaRwgzwlG9VqT843NbXOkCSlpKY79nfH74w1+hIbvbFGX6wlSl+SvN7YY42aIClJHo8n5ngwGJTP5zvssk9VVUAZGa2POn/HjtkaM3qU/vXEk5Fl+yoqNPmePyq/YId8Pp+uvGKY7pk8SRPvvFvV1TUx56rcV6a6YLARR2Wf9IxMVZSXxnWfpnCkN6nZcrc9Uy7X/t9Zj8ejKy75ida//rXKi/KVldVWV/3iSm3fXqBtWzYrHA7Htz7xOxMLfYmN3lijL9YSqS/e5OSYY40KMTf+erwGDRwQc/xPU6aqpqZGaWlpUctTU/2qOULgkKTs7A66d/JdWvjcC1q3bkNkeW1trbZtz5ckBQIBPT1vgQYO6Kc+fXpr48aPjjBj+MArXg49zRX/fwybN4d64z9RLvc3oTscljpltdKQW65Tmj9ZZWXl+vCjTXru+RcVDofiV1cEvzPW6Ets9MYafbGWaH2JfQyNCjGPPT5Ljz0+64jrtD+xvX5+8YVRy7p376a8vPUxt+nUsaPuvfcuvfTSYq1a9eZR63Dgf5hhosBuhUMNkSATCkt/WbpToU2zpQDfpgwAicK258SsX79BmZmZGjp0iDwej/r376euXTor571cy/U7deqo++79gxYtWqKVq944bLzXST3VqWNHuVwu+Xw+XT1qpMIKa/PmrXaVjERVVahwYZ7CoQZJUjjUoPDuXAIMACQY227sraoKaPqfZ2r8dWM1buxoFRXt0YyZj6q0tEySlJWVpZmPTNPtE+9WSUmJLrvkZ8rIaK1rrr5K11x9VWSeaQ89os8+36z2J7bXVSOvVJuMNgoGa7Vl63ZNnTpd1dXVdpWMBBb+YpXCJZuiPp0EAEgstj7sbvOWrbrr7vssx0pKSjR23A2R9/+Y9YT+MeuJmHPl5ORGPSoeOGZVhYQXAEhgfO0AAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEZKsnOy3r17afx1Y9WpY7a+LirSnDnztOnjTyzXPaFdOz3295mqqamJLPvwo02aMfPR7zQfAABoWWwLMX6/X3fdebueXfi81qx9W/369dXEO27VbbdPUllZecztbrr5NgUCAdvmAwAALYNtl5P69v2RysrKtGr1m6qvr9c77+Toyy93qv85/eI8nyvOLyf33dxf9Ia+0Bd6Q1/oi93H9A3bzsR069pF+QU7opblFxSoW9cuR9zukelT5fG4tXXbds1fsFC7du0+rvnSWrdRfV3ddziC45OekRn3fZqC3lijL9boS2z0xhp9sZYofUnyemOPNWqCpCR5PJ6Y48FgUD6f77DLQlVVAWVktLbcZl9FhSbf80flF+yQz+fTlVcM0z2TJ2ninXerurrmmOc7qHJfmeqCwcYclm3SMzJVUV4a132agt5Yoy/W6Ets9MYafbGWSH3xJifHHGtUiLnx1+M1aOCAmON/mjJVNTU1SktLi1qemupXTXWN5Ta1tbXatj1fkhQIBPT0vAUaOKCf+vTprY0bPzrm+b4RPvCKl0NPc8VzvyagN9boizX6Ehu9sUZfrCVaX2IfQ6NCzGOPz9Jjj8864jrtT2yvn198YdSy7t27KS9vfWN2IUkKH1Lnji++PO75AABA4rLtxt716zcoMzNTQ4cOkcfjUf/+/dS1S2flvJdruX6vk3qqU8eOcrlc8vl8unrUSIUV1ubNW7/TfAAAoGWx7cbeqqqApv95psZfN1bjxo5WUdEezZj5qEpLyyRJWVlZmvnINN0+8W6VlJSo/YntddXIK9Umo42CwVpt2bpdU6dOV3V1daPmAwAALZutD7vbvGWr7rr7PsuxkpISjR13Q+R9Tk6ucnKOfFblSPMBAICWja8dAAA6Vlu/AAAL2ElEQVQARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBIhBgAAGAkQgwAADASIQYAABiJEAMAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICRCDEAAMBISXZO1rt3L42/bqw6dczW10VFmjNnnjZ9/InlusMuv1TDh116yBKXfL5WmjHzUa1bv0EntGunx/4+UzU1NZE1Pvxok2bMfNTOkgEAgKFsCzF+v1933Xm7nl34vNasfVv9+vXVxDtu1W23T1JZWflh67+8eKleXrw08r5v3x/pxhvG6/0PNkatd9PNtykQCNhVJgAASBC2hZi+fX+ksrIyrVr9piTpnXdydOFPz1f/c/ppxSuvHXX7oUPO1bs5uaqrqzvOSlwHXk5war8moDfW6Is1+hIbvbFGX6wlQl9iH4NtIaZb1y7KL9gRtSy/oEDdunY56rZt22bqzDO/p8n33n/Y2CPTp8rjcWvrtu2av2Chdu3afcS50lq3Uf1xB6Fjl56RGfd9moLeWKMv1uhLbPTGGn2xlih9SfJ6Y481aoKkJHk8npjjwWBQPp/vsMs+VVUBZWS0Pur8Q84drB1ffKn8/ILIsn0VFZp8zx+VX7BDPp9PV14xTPdMnqSJd96t6uqamHNV7itTXTB49IOyUXpGpirKS+O6T1PQG2v0xRp9iY3eWKMv1hKpL97k5JhjjQoxN/56vAYNHBBz/E9TpqqmpkZpaWlRy1NT/ao5QuA4aMi5g7R8xatRy2pra7Vte74kKRAI6Ol5CzRwQD/16dNbGzd+dITZwgde8XLoaa547tcE9MYafbFGX2KjN9boi7VE60vsY2hUiHns8Vl67PFZR1yn/Ynt9fOLL4xa1r17N+XlrT/idt8743RlZrbRO+/mHLWOcCL8XQAAAFvY9pyY9es3KDMzU0OHDpHH41H//v3UtUtn5byXe8TtzjtvsNat26CqquhLUb1O6qlOHTvK5XLJ5/Pp6lEjFVZYmzdvtatkAABgMNtu7K2qCmj6n2dq/HVjNW7saBUV7dGMmY+qtLRMkpSVlaWZj0zT7RPvVklJiSQpNTVV//OjH+qhh2ccNl/7E9vrqpFXqk1GGwWDtdqydbumTp2u6upqu0oGAAAGs/Vhd5u3bNVdd99nOVZSUqKx426IWlZVVaUxYydYrp+Tk6ucnCOfxQEAAC0XXzsAAACMRIgBAABGIsQAAAAj2XpPDHDQ8GGXqt+P+yo7u4MqK6u0bv0GPfPs86qtrXW6NABAgiDEoEn06dNbS5au0NfFpfK6pesnXKsUn0//mPWE06UBABIEIQZN4uHpMyW5Io++fu75F3X9hHHSkZ+ZCABAo3FPDOIiPT39sAcaAgBwPAgxaHIpKSm65JKLtWbtW06XAgBIIFxOQpNKSvLo1pt/p6KiPXp58TKnywEAJBDOxMBeqR2kE86SUjvI7Xbrhglj5Uvx6ZEZf1MoFHK6OgBAAuFMDGzj6nq+XB1+LJfbI4UadPOQZLU/oZX+948P8tFqAIDtOBMDe6RmfxNgJF3/0w46rXcX/XP+CiUlJSkjI0MZGRlyuVwOFwoASBSciYE9/B0iAUaSzjsjQ5I0ZVL0F3z+9ne3a09xcVxLAwAkJkIM7BHYrXCoIRJkrv7bVoVDDfIVvKBA0WZJYWfrAwAkHC4nwR5VhQoX5ikcapAkhUMNCu/Ok6d2j8OFAQASFWdiYJvwF6sULtkk+bOlwG6p6mspI9PpsgAACYoQA3tVFe5/SZK4iRcA0HS4nAQAAIxEiAEAAEYixAAAACMRYgAAgJEIMQAAwEiEGAAAYCRCDAAAMBIhBgAAGIkQAwAAjESIAQAARiLEAAAAIxFiAACAkQgxAADASAn3LdZeb3Kc9+hSktcrb3KypHCc993c0Rtr9MUafYmN3lijL9YSqy9H+nc9YULMwYMcOeYmhysBAAB283qTVRcMRi1znd3/IvNj2gH+1DTV1QWPviIAADCG15usQFXlYcsT5kyMJMsDBAAAZvv2GZiDuLEXAAAYiRADAACMRIgBAABGIsQAAAAjEWIAAICREurTSU4666wzddmlP1fXLl0UCoe0ZfNWzZ23QIWFXztdmuPatMnQhPHj1LNHd2VltdWfpkzVJ59+5nRZjnC73RozepQGDxogl8ulvLz1mv3kXNXX1ztdmqMu+OlPdO7ggeratYu2bN2mKQ9Mc7qkZiEpKUnXjfuVzjj9NGVktFZpaZleX7laK155zenSHDf+urE6++wfyJ/iV01NtXLz1mve/GfV0NDgdGnNgtfr1SPTpyo9PU3XTUjc56d5srv0ut/pIhLBST176uNPPtVTc+dr2bJX1LNnd105/HK99voqp0tzXKtWreRNStIrr61Uvx//j3JycrWnuNjpshxxxfDL9IMzv6//vf9BrXjldV180YXq2DFbH374kdOlOSorK0uff75FJSUlyspqq7VvveN0Sc2C15uk7t266en5z2j+goX69LPPdf2EcSop3quvvtrpdHmOKirao0UvL9GLLy3WW2+/q4suPF9t27bVp5997nRpzcKoX46UL8WnNm0ytHjJcqfLaTJcTrLJuznv6b///UA1NTWqq6vTsuWvqlOnjkpNTXW6NMeVl+/T6ytXa8uWrQqHE+bZit/J0POG6KVFS1RaWqaKigq98OIiDTl3kFwul9OlOWrd+g1at36Dysv3OV1Ks1JbG9Rzz7+or78uUjgc1o4dX+j99zfq5JN7O12a43bu3KXa2m+eHRKW1KHDic4V1Iz06N5NZ531fS1evMzpUpocl5OayOmnn6qSkr2qqqpyuhQ0E36/X+3aZamgYEdk2fb8AqWlpSorq62Ki0scrA4mcLvdOvnkPlq6NHH/z/pYXH7ZJRo+7FKlpKRoX0WF5i9Y6HRJjnO73brh+uv073/PdbqUuCDENEJSUpI8Hk/M8WAwGHWGoWPHbI0ZPUr/euLJeJTnqGPtTUuW4vNJkgLVgciyQCAQNQYcybhrx6g6EOBy2wGLlyzT4iXL1KljRw0a1F9lZWVOl+S4S35+sb744kt98ulnOu3UU5wup8kRYhrhxl+P16CBA2KOH3qjanZ2B907+S4tfO4FrVu3IV4lOuZYetPSVdfUSJL8KX5VVOz/igy/3x81BsQyZvQonXxybz3w4EPcvPotO3ftUsGOL/SbG2/Qg1Mfdrocx5zYvr0uvOB83XX3fU6XEjeEmEZ47PFZeuzxWUddr1PHjrr33rv00kuLtWrVm3GozHmN7Q32n3UpLi5Rt+5d9XVRkaT9164rK6tUUrLX4erQnI391TU64/TTNOXBhyIBGNHcbneLvyfmlFP6qHXrdM3481RJ+8+Up6SkaNY/HtWMmY9q85atDldoP0KMTTp16qj77vmDXlq0WCtXveF0Oc2O1+uN/JyUlCSv16v6+voWd6npjTfX6Iphl2nLlq2qr2/QiBHDtWbt2y2uD9/mdrvl8Xjkdrvldrnk9XoVCoU44yDp2rGjdcbpp+lPD0xTRUWF0+U0C61atdI5/fpq3fr/KBAIqEuXzrpy+OXa2MI/5ZfzXp4+2PhND07u01s33ThBk/5wnyorEzP8us7uf1HL/q+nTW769QQNHjxQwW990+a0hx7RZ59vdqiq5mPhM4ffZNYSLzW53W79aszVGjRwgNxul3Jz1+nfc55WXV2d06U5asSVw/WLEcOjln38yact/nkx7dpl6fG//0XBYFChUCiy/NPPPtdDD89wsDJntWqVrIl33KaePborKSlJ5eX7lLdug55/4aXD/hvckp126imaeMetCf2cGEIMAAAwEs+JAQAARiLEAAAAIxFiAACAkQgxAADASIQYAABgJEIMAAAwEiEGAAAYiRADAACM9P8BBOJ1zHeT1WAAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_embedding(model, encoders, 'DayOfWeek')" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGhCAYAAACQ4eUqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xl4U2Xe//HPSRdK2lJKEVqgBWeEFlFn9FFANhVxFAUFxQUFFXHcxh2EwVGfn8uIo+DM+OgoDigiisiqoKJWFERkc6VAyyKlCF1outAmTZckvz8KgdiWtpA0nPb9uq5cQ+77Pud8+70Y+fSckxzjnH6XeQQAAGAylmAXAAAAcDwIMQAAwJQIMQAAwJQIMQAAwJRCg11AY1gjo1RZWRHsMgAAQBMKCwuXw15aY9w0IcYaGaXrxt4d7DIAAEAQvP/2qzWCjGlCzOEzMO+//SpnY46Loag2bVV6sEgSn6oPDHocePQ48Ohx4NHjxggLC9d1Y++u9d9+04SYwyorK1RZQYhpPENVlZWHesf/aQKDHgcePQ48ehx49NhfuLEXAACYEiEGAACYEiEGAACYEiEGAACYEiEGAACYEiEGAACYEiEGAACYEiEGAACYEiEGAACYEiEGAACYkt8fO2CxWDR2zGgNGthfhmFo/fqNmvXmHFVVVdVYO/62W3TOOX+UtbVVTmeZ1q3fqLnvvCeXy+XvsgAAQDPj9zMxI0cM1+k9UzRx0qN64KFJ6tKli0bfcF2ta1d8+rkenjBZ48bfqUl/fVxdkxJ11ZXD/F2SXz3x+BTNnzdHKck9aoyPumakHpn4oO6683afuXG3jtX8eXMUFxfnHWvTJlrz581R8m/2AwAAGsbvIWbwRRdq8ZIPVVhYpJKSEi1ctEQXXjBQhmHUWLtv336Vlx95mKNHUnx8x3qOYAT5JVVUVOi6a6+pMS5J27ZtV8+UZJ+5lOQeyrfZ1LNnypGxlBRVVFRo167dTVb3ydPD5vqix/S4ObzoMT0+GV+18+vlJKvVqvbt45SZucc79svuTEVFRSourp3y8201trnqymEaOWK4WrdurYMlJXrn3fnHPEZUm7aqqqz0Z9mNEhoapm/XbVLfvufqvN69lZ6xwzveKqK1tu/crfj4juqS2FXFBw+qdUSEEjolaPlHn+qsM8/UT5u3SZLOOuss7c7MUuvI6CatPzomtkmP1xLR48Cjx4FHjwOPHjdMaFhY3XP+PFDriAhJkqPM4R1zOBw+c7/1wYfL9cGHy9W5UycNHNhPRUVFxzxG6cGiQ48vD46qqkrl5eXq88+/0LChl2jjhg3e8XJnmdI2b5bT6VRi5476de8e/f7Us/TLL7v13abv9Jd77lBJcaEk6XenJumHH37yvm8K0TGxTXq8logeBx49Djx6HHj0uOHCwsPrnPPr5aQyp1OSZG1t9Y5ZrVafubrs279fmXuydM9dd9RzFE+QX9U++HC5unVL0h/OOsNn3O12afuOnUpJ6SHJo54pPZSRsV27MzMVF9dO0dFRatUqXN26Jik9PaPJ6z45ethcX/SYHjeHFz2mxyfjq3Z+DTEOh0P5+TZ17ZbkHTu1W1eVltplsxXUu73FYmnAPTEnh4MHS7Ti01Rde+3VNebS0zPUs2eyJCklOVnpGdvldru1c9cv6pmSrB49TpPH49H2HTuaumwAAJoNv9/Yu/LLr3T1iCsVG9tW0dHRGjVqpL5a9bU8Ht8k1apVK114wUDvmZrExC66ZuRV+unnzf4uyT8i46VTzpZCjpzWWrb8Y3Xu1EnnnP1Hn6XbtmUosUsXtW0bo1NP7aqMQ/fNpKdvV0pKD6UkJ2v37kyfm5oBAEDj+P17YpYsXabo6GhNe36qLBZD69Zt0HvzF0iSbh9/qyRp5qzZkjzq3/98jR0zWqGhoSouPqj1GzZpwcLF/i7phBlJQ2TE95FhCZFhTZARUypJKi0t1ScrPtN1117tc7lsx85dcrlcuuLyocrJzfXeF5SRsV2jb7hWjrIybUvPCMrPAgBAc+H3EON2uzX7rbma/dbcGnPV4aVaeXmF/v7s8/4+vP9FJngDjCTJMKSozpI1XnLkaPlHn+jSPw1ReHiYtmyp/uRRZWWlfvllty4ZcpG+XrPWu6vtO3YqMbGL3G63PvpoRTB+GgAAmg0eO1Afa/yRAHOIYVikyARJ1fcBffTxCoX/5u7pbekZat26tdLTt3vHnE6nfv11n8LCwpSesV0AAOD4+f1MTLPjyJbH7fIGmWcW7ZPH7ZLs2d4li5d8oMVLPvDZbN57CzTvvQU1dvfXR58IbL0AALQQnImpjz1Hnpz11cFFksftkid7neTICXJhAAC0bJyJaQBPVqo8tjTJmiA5siU7AQYAgGAjxDSUPYfwAgDASYTLSQAAwJQIMQAAwJQIMQAAwJQIMQAAwJQIMQAAwJQIMQAAwJQIMQAAwJQIMQAAwJQIMQAAwJQIMQAAwJR47EA9nnh8inqd3lOS5HK5ZCso0OrV32jBwsWaPOlhuVxuTZv+L59tJk54UBaLRc+/8GIwSgYAoEUgxDTAsuUfa9nyTxQSYlG3bl117z13ylZQoP/OfFPTX5iqvn17a926DZKkPr3PU6/TUzThkSlBrhoAgOaNy0kN4HSWq7i4WAUFhfr++x+1OW2LunVNUkFBod6ZN1/jbhmjyMhItW7dWrfeOkbvvDtfBQWFwS4bAIBmjTMxjZSQEK8ePbrrvfcWSJJSU79Uv/P76uYxo1VVVaXs/dlK/eLLIFcJAEDzR4hpgJEjhmv4sKGyWCwKDw/XN2u/1arVa7zzM16fpeef+7skjyZNfix4hQIA0IIQYhrg89SV+mTFZ7IYFnXocIpuHnujxtx0g+a+854kKTc3Txs3bar+c15eMEsFAKDF4J6YukTGS6ecLYWEq7TUrtzcPGXn5Oinnzdr4aKlunzopQoLC/Mud7nccrncQSwYAICWhTMxtTCShsiI7yPDEiLDmiAjptRn3uV2KSQkRKGhoaqsrAxSlQAAtGyEmN+KTPAGGEmSYSgiNlEx8afJcB5Qxw4ddPXIq7QtPUNlZWXBrRUAgBaMEPNb1vgjAeaQYee207Bzn5Db7VZx8UGlbdmid96dH6QCAQCARIipyZEtj9vlDTLPLNonj9sld9osyZFT52avvvbfpqoQAACIG3trsufIk7NeHrdLkuRxu+TJXnfMAAMAAJoeZ2Jq4clKlceWJlkTJEe2ZCfAAABwsiHE1MWeQ3gBAOAkxuUkAABgSoQYAABgSoQYAABgSoQYAABgSoQYAABgSoQYAABgSoQYtFhPPD5Fo64Zeczx+fPmeF9vvzVTL/zj7zq/b59j7vf/Xpqmd95+Q6ec0v4349N1waABkqRT2rfX/HlzdEr79rXtAgDQAHxPDFCP6S++pIztOxQeHqY+vc/VfffepT1ZWdq/P7vObdxut0ZdPUKvzpjZhJUCQMvCmRigHqWlpSouLtaBA/la/tEK2R0OJSV2OeY2qV98qQED+ik+vmMTVQkALQ8hBmiEc875o6ytWytzT9Yx12Vm7tF33/2gUdeMaKLKAKDl4XISWrSRI4Zr+LChPmPh4eHaujXd+37KXyfK7XYrNDRUhmFo9ltzlZOTW+++FyxcrOemPq0lS5dp3779fq8dAFo6QgxatM9TV+qTFZ/5jN13790+72e8Pks7du5SaEiofve7bhp361gVFBZq06bvj7nvvb/u04YNm3TtqJH6179f8XvtANDSEWLQ8kTGVz+hPCRcpaV25ebm+UxXVFT4vC8oKPSu2bd/v3r06K7hV1xeb4iRpAWLluiFf/xdSUmJ/qsfACCJEIMWxkgaIiO+jwxLiAxrgoyY0kbvw+12q1Wr8Aat3b8/W2u/Xa9rR9X8KDcA4MRwYy9ajsgEb4CRJBmGFNVZssYfc7OoqCjFxMQorl07nXfu/2jggH76/oefGnzYhYuW6Jyz/6i2MTEnUj0A4Dc4E4OWwxp/JMAcYhgWKTJBcuTUudmEh++XJFVVVclmK9Cnn6Vq0eKl3vknHp+iAwfy9eprtX8nTG5unlZ//Y0GX3SBH34IAMBhhBi0HI5sedwub5B5ZtE+edwuye77pXVPPT3V++frR99c725Pad9eq1Z97X1/3/0TJXl81sx4fZZmvD7L+/5Afn6D9g0AqBuXk9By2HPkyVlfHVwkedwuebLXHfMsTH3i4zvK6XRq9dff+KtKAEADcSYGLYonK1UeW1r1p5Mc2ZL9+AOMJOXk5OqRyX879M448QIBAA1GiEHLY8854fACAAg+LicBAABT8vuZGIvForFjRmvQwP4yDEPr12/UrDfnqKqqyvfAoaG6bdzNOqPX6YqJaaPCwiJ99vkX+viTT/1dEgAAaIb8HmJGjhiu03umaOKkR1VV5dKkiQ9p9A3X6e257/qsCwmxqKioWH+f+rzy8g4oKSlRj055RIWFRfp23Xp/lwUAAJoZv19OGnzRhVq85EMVFhappKRECxct0YUXDJRh+N70WF5eofcXLFJubp48Ho/27MnSDz/8pOTk7vUcweB1XC96SI+bw4se0+Pm8KLHJ9azI/x6JsZqtap9+zhlZu7xjv2yO1NRUZGKi2un/HxbndtaLBYlJ/fQsmUfHfMYUW3aqqqy0m81tzTRMbHBLqHZo8eBR48Djx4HHj1umNCwsLrn/Hmg1hERkiRHmcM75nA4fObqMu7WsSpzOLRq9Zpjris9WKTK3zygDw0THROrkuLCYJfRrNHjwKPHgUePA48eN1xYeN3PqvNriClzOiVJ1tZWlZRUP1jParX6zNVm7JjRSk7urqefeU4ul6ueo3j0229DRUMcfTqO/gUGPQ48ehx49Djw6HHj1N0jv94T43A4lJ9vU9duSd6xU7t1VWmpXTZbQa3b3HLzTTrrzDP09DP/8AYfAACA+vj9xt6VX36lq0dcqdjYtoqOjtaoUSP11aqv5fHUTFK33jJGZ57RS08985xKSkr8XQoAAGjG/P4R6yVLlyk6OlrTnp8qi8XQunUb9N78BZKk28ffKkmaOWu22reP09DL/qSKigq9/NJ07/bb0jP03D+m17JnAACAI/weYtxut2a/NVez35pbY27mrNneP+fn23iKLwAAOG48dgAAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJgSIQYAAJhSaLALQPP0xONT1Ov0npIkl8slW0GBVq/+RgsWLq4xf7Rnn3tBP/20uUlrBQCYEyEGAbNs+cdatvwThYRY1K1bV917z52yFRRo5cqvfOaPVlpaGoRKAQBmRIhBwDid5SouLpYkFRQUanPaFnXrmlTrPAAAjcU9MWgSCQnx6tGju3bt+iXYpQAAmgnOxCBgRo4YruHDhspisSg8PFzfrP1Wq1avqTF/tL/c9zCXlAAADUKIQcB8nrpSn6z4TBbDog4dTtHNY2/UmJtu0Nx33vOZP5rdbg9GqQAAEyLEwL8i4yVrghQSrtJSu3Jz8yRJ2Tk5Wrhoqe679y7Nf3+RJPnMAwDQWIQY+I2RNERGfB8ZlhAZ1gQZMb6XhVxul0JCQhQayl87AMCJ418T+EdkgjfASJIMQxGxiYqJP02G84A6duigq0depW3pGSorK5MkRUS0UkxMjM9uHA6HKisrm7p6AIAJEWLgH9b4IwHmkGHnttOwc5+Q2+1WcfFBpW3Zonfene+dHz7scg0fdrnPNjNen6WVX65qkpIBAOZGiIF/OLLlcbu8QeaZRfvkcbvkTpslOXJqLH/q6alNXSEAoJnhe2LgH/YceXLWy+N2SZI8bpc82etqDTAAAPgDZ2LgN56sVHlsadWfTnJkS3YCDAAgcAgx8C97DuEFANAkuJwEAABMiRADAABMiRADAABMiRADAABMiRADAABMiRADAABMiRADAABMiRADAABMiRADAABMiRADAABMiRADAABMiRADAABMiQdAAgAQBPPnvXXM+Xvve1ijRo2UJL362n995u6+68+1jrc0hBgAAILgjrvul+SRJA0fNlQ9enTX9Bdf8s4fPHgwSJWZByEGAIAgKC4u1uEQ43SWq6qq6tAYGop7YgAAgClxJgYAgJPYgP7nq2+f83zGQkNDteabb4NU0cmDEAMAwEnsu+9+0Dvz5vuM3TT6+iBVc3IhxAAA0MRcrU6RTukmOfZL9pxjri1zOpWbm1djDIQYAACalJF0scoT+ijECJHH7ZInZ32wSzItv4cYi8WisWNGa9DA/jIMQ+vXb9SsN+eoqqqqxto/XXKxLhg0QElJidqxc5eeenqqv8sBAODkEZkgI76PZIRIkgxLiBTfRwrLCnJh5uT3TyeNHDFcp/dM0cRJj+qBhyapS5cuGn3DdbWuLSoq1gcffqSPP/nU32UAAHDyscZXB5ejGJYQKTwqSAWZm9/PxAy+6ELNeftdFRYWSZIWLlqi+++7R3PfmSePx+OzdsPGTZKk9u3jGnEE49ALx4/+BR49Djx6HHj02O8cOfK4XT5BxuN2aeHCpZIjR7/t+auvzTz0p4aNN091/4x+DTFWq1Xt28cpM3OPd+yX3ZmKiopUXFw75efbTvgYUW3aqqqy8oT301JFx8QGu4Rmjx4HHj0OPHocKOUqL/hJrrg/VF9S8rgUWvCTWoWVS/S8VqFhYXXP+fNArSMiJEmOMod3zOFw+MydqNKDRaqsqPDLvlqa6JhYlRQXBruMZo0eBx49Djx6HGDFH8l6cIccnkjJkS2XPUf8q1a3sPDwOuf8GmIOf+TL2tqqkpLS6j9brT5zJ86jw1/TjMY4+nQc/QsMehx49Djw6HHgGQopPyAVbxc9boi6e+TXG3sdDofy823q2i3JO3Zqt64qLbXLZivw56EAAEAL5/dPJ6388itdPeJKxca2VXR0tEaNGqmvVn1d46Zeqfrj2GFhYbJYLLIYhsLCwhQSElLLXgEAAHz5/dNJS5YuU3R0tKY9P1UWi6F16zbovfkLJEm3j79VkjRz1mxJ0tUjr9K1o0Z6t507Z5a2bN3G98UAAIB6+T3EuN1uzX5rrma/NbfG3OHwctjCRUu0cNESf5cAAABaAL9fTgIAAGgKhBgAAGBKhBgAAGBKhBgAAGBKhBgAAGBKhBgAAGBKfv+INQAcFhvbVjeOvl5nntFLkZFWFRYWafuOHXr1tZka0P98jRo1UvfdP8FnmwsGDah1HAB+ixADIGAefvB+Ocudmjb9Xyo+eFDxHTuqT5/zZLFwEhjAiSPEAAiIyEirevQ4TY9M/puysvZKkg4cyNfmtC1BrgxAc8GvQwACwuksl9NZrvPOPUeGYdS/AQA0EmdiAASEy+XSf2e9qT+Pv1XDrrhcO3bs1M+b0/TVqq9VWloqSWofF6e33nzdZzuLxaKi4uIgVAzAbAgxAAJmzZq1+u6773VGr15KSemhy4deqmFXDNWUv/2vJKmwsEhPPv2szzZ9ep+rSy65OBjlAjAZQgwA/4qMl6wJkiNbsueorMypjZu+08ZN32nhoqX69z9f0JCLL1JeXp5cbpdyc/N8Ni8uPhikwgGYDSEGgN8YSUNkxPeRYQmRx+2SJ2e9PFmp3vmysjIVFRUpIqJVEKsE0FxwYy8A/4hM8AYYSYqJCteUu69R30GXqFOnBCUkxOu6a69RYmIX/fDDT0EuFkBzwJkYAP5hjfcGGEkqK3drr61CV195mTq0GyW326392Tn690uvKG3LVl0waEAQiwXQHBBiAPiHI1set8sbZCpdHr2zOk/utFmSI6fG8lWr12jV6jUNHgeA3+JyEgD/sOdU3wPjdklS9T0x2etqDTAA4A+ciQHgN56sVHlsaT6fTgKAQCHEAPAvew7hBUCT4HISAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwpVB/79BisWjsmNEaNLC/DMPQ+vUbNevNOaqqqjqhtQAAAEfz+5mYkSOG6/SeKZo46VE98NAkdenSRaNvuO6E1wIAABzN7yFm8EUXavGSD1VYWKSSkhItXLREF14wUIZhnNDaIwxex/Wih/S4ObzoMT1uDi96fGI9O8Kvl5OsVqvat49TZuYe79gvuzMVFRWpuLh2ys+3Hdfao0W1aauqykp/lt2iRMfEBruEZo8eBx49Djx6HHj0uGFCw8LqnvPngVpHREiSHGUO75jD4fCZO561Rys9WKTKigr/FNzCRMfEqqS4MNhlNGv0OPDoceDR48Cjxw0XFh5e55xfQ0yZ0ylJsra2qqSktPrPVqvP3PGs9eU59ELjHH06jv4FBj0OPHocePQ48Ohx49TdI7/eE+NwOJSfb1PXbknesVO7dVVpqV02W8FxrwUAAPgtv9/Yu/LLr3T1iCsVG9tW0dHRGjVqpL5a9bU8nppJqjFrAQAAjub374lZsnSZoqOjNe35qbJYDK1bt0HvzV8gSbp9/K2SpJmzZte7FgAA4Fj8HmLcbrdmvzVXs9+aW2PucHhpyFoAAIBj4bEDAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlAgxAADAlEKDXQAAADC32Ni2unH09TrzjF6KjLSqsLBI23fs0KuvzdSA/ufrnrvv0MqVX2nGf9/wbnPBoAEaNWqk7rt/wnEflxADAABOyMMP3i9nuVPTpv9LxQcPKr5jR/Xpc54sluoLPhUVFRo0aICWfrBcuXl5fjsul5MAAMBxi4y0qkeP0/T23HnauesXHTiQr81pWzRz1mxVVlZKkoqKi/XDDz9p1DUj/HpsQgwAADhuTme5nM5ynXfuOTIMo851CxYuVr9+fZWQEO+3YxNiAADAcXO5XPrvrDd15fAr9MbM1/ToXx/RsCuGKioqymfdnqy92rTpe107aqTfjk2IAQAAJ2TNmrW665779Z9XX9feX3/V5UMv1bTnn1VsbKzPuvcXLlaf3ucpsUtnvxyXEAMAABovMl465ezq/5VUVubUxk3f6e258zThkSmyWCwacvFFPpvs27df69Zt0Cg/nY3h00kAAKBRjKQhMuL7yLCEyON2yZOzXp6sVO98WVmZioqKFBHRqsa2Cxct1bQXnlVBQeEJ18GZGAAA0HCRCd4AI0kxUeGacvc16jvoEnXqlKCEhHhdd+01Skzsoh9++KnG5tk5OVrzzbc1ztIcD87EAACAhrPGewOMJJWVu7XXVqGrr7xMHdqNktvt1v7sHP37pVeUtmWrLhg0oMYuFi1eqgH9zz/hUggxAACg4RzZ8rhd3iBT6fLondV5cqfNkhw5NZavWr1Gq1av8RnLyzugm8bedsKlcDkJAAA0nD2n+h4Yt0uSqu+JyV5Xa4AJNM7EAACARvFkpcpjS5OsCZIjW7I3fYCRCDEAAOB42HOCFl4O43ISAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwJUIMAAAwpVB/7qx799M0/rZb1LlTgnLz8jR79lylbdla69q2bWN0+/hx+t2p3RQX105PPvWstm5L92c5AACgGfPbmRir1arJjzys1NSVGjf+Li39YLkmTnhAbdvG1Lre4/Ho558365//flkOh8NfZQAAgBbCb2dievc+V0VFRUr94ktJ0po1a3XpJUPU7/y++viTT2usLy4+qM8+/0JSdaBpOOPQC8eP/gUePQ48ehx49Djw6HH96u6R30JM16RE7c7c4zO2OzNTXZMS/XUISVJUm7aqqqz06z5bkuiY2GCX0OzR48Cjx4FHjwOPHjdMaFhY3XMN2kFoqEJCQuqcr6ioUERERI3LQna7QzExbRpYZsOUHixSZUWFX/fZUkTHxKqkuDDYZTRr9Djw6HHg0ePAo8cNFxYeXudcg0LMXXeO18AB/eucf/KpZ+V0OhUVFeUzHhlplbPM2cAyG8pz6IXGOfp0HP0LDHocePQ48Ohx4NHjxqm7Rw0KMS+/MkMvvzLjmGs6dOygK4Ze6jPWrVtXrV+/sSGHAAAAaBS/3ROzceMmjbnxBg0efKFWrfpaffqcp6TELvrnv/6vzm3CjrrOFRoaqrCwMFVVVTXyRt/AeeLxKep1ek9JktNZrpycHC1Z+qHWHQpmd9/1Z114wUBJ1ZfUcnJy9cGyj7Rmzdqg1QwAQEvhtxBjtzv0/Asvavxtt2jcLWOUl3dA0198SYWFRZKkuLg4vThtqh6eOEU2m02SNHfOLO/2f3t0kiSddN8Xs2z5x1q2/BOFh4fp/L69df999yg7+wntydorSVr77XrNfmuuWrUK16CB/fWXu+9Q/oF8pWdsD3LlAHBiYmPb6sbR1+vMM3opMtKqwsIibd+xQ4Zh0YD+59e53YKFS7Rw0ZImrBQtlV+/7G77jp2aPOXxWudsNptuGXeHz9j1o2/25+EDwuksV3FxsSTpw2Uf68rhw3T66T29IaaiosI7v3DRUvXv309nn/1HQgwA03v4wfvlLHdq2vR/qfjgQcV37Kg+fc7TO+/O15y335UktY9rp2f//qQe/dv/Kt9WIElyOv19LyRQO7+GmObMMAydd+7/KDLSKpfLVee6yooKhYbSVgDmFhlpVY8ep+mRyX9T1qFf2g4cyNfmtC2SpLKyMklS+KHbAg4eLPH+Qgc0Ff61rcfIEcM1fNhQhYaGKjQ0VDZbgb5dt6HGOsMw1L9fX3XtmqT3FyzRtJxbAAAVB0lEQVQOQqUA4D9OZ7mcznKdd+452rv315PmXkXgaISYenyeulKfrPhMsW3b6uaxN2rh4qUqKSnxzg/of7769jlPoaGhqqpyaeGipdr03fdBrBgATpzL5dJ/Z72pP4+/VcOuuFw7duzUz5vT9NWqr1VaWhrs8gBJhJiaIuMla4LkyJYklZbalZubp9zcPL38nxl68v89pomPPOo9bfrddz/onXnzVVFRqaKiIn5bAdBsrFmzVt99973O6NVLKSk9dPnQSzXsiqGa8rf/VWEhX9SG4PPbAyCbAyNpiCy9xivk98Nl6TVeRivfr4Tevz9b6dsydPXIK71jZU6ncnPzVFhYSIAB0Cy4Wp0inXK2FBmvsjKnNm76Tm/PnacJj0yRxWLRkIsvCnaJgCRCzBGRCTLi+8iwVD9ewbCESOFtpDDfbyFe8ennGnzRBYqJqf3p3ABgZkbSxSo/dZRCfj+s+pe5pCHeubKyMhUVFSkiolUQKwSO4HLSYdZ4b4DxMgwp3DfEbNm6Tdk5ORp2xWVNWBwANIFDv8zJCFEba4ju+VMnrdwcpT3f5spTdkADB/RXYmIX78ergWAjxBzmyJbH7fIJMk8vyJI7reYXNk2a/FhTVgYATeOoX+bKyt3Kyi/X1X3b65QhD8jtqtT+7Bz9+6VXlLZla5ALBaoRYg6z58iTs146dEnJ43bJk71OcuQEuzIAaBpH/TJX6fLo3TU2vbM6T+60WXX+t/BAfr4pvrgUzRMh5iierFR5bGlHPp1kJ8AAaEEO/TJnJFRfUuKXOZzsCDG/Zc8hvABosTxZX6iVc68cnkjJsZ//HuKkRogBAPgIKT8gFW+XxNdG4OTGR6wBAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIApEWIAAIAphfpzZ927n6bxt92izp0SlJuXp9mz5ypty9Za15599h905fArlJSYKLfHrR3bd2rO3HeVk5Prz5IAAEAz5bczMVarVZMfeVipqSs1bvxdWvrBck2c8IDato2pfX1rq5Yt/1h/ue8h3fOXB5Wbl6dJEx/yVzkAAKCZ89uZmN69z1VRUZFSv/hSkrRmzVpdeskQ9Tu/rz7+5NMa679Z+63P++UfrdDlQy9VZGSk7Hb7MY5kHHrh+NG/wKPHgUePA48eBx49rl/dPfJbiOmalKjdmXt8xnZnZqprUmKDtu/Vq6dstoJ6AowU1aatqiorj7vOli46JjbYJTR79Djw6HHg0ePAo8cNExoWVvdcg3YQGqqQkJA65ysqKhQRESGHw+Ezbrc7FBPTpt79d+qUoLFjRuu/M9+sd23pwSJVVlTUXzRqiI6JVUlxYbDLaNboceDR48Cjx4FHjxsuLDy8zrkGhZi77hyvgQP61zn/5FPPyul0Kioqymc8MtIqZ5nzmPtOSIjXY49O1vz3F2rDhk0NqMZz6IXGOfp0HP0LDHocePQ48Ohx4NHjxqm7Rw0KMS+/MkMvvzLjmGs6dOygK4Ze6jPWrVtXrV+/sc5tOnfqpMcem6zFiz9QauqXDSkFAABAkh8/nbRx4ybFxsZq8OALFRISon79+iopsYvWfruu1vWdO3fS44/9VUuWfKjPU1f6qwwAANBC+C3E2O0OPf/Ci7r0kos1+40ZumbkVZr+4ksqLCySJMXFxemtN19XXFycJOnKYZcrJqaNbrrxer315uveV0pyD3+VBAAAmjG/ftnd9h07NXnK47XO2Ww23TLuDu/7V2fM1KszZvrz8AAAoAXhsQMN9MTjUzTqmpHHHJ8/b45O75lSY01d4wAA4PgRYgAAgCkRYgAAgCkRYgAAgCn59cbe5m7kiOEaPmyoz1h4eLi2bk33vp/y14lyu91NXRoAAC0OIaYRPk9dqU9WfOYzdt+9d/u8n/H6LO3Yuctn7KV/TQt4bQAAtDSEmPpExkvWBCkkXKWlduXm5vlMV/zmOU4FBYU11gAAAP8jxByDkTRERnwfGZYQGdYEGTGlwS4JAAAcwo29dYlM8AYYSZJhSFGdJWt8cOsCAACSCDF1s8YfCTCHGIZFikwIUkEAAOBoXE6qiyNbHrfLG2SeWbRPHrdLsmf7LHvq6aneP18/+uZad1XXOAAAOH6ciamLPUeenPXVwUWSx+2SJ3ud5MgJcmEAAEDiTMwxebJS5bGlVX86yZEt2QkwAACcLAgx9bHnEF4AADgJcTkJAACYEiEGAACYEpeTmoEnHp+irVvTtXDRkjrH5897q9Ztx9w8XpWVlU1RJgAAfkWIaUH+89ob+vHHHyV5vGMEGACAWRFiWhC7w6Hi4mIdHWIAADAr7okBAACmxJmYZmLkiOEaPmyoz1h4eLi2bk33vn/g3jvkdt/uff/rr/v0t8efbLIaAQDwJ0JMM/F56kp9suIzn7H77r3b5/1bc+dr888/6/DlpKrKqqYqDwAAvyPEmFlkfPW3CYeEq7TUrtzcPJ/piooKn/dFRcWH1nBPDADA/AgxJmUkDZER30eGJUSGNUFGTGmwSwIAoEkRYswoMsEbYCRJhiFFdZas8cd8QGWk1aqYmBgdfSbm4MGD8ng4MwMAMB9CjBlZ448EmEMMwyJFJhwzxNxz1201xu5/cGKNy1AAAJgBIcaMHNnyuF3eIPPMon3yuF2SPdtn2VNPT/X++frRtyg6JlYlxYXinhgAQHPA98SYkT1Hnpz11cFFksftkid73THPwgAA0NxwJsakPFmp8tjSqj+d5MiW7AQYAEDLQogxM3sO4QUA0GIRYlCrJx6fol6n95QkuVwu2QoKtHr1N1qwcLGuuPwyXfqnIbr/wYne9b17n6sJD92vV/4zQ6u//sY7/uK057Tmm7VavOTDJv8ZAADNGyEGdVq2/GMtW/6JQkIs6tatq+69507ZCgq0LT1DN4+9Ue3axaqgoFCS1DMlWfk2m3qmJHtDTHR0lBIS4rVtW0YwfwwAQDPFjb2ok9NZruLiYhUUFOr773/U5rQt6tY1Sbt3Z8rhKFNKSrJ3bUpyD3300QqfseTkHqqqqtLOXb8Eo3wAQDNHiEGDJCTEq0eP7tq16xd5PB7t2LFTPQ8FloiICHXu3ElfrPxKbdvGKCamjaTqYLNr1y+qrKwMZukAgGaKy0mo0+EnY1ssFoWHh+ubtd9q1eo1kqRt6enq3+98SVJyj+7as2evysvLtWPnLqUkJ2v9ho1KSUnW5s1pwfwRAADNGGdiUKfPU1dq0l8f06TJj+nZqS+oa1KSxtx0gyQpPX27OnfupKioKKWk9FB6xnZJUkbGdvXsmazw8HCd2q0r98MAAAKGEIM6HX4ydnZOjn76ebMWLlqqy4deqrCwMO3YuUtVVVVKSe6hlJRkpWdUh5X0jO1KSUlW9+6/lyRlbN8RzB8BANCMcTkJR0TGH/nyvFq43C6FhIQoNDRUZWVl2rXrF515Zi/9/nenKiOjOqzs2LFLnTsl6Jyzz1Zm5h6Vl5c35U8AAGhBCDGQJBlJQ7xPxva4XTJaxSgiopViYmJkGFLHDh109cirtC09Q2VlZZKkbenbNfSyS3TgQL5KS0slSRUVFcrK2quLB1+g1C++DOaPBABo5ggxkCITvAFGUvX/hrfR8GGXa/iwy+V2u1VcfFBpW7bonXfnezdLT8/Q1SOv9N4P4x3P2KHTTvu9tqVzPwwAIHAIMZCs8d4Ac9gzi/fLtWuZdOCHOjf76efNun70zTXG3577rt6e+67fywQA4Gjc2AvJke19IvZhHrdLstd+bwwAACcDQgwke448Oeu9QcbjdsmTvU5y8HBJAMDJi8tJkCR5slLlsaUd+XQST8cGAJzkCDE4wp5DeAEAmAaXkwAAgCkRYgAAgCkRYgAAgCkRYgAAgCkRYgAAgCn59dNJ3bufpvG33aLOnRKUm5en2bPnKm3L1lrX/v53p+r28beqQ4cOMgzp1337NW/e+3xVPQAAaBC/hRir1arJjzys9+Yv0Fervlbfvr01ccIDevDhSSoqKq6xPu/AAU3/50vKz7fJMAydd+7/aPKkh3XHXfepoqLCX2UBAIBmym8hpnfvc1VUVOR9cvGaNWt16SVD1O/8vvr4k09rrC8pKVVJSan3vdvjVuvWrRUT00YHDuQf40jGoReOH/0LPHocePQ48Ohx4NHj+tXdI7+FmK5JidqducdnbHdmpromJR5zuzdmvqqIiAiFhIRo1eo19QQYKapNW1VVVp5wvS1VdExssEto9uhx4NHjwKPHgUePGyY0LKzuuQbtIDRUISEhdc5XVFQoIiJCDofDZ9xudygmps0x933b7XcrLCxMffqcp1bh4fXWUnqwSJVcbjou0TGxKikuDHYZzRo9Djx6HHj0OPDoccOFHSMbNCjE3HXneA0c0L/O+SefelZOp1NRUVE+45GRVjnLnPXuv7KyUmvWrNW0F57Vr/v2KyNj+zFWew690DhHn46jf4FBjwOPHgcePQ48etw4dfeoQSHm5Vdm6OVXZhxzTYeOHXTF0Et9xrp166r16zc25BCSpBBLiBLiO9YTYgAAAPz4PTEbN25SbGysBg++UCEhIerXr6+SErto7bfral1/ztl/VFJSogzDUKtW4Rpx1XDFxbXTtm18xBoAANTPbzf22u0OPf/Cixp/2y0ad8sY5eUd0PQXX1JhYZEkKS4uTi9Om6qHJ06RzWZTdHSUxo69Ue1i26qislJ7s/bqH8+/qNy8PH+VBAAAmjG/ftnd9h07NXnK47XO2Ww23TLuDu/7VavXaNXqNf48PAAAaEF47AAAADAlv56JaQphYfV/DBu1MRQaFnboo2rcDR8Y9Djw6HHg0ePAo8eNcax/900TYg7/ENeNvTvIlQAAgKYWFhZe43vijHP6XWaaGGiNjFJlJV90BwBASxIWFi6HvbTGuGnOxEiq9QcAAADNW13f1M+NvQAAwJQIMQAAwJQIMQAAwJQIMQAAwJQIMQAAwJRM9ekkNF737qdp/G23qHOnBOXm5Wn27LlK27K11rVnn/0HXTn8CiUlJsrtcWvH9p2aM/dd5eTkNnHVJy+LxaKxY0Zr0MD+MgxD69dv1Kw356iqquqE1uKIhvYtNDRUt427WWf0Ol0xMW1UWFikzz7/Qh9/8mmQKjeP4/m7GRYWpmnPP6vo6Cjddjvf11Wfxvb4nHP+qOtGXa2EhHiVlTn10ccrtGz5x01ctflwJqYZs1qtmvzIw0pNXalx4+/S0g+Wa+KEB9S2bUzt61tbtWz5x/rLfQ/pnr88qNy8PE2a+FATV31yGzliuE7vmaKJkx7VAw9NUpcuXTT6hutOeC2OaGjfQkIsKioq1t+nPq9bb7tT//z3y7rqqmE6v2+fIFRtLsfzd/O6a6+RraCgiSo0v8b0+A9/OFN/Hn+r5r7znsaNv0sPTZikH3/8qWkLNilCTDPWu/e5KioqUuoXX6qqqkpr1qzV3r371O/8vrWu/2btt/r++x/ldDpVWVmp5R+tUOfOnRQZGdnElZ+8Bl90oRYv+VCFhUUqKSnRwkVLdOEFA2UYxgmtxREN7Vt5eYXeX7BIubl58ng82rMnSz/88JOSk7sHqXLzaOzfzVO7ddXZZ5+lDz5Y3sSVmldjenz9tddo8ZIPlbZlq9xut8rKnNr7674gVG0+hJhmrGtSonZn7vEZ252Zqa5JiQ3avlevnrLZCmS32wNRnulYrVa1bx+nzKN6+svuTEVFRSourt1xr8URJ9I3i8Wi5OQeysraG+gyTa2xPbZYLLrjz7fpjTfmqLKysilLNa3G9LhVq3Cdemo3tWoVrukvTNWM1/5PEx6+X3FxcU1btEkRYkwqNDRUrVq1qvNlGIYiIiLkcDh8trPbHYpoHVHv/jt1StDYMaM1e87cQP0IptM6orpvjrIjPT3c38Nzx7MWR5xI38bdOlZlDodWrV4TuAKbgcb2eNgVQ5WVtVdbt6U3TYHNQGN6HBkZKYvFokGDBui5f0zXffdP0MHig5rw0H1NV7CJcWOvSd1153gNHNC/zvknn3pWTqdTUVFRPuORkVY5y5zH3HdCQrwee3Sy5r+/UBs2bPJLvc1BmbO6b9bWVpWUVD8Cw2q1+swdz1occbx9GztmtJKTu+vpZ56Ty+UKfKEm1pged+zQQZf+aYgmT3m8aYs0uUb9t+LQf49XrPhcB/LzJUnvvb9QM1//j2Jj26qwsKipyjYlQoxJvfzKDL38yoxjrunQsYOuGHqpz1i3bl21fv3GOrfp3KmTHntsshYv/kCpqV/6pdbmwuFwKD/fpq7dkpSblyep+l6B0lK7bLaC416LI46nb7fcfJPO6HW6nnrmOe8/GKhbY3qcktJDbdpEa/oLz0qqPgPcunVrzXj1JU1/8SVt37Gzyes3g8b0uKysTHkHDsjjOfIsZo9pHsscfCEJiaf9v2AXgcA4cOCARo64SqV2u7Ky9ur88/to8EUXaOas2XLW8ltt586d9Phjf9WSJR/q89SVQaj45Ne6dYSGDL5I333/g8LCwvTn28fp++9/1E8/bz6htTiiMX279ZYxOvOMXocCTEkQqjWnhvZ4//5sfZ76pfeVmZmls/94liY88qj3hmrUrjF/j8PDwjV48AX6/vsfVVVVpZtuvF6S9NHHK5q6bNMxzul3GX8Lm7Eeh74nplOnBOXlHdDst+Zqc9oWSVJcXJxenDZVD0+cIpvNprvvvF2DBg1QxW+eFjr1uWlKz9gejPJPOhaLRTePvVEDB/SXxWJo3boNemP226qsrNTt42+VJM2cNbvetahbQ3vcvn2cXvm/f6qiokJut9u7/bb0DD33j+nBKd4kGvP3+Gin90zRxAkP8D0xDdCYHhuGodE3XKuLLhwkw7AoPWO73pz9tmw2W9DqNwtCDAAAMCU+nQQAAEyJEAMAAEyJEAMAAEyJEAMAAEyJEAMAAEyJEAMAAEyJEAMAAEyJEAMAAEzp/wP7McWuMWd/1gAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_embedding(model, encoders, 'State')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" } }, "nbformat": 4, "nbformat_minor": 2 }