{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook contains an implementation of the third place result in the Rossman Kaggle competition as detailed in Guo/Berkhahn's [Entity Embeddings of Categorical Variables](https://arxiv.org/abs/1604.06737)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The motivation behind exploring this architecture is it's relevance to real-world application. Much of our focus has been computer-vision and NLP tasks, which largely deals with unstructured data.\n", "\n", "However, most of the data informing KPI's in industry are structured, time-series data. Here we explore the end-to-end process of using neural networks with practical structured data problems." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Using TensorFlow backend.\n" ] } ], "source": [ "import math, keras, datetime, pandas as pd, numpy as np, keras.backend as K\n", "import matplotlib.pyplot as plt, xgboost, operator, random, pickle" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from utils2 import *" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "np.set_printoptions(threshold=50, edgeitems=20)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "limit_mem()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from isoweek import Week\n", "from pandas_summary import DataFrameSummary" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/data/datasets/rossman\n" ] } ], "source": [ "%cd /data/datasets/rossman/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create datasets" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In addition to the provided data, we will be using external datasets put together by participants in the Kaggle competition. You can download all of them [here](http://files.fast.ai/part2/lesson14/rossmann.tgz).\n", "\n", "For completeness, the implementation used to put them together is included below." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def concat_csvs(dirname):\n", " os.chdir(dirname)\n", " filenames=glob.glob(\"*.csv\")\n", "\n", " wrote_header = False\n", " with open(\"../\"+dirname+\".csv\",\"w\") as outputfile:\n", " for filename in filenames:\n", " name = filename.split(\".\")[0]\n", " with open(filename) as f:\n", " line = f.readline()\n", " if not wrote_header:\n", " wrote_header = True\n", " outputfile.write(\"file,\"+line)\n", " for line in f:\n", " outputfile.write(name + \",\" + line)\n", " outputfile.write(\"\\n\")\n", "\n", " os.chdir(\"..\")" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# concat_csvs('googletrend')\n", "# concat_csvs('weather')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Feature Space:\n", "* train: Training set provided by competition\n", "* store: List of stores\n", "* store_states: mapping of store to the German state they are in\n", "* List of German state names\n", "* googletrend: trend of certain google keywords over time, found by users to correlate well w/ given data\n", "* weather: weather\n", "* test: testing set" ] }, { "cell_type": "code", "execution_count": 42, "metadata": { "collapsed": true }, "outputs": [], "source": [ "table_names = ['train', 'store', 'store_states', 'state_names', \n", " 'googletrend', 'weather', 'test']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll be using the popular data manipulation framework pandas.\n", "\n", "Among other things, pandas allows you to manipulate tables/data frames in python as one would in a database." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We're going to go ahead and load all of our csv's as dataframes into a list `tables`." ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "collapsed": true, "scrolled": true }, "outputs": [], "source": [ "tables = [pd.read_csv(fname+'.csv', low_memory=False) for fname in table_names]" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from IPython.display import HTML" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can use `head()` to get a quick look at the contents of each table:\n", "* train: Contains store information on a daily basis, tracks things like sales, customers, whether that day was a holdiay, etc.\n", "* store: general info about the store including competition, etc.\n", "* store_states: maps store to state it is in\n", "* state_names: Maps state abbreviations to names\n", "* googletrend: trend data for particular week/state\n", "* weather: weather conditions for each state\n", "* test: Same as training table, w/o sales and customers\n" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreDayOfWeekDateSalesCustomersOpenPromoStateHolidaySchoolHoliday
0152015-07-3152635551101
1252015-07-3160646251101
2352015-07-3183148211101
3452015-07-311399514981101
4552015-07-3148225591101
\n", "
" ], "text/plain": [ " Store DayOfWeek Date Sales Customers Open Promo StateHoliday \\\n", "0 1 5 2015-07-31 5263 555 1 1 0 \n", "1 2 5 2015-07-31 6064 625 1 1 0 \n", "2 3 5 2015-07-31 8314 821 1 1 0 \n", "3 4 5 2015-07-31 13995 1498 1 1 0 \n", "4 5 5 2015-07-31 4822 559 1 1 0 \n", "\n", " SchoolHoliday \n", "0 1 \n", "1 1 \n", "2 1 \n", "3 1 \n", "4 1 " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreStoreTypeAssortmentCompetitionDistanceCompetitionOpenSinceMonthCompetitionOpenSinceYearPromo2Promo2SinceWeekPromo2SinceYearPromoInterval
01ca1270.09.02008.00NaNNaNNaN
12aa570.011.02007.0113.02010.0Jan,Apr,Jul,Oct
23aa14130.012.02006.0114.02011.0Jan,Apr,Jul,Oct
34cc620.09.02009.00NaNNaNNaN
45aa29910.04.02015.00NaNNaNNaN
\n", "
" ], "text/plain": [ " Store StoreType Assortment CompetitionDistance CompetitionOpenSinceMonth \\\n", "0 1 c a 1270.0 9.0 \n", "1 2 a a 570.0 11.0 \n", "2 3 a a 14130.0 12.0 \n", "3 4 c c 620.0 9.0 \n", "4 5 a a 29910.0 4.0 \n", "\n", " CompetitionOpenSinceYear Promo2 Promo2SinceWeek Promo2SinceYear \\\n", "0 2008.0 0 NaN NaN \n", "1 2007.0 1 13.0 2010.0 \n", "2 2006.0 1 14.0 2011.0 \n", "3 2009.0 0 NaN NaN \n", "4 2015.0 0 NaN NaN \n", "\n", " PromoInterval \n", "0 NaN \n", "1 Jan,Apr,Jul,Oct \n", "2 Jan,Apr,Jul,Oct \n", "3 NaN \n", "4 NaN " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreState
01HE
12TH
23NW
34BE
45SN
\n", "
" ], "text/plain": [ " Store State\n", "0 1 HE\n", "1 2 TH\n", "2 3 NW\n", "3 4 BE\n", "4 5 SN" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StateNameState
0BadenWuerttembergBW
1BayernBY
2BerlinBE
3BrandenburgBB
4BremenHB
\n", "
" ], "text/plain": [ " StateName State\n", "0 BadenWuerttemberg BW\n", "1 Bayern BY\n", "2 Berlin BE\n", "3 Brandenburg BB\n", "4 Bremen HB" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
fileweektrend
0Rossmann_DE_SN2012-12-02 - 2012-12-0896
1Rossmann_DE_SN2012-12-09 - 2012-12-1595
2Rossmann_DE_SN2012-12-16 - 2012-12-2291
3Rossmann_DE_SN2012-12-23 - 2012-12-2948
4Rossmann_DE_SN2012-12-30 - 2013-01-0567
\n", "
" ], "text/plain": [ " file week trend\n", "0 Rossmann_DE_SN 2012-12-02 - 2012-12-08 96\n", "1 Rossmann_DE_SN 2012-12-09 - 2012-12-15 95\n", "2 Rossmann_DE_SN 2012-12-16 - 2012-12-22 91\n", "3 Rossmann_DE_SN 2012-12-23 - 2012-12-29 48\n", "4 Rossmann_DE_SN 2012-12-30 - 2013-01-05 67" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
fileDateMax_TemperatureCMean_TemperatureCMin_TemperatureCDew_PointCMeanDew_PointCMin_DewpointCMax_HumidityMean_Humidity...Max_VisibilityKmMean_VisibilityKmMin_VisibilitykMMax_Wind_SpeedKm_hMean_Wind_SpeedKm_hMax_Gust_SpeedKm_hPrecipitationmmCloudCoverEventsWindDirDegrees
0NordrheinWestfalen2013-01-018427519487...31.012.04.0392658.05.086.0Rain215
1NordrheinWestfalen2013-01-027415329385...31.014.010.02416NaN0.006.0Rain225
2NordrheinWestfalen2013-01-031186108410093...31.08.02.02621NaN1.027.0Rain240
3NordrheinWestfalen2013-01-0499899810094...11.05.02.02314NaN0.257.0Rain263
4NordrheinWestfalen2013-01-0588787610094...10.06.03.01610NaN0.007.0Rain268
\n", "

5 rows × 24 columns

\n", "
" ], "text/plain": [ " file Date Max_TemperatureC Mean_TemperatureC \\\n", "0 NordrheinWestfalen 2013-01-01 8 4 \n", "1 NordrheinWestfalen 2013-01-02 7 4 \n", "2 NordrheinWestfalen 2013-01-03 11 8 \n", "3 NordrheinWestfalen 2013-01-04 9 9 \n", "4 NordrheinWestfalen 2013-01-05 8 8 \n", "\n", " Min_TemperatureC Dew_PointC MeanDew_PointC Min_DewpointC Max_Humidity \\\n", "0 2 7 5 1 94 \n", "1 1 5 3 2 93 \n", "2 6 10 8 4 100 \n", "3 8 9 9 8 100 \n", "4 7 8 7 6 100 \n", "\n", " Mean_Humidity ... Max_VisibilityKm Mean_VisibilityKm \\\n", "0 87 ... 31.0 12.0 \n", "1 85 ... 31.0 14.0 \n", "2 93 ... 31.0 8.0 \n", "3 94 ... 11.0 5.0 \n", "4 94 ... 10.0 6.0 \n", "\n", " Min_VisibilitykM Max_Wind_SpeedKm_h Mean_Wind_SpeedKm_h \\\n", "0 4.0 39 26 \n", "1 10.0 24 16 \n", "2 2.0 26 21 \n", "3 2.0 23 14 \n", "4 3.0 16 10 \n", "\n", " Max_Gust_SpeedKm_h Precipitationmm CloudCover Events WindDirDegrees \n", "0 58.0 5.08 6.0 Rain 215 \n", "1 NaN 0.00 6.0 Rain 225 \n", "2 NaN 1.02 7.0 Rain 240 \n", "3 NaN 0.25 7.0 Rain 263 \n", "4 NaN 0.00 7.0 Rain 268 \n", "\n", "[5 rows x 24 columns]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
IdStoreDayOfWeekDateOpenPromoStateHolidaySchoolHoliday
01142015-09-171.0100
12342015-09-171.0100
23742015-09-171.0100
34842015-09-171.0100
45942015-09-171.0100
\n", "
" ], "text/plain": [ " Id Store DayOfWeek Date Open Promo StateHoliday SchoolHoliday\n", "0 1 1 4 2015-09-17 1.0 1 0 0\n", "1 2 3 4 2015-09-17 1.0 1 0 0\n", "2 3 7 4 2015-09-17 1.0 1 0 0\n", "3 4 8 4 2015-09-17 1.0 1 0 0\n", "4 5 9 4 2015-09-17 1.0 1 0 0" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "for t in tables: display(t.head())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is very representative of a typical industry dataset." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following returns summarized aggregate information to each table accross each field." ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreDayOfWeekDateSalesCustomersOpenPromoStateHolidaySchoolHolidayYearMonthWeekDay
count1.01721e+061.01721e+06NaN1.01721e+061.01721e+061.01721e+061.01721e+06NaN1.01721e+061.01721e+061.01721e+061.01721e+061.01721e+06
mean558.433.99834NaN5773.82633.1460.8301070.381515NaN0.1786472013.835.8467623.615515.7028
std321.9091.99739NaN3849.93464.4120.3755390.485759NaN0.3830560.7773963.326114.43348.78764
min11NaN0000NaN02013111
25%2802NaN372740510NaN020133118
50%5584NaN574460910NaN0201462216
75%8386NaN785683711NaN0201483523
max11157NaN41551738811NaN12015125231
counts1017209101720910172091017209101720910172091017209101720910172091017209101720910172091017209
uniques1115794221734408622223125231
missing0000000000000
missing_perc0%0%0%0%0%0%0%0%0%0%0%0%0%
typesnumericnumericdatenumericnumericboolboolboolboolnumericnumericnumericnumeric
\n", "
" ], "text/plain": [ " Store DayOfWeek Date Sales Customers \\\n", "count 1.01721e+06 1.01721e+06 NaN 1.01721e+06 1.01721e+06 \n", "mean 558.43 3.99834 NaN 5773.82 633.146 \n", "std 321.909 1.99739 NaN 3849.93 464.412 \n", "min 1 1 NaN 0 0 \n", "25% 280 2 NaN 3727 405 \n", "50% 558 4 NaN 5744 609 \n", "75% 838 6 NaN 7856 837 \n", "max 1115 7 NaN 41551 7388 \n", "counts 1017209 1017209 1017209 1017209 1017209 \n", "uniques 1115 7 942 21734 4086 \n", "missing 0 0 0 0 0 \n", "missing_perc 0% 0% 0% 0% 0% \n", "types numeric numeric date numeric numeric \n", "\n", " Open Promo StateHoliday SchoolHoliday \\\n", "count 1.01721e+06 1.01721e+06 NaN 1.01721e+06 \n", "mean 0.830107 0.381515 NaN 0.178647 \n", "std 0.375539 0.485759 NaN 0.383056 \n", "min 0 0 NaN 0 \n", "25% 1 0 NaN 0 \n", "50% 1 0 NaN 0 \n", "75% 1 1 NaN 0 \n", "max 1 1 NaN 1 \n", "counts 1017209 1017209 1017209 1017209 \n", "uniques 2 2 2 2 \n", "missing 0 0 0 0 \n", "missing_perc 0% 0% 0% 0% \n", "types bool bool bool bool \n", "\n", " Year Month Week Day \n", "count 1.01721e+06 1.01721e+06 1.01721e+06 1.01721e+06 \n", "mean 2013.83 5.84676 23.6155 15.7028 \n", "std 0.777396 3.3261 14.4334 8.78764 \n", "min 2013 1 1 1 \n", "25% 2013 3 11 8 \n", "50% 2014 6 22 16 \n", "75% 2014 8 35 23 \n", "max 2015 12 52 31 \n", "counts 1017209 1017209 1017209 1017209 \n", "uniques 3 12 52 31 \n", "missing 0 0 0 0 \n", "missing_perc 0% 0% 0% 0% \n", "types numeric numeric numeric numeric " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreStoreTypeAssortmentCompetitionDistanceCompetitionOpenSinceMonthCompetitionOpenSinceYearPromo2Promo2SinceWeekPromo2SinceYearPromoInterval
count1115NaNNaN11127617611115571571NaN
mean558NaNNaN5404.97.22472008.670.51210823.59542011.76NaN
std322.017NaNNaN7663.173.212356.195980.50007814.1421.67494NaN
min1NaNNaN2011900012009NaN
25%279.5NaNNaN717.5420060132011NaN
50%558NaNNaN2325820101222012NaN
75%836.5NaNNaN6882.51020131372013NaN
max1115NaNNaN758601220151502015NaN
counts11151115111511127617611115571571571
uniques111543654122322473
missing00033543540544544544
missing_perc0%0%0%0.27%31.75%31.75%0%48.79%48.79%48.79%
typesnumericcategoricalcategoricalnumericnumericnumericboolnumericnumericcategorical
\n", "
" ], "text/plain": [ " Store StoreType Assortment CompetitionDistance \\\n", "count 1115 NaN NaN 1112 \n", "mean 558 NaN NaN 5404.9 \n", "std 322.017 NaN NaN 7663.17 \n", "min 1 NaN NaN 20 \n", "25% 279.5 NaN NaN 717.5 \n", "50% 558 NaN NaN 2325 \n", "75% 836.5 NaN NaN 6882.5 \n", "max 1115 NaN NaN 75860 \n", "counts 1115 1115 1115 1112 \n", "uniques 1115 4 3 654 \n", "missing 0 0 0 3 \n", "missing_perc 0% 0% 0% 0.27% \n", "types numeric categorical categorical numeric \n", "\n", " CompetitionOpenSinceMonth CompetitionOpenSinceYear Promo2 \\\n", "count 761 761 1115 \n", "mean 7.2247 2008.67 0.512108 \n", "std 3.21235 6.19598 0.500078 \n", "min 1 1900 0 \n", "25% 4 2006 0 \n", "50% 8 2010 1 \n", "75% 10 2013 1 \n", "max 12 2015 1 \n", "counts 761 761 1115 \n", "uniques 12 23 2 \n", "missing 354 354 0 \n", "missing_perc 31.75% 31.75% 0% \n", "types numeric numeric bool \n", "\n", " Promo2SinceWeek Promo2SinceYear PromoInterval \n", "count 571 571 NaN \n", "mean 23.5954 2011.76 NaN \n", "std 14.142 1.67494 NaN \n", "min 1 2009 NaN \n", "25% 13 2011 NaN \n", "50% 22 2012 NaN \n", "75% 37 2013 NaN \n", "max 50 2015 NaN \n", "counts 571 571 571 \n", "uniques 24 7 3 \n", "missing 544 544 544 \n", "missing_perc 48.79% 48.79% 48.79% \n", "types numeric numeric categorical " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreState
count1115NaN
mean558NaN
std322.017NaN
min1NaN
25%279.5NaN
50%558NaN
75%836.5NaN
max1115NaN
counts11151115
uniques111512
missing00
missing_perc0%0%
typesnumericcategorical
\n", "
" ], "text/plain": [ " Store State\n", "count 1115 NaN\n", "mean 558 NaN\n", "std 322.017 NaN\n", "min 1 NaN\n", "25% 279.5 NaN\n", "50% 558 NaN\n", "75% 836.5 NaN\n", "max 1115 NaN\n", "counts 1115 1115\n", "uniques 1115 12\n", "missing 0 0\n", "missing_perc 0% 0%\n", "types numeric categorical" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StateNameState
count1616
unique1616
topSchleswigHolsteinBW
freq11
counts1616
uniques1616
missing00
missing_perc0%0%
typesuniqueunique
\n", "
" ], "text/plain": [ " StateName State\n", "count 16 16\n", "unique 16 16\n", "top SchleswigHolstein BW\n", "freq 1 1\n", "counts 16 16\n", "uniques 16 16\n", "missing 0 0\n", "missing_perc 0% 0%\n", "types unique unique" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
fileweektrendDateStateYearMonthWeekDay
countNaNNaN2072NaNNaN2072207220722072
meanNaNNaN63.8142NaNNaN2013.846.3378425.581115.6959
stdNaNNaN12.6502NaNNaN0.8521733.4030314.85878.83746
minNaNNaN0NaNNaN2012111
25%NaNNaN55NaNNaN20133138
50%NaNNaN64NaNNaN201462516
75%NaNNaN72NaNNaN2015937.2523
maxNaNNaN100NaNNaN2015125231
counts207220722072207219242072207220722072
uniques1414868148134125231
missing00001480000
missing_perc0%0%0%0%7.14%0%0%0%0%
typescategoricalcategoricalnumericdatecategoricalnumericnumericnumericnumeric
\n", "
" ], "text/plain": [ " file week trend Date State Year \\\n", "count NaN NaN 2072 NaN NaN 2072 \n", "mean NaN NaN 63.8142 NaN NaN 2013.84 \n", "std NaN NaN 12.6502 NaN NaN 0.852173 \n", "min NaN NaN 0 NaN NaN 2012 \n", "25% NaN NaN 55 NaN NaN 2013 \n", "50% NaN NaN 64 NaN NaN 2014 \n", "75% NaN NaN 72 NaN NaN 2015 \n", "max NaN NaN 100 NaN NaN 2015 \n", "counts 2072 2072 2072 2072 1924 2072 \n", "uniques 14 148 68 148 13 4 \n", "missing 0 0 0 0 148 0 \n", "missing_perc 0% 0% 0% 0% 7.14% 0% \n", "types categorical categorical numeric date categorical numeric \n", "\n", " Month Week Day \n", "count 2072 2072 2072 \n", "mean 6.33784 25.5811 15.6959 \n", "std 3.40303 14.8587 8.83746 \n", "min 1 1 1 \n", "25% 3 13 8 \n", "50% 6 25 16 \n", "75% 9 37.25 23 \n", "max 12 52 31 \n", "counts 2072 2072 2072 \n", "uniques 12 52 31 \n", "missing 0 0 0 \n", "missing_perc 0% 0% 0% \n", "types numeric numeric numeric " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
fileDateMax_TemperatureCMean_TemperatureCMin_TemperatureCDew_PointCMeanDew_PointCMin_DewpointCMax_HumidityMean_Humidity...Max_VisibilityKmMean_VisibilityKmMin_VisibilitykMMax_Wind_SpeedKm_hMean_Wind_SpeedKm_hMax_Gust_SpeedKm_hPrecipitationmmCloudCoverEventsWindDirDegrees
countNaNNaN1584015840158401584015840158401584015840...154591545915459158401584036041584014667NaN15840
meanNaNNaN14.644110.3896.198998.587826.205813.6261493.659674.2829...24.057612.23987.0251622.766611.972248.86430.8317185.55131NaN175.897
stdNaNNaN8.646017.379266.526396.244786.086776.128397.6785313.4866...8.97685.067944.98068.988625.8728413.0272.513511.68771NaN101.589
minNaNNaN-11-13-15-14-15-734430...000322100NaN-1
25%NaNNaN84142-190.7565...141031683905NaN80
50%NaNNaN151179749476...3111721114806NaN202
75%NaNNaN2116111311810085...3114102714550.257NaN256
maxNaNNaN393124252019100100...3131311015311158.938NaN360
counts15840158401584015840158401584015840158401584015840...1545915459154591584015840360415840146671188915840
uniques169905145404036405371...24322444294741921362
missing0000000000...38138138100122360117339510
missing_perc0%0%0%0%0%0%0%0%0%0%...2.41%2.41%2.41%0%0%77.25%0%7.41%24.94%0%
typescategoricalcategoricalnumericnumericnumericnumericnumericnumericnumericnumeric...numericnumericnumericnumericnumericnumericnumericnumericcategoricalnumeric
\n", "

13 rows × 24 columns

\n", "
" ], "text/plain": [ " file Date Max_TemperatureC Mean_TemperatureC \\\n", "count NaN NaN 15840 15840 \n", "mean NaN NaN 14.6441 10.389 \n", "std NaN NaN 8.64601 7.37926 \n", "min NaN NaN -11 -13 \n", "25% NaN NaN 8 4 \n", "50% NaN NaN 15 11 \n", "75% NaN NaN 21 16 \n", "max NaN NaN 39 31 \n", "counts 15840 15840 15840 15840 \n", "uniques 16 990 51 45 \n", "missing 0 0 0 0 \n", "missing_perc 0% 0% 0% 0% \n", "types categorical categorical numeric numeric \n", "\n", " Min_TemperatureC Dew_PointC MeanDew_PointC Min_DewpointC \\\n", "count 15840 15840 15840 15840 \n", "mean 6.19899 8.58782 6.20581 3.62614 \n", "std 6.52639 6.24478 6.08677 6.12839 \n", "min -15 -14 -15 -73 \n", "25% 1 4 2 -1 \n", "50% 7 9 7 4 \n", "75% 11 13 11 8 \n", "max 24 25 20 19 \n", "counts 15840 15840 15840 15840 \n", "uniques 40 40 36 40 \n", "missing 0 0 0 0 \n", "missing_perc 0% 0% 0% 0% \n", "types numeric numeric numeric numeric \n", "\n", " Max_Humidity Mean_Humidity ... Max_VisibilityKm \\\n", "count 15840 15840 ... 15459 \n", "mean 93.6596 74.2829 ... 24.0576 \n", "std 7.67853 13.4866 ... 8.9768 \n", "min 44 30 ... 0 \n", "25% 90.75 65 ... 14 \n", "50% 94 76 ... 31 \n", "75% 100 85 ... 31 \n", "max 100 100 ... 31 \n", "counts 15840 15840 ... 15459 \n", "uniques 53 71 ... 24 \n", "missing 0 0 ... 381 \n", "missing_perc 0% 0% ... 2.41% \n", "types numeric numeric ... numeric \n", "\n", " Mean_VisibilityKm Min_VisibilitykM Max_Wind_SpeedKm_h \\\n", "count 15459 15459 15840 \n", "mean 12.2398 7.02516 22.7666 \n", "std 5.06794 4.9806 8.98862 \n", "min 0 0 3 \n", "25% 10 3 16 \n", "50% 11 7 21 \n", "75% 14 10 27 \n", "max 31 31 101 \n", "counts 15459 15459 15840 \n", "uniques 32 24 44 \n", "missing 381 381 0 \n", "missing_perc 2.41% 2.41% 0% \n", "types numeric numeric numeric \n", "\n", " Mean_Wind_SpeedKm_h Max_Gust_SpeedKm_h Precipitationmm \\\n", "count 15840 3604 15840 \n", "mean 11.9722 48.8643 0.831718 \n", "std 5.87284 13.027 2.51351 \n", "min 2 21 0 \n", "25% 8 39 0 \n", "50% 11 48 0 \n", "75% 14 55 0.25 \n", "max 53 111 58.93 \n", "counts 15840 3604 15840 \n", "uniques 29 47 41 \n", "missing 0 12236 0 \n", "missing_perc 0% 77.25% 0% \n", "types numeric numeric numeric \n", "\n", " CloudCover Events WindDirDegrees \n", "count 14667 NaN 15840 \n", "mean 5.55131 NaN 175.897 \n", "std 1.68771 NaN 101.589 \n", "min 0 NaN -1 \n", "25% 5 NaN 80 \n", "50% 6 NaN 202 \n", "75% 7 NaN 256 \n", "max 8 NaN 360 \n", "counts 14667 11889 15840 \n", "uniques 9 21 362 \n", "missing 1173 3951 0 \n", "missing_perc 7.41% 24.94% 0% \n", "types numeric categorical numeric \n", "\n", "[13 rows x 24 columns]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
IdStoreDayOfWeekDateOpenPromoStateHolidaySchoolHolidayYearMonthWeekDay
count410884108841088NaN4107741088NaN4108841088410884108841088
mean20544.5555.93.97917NaN0.8543220.395833NaN0.44348720158.3541734.645813.5208
std11861.2320.2742.01548NaN0.3527870.489035NaN0.49680200.4782662.015488.44845
min111NaN00NaN020158311
25%10272.8279.752NaN10NaN020158336.75
50%20544.5553.54NaN10NaN0201583512.5
75%30816.2832.256NaN11NaN1201593619.25
max4108811157NaN11NaN1201593831
counts410884108841088410884107741088410884108841088410884108841088
uniques41088856748222212831
missing0000110000000
missing_perc0%0%0%0%0.03%0%0%0%0%0%0%0%
typesnumericnumericnumericdateboolboolboolboolconstantboolnumericnumeric
\n", "
" ], "text/plain": [ " Id Store DayOfWeek Date Open Promo \\\n", "count 41088 41088 41088 NaN 41077 41088 \n", "mean 20544.5 555.9 3.97917 NaN 0.854322 0.395833 \n", "std 11861.2 320.274 2.01548 NaN 0.352787 0.489035 \n", "min 1 1 1 NaN 0 0 \n", "25% 10272.8 279.75 2 NaN 1 0 \n", "50% 20544.5 553.5 4 NaN 1 0 \n", "75% 30816.2 832.25 6 NaN 1 1 \n", "max 41088 1115 7 NaN 1 1 \n", "counts 41088 41088 41088 41088 41077 41088 \n", "uniques 41088 856 7 48 2 2 \n", "missing 0 0 0 0 11 0 \n", "missing_perc 0% 0% 0% 0% 0.03% 0% \n", "types numeric numeric numeric date bool bool \n", "\n", " StateHoliday SchoolHoliday Year Month Week Day \n", "count NaN 41088 41088 41088 41088 41088 \n", "mean NaN 0.443487 2015 8.35417 34.6458 13.5208 \n", "std NaN 0.496802 0 0.478266 2.01548 8.44845 \n", "min NaN 0 2015 8 31 1 \n", "25% NaN 0 2015 8 33 6.75 \n", "50% NaN 0 2015 8 35 12.5 \n", "75% NaN 1 2015 9 36 19.25 \n", "max NaN 1 2015 9 38 31 \n", "counts 41088 41088 41088 41088 41088 41088 \n", "uniques 2 2 1 2 8 31 \n", "missing 0 0 0 0 0 0 \n", "missing_perc 0% 0% 0% 0% 0% 0% \n", "types bool bool constant bool numeric numeric " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "for t in tables: display(DataFrameSummary(t).summary())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data Cleaning / Feature Engineering" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As a structured data problem, we necessarily have to go through all the cleaning and feature engineering, even though we're using a neural network." ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "collapsed": true }, "outputs": [], "source": [ "train, store, store_states, state_names, googletrend, weather, test = tables" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(1017209, 41088)" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(train),len(test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Turn state Holidays to Bool" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "collapsed": true }, "outputs": [], "source": [ "train.StateHoliday = train.StateHoliday!='0'\n", "test.StateHoliday = test.StateHoliday!='0'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Define function for joining tables on specific fields.\n", "\n", "By default, we'll be doing a left outer join of `right` on the `left` argument using the given fields for each table.\n", "\n", "Pandas does joins using the `merge` method. The `suffixes` argument describes the naming convention for duplicate fields. We've elected to leave the duplicate field names on the left untouched, and append a \"_y\" to those on the right." ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def join_df(left, right, left_on, right_on=None):\n", " if right_on is None: right_on = left_on\n", " return left.merge(right, how='left', left_on=left_on, right_on=right_on, \n", " suffixes=(\"\", \"_y\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Join weather/state names." ] }, { "cell_type": "code", "execution_count": 48, "metadata": { "collapsed": true }, "outputs": [], "source": [ "weather = join_df(weather, state_names, \"file\", \"StateName\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In pandas you can add new columns to a dataframe by simply defining it. We'll do this for googletrends by extracting dates and state names from the given data and adding those columns.\n", "\n", "We're also going to replace all instances of state name 'NI' with the usage in the rest of the table, 'HB,NI'. This is a good opportunity to highlight pandas indexing. We can use `.ix[rows, cols]` to select a list of rows and a list of columns from the dataframe. In this case, we're selecting rows w/ statename 'NI' by using a boolean list `googletrend.State=='NI'` and selecting \"State\"." ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "collapsed": true }, "outputs": [], "source": [ "googletrend['Date'] = googletrend.week.str.split(' - ', expand=True)[0]\n", "googletrend['State'] = googletrend.file.str.split('_', expand=True)[2]\n", "googletrend.loc[googletrend.State=='NI', \"State\"] = 'HB,NI'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following extracts particular date fields from a complete datetime for the purpose of constructing categoricals.\n", "\n", "You should always consider this feature extraction step when working with date-time. Without expanding your date-time into these additional fields, you can't capture any trend/cyclical behavior as a function of time at any of these granularities." ] }, { "cell_type": "code", "execution_count": 50, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def add_datepart(df):\n", " df.Date = pd.to_datetime(df.Date)\n", " df[\"Year\"] = df.Date.dt.year\n", " df[\"Month\"] = df.Date.dt.month\n", " df[\"Week\"] = df.Date.dt.week\n", " df[\"Day\"] = df.Date.dt.day" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll add to every table w/ a date field." ] }, { "cell_type": "code", "execution_count": 51, "metadata": { "collapsed": true }, "outputs": [], "source": [ "add_datepart(weather)\n", "add_datepart(googletrend)\n", "add_datepart(train)\n", "add_datepart(test)" ] }, { "cell_type": "code", "execution_count": 52, "metadata": { "collapsed": true }, "outputs": [], "source": [ "trend_de = googletrend[googletrend.file == 'Rossmann_DE']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can outer join all of our data into a single dataframe.\n", "\n", "Recall that in outer joins everytime a value in the joining field on the left table does not have a corresponding value on the right table, the corresponding row in the new table has Null values for all right table fields.\n", "\n", "One way to check that all records are consistent and complete is to check for Null values post-join, as we do here.\n", "\n", "*Aside*: Why note just do an inner join?\n", "If you are assuming that all records are complete and match on the field you desire, an inner join will do the same thing as an outer join. However, in the event you are wrong or a mistake is made, an outer join followed by a null-check will catch it. (Comparing before/after # of rows for inner join is equivalent, but requires keeping track of before/after row #'s. Outer join is easier.)" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "store = join_df(store, store_states, \"Store\")\n", "len(store[store.State.isnull()])" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined = join_df(train, store, \"Store\")\n", "len(joined[joined.StoreType.isnull()])" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined = join_df(joined, googletrend, [\"State\",\"Year\", \"Week\"])\n", "len(joined[joined.trend.isnull()])" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined = joined.merge(trend_de, 'left', [\"Year\", \"Week\"], suffixes=('', '_DE'))\n", "len(joined[joined.trend_DE.isnull()])" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined = join_df(joined, weather, [\"State\",\"Date\"])\n", "len(joined[joined.Mean_TemperatureC.isnull()])" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "48" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined_test = test.merge(store, how='left', left_on='Store', right_index=True)\n", "len(joined_test[joined_test.StoreType.isnull()])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we'll fill in missing values to avoid complications w/ na's." ] }, { "cell_type": "code", "execution_count": 59, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined.CompetitionOpenSinceYear = joined.CompetitionOpenSinceYear.fillna(1900).astype(np.int32)\n", "joined.CompetitionOpenSinceMonth = joined.CompetitionOpenSinceMonth.fillna(1).astype(np.int32)\n", "joined.Promo2SinceYear = joined.Promo2SinceYear.fillna(1900).astype(np.int32)\n", "joined.Promo2SinceWeek = joined.Promo2SinceWeek.fillna(1).astype(np.int32)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we'll extract features \"CompetitionOpenSince\" and \"CompetitionDaysOpen\". Note the use of `apply()` in mapping a function across dataframe values." ] }, { "cell_type": "code", "execution_count": 60, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined[\"CompetitionOpenSince\"] = pd.to_datetime(joined.apply(lambda x: datetime.datetime(\n", " x.CompetitionOpenSinceYear, x.CompetitionOpenSinceMonth, 15), axis=1).astype(pd.datetime))\n", "joined[\"CompetitionDaysOpen\"] = joined.Date.subtract(joined[\"CompetitionOpenSince\"]).dt.days" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll replace some erroneous / outlying data." ] }, { "cell_type": "code", "execution_count": 63, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined.loc[joined.CompetitionDaysOpen<0, \"CompetitionDaysOpen\"] = 0\n", "joined.loc[joined.CompetitionOpenSinceYear<1990, \"CompetitionDaysOpen\"] = 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Added \"CompetitionMonthsOpen\" field, limit the maximum to 2 years to limit number of unique embeddings." ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([24, 3, 19, 9, 0, 16, 17, 7, 15, 22, 11, 13, 2, 23, 12, 4, 10,\n", " 1, 14, 20, 8, 18, 6, 21, 5])" ] }, "execution_count": 64, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined[\"CompetitionMonthsOpen\"] = joined[\"CompetitionDaysOpen\"]//30\n", "joined.loc[joined.CompetitionMonthsOpen>24, \"CompetitionMonthsOpen\"] = 24\n", "joined.CompetitionMonthsOpen.unique()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Same process for Promo dates." ] }, { "cell_type": "code", "execution_count": 65, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined[\"Promo2Since\"] = pd.to_datetime(joined.apply(lambda x: Week(\n", " x.Promo2SinceYear, x.Promo2SinceWeek).monday(), axis=1).astype(pd.datetime))\n", "joined[\"Promo2Days\"] = joined.Date.subtract(joined[\"Promo2Since\"]).dt.days" ] }, { "cell_type": "code", "execution_count": 66, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined.loc[joined.Promo2Days<0, \"Promo2Days\"] = 0\n", "joined.loc[joined.Promo2SinceYear<1990, \"Promo2Days\"] = 0" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 0, 25, 17, 8, 13, 24, 16, 7, 12, 23, 15, 6, 11, 22, 14, 5, 10,\n", " 21, 4, 9, 20, 3, 19, 2, 18, 1])" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined[\"Promo2Weeks\"] = joined[\"Promo2Days\"]//7\n", "joined.loc[joined.Promo2Weeks<0, \"Promo2Weeks\"] = 0\n", "joined.loc[joined.Promo2Weeks>25, \"Promo2Weeks\"] = 25\n", "joined.Promo2Weeks.unique()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Durations" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is common when working with time series data to extract data that explains relationships across rows as opposed to columns, e.g.:\n", "* Running averages\n", "* Time until next event\n", "* Time since last event\n", "\n", "This is often difficult to do with most table manipulation frameworks, since they are designed to work with relationships across columns. As such, we've created a class to handle this type of data." ] }, { "cell_type": "code", "execution_count": 72, "metadata": { "collapsed": true }, "outputs": [], "source": [ "columns = [\"Date\", \"Store\", \"Promo\", \"StateHoliday\", \"SchoolHoliday\"]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We've defined a class `elapsed` for cumulative counting across a sorted dataframe.\n", "\n", "Given a particular field `fld` to monitor, this object will start tracking time since the last occurrence of that field. When the field is seen again, the counter is set to zero.\n", "\n", "Upon initialization, this will result in datetime na's until the field is encountered. This is reset every time a new store is seen.\n", "\n", "We'll see how to use this shortly." ] }, { "cell_type": "code", "execution_count": 73, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class elapsed(object):\n", " def __init__(self, fld):\n", " self.fld = fld\n", " self.last = pd.to_datetime(np.nan)\n", " self.last_store = 0\n", " \n", " def get(self, row):\n", " if row.Store != self.last_store:\n", " self.last = pd.to_datetime(np.nan)\n", " self.last_store = row.Store\n", " if (row[self.fld]): self.last = row.Date\n", " return row.Date-self.last" ] }, { "cell_type": "code", "execution_count": 74, "metadata": { "collapsed": true, "scrolled": true }, "outputs": [], "source": [ "df = train[columns]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And a function for applying said class across dataframe rows and adding values to a new column." ] }, { "cell_type": "code", "execution_count": 75, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def add_elapsed(fld, prefix):\n", " sh_el = elapsed(fld)\n", " df[prefix+fld] = df.apply(sh_el.get, axis=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's walk through an example.\n", "\n", "Say we're looking at School Holiday. We'll first sort by Store, then Date, and then call `add_elapsed('SchoolHoliday', 'After')`:\n", "This will generate an instance of the `elapsed` class for School Holiday:\n", "* Instance applied to every row of the dataframe in order of store and date\n", "* Will add to the dataframe the days since seeing a School Holiday\n", "* If we sort in the other direction, this will count the days until another promotion." ] }, { "cell_type": "code", "execution_count": 76, "metadata": { "collapsed": true }, "outputs": [], "source": [ "fld = 'SchoolHoliday'\n", "df = df.sort_values(['Store', 'Date'])\n", "add_elapsed(fld, 'After')\n", "df = df.sort_values(['Store', 'Date'], ascending=[True, False])\n", "add_elapsed(fld, 'Before')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll do this for two more fields." ] }, { "cell_type": "code", "execution_count": 77, "metadata": { "collapsed": true }, "outputs": [], "source": [ "fld = 'StateHoliday'\n", "df = df.sort_values(['Store', 'Date'])\n", "add_elapsed(fld, 'After')\n", "df = df.sort_values(['Store', 'Date'], ascending=[True, False])\n", "add_elapsed(fld, 'Before')" ] }, { "cell_type": "code", "execution_count": 78, "metadata": { "collapsed": true }, "outputs": [], "source": [ "fld = 'Promo'\n", "df = df.sort_values(['Store', 'Date'])\n", "add_elapsed(fld, 'After')\n", "df = df.sort_values(['Store', 'Date'], ascending=[True, False])\n", "add_elapsed(fld, 'Before')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We're going to set the active index to Date." ] }, { "cell_type": "code", "execution_count": 79, "metadata": { "collapsed": true }, "outputs": [], "source": [ "df = df.set_index(\"Date\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then set null values from elapsed field calculations to 0." ] }, { "cell_type": "code", "execution_count": 80, "metadata": { "collapsed": true }, "outputs": [], "source": [ "columns = ['SchoolHoliday', 'StateHoliday', 'Promo']" ] }, { "cell_type": "code", "execution_count": 81, "metadata": { "collapsed": true }, "outputs": [], "source": [ "for o in ['Before', 'After']:\n", " for p in columns:\n", " a = o+p\n", " df[a] = df[a].fillna(pd.Timedelta(0)).dt.days" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we'll demonstrate window functions in pandas to calculate rolling quantities.\n", "\n", "Here we're sorting by date (`sort_index()`) and counting the number of events of interest (`sum()`) defined in `columns` in the following week (`rolling()`), grouped by Store (`groupby()`). We do the same in the opposite direction." ] }, { "cell_type": "code", "execution_count": 83, "metadata": { "collapsed": true }, "outputs": [], "source": [ "bwd = df[['Store']+columns].sort_index().groupby(\"Store\").rolling(7, min_periods=1).sum()" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "collapsed": true }, "outputs": [], "source": [ "fwd = df[['Store']+columns].sort_index(ascending=False\n", " ).groupby(\"Store\").rolling(7, min_periods=1).sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we want to drop the Store indices grouped together in the window function.\n", "\n", "Often in pandas, there is an option to do this in place. This is time and memory efficient when working with large datasets." ] }, { "cell_type": "code", "execution_count": 85, "metadata": { "collapsed": true }, "outputs": [], "source": [ "bwd.drop('Store',1,inplace=True)\n", "bwd.reset_index(inplace=True)" ] }, { "cell_type": "code", "execution_count": 90, "metadata": { "collapsed": true }, "outputs": [], "source": [ "fwd.drop('Store',1,inplace=True)\n", "fwd.reset_index(inplace=True)" ] }, { "cell_type": "code", "execution_count": 91, "metadata": { "collapsed": true }, "outputs": [], "source": [ "df.reset_index(inplace=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we'll merge these values onto the df." ] }, { "cell_type": "code", "execution_count": 92, "metadata": { "collapsed": true }, "outputs": [], "source": [ "df = df.merge(bwd, 'left', ['Date', 'Store'], suffixes=['', '_bw'])\n", "df = df.merge(fwd, 'left', ['Date', 'Store'], suffixes=['', '_fw'])" ] }, { "cell_type": "code", "execution_count": 94, "metadata": { "collapsed": true }, "outputs": [], "source": [ "df.drop(columns,1,inplace=True)" ] }, { "cell_type": "code", "execution_count": 100, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DateStoreAfterSchoolHolidayBeforeSchoolHolidayAfterStateHolidayBeforeStateHolidayAfterPromoBeforePromoSchoolHoliday_bwStateHoliday_bwPromo_bwSchoolHoliday_fwStateHoliday_fwPromo_fw
02015-07-31100570005.00.05.01.00.01.0
12015-07-30100560004.00.04.02.00.02.0
22015-07-29100550003.00.03.03.00.03.0
32015-07-28100540002.00.02.04.00.04.0
42015-07-27100530001.00.01.05.00.05.0
\n", "
" ], "text/plain": [ " Date Store AfterSchoolHoliday BeforeSchoolHoliday \\\n", "0 2015-07-31 1 0 0 \n", "1 2015-07-30 1 0 0 \n", "2 2015-07-29 1 0 0 \n", "3 2015-07-28 1 0 0 \n", "4 2015-07-27 1 0 0 \n", "\n", " AfterStateHoliday BeforeStateHoliday AfterPromo BeforePromo \\\n", "0 57 0 0 0 \n", "1 56 0 0 0 \n", "2 55 0 0 0 \n", "3 54 0 0 0 \n", "4 53 0 0 0 \n", "\n", " SchoolHoliday_bw StateHoliday_bw Promo_bw SchoolHoliday_fw \\\n", "0 5.0 0.0 5.0 1.0 \n", "1 4.0 0.0 4.0 2.0 \n", "2 3.0 0.0 3.0 3.0 \n", "3 2.0 0.0 2.0 4.0 \n", "4 1.0 0.0 1.0 5.0 \n", "\n", " StateHoliday_fw Promo_fw \n", "0 0.0 1.0 \n", "1 0.0 2.0 \n", "2 0.0 3.0 \n", "3 0.0 4.0 \n", "4 0.0 5.0 " ] }, "execution_count": 100, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It's usually a good idea to back up large tables of extracted / wrangled features before you join them onto another one, that way you can go back to it easily if you need to make changes to it." ] }, { "cell_type": "code", "execution_count": 96, "metadata": { "collapsed": true }, "outputs": [], "source": [ "df.to_csv('df.csv')" ] }, { "cell_type": "code", "execution_count": 97, "metadata": { "collapsed": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/bckenstler/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/lib/arraysetops.py:395: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n", " mask |= (ar1 == a)\n" ] } ], "source": [ "df = pd.read_csv('df.csv', index_col=0)" ] }, { "cell_type": "code", "execution_count": 98, "metadata": { "collapsed": true }, "outputs": [], "source": [ "df[\"Date\"] = pd.to_datetime(df.Date)" ] }, { "cell_type": "code", "execution_count": 99, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Index(['Date', 'Store', 'AfterSchoolHoliday', 'BeforeSchoolHoliday',\n", " 'AfterStateHoliday', 'BeforeStateHoliday', 'AfterPromo', 'BeforePromo',\n", " 'SchoolHoliday_bw', 'StateHoliday_bw', 'Promo_bw', 'SchoolHoliday_fw',\n", " 'StateHoliday_fw', 'Promo_fw'],\n", " dtype='object')" ] }, "execution_count": 99, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.columns" ] }, { "cell_type": "code", "execution_count": 101, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined = join_df(joined, df, ['Store', 'Date'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll back this up as well." ] }, { "cell_type": "code", "execution_count": 102, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined.to_csv('joined.csv')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We now have our final set of engineered features." ] }, { "cell_type": "code", "execution_count": 111, "metadata": { "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/bckenstler/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/lib/arraysetops.py:395: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n", " mask |= (ar1 == a)\n" ] }, { "data": { "text/plain": [ "Index(['Store', 'DayOfWeek', 'Date', 'Sales', 'Customers', 'Open', 'Promo',\n", " 'StateHoliday', 'SchoolHoliday', 'Year', 'Month', 'Week', 'Day',\n", " 'StoreType', 'Assortment', 'CompetitionDistance',\n", " 'CompetitionOpenSinceMonth', 'CompetitionOpenSinceYear', 'Promo2',\n", " 'Promo2SinceWeek', 'Promo2SinceYear', 'PromoInterval', 'State', 'file',\n", " 'week', 'trend', 'Date_y', 'Month_y', 'Day_y', 'file_DE', 'week_DE',\n", " 'trend_DE', 'Date_DE', 'State_DE', 'Month_DE', 'Day_DE', 'file_y',\n", " 'Max_TemperatureC', 'Mean_TemperatureC', 'Min_TemperatureC',\n", " 'Dew_PointC', 'MeanDew_PointC', 'Min_DewpointC', 'Max_Humidity',\n", " 'Mean_Humidity', 'Min_Humidity', 'Max_Sea_Level_PressurehPa',\n", " 'Mean_Sea_Level_PressurehPa', 'Min_Sea_Level_PressurehPa',\n", " 'Max_VisibilityKm', 'Mean_VisibilityKm', 'Min_VisibilitykM',\n", " 'Max_Wind_SpeedKm_h', 'Mean_Wind_SpeedKm_h', 'Max_Gust_SpeedKm_h',\n", " 'Precipitationmm', 'CloudCover', 'Events', 'WindDirDegrees',\n", " 'StateName', 'Year_y', 'Month_y.1', 'Week_y', 'Day_y.1',\n", " 'CompetitionOpenSince', 'CompetitionDaysOpen', 'CompetitionMonthsOpen',\n", " 'Promo2Since', 'Promo2Days', 'Promo2Weeks', 'AfterSchoolHoliday',\n", " 'BeforeSchoolHoliday', 'AfterStateHoliday', 'BeforeStateHoliday',\n", " 'AfterPromo', 'BeforePromo', 'SchoolHoliday_bw', 'StateHoliday_bw',\n", " 'Promo_bw', 'SchoolHoliday_fw', 'StateHoliday_fw', 'Promo_fw'],\n", " dtype='object')" ] }, "execution_count": 111, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined = pd.read_csv('joined.csv', index_col=0)\n", "joined[\"Date\"] = pd.to_datetime(joined.Date)\n", "joined.columns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "While these steps were explicitly outlined in the paper, these are all fairly typical feature engineering steps for dealing with time series data and are practical in any similar setting." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## Create features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we've engineered all our features, we need to convert to input compatible with a neural network.\n", "\n", "This includes converting categorical variables into contiguous integers or one-hot encodings, normalizing continuous features to standard normal, etc..." ] }, { "cell_type": "code", "execution_count": 104, "metadata": { "collapsed": true, "scrolled": false }, "outputs": [], "source": [ "from sklearn_pandas import DataFrameMapper\n", "from sklearn.preprocessing import LabelEncoder, Imputer, StandardScaler" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This dictionary maps categories to embedding dimensionality. In generally, categories we might expect to be conceptually more complex have larger dimension." ] }, { "cell_type": "code", "execution_count": 105, "metadata": { "collapsed": true }, "outputs": [], "source": [ "cat_var_dict = {'Store': 50, 'DayOfWeek': 6, 'Year': 2, 'Month': 6,\n", "'Day': 10, 'StateHoliday': 3, 'CompetitionMonthsOpen': 2,\n", "'Promo2Weeks': 1, 'StoreType': 2, 'Assortment': 3, 'PromoInterval': 3,\n", "'CompetitionOpenSinceYear': 4, 'Promo2SinceYear': 4, 'State': 6,\n", "'Week': 2, 'Events': 4, 'Promo_fw': 1,\n", "'Promo_bw': 1, 'StateHoliday_fw': 1,\n", "'StateHoliday_bw': 1, 'SchoolHoliday_fw': 1,\n", "'SchoolHoliday_bw': 1}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Name categorical variables" ] }, { "cell_type": "code", "execution_count": 106, "metadata": { "collapsed": true }, "outputs": [], "source": [ "cat_vars = [o[0] for o in \n", " sorted(cat_var_dict.items(), key=operator.itemgetter(1), reverse=True)]" ] }, { "cell_type": "code", "execution_count": 107, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "\"cat_vars = ['Store', 'DayOfWeek', 'Year', 'Month', 'Day', 'StateHoliday',\\n 'StoreType', 'Assortment', 'Week', 'Events', 'Promo2SinceYear',\\n 'CompetitionOpenSinceYear', 'PromoInterval', 'Promo', 'SchoolHoliday', 'State']\"" ] }, "execution_count": 107, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"\"\"cat_vars = ['Store', 'DayOfWeek', 'Year', 'Month', 'Day', 'StateHoliday',\n", " 'StoreType', 'Assortment', 'Week', 'Events', 'Promo2SinceYear',\n", " 'CompetitionOpenSinceYear', 'PromoInterval', 'Promo', 'SchoolHoliday', 'State']\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Likewise for continuous" ] }, { "cell_type": "code", "execution_count": 108, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# mean/max wind; min temp; cloud; min/mean humid; \n", "contin_vars = ['CompetitionDistance', \n", " 'Max_TemperatureC', 'Mean_TemperatureC', 'Min_TemperatureC',\n", " 'Max_Humidity', 'Mean_Humidity', 'Min_Humidity', 'Max_Wind_SpeedKm_h', \n", " 'Mean_Wind_SpeedKm_h', 'CloudCover', 'trend', 'trend_DE',\n", " 'AfterStateHoliday', 'BeforeStateHoliday', 'Promo', 'SchoolHoliday']" ] }, { "cell_type": "code", "execution_count": 109, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"contin_vars = ['CompetitionDistance', 'Max_TemperatureC', 'Mean_TemperatureC', \\n 'Max_Humidity', 'trend', 'trend_DE', 'AfterStateHoliday', 'BeforeStateHoliday']\"" ] }, "execution_count": 109, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"\"\"contin_vars = ['CompetitionDistance', 'Max_TemperatureC', 'Mean_TemperatureC', \n", " 'Max_Humidity', 'trend', 'trend_DE', 'AfterStateHoliday', 'BeforeStateHoliday']\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Replace nulls w/ 0 for continuous, \"\" for categorical." ] }, { "cell_type": "code", "execution_count": 112, "metadata": { "collapsed": true }, "outputs": [], "source": [ "for v in contin_vars: joined.loc[joined[v].isnull(), v] = 0\n", "for v in cat_vars: joined.loc[joined[v].isnull(), v] = \"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we create a list of tuples, each containing a variable and an instance of a transformer for that variable.\n", "\n", "For categoricals, we use a label encoder that maps categories to continuous integers. For continuous variables, we standardize them." ] }, { "cell_type": "code", "execution_count": 113, "metadata": { "collapsed": true }, "outputs": [], "source": [ "cat_maps = [(o, LabelEncoder()) for o in cat_vars]\n", "contin_maps = [([o], StandardScaler()) for o in contin_vars]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The same instances need to be used for the test set as well, so values are mapped/standardized appropriately.\n", "\n", "DataFrame mapper will keep track of these variable-instance mappings." ] }, { "cell_type": "code", "execution_count": 114, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "22" ] }, "execution_count": 114, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cat_mapper = DataFrameMapper(cat_maps)\n", "cat_map_fit = cat_mapper.fit(joined)\n", "cat_cols = len(cat_map_fit.features)\n", "cat_cols" ] }, { "cell_type": "code", "execution_count": 115, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/bckenstler/anaconda3/envs/py36/lib/python3.6/site-packages/sklearn/utils/validation.py:429: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.\n", " warnings.warn(msg, _DataConversionWarning)\n" ] }, { "data": { "text/plain": [ "16" ] }, "execution_count": 115, "metadata": {}, "output_type": "execute_result" } ], "source": [ "contin_mapper = DataFrameMapper(contin_maps)\n", "contin_map_fit = contin_mapper.fit(joined)\n", "contin_cols = len(contin_map_fit.features)\n", "contin_cols" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Example of first five rows of zeroth column being transformed appropriately." ] }, { "cell_type": "code", "execution_count": 116, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/bckenstler/anaconda3/envs/py36/lib/python3.6/site-packages/sklearn/utils/validation.py:429: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.\n", " warnings.warn(msg, _DataConversionWarning)\n" ] }, { "data": { "text/plain": [ "(array([ 0, 30, 4, 6, 4]),\n", " array([-0.53772351, 1.02444552, 0.81863226, 0.32504749, 0.61112004]))" ] }, "execution_count": 116, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cat_map_fit.transform(joined)[0,:5], contin_map_fit.transform(joined)[0,:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also pickle these mappings, which is great for portability!" ] }, { "cell_type": "code", "execution_count": 117, "metadata": { "collapsed": true }, "outputs": [], "source": [ "pickle.dump(contin_map_fit, open('contin_maps.pickle', 'wb'))\n", "pickle.dump(cat_map_fit, open('cat_maps.pickle', 'wb'))" ] }, { "cell_type": "code", "execution_count": 119, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1115, 31, 7, 12, 12, 23, 8, 22, 2, 3, 4, 3, 25, 4, 52, 26, 6, 6, 3, 3, 8, 8]" ] }, "execution_count": 119, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[len(o[1].classes_) for o in cat_map_fit.features]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Sample data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, the authors removed all instances where the store had zero sale / was closed." ] }, { "cell_type": "code", "execution_count": 121, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined_sales = joined[joined.Sales!=0]\n", "n = len(joined_sales)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We speculate that this may have cost them a higher standing in the competition. One reason this may be the case is that a little EDA reveals that there are often periods where stores are closed, typically for refurbishment. Before and after these periods, there are naturally spikes in sales that one might expect. Be ommitting this data from their training, the authors gave up the ability to leverage information about these periods to predict this otherwise volatile behavior." ] }, { "cell_type": "code", "execution_count": 122, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "844338" ] }, "execution_count": 122, "metadata": {}, "output_type": "execute_result" } ], "source": [ "n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We're going to run on a sample." ] }, { "cell_type": "code", "execution_count": 123, "metadata": { "collapsed": true }, "outputs": [], "source": [ "samp_size = 100000\n", "np.random.seed(42)\n", "idxs = sorted(np.random.choice(n, samp_size, replace=False))" ] }, { "cell_type": "code", "execution_count": 124, "metadata": { "collapsed": true }, "outputs": [], "source": [ "joined_samp = joined_sales.iloc[idxs].set_index(\"Date\")" ] }, { "cell_type": "code", "execution_count": 125, "metadata": { "collapsed": true }, "outputs": [], "source": [ "samp_size = n\n", "joined_samp = joined_sales.set_index(\"Date\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In time series data, cross-validation is not random. Instead, our holdout data is always the most recent data, as it would be in real application." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We've taken the last 10% as our validation set." ] }, { "cell_type": "code", "execution_count": 126, "metadata": { "collapsed": true }, "outputs": [], "source": [ "train_ratio = 0.9\n", "train_size = int(samp_size * train_ratio)" ] }, { "cell_type": "code", "execution_count": 127, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "759904" ] }, "execution_count": 127, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_size" ] }, { "cell_type": "code", "execution_count": 128, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(84434, 759904)" ] }, "execution_count": 128, "metadata": {}, "output_type": "execute_result" } ], "source": [ "joined_valid = joined_samp[train_size:]\n", "joined_train = joined_samp[:train_size]\n", "len(joined_valid), len(joined_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here's a preprocessor for our categoricals using our instance mapper." ] }, { "cell_type": "code", "execution_count": 129, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def cat_preproc(dat):\n", " return cat_map_fit.transform(dat).astype(np.int64)" ] }, { "cell_type": "code", "execution_count": 130, "metadata": { "collapsed": true }, "outputs": [], "source": [ "cat_map_train = cat_preproc(joined_train)\n", "cat_map_valid = cat_preproc(joined_valid)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Same for continuous." ] }, { "cell_type": "code", "execution_count": 131, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def contin_preproc(dat):\n", " return contin_map_fit.transform(dat).astype(np.float32)" ] }, { "cell_type": "code", "execution_count": 132, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/bckenstler/anaconda3/envs/py36/lib/python3.6/site-packages/sklearn/utils/validation.py:429: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.\n", " warnings.warn(msg, _DataConversionWarning)\n" ] } ], "source": [ "contin_map_train = contin_preproc(joined_train)\n", "contin_map_valid = contin_preproc(joined_valid)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Grab our targets." ] }, { "cell_type": "code", "execution_count": 133, "metadata": { "collapsed": true }, "outputs": [], "source": [ "y_train_orig = joined_train.Sales\n", "y_valid_orig = joined_valid.Sales" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, the authors modified the target values by applying a logarithmic transformation and normalizing to unit scale by dividing by the maximum log value.\n", "\n", "Log transformations are used on this type of data frequently to attain a nicer shape. \n", "\n", "Further by scaling to the unit interval we can now use a sigmoid output in our neural network. Then we can multiply by the maximum log value to get the original log value and transform back." ] }, { "cell_type": "code", "execution_count": 134, "metadata": { "collapsed": true }, "outputs": [], "source": [ "max_log_y = np.max(np.log(joined_samp.Sales))\n", "y_train = np.log(y_train_orig)/max_log_y\n", "y_valid = np.log(y_valid_orig)/max_log_y" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note: Some testing shows this doesn't make a big difference." ] }, { "cell_type": "code", "execution_count": 1066, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'#y_train = np.log(y_train)\\nymean=y_train_orig.mean()\\nystd=y_train_orig.std()\\ny_train = (y_train_orig-ymean)/ystd\\n#y_valid = np.log(y_valid)\\ny_valid = (y_valid_orig-ymean)/ystd'" ] }, "execution_count": 1066, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"\"\"#y_train = np.log(y_train)\n", "ymean=y_train_orig.mean()\n", "ystd=y_train_orig.std()\n", "y_train = (y_train_orig-ymean)/ystd\n", "#y_valid = np.log(y_valid)\n", "y_valid = (y_valid_orig-ymean)/ystd\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Root-mean-squared percent error is the metric Kaggle used for this competition." ] }, { "cell_type": "code", "execution_count": 136, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def rmspe(y_pred, targ = y_valid_orig):\n", " pct_var = (targ - y_pred)/targ\n", " return math.sqrt(np.square(pct_var).mean())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These undo the target transformations." ] }, { "cell_type": "code", "execution_count": 135, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def log_max_inv(preds, mx = max_log_y):\n", " return np.exp(preds * mx)" ] }, { "cell_type": "code", "execution_count": 137, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def normalize_inv(preds):\n", " return preds * ystd + ymean" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we're ready to put together our models." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Much of the following code has commented out portions / alternate implementations." ] }, { "cell_type": "code", "execution_count": 739, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'\\n1 97s - loss: 0.0104 - val_loss: 0.0083\\n2 93s - loss: 0.0076 - val_loss: 0.0076\\n3 90s - loss: 0.0071 - val_loss: 0.0076\\n4 90s - loss: 0.0068 - val_loss: 0.0075\\n5 93s - loss: 0.0066 - val_loss: 0.0075\\n6 95s - loss: 0.0064 - val_loss: 0.0076\\n7 98s - loss: 0.0063 - val_loss: 0.0077\\n8 97s - loss: 0.0062 - val_loss: 0.0075\\n9 95s - loss: 0.0061 - val_loss: 0.0073\\n0 101s - loss: 0.0061 - val_loss: 0.0074\\n'" ] }, "execution_count": 739, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"\"\"\n", "1 97s - loss: 0.0104 - val_loss: 0.0083\n", "2 93s - loss: 0.0076 - val_loss: 0.0076\n", "3 90s - loss: 0.0071 - val_loss: 0.0076\n", "4 90s - loss: 0.0068 - val_loss: 0.0075\n", "5 93s - loss: 0.0066 - val_loss: 0.0075\n", "6 95s - loss: 0.0064 - val_loss: 0.0076\n", "7 98s - loss: 0.0063 - val_loss: 0.0077\n", "8 97s - loss: 0.0062 - val_loss: 0.0075\n", "9 95s - loss: 0.0061 - val_loss: 0.0073\n", "0 101s - loss: 0.0061 - val_loss: 0.0074\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 150, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def split_cols(arr): return np.hsplit(arr,arr.shape[1])" ] }, { "cell_type": "code", "execution_count": 193, "metadata": { "collapsed": true }, "outputs": [], "source": [ "map_train = split_cols(cat_map_train) + [contin_map_train]\n", "map_valid = split_cols(cat_map_valid) + [contin_map_valid]" ] }, { "cell_type": "code", "execution_count": 194, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "23" ] }, "execution_count": 194, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(map_train)" ] }, { "cell_type": "code", "execution_count": 191, "metadata": { "collapsed": true }, "outputs": [], "source": [ "map_train = split_cols(cat_map_train) + split_cols(contin_map_train)\n", "map_valid = split_cols(cat_map_valid) + split_cols(contin_map_valid)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Helper function for getting categorical name and dim." ] }, { "cell_type": "code", "execution_count": 173, "metadata": { "collapsed": true, "scrolled": true }, "outputs": [], "source": [ "def cat_map_info(feat): return feat[0], len(feat[1].classes_)" ] }, { "cell_type": "code", "execution_count": 174, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "('Day', 31)" ] }, "execution_count": 174, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cat_map_info(cat_map_fit.features[1])" ] }, { "cell_type": "code", "execution_count": 175, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def my_init(scale):\n", " return lambda shape, name=None: initializations.uniform(shape, scale=scale, name=name)" ] }, { "cell_type": "code", "execution_count": 176, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def emb_init(shape, name=None): \n", " return initializations.uniform(shape, scale=2/(shape[1]+1), name=name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Helper function for constructing embeddings. Notice commented out codes, several different ways to compute embeddings at play.\n", "\n", "Also, note we're flattening the embedding. Embeddings in Keras come out as an element of a sequence like we might use in a sequence of words; here we just want to concatenate them so we flatten the 1-vector sequence into a vector." ] }, { "cell_type": "code", "execution_count": 177, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def get_emb(feat):\n", " name, c = cat_map_info(feat)\n", " #c2 = cat_var_dict[name]\n", " c2 = (c+1)//2\n", " if c2>50: c2=50\n", " inp = Input((1,), dtype='int64', name=name+'_in')\n", " # , W_regularizer=l2(1e-6)\n", " u = Flatten(name=name+'_flt')(Embedding(c, c2, input_length=1, init=emb_init)(inp))\n", "# u = Flatten(name=name+'_flt')(Embedding(c, c2, input_length=1)(inp))\n", " return inp,u" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Helper function for continuous inputs." ] }, { "cell_type": "code", "execution_count": 178, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def get_contin(feat):\n", " name = feat[0][0]\n", " inp = Input((1,), name=name+'_in')\n", " return inp, Dense(1, name=name+'_d', init=my_init(1.))(inp)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's build them." ] }, { "cell_type": "code", "execution_count": 179, "metadata": { "collapsed": true }, "outputs": [], "source": [ "contin_inp = Input((contin_cols,), name='contin')\n", "contin_out = Dense(contin_cols*10, activation='relu', name='contin_d')(contin_inp)\n", "#contin_out = BatchNormalization()(contin_out)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can put them together. Given the inputs, continuous and categorical embeddings, we're going to concatenate all of them.\n", "\n", "Next, we're going to pass through some dropout, then two dense layers w/ ReLU activations, then dropout again, then the sigmoid activation we mentioned earlier." ] }, { "cell_type": "code", "execution_count": 180, "metadata": { "collapsed": true }, "outputs": [], "source": [ "embs = [get_emb(feat) for feat in cat_map_fit.features]\n", "#conts = [get_contin(feat) for feat in contin_map_fit.features]\n", "#contin_d = [d for inp,d in conts]\n", "x = merge([emb for inp,emb in embs] + [contin_out], mode='concat')\n", "#x = merge([emb for inp,emb in embs] + contin_d, mode='concat')\n", "\n", "x = Dropout(0.02)(x)\n", "x = Dense(1000, activation='relu', init='uniform')(x)\n", "x = Dense(500, activation='relu', init='uniform')(x)\n", "x = Dropout(0.2)(x)\n", "x = Dense(1, activation='sigmoid')(x)\n", "\n", "model = Model([inp for inp,emb in embs] + [contin_inp], x)\n", "#model = Model([inp for inp,emb in embs] + [inp for inp,d in conts], x)\n", "model.compile('adam', 'mean_absolute_error')\n", "#model.compile(Adam(), 'mse')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Start training" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time \n", "hist = model.fit(map_train, y_train, batch_size=128, nb_epoch=25,\n", " verbose=0, validation_data=(map_valid, y_valid))" ] }, { "cell_type": "code", "execution_count": 133, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "{'loss': [0.017763014663192959,\n", " 0.011433798792295986,\n", " 0.010516694221132331,\n", " 0.0098951324205597234,\n", " 0.0093967079764604566,\n", " 0.0090409253494607086,\n", " 0.0087176129839817689,\n", " 0.0083871925427847441,\n", " 0.0081409312105178832,\n", " 0.0079449998010363843],\n", " 'val_loss': [0.013183931291202704,\n", " 0.012660176647603511,\n", " 0.01246179387887319,\n", " 0.012995950489441554,\n", " 0.012145203327039878,\n", " 0.011319018259445826,\n", " 0.01145879974504312,\n", " 0.01087024861852328,\n", " 0.012571461255550384,\n", " 0.010759200275043646]}" ] }, "execution_count": 133, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hist.history" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "scrolled": false }, "outputs": [], "source": [ "plot_train(hist)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "scrolled": false }, "outputs": [], "source": [ "preds = np.squeeze(model.predict(map_valid, 1024))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Result on validation data: 0.1678 (samp 150k, 0.75 trn)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "scrolled": true }, "outputs": [], "source": [ "log_max_inv(preds)" ] }, { "cell_type": "code", "execution_count": 1056, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.11097331560266578" ] }, "execution_count": 1056, "metadata": {}, "output_type": "execute_result" } ], "source": [ "normalize_inv(preds)" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "## Using 3rd place data" ] }, { "cell_type": "code", "execution_count": 377, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "pkl_path = '/data/jhoward/github/entity-embedding-rossmann/'" ] }, { "cell_type": "code", "execution_count": 401, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "def load_pickle(fname): \n", " return pickle.load(open(pkl_path+fname + '.pickle', 'rb'))" ] }, { "cell_type": "code", "execution_count": 402, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "[x_pkl_orig, y_pkl_orig] = load_pickle('feature_train_data')" ] }, { "cell_type": "code", "execution_count": 403, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "max_log_y_pkl = np.max(np.log(y_pkl_orig))\n", "y_pkl = np.log(y_pkl_orig)/max_log_y_pkl" ] }, { "cell_type": "code", "execution_count": 404, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "pkl_vars = ['Open', 'Store', 'DayOfWeek', 'Promo', 'Year', 'Month', 'Day', \n", " 'StateHoliday', 'SchoolHoliday', 'CompetitionMonthsOpen', 'Promo2Weeks', \n", " 'Promo2Weeks_L', 'CompetitionDistance',\n", " 'StoreType', 'Assortment', 'PromoInterval', 'CompetitionOpenSinceYear',\n", " 'Promo2SinceYear', 'State', 'Week', 'Max_TemperatureC', 'Mean_TemperatureC', \n", " 'Min_TemperatureC', 'Max_Humidity', 'Mean_Humidity', 'Min_Humidity', 'Max_Wind_SpeedKm_h', \n", " 'Mean_Wind_SpeedKm_h', 'CloudCover','Events', 'Promo_fw', 'Promo_bw', \n", " 'StateHoliday_fw', 'StateHoliday_bw', 'AfterStateHoliday', 'BeforeStateHoliday', \n", " 'SchoolHoliday_fw', 'SchoolHoliday_bw', 'trend_DE', 'trend']" ] }, { "cell_type": "code", "execution_count": 405, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "x_pkl = np.array(x_pkl_orig)" ] }, { "cell_type": "code", "execution_count": 406, "metadata": { "hidden": true, "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "StandardScaler(copy=True, with_mean=True, with_std=True)" ] }, "execution_count": 406, "metadata": {}, "output_type": "execute_result" } ], "source": [ "gt_enc = StandardScaler()\n", "gt_enc.fit(x_pkl[:,-2:])" ] }, { "cell_type": "code", "execution_count": 407, "metadata": { "collapsed": true, "hidden": true, "scrolled": false }, "outputs": [], "source": [ "x_pkl[:,-2:] = gt_enc.transform(x_pkl[:,-2:])" ] }, { "cell_type": "code", "execution_count": 408, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "(844338, 40)" ] }, "execution_count": 408, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x_pkl.shape" ] }, { "cell_type": "code", "execution_count": 386, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "x_pkl = x_pkl[idxs]\n", "y_pkl = y_pkl[idxs]" ] }, { "cell_type": "code", "execution_count": 409, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "x_pkl_trn, x_pkl_val = x_pkl[:train_size], x_pkl[train_size:]\n", "y_pkl_trn, y_pkl_val = y_pkl[:train_size], y_pkl[train_size:]" ] }, { "cell_type": "code", "execution_count": 355, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "(90000, 40)" ] }, "execution_count": 355, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x_pkl_trn.shape" ] }, { "cell_type": "code", "execution_count": 179, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "xgb_parms = {'learning_rate': 0.1, 'subsample': 0.6, \n", " 'colsample_bylevel': 0.6, 'silent': True, 'objective': 'reg:linear'}" ] }, { "cell_type": "code", "execution_count": 180, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "xdata_pkl = xgboost.DMatrix(x_pkl_trn, y_pkl_trn, feature_names=pkl_vars)" ] }, { "cell_type": "code", "execution_count": 181, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "xdata_val_pkl = xgboost.DMatrix(x_pkl_val, y_pkl_val, feature_names=pkl_vars)" ] }, { "cell_type": "code", "execution_count": 182, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "xgb_parms['seed'] = random.randint(0,1e9)\n", "model_pkl = xgboost.train(xgb_parms, xdata_pkl)" ] }, { "cell_type": "code", "execution_count": 183, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "b'[0]\\teval-rmse:0.120945'" ] }, "execution_count": 183, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model_pkl.eval(xdata_val_pkl)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "#0.117473" ] }, { "cell_type": "code", "execution_count": 184, "metadata": { "collapsed": true, "hidden": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAANYCAYAAAA/k9DFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xl8XVW5//HPl6HEircIERCkZYYwXlpQ5hkZfiKIqORS\nBEQFvUyCFi7cUhFRkVlA5YJFIBCQQUAUEGqZFFEbBIRQGTowlKEMoQxlaJ/fH2sd2N096UnSTE2+\n79drv5Kz9tprP3snheesPHsdRQRmZmZmZta+xfo6ADMzMzOz/s5Js5mZmZlZDU6azczMzMxqcNJs\nZmZmZlaDk2YzMzMzsxqcNJuZmZmZ1eCk2czMzMysBifNZmZmZmY1OGk2MzMzM6vBSbOZmZmZWQ1O\nms3MuomkyyW9LWnNKvuOlzRX0h6l9iGSjpB0j6RXJL0j6VlJN0raT9Jihb4j8hjFrU3SA5L+u9i3\nr0j6lqQDO9G/fD2V7bkejPGTksZJ2qinzrEwJN0p6aG+jqOr+vv9NeuqJfo6ADOzAeQYYA/gl8DO\nlUZJqwFjgWsi4g+F9nrgVmAT4DbgFOAVYMV8/BXAGsCppfNcCVTGGZbPeR4wHDiuuy+qk74NvARc\n2olj/ghcVmp7u9simt9KwDhgCtAfk9Po6wAWUn+/v2Zd4qTZzKybRMRLko4D/k/SARFxed71c+Bd\n4OjSIU3AxsA+EXFjad9pkkYC61Q5VUtEXFl4/QtJ9wP/Rd8nzV3x79L19DT1yKDSUsC7EbGoJ71d\nImlx0l+we+T+mvW1Pv9TnpnZQBIRFwN/Bs6Q9HFJ+wG7AidGxIxKP0mbA58FLqySMFfGaomI5g6e\n+gXg/XKjpG9L+pek2bns43xJw6r0+5Kkf0h6S9JLudRkpVKfFSRdIunpPN5zkm6QNDzvnwKsD2xf\nKLP4UwfjXyBJ60i6VtLLuQTm75L2LPX5uKQzJD0kaVYuXflDsUxA0nbA30izub/OMc6R9NW8f6qk\n8VXOf2fxWiRtl4/9iqQfSnoGeBP4WN4/TNI5kqbne/W4pDGSupRQ5nP9TNK+kh7JP6e/SNog7z80\nn+NtSRMrP5NS/A9JGinpz/n4pyQdWuVcn5D0K0nP5/H+Wbk/hT6VUqFjJB0l6QlgNukvDQu6v1tL\n+o2kafm+TJd0lqS60vi/zj/DlfLv2CxJL0o6vXwPlRyVr+/t3O+W/Kaz2G904Xf8ZUnNkj7VlZ+H\nDU6eaTYz636HAi2kMo1tgL9FxM9LffYkJRZXdGH8oZKWy9//B6k8Y1fgR8VOkr4PnEQqf/g5adb6\n28CmkraKiDm530HAeOB+4HhgBdKs+JaSNomI1/OQ1wMNwM+AacDywC6kspDpwFHA+cAs4IekGccX\nOnA9dYXrqZgVEe/m+NYH7gWeAX5MSk6/DNwgqThLvzrweeAaUmnACqSfxZ2S1ouI54HWfE9+AFwI\n3JOP/Uv+2t4scXvtY4F3gNOBpYB3JX0EuBv4JOl34Glgyxz7iqQynq7YNl/fBfn1CcDNkn4KfCu3\nf5z014bxFEqEcvzLAr8HfkMq8fky6a8U70TErwFy8noX6V6eB0wFvkRKgIdFxHmlmL6Wr/vCfB9+\nS3rj0N79/RLwEdLv48vAp4EjgJWBr5TiXYxUtvRX4Nh8PccAT+SxK8YDB+Zru4iU22wDbE76d4ik\nE3NMV+U+nwCOBO4q/Y6btS8ivHnz5s1bN2+kOuS5pLKMjavsvw6YA3ys1L4UsFxhG1bYNyKPOSd/\nnVt4fX5pnHrSzN8fSu3fzv0PzK+XAJ4H/gkMKfTbI489Lr8ell8fU+O6Hwb+1In71N71fLXQ5w7g\nAWCJ0rH3Ao8VXi9ZZfzhpProEwtto/J5vlql/xRgfJX2icXrArbLYzxevG953/8CrwOrl9p/lH8f\nVq5xTyYCD1W5T28BqxTavpHbnwWGln735gDDS2POAY4q3i9SUjkDWDy3HZX77VfotzjprydtwEdL\nv4uvAsuWYl3Q/V2qSttxpL+SfKrQdkmO44RS30mkN6GV1zvkc521gPs5HHgPOK7Uvl7+eRzf2X/f\n3gbn5vIMM7OeMTN/fQ54pMr+/8hf3yi1H0Z6kK6y3cP8/o8067YzsA9phvEwSWcV+uxMSorOKR17\nEWkm+P/l15uSZox/HnlmFyDSA4uPFfq9TUowtpe0TJWYFsaNfHg9O5Nmr2+DVHJBSoyuAYZJWq6y\nkWbQ15L0yRzze5UBJS0maVlSojkZmOdP9d3o18X7lu1L+rm1leKdQHqTsm0Xz3VHRDxdeH1//npt\nRLxVpX310vHvk353gA/u14Wkn/+o3Lw78HxEXFXoN4f014WlSW8Wiq6NiFc6egER8U7le0mVv5jc\nR5pV3qTKIReWXt/DvNf1RVLS/IMFnPaLpL96XFP6ebxIetOzQ0fjt8HN5RlmZt1M0irAyaRZ1w2A\nMZRKJ0iJK6REZFah/dp8HMBZVH/25PGIKNYK35DLPI+S9KuIeIQ0Ewjw7+KBEfGepKcK+0eQ/hQ+\nT7/sMWCrfNy7Sg85ngG8IOmvwM3AZRHRkRKMBXmmdD1Fa5ISnlNIJR9lQUr6ZuRa16NJpQqrkWZI\nK31mVjm2O0yt0rYWsCHpTU9ZJd6ueLr0ui1/faZKu0ilGkXPRUR5VZJ/576rkmqRR5ASybLW3G9E\nqX1qraCL8r+NU0jlScX4gvTXjKLZEfFyqe3V0nGrk67rtQWcdk3Sv6MnquwL0ptBs5qcNJuZdb/z\nSf8z3h04GzhR0pURMbXQ5zFgL1JSfV+lMSKeJf25HUmvkko0OmICcDhpFrPazPZCi4hzJd0E7E2q\nof4B8D+SdoiIB3vinHz4puEM8uxzFZVkqFK3ejGpROIV0izkuXT8wff2apcXp8qDllRfGm8x4Hbg\nNKqvJFHtDUpHzOlke2+sYtHhpQGV1hG/A1iGVN89mVSfvjJpicLyz6i96+qsxUi/B7vlr2Xlv/aY\nVeWk2cysG0n6AmkW7aiIeE7S0aQE8wI+LHWANEt7PLA/haR5IVT+e750/jotf12HwmygpCVJs7C3\nF/op97uzNOY6hXEAiIgppDcCZ0taA3iQ9JBWZXWF7l5u7an89b0FzEZXfJFUd/zNYmMuJynO+i4o\nxldJSV3ZCODJGueveBJYOiImdrB/b1lJ0kdKs83rkO7HlPx6GmmWvKyhsL+W9u7vhqRZ+AMi4oMH\nYCXt3E7/jngS+KykZRYw2/wk6Xd8akRUm2026xDXNJuZdRNJS5NqPyeRZpuJtMzcWGA3SV+s9I2I\nv5AS129K+nx7Q3bi9J8nJSuVGd87SA8/HVnq93VSPfXN+fU/SLWdh+WEunItu5MSpZvz648orUNc\nNIVUWlJsf5PqSWeXRMRLpGT+UEkrlvcrfUBMxRxK90zSl0gzmUVv5q/V4nwS2FzSB5NKkj4HrNKJ\nsH8DbCHps1XiHaa0nnFfWIJUM1+JZUnS6iIvkVeZIH1ozoqSvlLotzhphYtZpJU1amnv/lZmjsu5\nx9F0/c3WdXm8cQvocz35odZqO3Ptu1lNnmk2M+s+p5KWFNsrIopJwAWkJbHOkXRrRFSSitHALcBv\nJd1KSnRf5cNPBNyGDz/5r2iUpP3z9x/jwwcC742IPwJExExJPwZOymPfBKxLqvf9G3mpu4h4P9cq\njwfultScz38kaZa38iDh2sAESb8BHiWVKuxDqs8triU9iZSAn0gqm3ixG2Zc/5v0ANjDki7Kca0A\nbEFKiCsPkN0MjFVaZ/kvpJnN/Zl/hvhJ4LUc5xukJO/+XD5zMelBvtvyta5B+jl1ZobydNKbmJsl\n/Zp0Tz4KbES6Z6uSSkd623PAGEmrkkpE9ssxfSM/7AfpQcFDSUvMbcqHS85tQfrryZvUVu3+/pVU\nkvQkcGZeH/l10l8HuvwmKyLulHQ5cKSktUmfsLkY6d/OnyLi5xHxlKT/BX6k9OmcN5DeAKxOKjW6\nkPT8gNmC9fXyHd68efM2EDbS6gzvAue0s39T0szv2aX2IaRZvHtJCfM7pJrmG0nr1qrQdwRptq64\nvUN6cOvHFJYdKxzzLVKN82xS0nQe8B9V+u1LmnV+iw8/BvuThf3LkmbRHyElO6+QEtN9SuMsT0rQ\nX8vxLXD5udzn3A7c31VJy5A9m69ler5He5fu5U9JD8a9QZoV/TTwJ2BCabzPkR64fIf5l7g7Oo//\nVh5jE9KSbRMKfbbLx+3TTrxDSQ8uTibV/b5ASvyPJi/vtoBrnQg8WOs+FX4fvlNqny+2POZD+Vr+\nTEpknwIOq3L+etKbhxdy7P8klVTUPHet+0sqB7mN9LDiC8AvSHX95Z/BJUBblXHHAe+X2kRav/mR\nHO/zpDdQ/1nqt3f+eb6et0dI9e5r9vV/P7wtGpsiBuWnfZqZmQ0akiYCy0XERjU7m1lVrmk2MzMz\nM6vBSbOZmZmZWQ1Oms3MzAYH12OaLQTXNJuZmZmZ1eCZZjMzMzOzGrxOs1k/IGk50qfGTSUtp2Vm\nZmYdU0dalvK2iHi5p07ipNmsf9iV/GETZmZm1iX7A1f21OBOms36h6kATU1NNDQ09HEoPes73/kO\nZ599dl+H0SsGy7X6OgcWX+fAMhius7W1ldGjR0P+f2lPcdJs1j/MBmhoaGDkyJF9HUuPGjZs2IC/\nxorBcq2+zoHF1zmwDJbrzHq0vNEPApqZmZmZ1eCZZrN+pLW1ta9D6HFtbW20tLT0dRi9YrBcq69z\nYPF1DiyV66yvr2f48OF9Hc4izes02wckjQCmAP8ZEQ8toN9E4IGIOKbXgpv3/FOAsyPiZ31x/p4g\naSQwqa/jMDOzgamubiiTJ7cOyMS5paWFUaNGAYyKiB57J+SZ5oUgaQXgf4E9gJWBF4AHgXMi4k99\nGVstki4BhkXEPoXm6cCKwMzcZztgIrBMRLxe6PcF4L1ujmccMI70iVVzgdeAR4HrgV9ExLuF7psC\nb3Zw3EUswT6F9Os0kN0K7NbXQfSSwXKtvs6Bxdc5sNwKjGD27NHMnDlzQCbNvcVJcxflWdm/AK8A\nxwL/ApYk/Qs8H1iv76Lrmkh/dnix0CRSEqtSv9d6KIR/ATsBiwPLAdsDY4EDJG0XEW/m8/fYGox9\nbzVgoD+wMdCvr2iwXKuvc2DxdQ4sI4GBX4bSG/wgYNf9ApgDbBYRN0TEExHRGhFnA5sDSFpF0o2S\nZklqk3S1pOUrA0gaJ+kBSQdLmpb7nS9pMUljJM2Q9IKkE4onljRX0mGS/iDpLUlPSvpiqc+n8vle\nlfSypBtyol+Z1T0Q2CuPNUfStpJG5Ncb5b6V2fJXc5/x+fiJks4qnGsZSZdJekXSmzmuNQv7D8xx\nfFbSo/k6b8kz9UXvR8RLEfF8RDwSERcA2wEbAMcVxpsi6cjC6+/n+zdb0rOSzqnECYwAzq5cZ25f\nVtKVkp7J8T4kab/S/Zso6VxJp+X7NyPft2KfYZIulPS8pLfzOHsU9m8t6e78M5qWxxuKmZmZLXKc\nNHeBpI+TPozi/IiYb3mTiHhdkoCbgGWAbYCdgdWBq0rd1yDNTu8K7Ad8Hfg9sBKwLSlZ/KGkzUrH\n/QC4BtiI9KEYV0laJ8e3BHAb0AZsBWwJzAJuzfvOAH5D+pvNCsAnSbPmkGaWIZVqVBLxtXKfo9q5\nJZeS3sp+jvSGQcAfJC1e6DOUNCO/f74fw3McCxQRk4FbgH2q7Ze0L3A08A1gTWAv4OG8ex/gGdJs\n9Yr5GiB9ctA/gN2B9YELgcskbVoa/qvAG8CngTHASZJ2yucV6f5tAfwX0AB8j/RGCklr5LivISX9\nXyH9LM6rdc1mZmbW/7g8o2vWJCWGkxfQZ2dSQrZqRDwHIOmrwCOSRkVE5aEvAQdHxFvAY3l2dO2I\n2D3vf1zSccAOwN8L4/8mIi7J358kaRfgCOBwUvKtiPhmpbOkQ4BXge0j4g5JbwNDIuKlQp9KPERE\nSHol73qpVNNM4Zi1gD2BLSLi/ty2P/A0sDdwXe66BHBoREzNfc4nJbMd8RiwSzv7VgFmABMiYg4p\nSf5HvoZX8+zyGxHxQdlJ/nmcVRjjAkm7AV+uHJs9FBGn5O+flHQ4qXxkQo5nU2DdiHgy95laOPZ4\noCkiKknyU5KOBu6U9K1SjbaZmZn1c06au0a1u7Au8HQlYQaIiFZJr5FmJStJ89ScMFe8ALxfGusF\nYPlS219Lr+8DNs7fbwSsJWlWqc9SpJntOzoQf0etS3oo8G+Vhoh4RdJk0nVWvFVJmLMZzH9N7anU\nVldzDWmmeYqkW4E/AL/LCXT1waTFgBOBL5Ee4BySt/LDheUVRIoxbww8U0iYyzYGNpQ0unQdkAqX\n23nDdSZwdamtMW9mZmaDW3NzM83NzfO0tbW19cq5nTR3zeOkJG5d4MaFHKu8CkW009aZUpqlSTOm\n/8X8Cf5L83fvFdWuqSNvPiAl31Oq7YiIZyStTZrZ3wW4APiepG0XkDiPIc3KH0V6+PBN4FxS4lwr\n5srP4e0aMS9NKvs4l/mvc3r7h1UqWMzMzKyssbGRxsZ5J5IKS871KNc0d0FEvEqqGf5vSR8p75c0\nDGgFVpG0cqF9PVKN8yPdEMbmVV5XPhmjhVSH/FJEPFXaKrPP75JWqViQSgnBgvq1kt58fabSIGk5\nYB264TolrUuq+b62vT4R8U5E/D4ijiaVsWwBbJh3V7vOLYEbI6I5Ih4mJeRrdzK0h4BPFR94LGkB\n1ouIKVV+BuW/JJiZmVk/56S56/6blIz9TdI+ktaUtG5e1eEvEXEHaRbzCkmbSPo06YG5iRHxQDec\n/0tKq26sJelkYDPSUneQHgycCdyYV3BYVdL2efWGlXKfqcBGktaWtFx+QLBsGml2dU9J9ZI+Wu4Q\nEU+QHni8SNJWkjYGmkg1zTd18pqWkLSCpE9K2kDSEcCdpAS06kODeWWOr0laX9JqwAHAWzn2ynVu\nK2mlnMxD+kvBLpK2kNRAmhEur+SxQBFxN3APcJ2knfM93k3SrrnLacCWks6TtHH+/dhLkh8ENDMz\nWwQ5ae6iiJhCWjFiIimhexj4I/BZoPJJeZ8nPXx3V973BOkhvU6frkrbuDzWg8BoYL+IeCzH9jZp\n5Y3ppAfxHgUuItU0Vx7ou4hUV/sP0trMW5bPleuxxwE/AZ6n/ZUfDiLVaP8O+DPpw0n+34Lqitux\nPvAcKeGdCOwLnApsW6r7Lt6P10grZ9xLuhc7Ap/Lfw0AOAlYFXiSD9eg/iEpEb+VtKzeDOC3pVg6\n8lGZ+5AezrySNKt+GvnfVJ7B3o404393Pt/3gWc7MK6ZmZn1M/4Y7UWQpLnA3hHR2Zlc66f0wcdo\nD4ZPBDQzs97VCoxm0qRJjBw58D7UxR+jbTYojaXjK/GZmZl1TF3dUOrr6/s6jEWak+ZFk/88MEA1\nNTXR0NBQu6OZmVkn1NfXM3z48L4OY5HmpHkRFBG1Vr2wRVRDQ8OA/NOZmZnZos4PApqZmZmZ1eCk\n2czMzMysBifNZmZmZmY1OGk2MzMzM6vBSbOZmZmZWQ1Oms3MzMzManDSbGZmZmZWg5NmMzMzM7Ma\n/OEm1i9IGgfsFRGD+pM9Wltb+zoEM7NBwZ+QZ53lpLkfknQJcCDp47LfB6YDlwGnRsTcvoytTNII\nYCywI7Ai8CxwBSnW9wr9vgCMARpIf+GYDvwxIo7JXU4HftaLcd8OzImI3Urt3wZOBdaPiOd6K56K\n0aNH9/YpzcwGpbq6oUye3OrE2TrMSXP/dQtwEFAH7A78HHgH+Gmxk6TFgIiI6O0As3UBAd8AngQ2\nAC4GhpKSZCTtBFwF/A/wO9KbgfWAXSqDRMRbwFu9GPfBwEOSvhERF+U4VwNOAw7tyYRZ0uIRMaf6\n3lOAPXrq1GZmBkArs2ePZubMmU6areMiwls/24BLgOtLbbcCfybNQL8K7Ak8ArwLDCclricBTwOz\ngQeAXQvHjwDmAl8C7iYlqH8D1gI2A/4OzAL+ACxXOG6B47YT/3eBJwqvzwYm1DhmHPBA6R78FjgW\neA6YCZwPLF7oM4SU5E7Psf0bOLiwf4N8PbOA50mz9cVr+yrwOjAiv/4TcE0prm2Be/P9mgqcBXyk\nNMY/8jlmAJcD9YX9O+X7viswifTGZ8sq1z8SCGgKCG/evHnz1qPbpABi0qRJYYu+SZPSzxMYGdFz\n+ZkfBFx0zCYlifDhLO4hwPrAi8DRwHeAY4ANgduAmyStURrn+8APgE1IpR9XAj8BjgC2BtbM+ys6\nOm7RMsArhdfPA+tLWr/GNUbp9Q7A6sD2pOT0oLxVXA58BTicNOP9deANAEnDgAmkRHUkKWldHrj6\ng5NFXAbcAVwi6XDS7Pdhlf2S1gZ+DzST7nNjjuWcQgxLACeQ7s3ewBqkmfayH5HeTKxLerNjZmZm\ni5KezMi9dW2jNNMM7Ay8TUpuDwTmABuUjnkGOK7Udj9wXv5+BGnG86DC/q/ksbYrtB0HPNrRcavE\nvibwGvC1QttQUlnGHGAKKQk9GBhS6DMOaCndg6cAFdquBq7M36+dr2eHduI4Ebil1PapfMyahbZP\nkN50vAfsWeXncF6pbfvcd4l2zrt5vs6l8uvKTPNuNX7mnmn25s2bt17bPNM8kHim2faUNEvSbD6c\n7Tw573s3Iv5V6SjpY8BKwF9KY/yZ9OBd0cOF71/IX/9Valu+C+MiaWVSLfbVETG+0h4Rb0XEnqSE\n+hRSKcOZwN8k1c1/6R94JCKi8HpGJTZgY9JM+d3tHLsxsGO+h7MkzQJaSf+oPpglj4iXgAuB1oj4\nXZUxvl4a42ZSycqIfM2bSfqdpGmSXifNXAOsUhgnSDPeZmZmtojyg4D9159IpQLvAc9FXjVDEqRZ\n5656r/B9tNPW6TdTklYixXxvRBxarU9ETCHNNI+XdCrwOGm2+9IOxFqOrdY9WBq4iVTGotK+GaXX\n7+et2hgX5K08xvT8puLWfJ7/Is1Yr0lKrIeU+r9ZI97sTAoVJFlj3szMzAa35uZmmpub52lra2vr\nlXM7ae6/3sxJZk0RMUvSc8BWwD2FXVuRSik+6NqZADo6bp5h/hPpYcKvdXD46aSH6z7amZgKHiYl\n0Nvlc5e1APsA06Lry/S1kJaeq/pzkNRAqt8+PiJeyG1bdfFc2bHA/gs3hJmZ2QDV2NhIY+O8E0kt\nLS2MGjWqx8/tpHngOB34vqSngH+SkteNSTOgFeXZ0vbaOjxunmG+kzSDPAZYPs+GU0gkx5Hqmv8A\nTCMlmkeRfv9u79xlUhl7mqTLSLPWRwEPkkomlo+Ia0izw18HrpL0U9KDiWuRZrYPKZV9tOfHwH2S\nzgV+RUryNyDVUR+Vr+U94ChJF+X78j9duR4zMzPr35w0Dxw/A/4DOINU9/so6cG2Jwt9qiWKtZLH\nWuPuQlrhYnXSsnSQEvEAFs+v7wK+TSrDWIG0ZN4DwGcj4vEOXl81h5FWpbgAWI40e/0jgIiYkWd9\nTyOt+LEUKcm9tYMJMxHxoKTtgB+Slp0L0lrUzXn/C5K+lvcfTVp67ljgxoW4JjMzM+uH1MH8wcx6\nkKSRwCRowuUZZmY9rQUYxaRJkxg5cmRfB2MLqVCeMSoiWnrqPJ5pNutXppD+Y25mZj2nta8DsEWQ\nk2azfmVs3szMrCfV1Q2lvr6+r8OwRYiTZrN+pKmpiYaG+ZbANjOzblZfX8/w4cP7OgxbhDhpNutH\nGhoaXF9nZmbWD/kTAc3MzMzManDSbGZmZmZWg5NmMzMzM7ManDSbmZmZmdXgpNnMzMzMrAYnzWZm\nZmZmNThpNjMzMzOrwes0m/Ujra3+aFczs67yB5ZYT3LSbNaPjB49uq9DMDNbZNXVDWXy5FYnztYj\nnDTboCGpHjgF2ANYAXgVeBA4OSLukzQX2DsiburkuFOAsyPiZwsfZSU8MzPrnFZmzx7NzJkznTRb\nj3DSbIPJ9aTf+QOAKaTEeSdgub4Mal6rAf4YbTMzs/7GDwLaoCBpGLA1cFxE3B0RT0fEPyLitIi4\nOc8WB3CDpLmSnsrHrS7pBknPS5ol6W+SdiqMOxEYAZydj5tT2Le1pLslvSVpmqRzJQ3t3Ss3MzOz\n7uCk2QaLN/K2t6QhVfZvBgg4EFgxvwZYGvg9sAPwn8AtwE2SPpX37wM8A4zNx30SQNIaue81wAbA\nV4CtgPO6+8LMzMys5zlptkEhIuaQEuIDgdck3SvpVEkb5v0zc9e2iHgxIl7O7Q9FxEUR0RoRT0bE\nOOAp4PN5/6vAHOCNfNyLeZzjgaaIOC8inoqIvwJHAwe2k7SbmZlZP+aaZhs0IuK3kn4PbANsDuwO\njJF0SERcVu0YSR8FTiY9nfdJ0r+ZOqDWUyYbAxtKKi6Hofx1NWBy9cPOBK4utTXmzczMbHBrbm6m\nubl5nra2trZeObeTZhtUIuJdYELeTpV0ESkprpo0k7LYnYBjgSeBt4HrgFqzxUsDFwLn8mGyXDG9\n/cOOBfavMbSZmdng1NjYSGPjvBNJLS0tjBo1qsfP7aTZBrtWYK/8/XvA4qX9WwK/rixDJ2lpYNVS\nn3erHNcCrBcRU7o1WjMzM+sTrmm2QUHSspImSNpf0oaSVpX0JeB7wA2521RgJ0krSFomtz0O7CNp\nY0kbA1cw/8zxVGBbSStJqixfdxqwpaTz8rFrStpLkh8ENDMzWwQ5abbB4g2g8jDeXcDDpLKMC4Ej\ncp9jgV1I5RMtue0Y0oeg/Bm4Ebi1sK/iJNLs85PAiwAR8TCwHbAWcHc+5vvAs918XWZmZtYLXJ5h\ng0KuZT4xb+31uRm4udQ2Ddi51PUXpT73A5tUGW8SsFvnIp3C/Dm5mZnV1trXAdgA56TZrF8Zmzcz\nM+usurqh1NfX93UYNkA5aTbrR5qammhoaOjrMMzMFkn19fUMH15rRVCzrnHSbNaPNDQ0MHLkyL4O\nw8zMzEr8IKCZmZmZWQ1Oms3MzMzManDSbGZmZmZWg5NmMzMzM7ManDSbmZmZmdXgpNnMzMzMrAYn\nzWZmZmZmNThpNjMzMzOrwR9uYtaPtLa29nUIZpb50+XMrMhJsy00SZsD9wK3RMSefRzLROCBiDim\nL+OokDQv+7fcAAAgAElEQVQO2DsiNulI/9GjR/dwRGbWUXV1Q5k8udWJs5kBTpqtexwC/Aw4RNKK\nEfF8bwcgacmIeK+3z9tB0fGupwB79FggZtZRrcyePZqZM2c6aTYzwEmzLSRJHwW+AowCVgQOAn6S\n9y0DXADsAiwNPA38KCIulbQkcDawD/Bx4HnglxFxWj52FeB8YEdgLnArcEREvJj3jwP2zn1OBIZL\nuhzYDthW0tGkZHW1vE0EdsuxrQv8BWgENgXOBFYGbgYOiYjZ+RwCjge+ka9tMvDDiLgu798uj7sz\ncBqwHvBP4KCIeFzSgcA4ICTNzfEcHBGXtX9HVwNGdvDum5mZWW9x0mwL6ytAa04SrwDOISfNwA9J\nCequwMvAmsBH8r6jgM8B+5KS6VXyVklWbwJeB7YBlgR+DlxFSqIr1iQl3V8A5gDTgLWBh4GxgICX\nSJkopAT228DbwDXAb4DZwH7Ax4AbgCOA03P/E4D/Ar4JPAFsC1wu6cWIuKcQxw+B7wAzgQuB8Tnu\nq4EN8vXvlONpq31LzczMrL9x0mwL62vA5fn7W4H/kLRtRNxNSoIfiIgH8v7pheNWAR6PiL/k108X\n9u0MrA+sGhHPAUj6KvCIpFERMSn3WxI4ICJeqRwo6V3grYh4qdAGaZb3xIj4a277FfAjYPWImJbb\nrgV2AE6XNAT4H2CniLg/DzVV0jbAoUAlaQ7ghIi4N4/xE+BmSUMiYrakN4D3i/GYmZnZosdLzlmX\nSVoH+DRpBpiImEOavT0kd/kF0CjpAUmnSdqicPivgU0kTZZ0rqRdCvvWBZ6uJMx57FbgNaCh0G9a\nMWHugIcL379ASq6nldqWz9+vCQwFbpc0q7IBBwCrL2DcGfnr8piZmdmA4ZlmWxiHAIsDM/JsbsU7\nkg6PiFslDSc92bYLcIekCyJiTEQ8IGlVYHfSzPJvJN0eEV/uxPnf7GS8xQcFo/S60lZ5I7l0/roH\n8Fyp3zs1xoUuvyE9k1TVUdSYNzMzs8GtubmZ5ubmedra2nqn8tFJs3WJpMVJs67HALeXdt9AyvL+\nLyJeJpVvXC7pXuCnwBiAiHiDVFt8jaTrgFvyw4OtwCqSVo6IZ/P51gOWAR6pEdq7pER+YT1KSo5H\nVEovuqiT8RwL7L8QpzMzMxu4GhsbaWycdyKppaWFUaNG9fi5nTRbV+1JSmLHR8Ss4g5J1wNfl7Qy\nMImU6NaRHvx7NPf5DqmU4QHS7OyXgecj4jXSjPS/gCtyvyVJq3BMLNRHt2cq8BlJI4A3gEr5hto9\nooqIeEPSGcDZ+Q3CvcAwYCugLSIqddzVxi22TQVWk7Qx8AwwKyLe7UwsZmZm1vdc02xd9TXg9nLC\nnF1HWoLuPeDHwIPAncD7fFhnMIs04/x34H6gUsZR8XngVeAu4I+k1Sv260BcZ5BW0ngUeJG8Iged\nWis5HxAxlrRw8vF5vFtyjFOK3aodWvj+OtIDkhNzPB25BjMzM+tnFNHpXMLMupmkkcAkaMLlGWb9\nQQswikmTJjFypNdON+vPCuUZoyKipafO4/IMs35lCul/1mbWt1r7OgAz62ecNJv1K2PzZmZ9ra5u\nKPX19X0dhpn1E06azfqRpqYmGhoaanc0sx5XX1/P8OHD+zoMM+snnDSb9SMNDQ2unzQzM+uHvHqG\nmZmZmVkNTprNzMzMzGpw0mxmZmZmVoOTZjMzMzOzGpw0m5mZmZnV4KTZzMzMzKwGJ81mZmZmZjV4\nnWazfqS11R/da9aT/IElZtZVTprN+pHRo0f3dQhmA1pd3VAmT2514mxmneak2QYNSfXAKcAewArA\nq8CDwMkRcZ+kucDeEXFT30VZCc/Mul8rs2ePZubMmU6azazTnDTbYHI96Xf+AGAKKXHeCViuO08i\naYmIeL9rR68G+GO0zczM+hs/CGiDgqRhwNbAcRFxd0Q8HRH/iIjTIuJmSVOAAG6QNFfSU4VjvyXp\nCUnvSGqVNLo09lxJh0m6UdIbwAm5fQNJf5A0S9Lzki6T1K0JupmZmfUOJ802WLyRt70lDamyfzNA\nwIHAivk1kr4AnAOcDqwP/B9wiaTtSsePI81kbwCMz0n6BGASaep4V2B54OruvSwzMzPrDS7PsEEh\nIuZIOhC4CPiWpBbgLuCqiHg4ImZKAmiLiBcLhx4LjI+IC/PrsyVtDnw3H19xRURcWnkh6USgJSLG\nFtq+DkyXtGZEPNET12lmZmY9w0mzDRoR8VtJvwe2ATYHdgfGSDokIi5r57AG4MJS25+BI0ttk0qv\nNwZ2lDSrHAawBtBO0nwm809GN+bNzMxscGtubqa5uXmetra2tl45t5NmG1Qi4l1S2cQE4FRJFwEn\nA+0lzR31Zun10sBNwBhS2UfRjPaHORbYfyFDMTMzG5gaGxtpbJx3IqmlpYVRo0b1+Lld02yDXSvw\n0fz9e8DiVfZvVWrbCni0xrgtpBroaRHxVGl7e2GDNjMzs97lpNkGBUnLSpogaX9JG0paVdKXgO8B\nN+RuU4GdJK0gaZncdjpwUF4dY01JxwBfyO0LcgGwLHCVpE0lrS5pV0njlYunzczMbNHhpNkGizeA\nvwJHkx7ge5hUlnEhcETucyywCzCdNFNMRNwIHJX3/Qv4BnBQRNxTGDvKJ4uIGaQZ6cWA24CHgLOA\nVyNivv5mZmbWv7mm2QaFXMt8Yt7a63MzcHOV9guZ/2HA4v5ySUel/Ulg385FOoWcr5tZt2vt6wDM\nbBHmpNmsXxmbNzPrCXV1Q6mvr+/rMMxsEeSk2awfaWpqoqGhoa/DMBuw6uvrGT58eF+HYWaLICfN\nZv1IQ0MDI0eO7OswzMzMrMQPApqZmZmZ1eCk2czMzMysBifNZmZmZmY1OGk2MzMzM6vBSbOZmZmZ\nWQ1Oms3MzMzManDSbGZmZmZWg5NmMzMzM7Ma/OEmNmBI2hL4JbAucHNE7NOHsYwD9o6ITTpzXGtr\naw9FZNa7/Ml7ZjbQOGm2TpN0CXAgEMAc4BXgIaAZ+HVERDef73PAd4GRwOLAI8AFEXFpqetZwAPA\nrsBbkp4DzomInxbG+gkwBtg+Iu4utN8JTIuIA7sx9E7fh9GjR3fj6c36Tl3dUCZPbnXibGYDhpNm\n66pbgINIv0MrALsB5wJflPT5iJjbHSeRdARwNvBj4DDgXWAv4JeS1o+IMYXuawC/iIgZ+dg7ge2B\nnxb6bA9Mz1/vzv2WAj4DjO+OmBfOKcAefR2E2UJqZfbs0cycOdNJs5kNGE6araveiYiX8vczgH9K\nuh+YQEqmx0v6DnAwsDppNvp3wJiIeFPS0HzcwRFxfWVQSXsDTaREfFngDOCsiBhbOPfZkt4Dfibp\nGuBFYAppZvcSSePzeScCZ0haLCLmSloa2AQ4Gvgy8IM83pbAEODOQhx7AScB6wHPApcBP6y8GZA0\nDDgT+DywFPB34JiIeKjazZK0BvBH4PcRcWT7t3U10oS6mZmZ9Sd+ENC6TURMBB4EKrXEc4AjSInn\nV4EdgNNy37eAq0jJbdFBwG8i4k1gX9IbuzOrnO5C4A2gkTRz/ElgFnBk/v5qUtL8MWCzfMw2wGTg\neuAzkobk9u2BqRExHUDSNsClpBnudYFDSeUoJxbOfy2wHKkUZCTQAtwhaZlyoJI2Au4BmhacMJuZ\nmVl/5aTZuttjwKoAEfGziLgrIqZHxJ3AWNIMb8XFwK6SVgCQ9AlSbUKlTGItoC0iXiifJCLeA54C\n1o7kBdJM8+sR8WJEvBMRT5BmibfPh20P3JX7Tge2KLRPLAx/EvDjiGiKiGkRMSG3HZbj3BrYFPhy\nRDwQEU/mMpE2UqL/AUlb5LF/GhHjOnIDzczMrP9x0mzdTeQH4CTtLOkOSc9Ieh24HFhOUh1ARPwd\neJQ0iwtwAGnG995ujOdO5k2a78zf3wVsn2P5DPMmzRsDJ0maVdmAi4AVcv+NSDPYr5T6rEqqq64Y\nAdwOnBwR53TjNZmZmVkvc02zdbcGYIqkEaQa5guAE0g1zduQZpeHALNz/4uBb5Me1juIeR/G+zcw\nTNKKEfF88SSSliQlqH+qEc9E4BxJy5Lqme/K7XcB3ySVTSxZGmdp0szy9czvnbz/OWA70puEotcK\n37+Y+zVKuiQiZtWIlVSJcnWprTFvZmZmg1tzczPNzc3ztLW1tfXKuZ00W7eRtCOwISnzGwUoIr5b\n2L9flcOagNPyKhkNpAfuKq4j1UAfC3yvdNy3gKGkZe4WZCIpyT0G+HdEzMztdwO/AnYHHq+suJG1\nAOtExFPtXGcLsCIwp1IH3Y63gc+RVhq5TdIuuVZ7AY4F9q9xSWZmZoNTY2MjjY3zTiS1tLQwatSo\nHj+3k2brqqVyLfLipJUudgeOB24ilWFsCCwp6UjSjPPWpAfq5hERr0n6LXA6cFtEPFfY97SkMaQV\nMN7J474H7A2cCpyRSzzaFRFTJE0nPZDYVGh/Jq/j/E3gytJhPwB+J+lp0gN/c0klGxtExNiIuEPS\nfcANko4jzYivTKrHvj4iWgrneVvS/yMlzrdK2q124mxmZmb9jWuarat2I5UeTCElhNsBh0fE3vnB\nvIdIs7tjgIdJ9QXHtzPWr0glG/OtkxwR5wJfICXdf89j7QccGhHHlbu3M35ltnliqf2u3D5PiUdE\n/JE0Q7wL8DfgPtIydVML3fYgzVaPJ63IcSUwHKj20OKbpDcVADdL+kg7cZqZmVk/pW7+8DazTpN0\nAKmkY6WIeL+v4+kLkkYCk9JkuMszbFHXAoxi0qRJjBzpdcfNrGcVyjNGFf/a291cnmF9Js+4rgQc\nB/xysCbM85pCSjjMFmWtfR2AmVm3c9JsfWkM6QND7gR+0reh9Bdj82a2aKurG0p9fX1fh2Fm1m2c\nNFufiYiTgZP7Oo7+pKmpiYaGhr4Ow2yh1dfXM3z48L4Ow8ys2zhpNutHGhoaXANqZmbWD3n1DDMz\nMzOzGpw0m5mZmZnV4KTZzMzMzKwGJ81mZmZmZjU4aTYzMzMzq8FJs5mZmZlZDU6azczMzMxq8DrN\nZv1Ia6s/fth6lz+ExMysY5w0m/UASXOBvSPips4cN3r06B6KyKy6urqhTJ7c6sTZzKwGJ802YEk6\nFDgdWCYi5ua2jwKvAfdExI6FvtsDfwLWiIgpfRBudgqwR9+d3gaZVmbPHs3MmTOdNJuZ1eCk2Qay\nicBHgU2Bv+W2bYAZwGckDYmId3P79sC0vk2YAVYD/DHaZmZm/Y0fBLQBKyL+DTxPSogrtgduAKYA\nm5faJwJIGibpYkkvSmqTdIekjYpjS9pL0iRJb0t6QtJJkhZvLxZJJ0t6VtIG3XJxZmZm1qucNNtA\nNxHYofB6B+BO4K5Ku6Q64DOk8gyAa4HlgF1J074twB2Slsn9twEuBc4G1gUOBQ4ETqgWgKTzgNHA\n1hHxr+67NDMzM+stTpptoJsIbCVpMUkfA/6TlDDfw4cz0FsCQ4A7JW1NKuf4ckQ8EBFPRsQYoA3Y\nN/c/CfhxRDRFxLSImJDbDiude0lJV5CS8636vvTDzMzMuso1zTbQ3Umqa94MWBb4d0S8LOkuYLyk\nIaTk+amIeEbS54GPAa9IKo5TB6yev98Y2FLS/xb2Lw4MkVQXEbNz29nAbGDziHilY+GeCVxdamvM\nm5mZ2eDW3NxMc3PzPG1tbW29cm4nzTagRcSTkp4lzfYuS5plJiJmSHoa2IqUNFdKM5YGngO2A1Qa\n7rVCn5OA66ucb3bh5R9J2e5uwJUdi/hYYP+OdTUzMxtkGhsbaWycdyKppaWFUaNG9fi5nTTbYFCp\na/448NNC+93A7sCngZ/nthZgRWBORExvZ7wWYJ2IeKrGeW8Cfgc0S5oTEeUpZDMzM1tEOGm2wWAi\ncAHp9/2uQvvdwPnAkrkPEXGHpPuAGyQdB/wbWJm0ePL1EdEC/AD4XZ6pvhaYSyrZ2CAixhZPHBE3\nSjoAuEzS+xFxXQ9ep5mZmfUQJ802GEwk1SS3RsRLhfa7SKUWj0XEC4X2PYBTgfHAJ0jL1t0NvAAQ\nEX+U9DlSicYY4D3gMeDiwhjxwTcR10lajJQ4z4mIG7r5+szMzKyHOWm2AS8ippEe1Cu3T2+n/U3g\n6Ly1N+btwO0L2L946fU1wDW1o51Cqv4w6w2tfR2Amdkiw0mzWb8yNm9mvaOubij19fV9HYaZWb/n\npNmsH2lqaqKhoaGvw7BBpL6+nuHDh/d1GGZm/Z6TZrN+pKGhgZEjR/Z1GGZmZlbiTwQ0MzMzM6vB\nSbOZmZmZWQ1Oms3MzMzManDSbGZmZmZWg5NmMzMzM7ManDSbmZmZmdXgpNnMzMzMrAYnzWZmZmZm\nNfjDTcz6kdbW1r4OwTrBn6ZnZjZ4OGk2K5B0CXAgEMAc4BXgIaAZ+HVERE+ef/To0T05vHWzurqh\nTJ7c6sTZzGwQcNJsNr9bgINI/z5WAHYDzgW+KOnzETG35059CrBHzw1v3aiV2bNHM3PmTCfNZmaD\ngJNms/m9ExEv5e9nAP+UdD8wgZRMj5f0HeBgYHXSbPTvgDER8aakofm4gyPi+sqgkvYGmoAVIuLN\n6qdeDRjZE9dkZmZmC8EPApp1QERMBB4E9slNc4AjgPWArwI7AKflvm8BV5GS6qKDgN+0nzCbmZlZ\nf+WZZrOOewzYECAiflZony5pLPAL4PDcdjHwZ0krRMQLkj5BqrvYsTcDNjMzs+7hmWazjhPpAUEk\n7SzpDknPSHoduBxYTlIdQET8HXiU9FAhwAHA1Ii4tw/iNjMzs4XkmWazjmsApkgaQaphvgA4gVTT\nvA1pdnkIMDv3vxj4NvBTci107VOcCVxdamvMm5mZ2eDW3NxMc3PzPG1tbW29cm4nzWYdIGlHUmnG\nmcAoQBHx3cL+/aoc1gScJukIUsJ9We0zHQvs3w0Rm5mZDTyNjY00Ns47kdTS0sKoUaN6/NxOms3m\nt5SkFYDFSUvO7Q4cD9xEKsPYEFhS0pGkGeetgUPLg0TEa5J+C5wO3BYRz/VS/GZmZtbNXNNsNr/d\ngOeAKaQ1m7cDDo+IvSN5CDgGGAM8TKqdOL6dsX5FKtnoQGmGmZmZ9VeeaTYriIiDmX+puGr9ziV9\n4EnRFVW6fgqYSZqlNjMzs0WUk2azHiDpI8BKwHHALyPi/Y4dOQVo6bnArBu19nUAZmbWi5w0m/WM\nMcCJwJ3ATzp+2Ni82aKgrm4o9fX1fR2GmZn1AifNZj0gIk4GTu7scU1NTTQ0NPRARNYT6uvrGT58\neF+HYWZmvcBJs1k/0tDQwMiRI/s6DDMzMyvx6hlmZmZmZjU4aTYzMzMzq8FJs5mZmZlZDU6azczM\nzMxqcNJsZmZmZlaDk2YzMzMzsxqcNJuZmZmZ1eB1ms36kdZWfzRzZ/jDRczMrLc4aTbrR0aPHt3X\nISxS6uqGMnlyqxNnMzPrcU6arcdJqgdOAfYAVgBeBR4ETo6I+yTNBfaOiJt6IZa5QACqsjtyTD/o\n6TjaV7lNVlsrs2ePZubMmU6azcysxzlptt5wPel37QBgCilx3glYrjtPImmJiHi/RrcVC9/vB5wM\nrM2HSfQb3RlT560G+GO0zczM+hs/CGg9StIwYGvguIi4OyKejoh/RMRpEXGzpCmkGd4bJM2V9FTh\n2G9JekLSO5JaJY0ujT1X0mGSbpT0BnBCbt9A0h8kzZL0vKTLJC0HEBEvVjagLTXFS4W2tyVNl/Tt\n0rk2l/S+pJUlLZXP/XVJf5T0lqR/S9qzdMwISddKek3STEnXSfpU999lMzMz62lOmq2nvZG3vSUN\nqbJ/M9Is74GkWeDNACR9ATgHOB1YH/g/4BJJ25WOH0eayd4AGJ+T9AnAJNKU7a7A8sDVHQk2IgK4\nBDi4tOsg4PaIeLbQ9kPgcmDjHMO1klbN8Q8B7gCeB7YAtgHeA34vqVppiJmZmfVjTpqtR0XEHFJC\nfCDwmqR7JZ0qacO8f2bu2pZne1/Or48FxkfEhRHxREScTUpMv1s6xRURcWlETI2IZ4DDgZaIGBsR\nj0fEg8DXgR0lrdnBsMcD/ylpAwBJdcBXgF+V+jVFxOX5PMcDj+bzA3wVmBURh0dEa0S0khLxdYGt\nOhiHmZmZ9ROuabYeFxG/lfR70mzr5sDuwBhJh0TEZe0c1gBcWGr7M3BkqW1S6fXGpAR5VjkMYA3g\niQ7EO03SBOBrwDHAPsD7wI2lrn8tvb4vxw2wEbBBlTgWz3HcW/3sZzL/pHhj3szMzAa35uZmmpub\n52lra2vrlXM7abZeERHvksomJgCnSrqI9BBee0lzR71Zer00cBMwhvlXyJjRiXEvBi6QNIZUmnFl\nRLzXieOXBv5Cml0ux/Fi+4cdC+zfidOYmZkNHo2NjTQ2zjuR1NLSwqhRo3r83C7PsL7SCnw0f/8e\naQa2vL9cxrAVqQRiQVpINdDTIuKp0vZ2J+K7kTQ7/d/ADqSSjbLNq7yuxNcCrAM8XyWOPl6hw8zM\nzDrLSbP1KEnLSpogaX9JG0paVdKXgO8BN+RuU4GdJK0gaZncdjpwUF4dY01JxwBfyO0LcgGwLHCV\npE0lrS5pV0njO/MAXp5VbgJOAx7KtdFl+0s6QNJakn5MStZ/kfddSpoFv0HSlvm6d5R0fl632szM\nzBYhTpqtp71Bqv09GrgLeJhUlnEhcETucyywCzCdNENLRNwIHJX3/Qv4BnBQRNxTGDvKJ4uIGaQZ\n6cWA24CHgLOAV/PKGJ3xK2AI8z8AWDGWVLrxIPBFYN+IeCrHMYtUw/0C6c3Bo8AvSaUa5ZISMzMz\n6+dc02w9Ktcyn5i39vrcDNxcpf1C5n8YsLi/XNJRaX8S2LcDsV1KmhFuz6eA2cCV7eyfHhE7LWD8\nGaRVNDphCvl9g9XU2tcBmJnZ/2fvzuPrqsr9j3++gphbEVAig0oRGSTIIAmoDAIiyhVFpp9iIFgG\n5YJe5isyWGYUZR4cELVQAwG5oKBMAhZQQLk0jBJrgTIoYClCKUMp0Of3x1qn7OymOUma5Jwk3/fr\ndV45Z+211372Sfrqc9Z59tpjiJNmsxJJ7yDdtfAY0rJyLwzf0Sfmh/VFQ8M4Ghtd7WJmZkPPSbPZ\nwvYk1UbfxaJnyPtb6tEn7e3tNDU1Ve9oADQ2NjJ+/Phah2FmZmOAk2azkj6UhbzGwqt9DIqmpiaa\nm5uHYmgzMzNbDL4Q0MzMzMysCifNZmZmZmZVOGk2MzMzM6vCSbOZmZmZWRVOms3MzMzMqnDSbGZm\nZmZWhZNmMzMzM7MqnDSbmZmZmVXhm5vYmCVpS2AKsFxEvFjreAC6urpqHcKI4jsCmpnZcHHSbDUj\naQpwT0QcWsMw+nQ77EKCXek/B3gUuBE4MyKeKfQ9Fjg291VhmL9FxDq9Haetra3vkRsNDeOYNq3L\nibOZmQ05J81W1yQtERFv1jqOLIC1SAnzMkAz8G1gH0lbRsRfC30fBD5N96T5jeqHOBHYbpDCHe26\nmDu3jVmzZjlpNjOzIeek2WpC0iRgS2ALSQeTEtK9gUmkrPEkYF3gs8BtknYAjgHWAf4JTAZOriTU\nkuYDXwc+D2yb+xwWEb8tHHM74ExgFeDOPEZ/PZtLOWYCD0u6GrgH+DGwRaHfGxHxbP+HX42Ui5uZ\nmVk98YWAVisHkRLXC4AVgZWBJ/O275FmcJuA+yV9EriIlPCuDfwXMAE4qjTmMcClwHrAtcDFkpYD\nkPQB4ArgKmAD4GfAKYt7EhExF/gJsJmkxsUdz8zMzOqTk2ariTxbOw94JSKejYiZQKUMY2JE3BwR\nMyLiBVIy/L2IaI+IxyPi5ty2X2nYSRHxq4h4lJRQLw18LG/7BvBwRBweEdMjogO4cJBO52/55wcL\nbetLmlN4vCjpR4N0PDMzMxtmLs+wehPA1FLbBsCmkr5TaFsCWEpSQ57tBXhgwSARr0h6EVghN60N\n/KU07p2DFHOlbrl4UeHfgO3pXtNcFyt0mJmZWf85abZ69HLp9dKkmeUryx0LCTPA6+XNDM+3KZUV\nMR4rtM2LiBn9H+p04LJSW2t+mJmZjW0dHR10dHR0a5s9e/awHNtJs9XSPNKMcTWdwIdz2cVAdZFm\nfos2WYzxAJD0H6QLEG+NiOcWdzw4DNh98YcxMzMbhVpbW2lt7T6R1NnZSUtLy5Af20mz1dJjwMcl\nrQq8RJoVVg/9TgB+K+lJ4H+B+aSSjXUjYmIfj/UT4FBJPyBdBLgR6WLC/hCwYk6U35XH+BawPLBj\nqe+SklYstUWu3TYzM7MRxhcCWi2dRrr47yHSEm7j6eFmIxHxe+ALwGeAu0i1yAfTvRyip5uULGiL\niCeBXYAdgHuBfYEj+xlvkGqV/wncDRwO/J6UvE8r9f0I8FTh8XQpXjMzMxtBPNNsNRMR04HNSs0X\nLaLvjaS77y1qrIXKPCLiPaXX15KWoqt6vB7GupW+lZIQEccDx/el78JmkKpRrDrfctzMzIaPk2az\nujIxP6wvGhrG0djo5bHNzGzoOWk2AyRdC3yyh00BfDciFvtGKH3R3t5OU1PTcBxqVGhsbPQttM3M\nbFg4aTZL9gH+YxHb/j1cQTQ1NdHc7Ntom5mZ1RsnzWZARDxd6xjMzMysfnn1DDMzMzOzKpw0m5mZ\nmZlV4aTZzMzMzKwKJ81mZmZmZlU4aTYzMzMzq8JJs5mZmZlZFU6azczMzMyq8DrNZnWkq6ur1iHU\nJd/5z8zMas1Js9UlSfOBHSPiakmrAjOAj0bE/YvovyUwBVguIl4cphj3Bb4DvA84NCLOWdwx29ra\nFjuu0aihYRzTpnU5cTYzs5px0jyCSWoETgS2A1YEngfuA46PiDuLiWc/x50BnNnfJFDSFOCeiDi0\n1D4BOCsi3t2f8QqeAFYCZlXpFwMcv98kvQs4FzgYuAIYpES98uu0t3Qxd24bs2bNctJsZmY146R5\nZLuS9DvcgzQTuyLwaWD5Wga1CANOaCMigJmDGMtgWJX03l8bEYMY22pA8+ANZ2ZmZoPCFwKOUJKW\nBRkhNs4AACAASURBVDYHvh0Rt0XEkxFxd0R8PyJ+l2eLA/iNpPmSHs37fUjSbyQ9I2mOpLskfbow\n7hRSQnhm3u/NwrbNJd0m6RVJj0s6W9K4Aca/v6SHJb0mqUvSIusSJK2aY1m/0LadpGk5lpuBD5b2\neY+kSyT9Q9LLku6X9JXC9j0kzZL09tJ+v5F0UZXYJwCVMpEZkt6UtL6kNyQ15z6S9G9JdxT2a5P0\nRPV3x8zMzOqNk+aR66X82FHSUj1s3xgQMIFU2rBxbl8auAb4FPBR4DrgakkfyNt3Bv4BTMz7rQwg\nafXc93JgXWBXYDNSiUI16vZC2gk4CzgV+AjwU2BSrktelAUz1ZJWIZVEXAVsAPwMOKXUvwG4G/hc\nPsb5wGRJG+Xtl5P+/r9YGPe9pNqIn1c5n0uBbfLzjUjv0YPAPcBWuX09YD6wYeGDxRbALVXGNjMz\nszrkpHmEiog3SQnxBOAFSX+SdLKk9fL2Sv3v7IiYGRHP5fb7I+KCiOiKiEci4ljgUXLyGBHPA28C\nL+X9KqUHRwDtEXFuRDwaEX8m1fNOKCXt38wz2AsewI9L4R8G/CIizo+IhyPiTFKpyf/0csrFxHt/\n4OGIODwipkdEB3Bh6f15KiLOiIgHIuKxiPghcAPw5bx9LtAB7FXYbQ/g8Yi4rZc4iIjXgOfyy1n5\nfZoP3MpbSfNWwO+BLtI3ApW2W3sb28zMzOqTa5pHsIj4taRrgE8CnyDNqh4uaZ+ImNzTPpLeCRxP\nmlFdmfQ30ABUu8JqA2C9UhlFJZFdDZiWn7cDJ5f23QU4svC6iTTzW3Q7cGCVGCrWBv5Saruz+ELS\n24CjgS8B7weWyo+XC90uAO6StHJEPE36ADKpjzH05FZgb0kCtiQl6c8AW0l6AFiDqjPNpwOXldpa\n88PMzGxs6+jooKOjo1vb7Nmzh+XYTppHuIiYB9ycHydLuoCUFPeYNJOysk+TZnsfAV4llTr0VOJR\ntDQp0T2bUrkFaXWLitkR8Whxo6RaXMR3OHAAcBCpdOJlUuwLzjMi7pV0P/BVSTcC6wC91jNXcRvw\nLqCFVIpxJPAv0iz9/cA/I+KR3oc4DNh9MUIwMzMbvVpbW2lt7T6R1NnZSUtLy5Af20nz6NMF7JCf\nvw4sUdq+KXBhZRk6SUtTuogOmNfDfp3AOhExY5Bi3Az4ZaFtM+Chfuy/faltk9LrTYGrcukGefZ3\nLeCvpX4/I5WZfAC4KSL+2ccYFhIRs/OM8n8D8yLi75KeJU0dfwGXZpiZmY1YrmkeofLqEDdL2l3S\nepI+KOlLwLeA3+RujwGflrSipOVy23RgZ0kbSNoAuJiFZ44fA7aQ9D5JleXrvg9sKuncvO8aknaQ\n1JcLActOBfaUtF8e51Bgp9zeFz8B1pT0A0lrSdqNVFpRNB34jKRNJFXKQVbsYaxLSAnz16h+AWBZ\n+X2DVH6xOzlBzjXiXaQLJ500m5mZjVBOmkeul4DKxXi3Ag+QyjLOJ5UlQPqu/zOk8onO3HYo6SYo\nt5NWn7i+sK3iGNLs8yPk9ZEj4gFSne6apDKETuA4oDgz26e1mCPiKlLZxGGk0omvA3tGxB97GWvB\n64h4klQnvQNwL7Av3WumAU7KMV4P/AF4Gvh1D7G8SCpPeYn0fvRHT+d7K+nf1ZRC2y257ZZ+jm9m\nZmZ1Qum+EWZjl6SbgAci4pAaxtAMTPUdAXvSBbQxdepUmpt94xczM+uuUNPcEhHlicBB45pmG7Ny\nycqnSDPo+9c4nGxiflhRQ8M4Ghsbax2GmZmNYU6abSy7B1gOODwiphc3SHqQdGfEsgD+q3KB4WBr\nb2+nqalpKIYe0RobGxk/vtqqiGZmZkPHSbONWRGxWi+bPwe8fRHb/jUE4QDQ1NTkEgQzM7M65KTZ\nrAf5YkMzMzMzwKtnmJmZmZlV5aTZzMzMzKwKJ81mZmZmZlU4aTYzMzMzq8JJs5mZmZlZFU6azczM\nzMyqcNJsZmZmZlaF12keIpJWBWYAH42I+3vpNwW4JyIOHbbg6pSkGcCZEXFOrWOpla6urlqHUJd8\nR0AzM6u1ukmaJa0IfAfYDng/6a5r9wFnRcQfahlbNZImActGxM6F5ieAlYBZuc+WwBRguYh4sdBv\nJ+D1IYjp3cCxwI7AyjmO64HjanHjDkn/ARwDfIn0+50D/BU4IyJ+m7ttBLw8TPGsCdwL7BMRlxba\nBdwO/CMivjwcsRS1tbUN9yFHhIaGcUyb1uXE2czMaqYukuY8K3sH8G/gMOBB0i2M/xM4D1indtEN\nTEQEMLPQJCDyz2K/Fwb72Dlh/gswF9gXeAj4IHAy8H+SPhERjw32cas4H9gY+CbQBSwPbJp/AhAR\nzw1XMBExXdIRwHmSpkRE5dbY/0N6rz4/VMeW9PaIWMQHpRNJnxvtLV3MndvGrFmznDSbmVntRETN\nH8C1pJnZhh62LZN/rgJcRZqhnA1cBqxQ6HcscA+wF/B47nceqW77cOBp0uz1UaXx5wP75RheAR4B\ndin1+UA+3vPAc8BvgFULx50PvFn4uQWwan69fuF5sc8v8v5TSLOtlWMtB0wmfYB4Oce1RmH7hBzH\nZ0nJ8BzgOmDFQp8fAy8C7y2dRwPwJHBNoW0KcG5+vAA8C5xQ2m8p4DTgH8BLwJ3Alv2M6Xlgjyp/\nBzOAA0u/m32AK/N78Xdg+9I+6wC/zX8TLwK3AqsVtn8tx/Rq/rl/af+bgN/m52vnv4HPF7YLOBp4\nNG/rBHYqbF8S+HmO/RXgb8B/l47xS+ByYCLwFDCth3NvBgLaA8KPbo+pAcTUqVPDzMysbOrU9P8E\n0By95BmL+6j5hYB5VnRb4LyImFveHhEv5q/MryYllJ8EtgE+BFxa6r46aXZ6W+ArpITpGuB9pET2\n28BJkjYu7XcCKalZH7gYuFTSh3N8SwI3kJKyzUizo3OA6/O204BfkUofViSVQtxRCT//fALYJT9f\nM/c5aBFvyUWkBOoLwCdISdu1kpYo9BlHmpHfPb8f43MclfKCXYH2iHi2OHB+f38EbCtpucKmr5JK\nRDYGDgQOlbRPYfsPgY8DXwbWy+/VdZJW70tM2TPAdpKWXsR5L8oxpN/zeqQPEBdXYpf0PuA2UkK8\nFbAhcAH5GxRJuwPHAUeSEuKjgBMk7VEYf29gc0lfAyYBl0TENaXjV/6WmoBzgEskbZK3L0H6kLZz\n3n4icIqkHUvnsS1pBntrUsmMmZmZjSRDmZH35UFK1OYDO/TS5zPAPOB9hbamvF9Lfn0sKZkdV+hz\nHfBIaawu4PDC6/mkhL3Y585KG9AGPFTavhRp5nOb/HoScGWpz4KZ5vx6S9IM8zKlfgtmmkkJ9Xzg\n44Xt78nH2iW/npDH+WChz/7AU/n5CnmMA4vHKfTdMe+/UeH4D5b6fK/SRkp+XwdWKvW5ETipLzHl\n158kJZevAXcBZwCblsbsaab5uMLrcbnts/n1d4GHgSUWca7TgV1LbUcDt5faJgBv5OMvXWhvyO99\nS6n/JODCXv5ef0xKviuvf0ma4e8xztzHM82LfHim2czMFm3MzDRTqvFdhLWBJyPiqUpDRHSRygma\nCv0ei4hXCq//RfpKnlLbCqW2P5de31kYd31gTUlzKg9SicY7SDPbg2ltUoJ6V6UhIv4NTKP7eb4S\n3WuSn2bhc+rL+1rR0/mvmWet1yXNpv699B5sQffz7zWmiPgj6duBrUkz1esAf5R0dJXYHiiM8Qqp\nBKMy7gbAHyPizfJOksbl+H5eivtoYLVi34i4KMd7bkS8VNi0FvAfwJTSGK3Fc5d0gKS7JT2bt+9N\n+rBRdH9PcZqZmdnIUA8XAk4nfTpYm1SzvDjKF1fFItr682FhaeBuYDcWTkSfXbj7sOjpnCqxPcvC\nHyaK1sn9H+7jsZYmzcI2k2Z5i4oJZm8xpYaUNN6eH6fmhHmipO9HxBuLOH5vv79Xq8QNqazirtK2\nnpLXN/KjpzG2JX3YKpoLIKkNOAU4OB9nDqkcZINS/z6uCnI6qXy+qDU/zMzMxraOjg46Ojq6tc2e\nPXtYjl3zpDkinpd0A/BNSedERLdESNKypJKKVSS9PyL+mdvXIdU4/3UQwvgE0F563Zmfd5JqeZ8t\nzUIWzSPNxvZmXv7ZW78u0u/k4+TZX0nLAx+mj+cZESHpV8Buko6JiAUreORl3/YHro/uq3Z8vDTM\nJsD0PNY9OeYVI+L2vsTQD5XzbaB7At5X9wNflbREeRY3ImZKegpYPQpLyvXTg6Tf2/iIuHMRfTYF\nbouICyoNktYY4PF4qyzczMzMylpbW2lt7T6R1NnZSUtLy5Afux7KMyAtQ7YEcJeknSWtIWltSQcC\nd0TETaQE5mJJG0r6GOmCuSkRcc8gHP9LkvaStKak40l11uflbReT1ji+StLmkj4oaStJZ+cL0QAe\nA9aXtJak5fMFgmWPk2ZJt5fUKOmd5Q4R8TDpgscLJG0maQNSMv9kbu+ro0gX3t0o6T8lfUDSFqSL\nFZcE/rvUf7yk03L8rXn7WTmm6cAlwGRJO+Xz/5ikIyR9rq8BSZoiaV9JzZJWlbQdaQm8P/TyYaSa\n84BlgMskteS/m7a8BjOkOvcjc/nEmpLWlbSnpIP7Mnik9bTPBM7O434o//0dkC8yhPRNycclbZOP\ncTLpgkQzMzMbReoiaY6IGaSv/6eQVlx4APg9aQmzyp3yvkhatuzWvO1h0qoG/T5cD23H5rHuI134\n95WI+FuO7VVS/e4TwBWkGukLSDXNlZuUXECqO76btDbzpuVj5XrsY0lf5T9DWuKtJ3sCU0nLqN1O\nKon4fH/qYXMd9CdI7+dPSO/VpaQEb+NYeI3myaTa3btyXGdGxM9KMU0m/W7+RloCbiPSe9JX15NW\n6biB9B6eTbpQc9di6OVT6en0FjxJ57k18E7gFtL7/zVySUdE/Dy/3os0K30L6aK/Gb2N260x4kjS\nhZFH5bivI63QUhnjR6QPNL8i1YK/i/Sem5mZ2SiiiB5zhTFD0nxgx4joz0zuqCHfxrsuSGoGpvrm\nJj3pAtqYOnUqzc3NtQ7GzMzqTKE8oyUiOqv1H6ia1zSbWdHE/LCihoZxNDY21joMMzMbw5w0L+Jr\n+TFkrJ9/XWlvb6epaVELn4xdjY2NvoW2mZnV1JhPmiOi2qoXo1pEbF3rGOwtTU1NLkEwMzOrQ3Vx\nIaCZmZmZWT1z0mxmZmZmVoWTZjMzMzOzKpw0m5mZmZlV4aTZzMzMzKwKJ81mZmZmZlU4aTYzMzMz\nq8JJs5mZmZlZFWP+5iZm9aSrq6vWIdSc7/5nZmb1yEmzWR1pa2urdQg119AwjmnTupw4m5lZXXHS\nbKOGpEnABCCAN4AngMnAyRExv5ax9d2JwHa1DqKGupg7t41Zs2Y5aTYzs7ripNlGm+uAPYEG4HPA\nj4DXgB8UO0l6GxAREcMdYO9WA5prHYSZmZmV+EJAG21ei4hnI+LJiPgpcBOwg6QJkp6XtL2kvwJz\ngVWUHCPpSUlzJd0jadvKYJJWlTRf0pck3SbpFUl3SVpT0saS/k/SHEnXSlq+sF+v45qZmdnI4qTZ\nRru5wFL5+TjgcGAf4CPATOBg4BDgUGA94Abgakmrl8Y5DjgB2JBU+nEJcApwALA5sEbeXtHXcc3M\nzGwEcNJso5akbYBtgZtz05LA/hHx54iYHhFzgcOAUyLi8tx2BHAvKektOjUiboqIacDZpBqKE/JY\n9wE/Bz5V6N/Xcc3MzGwEcE2zjTbbS5oDvB0QcDFwPPBlYF5EPFjpKOldwPuAO0pj3A6sX2p7oPD8\nX/nng6W2FQYwbsnpwGWlttb8MDMzG9s6Ojro6Ojo1jZ79uxhObaTZhtt/gDsB7wOPFVZNUMSwKuL\nMe7rheexiLZB+ObmMGD3xR/GzMxsFGptbaW1tftEUmdnJy0tLUN+bJdn2GjzckTMiIh/VFtmLiLm\nAE8Bm5U2bQY8VOzanwD6Ma6ZmZmNEJ5ptrHuVOA4SY+Sao73BjYAdiv0UQ/79dTW33HNzMxshHDS\nbGPdOcAywGmkmuSHgO0j4pFCn55mmqvNPvdlXDMzMxshVHf3djAbgyQ1A1N9R8AuoI2pU6fS3Oyb\nvJiZWXWFmuaWiOgcquN4ptmsrkzMj7GroWEcjY2NtQ7DzMysGyfNZnWkvb2dpqamWodRU42NjYwf\nP77WYZiZmXXjpNmsjjQ1NbkswczMrA55yTkzMzMzsyqcNJuZmZmZVeGk2czMzMysCifNZmZmZmZV\nOGk2MzMzM6vCSbOZmZmZWRVOms3MzMzMqnDSbGZmZmZWhW9uMsJI2hT4CbA28LuI2LnGIY0okqYA\n90TEofn1DODMiDinl33mAztGxNVDHV9XV9dQH6Lu+Y6AZmZWj5w01yFJnwD+BFwXEduXNp8BdALb\nAi9L2hKYAiwXES8OwrHfBhwOTABWBV4FpgM/jYhf5D7dEs9+jD0JWLa/ib6kY0lJ64al9lWBGcBH\nI+L+/oxZsBHw8gD3HXRtbW21DqHmGhrGMW1alxNnMzOrK06a69M+wDnAPpJWiohnCttWB34cEU8D\nSBIQgBb3oJKWAI4Fvg58E5gKLENKLN+9uOMvpuhne98GjXhucfYffCcC29U6iBrqYu7cNmbNmuWk\n2czM6oqT5joj6Z3ArkALsBKwJ3BKYVY1gEmSfgHsDUzKbc9LCuCiiNg7J9NHkBLglYBpwEkRcUU+\nTmWGejvgJGBd4LPA9sCPIuLKQlgPFOKbBGwJbCHp4Hzs1YB/AD8Fts7HeyKPc07e71jS7HXkcocA\nPhURt0n6AHB6Pv584I/AQRHxeF/estL7tyXwA2AD4N/ARcDRETG/x51L5RmS1gB+AWwMPAIc3MM+\npwA7AR8AngEuBo6PiDfz7+lRYOOI6CzsczBwcER8sPfTWQ1ornbOZmZmNsx8IWD92RXoiojppGRs\nn9z+BCkZnQMcCKwM/ArYJW9fM7cdlF8fBbQB+wLrAGcCv5T0ydLxvgd8G2giJcfPAFtLalxEfAcB\ndwIXACvmYz5J+lt6MsfTBBwPnCzp/+X9TsvxXl/Y7w5JSwI3ALOBzYBN8zlen7f1ppwwvw+4BvgL\nsD6wH+n9+06VcSr7C/g1MJeUNO8HfJ+FZ7NfBL6az/NA4GvAIQA50b8R2Ku0z56kZNzMzMxGIM80\n15+9gV/m59cDy0jaIiJuA2bm2eQXI2ImgKR/577PVmqaJS0FHAl8OiL+krc/lhPm/yLN5FZMjIib\nKy8kHQpcDjwj6a/AHcBVEXE9QES8KGke8EpEPFsY5w1SolzxeL5o8cvA/0bEy5JeBZYq7idpd0AR\nsW+hbR/geWAr4KbcvL6kOaX3qlKaUvFN4ImIODC//nue4T4FOIHqPgOsBWwTEf/KsRwFXFfsFBHf\nLbx8QtLppA87p+W2nwM/lnRoRLwuqZk0k//FPsRgZmZmdcgzzXVE0oeBjwGXAkTEm6TZ2a/1c6g1\ngHHAjZLmVB7AHsCHCv2CVLf8VkNEV0SsC3yclPy9F/itpJ/2If5vSrpb0sx8vH2BaoWpGwBrluJ8\nDngHqX674m+k2eMNCo9y8e/apFnwotuBpXMJSDVrA09WEuasPB6SdpX0J0lP53hPovt5/oZUZrJT\nfr0nMCUinuhDDGZmZlaHPNNcX/YBlgCeTpUCC7wm6ZsRUZ5pXZSl88/tgKdK214rve5x5YiImEpK\nqM/Js8GTJZ28qDpjSV8BTiWVKfyZVGJxOOlDQLVY7wZ2Y+GLGYsz2fMiYkbpmG/2sM+QkrQJ0A5M\nBH5PKitpBRasJJJnlycDe0n6dd5+QN+OcDpwWamtNT/MzMzGto6ODjo6Orq1zZ49e1iO7aS5TuSV\nK/YgJV83ljb/hpQ19TTbOy//XKLQ9hApOV41Iv40COFVFg9+Z+GYS5T6bArcHhHnVxokrV7q09N+\nnaQSjmcj4qVBiLO8nN3mwJyI+Ecf919F0oqF2eZNSn02AR6LiFMqDZI+2MNYPwMeBL5BOudf9+H4\nwGHA7n3ramZmNsa0trbS2tp9Iqmzs5OWlpYhP7bLM+rH9sBywC8i4qHiA7iSty4ILHucVGaxvaRG\nSe/MyedpwJmSvirpQ5I2lPTfkvYo7LvQLK2kyyUdLOljksZL2go4j7T6xt9yt8eAj0taVdLy+QK6\n6cBGkj4raU1JJ5Aupit6jFSbvFbeb0nSxY6zgKskbS7pg5K2knR2vrCvP35ESnrPlfRhSTsAx5Gm\nb/vipnwekyWtn2vAT6J73fR0YHwu0fiQpAOBHcsDRcTfSDPu3wcuiYjyDL+ZmZmNIE6a68fewI2L\nKMG4gpSQrktpJYeIeIq0tvIppJUvzs3tE0mL/h5Bmnm+jlSuUSxx6GmN4+uBLwBXkxLlSXn/bQvL\ntp0GvJnbZwKrAOeTkvtLScnie4Aflsa+II95d95v04h4FdiCtDrIFXnMC0g1zX25WcuCc8jvxXak\nZP1eUhJ9AXByL+dc3D9ICXADaQWOn5JWIaHQ57eklUjOBe4BPsGiLzL8OfB2vGqGmZnZiKeUJ5jZ\nYJM0EdglIj7ah77NwFTf3KQLaGPq1Kk0N3u9ajMzq65QntFSvEfCYHNNs9kgyzeoWY20BN5RVbqX\nTMyPsauhYRyNjYtaJtzMzKw2nDSbDb7zgK+QLv6b1J8d29vbaWpqGpKgRorGxkbfQtvMzOqOk2az\nQRYRe7HwHQH7pKmpyWUJZmZmdcgXApqZmZmZVeGk2czMzMysCifNZmZmZmZVOGk2MzMzM6vCSbOZ\nmZmZWRVOms3MzMzMqnDSbGZmZmZWhZNmMzMzM7MqfHMTsyEgaT6wY0Rc3Z/9urq6hiiikcN3BDQz\ns3rkpNlGFUkXAl8FfhIR3yht+yGwP3BhROw9SMc7lpQcbzgY47W1tQ3GMCNaQ8M4pk3rcuJsZmZ1\nxUmzjTYBPAF8RdIhEfEagKR3AK3A40N0zEFyIrDd4A034nQxd24bs2bNctJsZmZ1xUmzjUb3AB8C\ndgY6ctvOpIR5RqWTpKWA04BdgWWAu4FDIuLuvH1LYAqwDfB9YB3gXmDPiJguaQJwLBC5HCOAvSJi\ncj7EeyVdCWwL/BM4LCJ+23voqwHNi3PuZmZmNgR8IaCNRgH8AiiWYOwNTAJUaDsV2AnYA9gQeBi4\nQdJypfFOAg4BWoA38tgAlwGnA38FVgRWzm0VxwCXAusB1wIX9zC2mZmZjQBOmm20uhjYXNIqklYF\nNgXaKxsljQP2A/4nIn4fEX8Dvg68CuxTGCeAoyLiT7nPKcCmkpaKiLnAS8AbEfFsRMyslINkkyLi\nVxHxKHAUsDTwsaE7ZTMzMxsqLs+wUSkiZkn6HbAXaXb5moj4t7Rgonl10t//HYV93pB0F9BUGu6B\nwvOn888VgH9UCWPBfhHxiqQX835mZmY2wjhpttFsEnAeabb4Gz1sVw9tPXm98Lxy0V9fvqV5vfQ6\nqu93Ot0rPCBdv9jah8OZmZmNbh0dHXR0dHRrmz179rAc20mzjWbXA0sBbwK/L217BJgHbEaqO0bS\nksDGwBn9OMY8YInFjnSBw4DdB284MzOzUaS1tZXW1u4TSZ2dnbS0tAz5sZ0026gVEfMlrZ2fR2nb\nK5J+DJwq6XngSeBw4D9460I/6Hk2utj2GLCapA1I5RpzImLe4J2FmZmZ1QMnzTaqRcRLvWw+gpQA\nTwbeRVpy7rMRUfyep6c1mIttV5BW4JgCLEuqoZ7ch/3MzMxsBHHSbKNKROxVZftOheevAQfnR099\nb6VUehER9xXb8qzyl3vYd6GSjYh4T5XwzczMrE45aTarKzOAzloHUUNdtQ7AzMysR06azerKxPwY\nuxoaxtHY2FjrMMzMzLpx0mxWR9rb22lqKi8TPbY0NjYyfvz4WodhZmbWjZNmszrS1NREc3NzrcMw\nMzOzEt9G28zMzMysCifNZmZmZmZVOGk2MzMzM6vCSbOZmZmZWRVOms3MzMzMqnDSbGZmZmZWhZNm\nMzMzM7MqnDSbmZmZmVXhm5uYDQNJWwJTgOUi4sVF9evq6hq+oIaA7+ZnZmajlZNmGxMkTQHuiYhD\naxhGVOvQ1tY2HHEMmYaGcUyb1uXE2czMRh0nzWaZpCUi4s3aRnEisF1tQxiwLubObWPWrFlOms3M\nbNRx0myjnqRJwJbAFpIOJs347g1MImWoJwHrAp8FbpO0A3AMsA7wT2AycHIloZY0H/g68Hlg29zn\nsIj4beGY2wFnAqsAd+Yx+mA1oHmxztfMzMwGny8EtLHgIFLiegGwIrAy8GTe9j3g20ATcL+kTwIX\nkRLetYH/AiYAR5XGPAa4FFgPuBa4WNJyAJI+AFwBXAVsAPwMOGWIzs3MzMyGgZNmG/XyhXfzgFci\n4tmImAlUyjAmRsTNETEjIl4gJcPfi4j2iHg8Im7ObfuVhp0UEb+KiEdJCfXSwMfytm8AD0fE4REx\nPSI6gAuH9izNzMxsKLk8w8ayAKaW2jYANpX0nULbEsBSkhoiYm5ue2DBIBGvSHoRWCE3rQ38pTTu\nnYMXtpmZmQ03J8021r1cer00aWb5ynLHQsIM8Hp5M4Pyzc3pwGWlttb8MDMzG9s6Ojro6Ojo1jZ7\n9uxhOfaAkmZJe5C+rl4N2CQiHs8XWM2IiKsGM0CzQTKPNGNcTSfw4Vx2MVBdwPaltk36tuthwO6L\ncWgzM7PRq7W1ldbW7hNJnZ2dtLS0DPmx+z0zJml/4AzSxU/L8VYi8gJw8OCFZjaoHgM+LmlVScuT\n/vbVQ78TgK9KOkbSOpLWlrSrpBP7cayfAGtK+oGktSTtRrqY0MzMzEaogXydfADw9Yg4mbcupgK4\nm7SSgFk9Oo309/oQMBMYTw83G4mI3wNfAD4D3EWqRT6YlHQv6NbD+AvaIuJJYBdgB+BeYF/gyEE4\nBzMzM6uRgZRnrAbc00P7a8A7Fy8cs6EREdOBzUrNFy2i743Ajb2MtVCZR0S8p/T6WtK3MVWP2FE2\n6wAAIABJREFU190MUoXISDSybwFuZmbWm4EkzTOAjwKPl9r/E/+vabaYJubHyNTQMI7GxsZah2Fm\nZjboBpI0nwH8UFIDqSb0Y5JaSV8/f20wgzMba9rb22lqaqp1GAPW2NjoW2ibmdmo1O+kOSJ+JulV\n0q2HxwGXAE8BB0XEpYMcn9mY0tTURHOzb6NtZmZWb/qVNEsSsApwRURcLGkcsHS+w5qZmZmZ2ajU\n39UzBDxMSpyJiFecMJuZmZnZaNevpDki5gPTgeWHJhwzMzMzs/ozkHWajwBOlbTuYAdjZmZmZlaP\nBrJ6xmTSBYD3SZoHvFrcWF6v1szMzMxspBtI0uxbZZuZmZnZmDKQJef6cFczMzMzM7PRo99Js6Re\n71wQEU8MPBwzMzMzs/ozkPKMx4DoZfsSAwvFbPSSNAW4JyIO7a1fV9fIvhO97whoZmaj1UCS5g1L\nr9+e2w4Fjl7siGzISZoETCB9+HkDeIJ0gefJeVnBuiJpPrBjRFzdx/4TgLMi4t1DG9nga2trq3UI\ni6WhYRzTpnU5cTYzs1FnIDXN9/XQfLekp4BvAVcudlQ2HK4D9gQagM8BPwJeA35Q7CTpbUBERG/f\nLtQb0fu3IX0bpCbnfiKw3fAdblB1MXduG7NmzXLSbGZmo85AZpoXZRqw8SCOZ0PrtYh4Nj//qaSd\ngR0k/Qs4C/gqcAqwJrCGpCeBicDXgfcCXcAREXEDgKRVgRnArsABwEbAg8DuwHKkpHxt4I/AHhHx\nXN5PvY1bVjjOLvk4HyfdcGe/iPizpC2BXwCRZ6gDOD4iTpC0FPBd4Cs5pgfysW7NY0/o4dy/Kekc\nYMWIeLEQx9nARyJiG0nvAc4DtgDeDTwCfDciLu3zb2OB1YDm/u9mZmZmQ6rfNzeRtEzpsayktYGT\nSMmLjUxzgaXy83HA4cA+wEeAmaSlBg8hleGsB9wAXC1p9dI4xwEnkEp23gAuISWgBwCbA2vk7RV9\nHbfsJNKs+AbA34FL8szwHXnMF4EVgZWB0/I+PyQl2V/Ox7ocuK50rPK5Xww8T0rSgQUz0F8G2nNT\nA3A3acb+I8D5wGRJG1U5BzMzMxshBjLT/AILf/Ut4EnSDJ6NMJK2AbYFzs5NSwL7R8SDhT6HAadE\nxOW56QhJnyIlqAcUhjs1Im7K+5xNSpq3jog/57afk+qpK/o6btmpEXF9HvNY0qz2GhHxd0mzSWUV\nlZl0JK1CKkdZJSKeyc1nSPocsBfwnV7O/TJgN2BSbtoGWJZcihQRTwFnFGL7oaT/JCXWd/dyDmZm\nZjZCDCRp/lTp9XzgWeDhiHhj8UOyYbK9pDmkCzlFmlE9npTozSslje8C3keaxS26HVi/1PZA4fm/\n8s8HS20rDGDcsuJxns7nsAJp1rkn65FWdvl7LgmpWAqYVXjd7dyzi4E7Ja2UE+7dgGsq5Rp55vlo\n4EvA+/OYSwEvVzkHMzMzGyEGkjQHcEc5QZa0pKQtIuK2wQnNhtgfgP2A14GnKqtm5Hzy1V72q+b1\nwvNYRFu/y4L6eJzexl2aVC7STPqgV/RS4flC5x4Rd0t6FPiKpJ8AO5HqnisOJ82KH0T6gPAyadZ+\nqfJY1Z0OXFZqa80PMzOzsa2jo4OOjo5ubbNnzx6WYw8kaZ5CqhOdWWpfNm/zOs0jw8sRMaMvHSNi\nTl4dZTPShXwVmwF/KXbtTwD9GHehXasMPY+F/w7vyW0rRsTt/YkzuxhoA/4JvAlcW9i2KXBVRHTA\ngosb1wL+2v/DHEa6dtLMzMzKWltbaW3tPpHU2dlJS0vLkB97IEnzopbzWh5/HT2anQocl2dc7wX2\nJl2Et1uhj3rYr6e2/o7b3zEfA5aWtDVwH/BKREyXdAnpAr3/ISXRKwBbA/dFxHVVxryYdJHj0cD/\nRkRxpns6sIukTUg1/4eQLkIcQNJsZmZm9ajPSbOkyvrLAVwo6bXC5iVINajl2lQbPc4BliGtRLEC\n8BCwfUQ8UujT04eparPCAxm31+NExJ25jOIy4D2kWu0TSBcCficf6/2kWuY/A7+tEiMR8Yiku0jL\nKh5U2nwSaa2464FXgJ8CvyZ9+9JbzGZmZjZCqK/3bch3kYO08sGv6F77OY80u3dBRMzCzPpFUjMw\nNa1iN1LLMzqBFqZOnUpzs9eaNjOz4VEoz2iJiM6hOk6fZ5ojYi8ASY8Bp0WESzHMBt0MUvI5EnXV\nOgAzM7MhM5DbaB8/FIGYGaSbI06sdRAD1tAwjsbGxlqHYWZmNugGdBttSf+PtJ7veErLakWEv5c1\nG6D29naamppqHcaANTY2Mn78+FqHYWZmNuj6nTRLOhA4GbgQ2IF0l7TVSRdI/XAwgzMba5qamlwP\nbGZmVocGcpOJbwD7RsQBpAsAfxARnyGtgrBsr3uamZmZmY1AA0max/PW0nKvAu/Kz3+Jb1tmZmZm\nZqPQQJLmZ0hr3wI8AXwiP1+N6jedMDMzMzMbcQaSNP8B+GJ+Pgk4U9KNpBtJ/HqwAjMzMzMzqxcD\nWT1jX3KyHRE/lPQcsClwNXD+IMZmZmZmZlYXBrJO83xgfuH1pcClgxmUmZmZmVk9GUh5BpI+Kald\n0p2S3p/b9pC0+eCGZ2ZmZmZWe/1OmiXtAtxAWjljQ+AdedOywFGDF5qZmZmZWX0YSE3zd4D9ImKy\npK8U2m/P28z6TdJ8YMeIuFrSqsAM4KMRcf8i+m8JTAGWi4gXhyG+Y3N8Gw7lcbq6uoZy+CHnOwKa\nmdloNZCk+cPAbT20zwaWW7xwrDeSGoETge2AFYHngfuA4yPizmLi2c9xZwBnRsQ5/dxvCnBPRBxa\nap8AnBUR7+7PeAVPACsBs6r0iwGOP1BDfry2trahPsSQamgYx7RpXU6czcxs1BlI0vwMsAbwWKl9\nc+DRxQ3IenUl6Xe2B2kmdkXg08DytQxqEQacYEZEADMHMZYRpPKZaCTqYu7cNmbNmuWk2czMRp2B\nJM0XAGdL2puUGL1P0ibAaaT/8W0ISFqW9MFky4j4Y25+Erg7b59B+n38RhLAYxHxIUkfAs4g3YTm\nnUAXcGRE3Jz3mwKsSlpv+yxSzrpE3rY58F1gI+BZ4Dd531cGEP/+wGHAKqQPVydHRPsi+i5UniFp\nO+DMvP+dwOTSPu8BzgO2AN4NPAJ8N6/ugqQ98v4rR8Trhf1+A8yOiAl9PI99SWVIywO/A74WEXMk\nfQS4H1ghIp6T9G7gOeDSiNgt7/sd4LMRscWij7Aa0NyXUMzMzGwY9elCQEnrS6qszfw94BLgZmBp\nUqnGz4DzI+LcoQrUeCk/dpS0VA/bNybdkXECqbRh49y+NHAN8Cngo8B1wNWSPpC37wz8A5iY91sZ\nQNLque/lwLrArsBmQF9+x93uDClpJ+As4FTgI8BPgUm5LnlRFsxUS1oFuAK4CtiA9Pd2Sql/A+kD\nxOfyMc4HJkvaKG+/nPT3XrkxD5LeS5rW/XkfzglgTeBLwOeBbUkXwv4IICL+SionqZzTJ0uvISX0\nt/TxWGZmZlZH+rp6xj1AI4CkR4GfkG6lvS5pBvO9ETFxSCI0ACLiTVJCPAF4QdKfJJ0sab28vVL/\nOzsiZkbEc7n9/oi4ICK6IuKRiDiWNNP7xbz9eeBN4KW8X6Us4gigPSLOjYhHI+LPwMHAhFLS/k1J\nc4oP4Mel8A8DfhER50fEwxFxJqnU5H96OeVi4r0/8HBEHB4R0yOiA7iw9P48FRFnRMQDEfFYRPyQ\ntMrLl/P2uUAHsFdhtz2AxyOipxr9nrwD2CMf40/AAUCrpBXy9j8CW+XnWwG/AN4haS1JS5JuAnRr\nH49lZmZmdaSv5RkvkL43ngl8EHhbRMwDHhqiuKwHEfFrSdeQZjE/QZpVPVzSPhExuad9JL0TOJ40\no7oy6XfeAFQrOt0AWE9S8cq0SiK7GjAtP28HTi7tuwtwZOF1EwvfLfJ24MAqMVSsDfyl1HZn8UX+\nJuRo0kzw+4Gl8uPlQrcLgLskrRwRT5M+gEzqYwwAT0TEM6UY3ka6OHYmKSH+et62Jek9WIuUQC9P\neu9v7/0Qp5PuSF/Umh9mZmZjW0dHBx0dHd3aZs+ePSzH7mvSfAVwq6SnSV+b3y3pzZ46RsSHBis4\nW1j+sHJzfpws6QJSUtxj0kzKwj5Nmu19hLS+9hWkhLI3S5MS3bMplVuQVreomB0R3S4AlVSLi/gO\nJ838HgQ8SEqWz6ZwnhFxr6T7ga9KuhFYB7hoEGO4hVQbvgbpg8Kf8s9Pkb6ZuTvPePfiMGD3QQzJ\nzMxs9GhtbaW1tftEUmdnJy0tLUN+7D4lzRGxr6QrSatmnEOasZszlIFZn3UBO+TnrwNLlLZvClxY\nWYZO0tKkbwuK5vWwXyewTkTMGKQYNwN+WWjbjL5/U9EFbF9q26T0elPgqly6gdLVkGsBfy31+xmp\nzOQDwE0R8c8+xgAwXtJKhdnmTUilLdMAIuIBSS+QLhS8NyJekXQL8G3SxYm39ONYZmZmVkf6vHpG\nRFwPIKkFODsinDQPo7w6xOWkOtn7SR9aNga+RVrVAtIygJ+WdAfwWkS8AEwHdpb0u9znBBaeOX4M\n2ELSZXm/54DvA3dKOpeUaL5MusBum4g4oJ/hnwpcJule4CZSPfVOpBnwvvgJcKikH+RYNiKVVhRN\nB3bJK7m8ABxCWpKvnDRfQlrp5Wukmub+eA24SNK3SHfAPBu4rFAHDunC2N1J5wzpd/UOYGvSrL+Z\nmZmNQP2+jXZE7OWEuSZeAioX490KPEAqyzifVJYA6bv9z5DKJzpz26Gkm6DcTlp94vrCtopjSLPP\nj5DXR46IB0h1uWuSEsFO4DigODPbp7WYI+IqUtnEYaTSia8DexaWzutprAWvI+JJUp30DsC9wL50\nr5kGOCnHeD3wB+Bp4Nc9xPIiqTzlJdL70R/TSRcwXpuPcy/wzVKfW0n/rm7JxwvS+zefqvXMZmZm\nVq+U/k83Gzsk3QQ8EBGH1DqWCknNwNSRfnMTaGPq1Kk0N3utaTMzGx6FmuaWiChPDA6agdzcxGxE\nkrQc6aK8LUnL2NWhifkxMjU0jKOxsbHWYZiZmQ06J802ltwDLAccHhHTixskPUi6M2JZAP9VucBw\nqLW3t9PU1DQchxoSjY2NvoW2mZmNSk6abcyIiNV62fw54O2L2PavIQinR01NTS5tMDMzq0NOms1Y\ncLGhmZmZWY/6vXqGmZmZmdlY46TZzMzMzKwKJ81mZmZmZlU4aTYzMzMzq8JJs5mZmZlZFU6azczM\nzMyqcNJsZmZmZlaF12k2GwBJHwYuBD4KdEXEoNyRpKurazCGqRnfEdDMzEYrJ802bCRNAiaQbk39\nBvAEMBk4OSLm1zK2ATgeeAlYE3h5sAZta2sbrKFqoqFhHNOmdTlxNjOzUcdJsw2364A9gQbSrat/\nBLwG/KDYSdLbgIiIGO4A+2h14HcR8Y/BHfZEYLvBHXLYdDF3bhuzZs1y0mxmZqOOk2Ybbq9FxLP5\n+U8l7QzsIOlfwFnAV4FTSDO4a0h6EpgIfB14L9AFHBERNwBIWhWYAewKHABsBDwI7A4sR0rK1wb+\nCOwREc/l/dTbuL2RNJ80W94i6RjSrPN6wDMRcUDucxZwILB2RPxd0tuB54EvRsQfFj36asCgVHqY\nmZnZIPKFgFZrc4Gl8vNxwOHAPsBHgJnAwcAhwKGkxPQG4GpJq5fGOQ44AdiQVPpxCSn5PgDYHFgj\nb6/o67g9WQl4CDgtPz8NuBXYstBnC+BZYKv8+mOkD6l39GF8MzMzqzNOmq1mJG0DbAvcnJuWBPaP\niD9HxPSImAscBpwSEZfntiOAe0lJb9GpEXFTREwDziZN156Qx7oP+DnwqUL/vo67kIiYSUrMX4qI\nZyPiFeAWYB1Jy0taDlgnx7FV3m1L4P/yOZmZmdkI46TZhtv2kuZImgtcA3SQyhsA5kXEg5WOkt4F\nvI+FZ2dvB5pKbQ8Unv8r/3yw1LbCAMbtkxz386Tk+JNAJ/A73pp93pKUWJuZmdkI5JpmG25/APYD\nXgeeqqyakUqMeXUxxn298DwW0TbUHxJvI81mv0ZKkB8A3iHpI8CmwKnVhzgduKzU1pofZmZmY1tH\nRwcdHR3d2mbPnj0sx3bSbMPt5YiY0ZeOETFH0lPAZqQL+So2A/5S7NqfAPoxbn/dSrqwcC5wdESE\npD8C3yLVbd9efYjDSNcwmpmZWVlrayutrd0nkjo7O2lpaRnyYztptnp3KnCcpEdJNcd7AxsAuxX6\nqIf9emrr77j9dQtwJmmm+U+FttOAuyJicWbSzczMrIacNFu9OwdYhpR4rkBatWL7iHik0KenmeZq\ns899Gbc3PY3/AKmueVq+OBBS0vw2YEofxzUzM7M6pPq9d4TZ2CGpGZgK7Yzc8oxOoIWpU6fS3Oy1\nps3MbHgUyjNaIqJzqI7jmWazujKDlHyORF21DsDMzGzIOGk2K5F0JHDUIjbfFhGfH7qjT8yPkamh\nYRyNjY21DsPMzGzQOWk2W9iPWXjdt4ohvZivvb2dpqYBLRVdFxobGxk/fnytwzAzMxt0TprNSiLi\nBeCFWhy7qanJ9cBmZmZ1yHcENDMzMzOrwkmzmZmZmVkVTprNzMzMzKpw0mxmZmZmVoWTZjMzMzOz\nKpw0m5mZmZlV4aTZzMzMzKwKJ81mZmZmZlX45iZmAyTpWGDHiNhwsMbs6uoarKGGnO/+Z2ZmY4mT\n5jFG0iRgAhDAG8ATwGTg5IiYX8vYyiStCkwEtgZWAv75/9m77zC7qrL949+bOoYqjiD6I7wISAaQ\nMiPyEqIJoqIg4ouNgaEICAiIQhQp0kFqpDcpoQyMWFAJSBEILaLATCiRIbTQpCWUkAAJkDy/P9Y6\nsHMyM2dmMi2Z+3Nd+5pz1lp7rbVP8PI56zx7beAq0lzfz21eBM6IiFMK550EHAyMioi7CuV3AM9G\nxK49OM3owb5oaGjoye56VVXVECZPbnXgbGZmg4KD5sHpRmA3oAr4JnAeMBs4pdhI0mJARESPBoZd\nMAwQ8GPgKWB94GJgCCkoBrgDGMW8cx9F+jIwCrgLQNLSwKbApb096QVzHLB1f0+iE1qZNauBadOm\nOWg2M7NBwUHz4DQ7Iqbm17+TtD2wnaRXgDOAXYCTgLWBtSQ9T1rx/THwSaAVOCQiboYPV4SnAD8E\nfgp8AZgE7ASsSArKhwF3AztHxGv5PHXUb/57c2Hez0g6DdiHj4Lm8cBpkhaLiLmSlgU2Bn4O/AA4\nNrcbDixFCrLJ428HHAmsS1rFvgI4vrTiLmkFYAzwbWBp4H7goIh4uK0PVdKawC3ADRFxgKShwDnA\niDz2FOCXEXFTW+cnawC17VebmZlZv/CNgAYwixTUwUeruHsA6wGvkgLQA4GDgM+TAtnrcpBYdDQp\nSN2YlPpxNSn4/ikpcFyLj4JYutBv0YrA64X344HlgE3y+y8Bk4FrgU0lla5rFPBMRDwHIOlLwOXA\n6aSAfm9S2srhhb7/BHwC2IoUybYAt0pasXxSkjYgfSlojIgDcvF5pM91BGmV/FfAzA6uzczMzAYo\nB82DnKSvkoLC23LREsBPIuJfEfFERMwCRgMnRcQfc9khwIOkoLfo1Ii4NSImA2eSAs1jc18PAZcA\nWxTad7bf0lzXAvYHLiiVRcSTpFXiUbloFHBnRLxCStHYrFA+vtDdkcCJEdEYEc9GxG25bJ881gjS\nivkPImJiRDwVEQcD04Hvlc1rs9z3KRFxVKFqNWBCRDwaEc9ExN8j4p62rs3MzMwGNqdnDE7bSpoB\nLEnKGb4KOIaUzvBeREwqNZS0HPBp4J9lfUwANigre6Tw+pX8d1JZ2crd6BdJnyHlYl8TEeV5yXeQ\nguKTmTe/+U5glKR/k/KZf1c4Z0NguKRfF8oWB5aSVJXnsBzwesoi+VAVUFwJXx34B3BYRJxVNq+z\ngPMlbQXcCvw5Ih6hQ2OAa8rK6vNhZmY2uDU1NdHU1DRP2fTp0/tkbAfNg9PtpBXV94EXCzm8AO8u\nQL/vF15HO2Vd/nVD0qdJc74nIvZuo8l44AxJK5FSQ+7M5XcCe5HSJpbMfZQsS1pZvraN/mbn+heB\nkaQvFkVvFl6/mtvVSxobETNKFRFxiaSbgG2ArwOHSBodEee2f7WjSangZmZmVq6+vp76+nkXklpa\nWqirq+v1sZ2eMTi9HRFTIuKFStvM5SDwRWDzsqrNgUeLTbsygc72m1eYx5Nuwtu9ne7Gk4Lcg4DH\nI2JaLr+LtML8TeCJiHipcE4LsE5EPN3GEbn+U8CcNuqLOdXvAt8iBdo3S1qm7Dr/GxG/i4jvAb8l\n3fRoZmZmCxmvNFtnnAocLelpUs7x7qT0hh0LbcpXY9sr63S/eYX5DtKuEwcDK5dSJXLOcun1FEnP\nkW44bCyUv5D3cd6LdFNi0bHAuLwzyJ+AuXns9SPiiIi4VdK9wF8l/Qp4HPgMaT+4ayOipTDOu5K2\nIaWP3CTpGxHxtqTTc9njwEqkfO7iFw0zMzNbSDhots44C1geOI2Uk/wosG1EPFVo09ZKc6XV50r9\nfg34bD6ez2XK/S5e1td40lZ548vK7yTtilFMzSAibpH0LVKKxsGkNJLHSPtAl2wNnEDa2/mTwMuk\n1etXKJOD5G8CNwHXS9o6z/Ec4P8Bb5EC6IM6+kDMzMxsYFL/PbfCzEok1QLNC9PDTaCB5uZmamu9\nr7SZmfWfQk5zXfGX4J7mlWazAeWIfAx8VVVDqK6u7u9pmJmZ9QkHzWYDSGNjIzU1Nf09jU6prq72\nI7TNzGzQcNBsNoDU1NQ43cHMzGwA8pZzZmZmZmYVOGg2MzMzM6vAQbOZmZmZWQUOms3MzMzMKnDQ\nbGZmZmZWgYNmMzMzM7MKHDSbmZmZmVXgoNnMzMzMrAI/3MSsAkmrA1OAjSLi4d4cq7W1tTe77zY/\n/c/MzAY7B83WaySNBXYFAvgAeA64AjghIub259y6IfpikIaGhr4YpsuqqoYweXKrA2czMxu0HDRb\nb7sR2A2oAr4JnAfMBk4pNpK0GBAR0SfBaTeob4Y5Dti6b4bqtFZmzWpg2rRpDprNzGzQctBsvW12\nREzNr38naXtgO0mvAGcAuwAnAWsDa0l6HjgC+DHwSaAVOCQiboZ5UiV+CPwU+AIwCdgJWJEUlA8D\n7gZ2jojX8nnqqN9OqpF0PlALPAnsFxF35f7vB5oi4rf5/V9J0e+KEfGOpM8AzwNrRcTT7Q+xRu7e\nzMzMBhLfCGh9bRawVH49BDgY2ANYD3gV+DlwIHAQ8HngZuA6SWuW9XM0cCywMSn142pS8P1TYASw\nVq4v6Wy/HTkFOBXYCLgXGCfp47nuTmBUoe0I4I38l1z3QscBs5mZmQ1UDpqtz0j6KrAVcFsuWgL4\nSUT8KyKeiIhZwGjgpIj4Yy47BHiQFPQWnRoRt0bEZOBM0vLssbmvh4BLgC0K7Tvbb0fOjoi/5jF/\nAkwnBfwAdwAjlGwAvAdcxUeB9EhSYG1mZmYLIQfN1tu2lTRD0izgBqAJOCbXvRcRk0oNJS0HfBr4\nZ1kfE4CasrJHCq9fyX8nlZWt3I1+O/Kv0ouImAM8UDj/bmA50sr3SFIQfQfzBs13dGEsMzMzG0Cc\n02y97XZgH+B94MXSrhkpxZh3F6Df9wuvo52yPvtSGBHTJT1EWt3eDLiFFEhfI2ltUs52J1aaxwDX\nlJXV58PMzGxwa2pqoqmpaZ6y6dOn98nYDpqtt70dEVM60zAiZkh6EdicFHCWbA78u9i0KxPoQr+V\n/C9wD4CkxYE64OxC/V2koHkT4LCIeEPSY8DhpC8MT1YeYjTpnkYzMzMrV19fT339vAtJLS0t1NXV\n9frYDpptoDkVOFrS06Sc492BDYEdC23a2v6t0pZwnem3kv0kPUnaeeMg0m4dlxbq7yDdiPhqRDxe\nKNsf+EMXxjEzM7MBxkGzDTRnAcsDp5Fykh8Fto2Ipwpt2lpprrT63Jl+OxLAIfnYkLTl3LYR8Xqh\nzd2k4P2OQtkdwAHA+E6OY2ZmZgOQBu6zJMwGD0m1QDM0MvDSM1qAOpqbm6mt9R7SZmY2sBTSM+oi\noqW3xvFKs9mAMoUUpA4krf09ATMzs37noNkGPUmHAoe1U31XRGzTd7M5Ih8DS1XVEKqrq/t7GmZm\nZv3GQbMZnM/8+7yVLMi2eF3W2NhITU1Xto7uG9XV1QwdOrS/p2FmZtZvHDTboBcRbwJv9vc8AGpq\napw3bGZmNgD5iYBmZmZmZhU4aDYzMzMzq8BBs5mZmZlZBQ6azczMzMwqcNBsZmZmZlaBg2YzMzMz\nswocNJuZmZmZVeCg2czMzMysAj/cxHqVpLnAdyLiul4eZwpwekSc1Zvj9LbW1tYFOt9P7jMzM+sd\nDpptgUhaBfg1sDXwGeAV4CHgjIi4vZ/ntjFwKPBlYAXgOeBO4NSIeKI/59aehoaGBTq/qmoIkye3\nOnA2MzPrYQ6ardskrQ78E3gdGA1MApYEvgGcA6zbj3P7FvAn4EZgR+ApYGXg+8CxQH0/zm2JiPig\n7drjSN8/uqOVWbMamDZtmoNmMzOzHuag2RbE+cAcYJOImFUob5V0SVsnSFofOBPYDHgH+DNwUES8\nnevHAxMj4qDCOX8B3oiI3fP7TwKXAlsCLwFHlI3xsVx/fUR8r1D1LHC/pOULbUcCpwAbkoL/y4HD\nI2KupB8DR0fEZ8r6/xswNSL2zO+3A44kfUn4L3AFcEJEzMn1c4F9gW8CXwFOJQXubVgDqG27yszM\nzPqNbwS0bpH0cWAr4JyygBmAiHirjXOGADcDrwF1wPeArwJnd3H4y0mpICNzH/sCnyzUfwP4BCkY\nnk9pbpI+DdwA/BvYANgH2IOUbgLwR2AlSVsUrqF03Y35/ZfyfE4HhgF7A7sCh5UNexRwLfB5UkBv\nZmZmCxEHzdZdawECJnfhnJ2ApYFdIqI1Iu4A9gd2yavHFUn6HCko3jMi7o+IiaRAd0i49arCAAAg\nAElEQVTZ3OjE3PYDnouIAyLi8Xyz4lGkVBMi4k3gJlJ6R8n3SavMd+T3RwInRkRjRDwbEbflsn3K\nxroqIi6PiGci4oXOXKuZmZkNHE7PsO5SN84ZBjxUtjI9gfTlbR1gaif7eD8iWkoFETFZ0pvdmNsw\n4N6ysgnAspL+Xw5urwJ+J2nfiHifFED/vtB+Q2C4pF8XyhYHlpJUVbjW5s5NaQxwTVlZPf2Ygm1m\nZjZgNDU10dTUNE/Z9OnT+2RsB83WXU8AQQo8/9aD/c5l/qB3yS728Xj+O4yUerEgxpGC+m0kPQB8\nCfh5oX5Z0sryteUnln05eLtzw40mLcibmZlZufr6eurr511Iamlpoa6urtfHdnqGdUtEvEHKT94v\n33g3D0krtHFaK7BhWfsRpJsJS6kUU4FVC/0sBqxfaP8YsISkukKbdYAVC21uIeVNH9zW3AtzayXd\nkFg0AphRSqGIiNmkgLiBtNz7WEQ8WGjfAqwTEU+XH22NbWZmZgsnB822IPYjpSLcJ2l7SWtJGibp\nANJWdOWuAmYBl0taL99gdxZwRUSUUjNuJ63qbp2D4fMpBMQR8TgpWP+dpC/m4Pki0k4cpTbvAHvm\nfv4maUtJq0uqk3Ry7hPgPGA1SWdLWifvgnE0KUeifN7bALvn10XHknKyj5S0br7+H0o6rrMfopmZ\nmQ18Dpqt2yJiCml/tPHAacAjpFXerwOlLeOi0P5d0s4TKwH3AX8A/gH8tNDtpaTdKC4H7iDtr1z+\nkJTdSFu73UHai/lC4NWyuV0HDAfeIwW6rcDVwCrk7d4i4kXSpsibAA+SguiLgBPKxrudtB3d2rmP\n4ji3AN8Cvpav6V5S+sYzxWaYmZnZQk0R/v9zs/4mqRZoXtCHm0ADzc3N1NZ6r2czMxscCjnNdcWN\nAnqabwQ0G1COoOxZLV1SVTWE6urqnpuOmZmZAQ6azQaUxsZGampqun1+dXW1H6FtZmbWCxw0mw0g\nNTU1Tq0wMzMbgHwjoJmZmZlZBQ6azczMzMwqcNBsZmZmZlaBg2YzMzMzswocNJuZmZmZVeCg2czM\nzMysAgfNZmZmZmYVOGg2MzMzM6vADzexQUvSSOB24OMR8VY7bXYFzoiIj+f3RwHbRUSvPIGktbV1\ngc73EwHNzMx6h4PmRYiky4BdgAsiYt+yunOBnwCXRcTu/TC9+UgaD0yMiIPKyucJVHvRBGDV9gLm\ngii8PhU4q/RG0lhghYjYvicm1NDQsEDnV1UNYfLkVgfOZmZmPcxB86IlgOeAHSQdGBGzASQtDdQD\nz/bn5LooKjdZwAEiPgBe7eI57wDv9M6MAI4Dtu7mua3MmtXAtGnTHDSbmZn1MOc0L3omAs8DxZXP\n7UkB88RSgZJDJT0t6R1JEyV9t1C/mKSLC/WPSTqgOJCksZL+Imm0pBclTZN0jqTFe/KC8jjXlpWd\nnleqS+/HSzorl78u6WVJe0gaIulSSW9JekLSNwrnjJQ0V9LyhbLdJD0raaakPwOfKBv3KEkTS6+B\nXYHtcj9zJH1Z0m2Szi47r1rSbElbdHy1awC13TxqKn+YZmZm1i0Omhc9AVwKFFMwdgfGAiqUHQY0\nAHsB6wKnA1dK+lKuX4wUfH+XFI0dA5wg6Xtl420BfBYYRUoN2S0fC0KVmwDzr0bvAkwFNiGlUFwA\n/JGUhrExcAtwhaSqtvqQtClwcT53I2A88OsOxj0N+ANwE7AKsCrwz9xHvaQlC+fsDLwQEeMxMzOz\nhY6D5kXTVcAISatJWh0YDjSWKiUtBRwK7B4Rt0bEMxFxRT5vb0ipCxFxTERMjIhnI6IJuAz4QdlY\nrwP7R8TjEfF34AZgyy7MdT9JM4oHcH73LpuHIuI3EfEUcBIwC5gaEZfksmOBamCDds4/ALgxIsZE\nxJMRcQ5wc3uDRcTbwLvA7IiYGhGv5pSPa0mB/3aF5ruSvriYmZnZQsg5zYugiJgm6XrgR6Tg7YaI\neF36cAF3LWAI8A8VCoElmTeFY7/cx1DgY8BSxfrsPxFRXPF9CVi/C9NtBE4oK/suKajvqodLLyJi\nrqTXgEcKZa/ky125nfNrSAFv0b3AVl2ZRETMlnQlaYX/T5JqgfWAbbvSj5mZmQ0cDpoXXWOBc0ip\nBPuW1S2b/24NvFhWV7p5cAfSThEHAv8CZgAHA18sa/9+2fuga79gTI+Ip4sFkspvzpvL/CkbSzK/\ntuZSXkYX59ddFwMTJX2a9MXj9oh4vvJpY4Brysrq82FmZja4NTU10dTUNE/Z9OnT+2RsB82LrptI\nK8NzSLm8RY+SguPVI+Keds4fDkyIiAtLBZLW7I2JdsJU0kpt0UbAez08TiuwaVnZZhXOeQ+Y78bH\niJgk6QFSzng9839xacdoYKfONTUzMxtk6uvrqa+fdyGppaWFurq6Xh/bQfMiKqcnDMuvo6xupqTT\ngNPzThf3ACsAm5NWfq8EngB2lvR1YArpRrZNgHlWhfvI7cAvJO1MSpdoIKWAtPRA38UV7LOAeySN\nBv4GfIPKqRnPAF+X9DngNdLn90Guu4S02j8T+GsPzNXMzMz6iW8EXIRFxMyImNlO3RGkTYEPIa08\n30hK15iSm1xIyu/9PSk9YyXg3J6eYqcaRdyS53oycB8pveTyTvTVmbIP30fEv4Efk24IfBD4ah63\nIxcBk4EHSHs+Dy/UNQEfAFdHRE+vipuZmVkfUtkipJn1EEn/AzwJ1EXEQxXa1gLN6b7I7qZntAB1\nNDc3U1vbK0/5NjMzG3AK6Rl1EdETv0K3yekZZj1M0hKkre2OB+6tFDDPawrdzzpp7eZ5ZmZmVomD\nZusVkkaQUj6C+Xe+iIhYfv6zFhmbkx6M8hjw/a6dekQ+uqeqagjV1dXdPt/MzMza5qDZesv9wIb9\nPYn+EBF30s37BRobG6mp6f7jsKurqxk6dGi3zzczM7O2OWi2XhERs+mfnTYWajU1Nc5HNjMzG4C8\ne4aZmZmZWQUOms3MzMzMKnDQbGZmZmZWgYNmMzMzM7MKHDSbmZmZmVXgoNnMzMzMrAIHzWZmZmZm\nFThoNjMzMzOrwA83GaAkjQcmRsRB/T0X6zutra0LdL6fCGhmZtY7HDT3IUmXAbsAF0TEvmV15wI/\nAS6LiN2B/wPe74ExdwXGAgGojSYBrBERzy3oWAORpJeAoyLid304pkj/lrsBNcB7wBPAlcAlETGr\nvXMbGhoWaOyqqiFMntzqwNnMzKyHOWjuWwE8B+wg6cD8qGkkLQ3UA89+2DDizR4a8/fAjYX3fwEe\nAY7goyB6ag+N1WckLR4Rc/pwvCUjorNfYv4AfAM4GhgPTAM2Bg4iBc+3tH/qccDW3ZxlK7NmNTBt\n2jQHzWZmZj3MOc19byLwPLB9oWx7UsA8sVQgabyk3xbeT5F0qKRLJL0l6VlJP640WETMjohXSwdp\n1fOdiJhaKI88xk8kPSbpXUn/kbRnYfx1JM2V9H+SJkh6R9K9kv5H0nBJLZJmSLpO0oqF85rycZyk\nqZLelHSWpMUKbRaTdGS+xrclNUv6dqF+qzz21yRNlDQbqMtzGifplfyZ3CtpZOG8e4FVgPPz+e/k\n8pNyHYW2v5LUWnhfmvfRebX6wVxeJekMSf/N13uPpOGF83bJ/57/FxFjIqIlIp6LiL9FxEhgnnHn\ntwZQ282jpuOuzczMrNscNPe9AC4Fdi+U7U5KoWgrfaLoIOB+YCPgPFIwuHZPTErSHsCvgF8Aw4Aj\ngVMkfb+s6dHAr4E60i8Vvyctj+4NfBlYn7SKXbQNsBrwJWBnYEfg0EL9McB3SZ/DevnarpH0xbJ+\nfgP8nBQdPgYsS1o5H0mKGu8ExklaJbffmrSKfjDwKWD1XB75KFdetg3waWCLPD+Ai4ANSYHxBsD1\nwC2SSku7OwIPR8StbfRPRMxoq9zMzMwGNqdn9I+rgJMkrUb64jIc+CEpOOvIDRFxQX59sqQD8zlP\n9MCcjgb2j4jr8/tnJW0E7AP8sdDuxIgYDyDpHNIXgOERcX8uuxzYrqzvmcCPc3rDY5KOIwXeJ0ha\nBhgNbBYRD+X2l0gaBewF3JfLAjg0Iu4s9Nucj5JDJH2XFOxeGhFvSJoLzMir7F31OrBPRMzN17YW\n6d/pUxHxem5zkqRtSLnqxwNrF+ZsZmZmiwgHzf0gIqZJuh74EWl1+YaIeD3dP9ahR8revwysvKDz\nkbQS8BmgsWwOi+cxSqJsDq/kv5PKysrn1FKWD3wvsJKkT5JWoKuAuzXv4EsyfypDMUBG0vLAscBW\npJXkJXJfPZXQ+1ApYM42IH0mz5TNdSk++uJS8R+xY2OAa8rK6vNhZmY2uDU1NdHU1DRP2fTp0/tk\nbAfN/WcscA4pEN23QtuS8hvRgp5JsVk2/90FeKis7oMO5hDtlHVlTsvmc7YEXiurK99l4u2y92cB\nm5LSL54G3iWlSyxVYcy5zB/cLtlGu/LxlgVmk9Izys8vpV08Tkpv6abRwE7dP93MzGwRVl9fT339\nvAtJLS0t1NXV9frYDpr7z02k4G4OHe6m0CeeJ+3w8NmI+Esv9F8raYmIKAXgmwGvR8RUSR+QAvOh\npRSPLhgOXBgR4wDyDYirlbV5j7Q6XDQVWLWsbONOjNcCLA2sFBHN7bS5Ghgr6WsR8Y/ySknLOa/Z\nzMxs4eMbAftJ/tl/GLBeafeKfpxLkG7GOzLvoLG2pM9L2kPSfoWm3U09WAb4naRhkrYFDgfOzGO/\nQVoxPkfSTpI+K6lW0gGSdqjQ7xPA9/NcNyblipdvQ/cMMErSqjkNBdI2cKtJ+pmkNSX9nMr55ETE\nJOBaoEnSt/POIZtKOlzSlrnZlcDfgD9L+mW+ltVz+ztIXxjMzMxsIeOguR9FxMyImFle3F7zTpZV\nHLaduZwL7E+6+e5h4HbSThBTFnA8gL8DLwH3kILK3wMnFsb+JXAK6ebAR4EbgK+TAt6OHEBKybgX\n+DMpoH20rM3hpN02pgAv5PEeAn5GyoVoIe3YcUYnr2VH0j7MZ5B28PgTKV2j1HeQdto4BPg+cBdp\nK8HDgXGkHT7MzMxsIaN+XuS0RZykJlIsuWN/z2Ugk1QLNC/ow02ggebmZmpra3tucmZmZgNYIae5\nLiJaemsc5zSbDShHMP82151XVTWE6urqnpuOmZmZAQ6aFwmSDgUOa6f6rojYpi/nY93X2NhITU33\nn+xXXV3tR2ibmZn1AgfNi4bzmX9z35J3+3Ii5SLCGwx3QU1NjVMrzMzMBiAHzYuAiHgTeLO/52Fm\nZma2qPLuGWZmZmZmFThoNjMzMzOrwEGzmZmZmVkFDprNzMzMzCpw0GxmZmZmVoGDZjMzMzOzChw0\nm5mZmZlV4H2azQaQ1tbWBTrfTwQ0MzPrHQ6ae5Gky4BdgAsiYt+yunOBnwCXRcTuvTiH1YEpQABq\no0kAP4qIK3prDv1JUhMQEbFjH4/7deAgYBOgCnga+DtwekS83N55DQ0NCzRuVdUQJk9udeBsZmbW\nwxw0964AngN2kHRgRMwGkLQ0UA882wdzeA74VOH9L4GtgC35KIie3gfz6FGSloiID/pwvMVIwXd0\nou0BwOnAhcDxpH/nNYDdgP2BX7d/9nHA1t2cZSuzZjUwbdo0B81mZmY9zDnNvW8i8DywfaFse1Ig\nNbFUIGkrSXdLekPSNEnjJH22UL+zpBmS1iyUnSfpUUlV7Q0eyaulA5gJfBARUwvlpWB+C0kTJL0j\n6RlJpxX7lvSSpF9KulrSTElPSfqGpE9Juj6XTZS0YeGcvfN538/t381ti4E8kr4n6cFc/7ikQ3Og\niqSlJc2VtIekGyTNBA6StJSksXmu70hqlfSTQp8nAj8EfpjPnyPpi/mznitpqULbTXPZymXz3l7S\nY8As4JO57ieSHstz/Y+kPQv9rAGcBpwYEftGxD0R8XxE3JV/UTilvX+rZA2gtptHTcddm5mZWbc5\naO59AVwKFFMwdgfGMm+6xDLAGFL08xVgDvCXDzuJuBK4Abha0mKStsn97BgRsxZ0kpJqgOuARmA9\nYCfgq3lORb8AbgY2BG7P7S8FLgI2Av6b3xetCBwI/AAYQVr5vrIw9ldJq7InkyK//YG9gdFl/RwL\nXJ3ndxXpl5KngO/k834DnCbpW7n98cDf8rEKsCrQnOvaWjEuL1sR+BmwM/B54A1JewC/yp/DMOBI\n4BRJ38/n7AAsDpzaRv9ExFttlZuZmdnA5vSMvnEVcJKk1UhfVIaTVkC3KDWIiGuLJ+TVy1clrRsR\nj+bifYCHgLNJq9VHRcSDPTTHw4CLI+L8/H6KpNHA3yX9NCLm5vK/RMTleY7HA3sAd0bE33LZqcDt\nkpYvBIhLAXtFxKTcZg9goqT1c9lRwLER0ZTbPyPpuDynYvA5NiKuKpv38YXXV0r6Mik4vz4i3pY0\ni7TgPrXUSGortbtNSwF7RMSThXOPBvaPiOtz0bOSNiIF+X8E1gJejYiFLuXFzMzM2ueguQ9ExDRJ\n1wM/Iq0u3xARrxeDN0lrA8cAmwLVpOA6gKHAo7mfN3MwfTMwISJO7sFpbgisVUw1yHNdHFiNj/Kv\nHynUv5L/TmqjbGWgFDS/WwqYASLiIUnvklaHJwEbALU5CC9ZHFiilKKRNVNG0s9JK8FDSTfcLQXc\n2/GldtrMsoB5JeAzQGNZ4L04ULq5r9MRuZmZmS08HDT3nbHAOaRAeN826seRdrnYE3iRFDT/hxQE\nFo0EPgBWlbRMRLzdQ/NblrSCfWEbdS8UXr9feB0dlHUq9Ucp+lwG+Dlpd4l5RMTcQoA6z7VK2o20\n0vwz4AFgBnAEsE6FYUur5sUAd8k22pV/tsvmv7uQVvyLSjclPg7sKmmF7q02jwGuKSurz4eZmdng\n1tTURFNT0zxl06f3zY+7Dpr7zk2kAHgOcEuxIq9gfo6UCjAhl40o70DScNLuF9uS8n/PJe3I0BNa\ngHUj4uke6q/oY4VUDPKNglXAoxERkh4E1omIc7rY73BgfERcUiqQtFZZm/fyWEWlVI1VgWfy6407\nMd7zwDTgsxHxl3ba/IG0BcbBwOHllZWD6dGkdHIzMzMrV19fT339vAtJLS0t1NXV9frYDpr7SF4x\nHZZfl99w9gbwGrCXpJeB1YETKdyYJmk54ArgzIi4WdJ/gfskjYuIP/fAFH8DTJD0W+Ay4F1gfeDL\nEXFgF/sqT1F4D7ggp1IIOI8U7P4n1x8D/FHSS3x08+NGwOci4pgOxnkC2F7SV0gB7R6kG/YeLbR5\nhrTl31qkz/mNXP8KcKykY0k3Fh5Q6aJygH8M8BtJ7wC3kgLyLwJVEXFuRDwt6ZekGxJXIuWzP0f6\nN90NeIkOt5wzMzOzgci7Z/ShiJgZETPbKA/Srgt1pJzhMaTdGeCjwPkMUvrB4fmcSfn1BZJW7YG5\ntQCjSEHnPaR0h1+TgtEPm7V1aifK3gDOJK3C3kkKWHcujD0O+D/SCvoDwATgp6R0lY7GOZuU0vFn\n4J+kAPaisjbnk4LWicCrwBci4j3S570RKc3iANpYFW5LRJxL2t1jL+Bh0g4iOxbnGhFnkDZbXoO0\nc8ejeR4zSL8OmJmZ2UJGnXhWg1m3SdqbtMvHp/t7LgOZpFqgOe3g1930jBagjubmZmpra3tucmZm\nZgNYIT2jLi8C9gqnZ5gNKFNIwW93tPbkRMzMzKzAQfMiIN80eCMphaE8nzgiYvm+n5V1zxH56J6q\nqiFUV1f33HTMzMwMcNC8qLiftM/ygBMRF9L2NnbWhsbGRmpquv847OrqaoYOHdqDMzIzMzNw0LxI\niIjZQG9sFWd9rKamxvnIZmZmA5B3zzAzMzMzq8BBs5mZmZlZBQ6azczMzMwqcNBsZmZmZlaBg2Yz\nMzMzswocNJuZmZmZVeCg2czMzMysAu/TbDaAtLZ27lHYfoiJmZlZ33LQbIOWpJHA7cDHI+Ktdtrs\nCpwRER/P748CtouIXnkCSUNDQ6faVVUNYfLkVgfOZmZmfcTpGYOMpMskzZV0Xht15+a6S/toLuMl\n/baN8l0lvdEHU5gArNpewFwQhdenAluW3kgaK+nanpvScUBzhaORWbPeYdq0aT03rJmZmXXIK82D\nTwDPATtIOjA/ghtJSwP1wLP9ObmCqNxkAQeI+AB4tYvnvAO80zszAlgD8GO0zczMBhqvNA9OE4Hn\nge0LZduTAuaJpQJJW0m6W9IbkqZJGifps4X6nSXNkLRmoew8SY9Kquqpyba1mivpdEnjC+/HSzor\nl78u6WVJe0gaIulSSW9JekLSNwrnjMwr68sXynaT9KykmZL+DHyibNyjJE0svQZ2BbbL/cyR9GVJ\nt0k6u+y8akmzJW3RU5+LmZmZ9R0HzYNTAJcCuxfKdgfGAiqULQOMIS19fgWYA/zlw04irgRuAK6W\ntJikbXI/O0bErAWYnyo3SVMoe78LMBXYBDgLuAD4IykNY2PgFuCKsoD+wz4kbQpcnM/dCBgP/LqD\ncU8D/gDcBKwCrAr8M/dRL2nJwjk7Ay9ExHjMzMxsoeOgefC6ChghaTVJqwPDgcZig4i4NiL+GhFT\nIuJhYE/g85LWLTTbB/gUcDYpWDwqIh7swjz2y6vVHx7A+d28poci4jcR8RRwEjALmBoRl+SyY4Fq\nYIN2zj8AuDEixkTEkxFxDnBze4NFxNvAu8DsiJgaEa/mlI9rSYH/doXmu5K+lJiZmdlCyDnNg1RE\nTJN0PfAjUoB3Q0S8Ln20yCtpbeAYYFNSsLkYaZV1KPBo7udNSXuSgssJEXFyF6fSCJxQVvZd4NAu\nXxQ8XHoREXMlvQY8Uih7JV/fyu2cX0MKeIvuBbbqyiQiYrakK0mr7n+SVAusB2xb+ewxwDVlZfX5\nMDMzG9yamppoamqap2z69Ol9MraD5sFtLHAOKRDet436ccAU0grzi6Sg+T/AUmXtRgIfAKtKWiav\nwHbW9Ih4ulggqfzmvLnMn7KxJPN7v+x9tFEGffMLy8XAREmfJn0xuT0inq982mhgp96dmZmZ2UKq\nvr6e+vp5F5JaWlqoq6vr9bGdnjG43UQKgJcg5ft+SNJKwOeA4yNifERMpuymuNxuOPBL0irqTODc\nXpjnVFK+cNFGvTBOK2lVvWizCue8ByxeXhgRk4AHgL1Iy8SX9MQEzczMrH84aB7EImIuMAxYLyLK\nb6p7A3gN2EvSmpK+QsodKN44txxwBXBmRNwMNAA/kPTdHp7q7cAX8m4da0k6Gli/h/ourmCfBXxD\n0ug8zv5UTs14BthA0uckfUJS8debS4BD8uu/9tB8zczMrB84aB7kImJmRMxsozyAHYA6Ul7wGOAX\nper89wxgBnB4PmdSfn2BpPKV4TaH7+QcbyE99eNk4D5gWeDyTvTVmbIP30fEv4Efk24IfBD4ah63\nIxcBk0mryq+SbqgsaSKlrVwdEe9V6MfMzMwGMM2/wGhmPUHS/wBPAnUR8VCFtrVAc4rRt67QcyvQ\nQHNzM7W1fhCKmZkNboWc5rqIaOmtcXwjoFkPyyka1cDxwL2VAuZ5HZGPjlVVDaG6urqbMzQzM7Ou\nctBsvULSCOBGUvpD+c4XERHLz3/WImNz0oNRHgO+35UTGxsbqampqdiuurqaoUOHdm92ZmZm1mUO\nmq233A9s2N+T6A8RcSfdvF+gpqbGKRdmZmYDkINm6xURMRt4umJDMzMzs4WAd88wMzMzM6vAQbOZ\nmZmZWQUOms3MzMzMKnDQbGZmZmZWgYNmMzMzM7MKHDSbmZmZmVXgoNnMzMzMrALv02w2gLS2tnaq\nnZ8IaGZm1rccNNugJekfwJyI+EZZ+b7ACcB6EfFiX86poaGhU+2qqoYweXKrA2czM7M+4qDZBrMf\nAQ9L+nFEXAQgaQ3gZGDv3gqYJS0eEXParj0O2LpCD63MmtXAtGnTHDSbmZn1Eec026AVES8APwfG\nSFo9F18C3BQRVwNI+rKkeyS9I+kZSb+V9LFSH5J2kfSApBmSXpJ0paTqQv2WkuZK2kpSs6TZwKbt\nz2oNoLbCUdNzH4KZmZl1ioNmG9Qi4grgVmCspP2BdYG9ASR9DrgBaALWA+qBUcAZhS6WAA4DPg98\nB1gTuLiNoX4D/AIYBvynFy7FzMzMepHTM8xSkPwf4EvA9hHxei4/FLgsIs7N76dIOgj4h6T9IuKD\niLi00M8zuX6CpKUjYnah7vCIGN/bF2JmZma9wyvNNuhFxFTgQqA1IsYVqjYE9sypFzMkzQCuBwSs\nDiBpE0njJD0r6S3SqjXAasUhgOZevxAzMzPrNV5pNks+yEfRssC5+VBZ3XOSlgNuAq4DdgReBdYi\nBdZLlbV/u3PTGANcU1ZWnw8zM7PBrampiaampnnKpk+f3idjO2g2a18Ladu5KW1VSqoBVgQOiYhX\nctnmCzbkaGCnBevCzMxsEVVfX099/bwLSS0tLdTV1fX62E7PMGvficBISWdK2kDSWpK+I+nMXP8s\n8D7wM0lrSPoOKQ/azMzMFjEOms3aEREPASNJO17cQ8pLPhJ4Ide/AuwO7EC6kfAg0lKxmZmZLWKc\nnmEGRMQxwDFtlN8PbNXBeVcDV5cVL16ov6343szMzBZODprNBpQppFTqjrT2xUTMzMyswEGz2YBy\nRD46VlU1hOrq6ortzMzMrGc4aDYbQBobG6mpqfyY7OrqaoYOHdoHMzIzMzNw0Gw2oNTU1FBbW9vf\n0zAzM7My3j3DzMzMzKwCB81mZmZmZhU4aDYzMzMzq8BBs5mZmZlZBQ6azczMzMwqcNBsZmZmZlaB\ng2YzMzMzswq8T7PZANLa2rlHZPvhJmZmZn3LQfNCTNLqwBRgo4h4eAH6OQr4TkRs3GOT6yeS5pKu\n5bp+nMNYYIWI2L6r5zY0NHSqXVXVECZPbnXgbGZm1kcGdXqGpMskzZV0Xht15+a6S/tgHidKai0r\nW6et8SXtJmmWpKWB54FPAZN6YBrRybl+LM/3SUnvSnpV0nhJ2/bAHHpc/gy/XX6s5ZEAACAASURB\nVHi/hKQmSc9LWrc/59a244DmCkcjs2a9w7Rp0/ptlmZmZoPNYF9pDuA5YAdJB0bEbIAckNYDz/bR\nPMYDB0taOSJezWVb5LmNKms7Cri3NFfgVfrWhcAmwH5AK/AJYHj+O6BJ+hhwLbAmsHlEPNfPU2rD\nGoAfo21mZjbQDOqV5mwiacW2+FP69qSAeWKpQNJWku6W9IakaZLGSfpsoX5nSTMkrVkoO0/So5Kq\nKszhHuAD5g2QRwHnAitJGlpWPj73v3peSd0gvx+Z339F0v2S3pY0QdLaxcEkHSLpZUnTJV0MVJpf\n0bbAbyLi5oh4LiImRsS5EXFZof8pkn4t6WpJMyW9IGnfsjmsIOnivFI9XdKtpesotNlOUnNe0X5S\n0pGSFivUryXprlw/SdJX25u0pBWAW4FVKATMhc/w+7mvdyTdJ2ltSZvkz3GGpL9L6tIXA0mjJb2Y\n/3s5R9LiXTnfzMzMBg4HzWm1+VJg90LZ7sBYQIWyZYAxpGXArwBzgL982EnElcANwNWSFpO0Te5n\nx4iY1eEEIt4B7ietLpeMAm4DJpTKc5A+lBw0F+Zf7njgQKCOFIx/mOIh6QfAUcAhwBeAl4B92+ij\nPS8DW0tatkK7X5C+dGwEnAScKWnLQv2fSKvTW5E+0xbgVkkr5nl+CbgcOB0YBuwN7AocnutF+vxn\nkVa+9wFOpu3PY1XgTtK/2aiImNpGm6OBY4GNSZ/Z1XnePwVGAGvl+s76CvBZ0r/jLsBu+TAzM7OF\nkIPm5CpghKTV8s11w4HGYoOIuDYi/hoRU/JNd3sCny/Li92HlGN8NnAxcFREPNjJOYwnrzTnPpcm\nBZ1389EK9CjgXeBfhfOKgT2koPGwiLgnIh4jBX7DJS2V638GXBQRl0XEExFxBPBoJ+cIsBfp83kt\nr8j+VtLwNtpNiIhTI+LJiDiHFCQfmK9vBClg/0FeqX4qIg4GpgPfy+cfCZwYEY0R8WxE3JbL9sn1\nXwM+B+wcEZMi4h7gsDY+D4AzgSWBr0fEW+1c16kRcWtETM7ta4FjI+JfEfEQcAnzfqmp5HVg/4h4\nPCL+TvpCtWWFc8zMzGyAGuw5zQBExDRJ1wM/IgVdN0TE62kxM8kpDscAmwLVpC8cQVr5fTT386ak\nPYGbSUHjyV2Yxh3AYZJWAUYC90RESLqTtMpKLv9nRLxfoa9HCq9fyn9XBl4AaoDzy9rfy/y5022K\niLvzivf/koLnLYG7JR0ZESeU9Vk+xs/y6w2A5YB5PmNSmkgp5WVDUrD/60L94sBSOd1lGPB8RLzS\nwZgl44DvkALuM9ppU/zMSn1OKitbuZ1z2/KfiCiuer8ErF/5tDHANWVl9fkwMzMb3Jqammhqapqn\nbPr06X0ytoPmj4wFziEFwm2lK4wjbe+2J/AiKWj+D7BUWbuRpJ/3V5W0TES83cnxJwDvkVYztyCl\nE0BK26iWtAYpsL2gE30Vg+pS4NZjvypExBzSfCcAp0o6HDhC0skR8UEnuliW9BmOZP6V4TcLbY4k\n3bhXbnYbZR25ErgOGCtJEXF6G23a+szKy7ryGZZ/senk+aOBnbowjJmZ2eBRX19Pff28C0ktLS3U\n1dX1+tgOmj9yEykAngPcUqyQtBIpFWCPiJiQy0aUd5DTFH5JulnuZNKNfLt1ZvCImCXpPlLA/GXg\nlFz+gaR/AXsA/49585m7o5W0Wl5MP/nfHuhzCdJK8cx2+vzf3A5S/vKngDkd7GDRAqwTEU+3Vam0\nRd9qklYprDZvRjtb50XElZKCFDgvFhFjitXtX5qZmZmZg+YPRcRcScPy6/Ig6g3gNWAvSS8DqwMn\nUgi2JC0HXAGcGRE3S/ovcJ+kcRHx505OYzwp7zdIQWPJXaQb694mrTx3pK2c3mLZmaTAsZm0UtwA\nrAc81ZkJShoPNAEPkD6T9YATgNsjYmah6eaSfgH8Dfg6KVd5a4CIuFXSvcBfJf0KeBz4TK6/NiJa\nSDfdjZP0PCkfei4pZWP9nId9K/AEcIWkXwIrkG6AbFdENCo9/OTyvOJ8WhufDx2UmZmZ2SDlGwEL\nImJmWeBXKg9gB9JuFI+QEk9/UarOf88AZpB3d4iISfn1BZJW7eQUxpPSEu6JiLmF8jtz+d05NWKe\n6VV4P09ZRPyB9ASNk0mB72rAfA936cBNpN0gbiblcp8J3Aj8sKzdGNLNfhNJN+gdGBG3Fuq3Jn0Z\nuBSYTNqtYig5nzgibgG+Rbrh7z5SvvLPgWdyfZDylKuAfwO/y+O0e+35vKuBnYHf5GB7vjYdlJmZ\nmdkgpfkXVc0WjKQpwOkRcVZ/z2VhIakWaE7fZ7au0LoVaKC5uZnaWj8IxczMBrdCTnNd/rW6Vzg9\nw2xAOSIfHauqGkJ1dXXvT8fMzMwAB819It80eCPpJ//59lWOiOX7flZtkzSDduYJfLN0I2QFi/TP\nFz30GbWpsbGRmpqaiu2qq6sZOnRoxXZmZmbWMxw09437STexLQw6mud/O9NBRHy2cquF2gJ/Ru2p\nqalxyoWZmdkA5KC5D0TEbKDNrdMGmva2eLOP+DMyMzMbfLx7hpmZmZlZBQ6azczMzMwqcNBsZmZm\nZlaBg2YzMzMzswocNJuZmZmZVeCg2czMzMysAgfNZmZmZmYVeJ/mRZCkXYEzIuLjvTzOWGCFiNi+\nF/qeApweEWfl93OB70TEde20Xx2YAmwUEQ/39HzaGfM7wKnA/wBnR8RBC9pna2trp9r5iYBmZmZ9\ny0HzACWpGjgO2BpYBXgDeBA4NiLu7UQXff4o6/aCaEkjgfHAihHxVje7/xTpM+hIX1/zBcAlwFnA\nzJ7osKGhoVPtqqqGMHlyqwNnMzOzPuKgeeC6lvTvszNpBXUVYEvgE/05qQWwQAFtRLzaiWZakDG6\nQtKywMrALRHxSs/1XPqe1JFWZs1qYNq0aQ6azczM+ohzmgcgSSsAI4BfRcRdEfF8RDwQESdHxPWl\nNpIulPSypHclPSxp67J+vi7pUUkzJN0oaZVCnSQdKel5SbMkTZS0Vdn560u6TdI7kqbl8ZbpoWv8\nrqRJeewpkjpMbZA0V9K3C++/KKklX/t9wMYUAnNJi0m6WNLTef6PSTqgUP8lSe9JWrlsnDMk3Vlh\nLiOBt/J44yXNkTRK0quSti+0e1DSfwvvR+TrrWq/9zWA2gpHTUfTMzMzs17goHlgmpmP70haqrxS\nkoCbgM2AHUlR1C+BOYVmywCjgZ2ALwFDgdMK9T8HDgQOAj4P3AxcJ2nNPMaQXPYaUAd8D/gqcHY3\nrmeeFWBJdcA1wNXA+sBRwHGSdulUZylwHwdMIkWRR5ddG6T/tp8Hvkv6fI4BTpD0PYCIuBt4irSS\nX+p3CdLneUmFKUwA1snX9X/AqrnsLmBU7mtFYBjwMUmfy+d9GbgvImZ15jrNzMxs4HB6xgAUEXPy\nzXwXAT+R1ALcCfw+Ih4BvgZ8ARgWEU/l054p62YJYO+IeAZA0jnAEYX60cBJEfHH/P4QSVuQgumf\nkoLtpYFdcpDXKml/YJykX0XE1Hamv62kGWVli5e9PxC4NSJ+k98/KWk9UuB/RTv9Fu1EClj3jIj3\n8txWA84rNYiID0iBcsmzkoYDPwD+lMsuBX4EjMnvv52v+Y90ICI+kFRKF3mjlDoi6Q5gr1z+ZaAF\neJkUSD+e/3a4im1mZmYDk1eaB6iI+AvwaWBb4EZgJNCcg+kNgRcKAXNb3ikFzNlLpBxcJC2X+/5n\n2TkT+Oi3/2HAQ2WrohNI/82s08G4twMb5DmWjj3L2tTkvsrHXjuvolcyDHg4B8wl890cKWk/SQ/k\ntIkZpIC2mAR8WR7zi/n9rsAfIuLdTsyhLXcC60r6BOnf6458jMqr2MPzezMzM1vIeKV5AMtB4W35\nOEHSRaTV0/JUhLa8X94dfXOj3NsRMaVYkFeB+5SkHUjbwR0I/AuYARwMlAJkImKqpHHAjyQ9A3yT\ntELcLRHxiKTXSSvKI4HDgFeAQ4BNSP97K/+iUmYMKXOlqD4fZmZmg1tTUxNNTU3zlE2fPr1PxnbQ\nvHBpBbYDHgJWk7RWRDzZ1U4iYoakF4HNgbsLVZuTAszSWLtK+lhh5XUEKW96cncvoND35mVlI4DH\nI6Izu2y0Ag2SliqsNm9W1mY4MCEiLiwVlPK1y1wMNAH/BZ6MiH+10aYr7iH9G62bX79LSvnYG3ig\n8ip2KQ3dzMzMyv3/9u48vq6q3vv450sFa6ngEAG5UEBEiArVRh6UQVBAEEWEq2igCOJFUa8+CD6A\nQwHhqgwKUq4MVwSU1DiBCDJUhgKCKJiUS5VQhhYZCpQylEIHoP09f6wV3Nk9J+ckzcnU7/v12q+c\nvfbaa9gn3fl1nbXXaW1tpbW150BSZ2cnLS0tDa/b0zOGIUlvyKtWHChpa0mbSvokac7vZfkhtpuB\nSyTtlo/vKelDfajmNOAYSftLepukk0lTKabm49OApcDPJL0jz3eeCvy8l/nMvXar8PqHwK6Svi1p\nizzl5Mu5TfX4BWnk/HxJzXnVkKNKee4D3pNXENlC0omk0d6y6aSVML5FmuO8qm4kDQvfGRGL838C\nbiZFwp7PbGZmNkI5aB6enieN+B5BCrRmkaZlnEd6SA9gP+AOUgD5D+AUVn7grjdTgdNJUz3uAj4E\n7N09TzqPiO4BvAG4Hfg1cG2h/r56ZQQ5ImaSHsj7FKlvJwDfjoiLK+WvcP4LpLne7yQ9bHcSaepF\n0Xmkta5/SbqWbwB+vFKjUlB7EenaXVw+Xm+fCm4i/buaUUi7Mafd2MfyzczMbJhQfZ+Gm41eks4H\nmiLi40PYhklAB7RRe3pGJ9BCR0cHkyZNanzjzMzMhrHC9IyWiOhsVD2e02yrLUnrkFb6OAD46BA3\nJ5tLCop70zUYDTEzM7MCB822Ovs9aZ7z2RFxQ/GApKtIXwpTFsD3IuLkxjRpCj2X065s7NhxNDU1\nNaYJZmZmthIHzbbaiogP9HL4c8Brqhx7ugHNAaCtrY3m5tpfk93U1MSECRNq5jMzM7OB4aDZrIKI\neGwo6m1ubvY8ZTMzs2HIq2eYmZmZmdXgoNnMzMzMrAYHzWZmZmZmNThoNjMzMzOrwUGzmZmZmVkN\nDprNzMzMzGpw0GxmZmZmVoPXaTYbRrq66vuKbH+5iZmZ2eBy0Gw2jEyePLmufGPHjmP27C4HzmZm\nZoPE0zNsRJN0oaQVkpbnn92vrxrENhwvaebAlHYS0FFja2Pp0sUsWLBgYKo0MzOzmjzSbKPB1cAh\ngAppywa5DTEwxWwG+Gu0zczMhhuPNNtosCwinoyI+YVtoaRpkn5ZzCjpVZKelDQ570vSNyTNkbRY\n0kxJ/17Iv3Mevf6gpDskvSDpVklb5OMHA8cDEwuj3J/Jx06Q9E9JSyU9IulHg3dJzMzMbCB5pNlG\ns2nAryWNi4jFOW1P4DXApXn/m8ABwOeB+4H3AxdLmh8RfyqU9V/A14AFwHnABcBOwK+AdwJ7ALuS\nRrsXSvoEcASwP3A3sAEwsUH9NDMzswZz0Gyjwd6SFhX2A/gecBqwGNiXFEADtAKXR8RiSWsB3wB2\njYi/5uMPStoJ+ALwp0J534yIWwAknQz8QdJaEbFU0vPAyxHxZHcDJG0MPAZcHxHLgUeAvw14z83M\nzGxQOGi20eAG4HB6zml+OiKWS/o1cCAwTdI4YB/S6C/AW4FxwLWSiueuCXSW6phVeP1Y/rkeKRiu\n5Dekkea5kq4BrgKuyAF0L35IGrwuas2bmZnZ6q29vZ329vYeaQsXLhyUuh0022jwQkTMrXJsGnCj\npCbSFIrFwPR8bHz+uRcwr3Re+UHClwqvux/6q/pMQEQ8IultwG7A7sCPga9L2rn3wPkoUoxvZmZm\nZa2trbS29hxI6uzspKWlpeF1O2i2US0ibpP0MPBp4MPAbwpB692k4HiT7qkX/fQiMKZC3cuAK4Er\nJZ0N3ANsDdy5CnWZmZnZEHDQbKPBqyWtX0p7OSKeyq/bSdM3tgA+0J0hIp6X9APgDEljgFuAdYEd\ngIURcXHOWpy6QYW0B4HNJE0kTddYRJpPMQb4K2l0+6D885/97aSZmZkNHQfNNhrsycrTK2YDb8+v\np5FWyXgwIv5czBQRUyTNB44F3gI8S5rP/L1itgp1FtMuIT1sOIMUdH82l3MsaZLyGNKc6I9GxDN9\n7ZyZmZkNPQfNNqJFxGdJQWpvee6hwvSJwvGzgLOqHLupfG5E/G8xLSJe5F8PFxb9vrd2VTaXlZ9B\nLOvqe7FmZma2Shw0mw0rU/LWu7Fjx9HU1NT45piZmRngoNlsWGlra6O5ublmvqamJiZMmDAILTIz\nMzNw0Gw2rDQ3NzNp0qShboaZmZmVVF1n1szMzMzMEgfNZmZmZmY1OGg2MzMzM6vBQbOZmZmZWQ0O\nms3MzMzManDQbGZmZmZWg4NmMzMzM7MaHDSbmZmZmdXgLzexPpE0A5gZEUcOdVsGgqTjgX0iouo3\niki6EFg3IvbL+w27Bl1dXXXl8zcCmpmZDS4HzYaki4DPAOdGxJdKx34MfBG4KCIOBfYFXhrAulcA\nH4+Iy0vpPQLVBjoNmNrHc3pcA0lzgTMioq/lrGTy5Ml15Rs7dhyzZ3c5cDYzMxskDpoNIICHgE9L\n+lpELAOQ9GqgFfjnKxkjnh2aJjZGRCwGFvfxnAZeg5OAvWrk6WLp0sksWLDAQbOZmdkg8Zxm6zYT\neBgojuzuRwqYZ3YnSJoh6fTC/lxJ35D0U0nPSfqnpMMGunG5nq+W0mZKOq6wv0LS5yVdIekFSXdL\neq+kzXO7n5d0q6TNCuccL6nYvzUknS7pGUlPSjoFUKneV65BnqqxCXBGrn+5pHGSFkrar3Tex3Mb\n1q7e082ASTW25r5cOjMzMxsADpqtWwAXAIcW0g4FLqQUNFZwJHAH8C7gbOAcSVs0opF1+DZwETAR\n6AJ+AZwLfBdoIfXlv0vnROH110lTVQ4BdgTeQJqOUc1+wCPAFGAD4M159PqXwGdLeQ8Bfh0RL/St\nS2ZmZjbUHDRb0TRgR0kbS9oE2B5oq+O8KyPi3IiYExGnAAuAD/Sh3nZJi4obcEDfmw/ABRFxSUTc\nD5wKbAq0RcR1ETEbOBPYpZfz/y/wvYj4fc5/OLCwWuaIeAZYDjwfEfMjYn4+dD6wh6T1ASS9iTTv\n4oJ+9svMzMyGkOc02ysiYoGkP5BGSEUKhp+Wag00M6u0/ziwXh+qPgK4vpR2Kv37T12xLU/kn38v\npY2VND4ini+eKGkd4M3A7d1pEbFc0t/62oiIuEPS3cDBpL4cBDwYEbf0tSwzMzMbeg6arexC0vSF\nAL5UI2+38moaQd8C3iciYk4xIY82r1tIWsHK00TWrNGW6CVtMD5lOZ90DU8lTc2oY5T5h8CvSmmt\neTMzM1u9tbe3097e3iNt4cKqHwgPKAfNVnYNsBZpysEfh7gtRU+SRoGBV0aFN6ue/RVRO0vOGPGc\npMeA7YBbcj1jSHOhO3o59UVgTIX0NuAUSV8hPb3389qtOAo4sN4mm5mZrVZaW1tpbe05kNTZ2UlL\nS0vD6/acZushIlYAWwHviIi6A85BcANwkKQdJW1Netjv5TrOqzS3pLf5JmcCx0raR9KWpAcbX1ej\njgeB90vaUNIbuxPz0nS/I60FPT0i5tXRXjMzMxuGHDTbSiLi+fJ8X6qP2FZK70uwXW/e7wM3AVfk\n7XfAA/1sS291/hC4mBSU/xl4Dri0xvnHkR44fACYXzr2U9LIvR8ANDMzG8E0vAYTzUYXSQeRAvEN\nI6LqyLikSUBHmtFRa3pGJ9BCR0cHkyZV/fZvMzOz1UJhekZLRHQ2qh7PaTZrAEmvATYEjiF9PXk9\nU0mAuaSguDddq9Q2MzMz6zsHzdYwkr4BfLPK4Zsj4iOD2Z5BdjTwLeBG4OT6T5uSt96NHTuOpqam\n/rXMzMzM+sxBszXSOay8flq3JYPZkMEWEd8BvtPX89ra2mhurv012U1NTUyYMKE/TTMzM7N+cNBs\nDZNXj3h2qNsxkjQ3N3uespmZ2TDk1TPMzMzMzGpw0GxmZmZmVoODZjMzMzOzGhw0m5mZmZnV4KDZ\nzMzMzKwGB81mZmZmZjU4aDYzMzMzq8HrNJsNI11d9X1Ftr/cxMzMbHA5aLbVmqTjgY9HxLuHui0A\nkydPrivf2LHjmD27y4GzmZnZIPH0jBFI0oWSVkhaLmmZpPskTZE07N5PSZtIOl/SHEmLc1tPkLRm\nIc88SUeXzjs59/H9pfQbJf1sgJsZA1xed79XSNqmb2eeBHTU2NpYunQxCxYsGNA2m5mZWXUeaR65\nrgYOAcYCHwbOBpYBpxYz5UA6ImLAA8M6bQUIOAx4AHgncD4wDugOlG8EdqFn23cBHso/bwaQ9Gpg\nO+CCRjd6gPTjmm8G+Gu0zczMhpthNzJpdVsWEU9GxMMR8T/AdcA+kg6W9IykvSX9A1gKbKzkOEkP\nS1oqaaakPboLK4yMflLSzXlU+HZJW0jaVtIdkhZJukrSGwvn9VpuREyPiM9FxPUR8WBE/AH4AbBf\noS8zgB26R8oljQfeDZwCfKCQb3tgLVKQ3V3/PpI6JC2RdH9uyxqF4+vmke75khZKuq630V9Jm0t6\nQNLUvD9B0uWSnpb0vKRZkvas8z1SnfnMzMxsmHPQPHosJQWU8K9R3M8B7wDmA0cAXwOOBLYGpgOX\nS9q8VM4JwImkoPVl4BfAycBXgB2Bt+bj3eott+h1wNOF/RnAa4Ft8/5OwGzgUmA7Sd392gV4MCIe\nApC0E/Az4AzSiPYXgIOBbxXK/i3wRmAP0hBuJ3CdpNeVG5WD6T8BbRHx1Zx8Num67kgaJT8GeL6X\nvpmZmdko5KB5FJC0GykovD4nvQr4YkT8JSLui4ilwFHAyRHxm5x2LHAnKegtOi0irouI2cCZpEDz\nxFzW/wI/pefob73ldrf1rcB/Aud2p0XE/cCjpKCY/POmiHiCNEXjfYX0GYXijgO+HxFtEfHPiLg+\npx2e69oReA+wf0TMjIgHIuJoYCHwiVK73pfLPjUiji8c2hi4NSLuziPlV0XELZX6ZmZmZqOX5zSP\nXHtLWgSsSZoGMA34DrA/8GJE/L07o6TXAhsCfy6VcStQnqowq/D6ifzz76W09fpRLpL+jTQX+1cR\nUZ6XfCMpKD6FnvObbwJ2kfRX0nzm/ymcMxHYXtK3C2ljgLUkjc1teC3wtNRjpsRYoDgSvglwLfDN\niJhaatdU4Jw85eQ64JKImEXD/BD4VSmtNW9mZmart/b2dtrb23ukLVy4cFDqdtA8ct1AGlF9CZgX\nESsAcnC4ZBXKfanwOqqk9fkTCkkbktp8S0R8oUKWGcCPJL2BNDXkppx+E/B50rSJNXMZ3caTRpYv\nrVDesnx8HrAzK88vfrbwen7O1yrpwohY1H0gIn4q6RrgI8CHgGMlHRURP67d6/44CjiwMUWbmZmN\ncK2trbS29hxI6uzspKWlpeF1e3rGyPVCRMyNiEe6A+ZqchA4D9ihdGgH4O5i1r40oN5y8wjzDOAO\n4NAqxc0gBblHAvdGRPd6ajeTRpg/DNwXEY8VzukEtoyIORW2yMc3AJZXOF6cU70E+Cgp0J4uae1S\nPx+NiP+JiE8Ap5NWAqnHUK1YYmZmZgPMI82rj9OAEyTNIc05PpQ0veGAQp5Kqz3UWgGi13LzCPON\nwFzSw4nrdU+VyHOWu1/PlfQQ6YHDtkL6I5LmkUabf1Gq+0TgCkkPkx74W5HrfmdETImI6yTdBlwm\n6RjgXuDfgL2ASyOis1DPEkkfIU0fuUbSnhHxgqQzctq9wBtI87mL/9HojYCtVJobAvw9IpbXWYaZ\nmZkNAw6aVx9TgXVIy72tRwr89o6IBwp5Ko2M1hotrVXu7sBb8vZwTlMud0yprBnAZ+j5sB+kKRoH\n03NqBhHxR0kfJU3ROJo0jeQe0jrQ3fYCvkta2/lNwOOk0esnKMlB8oeBa4A/SNort/G/gY2A50gB\n9JG9XZBikUB7hfSNSSP0ZmZmNkJo6L7zwsy6SZoEdKRvBNyrRu4uYDIdHR1MmuQvQjEzs9VbYU5z\nS/FT5IHmkWazYWVK3no3duw4mpqaGt8cMzMzAxw0m/WLpHOAyRUOBenLUb7Un3Lb2tpobm6uma+p\nqYkJEyb0pwozMzPrBwfNZv0zhfQQZCXP9bfQ5uZmT7kwMzMbhhw0m/VDXhJvQc2MZmZmNip4nWYz\nMzMzsxocNJuZmZmZ1eCg2czMzMysBgfNZmZmZmY1OGg2MzMzM6vBQbOZmZmZWQ0Oms3MzMzMavA6\nzSOEpBOAw4E3AftGxOVD26KRSdImwFzgXRFxl6SdgRuA10dExS8lkXQw8KOIeH2j29fV1bVSmr/9\nz8zMbOh5pLnBJF0oaUVhWyDpaklb96GMrYDjgMOADYCrG9XeXN+mkqZJelTSEkkPS/qdpLfl45vk\nvmzTj7JXSPpYP86bK+mrFdKPlzSzj8VF4fWtwJurBcxVzmmYyZMn09LS0mPbcstmHnroocGo3szM\nzKpw0Dw4rgbWJwW8HwReBq7ow/lvBSIiroiIJyPipf40QlLNTxZynmuBdYB9gbcB+wOzgNd1Z2OQ\ngsg69bUteuXEiJcjYv4At2cVnAR0FLY2li5dzIIF/vJBMzOzoeSgeXAsy8Hu/Ii4CzgZ2FjSGwEk\nbSTpV5KekfSUpMvyNAIkHQ9cnl+vkLQ8v5ak4/Io8FJJMyXt0V1hYTR4f0k3SloMHJCP7SjpZkmL\nJf1T0pmSxuVT3wFsDnwpIm6PiIcj4raIOC4ibs955uSfd+Y6bsjlvkfSHyU9KenZXO+7C22aSwpw\nL8vnzSkc20dSRx7Zvj/3bUxfL3St61Ih/865LesU0g7J1+V5SZcAbyyd85b8Hj0uaZGk2yXtWjg+\nRdKsCnXdKek7vfdgM2BSYWuus+dmZmbWSA6aB5mk8cBBwH0R8VQe2Z0OPkGmCgAAEaNJREFULAR2\nALYHFgHX5GOnAZ/Np68PvDm/PgL4GnAksHUu43JJm5eq/D7wI1L0NV3SW0gj378B3gl8Ktd7Vs7/\nJLAc+KSkar8f/4c0WvtB0uj5fjn9tcBFuQ/bAfcCV0laOx/fNp93cD5v23xNdgJ+BpwBbAV8Ief5\nZpX6i1Tar/e6FL0yUi1pO+B8YCrwLmAG8O1S/vHAlcAHcp6rcx0b5eMXAFtJaimU+27S9b6gjj6Z\nmZnZcBMR3hq4ARcCL5EC4UXACuAR0oNoAJOBu0vnrAW8AOyW9/cBlpfyPAIcU0r7K3BWfr1Jrus/\nS3l+ApxTStuRNGVkrbz/xdzWhcD1pKBxs0L+7rK3qdH3NXIZexXSVgAfK+W7tkJfDgQeLezPBZYU\nrmP3tgzo7Md12Sbv70z6T8I6eX8acEXp/Hbg6Rp9nUUane/evxL478L+VOD6Xs6fBAS0BURh6wgg\nOjo6wszMzFbW0ZH+VgKTooExnUeaB8cNwDbARNLo6nTSSPLGOX2L/DH/IkmLgKeAV5OmSaxE0muB\nDYE/lw7dysqf53eU9icCh5TquyYf2wwgIs4hjQQfkOv4BPCP4hSEKu1aT9JPJN0r6VlSwLw2UGvp\nh4nAcaU2/QRYX9LYQr7Tct7idm6h/r5cl2qaSUF20W3FHUlrS/qBpLvzlJpFpBHyYj9/ArRKWkvS\nmkAr8NM622BmZmbDjJecGxwvRMTc7h1Jh5ECysNIH/X/jRSglqcaPDkQdZf2xwPnAWdWqO+VJRoi\n4gXSaOmVwBRJ00kjztf3UtfPgdcDX8llLQP+Qho578140uogl5YPRMTSwu6CiJhTPC7p6RplN8IP\ngV2Bo4AHSCPgl9Czn1eQ+r8v6ZOGV+U8dRT9q8L+woFor5mZ2ajQ3t5Oe3t7j7SFCwfnb6WD5qET\nwGuATtLqFE9GxPN1nRixSNI80lzkPxUO7UDPUdJKq0p0Am8vBvF1ugd4X379Yv5ZflBve+CLETEd\nII+kN5XyvFThvE5gy3JA3Fd9uC696SLNxy56X2l/e+CiyGtl53nqm5baslzSz4FDSdfrlxGxrHb1\nR5FmpnTrBFqq5DUzM1u9tLa20tra2iOts7OTlpbG/6100Dw4Xi1p/fy6eyR2HGlVjL8B/w/4fV4p\n4xFSALYvcEpEzKtS5mnACXkFijtJwdlE8goZWXkkGeAU4DZJZ5EeeHuBtGLGbhHxFUkTge8AFwN3\nkwK+XXL5389lzCeNru4p6VFgaaR1ju8DDpLUAawLnAosLtX/ILCrpD+TVhV5FjgRuELSw8BvSXOO\nJwLvjIgpVfpfTT3Xpax4naYCt0g6Cvg9sCdQXn3jPmA/SX/I+ydS+VqfTwrCgxS4m5mZ2QjlOc2D\nY09gXt7+Qho6/ERE/CkilgA7kaYzXEIKVH9CmtPc2xduTAVOB34A3AV8CNg7Ih4o5FlppDkiZpEe\nftsCuJk0lHkC8GjO8gjpobvjcls7SEH+lIj4Xi5jeU77Qj7vsnzu50j/KeggrYZxJinALjoK2D33\ntzOX90fgozn9dtIc4iNIAXbVvlTRn+vyyn5E/JU0bearpKB7N9LiyUVHAs+Q5kr/njQnvLPckIi4\nnzS/+p6IuKPO9puZmdkwpIjh9B0VZqOLpPtIq2icWSPfJKAD2qg0PaOjo4NJkyY1sKVmZmYjU2F6\nRktErDSINVA8PcOsASQ1kVbMWJ+0dnWd5tJz0LprIJtlZmZm/eTpGWaNMZ+02shhEdGHx3qnkGbv\ndG+TGTt2HE1N5ecpR67yU8+j2erSV/dzdHE/R5fVpZ+DwUGzWQNExBoRsX5E/Kp27n9pa2ujo6Oj\nxzZ7dhcTJtRa6nrkWJ1u4KtLX93P0cX9HF1Wl34OBk/PMBtGmpubPXfZzMxsGPJIs5mZmZlZDQ6a\nzczMzMxq8PQMs+FhLEBX1+hfLWPhwoV0djZsRaBhZXXpq/s5urifo8vq0M/C386xjazH6zSbDQOS\nDgCmDXU7zMzMRrADI+IXjSrcQbPZMCDpjaSv634QWDq0rTEzMxtRxgKbAtMj4qlGVeKg2czMzMys\nBj8IaGZmZmZWg4NmMzMzM7MaHDSbmZmZmdXgoNnMzMzMrAYHzWYNIunLkuZKWiLpL5K2rZF/F0kd\nkpZKulfSwRXyfFJSVy7zfyV9uHE9qM9A91PSf0i6WdLTebu2VpmDoRHvZyHvpyWtkHTpwLe8bxr0\ne7uupB9Lmpfz3SNpz8b1orYG9fOI3LfFkh6SdLqkVzeuF7X1pZ+SNpA0TdJsScslnV4l34i+D9XT\nz+F6H4LGvKeF/CPyXtSH391VuxdFhDdv3gZ4Az5FWjruM8BWwHnA00BTlfybAs8DpwJbAl8GXgJ2\nL+TZPqcdmfOcCCwD3j7K+nkxcDiwDfA24ALgGeDNo6mfpbwPAzcCl47C39s1gTuAK4D3AhOAnYCt\nR1k/DwCW5LInALsBjwA/GEH93AQ4A5gMdACnV8gzGu5D9fRz2N2HGtXX0u/5SL0X1fOervK9aMgu\niDdvo3kD/gKcWdhX/gN6dJX8pwB3ldLagasK+78ELi/luQ04ezT1s8I5awALgcmjrZ+5b7cAnwUu\nHAZ/qBrxe3s4cB8wZij7Ngj9PAu4tpTnB8DNI6WfpXNnVAk8Rvx9qJ5+Vsg35PehRvZ1pN+L6unn\nQNyLPD3DbIBJWhNoAa7vTov0L/Y64H1VTntvPl40vZT/fXXkGTQN7GfZ2qQRgqf73dhV0OB+Hg88\nEREXDkxr+6+B/dybHFRJelzSLEnfkDQkf38a2M8/Ay3dHyFLeguwF3DlwLS8b/rZz3qMhvtQfwzp\nfQga3teRfi+qxyrfi161CpWbWWVNwBjgiVL6E6SPMyvZoEr+dSS9OiKW9ZJng1Vrbr81qp9lpwCP\nsvIf6sHSkH5K2pE0qjNxIBu7Chr1fr4F+CDQBnwYeCtwDunvz0kD0/Q+aUg/I6JdUhNwiyTlOs6N\niFMGsO190Z9+1mM03If6Y6jvQ9Cgvo6Se1E9Vvle5KDZzIYtSccC+wM7R8SLQ92egSJpPPBz4LCI\neGao29Nga5D+2H0+jxbNlLQR8HWGJmhuCEm7AN8kfQR8O+kP8lRJj0XEfw1l22zVjNb7EPhe1Nd7\nkYNms4G3AFgOrF9KXx94vMo5j1fJ/1xh9LVanmplNlqj+gmApK8DRwO7RsQ/Vr25/Tbg/ZS0FenB\nlSvyqCTk1YwkvQhsGRFzB6LxfdCo9/Mx4MX8R6pbF7CBpFdFxMur1uw+a1Q/TwQuLny8/Y8ckJwH\nDEXQ3J9+1mM03IfqNozuQ9CYvm7O6LgX1WOV70We02w2wCLiJdLTu7t2p+Wb0a6keY+V3FbMn30o\np/eWZ/dSnkHTwH4i6WjgW8AeETFzoNrcHw3q5z3A1sC7SB+JTgQuB27Irx8eoObXrYHv562kUdei\nLYHHhiBgbmQ/xwHl/qwolD+o+tnPeoyG+1BdhtN9CBrW1y5Gx72oHqt+LxrKpyO9eRutG+mjvMX0\nXC7nKeBN+fj3gZ8V8m8KLCLNm9sS+BLwIrBbIc/7SEs7dS/1dAJpSZ6hXOqpEf08JvdrX9LIQve2\n9mjqZ4U6hsMT6414PzcCngWmAlsAHyGNFh07yvp5fO7np3L+3UlP6v9ipPQzp00kBVB3kJZdmwg0\nF46P+PtQnf0cdvehRvW1Qh0j7l5U53u6yveiIbsg3ryN9i3/YX2QtHbrbcB7CscuBG4o5X8/6X/X\nS/If24MqlPnvpFHKJcBdpBGQUdVPYC7po7nydtxo6meF8of8D1UDf2+3I40QLc55jgE0mvpJ+uR2\nCnAv8EIueyqwzgjr54oK//bmlPKMhvtQr/0crvehRr2npfwj9V5Uz+/uKt2LlAsxMzMzM7MqPKfZ\nzMzMzKwGB81mZmZmZjU4aDYzMzMzq8FBs5mZmZlZDQ6azczMzMxqcNBsZmZmZlaDg2YzMzMzsxoc\nNJuZmZmZ1eCg2czMBpSk4yXNHKByOgeiTUNZh5mNDg6azcysEfr0dbOSVkj6WCn5NGDXgWtSRYNR\nxyqpcm3MbJC9aqgbYGZmI4OkNSPipcGqLyIWA4tHeh39NdjX28x655FmMzOrSNIMSWdJOkPSk8A1\nOX1dSedLmi9poaTrJG3TSznvkfRHSU9KelbSjZLeXTg+lzQyfVkeVZ2T00/onuYhaXdJSyStUyr7\nTEnXFfZ3lHSzpMWS/pmPj+ulbT2mkki6UNLvJH1D0uOSnpH0bUljJJ0q6SlJD0s6pHDOJrndn5J0\na27nLEnvL9W1s6S/SloqaZ6k70tao3B8peudrw0Vrs1bJF2W27hI0u2Sdi3VNzf346eSnsvX47BS\nnn+T1J779XwuZ9vC8X0kdeQ+3S/puGKbzVYn/sU3M7PefAZYBmwPHJ7Tfgu8EdgDmAR0AtdJel2V\nMl4LXJTL2A64F7hK0tr5+LaAgIOBDfI+pEC6e5rH9cAzwL93F5qDt/2Btry/OXA18BvgncCngB2A\ns2r0sTyV5IPAm4GdgK8BJwJ/AJ4G/g9wLnCepA1L551Kmu7xLuA24ApJr89t2xC4EvgrsA3pWn4O\n+HapjPL1fk9OL1+b8bm8D+T6rgYul7RRqbwjgTtynrOBcyRtkdu0NnBz7utHga2B75NjA0k7AT8D\nzgC2Ar6Q2/Gt8gU0Wy1EhDdv3rx587bSBswA/lZK24EUvK5ZSr8P+I/8+nigs5dy1wAWAnsV0lYA\nHyvl61EOKXi7trD/IdLUinXy/k+Ac0pl7Ai8DKxVpS3lOi4E5pTydAE3ltq/CNg/72+S2//1Qp4x\nwEPdacB3gbtL5X4RWNjb9a52bar0ZRbwpcL+XOCiUp7Hgc/n158HngXWrVLetcAxpbQDgUeH+nfT\nm7eh2Dyn2czMetNR2p9IGjl+WlIxfSyweaUCJK1HChp3BtYjBZSvASb0sS3TgNskbRARjwMHAFdG\nxHOFtm0taXKx+vxzM2B2nfX8o7T/BCkgBSAiVkh6itSXor8U8iyX9DegOSdtRRp9LroVGC9po4h4\nJKeVr3dFeZT4O8BepJHiV5Heg/I1nVXaf7zQ7onAzIhYWKWaicD2koqj4WOAtSSNjYil9bTVbLRw\n0GxmZr15obQ/HphHCoBVOvZslTJ+Drwe+App9HUZKcBcqy8NiYi/5Tm9n5Z0LrAvaTpDsW3nAWdW\naNtDfaiq/PBdVElrxBTH8vWu5oekVT+OAh4AlgCXsPI17a3dS2rUMR44Dri0fMABs62OHDSbmVlf\ndJLm1i6PiHoD0e2BL0bEdABJGwNNpTwvkUYxa5kGTAYeBZYDV5Xa9vaImFvpxEHwXuAWAEljgBZg\naj7WBexXyr8jsKgwylxNpWuzPWnqxeW5vvHApn1s713A5yS9LiIq/YenE9gyIub0sVyzUckPApqZ\nWd0i4jrSNIPL8ooWm0jaXtJ/SZpU5bT7gIMkbSVpO9KDe+Vl3h4EdpW0fi8PFEIKmieRHkb7bfRc\nku0U0nSCsyRNlPTWvPpDrQcBB8qXJX1c0pakh+5eR5ojTd7fOLdtS0n7ACeQRoxreZCVr819wH65\nnxNJ16U8ul5LO2nqyWX5PdxM0n75PYL0AORn8ooZb8/v36ckndTHesxGBQfNZmZWTbUvKNmLtOrC\nBaR5wr8gzaV9okr+Q0nTMzpIqzGcCcwv5TkK2B14mDTCWblBEQ8At5NWephWOjaLNG1ki9y+TlJg\n+mi18upU6TpUSjs2b3eSRoL3joinc9vmka7btvn42aQHF79bo0z417V5iH9dmyNJD2TeCvyetBxg\n+br12u78H47dSe/FlaSR52NII/hExB9Jq2rsTrrmtwFHkIJ4s9WOIvr0pU1mZmZWIGkTYA7w7oi4\na6jbY2aN4ZFmMzOzVdfXqRFmNsI4aDYzM1t1/tjWbJTz9AwzMzMzsxo80mxmZmZmVoODZjMzMzOz\nGhw0m5mZmZnV4KDZzMzMzKwGB81mZmZmZjU4aDYzMzMzq8FBs5mZmZlZDQ6azczMzMxqcNBsZmZm\nZlbD/wfAeuBK990W0gAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "importance = model_pkl.get_fscore()\n", "importance = sorted(importance.items(), key=operator.itemgetter(1))\n", "\n", "df = pd.DataFrame(importance, columns=['feature', 'fscore'])\n", "df['fscore'] = df['fscore'] / df['fscore'].sum()\n", "\n", "df.plot(kind='barh', x='feature', y='fscore', legend=False, figsize=(6, 10))\n", "plt.title('XGBoost Feature Importance')\n", "plt.xlabel('relative importance');" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true, "hidden": true }, "source": [ "### Neural net" ] }, { "cell_type": "code", "execution_count": 410, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "#np.savez_compressed('vars.npz', pkl_cats, pkl_contins)\n", "#np.savez_compressed('deps.npz', y_pkl)" ] }, { "cell_type": "code", "execution_count": 411, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "pkl_cats = np.stack([x_pkl[:,pkl_vars.index(f)] for f in cat_vars], 1)\n", "pkl_contins = np.stack([x_pkl[:,pkl_vars.index(f)] for f in contin_vars], 1)" ] }, { "cell_type": "code", "execution_count": 412, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "co_enc = StandardScaler().fit(pkl_contins)\n", "pkl_contins = co_enc.transform(pkl_contins)" ] }, { "cell_type": "code", "execution_count": 413, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "pkl_contins_trn, pkl_contins_val = pkl_contins[:train_size], pkl_contins[train_size:]\n", "pkl_cats_trn, pkl_cats_val = pkl_cats[:train_size], pkl_cats[train_size:]\n", "y_pkl_trn, y_pkl_val = y_pkl[:train_size], y_pkl[train_size:]" ] }, { "cell_type": "code", "execution_count": 414, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "def get_emb_pkl(feat):\n", " name, c = cat_map_info(feat)\n", " c2 = (c+2)//3\n", " if c2>50: c2=50\n", " inp = Input((1,), dtype='int64', name=name+'_in')\n", " u = Flatten(name=name+'_flt')(Embedding(c, c2, input_length=1, init=emb_init)(inp))\n", " return inp,u" ] }, { "cell_type": "code", "execution_count": 415, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "n_pkl_contin = pkl_contins_trn.shape[1]\n", "contin_inp = Input((n_pkl_contin,), name='contin')\n", "contin_out = BatchNormalization()(contin_inp)" ] }, { "cell_type": "code", "execution_count": 416, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "map_train_pkl = split_cols(pkl_cats_trn) + [pkl_contins_trn]\n", "map_valid_pkl = split_cols(pkl_cats_val) + [pkl_contins_val]" ] }, { "cell_type": "code", "execution_count": 417, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "def train_pkl(bs=128, ne=10):\n", " return model_pkl.fit(map_train_pkl, y_pkl_trn, batch_size=bs, nb_epoch=ne,\n", " verbose=0, validation_data=(map_valid_pkl, y_pkl_val))" ] }, { "cell_type": "code", "execution_count": 418, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "def get_model_pkl(): \n", " conts = [get_contin_pkl(feat) for feat in contin_map_fit.features]\n", " embs = [get_emb_pkl(feat) for feat in cat_map_fit.features]\n", " x = merge([emb for inp,emb in embs] + [contin_out], mode='concat')\n", "\n", " x = Dropout(0.02)(x)\n", " x = Dense(1000, activation='relu', init='uniform')(x)\n", " x = Dense(500, activation='relu', init='uniform')(x)\n", " x = Dense(1, activation='sigmoid')(x)\n", "\n", " model_pkl = Model([inp for inp,emb in embs] + [contin_inp], x)\n", " model_pkl.compile('adam', 'mean_absolute_error')\n", " #model.compile(Adam(), 'mse')\n", " return model_pkl" ] }, { "cell_type": "code", "execution_count": 458, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "model_pkl = get_model_pkl()" ] }, { "cell_type": "code", "execution_count": 459, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "[0.0086286964193351811,\n", " 0.0080550193002727963,\n", " 0.0080270905797575273,\n", " 0.00833844816263264,\n", " 0.0078600179936998815,\n", " 0.007661957152165411,\n", " 0.0076301282558615853,\n", " 0.0076333107689942779,\n", " 0.0074799654912108687,\n", " 0.0074509223625492749]" ] }, "execution_count": 459, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_pkl(128, 10).history['val_loss']" ] }, { "cell_type": "code", "execution_count": 460, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "[0.0074035965020400139,\n", " 0.0073704867357632167,\n", " 0.007380679178817344,\n", " 0.0073204016040114171,\n", " 0.0073323446760613217]" ] }, "execution_count": 460, "metadata": {}, "output_type": "execute_result" } ], "source": [ "K.set_value(model_pkl.optimizer.lr, 1e-4)\n", "train_pkl(128, 5).history['val_loss']" ] }, { "cell_type": "code", "execution_count": 739, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "'\\n1 97s - loss: 0.0104 - val_loss: 0.0083\\n2 93s - loss: 0.0076 - val_loss: 0.0076\\n3 90s - loss: 0.0071 - val_loss: 0.0076\\n4 90s - loss: 0.0068 - val_loss: 0.0075\\n5 93s - loss: 0.0066 - val_loss: 0.0075\\n6 95s - loss: 0.0064 - val_loss: 0.0076\\n7 98s - loss: 0.0063 - val_loss: 0.0077\\n8 97s - loss: 0.0062 - val_loss: 0.0075\\n9 95s - loss: 0.0061 - val_loss: 0.0073\\n0 101s - loss: 0.0061 - val_loss: 0.0074\\n'" ] }, "execution_count": 739, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"\"\"\n", "1 97s - loss: 0.0104 - val_loss: 0.0083\n", "2 93s - loss: 0.0076 - val_loss: 0.0076\n", "3 90s - loss: 0.0071 - val_loss: 0.0076\n", "4 90s - loss: 0.0068 - val_loss: 0.0075\n", "5 93s - loss: 0.0066 - val_loss: 0.0075\n", "6 95s - loss: 0.0064 - val_loss: 0.0076\n", "7 98s - loss: 0.0063 - val_loss: 0.0077\n", "8 97s - loss: 0.0062 - val_loss: 0.0075\n", "9 95s - loss: 0.0061 - val_loss: 0.0073\n", "0 101s - loss: 0.0061 - val_loss: 0.0074\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 116, "metadata": { "hidden": true, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjQAAAGHCAYAAACnPchFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xt8jvX/wPHX+97mMEyKUKEipwibcmakRqrd6VsaJVIq\nqX7UV6WDGRKVQ/qKIkM1lHLKIYccMlRblLNyPoacN6ft8/vjurbuzWYH97373ryfj8f9yP253tfn\n87589+Xt+nyuzyXGGJRSSiml8jOHtxNQSimllLpSWtAopZRSKt/TgkYppZRS+Z4WNEoppZTK97Sg\nUUoppVS+pwWNUkoppfI9LWiUUkople9pQaOUUkqpfE8LGqWUUkrle1rQKKXyPRGpJCLJItI5F+e2\nsM9tnkVcFzuuYu4zVUp5ihY0SikF2XkHjMlmnFLKC7SgUUoppVS+pwWNUkoppfI9LWiUUldMRCLt\n9SW3icgXInJcRP4WkSj7eAURmSEiJ0TkgIj0zqCPMiIyXkQOikiiiKzNaE2MiJQUkWh7jGMiMgG4\nJpO8qonINyJy1O7zFxF5wM3X3kNE1ovIWRHZJyIfi0jJdDFVRGS6fe2JIrJHRGJEpIRLzD0issK+\nplMisllEBrkzV6UKMn9vJ6CUKhBS1pZMBTYCrwHtgDdF5B/gWWAx0AfoBLwvIj8bY34CEJEiwDLg\nVmAUsBN4BIgWkZLGmFEuY80CGgOfAJuBh4CJpFvfIiK3Az8Be4HBwBngUWCGiLQ3xsy80osWkUjg\nHeAHYDRQDegB1BeRJsaYJBEJsI8HAB8BB4EbgfuxCrFTIlITmA2sBd4GzgFV7OtUSmWHMUY/+tGP\nfq7oA/QDkoHRLm0OYDdwEXjVpb0kVnHxuUvby0AS8JhLmx+wEjgBFLPbwu1xervECVYxlAR0dmlf\nBPwG+KfL9Sdgs8v3Fva5zbO4xiftuIr299LAWWBuurgedtyT9vc6ds4PXabvlOsv5e3/LfWjn/z6\n0SknpZS7GGB86hdjkoFfsQqOz13aTwBbsO7GpGgLHDTGTHGJS8K6o1Ecq+gAuA+4AIxxiTNYd3Uk\npU1ESgEtga+BkiJyXcoH627JbSJS/gqvtzXWXZcR6do/A05h3aECqyADaCMiRTPp67j934dERDKJ\nUUpdhhY0Sil32p3u+wngrDHmnwzaS7l8rwRsy6C/TViFSiX7e0XggDEmIV3clnTfq9jnDQAOp/tE\n2jHXX+5CsiElp62ujcaYC8D2lOPGmJ3Ah8DTwBERmW+vuwlyOW0q1t2oz4BD9vqaR7S4USr7dA2N\nUsqdkrLZBi53VDwg5R9rHwALMon504Pjp2GM+a+IRGNNmd2LdefpdRFpaIzZb4w5CzQXkZZYd3ba\nAB2AxSJyr30XSil1GXqHRinlC3YBt2XQXsP+706XuPIiEpgurnq679vt/14wxizJ5HPGDTmDtRA4\nlb0I+BaX4wAYYzYYY941xoQCTYGbgOfSxfxojHnVGFMLeBNohTV1ppTKghY0SilfMBcoJyIdUhpE\nxA94EWs9ynKXuADgeZc4hx2XehfDGHMYWAo8KyLl0g8mIqXdkPMirPU8L6VrfxoIAubYY5Wwr8XV\nBqyFwoXtmFJcah3WXazCbshVqQJPp5yUUr7gU6xHu6NFpD7/PrbdCHjZ5W7KbKy1Ju+JyC1Yj4i3\nB0pc0iO8AKwA/hCRz7Du2pS1+7wRqOcSm+PpL2PMEREZDLwjIvOxHievjlVs/Qx8aYe2Aj4Wka+x\n1tv4A52xnv76xo55x36X1PdYd3bK2v3sxnoqSymVBS1olFKeltn6D9c7KmdFpAXwHtZf9kFYC327\nGGMmu8QZe2O8EVj72RhgJtAb6xFtXGI32cVRP6xHrq8D/rbj+mczx8tfmDH9ReRvoCcwDPgH6wms\nN+2ntMC60zIfa9+ZG4EEu62NMeYXO2Ym1iLirliPgx/BusMUaYw5lZvclLraiK41U0oppVR+5zNr\naETkBRHZYW8LvlpE7swiPlRE4uztxreKyJPpjte0tzzfYW/Jnn6eO1vjisgE+3zXz9wrv2KllFJK\nuYtPFDT2QsAPsW4N18O6Hbsgs4V7InIz1oK7xVi7cI4ExonIPS5hgcBfWFuwH7jCcedhzWmXsz8R\nOb1GpZRSSnmOT0w5ichqYI0x5mX7uwB7gI+MMUMziB8CtDXG3OHSFgOUNMbcl0H8DmC4MeajnI5r\nv/iupDGmvXuuVimllFLu5vU7NPaeDSFYd1uA1K3MF2E9jZCRhvZxVwsuE3+l44aKyCH77bejReTa\n7I6jlFJKKc/zekGDtaLfDziUrv0Q1vRORsplEh8kItndsyG7487DeuqiFdabglsAc3VLcqWUUsp3\n6GPbWTDGTHP5ukFE/sBamxMK/Jg+3n75XRjWPhpn8yBFpZRSqqAoAtwMLDDGHM3Jib5Q0BzBetdL\n2XTtZYGDmZxzMJP4k8aYcx4cF2PMDhE5gvXyu0sKGqxi5ssM2pVSSimVPZ2Ar3JygtcLGmPMBRGJ\nA+7G2mkzZXHu3VgvcMvIKqBturZ77XZPjouI3IS1QVeGT05hv3Pmiy++oEaNGpmE5C+9evVi+PDh\n3k7DbQrS9RSkawG9Hl9WkK4F9Hp81aZNm3j88cfh3/e3ZZvXCxrbMKwtz+OwtgzvhfXYdTSAvb34\nDcaYlL1mxgAv2E87fY5VhPwHSH3CyV70WxNrS/NCwI0iUgc4bYz5K5vjFsN6pHs61l2bKsAQrO3L\nM3uD71mAGjVqEBwcnOvfEF9SsmTJAnMtULCupyBdC+j1+LKCdC2g15MP5HjJhk8UNMaYafbeL1FY\nUz5rgTD7BXNgLdKt4BK/U0TaAcOxXgy3F+hmjHF98ukGrC3OU55Lf9X+LMNa4JudcZOAO7AWBV8D\n7McqZN4xxlxw3++AUkoppa6ETxQ0AMaY0cDoTI51zaBtOdZj15n1t4tsPMWVxbhngTZZ9aGUUkop\n7/KFx7aVUkoppa6IFjQqSxERBetNDwXpegrStYBejy8rSNcCej0FkU+8+qAgEZFgIC4uLq6gLdBS\nSqk0du/ezZEjR7ydhspnSpcuTcWKFTM8Fh8fT0hICECIMSY+J/36zBoapZRS+cfu3bupUaMGCQkJ\n3k5F5TOBgYFs2rQp06Imt7SgUUoplWNHjhwhISGhQO25pTwvZZ+ZI0eOaEGjlFLKdxSkPbdU/qaL\ngpVSSimV72lBo5RSSql8TwsapZRSSuV7WtAopZRSKt/TgkYppZTKQzfffDNPPfWUt9MocLSgUUop\npdJZtWoV/fv35+TJk27v2+FwICJu7/dqp49tK6WUUunExsYSFRVF165dCQoKcmvfW7ZsweHQ+wnu\npr+jSimlVDrZfS2QMYZz587lqO+AgAD8/Pxyk5a6DC1olFJKKRf9+/enT58+gLXexeFw4Ofnx65d\nu3A4HLz00kt89dVX1KpViyJFirBgwQIAPvjgA5o0aULp0qUJDAykfv36TJ8+/ZL+06+hmThxIg6H\ng9jYWHr37s31119P8eLFad++PUePHs2biy4AdMpJKaWUcvHwww+zdetWpkyZwsiRI7nuuusQEcqU\nKQPA4sWLmTZtGj179qR06dLcfPPNAHz00UeEh4fz+OOPc/78eaZMmcKjjz7KnDlzaNu2bWr/ma2f\nefHFF7n22muJjIxk586dDB8+nJ49exITE+Pxay4ItKBRSimlXNSqVYvg4GCmTJlCeHj4Je8c2rp1\nK+vXr6datWpp2rdt20bhwoVTv/fs2ZN69eoxbNiwNAVNZsqUKcP8+fNTvyclJTFq1ChOnTpFiRIl\nrvCqCj4taJRSSnlcQgJs3uzZMapXh8BAz44BEBoaekkxA6QpZo4fP87Fixdp1qwZU6ZMybJPEaF7\n9+5p2po1a8aIESPYtWsXtWrVuvLECzgtaJRSSnnc5s0QEuLZMeLiIC/ek5kyxZTenDlzGDRoEGvX\nrk2zUDi7TzRVqFAhzfdSpUoBcOzYsdwlepXRgkYppZTHVa9uFRyeHiMvFC1a9JK2FStWEB4eTmho\nKJ988gnly5cnICCAzz//PNtrYDJ78im7T1xd7bSgUUop5XGBgXlz98Rdcrrx3bfffkvRokVZsGAB\n/v7//tU6fvx4d6emMqGPbXvIjh3ezkAppVRuFStWDLDWwmSHn58fIsLFixdT23bu3MnMmTM9kp+6\nlBY0HrJ0qbczUEoplVshISEYY+jbty9ffPEFU6dOJSEhIdP4du3acebMGcLCwhg7dixRUVE0bNiQ\n2267LVvjZTatpNNN2eczBY2IvCAiO0QkUURWi8idWcSHikiciJwVka0i8mS64zVF5Bu7z2QReSm3\n44pIlIjsF5EEEVkoIlWyup5ly7KKUEop5avq16/PwIED+f333+natSudOnXi8OHDiEiG01EtW7bk\n888/59ChQ/Tq1YupU6cydOhQnE7nJbEZ9ZHZFJe+8yn7xBeqPxHpAEwEugM/A72AR4CqxpgjGcTf\nDKwHRgPjgdbACOA+Y8xCO6a+3UccMBwYYoz5KKfjishrwGtAZ2AnMBCoDdQwxpzPILdga8w49u8P\npnz53P6uKKWU74qPjyckJIS4uDiC89PiGOVVWf3cpBwHQowx8Tnp21fu0PQCxhpjJhljNgPPAQlA\nZu9Xfx7YbozpY4zZYoz5H/CN3Q8AxphfjTGvGWOmAZcUHjkY92VggDFmjjFmPVZhcwNwadntwuGA\n2bOzuGqllFJKuYXXCxoRCQBCgMUpbca6bbQIaJTJaQ3t464WXCY+V+OKyC1AuXQxJ4E1WY1Vrx7M\nmJHdbJRSSil1Jbxe0AClAT/gULr2Q1jFREbKZRIfJCKFM4jP7bjlAJPD3AAIDYXFi+HUqWxmo5RS\nSqlc84WCpkAKDYXz58HltRxKKaWU8hBf2FjvCJAElE3XXhY4mMk5BzOJP2mMOZdBfG7HPQiI3XYo\nXcxvl+v8/fd7ERRUkl69YPJkqy0iIoKIiIhspqeUUkoVXDExMZfsonzixIlc9+f1gsYYc0FE4oC7\ngVkAYj2ndjfwUSanrQLSv7r0XrvdHeOOsmN2iMhBu+13OyYIaAD873L9Dx8+nFmzghkxAqZPh4CA\n7GamlFJKFXwZ/SPf5SmnHPOVKadhwDMi0llEqgNjgEAgGkBEBovIRJf4McCtIjJERKqJSA/gP3Y/\n2OcEiEgdEakLFAJutL9Xzsa4E1xiRgBvicgDIlIbmATsBbLc/jE8HE6c0D1plFJKKU/z+h0aAGPM\nNBEpDURhTeesBcKMMYftkHJABZf4nSLSDmt/mZewCoxuxhjXJ59uwJoWStlo51X7swxolc1xMcYM\nFZFAYCxwDbACaJvRHjTp1a0LFStaTzu1bp2j3xKllFJK5YBPFDQAxpjRWBvlZXSsawZty7Eeu86s\nv11k4w7U5cZ1iYkEIrPqKz0RcDrh229h1Cjru1JKKaXcz1emnAqs8HDYuxfic7TfoVJKKaVyQgsa\nD2vWDEqV0k32lFJKKU/SgsbDAgLg/vtB3yCvlFJXp+joaBwOB7t3705tCw0NpWXLllmeu2zZMhwO\nB8uXL3drTg6Hg6ioKLf26W1a0OSB8HD44w/Yvt3bmSillMprmb1d2+HI3l/BuX3j9rx58+jfv3+2\nc8rvfGZRcEEWFgaFC1t3aXr1yjpeKaVUwbZw4UKPjzF37lxGjx5Nv379LjmWmJiIv3/BKgH0Dk0e\nKF7cemxb19EopZQC8Pf393hBYb1vOWOFChXK9h2i/KJgXY0Pczrhp5/gyBFvZ6KUUupypk+fjsPh\nYMWKFZccGzt2LA6Hg40bN/LHH3/QpUsXKleuTNGiRSlfvjzdunXjn3/+yXKM0NBQWrVqlaZt3759\nOJ1OihcvTtmyZenduzfnzp27pDD56aefePTRR6lUqRJFihShYsWK9O7dm7Nnz6bGdO3aldGjrR1J\nHA4HDocDPz+/1OMZraH57bffaNu2LSVLlqREiRK0bt2aNWvWpImZOHEiDoeD2NhYevfuzfXXX0/x\n4sVp3749R48ezfK6Palg3W/yYQ88AN27w5w50KWLt7NRSimVmXbt2lG8eHGmTZtGs2bN0hybNm0a\ntWvXpmbNmgwbNoydO3fy1FNPUa5cOTZs2MDYsWPZuHEjq1Zd/k086devnD17llatWrF3715efvll\nypcvz+TJk1myZMklsV9//TWJiYn06NGD6667jp9//plRo0axb98+pk6dCsBzzz3H/v37WbRoEV9+\n+eVl79YAbNy4kebNm1OyZElef/11/P39GTt2LKGhoSxfvpw777wzTfyLL77ItddeS2RkJDt37mT4\n8OH07Nnzkncz5SljjH7c+AGCARMXF2fSa9zYmPDwS5qVUirfiYuLM5n9WVcQdOzY0ZQrV84kJyen\nth08eND4+fmZQYMGGWOMOXv27CXnTZkyxTgcDvPTTz+ltkVHRxuHw2F27dqV2hYaGmpatmyZ+n3E\niBHG4XCY6dOnp7YlJiaa2267zTgcDrNs2bLU9ozGfe+994yfn5/Zs2dPalvPnj2Nw+HI8PpExPTv\n3z/1u9PpNEWKFDE7d+5MbTtw4IAJCgoyoaGhaa5FRExYWFia/nr37m0CAgLMyZMnMxwvRVY/NynH\ngWCTw79/9Q5NHnI6oV8/SEiAwEBvZ6OUUnkn4UICm49s9ugY1UtXJzDAPX+4dujQgSlTprB06dLU\nx6u//vprjDE8+uijABQuXDg1/ty5c5w+fZoGDRpgjCE+Pp4mTZpke7x58+ZRvnx52rdvn9pWpEgR\nunfvzmuvvZYm1nXchIQEEhMTadSoEcnJyfz222/cdNNNObrW5ORkFi5cyEMPPUSlSpVS28uVK0fH\njh0ZN24cp0+fpnjx4oB1d6l79+5p+mjWrBkjRoxg165d1KpVK0fju4sWNHkoPBz69IGFC61fK6XU\n1WLzkc2EfJq7tyhnV1z3OILLB7ulrzZt2hAUFMTUqVNTC5pp06ZRt25dqlSpAsCxY8eIjIxk6tSp\n/P3336nniggnTpzI0Xi7du1K7ddVtWrVLmnbs2cPb7/9NrNnz+bYsWNXNC7A4cOHSUhIoGrVqpcc\nq1GjBsnJyezZs4caNWqktleoUCFNXKlSpQDS5JPXtKDJQ1WrQo0a1tNOWtAopa4m1UtXJ657nMfH\ncJdChQrhdDr57rvvGD16NAcOHGDlypW89957qTGPPPIIq1evpk+fPtSpU4fixYuTnJxMWFgYycnJ\nbsvFVXJyMq1bt+b48eO88cYbVKtWjWLFirFv3z6efPJJj42bnusCY1cmi7U6nqQFTR5zOuHTT+Hi\nRShgWwAopVSmAgMC3Xb3JK906NCBSZMmsXjxYjZs2ACQOt10/PhxlixZwoABA3jzzTdTz/nzzz9z\nNValSpVSx3C1eXPaabo//viDbdu2MXnyZDp16pTavmjRokvOze7GeWXKlCEwMJAtW7ZccmzTpk04\nHI5L7sj4In1sO4+Fh8PRoxAb6+1MlFJKXU7r1q0pVaoUU6ZMYdq0adx1112pa0xS7lCkvyMyfPjw\nXO3Ae99997F//36mT5+e2paQkMBnn32WJi6zcUeMGHHJuMWKFQPg5MmTlx3b4XBw7733MnPmzDSv\nZzh06BAxMTE0a9Ysdf2ML9N7BHnszjuhfHlr1+Dmzb2djVJKqcz4+/vTvn17pkyZQkJCAh9++GHq\nsRIlStC8eXOGDh3K+fPnufHGG/nhhx/YuXNnrqZdnnnmGT7++GOeeOIJfv3119THtlOKkhTVq1en\ncuXKvPLKK+zdu5egoCCmT5/O8ePHL+kzJCQEYwwvvvgiYWFh+Pn50aFDhwzHHzhwIIsWLaJJkyb0\n6NEDPz8/Pv30U86fP8/QoUPTxGZ2fd6cbgK9Q5PnHA548EFrHY2X/7dXSimVhQ4dOnDmzBlEhEce\neSTNsZiYGMLCwhg9ejR9+/alcOHCzJs3L9vvSXKNKVq0KEuWLCEsLIyPP/6YQYMGpRZMrvz9/Zkz\nZw716tXjvffeIyoqimrVqjFp0qRL+m/fvj0vvfQSCxYsoHPnznTs2DHN2K7j16xZkxUrVlC7dm3e\ne+89BgwYwC233MLSpUupX79+pnlnpz2viLcrqoJGRIKBuLi4OIKDM54vnj8f2raF33+H2rXzNj+l\nlHKH+Ph4QkJCuNyfdUqll9XPTcpxIMQYE5+TvvUOjRe0bAklSljTTkoppZS6clrQeEHhwtYdGn1Z\npVJKKeUeWtB4idMJcXGwZ4+3M1FKKaXyPy1oPGT38d2XPd62rbUPzaxZeZSQUkopVYBpQeMhkcsi\nSUpOyvT4NddYa2l02kkppZS6clrQeMi6g+sYuWbkZWOcTli6FDLYPkAppZRSOaAFjYdE1I7gzSVv\nsuXIpVtJp3jwQesVCHPn5mFiSimlVAHkMwWNiLwgIjtEJFFEVovInVnEh4pInIicFZGtIvJkBjGP\niMgmu891ItI23fHiIjJCRHaKSIKI/CQi9dPFTBCR5HSfLEuQF+56gZuCbqLrzK6ZTj3ddBPUr6/T\nTkoppdSV8olXH4hIB+BDoDvwM9ALWCAiVY0xRzKIvxmYA4wGOgKtgXEist8Ys9COaQx8BbwGfA90\nAmaISD1jzEa7q/FATfvYAeAJYJGI1DDGHHAZch7QBUjZBvFcVtdU1L8oE8In0HxCc0asHsErjV/J\nMC48HIYMgXPnrMe5lVIqP9m0aZO3U1D5iEd/XowxXv8Aq4GRLt8F2Av0ySR+CPB7urYYYK7L9ynA\nrHQxq4DR9q+LABeANulifgWiXL5PAL7NwbUEAyYuLs4YY8z/zfs/U2RgEbP58GaTkT/+MAaMmTs3\nw8NKKeWTdu3aZQIDAw2gH/3k6BMYGGh27dqV4c9VXFxcSlywyWEt4fU7NCISAIQA76a0GWOMiCwC\nGmVyWkMg/bvSFwDDXb43wrrrkz4m3P61P+DHpXdbEoGm6dpCReQQcAxYArxljPkns2tyNejuQXy/\n7Xu6zuzKiq4r8HP4pTl+++1QubK1a3Dbtpl0opRSPqZixYps2rSJI0cuuYmu1GWVLl2aihUrur1f\nrxc0QGmswuJQuvZDQLVMzimXSXyQiBQ2xpy7TEw5AGPMaRFZBbwtIpvtYx2xCqFtLufMA6YDO4DK\nwGBgrog0MibrF2EFBgQyIXwCzSY0Y/jq4bza+NU0x0WsaaevvoLRo62XVyqlVH5QsWJFj/zFpFRu\n+EJB402PA58D+4CLQDzWupuQlABjzDSX+A0i8gfwFxAK/JhZx7169aJkyZKp32/5+xbe2PAG91e9\nn+qlq6eJdTph2DD4+Wdo2PCKr0kppZTyeTExMcTExKRpO3HiRK7784WC5giQBJRN114WOJjJOQcz\niT9p3525XExqn8aYHUBLESkKBBljDonIFGB7ZskaY3aIyBGgCpcpaIYPH57mTaIJFxKoO6YuXWd2\n5aeuP6WZemrcGEqXtqadtKBRSil1NYiIiCAiIiJNm8vbtnPM6xMcxpgLQBxwd0qbiIj9PTaT01a5\nxtvutdsvF3NPupiUHBLtYqYUEAZk+iC1iNwEXIf1VFS2BQYEEu2MZs3eNQxbNSzNMT8/eOABfXxb\nKaWUyi2vFzS2YcAzItJZRKoDY4BAIBpARAaLyESX+DHArSIyRESqiUgP4D92PylGAm1EpLcdE4k1\nlfRxSoCI3CsiYSJys4jcg7Xgd6PLuMVEZKiINBCRSiJyN1axsxVrgXGONK7QmN6NevP2j2+z6XDa\nR9ecTti8GbZkvg+fUkoppTLhEwWNvU7lVSAK+A24Awgzxhy2Q8oBFVzidwLtsPafWYu1b003Y8wi\nl5hVWIt8u9sx7YFw8+8eNAAlgf8Bm7CKmOVYj3Gn7ISXZOcyE9gCfAb8AjS37yzl2ICWA7j5mpsv\n2XCvdWsoWtSadlJKKaVUzkg2HtRROSAiwUBcXFxcmjU0rlbtWUXTCU0ZfPdg+jTpk9r+0ENw6BDE\nZjbRppRSShVgLmtoQowx8Tk51yfu0FxtGlVoRO+GvXnnx3fYePjfG0ZOJ6xeDQczWwqtlFJKqQxp\nQeMlUS2juPmam+kyowsXky8C0K6dtS/N7NleTk4ppZTKZ7Sg8ZKiAUWJdkYTdyCOD2I/AKxHt5s1\n03U0SimlVE5pQeNFDW9qyCuNXqHf0n5s+HsDYO0avGgRnDrl5eSUUkqpfEQLGi+LahnFraVupctM\na+opPNx68/aCHD8UrpRSSl29tKDxsiL+RYgOjyb+QDzvr3yfW2+FO+7QaSellFIqJ7Sg8QENbmrA\nq41eJXJZJOv/Xk94OMyZAxdytdONUkopdfXRgsZH9G/Zn8qlKtN1Zlfuf/Aix4/DihXezkoppZTK\nH7Sg8RFF/IsQ7bSmnhYmDKVCBX23k1JKKZVdWtD4kLtuvIs+jfvQf1kkTduvZ8YM0I2clVJKqaxp\nQeNjIkMjue2624ir0IU9+y6wdq23M1JKKaV8nxY0Pqawf2Giw6P568xaitw9VKedlFJKqWzQgsYH\n3XnjnfRp0ofzjfsz5cc/vJ2OUkop5fO0oPFR/Vr048YiVdlaowtb/9Lnt5VSSqnL0YLGRxX2L8zk\nh6Oh3Dr+7+v3vJ2OUkop5dO0oPFhLW6rT+WDrzE/cQC/H/rd2+kopZRSPksLGh/XK/gdzOFqPD69\nCxeSdOpJKaWUyogWND7u4fDCMDOaDYd/Z/BPg72djlJKKeWTtKDxceXKQcOKIdx26HUGLB/AuoPr\nvJ2SUkop5XO0oMkHnE7Y/cXbVLu2Ol1m6tSTUkoplZ4WNPmA0wmJpwrTrXQ0fxz6g3dXvOvtlJRS\nSimfogVNPlCtmvVZ/0MIfZv1ZeCKgaw9qO9EUEoppVJoQZNPOJ0weza80eQtapapSZcZXTifdN7b\naSmllFI+wWcKGhF5QUR2iEiiiKwWkTuziA8VkTgROSsiW0XkyQxiHhGRTXaf60SkbbrjxUVkhIjs\nFJEEEflJROpn0E+UiOy3YxaKSJUrv+KcCQ+Hw4ch7udCRIdHs+HwBp16UkoppWw+UdCISAfgQ6Af\nUA9YByxAUK8QAAAgAElEQVQQkdKZxN8MzAEWA3WAkcA4EbnHJaYx8BXwGVAXmAnMEJGaLl2NB+4G\nOgG1gIXAIhEp79LPa0BPoDtwF3DGzq3QlV53TjRoAGXLwowZUK98Pfo27cugFYN06kkppZQCxBjj\n7RwQkdXAGmPMy/Z3AfYAHxljhmYQPwRoa4y5w6UtBihpjLnP/j4FCDTGPOgSswr4zRjTQ0SKAKeA\nB4wx811ifgXmGmPesb/vB943xgy3vwcBh4AnjTHTMsgtGIiLi4sjODj4yn5j0nn2WVi8GLZtgwvJ\n57nrs7swGH555hcK+eVpfaWUUkq5XXx8PCEhIQAhxpj4nJzr9Ts0IhIAhGDdbQHAWFXWIqBRJqc1\ntI+7WpAuvlEWMf6AH3AuXUwi0NTO7RagXLrcTgJrLpObx4SHw19/wcaNUMivENHOaDYe3sig5YPy\nOhWllFLKp3i9oAFKYxUWh9K1H8IqJjJSLpP4IBEpnEVMOQBjzGlgFfC2iJQXEYeIPI5VqJR36cPk\nMDePadUKihe3pp0A6pary5vN3uTdn94l/kCOClmllFKqQPGFgsabHgcE2AecxVor8xWQ7M2kMlOk\nCLRpAzNn/tvWt1lfbi9zuz71pJRS6qrm7+0EgCNAElA2XXtZ4GAm5xzMJP6kMeZcFjGpfRpjdgAt\nRaQoEGSMOWSvvdnu0ofY5x1K189vl7uoXr16UbJkyTRtERERREREXO60LDmd8PjjsG8f3Hjjv1NP\nd352JwOWDWBAqwFX1L9SSimVF2JiYoiJiUnTduLEiVz358uLgndjLQp+P4P497AWBddxafsKuCbd\nouCixphwl5iVwDpjTI9M8iiFVcy8aowZb7dltii4szHm6wz68NiiYIBjx+D66+Gjj+D55/9t77+0\nPwOWD2DN02sIuSHE7eMqpZRSnpavFwXbhgHPiEhnEakOjAECgWgAERksIhNd4scAt4rIEBGpJiI9\ngP/Y/aQYCbQRkd52TCTW4uOPUwJE5F4RCRORm+1HvpcAG1PGtY0A3hKRB0SkNjAJ2Iv1GHieK1UK\nWrT4dx1Nir7N+lK7bG26zOzCuYvp1zkrpZRSBZtPFDT248+vAlFYUzl3AGHGmMN2SDmggkv8TqAd\n0BpYC/QCuhljFrnErAI6Yu0fsxZoD4QbYza6DF0S+B+wCauIWQ60McYkufQzFBgFjMV6uqko1t0h\nry1YcTrhxx/B9c5cgF8A0eHRbD6ymQHLddpJKaXU1cUnppwKEk9POQHs2QMVK0JMDDz2WNpjUcui\niFoWxeqnV1P/hks2PVZKKaV8VkGYclI5UKECBAdfOu0E8EbTN7ij7B10maFTT0oppa4eWtDkU04n\nzJ0L59LVLAF+AUx0TmTr0a1ELYvyTnJKKaVUHtOCJp8KD4dTp2Dp0kuP1S5bm3davMOQlUP4df+v\neZ6bUkoplde0oMmnateGW27JeNoJ4LUmr1GnXB2delJKKXVV0IImnxKxpp1mzYLkDPY1TnnqaevR\nrfRf1j/vE1RKKaXykBY0+Vh4OOzfD79mMqtUu2xt+rXox5CVQ/hl3y95m5xSSimVh7SgyceaNIHr\nrkv7bqf0Xmv6GvXK1aPLzC6cvXg275JTSiml8pAWNPmYvz/cf3/m62gA/B3+RDuj+fOfP+m/VKee\nlFJKFUxa0ORzTids3AjbtmUeU+v6WvRr0Y+hsUNZs3dN3iWnlFJK5REtaPK5e++FokUvP+0E0KdJ\nH4LLB+vUk1JKqQJJC5p8LjAQ7rnn8tNOYE89hUez/dh2+v3YL2+SU0oppfKIFjQFgNMJsbHw99+X\nj7v9+tuJbBHJB6s+YPXe1XmTnFJKKZUHtKApAO6/39qXZvbsrGP/2+S/hJQPoevMrjr1pJRSqsDQ\ngqYAKFPGeoQ7q2kn+Pepp+3HtvPOj+94PjmllFIqD2hBU0A4nbBwIZw+nXVszTI1iQqN4sNVH+rU\nk1JKqQJBC5oCIjzcevP2Dz9kL/6Vxq9Q/4b6dJnRhcQLiZ5NTimllPIwLWgKiMqVoVatrB/fTpHy\n1NPO4zt16kkppVS+pwVNARIebi0Mvngxe/E1ytQgqqU19bRqzyrPJqeUUkp5kBY0BYjTCceOwYoV\n2T/nlUavcNeNd9Flpk49KaWUyr+0oClAQkLgxhuzP+0E4OfwI9oZza7ju3j7x7c9l5xSSinlQVrQ\nFCAi1rTTjBlgTPbPq166OgNaDmDYqmHE7on1XIJKKaWUh2hBU8A4nbBrF6xbl7PzejfqTYObGtBl\nRhcSLiR4JjmllFLKQ7SgKWBatICgoJxNO4E19TQhfAK7T+zmrSVveSY5pZRSykO0oClgChWCdu2y\nt2twetVLV2dgq4GMWD2Cn3b/5P7klFJKKQ/xmYJGRF4QkR0ikigiq0XkziziQ0UkTkTOishWEXky\ng5hHRGST3ec6EWmb7rhDRAaIyHYRSRCRP0XkrXQxE0QkOd1nrnuu2jOcTli71pp6yqleDXvR8KaG\ndJ3ZVaeelFJK5Rs+UdCISAfgQ6AfUA9YBywQkdKZxN8MzAEWA3WAkcA4EbnHJaYx8BXwGVAXmAnM\nEJGaLl29DjwL9ACqA32APiLSM92Q84CyQDn7E5H7q/W8Nm0gICDn007w79TT3pN7eXPxm+5PTiml\nlPIAnyhogF7AWGPMJGPMZuA5IAF4KpP454Htxpg+xpgtxpj/Ad/Y/aR4CZhnjBlmx7wDxAOuxUoj\nYKYxZr4xZrcx5lvgB+CudOOdM8YcNsb8bX9OXPEVe1BQENx9d+6mnQCqla7GoFaDGLlmJCt25WBT\nG6WUUspLvF7QiEgAEIJ1twUAY4wBFmEVHBlpaB93tSBdfKNsxMQCd4vIbXYudYAmQPoppVAROSQi\nm0VktIhcm+WFeVl4OCxfDv/8k7vzX27wMo0rNOapWU/p1JNSSimf5/WCBigN+AGH0rUfwpreyUi5\nTOKDRKRwFjGufb4HTAU2i8h5IA4YYYyZ4hIzD+gMtMKakmoBzBURyeK6vOrBByEpCb7/Pnfn+zn8\n+Dz8c/ae3MvTs57mQtIF9yaolFJKuZG/txPwsg5AR+AxYCPWWpuRIrLfGDMZwBgzzSV+g4j8AfwF\nhAI/ZtZxr169KFmyZJq2iIgIIiLyZvnNDTdAgwbWOponnshdH1Wvq8pE50Qe//Zx/kn8h68f+ZoS\nhUu4N1GllFJXpZiYGGJiYtK0nTiR+xUdvlDQHAGSsBbduioLHMzknIOZxJ80xpzLIsa1z6HAYGPM\n1/b3DfaC4zeAyRkNbIzZISJHgCpcpqAZPnw4wcHBmR3OE+HhMGgQJCZC0aK56+PR2x+ldGBpnFOc\nhE4M5fuO31OueGY3zpRSSqnsyegf+fHx8YSEhOSqP69PORljLmBN9dyd0mZP59yNtcYlI6tc4233\n2u2Xi7knXUwgVjHlKpnL/L6IyE3AdcCBzGJ8hdMJZ87A4sVZx15Oq1tasaLrCg6cOkDj8Y3ZenSr\nexJUSiml3MTrBY1tGPCMiHQWkerAGKxiIxpARAaLyESX+DHArSIyRESqiUgP4D92PylGAm1EpLcd\nE4m1+Phjl5jZwFsicp+IVBKRh7CelPrWHreYiAwVkQb28buBGcBWrAXGPq16dahaNXePb6dXp1wd\nVnVbRWH/wjQe35jVe1dfeadKKaWUm/hEQWOvU3kViAJ+A+4Awowxh+2QckAFl/idQDugNbAWqwjp\nZoxZ5BKzCmt9THc7pj0QbozZ6DJ0T6zHvf+HtYZmKPAJ8I59PMnOZSawBWtPm1+A5vadJZ+W8rLK\nWbOsBcJXqtI1lVj51Eqql65Oq4mtmL1l9pV3qpRSSrmBmJy8llllSUSCgbi4uDivr6EBiI2FJk1g\n5Upo3Ng9fSZeSKTTt52YuWUmn7T7hO4h3d3TsVJKqauayxqaEGNMfE7O9Yk7NMpzGjSA66/P/SZ7\nGSkaUJSvH/ma5+s/z7NznqXfj/3QwlgppZQ3aUFTwPn5WXvSzJgB7qw5/Bx+jGo7isF3DyZqeRTP\nzH6Gi8kX3TeAUkoplQO5KmhE5EkRaefyfaiIHBeRWBGp5L70lDs4nbBtG2ze7N5+RYTXm77OJOck\nJq6bSPiUcM6cP+PeQZRSSqlsyO0dmr5AIoCINAJewNpF9wgw3D2pKXe5+24oVsy9006unqjzBN93\n/J7lu5bTcmJL/j7zt2cGUkoppTKR24KmAvCn/WsnMN0Y8ynWhnTN3JGYcp8iRaw3cLvj8e3M3Fv5\nXpZ3Wc7uE7tpPL4xf/7zZ9YnKaWUUm6S24LmNNbmcmBtaLfQ/vVZIJd70ipPCg+HNWtg/37PjVGv\nfD1WdVuFn8OPxuMb88u+Xzw3mFJKKeUitwXNQmCciIwDqvLv26lvB3a6IS/lZu3aWQuEZ83y7Di3\nlLqFlU+tpPK1lQmdGMrcbelfXK6UUkq5X24LmhewXiFQBnjYGHPUbg8BYjI9S3nNtddCixaenXZK\nUTqwNIs7L6b1ra15MOZBPv/tc88PqpRS6qqWq5dTGmOOY+2ym7693xVnpDwmPBxefRVOnoSgIM+O\nFRgQyPRHp9Nzbk+6zerGvpP7eKv5W1iv6VJKKaXcK7ePbbcRkaYu318QkbUi8pWIlHJfesqdwsPh\nwgWYPz9vxvN3+PNJu08Y2HIg7yx9h+fmPKd71SillPKI3E45vQ8EAYhIbeBDrHU0t5D2BZHKh1Sq\nBHXreu7x7YyICG82f5MJ4RMY/9t42k9tT8KFhLxLQCml1FUhtwXNLVgvcwR4GJhjjOmLtbamrTsS\nU57hdML338P583k7bpe6XZgdMZslO5bQamIrjiQcydsElFJKFWi5LWjOA4H2r1sDP9i//gf7zo3y\nTU6ntYZm2bK8H7vtbW1Z2mUpO47voPH4xmw/tj3vk1BKKVUg5bag+QkYJiJvA3cB39vtVYG97khM\necYdd1hTT3k57eSq/g31iX0qFoOh8fjGxB/I0ctUlVJKqQzltqDpCVwE/gM8b4zZZ7e3BfJoyanK\nDRHrLs3Mme59WWVOVL62MrFPxVLpmkq0iG7BD3/9kPVJKo3jZ48zcPlAOn/XmW1Ht3k7HaWU8rpc\nFTTGmN3GmPuNMXWMMeNd2nsZY15yX3rKE8LDYd8+iIvzXg5lipVhSecltKjUgnZftWPSukneSyYf\nOXzmMH0X96XSiEoMWjGIJTuWUPuT2gxcPpDzSXm8MEoppXxIbu/QICJ+IvKwiLxlfx4SET93Jqc8\no1kzKFXKe9NOKYoVKsaMx2bQpU4XnpzxJINXDMZ467aRj9t7ci//N///qDSiEqN+HsWzIc+y4+Ud\nbH1xKy83eJnIpZHUG1uPlbtXejtVpZTyitzuQ1MF2ARMAtrbny+ADSJS2X3pKU/w94cHHsibXYOz\nzMXhz6cPfEpki0j6LulLz7k9SUpO8nZaPmP7se08O/tZbh15K5PWTaJPkz7s+r9dDL1nKOWKlyMw\nIJAh9wzh1+6/UiygGE0nNOX5Oc9z/Oxxb6eulFJ5Krd3aD4C/gIqGGOCjTHBQEVgh31M+bjwcFi/\nHv70gZdiiwj9Qvvx2QOfMTZuLP/5+j8kXkj0dlpetfHwRp747gmqjqrKjC0zGNhqILv+bxeRoZFc\nW/TaS+LrlqvLqm6r+KjNR3zxxxfU+F8Nvt7wtd7xUkpdNXJb0LQA+hhj/klpsN/n9Lp9TPm4sDAo\nUsQ37tKkeDr4aWY8NoMFfy6g9eTWHE04mvVJBUz8gXgenvYwtUbXYunOpQwPG87Ol3fSp0kfShQu\ncdlz/Rx+vNjgRTb22EiDGxvw6DeP8kDMA+w6viuPsldKKe/JbUFzDsjoT9fiWHvUKB9XrBi0bu1b\nBQ3A/VXv58cnf2Tr0a00ndD0qvnLeOXuldz35X2EfBrCuoPr+PSBT/nrpb94scGLFA0omqO+KpSs\nwIzHZvDto9/y28HfuH307QxfNVxfO6GUKtByW9DMAT4VkQbyr4bAGGCW+9JTnuR0wsqVcPiwtzNJ\nq8FNDYh9KpbzSedpNL4Raw+u9XZKHmGMYdH2RYRGh9J0QlN2n9jNl+2/ZHPPzTwd/DSF/ApdUf8P\n1XiITS9somvdrrzywys0GNdA9/1RShVYuS1oXsJaQ7MKOGt/YoE/gf9zT2rK0x54wNqLZs4cb2dy\nqduuu43Yp2K5ocQNNJ/QnMXbF3s7JbcxxjBryywajm/IPZPv4cyFM3zX4Tt+f/53OtbuiL/D321j\nBRUOYtR9o4jtFsuFpAvc+dmdvLLgFU6fP+22MZRSyhfkdh+a48aYcKydgf9jf6oaYx4yxuTq8Qr7\njd07RCRRRFaLyJ1ZxIeKSJyInBWRrSLyZAYxj4jIJrvPdSLSNt1xh4gMEJHtIpIgIn+KyFsZ9BMl\nIvvtmIX2U1753vXXQ+PG3n98OzNli5dlaZelNKnYhLZftuXL37/0dkpXJCk5iSnrp1BnTB3Cp4RT\n2K8w8zvN5+enf8ZZ3YlDcr2LQpYa3tSQuO5xvNvqXUb/OprbR9/O91u/z/pEpZTKJ7L9J6iIDEv/\nAXoALe1PD5f2HBGRDlhv7O4H1APWAQtEpHQm8TdjTXstBuoAI4FxInKPS0xj4CvgM6AuMBOYISI1\nXbp6HXjWvo7qQB+gj4j0dOnnNaydkbtjvebhjJ3blc0H+AinE374Ac6c8XYmGSteqDizHptFpzs6\n8fh3jzN05dB89+TOhaQLTPhtAjVH1yRiegQ3lLiB5V2Ws7zrcsKqhCEieZJHgF8ArzV9jfXPr6fa\nddW4P+Z+Hv36UQ6cOpAn4yullCdJdv9yEJEfs9mnMca0ylESIquBNcaYl+3vAuwBPjLGDM0gfgjQ\n1hhzh0tbDFDSGHOf/X0KEGiMedAlZhXwmzGmh/19NnDQGPOMS8w3QIIxprP9fT/wvjFmuP09CDgE\nPGmMmZZBbsFAXFxcHMHBwTn5bfCKbdugalX47juruPFVxhje+fEdBq4YyEt3vcSwsGH4OXx7H8fE\nC4l8/tvnDI0dyu4Tu3FWd/Jmszepf0N9b6eGMYav/viKXgt6cT7pPENaD+GZkGc8epdIKaWyEh8f\nT0hICECIMSZHi/6yPVlvjGmZ08SyQ0QCgBDgXZexjIgsAhplclpDYFG6tgXAcJfvjbDu+qSPCXf5\nHgs8IyK3GWO2iUgdoAnQy87tFqAc1p2glNxOisgau/9LCpr85rbboGZNa9rJlwsaEWFAqwHcFHQT\nPeb2YP/p/Ux+aDJF/It4O7VLnD5/mjG/juGD2A84nHCYx2o9xhtN36DW9bW8nVoqEaHTHZ1oU6UN\nfRb24bnvn2Py75MZe/9Ybr/+dm+np5RSOeYL/xwrDfhh3fVwdQirmMhIuUzig0SkcBYxrn2+B0wF\nNovIeSAOGGGMmeLSh8lhbvmO02ktDL6YD57qfbb+s3z76LfM2TqHeyffy7HEY95OKdWxxGNELYui\n0ohK9F3cl/ur3s+Wnlv4sv2XPlXMuLou8DrGh4/nxyd/5HDCYeqNrcfbS97m7MWz3k5NKZUNU9dP\n5daRt/JB7Ackm2Rvp+NVvlDQeFMHoCPwGNbanSeB/4rIE17NKo+Fh8PRo9Yj3PlBePVwlnRewobD\nG2g6oSl7Tuzxaj5/n/mbNxa9QaURlRj802A61e7EXy/9xbgHx1Hl2vyxfjz05lDWPbeO15u+zpCV\nQ7jjkzv4cUd2Z5mVUnntzPkzdJvZjcemP0aZYmX478L/ct+X93HodPp/f1893Pd8aO4dAZKAsuna\nywIHMznnYCbxJ40x57KIce1zKDDYGPO1/X2DveD4DWCyHSv2ea4/JWWB3zK9IqBXr16ULFkyTVtE\nRAQRERGXO80r6teHG26wpp1a5JN9nhtVaETsU7G0+bINjcY3Yl6nedQuWztPc9h7ci/vr3yfz+I/\nw8/hR4/6PejdqDdli6f/scsfivgXIaplFI/Veoxn5zxLq0mt6FK3Cx/c8wHXBV7n7fSUUrb4A/FE\nTI9g78m9fP7g53Sp24WF2xfyxHdPUGdMHSY9NIl7K9/r7TSzFBMTQ0xMTJq2EydO5L5DY4zXP8Bq\nYKTL95RFwf/NJP49YF26tq+AuS7fpwAz08WsBEa7fD8CdE8X8waw2eX7fqCXy/cgIBF4JJPcggET\nFxdn8pPnnzfmlluMSU72diY5s//kflN3TF0TNDjILNm+JE/G/PPon+aZWc+YgKgAU+q9Uibyx0hz\nNOFonoydV5KSk8ynv35qrnnvGlN6aGkzae0kk5zffjiUKmCSkpPMsNhhJiAqwNQbU89sPrw5zfGD\npw6asMlhhkjMf3/4rzl38ZyXMs29uLg4g7XUI9jktJbI6Qme+ACPAglAZ6zHp8cCR4Ey9vHBwESX\n+JuBU8AQoBrWY9fngdYuMY2wXtHQ246JxNoAsKZLzARgN3AfUAl4CPgbeNclpo+dywNAbWAGsA0o\nlMm15MuCZv5866dh3TpvZ5JzJ86eMPdMuscUGlDITPljisfGWX9ovek0vZNx9HeY69+/3gz9aag5\nefakx8bzBQdOHTAdvu5giMS0ntTa/Hn0T2+npNRV6eCpg6bNF20MkZje83ubsxfOZhiXlJxk3l/5\nvvGP8jd3fnpnvvv/bL4vaIxVCPQAdtp3P1YB9V2OTQCWpItvjrWIN9EuMJ7IoM+Hgc12zO9AWLrj\nxYBhWG8JP2P30x/wTxcXad+pScB6UqrKZa4jXxY0584ZExRkTFSUtzPJnXMXz5nHv33cEIn5MPZD\nt/b9675fzUNTHjJEYioMq2BGrRllEs4nuHUMXzd361xTaXglU2RgEfPu8nfN+YvnvZ2SUleNBX8u\nMGXfL2uuf/96M2/bvGyd8/Pen03lkZVNiXdLmC9//9LDGbpPgShoCsonvxY0xhjToYMxwcHeziL3\nkpOTzesLXzdEYnrN72WSkpOuqL8Vu1ak/ouoykdVzLi4cfnyFq67nD532ryy4BXj19/P1B5d26za\ns8rbKSlVoJ27eM68uuBVQyQmbHKYOXjqYI7OP3H2hOk0vZMhEvPkd0+aU+dOeShT97mSguZqf8pJ\nuXA6IT4edu/2dia5IyIMbj2YUW1HMWL1CDpO78i5i+eyPtGFMYYf/vqBFtEtaDahGXtP7uWr9l+x\n6YVNdAvudsUvjMzPihUqxgf3fsAvz/xCIb9CNB7fmBe+f4ETZ69gEZ9SKkPbjm6j8fjGjFwzkg/u\n+YC5nebm+IGDoMJBfNH+CyY6J/LNxm8IHhtcoF9QqwWNStW2LQQEwKx8/r70nnf15JtHv2HG5hm0\n+bINx89m/XqxZJPMzM0zaTCuAWFfhJF4IZEZHWaw7rl1RNSOcOsLI/O7euXrsebpNQwPG87EdROp\nObom3276NuUOpVLqChhjmLh2IvXG1uPkuZOs6raKVxq/ckW7eHeu05n4Z+MpUbgEDcc1ZMTqEQXy\n/69a0KhUJUtCy5a++7LKnGhfoz2LOi9i3cF1qXdaMpKUnETMHzHUGVMH51QnRQOKsuDxBax5eg3h\n1cP1VQCZ8HP48XLDl9n4wkZCyofw8LSHcU51en1PIKXysxNnT9Dp2050mdmFR25/hPhn4wm5IcQt\nfVe9riqxT8Xy4l0v0mtBL+6PuZ/DZw67pW9foX9aqzScTli2DI75zga8uda0YlNWPrWSk+dO0mh8\nIzb8vSH12Pmk83z+2+fU+F8NOn7bkZuCbmJF1xUs67KMeyvfm2cvjMzvKpasyMzHZvLNI9/wy75f\nqDm6JiNXjyQpOcnbqSmVr6zeu5p6Y+vx/bbv+ar9V0wIn0DxQsXdOkZh/8J8GPYh33f8nl/2/UKd\nMXVYvH1x1ifmE1rQqDQefNB6BcLcud7OxD1qlKnBqm6ruLbotTSd0JQf/vqBj3/+mCofVaHbrG7U\nur4WvzzzC/M6zaNpxabeTjdfEhEervkwm17YxBN3PEGvBb1oNL4Raw+u9XZqSvm8pOQk3l3xLk0/\nb0rZ4mVZ++xaImp7dgPW+267j3XPraNmmZrcM/ke+i7uy4WkCx4dMy9oQaPSuPFGuPPOgjHtlOKG\nEjewvMtygssHE/ZFGC/Pf5nmlZqz/vn1fNvhW594+3VBULJISUa3G83Kp1aSeDGR+p/W578//Jcz\n5894OzWlfNK+k/u4Z/I9vLXkLV5r8hrLuyznllK35MnY5UuU54cnfmDw3YN5P/Z9mkc3Z8exHXky\ntqdoQaMu4XTC/PlwtgC9n7BkkZLM6zSPMe3GsKXnFr5o/4W+VdpDGlVoRFz3OAa0HMCon0dR65Na\nzP9zvrfTUsqnzN4ymzpj6rDl6BYWd17MoLsHEeAXkKc5OMTBa01fY0XXFRw8fZC6Y+sydf3UPM3B\nnbSgUZcID4fTp2HJEm9n4l6F/ArxbP1n880LI/OzQn6FeKPZG6zvsZ7KpSrT9su2REyPuKpfnKcU\nQOKFRHrO7cmDUx6kScUmrHtuHS1vaenVnBre1JC1z66lbZW2PDb9MZ6e9XS+vLOqBY26RM2aUKUK\nzJzp7UxUflfl2iosfGIhk5yTWPjXQqr/rzrj4seRbJK9nZpSeW7j4Y00GNeAcfHj+N99/2NGhxmU\nDizt7bQA6y52zMMxjH9wPDHrY6j/WX3WHVzn7bRyRAsadQkR6y7NzJmQrH/vqCskIjxR5wk299xM\neLVwnpn9DKHRoWw6vMnbqSmVJ4wxjPl1DCGfhpBskvnlmV/ocWcPn3uaUkR4qt5TxHWPo7BfYe4a\ndxej1ozKN3vWaEGjMuR0wqFDsGaNtzNRBUXpwNJEO6NZ3HkxB/6/vfsOs6o827//vehFmoAMKNKU\nqqADqFiJggoJYosKGAuWoOaRBx9fyxuwJRY0ARFjMGKPgMSOigaiJIooMhQRsERRUbooKCJl5vr9\nca/N7NnMAFNg7T1zfo5jHZu99r3W3Atl5py7/riCLuO6cMuMW4q9mrNIJlm3aR1nTT6LK165gou6\nXJxJeKoAACAASURBVMTsy2ZzaJND467WTrVv1J53L32XIV2HcPVrV9N/Un/W/rQ27mrtkgKNFKpH\nD2jcWN1OUvZObHUiHwz5gOuOuY7b37qdLuO68O8v/h13tUTK3L+/+DddxnVhxhczeO6c5/jrr/5K\nraq14q7WbqlRpQZj+ozhpfNe4p1l72x/jnSmQCOFqlwZ+vUrX9O3JX3UrFqTP574R+b/dj4NazWk\n5+M9ueTFS1j63dKMad4WKcq2vG3c9OZNnPjEibRu0JoFQxZwRocz4q5WifRr148FQxbQtmFbTnz8\nREa8MYJtedvirlahTN88ypaZZQM5OTk5ZGdnx12dUpkyJSy0t2QJtG8fd22kvMrzPB7KeYjrp1/P\n+s3r2bfmvmQ3zSY7Kzu8Ns2mzb5ttA2FZIQvv/+Sgc8N5L2v3+OWnrdw47E3UrlS5birVWq5ebnc\n9fZd3DzjZo484EgmnDmBFvVblPnXmTt3Ll27dgXo6u7F2klTgaaMladAs2kTNGoEV14Jd98dBguL\n7CnrNq1j1rJZ5KzIYe6KucxdMZdlG8LeUHWq1eHwpofTtWnX7SGnXcN25eIHhZQfkxdN5vIpl1O/\nRn0mnDWBo5sfHXeVytw7y95hwLMD2LB5A+P7jeesjmeV6f0VaNJIeQo0AL//PdxxR2ipefBByMqK\nu0ZSkazZuIZ5K+dtDzhzV8zls+8+A6BW1Vp0adJle8DJbppNx8YdqVa5Wsy1lopm45aNDH1tKA/P\ne5hzOp3Dg796kPo16sddrT3mu03fcfnLl/PM4me4PPtyRp86uszGBinQpJHyFmggjKP57W8hNxfG\njYOzz467RlKRff/z98xbEYWclSHkfLz2YxynWuVqdG7SuUB31aFNDqVGlRpxV1vKqXkr5jHg2QEs\n27CMsX3GcvFhF6fddOw9wd0ZP3c8Q18bSqsGrZh01qQymb2lQJNGymOgAVizBq64Ap59FgYMgPvv\nh333jbtWIsGPW35kwcoFBULOotWLyPVcKltlOu3Xafu4nK7NutKlSRdqV6sdd7Ulg7k7Y94bw/XT\nr6dj445MOmsS7Rq1i7tae93iNYs575nz+HTdp4w6eRRDug0pVaBToEkj5TXQALjDxIlw1VVQsyaM\nHw99+8ZdK5HCbdq6iYWrFxborlq4eiFbcrdgGO0btS/QXXV41uHUq1Ev7mpLBli9cTUXv3gxr376\nKsOOGsadJ91J9SrV465WbDZt3cS1/7yWB+Y8wBntz2D8aePZt2bJfuNVoEkj5TnQJHzzDVx6adjA\n8tJLYdQoqFMn7lqJ7NqW3C0sXrO4QMiZv3I+m7ZtAqBNgzbbA07Xpl05vOnhabM0vaSHaZ9N4zfP\n/4Y8z+Px0x+nz8F94q5S2njhoxcY/OJgalerzYQzJ3Bci+OKfQ8FmjRSEQINhNaahx6Ca64JC/A9\n+ij07Bl3rUSKb1veNj759hNyluds77Kat2IeP2z5AYAD6x24wzTypnWaxlxr2du25G5h+BvDueed\ne+jdujdPnPEEWftolkSqZeuXMei5QcxcNpObjr+J4ccPL9ZsRAWaNFJRAk3C55/DxRfDf/4D//u/\nYUZUzZpx10qkdPI8j8/WfZbfkrNyLjnLc/ju5+8AyNonq8CYnOym2TSv27xCDAatiD799lMGPjeQ\nBSsXcMdJd3BNj2u0LtJO5Obl8sf//JHb/nMbxx54LH8/4+80r9d8t65VoEkjFS3QQNjAcswYuPFG\naNkSnngCjjgi7lqJlC1356v1XxVYJydnRQ6rN64GoGHNhgXG5GQ3zaZ1g9b6wZfhnlzwJFe+eiVZ\n+2Qx8ayJdGvWLe4qZYy3vnyLgc8NZOOWjTzS/xFOb3/6Lq9RoEkjFTHQJCxZAhdcAPPmhXAzYgRU\n05IgUo65Oyt+XFFgTE7ygoA1q9SkQ+MOdGzckU6NO21/bdWglYJOmtuweQNXvnIlTy18igu7XMjY\nPmOpU12DBYtr3aZ1XPrSpTz/0fNc2e1K/nTyn6hZtehm/HIRaMzsKuBaIAtYAPyPu7+/k/I9gT8D\nnYCvgNvd/fGUMr8GbgNaAp8AN7j71KTPlwKFrd38F3f/n6jMo8CFKZ+/5u6Fzu+pyIEGYNs2uOsu\nuPVW6NQptNZ07hx3rUT2rsSCgItWL2LRmkUsXrOYRWsWsWHzBiAEnfaN2tNpv050bNQxvDbuSKv6\nrbT6cRp47+v3GPjcQNZsXMO4X41j4KED465SRnN3xs0Zx7DXh9G2YVsmnT2Jjo07Flo24wONmZ0L\nPA5cDswGhgG/Btq6+w57lptZS+BD4AHgYaAXcC/Q192nRWWOBv4NXA+8AgyK/ny4uy+OyjQEkr97\nHAr8E+jp7m9FZR4F9gMuAhId5JvdfX0Rz1KhA03CvHmhtebjj+G22+Daa6FKlbhrJRIfd+ebH74J\n4WZ1fshJDjo1qtSgQ6OUFp39Oino7CV5nsfdM+9mxJsjyG6azcSzJtK6Qeu4q1VuLFy1kPOePY+l\n3y3l3lPv5bLsy3YYd1YeAs27wHvuPjR6b8Ay4D53v7uQ8iOBPu7eOencRKBeouXEzCYBtdz9tKQy\ns4B57n5lEfVIhKK2Seceje575m4+iwJNZPNmuPlmuOeeMKbm8cehbdtdXydSkbg7y39Ynt+Ss3oR\ni9eG1/Wbw+9NNarUCC06Sd1WHRt3pHWD1go6ZWT5D8v5zfO/4c2lb3LDsTdwa89bqVq5atzVKnd+\n2voT17x+DQ/mPMjZHc/mb7/6Gw1qNtj+eWkCTey/M5tZVaArcEfinLu7mU0HehRx2VHA9JRzrwOj\nk973IHRJpZbpv5N6DAL+VMjHPc1sFfAd8AYw3N3XFVE3iVSvHrqfTjsNLrwQDjsMRo4MC/NV0vAB\nEQDMjP3r7s/+dffn5DYnbz+fCDqJlpzE68ufvLxD0Ekdo6OgUzxTPp7CxS9eTLXK1Zh+wXRObHVi\n3FUqt2pVrcW4X42jV+teXDblMg578DAmnjWxTDbyjD3QAI0I3T6rUs6vAopaRzqriPJ1zay6u2/e\nSZmiFg44A6hH6PpKNhV4FlgKtAHuBF41sx6eDs1bGeDoo2H+fLjhBrj66rA31COPQIuy33lepNxI\nDjq92/Tefj4xEDm12+qVT17ZHnSqV65e6BidNg3aKOgk+Xnbz1w37TrGzh5Lv7b9eKT/I1pIcS85\nu+PZdG/WnUHPDeL4R4/n1p63csOxN5TqnukQaNLFYGCqu69MPunuk5PeLjKzhcBnQE/gzaJuNmzY\nMOrVK7iM+oABAxgwYECZVTiT1K4NY8dC//4weDAcemiY6n3RRaClO0R2n5nRrE4zmtVpVmjQSR2j\n8+qnr/L9z98D+UEndYxORQw6i9csZsCzA/h47ceM7TOWq7pfpXWE9rJ3pr5Dg+cb0Prb1gx/ajij\na43moFoHlfh+6RBo1gK5QJOU802AlTsWh+h8YeU3RK0zOyuzwz3N7EDCwOJdTpJ396VmthY4iJ0E\nmtGjR1f4MTSF6dULFi4Mi/ANHgzPPRdWHM7SgpsipZIcdHq17rX9vLuz8seVBcboLFqziKn/nVog\n6LRr1G6HMTpt9m1DlUp758eEu5PruWzN3crWvK27/bold0uxr9mweQN/y/kbrRq0YvZls+ncRFMx\n45D8S/6ML2Yw6LlBfPTlR/BOye4Xe6Bx961mlgOcBLwE2wcFnwTcV8Rls4DUDTROjs4nl0m9R++U\nMgmDCd1Rr+6qvmZ2ANAQWLGrslK4evXCVglnnAGXXx6md//1r3DOOXHXTKT8MTOa1mlK0zpNCw06\n27utosHIr/33te0rIlerXG17i06Lei3IzcstGBB2ER625G4pVtAoa1UqVaFqpapUrVx1h9dLDr+E\nkb1HUqtqrTL/ulJ8PVv2ZMGQBZx575m8xVsluke6zHI6B3gMGEL+tO2zgfbuvsbM7gSaufuFUfmW\nwELCtO1HCMElMUNpelSmBzADuJEwbXsAcAOQnZi2HZUzwviYp9z99yn1qg3cTBhDs5LQKjMSqA10\ndvcd/gVqllPxrF0LV1wBzzwD550H998PDRvGXSuRisvdWbVx1Q5r6Hyz4ZsQEFLCQbXK1QoNDFUr\nFXFuJ6/VKlcr9jU7e1UXUubJycmhW7dukImznCCMUzGzRoRF8JoA84FT3H1NVCQLaJ5U/gsz+yVh\nVtPVwNfAJYkwE5WZZWYDgduj41Ogf3KYifSK7v1oIVXLBToDFwD1geWEmVI3FRZmpPgaNYLJk2HS\npDD76ZBDYPx4+OUv466ZSMVkZmTtk0XWPlmc1PqkuKsjFUxpQmhatNCUJ2qhKbnly+HSS2HqVLjk\nEhg1CurWjbtWIiKyt5RmHRqtBiJpo1kzeOWVMEj46afDlglvFjnsWkREJJ8CjaQVs9BK88EHYefu\nE08MM6J++inumomISDpToJG01KoVvPEG3HsvPPggHH44vPtu3LUSEZF0pUAjaatSJRg6NGx0Wb8+\nHHMM/P73sGVL3DUTEZF0o0Ajaa99e5g5M+zafc890L07LFgQd61ERCSdKNBIRqhSJbTOzJ4N7iHU\n3HEHbNsWd81ERCQdKNBIRjnsMHj/fbj2WhgxAo49Fj7+OO5aiYhI3BRoJONUrx5aZ95+G9atCwOG\n77sP8vLirpmIiMRFgUYyVo8eMH9+mOY9dGjY+PLLL+OulYiIxEGBRjJarVqhdeZf/4LPPoNDD4WH\nHw7jbEREpOJQoJFy4cQTw2J8v/51aLHp1w9WaD90EZEKQ4FGyo169ULrzEsvwZw50KlT2PRSRETK\nPwUaKXf69YNFi6B3bxgwAM49F9aujbtWIiKyJynQSLnUsGHY4HLSJJg+HQ45BKZMibtWIiKypyjQ\nSLl27rnw4YfQrRucdhoMHgxLl8ZdKxERKWsKNFLuNW0aWmcefhiefRZat4YTTgjv16+Pu3YiIlIW\nFGikQjALrTPLl8OTT4bF+S67DLKyYOBAeO01baMgIpLJFGikQqldG84/H/75T1i2DG69NWx02acP\nNG8etlT44IO4aykiIsWlQCMV1v77w3XXhTE2OTlwzjnw+OPQpUvYM2r0aFi5Mu5aiojI7lCgkQrP\nDLKzYcyY0CX10ktw8MFwww1wwAHwy1+GGVObNsVdUxERKYoCjUiSqlXDOjb/+Edonbn/fvjuOzjv\nvDDe5rLL4K23tLWCiEi6UaARKUKDBjBkCLzzDnzySdgAc9o0OP54aNMGbrkl7B8lIiLxU6AR2Q0H\nHwy33Qaffw7//jf84hcwahQcdBAceyz87W/w/fdx11JEpOJKm0BjZleZ2VIz22Rm75pZ912U72lm\nOWb2s5l9YmYXFlLm12a2JLrnAjPrk/L5UjPLK+QYm1LuNjNbbmY/mdk0MzuobJ5aMk2lSqGF5uGH\nQ5fUhAlQpw5ccUXokjr3XHjlFdi6Ne6aiohULGkRaMzsXODPwM3A4cAC4HUza1RE+ZbAy8C/gC7A\nGGC8mfVOKnM0MAF4CDgMeBF4wcw6Jt2qG5CVdPQGHJicdJ/rgd8BlwNHABujulUr5WNLhqtVK+wV\nNXUqfP013H47LFkCv/pVGEw8bBjMm6fxNiIie4N5Gny3NbN3gffcfWj03oBlwH3ufnch5UcCfdy9\nc9K5iUA9d+8bvZ8E1HL305LKzALmufuVRdTjXqCvu7dNOrccuMfdR0fv6wKrgAvdfXIh98gGcnJy\ncsjOzi7uX4VkOPewrs0TT8BTT8Hq1WEfqQsvDAv4NWsWdw1FRNLX3Llz6dq1K0BXd59bnGtjb6Ex\ns6pAV0JrCwAeUtZ0oEcRlx0VfZ7s9ZTyPXajTGo9BgEPJ51rRWi5Sa7bBuC9ndRNKjCzsIbNqFHw\nzTeh+6lTJxg+PCzcd+qpoZvqp5/irqmISPkSe6ABGgGVCa0eyVYRwkRhsoooX9fMqu+iTFH3PAOo\nBzye8nW8mPcRAaBKFejbN+z4vXIljBsHGzfCoEFhvM3gwWGAcV5e3DUVEcl86RBo0sVgYKq7a21Y\nKXP16+evYfPf/8L//V8IMz17hs0yR4wIU8NFRKRkqsRdAWAtkAs0STnfBCgqXKwsovwGd9+8izI7\n3NPMDgR6AacX8nUsui65laYJMK+IugEwbNgw6tWrV+DcgAEDGDBgwM4ukwqgTRu4+Wa46SaYOTOM\ntxk7Fv74RzjqKLjggjBbat99466piMieM3HiRCZOnFjg3Pr160t8v3QeFPwVYVDwPYWUv4swKLhL\n0rkJQP2UQcE13b1/UpmZwILUQcFmdgtwGdDc3fNSPitqUPAF7v6PQuqmQcFSbJs2wZQpIdy89hpU\nrhxWLL7ggjDupprm1IlIBZDRg4Ijo4DLzOwCM2sPjANqAY8BmNmdZpY8tmUc0NrMRppZOzO7Ejg7\nuk/CGOBUM7smKnMLYfDx/clfOApPFwGPpYaZyL3AcDPrZ2aHAk8AXxOmgYuUiZo1w+aYL78cBhPf\ndVdYhbh//7CJ5tVXw5w5mgIuIlKUtAg00fTna4HbCF05nYFT3H1NVCQLaJ5U/gvgl4RuovnAMOAS\nd5+eVGYWMJCwfsx84Eygv7svTvnyvaJ7P1pE3e4GxgIPEmY31SS0Dm0p+ROLFK1Jk/w1bBYsgIsu\nCntLde8epoCPHBnWvRERkXxp0eVUnqjLSfaEbdvgX/8KXVLPPw8//wwnnRTWtjn1VGjaNO4aioiU\nXnnochKRnahSBU45JSzWt3IljB8ftlcYPDgs1te5M1x7Lbz+uta4EZGKSYFGJMPUrRuCzIwZYSXi\nCROga9ew3s2pp4bZUb17wz33hC4rrXMjIhWBAo1IBmvcOOwn9eijsGwZLFoUBhRXrRqmhh92WOiO\nOv/80F21YkXcNRYR2TPSYR0aESkDZtCxYzj+939h82Z45x345z/D8dRTodwhh8DJJ4fjuOPCJpsi\nIplOLTQi5VT16vCLX8Cdd0JOTuiemjgxzJZ6+umC3VN33w3z56t7SkQylwKNSAXRuDGcdx488kh+\n99TIkWHRvltvhcMPz++eevxxWL487hqLiOw+dTmJVEDJ3VNDhxbsnpo2Td1TIpJ51EIjIgW6p+bM\nKbx7qkED6NVL3VMikp4UaERkB6ndU4sXhyBTvXp+91RWFgwapO4pEUkP6nISkZ0ygw4dwpHonpo1\nK3/21MSJYY+pQw4JA4xPPhmOP17dUyKyd6mFRkSKpXp16NkT7rgjdE+tWhVCzRFHhD2n+vQp2D01\nb566p0Rkz1OgEZFSSXRPPfwwfPXVjt1T2dnqnhKRPU9dTiJSZna3e6pTp/zZU+qeEpGyoBYaEdlj\nUrunVq8Oe04deWTB7qmTTgpr4syaBZs2xV1rEclEaqERkb2mUSM499xwuMNHH+WvfXPbbWGn8MqV\nwwDj7t2hW7dwHHpoWABQRKQoCjQiEovU7qktW2DhwtCSM2cOvP9+2HQzNzeEmS5dQrhJBJ0OHaCK\nvoOJSETfDkQkLVSrBl27huO3vw3nNm0Ki/glQs6MGTBuXGjdqVUrrIeTaMXp3h0OPhgqqSNdpEJS\noBGRtFWzJvToEY6EH34IU8Hffz+EnFdegTFjwmd16oRAlGjF6d4dWrYMrUEiUr4p0IhIRqlTJ8yM\nOv74/HPffRd2FE+EnKefhnvuCZ/tu2/Brqpu3WD//RVyRMobBRoRyXiJhfx69co/t2pVCDmJ8Tjj\nx8Ptt4fPsrIKdlV16wb77RdP3UWkbCjQiEi51KQJ9O0bDgjjbpYvz2/Fef99uO8+WLcufN68ecFW\nnG7dQlASkcygQCMiFYJZ6Graf384/fRwzh2++CI/5MyZA3fdBRs2hM/btCnYipOdHbq8RCT9KNCI\nSIVlBq1aheOcc8K5vDz49NP8Vpw5c+Cll8KMKzNo375gyDnssDB4WUTipUAjIpKkUiVo1y4cgwaF\nc9u2wZIlBdfIefrpsHZOYiHA5DE5WghQZO9LmxUbzOwqM1tqZpvM7F0z676L8j3NLMfMfjazT8zs\nwkLK/NrMlkT3XGBmfQop08zMnjSztWb2U1QuO+nzR80sL+V4tWyeWkQyQZUqIaRcfDH85S8we3aY\nPj5nTnjfvXv48+9+F0JNnTrh3ODB8Kc/wauvhq4t7TousuekRQuNmZ0L/Bm4HJgNDANeN7O27r62\nkPItgZeBB4CBQC9gvJktd/dpUZmjgQnA9cArwCDgBTM73N0XR2XqAzOBfwGnAGuBg4HvUr7kVOAi\nIDHRc3NZPLeIZK6iFgJcsCC04OTkwIcfwuTJsHFj+LxWrbDCcceO4ejUKby2bBlaekSk5Mzd464D\nZvYu8J67D43eG7AMuM/d7y6k/Eigj7t3Tjo3Eajn7n2j95OAWu5+WlKZWcA8d78yen8X0MPdT9hJ\n3R6N7nvmbj5LNpCTk5NDdnb2LsuLSPmWlwfLlsHixTseicHHNWqEsTmJoJM42rTR9g5SscydO5eu\nXbsCdHX3ucW5NvZ/KmZWFegK3JE45+5uZtOBHkVcdhQwPeXc68DopPc9CK0+qWX6J73vB7xmZpOB\nE4BvgAfcfXzKdT3NbBWh5eYNYLi7r9vVs4mIVKoELVqEo09Sp3diGnlqyJk6NSwUCKEVqG3b/Jac\nxHHQQRqjI5Iq9kADNAIqA6tSzq8C2hVxTVYR5euaWXV337yTMllJ71sDVxCCz+3AEcB9ZrbZ3Z+M\nykwFngWWAm2AO4FXzayHp0PzlohkpORp5L175593D4sCpgadN96ANWtCmSpVwr5VqS06bduG1h6R\niigdAk2cKgGz3X1E9H6BmR0CDAGeBHD3yUnlF5nZQuAzoCfwZlE3HjZsGPXq1StwbsCAAQwYMKDs\nai8i5Y5ZWMk4KwtOPLHgZ2vWhNlWyUFn/HhYsSJ8XqlSaL1JDTrt2oXxOyLpZOLEiUycOLHAufXr\n15f4fukQaNYCuUCTlPNNgJVFXLOyiPIbotaZnZVJvucKYElKmSVAkeNl3H2pma0FDmIngWb06NEa\nQyMiZapx43Ak72MFoYsqtUXniSfg66/D54n1dpJDTqdOYdzOPvvs/ecQgcJ/yU8aQ1NssQcad99q\nZjnAScBLsH1Q8EnAfUVcNgtInYJ9cnQ+uUzqPXqnlJnJjt1a7YAvi6qvmR0ANCSEIRGR2DVoAMcc\nE45kGzbs2KIzeXKYQp7QosWOLTodOkBKA7NI2os90ERGAY9FwSYxbbsW8BiAmd0JNHP3xFoz44Cr\notlOjxCCy9lA36R7jgFmmNk1hGnbAwiDjy9LKjMamGlmNwKTgSOBSxNlzKw2cDNhDM1KQqvMSOAT\nwgBjEZG0VbcuHHlkOJL9+CN89FHBoPPCCzBqVBjDA2FsT/LU8vbtQytPs2aha0sk3aRFoHH3yWbW\nCLiN0C00HzjF3aMhcGQBzZPKf2FmvyQEkquBr4FL3H16UplZZjaQMNj3duBToH9iDZqozBwzOwO4\nCxhBGPg71N0nRUVygc7ABUB9YDkhyNzk7lvL+K9BRGSv2Gef/JWNk23aBB9/vOOsq7FjITc3lKlW\nLaybk9gyolUraN06/88NGoQuLpG9LS3WoSlPtA6NiJQ3mzfDZ5/B0qX5x+ef5/85sZ4OhFah5ICT\nHHpattS+V7JzGb0OjYiIpLfq1fPH16RyD4OSkwNO4pgyBb78Mux5lZCVtWOrTuI44AAtJCglp/91\nRESkxMxg333DkdqFBWGl5OXLd2zVWboUZswInyU6CqpUgQMPLLwrq1WrMMNL3VlSFAUaERHZYypV\nCi0vBxwAxx234+ebN4dWnNTAM28ePPts/qrJALVrF96VlfizpqBXbAo0IiISm+rVwwrHbdsW/vn6\n9YWP25k2Lbz+/HN+2UaNCm/Zad06tPxUrbp3nknioUAjIiJpq149OOywcKRyh5UrCw88774bNgXN\nywtlEy1FiZDTsmX+HlstWoTPtD9WZlOgERGRjGQGTZuG4+ijd/x861b46qsdZ2ctXgyvvgqrVxe8\nV7NmIdwceGDBsJM41KWV3hRoRESkXKpaFdq0CUdhNm0KrThffrnjMWtW2Doisf4OhIHPRYWdFi1C\nl5cGLcdHgUZERCqkmjV3Pn4nNzfMwios8EybFl5/+qng/XYWeJo107T0PUl/tSIiIoWoXBmaNw/H\nscfu+Lk7fPvtjmHnq68gJweeey58nny//fcvPOwceGA4tCt6ySnQiIiIlIBZ6GZq1AiK2iD6xx9D\nwEkNPMnr8CQGLkNYa6ewsJP4s7aWKJoCjYiIyB6yzz5Fr7IMYeDy118XDDuJP0+ZEt5v3lzwfoWF\nnaZNwyrMTZpA/foVM/Qo0IiIiMSkatX8qeSFycsLs7FSw86XX8LMmTBhQlirJ1m1aiHYJALOzl7r\n1Ck/4UeBRkREJE1VqhSCR1YWHHlk4WU2bAjr8axcCatW7fg6f37++61bC15bs+buh5/atff885aG\nAo2IiEgGq1s3HEXN1kpwh++/Lzr4rFwJ778fXlevhm3bCl5fu3bhQSf1XJMm8eyqrkAjIiJSAZiF\nQcUNGkCHDjsvm5cH69blB53Cws9nn4XXNWsKDmyGELB2p9Vnv/3C9hdlQYFGRERECqhUKX8GV6dO\nOy+bmwtr1xbd6rNqFSxZEl7Xrs3fXT2hQYP8gFOa7ScUaERERKTEKlfO72rq3HnnZbdtCy06RQWf\nTz8teT0UaERERGSvqFIlf/+twsydW/SaPrtSqeTVEhEREUkPCjQiIiKS8RRoREREJOMp0IiIiEjG\nU6ARERGRjJc2gcbMrjKzpWa2yczeNbPuuyjf08xyzOxnM/vEzC4spMyvzWxJdM8FZtankDLNzOxJ\nM1trZj9F5bJTytxmZsujz6eZ2UGlf+LMMXHixLirUKbK0/OUp2cBPU86K0/PAnqe8igtAo2ZnQv8\nGbgZOBxYALxuZo2KKN8SeBn4F9AFGAOMN7PeSWWOBiYADwGHAS8CL5hZx6Qy9YGZwGbgFKADhyGf\nmQAACwZJREFU8H/Ad0llrgd+B1wOHAFsjOpWiuV/Mkt5+4dSnp6nPD0L6HnSWXl6FtDzlEdpEWiA\nYcCD7v6Eu38EDAF+AgYXUf4K4HN3v87dP3b3vwDPRPdJuBqY6u6jojI3AXMJ4SThBuArd7/U3XPc\n/Ut3n+7uS5PKDAX+4O4vu/uHwAVAM+D0MnhuERERKQOxBxozqwp0JbS2AODuDkwHehRx2VHR58le\nTynfYzfK9APmmNlkM1tlZnPN7NKkurUCslLqtgF4byd1ExERkb0s9kADNAIqA6tSzq8ihInCZBVR\nvq6ZVd9FmeR7tia09nwMnAz8FbjPzH6TdA8vZt1ERERkL6voWx9UAma7+4jo/QIzO4TQ5fVkCe9Z\nA2DJkiVlUL30sH79eubOnRt3NcpMeXqe8vQsoOdJZ+XpWUDPk66SfnbWKO616RBo1gK5QJOU802A\nlUVcs7KI8hvcffMuyiTfcwWQmjyWAGcm3cOi65JbaZoA84qoW0uA888/v4iPM1PXkm6ukabK0/OU\np2cBPU86K0/PAnqeNNcSeKc4F8QeaNx9q5nlACcBLwGYmUXv7yvisllA6hTsk6PzyWVS79E7pcxM\noF3KfdoBX0Z1W2pmK6P7fBDVrS5wJPCXIur2OjAI+AL4uYgyIiIisqMahDDzenEvtDD+Nl5mdg7w\nGKGrZzZhttLZQHt3X2NmdwLN3P3CqHxLYCHwAPAIIXDcC/R19+lRmR7ADOBG4BVgAGFWU7a7L47K\ndCOEmluAyYSg8iBwmbtPispcB1wPXEQIKX8AOgGd3H3Lnvj7EBERkeJJi0ADYGZXAtcRunPmA//j\n7nOizx4FWrj7iUnljwdGAx2Br4Hb3P3JlHueBdwOtAA+Bf4/d389pUxf4C7gIGAp8Gd3fySlzC2E\ndWjqA28BV7n7f8vmyUVERKS00ibQiIiIiJRUOkzbFhERESkVBRoRERHJeAo0Zay4m2ymKzM7zsxe\nMrNvzCzPzE6Lu04lZWY3mtlsM9sQrQj9vJm1jbteJWVmQ6JNVNdHxztmdmrc9SoLZnZD9P/bqLjr\nUhJmdnNU/+Rjcdz1Ko3d2cA3U0Tfm1P/++SZ2di461ZcZlbJzP5gZp9H/13+a2bD465XaZjZPmZ2\nr5l9ET3T29Hknd2iQFOGirvJZpqrTRicfSVhteRMdhwwljCLrRdQFfinmdWMtVYlt4ww8y6bsG3I\nG8CLZtYh1lqVUhT+Lyf8u8lkHxImN2RFx7HxVqfkdmcD3wzTjfz/LlmEpTycMMs109wA/JbwPbo9\nYVLNdWb2u51eld4eJsxaHgQcAkwDpptZ0925WIOCy5CZvQu85+5Do/dG+OFzn7vfHWvlSsHM8oDT\n3f2luOtSFqKAuRo43t3fjrs+ZcHMvgWudfdH465LSZjZPkAOYSuSEcA8d78m3loVn5ndDPR394xs\nwUhlZncBPdz9hLjrsieYWWK5j4xrsTWzKcBKd78s6dwzwE/ufkF8NSsZM6sB/AD0c/fXks7PAV6N\nNpjeKbXQlJESbrIp8ahP+K1sXdwVKa2o2fk8oBYFF43MNH8Bprj7G3FXpAwcHHXVfmZmfzez5nFX\nqBR2uoFvJou+Zw8itApkoneAk8zsYAAz6wIcA7waa61KrgphX8fNKec3sZutnLGvFFyO7GyTzdTV\niCUmUavZvcDbiQUWM1G059gswqqaPwBnuPtH8daqZKJAdhihOyDTvUtYhPNjoClh0c7/mNkh7r4x\nxnqVVGID3z8T1vQ6grCB7+bUdb8y0BlAPeDxuCtSQncBdYGPzCyX0EDx+8SisJnG3X80s1nACDP7\niPCzcyChQeDT3bmHAo1UNA8QFmM8Ju6KlNJHQBfCN+SzgSfM7PhMCzVmdgAhYPZy961x16e0Uhbu\n/NDMZhO2UjkHyMTuwD2xgW+6GAxMdfei9gxMd+cSfuCfBywm/FIwxsyWZ3DYPJ+w+v83wDZgLjCB\n0PuxSwo0Zackm2zKXmRm9wN9gePcfUXc9SkNd98GfB69nWdmRwBDCb9NZ5KuQGNgbtR6BqGl8/ho\ncGN1z+CBfu6+3sw+IaxEnol2tYFvRjKzAwkTBE6Puy6lcDdwp7v/I3q/KNoW6EYyNGy6+1LgF9GE\njbruvsrMJpH/vW6nNIamjES/XSY22QQKbLJZrB1DpexFYaY/8At3/yru+uwBlYDqcVeiBKYDhxJ+\nu+wSHXOAvwNdMjnMwPbBzgcRgkEm2ukGvhlsMKFLI1PHm0AYN5ebci6PcvBz3d03RWGmAWF23Qu7\nc51aaMrWKOAxC7uHJzbZrEXYeDOjmFltwjfixG/NraNBZ+vcfVl8NSs+M3uAsDnpacBGM0u0oq13\n94zbEd3M7gCmAl8BdQgDG08g7DifUaJxJQXGMpnZRuBbd09tGUh7ZnYPMIXwA39/4FZgKzAxznqV\nwmhgppndSP4GvpcCl+30qjQW/aJ5EfCYu+fFXJ3SmAIMN7OvgUWEZRyGAeNjrVUpmNnJhJ85HwMH\nE1qhFrObP0MVaMqQu0+OpgTfRv4mm6e4+5p4a1Yi3YA3CbOBnDAoEMIAusFxVaqEhhCeYUbK+YuB\nJ/Z6bUpvP8J/h6bAeuAD4ORyMkMIMnvdowMIff4NgTXA28BR7v5trLUqIXefY2ZnEAagjiBs4Ds0\nUweeRnoBzcnMMU3Jfgf8gTBDcD9gOfDX6FymqgfcSfhlYB3wDDDc3VNbogqldWhEREQk42V8X5uI\niIiIAo2IiIhkPAUaERERyXgKNCIiIpLxFGhEREQk4ynQiIiISMZToBEREZGMp0AjIiIiGU+BRkRk\nF8zsBDPLM7O6cddFRAqnQCMisnu0rLpIGlOgERERkYynQCMiac+CG83sczP7yczmmdlZ0WeJ7qC+\nZrbAzDaZ2Swz65Ryj7PM7EMz+9nMlprZNSmfVzOzkWb2VVTmEzO7OKUq3czsfTPbaGYzzezgPfzo\nIrKbFGhEJBP8/8D5wOVAR2A08KSZHZdU5m5gGGGn+DXAS2ZWGcDMugJPE3bCPgS4GfiDmV2QdP2T\nwLmEXYzbA5cCPyZ9bsAfo6/RFdgGPFKmTykiJabdtkUkrZlZNWAdcJK7v5d0/iGgJvAQ8CZwjrs/\nE33WAPgauNDdnzGzvwON3P3UpOtHAn3d/VAzawt8FH2NNwupwwnAG9HnM6JzfYCXgZruvmUPPLqI\nFINaaEQk3R0E1AKmmdkPiQP4DdAmKuPAu4kL3P074GOgQ3SqAzAz5b4zgYPNzIAuhBaX/+yiLguT\n/rwiet2veI8jIntClbgrICKyC/tEr32B5SmfbSYEntLatJvltib9OdG8rV8MRdKA/iGKSLpbTAgu\nLdz985Tjm6iMAUclLoi6nNpG1wIsAY5Jue+xwCce+t0XEr4fnrAHn0NE9iC10IhIWnP3H83sT8Do\naJDv20A9QkBZD3wVFb3JzNYBq4HbCQODX4w++zMw28yGEwYHHw1cBQyJvsaXZvYE8IiZDQUWAC2A\n/dz9H9E9rJDqFXZORGKgQCMiac/dR5jZauAGoDXwPTAXuAOoTOj+uQEYQ+iCmgf0c/dt0fXzzOwc\n4DZgOGH8y3B3fzLpywyJ7vcXoCEhKN2RXI3CqlZWzygipaNZTiKS0ZJmIDVw9w1x10dE4qExNCJS\nHqjrR6SCU6ARkfJATc0iFZy6nERERCTjqYVGREREMp4CjYiIiGQ8BRoRERHJeAo0IiIikvEUaERE\nRCTjKdCIiIhIxlOgERERkYynQCMiIiIZT4FGREREMt7/AzL6Y24/YC0NAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_train(hist)" ] }, { "cell_type": "code", "execution_count": 1214, "metadata": { "collapsed": true, "hidden": true, "scrolled": false }, "outputs": [], "source": [ "preds = np.squeeze(model_pkl.predict(map_valid_pkl, 1024))" ] }, { "cell_type": "code", "execution_count": 1222, "metadata": { "collapsed": true, "hidden": true, "scrolled": true }, "outputs": [], "source": [ "y_orig_pkl_val = log_max_inv(y_pkl_val, max_log_y_pkl)" ] }, { "cell_type": "code", "execution_count": 1224, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "0.11084739924546773" ] }, "execution_count": 1224, "metadata": {}, "output_type": "execute_result" } ], "source": [ "rmspe(log_max_inv(preds, max_log_y_pkl), y_orig_pkl_val)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## XGBoost" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Xgboost is extremely quick and easy to use. Aside from being a powerful predictive model, it gives us information about feature importance." ] }, { "cell_type": "code", "execution_count": 52, "metadata": { "collapsed": true }, "outputs": [], "source": [ "X_train = np.concatenate([cat_map_train, contin_map_train], axis=1)" ] }, { "cell_type": "code", "execution_count": 53, "metadata": { "collapsed": true }, "outputs": [], "source": [ "X_valid = np.concatenate([cat_map_valid, contin_map_valid], axis=1)" ] }, { "cell_type": "code", "execution_count": 54, "metadata": { "collapsed": true }, "outputs": [], "source": [ "all_vars = cat_vars + contin_vars" ] }, { "cell_type": "code", "execution_count": 55, "metadata": { "collapsed": true }, "outputs": [], "source": [ "xgb_parms = {'learning_rate': 0.1, 'subsample': 0.6, \n", " 'colsample_bylevel': 0.6, 'silent': True, 'objective': 'reg:linear'}" ] }, { "cell_type": "code", "execution_count": 56, "metadata": { "collapsed": true }, "outputs": [], "source": [ "xdata = xgboost.DMatrix(X_train, y_train, feature_names=all_vars)" ] }, { "cell_type": "code", "execution_count": 57, "metadata": { "collapsed": true }, "outputs": [], "source": [ "xdata_val = xgboost.DMatrix(X_valid, y_valid, feature_names=all_vars)" ] }, { "cell_type": "code", "execution_count": 58, "metadata": { "collapsed": true }, "outputs": [], "source": [ "xgb_parms['seed'] = random.randint(0,1e9)\n", "model = xgboost.train(xgb_parms, xdata)" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'[0]\\teval-rmse:0.113812'" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.eval(xdata_val)" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'[0]\\teval-rmse:0.113812'" ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.eval(xdata_val)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Easily, competition distance is the most important, while events are not important at all.\n", "\n", "In real applications, putting together a feature importance plot is often a first step. Oftentimes, we can remove hundreds of thousands of features from consideration with importance plots. " ] }, { "cell_type": "code", "execution_count": 61, "metadata": { "scrolled": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAf4AAAJcCAYAAAAYU8ZdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmUXVWZ/vHvQ4AwBIISwIBIiQSZiaRCM08CDqhgS4sI\nSrA10ir8oAWNSsvQ2jLYSjOJAZkREZFBooAMiYhAqJCQIkwqBiUgEIVIGAJUnt8fZ1+5FDUmqSG5\nz2etWnXvPvvs/Z5TWXnP3ufcu2WbiIiIaAzLDXQAERER0X+S+CMiIhpIEn9EREQDSeKPiIhoIEn8\nERERDSSJPyIiooEk8UdERDSQJP6IZYykYZJmSzqormw1SX+WtH9dWbOk6yU9K+k5SQ9I+rakt5Tt\n4yS1SZpffh6V9B99HPtukh7vps6Fkl6pi2u+pAOWQN+WtNHittOL/sZJ+m1/9deVnpz3WHYk8Ucs\nY2zPBz4PnCZprVJ8CtBi+2cAknYAJgN3AJvYXgN4P/AasHVdc3faHmZ7GPAx4BRJ7+mfI+nSKbW4\nys8VAx2QpCEDHcOikLT8QMcQ/SuJP2IZZPtGYBJwuqTdgI8DX6ircgpwge3v2H6q7PNn28fZntxJ\nm9OBB4FNa2WSPiJpVpkxmCypftumpey5Uucjdds+WGYYnpc0R9LRklYFfgWsWzeSX7c3xy1pXUlX\nSXpG0p8kHVG3bVtJd5Z4npR0pqQVy7bflGr31WYQOhqR188KlJmHH0j6paQXgN0lDZX03TK78pSk\ncySt3MPYZ0s6RtJMSS9I+pGkdST9qpynm+tmY5pKLOMlPVGO5+i6toZKOq1se6K8Hlq27SbpcUlf\nlfRX4PKOzntX56vuXBwm6felzlmSVLf9c5IeLLE/IGmb7v5G0U9s5yc/+VkGf4C3AE8Cc4FD68pX\nBdqA3brZfxzw27r3Y4HngI3L+42BF4C9gBWArwB/AFYs7/8AfL283wN4Hnh32fdJYOe6OLcpr3cD\nHu8mrguBb3VQvhwwDfhm6XND4FHgfWX7GGA7YHmgieoi5si6/Q1s1Nnxt69T4pgH7Fj6Xgn4PnAd\n8FZgNeAXwHd6eH5nA3cB6wDrAU8D9wLvKW3fChxX6jaVWC4vf88tgWeAPcv2E0tbawNrAb8D/rvu\nHL8GnAwMBVbu6Lz38HxdD6wBvKP0//6y7d+AOVT/ZgRsBGzQ3d8oP/3zkxF/xDLK9rPALGAV4Od1\nm95C9R/wX2sFkk4po7YXJB1bV3e7Uv48MBW4BPh92XYAMMn2r22/CnyXKonsQJUwhgEn2X7F9q1U\nSeLAsu+rwGaSVrf9rO17e3l4R5e4npM0t5SNBdayfWLp81HgXOAT5XxMs32X7ddszwZ+COzay37b\nu9b2HbYXAguA8cBRtv9u+3ngf2r999AZtp+yPQe4Hbjb9nTbLwNXU10E1DvB9gu2W4ELeP38HgSc\naPtp288AJwCfqttvIdVFxALbL3UUSA/P10m2n7P9Z+A2YHQp/yzV7Zh7XPmD7cfo5m8U/SOJP2IZ\nJelgqpHazVSju5pnqf7jH1krsP0VV/f5r6Ya4dXcZXsN26sBbwM2p0pmAOsCj9W1sRD4C9VodV3g\nL6Ws5rGyDarnBT4IPCZpiqTte3l43y1xrWF7RCnbgGq6unZB8BzVjMM65XxsrOphxr9K+kc5jhEd\nN99jf6l7vRbVRda0uv5vKOU99VTd65c6eD+si/4fozrv0O5v024bwDPlYqJTPTxff617/WJdfOsD\nf+yg2S7/RtE/kvgjlkGS1qaadv4c1YN+H5e0M4DtF4C7gX/tTZuungW4CvhwKXqC6j/yWp+i+g9/\nTtm2vqT6/2PeUbZRRoL7Uk1FXwP8tNZNb2Jq5y/An+ouCNawvZrtD5btPwAeAkbZXp0q4aizxqhu\nY6xSd3xv66BOfbxzqZLz5nX9D3f1YGRfWb/u9Tuozju0+9u02wZvPs8dnffenq96fwHe1Ul5V3+j\n6AdJ/BHLpjOBa2zfZvtJqvvv59Ye8CrvPyNpQrlIQNLbgXd21qCkNYGPUt0+gCpZ7yPpvZJWAL5M\nNd39O6oLixeBr0haQdUDhh8GfiJpRUkHSRpebhH8g2oGAqoR7pqShi/CMU8Fni8Pra0saYikLSSN\nLdtXK33Nl7QJ0P6jiU9R3XOuuQ/YXNJoSSsBx3fVeZndOBf4ft05XU/S+xbhWHrqvyStImlz4FCg\n9umGy4FjJa0laQTVPfVLu2ino/Pe3fnqynlUt2PGqLKRpA3o/m8U/SCJP2IZI2k/YCfgmFqZ7fOo\nRnzfLO9/S/XA3S7AI3XT0pOBM+qa2772pDfVw13PAIeXNh4GDi7151Il9g+Xe7evlPcfKNvOBj5t\n+6HS7qeA2WUK+TCqe9KU7ZcDj5ap4B4/1W+7DfgQ1X3mP5V+zwNqyexo4JNUDxmey+tJsuZ44KLS\n78dtP0L1kNzNVM819OQz91+leqjxrnJsNwPv7ukxLIIppb9bqG5/3FTKvwW0ADOBVqqHBL/VWSOd\nnPfuzlenbF8JfBv4cdn/GuCtPfgbRT+QvTgzaxER0d8kNVElzhVsvzaw0cTSJiP+iIiIBpLEHxER\n0UAy1R8REdFAMuKPiIhoIFmcIQadESNGuKmpaaDDiIhYqkybNm2u7W6/MCqJPwadpqYmWlpaBjqM\niIiliqTHuq+Vqf6IiIiGksQfERHRQJL4IyIiGkju8ceg0zpnHk0TJg10GBER/Wr2Sfv0Sz/L9Ihf\n0tsk/UTSHyVNk/RLSRv3cwxNkj5Z975Z0unl9W6SdqjbdpikTy9iP7tJmidpuqSHJf1G0od62nb7\nWCIiYtm0zI74yxKhVwMX2f5EKduaat3nR/oxlCaqhS5+DGC7hWrxDIDdgPlUq5lh+5zF7Ot22x8C\nkDQauEbSS7Zv6UHbb4glIiKWTcvyiH934NX6hGf7PuC3kk6VdL+kVkkHwD9HvFMkXSvpUUknlaVD\np5Z67yr1LpR0jqQWSY/URtVleclTJd0jaaakz5duTwJ2ljRD0lGln+vLIhuHAUeVbTtLOl7S0aW9\n0ZLuKm1dLektpXyypJNLXI+orLHenu0ZVCuLfansV9/2EZIeKG3/pJNYPizp7jKDcLOkderaOb/E\n8aikI2p9Svp0afM+SZeUsrUkXVXOyz2SdlzcP2xERCy6ZXbED2wBTOug/F+ploTcGhgB3CPpN2Xb\n1sCmwN+BR4HzbG8r6f9RLUV6ZKnXBGwLvAu4TdJGwKeBebbHqlrz/A5JNwETgKPrRuK7AdieLekc\nYL7t75Zt762L82LgcNtTJJ0IHFfX//Ilrg+W8j07OQf3Urc0a50JwDttL5C0hu3nOojlLcB2ti3p\ns1Trt3+57L8J1YXVasDDkn4AbAwcC+xge66kt5a6/wd83/ZvJb0DuLGc4zeQNB4YDzBk9W6/fyIi\nIhbRspz4O7MTcHlZF/opSVOAscA/gHtsPwkg6Y9AbW3rVqpEV/NT2wuB30t6lCoR7g1sJWn/Umc4\nMAp4pbcBShoOrGF7Sim6CLiyrsrPy+9pVBchnTbVSflM4DJJ11Ctk92RtwNXSBoJrEi1BGjNJNsL\ngAWSnqa6fbIHcKXtuQC2/17q7glsVt15AWB1ScNsz6/vzPZEYCLA0JGjsoBEREQfWZan+mcBY3q5\nz4K61wvr3i/kjRdJ7ROTqZLs4bZHl5932r6JvlGLq42uL97eAzzYQfk+wFnANlQzHh21cQZwpu0t\ngc8DK3XQf09iWI5q5qB2XtZrn/QjIqL/LMuJ/1ZgaJlCBkDSVsBzwAHlnvxawC7A1F62/W+Sliv3\n/TcEHqaawv4PSSuUvjaWtCrwPNWUeEc63GZ7HvBs3f37TwFT2tfrSjnW/6JK8PXlywHr274N+CrV\nzMSwDmIZDswprw/pQZe3Up2XNUs/tan+m6huk9T6H92b44iIiCVrmZ3qL/emPwqcJumrwMvAbKr7\n5MOA+6hG6l+x/VdJm/Si+T9TXSysDhxm+2VJ51FNu99bPlHwDLAf1bR6m6T7gAuB6XXt/AL4maR9\nqUuOxSHAOZJWoXre4NAexLWzpOnAKsDTwBG2b2lXZwhwabmdIOD0co+/fSzHA1dKepYqqb+zq45t\nz5L0bWCKpLZynOOAI4CzJM2k+vf2G6oHCSMiYgDIzu3U3pB0IXC97Z8NdCzLqubmZmeRnoiI3pE0\nzXZzd/WW5an+iIiIaGeZnervK7bHDXQMERERiyoj/oiIiAaSxB8REdFAkvgjIiIaSBJ/REREA0ni\nj4iIaCBJ/BEREQ0kH+eLQad1zjyaJkwa6DAilrjZJ+0z0CFEZMQfbybpG5JmSZopaYakf5F0ZPn6\n4IiIWIplxB9vIGl74EPANrYXSBpBtSzvFcClwIu9aGtIWf44IiIGiYz4o72RwFzbCwBszwX2B9YF\nbpN0G4CkAyW1Srpf0sm1nSXNl/S/ZVGi7SWNkTRF0jRJN0oaOQDHFBERRRJ/tHcTsL6kRySdLWlX\n26cDTwC7295d0rrAycAewGhgrKT9yv6rAnfb3hq4GzgD2N/2GOB84NsddSppvKQWSS1tL87r2yOM\niGhgmeqPN7A9X9IYYGdgd+AKSRPaVRsLTLb9DICky4BdgGuANuCqUu/dwBbAr6uVihkCPNlJvxOB\niQBDR47KkpEREX0kiT/epNyXnwxMltQKHNKL3V+uu68vYJbt7ZdwiBERsYgy1R9vIOndkkbVFY0G\nHgOeB1YrZVOBXSWNkDQEOBCY0kFzDwNrlQcGkbSCpM37LvqIiOhORvzR3jDgDElrAK8BfwDGUyX3\nGyQ9Ue7zTwBuoxrVT7J9bfuGbL8iaX/gdEnDqf69nQbM6qdjiYiIdmTndmoMLs3NzW5paRnoMCIi\nliqSptlu7q5epvojIiIaSBJ/REREA0nij4iIaCBJ/BEREQ0kiT8iIqKBJPFHREQ0kCT+iIiIBpLE\nHxER0UDyzX0x6LTOmUfThEkDHUYsY2aftM9AhxAxKGTEP4hJ2k+SJW3Sx31s1lft93c/ERHRtST+\nwe1A4Lfl9xInaXlgP6A/EnJ/9RMREV1I4h+kJA0DdgL+HfhEKRsp6TeSZki6X9LOkoZIurC8b5V0\nVKk7WtJdkmZKulrSW0r5ZEmnSWoBvgp8BDi1tPmusv37klokPShprKSfS/q9pG/VxXewpKllvx+W\nVfqQNF/StyXdV/pfR9IO7fvpz3MZERGvS+IfvPYFbrD9CPA3SWOATwI32h4NbA3MoFo2dz3bW9je\nErig7H8x8FXbWwGtwHF1ba9ou9n2t4HrgGNsj7b9x7L9lbLQwznAtcAXgS2AcZLWlLQpcACwY4ml\nDTio7LsqcJftrYHfAJ+z/btO+omIiH6Wh/sGrwOB/yuvf1LeXwecL2kF4BrbMyQ9Cmwo6QxgEnBT\nWQJ3DdtTyv4XAVfWtX1FN31fV363ArNsPwlQ+lqfaiZiDHCPJICVgafLPq8A15fX04C9enKwksZT\nLf/LkNXX6skuERGxCJL4ByFJbwX2ALaUZGAIYOAYYBdgH+BCSd+zfbGkrYH3AYcBHweO6qaLF7rZ\nvqD8Xlj3uvZ+eUDARba/1sG+r/r1tZ7b6OG/MdsTgYkAQ0eOylrRERF9JFP9g9P+wCW2N7DdZHt9\n4E9USf8p2+cC5wHbSBoBLGf7KuBYYBvb84BnJe1c2vsUMOXN3QDwPLBaL+O7Bdhf0tpQXahI2qCb\nfRaln4iIWMIy4h+cDgRObld2FXAh8IKkV4H5wKeB9YALJNUu4mqj8EOAcyStAjwKHNpJXz8BzpV0\nBNUFR7dsPyDpWKrbCssBr1I9B/BYF7u9oZ/c54+IGBh6fVY2YnBobm52S0vLQIcREbFUkTStPJjd\npUz1R0RENJAk/oiIiAaSxB8REdFAkvgjIiIaSBJ/REREA0nij4iIaCBJ/BEREQ0kiT8iIqKB5Jv7\nYtBpnTOPpgmTBjqMGECzT9pnoEOIWGZlxB8REdFAkvjjTSR9Q9IsSTMlzZD0L5KOLN/7392+PaoX\nEREDI4k/3kDS9sCHqFb52wrYE/gLcCTQk4Te03oRETEAkvijvZHAXNsLAGzPpVq1b13gNkm3AUj6\ngaSWMjNwQik7ooN6e0u6U9K9kq6UNGwgDioiIipJ/NHeTcD6kh6RdLakXW2fDjwB7G5791LvG2UV\nqK2AXSVt1b6epBHAscCetrcBWoD/7KhTSePLhURL24vz+voYIyIaVp7qjzewPV/SGGBnYHfgCkkT\nOqj6cUnjqf4NjQQ2A2a2q7NdKb9DEsCKwJ2d9DsRmAgwdOSorBUdEdFHkvjjTWy3AZOByZJagUPq\nt0t6J3A0MNb2s5IuBFbqoCkBv7Z9YN9GHBERPZWp/ngDSe+WNKquaDTwGPA8sFopWx14AZgnaR3g\nA3X16+vdBewoaaPS9qqSNu7L+CMiomsZ8Ud7w4AzJK0BvAb8ARgPHAjcIOmJcv9+OvAQ1RP/d9Tt\nP7FdvXHA5ZKGlu3HAo90FcCW6w2nJV/gEhHRJ2TndmoMLs3NzW5paRnoMCIiliqSppWHrruUqf6I\niIgGksQfERHRQJL4IyIiGkgSf0RERANJ4o+IiGggSfwRERENJIk/IiKigeQLfGLQaZ0zj6YJkwY6\njFhMs/MlTBGDUkb8ERERDSSJvwuS2iTNkHR/WUt+lX7sey9J0yS1lt971G37TCmfWWLbt5SfKGnP\nJRjDtyWdXPd+A0mPlq/zjYiIpVCm+rv2ku3RAJIuAw4DvlfbqGqtWdle2Ad9zwU+bPsJSVsANwLr\nSXo78A1gG9vzJA0D1gKw/c0lHMO3gBmSLrT9IPB/wH/Zfm5xGpW0vO3XlkiEERHRKxnx99ztwEaS\nmiQ9LOli4H5gfUkHlhH4/e1GyPMlnSpplqSbJW0raXIZNX+k1FlJ0gVl/+mSdgewPd32E6WpWcDK\nZaGbtalWwJtf6s23/afS1oWS9i+vZ0s6QdK9pe1NSvmwuv5mSvpYKd9b0p2l/pWShtl+CTgKOEvS\nB4HVbF9W6o+VNKXMRvyqrNKHpMMk3SPpvtLOyqX8Ukk/kDQV+J+++iNFRETXkvh7QNLyVEvPtpai\nUcDZtjcHXgVOBvagWsJ2rKT9Sr1VgVtLveepRtB7AR8FTix1vgjY9pZUK+BdJKn92vYfA+61vQC4\nD3gK+FNJ4B/uIvS5trcBfgAcXcr+C5hne0vbWwG3ShpBtWrenqV+C/CfVIH9EngWuAj4QjkfQ6lG\n/x+zPQa4FPjv0v6Vtsfa3hr4IzCuLp6RwHa2v9I+UEnjJbVIaml7cV4XhxQREYsjU/1dW1nSjPL6\nduBHwLrAY7bvKuVjgcm2n4F/3hLYBbgGeAW4odRrBRbYflVSK9BUyncCzgCw/ZCkx4CNgZmlvc2p\nLiz2LnXaJL2/9Pte4PuSxtg+voP4f15+TwP+tbzeE/hErYLtZyV9CNgMuKO6e8GKwJ117ZwFrGz7\n4fJ+U2Bz4OZSfwjweNm2laQTgTWA1YDr69q5srPbIrYnUi3py9CRo7JkZEREH0ni79o/7/HXlET3\nQg/3f9Wvr3u8EFgAYHthmUXoUrmffzXwadt/rJWXNqcCUyX9GrgAOL6DJhaU3210/bcW8GvbB3ay\nfWH5qa8/0/bOHdS9GPiA7fslfRbYrm5bT89bRET0kUz1L76pwK6SRkgaQjVdP6UX+98OHAQgaWPg\nHcDD5cn5ScAE23fUKktaV9I2dfuPBh7rRX+/prq9UGvvLcBdwI6SNiplq5ZYOvMA1YOG25b6K5aZ\nCahub/xV0grAJ3sRV0RE9IOM+BeT7SclTQBuoxoJT7J9bS+aOBv4QZn+fw0YZ3uBpGOAjYBvSqo9\nrb83sALwXUnrAi8Dz1B92qCnvkX1sN79VDMBJ9j+uaRxwOXl/j1U9/wf6aiBEt/+wOmSVqea6v9f\nqocQvwncU+KaCrR/XqFbW643nJZ8+UtERJ/Q6zPREYNDc3OzW1paBjqMiIiliqRptpu7q5ep/oiI\niAaSxB8REdFAkvgjIiIaSBJ/REREA0nij4iIaCBJ/BEREQ0kiT8iIqKB5At8YtBpnTOPpgmTBjqM\nZd7sfElSREPKiD8iIqKBJPEPMpLaJM2QNKusaf9lSYv8d5K0k6Spkh4qP+Prtq0l6W5J0yUdJ+m0\num0/lHRz3fvDJZ2+iDHMLkv/RkTEAMtU/+DzzxUBJa0N/BhYHTiutw1JelvZfz/b95bke6OkObYn\nUS3r22r7s5KaqdYNqNkaGCJpiO02YAegN2sQRETEIJQR/yBm+2lgPPAlVZok3S7p3vKzA4CkiyXt\nV9tP0mWS9qVahe9C2/eW9uYCXwEmSBoNnALsK2kG8DCwsaSVJQ0HXgJmAFuWZncA7ijtH1xmEWaU\nmYEhpXxvSXeW2K6UNKz+eErbv5L0uT46ZRER0Y0k/kHO9qNUq9+tDTwN7GV7G+AAoDb1/iNgHEBJ\n2jtQLem7OTCtXZMtwOa2Z1CtpHeF7dG2nwemA2OB7YC7qZbr3UHSelQLOv1F0qal7x3LzEQbcFCZ\nTTgW2LPE1wL8Z12/w4BfAJfbPrf9cUoaL6lFUkvbi/MW8WxFRER3MtW/dFkBOLOM1tuAjQFsT5F0\ntqS1gI8BV9l+TVJv2/8d1UXDysCdwO+Br1Mtsfu7Uue9wBjgntL+ylQXJNsBmwF3lPIVSxs11wKn\n2L6so45tTwQmAgwdOSpLRkZE9JEk/kFO0oZUSf5pqvv8T1Hdf18OeLmu6sXAwcAngENL2QNUSbr+\n3vwYYFYn3d0BHAasBJxFlfA3442JX8BFtr/WLs4PA7+2fWAXbb9f0o+dtaAjIgZMpvoHsTKCPwc4\nsyTL4cCTthcCn6K6BVBzIXAkgO0HStlZwLgyQ4CkNYGTqe7td+ROqpH7WrafLn0+A+xLub8P3ALs\nXx48RNJbJW1AdVtgR0kblfJVJW1c1/Y3gWdLTBERMUAy4h98Vi4P260AvAZcAnyvbDsbuErSp4Eb\ngBdqO9l+StKDwDV1ZU9KOhg4V9JqVKP102z/oqOObT8r6RneOCNwJ7AjcF+p84CkY4GbyscMXwW+\naPsuSeOAyyUNLfseCzxS19b/A86XdIrtr3R2ArZcbzgt+XKZiIg+ocy6LhskrQK0AtvYXqqfjmtu\nbnZLS8tAhxERsVSRNM12c3f1MtW/DJC0J/AgcMbSnvQjIqJvZap/GWD7ZmCDgY4jIiIGv4z4IyIi\nGkgSf0RERANJ4o+IiGggSfwRERENJIk/IiKigeSp/hh0WufMo2nCpIEOY6kzO196FBE9kBF/RERE\nA8mIP3pMUhvVtwPWvk74YuD7Ze2AiIhYCiTxR2+8ZLu24M/awI+B1alWDYyIiKVApvpjkdh+GhgP\nfEmVJkm3S7q3/OwAIOliSfvV9pN0maR9ByruiIhGl8Qfi8z2o1RLA68NPA3sZXsb4ADg9FLtR8A4\nAEnDgR2ANz25J2m8pBZJLW0vZrmBiIi+ksQfS8oKVMv/tgJXApsB2J4CjJK0FnAgcJXt19rvbHui\n7WbbzUNWGd6fcUdENJTc449FJmlDoI1qtH8c8BSwNdUF5ct1VS8GDgY+ARzaz2FGRESdJP5YJGUE\nfw5wpm2XafzHbS+UdAjVLYCaC4GpwF9tP9D/0UZERE0Sf/TGypJm8PrH+S4Bvle2nQ1cJenTwA3A\nC7WdbD8l6UHgmn6ONyIi2pHtgY4hlnGSVqH6/P82trt9cq+5udktLS19H1hExDJE0jTbzd3Vy8N9\n0ack7Qk8CJzRk6QfERF9K1P90ads3wxsMNBxREREJSP+iIiIBpLEHxER0UCS+CMiIhpIEn9EREQD\nSeKPiIhoIEn8ERERDSQf54tBp3XOPJomvGkBv+jA7JP2GegQImIp06cjfklvk/QTSX+UNE3SLyVt\n3Jd9dhBDk6RP1r1vlnR6eb1bbd348v6w8pWzi9rXTpKmSnqo/IxfvOg77edDkqZLuk/SA5I+X8oX\nK/4O+vmcpCvq3q9e/pYbLqk+IiKif/XZiF+SgKuBi2x/opRtDawDPNJX/XagCfgk8GMA2y1A7ftg\ndwPmA78r285Z1E4kva30sZ/teyWNAG6UNMf2Ehu+SloBmAhsa/txSUOpjnGx4u/EecChkvYsX8Rz\nInC+7UcXp1FJy3e0NG9ERPS9vhzx7w68Wp+MbN8H/FbSqZLul9Qq6QD45+h7iqRrJT0q6SRJB5UR\ndKukd5V6F0o6R1KLpEckfaiUDynt3iNpZm0UDJwE7CxphqSjSj/XS2oCDgOOKtt2lnS8pKNLe6Ml\n3VXaulrSW0r5ZEknl7gekbRz6eeLwIW27y3HOhf4CjBhUeIucU6W9LMye3BZuZhajeqC7W+lnwW2\nHy771MffYZylv++W8z9T0uGlfEw5/9Mk3ShppKuFHA4DTpPUDLwXOLXUH1XqTZP0m9pMjqR9Jd1d\nZiRukrR2Kf+WpIsl3UG1Wl9ERAyAvkz8WwDTOij/V2A01brtewKnShpZtm1NlWg2BT4FbGx7W6qR\n5+F1bTQB2wL7AOdIWgn4d2Ce7bHAWOBzkt5JlXhvtz3a9vdrDdieTbWs7PfLttvbxXkx8FXbW1Et\nMHNc3bblS1xH1pVv3sHxtpTyRYkb4D2lj82ADYEdbf8duA54TNLl5eKos79jR3GOL3GMLsd2WZlF\nOAPY3/YY4Hzg2+U8zQRuBG4BDrf9SmlnIvCFUv9rwJml/DfAdrbfA/wc+HJdPJsA77V9cPtAJY0v\nF0UtbS/mK/0jIvrKQDzctxNwue024ClJU6gS3j+Ae2w/CSDpj8BNZZ9WqhmEmp/aXgj8XtKjVAll\nb2ArSfuXOsOBUcAr9JKqteXXsD2lFF0EXFlX5efl9zTKNHsP9TbuqbYfLzHNKH391vZnJW1JdeF0\nNLAXMK6D/jqKc0/gnNpUu+2/S9qC6kLt19WkAkOAJ+vaOQv4gO3JJZY1gO2oluGt1an9W3oH8NNy\n62Mob7ytc63tlzs6MbYnUl1MMHTkqCwZGRHRR/oy8c8C9u+21hstqHu9sO79Qt4Ya/vEYEBUI9Ib\n6zdI2q3Nda0zAAAgAElEQVSXMfRELa62urgeAMYA19bVG0N1HurjpN37ruKuPx/1fWG7FWiVdAnw\nJzpO/B3F2REBs2xv38n2heWnvv5c26M7qHsW8D+2f6lqZb4Jddte6CKGiIjoB3051X8rMFR1T7ZL\n2gp4Djig3GteC9gFmNrLtv9N0nLlvv+GwMNU09H/UaatkbSxpFWB56nui3ekw21l+dhn6+7ffwqY\n0r5eO2cB4ySNLv2vCZwMnLKIcXdI0rB2FzOjgce6ia3er4HPS1q+tPfWEsdakrYvZStI2ryzBmw/\nCzwp6aOl/nKqHtyEasZiTnke4ZBexBUREf2gz0b8tl0Sw2mSvgq8DMymut88DLiPasT7Fdt/lbRJ\nL5r/M9XFwurAYbZflnQe1XT2vSXpPAPsB8wE2iTdR/VQ2fS6dn4B/EzSvrzxGQKoktY5klYBHgUO\n7eZ4n5R0MHCupNWoRsWn2f7FIsbdGQFfkfRD4CWqUfS4rmJr5zxgY2CmpFeBc22fWW41nF5ucywP\nnMYbZyva+wTwA0nHAysCl1L9TY+n+jTH34HJwMiOd4+IiIGg6sHtpYekC4Hrbf9soGPpjaU17oHQ\n3NzslpaW7itGRMQ/SZpmu7m7evnK3oiIiAay1H1lr+1xAx3Dolha446IiGVLRvwRERENJIk/IiKi\ngSTxR0RENJAk/oiIiAaSxB8REdFAkvgjIiIayFL3cb5Y9rXOmUfThEkDHcagMvukfQY6hIhYRmTE\nHwBI+r6kI+ve31i+Trj2/n8l/ecitDt/ScUYERGLL4k/au4AdoBq0R1gBFC/UM8OwO8GIK6IiFiC\nkvij5ndAbVnezYH7geclvUXSUGBTqoWEjpF0j6SZkk6o7SzpYElTJc2Q9ENJQ+oblzRC0p2SMmcd\nETGAkvgDANtPAK9JegfV6P5O4G6qi4FmoBXYDRgFbEu1HPAYSbtI2hQ4ANjR9migDTio1rakdYBJ\nwDdtd3jzXtJ4SS2SWtpenNdHRxkREXm4L+r9jirp7wB8D1ivvJ5HdStg7/JTW9p4GNWFwFbAGOCe\namVhVgaeLnVWAG4Bvmh7Smcd254ITAQYOnLU0rVkZETEUiSJP+rV7vNvSTXV/xfgy8A/gAuAXYHv\n2P5h/U6SDgcusv21Dtp8DZgGvA/oNPFHRET/yFR/1Psd8CHg77bbbP8dWINquv93wI3AZyQNA5C0\nnqS1qUb0+5fXSHqrpA1KmwY+A2wi6av9ezgREdFeRvxRr5Xqaf4ftysbZnsucFO5n39nmdKfDxxs\n+wFJx5btywGvAl8EHgOw3SbpQOA6Sc/bPrv/DikiIurJzu3UGFyam5vd0tIy0GFERCxVJE2z3dxd\nvUz1R0RENJAk/oiIiAaSxB8REdFAkvgjIiIaSBJ/REREA0nij4iIaCBJ/BEREQ0kiT8iIqKB5Jv7\nYtBpnTOPpgkdLuK3zJh9UlYnjoiBkRH/UkjSNyTNkjRT0gxJ/yLpSEmrLKH2zyrtPiDppfJ6hqT9\nl0T7ERExcDLiX8pI2p5qIZ1tbC+QNAJYEbgCuBR4sRdtDbHd1r7c9hfL9ibgetujl0DoERExCGTE\nv/QZCcy1vQCgLJ6zP7AucJuk2wAkHSipVdL9kk6u7SxpvqT/lXQfsL2kMZKmSJom6UZJIzvrWNK7\nJd1T935TSVPL68clnVz6vFvShqV8HUk/l9Qiaaqk7frgnERERA8l8S99bgLWl/SIpLMl7Wr7dOAJ\nYHfbu0taFzgZ2AMYDYyVtF/Zf1XgbttbA3cDZwD72x4DnA98u7OObT8MvCRpi1J0KHBBXZW/294S\n+CHwvVJ2OnBKWTji48B5HbUtaXy5OGhpe3Fe785IRET0WKb6lzK250saA+wM7A5cIWlCu2pjgcm2\nnwGQdBmwC3AN0AZcVeq9G9gC+HVZZncI8GQ3IfwIOFTSV4F/A95Tt+3y8vsy4KTyek/g3aV9gLdI\nWtn2S+2OayIwEWDoyFFZMjIioo8k8S+Fyn35ycBkSa3AIb3Y/eW6+/oCZtnevhf7Xwl8HbgDuNP2\nc/WhdVBfwLa2X+lFHxER0Ucy1b+UKffZR9UVjQYeA54HVitlU4FdJY2QNAQ4EJjSQXMPA2uVBwaR\ntIKkzbvq3/aLwK3Ambxxmh/ggPL7QKoLA4CbgS/WxZ8HBSMiBlBG/EufYcAZktYAXgP+AIynSrY3\nSHqi3OefANxGNeKeZPva9g3ZfqV8RO90ScOp/j2cBszqJobLgA8Ct7QrHyFpJvBSiQeqpP8DSYeW\n9m+j7kIgIiL6l+zcTo3eKRcVQ22fUFf2OLBFu6n/RdLc3OyWlpbFbSYioqFImlYepO5SRvzRK5J+\nAaxP9YmBiIhYyiTxR6/Y/nAn5W/v71giIqL38nBfREREA0nij4iIaCBJ/BEREQ0kiT8iIqKBJPFH\nREQ0kCT+iIiIBpKP88Wg0zpnHk0TJg10GIts9kn7DHQIERGdatgRv6Q2STMk3SfpXkk79GCfIyQ9\nWFa7W9z+l5N0uqT7yxr290h6Z9n29R620dN6syWNqHu/m6Tru9lnnKQzy+vDJH26gzpNku7vSQwR\nETE4NPKI/yXbowEkvQ/4DrBrN/t8AdjT9uM96UDS8rZf62TzAcC6wFa2F0p6O/BC2fZ14H960EVP\n6y0W2+f0dR8REdE/GnbE387qwLO1N5KOKSPwmZJOKGXnABsCv5J0lKS3Srqm1LlL0lal3vGSLpF0\nB3CJpCGSTq1r7/Olm5HAk7YXAth+3Pazkk4CVi6zEZeVNq+RNE3SLEnjS1lH9Q6WNLWU/bCszNel\nzo6jXZ3jJR1dXo8psyT38cZV95ok3V5mT/45gyLpYkn71dW7TNK+Pfy7RETEEtbII/6VJc0AVqJK\nwnsASNobGAVsS7Wy3XWSdrF9mKT3A7vbnivpDGC67f0k7QFcTLVELsBmwE62XyqJep7tsZKGAndI\nugn4KfBbSTtTrXJ3qe3ptidI+lJtNqL4jO2/S1oZuEfSVe3rSdqUahZhR9uvSjobOKjEBXCbpLby\nehjwUHl9QhfH0ZELgC/Z/o2kU+vKnwb2sv1yWTb4cqAZ+BFwFHBNWQFwB+CQ9o2W8zQeYMjqa3XR\nfURELI5GHvG/ZHu07U2A9wMXSxKwd/mZDtwLbEJ1IdDeTsAlALZvBdaUtHrZdp3tl8rrvYFPl4uM\nu4E1gVHldsG7ga8BC4FbJL23k1iPKCPsu6gWyOkonvcCY6guDGaU9xvWbd+9HO9o4LM9PI43KEsB\nr2H7N6XokrrNKwDnSmoFrqS6+MH2FGCUpLWoluq9qqPbH7Yn2m623TxkleGdnIaIiFhcjTzi/yfb\nd5aH39aiGuV/x/YPF6PJF+peCzjc9o0d9LsA+BXV7YOngP1ot8a9pN2APYHtbb8oaTLVLEV7Ai6y\n/bXFiHtxHAU8BWxNdUH5ct22i4GDgU8Ah/Z/aBERUdPII/5/krQJMAT4G3Aj8BlJw8q29SSt3cFu\nt1NNpdeS81zb/+ig3o3Af0haodTdWNKqkraRtG4pWw7YCnis7PNqrT4wHHi2JP1NgO3q2q6vdwuw\nfy3Wcu9+gx4cfk+PA9vPAc9J2qkUHVS3eTivP7PwKarzWXMhcGRp44EexBQREX2kkUf8tXv8UI2W\nD7HdBtxU7pffWc38M59qtPp0u/2PB86XNBN4kQ7uWxfnAU3AveVWwjNUI/u1qabGh5Z6U4Ezy+uJ\nwExJ9wKfAQ6T9CDwMNV0P+3r2T5I0rEl/uWAV6kevnuMrvX0OGoOLfUN3FRXfjZwlaqP/d1A3ayH\n7adK/Nd003ZERPQx2R7oGGIZJ2kVoBXYxva87uo3Nze7paWl7wOLiFiGSJpmu7m7epnqjz4laU/g\nQeCMniT9iIjoW4081R/9wPbNQE+eNYiIiH6QEX9EREQDSeKPiIhoIEn8ERERDSSJPyIiooEk8UdE\nRDSQJP6IiIgGko/zxaDTOmceTRMmDXQY/zT7pH0GOoSIiCUmI/6IiIgGksTf4CRZ0qV175eX9Iyk\n6xexvTUkfaHu/W6L2lZERCx5SfzxArCFpJXL+72AOYvR3hrAF7qtFRERAyKJPwB+CdRuZB8IXF7b\nUJb3vUbSTEl3SdqqlB8v6XxJkyU9KumIsstJwLskzZB0aikbJulnkh6SdFlZpTAiIgZAEn8A/AT4\nhKSVgK2Au+u2nQBMt70V8HXg4rptmwDvA7YFjpO0AjAB+KPt0baPKfXeAxwJbAZsCOzYPgBJ4yW1\nSGppezFr+URE9JUk/sD2TKCJarT/y3abdwIuKfVuBdaUtHrZNsn2AttzgaeBdTrpYqrtx20vBGaU\nvtrHMNF2s+3mIasMX9xDioiITuTjfFFzHfBdYDdgzR7us6DudRud/3vqab2IiOhjGfFHzfnACbZb\n25XfDhwE1RP6wFzb/+iineeB1fokwoiIWGwZeQUAth8HTu9g0/HA+ZJmAi8Ch3TTzt8k3SHpfuBX\nQK+/iWfL9YbTki/NiYjoE7I90DFEvEFzc7NbWloGOoyIiKWKpGm2m7url6n+iIiIBpLEHxER0UCS\n+CMiIhpIEn9EREQDSeKPiIhoIEn8ERERDSSJPyIiooHkC3xi0GmdM4+mCb3+3p8lana+QCgillEZ\n8UdERDSQJP5+IKmtrE9/v6QrJa3Sj33vJun6buqsIekL/RTPbEkj+qOviIh4syT+/vFSWZ9+C+AV\n4LD6jaoM5N9iDaBXiX8QxBwREYsg/3H3v9uBjSQ1SXpY0sXA/cD6kg6U1FpmBk6u7SBpvqRTJc2S\ndLOkbSVNlvSopI+UOitJuqDsP13S7u07lnS8pPPr9j2ibDoJeFeZlTi11D1G0j2SZko6oZS1j/m/\navXL9nGSziyvr5E0rcQ8vk/OZERE9FoSfz+StDzwAaC29O0o4GzbmwOvAicDewCjgbGS9iv1VgVu\nLfWeB74F7AV8FDix1PkiYNtbAgcCF0laqYMwNgHeB2wLHCdpBWAC8McyK3GMpL1LbNuWWMZI2qWD\nmM8uMdQcAPykvP6M7TFAM3CEpDW7OTfjJbVIaml7cV5XVSMiYjEk8fePlSXNAFqAPwM/KuWP2b6r\nvB4LTLb9jO3XgMuAWrJ9BbihvG4Fpth+tbxuKuU7AZcC2H4IeAzYuINYJtleYHsu8DSwTgd19i4/\n04F7qS4WRrWP2fYzwKOStiuJfRPgjlLvCEn3AXcB69ft3yHbE203224essrwrqpGRMRiyMf5+sdL\ntkfXF0gCeKGH+7/q19dPXggsALC9sMwi9MaCutdtdPxvQMB3bP/wDYVSE2+O+SfAx4GHgKttW9Ju\nwJ7A9rZflDQZ6Gj2ISIi+llG/IPHVGBXSSMkDaGarp/Si/1vBw4CkLQx8A7g4R7u+zywWt37G4HP\nSBpW2ltP0tqd7Hs1sG+JtzbNPxx4tiT9TYDtenEcERHRhzLiHyRsPylpAnAb1Yh7ku1re9HE2cAP\nJLUCrwHjbC8oMwvd9f03SXdIuh/4VbnPvylwZ9l/PnAw1QxB+32flfQgsJntqaX4BuCwUv4w1XR/\nj2253nBa8gU6ERF9Qq/PIEcMDs3NzW5paRnoMCIiliqSptlu7q5epvojIiIaSBJ/REREA0nij4iI\naCBJ/BEREQ0kiT8iIqKBJPFHREQ0kCT+iIiIBpIv8IlBp3XOPJomTBrQGGbnC4QiYhmVEX9EREQD\nyYi/gUlqo1rhb3ngQeAQ2y8ObFQREdGXMuJvbC/ZHm17C6qlfw+r36hK/o1ERCxD8p961NwObCSp\nSdLDki4G7gfWl3SgpFZJ90s6ubaDpPmSTpU0S9LNkraVNFnSo5I+UuqsJOmCsv90SbsP0PFFRARJ\n/AFIWh74ANW0P8Ao4GzbmwOvAicDewCjgbGS9iv1VgVuLfWeB74F7AV8FDix1PkiYNtbUi3de5Gk\nlTqIYbykFkktbS/O64vDjIgIkvgb3cqSZgAtwJ+BH5Xyx2zXltIdC0y2/Yzt14DLgF3KtleoluCF\n6qJhiu1Xy+umUr4TcCmA7YeAx4CN2wdie6LtZtvNQ1YZvgQPMSIi6uXhvsb2ku3R9QWSAF7o4f6v\n+vV1nRcCCwBsLyyzCBERMchkxB/dmQrsKmmEpCFU0/VTerH/7cBBAJI2Bt4BPLzEo4yIiB7JqCy6\nZPtJSROA2wABk2xf24smzgZ+IKkVeA0YZ3tBVztsud5wWvIFOhERfUKvz9RGDA7Nzc1uaWkZ6DAi\nIpYqkqbZbu6uXqb6IyIiGkgSf0RERANJ4o+IiGggSfwRERENJIk/IiKigSTxR0RENJBuE7+kdST9\nSNKvyvvNJP1734cWERERS1pPvsDnQuAC4Bvl/SPAFbz+ve4RS1TrnHk0TZjU7/3OzpcGRUQD6MlU\n/wjbP6X6LnbKQi1tfRpVRERE9ImeJP4XJK0JGEDSdkDDrJsqaT9JlrRJXVltDfpTy/bNFqHddSRd\nL+k+SQ9I+mUpb5L0yR7s35t697crO17S0d3sd6Gk/cvr8zo6RknjJJ3ZXQwRETF49CTx/ydwHfAu\nSXcAFwOH92lUg8uBwG/L75rxwFa2jwH2A3qV+MvKdScCv7a9te3NgAllcxPQbULvRb3FZvuzth/o\nj74iIqJvdZn4JS0HrATsCuwAfB7Y3PbMfohtwEkaRrWe/L8Dnyhl1wHDgGmSjgM+ApwqaYakd5Wf\nGyRNk3R7baagjKDPkXQ3cAowEni81lfdOT0J2Lm0d1QZsd8u6d7ys0Mn9YaUGYh7JM2U9PkeHuNo\nSXeVfa6W9JYO6kyW1FxeHyrpEUlTgR3r6nxY0t2Spku6ucxoLCfp95LWKnWWk/SH2vuIiOh/XT7c\nV9ZVP8v2e4BZ/RTTYLIvcIPtRyT9TdIY2x+RNL+2jr2kdwLX2/5ZeX8LcJjt30v6F6rV6fYo7b0d\n2MF2m6T3AVdI+hJwM3CB7SeoRv5H2/5QaW8VYC/bL0saBVwONHdQbzwwz/ZYSUOBOyTdRHWL5l2S\nZtQd19uA75bXFwOH254i6UTgOODIjk6GpJHACcAYqts9twHTy+bfAtvZtqTPAl+x/WVJl1Ity3sa\nsCdwn+1nOmh7PNVMCkNWz3VBRERf6clT/bdI+hjwczfeUn4HAv9XXv+kvJ/WWeUyQ7ADcKWkWvHQ\nuipX2m4DsH2jpA2B9wMfAKZL2qKDZlcAzpQ0muqhyo076X5vYKvafXlgODCK6lMYf6xdqJQ4jy+/\nhwNr2J5SNl0EXNnZ8QH/AkyuJW5JV9TF83aqC5mRwIrAn0r5+cC1VIn/M1SfEHkT2xOBiQBDR45q\ntH9nERH9pieJ//NU9/lfk/Qy1Zrstr16n0Y2wCS9lWqkvqUkA0MASzqmi92WA56rT7LtvFD/xvbf\ngR8DP5Z0PbAL8Ld2+xwFPAVsXdp/ubOQqUbuN7Y7jqYu4l2SzgC+Z/s6SbsBxwPY/oukpyTtAWxL\nNfqPiIgB0u3DfbZXs72c7RVtr17eL9NJv9gfuMT2BrabbK9PNYrduV295/8/e3ceZ0dVr/v/8xiG\nJAxBRD0RwSAEEAgEaFAZFBRwBERBiCiG4xFRRNSLl/iTi+JxAPH8EFTAqEyCwMEJJFwmmcMQdgbS\nARkEghpRggfDEAiQPPePWpvsND2nx+zn/Xr1q2uvWrXqW5WGb61VtWsB6wDYfgp4RNJBAKps117j\nkt5VhvGRtA6wKfDnxvaKMcBjtpcBn6C6AKGdelcDn5W0emlzc0lrdXaAthcBT0qqH9MngJs62eRO\n4J2SXlP2c1CbOBeU5U+22e5nwAU0jHhERMTg6LLHL+kd7ZXbvrnvwxlSJgEntyn7NSs+3Q/VLYCf\nSvoC1cXCocCZko6nGqa/GLi7nfZ3pBrCf4nqAuxntu8qCXWppLupXp50BvBrSYcBV7F81GBum3qn\nUT3pP0vVfYaFVN846MongbPKRcjDwOEdVbT9WLlNcDvwL6DxuYFvUN3ieBK4HtikYd3lVEP87Q7z\ntzVhwzHU8jKdiIh+oa5u20v6fcPHkVTDtTNtv6uDTSJWUL4RcKrttqMl7WppaXGtVuvnqCIiVi2S\nZtpu6apelz1+2/u2aXgjqge1IrokaQrwWXJvPyJiSOjN7Hx/Bd7S14HEqsn2SeU5iVsHO5aIiOje\nPf4fUl7XS3WhMBGY1Z9BRURERP/oztf5Gm+2vgRcZHt6P8UTERER/ag7iX8926c1Fkg6pm1ZRERE\nDH3ducff9jvZAJP7OI6IiIgYAB32+CVNopr9bZMyMU3dOsD/9HdgERER0fc6G+q/DXgM2AD4r4by\np6leHhMRERHDTJcv8IkYaGuOHe+xnxyYV0XMzxsCI2IV0d0X+HR5j1/S28oc789IekHSUklP9U2Y\nq75yvuZImifp0vr7+Qdo33tLmimptfx+Vyk/RtIPGur9RNJ1DZ+PlnR6L/c5X9IGKx99RET0h+48\n3PcjqvfTPwiMAv4D+HF/BrWKec72RNvbAC8ARzauLBP59OZFSt3xBLCv7QlUD2n+opRPp5o+uG47\nYIyk+gRAu1Dd6omIiFVMtxKO7T8BI2wvtX0O1Rzy0XO3AJtJGifpfknnA/OAjSRNKj3zeZJenhyo\njLScIukeSddJ2lnSjZIelrRfqTNS0jll+9mS9gSwPdv230pT9wCjJK1JNbnO5pJGSRoDPFfKJpS6\nu1BdHCDp45JmlFGLn9QvDiTtI+l2SbPKSMbajQda2v6/kj4taS1J0yTdXY7v4P45vRER0ZXuJP7F\nktYA5kj6nqQvdXO7aCBpNeB9QGspGg+cYXtr4EWqmQDfRfVmxJ0k1WfWWwu4vtR7GvgWsDdwAPDN\nUucowKVnPwk4T9LINiF8BJhle4ntl4DZwE7A26im270D2EXShlTPfvxF0luAg4FdbU8ElgKHlqH8\n44G9bO9A9ZKnLzfsa23g91Qve/op1YXi32xvV0Y+rmrn/BwhqSaptnTxom6f14iI6JnuJPBPlHqf\np5oSdiOqJBLdM0rSHKrk+Gfg56X8Udt3lOWdgBttLyxJ+UKgPh3yCyxPlK3ATbZfLMvjSvluVPPd\nY/s+4FFg83oAkramurD4TENct1H17Hehmmb39obP9WH+d1NNH3xXOYZ3A2+muljYCpheyj8JvKmh\n7cuAc2yf3xD33pJOlrS77VdkdttTbbfYbhkxeky7JzIiIlZed2bne1TSKGCs7RMHIKZVzXOlt/wy\nSVBdRHXHi17+1YtlwBIA28vKKEKnJL0R+C1wmO2HGlZNp3reYCTVMxsLqZL5QpYnfgHn2f5qmzb3\nBa61PamD3U4H3ivpl648IGkH4P3AtyT9wfY3O9g2IiL6UXee6t+X6v7vVeXzxDYv9ImVNwN4p6QN\nyj30ScBNPdj+Fsq0t5I2BzYG7pe0HjANmNLO/Aq3U/XcX2v78XJxsRDYn3J/H/gDcKCk15W215f0\nJqrbArtK2qyUr1X2W3cC8CTlIVBJbwAW274AOAXYoQfHFhERfag7Q/3fAHYG/gVgew6wST/G1HRs\nPwZMAW4A7gZm2r6sB02cAbxKUitwCTDZ9hKq2zObASeUh/Pm1JO47SepEv09De3cDryuxIDte6nu\n5V8jaS5wLdXIz0Kq1zZfVMpvB7ZsE9MxVLc5vkf10OCMclvg61TPKURExCDo8gU+ku6w/TZJs21v\nX8rm2t52QCKMptPS0uJardZ1xYiIeFl3X+DTndn57pH0MWCEpPHAF8h3vCMiIoalDof6JdVf9vIQ\nsDXVQ2UXAU8BX+z/0CIiIqKvddbj37E8lHUwsCcrTtQzGni+PwOLiIiIvtdZ4j+L6qnuN1N9B71O\ngEt5REREDCMdDvXbPt32W4Czbb+54WcT20n6ERERw1CXX+ez/dmBCCQiIiL6X965HxER0USS+CMi\nIppId77HHzGgWhcsYtyUaSvdzvyTPtAH0URErFrS44+IiGgiSfzDjKSl5Z378yRdKmn0IMczWdKP\nBjOGiIjoviT+4ec52xNtbwO8QDW17stUyb9rRES0KwlieLsF2EzSOEn3SzofmAdsJGmSpNYyMnBy\nfQNJz0g6RdI9kq6TtLOkGyU9LGm/UmekpHPK9rMl7dlFHBuVNh6U9PXSxlckfaEsnyrp+rL8LkkX\ntm1A0hGSapJqSxcv6puzExERr5DEP0xJWg14H9BaisYDZ9jeGngROBl4FzAR2EnSh0q9tYDrS72n\nqabI3Rs4APhmqXMUYNsTgEnAeZJGdhLOzsBHgG2BgyS1UF2U7F7WtwBrS1q9lN3ctgHbU2232G4Z\nMXpMz05GRER0WxL/8DOqzGtfA/4M/LyUP2r7jrK8E3Cj7YW2XwIuBN5R1r0AXFWWW4GbbL9YlseV\n8t2ACwBs3wc8CmzeSUzX2v6n7eeA35TtZ1LN97Au1QRPt1NdAOxOdVEQERGDIF/nG36esz2xsUAS\nwLPd3P5F2y7Ly6iSMraXlVGE3nDbz7ZflPQIMJlqGue5VJM9bQb8sZf7iYiIlZQe/6ppBvBOSRtI\nGkE1XH9TD7a/BTgUQNLmwMbA/Z3U31vS+pJGAR8Cpje0cyzV0P4tVA8izm648IiIiAGWHv8qyPZj\nkqYAN1DNpjjN9mU9aOIM4ExJrcBLwGTbSzqpPwP4NfBG4ALb9dkcbwG+Btxu+1lJz9ONYf4JG46h\nlpfvRET0C6XzFUNNS0uLa7Va1xUjIuJlkmbabumqXob6IyIimkiG+qNbJL2H6iuCjR6xfcBgxBMR\nEb2TxB/dYvtq4OrBjiMiIlZOhvojIiKaSBJ/REREE0nij4iIaCJJ/BEREU0kD/fFkNO6YBHjpkzr\n1bbz8+KfiIhOpccfERHRRJL4V4Kkr5V57edKmiPprZK+KGl0N7btbr0byzS39c/jJM3rYps9JF1R\nlvcrr+9tr94zXe2/JySdUs7HKX3ZbkRE9J0M9feSpLcDHwR2sL1E0gbAGsAlVFPaLu6iiS92s95K\nsT5UGCQAACAASURBVH05cHl/7qPBEcD6tpcO0P4iIqKH0uPvvbHAE/XJa2w/ARwIvAG4QdINAJLO\nlFQrPeETS9kX2qm3j6TbJc2SdKmktbsKQNJISedIapU0W9Ke7dSZLOlHZXmTso9WSd9qqLO2pD+U\nfbdK2r+Uf1PSFxvqfVvSMR3EcjmwNjBT0sGSHlFlPUlLJb2j1LtZ0vh2tj+inKfa0sWLujr0iIjo\npST+3rsG2EjSA5LOkPRO26cDfwP2tF1Pwl8rkyZsSzVV7rZt65XRguOBvWzvANSALzfs68JyK2EO\ncGVD+VGAbU+gmnr3PEkjO4n5NODMUv+xhvLngQPKvvcE/kuSgLOBwwAkvQo4hGqU4hVs7wc8Z3ui\n7UuopvHdCtgNmAXsLmlNYCPbD7az/VTbLbZbRowe08khRETEykji7yXbzwA7Ug1vLwQukTS5naof\nlTQLmA1sTZUM23pbKZ9ekvsngTc1rD+0JNSJwPsbynejJGLb9wGPApt3EvauwEVl+RcN5QK+I2ku\ncB2wIfB62/OBf0raHtgHmG37n5203+gW4B3l57sl1p2Au7q5fURE9IPc418J5V72jcCNZe76Tzau\nl7QJcCywk+0nJZ0LtNcjF3Ct7Un9GzEA7c3DfCjwWmBH2y9Kms/yOH8GTAb+jWoEoLtuBj5LdUvj\nBOArwB5UFwQRETFI0uPvJUlbtLlXPZGqx/00sE4pWxd4Flgk6fXA+xrqN9a7A9hV0mal7bUkddZz\nr7uFKmlT6m9MNcTekelUw/XUtyvGAI+XpL8nK442/BZ4L1VvvSeT9MwAdgGW2X4emAN8huqCICIi\nBkl6/L23NvBDSesBLwF/ohr2nwRcJelv5f79bOA+4C9Uibduapt6k4GLyn1wqO75P9BFDGcAZ5bR\nhpeAyeUbBh3VPwb4paTjgMsayi8Efl/aqZV4AbD9QnkA8V89eVq/xPEXqosaqC5SJgGtXW07YcMx\n1PIinoiIfiG7vZHfiEp5qG8WcFB7D+X1h5aWFtdqtYHYVUTEKkPSzPIweacy1B8dkrQV1UjGHwYq\n6UdERP/KUH90yPa9wJsbyyRNYMVvBAAssf3WAQssIiJ6LYk/esR2K9WDjBERMQxlqD8iIqKJJPFH\nREQ0kST+iIiIJpLEHxER0UTycF8MOa0LFjFuyrRebTs/L/6JiOhUevwRERFNJIl/AEmypAsaPq8m\naaGkK8rn/SRN6UW74yTNa1P2DUnHrnzUIOmbkvZqp3yP9mKX9KHy8p+IiBhiMtQ/sJ4FtpE0yvZz\nwN7AgvpK25cDlw9WcB2xfUI36jTG/iHgCuDe/owrIiJ6Lj3+gXclUL8RPQm4qL5C0mRJPyrL50o6\nXdJtkh6WdGBvdyjpRkktZXmDMu1ufX+/k3StpPmSPi/py5JmS7pD0voNsRxYlt8r6T5Js4APt41d\n0i7AfsApkuZI2rTUrdcb3/i5ofwISTVJtaWLF/X2UCMiogtJ/APvYuAQSSOBbYE7O6k7FtgN+CBw\nUhftbloS7RxJc4AjuxnPNlQJfCfg28Bi29sDtwOHNVYsMf8U2BfYEfi3to3Zvo2q5/8V2xNtP0Q1\nLXH9bX+HA+e0s91U2y22W0aMHtPN0CMioqeS+AeY7bnAOKre/pVdVP+d7WXlnfmv76LuQyXRTrQ9\nETirmyHdYPtp2wuBRcDvS3lribPRlsAjth90Na3jBXTPz4DDJY0ADgZ+2c3tIiKijyXxD47Lge/T\nMMzfgSUNy1qJ/b3E8n/rkZ3sY1nD52X03TMgvwbeRzVyMdP2P/uo3YiI6KEk/sFxNnBimfBmIMyn\nGpoH6PWzAsB9wDhJm5bPkzqo9zSwTv2D7eeBq4EzaWeYPyIiBk6e6h8Etv8KnD6Au/w+8N+SjgB6\n92YcqgReb0PSYuAWGhJ8g4uBn0r6AnBguc9/IXAAcE1X+5mw4RhqeRFPRES/UHWrNqJ/lXcKjLH9\nf7qq29LS4lqtNgBRRUSsOiTNtN3SVb30+KPfSfotsCnwrsGOJSKi2SXxDyOSJgC/aFO8xPZbByOe\n7rJ9wGDHEBERlST+YaQ8DDixy4oREREdyFP9ERERTSSJPyIiookk8UdERDSRJP6IiIgmkof7Yshp\nXbCIcVN6956h+XnxT0REp9Ljj4iIaCJJ/MOUpKVlCt55ki6VNHoIxHSRpLmSvjTYsURERPuS+Iev\n58oUvNsALwBHNq5UZcD+fSX9G7CT7W1tnzpQ+42IiJ5J4l813AJsJmmcpPslnQ/MAzaSNElSaxkZ\nOLm+gaRnJJ0i6R5J10naWdKNkh6WtF+pM1LSOWX72ZL27CSGa4ANyyjEOyXNLG1sJ8mSNi6fHxoK\noxMREc0qiX+Yk7Qa1Vz39Sl+xwNn2N4aeBE4meod+ROBnSR9qNRbC7i+1Hsa+BawN9UMet8sdY4C\nbHsC1RS850ka2UEo+wEPlVGIm4CRktYFdgdqwO6S3gQ8bntxO8dxhKSapNrSxYt6fT4iIqJzSfzD\n1yhJc6iS6p+Bn5fyR23fUZZ3Am60vdD2S1RT476jrHsBuKostwI32X6xLI8r5bsBFwDYvg94FNi8\nm/HdBuxa9ved8nt3qtGJV7A91XaL7ZYRo8d0cxcREdFT+Trf8PWc7RXe2y8J4Nlubv+il8/JvAxY\nAmB7WRlFWFk3UyX6NwGXAccBBnr3Pb2IiOgT6fGv2mYA75S0gaQRVMP1N/Vg+1uAQwEkbQ5sDNzf\ng20/DjxoexnwP8D7gVt7sP+IiOhj6fGvwmw/JmkKcAMgYJrty3rQxBnAmZJagZeAybaXdHPf81UN\nQdxcim4F3mj7ya62nbDhGGp5EU9ERL/Q8tHeiKGhpaXFtVptsMOIiBhWJM203dJVvQz1R0RENJEM\n9UePSHoP1VcEGz1i+4DBiCciInomiT96xPbVwNWDHUdERPROhvojIiKaSBJ/REREE0nij4iIaCJJ\n/BEREU0kD/fFkNO6YBHjpnT+Zt/5ecFPRESvpMcfERHRRJL442WS1pP0uX5s/5n+ajsiIroniT8a\nrQe8IvH30Wx9ERExBOR/6NHoJGBTSXOAF4HngSeBLYHNJX0c+AKwBnAn8DnbS0tP/jTgg8BzwP62\n/yFpE+CXwNpUU/NGRMQgS48/Gk0BHrI9EfgKsANwjO3NJb0FOBjYtaxfSpmyF1gLuMP2dlSz8X26\nlJ8GnGl7AvBYZzuWdISkmqTa0sWL+vzAIiKiksQfnZlh+5Gy/G5gR+CuMiLwbuDNZd0LwBVleSYw\nrizvClxUln/R2Y5sT7XdYrtlxOgxfRR+RES0laH+6MyzDcsCzrP91Xbqvejl8zsvZcW/q8z7HBEx\nhKTHH42eBtbpYN0fgAMlvQ5A0vqS3tRFe9OBQ8ryoZ1VjIiIgZEef7zM9j8lTZc0j+ohvX80rLtX\n0vHANZJeRfXw31HAo500eQzwS0nH0YOH+yZsOIZaXtATEdEvtHyENmJoaGlpca1WG+wwIiKGFUkz\nbbd0VS9D/REREU0kiT8iIqKJJPFHREQ0kST+iIiIJpLEHxER0USS+CMiIppIEn9EREQTyQt8Yshp\nXbCIcVOmvaJ8fl7qExGx0tLjj4iIaCJJ/BEREU1k2CV+SZZ0QcPn1SQtlHRFZ9v1oH1JekLSq8vn\nsWWfuzXUWSjpNZKOlHRYD9u/UVKHr1SU9O+SWiXNlTRP0v69P5puxXOupAPbxiZpE0kPSnrPSrbf\n6fFGRMTAGo73+J8FtpE0yvZzwN7Agr5q3LYl3QG8HbgS2AWYXX7fKmkL4J+2/wmc1Vf7BZD0RuBr\nwA62F0laG3htX+6jB3FcBfwv21cP9P4jIqL/DLsef3ElUH/SaxJwUX2FpJ0l3S5ptqTbSqJG0pck\nnV2WJ5Te9OgO2r+NKtFTfp9KdSFQ/zy9tPMNSceW5RslnSxphqQHJO1eykdJuljSHyX9FhjVyXG9\njmpq3GcAbD9j+5GG9k+TNKfEvnMpX0vS2WW/s+sjBJJGSDpF0l1l9OAzpVySfiTpfknXlX02Ggtc\nA3zN9uVlm8mSfifpWknzJX1e0pfL/u6QtH4nxwRwUNvz0pakIyTVJNWWLl7URXMREdFbwzXxXwwc\nImkksC1wZ8O6+4DdbW8PnAB8p5SfBmwm6QDgHOAzthd30P50lif+nYHfAhuVz7tQXRi0ZzXbOwNf\nBL5eyj4LLLb9llK2YyfHdTfVVLiPSDpH0r5t1o+2PRH4HHB2KfsacH3Z757AKZLWAj4FLLK9E7AT\n8GlJmwAHAFsAWwGHNRxn3XnAj2z/qk35NsCHS1vfLse0PXB7aacz7Z2XFdiearvFdsuI0WO6aC4i\nInprOA71Y3uupHFUvf0r26weA5wnaTxgYPWyzTJJk4G5wE9sT+9kF3cB25cEurrtZyQ9LGkzqkT5\nXx1s95vyeyYwriy/Azi9Ie65nRzXUknvpUqu7wZOlbSj7W+UKheVejdLWlfSesA+wH71kQdgJLBx\nKd+2fv++nJfxJZ6LbC8F/ibp+jZhXAd8XNK5bS6MbrD9NPC0pEXA70t5K9XFV2faOy8RETEIhmuP\nH+By4Ps0DPMX/0mVpLYB9qVKhHXjqYbR39BZwyXhPQj8OzCrFN8BvJ9qaPz+DjZdUn4vpZcXVa7M\nsP1d4BDgI42r21YHBHzE9sTys7HtP5byoxvKN7F9TTdC+B7Vhc+lkhqPYUnD8rKGz8vo+lhX+rxE\nRETfGM6J/2zgRNutbcrHsPxhv8n1QkljqHre7wBe09AT7shtVEPTt5fPtwPHAHfYbpuAO3Mz8LES\nwzZ00juW9AZJOzQUTQQebfh8cKm3G9Uw/iLgauBoSSrrti91rwY+K2n1Ur55GcG4GTi4PAMwlur2\nQFtfBJ4Cfl5vdyBN2HAM80/6wCt+IiJi5Q3bxG/7r7ZPb2fV94DvSprNir3LU4Ef236A6v73SZLa\nPtjWaDrwZpYn/lnAG+n4/n5HzgTWlvRH4JtUw90dWR34vqT7JM2hSvTHNKx/vhzXWeUYoBrhWB2Y\nK+me8hngZ8C9wCxJ84CfUJ2P31KNZtwLnN9wfC8rFzafpHrQ73s9PN6IiBjC1LPOawwWSTcCx9qu\nDXYs/a2lpcW12ip/mBERfUrSTNtdvjdl2Pb4IyIiouea+kErSYez4lA6wHTbRw3Avu8E1mxT/Il2\nnlkAwPYe/R3TypD0Y2DXNsWn2T5nMOKJiIj2NXXiL0lpUBKT7bcOxn77y0BcLEVExMrLUH9EREQT\nSeKPiIhoIkn8ERERTaSp7/HH0NS6YBHjpkx7RXle4hMRsfLS44+IiGgiSfwRERFNJIl/mJO0nqTP\n9WP7z3Sybpyk5yTNlvRHSTPKDIj19ZMlLZQ0p+Fnq/6KNSIiupZ7/MPfesDngDMaCyWtZvulAdj/\nQ7a3L/t8M/AbSWp4cc8ltj8/AHFEREQ3pMc//J0EbFp603dJukXS5VST8CDp46UnPkfSTySNKOXP\nSPq2pLsl3SHp9aV8E0m3S2qV9K2eBGL7YeDLwBd6ehCSjpBUk1RbunhRTzePiIhuSuIf/qZQ9bon\nAl8BdgCOsb25pLdQzfC3a1m/FDi0bLcW1RTD21FN1fvpUn4acKbtCcBjvYhnFrBlw+eD2wz1j2pv\nI9tTbbfYbhkxekwvdhsREd2Rof5Vzwzbj5TldwM7AndJAhgFPF7WvQBcUZZnAnuX5V2Bj5TlXwAn\n93D/avM5Q/0REUNIEv+q59mGZQHn2f5qO/Ve9PI5mZey4t/CyszVvD3wx5XYPiIi+lES//D3NLBO\nB+v+AFwm6VTbj0taH1jH9qOdtDcdOAS4gOW3BbpF0jjg+8APe7JdWxM2HEMtL+uJiOgXSfzDnO1/\nSpouaR7wHPCPhnX3SjoeuEbSq4AXgaOAzhL/McAvJR0HXNaNEDaVNBsYSXURcrrtcxvWHyxpt4bP\nn7N9W3eOLSIi+p6Wj/ZGDA0tLS2u1WqDHUZExLAiaabtlq7q5an+iIiIJpKh/uiSpAlUT/g3WmL7\nrYMRT0RE9F4Sf3TJdiswcbDjiIiIlZeh/oiIiCaSxB8REdFEkvgjIiKaSO7xx5DTumAR46ZMe0X5\n/LzUJyJipaXHHxER0UTS429CkpYCrQ1FF9s+qQ/b3wN4IW/oi4gYepL4m9NzZZre/rIH8AyQxB8R\nMcRkqD8AkPReSZc2fN5D0hVleR9Jt0uaJelSSWuX8vmSTizlrZK2LBP1HAl8SdIcSbtLOkjSPEl3\nS7p5MI4vIiIq6fE3p1GS5jR8/i7wa2CqpLVsPwscDFwsaQPgeGAv28+WyXu+DHyzbPuE7R0kfQ44\n1vZ/SDoLeMb29wEktQLvsb1A0nrtBSTpCOAIgBHrvrbvjzgiIoAk/mbV7lC/pKuAfSX9CvgA8L+B\ndwJbAdMlAawB3N6w2W/K75nAhzvY33TgXEn/3VB/BbanAlMB1hw7PjNHRUT0kyT+aHQx8Hngf4Ca\n7adVZftrbU/qYJsl5fdSOvh7sn2kpLdSXUzMlLSj7X/2cewREdENuccfjW4CdgA+TXURAHAHsKuk\nzQAkrSVp8y7aeRpYp/5B0qa277R9ArAQ2KjPI4+IiG5J4m9Oo8qDd/WfkwBsLwWuAN5XfmN7ITAZ\nuEjSXKph/i27aP/3wAH1h/uAU8rDf/OonvS/u1+OKiIiuiQ7t1NjaGlpaXGtVhvsMCIihhVJM223\ndFUvPf6IiIgmksQfERHRRJL4IyIimkgSf0RERBNJ4o+IiGgiSfwRERFNJIk/IiKiiSTxR0RENJG8\nqz+GnNYFixg3Zdoryuef9IFBiCYiYtUy5Hr8kv5N0sWSHpI0U9KV3Xg3fF/HME7Sxxo+t0g6vSzv\nIWmXhnVHSjqsl/vZQ5Il/UdD2cRSdmwv25wo6f0Nn7/Rk7YkrSHpB5L+JOlBSZdJemNvYomIiKFn\nSCX+MhPcb4EbbW9qe0fgq8DrBziUccDLid92zfYXysc9gF0a1p1l+/yV2Nc84KMNnyexcu+ynwi8\nv8taHfsO1QQ7W9geD/wO+E35t4mIiGFuSCV+YE/gRdtn1Qts3w3cKukUSfPKZC8Hw8s95ptKr/Rh\nSSdJOlTSjFJv01LvXElnSapJekDSB0v5iNLuXZLmSvpM2e1JwO5lkpkvlf1cIWkccCTwpfoENI09\n6tLbvqO09VtJry7lN0o6ucT1QJm4pu5RYKSk15fk+l7g/9ZX9qRNSWsA3wQOLvEdXJrZqtR/WNIX\nyvZrSZom6e5yXg+WNBo4HPhSmbAH2+dQTb37rjIScp+kCyX9UdKvyjZI2rH8W8yUdLWksd049oiI\nGGBDLfFvA8xsp/zDVD3Z7YC9qGZ7G1vWbUeVjN8CfALY3PbOwM+AoxvaGAfsTDUn/FmSRgKfAhbZ\n3gnYCfi0pE2AKcAttifaPrXegO35wFnAqWXdLW3iPB84zva2QCvw9YZ1q5W4vtimHOBXwEFUIwmz\nWD7HfY/atP0CcAJwSYnvklJvS+A95fi/Lml1qguMv9nezvY2wFXAZsCfbT/VJr4asHVZ3gI4w/Zb\ngKeAz5X2fggcWEZpzga+3c1jB0DSEeXCrLZ08aL2qkRERB8Yaom/I7sBF9leavsfVPPG71TW3WX7\nMdtLgIeAa0p5K1Wyr/tv28tsPwg8TJUM9wEOkzQHuBN4DTC+NwFKGgOsZ/umUnQe8I6GKr8pv2e2\niQvgv6kS/yTgoj5qs9E020tsPwE8TnXrpBXYu/TGd7fd3Wz7F9vTy/IFVP82W1BdtF1bzuXxQONz\nAV3GaXuq7RbbLSNGj+lmKBER0VNDLfHfA+zYw20ae8fLGj4vY8VvLbSdf9iAgKNL73ii7U1sX0P/\nqMe1tE1c2P478CKwN/CHvmizg3ov17X9ALAD1QXAtySdQHXhtLGkddpsvyPVvw10fB7vaTiPE2zv\n04s4IyKinw21xH89sKakI+oFkrYF/kV133qEpNdS9Xpn9LDtgyS9qtz3fzNwP3A18NkyVI2kzSWt\nBTxN9YBbe9pdV3rMTzbcw/4E1chEd51ANaS/dCXb7Cz2l0l6A7DY9gXAKcAOtp+lGlX4/yWNKPUO\nA0ZT/dtAdWHw9rL8MeBWqnP52nq5pNUlbU1ERAw5Q6r3ZduSDgB+IOk44HlgPtW94bWpnnY38L9t\n/13Slj1o/s9UFwvrAkfafl7Sz6iGnmeVB+sWAh8C5gJLJd0NnAvMbmjn98CvJO3Pis8QAHyS6vmB\n0VS3Ew7vwbHf1sGqnrZ5AzClDLl/t5N6E6ielVhGNdrw2VL+VeD7wANl3X3AAeXfBqokf5Sks4F7\ngTNtvyDpQOD0cntiNeAHLB8liIiIIUJ225HbVY+kc4ErbP9qsGMZzsq3Gq4oDwP2m5aWFtdqtf7c\nRUTEKkfSTNstXdUbakP9ERER0Y+G1FB/f7E9ebBjWBWUrzP2a28/IiL6V3r8ERERTSSJPyIiookk\n8UdERDSRJP6IiIgmksQfERHRRJL4IyIimkhTfJ0vhpfWBYsYN2XaK8rnn/SBQYgmImLVkh5/RERE\nExmyiV+SJV3Q8Hk1SQslXTEA+95f0u8aPn9V0p8aPu8r6fKyfKWk9XrQ9jhJ8zpZP1rShZJaJc2T\ndKuktXt7LN2M6Zn2YpP0aUkzJb16Jdru9HgjImJgDeWh/meBbSSNsv0c1ZS1CwZo37cBP2n4/Hbg\nKUmvs/04sEupg+339/G+jwH+YXsCgKQtqCbRGVCSPkE1CdG7bD850PuPiIj+MWR7/MWVQP3G7iTg\novoKSWtJOlvSDEmzy2x59R7mLZJmlZ9dSvkekm6U9CtJ95Vetdrbqe2FVIl+s1K0IfBrqoRP+T29\ntDtf0gZlv3+U9FNJ90i6RtKoUmdHSXeX2f6O6uKYx9JwgWP7fttLSvv1uP9YjmN0Q/s3ld751ZLG\nlvJNJV1Vym+pz2YoaRNJt5dRhW+1DUDSR4EpwD62nyhlN0o6VVKt7H8nSb+R9GB7bbQxor3z0maf\nR5S2a0sXL+qiuYiI6K2hnvgvBg6RNBLYFrizYd3XgOtt7wzsSTXF7FrA48DetncADgZOb9hme6op\nfrcC3gzs2sm+pwO7lB73g8Ad5fNqwHbAXe1sMx74se2tgX8BHynl5wBH296uG8d8NnBcSczfkjS+\nYd0WwBm23wI8BXxO0urAD4EDbe9Ytv92qT+17HdH4FjgjFJ+GtV0uhOAx9rs/03Aj6iS/t/brHuh\nzPx0FnAZ1UXMNsBkSa/p5Jg6Oi8vsz3VdovtlhGjx3TSVERErIwhnfhtzwXGUfX2r2yzeh+Wzzt/\nIzAS2BhYHfippFbgUqokXzfD9l9tLwPmlLY7chtVz34X4HZgBvBWqouH+2w/3842j9ieU5ZnAuPK\n/f/1bN9cyn/RxTHPobooOQVYH7hL0lvK6r/Ynl6WLwB2o7oY2Aa4tpyL44E3lucCdgEuLeU/oRpN\ngOqCpz560jaehcCfgY+2E97l5XcrcI/tx2wvAR4GNurksF5xXjqpGxER/Wgo3+Ovuxz4PrAH0Nir\nFPAR2/c3Vpb0DeAfVL3yVwGNCXpJw/JSOj/+6VT3uEcAP7X9dBl52INyf78dbdt/xZB2d9h+BvgN\n8BtJy4D3U91qcNuqVOfhHttvb1whaV3gX7YndrSbDsoXl/3dIulx2xc2rKsf3zJWPNZldH4u++S8\nRETEyhvSPf7ibOBE261tyq8Gjq7fp5e0fSkfAzxWevWfoErcvfFH4A1UverZpWwOcCTl/n532P4X\n8C9Ju5WiQzurL2nX+lP0ktagGrF4tKzeWFI9wX8MuBW4H3htvVzS6pK2tv0U8Iikg0q5JNVvNUwH\nDukonvIA43uB70h6T3ePNSIihr4h3+O3/VdWvE9f95/AD4C5kl4FPAJ8kOo+9q8lHQZcRfXtgN7s\n15LuBMbYrj9VfztwBB33+DtyOHC2JAPXdFF3U+DMckHzKmAaVW//TVRJ/ihJZwP3Ut2nf0HSgcDp\nksZQ/Zv+ALiHKqmfKel4qlsgFwN3U31z4JeSjqO6V9/e8T8iaT/gSkkH9PB4V8qEDcdQy8t6IiL6\nheyORnxjKJE0DrjC9jaDHEq/a2lpca1WG+wwIiKGFUkzywPYnRoOQ/0RERHRR4b8UH9/k/RbYJM2\nxcfZvrqf9/se4OQ2xY/YbndY3fZ8qqf3h6Tydb4/tLPq3bb/OdDxRERE+5o+8XeUaAdgv1dTPaC4\nSijJvaNvEERExBCRof6IiIgmksQfERHRRJL4IyIimkgSf0RERBNp+of7YuhpXbCIcVOmvaJ8fl7q\nExGx0tLjj4iIaCJNm/glWdIFDZ9Xk7RQ0hV91P7hkuaUnxcktZblk/qi/f4gaX1JR/ZDu5MlzSvn\nYJakL/X1PiIionuaeaj/WWAbSaNsPwfsDSzoq8ZtnwOcAyBpPrCn7Sf6qv3ekrSa7Zc6WL0+1SRE\nZ/VVm5I+CHwe2Mv238sMhx/vSfsREdF3mrbHX1wJ1G8cT2L5HPVI2lnS7ZJmS7pN0hal/Etlkhwk\nTSg92dE92amktSWdK2lGaX/fUv4fkn4j6TpJj0r6rKSvNMSwXql3q6QflBGEVkkt3Wj3d5JuAK6W\ntK6k60vve25JzgAnAVvURyYk7SXpdw1xnyXp42X5r6XObOAASeMlXS1ppqSbJW1eNvv/gC/b/juA\n7edt/6ydc3KEpJqk2tLFi3pyOiMiogeaPfFfDBxSeqHbAnc2rLsP2N329sAJwHdK+WnAZmXGunOA\nz9he3MP9ngBcZXtn4F3Af5UYALYG9gd2pnql75Mlhpms2FNe0/ZEqpn2ftaNdrcHPmz73cBzwIds\n7wDsBZxa6kwB7rc90faUbhzH47a3t30pMBX4nO0dga8CP2o4npldNWR7qu0W2y0jRo/pxq4jIqI3\nmnmoH9tzy6x3k6h6/43GAOdJGg+YalpbbC+TNBmYC/zE9vRe7Hof4H2S6sl1JLBxWb7e9rPA6bHU\nXgAAFBFJREFUs5KeAX5fyluBzRvauKjEc72k10lau4t2r7H9ZFkWcJKk3YBlwEaSNujFcVwCUEYi\n3kY1HXJ9XVP/bUVEDFX5nzNcDnwf2AN4TUP5fwI32D6gXBzc2LBuPPAM8IZe7lNUPe6HViiU3gEs\naSha1vB5GSv+e7WdT9ldtPtsQ9FhVBc2O9h+SdJfqS4S2nqJFUeF2taptyngiTIC0da9wI7Aze2s\ni4iIAdbsQ/0AZwMn2m5tUz6G5Q/7Ta4XShoDnA68A3iNpAN7sc+rgaMb2ty+F20cXLbdA/hHGSXo\nbrtjqIbpX5K0N7BhKX8aWKeh3qPA1pLWkPRqqtsHr1BGEh4rtz+Q9CpJ25XV3wW+L+n1Zd2akj7V\n46ONiIg+0fQ9ftt/pUrkbX2Paqj/eKDxbTKnAj+2/UBJYDdIutn24z3Y7YnADyS1Ul18/Ynqvn5P\nvChpDjACOLyH7f4C+H2pNwN4EMD2P8rDea3ANNtTysN99wAPA7M6iecQ4ExJ3wDWAC4A7rZ9uaTX\nAteX2wAGftrZgU3YcAy1vKwnIqJfyG47YhxDnaRbgc/bnjPYsfSHlpYW12q1wQ4jImJYkTTTdktX\n9TLUHxER0USafqi/L0g6nOprdY2m2z6qP/Zne7f+aDciIlZ9Sfx9oPEtfREREUNZhvojIiKaSBJ/\nREREE0nij4iIaCJJ/BEREU0kiT+GnNYFixg3ZRrjpkzrunJERPRIEn90SJVbJb2voewgSVcNZlwR\nEdF7+TpfdMi2JR0JXCrpBqq/l+8A712ZdiWtZvulvogxIiJ6Jj3+6JTteVRTAx8HnACcb/shSZ+U\nNEPSHElnSHoVgKSpkmqS7pF0Qr0dSX+VdJKk2cABg3IwERGRHn90y4lUE/S8ALRI2oYqee9SZvib\nSjVJzy+BKbb/R9JqVBMY/cr2vaWdx233ZibCiIjoI0n80SXbz0q6BHjG9hJJewE7AbUy494o4C+l\n+qQya+FqwBuArYB64r+ko31IOgI4AmDEuq/tl+OIiIgk/ui+ZeUHQMDZtv9PYwVJ46nmLNjZ9r8k\nXQCMbKjybEeN254KTAVYc+z4TBkZEdFPco8/euM64KOSNgCQ9BpJGwPrAk8DT0kaC7xnEGOMiIh2\npMcfPWa7VdKJwHXlob4XgSOBGtWw/n3Ao8D0wYsyIiLaIzujqjG0tLS0uFarDXYYERHDiqSZtlu6\nqpeh/oiIiCaSxB8REdFEkvgjIiKaSBJ/REREE0nij4iIaCJJ/BEREU0kiT8iIqKJJPFHREQ0kST+\nGHJaFyxi3JRpjJsybbBDiYhY5STxR0RENJEk/oiIiCbS9Ilfksv0sfXPq0laKOmK8nk/SVN62OYE\nSXPKz/9IeqQsX9fX8fclSV+WNLLrmj1q822SbpV0n6TZkqZKGtWX+4iIiO7L7HzVHPHbSBpl+zlg\nb2BBfaXty4HLe9Kg7VZgIoCkc4ErbP+qzyLuJUmimphpWQdVvgycDTzfgzZXs/1SB+vGApcAB9me\nUfb/UWBt4LkeBR8REX2i6Xv8xZXAB8ryJOCi+gpJkyX9qCyfK+l0SbdJeljSgb3ZmaQpkmZImivp\nhFK2maR5kn4h6QFJ50t6T9nXg5JaSr1vSTpP0h2l/N+70e69ki4E7gHGll53TdI9DfW+BLwOuEXS\ndWXk418NbR8i6Wdl+QJJZ0qaAXxH0trl3Mwovfp9y2ZHAz+3PQPAlUtsL2znnBxRYqotXbyoN6c1\nIiK6IT3+ysXACWV4f1uqXu/uHdQdC+wGbEk1EtCjnryk9wMbA28FBFwpaRfgcWALqh7xfcAs4Hnb\nu0j6CDAFqF9oTAB2AdYFZkmaBuzYSbtbAofZrpUYptj+H0mrATdI+pXtUyX9L2B32/8q6zozFnib\n7WWSvgdcZXuypFcDd0q6FtgG+El3zovtqcBUgDXHjs9c0RER/SSJH7A9V9I4qt7+lV1U/10ZKr9X\n0ut7sbt9gPcBs8vntYHNqRL0n2zfCyDpXuAPpU4r8NU2MTwPPC/pZmAnYK9O2n2onvSLSZI+RfXv\n/wZgK+DeHh7HpQ23DPYB3tfwLMRIqouQiIgYYpL4l7sc+D6wB/CaTuotaVhWL/Yj4Fu2f75CobRZ\nm7aXNXxexor/Vm17xO6i3WcbPo8HjgF2Lj37C6gSdVvLWPH42tZ5tmFZwIdsP9Rm3/dQjUTkC/kR\nEUNE7vEvdzZwYnkwrz9dDXxK0loAkt4oaYMetvEhSWtKei3VLYlaD9pdF3gaeKo8fPeehnVPA+sA\nlN78k5LGS3oVcEAXx3R0/YOk7cviD0tM9ecTJOmgEneHJmw4hvknfYD5J32gs2oREdEL6fEXtv8K\nnD4A+7lS0pbAHdVD7jwNfKyHzcwDbqIamfi67X9Q3dPvTruzqIb17wMeBaY3rJsKXCfpL7b3Ao6j\nSuqPAzOBNTuI50TgB5JaqS4m/wTsb/tvkj4GnCbpNVQjEzcCV/TweCMioo/IznNUw4mkbwFP2P7B\nYMfSX1paWlyr1bquGBERL5M003ZLV/Uy1B8REdFEMtS/kiRNAH7RpniJ7bf2x/5sH98f7UZERHNI\n4l9JjW/pi4iIGOoy1B8REdFEkvgjIiKaSBJ/REREE0nijyGndcEixk2ZxrgpeeFfRERfS+KPiIho\nIkn8ERERTSSJvwOSvlbmq58raY6kdr+XL2mypB/10T7n19+vL+mZnu5H0jckHVuWvylpr3bq7FGm\nH+4TjfuMiIihL9/jb4ektwMfBHawvaQk4zUGOawesX3CYMcQERFDT3r87RtL9T78JQC2nygTzuwk\n6TZJd0uaIWmdUv8Nkq6S9KCk79UbkTRJUqukeZJO7qq8uySNk3R9GY34g6SN26lzrqQDy/J7Jd0n\naRbw4YY6O0u6XdLsclxblPKbJU1sqHerpO06CWm70s6Dkj5dtvmxpP3K8m8lnV2W/13St9uJ9whJ\nNUm1pYsX9fSURERENyXxt+8aYCNJD0g6Q9I7Ja0BXAIcY3s7YC/guVJ/InAwMAE4WNJGkt4AnAy8\nq6zfSdKHOipvJ4ZR5RbDHElzgG82rPshcJ7tbYEL6WRWQUkjgZ8C+wI7Av/WsPo+YHfb2wMnAN8p\n5T8HJpftNwdG2r67k/O1bTmetwMnlGO8hWrKYIANga3K8u7AzW0bsD3VdovtlhGjx3Syq4iIWBlJ\n/O2w/QxVkjwCWEiV8D8DPGb7rlLnKdsvlU3+YHuR7eepprx9E7ATcKPthaXehcA7Oilv6znbE+s/\nVIm57u3AL8vyL4DdOjmcLYFHbD/oairGCxrWjQEulTQPOBXYupRfCnxQ0urAvwPndtI+wGW2n7P9\nBHADsDMl8UvaqpyTf0gaW2K/rYv2IiKin+QefwdsL6WaO/7GMs/8UZ1UX9KwvJThc17/E7jB9gGS\nxlEdL7YXS7oW2B/4KNVFUGfazu1s2wskrQe8l6qHv35p6xnbT/fZEURERI+kx98OSVtIGt9QNBH4\nIzBW0k6lzjqSOkvwM4B3StpA0ghgEnBTJ+U9cRtwSFk+lKp33ZH7gHGSNi2fJzWsGwMsKMuT22z3\nM6pbCHfZfrKLePaXNFLSa4A9gLtK+R3AF6kS/y3AsV3ECsCEDccw/6QPMP+kD3RVNSIiemi49EwH\n2trAD0uP9SXgT1TD/ueU8lFU9/df8XW5OtuPSZpCNfQtYJrtywA6Ku+Bo4FzJH2F6lbE4Z3E8byk\nI4BpkhZTJd76Q4nfA86TdDwwrc12MyU9VY65K3PL8WwA/Kftv5XyW4B9bP9J0qNUvf4uE39ERPQf\nVbd9I1ZUHtC7EdjS9rKB3HdLS4trtdpA7jIiYtiTNNN2S1f1MtQfryDpMOBO4GsDnfQjIqJ/Zag/\nXsH2+cD5jWWSDgeOaVN1uu3OHnqMiIghJkP9MeRIehq4f7Dj6MQGwBODHUQHhnJskPhWVuLrvaEc\nG/RNfG+y/dquKqXHH0PR/d25TzVYJNWGanxDOTZIfCsr8fXeUI4NBja+3OOPiIhoIkn8ERERTSSJ\nP4aiqYMdQBeGcnxDOTZIfCsr8fXeUI4NBjC+PNwXERHRRNLjj4iIaCJJ/BEREU0kiT/6laT3Srpf\n0p/KHAVt10vS6WX9XEk7dLWtpPUlXSvpwfL71QMdn6SNJN0g6V5J90g6pmGbb0haIGlO+Xn/QMdX\n1s2X1FpiqDWUD4Xzt0XD+Zkj6SlJXyzr+uT8dSO2LSXdLmmJpGO7s+0An7t24xtCf3udnb+h8LfX\n0fkbCn97h5b/Hlol3SZpu6627ctzh+385KdffoARwEPAm4E1gLuBrdrUeT/wf6kmLHobcGdX21JN\nLjSlLE8BTh6E+MYCO5TldYAHGuL7BnDsYJ6/sm4+sEE77Q76+Wunnb9TvXykT85fN2N7HbAT8O3G\n/Q2hv72O4hsqf3vtxjeE/vY6jG8I/O3tAry6LL+PAfz/nu30+KNf7Qz8yfbDtl8ALgb2b1Nnf+B8\nV+4A1pM0tott9wfOK8vnAR8a6PhsP2Z7FoDtp6mmbd6wl3H0eXxdtDvo569NnXcDD9l+tJdx9Co2\n24/bvgt4sQfbDti56yi+ofK318n568ygn782Butv7zYvn+78DuCN3di2r85dEn/0qw2BvzR8/iuv\n/B9UR3U62/b1th8ry38HXj8I8b1M0jhge6qJjeqOLkN5Z6/EkNzKxmfgOkkzVU3NXDekzh9wCHBR\nm7KVPX/d2W9vth3Ic9elQf7b68xQ+NvrjqHwt/cpqlGxrrbtq3OXxB/Dm6txr0H7TqqktYFfA1+0\n/VQpPpNqqG4i8BjwX4MU3m62J1INJR4l6R1tKwyB87cGsB9waUPxUDl/nRoC5y5/eythKPztSdqT\nKvEf15PtVvbcJfFHf1oAbNTw+Y2lrDt1Otv2H/Xh4vL78UGID0mr8//au7cQq6o4juPfXw5pWSZe\nsgvVWBolFEEmRhEFIeSLRXaR7EIQSEH50OWhCC0oo+gtoiuRSJSgZdFFEzWRLG+pY4kFWlZkIXYT\ni9J/D2sdZ3eYM3Mcj+cc2b8PbGbf1t7/tWY765y9t+uf/vDOi4gFlR0iYldE7I+U0vgl0u27pscX\nEZWfPwMLC3G0Rftl1wDrI2JXZUWD2q+e2PpTtpltV1ObXHs1tcm115eWXnuSLgReBqZExO46yjaq\n7dzx2xG1BhgraXT+dH0zsKhqn0XAbUomAr/l21m9lV0E3J7nbwfeaXZ8kgS8AnwVEc8WC1Q9w74O\n6GpBfIMlnZjjGQxMKsTR8vYrbJ9G1a3WBrVfPbH1p2wz265HbXTt1YqvXa69vrTs2pN0JrAAuDUi\nttVZtlFt57f6PR3ZifRW9zbSm6oP53UzgBl5XsBzeftmYHxvZfP64cBS4GvgY2BYs+MDLifdatsE\nfJGnyXnb3LzvpvyP9dQWxHc26Y3gjcCWdmu/vG0wsBs4qeqYDWm/OmI7hfQM9Xfg1zw/pI2uvR7j\na6Nrr1Z87XLt9fb7bfW19zKwp/D7W9tb2Ua3nYfsNTMzKxHf6jczMysRd/xmZmYl4o7fzMysRNzx\nm5mZlYg7fjMzsxJxx29mbUfScknj+9hnpqTjC8vvSxragHM35DiHcL6hku5u1vnM3PGbWdPlAX0O\n9+/PTOBgxx8RkyPi18M8ZsOOUw9JHcBQwB2/NY07fjNrCkmdSnnGXyeNiHaGpElKOdPXS5qfx5+v\nLve8pLVKuedn53X3AqcByyQty+t2SBohaY6kewrlZynnY5f0gKQ1SklYZteIs3KcTklbJb0maZuk\neZKulrRKKSf6hMLx5+Z6fC3prrxekp6W1KWUd/2mvP5KSSslLQK+BOYA5yjlf39a0gmSluY22Sxp\nSqH9vpL0Um6LxZKOy9vGSPpY0sZc7px662sl1N+Rfzx58uTpUCagEzgATMzLI4BPgMF5+SHg0Ty/\nnO5RCIflnwPy+gvz8g4KOd8ry6RsdSsK678kjX8+CXiRNJrgMcB7wBU9xFk5TifwL3BB3n8d8Gou\nPwV4O+8/izRK3XG53E7Sh5LrgSU57lHAd8CpwJXAXmB0oV26CufvoHuEuRHAN/mclXguytveAqbn\n+c+A6/L8INKdkLrq66l8U0ftjwRmZg33bUSszvMTgXHAqjT8PMcCn/ZQ5kal1K4dpI5zHGlI1R5F\nxAZJJ0s6DRgJ7ImInZLuI3WGG/KuJwBjSR8+atkeEZsBJG0BlkZESNpM6ogr3omIfcC+fAdiAmlo\n3TciYj8pwcoK4BLSELKfR8T2GucU8IRSRrsDpLSslRSs2yPiizy/DujM4+KfHhELc/3/yvFO6kd9\nrQTc8ZtZM+0tzAtYEhHTau0saTRwP3BJROyR9BrpG21f5gNTSeO1v1k435MR8cIhxPt3Yf5AYfkA\n///7WT32eV9joe/tZdstpA8sF0fEP5J20F3nYjz7SXcZaulPfa0E/IzfzFplNXCZpDFwMKvbuVX7\nDCF1kr9JGkVKpVrxB3BijWO/ScpsNpXufOsfAXdW3iOQdLqkkxtSE5giaZCk4aRb+WuAlcBNkgZI\nGglcAXzeQ9nqepwE/Jw7/auAs3o7cUT8AXwv6VoASQPz/3Y4kvW1o5i/8ZtZS0TEL5LuAN6QNDCv\nfoSUmayyz0ZJG4CtpGfnqwqHeBH4UNKPEXFV1bG35FvgP0ROAxwRiyWdD3yaHy38CUznMPKaF2wC\nlpGeyT8eET9KWghcSnr+H8CDEfGTpPOqYt2dXxjsAj4AngLezY8T1ua69+VW4AVJjwH/ADcc4fra\nUczZ+czMDoOkWcCfEfFMq2Mxq4dv9ZuZmZWIv/GbmZmViL/xm5mZlYg7fjMzsxJxx29mZlYi7vjN\nzMxKxB2/mZlZifwHS9ro7WZdguYAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "importance = model.get_fscore()\n", "importance = sorted(importance.items(), key=operator.itemgetter(1))\n", "\n", "df = pd.DataFrame(importance, columns=['feature', 'fscore'])\n", "df['fscore'] = df['fscore'] / df['fscore'].sum()\n", "\n", "df.plot(kind='barh', x='feature', y='fscore', legend=False, figsize=(6, 10))\n", "plt.title('XGBoost Feature Importance')\n", "plt.xlabel('relative importance');" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## End" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.0" } }, "nbformat": 4, "nbformat_minor": 2 }