{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%reload_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from fastai.tabular import *" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Rossmann" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data preparation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To create the feature-engineered train_clean and test_clean from the Kaggle competition data, run `rossman_data_clean.ipynb`. One important step that deals with time series is this:\n", "\n", "```python\n", "add_datepart(train, \"Date\", drop=False)\n", "add_datepart(test, \"Date\", drop=False)\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "path = Config().data_path()/'rossmann'\n", "train_df = pd.read_pickle(path/'train_clean')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
01234
index01234
Store12345
DayOfWeek55555
Date2015-07-312015-07-312015-07-312015-07-312015-07-31
Sales526360648314139954822
Customers5556258211498559
Open11111
Promo11111
StateHolidayFalseFalseFalseFalseFalse
SchoolHoliday11111
Year20152015201520152015
Month77777
Week3131313131
Day3131313131
Dayofweek44444
Dayofyear212212212212212
Is_month_endTrueTrueTrueTrueTrue
Is_month_startFalseFalseFalseFalseFalse
Is_quarter_endFalseFalseFalseFalseFalse
Is_quarter_startFalseFalseFalseFalseFalse
Is_year_endFalseFalseFalseFalseFalse
Is_year_startFalseFalseFalseFalseFalse
Elapsed14383008001438300800143830080014383008001438300800
StoreTypecaaca
Assortmentaaaca
CompetitionDistance12705701413062029910
CompetitionOpenSinceMonth9111294
CompetitionOpenSinceYear20082007200620092015
Promo201100
Promo2SinceWeek1131411
..................
Min_Sea_Level_PressurehPa10151017101710141016
Max_VisibilityKm3110311010
Mean_VisibilityKm1510141010
Min_VisibilitykM1010101010
Max_Wind_SpeedKm_h2414142314
Mean_Wind_SpeedKm_h111151611
Max_Gust_SpeedKm_hNaNNaNNaNNaNNaN
Precipitationmm00000
CloudCover14264
EventsFogFogFogNaNNaN
WindDirDegrees13309354282290
StateNameHessenThueringenNordrheinWestfalenBerlinSachsen
CompetitionOpenSince2008-09-152007-11-152006-12-152009-09-152015-04-15
CompetitionDaysOpen2510281531502145107
CompetitionMonthsOpen242424243
Promo2Since1900-01-012010-03-292011-04-041900-01-011900-01-01
Promo2Days01950157900
Promo2Weeks0252500
AfterSchoolHoliday00000
BeforeSchoolHoliday00000
AfterStateHoliday5767576757
BeforeStateHoliday00000
AfterPromo00000
BeforePromo00000
SchoolHoliday_bw55555
StateHoliday_bw00000
Promo_bw55555
SchoolHoliday_fw71511
StateHoliday_fw00000
Promo_fw51511
\n", "

93 rows × 5 columns

\n", "
" ], "text/plain": [ " 0 1 2 \\\n", "index 0 1 2 \n", "Store 1 2 3 \n", "DayOfWeek 5 5 5 \n", "Date 2015-07-31 2015-07-31 2015-07-31 \n", "Sales 5263 6064 8314 \n", "Customers 555 625 821 \n", "Open 1 1 1 \n", "Promo 1 1 1 \n", "StateHoliday False False False \n", "SchoolHoliday 1 1 1 \n", "Year 2015 2015 2015 \n", "Month 7 7 7 \n", "Week 31 31 31 \n", "Day 31 31 31 \n", "Dayofweek 4 4 4 \n", "Dayofyear 212 212 212 \n", "Is_month_end True True True \n", "Is_month_start False False False \n", "Is_quarter_end False False False \n", "Is_quarter_start False False False \n", "Is_year_end False False False \n", "Is_year_start False False False \n", "Elapsed 1438300800 1438300800 1438300800 \n", "StoreType c a a \n", "Assortment a a a \n", "CompetitionDistance 1270 570 14130 \n", "CompetitionOpenSinceMonth 9 11 12 \n", "CompetitionOpenSinceYear 2008 2007 2006 \n", "Promo2 0 1 1 \n", "Promo2SinceWeek 1 13 14 \n", "... ... ... ... \n", "Min_Sea_Level_PressurehPa 1015 1017 1017 \n", "Max_VisibilityKm 31 10 31 \n", "Mean_VisibilityKm 15 10 14 \n", "Min_VisibilitykM 10 10 10 \n", "Max_Wind_SpeedKm_h 24 14 14 \n", "Mean_Wind_SpeedKm_h 11 11 5 \n", "Max_Gust_SpeedKm_h NaN NaN NaN \n", "Precipitationmm 0 0 0 \n", "CloudCover 1 4 2 \n", "Events Fog Fog Fog \n", "WindDirDegrees 13 309 354 \n", "StateName Hessen Thueringen NordrheinWestfalen \n", "CompetitionOpenSince 2008-09-15 2007-11-15 2006-12-15 \n", "CompetitionDaysOpen 2510 2815 3150 \n", "CompetitionMonthsOpen 24 24 24 \n", "Promo2Since 1900-01-01 2010-03-29 2011-04-04 \n", "Promo2Days 0 1950 1579 \n", "Promo2Weeks 0 25 25 \n", "AfterSchoolHoliday 0 0 0 \n", "BeforeSchoolHoliday 0 0 0 \n", "AfterStateHoliday 57 67 57 \n", "BeforeStateHoliday 0 0 0 \n", "AfterPromo 0 0 0 \n", "BeforePromo 0 0 0 \n", "SchoolHoliday_bw 5 5 5 \n", "StateHoliday_bw 0 0 0 \n", "Promo_bw 5 5 5 \n", "SchoolHoliday_fw 7 1 5 \n", "StateHoliday_fw 0 0 0 \n", "Promo_fw 5 1 5 \n", "\n", " 3 4 \n", "index 3 4 \n", "Store 4 5 \n", "DayOfWeek 5 5 \n", "Date 2015-07-31 2015-07-31 \n", "Sales 13995 4822 \n", "Customers 1498 559 \n", "Open 1 1 \n", "Promo 1 1 \n", "StateHoliday False False \n", "SchoolHoliday 1 1 \n", "Year 2015 2015 \n", "Month 7 7 \n", "Week 31 31 \n", "Day 31 31 \n", "Dayofweek 4 4 \n", "Dayofyear 212 212 \n", "Is_month_end True True \n", "Is_month_start False False \n", "Is_quarter_end False False \n", "Is_quarter_start False False \n", "Is_year_end False False \n", "Is_year_start False False \n", "Elapsed 1438300800 1438300800 \n", "StoreType c a \n", "Assortment c a \n", "CompetitionDistance 620 29910 \n", "CompetitionOpenSinceMonth 9 4 \n", "CompetitionOpenSinceYear 2009 2015 \n", "Promo2 0 0 \n", "Promo2SinceWeek 1 1 \n", "... ... ... \n", "Min_Sea_Level_PressurehPa 1014 1016 \n", "Max_VisibilityKm 10 10 \n", "Mean_VisibilityKm 10 10 \n", "Min_VisibilitykM 10 10 \n", "Max_Wind_SpeedKm_h 23 14 \n", "Mean_Wind_SpeedKm_h 16 11 \n", "Max_Gust_SpeedKm_h NaN NaN \n", "Precipitationmm 0 0 \n", "CloudCover 6 4 \n", "Events NaN NaN \n", "WindDirDegrees 282 290 \n", "StateName Berlin Sachsen \n", "CompetitionOpenSince 2009-09-15 2015-04-15 \n", "CompetitionDaysOpen 2145 107 \n", "CompetitionMonthsOpen 24 3 \n", "Promo2Since 1900-01-01 1900-01-01 \n", "Promo2Days 0 0 \n", "Promo2Weeks 0 0 \n", "AfterSchoolHoliday 0 0 \n", "BeforeSchoolHoliday 0 0 \n", "AfterStateHoliday 67 57 \n", "BeforeStateHoliday 0 0 \n", "AfterPromo 0 0 \n", "BeforePromo 0 0 \n", "SchoolHoliday_bw 5 5 \n", "StateHoliday_bw 0 0 \n", "Promo_bw 5 5 \n", "SchoolHoliday_fw 1 1 \n", "StateHoliday_fw 0 0 \n", "Promo_fw 1 1 \n", "\n", "[93 rows x 5 columns]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_df.head().T" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "844338" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "n = len(train_df); n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Experimenting with a sample" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "idx = np.random.permutation(range(n))[:2000]\n", "idx.sort()\n", "small_train_df = train_df.iloc[idx[:1000]]\n", "small_test_df = train_df.iloc[idx[1000:]]\n", "small_cont_vars = ['CompetitionDistance', 'Mean_Humidity']\n", "small_cat_vars = ['Store', 'DayOfWeek', 'PromoInterval']\n", "small_train_df = small_train_df[small_cat_vars + small_cont_vars + ['Sales']]\n", "small_test_df = small_test_df[small_cat_vars + small_cont_vars + ['Sales']]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreDayOfWeekPromoIntervalCompetitionDistanceMean_HumiditySales
2672685NaN4520.0677492
6046065NaN2260.0617187
9839865Feb,May,Aug,Nov620.0617051
16365254NaN1870.0559673
23481233NaN16760.05010007
\n", "
" ], "text/plain": [ " Store DayOfWeek PromoInterval CompetitionDistance Mean_Humidity \\\n", "267 268 5 NaN 4520.0 67 \n", "604 606 5 NaN 2260.0 61 \n", "983 986 5 Feb,May,Aug,Nov 620.0 61 \n", "1636 525 4 NaN 1870.0 55 \n", "2348 123 3 NaN 16760.0 50 \n", "\n", " Sales \n", "267 7492 \n", "604 7187 \n", "983 7051 \n", "1636 9673 \n", "2348 10007 " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "small_train_df.head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreDayOfWeekPromoIntervalCompetitionDistanceMean_HumiditySales
4205108293NaN110.0556802
4206549733Jan,Apr,Jul,Oct330.0596644
4209901942Feb,May,Aug,Nov16970.0554720
4213085122Mar,Jun,Sept,Dec590.0726248
42182410292NaN1590.0648004
\n", "
" ], "text/plain": [ " Store DayOfWeek PromoInterval CompetitionDistance \\\n", "420510 829 3 NaN 110.0 \n", "420654 973 3 Jan,Apr,Jul,Oct 330.0 \n", "420990 194 2 Feb,May,Aug,Nov 16970.0 \n", "421308 512 2 Mar,Jun,Sept,Dec 590.0 \n", "421824 1029 2 NaN 1590.0 \n", "\n", " Mean_Humidity Sales \n", "420510 55 6802 \n", "420654 59 6644 \n", "420990 55 4720 \n", "421308 72 6248 \n", "421824 64 8004 " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "small_test_df.head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "categorify = Categorify(small_cat_vars, small_cont_vars)\n", "categorify(small_train_df)\n", "categorify(small_test_df, test=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreDayOfWeekPromoIntervalCompetitionDistanceMean_HumiditySales
420510NaN3NaN110.0556802
420654973.03Jan,Apr,Jul,Oct330.0596644
420990NaN2Feb,May,Aug,Nov16970.0554720
421308512.02Mar,Jun,Sept,Dec590.0726248
4218241029.02NaN1590.0648004
\n", "
" ], "text/plain": [ " Store DayOfWeek PromoInterval CompetitionDistance \\\n", "420510 NaN 3 NaN 110.0 \n", "420654 973.0 3 Jan,Apr,Jul,Oct 330.0 \n", "420990 NaN 2 Feb,May,Aug,Nov 16970.0 \n", "421308 512.0 2 Mar,Jun,Sept,Dec 590.0 \n", "421824 1029.0 2 NaN 1590.0 \n", "\n", " Mean_Humidity Sales \n", "420510 55 6802 \n", "420654 59 6644 \n", "420990 55 4720 \n", "421308 72 6248 \n", "421824 64 8004 " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "small_test_df.head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Index(['Feb,May,Aug,Nov', 'Jan,Apr,Jul,Oct', 'Mar,Jun,Sept,Dec'], dtype='object')" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "small_train_df.PromoInterval.cat.categories" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "267 -1\n", "604 -1\n", "983 0\n", "1636 -1\n", "2348 -1\n", "dtype: int8" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "small_train_df['PromoInterval'].cat.codes[:5]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fill_missing = FillMissing(small_cat_vars, small_cont_vars)\n", "fill_missing(small_train_df)\n", "fill_missing(small_test_df, test=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StoreDayOfWeekPromoIntervalCompetitionDistanceMean_HumiditySalesCompetitionDistance_na
1857496222NaN2300.0934508True
\n", "
" ], "text/plain": [ " Store DayOfWeek PromoInterval CompetitionDistance Mean_Humidity \\\n", "185749 622 2 NaN 2300.0 93 \n", "\n", " Sales CompetitionDistance_na \n", "185749 4508 True " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "small_train_df[small_train_df['CompetitionDistance_na'] == True]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Preparing full data set" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "train_df = pd.read_pickle(path/'train_clean')\n", "test_df = pd.read_pickle(path/'test_clean')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(844338, 41088)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(train_df),len(test_df)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "procs=[FillMissing, Categorify, Normalize]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "cat_vars = ['Store', 'DayOfWeek', 'Year', 'Month', 'Day', 'StateHoliday', 'CompetitionMonthsOpen',\n", " 'Promo2Weeks', 'StoreType', 'Assortment', 'PromoInterval', 'CompetitionOpenSinceYear', 'Promo2SinceYear',\n", " 'State', 'Week', 'Events', 'Promo_fw', 'Promo_bw', 'StateHoliday_fw', 'StateHoliday_bw',\n", " 'SchoolHoliday_fw', 'SchoolHoliday_bw']\n", "\n", "cont_vars = ['CompetitionDistance', 'Max_TemperatureC', 'Mean_TemperatureC', 'Min_TemperatureC',\n", " 'Max_Humidity', 'Mean_Humidity', 'Min_Humidity', 'Max_Wind_SpeedKm_h', \n", " 'Mean_Wind_SpeedKm_h', 'CloudCover', 'trend', 'trend_DE',\n", " 'AfterStateHoliday', 'BeforeStateHoliday', 'Promo', 'SchoolHoliday']" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "dep_var = 'Sales'\n", "df = train_df[cat_vars + cont_vars + [dep_var,'Date']].copy()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "('2015-08-01', '2015-09-17')" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_df['Date'].min(), test_df['Date'].max()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "41395" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cut = train_df['Date'][(train_df['Date'] == train_df['Date'][len(test_df)])].index.max()\n", "cut" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "valid_idx = range(cut)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0 5263\n", "1 6064\n", "2 8314\n", "3 13995\n", "4 4822\n", "Name: Sales, dtype: int64" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df[dep_var].head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "data = (TabularList.from_df(df, path=path, cat_names=cat_vars, cont_names=cont_vars, procs=procs,)\n", " .split_by_idx(valid_idx)\n", " .label_from_df(cols=dep_var, label_cls=FloatList, log=True)\n", " .add_test(TabularList.from_df(test_df, path=path, cat_names=cat_vars, cont_names=cont_vars))\n", " .databunch())" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "doc(FloatList)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Model" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "max_log_y = np.log(np.max(train_df['Sales'])*1.2)\n", "y_range = torch.tensor([0, max_log_y], device=defaults.device)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = tabular_learner(data, layers=[1000,500], ps=[0.001,0.01], emb_drop=0.04, \n", " y_range=y_range, metrics=exp_rmspe)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "TabularModel(\n", " (embeds): ModuleList(\n", " (0): Embedding(1116, 81)\n", " (1): Embedding(8, 5)\n", " (2): Embedding(4, 3)\n", " (3): Embedding(13, 7)\n", " (4): Embedding(32, 11)\n", " (5): Embedding(3, 3)\n", " (6): Embedding(26, 10)\n", " (7): Embedding(27, 10)\n", " (8): Embedding(5, 4)\n", " (9): Embedding(4, 3)\n", " (10): Embedding(4, 3)\n", " (11): Embedding(24, 9)\n", " (12): Embedding(9, 5)\n", " (13): Embedding(13, 7)\n", " (14): Embedding(53, 15)\n", " (15): Embedding(22, 9)\n", " (16): Embedding(7, 5)\n", " (17): Embedding(7, 5)\n", " (18): Embedding(4, 3)\n", " (19): Embedding(4, 3)\n", " (20): Embedding(9, 5)\n", " (21): Embedding(9, 5)\n", " (22): Embedding(3, 3)\n", " (23): Embedding(3, 3)\n", " )\n", " (emb_drop): Dropout(p=0.04)\n", " (bn_cont): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (layers): Sequential(\n", " (0): Linear(in_features=233, out_features=1000, bias=True)\n", " (1): ReLU(inplace)\n", " (2): BatchNorm1d(1000, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (3): Dropout(p=0.001)\n", " (4): Linear(in_features=1000, out_features=500, bias=True)\n", " (5): ReLU(inplace)\n", " (6): BatchNorm1d(500, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", " (7): Dropout(p=0.01)\n", " (8): Linear(in_features=500, out_features=1, bias=True)\n", " )\n", ")" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.model" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "16" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(data.train_ds.cont_names)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n" ] } ], "source": [ "learn.lr_find()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAELCAYAAADURYGZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd8XNWd9/HPTxr1alnFsuXeC67CVJtiCBAglAAJgSwJbEiyG9gku5vdZ/Nsks1usklITzbFQIDNJmSpT2gxGAi2ARtbNrjj3uSiaku2ujTn+WPGIBxZzTNzp3zfr9e8PHPnju5X45F+uuece4455xARkcSV5HUAERHxlgqBiEiCUyEQEUlwKgQiIglOhUBEJMGpEIiIJLiwFQIz+42ZVZvZpm7bbjazzWbmN7PycB1bRET6L5xnBA8DV56ybRNwI7A8jMcVEZEB8IXrCzvnlpvZmFO2bQUws3AdVkREBkh9BCIiCS5sZwRnyszuBu4GyMrKmjdlyhSPE4mIxJa1a9fWOueK+tovaguBc24xsBigvLzcVVRUeJxIRCS2mNm+/uynpiERkQQXzuGjjwIrgclmVmlmd5nZDWZWCZwHPG9mL4br+CIi0j/hHDV062meejpcxxQRkYFT05CISIJTIRARSXAqBCIiCU6FQEQkwakQiIhEoeb2Tv7t2c3sq2sK+7FUCEREotDzGw7z0Bt7qWpsC/uxVAhERKLQYxUHGFeYxdljhoT9WCoEIiJRZlfNCdbsPcotZ4+MyGzNKgQiIlHmsTUHSE4ybpw7IiLHUyEQEYkiHV1+nlxXyaVTiinOSY/IMVUIRESiyKvvVlN7op2Pnz0yYsdUIRARiSL/u+YAxTlpXDSpz2UEQkaFQEQkShxpaOW1bdXcNK8MX3Lkfj2rEIiIRIkn1h7A7+CW8sg1C4EKgYhIVHDO8cTaSs4dV8CYwqyIHluFQEQkCmw62Mjeumaunx2ZIaPdqRCIiESB5zYcwpdkXDljWMSPrUIgIuIx5xzPbTjMgomF5GemRvz4KgQiIh5bt/8YB4+1cM3M4Z4cX4VARMRjz204RGpyEpdPL/Hk+CoEIiIe6vI7nt9wmIsmF5GbnuJJBhUCEREPrdlbT/XxNq6d5U2zEKgQiIh46rkNh0hPSWLRlGLPMqgQiIh4pLPLz582HmHRlBKy0nye5QhbITCz35hZtZlt6ratwMyWmtmO4L/hX3pHRCRKrdxdR11TO9fMLPU0RzjPCB4Grjxl2z8DrzjnJgKvBB+LiCSkl7dUkZ6SxCUeNgtBGAuBc245UH/K5uuAR4L3HwGuD9fxRUSi3YodtZw7bijpKcme5oh0H0GJc+4wQPBfb8ugiIhHDtQ3s7u2iQUTI7fuwOlEbWexmd1tZhVmVlFTU+N1HBGRkHp9Zy0ACycWepwk8oWgysxKAYL/Vp9uR+fcYudcuXOuvKjI+4opIhJKK3bUMCw3nQnF2V5HiXgheAa4I3j/DuCPET6+iIjnuvyO13fUsmBiIWbmdZywDh99FFgJTDazSjO7C/gOcLmZ7QAuDz4WEUkoGyqP0djayYIIrkvcm7BdweCcu/U0Ty0K1zFFRGLBih21mMGFE7zvH4Ao7iwWEYlXK3bUMGN4HgVZkV97oCcqBCIiEXS8tYN1+4+xIApGC52kQiAiEkErd9XR5XdRcf3ASSoEIiIRtGJHLZmpycwdne91lPeoEIiIRNCKHTWcO24oaT5vp5XoToVARCRCdlQdZ29dMxdFybDRk1QIREQi5LGKA/iSzPNpp0+lQiAiEgHtnX6eWneQy6aWMDQ7zes4H6BCICISAa++W0VdUzsfO3uk11H+ggqBiEgE/O+aAwzLTWdhlPUPgAqBiEjYHWloZdn2Gm6aV0ZykveTzJ1KhUBEJMyeWHsAv4NbyqOvWQhUCEREwsrvdzxWUcl544Yyamim13F6pEIgIhJGq/bUsb++OSo7iU9SIRARCaP/XXOAnHQfV84Y5nWU01IhEBEJk/qmdv608Qg3zBlBekr0TClxKhUCEZEwebziAO1dfm4/d7TXUXqlQiAiEgZ+v+P3q/czf0wBk0pyvI7TKxUCEZEweH1nLfvqmrnt3FFeR+mTCoGISBj87q19FGSlRnUn8UkqBCIiIXakoZWXt1Zzc3lZVK07cDoqBCIiIfaHNfvp8jtumx/dncQnqRCIiIRQZ5efP6w+wMJJRVF7JfGpVAhERELoz9tqONLYym3nRH8n8UmeFAIz+zsz22Rmm83si15kEBEJh2fWH2JIZgqXTin2Okq/RbwQmNkM4DPAfGAWcI2ZTYx0DhGRUGtp7+KVrVVcOaOUlOTYaXDxIulUYJVzrtk51wksA27wIIeISEi9+m41ze1dXBtlaxL3xYtCsAlYaGZDzSwT+DAQvdPyiYj003MbDlGYncY544Z6HWVAfJE+oHNuq5l9F1gKnADWA52n7mdmdwN3A4waFTudLiKSmE60dfLqu9V87OyRUbkKWW88acRyzj3onJvrnFsI1AM7ethnsXOu3DlXXlQUfWt8ioh098rWKto6/Vwzc7jXUQYs4mcEAGZW7JyrNrNRwI3AeV7kEBEJlWfXH2ZYbjrlo4d4HWXAPCkEwJNmNhToAP7WOXfUoxwiImesoaWD5dtr+OR5o0mKsWYh8KgQOOcWeHFcEZFwWLqlivYuP9fE2Gihk2JnoKuISJR6bsMhyoZkMHtkvtdRBkWFQETkDNSdaOP1HbVcPbMUs9hrFgIVAhGRM/LM+kN0+h03zinzOsqgqRCIiJyBp9YdZMaIXCYPi+7lKHujQiAiMkjbq46z8WBDTJ8NgAqBiMigPbmuEl+S8ZHZsXcRWXcqBCIig9Dld/y/tw9y8eQiCrPTvI5zRlQIREQG4c1dtVQ1tnHj3NhuFgIVAhGRQXlq3UFy030smho7C9CcjgqBiMgAnWjrZMmmI1w7azhpvmSv45wxFQIRkQH608bDtHR0xUWzEKgQiIgM2DPrDzF6aCZzR8XmlBKnUiEQERmAo03tvLmrjg+fFbtTSpxKhUBEZACWbqmiy++4+qzYnGm0JyoEIiID8MKmw4wsyGD68Fyvo4SMCoGISD81NHfwxs7auGoWAhUCEZF+W7q1io4ux4dnxE+zEKgQiIj02wsbDzMiP4OZZXleRwkpFQIRkX5obO1gxY4aPnzWsLhqFgIVAhGRfnl5S6BZ6Ko4Gi10kgqBiEg/vLDxCMPz0pkTo+sS90aFQESkD8dbO1i+o4YrZ8TXaKGTVAhERPqwclcd7Z1+PjS9xOsoYaFCICLSh1W760nzJTEnTuYWOpUnhcDMvmRmm81sk5k9ambpXuQQEemPt/bUMXfUkLiYcronES8EZjYCuBcod87NAJKBj0c6h4hIfzS0dLDlcCPnjCvwOkrYeNU05AMyzMwHZAKHPMohItKrNXvqcQ7OHTfU6yhhE/FC4Jw7CHwf2A8cBhqccy9FOoeISH+8taeOVF8Ss+Nw2OhJXjQNDQGuA8YCw4EsM7u9h/3uNrMKM6uoqamJdEwRESDQUTx7ZD7pKfHZPwD9LARmNt7M0oL3Lzaze81ssOXxMmCPc67GOdcBPAWcf+pOzrnFzrly51x5UVHRIA8lIjJ4ja0dbD7UENfNQtD/M4IngS4zmwA8SOCv+d8P8pj7gXPNLNMCV2YsArYO8muJiIRNxd56/A7OHRu/HcXQ/0Lgd851AjcAP3bOfQkY1IQbzrm3gCeAdcDGYIbFg/laIiLh9NbuelKTk5gzaojXUcLK18/9OszsVuAO4NrgtpTBHtQ593Xg64N9vYhIJKzaU8+skXlkpMZv/wD0/4zg08B5wLecc3vMbCzwP+GLJSLirRNtnWw62MA5Y+O7fwD6eUbgnNtC4CKwk6N+cpxz3wlnMBERL1XsrafL7+K+oxj6P2roNTPLNbMCYD3wkJn9MLzRRES8s2p3Pb4kY+7o+L1+4KT+Ng3lOecagRuBh5xz8wgMAxURiUurdtcxsyyPzNT+dqXGrv4WAp+ZlQK3AM+FMY+IiOcamjvYUHmMCyYUeh0lIvpbCL4JvAjscs6tMbNxwI7wxRIR8c7rO2vxO7hoUmJczNrfzuLHgce7Pd4NfDRcoUREvLRsezU56b64nl+ou/52FpeZ2dNmVm1mVWb2pJmVhTuciEikOedYvr2WCycU4ktOjLW7+vtdPgQ8Q2CSuBHAs8FtIiJxZXvVCY40tiZMsxD0vxAUOececs51Bm8PA4nzLolIwli+PTDb8UIVgr9Qa2a3m1ly8HY7UBfOYCIiXli2vYaJxdkMz8/wOkrE9LcQ3Elg6OgRAovJ3ERg2gkRkbjR3N7J6j31CdUsBP0sBM65/c65jzjnipxzxc656wlcXCYiEjfe2l1Pe5c/oZqF4MxWKPtyyFKIiESBZdtrSE9JYn6crz9wqjMpBBayFCIiUWD59hrOGTs0rpel7MmZFAIXshQiIh47UN/M7tqmhOsfgD6uLDaz4/T8C9+AxOlSF5G4tywBh42e1GshcM7lRCqIiIiXlm2vYUR+BuOLsryOEnGJcf20iEgv2jv9rNxVx0WTizBLvO5PFQIRSXjr9h/lRFtnQvYPgAqBiAjLttfgSzLOHx//y1L2RIVARBLesm01zBs9hJz0FK+jeEKFQEQSWnVjK1sON3LR5MRsFgIVAhFJcMt31AKJsxpZTyJeCMxsspm90+3WaGZfjHQOEREI9A8UZqcxdViu11E806+lKkPJObcNmA1gZsnAQeDpSOcQEenyO17fUcMlU4pJSkq8YaMned00tAjY5Zzb53EOEUlAGw82cLS5I6GbhcD7QvBx4FGPM4hIglq2rQYzWDBRhcATZpYKfAR4/DTP321mFWZWUVNTE9lwIpIQlm2vZmZZPgVZqV5H8ZSXZwRXAeucc1U9PemcW+ycK3fOlRcVJXa1FpHQO9bczjsHjnHRxEKvo3jOy0JwK2oWEhGPLNteg9/BJVOKvY7iOU8KgZllApcDT3lxfBGRl7dWU5idyqyyfK+jeC7iw0cBnHPNQGJO6iEinuvo8rNsWzVXTB+W0MNGT/J61JCISMSt3XeUxtZOFk1VsxCoEIhIAnplaxWpyUlcmODDRk9SIRCRhPPKu9WcM66A7DRPWsejjgqBiCSUPbVN7K5pYpFGC71HhUBEEsorWwOXLi2aWuJxkuihQiAiCeXVd6uZVJLNyIJMr6NEjYRqIOvyOxpaOjjW3M6xlg4aWjpobuuiqb2T5rZOhmansXBiEXmZiblKkUi8a2ztYPWeev56wTivo0SVhCgER5vaefjNvTyyci/Hmjt63Tc5ySgfPYSLJhfh9zsqj7ZQebQFh+P2c0aHZNyxc46jzR1UHm2m5ngbDS0dNLZ0cKKtk7yMFIpz0ynJTSc9JYn9dc3srw/cstN8zBiRx4zheYwsyMBM459FBmL59ho6/U7DRk8R14Xg0LEWHlixh0dX76elo4vLphZz4YRC8jNTyctMIS8jhaxUH5mpyWSmJrO3rolXtlbz6rvVfG/JNgAKs1MZMSSTo03tfP536xhXlMXnLhrPuWOH0tzRSVNbF01tnRxpbOVIQyuHG1qpbmylrqmdo83t1De1Y0Bmqo+M1GSSDA43tNLc3jWg7yUnzUdLRxedfhd4nO5jQnE24wqzGV+cxbjCLEYPzWL00EwyUz/43+qcU9EQAV7ZWk1+ZgpzRw3xOkpUietC8MOl23n67YNcN3s4n7toPJNKcnrdf2h2GvNGF/CVK6dQd6LtvV/eEGhWemHjYX7x2i6+8sSG036NwuxUinLSKcxOZfTQTIZkBmY1bG7vpLm9iy6/46JJxZQNyaBsSAYluenkZaSQm5FCdpqPhpYOqhpbqT7eSlNbFyMLMhldkEl+ZgptnX62Vx1n08FGNh9qYHdNEyt21PDkuspTMqThSzKa2ztp6ejCOZhUksNZI/KYUZbHzBF5TCnNIc2X/IHX+f2Olo4usjSkTuJQR5efV7ZWcdm0EpJ1NfEHmHPO6wx9Ki8vdxUVFQN+3cFjLTjnKBsSuk4h5xxv7KzjSGPre2cSWWk+SnLSKc5NIz0lue8vEmLHWzvYW9vMvvom9tU1c6C+GecgIzWZjNRkuvyOrYcb2RRchAMgJdmYWprLtNJcjjV3sKe2ib11TbR1+slJ81Gan05pXgZjC7OYWprD1NJcJpXkePL9iYTCih01fPLB1dz/V+VcPi0xRgyZ2VrnXHlf+8X1n34j8jNC/jXNjAujbNranPQUzirL46yyvF73c85x8FgLGysbWF/ZwPoDx1iy+QgFWamMK8xi4aRChmSlUt3YxqFjLRxqaGHN3vr3mrGSk4xJJTnMGZXP7JH5TC7JIT/YxJaTnqK/siSqLdl0hMzUZBZE2c9vNIjrQiAfZGaUDcmkbEgmV51V2q/X+P2O/fXNbD3cyOZDjayvPMaz6w/x+7f2/8W+WcGzo6w0H9lpvveKRH5mCiPyM5k+PJdpw3MpzE4L9bcm0qsuv+PFzVVcMrlYZ7U9UCGQXiUlGWMKsxhTmPVe8fD7HbtrT7CntpmG4DDchpYOTrR20tzeSVN7F8dbA9sOHm3haHP7e01SACW5aUwrzWVKaS5TS3MZX5RFQVYq+RmppKckqWNbQu7t/UepPdHGFTOGeR0lKqkQyIAlJRkTinOYUNx753t3Dc0dbD7cwJZDgTOLrYcbWbGj9r1RUCel+pIozUtnzNAsxhYGRkENy02nKCeN4pxAx7ov2fAlGylJSZpCWPplyaYjpCYncclkTTLXExUCiYi8zBTOH1/I+ePfb59t7/Szs/oE++qaONbSwbHmwMV+lcda2FvbRMXeepr6GGZbnJPGuKIsxhVlM64wi4klOUwqyWZYbrrOLAQI9I0t2XyECycWkpOui0V7okIgnkn1JTEt2G/QE+ccdU3tVDe2UXOijerGVhpbO+ns8tPpd7R1+jl4tIU9tSd4YePhD1wsmJ3mY1ppLvPGDKF89BDmjR5CfmZiL1CeqLYcbqTyaAv3XjrR6yhRS4VAopaZUZid1u/O5boTbeyoPhG4VR1nfWUD9y/fzS+DzU+jh2YyY3ge00fkBv4dnstQdVzHvRc3HSHJ0NXEvVAhkLgxNDuNodlpnDvu/VVQW9q7WF95jLX7jrLpYAMbDh7j+Y2H33u+NC+d6cMD10iMLQw0MY0vytLZQxxZsvkI88cWqOj3QoVA4lpGajLnjhv6geLQ0NzB5kMNbD4UuEJ706FGXttW84GO62mluVw8uYiLJxczd1Q+vmRN1BuLdtecYHvVCb5x7TSvo0Q1FQJJOHmZKZw/oZDzJ7zfcd3R5acy2N+w9fBxlm2v4dfLd/OL13aRkZLM9OG5nFWWx6yyfC6YUEhRjv66jAUvB9ceuHy6ho32Jq6nmBA5E42tHbyxo5bVe+vZWNnApkMNtHb4STKYP7aAq88q5YoZwyjOSfc6qpzGJ+5fRd2Jdl780kKvo3hCU0yInKHc9BSuOqv0vQvpOrv8vHvkOC9tqeKFjYf51z9u5mvPbGZWWT6XTyth0dRiJpfkaNhqlDjR1smavfXceeFYr6NEPRUCkX7yJScF1oMYkceXL5/E9qrjLNl0hFe2VnHfi9u478VtlA3J4LKpJXxoWglnjy0gRX0LnnlzZy0dXY6LJ2m0UF88KQRmlg88AMwAHHCnc26lF1lEBmtSSQ6TSnK4d9FEqhpbeWVrNS9vreL3q/fz8Jt7yctI4db5o7jrwrHqU/DAn7fVkJ3mo3yM1h7oi1dnBD8BljjnbjKzVECLh0pMK8lN5xPnjOIT54yiub2T5dtreXb9IRYv38VDb+zhY2eP5O6F40I6JbqcnnOOZduquWDCUJ2V9UPEC4GZ5QILgU8BOOfagfZI5xAJl8xUH1fOGMaVM4axp7aJX722i0dX7+fR1fu547wxfOHSCbpOIcx2VJ/gUEMr9y7S1cT94UWpHAfUAA+Z2dtm9oCZZXmQQyTsxhZm8d2bZrLsHy/hhjkjePCNPVx032s8sGI37Z1+r+PFrT+/Ww3ARZpkrl+8KAQ+YC7wS+fcHKAJ+OdTdzKzu82swswqampqIp1RJKSG52fwvZtm8cK9C5hZlsd/PL+VK3+8nGXb9dkOh9e21TBlWA6leaFfnCoeeVEIKoFK59xbwcdPECgMH+CcW+ycK3fOlRcVqapLfJhamstv7zqHhz59Ng644zerufu/KzhQ3+x1tLhxvLWDin31OhsYgIgXAufcEeCAmU0ObloEbIl0DhEvXTK5mCVfXMA/XTmF13fWctkPl/Fff96p5qIQeGNnHR1djksma9hof3nVnX4P8Dsz2wDMBr7tUQ4Rz6T5kvn8xeN55e8vYtHUYu57cRtX/3QFa/bWex0tpi3bXk1Omo95ozVstL88KQTOuXeCzT4znXPXO+eOepFDJBqU5mXwi9vm8eAd5TS3d3Hzr1byL09v5ERbp9fRYo7f7/jzuzVcMKFQw0YHQO+USJRYNLWEpV9eyGcWjOXR1fu56ifLWb1HZwcDsWpPHUcaW7nqLE0yNxAqBCJRJDPVx1evnsbjnz2PJDM+tngl335hK60dvS/ZKQFPrj1ITpqPD01TIRgIFQKRKFQ+poAX7l3AbeeMYvHy3Xzywbdo6LYUp/ylprZO/rTpMFfPLCUjNdnrODFFhUAkSmWl+fiP68/i55+Yw/oDDdz86zc5dKzF61hRa8mmIzS3d/HReWVeR4k5KgQiUe6amcN5+M6zOXyslRt/8Sbbq457HSkqPbG2ktFDMynXaKEBUyEQiQHnjy/ksc+dh985bvrlm1RoiOkHVB5tZuXuOj46t0zrQQyCCoFIjJhamstTf3M+Q7PTuP3Bt3htW7XXkaLG0+sOAnDDnBEeJ4lNKgQiMaRsSCaPf+48xhVm89ePVPDM+kNeR/Kcc44n11Vy3rihjCzQNN+DoUIgEmMKs9P4w2fPZe6oIfzdH97md2/t8zqSp9buO8reumZ1Ep8BFQKRGJSbnsIjd87nksnFfPXpTTywYrfXkTzzhzUHyExN5qoZunZgsFQIRGJURmoyv7p9HlfNGMZ/PL+Vn7+6w+tIEVfd2Mof3znIzfPKyErTEuyDpUIgEsNSfUn87NY53DBnBN9/aTv3vfguzjmvY0XMIyv30ul33HnhWK+jxDSVUJEY50tO4gc3zyI9JYn/+vMu0nzJCbFEY3N7J/+zaj9XTBvG6KFa5PBMqBCIxIGkJOPbN5xFW4efHy7dzqiCTK6P86GUT6ytpKGlg88s1NnAmVIhEIkTZsZ3PjqTQw0tfOWJDQzLS+fccUO9jhUWXX7HAyv2MGdUPvNGF3gdJ+apj0AkjqT6kvj17eWMLMjgs79dy66aE15HCoulW46wv76ZzywY53WUuKBCIBJn8jJTePjT80lJNj710GqqG1u9jhRy96/Yw8iCDK6YriGjoaBCIBKHRhZk8uAdZ1N3op07HlpDQ0v8TGH99v6jrN13lDsvGEtykuYVCgUVApE4NWtkPr/+5Dx2Vh/nM/9dETeL2zzw+h5y0n3cUj7S6yhxQ4VAJI4tmFjED2+ZzZq99dzz6Nt0dvm9jnRGDh5rYcmmI9w6f5QuIAshFQKROHftrOF849rpLN1Sxd8/vp6OGC4Gj7y5F4A7zh/jaY54o5IqkgDuOH8MTe2dfG/JNprbu/jZrXNIT4mt5Ryb2jp5dPV+rpwxjBH5GV7HiSs6IxBJEH9z8QS+eV3gzOCuR9bQ1NbpdaQBebziAMdbO7lL00mEnAqBSAL5q/PG8MNbZrFqdz23P/hWzAwt7fI7fvPGXuaMymfuKC1FGWqeFAIz22tmG83sHTOr8CKDSKK6cW4Zv7htLlsPN3LFj5fzp42HvY7Up5e3VrG/vpm/vlAXkIWDl2cElzjnZjvnyj3MIJKQrpg+jOfuWcDIgkw+/7t1fPmxd2hsjc5rDRqaO/jR0u2MyM/giuklXseJS2oaEklQE4qzefLz53Pvoon88Z1DXPaDZTy25gBd/uiZxrr6eCsfW7yS3TVNfPO66fiS9SsrHLx6Vx3wkpmtNbO7PcogkvBSkpP48uWTeOrz5zNiSAZfeXIDV/90Bcu213gdjQP1zdzyq5Xsq2vmwU+Vs2iqzgbCxbxYxMLMhjvnDplZMbAUuMc5t/yUfe4G7gYYNWrUvH37EntdVpFwc87xwsYjfHfJu+yvb+bqmaV8+/qzyMtMiXiWzYcauOvhCprbO3no0/OZN1odxINhZmv70/zuSSH4QACzbwAnnHPfP90+5eXlrqJCfcoikdDW2cX9y3fz45d3UJyTxo8+NptzIjSddWB66d18/6VtDMlM5ZE75zO1NDcix45H/S0EEW8aMrMsM8s5eR/4ELAp0jlEpGdpvmS+cOlEnvz8+aT6kvj4/av4wUvb8Ie57+DgsRZue2AV//mnd1k0pYQlX1yoIhAhXlxZXAI8bWYnj/9759wSD3KISC9mjczn+XsX8PVnNvOzV3dS19TOt66fQfBnN2TaOrt45M29/OyVnfid43s3zeTmeWUhP46cXsQLgXNuNzAr0scVkYHLSvNx300zKcxO41fLduFLMv7tI9ND8kvaOcfSLVV864Wt7Ktr5tIpxXzj2umMGpoZguQyEJprSER6ZWb805WT6fL7uX/FHpKTjK9dM+2MisHBYy38y1MbWba9hgnF2Txy53wumlQUwtQyECoEItInM+NfPjyVLj/85o09dPkdX7tm2oDH9TvneHT1Ab79wlb8zvH1a6fxyXNH6/oAj6kQiEi/mBn/es1UfMnG4uW72V3TxM8/MYf8zNQ+X9vU1skbO2t5ZOVe3thZx/njh/Ldj85kZIGagaKBCoGI9NvJM4MJxdn836c3cd1/vcH9f1XOpJKc9/ZxzlFzvI0thxvZcriRN3fWsXpPPe1dfnLSfXzrhhl8Yv4odQZHERUCERmwW8pHMr4om8/+di3X/fwNSvPTA084ONbSQX1T+3v7TizO5lMXjOHiyUWUjy4g1admoGijQiAigzJv9BCevecCfvLyDo63doKBAdlpPqYMy2FKaS5Th+V6cmWyDIwKgYgMWmleBt/56EyvY8gZ0jkR7OdCAAAIO0lEQVSaiEiCUyEQEUlwKgQiIglOhUBEJMGpEIiIJDgVAhGRBKdCICKS4FQIREQSnOdLVfaHmdUAx4CGU57K62NbX/dP/lsI1A4iWk/HH0i+vjL3lLX784PJPZjMveXq6XFPWc/kvY5k5u73o/3zES2Ze9quz0ffIvH5yHfO9T2/t3MuJm7A4oFu6+t+t38rQpUplJlPk7X7vgPOPZjMveXqz/t7pu91JDPH0ucjWjLr8xH9n4++brHUNPTsILb1db+n159ppr6eH0jm7o+9zNzT9t4e95T1THJHMnP3+9H++YiWzD1t1+ejb5H8fPQqJpqGws3MKpxz5V7nGKhYzK3MkRGLmSE2c8di5lPF0hlBOC32OsAgxWJuZY6MWMwMsZk7FjN/gM4IREQSnM4IREQSXNwVAjP7jZlVm9mmQbx2npltNLOdZvZT67aWnpndY2bbzGyzmX0v2jOb2TfM7KCZvRO8fTiUmcOVu9vz/2BmzswKQ5c4bO/1v5vZhuD7/JKZDY+BzPeZ2bvB3E+bWX4MZL45+PPnN7OQtcmfSdbTfL07zGxH8HZHt+29fuY9NZhhT9F8AxYCc4FNg3jtauA8Agst/Qm4Krj9EuBlIC34uDgGMn8D+IdYe6+Dz40EXgT2AYXRnhnI7bbPvcCvYiDzhwBf8P53ge/GQOapwGTgNaDc66zBHGNO2VYA7A7+OyR4f0hv31c03OLujMA5txyo777NzMab2RIzW2tmK8xsyqmvM7NSAj/QK13gf+2/geuDT38e+I5zri14jOoYyBx2Ycz9I+ArQMg7sMKR2TnX2G3XrFDnDlPml5xzncFdVwFlMZB5q3NuWyhznknW07gCWOqcq3fOHQWWAld6/bPal7grBKexGLjHOTcP+AfgFz3sMwKo7Pa4MrgNYBKwwMzeMrNlZnZ2WNMGnGlmgC8ET/1/Y2ZDwhf1A84ot5l9BDjonFsf7qDdnPF7bWbfMrMDwG3A18KY9aRQfD5OupPAX6jhFsrM4dafrD0ZARzo9vhk/mj5vnoU92sWm1k2cD7weLcmubSedu1h28m/7HwETvPOBc4GHjOzccHKHnIhyvxL4N+Dj/8d+AGBH/iwOdPcZpYJfJVAs0VEhOi9xjn3VeCrZvZ/gC8AXw9x1PeDhChz8Gt9FegEfhfKjH8RJISZw623rGb2aeDvgtsmAC+YWTuwxzl3A6fP7/n31Zu4LwQEznqOOedmd99oZsnA2uDDZwj84ux+elwGHArerwSeCv7iX21mfgLzi9REa2bnXFW3190PPBemrN2dae7xwFhgffAHsAxYZ2bznXNHojTzqX4PPE8YCwEhyhzsyLwGWBSuP2q6CfX7HE49ZgVwzj0EPARgZq8Bn3LO7e22SyVwcbfHZQT6Eirx/vs6Pa87KcJxA8bQreMHeBO4OXjfgFmned0aAn/1n+zM+XBw++eAbwbvTyJw6mdRnrm02z5fAv4QC+/1KfvsJcSdxWF6ryd22+ce4IkYyHwlsAUoCsfnIpyfDULcWTzYrJy+s3gPgRaEIcH7Bf39zHt18zxAGD58jwKHgQ4CVfguAn9lLgHWBz/8XzvNa8uBTcAu4Oe8f8FdKvA/wefWAZfGQObfAhuBDQT+0ioNZeZw5T5ln72EftRQON7rJ4PbNxCY32VEDGTeSeAPmneCt1CPdApH5huCX6sNqAJe9DIrPRSC4PY7g+/vTuDTA/nMe3XTlcUiIgkuUUYNiYjIaagQiIgkOBUCEZEEp0IgIpLgVAhERBKcCoHEJDM7EeHjPWBm00L0tbosMFPpJjN7tq+ZP80s38z+JhTHFumJho9KTDKzE8657BB+PZ97fxK2sOqe3cweAbY7577Vy/5jgOecczMikU8Sj84IJG6YWZGZPWlma4K3C4Lb55vZm2b2dvDfycHtnzKzx83sWeAlM7vYzF4zsycsMFf/707OGR/cXh68fyI4ydx6M1tlZiXB7eODj9eY2Tf7edaykvcn3Ms2s1fMbJ0F5q2/LrjPd4DxwbOI+4L7/mPwOBvM7N9C+DZKAlIhkHjyE+BHzrmzgY8CDwS3vwssdM7NITAz6Le7veY84A7n3KXBx3OALwLTgHHABT0cJwtY5ZybBSwHPtPt+D8JHr/PeWSC8+wsInDlN0ArcINzbi6BNTB+ECxE/wzscs7Nds79o5l9CJgIzAdmA/PMbGFfxxM5nUSYdE4Sx2XAtG4zRuaaWQ6QBzxiZhMJzPiY0u01S51z3eeiX+2cqwQws3cIzEHz+inHaef9SfzWApcH75/H+3PM/x74/mlyZnT72msJzFkPgTlovh38pe4ncKZQ0sPrPxS8vR18nE2gMCw/zfFEeqVCIPEkCTjPOdfSfaOZ/Qz4s3PuhmB7+2vdnm465Wu0dbvfRc8/Ix3u/c610+3Tmxbn3GwzyyNQUP4W+CmBtQyKgHnOuQ4z2wuk9/B6A/7TOffrAR5XpEdqGpJ48hKBtQAAMLOT0wjnAQeD9z8VxuOvItAkBfDxvnZ2zjUQWNryH8wshUDO6mARuAQYHdz1OJDT7aUvAncG583HzEaYWXGIvgdJQCoEEqsyzayy2+3LBH6plgc7ULcQmD4c4HvAf5rZG0ByGDN9Efiyma0GSoGGvl7gnHubwAyXHyewOEy5mVUQODt4N7hPHfBGcLjpfc65lwg0Pa00s43AE3ywUIgMiIaPioRIcIW1FuecM7OPA7c6567r63UiXlMfgUjozAN+Hhzpc4wwLw0qEio6IxARSXDqIxARSXAqBCIiCU6FQEQkwakQiIgkOBUCEZEEp0IgIpLg/j+Al2B53j4b5wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 11:27

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossexp_rmspe
10.0235870.0209410.140551
20.0176780.0234310.132211
30.0174530.0169290.120169
40.0126080.0162960.109245
50.0102220.0112380.105433
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit_one_cycle(5, 1e-3, wd=0.2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn.save('1')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXd4VFX6+D9vOoHQAoiCGqr0JoKIBURpKlhQwYqiWJYfu+vqirooYsNdv7aVVUFBbIBiQ8GGoggKUqUjAQKETigJJWWS9/fH3JnMTKZnJhngfJ5nntx77rnnnjtJ7nvf97xFVBWDwWAwGCJNXGVPwGAwGAwnJ0bAGAwGgyEqGAFjMBgMhqhgBIzBYDAYooIRMAaDwWCICkbAGAwGgyEqGAFjMBgMhqgQVQEjIn1FZIOIZIrIKC/Hk0VkunV8kYhkWO1dRGSF9flDRK4JNKaINLLG2GiNmRTNezMYDAaDf6ImYEQkHhgP9ANaAUNEpJVHt2HAQVVtCrwEPG+1rwY6q2oHoC/wpogkBBjzeeAlVW0GHLTGNhgMBkMlkRDFsbsAmaq6GUBEpgEDgbUufQYCY6ztGcBrIiKqesylTwrgSDfgdUwRWQdcCtxk9Ztijfu6vwnWqVNHMzIywrk3g8FgOGVZunTpflWtG6hfNAVMA2C7y3420NVXH1W1ichhIB3YLyJdgUnA2cCt1nFfY6YDh1TV5tLeINAEMzIyWLJkScg3ZjAYDKcyIrI1mH7RXIMRL22eic989lHVRaraGjgPeEREUvz0D+Za9guKDBeRJSKyZN++fT4nbzAYDIbyEU0Bkw2c6bLfENjpq4+IJAA1gAOuHVR1HXAUaONnzP1ATWsMX9dyjDdBVTuraue6dQNqeAaDwWAIk2gKmMVAM8u7KwkYDMz06DMTuN3aHgT8qKpqnZMAICJnA+cAWb7GVHtK6LnWGFhjfhG9WzMYDAZDIKK2BmOtmYwAvgXigUmqukZExgJLVHUm8DbwnohkYtdcBlunXwiMEpEioAS4X1X3A3gb0zrnYWCaiDwNLLfGNhgMpwhFRUVkZ2eTn59f2VM5aUhJSaFhw4YkJiaGdb6cyvVgOnfurGaR32A4OdiyZQtpaWmkp6cj4m1Z1hAKqkpOTg55eXk0atTI7ZiILFXVzoHGMJH8BoPhpCA/P98IlwgiIqSnp5dLIzQCxmAwnDQY4RJZyvt9GgFjCIm8/CK+WLGjsqdhMBhOAIyAMYTEqE9X8ddpK1i943BlT8VgiBlycnLo0KEDHTp0oH79+jRo0MC5X1hYGNQYd9xxBxs2bIjyTCuWaEbyG05Cdh+222OPFxVX8kwMhtghPT2dFStWADBmzBiqVavGgw8+6NZHVVFV4uK8v9dPnjw56vOsaIwGYwgJh0X2FHY+NBiCJjMzkzZt2nDvvffSqVMndu3axfDhw+ncuTOtW7dm7Nixzr4XXnghK1aswGazUbNmTUaNGkX79u3p1q0be/furcS7CB+jwRhCwrHmdyq7txtinye/XMPanbkRHbPVGdV54qrWIZ+3du1aJk+ezBtvvAHAuHHjqF27NjabjZ49ezJo0CBatXJPNH/48GEuueQSxo0bxwMPPMCkSZMYNapMxZOYx2gwhpAQS4cx4sVgCI4mTZpw3nnnOfenTp1Kp06d6NSpE+vWrWPt2rVlzqlSpQr9+vUD4NxzzyUrK6uiphtRjAZjCA3jBWo4AQhH04gWVatWdW5v3LiRV155hd9//52aNWtyyy23eI0zSUoqrZcYHx+PzWYr0+dEwGgwhrAwFjKDIXRyc3NJS0ujevXq7Nq1i2+//baypxRVjAZjCAnnIr8xkhkMIdOpUydatWpFmzZtaNy4Md27d6/sKUUVk4vM5CILicETfmPh5gN8eFdXLmhap7KnYzA4WbduHS1btqzsaZx0ePteTS4yQ1Qwi/wGgyFYjIAxhESpm3LlzsNgMMQ+RsAYQsLkEjQYDMFiBIwhLMwiv8FgCERUBYyI9BWRDSKSKSJlwlBFJFlEplvHF4lIhtV+uYgsFZFV1s9LrfY0EVnh8tkvIi9bx4aKyD6XY3dF895OVZxrMEa+GAyGAETNTVlE4oHxwOVANrBYRGaqqmvY6jDgoKo2FZHBwPPAjcB+4CpV3SkibbCXSG6gqnlAB5drLAU+dRlvuqqOiNY9GVzWYCp3GgaD4QQgmhpMFyBTVTeraiEwDRjo0WcgMMXangH0EhFR1eWqutNqXwOkiEiy64ki0gyoB/wStTuIAXKOFJC5N6+yp1GGU9m93WDwRo8ePcoETr788svcf//9Ps+pVq0aADt37mTQoEE+xw0UTvHyyy9z7Ngx537//v05dOhQsFOPGtEUMA2A7S772Vab1z6qagMOA+kefa4DlqtqgUf7EOwai+uT7joRWSkiM0TkzPLeQCxw2Ys/c9mL8yp7Gk4cFe6MeDEY3BkyZAjTpk1za5s2bRpDhgwJeO4ZZ5zBjBkzwr62p4CZPXs2NWvWDHu8SBFNAePN38jzueS3j4i0xm42u8dLv8HAVJf9L4EMVW0HzKFUM3K/oMhwEVkiIkv27dvnZ/qxwcFjRZU9BTecvzAjYQwGNwYNGsRXX31FQYH9XTgrK4udO3fSoUMHevXqRadOnWjbti1ffPFFmXOzsrJo06YNAMePH2fw4MG0a9eOG2+8kePHjzv73Xfffc5U/0888QQAr776Kjt37qRnz5707NkTgIyMDPbv3w/Aiy++SJs2bWjTpg0vv/yy83otW7bk7rvvpnXr1vTu3dvtOpEimqlisgFXLaIhsNNHn2wRSQBqAAcARKQh8Blwm6pucj1JRNoDCaq61NGmqjkuXSZiF0xlUNUJwASwR/KHflunNsZN2XBC8PUo2L0qsmPWbwv9xvk8nJ6eTpcuXfjmm28YOHAg06ZN48Ybb6RKlSp89tlnVK9enf3793P++eczYMAAn/XuX3/9dVJTU1m5ciUrV66kU6dOzmPPPPMMtWvXpri4mF69erFy5UpGjhzJiy++yNy5c6lTxz27xtKlS5k8eTKLFi1CVenatSuXXHIJtWrVYuPGjUydOpWJEydyww038Mknn3DLLbdE5ruyiKYGsxhoJiKNRCQJu8Yx06PPTOB2a3sQ8KOqqojUBGYBj6jqAi9jD8Fde0FETnfZHQCsi8A9GHxg3JQNhrK4mskc5jFV5dFHH6Vdu3Zcdtll7Nixgz179vgcY968ec4Hfbt27WjXrp3z2EcffUSnTp3o2LEja9as8Zrq35X58+dzzTXXULVqVapVq8a1117LL7/Yl60bNWpEhw52n6lolQSImgajqjYRGYHdAywemKSqa0RkLLBEVWcCbwPviUgmds1lsHX6CKApMFpERlttvVXVUdbtBqC/xyVHisgAwGaNNTRKt3ZKYypaGk4I/Gga0eTqq6/mgQceYNmyZRw/fpxOnTrxzjvvsG/fPpYuXUpiYiIZGRleU/S74k272bJlCy+88AKLFy+mVq1aDB06NOA4/pxxkpNL/abi4+OjYiKLahyMqs5W1eaq2kRVn7HaHreEC6qar6rXq2pTVe2iqput9qdVtaqqdnD57HUZt7Gqrve41iOq2lpV26tqT8/jhsjgXOQ3AsZgKEO1atXo0aMHd955p3Nx//Dhw9SrV4/ExETmzp3L1q1b/Y5x8cUX88EHHwCwevVqVq5cCdhT/VetWpUaNWqwZ88evv76a+c5aWlp5OWV9Ta9+OKL+fzzzzl27BhHjx7ls88+46KLLorU7QbEpOs3hERpun6DweCNIUOGcO211zpNZTfffDNXXXUVnTt3pkOHDrRo0cLv+ffddx933HEH7dq1o0OHDnTp0gWA9u3b07FjR1q3bl0m1f/w4cPp168fp59+OnPnznW2d+rUiaFDhzrHuOuuu+jYsWOFVcg06fpjPF1/xqhZAGQ+04+E+MrP7HPXlMXMWbeXCbeeS+/W9St7OgaDE5OuPzqYdP2nALaSWHkRMG5kBoMhOIyAMYRFrIg7g8EQuxgBc4IQK5ZMUw/GEMucyib/aFDe79MImBOEWIk7KTWQxcZ8DAYHKSkp5OTkGCETIVSVnJwcUlJSwh7DeJGdIMTK/4zRYAyxSsOGDcnOzuZESAF1opCSkkLDhg3DPt8ImBOEWHmeO+vBVPI8DAZPEhMTadSoUWVPw+CCMZFFkSm/ZrEvzzMJdHjEitpvNBiDwRAsRsBEicy9R3hi5hr+8uGyiIwXK8/zY4XFABQWF1fyTAwGQ6xjBEyUKCouAeBwiOn2x8xcw+2Tfi/TriURmVa5OW4JmLx8WyXPxGAwxDpmDSbKhJre/p1fs7yPEyOvAvWq2xPk1aiSWMkzMRgMsY4RMDHAgaOFrN2ZywMfrfDZJ1bWPHzVsDAYDAZPjICJATo//T0BM8HEioCxfsaKwDMYDLFLjBheTm2CSTMWK4GWcQ4vshiZj8FgiF2MgIkwj362ioxRsyL+hh8rGoPDRFYSI04HBoMhdomqgBGRviKyQUQyRWSUl+PJIjLdOr5IRDKs9stFZKmIrLJ+Xupyzk/WmCusTz1/Y1U0Hy7aBsDqHYeByHlbxYh8MfVgDAZD0ERNwIhIPDAe6Ae0AoaISCuPbsOAg6raFHgJeN5q3w9cpaptgduB9zzOu9lLpUtfY1UKr/ywEYAdh45zpMDGH9sPMeC1+RwtCE/gxEqgJc5AyxiZj8FgiFmiucjfBch0lEEWkWnAQGCtS5+BwBhrewbwmoiIqi536bMGSBGRZFX1Fxbva6xKeRLmF5UGIrZ54lvn9u9bDtCzRb2Qx4uVx3mcmFQxBoMhOKJpImsAbHfZz7bavPZRVRtwGEj36HMdsNxDuEy2zGOjpdRvNpixEJHhIrJERJZEMyleztFCr+3hLo7HisJQ6kUWIxMyGAwxSzQFjLeACc+nkt8+ItIau6nrHpfjN1ums4usz60hXA9VnaCqnVW1c926df1MP3TmrN0TsE+4i+Ox4rVlcpEZDIZgiaaAyQbOdNlvCOz01UdEEoAawAFrvyHwGXCbqm5ynKCqO6yfecCH2E1xfseqKO56d0nAPuE+l48WxEbuL5NN2WAwBEs0BcxioJmINBKRJGAwMNOjz0zsi/gAg4AfVVVFpCYwC3hEVRc4OotIgojUsbYTgSuB1f7GisJ9lYtwp/TH9kMRnkl4xFl/MbH3zRoMhlgjagLGWgcZAXwLrAM+UtU1IjJWRAZY3d4G0kUkE3gAcLgyjwCaAqM93JGTgW9FZCWwAtgBTAwwVkzx3sKtYZ1XaCtf4MnhY0XsOHS8XGPYseJgYljCqCoZo2bx3Ox1lT0Vg+GUJqqpYlR1NjDbo+1xl+184Hov5z0NPO1j2HN9XMvrWLHGLxv3u+1/tdLTauidAlv5TGQX/2cuh48XkTXuinKN41yDKdco0WXX4XwA3py3mUf6t6zk2RgMpy4mkr8S6P/KLyzdehCAH9bt9drngIcXWkF5NZjjoZUN8IXTkyKGNZjIaGoGg6G8GAFTCazdlct1r/8K4DPwstNT37vtl1fARIoTIQ4mziR8NhhiAiNgKplFW4JzdIsVAeMwkZUEk6GzkjAlBQyG2MAImEqmX5v6QfUr7xqMK2t35rIyOzyvtBMhF1mcETAGQ0xg6sFECMeaSqhsP3gsqH75hZETMP1f/QUgrAV/h3YQw0swxkRmMMQIRoOJEI41lVBZkJkTVL8pv7m7N3+xYgdZ+48GPE9Veen7P8OamzecJrIYljBGgzEYYgOjwVQyXRvVDnodBiA3v4gb31zIul25AIzq14J7Lm7sc91hb16BM7NzJHBE8m87cIxCWwlJCbH3jmLki8EQG8Te0+EE4JFP7UXFlm8LzyzmYPaqXZx7di2fx79YsaNM24KN+53CBWDc1+vZfsC3W24wikZufhGb9h0J3BGYtGALAO/+tpWHP1kZ1DkVTUKc+bM2GGIB858YBlN/txcVe2lO+TSDd3/LYvk234vt475eH9Q4thAyaB4rLHWL/mb1LrYfOMbgNxfS6/9+BuwF0zJGzWJvbn7AsT5bXlYA+qQ4MoXXgqFqcnyFXctgMPjGmMjKwbw/99Fh7HdhvzEv3OzfNOa6ltC+YQ2g/Oafu6aUJuS89/1lpKUkuFXd/GRZNgBbDxyjXvWU8l3MQdZ8+OoBuPF9qNs8MmMGQa3UxAq7lsFgKIvRYMrJoWNF7D/irw5aZCi2bF3HQvQm84zg/3WTu1OBZ0nn0novoc3PL/HJcPwAvHUZbJobwYH9E7tuCAbDqYERMDGMq7biSHb5wEd/lOm3/eBxtuV4d3fu8/K8sK8ZMc48D+7+EWo0gPevg8VvR+EiZYlhRzeD4ZTACJgYJvtg6eJ9UbHvp+Xtk37n4v+U1QxmLM0O6XrFLtH5Ea90UPMsGPYdNL0MZj0AX4+CkujWuFFVdh46TsaoWcxd7z3nm8FgiB5GwJwgXNPRs9q0f95buJUHPy6r7fjjpw17SwMpQzozMEcKbBwuToEhU+H8v8Ci12HqYMjPDXxymOTm25wZCxyOGQaDoeIwAuYEoV5aMkd8JMb0xujPVwfu5MGRAlt01mCALs/Mof3Y7yAuHvo+C1e+BJk/wKQ+cDC8GjnB4HCUiOHUaQbDSUtUBYyI9BWRDSKSKSJlCoCJSLKITLeOLxKRDKv9chFZKiKrrJ+XWu2pIjJLRNaLyBoRGecy1lAR2edSoOyuaN1Xl0a1ozW0TxT4bwQDJr1RUFTq7qwR1mHKOCd0vhNu+QRyd8DES2H77xG7lqtwLBUwRsIYDBVN1ASMiMQD44F+QCtgiIi08ug2DDioqk2Bl4Dnrfb9wFWq2hZ7GeT3XM55QVVbAB2B7iLSz+XYdFXtYH3eivxd2bmy3enRGtonJaq8OW9zVK9RYCsuXeSviOdxk54wbA4kp8E7V8LKjyM6fEKcEG8lJis2KozBUOFEU4PpAmSq6mZVLQSmAQM9+gwEpljbM4BeIiKqulxVHaUe1wApIpKsqsdUdS6ANeYyoGEU78ErvVsFlwE5klTE87G4RJ2pYEoU/jnjD1bvOBz0+SUlyr+/Wc+uwyEU/Krb3O5h1rAzfHoXzH02YvY5W4meELnTDIaTlWgKmAbAdpf9bKvNax9VtQGHgXSPPtcBy1XVLdhERGoCVwE/uPYVkZUiMkNEziz/LXinMjKRRNyryws2Fym2Jzefj5ZkM2zK4qDPX5F9iP/9tIm/T18R2oVTa8Otn0PHW+Dn52HGnVAUXlXK7uN+ZPCEhc79uErI/rxm52E+Wrw9cEeD4SQnmpH83iIqPP/N/fYRkdbYzWa93U4SSQCmAq+qqsNu9CUwVVULRORe7JrRpWUmJTIcGA5w1llnBXcnHlRGtt6KKPBVoqVv/A4hWuyShSaQNmOzXKnDMkclJMGA16BOc/j+CTi0FQZPhbTTQhrGs1yyw0RWkRrMFa/OB+CG86L2jmMwnBBE8108G3D9D2sI7PTVxxIaNYAD1n5D4DPgNlXd5HHeBGCjqr7saFDVHBctZyJwrrdJqeoEVe2sqp3r1q0b1o1VhoB5bW5m1K9RaCtxCpijBfZFedcsBbkeWQE8ceREO3isiLd+2Ry61iUC3f9qTymzdx1MvJTcLcsY8Np8th8Irm6OtyEhttdgsg8eqxAN1WCoaKIpYBYDzUSkkYgkAYOBmR59ZmJfxAcYBPyoqmqZv2YBj6jqAtcTRORp7ILobx7trivvA4B1EbsTDyqjoNX+I4UB+xw6VsimfUd477essK7xwnd/Otdgdh4qa6JaEaAKpiPnZubeIzw9ax3rduWFNQ9aXgl3fgNaQpX3+1N354+8/rPnO0ZwVIaJLBSWbTvIhc/PZboxqRlOQqJmIlNVm4iMAL4F4oFJqrpGRMYCS1R1JvA28J6IZGLXXAZbp48AmgKjRWS01dYbSAIeA9YDy6ygwNcsj7GRIjIAsFljDY3WvcVqzfceL/zEoWP+tYxA/J5lT8DpSE3jyuZ9/guceab8LyoOPsuzA1VlZfZh2p/ZHu7+kdy3rmWi7UVm7ckHbRNyLhuHgAklhqgiydxr/84WZx1kcJfwTLYGQ6wS1eVqVZ2tqs1VtYmqPmO1PW4JF1Q1X1WvV9WmqtrFsZ6iqk+ralUXl+MOqrpXVbNVVVS1pac7sqo+oqqtVbW9qvZU1eBy3YdBuBrMX3s1IyUxel95eYULlAqWLC+5zWwBBIbNwww1cPwCloVYM+fD37cxcPwCfly/B6qfzkdtJvB1yXlctXs8fPlXKA7tHuOtr3vtruhlDCgPTg3LpOY0nISYSP4w8KfB3N+jic9jCjzSr2UUZhR55qzbU6bt5z/3ee1bXKIsyNzv1RHhg4X+U7Ss25VLxqhZzjWWjXvsb/RZ++37x0liRNFI/mu7GpZNgfevhWPBVwCN9fLJ0cqcYDDEAkbARJiURN/FribN33JCl/M96ENDGj83k5vfWsS8jWUF0J4AhcveX2hPE/P16l1u7Y7nbfP6aShx/NFsBFzzJmxbaE/7vz84p4f4ylgwCwHH34NZ5DecjBgBE2Gqp5QuayV71KuP1XWA8rLZWnvZdbisMJmfud/vuR8ssms471mCxvOB63A6SEqIg/aD4baZkH8I3uoFWwKXIvDUYAptJRyNod+D834rdxoGQ1QwAiYMEny8FT91dRtuOf9s5/49Fzcu06drI8840pOH8sTqbD9g91oTj9CoUoFjNZzdDe76Aa1WH33vGlg6BX94aoy3vL2I1k98G/Y8I02se7kZDOXBCJgw8GYGa9OgOreefzYJ8aVfqbc3+nPqp3Fx8/Dib2Kdzfv9e5mBXTP5auVOZ1zKadWT3Y7HeQoUyu6vOlabdtkP8mtxK/hyJIveuI+f1rmb2ErHc5cwv28Jfv2mvIRi9jKpbAwnI0bARIgRPZuWaatRJZGnrm5Tpv2doedVxJQqhIxRs/h8hWf8rG++XLmLER8uZ4KVuHNA+zPcjrvmDiu0lTgFi+sD+KrX5pNHKrcVPARdhtN194cUfXgzFLi7SbuOVxkEo9BFq/6OwRALGAETRbo3rcOtLiYzB3F+Fp5bn1E9mlMKm183+V9LCZaRU5cDpYGcni/ucS4P3Ob/+pq/fLgMgO/WlvVqKyae4r7/5vGi27k0bhlM6svp5Lj1KQk9FCdiBMp8AC65koyEMZyEGAETJpe1dM+R5c3C0fJ0u7Do18aeffnJAa0Djjv+pk7ln1wUuGniooiO94PlBn3gmEeGghCzHy/fdpB3i/twZ9E/4dBWvkgeTXsp9TD7xYtnW0WxLYj0NiYOxnAyYwRMmLTy0DRqpCY6t//aqxkA9WukAJCalGD99O3C7CDW4zYixc7D+SzJOsCny3Y421bvOMybP9tNZ8EuSThMTD+XtIdh35OviUxPeoor4uwZlT2DPyuSYK7tNAlWoqZlMEQLI2DC5JzT0pzbE2/rzAVN6jj3/355c7LGXeHcD8UV9WSTLzWqJPo8du/7y9z2P1teKmy25rg7DJyXUcvrGG7WxnotuLrwKVZpI8YnvcqI+M/QMJ/c2w8cY3FW+RwCgtHCSuu7Vb4Gc/h4kVtyU4OhvEQzXf9JzTn1qwHwzDVtuLyV/5TywdjZJw3tTGJ8HLWqJkVmgjFCp7Nq+jzm+TBzFRYfLckO63oHqM7NhY/xXOJEHkz8mA0bjvBfbqSA0L7Xi/49F8DtRSFUgsngXMYNuxLp+uwc8otKynXPBoMrRoMJk6b10ljx+OXcFESCwmDqwrdtUJOLmtWlWnICn9x3gbN92IWN3PrVTUv2PDWmCeW56c886PjqcvPdF869jV9IIv8ouo9/F93AOXu/5oOkZ0nnsDOxZEURXIkAK3t1KFVAo0R+kbHTGSKLETDloGZqUlCZlR1xL63PqOH1+EXN6rgJjnQXLcZTJtWtVvECxlcOskjj77ssKi7h9km/027Md8GOxv+Kr+bhuH/QWrL4POlx7n/p/chMNEiCMZFlH7Q7AqzeEZvJOA2G8mAETAVwRbvTWf1kH9o29C5gTrecARwkuaSYKfZYQ6iMgLzv1uwO+9xQpusvbdgf2YfDEnQ/xF3ADYWPkyRFfJI0hh5xoZVzzjlSQMex3wWs5umNhCBqa8eCacxgiBZGwFQQ1ZLdl7v+O6Sjc9vTNJTokg3AX8wM2IVXtHHkC4s24SSmDPSAbtugOqu0MQMLnmKb1uPtxP9we/y3QS/+/7JxPwePFTkDQ0PB88XBGyebU4fB4IoRMJXEVS4R7J4PySQXAeOt8JcrvQM4GFQ2SqkZKBDhPWv9SxiH8N5NOtcXPsEPJZ14MnEKzH4wqNoy5UlGGYy2eaq4pRtOTaIqYESkr4hsEJFMERnl5XiyiEy3ji8SkQyr/XIRWSoiq6yfl7qcc67Vnikir4pluBeR2iLyvYhstH5692uNQdp4mM4SE0ofOp7ag+czK1A6/FjgwufnBtcxjIdtoGe4ayzKMVK4p+jvvGG7ClnyNnxwPRz3Xwa69Dqhi5hgBEyslxNwsC3nGDnGhdkQIlETMCISD4wH+gGtgCEi0sqj2zDgoKo2BV4Cnrfa9wNXqWpb4HbgPZdzXgeGA82sT1+rfRTwg6o2A36w9k8IPLMzu2ownuQcdf8nj3WvsnkhrJuE8xC/74Nlfo+38xDeShzjbEN4qGg4ZM2Hty+HA4HNX95mVlyiflPofLumbHobTwKZQGOFi/8zl27jfqzsaRhOMKKpwXQBMlV1s6oWAtOAgR59BgKOfOszgF4iIqq6XFUdGRTXACmWtnM6UF1Vf1P70+hd4GovY01xaY95PJ8xCX4EzP4j7qlVujet46PniceBo4WBO3mwL8//W/VPG7wLuI+Le8Btn8PRfTCxF3tW/si/v1lfRsj9sd1a3PciYd6ct4mbJi7yKUTbNvDu1OHKaTH+guBKIHOtweBJUAJGRJqISLK13UNERoqI7wg6Ow2A7S772Vab1z6qagMOA54FU64DlqtqgdXfNQLPdczTVHWXNdYuoF4w9xYLBOPq7I2scVdQLy2FD+/qGuEZVQ7RcCZY5cf7K69+V7jrB0hNp/an17Nn3mQ27XOPlXF48XmLtM+yyhM4zJcIAAAgAElEQVQ4Enc6cGiktVIDB3fWsQRMex8ehgbDiUywGswnQLGINAXeBhoBHwY4x9tT0/O/1G8fEWmN3Wx2Twhj+p+UyHARWSIiS/btq7xEiFD6htusXrWgz3nVxfvMwRk1q3jtO//hnjSqUzW8yfnhZHkYjv1yLXsTG3CLPM1SWvJ/SW9Q+7fn3BKDiZ+CYA5Ns8hHQKWifL92D5MXbPE5B8cf9B/ZobtBGwyxTrACpsTSMK4BXlbVvwOB/GOzgTNd9hsCnoVDnH1EJAGoARyw9hsCnwG3qeoml/4NfYy5xzKhYf3c621SqjpBVTuraue6dSu38Ncn913A+8O60vGs4P0R6nkxqew45D0KvGGtVOY+2MPveC/e0D7oazuIhtCqDA4fL+Ld37YyP7uYW/If4gNbL2ovHw8f30YV3J0nvAmYREtTsRW7m45Ka9rA3e8u4ckv1/qcQyAvsu/X7uFwEGn/I0lefsVez3DyEqyAKRKRIdgX3L+y2nxnMbSzGGgmIo1EJAkYDMz06DPTGhNgEPCjqqplfpsFPKKqCxydLdNXnoicb3mP3QZ84WWs213aY5akhDgubOZ9DaVPa7v7cb20ZKbc2cXZ7u1Bd8RLjflA+dEcnCheTNGgREu9zGwk8JjtTvZeMAbWz+KjpLGcxgF+WG9fqM8rKPvQ3bTPbiKbtdK9mqaj7HMoXmQNa5XVQnccOs7d7y7hr9OWB39TESAvv+zfk8EQDsEKmDuAbsAzqrpFRBoBfvNuWBrPCOBbYB3wkaquEZGxIjLA6vY2kC4imcADlHp+jQCaAqNFZIX1cayp3Ae8BWQCm4CvrfZxwOUishG43No/YWlQMxWAuy9qzCXN69KtsX1pypunVUZ6WY1ibxDuyzNHdA8vuDHkM2KTDXtyPTIlCAfaDYMh02gku/kieTTVD9q1jwWZ9kJmOUcKyDlSgKoyP9PuQbZk60Gv47/6w0af1+789Pe89P2fzv3khDgyRs3i3veWOtvyi4oB2JoTXBxRpDChOYZIEZSAUdW1qjpSVada8SVpqhrwAa6qs1W1uao2UdVnrLbHVXWmtZ2vqteralNV7aKqm632p1W1qqp2cPnstY4tUdU21pgjLG8yVDVHVXupajPrZ8UVX48CAzrYAzF7trCb8f52WTPSUhJo3aCGW64y8G42c9Vq6lSz9/esrtmuYc2QA/1evzk2C6KFw/YDxykqdheXqkDzPgwqHIONeD5OGkufuMUATPk1i3OfnsO5T8/h69Wl6XOGeCY8tb5SXx5sYPcGfMVFADneG75xScvjTOVv8skYTlCCStcvIj8BA6z+K4B9IvKzqj4Qxbmd0nQ4s6Zb2vSujdNZNaYPAN/87WJ2Hy6roVRPSSDXMm+4JtZ0aClpKWV/3aEqMM1Oq8bBY+l8scJzOe3ExOaRMsbxLF+vZ3F1wVNMSPo/3kx6iXFFg3lipuJ47LsGuNZKdbcWh6MAFLsIEVVFRML2LiwvRp4ZIkWwJrIaqpoLXAtMVtVzgcuiNy2DP+qmJbslzkxNtlfKvP2CDP51RUsA0quVajnJCfbjrs5OyVZCzVAfYiLCkC5n0qJ+WuDOJwDvL/TIlOBiANxPDYYU/ouZxd0YlTiNFxLfJAn7WoxrOWRPJ7Jw5ILrek2jR2aXKUtgMJyIBCtgEizPrBsoXeQ3xAjJCfFsfrY/D1ze3Ktb7ZQ7u3BfjyacVTvV2eY47M9EdpdHLRpHfxHhX1d4JmWAJ64q21aRRMKU5DlEAUmMLBrBS0XXMSh+Hu8lPUctcpm8IMvZJxIZrj1zb85ZW5oFoDyjHzhayOxVuwJ3dMEoMIZIEayAGYt9sX6Tqi4WkcaA7xVMQ4UTF2d/8HsTF43qVOXhvi04x1XrCOIp0sCLZ5PDpOYtgeUd3RuxYNSlZdoritmrwi8r4MC7rBBeKb6O/1c4gg6yic+THqeJlJZ3LvFQYcJJYOkpHGcszQ4z+ac79763lPs/WMbevNjPWWc4+Qh2kf9jVW2nqvdZ+5tV9broTs0QDqUleMs+Kc89uxYLRl2KCPzrSrspbcqvWT7H8szjBaUPz81WFLsnDXwEfVYEkXiITvzFd16yL0suYHDhv0iVfD5LeoIL41YB9vUT1+/bl2D4auVOHvr4D6/HPH9bv27KKT1WDpXCESNVYKpVGiqBYFPFNBSRz0Rkr4jsEZFPrEBIQ4zh9DzycbxBzSpsee4KbuuWAbh7oDWpG3wApedb+3PXtg1hlrHLLxv3sd9P1uDl2oyrC55ih6bzTuLz3BL/PZMXZNHokdl+c3V9sWIHIz5czsdLs70eD668cug4ap6FYsYzXmuGSBGsiWwy9kDGM7Dn/vrSajPEGP5Sm3ijepVSD6iP773A7Zi3Z55j3JTEeLf2Mq66lcCRCAQIlih0fnqO3z47qMugwjH8XNKepxMn80TCFOIppvm/7CFZ3hwn/jrNeyXNT5bZBY43AePwLPOWB82TA0cLyRg1iy//cPfuixdH0GfAIZwY+WKIFMEKmLqqOllVbdbnHaBy86wYIoJrqYDaVZPcXKM93W+h9GE3uMuZZY5VNn/uPRK4UwCCTctylCrcXfQP3rL1446Eb3kr8QWqYV+XCrR2cuhYadZoh7NAjpdM0jlW5uzjhcVljnlmkc607v3d37Lc2h0mzVA0pBOlhIChFFtxiTMwN5YIVsDsF5FbRCTe+twC5AQ8y1DhOMwbwa4z33ieb0HRtF5ZV2TH221qUlAhVBWK59t7tCkhjqdtt/JI0TAujFvNJ0lj4ODWgBLmrilLghrfUabZs0TDkqwDnPfMHGYGcb+OtbKtOd7XzLxhTGQnHoMnLKTF6G8qexplCFbA3IndRXk3sAt73rA7ojUpQ/gE437sSlMrk3PL06uHNP6pnMPMk6nFvbi96GHqywGYeCnNCnwntwTfqWU88ZV0cu2uXAB+31L2Hc+XbHh/4dagrulvDEPsEuzfVEUTrBfZNlUdoKp1VbWeql6NPejSEGOEulYsInx0Tzc+CLKmjMOm71mF81Tn15I2XFM4luPxVZma9AwD4hYEPikA+T6cBsTLukqg94mFm4PPnGQEjCFSlKeipUkTE4M40sHUrhq42JWDLo1qu/X/+N5uPNi7ude+Z6XbgzUT/VTdjARjKjloMxw26xl02/coy7UpryaN5+8JHyOE7x6c72XtBWCTtd7yoUuBtkDeg0XFwc8jEoGjoZCbXxQ1LzpD5VKep4R5hY1BruvUkKeubsO9lzQJe4zzMmoz4tJmAMx7qCeT7zgPgAHtz3D2SUqIK3clTUdJAm8M7V42i4Avzj07+Ho60eYQadxa+AjTbT34a8Jn/DfxNZIJvRQ0wLEi715x2w+UDXItrUHj/UHdKYSaQ6GY08pLoa2EdmO+Y/QXqyvsmoaKozwCxrxyxCDxccKt559NUkJkNIyz0lPpeU491o7tw0s3dnA7dkHTOsx54GLmPHBxyOP2b1ufl28sW50zHIIpTVyRFJHAw7a7eaboJvrHLWJ60lPU5VDI4xwr8K7BeHOD3rjHrtX4Uj4OHy8KevH+rfm+K3BGmkJLs/p8+Y4APQ0nIn6fQiKSJyK5Xj552GNiDKcIqUkJXhf2m9ZLK+NtdlX7M3jg8uacl+H7rfnhvi2okhRPl0a1yz233zbtd9uvmhTvo2dFIkwsvpJ7iv5Oc8nm8+TRtJTQNIOjhd41mNZnlHXIGPWpPauALyGyYU+eW9aGP/fkkTFqFiu2hy74IonjT8qs+5yc+BUwqpqmqtW9fNJUNfb8VA0xwX+HdGRkr2Zl2v/v+tLyzI5o+feGdSnTL1SaneYu4Cbc1pmH+7Yo97iR4PuSzlxf+ASCMiNpDL3ilnJGjZSgzs33kd6lih8B6u857Zp+ZqxVxvmrCnbt9iSU6p+GE4+ortSKSF8R2SAimSIyysvxZBGZbh1fJCIZVnu6iMwVkSMi8ppL/zSXCpcrRGS/iLxsHRsqIvtcjt0VzXszBGZxlrvrZNXk0neSo5b5x1FKoDx4erSlJMZxX4/w16AizRrNYGDBU2RqAyYmvsjVxz+lPBZmX+7L4K4J2DwW9l2v6KjGWVzJD3Zn7rxKnYUhWkRNwIhIPDAe6Ae0AoaIiKdr0DDgoKo2BV4Cnrfa84HRwIOunVU1z7XKJbAV+NSly3SX429F/q4M5SE5sfTPzbPQV3lIiPc03cWe/8k+anFj4Wi+LjmPf8a9z3MJb5FIeKlt/AllV02g3ZPfuR3zZj7znn8bJlXgOgxgJMxJSjQ1mC5AppV5uRCYBgz06DMQmGJtzwB6iYio6lFVnY9d0HhFRJoB9YBfIj91QzTo1jjduZ2WUpqG5tu/BXYSqJWayKP9vZu9cjwi3b3FhGSNu4LXburIjZ0rL8VNPsmMKBrJq7arGZIwlymJ46hB6Oltkv04cLh6+x7zcHP2pqx8vmIHGaNmcfmLP7u1j/3Kf7BopAjk/WY4sYmmgGkAbHfZz7bavPZRVRtwGEgnOIZg11hc/zKvE5GVIjJDRLw+SURkuIgsEZEl+/b5rpluKD89z3FPVycCl7WsB0Arl8wB5wRRHXP5470ZfnGp2cs184Bn9uMf1+31OsaV7c5g7NWtA088iihxvGi7gb8V3s+5cX/yWdLjNJLQCoLtyS2931fmbGTD7rzS8f08qL09xA9YOdA2+sjjtnFPnlvutGhhxMvJSTQFjDfd2/PvKJg+vhgMTHXZ/xLIUNV2wBxKNSP3wVUnqGpnVe1ct67J1xlNLmzm/v2qwms3dWLOA5e4rcd4Mmlo54Bjf/X/LqR3q9N47aaOZa6zO9d3XZiEuOgGiAbL5yUXclPhY9SQo3yW9Djd4tYEdd7h40VMWlBqvnppzp/0eXmec9+fIhBOLOPlL83jqtfmh35iiJj8Zycn0fxvywZctYiGgKfLirOPiCQANYCAOS1EpD2QoKpLHW2qmqOqjle7icC54U/dEAnu7J7BTw/24IXr21OnWjJJ8XGkJMY785/54tIWvgMwHcTHCRNu68yV7c7g2o7uivH5jX0rwQ5X6/Mbl989urws1XMYWPgUe7Um7yaO48b4uQHPCZQx119q/3DNUNsPHA/rvFAwgfwnJ9EUMIuBZiLSSESSsGscMz36zARut7YHAT9qcK8yQ3DXXhCR0112BwDrwpq1IWKICBl1qjLo3IYs+ddlUUsD38ojLiTQn1DWuCsYf1OnqMwlVLK1HtcVPsmCkjY8nziRRxM+IM5LepkDRwsZM3MNCzL3exmllD/3lL9kgcEQKaImYKw1lRHAt9gf9h+p6hoRGSsiA6xubwPpIpKJPbeZ05VZRLKAF4GhIpLt4YF2Ax4CBhgpImtE5A9gJDA0CrdlqEAe628v6/zLP3v67ecZAOqQL9dYms3MEd3LnJNeLblMW2WRRyrDih5ksq0PwxNm8Wbii6R6+Les2XmYd37N4oGPvJdcDgazkG6oaKIaLKmqs4HZHm2Pu2znA9f7ODfDz7iNvbQ9AjwS7lwNlUvVpHiOeng93XVRI4Z0PYtqLus1K8f0LnNuose6iuNB+vx17Xiwzzk0qFklCjOOLMXE86Ttdjbr6TyR8C5fJI1mQUlrDmh1DpBG7a05dJW95JDGAa3OIapR4uX9sHEd32WvN+8LviaML/KLipn6+zZu75ZhCpMZAmKi8Q0xwR3dG/Ha3Ey3NhFxEy4A1VPKVtmM94iDOc9KP5OUEBeUcBnRs2mZa1cW7xX3ZquexqMJH3JN/HxqiJXYcv5kprsoXcUqHKKaUwDlaHUOaBolBemwaAOkpnNB3CYOaHVyNI2DpLHrcD6zVu4qV3LQV3/YyP9+2kTN1ESu6diwnHdriDSq6jVXXWVhBIwhJrj74sZhP+Q9I/kz0n2/xXsjUolBI8W8kvbMK7Sn1UnARi3yGH/1Wbw88zfSyaW25FFbcp3b6ZJLM9lB7bhcahUdga8/AuBDjxyghzWVnBnV2ZtQkwmJVe1CydKIctwElX3bG3n5Nrefhthi8/6jNKnr34mmIjECxhATlKdCpqeACXWsWLb02EhgH7W44fM8oE3A/k3SU/jh/vZwdD83vvSVXRBJLrXJK9225XGm7KVD3CZqkUei+PBMe3YE85KqcIDqTg3pjJ1nUiW+hEbZG+HPlpBaB6qm238mVQ2+VrchKsSau7cRMIaYIL4cD6bylm+OtkmhSd2qbIrA+kcwFBMHVetA1Tos0k1BRJUp1TlKuuRRG0sAWdv/PL8OSxf8QTq5nCYHaRm3lXr7FtI9sRDWYP+4UBKfQlxVF4FTtY67AHLu14HUdEipEaVv4dQl1ty9jYAxxATlecaLCK8O6UhSvJCaFPqfdCjX7pJRm9+z7KFaHc+qyfJtgdPdd22cHlDAnN+4dkhljSOHkEs1crUaWzjdTSA91Kc/f//JzUeHu7pnMHXBOh7rUZcmqfm8+c3v3NiyClVsB1m3aQtXn53MafFH4Oh+yNkIR3OgyMe9xyWSlJrO10lJ5GgazPjELoBS070LpSq1IC4WSjHELjGmwBgBY4gNyluC2bXaZrBc2LSOM6uwN564qhUXNavDZS+WRso/P6gdPV/4CYD0IMtS+/PsctD6jBpOAXNNxwZ8FmYBrkg+X7yVMY6Lj+MoVTic0pCsKon8WGIjPaUhHy/NBi5g6p5UfnrIw6286Lhd4Bzbbxc4x/Y790uO7CN76TpqSy7sXG4/XnDY+4Qkzi5kXLUghwBKqw9tB53yWlGhLXJJZCOBETCGmKC8Zq5waNuwBvMz91Piw65wdnqqWzG1s9NTaVSnKv+6oiWtzqjOOwuygrpOTavi5uk1Uth12HsaG9c3zx7n1A1bwGzNOUbGqFkBY4eCobDY98PKV0xNVs4xDh4tpJar8E2sAjXPtH88sNmKuXvhN/ZzR15hNRbCMVdBlOMioFwE1b71kLUfjh8EFOa9AFe9DM37hH3PJzpfrdpJ24axI2SNgDGcsjjWfXzZrWtX9R6MeddF9jCsd38trVDpajrzpHfr0+i0qCYv3tCBHpb244nrA9ubK3ao/PRn+RO5fr1qd5k2x2uAL6EM8O2a3QzuclbI15u/cT8XNqsDCUlQ/XT7JxiKbbBzGcwcCR/eAO1uhL7jILXy0wFVNMXFsWUjiy3/TMMpT4cza1bYtRxKk6sp6PWbS1PIBJqLq9ZVLcX9Xe3nh3ow+Y7z2PJcf6qnJPLp/d3JCMJUBpFxxDp0tPwZkP/xse+sASVaWkvG85FWEKaZZunWg4E7eSM+Ac7sAvf8DJc8DKs/gfFdYO0X4Y13AlPZBeQ8MQLGEDOsebIPH93TrcKu17+d/Q25b5v6zrZmpwUfQ+BaQG3MVe5lAM5Or0rPc+qV8VAb2MH7WtE+l5IDkTAXRssxzuGsUKLqs67bLxv38fw36wMm5vTkzXmbAFi69QDbco6FPrmEZOj5KAz/CdJOh49us3+OeC/fcDLiT7OsDIyAMcQMVZMTKjTosUX96mSNu8Kttoy3/88eVl2b8zLcTS6PX9mKO7s34ru/X8xZ6alBXdOXM8Jul7WZuAhIBz/LJ+Vizro9AOw8dJw1O+yL8Z4vzXPW7eX1nzZxz3tLPU/3i6NA2nWv/8bF/wmcWdon9dvC3T9Cr8dhw9d2bWblR7HnYhUFjAZjMMQwNsuG3cKlCNq9l9gLnXkKh5qpSTx+VSuanxa4YJoDzzo4DjPcpS3qcUETe5mBTmeFn8rFwfrdueUewx978gqY8ttWv31+jsA6UNjEJ8JF/4B750N6U/j0bpg6GHI9K4bEBr9m7ufTZdnlHidaLxbhYhb5DQYX6qTZvZ9czWbnN05nzZN9/BZJCxbXWjj/uLw5Iy5typx1e+l5Tl2GXpDBzkPHqZJU/liPr1eXXaCPJAeOlpr09ub5LvBW6dQ9B+78Fha9CT+MhfFdoffT0Om2mMo6cNNbiwC4tlP58rsVl8SWhDECxmAA7u/RhIw6VamXlsLy0ZdTo4q7J1cwwkUksBWmjkuZgHZn1kREuLyVvcBaQnwczULQhiqT1TtKNaRDx4oiMmbUPNXj4qHb/XBOX7un2ZcjYc2ncNUrUCsj4OklJcrCLTlc0KROlCYYOWzGi8xgiD3+2bcFN3S2x2nUqpoUVir61WP68H/Xt2eVl5IC3jhZ6rOs2uEjMDJEol7OunZjuG0mXPEiZC+B/10AiyZAgLf+SQu2cNPERcxZuye684sAXWOgUqsrUf2NikhfEdkgIpkiMsrL8WQRmW4dXyQiGVZ7uojMFZEjIvKaxzk/WWOusD71/I1lMFQUVZMTuO7chqQFG8fiR768MrhDUEPUiaHCaeUl2vLFeZHzhsH9C+Gs8+Hrh+Cd/rDfdybvrBy759yuw9EvHV1eqoSRKimaRO1XKiLxwHigH9AKGOJRlRJgGHBQVZsCLwHPW+35wGjgQR/D36yqHayPwwfR11gGQ0yifiTMwA4NghrjzgszIjSbimfD7jxaPf6tc788CU9DpuaZcMsncPXrsHctvNEdFrxiD9r0wBHvE2MewF45ldyUuwCZqrpZVQuBacBAjz4DgSnW9gygl4iIqh5V1flAKKuHXscKf/oGQ3SJhIUsmIfyN3+7qPwXigLvL9zqFuQaCfdsX3y2PJs/tnskJhWBDjfBX36HJr3g+8fh7cthz9oy3SD2UuF7w1v+uMokmgKmAbDdZT/bavPaR1VtwGEgPYixJ1vmsdEuQiTcsQyGSiHYyH5/BBOU2aJ+9YB9KoqMUbN4/IvVgBcnrii+Dv59+h8MHL/A+8G0+jD4Axg0CQ5thTcvhp//DcV254UdB+2msQ17jkRvghHiVIqD8fbn4nn3wfTx5GZVbQtcZH1uDWUsERkuIktEZMm+fZXop2845QlUeXDkpU15f1hXNj3b32efRVvs+c9qpfpf9xl6QUbI84sET321tsyb/7tW/Myybe6pYYpL1K3vXVOWMOqTldGfJNilXZvr7NpMqwEw9xmY0BN2ruCH9XYr/PTF2ypmLuXgVDKRZQOu6VMbAp5RTs4+IpIA1AD8FsVQ1R3WzzzgQ+ymuKDHUtUJqtpZVTvXrVs3xFsyGCqOB3qfw4XN6vjVUjbszgPgYABX4TEDWvs9Hi3enr+FHi/8xPJtB92Ex+9bDri5OoM9kv+tX7Y49+es28O0xdupUKrWsWsygz+Eo/tg4qU8lDCNZApPiDWY6lXKnyg1kkRTwCwGmolIIxFJAgYDMz36zARut7YHAT+qH0OniCSISB1rOxG4ElgdzlgGQ2VxWct65R7jk/u68d6wLiTGx/4y49acY1zzv1/d1pxuePM3r32fmb2ugmYVgBZXwF8WQvsh/CVhJrOSHqWT/FnZs/KJI4C3SmJsFWSLmoCx1kFGAN8C64CPVHWNiIwVkQFWt7eBdBHJBB4AnK7MIpIFvAgMFZFsywMtGfhWRFYCK4AdwMRAYxkMscRbt59H1rgryjXGuWfX5qJmdd3yqIXKB3d1LdccQmVXbgxH/HujSi24ejx3lzxCFSlgRtKT8M2jUBhGIs4o41ByYy22KqpO06o6G5jt0fa4y3Y+cL2PczN8DHuuj/4+xzIYTlaevroNX63cxSf3XcCtby9yJowMhvMbV6wPTPdxP1bo9SLForiO9Ml/nocTpnHrwvGwYTYM+C80ih3vvLgAtY0qCxPJbzCcYNR2qRZZMzWJrHFXcO7ZtfjpwR5l+r55q9f3Mf7Ss0n0UrOcZCQlxHGEVEbb7oTbv7I3TrkSvnoACvIqd3IexJoGYwSMwXACMH34+cweaX9jLvKRMjchvuy/s6PUgCcP9WlRplbNicrCzTk8M2tt4I5hkuj6vTa6CO77FbqNgCWT4H/dIHNO1K4dLA4NJsbkixEwBsOJQNfG6ZxTP434OOHR/i299vEWdJlYIflXostHi7dTaCvhm9W7yRg1i/0uxdkABk9YyEQX77NIU6ZGUVIq9HkGhn0Hianw/nXw+f1wPMyKnBHA8WuONb+mE/+vz2A4RYiPEzY9258hPurdi5f/5nCSdsYa//xkJc9/s55737cXMLtj8mIADh0rxFYBBVASvWiGgFWmeZ697swf0+ylANbPivp8vGHWYAwGwwnDRc1iKzX92/NLNZRVOw5TYCumw9jvGf3FGmd7tNKk+BQwAIkp9sqZd/8IVevCtJtgxp1wdH9U5uILcQqY2JIwRsAYDAY3gbJyTG+ubHd6Jc4mMAU2u+Yy9ffS6Hpfa1PlJagy3md0gLvnQs/HYO1Me5nm1Z9U2KJIrLopGwFjMJwkpCUncM/FjZk+/PyQz31vWGlMTPWUxIhU74wm+UVl3bGjpcEkBRvMmpAEl/zTbjarebZdk5l2M+RFt7oolObJijH5YgSMwXCyICI80r8lLXwEXzqyKp8fRFGqs2qnRnRukabLMz+Uabvo33Pd9r9ZvYuuz86hwBZ8bJA3/JrIvHFaKxj2PVz+FGz6wa7NLP8gqk9/pxdZwFSOFUtsv6YYDIaQqZpkTxeSmuSeNqRF/er89GAPGtaq4vW8cde25ex0e4bndg1rRneSUeDA0ULn9p7cfO59fxkAuw6VL4NAyAIGID4Buo+Ec/rDzBHwxf12k9lVr9hr0fhAVX26j3+yNJsqSfH0b1vWfOlc5I++z0NIGA3GYDjJSIiPI2vcFawd27fMsYw6Vb3GywAM7nIW3ZqURvd/OeJC3rilU5l+7RvW8HntM2qkhDHjyJO1/6hzu7zv9GEJGAd1msLQ2dDvP7BtIfzvfFj8tk9J4M/K94+P/+D+D5Z5PSYxugZjNBiD4RRm6AUZdDzLu7bStmENjhSUVnicevf5ND+tGtVSEjjnX9+49W1WrxpTh59P7dQkGj8623OoCsd1Paa8sSFJCeV09Y6Lg67DoXkf+HIkzHoAVn8KA16F9CZuXW0lJcTHhZ6wMs54kRkMhgHUMDgAABlOSURBVFhjzIDWfsszu5YK6NYknfRqySQnlH0Afv/AJdSplhwTcTfFJYrNRcCUd+0/qTwajCu1zibv+o8p6P8K7F4Jr3eH38ZDSekaUbiOCo5ASxMHYzAYThjOqBkbJq9gWb87lyaPzub1nzY521zNZeHg6qa8x09G6OISZVuO/0zLbZ/8njZf1IP7F7IrvQt8+yhM6kMT2QHgJhhdCaSFOTSYU6lkssFgOMFpUNO7Q8DoK1s5t2eNvLCiphOQ5dsOAfDb5hxn245Dx8s1pqvG1vXZst5rDl6e8ycX/2cuW3P8C7SiYqWo2ul02zqcvxbeDzmZzE56hPvjv6C4sNRRYczMNWSMsmcG2LjXf7nmhDhjIjMYDCcYvjyahl3YyLnd+gzfi/4VjbfZnlM/zbkdznpM89P8l7Z28Nsmu1Dbk1sQoKcjUFT4ouRC+MvvzCnpxD8Tp1Pt/b6wexUA7/ya5ex/KEDFUocp01ZsBIzBYDgFqJnqv3zvyEubRvya3uSha9XPQBakvXn5ZIyaxeKs0mrrwSbSLM1oHPgh77pUlfH0Yv5S9DfuLfwbcUd2woQe8OMzJFEqVJJdzHR5+UX8usk9Fc0pucgvIn1FZIOIZIpImQqTIpIsItOt44tEJMNqTxeRuSJyRERec+mfKiKzRGS9iKwRkXEux4aKyD4RWWF97ormvRkMBv+seLy33+MD/DgXRBLXt/8xM9dwrNDms++slbsAePPn0jWcbQfc11VUlWm/byubmsYSGnvyAmsw3vimpAu7b/4Z2gyCef/my6THaC+ZACQnlj6qR05dzk0TF7lllXZoMKfMGoyIxAPjgX5AK2CIVfbYlWHAQVVtCrwEPG+15wOjgQe9DP2CqrYAOgLdRaSfy7HpqtrB+rwVwdsxGAwR5LH+LZ115CPJC9/9WaZt2JQlzu33Fm6l1ePfkpdvFzpv/ryJe99b6jz+7Ox1AMxZtxewOw148tjnqxn16SqaPfY1AM/NXsfSrQecWsnIqcvJGDXLbwaBowXejxUl14Jr34SbPqK6HOPTpCfgu9Eka6kw+XOPfT3muEv1Uof3XrEXDcZWXELGqFm89H3Z7ybaRFOD6QJkqupmVS0EpgEDPfoMBKZY2zOAXiIiqnpUVedjFzROVPWYqs61tguBZUDDKN6DwWDwwRu3nBv0Av9X/8+9n7cHYSTYF6T20HbMd3y+fAfPfb2eb9aU5gqL87Cx9X35lzLnfrR4u9v+m/M2c93rv5U5d90u39Uutx3w7ghwtNDGkQIbc0s60rvg30wv7gG/vkrD6b3pLOvtc3S6JJd+h45F/mIvazCOWKZXftjocz7RIpoCpgHg+pvIttq89lFVG3AYCKpQuIjUBK4CXN06rhORlSIyQ0R852MwGAzlpm+b+kEv8Dc/Lc1t3/Eovq6T/f0wmPxokeZv01c4t8fMXMOuw8edWZoBfli3J+AY/tZbprlkei57nvf24e8u5aGP/+COdxaTRyqP2u5mc78P0OIiPkp6iicSplBF7V5xrtYwR7E5b4L7mVnrAt5HtIimgPHm0OF598H0KTuwSAIwFXhVVTdbzV8CGaraDphDqWbkee5wEVkiIkv27dsX6FIGgyECiEDWuCtY82Qfhl6QwW3dMgD4vxvakzXuCnq3qu/Wf/LQ8yp0fu/8mkW35350a3M1rfkiv6hUIP26KcftmGsWBE98LZXsOHScLI9Ymks/E/rkj2NKcW9uj/+Od47/je5xqyi20s0Ulyj5ljmuxMvArhpaRRNNAZMNuGoRDYGdvvpYQqMGcIDATAA2qurLjgZVzVF1GionAud6O1FVJ6hqZ1XtXLeu93rlBoMhsjjMR1WTExgzoDVVkvynQ+nZoh7f/f3iiphauSjyk13Sn4BJ9FMCwFsyhC15wpO227mhcDTHS+L5IOk54r/6K+Qf5uFPVjJ7lV2IeNNgqiaVZgS77/2lLNycU6ZPtIimgFkMNBORRiKSBAwGZnr0mQncbm0PAn7UAD5+IvI0dkH0N4921xSjA4DK0wsNBoMb8QFSyLRpYDe19TynLhNv6wyUNavFCsG6Aufl+xYwU1xiXDxZs7OsY4GDJdqC/oXP8YbtKjK2fwbjz+fA8tLHqreaa67f/derd3P3u4E1s0gRtWSXqmoTkRHAt0A8MElV14jIWGCJqs4E3gbeE5FM7JrLYMf5IpIFVAeSRORqoDeQCzwGrAeWWUFgr1keYyNFZABgs8YaGq17MxhOJf51RUuahfmwv/eSJrzx8ya/aegBujSqzZJ/XUadasnhTrPCcLVCeTNJOWjbwH19yvXd+fMVnsac4CkgiXG2IZx/5Z10WPook5Je4NPiCxlbdKvTbOaKZyaDiswWF9Vsyqo6G5jt0fa4y3Y+cL2PczN8DOv1+1HVR4BHwpqowWDwyV0XNQ75nPSqSQCM6teCUf1aBHVOIOFy7tm1eO7atvR+aR4Na1Uh+2D5UsBEAl+5wwBmLM1mzIDWzv2vV0d2LSS3dlu452deGXMv98d/wUXJK5mbMwp7VEhsYCL5DQZDRNn4TD8WPdorImPd16M0nf3SrQedtVmqVWJJ56ou60f+NJgBHc5w28897j/dS6goQEIyL9kGMaDwaXZrbW7Y/ChH3h0CR/b6PC/Xj+ku0hgBYzAYIkpifJzPomah8nDfFrxxi91fp3vTdDLSU/lrr2bOdZrK4EyXctJd/CS/7NG8LqrKU1+tZcPuPOZtjKzXqqry5JdrAFinZ3N14VM8XzSYxE3fU/TqefDH9KiWaQ4GI2AMBkNM4yj4lZaciIjw98ubuz3kK5r1u30HULpSXKLsyS3g7flbuH3S7zSsFdk5/++nTUxekFV6PeJ5vXgA/QufZWV+XfhsOHx4I/WpOK8xT4yAMRgMMc0lzesx/OLGPH1NG6/H1z9VtjR0LOBe9ExpdXr1iI7/+xbvER2btAHXFz4BfZ6DLfP4Lvmf3Bg/l/IXjw4dI2AMBkNMEx8nPNq/ZRkngK/+34U82Ls5KYmhlxiuCI4U2Ci0MgPsDTMBZriUEMeMpAHcnPQSa0oa8XziRN5PfJaG4nttJhpU3kqZwWAwlIM2DWo442dikUc+XeW2/+Uf4bsmh8ODH/8BVOdXHmVIyVweSfiQ75Ie5nnbYCjpV5rULIoYDcZgMJzwdD67ltf2Lc/1r+CZ+Gbfkf/f3plHWVVdefj7UVVUMVQxllgCzSSDE4PigCiRQVQ0gjFGTLrF1izigK1tIo2xpUU7CcTuaBtcicY4ZalxioK2Ntpook0ERAVBIorKalFbnAKKE8LuP+55VfdVvVdFad0a7tvfWm/d8/Y959y933tV+55z7tm7eUcxGYx23LFzIsd8voCVu4Yxr+RWuGUKvLcx8Wu7g3Ecp81zQ56nyiRx0dFDgGQSnDWG2hsvm5u36MkZO2bzwy/Ohi3r4Y3liV/THYzjOG2eirL8s/3/MHEwm+Yfz0WThzajRnUZM2i3AsUnjLhv1zg4/3kY+b3Er+YOxnGcNk9Dsc6ak5NG5c7UOeuO55tZk3ro1CN3fukmxh2M4zhtntpxzvp278CFkwbXqbf8komcOXYAUw7Ys865puLqU0cm1ndTMeqKR5vlOu5gHMdJHU/NnsCFk4bUke/ZpYy539yXq08dyaEDurNszoSs85P26ZWzv8ryUn40uW5/+ejdtUPjFG5mPvxkB6+++3Hi13EH4zhOwVFaXMRdPxhD764d6B4Ccz532dHcOGM06684pk59M/jmiL3qyPNRO4Jxa2Tdm1sTv4Y7GMdxUkWHRm68vO+cw7ly2v7VjiYevmvxrLEA7Ni5qzrQZi4WnHxA4xVtJE0d4HPeg+ubtL9c+EZLx3FSxcMXHNmo+gN6dmJAz07V78tKiigpEvNO3J9eFWUAbP10R71r4n2bOM5YLurLkPlV+GD7F03aXy7cwTiOkyrizuKrUNROvPKTaIPmO9s+A6CspB2f78ifHrmqla+5tBSJTpFJOlbSBkkbJc3Jcb5U0l3h/ApJ/YO8h6QnJH0saWGtNgdJWhvaXKvw+Iik7pIek/RKOObe2us4TirZo7yUsXs37V6TzHRZRVkJXTqUVMtH9O3apNdJK4k5GElFwHXAcUQp1k6TVDvV2lnAh2a2N3A1sCDIPwMuA36Uo+tfATOBweGVCaU6B1hqZoOBpeG94zgFwspLJ3H79w9r0j7LwwbOkw/qQ6fYGsjcE7L/lWU2ep46um+DfeYLa7M7jOiTOxrAn2s9DddaSHIEcwiw0cxeM7MvgN8DU2vVmQrcGsr3AhMlycy2m9n/EDmaaiRVARVm9rRFCa5vA6bl6OvWmNxxHOcr0am0mL9ccSwXTx5K++Kaf5elxdn/Ont0LmXpD7/BldNypxSIE38abfzQyjrnjx9elbftollHVJf/9rC/AeCKqfuxVwNTdGcdMaCObPAenRvU9euSpIPpDbwRe785yHLWMbMvga1AfWPc3qGfXH32MrO3Q19vA3vk6kDSTEmrJK16992mzTDnOE766NC+iHYhUsAzl07i4mOG5oziPKiyc5YTykd8jeimMw6uLh/QuwtPzR7PL74zYrf0EpFOu5O08v+2fcawPcuzZJ2aIe10kg4m1zMXtT+K3anzderXrWx2g5mNNrPRlZV17x4cx3HyUVleynnjGw6aWV/omh07ax4WiEcgWPvmVvp270hpcd3HrIf2KmfqyOx9OJmmthseZvmr71NclK1TrkgHTU2SDmYzEJ+Q7APUTohQXUdSMdAFyJ2mraZ+nzx9vhOm0DJTac2bWcdxHCcQH+HU3rj56Y6dDba/4e8Oyoq+/F8XHsl/TB+VVacxkcSG9CqnqFb+l8y+nyRJ0sE8AwyWNEBSe2A6sLhWncXAjFD+NvC41eOOw9TXR5IOC0+PnQ4sytHXjJjccRynyblrZv4HCjIbL6cf3JeO7Yt59ac1eWmG9CrP16yayfvtyYPn16y31I61BlRP22VSM/94yjAAFn53FPNO3C+r7oDKThTXGlUN75P8k3CJTcKZ2ZeSZgFLgCLgJjN7UdIVwCozWwz8FvidpI1EI5fpmfaSNgEVQHtJ04DJZrYeOAe4BegAPBJeAPOBuyWdBfwvcEpStjmO4xw6sAdnHN6fL3fV3R8zbM8K1sydTEWH6F9sfMqsMZGfH7ngSJ7YkD0Zc+/ZY9jy0edsDxsv+/WI1nRmjhvEzHGDAPjTy9nry2bZ160vvUFTkuhVzOxh4OFasrmx8mfkcQRm1j+PfBVQ51ENM3sfmPg11HUcx2kUl9caKcTp0rEk6/2YgT3oVVHKoMrsp7emjtyLRatzp1Pep6qCfaoqsmSj+3cHorWXfaoqcj5w8I0hlSyeNZYTFy4D4PQx/bjyoZrQMLlGREngO/kdx3GagTvzTKn9+ykjWLT6Le4/9/BG9Scpp3PJEJ8C69ejI+2CU+nWsYTf5MkA2tS4g3Ecx2lBiovasWn+8YleQ4iykmjJ/brvHlg9CkoadzCO4zgpp6ykHT/71nCGLHudwwY2X+pmdzCO4zjNzM1nHEzH9o1LK/BV+O+LxvH4S1uQRGV5KbOPHZb4NeO4g3Ecx2lmxg/LGWikydl7j3L23qPhx6KTwhOOOY7jOIngDsZxHMdJBHcwjuM4TiK4g3Ecx3ESwR2M4ziOkwjuYBzHcZxEcAfjOI7jJII7GMdxHCcRtDvZ0NKKpI+ADS2tRzPTE3ivpZVoRgrNXig8mwvNXmh5m/uZWYMpgQt9J/8GM2uesKKtBEmrCsnmQrMXCs/mQrMX2o7NPkXmOI7jJII7GMdxHCcRCt3B3NDSCrQAhWZzodkLhWdzodkLbcTmgl7kdxzHcZKj0EcwjuM4TkK0eQcj6SZJWySti8m6S3pM0ivh2C3IJelaSRslvSDpwFibGaH+K5JmxOQHSVob2lwrhcTWLYikvpKekPQXSS9KuiDIU2m3pDJJKyWtCfbOC/IBklYE3e+S1D7IS8P7jeF8/1hflwT5BknHxOTHBtlGSXOa28Z8SCqS9Lykh8L7VNssaVP43a2WtCrIUvm7Dvp0lXSvpJfC3/OYVNlrZm36BYwDDgTWxWQ/B+aE8hxgQShPAR4BBBwGrAjy7sBr4dgtlLuFcyuBMaHNI8BxrcDmKuDAUC4HXgb2TavdQYfOoVwCrAh23A1MD/JfA+eE8rnAr0N5OnBXKO8LrAFKgQHAq0BReL0KDATahzr7tvT3HHS+CLgDeCi8T7XNwCagZy1ZKn/XQZ9bge+Hcnuga5rsbfE/oCb6kvqT7WA2AFWhXEW03wXgeuC02vWA04DrY/Lrg6wKeCkmz6rXWl7AIuDoQrAb6Ag8BxxKtNGsOMjHAEtCeQkwJpSLQz0BlwCXxPpaEtpVtw3yrHotaGsfYCkwAXgo2JB2mzdR18Gk8ncNVACvE9bC02hvm58iy0MvM3sbIBwz+Ul7A2/E6m0Osvrkm3PIWw1hKmQU0V19au0OU0WrgS3AY0R33381sy9DlbiO1XaF81uBHjT+c2hprgFmA7vC+x6k32YDHpX0rKSZQZbW3/VA4F3g5jANeqOkTqTI3rQ6mHzkmn+0ryBvFUjqDNwHXGhm2+qrmkPWpuw2s51mNpLorv4QYJ9c1cKxzdsr6QRgi5k9GxfnqJoamwNjzexA4DjgPEnj6qnb1m0uJpre/5WZjQK2E02J5aPN2ZtWB/OOpCqAcNwS5JuBvrF6fYC3GpD3ySFvcSSVEDmX283sD0GcervN7K/AH4nmoLtKyoQ7iutYbVc43wX4gMZ/Di3JWOBESZuA3xNNk11Dum3GzN4Kxy3A/UQ3E2n9XW8GNpvZivD+XiKHkx57W2r+sYnnMvuTvQZzFdmLZD8P5ePJXiRbGeTdieZCu4XX60D3cO6ZUDezSDalFdgr4DbgmlryVNoNVAJdQ7kD8BRwAnAP2Qve54byeWQveN8dyvuRveD9GtFid3EoD6BmwXu/lv6eY/YfRc0if2ptBjoB5bHyn4Fj0/q7Dvo8BQwN5cuDramxt8X/eJrgC7oTeBvYQeSxzyKae14KvBKOmQ9bwHVE8/drgdGxfs4ENobX38fko4F1oc1Cai3ItZDNRxANdV8AVofXlLTaDQwHng/2rgPmBvlAoqdkNhL94y0N8rLwfmM4PzDW16XBpg3EnqgJn9/L4dylLf0d17L/KGocTGptDratCa8XMzql9Xcd9BkJrAq/7QeIHERq7PWd/I7jOE4ipHUNxnEcx2lh3ME4juM4ieAOxnEcx0kEdzCO4zhOIriDcRzHcRLBHYyTaiTtDJF510h6TtLhDdTvKunc3ej3j5JafU705kTSLZK+3dJ6OK0HdzBO2vnUzEaa2QiigI4/a6B+V6LIxK2S2C5+x2n1uINxCokK4EOI4rhJWhpGNWslTQ115gODwqjnqlB3dqizRtL8WH+nKMpT87KkI0PdIklXSXom5Oz4QZBXSXoy9LsuUz9OyIWyIPS5UtLeQX6LpF9IegJYEPKFPBD6Xy5peMymm4OuL0g6OcgnS3o62HpPiGGHpPmS1oe6/xZkpwT91kh6sgGbJGlh6OM/qQnK6DhAFC7CcdJMhxCFuYwofPmEIP8MOMnMtknqCSyXtJgoNMf+FgXWRNJxwDTgUDP7RFL3WN/FZnaIpCnAvwCTiCJJbDWzgyWVAsskPQp8iyg8/k8kFRGlHcjFttDn6USxx04I8iHAJDPbKemXwPNmNk3SBKKwQSOBy8K1Dwi6dwu2/XNou13SPwEXSVoInAQMMzOT1DVcZy5wjJm9GZPls2kUMBQ4AOgFrAdu2q1vxSkI3ME4aefTmLMYA9wmaX+isBs/DdF6dxGFMe+Vo/0k4GYz+wTAzD6IncsEGX2WKB4ewGRgeGwtogswmCgm1E0hSOkDZrY6j753xo5Xx+T3mNnOUD4CODno87ikHpK6BF2nZxqY2YchKvO+RE4BorhjTwPbiJzsjWH08VBotgy4RdLdMfvy2TQOuDPo9Zakx/PY5BQo7mCcgsHMng539JVEcbgqgYPMbEeIWlyWo5nIH+L883DcSc3fkoDzzWxJnY4iZ3Y88DtJV5nZbbnUzFPeXkunXO1y6SrgMTM7LYc+hwATiZzSLGCCmZ0t6dCg52pJI/PZFEZuHmvKyYuvwTgFg6RhRJGE3ye6C98SnMt4oF+o9hFRGuoMjwJnSuoY+ohPkeViCXBOGKkgaYikTpL6hev9BvgtUVj2XJwaOz6dp86TwPdC/0cB71mUD+hRIkeRsbcbsBwYG1vP6Rh06gx0MbOHgQuJptiQNMjMVpjZXKKsmH3z2RT0mB7WaKqA8Q18Nk6B4SMYJ+1k1mAguhOfEdYxbgcelLSKKBr1SwBm9r6kZZLWAY+Y2cXhLn6VpC+Ah4Ef13O9G4mmy55TNCf1LtEazlHAxZJ2AB8Dp+dpXyppBdHNX51RR+ByoiyILwCfADOC/F+B64LuO4F5ZvYHSWcAd4b1E4jWZD4CFkkqC5/LP4ZzV0kaHGRLiSIbv5DHpvuJ1rTWEkVl/lM9n4tTgHg0ZcdpJYRputFm9l5L6+I4TYFPkTmO4ziJ4CMYx3EcJxF8BOM4juMkgjsYx3EcJxHcwTiO4ziJ4A7GcRzHSQR3MI7jOE4iuINxHMdxEuH/AQfJwynjF4jBAAAAAElFTkSuQmCC\n", "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot_losses(skip_start=10000)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn.load('1');" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 11:32

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossexp_rmspe
10.0122230.0143120.116988
20.0120010.0177890.117619
30.0114020.0355960.114396
40.0100670.0151250.113652
50.0091480.0313260.116344
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit_one_cycle(5, 3e-4)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 11:31

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossexp_rmspe
10.0118400.0132360.110483
20.0107650.0576640.129586
30.0101010.0427440.111584
40.0088200.1168930.135458
50.0091440.0179690.126323
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit_one_cycle(5, 3e-4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(10th place in the competition was 0.108)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_preds=learn.get_preds(DatasetType.Test)\n", "test_df[\"Sales\"]=np.exp(test_preds[0].data).numpy().T[0]\n", "test_df[[\"Id\",\"Sales\"]]=test_df[[\"Id\",\"Sales\"]].astype(\"int\")\n", "test_df[[\"Id\",\"Sales\"]].to_csv(\"rossmann_submission.csv\",index=False)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 2 }