{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import tensorflow as tf\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "np.random.seed(42)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Ch 14. Excercise 8" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In fact, Luis already put his solution to github(https://github.com/luisandresilva/Kaggle/tree/master/Rain-II). He used Keras API and python 2. Here, we will reproduce his model for latest tf.keras API and python 3." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, download data of \"How Much Did It Rain? II\" competition and put it on `datasets/rain2` folder. Read the `train.csv`" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "rain2 = pd.read_csv('datasets/rain2/train.csv')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's view some rows via `head()` method of pandas's `DataFrame`" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Idminutes_pastradardist_kmRefRef_5x5_10thRef_5x5_50thRef_5x5_90thRefCompositeRefComposite_5x5_10thRefComposite_5x5_50th...RhoHV_5x5_90thZdrZdr_5x5_10thZdr_5x5_50thZdr_5x5_90thKdpKdp_5x5_10thKdp_5x5_50thKdp_5x5_90thExpected
01310.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN0.254000
111610.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN0.254000
212510.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN0.254000
313510.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN0.254000
414510.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN0.254000
515510.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN0.254000
6212.09.05.07.510.515.010.516.5...0.9983330.3750-0.12500.31250.87501.059998-1.410004-0.3500061.0599981.016000
7262.026.522.525.531.526.526.528.5...1.0050000.0625-0.18750.25000.6875NaNNaNNaN1.4099881.016000
82112.021.515.520.525.026.523.525.0...1.0016670.3125-0.06250.31250.62500.349991NaN-0.3500061.7599941.016000
92162.018.014.017.521.020.518.020.5...1.0016670.25000.12500.37500.68750.349991-1.0599980.0000001.0599981.016000
102212.024.516.521.024.524.521.024.0...0.9983330.25000.06250.18750.5625-0.350006-1.059998-0.3500061.7599941.016000
112262.012.012.016.020.016.517.019.0...0.9983330.56250.25000.43750.6875-1.760010-1.760010-0.3500060.7099911.016000
122312.022.519.022.025.026.023.525.5...1.0016670.0000-0.18750.25000.6250-1.059998-2.120010-0.7100070.3499911.016000
132372.014.014.018.521.019.520.021.0...0.9983330.50000.18750.43750.81250.000000-1.760010-0.3500061.0599981.016000
142422.012.011.012.517.019.518.021.0...0.9983330.62500.37500.62500.8750-0.350006-0.3500060.0000000.3499911.016000
152472.01.53.57.010.518.016.518.5...0.9983330.37500.18750.50000.68750.349991-2.110001-0.3500061.0599981.016000
162532.016.014.518.023.528.023.526.5...0.9983330.87500.62500.93751.3750-0.350006-1.410004-0.3500062.1199951.016000
172582.022.016.522.526.531.526.529.0...1.0016670.37500.18750.37500.8750-1.410004NaN-0.3500060.6999971.016000
183410.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
193910.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2031410.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2131810.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2232310.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2332810.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2433310.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2533810.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2634310.0NaNNaNNaN8.5NaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2734810.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
2835310.0NaNNaNNaNNaNNaNNaNNaN...0.801667NaNNaNNaN2.0625NaNNaNNaNNaN26.162014
2935810.0NaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaN26.162014
\n", "

30 rows × 24 columns

\n", "
" ], "text/plain": [ " Id minutes_past radardist_km Ref Ref_5x5_10th Ref_5x5_50th \\\n", "0 1 3 10.0 NaN NaN NaN \n", "1 1 16 10.0 NaN NaN NaN \n", "2 1 25 10.0 NaN NaN NaN \n", "3 1 35 10.0 NaN NaN NaN \n", "4 1 45 10.0 NaN NaN NaN \n", "5 1 55 10.0 NaN NaN NaN \n", "6 2 1 2.0 9.0 5.0 7.5 \n", "7 2 6 2.0 26.5 22.5 25.5 \n", "8 2 11 2.0 21.5 15.5 20.5 \n", "9 2 16 2.0 18.0 14.0 17.5 \n", "10 2 21 2.0 24.5 16.5 21.0 \n", "11 2 26 2.0 12.0 12.0 16.0 \n", "12 2 31 2.0 22.5 19.0 22.0 \n", "13 2 37 2.0 14.0 14.0 18.5 \n", "14 2 42 2.0 12.0 11.0 12.5 \n", "15 2 47 2.0 1.5 3.5 7.0 \n", "16 2 53 2.0 16.0 14.5 18.0 \n", "17 2 58 2.0 22.0 16.5 22.5 \n", "18 3 4 10.0 NaN NaN NaN \n", "19 3 9 10.0 NaN NaN NaN \n", "20 3 14 10.0 NaN NaN NaN \n", "21 3 18 10.0 NaN NaN NaN \n", "22 3 23 10.0 NaN NaN NaN \n", "23 3 28 10.0 NaN NaN NaN \n", "24 3 33 10.0 NaN NaN NaN \n", "25 3 38 10.0 NaN NaN NaN \n", "26 3 43 10.0 NaN NaN NaN \n", "27 3 48 10.0 NaN NaN NaN \n", "28 3 53 10.0 NaN NaN NaN \n", "29 3 58 10.0 NaN NaN NaN \n", "\n", " Ref_5x5_90th RefComposite RefComposite_5x5_10th RefComposite_5x5_50th \\\n", "0 NaN NaN NaN NaN \n", "1 NaN NaN NaN NaN \n", "2 NaN NaN NaN NaN \n", "3 NaN NaN NaN NaN \n", "4 NaN NaN NaN NaN \n", "5 NaN NaN NaN NaN \n", "6 10.5 15.0 10.5 16.5 \n", "7 31.5 26.5 26.5 28.5 \n", "8 25.0 26.5 23.5 25.0 \n", "9 21.0 20.5 18.0 20.5 \n", "10 24.5 24.5 21.0 24.0 \n", "11 20.0 16.5 17.0 19.0 \n", "12 25.0 26.0 23.5 25.5 \n", "13 21.0 19.5 20.0 21.0 \n", "14 17.0 19.5 18.0 21.0 \n", "15 10.5 18.0 16.5 18.5 \n", "16 23.5 28.0 23.5 26.5 \n", "17 26.5 31.5 26.5 29.0 \n", "18 NaN NaN NaN NaN \n", "19 NaN NaN NaN NaN \n", "20 NaN NaN NaN NaN \n", "21 NaN NaN NaN NaN \n", "22 NaN NaN NaN NaN \n", "23 NaN NaN NaN NaN \n", "24 NaN NaN NaN NaN \n", "25 NaN NaN NaN NaN \n", "26 8.5 NaN NaN NaN \n", "27 NaN NaN NaN NaN \n", "28 NaN NaN NaN NaN \n", "29 NaN NaN NaN NaN \n", "\n", " ... RhoHV_5x5_90th Zdr Zdr_5x5_10th Zdr_5x5_50th \\\n", "0 ... NaN NaN NaN NaN \n", "1 ... NaN NaN NaN NaN \n", "2 ... NaN NaN NaN NaN \n", "3 ... NaN NaN NaN NaN \n", "4 ... NaN NaN NaN NaN \n", "5 ... NaN NaN NaN NaN \n", "6 ... 0.998333 0.3750 -0.1250 0.3125 \n", "7 ... 1.005000 0.0625 -0.1875 0.2500 \n", "8 ... 1.001667 0.3125 -0.0625 0.3125 \n", "9 ... 1.001667 0.2500 0.1250 0.3750 \n", "10 ... 0.998333 0.2500 0.0625 0.1875 \n", "11 ... 0.998333 0.5625 0.2500 0.4375 \n", "12 ... 1.001667 0.0000 -0.1875 0.2500 \n", "13 ... 0.998333 0.5000 0.1875 0.4375 \n", "14 ... 0.998333 0.6250 0.3750 0.6250 \n", "15 ... 0.998333 0.3750 0.1875 0.5000 \n", "16 ... 0.998333 0.8750 0.6250 0.9375 \n", "17 ... 1.001667 0.3750 0.1875 0.3750 \n", "18 ... NaN NaN NaN NaN \n", "19 ... NaN NaN NaN NaN \n", "20 ... NaN NaN NaN NaN \n", "21 ... NaN NaN NaN NaN \n", "22 ... NaN NaN NaN NaN \n", "23 ... NaN NaN NaN NaN \n", "24 ... NaN NaN NaN NaN \n", "25 ... NaN NaN NaN NaN \n", "26 ... NaN NaN NaN NaN \n", "27 ... NaN NaN NaN NaN \n", "28 ... 0.801667 NaN NaN NaN \n", "29 ... NaN NaN NaN NaN \n", "\n", " Zdr_5x5_90th Kdp Kdp_5x5_10th Kdp_5x5_50th Kdp_5x5_90th \\\n", "0 NaN NaN NaN NaN NaN \n", "1 NaN NaN NaN NaN NaN \n", "2 NaN NaN NaN NaN NaN \n", "3 NaN NaN NaN NaN NaN \n", "4 NaN NaN NaN NaN NaN \n", "5 NaN NaN NaN NaN NaN \n", "6 0.8750 1.059998 -1.410004 -0.350006 1.059998 \n", "7 0.6875 NaN NaN NaN 1.409988 \n", "8 0.6250 0.349991 NaN -0.350006 1.759994 \n", "9 0.6875 0.349991 -1.059998 0.000000 1.059998 \n", "10 0.5625 -0.350006 -1.059998 -0.350006 1.759994 \n", "11 0.6875 -1.760010 -1.760010 -0.350006 0.709991 \n", "12 0.6250 -1.059998 -2.120010 -0.710007 0.349991 \n", "13 0.8125 0.000000 -1.760010 -0.350006 1.059998 \n", "14 0.8750 -0.350006 -0.350006 0.000000 0.349991 \n", "15 0.6875 0.349991 -2.110001 -0.350006 1.059998 \n", "16 1.3750 -0.350006 -1.410004 -0.350006 2.119995 \n", "17 0.8750 -1.410004 NaN -0.350006 0.699997 \n", "18 NaN NaN NaN NaN NaN \n", "19 NaN NaN NaN NaN NaN \n", "20 NaN NaN NaN NaN NaN \n", "21 NaN NaN NaN NaN NaN \n", "22 NaN NaN NaN NaN NaN \n", "23 NaN NaN NaN NaN NaN \n", "24 NaN NaN NaN NaN NaN \n", "25 NaN NaN NaN NaN NaN \n", "26 NaN NaN NaN NaN NaN \n", "27 NaN NaN NaN NaN NaN \n", "28 2.0625 NaN NaN NaN NaN \n", "29 NaN NaN NaN NaN NaN \n", "\n", " Expected \n", "0 0.254000 \n", "1 0.254000 \n", "2 0.254000 \n", "3 0.254000 \n", "4 0.254000 \n", "5 0.254000 \n", "6 1.016000 \n", "7 1.016000 \n", "8 1.016000 \n", "9 1.016000 \n", "10 1.016000 \n", "11 1.016000 \n", "12 1.016000 \n", "13 1.016000 \n", "14 1.016000 \n", "15 1.016000 \n", "16 1.016000 \n", "17 1.016000 \n", "18 26.162014 \n", "19 26.162014 \n", "20 26.162014 \n", "21 26.162014 \n", "22 26.162014 \n", "23 26.162014 \n", "24 26.162014 \n", "25 26.162014 \n", "26 26.162014 \n", "27 26.162014 \n", "28 26.162014 \n", "29 26.162014 \n", "\n", "[30 rows x 24 columns]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "rain2.head(n=30)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are 24 columns. First column `Id` is unique value representing a sequence. Last column `Expected` is target. We simply remove rows that `Ref` column is `NaN`. If `Ref` column is `NaN`, all other columns is `NaN` too." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "train_df = rain2.dropna(subset=['Ref'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using `describe()` method, we get the statistics for dataset. The features excluding `Id` and `Expected` has different scale. But the order of magnitude is not significant." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Idminutes_pastradardist_kmRefRef_5x5_10thRef_5x5_50thRef_5x5_90thRefCompositeRefComposite_5x5_10thRefComposite_5x5_50th...RhoHV_5x5_90thZdrZdr_5x5_10thZdr_5x5_50thZdr_5x5_90thKdpKdp_5x5_10thKdp_5x5_50thKdp_5x5_90thExpected
count6.349375e+066.349375e+066.349375e+066.349375e+065.269105e+066.156846e+066.349288e+066.349375e+065.550921e+066.207490e+06...4.600898e+064.069799e+063.577674e+064.095586e+064.600898e+063.494665e+062.978691e+063.511354e+064.062702e+066.349375e+06
mean5.894813e+052.927051e+019.286182e+002.292666e+011.998385e+012.302103e+012.812518e+012.540071e+012.256900e+012.541082e+01...1.013113e+005.295568e-01-6.975932e-013.815768e-012.057199e+003.230033e-02-3.409001e+00-3.781228e-014.038652e+001.714876e+01
std3.406124e+051.715383e+014.068784e+001.035516e+019.195141e+009.882618e+001.028228e+011.044013e+019.547383e+001.003713e+01...4.179456e-021.476643e+001.017368e+009.260191e-011.617217e+003.699795e+002.719573e+002.087361e+003.902270e+002.025279e+02
min2.000000e+000.000000e+000.000000e+00-3.100000e+01-3.200000e+01-3.200000e+01-2.850000e+01-2.800000e+01-3.050000e+01-2.750000e+01...2.083333e-01-7.875000e+00-7.875000e+00-7.875000e+00-7.875000e+00-9.604000e+01-8.079000e+01-7.168000e+01-1.002000e+021.000000e-02
25%2.923290e+051.400000e+016.000000e+001.600000e+011.450000e+011.650000e+012.150000e+011.850000e+011.650000e+011.850000e+01...9.983333e-01-1.875000e-01-1.062500e+000.000000e+001.062500e+00-1.410004e+00-4.540008e+00-7.100067e-012.059998e+005.080003e-01
50%5.906240e+052.900000e+011.000000e+012.250000e+012.000000e+012.300000e+012.750000e+012.500000e+012.250000e+012.500000e+01...1.005000e+003.750000e-01-6.250000e-013.125000e-011.687500e+000.000000e+00-2.820007e+000.000000e+003.509994e+001.425001e+00
75%8.883170e+054.400000e+011.200000e+012.950000e+012.600000e+012.950000e+013.500000e+013.200000e+012.900000e+013.200000e+01...1.051667e+001.062500e+00-1.875000e-016.875000e-012.562500e+001.409988e+00-1.740006e+003.499908e-015.629990e+004.064002e+00
max1.180945e+065.900000e+012.100000e+017.100000e+016.250000e+016.900000e+017.250000e+019.250000e+016.600000e+017.100000e+01...1.051667e+007.937500e+007.937500e+007.937500e+007.937500e+001.797500e+023.169998e+001.280000e+011.446000e+023.301773e+04
\n", "

8 rows × 24 columns

\n", "
" ], "text/plain": [ " Id minutes_past radardist_km Ref Ref_5x5_10th \\\n", "count 6.349375e+06 6.349375e+06 6.349375e+06 6.349375e+06 5.269105e+06 \n", "mean 5.894813e+05 2.927051e+01 9.286182e+00 2.292666e+01 1.998385e+01 \n", "std 3.406124e+05 1.715383e+01 4.068784e+00 1.035516e+01 9.195141e+00 \n", "min 2.000000e+00 0.000000e+00 0.000000e+00 -3.100000e+01 -3.200000e+01 \n", "25% 2.923290e+05 1.400000e+01 6.000000e+00 1.600000e+01 1.450000e+01 \n", "50% 5.906240e+05 2.900000e+01 1.000000e+01 2.250000e+01 2.000000e+01 \n", "75% 8.883170e+05 4.400000e+01 1.200000e+01 2.950000e+01 2.600000e+01 \n", "max 1.180945e+06 5.900000e+01 2.100000e+01 7.100000e+01 6.250000e+01 \n", "\n", " Ref_5x5_50th Ref_5x5_90th RefComposite RefComposite_5x5_10th \\\n", "count 6.156846e+06 6.349288e+06 6.349375e+06 5.550921e+06 \n", "mean 2.302103e+01 2.812518e+01 2.540071e+01 2.256900e+01 \n", "std 9.882618e+00 1.028228e+01 1.044013e+01 9.547383e+00 \n", "min -3.200000e+01 -2.850000e+01 -2.800000e+01 -3.050000e+01 \n", "25% 1.650000e+01 2.150000e+01 1.850000e+01 1.650000e+01 \n", "50% 2.300000e+01 2.750000e+01 2.500000e+01 2.250000e+01 \n", "75% 2.950000e+01 3.500000e+01 3.200000e+01 2.900000e+01 \n", "max 6.900000e+01 7.250000e+01 9.250000e+01 6.600000e+01 \n", "\n", " RefComposite_5x5_50th ... RhoHV_5x5_90th Zdr \\\n", "count 6.207490e+06 ... 4.600898e+06 4.069799e+06 \n", "mean 2.541082e+01 ... 1.013113e+00 5.295568e-01 \n", "std 1.003713e+01 ... 4.179456e-02 1.476643e+00 \n", "min -2.750000e+01 ... 2.083333e-01 -7.875000e+00 \n", "25% 1.850000e+01 ... 9.983333e-01 -1.875000e-01 \n", "50% 2.500000e+01 ... 1.005000e+00 3.750000e-01 \n", "75% 3.200000e+01 ... 1.051667e+00 1.062500e+00 \n", "max 7.100000e+01 ... 1.051667e+00 7.937500e+00 \n", "\n", " Zdr_5x5_10th Zdr_5x5_50th Zdr_5x5_90th Kdp Kdp_5x5_10th \\\n", "count 3.577674e+06 4.095586e+06 4.600898e+06 3.494665e+06 2.978691e+06 \n", "mean -6.975932e-01 3.815768e-01 2.057199e+00 3.230033e-02 -3.409001e+00 \n", "std 1.017368e+00 9.260191e-01 1.617217e+00 3.699795e+00 2.719573e+00 \n", "min -7.875000e+00 -7.875000e+00 -7.875000e+00 -9.604000e+01 -8.079000e+01 \n", "25% -1.062500e+00 0.000000e+00 1.062500e+00 -1.410004e+00 -4.540008e+00 \n", "50% -6.250000e-01 3.125000e-01 1.687500e+00 0.000000e+00 -2.820007e+00 \n", "75% -1.875000e-01 6.875000e-01 2.562500e+00 1.409988e+00 -1.740006e+00 \n", "max 7.937500e+00 7.937500e+00 7.937500e+00 1.797500e+02 3.169998e+00 \n", "\n", " Kdp_5x5_50th Kdp_5x5_90th Expected \n", "count 3.511354e+06 4.062702e+06 6.349375e+06 \n", "mean -3.781228e-01 4.038652e+00 1.714876e+01 \n", "std 2.087361e+00 3.902270e+00 2.025279e+02 \n", "min -7.168000e+01 -1.002000e+02 1.000000e-02 \n", "25% -7.100067e-01 2.059998e+00 5.080003e-01 \n", "50% 0.000000e+00 3.509994e+00 1.425001e+00 \n", "75% 3.499908e-01 5.629990e+00 4.064002e+00 \n", "max 1.280000e+01 1.446000e+02 3.301773e+04 \n", "\n", "[8 rows x 24 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_df.describe()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We removed the rows that `Ref` column is `NaN`. But there are still `NaN` values. Let's check How many `NaN` are in the `train_df`." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Id 0\n", "minutes_past 0\n", "radardist_km 0\n", "Ref 0\n", "Ref_5x5_10th 1080270\n", "Ref_5x5_50th 192529\n", "Ref_5x5_90th 87\n", "RefComposite 0\n", "RefComposite_5x5_10th 798454\n", "RefComposite_5x5_50th 141885\n", "RefComposite_5x5_90th 67\n", "RhoHV 2279576\n", "RhoHV_5x5_10th 2771701\n", "RhoHV_5x5_50th 2253789\n", "RhoHV_5x5_90th 1748477\n", "Zdr 2279576\n", "Zdr_5x5_10th 2771701\n", "Zdr_5x5_50th 2253789\n", "Zdr_5x5_90th 1748477\n", "Kdp 2854710\n", "Kdp_5x5_10th 3370684\n", "Kdp_5x5_50th 2838021\n", "Kdp_5x5_90th 2286673\n", "Expected 0\n", "dtype: int64" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_df.isna().sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ok. For simplicity, just fill `NaN` cell with 0." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Id 0\n", "minutes_past 0\n", "radardist_km 0\n", "Ref 0\n", "Ref_5x5_10th 0\n", "Ref_5x5_50th 0\n", "Ref_5x5_90th 0\n", "RefComposite 0\n", "RefComposite_5x5_10th 0\n", "RefComposite_5x5_50th 0\n", "RefComposite_5x5_90th 0\n", "RhoHV 0\n", "RhoHV_5x5_10th 0\n", "RhoHV_5x5_50th 0\n", "RhoHV_5x5_90th 0\n", "Zdr 0\n", "Zdr_5x5_10th 0\n", "Zdr_5x5_50th 0\n", "Zdr_5x5_90th 0\n", "Kdp 0\n", "Kdp_5x5_10th 0\n", "Kdp_5x5_50th 0\n", "Kdp_5x5_90th 0\n", "Expected 0\n", "dtype: int64" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_df = train_df.fillna(0)\n", "train_df.isna().sum()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can group by `Id` column via `groupby` method. This is the sequence we feed into the model." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "train_seq = train_df.groupby(['Id'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's check the number of unique `Id` and maximum length of sequence. `size()` method return the length of each group. It's pandas's `Series` object that has `count()` and `max()` method." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(731556, 19)" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_seq_size = train_seq.size()\n", "train_seq_size.count(), train_seq_size.max()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, we know number of sequence and max length of each sequence. So we can make numpy array `X` and `y`. `X` has 731,556 sequences. Each sequence has up to 19 rows. Each row has 22 features. `y` has just one column for `Expected`." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "X = np.zeros((731556, 19, 22))\n", "y = np.zeros((731556, 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Luis applied some preprocessing and feature extraction. Here, we just try to apply raw dataset to LSTM without any preprocessing." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "731556\n" ] } ], "source": [ "i = 0\n", "for name, group in train_seq:\n", " # d.shape is (seq_length, 24)\n", " d = group.values\n", " # column 1~22 are features.\n", " # column 0 is Id and column 23 is target.\n", " # save 1~22 features to 0~21 index of dataset up to d.shape[0].\n", " X[i, :d.shape[0], 0:22] = d[:, 1:23]\n", " y[i, 0] = d[0, 23]\n", " i += 1;\n", "print(i)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We split dataset to train set and test set. In fact test set is validation set for this task. Total dataset size is 731,556. Test set is sufficient 10% of the data." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=42)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Calculate `steps_per_epoch` and `validation_steps` for `fit_generator()` method. We set `batch_size` is 1024." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(643.0, 72.0)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "batch_size = 1024\n", "steps_per_epoch = np.ceil(X_train.shape[0] / batch_size)\n", "validation_steps = np.ceil(X_test.shape[0] / batch_size)\n", "steps_per_epoch, validation_steps" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This model has three layers. `LSTM` layer has 50 units. It is just Luis choice. Final `Dense` layer has 1 unit without activation for `Expected` target. All other parameters of `LSTM` and `Dense` layer use default value. Loss is `'mae'` for regression task. We choose `'adam'` optimizer with default values." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.layers import Dense, LSTM, Bidirectional\n", "from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint\n", "\n", "model = Sequential()\n", "model.add(LSTM(50, input_shape=(None, 22), return_sequences=True))\n", "model.add(LSTM(50))\n", "model.add(Dense(1))\n", "\n", "model.compile(loss='mae', optimizer='adam')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Model training is simple. We just put the two generators and steps for epoch and validation. Let's try 150 epochs." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train on 493800 samples, validate on 164600 samples\n", "Epoch 1/150\n", "493800/493800 [==============================] - 42s 85us/step - loss: 23.2963 - val_loss: 23.1770\n", "Epoch 2/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.2118 - val_loss: 23.1109\n", "Epoch 3/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1849 - val_loss: 23.1646\n", "Epoch 4/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1757 - val_loss: 23.1062\n", "Epoch 5/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1713 - val_loss: 23.1158\n", "Epoch 6/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1618 - val_loss: 23.0918\n", "Epoch 7/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1564 - val_loss: 23.0897\n", "Epoch 8/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1573 - val_loss: 23.0747\n", "Epoch 9/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1489 - val_loss: 23.0772\n", "Epoch 10/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1484 - val_loss: 23.0792\n", "Epoch 11/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1462 - val_loss: 23.0741\n", "Epoch 12/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1386 - val_loss: 23.0949\n", "Epoch 13/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1352 - val_loss: 23.0888\n", "Epoch 14/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1343 - val_loss: 23.0835\n", "Epoch 15/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1309 - val_loss: 23.0896\n", "Epoch 16/150\n", "493800/493800 [==============================] - 40s 81us/step - loss: 23.1316 - val_loss: 23.1009\n" ] } ], "source": [ "callbacks_list = [EarlyStopping(monitor='val_loss', patience=5), \n", " ModelCheckpoint(filepath='14_ex_8.h5', monitor='val_loss', save_best_only=True)]\n", "history = model.fit(X_train, y_train,\n", " epochs=150, batch_size=512,\n", " validation_split=0.25,\n", " callbacks=callbacks_list)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd81fXZ//HXpSzZCLiAEqwWGYYQI4Ioy1HUukAFRHFFiqtabW9tsdZxc9f1c6GtUhUHURy4intgcVAwIAQQEQdqACVQQRQQA9fvj88JhHBCzknOyTlJ3s/H4zzOOd/zHddhnOv72ebuiIiI7JLqAEREJD0oIYiICKCEICIiEUoIIiICKCGIiEiEEoKIiABKCCIiEqGEICIigBKCiIhE1Et1APFo06aNZ2RkpDoMEZEaZfbs2avcvW1F+9WohJCRkUF+fn6qwxARqVHM7MtY9lOVkYiIADEkBDPrYGbTzGyRmS00s0sj228wswIzm2tmr5nZPlGO7WhmsyP7LDSzMaU+O8jM5pvZp2Z2l5lZYr+aiIjEI5YSQjFwhbt3AXoDF5lZV+AWd8909yxgKnBNlGNXAIdG9jkEuKpU4vgHMBrYP/IYXLWvIiIiVVFhG4K7ryD8sOPu68xsEdDO3T8qtVsTYId5tN19U6m3DYkkIDPbG2ju7jMi7x8BTgJeruT3EJEk+PnnnyksLGTjxo2pDkVi0KhRI9q3b0/9+vUrdXxcjcpmlgH0BGZG3o8DRgFrgYHlHNMBeBHYD/ijuy83sxygsNRuhUC7OGMXkSQrLCykWbNmZGRkoFrd9OburF69msLCQjp16lSpc8TcqGxmTYEpwGXu/n0kgLHu3gHIAy4uJ8iv3T2TkBDOMrM9gWj/sqKu1GNmo80s38zyi4qKYg13q7w8yMiAXXYJz3l5cZ9CpM7auHEjrVu3VjKoAcyM1q1bV6k0F1NCMLP6hGSQ5+7PRNnlMWDozs7h7suBhcDhhBJB+1IftweWl3PcBHfPcfectm0r7Ea7nbw8GD0avvwS3MPz6NFKCiLxUDKoOar6dxVLLyMDHgAWufttpbbvX2q3E4CPoxzb3sx2i7xuBfQFFkfaJdaZWe/I+UcBz1fpm0QxdiysX7/9tvXrw3YREdleLCWEvsCZwKBI99G5ZnYscKOZLTCzAuBooKQ7ao6Z3R85tgsw08zmAf8GbnX3+ZHPLgDuBz4FPiMJDcpffRXfdhFJL6tXryYrK4usrCz22msv2rVrt/X9pk2bKj4BcM4557B48eKd7nPPPfeQl6Cqg8MOO4y5c+cm5FzVLZZeRu8Svc7/pXL2zwdyI69fBzJ3sl/3mCOthF/8IlQTRdsuIomXlxdK4F99Ff6fjRsHI0dW/nytW7fe+uN67bXX0rRpU/7whz9st4+74+7sskv0+9uJEydWeJ2LLrqo8kHWIrV6pPK4cdC48fbbGjcO20Uksaqzze7TTz+le/fujBkzhuzsbFasWMHo0aPJycmhW7duXH/99Vv3LbljLy4upmXLllx11VX06NGDPn36sHLlSgCuvvpq7rjjjq37X3XVVfTq1YvOnTvz/vvvA/Djjz8ydOhQevTowYgRI8jJyamwJDBp0iQOPPBAunfvzp///GcAiouLOfPMM7duv+uuuwC4/fbb6dq1Kz169OCMM85I+J9ZLGp1Qhg5EiZMgI4dwSw8T5hQtTsWEYmuutvsPvroI8477zw+/PBD2rVrx4033kh+fj7z5s3j9ddf56OPPtrhmLVr19K/f3/mzZtHnz59ePDBB6Oe292ZNWsWt9xyy9bkMn78ePbaay/mzZvHVVddxYcffrjT+AoLC7n66quZNm0aH374Ie+99x5Tp05l9uzZrFq1ivnz57NgwQJGjRoFwM0338zcuXOZN28ed999dxX/dCqnVicECD/+S5fCli3hWclAJDmqu83ul7/8JQcffPDW948//jjZ2dlkZ2ezaNGiqAlht91245hjjgHgoIMOYunSpVHPPWTIkB32effddxk+fDgAPXr0oFu3bjuNb+bMmQwaNIg2bdpQv359Tj/9dKZPn85+++3H4sWLufTSS3n11Vdp0aIFAN26deOMM84gLy+v0gPLqqrWJwQRqR7ltc0lq82uSZMmW18vWbKEO++8k7feeouCggIGDx4ctT9+gwYNtr7eddddKS4ujnruhg0b7rCPe9ShUuUqb//WrVtTUFDAYYcdxl133cVvf/tbAF599VXGjBnDrFmzyMnJYfPmzXFdLxGUEEQkIVLZZvf999/TrFkzmjdvzooVK3j11VcTfo3DDjuMJ598EoD58+dHLYGU1rt3b6ZNm8bq1aspLi5m8uTJ9O/fn6KiItydU089leuuu445c+awefNmCgsLGTRoELfccgtFRUWsL1v/Vg1q1HoIIpK+SqpjE9nLKFbZ2dl07dqV7t27s++++9K3b9+EX+OSSy5h1KhRZGZmkp2dTffu3bdW90TTvn17rr/+egYMGIC7c/zxx3PccccxZ84czjvvPNwdM+Omm26iuLiY008/nXXr1rFlyxauvPJKmjVrlvDvUBGLtxiUSjk5Oa4FckSqz6JFi+jSpUuqw0gLxcXFFBcX06hRI5YsWcLRRx/NkiVLqFcvve6ro/2dmdlsd8+p6Nj0+iYiImnqhx9+4IgjjqC4uBh357777ku7ZFBVtevbiIgkScuWLZk9e3aqw0gqNSqLiAighCAiIhFKCCIiAighiIhIhBKCiKStAQMG7DDI7I477uDCCy/c6XFNmzYFYPny5Zxyyinlnruibux33HHHdgPEjj32WNasWRNL6Dt17bXXcuutt1b5PImmhCAiaWvEiBFMnjx5u22TJ09mxIgRMR2/zz778PTTT1f6+mUTwksvvUTLli0rfb50p4QgImnrlFNOYerUqfz0008ALF26lOXLl3PYYYdtHReQnZ3NgQceyPPP77jo4tKlS+nePSy7smHDBoYPH05mZibDhg1jw4YNW/e74IILtk6d/de//hWAu+66i+XLlzNw4EAGDhwIQEZGBqtWrQLgtttuo3v37nTv3n3r1NlLly6lS5cunH/++XTr1o2jjz56u+tEM3fuXHr37k1mZiYnn3wy33333dbrd+3alczMzK2T6v373//eukBQz549WbduXaX/bKPROAQRic1ll0GiVwLLyoLIj2k0rVu3plevXrzyyiuceOKJTJ48mWHDhmFmNGrUiGeffZbmzZuzatUqevfuzQknnFDuusL/+Mc/aNy4MQUFBRQUFJCdnb31s3HjxrH77ruzefNmjjjiCAoKCvjd737HbbfdxrRp02jTps1255o9ezYTJ05k5syZuDuHHHII/fv3p1WrVixZsoTHH3+cf/7zn5x22mlMmTJlp+sbjBo1ivHjx9O/f3+uueYarrvuOu644w5uvPFGvvjiCxo2bLi1murWW2/lnnvuoW/fvvzwww80atQonj/tCqmEICJprXS1UenqInfnz3/+M5mZmRx55JEsW7aMb7/9ttzzTJ8+fesPc2ZmJpmZ2xZzfPLJJ8nOzqZnz54sXLiwwonr3n33XU4++WSaNGlC06ZNGTJkCO+88w4AnTp1IisrC9j5FNsQ1mdYs2YN/fv3B+Css85i+vTpW2McOXIkkyZN2joium/fvlx++eXcddddrFmzJuEjpVVCEJHY7OROPplOOukkLr/8cubMmcOGDRu23tnn5eVRVFTE7NmzqV+/PhkZGVGnvC4tWunhiy++4NZbb+WDDz6gVatWnH322RWeZ2dzwJVMnQ1h+uyKqozK8+KLLzJ9+nReeOEFbrjhBhYuXMhVV13Fcccdx0svvUTv3r154403OOCAAyp1/mhUQhCRtNa0aVMGDBjAueeeu11j8tq1a9ljjz2oX78+06ZN48toC6iX0q9fP/Ii63kuWLCAgoICIEyd3aRJE1q0aMG3337Lyy+/vPWYZs2aRa2n79evH8899xzr16/nxx9/5Nlnn+Xwww+P+7u1aNGCVq1abS1dPProo/Tv358tW7bw9ddfM3DgQG6++WbWrFnDDz/8wGeffcaBBx7IlVdeSU5ODh9//HHc19wZlRBEJO2NGDGCIUOGbNfjaOTIkRx//PHk5OSQlZVV4Z3yBRdcwDnnnENmZiZZWVn06tULCKuf9ezZk27duu0wdfbo0aM55phj2HvvvZk2bdrW7dnZ2Zx99tlbz5Gbm0vPnj13Wj1UnocffpgxY8awfv169t13XyZOnMjmzZs544wzWLt2Le7O73//e1q2bMlf/vIXpk2bxq677krXrl23rv6WKJr+WkTKpemva56qTH+tKiMREQGUEEREJEIJQUR2qiZVK9d1Vf27UkIQkXI1atSI1atXKynUAO7O6tWrqzRYTb2MRKRc7du3p7CwkKKiolSHIjFo1KgR7du3r/TxSggiUq769evTqVOnVIch1URVRiIiAighiIhIhBKCiIgASggiIhKhhCAiIoASgoiIRCghiIgIoIQgIiIRFSYEM+tgZtPMbJGZLTSzSyPbbzCzAjOba2avmdk+UY7NMrMZkeMKzGxYqc8eMrMvIsfPNbOsxH41ERGJRywlhGLgCnfvAvQGLjKzrsAt7p7p7lnAVOCaKMeuB0a5ezdgMHCHmbUs9fkf3T0r8kjw6t0iIhKPCqeucPcVwIrI63Vmtgho5+6lV6FuAuww+5W7f1Lq9XIzWwm0BdZUNXAREUmsuNoQzCwD6AnMjLwfZ2ZfAyOJXkIofWwvoAHwWanN4yJVSbebWcNyjhttZvlmlq8JtkREkifmhGBmTYEpwGXu/j2Au4919w5AHnDxTo7dG3gUOMfdt0Q2/wk4ADgY2B24Mtqx7j7B3XPcPadt27axhisiInGKKSGYWX1CMshz92ei7PIYMLScY5sDLwJXu/t/Sra7+woPfgImAr3iDV5ERBInll5GBjwALHL320pt37/UbicAH0c5tgHwLPCIuz9V5rO9S53/JGBBZb6AiIgkRizrIfQFzgTmm1lJT6A/A+eZWWdgC/AlMAbAzHKAMe6eC5wG9ANam9nZkWPPjvQoyjOztoABc0uOFxGR1LCatDReTk6O5+fnpzoMEZEaxcxmu3tORftppLKIiABKCCIiEqGEICIigBKCiIhEKCGIiAighCAiIhFKCCIiAighiIhIhBKCiIgASggiIhKhhCAiIoASgoiIRCghiIgIoIQgIiIRSggiIgIoIYiISIQSgoiIAEoIIiISoYQgIiKAEoKIiEQoIYiICKCEICIiEUoIIiICKCGIiEiEEoKIiABKCHHLy4OMDNhll/Ccl5fqiEREEqNeqgOoSfLyYPRoWL8+vP/yy/AeYOTI1MUlIpIIKiHEYezYbcmgxPr1YbuISE2nhBCHr76Kb7uISE2ihBCHX/wivu0iIjWJEkIcxo2Dxo2339a4cdguIlLTKSHEYeRImDABOnYEs/A8YYIalEWkdlAvoziNHKkEICK1k0oIIiICKCGIiEhEhQnBzDqY2TQzW2RmC83s0sj2G8yswMzmmtlrZrZPlGOzzGxG5LgCMxtW6rNOZjbTzJaY2RNm1iCxX01EROIRSwmhGLjC3bsAvYGLzKwrcIu7Z7p7FjAVuCbKseuBUe7eDRgM3GFmLSOf3QTc7u77A98B51Xxu4iISBVUmBDcfYW7z4m8XgcsAtq5+/eldmsCeJRjP3H3JZHXy4GVQFszM2AQ8HRk14eBk6ryRUREpGri6mVkZhlAT2Bm5P04YBSwFhhYwbG9gAbAZ0BrYI27F0c+LgTaxROLiIgkVsyNymbWFJgCXFZSOnD3se7eAcgDLt7JsXsDjwLnuPsWwKLstkMJI3LsaDPLN7P8oqKiWMMVEZE4xZQQzKw+IRnkufszUXZ5DBhazrHNgReBq939P5HNq4CWZlZSQmkPLI92vLtPcPccd89p27ZtLOGKiEglxNLLyIAHgEXuflup7fuX2u0E4OMoxzYAngUecfenSra7uwPTgFMim84Cnq/MFxARkcSIpYTQFzgTGBTpYjrXzI4FbjSzBWZWABwNlHRHzTGz+yPHngb0A84udWxW5LMrgcvN7FNCm8IDCfxeIiISJws36zVDTk6O5+fnpzoMEZEaxcxmu3tORftppLKIiABKCCIiEqGEICIigBKCiIhEKCGIiAighJAW8vIgIwN22SU85+WlOiIRqYu0YlqK5eXB6NGwfn14/+WX4T1oZTYRqV4qIaTY2LHbkkGJ9evDdhGR6qSEkGJffRXfdhGRZFFCSLFf/CK+7SIiyaKEkGLjxkHjxttva9w4bBcRqU5KCCk2ciRMmAAdO4JZeJ4wQQ3KIlL91MsoDYwcqQQgIqmnEoKIiAB1JSHMnQuvvprqKKqNBrqJSGXUjSqjK6+ETz8ND4u2nHPtoYFuIlJZdaOEMHw4fP45fPBBqiNJOg10E5HKqhsJ4eSToUEDmDw51ZEkXTIGuqkKSqRuqBsJoWVLOOYYeOIJ2LIl1dEkVaIHupVUQX35Jbhvq4JSUhCpfepGQoBQbbR8ObzzTqojSapED3RTFZRI3VF3EsLxx4dfxlpebZTogW6aa0mk7qg7CaFJEzjhBHj6afj551RHk1QjR8LSpaF2bOnSqvUu0lxLInVH3UkIEKqNVq2CN99MdSQ1huZaEqk76lZCGDwYWrSo9dVGiaS5lkTqDnP3VMcQs5ycHM/Pz6/aSc49F6ZMgW+/hUaNEhOYiEgaM7PZ7p5T0X51q4QAodro++/h5ZdTHcn2Zs2q9W0bIpLe6l5CGDQI2rZNr2qjGTPgkEPgzjtTHUm10EA3kfRU9xJCvXpw6qnwr3/BDz+kOprgrrvC8z//GUZ/1WIa6CaSvupeQoBQbbRhA7zwQqojgRUrQlfYffeFTz6Bd99NdURJpYFuIumrbiaEvn2hffv0qDa67z4oLoZnnoHmzeH++1MdUVJpoJtI+qqbCWGXXWDYMHjlFfjvf1MXx6ZNISEccwz06AGnnw5PPQVr1qQupiSrKQPd1M4hdVHdTAgAI0aEXj3PPpu6GKZMgW++gUsuCe9zc0NV1mOPpS6mJEvWQLdE/oCrnUPqqro3DqGEO/zqV+HX4/XXE3POeB16KBQVweLF4ZfMHbKzwwiwOXNSE1M1yMsLbQZffRVKBuPGVW2gW9lFgSAkmcoOoMvICEmgrI4dw1QgIjWNxiFUxCw0Lr/1VrhLr26zZ4fuphddFJJBSUy5ufDhh7U6ISRyriVIfEO12jmkrqq7CQFCtdGWLaGXT3UbPz5MuHfOOdtvP/30MIK6ljcuJ1Kif8BrSjuHSKLV7YTQtSsceCA8/nj1XreoKPRwGjUqzK1UWqtWYZxEXt6Ot70SVaJ/wDWhn9RVFSYEM+tgZtPMbJGZLTSzSyPbbzCzAjOba2avmdk+5Rz/ipmtMbOpZbY/ZGZfRI6fa2ZZiflKcRo+HN5/P3qlcbLcfz/89FOoLoomNzdMr5GKkksNlOgfcE3oJ3WWu+/0AewNZEdeNwM+AboCzUvt8zvg3nKOPwI4HphaZvtDwCkVXb/046CDDvKE++wzd3C/+ebEnzuan39279DBfdCg8vfZssV9//3dDz+8emKqBSZNcu/Y0d0sPE+alOqItpfu8UntBuR7DL+xFZYQ3H2Fu8+JvF4HLALaufv3pXZrAkTtruTubwLr4sxT1WfffaFXr+qrNnr+efj6621dTaMpaVx+553QA0kqlOiG6kRKVjdWjZWQRIurDcHMMoCewMzI+3Fm9jUwErimEtcfF6l2ut3MGpZzzdFmlm9m+UVFRZW4RAxGjAg9e6rjx/fuu0Pl9vHH73y/UaPCvEsPPJD8mCSpkjFdh8ZKSDLEnBDMrCkwBbispHTg7mPdvQOQB1wc57X/BBwAHAzsDlwZbSd3n+DuOe6e07Zt2zgvEaNTTw135U88kZzzl5g/H95+Gy68EHbddef77rVXSBoPPxxGNEuNlYxurMlKMipx1G0xJQQzq09IBnnu/kyUXR4DhsZz4UhVlLv7T8BEoFc8xydUu3bQr1+oNkrmQL277w5dSnNzY9s/NxdWroSpUyveV9JWMrqxJjrJqMQhEFsvIwMeABa5+22ltu9farcTgI/jubCZ7V3q/CcBC+I5PuFGjICPP4aCguSc/7vvYNKkMM6gdevYjvn1r0Oy0piEGi0Z3VgTnWQ0C61AbCWEvsCZwKBSXUSPBW40swVmVgAcDZR0R80xs62/YGb2DvAUcISZFZrZryMf5ZnZfGA+0Ab438R9rUoYOjRU4yRrBtSJE8P/sIvjqFnbddew5Ocrr4SGaKmRktGNNdFJRqOzBeryXEbRHHNMKCV8/nn4n5somzeHeZP23jv+9Q6WLg09oa69Fq6pTLu91FaJnBNK8zfVbprLqDJGjAj/+mfOTOx5X345JJmddTUtT0YGHHlk6G20eXNi45IaLZFdbTU6W0AJYXsnnggNGyZ+TMLdd8M++8CQIZU7Pjc33Aa+8UZi4xKJ0OhsASWE7bVoAcceC08+mbi78cWL4dVXYcwYqF+/cuc48cTQEK3GZUmiRA/uUzfWmkcJoawRI8J02NOnJ+Z899wTEsH551f+HA0bhoFqzz8fuqGKpDl1Y62ZlBDKOu44aNo0MdVG69bBQw/BaaeFgWZVkZsbVnh79NGqxyWSZOrGWjMpIZTVuHGoopkypeojhB95JCSFyjQml9W1a1hh7f77kzt4TiQBktWNNdHVUKrW2p4SQjTDh8N//1u1Rlz30Jh88MFwyCGJiSs3N3SLff/9xJxPJEmSMTo70dVQqtbakRJCNEcfHRaqqUq10RtvhB/veAaiVeTUU6FZMzUuS9pLRjfWRFdDqVprR0oI0TRoEEYuP/ccbNhQuXPcfTe0bQvDhiUurqZNQ6P3k0/C2rWJO69IgiWjG2uiq6GSUa1V46u0Ylk0IV0eSVkgpzxvvBEWznnqqfiP/fzzsBLK2LGJj2vWrBDXvfcm/twiaaxjx/BPv+yjY8f0ON+kSe6NG29/rsaNK78YUiLPR6IWyKmzBgyAPfes3NxGf/97SOljxiQ8LHJyIDNT1UZS5yS6GirR56sNVVpKCOXZddfQXfTFF8P6xrFavz5MM3HyydC+feLjKllNLT8f5s5N/PlF0lSiq6ESfb6aUKVVESWEnRk+HDZuDAPCYvXYY2Gq60R0NS3PyJFhsJpWU5M6JtGjqRN5vkT3rEpGT62KKCHsTJ8+4bYh1mojdxg/PlTpHH548uLafffQ6D1pUuUbvUUkodK9SisWSgg7YxZ6Cb32GqxeXfH+77wTFti55JLETp8dTW4urFkDz0RbwE5Eqlu6V2nFQushVOTDDyE7G+67L4xa2ZlTT4U334TCwh1Te6Jt2RLWWOjQAaZNS+61RKRG03oIiZKVBZ07V1xtVFgIzz4L552X/GQAoRfTeefB22/DkiXJv56I1HpKCBUxC43Lb78Ny5eXv9+994a79gsvrLbQOOus0BvqwQer75oiUmspIcRi+PDQYPzUU9E/37gxVO4dfzx06lR9ce2zT5iddeLEMBOqiEgVKCHE4oADQtVRedVGTz4JRUWJnbcoVuefD99+G8ZLiIhUgRJCrIYPh//8B774YsfP7r47JI0jj6z+uAYPDiUFjVwWkSpSQohVySR1Tzyx/faZM+GDD0LpINldTaOpVw/OOQdefjk0bIuIVJISQqwyMsJAtbLVRuPHhympR41KSVgAnHtuaNB+6KHUxSAiNZ4SQjyGD4d582DRovD+229D+8HZZ4ekkCr77gtHHBGmstiyJXVxiEiNpoQQj9NOC/3/S0oJEyaE3j2paEwuKzc3TMby1luJPa97OOfQodC/f2g8F5FaSQkhHnvtFabFnjw5JIJ774Vf/zqMGE61k04KcxwlqnH5++9DY3m3bqH08fbbMGtW+L5r1iTmGiKSVpQQ4jV8OHzyCfzlL2GgWjJnNY1Ho0Zw5plhtPSqVZU/z8KFcNFF0K5d+G5NmoS2icLCMG/SggVw7LHwww8JC11E0oMSQryGDg09e266KdTdDx6c6oi2Oe882LQpzIIaj59/hqefhoEDoXv30BYxZMi2HlRnnQW77QbHHBNKR7NmwYknhgF5IlJrKCHEa/fdQ7UJhDvpXXdNbTylHXggHHJIqDaKZdLCb76BG24IPahOPTWMsbjxxlAaePhh6NVrx2OGDAkjo996C045JSQgEakVlBAq4+KLw5oH55yT6kh2lJsbqn1mzoz+uTu8+y6MGBFW2rjmmlAqeP55+OwzuPJKaNNm59c480z4xz/C6OgzzoDNmxP/PUSk2ikhVMbgwaH7aatWqY5kR8OGhXr/so3LP/4I//xnmILj8MPDQLaLLoLFi+HVV+GEE+Ir7YwZA7feGuZ3ys1Vd1eRWqBeqgOQBGvWLDR8T54Mt98eqoX+/vdQzbN2bSjZTJgAp58eEkdVXHFFaFy+9tpwrvHjUzNaW0QSQgmhNsrNDQ3DhxwSBtHVqxfq+y+6CPr2TeyP9jXXwLp18P/+HzRtCn/7m5KCSA2lhFAbHXJIeBQWwvXXhxlR99orOdcyg1tuCVVSN90USihjxybnWiKSVEoItZEZvPdeeN6lGpqJzOCee0JSuPrqUH102WXJv66IJFSFvxZm1sHMppnZIjNbaGaXRrbfYGYFZjbXzF4zs33KOf4VM1tjZlPLbO9kZjPNbImZPWFmDRLzlQQIDcTVkQxK7LJLWLltyBD4/e81HbdIDRTLL0YxcIW7dwF6AxeZWVfgFnfPdPcsYCpwTTnH3wKcGWX7TcDt7r4/8B1wXtzRS3qpVw8efzz0who9OrwWkRqjwoTg7ivcfU7k9TpgEdDO3b8vtVsTIOpIKHd/E1hXepuZGTAIeDqy6WHgpLijl/TToAFMmQL9+oXxCs8/n+qIRGqu4uIw88Chh4b5xZIsrjoFM8sAegIzI+/HmdnXwEjKLyFE0xpY4+7FkfeFQLt4YpE01rgx/OtfkJMTZoh9/fVURyRSsxQXwyOPQNeu4cbqhx9g2bKkXzbmhGBmTYEpwGUlpQN3H+vuHYA8IJ45oKP1S4xawjCz0WaWb2b5RZp6ueZo1iwMfjsgbihvAAAL9UlEQVTggDDv0TvvpDoikfRXXBwmk+zSJcwh1qRJmFRy7tywLcliSghmVp+QDPLc/ZkouzwGDI3juquAlmZW0supPbA82o7uPsHdc9w9p23btnFcQlKuVSt47bUwRcZxx0F+fqojEklPP/8cOmV07hymxGnWDJ57DubMgZNPrrYOIrH0MjLgAWCRu99Wavv+pXY7Afg41ou6uwPTgFMim84CVNlcG+25J7zxBrRuHSYFnD8/1RGJpI9Nm0KPvM6dw2zFrVrBCy/A7NmhZF3NgzxjSTt9Cb2EBkW6mM41s2OBG81sgZkVAEcDJd1Rc8xsa59DM3sHeAo4wswKzSwyVShXApeb2aeENoUHEve1JK20bw9vvhnWbDjqqLCehNQNa9fC9OlaP6OsTZvgvvvC4lrnnx8mlJw6NUw3f/zxKRvtbx7LNMlpIicnx/NV7VBzLVoUeh/ttltoU+jYMdURSTJ9+SUcfXS4AahfH/r0CTcERx0VOhyk09Tx1eWnn0LV0N/+Bl9/HWYU+OtfQ1ftJCYBM5vt7jkV7afZTqX6dOkSehytWxeW5VyxItURSbIsXBi6Sn77bZhX6/e/D6WEv/wFevcOd8RDhoRp1D/9NLb1O2qyjRvDaP799oMLLwyl5ldegRkzwsJTaTL/l6aukOqVlRV6Hx15ZFihbdiw0M6w117bPzdtmupIpbLefx9+85tQRfjOO2HhphKrVoXqw9dfD49nnw3bMzLCv4mjjgo3C61bpyT0hNu4MUw7f9NNodvoYYeFmYePOCJtkkBpqjKS1Hj77dCt7uuvo98dNmmyfYKIljRKnhs3rvbwpRwvvhhW32vfPvQwy8gof1/3UDooSQ5vvRUGX5lBdnZIDkceGWbobdSo2r5CQmzYEKaZv+mmUBLu1y9UDQ0cmJJEEGuVkRKCpFZxcbhr/OabUL2ws+fVq6Ofo1mzbQli5MgwbUZ1zuMkwcMPh54yWVnw0kuwxx7xHV9cHLomlySIGTPCtt12C4s6lbQ/HHhgev79/vRTiPn110M7wTffQP/+Yb2QAQNSGpoSgtQ+P/8MK1eWnzAWLw4DeAYMCF35fvnLVEdcd9xyC/zP/4Q7+meeCUm6qtatCz2UShLERx+F7W3bhqqXvn3DIzs7TJlS3dxhwYJt8U2fDuvXh8byQYPCNPD9+1d/XFEoIUjd4x4aMK+4ItxZ/t//wSWXpOfdZG3hHhLBrbeGaUoeeQQaNkzOtZYtC+0Pb74Z1gX//POwvVEjOPjgbQni0ENh992TF8Mbb4QE8MYb4WYEwoj8kjaQAQOgefPkXL+SlBCk7ioshN/+NlRb9O0bkkTnzqmOqvb5+efQh/7hh8NqfHfeWb1dSb/5Jqz78f774XnOnBAThB5tJcmhb1/Yf//K1d2vWwf//ve2UsCiRWH7HnuEBFDy6NAhcd8rCZQQpG5zh0cfhUsvDT09rr8eLr+8bvZ9T4b160OJ4MUX4brrQnfSVPea2bAhDOx6771tieK778JnbdtuSw59+8JBB0UvyRQXh3OUJID//GdbO0a/fttKAenajlEOJQQRCD08LrggTMPdq1fo8te1a6qjqtn++98wmnbGDPj732HMmFRHFN2WLfDxx9sniCVLwmcNG4bBcX37hgFzy5aFBDBt2raeTgcdtC0BHHpozevpVIoSgkgJd3jiCbj44lAF8Ne/wh//GEbPSnwKC8Oo2iVL4LHHYGg8c1qmgZUrt1UxvfdemDNo06bwWadO27q6DhpUe8ZCoIQgsqOVK0NSeOqp0DPlwQehR49UR1VzLF4cpqL47rtQ4ho4MNURVd3GjfDhh6FNoBb3StPUFSJl7bEHPPkkPP10uNPNyQl9xEvuEKV8H3wQunpu3BgaWWtDMoBQDdSnT61OBvFQQpC6Z+jQ0Kd92LDQIHrwwaHqoCbYvDmUdBYuDPXdU6bAvHlhe7K89lpIAM2bh2qWnj2Tdy1JKc1lJHVT69Zhrdphw0Kj6CGHhP7011xTvY2H7qEKpqgo/NCvXLnz16tWRZ/qo2nT0Gjep0949O6dmDrwxx8PU4x07RrmoNp776qfU9KW2hBE1qwJXVInTgz91ydODAmiKrZsCT/gX38dHoWF4XnZsu1/6IuKQrfGaFq1CtVcbduG52ivW7QIpZ0ZM8KjoGBbaeFXv9qWIPr0gW7d4ut2O3586LZ7+OFh0ZYWLar2ZyIpo0ZlkXi98kqYB2nZsjBd8w03hP7nZbmHO/XSP/RlXy9btmPbRMOGsM8+Yd6l8n7gS163aVO5XlA//hjq+2fMCH3oZ8wISQfCdBJlSxHRRvS6h5LS//4vnHRSKCXU4C6XooQgUjnffx+qju67L4xuvfDCHe/0CwtD42pp9euHGT7btw+jVjt02PF127bVP3jLPUzxUFKCKFuK6Nx5+1JE585huo8JEyA3N6xXUE81yzWdEoJIVbz5ZvhBXLo0VLO0a1f+D32HDuGuvqaMXC1diih5rFoVPmvQIJRsxo4NJaRUjz6WhFBCEKmqTZvCD+Wee9buKS/c4bPPQmKYNStUK515ZqqjkgSKNSGoLChSngYNQp1/bWcWlnbcbz8lgjquhpRxRUQk2ZQQREQEUEIQEZEIJQQREQGUEEREJEIJQUREACUEERGJUEIQERGgho1UNrMi4MtUxxHRBliV6iAqkO4xpnt8oBgTId3jg/SPsarxdXT3thXtVKMSQjoxs/xYhoKnUrrHmO7xgWJMhHSPD9I/xuqKT1VGIiICKCGIiEiEEkLlTUh1ADFI9xjTPT5QjImQ7vFB+sdYLfGpDUFERACVEEREJEIJIU5m1sHMppnZIjNbaGaXpjqmaMxsVzP70MympjqWaMyspZk9bWYfR/4s+6Q6prLM7PeRv+MFZva4maV0YWEze9DMVprZglLbdjez181sSeS5VRrGeEvk77nAzJ41s5bpFmOpz/5gZm5mbVIRWySGqPGZ2SVmtjjyb/LmZFxbCSF+xcAV7t4F6A1cZGZdUxxTNJcCi1IdxE7cCbzi7gcAPUizWM2sHfA7IMfduwO7AsNTGxUPAYPLbLsKeNPd9wfejLxPpYfYMcbXge7ungl8AvypuoMq4yF2jBEz6wAcBXxV3QGV8RBl4jOzgcCJQKa7dwNuTcaFlRDi5O4r3H1O5PU6wg9Zu9RGtT0zaw8cB9yf6liiMbPmQD/gAQB33+Tua1IbVVT1gN3MrB7QGFieymDcfTrw3zKbTwQejrx+GDipWoMqI1qM7v6auxdH3v4HaF/tgW0fT7Q/R4Dbgf8BUtqwWk58FwA3uvtPkX1WJuPaSghVYGYZQE9gZmoj2cEdhH/YW1IdSDn2BYqAiZFqrfvNrEmqgyrN3ZcR7sK+AlYAa939tdRGFdWe7r4Cws0KsEeK46nIucDLqQ6iLDM7AVjm7vNSHUs5fgUcbmYzzezfZnZwMi6ihFBJZtYUmAJc5u7fpzqeEmb2G2Clu89OdSw7UQ/IBv7h7j2BH0l9Vcd2InXxJwKdgH2AJmZ2RmqjqtnMbCyhyjUv1bGUZmaNgbHANamOZSfqAa0I1dR/BJ40M0v0RZQQKsHM6hOSQZ67P5PqeMroC5xgZkuBycAgM5uU2pB2UAgUuntJyeppQoJIJ0cCX7h7kbv/DDwDHJrimKL51sz2Bog8J6UqoarM7CzgN8BIT7++7r8kJP55kf837YE5ZrZXSqPaXiHwjAezCKX/hDd8KyHEKZKVHwAWufttqY6nLHf/k7u3d/cMQiPoW+6eVne27v4N8LWZdY5sOgL4KIUhRfMV0NvMGkf+zo8gzRq+I14Azoq8Pgt4PoWxRGVmg4ErgRPcfX2q4ynL3ee7+x7unhH5f1MIZEf+naaL54BBAGb2K6ABSZiMTwkhfn2BMwl33nMjj2NTHVQNdAmQZ2YFQBbwfymOZzuR0svTwBxgPuH/SkpHs5rZ48AMoLOZFZrZecCNwFFmtoTQQ+bGNIzxbqAZ8Hrk/8u9aRhj2ignvgeBfSNdUScDZyWjpKWRyiIiAqiEICIiEUoIIiICKCGIiEiEEoKIiABKCCIiEqGEICIigBKCiIhEKCGIiAgA/x/EQBstugsR3wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "loss = history.history['loss']\n", "val_loss = history.history['val_loss']\n", "epochs = range(1, len(loss) + 1)\n", "plt.plot(epochs, loss, 'bo', label='Training loss')\n", "plt.plot(epochs, val_loss, 'r', label='Validation loss')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "model.load_weights('14_ex_8.h5')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can make prediction for test data. Read the test.csv file and fill `NaN` with 0." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "rain2_test = pd.read_csv('datasets/rain2/test.csv')\n", "test_df = rain2_test.fillna(0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Test data has 23 columns not 24 columns. It doesn't have `Expected` column." ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Idminutes_pastradardist_kmRefRef_5x5_10thRef_5x5_50thRef_5x5_90thRefCompositeRefComposite_5x5_10thRefComposite_5x5_50th...RhoHV_5x5_50thRhoHV_5x5_90thZdrZdr_5x5_10thZdr_5x5_50thZdr_5x5_90thKdpKdp_5x5_10thKdp_5x5_50thKdp_5x5_90th
0118.00.00.00.014.00.00.00.0...0.00.00.00.00.00.00.00.00.00.0
1158.010.00.010.018.011.50.011.0...0.00.00.00.00.00.00.00.00.00.0
2188.00.00.07.014.50.00.07.0...0.00.00.00.00.00.00.00.00.00.0
31128.014.00.09.016.014.00.09.0...0.00.00.00.00.00.00.00.00.00.0
41158.010.50.09.015.513.50.09.0...0.00.00.00.00.00.00.00.00.00.0
\n", "

5 rows × 23 columns

\n", "
" ], "text/plain": [ " Id minutes_past radardist_km Ref Ref_5x5_10th Ref_5x5_50th \\\n", "0 1 1 8.0 0.0 0.0 0.0 \n", "1 1 5 8.0 10.0 0.0 10.0 \n", "2 1 8 8.0 0.0 0.0 7.0 \n", "3 1 12 8.0 14.0 0.0 9.0 \n", "4 1 15 8.0 10.5 0.0 9.0 \n", "\n", " Ref_5x5_90th RefComposite RefComposite_5x5_10th RefComposite_5x5_50th \\\n", "0 14.0 0.0 0.0 0.0 \n", "1 18.0 11.5 0.0 11.0 \n", "2 14.5 0.0 0.0 7.0 \n", "3 16.0 14.0 0.0 9.0 \n", "4 15.5 13.5 0.0 9.0 \n", "\n", " ... RhoHV_5x5_50th RhoHV_5x5_90th Zdr Zdr_5x5_10th \\\n", "0 ... 0.0 0.0 0.0 0.0 \n", "1 ... 0.0 0.0 0.0 0.0 \n", "2 ... 0.0 0.0 0.0 0.0 \n", "3 ... 0.0 0.0 0.0 0.0 \n", "4 ... 0.0 0.0 0.0 0.0 \n", "\n", " Zdr_5x5_50th Zdr_5x5_90th Kdp Kdp_5x5_10th Kdp_5x5_50th Kdp_5x5_90th \n", "0 0.0 0.0 0.0 0.0 0.0 0.0 \n", "1 0.0 0.0 0.0 0.0 0.0 0.0 \n", "2 0.0 0.0 0.0 0.0 0.0 0.0 \n", "3 0.0 0.0 0.0 0.0 0.0 0.0 \n", "4 0.0 0.0 0.0 0.0 0.0 0.0 \n", "\n", "[5 rows x 23 columns]" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ok. Let's check the number of sequences and maximum length just like above." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(717625, 19)" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_seq = test_df.groupby(['Id'])\n", "test_seq_size = test_seq.size()\n", "test_seq_size.count(), test_seq_size.max()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create `X_text` numpy array. Its size is (717625, 19, 22)." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "X_test = np.zeros((717625, 19, 22))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Just like train data, copy `test_seq` to `X_test`." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "717625\n" ] } ], "source": [ "i = 0\n", "for name, group in test_seq:\n", " # d.shape is (seq_length, 23)\n", " d = group.values\n", " # column 1~22 are features.\n", " # save 1~22 features to 0~21 index of dataset up to d.shape[0].\n", " X_test[i, :d.shape[0], 0:22] = d[:, 1:23]\n", " i += 1;\n", "print(i)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can feed `X_test` to our model and get the predictions!" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "pred = model.predict(X_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Making submission file, we stack `pred` array and its index horizontally. Using `numpy.savetxt`, we can make csv file with format(\"int, float\") and title(\"Id,Expected\")." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "pred_with_index = np.hstack((np.arange(1, pred.shape[0]+1).reshape(-1,1), pred))\n", "np.savetxt(\"datasets/rain2/test_prediction.csv\", pred_with_index, \"%d,%f\", \n", " delimiter=\",\", header=\"Id,Expected\", comments=\"\")" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Id,Expected\r\n", "1,0.473683\r\n", "2,0.250590\r\n", "3,2.757972\r\n", "4,5.341240\r\n", "5,0.266192\r\n", "6,0.785535\r\n", "7,2.634934\r\n", "8,0.696415\r\n", "9,0.169597\r\n" ] } ], "source": [ "!head \"datasets/rain2/test_prediction.csv\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you submit this file to Kaggle competition(https://www.kaggle.com/c/how-much-did-it-rain-ii/submit), you will get about 23.78 score. It may be ranked before 100. Cool!\n", "\n", "You can stack another `LSTM` layer and change weight initializer or optimizer. If you want to search more parameters, it's better to use `keras.wrappers.scikit_learn.KerasClassifier` with scikit-learn's `GridSearchCV`." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" } }, "nbformat": 4, "nbformat_minor": 2 }