{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Multiple Inputs in Keras\n", "> In this chapter, you will extend your 2-input model to 3 inputs, and learn how to use Keras' summary and plot functions to understand the parameters and topology of your neural networks. By the end of the chapter, you will understand how to extend a 2-input model to 3 inputs and beyond.This is the Summary of lecture \"Advanced Deep Learning with Keras\", via datacamp.\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- author: Chanseok Kang\n", "- categories: [Python, Datacamp, Tensorflow-Keras, Deep_Learning]\n", "- image: images/team_strength_model.png" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import tensorflow as tf\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "plt.rcParams['figure.figsize'] = (8, 8)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Three-input models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Make an input layer for home vs. away\n", "Now you will make an improvement to the model you used in the previous chapter for regular season games. You know there is a well-documented home-team advantage in basketball, so you will add a new input to your model to capture this effect.\n", "\n", "This model will have three inputs: team_id_1, team_id_2, and home. The team IDs will be integers that you look up in your team strength model from the previous chapter, and home will be a binary variable, 1 if team_1 is playing at home, 0 if they are not." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
seasonteam_1team_2homescore_diffscore_1score_2won
019853745666401781641
1198512674931777701
2198528835931763561
319851846988111670541
4198526751029811286741
\n", "
" ], "text/plain": [ " season team_1 team_2 home score_diff score_1 score_2 won\n", "0 1985 3745 6664 0 17 81 64 1\n", "1 1985 126 7493 1 7 77 70 1\n", "2 1985 288 3593 1 7 63 56 1\n", "3 1985 1846 9881 1 16 70 54 1\n", "4 1985 2675 10298 1 12 86 74 1" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "games_season = pd.read_csv('./dataset/games_season.csv')\n", "games_season.head()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
seasonteam_1team_2homeseed_diffscore_diffscore_1score_2won
01985288730-3-941500
1198559297304661551
2198598847305-459630
319857328803950411
41985392041001-954630
\n", "
" ], "text/plain": [ " season team_1 team_2 home seed_diff score_diff score_1 score_2 won\n", "0 1985 288 73 0 -3 -9 41 50 0\n", "1 1985 5929 73 0 4 6 61 55 1\n", "2 1985 9884 73 0 5 -4 59 63 0\n", "3 1985 73 288 0 3 9 50 41 1\n", "4 1985 3920 410 0 1 -9 54 63 0" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "games_tourney = pd.read_csv('./dataset/games_tourney.csv')\n", "games_tourney.head()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"Team-Strength-Model\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "input_1 (InputLayer) [(None, 1)] 0 \n", "_________________________________________________________________\n", "Team-Strength (Embedding) (None, 1, 1) 10888 \n", "_________________________________________________________________\n", "flatten (Flatten) (None, 1) 0 \n", "=================================================================\n", "Total params: 10,888\n", "Trainable params: 10,888\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "from tensorflow.keras.layers import Embedding, Input, Flatten\n", "from tensorflow.keras.models import Model\n", "\n", "# Count the unique number of teams\n", "n_teams = np.unique(games_season['team_1']).shape[0]\n", "\n", "# Create an embedding layer\n", "team_lookup = Embedding(input_dim=n_teams,\n", " output_dim=1,\n", " input_length=1,\n", " name='Team-Strength')\n", "\n", "# Create an input layer for the team ID\n", "teamid_in = Input(shape=(1, ))\n", "\n", "# Lookup the input in the team strength embedding layer\n", "strength_lookup = team_lookup(teamid_in)\n", "\n", "# Flatten the output\n", "strength_lookup_flat = Flatten()(strength_lookup)\n", "\n", "# Combine the operations into a single, re-usable model\n", "team_strength_model = Model(teamid_in, strength_lookup_flat, name='Team-Strength-Model')\n", "\n", "team_strength_model.summary()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras.layers import Concatenate, Dense\n", "\n", "# Create an Input for each team\n", "team_in_1 = Input(shape=(1, ), name='Team-1-In')\n", "team_in_2 = Input(shape=(1, ), name='Team-2-In')\n", "\n", "# Create an input for home vs away\n", "home_in = Input(shape=(1, ), name='Home-In')\n", "\n", "# Lookup the team inputs in the team strength model\n", "team_1_strength = team_strength_model(team_in_1)\n", "team_2_strength = team_strength_model(team_in_2)\n", "\n", "# Combine the team strengths with the home input using a Concatenate layer, \n", "# then add a Dense layer\n", "\n", "out = Concatenate()([team_1_strength, team_2_strength, home_in])\n", "out = Dense(1)(out)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Make a model and compile it\n", "Now that you've input and output layers for the 3-input model, wrap them up in a Keras model class, and then compile the model, so you can fit it to data and use it to make predictions on new data.\n", "\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# Make a model\n", "model = Model([team_in_1, team_in_2, home_in], out)\n", "\n", "# Compile the model\n", "model.compile(optimizer='adam', loss='mean_absolute_error')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fit the model and evaluate\n", "Now that you've defined a new model, fit it to the regular season basketball data.\n", "\n", "Use the `model` you fit in the previous exercise (which was trained on the regular season data) and evaluate the model on data for tournament games (`games_tourney`)." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "138/138 [==============================] - 0s 2ms/step - loss: 12.2876 - val_loss: 11.5519\n", "11.686502456665039\n" ] } ], "source": [ "# Fit the model to the games_season dataset\n", "model.fit([games_season['team_1'], games_season['team_2'], games_season['home']],\n", " games_season['score_diff'],\n", " epochs=1, verbose=True, validation_split=0.1, batch_size=2048)\n", "\n", "# Evaluate the model on the games_touney dataset\n", "print(model.evaluate([games_tourney['team_1'], games_tourney['team_2'], games_tourney['home']], \n", " games_tourney['score_diff'], verbose=False))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summarizing and plotting models\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Model summaries\n", "In this exercise, you will take a closer look at the summary of one of your 3-input models available in your workspace as model. Note how many layers the model has, how many parameters it has, and how many of those parameters are trainable/non-trainable.\n", "\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"model\"\n", "__________________________________________________________________________________________________\n", "Layer (type) Output Shape Param # Connected to \n", "==================================================================================================\n", "Team-1-In (InputLayer) [(None, 1)] 0 \n", "__________________________________________________________________________________________________\n", "Team-2-In (InputLayer) [(None, 1)] 0 \n", "__________________________________________________________________________________________________\n", "Team-Strength-Model (Model) (None, 1) 10888 Team-1-In[0][0] \n", " Team-2-In[0][0] \n", "__________________________________________________________________________________________________\n", "Home-In (InputLayer) [(None, 1)] 0 \n", "__________________________________________________________________________________________________\n", "concatenate (Concatenate) (None, 3) 0 Team-Strength-Model[1][0] \n", " Team-Strength-Model[2][0] \n", " Home-In[0][0] \n", "__________________________________________________________________________________________________\n", "dense (Dense) (None, 1) 4 concatenate[0][0] \n", "==================================================================================================\n", "Total params: 10,892\n", "Trainable params: 10,892\n", "Non-trainable params: 0\n", "__________________________________________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plotting models\n", "In addition to summarizing your model, you can also plot your model to get a more intuitive sense of it." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEVCAYAAAA1lUZ4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de1xVZdo//s8FbBFBAYnkIJ4a7aCGjWiWkqallhYMcXCy1PlW+ow1OeNkWvZ8c35jM/atpnGealKrqZ6cFKwsDWvQzLKhEgLNE6aTJoqJeEhIlMP1+2Mf2hs2sBE2a7H5vF+v/WKve611r+u+19r7Yh32WqKqICIiInPwMzoAIiIi+gkTMxERkYkwMRMREZkIEzMREZGJMDETERGZCBMzERGRiXgtMYvIRBEpEpH9IrLAW8shIiLyJeKN3zGLiD+AfQBuBlAMYBuAX6rq7lZfGBERkQ/x1h7zcAD7VfU/qnoBwCoASV5aFhERkc/wVmKOBXDYabjYVkZERESNCPBSveKmzOWYuYjMBDATAIKDg4deccUVXgqFiIjIfPLz80+oamTdcm8l5mIAcU7DPQEcdZ5AVZcDWA4ACQkJmpeX56VQiIiIzEdEDrkr91Zi3gagv4j0BXAEwBQAd3oyY1pampdCIvJcVlbWRc/LbZjMoCXbMBnLK+eYVbUawAMAPgSwB0Cmqu7yZN41a9Z4IyQij7V0G1yzZg2Ki4tbKRqi5vv888+NDoFawFt7zFDVbADZFzMv/9MjI4m4u0SieX73u98hPT29FaIhaj4etWnfeOcvIiIiE2FiJiIiMhEmZiIiIhNhYiYiIjIRJmYiIiITYWImIiIyESZmIiIiE2FiJiIiMpF2l5iHDBkCEWnwtXjxYqNDBABkZ2cjIKDp+7eEhIS4xN9anOt8+umnW63ei+HcRqNjMQOzb8OnTp3C2LFj0b17d/Tv3x9Tp05tdPq623BrrWNuw9RRtbvEDACqClXFrFmzAAAbNmxwlJnB7bffjkceecSjacvLy1FQUICkpKRWjb+goACAta8eeuihVqv3YpSXlwMAkpKSDI/FLMy8Dc+bNw9JSUnYvXs3XnnlFRQWFmLt2rUNTl93G26tdcxtmDqqdpmYze76669Hfn6+0WEYIiQkxOgQqBXMmTMHUVFRSExMxD//+U88/PDDRofUZrgNk9G8dq9sbyksLGx0/GOPPdZGkTRswYIFRodAJmb2bfill15yGf7Zz36GoKAgg6Ih6nh8co+5tLQUffr0QadOnZCSkuLyRVhdXY2bb74ZUVFRGDx4MJYuXeoY53xOKyMjA127dsXdd9+NU6dO4bbbbkPXrl1x33334ezZs16Lfe3atY4YDh48iIyMDERERGDy5MmtVm9GRgbCwsIwefJkHDhwAADw9NNPQ0TQs2dPbNu2DV27dkWXLl3w2WefAQAWL14MEcGoUaMcddY9Ly4iqKioaPb5cud1EhQUhKVLl6K2thanT5+ud/69urraMZyamgqg4fXt3OaioiJERERARHDixIkW9WVbMNM2nJWVhYULFzYrfm7D3IapBezntYx8DR06VO2sIXlm1qxZCkA3bNjgKJs+fbpLHSUlJRoYGOgYXrdunUsdd911l545c8YxnJSU5DL/wIEDXYb79u2rl19+uUfx+fv7ezRdQUGBJiUluZQlJSW5xJqamqqlpaWO4dGjR2t4eHijddbty7ptS01NdRmOj49XAFpQUOAoA6Dx8fGO4ZEjR7rU6bzuVFWDg4PdxgOgXhvt3K0Ti8XiMu/s2bMdw1u3btULFy6oatPr2z5/dna222U3FGtLANDVq1d7NK3Zt+EhQ4ZoRkZGk9NxGzbXNpyamurxtGQcAHnqJif63B7z2rVr4ef3U7OioqIwcOBAx/Nx6/7XHh8fj127Gn5UdExMjMtwbGwsjh49CgDYuXNns6+odp7+gQceaHTaYcOGOd7HxcU5lgsAH3/8MU6ePOnRMhsSFxdXryw4OBhDhgxxDMfExGD79u0tWk5T3K2Tqqoqx/DgwYPx6quvOoafeuopWCwWAE2vb7vhw4d7IXLvMMs2XFFRgauuugorV650Kec2XB+3YWpN7e4cc1NExGUjB+ByIdYNN9yA1NRU/PKXv0RkZCQCAgJQUVHRYH1+fn7w9/d3DPv7+6O2thYAMGjQoGZfRduc6UNDQx3vO3Xq5Fhua+nUqVO9spqaGqhqg/9oXLhwwWX49OnTLsMX85Mv53VyySWX1PuZ2bZt29C7d2+cP38eDz74oMuXcVPr2y44OLjZcRnFDNvwPffcg4MHD2LTpk0ArOeZ33jjDYwYMYLbsBvchqk1+dwec0pKCqqrq13KnnzySVRXV6OmpgafffYZHnzwQURGRgIAzp07Z0SYplVZWYlt27Y5ho8ePYr4+HjH8JEjRxzvjx07hu+++85l/i5dujjeL1++vMnl7dq1y2WdiEi9dRIYGIjZs2fjmWeewcqVKzFnzhzHuMbWd3tlhm14165dePfdd1u93rbAbZjaPXfHt9v6dTHnmP/xj38oAMfr7NmzjnFlZWXar18/tVgsOn78eM3JyXGMKy0t1bi4OLVYLDpjxgxdsGCBAtChQ4e61Ldw4ULdtm2bY/jPf/6zfvrpp47hxx9/vLHzBi6vFStWNDhtcHCwy7S5ubn14nAenjRpkqqqJiYmNnp+znmep556yqXehQsXukxjrzM+Pl5jY2N19+7d2rVrVw0KCtKtW7e61HvvvfdqdHS0BgUF6bZt2xz9Nn/+fFVV3bt3rwYHB2tcXFyDbXR+7dmzx2Wd9OjRw2Wd1G3Tli1b6rW1ofVdty891ZxpG5rfk3PMZt2GJ02a5HZd5ebmup2+7vp96qmnuA0bvA3zHHP7gAbOMYua4IYGCQkJmpeXB8B6WMcMMXVEQ4YMwYkTJ+qd2zKLhIQE2LcTb2rpNigiWL16NdLT01sxKvIEt2GrtLQ0ZGVleX051DIikq+qCXXLfe5QNvmuuXPnGh0CUYtwGyZPMDETAOtvQLdv344jR4606j27W+qll16CxWJBfHw87rzzTqPDIRPjNky+goeyiergoWxq73gou33goWwiIqJ2gImZiIjIRJiYiYiITISJmYiIyERadEtOETkI4CyAGgDVqpogIt0BrAbQB8BBAOmqeqplYRIREXUMrbHHfKOqDnG6smwBgE2q2h/AJtswERERecAbh7KTALxme/8agGQvLIOIiMgntTQxK4B/iUi+iMy0lfVQ1RIAsP291N2MIjJTRPJEJK+0tLSFYRAREfmGlj72caSqHhWRSwHkiMheT2dU1eUAlgPWG4y0MA4iIiKf0KLErKpHbX+Pi8g7AIYD+F5EolW1RESiARxvTp2pqalIS0trSVjkBe+88w4GDhyIAQMGGB2K16WmprZ4/qysLJ+/81JRURF2796NX/ziF0aHQuRTLvqWnCISDMBPVc/a3ucA+P8AjANQpqpLRGQBgO6q+nBjdTnfkpPMaenSpXj44Ydx6NAhREVFGR0OGay0tBQxMTFYuHAhFi1aZHQ4RO2SN27J2QPAVhHZDuBLAO+r6gcAlgC4WUS+AXCzbZjauTlz5uD8+fP43e9+B39/fyxYwIvtO6Lq6mr4+/vj17/+NaqqqpiUibzAdA+xIPN7/fXX8etf/xr79u1DbGys0eFQGzl06BCmTp2KtLQ0zJkzx+hwiNo9PsSCWs20adNQUVGBP/zhD/Dz88OFCxeMDom8qLKyEiKC//7v/8a//vUvJmUiL2Nipou2fPlyvPrqqxg5ciQOHDhgdDjkBbt378bw4cOxatUqvP766+jSpYvRIRH5vJb+XIo6uGnTpiE9PR3z58/HqVOn8OKLL/LL2wecOXMGs2bNQo8ePZCfnw+LxWJ0SEQdBs8xU6vp3r07YmJisHPnTqNDoRa67LLLUFFRgWPHjhkdCpHP4jlm8rrCwkKEhoZixYoVRodCF0lVsXTpUlx22WUoLCw0OhyiDomJmVpNr1698Nlnn6Fv376IiorC9u3bjQ6JmmHjxo3o168frr32WvzrX//i79WJDMLETK3upptuwvbt2zFixAgsXbrU6HDIA4sWLcKECRNQUFCAESNGGB0OUYfGxExe0aNHD5w7dw7x8fGIjY1Fbm6u0SGRG1lZWQgPD8edd96JmpoahIWFGR0SUYfHxExeNWbMGBQWFuKGG27gXaJMZs6cOcjIyMC0adM6xD3QidoLJmbyusjISFRVVWHSpEmIi4vDJ598YnRIHdry5cvRpUsXPPzww6itreXpBiKTYWKmNjNs2DCMHDkSY8eORU1NjdHhdEgnTpzA7Nmz8fDDD/N2qkQmxRuMUJtatWoVAGDAgAE4ffo0jh9v1lNB6SLV1NQgMDAQkydPRnV1tdHhEFEjuMdMhsjPz8eECRMwZ84c3mvby4qLizF27Fg888wzeOedd4wOh4iawDt/kaFCQkJw1VVX4csvvzQ6FJ91ySWXIDIyEnv27DE6FCJywjt/kSmVl5fjk08+gYhg2rRpOHfunNEh+YxrrrkG3bp1w4kTJ5iUidoRJmYyXOfOnfHWW29h/fr1GDlypNHh+ITXX38dAQEB+Oqrr4wOhYiaiYmZTCElJQUnT57El19+CX9/f0ybNs3okNql/v3749JLL8Ull1yCbdu24Wc/+5nRIRFRMzExk6kEBARg7dq1yM7Oxo4dO4wOp92wP3yiV69eKCwsxK233mp0SER0kZiYyXRuu+02nDhxAm+//TYCAgJw+vRpo0Mytc2bN6NPnz5ISEjApk2bEBMTY3RIRNQCTMxkWosWLUJOTg6GDBmCzz//3OhwTGnRokW46aabUFhYyPPzRD6CiZlM7cYbb8S3336LL774ArfffjvKysqMDskU3nnnHURERCAlJQU1NTUIDw83OiQiaiVMzGR6IoI5c+Zg586dGDJkCD799NN60/ji7+BLS0tRUlJSr7yyshIpKSmYNGkSrr76agMiIyJvYmKmdqOgoADXXXcdbrzxxnr32h4zZgz+85//GBSZd9x7772YOnWqS9mePXtw7bXX4s0338Trr79uUGRE5E28Vza1G6GhocjMzATw0722v/vuOwwdOhQVFRXo37+/zzwco7i4GOvWrYOq4t///jcGDRqEsLAw/OY3v0FeXh4sFovRIRKRlzAxU7uUl5eHWbNmITExEfv27QMA1NbWYteuXRg4cKDB0bVMbW0tpk6dCvvtcpOTkxEUFIScnByMGzfO4OiIyNuaPJQtIq+IyHER2elU1l1EckTkG9vfcFu5iMjfRGS/iOwQkZ97M3jquLp164bFixcjPz/f5WlJ8fHxWLx4sYGRtVxQUJDLM6tLS0tx+PBhJmWiDsKTc8yvAphYp2wBgE2q2h/AJtswANwCoL/tNRPA31snTCJXVVVVSE9PR92HsNTU1GDRokXGBNUKqqur67UJsN5AhDdcIeoYmkzMqvoJgJN1ipMAvGZ7/xqAZKfy19XqcwBhIhLdWsES2U2ePBk7duxAQED9szE1NTV49NFHDYiqZfLy8tC5c2dUVVXVG2exWDBx4kS8//77BkRGRG3pYq/K7qGqJQBg+3uprTwWwGGn6YptZUSt6sMPP8SZM2cwZcoUhISEoFOnTi7jlyxZgq1btxoU3cW56667ICKOYX9/fwQEBKB///6YO3cujhw5gkmTJhkYIRG1hda++EvclLl94LOIzIT1cDd69erVymG0b7m5uTh8+HDTExJuu+023HbbbY7hrKwsfP755zhy5AgSExNxyy23YMaMGcYF6KFdu3ahqKgIAHDDDTdg2LBhGD58uMs0WVlZRoTWrsTFxeG666676Pn52SNvS09Pb3oiVW3yBaAPgJ1Ow0UAom3vowEU2d4vA/BLd9M19ho6dKjST1JTUxXWf2j44ouvZrxSU1P52ePL1C9nAPLUTU682D3m9wBMB7DE9vddp/IHRGQVgGsBnFHbIW9qntTUVO4hETVTWlpai+vgZ4+8xdPts8nELCJvAhgD4BIRKQbwOKwJOVNE7gHwHQD70rIB3ApgP4AfAfyquYETERF1ZE0mZlX9ZQOj6v2o0rZrfn9LgyIiIuqoeK9sIiIiE2FiJiIiMhEmZiIiIhNhYiYiIjIRJmYiIiITYWImIiIyESbmdmjIkCEQkUZfRqupqcH111+P0NBQWCwW3HrrrXjuueeMDqtZ1q5d26z+LC8vd1kHubm5bqebN2/eRa+nVatWQUTQuXNnj6Z/+umnISLo2bOnR9PXbUND2rIN9uV42gZvCwkJcdtHPXv2dCl/7LHHDIzSyjnWp59+utXrNPq7pm4srdVGozExt1NZWVmO27fNmjULGzZscAxnZGQYHR7uvvtuJCcnY9euXTh79iyuueYaPPjgg0aH1SzJyclISkryePqQkBCXRzb+8Y9/rDdNWVkZXnzxRUydOtXt4x2bMmXKlGbN99BDDyE+Pt7j6eu2wZ3WaENzni2tqs1qg7eVl5ejoKAASUlJLu0vLi7Gt99+C8AasxmeC15eXg4ASEpKwkMPPdRqdbprvxHssQDWPm+tNhqttR9iQYRt27bhzTffxD//+U9H2RNPPIHTp0+7TBcSEuL44jCD1oonKCgIl156KTZs2FBv3LPPPou4uLgWL8Pb7G3Iy8tDQkKCy7j20gbyPaNGjWp3T427GNxjbocKCwuRmpra4PhVq1a1YTT1nThxwm35888/38aRGMPPzw9FRUWIjY3Fjh07XMYdOnQIy5cvNygyz9nbMHz48HbbBqL2ionZR5WWluLBBx9Ep06dEBkZiZSUFMe46upqrF69GlFRUQgKCsLSpUtRW1sLwPW86qFDh9C1a1dERETg1KlTOHjwILp27Yro6Gjcd999DS47MTERUVFRuOWWW/Dxxx876nYmIqioqHA5T+W87KKiIkREREBEHIn+wQcfRJ8+fdCpUyekpKSgsLCw3nwZGRkICwvD5MmTceDAAZdlJicnIzQ0FMOHD8f69etdzku5i8dZY/W6ExgYiHnz5uGJJ55wlJWXl+PRRx91O31ZWRkuu+wydOrUCbfccgs2b97sGLd3714kJycjODgYiYmJbvcY7H1jX9f2vmmJwMBAqKrHbZg7d67HbXCnbht8RUPr9mI/a6WlpW4/B55wXubBgwcRFhaGiIgIj7ZpT+vMyMhAREQEJk+e7JjG+TqBcePGoWvXrrjxxhvx2WefAQAWL14MEcGoUaMAAB988AFEBJdccgmAn66V+OyzzyAiCAjw/GDv6tWrcfPNNyMoKAiDBw/G0qVLAQCnT592xGU/7eDu3HlD26XztOnp6fW+r1rE3SOn2vrFxz66Sk1Nbdbj62bNmqUbNmxwKQOgK1eudAyXlJSovZ/XrVunY8aMcYy766671GKx1Jv//fffdxnesmWLY7hv375NxvXaa69pUlKSdu3aVQFoRESEy/jg4GC38wHQ7Oxsl7Lp06fXa09gYKBjOCkpyeWRavbH95WWlqqqalpamq5Zs8Yx/vjx4y7zNxRP3XrvvPNOl3rdca4HgN51111aUVGhPXr0UFXVTz/9VKdOneqYZsaMGS7LqKys1JiYGD127JijDufYjxw54hJ7U30THx+vsbGxjuHRo0dreHh4g/E7t+HJJ590xNZUG958802P2xAYGNhkG5y/F+q2oSGt8dhHT+ooKCjQpKSkeuXffvuty7psaN0GBQU5yprzWZs+fbpLfXXXtTsA6sUKQNetW+cy7LxNh4eH67///e8G63TX/qSkJJc6U1NTXeqMj493iX3Hjh0uw8HBwTpy5EiXOut+Z9Qdb48FdR6n6Mz5u07V+n135swZVVX98MMPXebdunWrxsbG6oULF1S16e2y7vdDU+puW2jgsY/cY/ZRfn5+Lv+xRkVFIT8/H8XFxZg8ebLL3kx8fDyqqqrq1VH33KLzcGxsrOO983+ODzzwgKN82rRpWLt2LU6ePIkpU6agrKzMcaFGU4YPH+4yvHbt2nrtGThwIIqLi93Obz8HevToUQDW/8AnTJjgGB8ZGelRHHXZ23306FHs3LmzyatTIyIi8Oabb2LZsmUYMWKE22neeecdl+HAwECMGzcOH374oaPMOfaYmBiX6ZvbNx9//DFOnjzZSCt/Mnv2bERERGD//v1NtmHSpEket2HAgAFNtsG+vbZnDa3bc+fOuZR7+llbu3Yt/Px++tp2XteebI/Ohg0b5jJs/6wAwMmTJ3Hdddc1WUdjdcbFxbnUCQDBwcGO94MHD0ZMTAxKSrz7ZGDn7zrA+n23a9cuAMD48eMxePBglJWVAQCeeuop/OY3v4HFYgFg3HbJxOyj/Pz80KVLF5cyVUXPnj1xww03QERQWlpqfSh3A4eFunXr5njv7+/vUp+/v79LvfbXc889h88++ww9evRwjA8ICMCbb76JDz74ACtWrHCUN/bl4fwBtk9btz35+fkN/oSmU6dOAOD2MHpDPPkys38p1tbWYtCgQS5td+fAgQMICQlBTk4O1q5d63EszdHcvmmOkJAQHDhwAP3792/VNtTd5ty1wb69dgSeftZExCUxAz+ta0+2R2ehoaEuw835rHhSZ6dOnerVWVNT0+C8fn5+uHDhQotjqEtE8Le//c3RJwEBAaioqHCM37ZtG6666iqcP38ecXFxmD9/vsu8RmyXTMw+qrq62nH+xq5Xr16O8qioKERGRkJE6v333lKqiuPHj9crz8vLwzXXXOMYdt7gm7qYKCUlpV57nnzySVRXV3sU0y233IIPPvjAMXzs2LF60zQnHk+FhoZi7ty5jf6m9Re/+IXL8Pnz57Fp0yaXPUzn2Ouew2pp3zQlNDQUoaGhTbbh/fffdww31YaioiKX+d21wb69tmcNrdugoKCLqi8lJaVen7Tmuva2yspKx/uvv/4aR48eRXR0NAAgOjoaR44ccYx39xm1J+7LL7+8yc9oQEAAdu3ahaioKDz44IOOf7zrft8FBgbi+PHjeOaZZzBnzhyXcYZtl+6Ob7f1i+eYXTXnHDMAl5ddWVmZzp07Vy0Wi0ZGRur48eMd40pLS3XWrFlqsVi0R48eumDBAgWgQ4cO1dzcXJf6tm3b5jL86aefugw//vjj9WKqqanRrVu36rXXXqsxMTEaEBCgCQkJ+qc//cllur1792pwcLDGxcWpqtZbdl1z587Vfv36qcVi0fHjx2tOTk69+RYuXFivX+ySk5O1W7duev311+uWLVu0S5cuHsfTWL3OgoODHeMnTJjg8fo6ceKE9u3bVy0Wi06YMEE3bdrkGFdUVKTJyckaFBSkw4YN0/Xr19eb39439nVt75unnnqqXhsSExMbPcfs3AZ3GmrDb3/7W4/bMG7cuCbb4G559jY0pC3OMTv3j3P8sbGxbmNtaN1e7GetrKzM7eegqVifeuopt58xd21p7Bxz3fbXrXPhwoUuw5MmTVLVn64TmDBhgnbt2lVHjx6tW7duddR7+vRpvffeezUoKEhHjRrl0h92iYmJGhwcrM8//7zbWOq+9uzZo7NmzdK4uDjt0aOHzpgxw+X7zu6+++5rcHv3ZLtsaN66PD3HLGrwD8QBICEhQfPy8owOwzTS0tIAWG8iQt7Tu3dvHDp0yOgwqBWlpaW16HPDz573DBkyBCdOnDDldQP/+Mc/8Pzzz8Pbeaju9iki+aqaUHc6HsqmDuHYsWMuF7gdPHgQY8eONTAiIjKLF198EXPnzjU6DAfe+Ys6hKioKKSkpKCgoADnzp3DTTfdhDfeeMPosIg6BOcLKx977DFT3K4UgOM30V988QV+/vOfGx2OAxMzdRhvv/220SEQdUhmOGXqjlnj4qFsIiIiE2FiJiIiMhEmZiIiIhNhYiYiIjIRJmYiIiITaTIxi8grInJcRHY6lS0SkSMiUmh73eo07hER2S8iRSIywX2tRERE5I4ne8yvApjopvxZVR1ie2UDgIhcBWAKgIG2eV4QEX838xIREZEbTSZmVf0EgGfPiAOSAKxS1fOq+i2A/QCGNzEPERER2bTkHPMDIrLDdqg73FYWC+Cw0zTFtjIiIiLywMXe+evvAP4I61M1/gjgGQD/B4C7B9q6vbWKiMwEMBOwPkaLXK1Zs8aj5wMT0U9SU1NbXAc/e2S0i0rMqvq9/b2IrACw3jZYDCDOadKeAI42UMdyAMsB69OlLiYOX8Un23Rc586dQ0hICCZOnOjyfGNqG7702RswYABKS0tx6tQpo0OhZrqoQ9kiEu00+AsA9iu23wMwRUQCRaQvgP4AvmxZiEQdR1BQEGpqapCVlQV/f39MnjzZ6JConenfvz+6d++Offv2MSm3U578XOpNALkALheRYhG5B8D/E5GvRWQHgBsB/A4AVHUXgEwAuwF8AOB+Va3xWvREPqpLly7YsmULioqKsHz5cqPDoXbgxx9/xIIFC3DFFVdg586dTc9ApiVmeLpGQkKCevsB1UTt1aeffopf/epXePzxx3H33XcbHQ6ZzA8//ICwsDCkpqbi+eefR2RkpNEhkYdEJF9VE+qW885fRCaXmJiIr776CtOnT0d6ejpKS0uNDolM4sMPP8TgwYORmZmJzMxMJmUfwcRM1A5069YNtbW1eOmll/DYY48hPT0dJ06cMDosMsgHH3yAXr16oby8HIcOHWqVq9HJPHgom6gd6tWrF86fP4/vv/++6YnJp5w5cwbh4eFITU1FZmam0eFQC/BQNpEP+frrr5GcnIz09HSUlZUZHQ61kQ0bNmDQoEFYs2YNk7IPY2ImaodCQ0OxbNkyrFixAo8++igTtI/Lzs5GXFwcKisrcfjwYaSkpBgdEnkRD2UT+YCePXuipqYGJSUlRodCrez06dMIDw9HWloa95J9DA9lE/mw4uJilJSUoGfPnoiOjsZ7771ndEjUQrNmzYKIYObMmVBVJuUOhImZyIfs3LkTt99+O5KSknDypKcPhSOzWb9+PdatW4d3332XCbkDYmIm8iFhYWFYtmwZVBWPPPIIRIS3ZWxH1q1bh9jYWKgqjh49ittvv93okMgATMxEPmrZsmVYt24dBg4ciPXr1zc9AxkqLS0NSUlJ2LlzJ2677TajwyEDMTm1K5UAABnoSURBVDET+bDJkyfj6NGjOHfuHPr06YONGzcaHRLVkZqaCj8/P7zyyiuora1FeHh40zORT2NiJuoA0tLSkJCQgPHjx+Ps2bNGh0M2WVlZyM/PR05ODrp27Wp0OGQSTMxEHcSaNWtQW1uLGTNmwM/PD+Xl5UaH1GFlZWWhX79+iIiIwLfffotx48YZHRKZCBMzUQfz1ltvYfXq1bj66quxefNmo8PpUI4dO4aUlBRkZGRgx44dGDt2rNEhkQkxMRN1QGlpaRgyZAjGjRuHWbNmGR1OhzFo0CAUFhZi06ZNCAkJMTocMqkAowMgImO8/fbbAKx7cX5+frjvvvuwbNkyg6PyTREREQgLC+MTwcgj3GMm6uCioqKwevVqrFmzBlu2bDE6HJ9SUlKC5ORkpKamYseOHUaHQ+0EEzMRIS0tDWVlZTh+/Di6d++OTz75xOiQ2rWSkhLcfvvtWLRoEVauXIlly5YhODjY6LConeBDLIjIhf2e2/feey8PbV8EVUVERAQiIiLwzTffGB0OmRgfYkFEHomOjkZNTQ0effRRBAQEYMGCBUaH1C4sX74c3bp1w6BBg3Dy5EkmZbpoTMxE5Fbv3r3xwgsv4IUXXsC2bduMDsfUDh48iPvvvx+zZ8/GV199ZXQ41M4xMRNRg2bOnIkffvgBBQUFjr1BZ1988QXi4+Nx/vx5gyJsOyNHjoS/v79L2cGDB2GxWPDiiy+iqqoKS5YsQWBgoEERkq9gYiaiJs2cORM7duxAjx49sGDBApw/fx6VlZWYOnUqdu/ejd///vdGh+h1X3zxBQCgsrISgPXQ9eDBg5Gbm4slS5YYGRr5GF78RUQeU1V069YNffr0wcSJE/HXv/4V1dXVjnG+Kjs7G5MmTQIAPPzww/iv//ovDBgwAL///e+ZlOmiNXTxFxMzETXbjz/+iG7duqGmpsZRduWVVyI/Px9BQUEGRta6qqqqkJiYiK+++gpVVVUAAH9/f8TExOC7774zODpq73hVNhG1mscee8wlKQPAN998g3nz5hkUkXfMnz8feXl5jqQMADU1NSgpKTEwKvJ1Te4xi0gcgNcBRAGoBbBcVZeKSHcAqwH0AXAQQLqqnhIRAbAUwK0AfgQwQ1UbvUyRe8xE7Uffvn1x6NAht4euRQRdunTxiSdX/ec//0H//v1RW1vrdvyAAQOwY8cOXuxFF60le8zVAH6vqlcCGAHgfhG5CsACAJtUtT+ATbZhALgFQH/bayaAv7dC/ERkEqNGjcIll1yCgIAABAS43m5fVXH+/Pl2f5X2+fPnkZKS4jYpWywWAMBll12GoqKitg6NOoAmE7Oqltj3eFX1LIA9AGIBJAF4zTbZawCSbe+TALyuVp8DCBOR6FaPnIgM8b//+784fvw4qqqqUFVVha+++gpPP/2049yyn58fOnfujKysLIMjvXidO3fG9u3bAQCBgYEYPXo0/v3vf6OqqgoXLlyAqiI7OxtXX321wZGSL2rW06VEpA+AawB8AaCHqpYA1uQtIpfaJosFcNhptmJbGU/KkE/KzMxERkaG0WGYxoULFwAA6enpBkfSOs6fP48tW7bg+uuvNzoUQ5nhQuGOwuPELCIhAN4C8FtV/cF6Ktn9pG7K6q1REZkJ66Fu9OrVy9MwiEyLX1zkqzIzM40OoUPx6KpsEbHAmpRXqurbtuLv7YeobX+P28qLAcQ5zd4TwNG6darqclVNUNWEyMjIi42fiIjIpzSZmG1XWb8MYI+q/sVp1HsAptveTwfwrlP5NLEaAeCM/ZA3ERERNc6TQ9kjAdwN4GsRKbSVPQpgCYBMEbkHwHcA0mzjsmH9qdR+WH8u9atWjZiIiMiHNZmYVXUr3J83BoBxbqZXAPe3MC4iIqIOiXf+IiIiMhEmZiIiIhNhYiYiIjIRJmYiIiITYWImIiIyESZmIvIZM2bMQN++fREUFIRBgwbhjjvuwIEDB4wOi6hZmnWvbCLyLSEhIRgyZAi2bt3apsts7cdCLly4EH/605/q3Ra1pqYGAQEBqKqqqvckrLZkVD+39TKpdXCPmYjavSVLlmDo0KH1yv39/Q2IhqhluMdMRO1ebW0t0tLS3I7jw0WoveEeM1EbOHbsGPr164f169fj7NmzWLx4MexPaHMe98MPP+COO+6An99PH83k5GSICHJzc1FRUYGNGzdi+PDhLnVHR0dj/fr1+P77713qXr9+Pf785z/j1KlT+J//+R+P9iBLSkoQFRWF7Oxs7Nq1C2PGjEFubm6D8XTp0sUlHm8sc+zYsYiIiGi0jhtvvLHR8XX7ed++ffDz88NLL73kmEZEkJyc3GA/N7QOndt84sQJ+Pv7Y968eU22uU+fPsjOzkZ5eTnGjBnjeKY18FM/NxSPu35u6TLJJFTV8NfQoUOVqL1avXq1Wj9KDZsxY0a9aSZOnOh2XGVlpcbExOixY8dUVTUpKanevAC0tLTUMf/q1avd1r1u3TqX8rvuukvPnDnjGA4ODtaRI0e6TDN9+nRduXKlY7ikpESdP6N140lNTXUZ9mSZdTW1zNGjR2t4eHi9+ewA6JdfftngeFX36yAmJkaDgoJc6nGOv24/N7QO3bXZYrE4hhvqZ+f6SkpKNDAw0DFs7+eG4vHGMhtSd/ui1gEgT93kRMOTsjIxUzvnSWIGoJ07d65XXllZ6XbcQw89pE8++aSqNpyYCwoKHPM35PTp0zpo0CANCwtTWJ+Lrhs3bnSMr/vlba/P3eujjz5yG8/8+fNdhj1Zprs+aGyZTQGgS5YsaXIad/3sHDsAPXfunMtwQUFBg/M31mbnet0lSU/7uaF4mrvMlvQzE7N3NJSYeSibqA2EhoaisrKyXnlgYKDbcd9//z2ioqKarNc+/9mzZ92Ov+2223Dfffdh3759UFU8++yzLudc7YdinesLCwtDVVVVvS+Lpg4VG7nMgIAArFmzxu04Pz8/7N27t8F+9lRD6xBwbXNtbS2effZZl/F12wwAYWFhCAgIaNV+bmyZ9n5uyTKpbTAxE7WBvXv3om/fvnj//fdx9uxZzJ4925F47ePs5y7vuOMOvPHGG5g2bZrHdV9++eV4//33UVxc7FK3xWLB+fPnISLYvHlzvS/vUaNGYefOncjNzYXFYgEAFBUV4fLLL8eGDRtw8uRJLFu2DJmZmR631ZNlHj58uFnLbOocc1VVFW666SYsWLAA+/btw4ULF/Dyyy8jJiYGZ8+exRVXXFGvn/ft24c33ngDy5cv96hdja1D5zZXVlY22c979uxBUVERevfujQ0bNuDMmTNYtmwZgoODPYql7jKbs25bskxqI+52o9v6xUPZ1J55cihbVfXEiRPat29ftVgsOmXKFN23b5/bcRMmTNBNmzapqmpubq7jcOPChQtVVd0etvztb3+rffv21ejoaJe6S0tLNS4uTi0Wi86YMUMXLFigABznb/fu3auJiYkaFxenzz//vKO+uXPnar9+/TQyMlLHjx/vKG8snkmTJnm8zODgYI+XqaqamJjY6Dlmu7vvvtux7BEjRujSpUsbXAehoaFu+9ner87D9rY1tA6d29yjRw+P+7msrEz79eunFotFx48frzk5OR7F05J129AyG8ND2d6BBg5lizodYjJKQkKC5uXlGR0G0UXJzMxERkYGzPBZIvKGzMxMpKenGx2GzxGRfFVNqFvOQ9lEREQmwsRMRERkIkzMREREJsLETEREZCJMzERERCbCxExERGQiTMxEREQmwsRMRERkIkzMREREJtJkYhaROBHZLCJ7RGSXiMyxlS8SkSMiUmh73eo0zyMisl9EikRkgjcbQERE5EsCPJimGsDvVfUrEekKIF9EcmzjnlXVp50nFpGrAEwBMBBADICNIjJAVWtaM3AiIiJf1GRiVtUSACW292dFZA+A2EZmSQKwSlXPA/hWRPYDGA4gtxXiJTItd4/2I/IVvFd22/Fkj9lBRPoAuAbAFwBGAnhARKYByIN1r/oUrEn7c6fZitF4Iidq166//nqsXr3a6DA6lIyMDPY5+SyPny4lIiEAtgB4QlXfFpEeAE7A+hiyPwKIVtX/IyLPA8hV1Tds870MIFtV36pT30wAMwGgV69eQw8dOtRabSIiHycifJoXtXsterqUiFgAvAVgpaq+DQCq+r2q1qhqLYAVsB6uBqx7yHFOs/cEcLRunaq6XFUTVDUhMjKyea0hIiLyUZ5clS0AXgawR1X/4lQe7TTZLwDstL1/D8AUEQkUkb4A+gP4svVCJiIi8l2enGMeCeBuAF+LSKGt7FEAvxSRIbAeyj4IYBYAqOouEckEsBvWK7rv5xXZREREnvHkquytANxdbprdyDxPAHiiBXERERF1SLzzFxERkYkwMRMREZkIEzMREZGJMDETERGZCBMzERGRiTAxExERmQgTMxERkYkwMRMREZkIEzMREZGJMDETERGZCBMzERGRiTAxExERmQgTMxERkYkwMRMREZkIEzMREZGJMDETERGZCBMzERGRiQQYHQARUVOqqqpQXl7uUnbq1CnH+/Dw8LYOichrmJiJyPTKysrQs2dP1NTUOMq6d+/ueK+qRoRF5BU8lE1EphcVFYUVK1a4Hefv79/G0RB5FxMzEbULd9xxR70yf39/TJw40YBoiLyHiZmI2oVu3bohIMD17Juq4q677jIoIiLvYGImonYjMDDQZTgjIwNTpkwxKBoi72BiJqJ2IyUlBRaLxTE8depUA6Mh8g4mZiJqN1555RV07doVAGCxWDBp0iSDIyJqfUzMRNRuBAQEOA5dV1VVGRwNkXc0mZhFpLOIfCki20Vkl4j8wVbeV0S+EJFvRGS1iHSylQfahvfbxvfxbhOIqCN57rnnAAAPPfSQwZEQeYcnNxg5D2CsqpaLiAXAVhHZAGAugGdVdZWIvAjgHgB/t/09pao/E5EpAJ4EkOGl+InIQ2lpaUaH0KoKCgp8pk3XXXcd5s6da3QYZBJNJma13lLHfi88i+2lAMYCuNNW/hqARbAm5iTbewBYA+A5ERHlrXmIDJWWlob09HSjw2gVmZmZPtOWtLQ05ObmGh0GmYhH55hFxF9ECgEcB5AD4ACA06pabZukGECs7X0sgMMAYBt/BkCEmzpnikieiOSVlpa2rBVE1KH4SlImcsejxKyqNao6BEBPAMMBXOluMttfaWScc53LVTVBVRMiIyM9jZeIiMinNeuqbFU9DeBjACMAhImI/VB4TwBHbe+LAcQBgG18KICTrREsERGRr/PkquxIEQmzvQ8CcBOAPQA2A0i1TTYdwLu29+/ZhmEb/xHPLxMREXnGk6uyowG8JiL+sCbyTFVdLyK7AawSkcUACgC8bJv+ZQD/KyL7Yd1T5v3yiIiIPOTJVdk7AFzjpvw/sJ5vrlteCcA3fsNARETUxnjnLyIiIhNhYiYiIjIRJmYiIiITYWImonpWrVoFEUHnzp2NDsUjIuJ4+fn5ITw8HLNnz0Z+fr7RoRE1GxMzEdUzZcoUtKdfORYUFAAAVBW1tbU4deoUXnjhBWRmZiIpKaldtYWIiZmIfNaSJUvw3nvvYdWqVUaHQuQxJmYi8lki1jsEv/DCCwZHQuQ5JmYiAgDs378fPXv2RE5ODr7++mtMmDDBZXxJSQn69OmD7OxslJeXY8yYMQgKCnKMT05OhoggOTkZFRUV2LhxI4YP/+lWB/fddx9ycnJw7tw5zJs3DyKCjz/+2FF3VFQUsrOzsWvXLowZM8bliUtjx45FRES9Z+F4bOvWrfWW01gbcnNzUVFRgS5dujTZBrs+ffo02gYij6mq4a+hQ4cqEXnX6tWrGx2flpama9ascQwfOXJEAwMDHcPTp09X61eGVUlJicv4pKQkBaDr1q1zlAHQ0tJSVVXt27evy/IGDBigmzdvdtS9cuVKl7qdvxdGjx6t4eHhDcZeUFDgEpszWB+i0+By3LXBLjU1tck2OC+nsTY0JDU1VVNTU5ucjnwPgDx1kxMNT8rKxEzUJhpLzJWVlW4Tmz1p2ce7e3300Ueq+lNSO3funGN+AFpQUKCqqrW1tTpu3Djt0qWLTpgwQd9++22P625KU4n5hhtuaFYb7ObPn+/1NjAxd1wNJWYeyiYiBAYGomvXrigvL29wfFhYGAICAup9idx4440eLUNEsHHjRpw+fRqqipSUFPzlL39x1F1VVXXRdTektrYWAHD//fe3ynLctQGw9k9AQIBX2kAdDxMzEQGw/uToiiuuQE5ODnbv3o2JEye6jC8qKkLv3r2xYcMGnDlzBsuWLUNwcLDH9YeFhWHHjh2ora3F9ddfDwA4d+6co+7LL78cGzZswMmTJ7Fs2TJkZmY65m3OOeba2locP34c48aNg7+/P3788Uekp6fXW05rtcGud+/ejbaByGPudqPb+sVD2UTe19Q5ZlXV5ORk7datmw4bNkzXr1/vcn5WVbWsrEz79eunFotFx48frzk5Oaqqmpub63L4VlXrDRcWFuqVV16pXbp00REjRuiKFSu0trbWUffcuXO1X79+GhkZqePHj3eJKzExsdFzzM7LEhENDQ3VX//615qfn19vWvtyGmvDwoULPW6Dc9801oaG8FB2x4UGDmWLdZyxEhISNC8vz+gwiHxaZmamY8+RzCMtzfowvqysLIMjobYmIvmqmlC3nIeyiYiITISJmYiIyESYmImIiEyEiZmIiMhEmJiJiIhMhImZiIjIRJiYiYiITISJmYiIyESYmImIiEyEiZmIiMhEmJiJiIhMJKCpCUSkM4BPAATapl+jqo+LyKsARgM4Y5t0hqoWiogAWArgVgA/2sq/8kbwROS5Z599lvdjNqHPP/8cI0aMMDoMMpEmEzOA8wDGqmq5iFgAbBWRDbZx81R1TZ3pbwHQ3/a6FsDfbX+JyEA9e/Y0OgRyY8SIEbjuuuuMDoNMpMnEbHs0lf3p6Rbbq7FHUiUBeN023+ciEiYi0apa0uJoieiicW+ZqH3w6ByziPiLSCGA4wByVPUL26gnRGSHiDwrIoG2slgAh51mL7aVERERURM8SsyqWqOqQwD0BDBcRAYBeATAFQCGAegOYL5tcnFXRd0CEZkpInkikldaWnpRwRMREfkaT84xO6jqaRH5GMBEVX3aVnxeRP4B4CHbcDGAOKfZegI46qau5QCWA4CIlIpIBYATzQvfp10C9kdd7JP62Cf1sU9csT/qM0uf9HZX6MlV2ZEAqmxJOQjATQCetJ83tl2FnQxgp22W9wA8ICKrYL3o60xT55dVNVJE8lQ1oRkN8mnsj/rYJ/WxT+pjn7hif9Rn9j7xZI85GsBrIuIP66HvTFVdLyIf2ZK2ACgE8F+26bNh/anUflh/LvWr1g+biIjIN3lyVfYOANe4KR/bwPQK4P6Wh0ZERNTxmOnOX8uNDsBk2B/1sU/qY5/Uxz5xxf6oz9R9ItYdXCIiIjIDM+0xExERdXiGJ2YRmSgiRSKyX0QWGB1PWxGRV0TkuIjsdCrrLiI5IvKN7W+4rVxE5G+2PtohIj83LnLvEJE4EdksIntEZJeIzLGVd+Q+6SwiX4rIdluf/MFW3ldEvrD1yWoR6WQrD7QN77eN72Nk/N5ku+lRgYistw136D4RkYMi8rWIFIpInq2sw352AMB218k1IrLX9r1yXXvpE0MTs+1K7+dhvb/2VQB+KSJXGRlTG3oVwMQ6ZQsAbFLV/gA22YYB1/uPz4T1/uO+phrA71X1SgAjANxv2xY6cp/Y71MfD2AIgIkiMgLAkwCetfXJKQD32Ka/B8ApVf0ZgGdt0/mqOQD2OA2zT4AbVXWI08+AOvJnB7A+TOkDVb0CQDys20v76BNVNewF4DoAHzoNPwLgESNjauP29wGw02m4CEC07X00gCLb+2UAfuluOl99AXgXwM3sE0f7ugD4CtZ7A5wAEGArd3yGAHwI4Drb+wDbdGJ07F7oi56wfqmOBbAe1p9sdvQ+OQjgkjplHfazA6AbgG/rruv20idGH8rmfbVd9VDbzVhsfy+1lXeofrIdbrwGwBfo4H0ide5TD+AAgNOqWm2bxLndjj6xjT8DIKJtI24TfwXwMIBa23AE2CcK4F8iki8iM21lHfmz0w9AKYB/2E55vCQiwWgnfWJ0YvbovtrUcfpJREIAvAXgt6r6Q2OTuinzuT7ROvepB3Clu8lsf32+T0RkMoDjqprvXOxm0g7TJzYjVfXnsB6SvV9Ebmhk2o7QJwEAfg7g76p6DYAK/HTY2h1T9YnRidmj+2p3IN+LSDQA2P4et5V3iH4S6/O+3wKwUlXfthV36D6xU9XTAD6G9fx7mIjYbw7k3G5Hn9jGhwI42baRet1IALeLyEEAq2A9nP1XdOw+gaoetf09DuAdWP+J68ifnWIAxfrTkxDXwJqo20WfGJ2YtwHob7uishOAKbDea7ujeg/AdNv76bCeZ7WXT7NdOTgCHtx/vL0REQHwMoA9qvoXp1EduU8iRSTM9t5+n/o9ADYDSLVNVrdP7H2VCuAjtZ0w8xWq+oiq9lTVPrB+X3ykqlPRgftERIJFpKv9PYDxsD67oMN+dlT1GIDDInK5rWgcgN1oL31igpP0twLYB+u5s4VGx9OG7X4TQAmAKlj/W7sH1nNfmwB8Y/vb3TatwHr1+gEAXwNIMDp+L/THKFgPHe2A9d7rhbZtoyP3ydUACmx9shPA/7WV9wPwJaz3o88CEGgr72wb3m8b38/oNni5f8YAWN/R+8TW9u221y7792hH/uzY2jkEQJ7t87MWQHh76RPe+YuIiMhEjD6UTURERE6YmImIiEyEiZmIiMhEmJiJiIhMhImZiIjIRJiYiYiITISJmYiIyESYmImIiEzk/weiKzUujhQX6AAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from tensorflow.keras.utils import plot_model\n", "\n", "# Plot model\n", "plot_model(model, to_file='../images/team_strength_model.png')\n", "\n", "# Display the image\n", "data = plt.imread('../images/team_strength_model.png')\n", "plt.imshow(data);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Stacking models\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Add the model predictions to the tournament data\n", "In lesson 1 of this chapter, you used the regular season model to make predictions on the tournament dataset, and got pretty good results! Try to improve your predictions for the tournament by modeling it specifically.\n", "\n", "You'll use the prediction from the regular season model as an input to the tournament model. This is a form of \"model stacking.\"\n", "\n", "To start, take the regular season model from the previous lesson, and predict on the tournament data. Add this prediction to the tournament data as a new column." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# Predict\n", "games_tourney['pred'] = model.predict([games_tourney['team_1'], \n", " games_tourney['team_2'], \n", " games_tourney['home']])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create an input layer with multiple columns\n", "In this exercise, you will look at a different way to create models with multiple inputs. This method only works for purely numeric data, but its a much simpler approach to making multi-variate neural networks.\n", "\n", "Now you have three numeric columns in the tournament dataset: `'seed_diff'`, `'home'`, and `'pred'`. In this exercise, you will create a neural network that uses a single input layer to process all three of these numeric inputs.\n", "\n", "This model should have a single output to predict the tournament game score difference." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# Create an input layer with 3 columns\n", "input_tensor = Input(shape=(3, ))\n", "\n", "# Pass it to a Dense layer with 1 unit\n", "output_tensor = Dense(1)(input_tensor)\n", "\n", "# Create a model\n", "model = Model(input_tensor, output_tensor)\n", "\n", "# Compile the model\n", "model.compile(optimizer='adam', loss='mean_absolute_error')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Fit the model\n", "Now that you've enriched the tournament dataset and built a model to make use of the new data, fit that model to the tournament data.\n", "\n", "Note that this `model` has only one input layer that is capable of handling all 3 inputs, so it's inputs and outputs do not need to be a list.\n", "\n", "Tournament games are split into a training set and a test set. The tournament games before 2010 are in the training set, and the ones after 2010 are in the test set." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "\n", "games_tourney_train = games_tourney[games_tourney['season'] <= 2010]\n", "games_tourney_test = games_tourney[games_tourney['season'] > 2010]" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "103/103 [==============================] - 0s 715us/step - loss: 9.2063\n" ] } ], "source": [ "# Fit the model\n", "model.fit(games_tourney_train[['home', 'seed_diff', 'pred']],\n", " games_tourney_train['score_diff'],\n", " epochs=1,\n", " verbose=True);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate the model\n", "Now that you've fit your model to the tournament training data, evaluate it on the tournament test data. Recall that the tournament test data contains games from after 2010.\n", "\n" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "30/30 [==============================] - 0s 763us/step - loss: 9.1752\n", "9.175172805786133\n" ] } ], "source": [ "# Evaluate the model on the games_tourney_test dataset\n", "print(model.evaluate(games_tourney_test[['home', 'seed_diff', 'pred']],\n", " games_tourney_test['score_diff'],\n", " verbose=True))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }