| Software | Version |
|---|
| Python | 3.6.8 64bit [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] |
| IPython | 7.2.0 |
| OS | Darwin 18.2.0 x86_64 i386 64bit |
| pandas | 0.23.4 |
| numpy | 1.15.4 |
| Tue Mar 19 21:16:05 2019 PDT |
"
],
"text/latex": [
"\\begin{tabular}{|l|l|}\\hline\n",
"{\\bf Software} & {\\bf Version} \\\\ \\hline\\hline\n",
"Python & 3.6.8 64bit [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE\\_401/final)] \\\\ \\hline\n",
"IPython & 7.2.0 \\\\ \\hline\n",
"OS & Darwin 18.2.0 x86\\_64 i386 64bit \\\\ \\hline\n",
"pandas & 0.23.4 \\\\ \\hline\n",
"numpy & 1.15.4 \\\\ \\hline\n",
"\\hline \\multicolumn{2}{|l|}{Tue Mar 19 21:16:05 2019 PDT} \\\\ \\hline\n",
"\\end{tabular}\n"
],
"text/plain": [
"Software versions\n",
"Python 3.6.8 64bit [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]\n",
"IPython 7.2.0\n",
"OS Darwin 18.2.0 x86_64 i386 64bit\n",
"pandas 0.23.4\n",
"numpy 1.15.4\n",
"Tue Mar 19 21:16:05 2019 PDT"
]
},
"execution_count": 79,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# %load jupyter_default.py\n",
"import pandas as pd\n",
"import numpy as np\n",
"import os\n",
"import re\n",
"import datetime\n",
"import time\n",
"import glob\n",
"from tqdm import tqdm_notebook\n",
"from colorama import Fore, Style\n",
"\n",
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.colors\n",
"import seaborn as sns\n",
"\n",
"%config InlineBackend.figure_format='retina'\n",
"sns.set() # Revert to matplotlib defaults\n",
"plt.rcParams['figure.figsize'] = (12, 8)\n",
"plt.rcParams['axes.labelpad'] = 20\n",
"plt.rcParams['legend.fancybox'] = True\n",
"plt.style.use('ggplot')\n",
"\n",
"SMALL_SIZE, MEDIUM_SIZE, BIGGER_SIZE = 14, 16, 20\n",
"plt.rc('font', size=SMALL_SIZE)\n",
"plt.rc('axes', titlesize=SMALL_SIZE)\n",
"plt.rc('axes', labelsize=MEDIUM_SIZE)\n",
"plt.rc('xtick', labelsize=SMALL_SIZE)\n",
"plt.rc('ytick', labelsize=SMALL_SIZE)\n",
"plt.rc('legend', fontsize=MEDIUM_SIZE)\n",
"plt.rc('axes', titlesize=BIGGER_SIZE)\n",
"\n",
"def savefig(plt, name):\n",
" plt.savefig(f'../../figures/{name}.png', bbox_inches='tight', dpi=300)\n",
"\n",
"%load_ext version_information\n",
"%version_information pandas, numpy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Bayesian Modeling Discussion\n",
"\n",
"We can model the probability of an outcome $y$ as $P_t(y)$ using a discrete **Poisson distribution** i.e. if discretizing the time $t$ in seconds.\n",
"\n",
"$$\n",
"P_t(\\mu) = \\frac{\\mu^te^{-\\mu}}{k!}\n",
"$$\n",
"\n",
"Instead we could also assume a Gamma posterior, which has the advantage of being continuous and has more parameters than can be optimized. For now we'll stick with using the simpler Poisson distribution.\n",
"\n",
"Based on a set of goalie pull observations $X$ from 2003-2007 NHL games, we'll solve for the posterior distribution $P_t(y|X)$, the probability of the outcome $y$, given the observations. This is done computationally using markov chain monte carlo and the `pymc3` library.\n",
"\n",
"The outcomes we're interested in are $y = \\big\\{\\mathrm{goal\\;for}, \\mathrm{goal\\;against}, \\mathrm{no\\;goal}\\big\\}$. \n",
"\n",
"We'll use a **uniform prior** over the domain of times (last 5mins). Note: when gathering the observations, we throw out goalie pulls greater than 5 minutes from the end of the game (due to high likelihood of false positives when parsing goalie pulls from the raw game table).\n",
"\n",
"Once we find the posteriors discussed above, we can study the risk reward of pulling a goalie. We'll compare posteriors to find the odds of scoring a goal (and the odds of getting scored on) over time $t$ where:\n",
" - **t = Time elapsed** e.g. if there's 3 minutes left, what is the chance that pulling the goalie will result in a goal for?\n",
" - **t = Time since goalie pull** e.g. after the goalie has been pulled for 1 minute, what is the chance of getting a goal?"
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {},
"outputs": [],
"source": [
"import pymc3 as pm"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load the training data"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"20032004_goalie_pulls_2019-03-01.pkl 20062007_goalie_pulls_2019-03-01.pkl\n",
"20052006_goalie_pulls_2019-03-01.pkl\n"
]
}
],
"source": [
"ls ../../data/processed/pkl/"
]
},
{
"cell_type": "code",
"execution_count": 82,
"metadata": {},
"outputs": [],
"source": [
"def load_data():\n",
" files = glob.glob('../../data/processed/pkl/*.pkl')\n",
" files = sorted(files)\n",
" print(files)\n",
" return pd.concat((pd.read_pickle(f) for f in files))\n",
"\n",
"def clean_df(df):\n",
" _df = df.copy()\n",
" \n",
" len_0 = _df.shape[0]\n",
" print('Removing goal_for_time < 15 mins')\n",
" _df = _df[~(_df.goal_for_time < datetime.timedelta(seconds=15*60))]\n",
" print(f'Removed {len_0 - _df.shape[0]} total rows')\n",
" \n",
" if 'game_end_time' in df.columns:\n",
" len_0 = _df.shape[0]\n",
" print('Removing game_end_time < 15 mins')\n",
" _df = _df[~(_df.game_end_time < datetime.timedelta(seconds=60*15))]\n",
" print(f'Removed {len_0 - _df.shape[0]} total rows')\n",
"\n",
" return _df"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['../../data/processed/pkl/20032004_goalie_pulls_2019-03-01.pkl', '../../data/processed/pkl/20052006_goalie_pulls_2019-03-01.pkl', '../../data/processed/pkl/20062007_goalie_pulls_2019-03-01.pkl']\n",
"Removing goal_for_time < 15 mins\n",
"Removed 1 total rows\n"
]
}
],
"source": [
"df = load_data()\n",
"df = clean_df(df)"
]
},
{
"cell_type": "code",
"execution_count": 84,
"metadata": {},
"outputs": [],
"source": [
"def load_training_samples(\n",
" df,\n",
" cols,\n",
" masks=[],\n",
" dtype='timedelta64[s]'\n",
") -> np.ndarray:\n",
" '''\n",
" Return buckets of training data.\n",
" '''\n",
" if not masks:\n",
" masks = [None] * len(cols)\n",
" out = []\n",
" for col, m in zip(cols, masks):\n",
" if m is None:\n",
" d = df[col].dropna().astype(dtype).values\n",
" else:\n",
" d = df[col][m].dropna().astype(dtype).values\n",
" out.append(d)\n",
" print(f'Loaded {len(d)} samples for col {col}')\n",
"\n",
" out = np.array(out)\n",
" print(f'Training data shape = {out.shape}')\n",
" return out"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Rough work"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Data loading"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def load_training_samples(\n",
" df,\n",
" cols,\n",
" masks=[],\n",
" dtype='timedelta64[s]'\n",
") -> np.ndarray:\n",
" '''\n",
" Return buckets of training data.\n",
" '''\n",
" if not masks:\n",
" masks = [None] * len(cols)\n",
" out = []\n",
" for col, m in zip(cols, masks):\n",
" if m is None:\n",
" d = df[col].dropna().astype(dtype).values\n",
" else:\n",
" d = df[col][m].dropna().astype(dtype).values\n",
" out.append(d)\n",
" print(f'Loaded {len(d)} samples for col {col}')\n",
"\n",
" out = np.array(out)\n",
" print(f'Training data shape = {out.shape}')\n",
" return out"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's start by modeling the 5 on 6 goal times in 3rd period, where time is a continuous (or rather, discretized by second) and measured in minutes."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Loaded 270 samples for col goal_for_time\n",
"Loaded 554 samples for col goal_against_time\n",
"Training data shape = (2,)\n"
]
}
],
"source": [
"features = ['goal_for_time', 'goal_against_time']\n",
"training_samples = load_training_samples(df, features)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(270,)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"training_samples[0].shape"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([1180., 1124., 1157., 1168., 1118., 1183., 1130., 1116., 1172.,\n",
" 1181.])"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"training_samples[0][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To get the proper probabilities, we should weight the "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Modeling"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [],
"source": [
"# with pm.Model() as model:\n",
"# prior_goal_for = pm.Uniform('prior_goal_for', 15, 20)\n",
"# prior_goal_against = pm.Uniform('prior_goal_against', 15, 20)\n",
"# obs_goal_for = pm.Gamma('obs_goal_for', observed=training_samples[0])\n",
"\n",
"# need to set up priors for all the parameters of the gamma!...\n",
"# THINK ABOUT IT"
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {},
"outputs": [],
"source": [
"from scipy.stats import poisson"
]
},
{
"cell_type": "code",
"execution_count": 61,
"metadata": {},
"outputs": [],
"source": [
"poisson?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```\n",
"pmf(k, mu, loc=0) \n",
" Probability mass function.\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"[