{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MTLR (N-MTLR) Example \n",
    "\n",
    "In this notebook, we will present a simple example of the neural network version of the `MTLR` method described in \n",
    "[this](https://papers.nips.cc/paper/4210-learning-patient-specific-cancer-survival-distributions-as-a-sequence-of-dependent-regressors) and [this paper](https://arxiv.org/pdf/1801.05512.pdf).\n",
    "\n",
    "For a more verbose introduction to `pycox` see [this notebook](https://nbviewer.jupyter.org/github/havakv/pycox/blob/master/examples/01_introduction.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# For preprocessing\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn_pandas import DataFrameMapper \n",
    "\n",
    "import torch # For building the networks \n",
    "import torchtuples as tt # Some useful functions\n",
    "\n",
    "from pycox.datasets import metabric\n",
    "from pycox.models import MTLR\n",
    "from pycox.evaluation import EvalSurv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Uncomment to install `sklearn-pandas`\n",
    "# ! pip install sklearn-pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(1234)\n",
    "_ = torch.manual_seed(123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "\n",
    "We load the METABRIC data set as a pandas DataFrame and split the data in in train, test and validation.\n",
    "\n",
    "The `duration` column gives the observed times and the `event` column contains indicators of whether the observation is an event (1) or a censored observation (0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = metabric.read_df()\n",
    "df_test = df_train.sample(frac=0.2)\n",
    "df_train = df_train.drop(df_test.index)\n",
    "df_val = df_train.sample(frac=0.2)\n",
    "df_train = df_train.drop(df_val.index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>x0</th>\n",
       "      <th>x1</th>\n",
       "      <th>x2</th>\n",
       "      <th>x3</th>\n",
       "      <th>x4</th>\n",
       "      <th>x5</th>\n",
       "      <th>x6</th>\n",
       "      <th>x7</th>\n",
       "      <th>x8</th>\n",
       "      <th>duration</th>\n",
       "      <th>event</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>5.603834</td>\n",
       "      <td>7.811392</td>\n",
       "      <td>10.797988</td>\n",
       "      <td>5.967607</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>56.840000</td>\n",
       "      <td>99.333336</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>5.284882</td>\n",
       "      <td>9.581043</td>\n",
       "      <td>10.204620</td>\n",
       "      <td>5.664970</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>85.940002</td>\n",
       "      <td>95.733330</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>6.654017</td>\n",
       "      <td>5.341846</td>\n",
       "      <td>8.646379</td>\n",
       "      <td>5.655888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>66.910004</td>\n",
       "      <td>239.300003</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>5.456747</td>\n",
       "      <td>5.339741</td>\n",
       "      <td>10.555724</td>\n",
       "      <td>6.008429</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>67.849998</td>\n",
       "      <td>56.933334</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>5.425826</td>\n",
       "      <td>6.331182</td>\n",
       "      <td>10.455145</td>\n",
       "      <td>5.749053</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>70.519997</td>\n",
       "      <td>123.533333</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         x0        x1         x2        x3   x4   x5   x6   x7         x8  \\\n",
       "0  5.603834  7.811392  10.797988  5.967607  1.0  1.0  0.0  1.0  56.840000   \n",
       "1  5.284882  9.581043  10.204620  5.664970  1.0  0.0  0.0  1.0  85.940002   \n",
       "3  6.654017  5.341846   8.646379  5.655888  0.0  0.0  0.0  0.0  66.910004   \n",
       "4  5.456747  5.339741  10.555724  6.008429  1.0  0.0  0.0  1.0  67.849998   \n",
       "5  5.425826  6.331182  10.455145  5.749053  1.0  1.0  0.0  1.0  70.519997   \n",
       "\n",
       "     duration  event  \n",
       "0   99.333336      0  \n",
       "1   95.733330      1  \n",
       "3  239.300003      0  \n",
       "4   56.933334      1  \n",
       "5  123.533333      0  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature transforms\n",
    "\n",
    "The METABRIC dataset has  9 covariates: `x0, ..., x8`.\n",
    "We will standardize the 5 numerical covariates, and leave the binary covariates as is.\n",
    "Note that PyTorch require variables of type `'float32'`.\n",
    "\n",
    "We like using the `sklearn_pandas.DataFrameMapper` to make feature mappers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "cols_standardize = ['x0', 'x1', 'x2', 'x3', 'x8']\n",
    "cols_leave = ['x4', 'x5', 'x6', 'x7']\n",
    "\n",
    "standardize = [([col], StandardScaler()) for col in cols_standardize]\n",
    "leave = [(col, None) for col in cols_leave]\n",
    "\n",
    "x_mapper = DataFrameMapper(standardize + leave)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_mapper.fit_transform(df_train).astype('float32')\n",
    "x_val = x_mapper.transform(df_val).astype('float32')\n",
    "x_test = x_mapper.transform(df_test).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Label transforms\n",
    "\n",
    "The survival methods require individual label transforms, so we have included a proposed `label_transform` for each method.\n",
    "In this case `label_transform` is just a shorthand for the class `pycox.preprocessing.label_transforms.LabTransDiscreteTime`.\n",
    "\n",
    "The `MTLR` is a discrete-time method, meaning it requires discretization of the event times to be applied to continuous-time data.\n",
    "We let `num_durations` define the size of this (equidistant) discretization grid, meaning our network will have `num_durations` output nodes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_durations = 10\n",
    "labtrans = MTLR.label_transform(num_durations)\n",
    "get_target = lambda df: (df['duration'].values, df['event'].values)\n",
    "y_train = labtrans.fit_transform(*get_target(df_train))\n",
    "y_val = labtrans.transform(*get_target(df_val))\n",
    "\n",
    "train = (x_train, y_train)\n",
    "val = (x_val, y_val)\n",
    "\n",
    "# We don't need to transform the test labels\n",
    "durations_test, events_test = get_target(df_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pycox.preprocessing.label_transforms.LabTransDiscreteTime"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(labtrans)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural net\n",
    "\n",
    "We make a neural net with `torch`.\n",
    "For simple network structures, we can use the `MLPVanilla` provided by `torchtuples`.\n",
    "For building more advanced network architectures, see for example [the tutorials by PyTroch](https://pytorch.org/tutorials/).\n",
    "\n",
    "The following net is an MLP with two hidden layers (with 32 nodes each), ReLU activations, and `num_nodes` output nodes.\n",
    "We also have batch normalization and dropout between the layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "in_features = x_train.shape[1]\n",
    "num_nodes = [32, 32]\n",
    "out_features = labtrans.out_features\n",
    "batch_norm = True\n",
    "dropout = 0.1\n",
    "\n",
    "net = tt.practical.MLPVanilla(in_features, num_nodes, out_features, batch_norm, dropout)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you instead want to build this network with `torch` you can uncomment the following code.\n",
    "It is essentially equivalent to the `MLPVanilla`, but without the `torch.nn.init.kaiming_normal_` weight initialization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# net = torch.nn.Sequential(\n",
    "#     torch.nn.Linear(in_features, 32),\n",
    "#     torch.nn.ReLU(),\n",
    "#     torch.nn.BatchNorm1d(32),\n",
    "#     torch.nn.Dropout(0.1),\n",
    "    \n",
    "#     torch.nn.Linear(32, 32),\n",
    "#     torch.nn.ReLU(),\n",
    "#     torch.nn.BatchNorm1d(32),\n",
    "#     torch.nn.Dropout(0.1),\n",
    "    \n",
    "#     torch.nn.Linear(32, out_features)\n",
    "# )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the model\n",
    "\n",
    "To train the model we need to define an optimizer. You can choose any `torch.optim` optimizer, but here we instead use one from `tt.optim` as it has some added functionality.\n",
    "We use the `Adam` optimizer, but instead of choosing a learning rate, we will use the scheme proposed by [Smith 2017](https://arxiv.org/pdf/1506.01186.pdf) to find a suitable learning rate with `model.lr_finder`. See [this post](https://towardsdatascience.com/finding-good-learning-rate-and-the-one-cycle-policy-7159fe1db5d6) for an explanation.\n",
    "\n",
    "We also set `duration_index` which connects the output nodes of the network the the discretization times. This is only useful for prediction and does not affect the training procedure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = MTLR(net, tt.optim.Adam, duration_index=labtrans.cuts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEKCAYAAAD5MJl4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXyU5bn/8c+VPYQsECBAAiSAbCKLBEHArSp1wb2lWjdwQU9tpZ7Wqj21Pba21V+r1q1aNxSreCpuoLbaKoICyib7IjuEnRDCkn3m/v0xCQZCyMxkJpPl+3698mLmmed57mvCJFfu3ZxziIiIVBcV6QBERKTxUXIQEZEalBxERKQGJQcREalByUFERGpQchARkRpiIh1AMNq1a+eys7MjHYaISJOycOHCvc659v6c2ySTQ3Z2NgsWLIh0GCIiTYqZbfb3XDUriYhIDUoOIiJSg5KDiIjU0CT7HESk+SkvLycvL4+SkpJIh9LkJSQkkJWVRWxsbND3UHIQkUYhLy+P5ORksrOzMbNIh9NkOefIz88nLy+PnJycoO+jZiURaRRKSkpIT09XYqgnMyM9Pb3eNTAlBxFpNJQYQuN438fZ6/YGdA8lBxGRFuD+d5cHdL6Sg4gIsH//fv76178GfN1FF13E/v37A75u3LhxTJ06NeDrglVa4Q3ofCUHERFqTw4ej+eE13344YekpaWFK6yQKfcElhw0WklEGp0Hpq9g5fYDIb1nv84p/OaSk2t9/d5772X9+vUMGjSI2NhYWrduTadOnVi8eDErV67k8ssvZ+vWrZSUlDBx4kQmTJgAfLucz6FDh7jwwgsZNWoUc+bMITMzk/fee4/ExMQ6Y/vkk0/4+c9/TkVFBUOHDuWZZ54hPj6ee++9l2nTphETE8Po0aP585//zJtvvskDDzxAdHQ0qampzJo1y6/3X6bkICISuIceeojly5ezePFiPvvsMy6++GKWL19+ZDjoSy+9RNu2bSkuLmbo0KFcddVVpKenH3WPtWvXMmXKFJ5//nnGjh3LW2+9xXXXXXfCcktKShg3bhyffPIJvXr14oYbbuCZZ57hhhtu4J133mH16tWY2ZGmq9/+9rd89NFHZGZmBtScVR5gs5KSg4g0Oif6C7+hnHbaaUfNE3jiiSd45513ANi6dStr166tkRxycnIYNGgQAEOGDGHTpk11lrNmzRpycnLo1asXADfeeCNPP/00P/7xj0lISOCWW27h4osvZsyYMQCMHDmScePGMXbsWK688kq/30+gNQf1OYiIHEdSUtKRx5999hn/+c9/mDt3LkuWLGHw4MHHnUcQHx9/5HF0dDQVFRV1luOcO+7xmJgY5s2bx1VXXcW7777LBRdcAMCzzz7Lgw8+yNatWxk0aBD5+fl1luH1Oso9xy+nNqo5iIgAycnJHDx48LivFRYW0qZNG1q1asXq1av58ssvQ1Zunz592LRpE+vWraNnz568+uqrnHXWWRw6dIiioiIuuugihg8fTs+ePQFYv349w4YNY9iwYUyfPp2tW7fWqMEcq9wbWK0BlBxERABIT09n5MiR9O/fn8TERDIyMo68dsEFF/Dss88yYMAAevfuzfDhw0NWbkJCApMmTeL73//+kQ7p22+/nX379nHZZZdRUlKCc47HHnsMgLvvvpu1a9finOPcc89l4MCBdZZRFmB/A4DVVqVpzHJzc502+xFpXlatWkXfvn0jHUazUf37ue9wGaf+7t9sfnjMQudcrj/Xq89BRKSZC6bmoGYlEZEwuuOOO5g9e/ZRxyZOnMj48eMbLAYlBxFp0pxzzW7xvaeffrrByzy2uyDQYaygZiURaSQSEhLIz8+vdWin+KdqP4eEhIQjx1RzEJEmKysri7y8PPbs2RPpUJq8qp3gqgS6rhIoOYhIIxEbG1uvncukdmpWEhGRGgJdVwmUHEREmr1S1RxERORYwXRIKzmIiDRzwXRIhzU5mFkXM5thZqvMbIWZTTzOOX3MbK6ZlZrZz8MZj4hIS9QYh7JWAD9zzi0ys2RgoZn92zm3sto5+4A7gcvDHIuISIvU6GoOzrkdzrlFlY8PAquAzGPO2e2cmw+UhzMWEZGWqlH3OZhZNjAY+CrI6yeY2QIzW6BJMiIi/isLcKMfaKDkYGatgbeAnzrngto13Dn3nHMu1zmX2759+9AGKCLSjDXKmoOZxeJLDK85594Od3kiInK0RpcczLe84ovAKufco+EsS0REjq/c4yUqwMVuwz1aaSRwPbDMzBZXHvsl0BXAOfesmXUEFgApgNfMfgr0C7b5SUREjlbm8RIXE1hdIKzJwTn3BXDCfOWc2wlknegcEREJXlmFl7jowJKDZkiLiDRzwdQclBxERJq5ctUcRETkWGUeL7GqOYiISHXqcxARkRrKPV5ilRxERKS60gp1SIuIyDHKNVpJRESOpT4HERGpodzjVHMQEZGjlVV4iY0ObHElJQcRkWbON0M6OqBrlBxERJo59TmIiEgNvpqDmpVERKSaco9qDiIicgxfh7SSg4iIVKNJcCIichSv11HucQHXHPzaCc7MooCBQGegGFjhnNsVcJQiItKgyjxegNBuE2pmPYB7gPOAtcAeIAHoZWZFwN+AV5xz3sBDFhGRcCuvTA7xId5D+kHgGeA255yr/oKZdQB+CFwPvBJQqSIi0iDKKnzJIaTNSs65a07w2m7gLwGVJiIiDarc4/u7PtTNSlee6HXn3NsBlSYiIg0qLDUH4JLKfzsAI4BPK5+fA3wGKDmIiDRiYemQds6NBzCz94F+zrkdlc87AU8HE6iIiDScqppDXJhWZc2uSgyVdgG9AipJREQaXFhqDtV8ZmYfAVMAB1wNzAioJBERaXBVQ1njogNbstuv5OCc+7GZXQGcWXnoOefcOwGVJCIiDe7bDunAmpX8rTkALAIOOuf+Y2atzCzZOXcwoNJERKRBBdus5NfZZnYrMBXfjGiATODdgEoSEZEGF+xQVn/PvgMYCRwAcM6txTe8VUREGrFgl8/w9+xS51xZ1RMzi8HXMS0iIo1YuGsOM83sl0CimZ0PvAlMD6gkERFpcEfmOYSp5nAvvhVZlwG3AR8CvwqoJBERaXDl4ZznULkk9/OVXyIi0kSUhmltJQDMbCTwv0C3ymsMcM657gGVJiIiDapqVdZQ7+dQ5UXgLmAh4AmoBBERiZhwrcpapdA5988AYxIRkQgr93iJjjKio0I4Q9rMTq18OMPM/oRvie7Sqtedc4sCDVRERBpOmccb8NIZUHfN4ZFjnudWe+yA7wRcooiINJiyCi9xATYpQd37OZwDYGbdnXMbqr9mZnV2RptZF2Ay0BHw4luw7/FjzjHgceAioAgYpxqJiEholHm8xMUEtiIr+D/PYepxjr3px3UVwM+cc32B4cAdZtbvmHMuBE6q/JoAPONnTCIiUgdfzSHEzUpm1gc4GUg9Zj/pFCChrptXbhC0o/LxQTNbhW/RvpXVTrsMmOycc8CXZpZmZp2O2VxIRESCUO7xBjwBDuruc+gNjAHS+HY/aYCDwK2BFGRm2cBg4KtjXsoEtlZ7nld57KjkYGYT8NUs6Nq1ayBFi4i0WGUV3oCHsULdfQ7vAe+Z2enOubnBBmdmrYG3gJ865w4c+/Lxij5OLM8BzwHk5uZq0T8RET8EW3Pw94qtZvaOme02s11m9paZZflzoZnF4ksMrznn3j7OKXlAl2rPs4DtfsYlIiInUBpkzcHfKyYB04DO+Jp8plceO6HKkUgvAqucc4/Wcto04AbzGY5vwp36G0REQqCsIjx9DlU6OOeqJ4OXzeynflw3ErgeWGZmiyuP/RLoCuCcexbfCq8XAevwDWUd72dMIiJSh3KPl6T4QHaE9vH3ij1mdh0wpfL5NUB+XRc5577g+H0K1c9x+HaaExGRECvzeEkLY7PSTcBYYGfl1/cqj4mISCNWXuFCP0O6inNuC3BpwHcXEZGIKvN4iQ3XaCUzywp2tJKIiEROsGsrhXW0koiIRFZZmOc5tHfOTXLOVVR+vQy0D7g0ERFpUMGureRvcthrZteZWXTl13X4MVpJREQiK9wzpKuPVtqBRiuJiDQJYVlbqYpGK4mIND1er6PC68I3Q9rMcoCfANnVr3HOKWGIiDRSZR4vQPhqDsC7+NZImo5vRzcREWnkyiuTQ3wY11Yqcc49EfDdm5DSCg/vfb2dHh2SGNKtbaTDERGpt7IKX3II58J7j5vZb4CPgdKqg81hr+eyCi//WLCVp2esY0dhCZlpicy8+2xigqiGiYg0Jg3RrHQKvtVVv8O3zUqu8nmT9dWGfP77H0vYtr+YId3acOWpmTw9Yz3/XrmLC0/pFOnwRETqpbzCty9a2NZWAq4AujvnygIuoRF75ONv8DrHKzedxpkntcPrYNqS7bz4xUYlBxFp8so8HoDwra0ELMG3j3SzcbCknEVbCrh8cCZn9WqPmREdZYwbkcOCzQUs2bo/0iGKiNRLWT1qDv5ekQGsNrOPzGxa1VfApTUic9bnU+F1nHnS0auAjM3NonV8DC/N3hihyEREQqOqzyEuJvDlM/xtVvpNwHdu5GZ9s4ekuGiGdGtz1PHkhFjG5nZh8txN3HdhXzqmJkQmQBGReqoayhoXHR3wtf7WHBYAnzvnZuJbPiMVmBNwaY2Ec45Za/dweo92xx3iNX5ktq8vYu6mBo9NRCRU6jOU1d8rZgEJZpYJfIJvn+eXAy6tkdiUX8TWfcWc1avdcV/v0rYVo/t15PWvtlBUVtHA0YmIhEZVcogN46qs5pwrAq4EnnTOXQGcHHBpjcTMNbsBOLNX7auO3zQqh8LicibN3tRAUYmIhNa3fQ7hqzmYmZ0OXAt8UHks8EasRmLW2r10S29Ft/SkWs8Zmt2GC07uyCMfr2FGZTIREWlKjjQrhXG00kTgPuAd59wKM+sOzAi4tEagtMLD3PX5NUYpHcvMeGTsQPp0TOEnr3/Nmp0HGyhCEZHQKA93zcE5N8s5d6lz7uHK5xucc3cGXFojsHBTAcXlnhM2KVVJio/hxXG5tIqL5uZX5rP3UGmd14iINBbf9jmEODmY2XNmdkotryWZ2U1mdm3ApUbQzLV7iIkyTu+R7tf5nVITef6GXPYcLOW2VxdyoKQ8zBGKiIRGOGsOfwXuN7NVZvammf3VzF4ys8/xDWVNBqYGXGoEzfpmL7nZbWgd7+8UDxjYJY1Hxw5i0ZYCznh4Bk/PWMfh0hOPYpq+ZDtvLcyjwqMVzkUkMkrDtSqrc24xMNbMWgO5QCegGFjlnFsTcGkRtvtACat2HOAXF/QO+NqLB3SiW/ooHvv3N/zpozW89MVG7jinJzeOyCY66uhhYq/O3cT9760A4JmZ6/nFd3tzfr8MzAIfTib145zj66378XgdfTomk5wQG+mQRBrMkdFKYdwm9BDwWcB3b0Scc7zwhW9JjLo6o2vTPzOVF8cNZdGWAh75eA2/fX8l/1qxk7/8YBCd0xIBeGPeFu5/bwXn9c3gylMz+fPHa5jw6kKGdGvDeX0zaJ8cT4fkeDqnJdKzQ+uQvT85Wkm5h2mLtzNpziZW7Thw5Hi39Fb065TCyJ7tOL9fBhkp386A33e4jJnf7GbFtgN4nMP5lqUhJSGGPp1S6NsphW5tWxEVpSQvTUPVqqzB9DmYq/oJaEJOOnmgW7tiid/nF5d5uHvqEt5fuoMrB2fyyNiBIfkr/u1Fedz/7nJioqN4+KpTOFTqK+esXu352/VDiI+JpsLj5c2FeTz16Tq27S8+6vpbz8jhlxf1DSiWfYfL+NW7y8hMS+TmUd2bzfIezvn2uq3rQ+z1OqYv3c5bi7bRplUsWW0SyWrTipSEWHYfLGHngRJ2FpYw65s9FBSV06djMuNGZJORksCK7YWs3HGAJVsLj/xfDOySRm63Nny9pYCvt+7HOUiIjSI2KgoMDDhUWoG38sekVVw0g7umcVav9pzduwMndWitGqE0Wn/6aDXPztzA+j9cBICZLXTO5fpzbZNMDkmZvdyhvDV+/VBu31/MhFcXsGL7AX7x3T7cflb3kP4wb9p7mIlvfM2SvELMYGSPdrxwYy4JsTWngRSVVbDnYCm7D5by9qJtTJm3hZtH5fCri/1LEBv3Hmb8pHls31+CxzmiDK4YnMmEM3s06VrInPV7+fV7K9i49zBZbRLJTk8ip10SfTslMyArjZM6tCY6ypi1di8P/3M1K3ccoGvbVnidY0dhCR7vt5/huOgoOqTE079zKjeM6Mbp3dNrfG+dc6zdfYh/r9zFxyt3sSxvP6dkpnJ27w58p08HTslMPap2UFLuYe2uQ6zacYCVOw4wZ/1evtl1CIDOqQncNCqHG0dkB/XXmUg4/eHDVbw6dzOrfncBEObkYGZRQGvn3IE6Tw6T+E4nuWWLF9ErI/mE5/1n5S7ufXsZxWUVPH71YM7rlxGWeMoqvDzxyVq27Cvi4asGkBhX9/xA5xwPTF/Jy3M2MX5kNr8e0++ECWL+pn1MmLwAM+OFG3Np3zqe5z/fwP/N30qZx8u5fTpw06ic4/4yrE1hUTnvLdlGRkoC5/bpUOvudzsLS5ixZjefrt5NYVE5N4zoxoX9O9Xoa6ni8TqKyiooKffSplVsrffdfbCEP3ywincXb6dL20TGDOjM1n1FbNx7mI17D1NU5luLPjE2mk5pCWzY40sePx/dm0sHdiYqyqjweNl1sJQDxeV0SI6nbVJcwMm/wuMNeOe/7fuLmfXNHqYv3c7sdfn0ymjNA5f293sUnEhD+N9pK3h7UR5L//e7QBiSg5m9DtwOeICF+Bbee9Q596ego66H+E4nub9N/ZhxI3OO+/ruAyU8MH0lHyzbQe+MZJ784eA6E0kkOOd48INVvPjFRsbmZnFu3wzaJsXRNimOmChjZ6GvmWTDnsM8M3M9WWmJTBo/9KiZ3XsPlTJ5zib+/tUW9h0uo2+nFG4ZlcPlgzNr/eW9o7CYl77YyOtfbeFw5S/gjJR4rh7ale8NyeJgSQXLtxWybFshi7YUsGK77++AzLRE4mKi2Lj3MN3bJ/Gjs3tySmYqCzbvY8GmAhZuLmD3wRJKyr8doRUdZXRKTaBLm1ZkpMRT4XWUlHsprfCweMt+Siu83H52D350do+jalter2NT/mGW5hWyeOt+1u0+xDl9OnDd8K7ExzSeyfnOOf6zajcPTF9BXkExFw/oxOWDMhma3Ya0VnFHnVtW4aWorILY6KjKL1OTlITVL99ZxscrdrHgV+cB4UkOi51zgyrnNAwB7gEWOucG1CPuoCVn9XbX/OE1nruh5nucujCPB6avoLTCy8RzT+LWM7oHNYyroTjneOifq/nbrA0nPG9Ej3T+eu2pNX7hVCkp9/De4m28+MVGvtl1iAFZqTx4eX8GZH27R9PaXQf526wNvLd4G14HYwZ04tYzurOjsITXvtrMzG/2UP3jkBQXzSlZqZzVy9fc0iujNV4H/1q+k6dmrDuqo7dd63iGZrehS9tWtIqLplVcNPEx0ew5WMrWgiLyCorZdaCEuOgo4mOjiY+JIqtNIj8b3ZucdrUvY9JUlJR7+Otn63lu1vojybFXRmtO6pDMrgMlvvd/sIRjf9x6ZyRz6aDOXDqwM13atopA5NKc/fzNJcxdn8/se307OocjOawABgGvA08552aa2RLn3MB6xB20jj1Odh2uf5Svfz36qL+ON+cf5qw/fcbQ7DY8fNUAurdvOu3wuw+UsPtgKfsOl1FQVEa5x9ExJYGOqfFkpCT4PQTTOce0Jdt58INV7D1UyvXDu3F+vwxenr2JT1bvJiE2iquHduXmUTk1fhlt3VfEv5bv9LXZZ6aSk55U68gc37Lne9l7sJTc7DZ0bdtKfwXjSxJLtu5n/qZ9zNtUwOb8w3RMSSCrTSuy2iSSmhhLucdLucdLSbmXLzfks2BzAQCDu6Zxw+nduGRA54CbuUSO584pX7M0bz+f3X0OEFhy8Hcm2N+ATfi2C51lZt2AiPU5tI6P4UBJBSu3H+CUrNQjx6ct3g7A41cPPjK0tKnokJJAh5T6jzwyMy4blMk5fTrwyEdrePXLzUyeu5m2SXHcdV4vrj+9G22Tjl/76NK2Fbee2d3vcs7yYwmSliYhNpph3dMZ1t3/voe8giKmL9nBW4vyuOv/lvCX/6zljrN7cvngzEZd65XGr9zjDfozFPRoJTOLcc5FZLODQYOHuP3f/S33XdiH287qAfj+kj3v0Zmkt47nH7edHomwGqUV2wtZt/sQo/t19KujXCLH63X8e9Uunvp0Hcu2FZKZlsj4kdn8YGgXTd6ToNz88nx2HijhgzvPAAKrOfiVUsxsopmlmM+LZrYI+E7wIddPTLTRs0Nr5qzPP3Js5Y4DrN9zmMsGdY5UWI3SyZ1TuWxQphJDExAVZXz35I5M+/FIJo0fSmZaIg9+sIrT//gpD0xfwbrdh/B6m97Qc4mcsnrUHPxtVrrJOfe4mX0XaI9vJ7hJwMdBlRoCI3qkM3VhHuUeL7HRUUxbvJ2YKOOi/p0iFZJISJgZ5/TuwDm9O7Asr5CXZm/k719uZtLsTcRFR5HVNpFubVuRm92Wm0bmKPFLrcoqvEHPv/F7s5/Kfy8CJjnnllQ7FhEjeqRTVOZhad5+vF5fJ+xZvdrTppb2dJGm6JSsVB77wSC+uOc7/OGKUxg/MpteHZLZUVjCnz5aw+i/zGTGam1GJcdX7vESH+aaw0Iz+xjIAe4zs2SgzuVGzewlYAyw2znX/zivtwFeAnoAJfhqKMv9CWhYTjpmMGddPuUe30zZ+y7q6+fbEWlaMlIS+OGwrkcdm7s+n/vfW874l+dzwckd+cm5PemdkayRTnJEmcdLmyA/D/4mh5vxDWXd4JwrMrN0fE1LdXkZeAqYXMvrvwQWO+euMLM+wNPAuf4E1CYpjr4dU5izPp/thSW0iovmvL4d/LlUpFk4vUc6H955Bs9/voEnP13Lv1bsJCE2ir6dUhiQmcq5fTMY1bOdFgpswerTrOTvqqxeM8sCflg5ln2mc266H9fNMrPsE5zSD/hj5bmrzSzbzDKcc7v8iWtEj3Qmf7mZlTsOMLpfBq3i/N+jQaQ5iIuJ4o5zevK9IVl8uSGfpXm+me1vLszjlbmbyUxL5KohWXx/SJYm2bVA5R4X3g5pM3sIGAq8VnnoTjMb4Zy7L6hSv7UEuBL4wsxOA7oBWYB/yaFnOi98sZGyCi+XDcqsZygiTVdGSgKXDco88nNQWuHh3yt38X/zt/Lkp2t58tO13H5WD/77/F5aILAFCXvNAV9H9CDnnBfAzF4BvgbqmxweAh43s8XAssp7HnfuhJlNACYAdO3qa3sdmt2W6CgjNTGWUSe1q2coIs1HfEw0YwZ0ZsyAzmzbX8yTn6zlmc/W8+WGfJ64erBqES1EfYayBnJVWrXHqbWeFQDn3AHn3Hjn3CDgBnzDZDfWcu5zzrlc51xu+/a+mbnJCbGMzc1iwpnd9deQSC0y0xJ56KoBPPXDwazbdYiLnvicD5buiHRY0gDKKsI/WumPwNdmNgPfENYzqX+tATNLA4qcc2XALcCsQJcC/+OVEVn7T6TJGTOgMwOz0vjJlK+54/VFzN+UzS8v6qslOpoxX7NScAMS/O2QnmJmn+HrdzDgHufczrquM7MpwNlAOzPLA34DxFbe81mgLzDZzDzASnyjokQkTLq0bcWbt5/OHz9czUuzN7J8WyFPX3vqUdulSvNRn7WVTpgczOzUYw7lVf7b2cw6O+cWneh659w1dbw+FzipzihFJGRio6P49SX9GNQ1jXumLuXiJ77gyWsGa6OiZsbr9W/r3drUVXN45ASvOSK4vpKI1M+lAzvTp2Myt726kGue/5LR/TL479G96NMxJdKhSQiUeXzzlMNSc3DOnePPTczsfOfcv4OKQEQipldGMu//ZBQvfrGR52dt4MLHP+eSAZ35+ejedE3XiKam7EhyCPPaSnV5OET3EZEGlhQfw53nnsTn95zD7Wf14N8rd3Hxk5/z+do9kQ5N6qG8on41h1AlB83PF2ni0lrFcc8Fffj4rjPJTEtk3KT5TJ67KdJhSZAaS81Bi8yLNBNd2rZi6n+N4JzeHfj1eyv41bvLKPfUuc6mNDJllTWHcC/ZLSItSOv4GP52/RBuP6sHf/9yC7e9upCSck+kw5IAlNezQzpUyWFTiO4jIo1EdJRx74V9+P0V/fl09W5unbyA4jIliKaitJ41B7+XMTWzEUB29Wucc5Mr/70yqNJFpNG7dlg3YqOjuOetpdz08nxeuDGXpHitgNzYlXt8rf3BLp/h7x7SrwJ/BkbhmyU9FPBrk2oRafrG5nbhsbGD+GpjPje+NI/C4vJIhyR1qG+fg7/pPxfo55xTx7NIC3X54Exio6OY+MbXXPHX2bx441By2iVFOiypRUP1OSwHOgZVgog0GxcP6MTfbxlGweEyLnvqC82FaMTKwjnPwcymm9k0oB2w0sw+MrNpVV9BlSgiTdrw7ulM+/EoOqX65kJMmr0RNSo0Pt92SIdnVdY/B3VXEWnWurRtxVs/GsFP31jMA9NXMuubPfzxygF0TNXqro1FVbNSWDqknXMznXMzgS3AV9WezwM2B1WiiDQLreNjeO76Ifzmkn7M3ZDP+Y/NZOrCPNUiGomGmgT3JlB9iqSn8piItGBRUcb4kTn8c+KZ9OmYzM/fXMKtkxeyv6gs0qG1eA3VIR1TuVsbAJWP44IqUUSanZx2Sbwx4XR+dXFfZn6zmzFPfsGyvMJIh9WiVa2tFO6awx4zu7TqiZldBuwNqkQRaZaio4xbzujOP247Ha/XcdWzc3hj3hY1M0VIWEcrVXM78Esz22JmW4F7gNuCKlFEmrXBXdvw/p1nMCynLfe+vYz/eXc5Hq8SREOr76qs/u4hvR4YbmatAXPOHQyqNBFpEdomxfHy+NP400dreHbmeg6VVPDI2IFBN3FI4I7UHBpgbaWLgZOBBDPfuFnn3G+DKlVEmr2qhftSEmP4f/9aQ0m5hyd/OJj4mOhIh9YilHu8xEQZUVHBzXPwd22lZ4EfAD/Bt7HP94FuQZUoIi3Kj87uyW8u6cfHK3cxYWLZvMsAAA8rSURBVPJCrezaQMoqvPWqqfl75Qjn3A1AgXPuAeB0oEvQpYpIizJ+ZA4PX3UKs9bu4bxHZzJl3hZtIBRm5R4XdGc0+J8ciiv/LTKzzkA5kBN0qSLS4vxgaFdeu2UY7ZPjue/tZXznkc94c8FWvOqsDovSBqo5vG9macCfgEX4NveZEnSpItIijejRjnd+NIKXxuWSmhjL3VOXct/byzTcNQzKPd6gl84A/0cr/a7y4Vtm9j6Q4JzTDBcRCZiZ8Z0+GZzTuwOPfPwNT81YR1J8DPeP6UvVYBepv0MlFcTHhjk5mFkC8CN8m/044Asze8Y5VxJ0ySLSopkZPxvdi0OlFbw0eyPJCTHcdX6vSIfVLDjnWLSlgOHd04O+h79DWScDB4EnK59fA7yKb9SSiEhQzIxfj+nH4dIKHv9kLa3jY7j1zO6RDqvJ25xfxO6DpZyW0zboe/ibHHo75wZWez7DzJYEXaqISKWoKOOhqwZQVObh9x+uYtm2Qu4f04/2yfGRDq3JmrdxHwDDuwefHPxtkPrazIZXPTGzYcDsoEsVEakmOsr4y9WDuOu8Xvxr+U7Oe3Qm/5i/VR3VQfpq4z7aJsXRo33roO9R105wy8xsKTAMmGNmm8xsIzAXODPoUkVEjhEbHcXE807iw4mj6J2RzC/eWsq1L3zFzkJ1bQbqq435nJbdtl4d/HU1K40J+s4iIkHo2SGZNyYM5435W3nwg5Vc9MTnPPL9gZzTp0OkQ2sStu0vJq+gmJtG1m8qWl07wW0+0Ve9ShYRqUVUlPHDYV2Z/pNRdEiOZ/zL8/n9ByuPLCYntZtf2d9Qn85o8L/PQUSkwfVo35p37xjJ9cO78fznG7n5lfla/rsOX23cR3JCDH07pdTrPkoOItKoJcRG87vL+/O7y/vz+dq9PPPZukiH1Kh9tTGfodltiQ5yNdYqSg4i0iRcN6wrlw7szGP/Wcv8TfsiHU6jtOdgKRv2HK53kxIoOYhIE2Fm/P6K/mSmJTJxytfsLyqr+6IWpippKjmISIuSnBDLk9cMZvfBUu55a6nmQRxj3sZ9JMZGc0pmar3vpeQgIk3KwC5p3HNBHz5asYtfvbuc3Qc1D6LKlxvyGdKtTUi2Y/V7m1ARkcbi5lE5bNlXxGtfbWbqwjyuOa0r/3V2DzJSEiIdWsTsLypjza6DXHRKp5DcL6w1BzN7ycx2m9nyWl5PNbPpZrbEzFaY2fhwxiMizUNUlPG7y/vz6c/O5tKBnXn1y82c8f9m8MHSHZEOLWIWbCrAudD0N0D4m5VeBi44wet3ACsrF/U7G3jEzOLCHJOINBPZ7ZL40/cHMuNnZ3NKZip3/WMxC1roSKbl231b7AzqkhaS+4U1OTjnZgEn+p9yQLL5FgBpXXluRThjEpHmp2t6K164IZfMtERumbyAjXsPRzqkBretoJiMlHgSYqNDcr9Id0g/BfQFtgPLgInOuePOjzezCWa2wMwW7NmzpyFjFJEmoE1SHC+PH0qUGeMmzSP/UGmkQ2pQeQXFZKYlhux+kU4O3wUWA52BQcBTZnbcOd/Oueecc7nOudz27ds3ZIwi0kR0S0/ihRtz2VlYwq2TF3CotOU0RGzbX0xmm1Yhu1+kk8N44G3nsw7YCPSJcEwi0oSd2rUNj189iCV5hXzvmTls218c6ZDCzuN17CgsJqtN86k5bAHOBTCzDKA3sCGiEYlIk3dB/05MGjeUbQXFXP70bJbm7Y90SGG1+2AJ5R7XdJqVzGwKvo2BeptZnpndbGa3m9ntlaf8DhhhZsuAT4B7nHN7wxmTiLQMZ/Zqz1s/GkFcdBRj/za3WQ9z3Vbgqx1lhrDmENZJcM65a+p4fTswOpwxiEjL1SsjmXfvGMmEVxdwx+uLeH1eOnd/t0/Ihns2FlVNZ12aUbOSiEhYtU+O540Jw7l/TD9W7TjI5U/P5rZXF7B+z6FIhxYyeZU1h85NpVlJRKQxiI+J5uZROcz6xTncdV4vZq/L5/KnZvP1loJIhxYSeQXFtE2Ko1Vc6BqDlBxEpMVoHR/DxPNO4qO7zqRNUhzXvziPhZubfoLYtj+0I5VAyUFEWqDMtET+77bhtGsdxw0vftXkl9zIKygK6UglUHIQkRaqU2oib0w4nYyUBG54aR5fbsiPdEhBcc6xfX9oZ0eDkoOItGAdUxN4Y8JwOqUmcMOL85rkcNf8w2WUlHvVrCQiEkodUhKYevsITslK5cdTFvHiFxsjHVJA8o7McQjd0hmg5CAiQpukOF67ZRij+2Xwu/dX8uD7K/F6m8YWpEcmwKlZSUQk9BJio/nrtUMYNyKbF77YyB2vL6K4zBPpsOq0bX8RENrZ0aDkICJyRHSU8ZtL+vGri/vyrxU7ufq5uY1+j+q8gmKSE2JITYwN6X2VHEREqjEzbjmjO89dn8s3uw5xxdNzWL3zQKTDqtW2EO/jUEXJQUTkOM7vl8Gbt59OhdfL956Zy1eNdKirbwJcaDujQclBRKRW/TNTefeOkWSkxDNu0nzmrm9cCcI5R15B6GdHg5KDiMgJVU2Wy2qTyPiX5zF7XePZVeBAcQWHSivUrCQiEgntk+OZMmE42elJ3PTyfGZ90zj2sc+rHKmkmoOISIS0ax3P67cOp3v71tzyyoJGMZs6Lwyb/FRRchAR8VPbpDim3DrsyGzql2dHdjZ1uCbAgZKDiEhA0lr5ZlOf3zeD/52+kof+uTpis6m37S8mMTaatklxIb+3koOISIASYqN55rohXDusK8/OXM/dU5fiiUCCyCsoIrNNImYW8nuHdQ9pEZHmKjrKePDy/nRITuCx/3xDucfLo2MHEhPdcH9zbwvDUt1VlBxERIJkZkw87yTiYqJ4+F+r8Xgdf7l6ELENlCC2FRQzMCstLPdWchARqaf/OrsHsdHGgx+sosLr5clrTiUuJrwJ4nBpBQVF5WEZqQRKDiIiIXHLGd2JjjIemL6Sy56ezcRzezK6X0eiomrvD5i/aR/Tl2xn76FS9h4sY+/hUtolxXPd6d24sH/HE9ZAthZUzXEI/dIZoOQgIhIy40fm0CE5gT9/vIbb/76I3hnJ3PGdnozul0FCbPSR8/IKivjjP1fzwdIdJMVF0yktkfSkOPp0TGbl9gPcOeVrOqYkcP3p3bjmtK41RiMt31bIj19fRHSU0b9zSljeiznXNDa0qC43N9ctWLAg0mGIiBxXhcfLB8t28OSn61i3+xAxUUbvjskMyEojMTaa177ajBncdmYPbjurO63ivv073et1zFizm5fnbOLztXuJi4nikgGduXFEN07JTGXy3M38/oNVtE2K44lrBnNaTlu/4zKzhc65XL/OVXIQEQkPj9cxa+0eFmzax9K8QpZs3c+BkgouHdiZey7sU+dIo7W7DjJ57mbeWpRHUZmHzLREtu0v5jt9OvDn7w8MeH6DkoOISCPk9ToOllYEvDHPgZJy3l6Yx4fLdzK6XwY3j8oJam6DkoOIiNQQSHLQDGkREalByUFERGpQchARkRqUHEREpAYlBxERqUHJQUREalByEBGRGpQcRESkhiY5Cc7MDgJrgrg0FSgM4hx/jlV/frzHVf+2A/YGFHXtMfh7zolirS3e6o9bcuwniq2u1xW7T7g+Ny0hdn/ire3x8WLv5pxrX1fwADjnmtwXsCDI654L5hx/jlV/frzH1f5t0NjrilWx1z9+xV577OH83LSE2P2J9wTvI+jPjXOuxTUrTQ/yHH+OTa/jsT9ln0iwsR/vuGIPTF33UOw1Hyv24F6v63dNoI+D1lSblRY4P9cHaWwUe2Qo9shpyvG35Nibas3huUgHUA+KPTIUe+Q05fhbbOxNsuYgIiLh1VRrDiIiEkZKDiIiUoOSg4iI1NDskoOZnWFmz5rZC2Y2J9LxBMLMoszs92b2pJndGOl4AmFmZ5vZ55Xf+7MjHU+gzCzJzBaa2ZhIxxIIM+tb+T2famb/Fel4AmFml5vZ82b2npmNjnQ8gTCz7mb2oplNjXQs/qj8fL9S+f2+1p9rGlVyMLOXzGy3mS0/5vgFZrbGzNaZ2b0nuodz7nPn3O3A+8Ar4Yy3ulDEDlwGZALlQF64Yj1WiGJ3wCEggaYXO8A9wD/CE+Xxhejzvqry8z4WaLAhlyGK/V3n3K3AOOAHYQz3KCGKfYNz7ubwRnpiAb6PK4Gpld/vS/0qoD4z6EL9BZwJnAosr3YsGlgPdAfigCVAP+AUfAmg+leHatf9A0hpSrED9wK3VV47tYnFHlV5XQbwWhOL/Tzgany/pMY0pdgrr7kUmAP8sKnFXnndI8CpTTT2Bvs5ref7uA8YVHnO6/7cP4ZGxDk3y8yyjzl8GrDOObcBwMzeAC5zzv0ROG4TgJl1BQqdcwfCGO5RQhG7meUBZZVPPeGL9mih+r5XKgDiwxHn8YTo+34OkITvh6jYzD50znnDGjih+74756YB08zsA+D18EV8VJmh+L4b8BDwT+fcovBG/K0Qf94jJpD3ga82nwUsxs8Wo0aVHGqRCWyt9jwPGFbHNTcDk8IWkf8Cjf1t4EkzOwOYFc7A/BBQ7GZ2JfBdIA14Kryh1Smg2J1z/wNgZuOAvQ2RGE4g0O/72fiaDOKBD8MaWd0C/bz/BF+tLdXMejrnng1ncHUI9PueDvweGGxm91UmkcagtvfxBPCUmV2Mn8trNIXkYMc5dsKZe86534QplkAFFLtzrghfYmsMAo39bXzJrTEI+DMD4Jx7OfShBCzQ7/tnwGfhCiZAgcb+BL5fWo1BoLHnA7eHL5ygHfd9OOcOA+MDuVGj6pCuRR7QpdrzLGB7hGIJlGKPDMUeGYo98kL2PppCcpgPnGRmOWYWh6/jcFqEY/KXYo8MxR4Zij3yQvc+ItXTXkvv+xRgB98O5by58vhFwDf4euH/J9JxKvbG86XYFXtLir0h34cW3hMRkRqaQrOSiIg0MCUHERGpQclBRERqUHIQEZEalBxERKQGJQcREalByUEkBMzsUKRjEAklJQeRMDGz6EjHIBIsJQeREDLfjngzzOx1YFmk4xEJVlNYlVWkqTkN6O+c2xjpQESCpZqDSOjNU2KQpk7JQST0Dkc6AJH6UnIQEZEalBxERKQGLdktIiI1qOYgIiI1KDmIiEgNSg4iIlKDkoOIiNSg5CAiIjUoOYiISA1KDiIiUoOSg4iI1PD/ASVMVSDoGx4QAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "batch_size = 256\n",
    "lr_finder = model.lr_finder(x_train, y_train, batch_size, tolerance=6)\n",
    "_ = lr_finder.plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0613590727341321"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_finder.get_best_lr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Often, this learning rate is a little high, so we instead set it manually to 0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.optimizer.set_lr(0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We include the `EarlyStopping` callback to stop training when the validation loss stops improving. After training, this callback will also load the best performing model in terms of validation loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\t[0s / 0s],\t\ttrain_loss: 1.7319,\tval_loss: 1.4576\n",
      "1:\t[0s / 0s],\t\ttrain_loss: 1.5639,\tval_loss: 1.4277\n",
      "2:\t[0s / 0s],\t\ttrain_loss: 1.5177,\tval_loss: 1.4187\n",
      "3:\t[0s / 0s],\t\ttrain_loss: 1.4696,\tval_loss: 1.4052\n",
      "4:\t[0s / 0s],\t\ttrain_loss: 1.4436,\tval_loss: 1.3784\n",
      "5:\t[0s / 0s],\t\ttrain_loss: 1.4135,\tval_loss: 1.3748\n",
      "6:\t[0s / 0s],\t\ttrain_loss: 1.3904,\tval_loss: 1.3758\n",
      "7:\t[0s / 0s],\t\ttrain_loss: 1.3808,\tval_loss: 1.3634\n",
      "8:\t[0s / 0s],\t\ttrain_loss: 1.3674,\tval_loss: 1.3626\n",
      "9:\t[0s / 0s],\t\ttrain_loss: 1.3438,\tval_loss: 1.3766\n",
      "10:\t[0s / 0s],\t\ttrain_loss: 1.3339,\tval_loss: 1.3773\n",
      "11:\t[0s / 0s],\t\ttrain_loss: 1.3352,\tval_loss: 1.3672\n",
      "12:\t[0s / 0s],\t\ttrain_loss: 1.3296,\tval_loss: 1.3674\n",
      "13:\t[0s / 0s],\t\ttrain_loss: 1.3137,\tval_loss: 1.3677\n",
      "14:\t[0s / 0s],\t\ttrain_loss: 1.3214,\tval_loss: 1.3727\n",
      "15:\t[0s / 0s],\t\ttrain_loss: 1.3094,\tval_loss: 1.3679\n",
      "16:\t[0s / 0s],\t\ttrain_loss: 1.3053,\tval_loss: 1.3698\n",
      "17:\t[0s / 0s],\t\ttrain_loss: 1.3078,\tval_loss: 1.3727\n",
      "18:\t[0s / 0s],\t\ttrain_loss: 1.2979,\tval_loss: 1.3648\n"
     ]
    }
   ],
   "source": [
    "epochs = 100\n",
    "callbacks = [tt.callbacks.EarlyStopping()]\n",
    "log = model.fit(x_train, y_train, batch_size, epochs, callbacks, val_data=val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xUdb7/8dc3mUnvlZCQQm+BAElAQUVsNMFVVOy47vXay726ur/d1dV177prd9eVdXeBa7mWBUVRsKGIDUhAOqEHEkoSEtL7zPf3x5mEAIG0Sc7M5PN8POYx7cw5n5DwPme+53u+X6W1RgghhOfyMrsAIYQQ3UuCXgghPJwEvRBCeDgJeiGE8HAS9EII4eEsZm04KipKJycnm7V5IYRwS+vXrz+mtY7uyGdMC/rk5GSys7PN2rwQQrglpdSBjn5Gmm6EEMLDSdALIYSHk6AXQggPZ1obvRDC8zQ0NJCfn09tba3Zpbg9Pz8/EhISsFqtXV6XBL0Qwmny8/MJDg4mOTkZpZTZ5bgtrTXFxcXk5+eTkpLS5fVJ040Qwmlqa2uJjIyUkO8ipRSRkZFO+2YkQS+EcCoJeedw5r+jaUFfWFFn1qaFEKJXMS3oC8prqaxrNGvzQgjRa5jadLPhwHEzNy+E8DClpaX87W9/6/Dnpk+fTmlpaYc/N2/ePBYvXtzhz/U004JeAWv3F5u1eSGEBzpT0NtstrN+bvny5YSFhXVXWaYzrXuln9WbdftLzNq8EKKbPbFsG9sPlzt1ncP7hvD45SPO+P6jjz7K3r17SUtLw2q1EhQURFxcHBs3bmT79u1cccUV5OXlUVtby/3338/tt98OnBh7q7KykmnTpjFp0iR++OEH4uPj+fDDD/H392+ztpUrV/LQQw/R2NhIRkYGr776Kr6+vjz66KN89NFHWCwWLr30Up599ln+/e9/88QTT+Dt7U1oaCirV6922r9Ra0wL+iBfC5vyyqhtsOFn9TarDCGEB3n66afZunUrGzduZNWqVcyYMYOtW7c290VfsGABERER1NTUkJGRwVVXXUVkZORJ69i9ezdvv/02//jHP7jmmmtYsmQJN95441m3W1tby7x581i5ciWDBw/m5ptv5tVXX+Xmm2/mgw8+ICcnB6VUc/PQk08+yWeffUZ8fHynmow6yrSgD/T1ps5mZ2NeKRP6R7b9ASGEWznbkXdPyczMPOmCo5dffpkPPvgAgLy8PHbv3n1a0KekpJCWlgbAuHHjyM3NbXM7O3fuJCUlhcGDBwNwyy238Morr3DPPffg5+fHL37xC2bMmMHMmTMBmDhxIvPmzeOaa67hyiuvdMaPelamtdEH+FhQCmm+EUJ0m8DAwObHq1at4ssvv+THH39k06ZNjBkzptULknx9fZsfe3t709jYdu9ArXWrr1ssFtatW8dVV13F0qVLmTp1KgDz58/nqaeeIi8vj7S0NIqLu/d8pWlH9N5eiqF9QhwnZAeZVYYQwoMEBwdTUVHR6ntlZWWEh4cTEBBATk4Oa9ascdp2hw4dSm5uLnv27GHgwIG88cYbXHDBBVRWVlJdXc306dOZMGECAwcOBGDv3r2MHz+e8ePHs2zZMvLy8k77ZuFMpo51Mz4lgneyDlLfaMfHIhfpCiG6JjIykokTJzJy5Ej8/f2JjY1tfm/q1KnMnz+fUaNGMWTIECZMmOC07fr5+bFw4UKuvvrq5pOxd9xxByUlJcyePZva2lq01rzwwgsAPPzww+zevRutNRdddBGjR492Wi2tUWf6ytHd0tPT9e8XLuPOtzbw/l3nMjYx3JQ6hBDOs2PHDoYNG2Z2GR6jtX9PpdR6rXV6R9Zj6mF0RkoEIO30QgjRnUwN+qggXwZEB7J2n1w4JYRwXXfffTdpaWkn3RYuXGh2We3WZhu9UmoBMBMo1FqPbOX9h4EbWqxvGBCttW7XYfr4/pEs23gYm13j7SWj3gkhXM8rr7xidgld0p4j+kXA1DO9qbV+RmudprVOA34FfNPekAfjhGxFXSM7jjj3CjohhBCGNoNea70aaG9wXwe83ZECMpKlnV4IIbqT09rolVIBGEf+S86yzO1KqWylVHZRUREAfcP86RfhLwOcCSFEN3HmydjLge/P1myjtX5Na52utU6Pjo5ufn18SiTr9pec8eoyIYQQnefMoJ9LB5ttmmSmRHC8uoE9hZVOLEcIIdoWFBR0xvdyc3MZOfK0PihuxylBr5QKBS4APuzM58c7+tOvlXZ6IYRwuvZ0r3wbmAxEKaXygccBK4DWer5jsZ8Bn2utqzpTRGJEALEhvqzdX8KNE5I6swohhKtZ8Sgc3eLcdfZJhWlPn3WRRx55hKSkJO666y4Afve736GUYvXq1Rw/fpyGhgaeeuopZs+e3aFN19bWcuedd5KdnY3FYuH555/nwgsvZNu2bdx6663U19djt9tZsmQJffv25ZprriE/Px+bzcZvf/tbrr322k7/2F3VZtBrra9rxzKLMLphdopSisyUSNbtL0ZrLbPICyE6be7cuTzwwAPNQf/ee+/x6aef8uCDDxISEsKxY8eYMGECs2bN6lDWNPWl37JlCzk5OVx66aXs2rWL+fPnc//993PDDTdQX1+PzWZj+fLl9O3bl08++QQwBlQzk6mDmrU0PiWCZZsOc7CkmqTIwLY/IIRwbW0ceXeXMWPGUFhYyOHDhykqKiI8PJy4uDgefPBBVq9ejZeXF4cOHaKgoIA+ffq0e73fffcd9957L2CMVpmUlMSuXbs455xz+MMf/kB+fj5XXnklgwYNIjU1lYceeohHHnmEmTNnct5553XXj9suLjNkpLTTCyGcZc6cOSxevJh3332XuXPn8tZbb1FUVMT69evZuHEjsbGxrY5FfzZn6hV4/fXX89FHH+Hv789ll13GV199xeDBg1m/fj2pqan86le/4sknn3TGj9VpLhP0A2OCiAj0kQunhBBdNnfuXN555x0WL17MnDlzKCsrIyYmBqvVytdff82BAwc6vM7zzz+ft956C4Bdu3Zx8OBBhgwZwr59++jfvz/33Xcfs2bNYvPmzRw+fJiAgABuvPFGHnroITZs2ODsH7FDXKbpRilFRnK4XDglhOiyESNGUFFRQXx8PHFxcdxwww1cfvnlpKenk5aWxtChQzu8zrvuuos77riD1NRULBYLixYtwtfXl3fffZc333wTq9VKnz59eOyxx8jKyuLhhx/Gy8sLq9XKq6++2g0/ZfuZOh59dnb2Sa8t+G4/T368nR8enULfsLZnXRdCuBYZj965PGI8+lNlOtrps3Kl+UYIIZzFZZpuAIbFhRDsa2Ht/hJmp8WbXY4QopfYsmULN91000mv+fr6snbtWpMqci6XCnpvL0V6crhMRCKEG3PHa2FSU1PZuHGj2WWcxJnN6i7VdAPGRCR7i6o4VllndilCiA7y8/OjuLhYBijsIq01xcXF+Pn5OWV9LnVEDy3a6feXMC01zuRqhBAdkZCQQH5+Pk3DkIvO8/PzIyEhwSnrcrmgH9k3FH+rN2sl6IVwO1arlZSUFLPLEKdwuaYbH4sXY5PC5ApZIYRwEpcLeoDM5EhyjpZTVt1gdilCCOH2XDLox/ePQGvIPiBH9UII0VUuGfRp/cLw8faScW+EEMIJXDLo/azejO4XyhoJeiGE6DKXDHowulluPVRGVV2j2aUIIYRbc9mgH58Sic2u2XDwuNmlCCGEW3PZoB+bFI63l5J2eiGE6CKXDfogXwsj+4ZIf3ohhOgilw16MNrpN+aVUttgM7sUIYRwWy4d9ONTIqlvtLMpr9TsUoQQwm25dNBnJEegFNJOL4QQXeDSQR8aYGVIbDDrZMYpIYToNJcOeoDxKRGsP3CcBpvd7FKEEMItuXzQZ6ZEUl1vY+uhMrNLEUIIt+QGQW9MRCLt9EII0TkuH/TRwb70jw6UoBdCiE5y+aAHo51+XW4JNrvMQymEEB3lFkGfmRJBRW0jOUfLzS5FCCHcjlsE/fiUSEDa6YUQojPcIuj7hvmTEO4vQS+EEJ3gFkEPRvPNuv0laC3t9EII0RFuE/TjUyIorqpnb1Gl2aUIIYRbcaOgN9rpZdhiIYToGLcJ+qTIAGKCfaWdXgghOqjNoFdKLVBKFSqltp5lmclKqY1KqW1KqW+cW2LzNshMiWDtPmmnF0KIjmjPEf0iYOqZ3lRKhQF/A2ZprUcAVzuntNONT4ngaHkteSU13bUJIYTwOG0GvdZ6NXC29pLrgfe11gcdyxc6qbbTZDa30xd31yaEEMLjOKONfjAQrpRapZRar5S6+UwLKqVuV0plK6Wyi4qKOryhQTFBhAdYpZ1eCCE6wBlBbwHGATOAy4DfKqUGt7ag1vo1rXW61jo9Ojq6wxvy8lJkJEfIRCRCCNEBzgj6fOBTrXWV1voYsBoY7YT1tiozJYIDxdUcLavtrk0IIYRHcUbQfwicp5SyKKUCgPHADiest1XjpZ1eCCE6pD3dK98GfgSGKKXylVK3KaXuUErdAaC13gF8CmwG1gH/1FqfsStmVw3vG0KQr0Xa6YUQop0sbS2gtb6uHcs8AzzjlIra4O2lSE8Ol6AXQoh2cpsrY1vKTIlgd2ElxZV1ZpcihBAuzy2DfrxjHtks6X0jhBBtcsugT40Pw8/qJQOcCSFEO7hl0PtYvBibKO30QgjRHm4Z9GC0028/Uk5ZTYPZpQghhEtz66DXGtYfkKN6IYQ4G7cN+jH9wrF6K2mnF0KINrht0Pv7eDM6IUza6YUQog1uG/RgNN9syS+jur7R7FKEEMJluX3QN9o1Gw6Uml2KEEK4LLcO+nFJ4Xh7KT746ZBMLyiEEGfg1kEf7GflF+elsGRDPv/4dp/Z5QghhEtqc1AzV/fIZUPJP17D/yzPoU+oP7NG9zW7JCGEcCluH/ReXornrh5NUUUdD723ieggX84ZEGl2WUII4TLcuummiZ/Vm3/clE5iZAC3v5HNroIKs0sSQgiX4RFBDxAaYGXRrRn4W72Zt2CdTDUohBAOHhP0AAnhASy8NYOymgbmLVxHRa2MgyOEEB4V9AAj+oby6o3j2FNYyZ1vbqC+0W52SUIIYSqPC3qA8wdH8/RVo/huzzEeXbJZ+tgLIXo1t+91cyZzxiVwpLSG577YRd8wfx66bIjZJQkhhCk8NugB7pkykMNlNfz16z3Ehflxw/gks0sSQoge59FBr5Ti97NHcrSslt8u3UqfED8uGhZrdllCCNGjPLKNviWLtxd/vX4sI+NDuef/fmJjngyAJoToXTw+6AECfS3865YMooJ9uG1RFgeKq8wuSQghekyvCHqA6GBfFt2aiU1rblmwjuLKOrNLEkKIHtFrgh5gQHQQ/7olnSNltfzi9Wxq6m1mlySEEN2uVwU9wLikCF6am8bGvFLue+cnbHbpYy+E8Gy9LugBpo6M4/GZw/liewG/+2ibXFAlhPBoHt298mzmTUzhcFktr63eR3y4P3dcMMDskoQQolv02qAHeHTqUA6X1vD0ihziQv2YnRZvdklCCOF0vTrovbwUz13jmLTk35uIDvbl3AFRZpclhBBO1Svb6FvytXjz2k3pJEcG8p+vr5dJS4QQHqfXBz04Ji35eSZ+Pt78fFEWx6SPvRDCg0jQO8SH+fPPm9M5VlnHf7yeTW2D9LEXQngGCfoWRvcL44Vr0vjpYCkPL5Zx7IUQnkGC/hTTUuN4ZOpQlm06zAtf7ja7HCGE6LJe3evmTO64oD/7j1Xy8srdpEQF8LMxCWaXJIQQndbmEb1SaoFSqlAptfUM709WSpUppTY6bo85v8yepZTiqStSmdA/gkcWbyErt8TskoQQotPa03SzCJjaxjLfaq3THLcnu16W+XwsXsy/cRwJ4f7c/nq2DG0shHBbbQa91no10CsPacMCfFgwLwMN3Looi7LqBrNLEkKIDnPWydhzlFKblFIrlFIjzrSQUup2pVS2Uiq7qKjISZvuXslRgfz9xnHklVRz51vrabDZzS5JCCE6xBlBvwFI0lqPBv4CLD3Tglrr17TW6Vrr9OjoaCdsumeM7x/J01eO4oe9xfzmg63S7VII4Va6HPRa63KtdaXj8XLAqpTyuAFjrhqXwL1TBvJudh6vrd5ndjlCCNFuXQ56pVQfpZRyPM50rLO4q+t1RQ9ePJgZo+J4+tMcPt161OxyhBCiXdrsR6+UehuYDEQppfKBxwErgNZ6PjAHuFMp1QjUAHO1h7ZteHkpnrt6NIeO1/DAuz/x77BzSU0INbssIYQ4K2VWJqenp+vs7GxTtt1VRRV1XPHK9zTY7Cy9eyJ9w/zNLkkI0UsopdZrrdM78hkZAqETooN9WXhrBjX1Nm7732yq6hrNLkkIIc5Igr6TBscG89cbxrKroIL73pZJxoUQrkuCvgsuGBzN72aNYGVOIX/4ZIfZ5QghRKtkULMuumlCEvuLqljw/X5SogK46Zxks0sSQoiTSNA7wa9nDONAcRW/W7adxMhALhjsPheDCSE8nzTdOIG3l+Ll68YwODaYe97awM6jMu+sEMJ1SNA7SaCvhX/dko6/Y97ZogqZd1YI4Rok6J2ob5g//7olg5Kqeq79+4+s298rB/0UQrgYCXonS00IZeGtGdTb7Fzz9x/5fx9sobxWhjcWQphHgr4bTOgfyecPns8vJqXwzrqDXPzcN3y69YjZZQkheikJ+m4S4GPhNzOHs/TuiUQF+XLHmxv4zzeyKSivNbs0IUQvI0HfzUYlhPHhPRN5dNpQVu0s4uLnvuHNNQewy5W0QogeYl7Qlx+CxnrTNt+TrN5e3HHBAD574HxSE0L5zdKtXPvaj+wplG6YQojuZ17QVxbCgsugZL9pJfS05KhA3vrFeJ6ZM4pdBZVMf+k7XvxyF3WNNrNLE0J4MPOCPiIFSvbC/PNgy2LTyuhpSimuTu/Hyv++gKkj+/Dil7uZ+fJ3rD8gXTGFEN3DvKD3C4M7voOYYbDkNvjwHqivMq2cnhYV5MvL141h4bwMquttzJn/I79dupUK6YophHAyc0/GhiXCrcvhvP+Gn96E1y6Egm2mltTTLhwaw+cPns+8c5N5c+0BLnl+NZ9vk2kKhRDOY36vG28rXPQY3PQB1JYaYZ/1T/DM2QhbFehr4fHLR/DBXRMJC7By+xvrueut9RRKV0whhBOYH/RNBlwId3wPyZPgk/+G926CmuNmV9Wj0vqFsezeSTx82RC+3FHIRc9/w9vrDuKhU/AKIXqI6wQ9QFA03LAYLvk97FxhnKg9uNbsqnqU1duLuy8cyKf3n8eIviH86v0t/HxRFiVVvaMrqhDC+Vwr6AG8vGDiffDzz0F5wcJpsPpZsPeuLoj9o4N4+z8m8MSsEXy/p5hpL61mzb5is8sSQrgh1wv6Jgnj4I5vYfhs+Or38MYVUNG7TlIqpbjl3GTev+tcAnwsXP+PNbz45S6Zn1YI0SGuG/QAfqEwZwHM+gvkZcGrE2H3F2ZX1eNGxoey7N5JXJEWz4tf7uaGf66RMXOEEO3m2kEPoBSMvRluXwVBsfDWHPj8N71m+IQmQb4Wnr82jWevHs2mvDKmvfQtX+8sNLssIYQbcP2gbxIzFP5jJaTfBj/8pdcNn9BkzrgElt07iZhgX25dmMUfPtlOfaPd7LKEEC7MfYIewOoPM5+Ha17vlcMnNBkYE8TSuydy04Qk/vHtfq6e/wMHi6vNLksI4aKUWX2009PTdXZ2dudXUHoQFt8G+esgNhX6ZUBCJvTLhIj+RpNPL7BiyxF+uWQzaPjjVanMHNXX7JKEEN1IKbVea53eoc+4bdAD2Bpg7d9hzxeQvx7qHcP++kdAQoYj/DMgfhz4Bne9aBeVV1LNfe/8xE8HS7kuM5HHLx+On9Xb7LKEEN2g9wV9S3YbFO00jvDzs4xeOsd2Ot5UEDP8RPAnZELkQKPPvodosNl57vNdzP9mL0Nig/nr9WMYFOu5OzcheqveHfStqSmFQ9lG6OdnQX421JUZ7/mFQUK6o7nHcdTvF9q99fSAb3YV8V/vbqSqvpEnZ43k6vQEVC9pxhKiN5Cgb4vdDsd2OUJ/nbEDKMoBNM1H/UOnw/ArIHaE27bzF5bX8sC7G/lhbzGz0/ry1BUjCfazml2WEMIJJOg7o7YMDq03jvb3r4YD34O2Q8QAGHGFcWVun1FuF/o2u+ZvX+/hhS93kRgRwF+uG0tqgvt/YxGit5Ogd4bKQsj5GLZ/CPu/BW2D8BQj8EdcAXFpbhX66/aXcP87P3Gsso5fXjaUWycmY/H2nHMTQvQ2EvTOVnUMcj6B7Uth3zdG6IclGaE//AqIH+sWoX+8qp6HF2/myx0FDIwJ4tczhjF5cLS03QvhhiTou1N1CexcDtuWwr5VYG+A0H4tQn+cS/fi0Vrz+fYC/rh8B7nF1Zw3KIrfzBjOkD7SM0cIdyJB31Nqjhvj5W//EPZ+BbZ6CImHYbOM5p2ETJcN/fpGO2+sOcBLX+6isq6RuZmJ/Nclg4kK8jW7NCFEO3RL0CulFgAzgUKt9cizLJcBrAGu1Vq3OS6BWwd9S7VlsPNTI/T3fAm2OgiOg3HzYNKDYHHNAD1eVc9LK3fzxpoD+Fu9ufvCgdw6MVkutBLCxXVX0J8PVAKvnynolVLewBdALbCgVwV9S7XlsPtzY/ydXSsgeijMfsXor++i9hRW8vSKHXy5o5CEcH8enTaUGalx0n4vhIvqTNC32b6gtV4NlLSx2L3AEqB3j5vrFwKpc+D6d+CGJVBXAf+6BD77NTTUmF1dqwbGBPHPWzJ487bxBPlauOf/fmLO/B/ZmFdqdmlCCCfpckOyUioe+Bkwvx3L3q6UylZKZRcVFXV1065t0MVw1xoYewv8+Fdj0pQDP5hd1RlNGhTFJ/edx9NXpnKguJorXvmeB975iUOlrrmDEkK0nzPOGL4IPKK1bnNSV631a1rrdK11enR0tBM27eL8QuDyF+Hmj8DeCAunw/JfQl2l2ZW1yttLMTczkVUPT+buCwewfOtRpjy7iuc+30lVXaPZ5QkhOqldvW6UUsnAx6210Sul9gNNDbpRQDVwu9Z66dnW6ZFt9GdTV2nMfbv27xDWz5gesf9ks6s6q/zj1fz50518tOkw0cG+PHzpEK4al4C3l7TfC2GWbmmjb4vWOkVrnay1TgYWA3e1FfK9km8QTPsT3LoCvH3g9dmw7H6j146LSggP4OXrxvD+XefSL9yfXy7ZzMy/fMcPe46ZXZoQogPa0+vmbWAyxtF6AfA4YAXQWs8/ZdlFGEf+vbPXTXs11MDX/2O03QfHweUvwaBLzK7qrLTWfLz5CE+vyOFQaQ2DYoJIT44gMyWc9KQIEsL9paeOED1ALphyN/nr4cO7oWgHjL4OLvsfCIgwu6qzqm2w8fa6g6zeVUT2geNU1Bpt931C/EhPDiczJYL0pAiG9AmWJh4huoEEvTtqrIPVz8C3z0NgFMx4HobNNLuqdrHZNbsKKsjOLSEr9zhZuSUcKasFINjXwtikcDKSw8lIjmB0vzC5GEsIJ5Cgd2dHNhlH90e3wIgrYfozRvC7Ea01h0pryHaEflZuCbsKjB5GVm9FanwoGckRpCdHkJ4UTnigj8kVC+F+JOjdna0Bvn8RVv3J6Jo57c8w8iq3GCHzTEqr61l/4HjzEf/m/FIabMbf3KCYIDJTIrh0RB/OHRCJVYZPFqJNEvSeonAHLL0LDm+AIdNh8GUQGH3yzSfQLXcAtQ02NueXNR/xZ+0voareRqi/lUuHxzI9NY6JA6PwsUjoC9EaCXpPYmuENX8zeuc0tnJ1qsXfCPygpvCPctzHnPw8KAb8I8DbcuZt2e3GsMu2euNbha3+9MeN9Sce2xshegiEJXb5x6xtsPHt7mOs2HKEL7YXUFHXSLCfhUuGxzJ9ZByTBkX1XNt+Qw3s+swYr6iu3Pgd2Bsd/zYtHtsbHc9be9xgTFRvbzAe+wZD0rmQNBGSJ0GfVPCScxWi8yToPVFjPVQVnX6rLDQmRqkqgqoWj+2tXcGqjN48vsFGKNnqjVE2m4K81c+0Q+xIGDLNuMWN6fLQzHWNNr7fc4zlW47y+bajlNc2EuRr4eJhMUxLjeOCwdHOD/3GOmOo6a1LjKGn6yshIBKCYo1A9rKClwW8HfenPm7rvaoiY3rKkn3G9nxDIekcI/STJxnTVErwiw6QoO/ttDbGym9tB1BVZAyy5u1jhJG3r+Pep8VrPsawyqe97nvyMsrLmFx95wo4+KMxx25QLAyeaoR+ygXgE9ClH6W+0c4Pe4+xYstRPtt+lNLqBgJ9vJkyLJbpI/sweUgM/j6dDEhbA+z/Bra+Dzs+hroy8A835hMYeZURwM4O3/LDkPs95H4Lud9ByV7jdd8QSDwl+M/27cudNdRAaR6UHoSyg8Z9063iqPE7CEuE0ARjUp/QBOMq8tBE4xuqqzRV2u3GN76a463cSlt/vbEWYoZB3Gjjdxw32vj5OvEzSdCLnlddAru/MGbf2rMS6iuMZqUBFxrBP3gqBMd2aRMNNjtr9hWzfMtRPtt2lJKqevyt3kwZGsO01D5cOCSGQN82wtFuM46st75vzB1QU2KE7NCZMPJKYzgKb2uX6uyQ8iNGPbnfGjuA4t3G6z7Bpxzxj3af4K+vOhHkpQegLO/kMK86ZSBDL+uJMA+OMwKxNM/4XP0p40FZ/Bw7gKadQD/HTsCxQwiJB0sbvbi0NnY29ZXGra7SqPm05xXGfV2lceX6qaFdW2oc3JyJT7Cx0/IPc9yHGwcOBdvh2M4Tn/WPgDhH6MeNNn7XEf3b/GYsQS/M1VgPB74zJmLZucI4agOIT4chU40TyzHDu3Rk1mizs25/Ccu3HuHTrQUcq6zDz+rF5MExXJvRj8lDWsyFa7dDfpbRLLN9KVQWgDXA+NYx8ioYcBFY/ZzwgztBxVHjSP/A98b9sV3G6z7BkDgBkica35q0dgSFPuWx3fFcn/L8lPebH9tB0+Kx/ZT37Cev80zL1Bw/EeTVxSf/TN4+jkBONEI5LNGYczks0Xg9uE/r35y0NsK0NA/K8o3gL8s7sRMoyzd+lydRxvpC+xnB2jLA66tOPD9bQLfkZQGfICHMDCsAAA5aSURBVPALNdYXEHEitM928ws7+w6nvhoKt8ORjXBks9GtunC70YTa9Pvuk+oIf8dOIGrISTt7CXrhOrSGgm3GBCw7V8Ch9cbroYkn2vWTJrZ9FHYWNrsmK7eEFVuOsGLrUQor6jhvYCR/GN9I4pFPYesHUJ5vND0NvtQI90GXdblZqUdUFJwI/dzvjCPB7qa8Tr6hTnmN09/3C3UEeFOYJ514HhjTfVNqNtRC+aETwd+8E8gzjsJ9goyeaU33vsGO502vBRnjT/kEGuHa9F7Tcj05M1xjPRTlwFFH8B/ZZFxP01BtvG/xMw6QHEf+KuPnEvTCRVUcNXq07FxhTK7eWGM0nSRNNK4Z8LIaRy1elhaPrY6Tm1bjyK/pcSvvNSoL27JWEZ77MYkUYFMW7P2nYB01x9ip+IWY/S/QNVXFRruw8jK+ETUHsTollJue045lvVos4yLt38Jgt0HxXkfwbzyxE6gtQz1RLkEv3EB9tXEydOdyyFtnnKhq6qJoa9lN0fFaeylvGhLPY5l9Ak/u6Q/+4fzXJYO5PjMRi1yMJdyd1lB6ABWRIkEvPIzWJ/dLtze22BE0nLyDCImHwEgAdhwp58ll2/lxXzGDY4N4bOYIJg1yryElhGiNtNEL0YLWms+2FfCH5dvJK6nh4mGx/GbGMJKjAs0uTYhOM2XiESFclVKKqSP78MWDF/DLqUP4Ye8xLnnhG/64YgcVtR1oEhLCzUnQC4/nZ/XmrskDWfXQZGanxfP3b/Zx4bPf8F5WHna7Od9ohehJEvSi14gJ8ePZq0fz4d0TSYwwpkac9cp3ZOWWmF2aEN1Kgl70OqP7hbHkznN5aW4axyrquXr+j9zzfxs4VNrK4HFCeAAJetErKaWYnRbPVw9dwH0XDeKL7QVMeXYVz3+xi+r6Tg7yJoSLkl43QgD5x6t5ekUOH28+Qp8QP2aMiiMzJYKM5AgiZCYs4UKke6UQXbRufwkvr9xNVm4JdY3GuCiDY42ZsDJTIhmfEkFsiIuMjyN6JQl6IZykrtGYCWvd/hLW7i9hfa4xExZAUmQAmckRZKZEMD4lkn4R/icGUhOim0nQC9FNGm12th8pbw7+rNwSSquNvvh9QvwcR/wRjE+JYGBMkAS/6DYS9EL0ELtds7uwknX7i1nrCP+iijoAIgJ9yEgOJzMlkiGxwTTY7dQ12KlrtFHXaDduDS0eN9oc77dYppXlIwJ9SIoMIDEikOTIABIjA0iKDCSorbH4hUeRoBfCJFprDhRXNx/xr8stJq+kfd01fS1exs3qja/FCx+LF74W75Ne9/FWHKus52BJNSVV9Sd9PjLQxwj9CCP4kyIDmncIUUE+8u3Cw0jQC+FCDpfWcLCk2hHY3vhavU577OPt1eEgLq9t4GBxNQdLqsktruJgcTUHHM8Pl9XQ8r90oI83/SICSHbsABIjAxgYHUR6cgTeXrIDcEedCXr5zidEN+kb5k/fMH+nrzfEz8rI+FBGxoee9l5do4384zWO8K8i17ED2F1YwVc7C6l39CSKC/Xj6nEJXJ3ej34RbjARi+gSOaIXopew2zVHy2vZmFfKe9l5fLOrCK1h0sAors3ox6UjYvG1OHlSdOF00nQjhGi3Q6U1LM7O573sPA6V1hAWYOVnY+K5NqMfQ/u4+YxcHkyCXgjRYXa75vu9x3gnK48vthVQb7Mzul8YczP6MXNUHMF+1m7Zrs2u2X+siu1HytlXVMnwuBAmDYoiwEdalM9Ggl4I0SUlVfV88NMh3s06yK6CSvyt3swcFcfczH6MTQzvdA+emnobOUfL2X6knO2Hy9l2uJyco+XUNthPWs7H4sWE/pFMGRLNlKGxJEbK+YNTSdALIZxCa83GvFLezcpj2abDVNXbGBAdyNyMRH42Np6oIN8zfvZYZR3bD7cM9TL2H6uiaej/ED8Lw/uGMKJvKMPjQhjeN4SkyAA2Hizlq5xCvtpZyL6iKgAGxgQxZWgMFw6JIT05HKvM/StBL4Rwvqq6Rj7ZfIR3sg6y4WApFi/FJcNjuTajH0mRgY5QL2sO94LyuubPxof5M7xvSHOgj+gbQnxY20NG5B6rMkI/p5C1+4tpsGmC/SycPziaKUNimDwkmsiz7Gw8mQS9EKJb7S6o4N2sPN7/6dBJF255eykGxQQ1B3pTuIcFdH3kz8q6Rr7bfYyvcgr4emcRRRV1KAVp/cKYMiSGKcNiGB4X0msuDJOgF0L0iPpGO1/lFFBW08DwuFAGxQbhZ+3+rpl2u2bb4XJW5hTwdU4hm/LLAGO8oQuHGu36E/pHdNsJZFcgQS+E6FUKK2pZtbOIr3MKWb2rqHmE0WA/C7EhfsSG+BIb7EdMiB8xwb4nXgvxIzrY1+k7J601dY12ymsaKK9toKymkfLaBmw2zaiEUGKcMMS1BL0Qoteqb7STlVvCpvxSCsvrKKyopaC8joLyWgrL66i32U/7TFiA1bEjOHknEON4TWtjyAkjuBubA7zcEeBNr1e0eL217TRJiQpsHuI6MyWChPCOD3HdLUGvlFoAzAQKtdYjW3l/NvB7wA40Ag9orb9ra8MS9EKInqK1prS6gYKTwv/E44KKOgrLaymsqMNmP3sm+lq8CPG3EuJncdxbCW7xOMTf4rg/sYzWmg0HSpuHuC6rMYa47hvq1zypTWZKBAOiA9sM/u4K+vOBSuD1MwR9EFCltdZKqVHAe1rroW1tWIJeCOFq7HZNcVW9sSOoqMXby+u0QO9qc4/drtlVWHFipNMWQ1xHBvo0H+1npkQwtE/IaYPPdcugZlrr1Uqp5LO8X9niaSBgTluQEEJ0kZeXIjrYl+hgX+D0QeOctY2hfUIY2ieEm89JRmtNbnF189wG6/aXsGLrUcA415DRoqlnZN/O1eSUa42VUj8D/gjEADPOstztwO0AiYmJzti0EEK4NaUUKVGBpEQFcm2GkYuHSmvIaj7iL+arnEIA/Dv5baJdJ2MdR/Qft9Z0c8py5wOPaa0vbmud0nQjhBDtc6yyrjn4n5g9ssNNN069nlhrvRoYoJSKcuZ6hRCiN4sK8mVaahy/mzWiU5/vctArpQYqx2lipdRYwAco7up6hRBCOEebbfRKqbeByUCUUiofeBywAmit5wNXATcrpRqAGuBabVbnfCGEEKdpT6+b69p4/0/An5xWkRBCCKeSMT+FEMLDSdALIYSHk6AXQggPJ0EvhBAeToJeCCE8nGnDFCulKoCdpmz8zKKAY2YX0QpXrEtqah+pqf1csS5XrGmI1jq4Ix9wylg3nbSzo5fxdjelVLar1QSuWZfU1D5SU/u5Yl2uWlNHPyNNN0II4eEk6IUQwsOZGfSvmbjtM3HFmsA165Ka2kdqaj9XrMsjajLtZKwQQoieIU03Qgjh4STohRDCw5kS9EqpqUqpnUqpPUqpR82o4ZR6+imlvlZK7VBKbVNK3W92TU2UUt5KqZ+UUh+bXQuAUipMKbVYKZXj+Pc6xwVqetDxe9uqlHpbKeVnUh0LlFKFSqmtLV6LUEp9oZTa7bgPd4GannH8/jYrpT5QSoWZXVOL9x5SSmkzJi86U11KqXsdebVNKfVns2tSSqUppdYopTYqpbKVUpltrafHg14p5Q28AkwDhgPXKaWG93Qdp2gE/ltrPQyYANztAjU1uR/YYXYRLbwEfKq1HgqMxuTalFLxwH1AumOqS29grknlLAKmnvLao8BKrfUgYKXjudk1fQGM1FqPAnYBv3KBmlBK9QMuAQ72cD1NFnFKXUqpC4HZwCit9QjgWbNrAv4MPKG1TgMeczw/KzOO6DOBPVrrfVrreuAdjH9I02itj2itNzgeV2CEV7yZNQEopRIwJlv/p9m1ACilQoDzgX8BaK3rtdal5lYFGBf++SulLEAAcNiMIhxTaZac8vJs4H8dj/8XuMLsmrTWn2utGx1P1wAJZtfk8ALwS8CUHiJnqOtO4GmtdZ1jmUIXqEkDIY7HobTj792MoI8H8lo8z8cFQrWJYyL0McBacysB4EWMP3y72YU49AeKgIWO5qR/KqUCzSxIa30I4yjrIHAEKNNaf25mTaeI1VofAeOAAogxuZ5T/RxYYXYRSqlZwCGt9SazaznFYOA8pdRapdQ3SqkMswsCHgCeUUrlYfztt/mNzIygV6285hJ9PJVSQcAS4AGtdbnJtcwECrXW682s4xQWYCzwqtZ6DFBFzzdFnMTR5j0bSAH6AoFKqRvNrMldKKV+jdFs+ZbJdQQAv8ZohnA1FiAco0n3YeC9pjmyTXQn8KDWuh/wII5v2GdjRtDnA/1aPE/ApK/aLSmlrBgh/5bW+n2z6wEmArOUUrkYzVtTlFJvmlsS+UC+1rrp285ijOA308XAfq11kda6AXgfONfkmloqUErFATjue/Sr/5kopW4BZgI3uMAczwMwdtSbHH/vCcAGpVQfU6sy5APva8M6jG/XPX6i+BS3YPydA/wbozn8rMwI+ixgkFIqRSnlg3Hi7CMT6mjm2EP/C9ihtX7ezFqaaK1/pbVO0FonY/wbfaW1NvVIVWt9FMhTSg1xvHQRsN3EksBospmglApw/B4vwrVOXn+E8R8Tx/2HJtYCGL3egEeAWVrrarPr0Vpv0VrHaK2THX/v+cBYx9+b2ZYCUwCUUoMBH8wfzfIwcIHj8RRgd5uf0Fr3+A2YjnG2fy/wazNqOKWeSRjNR5uBjY7bdLPralHfZOBjs+tw1JIGZDv+rZYC4S5Q0xNADrAVeAPwNamOtzHOEzRghNVtQCRGb5vdjvsIF6hpD8Z5sqa/9flm13TK+7lAlIv8/nyANx1/WxuAKS5Q0yRgPbAJ41ziuLbWI0MgCCGEh5MrY4UQwsNJ0AshhIeToBdCCA8nQS+EEB5Ogl4IITycBL0QQng4CXohhPBw/x8FDFr+4CqniQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_ = log.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prediction\n",
    "\n",
    "For evaluation we first need to obtain survival estimates for the test set.\n",
    "This can be done with `model.predict_surv` which returns an array of survival estimates, or with `model.predict_surv_df` which returns the survival estimates as a dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "surv = model.predict_surv_df(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can plot the survival estimates for the first 5 individuals.\n",
    "Note that the time scale is correct because we have set `model.duration_index` to be the grid points.\n",
    "We have, however, only defined the survival estimates at the 10 times in our discretization grid, so, the survival estimates is a step function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAdmElEQVR4nO3de3hUdZ7n8feXcAlNYiM3L4RrKwpRRIhRt9VBW7cxurDdiuJ4QdTBmdW+PK77tI476jjreOlhu+ltd1pWG1t7WtrLOrIaUXtoZ/rx8YKo3SAYZRClFOQmdwIk+e4fdQiVVEIqoU5+p5LP63nqSZ1Tv6rzyUmlvvU7t5+5OyIiIpl6hA4gIiLJo+IgIiJZVBxERCSLioOIiGRRcRARkSw9QwfoiEGDBvnIkSNDxxARKRhLly7d5O6Dc21fkMVh5MiRvPPOO6FjiIgUDDP7tD3ttVlJRESyqDiIiEgWFQcREclSkPscRERC2L9/P6lUitra2tBRWlVcXExZWRm9evU6rNeJtTiY2S+Bi4EN7n5SC48bMBeoAnYD17r7u3FmEhHpqFQqRWlpKSNHjiT98ZUs7s7mzZtJpVKMGjXqsF4r7s1KjwFTDvH4hcDx0W028I8x5xER6bDa2loGDhyYyMIAYGYMHDgwLz2bWIuDu/8bsOUQTaYBj3vam0B/MzsmzkwiIocjqYXhgHzlC73PYSiwNmM6Fc1b17yhmc0m3btg1IDhPDlzXqcEzNVxw3Zy2v+4JXQMEZG8CH20UkslrsUBJtx9nrtXuHtFg8FuPDG3bb2OZcVn/WJeVSIiaYsWLeKEE07guOOO4/77749lGaF7DilgWMZ0GfBFW0/aWLKdeZNfiS1Ue93w2gXUJbyrKSJdQ319PTfddBOvvvoqZWVlnHbaaUydOpVx48bldTmhi8NC4GYzWwCcDmxz96xNSs2NHfQN3pr1bOzhcvXoaw+HjiAi3cTbb7/Ncccdx+jRowGYMWMGzz//fGEVBzN7EpgMDDKzFHAX0AvA3X8BVJM+jHUV6UNZZ8WZR0QkX/72/33Aii+25/U1xx17BHf9p/JDtvn8888ZNuzgBpeysjLeeuutvOaAmIuDu1/RxuMO3BRnBhGRriT9sdlUHEdQhd6sJCJSkNr6hh+XsrIy1q49eJBnKpXi2GOPzftyQh+tJCIi7XDaaafx8ccf88knn7Bv3z4WLFjA1KlT874c9RxERApIz549+fnPf863v/1t6uvrue666ygvz38vRsVBRKTAVFVVUVVVFesytFlJRESyqOeQJw04sxYl60jcqtFVTB8zPXQMESlA6jnkQZFDjxavBBJOzZYaqldXh44hIgVKPYc86OXQ02H3p7NDR2lU3/sf2LB9b+gYIlKgVBzywMxgv3Pt0w+EjtJol3/G2yf1h8tCJxGRQqTikAe9+hVRt7ueccccETpKo01/2kvRim2hY4hIgVJxyIOeJT3pWdKTET9+PHSURu9eMCl0BBGJwXXXXccLL7zAkCFDWL58eWzL0Q5pEZECcu2117Jo0aLYl6PiICJSQM455xwGDBgQ+3K0WUlEpCNeug3WL8vvax59MlwYz8hu7aWeg4iIZFHPQUSkIxLyDT8u6jmIiEgWFQcRkQJyxRVXcOaZZ1JTU0NZWRmPPvpoLMvRZiURkQLy5JNPdspy1HMQEZEsKg4iIpJFxUFERLKoOIiISBYVBxERyaLiICIiWVQcREQKyNq1azn33HMZO3Ys5eXlzJ07N5bl6DwHEZEC0rNnT+bMmcPEiRPZsWMHkyZN4oILLmDcuHF5XY56DiIiBeSYY45h4sSJAJSWljJ27Fg+//zzvC9HPQcRkQ544O0H+HDLh3l9zRMHnMiPKn+Uc/s1a9bw3nvvcfrpp+c1B6jnICJSkHbu3Mkll1zCT3/6U444Iv/j16vnICLSAe35hp9v+/fv55JLLuHKK6/ku9/9bizLiLU4mNkUYC5QBDzi7vc3e3w48Cugf9TmNnevjjNTd9LAXmYtmhU6RhNVo6uYPmZ66BgiBcvduf766xk7diy33HJLbMuJbbOSmRUBDwEXAuOAK8ys+e70/w485e6nAjOA/x1Xnu6myEvpQZ/QMZqo2VJD9WrVfpHD8frrr/PEE0+wePFiJkyYwIQJE6iuzv//VZw9h0pglbuvBjCzBcA0YEVGGwcObCz7OvBFjHlitWn3YJ6b827oGI22DL2Okh01zJ9yb+gojZLWixEpRGeddRbuHvty4iwOQ4G1GdMpoPku9buBV8zse0A/4PzWXszMZgOzAYYPH57XoIdrzKCVsAng6NBRGu3rPZidpaFTiEihirM4WAvzmpe7K4DH3H2OmZ0JPGFmJ7l7Q9YT3ecB8wAqKiriL5vtUD5kGeVDlsGsmaGjNHp05uOhI4hIAYvzUNYUMCxjuozszUbXA08BuPsbQDEwKMZMIiKSgziLwxLgeDMbZWa9Se9wXtiszWfAtwDMbCzp4rAxxkwiIpKD2IqDu9cBNwMvAytJH5X0gZndY2ZTo2b/FfgLM/sj8CRwrXfGnhYRETmkWM9ziM5ZqG42786M+yuAb8aZQURE2k+XzxARKSC1tbVUVlZyyimnUF5ezl133RXLcnT5DBGRAtKnTx8WL15MSUkJ+/fv56yzzuLCCy/kjDPOyOtyVBzyZf0ymH9R6BSNihumUmf684p0NWZGSUkJkL7G0v79+zFr6cyBw6NPj3w4+dLQCbL0oIGeXhc6hkiXtf7v/569K/N7ye4+Y0/k6L/+6zbb1dfXM2nSJFatWsVNN90UyyW7VRzyoWJW+pYgDYsfDh1BRGJSVFTE+++/z9atW/nOd77D8uXLOemkk/K6DBWHrqwOPr36mtApGs3Y8iErJw2CKaGTiBy+XL7hx61///5MnjyZRYsW5b046GilLsr79khc6R/y+W7GLt0UOoZIQdu4cSNbt24FYM+ePfzud7/jxBNPzPtyEvbxIXnTtwfetwcj5ibnGksfXlQZOoJIwVu3bh0zZ86kvr6ehoYGLrvsMi6++OK8L0fFoQtraHAuf/iN0DEazdhXT68idVZFDsf48eN57733Yl+OikMX1auoB/vJurhtUPUNDgnLJCItU3HoonoX9aB3UQ9+e+OZoaM0eu6Z/B+LLSLxUB9fRESyqDiIiEgWFQcREcmi4iAiIllUHERECkx9fT2nnnpqLOc3HKDiICJSYObOncvYsWNjXYaKg4hIAUmlUrz44ovccMMNsS5H5zmIiHTAH576iE1rd+b1NQcNK+Hsy8Ycss0Pf/hDHnzwQXbs2JHXZTennoOISIF44YUXGDJkCJMmTYp9Weo5iIh0QFvf8OPw+uuvs3DhQqqrq6mtrWX79u1cddVV/PrXv877stRzEBEpEPfddx+pVIo1a9awYMECzjvvvFgKA6jnIJ2sgb3MWpSsUfOqRlcxfcz00DFEEkXFQTpNkZdCwq69V7OlBkDFQQrO5MmTmTx5cmyvr+IgnaYX/enl/Zk/ZX7oKI2S1osRSQrtcxARkSwqDiIi7eDuoSMcUr7yqTiIiOSouLiYzZs3J7ZAuDubN2+muLj4sF9L+xxERHJUVlZGKpVi48aNoaO0qri4mLKyssN+HRUHEZEc9erVi1GjRoWO0Sli3axkZlPMrMbMVpnZba20uczMVpjZB2b2mzjziIhIbmLrOZhZEfAQcAGQApaY2UJ3X5HR5njgduCb7v6VmQ2JK4+IiOQuzp5DJbDK3Ve7+z5gATCtWZu/AB5y968A3H1DjHlERCRHcRaHocDajOlUNC/TGGCMmb1uZm+a2ZQY84iISI7i3CHd0oUSmh//1RM4HpgMlAF/MLOT3H1r1ouZzQZmAwwfPjy/SUVEpIk4ew4pYFjGdBnwRQttnnf3/e7+CVBDulhkcfd57l7h7hWDBw+OJbCIiKTFWRyWAMeb2Sgz6w3MABY2a/PPwLkAZjaI9Gam1TFmEhGRHMRWHNy9DrgZeBlYCTzl7h+Y2T1mNjVq9jKw2cxWAL8H/pu7b44rk4iI5CbWk+DcvRqobjbvzoz7DtwS3STf9u2C+ReFTtFoxP7P2Najf+gYIpIDnSHdVfVL3n6ZYq+FhqxjDUQkgXIqDmZWAZwNHAvsAZYDv3P3LTFmk8NRenT6Nmtm6CSNah87OXQEEcnRIYuDmV0LfB/4BFhK+miiYuAs4Edmthz4G3f/LOac0gGbUjt5bs67oWM02j38Zo7avpRxoYM0U7OlJpGD/mj4UgmprZ5DP9KXttjT0oNmNoH0oacqDgkzpvKo0BGy7Cxufg5keFWjq0JHaJGGL5XQDlkc3P2h1h4zs97u/n7+I0k+lJ89lPKzk/Vh/E9XLwkdIcv0MdMT+QGcxJ6MdC85HcpqZq+Z2ciM6UrS5zGIiEgXlOvRSvcBi8zsZ6Svj3QhoK82IiJdVE7Fwd1fNrO/BF4FNgGnuvv6WJOJiEgwuW5W+hvgfwHnAHcDr5lZcs6uEhGRvMp1s9IgoDI6aukNM1sEPAK8GFsy6Zrq4NOrrwmdookjLr6YIy+/LHQMkUTJdbPSD5pNf0p6hDeRnHnfHhTtqYP1y0JHaVS7YR/sWK/iINKMLp8hnWZ7vwEc0XcrI847NnSURp8+9jHs2hg6hkjiqDhIp/mqaCBfFQ2EWQnaGvmbU0MnEEmkOMdzEBGRAtXWtZXOyfF11uj6SiIiXUdbm5VyPdHtOXR9JRGRLqOtayvpLGgRkW5I+xxERCSLioOIiGTRoazSqXbvrePyh98IHaPRXfvq6FWk70gizeV6baUncpknciiDSnrztT7J+j7S0ODsr28IHUMkcXL9Ty3PnDCzImBS/uNIVzaktJghpcXccePE0FEarfiNhY4gkkiH7DmY2e1mtgMYb2bbo9sOYAPwfKckFBGRTtfWoaz3AfeZ2X3ufnsnZRLpVMVeC/MTdgV6+5KaouQNF1o1uiqRw6pK/rXVcxgJ0FphsLSy/McS6RzbevSn1opDx8hSte0rTqgPnaKpmi01VK+uDh1DOklb+xx+bGY9SG9CWgpsBIqB44BzgW8BdwGpOEOKxGVr0UC2Fg1k3Kx/Dh2lienzL2K6A1Pmh47SKGm9GIlXW5uVppvZOOBK4DrgGGAPsJL0QD/3untt7ClFRKRTtXm0kruvAO7ohCwiIpIQbe1zOM3Mjs6YvsbMnjezn5nZgPjjiYhICG2dBPcwsA8aL999P/A4sA2YF280EREJpa3NSkXuviW6fzkwz92fBZ41s/fjjSYiIqG01XMoMrMDBeRbwOKMx9rcX2FmU8ysxsxWmdlth2h3qZm5mVW0HVlEROLW1gf8k8C/mtkm0kcp/QHAzI4jvWmpVdElNh4CLiB9qOsSM1sY7eDObFcKfB94q0O/gUhXtX5Zsk7Osy+h3+DQKaSTtHUo671m9i+kD2F9xd09eqgH8L02XrsSWOXuqwHMbAEwDVjRrN3fAQ8Ct7Yzu0heHL3xMz69+prQMZrasZ8jRozgyKPbbtpp9u0KnUA6US6Hsr7ZwryPcnjtocDajOkUcHpmAzM7FRjm7i+Y2SGLg5nNBmYDDB8+PIfFi7Rt2Qnpt2TSDr2r/XwrlJ7IkbMeDx3loMe01bc7ifP6yS1d7tIbH0yfef0T4NpcXszd5xEdIVVRUeFtNBfJydLxk1k6fjK/vfHM0FGaSFxPRrqdOEc5SQHDMqbLgC8ypkuBk4DXzGwNcAawUDulRUTCi7M4LAGON7NRZtYbmAEsPPCgu29z90HuPtLdRwJvAlPd/Z0YM4mISA5iKw7uXgfcDLxM+lpMT7n7B2Z2j5lNjWu5IiJy+GIds9Hdq4HqZvPubKXt5DiziIhI7pI1oK90eZtSO3luzruhYzQav24vG44sCh1DJHFUHKTTjKk8KnSELCV7GkJHEEkkFQfpNOVnD6X87KGhYzRx762vhY4gkkhxHq0kIiIFSsVBRESyaLOSiORu365kXQwQ4ORLoULjW+ebeg4ikpt+g6F3v9Apmlq/DJY9EzpFl6Seg4jkpvTo9G3K/NBJDkpaL6YLUXGQbm/33jouf/iN0DGauHbddgaW9GFE6CDSbak4SLc2qKQ3m0KHaMGuvXWhI0g3p+Ig3dqQ0mKGlBZzx40TQ0dp4qWn9a8pYWmHtIiIZFFxEBGRLOq7iiRU0sa2nrHlQ1ZOGgRTQieRzqDiIJJASRzbesjnuyGRu+8lDioOIgmUxLGtP7yoMnQE6UTa5yAiIllUHEREJIs2K4lIznbX7WHWogRd5M6+pMr7MT10ji5IPQcRycnAvgP5Ws++oWM0UcM+qm1X6BhdknoOIpKTwX0HM7jvYOYn6MJ7sx6rCB2hy1JxkG5vU2onz815N3SMJsav28uGI4tCx5BuTMVBurUxlUeFjtCikj0NoSNIN6fiIN1a+dlDKT97aOgYWe699bXQEaSb0w5pERHJouIgIiJZVBxERCSLioOIiGRRcRARkSwqDiIikkXFQUREssR6noOZTQHmAkXAI+5+f7PHbwFuAOqAjcB17v5pnJlEpIvZtwvmXxQ6RVMnXwoVCbpAYQfEVhzMrAh4CLgASAFLzGyhu6/IaPYeUOHuu83sr4AHgcvjyiQih6f2ww+TNXTpxjpWntgbTgmdJMP6ZemfKg6tqgRWuftqADNbAEwDGouDu/8+o/2bwFUx5hEpKLv31nH5w2+EjtFo0hFjmTx0L8Whg2QYsr4Oir4GP30xdJSDktaL6aA4i8NQYG3GdAo4/RDtrwdeau1BM5sNzAYYPnx4PvKJJNagkt6JG635icETNXRpNxJncbAW5nmLDc2uAiqAP2vtxdx9HjAPoKKiosXXEekqhpQWM6S0mDtunBg6SqMk9WIkfnEWhxQwLGO6DPiieSMzOx+4A/gzd98bYx4REclRnIeyLgGON7NRZtYbmAEszGxgZqcCDwNT3X1DjFlERKQdYisO7l4H3Ay8DKwEnnL3D8zsHjObGjX7MVACPG1m75vZwlZeTkREOlGs5zm4ezVQ3WzenRn3z49z+SIi0jEa7EdECtruuj3MWpSgcwrsS6q8H9ND5zhMunyGiBSsgX0H8rWefUPHaKKGfVTbrtAxDpt6DiJSsAb3HczgvoOZP2V+6CiNZj1WETpCXqjnICIiWVQcREQkizYriSTUptROnpvzbugYjcav28uGI4tCx5BOouIgkkBjKo8KHSFLyZ6G0BGkE6k4iCRQ+dlDKT97aOgYTdx762uhI0gnUnEQkZwl7TLi167bzsCSPowIHaQLUnEQkZwk8TLiu/bWhY7QZak4iEhOkngZ8Zee1kdYXHQoq4iIZFHZFZGCdvTGz5I1rvX6PawsL/yP1sL/DUSk21p2Qnrk4QGBc2QasqEBKPx9ISoOIlKwlo6fnLxxrS8oDx0hL1QcRETyzRtg/kWhUxwW7ZAWEcmnol5gCftoXb+s3U9Rz0FEJJ+Keqdvs14MneSg+RcBa9v1lISVNxERSQL1HESkoK1Ytz1Rl/SYsa8et72JG7q0vVQcRKRgTZuQrIsTAtTX9aMoYZ+sNexr93MS9iuIiOTuz08fzp+fPjx0jCZeenoA+IDEDV3a3r6V9jmIiEgW9RxEJGdJG50O0gMjJW3siyRe0uOxdj5HxUFEcpLE0ek2pXYCJKo4JPeSHu2j4iAiOUni6HRJ68VA17mkh/Y5iIhIFvUcRETyLGnnXsx0b/dzVBxERPIoiede0P7aoOIgIpJPSTz3ovrp9j9H+xxERLo478BHfaw9BzObAswFioBH3P3+Zo/3AR4HJgGbgcvdfU2cmUSka9G5F2375KihwAftek5sPQczKwIeAi4ExgFXmNm4Zs2uB75y9+OAnwAPxJVHRLqeMZVHMaisJHSMJjaldvLR2+2/0F2c/nDO3e1+Tpw9h0pglbuvBjCzBcA0YEVGm2nA3dH9Z4Cfm5m5d2DXuoh0O0k99yJpvZkzN9fxVDufE+c+h6E0HV0iFc1rsY271wHbgIEtvZiZzTazd8zsnY0bN8YQV0Tk8CWxNzNiYL92PyfOnoO1MK95jyCXNumZ7vOAeQAVFRXqWYhIIiWxNwPAre1rHmfPIQUMy5guA75orY2Z9QS+DmyJMZOIiOQgzuKwBDjezEaZWW9gBrCwWZuFwMzo/qXAYu1vEBEJL7bNSu5eZ2Y3Ay+TPpT1l+7+gZndA7zj7guBR4EnzGwV6R7DjLjyiIhI7mI9z8Hdq4HqZvPuzLhfC0yPM4OIiLSfzpAWEZEsKg4iIpJFxUFERLKoOIiISBYrxCNHzWwHUBM6RwcNAjaFDtFByh6GsofR1bKPcPfBub5AoY7nUOPuFaFDdISZvaPsnU/Zw1D2MPKRXZuVREQki4qDiIhkKdTiMC90gMOg7GEoexjKHsZhZy/IHdIiIhKvQu05iIhIjFQcREQkS0EVBzObYmY1ZrbKzG4LnactZrbGzJaZ2ftm9k40b4CZvWpmH0c/jwyd8wAz+6WZbTCz5RnzWsxraT+L/hZ/MrOJ4ZK3mv1uM/s8Wv/vm1lVxmO3R9lrzOzbYVKDmQ0zs9+b2Uoz+8DMfhDNT/x6P0T2xK/3KEuxmb1tZn+M8v9tNH+Umb0VrfvfRkMOYGZ9oulV0eMjE5j9MTP7JGPdT4jmt/994+4FcSN92e9/B0YDvYE/AuNC52oj8xpgULN5DwK3RfdvAx4InTMj2znARGB5W3mBKuAl0qP5nQG8lcDsdwO3ttB2XPT+6QOMit5XRYFyHwNMjO6XAh9F+RK/3g+RPfHrPcpjQEl0vxfwVrROnwJmRPN/AfxVdP+/AL+I7s8AfpvA7I8Bl7bQvt3vm0LqOVQCq9x9tbvvAxYA0wJn6ohpwK+i+78C/nPALE24+7+RPRJfa3mnAY972ptAfzM7pnOSZmsle2umAQvcfa+7fwKsIv3+6nTuvs7d343u7wBWkh5bPfHr/RDZW5OY9Q4QrcOd0WSv6ObAecAz0fzm6/7A3+QZ4Ftm1tJQx7E7RPbWtPt9U0jFYSiwNmM6xaHfiEngwCtmttTMZkfzjnL3dZD+5wKGBEuXm9byFsrf4+aoG/3LjE14icwebaY4lfS3wIJa782yQ4GsdzMrMrP3gQ3Aq6R7M1vdvS5qkpmxMX/0+DZgYOcmPqh5dnc/sO7vjdb9T8ysTzSv3eu+kIpDSxU66cfhftPdJwIXAjeZ2TmhA+VRIfw9/hH4BjABWAfMieYnLruZlQDPAj909+2HatrCvKRlL5j17u717j6B9Bj3lcDYlppFPxOVv3l2MzsJuB04ETgNGAD8KGre7uyFVBxSwLCM6TLgi0BZcuLuX0Q/NwDPkX7zfXmgOxf93BAuYU5ay5v4v4e7fxn9AzUA/4eDmzASld3MepH+cP0nd/+/0eyCWO8tZS+U9Z7J3bcCr5HeHt/fzA5cdy4zY2P+6PGvk/umzNhkZJ8Sbepzd98LzOcw1n0hFYclwPHRkQS9Se8QWhg4U6vMrJ+ZlR64D/xHYDnpzDOjZjOB58MkzFlreRcC10RHQZwBbDuwGSQpmm1T/Q7p9Q/p7DOio09GAccDb3d2PkgfRUJ6LPWV7v4/Mx5K/HpvLXshrHcAMxtsZv2j+32B80nvN/k9cGnUrPm6P/A3uRRY7NHe3s7WSvYPM75QGOl9JZnrvn3vm1B72ztyI73H/SPS2wXvCJ2njayjSR+Z8UfggwN5SW+j/Bfg4+jngNBZMzI/SXozwH7S3zSuby0v6W7qQ9HfYhlQkcDsT0TZ/hT9cxyT0f6OKHsNcGHA3GeR7t7/CXg/ulUVwno/RPbEr/coy3jgvSjncuDOaP5o0kVrFfA00CeaXxxNr4oeH53A7Iujdb8c+DUHj2hq9/tGl88QEZEshbRZSUREOomKg4iIZFFxEBGRLCoOIiKSRcVBRESy9Gy7iUj3ZWYHDikFOBqoBzZG07vd/T8ECSYSMx3KKpIjM7sb2Onu/xA6i0jctFlJpIPMbGf0c7KZ/auZPWVmH5nZ/WZ2ZXS9/WVm9o2o3WAze9bMlkS3b4b9DURap+Igkh+nAD8ATgauBsa4eyXwCPC9qM1c4CfufhpwSfSYSCJpn4NIfizx6Fo1ZvbvwCvR/GXAudH984FxGUMAHGFmpZ4eC0EkUVQcRPJjb8b9hozpBg7+n/UAznT3PZ0ZTKQjtFlJpPO8Atx8YOLA+L4iSaTiINJ5vg9URKN0rQD+MnQgkdboUFYREcminoOIiGRRcRARkSwqDiIikkXFQUREsqg4iIhIFhUHERHJouIgIiJZ/j8XEs8wCi3OMgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "surv.iloc[:, :5].plot(drawstyle='steps-post')\n",
    "plt.ylabel('S(t | x)')\n",
    "_ = plt.xlabel('Time')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is, therefore, often beneficial to interpolate the survival estimates, see [this paper](https://arxiv.org/abs/1910.06724) for a discussion.\n",
    "Linear interpolation (constant density interpolation) can be performed with the `interpolate` method. We also need to choose how many points we want to replace each grid point with. Her we will use 10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "surv = model.interpolate(10).predict_surv_df(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZwV9Znv8c/DJoZFWQVpNqMgtKjBFuK4hNxgROId4wiIxqjoHcYZTcxkGc14x2hmEpeXJjE33kmIV4xm3GccGUQ0xjAx3oTFiEFAkBGQBppVZZPV3/xx6nRXV58+S5+qU1Wnv+/Xq199qk71qR+nm376+T2/xZxziIiI+HWIuwEiIpI8Cg4iItKCgoOIiLSg4CAiIi0oOIiISAud4m5AW/Tt29cNGzYs7maIiKTG66+/vt0516/Y61MZHIYNG8aSJUviboaISGqY2fpSrle3koiItKDgICIiLSg4iIhIC6msOYiIxOHQoUPU19ezf//+uJvSqq5du1JTU0Pnzp3Lep1Ig4OZPQRcBGx1zp2S43kD7gcmA/uAa5xzf4yyTSIibVVfX0+PHj0YNmwYmV9fyeKcY8eOHdTX1zN8+PCyXivqbqWHgUl5nr8QOMn7mAn8c8TtERFps/3799OnT59EBgYAM6NPnz6hZDaRBgfn3G+BnXkuuRh4xGX8ATjWzAZG2SYRkXIkNTBkhdW+uGsOg4ANvuN679zm4IVmNpNMdsHw3kN4/OpZAJw4eA9n/tPXo2+piEg7EvdopVwhLucGE865Wc65Oudc3ccG+3B82Pl4VrzXLeImiogky/z58xk5ciQnnngid911VyT3iDs41AODfcc1wKZCX7St+y5mTXiJDofqsUOOZ88/o/Hjwe/cGFljRUTiduTIEW644QZeeOEFVqxYweOPP86KFStCv0/cwWEOcJVlfBr40DnXokspaFTfT7Jwxr/S4eiOHOrSlHycvGEfZz/5a178wrjGj5d/fHOEzRcRqaxFixZx4okncsIJJ9ClSxemT5/Oc889F/p9oh7K+jgwAehrZvXAd4DOAM65nwLzyAxjXUNmKOuMUl6/T48ubO/YDyb9HID5a1bRp+G39NizCMgEC/7vHF588TcAdLzgs0z86t0h/MtEpL274z+Ws2LTrlBfc/TxPfnO/6zNe83GjRsZPLipw6WmpoaFCxeG2g6IODg45y4v8LwDbmjr64/ouxI27YWGbQB0s37sGTaR50++CoB+f3iAz7y3gk4HjzB46z7WzZ3DjBFbG79+8gmTmTpialtvLyJScZlfm81FMYIq7tFKZan93Ehqlz3TePzs0i9Ap248+VdnAfDY6YN4ZulGAC597GZOaNjBn9/3JwCO2D5+N3oR8ybNa/x6BQsRKVahv/CjUlNTw4YNTYM86+vrOf7440O/T6qDA3UzMh9Z3/pFs6evGD+EK8YPAeDF969g28vz6eoF3WEbVlH73se8vVzBQkTS48wzz+Sdd95h7dq1DBo0iCeeeILHHnss9PukOzjksH1fP569r2kFjhHjjqP23EFccMv1cMv1jedfvOunfFxCsFCgEJEk6NSpEz/5yU+44IILOHLkCNdeey21teFnMVUVHEb0XQnbAQYAsL1+DwC15w5qcW2hYNG34V3ckU7ce9ou9nVYzZItS5j3rrIKEYnf5MmTmTx5cqT3qKrgUNt/GbUfPwa9xwDwbMM02F3crnjBYLHoi5dx4tp3uP1fjrD7UC9+V2ssPDMzMkHBQkSqXVUFB8ZMaX58cG+bX+qkyy9l19y59Ab2LV7FmC1w4epeAOw90otFp3Zi24WZa5dsWdIsWChQiEjaVVdwyFGg9tcgsvWHYvS6bBq9LpsGwPtPPsWuuXMZ7T23b/EqajfDuuU9AbiEPiwd24W1x8GqnasAFBxEJNWqKzgE+GsQ+eoPhfgDBTTVJ7IGNnzIR68dz/JPzuRIl3vZuutAuU0XEYlVVQcHfw2ilPpDIbnqEyPWvsM1T9/NXvce/3/Mx8zo2ZTBqJtJRNIm7rWVojVmCgzIFKc5uBf2bovkNiddfim9Tz2F0QN7Mnz7Yc5d3rHxuVU7VzUrXIuIpIHlmoqddHV1dW7JkiUlfc2zXv2h7wkDGs+VUoMo1gsTvwjAhS//OwAz5s9g1c5VjOw9svEaZRIi6bRy5UpGjRoVaxuuvfZa5s6dS//+/XnrrbdyXpOrnWb2unOurtj7VHfm4DOi70r6fqIpc9hev4fVi7ZEcq9+W9bzwsQv8sLEL3LJvWuZtLR743PKJESkHNdccw3z588vfGGZqrrm4Ffbfxm1/ZfBjKsBms2iDlOHiZPY5itWH7/lQzr/cSi33TMbyGQSIiJtdd5557Fu3brI79NuggMADctg9he8x9OgWzgFar9gsTrbzeS3aueqZkFC3UwiKfTCLZnfKWEaMAYujGZnt1K1n+CQY4Jca+swRWnyCc2nvGtehIgkUfsJDoEJciPuvgW2d6OYdZjKla1BAHQHpk+clMkwaCpYZzMJZREiKZGQv/Cj0m4K0kG1/ZdxyeinuOQbY7nkG2PpW9O98Be1QYeJk9h23NDG435b1jebQDf5hMmNI5lUrBaRpGg/mUMRttfvCb2bqVANYuqIqY2ZgorVIlLI5ZdfzoIFC9i+fTs1NTXccccdXHfddaHfp30HB1+BesThMXDMeWQ6fqLtZspHxWoRyefxxx+vyH3ab3AIFKhrP36M2qHLYMbzQHRDXaF5DQIyXU8X3HK9itUikhjtNzgEV3DNDnGNWHAeRL8t6zPHt1zfrIsJWharQZmEiFRG+w0ORfDXIMIa5lrMPIgsZRIiEhcFh1aMGHdc4+O46g+5MgkRkUpQcPDzFahrgdrPTIG6GZHWH6B5DaKDbw5ELupmEpFKaLfzHFrwL+8NmUCx7JnIb+ufBxGcAxHknxMBmhchItFR5pBVoEAdxRwIaF6DyFd/AHUziQhs2LCBq666ioaGBjp06MDMmTO56aabQr+PgkMR/PUHiLYGMWDbe6z/8lWNxz0vuqjZFqVBWnpDpH3p1KkT9913H2PHjmX37t2cccYZnH/++YwePbrwF5dyn1Bfrdp4NQh//QGimwOxbOR49h44TMPmXUAmUGzZfYBxrQQH/2gmjWQSaR8GDhzIwIEDAejRowejRo1i48aNCg4V458kl12Wty7abpyaq77E00snNB5P/eX36LbnQKvXB5feULFapHLuXnQ3b+98O9TXPLn3ydw87uair1+3bh1vvPEG48ePD7UdoODQOn8NIscEuShqEFeMH8IV44c0Hr/wdPHfHs2JEGlf9uzZw6WXXsqPfvQjevbsGfrrKzi0QRJrECpWi1RWKX/hh+3QoUNceumlfOlLX+Iv/uIvIrlHpMHBzCYB9wMdgQedc3cFnh8C/AI41rvmFudc4sdm1p47qFkgSEoNIkjdTCLVxznHddddx6hRo/j6178e2X0iCw5m1hF4ADgfqAcWm9kc59wK32X/G3jKOffPZjYamAcMi6pNZfFvMQqZmkTCahB+6mYSqU6vvfYajz76KGPGjOH0008H4Pvf/z6TJ08u8JWliTJzGAescc69C2BmTwAXA/7g4IBsZ9kxwKYI29N2wS1GcxSoo1iHqZwahLqZRKrTOeecg3Mu8vtEGRwGARt8x/VAsKR+O/CSmX0F6AZMbO3FzGwmMBNgyJAhrV0WjQIT5Cq5DlOp8yD8NCdCRIoVZXCwHOeC4e5y4GHn3H1mdhbwqJmd4pz7uMUXOjcLmAVQV1cXfdgsgb8GEeU6TOXUIDQnQkRKEWVwqAcG+45raNltdB0wCcA593sz6wr0BbZG2K7UKqcGoe1IRaQUUQaHxcBJZjYc2AhMB64IXPMe8DngYTMbBXQFtkXYpvDEUKAupwYRpJFMIpJPZMHBOXfYzG4EXiQzTPUh59xyM/susMQ5Nwf4BvBzM/tbMl1O17hKVFrKVaBAHdUifbn4axDF1h80kklECol0noM3Z2Fe4NxtvscrgLOjbEMk8hSoKzlBbtnITH2/N7D/7cw0/mKCg0YyiUghmiEdskpNkAN4/dQJPNpvLKMH9uSazXfTZ/cBhrbxtdTNJJIO+/fv57zzzuPAgQMcPnyYKVOmcMcdd4R+HwWHFLv49KYgtPfA4TYPc1U3k0h6HHXUUbzyyit0796dQ4cOcc4553DhhRfy6U9/OtT7KDiExV+grkBxGpoXqO/509l0W7WQ3t5z6mYSqU5mRvfu3YHMGkuHDh3CLNfMgfIoOIShwPLelShQv37qBF4/dQJP/tVZAM0yiLZQN5NIfg3f/z4HVoa7ZPdRo05mwN//fcHrjhw5whlnnMGaNWu44YYbtGR3YuVZ3ruSBeoVm3dx2c9+D8A1m3fRp/tRbapBqJtJJNk6duzI0qVL+eCDD7jkkkt46623OOWUU0K9h4JDFHxdTJXaRc5ff4DyahDqZhIprJi/8KN27LHHMmHCBObPn6/gkHhFLNIXheAEuXJqELloXSaRZNi2bRudO3fm2GOP5aOPPuLll1/m5pvD31tCwSFsBRbpg/TVILQuk0hybN68mauvvpojR47w8ccfM23aNC666KLQ76PgUGFprEFoXSaR5Dj11FN54403Ir+PgkMlJLAGUcpS30EaySRS/RQcopbAGkQ59QeNZBJpHxQcolZEDaIS/DWIcuoPGskk0j4oOMTBP5u6YRrbDwwKfYtREZFyKDhUWqCbaUTnl8nsjto78i1GswXqcorTuagGIVJ9FBwqLdDNVDv7C9TyFMy4OtIVXMNapC9INQiR6qTg0E6EtUhfkGoQIpV35MgR6urqGDRoEHPnzo3kHgoOSZCtQTRMg279Ir9d2Iv0BWk2tUi07r//fkaNGsWuXbsiu4eCQ9z8NYiDe9m+r1/FthiNgmZTi0Srvr6e559/nltvvZUf/OAHkd1HwSFuvhrEiLtvge3dgAFAZWdP1+ysD6UGodnU0l68+tRqtm/YE+pr9h3cnXOnjch7zde+9jXuuecedu/eHeq9gxQcEqS2/zJq+y+DGVcDlZs9/eKA07gAGO0dl7tIn59GMomEZ+7cufTv358zzjiDBQsWRHovBYekCcyBiKIGEZw9fRnwMJNCr0FoJJNUs0J/4UfhtddeY86cOcybN4/9+/eza9currzySn75y1+Gfi8FhyQJLrVxcG887QiJRjKJhOvOO+/kzjvvBGDBggXce++9kQQGUHBIluBSG9/6RWwF6v1vvx1KDSJI3Uwi6aDgkGAj+q6ETXuhYRsA2/f1g90NkReoz+g5igmDDtDVey6sGoS6mUTCM2HCBCZMmBDZ6ys4JFjt50ZSu+yZxuNnl34B9nYL/T7BAvWj/cZGMg8iVzeTMgmRZFJwSLIc3UxRaFGg9jKIqCmTEEkuBQeJTaFMQlmEJJFzDjOLuxmtcs6F8joKDimzfU8fnvVlECNO+wS1V0b/C9RfoA6rOB2k2dWSdF27dmXHjh306dMnkQHCOceOHTvo2rVr4YsLUHBIkRGnfQLe3NF4vH1PH3hzB7VXhn+v1grUYU6QCwrOrlY9QpKmpqaG+vp6tm3bFndTWtW1a1dqamrKfh0FhxSpvXJqs0DwbEQ1iHwF6rAX6WuN6hGSRJ07d2b48OFxN6MiIg0OZjYJuB/oCDzonLsrxzXTgNsBB7zpnLsiyjZJYXEVqP00skkkXpEFBzPrCDwAnA/UA4vNbI5zboXvmpOAbwNnO+feN7P+UbWnah3c27TcxpgpzUc3RSSqCXL5KJMQqawoM4dxwBrn3LsAZvYEcDGwwnfNXwIPOOfeB3DObY2wPdWnW7/MDOoV0+DgXkZsXUVtXbS37HnRRc2Oo6xB+Glkk0hlRRkcBgEbfMf1wPjANSMAzOw1Ml1Ptzvn5kfYpqoy4nOnwaItwAC2v9sA27tRG9G9mgrUg7n4xn9q7HaqVA0iSCObRKIVZXDINc4rOAC3E3ASMAGoAV41s1Occx+0eDGzmcBMgCFDhgSfbpdqzx3UuJTGs9/6RfMuJgitm8lfoF6xObPzlL8mEQeNbBKJVpTBoR4Y7DuuATbluOYPzrlDwFozW0UmWCwOvphzbhYwC6Curi6cWR7VJLi0d8OyzOcQgoO/QJ2rOB1HDcJP9QiR8EUZHBYDJ5nZcGAjMB0IjkT6d+By4GEz60umm+ndCNtUvXoMyHx4GwU1yyAiFFcNwk8jm0TCF1lwcM4dNrMbgRfJ1BMecs4tN7PvAkucc3O85z5vZiuAI8C3nHM7Wn9VyWd7/Z6m5b0bpjGi78rIahBZvS6b1iwQxFWD8FMmIVK+SOc5OOfmAfMC527zPXbA170PKcOIccc1O96+rx9s2kttBDUI/+xpyNQk4q5B+CmTECmfZkhXCX9xGuDZ2+c1X957/e8yH9klwNsYKIKzp3MVqCuxDlMpgpnEki1LWLJlCfPendf4vAKFSHNFBQczqwPOBY4HPgLeAl52zu2MsG1SjmANYsnspsAQDBRQdLAoNHvaX4OIo/6QSzCTeHr1042BQV1OIrlZvuVdzewa4KvAWuB1YCvQlUzh+GwyQeIfnHPvRd5Sn7q6OrdkyZJK3jJ1nr3vj2yv30Pfmu6N5xq3GPUHCsgECoCh5zSdKzJYZINDdmMgv2z2MPTRR9rwL6iMbJfTyN4jG88pk5BqZGavO+eKniZbKHPoRmZpi49audnpZIaeVjQ4SGEtahD1e4BM91OLTYRyBYs2ZhZpo+K1SG55M4e8X2jWxTl3MOT2FEWZQ+myo5gu+cbYwhcXyix8geKyn/2eFZt3MXpgz8bLswXqNGQOQcokpFqFnTlkX3QBcI1zbp13PA74OXBaG9ooSZcvswhkFT8+sJ/njvkzfk3mL/BggTruCXKlUiYhklHsaKU7gflm9mMyayZdCFRfH0OV88+DaKw/FMMfLAJZxXF732HmgK7MnPGPQPMCdRImyJUq1zBYkfaoqODgnHvRzK4HfgVsBz7lnGuItGUSKn8Noln9oVTBrGL2FzJLdXjzKW7b8SGvHf1Z4KxETpBrC82RkPao2G6lfwCmAecBpwILzOwbzrnno2ychKfZIn3ZWdRhGDOl+X0OLqP24DKY/cem51NcyFY3k7RXRRWkzex+4JbsqCUzG0pmZ7fzI25fTipIl6ek4nSJZv3wH/jUhy/ziS4dM0ECGgvZ6x/bxP4djq61pzZen/QaRFCwYK0sQtIikoK0c+6mwPF6Mju8SUo1W4eJEmsQeXT/s7/k3qWZv7ZHb/43pnT5feP6Tj2Pa4D9RzeuGLt/60HY3ZCq4KB9JKS9aPNQ1jgpcyjP8lc3snrRlsbj7GS5sDOJFhPkAsXs9Q+tZP8Hnek64BON53qefx69/u7+UNsRFQ17lTSJJHOQ6tJiHaYwaxD5BIrZPbfdBL/6bePx/oZ98B//Qa9+azInEl6vUD1CqpmCg8Sm19/dT6+/azpe/8XPw95tmYMQNyuKilZ/lWqWNziY2XlFvs66Sq+vJOGKqgZR0vLePQawf+MHrH+lDzT0oWfNSnoR/pLjUVEmIdWkUOZQ7P/EZ9H6SqmVdx2mMhSzvLdfsxVddzjgGHplT6RgvadCmYSyCEmTvMHBOZec/3kSmahqEIWW9w7yT5pb/+WrMktvvNIn8+Tu8fQc+hG9hnoXh7Q/RZQ0sknSTDUHSaQWS29s/AB6nEyvGd4ifv6RTwmtT/gzCdUjJG0UHCSnqGoQxcq19EbLRfyuzlwTWMIDSFwmoXqEpI2Cg7QQVQ0C2r7/dN5F/AJLeCSxPqGRTZI2xa6t9Khz7suFzkl1iKoGUWqB2i/vIn4p3LxImYQkXbFrK/3ROTfWd9wRWOacGx1l41qjGdKVFdxyNKwupnxbjBaS7WbqevLJQIE1mkrYvCgumm0tUQt1hrSZfRv4e+BoM9uVPQ0cBGa1uZWSKqEt951DGN1MBfeJyJdZJKSYrUxCkqbYzOFO59y3K9CeoihziE+YK7o+tvA9nlu6sfE4u91oqZlEMIuAElZ7zRazB4xpOpfATEJZhJQr7MxhmHNuXWuBwcwMGOScqy+xnSIlz4NoTVk7ziW0mK05EhK3vJmDmT0NdACeA14HtgFdgROBzwKfA77jnPtV9E1toswhPsH6A4Rbg8hmD1B8F1NQtlg99NFHSm9EofoEVDxYZEc0zZ40u2L3lOoTaubgnJtqZqOBLwHXAgOBj4CVwPPA95xz+8tor6RMlMNc/aOZShnJlEvLORFFdjO1ZeSTXwK6pETCUHAoq3NuBXBrBdoiKRDlct/+bqa2djFBmd1MQYWChV+EXVKaEyGVVqjmcCawwTnX4B1fBVwKrAdud87tjL6J0l61dSRTcbOr27g9aTBY+BXKMtoYKDSSSeJQKHP4GTARGpfvvgv4CnA6maGsU1r/Umkvolhqo5wJc0GhZhL55Msyysgqcs2uFolaoYL0m86507zHDwDbnHO3e8dLnXOnV6SVASpIJ0dsW46WoaQJdGEJsdCtCXPSFmFvE9rRzDo55w6TGZk0s4SvxcwmAfcDHYEHnXN3tXLdFOBp4EznnH7rp0gltxxtazdTUEkT6MIS4hIf6maSSij0C/5x4D/NbDuZUUqvApjZicCH+b7QW2LjAeB8oB5YbGZzvAK3/7oewFeBhW36F0i7EGY3U3DfiFiUGix8gULdTFIJhYayfs/Mfk1mCOtLrqkPqgOZ2kM+44A1zrl3AczsCeBiYEXgun8E7gG+WWLbJaH8NYiw5kCENWEul9CK1eWIqF4h0lbFDGX9Q45zq4t47UHABt9xPTDef4GZfQoY7Jyba2Z5g4OZzcTr1hoypG1j3yV6Ua7DFOTvZgqjiwkq2M1UiD9YBLOKHOtBaTtSCVuU+zlYjnON1W8z6wD8ELimmBdzzs3CW+yvrq6u8IJQEgt/DSLK+kNYE+YiHfYalmBWEdjcaDJ7oPsxgOoPEp4og0M9MNh3XANs8h33AE4BFmSWaGIAMMfM/lxF6eoR1Y5yYU2YC0psJuEXWA9q6vqlTAUYeg4z7CDsboilWVJdilqVtU0vbNYJWE1mlNNGYDFwhXNueSvXLwC+WUxg0FDWdKjkMFf/mkzQ9m6moLJWfK0UX7fTjAOrWdWlCyM7HN349OTjz2Xq538YV+skIcIeytpmzrnDZnYj8CKZoawPOeeWm9l3gSXOuTlR3VuSoVLDXMMcyRSUikzC1+00+aW/hU2vNj61yu2HTa+iTiYpVWSZQ5SUOaRTmHtB5FPJTCJxWUTAjIfr4OBeZnc+oemkRjq1S4nJHERyiaoG4VepTCKRWURQt36s4iAz8Lr39n/I5P+8lakaFisFKDhIxUS53LdflHMighPoEjeyKWDyadfCu/Maj1dtWwZdj2FqtsMgpMUBpfooOEjF5KpBVCKTgPCW3vBLQz2i1dnU2Y2DEriftiSDgoPEplKZRFTdTKmYI1GIfw5FYP4EoEyiHVNwkNgUyiTSsPSGXxoyCcizcVBC99OWeCg4SGKkbemNoEKZRBKyiLwrupaxGKBUHwUHSYzg0htR1SPC3Ks6nySObCppRdd8wUL1iaqn4CCJFGU9IqqlN4LSMrKp6EX7gvUJqWoKDpJIlRzZFMVIpqCk1iP83UwlL9qn4nVVU3CQVIgqk4hywpxfUkc2+buZSto0KFi8VjdT1VFwkFSIap2mSo1kCkpqJlG0AsuIA8okUk7BQVIrimGvUJlupqRmEq0Ocy1EmUTVUXCQVIpq2GulupmCkpBJ5B3mWkiuTEJSTauySupli9V9a7o3ngsrk4hyhdd8krD6azaDmJ1daqMU2W6mAWOazqmbKVZalVXanSiHvSYhk4izHqFupvZLmYNUnagzCYAn/+qssl+rWHHtRvf06qeZ51/RdecqRvYeGU4moSyi4pQ5SLsX9YJ+USy9kU9c9YiSZlMX4s8klEWkgoKDVJ0oJ9BVaukNv6SObCqJZlenjoKDVL0wM4ng0huVGPYaFOfIpqKX2ihEcyIST8FBql5US4PHVayOa/XXspba8FOxOhUUHKTdCWuORFyzq4MqNbIpuNRGm0cyaU5EKmi0krRrYY5simtOhF+lRjZFOpIJ1M0UAY1WEilBmPWIuLqZ/CpVj8g1kklzIqqLMgcRnygziUpnEaBMQpoocxApQ1SZRBxZBMSbSbSZMolEUOYgkkd2RNMl3xhb1uvEMbM6l0plEtluppG9Rzaea/PQ12zBesbzIbWufVLmIBKysCbQxTEnIqhSmURZK7zm4p8XoS6milBwEMkjrG6mJBSroXJzJAoVrEvKIrT0RizUrSRSgmDBOs3DXgHef/Ipds2dC9DY3TT00UdCv4+/YK1idTzUrSQSobAm0CUxk4hyzabQJtCpWF0xCg4iJfAvxVHOPtZJmV3tF6xH7Fu8mH2LFzdmFtlryg0W2nEuHSLtVjKzScD9QEfgQefcXYHnvw78L+AwsA241jm3vtDrqltJkqDaZlcH+bucILpup7JGNqmbqWiJ6VYys47AA8D5QD2w2MzmOOdW+C57A6hzzu0zs78G7gEui6pNImGqttnVQZUqXpeVSaibKTKRZQ5mdhZwu3PuAu/42wDOuTtbuf5TwE+cc2cXem1lDpJE1Ta7OqhSxetQ965WFtEoMZkDMAjY4DuuB8bnuf464IXWnjSzmcBMgCFD4v1PIpJLtc2uDqpU8RpC2rtaWURZogwOluNczjTFzK4E6oDPtPZizrlZwCzIZA5hNFAkTLn2jWirJGwqlE+Uk+lCK1irWF2W2LuVzGwi8H+Azzjnthbz2upWkjQIa07EYwvf47mlGxuPF67dCcD44b0bz8UdLKJcliNYsFaxum2S1K20GDjJzIYDG4HpwBX+C7w6w8+AScUGBpG0iGpToWCwSEK3U6UyCRWrKyfqoayTgR+RGcr6kHPue2b2XWCJc26Omb0MjAE2e1/ynnPuzwu9rjIHSZswi9VBSSxeR5VJaNhr2yUpc8A5Nw+YFzh3m+/xxCjvL5IUYRarg/zF64Vrd7Jw7c5mmUU1LfCnYa+Vo7WVRGIQ1lLgQbm6nEYP7Jm4pcKjqkdAiZkEtJulwBOVOYhI68JaCtwv1wsy9rsAAAtASURBVLIcSRjp5M8kEjOyCZovBQ7tqpupEGUOIjFY/upGVi/a0nicrUdEnUkkYaRTYkY2LZkNy55pOs7WI6o0k1DmIJICueZEVCKTSMJIp8SMbMq1iJ82FWqkzEEkASqVSQQlYaRTYkY2+TOJKswilDmIpFChTCKsYa9BSRjplJiRTcHZ1e28HqHMQSSB/JlEpbKIpIx0SkQmUYX1CGUOIlUguKlQFPWIoKSMdEpEJqFNhZQ5iCRdXPWIQiOdKlWfSMQciSqYXa3MQaTKVGpkU1C+kU6VrE8kYo5EO5xdrcxBJGXiyiT84po/kZg5EincVEiZg0iVi2tkk1+h+RNRZRaJmSPRDjYVUuYgknL+TGLTOx8AcPxJxzY+X4lgEVSpekUiRjalZI0mZQ4i7Yw/k8jV5ZS9ppIqVa8IZhL7Fi9m3+LFjXtdtzVQlDxHogrnRChzEKliUe4j0VZRzqd4/8mnGgPDvsWLAfjEmWc2Pt/WYJE3k0jJnIhSMwcFB5EqloTidSHBJTwgnG4nf6CA8oLF06ufZt67TVvTLNmS+f1Td1zmd22zLqeEDntVcBCRVqUhk4hq5FNUwSKbUcyeNDvzZEIzCQUHEWlVGjKJShWzCwWLYgNFweJ1QjIJBQcRKVowk4g7i8glWMyGaOZTtLVeEexyKphJrP9d5vPQczKfKxQoFBxEpGhxLPBXjri6oLLDZYc++kjBry2YSfiDRTBQQGTBQsFBRNokifWIQooJFn5tDRylzKcoVLyGeEY6KTiISJsE6xFJmVBXimCw8CsnyyinPpGUkU4KDiISijQUr0sRZpdUOfMp/MGiRVaxu4HJe/YxFS97C7E+oeAgIpFIQ/G6FGGNiipmiGxQNngU7ILyB4syu5wUHEQkEklcwylMYY2KCgaLoHxdUnmDRcMyJn/4PlN7NtU9SskkFBxEJHLF1CeC0hQ8ohwVVUqXVLPJdtuWMfIIzHbHZS4scaSTgoOIVFwwWAQFg0eaAgVENzGvlC6pt3e+zW9OPsKW80/NnChUn4BmwULBQUQSp9q6pAp1QQWF0SWVDRzvfbIHALsP7eZ3ozvw4aRxmQuCwSJQo1BwEJFEq4Yhs375hs9CeHMvWssyWg0WgRqFXTtPwUFE0qPU+kWaAgeUPvciqLXgUUywAOjhOsDHR5j0yioFBxFJr3z1i7RnGUHlZh1+Z/xpARM2LuW4HkcBsO2jbez4aAcAQ/5rN6NXvZ2c4GBmk4D7gY7Ag865uwLPHwU8ApwB7AAuc86tK/S6Cg4i7VNbRkn5pS2QFAoefvkCybm/vZ2vPPZSMoKDmXUEVgPnA/XAYuBy59wK3zV/A5zqnLvezKYDlzjnLiv02goOIgKFR0n5lRpIypGEfbv9Fq7dyfq7L0rMHtLjgDXOuXcBzOwJ4GJghe+ai4HbvcfPAD8xM3Np7OsSkYrz759dSCmBpByb3vmATe98UJF7+R0NTOeonM+d1aEnf1vi60UZHAYBG3zH9cD41q5xzh02sw+BPsD24IuZ2UxgJsCQIeWv3S4i7UspgaQclQpCpRjap1vJXxNlcLAc54IZQTHXZE46NwuYBZlupfKaJiISjUoFoZJ9s7TLO0TTCiCTKQz2HdcAm1q7xsw6AccAOyNsk4iIFCHK4LAYOMnMhptZF2A6MCdwzRzgau/xFOAV1RtEROIXWbeSV0O4EXiRzFDWh5xzy83su8AS59wc4P8Bj5rZGjIZw/So2iMiIsWLsuaAc24eMC9w7jbf4/3A1CjbICIipYuyW0lERFJKwUFERFpQcBARkRYUHEREpIVUrspqZruBVXG3o436kmMGeEqo7fFQ2+NRbW0f6pzrV+wLRDpaKUKrSllAKknMbInaXnlqezzU9niE0XZ1K4mISAsKDiIi0kJag8OsuBtQBrU9Hmp7PNT2eJTd9lQWpEVEJFppzRxERCRCCg4iItJCqoKDmU0ys1VmtsbMbom7PYWY2TozW2ZmS81siXeut5n9ysze8T73irudWWb2kJltNbO3fOdyttcyfux9L/5kZmPja3mrbb/dzDZ67/9SM5vse+7bXttXmdkF8bQazGywmf3GzFaa2XIzu8k7n/j3PU/bE/++e23pamaLzOxNr/13eOeHm9lC771/0ttyADM7yjte4z0/LIFtf9jM1vre+9O986X/3DjnUvFBZtnv/wJOALoAbwKj425XgTavA/oGzt0D3OI9vgW4O+52+tp2HjAWeKtQe4HJwAtkdvP7NLAwgW2/HfhmjmtHez8/RwHDvZ+rjjG1eyAw1nvcA1jttS/x73uetif+fffaY0B373FnYKH3nj4FTPfO/xT4a+/x3wA/9R5PB55MYNsfBqbkuL7kn5s0ZQ7jgDXOuXedcweBJ4CLY25TW1wM/MJ7/AvgizG2pRnn3G9puRNfa+29GHjEZfwBONbMBlampS210vbWXAw84Zw74JxbC6wh8/NVcc65zc65P3qPdwMryeytnvj3PU/bW5OY9x3Aew/3eIedvQ8H/A/gGe988L3Pfk+eAT5nZrm2Oo5cnra3puSfmzQFh0HABt9xPfl/EJPAAS+Z2etmNtM7d5xzbjNk/nMB/WNrXXFaa29avh83emn0Q74uvES23eum+BSZvwJT9b4H2g4ped/NrKOZLQW2Ar8ik8184Jw77F3ib2Nj+73nPwT6VLbFTYJtd85l3/vvee/9D83sKO9cye99moJDrgid9HG4ZzvnxgIXAjeY2XlxNyhEafh+/DPwSeB0YDNwn3c+cW03s+7AvwJfc87tyndpjnNJa3tq3nfn3BHn3Olk9rgfB4zKdZn3OVHtD7bdzE4Bvg2cDJwJ9AZu9i4vue1pCg71wGDfcQ2wKaa2FMU5t8n7vBV4lswP35ZsOud93hpfC4vSWnsT//1wzm3x/gN9DPycpi6MRLXdzDqT+eX6L865f/NOp+J9z9X2tLzvfs65D4AFZPrjjzWz7Lpz/jY2tt97/hiK78qMjK/tk7yuPuecOwDMpoz3Pk3BYTFwkjeSoAuZgtCcmNvUKjPrZmY9so+BzwNvkWnz1d5lVwPPxdPCorXW3jnAVd4oiE8DH2a7QZIi0Kd6CZn3HzJtn+6NPhkOnAQsqnT7IDOKhMxe6iudcz/wPZX49721tqfhfQcws35mdqz3+GhgIpm6yW+AKd5lwfc++z2ZArzivGpvpbXS9rd9f1AYmVqJ/70v7ecmrmp7Wz7IVNxXk+kXvDXu9hRo6wlkRma8CSzPtpdMH+WvgXe8z73jbquvzY+T6QY4ROYvjetaay+ZNPUB73uxDKhLYNsf9dr2J+8/x0Df9bd6bV8FXBhju88hk97/CVjqfUxOw/uep+2Jf9+9tpwKvOG18y3gNu/8CWSC1hrgaeAo73xX73iN9/wJCWz7K957/xbwS5pGNJX8c6PlM0REpIU0dSuJiEiFKDiIiEgLCg4iItKCgoOIiLSg4CAiIi10KnyJSPtlZtkhpQADgCPANu94n3Puz2JpmEjENJRVpEhmdjuwxzl3b9xtEYmaupVE2sjM9nifJ5jZf5rZU2a22szuMrMveevtLzOzT3rX9TOzfzWzxd7H2fH+C0Rap+AgEo7TgJuAMcCXgRHOuXHAg8BXvGvuB37onDsTuNR7TiSRVHMQCcdi561VY2b/BbzknV8GfNZ7PBEY7dsCoKeZ9XCZvRBEEkXBQSQcB3yPP/Ydf0zT/7MOwFnOuY8q2TCRtlC3kkjlvATcmD3I7u8rkkQKDiKV81WgztulawVwfdwNEmmNhrKKiEgLyhxERKQFBQcREWlBwUFERFpQcBARkRYUHEREpAUFBxERaUHBQUREWvhvGrZIox81cokAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "surv.iloc[:, :5].plot(drawstyle='steps-post')\n",
    "plt.ylabel('S(t | x)')\n",
    "_ = plt.xlabel('Time')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation\n",
    "\n",
    "The `EvalSurv` class contains some useful evaluation criteria for time-to-event prediction.\n",
    "We set `censor_surv = 'km'` to state that we want to use Kaplan-Meier for estimating the censoring distribution.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "ev = EvalSurv(surv, durations_test, events_test, censor_surv='km')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Concordance\n",
    "\n",
    "We start with the event-time concordance by [Antolini et al. 2005](https://onlinelibrary.wiley.com/doi/10.1002/sim.2427)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6572131078044852"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ev.concordance_td('antolini')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Brier Score\n",
    "\n",
    "We can plot the the [IPCW Brier score](https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291097-0258%2819990915/30%2918%3A17/18%3C2529%3A%3AAID-SIM274%3E3.0.CO%3B2-5) for a given set of times.\n",
    "Here we just use 100 time-points between the min and max duration in the test set.\n",
    "Note that the score becomes unstable for the highest times. It is therefore common to disregard the rightmost part of the graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3zU9f3A8df7LrksMiFhJQHCBgGBMBxVQLQ4qavOVqutWqt2t3b8nK3aWqttta1asdaKe4B1D0AcIHsTCGFkAEnIIHvd5/fHfcEQLsmF3PG9u7yfj0ceuft+v5d7f7nAm8/n/RlijEEppZRqy2F3AEoppYKTJgillFJeaYJQSinllSYIpZRSXmmCUEop5VWE3QH4S58+fczgwYPtDkMppULKqlWrSo0xqd7OhU2CGDx4MCtXrrQ7DKWUCikisru9c9rFpJRSyitNEEoppbzSBKGUUsorTRBKKaW80gShlFLKK00QSimlvNIEoZRSyquwmQeh1LFqanGzqegg6/IrmJSZzLj0RLtDUiooaIJQPU5js5t1BRV8seMAX+w4wJr8cuqb3AC4nA7uu2gcl0xOtzlKpeynCUL1CE0tbhbnlPDqqgKWbCuhrqkFERjTP4HLp2QyZXAKI/vFc+fCjfzs5XVs31/FL+aMwukQu0NXyjaaIFRYq6xt4u+Lc3llVQEHahrpHefiksnpnDKsD9OzUkiKdR1x/b+/M5V73tzM45/kkbO/ijvOG0NWai+bolfKXpogVFgyxvDq6kLuf3sL5bWNfH1sPy6elM7pI1OJdLY/NiPS6eDeb5zAiL69uO/trZz58CdcPGkgt50xnPTk2ON4B0rZT8JlT+rs7Gyji/Wpg/VNfLa9lKc/28WXu8qYmJnE775xAmMHdL3wXFLVwN8X5/Lcsj0YDGeO6csFEwYyc1QqURHOAESv1PEnIquMMdlez2mCUOHg4637eeKTPFbuKqfZbegd5+IXc0Zy6eQMHN2sIxRV1PHk0jwWri3iQE0jCdERzBqVxuTBKUzOTGZYWi/2H6xnT1ktheV1TMxMYnjf+G69pzGG4qoGNu89SH1jCxkpsQzqHUt8dGS3fq5SbWmCUGGttrGZafd9RGJMJOdPGMDMkWlMykwiooOupGPR3OLm09xSFqwt4tPcUkqqGrxeJwLnnNCfW88Yxqh+CT7//NLqBt7ftJ/3N+9jQ0ElB2oaj7qmb0IU150yhGtOHkx0pLZiVPd1lCC0BqFC3v/W7aWqvpl5105hyuCUgL1PhNPBjJFpzBiZhjGGgvI6Vu8pZ0dJDf0ToxnUO5a0+GjeWFPIvz/fxVsb9nJSVm9G9Y9naGovhqb24oSBCUe0AkqqGnh3417e2rCXL3eW4TYwuHcsZ4xOY3T/BEb3T6BXVAR7ymrZU1bLZ7ml3P/OVv79+S5+NHs4/RJjWL27nNV7yimvbSR7UArTs1KYOqQ3KXGuDu5Gqc5pC0KFvLmPfkptYwvv//g0RIJjWGpFbSPzPtvFR1v2k1dSQ11TC+BpXQxP68WJGUkUlNexLO8AbgNDU+M4d1x/zh7Xn1H94ju8j2V5B3jgna2sza8AwCEwsl8CSTGRrM2voK6phQiH8J/rpnLysD7H5X5V6NIuJhW2NhZWct7fPuWu88dw7SlD7A7HK7fbsO9gPdv2V7Euv5I1+eWsza8gJdbFueP7c+74/ozs23FSaMsYw2e5BxCBCRlJ9IrydAY0NrvZUFjBzc+tZnT/BP79namBui0VJrSLSYWt55bvITrSwYWTgnfms8MhDEiKYUBSDDNGpvnlZ4oIpw4/unXginAweVAKl03J5G8fb6egvFaH56pjpov1qZBVVd/EgrWFnD9+AIkxOrqntcumZADw0op8myNRoUwThAopbvdXXaJvrC2itrGFq6YPsjGi4DQwKYYZI1J5cWU+zS1uu8NRIUq7mFRQqW9qYdXuck4YkEhi7FetguV5B7j/na1sLjrIiRlJTM9K4a0Nexk7IIEJuvqqV1dMzeSGZ1exKKeEM8f0tTscFYI0QaigsbeyjhufXcX6gkoincKpw/pw5ph+fLRlPx9tLaZfQjSXT81gXX4Fjy7KxW3g/ovGBc3IpWAza1QaafFRzF++WxOEOiaaIFRQWLW7jBufXU1dYzO/v/AEdh+o5a31e1mUs4H4qAh+MWck150y5PDksKr6JrYXV3NiepLNkQevCKeDy6Zk8OiiXAor6hiYFGN3SCrE6DBXFXDFB+t55otdbN1bxaXZGZw1pu/h5S/KahqZv3w3f/loOwOTYnji29mMsJapMMawdV8V/ROjj1p1Vfkmv6yW0x5cxJRBKaQlRFFZ10Sk08GNp2UxLau33eGpIKDDXJUtcoureOKTPN5YU0ST202fXlF8tLWYrNQ4vjV9EBsKKvnfhr00NruZPTqNhy498Yi6g4gwur/vS1Woo2WkxHLxpHQ+2rKfkuoGEmIi2VdZx2VPLGPO2H786pxRDOodZ3eYKkhpC0L53c7SGv7y4TYWrCsiKsLBpZMzuP7UIaQnx/DOxn38Y/EONu89SK+oCC6cOJCrpw9iZL/uLW6nfFfX2MK/lubxjyU7aG4xXDRpIN86adAxrXirQp/OpFYBVVLVwM7SGnaWVvPlznLeWFtIpFO45uTB3PC1LHr3ijri+kNdR5kpscRFaSPWLvsP1vOXj7bz2uoC6pvcZA9K5tLsdKYMTmFInzgt/vcQtiUIEZkD/AVwAv8yxjzQ5vxPgO8CzUAJcJ0xZrd17hrgt9alvzPGPNPRe2mCsMcf393K3xfvOPw8OtLBFVMz+f6MoaTFR9sYmfJVZW0TL6/K57/LdrPrQC0AKXEupmelcM/cE+jTJsGr8GJLghARJ7ANOBMoAFYAVxhjNre6Ziaw3BhTKyLfB2YYYy4TkRRgJZANGGAVMNkYU97e+2mCOP4KK+qY8eAiZoxM4+rpg8jqE8eApBjdxzlEud2GvNJqVu4qZ9XuchasLeKssX159MpJdoemAsiuIvVUINcYk2cF8QIwFzicIIwxi1pdvwy42nr8deADY0yZ9doPgDnA8wGMV3XR3xflAnDXBWN1CGUYcDiEYWnxDEuL5/KpmaQnx/Lwh9u4eHIxM/20hpQKLYFcamMg0HohmALrWHuuB97pymtF5AYRWSkiK0tKSroZruqKooo6XlqZzzezMzQ5hKmbZmQxLK0Xv319I7WNzXaHo2wQyAThrZ/Ba3+WiFyNpzvpwa681hjzhDEm2xiTnZqaesyBqq77+2JP6+HmmcNsjkQFSlSEk/svGkdhRR0Pf7DN7nDC1oHqBu5csJFleQfsDuUogUwQBUBGq+fpQFHbi0RkNvAb4AJjTENXXqvsUVRRx4sr8rlUWw9hb8rgFK6YmsG8z3axsbDS7nDCzqKcYr7+yFKe+WI3C9YW2h3OUQKZIFYAw0VkiIi4gMuBha0vEJGJwON4kkNxq1PvAWeJSLKIJANnWcdUEDjcepgx1OZI1PFw+5zRJMdGcs//NhMuw+LtVtfYwp0LNvKdp1fQO85FWnwUZV72ILdbwBKEMaYZuAXPP+xbgJeMMZtE5B4RucC67EGgF/CyiKwVkYXWa8uAe/EkmRXAPYcK1speBeW1vLSigEsmZ+hGND1EYmwkt84azpc7y/h8R/B1g4SaVbvLOOevnlbD9acOYcEtpzC4TxzltU12h3aUgM5SMsa8Dbzd5tgdrR7P7uC184B5gYtOHYtHPtwOArfO0tpDT3LZlAz+uWQHf/5gGycP7a2T6I5BfVMLf3ovh6c+28nApBjmf3fa4T3Dk2Mj2VlaY3OER9MNg5TPcoureG11Ad+aPogBWnvoUaIjndwyaxirdpezZFv7IwZzi6vZUVJ9HCMLHXcs2Mi/Pt3JlVMzefdHpx1ODgDJsa6gbEFoglA+e+j9bcREOrX20ENdOjmD9OQYHv5g21G1iK37DnLzc6uY/eclXPC3T1m1W3uE21q1u5wzx/Tl9xeOo1ebJWaSYl2U1zQGXY1HE4TyyfqCCt7ZuI/rvaytpHoGV4SD22YNZ11BJR9tKaa8ppEFawu54T8rmfPIUj7ZVsrNM4aSlhDNtfNWsDa/wu6Qg0Zzi5s9ZbUMS+vl9XxKXCTNbkN1Q3DNN9GV0pRPHnwvh6TYSL73tSF2h6JsdOGkgTy2OJcfvbiW2sZm3AZ6x7m4bdYwrjt1CEmxLr510iAue3wZ335qOfO/N50TBuoqsfnldTS1GLL6eF9a/dB+JxW1TcRHR3q9xg7aglCd+nxHKUu3e/53GEy/vOr4i3Q6uPP8MYzuH88tM4fx+s0ns+I3s/nJWSMP/yPXPzGG+d+bRnx0JFc/tZz9B+ttjtp+O4o9dZmsVO8tiGTrz668NriGumqCUB1qcRvu/d8WBiRG8+2TBtsdjgoCs0b15eWbTuYnZ41kYmby4d0BW0tPjuXZ66dSXd/MP1qt9ttT5ZV6EsTQVO8tiGRro6xgK1RrglAdenFFPlv2HuRX54w+vB+0Ur7ISu3FxZPSmf/lnh7fisgrqSElztXu1rnJcVYLIsgmy2mCUO2qrGviofdzmDI4mfPG97c7HBWCfjBzGG636fGtiLySmnbrD6BdTCoE/e2j7ZTVNnLn+WN1YpQ6Jpm9Y7UVgaeLaWg79QeAxJhIRLSLSYWIHSXV/PvzXVyWnaGjUFS39PRWRGVdE6XVjWS1U38AcDqEhOhIKoKsBaHDXJVXD7yzlZhIJz89a6TdoagQl9k7losmDWT+l3uYPbovFXWNFJTXUdfYwsCkGAYmxzCod2zYru2VV9LxCKZDUuJcQbdgnyYIdZQtew/yweb9/Hj2CFLjdVKc6r5bZg7n1dWFXP3U8navue2M4fx49vCw687MK/GssdRRCwIgKTaSiiDrYtIEoY7yj8U7iHM5ufbkwXaHosJEZu9Y5n93Ggfrm0lPjiE9OYboSCf7KuvJL6/llZUF/PWj7TQ0t3D7nFGHk8S2/VVs2XuQCyYMCNnEsaOkmgiHkJnScQspOdYVdHUaTRDqCLtKa/jf+iK+d1oWibE6KU75z7Ss3kcdy0iJJSMllulDehMb5eTxJXk0Nru5cOJAHluUy3ub9gNgDHxjYkc7FgevvJIaMlNiiXR2XPJNio0kZ1/VcYrKN5og1BEe/ySPCKeD60/VJTXU8eNwCPfOPQGX08m8z3by9Ge7SIiO4LZZw1iaW8rdb27ia8P7HLEOWHlNI/XNLfRPDO6VhfNKqzvtXgJIidUahApi+yrreXVVAd+ckk5afLTd4ageRkT4v/NGMzA5huYWN1dOyyQ+OpLzJgzg3L8u5e43N/PXKyYCsLGwkmufXkFclJPFP5sRtN1PLW7DrgO1zBiZ1um1yXEu6ppaqG9qCZpJqZog1GH/WppHizHceJou563sISJHtV5H9I3nlpnDefjDbcw9cQAxLic3/GcV9U0tlFYbdpTUtLtKqt0Ky+tobHZ3OEnukCSrS7eitol+icGRIHQehAKgrKaR+V/u4YIJA8jopJim1PH2/RlDGdk3nl+8sp5r561gQFI0L9wwHaDDDYzstuPQGkw+JLBgnE2tCUIB8LePt1Pf1MIPZmrrQQUfV4SDP1wynoq6JsanJ/LyjSeTPTiFoalxQZ0gDg9x9aEFEYwJQruYFHsO1PLfZbu5bEoGw9Li7Q5HKa9OzEhi8c9m0DchGleE5/+2p41IZf7yPUHVb99aXkk1iTGRpMR5X6SvteQ4a0XXmuCZC6EtCMWD7+fgdAg/mj3C7lCU6lBGSuzh5ABw+ohUGprdLN8ZnFuc5pXUkJUa51MRPRhbEJogerj1BRW8ua6I756aRd8EHbmkQsv0rN5ERThYkhOc3Uw7SqrJ6uNbAf2rInXwJAjtYurBjDE88M5WUuJc3Hh6lt3hKNVl0ZFOpg5J4ZPtwZMg8stqWbq9lE9zSyiuavBpDgRAVISTOJczqFZ01QTRgy3OKeHzHQe48/wxupWoClmnj0jld29tobCijoFJnklz72/aR0ZKLKP7Jxy3OL7cWcYjH27j8x0HAOifGM03s9O5eFK6zz8jKdYVVJsGaYLooQ5UN/DLV9eTlRrHVdMG2R2OUsfsUIL4ZFsJV0zN5LFFuTz4Xg5REQ4e+uYEzhs/IKDvvy6/gj++t5XPcg/Qp1cUv5wzijPH9GWoj7WH1pLjIoOqBqEJogdyuw0/fXkdFXVNPP2dKUcU/ZQKNcPSejEgMZrFOcXsrajjrx/ncv6EAeyrrOOW+WvILa7mh2cEZpXYVbvLuPLJ5cRHR/Lbc0dz1bRBxLiOfTRVcqxLu5iUvZ5cmsfinBLunTuWsQN0MyAV2kSE00ak8uLKfN7btJ/LsjO476JxNLvd/Pq1jTzy4Xa2F1dz/0XjSPBjV2pucTXXP7OSAUkxvHLTSUesE3WskmJd5JfV+iE6/9D/OvYwq/eU8+B7OZx9Qj+unq5dSyo8zB7dF2PgW9MHcf9F43A6hKgIJ3+6dDy3nz2Kdzfu4+xHlrI874Bf3m//wXqumfclEQ4Hz3xnql+SA0BKbKS2IJQ9DtY3cdvza+iXGM0DF48P2gXOlOqq2WP68uFPTmNoaq8jfq9FhJtOH8rUISn8+MW1XP7kMq47ZQgpcS42Fx1k896DDEyK4bfnjWZUP98K2mU1jVz79Aoqaht58caTyOztv6VpkmJdVNY10dziJqKT5cGPB00QPYQxht+8vpG9lfW8fNNJJMboqCUVXjpaBWBSZjJv3/Y1fvfWZp76dCcAGSkxjO6XwIpdZZz710+55qTB/OjM4R12Q20uOsgNz66kuKqBf3072+/7tSdbcyEq65qOuVVSXFXPfz7fTYSz+5NfNUH0EK+tLuTNdUX87KwRTMpMtjscpY67uKgI7r9oPDfPGEZCTOTh/ySV1zTy4Ps5PP35Tt7aUMTrN5/CgKSj95j43/oifv7yehJjInn5xpOYkJHk9xiT4w7Npu56gthVWsPjn+Tx6uoCGpvdgKeA351RXPa3YVTA7Sqt4Y4FG5k6JIXvzxhmdzhK2SojJfaIFnRynIv7LhzHa98/mar6Zn756nqMMUe85vElO7hl/hrGDEhg4a2nBCQ5wFfLbXRlNrUxhqc+3cmZDy/h1dUFXDI5nQ9/cjoT0hP5vzc2UlLVcMzxaIIIc9UNzfzwhTU4HcIjl52I06F1B6W8mZiZzK/OHsXS7aU8/2X+4eML1hZy/ztbOW98f+Z/b1pAN9M6lCB83VmuoraRG55dxb3/28zpI9L49Jczue/CcQxL68VD35xATWMLv359w1EJz1faxRSm8stq+ffnu3hpRT5VDc3846pJXpvNSqmvXDVtEO9s3Mfv39rM14b3Yd/Ben7+8nqmDknhoW9OICoisCvGtt40qDMrd5XxwxfWUlxVz/+dN4brThl8RIF+WFo8Pz9rJL9/ewuvrynkoi7M6D4koAlCROYAfwGcwL+MMQ+0OX8a8AgwHrjcGPNKq3MtwAbr6R5jzAWBjDVcuN2Ge9/azDOf78Ihwjnj+nP9qUMC1iRWKpw4HMIfLxnPnEeWcuvza9h9oIb05Bie+NbkgCcHaF2DaL8F0dTi5q8fbeexRbkMTI7hlZtObvfv93WnDuH9zfu4Y8EmXl1d8NXxU4Zwxui+ncYTsAQhIk7gMeBMoABYISILjTGbW122B7gW+JmXH1FnjDkxUPGFo+YWNz9/ZT2vrynkymmZ3DprWNBv6K5UsElPjuW3547m9tc2kBLn4unvTCEptvP9HPwhzuXE5XS0OxdiZ2kNP3phDesKKrl0cjp3XjCWXlHt/zPudAh//uaJ3LlwEwfrPD9zfWElfROi7U0QwFQg1xiTByAiLwBzgcMJwhizyzrnDmAcPUJDcwu3Pb+G9zbt5+dfH8kPZmoxWqljddmUDGoaW5g2JIVBvX1bjdUfRISk2EivRep3N+7lZy+vx+kQ/n7VJM4Z19+nn5mREsu8a6ccfj7rT4sPj3LqTCATxEAgv9XzAmBaF14fLSIrgWbgAWPMG/4MLpzUN7Vw47OrWLKthDvOG8N1bTZ9V0p1jYhwvU1/j5JjXUcUqZtb3Dz4fg6PL8ljQkZSt+uJrghHUCQIb8NlulJKzzTGFIlIFvCxiGwwxuw44g1EbgBuAMjMzDz2SENYY7Obm59bzZJtJfzh4nFcNqVn/jkoFS48LQhPd9DO0hp+/doGvsg7wFXTMrnj/DHdroW4Ihw0ttifIAqAjFbP04EiX19sjCmyvueJyGJgIrCjzTVPAE8AZGdnH9s4rhDW1OLm1udX8/HWYu67UJODUuEgJc7FxqJK7n5zE89+sRtXhIMHLxnPpdkZnb/YBy5ncLQgVgDDRWQIUAhcDlzpywtFJBmoNcY0iEgf4BTgjwGLNAS1uA0/eWkd723az53nj+HKaZoclAoHnhVd63jm811cNiWDH585wq9zL4Kii8kY0ywitwDv4RnmOs8Ys0lE7gFWGmMWisgU4HUgGThfRO42xowFRgOPW8VrB54axOZ23qpH+ueSHby5rojbzx7Fd07RmoNS4eK88f1paGrhxtOHMrJf++tLHStXhIPqhmafrg3oPAhjzNvA222O3dHq8Qo8XU9tX/c5MC6QsYWy8ppG/rl4B7NH9+Wm04faHY5Syo9OGdaHU4b1CdjP70oXky61EYIeW5RLTWMzv5gz0u5QlFIhpitdTJogQkxhRR3/+WI3F09KZ0Rf/zc/lVLhzRXhoMFfCUJEYkXk/0TkSev5cBE5r5sxqmP08AfbQODHZ3ZvnXelVM8U1YVhrr60IJ4GGoCTrOcFwO+OLTTVHdv2V/Ha6gKuOWmQLrynlDom/q5BDDXG/BFoAjDG1OF9EpwKoKYWN3e/uYk4VwQ3654OSqljFOnnBNEoIjFYs6BFZCieFoU6Ttxuwy9eWc9nuQf49bmjD6/4qJRSXeXvmdR3Au8CGSLyHJ5Ja9cec3SqS4wx3LFwI6+vKeTnXx/JFVN1QpxS6ti5Ihy0uA0tbtPpBmIdJgjx7D6xFbgImI6na+mHxphSfwWr2meM4Y/v5fDfZXu48fQsbp6hcx6UUt3jivB0HDU2u4lxdbyuU4cJwhhjROQNY8xk4C2/Rag6tftADb99YyNLt5dy1bRMbp8z6ojdopRS6li4nFaCaHETQzcShGWZiEyxZj2rAGtsdvP4kh08uiiXSKeDu84fw7dPGqzJQSnlF1GtWhCd8SVBzARuFJHdQA2ebiZjjBnfjRhVO/70fg5PfJLHueP7c8d5Y+ibELgN0pVSPc/hLiYfCtW+JIizuxmP8pExhrc37OWMUWk8duUku8NRSoUhVxdaEJ0OczXG7AaSgPOtryTrmPKz3OJqCsrrmDU6ze5QlFJhyuX01B38kiBE5IfAc0Ca9fVfEbm1eyEqbz7eWgzAzJGaIJRSgdGVFoQvXUzXA9OMMTUAIvIH4Avgb8ceovLm463FjOoXr8toKKUC5qsaREun1/oyk1qA1j+pBV1qw+8q65pYubucWaO09aCUCpxDw1x9WdHVlxbE08ByEXndev4N4KljDU55t3R7CS1uowlCKRVQfu1iMsb8WUQWA6fiaTl8xxizpnshqrY+3lpMUmwkEzOT7Q5FKRXG/DoPQkSmA5uMMaut5/EiMs0Ys7ybcSqL221YklPC6SNSO10bRSmluqMr8yB8qUH8A6hu9bzGOqb8ZF1BBQdqGrV7SSkVcIeX2vDHMFdAjDHm0BNjjBvfahfKR4u2FuMQOG14qt2hKKXCXKQ/J8oBeSJym4hEWl8/BPK6F6JqbVFOCRMzk3WfB6VUwLVerK8zviSIm4CTgUI8241OA2449vBUa0UVdWworOQMnT2tlDoO/D2KqRi4vNtRKa/e3bgPgDlj+9kciVKqJzg0ismXeRC+LLXxRxFJsLqXPhKRUhG5uvthKvAkiJF948lK7WV3KEqpHsDfReqzjDEHgfPwdDGNAH7ejfiUpbiqnhW7y5hzgrYelFLHh8MhRDiEJj/VICKt7+cAzxtjyroTnPrK+5v2YwycPU4ThFLq+HFFOPy2WN+bIrIVqANuFpFUoL6b8SngvU37GNInjpF94+0ORSnVg7giHP4ZxWSMuR04Ccg2xjQBtcDcbkfYw1XUNvLFjgPMOaGfbieqlDquXE7/tSAwxpS3elyDZza16oYPNu+n2W04W+sPSqnjzNcuJl9qECoA3t24j4FJMYwbmGh3KEqpHsYV4aChu11M4pHht6gUAFX1TSzdXqrdS0opW/jaxdRhgrDWYHrDX0Epj+eW76Gxxa3dS0opW0T5sYtpmYhM6X5ICmB9QQUPvZ/DnLH9mDxI935QSh1//hzmOhO4SUR24SlOC57GxfhuRdgDVTc0c9vza0jtFcUDF4/T7iWllC1cEQ7qm/zTgjgbyAJmAefjmVF9vi9BiMgcEckRkVwRud3L+dNEZLWINIvIJW3OXSMi262va3x5v2B3x4KN7Cmr5ZHLJ5IUqyu3KqXs4ZcaBIAxZjeQAcyyHtf68joRcQKP4UkwY4ArRGRMm8v2ANcC89u8NgW4E8/KsVOBO0UkpPtjXlqRz2urC7ntjOFMHZJidzhKqR4s0l8JQkTuBH4J/OrQzwb+60MMU4FcY0yeMaYReIE2E+yMMbuMMeuBtpF+HfjAGFNmzcH4AJjjw3sGneYWN394dyu/eHU9J2X15paZw+wOSSnVw/k6k9qXGsSFwERgNYAxpkhEfFkbYiCQ3+r5ob0kfOHttQPbXiQiN2DtTZGZmenjjz5+SqoauPX51SzLK+OKqZncef4YIpw69UQpZS9/FqkbjTFGRAyAiMT5GIO3CqzxcuyYX2uMeQJ4AiA7O9vXn31c1DQ0M/fRTzlQ08ifLp3AJZPT7Q5JKaUAzzBXv+wHAbwkIo8DSSLyPeBD4EkfXleAp3ZxSDpQ5MPruvvaoLB0eylFlfX88+rJmhyUUkHFU6Ru6fQ6X3aU+5OInAkcBEYCdxhjPvAhhhXAcBEZgme70suBK314HcB7wH2tCtNn8VUNJCQs2VZCr6gITh3ex+5QlFLqCP6sQWAlBF+SQuvXNIvILXj+sXcC84wxm0TkHmClMWahNYGllu8AABBrSURBVAHvdSAZOF9E7jbGjDXGlInIvXiSDMA9obQPhTGGJTnFnDKsN5Fac1BKBRlXhIOmls575dtNECLyqTHmVBGp4sj+/0MT5RI6++HGmLeBt9scu6PV4xV4uo+8vXYeMK+z9whG24urKaqs59YzhtsdilJKHcXldNLiNrS4O04S7SYIY8yp1nfdzaaLluSUAHD6iFSbI1FKqaO5Inzbl7qz1VwdIrLRf2H1DIu3FTOiby8GJMXYHYpSSh3FLwnCGOMG1olI8E0yCFI1Dc2s2FnOjJFpdoeilFJeHUoQDS0dj2TypUjdH9gkIl/Saic5Y8wF3YgvbH2x4wCNLW7tXlJKBa0op28tCF8SxN1+iKfHWLKthFiXk+zBIb10lFIqjPnaxeTLPIglhx6LSB/ggLWRkGrDGMPibcWcPLQ3URFOu8NRSimvDieITuZCtFuDEJHpIrJYRF4TkYlWsXojsF9EQnLhvEDbWVpDflkdp2v9QSkVxFx+6GJ6FPg1kAh8DJxtjFkmIqOA54F3/RJpGPnf+r0AzND6g1IqiPljFFOEMeZ9Y8zLwD5jzDIAY8xWfwUZTnKLq3h0US6zR/clIyXW7nCUUqpdkT62IDpKEK1fWdfmnNYgWmlucfPTl9YR63Jy30Un2B2OUkp16KthrsfexTRBRA7iWVojxnqM9TzaDzGGjX8u2cG6gkoevXIiafH6R6OUCm5R3R3FZIzRYTg+2Fx0kL98tJ1zx/fnvPED7A5HKaU65ZeZ1KpjtY3N/OSltSTGuLh3rnYtKaVCgz9GMakOGGP41WsbyNlfxdPXTiElzmV3SEop5ZNuz4NQHXv6s10sWFvET88coesuKaVCyqEE0aQJwv+W5R3g929v4cwxfbl5xjC7w1FKqS7RGkSAlFY3cMv81QxKieWhb07A4RC7Q1JKqS45VINo0BqEfz25NI+ymkae++50EqIj7Q5HKaW6zNcitbYguuBgfRPzl+3h7HH9GdlPN9pTSoUmh0OIdIoWqf3pv8t2U9XQzPdPH2p3KEop1S0up0NbEP5S39TCvE938bXhfThhYKLd4SilVLe4IjRB+M2rqwsorW7Q1oNSKixogvCTFrfhyU/ymJCeyElDe9sdjlJKdZsrwqE1CH94d+M+dh2o5abThyKiw1qVUqFPaxB+smBtIQOTYjhrbD+7Q1FKKb+IdDo6nQehCcIHGwormTwoGadOilNKhYko7WLqvtLqBvZW1jM+XUcuKaXCh6dI3dLhNZogOrGhsBJAh7YqpcKKjmLygw0FngQxdkCCzZEopZT/uJzaxdRtGworyUqNI17XXVJKhRFtQfjBxsJKxmn3klIqzLginJoguqOkylOg1gShlAo3LqeDphbT4TWaIDqwUQvUSqkw5YrQeRDdsqGwEhEtUCulwk+UDnPtng2FlQzpowVqpVT4sX0tJhGZIyI5IpIrIrd7OR8lIi9a55eLyGDr+GARqRORtdbXPwMZZ3s2FGiBWikVnnxZiylgW46KiBN4DDgTKABWiMhCY8zmVpddD5QbY4aJyOXAH4DLrHM7jDEnBiq+zpRUNbDvoBaolVLhyRXhwN1xjTqgLYipQK4xJs8Y0wi8AMxtc81c4Bnr8SvAGRIky6UeKlBrglBKhSNXROf//AcyQQwE8ls9L7COeb3GGNMMVAKHNlwYIiJrRGSJiHzN2xuIyA0islJEVpaUlPg1+MMFak0QSqkw5HLamyC8tQTaNmjau2YvkGmMmQj8BJgvIkcNJTLGPGGMyTbGZKempnY74NbWF3gK1L2iAtYLp5RStrG7BVEAZLR6ng4UtXeNiEQAiUCZMabBGHMAwBizCtgBjAhgrEfZWFjJeG09KKXClN0JYgUwXESGiIgLuBxY2OaahcA11uNLgI+NMUZEUq0iNyKSBQwH8gIY6xF2H6hh38F6TsxIOl5vqZRSx5UvXUwB6z8xxjSLyC3Ae4ATmGeM2SQi9wArjTELgaeAZ0UkFyjDk0QATgPuEZFmoAW4yRhTFqhY2/poSzEAM0elHa+3VEqp48qXFkRAO9iNMW8Db7c5dkerx/XApV5e9yrwaiBj68iinGKGpsYxqHecXSEopVRA2V2kDknVDc0szytjlrYelFJhzO4aREj6dHspjS1uZo3qa3coSikVMJogjsGircXER0eQPTjZ7lCUUipgNEF0kdttWJRTzGnDU4n0oX9OKaVCldYgumhT0UGKqxq0/qCUCntR2oLomo+3FiMCM0b6d1a2UkoFG+1i6qKPc4o5MSOJ3r2i7A5FKaUCShNEF5RUNbAuv4JZI7V7SSkV/rQG0QVLt3tWg9XZ00qpnkBbEF2wNr+COJeTMf11/2mlVPjTBNEF6wsqOWFgIg5HUOxXpJRSAaVdTD5qbHazee9BJujqrUqpHkJEOk0SmiCAbfuraGx26/aiSqkepbNuJk0QeLYXBRifrglCKdVzRDo77lLXBIGn/pAYE0lmSqzdoSil1HGjLQgfrC+oYHx6IiJaoFZK9RyaIDpR39RCzr4qrT8opXocLVJ3Yuu+KprdRusPSqkexxXh7PB8j08Q6wsqABifrkNclVI9i3YxdWJ9QSV9ernonxhtdyhKKXVcRWkXU8c2FFQybqAWqJVSPY+2IDpQ29jM9uIqxmn3klKqB9IE0YFNRQdxG5igBWqlVA+ko5g6sL7AM4Nah7gqpXoibUF0YG1+Bf0SoklL0AK1Uqrn0QTRjme/2MWb64o4fYTuP62U6pk6SxARxymOoPLYolwefC+H2aPTuHvuWLvDUUopW3RWg+gRCcIYQ3ltE/lltSxYW8S8z3Yy98QB/OnSCUT6sGmGUkqFo6ie3oJ4ZVUBdy3cRHVD8+FjV03L5N65J+jucUqpHq1HdzEZY3h8yQ7SEqL48bQRZCTHMKRPHMPSeunEOKVUjzdpUHKH58M6QWworGR7cTX3XTiOK6dl2h2OUkoFlZkj0zo8H9Yd8K+sKiAqwsG54/vbHYpSSoWcsE0QDc0tLFxXxFlj+5EYE2l3OEopFXLCNkF8vKWYitomLpmcbncoSikVksI2Qby6uoC+CVGcOqyP3aEopVRICmiCEJE5IpIjIrkicruX81Ei8qJ1frmIDG517lfW8RwR+XpX3rekqoFFOSV8Y+JAnDqUVSmljknAEoSIOIHHgLOBMcAVIjKmzWXXA+XGmGHAw8AfrNeOAS4HxgJzgL9bP88nC9YW0uI2XDJJu5eUUupYBXKY61Qg1xiTByAiLwBzgc2trpkL3GU9fgV4VDwTFOYCLxhjGoCdIpJr/bwv2nuzbfurmP3nJQDsq6xnQnoiw/vG+/eOlFKqBwlkghgI5Ld6XgBMa+8aY0yziFQCva3jy9q8dmDbNxCRG4AbABIGZDHSSggj+8Zz1XSd96CUUt0RyAThrfPf+HiNL6/FGPME8ARAdna2eeyqSV2NUSmlVDsCWaQuADJaPU8Hitq7RkQigESgzMfXKqWUCqBAJogVwHARGSIiLjxF54VtrlkIXGM9vgT42BhjrOOXW6OchgDDgS8DGKtSSqk2AtbFZNUUbgHeA5zAPGPMJhG5B1hpjFkIPAU8axWhy/AkEazrXsJT0G4GfmCMaQlUrEoppY4mnv+wh77s7GyzcuVKu8NQSqmQIiKrjDHZ3s6F7UxqpZRS3aMJQimllFeaIJRSSnmlCUIppZRXYVOkFpEqIMfuOPykD1BqdxB+Ei73Ei73AXovwcjO+xhkjEn1diKcthzNaa8SH2pEZKXeS3AJl/sAvZdgFKz3oV1MSimlvNIEoZRSyqtwShBP2B2AH+m9BJ9wuQ/QewlGQXkfYVOkVkop5V/h1IJQSinlR5oglFJKeRUWCUJE5ohIjojkisjtdsfTVSKyS0Q2iMhaEVlpHUsRkQ9EZLv1PdnuONsSkXkiUiwiG1sd8xq3ePzV+ozWi0hQ7e7Uzr3cJSKF1ueyVkTOaXXuV9a95IjI1+2J+mgikiEii0Rki4hsEpEfWsdD7nPp4F5C8XOJFpEvRWSddS93W8eHiMhy63N50doaAWurgxete1kuIoNtCdwYE9JfeJYS3wFkAS5gHTDG7ri6eA+7gD5tjv0RuN16fDvwB7vj9BL3acAkYGNncQPnAO/g2S1wOrDc7vh9uJe7gJ95uXaM9XsWBQyxfv+cdt+DFVt/YJL1OB7YZsUbcp9LB/cSip+LAL2sx5HAcuvP+yXgcuv4P4HvW49vBv5pPb4ceNGOuMOhBTEVyDXG5BljGoEXgLk2x+QPc4FnrMfPAN+wMRavjDGf4NnHo7X24p4L/Md4LAOSRKT/8Ym0c+3cS3vmAi8YYxqMMTuBXDy/h7Yzxuw1xqy2HlcBW/Ds5x5yn0sH99KeYP5cjDGm2noaaX0ZYBbwinW87edy6PN6BThDRLxtxRxQ4ZAgBgL5rZ4X0PEvUTAywPsiskpEbrCO9TXG7AXPXxQgzbbouqa9uEP1c7rF6nqZ16qbLyTuxeqWmIjnf6sh/bm0uRcIwc9FRJwishYoBj7A08KpMMY0W5e0jvfwvVjnK4Hexzfi8EgQ3rJqqI3dPcUYMwk4G/iBiJxmd0ABEIqf0z+AocCJwF7gIet40N+LiPQCXgV+ZIw52NGlXo4F+72E5OdijGkxxpwIpONp2Yz2dpn1PSjuJRwSRAGQ0ep5OlBkUyzHxBhTZH0vBl7H88uz/1BT3/pebF+EXdJe3CH3ORlj9lt/qd3Ak3zVXRHU9yIikXj+QX3OGPOadTgkPxdv9xKqn8shxpgKYDGeGkSSiBxaE691vIfvxTqfiO9doH4TDgliBTDcGg3gwlPQWWhzTD4TkTgRiT/0GDgL2IjnHq6xLrsGWGBPhF3WXtwLgW9bo2amA5WHujyCVZu++AvxfC7guZfLrZEmQ4DhwJfHOz5vrH7qp4Atxpg/tzoVcp9Le/cSop9LqogkWY9jgNl4aiqLgEusy9p+Loc+r0uAj41VsT6u7K7u++MLz0iMbXj69H5jdzxdjD0Lz8iLdcCmQ/Hj6W/8CNhufU+xO1YvsT+Pp4nfhOd/PNe3FzeeJvNj1me0Aci2O34f7uVZK9b1eP7C9m91/W+se8kBzrY7/lZxnYqnK2I9sNb6OicUP5cO7iUUP5fxwBor5o3AHdbxLDxJLBd4GYiyjkdbz3Ot81l2xK1LbSillPIqHLqYlFJKBYAmCKWUUl5pglBKKeWVJgillFJeaYJQSinlVUTnlyilWhORQ0NGAfoBLUCJ9bzWGHOyLYEp5Wc6zFWpbhCRu4BqY8yf7I5FKX/TLial/EhEqq3vM0RkiYi8JCLbROQBEbnK2hNgg4gMta5LFZFXRWSF9XWKvXeg1Fc0QSgVOBOAHwLjgG8BI4wxU4F/Abda1/wFeNgYMwW42DqnVFDQGoRSgbPCWOsaicgO4H3r+AZgpvV4NjCm1VL/CSISbzz7HyhlK00QSgVOQ6vH7lbP3Xz1d88BnGSMqTuegSnlC+1iUspe7wO3HHoiIifaGItSR9AEoZS9bgOyrd3RNgM32R2QUofoMFellFJeaQtCKaWUV5oglFJKeaUJQimllFeaIJRSSnmlCUIppZRXmiCUUkp5pQlCKaWUV/8PQhtjQemiBssAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "time_grid = np.linspace(durations_test.min(), durations_test.max(), 100)\n",
    "ev.brier_score(time_grid).plot()\n",
    "plt.ylabel('Brier score')\n",
    "_ = plt.xlabel('Time')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Negative binomial log-likelihood\n",
    "\n",
    "In a similar manner, we can plot the the [IPCW negative binomial log-likelihood](https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291097-0258%2819990915/30%2918%3A17/18%3C2529%3A%3AAID-SIM274%3E3.0.CO%3B2-5)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXhU9bnA8e+bmWSyQkIIgYSwr5GdsCjiguJecBe11tYF27q0V7vQ5fpYu1m9trUtXkWr1x3XWqoIiisuIGE3hIQQloSQkITse2Z+948ZMMJkIclwZnk/z5MnM+f8ZvIeDpk3v12MMSillApdYVYHoJRSylqaCJRSKsRpIlBKqRCniUAppUKcJgKllApxdqsDOFH9+/c3w4YNszoMpZQKKBs3biwzxiR5OxdwiWDYsGFkZmZaHYZSSgUUEdnX3jltGlJKqRCniUAppUKcJgKllApxmgiUUirEaSJQSqkQp4lAKaVCnCYCpZQKcQE3j0Cp7nC6DHvK6sg+WM2hmiYun5pKQkyE1WEp5Rc0Eaig09zqYlthJV8dqCL7YA3ZxdXkFNfQ1Oo6Wubxj3fz8NWTmTva60RLpUKKJgIVFIqrGnlrWxGf5ZWxfs9h6pudACREhzN+UB++PXso4wf1YfygOFqchp+8upUb/vkl35szjJ9fMI7IcJvFV6CUdTQRqIBljOGL3eU8t24f7+4owekyjEiK4crpgzltZH+mDolnQJwDETnutW/deToPvLOTpz/by7+3FHHFtFQWzRzCyKRYC65EKWtJoG1VmZGRYXStodC1u7SWz/PK2LC3gg17D3OwqpH46HCuyUjj2plDGNY/5oTeb31+OU9/tpc12SW0ugwzhiWwYEoqF00YSGKsw0dXodTJJyIbjTEZXs9pIlCBYmtBJZf/7+c4XYbkPg4yhvVj3tgBXDxpUI+bdg7VNPLaxkJe31jI7tI6bGHCaSMTmTYkgbED4xiTHEtkuI2iykaKKhtoanVyyaQUYhxaqVaBQROBCgq3PZfJuvzD/Pv2OQxNjPba5NNTxhh2Ftfw1rYiVn1VTH5ZHe39iiT3cbDkwnEsnJxKWFjXY2l1usgpqeGrA1XsKatnX3kdhRUNpMRHMjktnimD45mcFq9JRvUqTQQq4OUdqmX+Xz7mjrNHcc95Y0/az21odrK7tJbckhqaW12kxEeREh9JeW0zv1+ZzbbCKqakxXPJpEEk94lkQJzDUyYKmyc5tDpdbDtQxdrcMr7cW86W/ZXUeTqzw21CWr9oUuOjKDhcz97yegAiw8OYnz6Qy6amcPqoJIoqG9haWMm2wioEGNg3kpT4KCanxZMaH3XS/j1U4NJEoALez17byoqtRXz283l+03bvchle31TI/7ybQ0l10zfORdjDGJYYTXKfSLYWVFLd2IoIjB/Yh4xhCUwfmsDkwfGk9Ys+mjAAKuub2VJQyfvZh3hrWxEV9S3YwgSny/17GhnungPa2OIeChvrsPPhT84iKc4//k2U/9JEoALawaoGznjwQ66dOYT7F06wOpzjGGOoamjhUE0Th6qbKKyoJ7+sjvzSWg5WNTIhpS9zx/Rnzsj+JzSJrbnVxSe5pazfU87IpFgmp8UzekAstjChsr6FHQerueGf61l8xkiWXDjOh1eogkFHiUAbIZXfe+rTPbgM3Dp3hNWheCUixEdHEB8dwZjkuF573wh7GOemJ3NuevJx5xJiIpgzqj+XTErhuS/2ctsZI3SmtOo2XWtI+ZWqhhae/WIvz36xl3X55ewrr+PF9fv51qRBpPWLtjo8v3P72aOoa3by9Od7rQ5FBTCf1ghE5ALgEcAGPGmMecBLmauB+wADbDXGXOfLmJT1Wpwu/vTOTrYdqCJjaAIzh/djYN9Iln9ZwCuZBUdnBbd125kjLYjU/40dGMcFpwzk6c/2cMvc4fSJDLc6JBWAfJYIRMQGLAXmA4XABhFZYYzZ0abMaOAXwBxjTIWIDPBVPMo/1DS28MMXNrF2VxnjBsbx+Cf5PPrRbsA9guZbk1K46fThJMU52FlcQ25xDXGRdsYP6mNx5P7rjnmjWJVVzHNf7OP2s0dZHY4KQL6sEcwE8owx+QAishxYCOxoU+ZWYKkxpgLAGHPIh/EoixVXNfK9/9tAbkkND14xiatnpFHX1Mrm/ZXsKa/jvPRkkvtEHi2f3CeSM8foonCdmZDal3njBvDk2nxGJsXQ1OqiscVJSnwUs4YnEmHXFmDVMV8mglSgoM3zQmDWMWXGAIjIZ7ibj+4zxqw69o1EZDGwGGDIkCE+CVb1zJ6yOl7eUMAbmwqJirCxcHIKC6emMiwxhsy9h1mdVcKKrQdoaHby1HdnHP2Aj3HYOX10f04f3d/iKwhsd84bxWWPfs73n9/0jeNxkXbmjRvA+acMZN64Abq4nvLKZ8NHReQq4HxjzC2e5zcAM40xd7Yp8xbQAlwNDAbWAhOMMZXtva8OH/UfDc1OVmUd5OUNBazLP4wtTDh7bBINLU4+312OMe5x7rVNrUTYwjh9dH9+ct5Y0lO0mccXdpfW0tjiJDLchsMeRk5xDauzilmTfYjDdc3EOuycf8pALp2awtQhCcTqzOWQYtXw0UIgrc3zwUCRlzLrjDEtwB4RyQFGAxt8GJfqhuKqRvIO1VLb1EpdUyub9lewYksRNU2tDOkXzU/PH8uV0wcfbdopqW7kP1uL2FVSy9wx/Tlr7AD94PGxY1dOHZwQzTnjk3G6DOvzy3lzywHe2V7M65sKAegTaSclPoqLJg7irnNGWxGy8hO+rBHYgVzgHOAA7g/364wxWW3KXABca4y5UUT6A5uBKcaY8vbeV2sEJ19RZQPz//zx0WURwD3D9aIJg7gqI41Zw/ud0Fo7yjqNLU7W7ipjd2ktRZUNbD9Qxeb9lay8a67W1IKcJTUCY0yriNwBrMbd/v+UMSZLRO4HMo0xKzznzhORHYAT+GlHSUBZ4+F3c2lxGZ7+3gwGxDmIddhJinMQHaF/4QeayHAb89OTmY97klpVfQunPfA+j328m79dO9Xi6JRVfPqbbIxZCaw85ti9bR4b4G7Pl/JDO4qqeWNzIYvnjuDssTq6N9j0jQ7n+tlDeXJtPj85byxDEnXSXijScWWqQ398J5u+UeH8UMenB62b5gzHFiY8sTbf6lCURTQRqHZ9klvK2l1l3HH2KPpG6YzVYDWwbySXTx3MK5kFlNU2df4CFXQ0ESivnC7DH1Zmk9YvihtOHWp1OMrHFp85gmani//7bK/VoSgLaCJQXi3fsJ+dxTX89PxxOOw6CSnYjUyK5bz0ZJ79Yi+1Ta1WhxOU9pXX8ceV2TS1Hr+WltU0EajjFFbU84e3szl1RCKXTBxkdTjqJPnhWaOobmxl2SfaV9Db8g7VcNVjX/D4J/l8daDa6nCOo4lAfYMxhiWvb8cAD145SecHhJDJni03l32ym6LKBqvDCRpZRVVc8/g6KhtaAPcudP5GE4H6hhe/3M+neWX88qLxuv5/CFpy4ThcBh5ctdPqUILC5v0VXLtsHRH2MJbdMB2AivoWi6M6niYCdVTBYXeT0JxRiVw/Sxf3C0WDE6K5de5w3txSxOb9FV7LHKxq4KUv9+sIo068m1XMtU+so290OK/cdirThiYAUFGnNQLlp1wuw89f3wbAn66YhIg2CYWqH5w1iqQ4B799awdHlqApqW7k1cwCrntiHac98AG/eGM71z+x3i8/1KxmjOGfn+7htuc3MjY5jjd+MIe0ftHEOezYw4QKP2wa0jUCFADPfrGXz3eX84fLJjI4QZuEQlmsw85PzxvLz17fxrVPrGNfeT0HqxoBGJoYzV3zRjMiKYafvraNG5/+khdumUWc7ox21G/fyuapz/Zw/inJ/PWaqURFuEfdHdnb2h+bhjQRKPJLa3lg1U7OGpvEtTPTOn+BCnpXTB/Mm1sOUFjRwIxh/ZiSFk/GsAQmpvY9WluMi7Sz+NmN3PR/G3jmppm69hTuEXdPfbaHRTPS+P1lE7EdM9giITrcL2tReudCXKvTxd2vbMVht2mTkDrKFia8eOvsDsvMG5fMI4umcudLm7j75a085ukMDWU5xTUAXJWRdlwSAEiIifDLpiHtIwhxj3+Sz5aCSu5feMo3tolUqisunjSIe84by6qsYtbn68LBuSW1AIxOjvV6PiE6XBOB8i95h2r565pcLp44iAWTU6wORwWom08fTnIfBw+uzsFX+5sEitySGlL6RtKnnT6TBD/tI9BEEMJ+//YOIu02frPwFG0SUt0WGW7jx+eOYeO+Ct7PPmR1OJbKKa5hzMC4ds8nxERQWd/sdwlTE0GI+ijnEB/mlHLnOaPoH+uwOhwV4K6aPpgR/WN4aHUOTpd/fcidLE6XIa+0ljHJHSSC6HBanMbv1nPSRBCCWpwufvd2NsMSo/nuacOtDkcFAbstjHvOG0tOSQ3/3nLA6nAssa+8juZWVyeJIAKASj9rHtJRQyHohXX7yDtUy7IbphNh178FVO+4cMJAJqb25U+rdvLprjIq6pupbGghpW8UU4fEM21oAqek9Ana1WxzS9wjhsa001EMXyeCw3XNfrWEiyaCEFNZ38xf1uxizqhE5qcnWx2OCiJhYcKvLh7P7S9sYv2ewyTEhNM3KpwtBZW8vf0gAMl9HPzzxhlMSO1rcbS9L7ekFhEYNaCDRBDj7kT2t5FDmghCzIOrc6hpbOG/L0nXDmLV62aPSGTjf88/7vih6kYy91Xw+7ezufrxL/jHdVOZN879h0iL08XaXaUMTojusFnF3+WU1DCkX3SHE+u0aUhZ7vPdZby4fj+3zh3OuIF9rA5HhZABfSK5aOIgMoYmcNMzG7jlmUx+dsE4KutbeG1jIWW1TQzsE8mae84k1hGYH0u7SmoYPaDjRNa2acifaANxiKhvbmXJ69sZlhjN3fPHWh2OClED+kTy8uJTOXvsAB54ZydPrM1n6pB47r0knZKaRv76Xq7VIXZLc6uL/NI6xg5sv1kIoE9UOGHif3sS+DT1isgFwCOADXjSGPPAMee/CzwEHBlm8A9jzJO+jClUPfxuLvsP17N88eyji2ApZYUYh53Hb5jOmuxDTBsSzwDPjPa80lqe/nwvl08bTHqKu8Za3djCox/u5srpgztse7fanrI6Wl2m06YtW5jQNyqcw36WCHxWIxARG7AUuBBIB64VkXQvRV82xkzxfGkS8IGN+yp46rM9fHv2EGaPSLQ6HKWw28K4YMLAo0kA4OfnjyM+Kpxfvbkdl8uwu7SWS5d+xmMf72bph3kWRtu5r0cMdd7H4Y+zi33ZNDQTyDPG5BtjmoHlwEIf/jzlRVVDCz99dSspfaNYcuF4q8NRql19o8P55UXj2by/kl/+azuX/uMzqupbmDEsgTU7Svxy0/cjcktqsIUJI5JiOi17ZHaxP/FlIkgFCto8L/QcO9YVIrJNRF4TEa9rIIvIYhHJFJHM0tJSX8QalFqcLm5/YRMFFfX8+erJAdsJp0LH5dNSmTW8H8s3FDAkMZoVd57OD88aRU1TK5/llVkdXrtyimsYlhjdpTkSCdHhHK4LnRqBt7GJx849/w8wzBgzCVgDPOPtjYwxy4wxGcaYjKSkpF4OMzgZY7j331l8mlfGHy6byCxtElIBQET4yzVT+MWF43jt+6eRGh/FaaMSiYu0s3J7sdXhtWvXoVrGdrDGUFsJ0aFVIygE2v6FPxgoalvAGFNujDmy8ekTgC5o3kv++ekeXvpyPz88ayRXZehmMypwpMRHcduZI48OanDYbcwfn8y7WcU0t7osju54jS1O9pbXdXkOREJMREgNH90AjBaR4SISASwCVrQtICKD2jxdAGT7MJ6Q8W5WMb9fmc1FEwfyk/N0qKgKfBdOHER1Yytf+OGeB3mHajGmax3FAPHR4TS1umho9p8+D58lAmNMK3AHsBr3B/wrxpgsEblfRBZ4it0lIlkishW4C/iur+IJFVsLKrlr+WYmDY7n4aumEOZllySlAs3c0f2JibDxjmepiiNandbXED7f7e676Goi6OeZVOZPy0z4tPfQGLMSWHnMsXvbPP4F8AtfxhBKCg7Xc/MzmfSPdfDkdzJ0voAKGpHhNs4Zn8zqrGJ+d+kEwkT42we73HMMMgZz9/wxJ3059QOVDdz/nyxWZ5UwIbUPwxK7tohcfJvZxSnxUb4Msct0GEmQqGpo4ab/20Bzq5Pli2eRFKd7DKjgctHEgazYWsSa7BJe23iANdklTB+awCsbCvjPliJunzeK780Z5vPVTZ0uw5Nr8/nrml0A/OyCsdxy+gjstq41sCREuxee6+l6Q61OF8s3FDBv3IAeJxRNBAGsqdXJ53nlrPqqmPeyS6hpbOGZm2YyqpP1TpQKRGeOGUBUuI0fvrCJMBF+s+AUvnPqUPLL6vjjymweeGcnH2Qf4pmbZvqsNry3rI57Xt3Kxn0VzE9P5r4Fp5B6gh/C/WJ63jTU2OLkrpc28+6OEhZMTuFv107t9nuBJoKAlbn3MN9/fhNltU3EOezMGz+Aa2fqzGEVvKIibHxr8iA+2HmIpddNOzokemRSLE/eOIM3Nx/g7le2sPi5TJ68MaNXawbGGF5Yv5/fv51NuE14ZNEUFkxO6dYKvvE97COoaWzh1mczWZd/mHED41iVVUxVfQt9o73vk9wVmggC0JubD/Cz17YxOCGKh66cxGmjEoN2sw+l2vr9ZRMBCPfSDHPp1FSaW1387PVt3PXSZpZeNw27LYyq+hb2ltcxflCfbm/E9Of3cvn7B3nMHd2fB6+cxKC+3W+Kifd8YFd0Y1JZeW0T33nqS3KKa3hk0RRGJsVyyd8/5c0tB7jxtGHdjkkTQQAxxvDXNbt45P1dzB7Rj8e+Pf3oXxdKhQJvCaCtq2ekUdfcym/+s4PLHv2c6sYW9pXXA+4NY3536YQTrjU/uTafv3+Qx6IZafzx8ok93scj3BZGXKT9hGsEjS1OFj+3kbxDtTxxYwZnjx0AwCkpfXh5Q0GPEoEuQx0gXC7DL/+1nUfe38VV0wfz7E2zNAko5cX35gzn1xePp6HFSfqgPvzsgrE8eMUkmlqdLFq2jrtf2UJ5bVPnbwS8sqGA372dzcUTB/H7y3qeBI5wLzzX9URgjOGXb2xn474K/nLNlKNJAOCaGWnsOFjNVwequh2P1ggCgNNlWPL6Nl7dWMgdZ4/invPG6O5iSnXglrkjuGXuiG8c+9bkFP7x4S6WfZJPbkkNK24/vcN5Nv/ecoAlb2xj7uj+/OWaKdh6cU5OQsyJrUD66Ee7eWPzAe6ZP4aLJg76xrmFk1P53dvZvLyhoNtbgGqNwM85XYafvrqVVzcW8uNzR/OT88dqElCqG6IibPz0/HH88fJJfHWgmtVZ3tcucroMf1q1kx8t30LGsH48fsP0bvcttCchOpyKLi4zseqrgzy0OocFk1O4Y96o4873jQ7nwgkDeXPLARpbujdbWROBn/v1m9uP/iXw43PHWB2OUgHvsqmpjEiK4S9rcnG6vrkOZlW9ez7O/360m+tmDeH5m2d1uAdxd/XrYtPQ2l2l3LV8C1PS4nnwyknt/hF4TUYaNY2trPqqewvzadOQH/tidzkvfVnAbWeM4M5zRlsdjlJBwRYm/PjcMdz10mbe2lbEwinu1fGLKhu4/sn1FFbU84fLJnLdrCE+iyE+OqLTGsEXu8u59dlMRvSP4envziAyvP2RgbNHJJLWL4r739rBk5/mAxATYWfp9dO6NONaawR+qtXp4jf/ySI1Por/mq81AaV60yUTBzE2OY5H1uyi1eniQGUDi5ato6ymiRdvne3TJADupqG6Zme7q6lm7j3Mzc9sIC0hmhdumUVCTMcDQ8LChF9fnM7UtHiS4yJx2G2s33OY3OKaLsWjNQI/9fy6fewsruGxb0/r8C8BpdSJCwsT/mv+aL7//CYe/Wg3r2QWUNXQwnO3zGJKWrzPf/6RD/bK+uZvbNcJsL2wiu8+vYGBfSJ54dZZJHZxDaXzTxnI+acMBGDz/goue/Rzmrq4KJ/WCPxQeW0Tf34vl9NH9T96Y5VSvev8UwZySkof/vxeLtUNLbxwkpIAuIePAsdtYr+7tJYbn/6SvlHhvHDrLAbERXp7eaeOTDBtatFEELAeWp1DfbOT+xak6wghpXxExN2cckpKH168dTaTBp+cJACQEHP87OKiygZueHI9Ajx/y6wezV4+MsqpuYs1Am0a8jOvbyzk5cwCbp4zXBePU8rHTh2ZyNt3zT3pPzfhmPWGDtc1852nvqSmsZWXFs9meP+YHr2/w5MImro4nFQTgZ8wxvDoR7t5aHUOc0YlagexUkGsbSL4OLeUJa9vo7yumWdvmtntSWFtObRGEHicLsO9//6KF9bv59IpKTx45eRen8CilPIfRxaee+zj3RQcbmD0gFge+/Z0JvdSH8XRpqEu7vGsicAixhi2H6ji7e0HWbn9IAWHG/j+mSP52fljdXtJpYJcZLiNOIedAxXu3/sfnzu6V0cHHkkETZoI/JcxhkXL1rF+z2HsYcKcUf35xYXjj1tDRCkVvJZeP42E6AgmDu55U9CxImxaI/B7WUXVrN9zmNvOGMEPzhqpq4gqFYLOGJPks/e228KwhQlNrV3rLNaGaAu8t6MEEVh8xghNAkopn3DYw7pcI9BEYIE12SVMH5LQ5RmDSil1oiL8JRGIyAUikiMieSKypINyV4qIEZEMX8bjDw5UNpBVVM389GSrQ1FKBbEIW1iXO4t9lghExAYsBS4E0oFrRSTdS7k44C5gva9i8SfvZ5cAcK4mAqWUDznC/aNGMBPIM8bkG2OageXAQi/lfgs8CDT6MBa/8d6OEkYkxTAyKdbqUJRSQcwvagRAKlDQ5nmh59hRIjIVSDPGvNXRG4nIYhHJFJHM0tLS3o/0JKlubGFdfjnzx2ttQCnlWw67zS8SgbdZUUe3AxKRMOAvwD2dvZExZpkxJsMYk5GU5LshV772cU4pLU6j/QNKKZ+LsId1eYkJXyaCQiCtzfPBQFGb53HABOAjEdkLzAZWBHOH8ZrsEhJjIpg6JMHqUJRSQS7CHtblRed8mQg2AKNFZLiIRACLgBVHThpjqowx/Y0xw4wxw4B1wAJjTKYPY7JMi9PFhzsPMW/cAGy6hIRSyscc/lAjMMa0AncAq4Fs4BVjTJaI3C8iC3z1c/3V+vzDVDe2arOQUuqkcNjDurwxjU+XmDDGrARWHnPs3nbKnuXLWKz2xqZC4hx2n04rV0qpIxx2m/U1AvW16sYWVn51kAVTUnT/YaXUSeE3M4uV21tbD9LY4uKqjLTOCyulVC9wzyOwvrNYeby6sYAxybFM9sFys0op5Y2/zCxWwK6SGjbvr+TqjDTdiF4pddL4y8xiBby6sRB7mHDp1NTOCyulVC/RGoGfaHG6eGNTIfPGDaC/LjmtlDqJImw2Wl0Gp8t0WlYTgQ99uPMQZbXNXK2dxEqpk+xENrDvdiIQkSu6+9pQ0Nji5NGPdtM/1sFZY3XugFLq5HKcjESAe8E45YXTZfjx8i1sKajkvgXp2G1a8VJKnVxHagRNzs6HkPbkE0qHwHhhjOG+FVmsyirmvy9J55JJKVaHpJQKQUdqBF1ZZqIniaDzHogQ9OhHu3lu3T5uO2MEN58+3OpwlFIh6mgfQReWmehwrSER2Y73D3wBBnYjtqDV6nTxwDs7efLTPVw2NZWfXzDO6pCUUiHsRGoEnS06d0kvxBP0qupbuOOlTazdVcaNpw7l15ekE6ZLTSulLOSwu9c163GNwBizr71zIvIZMOcEYws6BYfr+c5TX1JYUc8Dl09k0cwhVoeklFInNHy0J8tQ6yce8NjHuzlY1cBLt84mY1g/q8NRSimgzaihLiw8p53FPWCM4f3sQ5w9doAmAaWUXzmReQSddRZf3t4pIOpEAws2WUXVFFc3cs543XVMKeVfvq4R9Lxp6FsdnHur6yEFp/d2lBAmcLbOHFZK+ZmjncU9TQTGmO/1TkjBaU12CdOHJpCoC8oppfxMr641JCJnisgkz+OrReQfIvJfIhLSn35FlQ1kFVVzrjYLKaX8UISt653FnfURLAUmAZEikgPEAquA04CngOt7GGvAej+7BIBz0zURKKX8jyO89/oIzjbGpItIJHAAGGCMcYrI48C2ngYayN7LPsSI/jGMTIq1OhSllDrO1zWCnjcNNQIYYxqBfcYYp+e5AVo6e3MRuUBEckQkT0SWeDn/fRHZLiJbRORTEUnvNGI/UNvUyrrd5VobUEr5rV4bPgoMEJG7cQ8XPfIYz/MOh8qIiA1YCswHCoENIrLCGLOjTbEXjTGPecovAP4MXNBp1BZbm1tKs9Ol/QNKKb8lIkTYwnq+xATwBBDn5THAk528diaQZ4zJ9wS1HFgIHE0ExpjqNuVjCJBJau9llxAfHc60IfFWh6KUUu2KsIf1fNE5Y8xvehBDKlDQ5nkhMOvYQiJyO3A3EAHM68HPOym2FlTy1raDXDYlVTecUUr5NYc9jOYubEzT2aihezs4bYwxv+3o5d5e4+VNlgJLReQ64NfAjV7iWAwsBhgyxLoljspqm/jB8xtJinWw5EJdZlop5d+6WiPo7E/aOi9fADcDP+/ktYVA213bBwNFHZRfDlzq7YQxZpkxJsMYk5GUZM0s3laniztf3Ex5XTOP3zCdhJgIS+JQSqmuctcIet409PCRxyISB/wI+B7uD+2H23udxwZgtIgMxz30dBFwXdsCIjLaGLPL8/RiYBd+6k+rdvJFfjkPXzWZCal9rQ5HKaU6FWEP651lqEWkH+42/OuBZ4BpxpiKzl5njGkVkTuA1YANeMoYkyUi9wOZxpgVwB0ici7uoagVeGkW8gfvZ5fwxNo93HjqUK6YPtjqcJRSqksi7GE9n1AmIg8BlwPLgInGmNoTCcIYsxJYecyxe9s8/tGJvJ8Vqhtb+NW/vmJschy/ujggpjkopRTgXniuN9YaugdIwd2JWyQi1Z6vGhGp7uS1QeGPK7M5VNPIg1dOOrqIk1JKBYIIW1jP1xoyxoT0J99neWW89GUBt50xgslpOmdAKRVYHOFh1Ne1dloupD/oO9MCvnsAAA8DSURBVFLf3MqSN7YxvH8M/zV/jNXhKKXUCXPXCHy7Z3FQ+9v7eRQcbuDlxbOJDLdZHY5SSp2wro4a0hqBF9WNLTy/bh8LJqcwa0Si1eEopVS3OOy2Xll9NCQt/3I/tU2tLD5jhNWhKKVUt0V0cUKZJoJjtDhdPP3ZXk4dkagTx5RSAc1hD6OppfNRQ5oIjvH2toMcrGrU2oBSKuB1dYkJTQRtGGNY9kk+owbEcuYYa9Y0Ukqp3nJkZrF7L7H2aSJo4/Pd5ew4WM2tc4cTFuZt8VSllAocDnsYxkCrSxNBlz2xNp/+sREsnJJqdShKKdVjEV3crlITgUdNYwsf5ZSyaMYQnTeglAoKXd3AXhOBR26Jez29KbqUhFIqSDg8f9RqjaCLcktqABg7MK6TkkopFRi+rhF0PIRUE4FHTnEN0RE2UuOjrA5FKaV6hSNc+whOSG5JDWOS43S0kFIqaGgfwQnKKa5hbLI2CymlgseRUUOaCLqgrLaJ8rpmxmj/gFIqiDjs2lncZbnFno5irREopYLI1zUC7SzuVI5nxNCYgbEWR6KUUr3HoRPKui6nuIZ+MREkxTqsDkUppXrN0UTQycJzmghw1wjGJMcioiOGlFLB42jTUIsmgg4ZY8jVEUNKqSB0tLPYyhqBiFwgIjkikiciS7ycv1tEdojINhF5X0SG+jIebw5UNlDX7NQRQ0qpoPN1jcCizmIRsQFLgQuBdOBaEUk/pthmIMMYMwl4DXjQV/G0J0dHDCmlgpQ/9BHMBPKMMfnGmGZgObCwbQFjzIfGmHrP03XAYB/G49XXI4Y0ESilgos/LEOdChS0eV7oOdaem4F3vJ0QkcUikikimaWlpb0YonsOQUrfSPpEhvfq+yqllNXsYYKItTOLvQ3B8bpNjoh8G8gAHvJ23hizzBiTYYzJSErq3S0kc0pqtTaglApKIuLet9jCRFAIpLV5PhgoOraQiJwL/ApYYIxp8mE8x2l1uth9qFb7B5RSQSvCFmZpjWADMFpEhotIBLAIWNG2gIhMBR7HnQQO+TAWr/aW19HsdOkeBEqpoOUIt1mXCIwxrcAdwGogG3jFGJMlIveLyAJPsYeAWOBVEdkiIivaeTuf2OkZMTRGawRKqSAVYeu8acjuywCMMSuBlcccu7fN43N9+fM7sy6/nOgImyYCpVTQctjDdNG59hhj+Di3lNNGJh4dYqWUUsEmwuLOYr+2t7yegsMNnDGmd0chKaWUP3HXCDQRePVJrns+whmjNREopYKXw27TGkF7PsktZUi/aIb1j7E6FKWU8pkIe5guQ+1Nc6uLL/LLOWNMf6tDUUopn4rQzmLvMvcdpr7Zqc1CSqmgZ/XMYr/1SW4Z9jDhtFFaI1BKBTcdNdSOT3JLmT40gViHT6dRKKWU5axeYsIvldY0seNgtQ4bVUqFBEe41giOs3aXe9jomZoIlFIhIMJm4VpD/uqT3FISYyJIH9TH6lCUUsrntEbgxeaCSmaN6EdYmLftEpRSKrhE2HQewTfUNbWyr7ye8QO1NqCUCg1dWUstpBLBkf2Jx2mzkFIqRDg0EXxT9sFqAMbpRjRKqRChieAYOw/WEOewMzghyupQlFLqpHDYbZ2WCa1EUFzNuEFxiGhHsVIqNGgfQRvGGHYerGGcdhQrpUKIJoI2DlQ2UNPUyrhB2j+glAod2kfQRvZB94ih8TpiSCkVQrRG0MZOz4ihsbpRvVIqhGhncRs7i2sYmhhNjK44qpQKIZbXCETkAhHJEZE8EVni5fwZIrJJRFpF5EpfxpJdXK3zB5RSISfCZmEiEBEbsBS4EEgHrhWR9GOK7Qe+C7zoqzgAGpqd7C2r0xFDSqmQ4wjv/GPel+0kM4E8Y0w+gIgsBxYCO44UMMbs9ZzreEWkHsotqcFltKNYKRV6LK0RAKlAQZvnhZ5jJ0xEFotIpohklpaWnvDrdxa7O4rH69BRpVSI6UqNwJeJwNv0XdOdNzLGLDPGZBhjMpKSTnxDmeyDNURH2EhLiO7Oj1dKqYDlsFk7aqgQSGvzfDBQ5MOf166dxdWMHRinexAopUKO1aOGNgCjRWS4iEQAi4AVPvx5Xhlj2Flco/0DSqmQZGkiMMa0AncAq4Fs4BVjTJaI3C8iCwBEZIaIFAJXAY+LSFZvx7G3vJ7K+hYdOqqUCkm2MMHeSWuIT2dXGWNWAiuPOXZvm8cbcDcZ+czfP9hFhD2M+enJvvwxSinltzpbbyioZxbnFNfwr80H+O5pwxjUV/cgUEqFps6ah4I6ETy0eiexDjs/PGuk1aEopZRlQjYRZO49zJrsQ3z/zJHER0dYHY5SSlmms1UVgjIRGGP406qdJMU5+N6cYVaHo5RSlnrmppkdng+6RGCMYfmGAjbsreBH54wmOkJXG1VKqY4E1adk9sFq7v/PDr7IL2dKWjzXzEjr/EVKKRXigiYR/Pm9XP7xwS76RoXz20sncO2MNOxdWGxJKaVCXVAkgrqmVpZ+mMe8ccn8z1WTtHNYKaVOQFD8yby1oBKny/Dt2UM0CSil1AkKikSwcV8FAFOHJFgciVJKBZ6gSASZ+yoYkxxL36hwq0NRSqmAE/CJwOUybNpfwfSh/awORSmlAlLAJ4K80lpqGluZPlSbhZRSqjsCPhFk7nX3D2giUEqp7gn4RLBxXwWJMREMS9RtKJVSqjsCPhFs2l/BtKEJiOg2lEop1R0BnQjKapvYU1ZHhjYLKaVUtwV0Iti0T/sHlFKqpwI6EWzcX0GELYwJqX2tDkUppQJWYCeCvRVMSO1DZLjN6lCUUipgBWwiaGp1su1AlTYLKaVUDwXc6qO5JTWc8/BHtLoMza0uTQRKKdVDPq0RiMgFIpIjInkissTLeYeIvOw5v15EhnX2npHhNsYN7MOElL5cnTGYuaOTfBG6UkqFDJ/VCETEBiwF5gOFwAYRWWGM2dGm2M1AhTFmlIgsAv4EXNPR+w7pF83S66f5KmyllAo5vqwRzATyjDH5xphmYDmw8JgyC4FnPI9fA84RnRmmlFInlS8TQSpQ0OZ5oeeY1zLGmFagCkg89o1EZLGIZIpIZmlpqY/CVUqp0OTLRODtL3vTjTIYY5YZYzKMMRlJSdonoJRSvcmXiaAQSGvzfDBQ1F4ZEbEDfYHDPoxJKaXUMXyZCDYAo0VkuIhEAIuAFceUWQHc6Hl8JfCBMea4GoFSSinf8dmoIWNMq4jcAawGbMBTxpgsEbkfyDTGrAD+CTwnInm4awKLfBWPUkop73w6ocwYsxJYecyxe9s8bgSu8mUMSimlOhawS0wopZTqHRJoTfIiUgPkWB1HL+kPlFkdRC8IlusAvRZ/FCzXAdZey1BjjNdhlwG31hCQY4zJsDqI3iAimcFwLcFyHaDX4o+C5TrAf69Fm4aUUirEaSJQSqkQF4iJYJnVAfSiYLmWYLkO0GvxR8FyHeCn1xJwncVKKaV6VyDWCJRSSvUiTQRKKRXiAioRdLbjmT8Tkb0isl1EtohIpudYPxF5T0R2eb775b6bIvKUiBwSka/aHPMau7j9zXOPtomIX+0i1M613CciBzz3ZouIXNTm3C8815IjIudbE/XxRCRNRD4UkWwRyRKRH3mOB9x96eBaAuq+iEikiHwpIls91/Ebz/Hhnh0Yd3l2ZIzwHD/hHRp9xhgTEF+41yvaDYwAIoCtQLrVcZ1A/HuB/sccexBY4nm8BPiT1XG2E/sZwDTgq85iBy4C3sG9xPhsYL3V8XfhWu4DfuKlbLrn/5kDGO75/2ez+ho8sQ0CpnkexwG5nngD7r50cC0BdV88/7axnsfhwHrPv/UrwCLP8ceAH3ge/xB4zPN4EfCyVbEHUo2gKzueBZq2O7Q9A1xqYSztMsZ8wvHLg7cX+0LgWeO2DogXkUEnJ9LOtXMt7VkILDfGNBlj9gB5uP8fWs4Yc9AYs8nzuAbIxr3RU8Ddlw6upT1+eV88/7a1nqfhni8DzMO9AyMcf0/8YofGQEoEXdnxzJ8Z4F0R2Sgiiz3Hko0xB8H9ywAMsCy6E9de7IF6n+7wNJk81aaJLiCuxdOkMBX3X6ABfV+OuRYIsPsiIjYR2QIcAt7DXVupNO4dGOGbsXZph8aTIZASQZd2M/Njc4wx04ALgdtF5AyrA/KRQLxP/wuMBKYAB4GHPcf9/lpEJBZ4HfixMaa6o6Jejvn7tQTcfTHGOI0xU3BvxDUTGO+tmOe731xHICWCrux45reMMUWe74eAf+H+T1JypHru+X7IughPWHuxB9x9MsaUeH6BXcATfN3M4NfXIiLhuD84XzDGvOE5HJD3xdu1BOp9ATDGVAIf4e4jiBf3DozwzVj9ZofGQEoEXdnxzC+JSIyIxB15DJwHfMU3d2i7Efi3NRF2S3uxrwC+4xmlMhuoOtJU4a+OaSu/DPe9Afe1LPKM7hgOjAa+PNnxeeNpS/4nkG2M+XObUwF3X9q7lkC7LyKSJCLxnsdRwLm4+zs+xL0DIxx/T/xjh0are9pP5Av3yIdc3O1uv7I6nhOIewTuUQ5bgawjseNuD3wf2OX53s/qWNuJ/yXcVfMW3H/F3Nxe7Liru0s992g7kGF1/F24luc8sW7D/cs5qE35X3muJQe40Or428R1Ou5mhG3AFs/XRYF4Xzq4loC6L8AkYLMn3q+Aez3HR+BOVHnAq4DDczzS8zzPc36EVbHrEhNKKRXiAqlpSCmllA9oIlBKqRCniUAppUKcJgKllApxmgiUUirEBeLm9UqdFCJyZCgmwEDACZR6ntcbY06zJDClepkOH1WqC0TkPqDWGPM/VseiVG/TpiGlukFEaj3fzxKRj0XkFRHJFZEHROR6z7r020VkpKdckoi8LiIbPF9zrL0Cpb6miUCpnpsM/AiYCNwAjDHGzASeBO70lHkE+IsxZgZwheecUn5B+wiU6rkNxrNuj4jsBt71HN8OnO15fC6Q3ma5+T4iEmfc6+8rZSlNBEr1XFObx642z118/TsWBpxqjGk4mYEp1RXaNKTUyfEucMeRJyIyxcJYlPoGTQRKnRx3ARme3bZ2AN+3OiCljtDho0opFeK0RqCUUiFOE4FSSoU4TQRKKRXiNBEopVSI00SglFIhThOBUkqFOE0ESikV4v4fSAjfxj1Vl1MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ev.nbll(time_grid).plot()\n",
    "plt.ylabel('NBLL')\n",
    "_ = plt.xlabel('Time')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Integrated scores\n",
    "\n",
    "The two time-dependent scores above can be integrated over time to produce a single score [Graf et al. 1999](https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291097-0258%2819990915/30%2918%3A17/18%3C2529%3A%3AAID-SIM274%3E3.0.CO%3B2-5). In practice this is done by numerical integration over a defined `time_grid`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.16715497913570207"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ev.integrated_brier_score(time_grid) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4933129163806694"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ev.integrated_nbll(time_grid) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}