{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction to the pycox package\n", "\n", "\n", "In this notebook we introduce the use of `pycox` through an example dataset.\n", "We illustrate the procedure with the `LogisticHazard` method ([paper_link](https://arxiv.org/abs/1910.06724)), but we this can easily be replaced by for example `PMF`, `MTLR` or `DeepHitSingle`.\n", "\n", "In the following we will:\n", "\n", "- Load the METABRIC survival dataset.\n", "- Process the event labels so the they work with our methods.\n", "- Create a [PyTorch](https://pytorch.org) neural network.\n", "- Fit the model.\n", "- Evaluate the predictive performance using the concordance, Brier score, and negative binomial log-likelihood.\n", "\n", "While some knowledge of the [PyTorch](https://pytorch.org) framework is preferable, it is not required for the use of simple neural networks.\n", "For building more advanced network architectures, however, we would recommend looking at [the PyTorch tutorials](https://pytorch.org/tutorials/)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Imports\n", "\n", "You need `sklearn-pandas` which can be installed by uncommenting the following block" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# ! pip install sklearn-pandas" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "# For preprocessing\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn_pandas import DataFrameMapper " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`pycox` is built on top of [PyTorch](https://pytorch.org) and [torchtuples](https://github.com/havakv/torchtuples), where the latter is just a simple way of training neural nets with less boilerplate code." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import torch # For building the networks \n", "import torchtuples as tt # Some useful functions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We import the `metabric` dataset, the `LogisticHazard` method ([paper_link](https://arxiv.org/abs/1910.06724)) also known as [Nnet-survival](https://peerj.com/articles/6257/), and `EvalSurv` which simplifies the evaluation procedure at the end.\n", "\n", "You can alternatively replace `LogisticHazard` with, for example, `PMF` or `DeepHitSingle`, which should both work in this notebook." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from pycox.datasets import metabric\n", "from pycox.models import LogisticHazard\n", "# from pycox.models import PMF\n", "# from pycox.models import DeepHitSingle\n", "from pycox.evaluation import EvalSurv" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# We also set some seeds to make this reproducable.\n", "# Note that on gpu, there is still some randomness.\n", "np.random.seed(1234)\n", "_ = torch.manual_seed(123)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Dataset\n", "\n", "We load the METABRIC data set as a pandas DataFrame and split the data in in train, test and validation.\n", "\n", "The `duration` column gives the observed times and the `event` column contains indicators of whether the observation is an event (1) or a censored observation (0)." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "df_train = metabric.read_df()\n", "df_test = df_train.sample(frac=0.2)\n", "df_train = df_train.drop(df_test.index)\n", "df_val = df_train.sample(frac=0.2)\n", "df_train = df_train.drop(df_val.index)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
x0x1x2x3x4x5x6x7x8durationevent
05.6038347.81139210.7979885.9676071.01.00.01.056.84000099.3333360
15.2848829.58104310.2046205.6649701.00.00.01.085.94000295.7333301
36.6540175.3418468.6463795.6558880.00.00.00.066.910004239.3000030
45.4567475.33974110.5557246.0084291.00.00.01.067.84999856.9333341
55.4258266.33118210.4551455.7490531.01.00.01.070.519997123.5333330
\n", "
" ], "text/plain": [ " x0 x1 x2 x3 x4 x5 x6 x7 x8 \\\n", "0 5.603834 7.811392 10.797988 5.967607 1.0 1.0 0.0 1.0 56.840000 \n", "1 5.284882 9.581043 10.204620 5.664970 1.0 0.0 0.0 1.0 85.940002 \n", "3 6.654017 5.341846 8.646379 5.655888 0.0 0.0 0.0 0.0 66.910004 \n", "4 5.456747 5.339741 10.555724 6.008429 1.0 0.0 0.0 1.0 67.849998 \n", "5 5.425826 6.331182 10.455145 5.749053 1.0 1.0 0.0 1.0 70.519997 \n", "\n", " duration event \n", "0 99.333336 0 \n", "1 95.733330 1 \n", "3 239.300003 0 \n", "4 56.933334 1 \n", "5 123.533333 0 " ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_train.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Feature transforms\n", "\n", "The METABRIC dataset has 9 covariates: `x0, ..., x8`.\n", "We will standardize the 5 numerical covariates, and leave the binary covariates as is.\n", "Note that PyTorch require variables of type `'float32'`.\n", "\n", "Here we use the `sklearn_pandas.DataFrameMapper` to make feature mappers, but this has nothing to do the the `pycox` package." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "cols_standardize = ['x0', 'x1', 'x2', 'x3', 'x8']\n", "cols_leave = ['x4', 'x5', 'x6', 'x7']\n", "\n", "standardize = [([col], StandardScaler()) for col in cols_standardize]\n", "leave = [(col, None) for col in cols_leave]\n", "\n", "x_mapper = DataFrameMapper(standardize + leave)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "x_train = x_mapper.fit_transform(df_train).astype('float32')\n", "x_val = x_mapper.transform(df_val).astype('float32')\n", "x_test = x_mapper.transform(df_test).astype('float32')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Label transforms\n", "\n", "The survival methods require individual label transforms, so we have included a proposed `label_transform` for each method.\n", "For most of them the `label_transform` is just a shorthand for the class `pycox.preprocessing.label_transforms.LabTransDiscreteTime`.\n", "\n", "The `LogisticHazard` is a discrete-time method, meaning it requires discretization of the event times to be applied to continuous-time data.\n", "We let `num_durations` define the size of this (equidistant) discretization grid, meaning our network will have `num_durations` output nodes." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "num_durations = 10\n", "\n", "labtrans = LogisticHazard.label_transform(num_durations)\n", "# labtrans = PMF.label_transform(num_durations)\n", "# labtrans = DeepHitSingle.label_transform(num_durations)\n", "\n", "get_target = lambda df: (df['duration'].values, df['event'].values)\n", "y_train = labtrans.fit_transform(*get_target(df_train))\n", "y_val = labtrans.transform(*get_target(df_val))\n", "\n", "train = (x_train, y_train)\n", "val = (x_val, y_val)\n", "\n", "# We don't need to transform the test labels\n", "durations_test, events_test = get_target(df_test)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "pycox.preprocessing.label_transforms.LabTransDiscreteTime" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(labtrans)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `labtrans.cuts` contains the discretization grid. This will later be used to obtain the right time-scale for survival predictions." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 0. , 39.466667, 78.933334, 118.4 , 157.86667 ,\n", " 197.33334 , 236.8 , 276.26666 , 315.73334 , 355.2 ],\n", " dtype=float32)" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "labtrans.cuts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, `y_train` is a tuple with the indices of the discretized times, in addition to event indicators." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array([2, 3, 6, ..., 1, 5, 3]),\n", " array([0., 1., 0., ..., 1., 0., 0.], dtype=float32))" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_train" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 78.933334, 118.4 , 236.8 , ..., 39.466667, 197.33334 ,\n", " 118.4 ], dtype=float32)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "labtrans.cuts[y_train[0]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Neural net\n", "\n", "We make a neural net with `torch`.\n", "For simple network structures, we can use the `MLPVanilla` provided by `torchtuples`.\n", "For building more advanced network architectures, see for example [the tutorials by PyTroch](https://pytorch.org/tutorials/).\n", "\n", "The following net is an MLP with two hidden layers (with 32 nodes each), ReLU activations, and `out_features` output nodes.\n", "We also have batch normalization and dropout between the layers." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "in_features = x_train.shape[1]\n", "num_nodes = [32, 32]\n", "out_features = labtrans.out_features\n", "batch_norm = True\n", "dropout = 0.1\n", "\n", "net = tt.practical.MLPVanilla(in_features, num_nodes, out_features, batch_norm, dropout)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you instead want to build this network with `torch` you can uncomment the following code.\n", "It is essentially equivalent to the `MLPVanilla`, but without the `torch.nn.init.kaiming_normal_` weight initialization." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "# net = torch.nn.Sequential(\n", "# torch.nn.Linear(in_features, 32),\n", "# torch.nn.ReLU(),\n", "# torch.nn.BatchNorm1d(32),\n", "# torch.nn.Dropout(0.1),\n", " \n", "# torch.nn.Linear(32, 32),\n", "# torch.nn.ReLU(),\n", "# torch.nn.BatchNorm1d(32),\n", "# torch.nn.Dropout(0.1),\n", " \n", "# torch.nn.Linear(32, out_features)\n", "# )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Training the model\n", "\n", "To train the model we need to define an optimizer. You can choose any `torch.optim` optimizer, or use one from `tt.optim`.\n", "The latter is built on top of the `torch` optimizers, but with some added functionality (such as not requiring `net.parameters()` as input and the `model.lr_finder` for finding suitable learning rates).\n", "We will here use the `Adam` optimizer with learning rate 0.01.\n", "\n", "We also set `duration_index` which connects the output nodes of the network the the discretization times. This is only useful for prediction and does not affect the training procedure.\n", "\n", "The `LogisticHazard` can also take the argument `device` which can be use to choose between running on the CPU and GPU. The default behavior is to run on a GPU if it is available, and CPU if not.\n", "See `?LogisticHazard` for details." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "model = LogisticHazard(net, tt.optim.Adam(0.01), duration_index=labtrans.cuts)\n", "# model = PMF(net, tt.optim.Adam(0.01), duration_index=labtrans.cuts)\n", "# model = DeepHitSingle(net, tt.optim.Adam(0.01), duration_index=labtrans.cuts)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we set the `batch_size` and the number of training `epochs`.\n", "\n", "We also include the `EarlyStopping` callback to stop training when the validation loss stops improving." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "batch_size = 256\n", "epochs = 100\n", "callbacks = [tt.cb.EarlyStopping()]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now train the network and the `log` object keeps track of the training progress." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0:\t[0s / 0s],\t\ttrain_loss: 2.9525,\tval_loss: 2.7795\n", "1:\t[0s / 0s],\t\ttrain_loss: 2.6598,\tval_loss: 2.5419\n", "2:\t[0s / 0s],\t\ttrain_loss: 2.4027,\tval_loss: 2.2536\n", "3:\t[0s / 0s],\t\ttrain_loss: 2.0897,\tval_loss: 1.9271\n", "4:\t[0s / 0s],\t\ttrain_loss: 1.7825,\tval_loss: 1.6382\n", "5:\t[0s / 0s],\t\ttrain_loss: 1.5542,\tval_loss: 1.4681\n", "6:\t[0s / 0s],\t\ttrain_loss: 1.4341,\tval_loss: 1.4088\n", "7:\t[0s / 0s],\t\ttrain_loss: 1.4004,\tval_loss: 1.3898\n", "8:\t[0s / 0s],\t\ttrain_loss: 1.4069,\tval_loss: 1.3798\n", "9:\t[0s / 0s],\t\ttrain_loss: 1.3830,\tval_loss: 1.3746\n", "10:\t[0s / 0s],\t\ttrain_loss: 1.3433,\tval_loss: 1.3715\n", "11:\t[0s / 0s],\t\ttrain_loss: 1.3466,\tval_loss: 1.3710\n", "12:\t[0s / 0s],\t\ttrain_loss: 1.3187,\tval_loss: 1.3682\n", "13:\t[0s / 0s],\t\ttrain_loss: 1.3268,\tval_loss: 1.3692\n", "14:\t[0s / 0s],\t\ttrain_loss: 1.3174,\tval_loss: 1.3669\n", "15:\t[0s / 0s],\t\ttrain_loss: 1.3167,\tval_loss: 1.3642\n", "16:\t[0s / 0s],\t\ttrain_loss: 1.3138,\tval_loss: 1.3642\n", "17:\t[0s / 0s],\t\ttrain_loss: 1.3035,\tval_loss: 1.3680\n", "18:\t[0s / 0s],\t\ttrain_loss: 1.2987,\tval_loss: 1.3694\n", "19:\t[0s / 1s],\t\ttrain_loss: 1.2896,\tval_loss: 1.3761\n", "20:\t[0s / 1s],\t\ttrain_loss: 1.2905,\tval_loss: 1.3760\n", "21:\t[0s / 1s],\t\ttrain_loss: 1.3070,\tval_loss: 1.3732\n", "22:\t[0s / 1s],\t\ttrain_loss: 1.2847,\tval_loss: 1.3733\n", "23:\t[0s / 1s],\t\ttrain_loss: 1.2907,\tval_loss: 1.3821\n", "24:\t[0s / 1s],\t\ttrain_loss: 1.2582,\tval_loss: 1.3905\n", "25:\t[0s / 1s],\t\ttrain_loss: 1.2672,\tval_loss: 1.3977\n", "26:\t[0s / 1s],\t\ttrain_loss: 1.2636,\tval_loss: 1.3996\n" ] } ], "source": [ "log = model.fit(x_train, y_train, batch_size, epochs, callbacks, val_data=val)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xc1Znw8d8zRV1Wl2xJ7gZ3W7KFsTEhGBJaILYJxYSasl4CIWQ/IQvZN5WFfbObvEmWhLJkISSEUAIYO6EXgwM2GNmWi9wbtizbkiVZ1dJoZs77x72SZ9RllRmNnu/nM59755w7d54rj5+5c+6554gxBqWUUpHLEeoAlFJKDSxN9EopFeE00SulVITTRK+UUhFOE71SSkU4V6gD6Eh6eroZN25cqMNQSqkhY8OGDSeMMRkd1YVloh83bhyFhYWhDkMppYYMEfmss7pum25EJEZE1ovIZhEpFpGfdbBNtIg8LyJ7ReQTERkXUPcDu3yXiFx6pgehlFLqzPSkjb4JuMgYMxvIAy4TkflttvkGUGWMmQT8GvhPABGZBiwDpgOXAY+IiLO/gldKKdW9bhO9sdTZT932o+3ttIuBP9rrLwIXi4jY5c8ZY5qMMQeAvcC8folcKaVUj/Sojd4+C98ATAIeNsZ80maTHOAwgDHGKyLVQJpd/nHAdiV2WUfvsRxYDjBmzJheHIJSaihobm6mpKSExsbGUIcypMXExJCbm4vb7e7xa3qU6I0xPiBPRJKBFSIywxizLWAT6ehlXZR39B6PA48DFBQU6AA8SkWYkpISEhMTGTduHNYPftVbxhgqKiooKSlh/PjxPX5dr/rRG2NOAu9jtbcHKgFGA4iIC0gCKgPLbblAaW/eUykVGRobG0lLS9Mk3wciQlpaWq9/FfWk102GfSaPiMQCXwB2ttlsFXCrvX4N8J6xhsVcBSyze+WMB84C1vcqQqVUxNAk33dn8jfsSdPNKOCPdju9A3jBGPN3EbkfKDTGrAKeAJ4Wkb1YZ/LLAIwxxSLyArAd8AJ32s1AXWr2+Xt9IEoppTrWbaI3xmwB8jso/3HAeiNwbSevfxB4sDdBVTU092ZzpZRSXQjLsW5ONnjQCVGUUv3p5MmTPPLII71+3RVXXMHJkyd7/brbbruNF198sdevGwhhmeibvH42HqoKdRhKqQjSWaL3+bpuTX7ttddITk4eqLAGRViOdeMQ4cUNJcwdmxrqUJRSA+Bnfytme2lNv+5zWvYIfnLV9E7r77vvPvbt20deXh5ut5uEhARGjRpFUVER27dvZ8mSJRw+fJjGxkbuvvtuli9fDpwee6uuro7LL7+c888/n7Vr15KTk8PKlSuJjY3tNrZ3332Xe+65B6/XyznnnMOjjz5KdHQ09913H6tWrcLlcnHJJZfwy1/+kr/+9a/87Gc/w+l0kpSUxJo1a/r8twnLM/qkWDd/23yUU55ur9sqpVSP/PznP2fixIkUFRXxi1/8gvXr1/Pggw+yfft2AJ588kk2bNhAYWEhDz30EBUVFe32sWfPHu68806Ki4tJTk7mpZde6vZ9Gxsbue2223j++efZunUrXq+XRx99lMrKSlasWEFxcTFbtmzhhz/8IQD3338/b775Jps3b2bVqlX9cuxheUafEuemrsnLG8VHWZqfG+pwlFL9rKsz78Eyb968oJuOHnroIVasWAHA4cOH2bNnD2lpaUGvGT9+PHl5eQDMnTuXgwcPdvs+u3btYvz48Zx99tkA3HrrrTz88MN8+9vfJiYmhm9+85t86Utf4sorrwRg4cKF3HbbbVx33XVcffXV/XGo4XlGHx/tYnRqLC9uKAl1KEqpCBUfH9+6/v777/POO++wbt06Nm/eTH5+foc3JUVHR7euO51OvF5vt+/TWccSl8vF+vXr+cpXvsIrr7zCZZdZ96E+9thjPPDAAxw+fJi8vLwOf1n0VlgmeoBr5oxm7b4KSqoaQh2KUioCJCYmUltb22FddXU1KSkpxMXFsXPnTj7++OMOtzsTU6ZM4eDBg+zduxeAp59+ms9//vPU1dVRXV3NFVdcwW9+8xuKiooA2LdvH+eeey73338/6enpHD58uM8xhGXTDcDVc3L49Tu7eXnjEb5z8VmhDkcpNcSlpaWxcOFCZsyYQWxsLFlZWa11l112GY899hizZs1i8uTJzJ/fdiT2MxcTE8Mf/vAHrr322taLsbfffjuVlZUsXryYxsZGjDH8+te/BuD73/8+e/bswRjDxRdfzOzZs/scg4Rjf/WCggJTWFjIV3//MSVVp3j/ngtxOPTWaaWGsh07djB16tRQhxEROvpbisgGY0xBR9uHbdMNwLUFuRyqbODTg5WhDkUppYassE70l00fRUK0i7/qRVmlVJi68847ycvLC3r84Q9/CHVYQcK2jR4gNsrJlbNGsWpzKT/78nTio8M6XKXUMPTwww+HOoRuhfUZPcA1c3Np8Ph4bevRUIeilFJDUtgn+rljUxifHq/NN0opdYbCPtGLCNfMzWX9gUo+q6gPdThKKTXkhH2iB6tPvQi8pGf1SinVa0Mi0Y9KiuX8Sem8tPEIfn/49ftXSkWehISETusOHjzIjBkzBjGavhkSiR7g2oLRHDl5inX7+z7ug1JKDSfh2V/RtJ8z9pJpWSTGuHhxQwkLJ6WHICilVL95/T44trV/9zlyJlz+806r7733XsaOHcsdd9wBwE9/+lNEhDVr1lBVVUVzczMPPPAAixcv7tXbNjY28q1vfYvCwkJcLhe/+tWvWLRoEcXFxXzta1/D4/Hg9/t56aWXyM7O5rrrrqOkpASfz8ePfvQjrr/++j4ddk+E5xl9Q/uz9hi3ky/Pzub1bUepadQ5ZZVSvbNs2TKef/751ucvvPACX/va11ixYgUbN25k9erVfO973+v1NKYt/ei3bt3Ks88+y6233kpjYyOPPfYYd999N0VFRRQWFpKbm8sbb7xBdnY2mzdvZtu2ba0jVg60bs/oRWQ08CdgJOAHHjfG/Hebbb4P3Biwz6lAhjGmUkQOArWAD/B2NhZDkLoy8DWD0x1UfG3BaJ755BCvbTnKsnljut2NUipMdXHmPVDy8/MpKyujtLSU8vJyUlJSGDVqFP/yL//CmjVrcDgcHDlyhOPHjzNy5Mge7/fDDz/krrvuAqyRKseOHcvu3btZsGABDz74ICUlJVx99dWcddZZzJw5k3vuuYd7772XK6+8ks997nMDdbhBenJG7wW+Z4yZCswH7hSRaYEbGGN+YYzJM8bkAT8APjDGBA5Qs8iu7z7JA/g8sK39zC2zc5OYlJmgfeqVUmfkmmuu4cUXX+T5559n2bJlPPPMM5SXl7NhwwaKiorIysrqcBz6rnT2C+CrX/0qq1atIjY2lksvvZT33nuPs88+mw0bNjBz5kx+8IMfcP/99/fHYXWr20RvjDlqjNlor9cCO4CcLl5yA/Bsn6Jyx8JH/w1t/oAiwrVzc9nwWRX7y+v69BZKqeFn2bJlPPfcc7z44otcc801VFdXk5mZidvtZvXq1Xz22We93ucFF1zAM888A8Du3bs5dOgQkydPZv/+/UyYMIHvfOc7fPnLX2bLli2UlpYSFxfHTTfdxD333MPGjRv7+xA71Ks2ehEZB+QDn3RSHwdcBgSejhvgLRHZICLLu9j3chEpFJHCWn8slG2HPW+1225pfg5Oh+jsU0qpXps+fTq1tbXk5OQwatQobrzxRgoLCykoKOCZZ55hypQpvd7nHXfcgc/nY+bMmVx//fU89dRTREdH8/zzzzNjxgzy8vLYuXMnt9xyC1u3bmXevHnk5eXx4IMPts4TO9B6PB69iCQAHwAPGmNe7mSb64GbjDFXBZRlG2NKRSQTeBu4yxjT5bTmBQVzTeGNTZA0Gr7+erv6rz/1KdtLa/jovotw6jj1Sg0JOh59/xmQ8ehFxI11lv5MZ0netow2zTbGmFJ7WQasAOb14B1hwZ1waC0cXt+u9tq5uRyraeTDvSd6Er5SSg1r3SZ6ERHgCWCHMeZXXWyXBHweWBlQFi8iiS3rwCXAth5FNucWiE2BD3/TruqiqZkkx7n5a2Hf51JUSqnObN26td1Y8+eee26ow+q1ntwwtRC4GdgqIkV22b8BYwCMMY/ZZUuBt4wxgSOPZQErrO8KXMBfjDFv9CiyqHg4559gzX9B+S7ImNxaFe1ysiQvh7+sP0R1QzNJce4udqSUChfGGOx8MCTMnDmzddLucHEm07/2pNfNh8YYMcbMaulCaYx5zRjzWECSxxjzlDFmWZvX7jfGzLYf040xD/YqunP/GVyx8NFD7aqumZuLx+tn1ZbSXu1SKRUaMTExVFRUnFGiUhZjDBUVFcTExPTqdeE5BEKL+HTIvwk2PAWL/g2STvfqnJ49gikjE3lxQwk3zx8buhiVUj2Sm5tLSUkJ5eXloQ5lSIuJiSE3N7dXrwnvRA9w3reh8En4+BG49PQPgpZx6h94dQd7jtdyVlZiCINUSnXH7XYzfvz4UIcxLIXnWDeBUsbB9KXWWf2pqqCqJfk5uLRPvVJKdSn8Ez3AwrvBUwefPhFUnJ4QzaIpmby86QheX/sRL5VSSg2VRD9qFky8GD55DJpPBVVdVzCa8tom3tlRFqLglFIqvA2NRA9w/nehvhyK/hJUvGhyBjnJsfxx7cHQxKWUUmFu6CT6cZ+D7Dmw9rfg97UWu5wOblkwlnX7K9h5rCaEASqlVHgaOolexDqrrzoA21cGVV1/zmhi3A49q1dKqQ4MnUQPMOVKSJ0IH/0maAjj5LgolubnsGLTEarqPSEMUCmlws/QSvQOJyz8DhzdDPvfD6q69bxxNDb7eV7Hv1FKqSBDK9EDzFoGCVnWWX2AKSNHsGBCGk+v+0y7WiqlVIChl+jdMTD/W9YZfWnwYEO3LRzHkZOneGfH8dDEppRSYWjoJXqAgq9D9AhrusEAX5iaRU5yLH/46GBo4lJKqTA0NBN9TBIUfA22vwKV+1uLnQ7h1vPG8smBSnYc1a6WSikFQzXRA8y/AxwuWPu7oOLrC8YQ63ZqV0ullLIN3USfOBJmL4OiZ6Du9LCnSXFulmhXS6WUajV0Ez3AeXeDt8kaAyfAbeeNo8nr57lPtaulUkoN7USfPgmmXgmf/h6aaluLJ49M5LyJaTy97qB2tVRKDXtDO9EDnPcdaKyGbS8HFd923jhKqxt5e7t2tVRKDW9DP9HnngMZU2DTn4OKL56aRW5KLH/Qi7JKqWGu20QvIqNFZLWI7BCRYhG5u4NtLhSRahEpsh8/Dqi7TER2icheEbmvvw8AEWte2ZL1UL6rtdjpEG5dMI71ByopLq3u97dVSqmhoidn9F7ge8aYqcB84E4RmdbBdv8wxuTZj/sBRMQJPAxcDkwDbujktX0z63qrq2Wbs/rrCkZrV0ul1LDXbaI3xhw1xmy012uBHUBOD/c/D9hrjNlvjPEAzwGLzzTYTiVkwtmXwebnwNfcWpwU52bpnBxWFpVSqV0tlVLDVK/a6EVkHJAPfNJB9QIR2Swir4vIdLssBwjs41hCJ18SIrJcRApFpLC8vLyjTbqWfxPUl8Get4OKT3e1PNT7fSqlVATocaIXkQTgJeC7xpi24wtsBMYaY2YDvwVeaXlZB7syHZRhjHncGFNgjCnIyMjoaVinTfqiNaplm+abs7MSWThJR7VUSg1fPUr0IuLGSvLPGGNebltvjKkxxtTZ668BbhFJxzqDHx2waS5Q2ueoO+J0WXfK7n4DaoO7VN523niOVjfylna1VEoNQz3pdSPAE8AOY8yvOtlmpL0dIjLP3m8F8ClwloiMF5EoYBmwqr+CbyfvJjA+2PJ8UPFFUzIZnRrLUzqqpVJqGOrJGf1C4GbgooDuk1eIyO0icru9zTXANhHZDDwELDMWL/Bt4E2si7gvGGOKB+A4LBlnw+hzreabgKkGW7taHqxk2xHtaqmUGl7EmA6bzEOqoKDAFBYWntmLN/4JVt0F33gHRp/TWlx9qpn5//EuV84axS+und1PkSqlVHgQkQ3GmIKO6ob+nbFtTV8K7jjY9HRQcVKsm6/MzWHl5lIq6ppCFJxSSg2+yEv00YlWst/2Mnjqg6puXTAOj45qqZQaZiIv0YPVp95TC9uDr/uelZXI+ZPS+fPHn9GsXS2VUsNEZCb6MQsgdUK7PvVg3UB1tLqRt4q1q6VSaniIzETfMtDZZx9Cxb6gqkVTMhmTGsdTaw+EKDillBpckZnoAWbfAOKAor8EFTsdwi0LxvLpwSr2HK/t5MVKKRU5IjfRj8iGSV+wEr3fF1T15bxsHAIriwbmJl2llAonkZvowWq+qS2FfauDijMTY1g4KZ2Vm48QjvcRKKVUf4rsRH/25RCX1q5PPcDivBwOV55i0+GTIQhMKaUGT2QneleUNSnJzlehviKo6tLpWUS5HKzcdCREwSml1OCI7EQPVvONvxm2/jWoODHGzRemZvL3LUd1+GKlVESL/ESfNR2y863mmzbt8Yvzcqio9/DRvopOXqyUUkNf5Cd6sM7qj2+Do5uDii+cnEFijEubb5RSEW14JPoZ14Arpt2dstEuJ1fMGMWbxcc45fF18mKllBrahkeij02GqVfB1heguTGoanF+NvUeH+/u1CERlFKRaXgkerCabxqrYeffg4rPHZ9G1ohoXtmkN08ppSLT8En04y6ApDHtmm+cDuGqWdl8sLuMkw2eEAWnlFIDZ/gkeocD8m+E/e/DyUNBVYvzcmj2GV7fdiw0sSml1AAaPokeIO+r1rLo2aDiGTkjmJARz8oi7X2jlIo8wyvRJ4+BCZ+Hoj+D//RNUiLC4tk5fHKgkqPVp0IYoFJK9b9uE72IjBaR1SKyQ0SKReTuDra5UUS22I+1IjI7oO6giGwVkSIROcMZv/tR/s1W081nHwYVL87Lxhj422a9KKuUiiw9OaP3At8zxkwF5gN3isi0NtscAD5vjJkF/DvweJv6RcaYvM5mKB9UU74EMUntLsqOS49n9uhkHbpYKRVxuk30xpijxpiN9notsAPIabPNWmNMlf30YyC3vwPtN+5YmHktbF9pdbcMsHh2NsWlNewt0wlJlFKRo1dt9CIyDsgHPulis28Arwc8N8BbIrJBRJZ3se/lIlIoIoXl5eW9Cav3Zl0P3kbY9XpQ8ZWzRumEJEqpiNPjRC8iCcBLwHeNMTWdbLMIK9HfG1C80BgzB7gcq9nngo5ea4x53BhTYIwpyMjI6PEBnJHccyBpNBSvCCrOHBHDeRPTWVlUqhOSKKUiRo8SvYi4sZL8M8aYlzvZZhbwv8BiY0zrcJDGmFJ7WQasAOb1Neg+E4Fpi2Hvu3CqKqhqcV42hyobKNIJSZRSEaInvW4EeALYYYz5VSfbjAFeBm42xuwOKI8XkcSWdeASYFt/BN5nM662xqnf+VpQ8aUzRloTkmjzjVIqQvTkjH4hcDNwkd1FskhErhCR20XkdnubHwNpwCNtulFmAR+KyGZgPfCqMeaN/j6IM5I9B5LHQnHwD5QRMW4unpLJ37eU6oQkSqmI4OpuA2PMh4B0s803gW92UL4fmN3+FWFABKYvhXW/g4ZKiEttrVqcl8Pr246xdl8FF5w9wNcLlFJqgA2vO2Pbmr4U/F7Y8beg4pYJSV7RIRGUUhFgeCf6UbMhdUK73jcxbieXzxjJm9uO0disE5IopYa24Z3oW5pvDqyB+hNBVUvycqwJSXaUhSg4pZTqH8M70QNMvxqMD3asCio+d0IamYnR2nyjlBryNNFnTYe0s2BbcO8bp0O4anY27+8qo7qhOUTBKaVU32miF7H61H/2EdQGzxu7OC/bnpDkaIiCU0qpvtNED1Y7vfG3a76ZmZPEhPR4vXlKKTWkaaIHyJwKGVPbNd+ICF/Oy+bjAxUcq24MUXBKKdU3muhbzLgaDq2DmuCz98V5OTohiVJqSNNE32L6UsBY49QHGJ8ez+zcJFZu1t43SqmhSRN9i/SzIGtmu5unAL6cl8O2IzXsLasLQWBKKdU3mugDTV8Chz+B6pKg4qtmjUIEVmmfeqXUEKSJPtD0pday+JWgYmtCkjRWbtYJSZRSQ48m+kBpE63xb4rbz62yOC+Hzyoa2KQTkiilhhhN9G1NvxqObICqz4KKL5sxkmiXg1c2afONUmpo0UTf1vQl1rLNRdkRMW6+MC2Lv20upVknJFFKDSGa6NtKGQc5czvsfbM0L4eqhmbW7C4f/LiUUuoMaaLvyPSlcLQIKvYFFV9wdgYpcW5WaPONUmoI0UTfkWl288324N43US4HV87K5u3tx6lt1BEtlVJDgyb6jiSPhtx5sK19882S/ByavH7e2HYsBIEppVTvdZvoRWS0iKwWkR0iUiwid3ewjYjIQyKyV0S2iMicgLpbRWSP/bi1vw9gwMy4Go5vhRN7gornjElmTGqcTkiilBoyenJG7wW+Z4yZCswH7hSRaW22uRw4y34sBx4FEJFU4CfAucA84CciktJPsQ+saYsBaXdRVkRYkp/D2n0VHK/RES2VUuGv20RvjDlqjNlor9cCO4CcNpstBv5kLB8DySIyCrgUeNsYU2mMqQLeBi7r1yMYKCOyYcyCDnvfLMnLxhhYpePUK6WGgF610YvIOCAf+KRNVQ5wOOB5iV3WWXlH+14uIoUiUlheHibdF2dcDWXboWxnUPGEjARmj07W3jdKqSGhx4leRBKAl4DvGmNq2lZ38BLTRXn7QmMeN8YUGGMKMjIyehrWwJr6ZRBHh0MiLM3LZvvRGnYdqw1BYEop1XM9SvQi4sZK8s8YY9pnPetMfXTA81ygtIvyoSExC8YutJpv2gxmduXsbJwO0YuySqmw15NeNwI8Aewwxvyqk81WAbfYvW/mA9XGmKPAm8AlIpJiX4S9xC4bOqYvhRO74XhxUHF6QjQXnJXOyk1H8Pt1REulVPjqyRn9QuBm4CIRKbIfV4jI7SJyu73Na8B+YC/we+AOAGNMJfDvwKf24367bOiYtthuvum4T31pdSPrDw6tQ1JKDS+u7jYwxnxIx23tgdsY4M5O6p4Enjyj6MJBfDqMv8Bqp7/ohyCn/xSXTBtJfJSTVzYdYf6EtBAGqZRSndM7Y3ti+tVQuR+ObQkqjo1ycumMkby69SiNzb4QBaeUUl3TRN8TU68Chwu2ddD7Jj+H2kYvq3eWhSAwpZTqnib6nohLhQkXWs03/uCx6M+bmE5GYrT2qVdKhS1N9D018zo4eQgOrQ0qdjqExbOzWb2rjJMNnhAFp5RSndNE31NTr4LoEbDpmXZVS/JzaPYZXt16NASBKaVU1zTR91RUnDUkwvZXoDH4xuDp2SM4KzNB55NVSoUlTfS9kX8zNDd0OqLlpwerOFzZEKLglFKqY5roeyNnLmRMgU1/ble1OC8bgJU6JIJSKsxoou8NEci/CUrWQ/muoKrclDjmjU9lxaYjGKNDIiilwocm+t6adT2IE4raX5Rdmp/DvvJ6th1pO7inUkqFjib63krIhLMvg6JnwRc8QfgVM0YR5XRon3qlVFjRRH8m8m+C+jLY+05QcVKcm4umZLJqcylen7+TFyul1ODSRH8mzvoixGd2eFF2SX42J+qa+GhfRQgCU0qp9jTRnwmnG2Yvg91vQF3wtIcXTs5kRIyLldp8o5QKE5roz1T+TeD3wpbng4pj3E6+NGsUbxQfo8HjDVFwSil1mib6M5UxGXLPgU1Pt5tmcEleDg0eH29vPx6i4JRS6jRN9H2RfxOU74QjG4OKzxmXSk5yrPa+UUqFBU30fTH9anDFWmf1ARwOYXFeNv/Yc4Ly2qYQBaeUUhZN9H0RMwKmL4FtL4EneIybpfk5+PyGv28pDVFwSill0UTfV3k3QlMN7Px7UPFZWYlMzx7BXwtLdEgEpVRIdZvoReRJESkTkW2d1H9fRIrsxzYR8YlIql13UES22nWF/R18WBi7EFLGtWu+AbhlwVi2H63hg93l7V+nlFKDpCdn9E8Bl3VWaYz5hTEmzxiTB/wA+MAYUxmwySK7vqBvoYYphwPyboIDa6DqYFDV0vxcspNi+N17e/WsXikVMt0memPMGqCyu+1sNwDP9imioSjvBkCg6C9BxVEuB7dfOJHCz6r45EBP/4RKKdW/+q2NXkTisM78XwooNsBbIrJBRJb313uFnaRcmHiRlejbTB5+XcFo0hOi+d17e0MUnFJquOvPi7FXAR+1abZZaIyZA1wO3CkiF3T2YhFZLiKFIlJYXj4E27Tzb4Tqw3Dgg6DiGLeT5ReM58O9J9h0qCpEwSmlhrP+TPTLaNNsY4wptZdlwApgXmcvNsY8bowpMMYUZGRk9GNYg2TylyAmucOBzm48dyzJcW4eXq1n9UqpwdcviV5EkoDPAysDyuJFJLFlHbgE6LDnTkRwx8Cs62DH3+BU8Jl7fLSLry8czzs7yigurQ5RgEqp4aon3SufBdYBk0WkRES+ISK3i8jtAZstBd4yxtQHlGUBH4rIZmA98Kox5o3+DD7s5N8EvibY+mK7qlvPG0ditItHVu8LQWBKqeHM1d0GxpgberDNU1jdMAPL9gOzzzSwIWnUbBg505pmcN4/BVUlxbq55byxPPL+PvaW1TIpMzFEQSqlhhu9M7a/5d0EpZvgWPtWqq8vHE+My8kj7+tZvVJq8Gii72+zrgNnVIeTh6clRPPVc8ewsqiUQxUNHbxYKaX6nyb6/haXCpOvgM3PgdfTrnr5BRNwivDoB3pWr5QaHJroB0L+zXCqEna/3q4qa0QM152Ty0sbSjhafSoEwSmlhhtN9ANh4iJIzIZN7ZtvAP75gon4jeHxNfsHOTCl1HCkiX4gOJzW+Dd734aao+2qR6fGsSQ/h2fXH9KJSZRSA04T/UDJuxGMHzZ3PMbbHRdOpMnr54kPDwxyYEqp4UYT/UBJm2iNVf/pE9BU1656QkYCV87K5ul1BznZ0P6irVJK9RdN9APp4h9DzRF49/4Oq+9cNJF6j4+n1h4c3LiUUsOKJvqBNGY+zFsO6x+HQx+3q54ycgRfnJbFHz46SG1jcwgCVEoNB5roB9rFP7bGq191FzQ3tqv+9qJJVJ9q5s8fHzDl4QcAABL5SURBVApBcEqp4UAT/UCLToCr/htO7IY1v2hXPXt0MhecncETH+7nlMcXggCVUpFOE/1gmHSx1Qvno9/A0S3tqu+6aBIn6jw896me1Sul+p8m+sFyyQMQmwqrvg0+b1DVOeNSmTc+lf/5YD9NXj2rV0r1L030gyUuFb70Szi6Gdb9tl31XRdN4lhNIy9tOBKC4JRSkUwT/WCathimXgWr/y+cCJ5W8PxJ6cwencyjH+zF6/N3sgOllOo9TfSD7YpfWtMOrroL/KcTuohw16JJHK48xcqi0hAGqJSKNJroB1viSLj0P+DQWtjwZFDVxVMzmZ49gn9/dTt7jteGKEClVKTRRB8KeTfChEXw9k/g5OHWYhHh0Rvn4nY6uOXJ9Rw5qcMYK6X6ThN9KIhYfeuNgb//i7W0jUmL409fn0ddk5ebn/iEijod3VIp1Tea6EMlZax11+zet2HLC0FVU0eN4MnbzuFI1Sm+9tSn1DV5O9mJUkp1r9tELyJPikiZiLSf7dqqv1BEqkWkyH78OKDuMhHZJSJ7ReS+/gw8Isz7J8idB2/cC3XlQVXnjEvl0ZvmUFxaw/I/FWr/eqXUGevJGf1TwGXdbPMPY0ye/bgfQEScwMPA5cA04AYRmdaXYCOOwwmLfweeenj9X9tVXzQli19cM4u1+yq4+9kifH7TwU6UUqpr3SZ6Y8waoPIM9j0P2GuM2W+M8QDPAYvPYD+RLWMyfP5fofhl2Plqu+qr5+Tyoyun8UbxMX74ylaM0WSvlOqd/mqjXyAim0XkdRGZbpflAIcDtimxyzokIstFpFBECsvLyzvbLDIt/C5kzYBXvwenTrar/sb547lz0USeXX+YX761KwQBKqWGsv5I9BuBscaY2cBvgVfsculg205PR40xjxtjCowxBRkZGf0Q1hDidFtNOHXH4e0fd7jJPZdM5oZ5Y3h49T7+9x86qbhSquf6nOiNMTXGmDp7/TXALSLpWGfwowM2zQX0ls/OZOfDeXfBxj/CvvfaVYsIDyyZwRUzR/LAqzt4aUNJCIJUSg1FfU70IjJSRMRen2fvswL4FDhLRMaLSBSwDFjV1/eLaBf+ANImwZ+vgVXfgZrg70WnQ/j19XksnJTGv760hXe2Hw9RoEqpoaQn3SufBdYBk0WkRES+ISK3i8jt9ibXANtEZDPwELDMWLzAt4E3gR3AC8aY4oE5jAjhjoWvv2l1uyz6Czw0B975aVC7fbTLyf/cXMCM7BHc+ZeNrD9wJtfJlVLDiYRjL46CggJTWFgY6jBCq/IArP4P2PoCxCTD575nzT/rjrGq6z1c89haymubeH75AqZljwhxwEqpUBKRDcaYgo7q9M7YcJU6Hr7ye/jnf0BuAbz9I/jtXNj0Z/D7SI2P4ulvnEtCtItbnlzPxkNV2vVSKdUhPaMfKg6ssQZBK90IGVOt4RMmX87e8jqu+5+Pqaz3MCophi9MzeIL07KYPyGVaJcz1FErpQZJV2f0muiHEmNg+0p479+hYi+Mng9f/BlVaXN4Z8dx3tlxnDW7T3Cq2Ud8lJPPT87gC1OzuGhKJslxUaGOXik1gDTRRxpfM2x6Gt7/udX3fvIVkH8TZE6jMSGXdfureGv7cd7dcZyy2iacDqFgbApfnJbFF6ZmMS49vudv5TfUNXmpbWymsdlHbkocMW79paBUuNFEH6k89fDxo/DRf0NTjVXmjoOMKZA5DX/GFA44xvJ2RSqv7PGx83gdAJMyE7h4aiYjYtzUNDZT2+i1H81tlt52I2e6ncK0USPIG51M/pgU8kYnMzYtDruHrVKqL4wBTx001oC/GXxe8Hutdb+3i+deZPpiTfQRzVMPZTugbDsc324ty3ZAfdnpbWKSaUydzAHHWNbVZfJ2eSqH/WlUO1OIjokjMcZNYozLekS3rJ8uGxHjJsrlYOexWjYdqmLrkWoaPNaImilxbvJGJ5M3OoW8Mcnk5SaTFOcO0R9DqTDQfAoaKqChEk5VwqkqaKy2uko3ngxeP2U/byn3n9mw5PKzGk30w1L9CfsLYMfp5F+2A5qqg7eLHgHxGZCQaT3iMztZz7D6+gNen589ZXUUHT7JpkNVFB0+yZ6yutY5VCZkxJM3Opk5Y1K4ZHoWmYkxg3zwSvWT1qRdYf2fakneLYm8oSLgeZW19HYxO5zDZXWZjk2GmKSAdft5bLL1f9IVbW3b8nC6u3wuo2Zqolc2Y6w7bst3QM1Rq42/vhzqyqxHvb1sbD+4GgAON0TFQ1SCvYxvfd7siqXC4+bYKReH64UDNUJ5k5Nm3IwfmUr++Cxmj8skOiYWXDHgjAZX1OllS5nTbc3ChVhLcZxe72ypTUeDz++zmgwbq62mhsZq+3ngejU0N3Sxk07+3RxOcEadfrR8TlrXW8qjT687Orl21GmOM9BUBw0n7CRuJ/O2z5vrOw8/Jhni0iAu1VrGptrrbZ7HppxO5FHxA/J57aqN3tXv76bCmwgk5ViPrnibTn8B1Jef/kJoqrOaijz1Vltiy3rNEdyeekZ66hnZ3ECepw6MH1pacCrsx4B9f0vwf/qghGB/ebjsZUvCcDitGDt6+H32ugko91lfOg43OF32PjpYd7jt9w1YD0xOTndwXIHlLTFKL29xMcb6N/OeguZGa+ltss5GvY3tl95Gq97f0sZrH5/fax+7z1q2rtvlvubTydzTgwns3fHWr8COEltXJ5l+r/VevqYzbsroNXe8lZzj06xl+uTg53HpdgK3lzHJ1r/xEDA0olSDzxUNSbnW40wYYyUTTz14G/E3N1F8qJz3t5ewfm8pXk8jGbHC+eMSmD82gdEjnIjPYyUfn8dOAiZg6Q9Yp02dsZKRzwNej7X0NVmJwtt0OmG01DfVWet+L4gTHA4rsQY9nKfXHU4QO/kav/U6TwP4Tp5OSP5m+306WB+sRNUtsZKuKxpcsQFfKk7rGB3OgHWXte6KOr3eUh6TZD2iR9jrIzp4ngzRidb++8rvt/9N7UfLZ6T1uf3vbfxdH3tHohPsJJ7W2iwZiTTRq4EhdlKx//M4gJnpE5k5B5q8PlbvLGPFpiP8n51lNBcbJmUmsDQ/h8V52eSmxHW5a5/f4PH6afL6aPL68Xj9xLidZCRGD8KBnQG/307+noAvH3s9MGEFJq7OR/TuXEsCd8dYzWCuGDux20tn1NBs4nI4wBHTOvyH6j1to1chdbLBw2tbj7FiUwmfHqwCrMnRnQ7sZO6nqdmPx+enqdlK7N5OplQcnx7P/AlpLJiYxvwJqXoBWA0r2o9eDQmHKxtYWXSETw5U4nY6iHY5iHJZy2iXs+N1t4Mop4OTDc18vL+C9QcqqbX7/k/MiGfBxDQWTEjn3AmppCeE6Rm/Uv1AE70aNrw+P9uP1rBuXwXr9lfw6YFK6u3+/mdnJbDAPuOfNz6N1PgofH5Dzalmqho8VDU0czJgebLBKm9ZVjU043RAanw06fFRpCVEkZYQTVp8FOkJ0UHP9e5hNdg00athq9nnZ+uRaj7eX8G6fRUUHqziVLOV+JNirTuDO/sv4BBIjosiOc5NSlwUKXFu/AYq6po4UefhRF0TTd6OLwDGRzmtpJ8QRYzLaTUziyAiiL1v67k1e1jb526HEGX/onE77V82zuDnUS7r10zLr5vU+GjSE6LISIwmIdqldysPM9q9Ug1bbqeDOWNSmDMmhTsunITH62frkZOs21fB8ZomUuLcQcn8dFKPIjHGhcPRebI0xtDg8VFR56Givql1eaLOE1TW5PVZvRSNwW+spQH8xli9Gmmps+r9xuD1WRecPT4/zV4/TT7ronNPxbgdZCRGk5EQTUZiNOn2sm1ZfLSLuCgn0S7HkPtiaGz2caiygf3l9Rw4UY/bKczMSWJ6ThIJ0ZraAulfQw0rUS4Hc8emMndsap/3JSLER7uIj3YxJq3rnkL9wRiD1+5x5PH6afZZF6s9Pj+NzT6q6pspr2ukvLbp9KOuiQMn6ll/oJKqhuZO9+0QiHU7ibMTf6zbSVyUk7gol710EhvlIiHaSUp8FGnxUaTGR5Pasp4QReIA/Irw+w2l1adak/mBE/XsP1HPgRN1lFSd6vDXmAhMSI9nVm4yM3OSmJWbxLTsEcRFDd90N3yPXKkhRkRwOwW300H8GVxX9nj9VNZ77C+ARk7Ueqj3eGnw+Djl8VnLZuu59fBS7/Fyoq6ptayuqZnG5o5/WUQ5HaTEu0mNj7a/CKzHiBjX6V8wpuWXzOl101IW8Iunqr6ZAyfqOVhRH9Q8Fh/lZEJGAvmjU/jKnFzGp8czIT2BcelxNDb72Xakmi0l1Ww9Us3afSdYsekIYH2RTcpMYGZOMrNyk5iZm8S0USOIcTtp9vmpbzo9iF9dk5e6Ri+1TV7qA9brGr00eLycnZXIRVMyezUKbKhpG71SqldOeXxU1DdRWe+hot5DZZ3n9HprE5ZVVlnvoa7Ji9jXIJz2dQhHm+sSToe0XsNwCCTEuJiQHs+EjATGp8fbCT2ejMToXv1qOF7TyFY78W89Us2WkpOcqPMA1nu6HNLpdZZAIhAf5SLa5aCi3nr9hIx4LpqcyUVTMikYl0qUK7QT9unFWKVUyBhjwqb93xjDsZpGtpRUU3ykmiavnwS7+S0hxkWivUyItkZtTYh2kxDjIs7tbL1ec6iigfd2Hue9XeV8vK8Cj8/axwVnp7NociYXTs4Myc17fUr0IvIkcCVQZoyZ0UH9jcC99tM64FvGmM123UGgFvAB3s6CaEsTvVJqKKhv8vLR3hOs3lXGezvLOF7TBMDs3CQummLN7jY9ewQOh+D3G06eam791XP6V1Dwo6LeQ82pZtxOIdrlJMZt3TsSbS9jOll+94tn9ynRX4CVwP/USaI/D9hhjKkSkcuBnxpjzrXrDgIFxpgTvfnjaaJXSg01xhiKS2tYvbOMd3eWsbnkJMZAanwUAlQ1eOjkpm4So12kJkS1XtweEevG5zc02neDn17aQ3/Yy5bnzT7DZ/955Zl3rzTGrBGRcV3Urw14+jFwhqNgKaXU0CUizMhJYkZOEnddfBYn6pr4YFc56/ZX4HY6Wi9QpyWcvlCdFh9NSrybaFffbrDz+Q2u/+y8vr973XwDeD3guQHeEhED/I8x5vHOXigiy4HlAGPGjOnnsJRSanClJ0Tzlbm5fGXuwJ/7Oru43wP6MdGLyCKsRH9+QPFCY0ypiGQCb4vITmPMmo5eb38JPA5W001/xaWUUsNdv/QHEpFZwP8Ci40xFS3lxphSe1kGrADm9cf7KaWU6rk+J3oRGQO8DNxsjNkdUB4vIokt68AlwLa+vp9SSqne6bbpRkSeBS4E0kWkBPgJ9gRxxpjHgB8DacAjdl/Zlm6UWcAKu8wF/MUY88YAHINSSqku9KTXzQ3d1H8T+GYH5fuB2WcemlJKqf4Q2nt2lVJKDThN9EopFeE00SulVIQLy0HNRKQW2BXqOEIoHejVsBERZrgfP+jfQI+/98c/1hiT0VFFuI5Hv6unA6BFIhEp1OMfvscP+jfQ4+/f49emG6WUinCa6JVSKsKFa6LvdPCzYUKPXw33v4Eefz8Ky4uxSiml+k+4ntErpZTqJ5rolVIqwoVVoheRy0Rkl4jsFZH7Qh1PKIjIQRHZKiJFIhLx8ymKyJMiUiYi2wLKUkXkbRHZYy9TQhnjQOrk+H8qIkfsz0CRiFwRyhgHmoiMFpHVIrJDRIpF5G67fFh8Dro4/n77HIRNG72IOIHdwBeBEuBT4AZjzPaQBjbIznSe3aGqozmJReS/gEpjzM/tL/wUY8y9Xe1nqOrk+H8K1BljfhnK2AaLiIwCRhljNtpDm28AlgC3MQw+B10c/3X00+cgnM7o5wF7jTH7jTEe4DlgcYhjUgPMnnGssk3xYuCP9vofsT70EamT4x9WjDFHjTEb7fVaYAeQwzD5HHRx/P0mnBJ9DnA44HkJ/XywQ0TLPLsb7Hl0h6MsY8xRsP4TAJkhjicUvi0iW+ymnYhssuiIiIwD8oFPGIafgzbHD/30OQinRN/R7Lbh0a40uBYaY+YAlwN32j/t1fDyKDARyAOOAv8vtOEMDhFJAF4CvmuMqQl1PIOtg+Pvt89BOCX6EmB0wPNcoDREsYSMzrMLwHG73bKl/bIsxPEMKmPMcWOMzxjjB37PMPgMiIgbK8k9Y4x52S4eNp+Djo6/Pz8H4ZToPwXOEpHxIhIFLANWhTimQaXz7LZaBdxqr98KrAxhLIOuJbnZlhLhnwGx5ht9AthhjPlVQNWw+Bx0dvz9+TkIm143AHb3od8ATuBJY8yDIQ5pUInIBKyzeDg9z25E/w0C5yQGjmPNSfwK8AIwBjgEXGuMicgLlp0c/4VYP9cNcBD455a26kgkIucD/wC2An67+N+w2qkj/nPQxfHfQD99DsIq0SullOp/4dR0o5RSagBooldKqQiniV4ppSKcJnqllIpwmuiVUirCaaJXSqkIp4leKaUi3P8H31yvGsUt0BMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "_ = log.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After termination, the `EarlyStopping` callback loads the best performing model (in terms of validation loss).\n", "We can verify this by comparing the minimum validation loss to the validation score of the trained `model`." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1.3641669750213623" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "log.to_pandas().val_loss.min()" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'loss': 1.3641669750213623}" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.score_in_batches(val)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prediction\n", "\n", "For evaluation we first need to obtain survival estimates for the test set.\n", "This can be done with `model.predict_surv` which returns an array of survival estimates, or with `model.predict_surv_df` which returns the survival estimates as a dataframe." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "surv = model.predict_surv_df(x_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can plot the survival estimates for the first 5 individuals.\n", "Note that the time scale is correct because we have set `model.duration_index` to be the grid points.\n", "We have, however, only defined the survival estimates at the 10 times in our discretization grid, so, the survival estimates is a step function" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAdRklEQVR4nO3dfXRV9Z3v8feXEAiaYBSCYoIGR6mAZRAi6PgweKsjUgemFQWfeFCLvVdbXHbuEsc7tnpXx9bq7cPU1cqtwlQ74kOvI9dGrC0y0+X1gaBWEIlmFOXwIAFEQIFA8r1/nE04ZCfkhJydvU/yea11Vs7eZ5+9P+yc8D2/vX97/8zdERERydQr7gAiIpI8Kg4iIhKi4iAiIiEqDiIiEqLiICIiIb3jDnAkBg4c6JWVlXHHEBHJGytWrNji7mXZLp+XxaGyspKampq4Y4iI5A0z+6gjy+uwkoiIhKg4iIhIiIqDiIiE5OU5BxGROOzbt49UKsWePXvijtKmoqIiKioqKCws7NR6VBxERLKUSqUoKSmhsrISM4s7Toi7s3XrVlKpFEOHDu3UuiI9rGRmj5jZZjNb1cbrZmY/M7M6M3vbzMZEmUdEpDP27NnDgAEDElkYAMyMAQMG5KRlE/U5h4XAxMO8filwWvCYA/wi4jwiIp2S1MJwQK7yRXpYyd3/w8wqD7PIFODXnr5v+KtmVmpmg9194+HWu23dFh6b+3AOk3Ze6dACLrt1VtwxRERyIu5zDuXAuozpVDAvVBzMbA7p1gVDjzsJ27a/SwJmY1dROaw+bD0TEcmZJUuWMHfuXBobG7nxxhuZN29ezrcRd3Forf3T6uhD7j4fmA8w7JijfeC25LQcmvpfz/7CZDc1RaR7aGxs5Oabb+bFF1+koqKCs846i8mTJzNixIicbifu4pAChmRMVwAb2nvTtuP7sehbIyML1VF/9bguGBGRrvH6669z6qmncsoppwAwffp0nn322W5XHBYDt5jZImA88Fl75xsAKvtXsmDigsjDZevhxx+KO4KIdLG7/+87rN6wI6frHHFif777t4f/4rt+/XqGDDn4nbqiooLXXnstpzkg4uJgZo8DE4CBZpYCvgsUArj7L4FqYBJQB3wBzI4yj4hIvkv33zlUFD2oou6tdFU7rztwc5QZRESi0N43/KhUVFSwbt3BfjypVIoTTzwx59vRoXIRkTxy1lln8f777/Phhx/S0NDAokWLmDx5cs63E/c5BxER6YDevXvz85//nEsuuYTGxkauv/56Ro7MfStGxUFEJM9MmjSJSZMmRboNHVYSEZEQFQcREQlRcRARkRAVBxERCVFxEBGREBUHEREJUVfWHOnT4Hx03Yy4Yxyi/2WXcey0K+OOISI5dP311/Pcc88xaNAgVq1qdZDNnFDLIQc+7wcNfZJ1y+49a9aw47nn4o4hIjk2a9YslixZEvl21HLIgZ3Fxs5iOPn+X8cdpVnSWjEikhsXXHABa9eujXw7Kg4iIkfi+XmwaWVu13nCl+HSH+R2nUdIh5VERCRELQcRkSORkG/4UVFxyJE95sxekpyxiqZvW8OAfgM4Oe4gIpKXdFgpB45p7EWRJ6u30hf7d7N199a4Y4hIjl111VWcc8451NbWUlFRwcMPPxzJdtRyyIHSJqO0qYA7EzSu9Qv/PC7uCCISgccff7xLtqOWg4iIhKg4iIhIiIqDiIiEqDiIiEiITkjnSFOTM+2hV+KO0Wx6QyOFBar9InJkVBxyoLCgF/toijvGIRqbHBKWSUTyh4pDDvQp6EWfgl48cdM5cUdp9szTybruQkRyY926dcyYMYNNmzbRq1cv5syZw9y5c3O+HRUHEZE80rt3bx544AHGjBnDzp07GTt2LBdffDEjRozI7XZyuraerOFzWPDVuFM06+u7adSvV6TbGTx4MIMHDwagpKSE4cOHs379ehWHRDq6LO4EIb1oAvbHHUOk2/rh6z9kzbY1OV3n6cedzu3jbs96+bVr1/Lmm28yfvz4nOYAFYfcKDkh/Zg9M+4kzZp+k9tvESKSLLt27eLyyy/nJz/5Cf3798/5+lUcRESOQEe+4efavn37uPzyy7nmmmv4+te/Hsk21BFeRCSPuDs33HADw4cP57bbbotsO5EWBzObaGa1ZlZnZvNaef0kM3vJzN40s7fNbFKUeURE8t3LL7/Mo48+ytKlSxk9ejSjR4+muro659uJ7LCSmRUADwIXAylguZktdvfVGYv9D+BJd/+FmY0AqoHKqDKJiOS78847D3ePfDtRthzGAXXu/oG7NwCLgCktlnHgwJmUY4ANEeYREZEsRVkcyoF1GdOpYF6m7wHXmlmKdKvhW22tzMzmmFmNmdXU19fnOquIiGSIsrdSa/dvaNkWugpY6O4PmNk5wKNmdoa7h24K5O7zgfkAVVVV0bepOmhLahfPPPBG3DGabSufy9E7a+KOISJ5KsrikAKGZExXED5sdAMwEcDdXzGzImAgsDnCXDk3bNzxcUcIaehTDiVxpxCRfBVlcVgOnGZmQ4H1wHTg6hbLfAx8BVhoZsOBIiDvjhmNPL+ckee3PGIWr4dnLo87gojkscjOObj7fuAW4AXgXdK9kt4xs3vMbHKw2HeAb5jZn4HHgVneFafhRUTksCK9Qtrdq0mfaM6cd1fG89XAuVFmEBHpTvbs2cMFF1zA3r172b9/P1OnTuXuu+/O+XZ0+wwRkTzSt29fli5dSnFxMfv27eO8887j0ksv5eyzz87pdnT7DBGRPGJmFBcXA+l7LO3btw+z3A/upZZDN9anwfnouhlxxzhE/8su49hpV8YdQ6TTNv3TP7H33dzesrvv8NM54R/+od3lGhsbGTt2LHV1ddx88826Zbdk7/N+6RGk12zJ7Ye3Mwat/4L63fVUqTiIdEpBQQFvvfUW27dv52tf+xqrVq3ijDPOyOk2VBy6qV79Cthc3MT/u2pk3FGaTf5xDUft3hp3DJGcyOYbftRKS0uZMGECS5YsUXGQ7JQ2GaVNBdw5cUHcUZq98M/j4o4gkvfq6+spLCyktLSU3bt384c//IHbb8/92BIqDiIieWTjxo3MnDmTxsZGmpqauPLKK7nssstyvh0VBxGRPDJq1CjefPPNyLejrqwiIhKi4iAiIiEqDiIiEqLiICIiISoOIiISouIgIiIhKg4iInmmsbGRM888M5LrGw5QcRARyTM//elPGT58eKTbUHEQEckjqVSK3/3ud9x4442RbkdXSIuIHIE/PfkeW9btyuk6Bw4p5vwrhx12mVtvvZX77ruPnTt35nTbLak4dGNFvgcWfDXuGAc1fA4FhXGnEMlbzz33HIMGDWLs2LEsW7Ys0m2pOHRTO3qVQtP2uGMcypugcV/cKURyor1v+FF4+eWXWbx4MdXV1ezZs4cdO3Zw7bXX8thjj+V8WyoO3dSnBQP4tGAAzP5d3FEO+tfkjC0hko/uvfde7r33XgCWLVvG/fffH0lhAJ2QFhGRVqjlICKShyZMmMCECRMiW79aDiIiEqLiICIiISoOIiId4O5xRzisXOVTcRARyVJRURFbt25NbIFwd7Zu3UpRUVGn16UT0tK1vClZF+YBfHkqVM2OO4XkgYqKClKpFPX19XFHaVNRUREVFRWdXo+Kg3SdgsLkXQS3aWX6p4qDZKGwsJChQ4fGHaNLqDhI1ynok34k6cK8pLViRBIi0nMOZjbRzGrNrM7M5rWxzJVmttrM3jGzf40yj4iIZCeyloOZFQAPAhcDKWC5mS1299UZy5wG3AGc6+6fmtmgqPKIiEj2omw5jAPq3P0Dd28AFgFTWizzDeBBd/8UwN03R5hHRESyFGVxKAfWZUyngnmZhgHDzOxlM3vVzCa2tTIzm2NmNWZWk+SeAiIi3UGUxcFamdeyc3Bv4DRgAnAV8CszK21tZe4+392r3L2qrKwsp0FFRORQUfZWSgFDMqYrgA2tLPOqu+8DPjSzWtLFYnmEuSRGX+zfzewlCeo2ap8wyY/mirhziCRMlC2H5cBpZjbUzPoA04HFLZb5N+BCADMbSPow0wcRZpIYDeg3gKN694s7xiFqaaDaPo87hkjiRNZycPf9ZnYL8AJQADzi7u+Y2T1AjbsvDl77GzNbDTQC/93dt0aVSeJV1q+Msn5lLJi4IO4ozWYvrIo7gkgiRXoRnLtXA9Ut5t2V8dyB24KHSDwaPk/mxXC6rYfEKKviYGZVwPnAicBuYBXwB3ffFmE26aQv9u5n2kOvxB2j2ayNOxhQ3JeT4w6S6eigc0PS7qOm23pIzA5bHMxsFvBt4ENgBVALFAHnAbeb2SrgH93944hzSgcNLO7DlrhDtPD53v1xRwgrOSH9SNChLiCZLRnpUdprORxN+url3a29aGajSfcuUnFImEElRQwqKeLOm8bEHaXZ80/pVl4i+eKwf63u/mBbr5lZH3d/K/eRJFe2pHbxzANvxB2j2bbyr1O8szbuGCKShay6sprZMjOrzJgeh65FSLRh445nYEVx3DEO0dCnjF0lX4o7hohkIdt2/r3AEjP7GelbYFwK6ExZgo08v5yR57e8W0m8Hp7567gj5JdNK5N37kE9qHqMrIqDu79gZt8EXgS2AGe6+6ZIk4n0ZF+eGneCMPWg6lGy7cr6j8CVwAXAKGCZmX3H3RM0aotIN1I1O3n/CSetFSORyvaw0kBgXNBr6RUzWwL8ClBxEBHphrI9rDS3xfRHpAfxERGRbkgdz6VL9WnYy0fXzYg7RrPp29bw0umNzE5g/4pJp0ziimG6X6zEI9IxpEUyfd6vPw19+sYd4xDlm/Zx4ZqCuGOE1G6rpfqD6vYXFImIWg7SZXYWl7KzuJST709Ol9aPrpvB6ZCoO8UCyRrzQnqk9u6tdEGW61mr+yuJiHQf7bUcsv368gy6v5KISLfR3r2V1LYVEemBdEJaRERCVBxERCRExUFEREKyvWX3o9nMExGR7iHblsPIzAkzKwDG5j6OiIgkwWGLg5ndYWY7gVFmtiN47AQ2A892SUIREelyhy0O7n6vu5cAP3L3/sGjxN0HuPsdXZRRRES6WHtXSFe6+9q2CoGZGVDu7qlI0km388Xe/Ux76JW4YzSbtXEHA4r7cnLcQUQSpr0rpH9kZr1IH0JaAdQDRcCpwIXAV4DvAioO0q6BxX3YEneIFj7fuz/uCPlFQ5f2GO1dIX2FmY0ArgGuBwYDu4F3SQ/083133xN5SukWBpUUMaikiDtvGhN3lGbPP6V7T2ZNQ5f2KO3+Zbj7auDOLsgiEosT6j9O1BgTkNxxJiadOytZY0wkrRXTjbTXW+ksMzshY3qGmT1rZj8zs+OijycSrZVfGs+mspPijhGSxHEmNMZEz9Jey+Eh4CJovn33D4BvAaOB+UAC25ki2VsxagIrRk3giZvOiTvKIZI4zoTGmOhZ2isOBe6+LXg+DZjv7r8Ffmtmb0UbTURE4tLeFdIFZnaggHwFWJrxms7kiYh0U+0Vh8eBfzezZ0n3UvoTgJmdCnzW3srNbKKZ1ZpZnZnNO8xyU83MzayqA9lFRCQi7XVl/b6Z/ZF0F9bfu7sHL/Uife6hTcH9lx4ELiZ9HcRyM1sc9H7KXK4E+Dbw2pH9E0REJNey6cr6aivz3sti3eOAOnf/AMDMFgFTgNUtlvufwH3A32exThER6QJRjudQDqzLmE4F85qZ2ZnAEHd/rr2VmdkcM6sxs5r6+vrcJhURkUNEWRyslXne/GL6thw/Br6Tzcrcfb67V7l7VVlZWY4iiohIa6IsDilgSMZ0BbAhY7oEOANYZmZrgbOBxTopLSISvyiLw3LgNDMbamZ9gOnA4gMvuvtn7j7Q3SvdvRJ4FZjs7jURZhIRkSxEVhzcfT9wC/AC6Rv1Penu75jZPWY2OartiohI50V6IZu7VwPVLebd1cayE6LMIiIi2YvysJKIiOQpFQcREQlRcRARkRAVBxERCVFxEBGREN12W3q81Rt3MO2hV+KOcYhZG3dQsS2VqOFLp29bw7tjB8LEuJNIV1BxkB5tyujy9heKwQsn/CWXACPiDpJh0PovgC1xx5AuouIgXWpLahfPPPBG3DGa9QPuGncSI89PVpGYBixkYqKGL13z1XFxR5AupOIgXWbYuOPjjhCyJbULIHHFQSRuKg7SZUaeX564/4ST1IoRSRL1VhIRkRAVBxERCVFxEBGREBUHEREJUXEQEZEQFQcREQlRcRARkRBd5yCSUEm759P0hkYKC/R9sqdQcRBJoCTe86mxyYGmuGNIF1FxEEmgq8efxNXjT4o7xiGeedrijiBdSG1EEREJUXEQEZEQFQcREQlRcRARkRAVBxERCVFvJRHJWhN7mb1kdtwxDrJPmORHc0XcObohFQcRyUqBl0DCerPW0gCGikMEVBxEJCuFlFLopSyYuCDuKM1mL6yKO0K3pXMOIiISouIgIiIhkRYHM5toZrVmVmdm81p5/TYzW21mb5vZH83s5CjziIhIdiIrDmZWADwIXAqMAK4ysxEtFnsTqHL3UcDTwH1R5RERkexF2XIYB9S5+wfu3gAsAqZkLuDuL7n7F8Hkq0BFhHlERCRLURaHcmBdxnQqmNeWG4Dn23rRzOaYWY2Z1dTX1+coooiItCbKrqyt9Yj2Vhc0uxaoAv66rZW5+3xgPkBVVVWr6xGRaJ1Q/zEfXTcj7hjNpm/azbsj1SM/ClHu1RQwJGO6AtjQciEzuwi4E/hrd98bYR4R6YSVXxoPwHEx58g0aHMTsD/uGN1SlMVhOXCamQ0F1gPTgaszFzCzM4GHgInuvjnCLCLSSStGTeDRsjGMGNw/7ijNZm6sUX/8iERWHNx9v5ndArwAFACPuPs7ZnYPUOPui4EfAcXAU2YG8LG7T44qk4gcuSQOXYpDU+tHq6WTIj1Y5+7VQHWLeXdlPL8oyu2LSO4kcejS6qdgr3mybgYITDplElcMy+87PqlFJiJ565jGXvT1ZN0NsHZbLdUfVLe/YMLpNL+I5K1jm4xjmwqSdTPAhLVijpSKg/R4W1K7eOaBN+KOETJs3PGMPD+Bx/mlR1BxkB5t2Ljj447Qqi2pXQAqDhIbFQfp0UaeX57I/4CT2JKRnkUnpEVEJETFQUREQlQcREQkRMVBRERCdEJaRPJake+BBV+NO8ZB9gkcXRZ3ik5TcRCRvPVZr1Jo2h53jEM1fB53gpxQcRCRvLW9YADbCwYwYva/xR3loIVVcSfICZ1zEBGREBUHEREJUXEQEZEQnXMQkbymca2jkf//AhHpsTSudXRUHEQkb60YNYEVoybwxE3nxB2l2ZqLR8YdISd0zkFEREJUHEREJETFQUREQlQcREQkRCekRRIqiWNba1zrnkPFQSSBkji2tca17llUHEQSKIljWyetFZNkX+DMXjI77hidouIgIpJDAygAGuOOcYjabbUdfo+Kg4hIDpVRQBkFLJi4IO4ozWYvmc0rvNKh96g4iIjk2J7NDcm639O2NSzs4HtUHEQkr63euINpD3XsW3GU7iov4Ki9DbBpZdxRmg3avKfD71FxEJG8NWV0sk7aAzxdeSFTh73CyMHHxB2l2ZrffNDh96g4iEjeunr8SVw9/qS4Yxxi2kMwfeNFjGjoH3eUZjO5ocPvifQKaTObaGa1ZlZnZvNaeb2vmT0RvP6amVVGmUdEJGpTRpczYnByCgMA3vG3RNZyMLMC4EHgYiAFLDezxe6+OmOxG4BP3f1UM5sO/BCYFlUmEekcXbXdviS2Zqqf6vh7ojysNA6oc/cPAMxsETAFyCwOU4DvBc+fBn5uZubuR1DnRCRKSbxqe8P729nw/nbee/2TuKMk2rbyucA3O/SeKItDObAuYzoFjG9rGXffb2afAQOALS1XZmZzgDkAJ52UrKos0hMk8artd/60XoUhC37UUR1+T5TFwVqZ17JFkM0y6Znu84H5AFVVVWpZiEgiC1YyjeEbv+zYdRdRnpBOAUMypiuADW0tY2a9gWOAbRFmEhGRLERZHJYDp5nZUDPrA0wHFrdYZjEwM3g+FViq8w0iIvGL7LBScA7hFuAFoAB4xN3fMbN7gBp3Xww8DDxqZnWkWwzTo8ojIiLZi/QiOHevBqpbzLsr4/ke4IooM4iISMdpmFAREQlRcRARkRAVBxERCVFxEBGREMvHnqNmthPo+Lh3yTCQVq4AzxPKHg9lj0d3y36yu5dlu4J8vWV3rbtXxR3iSJhZjbJ3PWWPh7LHIxfZdVhJRERCVBxERCQkX4vD/LgDdIKyx0PZ46Hs8eh09rw8IS0iItHK15aDiIhESMVBRERC8qo4mNlEM6s1szozmxd3nvaY2VozW2lmb5lZTTDvODN70czeD34eG3fOA8zsETPbbGarMua1mtfSfhb8Lt42szHxJW8z+/fMbH2w/98ys0kZr90RZK81s0viSQ1mNsTMXjKzd83sHTObG8xP/H4/TPbE7/cgS5GZvW5mfw7y3x3MH2pmrwX7/olgyAHMrG8wXRe8XpnA7AvN7MOMfT86mN/xz42758WD9G2//xM4BegD/BkYEXeudjKvBQa2mHcfMC94Pg/4Ydw5M7JdAIwBVrWXF5gEPE96NL+zgdcSmP17wN+3suyI4PPTFxgafK4KYso9GBgTPC8B3gvyJX6/HyZ74vd7kMeA4uB5IfBasE+fBKYH838J/Nfg+X8Dfhk8nw48kcDsC4GprSzf4c9NPrUcxgF17v6BuzcAi4ApMWc6ElOAfwme/wvwdzFmOYS7/wfhkfjayjsF+LWnvQqUmtngrkka1kb2tkwBFrn7Xnf/EKgj/fnqcu6+0d3fCJ7vBN4lPbZ64vf7YbK3JTH7HSDYh7uCycLg4cB/AZ4O5rfc9wd+J08DXzGz1oY6jtxhsrelw5+bfCoO5cC6jOkUh/8gJoEDvzezFWY2J5h3vLtvhPQfFzAotnTZaStvvvw+bgma0Y9kHMJLZPbgMMWZpL8F5tV+b5Ed8mS/m1mBmb0FbAZeJN2a2e7u+4NFMjM25w9e/wwY0LWJD2qZ3d0P7PvvB/v+x2bWN5jX4X2fT8WhtQqd9H6457r7GOBS4GYzuyDuQDmUD7+PXwB/AYwGNgIPBPMTl93MioHfAre6+47DLdrKvKRlz5v97u6N7j6a9Bj344DhrS0W/ExU/pbZzewM4A7gdOAs4Djg9mDxDmfPp+KQAoZkTFcAG2LKkhV33xD83Aw8Q/rD98mB5lzwc3N8CbPSVt7E/z7c/ZPgD6gJ+N8cPISRqOxmVkj6P9ffuPv/CWbnxX5vLXu+7PdM7r4dWEb6eHypmR2471xmxub8wevHkP2hzMhkZJ8YHOpzd98LLKAT+z6fisNy4LSgJ0Ef0ieEFsecqU1mdrSZlRx4DvwNsIp05pnBYjOBZ+NJmLW28i4GZgS9IM4GPjtwGCQpWhxT/Rrp/Q/p7NOD3idDgdOA17s6H6R7kZAeS/1dd/9fGS8lfr+3lT0f9juAmZWZWWnwvB9wEenzJi8BU4PFWu77A7+TqcBSD872drU2sq/J+EJhpM+VZO77jn1u4jrbfiQP0mfc3yN9XPDOuPO0k/UU0j0z/gy8cyAv6WOUfwTeD34eF3fWjMyPkz4MsI/0N40b2spLupn6YPC7WAlUJTD7o0G2t4M/jsEZy98ZZK8FLo0x93mkm/dvA28Fj0n5sN8Pkz3x+z3IMgp4M8i5CrgrmH8K6aJVBzwF9A3mFwXTdcHrpyQw+9Jg368CHuNgj6YOf250+wwREQnJp8NKIiLSRVQcREQkRMVBRERCVBxERCRExUFEREJ6t7+ISM9lZge6lAKcADQC9cH0F+7+V7EEE4mYurKKZMnMvgfscvf7484iEjUdVhI5Qma2K/g5wcz+3cyeNLP3zOwHZnZNcL/9lWb2F8FyZWb2WzNbHjzOjfdfINI2FQeR3PhLYC7wZeA6YJi7jwN+BXwrWOanwI/d/Szg8uA1kUTSOQeR3Fjuwb1qzOw/gd8H81cCFwbPLwJGZAwB0N/MSjw9FoJIoqg4iOTG3oznTRnTTRz8O+sFnOPuu7symMiR0GElka7ze+CWAxMHxvcVSSIVB5Gu822gKhilazXwzbgDibRFXVlFRCRELQcREQlRcRARkRAVBxERCVFxEBGREBUHEREJUXEQEZEQFQcREQn5/5shoYIgLSpUAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "surv.iloc[:, :5].plot(drawstyle='steps-post')\n", "plt.ylabel('S(t | x)')\n", "_ = plt.xlabel('Time')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is, therefore, often beneficial to [interpolate the survival estimates](https://arxiv.org/abs/1910.06724).\n", "Linear interpolation (constant density interpolation) can be performed with the `interpolate` method. We also need to choose how many points we want to replace each grid point with. Her we will use 10." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "surv = model.interpolate(10).predict_surv_df(x_test)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZxU9Znv8c9DC7YKyKaANAhEkUWUAEI0aHDigq0TrlEUjVHRDMlczTJZbpLxjlFnRk0mxpiZ3EnMgqOZuOY6cgnjEpWJcRKlVRRBQIIgjYBACw02DU3zu3/UqeL0qepaus6pOlX9fb9evOhTdbrqZ4E8/TzPbzHnHCIiIn49yj0AERGJHwUHERFJo+AgIiJpFBxERCSNgoOIiKQ5rNwD6IpBgwa5kSNHlnsYIiIV45VXXtnunDsm3/srMjiMHDmShoaGcg9DRKRimNmGQu5XWUlERNIoOIiISBoFBxERSVORPQcRkXJoa2ujsbGR1tbWcg+lU7W1tdTV1dGzZ8+iXkfBQUQkT42NjfTp04eRI0diZuUeThrnHDt27KCxsZFRo0YV9VqRlpXM7Jdm9r6ZvdnJ82ZmPzKztWb2hplNjnI8IiLFaG1tZeDAgbEMDABmxsCBA0PJbKLuOdwHzMry/AXAid6v+cC/RjweEZGixDUwJIU1vkjLSs6535vZyCy3zAbud4l9w/9kZv3MbKhzbnO2123auJ1fffkXAPQbVcNFX7k2pBGLiAiUf7bSMGCj77rReyyNmc03swYza+CAw5oO8GHLYHaubCvJQEVE4uLJJ5/kpJNO4oQTTuDOO++M5D3K3ZDOlP9kPH3IOXcvcC/AmKOPcoOafgF9rgNgw2evTt3X96KL6H/5ZeGPVEQkBtrb27nhhht45plnqKur47TTTuNTn/oU48ePD/V9yp05NALDfdd1wHu5vqlp8BE89MUJbOlvtPYyVm5uZuXmZpreeJO3H/xNZIMVESm3l19+mRNOOIHRo0fTq1cv5s6dyxNPPBH6+5Q7c1gI3GhmDwHTgV25+g0AI/uOZMGsBdz+7M/Y0PsgT81YD8C1D+/nuHUrlEmISORu/X8rWPlec6ivOf64vnznLydkvWfTpk0MH37oZ+q6ujpeeumlUMcBEQcHM3sQmAkMMrNG4DtATwDn3E+AxUA9sBZoAeYV8vqDDj+avR8cxeS3bwDgrTEHaT66gdPZB0DL0qW0LF1K86JFgAKFiFS+xPydjqKYQRX1bKUrcjzvgBu6+vofP//jrHl5KzAUgHfXbWbFR+Du8U8D8IkBQ5n15x58BGhdtQpAwUFEQpHrJ/yo1NXVsXHjoXk8jY2NHHfccaG/T7nLSkWZcOYwJpx5aHLT//naLzjSOY5v+zMAS05t542JA3nyc/ez4bNX07pqlUpOIlLRTjvtNN5++23eeecdhg0bxkMPPcSvf/3r0N+nooND0NA+Qxn64Ta+3TMRMK7e/2dW99rBvCfnceqo7czYO5Ba795gyQkULEQk/g477DD+5V/+hfPPP5/29nauu+46JkwIP4upquBAnyFs39Wbx5v+HoBpja9y9MAGXmYbr47ezjMnn8BvL7sfgA8efqRDYFDZSUQqRX19PfX19ZG+R1UFhzHTBne47rXvOD7x/kS+0ONBvjNgH++4t5j3pNfzPhrq/+4vmTNmDoDKTiIiPlUVHII9iMdvWQwf9mbC0KO5uGkTT/Q4OvXc6qbVAKng0Peiizq8ljIJEenOqio4pOkzJPFr3jVMun0GY3a38303H4D2Xt9n+bb8MwllESLSnVR3cAC2N+7h8bteZf/26zmx5++4ecc3AFh8eCvPHt07dV+2TELNaxHpbqo6OPh7EM0HhrOh5jwuHvoIACPfXUf9vtFM+NwCAOY9OY/VTaszZhJqXotId1PVwcHfg3j8rleBATDvGgDW3z6Dlv3tXP7TPwLwQc1Y+vfdl/pefybR//LLOgQCNa9FpNpVdXDIZlDvw9m+51Aw2Nw4ifFDz2LBZacD6ZlE/ej6TpvX2qZDRErluuuuY9GiRRx77LG8+WbGQzZD0a2CQ7L/AMCWqxnT83c8PPQfAFjRaxcvtpwNJIJD/ehDc4iD/YhgJuEvO6nkJCJRuvbaa7nxxhu5+uqrc99chG4THIJrILbvGwacwwS8HkTbOq/MlAwKdcyedCtXTh+R3o+gYybhDxb+UpOISNjOOuss1q9fH/n7dJvgkLYGItCD2POjT3Kkr8y0cnNiK94rp4/okEVAeiYRpH6ESDfwn9+CLcvDfc0hE+GCaE52K1S3CQ65DO5Ty+AP3+bhXullpjlj5nQIBNkyCS2mE5Fq0K2DQ4cexO4bGHPE75lA4ieBYJlp9qRhXDl9BEDWTCLTzCYRqUIx+Qk/Kt02OKT1IHb1hj6XMWFe4g/cX2byl5iAnJmEvx8hIlKJum1wyNyDOMRfZlrRaxePbT6dy3966PnOMolM/Qj1IEQkLFdccQVLlixh+/bt1NXVceutt3L99deH/j7dNjjkNPHS1JcnHlzPpb3gNj4NZM8kgllE8BwJ9SBEpBgPPvhgSd5HwcGnQw+CUxkz7TwmnDmMXgsuZALw8LzEGojkqupMgv2IR8bu5PUzTmLBrMQ2HepBiEglUHDwpPUgGvcAHCo9bVkOCy4E4OYdu3hsf+YyU6Z+RJDKTCISdwoOnqw9CF+JCXKXmYJUZhKRSqPgkI+p8xK/PCoziUi1U3DIwt+DGDNtcIfMImjl5uZUkPDPZCq0zKQSk4jEgYJDJ/w9iLT+A3ToQfxoXytPHH0Gz1Kfs8QEnZeZVGISkbjoUe4BxNWEM4dx8dcmc/HXJjOornfHJydemtgDxTP4w7eZ3+9VHv786Ywf2jfr69aPruekASelrh8Zu5Mff24Ixz9wP7Vjx4b63yAi1Wfjxo2cffbZjBs3jgkTJnDPPfdE8j7KHLoi0INIZhD5yFVm0kwmEcnmsMMO46677mLy5Mns3r2bKVOmcO655zJ+/Phw3yfUV6tiHddAZOhBeGWmm3fs4sUjDp0LkY9kmUkzmUQkl6FDhzJ06FAA+vTpw7hx49i0aZOCQznkXAPhm+o6sm1dQa/tn82kmUwileO7L3+XVU2rQn3NsQPG8s1p38z7/vXr1/Paa68xffr0UMcBCg55ybUPk7/MFDybGjrOXgoKbr0hIpKPPXv2cMkll/DDH/6Qvn2z9zq7QsEhZIN6H07vD97i5h3fAKBlfzuvtZwD0/8+r+/3z2Sa27SKYVva1IMQiaFCfsIPW1tbG5dccgmf+cxn+PSnPx3Jeyg4dFFnPYjBZ1wFyx9jgvf4h+++xpF7n8/rNYML5p4f287Z9CQ5h0k9CBFxznH99dczbtw4vvrVr0b2PpEGBzObBdwD1AA/d87dGXh+BPBvQD/vnm855xZHOaYwZO1BBGYyrb99Rt6vmzaTiXk8dC7qQYhIyosvvsgDDzzAxIkTmTRpEgC333479fX1Ob6zMJEFBzOrAX4MnAs0AkvNbKFzbqXvtv8NPOKc+1czGw8sBkZGNaaw5OxBBIxsW9dxuuvESztOhc1CZSYR8ZsxYwbOucjfJ8pFcNOAtc65dc65/cBDwOzAPQ5IdlKOBt6LcDxl8eIRZ7O+5+hDD2xZDssfy+t7gwvmnh/bzqYhPVPXratW0bxoUWhjFRFJirKsNAzY6LtuBILzrW4BnjazLwJHAed09mJmNh+YDzBiROdbU8TNs0fW88+7ZjB+fyIG3uy+waDdrQzO8X2gMpOIlE+UwcEyPBbMha4A7nPO3WVmpwMPmNnJzrmDad/o3L3AvQBTp06NPqcqUGeb9M2e1HGzvpb97Wzfsy+v4JBJtjKTSkwiEpYog0MjMNx3XUd62eh6YBaAc+6PZlYLDALej3Bcocu2Sd+V00d0WOOw4vaaLvcgss1m0kwmEQlTlMFhKXCimY0CNgFzgSsD97wLfBK4z8zGAbXAtgjHFAl/gzpXczqxtQapqa5sWZ74PY/gkKnMdOuU1Zw0oIa5/wwD927j+IJHLyKSLrLg4Jw7YGY3Ak+RmKb6S+fcCjO7DWhwzi0Evgb8zMz+hkTJ6VpXijZ8GT17ZD3PHlmfOiiokE37gvyZRMuBvRy7dqNmMolIKCJd5+CtWVgceOxm39crgY9HOYZyyLVJn/9goJt37OLEg+vplQwSBUxz9WcSP/zveo58ZTsDvOdUZhKpTq2trZx11lns27ePAwcOcOmll3LrrbeG/j5aIR2yXJv0BRvUj+0/nUt7eWWmAkpMQa+fMZhHxu7kpAE1ACoziVSpww8/nOeee47evXvT1tbGjBkzuOCCC/jYxz4W6vsoOIQs1wK5YIP68p/CbXw6UWYKqcQEiTITe3d0+fVEJJ7MjN69EweQtbW10dbWhlmmyaHFUXCIG9/xo0DeZaZgs/qpf57GsZta1IMQiciW229n31vhbtl9+LixDPnbv815X3t7O1OmTGHt2rXccMMNkWzZrWNC4yRw/Gghq6mD3poyiPWDjVVNq1jVtIqm5a/y50cXhDRQESmnmpoali1bRmNjIy+//DJvvvlm6O+hzKEEcp4il1TE8aNBw66ax8IzDs0F+NTdDRypMpNIaPL5CT9q/fr1Y+bMmTz55JOcfPLJob62gkPEcp4iR8fZS2kHA4VYZhKRyrdt2zZ69uxJv3792Lt3L7/73e/45jfDP1tCwSFiuRrU/tlLKzc3AxwKDr7jR4GiZjMBDGrcw1MXJoJEzflnc86Xvtul1xGR8tm8eTPXXHMN7e3tHDx4kMsuu4yLLroo9PdRcCgz/+wl/9GiQKhlpprzz2b7U4lDhwY17kl8/aUuv5yIlMkpp5zCa6+9Fvn7KDh0E+d86bupYJDMHkREOqPgUAZ5N6gz6WIPIqjlwN7U7q6QWCfh71GISPem4FBi+TSoOxVSD2LgEQM7LJBb3bQaQMFBRFIUHEosV4PaP3MJArOXQupBHHPEMfTZsINb/r0dgFVNB3hrylZv83QREQWHWAnuu5Q2eykTf5kpzxJT38DMhmM3tQDbCxqriFQ3BYcYSPYgjgBunjYilVmkzV4K8peZCigx9b/8sg7baKy6cJp6ECLSgYJDmWU7RS4nf5mpiGmu6kGIVJb29namTp3KsGHDWLRoUSTvoeBQZoWcIpdTF2cyHXPEMRxzxDEsmJXYe8mfQYhI/Nxzzz2MGzeO5ubmyN5DwSHmsjao/YqcydS6alVqB9e5Tat4fmw781CZSSRuGhsb+e1vf8tNN93ED37wg8jeR8EhxgpqUBcxkynYoB62pY2z6clD5yauVWYSSffCI2vYvnFPqK85aHhvzrxsTNZ7vvKVr/C9732P3bt3h/reQQoOMeNfIFdwgzoozzJTsEG94bNXMxZUZhKJmUWLFnHssccyZcoUlixZEul7KTjESFEL5IKCZaYNf0j8Sp4PUeDK6tVNq1NBQiUmEXL+hB+FF198kYULF7J48WJaW1tpbm7mqquu4le/+lXo76XgECO5FsgVJFhmalhwKDAEAwWkBQt/D+KGvdv4w/h+vH6GSkwi5XTHHXdwxx13ALBkyRK+//3vRxIYQMGh+/AHC3+ggLRg0XdAMwzrl3q6z4YdXHzEWL5y2wKVmES6CQWHCpP37KVssmUVQP/+b9C/P3D8DAA2bGmD3VtSz/tLTKAyk0g5zJw5k5kzZ0b2+goOMedvUP/FbseQnkeS/Gc6r+018pEjWNC6K/FrwYXUswd6H516SmUmkeqk4BBjwQZ1j10HmFnXm4s/PxnowuylfAWDxRPnwYfbAJizYRlzIJVVzLP9HbIKEakOCg4xFmqDuhh9htC6aScbnhsIu6fT9/i99D/ee651F6sP7mXefVNTt9cfdyZzzru7PGMViZhzDjMr9zA65ZwL5XUUHCQn/yK51k07oc9Y+s+7H4D6p/8G3nsh9fzqg3th3SLmLFiT+cW6eDiRSBzU1tayY8cOBg4cGMsA4Zxjx44d1NbWFv1aCg6Sk3+RXHJ6a9Kc8+7G322Y9+gFiRJUph9e8phCKxJndXV1NDY2sm3btnIPpVO1tbXU1dUV/ToKDhXG36A+ZfM+Xu+xP9V76NLMpbD1GcLqtl3MG3AsEJjJlGMKLaBgIbHWs2dPRo0aVe5hlISCQwUJNqj774dTe/XiDUKcuVSk+tH1qa/TZjLlmhXVxWNPRSR8FlbzIuOLm80C7gFqgJ875+7McM9lwC0kChGvO+euzPW6U6dOdQ0NDSGPtvIkM4iLvzY5lT08/PnTI33PDZ+9mtZVq6gdOzb1WN+LLuqwN1NSci1Eco+mnBZcmAgQQyYmrpVFiITGzF5xzk3NfWdCZJmDmdUAPwbOBRqBpWa20Dm30nfPicC3gY875z4ws2OjGo+EI7iDa+uqVQAZg0PB/PtBqeQkUlZRlpWmAWudc+sAzOwhYDaw0nfPXwE/ds59AOCcez/C8VSlZA/ilM37eL9/TeTvl2kH12wKWk1dwBYfgIKFSISiDA7DgI2+60ZgeuCeMQBm9iKJ0tMtzrknM72Ymc0H5gOMGFHmpmtM+HsQvfcepGXfgeK31giRv/8ABa6mztWfKHKXWRHJLsrgkGkScLDBcRhwIjATqANeMLOTnXM7077RuXuBeyHRcwh3qJXJv0jup7f8N0fu2Z96Lg4N6jlj5nQIBEVt2lfkLrMiUpgog0MjMNx3XQe8l+GePznn2oB3zGw1iWCxNMJxVaVj+9RybJ9abop6a40M/Nt7Q+cNaghx0z6VoEQiFWVwWAqcaGajgE3AXCA4E+k/gCuA+8xsEIky07oIxyQhK6RBHSwzNWxtoGFrA4vXLe5wT8HBotASFChYiOQQWXBwzh0wsxuBp0j0E37pnFthZrcBDc65hd5z55nZSqAd+IZzbkdUY6p2wQVycWtQB8tMj655tENgCG2HV/UrRIoW6TqHqGidQ7oVL2xizctbU9fvrtvJniN6cNP3Z5Z0HMngcPwD9xf8vfOenMfqptWcNOAkIMJzIoL9CkjtMgsoWEhVKnSdQ17BwcymAmcCxwF7gTeB3znnmro60GIoOOT2j19fQsu+A6w9+ajUY6WYvRRcJJet/xDkzyQatib+fKcO9u32GkWwyJRVQMdgEaTgIRUo1EVwZnYt8CXgHeAVYDVQC8wAvmlmbwJ/55x7t8sjlkgM6t2L7b7rUs1e6rCDa4EL5Pxlp2DJKbT+RFCuElRQpv6FnwKHVImsmYOZ3UCiV7C3k+cnAQOdc89GNL6MlDnk5t9aAxKzl1Zubmb80L5A6bII6FqJKShTf+KkASflvzVHWLIFj1xZhwKHlFEkZaVO3qiXc25/7jvDp+CQ2+N3vcr2xj0MqusNwPu7W1nZs50tgw5LBYlS7MME4QSHoJL1JwpRTOAABQ+JVCR7K5nZEuBa59x673oa8DPg1C6MUUog2xGjpVwDERX/tNjISk6FCpao/LpSrlKwkDLKtyF9PondVX9EYluMC4DPOefKcm6lMofCBXdw9ZeYIJoyUyE7uBYjNiWnYuRqjCtQSJEiyRycc0+Z2ReAZ4DtwEedczpVvkLNnjSsw3VUzepId3D1ybRNR2grsUslW2Nc51xIGeSbOfwdcBmJje9OAf4G+Jpz7rfRDi8zZQ6FC/YgxkwbnNqXqZSZBETTg/DLNNMJSjAtNirBcy5AmYQULKrzHAYB07xZS380syeBnwNlCQ5SOH8PYnvjHoBUcChVJlEqJVuJXSr+cy5A/QkpCa2Q7oaC01yDosokSpU55BLLmU6FyHS86pCJME8/q0nnYnMSnFSuKDOJQnZwjUosZzoVItifSJadFlyYuFYWISFQcJA0V04f0SEQhDX1Ndigblm6lJalS2letKjDPVEHi2wrsSuu5AQdy05qXktIVFbqhoLNaejYoA6KanX1Bw8/0iEwJKe9lrPsFCw5QQVkEn5qXksnwt5b6aw8X2e99leqHMEFcsEGdZC/zBRmiSnTdt/+slO5S05QgWWnYPNamYR0Ua69lfJdRfS4dz5DSShzCFeuBrVflNNe/ZlEHLIIqIIFdsokxBNq5uCc098g6SDKZrU/kwhmEVCeTCLXArtYZxGgTEK6TD0HKbgH4RdVJhHsR7QsTRwrfuRpp6UeK0ewKMuZE2FSJtFtaSqrFKzQHoRfVJlEsB+RqXmdvK+UKn6mkzIJyZMyB0lTSA8iKDnttRTbgXf1xLmoVORMp2AmoSyiakW1ZfcDzrnP5npMBBLZQzJIRHWokH/NRLnWSwTlmukUy0ChNRLSiXw33nvVOTfZd10DLHfOjY9ycJ1R5hCtYnoQv37pXZ5YtgmgZIcKxXG9BHQsO1XELCf1I6paqCfBmdm3gb8FjgBakg8D+4F7nXPfLmKsXabgEK0VL2xizctbU9fJQFFomalUu70GleociUJURMlJezZVtbCnst4B3GFmd5QrEEjpTThzWIcsIdmDKFS5dnst1TkShaiIxXW59mwCZRLdSK7MYWTyaNBOnjdgmHOuMYKxdUqZQ2kV06D2UyZxSEUsrlMmUVXCbkj/k5n1AJ4AXgG2AbXACcDZwCeB7wAlDQ5SmeKSSQQb2HFcXAcVkEkoi6hqucpKc8xsPPAZ4DpgKLAXeIvEQT//6JxrjXyUUnbbG/ekMoh8m9NBmXZ79c9sgmgyiWxrJuJQcoL0slMs10xoZlO3onUOkpO/Qd3V5nQm/plNULrZTX5xLDlBBRxIlMweVGKqGGHvynoasNE5t8W7vhq4BNgA3OKcaypmsFIZ/A3qrjanM4nq3IhCxLF5DR0ziVhmEVL1cvUcfgqcA6ntu+8EvghMAu4FLu38W0UKV4oFdH65tg2H8vck/H0IkVLJFRxqfNnB5STWNvwG+I2ZLYt2aBJX/v4DdL0HERTVuRGFiGsmIVJqOYODmR3mnDtAYmbS/AK+V6pQMZv05eIvM5WqWR0U10widjOZpOrl+gf+QeC/zGw7iVlKLwCY2QnArlwvbmazgHuAGuDnzrk7O7nvUuBR4DTnnDrNMRbWArlcyjXtNSgOmURsZzJpgVxVyzlbycw+RmIK69POuQ+9x8YAvZ1znf7L4O2/tAY4l8Q6iKXAFc65lYH7+pCYFtsLuDGf4KDZSvER1gK5XMq1gC4oDrvBJjOIsi6Y0wK5ihP6rqzOuT9leGxNHq89DVjrnFvnDewhYDawMnDf3wPfA76ex2tKNxXHTKKcu8GWvcyUaYGcVJUo+wbDgI2+60Zguv8GM/soMNw5t8jMsgYHM5uP1/MYMaK0/yBIdlE1qP1yLaArVRbh70mU6wCi2JaZpKpEGRwsw2OpGpa3LcfdwLX5vJhz7l4S02eZOnVq5a3cq1JRNqizicPMpnI1rzNtvSEStiiDQyMw3HddB7znu+4DnAwsSezfxxBgoZl9Sk3pylGqBnVQcGZTHMSheS0SliiDw1LgRDMbBWwC5gJXJp90zu0CBiWvzWwJ8HUFBumKckx7DcqVSUTZj/D3IMo2zVWzl6pKZMHBOXfAzG4EniIxlfWXzrkVZnYb0OCcWxjVe0t5hbFJXyHi0qwO8mcSUWYRsdhqY2JgswRtzFfxtPGehCqqTfoKEZdpr36l2uAvFtNcQRvzxVDoU1lFChHVJn2FiGMmoX6EVBoFB6k65To3IptSzmwq+xoIqQoKDlL1ulMmoTUQEhb1HCQyj9/1aqrvkFSKBnUuwZ5EufsREF1Pomw9iOSRokMmHnpMs5fKSj0HiY1yLZDLJQ4L6IKqrieh2UsVT5mDlEypNukrRBxnNkF4G/wFjxuFMvUgNHup7JQ5iBQgjv0ICG+NhHoQ0lUKDlJSpdikrxBxnNkEHWc3+Wc0FUr7MElXKThIycS1B+EXzCReeqeJl95p4ollmzrcU+pgEYfT6KR7UXCQkinXJn2FCGYSv37p3Q6BIRgsShEowm5Wax2E5EPBQcoqbmWmoGzBolRZRaYFdF1V1h6Ef2M+TWuNPQUHKZtKKDMF+YNFrqwCosssulpmKlsPwj+1VdNaK4KCg5RNJZSZsilXCaoi10T4jxXVkaIVQcFBJCTZgkWYU2TDLDNBTM6CkNhRcJBYKfVZEFEKnlYX5RTZrh4qFIuzICSWFBwkNvw9iEroPxQiyimyxSyY8/cgtAZC/BQcJDbicBZEVArtT0D+wSKsBXMlpSNFY0/BQWIr7tNcixFlM7uYBXMlWQOhTfkqgoKDxFIlTnMtRljN7GJmMpVsDYR/5hJo9lJMaVdWqQhx3NG1VIrZOTaZQRz/wP0Fv2/JzoLQjq0loV1ZRapMsTvHal8m6QoFB6kY1dyDyKaYnWOLXTCnfZi6LwUHqQjdrQeRTSGZRDEL5nQWRPemnoNUpO7cgwgq5EzsYs6qjqwHofOmS0I9B+k2qmk1dTEKORM7lvsyaWprLCk4SEWq5tXUhSpkm45i92WKZB8mTW2NJQUHqUjVvJq6GF3ZpiPffZm0D1P3ouAgVaG7zmQKyrXyOlh2KmRfpuA+TJrJVN0UHKTiaSZT53JPgx3O7Bv/gSunjyhqJlPD1gYatjaweN3iDvcoWFQuBQepeJV+aFAp5ZoGm++CueCJco+uebRDYCg6WGhjvrKLNDiY2SzgHqAG+Llz7s7A818FPgccALYB1znnNkQ5JpHuLFsmMaXvOGYO20et91whM5kKDRZZA4VmL8VCZMHBzGqAHwPnAo3AUjNb6Jxb6bvtNWCqc67FzP4a+B5weVRjku5DPYj8+DOJB46ZzCunzOThz58OFLf9d7ZgkbOZrdlLsRBl5jANWOucWwdgZg8Bs4FUcHDOPe+7/0/AVRGOR7oJ9SDyl20a7LWbm6lragxlXyY1sytPlMFhGLDRd90ITM9y//XAf3b2pJnNB+YDjBgRztGKUp3Ug+iaYD/iqSGncj4w3rsOa8GcmtmVIbLtM8xsDnC+c+5z3vVngWnOuS9muPcq4EbgE865fbleW9tnSCEev+tVtjfuYVBd79RjKjPlFtyW49pHv0tdUyN9JyTCRVi7u2bqTwBMHezt9LBlOfXuKObM+6+i36s7i9P2GY3AcN91HfBe8CYzOwe4iTwDg0ihVGbqmmyZRJjbbuRsZts+GlZ/t1cAAA0bSURBVGwfi8NemS1ZRZk5HAasAT4JbAKWAlc651b47vko8Bgwyzn3dr6vrcxBiqFN+7rGn0kEswiI7pyIRxd8gsVt70Ovo2iwxM+PqawCBYt8FZo59IhqIM65AyRKRU8BbwGPOOdWmNltZvYp77Z/AnoDj5rZMjNbGNV4RKQ4sycNS5WYnhpyKo0D6lLPta5aRfOiRZG875yJ17Kg52gWuMHcvLOFqe7w1HOrm1Z3yDIkPNqyW7qdYA9C/YfCJWc0+ae9dnUr8IIEjhRNznw6acBJqVuUSWQWp56DSCxpR9dw+Ke9FrOArhg6kCg6Cg7S7WhH1+IFm9WZFtCV4uzqYDM7uIZCWUTXKThIt6fV1IXLtA2HX7kOFdK24uFRcJBuTdNcw9PZbq+QnkkUlUVk2ZQvuBJbuk7BQbq1TKuplUkULtdur4WcG5FVgZvyaZuOrlNwEPFRJtE1ucpM/uNJi+pHFLApn5rVxVFwEPFRJhGezs6yLlU/IlezGpRJZKPgIJKFMomuyVZm8mcRULqZTcokCqPgIJJFrkxCWURmucpMfuXMJKRzCg4iBdACuq7zl5mSJSag+EyiiCNFVWbqnIKDSAGCC+jUj8iPv8wUnMkUVFAmUcSRoiozZae9lUS6aMULm1jz8tbUdXK/Ju32ml1wX6Zcgvs2Zc0iFlyYCBBDJiauC8gigvs0VVsWob2VREpEM5u6rrOZTJkUtEbCn0kUkEWAVlcHKXMQCYkyifz8+qV3eWLZptR18oyIfDKJgnZ/DWYRkHcmkexDLJi1IOe9lUKZg0iZaGZTfgqZyRRUqn4EqFmt4CASEc1sCl9BM5syrabOc2aTmtUqK4mURPCAIVAmkeQ/fjQpWw/C74OHH+lwAl3L0qUAHHnaaUCg5NSwAJY/duibkyUn7+CgbKrhUCGVlURiSCutO5dr075sgpmEP1i0LF1Ky9KlHYJH34uuOXS/MomslDmIlIEyic4VOtW1M7myCnZvoe/xe+k/yctYCswkoLIa1socRCqAMonsCpnq2plsWQVA66ad0Gcs/efdn3ggmEnkmNlU7SfOKTiIlIHWSHSumDJTNjmb2bvb6Hv88fQfQs6ZTd1hTYSCg0gMKJM4pJiproUITottWbWRllXQ3DQKtgykb91b9Kf7njinnoNIDAV7Et01i4D02UxdKTHlI9jMBjhyeG3iyf0f0nfSYPrf80La91XKTCb1HESqgD+TeO/tnbz39s4Oq6+7U7AoZNO+YvjLTpma2S1Pbaf53I+mHut77ln0/1/3VO1MJmUOIjEX3Jbjvbd3AnDcif1Sj3WXYBHWTKZCffC9L9P8zO9T1y0bW4HM6ynimkkUmjkoOIhUmFzBopoDRTEL5sL0wZfPpHnZVuh1VFqg2LZ3G38Yb7x+RiL7a9ia+Ldq6uBD/y6XI1goOIh0M/5gUe1ZRTGb9oXKt9r6g2dfpXnDEVB7NACt7++ndtQwjv+PpwF4dM2jLF63+NC3BoJFqQKFgoNIN9bdSlDlKjN1ENiWY8N9b9O6sye1p0zJeLs/syhlVqHgICIp1R4sMpWZsilFCcpfcko56hjoMwTouFJ7295t7Ni7I3Xb7rbd/GF8D3bNmpbxtYsJHAoOItKpautXBMtM2bz0ThMA00cNCOW9Ow00wQ3+Nvwh8fvxMwD4YFkzzSv3ZHzNZP/i3Y/0ST321pRBoWQZCg4ikrfu3K8oRiGB5pMti/n43ufzet3jVv65Q/8i1eweXss22tlR0wNqegGZs4xswSJWwcHMZgH3ADXAz51zdwaePxy4H5gC7AAud86tz/W6Cg4i4av2ElSYwgw0fqPffZTZNf9Nn1pvCdqaVuydfQAcdfBDAD7skShX2dYDALxTZwC09Ej8W37kQUu93oYxI1hy6ncAeOQLZ8QjOJhZDbAGOBdoBJYCVzjnVvru+Z/AKc65L5jZXOBi59zluV5bwUEkevkEi7Ao6CRkCzppGYgvcAB80MOxq+Zg6npUY+Lf9nfqDBxc+NxbsQkOpwO3OOfO966/DeCcu8N3z1PePX80s8OALcAxLsegFBxESi8YLMISZdDpzj5YvwLX3AJA7T7HVQ//dWy2zxgGbPRdNwLTO7vHOXfAzHYBA4HtwRczs/nAfIARI0q74EVE0neSDUtUQae76z9yQurrHWtXFPz9UQYHy/BYMCPI557Eg87dC9wLicyhuKGJSFxEFXTEbzJ/9ZOrc9/m0yOikUAiUxjuu64D3uvsHq+sdDTQFOGYREQkD1EGh6XAiWY2ysx6AXOBhYF7FgLXeF9fCjyXq98gIiLRi6ys5PUQbgSeIjGV9ZfOuRVmdhvQ4JxbCPwCeMDM1pLIGOZGNR4REclfpOc5OOcWA4sDj93s+7oVqOxNz0VEqlCUZSUREalQCg4iIpJGwUFERNIoOIiISJqK3JXVzHYDq8s9ji4aRIYV4BVCYy8Pjb08qm3sxzvnjsn3BSKdrRSh1YXsERInZtagsZeexl4eGnt5hDF2lZVERCSNgoOIiKSp1OBwb7kHUASNvTw09vLQ2Muj6LFXZENaRESiVamZg4iIREjBQURE0lRUcDCzWWa22szWmtm3yj2eXMxsvZktN7NlZtbgPTbAzJ4xs7e93/uXe5xJZvZLM3vfzN70PZZxvJbwI+/P4g0zm1y+kXc69lvMbJP3+S8zs3rfc9/2xr7azM4vz6jBzIab2fNm9paZrTCzL3uPx/5zzzL22H/u3lhqzexlM3vdG/+t3uOjzOwl77N/2DtyADM73Lte6z0/MoZjv8/M3vF99pO8xwv/e+Ocq4hfJLb9/jMwGugFvA6ML/e4cox5PTAo8Nj3gG95X38L+G65x+kb21nAZODNXOMF6oH/JHGa38eAl2I49luAr2e4d7z39+dwYJT396qmTOMeCkz2vu4DrPHGF/vPPcvYY/+5e+MxoLf3dU/gJe8zfQSY6z3+E+Cvva//J/AT7+u5wMMxHPt9wKUZ7i/4700lZQ7TgLXOuXXOuf3AQ8DsMo+pK2YD/+Z9/W/A/yjjWDpwzv2e9JP4OhvvbOB+l/AnoJ+ZDS3NSNN1MvbOzAYecs7tc869A6wl8fer5Jxzm51zr3pf7wbeInG2euw/9yxj70xsPncA7zPc41329H454C+Ax7zHg5998s/kMeCTZpbpqOPIZRl7Zwr+e1NJwWEYsNF33Uj2v4hx4ICnzewVM5vvPTbYObcZEv9zAceWbXT56Wy8lfLncaOXRv/SV8KL5di9MsVHSfwUWFGfe2DsUCGfu5nVmNky4H3gGRLZzE7n3AHvFv8YU+P3nt8FDCztiA8Jjt05l/zs/9H77O82s8O9xwr+7CspOGSK0HGfh/tx59xk4ALgBjM7q9wDClEl/Hn8K/ARYBKwGbjLezx2Yzez3sBvgK8455qz3ZrhsbiNvWI+d+dcu3NuEokz7qcB4zLd5v0eq/EHx25mJwPfBsYCpwEDgG96txc89koKDo3AcN91HfBemcaSF+fce97v7wOPk/jLtzWZznm/v1++Eeals/HG/s/DObfV+x/oIPAzDpUwYjV2M+tJ4h/Xf3fO/V/v4Yr43DONvVI+dz/n3E5gCYl6fD8zS+475x9javze80eTfykzMr6xz/JKfc45tw9YQBGffSUFh6XAid5Mgl4kGkILyzymTpnZUWbWJ/k1cB7wJokxX+Pddg3wRHlGmLfOxrsQuNqbBfExYFeyDBIXgZrqxSQ+f0iMfa43+2QUcCLwcqnHB4lZJCTOUn/LOfcD31Ox/9w7G3slfO4AZnaMmfXzvj4COIdE3+R54FLvtuBnn/wzuRR4znnd3lLrZOyrfD9QGIleif+zL+zvTbm67V35RaLjvoZEXfCmco8nx1hHk5iZ8TqwIjleEjXKZ4G3vd8HlHusvjE/SKIM0EbiJ43rOxsviTT1x96fxXJgagzH/oA3tje8/zmG+u6/yRv7auCCMo57Bon0/g1gmfervhI+9yxjj/3n7o3lFOA1b5xvAjd7j48mEbTWAo8Ch3uP13rXa73nR8dw7M95n/2bwK84NKOp4L832j5DRETSVFJZSURESkTBQURE0ig4iIhIGgUHERFJo+AgIiJpDst9i0j3ZWbJKaUAQ4B2YJt33eKcO6MsAxOJmKayiuTJzG4B9jjnvl/usYhETWUlkS4ysz3e7zPN7L/M7BEzW2Nmd5rZZ7z99peb2Ue8+44xs9+Y2VLv18fL+18g0jkFB5FwnAp8GZgIfBYY45ybBvwc+KJ3zz3A3c6504BLvOdEYkk9B5FwLHXeXjVm9mfgae/x5cDZ3tfnAON9RwD0NbM+LnEWgkisKDiIhGOf7+uDvuuDHPr/rAdwunNubykHJtIVKiuJlM7TwI3Ji+T5viJxpOAgUjpfAqZ6p3StBL5Q7gGJdEZTWUVEJI0yBxERSaPgICIiaRQcREQkjYKDiIikUXAQEZE0Cg4iIpJGwUFERNL8f/3+XVfQYOH6AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "surv.iloc[:, :5].plot(drawstyle='steps-post')\n", "plt.ylabel('S(t | x)')\n", "_ = plt.xlabel('Time')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluation\n", "\n", "The `EvalSurv` class contains some useful evaluation criteria for time-to-event prediction.\n", "We set `censor_surv = 'km'` to state that we want to use Kaplan-Meier for estimating the censoring distribution.\n" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "ev = EvalSurv(surv, durations_test, events_test, censor_surv='km')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Concordance\n", "\n", "We start with the event-time concordance by [Antolini et al. 2005](https://onlinelibrary.wiley.com/doi/10.1002/sim.2427)." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.6579549790156429" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ev.concordance_td('antolini')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Brier Score\n", "\n", "We can plot the the [IPCW Brier score](https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291097-0258%2819990915/30%2918%3A17/18%3C2529%3A%3AAID-SIM274%3E3.0.CO%3B2-5) for a given set of times.\n", "Here we just use 100 time-points between the min and max duration in the test set.\n", "Note that the score becomes unstable for the highest times. It is therefore common to disregard the rightmost part of the graph." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xV9f348df73tybCSEJYQYIYW/BAKKCW3Friy1q6xbbarW1tl9Hq9ba39fur7ZV68JqXYgLt1YEXOwhhBnCCknIInvf+/n9cU8ghBtyQ25yknvfz8cjj9x7xr3vwwn3fT9bjDEopZRSzTnsDkAppVTXpAlCKaWUX5oglFJK+aUJQimllF+aIJRSSvkVYXcAwdK7d2+TmppqdxhKKdWtrFmzptAYk+xvX8gkiNTUVFavXm13GEop1a2IyJ6W9mkVk1JKKb80QSillPJLE4RSSim/NEEopZTySxOEUkopvzRBKKWU8ksThFJKKb9CZhyEUsejtsHDvuIqsgoq2VtcxUlpSYwfGG93WEp1CZogVFjKyCnl2S928e63OdR7Dq+J4o5w8NfvTeKiiQNsjE6prkEThAob1XUeFm/N56UVe/h6ZxExbidzpw5mypBepPWOIzHWzZ0L1nPby+vIPljNLbPSEBG7w1bKNpogVMj7fFs+b67dz2dbDlBV56F/fBT3nD+audMGEx/tOuLYF2+czl2vb+CRD7eyPa+cOekpnDgkgcgIp03RK2UfTRAqZJXX1HP/Oxm8tW4/ibFuLps8kIsm9Gd6WhJOh/+SQZTLyWNzJzM4MYZ/LcvizXX7iXY5mZ6WyA9PGsKZo/toqUKFDQmVNanT09ONTtanGm3YV8Ltr65jX3EVd5w1kp+cMQyXs22d9spr6lmeVcyXOwr475Z89pdUc8KgXtx5zkhmjuh9VKJYsi2fxz7bwakjkvnZWSNwtJCElOpKRGSNMSbd7z5NECpUZB+sYun2ApZuK2Dx1nz69Ijk0SsnMzU1sd2vXe/x8saabP6+OJP9JdWM7teD00YlM2tEMvHRLv748TaWbS8gIcbFwap6zhnbl799/wTiIrWQrro2TRAq5P1q4QYWrM4GYGCvaM4Z25efnz2S+BhXK2e2TW2Dh9dXZ/Petzms2XPwUA+onlER3H7WCK6ZkcrLK/bwu/e3MCw5lid/cCJpyXFBjUGpYNIEoULavuIqZv3pcy47YSC3njGcYcmxndJOUFnbwPKsIvYUVXH55IEkxLoP7ftyRyG3vryW0up64qNdpCbFkJYcx3UnpzJpUK8Oj02pQB0rQWj5V3V7C9f4Sg6/OHckKQkxnfa+sZERnDWmr999p47ozfu3n8pHm/LYXVTJ7sIqFm/N5611+7l88kB+NXsUyXGRrNxVzMcZeWzJLSe5RyT94qPoHx9FtNuJy+Egwin0i4/SnlTKFpogVLfm8RoWrsnm1OG9OzU5BCIlIYabZqYdel5eU88TS3byzJe7+HBTLlEuJyVV9US5HIwfEM+W3DI+23qAmnrvUa8V5XIwbWgSp49MZu60QcS49b+u6nj6V6a6ta93FrK/pJq7zx9tdyit6hHl4lezR3PV9MH8/bNM6r1ezh3bj9NGJhPt9pUOjDGUVTdQ0+Ch3uOl3mPIKqjgix2FLNtRwEPvbea5r3bx0KXjOHO0/9KLUsGibRCqW7vt5bV8mVnIinvPCosqmBVZRdz39iYy8yuYPa4f543vS029l5p6Dy6ng5F9ezCqb4+gN86r0KVtECokHays45OMA1w1fXBYJAeA6WlJfHD7TJ7+IovHPtvBRxl5fo9LSYjmsSsnM2VwQidHqEKJJgjVbb2zfj91Hi/fSx9kdyidyh3h4NYzhnPVtMGUVtcT6XIQFeGkqt7D9rxyth0o54Wvd3PX6xv48I6ZYZM8VfBpglDdQmVtA09/kcWaPQcZ0acHo/v34JWV+5gwMJ6xA3raHZ4tEmLdR3StTcA3BuSM0X0Y078n1z63kn9+vpM7zxlpX5CqW9MEobqUtXsP8vjnmQxLjuOU4b2ZMiSBdzfk8NdPt1NQXsvIvnGs2l18qKfPw5eNtznirum0kclcdsIAnliSycUT+zOibw+7Q1LdkDZSqy5j/b4SfvDMCpwOoaqu4Yh1Gk4cksB9F45hyuAEPF7D3uIq9hVXcfKwJCLaOMdSuCiqqOWsvy5leHIcC26ZoXNDKb+0kVp1eRuzS/nhsytIiHXx2rwZ9IpxsXJXMat3H2T8wJ6cN67fodHRTocwtHcsQ3vH2hx115YUF8mvLxzLXa9v4OH3t3DxpP6MHdBT2yRUwLQEoTrFkm35fLmjkEtOGMDElMNTTRhjWLX7IDe/sJq4yAheu+WkLjfgrTszxvCTl9by4SZfbye308GwPnHERTpxRziIdvkWTTp7rI6pCFc6F5OyTUVtA79/fzOvrNx3aNvU1ASunj6ErMJK3tuQQ1ZhJf3jo3ht3gwGJ2ly6Ah5pTWs33eQdXtL2H6gnNoGL7UNXvJKa9hfUs2tZwzjznNGtbhOhgpdmiCULZZnFXHX6xvYX1LNvFlpzJuZxlvr9vP817vJPliNCMxIS+LiSQO4YHx/Hdxlg5p6Dw8uyuDVVfuYOaI3j82dfETPKBX6bEsQIjIbeBRwAs8YYx5ptv9O4CagASgAbjDG7LH2XQv82jr0YWPMv4/1Xpoguo7ymnoe+XArL63Yy5CkGP5yxSTSm6zJ0ODxsnrPQdJ6x9KnZ5SNkapGr67cy/3vZCAC4wb0ZGJKL6amJnLBhH66gl6IsyVBiIgT2A6cA2QDq4ArjTGbmxxzBrDCGFMlIj8GTjfGfF9EEoHVQDpggDXAicaYgy29nyYI+zV4vCzems8DizLIK6vhhlOG8otzR+rEct1ERk4pb63dz7fZpWzKKaWqzsMf50wMu4GI4cauXkzTgExjTJYVxKvApcChBGGM+bzJ8cuBH1iPzwM+NcYUW+d+CswGXunAeNVx2FVYyfyvdrFxfylbcsuoqfcysm8cj199MpN1moduZdyAeMYNiAd8s+Re9PcveXLpTuZMSdEusmGqIzuQDwT2NXmebW1ryY3Ah205V0TmichqEVldUFDQznBVWxljuP2Vdby2ah8uh4Orpg3h0bkn8N5PZ2py6OacDuHHpw8jq6CSTzYfsDscZZOOLEH4+8rhtz5LRH6ArzrptLaca4x5CngKfFVMxxemOl4rdhWzcX8pv798PFdPH2J3OCrILhjfjz8nxvDE0p2cN66vtkWEoY4sQWQDTSsvU4Cc5geJyNnAfcAlxpjatpyr7PX0siwSY918d0qK3aGoDhDhdDBvVhob9pXwTVaR3eEoG3RkglgFjBCRoSLiBuYCi5oeICKTgX/hSw75TXZ9DJwrIgkikgCca21TXURmfjmfbc3nmhlDiHLpyNxQNefEFHrHRfLEkp12h6Js0GEJwhjTANyG74N9C7DAGJMhIg+JyCXWYX8C4oDXRWS9iCyyzi0GfocvyawCHmpssFZdwzNf7CIywsEPT9KqpVAW5XJyw6mpfLGjkE37S+0OR3UyHSin2iy/vIZTH/mcK9JT+P3lE+wOR3Wwspp6TvnfxUwZksDz10/VtogQc6xurjoNpmqzF7/ZQ73Xy42nDrU7FNUJeka5+Nk5I1m6vYC31++3OxzViTRBqDaprvPw4vI9nDOmL2nJcXaHozrJdSenMmVwL3777mYKymtbPG5XYSW7Cis7MTLVkTRBqDZ5a91+SqrqtfQQZpwO4Y9zJlFV5+GBRZuO2l9YUct9b23krL8s4eK/f6ntFW1QXlPPHz7aypo9LU4UYRudA0EFzBjD/K92MW5AT6YNTWz9BBVShveJ446zRvCnj7exaEMOEwbGs7uokg37Snj2i11U1Xu4avpgFm/J57r5K3n9Ryfrmh2t2JJbxk9eWsuuwkpKquo4cUjXGmCqCUIF7MvMQnbkV/DnKyZpQ2WYmjcrjQ835XL7K+uO2H7m6D7ce8EYhveJI/PkCq548mt++OwK3vjxyfTVCRn9WrB6H795exPx0S6SYt0UV9bZHdJRNEGogM3/aje949xcPKm/3aEom7icDv5x5RTeWZ/DwIRoUpNiGJIUS3KPyEPHDO8Tx/PXT+Oqp5dzzbMrWfCjGcRH61TujUqr67n/nU28sz6Hk4cl8ejcydz28loOVtbbHdpRtA1CBWRXYSWLt+Zz9fQhumRlmEvtHcsdZ49gzokppKcmHpEcGk0a1Iunrkknq7CCW19aS73Ha0OkXc/yrCIuePQL3vs2l5+fPZIXb5xOco9IEmPdFFd1vRKEJggVkH9/vRuXU7j6pMF2h6K6iVOG9+b3l0/gy8xC7n9nE6Ey5up4vbRiD1c+vRyXU1j4oxnccfaIQyv4JcS6OahVTKo7Kqup5/XV+7h44gD69ND6ZBW476UPYndhJY8v2cnQ3rHMmzXM7pBs8+rKfYwb0JPX5s0gNvLIj97EGDcHq+rwek2XmlpdE4Rq1ZNLdlJZ5+H6U7Rrq2q7u84dxZ6iKv73w60s215IncdLbb2HpLhIbpo5lBlpSSHf6cHrNWTmV3DltMFHJQfwlSC8xvdlrFdM11nyVauY1DFtzC7lX8uymHNiChNS4u0OR3VDDofwl+9N4pJJAyivbUCAXjFuNu4v5aqnV3DFk9+wZFt+SFdBZR+sprrew8i+/geXJsb6GvG7Wk8mLUGoFtU1ePnlwg0kxbr5zYVj7Q5HdWNRLiePzp18xLaaeg8LVu/jySU7uW7+Km45LY27Z48OydLE9gPlAIxoIUEkWKWGg12soVoThGrR40sy2ZpXzjPXpBMfo90UVXBFuZxcMyOVuVMH89B7GfxraRbGwD3nh16S2J7vSxDD+/Twuz8x1pcgirtYV1dNEMqvLbll/GNxJpedMICzx/a1OxwVwtwRDn536XicIjy1LAuP1/DrC8eEVJLYcaCCfj2jWhwPcqgEoVVMqjt4cFEGvWJcPHDxOLtDUWFARHjwknE4HMKzX+7i8235h8bbDE6M5m/fP4EYd/f9uNp+oLzF6iWApDirBNHFqpi0kVodZWteGSt2FXPLrGEkxHadHhUqtIkI9180lnvOH82w5DhSEqLpHx/FxxkHmP/V7qOO/yqzkHc3dP2ViD1WD6aRff1XLwFEu5xERji0BKG6vpeW78Ud4WDOibrWtOpcIsItpw3jlibbbvr3Kp5cspOrpg0+9IUlt7SaH/1nDQAXTOh/aMBZV5R9sIraBm+LPZjAd92JXXA+Ji1BqCNU1jbw1rr9XDShv5YeVJfwy/NGU1HXwBNLfetiG2O4982NlNc0UF7TwJbcMpsjPLbtByoAGHGMEgT42iHa24vJGMOuwkreWb+fFVlF7Xot0BKEauad9TlU1DZwta41rbqIUf168J3JKTz/9W6uOzmVr3cW8fm2AubNSuOpZVms2FXM+IFdd4zOoS6ufY69wFZ7ShBlNfXc/ca3fLmjkLKaBgB6RkWw9jfnEOE8/nKAliDUIcYY/rN8D2P692TK4F52h6PUIT8/ZwQYeGBRBg+9m8G01ETunj2aQYnRQfmm3JF2HChnQHwUPaKO3VU84TgTRFVdAzfMX8UnGQe4cGJ/HvnOBO67YAxlNQ3tXoRIE4Q6ZP2+EjbnlnH19MEh1cVQdX8pCTH8cMYQPt18gNoGL3+YMxGHQ5g+NImVu4vxervuKOztByparV4CSIxxtTlB1NR7uPmF1azde5BH507mf78zkbnTBjN32iBcTmHxtvzjDRvQBKGa+M/yvcS6nVw2eaDdoSh1lFvPGM6w5FgeuHjcoZXqpg9NpKSq/tBAtK7G4zXsLKg4ZgN1o4RYN2U1DQFPjV7X4OUnL63lq8wi/jRnEhdOPLxOS48oF9OGJrJ4iyYIFQRFFbW8920Ol00eSJyfycSUsltirJv/3nkaV00/POX8SWlJAKzIKrYrrCPsLaritVV7D80rta/Y14NpRAsjqJtqHE1dUtX6aOraBg+3vryWxVvzefiy8XzXT4/DM0b1YUd+BfuKq9p4FYdpglAAPL5kJ/Uer87Yqrq05lWfKQnRDIiPYsUu+9shCsprueqZ5fzPGxtZsHof0PocTE0FOh9TTb2HH/9nLZ9uPsBvLxnHD1roUHLWGN8MCIu3Hn8pQhOEYn9JNS8u38N3p6QwvJWeFkp1JSLC9LQkVu4qPmI22EUbcti0v7TT4qiu83DTC6sprKhl/MCePPTuZvYVV7EjP7AurtB0PqaWE0RNvYd5L65h8dZ8fn/5eK49ObXFY4f2jmVo71hNEKp9Hv3vdjDws3NG2h2KUm02fWgihRV17CzwfRj/d/MBbn9lHVc/s4I9RZUd/v5er+Hnr63n2+wSHps7mSeuPhGAXy38lm155QzsFR1QtW0g8zHd++ZGvthRwB++O4Grp7feFf3M0X34JquIqrqGAK/mSJogwlxmfgUL12Tzg5OGMLBXtN3hKNVm0612iOVZxRwoq+GXCzcwsm8cInDzC6upqD2+D8dA/emTbXyUkcevLxzLueP6MSgxht9cNJZvsor4YGNuQNVL0KQE0UIVkzGGxdvymTMlhe9PDWzp3zNH96GuwctXmcdXBacJIsz95ZNtRLuc3HpG+C4Fqbq31KQY+vSI5JusIn6xYAPV9R4ev/pE/nHlFDLzK/jFgvUd1g02t7Sap60FtW44JfXQ9u9PHcTpo5Jp8JpjzsHUVC9rSv2WShB5ZTWUVNUzsQ0Ld01NTSQuMuK4q5k0QYSxDftK+HBTHjfPSiMpLtLucJQ6Lo3tEB9szOXLzEIeuHgcw/vEceqI3tx34Vg+zjjAY4t3dMh7P//VbrzGcMdZI45oQBcR/vDdiYzu14PTRiYH9FpRLiexbmeLa0I0Tikypn/PgONzRziYOaI3n289vhX7tD9jmCqrqefnC9bTOy6Sm2am2R2OUu0yfWgi727IYfa4fsydOujQ9htOSWXT/lIe+2wH35mcwuCkmKC9Z0VtAy+v3Mv5E/ozKPHo1+3bM4qPfjarTa+ZENvyfExbcn09okb1C6xE0uiM0X34cFMew+794FAS+5/Zo5g3q/VaA00QYcjrNfz81fXsLaripZum67gH1e1dNLE/e4ur+Mnpw476Jv+r2aN4d0MO87/eFdT1TV5btY/ymgZuDuIXrKRjTLexObeMQYnRrU7Z0dwlkwZQUF5LdZ0HgBe+2X0o2bRGPxnC0P99toPPtubz20vGHWrgU6o76xXj5t4Lxvjd1z8+mosnDWDBqn387OyRLa7q1hYNHi/PfbmLqakJnDAoePOWHWs+pi25ZYzpF3j1UqMol5Nbzxh+6PmHm3KpC3C0trZBhJmPM/J47LMdzDkxhWtm6IytKjzcNHMolXUeXlm5Nyiv91FGHvtLqoNePZsY4z9BVNU1sKuwsk3tDy1xOR3UNWiCUM1s2l/Kna+tZ2JKPA9fNl4n5FNhY9yAeE4elsTzX+0OeK6jlhhjeHpZFqlJMZw9JrjrtSfEuv32YtqWV44xbWugbklkhCPgf4MOTRAiMltEtolIpojc7Wf/LBFZKyINIjKn2T6PiKy3fhZ1ZJzhIPtgFdc/v4r4aBdPX5NOlMtpd0hKdaqbZ6aRV1bD+9/mHvdr1Hu8PPLRVjZkl3LjqUODvpJdYqybyjoPNfWeI7Y3thmM7eQSRIe1QYiIE/gncA6QDawSkUXGmM1NDtsLXAfc5eclqo0xJ3RUfOGktLqe6+evoqbew0s/Ppm+PaPsDkmpTnfayGSGJcfy9BdZzB7fj8z8CnbklzOqb0/GDmj9g3d/STW3v7KONXsOcuW0wcydFthgtbZoHE1dUlVPv/jDX+K25JYRFxlBSkL7B7O621CC6MhG6mlApjEmC0BEXgUuBQ4lCGPMbmtf+8p86gjGGMpqGsgrrSGvrIYnlmSyu6iSf98wLeBBO0qFGodDuGlmGve8uZGx939E49g5t9PBkz+cwpmjW64uWpFVxLwX19Dg8fLYlZO5ZNKADokxMdbXgF5cWUe/+MNf5LbkljG6Xw8cQSixuJwOKgMcXd6RCWIgsK/J82xgehvOjxKR1UAD8Igx5u3mB4jIPGAewODBwc/m3VFZTT0/fXkdS7cXHNrmEPjL9yZx8rDeNkamlP0unzyQ7QfKiYuMYFS/HgxJjOW+tzdyy4tr+PuVU5g9vt9R5xhjePDdzfSMjuCFG6YfWouiI/ib0dXrNWzNK+fyIK3T4o5wUOcJbNBcRyYIf6muLUP5BhtjckQkDVgsIhuNMTuPeDFjngKeAkhPT++6S0p1kpySaq6fv4qdBRX89MzhjOzbg37xUQxJjKGPVispRZTLedRYiP/cNJ3rnlvJrS+v5f++fwIXNysdrNtXwpbcMh6+bHyHJgfwP6Nr9sFqKmobgtJADb4SU12Dp/UD6dgEkQ0MavI8BcgJ9GRjTI71O0tElgCTgZ3HPCmMbdpfyg3Pr6K6zsPz10/j1BFaWlAqED2jXLxw43RueH4Vd7y6jtSkWCY0me/opU5caTEh9ugSxOZDU2wEp3rY1wYR2PfpjuzFtAoYISJDRcQNzAUC6o0kIgkiEmk97g2cQpO2C3WYMYZXV+7liie/IcIhLPzxyZoclGqjuMgInrk2nYQYN799N+PQvEUlVXWdutJiL2sQX1HF4QSxJbcMkbZPsdESl1PsHwdhjGkAbgM+BrYAC4wxGSLykIhcAiAiU0UkG7gC+JeIZFinjwFWi8gG4HN8bRCaIJopqqhl3otruPvNjUwZ0ou3bz0laH9ESoWbnlEufnneKFbvOci7VlfYhWuyqW3wtrhqW7BFOB3ER7uOKEFsyS1jaFIsMe7gJKiu0osJY8wHwAfNtt3f5PEqfFVPzc/7GpjQkbF1d5v2l3Ld/FWUVdfz6wvHcMMpQ4PSw0GpcHZF+iBeXL6H//1gC2eP6cPLK/Zy4pCEoNX/ByKx2XQbW/LKmDgweNN56EjqEFfv8XLX6xtwOuCd207hpplpmhyUCgKnQ3jg4nHkltYw74U1ZBVWcvX0zu0hmRBzuARRUF7LvuLqoLU/QGMvpiAlCBGJEZHfiMjT1vMRInJRO2NU7TD/q11szSvnoUvHd+o3G6XCwbShiVw8aQBfZhaSEOPiggn9O/X9E2PdFJbX8crKvZz3f8twCJw6IrA1JQLhdvoSRCDrQwRSgpgP1AIzrOfZwMPHH55qj+yDVfzt0x2cPaYv5407us+2Uqr97jl/NLFuJ1dOG9zp09IkxLjZdqCce97cyPDkON796alBnTHW7XRgDHgCWGUvkDaIYcaY74vIlQDGmGrRWd5s8+AiX1v9g5eMtTkSpULXgF7RLPvVGfQMwtTgbTU1NZG1ew9yx9kjuXhi/6BPqumK8JUL6jxeIpzHLiMEkiDqRCQaa5CbiAzDV6JQneyTjDz+u+UA914wmpSE4K2MpZQ6ml3L8H5v6iC+N3VQ6wceJ7eVFOoavFgDt1sUSIJ4APgIGCQiL+Ebk3BduyJUbfb51nx+sWADo/v14PpThtodjlKqm2pagmjNMROEVZW0FfgOcBK+6TPuMMYUtjtKFRBjDE8s3cmfPt7GmH49efradFytFAuVUqolkU1KEK05ZoIwxhgRedsYcyLwflCiUwHxeg3f7i/l6WVZvL8xl4sm9udPcyYR7dZ1HJRSx89tlSACmW4jkCqm5SIy1RrUpjrY3qIq/u+/21m6vYCiyjqcDt+i6z8+bZiuAKeUajdXsEoQljOAW0RkD1CJr5rJGGMmtiNG1YKH3svgq8wizhvXlzNG92HWiORDE3gppVR7HS5BBCdBnN/OeFSAKmsbWLajkKumDebBS8a1foJSSrWRy+mriagNoATRamunMWYP0Au42PrpZW1TQbZ0ewF1DV4dAKeU6jBtKUEEMtXGHcBLQB/r5z8i8tP2haj8+SQjj4QYF1NTE+wORSkVotxBboO4EZhujKkEEJE/AN8Afz/+EFVzdQ1ePtuaz+xx/Vod3aiUUscrqCUIfI3STden8+B/OVHVDsuziiivadDqJaVUhwp2L6b5wAoRect6fhnw7PEGp/z7OCOPGLdTV4NTSnUod7BGUgMYY/5qrQl9Kr6Sw/XGmHXtC1E15fUaPt18gNNGJnf6zJFKqfAS1DYIETkJyDDGrLWe9xCR6caYFe2MU1nW7Sshv7xWq5eUUh2uLSOpA2mDeAKoaPK80tqmguSTjDwiHMIZo/vYHYpSKsQdboPwtHJkgI3UpsnSQ8YYLx28lnU4McbwcUYeM4YlEW/D3PNKqfAS7BJElojcLiIu6+cOIKt9IapG2w9UsLuoSquXlFKdonEkdSCN1IEkiB8BJwP78S03Oh2Yd/zhqaY+2pSHCJw7rq/doSilwkBQG6mNMfnA3HZHpfz6KCOP9CEJ9OkRZXcoSqkwICK4nBKcEoSI/FFEelrVS5+JSKGI/CAokYa5PUWVbMkt0+olpVSncjsd1Adjsj7gXGNMGXARviqmkcAv2xeeAt/gOEAThFKqU7kiHEFrg2jsWnMB8Ioxprg9ganDPtqUx7gBPRmUGGN3KEqpMOJ2OoI2F9O7IrIVSAc+E5FkoKad8YW9A2U1rN1bwmwtPSilOpnL6QjaehB3AzOAdGNMPVAFXNruCMPcJ1b10uzxmiCUUp0rMsIRtDWpMcYcbPK4Et9oatUOH2ccIC05luF94uwORSkVZlxOR9BGUqsgK6mq45usImaP64eIzpyulOpc7gBLEMdMEOIzKGhRKQD+tSwLj9dw/vj+doeilApDLqcENFDumAnCmoPp7WAFpeDzbfk8sWQnV04bxISUeLvDUUqFIXeEo/0JwrJcRKa2PySVU1LNna+tZ0z/njxw8Ti7w1FKhSmXM7BxEIE0Up8B/EhEduNrnBZ8hYuJ7YowzNR7vNz28lrqGrz886rJujCQUso2kREOioK05Oj57Q8nvNXUe/jN25tYu7eEv185mbRk7bmklLKPr5E6OOMg9gCDgDOtx1WBnAcgIrNFZJuIZIrI3X72zxKRtSLSICJzmu27VkR2WD/XBvJ+XdHWvDIu++dXvL4mm5+eOZyLJw2wOySlVJgLWhWTiDyAbwuvzGQAABH4SURBVBT1KGA+vqk3/gOc0sp5TuCfwDn45nBaJSKLjDGbmxy2F7gOuKvZuYlA4/saYI117kG6kRe+2c3D72+hZ5SL+ddP5YxRumKcUsp+wZys73LgEqzBccaYHKBHAOdNAzKNMVnGmDrgVZqNwDbG7DbGfAs0j/Q84FNjTLGVFD4FZgfwnl1GZn4F97+TwYy0JD762UxNDkqpLiOYk/XVWd1dDYCIxAYYw0BgX5Pn2da2oJ0rIvNEZLWIrC4oKAjwpTvHkm35APy/70ygd1ykzdEopdRhbmfwurkuEJF/Ab1E5Gbgv8DTAZznb4hw60P32nCuMeYpY0y6MSY9OTk5wJfuHEu3FzCiTxwDe0XbHYpSSh3BHawShDHmz8BC4A187RD3G2P+HkAM2fgatxulADkBnNfec21XVdfAiqxiThvZtZKWUkpB43TfwZus71N87QBtsQoYISJD8a1nPRe4KsBzPwb+n4gkWM/PBe5p4/vbZkVWMXUeL6eN0gShlOp6XE4HHq/B4z12kmixBCEiX1q/y0WkrMlPuYiUtRaAMaYBuA3fh/0WYIExJkNEHhKRS6zXnioi2cAVwL9EJMM6txj4Hb4kswp4qDstVLR0ewHRLidTUxPtDkUppY7ijvB99Lc2FqLFEoQx5lTrdyA9llp6jQ+AD5ptu7/J41X4qo/8nfsc8Nzxvredlm4vYMawJB0trZTqklxOXzNva4sGtTabq0NENgUvrNC3p6iSXYWV2v6glOqyIgMsQbQ2m6sX2CAig4MWWYhbtt3X3VYThFKqq3I5fR/9rXV1DaSRuj+QISIrabKSnDHmknbEF7KWbi9gSFIMqb0DHS6ilFKdq91tEE38NgjxhIXaBg9f7yxizol+m1WUUqpLCFoJwhiztPGxiPQGiqyR1aqZ1bsPUlXn0eolpVSX1liCaG2w3LG6uZ4kIktE5E0RmWw1Vm8CDohIt5oXqTMYY5j/1W6iXU5mDEuyOxyllGqROwgliH8A9wLxwGLgfGPMchEZDbwCfBSUSEPEh5vy+O+WA9x7wWhi3AGNP1RKKVscboM4zoFyQIQx5hNjzOtAnjFmOYAxZmuwggwVpVX1PLAog/EDe3LDKUPtDkcppY4pGG0QTc+sbrZP2yCaeOSjrRRX1jH/uqlEOANaS0kppWwTjF5Mk6wpNQSIbjK9hgBRQYgxJKzIKuKVlXuZNyuN8QPj7Q5HKaVaFehI6mNNtaHzRLSipKqOX73xLYMSo/n52SPtDkcppQIS6EhqbU09TvUeLz95aS25JTW8fPN0ot2aT5VS3UMwR1KrZowx3P9OBl/vLOKv35tEus7aqpTqRto9DkK1bP5Xu3ll5V5+cvowvjNFR00rpbqXxhJEuybrU0fbmF3Kw+9v5rxxfbnr3FF2h6OUUm12qATRnum+1dGeXLqT2MgI/nzFJBwOf0tnK6VU13ZoJLWWIIJnb1EVH27K5erpQ+gR5bI7HKWUOi6BTrWhCaINnvtqF06HcN3JqXaHopRSx83hECIcom0QwVJSVcdrq/ZxyaSB9IvXcYJKqe7N5XRoCSJYXlqxl+p6DzfP0rmWlFLdnzvC0a7J+pSltsHD81/vZtbIZEb362l3OEop1W4up6PVqTY0QQTgnfU5FJTXMm9mmt2hKKVUUERGOLQNIhg+3pRHalIMpwzXhYCUUqHB5RRtgwiGjJwyThjUCxEd96CUCg1uLUG0X1FFLXllNYwboFN5K6VCh/ZiCoKMHN8yGOMGaOO0Uip0uCMcOpK6vRoTxFhNEEqpEKIliCDIyCllYK9oesW47Q5FKaWCRnsxBcHm3DKtXlJKhRyXU6uY2qWytoFdhZVavaSUCjlup4P6Bh1Jfdy25pVhDNqDSSkVclzaSN0+2oNJKRWq3NpI3T4Z+8tIiHHRX2dvVUqFGHeEaAmiPTJySxk3IF5HUCulQo7baXMvJhGZLSLbRCRTRO72sz9SRF6z9q8QkVRre6qIVIvIeuvnyY6M0596j5fteRVavaSUCkmBjIOI6Kg3FxEn8E/gHCAbWCUii4wxm5scdiNw0BgzXETmAn8Avm/t22mMOaGj4mvNjgMV1Hm82oNJKRWS7J6LaRqQaYzJMsbUAa8ClzY75lLg39bjhcBZ0kXqczJySgHtwaSUCk0up70LBg0E9jV5nm1t83uMMaYBKAUa59QeKiLrRGSpiMz09wYiMk9EVovI6oKCgqAGn5FTRrTLydDesUF9XaWU6grcEa1//HdkgvBXEmierlo6JhcYbIyZDNwJvCwiR9X1GGOeMsakG2PSk5OT2x1wU5tzyxjdvwdOR5co0CilVFC5nfYmiGxgUJPnKUBOS8eISAQQDxQbY2qNMUUAxpg1wE5gZAfGegSv17AlR6fYUEqFLrtLEKuAESIyVETcwFxgUbNjFgHXWo/nAIuNMUZEkq1GbkQkDRgBZHVgrEfYuL+U8toG0ockdtZbKqVUp3IFUILosF5MxpgGEbkN+BhwAs8ZYzJE5CFgtTFmEfAs8KKIZALF+JIIwCzgIRFpADzAj4wxxR0Va3NLthUgArNGBrfaSimluopAShAdliAAjDEfAB8023Z/k8c1wBV+znsDeKMjYzuWJdvzmZjSi8RYneJbKRWaXM7W21d1JHUzByvrWL+vhNO19KCUCmGRNrdBdEvLdhRgDJw2ShOEUip02d1I3S0t3VZAQoyLSSm97A5FKaU6TCCN1JogmvB6Dct2FDBzRLKOf1BKhTS7x0F0Oxk5ZRRW1HG6Vi8ppUKcS6uY2mbJtnxAu7cqpUKfliDaaMn2AiYMjKd3XKTdoSilVIfSRuo2KKmqY93eg1q9pJQKC1qCaIOvMovwGjRBKKXCgrZBtMG6vQdxRziYqN1blVJhQEsQbbBxfylj+/cMqG+wUkp1d5ogAuT1GjJyypgwUFePU0qFB22kDtCuokoqahuYkKIJQikVHnSyvgBtzPatPz1RE4RSKkxEOB20NmGEJgjg2+xSolwOhifH2R2KUkp1mtbaXDVBAJusBuoIbaBWSoWR1tohwv4T0eM1bMop1e6tSqmw01pPprBPELsKK6iq8zBeezAppcKMliBa8a02UCulwpS2QbRi4/5Sol1OhmkDtVIqzGgJohUbs0sZN6CnLhCklAo7WoI4Bk/jCGqtXlJKhSEtQRzDzoIKqus92v6glApL7lZGU4d1gmhsoNY5mJRS4UhLEMewaX8psW4nQ3trA7VSKvxoG0QLtuaV8d63uUxM6aUN1EqpsKQD5fz4ZmcRVzz5DU4HPHDJWLvDUUopW7S2qlxEJ8VhO2MMZTUNLN56gP9ZuJEhSTE8f8M0BvaKtjs0pZSyRWQrJYiQTxDvbsjhkQ+3UlBeS53HC8C01ESeuuZEesW4bY5OKaXs01ojdcgniMeX7MThgOtPTSU5LpJ+8VGcPaYvUS6n3aEppZStWmukDukEkZFTypbcMn532Xh+eNIQu8NRSqkuJay7uS5ck43b6eCSiQPsDkUppbqcsO3mWtfg5Z31OZwzri/xMS67w1FKqS4nbEsQn2/Lp7iyjjlTUuwORSmluqQLJvQ75v6QTRAL12ST3COSmSN62x2KUkp1SaP79Tzm/g5NECIyW0S2iUimiNztZ3+kiLxm7V8hIqlN9t1jbd8mIue15X0LK2r5fGs+35k8UNeZVkqp49Rhn54i4gT+CZwPjAWuFJHmw5ZvBA4aY4YDfwP+YJ07FpgLjANmA49brxeQd9bn0OA1fPdErV5SSqnj1ZHdXKcBmcaYLAAReRW4FNjc5JhLgQetxwuBf4iIWNtfNcbUArtEJNN6vW9aerPtB8o5569LAcgtrWFSSjwj+/YI7hUppVQY6cgEMRDY1+R5NjC9pWOMMQ0iUgokWduXNzt3YPM3EJF5wDyAngPSGNHXNyvryL49uPqkwcG5CqWUClMdmSD8TZFqAjwmkHMxxjwFPAWQnp5uHr/6xLbGqJRSqgUd2YKbDQxq8jwFyGnpGBGJAOKB4gDPVUop1YE6MkGsAkaIyFARceNrdF7U7JhFwLXW4znAYmOMsbbPtXo5DQVGACs7MFallFLNdFgVk9WmcBvwMeAEnjPGZIjIQ8BqY8wi4FngRasRuhhfEsE6bgG+Bu0G4FZjjKejYlVKKXU08X1h7/7S09PN6tWr7Q5DKaW6FRFZY4xJ97dPR5EppZTySxOEUkopvzRBKKWU8ksThFJKKb9CppFaRMqBbXbHESS9gUK7gwiSULmWULkO0Gvpiuy8jiHGmGR/O0JpydFtLbXEdzcislqvpWsJlesAvZauqKteh1YxKaWU8ksThFJKKb9CKUE8ZXcAQaTX0vWEynWAXktX1CWvI2QaqZVSSgVXKJUglFJKBZEmCKWUUn6FRIIQkdkisk1EMkXkbrvjaSsR2S0iG0VkvYistrYlisinIrLD+p1gd5zNichzIpIvIpuabPMbt/g8Zt2jb0Vkin2RH62Fa3lQRPZb92W9iFzQZN891rVsE5Hz7In6aCIySEQ+F5EtIpIhIndY27vdfTnGtXTH+xIlIitFZIN1Lb+1tg8VkRXWfXnNWhoBa6mD16xrWSEiqbYEbozp1j/4phLfCaQBbmADMNbuuNp4DbuB3s22/RG423p8N/AHu+P0E/csYAqwqbW4gQuAD/GtFngSsMLu+AO4lgeBu/wcO9b6O4sEhlp/f067r8GKrT8wxXrcA9huxdvt7ssxrqU73hcB4qzHLmCF9e+9AJhrbX8S+LH1+CfAk9bjucBrdsQdCiWIaUCmMSbLGFMHvApcanNMwXAp8G/r8b+By2yMxS9jzDJ863g01VLclwIvGJ/lQC8R6d85kbauhWtpyaXAq8aYWmPMLiAT39+h7YwxucaYtdbjcmALvvXcu919Oca1tKQr3xdjjKmwnrqsHwOcCSy0tje/L433ayFwloj4W4q5Q4VCghgI7GvyPJtj/xF1RQb4RETWiMg8a1tfY0wu+P6jAH1si65tWoq7u96n26yql+eaVPN1i2uxqiUm4/u22q3vS7NrgW54X0TEKSLrgXzgU3wlnBJjTIN1SNN4D12Ltb8USOrciEMjQfjLqt2t7+4pxpgpwPnArSIyy+6AOkB3vE9PAMOAE4Bc4C/W9i5/LSISB7wB/MwYU3asQ/1s6+rX0i3vizHGY4w5AUjBV7IZ4+8w63eXuJZQSBDZwKAmz1OAHJtiOS7GmBzrdz7wFr4/ngONRX3rd759EbZJS3F3u/tkjDlg/af2Ak9zuLqiS1+LiLjwfaC+ZIx509rcLe+Lv2vprvelkTGmBFiCrw2il4g0zonXNN5D12LtjyfwKtCgCYUEsQoYYfUGcONr0Flkc0wBE5FYEenR+Bg4F9iE7xqutQ67FnjHngjbrKW4FwHXWL1mTgJKG6s8uqpmdfGX47sv4LuWuVZPk6HACGBlZ8fnj1VP/SywxRjz1ya7ut19aelauul9SRaRXtbjaOBsfG0qnwNzrMOa35fG+zUHWGysFutOZXfrfjB+8PXE2I6vTu8+u+NpY+xp+HpebAAyGuPHV9/4GbDD+p1od6x+Yn8FXxG/Ht83nhtbihtfkfmf1j3aCKTbHX8A1/KiFeu3+P7D9m9y/H3WtWwDzrc7/iZxnYqvKuJbYL31c0F3vC/HuJbueF8mAuusmDcB91vb0/AlsUzgdSDS2h5lPc+09qfZEbdOtaGUUsqvUKhiUkop1QE0QSillPJLE4RSSim/NEEopZTySxOEUkopvyJaP0Qp1ZSINHYZBegHeIAC63mVMeZkWwJTKsi0m6tS7SAiDwIVxpg/2x2LUsGmVUxKBZGIVFi/TxeRpSKyQES2i8gjInK1tSbARhEZZh2XLCJviMgq6+cUe69AqcM0QSjVcSYBdwATgB8CI40x04BngJ9axzwK/M0YMxX4rrVPqS5B2yCU6jirjDWvkYjsBD6xtm8EzrAenw2MbTLVf08R6WF86x8oZStNEEp1nNomj71Nnns5/H/PAcwwxlR3ZmBKBUKrmJSy1yfAbY1PROQEG2NR6giaIJSy1+1AurU62mbgR3YHpFQj7eaqlFLKLy1BKKWU8ksThFJKKb80QSillPJLE4RSSim/NEEopZTySxOEUkopvzRBKKWU8uv/A4muIZ4zNALTAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "time_grid = np.linspace(durations_test.min(), durations_test.max(), 100)\n", "ev.brier_score(time_grid).plot()\n", "plt.ylabel('Brier score')\n", "_ = plt.xlabel('Time')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Negative binomial log-likelihood\n", "\n", "In a similar manner, we can plot the the [IPCW negative binomial log-likelihood](https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291097-0258%2819990915/30%2918%3A17/18%3C2529%3A%3AAID-SIM274%3E3.0.CO%3B2-5)." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hU55X48e+ZkUZCBQkVECAEopvei/Fi3HEJjmvAwS0ucRwn2Xg3vzjN6zibTeI4ztprEvfuuMZ2SEzcsOOGKaKZKhBNEqBeUNeU9/fHjGQhVNEMd8r5PI8eZu68M3MuV5ozbxdjDEoppSKXzeoAlFJKWUsTgVJKRThNBEopFeE0ESilVITTRKCUUhEuyuoAeistLc2MGDHC6jCUUiqkbNy4scwYk97RYyGXCEaMGEFOTo7VYSilVEgRkUOdPaZNQ0opFeE0ESilVITTRKCUUhFOE4FSSkU4TQRKKRXhNBEopVSE00SglFIRLuTmESh1MhqdbvaX1rG3pIay2maumDGU5DiH1WEpFRQ0EaiwVVHXzNvbjvLW5sNszq/E02brjSc+3c8fvzGNeSNTrQtQqSChiUCFDY/HkFtcw5p95Xy2t5RP95bh8hjGDkrgO4tGMT6jP2MHJVLf7OLOV7ey7PG13HHWaL5/zhii7dpKqiKXhNoOZbNmzTK6xIRqq6bRyROfHuCFtYcor2sGYHhqHBdMzODr04Zy2uBEROS459Q1ubhn5Q5e21iIw25jWEo/stPimTQ0ievnj2BAfOfNRk63h8LKBrLT4gN6Xkr5k4hsNMbM6vAxTQQqVDU63byw9hArPsqjst7JeRMGccHEDOaPSmVocr8evca/cktYu7+Cg2V1HCirY09JDQkxUdy+aDQ3LhhBbLS9tawxhg92lfCbf+5if2kdjyyfyeJJGYE6PaX8ShOBChtr9pXx6d4yNh2q5MvCahqcbv5tTBo/umAcUzKT+/z6uUU13PfOblbvLmFQ/xhmj0hhcFIsg/rH8v7OYtYdqGBkejxRNqG0pol3/30hA/vH+uHMlAosTQQqLKzZV8Y1j68jyiZMHNKf6VkDOH/iIE4fleb391q7v5zHPtnPgbI6jlQ10OTykBrv4N/PG8vS2cM4VF7PJf/3KXOyU3n2xtknND0pFWy6SgTaWaxCxlOfHSQl3sEn/+8sEmIC+6s7b2Rq64giYwxV9U7iYuzERHmbikYPTOBnF0/gF29t57kvDnH96SMCGo9SgaSJQIWE/PJ6Vu8u5ruLRgc8CbQnIh12Hi+fm8WHu4r5n1W7iLbbGDqgH+kJMQxL6UdibPQpjVGpvtBEoELCc18cxC7C8nnDrQ6llYhw35VTufThz/jpm9tajzvsNi6YlMHS2cOYPzKVmiYXOQcrWH+wArsIU4clM31YsvYtqKChiUAFvbomF6/kFHDh5MFkJAXXh2d6Ygwf/WgRxdVNlNQ0UlLTxPoDFbyxqZC/bz1CaryDivpmjPEmCI8xuHwz2wbERRMTZSfKLjjsNsZlJLY2SY0dlKD9DuqU0USggt4bmwqpaXRx44IRVofSoZgoO1mpcWSlxgFw0eTB3HXheN7dUcTqXSWMTI9nbnYq07O8o5p2HDnG1oIq9pXW4nIbnB4PjU43Wwuq+ef2IgCGJMVy+YxMrpyZyYg28xU8viRis2mSUP6jo4ZUUPN4DOf98WMSYqJ467sLwv5bckFFPV/sK+ftbUf5dG8pHgOThvbH5TaU1TZTUdeEx4BNIMpuY+ygBJ69cQ6pCTFWh66CnI4aUiHB5fbw9rajPLPmIE1OD5kD+hHnsLOvtI7//ca0sE8CAMNS4hiWEsfVs4dRVN3IXzcV8uneUhJiopmelUxKvINouw23x9Dk8vDsmoPc+vxGXrx57nGT35TqjYAmAhFZDDwI2IEnjDG/7aDM1cA9gAG2GmOuCWRMynout4df/n0n6w6UM2ZgIuMyEomNtvHcF4corGxgVHo8w1PjOVheR0FFAyNS47ho8mCrwz7lMpJi+e5Zo/nuWaM7LTNtWDK3v7iJH//1y4hJlsr/ApYIRMQOrADOAwqBDSKy0hizs02ZMcBPgAXGmEoRGRioeFRwcLk93PnqVlZuPcK8kSl8ebiKt7cdBWB6VjJ3XzKBc08b1NoGbozBGG0T78xFkwfzowvG8ft3cxmVnsD3zxlzQhm3x9Ds8tDPoTUG1bFA1gjmAHnGmP0AIvIycCmws02ZW4AVxphKAGNMSQDjURZrmwT+3+Jx3L7I+023tslFeW0TWSlxJ3yjFRH0S27Xbl80in2ltTzw/h5e2VBAWmIM6QkxNLs95JfXUVjZAMAdZ3trF7rSqmovkIlgKFDQ5n4hMLddmbEAIvI53uaje4wx77R/IRG5FbgVICsrKyDBqr4xxvDP7UU8tHovUXbhmjnDuXTaEOJjonC6PWwpqOKJT/fz7o5ifrx4PN9ZNKr1uQkxUad8klg4ERF+c/lkRqUnsK+0ltKaJgor64m225g4NImLJg/mUEU9//vBXt7fWcz9V03ltMH9rQ5b9dGRqgZuf3ET3zt7NOecNqhPrxXIv76Ovse1H6IUBYwBFgGZwKciMskYU3Xck4x5DHgMvKOG/B+q6osv9pXz23d2s7WgijEDEwD46Zvb+M2qXUzLSmZzfhW1TS7sNuGnF43n1oWjunlF1VsxUfYu+xIAvjaliJ+/tY0lD3/GaYP7tybg+aNSuXFB9imKVPlDaU0Ty59Yx/6yOtYdqAjqRFAIDGtzPxM40kGZtcYYJ3BARHLxJoYNAYxL+ckX+8p5aPVevthfzuCkWO67cgpXzMjEJrApv5IX1uaz/XA1S6YNYeGYNOaPTCMpTpdesMriSRnMyU7hodV7OVheR22ji21l1Xy4u4TLp2fqtQkR1fVOrn1yHUeqG0iMiaK0pqnPrxnIRLABGCMi2cBhYCnQfkTQW8Ay4BkRScPbVLQ/gDGpk1R8rJGdR49RUdtMRV0z7+8qZv2BCtITY/j5xaexfN7w44YvzhyewszhKRZGrDqSEu/gniUTW+9vyq/k8j+t4cPcYi6bnmlhZKon6ppc3PDMevaX1vHE9bP44wd7KKsN4kRgjHGJyB3Au3jb/58yxuwQkXuBHGPMSt9j54vITsAN/MgYUx6omNTJKalp5Jw/fExtk6v1WEb/WO752gSWzsnS8eshbFpmMgMTY3h3uyaCUPD82kNszq/ikeUzWDg2nefXHqKgor7PrxvQHjpjzCpgVbtjd7e5bYA7fT8qSD356QHqm108fcNsRqTFkxLvoH9slI5ZDwM2m3DBxAxe31hIQ7Nbh5gGufd3FjNpaH8WT/LOq0lPjGHToco+v66OI1Ndqqhr5vm1h/ja1CGcNX4g2WnxJPWL1iQQRi6YmEGD080ne0utDkV1oay2iU35lZzbpmM4LSGGivpmXG5Pn15bE4Hq0tOfH6C+2d3tiBQVuuaOTCGpXzTv7iiyOhTVhQ93l2AMxyWC9MQYjPF+YesLTQSqU9UNTp75/CAXTspg7KBEq8NRARJtt3HOaQNZvasEZx+/WarA+WBnMYOTYpk45Ks5IOm+xQZL+9hhrIlAder5Lw5S0+TS2kAEuGBiBtUNTtYfqLA6FNWBRqebT/eWce5pg45rlk1P9O6c19chpJoIVIfqmlw8+dkBzh4/kElDk6wORwXYwjHpxEbbeGe7Ng8FozX7ymhwujl3wvETx9ITvBs1aSJQAfH05weorHdqbSBC9HPYOXNsOu/tLGrd/EYFj/d3lhDvsDNv5PFzc9J8NYKyWu0jUH5WUtPIn/+1j/MnDGLm8AFWh6NOkcWTMig+1qSjhwKk0elm9a5iersZmMdjWL2rmDPHpRMTdfzw3jhHFPEOu9YIlP/98f09NLk8/OSi06wORZ1CiycOJjstnp++sY1jjU6rwwkrxhh+8sY2bno2h51Hj/XqudsOV1NS03TcaKG20hJjtLNY+deuo8d4ZUMB180fQXabvXJV+OvnsPPA1VMprmninpU7rA4nrDy/9hBvbj4MQMmx3n1of7CrGJvAWeM63q4lPSGGMq0RKH8xxvDrt3eRGBvN98/RvoFIND1rAN89azRvbDrMO9uPnvB4o9PNazkF3Pj0ej7PK7MgwtCz8VAF9/59J5OGeod9lvdizL8xhlXbjjJrRAoD4h0dlkn3Q41AF4FXrf6VW8pneWX84pIJJMd1/Eunwt/3zh7Nv3JL+Mkb2xjUP5ZGp4fyuiY251fx+sZCqhucOKJsrDtQwUu3zGPqsGSrQw5apTVN3P7iJoYk9+PRa2ex4LcfUlHX8w/tnEOV7Cut49tdLN2elhDDF/v7tkSbJgIFQFV9M7/423ZGpMZx7bzhVoejLBRtt/HA1dO4+KFPuexPa1qPR/nWJVo+bzgj0+O58pE13PD0el677XRG+/ahUF9paHZz2wsbqap38ubtcxiSFIsjykZ5L0b4vLQ+n4SYKC6Z2vme3emJMVTVO2lyuU/oTO4pTQQKt8fwg5e3UHyskVe+PR9HlLYYRrrRAxP42x0L2FNcS1q8g9SEGDKSYknq99WeBc9/ay5XPrKG659az+vfmc/gpH4WRhxcXG4P33tpE5vyK3l42Qwm+GYDp8Y7etw0VF3v5O0vj3LFzEziHJ1/VKf5ZheX1zYzJPnkroH+xSse/GAPH+8p5b++NpEZWTpcVHmNz+jPkqlDOH10GuMyEo9LAgAj0uJ55sY5VDc4ue7J9VT2cb2bcGGM4advbuODXSXcu2QiF0/56tt8SryD8h6257+15TBNLg/XzOl6e970RG8i6Mu+BJoIItwHO4t56MM8rpqZyTfn6n7QqncmDU3i8etmcaiinhuf2UBdmz0rItUf3tvDqzmFfP/s0Vw7f8Rxj6UmxPRogThjDC+tz2fS0P7dzuxvSQR9mUugiSCCldY08cNXtzBpaH9+9fVJurS0OinzR6Xy8LLpbDtczW0vbKTJ5bY6JMuUHGvk4Y/yuGJGJj88b+wJj/e0aWhLQRW7i2pY1k1tACAtoe/rDWkfQQR78rMD1DW5eHDpdN1lTPXJ+RMz+N0VU/jP17Zy2/MbmTsylSanh2a3mwmDkzjntIER8TvWMlns6lmZHX6xSo139Kiz+OX1BfSLtrNk6pBuy7b0EfSlaUgTQYSqbnDywtpDXDh5MKPSdcSH6rsrZ2ZS3eDk12/v5KNc7zIVNgGPgYSYKC6YmMGyOcOYNSJ897LeXVQDePtXOpKS4KDB6e5yN7jaJhd///IIS6YOITE2usMybcVG20mM7dsm9poIItQLaw9R2+TiO2d2Pj5Zqd666Yxsls4ehgitQxnX7i/nrc2HeWd7EX/dVMjti0Zx53ljibKHX8t0blENg5NiSYrr+AM81TcprLyuiUxHXIdldh09Rn2zmwsmdbykREfSE2P6tPBc+F0J1a1Gp5unPz/AmWPTdYlp5XfxMVHEOaKw2wS7TVgwOo3fXzWV9T87l2VzsvjTv/ZxzRPrKD7WaHWofre7qIZxGZ1v4pQa/9VQz860bEafldLzJV7SE2K0RqB659WcAspqm7l9kdYG1KnTz2HnN5dPZk72AH76xnYu+N9PGJgYQ02ji5pGF5dNH8qvvj7J6jBPmtPtIa+khjPHpndaJsXXsdvVyKGCigYAMgf0fE5AWmIMO4/0bjG7tgJaIxCRxSKSKyJ5InJXB4/fICKlIrLF93NzIONR3l/WRz/ez4ysZOZkh29brQpel03PZOUdC5g/MpXstHjOGJ3GpKH9eXHdIQ6W1Vkd3kk7UFaH020Y32WNoKVpqItEUFnPwMSYXnWu93XhuYDVCETEDqwAzgMKgQ0istIYs7Nd0VeMMXcEKg51vLc2H+ZwVQO/XDJRh4sqy4wZlMifl89svV9S08gZv/uIP/9rH7+7cspxZV1uT0j0J7R0FHfZNNQ6C7jzD+2CinqGpXTcf9CZ9MQYappcXXZCdyWQ/7tzgDxjzH5jTDPwMnBpAN9PdaO6wcnv3sllSmYSZ4/veElbpawwMDGWpbOH8cbmQo5UNbQezy2qYeZ/f8ArG/ItjK5ndh89RpRNuhyFF++w44iyddk0VFjZwLBeNAvBV5vYn+wQ0kAmgqFAQZv7hb5j7V0hIl+KyOsiMiyA8US8+97ZTUVdE/9z2WRsNq0NqODy7TNHYQw89sl+ACrrmrn5uQ1UNzh5e1vw76WcW1TDqPSELtfqEpEuJ5U53R6OVjecVI0AOOnlqAOZCDr6pGm/R9vfgRHGmCnAB8CzHb6QyK0ikiMiOaWluo3eydh4qJIX1+Vz44JsHSmkgtLQ5H5cNn0oL63Pp6i6ke/+ZRPF1U2cPiqVdfvLaXQG94zl7kYMtUhN6Hy9oSNVDXgMDBvQu0TQMqnsZEcOBTIRFAJtv+FnAkfaFjDGlBtjWiJ/HJhJB4wxjxljZhljZqWnd94jrzrmdHv42ZvbGJwUy50dTHtXKlh8Z9EonG4Pl//pc9bsK+d/Lp/MLQtH0uTysO5AhdXhdepYo5PDVQ2MH9x9IkiJ73y9odYRQym9bBrq48JzgUwEG4AxIpItIg5gKbCybQERabvI9hJgVwDjiVhPfnaA3UU1/HLJROJjdMSwCl4j0xO4aPJgjlQ3cvMZ2Vw5M5N52ak4omx8sid4WwP2tM4o7kGNIN7R6eSvgkrvHILe1ghS+7jeUMA+FYwxLhG5A3gXsANPGWN2iMi9QI4xZiXwfRFZAriACuCGQMUTqT7YWcz97+Zy/oRBnD8xw+pwlOrW3ZdMYE52Suvyy/0cduZmpwR1ItjVOmKo46Ul2kqNd3RRI6jHbhMGJ8X26v2j7TYGxEUHXyIAMMasAla1O3Z3m9s/AX4SyBgi2cd7Srn9xU1MHNKf+6+eanU4SvXIwP6xXNdu+eaFY9L59apdHKlqOOnNVwIpt+gYibFRDOnBB3jLekP1za4TNpwpqGxgSHLsSQ2X9S4zEXxNQ8pCa/aVcetzOYwemMBz35pL/x4sXqVUsFrom60bDLUCt8dw63M5/PytbXg83vEvuUU1jM9I7NHcnLQulpkoqKjvdbNQ6+smxHCwrJ41eWWsyStja0EVxrQfn9MxTQRhaO3+cm56JofhqXG8cPPcThfAUipUjB2UQEb/WD7Z+1UiKKlp5IH3cvu0xs7JePKz/by3s5gX1ubzq7d3Yoxhd1FNpyuOtpcS3/kyE4WV9WT1cuhoi6yUOHKLa7jmiXVc88Q6Ll3xOdsP92zZCe05DDOf7i3lludyyBzgTQItv3RKhTIRYeHYNN7ZXoTL7aGu2c11T65nd1ENb2w+zDM3zmH0wMAvp55XUsv97+3hvAmDyBzQj6c/P0izy0NNo6tHQ0eh8/WG6ptdlNU293oOQYufXzKBy6Z7p2rlFtdw9992UNXQsxVJtUYQRj7cXcxNz+YwIjWel2+dx8DE3nU4KRXMFo5N51iji3UHKrj52Q3sK63lF5dMoNHp5oo/r2F9gIeXuj2GH72+lTiHnV9fNolfXOz94H1xnXfWc09GDMFXTUPt2/MLK3u/2FxbCTFRzB2ZytyRqUzNTAa8Q8d7QmsEIcrtMfxty2EKKxtodLqpbXLx0vp8xmf05/mb5pAcpzUBFV7OGJ2GTeC25zdS2+zi4WUzuHjKYM6fMIjrn17P8ifW8ci1Mzh7fM/X8e+NJz/bz+b8Kh5cOq31S9Z9V06husHJZ3llfa4RtCw/fbI1graifZ3NzS5NBGGrrLaJH76yhU/3lgEQZRNio+3MG5nKim/O0I5hFZaS4xxMHZbM5vwqfn3ZJC6e4p2GNCwljr/edjrLHl/L3X/bwb+NSW/9IPSXQ+V1rU1CbbePjLbbePTamRRVN/ZoNzHofL2h1kRwkp3FbbUsc9GkiSA8rT9Qwfde2kRlvZPfXD6Zq2ZmhsTKjEr5w71LJnG4qp7FkwYfd3xAvIMfLx7Pjc9s4I1NhXxjdvebvvfGA+/vwSbw31+fdMLIoGi7rVff4kWEtA4mlRVUNtAv2t66GX1fxET1rkagnyAhwuX28PCHe1n2+Fr6Rdt58/bTWTYnS5OAiiiTM5NOSAItFo1LZ0pmEv/3YV6P28Z7YnfRMVZuPcINp2czqL9/+t1SEhxU1B3fR1BQUU/mgH5+WR6+pUbkdOvw0bCRX17PNx5by/3v7eHiyYP5+/fOYOIQXThOqbZEhB+cM4bCygbe3HTYb6/7h/f2kOCI4rYzR/rtNTtab6igsverjnbG0Voj6NlCfZoIgtzftx7hwgc/YU9xDQ8uncZDy6b3uC1SqUhz9viBTB6axMMf+adWsKWgivd3FnPLwpF+HYDRvmnIGENhRX2v9yHoTGsi0FFDoW9LQRU/fGUL04Yl8+Cy6QwNwqn1SgWTllrBzc/l8PrGQkYPTODj3FJ2HKnmpjNGcsaYtF693v3v5pIS7+BbZ2T7Nc6UdusNVTc4qWly+a9G0MumIU0EQaq6wckdf9lERlIsT94wm6R+WgtQqifOOW0gk4b25ydvbAPAbhOS+0XzrWc38Oi1MzlrXM9251uzr4zP8sr4+cWnkeDnVXvbrzf01Yb1/kkE0XZvP4OOGgphxhh+/PqXFFU38tpt8zUJKNULIsKvvz6ZNzYVMm9kKgvGpOHxGJY/uY5vP7eRPy+fwTmndT3XYOOhSr774iaGJMWyfN5wv8fYdr2huJSor5af7uU+BJ0RERx2m44aCmXPrz3EOzuK+PHi8UzPGmB1OEqFnKnDkvnlpZO4cPJg+sdGkxzn4MWb5jF+cCK3vbCR1buKO33uezuKuObxtfTvF81fbplHbHTvN4PvTvv1hvw5mayFI0oTQUhqdnl45ON9/Pc/dnH2+IHc5Od2SaUiWVJcNM/fNJcxAxP52ZvbcXXQkfrC2kPc9sJGxg/uz1+/czoj0uIDEkvLRjLldU3kl9fz5GcHGJ4a59fJoNF26XGHuSaCIPHp3lIWP/gJv/3nbhaOTeeBq6fqBvNK+VlSv2j+/dwxFB1rZPXukuMe219ayy/+tp0zx6bz0i1zW/cBDoRUX9PQrqM1LH9yHc1uD49dO8uv79GbGoH2EVjA4zFc9egX7D56DAMYAw1ON8NT43j6htmcNb5nnVlKqd47e/xABifF8sLaQ1zQZte+Rz/ej8Nu474rp56wYYy/taw3dP97ucRF2/nLLfN6vFZRTzmibDp8NJjlHKpk46FKLpyUwdDkfojAkOR+LJuTFZD2SKXUV6LsNpbNyeKB9/dwoKyO7LR4iqobeWNzIUtnZ7VuBB9I8Q576zIQT1w/m6nDkv3+Hg67JoKg9rcth4mNtnH/VVN1M3mlLLB09jAeWr2Xv6w7xM8unsATn+7HY+DWhf6bPdwVEeGuC8czblAi80elBuQ9onXUUPBqdnl4e9tRzpuQoUlAKYsM7B/L+RMH8drGQoqqG/nL+ny+NmWwX0ftdOfGBdmcPrp3E9x6I0ZHDQWvz/JKqap3cmmbpWyVUqfe8rnDqap38q1nNlDf7OY7i0ZbHZJf6fDRILZyyxGS+kW3bsatlLLG/FGpjEyPZ+fRY5x72kC/d9ZaLdpuC47hoyKyWERyRSRPRO7qotyVImJExL/jp4JMfbOL93YWc9HkjNZFoZRS1hARrvPNGg632gAEyaghEbEDK4DzgEJgg4isNMbsbFcuEfg+sC5QsQSLD3aVUN/sZsnUoVaHopQCrps/ggWj0xgzKLxqA0DQLDExB8gzxuw3xjQDLwOXdlDuV8B9QGMAYwkKK7ccJqN/LHOyU6wORSkF2GwSlkkAelcjCGQiGAoUtLlf6DvWSkSmA8OMMf/o6oVE5FYRyRGRnNLSUv9HegpU1Tfz8Z5SvjZ1MHadMayUCrBgqRF09GnXuji2iNiAPwL/0d0LGWMeM8bMMsbMSk8PzU7Wf3x5FKfbaLOQUuqUCJZRQ4XAsDb3M4Ejbe4nApOAf4nIQWAesDJcO4xfzSlgfEYik4b2tzoUpVQECJamoQ3AGBHJFhEHsBRY2fKgMabaGJNmjBlhjBkBrAWWGGNyAhiTJXYdPcaXhdVcNWuYXzamVkqp7kTbbTitrhEYY1zAHcC7wC7gVWPMDhG5V0SWBOp9g9FrOYVE24XLpmuzkFLq1AiK4aMAxphVwKp2x+7upOyiQMZilSaXmzc3F3L+hIzWzSiUUirQHHYbTrfB4zHdLmmvs5oCbPWuEirrnVw1K9PqUJRSEaRl0qrT032tQBNBgL2yoYDBSbH825jQHO2klApNDrv3470nI4c0EQTQkaoGPtlbypUzM3XugFLqlGqpEWgisNjrGwsxBq6aOaz7wkop5UetiaAHHcaaCAJk46FKVnyUx6Jx6WSlnro1zpVSCrzDRwGcLtNNSU0EAXGwrI5bnsshIymWP1w11epwlFIR6KsagbvbsiedCETkipN9bjirqGvmhqfXY4zhmRvnkJoQ+P1PlVKqvZbO4qYA9xH8sQ/PDUulNU3c/OwGjlQ38sT1s8hOi7c6JKVUhHJEeQeoON3dNw31ZUKZDoPxMcawcusR/mvlDuqb3Tz4jWnMHK5LTSulrOOw24GejRrqSyLoPs1EgKr6Zn70+pe8v7OYacOSuf+qKYweGJ7rmyulQkdvho92mQhEZBsdf+ALkHESsYWdFR/l8eHuEn560XhuOmOkzhdQSgWF1pnFPRg+2l2N4BI/xBPWVu8q4fRRqdy6cJTVoSilVKtou/dLaU86i7tMBMaYQ509JiKfAwt6GVtYOVBWx/6yOq6bP9zqUJRS6jgxp2hCWVYfnhsWPtxdAsDZ4wdZHIlSSh2vN53FfUkEEd9Z/NHuEkYPTNCZw0qpoBPdOny0753Fl3f2ENCvt4GFk9omF+sOlPOtBdlWh6KUUifozeqj3XUWf62Lx/7R85DCz2d7S3G6DWeNH2h1KEopdQK/DR81xtzon5DCz+pdJfSPjWLm8AFWh6KUUifozeqj3U4oE5EzgUpjzJcicjWwENgH/MkY09S3UEOTx2P4KLeUhWPTW1f4U0qpYBJt89+EshXAFCBWRHKBBOAd4HTgKeCbfYw1JG07XE1ZbRPnnKbNQkqp4GSzCdF28UuN4CxjzAQRiQUOA8xk7PEAABHPSURBVAONMW4ReRT40g+xhqQPd5dgEzhzrCYCpVTwcthtfhk+2ghgjGkEDhlj3L77BnB29+IislhEckUkT0Tu6uDx20Rkm4hsEZHPRGRCtxFbzOn28Pa2o0zPGkBKvMPqcJRSqlPRUTa/LDExUETuxDtctOU2vvtd7sYuInZgBXAeUAhsEJGVxpidbYr9xRjziK/8EuABYHG3UVvo/vdyySup5ZHlM6wORSmlutTTGkF3ieBxILGD2wBPdPPcOUCeMWY/gIi8DFwKtCYCY8yxNuXjCfJJap/sKeXRj/ezbE4WiycNtjocpZTqkiPKD4nAGPPLPsQwFChoc78QmNu+kIh8F7gTcABnd/RCInIrcCtAVpY1K1uU1jRx56tbGTsogbsvCfoWLKWU8iYCP8wsvruLh40x5lddPb2j53TwIiuAFSJyDfBz4PoOyjwGPAYwa9asU15r8HgM//HaVmoanbx481z6OeynOgSllOo1f3UW13XwA3AT8ONunlsIDGtzPxM40kX5l4Gvd/Oalvjzx/v4ZE8pP79kAuMydNMZpVRo8EuNwBjzh5bbIpII/AC4Ee+H9h86e57PBmCMiGTjHXq6FLimbQERGWOM2eu7ezGwlyDzeV4Zf3gvl69NHcLyuRG/4KpSKoT4q7MYEUnB24b/TeBZYIYxprK75xljXCJyB/AuYAeeMsbsEJF7gRxjzErgDhE5F+9Q1Eo6aBayUlF1I99/aTMj0xP47eWTEdHdx5RSoSPa7ofhoyLye+ByvO3zk40xtb0JwhizCljV7tjdbW7/oDevdyo1uzzc/uJGGp1uHlk+k/iYvmzvrJRSp54jykZ9vavbct31EfwHMARvJ+4RETnm+6kRkWPdPDekPbR6L5vyq/jdlVMYPTDB6nCUUqrXHFE2v2xVGZErqlXXO3n68wNcMmUwl0wZYnU4Sil1Uhw9nFkckR/03Xlh3SHqmt3cvmi01aEopdRJc9h7NmpIE0E7jU43T39+gDPHpjNhSH+rw1FKqZPmr3kEEef1jYWU1TZz25mjrA5FKaX6pKdLTGgiaMPtMTz+6X6mZiYxb2SK1eEopVSfeIePdr8YgyaCNv65/SiHyuu57cxROmdAKRXytEZwEh79eD/ZafGcPzHD6lCUUqrPWpaY8G4h0zlNBD4Vdc1sO1zNN2YPw27T2oBSKvQ57N7Psu6ahzQR+OSVeCdNj9dF5ZRSYcIR5dvAvpshpJoIfPaW1ADoLGKlVNhw2H2JoJt+Ak0EPnkltcQ57AxJ6md1KEop5ReOKO/eKZoIeiivpJZR6QnYtH9AKRUmolv7CDQR9EheSS1jtFlIKRVGWvoIult4ThMBUNPo5Gh1I6M0ESilwkhMlPYR9Ni+Uu8OnNpRrJQKJ9G+zmJtGuqBlqGj2jSklAonOny0F/aW1OCw28hKibM6FKWU8hsdPtoL+0pqyU6LJ8qu/x1KqfDh0D6CnttbUqv9A0qpsNPSR6BNQ91odLopqKjXRKCUCjs6aqiHDpTV4TE6YkgpFX6ComlIRBaLSK6I5InIXR08fqeI7BSRL0VktYgMD2Q8HdnbMmJokCYCpVR4sXz4qIjYgRXAhcAEYJmITGhXbDMwyxgzBXgduC9Q8XQmr6QWm0B2WvypfmullAqoYBg+OgfIM8bsN8Y0Ay8Dl7YtYIz5yBhT77u7FsgMYDwdyiupYXhqPDG+xZmUUipcBEPT0FCgoM39Qt+xztwE/LOjB0TkVhHJEZGc0tJSP4b41WJzSikVblrmEVi51lBHy3h2uE2OiCwHZgG/7+hxY8xjxphZxphZ6enpfgvQ5fZwoKxOO4qVUmHJ0cM+gqgAxlAIDGtzPxM40r6QiJwL/Aw40xjTFMB4TnCooh6n2+jSEkqpsGSzCVE2sbRpaAMwRkSyRcQBLAVWti0gItOBR4ElxpiSAMbSob3F3hFDWiNQSoUrR5TNukRgjHEBdwDvAruAV40xO0TkXhFZ4iv2eyABeE1EtojIyk5eLiA25VfisNsYO0j3KVZKhadou83SpiGMMauAVe2O3d3m9rmBfP/ufJ5XxvSsZPo5dMSQUio8OaJsusREZyrrmtl59BgLRqdZHYpSSgWMw27THco6s3Z/OcbAgtGpVoeilFIBExNlw+nucMBmq4hNBJ/vKyPeYWdKZrLVoSilVMBE2200u9xdlonYRLAmr5y5I1Nb1+JQSqlwZOmooWB2tLqB/WV1nD5Km4WUUuFNO4s78XleOQCnj9KOYqVUeIu2C06X9hGcYE1eGSnxDsZn6PwBpVR4c0TZadIawfGMMXy+r4z5o1Kx2TpaDkkppcKHw659BCfYV1pH8bEmFmizkFIqAniHj2oiOM6afWWAzh9QSkWGaLu1i84FpTV55QxN7kdWSpzVoSilVMDp8NEObDtczYzhAxDR/gGlVPjT4aPtHGt0criqQUcLKaUiRrTdhlNrBF/ZU1QDwDhddlopFSEcUTYdPtrW7pZEoDUCpVSEiNHho8fLLaohISaKzAH9rA5FKaVOiZ6spxZZiaC4hrGDErSjWCkVMRxRmghaGWPILaphXEZ/q0NRSqlTRhNBG8XHmqhucOqIIaVURNFE0MbuomOAdhQrpSKL9hG0kesbMaQ1AqVUJImxukYgIotFJFdE8kTkrg4eXygim0TEJSJXBjKW3KIaBvWPITnOEci3UUqpoOKwskYgInZgBXAhMAFYJiIT2hXLB24A/hKoOFrs1o5ipVQEsrppaA6QZ4zZb4xpBl4GLm1bwBhz0BjzJdD1bIc+crk95JXWarOQUiriWN1ZPBQoaHO/0Hes10TkVhHJEZGc0tLSXj//YHkdzS6PLi2hlIo4VieCjmZtdb1xZieMMY8ZY2YZY2alp6f3+vm6tIRSKlJZnQgKgWFt7mcCRwL4fp3aU1SD3SaMHphgxdsrpZRlLO0sBjYAY0QkW0QcwFJgZQDfr1O7i2oYkRpHbLTdirdXSinLWFojMMa4gDuAd4FdwKvGmB0icq+ILAEQkdkiUghcBTwqIjsCEUtucQ3jdcSQUioC9aRGEBXIAIwxq4BV7Y7d3eb2BrxNRgFT3+wiv6KeK2YE9G2UUiooRVs9oSwY/GPrUYyBGVkDrA5FKaVOOav7CCzX7PLw0Id7mZKZxILRqVaHo5RSp5zVo4Ys9/rGQgorG/jheWN1DwKlVESyfK0hKzW53Dz84V6mZyWzaGzv5x4opVQ4sHqJCUu9uqGAI9WN3Km1AaVUBLPbBLut68/AsEwEjU43D3+Ux+wRAzhjdJrV4SillKW66zAOu0RQUFHPf762leJjTdo3oJRSQLS968/BgM4jOJWOVDXwfx/m8VpOATYRbl80itNHaW1AKaUcUV2vqhAWicAYw9LH1lJU3ciyOVncftYoBif1szospZQKCt2NHAqLpqH8inryK+r5xSWn8auvT9IkoJRSbTxw9dQuHw+LRLA5vwqAmcNTLI5EKaWCz9yRXU+oDYtEsCm/kjiHnbGDdJlppZTqrbBIBJvzq5iamUxUDyZOKKWUOl7If3I2NLvZdfQY07OSrQ5FKaVCUsgngm2Hq3F5jK4uqpRSJynkE8Hm/EoApmmNQCmlTkrIJ4JN+ZUMT40jLSHG6lCUUiokhXQiMMawKb9Km4WUUqoPQjoRHK5qoLSmSTuKlVKqD0I6EbRMJNMagVJKnbyQTwSx0TbGZSRaHYpSSoWskE4Em/IrmZKZ3KMdeJRSSnUsoJ+gIrJYRHJFJE9E7urg8RgRecX3+DoRGdHT125yudl5RCeSKaVUXwVsGWoRsQMrgPOAQmCDiKw0xuxsU+wmoNIYM1pElgK/A77R1evuKa7h3Ac+xun20Oz2aP+AUkr1USD3I5gD5Blj9gOIyMvApUDbRHApcI/v9uvAwyIixhjT2YvGRtsZN8jbJzAvO1W3olRKqT4KZCIYChS0uV8IzO2sjDHGJSLVQCpQ1raQiNwK3AqQlZXFim/OCFTMSikVcQLZR9DRJpntv+n3pAzGmMeMMbOMMbPS09P9EpxSSimvQCaCQmBYm/uZwJHOyohIFJAEVAQwJqWUUu0EMhFsAMaISLaIOIClwMp2ZVYC1/tuXwl82FX/gFJKKf8LWB+Br83/DuBdwA48ZYzZISL3AjnGmJXAk8DzIpKHtyawNFDxKKWU6lggO4sxxqwCVrU7dneb243AVYGMQSmlVNd0Sq5SSkU4TQRKKRXhNBEopVSEk1AbpCMiNUCu1XH4SRrtJs+FqHA5D9BzCUbhch5g7bkMN8Z0OBEroJ3FAZJrjJlldRD+ICI54XAu4XIeoOcSjMLlPCB4z0WbhpRSKsJpIlBKqQgXiongMasD8KNwOZdwOQ/QcwlG4XIeEKTnEnKdxUoppfwrFGsESiml/EgTgVJKRbiQSgTd7YEczETkoIhsE5EtIpLjO5YiIu+LyF7fv0G576aIPCUiJSKyvc2xDmMXr4d81+hLEQmqXYQ6OZd7ROSw79psEZGL2jz2E9+55IrIBdZEfSIRGSYiH4nILhHZISI/8B0PuevSxbmE1HURkVgRWS8iW33n8Uvf8Wzfnux7fXu0O3zHT3rPdr8zxoTED94VTPcBIwEHsBWYYHVcvYj/IJDW7th9wF2+23cBv7M6zk5iXwjMALZ3FztwEfBPvJsOzQPWWR1/D87lHuA/Oyg7wfd7FgNk+37/7Fafgy+2wcAM3+1EYI8v3pC7Ll2cS0hdF9//bYLvdjSwzvd//Sqw1Hf8EeA7vtu3A4/4bi8FXrEq9lCqEbTugWyMaQZa9kAOZZcCz/puPwt83cJYOmWM+YQTNwzqLPZLgeeM11ogWUQGn5pIu9fJuXTmUuBlY0yTMeYAkIf399ByxpijxphNvts1wC68W7+G3HXp4lw6E5TXxfd/W+u7G+37McDZePdkhxOvScu1eh04R0Q62rUx4EIpEXS0B3JXvyzBxgDvichG3x7MAIOMMUfB+8cADLQsut7rLPZQvU53+JpMnmrTRBcS5+JrUpiO9xtoSF+XducCIXZdRMQuIluAEuB9vLWVKmOMy1ekbazH7dkOtOzZfsqFUiLo0f7GQWyBMWYGcCHwXRFZaHVAARKK1+nPwChgGnAU+IPveNCfi4gkAH8F/t0Yc6yroh0cC/ZzCbnrYoxxG2Om4d2adw5wWkfFfP8GzXmEUiLoyR7IQcsYc8T3bwnwJt5fkuKW6rnv3xLrIuy1zmIPuetkjCn2/QF7gMf5qpkhqM9FRKLxfnC+aIx5w3c4JK9LR+cSqtcFwBhTBfwLbx9Bsnj3ZIfjYw2aPdtDKRH0ZA/koCQi8SKS2HIbOB/YzvF7Nl8P/M2aCE9KZ7GvBK7zjVKZB1S3NFUEq3Zt5ZfhvTbgPZelvtEd2cAYYP2pjq8jvrbkJ4FdxpgH2jwUctels3MJtesiIukikuy73Q84F29/x0d492SHE69JcOzZbnVPe29+8I582IO33e1nVsfTi7hH4h3lsBXY0RI73vbA1cBe378pVsfaSfwv4a2aO/F+i7mps9jxVndX+K7RNmCW1fH34Fye98X6Jd4/zsFtyv/Mdy65wIVWx98mrjPwNiN8CWzx/VwUiteli3MJqesCTAE2++LdDtztOz4Sb6LKA14DYnzHY33383yPj7Qqdl1iQimlIlwoNQ0ppZQKAE0ESikV4TQRKKVUhNNEoJRSEU4TgVJKRbhQ3LxeqVNCRFqGYgJkAG6g1He/3hhzuiWBKeVnOnxUqR4QkXuAWmPM/VbHopS/adOQUidBRGp9/y4SkY9F5FUR2SMivxWRb/rWpd8mIqN85dJF5K8issH3s8DaM1DqK5oIlOq7qcAPgMnAtcBYY8wc4Ange74yDwJ/NMbMBq7wPaZUUNA+AqX6boPxrdsjIvuA93zHtwFn+W6fC0xos9x8fxFJNN7195WylCYCpfquqc1tT5v7Hr76G7MB840xDacyMKV6QpuGlDo13gPuaLkjItMsjEWp42giUOrU+D4wy7fb1k7gNqsDUqqFDh9VSqkIpzUCpZSKcJoIlFIqwmkiUEqpCKeJQCmlIpwmAqWUinCaCJRSKsJpIlBKqQj3/wGVy+uLQ6/zowAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "ev.nbll(time_grid).plot()\n", "plt.ylabel('NBLL')\n", "_ = plt.xlabel('Time')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Integrated scores\n", "\n", "The two time-dependent scores above can be integrated over time to produce a single score [Graf et al. 1999](https://onlinelibrary.wiley.com/doi/abs/10.1002/%28SICI%291097-0258%2819990915/30%2918%3A17/18%3C2529%3A%3AAID-SIM274%3E3.0.CO%3B2-5). In practice this is done by numerical integration over a defined `time_grid`." ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.16609613377948" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ev.integrated_brier_score(time_grid) " ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.495728257223814" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ev.integrated_nbll(time_grid) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Next\n", "\n", "You can now look at other examples of survival methods in the [examples folder](https://nbviewer.jupyter.org/github/havakv/pycox/tree/master/examples).\n", "Or, alternatively take a look at\n", "\n", "- the more advanced training procedures in the notebook [02_introduction.ipynb](https://nbviewer.jupyter.org/github/havakv/pycox/blob/master/examples/02_introduction.ipynb).\n", "- other network architectures that combine autoencoders and survival networks in the notebook [03_network_architectures.ipynb](https://nbviewer.jupyter.org/github/havakv/pycox/blob/master/examples/03_network_architectures.ipynb).\n", "- working with DataLoaders and convolutional networks in the notebook [04_mnist_dataloaders_cnn.ipynb](https://nbviewer.jupyter.org/github/havakv/pycox/blob/master/examples/04_mnist_dataloaders_cnn.ipynb)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 4 }