{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# H2O Uplift Distributed Random Forest \n", "\n", "### Author: Veronika Maurerova veronika.maurerova@h2o.ai\n", "\n", "## Modeling Uplift \n", "\n", "Distributed Uplift Random Forest (Uplift DRF) is a classification tool for modeling uplift - the incremental impact of a treatment. This tool is very useful for example in marketing or in medicine. This machine learning approach is inspired by the A/B testing method. \n", "\n", "To model uplift, the analyst needs to collect data specifically - before the experiment, the objects are divided usually into two groups: \n", "\n", "- **treatment group**: receive some kind of treatment (for example customer get some type of discount) \n", "- **control group**: is separated from the treatment (customers in this group get no discount). \n", "\n", "Then the data are prepared and an analyst can gather information about the response - for example, whether customers bought a product, patients recovered from the disease, or similar. \n", "\n", "## Uplift approaches \n", "\n", "There are several approaches to model uplift: \n", "\n", "- Meta-learner algorithms\n", "- Instrumental variables algorithms\n", "- Neural-networks-based algorithms\n", "- Tree-based algorithms \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tree Based Uplift Algorithm\n", "\n", "Tree-based algorithm means in every tree, it takes information about treatment/control group assignment and information about response directly into a decision about splitting a node. The uplift score is the criterion to make a decision similar to the Gini coefficient in the standard decision tree. \n", "\n", "**Uplift metric to decide best split**\n", "\n", "The goal is to maximize the differences between the class distributions in the treatment and control sets, so the splitting criteria are based on distribution divergences. The distribution divergence is calculated based on the ``uplift_metric`` parameter. In H2O-3, three ``uplift_metric`` types are supported:\n", "\n", "- **Kullback-Leibler divergence** (``uplift_metric=\"KL\"``) - uses logarithms to calculate divergence, asymmetric, widely used, tends to infinity values (if treatment or control group distributions contain zero values). \n", "\n", "$ KL(P, Q) = \\sum_{i=0}^{N} p_i \\log{\\frac{p_i}{q_i}}$\n", "\n", "- **Squared Euclidean distance** (``uplift_metric=\"euclidean\"``) - symmetric and stable distribution, does not tend to infinity values. \n", "\n", "$ E(P, Q) = \\sum_{i=0}^{N} (p_i-q_i)^2$\n", "\n", "\n", "- **Chi-squared divergence** (``uplift_metric=\"chi_squared\"``) - Euclidean divergence normalized by control group distribution. Asymmetric and also tends to infinity values (if control group distribution contains zero values). \n", "\n", "$X^2(P, Q) = \\sum_{i=0}^{N} \\frac{(p_i-q_i)^2}{q_i}$\n", "\n", "where:\n", "\n", "- $P$ is treatment group distribution\n", "\n", "- $Q$ is control group distribution\n", "\n", "In a tree node the result value for a split is sum: $metric(P, Q) + metric(1-P, 1-Q)$. \n", "\n", "For the split gain value, the result within the node is normalized using a Gini coefficient (Eclidean or ChiSquared) or entropy (KL) for each distribution before and after the split.\n", "\n", "\n", "**Uplift score in each leaf is calculated as:**\n", "\n", "- $TP = (TY1 + 1) / (T + 2)$\n", "- $CP = (CY1 + 1) / (C + 2)$\n", "- $uplift\\_score = TP - CP $\n", "\n", "where:\n", "- $T$ how many observations in a leaf are from the treatment group (how many data rows in a leaf have ``treatment_column`` label == 1) \n", "- $C$ how many observations in a leaf are from the control group (how many data rows in the leaf have ``treatment_column`` label == 0)\n", "- $TY1$ how many observations in a leaf are from the treatment group and respond to the offer (how many data rows in the leaf have ``treatment_column`` label == 1 and ``response_column`` label == 1)\n", "- $CY1$ how many observations in a leaf are from the control group and respond to the offer (how many data rows in the leaf have ``treatment_column`` label == 0 and ``response_column`` label == 1)\n", "\n", "**Note**: A higher uplift score means more observations from treatment group respond to the offer than from control group. Which means offered treatment has positive effect. The uplift score can be negative, if more observations from control group respond to the offer without treatment.\n", "\n", "
\n", "
\n", "\n", "![Difference between SDT and Uplift DT](https://blog.h2o.ai/wp-content/uploads/2022/01/tree.png)\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## H2O Implementation (Major release 3.36)\n", "\n", "The H2O-3 implementation of Uplift DRF is based on DRF because the principle of training is similar to DRF. It is tree based uplift algorithm. Uplift DRF generates a forest of classification uplift trees, rather than a single classification tree. Each of these trees is a weak learner built on a subset of rows and columns. More trees will reduce the variance. Classification takes the average prediction over all of their trees to make a final prediction. \n", "\n", "Currently, in H2O-3 only binomial trees are supported, as well as the uplift curve metric and Area Under Uplift curve (AUUC) metric, normalized AUUC, and the Qini value. We are working on adding also regression trees and more metrics, for example, Qini coefficient, and more. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Start H2O-3" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "versionFromGradle='3.37.0',projectVersion='3.37.0.99999',branch='master',lastCommitHash='a1c95a407aec53a6cbc551484bd02d7d80b3bcb6',gitDescribe='jenkins-master-5950-dirty',compiledOn='2022-09-13 10:48:53',compiledBy='kurkami'\n" ] } ], "source": [ "import h2o\n", "from h2o.estimators.uplift_random_forest import H2OUpliftRandomForestEstimator\n", "\n", "import matplotlib as mpl\n", "import matplotlib.pyplot as plt\n", "import matplotlib.style as style\n", "\n", "import pandas as pd" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load data\n", "\n", "To demonstrate how Uplift DRF works, Criteo dataset is used. \n", "\n", "**Source:**\n", "\n", "Diemert Eustache, Betlei Artem} and Renaudin, Christophe and Massih-Reza, Amini, \"A Large Scale Benchmark for Uplift Modeling\", ACM, Proceedings of the AdKDD and TargetAd Workshop, KDD, London,United Kingdom, August, 20, 2018, https://ailab.criteo.com/criteo-uplift-prediction-dataset/.\n", "\n", "\n", "\n", "**Description:**\n", "\n", "- The dataset was created by The Criteo AI Lab\n", "- Consists of 13M rows, each one representing a user with 12 features, a treatment indicator and 2 binary labels (visits and conversions).\n", "- Positive labels mean the user visited/converted on the advertiser website during the test period (2 weeks).\n", "- The global treatment ratio is 84.6%.\n", "\n", "**Detailed description of the columns:**\n", "\n", "- **f0, f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11**: feature values (dense, float)\n", "- **treatment**: treatment group (1 = treated, 0 = control)\n", "- **conversion**: whether a conversion occured for this user (binary, label)\n", "- **visit**: whether a visit occured for this user (binary, label)\n", "- **exposure**: treatment effect, whether the user has been effectively exposed (binary)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
f0f1f2f3f4f5f6f7f8f9f10f11treatmentconversionvisitexposure
012.61636510.0596548.9764294.67988210.2805254.1154530.2944434.8338153.95539613.1900565.300375-0.1686791000
112.61636510.0596549.0026894.67988210.2805254.1154530.2944434.8338153.95539613.1900565.300375-0.1686791000
212.61636510.0596548.9647754.67988210.2805254.1154530.2944434.8338153.95539613.1900565.300375-0.1686791000
312.61636510.0596549.0028014.67988210.2805254.1154530.2944434.8338153.95539613.1900565.300375-0.1686791000
412.61636510.0596549.0379994.67988210.2805254.1154530.2944434.8338153.95539613.1900565.300375-0.1686791000
\n", "
" ], "text/plain": [ " f0 f1 f2 f3 f4 f5 f6 \\\n", "0 12.616365 10.059654 8.976429 4.679882 10.280525 4.115453 0.294443 \n", "1 12.616365 10.059654 9.002689 4.679882 10.280525 4.115453 0.294443 \n", "2 12.616365 10.059654 8.964775 4.679882 10.280525 4.115453 0.294443 \n", "3 12.616365 10.059654 9.002801 4.679882 10.280525 4.115453 0.294443 \n", "4 12.616365 10.059654 9.037999 4.679882 10.280525 4.115453 0.294443 \n", "\n", " f7 f8 f9 f10 f11 treatment conversion \\\n", "0 4.833815 3.955396 13.190056 5.300375 -0.168679 1 0 \n", "1 4.833815 3.955396 13.190056 5.300375 -0.168679 1 0 \n", "2 4.833815 3.955396 13.190056 5.300375 -0.168679 1 0 \n", "3 4.833815 3.955396 13.190056 5.300375 -0.168679 1 0 \n", "4 4.833815 3.955396 13.190056 5.300375 -0.168679 1 0 \n", "\n", " visit exposure \n", "0 0 0 \n", "1 0 0 \n", "2 0 0 \n", "3 0 0 \n", "4 0 0 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "control_name = \"control\"\n", "treatment_column = \"treatment\"\n", "response_column = \"visit\"\n", "feature_cols = [\"f\"+str(x) for x in range(0,12)]\n", "\n", "df = pd.read_csv(\"/home/0xdiag/bigdata/server/criteo/criteo-uplift-v2.1.csv\")\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prepare data\n", "\n", "Inspiration from: https://www.kaggle.com/code/hughhuyton/criteo-uplift-modelling/notebook\n", "\n", "To modeling uplift the treatment and control group data have to have similar distribution. In real world usually the control group is smaller than the treatment group. It is also a case of Crieteo dataset and we have to rebalanced the data to have a similar size." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Total number of samples: 13979592\n", "The dataset is largely imbalanced: \n", "1 0.85\n", "0 0.15\n", "Name: treatment, dtype: float64\n", "Percentage of users that visit: 4.7%\n", "Percentage of users that convert: 0.29%\n", "Percentage of visitors that convert: 6.21%\n" ] } ], "source": [ "print('Total number of samples: {}'.format(len(df)))\n", "print('The dataset is largely imbalanced: ')\n", "print(df['treatment'].value_counts(normalize = True))\n", "print('Percentage of users that visit: {}%'.format(100*round(df['visit'].mean(),4)))\n", "print('Percentage of users that convert: {}%'.format(100*round(df['conversion'].mean(),4)))\n", "print('Percentage of visitors that convert: {}%'.format(100*round(df[df[\"visit\"]==1][\"conversion\"].mean(),4)))" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Class 0: 2096937\n", "Class 1: 11882655\n", "Proportion: 6 : 1\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlsAAAF6CAYAAAAu3Z1/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAco0lEQVR4nO3de5SlVX3m8e/DRQGFtiONiDK0YpRBxzhjuxRHBy8otpdg8IJBswYHV4+zRp14v8S1bPES0BEykWSAiDgqygAiBhXl4oB3kyZeErGJGBGEiI1Wc7ERkf7NH+9b4eRQ3fVWd++uOtXfz1pnnTp7v/t9f+cUq3h6v7t2paqQJElSGzvNdwGSJEmLmWFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZpoSZYnqSS7zHct05K8NMlF2/B830/ylP7r1Uk+vg3P/bYkH9pW55N0T4YtaYFKctvIY2OS20dev3QbXucjSd69rc43x2sPCkpJHp7knCQ3Jbk5yfeSvC7Jztur1pFaPpLkN0lu7R//kORPkyyZPqaqzqyqZw4816yffVU9sqou28rSSfKUJD8dO/d7q+oVW3tuSZtm2JIWqKq67/QDuBZ43kjbmdPHLaQZnRaSHAh8C7gO+HdVtQR4EbAC2HOeynpfVe0JLANeDjwB+FqS+2zLiyz27620ozBsSRNmenYiyZuT/Aw4I8lOSd6S5EdJfpHk7CS/MzLmnCQ/62eFvpzkkX37KuClwJv6GbML+vZrkryxn0H6VZLTkzwgyYX9bM4lSZaOnP8JSb6eZH2S707f8ur7LkvyriRf68delGTvvvvL/fP6/vqHzPCW3wl8vapeV1X/DFBVV1XV0VW1fobP5+VJftBf65+S/NeRvr2TfLav85dJvpJkp77vzUmu78ddleTps30vqurXVfW3wO8D96cLXiQ5JslX+6+T5KQkP09yS5K/T/KoWT77Nyf5HvCrJLv0bYeNXHq3JP+3r/XvkvzeyHusJA8bef2RJO/ug+CFwH4jM6T7Zey2ZJLfT3fbcn3/vfu3I33XJHlD/9/FzX0Nu832OUk7OsOWNJn2BX4HOABYBbwaeD5wKLAfMAX8xcjxFwK/C+wD/B1wJkBVndZ//b5+xux5I2NeADwDeDjwvP4cb6ObzdkJeA1AkgcBnwPe3df0BuBTSZaNnOtouiCyD3Cv/hiA/9Q/36+//jdmeK+HAecO+1gA+DnwXGCv/ponJfkPfd/rgZ/27+EB/fupJI8AXgU8rp+xOhy4ZugFq+pW4GLgyTN0P5PufT4cWAK8GPjFLJ/9HwLPoftcfjvDOY8AzqH7vD8BnJ9k11lq/BWwErhhZIb0htFjkjwc+CTwx3Sf0eeBC5Lca+SwFwPPAh4CPBo4ZnPXlbTAw1aSD/f/GvyHAceelOQ7/eMfk6zfDiVK82Uj8I6quqOqbgdeCfxJVf20qu4AVgMvnL4NVVUfrqpbR/p+LyNrjDbhg1V1Y1VdD3wF+FZVfbuqfg18Gvj3/XEvAz5fVZ+vqo1VdTGwBnj2yLnOqKp/7Gs9G3jMHN7r/YF/HnpwVX2uqn5UncuBi7g7BN0JPBA4oKrurKqvVPcHYu8C7g0cnGTXqrqmqn40hxoBbqALP+PupLvdeRCQqvrB9AzdZvx5VV3Xf14zuaKqzq2qO4ETgd3obmVuraOAz1XVxf25/yewO/DEsdpuqKpfAhcwt++ltENa0GEL+Ajdv6BmVVWvrarHVNVjgA8C5zWsS5pv6/rQM+0A4NP9rZ/1wA/oAsQDkuyc5Pj+FuMt3D1jszebd+PI17fP8Pq+I9d+0fS1++s/iS7UTPvZyNcbRsYO8Yuxc21WkpVJvtnfJlxPF/qm3+v7gauBi/pbjG8BqKqr6WZzVgM/T3JWkv3mUCPAg4BfjjdW1ZeAk+lmGn+e5LQke81yruuG9lfVRrrZurnWO5P9gJ+Mnfs6uvc2bWu+l9IOaUGHrar6MmM/vJIcmOQLSa7o11scNMPQP6SbCpcWqxp7fR2wsqruN/LYrZ+VOpruttNhdLexlvdjsolzzdV1wMfGrn2fqjp+C97HTC6hu6U5qyT3Bj5FNyPzgKq6H92tsEB3u6+qXl9VD6VbZ/W66bVZVfWJqnoSXXgs4IQh1+yve1+6z/crM/VX1Z9X1WOBg+luJ75xumsTp5ztc9l/5No7AQ+mm1mDLgDtMXLsvnM47w1073/63Omvdf0s4yRtxoIOW5twGvDq/gfXG4C/HO1McgDdWoIvzUNt0nw5BXhP/98/SZYlOaLv2xO4g26GaA/gvWNjbwQeuhXX/jjwvCSH97Nou6VbxP/gAWPX0d0S3dz13wE8Mcn7k+wLkORhST6e5H5jx96L7nbgOuC3SVbSrZmiH/fcfmyAm+lm/zYmeUSSp/Vh7dd0M3cbZys+yb2TPBY4n26d3BkzHPO4JI/v11T9qj//9Lm39LN/bJIj+9vEf0z3/f1m3/cd4Oj+e/EsunV8024E7r+ZW8hnA89J8vS+3tf35/76FtQoqTdRYav/1+MTgXOSfAc4lXveXngJcG5V3bWdy5Pm0/8C/pru9titdP/jfXzf91G6W0PXA1dy9/+Up51Ot1ZpfZLz53rhqrqObubsbXQh5zq6mZtZf75U1QbgPXTbJqxPco91R/3aqUPoZuS+n+RmutmrNcCtY8feSrdw/2y68HM03ecy7XfpZspuA74B/GVV/T+6gHY8cBPdbbJ9gLdupvQ39Z/zL+g+3yuAJ/aL0MftBfxVX89P+jHv7/u29LP/DN36qingj4Aj+zVWAP+D7hca1tP9tuO/nLeq1tLN+v9Tf81/deuxqq6iW4P3QbrP4nl0W478Zg61SRqTbm3owpVkOfDZqnpUv87hqqra5PqNJN8G/ntV+S8xSZI07yZqZquqbgF+nORF8C/714zuL3MQsJTuX6ySJEnzbkGHrSSfpAtOj0i3ieOxdNPixyb5LvB9utsX014CnFULfbpOkiTtMBb8bURJkqRJtqBntiRJkiadYUuSJKmhBfsX5ffee+9avnz5fJchSZI0qyuuuOKmqlo2U9+CDVvLly9nzZo1812GJEnSrJL8ZFN93kaUJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgaFrSQPS3Jqku8luSvJZQPGPC7JGUmuTrIhyVVJ3pFkt62uWpIkaUIM3frhkcCzgW8Cuw4ccxRwIHAC8EPg0cC7+ucXzK1MSZKkyTQ0bF1QVZ8BSHIusPeAMcdX1U0jry9L8mvg1CQHVNUm96OQJElaLAbdRqyqjXM98VjQmvbt/nm/uZ5PkiRpEm3vBfKHABuBH23n60qSJM2L7Ra2kuwLvB34WFX9fBPHrEqyJsmadevWba/SJEmSmtkuYSvJvYCzgduA127quKo6rapWVNWKZctm/FuOkiRJE6X5H6JOEuCjdL/R+B+raqr1NSVJkhaK5mEL+DPgCOAZVbV2O1xvUXvIWz433yVoQvz4+OfMdwmSJBqHrSRvBV4FvLiqvtryWpIkSQvRoLCVZA+6TU0BHgTsleSF/evPV9WGJFcDl1fVsf2Yo4H3Ah8Brk/yhJFT/qiqXAEvSZIWvaEzW/sA54y1Tb9+CHBNf66dR/qf2T8f0z9GvZwuhEmSJC1qg8JWVV0DZJZjlo+9PoZ7hixJkqQdyvbe1FSSJGmHYtiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZIkNWTYkiRJasiwJUmS1JBhS5IkqSHDliRJUkOGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZIkNWTYkiRJasiwJUmS1JBhS5IkqSHDliRJUkOGLUmSpIYMW5IkSQ0NCltJHpbk1CTfS3JXkssGjluS5IwkU0luTnJmkvtvVcWSJEkTZJeBxz0SeDbwTWDXOZz/bODhwCuAjcAJwPnAk+dwDkmSpIk1NGxdUFWfAUhyLrD3bAOSHAI8Ezi0qr7ct10PfCvJYVV1yRbWLEmSNDEG3Uasqo1bcO6VwI3TQas/z98AP+77JEmSFr2WC+QPAtbO0P6Dvk+SJGnRaxm2lgLrZ2if6vvuIcmqJGuSrFm3bl3D0iRJkraPBbX1Q1WdVlUrqmrFsmXL5rscSZKkrdYybE0BS2ZoX9r3SZIkLXotw9ZaZl6btam1XJIkSYtOy7B1IbBvkidNNyRZATy075MkSVr0Bu2zlWQPuk1NAR4E7JXkhf3rz1fVhiRXA5dX1bEAVfWNJBcBH03yBu7e1PSr7rElSZJ2FEM3Nd0HOGesbfr1Q4Br+nPtPHbMUcBJwIfpZtE+C7xmSwqVJEmaRIPCVlVdA2SWY5bP0LYeeHn/kCRJ2uEsqK0fJEmSFhvDliRJUkOGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZIkNWTYkiRJasiwJUmS1JBhS5IkqSHDliRJUkOGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqaFDYSnJwkkuTbEhyQ5Ljkuw8YNyKJBcl+WX/uCTJ47e+bEmSpMkwa9hKshS4BCjgCOA44PXAO2cZt38/bhfgj/rHLsDFSQ7YurIlSZImwy4DjnklsDtwZFXdQheW9gJWJ3lf3zaT5wB7An9QVTcDJPk6cBPwbOB/b3X1kiRJC9yQ24grgS+Ohaqz6ALYoZsZtyvwW+BXI2239W2ZY52SJEkTaUjYOghYO9pQVdcCG/q+TflUf8wHkuyTZB/gJGAKOGfLypUkSZosQ8LWUmD9DO1Tfd+MquoG4KnAC4Ab+8eRwOFVtW7OlUqSJE2gZls/JHkg3QzWFXS3Ilf2X38uyb/ZxJhVSdYkWbNunXlMkiRNviFhawpYMkP70r5vU95It27rhVX1har6At0s113AG2YaUFWnVdWKqlqxbNmyAaVJkiQtbEPC1lrG1mb12zrswdharjEHAd+vqjunG6rqN8D3gQPnXqokSdLkGRK2LgQOT7LnSNtRwO3A5ZsZ9xPgUUnuNd2Q5N7Ao4Br5l6qJEnS5BkStk4B7gDOS3JYklXAauDE0e0gklyd5PSRcR8C9gM+neQ5SZ4LnA88EDhtG9UvSZK0oM0atqpqCng6sDNwAd3O8ScB7xg7dJf+mOlxVwDPotvY9GPAR+luPT6jqr67LYqXJEla6IbsIE9VXQk8bZZjls/Qdilw6RZVJkmStAg02/pBkiRJhi1JkqSmDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZIkNWTYkiRJasiwJUmS1JBhS5IkqSHDliRJUkOGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZIkNWTYkiRJasiwJUmS1JBhS5IkqaFBYSvJwUkuTbIhyQ1Jjkuy88CxRyb52yS3J/lFki8kuc/WlS1JkjQZZg1bSZYClwAFHAEcB7weeOeAsa8APgFcCKwEXgH8ENhly0uWJEmaHENCzyuB3YEjq+oW4OIkewGrk7yvb7uHJHsDJwGvrqq/Gun69NYWLUmSNCmG3EZcCXxxLFSdRRfADt3MuBf3z/9nC2uTJEmaeEPC1kHA2tGGqroW2ND3bcrjgauAY5P8NMmdSb6V5IlbXK0kSdKEGRK2lgLrZ2if6vs2ZV/gEcDbgTcDzwN+BXwhyQNmGpBkVZI1SdasW7duQGmSJEkLW8utHwLcFzi2qs6sqi8AzwfuAl4104CqOq2qVlTVimXLljUsTZIkafsYEramgCUztC/t+zY3roDLphv6dV9XAAcPL1GSJGlyDQlbaxlbm5Vkf2APxtZyjfkB3exWxtoDbJxDjZIkSRNrSNi6EDg8yZ4jbUcBtwOXb2bcZ/vnp043JFkCPBb47hzrlCRJmkhDwtYpwB3AeUkOS7IKWA2cOLodRJKrk5w+/bqq1gCfAU5P8p+TPAf4a+BO4C+24XuQJElasGYNW1U1BTwd2Bm4gG7n+JOAd4wdukt/zKiXAecDJwLn0gWtp/XnlCRJWvQG/dmcqroSeNosxyyfoe024L/1D0mSpB1Oy60fJEmSdniGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZIkNWTYkiRJasiwJUmS1JBhS5IkqSHDliRJUkOGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLU0KCwleTgJJcm2ZDkhiTHJdl56EWS7JRkTZJK8twtL1eSJGmy7DLbAUmWApcAVwJHAAcCH6ALam8feJ1XAA/ewholSZIm1pCZrVcCuwNHVtXFVXUK8E7gdUn2mm1wH9beA/zJVlUqSZI0gYaErZXAF6vqlpG2s+gC2KEDxr8L+Bpw6dzLkyRJmmxDwtZBwNrRhqq6FtjQ921SkkcD/wV4w5YWKEmSNMmGhK2lwPoZ2qf6vs35IHByVV09pJgkq/qF9GvWrVs3ZIgkSdKC1mzrhyQvAR4BvHvomKo6rapWVNWKZcuWtSpNkiRpuxkStqaAJTO0L+377iHJrsD7gROAnZLcD5heTH+fJHvOvVRJkqTJMyRsrWVsbVaS/YE9GFvLNeI+dFs9nEgXyKaA7/Z9ZwHf3pJiJUmSJs2s+2wBFwJvTLJnVd3atx0F3A5cvokxtwFPHWvbF/gk8DbgS1tQqyRJ0sQZErZOAV4DnJfkBOChwGrgxNHtIJJcDVxeVcdW1W+By0ZPkmR5/+XfV9W3tr50SZKkhW/WsFVVU0meDpwMXED3m4kn0QWu8XMN/hM+kiRJO4IhM1tU1ZXA02Y5Zvks/dcAGVqYJEnSYtBs6wdJkiQZtiRJkpoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZIkNWTYkiRJasiwJUmS1JBhS5IkqSHDliRJUkOGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4PCVpKDk1yaZEOSG5Icl2TnWcY8LskZSa7ux12V5B1Jdts2pUuSJC18u8x2QJKlwCXAlcARwIHAB+iC2ts3M/So/tgTgB8Cjwbe1T+/YKuqliRJmhCzhi3glcDuwJFVdQtwcZK9gNVJ3te3zeT4qrpp5PVlSX4NnJrkgKr6ydaVLkmStPANCVsrgS+Ohaqz6GasDgUumGnQWNCa9u3+eT/AsCVJC8XqJfNdgSbF6pvnu4KJM2TN1kHA2tGGqroW2ND3zcUhwEbgR3McJ0mSNJGGhK2lwPoZ2qf6vkGS7Eu3xutjVfXzoeMkSZIm2XbZ+iHJvYCzgduA127muFVJ1iRZs27duu1RmiRJUlNDwtYUMNPN/KV932YlCfBR4JHAs6tqk2Oq6rSqWlFVK5YtWzagNEmSpIVtyAL5tYytzUqyP7AHY2u5NuHP6LaMeEZVDTlekiRp0Rgys3UhcHiSPUfajgJuBy7f3MAkbwVeBbysqr66xVVKkiRNqCFh6xTgDuC8JIclWQWsBk4c3Q6i3yn+9JHXRwPvpbuFeH2SJ4w8vEcoSZJ2CLPeRqyqqSRPB06m21NrPXASXeAaP9fon/B5Zv98TP8Y9XLgI3OsVZIkaeIMWbNFVV0JPG2WY5aPvT6Ge4YsSZKkHcp22fpBkiRpR2XYkiRJasiwJUmS1JBhS5IkqSHDliRJUkOGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhgxbkiRJDRm2JEmSGjJsSZIkNWTYkiRJasiwJUmS1JBhS5IkqSHDliRJUkOGLUmSpIYMW5IkSQ0ZtiRJkhoybEmSJDVk2JIkSWrIsCVJktSQYUuSJKkhw5YkSVJDhi1JkqSGDFuSJEkNDQpbSQ5OcmmSDUluSHJckp0HjFuS5IwkU0luTnJmkvtvfdmSJEmTYZfZDkiyFLgEuBI4AjgQ+ABdUHv7LMPPBh4OvALYCJwAnA88eYsrliRJmiCzhi3glcDuwJFVdQtwcZK9gNVJ3te33UOSQ4BnAodW1Zf7tuuBbyU5rKou2TZvQZIkaeEachtxJfDFsVB1Fl0AO3SWcTdOBy2Aqvob4Md9nyRJ0qI3JGwdBKwdbaiqa4ENfd/gcb0fzDJOkiRp0RgStpYC62don+r7tvU4SZKkRWPImq3tJskqYFX/8rYkV81nPZooewM3zXcRC0lOmO8KpEXBny3j3pn5rmChOmBTHUPC1hSwZIb2pX3f5sYtm8u4qjoNOG1ATdK/kmRNVa2Y7zokLS7+bNG2MOQ24lrG1lgl2R/Yg5nXZG1yXG9Ta7kkSZIWnSFh60Lg8CR7jrQdBdwOXD7LuH2TPGm6IckK4KF9nyRJ0qI3JGydAtwBnJfksH5d1WrgxNHtIJJcneT06ddV9Q3gIuCjSY5M8nzgTOCr7rGlBrz9LKkFf7Zoq6WqZj8oORg4GTiE7jcMPwSsrqq7Ro65Brisqo4ZabsfcBLwB3TB7rPAa6rKxYaSJGmHMChsSZIkacsM+kPU0kKU5GFJTk3yvSR3JblsvmuSNPmSHJzk0iQbktyQ5LgkO893XZpcC2qfLWmOHgk8G/gmsOs81yJpEUiyFLgEuBI4AjgQ+ADd5MTb57E0TTBvI2piJdmpqjb2X58L7F1VT5nfqiRNsiRvBd4EHDD9S2BJ3kT3i2H7jv2dYGkQbyNqYk0HLUnahlYCXxwLVWcBuwOHzk9JmnSGLUmS7naPjber6lpgAzNv1C3NyrAlSdLdltJtcTRuqu+T5sywJUmS1JBhS5Kku00BS2ZoX9r3SXNm2JIk6W5rGVublWR/YA/G1nJJQxm2JEm624XA4Un2HGk7CrgduHx+StKkc1NTTawke9BtagrwIGCvJC/sX3++qjbMT2WSJtgpwGuA85KcADyUbo+tE91jS1vKTU01sZIsB368ie6HVNU1268aSYtFkoOBk4FD6H4z8UPA6qq6az7r0uQybEmSJDXkmi1JkqSGDFuSJEkNGbYkSZIaMmxJkiQ1ZNiSJElqyLAlSZLUkGFLkiSpIcOWJElSQ4YtSZKkhv4/7K3m0HMYDecAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Print proportion of a binary column\n", "# https://www.kaggle.com/code/hughhuyton/criteo-uplift-modelling/notebook\n", "def print_proportion(df, column):\n", " fig = plt.figure(figsize = (10,6))\n", " target_count = df[column].value_counts()\n", " print('Class 0:', target_count[0])\n", " print('Class 1:', target_count[1])\n", " print('Proportion:', int(round(target_count[1] / target_count[0])), ': 1')\n", " target_count.plot(kind='bar', title='Treatment Class Distribution', color=['#2077B4', '#FF7F0E'], fontsize = 15)\n", " plt.xticks(rotation=0) \n", " \n", "print_proportion(df, treatment_column)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(11183673, 16)\n", "(2795919, 16)\n" ] } ], "source": [ "from sklearn.model_selection import train_test_split\n", "train_df, test_df = train_test_split(df, test_size=0.2, random_state=42, stratify=df['treatment'])\n", "print(train_df.shape)\n", "print(test_df.shape)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "del(df)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Class 0: 1677550\n", "Class 1: 9506123\n", "Proportion: 6 : 1\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAF6CAYAAAAEbWzQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAVsklEQVR4nO3debCld13n8c83C0uEBEgamCCk2QKCDo7EQhhFEIY9YKEshpkpUCfDVLkNW4CZGoOCsogU68RWIKVsArInUTbZBKzpsCmBMCzBkADphHQWEgKS7/zxPBfuXLpzf52+p+9J9+tV1dX3nuec5/nec1M37/49zz2nujsAAFyzgzZ7AACA6wLRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AUuhqrZWVVfVIZs9y4qqelxVvXsD9/fZqrrP/PHJVfWaDdz3M6vqLzZqf8CPEk2wYFV1+ao/V1fVlas+f9wGHufUqnr2Ru1vD489FDxVdWxVvamqLqyqS6rqM1X1pKo6eF/NumqWU6vqu1V12fznn6vqj6vqiJX7dPdru/sBg/ta97nv7rt29wf2cvRU1X2q6mtr9v1H3f2be7tvYPdEEyxYd99o5U+Sf0ly/KrbXrtyv2VaYVmEqrp9kn9Mcm6Sn+ruI5I8KslxSW68SWM9v7tvnGRLkick+bkk/1BVP7aRB9nfv7dwoBBNsElWVguq6qSq+kaSV1fVQVX19Kr6UlVdVFVvrKqbrXrMm6rqG/MqzYeq6q7z7ScmeVySp80rWO+cbz+nqp46r+h8u6peWVW3qKoz5tWV91bVTVft/+eq6qNVtbOqPr1yKmne9oGq+sOq+of5se+uqqPmzR+a/945H/+eu/iSn5Xko939pO7+epJ099ndfUJ379zF8/OEqvrcfKwvV9V/XbXtqKp61zznt6rqw1V10LztpKo6b37c2VV1v/W+F939ne7+P0kenuTITAGVqnp8VX1k/riq6kVVdUFVXVpV/1RVP7nOc39SVX0myber6pD5tvuvOvQNquqv51k/UVV3W/U1dlXdYdXnp1bVs+egOyPJ0atWLI+uNaf7qurhNZ0O3Dl/735i1bZzquop838Xl8wz3GC95wkOdKIJNtctk9wsyTFJTkzy20l+OckvJjk6ycVJXr7q/mckuWOSmyf5RJLXJkl3b5s/fv68gnX8qsf8SpL/kOTYJMfP+3hmptWVg5L8TpJU1a2SnJbk2fNMT0nyN1W1ZdW+TsgUFDdPcr35Pkly7/nvm8zH/9guvtb7J3nz2NOSJLkgycOSHD4f80VV9TPzticn+dr8Ndxi/nq6qu6U5LeS/Oy8gvTAJOeMHrC7L0vyniS/sIvND8j0dR6b5Igkj05y0TrP/a8leWim5+Vfd7HPRyR5U6bn+3VJ3lZVh64z47eTPDjJ+atWLM9ffZ+qOjbJ65P8Xqbn6PQk76yq662626OTPCjJbZP82ySPv6bjAvsomqrqVfO/zv558P6Prqqz5n8lvW7R88EmujrJ73f3Vd19ZZInJvkf3f217r4qyclJfnXl9E53v6q7L1u17W616hqc3Xhpd3+zu89L8uEk/9jdn+zu7yR5a5J/N9/vPyY5vbtP7+6ru/s9SbYneciqfb26u78wz/rGJD+9B1/rkUm+Pnrn7j6tu7/Ukw8meXd+GDPfS/JvkhzT3d/r7g/39Eaa309y/SR3qapDu/uc7v7SHsyYJOdnipi1vpfpNOKdk1R3f25lxewavKS7z52fr105s7vf3N3fS/KnSW6Q6RTh3npMktO6+z3zvv8kyQ2T3GvNbOd397eSvDN79r2EA9K+Wmk6NdO/aNZVVXdM8owk/76775rpX0qwv9oxx8uKY5K8dT6lsjPJ5zKFwC2q6uCqeu586u7S/HAF5ahcs2+u+vjKXXx+o1XHftTKsefj/3ymOFnxjVUfX7HqsSMuWrOva1RVD66qj8+n33ZmireVr/UFSb6Y5N3zqbunJ0l3fzHTz4yTk1xQVW+oqqP3YMYkuVWSb629sbvfn+RlmVb+LqiqbVV1+Dr7Ond0e3dfnWn1bE/n3ZWjk3x1zb7PzfS1rdib7yUckPZJNHX3h7Lmh1BV3b6q/raqzpyvR7jzvOm/JHl5d188P/aCfTEjbJJe8/m5SR7c3TdZ9ecG8yrRCZlO59w/0+mhrfNjajf72lPnJvmrNcf+se5+7rX4OnblvZlOFa6rqq6f5G8yrZDcortvkukUUyXTabTufnJ33y7TdUhPWrl2qbtf190/nykCO8nzRo45H/dGmZ7fD+9qe3e/pLvvnuQumU7TPXVl0252ud7zcutVxz4oyY9nWulKppA5bNV9b7kH+z0/09e/su+aj3XeOo8DrsFmXtO0Lclvzz+AnpLkFfPtxyY5dr7Y9ONVNbRCBfuJU5I8p6qOSZKq2lJVj5i33TjJVZlWbA5L8kdrHvvNJLfbi2O/JsnxVfXAeVXrBjVdrP7jA4/dkelU4zUd//eT3KuqXlBVt0ySqrpDVb2mqm6y5r7Xy3SabUeSf62qB2e6pijz4x42P7aSXJJpNe7qqrpTVf3SHF3fybSSdvV6w1fV9avq7kneluk6slfv4j4/W1X3mK85+va8/5V9X9vn/u5V9cj59OvvZfr+fnze9qkkJ8zfiwdlus5txTeTHHkNp2bfmOShVXW/ed4nz/v+6LWYEZhtSjTN/5q7V5I3VdWnkvxZfrhsf0imC13vk+kiyj/fxQ9U2F+9OMk7Mp12uizT/0DvMW/7y0ynXM5LclZ++D/XFa/MdC3Pzqp6254euLvPzbSS9cxMsXJuppWUdX9OdPcVSZ6T6df1d1bVj1yXM19bdM9MK2SfrapLMq0mbU9y2Zr7XpbpAvU3ZoqYEzI9LyvumGnl6vIkH0vyiu7++0yh9dwkF2Y6/XTzTKf7d+dp8/N8Uabn98wk95ovtl7r8CR/Ps/z1fkxL5i3Xdvn/u2Zrj+6OMl/SvLI+RqkJPndTBfu78z023k/2G93fz7Thd5fno/5/53S6+6zM12j9tJMz8XxmV7q4rt7MBuwRk3XTu6DA1VtTfKu7v7J+TqAs7v7R65vqKpTMl2o+ur58/clefr868AAAJtiU1aauvvSJF+pqkclP3j9k5XXJ3lbplWm1PQaMMcm+fImjAkA8AP76iUHXp9pCf1ONb2Y329kWm7+jar6dJLPZjotkCR/l+Siqjoryd8neWp3X7Qv5gQA2J19dnoOAOC6zCuCAwAMEE0AAAMW/s7bRx11VG/dunXRhwEA2Gtnnnnmhd29ZVfbFh5NW7duzfbt2xd9GACAvVZVX93dNqfnAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGHLLZAxyobvv00zZ7BK4jvvLch272CADEShMAwBDRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AQAMEE0AAANEEwDAANEEADBANAEADBBNAAADRBMAwADRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AQAMEE0AAANEEwDAANEEADBANAEADBBNAAADRBMAwADRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AQAMEE0AAANEEwDAANEEADBANAEADBBNAAADRBMAwADRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AQAMGIqmqnpsVX2iqi6vqvOq6i+r6uhFDwcAsCzWjaaqeniS1yf5aJJHJDkpyb2TnFZVVqoAgAPCIQP3OSHJJ7r7t1ZuqKpLk7w9yZ2SfG5BswEALI2RlaJDk1yy5rad89+1odMAACypkWh6VZJfqKr/XFWHV9WxSZ6d5P3dfdZixwMAWA7rRlN3n5bk8Um2ZVpxOjvJwUl+ZXePqaoTq2p7VW3fsWPHBo0KALB5Ri4Ev2+SU5K8OMl9kzw2yc2SvLWqDt7VY7p7W3cf193HbdmyZSPnBQDYFCMXgr8wyTu6+6SVG6rqU0k+n+m36d6ymNEAAJbHyDVNd07yqdU3dPfZSa5McvsFzAQAsHRGoumrSX5m9Q1V9RNJbpjknAXMBACwdEZOz52S5EVVdX6SM5LcIsn/yhRMpy9uNACA5TESTS9J8t0k/y3JEzO9RtNHkjyju7+9uNEAAJbHutHU3Z3kf89/AAAOSN47DgBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABQ9FUVYdU1dOr6v9W1VVV9bWqetGihwMAWBaHDN7v1CS/lORZST6f5NZJ7rKgmQAAls660VRVD0rymCR36+6zFj8SAMDyGTk99+tJ3i+YAIAD2Ug03SPJF6rqZVV1aVVdUVVvqaqjFz0cAMCyGImmWyZ5fJKfTvLYJE9Icvckb62qWthkAABLZORC8Jr/PKK7L0qSqvp6kg9mujj8fT/ygKoTk5yYJLe5zW02bFgAgM0ystJ0cZJ/Wgmm2UeSfDe7+Q267t7W3cd193FbtmzZgDEBADbXSDR9LtNK01qV5OqNHQcAYDmNRNO7kvxUVR216rZ7Jzk0yacXMhUAwJIZiaZtSS5K8s6qOr6qTkjyV0ne290fWeh0AABLYt1o6u5LM13wfXGSNyR5eaaLvx+92NEAAJbH0NuodPcXkzxkwbMAACytoTfsBQA40IkmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAbscTRV1a2q6vKq6qq60SKGAgBYNtdmpekFSS7f6EEAAJbZHkVTVd07yYOS/MlixgEAWE6HjN6xqg5O8tIkf5Bk56IGAgBYRnuy0vTEJNdP8vIFzQIAsLSGoqmqjkzyh0me1N3fG7j/iVW1vaq279ixY29nBADYdKMrTc9J8vHuPn3kzt29rbuP6+7jtmzZcu2nAwBYEute01RVd03y60nuXVU3mW8+bP77iKr6fndfuaD5AACWwsiF4HdMcmiSj+1i29eSvDLJb27kUAAAy2Ykmj6S5L5rbntQkpOSPCTJlzd6KACAZbNuNHX3hUk+sPq2qto6f/jh7vZClwDAfs97zwEADLhW0dTdp3Z3WWUCAA4UVpoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGCCaAAAGiCYAgAGiCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGCAaAIAGLBuNFXVo6rqHVV1XlVdXlVnVtWv7YvhAACWxSED93lSkq8k+e9JLkzykCSvq6qjuvulixwOAGBZjETT8d194arP319VR2eKKdEEABwQ1j09tyaYVnwyydEbPw4AwHK6theC3zPJFzZyEACAZbbH0VRV90vyy0leeA33ObGqtlfV9h07duzFeAAAy2GPoqmqtiZ5XZK3d/epu7tfd2/r7uO6+7gtW7bs3YQAAEtgOJqq6mZJzkjy1SSPW9hEAABLaCiaquqwJO9Kcr0kD+vuKxY6FQDAkln3JQeq6pAkb0pyxyT36u4LFj4VAMCSGXmdpldkekHL301yZFUduWrbJ7v7qoVMBgCwREai6QHz3y/exbbbJjlnw6YBAFhS60ZTd2/dB3MAACy1a/vilgAABxTRBAAwQDQBAAwYuRAcgOuKk4/Y7Am4rjj5ks2e4DrHShMAwADRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AQAMEE0AAANEEwDAANEEADBANAEADBBNAAADRBMAwADRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AQAMEE0AAANEEwDAANEEADBANAEADBBNAAADRBMAwADRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AQAMEE0AAANEEwDAANEEADBANAEADBBNAAADRBMAwADRBAAwQDQBAAwQTQAAA0QTAMAA0QQAMEA0AQAMEE0AAANEEwDAANEEADBANAEADBBNAAADhqKpqu5SVe+rqiuq6vyq+oOqOnjRwwEALItD1rtDVd00yXuTnJXkEUlun+SFmYLrfy50OgCAJbFuNCV5YpIbJnlkd1+a5D1VdXiSk6vq+fNtAAD7tZHTcw9O8ndr4ugNmULqFxcyFQDAkhmJpjsn+fzqG7r7X5JcMW8DANjvjUTTTZPs3MXtF8/bAAD2eyPXNO2xqjoxyYnzp5dX1dmLOA77naOSXLjZQyybet5mTwDXeX627MqzarMnWFbH7G7DSDRdnOSIXdx+03nbj+jubUm2DY0Gs6ra3t3HbfYcwP7FzxY2ysjpuc9nzbVLVXXrJIdlzbVOAAD7q5FoOiPJA6vqxqtue0ySK5N8cCFTAQAsmZFoOiXJVUneUlX3n69XOjnJn3qNJjaYU7rAIvjZwoao7l7/TlV3SfKyJPfM9Jt0f5Hk5O7+/kKnAwBYEkPRBABwoBt6w15YlKq6Q1X9WVV9pqq+X1Uf2OyZgP2DN5tnoy3kdZpgD9w1yUOSfDzJoZs8C7Cf8GbzLILTc2yqqjqou6+eP35zkqO6+z6bOxVwXVdVz0jytCTHrPzSUlU9LdMvMt3SLzJxbTg9x6ZaCSaADebN5tlwogmA/ZE3m2fDiSYA9kfebJ4NJ5oAAAaIJgD2R3v8ZvOwHtEEwP7Im82z4UQTAPsjbzbPhvPilmyqqjos04tbJsmtkhxeVb86f356d1+xOZMB13GnJPmdTG82/7wkt4s3m2cveXFLNlVVbU3yld1svm13n7PvpgH2J95sno0mmgAABrimCQBggGgCABggmgAABogmAIABogkAYIBoAgAYIJoAAAaIJgCAAaIJAGDA/wMVGdfHu8yOMAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "print_proportion(train_df, treatment_column)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "# Random Undersampling (finding the majority class and undersampling it)\n", "# https://www.kaggle.com/code/hughhuyton/criteo-uplift-modelling/notebook\n", "def random_under(df, feature):\n", " \n", " target = df[feature].value_counts()\n", " \n", " if target.values[0]" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "print_proportion(train_df, treatment_column)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# method to transfor data for LGWUM method, will be explained later\n", "def target_class_lgwum(df, treatment, target, column_name):\n", " \n", " #CN:\n", " df[column_name] = 0 \n", " #CR:\n", " df.loc[(df[treatment] == 0) & (df[target] != 0), column_name] = 1 \n", " #TN:\n", " df.loc[(df[treatment] != 0) & (df[target] == 0), column_name] = 2 \n", " #TR:\n", " df.loc[(df[treatment] != 0) & (df[target] != 0), column_name] = 3 \n", " return df\n", "\n", "response_column_lgwum = \"lqwum_response\"\n", "train_df = target_class_lgwum(train_df, treatment_column, response_column, response_column_lgwum)\n", "test_df = target_class_lgwum(test_df, treatment_column, response_column, response_column_lgwum)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Start H2O" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Checking whether there is an H2O instance running at http://localhost:54321 ..... not found.\n", "Attempting to start a local H2O server...\n", " Java Version: openjdk version \"1.8.0_342\"; OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~22.04-b07); OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode)\n", " Starting server from /home/kurkami/git/h2o/h2o-3/build/h2o.jar\n", " Ice root: /tmp/tmp8edjah_q\n", " JVM stdout: /tmp/tmp8edjah_q/h2o_kurkami_started_from_python.out\n", " JVM stderr: /tmp/tmp8edjah_q/h2o_kurkami_started_from_python.err\n", " Server is running at http://127.0.0.1:54321\n", "Connecting to H2O server at http://127.0.0.1:54321 ... successful.\n", "Warning: Version mismatch. H2O is version 3.38.0.99999, but the h2o-python package is version 3.37.0.99999. This is a developer build, please contact your developer.\n" ] }, { "data": { "text/html": [ "\n", " \n", "
\n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
H2O_cluster_uptime:01 secs
H2O_cluster_timezone:America/New_York
H2O_data_parsing_timezone:UTC
H2O_cluster_version:3.38.0.99999
H2O_cluster_version_age:1 hour and 17 minutes
H2O_cluster_name:H2O_from_python_kurkami_q2bo8y
H2O_cluster_total_nodes:1
H2O_cluster_free_memory:8.88 Gb
H2O_cluster_total_cores:12
H2O_cluster_allowed_cores:12
H2O_cluster_status:locked, healthy
H2O_connection_url:http://127.0.0.1:54321
H2O_connection_proxy:{\"http\": null, \"https\": null}
H2O_internal_security:False
Python_version:3.10.4 final
\n", "
\n" ], "text/plain": [ "-------------------------- ------------------------------\n", "H2O_cluster_uptime: 01 secs\n", "H2O_cluster_timezone: America/New_York\n", "H2O_data_parsing_timezone: UTC\n", "H2O_cluster_version: 3.38.0.99999\n", "H2O_cluster_version_age: 1 hour and 17 minutes\n", "H2O_cluster_name: H2O_from_python_kurkami_q2bo8y\n", "H2O_cluster_total_nodes: 1\n", "H2O_cluster_free_memory: 8.88 Gb\n", "H2O_cluster_total_cores: 12\n", "H2O_cluster_allowed_cores: 12\n", "H2O_cluster_status: locked, healthy\n", "H2O_connection_url: http://127.0.0.1:54321\n", "H2O_connection_proxy: {\"http\": null, \"https\": null}\n", "H2O_internal_security: False\n", "Python_version: 3.10.4 final\n", "-------------------------- ------------------------------" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "h2o.init(strict_version_check=False) # max_mem_size=10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import data to H2O" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%\n", "Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%\n" ] } ], "source": [ "h2o_train_df = h2o.H2OFrame(train_df)\n", "del(train_df)\n", "\n", "h2o_train_df[treatment_column] = h2o_train_df[treatment_column].asfactor()\n", "h2o_train_df[response_column] = h2o_train_df[response_column].asfactor()\n", "h2o_train_df[response_column_lgwum] = h2o_train_df[response_column_lgwum].asfactor()\n", "h2o_train_df = h2o.assign(h2o_train_df, \"train_df\")\n", "\n", "h2o_test_df = h2o.H2OFrame(test_df)\n", "del(test_df)\n", "h2o_test_df[treatment_column] = h2o_test_df[treatment_column].asfactor()\n", "h2o_test_df[response_column] = h2o_test_df[response_column].asfactor()\n", "h2o_test_df[response_column_lgwum] = h2o_test_df[response_column_lgwum].asfactor()\n", "h2o_test_df = h2o.assign(h2o_test_df, \"test_df\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train H2O UpliftDRF model" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "upliftdrf Model Build progress: |████████████████████████████████████████████████| (done) 100%\n" ] }, { "data": { "text/html": [ "
H2OUpliftRandomForestEstimator : Uplift Distributed Random Forest\n",
       "Model Key: UpliftDRF_model_python_1665008325837_1\n",
       "
\n", "
\n", " \n", "
\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Model Summary:
number_of_treesnumber_of_internal_treesmodel_size_in_bytesmin_depthmax_depthmean_depthmin_leavesmax_leavesmean_leaves
20.040.0175049.015.015.015.0293.0394.0344.5
\n", "
\n", "
\n",
       "\n",
       "[tips]\n",
       "Use `model.show()` for more details.\n",
       "Use `model.explain()` to inspect the model.\n",
       "--\n",
       "Use `h2o.display.toggle_user_tips()` to switch on/off this section.
" ], "text/plain": [ "H2OUpliftRandomForestEstimator : Uplift Distributed Random Forest\n", "Model Key: UpliftDRF_model_python_1665008325837_1\n", "\n", "\n", "Model Summary: \n", " number_of_trees number_of_internal_trees model_size_in_bytes min_depth max_depth mean_depth min_leaves max_leaves mean_leaves\n", "-- ----------------- -------------------------- --------------------- ----------- ----------- ------------ ------------ ------------ -------------\n", " 20 40 175049 15 15 15 293 394 344.5\n", "\n", "[tips]\n", "Use `model.show()` for more details.\n", "Use `model.explain()` to inspect the model.\n", "--\n", "Use `h2o.display.toggle_user_tips()` to switch on/off this section." ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ntree = 20\n", "max_depth = 15\n", "metric=\"Euclidean\"\n", "\n", "h2o_uplift_model = H2OUpliftRandomForestEstimator(\n", " ntrees=ntree,\n", " max_depth=max_depth,\n", " min_rows=30,\n", " nbins=1000,\n", " sample_rate=0.80,\n", " score_each_iteration=True,\n", " treatment_column=treatment_column,\n", " uplift_metric=metric,\n", " auuc_nbins=1000,\n", " auuc_type=\"gain\",\n", " seed=42)\n", "\n", "h2o_uplift_model.train(y=response_column, x=feature_cols, training_frame=h2o_train_df)\n", "h2o_uplift_model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Predict and plot Uplift Score" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# Plot uplift score\n", "# source https://www.kaggle.com/code/hughhuyton/criteo-uplift-modelling/notebook\n", "def plot_uplift_score(uplift_score):\n", " plt.figure(figsize = (10,6))\n", " plt.xlim(-.05, .1)\n", " plt.hist(uplift_score, bins=1000, color=['#2077B4'])\n", " plt.xlabel('Uplift score')\n", " plt.ylabel('Number of observations in validation set')" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "upliftdrf prediction progress: |█████████████████████████████████████████████████| (done) 100%\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
uplift_predict p_y1_ct1 p_y1_ct0
0.0005053270.001459410.000954085
0.0005053270.001459410.000954085
0.0005053270.001459410.000954085
0.0328249 0.071748 0.0389231
0.0005053270.001459410.000954085
0.0007439870.001809470.00106548
0.0005053270.001459410.000954085
0.00146267 0.0345459 0.0330832
0.0005053270.001459410.000954085
0.0005053270.001459410.000954085
[2795919 rows x 3 columns]
" ], "text/plain": [ " uplift_predict p_y1_ct1 p_y1_ct0\n", "---------------- ---------- -----------\n", " 0.000505327 0.00145941 0.000954085\n", " 0.000505327 0.00145941 0.000954085\n", " 0.000505327 0.00145941 0.000954085\n", " 0.0328249 0.071748 0.0389231\n", " 0.000505327 0.00145941 0.000954085\n", " 0.000743987 0.00180947 0.00106548\n", " 0.000505327 0.00145941 0.000954085\n", " 0.00146267 0.0345459 0.0330832\n", " 0.000505327 0.00145941 0.000954085\n", " 0.000505327 0.00145941 0.000954085\n", "[2795919 rows x 3 columns]\n" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "h2o_uplift_pred = h2o_uplift_model.predict(h2o_test_df)\n", "h2o_uplift_pred" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmwAAAF+CAYAAAA/Y0YoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkBElEQVR4nO3deZglZX328e/NICoKAjImyoAzElyIC5oRjfIi4oaiYBAXXEEMJq87SjLGDTGLu1lEcVQEzauISMwoKMYNDG4MiwgoMgLqIBEEZI3I8nv/ONXDoe2lmO46Xd3z/VzXubq2U+d3Hpiee56qp55UFZIkSeqvjea6AEmSJE3NwCZJktRzBjZJkqSeM7BJkiT1nIFNkiSp5wxskiRJPTcvA1uSI5NcluSclsc/J8l5Sc5N8umu65MkSZpNmY/PYUuyK3Ad8MmqevA0x+4AHAvsXlVXJblXVV02ijolSZJmw7zsYauqU4Arh7cl2T7JV5KcnuTbSR7Y7PpL4PCquqp5r2FNkiTNK/MysE1iJfCqqvoz4A3Ah5rt9wfun+TUJN9LssecVShJkrQeNp7rAmZDkrsDjwE+l2Rs852bnxsDOwC7AUuAU5I8pKp+O+IyJUmS1suCCGwMegp/W1U7TbBvLfD9qroJuCjJTxkEuNNGWJ8kSdJ6WxCXRKvqGgZh7NkAGXhYs/sLDHrXSLI1g0ukF85BmZIkSetlXga2JJ8Bvgs8IMnaJAcCLwAOTPJD4Fxg7+bwk4ArkpwHfBM4pKqumIu6JUmS1se8fKyHJEnShmRe9rBJkiRtSAxskiRJPTfvRoluvfXWtXTp0rkuQ5IkaVqnn376b6pq8UzPM+8C29KlS1m9evVclyFJkjStJD+fjfN4SVSSJKnnDGySJEk9Z2CTJEnqOQObJElSzxnYJEmSes7AJkmS1HMGNkmSpJ4zsEmSJPWcgU2SJKnnOgtsSY5MclmSc6Y57pFJbk6yb1e1SJIkzWdd9rAdBewx1QFJFgHvAr7aYR2SJEnzWmeBrapOAa6c5rBXAZ8HLuuqDkmSpPluzu5hS7IN8BfAh1sce1CS1UlWX3755d0XJ0mS1CMbz+Fn/zPwt1V1a5IpD6yqlcBKgOXLl1f3pUlzb9mKE9YtX/TOPeewEknSXJvLwLYcOKYJa1sDT0tyc1V9YQ5rkiRJ6p05C2xVtWxsOclRwJcMa5IkSX+os8CW5DPAbsDWSdYCbwPuBFBVR3T1uZIkSQtNZ4Gtqva7A8fu31UdkiRJ850zHUiSJPWcgU2SJKnnDGySJEk9Z2CTJEnqOQObJElSzxnYJEmSes7AJkmS1HMGNkmSpJ4zsEmSJPWcgU2SJKnnDGySJEk9Z2CTJEnqOQObJElSzxnYJEmSes7AJkmS1HMGNkmSpJ4zsEmSJPWcgU2SJKnnDGySJEk9Z2CTJEnqOQObJElSzxnYJEmSes7AJkmS1HMGNkmSpJ4zsEmSJPWcgU2SJKnnDGySJEk9Z2CTJEnqOQObJElSzxnYJEmSes7AJkmS1HMGNkmSpJ4zsEmSJPWcgU2SJKnnDGySJEk911lgS3JkksuSnDPJ/hckOTvJj5J8J8nDuqpFkiRpPuuyh+0oYI8p9l8EPK6qHgK8A1jZYS2SJEnz1sZdnbiqTkmydIr93xla/R6wpKtaJEmS5rO+3MN2IPDluS5CkiSpjzrrYWsryeMZBLZdpjjmIOAggO22225ElUmSJPXDnPawJXko8DFg76q6YrLjqmplVS2vquWLFy8eXYGSJEk9MGeBLcl2wPHAi6rqp3NVhyRJUt9Ne0k0yZ2r6sbptk3wvs8AuwFbJ1kLvA24E0BVHQG8Fbgn8KEkADdX1fL1+RKSJEkLWZt72L4LPKLFttupqv2m2f8y4GUtPl+SJGmDNmlgS/LHwDbAXZM8HEiza3Ng0xHUJkmSJKbuYXsKsD+D56O9f2j7NcDfdViTJEmShkwa2KrqaODoJM+qqs+PsCZJkiQNaTNK9NQkH0/yZYAkOyY5sOO6JEmS1GgT2D4BnATcp1n/KfDargqSJEnS7bUJbFtX1bHArQBVdTNwS6dVSZIkaZ02ge36JPcECiDJo4GrO61KkiRJ67R5DtvBwCpg+ySnAouBfTutSpIkSetMG9iq6owkjwMewOBZbOdX1U2dVyZJkiSgxSXRJM8G7lpV5wLPBD6bZMpZDiRJkjR72tzD9paqujbJLsATgI8DH+62LEmSJI1pE9jGRoTuCXy0qk4ANumuJEmSJA1rE9guSfIR4LnAiUnu3PJ9kiRJmgVtgtdzGDw49ylV9VtgK+CQLouSJEnSbdqMEr0BOH5o/VLg0i6LkiRJ0m28tClJktRzBjZJkqSeM7BJkiT1XJsH5+6T5IIkVye5Jsm1Sa4ZRXGSJElqN5fou4FnVNWPuy5GkiRJf6jNJdFfG9YkSZLmTpsettVJPgt8AbhxbGNVHT/pOyRJkjRr2gS2zYEbgCcPbSuGns0mSZKk7rR5cO4BoyhEkiRJE2szSnRJkv9Iclnz+nySJaMoTpIkSe0GHXwCWAXcp3l9sdkmSZKkEWgT2BZX1Seq6ubmdRSwuOO6JEmS1GgT2K5I8sIki5rXC4Erui5MkiRJA20C20uB5wD/A1wK7As4EEGSJGlE2owS/Tmw1whqkSRJ0gQmDWxJ/qaq3p3k3xg8d+12qurVnVYmSZIkYOoetrHpqFaPohBJkiRNbNLAVlVfbBZvqKrPDe9L8uxOq5IkSdI6bQYdvLHlNkmSJHVgqnvYngo8Ddgmyb8O7docuLnrwiRJkjQw1T1sv2Jw/9pewOlD268FXtdlUZIkSbrNVPew/RD4YZJPV9VNd/TESY4Eng5cVlUPnmB/gH9h0It3A7B/VZ1xRz9HkiRpoWtzD9vSJMclOS/JhWOvFu87Cthjiv1PBXZoXgcBH25xTkmSpA1O28nfP8zgvrXHA58E/n26N1XVKcCVUxyyN/DJGvgesEWSe7eoR5IkaYPSJrDdtaq+DqSqfl5VhwJ7zsJnbwP8cmh9bbPtDyQ5KMnqJKsvv/zyWfhoSZKk+aNNYLsxyUbABUlemeQvgLt3XNftVNXKqlpeVcsXL148yo+WJEmac20C22uATYFXA38GvAh4ySx89iXAtkPrS5ptkiRJGtJm8vfTmsXrgANm8bNXAa9McgzwKODqqrp0Fs8vSZK0IEz14NwvMsGk72Oqaq+pTpzkM8BuwNZJ1gJvA+7UvPcI4EQGj/RYw+CxHrMZBiVJkhaMqXrY3tv83Af4Y24bGbof8OvpTlxV+02zv4BXtKhRkiRpgzbVg3NPBkjyvqpaPrTri0lWd16ZJEmSgHaDDu6W5H5jK0mWAXfrriRJkiQNm3bQAYN5Q7/VzG4Q4L7AyzutSpIkSeu0GSX6lSQ7AA9sNv2kqm7stixJkiSNmWqU6O5V9Y0k+4zbtX0Squr4jmuTJEkSU/ewPQ74BvCMCfYVYGCTJEkagalGib6t+enz0SRJkubQVJdED57qjVX1/tkvR5IkSeNNdUl0s5FVIUmSpElNdUn07aMsRJIkSROb9rEeSe4CHAj8KXCXse1V9dIO65IkSVKjzUwHn2Iwl+hTgJOBJcC1XRYlSZKk27QJbH9SVW8Brq+qo4E9gUd1W5YkSZLGtAlsNzU/f5vkwcA9gHt1V5IkSZKGtZlLdGWSLYG3AKuAuzfLkiRJGoE2ge0TVXULg/vX7tdxPZIkSRqnzSXRi5KsTPKEJOm8IkmSJN1Om8D2QOBrwCuAi5N8MMku3ZYlSZKkMdMGtqq6oaqOrap9gJ2AzRlcHpUkSdIItOlhI8njknwIOJ3Bw3Of02lVkiRJWqfNTAcXA2cCxwKHVNX1XRclSZKk27QZJfrQqrqm80okSZI0oTb3sBnWJEmS5lCre9gkSZI0dwxskiRJPddm0MGdgWcBS4ePr6rDuitLkiRJY9oMOvhP4GoGj/S4sdtyJEmSNF6bwLakqvbovBJJkiRNqM09bN9J8pDOK5EkSdKE2vSw7QLsn+QiBpdEA1RVPbTTyiRJkgS0C2xP7bwKSZIkTWrSwJZk8+ahudeOsB5JkiSNM1UP26eBpzMYHVoMLoWOKeB+HdYlSZKkxqSBraqe3vxcNrpyJEmSNJ4zHUiSJPVcp4EtyR5Jzk+yJsmKCfZvl+SbSc5McnaSp3VZjyRJ0nzUWWBLsgg4nMEo0x2B/ZLsOO6wNwPHVtXDgecBH+qqHkmSpPlq2sCWZPtmPlGS7Jbk1Um2aHHunYE1VXVhVf0eOAbYe9wxBWzeLN8D+FXryiVJkjYQbXrYPg/ckuRPgJXAtgxGkE5nG+CXQ+trm23DDgVemGQtcCLwqhbnlSRJ2qC0CWy3VtXNwF8A/1ZVhwD3nqXP3w84qqqWAE8DPpXkD2pKclCS1UlWX3755bP00ZIkSfNDm8B2U5L9gJcAX2q23anF+y5h0Bs3ZkmzbdiBwLEAVfVd4C7A1uNPVFUrq2p5VS1fvHhxi4+WJElaONoEtgOAPwf+oaouSrIM+FSL950G7JBkWZJNGAwqWDXumF8ATwBI8iAGgc0uNEmSpCHTziVaVecBrx5avwh4V4v33ZzklcBJwCLgyKo6N8lhwOqqWgW8HvhoktcxGICwf1XV+n0VSZKkhWnawJbksQwGB9y3OT5AVdW0U1NV1YkMBhMMb3vr0PJ5wGPvWMmSJEkblmkDG/Bx4HUM5hS9pdtyJEmSNF6bwHZ1VX2580okSZI0oTaB7ZtJ3gMcD9w4trGqzuisKkmSJK3TJrA9qvm5fGhbAbvPfjmSJEkar80o0cePohBJkiRNrM1covdI8v6xmQaSvC/JPUZRnCRJkto9OPdI4FrgOc3rGuATXRYlSZKk27S5h237qnrW0Prbk5zVUT2SJEkap00P2/8m2WVspXmQ7v92V5IkSZKGtelh+2vg6Oa+tQBXAvt3WZQkSZJu02aU6FnAw5Js3qxf03VRkiRJus2kgS3JC6vq35McPG47AFX1/o5rkyRJElP3sN2t+bnZBPuqg1okSZI0gUkDW1V9pFn8WlWdOryvGXggSZKkEWgzSvTfWm6TJElSB6a6h+3PgccAi8fdx7Y5sKjrwiRJkjQw1T1smwB3b44Zvo/tGmDfLouSJEnSbaa6h+1k4OQkR1XVz0dYkyRJkoa0eXDuDUneA/wpcJexjVW1e2dVSZIkaZ02gw7+H/ATYBnwduBi4LQOa5IkSdKQNoHtnlX1ceCmqjq5ql4K2LsmSZI0Im0uid7U/Lw0yZ7Ar4CtuitJkiRJw9oEtr9vJn5/PYPnr20OvK7TqiRJkrROm8D2/aq6GrgaeHzH9UiSJGmcNvewnZrkq0kOTLJl5xVJkiTpdqYNbFV1f+DNDB7rcXqSLyV5YeeVSZIkCWjXw0ZV/aCqDgZ2Bq4Eju60KkmSJK0zbWBLsnmSlyT5MvAd4FIGwU2SJEkj0GbQwQ+BLwCHVdV3uy1HkiRJ400Z2JIsAo6vqtePqB5JkiSNM+Ul0aq6BXjMiGqRJEnSBNpcEj0rySrgc8D1Yxur6vjOqpIkSdI6bQLbXYAruP38oQUY2CRJkkZg2sBWVQeMohBJkiRNrM1jPe6f5OtJzmnWH5rkzd2XJkmSJGj34NyPAm8EbgKoqrOB53VZlCRJkm7TJrBtWlU/GLft5jYnT7JHkvOTrEmyYpJjnpPkvCTnJvl0m/NKkiRtSNoMOvhNku0ZDDQgyb4MZjuYUvMMt8OBJwFrgdOSrKqq84aO2YFB791jq+qqJPdaj+8gSZK0oLUJbK8AVgIPTHIJcBHQZvL3nYE1VXUhQJJjgL2B84aO+Uvg8Kq6CqCqLrsDtUuSJG0Q2owSvRB4YpK7ARtV1bUtz70N8Muh9bXAo8Ydc3+AJKcCi4BDq+or40+U5CDgIIDtttuu5cdLkiQtDG1Gib4myebADcAHkpyR5Mmz9PkbAzsAuwH7AR9NssX4g6pqZVUtr6rlixcvnqWPliRJmh/aDDp4aVVdAzwZuCfwIuCdLd53CbDt0PqSZtuwtcCqqrqpqi4CfsogwEmSJKnRJrCl+fk04JNVde7QtqmcBuyQZFmSTRg8CmTVuGO+wKB3jSRbM7hEemGLc0uSJG0w2gS205N8lUFgOynJZsCt072pqm4GXgmcBPwYOLaqzk1yWJK9msNOAq5Ich7wTeCQqrpifb6IJEnSQtVmlOiBwE7AhVV1Q5J7Aq2mq6qqE4ETx21769ByAQc3L0mSJE2gzSjRW5MsBV6YpID/rqr/6LwySZIkAe1GiX4I+CvgR8A5wMuTHN51YZIkSRpoc0l0d+BBzeVLkhzN7R9+K0mSpA61GXSwBhh+Wu22wAXdlCNJkqTxJu1hS/JFBvOHbgb8OMnYBPA7A+Mng5ckSVJHprok+t6RVSFJkqRJTRrYqurkseUkfwQ8sln9gZO0S5IkjU6bUaLPYXAJ9NnAc4DvJ9m368IkSZI00GaU6JuAR471qiVZDHwNOK7LwiRJkjTQZpToRuMugV7R8n2SJEmaBW162L6S5CTgM836cxk33ZQkSZK602ZqqkOS7APs0mxa6dRUkiRJo9Omh42qOh44vuNaJEmSNAHvRZMkSeo5A5skSVLPTRrYkny9+fmu0ZUjSZKk8aa6h+3eSR4D7JXkGCDDO6vqjE4rkyRJEjB1YHsr8BZgCfD+cfsK2L2roiRJknSbqeYSPQ44LslbquodI6xJkiRJQ9o8h+0dSfYCdm02fauqvtRtWZIkSRrTZvL3fwJeA5zXvF6T5B+7LkySJEkDbR6cuyewU1XdCpDkaOBM4O+6LEySJEkDbZ/DtsXQ8j06qEOSJEmTaNPD9k/AmUm+yeDRHrsCKzqtSpIkSeu0GXTwmSTfAh7ZbPrbqvqfTquSJEnSOm0nf78UWNVxLZIkSZqAc4lKkiT1nIFNmgeWrTiBZStOmOsyJElzZMrAlmRRkp+MqhhJkiT9oSkDW1XdApyfZLsR1SNJkqRx2gw62BI4N8kPgOvHNlbVXp1VJUmSpHXaBLa3dF6FJEmSJtXmOWwnJ7kvsENVfS3JpsCi7kuTJEkStJv8/S+B44CPNJu2Ab7QYU2SJEka0uaxHq8AHgtcA1BVFwD36rIoSZIk3aZNYLuxqn4/tpJkY6DanDzJHknOT7ImyaTzjyZ5VpJKsrzNeSVJkjYkbQLbyUn+DrhrkicBnwO+ON2bkiwCDgeeCuwI7JdkxwmO2wx4DfD9O1K4JEnShqJNYFsBXA78CHg5cCLw5hbv2xlYU1UXNj10xwB7T3DcO4B3Ab9rVbEkSdIGps0o0VuTHM2gB6yA86uqzSXRbYBfDq2vBR41fECSRwDbVtUJSQ5pX7YkSdKGY9rAlmRP4AjgZ0CAZUleXlVfnskHJ9kIeD+wf4tjDwIOAthuOyddkCRJG5Y2l0TfBzy+qnarqscBjwc+0OJ9lwDbDq0vabaN2Qx4MPCtJBcDjwZWTTTwoKpWVtXyqlq+ePHiFh8tSZK0cLQJbNdW1Zqh9QuBa1u87zRghyTLkmwCPA9YNbazqq6uqq2ramlVLQW+B+xVVavbly9JkrTwTXpJNMk+zeLqJCcCxzK4h+3ZDMLYlKrq5iSvBE5iMDPCkVV1bpLDgNVVtWrqM0iSJAmmvoftGUPLvwYe1yxfDty1zcmr6kQGo0qHt711kmN3a3NOSZKkDc2kga2qDhhlIZIkSZpYm1Giy4BXAUuHj6+qvborS5IkSWOmDWwMJnr/OIPZDW7ttBpJkiT9gTaB7XdV9a+dVyJJkqQJtQls/5LkbcBXgRvHNlbVGZ1VJUmSpHXaBLaHAC8Cdue2S6LVrEuSJKljbQLbs4H7NRO4S5IkacTazHRwDrBFx3VIkiRpEm162LYAfpLkNG5/D5uP9ZAkSRqBNoHtbZ1XIUmSpElNG9iq6uRRFCJJkqSJtZnp4FoGo0IBNgHuBFxfVZt3WZgkSZIG2vSwbTa2nCTA3sCjuyxKkiRJt2kzSnSdGvgC8JRuypEkSdJ4bS6J7jO0uhGwHPhdZxVJkiTpdtqMEn3G0PLNwMUMLotKkiRpBNrcw3bAKAqRJEnSxCYNbEneOsX7qqre0UE9kiRJGmeqHrbrJ9h2N+BA4J6AgU2SJGkEJg1sVfW+seUkmwGvAQ4AjgHeN9n7JEmSNLumvIctyVbAwcALgKOBR1TVVaMoTJIkSQNT3cP2HmAfYCXwkKq6bmRVSZIkaZ2pHpz7euA+wJuBXyW5pnldm+Sa0ZQnSZKkqe5hu0OzIEiSJKkbhjJJkqSeM7BJkiT1nIFNkiSp5wxskiRJPWdgkyRJ6jkDmyRJUs8Z2CRJknrOwCZJktRzBjZJkqSeM7BJkiT1nIFNkiSp5wxsku6QZStOYNmKE+a6DEnaoHQa2JLskeT8JGuSrJhg/8FJzktydpKvJ7lvl/VIkiTNR50FtiSLgMOBpwI7Avsl2XHcYWcCy6vqocBxwLu7qkeSJGm+6rKHbWdgTVVdWFW/B44B9h4+oKq+WVU3NKvfA5Z0WI8kSdK81GVg2wb45dD62mbbZA4EvjzRjiQHJVmdZPXll18+iyVKkiT1Xy8GHSR5IbAceM9E+6tqZVUtr6rlixcvHm1xkiRJc2zjDs99CbDt0PqSZtvtJHki8CbgcVV1Y4f1SJIkzUtd9rCdBuyQZFmSTYDnAauGD0jycOAjwF5VdVmHtUiSJM1bnQW2qroZeCVwEvBj4NiqOjfJYUn2ag57D3B34HNJzkqyapLTSZIkbbC6vCRKVZ0InDhu21uHlp/Y5edLkiQtBL0YdCBJkqTJGdgkSZJ6zsAmSZLUcwY2SZKknjOwSZIk9ZyBTZIkqecMbJIkST1nYJMkSeo5A5s0jyxbcQLLVpww12VIkkbMwCZJktRzBjZJkqSeM7BJkiT1nIFNkiSp5wxskiRJPWdgkyRJ6jkDmyRJUs8Z2CRJknrOwCZJktRzBjZJkqSeM7BJkiT1nIFNkiSp5wxskiRJPWdgkyRJ6jkDmyRJUs8Z2CRJknrOwCZJktRzBjZJkqSeM7BJkiT13MZzXYCkO27ZihPWLV/0zj3nsBJJ0ijYwyZJktRzBjZJkqSeM7BJC8SyFSfc7lKpJGnhMLBJkiT1nIMOpHnOXjVJWvjsYZMkSeq5TnvYkuwB/AuwCPhYVb1z3P47A58E/gy4AnhuVV3cZU3ShmisF85HgNxxPkJFUh901sOWZBFwOPBUYEdgvyQ7jjvsQOCqqvoT4APAu7qqR1L3Rj3woU8DLfpUi6SFp8setp2BNVV1IUCSY4C9gfOGjtkbOLRZPg74YJJUVXVYl6RZtL4hZTZ7rmbag3hHa5lpMJvo8+7od1jf7zz+fX367zAq86VOaVi6ykZJ9gX2qKqXNesvAh5VVa8cOuac5pi1zfrPmmN+M9l5ly9fXqtXr+6kZmlUpvoLf6K/SOfKqGuZ6i/QPrSHNN5UgfuOBMO2wdmwOf8kOb2qls/4PPMhsCU5CDioWX0AcH4nRa+/rYFJQ6Zux7Zqx3Zqz7Zqx3Zqx3Zqz7Zq5wFVtdlMT9LlJdFLgG2H1pc02yY6Zm2SjYF7MBh8cDtVtRJY2VGdM5Zk9Wyk5w2BbdWO7dSebdWO7dSO7dSebdVOklm5LNjlYz1OA3ZIsizJJsDzgFXjjlkFvKRZ3hf4hvevSZIk3V5nPWxVdXOSVwInMXisx5FVdW6Sw4DVVbUK+DjwqSRrgCsZhDpJkiQN6fQ5bFV1InDiuG1vHVr+HfDsLmsYkd5eru0h26od26k926od26kd26k926qdWWmnzgYdSJIkaXY4NZUkSVLPGdhaSrJVkv9KckHzc8tJjntJc8wFSV4ywf5VzeNMFqyZtFWSTZOckOQnSc5N8s6J3jufJdkjyflJ1iRZMcH+Oyf5bLP/+0mWDu17Y7P9/CRPGWnhI7a+7ZTkSUlOT/Kj5ufuIy9+xGby/1Szf7sk1yV5w8iKngMz/LP30CTfbX4v/SjJXUZa/AjN4M/enZIc3bTPj5O8ceTFj1iLtto1yRlJbm4edza8b8q88AeqyleLF/BuYEWzvAJ41wTHbAVc2Pzcslnecmj/PsCngXPm+vv0ta2ATYHHN8dsAnwbeOpcf6dZbJtFwM+A+zXf74fAjuOO+b/AEc3y84DPNss7NsffGVjWnGfRXH+nHrbTw4H7NMsPBi6Z6+/T17Ya2n8c8DngDXP9ffrYTgzu9z4beFizfk//7E3YTs8HjmmWNwUuBpbO9Xea47ZaCjyUwbzp+w5tnzIvTPSyh629vYGjm+WjgWdOcMxTgP+qqiur6irgv4A9AJLcHTgY+PvuS51z691WVXVDVX0ToKp+D5zB4Bl+C8W6Kdua7zc2Zduw4fY7DnhCkjTbj6mqG6vqImBNc76FaL3bqarOrKpfNdvPBe6a5M4jqXpuzOT/KZI8E7iIQVstZDNppycDZ1fVDwGq6oqqumVEdY/aTNqpgLtl8FzVuwK/B64ZTdlzYtq2qqqLq+ps4NZx7500L0zGwNbeH1XVpc3y/wB/NMEx2wC/HFpf22wDeAfwPuCGzirsj5m2FQBJtgCeAXy9gxrnyrTfe/iYqroZuJrBv+jbvHehmEk7DXsWcEZV3dhRnX2w3m3V/EPyb4G3j6DOuTaT/6fuD1SSk5rLW38zgnrnykza6TjgeuBS4BfAe6vqyq4LnkMz+Z18h9/b6WM95pskXwP+eIJdbxpeqapK0np4bZKdgO2r6nXj7x2Zr7pqq6Hzbwx8BvjXqrpw/arUhizJnwLvYtA7ookdCnygqq5rOtw0sY2BXYBHMvhH99czmB9yIf1jcjbsDNwC3IfBZb5vJ/mav8Nnh4FtSFU9cbJ9SX6d5N5VdWmSewOXTXDYJcBuQ+tLgG8Bfw4sT3Ixgza/V5JvVdVuzFMdttWYlcAFVfXPM6+2V2YyZVub9y4UM5raLskS4D+AF1fVz7ovd07NpK0eBeyb5N3AFsCtSX5XVR/svOrRm0k7rQVOqWae6yQnAo9gYfX+j5lJOz0f+EpV3QRcluRUYDmD+7MWopn8Tp7u78A/4CXR9oan0XoJ8J8THHMS8OQkW2YwMvLJwElV9eGquk9VLWXwr7Sfzuew1sJ6txVAkr9n8Avgtd2XOnIzmbJtFfC8ZoTWMmAH4AcjqnvU1rudmkvpJzAY+HLqqAqeQ+vdVlX1f6pqafO76Z+Bf1ygYQ1m9mfvJOAhGYxi3xh4HHDeiOoetZm00y+A3QGS3A14NPCTkVQ9N9q01WQm/TtwUnM9ymK+vBhcn/86cAHwNWCrZvty4GNDx72Uwc3ga4ADJjjPUhb+KNH1bisG/8oo4MfAWc3rZXP9nWa5fZ4G/JTB6KI3NdsOA/Zqlu/CYMTeGgaB7H5D731T877zWUCjZ2eznYA3M7iP5qyh173m+vv0sa3GneNQFvAo0Zm2E/BCBgMzzgHePdffpY/tBNy92X4ug0B7yFx/lx601SMZ9NBez6AX8tyh906ZF8a/nOlAkiSp57wkKkmS1HMGNkmSpJ4zsEmSJPWcgU2SJKnnDGySJEk9Z2CT1EtJliY5Z9y2Q5O8YZr37Z/kg83yXyV5cbP8wCRnJTkzyfZJnt9d9ZI0uwxskhasqjqiqj7ZrD4TOK6qHs7g6eQjCWzNg1YlaUb8RSJpXkryLeCHDJ46vzHw0qr6wbhjDgWuY/AQz9cCtyR5AnBX4EFJzgKOrqoPDL3n3sBngc2b8/51VX07yR7APwKLgN9U1ROSbAUcCdyPwRyTB1XV2c3nbt9s/0WSVwNHANs1H/Pa2jBmYZA0SwxskuazTatqpyS7MghOD57ooKo6MckRwHVV9d4kuzF4qv/TJzj8+QymlPuHJIuATZMsBj4K7FpVFzVBDeDtwJlV9cwkuwOfBHZq9u0I7FJV/5vk0wwmWf/vJNsxmILmQbPw/SVtIAxskvpqsmlYhrd/BqCqTkmyeTOP6EydBhyZ5E7AF6rqrCbgnVJVFzWfd2Vz7C7As5pt30hyzySbN/tWVdX/NstPBHZMMvYZmye5e1VdNwv1StoAGNgk9dUVwJbjtm0FXDS0Pj7UzXiuvSb87QrsCRyV5P3AVetxquuHljcCHl1Vv5tpfZI2TA46kNRLTe/Tpc2lRprLkHsA/z102HObfbsAV1fV1S1Pfy2w2UQ7ktwX+HVVfRT4GPAI4HvArkmWDdUC8G3gBc223Rjc23bNBKf9KvCqoc/YqWWdkgTYwyap314MHN70cgG8vap+NrT/d0nOBO4EvPQOnPdsBgMQfggcNTzoANgNOCTJTQwGLLy4qi5PchBwfJKNgMuAJwGHMrh8ejaDQQcvmeTzXt18j7MZ/N49BfirO1CvpA1cqmZ8BUGSRq4ZJfqGqlo917VIUte8JCpJktRz9rBJkiT1nD1skiRJPWdgkyRJ6jkDmyRJUs8Z2CRJknrOwCZJktRzBjZJkqSe+/+yJC76e+5jFgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plot_uplift_score(h2o_uplift_pred['uplift_predict'].as_data_frame().uplift_predict)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate the model" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "perf_h2o = h2o_uplift_model.model_performance(h2o_test_df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "### Area Under Uplift Curve (AUUC) calculation\n", "\n", "To calculate AUUC for big data, the predictions are binned to histograms. Due to this feature the results should be different compared to exact computation.\n", "\n", "To define AUUC, binned predictions are sorted from largest to smallest value. For every group the cumulative sum of observations statistic is calculated. The uplift is defined based on these statistics.\n", "\n", "\n", "#### Types of AUUC\n", "\n", "\n", "| AUUC type |                    Formula                    |\n", "|:----------:|:-------------------------------------------:|\n", "| **Qini** | $TY1 - CY1 * \\frac{T}{C}$ |\n", "| **Lift** | $\\frac{TY1}{T} - \\frac{CY1}{C}$ |\n", "| **Gain** | $(\\frac{TY1}{T} - \\frac{CY1}{C}) * (T + C)$ |\n", "\n", "\n", "Where:\n", "\n", "- **T** how many observations are in the treatment group (how many data rows in the bin have treatment_column label == 1)\n", "- **C** how many observations are in the control group (how many data rows in the bin have treatment_column label == 0)\n", "- **TY1** how many observations are in the treatment group and respond to the offer (how many data rows in the bin have treatment_column label == 1 and response_column label == 1)\n", "- **CY1** how many observations are in the control group and respond to the offer (how many data rows in the bin have treatment_column label == 0 and response_column label == 1)\n", "\n", "\n", "The resulting AUUC value is:\n", "\n", "- Not normalized.\n", "- The result could be a positive or negative number.\n", "- Higher number means better model.\n", "\n", "More information about normalization is in **Normalized AUUC** section.\n", "\n", "\n", "For some observation groups the results should be NaN. In this case, the results from NaN groups are linearly interpolated to calculate AUUC and plot uplift curve.\n" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", "
\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
AUUC table (number of bins: 168): All types of AUUC value
uplift_typeqiniliftgain
AUUC value21585.48821570.025358625328.4067094
AUUC normalized0.88053300.02535860.8782349
AUUC random value15585.37521520.006558018335.7289893
\n", "
\n" ], "text/plain": [ "AUUC table (number of bins: 168): All types of AUUC value\n", "uplift_type qini lift gain\n", "----------------- -------- ---------- --------\n", "AUUC value 21585.5 0.0253586 25328.4\n", "AUUC normalized 0.880533 0.0253586 0.878235\n", "AUUC random value 15585.4 0.00655803 18335.7" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "perf_h2o.auuc_table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Cumulative Uplift curve plot\n", "\n", "To plot the uplift curve, the ``plot_uplift``method can be used. There is specific parameter ``metric`` which can be ``\"qini\", \"gain\", or \"lift\"``. The most popular is the Qini uplift curve which is similar to the ROC curve. The Gain and Lift curves are also known from traditional binomial models. \n", "\n", "Depending on these curves, you can decide how many observations (for example customers) from the test dataset you send an offer to get optimal gain." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABIQklEQVR4nO3deZzN1f/A8dd77EWyJeFrVJYo+1YqilTaS5uKaZO0IUlaCJWK9n6JUlIiLaJCKiPZsu+yy9j3fRvz/v1xPqNrzHJn/dy59/18PD6Pe+/5bO8zl3nP53zO5xxRVYwxxphQE+V3AMYYY0xyLEEZY4wJSZagjDHGhCRLUMYYY0KSJShjjDEhyRKUMcaYkGQJykQsEekpIl/6HUcwRCRGRP4K+LxfRM713hcSkTEiskdERvoXZWgQkbEi0jartzU5zxKUyXEi0lpEZnm/ZDd5vyQu9Tuu1IjI5yLSJxP7x4rIQ0nKmopIXEaOp6qFVXW197EVUBoooaq3BxOrOE+KyCIROSAicSIyUkQuykg8oURVr1XVIVm9rcl5lqBMjhKRzsA7wKu4X6r/A/4PuMnHsHK7CsByVY1Pxz7vAk8BTwLFgcrAKOC69J5cRPKmdx9jgqKqttiSIwtQFNgP3J7KNp8DfQI+NwXiAj6vBZ4BFgAHgE9xiW4ssA/4DSiW3L4B+zf33vcEvgxYNxLYDOwB/gSqe+XtgGPAUS/+MV75OcB3wDZgDfBkKvWKBR5KUpa0bopLGKuB7cCbQJS3Lgb4K8m25wMve3Ed82J7JLlYk5y3EnAcaBBsvCmc/zFghVf3j4B+SY7xI9A5vT+rIP4d5QH6eT+j1V4cCuRNGnti3N72u7xzX5va92JL6Cx2BWVy0sVAQeCHTB7nNuAq3F/9N+CSU3egFK5V4MkMHncs7pf3WcAc4CsAVR3ovX9DXdPaDSISBYwB5gNlgWZARxG5OqOV8twC1APq4K4qH0htY1XtgbsaHeHF9nHSWJPZrRkuMf6dyVhvBhoC1YCvgTtFRABEpBjQAhieDT+rh4Hrgdq4n1WrNLZvCPwDlATeAD5NjNOENktQJieVALZr+pqikvO+qm5R1Q3AZGCGqs5V1cO45Fc7IwdV1cGquk9Vj+CurmqKSNEUNq8PlFLVXqp6VN39oEHAXRk5d4DXVXWnqv6Lawq9O5PHS04JYFMWHOc1L9ZDuO9Bgcu8da2Aaaq6kaz/Wd0BvKOq61V1J/BaGtuvU9VBqnocGAKUwV11mxBnbccmJ+0ASopI3kwmqS0B7w8l87lweg8oInmAV4DbcVdiCd6qkrgmv6QqAOeIyO6Asjy4X9TJiQfyJSnLh2uOC7Q+4P06XNNYVtuB+yWdWSdiVVUVkeG4hPon0BpI7CEZ9M9KRP4HLAk4bnLf5Tmc+nNKzeaA4x30Lp7S/W/E5Dy7gjI5aRpwBNc0lJIDwGkBn8/OxPlOOpaXhEqlsG1rXJNac9y9sujE3bzXpMP+rwfWqOqZAUsRVW2ZwvH/DThmooqc+su1fMD7/wEbUzheatKaouB3oJyI1Etlm2C+h6Tn+RpoJSIVcM1q33nlQf+sVPVfr2mycArJCdzVX9KfkwlDlqBMjlHVPcBLwIcicrOInCYi+UTkWhF5w9tsHtBSRIqLyNlAx0yccjlQUESuE5F8wAtAgRS2LYJLnjtwv5hfTbJ+C3BuwOe/gX0i8qz3HFIeEblQROqncPwRwP0i0sDr4l0Z6AQMT7LdMyJSTETK43rZjQiyrqnFehJVXYHrOfm119U9v4gUFJG7RKSbt9k84FbvOzofeDCtk6rqXFzHhU+A8aq621uV3p9VWr4BnhSRct69rm5p7WByJ0tQJkepan+gMy5ZbMP9df04roszwFDczfS1wK9k7Bd04rn2AB1wvzA34K4KUnru6Avc1cwGXBPT9CTrPwWqichuERnl3c+4HqiF6xmW+Is52XtWqjoe94v0M1yT4S+4+yEDk2z6IzAblyB+9s6bXifFmsI2TwIfAB8Cu4FVuA4aY7z1b+N6Am7x4vwqyHMPw12FDkssSO/PKgiDgPG4fydzgO8zeBwT4kTVJiw0JhSIiAKVVHWl37HkJiISjUt8+bKgA44JIXYFZYwxJiRZgjLGGBOSrInPGGNMSLIrKGOMMSHJEpQxxpiQZAnKhDVvmotdIlIgSVmKU1+ISLSIaNJRupNOYyEiZUTkU2/KkH0iskxEXhaR09MR31ki8rWIbPTmc5oiIg2TnGO0t169HmtJYzrqTV2SuOQJWH+HiCz14lsiIjcHrIsRkeNJ9m0aRMwvebE0DygrLiIjRGSHiGwXka9E5IyA9bVEZLJXxzgReTFgXSMRmSAiO0Vkm7hpP7JipAuTy1mCMmHL+2V+GW7Egxuz+NjFcSNjFAIuVtUiuAFszwTOS8ehCgMzgbq4aS+GAD+LSOIoCgnAONwAuSl5I3D0Be+5I0SkLG64oc7AGbhR4IeJyFkB+05Lsm9sGvU+DzccVNKx/PoAxXCjY5yHG+uuZ8D6YbghkIoDTYAOIpL4nRTDPQ8WjRsWaR/ueTET4SxBmXDWBvfA7edAVs+a2hn3i/ReVV0L4A1e+pSqLgj2IKq6WlXfUtVNqnrcGzk9P1DFW79FVf8Pl8TSqxywW1XHqvMz7mHl9CTQpD4EnsU9xBuoIjBKVfd6D0j/AFQPWB8NfOXVcRVuCozqAF58I719D+IeIG6ciRhNmLAEZcJZG9wICF8BV4tIVo5g3Rz4XlUTUtpARBZ4ozkkt/xfCvvUwiWo9Dys28FrHpstIoFXWrOApSJyoze80M244ZwCE2htr0luuYi8mLRZM0lstwNHVPWXZFZ/CFzvDdNUDHfFNzZg/TtAG3FDW1XBTb3yWwqnuhxYnHqVTSSw0cxNWBI3hXwF4BtV3S4iq3ADwr6dRadIc8oKVa2RngN692yGAi97VyHBeA94Gjd8UgtghIhsVtUpqnpcRL7ANa8VxF313K6qB7x9/wQuxA3xVB03rFQ8yUxfISJFcOMTXpVCHHNwiXWH9/l33Hh/iX7CDSfVBTeSeS9VPeWqUERq4MZrtBmWjV1BmbDVFvhVVbd7n4fxXzNfWlNfxAeUpbRNVk1ZAYCIFMKNgzddVdOa3+gEVZ2jqjtUNd67svkKuNU7ZnPcBH1NccmjCfCJd5WW2Ly4RlUTVHUh0IuUJ//rCQxNbM5Mxje4wXmL4O53rcKbbsO7XzfOO35B3EjkV4tIhyQ/g/NxV11PqWpK05aYCGIJyoQd75f9HUATEdksIptxI4fXFJGapD31xSZcIkptm9+AW8TNFptSHIuT9JALXAYEbFcAN1huHG7K9sxQ/psipBbwp6rO8pLQTGAGrnkyrX2TaoYbQTzx51ke+EZEng0418eqekBV9wMDgMTpNM4FjqvqF14ijcON4n5iug1xU3T8BvRW1aHprrUJTxoC887bYktWLrhJ83bi5gk6O2D5E+gPXA1sBRrgfiFXBpYC7QOO8TVulOwSuCunu3Gjfpf21hfHjbg+FKjglZUF3gJqpCPWfLgrp1FA3hS2KQicjksgVYCCAeta4XoCRuGa+PYBTb11TXAjh9fyPtfGXfm18D5fG1CfqsAioEcKMZRI8rNcj+vNV9hbPxF4H9ersRCueW+qt+4M72fX2ovzbFwPyFcDfm6rgC5+/9uxJbQW3wOwxZasXnDNSf2TKb8DN7tqXuAB3I34vbgOCd2AqIBti/HfNB27gClA4yTHOwcY7B1zH7AM6AGclo5Ym3iJ5yCwP2C5LGAbTboErJuMu/+0Fzf9xF1Jjv+4V799wGrg6YB1/XDTaRzw1vXCjQieuH4xcE8Kca8Fmgd8rohLtDtwfxyMw43Mnrj+SlxPxD3ez2tQ4s/J+5lpkvrv9/vfkS3+LzYWnzHGmJBk96CMMcaEJEtQxhhjQpIlKGOMMSHJEpQxxpiQFHEjSZQsWVKjo6MzvP+BAwc4/fSgB6vOtaye4SVS6gmRU9dwqufs2bO3q2qppOURl6Cio6OZNWtWhvePjY2ladOmWRdQiLJ6hpdIqSdETl3DqZ4isi65cmviM8YYE5KyLUGJSHkRmehNkrZYRJ7yynuKyAYRmectgcOdPCciK0XkHxG5OqD8Gq9spYh0CyivKCIzvPIRIpI/u+pjjDEmZ2XnFVQ87qn1akAj4DERqeate1tVa3nLLwDeurtwoypfA/yfN0VAHtxQ/tcC1YC7A47zunes83FP+z+YjfUxxhiTg7ItQambgG2O934fbqyzsqnschMwXFWPqOoa3PAsDbxlpbqRl4/iBpm8SUQEN3zKt97+Q4Cbs6UyxhhjclyOdJLwpt6ujRtJuTHwuIi0wU2o9rSq7sIlr+kBu8XxX0Jbn6S8IW7wyt2qGp/M9knP3w5oB1C6dGliY2MzXJf9+/dnav/cwuoZXiKlnhA5dY2EemZ7ghKRwsB3QEdV3SsiHwG9cYND9saNLv1AdsagbhrtgQD16tXTzPR8CaeeM6mxeoaXSKknRE5dI6Ge2dqLT0Ty4ZLTV6r6PYCqblHV4+qmyh6Ea8IDN2p0+YDdy3llKZXvAM4MmKI6sdwYY0w2O3oUsnus8Wy7gvLuEX0KLFXVtwLKy6hq4lTZt+DmoAEYDQwTkbdw0xhUAv7GzddTSUQq4hLQXUBrVVURmYibD2c4brbUH7OrPsYYEy5U4dAh2L8f9u377zXwfWplixbBpk0AY2jQoBG//16KwoWzPs7sbOJrDNwHLBSReV5Zd1wvvFq4Jr61eDOIqupiEfkGWILrAfiYqh4HEJHHgfFAHmCwqi72jvcsMFxE+gBzcQnRGGPCSkKCSw6BSWPevDMzlFwSXxMSgjt3/vxQpAgULvzf64UXQsWKsGDBRv79dwCHD7+YuxKUqv5F8tNH/5LKPq8AryRT/kty+6nqav5rIjTGmJBw7Fj6kkZqiWTfPjh4MLmz1Er23KeddnJCKVIESpWCc889OckkrkurLH/A06UbN26ke/fuXHzxxTzyyCMcP/4QAHnyZPmPEIjAoY6MMSZQepq7gk0yR48Gd+6oqOQTRPnyaSeSVavmcemltU4qO/307EkWhw4don///rz22mvEx8dTtWpVAPJkV2byWIIyxuQqyTV3BSaLWbPKMGdO8Fcr+/fD8ePBnTt//uSTxtlnB39FEvhaqBBIcu1MQYiN3U29ehnbNz3GjRtHu3btWL9+PbfddhtvvPEG5557bvafGEtQxphsllpzV3qbvVJu7gpU5cS70047NVGUKuXun6QnoSTX3BXuVBURISoqihIlSjB06FCaNGmSozFYgjLGnKAKhw9n/v5JVjV3FS58anNXWolkwYKpXH31JdnW3BXu4uLieO655yhbtix9+/alRYsWNG/enKionB9b3BKUMblY0uaulJLGwoUV+OmntK9g0tPclS9f8okisbkrvTfkM9PcFWjTpqOccUbmjxNpDh48yJtvvsnrr79OQkIC3bqdGJfbl+QElqCMCXmqsHw5LFgAK1a494mv27cHe5SKJzV3Jb6WLAnR0env2VWkSGQ1d4W7P/74g7Zt2xIXF8cdd9zB66+/TmYmds0qlqCMCTHx8TB3LkyeDH/95ZZt2/5bf845ULky3HKLex9MQpk1K5ZmzZr6VicTmuLj48mbNy9nn3025cqVY9iwYVx22WV+h3WCJShjfHbgAMyY4RLS5MkwfborA3cz/9pr4bLLoG5dqFSJDD0QafdiTKD169fTrVs34uPjGTFiBNWqVWPq1KlIVrSxZiFLUMbksB073FVRYkKaM8ddNYlAjRoQE+MS0qWXQtnUJqgxJp0OHDjAG2+8wZtvvomq0qVLFxISEoiKigq55ASWoIzJVqqwbt3JCWnpUrcuf35o0ACeecYlo0sugTPP9DVcE8amT5/ObbfdxsaNG7nrrrvo27cvFSpU8DusVFmCMiYLJSTA4sUnJ6S4OLfujDOgcWO47z6XkOrXh4IF/Y3XhL9Dhw5RqFAhzjvvPKpXr87IkSO55JJL/A4rKJagjMmEo0dh1qz/OjRMmQK7drl1Zcq4prrE5cIL7V6QyTnr1q3j2WefZfXq1UyfPp1SpUrx66+/+h1WuliCMiYd9u6FadP+uzr6+2/3YCtAlSpw663/JaSKFbPmuR5j0mP//v307duXfv36ERUVRdeuXYmPjyd/LnwuwBKUManYvPnk5rr5810zXp48ULs2tG//X4eGs87yO1oT6RYtWsRVV13F5s2bueeee3jttdcoX7582juGKEtQxnhUYeXKkxPSypVuXaFC0KgRvPCCS0aNGrlnjIwJBbt27aJYsWJUqlSJJk2a0LFjRxo1auR3WJlmCcpErOPHYfnywixY8N89pM2b3brixV0ieuQR91qnjo2cYELPmjVrePbZZ5k+fTrLli3jtNNOY/jw4X6HlWUsQZmIceiQu2eUmIymToV9+9x8BRUqQLNm/90/qlrVDV5qTCjau3cvgwYN4rvvviNPnjx069YtJJ9jyixLUCZs7d59cnPdrFlu6gdwPeruuQdKlFjCI49UIxc305sI8++//9KgQQO2bNlCmzZtePXVVykbpk90W4IyYeP4cZeExo+HcePc8EEJCW7U7Xr1oFMn11zXuLFrwgOIjd1K+fLV/A3cmCBs2LCBsmXLUr58ee68806qVq3Ko48+6ndY2coSlMnVNm6EX391CWnCBNi503XtrlcPnn8errzSjdZw2ml+R2pMxqxatYquXbsyfvx4li9fzjnnnMO7775LbGys36FlO0tQJlc5csQ9DDtunLtSWrDAlZ99NtxwA1x9NVx1lZtGwpjcbO/evbzyyiu888475MuXj+eee45ixYr5HVaOsgRlQl58vLtKGjoUxoxxI33ny+ea615/3SWlGjXsoVgTPvbs2UOVKlXYsmULMTExvPLKK5xzzjl+h5XjLEGZkHX4MHz0Ebzxhuv+Xbw43HsvXHcdXHFFxqadMCaULVu2jKpVq1K0aFE6d+7MlVdeSb169fwOyzeWoExIWbMGfvvtv2XnTtf9+6OPoGVLexbJhKcVK1bwzDPPMHr0aObMmUOtWrXo2rWr32H5zhKU8VVCAkycCN984xLS6tWuvGxZuPFGaNPGXS0ZE452795Nnz59eO+99yhQoACvvvoqVatW9TuskGEJyvji339h8GD4/HM3X1KRIq7HXadO0Ly5G3jV7imZcHbs2DFq1arFv//+ywMPPECfPn04++yz/Q4rpFiCMjkmPh5+/hkGDoSxY11Z8+bQty/cfLPNjWQiw99//039+vXJly8fffr0oXr16tSuXdvvsEKSDeZist3Ro9CnD0RHu0Q0d657RmnNGtc77667LDmZ8Ld8+XJuvPFGGjZsyM8//wzAvffea8kpFXYFZbLNkSPw7LOuGW/PHmjRAj74AK6/HvLavzwTIXbt2kXv3r15//33KVSoEK+//jrNmzf3O6xcIduuoESkvIhMFJElIrJYRJ7yyouLyAQRWeG9FvPKRUTeE5GVIrJAROoEHKutt/0KEWkbUF5XRBZ6+7wn4ThaYi52113w7rtuJIeRI92DtTffbMnJRA5VpWnTprzzzjvcf//9rFixgq5du1LQmgyCkp1NfPHA06paDWgEPCYi1YBuwO+qWgn43fsMcC1QyVvaAR+BS2hAD6Ah0ADokZjUvG0eDtjvmmysjwmCKvTr555RGjUKevd2zXitWvkdmTE5548//uDo0aOICG+++SZz585l4MCBlC5d2u/QcpVsS1CquklV53jv9wFLgbLATcAQb7MhwM3e+5uAL9SZDpwpImWAq4EJqrpTVXcBE4BrvHVnqOp0VVXgi4BjGR+owtNPwzPPuNEeYmLce2MixbJly7juuuto1qwZn332GQAtWrSgZs2aPkeWO+VIY4uIRAO1gRlAaVXd5K3aDCT+SVEWWB+wW5xXllp5XDLlyZ2/He6qjNKlS2dqkMX9+/dHxCCN6a1nQgK8+24lRo8uy623xvHgg2s47bTjTJuWfTFmBfs+w48fdd2zZw9Dhgzhxx9/pFChQjz66KNUrFgxW+OIiO9UVbN1AQoDs4Fbvc+7k6zf5b3+BFwaUP47UA/oArwQUP6iV1YP+C2g/DLgp7TiqVu3rmbGxIkTM7V/bpGeesbHq95/vyqodu2qmpCQfXFlNfs+w48fdW3WrJlGRUVp+/btdcuWLTlyznD6ToFZmszv62y9ghKRfMB3wFeq+r1XvEVEyqjqJq+ZbqtXvgEInDaunFe2AWiapDzWKy+XzPYmB8XHQ9u2MGwY9OjhFuuqYiLBuHHjqF+/PiVKlOCNN94gX758XHTRRX6HFVaysxefAJ8CS1X1rYBVo4HEnnhtgR8Dytt4vfkaAXvUNQWOB1qISDGvc0QLYLy3bq+INPLO1SbgWCYHHD0Kd97pktNrr0HPnpacTPhbsmQJ1157Lddeey3vv/8+AHXq1LHklA2y8wqqMXAfsFBE5nll3YG+wDci8iCwDrjDW/cL0BJYCRwE7gdQ1Z0i0huY6W3XS1V3eu87AJ8DhYCx3mJywOHDrmfezz/D229Dx45+R2RM9tq+fTs9e/ZkwIABFClShLfffpsOHTr4HVZYy7YEpap/ASn9Pd0sme0VeCyFYw0GBidTPgu4MBNhmgw4eNA9zzRhghtlvH17vyMyJvt16tSJr7/+mvbt29OzZ09K2qyY2c6GOjLpsm+fm/bit9/gs88sOZnwpar89NNPLF++HIDevXszf/58PvjgA0tOOcQSlAnanj1u9tq//oKvvnLPORkTjhYtWsTVV1/NDTfcwNtvvw1AdHQ01atX9zmyyGIJygRlxw43ceCsWW7uprvv9jsiY7Letm3b6NChAzVr1mTWrFm8++67vPfee36HFbFsVDSTpq1b3bQYy5fDDz+4KdeNCUf9+/dn4MCBPPbYY/To0YMSJUr4HVJEswRlUrVxo7tyWrcOxoyBq67yOyJjso6qMnr0aEqWLEnjxo3p1q0bbdu25YILLvA7NIM18ZlU/PsvXH45xMW5CQYtOZlwsmDBApo3b87NN9/Mu+++C8CZZ55pySmEWIIyydq4sSCXXw7btrnRyJs08TsiY7LG1q1beeSRR6hduzbz5s3j/fff56uvvvI7LJMMa+Izp/jnH3jqqdokJMAff0Ddun5HZEzWGTlyJIMHD+aJJ57gpZdeonjx4n6HZFJgCcqcZNEi1yEiPl6YNAlq1PA7ImMyR1UZNWoUx48fp1WrVrRr146rrrqKypUr+x2aSYM18ZkT5s6Fpk0hKgreeWeeJSeT682bN48rr7ySW2+9lQEDBgCQL18+S065hCUoA8CMGXDllXDaafDnn1ChwkG/QzImw7Zs2cLDDz9MnTp1WLhwIf/3f//HuHHj/A7LpJMlKMNff7keesWLu+R0/vl+R2RM5syaNYshQ4bQqVMnVq5cyaOPPkrevHZHI7exbyzC/fEH3HADlCvn3pdNdk5iY0KbqvLdd9+xceNGatSoQcuWLVm1ahXly5dPe2cTsuwKKoKNG+dGhahYESZNsuRkcqc5c+bQtGlTbr/9dr788kuOHz+OiFhyCgOWoCLUjz/CTTdB1aoQGwtnn+13RMakz+bNm3nwwQepV68eS5cuZcCAAUydOpU8efL4HZrJItbEF4HWrYO77oJatdxVVLFifkdkTPpt27aNr7/+mi5duvD8889TtGhRv0MyWSzFBCUi76hqRxEZA2jS9ap6Y7ZGZrLNM8+4qdm//daSk8k9VJWRI0cya9Ys3njjDS666CLi4uLsQdswltoV1FDvtV9OBGJyRmwsjBwJL78M1kRvcotZs2bRsWNHpkyZQq1atTh48CCnnXaaJacwl2KCUtXZ3uuknAvHZKejR+HJJ+F//4MuXfyOxpi0bd26lWeeeYYvvviCs846i0GDBnH//ffbfaYIkeY9KBFpDPQEKnjbC6Cqem72hmaykio8+CAsXOiuoE47ze+IjEmbqjJ27FieffZZunfvzhlnnOF3SCYHBdNJ4lOgEzAbOJ694Zjs8v338OWX8OKL0KqV39EYkzxVZfjw4YwaNYrhw4dTunRp1qxZw+mnn+53aMYHwXQz36OqY1V1q6ruSFyyPTKTZZYuhQcegNq14fnn/Y7GmOTNmDGDxo0b07p1a1asWMG2bdsALDlFsGAS1EQReVNELhaROolLtkdmssTkydC4MRQq5J59KlDA74iMOdnOnTu57777aNSoEWvWrGHw4MHMnDmTs846y+/QjM+CaeJr6L3WCyhT4MqsD8dkpe3b4frr3ejkw4dbrz0TmgoVKsTMmTPp3r073bp1o0iRIn6HZEJEmglKVa/IiUBM1uvbF/bvdx0jqlXzOxpjnISEBIYNG8aAAQOYMGEChQoVYuHCheTLl8/v0EyISe1B3XtV9UsR6ZzcelV9K/vCMpm1YQN88AG0aWPJyYSOadOm0bFjR/7++2/q1q3Lli1biI6OtuRkkpXaPajEO5NFUlhMiJk4Edq3hwMH4JVXICEBevTwOypjYP/+/bRu3ZpLLrmE9evX8/nnn/P3338THR3td2gmhKX2oO7H3uvLOReOyahdu+Dmm2HvXti3zz3r9OCDYP//jZ8SEhKIiori9NNPZ8uWLbzwwgs8++yzFC5c2O/QTC4QzIO676W2XlWfzLpwTEb16OGunEqWhGHDIG9e6NXL76hMpEpISGDo0KH07duX2NhYSpcuzYQJE4iKsgkUTPCC+ddSEKgDrPCWWkB+3IO7s1PaSUQGi8hWEVkUUNZTRDaIyDxvaRmw7jkRWSki/4jI1QHl13hlK0WkW0B5RRGZ4ZWPEJH86ah3WJk7F95/H26/HUaNcmVt20KpUr6GZSLUlClTaNiwITExMZxxxhns2rULwJKTSbdgupnXAC5V1XgAERkATFbV9mns9znwAfBFkvK3VfWkAWhFpBpwF1AdOAf4TUQqe6s/BK4C4oCZIjJaVZcAr3vHGu7F9CDwURD1CSvLlsEVXj/Lzp2hfn1YsAAqV059P2OyWnx8PPfddx/Dhw+nbNmyDB06lNatW1tiMhkWzL+cYkDgAFiFvbJUqeqfwM4g47gJGK6qR1R1DbASaOAtK1V1taoeBYYDN4mI4J7D+tbbfwhwc5DnCivvv++6kk+f7pITwEUX2QO5JuccO3YMgLx581K4cGF69OjBP//8w7333mvJyWRKMFdQfYG5IjIRN1Ds5bjBYzPqcRFpA8wCnlbVXUBZYHrANnFeGcD6JOUNgRLA7sSruiTbn0JE2gHtAEqXLk1sbGyGg9+/f3+m9s8qCQkwZsw5fPrpeTRpsp1Dh5aSlWGFSj2zm9Uz4xISEhg/fjyfffYZffv25dxzz+Wee+4BYObMmVl6rvSw7zSMqGqaC3A27irnJuDsYPbx9osGFgV8Lg3kwV25vQIM9so/AO4N2O5ToJW3fBJQfp+3bUnclVViefnA86S21K1bVzNj4sSJmdo/q7z4oiqoVq2qumpV1h8/VOqZ3ayeGfPnn39qnTp1FNCGDRvq/Pnzs/T4mWHfae4DzNJkfl8HNeW7qm4GfsxcKgRV3ZL4XkQGAT95Hzd4SSZROa+MFMp3AGeKSF51V1GB24e9v/6CPn2gbFk3SkTeoL5FYzJPVWnbti1Dhw6lfPnyfPXVV9x99924VndjslaONhCLSJmAj7cAiT38RgN3iUgBEakIVAL+BmYClbwee/lxHSlGexl3Iu4KC6AtWZBAc4N9+9zoENHRbpRyS04mJxw4cABVRUSoWrUqvXr1YtmyZbRu3dqSk8k22fbrTUS+BpoCJUUkDugBNBWRWrjBZtcCjwCo6mIR+QZYAsQDj6nqce84jwPjcU2Dg1V1sXeKZ4HhItIHmItrFgx7zz8Pa9fCn3+Cjalpstvx48f5/PPPef755xk8eDAtW7ake/fufodlIkRQCUpELgUqqepnIlIKKKyut12KVPXuZIpTTCKq+gruvlTS8l+AX5IpX43r5RcxBg50vfZuvRUuvdTvaEy4i42NpVOnTsybN4/GjRtTpkyZtHcyJgul2cQnIj1wVyvPeUX5gC+zMyhzqq+/hkcegebN3XtjstOjjz7KFVdcwc6dOxk+fDiTJ0+mdu3afodlIkwwV1C3ALWBOQCqulFErHEph332mXsdNgzyR+yYGSY77d27l0KFCpEvXz4aN25MuXLl6Ny5M4UKFfI7NBOhgukkcdTrlKAAImLzL+ewHTvgjz/gueds+CKT9Y4fP86gQYOoVKkSH3/8MQD33nsvzz//vCUn46tgEtQ3IvIxrlv3w8BvwKDsDcsE6t0bjh+HVq3S3taY9Pjjjz+oU6cO7dq1o0qVKlxyySV+h2TMCcHMqNtPRK4C9gJVgJdUdUK2R2YAWLIE3n0XHn8c6tTxOxoTTrp27cqbb75JdHQ0I0eO5LbbbrMu4yakBDPdRmdghCUlf4wb5167dvU3DhMe9uzZg4hwxhln0LJlS4oVK0anTp0oWLCg36EZc4pgmviKAL+KyGQReVxESmd3UOY/v/wCVatC+fJpb2tMSo4fP86AAQM4//zz6dmzJwBNmzblueees+RkQlaaCUpVX1bV6sBjQBlgkoj8lu2RGVasgN9/h7uTe6LMmCBNmDCBhx56iEcffZTq1atz3333+R2SMUFJz1BHW4HNuHHwzsqecEygIUMgKgoeftjvSExu9corr9CiRQuOHDnCd999x8SJE+15JpNrBHMPqgNwB1AKGAk8rG7CQJONjh93Cerqq8Ee4DfpsWvXLg4dOsQ555xDq1atyJs3L7Vr16ZFixZ+h2ZMugRzBVUe6Kiq1VW1pyWnnPHppxAX56ZuNyYY8fHxfPjhh5x//vk88cQTAFSpUoVnn32W/PZ0t8mFUkxQIpI4i+6bwL8iUjxwyZnwItO//8Kjj7rpNG66ye9oTG4wfvx4atasyeOPP07NmjV56aWX/A7JmExLrYlvGHA9MBs3ikTgAxIKnJuNcUW0N990M+b++itYByuTlo8++ogOHTpw3nnnMWrUKG688UZ7nsmEhRQTlKpe771WzLlwzNGjMHQo3HMPVKvmdzQmVO3cuZOtW7dStWpV7rjjDg4fPkyHDh0oUKCA36EZk2WCGc3892DKTNb45RfYs8e6lpvkHTt2jPfff5/zzz+fNm3aoKqUKFGCTp06WXIyYSe1e1AFvXtNJUWkWMD9p2igbI5FGEFUoW9fqFABrMOVSWrs2LHUqFGDJ598krp16/Lpp59aU54Ja6ndg3oE6Aicg7sPlfg/YS/wQfaGFZm++QZmzIABAyBfPr+jMaHk22+/5fbbb6dy5cqMGTOG6667zpKTCXup3YN6F3hXRJ5Q1fdzMKaIdPgwdOsGNWtCTIzf0ZhQsH37dlauXEmjRo248cYbGThwIG3btrUu4yZiBDOa+fsiciFQDSgYUP5FdgYWaXr1grVr3eCwdishsh09epT/+7//4+WXX6ZIkSKsXr2a/Pnz87ANKWIiTLBTvr/vLVcAbwA3ZnNcEWX/fvjwQ7jzTjdyhIlMqspPP/3ERRddRKdOnWjQoAG//PILefMGM/G1MeEnmJEkWgHNgM2qej9QEyiarVFFmC++gL17oWNHvyMxfpo8eTI33HADIsLPP//MuHHjuPDCC/0OyxjfBJOgDqlqAhDvjS6xFTf8kckCqvDBB1CvHjRs6Hc0Jqdt27aNX375BYDLLruMESNGsHDhQlq2bGmdIEzEC6btYJaInImb5n02sB+Ylp1BRZLff4elS93AsPb7KHIcPXqU999/n969e6OqxMXFUaRIEe644w6/QzMmZAQzH1QHVd2tqgOAq4C2XlOfySRVuOoq9/7OO/2NxeQMVeXHH3+kevXqdOnShcaNGzN9+nSKFCnid2jGhJwUr6BEpE5q61R1TvaEFDlmz3avHTpYz71IsXz5cm655RaqVq3K2LFjueaaa/wOyZiQlVoTX/9U1ilwZRbHEnG+/Rby5oXevf2OxGSnrVu3MnbsWNq2bUuVKlWYMGECl19+OfnsaWxjUpXag7pX5GQgkeiPP6BRIyhuk5eEpSNHjvDee+/Rp08fDh8+TPPmzSlbtizNmjXzOzRjcoVgZtRtk1y5PaibOT/+CDNnurH3THhRVUaNGkWXLl1YvXo1119/Pf369aNsWRvC0pj0CKYXX/2A9wVxz0TNASxBZcJ330HRovDUU35HYrLa9u3bue+++4iOjmb8+PE21boxGRRML74nApaHgTpA4bT2E5HBIrJVRBYFlBUXkQkissJ7LeaVi4i8JyIrRWRBYAcNEWnrbb9CRNoGlNcVkYXePu9JLntoZO5cuPRSm5AwXGzevJnXX38dVaVUqVJMmjSJefPmWXIyJhOCeVA3qQNAMJMYfg4k7aLUDfhdVSsBv3ufAa4FKnlLO+AjcAkN6AE0BBoAPRKTmrfNwwH75ZruUIcOuWefatXyOxKTWYcPH6Zv375UqlSJF198kUWL3N9jdevWtSGKjMmkYMbiGyMio73lJ+Af4Ie09lPVP4GdSYpvAoZ474cANweUf6HOdOBMESkDXA1MUNWdqroLmABc4607Q1Wnq6rimhtvJpcYNAiOH4fGjf2OxGSUqvLtt99ywQUX8Nxzz9GsWTMWL17MRRdd5HdoxoSNYP7E6xfwPh5Yp6pxGTxfaVXd5L3fDJT23pcF1gdsF+eVpVYel0x5skSkHe7KjNKlSxMbG5vB8GH//v2Z2h/g009rce65eSlYcBaZPFS2yYp65gYZreeRI0fo0KEDhQsXpl+/ftStW5cNGzawYcOGrA8yC0TK9wmRU9dIqGcw021MAvDG4cvrvS+uqkmvjtJFVVVENDPHSMe5BgIDAerVq6dNmzbN8LFiY2PJzP5Ll8KCBW56jSuuyPhxsltm65lbpKeemzZt4u2336Z3794UKFCAKVOmULFixVzRlBcp3ydETl0joZ7BNPG1E5HNwAJgFm48vlkZPN8Wr3kO73WrV76BkwegLeeVpVZeLpnykDdkiJst95FH/I7EBOvQoUO88sorVKpUiXfeeYfp06cDUKlSpVyRnIzJrYLpJPEMcKGqRqvquapaUVXPzeD5RgOJPfHaAj8GlLfxevM1AvZ4TYHjgRYiUszrHNECGO+t2ysijbzee20CjhXSpkxxI5efdZbfkZi0qCojRoygatWqvPDCC7Ro0YIlS5bQpEkTv0MzJiIE8+ffKuBgeg8sIl8DTYGSIhKH643XF/hGRB4E1gGJQzf/ArQEVnrnuh9AVXeKSG9gprddr4CmxQ64noKFgLHeEtKOHnUP5z72mN+RmGCoKv3796dYsWIMGTIk7JtTjAk1wSSo54CpIjIDOJJYqKpPpraTqt6dwqpTxnnxeuIl+2tbVQcDg5MpnwXkqtncYmPhyBH3/JMJTRs2bKBXr1706dOHUqVKMXr0aEqVKkWePHn8Ds2YiBNME9/HwB/AdNz9p8TFpNOgQVCiBLRs6XckJqmDBw/Su3dvKleuzOeff87UqVMBOPvssy05GeOTYK6g8qlq52yPJMwNG+ZGL+/UyabWCDVff/01zz77LOvXr6dVq1a8/vrrnHtuRm+zGmOySjAJaqz3HNEYTm7iy1Q380iyezc8+KCbMTcmxu9oTFLff/89JUuW5Msvv+Tyyy/3OxxjjCeYBJV4L+m5gDIF7E/MIC1ZAocPw88/Q40afkdj4uLi6N69O926uZG2PvnkE4oUKUJUVEZG/jLGZJdgHtQNZtw9k4q5c91rlSr+xhHpDhw4wJtvvskbb7xBQkICV111FeXLl6do0aJ+h2aMSYbNB5UDRo6EatXAbmv4Z/jw4XTp0oUNGzZw55130rdvX6Kjo8N+qBhjcjObDyqbHT4M06bBk0+6e1DGH3PmzKFMmTIMHz6cS62fvzG5QjBNfE8EfhaRM4Hh2RVQuFmxwj2gW6+e35FEln///Zdu3boRExNDixYt6NWrF/nz57f7TMbkItk5H5QBVq50r+ef728ckWL//v28+OKLVKlShR9++IE1a9YAULBgQUtOxuQywdyDGoPrtQcuoVUDvsnOoMLJ+PFucNhKlfyOJPyNHDmSjh07snHjRlq3bs1rr73G//73P7/DMsZkUE7PBxVRtm6FgQPdyOVnnOF3NOFLVRERduzYQbly5fj222+5+OKL/Q7LGJNJKbZ5iMj5ItJYVScFLFOACiJyXg7GmGtNnQqq0CbZfpAms9auXcudd97JoEGDAHj44YeZNm2aJSdjwkRqjfLvAHuTKd/rrTNpmDoV8ueHOnX8jiS87Nu3j+eff56qVasyZswYDh50g+3nyZPH7jMZE0ZSa+IrraoLkxaq6kIRic6+kMLHr79CgwY29l5W+vHHH2nfvj2bN2/m3nvv5bXXXqNcuXJp72iMyXVSS1BnprKuUBbHEXbi4mD+fOjf3+9IwkNCQgJRUVEULFiQ6OhoRo0aRcOGDf0OyxiTjVJrD5klIg8nLRSRh7DpNtKUOLyR/Q7NnNWrV9OqVSu6d+8OwNVXX83UqVMtORkTAVJLUB2B+0UkVkT6e8sk4EHgqRyJLheLjXX3n2rV8juS3Gnv3r1069aNCy64gLFjx1KsWLET68SG5DAmIqTYxKeqW4BLROQK/pu59mdV/SNHIsvlxo6FJk3g9NP9jiT3GT9+PG3btmXLli20adOGV199lbJly/odljEmhwUz1NFEYGIOxBI2Vq6EpUuhXTu/I8ldjh49Sv78+SlXrhxVqlRhzJgx1K9fP+0djTFhKZgHdU06ffede73lFn/jyC1WrVrFM888Q758+RgxYgTVq1dn0qRJfodljPGZPTSSDb77zg0OW6GC35GEtj179tC1a1eqVavGr7/+Ss2aNVHVtHc0xkQEu4LKYmvXwsyZ8NprfkcS2iZPnsxtt93G9u3biYmJ4ZVXXqFMmTJ+h2WMCSF2BZXFRoxwr3fe6W8coWr//v0AVK1alQYNGjBz5kwGDx5syckYcwq7gspCqvDZZ3DJJVDRJiQ5yYoVK3jmmWfYtGkT06ZNo1SpUvz0009+h2WMCWF2BZWFpk2Df/6Bhx7yO5LQsXv3brp06UL16tX5/fffueWWWzh+/LjfYRljcgG7gspCkye71xtv9DeOUDF37lxatGjBjh07eOCBB+jTpw9nn32232EZY3IJu4LKQrNmuaa9EiX8jsRf27dvB6BatWpcc801zJ49m08++cSSkzEmXSxBZZH9++GHH1z38kj1zz//cMMNN1CnTh0OHjxIgQIFGDp0KLVr1/Y7NGNMLmQJKot8+SUcPw433eR3JDlv165ddOrUiQsvvJBJkybxxBNPkCdPHr/DMsbkcr7cgxKRtcA+4DgQr6r1RKQ4MAKIBtYCd6jqLnEjg74LtAQOAjGqOsc7TlvgBe+wfVR1SE7WI9DUqVCmDLRu7VcE/li9ejX169dn9+7dPPTQQ/Tq1YvSpUv7HZYxJgz4eQV1harWUtXERrFuwO+qWgn43fsMcC1QyVvaAR8BeAmtB9AQaAD0EJFi+GTFCrjgAoiUgbbXrVsHQMWKFYmJiWHOnDl8/PHHlpyMMVkmlJr4bgISr4CGADcHlH+hznTgTBEpA1wNTFDVnaq6C5gAXJPDMQOuae+ff+D88/04e85aunQpLVu2pHr16mzcuBERoX///tSsWdPv0IwxYcavbuYK/CoiCnysqgNxU8xv8tZvBhL/FC8LrA/YN84rS6n8FCLSDnf1RenSpYmNjc1w4Pv37z9l/4ULz2DXrjqULr2Y2NhtGT52KElazz179jBkyBB+/PFHChUqRJs2bVi0aBHLly/3L8gskNz3GY4ipZ4QOXWNhHr6laAuVdUNInIWMEFElgWuVFX1kleW8BLgQIB69epp06ZNM3ys2NhYku6/aJF7bd++Oueck+FDh5TAeu7cuZNKlSqxe/du2rVrR69evShVqpS/AWaR5L7PcBQp9YTIqWsk1NOXJj5V3eC9bgV+wN1D2uI13eG9bvU23wCUD9i9nFeWUnmOW7wYzjzTdZIIF6rKwoULAShevDjdu3dn3rx5fPTRR2GTnIwxoS3HE5SInC4iRRLfAy2ARcBooK23WVvgR+/9aKCNOI2APV5T4HighYgU8zpHtPDKctyiRXDhheHTQWLx4sV07dqVmjVrsmDBAgCefvppLrroIp8jM8ZEEj+a+EoDP7je4+QFhqnqOBGZCXwjIg8C64A7vO1/wXUxX4nrZn4/gKruFJHewExvu16qujPnquEkJLgEFQ6jl2/fvp0ePXrw8ccfU6hQId566y2qVq3qd1jGmAiV4wlKVVcDp3T5UtUdQLNkyhV4LIVjDQYGZ3WM6bFyJezeDbl9ZvIjR45Qs2ZNtmzZQvv27bnqqqu4KRKfOjbGhIxQ6maeK82Y4V4bNvQ3joxQVSZPnoyqUqBAAd544w3mz5/PBx98QNGiRf0OzxgT4SxBZdKMGVC4sHtINzdZuHAhLVq04PLLL2f8eHfr7p577qF69eo+R2aMMY4lqEyaMcM17+WWoee2bdvGo48+Sq1atZg9ezbvvfcezZqd0rJqjDG+s/mgMuHIEZg/Hzp39juS4KgqTZo0Yfny5Tz++OP06NGD4sWL+x2WMbnesWPHiIuL4/Dhwzl2zqJFi7J06dIcO19WKFiwIOXKlSNfvnxBbW8JKhPmz4djx0K7g4SqMm7cOJo1a0b+/Pl57733KFu2LBfktjZJY0JYXFwcRYoUITo6Gsmh50327dtHkSJFcuRcWUFV2bFjB3FxcVSsWDGofayJLxNmeh3cQzVBLViwgObNm9OyZUuGDh0KQPPmzS05GZPFDh8+TIkSJXIsOeVGIkKJEiXSdZVpCSoTZs6Es86C8uXT3jYnbd26lUceeYTatWszb948PvjgA9q2bZv2jsaYDLPklLb0/oysiS8TZs50V0+h9u/yjjvuYMqUKTz55JO89NJLFCvm2ywkxhiTYXYFlUH79sHSpaHRvKeq/PDDD+zc6QbSeOedd1i0aBFvv/22JSdjTLI+//xzHn/8cQAGDBjAF198AcCyZcuoVasWtWvXZtWqVQwbNsy3GC1BZdDMmaAKDRr4G8fcuXO54ooruPXWW/noo48AqFWrFlWqVPE3MGNMrtG+fXvatGkDwKhRo2jVqhVz585l/fr1viYoa+LLoEmTICoKGjf25/ybN2/mhRdeYPDgwZQoUYKPPvqIhx56yJ9gjDEndOwI8+Zl7TFr1YJ33kl9m7Vr13L99dezyJv/p1+/fifmjKpZsyaTJk0iPj6ewYMH0yDJX9Y9e/akcOHCVKtWjXfeeYc8efLw+++/c+jQIZYuXUqtWrVo27YtnTp1ytqKpcESVAZNmgS1a8MZZ/hz/s6dO/Ptt9/SuXNnXnjhBc4880x/AjHGhLyDBw8yb948/vzzTx544IETSSypli1b0r59ewoXLkyXLl2IjY2lX79+/PTTTzkcsWMJKgMOH4bp0+GxZIewzR6qynfffUeNGjWoXLkyr732Gi+//DKVKlXKuSCMMWlK60rHD3fffTcAl19+OXv37mX37t3+BhQkuweVAX//7UaRaNIkZ843e/ZsmjRpwu23386HH34IQIUKFSw5GWNOyJs3LwkJCSc+Bz5vlLR7d27pEm8JKgP++su9Xnpp9p5n48aN3H///dSvX59ly5bx8ccf89Zbb2XvSY0xuVLp0qXZunUrO3bs4MiRIyc1y40YMQKAv/76i6JFiwY9W0GRIkXYt29ftsQbDGviy4B//oFy5SC7h7F7++23GTZsGM888wzdu3e3KTCMMSnKly8fL730Eg0aNKBs2bInTTZasGBBateuzbFjxxg8OPgp9GrUqEGePHmoWbMmMTExOd5JAlWNqKVu3bqaGRMnTtRLLlFt2jRTh0lWQkKCDh8+XCdPnqyqqrt27dKVK1dm/YmCMHHiRF/Om9OsnuHHj7ouWbIkx8+5d+/eoLZr0qSJzpw5M5ujCV5yPytglibz+9qa+DJg1So477ysPebMmTO59NJLueuuuxgwYAAAZ555Judl9YmMMSaXsASVTgcP5mHLFjj//Kw53oYNG2jbti0NGjRg1apVfPLJJwwZMiRrDm6MiXixsbHUq1fP7zAyxO5BpdPatacDkFUDNXz//fcMHz6cbt268dxzz3GGXw9WGWNMiLEElU4LF7qOCpdckrH9VZWvv/6afPnycfvtt9O+fXuuv/76oOdHMcaYSGFNfOm0YEFRKlWC0qXTv+/06dO55JJLuOeee/j8888B1/PGkpMxxpzKElQ6JCTAokVF0/380/r167n33nu5+OKLWbt2LZ999hljxozJniCNMSZMWIJKh9WrYe/efOlu3ps/fz7ffvstzz//PCtWrCAmJoaoKPvRG2NCV3R0NNu3b/c1BrsHlQ5r1rjXtEYYSkhIYNiwYezYsYOnnnqK6667jjVr1lCmTJnsD9IYE/FOPEeUy/8QtgSVDuvWudcKFVLeZtq0aXTs2JG///6byy67jCeeeIKoqChLTsZEkKZNm55Sdscdd9ChQwcOHjxIy5YtT1kfExNDTEwM27dvp1WrVieti42NTfOca9eu5eqrr6Zhw4bMnj2bBg0asHDhQg4dOkSrVq14+eWXAXdl1LZtW8aMGcOxY8cYOXIkVatWZceOHdx9991s2LCBiy++GPf8rPPWW2+dGIHioYceomPHjqxdu5ZrrrmGRo0aMXXqVOrXr8/9999Pjx492Lp1K1999dUp03qkV+5Orzls3TqIilLKlTt1XVxcHK1bt+aSSy4hLi6OIUOGEBsbm+v/gjHG5B4rVqygQ4cOLF68mP79+zNr1iwWLFjApEmTWLBgwYntSpYsyZw5c3j00Ufp168fAC+//DKXXnopixcv5pZbbuHff/8F3GDVn332GTNmzGD69OkMGjSIuXPnArBy5Uqefvppli1bxrJlyxg2bBh//fUX/fr149VXX810fewKKh3WroVSpY6QN2/BU9bt3LmT0aNH8+KLL9K1a1cKFy6c8wEaY0JCalc8p512WqrrS5YsGdQVU3IqVKhAo0aNAPjmm28YOHAg8fHxbNq0iSVLllCjRg0Abr31VgDq1q3L999/D8Cff/554v11111HsWLFADfA7C233MLpp59+Yt/Jkydz4403UrFiRS666CIAqlevTrNmzRARLrroItauXZuhOgTK9QlKRK4B3gXyAJ+oat/sOte6dVC69GGgIAkJCQwdOpQFCxbQv39/atSowYYNG2xAV2OMbxKTyJo1a+jXrx8zZ86kWLFixMTEnDT9RoECBQDIkycP8fHxGT5f4nEAoqKiTnyOiorK1HFPHDPTR/CRiOQBPgSuBaoBd4tItew63333QcuWm/jrr79o2LAhMTExTJky5cQXb8nJGBMK9u7dy+mnn07RokXZsmULY8eOTXOfyy+/nGHDhgEwduxYdu3aBcBll13GqFGjOHjwIAcOHOCHH37gsssuy9b4E+XqBAU0AFaq6mpVPQoMB27KrpNdf/0mpk/vwGWXXcamTZv48ssvmTp1KgULntrkZ4wxfqlZsya1a9ematWqtG7dmsaNG6e5T48ePfjzzz+pXr0633//Pf/73/8AqFOnDjExMTRo0ICGDRvy0EMPUbt27eyuApD7m/jKAusDPscBDbPrZCLC/Pnz6dmzJ126dDlxOW2MMX6Ljo5m0aJFJz4njlaTVOC9oXr16p2431WiRAl+/fXXZPfp3LkznTt3Dvp8SddllAR2JcxtRKQVcI2qPuR9vg9oqKqPJ9muHdAOoHTp0nWHDx+e4XPu3LmT4tk9U2EI2L9/f0R09LB6hh8/6lq0aFHOz6opDoJ0/Phx8uTJk6PnzAorV65kz549J5VdccUVs1X1lCHXc/sV1AagfMDncl7ZSVR1IDAQoF69eprcMwrBio2NTfYZh3Bj9QwvkVJP8KeuS5cupUiRIjl6zn379uX4ObNC4uy+wcjt96BmApVEpKKI5AfuAkb7HJMxJgLl5taonJLen1GuTlCqGg88DowHlgLfqOpif6MyxkSaggULsmPHDktSqVBVduzYka5OZbm9iQ9V/QX4xe84jDGRq1y5csTFxbFt27YcO+fhw4dzXQ/iggULUi65oXhSkOsTlDHG+M2Ped1iY2NzrLu3X3J1E58xxpjwZQnKGGNMSLIEZYwxJiTl6gd1M0JEtgHrMnGIkoC/00zmDKtneImUekLk1DWc6llBVUslLYy4BJVZIjIruSeew43VM7xESj0hcuoaCfW0Jj5jjDEhyRKUMcaYkGQJKv0G+h1ADrF6hpdIqSdETl3Dvp52D8oYY0xIsisoY4wxIckSlDHGmJBkCSoZInKNiPwjIitFpFsy6wuIyAhv/QwRifYhzCwRRF1jRGSbiMzzlof8iDMzRGSwiGwVkWSn+BTnPe9nsEBE6uR0jFkhiHo2FZE9Ad/lSzkdY1YQkfIiMlFElojIYhF5KpltwuU7DaauYfG9JktVbQlYgDzAKuBcID8wH6iWZJsOwADv/V3ACL/jzsa6xgAf+B1rJut5OVAHWJTC+pbAWECARsAMv2POpno2BX7yO84sqGcZoI73vgiwPJl/t+HynQZT17D4XpNb7ArqVA2Alaq6WlWPAsOBm5JscxMwxHv/LdBMRCQHY8wqwdQ111PVP4GdqWxyE/CFOtOBM0WkTM5El3WCqGdYUNVNqjrHe78PNxdc2SSbhct3Gkxdw5YlqFOVBdYHfI7j1H8QJ7ZRN2niHqBEjkSXtYKpK8BtXjPJtyJSPmdCy1HB/hzCwcUiMl9ExopIdb+DySyveb02MCPJqrD7TlOpK4TZ95rIEpRJyxggWlVrABP478rR5D5zcGOe1QTeB0b5G07miEhh4Dugo6ru9Tue7JRGXcPqew1kCepUG4DAq4RyXlmy24hIXqAosCNHostaadZVVXeo6hHv4ydA3RyKLScF853neqq6V1X3e+9/AfKJSEmfw8oQEcmH+4X9lap+n8wmYfOdplXXcPpek7IEdaqZQCURqSgi+XGdIEYn2WY00NZ73wr4Q727lblMmnVN0m5/I64NPNyMBtp4Pb8aAXtUdZPfQWU1ETk78V6piDTA/f/PdX9YeXX4FFiqqm+lsFlYfKfB1DVcvtfk2JTvSahqvIg8DozH9XIbrKqLRaQXMEtVR+P+wQwVkZW4m9J3+RdxxgVZ1ydF5EYgHlfXGN8CziAR+RrX06mkiMQBPYB8AKo6APgF1+trJXAQuN+fSDMniHq2Ah4VkXjgEHBXLv3DqjFwH7BQROZ5Zd2B/0F4facEV9dw+V5PYUMdGWOMCUnWxGeMMSYkWYIyxhgTkixBGWOMCUmWoIwxxoQkS1DGGGMyJK0BipPZ/o6AgW+HpbW9JShjAoiIikj/gM9dRKRnFh37cxFplUXHmuGNXP2vnDzafHRWHD+Z83XPwD4xIvJBdsRjQsbnwDXBbCgilYDngMaqWh3omNY+lqCMOdkR4NZQexLfG7HkBFVtqKq1gJdwo+nX8pa16TlOOqQ7QZnwl9wAxSJynoiME5HZIjJZRKp6qx4GPlTVXd6+W9M6viUoY04WDwwEOiVdkfQKSET2e69NRWSSiPwoIqtFpK+I3CMif4vIQhE5L+AwzUVklogsF5Hrvf3ziMibIjLTG5T3kYDjThaR0cCStAIXkRu8K6u5IvKbiJT2ynuKyFARmYJ7wLyUiEzwmlk+EZF1iQlZRO714p4nIh97sfUFCnllX6W0nVd+v1e3v3EPmZrIMxB4QlXrAl2A//PKKwOVRWSKiEwXkTSvvCxBGXOqD4F7RKRoOvapCbQHLsA9+V9ZVRvgxi98ImC7aNw0J9cBA0SkIPAgbiie+kB94GERqehtXwd4SlUrBxHDX0AjVa2Nmzqla8C6akBzVb0bN8LEH14zy7d4oxKIyAXAnbgmmFrAceAeVe0GHPKu0O5JaTtxw2K9jEtMl3rnNBFE3KC2lwAjvZEvPsbNaQVu5KJKuNFO7gYGiciZqR3PhjoyJglV3SsiXwBP4oaOCcbMxLHeRGQV8KtXvhC4ImC7b1Q1AVghIquBqkALoEbA1VlR3H/ko8DfqromyBjKASO8RJEfCNxvtKom1uVS4BavruNEZJdX3gw3GPBMb2i3QkByzTApbdcQiFXVbd7PYQTur2YTOaKA3d4fLknF4SaOPAasEZHluH/nM1M7mDHmVO/grmxODyiLx/s/IyJRuCSQ6EjA+4SAzwmc/Idg0rHFFDfr6xMB95EqqmpigjuQjpjfx81+fBHwCFAwYF0wxxFgSEAcVVS1Zya2MxHGmwpkjYjcDm6wWxGp6a0ehbt6wmtSrgysTu14lqCMSYaq7gS+wSWpRGv5b7qRG/EGYk2n20UkyrsvdS7wD26w3kfFTauAiFQWkdNTO0gKivLflBJtU9luCnCHd64WQDGv/HeglYic5a0rLiIVvHXHEuNLZbsZQBMRKeFte3sG6mByEXEDFE8DqohInIg8CNwDPCgi84HF/DdL93hgh4gsASYCz6hqqqOuWxOfMSnrDzwe8HkQ8KP3H28c6bu6SfQv8DdwBtBeVQ+LyCe4e1NzxLWZbQNuzsCxe+La/ncBfwAVU9juZeBrEbkP98tlM7BPVbeLyAvAr94V4jHgMWAd7sb3AhGZ492HOmU7VZ0urkv+NGA3MC8DdTC5iHdPMzmndIDwRljv7C1BsdHMjYkwIlIAOO5Nt3Ix8FEK9wyM8ZVdQRkTef4HfONd/RzFPZ9iTMixKyhjjDEhyTpJGGOMCUmWoIwxxoQkS1DGGGNCkiUoY4wxIckSlDHGmJD0/5QVvnacn4RvAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "perf_h2o.plot_uplift(metric=\"qini\")" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABK+ElEQVR4nO3debxN1fvA8c9jJvN0E4UyFJnHSCjfSH6NGvgmVOiLShIaUX2jIs2GJEqi0sA3itRVJFPJlAxRLjLP83Wf3x9rXx3XHc4d9z3nPO/Xa7/OOWtPzzqX+9y99tpriapijDHGZDc5/A7AGGOMSYwlKGOMMdmSJShjjDHZkiUoY4wx2ZIlKGOMMdmSJShjjDHZkiUoE3FEZLCITPI7jmCISBcRmR/w+bCIXOy9zy8iM0TkgIh87F+U2Z+INBOR3/2Ow6SOJSiTZUSko4gs9X7JbheRWSJypd9xJUdEJojIc+nYP1pE7ktQ1kJEYtJyPFUtqKp/eB/bA1FACVW9LZhYxXlQRFaJyBERiRGRj0WkRlriCRWq+oOqVvU7DpM6lqBMlhCRvsArwPO4X6oXAW8BN/oYVqgrD6xT1dhU7PMq8BDwIFAcqAJ8Dlyf2pOLSK7U7mNMqqiqLbZk6gIUAQ4DtyWzzQTguYDPLYCYgM+bgUeBFcAR4B1copsFHAK+AYoltm/A/q2894OBSQHrPgb+Bg4A3wPVvfLuwCngpBf/DK/8AmAasAvYBDyYTL2igfsSlCWsm+ISxh/AbuAlIIe3rgswP8G2lYAhXlynvNh6JBZrgvNWBk4DDYONN4nz9wLWe3UfBQxPcIwvgL6p/a6C+HeUExjhfUebgN5ePLm89V2B37x/D38APVL499TP+/d0AJgK5PP7/4otZy92BWWywhVAPuCzdB7nVuBfuL/6/w+XnB4HSuFaAx5M43Fn4X55lwZ+Bj4AUNWx3vsX1TWt/Z+I5ABmAL8CZYFrgD4i0jqtlfLcDNQH6uKuKu9JbmNVHYS7Gp3qxTYmYayJ7HYN7pf04nTGehPQCKgGfAjcISICICLFgGuBKZnwXXUDrgNq476nmxKs3wm0AwrjktVIEambzPFuB9oAFYGauGRsshFLUCYrlAB2a+qaohLzuqruUNWtwA/AIlX9RVWP45JfnbQcVFXHq+ohVT2Bu7qqJSJFkti8AVBKVZ9R1ZPq7ge9DdyZlnMHeEFV96rqX7im0A7pPF5iSgDbM+A4Q71Yj+F+Dgo089a1Bxaq6jYy/ru6HXhVVWNUdR8wLHClqn6pqhvVmQfMDogrMa+p6jZV3YtLpLXTGJfJJNaGbLLCHqCkiORKZ5LaEfD+WCKfC6b2gCKSE/gvcBvuSizOW1US1/STUHngAhHZH1CWE/eLOjGxQO4EZblxzXGBtgS8/xPXNJbR9gBlMuA4Z2JVVRWRKbiE+j3QEYjvIRn0dyUiFwFrAo6b2M/yAs7+ngLfIyLXAYNwV9g5gALAymTq8XfA+6Nkzndu0sGuoExWWAic4NwmmUBHcL9Q4p2fjvOddSwvCZVKYtuOuCa1Vrh7ZRXid/NeEw73vwXYpKpFA5ZCqto2ieP/FXDMeBVxSSjQhQHvLwK2JXG85KQ0NcFcoJyI1E9mm2B+DgnP8yHQXkTK45r+pnnlQX9XqvqX1zRZMInkBO7qr1zA5zPfmYjk9c47HIhS1aLATP75OZoQZAnKZDpVPQA8DbwpIjeJSAERyS0i14nIi95my4G2IlJcRM4H+qTjlOuAfCJyvYjkBp4E8iaxbSFc8tyD+8X8fIL1O4CLAz4vBg6JyADvOaScInK5iDRI4vhTga4i0tDr4l0FeBiYkmC7R0WkmIhciOtlNzXIuiYX61lUdT2u5+SHXlf3PCKST0TuFJGB3mbLgVu8n1El4N6UTqqqv+A6LowDvlbV/d6q1H5XKfkIeEhEyopIUWBAwLo8uJ/xLiDWu5q6No3nMdmEJSiTJVR1BNAXlyx24f667o3r4gzwPu5m+mbcvYO0/IKOP9cBoCfuF+ZW3FVBUs8dvYe7mtmKa2L6KcH6d4BqIrJfRD5X1dO4G/G1cT3J4n8xJ3rPSlW/BgYC7+KaDGcCE4GxCTb9AliGSxBfeudNrbNiTWKbB4E3gDeB/cBGXAeNGd76kbiegDu8OD8I8tyTcVehk+MLUvtdBeFt3L+NFcAvuO8yFjitqoe8un0E7MNdGU9P43lMNiGqNmGhMX4SEQUqq+oGv2MJJd5V0mhVLe93LCZz2BWUMSYkeM2EbUUkl4iUxXWISO+jCyYbswRljAkVgntAeR+uie833L1NE6asic8YY0y2ZFdQxhhjsiVLUMYYY7IlS1AmrHnTXezzHuQMLEtyCgwRqSAimnC07oTTWYhIGRF5x5s65JCIrBWRISJyXiriKy0iH4rINm9epwUi0ihBXHHeFCXxS+eA9ZO88x8UkXWB9RKRxiIyR0T2isgub1qNMgHr84rIaBHZ4W0zw+t8kFLMd3vfT+C5REReEJE93vJC/Ph83vqcIvKcV89DIvKL9ywTXgyB9TshIoeC/Q5N+LIEZcKWiFTAjcWmwA0ZfOziuBEy8gNXqGoh3EC2RYFLUnGogsASoB5u+ouJwJciEjiawrbAURZUdWLAuqFABVUtjKvjcyJSz1tXDPe8VQXcsEOHcM9jxXsIN5BvTdwwP/uA11OodzHcAL2rE6zqjhsppJZ3vP/DjbAebwjQxDtfYaATcBxAVe9PMIrEh7gR5k2EswRlwtnduAdvJwCdk9801frifuHfpaqbAVR1i6o+pKorgj2Iqv6hqi+r6nZVPe2NoJ4HCGpyPVVd7Q1yCy4RK16CVNVZqvqxqh5U1aO4B3SbBuxeETfyww5vwN2pQPUUTjkUeA330G2gzsAIbyDXrbhpMbrAmaTWB+imqn96g7mu8s55Fu/q81ZcojYRzhKUCWd340ZC+ABoLSJRGXjsVsCnqhqX1AYissIb1SGx5a0k9qmNS1CBD+2W9prhNonIyIRNiCLylogcBdbixqubmURIV3H2lc87QFMRuUBECgD/xk09klR9GuKmBBmdyOrquJFA4v3KP8muBm7Eh/Yi8rfXFNkridPcihtp5Puk4jCRw0YzN2FJ3FTy5YGPVHW3iGzEDX8zMoNOkeLUFapaMzUHFJHCuCGfhnjDNYFLOrW91/K4K4uXCWg+U9WeIvIArvmsBW5swYTHrol7ZihwBuP1uCGntuImMlyJG34qsdhy4sbx662qcQG3l+IV5OzR3w8ABb37UOVwwxtVwV21VQbmisg6VZ2T4DidgffUnn8x2BWUCV+dgdmqGt8UNZl/mvlSmgIjNqAsqW0yauoKwI2SgBsP7ydVHRpfrqp/q+oaVY1T1U1Af9xVxlm85sH5uGTwnwTHroS7MnpIVQOnungTN8BqCeA84FOSvoLqCaxQ1YRjFcY7jLu3FK8wcNhLNMe8smdU9ZjXBDoFOGtUc3FTbrTAjY9ojCUoE368X/a3A829JqW/cSOI1xKRWqQ8BcZ2XCJKbptvgJvFzRqbVByrE/ROC1xGB2yXFzdobgxndyxIjJL8/9tcBHTSEDcFxjfAs6r6foJtawMTvMkHT+A6SDQUkZKJHPcaXH3jv88mwAgRecNbvxrXQSJeLf5pToy/Jxd4VZTYFVInYIE3saEx+D7nvC22ZPSCmzxvL25epfMDlu9xN+9b46YHb4gbPqcKbtic+wOO8SHuiqIE7sqpA2707yhvfXHcyOvvA+W9srK45reaqYg1N+7K6XMgVyLrW+Ka9gQ3/9F3wLveutK42WkL4iYCbI0buf2GgHg2Av2SOPe7uDmUinhxPA5sTWLbogm+yx9xHUWKeOvv977DsrgegasTfJ/fA2NwV2yXed//NQnO8Ttwj9//fmzJPovvAdhiS0YvwFe4HmUJy2/HzaKaC7jH+yV6ENchYSCQI2DbYvwzXcc+YAHQNMHxLgDGe8c8hLtPNAgokIpYm+OuJo7imsnil2be+r5eDEdx94teAwp560oB87zEeRB3D6lbwLEHeccOPO7hgPUlcB1IdnrHmA80DFg/C3g8ibijgfsCPgvwIu4Pg73eewlYX9b7uRwG/gB6JDjeFV5yLeT3vx9bss9iY/EZY4zJluwelDHGmGzJEpQxxphsyRKUMcaYbCnTEpSI5BORxSLyq9fddohXXlFEFonIBhGZKiJ5vPK83ucN3voKAcd6zCv/XURaB5S38co2iMjAzKqLMcaYrJdpnSS8J8jPU9XDIpIb10PoIVyvpE9VdYr3LMivqjpKRHriuufeLyJ3Ajer6h0iUg3X5bchrtfUN7huwQDrcAN0xuAG3OygqmuSi6tkyZJaoUKFNNfryJEjnHde0INVhyyrZ3iJlHpC5NQ1nOq5bNmy3apa6pwVWdFVECgA/Aw0wg0ymcsrvwI3WCXA17hRocF1A96N67r6GPBYwLG+9vY7s69XftZ2SS316tXT9Pjuu+/StX+osHqGl0ipp2rk1DWc6gks1UR+X2fqPShvDpjluOcs5uAeGtyvqvFDycTgno/Ae90C4K0/gHtO40x5gn2SKjfGGBMGMnWwWFU9DdT2Jib7DLg0M8+XFBHpjpuvhqioKKKjo9N8rMOHD6dr/1Bh9QwvkVJPiJy6RkI9s2Q0c1XdLyLf4ZrliopILu8qqRzuKXm81wuBGHEzmRbBDcgZXx4vcJ+kyhOefyxu4jbq16+vLVq0SHNdoqOjSc/+ocLqGV4ipZ4QOXWNhHpmZi++UgFTOufHdWb4DTeWWHtvs87AF9776fwz2nR74FuvbXI6cKfXyy9+qP7FuE4Rlb1egXlwY5JNz6z6GGOMyVqZeQVVBpjozSOTAzcvz/9EZA0wRUSeA37BTZqG9/q+iGzAjeV1J7gZQ0XkI2ANbhqEXl7TISLSG9dpIicwXlUTTkNtjDEmRGVaglI350udRMr/wHUZT1h+HLgtiWP9F/hvIuUzSXr2UGOMMSHMRpIwxhiTasePQ1xc5p7Dpnw3xpgIdOoUHDp07nL4cOLlgcuqVbB9O8BMrriiEbNnl6BgwYyP0RKUMcaEgNjYs5PHmjWFiI1NfXKJ3+bEieDOmysXFCp09nL55VC1Kixf/hcbNy7j+PGnLEEZY0yoOH3aJYPUJI3kluPHE56hXqLnzZHj3IRSqBCULu1eCxZMfH3gErhN3rwg4o79999/8+STT9KgQQN69OjB6dPdUFVyZVImsQRljDG4+ylHjqTtaiSx5ejR4M4rknjSKF8+6aRRqBBs2rSCpk1rnrNN/vz/JJSMcvz4cV555RX++9//cuLECeLHM82ZM2fGnigBS1DGmJCk6pJAwsSwcGEJ/v479cnlyBF3zGDEJ4vApFG2bMpXI4ktBQqkLaFER++lWbPU75das2fPpkePHmzevJmbbrqJF198kcqVK2f+ibEEZYzJIqqumSq9TV3x2xw+nFQvshrnlBQocG7SiIqCSpWCa+YKXM47zzWjhTtVRbzMWbhwYebOncvVV1+dpTFYgjLGJErV3UjPiOau+PWnTwd37nz5zk0MpUpBxYop3z/5/fdltGhR76xEk8ktUWFl27ZtPPHEE5x//vkMHTqUa6+9llatWpHDh6xsCcqYMHLyZOJJY+HCkmzenPrEEhub4ikByJPn3ERRrBhcdFHabsyn56Z7jhyHuOyytO8fqY4dO8bLL7/M0KFDOXnyJP379z+zzo/kBJagjAkJsbHwxx+wbp1b1q93y86dZyeUkyeTOsLlZ31KrOtwoUJwwQWpu38Sv02ePJn+FZhMNG/ePO6++27++usvbrnlFl588UUuueQSv8OyBGVMdnTiBCxZAvPmueXHH91N/HhFi0KVKnDJJcHdP1m7dglXX93grK7Dxpw+fZqcOXNSqlQpSpcuzcSJE7PVCOmWoIzJBo4dg59+csno++9h4cJ/nnupUQO6dIEGDVxSqlwZSpRIXc+vU6eOULFipoRuQtDWrVt57LHHOHHiBFOnTqVatWosXrz4TKeI7MISlDE+OHzYXRXFJ6TFi13zXI4cULs23H8/NG8OzZq5ZGRMRjh69CjDhw/nhRde4PTp0/Tt25e4uDhy5MiR7ZITWIIyJkscOADz57tkNG8eLFvm7ivlzAn16sFDD7mEdOWVUKSI39GacLR48WJuvfVWYmJiuO2223jhhReomM0vqy1BGZMJ9u6FH3745x7S8uXumZ3cuaFhQ+jf3yWkJk3IlDHMjIl3/Phx8uXLx8UXX0yVKlWYPHkyzbLiCd8MYAnKmAywc+c/V0fz5sHKla48Xz5o3BieesolpEaN3EOjxmS2LVu28Nhjj7F+/XoWLlxIyZIlmTt3rt9hpYolKGPSYNu2f5LRvHmwdq0rL1AAmjaFO+6Aq65yV0vWY85kpSNHjvDSSy/x4osvEhcXR79+/Th16hR5Q/AfoiUoY4Lw559nJ6SNG1154cLuvlGXLu4KqV4914xnjB9Wr15N69at2bp1K3fccQcvvPAC5cuX9zusNLMEZUwCqi4BxfewmzfPJShwoyNcdRX07OkSUu3aNoyO8d/+/fspWrQolSpVokmTJjz00EM0bdrU77DSzRKUiXiqrokuMCFt2+bWlSrlEtIjj7iEdPnlkTFQqAkNf/75JwMHDmTBggWsXbuWAgUK8NFHH/kdVoaxBGUiTlycm7J63jyYNq0av/3mOjkAlCnjElH8cumlGT+3jjHpdfjwYcaPH8/HH38McNa4eeHEEpQJe6dPu27e8VdHP/zguoEDREUVpnXrfxLSJZdYQjLZ25YtW2jUqBHbt2+nY8eODB06lIsuusjvsDKFJSgTduKb7GbOhG+/dQ/IHjzo1l1yCdx0k0tGV10Fmzf/lK3GHjMmKdu3b6dMmTKUK1eO9u3bU7VqVXr16uV3WJnKEpQJC8eOQXQ0fPmlWzZvduWXXgodOrhk1Ly5m/U0UPx2xmRXmzdvZsCAAXz55ZesW7eOCy64gNdee43o6Gi/Q8t0mZagRORC4D0gClBgrKq+KiKDgW7ALm/Tx1V1prfPY8C9wGngQVX92itvA7wK5ATGqeowr7wiMAUoASwDOqlqkhMOmPDy11//JKRvv3VJqkABuOYaGDgQ2raFCy/0O0pj0ubQoUMMGzaMESNGkCNHDgYMGECRCBsHKzOvoGKBR1T1ZxEpBCwTkTneupGqOjxwYxGpBtwJVAcuAL4RkSre6jeBfwExwBIRma6qa4AXvGNNEZHRuOQ2KhPrZHx0+jQsWPBPUlq92pVffDHcdx9cf727SsqXz984jUmvAwcOUK1aNbZt20anTp14/vnnKVeunN9hZblMS1Cquh3Y7r0/JCK/AWWT2eVGYIqqngA2icgGoKG3boOq/gEgIlOAG73jXQ109LaZCAzGElTY2bQJxo+HCRMgJsZNtnfVVXDPPe4qqWpV69hgwsP69eupXLkyRYoU4cEHH6Rly5Y0bNgw5R3DlKhq5p9EpALwPW5az75AF+AgsBR3lbVPRN4AflLVSd4+7wCzvEO0UdX7vPJOQCNcMvpJVSt55RcCs1T17KlD3bruQHeAqKioelOmTElzXQ4fPkzBCBjdMzvU8++/8zFxYnm+/vp8ABo02Evr1n/TqNFezjvvdIacIzvUMytESj0hNOu6bds2xowZww8//MCYMWOoXLlyivuEYj2T0rJly2WqWv+cFaqaqQtQEHd/6BbvcxTuXlIO4L/AeK/8DeCugP3eAdp7y7iA8k7etiVxV1bx5RcCq1KKp169epoe3333Xbr2DxV+1PPkSdV581Qfe0y1dm1VUM2bV/WRR1T/+itzzmk/z/ATSnU9cOCA9u/fX/PkyaMFChTQZ599Vo8ePRrUvqFUz5QASzWR39eZ2otPRHID04APVPVTLyHuCFj/NvA/7+NWL8nEK+eVkUT5HqCoiORS1dgE25sQceqUu580aRLMmeO6g+fM6QZcHToUOnaEMH3Ew0S4U6dOUbt2bTZt2kTnzp15/vnnueCCC/wOK1vJzF58grsK+k1VXw4oL6Pu/hTAzcAq7/10YLKIvIzrJFEZWAwIUNnrsbcV15Gio6qqiHyHu8KaAnQGvsis+piM9ddf8Pbb8M47sH07nH8+3H47XHed64UXYZ2VTARZunQp9erVI3fu3AwePJhq1apRv/65rVsmc3vxNcU1x60UkeVe2eNABxGpjet6vhnoAaCqq0XkI2ANrgdgL1U9DSAivYGvcU2D41XV67/FAGCKiDwH/IJLiCabiouDb76Bt96CGTPcA7Vt20L37u41lz2VZ8LYhg0bePTRR/n888+ZMWMG7dq14+677/Y7rGwtM3vxzcdd/SQ0M5l9/ou7L5WwfGZi+6nr2Re5XVxCRGwsvPkmvPEGbNjgBmAdONAlphCeCcCYoBw4cIDnnnuOV199lbx58/L888/TqlUrv8MKCfY3q8k0cXHuPtKYMbBlC1xxBQwZArfeapP4mcigqrRo0YJff/2Vrl278txzz1GmTBm/wwoZlqBMpune3d1jqlsXnn7aPUxrTCSIjo6mSZMm5MmTh2HDhlGqVCnq1q3rd1ghx2a2MRluyhQ3Bt4770CfPrB0qSUnExnWrVvHDTfcQMuWLRk/fjwArVu3tuSURpagTIZ67TU3OOvvv0Pr1vDMMzbKgwl/+/bto2/fvlSvXp3o6GiGDRtGly5d/A4r5FkTn8kwI0dC375uOouhQ6FKFZt91kSGO+64g2+++Yb77ruPZ599lqioKL9DCguWoEyGeOkl6N/fdYD48EPIndvviIzJXLNnz6ZevXqUKFGCYcOGkSNHDmrXru13WGHF/r416TZsmEtOt99uycmEv7Vr19KuXTtat27Na6+9BkDdunUtOWUCS1AmXZ57Dh57zN13+uADS04mfO3du5c+ffpQo0YNfvjhB1566SUef/xxv8MKa9bEZ9JsyBAYPBg6dYJ333Vj6BkTrh5++GEmTZpEt27deOaZZyhdurTfIYU9u4IyqabqnmsaPBi6dLHkZMLXrFmzWLduHQBDhgzhl19+YfTo0ZacsoglKJMqqvDEE/Dss3Dvve5ZJ0tOJtysWbOG6667jrZt2/Lyy26s6woVKlCzZk2fI4sslqBM0FRhwADXhbx7dxg71rqRm/Cye/duevfuTc2aNVm4cCEvv/zymY4QJuvZPSgTFFV45BH3rFPPnvD665acTPgZMWIEo0ePpkePHgwZMoSSJUv6HVJEswRlUqTqhix67TV48EF45RUbHcKEB1Vl5syZFC1alKZNmzJgwADuuusuqlev7ndoBmviMymIi4PevV1yevhhS04mfKxatYrWrVvTrl27M/eZihYtaskpG7EEZZIUF+ea8956C/r1gxEjLDmZ0Ldr1y569uxJrVq1WLJkCa+88gpTpkzxOyyTCGviM4mKi4MePWDcODe54PPPW3Iy4eHjjz9m7Nix9OrVi0GDBlGiRAm/QzJJsARlznH6NLz0UlW++gqefNJGJDehTVWZPn06sbGx3HrrrXTr1o2rr76aSy+91O/QTAqsic+c5fRp6NoVvvqqDIMHu+edLDmZULVixQpatWrFTTfdxBtvvAFA7ty5LTmFCEtQ5ozYWLj7bnj/fejadRODBvkdkTFps3PnTnr06EGdOnVYvnw5r7/+OrNnz/Y7LJNK1sRnAJec7roLpk5195uuuOJPoKLfYRmTJkuWLGH8+PE88MADPP300xQvXtzvkEwaWIIynDoFHTvCJ5/Aiy/Co49CdLTfURkTPFXl888/JyYmhho1atC2bVs2btzIRRdd5HdoJh2siS/CnTwJd9zhktPLL7vkZEwoWb58OVdffTW33HILEydO5PTp04iIJacwkGKCEpFbRGS9iBwQkYMickhEDgax34Ui8p2IrBGR1SLykFdeXETmeMecIyLFvHIRkddEZIOIrBCRugHH6uxtv15EOgeU1xORld4+r4nY7fzUOHECbrsNPvsMXn3VPYhrTKjYsWMH3bp1o27duqxcuZK33nqLn376iZw2enHYCOYK6kXgBlUtoqqFVbWQqhYOYr9Y4BFVrQY0BnqJSDVgIDBXVSsDc73PANcBlb2lOzAKXEIDBgGNgIbAoPik5m3TLWC/NkHEZYDjx9307NOnwxtvuCGMjAklO3bsYNKkSTz88MNs2LCB//znP+TKZXctwkkwP80dqvpbag+sqtuB7d77QyLyG1AWuBFo4W02EYgGBnjl76mqAj+JSFERKeNtO0dV9wKIyBygjYhEA4VV9Sev/D3gJmBWamONNKpuevYvv4TRo90DucZkd6rKtGnTWLJkCS+88AI1a9Zky5YtNqBrGAsmQS0VkanA58CJ+EJV/TTYk4hIBaAOsAiI8pIXwN9AlPe+LLAlYLcYryy58phEyhM7f3fcVRlRUVFEp6MHwOHDh9O1f3bw7belmTGjGj17bqBq1ZhEO0SEQz2DYfUMDevWrePNN99kxYoVXHzxxbRs2ZJ8+fIlum2o1zVYkVDPYBJUYeAocG1AmQJBJSgRKQhMA/qo6sHA20SqqiKiwYebNqo6FhgLUL9+fW3RokWajxUdHU169vfb0aNuiva6deG11yqRM2elRLcL9XoGy+qZve3atYsBAwYwYcIESpYsyZgxY7j33nuTvc8UqnVNrUioZ4oJSlW7pvXgIpIbl5w+CLji2iEiZVR1u9eEt9Mr3wpcGLB7Oa9sK/80CcaXR3vl5RLZ3iTjhRcgJgY+/NBmwjXZX1xcHDNmzKBfv3488cQTFClSxO+QTBZKMkGJSH9VfVFEXsddMZ1FVZO9re71qHsH+E1VXw5YNR3oDAzzXr8IKO8tIlNwHSIOeEnsa+D5gI4R1wKPqeper1dhY1zT4d3A6ylXOXItWeIS1J13wpVX+h2NMedSVT7++GM+++wzJk+eTFRUFJs3b+a8887zOzTjg+SuoOI7RixN47GbAp2AlSKy3Ct7HJeYPhKRe4E/gdu9dTOBtsAGXJNiVwAvET0LLPG2eya+wwTQE5gA5Md1jrAOEknYtw9uuglKl4bhw/2OxphzLV26lD59+rBgwQJq1arFrl27KF26tCWnCJZkglLVGd7rxLQcWFXnA0k9l3RNItsr0CuJY40HxidSvhS4PC3xRRJVeOAB2LYNFi2Csol2JTHGH3v37uXhhx/mvffeo3Tp0rz99tt07drVnmcyKd+DEpFSuG7g1YAz3WZU9epMjMtkoJdfhg8+cCOTN2zodzTGnC1//vwsWrSIAQMG8Pjjj1O4cDCPWZpIEEwvvg+AqcD1wP24+0a7MjMok3F69ICxY+GWW+CJJ/yOxhh3n2nKlCmMGjWK2bNnkz9/flauXEnu3Ln9Ds1kM8GMJFFCVd8BTqnqPFW9B7CrpxDw448uOXXoAOPH27xOxn+LFi2iadOmdOzYkcOHD7N9u3sk0pKTSUwwCeqU97pdRK4XkTqAjV0fAp56ynWKePttsN65xk9HjhyhU6dONG7cmE2bNjF+/HiWLFlCxYo2pYtJWjBNfM+JSBHgEVw37sKADSuazUVHw7ffuvtP1gnK+EVVEREKFCjA1q1beeKJJxgwYACFChXyOzQTAlK8glLV/6nqAVVdpaotVbWeqk7PiuBM6gwbBhdeCBs3wpNPwgUXwP33+x2ViURxcXFMmjSJ6tWrs2PHDkSEb775hueee86SkwlaML34Xkuk+ACwVFW/SGSd8cGKFfDYY+59JW/0onHjIH9+/2IykWnhwoX06dOHxYsXU69ePfbu3UtUVBQ5ctj0cyZ1gvkXkw+oDaz3lpq4YYXuFZFXMi0ykyoPPAAlS8Ldd7vPpUvDPff4G5OJLLGxsfz73/+mSZMmbNmyhQkTJrB48WIuu+wyv0MzISqYe1A1gaaqehpAREYBPwBXAiszMTYTpLfegu+/hz59YOBAWLvWXU1Zrz2TFWJjY8mVKxe5cuUif/78PPXUU/Tv35+CBQv6HZoJccEkqGJAQVyzHsB5QHFVPS0iJ5LezWSFr76CXr2gVCn3nFPJkm60CGMyW1xcHO+//z5PPfUUX375JTVq1GDcuHF+h2XCSLAz6i4XkXdFZALwC/CSiJwHfJOZwZmUjRoFJUrAhg0uORmTFebPn0+jRo3o0qULZcqUIS4uzu+QTBgKphffO0AT3ISFnwFXquo4VT2iqo9mcnwmCQcP/jNle7duYKPDmKygqnTp0oVmzZqxfft2Jk2axMKFC6lVq5bfoZkwFEwTX/z07dZjL5tQhbvughkzoGdPGDTI74hMuDt69Cj58+dHRKhcuTKDBg3i0UcftZHGTaayfp8haMIEl5w6doQ334QkZr42Jt3i4uJ49913ueSSS5g1y81m88QTTzB48GBLTibTWYIKMX/9BQ89BM2bw/vv+x2NCWfff/89DRo04J577qFChQpERUX5HZKJMEElKBG5UkS6eu9LiYgNoOWD2Fg3MsTJk/Duu2DPPZrM0qtXL5o3b86uXbuYPHkyP/74I/Xq1fM7LBNhghlJYhBQH6gKvAvkBibhZsw1WeTUKbjtNpg1y3UntzE2TUY7ePAg+fPnJ3fu3DRu3Jjzzz+fRx55hAIFCvgdmolQwfwNfjNwA3AEQFW3ATaYVhZ75RX44gt3BfXcc35HY8LJ6dOnGTduHJUrV2bMmDEAdOrUiaeeesqSk/FVMAnqpDcduwJ4zz+ZLDZ1qusM8VpiIyMak0bR0dHUq1ePbt26UalSJRo3bux3SMacEUyC+khExgBFRaQb7uHctzM3LBNo0yZYtsxN2W7zupmMMnDgQFq2bMn+/fuZOnUq8+fPp379+n6HZcwZKd6DUtXhIvIv4CDuPtTTqjon0yMzAMTFQffuLjHddpvf0ZhQd+DAAUSEwoUL06ZNGwoXLszDDz9Mfhv23mRDKV5BiUhfYI2qPqqq/Sw5Za0ff4RvvoERI6B8eb+jMaHq9OnTjB07lsqVKzNkyBAAWrRoweOPP27JyWRbwYwkUQiYLSJ7ganAx6q6I3PDMvHmzHHdyTt18jsSE6rmzp1L9+7d+eOPP2jWrBkdO3b0OyRjghLMWHxDVLU60AsoA8wTkRQHiRWR8SKyU0RWBZQNFpGtIrLcW9oGrHtMRDaIyO8i0jqgvI1XtkFEBgaUVxSRRV75VBHJk4p6hwRV+OwzaNgQihb1OxoTioYOHUqrVq04evQoH3/8MfPmzbPnmUzICGosPs9O4G9gD1A6iO0nAG8A7yUoH6mqwwMLRKQacCdQHbgA+EZEqnir3wT+BcQAS0RkuqquAV7wjjVFREYD9wKjUlGfbG/+fFi5Et55x+9ITCjZv38/x44do0yZMtxyyy2ICHXr1uXaa6/1OzRjUiWYe1A9RSQamAuUALqpas2U9lPV74G9QcZxIzBFVU+o6iZgA9DQWzao6h+qehKYAtwoIgJcDXzi7T8RuCnIc4WMCROgYEG44w6/IzGhIDY2llGjRlG5cmV69eoFQNWqVRk4cCB58oRdA4OJAMFcQV0I9FHV5Rl0zt4icjewFHhEVfcBZYGfAraJ8coAtiQob4RLlPtVNTaR7c8hIt2B7gBRUVFER0enOfjDhw+na/9gHTuWgylTmnDVVbtYsuT3TD9fQllVT7+FSz2XLFnCW2+9xebNm6lVqxZt27Y9q17hUs9gREpdI6KeqproAhT2XosntiS1X4JjVABWBXyOAnLirtz+C4z3yt8A7grY7h2gvbeMCyjv5G1bEndlFV9+YeB5klvq1aun6fHdd9+la/9gxMWp9uunCqrR0Zl+ukRlRT2zg3Co56hRoxTQiy++WD/99FONi4s7Z5twqGewIqWu4VRPYKkm8vs6uSuoyUA7YBluFAkJzGvAxWlIhmd6/4nI28D/vI9bvSQTr5xXRhLle3APDudSdxUVuH3IW7YMhg+HFi3gqqv8jsZkR/v27WPnzp1UrVqV2267jaNHj9KrVy/y5s3rd2jGZJgk70GpajvvtaKqXuy9xi+pTk4AIlIm4OPNQHwPv+nAnSKS1xspvTKwGFgCVPZ67OXBdaSY7mXc73BXWACdCaMJFYcOdb32Pv0URFLc3ESQU6dO8cYbb1CpUiXuuusuVJUSJUrQt29fS04m7ATTSWJuMGWJbPMhsBCoKiIxInIv8KKIrBSRFUBL4GEAVV0NfASsAb4Ceqnqae/qqDfwNfAb8JG3LcAAoK+IbMDdkwqLvm7798P//gddukCxYn5HY7KTr776ilq1avHAAw9Qq1Ytxo0bh9hfMCaMJdnEJyL5gAJASREpxj9NfIVJpkNCPFXtkEhxkklEVf+Luy+VsHwmMDOR8j9wvfzCypQpbr4n67lnAk2bNo327dtTqVIlPv/8c2644QZLTibsJXcPqgfQB/dc0jL+SVAHcR0VTAY7dQpeeME9mNuokd/RGL/t2bOHDRs20KhRI/7v//6P0aNH07VrV+sybiJGkglKVV8FXhWRB1T19SyMKWK99BJs3gyvv273niLZqVOneOuttxgyZAgFCxbkjz/+IE+ePPTo0cPv0IzJUsGMZv66iFwOVAPyBZQnHCHCpMOOHS5BXX+9W0zkUVVmzpzJI488wu+//06rVq0YOXIkuXKlZsAXY8JHsFO+t8AlqJnAdcB8zh3CyKRD375w/DgMG2ZXT5Fq/vz5tGvXjipVqjBjxgyuv/56u89kIlowExa2B64B/lbVrkAtoEimRhVhtm2Djz6C//wHLr/c72hMVtq9ezczZ7o+QFdeeSUffvghK1eupF27dpacTMQLJkEdU9U4IFZECuMGjb0whX1MKowZA6dPgzd8mokAJ0+eZOTIkVSqVIkOHTpw6NAhRIQ777zTOkEY4wkmQS0VkaK4ad6XAT/jnm8yGeDkSZeg2raFSy7xOxqT2VSVGTNmcPnll9O3b18aN27MwoULKVSokN+hGZPtBNNJoqf3drSIfIUbo29F5oYVOT7+2HWQ6N3b70hMVli/fj033ngjVatWZebMmVx33XV+h2RMtpXcg7p1k1unqj9nTkiRIzYW7rrLvbepesLXrl27mDlzJp07d6ZKlSrMnj2b5s2bkzt3br9DMyZbS+4KakQy6xQ3H5NJh3nz3OvQoW5adxNeTpw4weuvv86zzz7LsWPHaNWqFWXLlqVVq1Z+h2ZMSEjuQd2WWRlIJPr0UyhQAB580O9ITEZSVb744gv69evHxo0badu2LcOHD6ds2RRHCDPGBAjmOai7Eyu3B3XTRxW++QaaNXNJyoSP3bt3c9ddd3HRRRcxa9Ys2rRp43dIxoSkYB5RbxDwPh/umaifsQd10+Xtt2HdOhgwwO9ITEbYsWMHEydO5NFHH6VUqVLMmzePWrVq2SgQxqRDML34Hgj87HU5n5JZAUWKOXPgggvctBomdJ04cYJXX32V5557jmPHjtGmTRtq1qxJvXr1/A7NmJCXllvzR4CKGR1IpPnlF7jiCuscEapUlWnTpnHZZZcxYMAAWrRowerVq6lZs6bfoRkTNoK5BzUD12sPXEKrhptc0KTRnj2wcSN07ep3JCatjh8/Tp8+fShatChz5syxnnnGZIJgGsiHB7yPBf5U1ZhMiicijBzpXu3Zp9Cyfft2XnnlFZ555hny58/Pt99+S8WKFe0+kzGZJMUGJlWdp6rzgF9w064fFZHimR5ZmFKFL76Aq66CBg1S3t747/jx4wwdOpQqVaowcuRIFi50I31VrlzZkpMxmSjFBCUi3UXkb2AFsBQ3Ht/SzA4sXH3/PaxaBf/+t9+RmJSoKh9//DGXXnopjz/+OK1atWL16tW0aNHC79CMiQjB/Pn3KHC5qu7O7GAiwZQpUKgQdOrkdyQmJarKiy++SJEiRZg7dy5XX22DpxiTlYLpQ7YROJrZgUSKBQugSRPIn9/vSExitm3bxv3338/u3bvJkSMH06dP5+eff7bkZIwPgrmCegz4UUQWASfiC1XVBuhJpQMHXPNe+/Z+R2ISOnbsGCNGjGDYsGGcOnWKNm3acNNNN1GmTBm/QzMmYgWToMYA3wIrgbjMDSe8TZvmOklcc43fkZhAU6dOpX///vz111/ceuutvPjii1x88cV+h2VMxAumiS+3qvZV1XdVdWL8ktJOIjJeRHaKyKqAsuIiMkdE1nuvxbxyEZHXRGSDiKwInOpDRDp7268Xkc4B5fVEZKW3z2sSAvNjjx0Ll13mmvhM9jFt2jSKFy9OdHQ0n3zyiSUnY7KJYBLULK8nXxkvwRQPspv5BCDhKJkDgbmqWhmY630GuA6o7C3dgVHgEhowCGgENAQGxSc1b5tuAftl6xE5e/SARYvg3nsh+6fS8BYTE0Pnzp1Zs2YNAG+//TZLly6lefPmPkdmjAkUTILqgHcfCtfFPKhu5qr6PbA3QfGNQPzV10TgpoDy99T5CSgqImWA1sAcVd2rqvuAOUAbb11hVf1JVRU3cO1NZFMxMe7qqUEDG3vPT0ePHmXIkCFUrVqVqVOnsmzZMgCKFClCzpw5fY7OGJNQMIPFZuS4e1Gqut17/zcQ5b0vC2wJ2C7GK0uuPCaR8kSJSHfclRlRUVFER0enuQKHDx9O9f5LlxYDatGx43JWrtyf5nNnpbTUMzv79ttvGT16NLt27aJ58+b06NGDMmXKhF09kxIp9YTIqWsk1NO3+aBUVUVEU94y/VR1LDAWoH79+pqeBy2jo6NT/aDmnDluUNi77qpNyZJpPnWWSks9s7NZs2Zx4YUXMm3aNJo1a3amPNzqmZRIqSdETl0joZ7BNPE1CFiaAYOBG9J4vh1e8xze606vfCtwYcB25byy5MrLJVKeLX36qeu5FyrJKRxs2bKFu+66i9mzZwMwZMgQlixZclZyMsZkb8GMxfdAwNINqAsUTOP5pgPxPfE6A18ElN/t9eZrDBzwmgK/Bq4VkWJe54hrga+9dQdFpLHXe+/ugGNlK3v2wNq1YM95Zo0jR44waNAgqlatyrRp0/jjjz8AyJcvHzlsbhNjQkpaRroMaj4oEfkQaAGUFJEYXG+8YcBHInIv8Cdwu7f5TKAtsAE3akVXAFXdKyLPAku87Z5R1fiOFz1xPQXzA7O8Jdv5/Xf3atMEZb5PPvmEhx56iG3btnHnnXcybNgwypcv73dYxpg0yrT5oFS1QxKrznlM1euJ1yuJ44wHxidSvhS4PKU4/Pbbb+61UiV/4whnqoqIsHv3bsqWLctHH31E06ZN/Q7LGJNONh9UJoqLg1degYsvhksu8Tua8PPnn38yYMAAWrZsSY8ePejWrRvdu3e3pjxjwkSSCUpEKuG6hc9LUN5URPKq6sZMjy7ErV7txt57+22wx2wyzuHDhxk2bBgjRoxARGjYsCGAPctkTJhJ7k/NV4CDiZQf9NaZFCxY4F5btvQ3jnDyxRdfUKVKFf773/9y66238vvvv9O3b1+/wzLGZILkmviiVHVlwkJVXSkiFTIvpPAxfz5ERbkmPpM+cXFx5MiRg7x583LRRRfx6aef0rhxY7/DMsZkouSuoIoms85mM0rByZMwYwZce62NvZcemzZt4vbbb+fxxx8HoE2bNixcuNCSkzERILkEtVREuiUsFJH7cOPxmWSsWgUHD0K7dn5HEpoOHTrE448/zmWXXcaXX35J0aJFz6wLgYHrjTEZILkmvj7AZyLyb/5JSPWBPMDNmRxXyFu82L3WrZv8duZcs2fP5u6772bHjh106tSJ559/nnLlyqW8ozEmrCSZoFR1B9BERFryz/NGX6rqt1kSWYj76isoX966l6fGqVOnyJ07N2XLlqVq1apMnz79TA89Y0zkCWY08++A77IglrBx/Dh88w106mT3n4KxceNG+vfvT65cuZg6dSrVq1dn3rx5Ke9ojAlr9kRjJvj6azhyBG62htBkHTx4kAEDBlCtWjW++uoratSogRtUxBhj0jYWn0nBJ59AsWL2/FNyfvjhB9q3b8/OnTvp3Lkzzz//PBdccIHfYRljshG7gspgJ07A9Olw002QO7ff0WQ/R44cAeDSSy+lfv36LFmyhAkTJlhyMsacw66gMtjXX7vu5bfd5nck2cv69et59NFH2b59OwsXLqRUqVJ8+eWXfodljMnG7Aoqg33wAZQoAa1a+R1J9rB//3769etH9erVmTt3LjfffDOnT5/2OyxjTAiwK6gMtGcPfP45/Oc/1rwHsHz5cv71r3+xZ88e7rnnHp577jnOP/98v8MyxoQIu4LKQLNmuSGOOnXyOxJ/7d69G4DLLruM1q1bs3TpUsaNG2fJyRiTKpagMtDixXDeeVC7tt+R+GPdunXccMMN1K1bl6NHj5I3b14mTZpEXRtOwxiTBpagMtC8eW5oo0iblmjfvn307duX6tWrEx0dTe/evW1uJmNMutk9qAzyww+wYgUMHOh3JFlr06ZNNGjQgL1793Lffffx7LPPEhUV5XdYxpgwYAkqg3zxhXuNlAT1559/Ur58eSpUqEDnzp3p1KkTtSO1bdMYkymsiS+DrFoF9epBkSJ+R5K51q5dy/XXX0/16tXZvn07IsKIESMsORljMpwlqAyybh1Urux3FJln7969PPTQQ9SoUYP58+czZMgQihcv7ndYxpgwZk18GWDTJrf07Ol3JJlj7969VKlShX379tG9e3eGDBlC6dKl/Q7LGBPmfLmCEpHNIrJSRJaLyFKvrLiIzBGR9d5rMa9cROQ1EdkgIitEpG7AcTp7268Xkc5+1AVgwQL3et11fkWQOVatWgVA8eLFeeyxx1i+fDmjRo2y5GSMyRJ+NvG1VNXaqlrf+zwQmKuqlYG53meA64DK3tIdGAUuoQGDgEZAQ2BQfFLLaitXQp48UKWKH2fPeGvWrKF///7UrFmTX3/9FYBHHnmEGjVq+ByZMSaSZKd7UDcCE733E4GbAsrfU+cnoKiIlAFaA3NUda+q7gPmAG2yOGbAdZC49NLQH95o9+7d9O7dm5o1a7JmzRpGjBjBZZdd5ndYxpgI5dc9KAVmi4gCY1R1LBClqtu99X8D8Q/TlAW2BOwb45UlVZ7lVq6EK6/048wZ58SJE9SqVYsdO3bQo0cPrr32Wm688Ua/wzLGRDC/EtSVqrpVREoDc0RkbeBKVVUveWUIEemOax4kKiqK6OjoNB/r8OHDZ+2/d28etmxpQtGiG4iOjklnpFlLVVm1ahWXX345IkKXLl2oVKkSFStWPKee4crqGX4ipa4RUU9V9XUBBgP9gN+BMl5ZGeB37/0YoEPA9r976zvgrr5IbLuklnr16ml6fPfdd2d9/uwzVVBdsCBdh81yK1eu1H/9618K6MyZM89Zn7Ce4crqGX4ipa7hVE9gqSby+zrL70GJyHkiUij+PXAtsAqYDsT3xOsMeGMzMB242+vN1xg4oK4p8GvgWhEp5nWOuNYry1ILF7p7T6EyHuquXbvo2bMntWrVYunSpbz66qu0ssmrjDHZkB9NfFHAZyISf/7JqvqViCwBPhKRe4E/gdu97WcCbYENwFGgK4Cq7hWRZ4El3nbPqOrerKuG89NPUKcO5MuX1WdOPVWlefPmrFu3jl69ejFo0CBKlCjhd1jGhLxTp04RExPD8ePHs+ycRYoU4bfffsuy82WEfPnyUa5cOXIH2aMsyxOUqv4B1EqkfA9wTSLlCvRK4ljjgfEZHWOwTp2CJUuge3e/IkiZqvLVV19xzTXXkCdPHl599VXKlStnvfOMyUAxMTEUKlSIChUq4P3xnekOHTpEoUKFsuRcGUFV2bNnDzExMVSsWDGofbJTN/OQs3IlHDsGjRv7HUniVqxYQatWrWjbti3vv/8+AP/6178sORmTwY4fP06JEiWyLDmFIhGhRIkSqbrKtASVDosWudfslqB27txJjx49qFOnDsuXL+eNN97g7rvv9jssY8KaJaeUpfY7srH40mHRIihdGsqX9zuSs91+++0sWLCABx98kKeffppixXwZYMMYY9LFrqDSYdEiaNgQ/P7DSVX57LPP2LvX9REZOXIkq1atYuTIkZacjDGJmjBhAr179wZg9OjRvPfee4CbUqd27drUqVOHjRs3MnnyZN9itASVRvv3w9q10KiRv3H88ssvtGzZkltuuYVRo0YBUKdOHapWrepvYMaYkHH//fefuQ3w+eef0759e3755Re2bNnia4KyJr40+ukn9+rX/ae///6bJ598kvHjx1OiRAlGjRrFfffd508wxpgz+vSB5csz9pi1a8MrryS/zebNm2nXrt2ZWQiGDx9+ZrSJWrVqMW/ePGJjYxk/fjwNGzY8a9/BgwdTsGBBqlWrxiuvvELOnDmZO3cux44d47fffqN27dp07tyZhx9+OGMrlgJLUGn07bfuAd0mTfw5f9++ffnkk0/o27cvTz75JEWLFvUnEGNMtnf06FGWL1/O999/zz333HMmiSXUtm1b7r//fgoWLEi/fv2Ijo5m+PDh/O9//8viiB1LUGn03XdwxRVQoEDWnE9VmTZtGrVq1aJy5coMHTqUIUOGUDmcp/E1JgSldKXjhw4dOgBw1VVXcfDgQfbv3+9vQEGye1BpsH8//PwztGyZNedbtmwZzZs357bbbuONN94AoHz58pacjDFn5MqVi7i4uDOfA583Sti9O1S6xFuCSoPvv4e4OLj66sw9z/bt27nnnnto0KABa9euZcyYMbz88suZe1JjTEiKiopi586d7NmzhxMnTpzVLDd16lQA5s+fT5EiRShSpEhQxyxUqBCHDh3KlHiDYU18afDjj+7+U2b34Bs5ciSTJk2iX79+PPHEE0H/ozLGRJ7cuXPz9NNP07BhQ8qWLcull156Zl2+fPmoU6cOp06dYvz44EeHq1mzJjlz5qRWrVp06dIlyztJ+D7dRlYvGTHdxs03q1atmq7DJCouLk6nTJmiP/zwg6qq7tu3T9evX5/xJwpCOA3lnxyrZ/jxo65r1qzJ8nMePHgwqO2aN2+uS5YsyeRogpfYd0V2mW4jHKxfDxl9+2fJkiU0a9aMO++8k9GjRwNQtGhRKlWqlLEnMsaYEGEJKpXi4mDjxoxLUFu3bqVz5840bNiQDRs2MG7cOCZOnJgxBzfGRLzo6Gjq16/vdxhpYvegUmn37rwcO5ZxCerTTz9lypQpDBw4kMcee4zChQtnzIGNMSbEWYJKpTVrXAJJ6wy6qsqUKVPIlSsXt912G/fffz/t2rULen4UY4yJFNbEl0q//lqU885LW4JatGgRTZo0oWPHjkyYMAFwPW8sORljzLksQaXSr78WoWlT1808WFu2bOGuu+6icePGbN68mXfffZcZM2ZkXpDGGBMGLEGlwt69sGlTQa66KnX7LV++nE8++YQnnniCdevW0aVLF3LksK/eGJN9VahQgd27d/sag92DSoXff3evtWsnv11cXByTJ09mz549PPTQQ7Rr145NmzZRpkyZTI/RGGPOPEcU4n8IW4JKhU2b3Gtyt4wWLlxInz59WLx4Mc2aNeOBBx4gR44clpyMiSAtWrQ4p+z222+nZ8+eHD16lLZt256zvkuXLnTp0oXdu3fTvn37s9ZFR0eneM7NmzfTunVrGjVqxLJly2jYsCErV67k2LFjtG/fniFDhgDuyqhz587MmDGDU6dO8fHHH3PppZeyZ88eOnTowNatW7niiitwz886L7/88pkRKO677z769OnD5s2badOmDY0bN+bHH3+kQYMGdO3alUGDBrFz504++OCDc6b1SK3QTq9ZLD5BVahw7rqYmBg6duxIkyZNiImJYeLEiURHR4f8XzDGmNCxfv16evbsyerVqxkxYgRLly5lxYoVzJs3jxUrVpzZrmTJkvz888/85z//Yfjw4QAMGTKEK6+8ktWrV3PzzTfz119/AW6w6nfffZdFixbx008/8fbbb/PLL78AsGHDBh555BHWrl3L2rVrmTx5MvPnz2f48OE8//zz6a6PXUGlwqZNUKzYSQoUyHPOur179zJ9+nSeeuop+vfvT8GCBX2I0BiTHSR3xVOgQIFk15csWTKoK6bElC9fnsbeLKofffQRY8eOJTY2lu3bt7NmzRpq1qwJwC233AJAvXr1+PTTTwH4/vvvz7y//vrrKVasGOAGmL355ps577zzzuz7ww8/cMMNN1CxYkVq1KgBQPXq1bnmmmsQEWrUqMHmzZvTVIdAIZ+gRKQN8CqQExinqsMy61ybN8P55x8H8hAXF8f777/PihUrGDFiBDVr1mTr1q02oKsxxjfxSWTTpk0MHz6cJUuWUKxYMbp06XLW9Bt58+YFIGfOnMTGxqb5fPHHAciRI8eZzzly5EjXcc8cM91H8JGI5ATeBK4DqgEdRKRaZp2vXj1o1GgP8+fPp1GjRnTp0oUFCxac+cFbcjLGZAcHDx7kvPPOo0iRIuzYsYNZs2aluM9VV13F5MmTAZg1axb79u0DoFmzZnz++eccPXqUI0eO8Nlnn9GsWbNMjT9eqF9BNQQ2qOofACIyBbgRWJMZJ+vTZzsdO95Ds2bRlC1blkmTJtGhQwe7z2SMyVZq1apFnTp1uPTSS7nwwgtp2rRpivsMGjSIDh06UL16dZo0acJFF10EQN26denSpcuZDg/33XcfderUyZAmvJSEeoIqC2wJ+BwDZNosTSLCr7/+yuDBg+nXr9+Zy2ljjPFbhQoVWLVq1ZnP8aPVJBSYWOrXr3/mfleJEiWYPXt2ovv07duXvn37Bn2+hOvSSgK7EoYaEWkPtFHV+7zPnYBGqto7wXbdge4AUVFR9aZMmZLmc+7du5fixYunPegQcfjw4Yjo6GH1DD9+1LVIkSJZPjXO6dOnyZkzZ5aeMyNs2LCBAwcOnFXWsmXLZap6zpDroX4FtRW4MOBzOa/sLKo6FhgLUL9+fU3sGYVgRUdHJ/qMQ7ixeoaXSKkn+FPX3377jUKFCmXpOQ8dOpTl58wI8bP7BiPUb54sASqLSEURyQPcCUz3OSZjTAQK5daorJLa7yikE5SqxgK9ga+B34CPVHW1v1EZYyJNvnz52LNnjyWpZKgqe/bsIV++fEHvE+pNfKjqTGCm33EYYyJXuXLliImJYdeuXVl2zuPHj6fql312kC9fPsqVKxf09iGfoIwxxm9+zOsWHR0d9L2cUBXSTXzGGGPClyUoY4wx2ZIlKGOMMdlSSD+omxYisgv4Mx2HKAn4O81k1rB6hpdIqSdETl3DqZ7lVbVUwsKIS1DpJSJLE3viOdxYPcNLpNQTIqeukVBPa+IzxhiTLVmCMsYYky1Zgkq9sX4HkEWsnuElUuoJkVPXsK+n3YMyxhiTLdkVlDHGmGzJEpQxxphsyRJUIkSkjYj8LiIbRGRgIuvzishUb/0iEangQ5gZIoi6dhGRXSKy3Fvu8yPO9BCR8SKyU0QSneJTnNe872CFiNTN6hgzQhD1bCEiBwJ+lk9ndYwZQUQuFJHvRGSNiKwWkYcS2SZcfqbB1DUsfq6JUlVbAhYgJ7ARuBjIA/wKVEuwTU9gtPf+TmCq33FnYl27AG/4HWs663kVUBdYlcT6tsAsQIDGwCK/Y86kerYA/ud3nBlQzzJAXe99IWBdIv9uw+VnGkxdw+LnmthiV1DnaghsUNU/VPUkMAW4McE2NwITvfefANeIiGRhjBklmLqGPFX9HtibzCY3Au+p8xNQVETKZE10GSeIeoYFVd2uqj977w/h5oIrm2CzcPmZBlPXsGUJ6lxlgS0Bn2M49x/EmW3UTZp4ACiRJdFlrGDqCnCr10zyiYhcmDWhZalgv4dwcIWI/Cois0Skut/BpJfXvF4HWJRgVdj9TJOpK4TZzzWeJSiTkhlABVWtCczhnytHE3p+xo15Vgt4Hfjc33DSR0QKAtOAPqp60O94MlMKdQ2rn2sgS1Dn2goEXiWU88oS3UZEcgFFgD1ZEl3GSrGuqrpHVU94H8cB9bIotqwUzM885KnqQVU97L2fCeQWkZI+h5UmIpIb9wv7A1X9NJFNwuZnmlJdw+nnmpAlqHMtASqLSEURyYPrBDE9wTbTgc7e+/bAt+rdrQwxKdY1Qbv9Dbg28HAzHbjb6/nVGDigqtv9Diqjicj58fdKRaQh7v9/yP1h5dXhHeA3VX05ic3C4mcaTF3D5eeaGJvyPQFVjRWR3sDXuF5u41V1tYg8AyxV1em4fzDvi8gG3E3pO/2LOO2CrOuDInIDEIuraxffAk4jEfkQ19OppIjEAIOA3ACqOhqYiev1tQE4CnT1J9L0CaKe7YH/iEgscAy4M0T/sGoKdAJWishyr+xx4CIIr58pwdU1XH6u57ChjowxxmRL1sRnjDEmW7IEZYwxJluyBGWMMSZbsgRljDEmW7IEZYwxJk1SGqA4ke1vDxj4dnJK21uCMiaAiKiIjAj43E9EBmfQsSeISPsMOtYib+Tqv+Ts0eYrZMTxEznf42nYp4uIvJEZ8ZhsYwLQJpgNRaQy8BjQVFWrA31S2scSlDFnOwHckt2exPdGLDlDVRupam3gadxo+rW9ZXNqjpMKqU5QJvwlNkCxiFwiIl+JyDIR+UFELvVWdQPeVNV93r47Uzq+JShjzhYLjAUeTrgi4RWQiBz2XluIyDwR+UJE/hCRYSLybxFZLCIrReSSgMO0EpGlIrJORNp5++cUkZdEZIk3KG+PgOP+ICLTgTUpBS4i/+ddWf0iIt+ISJRXPlhE3heRBbgHzEuJyByvmWWciPwZn5BF5C4v7uUiMsaLbRiQ3yv7IKntvPKuXt0W4x4yNZFnLPCAqtYD+gFveeVVgCoiskBEfhKRFK+8LEEZc643gX+LSJFU7FMLuB+4DPfkfxVVbYgbv/CBgO0q4KY5uR4YLSL5gHtxQ/E0ABoA3USkord9XeAhVa0SRAzzgcaqWgc3dUr/gHXVgFaq2gE3wsS3XjPLJ3ijEojIZcAduCaY2sBp4N+qOhA45l2h/Tup7cQNizUEl5iu9M5pIoi4QW2bAB97I1+Mwc1pBW7kosq40U46AG+LSNHkjmdDHRmTgKoeFJH3gAdxQ8cEY0n8WG8ishGY7ZWvBFoGbPeRqsYB60XkD+BS4FqgZsDVWRHcf+STwGJV3RRkDOWAqV6iyAME7jddVePrciVws1fXr0Rkn1d+DW4w4CXe0G75gcSaYZLarhEQraq7vO9hKu6vZhM5cgD7vT9cEorBTRx5CtgkIutw/86XJHcwY8y5XsFd2ZwXUBaL939GRHLgkkC8EwHv4wI+x3H2H4IJxxZT3KyvDwTcR6qoqvEJ7kgqYn4dN/txDaAHkC9gXTDHEWBiQBxVVXVwOrYzEcabCmSTiNwGbrBbEanlrf4cd/WE16RcBfgjueNZgjImEaq6F/gIl6Tibeaf6UZuwBuINZVuE5Ec3n2pi4HfcYP1/kfctAqISBUROS+5gyShCP9MKdE5me0WALd757oWKOaVzwXai0hpb11xESnvrTsVH18y2y0CmotICW/b29JQBxNCxA1QvBCoKiIxInIv8G/gXhH5FVjNP7N0fw3sEZE1wHfAo6qa7Kjr1sRnTNJGAL0DPr8NfOH9x/uK1F3dxPsLWAwUBu5X1eMiMg53b+pncW1mu4Cb0nDswbi2/33At0DFJLYbAnwoIp1wv1z+Bg6p6m4ReRKY7V0hngJ6AX/ibnyvEJGfvftQ52ynqj+J65K/ENgPLE9DHUwI8e5pJuacDhDeCOt9vSUoNpq5MRFGRPICp73pVq4ARiVxz8AYX9kVlDGR5yLgI+/q5yTu+RRjsh27gjLGGJMtWScJY4wx2ZIlKGOMMdmSJShjjDHZkiUoY4wx2ZIlKGOMMdnS/wPCq4hLSRDuQwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "perf_h2o.plot_uplift(metric=\"gain\")" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABAFklEQVR4nO3deXhU1fnA8e9LEkhIIASEiOwiVgFlC8ENGn+4txVRXFAUWil1QdparVZbQG3VKu5ilSIiuADaiqgobkTcARWVQFFA0CCCEBBCEiDk/f1x7oTJZJssk5nMvJ/nuU/m3nvunffM6Lyce889R1QVY4wxJtI0CXcAxhhjTEUsQRljjIlIlqCMMcZEJEtQxhhjIpIlKGOMMRHJEpQxxpiIZAnKxCQRmSwiT4U7jmCIyBgRec9vPV9EDvdeJ4nISyLyk4g8F74oG4Z9FrHFEpRpUCJysYgs935YNovIqyJyUrjjqoqIzBSRv9fh+GwRGRuwLUtEcmtzPlVNUdX13uoIIB1oo6rnBxOrOBNEZKWI7BGRXBF5TkSOqU084VTXz8JENktQpsGIyLXA/cDtuB+SzsAjwLAwhtXYdQG+UtXiGhzzAPB7YALQGjgSmA/8oqZvLiLxNT0mhGrzWZhIpqq22BLyBUgF8oHzqygzE/i733oWkOu3vgG4HvgC2AM8jkt0rwK7gTeBtIqO9Tv+FO/1ZOApv33PAT8APwFLgF7e9nHAfmCfF/9L3vbDgP8APwLfABOqqFc2MDZgW2DdFJcw1gPbgLuBJt6+McB7AWWPAG7x4trvxfa7imINeN8ewAEgM9h4K3n/q4Gvvbr/C5gScI4XgWtr+lkF8d9RvX0WtkT+Ekn/+jHR7XggEXihjuc5DzgViAc+A/oBlwOrgYW4H/lbanHeV4Hf4H7Q/gk8DfRV1WkicgIumfwVQESaAC/hfoRHAh2BN0VkjaouqkPdhgMZQAou2a4BpldWWFUniYgCR6jqKC+24/1jrcBQb//SOsQJcA4wCCgEBgJPi8j1qqoikgacBlwZws+qjFp+FibC2SU+01DaANu07pdfHlLVLaq6CXgX+FhVP1PVIlzy61ebk6rqDFXdrap7ca2rPiKSWknxgUBbVb1VVfepuwfyb+Ci2ry3n3+qap6qfou7FDqyjuerSBtgcz2c5w4v1kLc96DAYG/fCOBDVf2e0H1WJgZYC8o0lO3AISISX8cktcXvdWEF6yk1PaGIxAH/AM4H2gIl3q5DcJf8AnUBDhORnX7b4nA/1BUpBhICtiXgLkH5+87v9UbcpbH6th1oXw/nKY3VazXNwSXUJcDFgK+HZNCflYh0Blb5nbfG36WJLtaCMg3lQ2Av7tJQZfYAzf3WD63D+5U5l5eE2lZS9mJcR41TcPfKuvoO8/4GDvn/HfCNqrbyW1qo6lmVnP9bv3P6dMMlIX+d/F53Br6v5HxVqW56greAjiKSUUWZYL6HwPd5FhghIl1wl/7+420P+rNS1W/V9cpLqafkZFM1NHKWoEyDUNWfgInAVBE5R0Sai0iCiJwpInd5xVYAZ4lIaxE5FPhDHd7yKyBRRH4hIgnAX4FmlZRtgUue23E/zLcH7N8CHO63vhTYLSI3eM/exIlIbxEZWMn55wK/FpFMr4v3kcAfgTkB5a4XkTQR6YTrZTc3yLpWFWsZqvo1rufks15X96YikigiF4nIjV6xFcC53nd0BO4eX5VU9TNc547pwCJV3entqulnVZ+q/CxM5LMEZRqMqt4DXItLFj/i/nU9HtfFGWA28Dmut93r1O4H2vdePwFX4X4wN+FaBZU9dzQL15rZhLvE9FHA/seBniKyU0Tmq+oB4JdAX1yvNN8Pc4X3rLzOADcCT+AuGS4EngSmBRR9EfgElyBe8d63psrEWkmZCcDDwFRgJ7AO10HjJW//fbjOIlu8OJ8O8r2fwbVCn/FtqOlnVc+C+SxMBBNVawUbE25eD7Qeqro23LEYEymsBWWMMSYiWYIyxhgTkewSnzHGmIhkLShjjDERyRKUMcaYiGQJyhhKp8TYISLNArZVOk2GiHQVEQ0c0TtwmgcRaS8ij3vTi+wWkf+JyC0iklzDGLuKyGIRKfDOcUoVZZuJyAwR2SUiP3gjyfv2HScib4hInoj86E210d5v/2QR2e9NieJbyj1PJCKXefUfG7jPmPpgCcrEPBHpihtHToGz6/ncrXGjaCQBx6tqC9xgt62A7jU83bO4AXLbADcDz4tIZaNjTMaNXN4FOBn4s4ic4e1Lwz2D1dXbvxv3jJa/uf6jOujBOZd89UoDbgJyalgHY4JmCcoYuAz3cO5MYHQ9n/taXAIYpaobAFT1O1X9vap+EexJvNEn+gOTVLVQVf8DfIkb3b0io4HbVHWHqq7GDdA6xnv/V1X1OVXdpaoFuId2T6xhve4AHsQ9eGtMSFiCMsYlqKe95XQRSa/Hc58C/FdVSyorICJfeKMdVLQ84hXrBaxX1d1+h37ubQ88XxpuQNjPqyvrGUL5ltCvvEuAOSJyZcD5M3HTgjxaWZ2MqQ82mrmJaeKmm+8CzFPVbSKyDjd47H319BbVTm+hqscGcZ4Uyo+s/hPQoZKyvv3+ZVsEFhSRY3FjJPrPajwPdwlwC97AryKyU1Wf9QbdfQQYr6olIhJ4SmPqjbWgTKwbDbyuqr5LVc9w8DJfddNkFPttq6xMfU1vkQ+0DNjWEnf5sKKyvv2VlvUGgn0V+L2qlk5/oaqrVPV7VT2gqh/gpogf4e2+CvhCVQPHKzSm3lmCMjFLRJKAC4Cfez3dfsCNMt5HRPpQ/TQZm3GJqKoybwLDxc0sW1kcOQE95vwX32W0HOBwEfFvBfWhgk4KqrrDi61PZWXFTYvxJu4+1ezKYvOdkoNTjwz16uP7vE4A7hGRh6s5hzE115Dzy9tiSyQtuAn28nBzLx3qtywB7gFOB7YCmbgf6CNxU8tf4XeOZ4H/4i7lJXjn3Amke/tb40Znnw108bZ1AO4Fjq1hvB8BU4BE3OjjO3Gz1VZU9k7gHVyPvaNwCesMv/dfB1xXybHDvOPEq/smYLS3r1XAZ/UBriNIari/T1uibwl7ALbYEq4FeA24p4LtFwA/4O7R/gbX8tgFrMVNm9HEr2waB6f02AG8D5wYcL7DgBneOXcD/wMmAc1rGG9XIBs3c/Aa4BS/fZcAOX7rzbz33IW7l3St375JuFZRvv/it/9Z3KXJfC/WCVXElA2MDfd3aUt0LjYWnzHGmIhk96CMMcZEJEtQxhhjIpIlKGOMMRHJEpQxxpiIFNKRJLzBKR8A4oDpqnpnwP4hwP3AscBFqvq8377OuN5RnXA9js5SbyyzihxyyCHatWvXOsW7Z88ekpNrNMB0oxZL9Y2luoLVN9pFW30/+eSTbapafuDjUHUPxCWldcDhQFPcWGA9A8p0xSWnWcCIgH3ZwKne6xSq6ZI7YMAAravFixfX+RyNSSzVN5bqqmr1jXbRVl9guVbwux7KFlQmsFa9YfpFZA7uAcBVfslxg7evzECaItITiFfVN7xy+RhjjIkpoUxQHYDv/NZzcQNPBuNIYKeI/Bc3bMybwI2qesC/kIiMA8YBpKenk52dXaeA8/Pz63yOxiSW6htLdQWrb7SLlfpG6mjm8bgJ5PrhxkObi5vL5nH/Qqo6DTfqMhkZGZqVlVWnN83Ozqau52hMYqm+sVRXsPpGu1ipbygT1CZcBwefjt62YOQCK/wuD84HjiMgQRljTCTYv38/ubm5FBUVNcj7paamsnr16gZ5r/qUmJhIx44dSUgInACgYqFMUMuAHiLSDZeYLsLNsxPssa1EpK2q/gj8H7A8NGEaY0zd5Obm0qJFC7p27UpDzJG1e/duWrQoN71XRFNVtm/fTm5uLt26dQvqmJA9B6WqxcB4YBFuBOh5qpojIreKyNkAIjJQRHKB84HHRCTHO/YAcB3wloh8iRtV+d+hitUYY+qiqKiINm3aNEhyaqxEhDZt2tSolRnSe1CquhBYGLBtot/rZbhLfxUd+wauC7oxxkQ8S07Vq+lnZCNJeN56C04+OYs1a8IdiTHGGLAEVeruu93fnHLzkxpjTPSZOXMm48ePB+DRRx9l1qxZAPzvf/+jb9++9OvXj3Xr1vHMM8+ELUZLUJ6vv3Z/o2j0EGOMCcoVV1zBZZddBsD8+fMZMWIEn332Gd99911YE1SkPgfV4ObOhYEDYe/ecEdijDE1t2HDBn75y1+ycuVKAKZMmVL6QG+fPn145513KC4uZsaMGWRmZpY5dvLkyaSkpNCzZ0/uv/9+4uLieOuttygsLGT16tX07duX0aNH88c//rFB62QJypOU5P420GMMxpgo9Yc/wIoV9XvOvn3h/vtrf3xBQQErVqxgyZIl/OY3vylNYoHOOussrrjiClJSUrjuuuvIzs5mypQpvPzyy7V/8zqwS3yexET311pQxphoM3LkSACGDBnCrl272LlzZ3gDCpK1oDzNmrm/1oIyxtRFXVo6dREfH09JycFxt/2fNwrs3t1YusRbC8rja0FZgjLGNEbp6els3bqV7du3s3fv3jKX5ebOnQvAe++9R2pqKqmpqUGds0WLFuzevTsk8QbDWlAeu8RnjGnMEhISmDhxIpmZmXTo0IGjjjqqdF9iYiL9+vVj//79zJgxI+hzHnvsscTFxdGnTx/GjBljnSTCxS7xGWMauwkTJjBhwoQy27Kyshg1ahT3B1x7HDNmDGPGjAFcLz4f/9cJCQm8/fbbIYq2enaJzxMfD02aqLWgjDEmQlgLyiMCCQklFBbGhTsUY4ypN415YkNrQfmJj1f27w93FMYYY8ASVBnx8SUUF4c7CmOMMWAJqoy4OLUEZYwxESKkCUpEzhCRNSKyVkRurGD/EBH5VESKRWREBftbikiuiDwcyjh9LEEZY0zkCFmCEpE4YCpwJtATGCkiPQOKfQuMASobLvc2YEmoYgxkCcoYY5yuXbuybdu2sMYQyhZUJrBWVder6j5gDjDMv4CqblDVL4CSwINFZACQDrwewhjLiIuzThLGmMZPVcsMe9RYhbKbeQfgO7/1XGBQMAeKSBPgHmAUcEoV5cYB48AN81HX7pQiA/j++61kZ6+q03kaC99Q/LEgluoKVt+Glpqa2qBDAh04cKDc+23cuJHhw4eTkZHBihUrGDBgAKtWraKwsJBhw4Zx8803A9C7d29GjhzJa6+9xv79+5k1axZHHnkk27dv5ze/+Q2bN28mMzOTkpIS8vPzadasGQ8//DCzZ88G4LLLLuPqq69m48aNnHvuuQwcOJCPP/6Y/v37M2rUKG6//XZ+/PFHpk+fTkZGRrnYi4qKgv6uIvU5qKuAhaqaW9Wghqo6DZgGkJGRoVlZWXV606ZN82nduh1ZWe3qdJ7GIjs7m7p+Zo1FLNUVrL4NbfXq1bRo0aJ0vaJYLrjgAq666ioKCgo466yzyu33jeywbds2Rowoe0s+8Ad99+7dZd4PICUlhXXr1jF79myOO+448vLyaN26NQcOHGDo0KF88803HHvssYgIHTp0YMWKFTzyyCP861//Yvr06dx8881kZWUxceJEXnnlFWbNmkVKSgpfffUVzzzzDMuWLUNVGTRoEKeffjppaWmsX7+e//znP/Tq1YuBAwcyf/58PvzwQxYsWMADDzzA/Pnzy9XTN+xSMEJ5iW8T0MlvvaO3LRjHA+NFZAMwBbhMRO6s3/DKs3tQxpjGrEuXLhx33HEAzJs3j/79+9OvXz9ycnJYterglaFzzz0XgAEDBrBhwwYAlixZwqhRowD4xS9+QVpaGuAGmB0+fDjJycmkpKRw7rnn8u677wLQrVs3jjnmGJo0aUKvXr0YOnQoIsIxxxxTet66CGULahnQQ0S64RLTRcDFwRyoqpf4XovIGCBDVcv1AqxvlqCMMfWhqktYzZs3r3L/IYccUuvLlcnJyQB88803TJkyhWXLlpGWlsaYMWPKTL/RzBt8NC4ujuI6/Oj5zgPQpEmT0vUmTZrU6byl56zzGSqhqsXAeGARsBqYp6o5InKriJwNICIDRSQXOB94TERyQhVPMCxBGWOiwa5du0hOTiY1NZUtW7bw6quvVnvMkCFDeOYZ16H61VdfZceOHQAMHjyY+fPnU1BQwJ49e3jhhRcYPHhwSOP3Cek9KFVdCCwM2DbR7/Uy3KW/qs4xE5gZgvDKsV58xpho0KdPH/r168dRRx1Fp06dOPHEE6s9ZtKkSYwcOZJevXpxwgkn0LlzZwD69+/PmDFjyMzMBGDs2LH069evXi7hVSdSO0mEhbWgjDGNVdeuXVm5cmXp+syZMyss559YMjIySi8ntmnThtdfr/ipnmuvvZZrr7026PcL3FdbNtSRH0tQxhgTOSxB+bEEZYwxkcMSlB9LUMaY2lLVcIcQ8Wr6GVmC8mMJyhhTG4mJiWzfvt2SVBVUle3bt5OYmBj0MdZJwo/14jPG1EbHjh3Jzc3lxx9/bJD3KyoqqtEPfaRITEykY8cqO26XYQnKj7WgjDG1kZCQQLdu3Rrs/bKzs4MeLqgxs0t8fuLjLUEZY0yksATlJz6+hH37wh2FMcYYsARVRlLSAfbsCXcUxhhjwBJUGUlJB8jPB+uIY4wx4WcJyk9S0gFKSqCwMNyRGGOMsQTlJzHxAAD5+WEOxBhjjCUof82buwTVgDM3G2OMqYQlKD9JSdaCMsaYSBHSBCUiZ4jIGhFZKyLlZsQVkSEi8qmIFIvICL/tfUXkQxHJEZEvROTCUMbpY5f4jDEmcoQsQYlIHDAVOBPoCYwUkZ4Bxb4FxgDPBGwvAC5T1V7AGcD9ItIqVLH6JCS47ns23JExxoRfKIc6ygTWqup6ABGZAwwDVvkKqOoGb1+J/4Gq+pXf6+9FZCvQFtgZwniJj7cEZYwxkSKUCaoD8J3fei4wqKYnEZFMoCmwroJ944BxAOnp6aUzQ9bW/v3u41i+/AsSEvLqdK7GID8/v86fWWMRS3UFq2+0i5X6RvRgsSLSHpgNjFbVksD9qjoNmAaQkZGhWVlZdXq/r79eDsDRRx9LHU/VKGRnZ1PXz6yxiKW6gtU32sVKfUPZSWIT0MlvvaO3LSgi0hJ4BbhZVT+q59gqFB/vcqCNx2eMMeEXygS1DOghIt1EpClwEbAgmAO98i8As1T1+RDGWIbdgzLGmMgRsgSlqsXAeGARsBqYp6o5InKriJwNICIDRSQXOB94TERyvMMvAIYAY0Rkhbf0DVWsPtaLzxhjIkdI70Gp6kJgYcC2iX6vl+Eu/QUe9xTwVChjq0hcnF3iM8aYSGEjSfjxtaByc8MciDHGGEtQ/nz3oG67DYqKwhyMMcbEOEtQfnyX+ACWLw9jIMYYYyxB+fNd4gPYuTN8cRhjjLEEVUZc3MEEtWtXGAMxxhhjCcqfyMHXP/0UvjiMMcZYgqqUtaCMMSa8LEEFuOoq99daUMYYE16WoAJMnQpt28L27eGOxBhjYpslqAq0bw8//BDuKIwxJrZVm6BEpFkw26JJ+/aweXO4ozDGmNgWTAvqwyC3RY1DD7UEZYwx4VbpYLEicihuVtwkEekH+DphtwSaN0BsYZOWZp0kjDEm3Koazfx0YAxutPF7OJigdgE3hTas8EpNhd274cABiIsLdzTGGBObqkpQPVX1ZBG5QFXnNVhEEaBlS/c3P98lK2OMMQ2vqntQZ4mIADfW9uQicoaIrBGRtSJS7jwiMkREPhWRYhEZEbBvtIh87S2jaxtDbfgSlF3mM8aY8KmqBfUasANIERH/cRUEUFVtWdWJRSQOmAqcCuQCy0Rkgaqu8iv2Le4y4nUBx7YGJgEZgAKfeMfuCKpWdeRrNdloEsYYEz6VtqBU9XpVbQW8oqot/ZYW1SUnTyawVlXXq+o+YA4wLOA9NqjqF0BJwLGnA2+oap6XlN4AzqhBveokLc393batod7RGGNMoGqnfFfVYdWVqUQH4Du/9VxgUB2O7RBYSETGAeMA0tPTyc7OrlWgPvn5+WRnZ5OX1ww4nvnzvwK+r9M5I5mvvrEgluoKVt9oFyv1raqb+XuqepKI7MZdZhP/v0G2okJKVacB0wAyMjI0KyurTufLzs4mKysLVZgwAZYvP5J77z2SJlE63oavvrEgluoKVt9oFyv1reoS30ne3xb+l/ZqcIlvE9DJb72jty0YdTm2zkTg1lvh/ffhgw8a6l2NMcb4qzRBiUjrqpYgzr0M6CEi3USkKXARsCDIuBYBp4lImoikAad52xrMKae4v2vWNOS7GmOM8anqHtQnHLykF0iBw6s6saoWi8h4XGKJA2aoao6I3AosV9UFIjIQeAFIA34lIreoai9VzROR23BJDuBWVc2rWdXqpnNniI+Hr79uyHc1xhjjU2mCUtVudT25qi4EFgZsm+j3ehnu8l1Fx84AZtQ1htqKj4f+/eGdd8IVgTHGxLYovf1fP846Cz76yA139Pnn4Y7GGGNiiyWoKlxyiftbUgJ9+8JXX4U1HGOMiSmWoKpwxBFw/PEH1x96KHyxGGNMrAkqQYnISSLya+91WxGp8/2pxmLRIujVy73esye8sRhjTCwJZkbdScANwF+8TQnAU6EMKpK0aAHLl0OnTjb0kTHGNKRgWlDDgbOBPQCq+j3QIpRBRZrERDjySEtQxhjTkIJJUPtUVXHPPiEiyaENKTK1bQsffgi5ueGOxBhjYkMwCWqeiDwGtBKR3wJvAv8ObViRZ8gQ93fy5LCGYYwxMSOY0cyniMipuKnefwZMVNU3Qh5ZhLnySnjxRXc/yhhjTOhVm6BE5FpgbiwmpUD9+8Pdd8PevdCsWbijMcaY6BbMJb4WwOsi8q6IjBeR9FAHFan694fiYli5MtyRGGNM9Ks2QanqLaraC7gaaA+8IyJvhjyyCDRokJuK49VXwx2JMcZEv5qMJLEV+AHYDrQLTTiRrVMnN+TRkiXhjsQYY6JfMA/qXiUi2cBbQBvgt6p6bKgDi1SZmfDxx+4+lDHGmNAJpgXVCfiDN0/TZFVdFeqgItk558CuXfDaa+GOxBhjoltVM+r6pnW/G/i2FjPqIiJniMgaEVkrIjdWsL+ZiMz19n8sIl297Qki8qSIfCkiq0XkL+VOHiZDh8Ihh8CcOeGOxBhjoltVLahnvL+fAMu9v5/4rVdJROKAqcCZQE9gpIj0DCh2ObBDVY8A7gP+6W0/H2imqscAA4Df+ZJXuCUkwNlnuwS1dGm4ozHGmOhVaYJS1V96f7up6uHeX99S5XTvnkxgraquV9V9wBxgWECZYcCT3uvngaEiIrhhlZJFJB5IAvbhHhSOCCNGuL+DBtkcUcYYEyrBPKj7lqoOrW5bBToA3/mt5wKDKiujqsUi8hOuI8bzuOS1GWgO/FFV8yqIbRwwDiA9PZ3s7OzqqlOl/Pz8oM6RmAhZWT3Jzm5H//7FvPjie8TF1emtwyLY+kaDWKorWH2jXczUV1UrXIBEoDXwOZDmvW4NdAX+V9lxfsePAKb7rV8KPBxQZiXQ0W99HXAIcCLwNG5qj3bAGuDwqt5vwIABWleLFy+uUfm//lUV3HLppXV++wZX0/o2ZrFUV1Wrb7SLtvoCy7WC3/Wq7kH9Dne/6SjK3n96EXg4iNy3CdcD0Kejt63CMt7lvFTcc1YXA6+p6n5V3Qq8D2QE8Z4N6oorDr6ePRs2bgxfLMYYE22qugf1gKp2A67Tsveg+qhqMAlqGdBDRLqJSFPgImBBQJkFwGjv9QjgbS+bfgv8H5RO73Ec8L8a1awBdOgAZ555cP2DD8IXizHGRJtgRjN/SER643riJfptn1XNccUiMh5YBMQBM1Q1R0RuxTXnFgCPA7NFZC2Qh0ti4Hr/PSEiOYAAT6jqFzWvXugtXAj79kFyMnzxBYwcGe6IjDEmOgTTSWISkIVLUAtx3cbfA6pMUACqutA7xn/bRL/XRbgu5YHH5Ve0PVI1bepaUy+8AHfcEe5ojDEmOgQzksQIYCjwg6r+GuiDu1dk/OTmwpo18M034Y7EGGOiQzAJqlBVS4Bib3SJrZTt/GCA9993fw8/HHJywhuLMcZEg2AS1HIRaYWb5v0T4FPgw1AG1RgNGgTnexclR41ync+NMcbUXjDzQV2lqjtV9VHgVGC0d6nPBJg1CyZPhhUroG1bmD8/zAEZY0wjVmknCRHpX9U+Vf00NCE1XomJ8Ne/wiOPwNatMHy4taSMMaa2qurFd08V+xTvOSVTVlwcPP88DBni1k87DV5+2fX0M8YYE7xKE5SqntyQgUSTwYPdhIYTJsBjj8GMGWVHnTDGGFO9YJ6Duqyi7dU9qBvrmjaFf/3LPbx7ww0wYAAMHBjuqIwxpvEIphffQL9lMDAZODuEMUUNETdvVGqq6+G3Z0+4IzLGmMYjmF581/gtvwX6AymhDy06dO4MTz/tBpL9xz/CHY0xxjQewbSgAu0ButV3INFs8GC46CJ46CHYsiXc0RhjTONQbYISkZdEZIG3vIybm+mF0IcWXf72N9i/H8aPD3ckxhjTOFTbSQKY4ve6GNioqrkhiidq9ewJ118Pf/87PP44XH55uCMyxpjIFsx0G+8AeOPwxXuvW2sFU7Cbqk2aBIsXw9ix7lLfTTeFOyJjjIlcwVziGyciPwBfAMtx4/EtD3Vg0Sg+Hl56yY0wcfPN8Mor4Y7IGGMiVzCdJK4HeqtqV7+ZdQ8P5uQicoaIrBGRtSJyYwX7m4nIXG//xyLS1W/fsSLyoYjkiMiXIpIYeHxjlJYGzzwD7dvDn/4Ezz0X7oiMMSYyBZOg1gEFNT2xiMThZsY9EzfZ4UgR6RlQ7HJgh6oeAdwH/NM7Nh54CrhCVXvhJkzcX9MYIlViIkybBj/+CBdcAH36wJdfhjsqY4yJLMEkqL8AH4jIYyLyoG8J4rhMYK2qrlfVfcAcYFhAmWHAk97r54GhIiLAacAXqvo5gKpuV9UDwVSosfjlL919qLFj3WgTF14IL74Y7qiMMSZyiFYz3LaILMVN8f4lUOLbrqpPVnqQO24EcIaqjvXWLwUGqep4vzIrvTK53vo6YBAwChgAtAPaAnNU9a4K3mMcMA4gPT19wJw5c6qrb5Xy8/NJSWn4Z5CXLm3NTTf15sCBJjz00Kf07r2rQd43XPUNh1iqK1h9o1201ffkk0/+RFUzyu1Q1SoX4LPqylRy3Ahgut/6pcDDAWVWAh391tcBhwDXAd94r5vjJkgcWtX7DRgwQOtq8eLFdT5Hbe3cqXrYYapduqhu3dow7xnO+ja0WKqrqtU32kVbfYHlWsHvejCX+F71evK1F5HWviWI4zZRdmr4jt62Cst4951Sge1ALrBEVbepagGwEDfEUtRKTXWX+LZsgREj3GjoxhgTy4JJUCPx7kPhupgH2818GdBDRLqJSFPgImBBQJkFwGjv9QjgbS+bLgKOEZHmXuL6ObAqiPds1DIy3EO8S5a4zhP7o6ZbiDHG1FwwD+rWatw9VS0WkfG4ZBMHzFDVHBG5FdecWwA8DswWkbVAHi6Joao7ROReXJJTYKGqxsRTQxdfDDt3wtVXw29+A08+CU1qM2KiMcY0ciGdD0pVF+Iuz/lvm+j3ugg4v5Jjn8J1NY85V10FGzbA3XdDjx4wcWK1hxhjTNQJZiw+/2n2EoGhwKeATVgYQnfe6ZLUbbdBt25wySXWkjLGxJZgLvFd478uIq1wzzSZEGrSxD3M+8kncNll8PnnrkUlEu7IjDGmYdh8UBGsVStYsQJOPBHuuQd+/nPYsSPcURljTMMI5h7US7iOCuASWk9gXiiDMge1aAHZ2fDrX8NTT8EJJ8Cbb0KHDuGOzBhjQsvmg2oE4uNh9mz47W/htNPg9tth6tRwR2WMMaFV6SU+ETlCRE5U1Xf8lveBLiLSvQFjNJ4hQ9zU8U8+CT/9FO5ojDEmtKq6B3U/UNGgcLu8fSYMrrkG9uyBG26AghqPMW+MMY1HVQkqXVXLTQLhbesasohMlQYMgKFD4bHH3DNSN9wAq6J+jA1jTCyqKkG1qmJfUj3HYWrglVdg/nzo2xfuvdfNJ/VklWPLG2NM41NVglouIr8N3CgiY3Hj8ZkwadYMhg1zier77929qTFj4MwzYfXqcEdnjDH1o6pefH8AXhCRSziYkDKApsDwEMdlgtS2Lbz6Kjz8MNx6q2tV/fa3MHIkHH+8jT5hjGm8Kv35UtUtqnoCcAuwwVtuUdXjVfWHhgnPBKNpU7j2WvjqK5eYpk+Hk06C7t1hyhR7uNcY0zhV++9rVV2sqg95y9sNEZSpnXbtYOZM2LrVPTfVuTNcfz107OhGR1+zJtwRGmNM8OwCUBRq2RJGjYJ33oHPPnNzS02fDkcdBb/4BbzxBqhWfx5jjAknS1BRrm9feOIJ+PZbmDwZli93o1H07g0vv9yewsJwR2iMMRULaYISkTNEZI2IrBWRGyvY30xE5nr7PxaRrgH7O4tIvohcF8o4Y0F6Okya5BLVzJnuvtU99/yMTp3g5pth06ZwR2iMMWWFLEGJSBwwFTgTN8DsSBHpGVDscmCHqh4B3Af8M2D/vcCroYoxFjVrBqNHw6efwn33fcbgwXDHHdC1q5vNd+nScEdojDFOKFtQmcBaVV2vqvtwc0gNCygzDPA9Yvo8MFTEzXgkIucA3wA5IYwxZolA374/8cILsHatG0Lp5Zdh0CA3Yvq8eVBcHO4ojTGxTDREd8tFZARwhqqO9dYvBQap6ni/Miu9Mrne+jpgEFAEvAGcClwH5KvqlIC3QETGAeMA0tPTB8yZU7d5FPPz80lJSanTORqTwPru2RPHa68dyn//25Hvv0+iXbsizjlnE7/4xWZatmzc2SrWv9toZ/Vt3E4++eRPVDWj3A5VDckCjACm+61fCjwcUGYl0NFvfR1wCG6Kjwu8bZOB66p7vwEDBmhdLV68uM7naEwqq29xseqLL6qefLIqqCYlqV5xherq1Q0bX32y7za6WX0bN2C5VvC7HspLfJuATn7rHb1tFZYRkXggFdiOa0XdJSIbcCNa3CQi4zENIi4Ozj4b3n7bTTU/cqTrCXj00W44pUWLrJu6MSb0QpmglgE9RKSbiDQFLgIWBJRZAIz2Xo8A3vYS6mBV7aqqXXFTe9yuqg+HMFZTiWOPhccfd73/br3VTUF/xhnQq5d7tqqkJNwRGmOiVcgSlKoWA+OBRcBqYJ6q5ojIrSJytlfscaCNiKwFrgXKdUU3kaFdO/jb32DDBpg1C5KS3Jh/7dq5SRTnzoV9+8IdpTEmmgQz5XutqepCYGHAtol+r4uA86s5x+SQBGdqpVkzuPRSN1LFc8+5EdXfeMMlqEMPdaOqjxkDP/tZuCM1xjR2NpKEqRURN4TSk09Cbq4bUT0jA+6+2w2pdMIJ8O9/w66K5mQ2xpggWIIyddakibsv9dJL8N13cNddsHMnjBvnWlWjRsFbb9n9KmNMzViCMvWqfXs3gnpODnz8sRu14uWX4ZRToFs3+OtfbYp6Y0xwLEGZkBCBzEz4179g82Z45hl36e+OO1wPwL593eXA774Ld6TGmEhlCcqEXFKSe5Zq0SI3KO0DD0BiIvz5z27OqsGD4fbb3UjrdhnQGONjCco0qEMPhQkT4KOP3BiAt90Gu3e7EdUHDjzYbf3xx611ZUysswRlwqZ7d3dPasUK+OEHeOop+OUvYckSGDvWta6OPhp+/3t3Hys/P9wRG2MaUkifgzImWOnpcMklblF1nSxef909Y/Xvf8ODD0JCguu+ftppcOqp0L+/G5bJGBOdrAVlIo6Im/H32mvd81V5efDmm/DHP7rnqm6+2XXAaNcOLrzQXQ5cudJGsjAm2lgLykS8xEQYOtQt//wnbN3qEpavhTVvnisXHw89erhegr6ld2844ojwxm+MqR1LUKbRadfOzf578cXucuDq1e4+1sqV7tLgZ5/Bf/5zcMT1hATo2DGDzMyyyat7d5fUjDGRyf73NI2aCPTs6RZ/BQXwv/+5hJWTA0uWFLF0aQpz5x4s07SpezbLP2n16gWHH273toyJBJagTFRq3tx1oujf361nZ68kKyuLPXtci8uXuHJy4IMP4NlnDx6bmFhx4urWzQ3rZIxpGJagTExJTnaD2mYETC69e3f5xLVkCTz99MEySUmu27v//a1evVx3eEtcxtQ/S1DGAC1auJ6BmZllt+/a5cYOzMk5eI/rrbdg9uyDZZKT3SXGwBZXp07uEqQxpnZCmqBE5AzgASAOmK6qdwbsbwbMAgbgpnq/UFU3iMipwJ1AU2AfcL2qvh3KWI2pSMuWcNxxbvG3c2fZ1lZODrz2GsycebBMixYVJ64OHSxxGROMkCUoEYkDpgKnArnAMhFZoKr+Y1lfDuxQ1SNE5CLgn8CFwDbgV6r6vYj0xs3K2yFUsRpTU61awYknusVfXl75xPXyyzBjxsEyqakVJ6727S1xGeMvlC2oTGCtqq4HEJE5wDDAP0ENAyZ7r58HHhYRUdXP/MrkAEki0kxV94YwXmPqrHVrN/jt4MFlt2/bVj5xvfACTJ9+sExaWvnEddRRLnHZPS4Ti0R9D4vU94lFRgBnqOpYb/1SYJCqjvcrs9Irk+utr/PKbAs4zxWqekoF7zEOGAeQnp4+YM6cOXWKOT8/n5SUlDqdozGJpfpGYl1VYceOBDZsSPZbmrNhQzK7dyeUlouPL6Fdu720a1dEixbFtGy5n5SUYlq0cEtKyn5atjz4ukWLYlR3kpoaWfUNpUj8fkMp2up78sknf6KqGYHbI7qThIj0wl32O62i/ao6DZgGkJGRoVlZWXV6v+zsbOp6jsYklurbmOqq6gbPzclxI75v3NiEDRuSyM1NYvt2t23HDigqqvwcIkpqqpCWRpmldWuq3dayZeNrsTWm77c+xEp9Q5mgNgGd/NY7etsqKpMrIvFAKq6zBCLSEXgBuExV14UwTmMiioi7rNe+vZuJuDJFRS5R5eW5v/7Lp59uJDW1a5ltmzYdfF3VuIVNmrh7bDVNbGlprmOI3Ucz9SWUCWoZ0ENEuuES0UXAxQFlFgCjgQ+BEcDbqqoi0gp4BbhRVd8PYYzGNFqJiQcTWaDs7A1kZXWt8DhVKCysOLFVtm3jxoPrxcWVxxQXV30iqyzZNW9uyc2UFbIEparFIjIe1wMvDpihqjkiciuwXFUXAI8Ds0VkLZCHS2IA44EjgIkiMtHbdpqqbg1VvMbEChGXDJo3h44da3asKuzZUz6RVZbY8vJg3bqD26qaMTkhoXatttat6/Z5mMgV0ntQqroQWBiwbaLf6yLg/AqO+zvw91DGZoypORFISXFL5841O7akxI3YEWyrbcsWN57ijh3uubOq+nMlJAyhTZuaJTfferNmdfpITAhFdCcJY0z0aNLEPQOWmgpdu9bs2JIS+OmnyhPb55/n0qJF59Jtmza5kT927HDHVSUpqXattrQ01+ozoWMJyhgT8Zo0OZgcKpKdvZ6srIqbdMXFVSe3wPWNG930LXl5kJ9fdVzJyTW/15aW5jqh2FQv1bOPyBgT1eLjoU0bt9TU/v3u8mJ1lyN9r9etO7heUFD1uVu0qF2rLTW1Vh9Do2QJyhhjKpGQAG3buqWm9u0LPrHt2AFr1hzcVtUzbgDJySfRtm3NWm2N8Rk3S1DGGBMCTZtCerpbasr3jFtllyO//PIHkpM71uoZt9TUmrfawvWMmyUoY4yJMFU94waQnb2WrKzyzwj4nnELttWWl6esX7+bnTsL2bmzkAMHCoBCoD1wGLAL90hqASKFJCUVkJhYyKGHnkWnTgNJTt7BKaekceWVofkcLEEZY0wEKS4upri4mMTERADWrFlDfn4+BQUFFBYWUlhYyObNm0uHOpoyZQp79uwp3V9QUEBWVhajRo2iqKiI008/vcy+wsJCrrnmGm666Sa2bNnKoYceWi6G3//+Ts455wZWr/6Rq65y4yuouvtqBQWQnt6OvLyBrFmzr1YtxGBZgjLGmGoUFxeXJofCwkJKSkro1q0bAEuXLmXz5s2lCaCgoIDWrVtz8cXuh/2OO+5g/fr1ZRJMr169mDJlCgBDhgzh66+/Lt2/f/9+hg8fzn//+18ATjzxRLZv314mnlNPPZUrrrgCgL/97W8UFRXRrFkzkpKSaN68Oe29pldCQgJNmjShbdu2NG/enKSkJJKSkujZsycAqampTJkypfQ439+ePXvSvTuceGJn/u///ldmf2JiInFxcV4kIcxOWIIyxjRyu3bt4ocffmD16tVlWgmnnnoqAO+99x6ff/55mVZESUkJd911FwD33nsvb7zxRpkE07JlSz744AMAzjnnHF588cUy79mjRw+++uorAP785z/zzjvvlNnfr1+/0gT15ptvsnr1apo3b176I7/P70bRcccdx1FHHVUuQfjM8CYT808w69YdHJ5027ZtAUnjoLi4OBYvXlzpZ5eYmMif/vSnSvcnJCTws5/9rNL9oWYJyhhTbw4cOFDaSkhNTaVp06b8+OOPfPXVV+UuMw0bNozWrVvz0UcfMX/+/NLjfPsfffRR2rZty+OPP879999f7vgffviBtLQ0/vGPf5QmG3/79u0jISGBZ599lkceeaR0e9OmTUlNTS09ZseOHeTl5ZGUlESbNm3o1KkTbf267V1wwQX079+/TIJp49dnferUqezdu7c0ufgWn7feeqvKz6yi2P2dffbZ5bbt2rWr9HVycnKVxzdmlqCMiXKqWtoy8P+B79ChA+3atSMvL4/XX3+9zL6CggLOPfdcevfuzapVq7jjjjvKHX/vvfdy4oknsnDhQi6++GIKCwvLtAzeffddTjrpJF577TUuu+yycnF9+umntG7dms8++4z77ruv3GWmwsJCAFq1akWPHj3K7EtKSiLee9J1xIgRgGu1+O9v4vWn/vvf/86kSZNKtwe2NG677TZuu+22Sj8/X0uoMr169QriWzC1YQnKmDAoKSmhuLiYpk2bAu5GeGAS6dKlC3369GHfvn08+OCD5RLImWeeyfDhw9m+fTvnnXdemf07d+7klltu4eqrr+brr7+u8DLNI488wpVXXsnGjRsZOXJkuf2HH344vXv3Jj8/nw8++KDMj39aWlrpD33nzp0ZPXp0uQTju0czdOhQFi1aVC7BHHbYYQBcccUVXFlFN7DzzjuP8847r9L9AwcOZM+ePZXOj5RW2fATJuJZgjLGU1JSQlFRUWmiEBE6dXJTmn344Yds27atTBJJT08v/eG89dZb2bRpU5kEkpmZyeTJkwHIyMhg8+bNpfv37t3LmDFjeOKJJwA45phj2L9/f5l4rr76ah5++GEArr/+esDdE/D9wHfv3r10m6qSlpZGhw4dSEpK4qeffirdn56ezl133VUmOSQlJdG3b18Ajj76aFatWlVuv6+FkpmZWeaeR6DevXvzwAMPVLr/sMMOK01GFRGbY8NUwhKUaRR2797Nzp07yySAffv2lf6refHixaU3yX37mzVrxqRJkwB46qmnuPvuu8skmMMOO4zXXnsNcP/Kf/vtt8u8Z0ZGBsuWLQPgqquuYsWKFWX2//znPy9NUAsWLGDTpk2lP+7+l6gABg0axL59+8okgX79+pXuf/rpp8skn+bNm5d2/01ISGDXrl1lkoa/li1blrtJ7z/jampqammCq0hiYiJHH310pfuNCRdLUKbGfC0N343w+Ph4tmzZwrp168pdhjr//PNJTk5myZIlLFq0qNx9jJkzZ5KcnMyDDz7I9OnTy90oz8/PJy4ujuuvv57HHnusTByJiYmlSWDGjBk89dRTpfsSEhLo2LFjaYLasWMHW7ZsoXnz5qSlpXHYYYfRpUuX0vIjR45kyJAhZRJMe7+nJJ944gmKi4vL3Cj3vzm9fPnyKj+zqVOnVrn//PPLzTpTSkRo0aJFlccbE41CmqBE5AzgAdyEhdNV9c6A/c2AWcAA3FTvF6rqBm/fX4DLgQPABFVdFMpYGztVLffjXlBQQOfOnWndujVbt24lOzu7zP6cnBw6duzIEUccwaeffsr9999f7vjHHnuMPn36MG/ePK688koKCgoo8hso7IsvvuCYY47hueee45prrikX15AhQzj88MP54IMPyl1mat68OXv37iU5OZlWrVrRvXv3cvtLSkqIi4vjkksuISMjo8w+/55S9913X+nzHElJSSQEzINwzTXXVHqPAmDs2LFVfr6+y2HGmIYTsgQlInHAVOBUIBdYJiILVHWVX7HLgR2qeoSIXAT8E7hQRHriZtfthRtv400ROVJVD4Qq3lBQVQ4cOEB8fDwlJSWsXbu2XALo3r07Rx99NPn5+UybNq3c/uHDh3P66afz3XffMXr06HL777jjDi699FKWL19OZmZmuRiefvppLr74YlatWsWFF15Ybv+wYcM44ogjyMvL49133y2TIFJTU0vvD3Tr1o2LL7643H2KdO8x8l/96lf06NGjXILp6E3ZesMNN3DjjTdW+llddtllFfb08hk8eDCDBw+udP8hhxxS6T5jTOMUyhZUJrBWVdcDiMgcYBjgn6CGAZO9188DD4v7RRwGzFHVvcA33pTwmcCHIYyXL7/8kvz8/DJJoHPnzpx11lkA3HTTTeTl5ZXZn5WVxXXXXQdQmmj8nxifMGECDzzwAHv37q2wJ9Vf/vIXbr/9doqKikofmIuLiyv9ke/duzenn346cXFx7N+/n5YtW5Kenl6633cTv3Pnztx5553lEsTAgQMB19Np5cqVZfYvXbqUU045BYBTTjmFb775ptLPZuDAgaXnqkiXLl3KXDILZDfCjTE1FcoE1QH4zm89FxhUWRlVLRaRn4A23vaPAo7tELpQnTvvvJPvv/++zLazzz67NEHNnTuX/Pz8Mj/ye/bsKS07aNAg4uLiytzHOP744wF3v+Spp54q19XW18Jo3bo1O3fupHnz5uUuT4HrCfXuu+9WGnt6ejo33HBDpfuTk5PLPa+RkJBgicMYE7FEVUNzYpERwBmqOtZbvxQYpKrj/cqs9MrkeuvrcElsMvCRqj7lbX8ceFVVnw94j3HAOID09PQBc+bMqVPMn332GUlJSTRr1oymTZuSmJhYmkyiUX5+PikpKeEOo0HEUl3B6hvtoq2+J5988ieqmhG4PZQtqE1AJ7/1jt62isrkikg8kIrrLBHMsajqNGAaQEZGhlZ1EzxY9XGOxsK/K3K0i6W6gtU32sVKfUM5t+IyoIeIdBORprhODwsCyiwARnuvRwBvq2vSLQAuEpFmItIN6AEsDWGsxhhjIkzIWlDePaXxwCJcN/MZqpojIrcCy1V1AfA4MNvrBJGHS2J45ebhOlQUA1c3th58xhhj6iakz0Gp6kJgYcC2iX6vi4AKn1BU1X8A/whlfMYYYyJXKC/xGWOMMbVmCcoYY0xEsgRljDEmIlmCMsYYE5FC9qBuQxORH4GNdTzNIcC2eginsYil+sZSXcHqG+2irb5dVLVt4MaoSVD1QUSWV/Q0c7SKpfrGUl3B6hvtYqW+donPGGNMRLIEZYwxJiJZgiprWrgDaGCxVN9YqitYfaNdTNTX7kEZY4yJSNaCMsYYE5EsQRljjIlIMZmgROQMEVkjImtF5MYK9jcTkbne/o9FpGsYwqwXQdR1jIj8KCIrvGVsOOKsDyIyQ0S2ehNhVrRfRORB77P4QkT6N3SM9SmI+maJyE9+3+3Eiso1FiLSSUQWi8gqEckRkd9XUCYqvuMg6xpV32+FVDWmFtzUH+uAw4GmwOdAz4AyVwGPeq8vAuaGO+4Q1nUM8HC4Y62n+g4B+gMrK9l/FvAqIMBxwMfhjjnE9c0CXg53nPVY3/ZAf+91C+CrCv57jorvOMi6RtX3W9ESiy2oTGCtqq5X1X3AHGBYQJlhwJPe6+eBoSIiDRhjfQmmrlFDVZfg5hWrzDBgljofAa1EpH3DRFf/gqhvVFHVzar6qfd6N7Aa6BBQLCq+4yDrGvViMUF1AL7zW8+l/BdfWkZVi4GfgDYNEl39CqauAOd5l0OeF5FODRNaWAT7eUST40XkcxF5VUR6hTuY+uJddu8HfBywK+q+4yrqClH6/frEYoIyZb0EdFXVY4E3ONhyNI3fp7gxzvoADwHzwxtO/RCRFOA/wB9UdVe44wmlauoald+vv1hMUJsA/1ZCR29bhWVEJB5IBbY3SHT1q9q6qup2Vd3rrU4HBjRQbOEQzHcfNVR1l6rme68XAgkickiYw6oTEUnA/WA/rar/raBI1HzH1dU1Gr/fQLGYoJYBPUSkm4g0xXWCWBBQZgEw2ns9AnhbvbuSjUy1dQ24Pn827lp3tFoAXOb19DoO+ElVN4c7qFARkUN9905FJBP3/3tj/IcW4HroAY8Dq1X13kqKRcV3HExdo+37rUh8uANoaKpaLCLjgUW4Xm4zVDVHRG4FlqvqAtx/GLNFZC3uJvRF4Yu49oKs6wQRORsoxtV1TNgCriMReRbXs+kQEckFJgEJAKr6KLAQ18trLVAA/Do8kdaPIOo7ArhSRIqBQuCiRvoPLZ8TgUuBL0VkhbftJqAzRN13HExdo+37LceGOjLGGBORYvESnzHGmEbAEpQxxpiIZAnKGGNMRLIEZYwxJiJZgjLGGFMr1Q1YXEH5C/wGwH2muvKWoExMExEVkXv81q8Tkcn1dO6ZIjKins71sTdi9bdSdvT5rvVx/gre76ZaHDNGRB4ORTwmYs0EzgimoIj0AP4CnKiqvYA/VHeMJSgT6/YC50baE/jeCCalVHWQqvYFJuJG1+/rLRtqcp4aqHGCMrGnogGLRaS7iLwmIp+IyLsicpS367fAVFXd4R27tbrzW4Iysa4YmAb8MXBHYAtIRPK9v1ki8o6IvCgi60XkThG5RESWisiXItLd7zSniMhyEflKRH7pHR8nIneLyDJvkN7f+Z33XRFZAKyqLnAR+ZXXsvpMRN4UkXRv+2QRmS0i7+MeOG8rIm94l1Wmi8hGX0IWkVFe3CtE5DEvtjuBJG/b05WV87b/2qvbUtzDpcZMA65R1QHAdcAj3vYjgSNF5H0R+UhEqm15WYIyBqYCl4hIag2O6QNcARyNe+L/SFXNxI1neI1fua64aU9+ATwqIonA5bgheAYCA4Hfikg3r3x/4PeqemQQMbwHHKeq/XBTqfzZb19P4BRVHYkbYeJt77LK83ijEYjI0cCFuEsufYEDwCWqeiNQ6LXQLqmsnLhhsm7BJaaTvPc0MUzc4LYnAM95I2A8hpvbCtzIRT1wo5+MBP4tIq2qOl/MDXVkTCBV3SUis4AJuCFjgrHMN8abiKwDXve2fwmc7FdunqqWAF+LyHrgKOA04Fi/1lkq7n/cfcBSVf0myBg6AnO9RNEU8D9ugar66nISMNyr62sissPbPhQ3OPAyb0i3JKCiyy6VlRsEZKvqj97nMBf3r2QTu5oAO71/yATKxU0guR/4RkS+wv13v6yqkxlj4H5cyybZb1sx3v8jItIElwR89vq9LvFbL6HsP/wCxxJT3Gyv1/jdR+qmqr4Et6cGMT+Emw35GOB3QKLfvmDOI8CTfnH8TFUn16GciXHelCDfiMj54Aa9FZE+3u75uNYT3iXmI4H1VZ3PEpQxgKrmAfNwScpnAwenHzkbbyDWGjpfRJp496UOB9bgBu+9Utx0CojIkSKSXNVJKpHKwakkRldR7n3gAu+9TgPSvO1vASNEpJ23r7WIdPH27ffFV0W5j4Gfi0gbr+z5taiDacTEDVj8IfAzEckVkcuBS4DLReRzIIeDs3gvAraLyCpgMXC9qlY5+rpd4jPmoHuA8X7r/wZe9P5He42atW58vgWWAi2BK1S1SESm4+5NfSrumtmPwDm1OPdk3LX+HcDbQLdKyt0CPCsil+J+TH4AdqvqNhH5K/C610LcD1wNbMTd6P5CRD717kOVK6eqH4nrkv8hsBNYUYs6mEbMu8dZkXIdILyR1q/1lqDYaObGRDkRaQYc8KZfOR74VyX3CIyJKNaCMib6dQbmea2ffbjnUYyJeNaCMsYYE5Gsk4QxxpiIZAnKGGNMRLIEZYwxJiJZgjLGGBORLEEZY4yJSP8P1W7wkAkSNJQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "perf_h2o.plot_uplift(metric=\"lift\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Qini value and Average Excess Cumulative Uplift (AECU)\n", "\n", "Qini value is calculated as the difference between the Qini AUUC and area under the random uplift curve (random AUUC). The random AUUC is computed as diagonal from zero to overall gain uplift. \n", "\n", "The Qini value can be generalized for all AUUC metric types. So AECU for Qini metric is the same as Qini value, but the AECU can be also calculated for Gain and Lift metric type. These values are stored in ``aecu_table``.\n" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", "
\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "
AECU values table: All types of AECU value
uplift_typeqiniliftgain
AECU value6000.11300050.01880056992.6777202
\n", "
\n" ], "text/plain": [ "AECU values table: All types of AECU value\n", "uplift_type qini lift gain\n", "------------- ------- --------- -------\n", "AECU value 6000.11 0.0188005 6992.68" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "perf_h2o.aecu_table()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Normalized AUUC\n", "\n", "To get normalized AUUC, you have to call ``auuc_normalized`` method. The normalized AUUC is calculated from uplift values which are normalized by uplift value from maximal treated number of observations. So if you have for example uplift values [10, 20, 30] the normalized uplift is [1/3, 2/3, 1]. If the maximal value is negative, the normalization factor is the absolute value from this number. The normalized AUUC can be again negative and positive and can be outside of (0, 1) interval. The normalized AUUC for ``auuc_metric=\"lift\"`` is not defined, so the normalized AUUC = AUUC for this case. Also the ``plot_uplift`` with ``metric=\"lift\"`` is the same for ``normalize=False`` and ``normalize=True``.\n" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABHh0lEQVR4nO3deZxN9f/A8dfbWMa+9pOSJUSkbJEilFJUWiRK0UJR+aJsbYpvK+3fFopEWVtVlJQpLWQJkV3KJEKWGeuMef/++JzRNWa5Y+bOuXPv+/l43Mfce865577Pvdz3/ZzP57w/oqoYY4wx4aaA3wEYY4wx6bEEZYwxJixZgjLGGBOWLEEZY4wJS5agjDHGhCVLUMYYY8KSJSgTcUTkURF5x+84giEiPUTku4DHiSJyune/qIh8IiJ7RGS6f1GGPxFpKSJr/I7D5C5LUCbXiMiNIrLI+5L9S0RmiUgLv+PKjIiMF5H/5uD5cSJyR5plrUUk/kT2p6olVHWj97ATUBEor6rXBxOrOH1FZIWI7BOReBGZLiL1TySe/EJV56lqbb/jMLnLEpTJFSIyAHgBeAL3pVoFeBXo6GNY+V1VYK2qJmfjOS8C/wH6AuWAM4CPgA7ZfXERKZjd5xiTq1TVbnbL0Q0oDSQC12eyzXjgvwGPWwPxAY83AQOB5cA+YCwu0c0CEoA5QNn0nhvw/Lbe/UeBdwLWTQe2AnuAb4F63vJeQBJw2Iv/E2/5KcD7wHbgN6BvJscVB9yRZlnaY1NcwtgI7ABGAgW8dT2A79JsWxN4zIsryYvtzvRiTfO6tYAjQNNg483g9e8G1nnH/howKs0+PgYGZPe9CuLfUQzwrPce/Qbc48VT0Ft/K7DK+/ewEbgzi39P93v/nvYAU4FYv/+v2C17N2tBmdzQHIgFPszhfq4DLsH96r8Sl5weAE7Ctfb7nuB+Z+G+vP8PWAK8C6CqY7z7z6g7tXaliBQAPgGWAacCFwP9RKTdiR6U5xqgCdAI16q8LbONVXUYrjU61YttdNpY03naxbgv6Z9yGOvVQDOgLjAZuEFEBEBEygKXAlNC8F71BC4HGuDep6vTrP8buAIohUtWz4tIo0z21xm4DKgOnI1LxiYfsQRlckN5YIdm71RUel5W1W2q+icwD1igqj+r6kFc8mt4IjtV1XGqmqCqh3Ctq3NEpHQGm58LnKSqw1X1sLr+oDeALify2gGeVtV/VPUP3KnQrjncX3rKA3/lwn6e9GI9gPscFGjpresE/KiqW8j996oz8KKqxqvqLuCpwJWq+pmqblDnG2B2QFzpeUlVt6jqP7hE2uAE4zI+sXPMJjfsBCqISMEcJqltAfcPpPO4RHZ3KCIxwOPA9biWWIq3qgLu1E9aVYFTRGR3wLIY3Bd1epKBQmmWFcKdjgu0OeD+77hTY7ltJ1ApF/ZzNFZVVRGZgkuo3wI3AqkjJIN+r0SkCvBrwH7T+yxP4dj3KfA+InI5MAzXwi4AFAN+yeQ4tgbc309o3nMTQtaCMrnhR+AQx5+SCbQP94WS6uQcvN4x+/KS0EkZbHsj7pRaW1xfWbXUp3l/05bz3wz8pqplAm4lVbV9Bvv/I2CfqarjklCg0wLuVwG2ZLC/zGQ19cBXQGURaZLJNsF8DmlfZzLQSUSq4k79ve8tD/q9UtU/vFOTJTJITuBaf5UDHh99z0SkiPe6o4CKqloGmMm/n6OJQJagTI6p6h7gEeAVEblaRIqJSCERuVxEnvE2Wwq0F5FyInIy0C8HL7kWiBWRDiJSCHgIKJLBtiVxyXMn7ov5iTTrtwGnBzz+CUgQkcHedUgxInKWiJybwf6nAreKSFNviPcZQH9gSprtBopIWRE5DTfKbmqQx5pZrMdQ1XW4kZOTvaHuhUUkVkS6iMgQb7OlwLXeZ1QTuD2rF1XVn3EDF94EvlDV3d6q7L5XWZkG/EdEThWRMsDggHWFcZ/xdiDZa01deoKvY/IJS1AmV6jqs8AAXLLYjvt1fQ9uiDPARFxn+iZc38GJfEGnvtYeoA/uC/NPXKsgo+uOJuBaM3/iTjHNT7N+LFBXRHaLyEeqegTXEd8AN5Is9Ys53T4rVf0CGAK8hTtlOBN4GxiTZtOPgcW4BPGZ97rZdUysGWzTF/gf8AqwG9iAG6Dxibf+edxIwG1enO8G+dqTcK3QSakLsvteBeEN3L+N5cDPuPcyGTiiqgnesU0DduFaxjNO8HVMPiGqNmGhMaEkIgrUUtX1fseSn3itpNdVtarfsRh/WAvKGBMWvNOE7UWkoIicihsQkdNLF0w+ZgnKGBMuBHeB8i7cKb5VuL5NE6XsFJ8xxpiwZC0oY4wxYckSlIl6gRXJReQmEZmdy/uvJiJqxVeNyR5LUCbkRGSTiPwtIsUDlt0hInE+hpUuVX1XVX25vsZLlLu8i1LTLs9wSo+MEmDa6TlEpJKIjPWmQkkQkdUi8ljg5xJknNVEZK6I7Pf20TaTbcuJyFQR2SkiO0TkXREp5a2rIm5qlsCbish9Ac+/V0R+E5G94qZyaRGwbqC4aUUSvG0GZuc4TPizBGXySgzuAtUc8S6Gjbh/tyJSDVdXToGrQrD/criKH0WB5qpaEleYtwxQI5u7m4wbxFAeeBB4T0QyquTxX6AsrrpGDVyF+kch3eoS9XGlqN73Ym6Gq8fXCXdt1VjgQ69yCLhBFbd4+78MuEdEcloz0YSRiPuPbsLWSOB+r0LAcUTkfBFZKG722IUicn7AujgReVxEvsfVVDvd+6XdR0TWeb+gR4hIDRH5wfu1PU1ECnvPLysin4rIdq+F8qmIVM4gjqMz3IrIoDS/7pNEZLy3rnRAa+RPEflv6henV1FhlNdi2EhwczHdgruIeDzQPah3NHsG4Kap6KaqmwBUdbOq/kdVlwe7E3GVMhoBw1T1gKq+j6uHd10GT6kOfKSqe70LrD8E6mWw7S3At6nx4UpIrVTVxepGc03A1VD8Py/+Z1R1iaomq+oa3MXQFwR7LCb8WYIyeWURbi6i+9Ou8H7dfwa8hPtV/hzwmYiUD9jsZtz8TSX5t85dO6AxcB4wCFe9oRuuhttZ/FsxvACu0kNVXB28A7hqC5nyvgBTf92fiauQkVoBYzyuykFNXJX1S4HU03A9cRUWGuKm2OiU1Wvhvpzf9W7tRKRiEM/JjrbAB6qaktEGIrJcXJWK9G6vepvVAzZ6lR1SLSPjpPMKcIX3I6EsLpHNSue1U1tDbwcsngXEiEgzL/nfhqvEsTWD57cEVmZ0fCb/sQRl8tIjwL3pnA7qAKxT1Yner+HJwGrcnFCpxqvqSm99aqXwZ7xf5iuBFcBsVd3o/VKfhTc9h6ruVNX3VXW/98X6ONAq2KBFpCiuZNOLqjrLSx7tgX6quk9V/8aVEEo9vdQZeMFrofwDPJnF/lvgkuc0VV2MK090Y7DxBSnLqThU9ew0hV8Db328zUpwfBX4PbgfDulZgqujt9O7HcHVC0yrBe7033sByxJwp/u+w9VTHAb00vSvjXmUf3+ImAhhCcrkGVVdAXyKq10X6BSOr/79O24SvFSbOV5Q03OIK4w6WkR+F5G9uGkjygT0ZWRlLLBGVZ/2HlfFTanxV2oLAxiNd+qJ46eNSHtsaXXHJdcd3uNJHHuaL6spPZIDlmW0TW5NxZGImzAwUClcMknPNFxx35Ledhv4d7qOQN2B91U1MWDZ7biJCevhklw34FMROWbaDBG5B9f66uDN+WUihCUok9eG4U6BBSafLbgv/UBVcAVeU+XkivL7gNpAM1UtBVzoLc9yqgZxVcDP4Niq35txv+grBLQwSqlq6mmuvzh+eo2M9l8U1+JqJSJbRWQrrhr6OSJyjrdZVlN6/IVLRJltMwe4JrMBJiKyMp1Rdam3173NVuL6AANbTOeQ8am1BsBor6WZCLyOa32mfQ+u59jTe6nP/VRV16pqiqp+7h1rYP/kbbgfPBerakYFg00+ZQnK5CmvYOpUjp2+fSZwhojcKK4O2w246cY/zaWXLYlrUe32+ruGBfMkccVK+wLXeLPLph7DX7iq28+KSCkRKeAN0Eg9bTgN6Csilb1+l7QtxkBX40571cV9ITfA9XfNw7UKIIspPbyq4u8Dj4tIeXFTnXT19pna3/McrgXztrh5nRA3rcVzInK2t596gaPq0tzu8rZZi+sHGiZuKo9rcNOpp84RldZC4A5xdfaK4voR0w7KuAZX3mhuOs/tICKne8d9Ce7Hwgov/ptw06dcom42XxNhLEEZPwwHjl57o6o7cYMK7sOdihoEXBFwyiunXsANr96BGyn3eZDPuwE3EeKqdFoSt+BOO/2K+3J9j39Pob0BfIEbPLAE+CCT1+gOvOUNud6aesMN4rhJ3CzFwUzp0Qf4B/fl/zduqpMOqroNwOsLOx/X0logIgm4CQ73ANmtst4FN/gjdVr2Tqq6HY5e6BzYmroN17KLx7WIT+f4UYrdgYnp9C1NwCXhOGAvbhDNnaq62lv/X1zf2sJ0Ph8TAawWnzHGmLBkLShjjDFhyRKUMcaYsGQJyhhjTFiyBGWMMSYs5bvy/xUqVNBq1arlaB/79u2jePFsFXDO16LteMGOOVrYMUeGxYsX71DV4woO57sEVa1aNRYtWpSjfcTFxdG6devcCSgfiLbjBTvmaGHHHBlEJN1qK3aKzxhjTFiyBGWMMSYsWYIyxhgTlixBGWOMCUuWoIwxxoSlkCUoERknIn+LyIoM1ouIvCQi672ZPBuFKhZjjDH5TyhbUOOByzJZfzlQy7v1Al4LYSzGGGNySVISJCdnvV1Ohew6KFX9VkSqZbJJR2CCV2J/voiUEZFK3lw7xhhjQkAV9u+HxERISHB/s3N/61ZYtgxgGVWrJjJ+/AWE6rIsPy/UPZVjp8WO95Ydl6BEpBeulUXFihWJi4vL0QsnJibmeB/5SbQdL9gxR4tIP2ZVSEoqwIEDMRw4EMP+/TH8809BFi1axv79MUeXB9727y94zPbpbaOa5WTSAMTEpFCs2BGKFTtCbOwRihY9QsGCv1OhwuPs2DGNLVua8d13Y3DTuOW+fFFJQlXH4E3O1qRJE83pVdSReCV2ZqLteMGOOVqE2zEnJ59YqyTt/cBlwZ5KK1AASpRwt5Il3d+yZaFKlWOXZed+4cIFcD1Bhdi9ezdPPvkkL774IgCDBg1kyJAhlC1bNmTvp58J6k/gtIDHlb1lxhgTcikpsG/fiSeO9O4fOhT86xcvfnxiqFABqlXLPHFs3LiMCy4457jlRYuCBNcwOiG7d+/mpZde4oYbbmDEiBFUqVIldC/m8TNBzQDuEZEpQDNgj/U/GWPSowoHDhyfGH76qRw7dmQviaTe378/+NcvUuT4hFGyJFSq9O/y7LRQihWDmJgTey/i4nZxwQUn9tzsSElJYcqUKcTFxTFmzBiqVavGpk2bqFixYuhf3BOyBCUik4HWQAURiQeGAYUAVPV1YCbQHlgP7AduDVUsxpi8dehQzloiae8nJroWz/HOPm5JTMy/CSQwMVStemySCDahFC8OhQqF/C0LK19//TUDBw5kyZIlNGjQgN27d1OmTJk8TU4Q2lF8XbNYr8DdoXp9Y0xwkpOPTQQnmkQC7yclBffaIuknhpNPhpo1s04oa9cuoVWrRmn6TUJ7qiuSxcfH06tXL2bNmkWVKlWYOHEiN954IwUK+FPTIV8MkjDGOCkp7tRUeklk/vyT2LAh+wnl4MHgX79Yscw74rPbQilWLGfJpFChvZx11ok/3zhHjhwhJiaGUqVKsWHDBkaOHMk999xDbGysr3FZgjImDCQnw6ZNsH79sbc//zw2oezbl9le6h3zqHDh9BNDxYon1m9SvPiJ95uY8LR7926eeuop5syZw/z58ylVqhS//vorMWHyQVuCMsYH+/bBggXw3Xcwbx78+OOxyad4cXeK67TT0u9PSS+J/PrrT1x0UdOj/SaFC/t3fCa8HTp0iNdee40RI0awa9cuunXrxr59+yhdunTYJCewBGVMnvj7b5eMUm9LlsCRI+701jnnwK23QqNGUKuWS0wVK2b/1Nf+/fvJg5G/Jp/buHEjbdu25bfffuOSSy7h6aefpmHDhn6HlS5LUMbkMlV3ei4wIa1d69bFxkKzZjBkCLRoAc2bQ+nS/sZrosO2bduoWLEiVapUoWnTprz++utceumlfoeVKUtQxuRQcjIsXXpsQtq2za0rV84lop493d9GjezUm8lbK1asYMiQISxYsIANGzZQqlQppkyZ4ndYQbEEZUw2uRFz/yaj+fP/7T+qXh3atXPJqEULqF3blaAxJq/9+eefPPLII4wfP56SJUvywAMPUDif/TqyBGVMFrZtO7Z19PPPrv+oQAHXf3TbbS4ZXXABnHqq39EaA7/99hv16tXjyJEj9OvXjwceeIDy5cv7HVa2WYIyJoAqrFv3bzKaN8/1J4GrddasGQwd+m//UalS/sZrTKrDhw+zYMECWrZsSfXq1Rk2bBidO3emevXqfod2wixBmaiWlOT6j+bN+zcpbd/u1pUv7xLRXXe5vw0bWv+RCT+qyvTp0xk6dCjx8fH89ttvnHLKKQwePNjv0HLMEpSJKgkJx/cfpRYNPf10aN/+2P4jK5ljwtk333zDwIEDWbhwIfXr1+fjjz+mUqVKfoeVayxBmYj211/w/ff/nq5butSVCypQABo0gDvu+Lf/6JRT/I7WmODFx8dz8cUXU6lSJcaPH0+3bt3C6iLb3GAJykQMVXe90bx58N57dbjjDtiwwa0rWhTOOw8efPDf/qOSJf2N15js2rJlCx9//DGtW7emcuXKfPbZZ1x44YUULVrU79BCwhKUybcOH3Yj6gJH2O3Y4daVLl2ONm2gd29o2dL1H0XblAkmcuzdu5eRI0fy7LPPkpycTP/+/alSpQrt2rXzO7SQsgRl8o3kZFe/bvZs10qaP99NYgdQowZcccW//UdbtvxAmzatfYzWmJw7fPgwY8aMYfjw4Wzfvp0uXbpw5ZVX5slstuHAEpQJa5s3wxdfwOefw5w5sGeP6z9q2BB69fq3/yhtv/BfNjeziQAJCQk89NBDNGzYkJEjR9KkSRPi4uL8DivPWIIyYeXAAdc6+vxzl5h+/dUtr1wZOnWCyy6Diy92cxAZE4m+/fZbJk6cyOjRoylfvjxLly6latWqSBQOKbUEZXy3Zg3MmuUSUlycm0CvSBG48EK4/XZXOqhuXRvybSLbqlWrGDJkCDNmzODUU0/ljz/+oFq1alSrVs3v0HxjCcr4Yvt2mDwZ3n7bTT0B7rqjXr1cK6lVKzfbqjGRbs+ePQwcOJCxY8dSokQJnnzySfr27Usx+w9gCcrkra1bYdQoeOUV11Jq1Aiefx6uvhqi+IeiiUKqiohQtGhR5s2bx7333stDDz1EhQoV/A4tbFiCMiF1+DD88IMbeTd7tmsticAtt8CAAVC/vt8RGpO3kpKSeOONNxg7dizz5s2jWLFiLFu2LN9VGs8LlqBMrktMhOnT4b334Jtv3FQUMTHu4tjHHoPOnd3pPGOiiarywQcfMHToUNatW0erVq3YuXMnxYoVs+SUAUtQJleouuuSxo6FqVNdkjr9dOjRAy69FFq3tsrfJnrt2rWLDh068OOPP1K3bl0++eQTOnToEJUj87LDEpTJkV274J13YMwYWLECiheHG25wcySdf76NvDPRbffu3ZQpU4YyZcpw2mmncdttt9GjRw8KFrSv3mDYu2ROyMqV8PTT7lTewYNw7rnwxhsuOVmNOxPttm7dyqOPPsrkyZNZvXo1lSpVYurUqX6Hle9YgjLZsnixm7Dvyy9da+nWW6FnT1fZwZhol5iYyKhRoxg1ahSHDh2id+/e1r+UA5agTNC++w7atnUDHu66CwYPtqHhxqTau3cvderU4a+//uL666/niSeeoGbNmn6Hla9ZgjJZ+uUXeOABVwvv5JNdwdaKFf2Oyhj/qSqLFy+mSZMmlCpVin79+tGqVSuaNWvmd2gRoUAody4il4nIGhFZLyJD0llfRUTmisjPIrJcRNqHMh6TffPnu5JDn34KZcrAxImWnIwB+OGHH2jRogVNmzZl+fLlAAwaNMiSUy4KWYISkRjgFeByoC7QVUTqptnsIWCaqjYEugCvhioek31ff+1O6ZUr5xLVmjVubiVjotmaNWu49tprueCCC9i4cSOjR4+mbt20X20mN4TyFF9TYL2qbgQQkSlAR+DXgG0USL06pjSwJYTxmGz45BO4/no3z9KXX9p06MYA7Nu3j2bNmnHkyBGGDx/OgAEDKF68uN9hRSxR1dDsWKQTcJmq3uE9vhlopqr3BGxTCZgNlAWKA21VdXE6++oF9AKoWLFi4ylTpuQotsTEREqUKJGjfeQn2T3er7/+P554og41auzjmWeWU7p0UgijC41o+4zBjjlUDhw4wNdff0379u0REX788Ufq1KlDWZ/mfInEz7lNmzaLVbXJcStUNSQ3oBPwZsDjm4H/pdlmAHCfd785rnVVILP9Nm7cWHNq7ty5Od5HfpKd4x0zRlVEtWVL1T17QhdTqEXbZ6xqx5zbkpKSdPTo0XryyScroPPnzw/Za2VHJH7OwCJN5/s+lIMk/gROC3hc2VsW6HZgGoCq/gjEAlbK1yfPPeemu2jXzk0YaKWJTDRSVT7++GPq16/PnXfeSY0aNfjhhx9s8IMPQpmgFgK1RKS6iBTGDYKYkWabP4CLAUTkTFyC2h7CmEw6VOHRR+G++9ystR9/bHMxmeh16NAh7r77blSVjz76iHnz5tG8eXO/w4pKIUtQqpoM3AN8AazCjdZbKSLDReQqb7P7gJ4isgyYDPTwmnsmj6i6xPTYY66w6+TJYBe+m2izdu1a7r77bg4fPkxsbCxfffUVK1asoGPHjlbQ1UchvVBXVWcCM9MseyTg/q/ABaGMwWTsyBFXEeLNN6FvXzdxYIGQXhlnTHjZtm0bw4cPZ8yYMRQpUoRbbrmFZs2aUdvmgwkL9nUUpQ4fhptucsnpoYfghRcsOZnocfjwYUaMGEHNmjUZPXo0vXr1YsOGDdbPFGas1FEUOnDAXeP02WeuIvmgQX5HZEzeKliwIDNmzODSSy/liSeesBZTmLLfzFEmIQHat4eZM+G11yw5meigqnzyySecf/757NixgwIFChAXF8f7779vySmMWYKKIv/840oXzZvnaurddZffERkTegsWLKB169ZcddVV7Ny5k/j4eACrAJEPWIKKElu3umnXly6F9993/U/GRLKkpCRuuOEGzjvvPFavXs1rr73GihUraNCggd+hmSBZgooC27YV4cILYcMG1+/UsaPfERkTOvv37wegUKFCxMbGMmzYMNavX89dd91FoUKFfI7OZIcNkohwa9dC374NOXTIFX09/3y/IzImNPbv38/zzz/Ps88+yw8//ECdOnV4++23/Q7L5IC1oCLY8uVueozDhwswd64lJxOZkpOTGTt2LLVq1eKhhx6iVatWNs16hLAWVIRasAAuuwxKlICRI5fSsGFTv0MyJtcdOXKEZs2asWTJEs477zymTp1KixYt/A7L5BJrQUWguXPh4ouhfHk3Yq9Klf1+h2RMrlq9ejUAMTEx3HTTTbz33ntHZ7g1kcMSVIT59FO4/HKoVs0lp2rV/I7ImNyzYcMGunTpwplnnsmcOXMAGDBgANddd53VzItAWSYoEblWRNaJyB4R2SsiCSKyNy+CM9kzdSpccw3Urw/ffAOVKvkdkTG5Y8eOHfTr148zzzyTTz75hIcffpimTe20daQLpg/qGeBKVV0V6mDMiXvzTTeXU8uWbrp2m8vJRIqUlBSaN2/Oxo0buf3223n00Uc55ZRT/A7L5IFgEtQ2S07h7Y03XHK67DJ3Ea7N5WTyuyNHjvDee+9x3XXXUbBgQV544QWqV69O3bp1/Q7N5KFg+qAWichUEenqne67VkSuDXlkJijx8dC/P1xyiU00aPI/VWXWrFk0aNCALl268NFHHwHQoUMHS05RKJgEVQrYD1wKXOndrghlUCZ4/fu7eZ1Gj7aJBk3+tnjxYtq2bUv79u05cOAA06ZN47rrrvM7LOOjLE/xqeqteRGIyb7Zs+G992DECKhe3e9ojDlxqkqPHj3YunUrL7/8Mr169bKLbU3GCUpEBqnqMyLyMnDcNOyq2jekkZlMbdni+p1q1oT77/c7GmOyb+fOnTz77LMMGTKEUqVKMXXqVCpXrkwpG+FjPJm1oFIHRizKi0BM8I4cgWuvhe3b4fPPITbW74iMCd6BAwd46aWXePLJJ0lISKBJkyZce+211sdkjpNhglLVT7y/Vm0xzAwf7koZvf22G1ZuTH6gqkyYMIGHHnqI+Ph4rrjiCp566inq1avnd2gmTGXZByUiJwGDgbrA0d/qqnpRCOMyGZg0ySWoW2+Fm2/2OxpjgiciTJ48mUqVKjFx4kRat27td0gmzAUziu9d3Om+6sBjwCZgYQhjMhkYMMBNNNiyJbz+OlhlFxPulixZQvv27dm0aRMAkydPZv78+ZacTFCCSVDlVXUskKSq36jqbYC1nvLYokXw/PNw9dXuYlwb4GTC2aZNm+jWrRuNGzfmp59+Ys2aNQCULVuWAgWsBKgJTjD/UpK8v3+JSAcRaQiUC2FMJh1Dh0KFCq7f6aST/I7GmPSpKoMHD6Z27dq8//77PPDAA2zYsIF27dr5HZrJh4IpdfRfESkN3Ae8jLtwt39IozLHmDPH3Z5/3mrsmfCUlJREoUKFEBH27dtHt27deOyxx6hcubLfoZl8LMsWlKp+qqp7VHWFqrZR1caqOiMvgotWqtCjB5x1FuzaBYMHQ9Wq0Lu335EZc6yUlBQmTpxIzZo1+eGHHwB4+eWXGTt2rCUnk2PBjOJ7KZ3Fe4BFqvpx7odkpk51p/IATj0VDhxwy4oU8TcuYwLNnj2bQYMGsWzZMho3bkzBgu7rxOZlMrklmD6oWKABsM67nQ1UBm4XkRdCFlmUSkqCnj2haVPo2NElp3PPhc6d/Y7MmH917tyZdu3asXfvXiZNmsRPP/1k8zOZXBdMH9TZwAWqegRARF4D5gEtgF8ye6KIXAa8CMQAb6rqU+ls0xl4FFdOaZmq3pidA4gkyclw3XWQmAj33QcNGsDvv8PIkX5HZgzEx8dzyimnUKBAAdq0acP5559P7969KWJNexMiwSSoskAJ3Gk9gOJAOVU9IiKHMnqSiMQArwCXAPHAQhGZoaq/BmxTCxiKS4C7ROT/TvA4IsL997vJBi++2CWqmBj4+We/ozLRbteuXTzxxBO8/PLLjBs3jhtvvJHe1iFq8kCwM+ouFZE4QIALgSdEpDgwJ5PnNQXWq+pGABGZAnQEfg3YpifwiqruAlDVv7N9BBEiMRHGjXOn9T74AOxSEeO3gwcP8sorr/D444+ze/duunfvTkurrWXykKgeV6j8+I1EKuESDsBCVd0SxHM6AZep6h3e45uBZqp6T8A2HwFrgQtwpwEfVdXP09lXL6AXQMWKFRtPmTIly5gzk5iYSIkSJXK0j9z0yy+lGTmyNps3F+N//1tCvXp7c3X/4Xa8ecGOOefuu+8+lixZQtOmTenVqxc1atTItX3nFvucI0ObNm0Wq2qT41aoakhuQCdcv1Pq45uB/6XZ5lPgQ6AQrpTSZqBMZvtt3Lix5tTcuXNzvI/csnGjarFiqqA6dWpoXiOcjjev2DGfmDlz5ui+fftUVXX27Nn65Zdf5nifoWSfc2TAjQo/7vs+lCeS/gROC3hc2VsWKB6YoapJqvobrjVVK4QxhZWkJOjaFQ4ehC++sJF6xj/Lli2jXbt2tG3bltGjRwNwySWX0LZtW58jM9EslAlqIVBLRKqLSGGgC5D2At+PgNYAIlIBOAPYGMKYwsqIEW7ajClT4NJL/Y7GRKM//viD7t2707BhQxYuXMhzzz1Hnz59/A7LGCC4QRKISAuglqq+5U2/UcJr8WRIVZNF5B7gC1z/0jhVXSkiw3HNuRneuktF5FfgCDBQVXfm5IDyiwUL4PHHXXXy66/3OxoTrXr27Mk333zDwIEDGTJkCGXLlvU7JGOOCqaSxDCgCVAbeAvXX/QObmBDplR1JjAzzbJHAu4rMMC7RY05c9xovZNOgsce8zsaE00OHTrEq6++yg033MApp5zCiy++SLFixahSpYrfoRlznGBaUNcADYElAKq6RURKhjSqCLZ3L1x5pet3+uUXOP10vyMy0SAlJYUpU6bw4IMPsmnTJkSEfv36UadOHb9DMyZDwfRBHfZaOgrgXf9kTtDHH7vkNHGiJSeTN77++mvOPfdcbrrpJsqUKcPs2bPp16+f32EZk6VgWlDTRGQ0UEZEegK3AW+ENqzINWUKVKni+p6MyQvjx49nx44dTJw4kRtvvNEmDDT5RjDTbYwC3gPex/VDPaKqL4c6sEg0aRLMnAm33GLTtZvQ2bx5M7feeis/e3WyXnjhBdasWUO3bt0sOZl8JZhBEgOAqar6ZR7EE9EeecRVJn/4Yb8jMZEoMTGRIUOG8OKLL5KSkkKLFi1o2LAh5crZBNgmfwrmFF9JYLaI/ANMBaar6rbQhhV51q2DDRugf38oXNjvaEykef311xk8eDAJCQl069aNESNGULVqVb/DMiZHgjnF95iq1gPuBioB34hIZkViTTqmT3d/L7/c3zhM5EhJSUktGcZff/3FGWecweLFi5kwYYIlJxMRsnNC+m9gK7ATiOppMbIrORn+9z9XLcJG7pncEBcXR7NmzZgxwxVneeSRRxg5ciQNGzb0OTJjck+WCUpE+nhTbXwFlAd6qurZoQ4sksyaBX/9BTaFjsmpFStWcMUVV9CmTRu2bdt2dHr1mJgYnyMzJvcF0wd1GtBPVZeGOJaINXYsVKwIHTr4HYnJzx588EGeeuopSpYsydNPP829995L0aJF/Q7LmJDJMEGJSClV3QuM9B4fMxRIVf8JcWwR4fPPYcYMGDgQChXyOxqT3+zZs4fY2FiKFClCnTp16NevHw888ADly5f3OzRjQi6zU3yTvL+LgUXe38UBj00W9u+Hbt2gVCkYEFXVBk1OHT58mJdeeokaNWrw2muvAXDzzTfz7LPPWnIyUSPDFpSqXuH9rZ534USWV1+FnTth3jx3is+YrKgq06dPZ+jQoWzcuJGLLrqICy+80O+wjPFFMIMkvgpmmTne229D8+bQooXfkZj8olevXtxwww0UL16cWbNmMWfOHBo1auR3WMb4IrM+qFigGFBBRMoCqcV5SgGn5kFs+dqCBbBihRtebkxmfv31VypWrEj58uXp0aMHLVq0oFu3bjYyz0S9zFpQd+L6m+pwbP/Tx4B97WZh2DAoX97V3TMmPVu2bKFnz57Ur1+fp556CoALLriA7t27W3Iyhsz7oF4EXhSRe604bPZMmwZffAHPPAMlbeYsk8bevXt55plneO6550hOTqZv374MHjzY77CMCTtZXgelqi+LyFlAXSA2YPmEUAaWXyUmwoMPQv360Lev39GYcPSf//yH8ePH06VLFx5//HFOt/IixqQr2CnfW+MS1EzgcuA7wBJUOh55xBWFnTMHihTxOxoTDlSV9957j7PPPpvatWvz8MMP06dPH84991y/QzMmrAVTi68TcDGwVVVvBc4BSoc0qnxqzx544w03GeFFF/kdjQkH3377Lc2bN6dz5868+uqrAJx++umWnIwJQjAJ6oCqpgDJIlIKVzT2tNCGlT+9+aY7xde/v9+RGL+tWrWKjh070qpVK+Lj4xk3bhzPPfec32EZk68EU4tvkYiUwU3zvhhIBH4MZVD5UXIyvPQStGoFdtmKGTt2LHFxcTzxxBP85z//oVixYn6HZEy+E8wgiT7e3ddF5HOglKouD21Y+c8HH8Aff7gkZaJPQkICo0aNolWrVlx00UU89NBDDBkyhAoVKvgdmjH5Voan+ESkUdobUA4o6N03nqQkuOEGd/+KK/yNxeStpKQkXn31VWrWrMnw4cOZO3cuAGXKlLHkZEwOZdaCejaTdQrYMADPV17hp2eeAbu+Mnp89tln9O/fn3Xr1tGqVSs++eQTmjZt6ndYxkSMzC7UbZOXgeRn06dD6dJ23VO0UFVEhE2bNlG4cGE+/fRT2rdvf3TyQGNM7gjmOqh0i/XYhbrOkSPw5Zdw8cV23VOkW716NUOHDqV9+/b07NmTO++8kzvvvJOCBYMZa2SMya5ghpmfG3BrCTwKXBXCmPKVkSNh82bo0sXvSEyobN26ld69e3PWWWfx1VdfkZSUBEDBggUtORkTQsGM4rs38LE35HxKMDsXkcuAF4EY4E1VfSqD7a4D3gPOVdV8NRniF1/AWWdBp05+R2JC4Y033qB///4cOnSIPn368PDDD3PSSSf5HZYxUSGYFlRa+4AsJzEUkRjgFVxppLpAVxGpm852JYH/AAtOIBZfpaTA4sXQsiVY90PkSEpK4sCBAwCccsoptG/fnlWrVvHSSy9ZcjImDwUzYeEnIjLDu30KrAE+DGLfTYH1qrpRVQ/jWl0d09luBPA0cDAbcYeFVasgIQEaN/Y7EpMbVJUPP/yQs846i//+978AdOjQgWnTplGzZk2fozMm+gRzAn1UwP1k4HdVjQ/ieacCmwMexwPNAjfwrqc6TVU/E5GBGe1IRHoBvQAqVqxIXFxcEC+fscTExBzvA+CJJ+pQqND/UazYT8TFhW9+za3jzU+ye8wrVqzg9ddfZ+XKlVStWpWSJUvmu/fMPufoEFXHrKpB3XAz6ZZLvQWxfSdcv1Pq45uB/wU8LgDEAdW8x3FAk6z227hxY82puXPn5ngfBw6oli2r2r17jncVcrlxvPlNdo75ySefVEArVaqkY8aM0aSkpNAFFkL2OUeHSDxmYJGm830fzDDzXsBw3Cm4FNzU7wpkNYnNnxxbVLaytyxVSeAsIM67fuRkYIaIXKX5YKDEhAmwa5fNmJtfbdu2jZSUFCpVqkSHDh1ITk6mf//+FC9e3O/QjDGeYE7xDQTOUtUd2dz3QqCWiFTHJaYuwI2pK1V1D3C0FoyIxAH354fkBPDOO25SwjZ2OXO+kpiYyHPPPcfIkSPp2LEj77zzDvXr16d+/fp+h2aMSSOYUXwbgP3Z3bGqJgP3AF8Aq4BpqrpSRIaLSL6+jurQIfjpJ7jkEhu9l18kJyczevRoatWqxbBhw2jXrh2PPPKI32EZYzIRTAtqKPCDiCwADqUuVNUsC/uo6kzcLLyBy9L9VlDV1kHEEhYWLXJJqmVLvyMxwRo+fDgjRoygRYsWfPDBBzRv3tzvkIwxWQgmQY0GvgZ+wfVBRb1Jk6BwYbjwQr8jMZmZP38+RYoUoWHDhvTp04dGjRrRsWNHq5lnTD4RTIIqpKoDQh5JPpGQ4AZIdOkC5cr5HY1Jz9q1axk2bBjffvstHTt25KOPPuLkk0/m6quv9js0Y0w2BNMHNUtEeolIJREpl3oLeWRhaN8+qFPHTet+551+R2PS2rZtG3fffTf16tVj4cKFPPbYY7zzzjt+h2WMOUHBtKC6en+HBiwLZph5xJk2DbZsccnJujDCz4QJExg9ejR33nknF198Mddee63fIRljciCYYrFZ1t2LFitWQGwsvPqqjd4LB8nJyYwfP54KFSpw9dVXc88993DVVVdRu3bt6LnS3pgIZvNBBUkV5syBBg2gwImU2DW5RlX59NNPGTx4MKtWraJLly5cffXVFC1alNq1a/sdnjEml9h8UEFatw6WL4cbb8x6WxM6S5YsoXXr1lx11VUcOXKEDz74gEmTJvkdljEmBEI6H1QkmTfP/b30Un/jiHbr169n9erVvPrqq9xxxx0UKlTI75CMMSFyItOBBjUfVKT59VcoWhRq1fI7kuiyfft2RowYQeXKlRk0aBDXX3897du3p0SJEn6HZowJsWD6oD7BjdoDd0qwLjAtlEGFo1WrXHKy/qe8sX//fp5//nmefvpp9u/fT//+/QEQEUtOxkSJUM4HFTGWL4fZs6FvlsWdTG6YOXMmPXv2ZMuWLVx99dU8+eST1KlTx++wjDF5LMMEJSI1gYqq+k2a5ReISBFV3RDy6MLExImu5fTgg35HErlUlUOHDhEbG0uFChWoWrUqU6dOpUWLFn6HZozxSWYnrF4A9qazfK+3LmrMmwfNmkH58n5HEpkWLlzIRRddxL33uvE4TZs25fvvv7fkZEyUyyxBVVTVX9Iu9JZVC1lEYSYxERYvtsrlobBx40a6du1K06ZNWblyJY0aNTq6zgq6GmMy64Mqk8m6orkcR9j68ENITobLL/c7ksgyefJkunfvTqFChXjooYcYOHAgpUqV8jssY0wYyawFtUhEeqZdKCJ3AItDF1J4+fZbV7Xczjbl3P79+/nzzz8BuOCCC7jttttYt24dI0aMsORkjDlOZi2ofsCHInIT/yakJkBh4JoQxxU25s+HJk2s9l5OHDlyhAkTJvDwww9z5pln8uWXX1KlShVef/11v0MzxoSxDFtQqrpNVc8HHgM2ebfHVLW5qm7Nm/D89fvvrkBsu3Z+R5I/qSozZ86kQYMG3HbbbZx66qk8/PDDfodljMkngil1NBeYmwexhJ0ZM9zfK6/0N478aty4cdxxxx3UrFmT6dOnc91119ngB2NM0E6k1FHUmDoV6te38kbZ8dtvv7F9+3aaNm1K586dOXz4MLfffjuFCxf2OzRjTD5jhXsysHkzfP893HCD35HkDzt37qR///7Url2b3r17o6qULFmS3r17W3IyxpwQS1AZmDzZ/bUElbkDBw7w1FNPUaNGDV566SVuueUWZsyYYafyjDE5ZgkqA2++6S7OrVnT70jC2/vvv8/QoUNp2bIly5cv58033+TUU0/1OyxjTASwPqh0LFvmJijs18/vSMKPqvLFF1+wa9cuunbtSteuXTn99NM5//zz/Q7NGBNhrAWVjjfegCJFoEsXvyMJL0uWLOGSSy7h8ssv54UXXkBViYmJseRkjAkJS1BpqML06XD11a6ChIHff/+dbt260bhxY5YuXcqLL77IvHnzrJ/JGBNSdoovjU2b4O+/oVUrvyMJHxs3bjza1zR48GBKly7td0jGmChgCSqNr792f5s18zcOPx08eJCXX36ZhIQEhg8fTps2bdi8eTMVKlTwOzRjTBQJ6Sk+EblMRNaIyHoRGZLO+gEi8quILBeRr0SkaijjyYoqPPEElC7tLtCNNqk188444wwGDRrEL7/8QkpKCoAlJ2NMngtZghKRGOAV4HKgLtBVROqm2exnoImqng28BzwTqniCsX49bNwIgwdDoUJ+RpL3Fi9eTOPGjenevTv/93//x9dff82HH35IgQLWTWmM8Ucov32aAutVdaOqHgamAB0DN1DVuaq633s4H6gcwniytGyZ+3vZZX5GkbeSk5MBKFWqFAcOHGDy5Mn89NNPtGnTxufIjDHRLpR9UKcCmwMexwOZ9ezcDswKYTxZWrPG/Y2G2nu///47Dz/8MAkJCXz44YfUqlWLVatWWYvJGBM2wmKQhIh0w801le7YORHpBfQCqFixInFxcTl6vcTExHT3MW1aA6pWLcSiRQtztP9wE3i8CQkJvPvuu3zwwQeICNdddx1fffUVMTEx/gaZyzL6jCOZHXN0iKpjVtWQ3IDmwBcBj4cCQ9PZri2wCvi/YPbbuHFjzam5c+cet+zQIdVChVQHDcrx7sNO6vHGxcVp2bJlVUS0R48e+scff/gbWAil9xlHOjvm6BCJxwws0nS+70N5PmchUEtEqotIYaALMCNwAxFpCIwGrlLVv0MYS5bWrIGkJGjQwM8ocl9KSgo7d+4E4Oyzz6Zt27YsXbqUt956i9NOO83n6IwxJmMhO8Wnqskicg/wBRADjFPVlSIyHJctZwAjgRLAdK8qwR+qelWoYsrM8uXu79ln+/HqofHll18yaNAgEhMTueaaayhbtizTpk3zOyxjjAlKSPugVHUmMDPNskcC7rcN5etnx7JlULgwnHGG35Hk3LJlyxg0aBCzZ8+mWrVq3HTTTX6HZIwx2WZDtjzz57vTe/n9+qcvv/yShg0bsnDhQp577jlWr15N27ZtbXSeMSbfsW8t4PBhWLgQWrTwO5ITs3v3bhYsWABA69atefzxx9mwYQP9+/enSJEiPkdnjDEnxhIUsGQJHDwIF1zgdyTZc+jQIZ5//nlq1KjBtddeS1JSEoUKFWLo0KGULVvW7/CMMSZHwuI6KL99/737m18SVEpKClOmTOHBBx9k06ZNXHrppTz99NMUyu/nJ43Jp5KSkoiPj+fgwYMhf63SpUuzatWqkL9OKMTGxlK5cuWgv6ssQeESVI0aULGi35EE55tvvuGmm26iQYMGzJ49m0suucTvkIyJavHx8ZQsWZJq1aqFfJ60hIQESpYsGdLXCAVVZefOncTHx1O9evWgnhP1p/hUXYIK99bTL7/8wsSJEwHXzzRz5kwWL15sycmYMHDw4EHKly9vk3hmQkQoX758tlqZUZ+gNm50ExSG66zlmzdv5tZbb+Wcc85hyJAhHDp0CBHh8ssvt5F5xoQRS05Zy+57FPXfcD/+6P42b+5vHGnt3r2boUOHcsYZZzBp0iTuu+8+fvnlFxuVZ4yJGpagfoSSJaFePb8jOdbmzZsZOXIk119/PWvXrmXkyJGUK1fO77CMMRFi/Pjx3HPPPQC8/vrrTJgwAYDVq1fToEEDGjZsyIYNG5g0aZJvMVqC+hGaNgW/i3mnpKQwdepUBg8eDED9+vX57bffmDBhAlWr+jrRsDEmwt11113ccsstAHz00Ud06tSJn3/+mc2bN/uaoKJ6FN/u3a7E0YMP+htHXFwcAwcOZNGiRTRo0IBHH32UokWLWjFXY/Khfv1g6dLc3WeDBvDCC5lvs2nTJq644gpWrFgBwKhRo45OzXHOOefwzTffkJyczLhx42jatOkxz3300UcpUaIEdevW5YUXXiAmJoavvvqKAwcOsGrVKho0aED37t3p379/7h5YFqI6QcXFQUoKtPWpIuDvv//O3XffzWeffcZpp53G22+/zU033RRxczMZY/y1f/9+li5dyrfffsttt912NIml1b59e+666y5KlCjB/fffT1xcHKNGjeLTTz/N44idqE5QX34JxYvDeefl7euqKiJCbGwsy5Yt4+mnn+bee++laNGieRuIMSbXZdXS8UPXrl0BuPDCC9m7dy+7d+/2N6AgRXWCmjMHWrVyVczzwp49e3j66af56aef+PLLL6lYsSIbN260ChDGmBwrWLAgKSkpRx8HXm+Udnh3fhkSH7WDJP74A9auzZvTe4cPH+all16iRo0aPPnkk1SsWJH9+/cDWHIyxuSKihUr8vfff7Nz504OHTp0zGm5qVOnAvDdd99RunRpSpcuHdQ+S5YsSUJCQkjiDUbUtqB++MH9bdMmtK+zatUqrrjiCjZu3MhFF13EM888Q+PGjUP7osaYqFOoUCEeeeQRmjZtyqmnnkqdOnWOrouNjaVhw4YkJSUxbty4oPd59tlnExMTwznnnEOPHj1skEReWb0aRCDgM8xV//zzD+XKlaNatWrUqVOHV155hXbt2uWbprUxJv/p27cvffv2PWZZ69at6datGy+k6Rzr0aMHPXr0ANwovlSB9wsVKsTXX38domizFrUJas0aqFoVYmNzd78rV65kyJAhrFixgtWrV1O0aFE+++yz3H0RY4yJAlGboNauhdq1c29/f/75J8OGDeOtt96iRIkSPPDAA7m3c2OMOUFxcXF+h3DCojJBpaS4BJVbFcxXrlzJueeeS3JyMn379uXBBx+kQoUKubNzY4yJUlE5im/jxuIkJkJOxiocPnyYRYsWAVC3bl3uu+8+Vq9ezfPPP2/JyRhjckFUJqiff3bToV90Ufafq6pMnz6devXqcdFFF/HPP/8gIowYMYLTTz89lyM1xpjoFZUJaunSMtSsCdktdTdv3jyaN29O586diY2NZerUqZQtWzY0QRpjTJSLugSVkgLLlpXJ9vVPa9eu5cILLyQ+Pp5x48axdOlSLr/8chs2boyJSNWqVWPHjh2+xhB1CerPP2HfvoJB9T9t2bLl6DTrZ5xxBu+//z5r167l1ltvtYKuxpiwparHlD3Kr6JuFN/Gje5vZt1FCQkJjBw5kmeffZaUlBQuu+wyTjrpJK699tq8CdIYk6+1bt36uGWdO3emT58+7N+/n/bt2x+3PvXC2R07dtCpU6dj1gUzVHzTpk20a9eOZs2asXjxYpo2bcovv/zCgQMH6NSpE4899hjgWkbdu3fnk08+ISkpienTp1OnTh127txJ165d+fPPP2nevDmqenTfzz333NEKFHfccQf9+vVj06ZNXHbZZZx33nn88MMPnHvuudx6660MGzaMv//+m3ffffe4aT2yK+paUJklqKSkJF555RVq1KjBiBEjuPLKK1mxYgUnnXRS3gZpjDEnYN26dfTp04eVK1fy7LPPsmjRIpYvX84333zD8uXLj25XoUIFlixZQu/evRk1ahQAjz32GC1atGDlypVcc801/PHHHwAsXryYt956iwULFjB//nzeeOMNfv75ZwDWr19/dATz6tWrmTRpEt999x2jRo3iiSeeyPHxRF0LasMGKFBAqVLl+L6jrVu3ct9993HeeecxcuRIzj33XB8iNMbkd5m1eIoVK5bp+goVKpzwxbVVq1blPG/+oGnTpjFmzBiSk5P566+/+PXXXzn77LMBjp4Naty4MR988AEA33777dH7HTp0ODoA7LvvvuOaa66hePHiR587b948rrrqKqpXr079+vUBqFevHhdffDEiQv369dm0adMJHUOgkLagROQyEVkjIutFZEg664uIyFRv/QIRqRbKeMC1oCpWPEhqEfHvvvuO++67D1XltNNOY9myZcydO9eSkzEm30lNIr/99hujRo3iq6++Yvny5XTo0OGY6TeKFCkCQExMDMnJySf8eqn7AShQoMDRxwUKFMjRfo/uM8d7yICIxACvAJcDdYGuIlI3zWa3A7tUtSbwPPB0qOJJddZZ0KLFDlavXs0111xDy5YtmTx5Mlu3bgWgdu3aNjLPGJOv7d27l+LFi1O6dGm2bdvGrFmzsnzOhRdeyKRJkwCYNWsWu3btAqBly5Z89NFH7N+/n3379vHhhx/SsmXLkMafKpQtqKbAelXdqKqHgSlAxzTbdATe9u6/B1wsIc4OvXvv4tChezjrrLP46quv+O9//8u6deuoVKlSKF/WGGPyzDnnnEPDhg2pU6cON954IxcEUddt2LBhfPvtt9SrV48PPviAKlWqANCoUSN69OhB06ZNadasGXfccQcNGzYM9SEAoe2DOhXYHPA4HmiW0Taqmiwie4DyQMgG3xcsWJAff/yRPn368PDDD9sACGNMRKhWrRorVqw4+nj8+PHpbhfYN9SkSZOj/V3ly5dn9uzZ6T5nwIABDBgwIOjXS7vuROWLQRIi0gvoBW7WyJxW533ttdcoX748K1euzIXowl9iYmK+rmh8IuyYo0O4HHPp0qXzbObZI0eO+DrLbU4dPHgw6M8slAnqTyCwmFBlb1l628SLSEGgNLAz7Y5UdQwwBqBJkyaa3jUG2REXF5fudQqRKtqOF+yYo0W4HPOqVasoWbJknrxWQkJCnr1WKKTO7huMUPZBLQRqiUh1ESkMdAFmpNlmBtDdu98J+FoDrw4zxph8wr66spbd9yhkCUpVk4F7gC+AVcA0VV0pIsNF5Cpvs7FAeRFZDwwAjhuKbowx4S42NpadO3daksqEqrJz505iszGNeUj7oFR1JjAzzbJHAu4fBK4PZQzGGBNqlStXJj4+nu3bt4f8tQ4ePJitL/lwEhsbS+XKlYPePl8MkjDGmHBWqFAhqlevnievFRcXl2fDvP0WdbX4jDHG5A+WoIwxxoQlS1DGGGPCkuS3UScish34PYe7qUAIq1WEoWg7XrBjjhZ2zJGhqqoeV9Yn3yWo3CAii1S1id9x5JVoO16wY44WdsyRzU7xGWOMCUuWoIwxxoSlaE1QY/wOII9F2/GCHXO0sGOOYFHZB2WMMSb8RWsLyhhjTJizBGWMMSYsRWyCEpHLRGSNiKwXkeOqpItIERGZ6q1fICLVfAgzVwVxzD1EZLuILPVud/gRZ24RkXEi8reIpDt1pzgvee/HchFplNcx5rYgjrm1iOwJ+IwfSW+7/EREThORuSLyq4isFJH/pLNNxHzWQR5vxH3O6VLViLsBMcAG4HSgMLAMqJtmmz7A6979LsBUv+POg2PuAfzP71hz8ZgvBBoBKzJY3x6YBQhwHrDA75jz4JhbA5/6HWcuH3MloJF3vySwNp1/2xHzWQd5vBH3Oad3i9QWVFNgvapuVNXDwBSgY5ptOgJve/ffAy4WEcnDGHNbMMccUVT1W+CfTDbpCExQZz5QRkQq5U10oRHEMUccVf1LVZd49xNw88udmmaziPmsgzzeqBCpCepUYHPA43iO/4CPbqNucsU9QPk8iS40gjlmgOu8UyDvichpeROab4J9TyJNcxFZJiKzRKSe38HkJu9UfENgQZpVEflZZ3K8EMGfc6pITVAmfZ8A1VT1bOBL/m1BmsixBFfX7BzgZeAjf8PJPSJSAngf6Keqe/2OJ9SyON6I/ZwDRWqC+hMIbB1U9palu42IFARKAzvzJLrQyPKYVXWnqh7yHr4JNM6j2PwSzL+DiKKqe1U10bs/EygkIhV8DivHRKQQ7sv6XVX9IJ1NIuqzzup4I/VzTitSE9RCoJaIVBeRwrhBEDPSbDMD6O7d7wR8rV7vYz6V5TGnOSd/Fe7cdiSbAdzijfA6D9ijqn/5HVQoicjJqX2pItIU9388P//wwjuescAqVX0ug80i5rMO5ngj8XNOT0RO+a6qySJyD/AFbnTbOFVdKSLDgUWqOgP3D2CiiKzHdTp38S/inAvymPuKyFVAMu6Ye/gWcC4Qkcm40UwVRCQeGAYUAlDV14GZuNFd64H9wK3+RJp7gjjmTkBvEUkGDgBd8vkPL4ALgJuBX0RkqbfsAaAKRORnHczxRuLnfBwrdWSMMSYsReopPmOMMfmcJShjjDFhyRKUMcaYsGQJyhhjTFiyBGWMMeaEZFW8OJ3tOwcUwZ2U1faWoEzUEREVkWcDHt8vIo/m0r7Hi0inXNrXAq9S9R9ybBX6armx/3Re74ETeE4PEflfKOIx+cJ44LJgNhSRWsBQ4AJVrQf0y+o5lqBMNDoEXBtuV957FU2OUtVmqtoAeARXbb+Bd9uUnf1kQ7YTlIlu6RUvFpEaIvK5iCwWkXkiUsdb1RN4RVV3ec/9O6v9W4Iy0SgZGAP0T7sibQtIRBK9v61F5BsR+VhENorIUyJyk4j8JCK/iEiNgN20FZFFIrJWRK7wnh8jIiNFZKFXrPfOgP3OE5EZwK9ZBS4iV3otq59FZI6IVPSWPyoiE0Xke9wF6CeJyJfeqZQ3ReT31IQsIt28uJeKyGgvtqeAot6ydzPazlt+q3dsP+EuKjUm0BjgXlVtDNwPvOotPwM4Q0S+F5H5IpJly8sSlIlWrwA3iUjpbDznHOAu4Ezclf5nqGpTXF3DewO2q4ab/qQD8LqIxAK348rvnAucC/QUkere9o2A/6jqGUHE8B1wnqo2xE2pMihgXV2grap2xVWY+No7lfIeXhUCETkTuAF3mqUBcAS4SVWHAAe8FtpNGW0nrlzWY7jE1MJ7TWOAowVuzweme1UwRuPmtwJXuagWrhJKV+ANESmT2f4istSRMVlR1b0iMgHoiysVE4yFqfXdRGQDMNtb/gvQJmC7aaqaAqwTkY1AHeBS4OyA1llp3H/Ww8BPqvpbkDFUBqZ6iaIwEPi8GaqaeiwtgGu8Y/1cRHZ5yy/GFQle6JVyKwqkd6olo+2aAXGqut17H6bifhkbA67Rs9v7UZNWPG4iySTgNxFZi/s/sDCznRkTrV7AtWyKByxLxvt/ISIFcEkg1aGA+ykBj1M49sde2vphipvp9d6AfqTqqpqa4PZlI+aXcbMi1wfuBGID1gWzHwHeDoijtqo+moPtjDnKmxbkNxG5HlzhWxE5x1v9Ea71hHe6+QxgY2b7swRlopaq/gNMwyWpVJv4dxqSq/AKsWbT9SJSwOuXOh1Ygyvi21vcNAqIyBkiUjyznWSgNP9OI9E9k+2+Bzp7r3UpUNZb/hXQSUT+z1tXTkSqeuuSUuPLZLsFQCsRKe9te/0JHIOJEOKKF/8I1BaReBG5HbgJuF1ElgEr+Xdm7y+AnSLyKzAXGKiqmVZgt1N8Jto9C9wT8PgN4GPvP9fnZK91k+oP4CegFHCXqh4UkTdxfVNLxJ0z2w5cfQL7fhR3fn8X8DVQPYPtHgMmi8jNuC+QrUCCqu4QkYeA2V4LMQm4G/gd17m9XESWeP1Qx22nqvPFDcn/EdgNLD2BYzARwuvvTM9xAyC8ausDvFtQrJq5MRFIRIoAR7xpWJoDr2XQL2BM2LIWlDGRqQowzWv9HMZdg2JMvmItKGOMMWHJBkkYY4wJS5agjDHGhCVLUMYYY8KSJShjjDFhyRKUMcaYsPT/+sSJcc7DJ4QAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "perf_h2o.plot_uplift(metric=\"gain\", normalize=True)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8782349075050278" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "perf_h2o.auuc_normalized()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Scoring histrory and importance of number of trees\n", "\n", "To speed up the calculation of AUUC, the predictions are binned into quantile histograms. To calculate precision AUUC the more bins the better. The more trees usually produce more various predictions and then the algorithm creates histograms with more bins. So the algorithm needs more iterations to get meaningful AUUC results. \n", "You can see in the scoring history table the number of bins as well as the result AUUC. There is also Qini value parameter, which reflects the number of bins and then is a better pointer of the model improvement. In the scoring history table below you can see the algorithm stabilized after building 6 trees. But it depends on data and model settings on how many trees are necessary." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timestampdurationnumber_of_treestraining_auuc_nbinstraining_auuctraining_auuc_normalizedtraining_qini_value
02022-10-05 18:19:390.030 sec0.00NaNNaNNaN
12022-10-05 18:19:412.314 sec1.01929533.0388040.833748128.764676
22022-10-05 18:19:478.248 sec2.04226565.1145230.749961657.003459
32022-10-05 18:19:5314.429 sec3.07225346.4913620.7155581373.757472
42022-10-05 18:20:0021.083 sec4.011225200.1519040.7114272198.512852
52022-10-05 18:20:0627.502 sec5.016125482.4700080.7193972929.990639
62022-10-05 18:20:1334.462 sec6.022825976.9637490.7333573551.194616
72022-10-05 18:20:2141.969 sec7.030526589.2084580.7506414101.071634
82022-10-05 18:20:2950.484 sec8.038827380.9731230.7729934660.496077
92022-10-05 18:20:3858.928 sec9.047927672.1567770.7812144908.823402
102022-10-05 18:20:471 min 8.312 sec10.056128108.4987830.7935325200.407577
112022-10-05 18:20:571 min 18.075 sec11.063328493.1049290.8043905441.356926
122022-10-05 18:21:071 min 28.003 sec12.071428828.6974600.8138645641.677461
132022-10-05 18:21:171 min 38.369 sec13.077329066.7877520.8205865783.946316
142022-10-05 18:21:281 min 48.844 sec14.082929181.5610070.8238265855.454094
152022-10-05 18:21:391 min 59.705 sec15.088429331.7346390.8280655941.529970
162022-10-05 18:21:502 min 10.810 sec16.092329380.7531710.8294495973.314654
172022-10-05 18:22:012 min 22.097 sec17.094229485.8605950.8324176031.958517
182022-10-05 18:22:122 min 33.476 sec18.095429638.0005890.8367126112.091663
192022-10-05 18:22:242 min 45.046 sec19.096829690.0000050.8381806139.355404
202022-10-05 18:22:362 min 57.092 sec20.097829708.2202360.8386946150.467978
\n", "
" ], "text/plain": [ " timestamp duration number_of_trees \\\n", "0 2022-10-05 18:19:39 0.030 sec 0.0 \n", "1 2022-10-05 18:19:41 2.314 sec 1.0 \n", "2 2022-10-05 18:19:47 8.248 sec 2.0 \n", "3 2022-10-05 18:19:53 14.429 sec 3.0 \n", "4 2022-10-05 18:20:00 21.083 sec 4.0 \n", "5 2022-10-05 18:20:06 27.502 sec 5.0 \n", "6 2022-10-05 18:20:13 34.462 sec 6.0 \n", "7 2022-10-05 18:20:21 41.969 sec 7.0 \n", "8 2022-10-05 18:20:29 50.484 sec 8.0 \n", "9 2022-10-05 18:20:38 58.928 sec 9.0 \n", "10 2022-10-05 18:20:47 1 min 8.312 sec 10.0 \n", "11 2022-10-05 18:20:57 1 min 18.075 sec 11.0 \n", "12 2022-10-05 18:21:07 1 min 28.003 sec 12.0 \n", "13 2022-10-05 18:21:17 1 min 38.369 sec 13.0 \n", "14 2022-10-05 18:21:28 1 min 48.844 sec 14.0 \n", "15 2022-10-05 18:21:39 1 min 59.705 sec 15.0 \n", "16 2022-10-05 18:21:50 2 min 10.810 sec 16.0 \n", "17 2022-10-05 18:22:01 2 min 22.097 sec 17.0 \n", "18 2022-10-05 18:22:12 2 min 33.476 sec 18.0 \n", "19 2022-10-05 18:22:24 2 min 45.046 sec 19.0 \n", "20 2022-10-05 18:22:36 2 min 57.092 sec 20.0 \n", "\n", " training_auuc_nbins training_auuc training_auuc_normalized \\\n", "0 0 NaN NaN \n", "1 19 29533.038804 0.833748 \n", "2 42 26565.114523 0.749961 \n", "3 72 25346.491362 0.715558 \n", "4 112 25200.151904 0.711427 \n", "5 161 25482.470008 0.719397 \n", "6 228 25976.963749 0.733357 \n", "7 305 26589.208458 0.750641 \n", "8 388 27380.973123 0.772993 \n", "9 479 27672.156777 0.781214 \n", "10 561 28108.498783 0.793532 \n", "11 633 28493.104929 0.804390 \n", "12 714 28828.697460 0.813864 \n", "13 773 29066.787752 0.820586 \n", "14 829 29181.561007 0.823826 \n", "15 884 29331.734639 0.828065 \n", "16 923 29380.753171 0.829449 \n", "17 942 29485.860595 0.832417 \n", "18 954 29638.000589 0.836712 \n", "19 968 29690.000005 0.838180 \n", "20 978 29708.220236 0.838694 \n", "\n", " training_qini_value \n", "0 NaN \n", "1 128.764676 \n", "2 657.003459 \n", "3 1373.757472 \n", "4 2198.512852 \n", "5 2929.990639 \n", "6 3551.194616 \n", "7 4101.071634 \n", "8 4660.496077 \n", "9 4908.823402 \n", "10 5200.407577 \n", "11 5441.356926 \n", "12 5641.677461 \n", "13 5783.946316 \n", "14 5855.454094 \n", "15 5941.529970 \n", "16 5973.314654 \n", "17 6031.958517 \n", "18 6112.091663 \n", "19 6139.355404 \n", "20 6150.467978 " ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "h2o_uplift_model.scoring_history()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Comparasion Tree-based approach and Generalized Weighed Uplift (LGWUM)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "LGWUM (Kane et al., 2014) is one of several methods available for Uplift Modeling, and uses an approach to Uplift Modelling better known as Class Variable Transformation. LGWUM assumes that positive uplift lies in treating treatment-group responders (TR) and control-group non-responders (CN), whilst avoiding treatment-group non-responders (TN) and control-group responders (CR). This is visually shown as:\n", "\n", "𝑈𝑝𝑙𝑖𝑓𝑡 𝐿𝐺𝑊𝑈𝑀 = P(TR)/P(T) + P(CN)/P(C) - P(TN)/P(T) - P(CR)/P(C)\n", "\n", "source: https://www.kaggle.com/code/hughhuyton/criteo-uplift-modelling/notebook" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gbm Model Build progress: |██████████████████████████████████████████████████████| (done) 100%\n" ] }, { "data": { "text/html": [ "
H2OGradientBoostingEstimator : Gradient Boosting Machine\n",
       "Model Key: GBM_model_python_1665008325837_24\n",
       "
\n", "
\n", " \n", "
\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Model Summary:
number_of_treesnumber_of_internal_treesmodel_size_in_bytesmin_depthmax_depthmean_depthmin_leavesmax_leavesmean_leaves
20.080.04011303.015.015.015.01341.06950.03990.975
\n", "
\n", "
\n",
       "\n",
       "[tips]\n",
       "Use `model.show()` for more details.\n",
       "Use `model.explain()` to inspect the model.\n",
       "--\n",
       "Use `h2o.display.toggle_user_tips()` to switch on/off this section.
" ], "text/plain": [ "H2OGradientBoostingEstimator : Gradient Boosting Machine\n", "Model Key: GBM_model_python_1665008325837_24\n", "\n", "\n", "Model Summary: \n", " number_of_trees number_of_internal_trees model_size_in_bytes min_depth max_depth mean_depth min_leaves max_leaves mean_leaves\n", "-- ----------------- -------------------------- --------------------- ----------- ----------- ------------ ------------ ------------ -------------\n", " 20 80 4.0113e+06 15 15 15 1341 6950 3990.97\n", "\n", "[tips]\n", "Use `model.show()` for more details.\n", "Use `model.explain()` to inspect the model.\n", "--\n", "Use `h2o.display.toggle_user_tips()` to switch on/off this section." ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from h2o.estimators.gbm import H2OGradientBoostingEstimator\n", "\n", "h2o_gbm_lgwum = H2OGradientBoostingEstimator(ntrees=ntree,\n", " max_depth=max_depth,\n", " min_rows=30,\n", " nbins=1000,\n", " score_each_iteration=False,\n", " seed=42)\n", "\n", "h2o_gbm_lgwum.train(y=response_column_lgwum, x=feature_cols, training_frame=h2o_train_df)\n", "h2o_gbm_lgwum" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gbm prediction progress: |███████████████████████████████████████████████████████| (done) 100%\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
predictp_cnp_crp_tnp_truplift_score
000.4631860.0390680.4586990.0390470.001324
100.4614040.0398720.4590650.039659-0.000040
200.4619670.0392860.4594310.0393150.000904
320.3947160.0582610.4895730.057449-0.047193
400.4635720.0390380.4583460.0390440.001654
.....................
279591420.2724730.2434910.2807080.203327-0.103698
279591520.3740560.0835780.4333720.1089940.036659
279591600.4631080.0390590.4586210.0392120.001971
279591700.4486330.0561530.4432690.051945-0.012692
279591800.4631350.0390640.4586490.0391520.001727
\n", "

2795919 rows × 6 columns

\n", "
" ], "text/plain": [ " predict p_cn p_cr p_tn p_tr uplift_score\n", "0 0 0.463186 0.039068 0.458699 0.039047 0.001324\n", "1 0 0.461404 0.039872 0.459065 0.039659 -0.000040\n", "2 0 0.461967 0.039286 0.459431 0.039315 0.000904\n", "3 2 0.394716 0.058261 0.489573 0.057449 -0.047193\n", "4 0 0.463572 0.039038 0.458346 0.039044 0.001654\n", "... ... ... ... ... ... ...\n", "2795914 2 0.272473 0.243491 0.280708 0.203327 -0.103698\n", "2795915 2 0.374056 0.083578 0.433372 0.108994 0.036659\n", "2795916 0 0.463108 0.039059 0.458621 0.039212 0.001971\n", "2795917 0 0.448633 0.056153 0.443269 0.051945 -0.012692\n", "2795918 0 0.463135 0.039064 0.458649 0.039152 0.001727\n", "\n", "[2795919 rows x 6 columns]" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "uplift_predict_lgwum = h2o_gbm_lgwum.predict(h2o_test_df)\n", "\n", "result = uplift_predict_lgwum.as_data_frame()\n", "result.columns = ['predict', 'p_cn', 'p_cr', 'p_tn', 'p_tr']\n", "result['uplift_score'] = result.eval('\\\n", " p_cn/(p_cn + p_cr) \\\n", " + p_tr/(p_tn + p_tr) \\\n", " - p_tn/(p_tn + p_tr) \\\n", " - p_cr/(p_cn + p_cr)')\n", "result" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmwAAAF+CAYAAAA/Y0YoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjLUlEQVR4nO3deZwlZX3v8c+XVVZlGRNlwEGCUeKCZgSjXETcUBS8iIgrKgkm1x3lilERMYt7EiMuoyJgriISYsaAYkQFgxvDKqDICKiDRBCQNSLL7/5xqmcOTS/F9NTp6p7P+/U6r67t1Pmdh5nmO0/VU0+qCkmSJPXXOrNdgCRJkqZmYJMkSeo5A5skSVLPGdgkSZJ6zsAmSZLUcwY2SZKknpuTgS3JMUmuSXJRy+MPSHJJkouTfL7r+iRJktakzMXnsCXZHbgFOL6qHjnNsTsCJwJ7VtUNSR5YVdeMok5JkqQ1YU72sFXVmcD1w9uS7JDka0nOSfKdJA9vdv0FcHRV3dC817AmSZLmlDkZ2CaxBHhdVf0p8BbgY832hwEPS3JWku8n2WvWKpQkSVoN6812AWtCkk2BJwJfSjK2ecPm53rAjsAewELgzCSPqqrfjrhMSZKk1TIvAhuDnsLfVtXOE+xbAfygqu4ArkjyUwYB7uwR1idJkrTa5sUl0aq6iUEYewFABh7T7P4yg941kmzN4BLp5bNQpiRJ0mqZk4EtyReA7wF/nGRFkoOBlwAHJ7kAuBjYtzn8NOC6JJcA3wIOq6rrZqNuSZKk1TEnH+shSZK0NpmTPWySJElrEwObJElSz825UaJbb711LVq0aLbLkCRJmtY555zzm6paMNPzzLnAtmjRIpYtWzbbZUiSJE0ryc/XxHm8JCpJktRzBjZJkqSeM7BJkiT1nIFNkiSp5wxskiRJPWdgkyRJ6jkDmyRJUs8Z2CRJknrOwCZJktRzBjZJkqSeM7BJkiT1nIFNkiSp5wxskiRJPbfebBcgaXLbH37KtMdc8d69R1CJJGk22cMmSZLUcwY2SZKknjOwSZIk9ZyBTZIkqecMbJIkST1nYJMkSeo5A5skSVLPGdgkSZJ6zsAmSZLUcwY2SZKknjOwSZIk9ZyBTZIkqecMbJIkST1nYJMkSeo5A5skSVLPGdgkSZJ6zsAmSZLUcwY2SZKknjOwSZIk9ZyBTZIkqecMbJIkST1nYJMkSeo5A5skSVLPdRbYkhyT5JokF02y/yVJLkzyoyTfTfKYrmqRJEmay7rsYTsW2GuK/VcAT66qRwHvAZZ0WIskSdKctV5XJ66qM5MsmmL/d4dWvw8s7KoWSZKkuawv97AdDHx1sp1JDkmyLMmya6+9doRlSZIkzb5ZD2xJnsIgsL11smOqaklVLa6qxQsWLBhdcZIkST3Q2SXRNpI8Gvg08Kyqum42a5EkSeqrWethS7IdcDLwsqr66WzVIUmS1Hed9bAl+QKwB7B1khXAu4D1AarqE8ARwFbAx5IA3FlVi7uqR5Ikaa7qcpToi6bZ/+fAn3f1+ZIkSfPFrA86kCRJ0tQMbJIkST1nYJMkSeo5A5skSVLPGdgkSZJ6zsAmSZLUcwY2SZKknjOwSZIk9ZyBTZIkqecMbJIkST1nYJMkSeo5A5skSVLPGdgkSZJ6zsAmSZLUc9MGtiQbttkmSZKkbrTpYftey22SJEnqwHqT7Ujyh8A2wEZJHguk2bU5sPEIapMkSRJTBDbgmcArgIXAh4e23wT8dYc1SZIkacikga2qjgOOS/L8qvrXEdYkSZKkIW3uYTsryWeSfBUgyU5JDu64LkmSJDXaBLbPAqcBD27Wfwq8sauCJEmSdE9tAtvWVXUicDdAVd0J3NVpVZIkSVqpTWC7NclWQAEkeQJwY6dVSZIkaaWpRomOORRYCuyQ5CxgAbB/p1VJkiRppWkDW1Wdm+TJwB8zeBbbpVV1R+eVSZIkCWg3NdULgI2q6mLgecAXkzyu68IkSZI00OYetndW1c1JdgOeCnwG+Hi3ZUmSJGlMm8A2NiJ0b+BTVXUKsEF3JUmSJGlYm8B2VZJPAi8ETk2yYcv3SZIkaQ1oE7wOYPDg3GdW1W+BLYHDuixKkiRJq7QZJXobcPLQ+tXA1V0WJUmSpFW8tClJktRzBjZJkqSeM7BJkiT1XJsH5+6X5LIkNya5KcnNSW4aRXGSJElqN5fo+4HnVtWPuy5GkiRJ99bmkuivDWuSJEmzp00P27IkXwS+DNw+trGqTp70HZIkSVpj2vSwbQ7cBjwDeG7zes50b0pyTJJrklw0yf4k+UiS5UkudEJ5SZKkibV5cO4rV/PcxwIfBY6fZP+zgB2b164MJpTfdTU/S5Ikad5qM0p0YZJ/a3rLrknyr0kWTve+qjoTuH6KQ/YFjq+B7wMPSPKg9qVLkiStHdpcEv0ssBR4cPP6SrNtprYBfjm0vqLZJkmSpCFtAtuCqvpsVd3ZvI4FFnRc1z0kOSTJsiTLrr322lF+tCRJ0qxrE9iuS/LSJOs2r5cC162Bz74K2HZofWGz7V6qaklVLa6qxQsWjDQrSpIkzbo2ge1VwAHAfwNXA/sDqzsQYdhS4OXNaNEnADdW1dVr4LySJEnzSptRoj8H9rmvJ07yBWAPYOskK4B3Aes35/wEcCrwbGA5g8eGrIkQKEmSNO9MGtiS/N+qen+SfwZq/P6qev1UJ66qF02zv4DXtC1UkiRpbTVVD9vYdFTLRlGIJEmSJjZpYKuqrzSLt1XVl4b3JXlBp1VJkiRppTaDDt7WcpskSZI6MNU9bM9iMChgmyQfGdq1OXBn14VJkiRpYKp72H7F4P61fYBzhrbfDLypy6IkSZK0ylT3sF0AXJDk81V1xwhrkiRJ0pBpn8MGLEry98BOwP3GNlbVQzurSpIkSSu1nfz94wzuW3sKcDzwL10WJUmSpFXaBLaNqup0IFX186o6Eti727IkSZI0ps0l0duTrANcluS1DCZo37TbsiRJkjSmTQ/bG4CNgdcDfwq8DDioy6IkSZK0SpvJ389uFm/BCdolSZJGbqoH536FCSZ9H1NV+3RSkSRJku5hqh62DzY/9wP+kFUjQ18E/LrLoiRJkrTKVA/OPQMgyYeqavHQrq8kWdZ5ZZIkSQLaDTrYJMnKh+Qm2R7YpLuSJEmSNKzNYz3eBHw7yeVAgIcAr+60KkmSJK3UZpTo15LsCDy82fSTqrq927IkSZI0ZqpRontW1TeT7Ddu1w5JqKqTO65NkiRJTN3D9mTgm8BzJ9hXgIFNkiRpBKYaJfqu5qcPy5UkSZpFU10SPXSqN1bVh9d8OZIkSRpvqkuim42sCkmSJE1qqkui7x5lIZIkSZrYtI/1SHI/4GDgT4D7jW2vqld1WJckSZIabWY6+ByDuUSfCZwBLARu7rIoSZIkrdImsP1RVb0TuLWqjgP2BnbttixJkiSNaRPY7mh+/jbJI4H7Aw/sriRJkiQNazOX6JIkWwDvBJYCmzbLkiRJGoE2ge2zVXUXg/vXHtpxPZIkSRqnzSXRK5IsSfLUJOm8IkmSJN1Dm8D2cOAbwGuAK5N8NMlu3ZYlSZKkMdMGtqq6rapOrKr9gJ2BzRlcHpUkSdIItOlhI8mTk3wMOIfBw3MP6LQqSZIkrdRmpoMrgfOAE4HDqurWrouSJEnSKm1GiT66qm7qvBJJkiRNqM09bIY1SZKkWdTqHjZJkiTNnk4DW5K9klyaZHmSwyfYv12SbyU5L8mFSZ7dZT2SJElzUZtBBxsCzwcWDR9fVUdN8751gaOBpwMrgLOTLK2qS4YOewdwYlV9PMlOwKnN50iSJKnRZtDBvwM3Mnikx+334dy7AMur6nKAJCcA+wLDga0YPNcNBpPK/+o+nF+SJGmt0CawLayqvVbj3NsAvxxaXwHsOu6YI4GvJ3kdsAnwtNX4HEmSpHmtzT1s303yqI4+/0XAsVW1EHg28Lkk96opySFJliVZdu2113ZUiiRJUj+1CWy7Aec0gwcuTPKjJBe2eN9VwLZD6wubbcMOZvBAXqrqewxmUdh6/ImqaklVLa6qxQsWLGjx0ZIkSfNHm0uiz1rNc58N7JhkewZB7UDgxeOO+QXwVODYJI9gENjsQpMkSRoyaWBLsnnz0NybV+fEVXVnktcCpwHrAsdU1cVJjgKWVdVS4M3Ap5K8icEAhFdUVa3O50mSJM1XU/WwfR54DoPRoQVkaF8BD53u5FV1KoNHdQxvO2Jo+RLgSfehXkmSpLXOpIGtqp7T/Nx+dOVIkiRpPKemkiRJ6jkDmyRJUs8Z2CRJknpu2sCWZIdmPlGS7JHk9Uke0HllkiRJAtr1sP0rcFeSPwKWMHgY7uc7rUqSJEkrtQlsd1fVncD/Bv65qg4DHtRtWZIkSRrTJrDdkeRFwEHAfzTb1u+uJEmSJA1rE9heCfwZ8LdVdUUz1dTnui1LkiRJY6adS7SZjeD1Q+tXAO/rsihJkiStMm1gS/Ik4EjgIc3xAaqqpp2aSpIkSTM3bWADPgO8icGcond1W44kSZLGaxPYbqyqr3ZeiSRJkibUJrB9K8kHgJOB28c2VtW5nVUlSZKkldoEtl2bn4uHthWw55ovR5IkSeO1GSX6lFEUIkmSpIm1mUv0/kk+nGRZ8/pQkvuPojhJkiS1e3DuMcDNwAHN6ybgs10WJUmSpFXa3MO2Q1U9f2j93UnO76geSZIkjdOmh+1/kuw2ttI8SPd/uitJkiRJw9r0sP0VcFxz31qA64FXdFmUJEmSVmkzSvR84DFJNm/Wb+q6KEmSJK0yaWBL8tKq+pckh47bDkBVfbjj2iRJksTUPWybND83m2BfdVCLJEmSJjBpYKuqTzaL36iqs4b3NQMPJEmSNAJtRon+c8ttkiRJ6sBU97D9GfBEYMG4+9g2B9btujBJkiQNTHUP2wbAps0xw/ex3QTs32VRkiRJWmWqe9jOAM5IcmxV/XyENUmSJGlImwfn3pbkA8CfAPcb21hVe3ZWlSRJklZqM+jg/wE/AbYH3g1cCZzdYU2SJEka0iawbVVVnwHuqKozqupVgL1rkiRJI9Lmkugdzc+rk+wN/ArYsruSJEmSNKxNYPubZuL3NzN4/trmwJs6rUqSJEkrtQlsP6iqG4Ebgad0XI8kSZLGaXMP21lJvp7k4CRbdF6RJEmS7mHawFZVDwPeweCxHuck+Y8kL+28MkmSJAHtetioqh9W1aHALsD1wHGdViVJkqSVpg1sSTZPclCSrwLfBa5mENwkSZI0Am162C4AdgaOqqqHVdVbq+qcNidPsleSS5MsT3L4JMcckOSSJBcn+Xz70iVJktYOU44STbIucHJVvfm+nrh579HA04EVwNlJllbVJUPH7Ai8DXhSVd2Q5IH39XMkSZLmuyl72KrqLuCJq3nuXYDlVXV5Vf0eOAHYd9wxfwEcXVU3NJ93zWp+liRJ0rzV5jls5ydZCnwJuHVsY1WdPM37tgF+ObS+Ath13DEPA0hyFrAucGRVfW38iZIcAhwCsN1227UoWZIkaf5oE9juB1zHPecPLWC6wNb283cE9gAWAmcmeVRV/Xb4oKpaAiwBWLx4ca2Bz5UkSZozpg1sVfXK1Tz3VcC2Q+sLm23DVjCYSeEO4IokP2UQ4M5ezc+UJEmad9o81uNhSU5PclGz/ugk72hx7rOBHZNsn2QD4EBg6bhjvsygd40kWzO4RHp5+/IlSZLmvzaP9fgUg5GcdwBU1YUMwteUqupO4LXAacCPgROr6uIkRyXZpznsNOC6JJcA3wIOq6rr7vvXkCRJmr/a3MO2cVX9MMnwtjvbnLyqTgVOHbftiKHlAg5tXpIkSZpAmx623yTZgcFAA5Lsz2C2A0mSJI1Amx621zAYofnwJFcBVwBO/i5JkjQibUaJXg48LckmwDpVdXP3ZUmSJGlMm1Gib0iyOXAb8A9Jzk3yjO5LkyRJErS7h+1VVXUT8AxgK+BlwHs7rUqSJEkrtQlsY8NDnw0cX1UXD22TJElSx9oEtnOSfJ1BYDstyWbA3d2WJUmSpDFtRokeDOwMXF5VtyXZCljd6aokSZJ0H7UZJXp3kkXAS5MU8F9V9W+dVyZJkiSg3SjRjwF/CfwIuAh4dZKjuy5MkiRJA20uie4JPKKZRookxwGXdFqVJEmSVmoz6GA5sN3Q+rbAZd2UI0mSpPEm7WFL8hUG84duBvw4yQ+bXbsAP5zsfZIkSVqzprok+sGRVSFJkqRJTRrYquqMseUkfwA8vln9YVVd03VhkiRJGmgzSvQABpdAXwAcAPwgyf5dFyZJkqSBNqNE3w48fqxXLckC4BvASV0WJkmSpIE2o0TXGXcJ9LqW75MkSdIa0KaH7WtJTgO+0Ky/EDi1u5IkSZI0rM3UVIcl2Q/Yrdm0xKmpJEmSRqdNDxtVdTJwcse1SJIkaQLeiyZJktRzBjZJkqSemzSwJTm9+fm+0ZUjSZKk8aa6h+1BSZ4I7JPkBCDDO6vq3E4rkyRJEjB1YDsCeCewEPjwuH0F7NlVUZIkSVplqrlETwJOSvLOqnrPCGuSJEnSkDbPYXtPkn2A3ZtN366q/+i2LEmSJI1pM/n73wNvAC5pXm9I8nddFyZJkqSBNg/O3RvYuaruBkhyHHAe8NddFiZJkqSBts9he8DQ8v07qEOSJEmTaNPD9vfAeUm+xeDRHrsDh3dalSRJklZqM+jgC0m+DTy+2fTWqvrvTquSJEnSSm0nf78aWNpxLZIkSZqAc4lKkiT1nIFNkiSp56YMbEnWTfKTURUjSZKke5sysFXVXcClSbYbUT2SJEkap80l0S2Ai5OcnmTp2KvNyZPsleTSJMuTTPookCTPT1JJFrctXJIkaW3RZpToO1fnxEnWBY4Gng6sAM5OsrSqLhl33GYMpr76wep8jiRJ0nw3bQ9bVZ0BXAms3yyfDZzb4ty7AMur6vKq+j1wArDvBMe9B3gf8Lu2RUuSJK1N2kz+/hfAScAnm03bAF9uce5tgF8Ora9otg2f+3HAtlV1yjQ1HJJkWZJl1157bYuPliRJmj/a3MP2GuBJwE0AVXUZ8MCZfnCSdYAPA2+e7tiqWlJVi6tq8YIFC2b60ZIkSXNKm8B2e3NJE4Ak6wHV4n1XAdsOrS9sto3ZDHgk8O0kVwJPAJY68ECSJOme2gS2M5L8NbBRkqcDXwK+0uJ9ZwM7Jtk+yQbAgQxNb1VVN1bV1lW1qKoWAd8H9qmqZff5W0iSJM1jbQLb4cC1wI+AVwOnAu+Y7k1VdSfwWuA04MfAiVV1cZKjkuyz+iVLkiStXaZ9rEdV3Z3kOAaP3Sjg0qpqc0mUqjqVQcAb3nbEJMfu0eackiRJa5tpA1uSvYFPAD8DAmyf5NVV9dWui5MkSVK7B+d+CHhKVS0HSLIDcApgYJMkSRqBNvew3TwW1hqXAzd3VI8kSZLGmbSHLcl+zeKyJKcCJzK4h+0FDEaASpIkaQSmuiT63KHlXwNPbpavBTbqrCJJkiTdw6SBrapeOcpCJEmSNLE2o0S3B14HLBo+vqp8lpokSdIItBkl+mXgMwxmN7i702okSZJ0L20C2++q6iOdVyJJkqQJtQls/5TkXcDXgdvHNlbVuZ1VJUmSpJXaBLZHAS8D9mTVJdFq1iVJktSxNoHtBcBDq+r3XRcjSZKke2sz08FFwAM6rkOSJEmTaNPD9gDgJ0nO5p73sPlYD0mSpBFoE9je1XkVkiRJmtS0ga2qzhhFIZIkSZpYm5kObmYwKhRgA2B94Naq2rzLwiRJkjTQpodts7HlJAH2BZ7QZVGSJElapc0o0ZVq4MvAM7spR5IkSeO1uSS639DqOsBi4HedVSRJkqR7aDNK9LlDy3cCVzK4LCpJkqQRaHMP2ytHUYgkSZImNmlgS3LEFO+rqnpPB/VIkiRpnKl62G6dYNsmwMHAVoCBTZIkaQQmDWxV9aGx5SSbAW8AXgmcAHxosvdJGq3tDz9l2mOueO/eI6hEktSVKe9hS7IlcCjwEuA44HFVdcMoCpMkSdLAVPewfQDYD1gCPKqqbhlZVZIkSVppqgfnvhl4MPAO4FdJbmpeNye5aTTlSZIkaap72O7TLAiSJEnqhqFMkiSp5wxskiRJPWdgkyRJ6jkDmyRJUs8Z2CRJknrOwCZJktRzBjZJkqSeM7BJkiT13JRziUqaH5wgXpLmtk572JLsleTSJMuTHD7B/kOTXJLkwiSnJ3lIl/VIkiTNRZ0FtiTrAkcDzwJ2Al6UZKdxh50HLK6qRwMnAe/vqh5JkqS5qssetl2A5VV1eVX9HjgB2Hf4gKr6VlXd1qx+H1jYYT2SJElzUpeBbRvgl0PrK5ptkzkY+GqH9UiSJM1JvRh0kOSlwGLgyZPsPwQ4BGC77bYbYWWSJEmzr8setquAbYfWFzbb7iHJ04C3A/tU1e0TnaiqllTV4qpavGDBgk6KlSRJ6qsuA9vZwI5Jtk+yAXAgsHT4gCSPBT7JIKxd02EtkiRJc1Znga2q7gReC5wG/Bg4saouTnJUkn2awz4AbAp8Kcn5SZZOcjpJkqS1Vqf3sFXVqcCp47YdMbT8tC4/X5IkaT5waipJkqSeM7BJkiT1nIFNkiSp5wxskiRJPWdgkyRJ6jkDmyRJUs8Z2CRJknrOwCZJktRzBjZJkqSeM7BJkiT1nIFNkiSp5wxskiRJPWdgkyRJ6jkDmyRJUs8Z2CRJknrOwCZJktRzBjZJkqSeM7BJkiT13HqzXYCkftj+8FOmPeaK9+49gkokSePZwyZJktRzBjZJkqSeM7BJkiT1nIFNkiSp5wxskiRJPWdgkyRJ6jkDmyRJUs/5HDZplrR57lnfTFezz2mTpG7YwyZJktRzBjZJkqSeM7BJkiT1nIFNkiSp5xx0IGmNcQJ5SeqGPWySJEk9Zw+b1IG5+MgOSVJ/GdgkjZSXTSXpvvOSqCRJUs/ZwybdR17u7J69cJJ0TwY2aYhhbO4w1Elam3Qa2JLsBfwTsC7w6ap677j9GwLHA38KXAe8sKqu7LImrb0MY2ufNfXf3OAnabZ1FtiSrAscDTwdWAGcnWRpVV0ydNjBwA1V9UdJDgTeB7ywq5o0Nxm0NNtG+WfQcChpIl32sO0CLK+qywGSnADsCwwHtn2BI5vlk4CPJklVVYd1rdUMP1K/+Xd05gy9mo+6DGzbAL8cWl8B7DrZMVV1Z5Ibga2A30x20h9ddaO/0CRJk/L/EZqP5sSggySHAIc0q7dc+b7nXDqb9Uxga6YImboH26od26k926od26kd26k926qdP14TJ+kysF0FbDu0vrDZNtExK5KsB9yfweCDe6iqJcCSjuqcsSTLqmrxbNcxF9hW7dhO7dlW7dhO7dhO7dlW7SRZtibO0+WDc88GdkyyfZINgAOBpeOOWQoc1CzvD3zT+9ckSZLuqbMetuaetNcCpzF4rMcxVXVxkqOAZVW1FPgM8Lkky4HrGYQ6SZIkDen0HraqOhU4ddy2I4aWfwe8oMsaRqS3l2t7yLZqx3Zqz7Zqx3Zqx3Zqz7ZqZ420U7wCKUmS1G9O/i5JktRzBraWkmyZ5D+TXNb83GKS4w5qjrksyUET7F+a5KLuK549M2mrJBsnOSXJT5JcnOS9E713LkuyV5JLkyxPcvgE+zdM8sVm/w+SLBra97Zm+6VJnjnSwkdsddspydOTnJPkR83PPUde/IjN5M9Us3+7JLckecvIip4FM/y79+gk32t+L/0oyf1GWvwIzeDv3vpJjmva58dJ3jby4kesRVvtnuTcJHcm2X/cvinzwr1Ula8WL+D9wOHN8uHA+yY4Zkvg8ubnFs3yFkP79wM+D1w029+nr20FbAw8pTlmA+A7wLNm+zutwbZZF/gZ8NDm+10A7DTumP8DfKJZPhD4YrO8U3P8hsD2zXnWne3v1MN2eizw4Gb5kcBVs/19+tpWQ/tPAr4EvGW2v08f24nB/d4XAo9p1rfy796E7fRi4IRmeWPgSmDRbH+nWW6rRcCjGcybvv/Q9inzwkQve9ja2xc4rlk+DnjeBMc8E/jPqrq+qm4A/hPYCyDJpsChwN90X+qsW+22qqrbqupbAFX1e+BcBs/wmy9WTtnWfL+xKduGDbffScBTk6TZfkJV3V5VVwDLm/PNR6vdTlV1XlX9qtl+MbBRkg1HUvXsmMmfKZI8D7iCQVvNZzNpp2cAF1bVBQBVdV1V3TWiukdtJu1UwCYZPFd1I+D3wE2jKXtWTNtWVXVlVV0I3D3uvZPmhckY2Nr7g6q6uln+b+APJjhmoum4tmmW3wN8CLitswr7Y6ZtBUCSBwDPBU7voMbZMu33ZtyUbcDYlG1t3jtfzKSdhj0fOLeqbu+ozj5Y7bZq/iH5VuDdI6hzts3kz9TDgEpyWnN56/+OoN7ZMpN2Ogm4Fbga+AXwwaq6vuuCZ9FMfiff5/fOiampRiXJN4A/nGDX24dXqqqStB5em2RnYIeqetP4e0fmqq7aauj86wFfAD5SVZevXpVamyX5E+B9DHpHNLEjgX+oqluaDjdNbD1gN+DxDP7RfXqSc6pqPv1jck3YBbgLeDCDy3zfSfINf4evGQa2IVX1tMn2Jfl1kgdV1dVJHgRcM8FhVwF7DK0vBL4N/BmwOMmVDNr8gUm+XVV7MEd12FZjlgCXVdU/zrzaXpnJlG1t3jtfzGhquyQLgX8DXl5VP+u+3Fk1k7baFdg/yfuBBwB3J/ldVX2086pHbybttAI4s6p+A5DkVOBxzK/e/zEzaacXA1+rqjuAa5KcBSxmcH/WfDST38nT/T/wXrwk2t7wNFoHAf8+wTGnAc9IskUGIyOfAZxWVR+vqgdX1SIG/0r76VwOay2sdlsBJPkbBr8A3th9qSM3kynblgIHNiO0tgd2BH44orpHbbXbqbmUfgqDgS9njargWbTabVVV/6uqFjW/m/4R+Lt5GtZgZn/3TgMelcEo9vWAJwOXjKjuUZtJO/0C2BMgySbAE4CfjKTq2dGmrSYz6f8DJzXboyzmyovB9fnTgcuAbwBbNtsXA58eOu5VDG4GXw68coLzLGL+jxJd7bZi8K+MAn4MnN+8/ny2v9Mabp9nAz9lMLro7c22o4B9muX7MRixt5xBIHvo0Hvf3rzvUubR6Nk12U7AOxjcR3P+0OuBs/19+thW485xJPN4lOhM2wl4KYOBGRcB75/t79LHdgI2bbZfzCDQHjbb36UHbfV4Bj20tzLohbx46L1T5oXxL2c6kCRJ6jkviUqSJPWcgU2SJKnnDGySJEk9Z2CTJEnqOQObJElSzxnYJPVSkkVJLhq37cgkb5nmfa9I8tFm+S+TvLxZfniS85Ocl2SHJC/urnpJWrMMbJLmrar6RFUd36w+Dzipqh7L4OnkIwlszYNWJWlG/EUiaU5K8m3gAgZPnV8PeFVV/XDcMUcCtzB4iOcbgbuSPBXYCHhEkvOB46rqH4be8yDgi8DmzXn/qqq+k2Qv4O+AdYHfVNVTk2wJHAM8lMEck4dU1YXN5+7QbP9FktcDnwC2az7mjbV2zMIgaQ0xsEmayzauqp2T7M4gOD1yooOq6tQknwBuqaoPJtmDwVP9nzPB4S9mMKXc3yZZF9g4yQLgU8DuVXVFE9QA3g2cV1XPS7IncDywc7NvJ2C3qvqfJJ9nMMn6fyXZjsEUNI9YA99f0lrCwCapryabhmV4+xcAqurMJJs384jO1NnAMUnWB75cVec3Ae/Mqrqi+bzrm2N3A57fbPtmkq2SbN7sW1pV/9MsPw3YKcnYZ2yeZNOqumUN1CtpLWBgk9RX1wFbjNu2JXDF0Pr4UDfjufaa8Lc7sDdwbJIPAzesxqluHVpeB3hCVf1upvVJWjs56EBSLzW9T1c3lxppLkPuBfzX0GEvbPbtBtxYVTe2PP3NwGYT7UjyEODXVfUp4NPA44DvA7sn2X6oFoDvAC9ptu3B4N62myY47deB1w19xs4t65QkwB42Sf32cuDoppcL4N1V9bOh/b9Lch6wPvCq+3DeCxkMQLgAOHZ40AGwB3BYkjsYDFh4eVVdm+QQ4OQk6wDXAE8HjmRw+fRCBoMODprk817ffI8LGfzePRP4y/tQr6S1XKpmfAVBkkauGSX6lqpaNtu1SFLXvCQqSZLUc/awSZIk9Zw9bJIkST1nYJMkSeo5A5skSVLPGdgkSZJ6zsAmSZLUcwY2SZKknvv/fhKpiSWcGj8AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plot_uplift_score(result.uplift_score)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%\n" ] }, { "data": { "text/html": [ "
ModelMetricsBinomialUplift: \n",
       "** Reported on test data. **\n",
       "\n",
       "AUUC: 20878.6347962265\n",
       "AUUC normalized: 0.7239439144140724
\n", "
\n", " \n", "
\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
AUUC table (number of bins: 79): All types of AUUC value
uplift_typeqiniliftgain
AUUC value17782.56991260.024070320878.6347962
AUUC normalized0.72540120.02407030.7239439
AUUC random value12420.20480840.005226214612.0004307
\n", "
\n", "
\n", "
Qini value: 5362.365104164344
\n", "
\n", " \n", "
\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", "\n", "\n", "\n", "
AECU values table: All types of AECU value
uplift_typeqiniliftgain
AECU value5362.36510420.01884416266.6343655
\n", "
\n", "
" ], "text/plain": [ "ModelMetricsBinomialUplift: \n", "** Reported on test data. **\n", "\n", "AUUC: 20878.6347962265\n", "AUUC normalized: 0.7239439144140724\n", "\n", "AUUC table (number of bins: 79): All types of AUUC value\n", "uplift_type qini lift gain\n", "----------------- -------- ---------- --------\n", "AUUC value 17782.6 0.0240703 20878.6\n", "AUUC normalized 0.725401 0.0240703 0.723944\n", "AUUC random value 12420.2 0.00522619 14612\n", "\n", "Qini value: 5362.365104164344\n", "\n", "AECU values table: All types of AECU value\n", "uplift_type qini lift gain\n", "------------- ------- --------- -------\n", "AECU value 5362.37 0.0188441 6266.63" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lgwum_predict = h2o.H2OFrame(result['uplift_score'].tolist())\n", "perf_lgwum = h2o.make_metrics(lgwum_predict, h2o_test_df[response_column], treatment=h2o_test_df[treatment_column], auuc_type=\"gain\", auuc_nbins=81)\n", "perf_lgwum" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABIQklEQVR4nO3deZzN1f/A8dd77EWyJeFrVJYo+1YqilTaS5uKaZO0IUlaCJWK9n6JUlIiLaJCKiPZsu+yy9j3fRvz/v1xPqNrzHJn/dy59/18PD6Pe+/5bO8zl3nP53zO5xxRVYwxxphQE+V3AMYYY0xyLEEZY4wJSZagjDHGhCRLUMYYY0KSJShjjDEhyRKUMcaYkGQJykQsEekpIl/6HUcwRCRGRP4K+LxfRM713hcSkTEiskdERvoXZWgQkbEi0jartzU5zxKUyXEi0lpEZnm/ZDd5vyQu9Tuu1IjI5yLSJxP7x4rIQ0nKmopIXEaOp6qFVXW197EVUBoooaq3BxOrOE+KyCIROSAicSIyUkQuykg8oURVr1XVIVm9rcl5lqBMjhKRzsA7wKu4X6r/A/4PuMnHsHK7CsByVY1Pxz7vAk8BTwLFgcrAKOC69J5cRPKmdx9jgqKqttiSIwtQFNgP3J7KNp8DfQI+NwXiAj6vBZ4BFgAHgE9xiW4ssA/4DSiW3L4B+zf33vcEvgxYNxLYDOwB/gSqe+XtgGPAUS/+MV75OcB3wDZgDfBkKvWKBR5KUpa0bopLGKuB7cCbQJS3Lgb4K8m25wMve3Ed82J7JLlYk5y3EnAcaBBsvCmc/zFghVf3j4B+SY7xI9A5vT+rIP4d5QH6eT+j1V4cCuRNGnti3N72u7xzX5va92JL6Cx2BWVy0sVAQeCHTB7nNuAq3F/9N+CSU3egFK5V4MkMHncs7pf3WcAc4CsAVR3ovX9DXdPaDSISBYwB5gNlgWZARxG5OqOV8twC1APq4K4qH0htY1XtgbsaHeHF9nHSWJPZrRkuMf6dyVhvBhoC1YCvgTtFRABEpBjQAhieDT+rh4Hrgdq4n1WrNLZvCPwDlATeAD5NjNOENktQJieVALZr+pqikvO+qm5R1Q3AZGCGqs5V1cO45Fc7IwdV1cGquk9Vj+CurmqKSNEUNq8PlFLVXqp6VN39oEHAXRk5d4DXVXWnqv6Lawq9O5PHS04JYFMWHOc1L9ZDuO9Bgcu8da2Aaaq6kaz/Wd0BvKOq61V1J/BaGtuvU9VBqnocGAKUwV11mxBnbccmJ+0ASopI3kwmqS0B7w8l87lweg8oInmAV4DbcVdiCd6qkrgmv6QqAOeIyO6Asjy4X9TJiQfyJSnLh2uOC7Q+4P06XNNYVtuB+yWdWSdiVVUVkeG4hPon0BpI7CEZ9M9KRP4HLAk4bnLf5Tmc+nNKzeaA4x30Lp7S/W/E5Dy7gjI5aRpwBNc0lJIDwGkBn8/OxPlOOpaXhEqlsG1rXJNac9y9sujE3bzXpMP+rwfWqOqZAUsRVW2ZwvH/DThmooqc+su1fMD7/wEbUzheatKaouB3oJyI1Etlm2C+h6Tn+RpoJSIVcM1q33nlQf+sVPVfr2mycArJCdzVX9KfkwlDlqBMjlHVPcBLwIcicrOInCYi+UTkWhF5w9tsHtBSRIqLyNlAx0yccjlQUESuE5F8wAtAgRS2LYJLnjtwv5hfTbJ+C3BuwOe/gX0i8qz3HFIeEblQROqncPwRwP0i0sDr4l0Z6AQMT7LdMyJSTETK43rZjQiyrqnFehJVXYHrOfm119U9v4gUFJG7RKSbt9k84FbvOzofeDCtk6rqXFzHhU+A8aq621uV3p9VWr4BnhSRct69rm5p7WByJ0tQJkepan+gMy5ZbMP9df04roszwFDczfS1wK9k7Bd04rn2AB1wvzA34K4KUnru6Avc1cwGXBPT9CTrPwWqichuERnl3c+4HqiF6xmW+Is52XtWqjoe94v0M1yT4S+4+yEDk2z6IzAblyB+9s6bXifFmsI2TwIfAB8Cu4FVuA4aY7z1b+N6Am7x4vwqyHMPw12FDkssSO/PKgiDgPG4fydzgO8zeBwT4kTVJiw0JhSIiAKVVHWl37HkJiISjUt8+bKgA44JIXYFZYwxJiRZgjLGGBOSrInPGGNMSLIrKGOMMSHJEpQxxpiQZAnKhDVvmotdIlIgSVmKU1+ISLSIaNJRupNOYyEiZUTkU2/KkH0iskxEXhaR09MR31ki8rWIbPTmc5oiIg2TnGO0t169HmtJYzrqTV2SuOQJWH+HiCz14lsiIjcHrIsRkeNJ9m0aRMwvebE0DygrLiIjRGSHiGwXka9E5IyA9bVEZLJXxzgReTFgXSMRmSAiO0Vkm7hpP7JipAuTy1mCMmHL+2V+GW7Egxuz+NjFcSNjFAIuVtUiuAFszwTOS8ehCgMzgbq4aS+GAD+LSOIoCgnAONwAuSl5I3D0Be+5I0SkLG64oc7AGbhR4IeJyFkB+05Lsm9sGvU+DzccVNKx/PoAxXCjY5yHG+uuZ8D6YbghkIoDTYAOIpL4nRTDPQ8WjRsWaR/ueTET4SxBmXDWBvfA7edAVs+a2hn3i/ReVV0L4A1e+pSqLgj2IKq6WlXfUtVNqnrcGzk9P1DFW79FVf8Pl8TSqxywW1XHqvMz7mHl9CTQpD4EnsU9xBuoIjBKVfd6D0j/AFQPWB8NfOXVcRVuCozqAF58I719D+IeIG6ciRhNmLAEZcJZG9wICF8BV4tIVo5g3Rz4XlUTUtpARBZ4ozkkt/xfCvvUwiWo9Dys28FrHpstIoFXWrOApSJyoze80M244ZwCE2htr0luuYi8mLRZM0lstwNHVPWXZFZ/CFzvDdNUDHfFNzZg/TtAG3FDW1XBTb3yWwqnuhxYnHqVTSSw0cxNWBI3hXwF4BtV3S4iq3ADwr6dRadIc8oKVa2RngN692yGAi97VyHBeA94Gjd8UgtghIhsVtUpqnpcRL7ANa8VxF313K6qB7x9/wQuxA3xVB03rFQ8yUxfISJFcOMTXpVCHHNwiXWH9/l33Hh/iX7CDSfVBTeSeS9VPeWqUERq4MZrtBmWjV1BmbDVFvhVVbd7n4fxXzNfWlNfxAeUpbRNVk1ZAYCIFMKNgzddVdOa3+gEVZ2jqjtUNd67svkKuNU7ZnPcBH1NccmjCfCJd5WW2Ly4RlUTVHUh0IuUJ//rCQxNbM5Mxje4wXmL4O53rcKbbsO7XzfOO35B3EjkV4tIhyQ/g/NxV11PqWpK05aYCGIJyoQd75f9HUATEdksIptxI4fXFJGapD31xSZcIkptm9+AW8TNFptSHIuT9JALXAYEbFcAN1huHG7K9sxQ/psipBbwp6rO8pLQTGAGrnkyrX2TaoYbQTzx51ke+EZEng0418eqekBV9wMDgMTpNM4FjqvqF14ijcON4n5iug1xU3T8BvRW1aHprrUJTxoC887bYktWLrhJ83bi5gk6O2D5E+gPXA1sBRrgfiFXBpYC7QOO8TVulOwSuCunu3Gjfpf21hfHjbg+FKjglZUF3gJqpCPWfLgrp1FA3hS2KQicjksgVYCCAeta4XoCRuGa+PYBTb11TXAjh9fyPtfGXfm18D5fG1CfqsAioEcKMZRI8rNcj+vNV9hbPxF4H9ersRCueW+qt+4M72fX2ovzbFwPyFcDfm6rgC5+/9uxJbQW3wOwxZasXnDNSf2TKb8DN7tqXuAB3I34vbgOCd2AqIBti/HfNB27gClA4yTHOwcY7B1zH7AM6AGclo5Ym3iJ5yCwP2C5LGAbTboErJuMu/+0Fzf9xF1Jjv+4V799wGrg6YB1/XDTaRzw1vXCjQieuH4xcE8Kca8Fmgd8rohLtDtwfxyMw43Mnrj+SlxPxD3ez2tQ4s/J+5lpkvrv9/vfkS3+LzYWnzHGmJBk96CMMcaEJEtQxhhjQpIlKGOMMSHJEpQxxpiQFHEjSZQsWVKjo6MzvP+BAwc4/fSgB6vOtaye4SVS6gmRU9dwqufs2bO3q2qppOURl6Cio6OZNWtWhvePjY2ladOmWRdQiLJ6hpdIqSdETl3DqZ4isi65cmviM8YYE5KyLUGJSHkRmehNkrZYRJ7yynuKyAYRmectgcOdPCciK0XkHxG5OqD8Gq9spYh0CyivKCIzvPIRIpI/u+pjjDEmZ2XnFVQ87qn1akAj4DERqeate1tVa3nLLwDeurtwoypfA/yfN0VAHtxQ/tcC1YC7A47zunes83FP+z+YjfUxxhiTg7ItQambgG2O934fbqyzsqnschMwXFWPqOoa3PAsDbxlpbqRl4/iBpm8SUQEN3zKt97+Q4Cbs6UyxhhjclyOdJLwpt6ujRtJuTHwuIi0wU2o9rSq7sIlr+kBu8XxX0Jbn6S8IW7wyt2qGp/M9knP3w5oB1C6dGliY2MzXJf9+/dnav/cwuoZXiKlnhA5dY2EemZ7ghKRwsB3QEdV3SsiHwG9cYND9saNLv1AdsagbhrtgQD16tXTzPR8CaeeM6mxeoaXSKknRE5dI6Ge2dqLT0Ty4ZLTV6r6PYCqblHV4+qmyh6Ea8IDN2p0+YDdy3llKZXvAM4MmKI6sdwYY0w2O3oUsnus8Wy7gvLuEX0KLFXVtwLKy6hq4lTZt+DmoAEYDQwTkbdw0xhUAv7GzddTSUQq4hLQXUBrVVURmYibD2c4brbUH7OrPsYYEy5U4dAh2L8f9u377zXwfWplixbBpk0AY2jQoBG//16KwoWzPs7sbOJrDNwHLBSReV5Zd1wvvFq4Jr61eDOIqupiEfkGWILrAfiYqh4HEJHHgfFAHmCwqi72jvcsMFxE+gBzcQnRGGPCSkKCSw6BSWPevDMzlFwSXxMSgjt3/vxQpAgULvzf64UXQsWKsGDBRv79dwCHD7+YuxKUqv5F8tNH/5LKPq8AryRT/kty+6nqav5rIjTGmJBw7Fj6kkZqiWTfPjh4MLmz1Er23KeddnJCKVIESpWCc889OckkrkurLH/A06UbN26ke/fuXHzxxTzyyCMcP/4QAHnyZPmPEIjAoY6MMSZQepq7gk0yR48Gd+6oqOQTRPnyaSeSVavmcemltU4qO/307EkWhw4don///rz22mvEx8dTtWpVAPJkV2byWIIyxuQqyTV3BSaLWbPKMGdO8Fcr+/fD8ePBnTt//uSTxtlnB39FEvhaqBBIcu1MQYiN3U29ehnbNz3GjRtHu3btWL9+PbfddhtvvPEG5557bvafGEtQxphsllpzV3qbvVJu7gpU5cS70047NVGUKuXun6QnoSTX3BXuVBURISoqihIlSjB06FCaNGmSozFYgjLGnKAKhw9n/v5JVjV3FS58anNXWolkwYKpXH31JdnW3BXu4uLieO655yhbtix9+/alRYsWNG/enKionB9b3BKUMblY0uaulJLGwoUV+OmntK9g0tPclS9f8okisbkrvTfkM9PcFWjTpqOccUbmjxNpDh48yJtvvsnrr79OQkIC3bqdGJfbl+QElqCMCXmqsHw5LFgAK1a494mv27cHe5SKJzV3Jb6WLAnR0env2VWkSGQ1d4W7P/74g7Zt2xIXF8cdd9zB66+/TmYmds0qlqCMCTHx8TB3LkyeDH/95ZZt2/5bf845ULky3HKLex9MQpk1K5ZmzZr6VicTmuLj48mbNy9nn3025cqVY9iwYVx22WV+h3WCJShjfHbgAMyY4RLS5MkwfborA3cz/9pr4bLLoG5dqFSJDD0QafdiTKD169fTrVs34uPjGTFiBNWqVWPq1KlIVrSxZiFLUMbksB073FVRYkKaM8ddNYlAjRoQE+MS0qWXQtnUJqgxJp0OHDjAG2+8wZtvvomq0qVLFxISEoiKigq55ASWoIzJVqqwbt3JCWnpUrcuf35o0ACeecYlo0sugTPP9DVcE8amT5/ObbfdxsaNG7nrrrvo27cvFSpU8DusVFmCMiYLJSTA4sUnJ6S4OLfujDOgcWO47z6XkOrXh4IF/Y3XhL9Dhw5RqFAhzjvvPKpXr87IkSO55JJL/A4rKJagjMmEo0dh1qz/OjRMmQK7drl1Zcq4prrE5cIL7V6QyTnr1q3j2WefZfXq1UyfPp1SpUrx66+/+h1WuliCMiYd9u6FadP+uzr6+2/3YCtAlSpw663/JaSKFbPmuR5j0mP//v307duXfv36ERUVRdeuXYmPjyd/LnwuwBKUManYvPnk5rr5810zXp48ULs2tG//X4eGs87yO1oT6RYtWsRVV13F5s2bueeee3jttdcoX7582juGKEtQxnhUYeXKkxPSypVuXaFC0KgRvPCCS0aNGrlnjIwJBbt27aJYsWJUqlSJJk2a0LFjRxo1auR3WJlmCcpErOPHYfnywixY8N89pM2b3brixV0ieuQR91qnjo2cYELPmjVrePbZZ5k+fTrLli3jtNNOY/jw4X6HlWUsQZmIceiQu2eUmIymToV9+9x8BRUqQLNm/90/qlrVDV5qTCjau3cvgwYN4rvvviNPnjx069YtJJ9jyixLUCZs7d59cnPdrFlu6gdwPeruuQdKlFjCI49UIxc305sI8++//9KgQQO2bNlCmzZtePXVVykbpk90W4IyYeP4cZeExo+HcePc8EEJCW7U7Xr1oFMn11zXuLFrwgOIjd1K+fLV/A3cmCBs2LCBsmXLUr58ee68806qVq3Ko48+6ndY2coSlMnVNm6EX391CWnCBNi503XtrlcPnn8errzSjdZw2ml+R2pMxqxatYquXbsyfvx4li9fzjnnnMO7775LbGys36FlO0tQJlc5csQ9DDtunLtSWrDAlZ99NtxwA1x9NVx1lZtGwpjcbO/evbzyyiu888475MuXj+eee45ixYr5HVaOsgRlQl58vLtKGjoUxoxxI33ny+ea615/3SWlGjXsoVgTPvbs2UOVKlXYsmULMTExvPLKK5xzzjl+h5XjLEGZkHX4MHz0Ebzxhuv+Xbw43HsvXHcdXHFFxqadMCaULVu2jKpVq1K0aFE6d+7MlVdeSb169fwOyzeWoExIWbMGfvvtv2XnTtf9+6OPoGVLexbJhKcVK1bwzDPPMHr0aObMmUOtWrXo2rWr32H5zhKU8VVCAkycCN984xLS6tWuvGxZuPFGaNPGXS0ZE452795Nnz59eO+99yhQoACvvvoqVatW9TuskGEJyvji339h8GD4/HM3X1KRIq7HXadO0Ly5G3jV7imZcHbs2DFq1arFv//+ywMPPECfPn04++yz/Q4rpFiCMjkmPh5+/hkGDoSxY11Z8+bQty/cfLPNjWQiw99//039+vXJly8fffr0oXr16tSuXdvvsEKSDeZist3Ro9CnD0RHu0Q0d657RmnNGtc77667LDmZ8Ld8+XJuvPFGGjZsyM8//wzAvffea8kpFXYFZbLNkSPw7LOuGW/PHmjRAj74AK6/HvLavzwTIXbt2kXv3r15//33KVSoEK+//jrNmzf3O6xcIduuoESkvIhMFJElIrJYRJ7yyouLyAQRWeG9FvPKRUTeE5GVIrJAROoEHKutt/0KEWkbUF5XRBZ6+7wn4ThaYi52113w7rtuJIeRI92DtTffbMnJRA5VpWnTprzzzjvcf//9rFixgq5du1LQmgyCkp1NfPHA06paDWgEPCYi1YBuwO+qWgn43fsMcC1QyVvaAR+BS2hAD6Ah0ADokZjUvG0eDtjvmmysjwmCKvTr555RGjUKevd2zXitWvkdmTE5548//uDo0aOICG+++SZz585l4MCBlC5d2u/QcpVsS1CquklV53jv9wFLgbLATcAQb7MhwM3e+5uAL9SZDpwpImWAq4EJqrpTVXcBE4BrvHVnqOp0VVXgi4BjGR+owtNPwzPPuNEeYmLce2MixbJly7juuuto1qwZn332GQAtWrSgZs2aPkeWO+VIY4uIRAO1gRlAaVXd5K3aDCT+SVEWWB+wW5xXllp5XDLlyZ2/He6qjNKlS2dqkMX9+/dHxCCN6a1nQgK8+24lRo8uy623xvHgg2s47bTjTJuWfTFmBfs+w48fdd2zZw9Dhgzhxx9/pFChQjz66KNUrFgxW+OIiO9UVbN1AQoDs4Fbvc+7k6zf5b3+BFwaUP47UA/oArwQUP6iV1YP+C2g/DLgp7TiqVu3rmbGxIkTM7V/bpGeesbHq95/vyqodu2qmpCQfXFlNfs+w48fdW3WrJlGRUVp+/btdcuWLTlyznD6ToFZmszv62y9ghKRfMB3wFeq+r1XvEVEyqjqJq+ZbqtXvgEInDaunFe2AWiapDzWKy+XzPYmB8XHQ9u2MGwY9OjhFuuqYiLBuHHjqF+/PiVKlOCNN94gX758XHTRRX6HFVaysxefAJ8CS1X1rYBVo4HEnnhtgR8Dytt4vfkaAXvUNQWOB1qISDGvc0QLYLy3bq+INPLO1SbgWCYHHD0Kd97pktNrr0HPnpacTPhbsmQJ1157Lddeey3vv/8+AHXq1LHklA2y8wqqMXAfsFBE5nll3YG+wDci8iCwDrjDW/cL0BJYCRwE7gdQ1Z0i0huY6W3XS1V3eu87AJ8DhYCx3mJywOHDrmfezz/D229Dx45+R2RM9tq+fTs9e/ZkwIABFClShLfffpsOHTr4HVZYy7YEpap/ASn9Pd0sme0VeCyFYw0GBidTPgu4MBNhmgw4eNA9zzRhghtlvH17vyMyJvt16tSJr7/+mvbt29OzZ09K2qyY2c6GOjLpsm+fm/bit9/gs88sOZnwpar89NNPLF++HIDevXszf/58PvjgA0tOOcQSlAnanj1u9tq//oKvvnLPORkTjhYtWsTVV1/NDTfcwNtvvw1AdHQ01atX9zmyyGIJygRlxw43ceCsWW7uprvv9jsiY7Letm3b6NChAzVr1mTWrFm8++67vPfee36HFbFsVDSTpq1b3bQYy5fDDz+4KdeNCUf9+/dn4MCBPPbYY/To0YMSJUr4HVJEswRlUrVxo7tyWrcOxoyBq67yOyJjso6qMnr0aEqWLEnjxo3p1q0bbdu25YILLvA7NIM18ZlU/PsvXH45xMW5CQYtOZlwsmDBApo3b87NN9/Mu+++C8CZZ55pySmEWIIyydq4sSCXXw7btrnRyJs08TsiY7LG1q1beeSRR6hduzbz5s3j/fff56uvvvI7LJMMa+Izp/jnH3jqqdokJMAff0Ddun5HZEzWGTlyJIMHD+aJJ57gpZdeonjx4n6HZFJgCcqcZNEi1yEiPl6YNAlq1PA7ImMyR1UZNWoUx48fp1WrVrRr146rrrqKypUr+x2aSYM18ZkT5s6Fpk0hKgreeWeeJSeT682bN48rr7ySW2+9lQEDBgCQL18+S065hCUoA8CMGXDllXDaafDnn1ChwkG/QzImw7Zs2cLDDz9MnTp1WLhwIf/3f//HuHHj/A7LpJMlKMNff7keesWLu+R0/vl+R2RM5syaNYshQ4bQqVMnVq5cyaOPPkrevHZHI7exbyzC/fEH3HADlCvn3pdNdk5iY0KbqvLdd9+xceNGatSoQcuWLVm1ahXly5dPe2cTsuwKKoKNG+dGhahYESZNsuRkcqc5c+bQtGlTbr/9dr788kuOHz+OiFhyCgOWoCLUjz/CTTdB1aoQGwtnn+13RMakz+bNm3nwwQepV68eS5cuZcCAAUydOpU8efL4HZrJItbEF4HWrYO77oJatdxVVLFifkdkTPpt27aNr7/+mi5duvD8889TtGhRv0MyWSzFBCUi76hqRxEZA2jS9ap6Y7ZGZrLNM8+4qdm//daSk8k9VJWRI0cya9Ys3njjDS666CLi4uLsQdswltoV1FDvtV9OBGJyRmwsjBwJL78M1kRvcotZs2bRsWNHpkyZQq1atTh48CCnnXaaJacwl2KCUtXZ3uuknAvHZKejR+HJJ+F//4MuXfyOxpi0bd26lWeeeYYvvviCs846i0GDBnH//ffbfaYIkeY9KBFpDPQEKnjbC6Cqem72hmaykio8+CAsXOiuoE47ze+IjEmbqjJ27FieffZZunfvzhlnnOF3SCYHBdNJ4lOgEzAbOJ694Zjs8v338OWX8OKL0KqV39EYkzxVZfjw4YwaNYrhw4dTunRp1qxZw+mnn+53aMYHwXQz36OqY1V1q6ruSFyyPTKTZZYuhQcegNq14fnn/Y7GmOTNmDGDxo0b07p1a1asWMG2bdsALDlFsGAS1EQReVNELhaROolLtkdmssTkydC4MRQq5J59KlDA74iMOdnOnTu57777aNSoEWvWrGHw4MHMnDmTs846y+/QjM+CaeJr6L3WCyhT4MqsD8dkpe3b4frr3ejkw4dbrz0TmgoVKsTMmTPp3r073bp1o0iRIn6HZEJEmglKVa/IiUBM1uvbF/bvdx0jqlXzOxpjnISEBIYNG8aAAQOYMGEChQoVYuHCheTLl8/v0EyISe1B3XtV9UsR6ZzcelV9K/vCMpm1YQN88AG0aWPJyYSOadOm0bFjR/7++2/q1q3Lli1biI6OtuRkkpXaPajEO5NFUlhMiJk4Edq3hwMH4JVXICEBevTwOypjYP/+/bRu3ZpLLrmE9evX8/nnn/P3338THR3td2gmhKX2oO7H3uvLOReOyahdu+Dmm2HvXti3zz3r9OCDYP//jZ8SEhKIiori9NNPZ8uWLbzwwgs8++yzFC5c2O/QTC4QzIO676W2XlWfzLpwTEb16OGunEqWhGHDIG9e6NXL76hMpEpISGDo0KH07duX2NhYSpcuzYQJE4iKsgkUTPCC+ddSEKgDrPCWWkB+3IO7s1PaSUQGi8hWEVkUUNZTRDaIyDxvaRmw7jkRWSki/4jI1QHl13hlK0WkW0B5RRGZ4ZWPEJH86ah3WJk7F95/H26/HUaNcmVt20KpUr6GZSLUlClTaNiwITExMZxxxhns2rULwJKTSbdgupnXAC5V1XgAERkATFbV9mns9znwAfBFkvK3VfWkAWhFpBpwF1AdOAf4TUQqe6s/BK4C4oCZIjJaVZcAr3vHGu7F9CDwURD1CSvLlsEVXj/Lzp2hfn1YsAAqV059P2OyWnx8PPfddx/Dhw+nbNmyDB06lNatW1tiMhkWzL+cYkDgAFiFvbJUqeqfwM4g47gJGK6qR1R1DbASaOAtK1V1taoeBYYDN4mI4J7D+tbbfwhwc5DnCivvv++6kk+f7pITwEUX2QO5JuccO3YMgLx581K4cGF69OjBP//8w7333mvJyWRKMFdQfYG5IjIRN1Ds5bjBYzPqcRFpA8wCnlbVXUBZYHrANnFeGcD6JOUNgRLA7sSruiTbn0JE2gHtAEqXLk1sbGyGg9+/f3+m9s8qCQkwZsw5fPrpeTRpsp1Dh5aSlWGFSj2zm9Uz4xISEhg/fjyfffYZffv25dxzz+Wee+4BYObMmVl6rvSw7zSMqGqaC3A27irnJuDsYPbx9osGFgV8Lg3kwV25vQIM9so/AO4N2O5ToJW3fBJQfp+3bUnclVViefnA86S21K1bVzNj4sSJmdo/q7z4oiqoVq2qumpV1h8/VOqZ3ayeGfPnn39qnTp1FNCGDRvq/Pnzs/T4mWHfae4DzNJkfl8HNeW7qm4GfsxcKgRV3ZL4XkQGAT95Hzd4SSZROa+MFMp3AGeKSF51V1GB24e9v/6CPn2gbFk3SkTeoL5FYzJPVWnbti1Dhw6lfPnyfPXVV9x99924VndjslaONhCLSJmAj7cAiT38RgN3iUgBEakIVAL+BmYClbwee/lxHSlGexl3Iu4KC6AtWZBAc4N9+9zoENHRbpRyS04mJxw4cABVRUSoWrUqvXr1YtmyZbRu3dqSk8k22fbrTUS+BpoCJUUkDugBNBWRWrjBZtcCjwCo6mIR+QZYAsQDj6nqce84jwPjcU2Dg1V1sXeKZ4HhItIHmItrFgx7zz8Pa9fCn3+Cjalpstvx48f5/PPPef755xk8eDAtW7ake/fufodlIkRQCUpELgUqqepnIlIKKKyut12KVPXuZIpTTCKq+gruvlTS8l+AX5IpX43r5RcxBg50vfZuvRUuvdTvaEy4i42NpVOnTsybN4/GjRtTpkyZtHcyJgul2cQnIj1wVyvPeUX5gC+zMyhzqq+/hkcegebN3XtjstOjjz7KFVdcwc6dOxk+fDiTJ0+mdu3afodlIkwwV1C3ALWBOQCqulFErHEph332mXsdNgzyR+yYGSY77d27l0KFCpEvXz4aN25MuXLl6Ny5M4UKFfI7NBOhgukkcdTrlKAAImLzL+ewHTvgjz/gueds+CKT9Y4fP86gQYOoVKkSH3/8MQD33nsvzz//vCUn46tgEtQ3IvIxrlv3w8BvwKDsDcsE6t0bjh+HVq3S3taY9Pjjjz+oU6cO7dq1o0qVKlxyySV+h2TMCcHMqNtPRK4C9gJVgJdUdUK2R2YAWLIE3n0XHn8c6tTxOxoTTrp27cqbb75JdHQ0I0eO5LbbbrMu4yakBDPdRmdghCUlf4wb5167dvU3DhMe9uzZg4hwxhln0LJlS4oVK0anTp0oWLCg36EZc4pgmviKAL+KyGQReVxESmd3UOY/v/wCVatC+fJpb2tMSo4fP86AAQM4//zz6dmzJwBNmzblueees+RkQlaaCUpVX1bV6sBjQBlgkoj8lu2RGVasgN9/h7uTe6LMmCBNmDCBhx56iEcffZTq1atz3333+R2SMUFJz1BHW4HNuHHwzsqecEygIUMgKgoeftjvSExu9corr9CiRQuOHDnCd999x8SJE+15JpNrBHMPqgNwB1AKGAk8rG7CQJONjh93Cerqq8Ee4DfpsWvXLg4dOsQ555xDq1atyJs3L7Vr16ZFixZ+h2ZMugRzBVUe6Kiq1VW1pyWnnPHppxAX56ZuNyYY8fHxfPjhh5x//vk88cQTAFSpUoVnn32W/PZ0t8mFUkxQIpI4i+6bwL8iUjxwyZnwItO//8Kjj7rpNG66ye9oTG4wfvx4atasyeOPP07NmjV56aWX/A7JmExLrYlvGHA9MBs3ikTgAxIKnJuNcUW0N990M+b++itYByuTlo8++ogOHTpw3nnnMWrUKG688UZ7nsmEhRQTlKpe771WzLlwzNGjMHQo3HMPVKvmdzQmVO3cuZOtW7dStWpV7rjjDg4fPkyHDh0oUKCA36EZk2WCGc3892DKTNb45RfYs8e6lpvkHTt2jPfff5/zzz+fNm3aoKqUKFGCTp06WXIyYSe1e1AFvXtNJUWkWMD9p2igbI5FGEFUoW9fqFABrMOVSWrs2LHUqFGDJ598krp16/Lpp59aU54Ja6ndg3oE6Aicg7sPlfg/YS/wQfaGFZm++QZmzIABAyBfPr+jMaHk22+/5fbbb6dy5cqMGTOG6667zpKTCXup3YN6F3hXRJ5Q1fdzMKaIdPgwdOsGNWtCTIzf0ZhQsH37dlauXEmjRo248cYbGThwIG3btrUu4yZiBDOa+fsiciFQDSgYUP5FdgYWaXr1grVr3eCwdishsh09epT/+7//4+WXX6ZIkSKsXr2a/Pnz87ANKWIiTLBTvr/vLVcAbwA3ZnNcEWX/fvjwQ7jzTjdyhIlMqspPP/3ERRddRKdOnWjQoAG//PILefMGM/G1MeEnmJEkWgHNgM2qej9QEyiarVFFmC++gL17oWNHvyMxfpo8eTI33HADIsLPP//MuHHjuPDCC/0OyxjfBJOgDqlqAhDvjS6xFTf8kckCqvDBB1CvHjRs6Hc0Jqdt27aNX375BYDLLruMESNGsHDhQlq2bGmdIEzEC6btYJaInImb5n02sB+Ylp1BRZLff4elS93AsPb7KHIcPXqU999/n969e6OqxMXFUaRIEe644w6/QzMmZAQzH1QHVd2tqgOAq4C2XlOfySRVuOoq9/7OO/2NxeQMVeXHH3+kevXqdOnShcaNGzN9+nSKFCnid2jGhJwUr6BEpE5q61R1TvaEFDlmz3avHTpYz71IsXz5cm655RaqVq3K2LFjueaaa/wOyZiQlVoTX/9U1ilwZRbHEnG+/Rby5oXevf2OxGSnrVu3MnbsWNq2bUuVKlWYMGECl19+OfnsaWxjUpXag7pX5GQgkeiPP6BRIyhuk5eEpSNHjvDee+/Rp08fDh8+TPPmzSlbtizNmjXzOzRjcoVgZtRtk1y5PaibOT/+CDNnurH3THhRVUaNGkWXLl1YvXo1119/Pf369aNsWRvC0pj0CKYXX/2A9wVxz0TNASxBZcJ330HRovDUU35HYrLa9u3bue+++4iOjmb8+PE21boxGRRML74nApaHgTpA4bT2E5HBIrJVRBYFlBUXkQkissJ7LeaVi4i8JyIrRWRBYAcNEWnrbb9CRNoGlNcVkYXePu9JLntoZO5cuPRSm5AwXGzevJnXX38dVaVUqVJMmjSJefPmWXIyJhOCeVA3qQNAMJMYfg4k7aLUDfhdVSsBv3ufAa4FKnlLO+AjcAkN6AE0BBoAPRKTmrfNwwH75ZruUIcOuWefatXyOxKTWYcPH6Zv375UqlSJF198kUWL3N9jdevWtSGKjMmkYMbiGyMio73lJ+Af4Ie09lPVP4GdSYpvAoZ474cANweUf6HOdOBMESkDXA1MUNWdqroLmABc4607Q1Wnq6rimhtvJpcYNAiOH4fGjf2OxGSUqvLtt99ywQUX8Nxzz9GsWTMWL17MRRdd5HdoxoSNYP7E6xfwPh5Yp6pxGTxfaVXd5L3fDJT23pcF1gdsF+eVpVYel0x5skSkHe7KjNKlSxMbG5vB8GH//v2Z2h/g009rce65eSlYcBaZPFS2yYp65gYZreeRI0fo0KEDhQsXpl+/ftStW5cNGzawYcOGrA8yC0TK9wmRU9dIqGcw021MAvDG4cvrvS+uqkmvjtJFVVVENDPHSMe5BgIDAerVq6dNmzbN8LFiY2PJzP5Ll8KCBW56jSuuyPhxsltm65lbpKeemzZt4u2336Z3794UKFCAKVOmULFixVzRlBcp3ydETl0joZ7BNPG1E5HNwAJgFm48vlkZPN8Wr3kO73WrV76BkwegLeeVpVZeLpnykDdkiJst95FH/I7EBOvQoUO88sorVKpUiXfeeYfp06cDUKlSpVyRnIzJrYLpJPEMcKGqRqvquapaUVXPzeD5RgOJPfHaAj8GlLfxevM1AvZ4TYHjgRYiUszrHNECGO+t2ysijbzee20CjhXSpkxxI5efdZbfkZi0qCojRoygatWqvPDCC7Ro0YIlS5bQpEkTv0MzJiIE8+ffKuBgeg8sIl8DTYGSIhKH643XF/hGRB4E1gGJQzf/ArQEVnrnuh9AVXeKSG9gprddr4CmxQ64noKFgLHeEtKOHnUP5z72mN+RmGCoKv3796dYsWIMGTIk7JtTjAk1wSSo54CpIjIDOJJYqKpPpraTqt6dwqpTxnnxeuIl+2tbVQcDg5MpnwXkqtncYmPhyBH3/JMJTRs2bKBXr1706dOHUqVKMXr0aEqVKkWePHn8Ds2YiBNME9/HwB/AdNz9p8TFpNOgQVCiBLRs6XckJqmDBw/Su3dvKleuzOeff87UqVMBOPvssy05GeOTYK6g8qlq52yPJMwNG+ZGL+/UyabWCDVff/01zz77LOvXr6dVq1a8/vrrnHtuRm+zGmOySjAJaqz3HNEYTm7iy1Q380iyezc8+KCbMTcmxu9oTFLff/89JUuW5Msvv+Tyyy/3OxxjjCeYBJV4L+m5gDIF7E/MIC1ZAocPw88/Q40afkdj4uLi6N69O926uZG2PvnkE4oUKUJUVEZG/jLGZJdgHtQNZtw9k4q5c91rlSr+xhHpDhw4wJtvvskbb7xBQkICV111FeXLl6do0aJ+h2aMSYbNB5UDRo6EatXAbmv4Z/jw4XTp0oUNGzZw55130rdvX6Kjo8N+qBhjcjObDyqbHT4M06bBk0+6e1DGH3PmzKFMmTIMHz6cS62fvzG5QjBNfE8EfhaRM4Hh2RVQuFmxwj2gW6+e35FEln///Zdu3boRExNDixYt6NWrF/nz57f7TMbkItk5H5QBVq50r+ef728ckWL//v28+OKLVKlShR9++IE1a9YAULBgQUtOxuQywdyDGoPrtQcuoVUDvsnOoMLJ+PFucNhKlfyOJPyNHDmSjh07snHjRlq3bs1rr73G//73P7/DMsZkUE7PBxVRtm6FgQPdyOVnnOF3NOFLVRERduzYQbly5fj222+5+OKL/Q7LGJNJKbZ5iMj5ItJYVScFLFOACiJyXg7GmGtNnQqq0CbZfpAms9auXcudd97JoEGDAHj44YeZNm2aJSdjwkRqjfLvAHuTKd/rrTNpmDoV8ueHOnX8jiS87Nu3j+eff56qVasyZswYDh50g+3nyZPH7jMZE0ZSa+IrraoLkxaq6kIRic6+kMLHr79CgwY29l5W+vHHH2nfvj2bN2/m3nvv5bXXXqNcuXJp72iMyXVSS1BnprKuUBbHEXbi4mD+fOjf3+9IwkNCQgJRUVEULFiQ6OhoRo0aRcOGDf0OyxiTjVJrD5klIg8nLRSRh7DpNtKUOLyR/Q7NnNWrV9OqVSu6d+8OwNVXX83UqVMtORkTAVJLUB2B+0UkVkT6e8sk4EHgqRyJLheLjXX3n2rV8juS3Gnv3r1069aNCy64gLFjx1KsWLET68SG5DAmIqTYxKeqW4BLROQK/pu59mdV/SNHIsvlxo6FJk3g9NP9jiT3GT9+PG3btmXLli20adOGV199lbJly/odljEmhwUz1NFEYGIOxBI2Vq6EpUuhXTu/I8ldjh49Sv78+SlXrhxVqlRhzJgx1K9fP+0djTFhKZgHdU06ffede73lFn/jyC1WrVrFM888Q758+RgxYgTVq1dn0qRJfodljPGZPTSSDb77zg0OW6GC35GEtj179tC1a1eqVavGr7/+Ss2aNVHVtHc0xkQEu4LKYmvXwsyZ8NprfkcS2iZPnsxtt93G9u3biYmJ4ZVXXqFMmTJ+h2WMCSF2BZXFRoxwr3fe6W8coWr//v0AVK1alQYNGjBz5kwGDx5syckYcwq7gspCqvDZZ3DJJVDRJiQ5yYoVK3jmmWfYtGkT06ZNo1SpUvz0009+h2WMCWF2BZWFpk2Df/6Bhx7yO5LQsXv3brp06UL16tX5/fffueWWWzh+/LjfYRljcgG7gspCkye71xtv9DeOUDF37lxatGjBjh07eOCBB+jTpw9nn32232EZY3IJu4LKQrNmuaa9EiX8jsRf27dvB6BatWpcc801zJ49m08++cSSkzEmXSxBZZH9++GHH1z38kj1zz//cMMNN1CnTh0OHjxIgQIFGDp0KLVr1/Y7NGNMLmQJKot8+SUcPw433eR3JDlv165ddOrUiQsvvJBJkybxxBNPkCdPHr/DMsbkcr7cgxKRtcA+4DgQr6r1RKQ4MAKIBtYCd6jqLnEjg74LtAQOAjGqOsc7TlvgBe+wfVR1SE7WI9DUqVCmDLRu7VcE/li9ejX169dn9+7dPPTQQ/Tq1YvSpUv7HZYxJgz4eQV1harWUtXERrFuwO+qWgn43fsMcC1QyVvaAR8BeAmtB9AQaAD0EJFi+GTFCrjgAoiUgbbXrVsHQMWKFYmJiWHOnDl8/PHHlpyMMVkmlJr4bgISr4CGADcHlH+hznTgTBEpA1wNTFDVnaq6C5gAXJPDMQOuae+ff+D88/04e85aunQpLVu2pHr16mzcuBERoX///tSsWdPv0IwxYcavbuYK/CoiCnysqgNxU8xv8tZvBhL/FC8LrA/YN84rS6n8FCLSDnf1RenSpYmNjc1w4Pv37z9l/4ULz2DXrjqULr2Y2NhtGT52KElazz179jBkyBB+/PFHChUqRJs2bVi0aBHLly/3L8gskNz3GY4ipZ4QOXWNhHr6laAuVdUNInIWMEFElgWuVFX1kleW8BLgQIB69epp06ZNM3ys2NhYku6/aJF7bd++Oueck+FDh5TAeu7cuZNKlSqxe/du2rVrR69evShVqpS/AWaR5L7PcBQp9YTIqWsk1NOXJj5V3eC9bgV+wN1D2uI13eG9bvU23wCUD9i9nFeWUnmOW7wYzjzTdZIIF6rKwoULAShevDjdu3dn3rx5fPTRR2GTnIwxoS3HE5SInC4iRRLfAy2ARcBooK23WVvgR+/9aKCNOI2APV5T4HighYgU8zpHtPDKctyiRXDhheHTQWLx4sV07dqVmjVrsmDBAgCefvppLrroIp8jM8ZEEj+a+EoDP7je4+QFhqnqOBGZCXwjIg8C64A7vO1/wXUxX4nrZn4/gKruFJHewExvu16qujPnquEkJLgEFQ6jl2/fvp0ePXrw8ccfU6hQId566y2qVq3qd1jGmAiV4wlKVVcDp3T5UtUdQLNkyhV4LIVjDQYGZ3WM6bFyJezeDbl9ZvIjR45Qs2ZNtmzZQvv27bnqqqu4KRKfOjbGhIxQ6maeK82Y4V4bNvQ3joxQVSZPnoyqUqBAAd544w3mz5/PBx98QNGiRf0OzxgT4SxBZdKMGVC4sHtINzdZuHAhLVq04PLLL2f8eHfr7p577qF69eo+R2aMMY4lqEyaMcM17+WWoee2bdvGo48+Sq1atZg9ezbvvfcezZqd0rJqjDG+s/mgMuHIEZg/Hzp39juS4KgqTZo0Yfny5Tz++OP06NGD4sWL+x2WMbnesWPHiIuL4/Dhwzl2zqJFi7J06dIcO19WKFiwIOXKlSNfvnxBbW8JKhPmz4djx0K7g4SqMm7cOJo1a0b+/Pl57733KFu2LBfktjZJY0JYXFwcRYoUITo6Gsmh50327dtHkSJFcuRcWUFV2bFjB3FxcVSsWDGofayJLxNmeh3cQzVBLViwgObNm9OyZUuGDh0KQPPmzS05GZPFDh8+TIkSJXIsOeVGIkKJEiXSdZVpCSoTZs6Es86C8uXT3jYnbd26lUceeYTatWszb948PvjgA9q2bZv2jsaYDLPklLb0/oysiS8TZs50V0+h9u/yjjvuYMqUKTz55JO89NJLFCvm2ywkxhiTYXYFlUH79sHSpaHRvKeq/PDDD+zc6QbSeOedd1i0aBFvv/22JSdjTLI+//xzHn/8cQAGDBjAF198AcCyZcuoVasWtWvXZtWqVQwbNsy3GC1BZdDMmaAKDRr4G8fcuXO54ooruPXWW/noo48AqFWrFlWqVPE3MGNMrtG+fXvatGkDwKhRo2jVqhVz585l/fr1viYoa+LLoEmTICoKGjf25/ybN2/mhRdeYPDgwZQoUYKPPvqIhx56yJ9gjDEndOwI8+Zl7TFr1YJ33kl9m7Vr13L99dezyJv/p1+/fifmjKpZsyaTJk0iPj6ewYMH0yDJX9Y9e/akcOHCVKtWjXfeeYc8efLw+++/c+jQIZYuXUqtWrVo27YtnTp1ytqKpcESVAZNmgS1a8MZZ/hz/s6dO/Ptt9/SuXNnXnjhBc4880x/AjHGhLyDBw8yb948/vzzTx544IETSSypli1b0r59ewoXLkyXLl2IjY2lX79+/PTTTzkcsWMJKgMOH4bp0+GxZIewzR6qynfffUeNGjWoXLkyr732Gi+//DKVKlXKuSCMMWlK60rHD3fffTcAl19+OXv37mX37t3+BhQkuweVAX//7UaRaNIkZ843e/ZsmjRpwu23386HH34IQIUKFSw5GWNOyJs3LwkJCSc+Bz5vlLR7d27pEm8JKgP++su9Xnpp9p5n48aN3H///dSvX59ly5bx8ccf89Zbb2XvSY0xuVLp0qXZunUrO3bs4MiRIyc1y40YMQKAv/76i6JFiwY9W0GRIkXYt29ftsQbDGviy4B//oFy5SC7h7F7++23GTZsGM888wzdu3e3KTCMMSnKly8fL730Eg0aNKBs2bInTTZasGBBateuzbFjxxg8OPgp9GrUqEGePHmoWbMmMTExOd5JAlWNqKVu3bqaGRMnTtRLLlFt2jRTh0lWQkKCDh8+XCdPnqyqqrt27dKVK1dm/YmCMHHiRF/Om9OsnuHHj7ouWbIkx8+5d+/eoLZr0qSJzpw5M5ujCV5yPytglibz+9qa+DJg1So477ysPebMmTO59NJLueuuuxgwYAAAZ555Judl9YmMMSaXsASVTgcP5mHLFjj//Kw53oYNG2jbti0NGjRg1apVfPLJJwwZMiRrDm6MiXixsbHUq1fP7zAyxO5BpdPatacDkFUDNXz//fcMHz6cbt268dxzz3GGXw9WGWNMiLEElU4LF7qOCpdckrH9VZWvv/6afPnycfvtt9O+fXuuv/76oOdHMcaYSGFNfOm0YEFRKlWC0qXTv+/06dO55JJLuOeee/j8888B1/PGkpMxxpzKElQ6JCTAokVF0/380/r167n33nu5+OKLWbt2LZ999hljxozJniCNMSZMWIJKh9WrYe/efOlu3ps/fz7ffvstzz//PCtWrCAmJoaoKPvRG2NCV3R0NNu3b/c1BrsHlQ5r1rjXtEYYSkhIYNiwYezYsYOnnnqK6667jjVr1lCmTJnsD9IYE/FOPEeUy/8QtgSVDuvWudcKFVLeZtq0aXTs2JG///6byy67jCeeeIKoqChLTsZEkKZNm55Sdscdd9ChQwcOHjxIy5YtT1kfExNDTEwM27dvp1WrVieti42NTfOca9eu5eqrr6Zhw4bMnj2bBg0asHDhQg4dOkSrVq14+eWXAXdl1LZtW8aMGcOxY8cYOXIkVatWZceOHdx9991s2LCBiy++GPf8rPPWW2+dGIHioYceomPHjqxdu5ZrrrmGRo0aMXXqVOrXr8/9999Pjx492Lp1K1999dUp03qkV+5Orzls3TqIilLKlTt1XVxcHK1bt+aSSy4hLi6OIUOGEBsbm+v/gjHG5B4rVqygQ4cOLF68mP79+zNr1iwWLFjApEmTWLBgwYntSpYsyZw5c3j00Ufp168fAC+//DKXXnopixcv5pZbbuHff/8F3GDVn332GTNmzGD69OkMGjSIuXPnArBy5Uqefvppli1bxrJlyxg2bBh//fUX/fr149VXX810fewKKh3WroVSpY6QN2/BU9bt3LmT0aNH8+KLL9K1a1cKFy6c8wEaY0JCalc8p512WqrrS5YsGdQVU3IqVKhAo0aNAPjmm28YOHAg8fHxbNq0iSVLllCjRg0Abr31VgDq1q3L999/D8Cff/554v11111HsWLFADfA7C233MLpp59+Yt/Jkydz4403UrFiRS666CIAqlevTrNmzRARLrroItauXZuhOgTK9QlKRK4B3gXyAJ+oat/sOte6dVC69GGgIAkJCQwdOpQFCxbQv39/atSowYYNG2xAV2OMbxKTyJo1a+jXrx8zZ86kWLFixMTEnDT9RoECBQDIkycP8fHxGT5f4nEAoqKiTnyOiorK1HFPHDPTR/CRiOQBPgSuBaoBd4tItew63333QcuWm/jrr79o2LAhMTExTJky5cQXb8nJGBMK9u7dy+mnn07RokXZsmULY8eOTXOfyy+/nGHDhgEwduxYdu3aBcBll13GqFGjOHjwIAcOHOCHH37gsssuy9b4E+XqBAU0AFaq6mpVPQoMB27KrpNdf/0mpk/vwGWXXcamTZv48ssvmTp1KgULntrkZ4wxfqlZsya1a9ematWqtG7dmsaNG6e5T48ePfjzzz+pXr0633//Pf/73/8AqFOnDjExMTRo0ICGDRvy0EMPUbt27eyuApD7m/jKAusDPscBDbPrZCLC/Pnz6dmzJ126dDlxOW2MMX6Ljo5m0aJFJz4njlaTVOC9oXr16p2431WiRAl+/fXXZPfp3LkznTt3Dvp8SddllAR2JcxtRKQVcI2qPuR9vg9oqKqPJ9muHdAOoHTp0nWHDx+e4XPu3LmT4tk9U2EI2L9/f0R09LB6hh8/6lq0aFHOz6opDoJ0/Phx8uTJk6PnzAorV65kz549J5VdccUVs1X1lCHXc/sV1AagfMDncl7ZSVR1IDAQoF69eprcMwrBio2NTfYZh3Bj9QwvkVJP8KeuS5cupUiRIjl6zn379uX4ObNC4uy+wcjt96BmApVEpKKI5AfuAkb7HJMxJgLl5taonJLen1GuTlCqGg88DowHlgLfqOpif6MyxkSaggULsmPHDktSqVBVduzYka5OZbm9iQ9V/QX4xe84jDGRq1y5csTFxbFt27YcO+fhw4dzXQ/iggULUi65oXhSkOsTlDHG+M2Ped1iY2NzrLu3X3J1E58xxpjwZQnKGGNMSLIEZYwxJiTl6gd1M0JEtgHrMnGIkoC/00zmDKtneImUekLk1DWc6llBVUslLYy4BJVZIjIruSeew43VM7xESj0hcuoaCfW0Jj5jjDEhyRKUMcaYkGQJKv0G+h1ADrF6hpdIqSdETl3Dvp52D8oYY0xIsisoY4wxIckSlDHGmJBkCSoZInKNiPwjIitFpFsy6wuIyAhv/QwRifYhzCwRRF1jRGSbiMzzlof8iDMzRGSwiGwVkWSn+BTnPe9nsEBE6uR0jFkhiHo2FZE9Ad/lSzkdY1YQkfIiMlFElojIYhF5KpltwuU7DaauYfG9JktVbQlYgDzAKuBcID8wH6iWZJsOwADv/V3ACL/jzsa6xgAf+B1rJut5OVAHWJTC+pbAWECARsAMv2POpno2BX7yO84sqGcZoI73vgiwPJl/t+HynQZT17D4XpNb7ArqVA2Alaq6WlWPAsOBm5JscxMwxHv/LdBMRCQHY8wqwdQ111PVP4GdqWxyE/CFOtOBM0WkTM5El3WCqGdYUNVNqjrHe78PNxdc2SSbhct3Gkxdw5YlqFOVBdYHfI7j1H8QJ7ZRN2niHqBEjkSXtYKpK8BtXjPJtyJSPmdCy1HB/hzCwcUiMl9ExopIdb+DySyveb02MCPJqrD7TlOpK4TZ95rIEpRJyxggWlVrABP478rR5D5zcGOe1QTeB0b5G07miEhh4Dugo6ru9Tue7JRGXcPqew1kCepUG4DAq4RyXlmy24hIXqAosCNHostaadZVVXeo6hHv4ydA3RyKLScF853neqq6V1X3e+9/AfKJSEmfw8oQEcmH+4X9lap+n8wmYfOdplXXcPpek7IEdaqZQCURqSgi+XGdIEYn2WY00NZ73wr4Q727lblMmnVN0m5/I64NPNyMBtp4Pb8aAXtUdZPfQWU1ETk78V6piDTA/f/PdX9YeXX4FFiqqm+lsFlYfKfB1DVcvtfk2JTvSahqvIg8DozH9XIbrKqLRaQXMEtVR+P+wQwVkZW4m9J3+RdxxgVZ1ydF5EYgHlfXGN8CziAR+RrX06mkiMQBPYB8AKo6APgF1+trJXAQuN+fSDMniHq2Ah4VkXjgEHBXLv3DqjFwH7BQROZ5Zd2B/0F4facEV9dw+V5PYUMdGWOMCUnWxGeMMSYkWYIyxhgTkixBGWOMCUmWoIwxxoQkS1DGGGMyJK0BipPZ/o6AgW+HpbW9JShjAoiIikj/gM9dRKRnFh37cxFplUXHmuGNXP2vnDzafHRWHD+Z83XPwD4xIvJBdsRjQsbnwDXBbCgilYDngMaqWh3omNY+lqCMOdkR4NZQexLfG7HkBFVtqKq1gJdwo+nX8pa16TlOOqQ7QZnwl9wAxSJynoiME5HZIjJZRKp6qx4GPlTVXd6+W9M6viUoY04WDwwEOiVdkfQKSET2e69NRWSSiPwoIqtFpK+I3CMif4vIQhE5L+AwzUVklogsF5Hrvf3ziMibIjLTG5T3kYDjThaR0cCStAIXkRu8K6u5IvKbiJT2ynuKyFARmYJ7wLyUiEzwmlk+EZF1iQlZRO714p4nIh97sfUFCnllX6W0nVd+v1e3v3EPmZrIMxB4QlXrAl2A//PKKwOVRWSKiEwXkTSvvCxBGXOqD4F7RKRoOvapCbQHLsA9+V9ZVRvgxi98ImC7aNw0J9cBA0SkIPAgbiie+kB94GERqehtXwd4SlUrBxHDX0AjVa2Nmzqla8C6akBzVb0bN8LEH14zy7d4oxKIyAXAnbgmmFrAceAeVe0GHPKu0O5JaTtxw2K9jEtMl3rnNBFE3KC2lwAjvZEvPsbNaQVu5KJKuNFO7gYGiciZqR3PhjoyJglV3SsiXwBP4oaOCcbMxLHeRGQV8KtXvhC4ImC7b1Q1AVghIquBqkALoEbA1VlR3H/ko8DfqromyBjKASO8RJEfCNxvtKom1uVS4BavruNEZJdX3gw3GPBMb2i3QkByzTApbdcQiFXVbd7PYQTur2YTOaKA3d4fLknF4SaOPAasEZHluH/nM1M7mDHmVO/grmxODyiLx/s/IyJRuCSQ6EjA+4SAzwmc/Idg0rHFFDfr6xMB95EqqmpigjuQjpjfx81+fBHwCFAwYF0wxxFgSEAcVVS1Zya2MxHGmwpkjYjcDm6wWxGp6a0ehbt6wmtSrgysTu14lqCMSYaq7gS+wSWpRGv5b7qRG/EGYk2n20UkyrsvdS7wD26w3kfFTauAiFQWkdNTO0gKivLflBJtU9luCnCHd64WQDGv/HeglYic5a0rLiIVvHXHEuNLZbsZQBMRKeFte3sG6mByEXEDFE8DqohInIg8CNwDPCgi84HF/DdL93hgh4gsASYCz6hqqqOuWxOfMSnrDzwe8HkQ8KP3H28c6bu6SfQv8DdwBtBeVQ+LyCe4e1NzxLWZbQNuzsCxe+La/ncBfwAVU9juZeBrEbkP98tlM7BPVbeLyAvAr94V4jHgMWAd7sb3AhGZ492HOmU7VZ0urkv+NGA3MC8DdTC5iHdPMzmndIDwRljv7C1BsdHMjYkwIlIAOO5Nt3Ix8FEK9wyM8ZVdQRkTef4HfONd/RzFPZ9iTMixKyhjjDEhyTpJGGOMCUmWoIwxxoQkS1DGGGNCkiUoY4wxIckSlDHGmJD0/5QVvnacn4RvAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "perf_h2o.plot_uplift(metric=\"qini\")" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABG30lEQVR4nO3deZyNdfvA8c9lF0LUVAhpEGXfIiUK4an0oKRCqyRLydJGiqceSwvtJYSkFT2kUqOfsqtsEbKN7Et2M2Ou3x/fezjGLGfGnDnLXO/X636dM/d63XM413yX+/sVVcUYY4wJNbmCHYAxxhiTEktQxhhjQpIlKGOMMSHJEpQxxpiQZAnKGGNMSLIEZYwxJiRZgjI5logMFpGJwY7DHyLSRUTm+fx8WEQu994XFJEZIvKPiHwavChDg4jMEpHOWb2vyX6WoEy2E5G7RGSJ9yW73fuSuDbYcaVFRMaJyIvncHyMiDyQbF0TEYnNzPlUtbCq/uX92A6IAkqoant/YhWnp4isFJEjIhIrIp+KyNWZiSeUqOrNqjo+q/c12c8SlMlWIvI48CowDPelehnwJnBrEMMKd2WBP1U1IQPHvAb0AnoCFwAVga+A1hm9uIjkyegxxvhFVW2xJVsWoChwGGifxj7jgBd9fm4CxPr8vAl4ElgOHAE+wCW6WcAh4HugeErH+hx/o/d+MDDRZ9unwA7gH+AnoKq3/iEgHojz4p/hrb8U+BzYDWwEeqZxXzHAA8nWJb83xSWMv4A9wHAgl7etCzAv2b5XAM97ccV7sT2cUqzJrhsNnATq+RtvKtd/FFjn3ftbwIhk55gGPJ7R35Uf/45yAyO839FfXhwK5Ekee1Lc3v77vWvfnNbnYkvoLFaCMtnpGqAA8OU5nuffwE24v/r/hUtOTwEX4moFembyvLNwX94XAcuASQCq+q73/r/qqtb+JSK5gBnA70ApoBnQW0RaZPamPG2BOkAtXKnyvrR2VtVBuNLoJ15s7ySPNYXDmuES46JzjPU2oD5QBfgYuENEBEBEigPNgSkB+F09CLQBauJ+V+3S2b8+sBYoCfwX+CApThPaLEGZ7FQC2KMZq4pKyWhV3amq24D/Axaq6q+qehyX/Gpm5qSqOlZVD6nqCVzpqrqIFE1l97rAhao6RFXj1LUHvQfcmZlr+3hZVfep6hZcVWjHczxfSkoA27PgPP/xYj2G+xwUaOxtawfMV9W/yfrfVQfgVVXdqqr7gP+ks/9mVX1PVU8C44FLcKVuE+Ks7thkp71ASRHJc45JaqfP+2Mp/Fw4oycUkdzAUKA9riSW6G0qiavyS64scKmIHPBZlxv3RZ2SBCBvsnV5cdVxvrb6vN+MqxrLantxX9Ln6lSsqqoiMgWXUH8C7gKSekj6/bsSkcuA1T7nTemzvJSzf09p2eFzvqNe4SnD/0ZM9rMSlMlO84ETuKqh1BwBzvP5+eJzuN4Z5/KS0IWp7HsXrkrtRlxbWbmkw7zX5MP+bwU2qmoxn6WIqrZK5fxbfM6ZpDxnf7mW8Xl/GfB3KudLS3pTFMwBSotInTT28edzSH6dj4F2IlIWV632ubfe79+Vqm7xqiYLp5KcwJX+kv+eTASyBGWyjar+AzwHvCEit4nIeSKSV0RuFpH/erv9BrQSkQtE5GKg9zlc8k+ggIi0FpG8wDNA/lT2LYJLnntxX8zDkm3fCVzu8/Mi4JCI9PeeQ8otIleJSN1Uzv8J0FVE6nldvCsCfYApyfZ7UkSKi0gZXC+7T/y817RiPYOqrsP1nPzY6+qeT0QKiMidIjLA2+034HbvM7oCuD+9i6rqr7iOC+8Ds1X1gLcpo7+r9EwFeopIaa+ta0B6B5jwZAnKZCtVHQk8jksWu3F/XffAdXEG+AjXmL4J+JbMfUEnXesfoDvuC3MbrlSQ2nNHE3ClmW24KqYFybZ/AFQRkQMi8pXXntEGqIHrGZb0xZxim5WqzsZ9kX6IqzKciWsPeTfZrtOApbgE8T/vuhl1Rqyp7NMTGAO8ARwANuA6aMzwtr+C6wm404tzkp/XnowrhU5OWpHR35Uf3gNm4/6dLAO+yOR5TIgTVZuw0JhQICIKRKvq+mDHEk5EpBwu8eXNgg44JoRYCcoYY0xIsgRljDEmJFkVnzHGmJBkJShjjDEhyRKUMcaYkGQJykQ0b5qL/SKSP9m6VKe+EJFyIqLJR+lOPo2FiFwiIh94U4YcEpE1IvK8iBTKYIwviMgKEUkQkcHJtj3lTUuStBwTkUQRKeltX5Vse4KIzPA5vqmILBORgyLyl4g85LOttYjM87qj7xCR90WkSBpxbvKun3Stb5Ntv1xEvvZ+F3t8nm1DRK4UkR/EzVm1XkTaJjv2AW/9YRH5RkQCMYKGCTOWoEzE8rofN8aNeHBLFp/7AtzIGAWBa1S1CG4A22JAhQyebj3QD/fc0xlUdViykRVeBmJUdY+3varPtiK458o+9WLMixub8B3cM0d3AKNEpLp3+qLAi7ihg67EDeQ6PJ1Y/+UTT3Of30c+4DvgB9yoE6XxhjryEv004Gvc1B4PARO9h5URkSa4B6Nv9bZvxI1KYXI4S1Amkt2Le+B2HJDVs6Y+jpve425V3QTgDV7aS1WXZ+REqjpeVZOmC0mVuEHk7sU9OJuS63BjByYNMXQBcD7wkTqLgT9wo4+jqpNV9RtVPaqq+3EPwDbKSOw+ugB/q+ooVT2iqsd9fg+VcUnwFVU9qao/AD8D93jb2wCfquoqVY0DXgCuE5GMJnoTYSxBmUh2L24EhElACxHJyhGsbwS+UNXE1HYQkeVe9VlKy5uZuGZj3FQgn6eyvTPwuaoeAVDVnbiSSFdveKFrcAO3zkvl+OuAVenEMElEdovItz4lMYAGwCZxsyPv8apR05qdV4Crkv2c/L3vdpMDWYIyEUncFPJlgamquhQ3lM9dWXiJdKesUNVqyQZI9V26Z+KanYHPVPVw8g0ich5uiotxyTZ9jBv/8ARu9PCnVXVrsn0QkZu88z+XxvU74Qa8LQv8CMwWkWLettK46TNex5WW/gdM86r+1gK7cOMM5hWR5sD1nB6M9hugg4hUE5GCXgzKmYPVmhzIEpSJVJ2Bb5PaanBjwyVV86U39UWCz7rU9smqKSv84iWg9qRevXc7sA+Y63NMZdxgtPcC+YCqQD8ROWNadxFpgPv9tFPVP1OLQVV/VtVjXpXgf3Bj+CXN/3QMN+PuLK+abgQuiV+pqvG4Eexb46a+eAI34Gusd97vgUG4kuEmbzlE6uMmmhzCEpSJON5f4R2A673eaTtwI4dX96ql0pv6YjsuEaW1z/dAW3GzxaYWR/Iedr7L2xm8rba4BBSTyvbOwAQ988n7q4A/VXW2qiaq6lpcyeZmnxhrAtOB+1R1TgZjUk5Xxy0njWk+VHW5ql6vqiVUtQVutPVFPtvfUNVoVY3CJao8wMoMxmMijYbAvPO22JKVC27SvH24eYIu9ll+AkYCLXBVTvVwX7AVcZ0Huvmc42PcKNklcCWnjrgSQ5S3/QLcX/ofAWW9daWAUUC1DMabFyiAK8W86L3PnWyfb4EhqRxfGlfqq5BsfQXgMNDUu88KuB6DD3nbr8KNVn6HHzFehutAkc+L70ncaPQlvO2VgKO4trncuD8INgD5vO3VvOPOA/rieurl97YV8GIR7zoxwLBg/zuyJfhL0AOwxZasXnBtGiNTWN8BV8WUB7gP1yHgoPelPQDI5bNvcU5P07Ef1+usUbLzXQqM9c55CFiDq6o6L4PxjsOVPnyXLj7bS3kJ6IpUjh8I/F8q2zrgSiJJVWYvJ90nbuqPRC+JJS2rfI59G3jbe18VV0o6gqvenAPUSXat273f5UEvyVT12Tbc+z0eBmb53guua37SuXfgpnDP7c/vzpbIXmwsPmOMMSHJ2qCMMcaEJEtQxhhjQpIlKGOMMSHJEpQxxpiQlCf9XSJLyZIltVy5cn7vf+TIEQoVytDg1GHL7jUy2b1Gnki7z6VLl+5R1QuTr89xCapcuXIsWbLE7/1jYmJo0qRJ4AIKIXavkcnuNfJE2n2KyOaU1lsVnzHGmJAUsAQlImVE5EcRWe0N+dLLWz9YRLaJyG/e0srnmIHepGVrRaSFz/qW3rr1IjLAZ315EVnorf/EG5jSGGNMBAhkCSoBeEJVq+CG4n9URKp4215R1RreMhPA23Yn7on1lsCb3hQBuYE3cOOHVQE6+pznZe9cV+CeUr8/gPdjjDEmGwUsQanqdlVd5r0/hBvrrFQah9wKTFHVE6q6ETdkSj1vWa+qf6kbJXkKcKs3eVtT4DPv+PG4EZONMcZEgGzpJOFNvV0TWIgbcLKHiNwLLMGVsvbjktcCn8NiOZ3QtiZbXx83iOcBVU1IYf/k138IN800UVFRxMTE+B374cOHM7R/OLN7jUx2r5Enp9xnwBOUiBTGDZ/fW1UPishbuCmd1XsdiRu4M2BU9V3gXYA6depoRnq/RFpvmbTYvUYmu9fIk1PuM6C9+EQkLy45TVLVL8BNQ62qJ9VNlf0ergoP3KjRZXwOL+2tS239XqCYiORJtt4YY0w2OH4ctqc5r/S5CWQvPgE+AP5Q1VE+631nIW3L6UnJpgN3ikh+ESkPROMmNFsMRHs99vLhOlJMVzcM+4+4aa7BTdg2LVD3Y4wx5kxNm95H2bK/sG5dYM4fyCq+RsA9wAoR+c1b9xSuF14NXBXfJuBhAFVdJSJTgdW4HoCPqupJABHpAczGTYQ2VlVXeefrD0wRkReBX3EJ0RhjTIDs37+fIkWKAHlYt64JZctu5oorGgbkWgFLUKo6j9PTQfuamcYxQ4GhKayfmdJxqvoXp6sIjTHGBEhCQgLvvPMOzz33HEOHDqVw4W7s2XMvH3wAktI3fRbIcUMdGWOMyZhvv/2WPn36sHr1am644QYaNGhIp05w1VXQpk3grmtDHRljjElVnz59aNGiBSdOnOCrr75izpw5bN5cjdWrYeBAyBXALGIlKGOMMWfYv38/efLkoUiRIrRp04ZSpUrx2GOPkT9/flRh2DC4/HLo0CGwcVgJyhhjDODamd544w2uuOIKXnjhBQCaNWtG3759yZ8/PwA//giLFkG/fpAnwEUcS1DGGGOYPXs21atXp0ePHlSvXp277747xf2GDYNLLoHOnQMfkyUoY4zJ4YYMGULLli05ceIEX375JXPmzKFatWpn7bdwIcyZA48/DgUKBD4ua4MyxpgcaN++fcTFxXHxxRfTrl07zjvvvFPtTCn55x9XarroInj44eyJ0UpQxhiTg8THxzNmzBiio6Pp2bMnAFWqVDmjnSm5xES4+27YsAGmToUiRbInVktQxhiTQyS1Mz322GNUr16dp59+2q/jnnsOvv4aXn0Vrr8+sDH6sgRljDE5wBtvvEHLli2Ji4s79TxT9erV0z3us89g6FC4/37o3j0bAvVhbVDGGBOh9u3bx549e6hYsSIdOnQgLi6O7t27p1qVl9yKFa7d6Zpr4I03AjekUWqsBGWMMRHGt52pc+fOqCoXXnghffr08Ts5JSbCffe59qbPPwc/D8tSlqCMMSaC+LYz1axZk3feeQfJRNHnww9hyRIYOdI99xQMlqCMMSZCfPrpp7Rs2ZL4+HimTZvGd999l+LzTOk5cMCNs9eoEdx1V9bH6S9rgzLGmDC2b98+NmzYQN26dbnlllt48803ue+++/yuykvJ4MGwZw/Mnp397U6+rARljDFhKD4+ntGjR3PFFVfQvn17EhISyJ8/P4888sg5JadVq2DMGHjoIahZMwsDzgRLUMYYE2YWLlxItWrV6NmzJ7Vq1WLGjBnkyYKRW1WhZ084/3x48cUsCPQcWRWfMcaEkblz5zJgwACuuOIKpk2bxr/+9a9MdYJIyeefww8/uBJUyZJZcspzYiUoY4wJcXv37uXbb78F4LrrruOpp55i1apV3HLLLVmWnHbscA/iVq+efWPtpccSlDHGhKj4+Hhef/11oqOjad++PYcOHUJEuOmmm8iXL1+WXScxEbp2hUOHYNKkwM/z5C9LUMYYE4JmzZpFtWrV6NWrF7Vr1+bnn3+mSIBGaR09Gr75xj3zVLVqQC6RKSGSJ40xxiRZu3YtrVq1Ijo6OsvbmZL7/Xc3O+6//gWPPBKQS2SalaCMMSYE7N27l8mTJwNQqVIlZs2axcqVK7O0nSm5Y8fcg7gXXAAffBDcZ55SYgnKGGOCyLedqUuXLmzbtg2Ali1bZmk7U0p694bVq2HCBLjwwoBeKlMsQRljTJDMnDmTq6+++lQ707JlyyhVqlS2XPvtt+Hdd2HAALjppmy5ZIZZG5QxxgTB7t27adeuHaVLl2bGjBm0bt06YFV5yf30Ezz2GLRqFRoP5KbGSlDGGJNN9u7dy+uvv35q+osffviBlStX0qZNm2xLTps3w7//DRUqwOTJkDt3tlw2UyxBGWNMgPm2M/Xp04cVK1YA0KBBg4C3M/k6cgRuvRXi42HaNChaNNsunSmWoIwxJkBU9Yx2pjp16vD7779nagqMc5GQAF99BTfeCMuXw8cfQ6VK2RpCpgQsQYlIGRH5UURWi8gqEenlrb9ARL4TkXXea3FvvYjI6yKyXkSWi0gtn3N19vZfJyKdfdbXFpEV3jGvS3aVkY0xxg/Hjx/ngQceQFWZMWMGs2fP5qqrrsq262/fDkOHwuWXQ9u2EBsL48bBzTdnWwjnJJAlqATgCVWtAjQAHhWRKsAAYI6qRgNzvJ8BbgaiveUh4C1wCQ0YBNQH6gGDkpKat8+DPse1DOD9GGNMuvbs2cOgQYOIi4ujYMGCfP/996xYsSJb2plU3YO3L74I9evDpZfCM8+40tKXX8LGjXDvvQENIUsFrBefqm4HtnvvD4nIH0Ap4FagibfbeCAG6O+tn6CqCiwQkWIicom373equg9ARL4DWopIDHC+qi7w1k8AbgNmBeqejDEmNfHx8bz55psMHjyYQ4cOcf3119O0aVOqVKkS0OsePw4xMTBjBnz9NWzZ4tbXqwcvvADt24dHdV5KsqWbuYiUA2oCC4EoL3kB7ACivPelgK0+h8V669JaH5vC+pSu/xCuVEZUVBQxMTF+x3748OEM7R/O7F4jk91rYKkqCxYs4K233mLr1q3UqVOH7t27kytXroDFsmPHCZ566g/mzSvJ4sUXcPx4bgoUOEnt2vvp0GEv11yzlwsuiANcNd/27emcMEQFPEGJSGHgc6C3qh70LeKqqoqIBjoGVX0XeBegTp062qRJE7+PjYmJISP7hzO718hk9xpYiYmJ9O3bl4IFC/L111/TqlWrLK/K27XLzXS7fLkrJf34YyInT+bi0kuhSxc3jt4NN+SmYMGSQAhM5JRFApqgRCQvLjlNUtUvvNU7ReQSVd3uVeHt8tZvA8r4HF7aW7eN01WCSetjvPWlU9jfGGMCas+ePfznP//hqaeeokSJEnz55ZdcfPHFZ3QZP3YM/v7btQvlyeMWETh8GA4ePL0cOnT69dAh+OcfOHDAve7fD3/+CXv2nL52xYrQoUMsvXpdRt26kCuC+2IHLEF5Peo+AP5Q1VE+m6YDnYGXvNdpPut7iMgUXIeIf7wkNhsY5tMxojkwUFX3ichBEWmAqzq8FxgdqPsxxoSWxEQ3yd7GjaeXhQsrMnEiFChwesmf//R7EdflOj7evSYmpnxuEciXzx178iTExbljDh+OY968N1m06Hni4w8xe3YDSpZsT2LiZZw86c534oTrLbd7d8bvKXdu92xSsWKnX9u2hSpV3DQYVaq4jg9z5/5F/fqXncuvLywEsgTVCLgHWCEiv3nrnsIlpqkicj+wGejgbZsJtALWA0eBrgBeInoBWOztNySpwwTQHRgHFMR1jrAOEsYEiaorHRw65EoPx47B0aPu9fhxTn2Bnzzp1h85cvo1+fvDh13C2b/fnTdpSbpOfLwrZSQknBlD8eIlOe88d73jx921U0tCGfc/4HHgTwoUaE7FiqMoXtxNnpQnj0touXJBiRJQty5cdhmUKuW2JSScToiFC8P557ulSBG3JL1PSqLGCWQvvnlAar/qZinsr8CjqZxrLDA2hfVLgOx7qMCYEHXihPvCPnDAfaknve7eDYsXl2fyZNeOcezY6S/LpFKEyOkSRsGC7q/4pCSRtBw54vZP+jI9/3z3xZtUHZVUJZXZZCAChQqduVx2GdSq5bb5LuCuXbw4lC4N5cu7pWxZWLjwl7PaoBISTiespGOTltSG+UlMdKWmEyfcfnnzwl13vc/atcLIkYFpZzJns8FijQmSpBLHvn3uyz3pr/CkZd8+2LbNLX//7X72TT7798PevW79kSOpXydXrsu46CI3nUKhQqe/cPPnd++TqqX++Qd27nRf6IULu6VEidPvc+c+3X5y6JD7Ai9VylU9FSt2eilcGM47zyW7ggXd+wIF3PEi7vW881wsSa/58weu5JAnz+l7yIhDh/bw/PPP06NHDypVqsTYse9z/vnnkzdv3sAEas5iCcqYANmzB9audctff7kkk9Tld8cOl1ji4/0/X6FCp5NA0aKu9FC9uptsLmkpVsyVLJL2u/BCWL58Lk2bNgnELUakuLg43njjDZ5//nkOHz5MtWrVqFSpEiVKlAh2aDmOJShjMigx0ZVqli8/vfz5pyvFJLV7HD3qShtJcuWCqCjXwF2mjGujKFnSJZUSJVzCSWqMP3HCvRYv7koopUq54woUyFy8kdzLK6vNnDmTPn368Oeff9K8eXNGjRpF1apVgx1WjmUJyphUbNgAs2e75c8/T3cL9k08AOXKwZVXunYZ395jl13mnuCvVMntk8f+t4W8OXPmAATseSaTMfZfxuRoqi4RrV7tugbHxsLWrbBgAaxf7/YpXx7q1Dmz59XFF0O1anDVVaE/ZYFJXdK4ef/+979p2rQpQ4YM4aWXXrJ2phBhCcrkKPHxLiH9738X88EH8OOPrrouSZ48rkrtqqugVy9o0QKuuMK6/kaa5O1Ml19+OU2bNqVQoULBDs34sARlItaOHbBokVtWrYI1a1ypyD07U5mLLoImTeCGG1x35jJlXDuRtdlEttmzZ9OzZ0/+/PNPWrRowahRowI+oKvJHEtQJuzFx7vEs3KlS0QrVsCSJadHdc6d2w0Pc+WV7qn8SpVAdRGdO9ezklEOtG7dOnLlysXMmTO5OVwmRsqhLEGZsLJ/Pyxd6ua8SepBt3q16/UGriquQgW45hro3dtNOVCzpnvexldMzFFLTjnE7t27GTRoEHXr1qVr165069aNhx9+2NqZwoAlKBPSdu6Ezz5znRYWLXK96ZJceqnrqHDTTXD11a7dqHJl93CoMXFxcYwZM4YhQ4Zw+PBhLrroIgDyWHfKsGGflAk5iYnw3Xfw3nswbZprM7rkElca6tzZPUNUo4Z7CNWYlPzwww9069aNdevWWTtTGLMEZULG9u0wdqxLTJs3uwdZe/WC++937UfG+OvYsWPkzp2b//3vf7Rq1SrY4ZhMsgRlgioxEX74Ad5++3RpqVkz+O9/4dZb3RhtxqQnqZ3pyJEjNGnShFatWtGiRQurzgtz9umZbKfqetlNmQKffOKeQypRwnVqeOghiI4OdoQmXCRvZ2rfvj0AImLJKQLYJ2iyzb59rvru/fddt/C8eeHmm2H4cNf9O7NjzZmc6eeff6ZLly6sX7+em2++mZEjR7Jz585gh2WykCUoE3Br1sDrr8P48W4Q1SZNYOBAl5SKF0/3cGPOkJiYSK5cuShSpAgFChQ443kmS1CRJdUEJSKvqmpvEZkBaPLtqnpLQCMzYW/RIhg6FKZPd21JnTq5Tg/VqgU7MhOOdu/ezXPPPceRI0eYMGEC1apVY/ny5TagawRLqwT1kfc6IjsCMZFBFX76CV58Eb7/3pWQBg2C7t3BewzFmAyJi4tj9OjRDBkyhCNHjtCjR49TpShLTpEt1QSlqku917nZF44JV7Gx8PHHMGmSG+UhKsr1xOvWzY3+bUxmLFu2jDvuuIP169fTqlUrRowYwZX2zEGOkW4blIg0AgYDZb39BVBVvTywoZlwMGuW6+QQE+NKT/Xrw1tvuQdqbUQHk1nx8fHkzZuXUqVKUaJECUaPHk3Lli2DHZbJZv50kvgA6AMsBU4GNhwTLg4fhscfd73yLr/cVeN16uSmpjAms5LamVatWsXcuXOJiopiwYIFwQ7LBIk/CeofVZ0V8EhM2PjlF7jnHti4Efr3h+eftwdqzbnxbWc6evQojz76KCdOnKCAPXuQo/mToH4UkeHAF8CJpJWquixgUZmQs2sXzJkD33wDEye66cznzoXGjYMdmQl3a9eupU2bNqfamUaOHEnlypWDHZYJAf4kqPreax2fdQo0zfpwTCg5cQJefRUmT3bTWoDrlffww/DSS276c2My6+jRo5x33nmULVuW6Ohoa2cyZ0k3QanqDdkRiAktP/zguoavXQvXXQfDhsGNN7qZZ3PnDnZ0Jpzt2rWLZ599lu+//56VK1dSsGBBZs6cGeywTAhK60Hdu1V1oog8ntJ2VR0VuLBMsOzcCX37umq8yy93vfTsj1qTFU6cOMHo0aN54YUXTrUzJSQkBDssE8LSKkEV8l7tKZYc4MQJmDKlDB9/DMeOwbPPuuGIrKu4yQrbtm3j+uuvZ8OGDdx8882MGjXK2plMutJ6UPcd7/X57AvHZDdV+Pxz6NcPNm6sQOvWMHIkVKoU7MhMJDhw4ADFihXj0ksvpXHjxowZM8bamYzf/HlQ9/W0tqtqz6wLx2SnpUvdFBfz5rkp04cP/52+fasHOywTAZLamaZOncqaNWuIioriww8/DHZYJszk8mOfAkAtYJ231ADy4R7cXRqwyEzAbN8OXbu6qdPXroV33oFff4U6dfYHOzQT5k6cOMGIESOIjo5m7NixdOnShfz2kJzJJH8SVDWgiaqOVtXRQDOghqqOV9XxqR0kImNFZJeIrPRZN1hEtonIb97SymfbQBFZLyJrRaSFz/qW3rr1IjLAZ315EVnorf9ERPJl9OZzGlVXfRcd7bqOP/kkrFvnJgm0nnnmXB06dIirrrqKJ598ksaNG7Ny5UpeeeUVihUrFuzQTJjyJ0EVB3yfeCnsrUvPOCClyuZXVLWGt8wEEJEqwJ1AVe+YN0Ukt4jkBt4AbgaqAB29fQFe9s51BbAfuN+PmHIsVZeQ+vaFpk1h9Wp4+WUoWjTYkZlwt337dgCKFCnCnXfeyTfffMPXX39NJWvINOfInwT1EvCriIwTkfHAMmBYegep6k/APj/juBWYoqonVHUjsB6o5y3rVfUvVY0DpgC3ihtjvynwmXf8eOA2P6+V4yQlp5EjoUcPmDYNKlQIdlQm3O3atYuHH36YsmXLsmLFCgBeeOEFWrRokc6RxvjHnwd1PxSRWZweUaK/qu44h2v2EJF7gSXAE6q6HygF+I4IGeutA9iabH19oARwQFUTUtj/LCLyEPAQQFRUFDExMX4He/jw4QztH2pU4e23KzB1ahnato3l9tvXMzeVCVTC/V4zwu418+Li4vjyyy/56KOPOH78OG3btmXTpk3s3bs3y66RWTnlc80p94mqBmwBygErfX6OAnLjSm5DgbHe+jHA3T77fQC085b3fdbf4+1bEleySlpfxvc6aS21a9fWjPjxxx8ztH8oOXJEtWdPVVDt0UM1MTHt/cP5XjPK7jVz4uPj9aqrrlJAW7durWvWrMmyc2eFnPK5Rtp9Aks0he9rf8biyzKqujPpvYi8B3zt/bjNSzJJSnvrSGX9XqCYiORRV4ry3T/H++cfePNNeOUV2L0bevZ0Y+rZ5KMms9avX0+FChXIkycP3bt3p0KFCjRv3jzYYZkI508bVJYRkUt8fmwLJPXwmw7cKSL5RaQ8EA0sAhYD0V6PvXy4jhTTvYz7I66EBdAZmJYd9xDKEhLguefcSONPPQV16rjp1197zZKTyZydO3fy0EMPUbFixVPj5T3yyCOWnEy28KsEJSLXAtHq2qMuBAqr68yQ1jEfA02AkiISCwwCmohIDdxo6JuAhwFUdZWITAVWAwnAo6p60jtPD2A2rmpwrKqu8i7RH5giIi8Cv+KqBXMsVTfK+Nix0K6dS1A1awY7KhOuTpw4wWuvvcaLL77IsWPH6NWrFw0bNgx2WCaH8WckiUG4qTYqAR8CeYGJQKO0jlPVjimsTjWJqOpQXLtU8vUzgbOGOlbVv3C9/AwuIY0d68bQGzIk2NGYcKaqNGvWjJ9//pk2bdowYsQI6zJugsKfKr62wC3AEQBV/RsbQDakjBrl5md6+GE3u60xmbFixQri4+MREZ544glmz57NjBkzLDmZoPEnQcV5bT4KICKF0tnfZKOPPoInnoB//xveeMPamkzGJbUzVa9enffffx+Atm3bWjuTCTp/EtRUEXkH12vuQeB74L3AhmX88fXXcN99bmSISZNsuCKTMcePH+fll18mOjqaDz/8kD59+tCxY0o188YEhz8P6o4QkZuAg7h2qOdU9buAR2bSFBMD7dtDjRrw5Zdg43GajLrjjjuYPn06bdq0YeTIkVSsWDHYIRlzBn86STwOfGJJKXQsXgz/+tfpGW/PPz/9Y4wB+O233yhXrhzFihWjf//+PProo1aVZ0KWP1V8RYBvReT/RKSHiEQFOiiTulWr3BTsF14I334LJUsGOyITDnbu3MmDDz5IrVq1+O9//wtAw4YNLTmZkJZuglLV51W1KvAocAkwV0S+D3hk5iyxsdC8uavO+/57KJXq6IPGOCdOnODjjz8mOjqacePG0bt3b/r16xfssIzxS0aGOtoF7MANM3RRYMIxqTl5Ejp1goMHYf58V71nTHoee+wx3nvvPW655RaGDx9u7UwmrPjTBtUd6ABcCHwKPKiqqwMdmDnTsGFu2KLx4+Gqq4IdjQllv/76K8WKFaN8+fI8+eSTVKxYkb59+wY7LGMyzJ82qDJAb1WtqqqDLTllv59/hsGDXQnqnnuCHY0JVTt27OCBBx6gdu3aDB48GIDo6Gjq1KkT3MCMyaRUS1Aicr6qHgSGez9f4LtdVf2djNCcg/374a67oFw5N0K5PYhrkjt+/DivvfYaQ4cO5dixY/Tp04dnn3022GEZc87SquKbDLQBluJGkfD9alTAWkECLGkA2L//dqUo605uUjJs2DBeeOEFa2cyESfVBKWqbbzX8tkXjvH1ySfw6afw8stQz4bFNT5+/fVXEhMTqV27Nr169eK6667jxhtvDHZYxmSpdNugRGSOP+tM1jp+HAYMcCNFWPu2SbJjxw7uv/9+ateuzcCBA1FVLrjgAktOJiKl1QZVADgPN59TcU5X8Z0P2BM4ATZmDGzeDB98ALmydVpJE4qOHz/Oq6++ytChQzlx4gSPP/44zzzzDGKNkiaCpdUG9TDQG7gU1w6V9D/hIDAmsGHlbHv3wosvQqtW0KxZsKMxoWDcuHEMHDiQW265hREjRhAdHR3skIwJuLTaoF4DXhORx1R1dDbGlOO98AIcOgTeiDQmh/r111/ZtWsXLVq04L777qNy5co0adIk2GEZk238Gc18tIhcBVQBCvisnxDIwHKqdevcvE4PPABVqwY7GhMMO3bs4Omnn+bDDz+kWrVqNG/enHz58llyMjmOP50kBgGjveUG4L+4GXZNAAwc6Mbas5lxc57jx4/z0ksvER0dzUcffcTjjz/O3LlzrZ3J5Fj+NL+3A5oBO1S1K1AdKBrQqHKon36Czz+Hfv3g4ouDHY3Jbt9//z0DBw6kWbNmrF69mhEjRlC0qP1XMzmXP4PFHlPVRBFJEJHzcYPGlglwXDnOoUPQpQuUL++mcDc5w7Jly1i1ahX33HMPrVu3ZtGiRdStWzfYYRkTEvwpQS0RkWK4ad6XAsuA+YEMKifq1ct1K//oIyhUKNjRmEDbvn079913H3Xq1OHZZ58lLi4OEbHkZIwPf+aD6q6qB1T1beAmoLNX1WeyyBdfwIcfuvanRo2CHY0JpOPHj/Of//yHihUrMnHiRJ544gl+//138uXLF+zQjAk5aT2oWyutbaq6LDAh5Sx//w0PPgh16sCgQcGOxgTamjVrePrpp0+Nm2fPMxmTurTaoEamsU2BplkcS46jCvfdB8eOwcSJkDdvsCMygbBs2TLmzJnDk08+SY0aNVi9ejWVK1cOdljGhLy0HtS9ITsDyYnGjoXZs900GpUqBTsak9W2b9/O008/zbhx47jwwgt58MEHKVasmCUnY/zkz4y696a03h7UPTdxcTBkCDRoAN26BTsak5WOHTvGK6+8wrBhw4iLi+OJJ57gmWeesS7jxmSQP93MfbsVFcA9E7UMsAR1DsaPhy1b4J13bBLCSLN//36GDRvGjTfeyIgRI7jiiiuCHZIxYcmfoY4e8/3Z63I+JVAB5QRxcTB0qJvjqUWLYEdjssLSpUuZOHEio0aN4tJLL+WPP/6gTBl7XNCYc5GZiRyOAOlOYigiY0Vkl4is9Fl3gYh8JyLrvNfi3noRkddFZL2ILPftQSginb3914lIZ5/1tUVkhXfM6xJG48FMmOCeeRo0yEpP4W779u107dqVunXrMmnSJLZs2QJgycmYLODPWHwzRGS6t3wNrAW+9OPc44CWydYNAOaoajQwx/sZ4GYg2lseAt7yrn0BMAioD9QDBiUlNW+fB32OS36tkBQf70pPderAzTcHOxqTWcePH2fYsGFER0czefJk+vbty7p16yhbtmywQzMmYvjTBjXC530CsFlVY9M7SFV/EpFyyVbfCjTx3o8HYoD+3voJqqrAAhEpJiKXePt+p6r7AETkO6CliMQA56vqAm/9BOA2YJYf9xNUH30EmzbB6NFWegpniYmJvPXWWzRv3pzhw4dToUKFYIdkTMTxpw1qLoA3Dl8e7/0FSUkjg6JUdbv3fgcQ5b0vBWz12S/WW5fW+tgU1qdIRB7ClcyIiooiJibG74APHz6cof3TkpAgPPNMPSpWTKBQoaVk0WmzTFbea6jLzL2uXbuWTz/9lH79+pEvXz7GjBlD0aJF2bp1K1u3bk3/BEFin2vkySn36U8384eAIcBxIBE3s64Cl5/LhVVVRUTP5RwZuNa7wLsAderU0YzMqxMTE5Nl8/CMGwfbt7ueezfckDXnzEpZea+hLiP3+vfff5/xPFNUVBTVq1cPbIBZyD7XyJNT7tOfThJPAlepajlVvVxVy6tqZpPTTq/qDu91l7d+G2eOkF7aW5fW+tIprA9ZiYluhtxq1aBNm2BHY/wRHx/P0KFDqVixIpMnT6Zfv36sW7curJKTMeHMnwS1ATiaRdebDiT1xOsMTPNZf6/Xm68B8I9XFTgbaC4ixb3OEc2B2d62gyLSwOu9d6/PuULSzJnwxx/w5JPW9hQucufOzbRp02jRogWrV6/m5ZdftodtjclG/nSSGAj8IiILgRNJK1W1Z1oHicjHuE4OJUUkFtcb7yVgqojcD2wGOni7zwRaAetxybCrd419IvICsNjbb4hP21d3XE/BgrjOESHdQWLECChTBu64I9iRmLQsWbKEQYMGMX78eEqWLMmPP/5IIZv/xJig8CdBvQP8AKzAtUH5RVU7prKpWQr7KvBoKucZC4xNYf0S4Cp/4wmmxYth7lwYOdIGhA1Vvu1MF110EWvWrOHaa6+15GRMEPmToPKq6uMBjySCDR8O558PDzwQ7EhMcqrKsGHD+M9//kN8fDz9+/fnqaee4vzzzw92aMbkeP4kqFleT74ZnFnFl5lu5jnOX3/B559D374uSZnQIiL8/vvvtGzZkv/+979cfvk5dU41xmQhfxJUUlXdQJ9159zNPKd45RXInRt6ptliZ7LTkiVL6N27N5MnT6ZKlSpMmjSJvFb3akzI8WfK9/IpLJac/LB3r5vz6a67oFSqjxGb7PL333/TuXNn6taty5YtW06Nm2fJyZjQZPNBBdAbb8DRo656zwTX8OHDGTx4MAkJCfTv35/GjRvTsmVYDN9oTI5l80EFyD//uOq9Nm3gqrDoaxh5VJWkQe537959qp2pQoUKOWKYGGPCnc0HFSCvvw4HDsDgwcGOJGdavHgxvXv3ZtCgQTRv3pyXXnqJXLkyM7uMMSZYAjYfVE524ACMGgW33AK1awc7mpwlqZ2pXr16rF+/niNHjgBYcjImDPnTBjUD12sPXEKrAkwNZFDh7rXXrPQUDKNHj2bAgAGn2pnseSZjwlvA5oPKqQ4ccG1Pt90GNWsGO5rIp6qoKrly5SJv3ry0atWKl19+2Z5nMiYCpFrvISJXiEgjVZ3rs/wMlBURm50tFa+84jpIWOkp8BYvXsy1117Le++9B8DDDz/Mp59+asnJmAiRVsX8q8DBFNYf9LaZZPbvh1dfhdtvB5uRIXC2bdt2qp1pw4YNp6rxxIaJNyaipFXFF6WqK5KvVNUVKUzlbnBtTwcPwqBBwY4kcr3//vv06tWLhIQEBgwYwMCBA62dyZgIlVaCKpbGtoJZHEfYU4WPPoLmzd2khCbrqCrx8fHky5ePMmXKcPPNN9u4ecbkAGlV8S0RkQeTrxSRB4ClgQspPK1Y4QaGbdcu2JFElkWLFnHttdfy3HPPAdCiRQs+++wzS07G5ABpJajeQFcRiRGRkd4yF7gf6JUt0YWRL75wM+XeemuwI4kM27Zt495776V+/fps2LCBKlWqBDskY0w2S7WKT1V3Ag1F5AZOTwz4P1X9IVsiCzNffAGNG8NFFwU7kvA3ZcoU7r///lPtTE899RRFihQJdljGmGzmz1BHPwI/ZkMsYWvdOlfF9+qrwY4kfKkqR44coXDhwlSrVo3WrVvz8ssvU768DVpiTE5l479kgS+/dK+33RbUMMLWokWLaNSoEffffz8AVapUYerUqZacjMnhLEFlgS++cGPulS0b7EjCi287019//UWLFi2CHZIxJoT4M9SRSUNsLCxcCEOHBjuS8DJz5kzat2/PyZMnGThwIAMHDrR2JmPMGawEdY6++sq93n57UMMIC6rKnj17AKhbty7t2rXjjz/+YNiwYZacjDFnsQR1jr78Eq68EipXDnYkoW3hwoU0atSI1q1bk5iYyIUXXsj48eOtnckYkypLUOdgzx6YOxfatg12JKErNjaWe+65hwYNGrBx40a6desW7JCMMWHC2qDOwYwZcPKkVe+l5pdffuGmm27i5MmTPPXUUwwYMMCq8owxfrMS1DmYPh0uuwxq1Qp2JKFDVdmyZQsAtWvXpmvXrqxZs4ahQ4dacjLGZIglqExShXnzoGlTN8SRgQULFtCwYUMaNWrE0aNHyZ8/P2PGjKFcuXLBDs0YE4YsQWXShg2uDeqaa4IdSfBt3bqVu+++m2uuuYZNmzYxZMgQChQoEOywjDFhztqgMumXX9xrTk9Qa9asoVatWiQmJlo7kzEmSwWlBCUim0RkhYj8JiJLvHUXiMh3IrLOey3urRcReV1E1ovIchGp5XOezt7+60Skc3bew/z5UKQI5MRBthMTE1m9ejUAlSpVol+/ftbOZIzJcsGs4rtBVWuoah3v5wHAHFWNBuZ4PwPcDER7y0PAW+ASGjAIqA/UAwYlJbXsMH8+NGgAuXNn1xVDw8KFC2nYsCH16tVjx44diAiDBw+2diZjTJYLpTaoW4Hx3vvxwG0+6yeoswAoJiKXAC2A71R1n6ruB74DWmZHoIcOudHLc1L1XlI7U4MGDdi8eTNjxozhIptbxBgTQMFqg1LgWxFR4B1VfReIUtXt3vYdQJT3vhSw1efYWG9dauvPIiIP4UpfREVFERMT43eghw8fPmv/ZcuKkZhYg0KFlhMTs8/vc4W6lO4VYN++fXTq1ImTJ09y9913c9ddd1GwYEF++umn7A8yi6R2r5HI7jXy5JT7DFaCulZVt4nIRcB3IrLGd6Oqqpe8soSXAN8FqFOnjjZp0sTvY2NiYki+/88/u9cHH6xG8WyrVAw833tNTExk0aJFNGjQAIBRo0bRqlUrykbIkO0pfa6Ryu418uSU+wxKFZ+qbvNedwFf4tqQdnpVd3ivu7zdtwFlfA4v7a1LbX3AzZ/vxt+LpOTkK+l5poYNG7Jq1SoAHnnkkYhJTsaY8JDtCUpEColIkaT3QHNgJTAdSOqJ1xmY5r2fDtzr9eZrAPzjVQXOBpqLSHGvc0Rzb11AqcKCBZHZ/rRr1y46derENddcw5YtW/jwww+58sorgx2WMSaHCkYVXxTwpbjhF/IAk1X1GxFZDEwVkfuBzUAHb/+ZQCtgPXAU6AqgqvtE5AVgsbffEFUNeIPQunWwd2/kJaijR4/y4IMPEhcXx9NPP82AAQMoXLhwsMMyxuRg2Z6gVPUvoHoK6/cCzVJYr8CjqZxrLDA2q2NMy/z57rVhw+y8amAkJiby3Xff0bx5c8477zwef/xxunTpYlV5xpiQEErdzMPCL79AsWLhP/9TUjtTy5YtmTNnDgDXX3+9JSdjTMiwBJVB8+dD/fqQK0x/c1u3bj2rnalp06bBDssYY85iY/FlwMGDsHIl/PvfwY4kcxITE7nhhhvYtm0bzzzzDP3797d2JmOyQHx8PLGxsRw/fjxbrle0aFH++OOPbLlWVipQoAClS5cmb968fu1vCSoDFi1yvfjCqYNEYmIiX3zxBbfeeit58+bl/fffp3z58laVZ0wWio2NpUiRIpQrVw7Jhvl3Dh06FHbjXqoqe/fuJTY2lvLly/t1TJhWVAXH/Plu7qf69YMdiX/mz5/PNddcQ/v27fn4448BaNKkiSUnY7LY8ePHKVGiRLYkp3AlIpQoUSJDpUxLUBkwfz5UrQpFiwY7krRt3bqVu+66i4YNGxIbG8u4ceO4++67gx2WMRHNklP6Mvo7siq+DKhVCxo3DnYU6bvzzjtZtmwZzz77LP369bN2JmNMWLISVAa8+CIMHBjsKM6WmJjIxIkT2bfPPaf81ltvsWbNGoYMGWLJyRiTonHjxtGjRw8A3n77bSZMmAC4SUhr1KhBzZo12bBhA5MnTw5ajJagwtwvv/xCgwYNuOeee/jggw8AqFatmrUzGWP81q1bN+69914AvvrqK9q1a8evv/7K1q1bg5qgrIovTG3ZsoX+/fszZcoULr30UiZMmECnTp2CHZYxOV7v3vDbb1l7zho14NVX095n06ZNtGnThpUrVwIwYsSIU9NyVK9enblz55KQkMDYsWOpV6/eGccOHjyYwoULU6VKFV599VVy587NnDlzOHbsGH/88Qc1atSgc+fO9OnTJ2tvLB2WoMLUk08+yfTp03n22Wfp378/hQoVCnZIxpgQdfToUX777Td++ukn7rvvvlNJLLlWrVrRrVs3ChcuTN++fYmJiWHEiBF8/fXX2RyxYwkqTCQmJjJp0iTq169PxYoVGT58OMOHD+eyyy4LdmjGGB/plXSCoWPHjgBcd911HDx4kAMHDgQ3ID9ZG1QYSGpnuvfee3nvvfcAuOyyyyw5GWNOyZMnD4mJiad+9n3eKHn37nDpEm8JKoRt3ryZjh070qhRI7Zt28aECRN4+eWXgx2WMSYERUVFsWvXLvbu3cuJEyfOqJb75JNPAJg3bx5FixalqJ8PcxYpUoRDhw4FJF5/WBVfCBs9ejRfffUVzz33HP369bN2JmNMqvLmzctzzz1HvXr1KFWqFJV9plwoUKAANWvWJD4+nrFj/Z+hqFq1auTOnZvq1avTpUuXbO8kgarmqKV27dqaET/++GOG9j8XJ0+e1PHjx+tPP/2kqqr79+/XzZs3Z9v1s/Neg83uNTIF615Xr16drdc7ePCg3/tef/31unjx4gBGkzEp/a6AJZrC97VV8YWIpHamzp07n/oLp1ixYtbOZIzJsSxBBdmWLVtOtTP9/fffTJgw4dQDt8YYc65iYmKoU6dOsMPIFGuDCrJp06Yxbdo0Bg0axJNPPmntTMYY47EElc0SExP56KOPKFiwIB06dKBbt27cdtttlClTJtihGWNMSLEqvmz0888/U79+fbp06XJqfKu8efNacjLGmBRYgsoGmzdv5s477+Taa69l+/btTJw4kS+++CLYYRljTEizBJUNVqxYwfTp0xk0aBBr166lU6dO5Mplv3pjTOgqV64ce/bsCWoM1gYVAEntTPv27aNPnz60bt2ajRs3EhUVFezQjDE5wKnniML8D2FLUFls3rx59O7dm6VLl9KkSRN69epFrly5LDkZk4M0adLkrHUdOnSge/fuHD16lFatWp21vUuXLnTp0oU9e/bQrl27M7bFxMSke81NmzbRokUL6tevz9KlS6lXrx4rVqzg2LFjtGvXjueffx5wJaPOnTszY8YM4uPj+fTTT6lcuTJ79+6lY8eObNu2jWuuuQb3/KwzatSoU89nPvDAA/Tu3ZtNmzbRsmVLGjRowC+//ELdunXp2rUrgwYNYteuXUyaNOmsaT0yKrzTawjZunUrd9xxB40bN2bHjh1MnDiROXPmhP1fMMaY8LFu3Tq6d+/OqlWrGDlyJEuWLGH58uXMnTuX5cuXn9qvZMmSLFu2jEceeYQRI0YA8Pzzz3PttdeyatUq2rZty5YtWwBYunQpH374IQsXLmTBggW89957/PrrrwCsX7+eJ554gjVr1rBmzRomT57MvHnzGDFiBMOGDTvn+7ESVBbZv38/M2fOZPDgwfTt29eeZzImB0urxHPeeeelub1kyZJ+lZhSUrZsWRo0aADA1KlTeffdd0lISGD79u2sXr2aatWqAXD77bcDULt27VMdtn766adT71u3bk3x4sUBVyvUtm3bU99pt99+O//3f//HLbfcQvny5bn66qsBqFq1Ks2aNUNEuPrqq9m0aVOm7sGXJahMSkxMZMKECaxYsYKRI0dSrVo1YmNj/R4l2BhjslpSEtm4cSMjRoxg8eLFFC9enC5dupwx/Ub+/PkByJ07NwkJCZm+XtJ5AHLlynXq51y5cp3TeU+d85zPkAPNmzePevXq0bVrV+bPn3/qg7fkZIwJBQcPHqRQoUIULVqUnTt3MmvWrHSPue666049nzlr1iz2798PQOPGjfnqq684evQoR44c4csvv6Rx48YBjT+JlaAy4O+//6ZPnz5MnTqV0qVLM2nSJDp27Bg2k38ZY3KG6tWrU7NmTSpXrkyZMmVo1KhRuscMGjSIjh07UrVqVRo2bHhqoOpatWrRpUuXUx0eHnjgAWrWrJklVXjpSmmI83BagJbAWmA9MCC9/c9luo3t27drVFSUDh48WI8cOZKh84QDm5YhMtm9Bl4oT7cRajIy3UZYl6BEJDfwBnATEAssFpHpqro6ENe7+OKL2bRpEwUKFAjE6Y0xxvgI9zaoesB6Vf1LVeOAKcCtgbygJSdjjMkeoj4PY4UbEWkHtFTVB7yf7wHqq2qPZPs9BDwEEBUVVXvKlCl+X+Pw4cMULlw464IOYXavkcnuNfCKFi1KhQoVsq09+uTJk+TOnTtbrpWVVJUNGzbwzz//nLH+hhtuWKqqZ01aFdZVfP5S1XeBdwHq1KmjKT3lnZqYmJgUnwqPRHavkcnuNfA2btxIXFwcJUqUyJYkdejQIYoUKRLw62QlVWXv3r0UK1aMmjVr+nVMuCeobYDvXBWlvXXGGJNtSpcuTWxsLLt3786W6x0/fjwsmxsKFChA6dKl/d4/3BPUYiBaRMrjEtOdwF3BDckYk9PkzZuX8uXLZ9v1YmJi/C6FhLOwTlCqmiAiPYDZQG5grKquCnJYxhhjskBYJygAVZ0JzAx2HMYYY7JWuHczN8YYE6HCupt5ZojIbmBzBg4pCQR3WsnsY/camexeI0+k3WdZVb0w+cocl6AySkSWpNQ/PxLZvUYmu9fIk1Pu06r4jDHGhCRLUMYYY0KSJaj0vRvsALKR3WtksnuNPDniPq0NyhhjTEiyEpQxxpiQZAnKGGNMSLIE5RGRliKyVkTWi8iAFLbnF5FPvO0LRaRcEMLMEn7caxcR2S0iv3nLA8GI81yJyFgR2SUiK1PZLiLyuvd7WC4itbI7xqzix702EZF/fD7T57I7xqwgImVE5EcRWS0iq0SkVwr7RMTn6ue9RsTnmqqUptnNaQtuHL8NwOVAPuB3oEqyfboDb3vv7wQ+CXbcAbzXLsCYYMeaBfd6HVALWJnK9lbALECABsDCYMccwHttAnwd7Diz4D4vAWp574sAf6bw7zciPlc/7zUiPtfUFitBOf7MzHsrMN57/xnQTLJrdrKsle2zEAeLqv4E7Etjl1uBCeosAIqJyCXZE13W8uNeI4KqblfVZd77Q8AfQKlku0XE5+rnvUY0S1BOKWCrz8+xnP0P4dQ+qpoA/AOUyJbospY/9wrwb6965DMRKZPC9kjg7+8iUlwjIr+LyCwRqRrsYM6VV81eE1iYbFPEfa5p3CtE2OfqyxKUSckMoJyqVgO+43TJ0YSvZbjxzqoDo4GvghvOuRGRwsDnQG9VPRjseAIpnXuNqM81OUtQjj8z857aR0TyAEWBvdkSXdZK915Vda+qnvB+fB+onU2xZbccMyOzqh5U1cPe+5lAXhEpGeSwMkVE8uK+sCep6hcp7BIxn2t69xpJn2tKLEE5p2bmFZF8uE4Q05PtMx3o7L1vB/ygXitlmEn3XpPV19+Cq/uORNOBe71eXw2Af1R1e7CDCgQRuTipzVRE6uH+74fdH1jePXwA/KGqo1LZLSI+V3/uNVI+19SE/YSFWUFTmZlXRIYAS1R1Ou4fykcish7XGH1n8CLOPD/vtaeI3AIk4O61S9ACPgci8jGul1NJEYkFBgF5AVT1bdxEl62A9cBRoGtwIj13ftxrO+AREUkAjgF3hukfWI2Ae4AVIvKbt+4p4DKIuM/Vn3uNlM81RTbUkTHGmJBkVXzGGGNCkiUoY4wxIckSlDHGmJBkCcoYY0xIsgRljDEmU9IbpDiF/Tv4DH47Ob39LUEZ40NEVERG+vzcV0QGZ9G5x4lIuyw610Jv9OotcubI8+Wy4vwpXO+pTBzTRUTGBCIeEzLGAS392VFEooGBQCNVrQr0Tu8YS1DGnOkEcHuoPY3vjV5yiqrWV9UawHO4kfVreMumjJwnAzKcoEzkS2mQYhGpICLfiMhSEfk/EansbXoQeENV93vH7krv/JagjDlTAvAu0Cf5huQlIBE57L02EZG5IjJNRP4SkZdEpJOILBKRFSJSwec0N4rIEhH5U0TaeMfnFpHhIrLYG6D3YZ/z/p+ITAdWpxe4iPzLK1n9KiLfi0iUt36wiHwkIj/jHja/UES+86pZ3heRzUkJWUTu9uL+TUTe8WJ7CSjorZuU2n7e+q7evS3CPWhqcp53gcdUtTbQF3jTW18RqCgiP4vIAhFJt+RlCcqYs70BdBKRohk4pjrQDbgS9/R/RVWthxvL8DGf/crhpjxpDbwtIgWA+3HD8dQF6gIPikh5b/9aQC9VrehHDPOABqpaEzeNSj+fbVWAG1W1I26UiR+8apbP8EYmEJErgTtwVTA1gJNAJ1UdABzzSmidUttP3BBZz+MS07XeNU0OIm5g24bAp97oF+/g5rUCN3JRNG7Ek47AeyJSLK3z2VBHxiSjqgdFZALQEzd8jD8WJ433JiIbgG+99SuAG3z2m6qqicA6EfkLqAw0B6r5lM6K4v4jxwGLVHWjnzGUBj7xEkU+wPe46aqadC/XAm29e/1GRPZ765vhBgZe7A3vVhBIqRomtf3qAzGqutv7PXyC+6vZ5By5gAPeHy7JxeImj4wHNorIn7h/54vTOpkx5myv4ko2hXzWJeD9nxGRXLgkkOSEz/tEn58TOfMPweRjiylu5tfHfNqRyqtqUoI7koGYR+NmQr4aeBgo4LPNn/MIMN4njkqqOvgc9jM5jDcdyEYRaQ9uwFsRqe5t/gpXesKrUq4I/JXW+SxBGZMCVd0HTMUlqSSbOD31yC14g7FmUHsRyeW1S10OrMUN3PuIuKkVEJGKIlIorZOkoiinp5XonMZ+PwMdvGs1B4p76+cA7UTkIm/bBSJS1tsWnxRfGvstBK4XkRLevu0zcQ8mjIgbpHg+UElEYkXkfqATcL+I/A6s4vSM3bOBvSKyGvgReFJV0xx53ar4jEndSKCHz8/vAdO8/3jfkLHSTZItwCLgfKCbqh4XkfdxbVPLxNWZ7QZuy8S5B+Pq/vcDPwDlU9nveeBjEbkH9+WyAzikqntE5BngW6+EGA88CmzGNXwvF5FlXjvUWfup6gJxXfLnAweA3zJxDyaMeG2aKTmrA4Q3yvrj3uIXG83cmBxGRPIDJ72pV64B3kqlzcCYoLISlDE5z2XAVK/0E4d7PsWYkGMlKGOMMSHJOkkYY4wJSZagjDHGhCRLUMYYY0KSJShjjDEhyRKUMcaYkPT/CXalnXRcCR8AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "perf_lgwum.plot_uplift(metric=\"qini\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Conclusion\n", "From the Qini curves, you can see the Uplift DRF algorithm performs better than the LGWUM algorithm. The main reason is, that the split in Uplift DRF can be more precious thanks to information about both treatment and control groups." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Uplift trees modeling sources:\n", "\n", "- N. J. Radcliffe, and P. D. Surry, “Real-World Uplift Modelling withSignificance-Based Uplift Trees”, Stochastic Solutions White Paper, 2011.\n", "\n", "- P. D. Surry, and N. J. Radcliffe, “Quality measures for uplift models”, 2011.\n", "\n", "## References\n", "\n", "- P. Rzepakowski, and S. Jaroszewicz, “Decision trees for uplift modeling with single and multiple treatments”, 2012.\n", "\n", "- Hugh Huyton, “Criteo Uplift Modelling“, 2021, https://www.kaggle.com/code/hughhuyton/criteo-uplift-modelling/notebook.\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.4" } }, "nbformat": 4, "nbformat_minor": 4 }