{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# \"Causality modelling in Python for data scientists\"\n", "> \"Data science is increasingly commonplace in industry and the enterprise. Industrial data scientists have a vast toolbox for descriptive and predictive analyses at their disposal. However, data science tools for decision-making in industry and the enterprise are less well established. Here we survey Python packages that can aid industrial data scientists facilitate intelligent decision-making through causality modelling.\"\n", "- hidden: true\n", "- toc: true\n", "- branch: master\n", "- badges: true\n", "- comments: true\n", "- categories: [causal inference, causal discovery, causality modelling, python]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The need for causality modelling\n", "\n", "Intelligent planning and decision-making lie at the heart of most business success.\n", "\n", "The decisions that our business needs to evaluate can range from those that are relatively low effort and we take potentially thousands or millions of times a day to those that are high effort and are taken every couple of months:\n", "\n", "1. What will happen if I show an advertising banner to a particular user?\n", "2. What will happen if I change the retail prices for certain products in my shop?\n", "3. What will happen if I alter my manufacturing process?\n", "4. What will happen if I swap out a particular mechanical piece in a vehicle I develop?\n", "5. What will happen if I invest in new property, machinery, or processes?\n", "6. What will happen if I hire this applicant?\n", "7. What if I increase remuneration of my workforce?\n", "\n", "As industrial data scientists we are oftentimes called upon to evaluate these proposed business decisions using analytics, machine learning methodologies, and past data.\n", "\n", "What we may end up doing for the above proposed business decisions is:\n", "\n", "1. Compute and rank past click-through rates for given pairs of ad banner and user,\n", "2. Correlate past demand with set retail prices for product groups of interest,\n", "3. Correlate past manufacturing parameters with achieved output quality,\n", "4. Correlate the mechanical behavior of my vehicles with the mechnical parts used in it,\n", "5. Use past data to forecast the development of real estate prices,\n", "6. Use past data to correlate and predict the productivity of my team given e.g. its size or makeup, and\n", "7. Use past data to correlate productivity and remuneration levels.\n", "\n", "The way I formulated these is already pretty suggestive - but essentially some of our common approaches to evaluating business decisions do not compare our business outcomes with and without said business decisions but they rather look at our data outside the context of decision-making.\n", "\n", "Put another way, we oftentimes analyze past data without considering the state our business or customer is in when those data were generated.\n", "For illustration:\n", "\n", "![Data fusion process (5)](https://user-images.githubusercontent.com/3273502/85201681-a999f580-b301-11ea-9174-056649a1bebb.png)\n", "\n", "So really when tasked with evaluating the above proposed business decisions we should instead think in terms of questions akin the following:\n", "\n", "1. How would the user of interest behave differently if we didn't show them (and pay for) a banner now?\n", "2. For each Euro we shave off a price tag how much higher will our revenue be since more customers are inclinced to place an order?\n", "3. \n", "4.\n", "5.\n", "6.\n", "7." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## How to do causality modelling\n", "\n", "The authors Hünermund and Bareinboim (https://arxiv.org/abs/1912.09104) proposed a methodology they called data-fusion process.\n", "\n", "The data-fusion process maps out the individual steps necessary for evaluating the impact of past and potential future decisions:\n", "\n", "![The data-fusion process.](https://user-images.githubusercontent.com/3273502/85201682-aacb2280-b301-11ea-9529-59e63c19f945.png \"With the data-fusion process we iterate through applying and validating our causal understanding of our system, modelling the impact of our proposed business decision on our system, and estimate its impact using our validated causal model and historical data. Adapted from Hünermund and Bareinboim: https://arxiv.org/abs/1912.09104.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use case: The impact of direct marketing on customer behavior\n", "\n", "We'll use a data set provided by UCI (https://archive.ics.uci.edu/ml/datasets/Bank+Marketing) that demonstrates the potential impact of direct marketing on customer success.\n", "\n", "Let's dive right in, download the data set and see what we are working with." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The direct marketing success data set" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "!wget --quiet https://archive.ics.uci.edu/ml/machine-learning-databases/00222/bank.zip" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "!unzip -oqq bank.zip" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [], "source": [ "#collapse\n", "df = pd.read_csv('bank.csv', delimiter=';')\n", "df['success'] = df['y']\n", "del df['y']\n", "df['success'] = df['success'].replace('no', 0)\n", "df['success'] = df['success'].replace('yes', 1)\n", "del df['duration']\n", "df['no_contacts'] = df['campaign']\n", "del df['campaign']" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
agejobmaritaleducationdefaultbalancehousingloancontactdaymonthpdayspreviouspoutcomesuccessno_contacts
030unemployedmarriedprimaryno1787nonocellular19oct-10unknown01
133servicesmarriedsecondaryno4789yesyescellular11may3394failure01
235managementsingletertiaryno1350yesnocellular16apr3301failure01
330managementmarriedtertiaryno1476yesyesunknown3jun-10unknown04
459blue-collarmarriedsecondaryno0yesnounknown5may-10unknown01
\n", "
" ], "text/plain": [ " age job marital education default balance housing loan \\\n", "0 30 unemployed married primary no 1787 no no \n", "1 33 services married secondary no 4789 yes yes \n", "2 35 management single tertiary no 1350 yes no \n", "3 30 management married tertiary no 1476 yes yes \n", "4 59 blue-collar married secondary no 0 yes no \n", "\n", " contact day month pdays previous poutcome success no_contacts \n", "0 cellular 19 oct -1 0 unknown 0 1 \n", "1 cellular 11 may 339 4 failure 0 1 \n", "2 cellular 16 apr 330 1 failure 0 1 \n", "3 unknown 3 jun -1 0 unknown 0 4 \n", "4 unknown 5 may -1 0 unknown 0 1 " ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our tabular marketing and sales data shows a number of features we observe about a given customer and our interaction with them:\n", "\n", "- The customer's age, job, marital status, education, current account balance, and whether or not they already took out a loan are recorded,\n", "- Our direct marketing interaction with a given customer is also recorded, for instance, how often we already contacted them.\n", "\n", "A more detailed description of the features in our data can be found here:\n", "\n", "https://archive.ics.uci.edu/ml/datasets/Bank+Marketing\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Trying to help our business with machine learning only" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [], "source": [ "target = 'success'\n", "features = [column for column in df.columns if column != target]" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [], "source": [ "import lightgbm as lgb\n", "from sklearn.preprocessing import OrdinalEncoder" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [], "source": [ "model = lgb.LGBMClassifier()" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [], "source": [ "X, y = df[features], df[target]" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [], "source": [ "numerical_features = ['age', 'balance', 'no_contacts', 'previous', 'pdays']\n", "categorical_features = [feature for feature in features if feature not in numerical_features]" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [], "source": [ "encoder = OrdinalEncoder(dtype=int)" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [], "source": [ "X_numeric = pd.concat(\n", " [\n", " X[numerical_features],\n", " pd.DataFrame(\n", " data=encoder.fit_transform(X[categorical_features]),\n", " columns=categorical_features\n", " )\n", " ],\n", " axis=1\n", ")" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
agebalanceno_contactspreviouspdaysjobmaritaleducationdefaulthousingloancontactdaymonthpoutcome
030178710-11010000018103
13347891433971101101080
23513501133042201001500
330147640-14120112263
459010-11110102483
\n", "
" ], "text/plain": [ " age balance no_contacts previous pdays job marital education \\\n", "0 30 1787 1 0 -1 10 1 0 \n", "1 33 4789 1 4 339 7 1 1 \n", "2 35 1350 1 1 330 4 2 2 \n", "3 30 1476 4 0 -1 4 1 2 \n", "4 59 0 1 0 -1 1 1 1 \n", "\n", " default housing loan contact day month poutcome \n", "0 0 0 0 0 18 10 3 \n", "1 0 1 1 0 10 8 0 \n", "2 0 1 0 0 15 0 0 \n", "3 0 1 1 2 2 6 3 \n", "4 0 1 0 2 4 8 3 " ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_numeric.head()" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "LGBMClassifier()" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.fit(X_numeric, y)" ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAAEWCAYAAAA+bHOCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzde5xXA/7H8de7qxQTSptWjdBtppouxG40SbklWVmXXCqWXZfY/RXtZgmLWYSU+2VLIisqt3XZakiKSpNCE6txqZRIuqCpPr8/zpnxbfrOpebynTPzeT4e38ec+3l/R4/5OLfPkZnhnHPORUWNRAdwzjnndocXLuecc5Hihcs551ykeOFyzjkXKV64nHPORYoXLuecc5Hihcu5KkjS3yQ9mugczpUH+XNczu1MUg7QBNgeM7mVma0q5TYvNrP/li5d9EgaBRxmZuclOourGvyIy7n4TjWzBjGfPS5aZUFSrUTuf09FNber3LxwOVdCkpIkPSZptaSVkv4hqWY471BJMyV9K2mdpEmSGobzJgLNgRclbZJ0jaR0SV8V2H6OpOPD4VGSpkh6UtIPwKCi9h8n6yhJT4bDyZJM0mBJX0paL+mPko6Q9IGk7yWNi1l3kKQ5ksZJ2iBpmaReMfMPkvSCpO8kfSrpDwX2G5v7j8DfgLPC7744XG6wpI8lbZT0maRLY7aRLukrSf8naW34fQfHzK8nabSkz8N8b0uqF847StI74XdaLCl9j/5ju0rNC5dzJTce2AYcBnQC+gAXh/ME3AYcBLQFDgZGAZjZ+cAX/HIUd3sJ93caMAVoCEwqZv8l0Q04HDgLuAcYCRwPpAC/l9SjwLL/AxoBNwDPS9o/nDcZ+Cr8rgOAWyUdV0jux4BbgWfC794xXGYt0BfYFxgM3C2pc8w2fgUkAc2Ai4D7JO0XzrsT6AL8BtgfuAbYIakZ8DLwj3D6MOA5SY1343fkIsALl3PxTQv/r/17SdMkNQFOBq42s81mtha4GzgbwMw+NbM3zOxnM/sGuAvoUfjmS2SumU0zsx0Ef+AL3X8J3WxmP5nZ68Bm4GkzW2tmK4HZBMUwz1rgHjPLNbNngGzgFEkHA78Frg23lQU8ClwQL7eZ/RgviJm9bGb/s8CbwOvAMTGL5AI3hft/BdgEtJZUAxgCXGVmK81su5m9Y2Y/A+cBr5jZK+G+3wAWhL83V4X4+Wfn4usfeyOFpCOB2sBqSXmTawBfhvObAGMI/vjuE85bX8oMX8YMtyhq/yW0Jmb4xzjjDWLGV9rOd259TnCEdRDwnZltLDCvayG545J0EsGRXCuC77E3sCRmkW/NbFvM+JYwXyNgL4KjwYJaAGdKOjVmWm1gVnF5XLR44XKuZL4EfgYaFfiDmudWwID2ZvadpP7AuJj5BW/f3UzwxxqA8FpVwVNasesUt/+y1kySYopXc+AFYBWwv6R9YopXc2BlzLoFv+tO45LqAs8RHKVNN7NcSdMITrcWZx3wE3AosLjAvC+BiWb2h13WclWKnyp0rgTMbDXB6azRkvaVVCO8ISPvdOA+BKezNoTXWoYX2MQaoGXM+HJgL0mnSKoNXAfULcX+y9qBwFBJtSWdSXDd7hUz+xJ4B7hN0l6SOhBcg3qyiG2tAZLD03wAdQi+6zfAtvDoq09JQoWnTR8H7gpvEqkp6eiwGD4JnCrphHD6XuGNHr/e/a/vKjMvXM6V3AUEf3Q/IjgNOAVoGs67EegMbCC4QeD5AuveBlwXXjMbZmYbgMsIrg+tJDgC+4qiFbX/svYuwY0c64BbgAFm9m047xwgmeDoaypwQzHPpz0b/vxW0vvhkdpQ4N8E3+NcgqO5khpGcFpxPvAd8E+gRlhUTyO4i/EbgiOw4fjfuSrHH0B2zu1E0iCCh6W7JzqLc/H4/4k455yLFC9czjnnIsVPFTrnnIsUP+JyzjkXKf4cVzlr2LChHXbYYYmOsUc2b95M/fr1Ex1jt0U1N3j2RIhqbohu9pLkXrhw4Tozi9uuywtXOWvSpAkLFixIdIw9kpmZSXp6eqJj7Lao5gbPnghRzQ3RzV6S3JI+L2yenyp0zjkXKV64nHPORYoXLuecc5Hihcs551ykeOFyzjkXKV64nHPORYoXLuecc5Hihcs551ykeOFyzjlXrO+//54BAwbQpk0b2rZty9y5cxk1ahTNmjUjLS2NtLQ0XnnlFQDee++9/GkdO3Zk6tSpZZolUp0zJCUDL5lZagmXHx8uP6UcYznnXJV31VVXceKJJzJlyhS2bt3Kli1beO211/jzn//MsGHDdlo2NTWVBQsWUKtWLVavXk3Hjh059dRTqVWrbEpOpAqXc865irdhwwbeeustxo8fD0CdOnWoU6dOocvvvffe+cM//fQTkso0TxQLVy1Jkwhek/4hwevMhwGnAvWAd4BLrcD7WiRdH28ZSZkErynvCTQELjKz2ZJqErwS/ERgB/CImY2V1AW4C2hA8FrzQWa2urCwP+ZuJ3nEy2X25SvS/7XfxqAIZo9qbvDsiRDV3FD+2XMyTgFgxYoVNG7cmMGDB7N48WK6dOnCmDFjABg3bhxPPPEEXbt2ZfTo0ey3334AvPvuuwwZMoTPP/+ciRMnltnRFkTsfVzhqcIVQHczmyPpceAj4HEz+y5cZiLwbzN7MfZUoaT9C1kmE1hoZv8n6WTgL2Z2vKQ/Ab2As81sm6T9gY3Am8BpZvaNpLOAE8xsSIGclwCXADRq1LjL9fc8Ur6/mHLSpB6s+THRKXZfVHODZ0+EqOaG8s/evlkSANnZ2Vx22WWMHTuWdu3aMXbsWOrXr0///v1JSkpCEo8//jjffvst11577U7b+Pzzz8nIyGDMmDH5R2mbNm2iQYMGRe67Z8+eC82sa9yZZhaZD5AMfBEzfhwwDTiD4KhpCbASGBHOHw8MCIcLWyYT+G043AT4NBx+DuhdYP+pwA9AVvhZArxeVOZWrVpZVM2aNSvREfZIVHObefZEiGpus4rLvnr1amvRokX++FtvvWUnn3zyTsusWLHCUlJS4q7fs2dPmz9/fv54SXIDC6yQv6tRvKuw4CGiAfcTFKj2wCPAXrELSNqrmGV+Dn9up+jTpwI+NLO08NPezPrs+VdxzrnK71e/+hUHH3ww2dnZAMyYMYN27dqxevUvV0mmTp1Kampw39yKFSvYtm0bEBxxLVu2jOTk5DLLE8VrXM0lHW1mc4FzgbeB3wDrJDUABgAF7yLMK1JFLVPQG8ClkmbZL6cKs4HGefuXVBtoZWYfltF3c865Smns2LEMHDiQrVu30rJlS/71r38xdOhQsrKykERycjIPPfQQAG+//TYZGRnUrl2bGjVqcP/999OoUaMyyxLFwpUNXB5zfesBYD9gKfA1ML/gCmb2vaRHilomjkeBVsAHknIJbs4YJ2kAcK+kJILf3z0EN4k451yVlZaWtstLcSdOnBh32fPPP5/zzz+/3LJEqnCZWQ7QJs6s68JPweUHxQwXtkx6zPA6gutomNk24C/hJ3b5LODY3U/vnHOuLETxGpdzzrlqzAuXc865SPHC5ZxzLlK8cDnnImv79u106tSJvn37AjBo0CAOOeSQ/AavWVlZ+ctmZmaSlpZGSkoKPXr0SFRkVwYidXNGeZM0CthkZncmOotzrnhjxoyhbdu2/PDDD/nT7rjjDgYMGLDTct9//z2XXXYZr776Ks2bN2ft2rUVHdWVIT/ics5F0ldffcXLL7/MxRdfXOyyTz31FL/73e9o3rw5AAceeGB5x3PlqNofcUkaCVwIrAW+BBZK+gNBr8E6wKfA+UBN4AOCB45zJe0LLM4bL2z73mS34kU1N3j2kshr/Hr11Vdz++23s3Hjxp3mjxw5kptuuolevXqRkZFB3bp1Wb58Obm5uaSnp7Nx40auuuoqLrjggnLP6spHtT7iCju9nw2kAScDR4SznjezI8ysI/AxQcf4jQR9DU8Jlzk7XK7QouWcKx8vvfQSBx54IF26dNlp+m233cayZcuYP38+3333Hf/85z8B2LZtGwsXLuTll1/mtdde4+abb2b58uWJiO7KQKS6w5c1SVcD+5vZ9eH4XcAqgs4a/yB4zUkD4DUz+6Ok3wLXmNlpkuYCfzCzpXG2693hEyiqucGzl0T7Zkk88sgjvP7669SsWTP/pYbHHHMMI0eOzF8uKyuLZ555httuu42nnnqKn3/+mcGDBwNw++23c+SRR5Kenl6iTuWVVVSzV6vu8GX9Aa4GbooZv4vg3V4rgI7htEHA+JhlFgPpwHsl2Yd3h694Uc1t5tn3dL+nnHKKmZmtWrXKzMx27NhhV111lV177bVmZvbRRx/ZcccdZ7m5ubZ582ZLSUmxJUuWJDR3WYhq9tJ2h6/u17jeAsZLuo3get+pwEPAPsDqsInuQILXoOR5AngKuLmCszrnijFw4EC++eYbzIy0tDQefPBBANq2bcuJJ55Ihw4dqFGjBhdffHF+J3MXPdW6cJnZ+5KeITiKWssvzXf/TvDurm/Cn/vErDaJ4DTi0xUY1TlXiPT0dNLT0wGYOXNmocsNHz6c4cOHV1AqV56qdeECMLNbgFvizHqgkFW6A1PM7PvyS+Wcc64w1b5w7Q5JY4GTCO5AdM45lwBeuHaDmV2Z6AzOOVfdVevnuJxzzkWPFy7nnHOR4oXLOVepFewAn2fo0KE7PcR611130a5dOzp06ECvXr34/PPPKzqqqyBeuJxzlVpeB/hYCxYsYP369TtN69SpEwsWLOCDDz5gwIABXHPNNRUZ01UgL1zOuUorXgf47du3M3z4cG6//fadlu3Zsyd77703AEcddRRfffVVhWZ1Fafa31UoaRpwMLAXMMbMHpZ0EXAt8D3Bw8k/m9kVkhoDDwLNw9WvNrM5RW3fu8NXvKjmBs+ep6gO8OPGjaNfv340bdq00PUfe+wxTjrppDLJ4iqfal+4gCFm9p2kesB8SS8TdM7oDGwEZhIUL4AxwN1m9rak5sBrQNuCGyzQZJfr22+rgK9R9prUC/4YRU1Uc4Nnz5OZmcncuXPJzc1l48aNZGVl8e233zJlyhQeffRR7rnnHjIzM9m+fTuZmZk7rfvGG28wc+bM/GWKs2nTphItVxlFNXupcxfWxLC6fIBRBIVpMbABGAFMiJk/FBgXDq8FsmI+K4EGRW3fm+xWvKjmNvPssUaMGGHNmjWzFi1aWJMmTaxevXrWsGFDa9KkibVo0cJatGhhkuzQQw/NX+eNN96wNm3a2Jo1axKWuyJFNXtpm+xW62tcktKB44GjLXj31iJgWRGr1ACOMrO08NPMzDZVQFTnqp3bbruNr776ipycHCZPnsxxxx3H+vXr+frrr8nJySEnJ4e9996bTz/9FIBFixZx6aWX8sILL/gbjqu4al24gCRgvZltkdQGOAqoD/SQtJ+kWsAZMcu/DuR3z5CUVqFpnXOFGj58OJs2beLMM88kLS2Nfv36JTqSKyfV/RrXq8AfJX0MZAPzCE7/3Qq8B3xHcAS2IVx+KHCfpA8IfndvAX+s6NDOVTexHeBjbdr0ywmP//73vxWYyCVStS5cZvYzQdPcnUhaYMHdhbWAqcC0cPl1wFkVm9I551ys6n6qsDCjJGUBSwnehjwtwXmcc86FqvURV2HMbFiiMzjnnIvPj7icc85Fihcu55xzkeKFy7ly8OWXX9KzZ0/atWtHSkoKY8aMyZ83duxY2rRpQ0pKSn4j2EmTJpGWlsbFF19MWloaNWrUICsrK1HxnavU/BpXESQ1BM41s/vD8XRgmJn1LXJFV+3VqlWL0aNH07lzZzZu3EiXLl3o3bs3a9asYfr06SxevJi6deuydu1aAAYOHMjAgQPJzMzkgAMOoH///qSl+WOCzsXjhatoDYHLgPv3dAPeZLfiJTp3TsYpNG3aNL8J7D777EPbtm1ZuXIljzzyCCNGjKBu3boAcTs8PP3005x99tkVmtm5KKkypwolJUtaJmm8pOWSJkk6XtIcSZ9IOlLS/pKmSfpA0jxJHcJ1R0l6XFKmpM8kDQ03mwEcKilL0h3htAaSpoT7miRJCfnCLjJycnJYtGgR3bp1Y/ny5cyePZtu3brRo0cP5s+fv8vyzzzzDOecc04CkjoXDVXtiOsw4ExgCDAfOBfoDvQD/gZ8CSwys/6SjgOeAPLOx7QBegL7ANmSHiBouJtqZmmQf6qwE5ACrALmAL8F3o4N4d3hEyvRuWO7Xv/4449cddVVXHzxxbz//vts2LCBJUuWkJGRwbJly+jXrx9PPfUUef//s3DhQsyMdevWRa7rd7XtVJ5AUc3u3eF/6eKeDHwSM/4EMDAcbknQzX0R0DJmmS+BfQk6xI+Mmf4x8Otwm0tjpqcDb8SMPwCcV1Qu7w5f8SpL7q1bt1qfPn1s9OjR+dNOOOEEmzlzZv54y5Ytbe3atfnjAwYMsFtuuaVCc5aVyvJ7311RzW0W3ezeHX5nP8cM74gZ30HxR5ex624vYvmSLueqMTPjoosuom3btvzlL3/Jn96/f39mzZoFwPLly9m6dSuNGjUCYMeOHWRmZvr1LeeKUdUKV3FmAwMh/7TfOjP7oYjlNxKcOnRut8yZM4eJEycyc+ZM0tLSSEtL45VXXmHIkCF89tlnpKamcvbZZzNhwoT804RvvfUWjRs3pmXLlglO71zlVt2OFkYBj4fd3bcAFxa1sJl9G97csRT4DxC9W+xcQnTv3j3vdPIunnzyybjT09PTuf/+Pb6B1blqo8oULjPLAVJjxgcVMq9/nHVHFRiP3c65BRbPjJl3xR4Hds45t0eq26lC55xzEeeFyznnXKR44XLOORcpXricK0JhzXL//ve/06FDB9LS0ujTpw+rVq0C4I477si/izA1NZWaNWvy3XffJfIrOFfleOEqQNI7RcxLl/RSReZxiZXXLPejjz5i3rx53HfffXz00UcMHz6cDz74gKysLPr27ctNN90EwPDhw8nKyiIrK4vbbruNHj16sP/++yf4WzhXtXjhKsDMfpPoDK7yaNq0KZ07dwZ2bpa777775i+zefNm4rWsfPrpp73noHPloMrcDl9WJG0ieOj4duAkwIB/mNkz4SL7SnqZoC/iLOAyM9tR2Pa8O3zFK6vcORmn7Dwe0ywXYOTIkTzxxBMkJSXld8PIs2XLFl599VXGjRtX6hzOuZ2psIckq6uwcF0I/BE4EWhE0LC3G9AaeBVoB3weDj9kZlMKbCO2yW6X6+95pMLyl6Um9WDNj4lOsfvKKnf7Zkn5w3nNcs877zyOPfbYnZabNGkSW7duZfDgwfnTZs6cyX//+19uvfXW3drnpk2baNCgQemCJ0hUs0c1N0Q3e0ly9+zZc6GZdY03zwtXAWHhegRYYmaPh9MmAs8CPwA3mdmx4fQhQAczu7qw7TVveZjV+P2YwmZXav/Xfhujl0TvoLyscucdceXm5tK3b19OOOGEnfoO5vniiy84+eSTWbp0af60008/nTPPPJNzzy34/HrRMjMzSU9PL1XuRIlq9qjmhuhmL0luSYUWruj9VUq8gpW+yMpfr3ZNsguccoqKzMxMcgamJzrGbivL3IU1y/3kk084/PDDAZg+fTpt2rTJn7dhwwbefPPNQls7OedKxwtXfLOBSyVNAPYHjgWGE7yz60hJhxCcKjwLeDhhKV25y2uW2759e9LSgle33XrrrTz22GNkZ2dTo0YNWrRowYMPPpi/ztSpU+nTpw/169dPVGznqjQvXLsyYCpwNLA4HL/GzL6W1Ibgetc4frk5Y2qigrryV1iz3JNPPrnQdQYNGsSgQYPKMZVz1ZsXrhiSDgC+C19iNjz85DOzTIKjL+eccwniz3GFJB0EzAXuTHQW55xzhfMjrpCZrQJaJTqHc865ovkRl3POuUjxwuWccy5SqkXhknS1pL33cN00SYXfQuaqpN3tCp9n/vz51KpViylTpsTbrHOuDFSLwgVcDexR4QLSAC9c1czudoUH2L59O9deey19+vRJYHLnqr5yK1ySkiV9LOkRSR9Kel1SvfAIZp6kDyRNlbRfEds4TNJ/JS2W9L6kQxW4Q9JSSUsknRUumy4pU9IUScskTQqXHQocBMySNCtc9gFJC8JcN8bs7whJ74T7e09SEnATcJakLElnSeoRDmdJWiRpn/L6HbrE2ZOu8GPHjuWMM87gwAMPrPC8zlUn5X1X4eHAOWb2B0n/Bs4ArgGuNLM3Jd0E3EBwRBTPJCDDzKZK2oug0P6O4CioI2EDXElvhct3AlKAVcAc4Ldmdq+kvwA9zWxduNxIM/tOUk1ghqQOwDLgGeAsM5svaV9gC3A90NXMrgCQ9CJwuZnNkdQA+KmoX4B3h694pc29J13hV65cydSpU5k1axbz58/f8/DOuWKVd+FaYWZZ4fBC4FCgoZm9GU6bQNC8dhfhkUwzM5sKYGY/hdO7A0+b2XZgjaQ3gSMIGuC+Z2ZfhctlAcnA23E2//uwg3stoClBt3cDVpvZ/HB/P4TbKbjuHOAuSZOA5/P2VyB7bHd4rm+/rdBfUGXWpF5QBKKmtLkzMzPzh/O6wl988cW8//77APTu3ZvevXszadIkhg0bxuDBgxk1ahRnnXUWb731Fl9//TUffvghjRo12u19b9q0aaf9R0lUs0c1N0Q3e6lzm1m5fAiKxtKY8WHA3cAXMdMOBd4vZP19gK/iTL8bGBIzPhHoB6QDL8VMHwcMCodzgEbh8CHAp8B+4fh4YBDQHpgTZ3+DgHEFprUHriXoV9imqN9Dq1atLKpmzZqV6Ah7pKxyb9261fr06WOjR4+OO//zzz+3lJQUMzNLTk62Fi1aWIsWLax+/frWuHFjmzp16m7vM6q/c7PoZo9qbrPoZi9JbmCBFfJ3taJvztgArJd0TDh+PvBmvAXNbCPwlaT+AJLqhncGzia45lRTUmOCFkzvFbPfjQSFEGBfYDOwQVITgpdFAmQDTSUdEe5vH0m1CqyLpEPNbImZ/ZOgb+EvbcFdlWFFdIXPE9sVfsWKFeTk5JCTk8OAAQO4//776d+/f4Xndq46SETnjAuBB8Mi9BkwuIhlzwceCq+F5QJnUnQD3MI8DLwqaZWZ9ZS0iOCa1pcEp/4ws63hjR5jJdUDfgSOJ2ikOyI89Xgb0F1ST2AH8CHwnz36LbhKbU+6wjvnKka5FS4zywFSY8ZjewAeVcJtfAIcF2dWYQ1wM2PGr4gZHguMjRkfVMj+5heS7YiY4WeKDe4ib0+6wucZP358OSRyzuWpLs9xOeecqyIqRZNdSfcBvy0weYyZ/SsReZxzzlVelaJwmdnlic7gnHMuGkp0qjDsWFE3HE6XNFRSw/KN5pxzzu2qpNe4ngO2SzqM4A69g4Gnyi2Vc2WosIa53333Hb179+bwww+nd+/erF+/HggeQE5KSiItLY20tLSd+hE65xKvpIVrh5ltA04HxprZcIKOE5EUHjW+lOgcrmIU1jA3IyODXr168cknn9CrVy8yMjLy1znmmGPIysoiKyuL66+/PoHpnXMFlbRw5Uo6h+AZrLw/+LXLJ5JzZauwhrnTp0/nwgsvBODCCy9k2rRpiYzpnCuhkt6cMRj4I3CLma2QdAhBq6VKR1Iy8CpBb8TOBA8JX0DQYeMegsa5b8csfyQwBtiL4KHjwWaWHTbuHWphr0VJbwOXAw3D5SF4APrYsMtHXN5kt+LF5i6qYe6aNWto2jQ4cfCrX/2KNWvW5C83d+5cOnbsyEEHHcSdd95JSkpKxX0B51yRFO8hy7gLBt0kmptZdvlGKp2wcK0AulvQwf1xgg4dlxI8zPwpwUPEe5tZ37wu8Ga2TdLxwJ/M7AxJFwKdzOxqSa2Ap8ysa9gdPsNiusOHp1FjM8Q22e1y/T2PVMh3L2tN6sGaHxOdYvfF5m7fLCl/el7D3PPOO49jjz2Wvn378tJLv5wxPvXUU3nxxRfZvHkzNWrUoF69esybN49x48bx5JNPVkj2TZs20aBBgwrZV1mLavao5oboZi9J7p49ey40s67x5pWocEk6FbgTqGNmh0hKA24ys357kLlchYXrLTNrHo4fBwwF9jezY8Np/YBLwsJ1MHAvwStYDKhtZm3CllQfAG2Bmwka/o6TNILgWl+h3eFjNW95mNX4/ZiiFqm0/q/9NkYvqRRPTOyW2Nx5R1y5ubn07duXE044Ib/3YOvWrcnMzKRp06asXr2a9PR0srN3/f+y5ORkFixYsEfd3ndXZmYm6enp5b6f8hDV7FHNDdHNXpLckgotXCX9qzQKOJKwpZKZZUlqWeKUFa9gNU6Ku1TgZmCWmZ0eFr1MADPbIukN4DTg90CXcHqGpJcJ3oo8R9IJZrassI3Xq12T7AKnq6IiMzOTnIHpiY6x2wrmLqxhbr9+/ZgwYQIjRoxgwoQJnHbaaQB8/fXXNGnSBEm899577NixgwMOOKCiv4ZzrhAlLVy5ZrahwLupdpRDnrLSXNLRZjYXOBf4L3Bp2Nn9f8A5McsmASvD4UEFtvMo8CIw28zWwy/d4YElYSf5NgQNe10lVVjD3BEjRvD73/+exx57jBYtWvDvf/8bgClTpvDAAw9Qq1Yt6tWrx+TJk+O9l805lyAlLVwfSjoXqCnpcIJTb++UX6xSywYuD69vfUSQdyHwsqQtBK9GyXtVye3ABEnXATvdiWBmCyX9AMS2nrrau8NHS2ENcwFmzJixy7QrrriCK664Is7SzrnKoKSF60pgJPAzwYPHrwH/KK9QZWCbmZ1XYNqrxHl3VnhU1ipm0nV5A5IOInhk4PWY5a8s26jOOed2R7GFS1JN4GUz60lQvKoFSRcAtwB/MbPKfFrUOeeqlWILl5ltl7RDUpKZbaiIUKVR8D1gpdjOE8ATpQ7knHOuTJX0VOEmgpsR3iB47T0AZja0XFI555xzhShp4Xo+/DjnnHMJVaJehWY2Id6nvMO5xBkzZgyDBw8mJSWFe+65B4BRo0bRrFmz/K7pr7zySoJTOueqoxIdcUlawa4P9WJmlfkh5EJJegU418y+T3SWymjp0qU88sgjPPDAAxx//PGceOKJ9O3bF4A///nPDBs2LMEJnXPVWUlPFca23dgLOBPYv+zj7B8v7WEAACAASURBVD5JNc1s++6sY2Ynl1eequDjjz+mW7du7LXXXtSqVYsePXrw/PN+ptg5VzmUqHCZ2bcFJt0jaSFQri8qKqLT+0cEjXJ7A7dL+g64EagL/I+gm3134CIzOzPcVjowLOxPmAN0NbN1kv4CDAl3+aiZ3RPu9yUzSw3XHQY0MLNRkoYSdMrfBnxkZmcX9R2i1h0+J+MUUlNTGTlyJP369WPLli288sordO3alQMOOIBx48bxxBNP0LVrV0aPHs1+++2X6MjOuWqmpE12O8eM1iA4AvuTmXUsr2DhfpPZtdP7R8AVwP1mdrukRgQ3jpxkZpslXUtQwG4l6ArfNpz+ADDHzJ7MK1xAC2A8cBQg4F3gPGA9hReuVcAhZvazpIbxTjdGuTt8Xjf1l19+malTp1K/fn2Sk5OpXbs25557LklJSUji8ccf59tvv+Xaa69NcOJdRbVjNnj2RIhqbohu9tJ2h8fMiv0As2I+bwAPA61Lsm5pPkAy8EXM+HHANCAHaBFO6wusA7LCz0fAY+G8h4GzCY4svwD2CafnAI2Aqwi63Odt/2aC9lDJwNKY6cOAUeHwq8AUggLXoLjv0KpVK4uqWbNmmZnZX//6V7vvvvt2mrdixQpLSUlJQKri5eWOIs9e8aKa2yy62UuSG1hghfxdLek1rovM7LPYCeHLJCtCwUPCvPG858kEvGFm57CryQRHZ98R/BIKfeFjAdvY+Y7LvWKGTyF4KeWpwEhJ7a3A+7iqgrVr1wLwxRdf8PzzzzNv3jxWr16d/+LFqVOnkppa6ue8nXNut5W0cE0huMZUcFqXso0TV8FO728DnWLmzwPuk3SYmX0qqT7QzMyWA28CjwN/IChiBc0GxkvKICiApwPnA2uAAyUdQPDwdV/gVUk1gIPNbFb4RuSzgQZAlbs78YwzzuDLL78kKSmJ++67j4YNG3LllVeSlZWFJJKTk3nooYcSHdM5Vw0VWbgktQFSgCRJv4uZtS87H4WUp4Kd3h8gaPoLgJl9I2kQ8LSkuuHk64DlFrSreongdSUXFtywmb0vaTzwXjjpUTNbBCDppnD6Sn55bUlN4ElJSQSF7l6rorfUz549e5eXvU2cODFxgZxzLlTcEVdrgqONhgSnxvJsJDiKqQjxOr0nx46Y2UzgiHgrm9kVBKcLY6clxwzfBdwVZ717Cd6MXFD3koR2zjlXPoosXGY2HZgec6rOOeecS6iSXuNaJOlygtOG+acIzWxI4auUnpVRp3fnnHNVR4l6FQITgV8BJxDc8PBrgtOFzjnnXIUqaeE6zMz+Dmy2oLnuKUC38ovlnHPOxVfSwpUb/vxeUiqQBBxYPpFcZRCvO/zf//53OnToQFpaGn369GHVqlUJTumcq45KWrgelrQf8HfgBYLb0m8vt1QFSBokaVwZb7O/pHYx4zdJOr4s9xFVsd3hFy9ezEsvvcSnn37K8OHD+eCDD8jKyqJv377cdNNNiY7qnKuGStpk99Fw8E0gkq8yiaM/8BJBEcbMyqVhcJSa7OZknAIU3h3+mmuuyV928+bNSEpUVOdcNVaiIy5JTSQ9Juk/4Xg7SReVVQhJ50l6T1KWpIck1ZQ0WNJySe8Bv41ZdrykATHjm2KGr5W0RNLisBsGkv4gaX447TlJe0v6DdAPuCPc56Gx25XUS9KicFuP5z3YLClH0o2S3g/ntSmr30FlkpqayuzZs9mwYUN+d/gvv/wSgJEjR3LwwQczadIkP+JyziVESbvD/wf4FzDSzDpKqgUsMrP2pQ4gtSU47fg7M8uVdD9Bl/abCVpKbSBo7rvIzK4IO128ZGZTwvU3mVkDSScRnMo83sy2SNrfzL6TdICFr2WR9A9gjZmNjbOd8QRHYC8BnwC9zGy5pCeA9y143UkOMDpc/zKgs5ldHOc7RbI7fF5neIjfHf6KK355jnvSpEls3bqVwYMHJyJqkaLaMRs8eyJENTdEN3tFdYefH/5cFDMtqyTrlmDbVwCr+KW7e3b484mYZYYC48Lh8cCAmHmbwp+jgT/E2X4Pgp6ESwhekfJgIdsZDwwAOgJvxUzvBTxvv3SVbxYOdwP+W9z3q6rd4T///HPvDl8OPHvFi2pus+hmL213+JLenLE5bDhrAJKOIjgSKgsCJphZWvhpDYwqYvn8zu1h09s6xWx/PHCFBUeHN1L6Hos/hz+3U/IHuCOnYHf4c889l08++SR//vTp02nTpkqeKXXOVXIl/cP7F4K7CQ+VNAdoTHB0UhZmELSVutvM1kraH1gEjAmL5Q/AmcDicPkcglOI/ya4TlU7nP4GcL2kSRZzqhDYB1gtqTYwkKBpLgQPUO8TJ082kJzXbZ6gW/ybZfRdIyNed/iLLrqI7OxsatSoQYsWLXjwwQcTHdM5Vw0V1x2+uZl9YUEX9R4ETXcFZJtZblHrlpSZfSTpOuD18AgqF7ic4KhrLsErQ7JiVnmEoNAtJnip4+ZwO69KSgMWSNoKvAL8jeC617vAN+HPvGI1GXhE0lBiirCZ/SRpMPBseC1vPlDt/kLH6w7/3HPPJS6Qc86FijvimsYv7+F6xszOKI8QZvYM8EyByfMIbggpuOwa4KiYSdfGzMsAMgos/wDBq1AKbmcO0C5m0qCYeTPY+Z1fedOTY4YXAOm7fhvnnHPlqbhrXLEP6lSV57ecc85FWHGFywoZds455xKiuFOFHSX9QHDkVS8cJhw3M9u3XNM555xzBRR5xGVmNc1sXzPbx8xqhcN54160qpC7776blJQUUlNTOeecc/jpp5/IyMjgkEMOIS0tjbS0NLKysorfkHPOlbOSPsdVpUhKlnRuKdYfJOmgssyUSCtXruTee+9lwYIFLF26lO3btzN58mQA7rjjDrKyssjKyiItLS3BSZ1zrpoWLiAZ2OPCRXAHYpUpXADbtm3jxx9/ZNu2bWzZsoWDDqpSX885V4VEsvODpAuAYQQ3jHxA8KzW40Ajgue1BpvZF2H/wR+ArgRvcL7Ggt6EGUBbSVnABGAqwVue64e7uMLM3gn3dS1wHrAD+A+wINzeJEk/Akeb2Y+FZa3s3eFzMk6hWbNmDBs2jObNm1OvXj369OlDnz59uOuuuxg5ciQ33XQTvXr1IiMjg7p16yY6snOumitRk93KRFIKQaH5jZmtCzttTACmmNkESUOAfmbWPyxc9YGzgDbAC2Z2mKR0YJiZ9Q23uTewI3z4+HDgaTPrWkTj3sxw/QWFZIxMk932zZLYuHEjN9xwA9dffz0NGjRg1KhR9OjRg9atW3PwwQeTm5vL6NGjOeigg7jwwgsTHblYUW08Cp49EaKaG6KbvbRNdqN4xHUc8KyZrQMIC8nRwO/C+RPZ+SWX08xsB/CRpCaFbLM2MC7svLEdaBVOPx74l5ltydtXSQKa2cPAwwDNWx5mo5dU3l9zzsB0nn32WTp16kT//v0BWLVqFfPmzaN58+b5nTPq1KnDnXfeuVMnjcqqYMePKPHsFS+quSG62Uubu/L+RS07P8cMF/bmwz8Dawg6w9cAfiqrnderXZPs8AWNlVXz5s2ZN28eW7ZsoV69esyYMYOuXbvy7bffAsEbBKZNm0ZqamqCkzrnXDRvzpgJnBk24CU8VfgOcHY4fyDBa0yKUrDBbhKwOjwyOx+oGU5/AxgcnkrM21e89SOtW7duDBgwgM6dO9O+fXt27NjBJZdcwi233EL79u1p374969at47rrrkt0VOeci94Rl5l9KOkW4E1J2wk6yV8J/EvScMKbM4rZzAfA9rBR73jgfuC58KaPkjTuHQ88WJKbM6Lixhtv5MYbb9xp2l133RXJ0xDOuaotcoULwMwmENyQEeu4OMsNKjDeIPyZG2f5DjHDxTXufQ7wVunOOZcAUTxV6JxzrhrzwuWccy5SvHA555yLFC9czjnnIsULVzWUnZ2d3/E9LS2Nfffdl3vuuQeAsWPH0qZNG1JSUnjwwQcTnNQ553YVybsKy1rY6f1eMxsQ3v5+kJm9Usw66cS0jYqS1q1b57+iZPv27TRr1ozTTz+dWbNmMX36dBYvXkzdunWZOnVqgpM659yuqn3hklTLzFYBA8JJaQRNdIssXCVVmZrs5sTp4DFjxgwOPfRQWrRowfDhwxkxYkR+I9399tuvoiM651yxInuqMHyn1jJJ4yUtlzRJ0vGS5kj6RNKR4WeupEWS3pHUOlx3kKQXJM0EZoTbWiqpDnATcJakLElnFbaNqmLy5Mmcc845ACxfvpzZs2fTrVs3evTowbJlyxKczjnndhX1I67DgDOBIcB8gndsdQf6EXS4uAA4xsy2SToeuBU4I1y3M9AhbNKbDGBmWyVdD3Q1sysAJO1bxDbiKtAdnuvbbyu7b1wKmZmZO43n5uby3HPP0bdvXzIzM9mwYQNLliwhIyODZcuWccMNN9C6dWukwlo8Vk6bNm3a5btGhWeveFHNDdHNXurcZhbJD8HLID+JGX8CGBgOtwSygIMJXoGyFFgCLAvnDyLo+h67raUx88bFzCtsG+nAS8XlbNWqlVVW06ZNs969e+ePn3DCCTZz5sz88YMOOsjWrl2biGilMmvWrERH2GOeveJFNbdZdLOXJDewwAr5uxrZU4Wh2M7vO2LGdxAcTd4MzDKzVOBUYK+Y5TeXcB9FbSPSnn766fzThAD9+/dn1qxZQHDaMDc3l0aNGiUqnnPOxRX1wlWcJGBlODyohOvE6xy/u9uo9DZv3swbb7zB7373u/xpQ4YM4bPPPiM1NZWzzz6bESNGRO40oXOu6qvqhet24DZJiyj59bxZQLu8mzP2cBuVXv369fn2229JSkrKn1anTh2efPJJli5dyvvvv0/nzp0TmNA55+KL7B9iM8sBUmPGBxUyr1XMateF88cTvJpkl+UteMvxEQV2F28bmUDmnuZ3zjm3Z6r6EZdzzrkqxguXc865SPHC5ZxzLlK8cFVh33//PQMGDKBNmza0bduWuXPn8uyzz5KSkkKNGjVYsGBBoiM659xui0zhymvLVI7bf6e8tp0oV111FSeeeCLLli1j8eLFtG3bltTUVJ5//nmOPfbYRMdzzrk9Etm7Csuamf0m0RnK0oYNG3jrrbcYP348ENzqXqdOHRo2bJjYYM45V0pRK1w1JT0C/IbgoeDTgNbAg8DewP+AIWa2XlImwWtHFkhqRNA+JFlSCvAvoA7BEecZZvaJpE1m1iB8XckoYB3BLfILgfPMzCSdDNxF0HVjDtDSinmtSSK6w+dknMKKFSto3LgxgwcPZvHixXTp0oUxY8ZQv379Cs3inHNlTUFLqMovbIT7KUED3CxJ/wZeAK4BrjSzNyXdBOxrZlcXUbjGAvPMbFLYDb6mmf1YoHBNB1KAVQQFajiwAPgEONbMVkh6GtgnXuEq0GS3y/X3PFJ+v5g42jdLIjs7m8suu4yxY8fSrl07xo4dS/369RkyZAgAV199NX/6059o3brwZvebNm2iQYMGFRW7zEQ1N3j2RIhqbohu9pLk7tmz50Iz6xpvXtSOuFaYWVY4vBA4FGhoZm+G0yYAzxazjbnASEm/Bp43s0/iLPOemX0FICmLoAnvJuAzM1sRLvM0YXEqyMweBh4GaN7yMBu9pGJ/zTkD02nTpg233XYbl112GQA1a9YkIyOD9PR0ABo2bEiXLl3o2jXuvwsg6Caft3yURDU3ePZEiGpuiG720uaOWuGKbaq7HSjqgs02frn5JL8xrpk9Jeld4BTgFUmXmtnMYvazx7+nerVrkh3nBY7l7Ve/+hUHH3ww2dnZtG7dmhkzZtCuXbsKz+Gcc2UtMncVFmIDsF7SMeH4+UDe0VcO0CUcznu7MZJaEhw53UtwSrBDCfeVDbTMe3cXcNYep64gY8eOZeDAgXTo0IGsrCz+9re/MXXqVH79618zd+5cTjnlFE444YREx3TOud0StSOueC4EHpS0N/AZMDicfifw7/B6U+zdEb8HzpeUC3xN8GLIYoXXwS4DXpW0meDFlZVaWlraLs9qnX766Zx++ukJSuScc6UXmcIVp6nunTGzj4qz/DJ2PprKa46bAWTEWb5B+DOTmOa5Fr4JOTTLzNooeNfHfQQ3bDjnnKtAUT9VWNH+EN6s8SHBe7oeSnAe55yrdiJzxFUZmNndwN2JzuGcc9WZH3E555yLFC9czjnnIsULVxWRnJxM+/btSUtLy3+o2DvBO+eqoipzjSts1bTVzKpcl/eSmjVrFo0aNcofz+sEf+mllyYwlXPOla0qU7iAdIK2TNW2cBXUtm3bREdwzrkyl/DCFXaieJWg92BnglvNLwCOJniIuBbBw75/MrOfJeUQNNpdJ6lruMwg4I/AdknnAVcCywm6xrcMd/UnM3tH0l+AIeG0R83snpgM8wg6z88n6CB/I3AgMNDM3pNUHxhL8DxZbWCUmU0v6vuVd3f4nLCdlCT69OmDJC699FIuuSRuG0XnnIu8hHeHD4vGCqC7mc2R9DhBB4xLgV5mtlzSE8D7YZHJoUDhMrN0SaOATXkPJkt6BpgbrlMTaAAcBowneGBZwLvAecB6gs7znQgK53xgMXAR0A8YbGb9Jd0KfGRmT0pqCLwHdDKzzQW+U4V1h2/fLAmAb775hsaNG7N+/XqGDRvG0KFD6dixI1CyTvDxVOXO05WVZ694Uc0N0c1eVbrDf2lmc8LhJ4G/E3SCXx5OmwBcDtyzG9s8juDIDTPbDmyQ1B2YmldoJD0PHEPwepQVZrYknP4hMCN8B9cSgu7wAH2AfpKGheN7Ac2Bj2N3HNsdvnXr1nblwNN2I3bpLV68mNzc3N3qBB9Pde08nUieveJFNTdEN3tpc1eWuwoLHvZ9X8Sycbu+l4HYjvA7YsZ38EuBF8GLJ9PCT3Mz26loJcLmzZvZuHFj/vDrr79OampqMWs551w0VZbC1VzS0eHwuQQ9AJMlHRZOK6zr+xkx29gI7BMzPgP4E4CkmpKSgNlAf0l7h9erTg+nldRrwJVhr0IkddqNdcvNmjVr6N69Ox07duTII4/klFNO4cQTT/RO8M65KqmynCrMBi4Pr299BAwluFHiWUl5N2c8GC57I/CYpJuJaYYLvAhMkXQawc0ZVwEPS7qI4J1afzKzuZLGE1ybguDmjEUxryopzs0Epys/kFSD4NrcLm9ArmgtW7Zk8eLFu0z3TvDOuaqoshSubWZ2XoFpMwhultiJmc0GWsWZvpxd3621y8UlM7sLuKvAtBx27jw/KN48M/uR4KYR55xzCVJZThU655xzJZLwI66CRzvOOedcUfyIyznnXKQk/IjLldxPP/3Esccey88//8y2bdsYMGAAN954I8ccc0z+7fBr167lyCOPZNq0aQlO65xz5aNaFi5Jm8wsco+b161bl5kzZ9KgQQNyc3Pp3r07J510ErNn/3JH/xlnnMFpp1XsA8/OOVeR/FRhhEjKb5OSm5tLbm4u4SNlAPzwww/MnDmT/v37Jyqic86Vu2p5xJUnfJD4duAkgu4d/zCzZyQ1AKYD+xE0073OzKaHz3v9B3iboBnvSuC08Db5uMqiyW5eI12A7du306VLFz799FMuv/xyunXrlj9v2rRp9OrVi3333bdU+3POucqsuh9x/Q5IAzoCxwN3SGoK/AScbmadgZ7AaP1yaHM4cJ+ZpRC0pjpj182Wn5o1a5KVlcVXX33Fe++9x9KlS/PnPf3005xzzjkVGcc55ypcwrvDJ0LeNS5JdwNLzOzxcPpE4FmCo6q7gWMJehW2Bg4h6I34hpkdHi5/LVDbzP5RYPtl2h0+rwN8QRMmTGCvvfbirLPOYsOGDVxwwQU8++yz1KlTp1T7y1OVO09XVp694kU1N0Q3e2m7w2Nm1e5D8PoTCIrTkJjpEwleYzIIeIagKEHQHzE5/CyNWX4YwTu5Ct1Xq1atrKysXbvW1q9fb2ZmW7Zsse7du9uLL75oZmYPPPCAXXDBBWW2LzOzWbNmlen2KkpUc5t59kSIam6z6GYvSW5ggRXyd7W6nyqcDZwVNuFtTHCE9R6QBKw1s1xJPYEWiQyZZ/Xq1fTs2ZMOHTpwxBFH0Lt3b/r2DVolTp482U8TOueqhWp9cwYwleBNy4sJbs64xsy+ljQJeDF8F9cCYFkCM+br0KEDixYtijsvMzOzYsM451yCVMvCZeEzXOHh6PDwEzt/HUFBiye2Ge+d5ZXROedcfNX9VKFzzrmI8cLlnHMuUrxwOeecixQvXM455yLFC1eC3H333aSkpJCamso555zDTz/9lOhIzjkXCVWmcEkaJWlYEfMbS3pX0iJJx+zB9gdJGhcO95fUbk+zrly5knvvvZcFCxawdOlStm/fzuTJk/d0c845V61UmcJVAr0I2jt1MrPZxS5dtP7AHhcugG3btvHjjz+ybds2tmzZwkEHHVTKSM45Vz1EunBJGilpuaS3CfoJIulQSa9KWihptqQ2ktIIusCfJilLUj1JD0haIOlDSTfGbDNHUqNwuKukzAL7/A1BW6g7wm0dWlTGvO7weR+AZs2aMWzYMJo3b07Tpk1JSkqiT58+Zfibcc65qiuyTXYldQHGA90IHqR+H3iQ4BUlfzSzTyR1A24zs+MkDQK6mtkV4fr7m9l3kmoCM4ChZvaBpJxwuXWSugJ3mll67PqSxgMvmdmUQrIV2mS3fbMkNm7cyA033MD1119PgwYNGDVqFD169KB3795l/WsqlarcwLOy8uwVL6q5IbrZS9tkN8qdM44BpprZFgBJLxB0b/8N8GzMCxbrFrL+78MCUwtoSnDq74OyCGZmDwMPAzRveZiNXvLLrzlnYDrPPvssnTp1yn/h46pVq5g3bx7p6ellsfsyk5mZWekylURUc4NnT4So5oboZi9t7igXrnhqAN+bWVpRC0k6hKCz+xFmtj48gtornL2NX06h7hVn9d1Sr3ZNsmNeBAnQvHlz5s2bx5YtW6hXrx4zZsyga9f43fudc87tLMrXuN4C+ofXq/YBTgW2ACsknQnBG44ldYyz7r7AZmCDpCYEpxfz5ABdwuHCXhK5EdhnT4N369aNAQMG0LlzZ9q3b8+OHTu45JJL9nRzzjlXrUS2cJnZ+wTvzFpM8OLH+eGsgcBFkhYDHwKnxVl3MbCIoOv7U8CcmNk3AmMkLQC2F7L7ycDw8Nb6Im/OKMyNN97IsmXLWLp0KRMnTqRu3cLOaDrnnIsV6VOFZnYLcEucWSfGWXY8wc0ceeODCtnmbKBVUeub2RxKeTu8c865PRPZIy7nnHPVkxcu55xzkeKFyznnXKR44XLOORcpXricc85Fihcu55xzkeKFyznnXKR44XLOORcpke0OHxWSNgLZic6xhxoB6xIdYg9ENTd49kSIam6IbvaS5G5hZo3jzYh054yIyC6sNX9lJ2lBFLNHNTd49kSIam6IbvbS5vZThc455yLFC5dzzrlI8cJV/h5OdIBSiGr2qOYGz54IUc0N0c1eqtx+c4ZzzrlI8SMu55xzkeKFyznnXKR44SpHkk6UlC3pU0kjEp0nlqTHJa2VtDRm2v6S3pD0Sfhzv3C6JN0bfo8PJHVOXHKQdLCkWZI+kvShpKuikF/SXpLek7Q4zH1jOP0QSe+G+Z6RVCecXjcc/zScn5yI3LEk1Qzf/P1SOB6J7JJyJC2RlBW+3bzS/3sJszSUNEXSMkkfSzo6Irlbh7/rvM8Pkq4us+xm5p9y+AA1gf8BLYE6wGKgXaJzxeQ7FugMLI2ZdjswIhweAfwzHD4Z+A8g4Cjg3QRnbwp0Dof3AZYTvJG6UucP998gHK4NvBvm+Tdwdjj9QeBP4fBlwIPh8NnAM5Xg381fgKeAl8LxSGQHcoBGBaZV6n8vYZYJwMXhcB2gYRRyF/gONYGvgRZllT3hX6qqfoCjgddixv8K/DXRuQpkTC5QuLKBpuFwU4KHpwEeAs6Jt1xl+ADTgd5Ryg/sDbwPdCPoIFCr4L8b4DXg6HC4VricEpj518AM4DjgpfCPTFSyxytclfrfC5AErCj4e6vsueN8jz7AnLLM7qcKy08z4MuY8a/CaZVZEzNbHQ5/DTQJhyvtdwlPQXUiOHqp9PnDU21ZwFrgDYKj8u/NbFucbPm5w/kbgAMqNvFO7gGuAXaE4wcQnewGvC5poaRLwmmV/d/LIcA3wL/C07OPSqpP5c9d0NnA0+FwmWT3wuXisuB/eyr1sxKSGgDPAVeb2Q+x8yprfjPbbmZpBEcvRwJtEhypRCT1Bdaa2cJEZ9lD3c2sM3AScLmkY2NnVtJ/L7UITuc/YGadgM0Ep9fyVdLc+cJrnv2AZwvOK012L1zlZyVwcMz4r8NpldkaSU0Bwp9rw+mV7rtIqk1QtCaZ2fPh5MjkN7PvgVkEp9caSsrrGxqbLT93OD8J+LaCo+b5LdBPUg4wmeB04RiikR0zWxn+XAtMJfifhsr+7+Ur4Cszezccn0JQyCp77lgnAe+b2ZpwvEyye+EqP/OBw8O7ruoQHC6/kOBMxXkBuDAcvpDg2lHe9AvCO3+OAjbEHO5XOEkCHgM+NrO7YmZV6vySGktqGA7XI7gu9zFBARsQLlYwd973GQDMDP8vtcKZ2V/N7Nf2/+3dW4hVVRzH8e+PjKbMMUfnpYcYlQhMZGIGQRlEKiJ8yTAYwh4GoijIl7CnqCSliwURFQSBGD5YkCYGkdiMk6FJWep4IUtqIsigCETEbrp6WP/T2R7mos4Zz2zn94EN66y111n/PZw5a/Zl/iulNvJnuS+ltJISxC5pqqRplTL5nssRJvjnJaX0K/CzpNui6i7gGBM87hoPUr1MCPWKvdE37q7mjfykzHfk+xhPNzqemtg2AyeBf8h/2T1MvgfRC3wPfAq0xL4C3orjOAx0Njj2LvIlhgHgYGzLJnr8wALgQMR9BHg26ucAXwInyJdUrov6pnh9ItrnNPpzE3EtpfpU4YSPPWI8FNvR/Fi2AQAAA7pJREFUyu/iRP+8RCztwP74zGwDZpQh7ohnKvkse3qhri6xO+WTmZmVii8VmplZqXjiMjOzUvHEZWZmpeKJy8zMSsUTl5mZlYonLrMxkHSuJgt222W8x3JJ8+ofHUi6WdIH4/HeI4zZLmnZlRzTJpcpo+9iZiM4m3IKp7FYTk5ae+xiO0iakqo5AoeVUvqF6j8Ij7vIktEOdAIfX6lxbXLxGZdZnUnqkPRZJHTdUUhx84ikr5TX49oi6QZJi8m53F6JM7a5kvoldUafWZFmCUk9krZL6gN6IyPEBuU1vg5Ium+IWNoUa65F/22xDtKgpCckPRl990lqif36Jb0e8RyRtDDqW6L/QOy/IOrXSNokaQ+wCXge6I7+3ZIWSvoixtlbyQQR8WyV9Iny+kzrC3HfK+mb+Fn1Rt2ox2uTRCP/s9qbt7JvwDmq2Ts+JK+ztRdojfZuYEOUZxb6rQNWRXkj8EChrZ/IHADMAgaj3EPOclLJNvAC8FCUbyJnaZlaE18bsXRN9D9BXsOslZyx/bFoe42crLgy/jtRXlLo/wbwXJTvBA5GeQ3wNXB9YZw3CzE0U1365G5gS2G/H8h5DJuAn8j56lrJmcJnx34XfbzeJsfmS4VmY3PBpUJJ84H5wM6cUpFryKm1AOZLWkf+0r2RvGbVpdqZUvojyveQE9+ujtdNwC3k/IfD2ZVSOg2clnQK+CjqD5NTUlVsBkgp7ZbUHDkWu4AVUd8naaak5th/e0rp7DBjTgfelXQrOVXXtYW23pTSKQBJx8iLDc4AdqeUfoyxxnK8dhXyxGVWXwKOppQWDdG2EVieUjokqYec828o/1K9jN9U03amZqwVKaXjlxDfX4Xy+cLr81z4fVCbC2603HBnRmhbS54w74+HV/qHieccI38nXc7x2lXI97jM6us40CppEeTlVyTdHm3TgJPKS7KsLPQ5HW0Vg0BHlEd6sGIHsEpxaifpjrGH/7/ueM8ucqbuU8DnRNySlgK/p5p10ELt8UynukRFz0WMvQ9YIml2jNUS9eN5vFYinrjM6iil9Dd5snlZ0iHyva/F0fwMeaXmPcC3hW7vAU/FAwdzgVeBxyUdIN/jGs5a8mW3AUlH43W9/Bnjv01eOQDyvawOSQPAS1SXp6i1C5hXeTgDWA+8GO836lWelNJvwKPA1vgZvh9N43m8ViLODm9mF5DUD6xOKe1vdCxmQ/EZl5mZlYrPuMzMrFR8xmVmZqXiicvMzErFE5eZmZWKJy4zMysVT1xmZlYq/wHMtjIMs5w6egAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "lgb.plot_importance(model);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are numerous ways to compute feature importance and this one implemented in the LightGBM library measures the number of times a given feature is used in the constructed trees:\n", "\n", "https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.plot_importance.html\n", "\n", "In general, feature importance gives us a measure of how well a given measured variable correlates with the target (marketing success in our case).\n", "\n", "The question here is: How can we use our trained success predictor and our feature importances to aid intelligent plannning and decision-making in our business?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Uses for " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" } }, "nbformat": 4, "nbformat_minor": 4 }