{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Decision tree for classification\n",
"> A Summary of lecture \"Machine Learning with Tree-Based Models in Python\n",
"\", via datacamp\n",
"\n",
"- toc: true \n",
"- badges: true\n",
"- comments: true\n",
"- author: Chanseok Kang\n",
"- categories: [Python, Datacamp, Machine_Learning]\n",
"- image: images/decision-boundary.png"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import seaborn as sns"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Decision tree for classification\n",
"- Classification-tree\n",
" - Sequence of if-else questions about individual features.\n",
" - **Objective**: infer class labels\n",
" - Able to caputre non-linear relationships between features and labels\n",
" - Don't require feature scaling(e.g. Standardization)\n",
"- Decision Regions\n",
" - Decision region: region in the feature space where all instances are assigned to one class label\n",
" - Decision Boundary: surface separating different decision regions\n",
"![decision region](image/decision_boundary.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Train your first classification tree\n",
"In this exercise you'll work with the [Wisconsin Breast Cancer Dataset](https://www.kaggle.com/uciml/breast-cancer-wisconsin-data) from the UCI machine learning repository. You'll predict whether a tumor is malignant or benign based on two features: the mean radius of the tumor (```radius_mean```) and its mean number of concave points (```concave points_mean```)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Preprocess"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
id
\n",
"
diagnosis
\n",
"
radius_mean
\n",
"
texture_mean
\n",
"
perimeter_mean
\n",
"
area_mean
\n",
"
smoothness_mean
\n",
"
compactness_mean
\n",
"
concavity_mean
\n",
"
concave points_mean
\n",
"
...
\n",
"
texture_worst
\n",
"
perimeter_worst
\n",
"
area_worst
\n",
"
smoothness_worst
\n",
"
compactness_worst
\n",
"
concavity_worst
\n",
"
concave points_worst
\n",
"
symmetry_worst
\n",
"
fractal_dimension_worst
\n",
"
Unnamed: 32
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
842302
\n",
"
M
\n",
"
17.99
\n",
"
10.38
\n",
"
122.80
\n",
"
1001.0
\n",
"
0.11840
\n",
"
0.27760
\n",
"
0.3001
\n",
"
0.14710
\n",
"
...
\n",
"
17.33
\n",
"
184.60
\n",
"
2019.0
\n",
"
0.1622
\n",
"
0.6656
\n",
"
0.7119
\n",
"
0.2654
\n",
"
0.4601
\n",
"
0.11890
\n",
"
NaN
\n",
"
\n",
"
\n",
"
1
\n",
"
842517
\n",
"
M
\n",
"
20.57
\n",
"
17.77
\n",
"
132.90
\n",
"
1326.0
\n",
"
0.08474
\n",
"
0.07864
\n",
"
0.0869
\n",
"
0.07017
\n",
"
...
\n",
"
23.41
\n",
"
158.80
\n",
"
1956.0
\n",
"
0.1238
\n",
"
0.1866
\n",
"
0.2416
\n",
"
0.1860
\n",
"
0.2750
\n",
"
0.08902
\n",
"
NaN
\n",
"
\n",
"
\n",
"
2
\n",
"
84300903
\n",
"
M
\n",
"
19.69
\n",
"
21.25
\n",
"
130.00
\n",
"
1203.0
\n",
"
0.10960
\n",
"
0.15990
\n",
"
0.1974
\n",
"
0.12790
\n",
"
...
\n",
"
25.53
\n",
"
152.50
\n",
"
1709.0
\n",
"
0.1444
\n",
"
0.4245
\n",
"
0.4504
\n",
"
0.2430
\n",
"
0.3613
\n",
"
0.08758
\n",
"
NaN
\n",
"
\n",
"
\n",
"
3
\n",
"
84348301
\n",
"
M
\n",
"
11.42
\n",
"
20.38
\n",
"
77.58
\n",
"
386.1
\n",
"
0.14250
\n",
"
0.28390
\n",
"
0.2414
\n",
"
0.10520
\n",
"
...
\n",
"
26.50
\n",
"
98.87
\n",
"
567.7
\n",
"
0.2098
\n",
"
0.8663
\n",
"
0.6869
\n",
"
0.2575
\n",
"
0.6638
\n",
"
0.17300
\n",
"
NaN
\n",
"
\n",
"
\n",
"
4
\n",
"
84358402
\n",
"
M
\n",
"
20.29
\n",
"
14.34
\n",
"
135.10
\n",
"
1297.0
\n",
"
0.10030
\n",
"
0.13280
\n",
"
0.1980
\n",
"
0.10430
\n",
"
...
\n",
"
16.67
\n",
"
152.20
\n",
"
1575.0
\n",
"
0.1374
\n",
"
0.2050
\n",
"
0.4000
\n",
"
0.1625
\n",
"
0.2364
\n",
"
0.07678
\n",
"
NaN
\n",
"
\n",
" \n",
"
\n",
"
5 rows × 33 columns
\n",
"
"
],
"text/plain": [
" id diagnosis radius_mean texture_mean perimeter_mean area_mean \\\n",
"0 842302 M 17.99 10.38 122.80 1001.0 \n",
"1 842517 M 20.57 17.77 132.90 1326.0 \n",
"2 84300903 M 19.69 21.25 130.00 1203.0 \n",
"3 84348301 M 11.42 20.38 77.58 386.1 \n",
"4 84358402 M 20.29 14.34 135.10 1297.0 \n",
"\n",
" smoothness_mean compactness_mean concavity_mean concave points_mean \\\n",
"0 0.11840 0.27760 0.3001 0.14710 \n",
"1 0.08474 0.07864 0.0869 0.07017 \n",
"2 0.10960 0.15990 0.1974 0.12790 \n",
"3 0.14250 0.28390 0.2414 0.10520 \n",
"4 0.10030 0.13280 0.1980 0.10430 \n",
"\n",
" ... texture_worst perimeter_worst area_worst smoothness_worst \\\n",
"0 ... 17.33 184.60 2019.0 0.1622 \n",
"1 ... 23.41 158.80 1956.0 0.1238 \n",
"2 ... 25.53 152.50 1709.0 0.1444 \n",
"3 ... 26.50 98.87 567.7 0.2098 \n",
"4 ... 16.67 152.20 1575.0 0.1374 \n",
"\n",
" compactness_worst concavity_worst concave points_worst symmetry_worst \\\n",
"0 0.6656 0.7119 0.2654 0.4601 \n",
"1 0.1866 0.2416 0.1860 0.2750 \n",
"2 0.4245 0.4504 0.2430 0.3613 \n",
"3 0.8663 0.6869 0.2575 0.6638 \n",
"4 0.2050 0.4000 0.1625 0.2364 \n",
"\n",
" fractal_dimension_worst Unnamed: 32 \n",
"0 0.11890 NaN \n",
"1 0.08902 NaN \n",
"2 0.08758 NaN \n",
"3 0.17300 NaN \n",
"4 0.07678 NaN \n",
"\n",
"[5 rows x 33 columns]"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"wbc = pd.read_csv('./dataset/wbc.csv')\n",
"wbc.head()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"X = wbc[['radius_mean', 'concave points_mean']]\n",
"y = wbc['diagnosis']\n",
"y = y.map({'M':1, 'B':0})"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.model_selection import train_test_split\n",
"\n",
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1 0 0 1 0]\n"
]
}
],
"source": [
"from sklearn.tree import DecisionTreeClassifier\n",
"\n",
"# Instantiate a DecisionTreeClassifier 'dt' with a maximum depth of 6\n",
"dt = DecisionTreeClassifier(max_depth=6, random_state=1)\n",
"\n",
"# Fit dt to the training set\n",
"dt.fit(X_train, y_train)\n",
"\n",
"# Predict test set labels\n",
"y_pred = dt.predict(X_test)\n",
"print(y_pred[0:5])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Evaluate the classification tree\n",
"Now that you've fit your first classification tree, it's time to evaluate its performance on the test set. You'll do so using the accuracy metric which corresponds to the fraction of correct predictions made on the test set."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test set accuracy: 0.89\n"
]
}
],
"source": [
"from sklearn.metrics import accuracy_score\n",
"\n",
"# Predict test set labels\n",
"y_pred = dt.predict(X_test)\n",
"\n",
"# Compute test set accuracy\n",
"acc = accuracy_score(y_test, y_pred)\n",
"print(\"Test set accuracy: {:.2f}\".format(acc))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Logistic regression vs classification tree\n",
"A classification tree divides the feature space into rectangular regions. In contrast, a linear model such as logistic regression produces only a single linear decision boundary dividing the feature space into two decision regions."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Helper function"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"from mlxtend.plotting import plot_decision_regions\n",
"\n",
"def plot_labeled_decision_regions(X,y, models):\n",
" '''Function producing a scatter plot of the instances contained \n",
" in the 2D dataset (X,y) along with the decision \n",
" regions of two trained classification models contained in the\n",
" list 'models'.\n",
" \n",
" Parameters\n",
" ----------\n",
" X: pandas DataFrame corresponding to two numerical features \n",
" y: pandas Series corresponding the class labels\n",
" models: list containing two trained classifiers \n",
" \n",
" '''\n",
" if len(models) != 2:\n",
" raise Exception('''Models should be a list containing only two trained classifiers.''')\n",
" if not isinstance(X, pd.DataFrame):\n",
" raise Exception('''X has to be a pandas DataFrame with two numerical features.''')\n",
" if not isinstance(y, pd.Series):\n",
" raise Exception('''y has to be a pandas Series corresponding to the labels.''')\n",
" fig, ax = plt.subplots(1, 2, figsize=(10.0, 5), sharey=True)\n",
" for i, model in enumerate(models):\n",
" plot_decision_regions(X.values, y.values, model, legend= 2, ax = ax[i])\n",
" ax[i].set_title(model.__class__.__name__)\n",
" ax[i].set_xlabel(X.columns[0])\n",
" if i == 0:\n",
" ax[i].set_ylabel(X.columns[1])\n",
" ax[i].set_ylim(X.values[:,1].min(), X.values[:,1].max())\n",
" ax[i].set_xlim(X.values[:,0].min(), X.values[:,0].max())\n",
" plt.tight_layout()"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAskAAAFgCAYAAABJzuRWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzde5xVdb3/8ddnLgwDA4MyXkHzAnmtY6h0t5OdTD2oeUPJRE2zLDqVHkuzU52yMstLqaFoqRwlJYvih5paJpSUjpeC0ATFC4hAA8wwA8wMM/P5/bHWhj17bmvP7LWv7+fjsR/MXmuvtb57A+/57u/6XszdERERERGRncpyXQARERERkXyjSrKIiIiISApVkkVEREREUqiSLCIiIiKSQpVkEREREZEUqiSLiIiIiKRQJVkKjpmdY2aPDvLYZWb27xkuUt4xs1vN7H9yXQ4RyV9m9rCZnRfhdS1mdkA2ypQtZvYfZvZajOe/w8y+lvR8hpmtDz/L2vDPfeO6vmSGaZ5kiVsYRBe5+++zfN27gNXu/vVBHOvAVsCBJuB+4HJ378xoIUVEBhBm6B5AB9AJvADMBma5e1cOixaZmS0D3hY+rQa2E7wfgO+5+/diuOZ7gG8B7yX43FYAt7j7bDP7D+AOd98v09ftpRzDgUbgSHdfFvf1JHPUkizSt39z9xrgQ8BZwKcyfQEL6P+hiAzkJHcfRVDRvAb4KvCz3BYpOnc/zN1rwkz9EzAj8by3CrKZVQzlemb2AeD3wB+AA4CxwAzgxKGcd5D2BKoyUUEe6uci6dEvZ8kZM/u0mb1sZhvNbL6Z7Z207zgze8nMmszsp2a20MwuCvedb2Z/Dn82M7shvI3VZGZLzOxwM7sYOAf4Snhb6/+Fr38tbEHAzMrN7Gtm9oqZNZvZs2a2T2o53f1l4EngiKTy1ZrZz8zsLTN708yuNrPypPNeZ2YNZvZqeJvNE+FmZk+Y2XfN7EmC1uoDBjjfhPD9N4XnvL+/9x7uu8vMro74WbuZfdbMVpjZJjO7xcwsE3/HIpJZ7t7k7vMJvrifF+ZdlZn9yMzeMLN1YXer6sQxZnaKmf3NzDaHeXd8uP2JpFztNWfCfW5mE8Kfa81stpn9y8xeN7OvJ77oJ7I5LMumMP9OiPK+zOwiM1tkZj8xs43A15O2/zM838PJGW1mh5rZ78Nc+6eZnZ50yh8BP3P3H7r7Bg/Uu/vZfVz/62a2MvxdsMzMTk7a9/awbInPZk64vSwsb3IGHxruu8fMvmVmhwDLwm0tZvaomVWEn+l+4fbhZna9ma0K//5+akHr845uIRb8rloL3B7l85TMUCVZcsLMjgW+D0wF9gJeB+4L99UBDwBXEnz7fwl4Xx+nOg44Bng7MIbgF8cGd58F3AtcG7ZUnNTLsZcC0whaFkYTtBRv7aWsBwMfBF5O2nw3wa3CCcC7wnJcFO77NHACQaV6EvDxXq59LnAxMCp87/2d7zvAo8AuwHjgpv7eey/l7/OzTjIFOBr4t/B1H+ulzCKSJ9z9aWA1QTb9gCAHjiDIkHHANwDMbDJB14zLCXLiGOC1Xk7ZV86kugmoJWid/RAwHbggaf+7CTK7DrgW+FkaX7rfB7wI7Ab8wMzOCMt9SrjtKSBRQR0FPBa+t90JGkVmmdlB4b7JBL9HoloOvD98b98F5pjZHuG+7wIPsvOzuSXcfgLwHmBiuO9sYGPySd39RYJcJfxddFwv1/4RsD/wzvBc+wFXJe0fD9QA+wKfS+M9yRCpkiy5cg7wc3d/zt3bCCrE7w2/WZ8ILHP3X7t7B/ATYG0f59lOUNE8mKCP/Yvu/lbEMlwEfN3dXwpbGf7u7smVzOfMbAtBaD8B/BQgDM4TgC+5+xZ3Xw/cQBCQEFQyf+zuq919E8Gt0VR3ufuy8P3tOsD5thPcYt3b3Vvd/c9pvvf+PuuEa9y90d3fAP5IUqu5iOStNQT58Wngy+6+0d2bge+xMz8uJPj//5i7d7n7m+7+z17O1VfO7GDB3a2zgCvdvdndXwOuI/jSn/C6u98ejt+4m+CL+R6p5+rDG+4+09073X0b8BmC/sovhVl5NTDZzMYBJwPL3X22u3e4+7PAb4Azws/EgKi/C3D3ue7+VvgZzSH4InFU0mezH7BX+Nk8mbR9NEEG4+4vuHtfv6t6FbbCX0SQ/5vcfTNBo0Zyi3cH8C13bw8/F8kSVZIlV/YmaNEEwN1bCFpBx4X7ViXtc4IWkx7c/XHgZoJv9uvMbJaZjY5Yhn2AV/rZP4ng2/tZBK0jI8PtbwMqgbfMrNHMGoHbCFozEu9tVdJ5kn/ubdtA5/sKQeA/Hd4G/BSk9d77+6wTkoN9a/i+RSS/jQMqgBHAs0n58TuCllcYOOcSes2ZFHXAMJLyJPy51yxx98Sduah5kpqVbwNuSXpfDUAXQcvq24D3J/aF+88iqJRvJBh0vVfE6ya6ivw96VwHE7xfgMsIMvoZM1tq4Ywg7v4ocCswkyCDbw1bsdOxJ1AFJF97ATvzH2Cdu7eneV7JAFWSJVfWsHOkM2Y2kqBrxZsE3/7HJ+2z5Oep3P0n7n4kcBjBLcfLE7sGKMMq4MD+XhC2MM8F/kJ4+zI8rg2oc/cx4WO0ux8W7u9WfoJfUj1OnVKOPs/n7mvd/dPuvjdBy8pPE/0D+3nvyfr7rEWkAJnZ0QSV098A24DDkvKjNhwgBxFyDvrPmSQN7GxxTtiXzGVJamavAi5Mel9j3L3a3Z8K9/0hZV+Nu88IW9OfBk5PvUBvLJjebiZwCTDW3ccA/yT40kDYwnyRu+8FfJ6gW8f+4b4b3X0ScDhwKEE3vnSsA9qBg1L+/mr7+VwkS1RJlmypDAcnDA8HJMwFLjCzI8ysiuD24FPh7bsHgXeY2cctGOz2eYJv2z2Y2dFm9m4zqwS2AK0EU/1AED79ze15B/AdM5togXea2dg+XnsNcLGZ7Rl2aXgUuM7MRoeDNw40sw+Fr50LfNHMxpnZGIJR6H0a6HxmdqaZJSrdmwgCs3OA955sDn1/1iJSQMKMmEIwruAed/87wWCuG8xs9/A148wsMa7gZwT//z8SZsu4cJxF6nl7zZnk14RdKOYC3zWzUWb2NoJK4T0xvFUIWmmvCge/YWZjwn7KAPOBw8zsE2ZWGT4mm9lB4f7LgYvM7FIz2zU8/l0WDrpLUUPwfv8VvMwuIuxCER43NeziAcFUbokMnhw+KggyuJ3eM7hP4Wd6B3Cjme0W/i4ab2a99V2WLFMlWbLlIYLWjsTjg8D/AL8iaHk9kLAPlrs3AGcSDPrYQPDt/BmC1tZUowl+QWwiuO23gWAQBAS/HA4Nb2H9ppdjrycI/EeBzeHrq3t5He6+FFjIzpba6QS3HV8Ir/0AO2/t3R6ecwnwfPjeE/Ob9qW/8x0NPGVmLQS/GL7o7q8O8N6Ty/4H+visRaRg/D8zayZoQb2KIL8SA+a+SjCw+K9mtplg6rODYMcAvwsIxjk0EeTY2+ipr5xJ9QWCCuFK4M8EX8J/nok3mMrdf0nwPn8Zvq8lhIOK3b0p/PmTBLm2lqAvb1W4/0/Af4Svec2CGTNmEuRx6nWWEIx9eTo818EEgwQT3g3UWzBG5dfA58PxG2MIfm80EvRhfovgc07XZQQZ/jTB39GjBAP4JMe0mIjkvXBgw2rgHHf/Y67Lky4LpkC61d17+8UkIiIieUgtyZKXzOxj4a21KuBrBH3D/prjYkViZtVmdqIFc2GOA74JzMt1uURERCQ6VZIlX72XYER2A3AS8PECmvrGgP8l6AbxPMEUct/o9wgRERHJK+puISIiIiKSQi3JIiIiIiIpKnJdgGy5fdFKNZkXgL/+8mZ+fMre1IyoynVRRLLu+nsfhTef5dIjtjzLt5qOGviI/FNMWbts8aOcu/srHHnwvrkuioj0YUduHlPL9YuaYNyRXHpOxBn09jgMDjy2z2XT1ZIseaVyeA3NW1tzXQyRrGtobGHBwnqmTxo58ItFRKRHbk6fNJIFC+vZ0LQlI+dXJVnySkX1SFq29TYdskhxm/3gYqZMKKOupmRu8ImIDElqbtbVVDBlQhl3L3gyI+dXGkteqRg+iuatG3JdDJGse+K55axZ38acpetZva65bu23cl0iEZH8lpybyfZetzx6l4t+qJIseaW8epS6W0hJmn/djJ1P3veF/XJWEBGRAtEtN2NQ0pVkw6mt7GJ4OZj12W87Z9yd1k5o2l6Gk3/li0PViBqamjtyXQwRySBlrYgUopKuJNdWdjFm5HC6rALyMLhxZ7h3wJZWGreX57o0WVFVPYLm9aokixQTZa2IFKKSHrg3vJz8DW0AM7qsguEllNnDR4ykaVt7roshIhmkrBWRQlTSlWQzy9/QTjDLy9uTcamqHknTVlWSRYqJslZEClFJV5Il/1RUDmN7Z1euiyEiIiIlTpXkHHvmz49z4Ukf4IIT38v9d9yU6+KIiBQt5a2IpEOV5Bzq7Ozklu9+jat/ei+zfruQJx7+Da+/8lKuiyUiUnSUtyKSrpKe3SIdX5x+Kk2bN/fYXjt6ND+ePW9Q53xp6fPste9+7LXP2wD40Amn8Jc/PsLbDjxoSGUVESlUcWQtKG9FJH2qJEfUtHkzEy++ucf2FbMGP5H1hvVr2W3PcTue1+2xFy8teX7Q5xMRKXRxZC0ob0UkfepukUPu3mObRleLiGSe8lZE0qVKcg7V7bEX/1r75o7nDeveYtfd98hhiUREipPyVkTSpUpyDh10+BGsef1V1q5+g+3b21n48G95z79/LNfFEhEpOspbEUlX7JVkMzvezF4ys5fN7Ipe9h9jZs+ZWYeZnZG0/cNm9rekR6uZfTzcd5eZvZq074i430ccyisq+NzXvsdVn53GxScfwzEfO4n9JmgQiYikT1nbP+WtiKQr1oF7ZlYO3AJ8FFgN1JvZfHd/IellbwDnA/+dfKy7/xE4IjzPrsDLwKNJL7nc3R+Ir/Td1Y4e3evAkdrRo4d03snHfITJx3xkSOcQkdKmrI1GeSsi6Yh7dovJwMvuvhLAzO4DTgF2BLe7vxbu62+ZtTOAh919a3xF7d9Qph4SEYmZslZEJMPi7m4xDliV9Hx1uC1dZwO/SNn2XTNbYmY3mFlVbweZ2cVm9oyZPbNofurhIiJFQ1krIpJhcVeSe5tfp+c8PP2dwGwv4B3AI0mbrwQOBo4GdgW+2tux7j7L3Y9y96OOOXlaOpcVESkkyloRkQyLu5K8Gtgn6fl4YE2a55gKzHP37YkN7v6WB9qAOwluNYqIlCplrYhIhsVdSa4HJprZ/mY2jOBW3vw0zzGNlNt/YYsHFswE/3HgHxkoq4hIoVLWiohkWKyVZHfvAGYQ3L57EZjr7svM7NtmdjKAmR1tZquBM4HbzGxZ4ngz24+gdWRhyqnvNbOlwFKgDrg6zvchIpLPlLUiIpkX9+wWuPtDwEMp276R9HM9wa3B3o59jV4Gn7j7sZktZe5c/z9f5qlFjzFm1zpum/dErosjIgVKWds/Za2IpEsr7uXYR0+ZytUz5+S6GCIiRU1ZKyLpUiU5TU2bNvDd//okmxs3ZuR87zjqvYyq3SUj5xIRKRbKWhHJNVWS0/T4b+6la83f+cO8e3JdFBGRoqWsFZFcUyU5DU2bNvD8Yw9w42njef6xBzLWwiEiIjspa0UkH6iSnIbHf3MvJ02AiXtUc9IE1MIhIhIDZa2I5ANVkiNKtGx84shaAD5xZK1aOEREMkxZKyL5QpXkiBItG2NrKoHgz0y0cHz/K5fw5U9OYfVrr/DJj0zid7/W6GsRKV3KWhHJF7HPk1wslj79J/70Viu/WLK62/Yx//oTp17wX4M+75XXzhxq0UREioayVkTyhSrJEX1j5i9zXQQRkaKnrBWRfKHuFiIiIiIiKUq6kuzu4J7rYvTPPSiniEiBUtaKSCEq6UpyayeUeUf+hrc7Zd5Ba2euC5I9ZWVldHTm6d+HiAyKslZEClFJ90lu2l4GW1oZXg5mluvi9ODutHaG5SwRFZXD2Nau31QixURZKyKFqKQryY7RuL0ctue6JJJgZnSZflGJFBNlrYgUItVGJO90uf5ZioiISG6pNiJ5Ry3JIiIikmuqjUje6dQ/SxEREckx1UYk76i7hYiIiOSaaiOSd9TdQkRERHJNtRHJO136ZykiIiI5ptqI5J1Oz795VEVERKS0qJIsecdRJVlERERyS5VkEREREZEUqiSLiIiIiKRQJVlEREREJIUqySIiIiIiKVRJFhERERFJoUqyiIiIiEgKVZJFRERERFKokiwiIiIikiL2SrKZHW9mL5nZy2Z2RS/7jzGz58ysw8zOSNnXaWZ/Cx/zk7bvb2ZPmdkKM7vfzIbF/T5ERPKZslZEJLNirSSbWTlwC3ACcCgwzcwOTXnZG8D5wJxeTrHN3Y8IHycnbf8BcIO7TwQ2ARdmvPAiIgVCWSsiknlxtyRPBl5295Xu3g7cB5yS/AJ3f83dlwBdUU5oZgYcCzwQbrob+HjmiiwiUnCUtSIiGRZ3JXkcsCrp+epwW1TDzewZM/urmSXCeSzQ6O4dA53TzC4Oj39m0fxfpFt2EZFCoawVEcmwuCvJ1ss2T+P4fd39KOATwI1mdmA653T3We5+lLsfdczJ09K4rMjQNTS2cPoVt7KhaUuuiyLFT1krJUtZK3GJu5K8Gtgn6fl4YE3Ug919TfjnSuAJ4F1AAzDGzCoGc06RbJn94GI2rV3F3QuezHVRpPgpa6VkKWslLnFXkuuBieEI6WHA2cD8AY4BwMx2MbOq8Oc64P3AC+7uwB+BxOjs84DfZrzkIkPQ0NjCgoX1zDytjgUL69XCIXFT1kpJUtZKnGKtJId92WYAjwAvAnPdfZmZfdvMTgYws6PNbDVwJnCbmS0LDz8EeMbM/k4Q1Ne4+wvhvq8Cl5rZywT95n4W5/sQSdfsBxczZUIZB+1exZQJZWrhkFgpa6VUKWslThUDv2Ro3P0h4KGUbd9I+rme4DZe6nGLgXf0cc6VBKO5RfJOomVj7tRRAEyfNJKpc+s5b8r7GVs7Mselk2KlrJVSo6yVuGnFPZEMS7Rs1NUE30HrairUwiEikmHKWolb7C3JIqXmieeWs2Z9G3OWru+2fe91y7n0nONyVCoRkeKirJW4qZIskmHzr5uR6yKIiBQ9Za3ETd0tRERERERSqJIsIiIiIpJClWQRERERkRSqJIuIiIiIpFAlWUREREQkhSrJIpK2hsYWTr/iVi0BKyKSIcrV/KNKsoikbfaDi9m0dpUm7RcRyRDlav5RJVmKhr6FZ0diKdiZp9WxYGG9Pm+REqbczQzlan5SJVmKhr6FZ0diKdiDdq/SErAiJU65mxnK1fykSrIUBX0Lz47E5zx90kgApk8aqc9bpEQpdzNDuZq/VEmWoqBv4dmR+JzraoIV7etqKvR5i5Qo5W5mKFfzV0WuCyAyVIlv4XOnjgKCb+FT59Zz3pT3M7Z2ZI5LV1yeeG45a9a3MWfp+m7b9163nEvPOS5HpRKRbFPuZo5yNX+pkiwFr79v4QqYzJp/3YxcF0FE8oByN3OUq/lLlWQpePoWLiKSXcpdKQWqJEvB07dwEZHsUu5KKdDAPRERERGRFKoki4iIiIikUCVZ8k4XlusiiIiISIlTJVnyjlVU0da+PdfFEBERkRKmSrLknYrqGlq2tee6GCIiIlLCVEmWvDOsuobmra25LoaIiIiUMFWSJe9UVNfQvLUt18UQERGREqZKsuSdsuGjaNmmSrKIiIjkjirJkncqq0exeav6JIuIiEjuqJIseWf4iJE0bdXsFiIiIpI7qiRL3qmqHsHmbaokZ0pDYwunX3ErG5q25LooIiJFQ9la/CJVks1sNzP7mpnNMrOfJx4Rjz3ezF4ys5fN7Ipe9h9jZs+ZWYeZnZG0/Qgz+4uZLTOzJWZ2VtK+u8zsVTP7W/g4IkpZpDCUV1TS4Z7rYhSN2Q8uZtPaVdy94MlcF0VipKwVyS5la/GL2pL8W6AW+D3wYNKjX2ZWDtwCnAAcCkwzs0NTXvYGcD4wJ2X7VmC6ux8GHA/caGZjkvZf7u5HhI+/RXwfIiWlobGFBQvrmXlaHQsW1qvFo0gpa0WyS9laGqJWkke4+1fdfa67/yrxiHDcZOBld1/p7u3AfcApyS9w99fcfQnQlbJ9ubuvCH9eA6wHdotYXilyus0VzewHFzNlQhkH7V7FlAllavEoXspaiZ1ydydla2mIWkleYGYnDuL844BVSc9Xh9vSYmaTgWHAK0mbvxveGrzBzKoGUTYpYLrNNbBES8f0SSMBmD5ppFo8ipeyVmKn3A0oW0tH1EryFwkqytvMbLOZNZvZ5gjHWS/b0upsamZ7Af8HXODuiRaQK4GDgaOBXYGv9nHsxWb2jJk9s2j+L9K5rOQx3eaKJtHSUVdTAUBdTYVaPIqXslZipdzdSdlaOiqivMjdRw3y/KuBfZKejwfWRD3YzEYT9H3+urv/Nak8b4U/tpnZncB/93a8u88CZgHcvmilRoIVie63uVq5e8GTXHrOcbkuVt554rnlrFnfxpyl67tt33vdcn1exUdZK7FS7u6kbC0dkSrJAGa2CzARGJ7Y5u6LBjisHphoZvsDbwJnA5+IeL1hwDxgtrv/MmXfXu7+lpkZ8HHgH1HfhxS2RGvG3KnB97bpk0YydW495015P2NrR+a4dPll/nUzcl0EyR5lrcRGududsrV0RJ0C7iJgEfAI8L/hn98a6Dh37wBmhK9/EZjr7svM7NtmdnJ47qPNbDVwJnCbmS0LD58KHAOc38v0Q/ea2VJgKVAHXB3p3UrB020ukZ6UtRIn5a6UqqgtyV8k6JP2V3f/sJkdTFBZHpC7PwQ8lLLtG0k/1xPcGkw97h7gnj7OeWzEckuR0W0ukd4payUuyl0pVVErya3u3mpmmFmVu//TzA6KtWQivSiU21wNjS185pp7mHXluSV5O1JEikc+566yVuIUdXaL1eHk8r8BHjOz35LGoBCRUqOpkkRE4qeslThFqiS7+6nu3uju3wL+B/gZwSAOEUmhqZJEROKnrJW4RW1Jxsw+YGYXuPtC4C8MYqJ6kVKglZhEROKnrJW4RZ3d4psEk8hfGW6qpI+BHiKlTCsxiYjET1kr2RC1JflU4GRgC4C7rwEGu8CISNHSVEkiIvFT1ko2RJ3dot3d3cwcwMw0hFSkF5oqSUQkfspayYaoleS5ZnYbMMbMPg18Crg9vmKJFKZ8nipJRKRYKGslGyJVkt39R2b2UWAzcBDwDXd/LNaSiYiIiIjkSNSWZNz9MTN7KnGMme3q7htjK5mIiIiISI5EqiSb2WeAbwPbgC7AAAcOiK9oIiIiIiK5EbUl+b+Bw9y9Ic7CiIiIiIjkg6hTwL0CbI2zICIiIiIi+SJqS/KVwOKwT3JbYqO7/1cspRIRERERyaGoleTbgMeBpQR9kkVEREREilbU7hYd7n6pu9/p7ncnHrGWTCSChsYWTr/iVi1FKiKSJcpdKRVRK8l/NLOLzWwvM9s18Yi1ZCIRzH5wMZvWrtJSpCIiWaLclVIRtZL8CcJ+ycCz4eOZuAolEkVDYwsLFtYz87Q6FiysV6uGiEjMlLtSSiJVkt19/14eO+ZIDlfjE8mq2Q8uZsqEMg7avYopE8rUqiEiEjPlrpSSqC3JA/lBhs4jQnlFBe3b+x8fmmjNmD5pJADTJ41Uq4aISIyUu1JqMlVJtgydR4Th1SPYvK2939ckWjPqaoIJWupqKtSqISISI+WulJqoU8ANxDN0HhGGDR9By7bt/b7mieeWs2Z9G3OWru+2fe91y7n0nOPiLJ6ISElS7kqpyVQlWXKsuXEj9/3wcqZ95UfU1O6S6+IMSVlZGR3e/82J+dfNyFJpRETS09DYwmeuuYdZV57L2NqRuS5Oxih3pdRkqrvFaxk6jwxS/cP3U7FuKU8/dF+ui5IRXRn7pykikl2aIk2kOESqiZjZmWY2Kvz562b2azOblNjv7qfFVUAZWHPjRl5aNI/rTh3HS4vm0dK0KddFGrIuUyVZRAqPpkgTKR5RayL/4+7NZvYB4GPA3cDM+Iol6ah/+H5OmggTdq/mpIkURWtyl6uSLCKFR1OkiRSPqDWRzvDP/wRmuvtvgWHxFEnSkWhFnjapFoBpk2qLojW5y8ozfk4tpSoicdIUad0pc6XQRa0kv2lmtwFTgYfMrCqNYyVGiVbksSMrgeDPYmhN7oxhVkH1ExSROGmKtO6UuVLoos5uMRU4HviRuzea2V7A5fEVS6Ja8fyTPL++lfuXrO62vWbtkxw77ZIclWroOjPc3SK5n+AlC+o5b8r7i2rUuYjknqZI20mZK8UgaiX5Nnc/N/HE3d8ys2uBR+MplkT1mWvvyXURYmEZbkju3k+wlbsXPFlyv7REJF6aIm0nZa4Ug6jNdYclPzGzcuDIKAea2fFm9pKZvWxmV/Sy/xgze87MOszsjJR955nZivBxXtL2I81saXjOn5hlukolxUT9BKUUKGslXyhzpVj0W0k2syvNrBl4p5ltDh/NwHrgtwOdPKxM3wKcABwKTDOzQ1Ne9gZwPjAn5dhdgW8C7wYmA980s8QqGTOBi4GJ4eP4gcoipWuo/QQ1+ETynbJW8slQMld5K/mk3+4W7v594Ptm9n13v3IQ558MvOzuKwHM7D7gFOCFpGu8Fu7rSjn2Y8Bj7r4x3P8YcLyZPQGMdve/hNtnAx8HHh5E+aQEDLWfYPLgE90ulDylrJW8MZTMVd5KPonUJ9ndrzSzccDbko9x90UDHDoOWJX0fDVBa0UUvR07Lnys7mV7D2Z2MUErCJ+87GqOOXlaxEtLMRlKP0ENPpECoayVvDHYzFXeSr6JuuLeNcCTwNcJZrW4HPjvKIf2ss0jlq2vYyOf091nuftR7n6UQlsGQwsDSIFQ1krBU95Kvok6cO9U4CB3P9HdTwofJ0c4bjWwT9Lz8cCaiNfs69jV4c+DOadIZPk0+ET99GQAylopaLnMW+Wr9CVqJXklUDmI89cDE81sfzMbBpwNzI947CPAcWa2SziI5DjgEXd/C2g2s/eEI62nE2EQoUi68mlhABCY9pYAACAASURBVE3KLwNQ1kpBy2XeKl+lL1HnSd4K/M3M/gC0JTa6+3/1d5C7d5jZDIIQLgd+7u7LzOzbwDPuPt/MjgbmAbsAJ5nZ/7r7Ye6+0cy+QxD+AN9ODCwBLgHuAqoJBpFoIIlkXL4sDKB+ejIQZa0UulzlrfJV+mPuA3dbS543M5m7353xEsXk9kUro/bPkzzwzP99h9sumJTrYuSF6+99FN58lkuPqeX6RU0w7kiN+i527/tCwc5HXExZu2zxo5y7+yscefC+uS6KxET5WuL2OAwOPLbPvI3U3cLd7+7tkblSikhv8qlftIhIMVG+ykAGWkxkbvjnUjNbkvrIThFFSlc+9YsWESkmylcZyEB9kr8Y/jkl7oKISE/50i9aRKTYKF9lIAOtuPdW+OfrZrYHcHS462l3X9/3kSKSCUNZCEVERPqmfJWBRF1MZCrwNHAmMBV4yszOiLNgIiIiIiK5EnUKuKuAoxOtx2a2G/B74IG4CiYiIiIikitRFxMpS+lesSGNY0XStq21NZYVkIptZaViez8ikluZzpRCzqhCLrtkRtSK7u/M7BEzO9/MzgceBB6Kr1hS6lYsXx7LCkjFtrJSsb0fEcmtTGdKIWdUIZddMiPqPMmXA7cB7wT+DZjl7l+Ns2BSupobN7L61VeYeVpdRuesTF5ZqRjmwiy29yMiuZXpTCnkjCrkskvmpNNlYjGwEHgc+Es8xRGB+ofv5+27wkG7V2V0zsrEnJiZPm+uFNv7EZHcynSmFHJGFXLZJXOizm5xEcHsFqcCZwB/NbNPxVkwKU3NjRt5adE83rFXFZC5FZBSV1b6z4OGc/2cx1ixKnczGQ6lv5tWihKRTMp0puRD5g42Y5WvkhC1Jfly4F3ufr67nwccCai7hWRc/cP3c9JEGDEs+KeZqRWQUldWevDFFvap6eQrN/1yyGUeSpkG299NK0WJSCZlOlPyIXMHm7HKV0mIOgXcaqA56XkzsCrzxZFS0dy4kft+eDnTvvIjamp32bF9xfNP8vz6VjZtaubRl9t3bB/qCkjJKyt1dHaxubmFm06o4tJHX2ND0xbG1o4c0vtJV3J/t0sW1HPelPenVQatFCVSONydF15dS5d77NfadfQIxu02ps/9DY0tfOaae5h15bndMic5U1rbO3GCsu726lKOf987OGS/PTGzyOXIdeYOJWOVr5JgHuE/rZnNBt4B/BZw4BSC7hfLAdz9+hjLmBG3L1oZfzpJZI//Yiav/+FO3vaRCzh22iU99j/zf9/htgsmxXLt6+99FN58lkuPqeX6RU0w7sisB18+lEHy2Pu+EL02kmeKKWuXLX6Uc3d/hSMP3ndI53ntrQ184b6XOODf3puhkvVtw99/zz1f/mif+6+/91EWPLaQKR/9UK+Z07K1jdOuf4JD3rdz34pnFvHzC9/FnmNHD6pMucg7ZaxEssdhcOCxfeZt1JbkV8JHwm/DP0cNtlySX/pq2Y3rWi8tmsctp47j8wvmMfnEs2O/ZkKidWHu1OCf7vRJI5k6N/2W3EIvg4hkT8u2Ng5453s4/Jgp/b4uEzn83OvP9bkvSutqy7Y2DjjsXd3K2tm2jS3bhtY3OZt5p4yVTIk6Bdz/9veIu5ASv/qH76di3VKefui+rFzrpIkwYfdqTppIVq6ZkA99zfKhDCKSPS1b2yirGrhylokc7qTvmxBRZmzYsq2N8pSylg8fQcu29h6vjSIXeaeMlUyJ2pIsRSybLbuJa33zrFoApk2q5RP3Z681OR/6muVDGUQke1pa26morun3NZnK4S7vvZIctXV1S2s7FcPruh1bVjWSLa1taZcFcpN3yljJFFWSJaVldwtPP3Rfr/2EM3mtsSMrgeDPRGtyXNdMNv+6GbFfoxDK0NfgHRHJvObWTqp2qe73NZnK4c4+bhD317qaXHHc0tpO2fDuFfph1TVs3jq4luRc5F0uM1bZWlzSWUxEilCi9WLapJ0tuy8tmkdL06ZYrrfi+Se5f0krH7xl9Y7H/UtaWfG8boNlk5ZbFcmejk6nrLzvNqlM5rD30d3iieeWM2dpG0fdsn7HY87SNp54bnm313V2dmFl5d22lZVX0NGVdlFKkrK1uERqSTaztwMzgT3c/XAzeydwsrtfHWvpJHbZbtn9zLX3ZPyckp6hTj8nIpmVjRzOhztYxU7ZWnyidre4nWBBkdsA3H2Jmc0BVEkucIl5ie9fsrrb9pq1T2al+4NkX/fBO609breKSHYph4uDsrX4RK0kj3D3p1MmEu+IoTySZYmW3WxOASe5o6mRRPKPcrjwKVuLU9Q+yQ1mdiDBQiKY2RnAW7GVSrIum1PASe5oaiSR/KUcLlzK1uIUtZL8eYKuFgeb2ZvAl4DPxlYqyarEoJHrTh0X66C9YtXQ2MLpV9zKhqbBTbafTVEH74hIdimHB5bPWatsLU5Ru1u87u7/YWYjgTJ3b46zUJJd2ZwCrhglj2bO9/5nGrwjkp+UwwPL56xVthanqC3Jr5rZLOA9QEuM5ZEsy/YUcMUmeTTzgoX1ednCISL5TTk8MGWt5ELUSvJBwO8Jul28amY3m9kH4iuWZEt/Uw8Vmzhu1Q20zGs+3x4UkfxQrDmcyfzrL2uVsxKXSJVkd9/m7nPd/TTgXcBoYGGsJZOsyNfFPbo88+fM9CTviZaN6ZOCkcvTJ43s0cKhieVFZCD5msNDlan8GyhrlbMSl8jLUpvZh4CzgBOAemBqXIWS7MnXxT3cMrsYZByTvA+0zKsmlheRKPI1h4cik/nXX9ZO/8/3KWclNlFX3HsV+BswF7jc3XVPQ2LV0ZXZSnIck7w/8dxy1qxvY87S9d22771uOZeec5wmlheRkpXJ/OsvawHlrMQmakvyv7n75sFcwMyOB34MlAN3uPs1KfurgNnAkcAG4Cx3f83MziFY5S/hncAkd/+bmT0B7AVsC/cd5+7d//dIQesqK6ezs4vy8qFXluOa5L2/0cyaWF6yTVkr+SLT+ddX1jY0tjD1Kz9WzkpsotZA2s3s82b2UzP7eeIx0EFmVg7cQtBF41BgmpkdmvKyC4FN7j4BuAH4AYC73+vuR7j7EcC5wGvu/rek485J7FdoF59hw0ewpbUtI+fKxSTvmlhesklZK/kkW/mnnJW4RW1J/j/gn8DHgG8D5wAvRjhuMvCyu68EMLP7gFOAF5JecwrwrfDnB4CbzczcPXno1jTgFxHLKjHLxtKpFdU1tGxtZ/TI6iGfa6BuEXHIxTWlpClrS0i+L1+drfxTzkrcolaSJ7j7mWZ2irvfbWZzgEciHDcOWJX0fDXw7r5e4+4dZtYEjAUakl5zFkHAJ7vTzDqBXwFXpwQ9AGZ2MXAxwCcvu5pjTp4WocgykOSlU+Oa7L6yehTNW5uA2iGfKxeTvGtieckyZW0JyUYGD0W28k85K3GL2t1ie/hno5kdTlBz2S/CcdbLttSA7fc1ZvZuYKu7/yNp/znu/g7gg+Hj3N4u7u6z3P0odz9KoZ0Z2Vo6tXx4DS3bMtPdQqQEKGtLhJavFsmeqJXkWWa2C/B1YD7BLbxrIxy3Gtgn6fl4YE1frzGzCoIK+Mak/WeTcvvP3d8M/2wG5hDcapQs6L50anyT3VdUj6J5qyrJIhEpa0tEtjJYRKIvJnKHu29y90XufoC77+7ut0Y4tB6YaGb7m9kwghCen/Ka+cB54c9nAI8nbueZWRlwJrAjBcyswszqwp8rgSnAP5DYZXPp1KrqGppaOzJ+XpEipawtAVq+WiS7IlWSzex7ZjYm6fkuZnb1QMe5ewcwg6D/8ovAXHdfZmbfNrOTw5f9DBhrZi8DlwJXJJ3iGGB1YjBKqAp4xMyWEMzd/CZwe5T3IUOTzaVTrbyMrjiW3RMpQsra0lCsy1eL5KuoA/dOcPevJZ64+yYzO5Gg+0W/3P0h4KGUbd9I+rmVoAWjt2OfAN6Tsm0LwTyfkmUrnn+S59e3cv+S1d2216x9Mi8Hj4iUEmVt8VMGi2RX1EpyuZlVuXsbgJlVE7QySAkpxqVTRUQKhTJYJLuiVpLvAf5gZncSjIb+FHB3bKUSEREREcmhqAP3rgW+CxwCHAZ8J9wm0qfmxo3cftWFGlQiIpJlDY0tnH7FrWxo2pLroogUrKhTwOHuD7v7f7v7Ze4eZSERKXHJE96LiEj2zH5wMZvWrtISzSJDEHV2i9PMbIWZNZnZZjNrNrPNcRdOCpcmvBcRyY2GxhYWLKxn5ml1LFhYr9ZkkUGK2pJ8LXCyu9e6+2h3H+Xuo+MsmBQ2TXgvIpIbsx9czJQJZRy0exVTJpSpNVlkkKIO3Fvn7i/GWhIpGolW5G+etXPC+0/cP4/JJ55NTe0u3V77/RnTaGlp7nGOyvIyjvz0B7NSXhGRYtHa2sqCv9Yzd+ooAKZPGsnUufWcN+X9jK0dueN1ky+5hYbmnquaVpV1csLn3pW18orks6iV5GfM7H7gN8CO/1Xu/utYSiUFrbcJ76cc0MXNXzqTGTf+Enfnvh9ezrSv/IiWlmYOuOimHud46ZaLsl1sEZGCt3L5Pzl7Qhl1NcGv97qaCk44AD76hRt47KYv4+585pp7WNe4lXd+9sYexz970+d7bCsvL2d7Z1fsZRfJN1EryaOBrcBxSdscUCVZeuhtwvvWLS1sb9u6o9uFBvSJiGTeurfeZM6GNuYsXb9j2+YtrWxrbdvR7WLT2lW0b4366x+GDR9BS2tHxssqku8i/S9x9wviLogUj9QJ75sbN3LPlWdzy5SRXDL/Abq64LZTx/H5BfPo6tzZLX7tqpV0dnaGx2zis9f9mstu/R11o6p4embP1g0REenuwx/7T+761Dt3PG9obGHqV37MzCl7cPFvn6bLnTtOq+Njs96kfWszw0aM4sU31tPR6QD8q3ELP//hNxhWPZKamlFcefMvqKquplmVZClBUWe3GG9m88xsvZmtM7Nfmdn4uAsnxSF5EN9x47YwftjmHQP6urY27nhdZ2cnVXX7UlW3LxUjRrPvxz7FYZ++rtd+cyIiMrDkQXwfGtdOXWUrB+1exSkHl7Pu2UcB6Oh0qncbT/Vu46kYMYq9T/g8B1x0047xIlXVI2lu3Z7LtyGSE1Hvt9wJzAHODJ9/Mtz20TgKJcUjeRBfZ2cnp+y/ncWvtLJxy3amTarlzkWr2b61icoRtbkuqohIUUlMBTd36ii2d3Rx4v6dLHyljQ1bOjn9kGE8/MdFtB953IDnqRpezYL65WxrG1pFecZ/HkHdmJohnUMkm6JWkndz9zuTnt9lZl+Ko0BSXJIH8TVv/Bf77VLGyQdVMrd+PZ/993Gcekgl/3fbZ6kYVcemhvVU1uwKQNmw4TkuuYhIYUu0ItfVVLB2w2b226WcUw6q5O76Rk7Yv4IT9tzA3Fu/zMY2o7ImmHmoclhVj/NUVA7jtG/eHYxEGqQVS+pZ+ko9Hz7y7YM/iUiWRa0kN5jZJ4FfhM+nARviKZIUk+RBfM2bGvCuoM9xh7dz7zIHajngwH35zLX3cNX5U3bMdLHlzRXQ/HoOSy4iUtieeG45a9YHg/jWb2qmM5yhoot25iwbBVRz9IQ6/rGuncM+fS0AG197kU29NBj3VnlOx4hRo2lqUr9mKSxRK8mfAm4GbiD4Lrk43CZFpK85ixODNwYjdRCf5IeGxhY+c809zLry3G5zp4pIbmUyh+dfNyPS6w745PVpnXcwqoaPoGVtcVeSlavFJ+rsFm8AJ8dclpIXRyU1HX3NWbzyji/Efm0I3mfiWtvbtjGsYysb/nwfdaOG1oIhPc1+cDGb1q7i7gVPcuk5A/dJFClF9912Pb9sb6GqqnsGxTnjTi5yuG5UFctuvwwIFiPZ6sNoCGe3yJSqESOKfoYM5WrxiVRJNrO7gS+6e2P4fBfgOndXa3IG5bqSmmvJXwReX/4Pjtz4MKd+4NAclqg4JQbzzDytjksW9FyJS0QC27Zt452nf4kx4yd0256oUEbV5Y6ZZbJoGZVc4f/z31/h19vexSFHZXbF02FV1bQU8QwZytXiFGkKOOCdiQoygLtvArRupUiaGhpbOP2KW9nQtGVQ+zMheUqoKRPKdiwwICLdmZXRuX3oU1BuaetkWFVpD0a2sjKytWZffzkaV8YqV4tT1EpyWdh6DICZ7Ur0/swiJS05lJNvx/VmoP2ZKMuChfVMnxS0cEyfNJIFC+tjrZSLFCqzMjraW4d8npbW7VRVq1UxTlFzNo6MVa4Wr6iV5OuAxWb2HTP7NsHAvWvjK5b0prlxI7dfdSEtTZtyXRRJQyKUf/rAH3fcjustQJNv18UVsMlTQgHU1VSo1UOkD1ZWRmd7e6/70mmR3Lytnarq6kwXT5JEydm4Mla5WryiDtybbWbPAMcCBpzm7i/EWjLpof7h+6lYt5SnH7qPY6ddkvHzJw+cS91eDLI98rihsYULvnM3jZs2csdpdZw15y+cfviI8HZca4/BHd1v1/XcnwnJU0Il23vdcg00EUlhZrRta+7RmtzV1cXtv1lEw5o3mDVvIf911kf6PU9jSyu7VUWrJBdTDjc3buS+H17OtK/8CCuL2iaXnnRyNq6MVa4Wr8hdJsJKsSrGMeovHBMr191y6jg+v2Aek088m5raXXo5y+BlYwaNXMr2yOPZDy6mYc3r7Dm6kgN23YWP7LMdOoOBK9MnjWTq3J2DO5JXxuptf6ZEnRJKRGB07S68vmgebyz8dbftlZUV/HT+U5x2yEh+Ov8p/tG1L8OH910JLt/lUMrKyyNds5hyOLlh591TPhHLNaLmrLvHlrHK1eKlfsV5pL9wfPwXMzlpIkzYvZqTJm6JrTW5WGV75HFDYwu/ffxpvnFMBdf8uY0Vb23mU+8axhce3sLnPtDZ7Xbcpecc1+/tOrVEiOTG1356f6/bH//FTN7+1jxmfLCOMbUNLK8czXumKo+TpTbsHH7MiRmvcKSTs4AyVtKmSnIBSITNN8+qBWDapFo+cX88rcnFKhtdGVKv96Fx7bxrrwpOOci557kmPvnOSv5tDzj6ptXsOipodUrcjtPtOpHCoDyOpv7h+7s17Dz32K+ZPCaz10gnZwFlrKRNleQCkAibsSMrgeDPkyai1uSIhtKVYTD9mBPX+/4HYOO2Lt493jh33nYeer2KivKRHH5gXY/bc7pdJ1IYlMcD6+2LxNQ5Czj8w0f1e1w6eTuYnBVJlyrJBWDF80/y/PpW7l+yutv2mrVPxhrKuV4BMFOG0pVhMP2YE9d799v33LFtRkMTjDtSLRYiBS6beZyLDHaHYHz+4PX2ReL4A+DZZSuAY/o8Lp28Vc5KNqiSXAA+c+09OblusawAONiuDIPtx6yuEyLFK5t5PJgM9iFes62jg4rKqoFf2I/evkh0dXVR5m/1eUy6eauclWxQJbkEJE/DU4p95gZ7y22w/Zh1i09EepONLLbyYbRv72BY5eB+vW/e1smwsUOb07m3LxJdXV28OueKPo9JN2+Vs5INqiSXgKjzK6fe2tvUsJ4lN19C+fARHHbRddkoat7I1pRsIlI6omRxcg4nMhiInMOV1TVs2dY+6Epyc+t2hsewOmBZWRmd3vtcycpbyVfxzO6dxMyON7OXzOxlM+vxNdLMqszs/nD/U2a2X7h9PzPbZmZ/Cx+3Jh1zpJktDY/5iZkNrQNVEUsMoLju1HG8tGhev6v1JW7tJR57nn01e59/I52tW7NY4vygFZSk0Chr81vULE7O4UQGp5PDlcNH0LKtbfDl3NbBsH7mfB6Krj76OitvJV/F2pJsZuXALcBHgdVAvZnNT1mt70Jgk7tPMLOzgR8AZ4X7XnH3I3o59UzgYuCvwEPA8cDDMb2NgpY6DY9GYEej/m5SSJS1+S9bWTxyt/F855fzqRk5uIruijfWcdy/x7O632sbWrn0zp4V39/+7mm2tLRw7cKmbttH1jzN6na1JEt8jpzcyTkHHtvn/ri7W0wGXnb3lQBmdh9wCt1X7jsF+Fb48wPAzf21VpjZXsBod/9L+Hw28HEU3D0MdT7P8vJy2hreYHvLxm4DRQpxedR0qb+bFBhlbR4bbBYnMhjolsP9ZfDbj/4wHP3hQZf1kEEfObDjv3RD79c8N8aLivRj3736r8/EXUkeB6xKer4aeHdfr3H3DjNrAsaG+/Y3s+eBzcDX3f1P4euT595ZHW6TFEOdz3PPfQ4AoK1ud75714JYyyoiQ6KszWODzeJEBoNyWCQX4q4k99ZKkTpDTV+veQvY1903mNmRwG/M7LCI5wxObHYxwa1CPnnZ1Rxz8rTIBS8GuZpfWUSyTlmbx5TFIoUp7kryamCfpOfjgTV9vGa1mVUAtcBGd3egDcDdnzWzV4C3h68fP8A5CY+bBcwCuH3RyqFOH1lw0p3Ps6ZmVK/zb5ZC94pMG8xKfSJDoKzNY+lksXI4PaU+xanEK+5Kcj0w0cz2B94EzgY+kfKa+cB5wF+AM4DH3d3NbDeCAO80swOAicBKd99oZs1m9h7gKWA60HO2dUlbPq2iV2ZldHQV7u/awazUJzIEytoikU85XAiiTnEqMhixTgHn7h3ADOAR4EVgrrsvM7Nvm9nJ4ct+Bow1s5eBS4HE1EXHAEvM7O8Eg0w+6+4bw32XAHcALwOvoIEkRaeqegQtrR25LsagJK8ctWBhPRuatuS6SFLklLVSitKZ4lRkMGJfTMTdHyKYOih52zeSfm4FzuzluF8Bv+rjnM8Ah2e2pJJPqqpHsnlre66LMSiDXalPZCiUtVJqNMWpxE0r7smQpa7Ul1BTM2rQtw6HjxjJpm3bh1q0rNPKUSKSbXFkcL4b6hSnIlGokpyGUgyiKBIrRKXqbfBJVMOGV9PSWniV5P5WjlJrskh8Sjmf48jghE3/Wkvzpg1DPk+m7LHP/lRVjxjyFKciUaiSnIY4g0i6MzM6vfBWwNVKfSK5oXyOx5IHbuDUI8YO/MIs2Lh5C8+9PIFJJ56rafUkK1RJlrzlVp7rIqRNK/WJSDEZU1PN6R88LNfFAGDT5q08/btgTGm6U5yKDEass1uIDEWX/nmKiORUGV25LsIOI6uHsX1bS66LISVEtRDJW5365ykiklP5VEkeVlmBdxbmrEdSmNTdQoYsrhWiOntdFVdERJLFuUqf5VElGaC895XRRWKhSnIatFxo7+IbOa5KsohEU8r5HOfsHfl2P0+/FSSbVElOQ7FPIyQiUqiUzyKSafn2JVFEREREJOfUklzkSnmCfRGRfKEsFik8qiQXoeQw3tSwnj3PvhqA8vJy9tznACC/Jtjv65eHbd3ILRccnYMSiYhkRiLfkrMYduZxLrNYFXeR/qmSXISSV55acvMlVNXtC0Bbwxu5LFaf+lop64UfTs1BaUREMieRb8lZDPmRx1qlUKR/6pMsIiIiIpJCLcmSV5bdcRmdrVsBaG3ZQtXx3wKgzDvZa7ddAKgbVcXTMz+fqyKKiBSdRPa2t2zCuzr43JTJAHhrC/P+8JRyV0qSKsmSVzpbt7L3+TcCsO7hmxl/4iUArLrzyxz26WsBWHb7ZTkrn4hIMUpk77b1b1AxZg8qh1UB8MZNn+SwT1+n3JWSpEpykSsfPoI1d30JgO0tG2mr2x0ojQn2RUTyRXIWw848zvcs1vp2UspUSS4gUUciJ688VQ0wPPhrrqk7MO0RywNdMxOjo5PLu71lY14MaBER6U+6eZycxRA9j+PM4N6y17s6tKqdSEiV5AISdSRyJqfuGeiamRgdnVzeq86fwrj9JgKw+VmNKxWR/JStPI4zg3vL3jdfW0FF2NVCpNSpklyE/uvkd9PR1fMmWUWZ8ZP5T+WgRCIipUdZLFLYVEkuMt+fMY32jg7Gf/bn3bYb8OasT+emUGlIvv23Zc1rrFzxFyCY3SIxcKRulFo5RCT/be/oZNxnf9Zj+5u3XpiD0vQvkb1NGxro8q4d27vatvDUDZ9lVHUlcx5fumP7f06eQG1NdS6KKpI1qiTniUytfNTS0oxZOVYxrNt272gfchmzIfm9/u3/vqUV90QkJzKRyY73yOLE9nzT13vavKmBlqZNAKwIt61+5Z+MXvoSU957SJZKJ5IbqiTnCa181POXUuumdTz8hz9pfk4RybpSyuSBvhCM3qWu2/auzi5a1ryQreKJ5IwqyQUkuStC6vZMSg7Mpg0NPHvNWQCYdzFmtz27XTMTZUpcb1PDeiprdt2xvYsKzc8pInkp7jxO5GJyBsPOHM5UBn9/xjRWvfZKt+yFYMo6eqk4A1RVV9PSuj3qWxEpWKokF5BMzlrRn/5aUL5714KMlik5oG34KDo9uA1ZNmwEnc0NAHR2dnL6Fbcy68pzGVs7ckjXExHJhLjzOGoOD7YcyY0Tqdk77pzvs+auL9FVCbdfdSHTvvIjamp32XFs1fARtLR2DOq6IoVEleQC1dftscZ/rQWM1bec1227eyfDKqL9dTf+ay1Lbr6kx/aOsNKaSS0tzex59tVU1e3L9va2Hf333rp756T77Vtb2LS2kbsXPMml5xyX8TKIiAxFX3nsHdt7ZDEAnQNXMOPO4UQl/M3XVlA2eo9es7drayMV6zby9EP3cey0nWUZNryaRrUkSwlQJblA9dXK8PwPprHr2Loe29MabGJlO5aGTvb6zdN3/JypgYYJrQ2r6erq3PG8s2UjXW3b+NOtVzGmcwszTxvHJQvqOW/K+9WaLCJ5pa88bvrBNGr7yOOBDJTDmcrgzo7tdDau3fm8ZSOr7vwiHZvXU1fVxnWnHsTnF8xj8oln72hNLisvp6uXqe1Eio0qyTmUTt/fqGrH1vXoEjFYnVub2Py76xl9/GWUjxjdbV+6g1r6CvSmDQ3sDnhXF5Vjx2MWLCBSXrMr1RPfQ+erf+WUQ8o5aPcqTjhgKx/9wg08dtOXVVEWkYzLdCZnIo/7yuF0Mri3/N3UsJ61q1YC1iN7hTJD/QAAGlVJREFU9zrvRtb95ExOObiCCbtXM+WAZm7+0pnMuPGX3bpdiBQ7VZJzKJ2+v31ZdsdldLZu3fF8e8tGrjp/yqBbdJO1/eMR9m17mXX/+B0jJk/dsf37M6axqWF9j1uB5cNH0NesmX29124DUgBPzM/pjm9vo7ZrM2cdHvxSOvFAuO3JDfz0gT/yPxdOGdJ7ExFJNdRMjiOPe8vhdDO4t/e15OZL6OwM7t6lZm/H5gZ2rWznzMODCvHJE7q4d/Gb/OnXd3LCBZcO6n2IFCJVkgtcZ+vWbrfk2hreYNx+E4c0TVGZlbF19QtUvvQI/3virnzhoUfYuvfhlIUtDcn9iJOtuetLMDz9f1Ll5eV0drThSf30vKsTGldzxmGVvO/gPdje0UVFZysXHTmMu3+3mM+d8WG1JotIXslkHveXw5nO4NTsbX9+HmceXMkhBx1IZ2cnIzqb+dSkKu545Bd88LQLGDZci4hIaYi9kmxmxwM/BsqBO9z9mpT9VcBs4EhgA3CWu79mZh8FrgGGAe3A5e7+eHjME8BewLbwNMe5+/q430upqB1bx4iGf3LyoVVM3LuW0w/tZH7DP+nopW/dUJl3sfWRG3pM/1ZVU4ttXsvcZdt5ct16Nm9phY52Rg83qujUID6RFMra4hJ3DpcPH8Ha+74O0CN7Kxrf4FdruvjDLatp3dJCRedWRg8vo8a6ePqh+/jAaRdkpAwi+S7WSrKZlQO3AB8FVgP1Zjbf3ZNnIb8Q2OTuE8zsbOAHwFlAA3CSu68xs8OBR4BxSced4+7PxFn+gQxl4ETidtmbr63otr28vDzStRr/tZZnrzkL7+qireGNAY9PR/Xw4bT/9R5OOnUEbf9q4aTxXTww7x6q9zy0z2PevPdK2jf/i7bN8Lkpk3dsN+9i/P4T+zxuzG578t27FnDV+VN63A5c//jPYeVifnfTxUz9yo+ZO3UUdTUVNLR0MHWuBvGJJBR71qZrsNnctKGhRyYntvd3jUQed3V1snXtSsrDmYSGksf95fC21tYer19192V0NPXM4IoyY9Suu/V4/WEXXbejhbuvLiYX3zibe648mzln1TJ2ZCUbtmznE/fP44iPfHzQ70ukkMTdkjwZeNndVwKY2X3AKUBycJ8CfCv8+QHgZjMzd38+6TXLgOFmVuXubTGXuYf+plubdOUve2yPcmutpaWZyppde9wuS67wpr6+rz694/bruxI6GO/70Ed4+1ubqatppr2tldGVxkkTjZ8v/SefmzIZ7+qi4qEfs9uJX9xxTOe2zewx9dsM23U8FcOqdmxfc9eXev3sUvU2IX7rpnUcvO9uzH5wMVMmlFFXE/xzraupYMqEMrUmi+xUFFmbrkxnc5d39cjkxPZUffXzLa+oyEgmJ3J49LAGOju6GF3JjhxuL6uiYsGN7D5l53Rtvr2VPaZ9j2Fj9uyWwcmzEv3/9u49zK26zuP4+5uZaUtpt5QO0FIuLVpdwUUFwV1XEdcV0adQXVEpXiiiFaSu+uC69Kn6sCwK1QcvWNfVclPktuqCXS6Wi7o8sqJAgUIX1kIpdAq9TNtpOwxzS777xzmZJpmcTGYmJzlJPq/nmWcyJycn35w5+eSX37n8iik1GMlDd93CafNgxv5tQPD7tHnw8Oqf8/rho22LNJy4G8mzgU05f3cAb4max90HzWw3MIOgdyPrg8CjBaF9rZmlgV8Cl7r7sOvRmNliYDHAxy68lJNOXzimF1HOSWdj0TJpcnAMWY6B7p0cPudV41rueK1/9AEe3dbLj37bRcadlv0PBCYxZfZMpn3oG/Rs2UDXr6/M+yDY0doGWF445xppVKhivTuPXX8xPzjnBE6/cAUvbuvjxify9/IeuvXPaiSLBBoia0er0tlsnhmWydnp1ZbN4R/e203L1OwhFkEOT3jHeWxftTwvg7cCFpHBpfK3VM/6j778MR7d1sstazvypk9u/wOvP2HumF6XSD2Ju5FsRaYVBmzJeczsGILdgrmtoY+6+2Yzm0oQ3B8nONYufyHuPwZ+DLDy/g2Ju6jjMZ+6Yti0DVd9rqyzoLNnUWcG+vM+EFKWYtqM9nENjfqZb/4MgGWL5tPdO1jkWp3OQPfOvNAd6N7J8H/tPuO50saqK5aM+bEiTUJZWwEHHDRzVJe2zMrmcf+e7bx441K2pILDLMaTx9kc/uz8EznynGvy7uvt7CDd05VXV7qnC0ulii5rrPmbraFQf18vnbddMqZlitSTuBvJHcDhOX8fBrwYMU+HmbUC04CdAGZ2GHAr8Al3fzb7AHffHP7ea2Y3EuxqHBbcjSzd24O37UfL1Bm0trYNTR/o3lmRy7+V0tLaxvT2g/MuibRs0XxacuqoBF2rXqRsytoa2pfH7Xh6cCiT48rjSe2HkUq15GXwZ+efyKT2wyr6PCLNLu5G8kPAPDObC2wGzgTOKphnFXA28AfgDOA37u5mdgBwB7DU3R/IzhyG+wHu3mlmbcB84N6YX0ciZfp7mPWJb9OWs3ut4ycXsmnjsyxblH8d4bgbznFwG/9JiCJNQllbY5n+Hmad/V18sJ+2CRPZfMNSbNLUYXlcj1ks0qxibSSHx70tIThbugW4xt3XmdklwMPuvgq4GrjezJ4h6NU4M3z4EuDVwFfN7KvhtFOAl4HVYWi3EIT2yjhfR5SUpUoeZ1vKSMfojjT/QPdObNLUYftPMwO9zDzz0mEnjoznusm5Nt+wlPQre/DevUPBn0kP8sq25+m66St4QUHmGaa0j+0klsHChYlIUY2etaM11mweTS7nzpvNYx/sH8rkbCdGZs/WvDwebxZvvmEpmf4eMpl0Xga39e1m4/c/NjRyXlZrSjkqMlaxXyfZ3e8E7iyY9rWc273Ah4o87lLg0ojFHl/JGkcSFZyzj5wb2SMw0iWIRtuTUDh/9njhqBPlKmHKlKl0bV+fd3Z0JpPmkA//KxMmTmTm4UcB0Pn7mzhs4Ke8bsE5/N3C86MWN2oZih9fJyLDNULWjtZos/myJQuH7WXLLme0uZw7bzaP22LI49aUFc1gAw5/9euAbAZv5HULzq1oBos0O424V4ax7BorNbxpUoylIb9s0XyOCIMZYODl3fjT93DxO/fnkvtv5X/++76i1/Acyy7GVbffwdz77sUsvyekfepE/vTDC0a1LBFpPKPNlCTm8kg5fOWqP+ZNr0YGj2cMAJFGokZynZoyZSq7Op8ddl3lwgZlKZX4wNj96J0smGfMmd7CafPg+w908JovDg/RsXwI9fUPcuJnv0NLW/4FOdetvHDUyxIRiUuxPPb0YNHLiRQabw7HkcFJ/DIhUgtqJNeppStuCr7t33tl3vTMK3vGPepe1/Ytkbskc2V7MM44Yxr0dLPwuGlce38HAz27aZs8bVw1AFgqxWDfK8MaySIiSVIsj713L+k9W8ecx1E5nDv6X9wZLNLs1EiuY1HH3HWv/g4bCqaP5jqdbqmyehGyPRjT92+lrycYjWnBX7Zy15o7aX9b6cEEytmdZ5Yi3d9LcKUqEZHkKszj3EZzbh6Xm8VROZx7XfyxZrAOpxApjxrJDWakgMuGY9f2LaTd2XlZGLhmTJgynZZJk0s+Pu+M7s7nudEGufHBlzAzph3YQdeOAVKHroERGsll7c4Le5JFROpNOVnc8dz6/BwGMGO/9tmRjzPPjDuDdTiFSHnUSI7JaC/xVmlRPQVd27dw3NKfs3bF+cw6aznWGhzKMNC5if0OPqLokKy5Rgr+ZYvmM+fj3xp74TnMWkgP9I08o4hIGWqRy6WyuHVqe14OQ5DFu+78duTyDjhoZt4gIoUqmcEizU6N5JjUepdVVE/Bw18/g7Urzqd/704GurbsuyM9SF/nCwx07yRlY7/0WiU/hCbttx8bfnUlkyZNypvePjW+y96JSOOqRS4Xy+Itmzaw4/ovkynIYTPDM4PjyuFKZHCtO3lEkkKN5Aqp5TFexZ57V+c21l11Icd86oq86W5w6KLvsunazzOh/Yih6f3bnmP2nHn0tR88rloq+VrPWnIRZ0xZy1v/6qiKLVNEmks1s7ncLE6n07RMPgBrac3L4YHOTbS0tjJ9HDlciddU604ekaRQI7lCCnsL1l11IeneHnZ1xj8kaXf3Xia/54uk0+mhaQcNDtJ5+xVs2bRhaNCPcqkXQUQaRW42Z3MZyMvmSuVy9rm2bNowlMfZLN68cT0tLS1l57FyWKT2mqaRHHdDtVC6t4dDF32Xvs4XKjokaeTzpdNMzOmReGXbC1hLa17DuVxDlzMq6BHp7t7LZUsWlrXuRuq90QeAiFR7D1w2l4G8bK50LufmcTaLJ7YfMey69nEqtW6VvyLlaZpGcm4vb6Oewbv5hqVk+oNeEk8PknllD1tvW07XlGlDu/oMo6/zBSzVyovXLBl6bKZnN4MHzxwKyfGe/TzS47U7T0Qa9SoL2+/4Hp4ZBPZl8aZrP4+lWpl9wQogOP64MIfTPV20trQx+8i5sWZwqRP/RGSfpmkkN4NMfw+zzg56SXywn4GdmwHY/ouLh4I1ZTB7zryhoM5ScIqIVEZmoJdDPxlkbDaLLdXKlpuXsuGqz7GrcxuT2g8bds5Ibg4XG0hERKpLjeQGZa0TsFQrqRRMbz94KHgvW7JQu9lERKokm8X7HXwEqVQLX7/u9qFDIQqzWDkskixqJFdI4TFeA9076et8YVRDko71+LwpU6ay6eav4B6cHZ1lqRSQyZt3vIc57N7RGTlktQ6hEJGkyRsAKcxloKxsHm0mZ58r3dNVJIv3GU9WKoNFqkeN5AqpxJCkYz0GLXui3aaNz5KfxZlRNdLLkfFMQx5DKCKNKTebc3MZ9mVzVC6PNpOzz7XktL8elsV9nS+M6xr0Q0tSBotUTdM0knMDpBq7tKr9jX7piptYtmh+3pU0sjYUmX8kUWc/m2eKzL1Ptudl945OHrl831Cr5hkOOGimdieKyJBqX2WhWrk8bUZ70Szum9Fe9jKUwSK11zSN5GY4Ka2SHzhRHyYjnUyiM6pFpFyNenhAJbJYGSxSe03TSG4GpT5wajkioIhIM1EWizQGNZKLaMQQa9TrkYpIc6n3fFYWi9QPNZKLqFWI1cMoSPVQo4g0rmrmcxLzLok1iTQqNZITpFgvSLbXpPA4tFr1mtRDT42ISCUkMZOVwSLVo0ZywtXbrjn1cohII0t6JiuDRSpHjeQqqvdj6crRKK9DRJpDo+VyPdYsklRqJFdRLXsg4updaLQPGBFpLtXO5UpnsTJYJD5qJBfRiLur4grLpO96FJHGUu/5XOksVgaLxEeN5CKa8du3eiNEpB40ch4ph0WSRY3khKtWr4l6I0RERhZnJiuHRZJFjeSEU++BiEhyKJNFmocayVVU78fSiYg0GuWyiERRI7mKGrEHQh8wIlLP6j2XlcEi8Ym9kWxmpwLfA1qAq9z98oL7JwI/BY4HdgAfcfeN4X1LgXOBNPCP7r66nGU2kqSfyJGEGkREWVsNScxjZbBIfGJtJJtZC/AD4N1AB/CQma1y9//Nme1cYJe7v9rMzgSWAx8xs6OBM4FjgEOBe83sNeFjRlpmw6jWiRzqjRCpX8ra6og7j5XDIskSd0/yicAz7r4BwMxuBhYAuSG7ALg4vP0LYIWZWTj9ZnfvA54zs2fC5VHGMmWU1BshUteUtQ1AOSySLHE3kmcDm3L+7gDeEjWPuw+a2W5gRjj9wYLHzg5vj7RMAMxsMbA4/PMz7v7jMbyGijOzxeXWct7WlzZO3tPbWTh9x9aX2j990lFzallb7E46auhmouoqkNTakloXJLe2pNZVhrrM2iSt73JqqVYe18V6effNyamlBlRLcY1WS9yNZCsyzcucJ2p6qoxlBhODlZOIf1aBxZRZV/rlXXOK3nH5rgqWk6fs2qosqXVBcmtLal2Q3NqSWtdI6jVrk7S+R6ylinlcV+ulilRLcaqluHHXUiwEK6kDODzn78OAF6PmMbNWYBqws8Rjy1mmiEgzUdaKiFRY3I3kh4B5ZjbXzCYQnByyqmCeVcDZ4e0zgN+4u4fTzzSziWY2F5gH/KnMZYqINBNlrYhIhcV6uEV43NsSYDXBJYSucfd1ZnYJ8LC7rwKuBq4PTxbZSRDEhPP9B8FJIoPABe6eBii2zDhfRwySsiuimKTWltS6ILm1JbUuSG5tSa2rpDrO2iStb9VSnGopTrUU11C1WNCRICIiIiIiWXEfbiEiIiIiUnfUSBYRERERKaBGcgWZ2TVmts3MnsyZdqCZ3WNm68Pf0yMee3Y4z3ozO7vYPDHU9i0ze9rM1prZrWZ2QMRjN5rZE2b2mJk9XIW6LjazzeHzPWZm74t47Klm9n9m9oyZXVTJukrUdktOXRvN7LGIx8a5zg43s9+a2VNmts7MPh9Or+m2VqKuJGxnUbUlYltrZGb22pz1+5iZ7TGzLxTMc7KZ7c6Z52sVfP7E5HKScjhJ2ZuUrE1StiYpT5OUnyVqiWd7cXf9VOgHOAk4DngyZ9o3gYvC2xcBy4s87kBgQ/h7enh7ehVqOwVoDW8vL1ZbeN9GoL2K6+xi4EsjPK4FeBY4CpgAPA4cHXdtBfdfAXytButsFnBceHsq8Gfg6FpvayXqSsJ2FlVbIra1ZvkJ1+UW4MiC6ScDt8f0nInJ5STlcJKyNylZm6RsTVKeJik/o2qJa3tRT3IFufv9BGeN51oA/CS8/RPg/UUe+h7gHnff6e67gHuAU+Ouzd3vdvfB8M8HCa6DWlUR66wcQ8Pwuns/kB0ytyq1mZkBHwaqPo6su7/k7mvC23uBpwhGSKvpthZVV0K2s6h1Vo7Yt7Um8i7gWXd/vlpPmKRcTlIOJyl7k5K1ScrWJOVpkvJzpFoqvb2okRy/Q9z9JQj+ucDBReYpNqRsuRtgpXwSuCviPgfuNrNHLBh+thqWhLuTronYtVXrdfZ2YKu7r4+4vyrrzMzmAG8C/kiCtrWCunLVfDsrUlvSt7VGcibRH15/Y2aPm9ldZnZMzHUk5r1SoObvD5L3fqhJ1iYpW5OUp0nKz4j1UtHtRY3kZChnSNn4ntxsGcH1UW+ImOVv3f044L3ABWZ2Uswl/RB4FfBG4CWCXSeFarrOgIWU/qYa+zozsynAL4EvuPuech9WZFpF11tUXUnYzorUVg/bWkOwYECS04GfF7l7DcEhGG8Avg/cVs3aIlT1/56E9wfJfD9UPWuTlK1JytMk5WeJ/1FFtxc1kuO31cxmAYS/txWZp2bDv4YnF8wHPurhATuF3P3F8Pc24FaC3Sexcfet7p529wywMuL5arnOWoF/AG6JmifudWZmbQQBcYO7/2c4uebbWkRdidjOitWW9G2twbwXWOPuWwvvcPc97t4d3r4TaDOz9hhrqfl7JVcS3h/hshP1fqhF1iYpW5OUp0nKzxLrpeLbixrJ8csdCvZs4FdF5lkNnGJm08PdFaeE02JlZqcC/wyc7u49EfPsb2ZTs7fD2p4sNm8F65qV8+cHIp6vlkPm/j3wtLt3FLsz7nUWHnN1NfCUu387566abmtRdSVhOytRW9K3tUYS2cNjZjPD/xFmdiLBZ9OOGGtJTC4n4f2R8zxJez9UNWuTlK1JytMk5WeJ/xHEsb14Bc581M/QWZM3EexyGCD49nQuMAO4D1gf/j4wnPfNwFU5j/0k8Ez4c06VanuG4Fihx8Kffw/nPRS4M7x9FMHZqI8D64BlVajreuAJYC3Bm2lWYV3h3+8jOLP12UrXFVVbOP064LyCeau5zt5GsLtqbc7/7n213tZK1JWE7SyqtkRsa43+A0wmaPROy5l2XvZ9BCwJ/++PE5yM9NYKPndicjmilpq8PyJqqcn7oVgt4fTrqGLWlsiJqm8vJWqp+vZSopaqby9RtcS1vWhYahERERGRAjrcQkRERESkgBrJIiIiIiIF1EgWERERESmgRrKIiIiISAE1kkVERERECqiRLCIiIiJSQI1kaUpmdrKZ3R7ePt3MLqp1TSIijUZZK/WstdYFiFRSOBqPeTBMZlncfRUaQU1EpGzKWmkG6kmWumdmc8zsKTP7N2ANcLWZPWxm68zsX3LmO9XMnjaz3xOM756dvsjMVoS3rzOzM3Lu6w5/zzKz+83sMTN70szeXqKebjNbbmaPmNm9Znaimf3OzDaY2enhPC1m9i0ze8jM1prZZ8LpU8zsPjNbY2ZPmNmCgte4Mnxdd5vZfhVdkSIiJShrpdmokSyN4rXAT939TcCF7v5m4FjgHWZ2rJlNAlYCpwFvB2aOcvlnAavd/Y3AGwiGwoyyP/A7dz8e2AtcCrybYGz7S8J5zgV2u/sJwAnAp81sLtALfMDdjwPeCVwR9tgAzAN+4O7HAF3AB0f5GkRExktZK01Dh1tIo3je3R8Mb3/YzBYTbN+zgKMJvhA+5+7rAczsZ8DiUSz/IeAaM2sDbnP3UsHdD/w6vP0E0OfuA2b2BDAnnH4KcGxOT8o0gmDuAL5hZicBGWA2cEg4z3M5z/tIzrJERKpFWStNQz3J0iheBgh7CL4EvMvdjwXuACaF83gZyxkkfF+EvQoTANz9fuAkYDNwvZl9osQyBtw9+1wZoC9cRoZ9X0wN+Jy7vzH8mevudwMfBQ4Cjg97Urbm1N+X8xxp9CVXRKpPWStNQ41kaTR/QRDiu83sEOC94fSngblm9qrw74URj98IHB/eXgC0AZjZkcA2d18JXA0cN846VwPnh70lmNlrzGx/gl6ObWFvyDuBI8f5PCIicVDWSsPTtyNpKO7+uJk9CqwDNgAPhNN7w92Cd5hZJ/B74PVFFrES+JWZ/Qm4j7DXBDgZ+CczGwC6gVK9G+W4imAX3pqwF2U78H7gBuC/zOxhgmPxnh7n84iIVJyyVpqB7dtTISIiIiIioMMtRERERESG0eEWImNkZn8EJhZM/ri7P1GLekREGpGyVmpFh1uIiIiIiBTQ4RYiIiIiIgXUSBYRERERKaBGsoiIiIhIATWSRUREREQK/D/p3UrSxUYWJwAAAABJRU5ErkJggg==\n",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"from sklearn.linear_model import LogisticRegression\n",
"\n",
"# Instantiate logreg\n",
"logreg = LogisticRegression(random_state=1)\n",
"\n",
"# Fit logreg to the training set\n",
"logreg.fit(X_train, y_train)\n",
"\n",
"# Define a list called clfs containing the two classifiers logreg and dt\n",
"clfs = [logreg, dt]\n",
"\n",
"# Review the decision regions of the two classifier\n",
"plot_labeled_decision_regions(X_test, y_test, clfs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Classification tree Learning\n",
"- Building Blocks of a Decision-Tree\n",
" - Decision-Tree: data structure consisting of a hierarchy of nodes\n",
" - Node: question or prediction\n",
" - Three kinds of nodes\n",
" - **Root**: no parent node, question giving rise to two children nodes.\n",
" - **Internal node**: one parent node, question giving rise to two children nodes.\n",
" - **Leaf**: one parent node, no children nodes --> prediction.\n",
"- Information Gain (IG)\n",
"![information gain](./image/ig.png)\n",
"$$ IG(\\underbrace{f}_{\\text{feature}}, \\underbrace{sp}_{\\text{split-point}} ) = I(\\text{parent}) - \\big( \\frac{N_{\\text{left}}}{N}I(\\text{left}) + \\frac{N_{\\text{right}}}{N}I(\\text{right}) \\big) $$\n",
" - Criteria to measure the impurity of a note $I(\\text{node})$:\n",
" - gini index\n",
" - entropy\n",
" - etc...\n",
"- Classification-Tree Learning\n",
" - Nodes are grown recursively.\n",
" - At each node, split the data based on:\n",
" - feature $f$ and split-point $sp$ to maximize $IG(\\text{node})$.\n",
" - If $IG(\\text{node}) = 0$, declare the node a leaf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Using entropy as a criterion\n",
"In this exercise, you'll train a classification tree on the Wisconsin Breast Cancer dataset using entropy as an information criterion. You'll do so using all the 30 features in the dataset, which is split into 80% train and 20% test.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"DecisionTreeClassifier(ccp_alpha=0.0, class_weight=None, criterion='entropy',\n",
" max_depth=8, max_features=None, max_leaf_nodes=None,\n",
" min_impurity_decrease=0.0, min_impurity_split=None,\n",
" min_samples_leaf=1, min_samples_split=2,\n",
" min_weight_fraction_leaf=0.0, presort='deprecated',\n",
" random_state=1, splitter='best')"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn.tree import DecisionTreeClassifier\n",
"\n",
"# Instantiate dt_entropy, set 'entropy' as the information criterion\n",
"dt_entropy = DecisionTreeClassifier(max_depth=8, criterion='entropy', random_state=1)\n",
"\n",
"# Fit dt_entropy to the training set\n",
"dt_entropy.fit(X_train, y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Entropy vs Gini index\n",
"In this exercise you'll compare the test set accuracy of dt_entropy to the accuracy of another tree named ```dt_gini```. The tree ```dt_gini``` was trained on the same dataset using the same parameters except for the information criterion which was set to the gini index using the keyword ```'gini'```."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"DecisionTreeClassifier(ccp_alpha=0.0, class_weight=None, criterion='gini',\n",
" max_depth=8, max_features=None, max_leaf_nodes=None,\n",
" min_impurity_decrease=0.0, min_impurity_split=None,\n",
" min_samples_leaf=1, min_samples_split=2,\n",
" min_weight_fraction_leaf=0.0, presort='deprecated',\n",
" random_state=1, splitter='best')"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dt_gini = DecisionTreeClassifier(max_depth=8, criterion='gini', random_state=1)\n",
"dt_gini.fit(X_train, y_train)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Accuracy achieved by using entropy: 0.8947368421052632\n",
"Accuracy achieved by using gini: 0.8859649122807017\n"
]
}
],
"source": [
"from sklearn.metrics import accuracy_score\n",
"\n",
"# Use dt_entropy to predict test set labels\n",
"y_pred = dt_entropy.predict(X_test)\n",
"y_pred_gini = dt_gini.predict(X_test)\n",
"\n",
"# Evaluate accuracy_entropy\n",
"accuracy_entropy = accuracy_score(y_test, y_pred)\n",
"accuracy_gini = accuracy_score(y_test, y_pred_gini)\n",
"\n",
"# Print accuracy_entropy\n",
"print(\"Accuracy achieved by using entropy: \", accuracy_entropy)\n",
"\n",
"# Print accuracy_gini\n",
"print(\"Accuracy achieved by using gini: \", accuracy_gini)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Decision tree for regression\n",
"- Information Criterion for Regression Tree\n",
"$$ I(\\text{node}) = \\underbrace{\\text{MSE}(\\text{node})}_{\\text{mean-squared-error}} = \\dfrac{1}{N_{\\text{node}}} \\sum_{i \\in \\text{node}} \\big(y^{(i)} - \\hat{y}_{\\text{node}} \\big)^2 $$\n",
"$$ \\underbrace{\\hat{y}_{\\text{node}}}_{\\text{mean-target-value}} = \\dfrac{1}{N_{\\text{node}}} \\sum_{i \\in \\text{node}}y^{(i)}$$\n",
"- Prediction\n",
"$$ \\hat{y}_{\\text{pred}}(\\text{leaf}) = \\dfrac{1}{N_{\\text{leaf}}} \\sum_{i \\in \\text{leaf}} y^{(i)}$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Train your first regression tree\n",
"In this exercise, you'll train a regression tree to predict the mpg (miles per gallon) consumption of cars in the [auto-mpg dataset](https://www.kaggle.com/uciml/autompg-dataset) using all the six available features.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Preprocess"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"