{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "8aec88d2-8c79-452c-927b-17253668a176",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"# Install the necessary dependencies\n",
"\n",
"import os\n",
"import sys\n",
"!{sys.executable} -m pip install --quiet seaborn pandas scikit-learn numpy matplotlib jupyterlab_myst ipython "
]
},
{
"cell_type": "markdown",
"id": "28bfa3a1",
"metadata": {
"tags": [
"remove-cell"
]
},
"source": [
"---\n",
"license:\n",
" code: MIT\n",
" content: CC-BY-4.0\n",
"github: https://github.com/ocademy-ai/machine-learning\n",
"venue: By Ocademy\n",
"open_access: true\n",
"bibliography:\n",
" - https://raw.githubusercontent.com/ocademy-ai/machine-learning/main/open-machine-learning-jupyter-book/references.bib\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "f6ca5fc4",
"metadata": {},
"source": [
"# Bagging"
]
},
{
"cell_type": "markdown",
"id": "0164eba3",
"metadata": {},
"source": [
"## Bootstrapping\n",
"\n",
"*Bagging* (also known as [Bootstrap aggregation](https://en.wikipedia.org/wiki/Bootstrap_aggregating)) is one of the first and most basic ensemble techniques. It was proposed by [Leo Breiman](https://en.wikipedia.org/wiki/Leo_Breiman) in 1994. Bagging is based on the statistical method of [bootstrapping](https://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29), which makes the evaluation of many statistics of complex models feasible.\n",
"\n",
"The bootstrap method goes as follows. Let there be a sample $\\large X$ of size $\\large N$. We can make a new sample from the original sample by drawing $\\large N$ elements from the latter randomly and uniformly, with replacement. In other words, we select a random element from the original sample of size $\\large N$ and do this $\\large N$ times. All elements are equally likely to be selected, thus each element is drawn with the equal probability $\\large \\frac{1}{N}$.\n",
"\n",
"Let's say we are drawing balls from a bag one at a time. At each step, the selected ball is put back into the bag so that the next selection is made equiprobably i.e. from the same number of balls $\\large N$. Note that, because we put the balls back, there may be duplicates in the new sample. Let's call this new sample $\\large X_1$.\n",
"\n",
"By repeating this procedure $\\large M$ times, we create $\\large M$ *bootstrap samples* $\\large X_1, \\dots, X_M$. In the end, we have a sufficient number of samples and can compute various statistics of the original distribution."
]
},
{
"cell_type": "markdown",
"id": "f878eb84",
"metadata": {},
"source": [
":::{figure} https://static-1300131294.cos.ap-shanghai.myqcloud.com/images/ml-advanced/ensemble-learning/bagging/Process_of_bootstrap.png\n",
"---\n",
"name: 'process of bootstrap'\n",
"width: 90%\n",
"---\n",
"Process of bootstrap\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "44186e16",
"metadata": {},
"source": [
"For our example, we'll use the familiar `telecom_churn` dataset. Previously, when we discussed feature importance, we saw that one of the most important features in this dataset is the number of calls to customer service. Let's visualize the data and look at the distribution of this feature."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "92d03822",
"metadata": {
"attributes": {
"classes": [
"code-cell"
],
"id": ""
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import seaborn as sns\n",
"from matplotlib import pyplot as plt"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "65b1b554",
"metadata": {},
"outputs": [],
"source": [
"# Graphics in retina format are more sharp and legible\n",
"%config InlineBackend.figure_format = 'retina'"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "c336331c",
"metadata": {
"attributes": {
"classes": [
"code-cell"
],
"id": ""
},
"tags": [
"output-scoll"
]
},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
State
\n",
"
Account length
\n",
"
Area code
\n",
"
International plan
\n",
"
Voice mail plan
\n",
"
Number vmail messages
\n",
"
Total day minutes
\n",
"
Total day calls
\n",
"
Total day charge
\n",
"
Total eve minutes
\n",
"
Total eve calls
\n",
"
Total eve charge
\n",
"
Total night minutes
\n",
"
Total night calls
\n",
"
Total night charge
\n",
"
Total intl minutes
\n",
"
Total intl calls
\n",
"
Total intl charge
\n",
"
Customer service calls
\n",
"
Churn
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
KS
\n",
"
128
\n",
"
415
\n",
"
No
\n",
"
Yes
\n",
"
25
\n",
"
265.1
\n",
"
110
\n",
"
45.07
\n",
"
197.4
\n",
"
99
\n",
"
16.78
\n",
"
244.7
\n",
"
91
\n",
"
11.01
\n",
"
10.0
\n",
"
3
\n",
"
2.70
\n",
"
1
\n",
"
False
\n",
"
\n",
"
\n",
"
1
\n",
"
OH
\n",
"
107
\n",
"
415
\n",
"
No
\n",
"
Yes
\n",
"
26
\n",
"
161.6
\n",
"
123
\n",
"
27.47
\n",
"
195.5
\n",
"
103
\n",
"
16.62
\n",
"
254.4
\n",
"
103
\n",
"
11.45
\n",
"
13.7
\n",
"
3
\n",
"
3.70
\n",
"
1
\n",
"
False
\n",
"
\n",
"
\n",
"
2
\n",
"
NJ
\n",
"
137
\n",
"
415
\n",
"
No
\n",
"
No
\n",
"
0
\n",
"
243.4
\n",
"
114
\n",
"
41.38
\n",
"
121.2
\n",
"
110
\n",
"
10.30
\n",
"
162.6
\n",
"
104
\n",
"
7.32
\n",
"
12.2
\n",
"
5
\n",
"
3.29
\n",
"
0
\n",
"
False
\n",
"
\n",
"
\n",
"
3
\n",
"
OH
\n",
"
84
\n",
"
408
\n",
"
Yes
\n",
"
No
\n",
"
0
\n",
"
299.4
\n",
"
71
\n",
"
50.90
\n",
"
61.9
\n",
"
88
\n",
"
5.26
\n",
"
196.9
\n",
"
89
\n",
"
8.86
\n",
"
6.6
\n",
"
7
\n",
"
1.78
\n",
"
2
\n",
"
False
\n",
"
\n",
"
\n",
"
4
\n",
"
OK
\n",
"
75
\n",
"
415
\n",
"
Yes
\n",
"
No
\n",
"
0
\n",
"
166.7
\n",
"
113
\n",
"
28.34
\n",
"
148.3
\n",
"
122
\n",
"
12.61
\n",
"
186.9
\n",
"
121
\n",
"
8.41
\n",
"
10.1
\n",
"
3
\n",
"
2.73
\n",
"
3
\n",
"
False
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" State Account length Area code International plan Voice mail plan \\\n",
"0 KS 128 415 No Yes \n",
"1 OH 107 415 No Yes \n",
"2 NJ 137 415 No No \n",
"3 OH 84 408 Yes No \n",
"4 OK 75 415 Yes No \n",
"\n",
" Number vmail messages Total day minutes Total day calls \\\n",
"0 25 265.1 110 \n",
"1 26 161.6 123 \n",
"2 0 243.4 114 \n",
"3 0 299.4 71 \n",
"4 0 166.7 113 \n",
"\n",
" Total day charge Total eve minutes Total eve calls Total eve charge \\\n",
"0 45.07 197.4 99 16.78 \n",
"1 27.47 195.5 103 16.62 \n",
"2 41.38 121.2 110 10.30 \n",
"3 50.90 61.9 88 5.26 \n",
"4 28.34 148.3 122 12.61 \n",
"\n",
" Total night minutes Total night calls Total night charge \\\n",
"0 244.7 91 11.01 \n",
"1 254.4 103 11.45 \n",
"2 162.6 104 7.32 \n",
"3 196.9 89 8.86 \n",
"4 186.9 121 8.41 \n",
"\n",
" Total intl minutes Total intl calls Total intl charge \\\n",
"0 10.0 3 2.70 \n",
"1 13.7 3 3.70 \n",
"2 12.2 5 3.29 \n",
"3 6.6 7 1.78 \n",
"4 10.1 3 2.73 \n",
"\n",
" Customer service calls Churn \n",
"0 1 False \n",
"1 1 False \n",
"2 0 False \n",
"3 2 False \n",
"4 3 False "
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"telecom_data = pd.read_csv(\"https://static-1300131294.cos.ap-shanghai.myqcloud.com/data/telecom_churn.csv\")\n",
"\n",
"telecom_data.head()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "30b50e86",
"metadata": {
"attributes": {
"classes": [
"code-cell"
],
"id": ""
}
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABIkAAANhCAYAAABjAus5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAB7CAAAewgFu0HU+AAB9zUlEQVR4nOzdeZzWZb0//tc9DDuispgglAsialkcxKOBkLl0RJNI82idXFEzc6lc8quZpaYeNU3QYwZudTQ1U1MjLTcQRcC04wLhRgKCiqgIsghz//7gx92MzAzrzA3M8/l4zONxzf25PtfnfcN8hpkX13V9CsVisRgAAAAAmrSKchcAAAAAQPkJiQAAAAAQEgEAAAAgJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgSWW5C2DjsXDhwjz//PNJks6dO6ey0pcXAAAArGtLlizJO++8kyT53Oc+l1atWq2Tcf0Wzzrz/PPPZ7fddit3GQAAANBkjB8/Pn379l0nY1luBgAAAICZRKw7nTt3LrXHjx+fLl26lLGaVbdgwYKMHj06STJgwIC0bt26zBXB+sP9AXVzf0D93CNQN/cHa2vmzJmllTzVfxdfW0Ii1pnqexB16dIl3bp1K2M1q27BggXp1KlTkqRbt26+QUM17g+om/sD6ucegbq5P1iX1uV+wJabAQAAACAkAgAAAEBIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAAAkqSx3AQAAALA6qqqqMm/evMydOzeLFy/O0qVLy13SaqmqqkrHjh2TJG+88UYqKszfaGqaNWuWFi1apH379mnXrt168zUgJAIAAGCD8eGHH2bGjBkpFovlLmWNFYvFtG7dOkmydOnSVFVVlbkiGtuSJUuyaNGifPjhhykUCtlqq62yySablLssIREAAAAbhtoCokKhkGbNmpWxqjVTKBSSJJWVfi1vipYuXVr6Oi4Wi5kxY8Z6ERT5agQAAGC9V1VVVSMgateuXTp06JA2bdqUApcNRVVVVebOnZskad++/Xqz1IjGUywW89FHH2XOnDmZN29eKSjq2bNnWb8efCUCAACw3lv+i3SyLCDq1q1b2rZtu8EFRJAsm0nWtm3bdOvWLe3atUuyLDiaN29eWesSEgEAALDeWz7zJkk6dOggHGKjUCgU0qFDh9Ln1b/Oy0FIBAAAwHpv8eLFSZb9Ut2mTZsyVwPrTvUlk8u/zstFSAQAAMB6b/lj7ps1a2YWERuV6puvL/86LxchEQAAAABCIgAAAACERAAAAABESAQAAABAhEQAAAAAREgEAAAAsIKpU6emUCikUCjkpptuKnc5jUJIBAAAAKzgscceK4Uk559/frnLoRFUlrsAAAAAWNe2/tED5S6hQU295IByl8BGyEwiAAAAAIREAAAAAAiJAAAAAIiQCAAAAGgAixcvzrXXXpu99tornTt3TosWLbLllltm0KBB+e1vf5uqqqoVzpk9e3ZatmyZQqGQ73znOyu9xn333VfaXPuOO+6ocWzmzJm59tprc8ghh2T77bdP27Zt07Jly2y11VYZPHhwbr/99lpraMqERAAAAMA6NXXq1Hz+85/PSSedlMceeyyzZ8/Oxx9/nLfeeiujRo3Kt7/97QwcODBz5sypcV6nTp0yePDgJMntt9+ehQsX1nudG2+8MUnSoUOH0nlJsnTp0nTr1i0nnXRS7rrrrrzyyiv56KOPsnjx4rz55pv54x//mMMOOyz/8R//kXnz5q3jd7/hEhIBAAAA68y8efOy9957Z/LkyUmSr33ta/njH/+YiRMn5s4778zAgQOTJE888US++tWvZunSpTXOHzp0aJLk/fffz913313ndd55553cf//9SZJvfetbadmyZelYsVhMknz5y1/OZZddlj//+c955pln8thjj+WGG27IHnvskST5y1/+kpNOOmkdvfMNX2W5CwA2Dhv7I0YbmkeYAgCwsfjpT3+a1157LUly7rnn5oILLigd69OnTw4++OB8+9vfzv/+7//mySefzPXXX58TTzyx1GefffbJZz7zmfzzn//MjTfemMMPP7zW6/z2t7/Nxx9/nCQ55phjahxr1qxZ/vGPf6RHjx4rnDdw4MAcffTR+clPfpKf/exn+c1vfpNzzz0322+//Vq/9w2dmUQAAADAOrFo0aKMGDEiSbLzzjvn/PPPX6FPoVDItddem44dOyZJhg8fXuN4RUVFKfR5+OGHM23atFqvtXypWe/evfOFL3xhhWvUFhBVd95556VTp04pFov54x//uNL31hQIiQAAAIB14plnnsn777+fJDnqqKPSrFmzWvu1b98+hx56aJLkpZdeysyZM2scP+aYY1JRUZGqqqrcfPPNtV7n+eefL/Vdmaqqqrz55pv5xz/+kRdeeCEvvPBCJk2alG7duiVJ/v73v6/ye9yYCYkAAACAdeKFF14otf/93/+93r7Vj1c/L0m6deuWr3zlK0mSm266aYVzl88iatmyZb71rW/VOn6xWMxvf/vb7LXXXmnXrl222mqr9OrVK5/73OdKH88991ySZU9VQ0gEAAAArCPVn1a2xRZb1Nt3yy23rPW85ZZvYP3qq69m9OjRpdcXLVqUW2+9NcmyTbE333zzFc5duHBhDjjggHz729/OY489lgULFtRby8qONxVCIgAAAGCdKxQKa3X+V7/61XzqU59K8q+ZQ0lyzz335L333ktS91Kziy66KKNGjUqybKPqO+64I6+88krmzZuXpUuXplgsplgsZs8990zyr6ehNXVCIgAAAGCd6NChQ6n91ltv1dt31qxZtZ63XPPmzXPEEUckSe68887Mmzcvyb8Co09/+tPZZ599VjivWCyWNs/ec88988gjj+Qb3/hGtttuu7Rt2zYVFf+KQmqbwdSUCYkAAACAdeKzn/1sqf3000/X23f8+PG1nlfd8iVn8+fPz5133pnp06fnL3/5S5LkyCOPrBH4LDdnzpxSAPWNb3yj1j5JMm/evPzjH/+ot8amRkgEAAAArBN9+vTJZpttliS5+eabU1VVVWu/Dz/8MHfccUeSZKeddkqXLl1q7dezZ8/SkrAbb7yxNGahUMjRRx9d6zlLliwptefPn19nrSNGjKjRFyERAAAAsI60bNmyNPvnhRdeyAUXXLBCn2KxmO9973ulJ4p973vfq3fM5eONGTMmw4YNS5J86UtfyjbbbFNr/86dO5eCqttuuy2LFi1aoc+ECRPy4x//eNXeVBNSWe4CAAAAgPXbc889V+uj6D/py1/+cs4777z84Q9/yGuvvZbzzz8/zz//fI4++uh06dIlr7/+eoYPH57HHnssSbLHHnvk+OOPr3fMb3zjGznllFPywQcflPY5qmvD6iSpqKjIt771rVxzzTX5v//7v/Tv3z8/+MEPsv322+eDDz7In/70p1x77bVp165dunbtmilTpqzyn8PGTkgEAAAA1Ovee+/Nvffeu9J+d999d772ta/l4Ycfzv7775/Jkyfnrrvuyl133bVC3379+uWPf/xjmjVrVu+YrVu3zje/+c38z//8T5Jk0003zcEHH1zvORdddFHGjh2b5557LhMnTsw3v/nNGsc7dOiQu+66K+edd56QqBohEQAAABudqZccUO4S6lRVVZW5c+cmSdq3b1/nxsobsq233jp///vf8+tf/zp33nlnXnjhhcydOzcdOnRI7969861vfSvf/OY3V/m9f/vb3y6FRIcddlhat25db/9NN900Y8eOzS9+8Yvccccdefnll1NZWZnu3bvngAMOyKmnnppu3bqt9fvc2AiJAAAAgBV86UtfSrFYXOPzW7RokZNOOiknnXTSWtfywgsvlNr1LTWrrk2bNjn33HNz7rnn1tln+bK32my99dZr9f43RBtfXAkAAABsVG644YYkyWc/+9nstttuZa5m4yUkAgAAANZbo0ePzrhx45Ik3/nOd8pczcbNcjMAAABgvfLPf/4zixYtyosvvpjvf//7SZItt9xylZeasWaERAAAAMB6ZeDAgfnnP/9Z47Vhw4atdMNq1o7lZgAAAMB6aZNNNskee+yR+++/P4cccki5y9nomUkEAAAArFemTp1a7hKaJDOJAAAAABASAQAAACAkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAADShEOit99+O/fff3/OO++87L///unUqVMKhUIKhUKOOuqo1R5v1KhRGTJkSLp165aWLVumW7duGTJkSEaNGrXKYyxZsiTXXXdd9txzz3Tu3DmtW7fOdtttlxNOOCEvvvjiKo8ze/bsnHfeedlll13Svn37tG/fPrvsskvOO++8vPvuu6v93gAAAICNX2W5CyiXT33qU+tknKqqqhx//PEZOXJkjddnzJiRGTNm5J577snQoUPzq1/9KhUVdWdys2fPzqBBgzJhwoQar7/22mu5/vrrc/PNN2f48OEZOnRovfU8/fTT+drXvpZZs2bVeP3555/P888/nxEjRuSee+7JbrvttprvFAAAANiYNdmZRNV9+tOfzn777bdG555zzjmlgKh379657bbbMn78+Nx2223p3bt3kmTEiBE599xz6xxj6dKlGTJkSCkg+vrXv55Ro0bl6aefztVXX50tttgiixYtygknnFDvzKRp06blq1/9ambNmpXKysqceeaZGT16dEaPHp0zzzwzlZWVmTlzZr761a9m+vTpa/R+AQAAgI1Tk51JdN5556Vv377p27dvPvWpT2Xq1KnZZpttVmuMKVOm5PLLL0+S7Lrrrhk9enRat26dJOnbt28OOuigDBw4MBMnTsxll12WY445Jj169FhhnJtvvjlPPPFEkuS73/1urrnmmtKx3XbbLfvvv3/69OmTuXPn5pRTTsmkSZNSWbniX90555yTd955J0ly66235hvf+Ebp2J577pk+ffrkP//zP/P222/n3HPPzU033bRa7xcAAABWV/Xft2+88cY12uKFxtFkZxL99Kc/zYEHHrhWy86uuuqqLFmyJEkybNiwUkC0XJs2bTJs2LAky/YbuvLKK2sdZ3nQ1KFDh1x22WUrHO/Ro0fOPvvsJMkrr7ySu+++e4U+s2bNyv/+7/8mSb7yla/UCIiWO/TQQ/OVr3wlSfKb3/xmhSVpAAAAUJfFixfntttuyxFHHJFevXqlY8eOad68eTp16pQ+ffrkxBNPzF//+tdUVVWVu1TWUJOdSbS2isVi7r333iRJr169svvuu9fab/fdd88OO+yQf/zjH7n33nszfPjwFAqF0vEpU6Zk0qRJSZaFOG3atKl1nKOOOqoUFN19990rhEB//OMfSzfi0UcfXWfdRx11VB588MFUVVXlj3/8Y44//vhVfMcAAAAbkPM3LXcFdapIstnaDnL+B2tfyGr4wx/+kB/+8IeZOnXqCsfefffdvPvuu/nb3/6W6667Lj179swvfvGLHHDAAY1aI2uvyc4kWluvv/563nzzzSTJwIED6+27/PiMGTNWuKGWLzNb2ThbbrllevbsmSQZO3bsCsdXdZzqx2obBwAAAKq74IILcvDBB5d+n913330zbNiwPPzww3nmmWfyl7/8JcOHD89XvvKVVFRUZMqUKTnnnHPKWzRrxEyiNfTSSy+V2r169aq3b/XjkyZNqrH30eqOM2XKlEybNi3z589P27ZtVxhn0003zZZbblnnGF26dEn79u0zd+7c0gwmAAAAqM2NN96Y8847L0myxRZb5I477qh1YsI+++yTk046KS+88EK+//3vl/bLZcMiJFpD1Z8O1q1bt3r7du/evdSeNm3aWo9TLBYzffr07LDDDiuMs7Ixlo/z4osvrlDLyqzsiWgzZ84stRcsWJAFCxas1vjlsnDhwlrbrJ5NWxTLXcIGbX29X9wfUDf3B9TPPcK6VlVVlWKxWGqvzMa+bKYx9v2ZMWNGvve97yVJ2rZtm0cffTS9evWq99o77bRTRo0alVtvvbXUr3r/qqoqexbVoVgspqqqapV+N2io3x+ERGvoww8/LLXbtWtXb9/qM37mzZvXoOOsbIzq43xyjJWpHnatzOjRo9OpU6fVGn99MHr06HKXsMH6WZ9yV7Bhe+ihh8pdwkq5P6Bu7g+on3uEdaFjx45p3bp1CoVC5s6du9L+mzV8SWW1Kn8Ga+u///u/89FHHyVJzj777HTt2nWVr3vQQQeV+lb/3XPhwoWZO3duHn300fzqV7/Ks88+m/fffz9bbrll9t577/zwhz/MVlttVeuY3/3ud3Pbbbele/fu+b//+786r33rrbfmpJNOSpL8/e9/z6c//ekax3fZZZdMmzYthx9+eK699to899xzuf766zN27NjMmjUrixcvznvvvVdr35dffjnDhw/Po48+mrfeeiubbrppdtttt5x66qnp27fvKv3Z1GbJkiWlyRaTJ09eaf/Zs2ev8bXqIyRaQ9X/N6RFixb19m3ZsmWp/cm0b12Ps7Ixqo+zvs5cAAAAoLyKxWJuu+22JMsmGhxxxBHrbOyf/vSnueqqq2q89sYbb+TGG2/Mfffdl/vvv7/GypmGdMMNN+Sss84qPbm8Pvfff39OOOGEUnCWJO+8804eeOCB/PnPf87111+fr3/96w1ZboMTEq2hVq1aldqLFy+ut++iRYtK7datW9c7TvXPV3ecjz76aKW1VB/nk2OszMqWp82cOTO77bZbkmTAgAGrtPRtfbBw4cLS/24NGDCg3r8D6rb7xQ+Xu4QN2riz9y53CbVyf0Dd3B9QP/cI69obb7yRpUuXprKyMu3bty93OWXX0H8GL7zwQt59990kSf/+/euc3bMqqq94+d///d88+eSTGThwYI477rj07Nkz77//fn7zm9/kN7/5TWbPnp3TTjut1gctNW/ePElSUVFR7/uv/v2mXbt2K/StqFi2GPHvf/977rjjjnTv3j0//OEP06dPnyxZsiRPPPFE6Zzlff/xj3/k7rvvTpcuXfKDH/wgffr0SbFYzEMPPZRLL700CxcuzPe///0ccMAB6dy582r/Gb3zzjtp3bp12rVrlz59Vr5MY2XbwawpIdEa2mSTTUrtlS3bmj9/fqn9yeVgnxynvn88VzbORx99tEpLyJaPsypL06pbndCndevWqx1CrQ9atWq1Qda9PvhgcaHcJWzQNoSvO/cH1M39AfVzj7AuVFRUlPayWf6Le1PW0H8Gzz//fKm96667rtX1qp/75JNP5rjjjsuvfvWrFAr/+h1i3333TcuWLTNixIiMGzcuf//739O7d+8a41TvX1891Y9VVFTU2fell17K5z73uYwePTqbbbZZ6fU999xzhb5/+9vf0qdPnzzyyCM1QqcvfvGL2X777fNf//VfmTt3bm699dZ8//vfr7O2+hQKhVRUVKzS98uG+p7qzlpD1QOTlSV41WfgfHJfnzUZp1AorBDYLP98VdLE5eOszh5DAAAANB3LZxEly55qtq506dIlw4YNqxH4LHf66aeX2mPGjFln16zPNddcUyMgqs8NN9xQ6wymb37zm+natWuSxqu7oQiJ1tBOO+1Uaq9sU6nqx3fccce1Hqd79+41NrGuPs4HH3yQWbNm1TnGzJkzS5uHfbIWAAAASGo+ZOmTv3+ujUMOOaTGfrvV7bDDDqUVL6+99to6u2ZdunfvXuusodp87nOfyy677FLrsUKhUJr11Bh1NyQh0RraZpttSknh448/Xm/f5Wuxt9pqq2y99dY1jvXv37/Urm+cWbNmZcqUKUmSfv36rXB8Vcepfqy2cQAAAKD61ijVtz5ZW7169ar3+Oabb56kZkjVUOoKfWqzsro7dOiQpHHqbkhCojVUKBQyePDgJMtm+IwbN67WfuPGjSvNABo8ePAKU+p69uxZmtFzxx131Nglvbqbbrqp1B4yZMgKxw866KDSOssbb7yxzrqXj1NRUZGDDjqozn4AAAA0XR07diy133rrrXU2bps2beo9vvz32qVLl66za9ZleSC1KtanuhuSkGgtnHbaaWnWrFmS5OSTT17hkfILFizIySefnCSprKzMaaedVus4y9ddzpkzJ2eeeeYKx1999dVcfPHFSZIePXrUGhJtueWW+da3vpUkefDBB/P73/9+hT533nlnHnzwwSTJt7/97Wy55Zar8jYBAABoYj7/+c+X2n/729/KWEnDWf77PP/SZJ9u9sQTT+SVV14pfT579uxS+5VXXqkxcydJjjrqqBXG6NmzZ84444xccsklmThxYvr165ezzjor2223XV599dVceumlefbZZ5MkZ5xxRrbffvtaaznyyCNzww03ZOzYsbnmmmsya9asHHfccdl8880zfvz4XHDBBZk7d24qKipy9dVXp7Ky9r+2iy66KH/+85/zzjvv5PDDD8/EiRNz4IEHJknuv//+XHHFFUmSzp0758ILL1zlPysAAACalp133jmdOnXK7NmzM2bMmMydO7fex843huWzdZY/5a4u63J5XFPTZEOiESNG5Oabb6712NixYzN27Ngar9UWEiXLgpm33347N9xwQ5599tkcdthhK/Q59thj6w1lmjVrlnvuuSeDBg3KhAkTctddd+Wuu+6q0adly5YZPnx49t9//zrH6d69e+6777587Wtfy6xZs3LppZfm0ksvrdFnyy23zD333LNaj7MHAACgaSkUCjnyyCNzxRVXZP78+RkxYkR+8IMflLWm5fskvf/++/X2W76fL6vPcrO1VFFRkZEjR+aBBx7I4MGD07Vr17Ro0SJdu3bN4MGD86c//SkjRowoJZ516dSpU5588slce+216d+/fzp27JhWrVpl2223zXHHHZdnnnkmQ4cOXWk9//7v/57nn38+5557bj772c+mXbt2adeuXT73uc/l3HPPzQsvvJB///d/X1dvHwAAgI3U97///dJePOedd95Kn8i9XFVVVf73f/93ndezzTbbJFm2OfQ//vGPWvssXrx4hUkXrLomO5PopptuWmFJ2doYNGhQBg0atFZjVFZW5sQTT8yJJ564VuN06tQpF1xwQS644IK1GgcAAICma6uttsrw4cNzzDHHZP78+Rk4cGDuuOOODBw4sM5zXnrppZx22ml5++23S/vmrivVr3vFFVfk+uuvX6HPD37wg8yYMWOdXrcpabIhEQAAAFC/o48+OtOnT895552Xt99+O1/60pey3377ZfDgwdlxxx2z2WabZc6cOZkyZUoeeOCB/PnPf87SpUtrbHy9rvTu3Tt77LFHnnrqqfz617/O4sWLc+SRR2bTTTfNyy+/nOuvvz6PPPJIvvjFL+bJJ59c59dvCoREAAAAQJ1+/OMfZ+edd84Pf/jDTJ06NQ899FAeeuihOvvvvPPO+e///u8GqeWGG27IwIED8/bbb+fmm29eYa/h008/PTvvvLOQaA0JiQAAANj4nP9BuSuoU1VVVebOnZskad++/Ur3sF0ffP3rX8+BBx6Y3//+9xk1alQmTJiQt99+Ox9++GHat2+frbfeOrvvvnsOOeSQfOlLX0qhUGiQOnr16pW//e1vueiii/KnP/0pM2fOzKabbpo+ffrk5JNPzqBBg9bp1jJNjZAIAAAAWKkWLVrkm9/8Zr75zW+u1nlbb711isXiKvWdOnXqSvtstdVWufbaa+s8ftRRR9X5hPJVvcbq9l3X+x6Xy/ofVwIAAADQ4IREAAAAAAiJAAAAABASAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAEBZFYvFcpeQREgEAADABqBZs2ZJkiVLlmTp0qVlrgbWnaVLl5a+ppd/nZeLkAgAAID1Xps2bUrt999/v3yFwDpW/eu5+td5OVSW9eoAAACwCjbbbLO89957SZK33347S5cuTfv27dOyZcsUCoUyVwerp1gsZtGiRZk7d27efffd0uubb755GasSEgEAALABaNWqVTbddNN88MEHSZJ333037777bgqFQtmX6KyJJUuWJEneeeedMldCOSxdunSFfYg23XTTtGzZskwVLSMkAgAAYIPQpUuXtGjRokawUiwWS4HLhqJYLGbBggVJktatW5sJRTp37pyOHTuWuwwhEQAAABuGQqGQTp06pX379pk3b17mz5+fxYsXp6qqqtylrZaqqqpSSNSuXbtUVNguuKmpqKhIixYt0rZt27Rr1y4tWrQod0lJhEQAAABsYFq0aJEOHTqkQ4cO5S5ljSxYsCCTJ09OkvTp0yetW7cuc0WwjLgSAAAAACERAAAAAEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJFpnFi9enBEjRuQrX/lKunTpkpYtW6Zdu3bZYYcdcvTRR+fJJ59cpXFGjRqVIUOGpFu3bmnZsmW6deuWIUOGZNSoUatcy5IlS3Lddddlzz33TOfOndO6detst912OeGEE/Liiy+u6VsEAAAANmKV5S5gY/DPf/4zBxxwwAoBzOLFizNlypRMmTIlN910U04++eT88pe/TKFQWGGMqqqqHH/88Rk5cmSN12fMmJEZM2bknnvuydChQ/OrX/0qFRV1Z3uzZ8/OoEGDMmHChBqvv/baa7n++utz8803Z/jw4Rk6dOhavGMAAABgY2Mm0Vr6+OOPawREu+yyS2666aY89dRTeeihh3Leeeelbdu2SZJhw4bl0ksvrXWcc845pxQQ9e7dO7fddlvGjx+f2267Lb17906SjBgxIueee26dtSxdujRDhgwpBURf//rXM2rUqDz99NO5+uqrs8UWW2TRokU54YQTVmtmEgAAALDxM5NoLd17772lgGiPPfbImDFj0qxZs9LxfffdNwcddFD22GOPfPzxx7n00ktz+umnp7LyX3/0U6ZMyeWXX54k2XXXXTN69Oi0bt06SdK3b98cdNBBGThwYCZOnJjLLrssxxxzTHr06LFCLTfffHOeeOKJJMl3v/vdXHPNNaVju+22W/bff//06dMnc+fOzSmnnJJJkybVqAMAAABouswkWkvV9xo6++yzawREy/Xp0ycHHnhgkuT999/PpEmTahy/6qqrsmTJkiTLZhstD4iWa9OmTYYNG5Zk2X5DV155Za21LA+aOnTokMsuu2yF4z169MjZZ5+dJHnllVdy9913r9J7BAAAADZ+QqK1tHjx4lJ72223rbPfdtttV+s5xWIx9957b5KkV69e2X333Ws9f/fdd88OO+yQZNnspWKxWOP4lClTSuHToYcemjZt2tQ6zlFHHVVqC4kAAACA5YREa2l5cJMs2xy6Lq+++mqSpFAoZPvtty+9/vrrr+fNN99MkgwcOLDeay0/PmPGjEydOrXGseXLzFY2zpZbbpmePXsmScaOHVvv9QAAAICmQ0i0lg4//PC0b98+SXLppZdm6dKlK/R59tln88ADDyRJvvnNb5b6J8lLL71Uavfq1avea1U//skla2syzrRp0zJ//vx6+wIAAABNg12L11KnTp3ym9/8JocffnjGjh2bvn375rTTTkvPnj0zb968jB07NldccUUWL16cf/u3f8sVV1xR4/zp06eX2t26dav3Wt27dy+1p02bttbjFIvFTJ8+vcZsqPpUv0ZtZs6cWWovWLAgCxYsWKVxy23hwoW1tlk9m7YorrwTdVpf7xf3B9TN/QH1c49A3dwfrK2G+v1BSLQOHHTQQXnmmWdyxRVXZOTIkTnyyCNrHP/Upz6VCy64IMcdd9wKewV9+OGHpXa7du3qvU7btm1L7Xnz5jXIOPWpHlKtzOjRo9OpU6dV7r++GD16dLlL2GD9rE+5K9iwPfTQQ+UuYaXcH1A39wfUzz0CdXN/sCZmz57dIONabrYOLF68OLfcckutG0onyVtvvZXf/va3+etf/7rCseqpcYsWLeq9TsuWLUvtT6aG62ocAAAAoGkyk2gtzZ8/P/vvv3/GjBmTZs2a5cwzz8zRRx+dbbfdNgsXLszTTz+dn/3sZ3niiSfyta99LZdffnl+8IMflM5v1apVqV39qWe1WbRoUandunXrGsc+OU71z1dnnPp8conbJ82cOTO77bZbkmTAgAErXfa2vli4cGEpvR8wYEC9f3bUbfeLHy53CRu0cWfvXe4SauX+gLq5P6B+7hGom/uDtbWy7WDWlJBoLZ1//vkZM2ZMkqyw1KxFixbZd999s9dee2W//fbLo48+mjPOOCN77713Pv/5zydJNtlkk1L/lS39qr7J9CeXlH1ynPq+ydQ3Tn1WJ/Rp3br1agVQ64tWrVptkHWvDz5YXCh3CRu0DeHrzv0BdXN/QP3cI1A39wdroqG+Ziw3WwvFYjE33HBDkqRnz54r7EW0XGVlZS644IIkSVVVVW666abSserBy8qSwOozeT65P9CajFMoFDaY2T4AAABAwxISrYW33norc+bMSZL07t273r59+vxrV9/JkyeX2jvttFOtr9em+vEdd9yxxrE1Gad79+41NrEGAAAAmi4h0VqorPzXar0lS5bU2/fjjz+u9bxtttkmXbt2TZI8/vjj9Y6xfM3qVlttla233rrGsf79+5fa9Y0za9asTJkyJUnSr1+/eq8HAAAANB1CorXQoUOHtG/fPkny1FNP1RsUVQ9uttlmm1K7UChk8ODBSZbN8Bk3blyt548bN640A2jw4MEpFGru/9KzZ8/S7KI77rgjH330Ua3jVF/qNmTIkDrrBQAAAJoWIdFaqKioyAEHHJAkefPNN3PRRRfV2u+9997LWWedVfr8wAMPrHH8tNNOS7NmzZIkJ5988gqPpV+wYEFOPvnkJMtmIZ122mm1Xuf0009PksyZMydnnnnmCsdfffXVXHzxxUmSHj16CIkAAACAEiHRWjrvvPPSpk2bJMuedHbQQQflrrvuyrPPPpunnnoqV155Zb7whS/kpZdeSpLsvffe2W+//WqM0bNnz5xxxhlJkokTJ6Zfv365/fbbM3HixNx+++3p169fJk6cmCQ544wzsv3229day5FHHllaQnbNNdfkkEMOyYMPPpjx48dn+PDh+eIXv5i5c+emoqIiV199dY1lbwAAAEDTJiVYS7169cq9996bww8/PLNnz859992X++67r9a+X/7yl3PnnXfWeuyiiy7K22+/nRtuuCHPPvtsDjvssBX6HHvssbnwwgvrrKVZs2a55557MmjQoEyYMCF33XVX7rrrrhp9WrZsmeHDh2f//fdfjXcJAAAAbOzMJFoH9tlnn0yePDmXXnppvvSlL6Vz585p3rx5WrdunW222SaHHnpo7rnnnvz1r3/N5ptvXusYFRUVGTlyZB544IEMHjw4Xbt2TYsWLdK1a9cMHjw4f/rTnzJixIhUVNT/V9apU6c8+eSTufbaa9O/f/907NgxrVq1yrbbbpvjjjsuzzzzTIYOHdoQfwwAAADABsxMonWkY8eOOfPMM2vdC2h1DBo0KIMGDVqrMSorK3PiiSfmxBNPXKtxAAAAgKbDTCIAAAAAhEQAAAAACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAABIUlnuAgBItv7RA+UuoVabtijmZ32WtXe/+OF8sLhQ3oLqMPWSA8pdAgAAbPDMJAIAAABASAQAAACAkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAASBlCou9973t59tlnG/uyAAAAANSj0UOia6+9Nrvuumt69+6dYcOGZc6cOY1dAgAAAACf0OghUfPmzVMsFvP3v/89p512Wrbaaqv853/+Z/785z+nWCw2djkAAAAApAwh0cyZM3PVVVflC1/4QorFYhYtWpTf//73OeCAA/KZz3wmP/7xj/Pqq682dlkAAAAATVqjh0QdOnTIKaeckr/97W/529/+lu9973vp0KFDisVipk+fnp///Ofp2bNnvvSlL+U3v/lNFixY0NglAgAAADQ5ZX262Re+8IVcffXVefPNN3PnnXdm//33T0VFRYrFYsaMGZOjjjoqXbp0yQknnJBx48aVs1QAAACAjVpZQ6LlmjdvnoMPPjgPPPBA3njjjfz85z/P9ttvn2KxmLlz52bEiBHp169fdt5551xxxRV5++23y10yAAAAwEZlvQiJquvSpUt+9KMfZfLkyRk1alS23HLLJEmxWMzkyZNz5plnpnv37jnssMPy3HPPlbdYAAAAgI3EehcSJcno0aNz9NFH55BDDslbb71VeupZ27ZtUywW8/HHH+fOO+/MrrvumlNPPTVVVVVlrhgAAABgw7behETTpk3LhRdemB49emSvvfbKLbfckvnz5ydJ9t1339x+++159913M2XKlJx11lnZfPPNU1VVleHDh2f48OFlrv5f3njjjfzkJz/Jrrvums6dO6dVq1bp3r179txzz5x33nl54YUX6j1/1KhRGTJkSLp165aWLVumW7duGTJkSEaNGrXKNSxZsiTXXXdd9txzz3Tu3DmtW7fOdtttlxNOOCEvvvji2r5FAAAAYCNUWc6LL1q0KH/4wx9y44035pFHHkmxWCzNGurWrVuOPvroHHPMMfnMZz5TOqdHjx65+OKL86Mf/ShDhgzJY489luuvvz6nnHJKud5GybBhw3L22WeXwq3lpk+fnunTp+eJJ57I3Llzc9VVV61wblVVVY4//viMHDmyxuszZszIjBkzcs8992To0KH51a9+lYqKurO92bNnZ9CgQZkwYUKN11977bVcf/31ufnmmzN8+PAMHTp0zd8oAAAAsNEpS0g0fvz43Hjjjbn99tvzwQcfJFm251Dz5s1z4IEHZujQofmP//iPFAqFOsfYdNNN89Of/jQDBw7Mq6++2lil1+nCCy/Mj3/84yRJz549c9xxx6Vv377ZdNNN8+677+bZZ5/N3XffXWfAc84555QCot69e+fMM8/Mdtttl1dffTX//d//nWeffTYjRoxI586d8/Of/7zWMZYuXZohQ4aUAqKvf/3rOe6449KhQ4c8/fTTufDCC/P222/nhBNOyFZbbZX999+/Af4kAAAAgA1Ro4dEO++8cyZPnpwkpVlDPXv2zLHHHpsjjzwyW2yxxSqP1bVr1yTJ4sWL132hq+Hhhx8uBURHHHFERowYkebNm9fos/fee+f000+vtdYpU6bk8ssvT5LsuuuuGT16dFq3bp0k6du3bw466KAMHDgwEydOzGWXXZZjjjkmPXr0WGGcm2++OU888USS5Lvf/W6uueaa0rHddtst+++/f/r06ZO5c+fmlFNOyaRJk1JZWdbJZAAAAMB6otH3JJo0aVKKxWJatWqVb3/723n88cczefLknHHGGasVECVJ+/btc8QRR+SII45ooGpXrqqqKieeeGKS5POf/3xGjhy5QkBUXYsWLVZ47aqrrsqSJUuSLFuytjwgWq5NmzYZNmxYkmX7DV155ZW1jr08aOrQoUMuu+yyFY736NEjZ599dpLklVdeyd13372ytwcAAAA0EY0eEvXu3TvXXHNNZs6cmZtvvjl77rnnGo/VuXPn3HTTTbnxxhvXYYWr56GHHsrLL7+cJDnrrLNWe2ZOsVjMvffemyTp1atXdt9991r77b777tlhhx2SJPfee29pFtZyU6ZMyaRJk5Ikhx56aNq0aVPrOEcddVSpLSQCAAAAlmv0tUbPPPNMY1+yQd15551JkkKhkAMPPLD0+pw5c/Luu++mY8eO6dChQ53nv/7663nzzTeTJAMHDqz3WgMHDsw//vGPzJgxI1OnTs0222xTOrZ8mdnKxtlyyy3Ts2fPTJkyJWPHjq3/zTVBu1/8cD5YXPdeWAAAALCxavSZRD/72c/ys5/9LLNnz17lc957773SeeubcePGJUm23nrrbLLJJrn11lvzuc99Lh07dkzPnj3TsWPH7LDDDrn88suzaNGiFc5/6aWXSu1evXrVe63qx5fPGlqbcaZNm7bCk9gAAACApqnRZxKdf/75KRQKOeSQQ9KpU6dVOmfOnDml884777wGrnDVVVVVlTbh7tSpU0499dRcffXVK/SbMmVKzjjjjNx999154IEHstlmm5WOTZ8+vdTu1q1bvdfr3r17qT1t2rQax9ZknGKxmOnTp5eWsa1M9WvUZubMmaX2ggULsmDBglUat9wWLlxYardvXqynJzQ91e+J9fn+2FC+37Bxqf7vR/U2sIx7BOrm/mBtNdTPvx5ttRY++OCDVFVVJUmef/75TJgwIV26dMlll12WQYMGpVWrVpkwYULOOuusjBs3Lk8++WSOOeaY/OEPfyiN8eGHH5ba7dq1q/d6bdu2LbXnzZtX49i6Gqc+1UOqlRk9evQqh4Drk9N3qSp3CbDeWp/vj4ceeqjcJdDEjR49utwlwHrNPQJ1c3+wJlZnddbqaPTlZmvi448/TpJ6nxpWDtWXai1cuDBt2rTJo48+mm9961vZfPPN07p16wwYMCCPPPJIPv/5zydZtln0008/XeO85Wp78ll1LVu2LLU/mRquq3EAAACApmmDmEn03HPPJVn2NLP1SatWrWp8PnTo0FqXbrVu3ToXXXRRaWPr22+/Pf/+7/++whiLFy+u93rV9zRq3bp1nbUsXrx4hdpWdZz6fHKJ2yfNnDkzu+22W5JkwIABK132tr5YuHBhKb2//P8qMvdjG1fDcu2bF0sziNbn+2Pc2XuXuwSaoOr/fgwYMKDef3uhKXKPQN3cH6ytlW0Hs6YaPCS65ZZban393nvvzcSJE+s9d9GiRXn11Vdzww03pFAopG/fvg1R4hrbZJNNany+33771dl37733TmVlZZYsWZIJEybUOsbKln5Vn7n0ySVlnxynvm8y9Y1Tn9UJfVq3br1aAdT6Yu7HBU83gzqsz/fHhvj9ho1Lq1atfB1CPdwjUDf3B2uiob5mGjwkOuqoo1Io1Pylolgs5txzz13lMYrFYioqKnLqqaeu6/LWSsuWLdO5c+e88847Serfs6dVq1bp1KlTZs2aVeqf1AxeVpYEVp/J88lrfXKc+vYDWj5OoVDYYGb7AAAAAA2rUfYkKhaLpY/aXqvvo3nz5unXr1/++Mc/ZuDAgY1R7mrZeeedS+2lS5fW23f58crKf2VzO+20U6m9/Elpdal+fMcdd6xxbE3G6d69e41NrAEAAICmq8FnEr3++uuldrFYzLbbbptCoZAHH3ww22+/fZ3nFQqFtGrVKh07dkyzZs0ausw1NmDAgDz22GNJktdeey29e/eutd/cuXNLu49vtdVWpde32WabdO3aNW+++WYef/zxeq+1fM3qVlttla233rrGsf79+5fajz/+eA477LBax5g1a1amTJmSJOnXr1+91wMAAACajgafSfSZz3ym9FE92OjatWuNY5/8+PSnP50ttthivQ6IkuTggw8ute++++46+919992lmVR77rln6fVCoZDBgwcnWTbDZ9y4cbWeP27cuNIMoMGDB6+whK9nz56l2UV33HFHPvroo1rHuemmm0rtIUOG1FkvAAAA0LQ0ynKz6qqqqrJ06dIay6M2ZLvsskv233//JMltt92Whx9+eIU+s2bNKu3B1KJFixx99NE1jp922mmlMOzkk09e4bH0CxYsyMknn5xk2VK10047rdZaTj/99CTJnDlzcuaZZ65w/NVXX83FF1+cJOnRo4eQCAAAAChp9JBoY3TVVVdls802S1VVVQ488MCcffbZGTNmTCZOnJhrr702ffv2LW1KfcEFF9RYbpYsmwV0xhlnJEkmTpyYfv365fbbb8/EiRNz++23p1+/fqUnwZ1xxhl1LtM78sgjS0vIrrnmmhxyyCF58MEHM378+AwfPjxf/OIXM3fu3FRUVOTqq6+usTcSAAAA0LRJCdaBnj175r777sshhxySt956K5dcckkuueSSGn0KhULOOeecWmf4JMlFF12Ut99+OzfccEOeffbZWvcUOvbYY3PhhRfWWUezZs1yzz33ZNCgQZkwYULuuuuu3HXXXTX6tGzZMsOHDy/NfgIAAABIGjAkOuaYY5IsC0dGjhy5wutr4pNjrU/69++fF198McOGDcs999yT119/PYsXL06XLl3ypS99KSeffHKdm1onSUVFRUaOHJmDDz44119/fSZMmJDZs2enU6dO6du3b0444YRVCnY6deqUJ598Mr/+9a9z6623ZtKkSZk/f366du2avffeO6eeemqNJ7IBAAAAJA0YEt10002lzZWrBzvVX18dxWJxvQ6JkqRjx445//zzc/7556/xGIMGDcqgQYPWqo7KysqceOKJOfHEE9dqHAAAAKDpaLCQ6NOf/nStYVBdrwMAAABQPg0WEk2dOnW1XgcAAACgfDzdDAAAAAAhEQAAAADrcUi0aNGivPXWW6mqqip3KQAAAAAbvUYPiebNm5c//elP+dOf/pR58+atcHz27Nk5+OCD0759+3Tt2jWbb755fvjDH2bRokWNXSoAAABAk9FgG1fX5a677srRRx+dbt26rbCJdVVVVfbff//87W9/S7FYTJJ8+OGHueqqqzJ16tTcddddjV0uAAAAQJPQ6DOJHnzwwSTJkCFDUlFR8/K33357nnnmmSTJv/3bv+X73/9+/u3f/i3FYjH33HNP/vznPzd2uQAAAABNQqPPJHrhhRdSKBTyxS9+cYVjt9xyS5KkT58+efLJJ1NZWZmPP/44e+65ZyZMmJCbb745//Ef/9HYJQMAAABs9Bp9JtHbb7+dJNlmm21qvP7xxx9n9OjRKRQKOemkk1JZuSy/at68eb7zne+kWCxm/PjxjV0uAAAAQJPQ6CHRnDlzkiQtWrSo8fqECROyYMGCJFlhtlDPnj2TJLNmzWqECgEAAACankYPidq0aZPkXzOKlhs9enSSpEePHvnUpz5V41jr1q0bpzgAAACAJqrRQ6LtttsuSfLYY4/VeP3uu+9OoVDIgAEDVjjnnXfeSZJsscUWDV4fAAAAQFPU6CHRvvvum2KxmGuvvTajRo3KvHnzMmzYsEyYMCFJ8tWvfnWFc/7v//4vSdK1a9dGrRUAAACgqWj0p5udeuqpue666/Lhhx/mwAMPrHFsxx13rDUkeuCBB1IoFNK7d+/GKhMAAACgSWn0mURdunTJfffdly233DLFYrH0se222+b3v/99CoVCjf6vvvpqxowZkyTZZ599GrtcAAAAgCah0WcSJcmee+6Z119/PWPHjs2sWbPSpUuX9O/fv/TY++pmzpyZH//4x0mS/fbbr7FLBQAAAGgSyhISJUmLFi2y1157rbRf//79079//0aoCAAAAKDpavTlZgAAAACsf4REAAAAAJRvuVmS/P3vf8+YMWPy2muv5cMPP8zSpUvr7V8oFDJy5MhGqg4AAACg6ShLSPSPf/wjxxxzTMaNG7fK5xSLRSERAAAAQANp9JBoxowZGTBgQGbPnp1isZgkadeuXTbffPNUVFj9BgAAAFAOjR4SXXTRRXnnnXdSKBQydOjQnH766enZs2djlwEAAABANY0eEv35z39OoVDIEUcckeuvv76xLw8AAABALRp9fdebb76ZJDniiCMa+9IAAAAA1KHRQ6LNN988SbLZZps19qUBAAAAqEOjh0S77rprkmTKlCmNfWkAAAAA6tDoIdEpp5ySYrFoPyIAAACA9Uijh0T77rtvzjrrrDz66KM58cQT8/HHHzd2CQAAAAB8QqM/3eyWW27JjjvumC9+8Yu5/vrrc9999+WQQw5Jr1690qZNm5Web8NrAAAAgHWv0UOio446KoVCofT5zJkzM2zYsFU6t1AoCIkAAAAAGkCjh0RJUiwWy3FZAAAAAOrQ6CHR66+/3tiXBAAAAGAlGj0k+sxnPtPYlwQAAABgJRr96WYAAAAArH+ERAAAAACUZ+Pq5V5++eXccssteeqppzJr1qwsWLAgDz74YHr06FHq88ILL+SNN95I27ZtM3DgwDJWCwAAALDxKktIVFVVlTPPPDO//OUvU1VVVXraWaFQyOLFi2v0feONN3LggQemsrIyr7/+erbaaqtylAwAAACwUSvLcrMTTjghV155ZZYuXZquXbvmkEMOqbPvoEGDss0222Tp0qX5/e9/34hVAgAAADQdjR4SPfzwwxk5cmSS5P/9v/+XqVOn5o477qj3nG984xspFot55JFHGqNEAAAAgCan0ZebXX/99UmWzRC68MILV+mc3XbbLUny4osvNlhdAAAAAE1Zo88keuqpp1IoFHLssceu8jndunVLksyaNauhygIAAABo0ho9JHr77beTJFtvvfUqn9O8efMkyZIlSxqiJAAAAIAmr9FDorZt2yZJ3nnnnVU+Z/r06UmSDh06NEhNAAAAAE1do4dE2267bZLkpZdeWuVzRo0alSTZeeedG6QmAAAAgKau0UOi/fbbL8ViMddcc02qqqpW2v+ll17KTTfdlEKhkEGDBjVChQAAAABNT6OHRKecckratm2bV199Nd/5znfq3WfoL3/5S/bbb78sXLgwHTp0yHHHHdeIlQIAAAA0HZWNfcFPfepTue6663LEEUdk5MiRefDBB3PAAQeUjv/yl79MsVjM2LFjM3ny5BSLxVRUVOSmm25Ku3btGrtcAAAAgCah0UOiJPnWt76V5s2b54QTTsi0adPyq1/9KoVCIUkyYsSIJEmxWEyStGvXLjfffHONIAkAAACAdavRl5std+ihh+aVV17JT3/60/Tp0yfNmjVLsVgsfey88845++yz88orr2TIkCHlKhMAAACgSSjLTKLlOnbsmB//+Mf58Y9/nKqqqsyZMydLly5Nhw4d0rx583KWBgAAANCklDUkqq6ioiKdOnUqdxkAAAAATVKjh0RLly7NhAkTMmbMmEyZMiXvvfdePvzww7Rv3z4dOnTIDjvskP79+2fXXXdNRUXZVsMBAAAANCmNFhItWbIk11xzTS6//PK8+eabK+3fvXv3nHHGGfnOd76TZs2aNUKFAAAAAE1Xo0zVeffdd/PlL385P/jBD/Lmm2/W2KC6ro9p06bllFNOyX777Zc5c+Y0RpkAAAAATVaDzyRaunRpDjjggEyYMCHFYjGFQiH77bdf9tlnn/zbv/1bOnbsmHbt2uXDDz/M7Nmz8+yzz+Yvf/lLHn744RSLxTz22GM56KCDMnr0aMvPAAAAABpIg4dEl1xyScaPH59CoZDevXvnN7/5TXbaaac6+++zzz4544wz8sILL+SII47Ic889l6eeeiqXXXZZzjrrrIYuFwAAAKBJatCpOR9//HGuvvrqUkA0duzYegOi6j772c/mySefTO/evVMsFnPllVdmyZIlDVkuAAAAQJPVoCHRfffdl3feeSeFQiG//e1v06pVq9U6v1WrVvnNb36TQqGQd955J/fff38DVQoAAADQtDVoSPTEE08kWbaErFevXms0xk477ZR99903STJmzJh1VhsAAAAA/9KgIdEzzzyTQqGQvffee63G2XvvvVMsFvPMM8+so8oAAAAAqK5BQ6Jp06YlSXbZZZe1Gmf5+f/85z/XuiYAAAAAVtSgIdEHH3yQJNl8883Xapzl5y8fDwAAAIB1q0FDorlz5yZJ2rVrt1bjtG3bNkny4YcfrnVNAAAAAKyoQUOipUuXrtPxqqqq1ul4AAAAACzToCERAAAAABuGysa4yLXXXpsttthijc9/++2312E1AAAAAHxSo4RE//M//9MYlwEAAABgDTV4SFQsFhv6EgAAAACspQYNiR599NGGHB4AAACAdaRBQ6KBAwc25PAAAAAArCOebgYAAACAkAgAAAAAIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERI1mLPOOiuFQqH08dhjj630nFGjRmXIkCHp1q1bWrZsmW7dumXIkCEZNWrUKl93yZIlue6667Lnnnumc+fOad26dbbbbruccMIJefHFF9fiHQEAAAAbs8pyF7Axeu655/KLX/xilftXVVXl+OOPz8iRI2u8PmPGjMyYMSP33HNPhg4dml/96lepqKg715s9e3YGDRqUCRMm1Hj9tddey/XXX5+bb745w4cPz9ChQ1fvDQEAAAAbPTOJ1rHlgc+SJUuyxRZbrNI555xzTikg6t27d2677baMHz8+t912W3r37p0kGTFiRM4999w6x1i6dGmGDBlSCoi+/vWvZ9SoUXn66adz9dVXZ4sttsiiRYtywgknrNbMJAAAAKBpEBKtY1dffXUmTJiQXr165dhjj11p/ylTpuTyyy9Pkuy6664ZO3ZsDjvssPTt2zeHHXZYnnjiiey6665JkssuuyyvvPJKrePcfPPNeeKJJ5Ik3/3ud3PXXXflP/7jP7Lbbrvl5JNPztixY9O+fftUVVXllFNOyZIlS9bROwYAAAA2BkKideiNN97Ij3/84yTJddddlxYtWqz0nKuuuqoU2AwbNiytW7eucbxNmzYZNmxYkmX7DV155ZW1jrM8aOrQoUMuu+yyFY736NEjZ599dpLklVdeyd13372K7woAAABoCoRE69BJJ52UefPm5cgjj8zAgQNX2r9YLObee+9NkvTq1Su77757rf1233337LDDDkmSe++9N8ViscbxKVOmZNKkSUmSQw89NG3atKl1nKOOOqrUFhIBAAAA1QmJ1pE77rgj999/fzp06FCa1bMyr7/+et58880kWWmotPz4jBkzMnXq1BrHli8zW9k4W265ZXr27JkkGTt27CrVCAAAADQNQqJ14P3338+pp56aJLn00kvTqVOnVTrvpZdeKrV79epVb9/qx5fPGlqbcaZNm5b58+evUp0AAADAxq+y3AVsDM4888zMmjUr/fr1W6XNqpebPn16qd2tW7d6+3bv3r3UnjZt2lqPUywWM3369NIyttWttzYzZ84stRcsWJAFCxas8tjltHDhwlK7ffNiPT2h6al+T6zP98eG8v2GjUv1fz+qt4Fl3CNQN/cHa6uhfv4VEq2lMWPGZMSIEamsrMx1112XQqGwyud++OGHpXa7du3q7du2bdtSe968eQ0yzspUD6pWZvTo0as8o2p9cvouVeUuAdZb6/P98dBDD5W7BJq40aNHl7sEWK+5R6Bu7g/WxOzZsxtkXMvN1sLixYtz/PHHp1gs5vvf/34++9nPrtb51RPjlT0JrWXLlqX2JxPDdTUOAAAA0HSZSbQWfv7zn2fy5Mn59Kc/nZ/85CerfX6rVq1K7cWLF9fbd9GiRaV269at6x2n+uerM87KfHKZ2yfNnDkzu+22W5JkwIABK136tr5YuHBhKb2//P8qMvfjVZ8NBhu79s2LpRlE6/P9Me7svctdAk1Q9X8/BgwYUO+/v9AUuUegbu4P1tbKtoNZU0KiNTR58uRcfPHFSZJhw4bVWMa1qjbZZJNSe2VLv6pvMv3JJWWfHKe+bzD1jbMyqxP6tG7derVDqPXB3I8L+WDx+vlLMJTb+nx/bIjfb9i4tGrVytch1MM9AnVzf7AmGuprRki0hq688sosXrw42267bT766KP87ne/W6HPCy+8UGo/8sgjmTVrVpLkq1/9atq2bVsjdFlZClh9Fs8n9wb65Dj17QW0fJxCobDBzPQBAAAAGp6QaA0tX7b12muv5fDDD19p/wsuuKDUfv3119O2bdvstNNOpdcmT55c7/nVj++44441jn1ynC984QsrHad79+5rNPsJAAAA2DjZuLqMttlmm3Tt2jVJ8vjjj9fbd/l61a222ipbb711jWP9+/cvtesbZ9asWZkyZUqSpF+/fmtSMgAAALCREhKtoZtuuinFYrHej+qbWT/66KOl15eHPIVCIYMHD06ybIbPuHHjar3WuHHjSjOABg8enEKh5p4gPXv2LM0uuuOOO/LRRx/VWfNyQ4YMWaP3DQAAAGychERldtppp6VZs2ZJkpNPPnmFx9IvWLAgJ598cpKksrIyp512Wq3jnH766UmSOXPm5Mwzz1zh+KuvvlraaLtHjx5CIgAAAKAGIVGZ9ezZM2eccUaSZOLEienXr19uv/32TJw4Mbfffnv69euXiRMnJknOOOOMbL/99rWOc+SRR5aWkF1zzTU55JBD8uCDD2b8+PEZPnx4vvjFL2bu3LmpqKjI1VdfncpK21EBAAAA/yIpWA9cdNFFefvtt3PDDTfk2WefzWGHHbZCn2OPPTYXXnhhnWM0a9Ys99xzTwYNGpQJEybkrrvuyl133VWjT8uWLTN8+PDsv//+6/w9AAAAABs2IdF6oKKiIiNHjszBBx+c66+/PhMmTMjs2bPTqVOn9O3bNyeccMIqBTudOnXKk08+mV//+te59dZbM2nSpMyfPz9du3bN3nvvnVNPPTU777xzI7wjgMa19Y8eKHcJG7SplxxQ7hIAAFgPCIka0Pnnn5/zzz9/lfsPGjQogwYNWqtrVlZW5sQTT8yJJ564VuMAAAAATYs9iQAAAAAQEgEAAAAgJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQqK1NnHixPzsZz/Lfvvtl27duqVly5Zp165devbsmaOPPjpPPPHEao03atSoDBkypDRWt27dMmTIkIwaNWqVx1iyZEmuu+667LnnnuncuXNat26d7bbbLieccEJefPHF1X2LAAAAQBNQWe4CNmQDBgzImDFjVnh98eLFefnll/Pyyy/npptuyhFHHJFf//rXadGiRZ1jVVVV5fjjj8/IkSNrvD5jxozMmDEj99xzT4YOHZpf/epXqaioO9ubPXt2Bg0alAkTJtR4/bXXXsv111+fm2++OcOHD8/QoUNX890CAAAAGzMzidbCm2++mSTp2rVrTj311Pz+97/P+PHj89RTT+UXv/hFttpqqyTJLbfckqOOOqresc4555xSQNS7d+/cdtttGT9+fG677bb07t07STJixIice+65dY6xdOnSDBkypBQQff3rX8+oUaPy9NNP5+qrr84WW2yRRYsW5YQTTlitmUkAAADAxs9MorXQq1ev/PznP8/BBx+cZs2a1Ti2++6759vf/nb69euXKVOm5Lbbbst3vvOdDBgwYIVxpkyZkssvvzxJsuuuu2b06NFp3bp1kqRv37456KCDMnDgwEycODGXXXZZjjnmmPTo0WOFcW6++ebS8rbvfve7ueaaa0rHdtttt+y///7p06dP5s6dm1NOOSWTJk1KZaUvAQAAAMBMorVy//3359BDD10hIFquU6dOueKKK0qf//73v6+131VXXZUlS5YkSYYNG1YKiJZr06ZNhg0blmTZfkNXXnllreMsD5o6dOiQyy67bIXjPXr0yNlnn50keeWVV3L33XfX9/YAAACAJkRI1MD22muvUvvVV19d4XixWMy9996bZNnMpN13373WcXbffffssMMOSZJ77703xWKxxvEpU6Zk0qRJSZJDDz00bdq0qXWc6svehEQAAADAckKiBrZo0aJSu7YZR6+//nppb6OBAwfWO9by4zNmzMjUqVNrHKv+FLX6xtlyyy3Ts2fPJMnYsWPrLx4AAABoMmxI08Aef/zxUnvHHXdc4fhLL71Uavfq1avesaofnzRpUrbZZps1HmfKlCmZNm1a5s+fn7Zt29bbf7np06fXe3zmzJml9oIFC7JgwYJVGrfcFi5cWGq3b16spyc0PdXvCffHxmtD+X69vqn+70f1NrCMewTq5v5gbTXUz29CogZUVVWVSy65pPT5oYceukKf6sFLt27d6h2ve/fupfa0adPWepxisZjp06eXlrGtTPXrr8zo0aPTqVOnVe6/vjh9l6pylwDrLffHxuuhhx4qdwkbvNGjR5e7BFivuUegbu4P1sTs2bMbZFzLzRrQlVdemfHjxydZ9jj6Pn36rNDnww8/LLXbtWtX73jVZ/zMmzevQcYBAAAAmiYziRrI448/nh/96EdJki222CL/8z//U2u/6lMLW7RoUe+YLVu2LLU/ObVsXY1Tn0/OXvqkmTNnZrfddkuSDBgwYKUzmtYXCxcuLKX3l/9fReZ+XChzRbD+aN+8WJpB5P7YeI07e+9yl7BBqv7vx4ABA9KqVasyVwTrF/cI1M39wdpa2XYwa0pI1ABefPHFDBkyJEuWLEmrVq1y5513Zosttqi1b/VvBosXL6533OqbYLdu3brecer7JlPfOPVZndCndevWqzX2+mLux4V8sNgvwVAb98fGa0P8fr2+adWqlT9HqId7BOrm/mBNNNTXjOVm69jrr7+e/fbbL++9916aNWuW3/3udxkwYECd/TfZZJNSe2VLv+bPn19qf3JJ2boaBwAAAGiahETr0Jtvvpl99tknb775ZgqFQm644YYMHjy43nOqz85Z2XSx6su9PrmJ9JqMUygUNpglYQAAAEDDEhKtI7Nnz86+++6b1157LUkybNiwHHHEESs9b6eddiq1J0+eXG/f6sd33HHHtR6ne/fuNTaxBgAAAJouIdE68MEHH+QrX/lKXnrppSTJJZdckpNOOmmVzt1mm23StWvXJMs2u67P8o3Nttpqq2y99dY1jvXv37/Urm+cWbNmZcqUKUmSfv36rVKNAAAAwMZPSLSWPvrooxxwwAH529/+liQ555xzctZZZ63y+YVCobQkbfLkyRk3blyt/caNG1eaATR48OAUCjU3j+3Zs2dpdtEdd9yRjz76qNZxbrrpplJ7yJAhq1wnAAAAsHETEq2FxYsXZ8iQIRk7dmyS5NRTT82FF1642uOcdtppadasWZLk5JNPXuGx9AsWLMjJJ5+cJKmsrMxpp51W6zinn356kmTOnDk588wzVzj+6quv5uKLL06S9OjRQ0gEAAAAlFSWu4AN2eGHH56HHnooSfLlL385xx57bF544YU6+7do0SI9e/Zc4fWePXvmjDPOyCWXXJKJEyemX79+Oeuss7Lddtvl1VdfzaWXXppnn302SXLGGWdk++23r3X8I488MjfccEPGjh2ba665JrNmzcpxxx2XzTffPOPHj88FF1yQuXPnpqKiIldffXUqK/31AwAAAMtICdbCH/7wh1L7kUceyS677FJv/8985jOZOnVqrccuuuiivP3227nhhhvy7LPP5rDDDluhz7HHHlvvTKVmzZrlnnvuyaBBgzJhwoTcddddueuuu2r0admyZYYPH57999+/3loBAACApsVys/VERUVFRo4cmQceeCCDBw9O165d06JFi3Tt2jWDBw/On/70p4wYMSIVFfX/lXXq1ClPPvlkrr322vTv3z8dO3ZMq1atsu222+a4447LM888k6FDhzbSuwIAAAA2FGYSrYVisbjOxxw0aFAGDRq0VmNUVlbmxBNPzIknnriOqgIAAAA2dmYSAQAAACAkAgAAAEBIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECSynIXADQtU1t9s9wlNAlbL7y13CUAAAAbGDOJAAAAABASAQAAACAkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIEIiAAAAACIkAgAAACBCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIElluQuA9cm4lt9L64r3yl0GAAAANDoziQAAAAAwkwgAYG3tfvHD+WBxodxlbJCmXnJAuUsAAP5/ZhIBAAAAYCYRwMZoaqtvrpNxFjTfPA/ll0ns2VWbrRfeWu4SAABgnTGTCAAAAAAhEQAAAABCIgAAAAAiJAIAAAAgQiIAAAAAIiQCAAAAIElluQsAAKjX+ZuWu4LaNd88+ewvkyTjWn4vrSveK3NBa2frhbeWuwQAoMzMJAIAAABASAQAAACAkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAACIkAgAAACACIkAAAAAiJAIAAAAgAiJAAAAAIiQCAAAAIAIiQAAAABIUlnuAgBgQzW11TfLXcK6cX65CwAAYH1gJhEAAAAAQiIAAAAALDcDACBlXD55fnkuW1bnf1DuCgCgVmYSAQAAACAkAgAAAEBIBAAAAECERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAACSpLHcBAADQlGz9owfW6XhTLzlgnY4HQNNlJhEAAAAAQiIAAAAAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAAAkqSx3ATSMf/7zn7n66qvzwAMPZNq0aWnZsmW22267HHrooTnppJPSpk2bcpcIANAkTW31zXU74Pnrdrh1pvnmyWd/uaz9ix2Tj99r3Ouf/0HjXg9gIyAk2gjdd999+a//+q/MnTu39NpHH32UiRMnZuLEiRkxYkQeeOCB9OjRo4xVAgAAAOsTy802Ms8++2z+8z//M3Pnzk27du1y0UUX5cknn8zDDz+c4447LkkyZcqUHHDAAfnwww/LXC0AAACwvjCTaCNz6qmnZsGCBamsrMxDDz2UPfbYo3Tsy1/+crbffvuceeaZmTJlSq644oqcf/755SsWAAAAWG8IiTYi48ePz5gxY5Ikxx57bI2AaLkf/vCHufHGGzNp0qT88pe/zDnnnJPmzZs3dqkAANCgtv7RA6vUb+olBzRwJQAbDsvNNiL33HNPqX300UfX2qeioiJHHHFEkuT999/Po48+2hilAQAAAOs5M4k2Ik888USSpG3btunTp0+d/QYOHFhqjx07Nvvtt1+D1wYAAI1plZ8id36DlrHx8xQ52KgIiTYikyZNSpL06NEjlZV1/9X26tVrhXNWxfTp0+s9Pm3atFL79ddfz4IFC1Z57HJatGhRZs+enSR5dX7rtFyypMwVwfpjUWVr9wfUwf0B9XOPNBEvv1zuCjZINX4HefXVtGzZsswVsaGZNWtWqb1kHX5/LRSLxeI6G42yWbhwYVq3bp0kOeCAA3L//ffX279du3aZP39+dt999zz11FOrdI1CobDWdQIAAADrzvjx49O3b991MpY9iTYS1R9n365du5X2b9u2bZJk3rx5DVYTAAAAsOGw3GwjsXDhwlK7RYsWK+2/fDrj6iwJq76crK4aJk+enE996lPp3LlzvUve1iczZ87MbrvtlmRZAtulS5cyVwTrD/cH1M39AfVzj0Dd3B+srSVLluSdd95Jknzuc59bZ+NuGL/Fs1KtWrUqtRcvXrzS/osWLUqS0hK1VdGtW7eV9unRo8cqj7c+6tKlyyq9T2iK3B9QN/cH1M89AnVzf7Cmtt5663U+puVmG4lNNtmk1F6VJWTz589PsmpL0wAAAICNn5BoI9GqVat07NgxycqfQvbee++VQqLu3bs3eG0AAADA+k9ItBHZaaedkiSvvPJKvY/Amzx5cqm94447NnhdAAAAwPpPSLQR6d+/f5JlS8meeeaZOvs9/vjjpXa/fv0avC4AAABg/Sck2oh87WtfK7VvvPHGWvtUVVXllltuSZJsttlm2WuvvRqjNAAAAGA9JyTaiOy2227Zc889kyQjR47MU089tUKfK664IpMmTUqSnHrqqWnevHmj1ggAAACsnyrLXQDr1i9/+cv069cvCxYsyH777Zf/9//+X/baa68sWLAgv/vd73L99dcnSXr27Jkf/vCHZa4WAAAAWF8UisVisdxFsG7dd999+a//+q/MnTu31uM9e/bMAw88kB49ejRyZQAAAMD6Ski0kfrnP/+ZX/7yl3nggQcyffr0tGjRIj169Mg3vvGNfO9730ubNm3KXSIAAACwHhESAQAAAGDjagAAAACERAAAAABESAQAAABAhEQAAAAAREgEAAAAQIREAAAAAERIBAAAAECERAAAAABESAQAAABAhEQ0cf/85z/zwx/+ML169Urbtm3ToUOH9O3bN5dddlk++uijcpcHjW7ixIn52c9+lv322y/dunVLy5Yt065du/Ts2TNHH310nnjiiXKXCOuls846K4VCofTx2GOPlbskKLs33ngjP/nJT7Lrrrumc+fOadWqVbp3754999wz5513Xl544YVylwiNbvHixRkxYkS+8pWvpEuXLqWftXbYYYccffTRefLJJ8tdIk1coVgsFstdBJTDfffdl//6r//K3Llzaz3es2fPPPDAA+nRo0cjVwblMWDAgIwZM2al/Y444oj8+te/TosWLRqhKlj/Pffcc+nbt2+WLFlSeu3RRx/Nl770pfIVBWU2bNiwnH322Zk/f36dfU499dRcddVVjVcUlNk///nPHHDAAXnxxRfr7XfyySfnl7/8ZQqFQiNVBv9SWe4CoByeffbZ/Od//mcWLFiQdu3a5eyzz85ee+2VBQsW5He/+11+/etfZ8qUKTnggAMyceLEbLLJJuUuGRrcm2++mSTp2rVrvvGNb2TPPffMpz/96SxdujRPPfVUrrjiisyYMSO33HJLPv7449x6661lrhjKr6qqKscff3yWLFmSLbbYIm+//Xa5S4Kyu/DCC/PjH/84ybL/dDvuuOPSt2/fbLrppnn33Xfz7LPP5u67705FhUUNNB0ff/xxjYBol112yQ9+8IPssMMO+fDDD/PEE0/kiiuuyPz58zNs2LB07do1P/rRj8pcNU2RmUQ0SctnTFRWVmb06NHZY489ahy/7LLLcuaZZyZJfvKTn+T8888vQ5XQuA488MAcccQROfjgg9OsWbMVjs+ePTv9+vXLlClTkiSPP/54BgwY0Nhlwnrlqquuyve///306tUrQ4YMycUXX5zETCKarocffjj77LNPkmUzT0eMGJHmzZvX2nfx4sVmpdJk/P73v883vvGNJMkee+yRMWPGrPDz1jPPPJM99tgjH3/8cTbbbLO88847qaw0r4PGJb6nyRk/fnxpSc2xxx67QkCUJD/84Q+z4447Jkl++ctf5uOPP27UGqEc7r///hx66KG1BkRJ0qlTp1xxxRWlz3//+983VmmwXnrjjTdKsyWuu+46v+zS5FVVVeXEE09Mknz+85/PyJEj6wyIkrhnaFKq7zV09tln1/rzVp8+fXLggQcmSd5///1MmjSp0eqD5YRENDn33HNPqX300UfX2qeioiJHHHFEkmXfoB999NHGKA3We3vttVep/eqrr5axEii/k046KfPmzcuRRx6ZgQMHlrscKLuHHnooL7/8cpJlm7mbAQH/snjx4lJ72223rbPfdtttV+s50FiERDQ5y5/O1LZt2/Tp06fOftV/4B87dmyD1wUbgkWLFpXadc04gqbgjjvuyP33358OHTrk8ssvL3c5sF648847kySFQqE0GyJJ5syZk5dffjlz5swpV2lQdjvssEOp/dprr9XZb/l/whUKhWy//fYNXhd8kpCIJmf5tM0ePXrU+z9cvXr1WuEcaOoef/zxUnv5kkxoat5///2ceuqpSZJLL700nTp1KnNFsH4YN25ckmTrrbfOJptskltvvTWf+9zn0rFjx/Ts2TMdO3bMDjvskMsvv7zGfzpAU3D44Yenffv2SZb927F06dIV+jz77LN54IEHkiTf/OY3S/2hMQmJaFIWLlyY2bNnJ0m6detWb9/NN988bdu2TZJMmzatwWuD9V1VVVUuueSS0ueHHnpoGauB8jnzzDMza9as9OvXL8cee2y5y4H1QlVVVSZPnpxk2R52p556ar71rW/lhRdeqNFvypQpOeOMM/LlL38577//fhkqhfLo1KlTfvOb36RNmzYZO3Zs+vbtm1tuuSXjxo3LX//61/z0pz/NwIEDs3jx4vzbv/1bjX0goTEJiWhSPvzww1K7Xbt2K+2/PCSaN29eg9UEG4orr7wy48ePT5J8/etfr3e5JmysxowZkxEjRqSysjLXXXddCoVCuUuC9cIHH3yQqqqqJMnzzz+fq6++Ol26dMlvf/vbzJkzJx999FEef/zx7L777kmWbeJ7zDHHlLNkaHQHHXRQnnnmmQwdOjTPPfdcjjzyyOyxxx7Zd999c/7556dNmza56qqrMmbMmHzqU58qd7k0UUIimpSFCxeW2qvyRI2WLVsmSRYsWNBgNcGG4PHHH8+PfvSjJMkWW2yR//mf/ylzRdD4Fi9enOOPPz7FYjHf//7389nPfrbcJcF6Y/78+aX2woUL06ZNmzz66KP51re+lc033zytW7fOgAED8sgjj+Tzn/98kuTuu+/O008/Xa6SodEtXrw4t9xyS+69994Ui8UVjr/11lv57W9/m7/+9a9lqA6WERLRpLRq1arUXpWnBSxfL9+6desGqwnWdy+++GKGDBmSJUuWpFWrVrnzzjuzxRZblLssaHQ///nPM3ny5Hz605/OT37yk3KXA+uV6j9jJcnQoUNrbNS7XOvWrXPRRReVPr/99tsbvDZYH8yfPz/77LNPLr744syZMydnnnlmJk2alEWLFuWDDz7IQw89lP79+2fixIn52te+ll/84hflLpkmSkhEk7LJJpuU2quyhGz5/4qtytI02Bi9/vrr2W+//fLee++lWbNm+d3vfpcBAwaUuyxodJMnT87FF1+cJBk2bFhpOTKwTPWfsZJkv/32q7Pv3nvvXXp4yIQJExq0LlhfnH/++RkzZkySZOTIkbn00kvTq1evtGjRIu3bt8++++6bRx99NHvttVeKxWLOOOOM/P3vfy9z1TRFdT/aCTZCrVq1SseOHfPuu+9m+vTp9fZ97733SiFR9+7dG6M8WK+8+eab2WefffLmm2+mUCjkhhtuyODBg8tdFpTFlVdemcWLF2fbbbfNRx99lN/97ncr9Km+Qe8jjzySWbNmJUm++tWvCpXY6LVs2TKdO3fOO++8k6T+n51atWqVTp06ZdasWaX+sDErFou54YYbkiQ9e/bMkUceWWu/ysrKXHDBBenfv3+qqqpy00035corr2zMUkFIRNOz0047ZcyYMXnllVeyZMmS0v9kfdLyJ3QkHvVN0zN79uzsu+++ee2115IsmzlxxBFHlLkqKJ/ly49fe+21HH744Svtf8EFF5Tar7/+upCIJmHnnXfOY489liS1Pt67uuXH6/o5DDYmb731VubMmZMk6d27d719qz8YpPrvI9BYLDejyenfv3+SZUvJnnnmmTr7Pf7446V2v379GrwuWF988MEH+cpXvpKXXnopSXLJJZfkpJNOKnNVAKzvqi9HXv6fDLWZO3duZs+enSTZaqutGrwuKLfqYeiSJUvq7fvxxx/Xeh40FiERTc7Xvva1UvvGG2+stU9VVVVuueWWJMlmm22WvfbaqzFKg7L76KOPcsABB+Rvf/tbkuScc87JWWedVeaqoPxuuummFIvFej+qb2b96KOPll7feuuty1c4NKKDDz641L777rvr7Hf33XeXnuy05557NnhdUG4dOnRI+/btkyRPPfVUvUFR9f+o3mabbRq8NvgkIRFNzm677Vb6gWTkyJF56qmnVuhzxRVXZNKkSUmSU089Nc2bN2/UGqEcFi9enCFDhmTs2LFJln3tX3jhhWWuCoANxS677JL9998/SXLbbbfl4YcfXqHPrFmzcu655yZJWrRokaOPPrpRa4RyqKioyAEHHJBk2Z6P1Z/wV917771X4z/nDjzw/2vv7oOiuu4/jn8WFWEXUBEQgowPVZSmREXzMKM1yiSZBtQoYh0wiClKg2KDMTGtjYLJmLS1Y6E2VpFUSTPSisWHam0jEWuDiULkuUHSqcYYUdukRHwAA9zfH5QtuiygAusvvl8zO3PZc86937vLjPrx3HOm9Uh9QGsmoyXGB+4hRUVFmjhxoq5duyY3NzetXLlSU6dO1bVr1/S73/1O6enpkpoXlissLLTZsQP4Opo9e7ZycnIkSaGhoUpNTZXJZLLb39nZWYGBgT1VHnDXS0lJ0Zo1ayQ1zySaMmWKYwsCHKCqqkoPP/ywampq5OLioqSkJIWFhcnV1VXHjx/X66+/bt085Kc//alWrFjh4IqBnlFZWanx48fr6tWrkpo3NYiNjdXw4cNVV1enDz74QKmpqTpz5oyk5l0Ac3NzHVky7lGERLhn/fGPf9TTTz+tS5cutdkeGBio/fv3a8SIET1cGeAY7QVCbRkyZIhOnz7dPcUA/w8REgHN3nvvPUVGRurChQtttptMJv34xz++YYF34F6Qm5urqKgo65pc9oSGhmrnzp0aMGBAD1UG/A8rYeGeNX36dJWWliotLU379+/X2bNn5ezsrBEjRmjOnDlKTEyU2Wx2dJkAAAD/r0yaNEkVFRXasGGDdu/erVOnTun69evy8/PTlClTtHTp0g53eAK+jh577DFVVlbqzTff1IEDB1RRUaGamhr17t1bvr6+evDBBxUdHa0ZM2bc8n/eAV2FmUQAAAAAAABg4WoAAAAAAAAQEgEAAAAAAECERAAAAAAAABAhEQAAAAAAAERIBAAAAAAAABESAQAAAAAAQIREAAAAAAAAECERAAAAAAAAREgEAAAAAAAAERIBAAAAAABAhEQAAAAAAAAQIREAAAAAAABESAQAAAAAAAAREgEAAAAAAECERAAAAAAAABAhEQAAAAAAAERIBAAA0O1MJpNMJpNSUlIcXcpdq7GxUWlpaXrooYfk4eFh/cxmzpzp6NI67fTp09a6t23bZtO+bds2a/vp06d7vD4AADpCSAQAALrN4cOHrf8oNplMmjt3bodjFixYYO2Pe0dUVJSSkpJUUFCg2tpaR5cDAMA9iZAIAAD0mOzsbJWVlTm6DNxljh49quzsbElSeHi4Dh48qNLSUpWVlemXv/ylg6sDAODe0dvRBQAAgHuHYRhKTk5WTk6Oo0vBXSQ3N1eS1KtXL23fvl0eHh4OrggAgHsTM4kAAECP8PLykiTt2rVLRUVFDq4Gd5PPPvtMkjRo0CACIgAAHIiQCAAA9Igf/OAH6tu3ryRp9erVDq4Gd5P6+npJUp8+fRxcCQAA9zZCIgAA0CMCAgIUHx8vSdq3b5+OHz9+W+cZOnSoTCaTFixY0G6/lgWwhw4datPW1i5UOTk5euKJJ+Tj4yOLxaIxY8Zow4YN+uqrr6zjDMPQ9u3bNWXKFPn4+MhsNiskJESbNm2SYRidvofc3FzNmDFDfn5+cnFx0fDhw5WYmGidUdOREydO6Nlnn9WoUaPk5uYmi8WiUaNGKSEhQVVVVXbH3by7Vn19vVJTU/XII4/Iy8vrjnZgKysrU3x8vEaOHCmz2Sx3d3fdf//9WrZsmd2dvFpqyczMlCR98sknNyx0fieLl+fn52vhwoUaNWqUPDw85OzsrMGDB2vatGl64403VFNTYzOmurpaGzduVGRkpEaOHCmLxaK+ffvK399fTz31lH7/+9+rqanptmvqjA8//FBxcXEKDAyUxWKRi4uLAgICNH78eC1ZskR79+69pd81AABuiQEAANBN8vLyDEmGJGPr1q3GuXPnDFdXV0OS8cQTT7Q5JjY21jqmLUOGDDEkGbGxse1eu+U8Q4YMsWk7derUDXUlJCRYf775FRERYTQ0NBh1dXVGZGSk3X6LFi2yW0tLn+TkZCMlJcXuOfr162ccOXLE7nkaGxuNZcuWGSaTye45evfubWzevLnN8Vu3brX2KygoMMaOHWszPjk5ud3PtS2vvfaa4eTkZLemvn37GpmZmXY/l/Zet+rq1atGVFRUh+e9+T4bGhravYeW1+OPP27U1ta2ee2bf69u1vrzP3XqlE37+vXrO1WDvesDAHCnWLgaAAD0GD8/PyUkJGj9+vV655139N5772nSpEkOrWnTpk06duyYwsLCtHDhQg0ZMkSffvqpXn/9dR07dkw5OTnaunWrSktLtXPnTkVHRys6Olp+fn76+OOPlZKSosrKSm3ZskURERH6zne+Y/da+/fvV2FhoUaNGqUVK1bogQce0Jdffqns7Gxt2bJFX375paZNm6by8nIFBATYjF+6dKk2btwoSZo8ebIWLFig4cOHy2w2q6SkRKmpqaqoqND3v/99+fr6asaMGXZriYuLU1lZmebPn6+5c+fK19dXZ86csT4S2FkbN27UypUrJUne3t566aWXNHHiRDU2Nio3N1fr1q3TlStXtGDBAnl5eSksLMw6tmWnu5dffll79uzRfffdp7/85S+3dP3Wmpqa9NRTT+ngwYOSpJEjR2rx4sWaMGGCzGazqqurdfToUe3YscNmrPHf2TmhoaF68sknFRwcLG9vb9XW1uqf//yntmzZovfff18HDx7UkiVLrLOfukppaaleeOEFNTU1adiwYUpMTNTYsWPl6emp2tpanTx5Unl5edqzZ0+XXhcAgBs4OqUCAABfXzfPJDIMw7hw4YJhsVgMScbUqVNtxvT0TCJJRlJSkk2fK1euWK81cOBAw2QyGampqTb9qqurDXd3d0OSMWPGjDZraX2tkJCQNmeCvPXWW9Y+c+bMsWl/5513rO0ZGRltXufatWtGaGio9b6/+uqrG9pbz2Rp7zyddfHiRcNsNhuSjPvuu884c+aMTZ8TJ05Yv29/f3/j+vXrNn3a+65uRVpamvXeZs2aZdTV1bXZr7Gx0Th79uwN7zU1NRkff/xxu+dfvXq1IckwmUxGVVWVTfudzCRatWqVIcmwWCzG+fPn7dZQU1NjNDY2tlsnAAC3izWJAABAj/Lx8VFiYqIkKS8vT3l5eQ6tJyAgQD/72c9s3jebzYqNjZUkff7553r44Yf13HPP2fTz9fXVrFmzJEl/+9vfOrxeenq63NzcbN6PiYnRk08+Kal5B7jz58/f0P6Tn/xEkjR79mzFxcW1eW4XFxf96le/ktS8vk97n21oaKjd83TW1q1bdfXqVUnS+vXr25z9NG7cOP3oRz+S1LyL2e7du+/omvY0NTVp3bp1kqTBgwfrrbfesjsrysnJSf7+/je8ZzKZNGLEiHavsXr1anl5eckwDO3du7drCv+vlu87MDBQgwYNstuvX79+cnLir/AAgO7BnzAAAKDHvfjii3J3d5ckrVq1yqG1RERE2N1Va8yYMdbjuXPn2j1HS7///Oc/bS6I3CI4OFjjx4+32/69731PktTQ0KDDhw9b37906ZL158jISLvjJSkoKEheXl6SpPfff99uv3nz5rV7ns7Izc2VJPXv318RERF2+y1cuNBmTFcrLi7W2bNnJUmLFi1qM4i7FU1NTTp37pxOnjyp8vJylZeX66OPPtLgwYMlSSUlJXdcc2t+fn6SpL///e+3vag7AAB3ipAIAAD0uIEDByopKUlS8y5Ud7IOzZ0KDAy029a/f/9b7ldbW2u334MPPthuLQ899JD1uGW9HkkqKiqy7qoVFRVlswPYza9///vfkmQzG6m1Bx54oN1aOqO8vFySFBIS0u729YMGDbLuMtcypqsVFRVZj7/97W/f1jkMw9Dbb7+tqVOnys3NTf7+/ho9erSCg4Otr+LiYkmyfsZdJSoqSn369FF9fb0mTpyo6dOna9OmTSovL2c3MwBAjyEkAgAADvH8889bw5Xk5GSH1WE2m+22tX6sp7P9Ghsb7fbz8fFpt5bWjxl98cUX1uOLFy+2O86elkfB2jJgwIDbOmdrLTV2dF9S82N5rcd0tdahTcusnFtRV1en8PBwxcTE6PDhw7p27Vq7/Ttqv1WjR49WVlaWBgwYoIaGBu3bt08JCQkKDg6Wj4+PYmJiOvU4IwAAd4KQCAAAOET//v31/PPPS5KOHTumffv2Obii7mcymW5rXOvgafPmzSorK+vUa+3atXbP2atXr9uqpS23e193k7Vr1+rAgQOSpEcffVQ7duzQP/7xD12+fFmNjY0yDEOGYVhnKXXH7J7Zs2fr1KlT2rx5syIiIuTt7S2pOQB7++23rTvatcwqAwCgq/V2dAEAAODelZSUpLS0NH3++edKTk7WtGnTOhzTMmuno38oX7lypUtq7EoXLlzodLunp6f1eODAgdZjs9msb33rW11f3G3w9PRUdXV1h/cl/e/Rt9b31ZVa1mGSpOrqao0ePbrTYw3DUEZGhqTmR9UOHTpkd3Ho7poJ1aJfv36Kj49XfHy8JOmjjz7Snj17tGHDBp07d06ZmZkaN25cm4uoAwBwp5hJBAAAHMbd3V0vvviiJOnEiRPatWtXp8ZIzYtEt6eqqurOC+xiBQUFnW5vHQSNHTvWOlsnPz+/e4q7DS01njhxQg0NDXb7Xbx4UZ988skNY7paSEiI9fjIkSO3NPaLL76whlhz5syxGxBdvnxZJ0+evP0ib0NQUJB++MMf6oMPPpDFYpEk7dixo0drAADcOwiJAACAQyUmJlrXtElOTu7wMZ5hw4ZJag4m7PWtqKhQaWlp1xbaBcrKym5YYPlmv/nNbyQ1Pwo2ZcoU6/ve3t565JFHJEnbt2/Xv/71r26ts7Mee+wxSVJNTY1ycnLs9nvzzTet31XLmK42ZswYBQQESJIyMjJ0+fLlTo9tHXC1NwMtIyOj3TCsOwUEBFgXT+/qRbMBAGhBSAQAABzKYrHopZdektQcovzpT39qt/+jjz4qSTp37pyysrJs2mtraxUXF9f1hXaR+Pj4NoOI7du3W+995syZNosvv/zyy5KkS5cuKTIyUjU1NXavUV9frzfeeEN1dXVdV3gbnnnmGeuC3suXL9dnn31m06ekpESvvfaaJMnf318zZ87sllqcnJyss9LOnj2r+fPn6/r16232bdnevoW3t7d1EfWsrCzV19fbjCkoKNCqVau6vvD/2r17d7vf6aeffqrKykpJ/wtKAQDoaoREAADA4RISEqyhSEezJJ5++ml5eHhIkuLi4vTKK6/o2LFjOn78uH79618rJCREJSUlGjduXLfXfasmTJigwsJCTZgwQdu2bdOHH36oQ4cOafHixYqJiZHU/Djdz3/+c5uxYWFh1nVojhw5oqCgIK1Zs0bvvvuuiouLlZ+fr8zMTC1cuFB+fn5KTEzs9lkv3t7eWrdunaTmYGb8+PFKTU3V8ePHdfToUb3yyiuaNGmSLl++LJPJpPT0dPXp06fb6lmyZIkef/xxSdKuXbsUHBystLQ05efnq6ioSAcOHFBycrJGjx6t9PR06zgnJyfNmzdPklRaWqpJkyYpKytLhYWFevfdd7V8+XJNnjxZLi4u1tk8XS01NVX+/v767ne/q02bNumvf/2riouLlZeXp3Xr1mnixInWHdWeffbZbqkBAAAWrgYAAA7n6uqqlStXaunSpR329fb2VkZGhqKiolRXV6fk5GQlJyffcK7f/va32rdvX7uPdjlCeHi4wsPDtWbNGj3zzDM27R4eHtq7d6+GDh3a5vhf/OIX8vT01Kuvvqrz588rJSXF7rUsFkuX7mBmz+LFi1VTU6NVq1bpwoULWrZsmU2fvn37Kj09XWFhYd1ai5OTk3bv3q3Y2Fjt3LlTVVVVSkpK6tTYtWvXKj8/X8XFxSosLFR0dPQN7Z6envrDH/6g1atXd9t6V1evXlV2drays7PbbHdyctKaNWu6bTYWAADMJAIAAHeFRYsWWdeU6cicOXN09OhRzZo1S97e3nJ2dlZAQIBiY2NVUFCgyMjIbq729qWkpOjPf/6zwsPDNWjQIDk7O2vo0KFavHixKioqrI/TtcVkMllDihUrVmjChAny9PRUr1695O7urm9+85uaN2+eMjMzVV1dLVdX1x65p5UrV6qoqEiLFi3SN77xDbm6uspisSgoKEjPPfecKisrNX/+/B6pxWw2Kzs7W4cOHVJMTIyGDRsmV1dX6+/I9OnTtXnzZi1fvvyGcf369VN+fr5effVVBQcHy8XFRW5ubgoKCtILL7ygkpISTZ48udvqzsrKUnp6uqKjozV27Fj5+vqqd+/ecnNz0/3336+EhAQVFRVZHzsEAKA7mIyOVocEAAAAAADA1x4ziQAAAAAAAEBIBAAAAAAAAEIiAAAAAAAAiJAIAAAAAAAAIiQCAAAAAACACIkAAAAAAAAgQiIAAAAAAACIkAgAAAAAAAAiJAIAAAAAAIAIiQAAAAAAACBCIgAAAAAAAIiQCAAAAAAAACIkAgAAAAAAgAiJAAAAAAAAIEIiAAAAAAAAiJAIAAAAAAAAIiQCAAAAAACACIkAAAAAAAAgQiIAAAAAAACIkAgAAAAAAAAiJAIAAAAAAIAIiQAAAAAAACBCIgAAAAAAAIiQCAAAAAAAAJL+D3sGAnVFMr2pAAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {
"image/png": {
"height": 432,
"width": 580
}
},
"output_type": "display_data"
}
],
"source": [
"telecom_data.loc[telecom_data[\"Churn\"] == False, \"Customer service calls\"].hist(\n",
" label=\"Loyal\"\n",
")\n",
"telecom_data.loc[telecom_data[\"Churn\"] == True, \"Customer service calls\"].hist(\n",
" label=\"Churn\"\n",
")\n",
"plt.xlabel(\"Number of calls\")\n",
"plt.ylabel(\"Density\")\n",
"plt.legend()"
]
},
{
"cell_type": "markdown",
"id": "74d5c1ba",
"metadata": {},
"source": [
"Looks like loyal customers make fewer calls to customer service than those who eventually leave. Now, it might be a good idea to estimate the average number of customer service calls in each group. Since our dataset is small, we would not get a good estimate by simply calculating the mean of the original sample. We will be better off applying the bootstrap method. Let's generate 1000 new bootstrap samples from our original population and produce an interval estimate of the mean."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "0bc8bc53",
"metadata": {
"attributes": {
"classes": [
"code-cell"
],
"id": ""
}
},
"outputs": [],
"source": [
"def get_bootstrap_samples(data, n_samples):\n",
" \"\"\"Generate bootstrap samples using the bootstrap method.\"\"\"\n",
" indices = np.random.randint(0, len(data), (n_samples, len(data)))\n",
" samples = data[indices]\n",
" return samples\n",
"\n",
"\n",
"def stat_intervals(stat, alpha):\n",
" \"\"\"Produce an interval estimate.\"\"\"\n",
" boundaries = np.percentile(stat, [100 * alpha / 2.0, 100 * (1 - alpha / 2.0)])\n",
" return boundaries"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "f95f6c16",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Service calls from loyal: mean interval [1.4077193 1.49473684]\n",
"Service calls from churn: mean interval [2.0621118 2.39761905]\n"
]
}
],
"source": [
"# Save the data about the loyal and former customers to split the dataset\n",
"loyal_calls = telecom_data.loc[\n",
" telecom_data[\"Churn\"] == False, \"Customer service calls\"\n",
"].values\n",
"churn_calls = telecom_data.loc[\n",
" telecom_data[\"Churn\"] == True, \"Customer service calls\"\n",
"].values\n",
"\n",
"# Set the seed for reproducibility of the results\n",
"np.random.seed(0)\n",
"\n",
"# Generate the samples using bootstrapping and calculate the mean for each of them\n",
"loyal_mean_scores = [\n",
" np.mean(sample) for sample in get_bootstrap_samples(loyal_calls, 1000)\n",
"]\n",
"churn_mean_scores = [\n",
" np.mean(sample) for sample in get_bootstrap_samples(churn_calls, 1000)\n",
"]\n",
"\n",
"# Print the resulting interval estimates\n",
"print(\n",
" \"Service calls from loyal: mean interval\", stat_intervals(loyal_mean_scores, 0.05)\n",
")\n",
"print(\n",
" \"Service calls from churn: mean interval\", stat_intervals(churn_mean_scores, 0.05)\n",
")"
]
},
{
"cell_type": "markdown",
"id": "53d724e8",
"metadata": {},
"source": [
"For the interpretation of confidence intervals, you can address this [concise note](https://www.graphpad.com/guides/prism/7/statistics/stat_more_about_confidence_interval.htm?toc=0&printWindow) or any course on statistics. It's not correct to say that a confidence interval contains 95% of values. Note that the interval for the loyal customers is narrower, which is reasonable since they make fewer calls (0, 1 or 2) in comparison with the churned clients who call until they are fed up and decide to switch providers. "
]
},
{
"cell_type": "markdown",
"id": "80bb5376",
"metadata": {},
"source": [
"## Bagging\n",
"\n",
"Now that you've grasped the idea of bootstrapping, we can move on to *bagging*. \n",
"\n",
"Suppose that we have a training set $\\large X$. Using bootstrapping, we generate samples $\\large X_1, \\dots, X_M$. Now, for each bootstrap sample, we train its own classifier $\\large a_i(x)$. The final classifier will average the outputs from all these individual classifiers. In the case of classification, this technique corresponds to voting:\n",
"$$\\large a(x) = \\frac{1}{M}\\sum_{i = 1}^M a_i(x).$$\n",
"\n",
"The picture below illustrates this algorithm:"
]
},
{
"cell_type": "markdown",
"id": "bdaf7705",
"metadata": {},
"source": [
":::{figure} https://static-1300131294.cos.ap-shanghai.myqcloud.com/images/ml-advanced/ensemble-learning/bagging/Illustration_of_Bagging.png\n",
"---\n",
"name: 'illustration of bagging'\n",
"width: 90%\n",
"---\n",
"Illustration of Bagging\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "ddfc652f",
"metadata": {},
"source": [
"Let's consider a regression problem with base algorithms $\\large b_1(x), \\dots , b_n(x)$. Assume that there exists an ideal target function of true answers $\\large y(x)$ defined for all inputs and that the distribution $\\large p(x)$ is defined. We can then express the error for each regression function as follows: \n",
"\n",
"$$\\large \\varepsilon_i(x) = b_i(x) - y(x), \\quad i = 1, \\dots, n$$\n",
"\n",
"And the expected value of the mean squared error: \n",
"\n",
"$$\\large E_x\\left[\\left(b_i(x) - y(x)\\right)^{2}\\right] = E_x\\left[\\varepsilon_i^{2}(x)\\right]$$\n",
"\n",
"Then, the mean error over all regression functions will look as follows:\n",
"\n",
"$$\\large E_1 = \\frac{1}{n} E_x\\left[ \\sum_i^n \\varepsilon_i^{2}(x)\\right]$$\n",
"\n",
"We'll assume that the errors are unbiased and uncorrelated, that is: \n",
"\n",
"$$\\large \\begin{array}{rcl} E_x\\left[\\varepsilon_i(x)\\right] &=& 0, \\\\\n",
"E_x\\left[\\varepsilon_i(x)\\varepsilon_j(x)\\right] &=& 0, \\quad i \\neq j. \\end{array}$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let's construct a new regression function that will average the values from the individual functions: \n",
"\n",
"$$\\large a(x) = \\frac{1}{n}\\sum_{i=1}^{n}b_i(x)$$\n",
"\n",
"Let's find its mean squared error:\n",
"\n",
"$$\\large \\begin{array}{rcl} E_n &=& E_x\\left[\\frac{1}{n}\\sum_{i=1}^{n}b_i(x)-y(x)\\right]^2 \\\\\n",
"&=& E_x\\left[\\frac{1}{n}\\sum_{i=1}^{n}\\varepsilon_i\\right]^2 \\\\\n",
"&=& \\frac{1}{n^2} E_x\\left[\\sum_{i=1}^{n}\\varepsilon_i^2(x) + \\sum_{i \\neq j}\\varepsilon_i(x)\\varepsilon_j(x)\\right] \\\\\n",
"&=& \\frac{1}{n} E_1\\end{array}$$\n",
"\n",
"Thus, by averaging the individual answers, we reduced the mean squared error by a factor of $\\large n$.\n",
"\n",
"From our previous section, let's recall the components that make up the total out-of-sample error:\n",
"\n",
"$$\\large \\begin{array}{rcl} \n",
" Err\\left(\\vec{x}\\right) &=& E\\left[\\left(y - \\hat{f}\\left(\\vec{x}\\right)\\right)^2\\right] \\\\\n",
"&=& \\sigma^2 + f^2 + Var\\left(\\hat{f}\\right) + E\\left[\\hat{f}\\right]^2 - 2fE\\left[\\hat{f}\\right] \\\\\n",
"&=& \\left(f - E\\left[\\hat{f}\\right]\\right)^2 + Var\\left(\\hat{f}\\right) + \\sigma^2 \\\\\n",
"&=& Bias\\left(\\hat{f}\\right)^2 + Var\\left(\\hat{f}\\right) + \\sigma^2\n",
"\\end{array}$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Bagging reduces the variance of a classifier by decreasing the difference in error when we train the model on different datasets. In other words, bagging prevents overfitting. The efficiency of bagging comes from the fact that the individual models are quite different due to the different training data and their errors cancel each other out during voting. Additionally, outliers are likely omitted in some of the training bootstrap samples.\n",
"\n",
"The `scikit-learn` library supports bagging with meta-estimators `BaggingRegressor` and `BaggingClassifier`. You can use most of the algorithms as a base.\n",
"\n",
"Let's examine how bagging works in practice and compare it with a decision tree. For this, we will use an example from [sklearn's documentation](http://scikit-learn.org/stable/auto_examples/ensemble/plot_bias_variance.html#sphx-glr-auto-examples-ensemble-plot-bias-variance-py)."
]
},
{
"cell_type": "markdown",
"id": "52f342e0",
"metadata": {},
"source": [
":::{figure} https://static-1300131294.cos.ap-shanghai.myqcloud.com/images/ml-advanced/ensemble-learning/bagging/Comparison_of_Bagging_and_Tree.png\n",
"---\n",
"name: 'comparison of bagging and tree'\n",
"width: 90%\n",
"---\n",
"Comparison of Bagging and Tree\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "85f44f0e",
"metadata": {},
"source": [
"The error for the decision tree:\n",
"\n",
"$$ \\large 0.0255 \\, (Err) = 0.0003 \\, (Bias^2) + 0.0152 \\, (Var) + 0.0098 \\, (\\sigma^2) $$\n",
"\n",
"The error when using bagging:\n",
"\n",
"$$ \\large 0.0196 \\, (Err) = 0.0004 \\, (Bias^2) + 0.0092 \\, (Var) + 0.0098 \\, (\\sigma^2) $$ "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you can see from the graph above, the variance in the error is much lower for bagging. Remember that we have already proved this theoretically.\n",
"\n",
"Bagging is effective on small datasets. Dropping even a small part of training data leads to constructing substantially different base classifiers. If you have a large dataset, you would generate bootstrap samples of a much smaller size.\n",
"\n",
"The example above is unlikely to be applicable to any real work. This is because we made a strong assumption that our individual errors are uncorrelated. More often than not, this is way too optimistic for real-world applications. When this assumption is false, the reduction in error will not be as significant. In the following sections, we will discuss some more sophisticated ensemble methods, which enable more accurate predictions in real-world problems."
]
},
{
"cell_type": "markdown",
"id": "cd120ad8",
"metadata": {},
"source": [
"## Out of bag error\n",
"\n",
"In case of Random Forest, there is no need to use cross-validation or hold-out samples in order to get an unbiased error estimation. Why? Because, in ensemble techniques, the error estimation takes place internally.\n",
" \n",
"Random trees are constructed using different bootstrap samples of the original dataset. Approximately 37% of inputs are left out of a particular bootstrap sample and are not used in the construction of the $\\large k$-th tree.\n",
"\n",
"This is easy to prove. Suppose there are $\\large \\ell$ examples in our dataset. At each step, each data point has equal probability of ending up in a bootstrap sample with replacement, probability $\\large\\frac{1}{\\ell}.$ The probability that there is no such bootstrap sample that contains a particular dataset element (i.e. it has been omitted $\\large \\ell$ times) equals $\\large (1 - \\frac{1}{\\ell})^\\ell$. When $\\large \\ell \\rightarrow +\\infty$, it becomes equal to the [Second Remarkable Limit](https://en.wikipedia.org/wiki/List_of_limits) $\\large \\frac{1}{e}$. Then, the probability of selecting a specific example is $\\large \\approx 1 - \\frac{1}{e} \\approx 63\\%$.\n",
"\n",
"Let's visualize how **O**ut-**o**f-**B**ag **E**rror (or OOBE) estimation works:"
]
},
{
"cell_type": "markdown",
"id": "181da1e6",
"metadata": {},
"source": [
":::{figure} https://static-1300131294.cos.ap-shanghai.myqcloud.com/images/ml-advanced/ensemble-learning/bagging/Principle_of_OOBE.png\n",
"---\n",
"name: 'principle of OOBE'\n",
"width: 90%\n",
"---\n",
"Principle of OOBE\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "197e7e57",
"metadata": {},
"source": [
"The top part of the figure above represents our original dataset. We split it into the training (left) and test (right) sets. In the left image, we draw a grid that perfectly divides our dataset according to classes. Now, we use the same grid to estimate the share of the correct answers on our test set. We can see that our classifier gave incorrect answers in those 4 cases that have not been used during training (on the left). Hence, the accuracy of our classifier is $\\large \\frac{11}{15}*100\\% = 73.33\\%$.\n",
"\n",
"To sum up, each base algorithm is trained on $\\large \\approx 63\\%$ of the original examples. It can be validated on the remaining $\\large \\approx 37\\%$. The Out-of-Bag estimate is nothing more than the mean estimate of the base algorithms on those $\\large \\approx 37\\%$ of inputs that were left out of training."
]
},
{
"cell_type": "markdown",
"id": "321d7028",
"metadata": {},
"source": [
"## Your turn! 🚀\n",
"\n",
"TBD"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Acknowledgments\n",
"\n",
"Thanks to [Yury Kashnitsky](https://www.kaggle.com/kashnitsky) for creating the open-source content [Ensembles and random forest](https://www.kaggle.com/code/kashnitsky/topic-5-ensembles-part-1-bagging/notebook). They inspire the majority of the content in this chapter."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.19"
}
},
"nbformat": 4,
"nbformat_minor": 5
}