\n",
"\n",
"Documentation can be found here:\n",
"http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier\n",
" \n",
"Also, here is another useful example:\n",
"http://nbviewer.ipython.org/github/sujitpal/statlearning-notebooks/blob/master/src/chapter8.ipynb\n",
"\n",
"
The above is a pretty simple tree, which we can see in the above plot.
\n",
"\n",
"As we allow for more complex trees, visualing the tree becomes very cumbersome, but we can still look at the decision surface. \n",
"\n",
"\n",
"
As we let the trees grow without any restrictions, we notice that the decision surface gets very fragmented. This defies our intuition that nearest neighbors of a point should have the same class assignment. Since we know that this data was generated from a few simple normal mixtures, we should understand that these more complex surfaces are terribly over fit.\n",
"
\n",
"Beyond toy examples, let's see this on real data.\n",
"
\n",
"\n",
"### Decision Tree on Real Data\n",
"
We don't just want to build a tree, we want to optimize the configuration of the tree. We'll use AUC as a metric on our test set and vary the max_depth and min_leaf_size.
\n",
"\n",
"The underlying data has a very rare outcome (less than 5%) so we'll down sample the data first to reach a 50/50 split.\n",
"\n",
"\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from sklearn.metrics import confusion_matrix, roc_auc_score\n",
"import sys\n",
"#Get the datadir path set up - note this may only work in Linux/MAC\n",
"import os\n",
"cwd = os.getcwd()\n",
"datadir = '/'.join(cwd.split('/')[0:-1]) + '/data/'\n",
"#If it doesn't work, hard code the datadir\n",
"#datadir = ${path}\n",
"import imp\n",
"import course_utils as bd\n",
"imp.reload(bd)\n",
"\n",
"#Load data and downsample for a 50/50 split, then split into a train/test\n",
"f = datadir + 'ads_dataset_cut.txt'\n",
"\n",
"train_split = 0.5\n",
"tdat = pd.read_csv(f, header = 0, sep = '\\t')\n",
"\n",
"moddat = bd.downSample(tdat, 'y_buy', 9)\n",
"moddat = moddat.sample(frac=1).reset_index(drop=True) #shuffle rows\n",
"\n",
"#We know the dataset is sorted randomly so we can just split by index\n",
"train = moddat[:int(np.floor(moddat.shape[0] * train_split))]\n",
"test = moddat[int(np.floor(moddat.shape[0] * train_split)):]\n",
"\n",
"def testTrees(X_train, y_train, X_test, y_test, dep, leaf, auc):\n",
" clf = DecisionTreeClassifier(criterion='entropy', min_samples_leaf = leaf, max_depth = dep)\n",
" clf = clf.fit(X_train, y_train)\n",
" if (auc==0):\n",
" cm = confusion_matrix(clf.predict(X_test), y_test)\n",
" return (cm[0][0] + cm[1][1]) / float(sum(cm))\n",
" else:\n",
" return roc_auc_score(y_test, clf.predict_proba(X_test)[:,1])\n",
" \n",
"\n",
" "
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"Text(0.5,1,'Holdout AUC by Hyperparameters')"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnWd4VNXWgN+VHkgFQu9FpSNNEREV\nRUQFCyqoV7GXK3jt+tmwIF4bNmyogCIiYkMBUVHxinRBpEgPIaElIb1PZn0/9sRMQjKZkEwCZL/P\nc57M2W3WmZmcdfZea68lqorFYrFYLEeKX20LYLFYLJZjG6tILBaLxVIlrCKxWCwWS5WwisRisVgs\nVcIqEovFYrFUCatILBaLxVIlrCKxeI2ITBeRZzzUq4h0rEmZapLj/fosliPFKpI6hIjEisg5pcrG\nishvtSVTWYjILyJykxftwkQkU0QWllF32E1fRCaIyEy38wgReUVE4lzj7HCdN6qeKylX7jI/87K+\nH4tnSn+nltrBKhLLscxlQB5wrog0rUxHEQkCFgNdgWFABDAASAb6V7OcRy0iEnA0j+drjjV5j1as\nIrGUQEQ6u2YEqSKyUURGeGh7v4jsE5G9InJDqbpIEflQRBJFZLeIPCoifq660jODtq4ZRICITAQG\nAW+4ZglveBD3OuBtYD1wTSUv9VqgNXCJqm5SVaeqHlTVp1V1gYd+w0Vkp4gkicgLIuInIkEickhE\nurtdU2MRyRaRmErKRUXjiciZIhIvIv/nkiNWRK52axssIi+6ZloHRORtEQl11RX1fVBE9gPTvBjv\nAhFZKyLpIrJHRCa41RV9dzeKSBzwk6v8MxHZLyJpIvKriHR16zNdRN4UkYWu73ipiDR1zQZTRORv\nETnZrX1zEfnc9VvaJSLjXeXDgP8DrnSN86erPFJE3nf9NhNE5BkR8XfVjXW932QRSQYmiEhHEVni\nkjVJRD6t7HdW17GKxPIPIhIIfAN8DzQGxgEfi8iJZbQdBtwHnAt0AkovybwORALtgcGYG/f1Fcmg\nqo8A/wPuVNUwVb2zHFnbAGcCH7uOayu+whKcA3ynqpmV7HcJ0BfoDYwEblDVfGA2JZXZGGCxqiZW\ncny8HK8p0AhogVGo77p9T88BJwC9gI6uNo+7jdUUaAC0AW7xYrwszOcbBVwA3C4iF5cSezDQGTjP\ndb4Q87toDPyB+Y7cuQJ41PWeecAyV7tGwFzgZQDXw8c3wJ8u2YYA/xGR81T1O+BZ4FPXb6Wna+zp\ngMN17ScDQwH3pdJTgJ1AE2Ai8DTmNx8NtMT8di2VQVXtUUcOIBbIBFLdjmzgN1f9IGA/4OfW5xNg\nguv1dOAZ1+sPgOfc2p0AKOaf1x/IB7q41d8K/OJ6PQGY6VbX1tU3wHX+C3BTBdfyKLDO9boFUAic\n7FavQMdSff55X+AHd/m9/PwUGOZ2fgfm5g7m5hQHiOt8NXBFOeOMxdzoUksdTuCcisbDKFAHUN9t\nzDnAY4Bgbvwd3OoGALvc+uYDIW715Y5XjvyvAJNLfXftPXxuUa42kW6/o6lu9eOAzW7n3YFU98+h\n1HgPA9PK+S01wSimULeyMcDPbp996fE+BN4FWtb2/+ixetgZSd3jYlWNKjowN8MimgN7VNXpVrYb\nc6MuTXNgT6l2RTQCAkuVlTfOkXItrqdcVU0AlmCepIsodMngTiBQ4HqdDDQ7gvctfc3NXTKswCjl\nM0XkJIxCnedhnOXu34Pru4grqvRivBRVzSpDlhigHrDGtTyZCnznKi8iUVVzS8lT3niIyCki8rNr\naSkNuA3zHZf5uYiIv4g8J8Z5IR3zAEOpPgfcXueUcR7met0GaF50La7r+T+MwiiLNpjveZ9b+3cw\nM6PDZHXxAEYBrxSznHsDlkphFYnFnb1AqyJbhovWQEIZbfcBrUq1KyIJc8NuU844WZibXRGlDeUe\nQ1KLyGmYZZOHXevw+zFPrldJsfE0DvO07E47ipXbj8B5IlLf03uVQelr3ut2PgOzHPUvYG4ZN+vK\n4mm86FKyF8mShLkRd3VTUpGqGubWtqzPt7zxAGZhlFgrVY3E2KWkVH/3Ma/CLPudg1nebOsqL93H\nG/ZgZlPuSjdcVYeXcy17MDOSRm7tI1S1q1ubEn1Udb+q3qyqzTEz5zfFunlXCqtILO4UPQU/ICKB\nInImcBFmvb40c4CxItJFROoBTxRVqGqhq36iiIS77Bn3AEUG9nXAGSLSWkQiMUsV7hzA2FbK4zrM\n0lQXjB2gF9ANCAXOd7X5FHhURFqKMYif47qWua76jzA3nc9F5CRXm4Yug/Nwyud+EYkWkVbAXa73\nKWImxoZyDWa5pKpUNN6TYgzzg4ALgc9cs8mpwGQRaQwgIi1E5Lwy+lc4nqs8HDikqrki0h+jKDwR\njrmZJ2MeGJ714r3LYyWQ4XIOCHXNdrqJSD9X/QGgbdHDj6ruw9g7XhLj3u0nIh1EZHB5byAil4tI\nS9dpCkbROMtrbzkcq0gs/6DGyHsR5macBLwJXKuqf5fRdiFmrfwnYLvrrzvjMDOPncBvmKfaD1x9\nf8DcgNcDa4BvS/V9FRjl8uB5zb1CREIwhtrXXU+SRccujHIoWt56Cvjd9d4pwPPA1aq6wSVDHuaJ\n+W+MUkrH3LQaYRRqeXztknkdMB943+0z2YMxGCvGYaBKVDDeftd17cUs8d3m9j09iPlOlruWln4E\nDnOYqMR4dwBPiUgGxmg/p4KxPsTM/BKATcDyCtqXi+uh5ELMw8IuzO/yPcxMB4qVXbKI/OF6fS0Q\n5HrvFMzDg6dlzH7AChHJxMy87lLVnUcqc12kyJBnsViqARH5ANirqo/6ajzXTHGmqrYst2Pl3qNa\nx7PUPexmHIulmhCRtsClGJfTo248i8VX2KUti6UaEJGngQ3AC65ltqNqPIvFl9ilLYvFYrFUCTsj\nsVgsFkuVqBM2kkaNGmnbtm1rWwyLxWI5plizZk2SqlYYL65OKJK2bduyevXq2hbDYrFYjilEZHfF\nrezSlsVisViqiFUkFovFYqkSVpFYLBaLpUpYRWKxWCyWKmEVicVisViqhFUkFovFYqkSVpFYLBaL\npUrUiX0kFh+TkwNbtsCOHbBrF/TvD2ecUdtSWSyWGsKnikREhmFyS/gD76nqc6XqW2OywEW52jyk\nqgtcUU83A1tcTZer6m2uPn0wOZ9DgQWY3AE2YFhtkZEBvXrBzlLpG269FV54AcLDa0cui8VSY/hs\naUtE/IEpmCRJXYAxItKlVLNHgTmqejIwGpNIqYgdqtrLddzmVv4WcDMm1WonYJivrsHiBc88Y5TI\nu+/CH3/AgQNw333mvFs3mDED4uNrW0qLxeJDfDkj6Q9sL8o0JiKzMXmcN7m1USDC9TqSkvmvD0NE\nmgERqrrcdf4hcDGwsHpFt3jF1q0weTJcfz3cfHNx+QsvwCWXwA03wNixpqx9ezjzTBgyBM46C5p5\nSFjncEBampntFFG/PsRUGPLHYrHUAr5UJC0wObGLiAdOKdVmAvC9iIwD6mNSnxbRTkTWYlKgPqqq\n/3ON6f54G+8qOwwRuQW4BaB169ZHfhWW8rn7bggNhUmTDq877TTYuBH+/BN+/RWWLIEvv4QPPjD1\njRuDn2tC7HSao7AQ8vMhK+vw8URg/Hh49lmoV89312SxWCpNbRvbxwDTVfUlERkAfCQi3YB9QGtV\nTXbZRL4Ska6VGVhV3wXeBejbt6+1oVQ38+fDggXw0kvQpEnZbfz9oXdvc/znP0ZRrFsHixcbw3wR\nIqatnx8EBUFUlDnCwkwdwPLl8Oqr8O23Ztmse3dTHhAA0dG+vVaLxeIRXyqSBKCV23lLV5k7N+Ky\ncajqMhEJARqp6kEgz1W+RkR2ACe4+rvnlS5rTEtuLqxcCampZomoRQs4++zqGz89He66C046Ce68\n0/t+/v7Qp485KsvYsTB6tFkuGzKkZF3v3nD11aa+efPKj22xWKqELxXJKqCTiLTD3OxHA1eVahMH\nDAGmi0hnIARIFJEY4JCqFopIe4xRfaeqHhKRdBE5FVgBXAu87sNrOPZIToZzz4W1a4vL/PyMwduT\nXcJbCgvhqqsgNhZ++snMIGqKM8+Ev/6CTz81LsdglNoXX8C995qjaLnM3x+mTTMKxmKx+BSfKRJV\ndYjIncAijGvvB6q6UUSeAlar6jzgXmCqiNyNMbyPVVUVkTOAp0SkAHACt6nqIdfQd1Ds/rsQa2gv\nJikJzjkH/v7b3ES7dTM32iFDYPZsY9OoKo88Ypa1pkypnb0i9eubWYk7Dz9s9rF89RVkZpqyadPg\nww+tIrFYaoA6kbO9b9++etwntjpwwMxEtm2Dr7+GoUOL6/r1M8bsNWuq9h4ffwzXXAO33QZvvVW1\nsXzNvffCG2+YGVpYWG1LY7Eck4jIGlXtW1E7GyLlWMPpNLOM/fuNwfrjj2HkSGjTBrZvh2++KalE\nAP71L7PHY9Omssf0hs2b4aabYPBgeO21ElW5ufE4HBnldKwlLrzQeID9+GNtS2KxHPdYRVKdFBSY\nm3l1ExsL77wDo0ZBo0YQGWnsHR07mhnCmjVw++3GwH7OOYf3v/JKYzOYOfPI3r+w0Cwn1atnlsgC\nA/+pcjgyWL26OytXnsShQz8c2fi+4PTTISLCeHlZLBafUtvuv8cXM2eajXnbtkG7dpXrq2r2W6Sm\nFt+oly6FefNgwwZz3qqV2ejXpYu5qderB506wamnFhuZy6JJEzNL+fhjsxPdU9uyePVV4347cyY0\nbVqi6sCBj3A4UgkOjmT9+qEcCrmCPic+Q5voTpV7j+omMBCGDTMuyk5n5a/ZYrF4jVUk1cmmTebp\nfd484x7rDU4nfP65ucGvX1+yzt8fBg2Cl1+G4cPhhBOK91VUlmuuMYbn336rnJF82zZjYB8xgh2n\nXEXSiuIqVSUvbwoifdia8TpvbRnDikNzaLZkHtNP201kUGOv3qL0fsQir2X3je3h4cZZLDTUe9G5\n4AKYM8d4sB2Jy7HFa2aueoQFW7+usF1kcDj9Wp7OoPaj6NCoH35WwR8XWEVSnezebf6WpUiWLoVl\ny8yGvE2bzDIYmLtmfDyceCJMn2422hUUmDAhnTtDgwbVI9vIkcbjaebMkookKckYpT//3MxcOnQw\n9pbgYFP/6acQEsK+J97mpM6Cw1HctWfPJTz+/Cbu/OYU9jU6DbIb0CblfBJaLuTW7/oTO2knaPXd\nKF54wThjDRjgZYfzzzeK99tvrSLxIat3z+P6hc8S5AdBfp4fdDIdytsblwMvEuoP/q7mTUKCWfiv\nJXRqXDr4heVYwHptVSf9+8OqVWYmkZhYvOP6yy/h0kvN65YtjVtuUZgPf3+47DJj//D39618114L\nn31m3IE7doS8POMim51t9mjk5BgDflJScZ/AQJgxgzeSxzBunInBGBNjZiP/SzuFN3euJrswkMua\n3c+opvdTPyCS+YlDmLLrJ65rdTFXNvuyQrG82di+YgXccgvs2WP2QLpvbB8yBMqNgnPaaUYxr1pV\npY/OUjaFTgf9pjRkR3oGm/69kRZRnT22z8lPZ3nsl/y+ez7bkrcB5rc0d8eftA+vz8rb9xIaFOFx\nDEvN4a3XFqp63B99+vTRGqFxY9UePVRBddas4vKBA1Xbt1dNTKwZOcpj0ybVUaNUe/ZUrV9fNSBA\n9brrVDduLNkuK0s1Pd0c2dmqqnrmmaqdO5tqp9Opl386QpmAnvxGc910cFOJ7g5Hvp7/XlOVCejn\n616uNvHT0lRvu818vKWPQYNU33xTdd48c8yfb8TXiRNNg337qk0OSzEv/TRWmYC++NN1VRrn/WX3\nKRPQqz/uXj2CWaoFzJ6/Cu+xdkZSXWRnm6Wjp5+G1183EW5nz4bVq80+jsmTTbypowVVY5AoWsLy\nQGKisbH/3/+Zy9uVsov2r7VnVAuYNmYrYfUPN6ynZu2hz9sdic/O541z7ufmAc+73lZJTv6WlJSf\n2JmewooDe9iXKzSJHkh4cAT1AusRHBCMn2YTUriby05+gsCAkvtAkpNNFBgo3tg+c6bZh+lOmzYw\n97E/6XtTLxgzxni0dexoZo4hIUf2uVn+ISF1M12mdKV9RDir/52Mv1/VVspvmtOX9zevYcrZt3PH\noDcr7mDxOd7OSKwiqS7+/tvYNGbOhJ9/NktIiYlw441mg2B8vHFHPQZ57z3jjPbHH3DyyTBj3XTG\nfn09c88+ncsG/a/cfvEp67nwo9P4MyWL+3qfx4ODJjJr+Y0siP2TVSlwKN+0CxQoKOdn2CEskCcG\njefqvs97NMyqmqj2RRvb9+83envHduXvVufSae8vSGGhqbzlFuNObYF9+7wOneN0FvDw/OH8nWyS\nmG1P3c+W9GyWXfcV/dqMrLIoeQXZDHi7GWsPpRPoZmrxl6JD8Pdz/RVB/qkXXhzyKKP7PFFlGSwl\nsYrEjRpRJN99Z4y7v/1mHplHjjT2hxtvhDvugFde8e37+5Dhw42e3LHD2CzGfn4Rczd/y/abP6Vp\nkys89s3OS+GKWb2YHxf3j8KIDApl+AkjGdLuHAa3HUz9vGX8vfU28jWY4LBB7E+aR1BoFw74n82z\ny94lPjufntHhvHPRNE5pd5nXcmdlwQMPwJtvgj8OOgbGMS3gZrpGxROxd0vFAxwt7NjBf6cOoCA3\ni0ezrjIRDHr0KDYgtWxpZsOV5fvv4bzzzKbN0oEwS1FQkMqHS4Zw09I/aBTsT5CfuZHf1utiHh36\nWeXfuxwSUjfz4i93kO1wxVJTpVALKVQnBYUO89pZSIGz2Otj6b5dNAwOZuNdmdYLrJqxNpKatpG8\n/bZZi9+zx9gYQkNVw8NVRVS3b/f9+6uq06m6e7fqZ5+pTp2qunSpsStUhZQU1cBA1XvvLS7rOLmB\nnvKanzocmV6NUVhYqBMWjtQrZ3bXeZtna54j77A2mZmbdeXKHvrzz6Lbtz+ghYWmTV5Brj6/+DqN\nmiga8CR6z1dDNDc/q1LXsGqV6ltvqT7wgOpT9Z8z31Nt26u85eOPdUfnUA14EmUCOunWgMMNRO3b\nm99cZTnnHNP/yis9NsvJidXlyztrj8miTZ6P0qxKfv6+5rkfr1ImoN9ueK22RTnuwEsbiXX/rS5i\nY3EGBNL3gmb8utSfsHPPNW7AI0YYl9pqQNV4TS1fbkwyRUdOjjliY03IrdI0alTsEBYYaB5gW7c2\n21JuusnYEspj/nzj9HSZayKQnJ3E9rRDDO/aGX9/756C/fz8eGLYVx7b1K9/En36rCQvL57Q0OLP\nKyggmPvPns7VvR/mpi+G8vK6xXy1vQEdIhuZ6/ELYPyABzmv8+3ljt23rzkAHl46AJZiPsQLL/RK\n/lrjgQfghReYNKE+ToXeDSN5hDRaf3k9V+Wca9rs3w/33GPC1jz0kPdjb9gAP/7I6kHh9PjmC4JS\nUsrM65KY+CVbt97KiqRs1qcpb5z/DPUCy0gs9uGHZvNnRURGmi+jf3+zsbbohylyxHuk7hj4KpNW\nfMJLS5/lgq7jjmgMS9WwS1vVxVVXkbpoBdGHdrB4MZwdN92koP3pJ2N4ryIZGeamP2cONGxoNujV\nq2c26BX9bdbM2PW79JiIQ1aRm/Ilf/0lxMUVj5Oba8w1cXEm1bpLdO6/33gll/5fvvRS43q7Z49x\nz52z7iWu/Po+vrjoES7p/UyVr6uyTFv+AC8un0Kuy95xKC+f1AJlTMeuvD5yAQ3DPGfDfODObCZO\niSTgofuRSc/WhMhHRloaxMRw8JphtG3/DWe3aMPMK5cz4J32xGbm8O55D9Mi0ijcro/PpMn368wX\n6m2Sr5tuYmbqDP7V3UGjABgT3ZE7R33ICY3NJp2CgkNs2zaegwc/pn79ntz+h5PEnDS23rmV4IBS\nDhp//GF+eE2bmh+mJw4ehJSUw8tbtDC2xU5HFhFh3BeDmPLXb6y76Xt6tDj3iMawHI61kbhRI4rk\ntNPYsTeUjrsXM2kSPPSA02w+7N27SsPm55thrrvOGJOffdbc9MtbCj5wYBZ3fnM1iw/Cuhu+pnXT\nEeWOHR9vEhy++66Z2YSGmtTqbdsWO3MtWGDMPG+8Yc5vnXsKH2xaSfJ9e4moVw35TapIavZ+7v5m\nOB/+vZboID9OijYbOP3Fj7sH3MfFPe4v0f6112DAXf3oeVoYQUt/rg2RvWP2bBgzhntn9OHlXWtY\ndu1nnNpuFLuT13Pqe73Zn1v4T9OGQX5se95J9C0PwH//W/HYiYnQqhV9nlbiCws5wQlLCwpRTPC9\nID8I9of+DYRbet+Af/j5XDpnFO9d9B439r6x5FiFhSZET3y8CewZFeX5vVWNwlu50kRNKCp7/XWT\nlGzZsiOy9+xOXk+nKT25omN3Zl61vuIOFq+wisSNGlEkLVrwe/h5DNzyAZdcYlxSK0NCglEQ+/aZ\nc6fTlMXGmv/Vxo3NvcXT5CY9fTXr1g3ipjX+bEvP4t5uJ/LiZX+X2XZL0hbmbJzDAwMfICM1mLlz\njaLascPMVop2sAcFmTTrPXsae1qv1+qBXwh/jivjqbIW+XnrDB776T5S84yR9mBODqn5Tj4Z8SyX\n9Xr4n3bffAM7R9zFnSHv4Z+RZnY0Ho2MHk3G8sW0viWZ7g0a8ettB/+p2p+2jSU7PgEgIW0X9/46\nnfFBjXn1qXRzc27ZsrxRDU8/zf++fZwzhsNj/Ufw1ObB/P3WvXz0zGBSAxzkF+aRnJfN4vg40vMz\nCfALoG1UWzb/ezMBpV18X38dxo+HTz4xGSqPlEWLjLPK1VebZbIjWOa6dEYnFsZtJ/au7TSJqJ7l\n5LqOVSRu+FyR5OVBSAifdXuSKzY8TosW5gHNW374wSwv5eSUjOTRtKmxY3TqZOIPNi4VumrHjodI\nSfmByMhBREScyo4d95FT6MfQnxNQVaKDlK13rKFhZPGsSFVZt38dQ2cOJSk7iefPeZ77B5Z8ai+P\npNRVNH+tPzd1H8qblyzy/gJrgf1p2zjj/R7szspl1kWTuKyXsR9s3AhPd5vNbMaYqMlVnDH6hLw8\niInhmYca8lhBLPNHvcHwrv8ut/nQ91ry2/4Etk71p2WP84ttP/7+ZqkrOtq4nvv5mSeUiy5i5Pg0\nfijMYc9/dtMwK9gsLd17b4kZTVZ+FrM3zGbWhlncf9r9DOs4rOQbJyQYl/cBA4zX4pHGgSviqafg\niSfM9Pf6673v5+cHISGs2PU5p344Cj+gKFKLAAF+xn242GG48jQJDWbN7buJCPUuftzxglUkbvhc\nkWzfDp068WK36dy/4TrAKJIWLYqbxMUZO8OBAyYCidNpyrdtM3sVu3SBuXNNGnRvcDgy+P33xgQG\nNqKgIAmnMxc/v3pkx7zG+XNu4j/97+CVlW/yeJ/TePLCpagqO3c+xKLNU3hofSFRoY1oF92etfvW\nsm3cNpqENanwPecsv54rF03ns8umM6rbdZX9lGqcvalbGPxBL+Kychl7Un86x3SnU3R//t3/dGLz\nu5qn6crknK8pFi7EedFw2j3rT1RIPdaNS0M83KTXxS+kz/vDuSaiMTPuOVhuuyJiW0Knm+GqTj2Z\nMWadKRwxwmye3bbNGNLy8ozRzf194+LM2mrRU9LOnbBrlzHcV4dDidNplODCI0h6+t57cOONTP7l\nBrYlF8/CC9VJodNBQaED5cjudZn52XwZu53/nn41Dww5wlQMxyjeKpKjdF5/jBEbC8B2R1uio40t\nceVKE/EdYMkSE8qqPK691ux1KFoadjgd3PrNrXRt3JW7TrkLf7/DY3AlJ8/D6cylS5fZhIf3JT19\nFQEBUUxZNx+ARwc/xS87PuedDb/zwJB4Eva8wIdrX+OlbUKjIOW1XtC05SjOmL2MR356hPdGvFfh\nZf4aa/7BB7cb7vVHU5s0jzqRJTesY+THp/L+5pUUbloJvE+vhzqR8mJzopct860iOXjQGKHuuMOz\na1xpvvqKVSODiMvJ57a+YzwqEYBeLc/nsvYnMWvX3zy06Us6R7kCHxYUmKCghw6ZEAAuXkh+hMK4\nDTww+OXiQcaONet+7tkkO3Uy5ZdeajbavvSSKe/a1fytX98Y2KrJKxE/P7N+O2OGmZ57y0cfwaRJ\nMHYsd5/5QfXI4obT6aTLq2G8s24u9531od2rUgZWkVQHrqi/W/PacMYZxmXWXZFMn25WFj791ATY\njYkpXpoPDDReWO5MXjaZD9aZf4ivt3zNjItn0DaqbYk2Bw58QnBwKyIiBiDiR1TU6QCsSHicjg06\n0rBeQx4b/CSXfXEbd33eg7XJKfyRCqe06M/0YfeRse9pshLGc0O3Yby79gP+3e/fnNzs5HIvMSNj\nHWsSD9Axsgkx9WOq/JHVFM2jTmTVv1PId+Ty94H/ce+C61i6fxsro0dy3u+/++6NDxyAs882kZ7n\nzDEeSW3bVtzP6US//oq5T/rBfhjV816v3u658z/i67f6MfanGxjS9lQA/MWf6NBGNKjXmKgGjfDD\nHyeFfLxhI2c2a0rX5mcXDzBihElXkJdn3ABVTbDRRx4xB5j110mTPETIrAYiImBcJV1427eHyy83\nivDii6tdJD8/P249eTT3LJnGtxteYUSPe6r9PY51fLq0JSLDgFcBf+A9VX2uVH1rYAYQ5WrzkKou\nEJFzgeeAICAfuF9Vf3L1+QVoBhQ9sgxVVY/zeZ8vbT32GEyaRLPoXC4eFcCqVcZdfvFi82DVpIkJ\n7vuBFw9L25K30ePtHpzXYSgXn3QJ4xeOB2DuFXMZ2sGk0C0oOMTvvzehZcv/0KHDC//0VVVavNyC\ns9udzcxLZ6KqdHk1ir/T0gkPDGbSOS9xW9/b8Pfzx+l0sH79UBKSf2fsmnp0junGkrFLyn36/WnN\nFYz87jOu6HYt74+cUfXPrJb4Yt1ELvv6UW7cPIT3Pl0Me/d6HSLEa/btM0okLg6ef978PsLD4Zdf\nKk54tmwZyf85jQuugYOOxuy8u4yNQeXw4DfDeP4P72xX3456zbs9Fzt3mv1QAwbAKUdpiHeHw8ye\nWrUyyeF8QFZeKi1fakDvmKYsvnmvV30OZcWzInYe6/b+j1xHJWZY1cwdA186YueDWl/aEhF/YApw\nLhAPrBKRearqnjj8UWCOqr66e41lAAAgAElEQVQlIl2ABUBbIAm4SFX3ikg3YBHgZnHgalWtgbjw\nXrJ7N9qiBUl7A4iONnutZs40S77z55s9IFddVfEwTnVy8zc3E+wfyPWNl9PCsYefLp/I6AWvMuGX\nCf8oksTEL1B10LjxmBL949Pj2Ze5j1NbmidSEeHdkR/z8bq3eeKcqTQLL75h+vkF0KXLp+Ss6cMN\n7bJ4ftP/+Hbrt1x04kUlxtyWvI1nf53AR399hp/4cVX3a6v4YdUuF3W7m+gFj7Gy00ZTsGxZcYh/\nb1m50hiEf/3VTC9btDCeEEWb63780SiThQtN7pcBA0zAyDPOKDaEi5h+J5xgjmbNjFH8q6/YPhrW\npQo39fE+HAzAfy/6jsfOPYRiDHD5jmySsvaQmBlHWk7xs1ZkaGNO7zCmvGFK0r790RVstCwCAkz+\nn7vvNukC+vWr9reoHxzFmBMH8M6G39m871c6NyuZHO69Zffw32VvkeNyd3SocsDNRbs2Gdn1Fp97\nsflyaas/sF1VdwKIyGxgJOCuSBQoimQYCewFUNW1bm02AqEiEqyqeT6U98iJjcXZqi2OOONGf8IJ\n8NZbsGULzJplvK+82ZM4dc1UluxewiPdGtMwyEFhYRY58eM5u2E0U7fvYF/GPpqFN+PgwU8IDe1E\nWFjJpagVCSZ94Sktip8cB7W7kEHtyt7BHRQUQ9euX5CdO5CZ9UJ45tenuPCEC/+Zlazdt5ZT3z8V\nwcmI5jDpgu85sZnnmExHO4EB9RgYdQKLHFs4FBlIg6efhilTjN9zamrZnUJCzB6H5s3NTvI1a4wt\nYdgw85Swc6fZKa/KT31yuO+GTN7p9Qz9ihKI9e5N/PypXL94NBf9+SF3/K8+AQ6niclWivTO8Osz\nkPeXMrxT5Xfeh4WUTITWMKw1JzYZWOlxjjluuMF4fE2ebP7pfMA9g17hnQ39efl/9zL1CpPf5kD6\nDq6fezYL98TRMTyEk2OM67W/+HFCww70bj6Qvq2GE12vYmcWXxEZ2rTiRlXEl4qkBbDH7TweKD03\nngB8LyLjgPrAOWWMcxnwRyklMk1ECoHPgWe0jPU5EbkFuAWgtS/XdAFiY8ntfybAPzMSMDHx5s83\nttaKclYlZiVy/w/3079RBOc2SqN795+JiOhPUtLXnO18mKnbU5i+7N/cM3gKqak/06bNY2TmZ+In\nftQPMlb65fHLCfYPpmfTnl6LHhHRly4nvcOV+65n8rbVzFl1F5f3fQmRAMYtHEdkcCTv9Q2gTYOu\nx7wSKWJo8+v5NvEh3r+uA/d/kmAUxemnG2NVWUt7WVlmhpGQYL7IN97g59P9mL7pHV6+aN4/u+kT\nUjcz5u3uHMxTLt/4JGvPGEt0/ebkO3K5eM2NrCl08GM3B2+f2YoXh77M+U0HITt2GE8p147vuI4z\nWHVoN8H+fpzZ9sya/WCOZSIiTIjqV14xs5PShsdqoCPRnNu0GTO3rGbVq8YpYXdWNhkO5e4TT2XS\nye8Q7O8KH5Ofb+xjX66DDROL8x7UBlOnmmU/H1LbxvYxwHRVfUlEBgAfiUg3VXUCiEhX4L/AULc+\nV6tqgoiEYxTJv4APSw+squ8C74KxkfjsCgoKICGBrEZtAaNITjzRLIlPnGh+T1dfXfEwz/7vWbLy\nM/h3O6FLly+IjDShKmJiLmXUGcN44I8mfPn3lwyJ+htQYmKu4KwPzyYlJ4U/bv2DiOAIViSsoHez\n3gT5B1XqEpo1G8uD57Xjo7ih/HfZ67TVhazJP5ele5by8pm3EaFv06LF8RN2fXj38Tyz+WFmt8nk\n/oMVu8uWZl38Qi6ecQHpDmXZuyfy/XXLaBnVhctmnU5KfiFPnnopTy7/gjGzT2HB9bu5/fPTWZOc\nxgunXUhO2ndMjd3DBbMuYFz/cbw67FWkVy8A4uNfJ2n7dtasbcSZbfuUHdPqKGXtWmMCqojISFcY\nny4+SAg6frxRJKeeWs0DF/NUL7jnUijIyAKgdyFMWACDnlgOlPEA5+9ffEOoLQp9v8TmS0WSALir\nwZauMnduBIYBqOoyEQkBGgEHRaQl8CVwraruKOqgqgmuvxkiMguzhHaYIqkx4uPB6SQ10rh3RkWZ\n307fvsWhgypKF74nbQ9vrZ7CeU3hrO4vERNT0vMkIKAel3e/hddXvsLB9M00iezBL3t3sXqvMRPd\nMf8Opo2cxpq9a7i1z61HdBlNGg7moTOe5b4f7mNVciET/nyLzpEhDAhZAtqOhg2PDZdfb2jTJpSO\nuZ1Znr+JAxl7aRLe3Ou+e1I2cMGsEfgJvDDoWh7//UMGvt+PQc07sSLxEC8Mupb7zp5BTsH5PLfm\nO4a+35LFe/dxzQndue/cb9i79z1ObXAzs5JO4fWVr+Mnfkw+bzKHDi1g+/b/kBMyhJ3pixk3YFjF\nwhwlJCTA4MFmlc9b6tc399ciZdKqFbz/fsURVjzSurXxtd+1qwqDeKY/8FvpwrJ8Fvz8zAV27Von\nkqj5UpGsAjqJSDuMAhkNlDY5xwFDgOki0hkIARJFJAqYj/HiWlrUWEQCgChVTRKRQOBC4EcfXkPF\nuFx/k8LaAsUx8045xSiSq6+ueMPvk0uexOl0cOtJHWjRomxPmks7X8rLy19mf9hDDD7pSsZ9ehvt\notpxTY9rePrXp4mpF0OOI4dTWh65Z81tfW9j0m+TeHDdAbIL4Pk+TcnN2UyHDi9jfCeODwICIGT7\nTThPuIf7Fl5Hi/Cm7Er+k/S8YhtJ/cBQ2kW1pkODk2gR2YGggGAC/YIYt3A8yXkOFl75AWedcD29\nW5zBpZ/fzKc7NnNJ247cc+Y0ACYOn8/yvS1YvHcfPaPDmTrKuBo3a3YjaWm/ca3MQAu78eqKV8nL\ni2d01ELCwnqx0XkhsJjzO57v888hKck8xGdnH14XFGSCCXuz8f+uu8zE/K+/Kl5BOXjQ2MOLQm0V\nxcKfN8/Ek/vyy/LjyHnFwIHmsNQs3sSaP9IDGA5sBXYAj7jKngJGuF53wQT1/hNYh3HlBePNleUq\nKzoaY+woa4D1GCP8q4B/RXL4NB/JtGmqoF88v02hOPXIL7+ohoVVnIpkS9IW9X/STy+dih48OLfc\ndo5ChzZ+obFe8dkV+uOOH5UJ6Nur3lZHoUPPmHaGMsHkq9iVsqtKl/P0kqeVCejYr8aq0+nQ1NTf\n1el0VGnMo5Fzh2Zp++f9/vncoieirZ8P1DYvBGmbF4K0wbPFde6H3wT0o5X/V2KstXsW6LgvztD0\nnJI5Tg6mx+r4LwZrbNKfJcodjizduHGMLlkSppdONeO2fD5AL/90pHZ/s7u2e6WdOp3OcmWfO1d1\n9Oji4623juwzeOwxky6nZ8/DjwYNVBs2VP37b89jfPutUQUTJx6ZDEW88ooZ57nnqjaOpXrBy3wk\ntZ50qiYOnykSp1P1ggtUQ0L09RdzFVSTk0tWV8QVcy7XkKdEFy3t6fHmoap687ybNezZMB34/kBt\n/lJzzS3IVVXVuNQ4jX4uWhu/0LjCMSoiIy9DJ/46UZOykqo0ztHOTTepjrhiii5ee4vuPfitOhyH\nJ2tKyz6gK2O/0q/+fFE/++MZnbX6CV2+s3xlX1kKCws0JeV3fWbRJXrhzCHa6uVWygT0vkX3ldvn\nu+9U/fxUmzZVPeEE1RYtzH/xqlWVe++cHNVGjVRHjCi7futW1caNVVu3Vo2PL7tNZqZqmzaqXbqo\n5h2eq6xSOJ2qV1xhru2nn6o2lqX68FaR2FhbVeHVV42P/auvMuHQeJ580uyN8taIGJ8eT6vJrbiq\nFbx+8fc0aOA5j8KCbQu4YNYFALxy3ivcdepd/9StiF9BSm7K4YH1LGXy7LNmw3Zm5pFlqfUVKTkp\nRARHlBkWZ9MmsyWlbVtYutR4IKenmwglPXua7Sve8v77Jr+Np3Q5f/xhQvu0bm1ybGVkmPcrst2u\nWQNffWW20wwaVOlLPYyMDOPxmJDgXRCAIvz9TfSWs8+uuK2lctT6hsTjnjVrTNz3ESNg3DhS7zYe\niJXxRFm6ewkAw9v1ITq6LM/nkgxpN4SwoDBCA0K5uc/NJeqqYhupixRtMI+NLQ4ddTQQHVp2Yqqk\nJLjoIpMzxj0kVkSEUYh3320UyTkV/4xQNdstevb0HAOud2/4+msT3f26cmJ03nNP9SgRMI5N8+bB\nk0+Wbbcpj6VLTfAAq0hqD6tIjoT0dLjyShP75IMPQIRyMpV65NedXxAocHbXxyoMzAcQHBDM2xe8\nTXRo9DHlGno00r69+btzZ/UqEqcT1q83N+mqRlUv4vvvzcQ3IcG42JbeFnX77cbr9aGHjBG7ImP1\n4sUmnP60aRXLeNZZJmp1WppRWuHhJj5cEdWdzqVTJxMVojIUpURZvtynnr8WD1hFciRMmmRcDJcs\n+WfjU0pK5V0XVySs4MRwaNzQi8dIF1f38GJTiqVCimYk1e0pet995mn/mWeKYx164tAhk3ojKal4\n83zjxibySlSUeU5ZtMgovnnzyr5RBgebVB7XXWdSEVxxhef3nDzZvMcYL6OkxMSY42jl+uvNjGTy\nZBMY1VLzWEVyJMybB0OGmN3QLlJTKzcjKSgsYEPyXka1bY6//1G0SF9HiIkxQW6rU5FMnWpuZq1a\nwaOPmtwyl3kIl5WebqKsrF8Pp51m0nosWlRyP0Z0tAnKe8cdxemPy+Lqq+GFF4wyKQqNVV5eqwUL\nYMIEz+MdS4SFwa23wosvmqXKythXLNWDVSSVZc8eY/W84YYSxSkpZlruLX8kLCXPqZza8rRqFtDi\nDSLmRj97tok8Pnhw1cb75Rdzsz/vPPj8c2Or+Ne/zMynrL0Y2dkmfuPatSYt80VusTJzckwYrqQk\n0z8ysuL39/c31/LGG8XGcPd0JO4ZOwcPNrIeT4wbZxTua6+Zv5aaxSqSyrLIFab7vPNKFFd2aesX\nV87tMzpcXl2SWSrJ+++bZaCzzjLLUI8/XnL9vyxUjXH3jTfMymbjxmY5auVK8yDx6afGC+yrr0wo\nkAsvNLOO0vz1l/GKmjWrpBIBY1Bv2bLi1Oul6drVBAuti7Rsab7L994zsRu9Ub6+RNXYtNatM991\nbYbauvNO3y9NWkVSWRYtMuG/S1loK7u0tSzuVxoEQdcWI6pZQIu39OplbubjxxubxvPPl+115x78\n98ABsxQVFQUXXGCWofbuNRGfZ84svoE1aWK8q669tmy33MBAY+y+8krfXmNd4u67jWJu3briBwJf\nk59fuZAxvmTMGKtIji4cDvjhB7Pw7ebuUlBgAsRWZkbyx8Gd9GgQg7//8R+H52gmLMwYtC++GH47\nLIiSwT34b/36xhZy1VXGxuKJnj3hzz+rX2ZL2fTta7Z2bd1a25IUx2rs1Qu6d6/dmI01gVUklWHl\nSuMHWWqtoiiNhbczkv1pf7MnO58xnb0IZGSpEUaMMIfl2Gb8+NqWoG5is9hXhu++M64vpXZ9paSY\nv94qkp+3TgdgULuR1SicxWKx1A5WkVSGRYtMWN9SGqNIkXi7tLV09w8IcIa36U4tFovlKMYqEm9J\nTjbxr0t5a0Hll7bWHNhMx4gIIkKqknzBYrFYjg6sjcRbfvjB+PSV4cvpzYzE6XSQlvYrBw58ysbU\nHC5s38tHglosFkvNYhWJt3z6KTRoYFxDSlGRjSQnZxdr1w4iPz+BLZnBZDhgcEe7rGWxWI4P7NKW\nN/z6q9lhdtddZW40qGhpa//+6eTn76PTiTN5J+EkGtdvzOXdbMwsi8VyfGBnJBVRWGgUSKtWJiJf\nGaSkmLhF5aVmTkycS2TkID7cHsfa/X8y9/K5NAht4EOhLRaLpeawM5KKmDbNxDl4/vlyd6B5CiGf\nlfU32dmbSA08nSeXPMmoLqO4rIuHSH4Wi8VyjGEViSfS0kwQpoEDPcaySE0t39CelPQ5hQr/t/w7\n6gfV543z3/CRsBaLxVI7+FSRiMgwEdkiIttF5KEy6luLyM8islZE1ovIcLe6h139tojIed6OWa08\n8wwkJpq4Cx4yAHmakSQmzmVbfheWJ6zhpaEv0SSsiY+EtVgsltrBZ4pERPyBKcD5QBdgjIh0KdXs\nUWCOqp4MjAbedPXt4jrvCgwD3hQRfy/HrD6CguCWW6BPH4/NygvYmJOzk8zMdSRyIgDndTh8D4rF\nYrEc6/jS2N4f2K6qOwFEZDYwEtjk1kaBCNfrSGCv6/VIYLaq5gG7RGS7azy8GLP6mDjR7B2pgJQU\nE6CtNImJnwNwoCCKsKAwmoY1rW4JLRaLpdbx5dJWC2CP23m8q8ydCcA1IhIPLADGVdDXmzEBEJFb\nRGS1iKxOTEw80mvwKvF2eUtbiYlzCQ/vy670A3Rs0NGrvOwWi8VyrFHbxvYxwHRVbQkMBz4SkWqR\nSVXfVdW+qto3xofB+J1OY5MvbWzPzd1DRsZKGjW6jO2HttOxQUefyWCxWCy1iS8VSQLQyu28pavM\nnRuBOQCqugwIARp56OvNmDVKRoZRJqVnJImJcwCIbjiSnSk76dSgEnl4LRaL5RjCl4pkFdBJRNqJ\nSBDGeD6vVJs4YAiAiHTGKJJEV7vRIhIsIu2ATsBKL8esUcoLj3LgwEzCw/uTVBCMw+mwMxKLxXLc\n4jNju6o6ROROYBHgD3ygqhtF5ClgtarOA+4FporI3RjD+1hVVWCjiMzBGNEdwL9VtRCgrDF9dQ3e\nUBQexX1pKzNzA5mZ6+jY8XU2Jm8DsDMSi8Vy3OLTECmqugBjRHcve9zt9SZgYDl9JwITvRmzNnGf\nkeQ6ctl4cCNROZ8B/jRufCVfrzNLXHZGYrFYjldq29h+zPDdd/DWW4eXuyuSWX/Not/UfqyLnUGD\nBsMICoph26Ft1A+sb11/LRbLcYtVJF7gcMCtt8IDDxjDujvuS1sJ6QkoysrE/TRpcg3APx5b1vXX\nYrEcr1hF4gVffglxcZCZCbGxJevcZyQpueZkfVoAjRqNAGDboW10amjtIxaL5filXEUiIu1FZEAZ\n5QNcnlR1hsmTITTUvF6/vmRdbCyEhUF4OCRnm42PGzJC8Pevh8PpYFfKLjpGW/uIxWI5fvE0I3kV\nyC6jPAt4xTfiHH2sWAHLlsFjj5lN7qUVyapV0Ls3+PnBwfQtAMRnZbInbQ9xaXEUOAvsjMRisRzX\neFIkTVX1z9KFqroeaO87kY4uXnkFIiLgzjuhQ4eSiqSgwKQq6dfPnCdlHyDC5Qf36+5f2WZdfy0W\nSx3AkyKJ9FAXWt2CHI3s2QOffQY332yWrnr0gL/+Kq7fuBHy8orTuKfmptMzOoTI4EiW7F7C9kPb\nAev6a7FYjm88KZK1InJ96UIRGQus9ZlERxFvvGGC/45zhZLs0QO2bYNs14Lf6tXm7z+KJC+bhqER\nnN76dDMjsa6/FoulDuBpQ+J/gK9E5GpgjausLxCOCd1+3JOdDWPGQJs25rxHD6NYNm40y1mrV0Nk\npFnyAkgvKCA6tAGdWw1m/rb5hAWFWddfi8Vy3FOuIlHVfcApInIu0M1V/F9V/b5GJDsKeP31kulI\nevQwf9evN4pk1SozGxGB7Pws8p1Kw9AmnNHmDADW7FvDqC6jakFyi8ViqTnKVSQiUpRwaoXrUFXN\nqBGpjiLcJxPt2kH9+kaR5OYae8m995q6Ay6PrUb1m9G7WW/qB9YnqyDLuv5aLJbjHk82ko3ABrdj\no4gki8h3ItK6RqQ7yvDzg+7djSL56y/jtVVkH9mfthmAmPC2BPoHMrC1CSFmXX8tFsvxTrmKRFVb\nqWrrUkdD4APgnZoT8eiiRw+jSFatMudFiiQxw7j6xoQZz+gzWpvlLeuxZbFYjncqHSJFVecATXwg\nyzFBjx5w6BDMmweNGkFr19wsMTMWgCYRJwHwr57/4vpe19Oveb9aktRisVhqhkorEhGpfyT9jheK\nDO7ff19saAdIzDSp5GPCWgLQOrI1H4z8gNDAOrHlxmKx1GE8GdvHl1EcDVwMlBFQvW7Qvbv5q1q8\nox0gOXsfAA1CG9SCVBaLxVJ7eNpHElPqXIFk4AZVrRMbEssiKsosZ8XFFdtHAA7lJOEHhAeH15ps\nFovFUht42kfyWFnlIhIkIpeo6pe+E+vopkePkopEVUnJSSUiOAQ/qbOrfhaLpY7i1V1PRPxEZKiI\nTAP2ANd52W+YiGwRke0i8lAZ9ZNFZJ3r2Coiqa7ys9zK14lIrohc7KqbLiK73Op6eX+51cPIkXDO\nOdC8uTl3OFJJKyggKjispkWxWCyWWsdjznYRGQhcBVyEia91KtBBVTMrGlhE/IEpwLlAPLBKROa5\n8rQDoKp3u7UfB5zsKv8Z6OUqbwBsB9x31N+vqnO9uUBfcNNN5igiN3c3mQXQICSqtkSyWCyWWsNT\nYqvdwIvAaqCHqo4Esr1RIi76A9tVdaeq5gOz8RyjawzwSRnlo4CFqlpWbpSjgry83aQ7IDq0UW2L\nYrFYLDWOp6Wtb4AWmJv/UBEJxRjcvaUFZhmsiHhX2WGISBugHfBTGdWjOVzBTBSR9a6lseBKyOQT\ncnN3k+kw4VEsFoulruFpZ/udQFvM8tQwzPJSjIhcKiL1qlmO0cBcVS10LxSRZkB3YJFb8cPASUA/\noAHwYFkDisgtIrJaRFYnJiZWs7glyc3dTYYDGtSz4eItFkvdw6OxXVWdqvqDqt6AmTH8C7gSiPNi\n7ASgldt5S1dZWZQ16wC4AvhSVQvcZNqnhjxgGmYJrSzZ31XVvqraNyamtCdz9ZKTu5uMAruHxGKx\n1E289lVV1XxV/UpVrwTaeNFlFdBJRNqJSBBGWcwr3UhETsJsdFxWxhiH2U1csxTEJPm4GBNQslY5\nlLkTJ1aRWCyWuolHr63yUNUsL9o4ROROzLKUP/CBqm4UkaeA1apapFRGA7NVtYT9RUTaYmY0S0oN\n/bGIxAACrANuO5JrqE4SM80ELTokupYlsVgslprniBSJt6jqAmBBqbLHS51PKKdvLGUY51X17OqT\nsOoUFuZwKCcZsDMSi8VSN6lwaUtELvWmrK6SlxdHhsO8jg61MxKLxVL38MZG8mgZZY9UtyDHKkUe\nW2BnJBaLpW7iKfrveRi33xYi8rJbVQTg9LVgxwq5Lo8tsDYSi8VSN/FkIzmI8YjKxaTdLSIDOCxu\nVl0lLy/BzkgsFkudxlP037XAWhH5GDMDaa2q22tMsmMEh+MQmYVBBPuLTWJlsVjqJN7YSIYAfwE/\nAIhILxGpsyHkS+NwpJBVGGRnIxaLpc7ijSJ5CjgFSAVQ1XVAR18KdSxRUJBCZmGA9diyWCx1Fm8U\nSYGqppYqq0zwxuMahyOFDIfYGYnFYqmzeKNINovIFYCfK9zJZGC5j+U6ZnA4DpHhUOuxZbFY6ize\nKJI7gT4Yg/uXQD7wH18KdSxRUJBCer7DzkgsFkudpcIQKa64Wg8CD4pIuKpm+F6sYwNVxeFIIa3A\n7iGxWCx1F08ZEh9xReZFRIJE5Htgj4gcEJGjKt5VbeF0ZlNQmE9WQb6dkVgsljqLp6Wtq4AtrtfX\nAiFADHA2MMnHch0TFBSkkGnjbFksljqOJ0WS7xbafRgwS1ULVHUjEOh70Y5+HI4U0u2udovFUsfx\npEjyRKSziDTEzEK+d6uzW7gxiuSfGYm1kVgsljqKJ2P7vZiMho2AV1V1J4CIDMfsdK/zFBQcKg7Y\naJe2LBZLHcVTrK2lQKcyyg9LVlVXsUtbFovFUomc7ZbDsUtbFovFYhVJlXA4UkgtAEGIComqbXEs\nFoulVvAm1e5hy19llZXTd5iIbBGR7SJyWA4TEZksIutcx1YRSXWrK3Srm+dW3k5EVrjG/FREgryR\nxRcUFBwiMT+I5uHNCfS3jmwWi6Vu4s2MZKWXZSUQEX9gCnA+0AUYIyJd3Nuo6t2q2ktVewGvA1+4\nVecU1anqCLfy/wKTVbUjkALc6MU1+ASHI4UDuX60jWpbWyJYLBZLreNpZ3tjEekJhIpIdxHp4TpO\nB+p5MXZ/YLuq7lTVfGA2MNJD+zHAJ54GFBHBuCLPdRXNAC72Qhaf4HCksD/XaRWJxWKp03haoroA\nuAFoiZlZiKs8A3jMi7FbAHvczuMxeU0OQ0TaAO2An9yKQ0RkNeAAnlPVr4CGQKqqOtzGbFHOmLcA\ntwC0bt3aC3ErT05eMgdzC6wisVgsdRpP7r/TgGkicoWqzvGxHKOBuapa6FbWRlUTRKQ98JOI/AWk\neTugqr4LvAvQt29fn+RP2Z+ViEOVNpFtfDG8xWKxHBN4YyNpLCIRACLytoisFJEhXvRLAFq5nbd0\nlZXFaEota6lqguvvTuAX4GQgGYhyM/Z7GtPnxGemANgZicViqdN4o0huUdV0ERkKNANuBp73ot8q\noJPLyyoIoyzmlW7kijAcDSxzK4sWkWDX60bAQGCTK/bXz8AoV9PrgK+9kKXaUVXiM9MBq0gsFkvd\nxhtFUrQsNBz4UFX/9Kafy45xJ7AI2AzMUdWNIvKUiLh7YY0GZrsFiAToDKwWkT8xiuM5Vd3kqnsQ\nuEdEtmNsJu97cQ3VTmFhBgfyjMitIltV0NpisViOX7zZD/KniCwATgD+T0TC8DJne1nhVFT18VLn\nE8ro9zvQvZwxd2I8wmoV47EFTepFEhIQUtviWCwWS63hjSK5HpNqd7uqZruWmmpt78bRQkGBUSSt\nwpvWtigWi8VSq3izRFUItAdudxWFetPveMfhOMSBXGgTWab3scVisdQZvAmR8gZwFnCNqygLeNuX\nQh0L5OUncTAP2lhDu8ViqeN4s7R1mqr2FpG1AKp6qDbjWx0txKfF4lBoF92xtkWxWCyWWsWbJaoC\nEfHDZWB3ZUx0+lSqY4DdabEAtI/uXLuCWCwWSy3jKdZW0WxlCvA5ECMiTwK/YQIn1mni0uIBaNfg\nxFqWxGKxWGoXT0tbK4HeqvqhiKwBzsHE27pcVTfUiHRHMXHp+wG7GdFisVg8KZKiII2o6kZgo+/F\nOXbYk5lEgyB/QgNDa/bn3sUAABhUSURBVFsUi8ViqVU8KZIYEbmnvEpVfdkH8hwzJGSm0aye3Yho\nsVgsnhSJPxCG28zEUsze7CxOirJ52i0Wi8WTItmnqk/VmCTHEE51si87j3NaWkVisVgsntx/7Uyk\nHPZn7qdAoVV449oWxWKxWGodT4rEm5wjdZLYlF0AtI5oVsuSWCwWS+1TriJR1UM1KcixxI5DmwFo\nE+mbFL4Wi8VyLFHngy8eCbtStgDQNqp9LUtisVgstY9VJEdAbMpOIgIgsp4NIW+xWCxWkRwBe9L3\n0CQEAgIa1LYoFovFUutYRXIExGfsp0kwBARY91+LxWKxiuQISMhIpHEIBAZaRWKxWCw+VSQiMkxE\ntojIdhF5qIz6ySKyznVsFZFUV3kvEVkmIhtFZL2IXOnWZ7qI7HLr18uX11CatNw0MgpyaWxnJBaL\nxQJ4l9jqiBARf0wI+nOBeGCViMxT1U1FbVT1brf244CTXafZwLWquk1EmgNrRGSRqqa66u9X1bm+\nkt0TcWlxADQJ8cfPzwZstFhqioKCAuLj48nNza1tUY47QkJCaNmyJYGBgUfU32eKBOgPbFfVnQAi\nMhsYCWwqp/0Y4AkAVd1aVKiqe0XkIBADpJbTt8YoUiStIlohYjf/Wyw1RXx8POHh4bRt29b+71Uj\nqkpycjLx8fG0a9fuiMbw5dJWC2CP23m8q+wwRKQN0A74qYy6/kAQsMOteKJryWuyiASXM+YtIrJa\nRFYnJiYe6TUcxp50c0kdY/pU25gWi6VicnNzadiwoVUi1YyI0LBhwyrN9I4WY/toYK6qFroXikgz\n4CPgelUtSu/7MHAS0A9oADxY1oCq+q6q9lXVvjExMdUm6M5DmwgQaBczsNrGtFgs3mGViG+o6ufq\nS0WSALRyO2/pKiuL0cAn7gUiEgHMBx5R1eVF5aq6Tw15wDTMElqNsSv5LxoFQ0R4v5p8W4vFYjlq\n8aUiWQV0EpF2IhKEURbzSjcSkZOAaGCZW1kQ8CXwYWmjumuWghgVejFQo2l/d6fupHEwhIXVqLOY\nxWI5CvD396dXr1507dqVnj178tJLL+F0OivuWA7PPvvsP69jY2Pp1q1bleSLi4sjLCyMF198sUrj\nVBafKRJVdQB3AouAzcAcVf3/9u4/OKryXOD49zE/CBJ+SqiYFAg/WkmBhiQgHagGLuVH6ajMZbwg\nF6VTBLxEqtyiyNxBxKJ4ZaqiYi9WVG4dpQPeSK0ITCHVjlUgSMFAm8SANhRNDEISCCEhz/3jnCRL\nfm2S3c0um+czs8PuOWfPvu++zD553/ec580RkTUicqvHobOBN1VVPbbdAdwMzG/iMt/XReQocBTo\nC/wyUHVoyqnyYm7o1oPIyNiO/FhjTAjo2rUrhw8fJicnhz179rBz504effTRdp/PM5D4w7Jly5g+\nfbpfz9kagbxqC1V9F3i3wbZVDV6vbuJ9vwV+28w5J/mxiG1yueYyX1VUMGDgkGAVwRgD3H8/HD7s\n33MmJ8Mzz7T++H79+rFp0ybGjBnD6tWrqampYcWKFWRlZVFZWcmSJUtYtGgRWVlZrFq1iu7du5Of\nn8/EiRPZuHEjK1eupKKioq6Hs3btWi5fvsw999zDhx9+SHx8PG+//TZdu7buNoPMzEwSExPp1q1b\nO7+B9guVyfarwudnjnBZYVDv4cEuijEmBAwePJjLly9TVFTEyy+/TM+ePTlw4AAHDhzgpZde4sQJ\nZ+2i/fv389xzz3Hs2DE+++wz3nrrLdatW1fXw3n99dcByMvLY8mSJeTk5NCrVy+2b98OwFNPPUVy\ncnKjx9KlSwEoLy/nySef5JFHHgnK9xDQHkm4yf3yjwAMjbOJdmOCqS09h46ye/dujhw5wrZtzrTu\nuXPnyMvLIzo6mrFjxzJ4sLPsxJw5c/jzn//MrFmzGp0jMTGR5GRnFD81NZWTJ08CsHz5cpYvX97s\nZ69evZoHHniA2NjgDLlbIGmD/OL9AAzrd0uQS2KMCQUFBQVERETQr18/VJXnnnuOqVOnXnFMVlZW\no8trm7vctkuX+tviIiIiqKioAJweSW2vxdPNN9/Mhg0b+Pjjj9m2bRsPPvggZ8+e5ZprriEmJoaM\njAxfq9gqFkjaoOBMDgCDr7sxyCUxxgRbcXExixcvJiMjAxFh6tSpvPjii0yaNImoqChyc3OJj3fu\nwd6/fz8nTpxg4MCBbN26lYULFwIQFRVFVVWV19Qk3nokH3zwQd3z1atXExsb22FBBCyQtMkX5z6n\ne1QUPbr0CHZRjDFBUDs5XlVVRWRkJPPmzWPZsmUALFiwgJMnT5KSkoKqEhcXR2ZmJgBjxowhIyOj\nbrJ95syZACxcuJBRo0aRkpLC2rVrg1YvX8mVV92Gp7S0ND148KBP57h06Ssmbbqekpr+HF/6Tz+V\nzBjTWsePH2f48KvvQpesrCzWr1/PO++8E+yitKip71dEslU1zdt77aqtViory6aoEgb0HBTsohhj\nTEixoa1WKivL5quLMLFPUrCLYoy5iqSnp5Oenh7sYgSU9UhaqejsfkqrYVAvuxnRGGM8WSBppc++\nzgZgQM8BQS6JMcaEFgskrXDp0tcUlp4GLJAYY0xDFkhaobz8EEWVznMLJMYYcyULJK1QVuYEEkG4\nofsNwS6OMSZIQjWN/MmTJ+natWtdDq7Fixe3u0ztYVdttUJ5+SFKqrvTv3t3oiJavgPVGBO+apMs\nAhQVFXHnnXdSWlra7lTyjz/+OCtXrvRL2YYMGVJXto5mgaQVysqyKb3cjf6x/YNdFGMMhEQe+VBL\nIx9MNrTlRVXVN1y8WMC5qgj6desX7OIYY0JIqKSRBzhx4gSjR4/mlltuuSL3VkewHokX5eXOXz1n\nLlWR3C0uyKUxxgAhmUc+mGnk+/fvzxdffMF1111HdnY2t99+Ozk5OfTo0TF5AS2QeFFWlo0qfF1R\nSr9rrUdijKkXKmnku3TpUvfe1NRUhgwZQm5uLmlpXtNk+YUFEi/Kyw9RE5nAxepCG9oyxtQJpTTy\nxcXF9OnTh4iICAoKCsjLy6vrAXWEgM6RiMg0Efm7iOSLyIom9j8tIofdR66InPXYd7eI5LmPuz22\np4rIUfecG6S50O4nZWWHuBTlZMSMs6EtYzo1z8nxyZMnM2XKlLrlbRcsWEBSUhIpKSmMGDGCRYsW\nUV1dDdSnkR8+fDiJiYmN0sjPnTvXp3K9//77jBo1iuTkZGbNmsWvf/1r+vTp41tl20JVA/IAIoDP\ngMFANPBXIKmF4+8DNrvP+wAF7r+93ee93X37gXGAADuB6d7Kkpqaqu1RVVWq+/aJvrV/obIa/UPu\nH9p1HmOM744dOxbsIrTLvn37dMaMGcEuhldNfb/AQW3F730geyRjgXxVLVDVS8CbwG0tHD8HeMN9\nPhXYo6pnVPUbYA8wTUT6Az1U9SO3kluA2wNVAWeiXbkg1wPY0JYxxjQhkIEkHviHx+tCd1sjIjIQ\nSAT2enlvvPu8NedcKCIHReRgcXFxuypQXn4IgLKanoAFEmNM26Wnp4f8ola+CpX7SGYD21T1sr9O\nqKqbVDVNVdPi4to3t1FWdojo6Bv4ptJJtBV3rc2RGGNMQ4EMJKeAb3u8TnC3NWU29cNaLb33lPu8\nNef0WUzMQOLiZlF8oZjY6Fi6RoX+HabGGNPRAhlIDgDDRCRRRKJxgsWOhgeJyI04E+p/8di8C5gi\nIr1FpDcwBdilqqeBUhEZ516tdRfwdqAqkJi4hmHDnqXofJENaxljTDMCdh+JqlaLSAZOUIjAuSIr\nR0TW4FwJUBtUZgNvupPnte89IyKP4QQjgDWqesZ9/h/Aq0BXnKu2dgaqDrWKzhfZsJYxxjQjoHMk\nqvquqn5HVYeo6lp32yqPIIKqrlbVRveYqOpmVR3qPl7x2H5QVUe458zwDECBUnyh2HokxpiQTSNf\nUlLCxIkTiY2NJSMj44p92dnZjBw5kqFDh7J06VIC8ZMZKpPtIc2GtowxUJ9GPicnhz179rBz5852\np5CHKwOJL2JiYnjsscdYv359o3333nsvL730Enl5eeTl5fHee+/55TM9WYoUL1TVAokxISYv7/66\nhKr+EhubzLBhV2ca+W7dujFhwgTy8/Ov2H769GlKS0sZN24cAHfddReZmZlMnz69bV+OF9Yj8eLs\nxbNU11TbHIkxppFQSiPflFOnTpGQUH+ha0JCAqdO+f9CV+uReFF8wbmZ0XokxoSOtvQcOkow08gH\nmwUSL4rOFwEWSIwxjYVKGvnmxMfHU1hYnwyksLCwLiOxP9nQlhe1gcQy/xpjPDWXRr6qqgqA3Nxc\nzp8/D9Snka+pqWHr1q1MmDABqE8j783y5cs5fPhwo0dLQQScBa969OjBRx99hKqyZcsWbrutpZSH\n7WM9Ei+Kz9vQljHGUTs5XlVVRWRkJPPmzWPZsmWAk0b+5MmTpKSkoKrExcWRmZkJ1KeRr51sb5hG\nPiUlhbVr1/pUtkGDBlFaWsqlS5fIzMxk9+7dJCUlsXHjRubPn09FRQXTp0/3+0Q7gHTAbRhBl5aW\npgcPHmzXex/702OsylpF5X9VEh0R7eeSGWNa6/jx4wwfPjzYxWizrKws1q9fH/KJG5v6fkUkW1W9\nLrNoQ1teFJ0vondMbwsixhjTDBva8qL4QrHNjxhj2i09PZ309PRgFyOgrEfihd2MaIwxLbNA4oUF\nEmOMaZkFEi8s868xxrTMAkkLLtdcpqSixHokxhjTAgskLThTcYYarbFAYowBQjeN/J49e0hNTWXk\nyJGkpqayd+/eun2WRj7I6u5qt6EtYwyhm0a+b9++/P73v+fo0aO89tprzJs3r26fpZEPMkvYaExo\nuv+9+zn8pX/TyCdfn8wz067ONPKjR4+ue/69732PiooKKisrOXPmjKWRDzZL2GiMaUkoppHfvn07\nKSkpdOnSxdLIhwILJMaEprb0HDpKKKSRz8nJ4aGHHmL37t1+qlXrBDSQiMg04FkgAviNqq5r4pg7\ngNWAAn9V1TtFZCLwtMdhNwKzVTVTRF4FbgHOufvmq6p/+7iu4vPFCEKfrn0CcXpjzFUulNLIFxYW\nMnPmTLZs2cKQIUOAjksjH7BAIiIRwAvAj4BC4ICI7FDVYx7HDAMeBsar6jci0g9AVfcBye4xfYB8\nwDPELlfVbYEqe62i80X0vbYvEddEBPqjjDFXmebSyE+aNImoqChyc3PrfrRr08gPHDiQrVu3snDh\nQqA+jXxUVFSLn+WtR3L27FlmzJjBunXrGD9+fN12zzTyN910E1u2bOG+++7zQ+2vFMg5krFAvqoW\nqOol4E2gYSL8e4AXVPUbAFUtauI8s4CdqnohgGVtUtEFu6vdGFPPc3J88uTJTJkyhUceeQRw0sgn\nJSWRkpLCiBEjWLRoEdXV1UB9Gvnhw4eTmJjYKI383LlzfSrX888/T35+PmvWrKmbPykqcn5ON27c\nyIIFCxg6dChDhgy5utLIi8gsYJqqLnBfzwNuUtUMj2MygVxgPM7w12pVfa/BefYCv1LVd9zXrwI/\nACqBPwIrVLWyic9fCCwEGDBgQOrnn3/e5jo88cETlFaW8sTkJ9r8XmOMf1ka+cDyJY18sCfbI4Fh\nQDqQALwvIiNV9SyAiPQHRgK7PN7zMPAlEA1sAh4C1jQ8sapucveTlpbWrmj58A8fbs/bjDGmUwnk\n0NYp4NserxPcbZ4KgR2qWqWqJ3B6J8M89t8B/J+q1q1Fqaqn1VEJvIIzhGaMMSEpPT095Hsjvgpk\nIDkADBORRBGJBmYDOxock4nTG0FE+gLfAQo89s8B3vB8g9tLQZzLHm4HPg1E4Y0xoaczrOgaDL5+\nrwELJKpaDWTgDEsdB36nqjkiskZEbnUP2wWUiMgxYB/O1VglACIyCKdH86cGp35dRI4CR4G+wC8D\nVQdjTOiIiYmhpKTEgomfqSolJSXExMS0+xy2Zrsx5qpQVVVFYWEhFy9eDHZRwk5MTAwJCQmNLkO+\nWibbjTGmVaKiokhMTAx2MUwTLNeWMcYYn1ggMcYY4xMLJMYYY3zSKSbbRaQYaMut7X2BrwNUnFDV\nGesMnbPenbHO0Dnr7WudB6qq15X9OkUgaSsROdiaKxXCSWesM3TOenfGOkPnrHdH1dmGtowxxvjE\nAokxxhifWCBp2qZgFyAIOmOdoXPWuzPWGTpnvTukzjZHYowxxifWIzHGGOMTCyTGGGN8YoHEg4hM\nE5G/i0i+iKwIdnkCRUS+LSL7ROSYiOSIyM/d7X1EZI+I5Ln/9g52Wf1NRCJE5BMRqV1xM1FEPnbb\nfKu75EFYEZFeIrJNRP4mIsdF5Afh3tYi8oD7f/tTEXlDRGLCsa1FZLOIFInIpx7bmmxbcWxw639E\nRFL8VQ4LJC4RiQBeAKYDScAcEUkKbqkCphr4T1VNAsYBS9y6rgD+qKrDcJcxDmIZA+XnOMsa1HoS\neFpVhwLfAD8LSqkC61ngPVW9Efg+Tv3Dtq1FJB5YCqSp6gicZbxnE55t/SowrcG25tp2Os7CgcNw\nliF/0V+FsEBSbyyQr6oFqnoJeBO4LchlCgh3lclD7vMynB+WeJz6vuYe9hrOwmFhQ0QSgBnAb9zX\nAkwCtrmHhGOdewI3Ay8DqOoldynrsG5rnMzmXUUkErgWOE0YtrWqvg+cabC5uba9DdjirjD7EdCr\ndqFAX1kgqRcP/MPjdaG7Lay5C4iNBj4GvqWqp91dXwLfClKxAuUZ4EGgxn19HXDWXYQNwrPNE4Fi\n4BV3SO83ItKNMG5rVT0FrAe+wAkg54Bswr+tazXXtgH7jbNA0omJSCywHbhfVUs996lzXXjYXBsu\nIj8BilQ1O9hl6WCRQArwoqqOBs7TYBgrDNu6N85f34nADUA3Gg//dAod1bYWSOqdwlnat1aCuy0s\niUgUThB5XVXfcjd/VdvVdf8tClb5AmA8cKuInMQZtpyEM3fQyx3+gPBs80KgUFU/dl9vwwks4dzW\nk4ETqlqsqlXAWzjtH+5tXau5tg3Yb5wFknoHgGHulR3ROJNzO4JcpoBw5wZeBo6r6q88du0A7naf\n3w283dFlCxRVfVhVE1R1EE7b7lXVucA+YJZ7WFjVGUBVvwT+ISLfdTf9C3CMMG5rnCGtcSJyrft/\nvbbOYd3WHppr2x3AXe7VW+OAcx5DYD6xO9s9iMiPccbRI4DNqro2yEUKCBGZAHwAHKV+vmAlzjzJ\n74ABOGn371DVhhN5Vz0RSQd+oao/EZHBOD2UPsAnwL+ramUwy+dvIpKMc4FBNFAA/BTnj8iwbWsR\neRT4N5wrFD8BFuDMB4RVW4vIG0A6Trr4r4BHgEyaaFs3qD6PM8x3Afipqh70SzkskBhjjPGFDW0Z\nY4zxiQUSY4wxPrFAYowxxicWSIwxxvjEAokxxhifWCAxnZqIqIj81uN1pIgUe2QHvrWtmaBFpNxP\nZXvDzdL6QIPt3xWRLBE57Gbz3eRuTxORDf74bGPaItL7IcaEtfPACBHpqqoVwI/wuNtXVXcQhBtT\nReR6YIybqbahDThZbN92jx0J4N4T4Jf7AoxpC+uRGAPv4mQFBpgDvFG7Q0Tmi8jz7vNX3fUcPhSR\nAhGZ1cS5miQicSKyXUQOuI/x7vaxIvIXN6Hihx53oO8G4t1exw8bnK4/TuoTAFT1qHuudI+e1Lvu\new+LyDkRuVuctViecj//iIgsatO3ZEwzLJAY49ztPFtEYoBROHf4N6c/MAH4CbCuDZ/xLE4vYgzw\nr7ip7IG/AT90EyquAh53t98KfKaqyar6QYNzPQ3sFZGd4izg1Kvhh6nqj1U1GWfNjc9x7nb+GU5a\njDHAGOAeEUlsQx2MaZINbZlOT1WPuOn05+D0TlqSqao1wDERaUvq9clAkpOlAoAebvblnsBrIjIM\nJ0trVCvK+4qI7MJJdXEbsEhEvt/wOBHpC/wvToqMcyIyBRjl0ZPqibPI0Yk21MOYRiyQGOPYgbOG\nRTrOOiXN8czNJM0e1dg1wDhVvei50R0226eqM91gltWak6nqP4HNwGZxllkd0eC8ETg9rTWqWrsM\nqwD3qequNpTbGK9saMsYx2bg0dr5hgDYDdxX+8JNpAhOr6B2cn9+a04kItPcZQBqJ+Wvo3E68HXA\nEVV902PbLuBej/d+x13kyhifWCAxBlDVQlX116Wz14pIocdjGe4a4u4k9zFgsXvsfwNPiMgntH6E\nYArwqYj8FSc4LHfTxXv6BTDFY8L9Vpx5mWPAIbcX8z9t+ExjmmXZf40xxvjEeiTGGGN8YoHEGGOM\nTyyQGGOM8YkFEmOMMT6xQGKMMcYnFkiMMcb4xAKJMcYYn/w/Mz9buW2H1Z0AAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"lab = 'y_buy'\n",
"\n",
"depths = [4, 5, 10, 20]\n",
"leaves = np.arange(1, 101)\n",
"\n",
"#Run all of the options\n",
"run=1\n",
"if (run == 1):\n",
" #Initialize dictionary of results\n",
" res = dict()\n",
" for d in depths:\n",
" res[d] = list()\n",
"\n",
" #Now train and get results for each option\n",
" for d in depths:\n",
" for l in leaves:\n",
" res[d].append(testTrees(train.drop(lab, 1), train[lab], test.drop(lab, 1), test[lab], d, l, 1))\n",
"\n",
"\n",
"#Now plot \n",
"fig = plt.figure()\n",
"ax=fig.add_subplot(111)\n",
"plt.plot(leaves,res[depths[0]], 'b-', label = 'Depth={}'.format(depths[0]))\n",
"plt.plot(leaves,res[depths[1]], 'r-', label = 'Depth={}'.format(depths[1]))\n",
"plt.plot(leaves,res[depths[2]], 'y-', label = 'Depth={}'.format(depths[2]))\n",
"plt.plot(leaves,res[depths[3]], 'g-', label = 'Depth={}'.format(depths[3]))\n",
"plt.legend(loc = 4)\n",
"ax.set_xlabel('Min Leaf Size')\n",
"ax.set_ylabel('Test Set AUC')\n",
"plt.title('Holdout AUC by Hyperparameters')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
The above plot shows how holdout AUC varies by parameters. As we generally expect with most hyper-parameter searches, there is a \"sweet spot\" between letting the model be flexible and over fitting. Here we see that as long as we restrict the min_leaf_size, the max_depth doesn't affect the optimal choice too much.
\n",
"\n",
"We might want to peek at the tree to determine which features it thinks are important. The DT class objects actually have an attribute that gives each feature a score for how important it is in building the tree. \n",
"\n",
"\n",
"