{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "1. Draw an example (of your own invention) of a partition of two- dimensional feature space that could result from recursive binary splitting. Your example should contain at least six regions. Draw a decision tree corresponding to this partition. Be sure to label all as- pects of your figures, including the regions R1, R2, . . ., the cutpoints t1,t2,..., and so forth." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is mentioned in Section 8.2.3 that boosting using depth-one trees (or stumps) leads to an additive model: that is, a model of the form: `SEE ISL p332`\n", "\n", "Explain why this is the case. You can begin with (8.12) in Algorithm 8.2.\n", "\n", "... We can see this by inspection?!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "3. Consider the Gini index, classification error, and entropy in a simple classification setting with two classes. Create a single plot that displays each of these quantities as a function of pˆm1. The x- axis should display pˆm1, ranging from 0 to 1, and the y-axis should display the value of the Gini index, classification error, and entropy.\n", "\n", "*Hint: In a setting with two classes, pˆm1 = 1 − pˆm2. You could make this plot by hand, but it will be much easier to make in R.*" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "<matplotlib.axes._subplots.AxesSubplot at 0x10f565320>" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xd0VNX2wPHvmUnvCQlJSIDQexApUgQRKUGwgSJFCVGqFCtFARVUUASUJtIJShEFn0gJVelINyH0EiAhISGN9DJzfn8kvB/6KEnI5E6S81kr6zGTe+/Z8haz5+57ztlCSomiKIqi3KHTOgBFURTFvKjEoCiKovyDSgyKoijKP6jEoCiKovyDSgyKoijKP6jEoCiKovyDSgyKoijKP6jEoCiKovyDSgyKoijKP1hoHUBRuLu7Sz8/P63DUBRFKVWOHTt2S0rp8bDjSmVi8PPz4+jRo1qHoSiKUqoIIa4W5DhVSlIURVH+weSJQQgRIIQ4J4S4KIQYd4/ffyOEOJn/c14IkWTqmBRFUZT7M2kpSQihB+YBnYBI4IgQYoOU8vSdY6SU7951/EigiSljUhRFUR7M1HcMLYCLUsrLUspsYA3wwgOO7wOsNnFMiqIoygOYOjH4ANfveh2Z/97/EEJUBaoBu+7z+8FCiKNCiKNxcXHFHqiiKIqSx5wePvcGfpFSGu71SynlQillMyllMw+Ph862UhRFUYrI1IkhCqh812vf/PfupTeqjKQoiqI5U69jOALUEkJUIy8h9Ab6/vsgIURdwBU4aOJ4FOWeLtxMIeRUDDkGo9ah4GxnxQuPVcLdwVrrUJRyyqSJQUqZK4QYAWwF9MBSKWW4EGIycFRKuSH/0N7AGqkaUCslyGCU/HE2luUHIth38RYAQmgcFCAlfLXlLM81rsSA1n408nXWOiSlnBGl8bO4WbNmUq18VooqOSOHn49eZ8XBq1xLSMfLyYbXW1WlT4squNlbaR0eF2NTWXEwgl+ORZKebaBZVVcGtPGjSwMvLPXm9FhQKW2EEMeklM0eepxKDEp5cTE2heADV1l3PO8Dt7mfKwNaV6NzA0+z/MC9nZnDz0cjWXEwgqvxeQnstZZV6NOiChVUmUkpApUYFAUwGiV/nMsrF+29cAsrCx0vNK5EYGs/GvqUjhKN0Sj583wsy/b//3/D8/llptLy36CYB5UYlHLtXt+2X29Vld7NK5fqb9ul7a5HMS8qMSjlUnmpzydn5PDLsUiCD0RwLSEdb2cbXmtZ+hOfYloqMSjlhtEo2X0+jmUHIthzPg4rva7czOgxGCV/3qNUNqCNHw0qle3/dqXwVGJQyryUzP//1hwRn46nkzWvPVGVPk9UKZdrAC7cTCH4YATrjkWRkWOghZ8bA9r40bm+JxZl6G5JKTqVGJQy61JcKisO5JWL0rINNK3qyoDWfgQ0LFvloqK6Mx03+GAE1xMy8Ha+83zFPKbjKtpRiUEpU4xGye4LcSzfH8Hu/HJR98beDGjth7+vi9bhmSWDUbLrbCzLD1xh/8V4rC10vPBY3owsVWYqn1RiUMqElMwc1h2LJPjgVa7cSqOiozWvtcxbjObhWP7KRUV1/mYKwQciWH88v8xUzY2g1n50UmWmckUlBqVUuxyXyoqDV/nlWCSpWbk8XsWFAW2qEdDACysL9UFWVMnpOazNLzNFJmZQydmG11v50bt5ZVxVmanMU4lBKXWMRsmeC3EsPxDBn+fisNQLnvPPK300rqzKRcXJYJTsPHOT5QciOHApr8z04mM+DGjjRz1vJ63DU0xEJQal1EjNys0rFx2I4PKtNDwc78wuqkxFRxutwyvzzsWksPxABL+eiCQzx8gT1dwIauNHx3qqzFTWqMSglAqHLscz9MdjJKXn8FhlF4La+NG1obcqF2kgKT07r8x04CpRSRnU9XJkWVBzvJ1ttQ5NKSYqMShmL+RUDKPWnKCyqy3TX2lMkyquWoekkFdm2hwWzYfrw3CysWDFmy2oWdFR67CUYlDQxKC+limaWH34Gm+tPEZ9byd+GdpaJQUzotcJnmtciTWDW5JtkLz8/UGOX0vUOiylBKnEoJQoKSVzdl7gw/VhtKvtwapBT6jZMGaqoY8z64a1wtnWkn6L/uKPc7Fah6SUEJUYlBJjMEo+2RDOjO3neamJD4v6N8POytTdZZVHUbWCPb8MbU11D3sGBR9l/fFIrUNSSoBKDEqJyMo1MGrNCVYcvMqgttWY8UpjtX1FKeHhaM2awS1pUc2N99b+zaI9l7UOSTEx9S9TMbnUrFzeWH6ETaHRfNi1LuO71UenM4PmykqBOdpYsiyoOc828uKLzWeYuvkMpXHiilIw6j5eMalbqVkMWHaYM9EpTH+lMS839dU6JKWIrC30zOnzOG72p1iw5zK3UrP5smcjdedXBpn8/1EhRIAQ4pwQ4qIQYtx9juklhDgthAgXQqwydUxKybiekM7L8w9wMTaVRf2bqqRQBuh1gs9eaMi7HWuz7ngkQ344Rka2QeuwlGJm0sQghNAD84CuQH2gjxCi/r+OqQV8CLSRUjYA3jFlTErJOH3jNj3mHyAxPYeVA1vSoa6n1iEpxUQIwdsda/HFSw3581ws/RYfIik9W+uwlGJk6juGFsBFKeVlKWU2sAZ44V/HDALmSSkTAaSUak5cKXfocjyvLjiIhU7wy9BWNK2q1iiURf2eqMp3/R7nVNRtXv7+IDeSMrQOSSkmpk4MPsD1u15H5r93t9pAbSHEfiHEISFEwL0uJIQYLIQ4KoQ4GhcXZ6JwlUcVciqa/ksP4+lsw7phranlqVbMlmUBDb0JfqMFN5Mz6Tn/ABdjU7QOSSkG5vDUyAKoBbQH+gCLhBD/s5WmlHKhlLKZlLKZh4dHCYeoFMSqv67x1srjNKjkxM9DWlHJRe2xUx60qlGBNUNakqNWSZcZpk4MUUDlu1775r93t0hgg5QyR0p5BThPXqJQSgkpJbN3XuCjX/NWM68cqFYzlzcNKjmzflhrnG0t6bvoEH+cVRXh0szUieEIUEsIUU0IYQX0Bjb865j/kHe3gBDCnbzSklpBU0rcWc08c/t5eqjVzOValQp2/DK0NTU8HBi4Qq2SLs1MmhiklLnACGArcAZYK6UMF0JMFkI8n3/YViBeCHEa+AMYLaWMN2VcSvG4ezXz4HbVma5WM5d7d1ZJP6FWSZdqatttpUhSMnMY8sMxDlyK56Nn6zK4XQ2tQ1LMSFaugfd++ptNYdEMbledcQF11Wp3M1DQbbfVPb9SaHEpWQQtz1vNPOOVxvRUC9eUf7G20DO7TxPc7K1YuOcyt1Kz+Kqnv7qjLCVUYlAK5Vp8Ov2X/kXM7UwW92/G03Urah2SYqb0OsHkFxrg4WjNzO3nSUzLZl6/x9UzqFJApW+lwMJvJNPz+wMkZeSwalBLlRSUhxJCMOqZWkx5qRG7z8fRb/FfJKapVdLmTiUGpUAOXoqn94JD/13N/LjquKYUQt8nqvBdv8cJv3GbVxaoVdLmTiUG5aFCTkUTuOz/VzOr/r9KUQQ09CY4SK2SLg1UYlAeaOVfV3lr5XEaVnLil6FqNbPyaFrVqMBPQ1qRa8xbJX3sqlolbY5UYlDuSUrJrB0XGP/rKZ6q7cHKgS1xsVOrmZVHV7+SE+uGtsbF1pJ+i9UqaXOkEoPyP+6sZv5mx3l6PO7Dwv7NsLXSax2WUoZUqWDHz0NbU7Ni3irpdcfUKmlzohKD8g9ZuQZGrc5bzTykXXXVm1kxGQ9Ha1YPaknL6m68//PfLNxzSeuQlHzqX7zyXymZOQQtO8KmsGjGP1uPD5+thxBqtapiOo42liwd0Jxu/t5M2XyWKZvPYDSWvt0Yyhq10kQB/n8189noFGb2akyPx9VqZqVkWFvomd27CRXUKmmzoRKDwrX4dF5f+hext7NYFNiMp+uohWtKydLrBJOeb4CHgzUz1CppzamUXM6F30imx/wDJGfksHLQEyopKJoRQjDymVpM7aFWSWtNJYZy7M5qZiu9Ws2smI8+LarwXb+mapW0hlRiKKeikjIYGHwEL2cbflGrmRUzE9DQixX5vaQHBh8lx2DUOqRyRSWGckhKyUfrw5DA0gHN1WpmxSy1rF6Br19pzOno2yxUDX9KlEoM5dCvJ6LYfT6OMV3qUNnNTutwFOW+Ahp68WwjL2btvMDF2FStwyk3VGIoZ+JSspi88TTNqrrSv5Wf1uEoykNNer4htpZ6xq0LVWscSohKDOXMpxvCSc8y8GVPf9VqUSkVPByt+bh7fY5eTeSHQ1e1DqdcMHliEEIECCHOCSEuCiHG3eP3A4QQcUKIk/k/A00dU3kVciqGTWHRvN2xFjUrOmgdjqIUWI/HfXiqtgdfhZwlMjFd63DKPJMmBiGEHpgHdAXqA32EEPXvcehPUsrH8n8WmzKm8io5PYeJv52ivrcTg9tV1zocRSkUIQRfvNQQAXy4PgwpVUnJlEx9x9ACuCilvCylzAbWAC+YeEzlHr7YfJqEtGymvay2GlBKJ19XO8Z2rcveC7dYdzxK63DKNFN/QvgA1+96HZn/3r/1FEKECiF+EUJUNnFM5c6+C7dYezSSwe2q09DHWetwFKXIXnuiKs39XPls42liUzK1DqfMMoevjr8DflJKf2A7EHyvg4QQg4UQR4UQR+Pi4ko0wNIsLSuXcetDqe5uz9vP1NI6HEV5JDqd4Mue/mTkGPjkt3CtwymzTL1DVRRw9x2Ab/57/yWljL/r5WJg2r0uJKVcCCwEaNasmSowFtD0beeITMzg56GtsLFUzXbuJfPceRJXrSJlxw5kTo7W4aB3csL5pRdx7dULCw8PrcMxOzU8HHinYy2mhZxjS1g0XRt5ax1SmWPqxHAEqCWEqEZeQugN9L37ACGEt5QyOv/l88AZE8dUbhy7msjyAxH0b1WV5n5uWodjVmRODik7d5L440rSjx5FWFvj2LEjejft/56yL1/m1py53Pp+AU6dOuH6Wj9smzRRvTHuMqhtdTaFRjPxt3Ba13DH2c5S65DKFJMmBillrhBiBLAV0ANLpZThQojJwFEp5QZglBDieSAXSAAGmDKm8iIr18DYdaFUcrZlTEBdrcMxG7lxcSSuXUvST2vJjY3F0seHiqM/wLlHDyxczWcTweyICBJXryZp/a/c3rwZ67p1ce3XF+fu3dHZqi1MLPU6pr3sz/Nz9/PZptNMf6Wx1iGVKaI0Tvtq1qyZPHr0qNZhmLUZ284xZ9dFlgc1p30530pbSknGiRMk/riS29u3Q04O9k8+iWu/vji0a4fQm2+JzZieTvLvG0lcuZKs8+fROTnh0qMHrn37YFWlitbhae7rrWeZ98clVrzRgna1VdntYYQQx6SUzR56nEoMZc/pG7d5fu4+nn+sEjN7PaZ1OJoxZmSQvHEjiStXkXX2LDpHR1x6vIRL795YV6umdXiFIqUk49gxElauJGX7DjAYsG/XFre+fbFv2xahM4d5JCUvM8dAt9l7ycwxsu3ddthbq8Y+D6ISQzmVazDy0ncHiE7OYMd7T+FiZ6V1SCUu+9o1EletJmn9eoy3b2Nduzau/frh/Fx3dHalf9PAnJuxJK1dS+LanzDE3cKyShVce/fGpcdL6F1ctA6vxB2NSOCVBQcJbOXHp8830Docs6YSQzn1/e5LfLnlLN/1e5xny9FsDWk0krZ3LwkrV5K2dx/o9Th26ohbv37YNm1aJh/cyuxsUnbsIGHlKjKOHUPY2ODUvRtufftiU/9eGwyUXZ9uCCf4YAQ/D2lFMzXR4r5UYiiHLsel0nXWXtrX8eD718rmh+G/GZKSSFr/K4mrV5Nz/Tp6D3dce72KS69eWHqWn2crmWfPkrhyFckbNyIzMrBt0gTXfv1w6twJYVX27xrTsnLp/M0erC11bB7VVk3Nvg+VGMoZo1HSe9EhzkbfZsd7T1HRyUbrkEwq8/RpElat4vbGTcjMTGybNsWtX18cO3YsFx+E92NITibp1/xEefUaend3XHu9gsurr2Lp6al1eCa153wc/ZceZvjTNRjdRc3EuxeVGMqZHw5dZeJ/TjHtZX96NSubu4rI7Gxub9tO4sqVZJw4gbCxwfm553Dt1xebuuqD4G7SaCRt/34Sf1xJ6p49oNPh2LEjrv36Yte8eZm9m/zg57/59UQUG0a0oUEltf3Lv6nEUI5EJWXQeeZuHq/qyoo3WpS5f/Q5N2+S9NNPJK79GcOt/Ietffvg8tJL6J3VP/6Hyb5+ncTVa0hatw5jcjLWtWrlrYl47jl09vZah1esktKz6ThzD55O1vxneBu1YeS/qMRQTkgpCVp+hMNXEtj6Trsy06pTSkn6kSMkrszbqgKjEYd27XB9rR/2bdqU2+mZj8KYkcHtzZtJWLmSrNNn0Dk44NzjJVz79Cl103cfJORUNEN/PM6YgDq81b6m1uGYlYImBjXpt5T7z8ko/jwXxyfP1S8TScGYlkby77/nrT24cAGdszNugYG49umNVeWyWSIrKTpbW1x69sS5Rw8yTp4kceUqElevIXHFD9i3aZO34O+pp8x6wV9BBDT0pmtDL77dcYEuDbyo4aGaUhWWumMoxW6lZtFx5m6qu9vz89DW6Etxq86sy1dIXL2a5F9/xZiainX9erj164fTs8+qLSBMKPfWLZJ+/pnENT+Re/MmlpUq4dKnNy4vv2xWW4QUVmxKJp1m7qG2pwM/DW6l2tjmU6WkcmDEquNsC7/J5refpGZFR63DKZKc2FiiJ04kbfcesLTEqUsXXPv1xfaxx0r0WcmlpEvMOTGHjNyMEhvzfjztPBndfDSOViX3/6nMzSVl5y4SV64k/fBhhJUVLr16UXHMaHSldJbXL8ci+eDnv5n8QgP6t/LTOhyzoEpJZdy28Bg2hkbzQefapTYpZJ4+zfVhb2FIScF91Mi8babd3Us8jvD4cIZuH4pEUtWpaomP/w8SDscc5mzCWb7v9D1uNiWzWEtYWODUpTNOXTqTdeECCStWkPjjj2SePYPvnDml8u6h5+M+/HYyiq+2nKVD3Yr4upb+UmtJUXcMpVByRg6dZu6mgoM1G0aUzpkXKTt2EDV6DHoXFyp/Nw+bevU0iePYzWMM3zkcF2sXFnVeRGVH7Z9j7I3cy7t/vouPgw8LOy3E016b9QfJmzYR/dF4LDw8qPz9fKxrlr4HudcT0uny7R6a+bkRHFR2p+kWVEHvGErfJ4rC1M1niE/L5utS2L9ZSsmtRYuIHDkK61q1qLb2J82Swr6ofQzdPpSKdhUJDgg2i6QA0Na3Ld93/J6b6TcJDAnkesr1h59kAs7dulF1RTDGzEwievchde8+TeJ4FJXd7BgbUJc95+NYr/pEF1jp+lRR2H/xFmuOXGdQ29LXv9mYnU30hx8RN2MmTl0DqLoiWLMOZduvbmfkrpH4OfuxrMsyzb6V308zr2Ys7ryY1JxUBmwZwOWky5rEYdu4MdXW/oSlry/Xhwwh4ceVmsTxKF5vWZVmVV2ZvPE0cSlZWodTKqjEUIqkZ+f1b67mbs87HUtX/+bcxESuBb1B8n/+g/vw4VSaMQOdjTbbdmy4tIEPdn9AwwoNWdJlCRVsK2gSx8M0dG/Isi7LMGJkQMgATsef1iQOy0qV8Fv5Iw7t23Pz88+JmTwZmZurSSxF8d8+0dkGPtlwSutwSgWVGEqRGdvOcz0hgy97NCpVm4RlXbxIxCu9yDx1ikozpuMxcoRmtd7VZ1czft94Wni1YEGnBThZOWkSR0HVcq1FcEAwtha2vLn1TU7EntAkDp29Pb5zZuP25hskrlrN9cFDMNy+rUksRVGzogNvd6zF5rAYQk5FP/yEck4lhlLi+LVElu6/wustq/JEdfP8hnsvqXv3EdG7D8bMTKquCMa5WzfNYlkctpgpf03h6cpPM/eZudhZlo5ZKlWcqhDcNRh3W3eGbB/CgRsHNIlD6PV4jh6N9xefk3bkCBG9+5B97ZomsRTF4HbVqe/txMTfwklOz9E6HLOmEkMpkJVrYOwvoXg72TAmoI7W4RRYwo8ruT5kCJa+vlRb+xO2jbXpyyul5Ntj3zLr+Cy6Ve/GjPYzsNZbaxJLUXnZe7E8YDlVHKswYucIdl7dqVksLj17UmXJYgzx8UT0epX0I0c0i6Uw7vSJTkjL5vNN2pTlSguVGEqBeX9c4kJsKl/0aISjjaXW4TyUzMkhZvJkbn7+OQ7t2+O38kcsK1XSJBajNDLlryksObWEV2q/wpQnp2CpM/+/w3upYFuBJV2WUK9CPd7f/T6/X/pds1jsW7TA7+e16N3cuPrGmyStW69ZLIXR0MeZwe2q8/OxSPZeiNM6HLNVoMQghNAJIXoVZQAhRIAQ4pwQ4qIQYtwDjusphJBCiIfOsS1PzkTf5rs/LtKjiQ9P1zH/xjOG27e5PmQoiatW4/bmG/jOma3ZDp65xlwm7p/ImnNrCGoQxMSWE9GJ0v1dyNnamUWdFtHMsxkf7fuIn87+pFksVlWq4LdmNfbNmxM9fjw3v/4aaTBoFk9Bvf1MLaq72/Ph+jDSskrPQ/SSVKB/JVJKIzCmsBcXQuiBeUBXoD7QRwjxPz0HhRCOwNvAX4UdoyzLNRgZuy4UZ1tLJnY3/1aN2VevEtG7D2lHjuD9xed4jh6t2YZs2YZsRu8ezYZLGxjZZCTvNn23zCxusrO0Y17HebT3bc/nf33O0lNLNYtF7+RE5YULcO3bh4QlS4kcOQpjWppm8RSEjaWer172JzIxg6+3ntM6HLNUmK9PO4QQHwghKgsh3O78POScFsBFKeVlKWU2sAZ44R7HfQZ8BWQWIp4yb+n+K4RGJjPphQa42pv3fjVphw8T0etVDPHxVFmyGJeePTWLJSM3g1G7RrHj2g7GtRjHYP/BZSYp3GGtt2bm0zPpWq0r3xz7htnHZ6PVLgbCwgKvjz/Gc8IEUv/8k4h+r5Fz44YmsRRUcz83+reqSvDBCI5dTdA6HLNTmMTwKjAc2AMcy/952L4UPsDdyzYj89/7LyHE40BlKeWmQsRS5l25lcaMbefpXN+Tbo28tQ7ngZLWrePamwPRV6iA389rsW/RQrNYUrJTGLp9KAejDzK59WT61eunWSymZqmzZOqTU+lZqyeLwhbx5eEvMUqjZvG4vdaPygsWkBMZyZVer5Lx99+axVIQYwLqUsnZljG/hJKZY/4lsJJU4GcMwGtSymr/+qn+KIPnX3cm8H4Bjh0shDgqhDgaF1e2HxoZjZJx60KxstDx2YsNzfbbrjQYuDnta6LHT8C+eXP81qzGqkoVzeJJzExk4LaBhMaF8lW7r3ip1kuaxVJS9Do9n7T6hP71+7Pq7Co+3v8xuUbt6uYObZ/Eb81qdDY2XO0fSPIm8/2+52BtwRcvNeRSXBpzd13UOhyzUphnDHOLcP0o4O4NaHzz37vDEWgI/CmEiABaAhvu9QBaSrlQStlMStnMQ6NtFErK6iPX+OtKAhO61cPTSZvVwQ9jTEsjcuQoEpYuxbVvHyovXIDeSbvFYrHpsQSFBHEp6RKzOswiwC9As1hKmhCCD5p9wFuPvcVvl35jzJ4x5Bi0m6dvXbMmfmt/wqZhQ268/wFxc+dpVuZ6mPZ1KtLjcR++332J8BvJWodjNgpTStqZP3OoMF9fjwC1hBDVhBBWQG9gw51fSimTpZTuUko/KaUfcAh4XkpZbrdOvZGUwdTNZ2lTswK9mpnHpm7/lnPjBhH9XiP1zz/xnDABr48/Rlhot4N7ZEokgVsCiU6LZn7H+bTzbadZLFoRQjCs8TBGNxudtw/UHyM17S1h4eZGlWVLcX7xRW7NncuN9z/AmGmejxA/7l4fFztLxq4LJdegXSnOnBQmMQwBfgayhRC3hRApQogHromXUuYCI4CtwBlgrZQyXAgxWQjxfJGjLqOklEz4zykMRsnUl/zNsoSU8fffXOn1KjmRkVResAC317St4V9OvkxgSCC3s2+zuPNimns11zQerfVv0J9PW33KgagDDNsxjNTsVM1i0VlZ4T11ChU/eJ/bW7ZwNTCQXDMsA7vYWTH5hYacirrNor1XtA7HLBQ4MUgpHaWUOimlpZTSKf/1Q2sHUsrNUsraUsoaUsov8t/7WEq54R7Hti/Pdwsb/r7BrrOxfNClDlUqmN92DcmbNnH19f7obG3xW7Mah7ZPahrPmfgzDNgyAIPRwLKAZTTyaKRpPOaiZ+2eTGs3jb9j/2bgtoEkZSZpFosQggoDB+I7ZzZZ5y9wpderZJ49q1k899O1oRddGnjyzY7zXI7TLpmaiwInBpHnNSHExPzXlYUQ2k0/KWNupWbx6YZwmlRxYUBrP63D+QcpJXFz8soBNv6N8Fv7k+ZNW07GnuTNrW9iY2FDcNdgarvW1jQecxNQLYBvn/6WC4kXCNoaRFy6tt/UHTt2xG/ljyAlEX37kbJrl6bx/JsQgs9eaIiNhY6x60IxGs3zmUhJKUwp6TugFdA3/3UqeYvXlGIw6ffTpGUZmNbTH70ZNS43ZmZy4/33uTVvHs4vvkiVpUs1b/N48MZBBm8fjJutG8EBwdq34zRTT1V+ivkd5xOVGkVgSCBRqdo2qrGpXz/vS0WNGkQOH0H8kiVm9VC6opMNE7rX50hEIiv/uqp1OJoqTGJ4Qko5nPxFaFLKRMC8V12VEttP3+T3v28wokNNanmaT//m3Lg4rvYP5PaWECp+8D7eU6do3hh+17VdDN85nMqOlVkesBxvB/Ne46G1Ft4tWNR5EUlZSQRuCeRKsrY1dMuKFam6IhjHgC7Efj2d6AkTkNnZmsZ0t1ea+tK2ljtfbjlLVJJ2D++1VpjEkJO/xYUEEEJ4AOoR/iNKzshhwn/CqOvlyNCnamgdzn9lnj3LlV6vknXhAr5zZlNh4EDNH4ZvvLyR9/58j3pu9VjaZSnutu6axlNaNPZozLIuy8gx5jAgZADnErTdBkJna4vPjBm4v/UWyevWc+3NgeQmJmoa0x1CCKa81AgJfLQ+zKzuaEpSYRLDbOBXoKIQ4gtgHzDFJFGVI19uOUNcShbTXvbHysI8NnhL2bWLiL79QEr8Vv6IY8eOWofE2nNr+WjvRzT1bMr+md9zAAAgAElEQVTCzgtxti5dbU21VsetDsEBwVjprQjaGsTJ2JOaxiN0OjxGjaTS11+T8fffRLzam6zL2rQv/bfKbnaM7lKH3efj+PVE+ewTXZhZSSvJ20hvKhANvCil/PnO74UQ2haeS6EDF2+x+nBe/2Z/Xxetw0FKSfySJUQOH4F1jRp5i5Tqa7953/JTy/ns0Ge09W3LvGfmYW+pzW6tpZ2fsx/BAcG4WrsyePtgDkUf0joknJ/rTpXg5RjT0oh4tTep+/drHRIA/Vv58XgVl3LbJ7pQX1GllGellPOklHOllGf+9WvtOoeUQnn9m8Pwq2DHu520n1Ejs7OJHj+B2K+n4xjQhao/rMCyorbbfEspmXtiLjOOzSDAL2+WjY2Fea4ELy0qOVQiuGswPg4+DN8xnD+v/6l1SNg1aUK1tT9h6e3N9cFDSFi1SuuQ0OsE0172Jz3LwKcbwrUOp8QVZ+3CfKbSlAIzt53nWkI6X/b017x/c25iItfeeJPk9etxf+stfGbMQGej7QewURqZdmQaC0IX0LNWT75s+2WpbbBjbtxt3VkesJzarrV554932Hx5s9YhYenjQ9VVq3Bo25abkz8j5vMvkLna9kqoWdGRUc/UZFNYNCGnYjSNpaQVZ2Ion09piuBEfv/mfk9UoaXG/ZuzLl8m4tXeZISGUmn6dDxGjUTotH3WYTAa+PTAp/x45kder/86n7T6BL1O2+RZ1jhbO7O4y2KaVGzCuL3j+OX8L1qHhN7BHt95c3ELCiLxxx+5PnQYhpQUTWMa8lQN6nk7MfG3U+WqT7R5PO0sZ6ZuOUtFRxvGda2raRyp+/cT8WpvjGlpVF0RjHP3bprGA5BjyGHMnjH8evHX/+79o/VsqLLK3tKe7zp+RxufNkw6OIng8GCtQ0Lo9XiOHYPXZ5NJO3SIiN59yL5+/eEnmoilXsfXL/sTn5rFor3m8XC8JKhSUgmLSc7k8JUE+j1RRdP+zRmnwokc9haW3t5UW/sTto89plksd2TmZvL2H2+z7eq2/+4WqpKCadla2DL76dl0rtqZ6Uen893J78xiiqbrK69QZfFicm/d4tobb2JI0m5bj4Y+zrSp6c7voTfM4u+mJBRmS4waQgjr/D+3F0KMEkLcPZXmmWKPrgzaHBYNwLP+2i3Myk1IIHLkSPTuFagSvBxLH5+Hn2RiaTlpDNsxjH1R+/i41ccENgjUOqRyw1JvybR203ix5ovM/3s+Xx/92iw+AO1bPkHl7+eTExND1OgxmvaT7tbIm6vx6YTfeOC+oWVGYe4Y1gEGIURNYCF5fRb+O31ASqn64xXAxtAb1PN2ooaHgybjy9xcot57H0N8PL6z52i+vQVAclYyg7YN4mTsSb5s+yWv1H5F65DKHb1Oz6TWk+hXrx8/nP6BSQcnYTBq39XMrkkTvMZ/RNrevcTNmaNZHF0aeKHXCX4PNe+WpcWlMInBmL+N9kvAHCnlaEDtR1AIUUkZHL+WRHcN7xZiZ8wk/dAhvCZNwrZhA83iuONWxi2CtgZxLuEc3zz9Dc9Wf1brkMotndAxtvlYBvsPZt2FdYzbO07Thj93uLz6Ks49exD//QJSduzQJAZXeyva1HRnU2i0WdxNmVpht8ToAwQCG/PfU/MHC2FLfhlJqx7OyZs2kbBsGa59++Ly0ouaxHC3G6k3CNwSSGRKJPM6zqN95fZah1TuCSEY2WQk7zV9j5CIEN758x0yc7VtsCOEwOvjj7Fp1IgbY8dptkK6u783kYkZhEaW/U5vhUkMQeTtrvqFlPKKEKIa8INpwiqbNoZG09DHCT/3kl+5m3nuHNETJmL7+ON4jhtb4uP/W0RyBIEhgSRmJbKw00JaerfUOiTlLkENg5jYciJ7I/cyfOdw0nLSNI1HZ22N7+xZCGtrIoePwJBa8j0TutT3wlIv2JT/Ba8sK8yWGKellKOklKvzt79wlFJ+ZcLYypTrCemcvJ5Et0aVSnxsQ3IykSNGondwwHfWtwiNd0g9l3COwJBAsg3ZLO2ylMcqaj8jSvlfver0YkrbKRy7eYzB2waTnKXtN2VLb298vvmG7GvXuDFuHNJYsnt4OttZ8mQ5KScVZlbSn0IIJyGEG3AcWCSEmGm60MqWzRqVkaTBQNTo0eTExOAzexYWHh4lOv6/hcaFErQ1CEudJcsDllPXTdu1HMqDda/enZntZ3Im4QxvbH2DWxm3NI3H/okWeI4ZTeqOncQvXFji43fzr0RUUgYnrms3fbYkFKaU5CylvA30AFZIKZ8AtN92s5TYFBZNY1/nEm/ZGTd3Lml79uI1fjx2TZqU6Nj/djj6MAO3DcTF2oXgrsFUc66maTxKwXSo0oG5z8zlesp1BoQMIDpV21KKa//+OHXvTtys2aTu2VOiY3eq74mVXsem0LJdTipMYrAQQngDvfj/h89KAVyLTyc0MpluJTwbKWXHDuLnf4/zyz1xebVXiY79b3si9zBsxzB8HHwIDsjbxE0pPVpXas2CTguIz4gnMCSQq7e163AmhMD7s8lY16lD1Aejyb52rcTGdra1pF1tdzaHRZfp9p+FSQyTga3ARSnlESFEdeDCw04SQgQIIc4JIS4KIcbd4/dDhRBhQoiTQoh9Qgjt93kuZhvD8uY+P1uCZaSsy5e5MXYcNo0a4TVxoqYriEOuhPD2rrep5VqLZV2W4WGnbTlLKZomFZuwtMtSMnMzCdwSyPnE85rForO1xXfObBCCyBEjMaanl9jY3fy9iU7O5MR182guZAqFefj8s5TSX0r5Vv7ry1LKng86J7/j2zygK1Af6HOPD/5VUspGUsrHgGlAmXtusSk0mscqu+DrWjJlJENqKpHDRyDyZ3LorK1LZNx7WX9hPWP2jKFxxcYs7rwYFxvt+04oRVevQj2WByxHr9MTFBJEWFyYZrFYVa6Mz/TpZF24QPSEiSX2QLhjPU+sLHT8/nfZLScV5uGzjRBiuBDiOyHE0js/DzmtBXl3GJellNnAGuCFuw/If25xhz1lbJfWK7fSCL9xu8QWtUmjkRvjxpF97Ro+336Dpbd2i+l+OP0Dnxz4hNY+rZnfcT4OVtqs9laKV3WX6gQHBONo5cjAbQM5EnNEs1gc2j6JxzvvcHvzZhKWl8wmgI42lrSv7VGmy0mFKSX9AHgBXYDdgC/wsD1xfYC7t0aMzH/vH/ITziXy7hhGFSIms/ffvZFKqIwUv3AhqTt24jlmNPYtWpTImP8mpWT+3/OZdmQanap2Ys7Tc7C1sNUkFsU0fB19CQ4Ixsvei2E7hrEnsmQfAt+twuBBOHbqROz06aQdKpmudN38vYlNyeLo1bJZTipMYqgppZwIpEkpg4FuwBPFEUR+V7gawFhgwr2OEUIMFkIcFUIcjYuLK45hS8Tvf9+gaVVXKrmY/oMxdc8e4mbNxql7d1z79zf5ePcipWTG0Rl8d/I7XqjxAtPaTcNSrxbIl0We9p4sD1hOdefqvL3rbbZGbNUkDiEE3lOnYuXnR9S775Fzw/T7GT1TzxNrCx2byujeSYXaEiP/f5OEEA0BZ+BhvR+jyNts7w7f/PfuZw1wz70apJQLpZTNpJTNPDSei19QF2NTORuTUiJrF7KvXSPqg9FY16mD92eTNXnYbDAamHxoMsGng+lbty+T20zGQmdR4nEoJcfVxpUlXZbg7+Gf10fjwq+axKF3sMd3zhxkTg6RI0dhzDTtNh4O1hY8Xacim0/FYCiD5aTCJIaF+SueJwIbgNPklX4e5AhQSwhRTQhhBfTOP/e/hBC17nrZjQLMdCotNodFI4Tpy0jG9HQiR4wEIfCdMxudbcmXbXKMOXy470N+Of8LgxoNYlyLceiE6gNVHjhaOTK/43xaerfk4wMfs/LMSk3isK5ejUrTviIzPJyYSZNN/jC6e2Nv4lKyOHyl7G0sXZhZSYullIlSyt1SyupSyopSyu8fck4uMIK8aa5ngLVSynAhxGQhxPP5h40QQoQLIU4C75G3SV+ZsCk0muZV3fByNl3/ZCkl0RMmknXhAj7Tp2NVufLDTypmWYYs3vvjPbZc2cI7j7/DqMdHqQY75YydpR1zOszhmSrP8OXhL1kYulCTbSMcO3TA/a1hJP/6K4mrV5t0rA51K2JjqWNTWNkrJz30Pl8I8d6Dfi+lfOD0UinlZmDzv977+K4/v/2wGEqjCzdTOHczhUnPm3Zr64TlwdzevBmPd9/Foe2TJh3rXtJz0hn1xyj+iv6L8U+Mp3fd3iUeg2IerPRWTH9qOh/v/5g5J+aQmp3Ku03fLfEvCe4jRpARHs7NKVOxqVsXu8cfN8k4dlYWPFPXk5BTMXz6XAMs9GXnDrkg/yWO+T8Od/357veUe9gYmldG6trQy2RjpB36i9jp03Hs1IkKgweZbJz7uZ19m8HbB3M05ihTnpyikoKChc6Cz5/8nFfrvMqy8GV8fuhzjLJkN7sTOh0+06ZhWakSkW+/Tc7NWJON1c3fm1up2fxVxspJD00MUspJUspJQA1g1l2vZwN+Jo6vVJJSsiksmiequVHRyTRlpJwbN4h6912s/Pzwnjq1xL+VxWfE80bIG4THhzPjqRk8V+O5Eh1fMV86oWP8E+N5o+EbrD2/lo/2fUSOsWQb/uidnfGdOwdjWjpR77yDzM42yThP16mInZWejWVs76TC3Pv4Syn/u6WglDIR0HZXNjN1/mYqF2NT6eZvmi22jZmZRI4chczJwXfuHPQOJdvfISYthgEhA7h6+ypzO8zlmaqq3bfyT0II3m36Lm8//jabLm/i/T/fJ9tgmg/n+7GpXZtKX3xOxokTxEydapIxbK30PFPPk5BT0eQaSvbOyJQKkxh0+bOSAMjfflvNRbyHjaE30AkIaFD8ZSQpJTGTJpMZHk6laV9hXa1kdyi9dvsagVsCuZVxiwWdFtDGp02Jjq+ULgMbDeTDFh/yx/U/GLFzBOk5JbenEYBT1664vfkGSavXkLRuvUnG6NbIm8T0HA5ejjfJ9bVQmMQwAzgohPhMCPEZcICHT1ctd6SUbAqNpmX1Cng4Fv8eRYmrV5P866+4v/UWjh06FPv1H+RC4gUCQwJJz01nSZclPO5pmod6StnSt15fPm/zOX/F/MWQ7UO4nX374ScVo4rvvotdq5bETJpERtipYr9++zoe2Fvp2ViG9k4qzHTVFeT1YriZ/9NDSqlae/7LmegULt9Ko7sJykjpx49zc8pUHJ56CvcRw4v9+g8SfiucoK1BCATLA5ZTv0KZ2wRXMaEXar7A1+2+5lT8Kd7c+iYJmSX3sFZYWOAzcyYW7u5EjhpFbnzxfrO3sdTTqb4nIeEx5JSRclKh5lflt/ecm/9z2lRBlWabwm6g1wm6NPAs1uvm3Iwl8u23sfSpRKWvpyF0JTc17mjMUd7c9iYOlg4Edw2mhkuNEhtbKTs6+3Vm9tOzuZJ8hQEhA7iZdrPExrZwdcVnzmwMCQlEvfseMje3WK/fzb8SyRk57L+obYe74lJ2Jt6aASklG0OjaV2jAhUciq+MJLOziXrnHYxp6fjOmYPeyanYrv0w+6L2MXTHUCraVSQ4IJjKjiW/gE4pO9r6tuX7jt8Tmx5LYEgg11OuP/ykYmLboAFekz4l/fBhYqfPKNZrt63ljqO1RZnp7KYSQzEKv3Gbq/Hpxb43UszUqWScOEGlLz7HpnbtYr32g2yL2MbIXSOp7lyd5QHL8bQv3rsgpXxq5tWMJZ2XkJqTSuCWQC4lXSqxsV1efBHXfv1IWL6c5N+LrxHlnXLS1vAYsnNLfzlJJYZitDE0GgudoEsxzkZKWreepNVrcHvzDZy6di226z7Mbxd/Y/Se0TRyb8TiLotxs3ErsbGVsq+BewOWdVmGRDIgZADh8eElNrbnuLHYNm1K9MSJZJ49W2zX7d7Ym9uZuey7WHp2f74flRiKSd6ithu0qemOq71VsVwzIyyMmEmTsGvVkorvvlss1yyIVWdWMWH/BFp4teD7jt/jZFVypSul/KjlWovggGDsLOwYuHUgx28eL5FxhaUlvt9+g97JicgRIzEkJT38pAJ4sqYHjjYWZWKxm0oMxSQ0MpnrCRl0K6ZObbnx8USOHIWFuzs+M2ciLEpmycjisMVMPTyVpys/zdxn5mJnWTLtSJXyqYpTFYK7BuNu686Q7UM4EHWgRMa18PDAZ9a35Ny8SdQHo5EGwyNf08pCR5cGXmwPv0lW7qNfT0sqMRSTTWHRWOoFXeo/ehlJ5uYS9e57GBIT8ZkzGwtX14ef9KhjSsk3x75h1vFZdKvejRntZ2Ct165XtFJ+eNl7sTxgOVWdqjJi1wh2Xt1ZIuPaNWmC1/jxpO3bR9zsOcVyzW7+3qRk5bLnfOmenaQSQzG4s6itbS0PnO0evVtZ7NfTST98GK9Jn2LbwLS7swIYpZEv/vqCpaeW0qt2L6Y8OQVLneq6ppScCrYVWNJlCfUr1Of93e/z+6XfS2Rcl1d74fxyT+IXLOD29u2PfL0na7rjbGtZ6ju7qcRQDE5eTyIqKaNYZiMl/76RhOBgXF97DZcX79nMrljlGnOZsG8CP537iaAGQUxoOUE12FE04WztzMJOC2nm2YyP9n3EmrNrTD6mEAKviROx8fcneuw4si492gwpS72OgAZebD99k8yc0ltOUp8AxWBjaDRWeh0d6z/adM7Ms2eJnjgR22ZN8Rw7ppiiu79sQzYf7P6A3y//zsgmIzXZO19R7mZnace8jvNo79ueL/76giVhS0w+ps7aGt/ZsxC2tnkPo1NTH+l63fy9Scs2sPt86Z2dpBLDIzIaJZvDomlXO+8WsqgMSUlEjhiJ3skJ32++QViatpSTnpPOyF0j2XltJ+NajGOw/2CVFBSzYK23ZubTM+larSvfHv+W2cdnm7wbnKWXFz7fzCT72jVujB2HNBZ9LUKrGhVwtbMs1bOTVGJ4RCeuJxKdnPlIeyNJg4GoD0aTc/MmvrNnYeHhUYwR/q+U7BSG7hjKoehDTG49mX71+pl0PEUpLEudJVOfnErPWj1ZFLaIqYenmrzhj32LFniOHUPqzp3EL1hQ5OtY6nUENPRm55mbZGSXznKSSgyPaGNoNFYWOp6pV7HI14ibPYe0ffvwmjAB28ceK8bo/ldCZgJvbn2TsLgwprWbxku1XjLpeIpSVHqdnk9afUJg/UBWn13NxP0TyTUW7x5H/+b6+us4PfcccbPnkLpnT5Gv093fm/RsA3+eM133OFNSieER3Ckjta/tgaNN0Uo/t7dvJ37BAlxeeRnXV3sVc4T/FJseS1BIEJeTLzOrwyy6+HUx6XiK8qiEELzf7H3eeuwtNlzawJg9Y8gxmK4bnBAC78mTsK5bl6gPRpN99WqRrvNENTcq2FuxMax0lpNMnhiEEAFCiHNCiItCiHH3+P17QojTQohQIcROIURVU8dUXI5eTeTm7awiL2rLunSJ6LHjsPH3x3PixGKO7p8iUyIJ3BJITFoM8zvOp51vO5OOpyjFRQjBsMbDGN1sNNuvbmfkHyPJyM0w2Xg6W1t858xGCEHkiJEY09IKfQ0LvY6Ahl7sOhNLerZp73JMwaSJQQihB+YBXYH6QB8hxL838j8BNJNS+gO/UIqa/2wKvYG1hY6O9Qo/G8mQkkLk8BEIW1t8Z89CZ1U822jcy+WkywRuCeR29m0Wd15Mc6/mJhtLUUylf4P+fNrqUw5EHWDo9qGkZj/a7KEHsfL1pdKMGWRdusSNCROK9PC7u38lMnIM7Dpb+spJpr5jaAFclFJellJmA2uAF+4+QEr5h5TyTr+/Q4CviWMqFgajZPOpGDrUrYi9deG2q5BGIzfGjiP7+nV8vpmJpVfxtwC943T8aQaEDMAgDSwLWEYjj0YmG0tRTK1n7Z5MazeN0LhQBm4bSFJm8exzdC8OT7bB4513SNkSQsLSZYU+v0U1N9wdrEvlVtymTgw+wN0brkfmv3c/bwJb7vULIcRgIcRRIcTRuDjt5wcfvpJAXErRykjxixaTumsXnmPHYN+ihQmiy3Mi9gRvbn0TGwsbgrsGU9u15LbsVhRTCagWwLdPf8uFxAsEbQ0iLt10nwcVBg3EsXNnYmfMIO2vw4U6V68TPNvIi11nY0nLKl3lJLN5+CyEeA1oBnx9r99LKRdKKZtJKZt5mHg6Z0FsCruBraWeDnULNxsp++pVbs2di2NAAK6vv26i6ODAjQMM2T6ECrYVCA4IpqpTqXl0oygP9VTlp5jfcT5RqVEEhgQSlRplknGEEHhPmYJlZV9iPvkEmZ1dqPO7NfImK9fIjjMl162uOJg6MUQBd7f88s1/7x+EEB2B8cDzUsosE8f0yHINRkJOxdChXkXsrApXRro5ZSrC0hLPjz402YKyndd2MmLnCCo7VmZ5wHK8HYq3cZCimIMW3i1Y1HkRSVlJBG4J5EryFZOMo3ewx2v8eLIjIogPDi7Uuc393KjoWPrKSaZODEeAWkKIakIIK6A3sOHuA4QQTYAF5CWFUvGU5vCVBG6lZtO9kHsjpfzxB6m7d+M+YgSWFYu+7uFBfr/0O+//+T713OqxtMtS3G3dTTKOopiDxh6NWdZlGTnGHAaEDOBsQvE13rmbQ7t2OHTowK3535MTE1Pg83Q6wbONvPnzfBwpmaabZlvcTJoYpJS5wAhgK3AGWCulDBdCTBZCPJ9/2NeAA/CzEOKkEGLDfS5nNn4PjcbOSk/7OgX/cDdmZXFzylSsatTA7fXXTBLX2nNrGb9vPE09m7Kw80KcrZ1NMo6imJM6bnUIDgjGSm/FGyFvcDL2pEnG8fzoQ8jNJXbaPavd99Xd35vsXCM7z5SK771ACTxjkFJullLWllLWkFJ+kf/ex1LKDfl/7iil9JRSPpb/8/yDr6itvDJSNM/U88TWSl/g8+KXLCHn+nW8Jow3yT5Iy04t47NDn9HOtx3fdfwOe0v7Yh9DUcyVn7MfwQHBuNq4Mnj7YA5FHyr2Max8fakwaBC3N28u1IPox6u44uVkU6r2TjKbh8+lxcHL8SSm59C9ELORsiOjiF+wEMeAAOxbtSrWeKSUzDkxh5nHZhLgF8A3T3+jGuwo5VIlh0oEdw3Gx8GHt3a8xR/X/ij2MSoMGoiljw83P/8MmVOw0pBOJ+jm782e83EkZ5SOcpJKDIW0KTQaeys9T9Uu+Myo2K++Ap2u2LfSNkojXx35ioWhC+lZqydftv1SNdhRyjV3W3eWByynjmsd3v3zXTZf3lys19fZ2OD50YdkXbhI4qpVBT6vm7832QYjO06XjtlJKjEUQo7BSEh4DJ3qe2JjWbAyUuq+/aRs34770KFYehff7CCD0cAnBz5h5ZmVvF7/dT5p9Ql6XcFLW4pSVjlbO7O4y2KaVGzCuL3j+Pn8z8V6fYcOHbBv25a4OXPJLeCaqiaVXfBxsWVTKdk7SSWGQth/8RZJ6Tl0K+AW2zI7m5uff45V1aq4BQ0otjhyDDmM2TOG/1z8D281fovRzUarXgqKchd7S3vmd5zPkz5PMvngZILDCzfN9EGEEHh+9CHGrCxip88o8DnPNvJi74U4ktPNv5ykEkMhbAqNxtHagna1CzYFNGHFCrIjIvCcML7Y9kLKzM1k1B+j2HZ1Gx80+4Bhjw1TSUFR7sHGwoZZT8+ic9XOTD86nXkn5xVbwx/ratWoEBRE8m+/kX78eIHO6e5fiRyDZOvpgk931YpKDAWUnWtka3gMnRp4Ym3x8JJNTkwMcd/Nx+GZZ3Bo27ZYYkjNTmXYjmHsj9rPx60+JrBBYLFcV1HKKku9JdPaTePFmi/y/d/f8/XRr4stObgPHYKFlxcxn32ONDy8IY+/rzO+rralYrGbSgwFtO9iHLczcws8Gyl22teQm4vnh/+z03iRJGclM2jbIE7GnuTLtl/ySu1XiuW6ilLW6XV6JrWeRL96/fjh9A9MOjgJg/HRO6vp7OzwHDuGrDNnSPzpp4ceL0Te7KT9F2+RmFa4rTVKmkoMBbQxNBonGwuerPnw2Uhpfx3m9ubNVBg0CCvfR98s9lbGLQaEDOB84nm+efobnq3+7CNfU1HKE53QMbb5WIb4D2HdhXWM2zuuWBr+OAYEYNeyJXHfziI3IeGhx3dvVIlco2SbmZeTVGIogKxcA9vDb9KlgRdWFg/+K5M5Odz8/DMsfXyoMGjgI499I/UGgVvyNgn7ruN3tK/c/pGvqSjlkRCCEU1G8F7T9wiJCOGdP98hMzfzka/pNWE8xvR04r759qHHN/RxomoFO7Nf7KYSQwHsOX+LlKzcAm2xnbhqFVkXLuL50YfobGweadyI5AgCQwJJzEpkUedFPOH9xCNdT1EUCGoYxMSWE9kbuZfhO4eTllP4Dm13s65ZE7fXXiPpl1/ICAt74LFCCLo18ubApXjiU813v1CVGApgU+gNnG0taVPzwbORcuPiiJszF/u2bXHo0OGRxjyXcI7AkECyDdks7bKUxh6NH+l6iqL8v151ejGl7RSO3TzG4G2DSc5KfqTruY8Yjt69AjGTP0MajQ88tpu/NwajZGu4+S52U4nhITJzDGw/fZOABl5Y6h/81xU7fQYyKwuv8R890hTSv+P+JmhrEJY6S5YHLKeuW90iX0tRlHvrXr07M9vP5EzCGd7Y+ga3Mm4V+Vp6Bwc8R48mMyyM5PXrH3hsfW8nqrnbsynsRpHHMzWVGB5i9/k40rINdG/84DJS+vHjJP/2G25BQVj5+RV5vMPRhxm0bRAu1i6s6LqCas7VinwtRVEerEOVDsx7Zh7XU64zIGQA0alFr/07Pfcctk2bEjtjJoak+7ccFULQ3d+bg5fiiUsxz3KSSgwPsTE0Glc7S1pVr3DfY6TBQMxnn2Ph5YX70CFFHmv39d0M2zEMHwcfggOCqeRQsBXWiqIUXatKrVjYaSEJGQn0D+nP1dtXi3QdIQReEydgSGs3VZsAABVGSURBVE4mbvacBx7bzd8bo4SQcPOcnaQSwwNkZBvYeeYmAQ29sXhAGSnxp5/IOnMGz3Fj0dnZFWmskCshvPPHO9RyrcWyLsvwsNO+familBePVXyMJV2WkJWbReCWQM4nni/SdWzq1sW1Tx8S16wh88yZ+x5Xx9ORGh72bAo1z3KSSgwP8Oe5WNKzDQ9c1JabkEDct7Owa9kSxy5dijTOuvPrGLNnDI0rNmZx58W42LgUNWRFUYqoXoV6LA9Yjl6nJygkiLC4B88wuh+PUSPROzvnrYi+zyrrvMVulfjrSgKxKY82ZdYUVGJ4gI1h0bg7WPFENbf7HhP3zbcY09PzGvAU4YHzD6d/4NODn9LapzXzO87HwcrhUUJWFOURVHepTnBAME5WTgzcNpAjMUcKfQ29szMV33+PjOPHub3h/g0pu/t7IyVsCTO/cpJKDPeRnp3LrjOxBDT0um8ZKSMsjKRffsHt9dexrlmzUNeXUjL/7/lMOzKNTlU7MefpOdha2BZH6IqiPAJfR1+Cuwbjbe/NsB3D2BO5p9DXcO7RAxt/f25+PR1Dauo9j6nt6UhtTwez3DtJJYb72HU2lowcA90a3fsBsDQaiZn8GXr3CrgPf6tQ15ZSMuPoDL47+R0v1HiBae2mYalXDXYUxVxUtKvIsoBlVHeuztu73iYkIqRQ5wudLu9BdHw8t+bOu+9x3RpV4sjVBGKSzaucZPLEIIQIEEKcE0JcFEL8z45yQoh2QojjQohcIcTLpo6noDaFRuPhaE2L+5SRktevJzMsDM/Ro9E7FLz8YzAamHxoMsGng+lbty+T20zGQmdRXGErilJMXG1cWdJlCf4e/ozdM5ZfL/xaqPNtGzXC5eWXSfjhB7IuXLjnMd3ulJNOmdddg0kTgxBCD8wDugL1gT5CiPr/OuwaMAAoeJ88E0vLymXX2ViebeiFXve/zw0MycnEzpiJbdOmOD33XIGvm2PM4cN9H/5fe3caXWV1NXD8vzMyJBAgIYmQAAoqQ7AgU5GKikAEKrSvotRWQESxAgpUSwWsioAUtQuFGuagvr5Y/dBGGVIGCwUEDbQFYgQiMyEhCRhCAhn3++FekEAgV8i9N8P+rZW1nuE89+6Tm2TnnPM85/Dpvk8ZHTOayd0m4yPWaDOmqgoOCCaubxw9Invw8taX+fCbD3/U9WETJ+ATFET66zPKHYhu3TSI2yOCq9zcSe7+q9QNSFXVA6paCKwABl9aQFUPqeou4NrPkXvQupQMCopLr7pSW+bcdyjJySFi2lSXB5wLSgqY+MVEVh9czYQ7JzC+83hbYMeYaqCuX13eve9d7o++n9lfz2bBfxe4vKaDX6NGNH3+OfK3byd3TfndUYM6RrLj8GnSvj9XmWHfEHcnhmbA0Uv2jzmPVWkrd52gaXAgXVo0uuLc+ZQUTq9YQaNhw6hzu2tTVeQX5fPsumfZeGwjU7tP5YkOT1R2yMYYNwrwDWBO7zk8eMuDzPvPPN7e8bbLySFk6FAC27Yl443ZlOZdOWHfgBjH7fCrqtB60NWmH0NEnhKRJBFJynRxAe7rkXu+iH/uy2RATCQ+l3UjqSrp01/HNySEsPHjXHq9nIIcRq8dTVJGEjN6zeCR2x9xR9jGGDfz8/Fj+l3TeeS2R4hPjmf6tumUasUdHeLrS8S0aRRnZJAVt+CK8zeHBdEusgEra1FiOA5EXbLf3HnsR1PVharaRVW7hIW576ng9SknKSwu5eflzI10JiGBczt30nTSRHwbNqzwtbLPZTMqcRQp2Sm81fstfn6L6+MRxpiqx0d8mNJ9CqM6jOKTfZ/w0uaXKCqteMGfep070XDIELLj4yk4ePCK84PuiOTfR77n2Ol8d4T9o7k7MXwNtBGRViISADwKXP2Jjyrg811pRDasQ6eost1IJWfPkjHnTep07EjDX/yiwtdJz0tnxJoRHD5zmHn3zaNPiz7uCtkY40EiwvN3Ps9znZ9j5YGVTPrnJApKKp4Mr+mkifgEBpIxY+YV3VADq1h3klsTg6oWA2OBRCAF+KuqJovIayLyIICIdBWRY8DDwAIRSXZnTNeSc66ITfuyyu1Gypo3n5LsbCKmTUN8rv1tO3LmCMNXDyfrXBYL+i6gZ7Oe7gzbGOMFT8Y8yUvdX+KLo18wdv1Y8ouu/d++X1gYYePGkrd5M2fXry9zrkWT+sQ0a1hlHnZz+xiDqq5S1VtV9RZVneE89rKqJji3v1bV5qpaX1WbqGp7d8d0Neu+yaCwpPSKldoK9u/n1AcfEPLww9SN6XDN19h/ej/D1wwnvzifJf2X0Dm8sztDNsZ40bDbh/H6Xa/zVfpXPL32ac4Unrlm+UaPPUZgmzZkzJxF6fmyD7UN7BjJf4/lcPSU97uTqs3gsyes3H2CZiF16RT1wyR2qkr66zPwCQoibMLz17w+OSuZkYkj8cGH+Nh42jW5/JENY0xNM7j1YN7s/SZ7svcwKnEUp86fumpZ8fMjfOpUitLSyF64qMy5C91JVeGZBksMTjn5RfxrfyYDO0aWeb4gd80a8rdvp+nzz+HX6MrbVy9ISk9i1D9GEeQfRPwD8dwScosnwjbGVAF9W/Tl3fve5VDOIUasGUF63tUnxqvfvRsNBgwge/FiCo/+cDd/VON63BEVUiVWdrPE4JT4TTpFJXoxawOU5uWR8cZsAtu1JWTo0Kteu/n4ZsasG0N4vXCWxy4nKjjqqmWNMTVTr2a9iOsbx8n8k4xYM4KjZ45etWzT378Ifn5kzHqjzPFBMZHsOX6GQ1lXPu/gSZYYnFbuOkFU47p0bP7DbahZcQsozsggYuo0xNe33Ov+cegfjNswjpsb3syy2GWE1w/3VMjGmCrmzvA7WdJvCXlFeQxfM5zU06nllvMPDyfst89wdsMGzm7cePH4AOf4prefabDEAJzOK2RLahYDY2662I1UcPAg2fHxNBwyhHqdO5V73d9T/84Lm14gJjSGJf2X0LjO1ddtMMbUDu1D27Os/zIUZWTiSJKzy7/RsvHjjxPQqhXpM2dSWlgIQLOQunSODvH6OIMlBiAxOZ3iUr24UpuqkjFjJj6BgTT93aRyr/ko5SOmbplK94juxN0fR3BAsCdDNsZUYa0btWZ57HLq+dXjycQn2ZGx44oyEhBA+JQpFB0+wqmlyy4eH9jxJlJOnOG7zPLXcfAESww4mm0tmtSj/U0NADi7fj15mzcTNn4cfqGhV5RfvHsxs76axX1R9zGvzzzq+V/fOs/GmJorukE0yx9YTmjdUMasHcOW41uuKBPU6y6C+/YlKy6OojTHoPOAmAgAVnmx1VDrE0P22QK2fpfNwBjH3Uil58+TMXMWgW3a0OhXvypTVlX5844/M3fnXAbdPIi37nmLAN8AL0VujKnqIupHEB8bT8uGLRm7YSzrDq+7okz45N8DkDH7TwBENqxLlxaNvDrOUOsTQ2JyBiWlyiDnFNvZCxdRlJZG+LSpiN8PC+iUaikzts9g6Z6lDL11KDN6zbAFdowxFWpStwlL+i+hfZP2TNo4iYTvys4K5N+sGaFPP0VuYiJ5W7cCjqm4v03PJfVkrjdCtsTw+a40bg6tT9vIYAqPHiV78WIaDBxI/W7dLpYpLi1m6uapfLz3Y0Z2GMnUHlNtgR1jjMsaBDRgYd+FdI3oypTNU1jx7Yoy5xs/8QT+UVGOBX0KC3kgJhIR7z3sVqv/umXmFrDtQPbFh9oyZr0Bfn40ffGFi2UKSwr53cbf8dmBzxjfaTwTOk+wBXaMMT9aPf96zO8zn3ui7mHG9hks3r344jmfwEDCX/oDhQcOcOqDDwlvUIeuLRt7be6kWp0Y1iSnU6qOOUrObtzI2Q0bCPvtM/iHO55FyC/KZ9yGcaw/sp7J3SYzuuNoSwrGmOsW6BvI2/e8zYBWA5i7cy5zd869ONNq8L33EtS7N1nz51OUcZJBHSPZf/Is+zI8351UqxPDyl1ptG4aRJuQANJnzCSgVSsaP/44ALmFuYxZN4ZtJ7Yx/a7pPNb2MS9Ha4ypCfx9/JnZayYP3foQi3cvZub2mRcX/Amf8hJaVMTJN98ktkMEPl7qTqq1ieHkmfNsP3iKgTGRnI5fTtGRI4RPmYIEBHD6/GlGJY5id9Zu5tw9hyGth3g7XGNMDeLr48vLPV5meLvhrNi7gmlbplFcWkxAdDSNnxzFmc8+I+jb3XRv1YTPd6W5vIxoZam1iWH1nnRUoX+EL1lxcQT37UtQr7s4mX+SkWtGciDnAO/c+w79WvbzdqjGmBpIRJjUZRLP/uRZEr5L4MVNL1JYUkjoU0/hd1Mk6dNfZ2CHcA5k5vFtume7k2ptYli56wS3hgfRYPG7gONe4mO5xxi+ejjp+em8d/97/Kz5z7wcpTGmJhMRxtwxhhe7vsjaw2sZv2E8Bf4QPnkyBfv20WPvVnwEjw9C18rEkJ5znq8Pn6JfSDG5iYmEPv0UR+sXMHz1cHKLclncbzFdI7p6O0xjTC3xm3a/4dWer7I1bStj1o6B3j2o37MnJX+Zy0+jG7By9wmPdifVysSwavcJVOHOhKX4R0eTMaQnI9aMoERLWNZ/GR1Cr71KmzHGVLZftvklf7r7T+zK3MXotaMJ/P04Ss+fp9eRnRzMyiM57dqrw1WmWpkYVu4+QZuAIiJSdpIx8RGe/GIMdfzqsPyB5bRp1Mbb4RljaqnYVrHMvW8uqadTeTrlFRj5EJ1Wvo+veHYq7lqXGNK+P8eOw6fpmbyRfYPvYEJ2HKF1Q3n/gfdp0aCFt8MzxtRydze/m/fuf48TeSeYFP0lJc3r0unscVZ68O4ktycGEYkVkb0ikioik8s5HygiHzvPbxeRlu6MZ5Uz64b4f8Wr7fcSHRzNsthlRNSPcOfbGmOMy7pFdmNRv0XkFJ1h2jClw4l1HDl1jj3HPdOd5NbEICK+wHzgAaAdMExE2l1WbBRwWlVbA38GZrszpoQv99PMbz3xsbm0DW3Hkv5LCK175dTaxhjjTR3DOrK0/1JKA/xZ238/fgFHSfjqgEfe290thm5AqqoeUNVCYAUw+LIyg4Hlzu1PgT7ipnknjmbmklK6mTOt19Il/E4W9V1Ew8CGFV9ojDFecFvj24iPjaduUAj1Wyzkb7s3eaQ7yd2JoRlw6YrYx5zHyi2jqsVADtDEHcG88skfqRP5Nzr7t+Uv/eJsgR1jTJXXsmFLPvj5RwRrPc5Fv8/7iR+5/T2rzeCziDwlIkkikpSZmXldrxFVN5Kw729n0aMfEugbWMkRGmOMe0QGRRIfG0/Q2QhCfBq5/f3Enc0SEfkp8Iqq9nfu/wFAVWddUibRWeZLEfED0oEwvUZgXbp00aSkJLfFbYwxNZGI7FDVLhWVc3eL4WugjYi0EpEA4FEg4bIyCcBw5/ZDwIZrJQVjjDHu5da1KVW1WETGAomAL7BUVZNF5DUgSVUTgCXAByKSCpzCkTyMMcZ4idsXLVbVVcCqy469fMn2eeBhd8dhjDHGNdVm8NkYY4xnWGIwxhhThiUGY4wxZVhiMMYYU4YlBmOMMWW49QE3dxGRTODwdV4eCmRVYjjVgdW5drA61w43UucWqhpWUaFqmRhuhIgkufLkX01ida4drM61gyfqbF1JxhhjyrDEYIwxpozamBgWejsAL7A61w5W59rB7XWudWMMxhhjrq02thiMMcZcQ41NDCISKyJ7RSRVRCaXcz5QRD52nt8uIi09H2XlcqHOE0XkGxHZJSLrRaSFN+KsTBXV+ZJy/yMiKiLV/g4WV+osIkOdn3WyiLh/yS83cuHnOlpEvhCRfzt/tgd4I87KJCJLReSkiOy5ynkRkXec35NdItK5UgNQ1Rr3hWOK7++Am4EA4L9Au8vK/BaIc24/Cnzs7bg9UOd7gXrO7WdqQ52d5YKBTcA2oIu34/bA59wG+DfQyLnf1Ntxu7m+C4FnnNvtgEPejrsS6n030BnYc5XzA4DVgAA9gO2V+f41tcXQDUhV1QOqWgisAAZfVmYwsNy5/SnQR0TEgzFWtgrrrKpfqGq+c3cb0NzDMVY2Vz5ngOnAbOC8J4NzE1fqPBqYr6qnAVT1pIdjrEyu1FeBBs7thkCaB+NzC1XdhGN9mqsZDLyvDtuAEBGJrKz3r6mJoRlw9JL9Y85j5ZZR1WIgB2jikejcw5U6X2oUjv84qrMK6+xsYkep6kpPBuZGrnzOtwK3isgWEdkmIrEei67yuVLfV4Bfi8gxHGu/jPNMaF71Y3/ffxS3L9Rjqh4R+TXQBejt7VjcSUR8gLeBEV4OxdP8cHQn3YOjVbhJRGJU9XuvRuU+w4B4VX3Luc78ByLSQVVLvR1YdVVTWwzHgahL9ps7j5VbRkT8cDRBsz0SnXu4UmdE5H5gCvCgqhZ4KDZ3qajOwUAH4J8icghHX2xCNR+AduVzPgYkqGqRqh4E9uFIFNWRK/UdBfwVQFW/BOrgmE+oJnPp9/161dTE8DXQRkRaiUgAjsHlhMvKJADDndsPARvUOapTTVVYZxHpBCzAkRSqc7/zBdess6rmqGqoqrZU1ZY4xlUeVNUk74RbKVz52f4bjtYCIhKKo2vpgCeDrESu1PcI0AdARNriSAyZHo3S8xKAx513J/UAclT1RGW9eI3sSlLVYhEZCyTiuKthqaomi8hrQJKqJgBLcDQ5U3EM8jzqvYhvnIt1ngMEAZ84x9mPqOqDXgv6BrlY5xrFxTonAv1E5BugBHhBVatla9jF+k4CFonIBBwD0SOq+T95iMj/4Ujuoc6xkz8C/gCqGodjLGUAkArkAyMr9f2r+ffPGGNMJaupXUnGGGOukyUGY4wxZVhiMMYYU4YlBmOMMWVYYjDGGFOGJQZjPEBE1ojI9yLyubdjMaYilhiM8Yw5wG+8HYQxrrDEYIwLRKSliHwrIv8rIiki8qmI1BORQyIyS0T+IyJJItJZRBJF5DsRGXPhelVdD+R6sQrGuMwSgzGuuw34i6q2Bc7gWNMDHE+Q/wT4FxCPY4qVHsCr3gjSmBtlicEY1x1V1S3O7Q+BXs7tC1Nv7MaxYEquqmYCBSIS4ukgjblRlhiMcd3l88dc2L8wS23pJdsX9mvkfGSmZrPEYIzrop3z/QP8CtjszWCMcRdLDMa4bi/wrIikAI2A91y9UET+BXyCYwnZYyLS300xGnPDbHZVY1wgIi2Bz1W1g5dDMcbtrMVgjDGmDGsxGGOMKcNaDMYYY8qwxGCMMaYMSwzGGGPKsMRgjDGmDEsMxhhjyrDEYIwxpoz/Bz6SJ/mrKEZPAAAAAElFTkSuQmCC\n", "text/plain": [ "<Figure size 432x288 with 1 Axes>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import seaborn as sns\n", "\n", "# binary classification: pm1 is the complement of pm2\n", "pm1 = np.linspace(0, 1, 10)\n", "pm2 = np.repeat(1, 10) - pm1\n", "pmk = pd.DataFrame({'pm1':pm1,'pm2':pm2})\n", "\n", "# binary classification: k = 2\n", "\n", "# Gini\n", "g1 = pmk.pm1 * (1 - pmk.pm1)\n", "g2 = pmk.pm2 * (1 - pmk.pm2)\n", "g = g1 + g2\n", "\n", "# Entropy\n", "e1 = pmk.pm1 * np.log(pmk.pm1.replace([0], 1))\n", "e2 = pmk.pm2 * np.log(pmk.pm2.replace([0], 1))\n", "e = -(e1 + e2)\n", "\n", "\n", "# Classification error\n", "# it is the fraction not belonging to most common class\n", "ce = pmk.min(axis=1)\n", "\n", "\n", "\n", "# plot\n", "line_df = pd.DataFrame({'gini' : g,\n", " 'pm1': pmk.pm1,\n", " 'class_err': ce,\n", " 'entropy' : e})\n", "sns.lineplot(x='pm1', y='gini', data=line_df, color='tab:red')\n", "sns.lineplot(x='pm1', y='entropy', data=line_df, color='tab:blue')\n", "sns.lineplot(x='pm1', y='class_err', data=line_df, color='tab:green')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "4. This question relates to the plots in Figure 8.12.\n", "\n", "(a) Sketch the tree corresponding to the partition of the predictor space illustrated in the left-hand panel of Figure 8.12. The num- bers inside the boxes indicate the mean of Y within each region.\n", "\n", "(b) Create a diagram similar to the left-hand panel of Figure 8.12, using the tree illustrated in the right-hand panel of the same figure. You should divide up the predictor space into the correct regions, and indicate the mean for each region." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "5. Suppose we produce ten bootstrapped samples from a data set containing red and green classes. We then apply a classification tree to each bootstrapped sample and, for a specific value of X, produce 10 estimates of:\n", "\n", "`P(Class is Red|X): 0.1, 0.15, 0.2, 0.2, 0.55, 0.6, 0.6, 0.65, 0.7, and 0.75.`\n", "\n", "3There are two common ways to combine these results together into a single class prediction. One is the majority vote approach discussed in this chapter. The second approach is to classify based on the average probability. In this example, what is the final classification under each of these two approaches?" ] }, { "cell_type": "code", "execution_count": 81, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "majority vote: Red\n", "mean probability: Green\n" ] } ], "source": [ "# marjority vote\n", "ps = np.array([0.1, 0.15, 0.2, 0.2, 0.55, 0.6, 0.6, 0.65, 0.7, 0.75])\n", "res = 'Red' if (((ps > 0.5).sum() / len(ps)) > 0.5) else 'Green'\n", "print('majority vote: {}'.format(res))\n", "\n", "# avg probability\n", "res = 'Red' if (ps.mean() > 0.5) else 'Green'\n", "print('mean probability: {}'.format(res))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "6. Provide a detailed explanation of the algorithm that is used to fit a regression tree." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- (1) Recursive binary splitting: select split such that RSS is minimized, repeat until some stopping condition - for example minimum number of observations in each terminal node. How do we try all such split for a continuous variable? The answer is here: https://stats.stackexchange.com/questions/220350/regression-trees-how-are-splits-decided. We try splits at each value of the predictor for every observation.\n", "\n", "If we decide on some small number for the minimum number of observations in the terminal nodes, (1) will yield a tree with low bias and high variance. To obtain a lower vairance model one can use cost complexity pruning, to collapse nodes; increasing bias in the hope of a bigger dividend in reduced variance with the aim of improving overall predictive performance. In practice, this technique is not commonplace in ML, the preferred means of reducing variance is to use an ensemble method: bagging and random forests or to use the additive boosted tree model. These techniques are not as interpretable as the simple tree." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.1" } }, "nbformat": 4, "nbformat_minor": 2 }