{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sebastian Raschka 07/10/2015 \n", "\n", "CPython 3.4.3\n", "IPython 3.2.0\n" ] } ], "source": [ "%load_ext watermark\n", "%watermark -a 'Sebastian Raschka' -d -v" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Cheatsheet for Decision Tree Classification " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Algorithm\n", "\n", "1. Start at the root node as parent node\n", "2. Split the parent node at the feature ***a*** to minimize the sum of the child node impurities (maximize information gain)\n", "3. Assign training samples to new child nodes\n", "3. Stop if leave nodes are pure or early stopping criteria is satisfied, else repeat steps 1 and 2 for each new child node" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Stopping Rules\n", "\n", "- a maximal node depth is reached\n", "- splitting a note does not lead to an information gain" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Criterion\n", "\n", "Splitting criterion: Information Gain (IG), sum of node impurities\n", "\n", "Objective function: Maximize IG at each split, eqiv. minimize the the impurity criterion\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Information Gain (IG)\n", "\n", "***Examples below are given for binary splits.***\n", "\n", "$$IG(D_{p}, a) = I(D_{p}) - \\frac{N_{left}}{N_p} I(D_{left}) - \\frac{N_{right}}{N_p} I(D_{right})$$\n", "\n", "- $IG$: Information Gain\n", "- $a$: feature to perform the split\n", "- $N_p$: number of samples in the parent node\n", "- $N_{left}$: number of samples in the left child node\n", "- $N_{right}$: number of samples in the right child node\n", "- $I$: impurity\n", "- $D_{p}$: training subset of the parent node\n", "- $D_{left}$: training subset of the left child node\n", "- $D_{right}$: training subset of the right child node" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Impurity (I) Indices\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Entropy\n", "\n", "The entropy is defined as\n", "$$I_H(t) = - \\sum_{i =1}^{C} p(i \\mid t) \\;log_2 \\,p(i \\mid t)$$\n", "\n", "for all non-empty classes ($p(i \\mid t) \\neq 0$), where $p(i \\mid t)$ is the proportion (or frequency or probability) of the samples that belong to class $i$ for a particular node $t$; $C$ is the number of unique class labels.\n", "\n", "The entropy is therefore 0 if all samples at a node belong to the same class, and the entropy is maximal if we have an uniform class distribution. For example, in a binary class setting, the entropy is 0 if $p(i =1 \\mid t) =1$ or $p(i =0 \\mid t) =1$. And if the classes are distributed uniformly with $p(i =1 \\mid t) = 0.5$ and $p(i =0 \\mid t) =0.5$ the entropy is 1 (maximal), which we can visualize by plotting the entropy for binary class setting below.\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8lnP+x/HXp0XETKTfYCxlFGUpSSnrQRJ+lmxlX2cM\nsowx0sxw0jzkZ8ZgFKlow7RZU8jWoUikRaHUEIWxhIRoOZ/fH9+7Ouc4nXOf033d13Xf9/v5eNyP\n7uU69/VxOef+3N/t8zV3R0REZJ06cQcgIiLJosQgIiLlKDGIiEg5SgwiIlKOEoOIiJSjxCAiIuXU\nizuAdJiZ5tSKiNSCu1tNfyZnWgzurps7xcXFsceQlJuuha6FrkXVt9rKmcQgIiLZocQgIiLlKDHk\nmKKiorhDSAxdiw10LTbQtdh0tin9UNliZp4LcYqIJImZ4fk8+CwiItmhxCAiIuUoMYiISDlKDCIi\nUo4Sg4iIlKPEICIi5SgxiIhIOUoMIiJSjhKDiIiUE2liMLOhZvaZmc2t4pi7zGyhmc0xs7ZRxiMi\nItWLusUwDOi6sRfN7Figubu3AH4HDIw4HhERqUakicHdpwBfV3HICcCI1LHTga3NbLsoYxIRkarF\nPcawI7CkzOOlwE4xxSIiIiRja8+Klf9URlVitXYtfPbZhtvnn8Py5bBiRbitXAlr1sDq1eHYevWg\nfv3w71ZbwS9+EW6NG8OvfgXbbQfbbw/bbANW4zqXIjHIwtZyzYC5G3ntXqBHmcfzge0qOc4ruxUX\nF3tliouLdbyOr/b4K68s9gkT3P/xD/eLLnIvKnLfdVf3OnUqP/6gg4r95pvdb7/dvX9/94ED3QcP\ndj/22MqPb9Wq2A891L1lS/dGjdy33NJ9zz3dmzfPjeuj43Pv+MmTJ3txcfH6G+Bei8/tyPdjMLNm\nwJPuvk8lrx0L9HT3Y82sI3Cnu3es5DiPOk7Jb0uXwquvwvTpMHMmzJoVvtXvtRe0bBluzZtDs2aw\n887QoEHmY1i+HD78EBYvhvfeg/nzw23uXGjUCPbbL9w6dYKOHUN8IpuitvsxRJoYzGwUcBjQBPgM\nKAbqA7j7oNQxAwgzl74HLnD3mZW8jxKD1MhHH8ELL4TblCnwww9w0EHhA7ddO2jbFpo0iTvKoLQU\nPvggJKsZM0ICmzkTWrSAww6Dzp3Dv0oUUlOJTAyZosQg1Vm9Gl55BZ58EiZMgGXL4MgjN3yotmiR\nW/37q1aF5DB5Mjz/fGjptG0Lxx8fbi1b5tZ/j8RDiUEKzqpV8NxzMHZsSAi/+U340Pzf/w0fonXi\nnnOXQT/8AC+9FP47n3wSNt8cTj0VTj8d9t1XSUIqp8QgBaG0FKZOhREj4LHHwhjB6afDySfDjjvG\nHV12uIdup3HjYMwYqFsXzjwTzj0Xdtst7ugkSZQYJK8tXQr33x8SQsOGcN55cMYZsFOBr3pxD+MS\nDzwAo0ZBq1Zw4YUhWTZsGHd0EjclBsk7paWhf33gwNCN0qNH+NBr105dJ5VZtQomToT77gtjEuec\nA7//PeyxR9yRSVyUGCRvrFwZvgHfcUeYNnrppaGrRLNy0vfBBzBkSGhltWsHf/wjHHGEEmqhUWKQ\nnPfVV3DXXXDPPWFa6TXXhBlF+jCrvR9/hIcegttvDyuzr7sOuncP9yX/KTFIzvr88/DBNWQIdOsG\nf/qTuj8yzR0mTYJ+/eDTT6F3bzj7bNhss7gjkyjVNjHk0YQ+yTXLlkGvXmFO/ooVYabNffcpKUTB\nDLp2hZdfDt1Lo0bB7rvDsGGh7pNIWUoMknUrVkDfviEBLF8Ob70Fd98Nu+wSd2SF4dBDw/qPBx+E\n4cNh773DWhA1ymUddSVJ1qxdC0OHwo03hlXJN92kefdxcw9JonfvUCH29tvhwAPjjkoyRWMMkmjP\nPx8Gkxs3hn/+M8yUkeQoLQ2D1H/5Sxj4//vfQ0FByW0aY5BE+uijULrhkktCC2HyZCWFJKpTJ6x7\nmD8f9tkH9t8f/va3MKtJCo8Sg0Ri1Sq45ZZQRrp1a3j77TDjSFNPk61hQ7jhBnjzTZg9O4w/PPVU\n3FFJtqkrSTLutdfg4ouhaVPo3z8Ut5PcNGkSXH45tG8Pd94ZdqOT3KGuJIndd9/BVVeFlsENN4Ty\n10oKue3oo8OssV12CS2/4cM1e6kQqMUgGfHSS3DBBWEq5O23h0FmyS+zZsFFF4X9q4cMKZxqtrlM\nLQaJxcqV8Ic/hFpGd90VvlEqKeSntm1Dcb4OHcL9hx5S6yFfqcUgtTZ7dih93aZNWKC27bZxRyTZ\n8uabofT5nnvCoEGwzTZxRySVUYtBsqa0NFQ+PeqoMJYwerSSQqFp1y7sA7H99mEHualT445IMkkt\nBqmRL74IO4V9803oStDgskyYAL/9bVircsMNYUc5SQa1GCRyr7wS1iW0bRuKsSkpCIQ9tmfODL8T\nRx8dquVKblNikGq5hzIWp5wS+pP79Qt1dUTW2WGHUHOpU6fw5WHKlLgjkk2hriSp0ooVcP75Yc/l\nsWPDojWRqjzzTPiduf76sK5Fq93jo64kybhFi8I3wMaNQzeBkoKko2vXsPp9xIiQIFaujDsiqSkl\nBqnUpElw0EHQsycMHhz2XhZJV7NmYUxq9Wo45BBYsiTuiKQmlBjkZ/r3D9/0HnkEfv97dQVI7TRs\nGGaude8eWp5vvBF3RJIujTHIemvWwNVXh9LYEybArrvGHZHki/HjQ2HFe+4JZdglO2o7xlAvimAk\n96xYAaefHhavvfoqNGoUd0SST044IXRPnnACLFwYBqbVEk0udSUJn34Khx0WKmhOnKikINFo2zYM\nSo8dG0p5r10bd0SyMUoMBW7BgrDH78knw733Qj21ISVCO+4YKvEuXBi6lDRjKZmUGArYa6+FlsKN\nN8Jf/6qmvWTHL38ZWqZbbgmdO8NXX8UdkVSkxFCgnnsOjj8ehg4N+yiIZNNmm8HIkWG20mGHhe5M\nSY5IE4OZdTWz+Wa20Mx6VfJ6EzN7xsxmm9k8Mzs/yngkeOwxOOus8O+xx8YdjRSqOnXgH/8IpdsP\nOQQWL447IlknsumqZlYXWAB0Bj4G3gDOcPd3yxzTB2jg7r3NrEnq+O3cfU2F99J01QwZORJ69QpN\n+f32izsakeDuu+H//g+efRZatYo7mvyRxOmqHYBF7r4YwMxGAycC75Y55lOgder+L4FlFZOCZM6Q\nIdC3b1in0LJl3NGIbHD55WHs4cgjQ3LYe++4IypsUSaGHYGyC+GXAgdUOGYI8KKZfQL8Ajg9wngK\n2sCB4RvZ5MnQvHnc0Yj83DnnhKq9Rx0VCvG1aRN3RIUrysSQTt/Pn4HZ7l5kZrsBz5lZG3dfUfHA\nPn36rL9fVFREUVFRpuLMe/37h7LZkydrDwVJth49wkY/Rx8NTz2l7s6aKikpoaSkZJPfJ8oxho5A\nH3fvmnrcGyh191vLHPMUcLO7v5J6/ALQy91nVHgvjTHU0oABG5JCs2ZxRyOSnkcfhUsvDaul9903\n7mhyVxLLbs8AWphZMzPbDOgOjK9wzHzC4DRmth2wB/B+hDEVlCFDwqwPJQXJNSefHAakjzkG3n47\n7mgKT2RdSe6+xsx6ApOAusD97v6umV2Sen0Q0A8YZmZzCEnqOnfXcpcMGDkSbroJSkqUFCQ3nXoq\nrFoFXbqELze77x53RIVD1VXz0NixoUrqiy9q9pHkvmHDoLg4lNJQxd+aSeJ0VYnBM8/AFVeElc1K\nCpIPLrgg1FQ66iiYOhW23z7uiPKfEkMeefXVMOXv8cehdevqjxfJFZddBsuWhW1DS0pg663jjii/\nqSspT8ydGwqSjRgR/nhE8o176CJ9882wCK5hw7gjSr7adiUpMeSBDz8M+zOvqzsjkq9KS+G88+Cb\nb0KtL5WJr1oSp6tKFnz9dZjSd+21SgqS/+rUCRWBf/oplNHQ98VoKDHksB9/hJNOCl1HV18ddzQi\n2VG/Pjz8MLz+OtxyS9zR5Cd1JeWo0tLQQnCH0aPDNymRQvLJJ2H3wb594dxz444mmTRdtcD85S/h\nD+O555QUpDD9+tfw9NNQVBT2K1f5tMzRR0oOGj48LGJ77DHYfPO4oxGJT6tW8O9/Q/fuYR9pyQx1\nJeWYl18OpQJeekkbmoisM3hwKBb52muwzTZxR5Mcmq5aABYtgoMPhgceCKtARWSDa66BOXPC6v/6\n9eOOJhmUGPLct99Cx47Qs2dYBSoi5a1dCyeeCE2bhsqsosSQ10pLQxni7baDQYPijkYkuZYvhwMO\nCOt6Lr447mjip1lJeaxvX/jyyzDgLCIb16gRPPEEHHII7LUXdOoUd0S5SbOSEu7xx8NKz4cfhs02\nizsakeTbY49Qqvu008KUbqk5dSUl2IIF4ZvPxInQvn3c0Yjkln794Mknwwy+Qv1SpTGGPPP996Gv\n9Oqr1VcqUhulpdCtW9jB8F//ijuaeCgx5BF3OPvs8C1n6FCwGv9vFREIVVjbtYObb4YePeKOJvs0\n+JxHBg6EefNg2jQlBZFNsfXWYXyuSxdo00aLQtOlFkPCvPEGHHdc2I2tefO4oxHJD0OHwm23hb+v\nLbeMO5rsUVdSHli+HPbbD/7+dzjllLijEckv558fWuDDhsUdSfYoMeQ499AHuu22cM89cUcjkn++\n+w723x/++tcwhlcINMaQ4+67D+bPh+nT445EJD9ttRWMGRP2Rj/gAGjRIu6IkksthgSYNw8OPxym\nTIGWLeOORiS/3XNP+CI2bRo0aBB3NNFSV1KO+vHH0Lz94x/hggvijkYk/7mH0vVNm8Ltt8cdTbSU\nGHLU1VeHZftjxmhqqki2LFsWpq8OHx66lvKVEkMOevZZuOiiUEO+ceO4oxEpLM89BxdeCLNnh0kf\n+UiJIcd8+WX4xjJyJBx5ZNzRiBSma66Bjz6CcePys8Ve28RQbXVVM8vTXBofd/jd7+CMM5QUROLU\nrx+89x6MGBF3JMlSbYvBzBYCs4FhwNNxfHXPtxbDAw+ERWwzZuT/rAiRpHvrrfAF7c03YZdd4o4m\nsyLrSjKzOkBn4EKgPTAWGObu79Um0NrIp8Tw8cfQti1MmhT+FZH49esHkyeHcb986lLKyhiDmR0B\nPAhsSWhF9Hb3V2t60prKl8TgHuogHXAAFBfHHY2IrLNmDRx0UCibcemlcUeTOVGOMTQxs6vM7E3g\nWqAn0AT4I/Dvan62q5nNN7OFZtZrI8cUmdksM5tnZiU1/Q/IJUOHwn//C3/+c9yRiEhZ9eqFqas3\n3ADvvx93NPFLpyvpPUIrYai7L63w2vXu/n8b+bm6wAJCN9THwBvAGe7+bpljtgZeAY5296Vm1sTd\nv6zkvXK+xfDRR6Eu/Isvwj77xB2NiFTmttvCrm+TJ0OdPNj4OLIWA9DS3fsC35rZL8q+sLGkkNIB\nWOTui919NTAaOLHCMWcCj6xLOJUlhXzgDpdcEhazKSmIJNcf/gCrV8OgQXFHEq90EkM7M5sLzAXm\nmdkcM9s/jZ/bEVhS5vHS1HNltQAam9lkM5thZuekFXWOeeihsLr5uuvijkREqlK3bqijdOONsHRp\n9cfnq3Sqqw4FLnP3KQBmdnDqudbV/Fw6fT/1gf2AI4GGwDQze83dF1Y8sE+fPuvvFxUVUVRUlMbb\nx++LL0IdpIkToX79uKMRkersuSf07BkGocePz61ZSiUlJZSUlGzy+6QzxjDL3dtWeG6mu+9Xzc91\nBPq4e9fU495AqbvfWuaYXsAW7t4n9fg+4Bl3f7jCe+XsGMOZZ8Kvfx36LkUkN6xaFTbN+utfc3uv\n6CjXMdwJbAGMSj3VHfgReADA3Wdu5OfqEQafjwQ+AV7n54PPLYEBwNFAA2A60N3d36nwXjmZGCZO\nhCuvhLlzoWHDuKMRkZp47TU46aRQFr9Jk7ijqZ0oE0MJ5buFrOxjdz+8ip89BrgTqAvc7+63mNkl\nqZ8blDrmWuACoBQY4u53VfI+OZcYvv8+NEnvvz+/qzeK5LOrroIVK8JU81ykInoJ06tXWOX84INx\nRyIitfXtt+EL3qhRcMghcUdTc1G2GLYGioFDU0+VAH3dfXlNT1ZbuZYY5s2DI44IXUjbbRd3NCKy\nKR5+GPr0gZkzYbPN4o6mZqJcxzAU+BY4DTgdWEEoqCeVKC0NsxluuklJQSQfnHIK7Lwz3HFH3JFk\nTzothjnu3qa656KUSy2GoUPD4phXXw1zokUk9/3nP9ChQ6jA2qxZ3NGkL8oWw0ozW9+7llrH8ENN\nT1QIli2D3r3h3nuVFETyyW67hVXRV14ZdyTZkU6LYV9gJNAo9dTXwHnuPifi2MrGkBMthssvD4th\nBgyIOxIRybSffoK994a77oJjjok7mvREMvicKoR3q7tfa2aNALI56FwmjsQnhjlzoEsXePdd7d8s\nkq8mTgzbgc6dmxsD0ZF0Jbn7WuBgC5/My+NICrnAHa64Avr2VVIQyWfHHQfNm8O//hV3JNFKpyvp\nXuDXwDg2jC24uz8acWxlY0h0i2HUKPjHP+CNNzS2IJLvFi6ETp1Cq2GHHeKOpmpRrmMYTiUF8dz9\ngpqerLaSnBi++w5atYLRo8MOUCKS/66/Hj79FEaMiDuSqkWZGA5296nVPRelJCeGG26ADz7QCmeR\nQvLdd9CyJTzySNiqN6miTAw/q6SaTnXVTEpqYliyBPbdF2bPDgtgRKRwDBsW9m6YOjW5pbkznhjM\nrBNwIPAH4HZC8TyAXwDdtMANzj0XmjaFv/0t7khEJNvWroX994e//AVOPTXuaCpX28RQ1UY9mxGS\nQN3Uv+t8CyT0MmTPjBnw/POwYEHckYhIHOrWhX/+E377Wzj+eGjQIO6IMiedrqRm7r44O+FsNIZE\ntRjcoagIzjkHLr447mhEJE4nnACHHRZ2akyaKMcY9gCuBZqxoYXh7n5ETU9WW0lLDI89BsXFMGuW\npqeKFLr580NJ7nffTd6GPlEmhreAgcBMYG3qaXf3N2scZS0lKTGsWQN77QX9+4eVziIiPXtCvXpw\n551xR1JelInhTXdvV+vIMiBJiWHwYBg7NowviIgAfPZZ2NAnadVXo0wMfYAvgEeBn9Y97+5f1fRk\ntZWUxPDDD9CiBTz+OLRvH3c0IpIkxcWweHGyFr1FmRgWU/nK511rerLaSkpiuPXWMBtp3Li4IxGR\npPn22/DF8fnnYZ994o4m0J7PEfv6a9h997CYZY89Yg1FRBLqzjvhxRdh/Pi4IwkyXl3VzK4rc/+0\nCq/1q+mJct2tt0K3bkoKIrJxv/99KME/NWsFg6JR1crnWe7etuL9yh5HLe4Ww6efhg065syBnXaK\nLQwRyQEjRoRSGS+/HH+pjCi39ix4t9wC55+vpCAi1Tv7bPjii9yeuVhVSQwBli6Fhx6Cd96JOxIR\nyQV164YZSsXF0Llz/K2G2qiqK2ktGzbm2QJYWeblLdw9a0klzq6kyy6DrbaCv/89ltOLSA5auxZa\ntw61lLp2jS8OzUqKwIcfQtu2oVDe//xP1k8vIjls3Liws+P06fG1GjTGEIGbbw6zDJQURKSmTjkF\nfvwRJk6MO5KaU4thIz74INRaf+892HbbrJ5aRPLEo4+GL5gzZsTTalCLIcP69YNLL1VSEJHaO+mk\nMN4wYULckdSMWgyVWLIE2rQJrYWkldEVkdwyblwYhJ42LfutBrUYMui22+DCC5UURGTTnXwyfPNN\nKJWRKyJNDGbW1czmm9lCM+tVxXHtzWyNmZ0cZTzp+PxzeOCBZO7GJCK5p25d6N07jDXkisgSg5nV\nBQYAXYE9gTPMrNVGjrsVeAaIfSnIHXdAjx6www5xRyIi+eLMM8OElldfjTuS9ETZYugALHL3xe6+\nGhgNnFjJcVcADxP2fIjV11+HjXiuu676Y0VE0lW/fvhcyZVWQ5SJYUdgSZnHS1PPrWdmOxKSxcDU\nU7GOhPfvD8cfn6wdmEQkP1xwQdgnftasuCOpXpSJIZ0P+TuB61NTjowYu5J++AEGDIDrr48rAhHJ\nZ5tvHsYuc6G8TpT1jj4Gdi7zeGdCq6GsdsBoC3O4mgDHmNlqd//ZNhd9+vRZf7+oqIiioqKMBjt8\nOBx4ILRsmdG3FRFZ77e/DWukFi+OpmeipKSEkpKSTX6fyNYxmFk9YAFwJPAJ8Dpwhru/u5HjhwFP\nuvujlbwW6TqGtWvDBjzDh8PBB0d2GhERrrsOVq0Ku71FLXHrGNx9DdATmAS8A4xx93fN7BIzuySq\n89bG44+HekgHHRR3JCKS7666CkaODJNdkqrgVz67Q6dO8Kc/haJXIiJRO++80G3du3e051HZ7Vqa\nOjXszrZgQViIIiIStblz4eijw9qGBg2iO0/iupJyxW23wTXXKCmISPbss0/YyOehh+KOpHIF3WJY\nuDCMKyxeDA0bZvztRUQ26oUX4MorYd686IrrqcVQCwMGwMUXKymISPYdcQTUqZPM4noF22JYsQKa\nNoU5c2Dnnas/XkQk0wYPDju8PfFENO+vFkMNjRgBnTsrKYhIfM46C155Bd5/P+5IyivIxFBaGuoi\nXXll3JGISCHbcsuw98vdd8cdSXkFmRiefTaMK2hBm4jE7bLLQtWF776LO5INCjIx3HVXaC3EsTm3\niEhZzZrBoYfCgw/GHckGBTf4vG6K6kcfhWqHIiJxmzwZLr8c3n47s19YNficpnvuCVNUlRREJCmK\nisLU1cmT444kKKgWw8qVsMsu8MYb2oxHRJJlwACYMgXGjMnce6rFkIaHH4b991dSEJHkOftsmDQJ\nPv887kgKLDEMGgSXJKrgt4hIsPXWcPLJMGxY3JEUUFfS229Dly7w4YdQL8p960REamn6dDjzzDBJ\npk4GvrarK6kagwaFhSRKCiKSVB06wFZbxV8/qSBaDD/8EEpfzJwZ6iOJiCTVPfeE2Unjxm36e6nF\nUIWxY6FjRyUFEUm+s86C55+H//43vhgKIjEMHqxBZxHJDY0ahW2G4xyEzvuupAULwuKRJUs0viAi\nuWHatLDl8Pz5m7YSWl1JGzFiRJgfrKQgIrmiY8fw7/Tp8Zw/rxPD2rUwciScd17ckYiIpM8stBiG\nD4/n/HmdGF54AbbfHvbeO+5IRERq5pxzwsSZlSuzf+68TgzDh4esKyKSa3baKZTwiWrbz6rk7eDz\n8uVheup//gPbbhtRYCIiEfr3v+GBB+Dpp2v38xp8rmDs2LCns5KCiOSqk04KA9Aff5zd8+ZtYhg+\nXIPOIpLbGjYMaxqyvbtbXiaGRYvCrWvXuCMREdk0558fpt1ns9c/LxPDqFHQvTvUrx93JCIim+bA\nA0O9t7feyt458y4xuIfE0KNH3JGIiGw6s/B5Nnp09s6Zd4lh3jz4/vsNKwdFRHLdusSQre6kvEsM\no0eHbqRMbHIhIpIEbdpAgwbw+uvZOV/kH59m1tXM5pvZQjPrVcnrZ5nZHDN7y8xeMbPWtT2Xe0gM\n6kYSkXyS7e6kSBODmdUFBgBdgT2BM8ysVYXD3gcOdffWwN+AwbU934wZoVhe27a1fQcRkWTq0SOs\nz1q7NvpzRd1i6AAscvfF7r4aGA2cWPYAd5/m7stTD6cDO9X2ZOsGnTelTK2ISBK1bAm/+hVMmRL9\nuaJODDsCS8o8Xpp6bmMuAp6qzYlKS2HMmDC+ICKSj7LVnRT1LgVpj6Gb2eHAhcBBlb3ep0+f9feL\nioooKioq9/rUqdCkCey5Z23CFBFJvu7doX176N+/8nVaJSUllJSUbPJ5Ii2iZ2YdgT7u3jX1uDdQ\n6u63VjiuNfAo0NXdF1XyPtUW0bviCthhB/jznzMWvohI4nTsCH37Qpcu1R+b1CJ6M4AWZtbMzDYD\nugPjyx5gZrsQksLZlSWFdLjD449Dt26bHK+ISKJ16xZ9Ke5IE4O7rwF6ApOAd4Ax7v6umV1iZpek\nDrsR2AYYaGazzKzGM3VnzgzFplq2zFjoIiKJdNJJITGUlkZ3jrzYj+GGG2DVKrj11o0eIiKSN1q1\nCtsWt29f9XFJ7UrKiscfD1lURKQQnHRS+NyLSs4nhkWL4Isv4IAD4o5ERCQ7TjxRiaFKTzwBJ5yg\n2kgiUjg6dICvvoKFC6N5/5z/OH3iCXUjiUhhqVMntBqimp2U04nh88/D5hVHHBF3JCIi2RXlOENO\nJ4YJE+Coo2DzzeOOREQkuw4/POw/89lnmX/vnE4Mmo0kIoWqQQM4+mgYP776Y2sqZxPDTz/B5Mlw\nzDFxRyIiEo/jj4enn878++ZsYnjlFdhrL2jcOO5IRETi0blz+IK8Zk1m3zdnE8Nzz6VXREpEJF9t\nvz00bQpvvJHZ983ZxPDss2HgWUSkkB11VPg8zKScTAxffBFWPHfsGHckIiLx6tJFiQGAF16AoqLK\nN6oQESkkBx8c1nMtX179senKycSgbiQRkWCLLaBTpzAInSk5lxjcQ2LQwLOISJDp7qScSwzz50O9\netCiRdyRiIgkQ5cuYaZmpuRcYljXjWQ13npCRCQ/7bMPrFgBH3yQmffLucSg9QsiIuWZhS/MmWo1\n5FRi+OknePllVVMVEakok+sZcioxLFgA7drBttvGHYmISLIcdVQYg3Xf9Pcyz8S7RMzMfF2c7hpf\nEBGpTMXPRzPD3Wv8iZlTLQZQUhAR2ZhMfT7mXGIQEZFoKTGIiEg5SgwiIlKOEoOIiJSjxCAiIuUo\nMYiISDlKDCIiUo4Sg4iIlKPEICIi5USaGMysq5nNN7OFZtZrI8fclXp9jpm1jTIeERGpXmSJwczq\nAgOArsCewBlm1qrCMccCzd29BfA7YGBU8eSLkpKSuENIDF2LDXQtNtC12HRRthg6AIvcfbG7rwZG\nAydWOOYEYASAu08Htjaz7SKMKefpl34DXYsNdC020LXYdFEmhh2BJWUeL009V90xO0UYk4iIVCPK\nxJBuPe+K9QCTXwdcRCSPRbYfg5l1BPq4e9fU495AqbvfWuaYe4ESdx+dejwfOMzdP6vwXkoWIiK1\nUJv9GOpFEUjKDKCFmTUDPgG6A2dUOGY80BMYnUok31RMClC7/zAREamdyBKDu68xs57AJKAucL+7\nv2tml6ThwcmRAAAEuklEQVReH+TuT5nZsWa2CPgeuCCqeEREJD05sbWniIhkT6JWPmtB3AbVXQsz\nOyt1Dd4ys1fMrHUccWZDOr8XqePam9kaMzs5m/FlS5p/H0VmNsvM5plZSZZDzJo0/j6amNkzZjY7\ndS3OjyHMrDCzoWb2mZnNreKYmn1uunsiboTupkVAM6A+MBtoVeGYY4GnUvcPAF6LO+4Yr0UnoFHq\nftdCvhZljnsRmACcEnfcMf1ObA28DeyUetwk7rhjvBZ9gFvWXQdgGVAv7tgjuh6HAG2BuRt5vcaf\nm0lqMWhB3AbVXgt3n+buy1MPp5O/6z/S+b0AuAJ4GPgim8FlUTrX4UzgEXdfCuDuX2Y5xmxJ51p8\nCvwydf+XwDJ3X5PFGLPG3acAX1dxSI0/N5OUGLQgboN0rkVZFwFPRRpRfKq9Fma2I+GDYV1JlXwc\nOEvnd6IF0NjMJpvZDDM7J2vRZVc612IIsJeZfQLMAa7KUmxJVOPPzSinq9aUFsRtkPZ/k5kdDlwI\nHBRdOLFK51rcCVzv7m5mxs9/R/JBOtehPrAfcCTQEJhmZq+5+8JII8u+dK7Fn4HZ7l5kZrsBz5lZ\nG3dfEXFsSVWjz80kJYaPgZ3LPN6ZkNmqOman1HP5Jp1rQWrAeQjQ1d2rakrmsnSuRTvCWhgI/cnH\nmNlqdx+fnRCzIp3rsAT40t1XAivN7GWgDZBviSGda3EgcDOAu//HzD4A9iCsryo0Nf7cTFJX0voF\ncWa2GWFBXMU/7PHAubB+ZXWlC+LyQLXXwsx2AR4Fznb3RTHEmC3VXgt3/4277+ruuxLGGS7Ns6QA\n6f19PAEcbGZ1zawhYaDxnSzHmQ3pXIv5QGeAVH/6HsD7WY0yOWr8uZmYFoNrQdx66VwL4EZgG2Bg\n6pvyanfvEFfMUUnzWuS9NP8+5pvZM8BbQCkwxN3zLjGk+TvRDxhmZnMIX4Cvc/evYgs6QmY2CjgM\naGJmS4BiQrdirT83tcBNRETKSVJXkoiIJIASg4iIlKPEICIi5SgxiIhIOUoMIiJSjhKDiIiUo8Qg\nUgUzG2Nmv0ndn2hmv6zuZ1LHnmZmb5vZWjPbr8zzrc3s/qjiFckEJQaRjTCz5sCW7v4+gLsf5+7f\npvnjc4FuwMtln3T3t4DdzOxXGQ1WJIOUGKTgpUorzDezB83sHTMbZ2ZbAD0oU2rBzBabWeN03tPd\n57v7ext5+WngtE2PXCQaSgwiwe7A3e6+J/AtcBmhYm3ZomvrywSY2cupndIq3o5I41yvA4dmMniR\nTEpMrSSRmC1x92mp+w8S6vc3JWz48jPuvikf7J8Sdh8TSSQlBpGgbNEwIxShg420qs1sCrBVJS9d\n6+4vVHMuIz/3EZE8ocQgEuxiZh3d/TXCFplTgS2A7amkdr27H1KD9664ScoOwIe1DVQkahpjEAkW\nAJeb2TtAI8I2oVOB/cscU5Od9bqlSiB3BCaa2dNlXu5AhdlKIkmisttS8MysGfCku+9T4fnfAP3d\n/bgMn68EON3dP8/k+4pkiloMIsHPviGl1i+sSO0ZnBGp7VgXKSlIkqnFICIi5ajFICIi5SgxiIhI\nOUoMIiJSjhKDiIiUo8QgIiLlKDGIiEg5/w9bOkxL02j27AAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "\n", "\n", "def entropy(p):\n", " return - p*np.log2(p) - (1 - p)*np.log2((1 - p))\n", "x = np.arange(0.0, 1.0, 0.01)\n", "ent = [entropy(p) if p != 0 else None for p in x]\n", "plt.plot(x, ent)\n", "plt.ylim([0,1.1])\n", "plt.xlabel('p(i=1)')\n", "plt.axhline(y=1.0, linewidth=1, color='k', linestyle='--')\n", "plt.ylabel('Entropy')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Gini Impurity\n", "\n", "$$I_G(t) = \\sum_{i =1}^{C}p(i \\mid t) \\big(1-p(i \\mid t)\\big)$$" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYVOWZ9/HvTbOocQEGxYgYBJegUaMCsri04tJBBDUx\nCm64ROIb1KgZFTIJ/SZjEpKJu0FEFMO4xhUjSgQpEVRADYsjEDqoI6CgKO4Smr7nj6eQrraXqu46\ndWr5fa6rLutUPX3q5th97np2c3dERES2aBV3ACIikl+UGEREJIUSg4iIpFBiEBGRFEoMIiKSQolB\nRERStI47gHSYmcbUiog0g7tbpj9TMDUGd9fDnbFjx8YeQ748dC10LXQtGn80V8EkBhERyQ0lBhER\nSaHEUGDKy8vjDiFv6FpspWuxla5Fy1lL2qFyxcy8EOIUEcknZoYXc+eziIjkhhKDiIikUGIQEZEU\nSgwiIpJCiUFERFIoMYiISAolBhERSaHEICIiKZQYREQkRaSJwczuNLO1ZrakkTI3mdkKM1tkZgdH\nGY+IiDQt6hrDXUBFQ2+a2SBgL3ffG7gIGB9xPCIi0oRIE4O7Pw982EiRIcDdybLzgPZm1jnKmERE\npHFx9zF0Ad6udbwK2D2mWEREhPgTA0Ddlf+0jKqISIzi3vN5NdC11vHuyde+prKy8qvn5eXlWnNd\nRKSORCJBIpFo8Xki34/BzLoBT7j7AfW8NwgY5e6DzKwvcIO7962nnPZjEBHJUHP3Y4i0xmBm9wFH\nAZ3M7G1gLNAGwN0nuPs0MxtkZlXAZ8B5UcYjIiJN0w5uIiJFSju4iYhIVigxiIhICiUGERFJocQg\nIiIplBhERCSFEoOIiKRQYhARkRRKDCIikkKJQUREUigxiIhICiUGERFJocQgIiIplBhERCSFEoOI\niKRQYhARkRRKDCIikkKJQUREUigxiIhICiUGERFJocQgIiIplBhERCSFEoOIiKRQYhARkRRKDCIi\nkkKJQUREUigxiIhICiUGERFJocQgIiIplBhERCSFEoOIiKRQYhARkRRKDCIikiLSxGBmFWa2zMxW\nmNnV9bzfycyeNrOFZvaamY2IMh4REWmauXs0JzYrA5YDxwKrgQXAMHdfWqtMJdDO3UebWadk+c7u\nXl3nXB5VnCIixcrMcHfL9OeirDH0Aarc/U133wTcDwytU+YdYMfk8x2B9XWTgoiI5FbrCM/dBXi7\n1vEq4LA6ZSYCz5rZGmAH4IcRxiMiImmIMjGk0/YzBljo7uVm1gN4xswOcvdP6hasrKz86nl5eTnl\n5eXZilNEpCgkEgkSiUSLzxNlH0NfoNLdK5LHo4Eadx9Xq8w04Fp3n5s8nglc7e4v1zmX+hhERDKU\nj30MLwN7m1k3M2sLnA5MrVNmGaFzGjPrDOwLrIwwJhERaUJkTUnuXm1mo4DpQBkwyd2XmtnI5PsT\ngN8Ad5nZIkKSusrdP4gqJhERaVpkTUnZpKYkEZHM5WNTkoiIFCAlBhERSaHEICIiKZQYREQkhRKD\niIikUGIQEZEUSgwiIpJCiUFERFIoMYiISAolBhERSaHEICIiKZQYREQkhRKDiIikUGIQEZEUSgwi\nIpKiycRgZpeaWYdcBCMiIvFLp8bQGVhgZg+aWYWZZbzpg4iIFI60dnAzs1bA8cAIoBfwIGGrzn9G\nGt3Wz9cObiIiGYp0Bzd3rwHeBdYCm4EOwENm9odMP1BERPJbkzUGM7sMOAdYD9wBPOrum5K1iBXu\n3iPyIFVjEBHJWHNrDK3TKNMRONXd36r9orvXmNlJmX6giIjkt3SaknrUTQpmNgXA3V+PJCoREYlN\nOolh/9oHZtYaODSacEREJG4NJgYzG2NmnwAHmNknWx7AOmBqziIUEZGcSqfz+Xfufk2O4mkoBnU+\ni4hkqLmdzw0mBjP7trsvM7NDga8VcvdXMw+zeZQYREQyF0VimOjuPzKzBPUnhqMzjrKZlBhERDKX\n9cSQPGkroJ+7z21JcC2lxCAikrlIZj4nZzzf2uyoRESk4KQzXHWGmf1Ai+eJiJSGdEYlfQpsR1gj\n6cvky+7uO0YcW+0Y1JQkIpKhyJbEcPftmxeSiIgUoiYTg5kdWd/r7j47jZ+tAG4AyoA73H1cPWXK\ngeuBNsD77l7e1HlFRCQ66TQl/ZWtw1W3AfoAr7j7MU38XBmwHDgWWA0sAIa5+9JaZdoDc4ET3H2V\nmXVy9/frOZeakkREMhRlU9LgOh/UFbgxjXP3Aarc/c3kz90PDAWW1iozHHjY3VclP+trSUFERHIr\nrY166lgF9EyjXBfg7To/16VOmb2BjmY2y8xeNrOzmxGPiIhkUTp9DDfXOmwFfBd4JY1zp9P20wY4\nBBhIGPn0opm95O4r6hasrKz86nl5eTnl5eVpnF5EpHQkEgkSiUSLz5NOH8MItt7kq4E305kJbWZ9\ngUp3r0gejwZqandAm9nVwLbuXpk8vgN42t0fqnMu9TGIiGQokiUxap28HfBtoAZY7u7/SuNnWhM6\nnwcCa4D5fL3z+dvALcAJQDtgHnB63Q2AlBhERDIXWeezmZ0I3AasTL7U3cxGuvu0xn7O3avNbBQw\nnTBcdZK7LzWzkcn3JyRXb30aWExIOhO1K5yISLzSaUpaDpzo7lXJ4x7ANHffNwfxbYlBNQYRkQxF\nsohe0sdbkkLSSuDjTD9IREQKQzo1htuAPYAHky+dBvwv8AyAuz8SZYDJGFRjEBHJUGSdz2Y2Ofl0\nS0Gr9Rx3Py/TD82UEoOISOYiHZUUNyUGEZHMRTkqqTtwCdCtVnl39yGZfpiIiOS/JhMD8BhwB/AE\nYUgppDerWUREClA6ieFLd78p8khERCQvpNP5fDbQgzBRbeOW19391WhDS4lBfQwiIhmKrI8B2B84\nGziarU1JJI9FRKTIpFNj+CfQM531kaKiGoOISOainPm8BOiQeUgiIlKI0mlK6gAsM7MFbO1j0HBV\nEZEilU5iGBt5FCIikjc081lEpEhlfVSSmX1KwxPZ3N13zPTDRAqRO/zrX/DFF+GxcSNs3hweNTVQ\nVhYerVtDu3aw3Xaw7bbhWKQguXvePwgJ6muPsWPHen3Gjh2r8irfZPlLLx3rzz3nfs897uPGuV9+\nufuZZ7p3715/+bZtx3rnzu5du7rvuaf7Xnu577OPe4cO9ZcvKxvru+wSyvTp4z54sPt557kPGFAY\n10flC7884N6Me66akqRoucM778Drr8PSpbBsGfzjH7ByJaxaBbvuCnvsAV26wO67h+NddgmPTp2g\nffvw2GknaNMm88/+4gv46KPw+OADeP99WLcuPNasCTGsWgVvvQWffw7du0OPHrDvvvDtb4fH/vvD\njqqbSzNpdVUpae5QVQULFsArr8DChbBoEZjBd74TbrI9e8I++4Sb77e+BW3bxh31Vh9/DG+8Ef4N\ny5aFx9Kl4dG5Mxx0EBx8MPTqBb17w847xx2xFAIlBikpn30G8+bB3LnhMW9e+Gbdu3e4eX73u+Fm\nuuuuITkUqs2bQ7JYtAj+/veQ+F5+OdRk+veHAQPC44ADQj+HSG1KDFLUNm6EF16AZ5+FWbNCjeDA\nA7feGPv1C9+sS0FNDaxYEa7H3LkwZw6sXQtHHglHHw0DB4ZaUiEnRMmOrCcGM5vr7gMaGJ3knsNR\nSUoMpWnlSpg2DaZPh+eeC01BAweGm9+AAWH0jwTvvguJREiazzwTEukJJ0BFRfjvTjvFHaHEQTUG\nKXg1NTB/Pjz+ODzxROisHTQo3NyOPRY6dow7wsJRVQVPPx0es2eHJrYhQ+Dkk0P/ipSGSBODmZUB\nnak178Hd/zfTD2suJYbiVVMTmkMeeggeeQR22AFOOSXcxHr3hlbprOYljfrsM5gxA6ZODY9u3eD7\n34fTTgsd8VK8IksMZnYJYVmMdcDmLa+7+wGZflhzKTEUF/fQmXrvvXDffaEj9Yc/DDer/faLO7ri\nVl0dmuUefjg8unWDM88M13/XXeOOTrItysTwT6CPu69vbnAtpcRQHNauhXvugcmTw/DM4cNh2LAw\nokZyr7oaZs4MCXrq1NCBf955cNJJsM02cUcn2RBlYpgFHO/um5obXEspMRSuzZtDO/ftt4e27lNO\ngREj4PDD1UyUTz7/PDTlTZ4cRnwNHw4XXRRGN0nhijIx3AnsAzwJbNmsx939uoyjbCYlhsKzZg1M\nnAiTJsFuu8GPfgSnnw7bbx93ZNKUt96CO+8M/+/22ANGjgz/71SLKDxRJobK5NMtBY2QGP5/ph/W\nXEoMhcE9jK2/+eYwxPSMM8JN5bvfjTsyaY7q6jBcePz4MJv8wgvhxz8OyUIKg4arSmw2bQqjiq67\nDjZsgJ/8JDQXtW8fd2SSLf/4B9x6K0yZAscdB1deCX36xB2VNCWKCW43uvtlZvZEPW+753AHNyWG\n/PTpp6Hv4MYbYc894YorYPBg9R0Us48/Dk1MN94IXbvCv/+7/p/nsygSw6Hu/oqZldfztrv7c5l+\nWHMpMeSX9evhppvgT3+CY44JN4deveKOSnKpujp0Vo8bF2ZZX3NNaDrUHhT5JS+bksysArgBKAPu\ncPdxDZTrDbwI/NDdH6nnfSWGPLBuHfzhD+Eb4/e/D1ddBXvvHXdUEif3sATHb38bOq3HjIFzzsmv\nlWtLWXMTQ4MVQDM72cxG1Tqeb2ZvJB+npRFQGXALUAHsBwwzs54NlBsHPE3o2JY8s24d/OxnYenq\nL76AxYvDiCMlBTGD448PazRNmQIPPhiWNr/99rDrnRSmxloGrwKm1jpuC/QCjgIuTuPcfYAqd38z\nOQfifmBoPeUuAR4C3ksrYsmZDz8M3wB79gzNBYsXwy23hE1tROoaMAD+9rcwYe6hh8IXiSlTwlwW\nKSyNJYa2ddZDmuPu65OvfSONc3cB3q51vCr52lfMrAshWYxPvqT2ojzw+efwm9+Eb37r1oV9AG6+\nWQlB0tO/f0gQd90FEyaE5dEfeyw0O0lhaCwxdKh94O6jah2ms39UOr8GNwDXJDsQDDUlxaq6Gu64\nIySEhQvD4nZ33KFx69I8Rx0Fzz8Pv/89/PKXYbb7Cy/EHZWko7ExBPPM7CJ3v732i2b2Y2BeGude\nDXStddyVUGuo7VDgfgs7inQCvmdmm9x9ap1yVFZWfvW8vLyc8vLyNEKQdD31VOhH6NQpLK522GFx\nRyTFwAxOPDEsnT5lShi51KtXSBZ77RV3dMUnkUiQSCRafJ7Ghqt2Bh4DNgKvJl8+BNgGONnd3230\nxGatgeXAQGANMB8Y5u5LGyh/F/CERiXl1uuvh8lKK1eGEUcnnaSdvyQ6X3wBN9wAf/wjnH8+/Pzn\n2kQoSlkfleTua4H+wK+BN4E3gF+5e9+mkkLy56uBUcB04HXgAXdfamYjzWxkpoFKdm3YAJddFqr7\nJ5wAS5aEPRCUFCRK224Lo0fDa6/BBx/AvvuG4c81NXFHJrVpSYwSU1MDd98dRhsNHQr/+Z+h+Ugk\nDq++CqNGhf6tW28NmzNJ9uTlBLdsUWLIjoUL4eKLw+iQW2+FQw+NOyKR8GVlypQwe/qkk+B3v9M2\nrtmS9aYkKR6ffBLWMTrhBLjggjAyRElB8kWrVnDuubBsGbRrB/vvHxKFvgvGR4mhyE2dGv7QPvgg\ntOteeKEWPJP8tNNOYb7M1Klw/fUwcCCsWBF3VKVJt4gitXZt2FzlyitDn8LkybBzOrNPRGLWuzfM\nnx+alfr1Cwv1bYpt/8jSpMRQZNxDIjjwQOjePSxjcfTRcUclkpnWreHyy2HBAnj22bD3w9//HndU\npUOdz0Vk9eqwT+/q1WE5goMPjjsikZZzhz//OSzvfvHFYe6DVm9NjzqfS9iWWsLBB4dvVvPnKylI\n8TALndMLF4YtRnv3Vu0haqoxFLh160It4Y03QnLQ/spSzNzDiKWf/Qx++tOwJ4g2B2qYagwl6PHH\n4aCDwvLG8+crKUjxMwsbAb3yCsycCUceCVVVcUdVfJQYCtBnn4VawuWXw1/+EiYEtWsXd1QiudO1\na9g57owzwsilSZM07yGblBgKzKuvhslpGzeGNtfDD487IpF4tGoFl14KiQTceCOcdlqYryMtp8RQ\nIGpqwoqUFRVQWRn6E3bcMe6oROK3//6hKXX33UNz6uzZcUdU+NT5XADeey+MyvjwQ7jvPujWLe6I\nRPLTU0+F5by3DGstK4s7onip87lIPfccHHJImLA2e7aSgkhjvve90DE9axYcdxy8807cERUmJYY8\nVVMT9l0+4wyYODF0MLdpE3dUIvlvt91gxowwYunQQ8PMacmMmpLy0Pr1YUjehg3wwAOh7VREMjdj\nBpx9dtjzYfTo0ltAUk1JRWLBgvAtp2fPMNpCSUGk+Y49Fl5+OfQ9DB6sUUvpUmLII5MmhY3Tr7sO\n/uu/1HQkkg1duoQ+h549w3IaCxfGHVH+U1NSHti4MYzHnj0bHn00zGQWkex74IHQrHT99XDWWXFH\nEz1t7Vmg1qyBU08NTUZ33QU77BB3RCLF7bXXwt/coEGhZl7May2pj6EAvfRSWA11yJCwtIWSgkj0\nvvOdMCFu+fKw3e369XFHlH+UGGIyeXJICOPHw5gxYXEwEcmN9u3hr3+FXr1Cv8PixXFHlF/UlJRj\nmzfD1VeHlVGnTg0dYiISn3vvhcsuC4M/hgyJO5rsam5TUhG3ruWfjz+G4cPh889h3jzo2DHuiERk\n+HDo0SP0OyxfHvZ6KPUavJqScuStt2DAgDArc/p0JQWRfHLYYaHP7557wlpL//pX3BHFS4khB+bP\nh/794YILYMIEzU8QyUddu8KcOWGxyhNOCP8tVUoMEXv00TBpbfz4sBVhqVdRRfLZ9tvDww+HhSv7\n9YOVK+OOKB7qY4iIe5hE88c/wtNPh2UuRCT/lZWFv9sePULz76OPQt++cUeVWxqVFIHNm+HKK8PW\ng089BXvsEXdEItIcTz4JI0YU7ogljUrKE19+Gabav/9+aK/s0CHuiESkuU48EaZNg6FDYfXqsAFQ\nKVAfQxZ9+CEcf3yYYj99upKCSDHo3Ruefz40Df/856GZuNgpMWTJ6tVhY5BevcKEmXbt4o5IRLKl\nRw944YWwv8OFF0J1ddwRRSvyxGBmFWa2zMxWmNnV9bx/ppktMrPFZjbXzA6MOqZsW7YsdFKdfXbo\ntCq1zUBESkGnTjBzZvgSeOqpYaJqsYr0FmZmZcAtQAWwHzDMzOouArESONLdDwR+DdweZUzZtmAB\nlJdDZSVcdZWGo4oUs+23D0vZ7LBDaDbesCHuiKIR9XfbPkCVu7/p7puA+4GhtQu4+4vu/lHycB5Q\nMHuWzZoVOqcmTgwjF0Sk+LVtC1OmhGbj8nJYuzbuiLIv6sTQBXi71vGq5GsNuQCYFmlEWfL443D6\n6WG57JNOijsaEcmlVq1CZ/Qpp8ARR4Qlb4pJ1MNV0+6/N7OjgfOBAfW9X1lZ+dXz8vJyysvLWxha\n802ZEpqNpk0L3xpEpPSYwdixYQnvI46Av/0t/t0XE4kEiUSixeeJdIKbmfUFKt29Ink8Gqhx93F1\nyh0IPAJUuHtVPefJmwlut90G114bfgm0ZLaIANx9N4weHSa0HnRQ3NFsla8T3F4G9jazbsAa4HRg\nWO0CZrYHISmcVV9SyCfXXQc33wyJRBi+JiICcO65sN12oUP6iSfCzoyFLNLE4O7VZjYKmA6UAZPc\nfamZjUy+PwH4JdABGG9hSM8md8+ry+oeagl//jPMnh1WYRQRqe2002CbbWDw4LAQ3xFHxB1R82mt\npCa4wy9+AY89Fia37LprLGGISIGYMQOGDYMHHoBjjok3luY2JWkqViPc4ZprQtVw1iwlBRFp2rHH\nhtGKp58eFtIsREoMDXAPW/w98ww8+yzsvHPcEYlIoSgvD8t1n3lmWHa/0Cgx1MMdLr889CfMnAn/\n9m9xRyQihebww8N8p3POCUPbC4kSQx3ucMUVMHduqC1ohVQRaa5+/cISGiNGhKGshUKJoZYtzUfP\nPx/mKbRvH3dEIlLo+vYNNYdzzy2cZiUlhiT3MJs5kVBNQUSyq1+/MLLxnHPCl858p8SQ9ItfhGFm\nSgoiEoX+/UOH9FlnhVGO+UyJAfj1r0M2f+YZ6Ngx7mhEpFgNGAAPPhiGss6ZE3c0DSv5xPD738M9\n94TRR506xR2NiBS78vJwzzn1VJg3L+5o6lfSieGWW2DChJAUOneOOxoRKRXHHQd33QVDhsDChXFH\n83UlmxgmTw61hZkzoUtjO0SIiETgxBPh1lth0CBYvjzuaFJFvbpqXvrLX2DMmNAB1K1b3NGISKn6\nwQ/g009DDWL27Py5H5VcYnjqKRg1KgwZ23ffuKMRkVI3YkRIDgMHhg7pb34z7ohKLDHMmRPGEU+d\nml+baYhIaRs1Cj76KOzn8Nxz8Y+OLJlltxctChd9ypTwXxGRfLJl5YUXXwxD57/xjZafs7nLbpdE\nYqiqgiOPhBtvDJtpiIjkI3c4/3xYsyYs99+2bcvOp8TQgHfeCZNKrrkGLrooy4GJiGRZdXX4Atuu\nHdx7L7RqwdhRbdRTj48+gooKuOACJQURKQytW8N994UvtT/9aahF5FrR1hi+/DIkhQMOgJtuAss4\nZ4qIxGfDBjjqqLB8xpgxzTtHc2sMRTkqafPmsHNS586hX0FJQUQKTfv2YXj9gAGwyy5w4YW5++yi\nSwzucOmlIdtOm9ay9jkRkTjtthtMnx5qDt/8ZpgtnQtFd9scNy7MV3jkkdB5IyJSyPbZJ6z+PGIE\nLFiQm88sqsQwZQrcdluofu20U9zRiIhkx2GHwZ13hkX3qqqi/7yi6XyeMSP0K8yaBfvtl6PARERy\naMIE+MMfwiS4nXduunxJz2NYsiSsM/LQQ2Eim4hIsfqP/wirQj/7LGy7beNlSzYxrFkT9lP97W9h\n+PAcByYikmPuYXvQjRvDbnCNDbApyQlun3wSeulHjlRSEJHSYBb6G957D666KqLPKNQaQ3U1DB0a\nhnPdfrvmKohIafngA+jfHy65BH7yk/rLlNwEtyuugE2b4E9/UlIQkdLTsWOYqzVgAPToEVZ6yJaC\nbEq65ZYwCunBB6FNm7ijERGJR/fuYdDNOefAa69l77wFlximTYNrr4UnnwxTxkVEStmAAXDDDTB4\nMLz7bnbOGWliMLMKM1tmZivM7OoGytyUfH+RmR3c2PmWLAmz/x5+GPbcM5KQRUQKzvDh4d548slh\nrbiWiiwxmFkZcAtQAewHDDOznnXKDAL2cve9gYuA8Q2db926MOvv+utDh0upSiQScYeQN3QtttK1\n2KpUr8XYsWHyW1lZy88VZY2hD1Dl7m+6+ybgfmBonTJDgLsB3H0e0N7MOtd3slNPDTObzzwzwogL\nQKn+0tdH12IrXYutSvVamMERR2TnXFEmhi7A27WOVyVfa6rM7vWdbJdd4Fe/ymp8IiJSjygTQ7oT\nJOoONq3356ZM0RLaIiK5ENkENzPrC1S6e0XyeDRQ4+7japW5DUi4+/3J42XAUe6+ts658n8WnohI\nHsq3CW4vA3ubWTdgDXA6MKxOmanAKOD+ZCLZUDcpQPP+YSIi0jyRJQZ3rzazUcB0oAyY5O5LzWxk\n8v0J7j7NzAaZWRXwGXBeVPGIiEh6CmKtJBERyZ286s7N9oS4QtbUtTCzM5PXYLGZzTWzA+OIMxfS\n+b1IluttZtVmdmou48uVNP8+ys3s72b2mpklchxizqTx99HJzJ42s4XJazEihjBzwszuNLO1Zrak\nkTKZ3TfdPS8ehOamKqAb0AZYCPSsU2YQMC35/DDgpbjjjvFa9AN2Sj6vKOVrUavcs8Bfge/HHXdM\nvxPtgf8Bdk8ed4o77hivRSXw2y3XAVgPtI479oiuxxHAwcCSBt7P+L6ZTzWGrE6IK3BNXgt3f9Hd\nP0oezqOB+R9FIJ3fC4BLgIeA93IZXA6lcx2GAw+7+yoAd38/xzHmSjrX4h1gx+TzHYH17l6dwxhz\nxt2fBz5spEjG9818SgxZnRBX4NK5FrVdAEyLNKL4NHktzKwL4cawZUmVYuw4S+d3Ym+go5nNMrOX\nzezsnEWXW+lci4nA/ma2BlgEXJaj2PJRxvfNfNqPIasT4gpc2v8mMzsaOB8YEF04sUrnWtwAXOPu\nbmbG139HikE616ENcAgwENgOeNHMXnL3FZFGlnvpXIsxwEJ3LzezHsAzZnaQu38ScWz5KqP7Zj4l\nhtVA11rHXQmZrbEyuydfKzbpXAuSHc4TgQp3b6wqWcjSuRaHEubCQGhP/p6ZbXL3qbkJMSfSuQ5v\nA++7+xfAF2Y2GzgIKLbEkM616A9cC+Du/zSzN4B9CfOrSk3G9818akr6akKcmbUlTIir+4c9FTgH\nvppZXe+EuCLQ5LUwsz2AR4Cz3L0qhhhzpclr4e7d3X1Pd9+T0M9wcZElBUjv7+Nx4HAzKzOz7Qgd\nja/nOM5cSOdaLAOOBUi2p+8LrMxplPkj4/tm3tQYXBPivpLOtQB+CXQAxie/KW9y9z5xxRyVNK9F\n0Uvz72OZmT0NLAZqgInuXnSJIc3fid8Ad5nZIsIX4Kvc/YPYgo6Qmd0HHAV0MrO3gbGEZsVm3zc1\nwU1ERFLkU1OSiIjkASUGERFJocQgIiIplBhERCSFEoOIiKRQYhARkRRKDCKNMLMHzKx78vmTZrZj\nUz+TLHuamf2PmW02s0NqvX6gmU2KKl6RbFBiEGmAme0FfMPdVwK4+4nu/nGaP74EOAWYXftFd18M\n9DCzXbIarEgWKTFIyUsurbDMzP7bzF43s7+Y2bbAGdRaasHM3jSzjumc092Xufs/Gnj7KeC0lkcu\nEg0lBpFgH+BWd98P+Bj4f4QVa2svuvbVMgFmNju5U1rdxzFpfNZ84MhsBi+STXmzVpJIzN529xeT\nz/+bsH7/twgbvnyNu7fkxv4OYfcxkbykxCAS1F40zAiL0EEDtWozex7Yvp63fubuM5v4LKM49xGR\nIqHEIBLsYWZ93f0lwhaZc4BtgV2pZ+16dz8ig3PX3STlm8BbzQ1UJGrqYxAJlgM/MbPXgZ0I24TO\nAXrVKpONKRwLAAAAf0lEQVTJznqnJJdA7gs8aWZP1Xq7D3VGK4nkEy27LSXPzLoBT7j7AXVe7w7c\n7O4nZvnzEsAP3X1dNs8rki2qMYgEX/uGlJy/8Elyz+CsSG7HWqWkIPlMNQYREUmhGoOIiKRQYhAR\nkRRKDCIikkKJQUREUigxiIhICiUGERFJ8X8UyJrT1OdTiQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def gini(p):\n", " return (p)*(1 - (p)) + (1-p)*(1 - (1-p))\n", "\n", "x = np.arange(0.0, 1.0, 0.01)\n", "plt.plot(x, gini(x))\n", "plt.ylim([0,1.1])\n", "plt.xlabel('p(i=1)')\n", "plt.axhline(y=0.5, linewidth=1, color='k', linestyle='--')\n", "plt.ylabel('Gini Impurity')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Misclassification Error\n", "\n", "$$I_M(t) = 1 - max\\{{p_i}\\}$$\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH+dJREFUeJzt3XmUVOWZx/HvQ4sGQgCXuKEOLujQBJcYEDWGchsRjJ6I\na4zjBPdE49Exbpkz9MTEqDMT921wSeLEoKKjTVRQooXGKAgqioBC1DngFjEhMmKQhmf+eG9DVVnd\nfau7bt1afp9z6nQtt2493NPU0+/6mLsjIiLSrlfaAYiISHVRYhARkTxKDCIikkeJQURE8igxiIhI\nHiUGERHJs1HaAcRhZppTKyLSDe5upb6nZloM7q6bOxMnTkw9hmq56VroWuhadH7rrppJDCIiUhlK\nDCIikkeJocZkMpm0Q6gauhYb6FpsoGvRc9aTfqhKMTOvhThFRKqJmeH1PPgsIiKVocQgIiJ5lBhE\nRCSPEoOIiORRYhARkTxKDCIikkeJQURE8igxiIhIHiUGERHJk2hiMLM7zewDM3u1k2OuN7PFZjbP\nzPZKMh4REela0i2Gu4AxHb1oZmOBXdx9CHAGcEvC8YiISBcSTQzu/gzwl04OORL4ZXTsLGCgmW2V\nZEwiItK5tMcYBgFLcx4vA7ZLKRYRESH9xABQuPOftlEVEUlR2jWf3wG2z3m8XfTc57S0tKy/n8lk\ntOe6iEiBbDZLNpvt8XkSr8dgZoOBqe4+vMhrY4Fz3H2smY0CrnX3UUWOUz0GEZESdbceQ6ItBjP7\nDTAa2MLMlgITgd4A7n6buz9qZmPNbAnwCfDdJOMREZGuqYKbiEidUgU3EREpCyUGERHJo8QgIiJ5\nlBhERCSPEoOIiORRYhARkTxKDCIikkeJQURE8igxiIhIHiUGERHJo8QgIiJ5lBhERCSPEoOIiORR\nYhARkTxKDCIikkeJQURE8igxiIhIHiUGERHJo8QgIiJ5lBhERCSPEoOIiORRYhARkTxKDCIikkeJ\nQURE8igxiIhIHiUGERHJo8QgIiJ5lBhERCSPEoOIiORRYhARkTxKDCIikkeJQURE8iSaGMxsjJkt\nMrPFZnZxkde3MLNpZvaymc03s39KMh4REemauXvHL5r1Aka5+x9KPrFZE/A6cAjwDvACcKK7L8w5\npgXYxN0vNbMtouO3cve2gnN5Z3GKiMjnmRnubqW+r9MWg7uvA27uZkwjgSXu/ra7rwEmA0cVHPMe\n0D+63x/4qDApiIhIZcXpSpphZseYWalZZxCwNOfxsui5XJOAYWb2LjAPOK/EzxARkTLbKMYxZwEX\nAGvN7G/Rc+7u/Tt5D0Ccvp/LgJfdPWNmOwNPmNke7r6y8MCWlpb19zOZDJlMJsbpRUQaRzabJZvN\n9vg8nY4x9OjEZqOAFncfEz2+FFjn7lflHPMo8FN3fzZ6/DvgYnefU3AujTGIiJSou2MMcVoMmNlR\nwDcIrYCZ7j41xtvmAEPMbDDwLnA8cGLBMYsIg9PPmtlWwG7Am7EiFxGRRHSZGMzsSmAE8GvAgB+Y\n2X7ufmln73P3NjM7B5gONAF3uPtCMzszev024ArgLjObRxjvuMjd/9yjf5GIiPRIl11JZvYqsKe7\nr40eNxHGBYZXIL72GNSVJCJSokSmq0YcGJjzeCDxBpZFRKQGxRlj+Bnwopk9RehKGg1ckmhUIiKS\nmk4TQ7TyeR2wL2GcwYFL3P29CsQmIiIpiDPGMNfd965QPB3FoDEGEZESdXeMIU5iuBJYDtwLfNL+\nfCVnDykxiIiULsnE8DafH2x2d9+p1A/rLiUGEZHSJZIYojGGY9393p4E11NKDCIipUtyd9WLuh2V\niIjUHI0xiIjUqUqPMeDuO5b6Yd2lxCAiUrrEEkM1UGIQESld2ccYzOyinPvHFrx2RakfJCIitaGz\nwefcLbIvK3jt8ARiERGRKhBnEz0REWkgSgwiIpKnw8FnM1sLrIoe9gE+zXm5j7vHqv5WDhp8FhEp\nXdlLe7p7U89CEhGRWqSuJBERyaPEICIieZQYREQkjxKDiIjk6TIxmNl4M1tsZh+b2cro9nElghMR\nkcqLs4neH4Ej3H1hZUIqGoOmq4qIlCiRegyR99NMCiIiUllxFqnNMbN7gYeAz6Ln3N0fTC4sERFJ\nS5zEMICw6vkfCp5XYhARqUOqxyAiUqcSG2Mws+3N7H/M7MPo9oCZbde9MEVEpNrFGXy+C2gFto1u\nU6PnRESkDsWZrjrP3ffo6rkkqStJRKR0SU5X/cjMTjazJjPbyMy+AywvPUQREakFcRLDBOA44H3g\nPeBY4LtJBiUiIulJdFaSmY0BrgWagNvd/aoix2SAa4DewHJ3zxQ5Rl1JIiIl6m5XUmcV3C5296vM\n7IYiL7u7/6CLgJqA14FDgHeAF4ATc1dRm9lA4FngMHdfZmZbuPvnuqmUGERESlf2Cm7AgujnXCD3\nW9kKHndkJLDE3d+OApwMHAXkbq/xbeABd18GUCwpiIhIZXVW2nNqdHeVu9+X+5qZHRfj3IOApTmP\nlwH7FBwzBOhtZk8BXwKuc/e7Y5xbREQSEmdLjEuB+2I8VyhOq6I38FXgYKAv8JyZPe/uiwsPbGlp\nWX8/k8mQyWRinF5EpHFks1my2WyPz9PZGMPhwFjgeGAyoQsJwl/2ze4+stMTm40CWtx9TPT4UmBd\n7gC0mV0M9HH3lujx7cA0d59ScC6NMYiIlCiJdQzvEsYX/hb9bL+1AofFOPccYIiZDTazjQkJprXg\nmIeBr0drJPoSupoWICIiqelsjGEeMM/M7nH3zzo6rpP3t5nZOcB0wnTVO9x9oZmdGb1+m7svMrNp\nwCvAOmCSuysxiIikKM6WGLsCVwDNQJ/oaXf3nRKOLTcGdSWJiJQoyS0x7gJuBdqADPBL4NelfpCI\niNSGOImhj7vPILQu/jcaKB6XbFgiIpKWONNV/xatYl4SjRm8C3wx2bBERCQtccYYRhJWKw8ELgf6\nA1e7+/PJh7c+Bo0xiIiUqOx7JVUTJQYRkdIlWdrziWizu/bHm5rZ9FI/SEREakOcwecvu/uK9gfu\n/hdgq+RCEhGRNMVJDGvN7O/aH5jZYMJiNBERqUNxZiX9CHjGzJ6OHn8DOCO5kEREJE2xBp/N7MvA\nKMKOqc9Xum6CBp9FREqXRAW3odHeRnsTEkL7yR3A3V/sbrClUmIQESldEolhkrufbmZZitRWcPcD\nS46ym5QYRERKl0RpzyeinxPc/c3uhSUiIrWms1lJl0Q/p3RyjIiI1JnOupJmELqQRgDPFLzs7n5k\nwrHlxqKuJKk67rByJfTvn3YkIsV1tysJdy96AzYhzERaAowmbLndfhvd0fuSuBES1OduEydO9GIm\nTpyo43V8oscvWuR+6KHuG2/svu++6cej43V8seMB925858bZRO/L7v5hpwclTC0GqRarVsFPfwq3\n3QaXXQbHHx9+Pvkk/Od/wrHHgpX+95lIIpKYlXSdu59nZlOLvOyuriRpMK2t8IMfwKhRIQkMGrTh\ntWeege99D7bZBm68EXbdNb04RdolkRj2dve5ZpYp8rK7+8xSP6y7lBgkTW+9FRLC4sVw001w8MHF\nj1uzBm64Aa64As46K7Qk+vatbKwiucq+u6q7z41+ZttvwCvAnyuZFETSsno1/OQnMGIE7LsvzJvX\ncVIA6N0bLrggHLd4MQwbBlOLtbdFqlycMYYscCRhzcNc4EPgWXc/P/HoNsSgFoNU1PTpcM454cv9\n2mth8ODSzzFjBnz/+7DbbnDddbDjjmUPU6RTidVjAAa6+8fA0cCv3H0kcEipHyRSC5YtCwPIZ58d\nEsJDD3UvKQAccgi88kpobYwYEVofq1eXNVyRRMRJDE1mtg1wHPBI9Jz+fJe6smYN/Pu/w557QnMz\nvPYajBvX8/NusglceinMmRNuw4fD44/3/LwiSYqz7faPgemE7qPZZrYzsDjZsEQqZ+bM0OWz3Xbw\n3HMwZEj5P2Pw4ND6eOSRMDD9ta/Bz38ePlOk2qjmszSs99+HH/4wJIZrroGjj67MGoRPP4Wf/Qxu\nvhkuuQTOOy8MXIuUW5I1n682s/5m1tvMfmdmy83s5O6FKZK+trYwrXT4cNh2W1iwAMaPr9zCtD59\n4Mc/Dq2TGTNgr73g6ae7fp9IpcSZlTTP3fcws28BRwAXAM+4++6VCDCKQS0GKYvnnw8DywMHhjUJ\nzc3pxuMODz4I558PmUwY59hKFdWlTJKcldQ+DnEEMMXd/4oGn6XGLF8Op58euot++MOwhUXaSQFC\nK2X8+NBq2WYb+MpXwsrptWvTjkwaWZzEMNXMFgF7A78zsy2BvyUblkh5rFsHkyaF9Qh9+8LChfDt\nb1fffkb9+sFVV0E2C1OmhOmtzz+fdlTSqOLWfN4cWOHua83si8CX3P39xKPb8PnqSpKSvfhi2L+o\nV68w0LvnnmlHFI873HNPaNmMGwdXXgmbb552VFKLkuxKAtgWGG9mpwDjgX8o9YNEKmXFCjj3XBg7\nFs44A37/+9pJChBaMyedFFo3ffuGLq/bbw+tH5FKiDMrqQW4HriRUIvhasIWGV0yszFmtsjMFpvZ\nxZ0cN8LM2szs6FhRixThDnffHb5I16wJ/fYTJoQWQy0aMCBspTF9Otx5J+y3X2gFiSQtzqyk+cAe\nwIvR7KStgF+7e6fbYphZE/A6YfuMd4AXgBPdfWGR454AVgF3ufsDRc6lriTp1Pz5YZHaJ5+EbqOR\nI9OOqLzWrYNf/CLs2HrssXD55WFmlUhnkuxK+tTd1wJtZjYA+BOwfYz3jQSWuPvb7r4GmAwcVeS4\ncwl1pVMtBiS1aeVKuPBCOOggOOEEmDWr/pIChFbPhAlhq47PPoOhQ0PrSH8vSRLiJIYXzGxTYBIw\nB3gJ+EOM9w0CluY8XhY9t56ZDSIki1uip/RrLrG4w333hW6j5ctDi+Hss6GpKe3IkrX55qF63MMP\nh03+Mpnwbxcppy73SnL370V3bzWz6UB/d58X49xxvuSvBS5xdzczA6psEqFUo9dfD1tif/BBmL1z\nwAFpR1R5I0fC7NkhSRx4IJxyCkycCF/6UtqRST3oMDGY2d508OVuZl91966Gwd4hv8tpe0KrIdfe\nwOSQE9gCONzM1rh7a+HJWlpa1t/PZDJkMpkuPl7qzapVoTrarbeGvvZzz23sPYaamsJ03GOOgYsu\nCq2nn/88PK62dRpSGdlslmw22+PzdFbaM0snf/W7+4GdnthsI8Lg88HAu8Bsigw+5xx/FzDV3R8s\n8poGnxtca2vYbG6ffT5fb1kC1Z2WQt0dfO6wxeDumZ4E5O5tZnYOYcvuJuAOd19oZmdGr9/Wk/NL\nY8ittzxpUih+I8UdcECYznrDDWFqq+pOS3fFma76feAed/9L9HhTwl/+N1cgvvYY1GJoMKtXw9VX\nh3n8//zPoZbyJpukHVXteOedcN1mzYLrr4dvfjPtiCQN3W0xxN5dteC5l929YmtJlRgay+OPh8Hl\n5ubu11uWQHWnG1uS6xh6mdn646IFaQ085CdJaa+3fNZZoXBOT+otS9Bed3rUqFA1TnWnJY44iWE6\nYebQwWZ2CGGh2rRkw5JGkltveejQ8tVblmCTTcJYw9y58MILqjstXYvTldQEnEGYXQRh+4rbo9XQ\nFaGupPo1c2aYSbPDDmHQdJdd0o6o/v32t2FAX3Wn619iYwwFH7IZsH3MBW5lo8RQf9KqtyyB6k43\nhiRrPs+Maj5vBswFJpnZNd0JUqStLcyxT6vesgSqOy2didOV9LK772lmpxFaCxPN7FV3H16ZENVi\nqBft9ZYHDAj1locNSzsigfy606NHh/GerbdOOyophyRnJTWZ2TbAccAj0XP6lpbYcustX3ghPPWU\nkkI1ya07ve22oTWnutONLU5i+DFhZtIf3X22me0MLE42LKkHxeotn3SSuo2qVXvd6Zkz4YEHVHe6\nkZU0+JwWdSXVnpdeCrONzGqr3rIEuXWnjzgiDFSr7nTtKXtXUnspTjO7wcyuj362367vSbBSv9rr\nLY8ZA6edVnv1liXIrTv9hS+o7nSj6awraUH0cy7wYvQz9yayXrF6y6eeWrv1liUYMCDstTRtGtxx\nh+pONwp1JUmP1Xu9ZQnWrYO77tpQd/onP1Hd6WpX9gVuZjaVMPuo2End3Y8s9cO6S4mhOq1cCf/2\nb/CrX4WfZ5xR/6U1BT76KCSH1tYwWH3yyZpQUK2SSAwfEiqu/QaY1f509NPdfWZ3Au0OJYbq4g73\n3x+2dT744LA99pZbph2VVNrs2WFdSr9+YV3KV76SdkRSKInEsBFwKHAiMJywhuE37v5aTwLtDiWG\n6vH662Fw+f33Q7fR17+edkSSprVrQ93piRNVd7oalX1Wkru3uftj7v6PwChgCTAzqsomDWbVKvjR\nj2D//cOMo7lzlRRkQ93p+fPhww/D5IP77gutSqldnQ4+m9kXgHHACcBgoBW4093fqUh0G+JQiyFF\nra1hN85Ro1RvWTqnutPVJYmupLuBYcCjwL3u/mrPQuw+JYZ05NZbvvFG1VuWeNasCVuoX3GF6k6n\nLYm9kk4ChgDnAX8ws5U5t4+7G6hUv9Wrw1TEESPCvPV585QUJL7evUON7nnzYMmSsCXK1KlpRyWl\n0DoGyTN9eqi3PGyY6i1LeTzxRPidUt3pyktyd1VpAO31ls8+OyQE1VuWcjn00A11p0eMUN3pWqDE\n0OBy6y03N6vesiSjve70nDmqO10L1JXUwGbODFtZbLddGCwcMiTtiKRRqO50ZagrSWJ7//2wjcHJ\nJ4etLB57TElBKuuII0Lr9O//PrRW/+M/QutVqoMSQwNpawstA9VblmqQW3f6iSdUd7qaqCupQeTW\nW7755jCeIFIt3EPVuAsugEwmjHtttVXaUdU+dSVJUcXqLSspSLUxg2OOCa3YrbcOG/Kp7nR6lBjq\nVG695T59VG9ZakO/fmG33mwWpkxR3em0qCupDr34Ytivplcv1VuW2pVbd3rcOLjyStWdLpW6koQV\nK8IK08MPD91HqrcstSy37nTfvqELdNIk1Z2uBCWGOuAeqqgNHRpmHi1cqHrLUj8GDAhbaUyfHkqL\nqu508hL/6jCzMWa2yMwWm9nFRV4/yczmmdkrZvasme2edEz1ZP78MIvj+uvD9ti33gqbbZZ2VCLl\nt+eeoRV8xhkwdmwoGLViRdpR1adEE4OZNQE3AmOAZuBEMxtacNibwDfcfXfgcuC/koypXqxcGfpe\nDzoITjgBZs0KA3Ui9axXL5gwISyO++yz0L10990qDFRuSbcYRgJL3P1td18DTAaOyj3A3Z9z979G\nD2cBWhzfCfdQIau5OVTMmj8/rE9oako7MpHK2XzzUFL0oYfCpo+ZTPi/IOWRdGIYBCzNebwseq4j\npxIKA0kRb7wBhx0Wdqe85x74xS9gyy3TjkokPSNHwuzZcPzxcOCBYa3OypVpR1X7Nkr4/LEbeGZ2\nIDAB2L/Y6y0tLevvZzIZMplMD0OrHatWhWpYt94adqg899xQDEVENtSdPuaY0L3a3Bw25jvmmMZb\nt5PNZslmsz0+T6LrGMxsFNDi7mOix5cC69z9qoLjdgceBMa4+5Ii52nYdQxTp4ZdKPfZR/WWReJQ\n3ekNqnUdwxxgiJkNNrONgeOB1twDzGwHQlL4TrGk0Kjeegu++c3QNJ40CSZPVlIQieOAA8J01jFj\nwtTWf/mX0OqW+BJNDO7eBpwDTAcWAPe6+0IzO9PMzowO+1dgU+AWM3vJzGYnGVO1W70aLr98Q73l\nV15RvWWRUhWrO93a2vX7JNCWGFVE9ZZFkjFjRihKteuuYc1Po9SdrtauJIlB9ZZFknXIIaH1ve++\nqjsdhxJDinLrLQ8dqnrLIklS3en41JWUkpkzw8yJHXYIVdV22SXtiEQaSyPUnVZXUo3Irbd8+eXw\n6KNKCiJpUN3pjikxVMjatWFOdW695aOPbrwFOCLVJLfu9IwZqjvdTl1JFaB6yyLVr73u9Pnnb6g7\nvfXWaUfVM+pKqkKqtyxSO9rrTi9cGFZNDx8exv8ase60EkMCcust9+2ressitSS37vQDDzRm3Wl1\nJZWZ6i2L1I9arzutrqSUrVgRdj0dOzZUmFK9ZZHaV1h3etgwuP32+q87rcTQQ7n1ltesCbONJkxQ\nvWWRetJed3raNLjzTth/f3jppbSjSo66knpg/vyw/8r//R/ccksoGiIi9W3dulAk67LLwlY2l18O\nAwemHVVx6kqqoJUrwyyjAw8MlaNmz1ZSEGkUjVB3WomhBIX1ll97LQw0q96ySOOp57rT6kqK6Y03\nwpbY770XZhsdcECq4YhIFVm7NiSJlhY45RSYODFMe02bupISsmpVqAC1336hItSLLyopiEi+9rrT\nr74Kf/pTmIxy//21272kFkMnWlvhvPNUb1lESlMtdafVYiij9nrLF10U5iyr3rKIlKLW604rMeRY\nvTpUdmqvtzxvHhx8cNpRiUgtKlZ3eurUtKOKR11JkccfD4PLzc2qtywi5dded3q33cJiuUrUnVZX\nUje111s+6yy45hrVWxaRZLTXnR41qvrrTjdsYsitt9zcrHrLIpK83LrTc+ZUb93phuxKUr1lEakG\njzwSNt9Mqu60upJiUL1lEakm48ZVZ93phkgMbW2hZaB6yyJSbYrVnZ45M92Y6r4rqb3e8sCBcNNN\nKq0pItXLHR58MNSdHj2653Wn1ZVUYPlyOO00GD8+7IT65JNKCiJS3czCd9aCBaF3Y/jwsHK60nWn\n6y4x5NZb7tcvXGDVWxaRWtKvH1x1VehSmjKl8nWn66orSfWWRaTe9KTudEN3Ja1YEVYtq96yiNSb\nwrrTzc2hVyTJutM1nRgK6y2/9prqLYtIfSqsO73ffqGXJAmJfoWa2RgzW2Rmi83s4g6OuT56fZ6Z\n7RX33PPnh1H7666Dhx8ORTLiNq9ERGrVXnvBs8/C6afD4YeH3pIVK8r7GYklBjNrAm4ExgDNwIlm\nNrTgmLHALu4+BDgDuKWr8+bWWz7hhMart5zNZtMOoWroWmyga7FBI1yLXr3g1FPD5Jo1a0KvSTnr\nTifZYhgJLHH3t919DTAZOKrgmCOBXwK4+yxgoJltVexk7qEiUnNzmIraqPWWG+GXPi5diw10LTZo\npGvRXnf64YfDrtCHHlqeqa0b9fwUHRoELM15vAzYJ8Yx2wEfFJ7ssMNCveV77lFpTRGRXCNHht6T\nWbPK88dyki2GuI2awqlURd+nessiIh1ragoD0uWQ2DoGMxsFtLj7mOjxpcA6d78q55hbgay7T44e\nLwJGu/sHBeeq/sUWIiJVqDvrGJLsSpoDDDGzwcC7wPHAiQXHtALnAJOjRLKiMClA9/5hIiLSPYkl\nBndvM7NzgOlAE3CHuy80szOj129z90fNbKyZLQE+Ab6bVDwiIhJPTWyJISIilVNVa4STXBBXa7q6\nFmZ2UnQNXjGzZ81s9zTirIQ4vxfRcSPMrM3Mjq5kfJUS8/9HxsxeMrP5ZpatcIgVE+P/xxZmNs3M\nXo6uxT+lEGZFmNmdZvaBmb3ayTGlfW+6e1XcCN1NS4DBQG/gZWBowTFjgUej+/sAz6cdd4rXYl9g\nQHR/TCNfi5zjngR+C4xPO+6UficGAq8B20WPt0g77hSvRQvws/brAHwEbJR27AldjwOAvYBXO3i9\n5O/NamoxlHVBXI3r8lq4+3Pu/tfo4SzC+o96FOf3AuBcYArwYSWDq6A41+HbwAPuvgzA3ZdXOMZK\niXMt3gP6R/f7Ax+5e1sFY6wYd38G+Esnh5T8vVlNiaHYYrdBMY6pxy/EONci16nAo4lGlJ4ur4WZ\nDSJ8MbRvqVKPA2dxfieGAJuZ2VNmNsfMTq5YdJUV51pMAoaZ2bvAPOC8CsVWjUr+3kxyumqpyrog\nrsbF/jeZ2YHABGD/5MJJVZxrcS1wibu7mRmf/x2pB3GuQ2/gq8DBQF/gOTN73t0XJxpZ5cW5FpcB\nL7t7xsx2Bp4wsz3cfWXCsVWrkr43qykxvANsn/N4e0Jm6+yY7aLn6k2ca0E04DwJGOPunTUla1mc\na7E3YS0MhP7kw81sjbu3VibEiohzHZYCy939U+BTM3sa2AOot8QQ51rsB/wUwN3/aGZvAbsR1lc1\nmpK/N6upK2n9gjgz25iwIK7wP3Yr8I+wfmV10QVxdaDLa2FmOwAPAt9x9yUpxFgpXV4Ld9/J3Xd0\n9x0J4wxn11lSgHj/Px4Gvm5mTWbWlzDQuKDCcVZCnGuxCDgEIOpP3w14s6JRVo+SvzerpsXgWhC3\nXpxrAfwrsClwS/SX8hp3r7sNyGNei7oX8//HIjObBrwCrAMmuXvdJYaYvxNXAHeZ2TzCH8AXufuf\nUws6QWb2G2A0sIWZLQUmEroVu/29qQVuIiKSp5q6kkREpAooMYiISB4lBhERyaPEICIieZQYREQk\njxKDiIjkUWIQ6YSZ3WtmO0X3HzGz/l29Jzr2WDN7zczWmtlXc57f3czuSCpekXJQYhDpgJntAnzR\n3d8EcPdx7v5xzLe/CnwLeDr3SXd/BdjZzLYsa7AiZaTEIA0v2lphkZn9t5ktMLP7zawPcAI5Wy2Y\n2dtmtlmcc7r7Ind/o4OXHwOO7XnkIslQYhAJdgVucvdm4GPge4Qda3M3XVu/TYCZPR1VSiu8HRTj\ns2YD3yhn8CLlVDV7JYmkbKm7Pxfd/2/C/v1/Ryj48jnu3pMv9vcI1cdEqpISg0iQu2mYETahgw5a\n1Wb2DNCvyEsXuvvvuvgsoz7riEidUGIQCXYws1Hu/jyhRObvgT7A1hTZu97dDyjh3IVFUrYB/re7\ngYokTWMMIsHrwPfNbAEwgFAm9PfA13KOKaWy3reiLZBHAY+Y2WM5L4+kYLaSSDXRttvS8MxsMDDV\n3YcXPL8TcIO7jyvz52WB49z9T+U8r0i5qMUgEnzuL6Ro/cLKqGZwWUTlWJcoKUg1U4tBRETyqMUg\nIiJ5lBhERCSPEoOIiORRYhARkTxKDCIikkeJQURE8vw/cdS2PfLTuCMAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def error(p):\n", " return 1 - np.max([p, 1-p])\n", "\n", "x = np.arange(0.0, 1.0, 0.01)\n", "err = [error(i) for i in x]\n", "plt.plot(x, err)\n", "plt.ylim([0,1.1])\n", "plt.xlabel('p(i=1)')\n", "plt.axhline(y=0.5, linewidth=1, color='k', linestyle='--')\n", "plt.ylabel('Misclassification Error')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Comparison" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbcAAAE4CAYAAADYcb3mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmcjdUfwPHPMcwY+77vJGtEdqLInl3IVqmklKWktNBe\npNBCsmWIKLKUZBtk32ks2XfGPoPBLN/fH4f5DWbGYO597tz5vl+v++LeeeZ5vnPnzvN9znnO+R4j\nIiillFLeJIXTASillFKJTZObUkopr6PJTSmllNfR5KaUUsrraHJTSinldVIm9g6NMX4ZMmQIvHjx\nYuWoqChNnkoppRJVihQpotKlS7c2JCSkjohcjW2bRE9uadKk+ah69eoPzZgxI4W/v39i714ppVQy\nFxYWlqJly5YPLV++/EOgf2zbmMSe55YmTZqQf//9N32RIkUSdb9KKaXUDXv37qVs2bIhly9fzhjb\n1xM9uRljosLDw03KlIneKFRKKaUAiIiIIFWqVCIisd7+ckVyE616opRSytWMMYiIie1rOuBDKaWU\n19HkppKMHj168PHHHyf6tk4rU6YMy5YtczqMJO9+f+cTJkygVq1aiRjRzRo3bkxAQED083fffZfs\n2bOTJ08eDh8+TPr06XFFr1f69Ok5cOBAou/X44lIoj7sLj1PwYIFxd/fX9KlSxf9ePXVV+/4fbVr\n15YxY8a4IUI1ZcoUqVy5sqRNm1Zy5MghVapUke+//z7RjzN+/HipWbNmou83MQwcOFA6derkdBge\npWDBguLr6yunT5++6fXy5cuLMUYOHjyYKMdx5+fi4MGD4u/vf9vPdL/ceb6613NqYrqeb2LNRcmm\n5WaMYe7cuYSGhkY/RowYkaDvi09ERERihZisDR06lN69e9O/f39OnjzJyZMnGTVqFCtWrODatWtO\nh+cS+tlJGGMMRYoUYcqUKdGvbdu2jbCwsDv+fXqqQ4cOkTVrVrJmzZqo+3Xn+3E359TIyMjbXouK\nirqr493t9smm5VaoUCFZtGjRba+PHz9eatSoIW+88YZkzpxZChcuLPPmzRMRkQEDBoiPj4+kTp36\npqsSY4x89913UqxYMSlSpIiIiIwePVqKFSsmWbJkkWbNmsmxY8eij2GMkREjRkiRIkUkW7Zs0q9f\nP4mKipKrV69K5syZZdu2bdHbnjx5UtKkSZPoV3Se7Pz585I2bVqZMWNGvNt17dpV3n33XRERWbJk\nieTNm1eGDh0qOXLkkNy5c8v48eNj3fZWt16hFyxYUIYMGSJly5aVdOnSyXPPPScnTpyQhg0bSoYM\nGaRevXpy7tw5ERHZv3+/GGNk9OjRkidPHsmdO7d8+eWXcR53yZIlki9fvpuO9cUXX0jZsmUlderU\nEhERIQULFpSFCxfKvHnzxNfXV1KlSiXp0qWT8uXLy/Tp06VixYo3xT906FBp3rz5Hd5V71GoUCH5\n+OOPpVKlStGvvf766/LJJ5/c1HKL+d6fOnVKmjRpIpkyZZIsWbJIrVq1JCoqSkREDh06JC1btpTs\n2bNL1qxZpWfPniJy++fitddek/z580uGDBmkYsWKsnz58uivrVmzRipWrCgZMmSQnDlzSt++fUVE\nJCwsTDp27ChZs2aVTJkySaVKlSQ4OFhE/t+qWrhwofj7+0uKFCkkXbp08uyzz0Z/riIjI0VE5MyZ\nM/LMM89Injx5JHPmzNKiRQsRETl79qw0adJEsmfPLpkzZ5amTZvKkSNHRCT+89XevXtFxP6tde7c\nWbJnzy4FCxaUjz/+OPp9ie9cGNfvJbZz6o19Va9eXfr06SNZs2aVd999V5555hl56aWXpFGjRpI2\nbVpZtGiRbN++XWrXri2ZMmWS0qVLy+zZs6P30bVr19u2vxXacrMkjv7stWvXUqJECc6cOcObb75J\nt27dAPjkk0+oVasW33333W1XJbNmzWLdunVs376dxYsXM2DAAKZPn87x48cpWLAg7du3v+kYv//+\nOxs2bGDjxo3MmjWLcePG4evrS4cOHZg0aVL0dlOmTKFevXqJfkXnyVatWsXVq1dp3rx5vNsZY266\nMj158iQhISEcO3aMsWPH8sorr3DhwoVYt73TfmfMmMGiRYvYtWsXc+fOpVGjRnz++ecEBwcTFRV1\n2xVpYGAge/bs4e+//+aLL75g0aJFCT7u1KlTmTdvHufPn8fHxyf6exo2bMiAAQNo3749oaGhbNq0\niWbNmrF//3527twZ/f0BAQF07do1QT+bt6hatSohISHs3LmTyMhIfvnlFzp16nTTNjHf+6FDh5I/\nf35Onz5NcHAwn332GcYYIiMjadq0KYULF+bgwYMcPXqUDh06xHrMypUrs2XLFs6dO8fTTz9N27Zt\no3sRevXqRZ8+fbhw4QL79u2jXbt2APz000+EhIRw5MgRzp49yw8//EDq1Klviq9u3brMmzePPHny\nEBoayrhx4247dufOnbly5Qrbt28nODiYvn37AvYc1q1bNw4dOsShQ4fw9/enZ8+eQPznqxteffVV\nQkND2b9/P0uXLmXixImMHz8++utxnQvjEtc59ca+ihYtSnBwMO+88w4iwpQpU3jvvfe4ePEilSpV\n4sknn6Rhw4acOnWKb775ho4dO/Lff/9F7yPm9jVq1Ig3llu5PbkZkziPuyUitGjRgsyZM0c/xowZ\nA0DBggXp1q0bxhi6dOnC8ePHCQ4Ovul7b/X222+TKVMm/Pz8mDx5Mt26daN8+fL4+vry2WefsWrV\nKg4dOhS9ff/+/cmUKRP58+end+/e0V0sXbp0uam7JSAggM6dO9/9D5gIzAcmUR536/Tp02TLlo0U\nKf7/caxevTqZM2cmTZo0/PPPP9Gvx/xdpEqVivfffx8fHx8aNWpEunTp2LVrV6zb3smrr74afXO/\nVq1aVKtWjXLlyuHn50fLli3ZtGnTTdsPHDgQf39/ypQpw7PPPnvT7zC+4xpjeO2118ibNy9+fn63\nfV3+3wMCgK+vL0899VT0BVBQUBAHDx6kadOmCf7ZEoVTf7gxdO7cmYkTJ7JgwQJKlSpF3rx549zW\n19eX48ePc+DAAXx8fKJPjGvXruX48eMMGTIEf39//Pz8qF69eqz76NixI5kzZyZFihT07duXq1ev\nRn++fH192b17N6dPnyZNmjRUrlw5+vUzZ86we/dujDE8/PDDpE+f/rZ9x/cZOX78OH/99RejRo0i\nY8aMpEyZMnqgS5YsWWjZsiWpU6cmXbp0DBgwgKVLlyZo3zcuCj777DPSpk1LwYIFef31128a5HKn\nc+Gtx7n1nDp27Njor+fJk4dXXnmFFClSkDp1aowxtGjRgmrVqgGwefNmLl26xFtvvUXKlCl57LHH\naNq06U1/SzG3j+3vJT5un2nt1BQ4YwyzZs3i8ccfv+n1CRMmkCtXrujnadKkAeDixYvkyJEj+ntv\nlT9//uj/Hz9+nEceeST6edq0acmaNStHjx6lQIECt21foEABjh07BkCVKlXw9/cnMDCQXLlysXfv\nXpo1a3a/P+49kYHO/HKyZs3K6dOniYqKik5wK1euBOz7Fldfe9asWW9KiGnSpOHixYv3FEPOnDmj\n/+/v73/T89SpU9+231t/n9u2bUvwsWJ+b0J07dqVp59+mo8//piAgADatWtHqlSp7mof983huavG\nGDp37kytWrXYv38/Xbp0ifUkfuO1fv36MWjQIOrXrw/Aiy++SP/+/Tl8+DAFCxa86XMTly+//JJx\n48Zx7NgxjDGEhIRw+vRpAMaOHcv7779PyZIlKVy4MAMHDqRJkyZ07tyZw4cP0759e86fP0+nTp34\n5JNPuJuiFocPHyZLlixkzHh74Y3Lly/Tp08f5s+fz7lz5wB7rhKR6PNUXD0Hp0+fJjw8nIIFC0a/\nVqBAAY4ePRr9/E7nwpjiOqfeENvnPF++fNH/P3bs2G3bFCxYMPrcaIy5afu7lay6Je9FXB+UmK/n\nyZPnpqG2ly5d4syZMzddWcZsxR06dOimr3Xt2pVJkyYREBBA27Zt8fX1TcSfwPNVq1YNPz8/fv/9\n9ztu664b5ndq9cX1+0ybNi2XL1+O/tqJEydu+974fobYvla1alV8fX1ZtmwZU6ZMcaxl77QCBQpQ\npEgR5s2bR6tWreLdNl26dHz55Zfs3buX2bNn89VXX7F48WIKFCjAoUOHYh3gENPy5csZMmQI06dP\n5/z585w7d46MGTNGfy6KFSvGzz//zKlTp+jfvz9t2rQhLCyMlClT8v777xMUFMTKlSuZO3cuEydO\nvKufM3/+/Jw9eza6iz2moUOH8t9//7F27VouXLjA0qVLb2rtx/fZypYtG6lSpbrpXHXo0KH7SiDx\niS2WW8+bhw8fvulv7eDBg/G2yO9Gskpud9NNdUPOnDnZu3dvvNt06NCB8ePHs2XLFq5evcqAAQOo\nWrVqdKsN7FXg+fPnOXz4MCNGjIjuowfo1KkTM2bMYPLkyXTp0uWuY0zqMmXKxMCBA3n55Zf57bff\nCA0NJSoqKrrb4oZbu+zicy+/67vx8ccfExYWRlBQEBMmTIj+fZYvX54///yTc+fOceLECYYNG3ZX\n+82VKxcHDhy4Lf7OnTvTs2dPfH194+xGSw7Gjh3L4sWLia0oe8z3bO7cuezZswcRIUOGDPj4+ODj\n40PlypXJnTs3b731FpcvX+bKlSvRvQQxhYaGkjJlSrJly8a1a9f48MMPCQkJif76pEmTOHXqFAAZ\nM2bEGEOKFClYsmQJ27ZtIzIykvTp05MqVSp8fHzu6mfMnTs3jRo14uWXX+b8+fOEh4ezfPlywLai\n/P39yZgxI2fPnuWDDz646XvjO1/5+Pjw1FNP8c4773Dx4kUOHjzI119/fdu9y7txN39nt25btWpV\n0qRJw+DBgwkPDycwMJC5c+dGj1e437/hZJXcnnzySdKnTx/9aNWqVawDAGI+79WrF7/++itZsmSh\nd+/ese63bt26fPTRR7Ru3Zo8efKwf/9+pk6detM2zZs3p2LFijz88MM0bdqU5557Lvpr+fPnp0KF\nCqRIkYKaNWsm4k+cdPTr14+vvvqKwYMHkytXLnLlysVLL73E4MGDo/vcb/1d3akFFF+r+04twFuP\nc+v2tWvXplixYtSrV49+/fpRr149wCahcuXKUahQIRo2bEj79u3vqrXZtm1bwHa5xuzq7ty5M0FB\nQfd1IvIGRYoUoUKFCtHP4/o97dmzhyeeeIL06dNTvXp1XnnlFWrXrk2KFCmYM2cOe/bsoUCBAuTP\nn59p06bd9v0NGzakYcOGFC9enEKFCuHv73/Txer8+fMpU6YM6dOnp0+fPkydOhU/Pz9OnjxJ27Zt\nyZgxI6VKlaJOnTpxtrTjO+8EBASQKlUqSpQoQc6cORk+fDgAvXv3JiwsjGzZslG9enUaNWp0V+er\nb775hrRp01KkSBFq1apFx44defbZZ2/7+eOK8Va3nlNbt24d775ivpYqVSrmzJnDvHnzyJ49Oz17\n9iQgIIDixYvHuY+7obUl3SBFihTs2bOH+FZK6NatG3nz5uXDDz90Y2Tqbh04cIAiRYoQERGRoPs2\niSUsLIycOXOyadMmihYt6rbjKuXJ4qstqaX7PcCBAweYMWMGmzdvdjoU5aFGjhxJ5cqVNbEplUCa\n3Nwgvqb1e++9x7BhwxgwYMBNo5iU53J3VYxChQphjEnQgBullKXdkkoppZIkXfJGKaVUspLoyc0Y\nI95a6FYppZRnuHbtGsaYOLsJEz25pUuX7r/BgwdHaIJTSinlCteuXWPw4MER6dKl+y+ubVxxzy1v\n+vTpF128eLF4XH2hSiml1L0yxki6dOn+Cw0NrSsiR2PdRgd/KKWU8jY6oEQppZTX0eSmlFLK62hy\nU0op5XU0uSmllPI6mtyUUkp5nSRRWzK+iXpKKaWSBndOD0syLbcbC1XqI+7HwIEDHY8hKTz0fdL3\nSd8n9z/cLckkN6WUUiqhNLkppZTyOprcvEidOnWcDiFJ0PcpYfR9Shh9nzxTkii/pWvEKaVU0hbf\n2muuoC03pZRSXkeTm1JKKa+jyU0ppZTX0eSmlFLK62hyU0op5XU0uSmllPI6mtyUUkp5HU1uSiml\nvI4mN6WUUl5Hk5tSSimvo8lNKaWU19HkppRSyuu4NLkZY8YZY04aY7bFs80IY8xuY8wWY8zDroxH\nKaVU8uDqltt4oGFcXzTGNAaKicgDwIvASBfHo5RSKhlwaXITkeXAuXg2aQb8dH3bNUAmY0xOV8ak\nlFLK+6V0+Ph5gcMxnh8B8gEnnQlHqcQXFQXnz8OFCxAaah+XLkF4OERE2H9FIGVK+0iVCvz9IX16\n+8iQAbJksa8rpRJIRFz6AAoB2+L42hygRoznC4EKsWwnsT0GDhwosRk4cKBur9u7bfv+/WPfPmfO\ngZI7t0jKlCIZMogUKCBSurRI3ryxb//ggwOlSROR+vVFatUSKV9epGhREX//2LevVWugjB4tsmiR\nyJEjIlFRnvn+6PbJc/uuXbvKwIEDox+AiIvzTcyHy1fiNsYUAuaISNlYvjYKCBSRqdef7wRqi8jJ\nW7YTV8ep1J1cuwY7dsDmzbBpE2zZYp9fvAjFi8MDD0ChQlCwIBQoAHnyQM6ckC0b+Pnd37EjI+Hc\nOQgOhuPH4fBhOHjQPvbuhV27bGuweHF46CEoX94+ypWDTJkS5cdX6r64eyVup5NbY6CniDQ2xlQF\nholI1Vi20+Sm3C44GJYvh9WrYdUqm9QKFrw5cZQqBXnzgnHbn2zczp+HnTth61abeDdvtv/Pnx+q\nVoVq1aBGDShZ0jPiVcmLVyU3Y8wUoDaQDXsfbSCQCkBEfri+zbfYEZWXgGdFZGMs+9HkplwuNBQW\nL/7/4/BhqF7dPqpVg8qV7T2wpCQiAv791ybnVatg2TK4cgUefxweewwaNLCtTKVczauSW2LR5KZc\nZc8emDMH/vwT1qyBKlWgbl178q9QwQ7w8Db798OSJbBoEfz9t+06bdLEPmrUAB8fpyNU3kiTWyw0\nuanEIgLbt8Nvv8Gvv8KpU9CsGTRubJNaunROR+hekZGwfr1N7nPmwNGj0KIFtG5tW3Y6QlMlFk1u\nsdDkpu7X4cPw888waZK9N9WmjX1UqwYptAhdtP37beL/7Tfbqn3qKejc2bZo9T6duh+a3GKhyU3d\niytX7El67Fg7uKJNG3uirlFDE1pC7N8PkydDQIBt4T3zDDz3nB0FqtTd0uQWC01u6m7s2AGjR9uT\ncoUK8PzztusxdWqnI0uaRGDdOnuRMG0a1K4NL75oB6Po/TmVUJrcYqHJTd1JVJS9bzR8OGzbBt26\n2UeRIk5H5l0uXoSpU2HUKDvv7tVXbWsuQwanI1OeTpNbLDS5qbhcvgzjx8PXX9vJyr162ftE9ztp\nWsVPxE4tGDHCjrjs3Bn69rXzAJWKjbuTm955UEnSuXPwySdQuDAsWAATJtius86dNbG5gzF2/t/U\nqXaiuJ+f7QLu0gWCgpyOTilNbiqJOXsW3nkHihWD3bvtfK3ff4eaNXU0n1Py5YPBg20ZsJIl7ZSK\nli1tlRSlnKLJTSUJ587B++/b2omnTsGGDba1VqqU05GpGzJlgrfftqMsa9eGhg3tCNV//3U6MpUc\naXJTHi0sDL74wia1I0dg7Vo7ErJQIacjU3Hx94feve08uapVoV496NQJDhxwOjKVnGhyUx4pMtIO\nFCle3JbF+ucfGDdORz8mJWnTwhtv2O7jokWhYkV4/XU4c8bpyFRyoMlNeZylS+3ghLFj4ZdfYMYM\nePBBp6NS9yp9evjgAzvQJCwMSpSwUzbCw52OTHkznQqgPMbBg9Cvn22pffmlvV+jg0S8z/btttvy\nyBEYNgzq13c6IuUOOhVAJTtXr8LHH9vWWunStsJI27aa2LxVqVIwfz589hn06GFHVh465HRUytto\nclOOCgy0i36uWWNHQA4cCGnSOB2VcjVjoHlz21X58MP2wmboULv+nFKJQbsllSPOnLGDCxYvtlUu\nmjfXllpytnu3bcWdPg0//giVKjkdkUps2i2pvN6MGVC2LGTMaK/cW7TQxJbcPfCArTTTrx80bQr9\n+9tVHZS6V9pyU24THAw9e9rKFePG2aVnlLrVyZO2IPPWrfZzUr260xGpxKAtN+WVZs+299YKFbJr\nq2liU3HJmdMurfPJJ3ZF8LfegmvXnI5KJTXaclMuFRoKffrYe2sTJ9oakEolVHAwvPCCHU05aZId\nTauSJm25Ka+xerUdCSdiW2ua2NTdypHDFsZ+5RWoU8dO/tbrXJUQ2nJTiS4qyk7CHjoURo6EVq2c\njkh5g717oUMHyJXLlmbLmtXpiNTd0JabStKCg6FxY5g1y66vpolNJZaiRW2N0eLFbY/A8uVOR6Q8\nmSY3lWiWL7eTcStUsJOzCxRwOiLlbXx9ba/AyJG2is1nn9meAqVupd2S6r6J2BqBn38OP/1k1/FS\nytWOHrUJLkcO+7nLmNHpiFR8tFtSJSmhodC+PUyebEtoaWJT7pI3r+0hyJ8fHnkEtm1zOiLlSTS5\nqXt2YzHKDBnsvRBdQFS5m68vfPMNDBoEjz9u58cpBdotqe7RokXw9NN2na6XXnI6GqXsdJMWLaBL\nF5vsUuilu0dxd7ekJjd1V0Tgu+/sEjVTp9q5R0p5iuBgO0I3e3YICIB06ZyOSN2g99yUx4qIgJdf\nhh9+gJUrNbEpz5Mjh+1VyJrV1qQ8fNjpiJRTNLmpBAkNhSefhAMHYMUKKFLE6YiUip2fn102p2tX\nqFYNNm50OiLlBE1u6o6OHLGlswoWhDlz7AASpTyZMXa9wOHDoUEDmDvX6YiUu2lyU/HautVe/Xbq\nZCfOpkzpdERKJVzr1vaC7IUX7OdXJR86oETFadkyO0l2xAho187paJS6d/v22Rbc00/bkZS6OK77\nedWAEmNMQ2PMTmPMbmNM/1i+ns0Y85cxZrMx5l9jzDOujEcl3O+/Q5s28PPPmthU0lekiL1X/Mcf\n0KMHREY6HZFyNZe13IwxPsAuoB5wFFgHdBCRHTG2GQT4icjbxphs17fPKSIRt+xLW25u9OOPMHCg\n7c6pWNHpaJRKPKGh0LKlLdU1eTKkTu10RMmHN7XcKgN7ROSAiIQDU4Hmt2xzHLgxPCEDcObWxKbc\na+hQ+PRTWLpUE5vyPunT29abj48d/XvpktMRKVdxZXLLC8ScZXLk+msx/QiUNsYcA7YAvVwYj4qH\niK02Mnq0vdf2wANOR6SUa/j5wZQpkC+fvQ934YLTESlXcGVyS0g/4gBgs4jkAcoD3xlj0rswJhUL\nEejfH377zSa2/Pmdjkgp1/LxgbFjoXx5qFcPzpxxOiKV2Fw5sPsoEPM0mR/beoupOvAJgIjsNcbs\nBx4E1t+6s0GDBkX/v06dOtTR8hiJQgRefRXWrrUV1rNkcToipdwjRQpbdPmtt2y1nUWLbIUTlTgC\nAwMJDAx07PiuHFCSEjtApC5wDFjL7QNKvgIuiMgHxpicwAbgIRE5e8u+dECJC4hAz562gsNff+l6\nWCp5ErEDqGbO1ATnSu4eUOKylpuIRBhjegLzAR9grIjsMMZ0v/71H4BPgfHGmC3YLtI3b01syjU0\nsSllGWPvNxsDdetqgvMWOok7GdLEptTtROwE7xkzNMG5gte03JRnEoHevWHDBpg/XxObUjcYY5Ob\niB1kovegkzZtuSUz77xjW2uLFkGmTE5Ho5TnuTF6ODAQFi7UQuGJRRcrjYUmt8Tx6ae2KsPSpZAt\nm9PRKOW5bnTdb91qLwbTpnU6oqRPk1ssNLndv+HD4dtv7Ty23LmdjkYpzxcVBc89B8eOwezZWqrr\nfmlyi4Umt/szcSK8955NbAULOh2NUklHRIRdSSAiAqZPt5O/1b3R5BYLTW737o8/oFs3e/+gRAmn\no1Eq6bl6FZo0gaJFYdQoXS7nXnlT4WTlsJUr4dlnYdYsTWxK3Ss/PzvBe8MGO9lbJQ06FcBLBQVB\nq1YQEABVqjgdjVJJW/r08OefULOmnf/Ws6fTEak70eTmhY4ehUaN7PI1DRo4HY1S3iFHDvj7b5vg\ncueG1q2djkjFR++5eZnQUKhVC9q3twVhlVKJa9Mme9E4axZUq+Z0NEmHDiiJhSa3hAkPh2bNoEAB\nvfGtlCv9+acdqLV8ORQr5nQ0SYMOKFH3RAReecX+/7vvNLEp5UqNG9tSXY0b61pwnkrvuXmJIUNg\n3To7ly2l/laVcrnu3WHfPmjRwpbp8vNzOiIVk3ZLeoHZs+Hll2H1asiXz+lolEo+oqKgbVtbf3Lc\nOO0xiY92S6q7snWr7fufMUMTm1LuliKFrQC0ebMdnaw8xz11YBljfEXkWmIHo+7OyZN2AMmIEVC5\nstPRKJU8pU1re0+qVrXFEpo2dToiBQlouRljlhpjCsd4XhlY79Ko1B1dvWonaXfpAh06OB2NUslb\n/vzw22+20PK//zodjYIE3HMzxjQAhgPfAHmBRkA3Edno+vCiY9B7brd48UU7Smv6dNs1opRy3uTJ\ntkTXunWQObPT0XgWj5znZox5DFgAnAIeFpETrg7sluNrcoth9Gi7hM3q1bYskFLKc/TpA7t2wZw5\nuopATB43oMQY8x621VYLGAQsNcZor7JDVq2Cd9+1hVw1sSnleQYPhrAwOw9OOSchA0qyApVEJAxY\nZYz5CxgDzHVpZOo2x4/bYcfjxkHx4k5Ho5SKTapU8MsvUKkSVKgALVs6HVHylNBuSX+ggIjscn1I\nsR4/2XdLhofD449DvXq67IZSScG6dbaCyfLluuQUeGa3ZDNgMzD/+vOHjTGzXR2Yutnbb9tuyPfe\nczoSpVRCVKoEn34KbdrApUtOR5P8JGS05EbgcWCJiDx8/bV/RaSMG+K7EUOybrn9/jv06gUbN0LW\nrE5Ho5RKKBHo2tVWLpkwIXlXMPG4lhsQLiLnb3ktyhXBqNvt22eH/f/yiyY2pZIaY2DkSLuK99ix\nTkeTvCRkQEmQMaYjkNIY8wDwGrDStWEpgCtX7ACSd96x1Q+UUklP2rR2Puqjj8Ijj0D58k5HlDwk\npOX2KlCrgWW0AAAgAElEQVQauApMAUKA3q4MSlmvvw6FC8NrrzkdiVLqfpQsacvktW1rFxRWrqer\nAniomTNtctu0CTJmdDoapVRiePFF2yMzcaLTkbifx1QoMcbMifFUgJhBiYg0c2Vgt8SSrJLb4cO2\n+2LWLO2OVMqbXL5s/7bffhs6d3Y6GvfypORW5/p/WwK5gEnYBNcBOCkibuuaTE7JLTISHnsMGjWy\nfwBKKe+yZYudr7pqFRQr5nQ07uMxyS16A2M2iEjFO73mSskpuX34ISxdCn//rXXplPJW334LP/0E\nK1aAr6/T0biHJ04FSGOMKXrjiTGmCJDGdSElXytWwPffQ0CAJjalvNkrr0Du3LZOrHKNhLTcGgKj\ngf3XXyoEvCgi810b2k0xeH3LLTTUDhH+6ito3tzpaJRSrnb6NJQrBz//DLVrOx2N63lctySAMSY1\nUAI7sGSniFx1dWC3HN/rk9vzz9t/x4xxNg6llPv8+Se8/LK9D+fto6I9NblVBwpjJ30LgIi4bTCr\ntye3WbOgb1/YvFmXsVEquenRw46i/OknpyNxLY+752aMmQR8CdQAHgEqXX/ckTGmoTFmpzFmtzGm\nfxzb1DHGbDLG/GuMCUx46N7h5El46SU770UTm1LJz5dfwsqV8OuvTkfiXRJyz20HUOpum07GGB9g\nF1APOAqsAzqIyI4Y22QCVgANROSIMSabiJyOZV9e2XITsffXypSx1cOVUsnTmjXQrJntvcmd2+lo\nXMPjWm7Av8C9vN2VgT0ickBEwoGpwK1DJZ4GfhORIwCxJTZvFhAAhw7pir1KJXdVqtjqJd2724te\ndf8SktyyA9uNMX8bY+ZcfyRkPbe8wOEYz49cfy2mB4Asxpglxpj1xphkM2f/2DF44w27DEZymeei\nlIrbe+/BgQMwebLTkXiHhKwKMOge952Q649UQAWgLnbu3CpjzGoR2X1bEDGaN3Xq1KFOnTr3GJbz\nROxV2ssva4VwpZTl6wvjx9vqRHXrJv3uycDAQAIDAx07vssKJxtjqgKDRKTh9edvA1Ei8kWMbfoD\n/iIy6PrzMcBfIvLrLfvyqntuAQH2JvK6ddpqU0rd7N13Yds2u0ixNy1u6jH33IwxF40xoXE8QhKw\n7/XAA8aYQsYYX6AdcGt35iygpjHGxxiTBqgCbL/XHyYpOHbMVvvX7kilVGzee88uUvzzz05HkrTF\n2S0pIunuZ8ciEmGM6QnMB3yAsSKywxjT/frXfxCRncaYv4Ct2NW9fxQRr01uInZOS/fu8PDDTkej\nlPJEfn62e7JJE1tgOWdOpyNKmnQ9Nzf69Vd4/327Rpufn9PRKKU82Ztv2uWvpkxxOpLE4ZEVSpzm\nDcnt/HkoXRp++QVq1nQ6GqWUp7t82c6B/fZbaNzY6Wjunya3WHhDcuve3d4cHjXK6UiUUknFggXw\nwgvw77+Q7r5uFDnP45KbMeY1IEBEzrknpFhjSNLJbfly6NABgoK8vziqUipxdekC2bLZFUOSMo8Z\nLRlDTmCdMWba9VqRXjQ41fWuXrVz2kaM0MSmlLp7X31lR06uW+d0JElLQlcFSAHUB57BFk+ehh39\nuNel0f3/+Em25fbhh7BxI8yc6V1zVpRS7hMQYJPcunWQMiGlNzyQJ7bcEJEo4ARwEogEMgO/GmOG\nuDC2JG/vXhg+3LbaNLEppe5Vp06QKROMHOl0JElHQu659QK6AGeAMcBMEQm/3prbLSJFXR5kEmy5\niUDTplCrFrz1ltPRKKWSuu3b7Yrd27ZBrlxOR3P33N1yS0gDNwvQSkQOxnxRRKKMMU+6Jqykb9Ys\nW2Vg5kynI1FKeYNSpeC556BfP9tNqeKXkG7JorcmNmNMAIA3VxO5H5cuQe/e8P33WmJLKZV43nsP\nli61DxW/hCS30jGfGGNSAhVdE453+OQTqF4dHnvM6UiUUt4kXTr4+mt45RUID3c6Gs8WX+HkAcaY\nUKBszKLJQDC3F0BW1/33H4weDUOHOh2JUsobtWoF+fLZgWoqbgkZUPK5iDg6JCIpDShp2hTq1LEL\nkSqllCvs2gU1atjCEEmlsLLHVCgxxpS4XrW/IrEsPCoiG10dXIxYkkRymzcPevWypXL0XptSypVe\nfx1CQuDHH52OJGE8Kbn9KCIvGGMCiT25ue2OUlJIbuHhULasXYS0aVOno1FKebvz56FECfjzT6hQ\nwelo7sxjktv1YFIA1URkhbsCiiMOj09uw4bBX3/Z1ptO2FZKucOPP8LEibBsmeefdzwquQEYYzaL\nSHk3xRNXDB6d3E6dsnNQli2DkiWdjkYplVxERsIjj9hCEe3aOR1N/DwxuX0JrAZ+cyrDeHpye+kl\nSJ3att6UioyK5EzYGc6GneXClQtcuHqBC1cucCn8EmHhYYRFhHE14ioRURFEREWQe/shMh89x/bH\ny+CTyhcf44NfSj9Sp0yNf0p/0qRKQwa/DGRMnZGMfhnJ7J+ZbGmykTplaqd/VOUBli2z5bl27oQ0\naZyOJm7uTm6ISLwP4CIQBYQDodcfIXf6vsR8YO/53fYYOHCgxGbgwIG6vW7vku0vX7ss24O3yx//\n/SHfrf1O3lrwljz01EOxbp+lURapNLqS1JtYT1r/0lq6zOwi3ed0lyodq8S6fd8saeXHEc/IgIUD\n5PX5r8vLc1+WZ39/Vkq3LR3r9qkeTyWFhxWWamOqSZtpbaTXvF4yZMUQadOjTZJ5P3X7xNv+8cc9\nK55btwdE3Jg3dLHS+9S0KdStC336OB2JSkwnL55kW/A2goKD2Hl6J7vO7GLXmV2cuXyGAhkLUDhz\nYQpnKkz+DPnJmyEvedPnJXf63ORIm4Os/lnxSeET986vXYNvvoHPPrP1lN57z87OnToV3nzTFhAc\nPBjy5IlzFyJCyNUQTl0+xYmLJzgacpSjoUc5EnKEA+cPsP/8fvaf2w9A8azFeTDbgzyY9UFKZitJ\n2ZxlKZq5aPwxqiRlzx6oWtXWn8yRw+loYueJ3ZKPxva6iCxzSUSxx+CRyW3JEujWDXbsAD8/p6NR\n90JE2HtuLxuObWDD8Q1sPL6RrSe3EhEVQdmcZSmTvQwlspWITg75M+YnhUnQYhqxmz/fzhcpXNj2\nYz/44M1fv3gRPv3UVgLo18/WcbuPD9eZy2f478x/7Dqzi52nd7L91Ha2BW8j+FIwpbKXonzO8jyS\n5xEq5qlI2Rxl8UupH+Skqlcvew/u22+djiR2npjc5kL0VIDUQGVgg4g87uLYYsbgccktKgoqVbIX\n2p5+I1f938VrF1l9ZDWrDq9i1ZFVrD6ymnS+6aiYpyKP5H6ECrkrUC5XOXKny02irsu7bx/07Wsn\nQX79tW3yx7f/PXvs9jt32iTYuHHixQKEXA0hKDiIjcc3suG4Tex7zu6hTI4yVMtXjWr5qlGjQA3y\nZciXqMdVrnP6tB3QtmIFFC/udDS387jkdts3GJMfGC4irVwTUqzH9LjkNnmyLX+zerXnD8FNzi5e\nu8jyg8sJPBBI4MFAgoKDeDj3w1TLV43q+atTNV9VcqVz4fohly7Z7seRI+2s27597eijhJo3z7be\nHnjAJrlixVwW6uXwy6w/tj468a84vIIMfhmoXbA2tQvW5vHCj5M/Y36XHV/dvy++gLVr4bffnI7k\ndkkhuRlgu4i4bdC7pyW3K1fs5MmAALtem/IcURLFhmMbmL93Pgv2LWDj8Y1UzF2Rxwo9Ru1CtamS\ntwr+qfxdH4gITJtmuxZr1rT30PLdYyvo2jWb2AYPhhdegHfesffoXCxKothxagdLDy4l8EAgSw4s\nIat/VuoXrc8TRZ7g8cKPk9Y3rcvjUAkXFmbPTT//bMtzeRKPS27GmG9iPE0BlAf2i0gnVwZ2Swwe\nldy+/BKWL7drtinnhVwNYf6e+fyx+w/m7ZlHtjTZaFC0AU8UeYJHCz7q/hPwtm3w2mtw9qwdOPJo\nrLet796xY3ZC0+LF9hL96afd2m0QJVFsPrGZBXsX8Pe+v1l7dC018tegyQNNaFq8KYUzF3ZbLCpu\nAQG2o2DFCs/qVfLE5PYM/7/nFgEcEDdXLPGk5BYSYnuGAgPtxG3ljOBLwczaOYuZO2fyz6F/qFGg\nBk0faErjBxo7d5I9dw7efx9++QU++MC2slImZD3gu7RihU2eadLY5FnemRoLIVdDWLB3AX/s/oM/\ndv9BrnS5aFWiFS1LtqRsjrKJe89SJVhUlP1IfPIJPOlBy0l7XHIDMMb4ASWw8912icg1Vwd2y/E9\nJrkNGgT798NPPzkdSfJz+vJpftv+G78E/cLG4xtpWKwhLUu0pNEDjcjgl8G5wCIjYdw4O6S/VSv4\n6CPImtX1xxw79v/H/Phj1x8zvnCiIll5eCUzd85k5s6ZpEyRknal29G+THvK5CjjWFzJ1ezZ8O67\nsHkzpLiPwb2JyeOSmzGmCTAK2Hf9pSJAdxH508WxxYzBI5Lb6dN25Pb69XYkt3K9S9cuMXPnTCZt\nncTqI6tpWKwh7Uq3o9EDjTyjQsfKlfDqq7YVNWIEPPywe49/7hwMHGjnyA0aBN27g4+z89dEhA3H\nNzD136lMC5pGer/0dCzbkY5lO1IwU0FHY0suRKBaNTs9oEMHp6OxPDG57QKaiMie68+LAn+KyIPx\nfmMi8pTk9sYb9obtd985HYl3i5Ioluxfwk9bfmL2rtnUKFCDzg915sniT3rOAIZjx6B/f9s/PXgw\ntG/v7A2OG/f5zp2zSTax7vPdpyiJYvWR1UzaOolpQdMok6MMXcp1oW2ptqT3S+90eF5t8WJ7rbN9\nO6RK5XQ0npnc1olIpRjPDbA25muu5gnJ7cgRKFfOTlPKndvRULzWkZAjjN80nvGbx5PBLwPPlH+G\nDmU6kDOdB63GePUqDB/u9pGLCSICv/5qr8KqV4chQ+59hKYLXI24yp+7/2TClgksO7iMViVa8XyF\n56mar6ren3ORevXsPNwXXnA6Es9MbqOAAsC06y+1BQ4BCwBEZIYrA7weg+PJ7aWXIGNGO0hNJZ7I\nqEjm753PyPUjWXFoBe1Kt+P5Cs9TIXcFzzvh3ViN9sEH7URsF845uy+XLtkP6vff23l1dzu3zg2O\nhx5n4paJjN00Fl8fX3o80oPO5To7e+/UC61ZA23awO7dzn8EPDG5Tbj+3xsbmhj/R0SedUlkN8fg\naHLbuxeqVLFLuzt4z96rnLl8hrGbxjJq/Siy+GehxyM9aF+mved0O8a0Z48tHnqjWkiTJk5HlDB3\nWxXFASJC4IFARq4fyYJ9C3iq1FP0rNyTsjnLOh2a12jRwpYrdbr+rcclN0/gdHJ77jnIn9+O7lb3\nJyg4iOFrhjN9+3SaPdiMnpV6Uimv23q4704i13l0zJ3qWXqI46HH+XHjj4xaP4oS2UrQq0ovmhZv\nqgWe79OWLdCwob1Id3JJHI9LbsaYIsCrQCHgxqQdEZFmrg3tphgcS2779tkaknv2QObMjoSQ5IkI\nC/ctZMjKIWwL3kaPR3rQvWJ3z7qXFpPIXVXoTxJuXYng3Xchg2d2AV6LvMb0oOkMXzOcM2Fn6Fu1\nL88+/CxpUnnwYmUermVL+1Hu3du5GDwxuW0FxgD/Yue5gU1uS10cW8wYHEtuzz9vB5B89JEjh0/S\nIqIimBY0jSErhxAeGc4b1d+gQ5kOnl15fssWO7T/4kWbDDythtH9OnHCVjlZsAA+/xw6dvSciVC3\nEBFWHVnFkJVDWHFoBT0e6UHPyj3Jnja706ElOZs22d70vXvB3w3V52LjiYuVrr3XxeKAhsBOYDfQ\nP57tKmGrn7SK4+vihP37RbJkETlzxpHDJ1lXwq/IqHWjpNCwQlJ7fG35478/JCoqyumw4nf6tEiP\nHiI5coiMGiUSEeF0RK61apXII4+IVKsmsn6909Hc0c5TO+XF2S9K5s8zS+95veXIhSNOh5TkNGsm\nMny4c8fHzYuVJiRBdQYGAdWACjceCfg+H2APtjszFbAZKBnHdouBuUDrOPaVqG9yQr3wgsiAAY4c\nOkm6dO2SfLXyK8k7NK80mdxEVh5a6XRIdxYRIfLddyLZs4v07Jm8rmQiI0XGjBHJmdN+2IODnY7o\njo6GHJW+f/WVzJ9nlhdnvyj7zu5zOqQkY/16kTx5RMLCnDm+u5NbQvojSgMvAJ8DQ2M87qQysEdE\nDohIODAVaB7Ldq8CvwKnErBPtzl40C4b0bev05F4vrDwMIatHkbREUX55/A/zOkwh7lPz6Va/mpO\nhxa/ZcugYkVbvX/hQtsNmSWL01G5T4oUdrXdnTshbVooXdq+BxERTkcWpzzp8zC0wVD+e/U/sqfN\nTqUfK/HinBc5eP6g06F5vIoVoUIFGDPG6Ujc5E7ZD9gL+N5t1gTaAD/GeN4J+OaWbfICS7DTC8bj\nQd2S3buLvPWW2w+bpFwJvyIjVo+QPEPzSIupLWTz8c1Oh5Qwhw+LtG8vkj+/yNSpIp7eZeouQUEi\ndeuKlCkjsnix09EkyJnLZ2TAwgGS5Yss0n1Odzl84bDTIXm0detE8uZ1pvWGB7bctgH3Mk4wISNA\nhgFvXf/BzfWH444etRfzr7/udCSeKTIqkgmbJ/Dgtw8yf+985naYy8x2MymXq5zTocXvyhU7WrB8\neTsBe8cOW77Bw+Z+OaZUKTvQ5IMP4NlnoW1bOHTI6ajilcU/C5/U/YRdPXeRKXUmyo0qxxt/v8GZ\ny2ecDs0jPfKIrbQ0YYLTkbheQkZLLgUeAtYBV6+/LHKHqQDGmKrAIBFpeP3520CUiHwRY5t9/D+h\nZQMuAy+IyOxb9iUDBw6Mfl6nTh3q1Klzxx/uXr3xhi26/vXXLjtEkiQizNo1iwGLBpA1TVY+q/sZ\nNQvUdDqshJk7146DLlMGhg6FokWdjsizXb5sy3eNGGHft379nC9xkQDHQo/x0dKPmL59Or2q9KJv\ntb6eWRjAQf/8A1272qIUrliR6YbAwEACAwOjn3/wwQeIh00FqBPb6yISeIfvSwnsAuoCx4C1QAcR\n2RHH9uOBORJLOS93TgU4e9Ze1G/ZYiduK2vNkTX0W9CPc1fO8Xndz2n8QGPPK48Vm//+s6UZ9u61\nNSEbNHA6oqTlwAF7tbdxI3z1FTRvniRaunvO7uHdxe+y/NByPqzzIc+Uf0Yng8dQs6ad8dKunfuO\n6XFTAe7nATTCJrg9wNvXX+uOXTLn1m094p7bhx+KPPOM2w7n8faf2y/tf20veYbmkTEbxkhEZBIZ\nIh8SIvLmmyJZs4oMGSJy9arTESVtCxeKlColUr++yI4dTkeTYKsPr5aa42pKme/LyF+7/3I6HI8x\nZ45IuXLuvd2Mp0wFAC4CoXE8QtwapJuS28WLdkT49u1uOZxHC70aKu8sekeyfJFFBi0ZJBevXnQ6\npISJihIJCLB3zbt2FTl2zOmIvMe1ayLDholkyybSt6/IhQtOR5QgUVFRMnPHTCk2opg0/bmp7Dq9\ny+mQHBcVZccNzZvnvmO6O7nFOaBERNKJSPo4Hp5Zt+c+jRtnm+slSzodiXNEhElbJ1Hi2xIcOH+A\nLS9tYWCdgUnjvsXGjfYXOHy4XfplwgRdnygxpUpla1QGBcGFC1CihH2Po6Lu+K1OMsbQokQL/u3x\nL48WeJTqY6vz+vzXuXDlgtOhOcYYW6jm88+djsR1tHDydeHh9l7btGl2BYDkaMuJLfSc15Ow8DC+\nafSN589Tu+H0abuu2qxZ8MkndqSfh5aU8irr1tkbNyJ2flzlyk5HlCAnL55kwKIBzNszjy/qfUGn\nhzoljfvHiSwiAh54AH7+2a7a7WruvuemZ4Drpk61A+iSY2K7cOUCveb1ov6k+nR+qDNrnl+TNBJb\nRIQ9qZYsaUfy7dxpJyVrYnOPSpVg5Uro0cMONOnWDU6edDqqO8qZLidjm49lZruZDF8znNoTarP1\n5Fanw3K7lCntWCFvbb3pWQB74TlkiC0Cn5yICFP/nUrJ70oSFhHG9pe382LFF5PGqLLAQFtuYeZM\nWLLEdkVmyuR0VMlPihTwzDP2wiJTJjvVYtgw2xXi4arkq8Ka59fQsWxH6k2sxxt/v8HFaxedDsut\nnn0WVq+20wK8jXZLAosWwWuv2TUdk0vvxN6ze3n5z5c5HnqcH5r+kDRaamAnFffrZ/8ihw6F1q2T\nzy8tKdixw96XO3LEzpGrV8/piBIk+FIwb/z9BssOLuObRt/w5INPOh2S27z3Hpw5YxdudyWPW/LG\nE7g6uTVtantVXnjBZYfwGOGR4QxdNZQhK4fQv0Z/+lTtQyqfVE6HdWdXrsCXX9pWQc+etpnt5MqL\nKm4i9v5nnz62dT10KBQq5HRUCbJo3yJe+uMlHsr5EN82+pbc6b1/QNLx47Y4zd69ri2tqvfc3Gzn\nTntfvFMnpyNxvU3HN1F5TGUW71/M+hfW82aNNz0/sd04UZYqZRelWr8eBg3SxObJjIEWLWD7dlvq\nrGJFGDjQVj3xcHWL1GVbj22UzFaScqPKMW7TOJJCA+B+5M5tL+5/+MHpSBJXsm+59egB2bPDhx+6\nZPce4UrEFT4I/IBxm8cxuN5gupTrkjRGh+3YYUs/HT6cpLq41C1udCWvWWNbca1aJYmu5C0nttBt\ndjcy+2dmdNPRFM5c2OmQXGbLFmjcGPbvB19f1xxDW25udOaMHSX58stOR+I6a4+upcIPFdh9djdb\nX9pK1/JdPT+xhYTYqtWPPgqNGtm/PE1sSVeBAvDLLzB+vG1116tn58p5uHK5yrH6+dXUL1KfymMq\nM3LdSKLEs+f03aty5eDBB2H6dKcjSTzJOrmNHm2b47lyOR1J4rsacZW3F75NsynNGFRnEL8+9Ss5\n0+V0Oqz4RUXZScElSsD583aET+/edvKwSvoee8x2LbdoYf/fu7f9PXuwlClS0q9GP5Y/u5yftvzE\nEwFPcOD8AafDcok+fWz50CTQmZcgyTa5XbsG335rf6HeZtPxTVQcXZFdZ3ax5aUtPFX6KadDurP1\n66FGDRg5En7/HcaOhZwenozV3UuZ0k78DgqCsDB7ITN2rMdXOSmRrQQrnltBg6INqPRjJcZsHON1\n9+KaNIHQUFi+3OlIEkeyvef28892RdrFixN1t46KjIpk8IrBfL36a75q8BUdy3b0/C7I4GAYMAD+\n/NNWF+naVSdhJycbNth5ONeu2Qn5Vas6HdEdBQUH0WlmJ/JnyM+PT/7o+T0id+G77+zUqBm3rc1y\n//Sem5t89529gPQW+87t49EJj7Jg3wLWv7je80sKhYfbidelS0PGjHbwiJbNSn4qVrQLjPXqZecs\nPvMMnDjhdFTxKp2jNGueX0OZHGUo/0N5Zu2c5XRIiaZLF1sf4cgRpyO5f8nyTLJ5Mxw8CE96wTxN\nESFgSwBVxlShdcnWLOyykAIZCzgdVvwWLYKHH4Y//oBly+wIuowZnY5KOcUYOxdn507IkeP/C8pe\nu+Z0ZHHy9fHl07qf8mvbX+kzvw8vzX2Jy+GeP9XhTtKnh6eftuMRkrpk2S3ZvTvky2dn5idlF65c\noMcfPdh8YjNTWk+hXK5yTocUv4MH7SjIJLbwpXKzXbvsYJMDB2zrvn59pyOKV8jVEF758xXWH1vP\nlNZTKJ+rvNMh3ZegIDug9eDBxJ0WoN2SLnbhgq38//zzTkdyf1YdXkX5H8qTKXUm1r+43rMTW1gY\nfPCBrVbx0EP2r6dFC01sKnYPPmjvwQ4ZYufptGgB+/Y5HVWcMvhlIKBlAO/Wepf6AfX5etXXSXqw\nSenS9lfw++9OR3J/kl1ymzjRXggm1WW+oiSKwSsG0+KXFnzd4Gu+b/I9aVJ5aLUOEfjtN1u1PyjI\nttjefx/8/Z2OTHk6Y2xdvH//tUvpVKpku1ouXXI6sjh1fKgja55fw9SgqTSf2pwzl884HdI9e/ll\n19eadLVk1S0pYqs4jRoFtWsnQmBudurSKbr83oWQqyFMaT3Fs++tBQXZQQInTtjqIo8/7nREKik7\ncsRWOVmxwrbonnrKY1v+1yKvMWDRAKYFTePn1j9Ts0BNp0O6a9eu2XKgCxbYllxi0G5JFwoMBB8f\nW/giqfnn0D9UGF2B8jnLE9g10HMT2/nzdvJgnTr2ntrmzZrY1P3Llw+mTIFJk+DTT+0k8K2euQab\nr48vX9b/kpFNRtJmWhu++OeLJFfZxNfXFpIfOdLpSO5dskpu339vm9seesEXKxHhy5Vf0mZaG0Y3\nHc1n9T7zzGLHUVEwbpztgrx40RbNffVVO2lXqcTy6KN2btxTT9lRDz17wtmzTkcVqybFm7DuhXXM\n2jWL5lObcy7snNMh3ZUXXrDzgUNDnY7k3iSb5HbiBCxcmLSq/5+/cp5W01oxfft01r6wlkYPNHI6\npNitWWOXMB8zBubMgR9/tNWolXKFlCntVeqOHfaiqmRJW9I+MtLpyG6TP2N+Ap8JpFjmYlQcXZH1\nx9Y7HVKC5ctnO2B+/tnpSO5NskluP/1ki5FnyOB0JAmz9eRWHhn9CPnS52P5s8s9sxvyxAk78bpV\nK1tl4p9/4JFHnI5KJRdZs9rumPnzYfJkO+hkxQqno7qNr48vXzf8miFPDKHx5MaM2TjG6ZAS7Pnn\nbXW0JElEPP5hw7x3UVEixYuLrFhxX7txm8lbJ0u2wdlk8tbJTocSu6tXRb78UiRrVpE33hC5cMHp\niFRyFxUlMnmySL58Ip06iRw96nREsdp5aqeU/LakPD/reQkLD3M6nDuKiBDJm1dk69b739f187jb\n8kayaLn984+t6lStmtORxC88Mpzef/XmvSXvsajLIp4u+7TTId3u77/t+hgLF/5/5FpSaQ4r72WM\nLa2xYwfkz2/nU37xBVy96nRkN3kw24OseX4N56+ep9b4Why6cMjpkOLl42Mroo0b53Qkdy9ZTAV4\n5hlb0eeNNxIvpsR26tIpnvr1KfxT+jO51WQy+2d2OqSb7dsHffvCtm3w9de2dllSGpmjkpc9e+yo\n3UgwNHUAABqGSURBVF27YNgwuxKnBxERhqwcwterv2Zam2nUKljL6ZDitG+frWd9+DD4+d37fnQq\nQCILCbEz7bt0cTqSuG0+sZlKP1aiWr5qzOkwx7MS2+XLduJ15cr2ERQEzZppYlOerVgxO7hp2DBb\nyuvJJ23C8xDGGN6s8SYTmk+gzfQ2jFo/yumQ4lSkiG0czJ7tdCR3x+uT2y+/2GlWOXI4HUnspgVN\n44mAJ/ii3hd8WvdTfFL4OB2SJWKX5S1ZEnbvtvPVBgyA1KmdjkyphGvc2FY5qVXLNj8GDLBTVTxE\ng2INWPHcCr5Z+w3d53TnWqRnFot+7rmkN7DE67slq1a1VXuaNEnkoO5TlETxQeAH/LTlJ35v/7tn\nFVv99187+vHMGbvGVlKc9a7UrY4dg/79bTWHwYOhfXuP6YEIvRpK55mdOXflHL899RvZ0mRzOqSb\nhIXZqQGbN9tbmvdCuyUTUVCQ7Sdu0MDpSG52Ofwy7X5tx8L9C1nz/BrPSWznztmk9vjj0KaNnSyr\niU15izx5ICAApk61A6Fq14YtW5yOCoD0fumZ0W4G1fNVp8qYKmw/td3pkG7i7w/t2sGECU5HknBe\nndzGjrWDSTypSMbRkKM8Ov5R/FP6s6jLIs9YxTcy0i7gVLIkRETYEWcvv+xZb5xSiaVGDVi3zlZ0\nqF/fftbPOF/kOIVJwWf1PmNQ7UHUmVCHebvnOR3STbp1s6Mmo5JIJTGvTW4REXZmfdeuTkfyfxuP\nb6Tq2Kq0KdWGn1r8ROqUHnD/auVKO1Bk4kSYN89Ois2a1emolHItHx948UV7IefjYyuqjxzpEVVO\nOpfrzO/tf6fb7G58s+Ybp8OJVqECpEtnp1YlBV6b3BYtggIFoHhxpyOx5uyaQ4NJDRjWYBhv1XwL\n43Rf//Hj0LmzrdHXty8sX25Xx1YqOcmSxd5XXrDAjj6rWNGuDu+w6vmrs+K5FYxcP5LX5r1GZJTz\nSdcY6NjRFoNJCrw2uU2ebH8RnmDEmhF0n9uduR3m0rpUa2eDuXbN3m8oW9beId6xw75RTidbpZz0\n0EOwZAm8/bbtruzQwS6z46DCmQuzsttKgk4F0eKXFly85vwozw4d7BKN1zxzUOdNvDK5Xb5s52S0\na+dsHJFRkfSa14sfNvzAym4rqZKvirMBzZtnk9rSpbBqFXz2GaRP72xMSnkKY+xJY8cOO0+uXDm7\nvM6VK46FlCl1JuZ1nEeONDl4dPyjHA897lgsAAUL2h7ceZ51OzBWXpncZs+2Repz5XIuhrDwMNpO\nb8vW4K2seG4FhTIVci6YPXvsJNbXXoOvvoK5c+GBB5yLRylPljYtfPSRHXSydq2dwTxnjp376QBf\nH1/GNBtDq5KtqDa2GjtO7XAkjhuSSteky5ObMaahMWanMWa3MaZ/LF/vaIzZYozZaoxZYYx56H6P\n6XSX5OnLp6k7sS7+qfz5q+NfZEqdyZlALl6Ed96xk/1q1LDz1zxtwp9SnqpIEVve6Lvv7CrgTZrA\nf/85EooxhncffZcPH/uQOj/VYfnB5Y7EAdC2rV2IISTEsRASxKXJzRjjA3wLNARKAR2MMSVv2Wwf\n8KiIPAR8BIy+n2OePm3vB7dseT97uXf7z+2nxrga1C5Ym4CWAfilvI9ibPdKxK5aXLIkHDxo5/K8\n9db9FYZTKrlq0MCu+l23LlSvDm++6dgKnl3KdWFSy0m0ntaa6UHTHYkhSxa7ztuMGY4cPsFc3XKr\nDOwRkQMiEg5MBZrH3EBEVonIhetP1wD57ueAv/4KjRo5cytp84nN1Bxfk1crv8pn9T4jhXGg13fL\nFjs5dfBgm+AmTYK8ed0fh1LexNcXXn/d9n4EB0OJEnZCuAOTvp4o+gQLOi+gz/w+fLf2O7cfH5JG\n16Srz755gcMxnh+5/lpcugF/3s8BneqSDDwQSP2A+gxrMIyelXu6P4AzZ+xk1Pr17Ruwfj3UrOn+\nOJTyZrly2TIdv/0GI0bYv7ENG9weRrlc5Vj+7HKGrxnOe4vfw91lFJ980p5ijjs7viVeri5BkeB3\n3BjzGPAcUCO2rw8aNCj6/3Xq1KFOnTq3bXPwIOzc6f5yW79t/40ef/Tglza/8Fjhx9x78BvVRQYO\ntHPWduyw/QZKKdepWhXWrIHx4+29uGbN4JNPIHt2t4VQOHNhVjy3gsY/N+bExROMbDqSlCncU1XI\n3x+aN7eVzPr0iX2bwMBAAgMD3RJPrFy5EipQFfgrxvO3gf6xbPcQsAcoFsd+ErTS6+efi3TvnqBN\nE83o9aMl95e5ZeOxje49sIjIsmUi5cqJ1K4tsmWL+4+vlBI5d06kVy+RbNlERowQCQ936+FDr4bK\nExOfkJZTW7p1de+//xapVCnh2+PmlbhdndxSAnuBQoAvsBkoecs2Ba4ntqrx7CdBb16lSiILFyZo\n00Tx+fLPpdCwQvLf6f/cd1ARkSNHRJ5+WiRfPpGpU0Wiotx7fKXU7f79V+Txx0XKlBFZssSth74S\nfkXaTGsjdX+qKyFXQtxyzGvXRLJmFTl4MGHbuzu5ufSem4hEAD2B+cB24BcR2WGM6W6M6X59s/eB\nzMBIY8wmY8zaeznWoUN2xVh3FLEXEfov6M/ErRP559l/eCCrm+aMXb0K/2vvzqOrqq4Hjn83g8qk\nqKg/FUWglBVAFBFBJAIawiOBgERACNJaFlJxbGVVkQo41GrVH0gtLESUIUKQQqkJk4CiIEPlBwgG\nIuCEKIKGeU7g/P7YTxuZcpPcN7I/a7FWknffvSeH5O7cc87e5/nnNbm0dm0dg+3Rw6qLGBMNGjaE\nBQtg2DCt2N6jh25LEgbnVjiXrPQsalevTdKkJPIPhr4QdMWKOvf2r3+F/FKlEjf7uY0cqXsNvfFG\naNty7PgxBswawOrvVzMnYw4XVw5TkeFZs3RH4YYN4eWXoW7d8FzXGFNyBw/qiuVXX9Xf24EDw7LR\nr3OOQQsHkbMxh/l3z+fyapeH9Ho5OVrN74MPij823Pu5xU1wa91a8yw7dgxdOwqPF/Lbmb9l696t\nZPfMptq5Ycg32LhRZ2w//xxeeSX6NqczxpzeV19pCsHq1TB8uC48CcNIy3OLn+PNNW+y4O4F1Kpe\nK2TXOXxYF5B+9hlcVszuXbZZaSls367pXUlJobvGkcIjdJ/WnfxD+czOmB36wLZvn+4a3LIltG2r\nSaQW2IyJLddco2kDr72mRZkDAZ1OCLEnEp/gwZse5Nbxt7IxP3RVVc47T7+lf/87ZJcotbgIbu+8\no4nboXrqP1hwkM5ZnRERZvaYSeWKlUNzIdDqIpmZmiT6/fewbp0OaZxzTuiuaYwJraQk/Qs8EIDE\nRP2dDnH9qoeaP8TQ1kNpO6Et67avC9l10tOjs1pJXAS36dOha9fQnHv/0f2kTk7lkiqXMPXOqaEt\np7VqlSaFjhihpVYmTIDLQztmbowJk4oVdYrh009h1y79A3b8+JBWOfldk9/xcvLLtJvUjtXbVofk\nGh066J7Hu3aF5PSlFvPBbfdu7dgOHfw/994jewlkBqhTvQ7jO48PXYLkjz9C//6QkqKrrFasgJtv\nDs21jDGRddllMG6cFmUePVqnHj7+OGSXu6vRXYxKHUXgrQAff+v/dapW1ZmTnBzfT10mMR/ccnK0\nY6tW9fe8uw/vJnlSMtdeei1j08ZSvlx5fy8AUFioq6kaNNAx1Q0boF8/3fbeGBPfbrpJ91Xs318X\nmvTtq3UrQ6BrQlde7/Q6qZNTWfrNUv/P3zX6hiZjPrjNmKFjvn7adWgXSROTaH5lc0aljgpNAeRF\ni+CGG/QbeO89XQl54YX+X8cYE73KlYN77tFFJhdcoKk+I0ZAQYHvl+pUvxMT75hI56zOvm+Z06kT\nLFwIBw74etoyielUgAMH4Ior4Msv/SunuOvQLpImJdG6VmteTn4Z8XvZ7pYtmrOwfLnmq6WnWxK2\nMUZt2AAPPwzffqvJu7ff7vsl5n8+n14zejGj+wwSayX6dt7kZLj3XrjzzlO/bqkAJfD++9C0qf+B\nrU2tNv4HtsOH4dlnoUkTnUjesEF/CiywGWN+kpCgO4H+5S86RZGerrlyPmpXtx1T0qeQ/na6r09w\nnTtrrYloEdPBbe5c/xaSFA1sLyW/5F9gc06TQBo00ETOlSvhqaegcgjTCYwxsUsEunSB3Fy4/nq4\n8Ua9Zxw65NslkuokMTl9sq8BLhDQe3K0DAbGfHDzI6959+HdtJvUzv/Alpen0XfQIBgzRnMWatf2\n59zGmPhWqRI8+aSmCOXm6lPd9Om+RY+iAW7JliVlPl/durqwb+1aHxrng5gNbps3a/m2a68t23n2\nHtlL+8z2tLq6lX+Bbe9enVdLTNTo+8kn0K5d2c9rjDn7XH01vP227h03bJjeS3JzfTl1Up0kMrtm\n0nVqV5ZvXV7m8/309BYNYja4zZ2rHVmWWLTvyD46vNWBppc3ZXj74WUPbMePw8SJOqeWn6/Jmn/4\ngyZvGmNMWbRtq1MbnTtDmzZ6b9m9u8ynTa6bzPgu40mbksbK71aW6Vzt21twK7OfgltpHTh6gI5T\nOtKgRgNeTXm17IFt5Uq45Rb4xz80OfONN4qvJGqMMSVRoQI8+CCsX69DVwkJmhBexionKfVSeD1N\n8+DKUsmkTRu9Fe7bV6bm+CImg9uRI7B4cekLJR8uPEyXqV2oXb02YzqNKVse244duqqpUyddB7ts\nmSZnGmNMqFxyic7j5+RocGvRQisblUFa/TRGpYwiZXIK639YX6pzVK0KzZvrSvZIi8ngtmSJ5jqW\nJgWg4FgB3ad156JKFzEubVzpA1tBgeahNGyo/6N5eZqMWS4mu9QYE4uaNtUb4gMPaJmQe+7Rguul\nlN4gnZfavUTypGQ279xcqnNEy7xbTN6JS7tK8tjxY/T+V28AMu/ILH1Jrffe03y17GzdpW/4cK0u\nYIwx4VauHPTpo39gX3IJNGqkBSKOHi3V6TIaZzC09VCSJiaxZc+WEr8/EIA5cyKfEhCzwa2k823H\n3XH6Zfdj56GdvN3tbSqWL8Uij6+/1sTrvn3h6afh3Xc1f80YYyKtWjXd/fujj2DBArjuOr1HlUK/\npv14pMUj3D7xdrbt21ai9zZsqANbmzaV6tK+ibngtnUrbNumeY1eOed4ZO4jbMzfyMweMzmvQgk3\nfjt0SJMob7gBGjfWydyuXa26iDEm+tSvD7Nna6C77z5NCP/iixKf5pEWj/Cb635DcmYyOw/t9Pw+\nkegYmoy54Pbuu5rmUZLC+cMWDWPxlsXk9MqhyjlVvL/ROS1s3KCBbhq6ahUMGaLJlcYYE61EdJFb\nbq4ucGvWTBPCS1jZeHDiYNrXbU/KWynsP7rf8/sCAa0iFkkxF9xKOiQ5fNlwsnKzmNd7HtXPq+79\njevXayXQIUN0NdI//wm1apW8wcYYEynnnQdPPAFr1mjli4QETQj3OCEmIrzY7kUaXdqILlldOFx4\n2NP7kpJ0Rfthb4eHRMwFt0qVNOZ48ebqNxmxYgTz757PpVUu9famPXvgj3+E1q31L581a+C220rf\nYGOMibSrroIpUyAzU4sy33abjkZ5ICKM6TiGiypdRM/pPSk8Xljse6pXh169dAopUmJ6y5szmZk3\nk/tm3cei3yyifo36xb/h+HHd8n3wYOjYEZ57TlceGWNMPCkshNde01JePXroegIPeVVHjx2l05RO\n1KxWk9fTXi9x4Qvb8sYHH3z1Afdm30t2z2xvgW3FCrj5Zv0Pz86GsWMtsBlj4lOFCjBggG67deyY\nDlW+9pp+fAbnlD+H6d2nk/tDLoMWDgpTY0sv7p7c1ny/huRJyUxJn8LtdYrZ6G/7dnj8cZ35fP55\n6N3bkrCNMWeX1avhoYe0nNff/w4tW57x8PyD+SS+mUjfJn15tOWjni9jT25l8PnOz0mdnMqo1FFn\nDmwFBZp43aiRPqHl5WkSpAU2Y8zZpkkT+PBDGDhQhyn79DnjZNnFlS9mXu95jPzPSCasmRDGhpZM\n3NzNdxzYQeCtAH9O/DN3NjjNPucA8+drcuO8ebqc529/g/PPD19DjTEm2ohAz546VHnllbqX2Isv\nnrbKyVUXXMXcjLk8tuAx5myaE+bGehMXw5L7j+6n7YS2BOoGeOa2Z0590JdfwqOP6t5qw4frSkhL\nwjbGmJNt2qSrxjduhBEjdNPlU1j6zVI6Z3Vmdq/ZNLuy2RlPacOSJVRwrIBu07rR+NLGPN326ZMP\nOHhQc9WaNdMio7m5kJZmgc0YY06nXj1dXDd8ODz8sN4zN59cSLnlVS0ZlzaOtKw0NuVHuN7WCWI6\nuDnn6Jfdj/JSnjGdxvxyaapzMG2argTauFEnTQcP1qRGY4wxxUtJ0Xy4W27RbXUGD4b9v6xUklY/\njafbPE3grQDb92+PUENPFtPBbcj7Q9jw4wam3jmVCuUq/PeFdes0SfGZZ3Rn7KwsTWI0xhhTMuee\nC489BmvXavH4hARNCC8yVdSvaT/ubnw3qZNTOXC0ZCW+QiVm59zG/t9YXvjoBZb2Xfrf6iO7dsHQ\noRrMhg6F/v01p8MYY4w/lizR3cCrVdM9La+/HtCRtL7v9GXHgR3MvGvmLx84sDk3T+ZsmsOT7z/J\n7IzZGtiOHdPE64QE3aZ7/Xq4/34LbMYY47dWrWDlSsjI0I01BwyA/Pyfy3QVHC/ggdkPEOkHp5gL\nbqu2raLPzD7M6DGDX1/8a1i6VPc1Hz9et3kYMwZq1Ih0M40xJn6VL68jYxs2aH5wQgKMHk1FyjGt\n2zSWb13OCx+9ENEmxtSw5JY9W2g5riWvBF4h/cKWOg68cKHmqvXqZSsgjTEmEtau1aHKPXtg5Ei+\na/IrWr3Rinm951Hv4npAnA1LikhARPJEZJOIPHaaY0YGX/9ERJqc7lx7j+yl4+SODLzxIdJnfaFJ\nhldcodVFMjIssAGLFi2KdBNigvWTN9ZP3lg/oZs4L1oEgwZBRgZX9B9Ibtp/A1skhCy4iUh54FUg\nADQAeopIwgnHpAC/cs7VA+4FRp/ufN2mdePeH67m4X7j4IMPYNkyrQdZrVqovoWYY79k3lg/eWP9\n5I31U5CIlu/Ky4M6dajU7GZNw4qQUK64uAnY7Jz7CkBEsoDOwIYix6QBEwCccytEpLqIXOacOylZ\n4qn/XU3zfRcgI0ZAamoIm22MMabUqlSBZ5+F3/9eS3lFSCiHJa8Eviny+dbg14o7puapTnZ91wHI\np59aYDPGmFhQs2ZEp4tCtqBERNKBgHOuX/Dz3kBz59yDRY7JBp53zn0U/HwB8Cfn3KoTzhX9q16M\nMcacUTgXlIRyWPJboGhZkKvQJ7MzHVMz+LVfCGeHGGOMiX2hHJZcCdQTkWtE5BygB/DOCce8A/QB\nEJEWwO5TzbcZY4wxJRGyJzfnXKGIPADMA8oD45xzG0Skf/D1Mc652SKSIiKbgQPAPaFqjzHGmLNH\nTCRxG2OMMSURVeW3/Ez6jmfF9ZOIZAT7Z62IfCQijSPRzkjz8vMUPK6ZiBSKSNdwti9aePy9ayMi\nq0XkUxFZFOYmRgUPv3c1RGSuiKwJ9tNvI9DMiBKRN0Rku4isO8Mx4bmHO+ei4h86dLkZuAaoCKwB\nEk44JgWYHfy4ObA80u2O0n66Gbgg+HHA+unU/VTkuPeAHCA90u2Oxn4CqgO5QM3g5zUi3e4o7adh\nwF9/6iMgH6gQ6baHuZ8SgSbAutO8HrZ7eDQ9uf2c9O2cKwB+Svou6hdJ30B1EbksvM2MuGL7yTm3\nzDm3J/jpCk6TOxjnvPw8ATwI/BP4IZyNiyJe+qkXMN05txXAOfdjmNsYDbz00zbg/ODH5wP5zrnC\nMLYx4pxzi4FdZzgkbPfwaApuviZ9xzEv/VRUX2B2SFsUnYrtJxG5Er1B/VT27WycgPby81QPuEhE\n3heRlSJyd9haFz289NNYoKGIfAd8AjwcprbFkrDdw6NpwzOvN5YTc97OthuS5+9XRNoCvwNuCV1z\nopaXfhoBPO6ccyIinPyzdTbw0k8VgRuA24HKwDIRWe6c2xTSlkUXL/30BLDGOddGROoC80XkOufc\nvhC3LdaE5R4eTcHNt6TvOOelnwguIhmLVok50zBBvPLST02BLI1r1AA6iEiBc+7EfMx45qWfvgF+\ndM4dAg6JyIfAdcDZFNy89FNL4C8AzrnPReRLoD6a82tU2O7h0TQsaUnf3hTbTyJyNTAD6O2c2xyB\nNkaDYvvJOVfHOVfbOVcbnXe77ywLbODt9+7fQCsRKS8ildGFAOvD3M5I89JPeUASQHAeqT7wRVhb\nGf3Cdg+Pmic3Z0nfnnjpJ2AIcCEwOvhUUuCcuylSbY4Ej/101vP4e5cnInOBtcBxYKxz7qwKbh5/\nnp4D3hSRT9AHhz8553ZGrNERICJTgNZADRH5BhiKDmuH/R5uSdzGGGPiTjQNSxpjjDG+sOBmjDEm\n7lhwM8YYE3csuBljjIk7FtyMMcbEHQtuxhhj4o4FN2PCQESmikid4MezROT84t4TPLabiOSKyDER\nuaHI1xuLyLhQtdeYWGfBzZgQE5FfAVWcc18AOOdSnXN7Pb59HXAH8GHRLzrn1gJ1ReRSXxtrTJyw\n4GaMT4KlmfJEJFNE1ovINBGpBNxFkVJNIvKViFzk5ZzOuTzn3MbTvDwH6Fb2lhsTfyy4GeOvXwP/\ncM41APYCA9BdGYoWz/25LJCIfBjc4frEf7d5uNZ/gFv9bLwx8SJqaksaEye+cc4tC36cie7pVQvd\nyPIkzrmyBKdt6M7QxpgTWHAzxl9Fi7UKWmgYTjNKIiKLgaqneGmgc25hMdcSzr79DI3xxIKbMf66\nWkRaOOeWA72AJUAl4H84xb5VzrnEEpz7xE0eLwe+Lm1DjYlnNudmjL8+A+4XkfXABcBoNMDdWOSY\nkuymfkdw65AWwCwRmVPk5Zs4YRWlMUbZljfG+ERErgGynXPXnvD1OsDfnXOpPl9vEdDdObfDz/Ma\nEw/syc0Yf53012Iwv22fiNT16yIi0hjYbIHNmFOzJzdjjDFxx57cjDHGxB0LbsYYY+KOBTdjjDFx\nx4KbMcaYuGPBzRhjTNz5f3HqQLV7xm7TAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = plt.figure()\n", "ax = plt.subplot(111)\n", "\n", "for i, lab in zip([ent, gini(x), err], \n", " ['Entropy', 'Gini Impurity', 'Misclassification Error']):\n", " line, = ax.plot(x, i, label=lab)\n", "\n", "ax.legend(loc='upper center', bbox_to_anchor=(0.5, 1.15),\n", " ncol=3, fancybox=True, shadow=False)\n", "\n", "ax.axhline(y=0.5, linewidth=1, color='k', linestyle='--')\n", "ax.axhline(y=1.0, linewidth=1, color='k', linestyle='--')\n", "plt.ylim([0,1.1])\n", "plt.xlabel('p(i=1)')\n", "plt.ylabel('Impurity Index')\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.4.3" } }, "nbformat": 4, "nbformat_minor": 0 }