{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#Clustering and Unsupervised Analysis\n",
"\n",
"##Clustering Overview\n",
"<p>This notebook covers different clustering methods. We'll cover both k-means and hierarchical based clustering. We'll also cover how to incorporate SVD into clustering methods. <br><br>\n",
"\n",
"Clustering is an unsupervised technique that doesn't require a particular outcome variable. The core idea of clustering follows this logic:<br>\n",
"<ul>\n",
" <li>Identify an object/instance $i$ by its feature vector $X_i=<X_i^1,X_i^2,...,X_m^1>$ </li>\n",
" <li>Define a distance function $d(X_i,X_j)$ to measure distance between $X_i$ and $X_j$.</li>\n",
" <li>Create distinct groupings of instances that:</li>\n",
" <ul>\n",
" <li>Minimize intra-group distances</li>\n",
" <li>Maximize inter-group distances</li>\n",
" </ul>\n",
"</ul>\n",
"<br>\n",
"Note that in the above, if we use a similarity function instead of traditional distance metric (i.e., cosine similarity as opposed to Euclidean distance), we want to maximize similarity within clusters.\n",
"<br><br>\n",
"The key questions we often consider when clustering are:\n",
"<ul>\n",
" <li>What is a good distance/similarity function?</li>\n",
" <li>How many clusters is appropriate?</li>\n",
" <li>How do we interpret/assess the clustering?</li>\n",
"</ul>\n",
"<br>\n",
"We'll explore these in the options below.\n",
"\n",
"<br><br>\n",
"The data we'll use is from the student survey that everyone filled out at the beggining of the class. The survey asked for each student to rank themselves on a scale of 1 to 10 in each of the following DS related skill sets - Visualization, Computer Science, Math, Statistics, Machine Learning, Business, Communication. In the next section we load and do some basic distributions of the results.\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADHCAYAAABMZ8f7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXtUU8e+x78BgqC8EYISFEVBXgYUtXqlPrHWqlctWvEB\nlWLPuXo91WOVek5vrV5f1NrWPk9PK4rVquv0WqUWvGIVpVqvUlSstmoRFBS1oogICsG5f1BSIDvJ\nzs5Odgi/z1pZC/ZMZn57vnsys2d+MyNjjDEQBEEQhJVhJ7UBBEEQBMEFNVAEQRCEVUINFEEQBGGV\nUANFEARBWCXUQBEEQRBWCTVQBEEQhFXSrhqoiIgIHD16VGozCMKm2bJlC2JjY6U2g7ABbKqBGjt2\nLJYvX651fe/evejSpQsKCwvx9NNPS2AZwcWXX36JmJgYuLq6omvXrhg3bhyOHTuGyspKJCcno0uX\nLnBzc0NISAjS0tKkNtcmCQwMRIcOHVBRUdHienR0NOzs7HDt2jW93y8pKYGdnR2ePHliTjPbNd9/\n/z2GDBkCDw8PeHt7Y+jQocjPzze6I9AWtbKpBurFF1/Etm3btK5/8cUXmDVrFuzsbOp22zTvvPMO\nFi1ahNdffx23b99GaWkp5s+fj8zMTCxatAg1NTX45ZdfUFVVhczMTPTq1Utqk20SmUyGnj17YseO\nHZpr586dQ21tLWQyGe90aL2/eaiqqsL48ePxyiuv4N69e7h+/TqWL1+ODh06CE6zTWnFbIiamhrm\n7u7Ojh49qrl29+5d5uTkxM6ePcu6d+/OvvvuO8YYY+7u7szFxYW5uLiwTp06MZlMxq5evSqV6e2K\nyspK5uLiwr766ivO8IiICLZnzx4LW9U+CQwMZKtWrWIDBgzQXFu8eDFbvXq1pk7s27ePRUVFMTc3\nNxYQEMDefPNNTdyAgAAmk8mYi4sLc3V1ZT/88APbsmULGzp0KHv11VeZp6cn69GjB8vOzpbi9to8\np06dYh4eHlrXL1y4wJycnJi9vT1zcXFhnp6ejDHGWysXFxd24sQJdvnyZfb0008zd3d31rlzZ/bC\nCy9Y7N74YFMNFGOMzZ07l6WkpGj+/8c//sGio6MZY42VsamBas6yZcvYsGHDmFqttpid7Zns7Gzm\n4ODAGhoaOMNTUlJYeHg427x5M7t06ZKFrWtfBAYGsoMHD7KQkBD2888/M7VazZRKJbt69aqmgcrN\nzWU//fQTY4yxwsJCplAoNB2IkpISJpPJWmi5efNmJpfL2eeff86ePHnCPvnkE9a1a1dJ7q+tU1VV\nxby9vVlSUhLLzs5md+/e1YQ1dQSaY6xW06dPZ2vWrGGMMfb48WN27Ngxc9+SUdjcmFdSUhK++uor\n1NXVAQC2bt2KpKQknfF37dqFHTt24H/+539gb29vKTPbNRUVFejcubPOIdcPPvgAM2fOxIcffojw\n8HD07t0b+/fvt7CV7YvZs2dj69atyMnJQVhYGPz9/TVhw4YNQ3h4OAAgMjIS06dPx5EjRwDoHi7q\n3r07XnrpJchkMiQmJqK8vBy3b982/43YGK6urvj+++8hk8kwd+5c+Pr64t///d9x+/ZtzrI3VitH\nR0eUlJTg+vXrcHR0xJAhQ8x7Q0Zicw3Uv/3bv6Fz5874+uuvUVRUhFOnTmHGjBmccU+fPo0FCxZg\nz5498Pb2trCl7Rdvb2/cuXNH52Stk5MTli1bhvz8fFRUVGDatGmYOnUq7t27Z2FL2wcymQyzZ8/G\n9u3bkZGRgcTExBY/Zv/3f/+HESNGwNfXFx4eHvj000+1nCpa4+fnp/m7Y8eOAIDq6mrz3ICN06dP\nH2zevBmlpaX46aefcOPGDSxcuJBzjtBYrd566y0wxjBw4EBERERg8+bN5rwVo7G5BgoAEhMTsXXr\nVmzbtg1jx46Fj4+PVpzbt29j8uTJ+Pjjj6FSqSSwsv0yePBgdOjQAV9//bXBuK6urli2bBkePnyI\nkpIS8xvXTunWrRt69uyJ7OxsTJkyRXOdMYYZM2Zg0qRJKCsrQ2VlJf785z9rOhfGOFIQphMSEoKk\npCT89NNPnGVvrFYKhQL//Oc/cf36dXz66aeYN28erly5Yvb74IvNNlA5OTn4/PPPOYf31Go14uPj\nMWvWLMTHx0tgYfvG3d0dK1euxPz587F3717U1NSgvr4e2dnZSE1NxapVq5Cfn4+6ujo8evQIGzdu\nhKenJ0JCQqQ23abZtGkTDh06BGdn5xbXq6ur4enpCUdHR5w8eRJffvml5sfOx8cHdnZ2KCoqksJk\nm+fixYt45513cP36dQBAaWkpduzYgcGDB0OhUKCsrAz19fWa+MZq9a9//QtlZWUAAA8PD8hkMuvy\ndpZyAsycDB8+nHl5ebG6ujrNtSYnieLiYiaTyVinTp00Hi2urq6stLRUQovbH9u3b2cxMTGsU6dO\nzM/Pj40fP5798MMPbNWqVSwiIoK5ubkxLy8vNmLECPbDDz9Iba5NostxqL6+ntnZ2bGrV6+yr776\ninXv3p25urqy8ePHswULFrDZs2dr4r7xxhvMx8eHeXp6shMnTrAtW7aw2NjYFunZ2dmxoqIis9+P\nrXH9+nU2bdo05u/vzzp16sT8/f3Zn//8Z/bgwQNWV1fHnnvuOebl5cV8fHwYY8xorZYuXcr8/f2Z\ni4sLCwoKYp999plUt8qJ3gZqzpw5zNfXl0VERGiFvf3220wmk7GKigqzGUfwg0unV199lfXp04f1\n7duXTZ48mVVWVkpoIaGrLr3//vusT58+LDw8nC1dulQi6wjGqB5ZI3rf5ebMmcPpPVVaWoqcnBx0\n797dbG92BH+4dBozZgzOnz+Ps2fPIjg4GGvXrpXIOgLg1ujw4cPIzMxEYWEhfvrpJ7z66qsSWUcA\nVI+sEb0NVGxsLDw9PbWu//Wvf8Vbb71lNqMI4+DSKS4uTjOWPGjQIM04MyENXBp98sknWLZsGeRy\nOQBwOvMQloPqkfVh9GzY3r17oVQq0bdvX3PYQ5iB9PR0jBs3TmoziFZcvnwZR48exVNPPYXhw4cj\nPz9fapMIPVA9sjwOxkSuqanBmjVrkJOTo7nG2tK+Tu2Q1atXw9HRUedaMEI61Go17t27hxMnTuDU\nqVOYNm2aVbn4En9A9UgajGqgioqKUFJSolk3VFZWhv79++PkyZPw9fVtEbdXr17kesqDoKAg/Prr\nr2ZJe8uWLcjKysJ3333HGU4a8cNcGimVSs2aowEDBsDOzg4VFRVai8ZJJ36YSydD9QggjfhitEaG\nvCiKi4s5vfgYa3RR1eXFZyhpAAxYzgDG8Wn53eXLlxsy06JxxEyLhwS8aK1TdnY2CwsLY7/99ptJ\neRu6B3OHW4MN5tLoH//4B3vjjTcYY4xdvHiRBQQECM7fmp7/xrrNdNRv89TtpnxNRUg94pu3Nf7+\nWNomYzXSOweVkJCAIUOG4NKlSwgICNDaBoNWkVsHTTpdvHgRAQEBSE9Px4IFC1BdXY24uDhER0dj\n3rx5UpvZruGqS8nJybhy5QoiIyORkJCArVu3Sm1mu4bqkfWhd4iv+RkxXNB4uXXApVNycrIElhC6\ncHZ2RkNDA4KDg3Hu3DnN9S+++AIbNmzAkiVLyPFIYqgeWR8GvfiSk5OhUCgQGRmpubZkyRKEhoZC\npVJhypQpuH//vsDsh/OLNdxwPEvGETsta8bQPZg73FpsMBVzrym0xuefT/1uL/UIsM7fH8s/E0Zi\naAzw6NGjrKCgoMW47IEDBzRniqSmprLU1FSjxxqhGac2PAdly4hxr1wr4CsqKtjo0aNZ7969WVxc\nHLt3755Z8rYErq6evz8vLT+urp4WyV+scuKaz42Pj2dnz541aT7X2pCqbpuattB6JEbeutD17Fu6\nDoiFseVk8A2KFq9ZP1y983Xr1iEuLg6XLl3CqFGjsG7dOomsM50HD+6Bq342Xm+70JpC68Ia65Gu\nZ99W6oAhjHIz5yI9PR0JCQli2EIIJDY2VusoiszMTM1BZUlJSRg+fHibbqRsDWPXFL755puav4cP\nH24zw16mkJubi9zcXNHSo3pkfZjUQNHiNevl1q1bUCgUABrPfLl165bEFhHNMWZNIdCygSIaad1Q\nr1ixQvQ8qB5Ji+AGis/iNer1aSN2r48PMpmMlgRYGZGRkS1+7Hr06IEff/wRXl5eElpF6IPqkeUR\n1EDt378f69evx5EjR+Dk5KQzHvX6tLFErw9o7O3dvHkTfn5+KC8v5+yVA9SJ4MIcnYiePXvi6tWr\nYIwhICAAK1euxIULF7Bv3z44Ojri9u3bqKqqogbKyuBbjwCqS1yYWpdkTN/ANxoXrx05cgR37tyB\nQqHAihUrsHbtWtTV1Wkq0+DBg/Hxxx+3TFgm0zum3tgT0RWu/7u2hKFy4ktJSQkmTJigWWOzdOlS\neHt7IzU1FevWrUNlZaXW2LlYeZsb3c+KZewXo5zy8vLg4uKCxMREjUY5OTkYNWoU7Ozs8NprrwEA\n5/xGW9GpCanqthjlJKQeiZU3F/rLEmhrv5VGl5M+Fz9zul2C3MwZY+Lc6/Tp01mXLl2YXC5nSqWS\npaens4qKCjZq1CibcDPX/axYxn6x8tG3bdju3bvZzJkzzZq/pZCqbpuattB6JEbeutBflm3z2TAG\nvW9QXL2+pUuXonPnzli6dCnS0tJw7949QT0KeoNqRMrecVvpmdvCGxSg3TtvzoQJE5CQkMDpcNRW\ndGqiLb9BWVve7f0NyugDCzMzM5GUlASg0e1yz549AswkCKIJ8oYlCG6MdpIgt8u2w9q1a7Ft2zbY\n2dkhMjISmzdvRocOHcyer5ubF+cCQldXT1RV3TV7/m0JPt6wAE3Ac2FJj1ip6pIp6KqHjcgB1Ov8\nrtXUVUNjgK3HzT08PFqEe3pyb7VhKGnQHBRjzHz3WlxczHr06MEePXrEGGNs2rRpbMuWLRbJW7e2\nwvITOz0h+YuBOY9ysCakqtvtqy4ZLk9DOkgxt2Vsuka/QZHbpWlYqtfn5uYGuVyOmpoa2Nvbo6am\nBv7+/mbPl+CmtZv5ihUrsHr1aty4cQNKpRKOjo6YOnUqNm3aJLWpRCuoLkmH0Q3UxIkTkZGRgdTU\nVGRkZGDSpEk649I6KG0stQ7Ky8sLixcvRrdu3eDs7IxnnnkGo0ePNktehGEyMjK0HI5++eUXLYcj\nwvqguiQh+l6vzOl2CRriY4yZ715//fVXFhoayu7cucPq6+vZpEmT2LZt2yySt25taYiv+RBfSEgI\nu3nzJmOMsfLychYSEmLW/Fujb6dsU3bJlqput6+6REN8Og8sPHjwoGbSMDY2ts1MGrYn8vPzMWTI\nEHh7ewMApkyZguPHj2PmzJkt4tEwrDaWGoaV2uHoj52yucJoS58mqC4Jx+S6JKQVFGPSEHpbcAez\n9OysEYESGOTMmTMsPDyc1dTUsCdPnrDExET24YcfWiRv3drSG5Q5HI6Eor8OCs/TXOnyydccWGdd\nojconZh/0lAN6tmZhkqlQmJiImJiYmBnZ4d+/frh5ZdfltosohnkcGQalnrTpbokHQb34tPFP//5\nTyxevFgzafjFF1+0TNjEnST0hQk02SppX6vfheVnqztJWPc+b8LzpJ0kxE1Xd1kC+spT+G+s/nRN\nQdSdJHRRVFSE9957DyUlJbhx4waqq6uxfft2IUkRRLsgISEBQ4YMwcWLFxEQEIDNmzfjtddeQ05O\nDoKDg3Ho0CHNhrEEQTQiaIiPJg2FY8nV75WVlUhJScH58+chk8mQnp6Op556yiJ5Ey0hh6O2Tfur\nSw46z76y5C4Tgob4zp49i5kzZ+LUqVNwcnLCiy++iIEDB2L+/Pl/JExDfLww57BEUlIShg0bhuTk\nZKjVajx8+BDu7u5mz5uG+PhTUlKCkSNH4ueff0aHDh3wwgsvYNy4cZr9Ls2ZPw3x8cf66pImhtmG\n+Mz1bBjzXUFvUDRpaP3cv38feXl5yMjIAAA4ODi0qFCEdUC7FFg/VJekQ9AcFNA4wXv+/HmcO3cO\nGRkZkMvlYtpFmEhxcTF8fHwwZ84c9OvXD3PnzkVNTY3UZhGtaL5LQdeuXeHh4UG7FFgZVJekQ3AD\nVVlZifj4eISGhiIsLAwnTpwQ0y7CRNRqNQoKCjBv3jwUFBSgU6dOnB5ihLSQw5H1Q3VJOgQN8QHA\nK6+8gnHjxuGrr77SjMm2dfRtT28128/zRKlUQqlUYsCAAQCA+Ph4zkpFjizaWNKRhRyOhGMpnagu\nCcdUjQQ5Sdy/fx/R0dG4cuWK7oTboJOEFJO75pzYffrpp/H5558jODgYb775Jmpra5GWlmb2vMlJ\ngj9iOBwJhZwk+GN9dUkTg5wkWtN8TPbs2bPo378/Nm7ciI4dOwpJjjATH3zwAWbOnIm6ujoEBQVh\n8+bNUptEtEKlUmHq1KlQKBSoq6uDk5MT5s6dK7VZRCuoLkmDoDeo/Px8DB48GMePH8eAAQOwcOFC\nuLm5YeXKlX8kTG9QvPNsP6vf6Q2KC+t0Ydafp/7TWiE4XVNoX3VJE4PeoFpDY7LCseT8BgA0NDQg\nJiYGSqUS33zzjVHffeqpkTh58ghn2LPPTsG33/5LDBPbNW3VhVnfTuiNP262hyl1iRCGoAbKz88P\nAQEBuHTpEoKDg3Hw4EGEh4drxaMDC7Wx1IGFTWzcuBFhYWF48OCB0d8tL/8NjP0IILJVyGHcuLFa\ngDXWsTrdmqDh8raDKXWJEIZgN/OmMVmVSoXCwkL87W9/E9MuQgTKysqQlZWFlJQUE4Yf7HV8hNC0\nS732R/9wke1CLsxtA3HqEmEsgt3MVSoVTpw4gZiYGNTX17eJYYn2xqJFi7B+/XpUVVVJbQqhg/Y3\nXK77LRow7k3aksPlVJekQXADBdArrzWzb98++Pr6Ijo62qJzXoRxtL/hct1nvQHGnfdmqeFyqkvS\nIbiBanrl/fvf/4533nlHTJsIETh+/DgyMzORlZWFR48eoaqqComJidi6dWuLeEJ75oWFP+jtCbdF\nDHummYf33nsPUVFRcHBwwOjRo8mF2cowd12yZSQ58p0xxuLj41lBQQHLzc1l48eP1wo3lDQEH0ds\n3iOjrf0IZCEI1ahbtwgGFHKUxSGB+gkrX33piYml8mnNhg0b2IwZM9iECRN02mUOTHneTam/usNM\nu1drrktCMaW8TNXBGjQS5CTR/JWX0YRhm8DW3nZsBZp8b3tQXbIcgob4pH3lFW+SVax8rXVit4lh\nw4Zh2LBhFs2T4AdNvrctqC5ZFkE7STTnyJEjePvtt7UWrplzJwndYY3hQm9Jit0trHn1e/fukbh2\n7UtwrYMCRsL4shJWhpbaScLSO1bs27cP2dnZ+Oijj5Cbm4sNGzZwLgC1xp0krLX+WmtdMiVdoeXV\nbneS4MqUsD5KS0uRmJiI27dvQyaT4eWXX8Zf/vIXqc0SEePfaqVyhOCC70gEQBPwXFhqNMLUemT4\nmZMDqDfZTptE8GyXAQwlDSudZLX2SUNjKC8vZ6dPn2aMMfbgwQMWHBzMLly4wDvvtuAkYWx6Qu02\nN7om35tsNgemPNPWWn/NgaF6ZChvPvdsjvIyNV1r0EjwThKlpaUYMWIEwsPDERERgffff19oUhbD\nzc0LMplM50c4DnrTdXPzEu0ejMHPzw9RUVEAABcXF4SGhuLGjRuS2EIYhkYirBOqR9IheIhPLpfj\n3XffRVRUFKqrq9G/f3/ExcUhNDRUTPtERf8Gl4DwTS7FW3xoLkpKSnD69GkMGjRIalOIZrQePnr/\n/fdtbBjWtqB6ZFkEN1B+fn7w8/MD0LJXYc0NVHuluroa8fHx2LhxI1xcXKQ2h2hGW+zotVcM1aPD\nhw9LYJVtI4qTBPUqrJf6+no8//zzmDVrFiZNmqQVbruT7/qXI+gn9/eP+aGOXtvAUD0CgGefnaP5\n28HBAw4Onnjy5JGlTLRKJDnyvTnV1dUYPnw4Xn/99RbCWaObOR+XTSlsMlECnTDGkJSUBG9vb7z7\n7rtG590W3MzFtkHXd8ylUXNKSkowbNgwnD9/vkUPndzM+WGucjJUj5ry5r6vUgDddIRpvq0nnNzM\nBdN+e+fCseRC3WPHjmHbtm3o27cvoqOjAQBr167F2LFjLZI/wR8ahrVeqB5Jh+A3KFN75/QG9YdN\nluidC8mb3qAM2yYG9fX1GD9+PJ599lksXLhQO3eZDMuXL9f8L1ZnzzrfoORodDrSpvXattadvRUr\nVkhal+gNyjDG/t4JbqC+//57PP300+jbt69mrL95r4IaKP42UQPVGNYeGyi+w0ftaYjPFJuogeJj\nE790raGBEjzEV11djeDgYDx8+BApKSlITU3VijNx4kzO77q5OQvNljCC/fv3Y+HChWhoaNCpESEt\nTcNHjo6O+Pjjj+Ht7Y309HSt4aNr165xft/X1xdOTk6WMLVdQ3WpOYYckETcGcOoZb2/o1arWVBQ\nECsuLmZ1dXVMpVJxrqwGtnF+5HLX38MPM36rmA/rCWsdx9QV3eawyeH3ONwfc8BXI3007iSxieO+\nmu8kwVVe4CgDXeVkim7i26ArH3PBV6dOnQK0Po6OnszBwVnvswXIDYQbr4d+TYzRhSuOPpv01yOp\n6xJ3WVwzUB76nlU+vy1Cf+/Ma5O+fI1B0E4SJ0+eRK9evRAYGAi5XI7p06dj7969HDFncn4cHJre\noHJ55sgnHt+0xEqHT7ymOE0Lebk+5oG/RoY4ZSA818zhfLAGG4TBV6eHD69pferqNkCtrkXjc7Qc\n3M9XfbO/W8fhIlfEu+OTljFx9NWjtlCXckWKY0w8MdLhmxffePwR1EBdv34dAQEBmv+VSiWuX78u\nmlGE6ZBGbQPSyfohjaRD0BwU/wWQGzmvqtU1QrIljEC8fd0Koa3jZZHSJkyrS/8npimEDkzTyDp2\nzm+zCBmT/eGHH9gzzzyj+X/NmjVs3bp1LeIEBQUZHC+mD1hQUJAQCUgjG9CIdGobOpFG0mkkqIGq\nr69nPXv2ZMXFxezx48eck4aEtJBGbQPSyfohjaRD0BCfg4MDPvzwQzzzzDNoaGjASy+9RHuHWRmk\nUduAdLJ+SCPpMHkvPoIgCIIwB4IPLNTH/v370adPH/Tu3RtpaWla4cnJyVAoFIiMbL1DwR/wPRDx\n0aNHGDRoEKKiohAWFoZly5bpTLOhoQHR0dGYMGECZ3hgYKBmv62BAwdyxqmsrER8fDxCQ0MRFhaG\nEydOaMW5ePEioqOjNR93d3dO+9euXYvw8HBERkZixowZePz4sU7bxcZUjQzpw1cXfZoY0sOQFnx0\nMKTBxo0bERkZiYiICGzcyO30Yw7o+W+Eq/y5ns27d+8iLi4OwcHBGDNmDCorK3WWg5jw0Yk0akSQ\nRmKPGfJZ1Hb06FFWUFDAIiIidKbD55jlJh4+fMgYaxwrHjRoEMvLy+OMt2HDBjZjxgw2YcIEzvDA\nwEBWUVGh9/4SExPZpk2bNPlVVlbqjd/Q0MD8/PzYtWvXWlwvLi5mPXr0YI8ePWKMMTZt2jS2ZcsW\nvWmJhRga8dGHjy76NDGkhzFacOlgSINz586xiIgIVltby9RqNRs9ejT79ddfdeYhJvT86y5/rmdz\nyZIlLC0tjTHG2Lp161hqaqpeu8SCr06kkTCNRH+D4rOoLTY2Fp6ennrTMeaY5Y4dOwIA6urq0NDQ\nAC8v7SPWy8rKkJWVhZSUFDA9o5r6wu7fv4+8vDwkJycDaBybdnd313sfBw8eRFBQUIt1FADg5uYG\nuVyOmpoaqNVq1NTUwN/fX29aYiGGRnz0MaQLH010XTdWCy4dDGnwyy+/YNCgQXBycoK9vT2GDRuG\n3bt368xDTOj5113+XM9mZmYmkpKSAABJSUnYs2ePXrvEgq9OpJEwjURvoMyxqM3QgYhPnjxBVFQU\nFAoFRowYgbCwMK04ixYtwvr162Fnp/uWZTIZRo8ejZiYGHz22Wda4cXFxfDx8cGcOXPQr18/zJ07\nFzU1+td07dy5EzNmzNC67uXlhcWLF6Nbt27o2rUrPDw8MHr0aL1piYXYGunSx5AuhjTRp4exWnDp\nYEiDiIgI5OXl4e7du6ipqcG3336LsrIyXmUiJu31+Tem/G/dugWFQgEAUCgUuHXrll67zIE+nUgj\nYRqJ3kCJt0C0ET7n5NjZ2eHMmTMoKyvD0aNHtc5b2rdvH3x9fREdHa2393Hs2DGcPn0a2dnZ+Oij\nj5CXl9ciXK1Wo6CgAPPmzUNBQQE6deqEdevW6Uyvrq4O33zzDaZOnaoVVlRUhPfeew8lJSW4ceMG\nqqursX37dj0lIR5iaqRPH3268NFEnx7GaKFLB0Ma9OnTB6mpqRgzZgyeffZZREdH6/3xMAft+fkX\nWv4ymUz03yFDGNKJNGoJX41Er23+/v4oLS3V/F9aWgqlUikoLT7HLDfH3d0dzz33HPLz81tcP378\nODIzM9GjRw8kJCTg0KFDSExM1Pp+ly5dAAA+Pj6YPHkyTp482SJcqVRCqVRiwIABAID4+HgUFBTo\ntCc7Oxv9+/eHj4+PVlh+fj6GDBkCb29vODg4YMqUKTh+/LjBexQDsTTiqw+XLnw00aeHMVro0oGP\nBsnJycjPz8eRI0fg4eGBkJAQHiUjDvT88y9/hUKBmzdvAgDKy8vh6+ur0y6xMUYn0sg4jURvoGJi\nYnD58mWUlJSgrq4Ou3btwsSJE41OhzGGl156CWFhYZyHuDVx584djTdIbW0tcnJyNKdeNrFmzRqU\nlpaiuLgYO3fuxMiRI7F169YWcWpqavDgwQMAwMOHD3HgwAEtDzY/Pz8EBATg0qVLABrHbsPDw3Xa\ntmPHDiQkJHCG9enTBydOnEBtbS0YYzh48CDna785EEMjQ/oY0sWQJob0MEYLXTrw0eD27dsAGo+7\n+PrrrzmHQswBPf+N8C3/iRMnIiMjAwCQkZHBq0EXAz46kUaNCNLIoBuFALKyslhwcDALCgpia9as\n0QqfPn0669KlC3N0dGRKpZKlp6drxcnLy2MymYypVCoWFRXFoqKiWHZ2tla8wsJCFh0dzVQqFYuM\njGRvvfXVrMXsAAAdlElEQVSWXttyc3M5PWSuXLnCVCoVU6lULDw8nNNuxhg7c+YMi4mJYX379mWT\nJ0/W6SFTXV3NvL29WVVVlU5b0tLSWFhYGIuIiGCJiYmsrq5Or+1iYqpGhvQxRhcuTfjowUcLQzoY\n0iA2NpaFhYUxlUrFDh06pPMexIae/0a4yr/p2ZTL5Zpns6Kigo0aNYr17t2bxcXFsXv37uktB7Hg\noxNpJFwjWqhLEARBWCWWnfElCIIgCJ5QA0UQBEFYJdRAEQRBEFYJNVAEQRCEVUINFEEQBGGVUANF\nEARBWCXUQBEEQRBWCTVQBEEQhFVCDRRBEARhlVADRRAEQVgl1EARBEEQVolNNVCBgYHo0KEDKioq\nWlxvOqPk6tWrePHFF/Ff//VfElnYfgkMDETHjh3h6uoKLy8vjB8/XpLD/wjdfPnll4iJiYGrqyu6\ndu2KcePG4dixY1KbRbRjbKqBkslk6NmzJ3bs2KG5du7cOdTW1moOx5LiMDOisdz37duHBw8eoLy8\nHAqFAgsWLJDaLOJ33nnnHSxatAivv/46bt++jdLSUsyfPx+ZmZlSm0a0Y2yqgQKAWbNmtThHJSMj\nA4mJiS1OqaQN3KWlQ4cOeP7553HhwgUAwPDhw7Fp0yZN+JYtWxAbGwugUatFixZBoVDA3d0dffv2\nxfnz5yWx21a5f/8+li9fjo8//hiTJk2Cs7Mz7O3t8dxzzyEtLQ2PHz/GwoUL4e/vD39/fyxatAh1\ndXUAgNzcXCiVSqxfvx6+vr7o2rUr9uzZg6ysLAQHB8Pb21vvia4EoQ+ba6CeeuopVFVV4ZdffkFD\nQwN27dqFWbNmSW0WgT86BjU1Ndi1axcGDx4MQP9b7YEDB5CXl4fLly/j/v37+Ne//gVvb2+L2dwe\n+OGHH/Do0SNMnjyZM3z16tU4efIkzp49i7Nnz+LkyZNYtWqVJvzWrVt4/PgxysvLsXLlSqSkpGD7\n9u04ffo08vLysHLlSly9etVSt0PYEDbXQAHA7NmzsXXrVuTk5CAsLAz+/v5Sm9TuYYxh0qRJ8PT0\nhIeHB7777ju8+uqrBr8nl8vx4MED/Pzzz3jy5AlCQkLg5+dnAYvbDxUVFejcuTPs7Lh/Dr788ku8\n8cYb6Ny5Mzp37ozly5fjiy++0ITL5XL8/e9/h729PV544QXcvXsXCxcuRKdOnRAWFoawsDCcOXPG\nUrdD2BA210DJZDLMnj0b27dv5xzeI6RBJpNh7969uHfvHh4/fowPPvgAw4YNw61bt/R+b+TIkfjP\n//xPzJ8/HwqFAn/60580x14T4uDt7Y07d+7gyZMnnOE3btxA9+7dNf9369YNN27caPH9pjdgZ2dn\nAIBCodCEOzs74+HDh+YwnbBxbK6BAhorUM+ePZGdnY0pU6ZohZOThLTIZDJMnjwZ9vb2+P7779Gp\nU6cWP2A3b95sEX/BggXIz8/HhQsXcOnSJaxfv97SJts0gwcPRocOHfD1119zhnft2hUlJSWa/69d\nu4auXbtayDqiPeMgtQHmYtOmTaisrISzszPUarXmOmMMarUajx490lyzs7ODo6OjFGa2K5reZBlj\nyMzMRGVlJcLCwhAVFYXdu3cjJSUF169fx6ZNm9ClSxcAQH5+PhoaGtCvXz907NgRTk5OsLe3l/I2\nbA53d3esXLkS8+fPh4ODA+Li4iCXy3Hw4EHk5uYiISEBq1atwoABAwAAK1euxOzZsyW2mmgP2GwD\n1bNnzxb/N3czX7duXQvPoqFDh+Lo0aMWta89MmHCBNjb20MmkyEwMBAZGRkIDQ3FokWLcOrUKSgU\nCqhUKsyaNQvfffcdAKCqqgqLFi3ClStX4OTkhLFjx2LJkiUS34nt8de//hV+fn5YtWoVZs6cCVdX\nV8TExODvf/87oqOjUVVVhb59+wIApk2bhtdff13z3dYjEjRCQYgG08OcOXOYr68vi4iIaHH9/fff\nZ3369GHh4eFs6dKl+pIgzIwujRhj7O2332YymYxVVFRIYBnRHNKJIIxH7xzUnDlzsH///hbXDh8+\njMzMTBQWFuKnn37i5YlFmA8ujQCgtLQUOTk5LSa3CekgnQjCePQ2ULGxsfD09Gxx7ZNPPsGyZcsg\nl8sBAD4+PuazjjAIl0ZA45DNW2+9JYFFBBekE0EYj9FefJcvX8bRo0fx1FNPYfjw4cjPzzeHXYQJ\n7N27F0qlUjNnQFgnpBNB6MdoJwm1Wo179+7hxIkTOHXqFKZNm4YrV66YwzZCADU1NVizZg1ycnI0\n1xitA7M6SCeCMIzRb1BKpVKztmjAgAGws7PT2j0cAHr16qXZwoY+uj+9evUyXcVmFBUVoaSkBCqV\nCj169EBZWRn69++P27dvk0ZWohHp1HZ0IqTF6AZq0qRJOHToEADg0qVLqKur49wbraioCIwxnZ9G\nlgNgHB+0iLt8+XK9aVk6jphpFRUVCRJOF5GRkbh16xaKi4tRXFwMpVKJgoIC+Pr6Gq0Rn3swd7g1\n2CC2RubQScgzLDS+JfIQYpM5dCKkRW8D1bNnTwQFBeH8+fMICAjA5s2bkZycjCtXrqBLly4ICQnB\nhx9+aClbCQ4SEhIwZMgQXLp0SaNRc2QyWpNiDZBOBGE8euegMjIy4OLigsTERJw7d05zfc2aNfjt\nt9/g5OSEoUOHmt1IQjc7duxAcnIyvv32W3h4eGDOnDkAgCVLlmDfvn1wdXVFSkoKNm/eDHd3d6PS\n/t///V+cPHmSsxMSGRmJYcOGiXIP7QFnZ2c0NDQgODhYU5eaNHJ0dERUVBTtkEEQrWEGKC4u1lpc\nGB8fz86ePcsCAwN1Li40lDQABhxmAOP4tPzu4cOHDZlp0ThipsVDAoMcPXqUFRQUtNDpwIEDrKGh\ngTHGWGpqKktNTTU67+7dI5iDwwjWocP8Fh+5/N9ZdPRwxpjhezQ13BJ5GAqXUiMh+fN9hnXFd3X1\n5Bp3Z66unqLlIXZ8xsTRibAujG6g9uzZwxYuXMgYYyI0UFyNU/t60MS6V66ORBO7d+9mM2fONDrv\nbt0iGFDIoc8hFhU1XBS72wJSaiRm/nzRXTetu15au32E8RjlZk6usW2T9PR0JCQkSG0GoQfSiCC0\nMaqBau4aC0DjGnvy5ElO76M333xT8/fw4cMxfPhwk4y1BXJzc5Gbm2ux/FavXg1HR0fMmDHDYnkS\nxkEaEQQ3RjVQTa6xTfTo0QM//vgjvLy8OOM3b6CIRlo31CtWrDBbXlu2bEFWVpZmZ3AuqBOhjSU7\nEXw0AkgnLizd2SMkQN/43/Tp01mXLl2Yo6MjUyqVLD09vUV4jx49aA7KRMS619bzG9nZ2SwsLIz9\n9ttvgvOmOahGpNRIzPz5ortuWne9tHb7COPRuw5qx44dGDt2LDw8PLRcmENDQzUuzPfv3zdjE0oY\nommNzcWLFxEQEID09HQsWLAA1dXViIuLQ3R0NObNmye1me0a0oggjEfGmH4vh7y8PK21UDk5ORg1\nahTs7Ozw2muvAUCLAwCBxoWH+pJuXJioK1z/d20JQ+UkZd7du0fi2rUvAUS2CjmMqKiVOH36sFnt\nsxak1EiK/HXXTeuul1LrRIiPwa2OuI4JiIuLg51d41cHDRqEsrIy81hH8CI5ORkKhQKRkX80JHfv\n3kVcXByCg4MxZswYVFZWSmghYasaubl5ce6L5+bGPS9NEMZg9F58rUlPT8e4cePEsIUQCNdheOvW\nrUNcXBwuXbqEUaNGab3hEpbFVjV68OAeuPbTbLxOEKZhUgNF7rHWAddbbmZmJpKSkgAASUlJ2LNn\njxSmEb9DGhGE8Rh9HlQT5MIsDEu5xt66dQsKhQIAoFAoWiwPIKwD0ogg9COogdq/fz/Wr1+PI0eO\nwMnJSWc8WgeljSXXQTXRNC9AWC+kEUFoo7eBSk5OxpdffomGhgbIZDIEBARg6dKlSE1NRX19PXr2\n7InAwEAMHToUH3/8saVsJnigUChw8+ZN+Pn5oby8nHOnD4Decrmw1FsuX40A0okLWqhr++h1M+dy\nMV+6dCk6d+6MpUuXIi0tDffu3eOc3CU3c36I5RpbUlKCCRMmtNDJ29sbqampWLduHSorK41eCkBu\n5o1IqZGY+fPFGDdza3JJJzdzG8TQSt7Wq99DQkLYzZs3GWOMlZeXs5CQEM7vGUoatJMEY0yce23a\n8UMul2t2/KioqGCjRo1ivXv3ZnFxcezevXtG5007STQipUZi5W8Muuumth3GxLWE3YRtYfQcFE3s\nWh87duzgvD5q1Chs27YN5eXlmDdvHjZv3owOHTpY2DoC0K3RwYMHsXbtWmzbtg2xsbGIjIwknQji\nd0xyM6eJXeulpKQEn332GQoKCnDu3Dk0NDRg586dUptFtIJ0IgjdGP0GRRO7pmGpiV03NzfI5XLU\n1NTA3t4eNTU18Pf3N3u+hHGQTgShG6MbqIkTJyIjIwOpqanIyMjApEmTdMYlN3NtLOVm7uXlhcWL\nF6Nbt25wdnbGM888g9GjR5slL0I4pBNB6EbvEF/rHZg3b96M1157DTk5OejcuTNWrVqFzMxMzJgx\nA48fP7aUzQQPioqK8N5776GkpAQ3btxAdXU1tm/fLrVZRCtIJ4LQjd43KF0Tu59//jlGjhyJ69ev\no0OHDnjhhRewc+dOzbYthPTk5+djyJAh8Pb2BgBMmTIFx48fx8yZM1vEo2FYbSy5voZ0Eg6tg7J9\nBO0kYf5xcwedzheurp6oqrorYl62SZ8+ffDf//3fqK2thZOTEw4ePIiBAwdqxaNhWG0sudsH6SQc\nKXZlISyLIC++5uPmXbt2hYeHh8jj5mpw7ZBMuyTzR6VSITExETExMejbty8A4OWXX5bYKqI1pBNB\n6MbggYVcFBUVYcKECcjLy4O7uzumTp2K+Pj4FsMSpu4k0V52maADC60fqXcooJ0k+CG1ToT4CBri\no3Fz4Vhy3LyyshIpKSk4f/48ZDIZ0tPT8dRTT1kkbzFxc/PifHO2leFeW9GJIMRGUANF4+bCseS4\n+SuvvIJx48bhq6++glqtxsOHD82Wlzn541C81tdtY5G4rehEEGIjaA6Kxs2tn/v37yMvLw/JyckA\nAAcHB7i7u0tsFdEa0okgdCN4q6OXX34ZoaGhUKvVOHXqFH788Ucx7SJMpLi4GD4+PpgzZw769euH\nuXPnoqamRmqziFaQTgShG8ENVNOwxM8//4zCwkKEhoaKaRdhImq1GgUFBZg3bx4KCgrQqVMnzqMc\nCGkhnQhCN4LmoJqGJTIyMhoToWEJq0OpVEKpVGLAgAEAgPj4eM4fPlt1ZDHFscKSjiy2q5P2Wkax\nnVpooa7tI8jN/MyZM/jTn/6EsLAwnD17Fv3798fGjRvRsWPHPxImN3NemNM19umnn8bnn3+O4OBg\nvPnmm6itrUVaWhrvvK3FzVyIK7OY7s/mdl82VSexEcvNXPu6ee+D3MxtD0FDfDQs0Tb44IMPMHPm\nTKhUKhQWFuJvf/ub1CYRHJBOBMGNoCE+2x2WMD+WHJZQqVQ4ceIEYmJiUF9fT8OwVkpERATUajW6\ndeuG3bt3S20OQVgNghooPz8/BAQE4NKlSwgODsbBgwcRHh6uFY/WQWlj6f3DNm7ciLCwMDx48MCs\n+RDCIY0IghvBXnxNwxLOzs749NNPaVjCCikrK0NWVhZSUlJobN5KIY0IQjeC3qCAxuGjhIQE/Pjj\nj3jw4IEFh49073QOmOYppMvzy9R0pWLRokVYv349qqqqpDaF0AFpRBC6EfwGJV3PT/dO56budv7H\nljptfxf1ffv2wdfXF9HR0dQzt1JII4LQj+A3KOr5WTfHjx9HZmYmsrKy8OjRI1RVVSExMRFbt25t\nEU+oI0th4Q963mTlAOqNuG7JN1TDZ41ZypGFr0aA6Q5H4my4q3/0wtLQOqh2ABPAN998w+bNm8cY\nY+zw4cNs/PjxWnEMJQ2AAUzHR2iY4XxNsckcmCvd5uTm5grSqFu3CAYUcpTFIYEaCStffekJ+Y6x\nNkipkVj5G1OGxpWdcXHNiSV0IiyLoDcoc/fObRmpen3W1PMluCGNCKIlgnaSaM6RI0fw9ttv45tv\nvmmZsBl3ktAd1hgu9JYM2WRiUenM0xzpAkBpaSkSExNx+/ZtyGQyvPzyy/jLX/7CO299O0kAI2G8\nRsLKV+ydJIy1QUqNxMrfnLtDGBPXXOUI0E4StojgOajmUM/POpHL5Xj33XcRFRWF6upq9O/fH3Fx\ncbSxrxUhvUbWNa9EEM0R7MVXWlqKESNGYN68ebhy5Qref/99Me0iRMDPzw9RUVEAABcXF4SGhuLG\njRsSW0U0R3qNuLxiCcI6EPwGJX3Pz3j0rXMyjL6epm7vNMA61lCVlJTg9OnTGDRokKR2ELohjQii\nJYLfoKTv+RmPvnVOhnuO+tZf1etNV+o1VNXV1YiPj8fGjRvh4uIiqS0EN6QRQWgjyhwU9fysl/r6\nejz//POYNWsWJk2apBXetj0txZ4/kWY+xpBGQFvXST9cIxvWdm4XIQ0me/FVV1dj+PDheP3111tU\nLplMhuXLl2v+b12ppPDi05+nKfnyt6l1pVqxYoXZPI8YY0hKSoK3tzfeffddbatswItPbBuM9RY0\nFUMaAeb34jPVM89ULz5u26zv3C7C8pjUQNXX12P8+PF49tlnsXDhwpYJW6GbuTU0UFw2matSff/9\n93j66afRt29fzZvB2rVrMXbsWF55UwNl2DZTMaQRQA0UX6iBsj0ED/ExxvDSSy8hLCxMq3EirIOh\nQ4fiyZMnUptB6IE0IgjdCG6gNmzYgC+++AKOjo7YtWsXFAqFVs8vPT2d87vOzs5CsyWMYP/+/Vi4\ncCEaGhqQkpKC1NRUqU0iOBCi0xtvrMJ7732gdb1nzyCcOXPcHGYShOURsj+SWq1mQUFBrLi4mNXV\n1TGVSsUuXLjQIg4A1qnTHM6PvX2H313cDjN++3gd1hPWOo6Q/dmap20OmxxauvW1+pgDvhrpo3Ev\nvk0c99V8Lz6u8gJHGegqJ1N0E98GXfmYC6E6vfBCMgM2MOBms8/PDJDredb4PvN89tczXGbaZayv\nHphe5ubUiZAGQW7mJ0+eRK9evRAYGAi5XI7p06dj7969WvEePkzn/Dg6Np0dlcszRz7x+KYlVjp8\n4jXF0eeibh74amSYUwbCc80czgdrsEEYpunkDkDR7OOLxiUPy2Hcc5YrwHJjv5ML3fWAILgR1EBd\nv34dAQEBmv+VSiWuX78umlGE6ZBGbQPSiSB0I2gOiu9aETe3CZzXHz5sW4f/tUXEWM/j4GAHuTwL\nzs4tdWxouIOHD01OnoBwnRwc7ODs/D7k8j2aa4zV4cEDsSwjCOkR1ED5+/ujtLRU839paSmUSmWL\nOEFBQSgq2mcgpRW/f7hoXXFX6AlrGUd/pTf0g2Aem3SFBwUFGbBHGHw14vMDWV9/TUdI03e5yqt5\nuobKyRTdxLWBK8xcGgGm61RbW8hxVdczrO8ZbR3fUHk3fY9vXGPS/f2KkY23OXUipEHQOii1Wo2Q\nkBB899136Nq1KwYOHIgdO3ZY9T587Q3SqG1AOhGEbgS9QTk4OODDDz/EM888g4aGBrz00ktUoawM\n0qhtQDoRhG5M3uqIIAiCIMyB4N3M9bF//3706dMHvXv3RlpamlZ4cnIyFAoFIiNbb6HzB03nTYWH\nhyMiIkLneVOPHj3CoEGDEBUVhbCwMCxbtkxnmg0NDYiOjsaECdzOG4GBgejbty+io6MxcOBAzjiV\nlZWIj49HaGgowsLCcOLECa04Fy9eRHR0tObj7u7Oaf/atWsRHh6OyMhIzJgxA48fP9Zpu9iYqpEh\nffjqok8TQ3oY0oKPDoY02LhxIyIjIxEREYGNGzdy3oO5MKRRa/jWmdYYqhfN4fP8t8ZQGXM9a3fv\n3kVcXByCg4MxZswYVFZW6o2/ZMkShIaGQqVSYcqUKbh//z6fWyesHbEXVvFZeHj06FFWUFDAIiIi\ndKZTXl7OTp8+zRhj7MGDByw4OFgrnSYePnzIGGOsvr6eDRo0iOXl5XHG27BhA5sxYwabMGECZ3hg\nYCCrqKjQe3+JiYls06ZNmvwqKyv1xm9oaGB+fn7s2rVrLa4XFxezHj16sEePHjHGGJs2bRrbsmWL\n3rTEQgyN+OjDRxd9mhjSwxgtuHQwpMG5c+dYREQEq62tZWq1mo0ePZr9+uuvOvMQEz4atcaYOtMc\nQ/WiOcY+/3yec65nbcmSJSwtLY0xxti6detYamqq3vgHDhxgDQ0NjDHGUlNTW8Qn2i6iv0HxWXgY\nGxsLT09PvekYc95Ux44dAQB1dXVoaGiAl5eXVpyysjJkZWUhJSVF74aS+sLu37+PvLw8JCcnA2ic\nP3B3d9cZHwAOHjyIoKCgFmtdAMDNzQ1yuRw1NTVQq9WoqamBv7+/3rTEQgyN+OhjSBc+mui6bqwW\nXDoY0uCXX37BoEGD4OTkBHt7ewwbNgy7d+/WmYeYCFnAK+SMNr71AhD2/PN5zrmetczMTCQlJQEA\nkpKSsGfPHr3x4+LiYGfX+HM2aNAglJWV6bWLaBuI3kCZY+GhofOmnjx5gqioKCgUCowYMQJhYWFa\ncRYtWoT169drHmIuZDIZRo8ejZiYGHz22Wda4cXFxfDx8cGcOXPQr18/zJ07FzU1NXpt37lzJ2bM\nmKF13cvLC4sXL0a3bt3QtWtXeHh4YPTo0XrTEguxNdKljyFdDGmiTw9jteDSwZAGERERyMvLw927\nd1FTU4Nvv/3WYj98pmrE94w2PvWiCSHPv9Dn/NatW1AoFAAAhUKBW7duGfxOE+np6Rg3bhzv+IT1\nInoDJfaBb3xOGrWzs8OZM2dQVlaGo0ePah1itm/fPvj6+iI6OlpvL/HYsWM4ffo0srOz8dFHHyEv\nL69FuFqtRkFBAebNm4eCggJ06tQJ69at05leXV0dvvnmG0ydOlUrrKioCO+99x5KSkpw48YNVFdX\nY/v27XpKQjzE1EifPvp04aOJPj2M0UKXDoY06NOnD1JTUzFmzBg8++yziI6O5vVDLgamaMT3dF6+\n9aIJY59/QJznXCaT8S6P1atXw9HRkbNTSLQ9RK9tfBYe8oXPSaPNcXd3x3PPPYf8/PwW148fP47M\nzEz06NEDCQkJOHToEBITE7W+36VLFwCAj48PJk+ejJMnT7YIVyqVUCqVGDBgAAAgPj4eBQUFOu3J\nzs5G//794ePjoxWWn5+PIUOGwNvbGw4ODpgyZQqOH7fMLtRiacRXHy5d+GiiTw9jtNClAx8NkpOT\nkZ+fjyNHjsDDwwMhISE8SsZ0hGpkTJ3hWy+aMPb5B4Q/5wqFAjdv3gQAlJeXw9fX1+B3tmzZgqys\nLIt19AjzI3oDFRMTg8uXL6OkpAR1dXXYtWsXJk6caHQ6jPE7b+rOnTsaD5/a2lrk5OQgOjq6RZw1\na9agtLQUxcXF2LlzJ0aOHImtW7e2iFNTU4MHv+8T8/DhQxw4cEDLg83Pzw8BAQG4dOkSgMZ5jfDw\ncJ227dixAwkJCZxhffr0wYkTJ1BbWwvGGA4ePMg5NGkOxNDIkD6GdDGkiSE9jNFClw58NLh9+zYA\n4Nq1a/j6668t1jMXohHfOtMEn3rRHGOff0D4cz5x4kRkZGQAADIyMgw2tvv378f69euxd+9eODk5\nGUyfaCOYw/MiKyuLBQcHs6CgILZmzRqt8OnTp7MuXbowR0dHplQqWXp6ulacvLw8JpPJmEqlYlFR\nUSwqKoplZ2drxSssLGTR0dFMpVKxyMhI9tZbb+m1LTc3l9Nb6cqVK0ylUjGVSsXCw8M57WaMsTNn\nzrCYmBjWt29fNnnyZJ1eTNXV1czb25tVVVXptCUtLY2FhYWxiIgIlpiYyOrq6vTaLiamamRIH2N0\n4dKEjx58tDCkgyENYmNjWVhYGFOpVOzQoUM678EcGNKoNXzrDBe66kVr+D7/zTFUxk3Pmlwu1zxr\nFRUVbNSoUax3794sLi6O3bt3T2f8TZs2sV69erFu3bpp7vs//uM/eN03Yd3QQl2CIAjCKrHMjC9B\nEARBGAk1UARBEIRVQg0UQRAEYZVQA0UQBEFYJdRAEQRBEFYJNVAEQRCEVUINFEEQBGGVUANFEARB\nWCX/DxIb1LMM/IshAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x106fd4e90>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"\n",
"d = pd.read_csv('/Users/briand/Desktop/ds course/datasets/survey.csv',header=0,index_col=0)\n",
"dpro = d[['profile_{}'.format(k+1) for k in range(7)]]\n",
"dpro.columns = ['Viz','CS','Math','Stats','ML','Bus','Com']\n",
"\n",
"fig=plt.figure()\n",
"for i in range(7):\n",
" plt.subplot(3,4,i+1)\n",
" hist(dpro[dpro.columns.values[i]])\n",
" plt.title(dpro.columns.values[i])\n",
"\n",
"fig.tight_layout()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>We can see that most categories have a full range of values. An important question is how correlated are the values to each other. We'll explore this question in two different ways.<br><br>\n",
"\n",
"First let's look at the correlation of the different categories.\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD7CAYAAACPDORaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG7RJREFUeJzt3XtYFGeeL/BvNdpR8IIYcRVJMGDiLDTQIGjA2ALRoCDG\nyxHxGcdgRvckB1yTEY96NoZoMt7GgJfdJ6tJRLNHLoOPzmSIGR0EdBwNEVtAkYhriJvlcrwsIiq4\nDXX+8LEigQakqyl5/X7+kXrr7bd+ZdNfiurmfSVZlmUQEZGQdFoXQERE9sOQJyISGEOeiEhgDHki\nIoEx5ImIBMaQJyISWB+tC3hU4ISJOPvNSa3LICLqVUwmE/Lz89vdJz1Jn5OXJAlnrv6X3cb/1483\n4h/eXWW38X2fqbPb2ACwbksq1iYtt9v4uS/PstvYAPBvN6vxS5cRdhvf+A8T7TY2AGw5+g2Spoy3\n2/iD/n6s3cYGgA/35+CfFkTZbfzB7xXZbWwAaK42w2GE0S5j3/kkxi7jPrRudwbWLplvt/H1E2bB\nWpTzdg0RkcAY8kREAnuqQj7wZfv+Om9vppAJWpdgE9/+A7QuwSYhL7hpXYJNJhnGaF2CTaQBf6d1\nCd1mCvDR7NhPVciP6+0hH9rbQ36g1iXYJNRzlNYl2GSS4UWtS7CJbqD93s+xN1MgQ56IiOyAIU9E\nJDCGPBGRwBjyREQCY8gTEQnssUK+pqYG8+fPh5eXF8aNG4eoqChUVFTYqzYiIrJRl+eukWUZs2bN\nQnx8PDIyMgAAJSUlqK2txZgxvfvzt0REourylXxeXh70ej2WLl2qtPn6+mLixIlISkqCwWCAr68v\nsrKyAAD5+fkwmUx4/fXX4enpiVWrVuGLL75AcHAwfH19ceXKFfXPhoiIWunylfz58+cRGBjYpv3A\ngQMoLi5GSUkJrl27hqCgIEyaNAnAgyv98vJyDBkyBKNHj8aSJUtQWFiI7du3Y8eOHUhJSVHvTIiI\nqI0uX8lLktRu+8mTJ7FgwQJIkgRXV1eYTCZ8++23kCQJQUFBGD58OPR6Pby8vPDaa68BAHx8fFBZ\nWanKCRARkXVdvpL39vZGdnZ2u/t+PsXlwx8IzzzzjNKm0+mUbZ1OB4vF0u5Y//rxRuXrwJcn9vqp\nCIiI1FZQdB4FZ893qW+XQz48PBxr1qzB7t27sWTJEgAPbsc4OzsjMzMTixYtwo0bN3D8+HH87ne/\nQ1lZWbeKt+d870REIjAF+rSaD+fDzzKt9n2slaEOHjyI5cuXY9OmTejXrx9Gjx6NlJQUNDQ0wM/P\nD5IkYcuWLXB1dcXFixet3uKRJMnqPiIiUs9TtTKUvdl7ZSh7s/fKUPZm75Wh7M3eK0PZm71XhrIn\ne68MZW9cGYqI6CnFkCciEhhDnohIYAx5IiKBMeSJiATGkCciEhhDnohIYAx5IiKBMeSJiATGkCci\nEhhDnohIYI81QVlPyDz3n1qX0G3Gl3r33DWOz/bXugSbOAdP0LoEmzgMGaZ1CTa584mb1iV0m9P/\n/KPWJdgNr+SJiATGkCciEhhDnohIYAx5IiKBMeSJiATGkCciEhhDnohIYAx5IiKBMeSJiATGkCci\nEhhDnohIYN0OeQcHBxiNRvj7+yMwMBCnTp1Ssy4iIlJBtycoc3R0hNlsBgAcOXIEq1evRn5+vlp1\nERGRClS5XXPr1i24uLgAAPLz8zFjxgxlX0JCAvbu3QsAWLVqFby9veHn54ekpCQ1Dk1ERB3o9pX8\nvXv3YDQa0djYiOrqauTl5bXbT5IkSJKEmzdv4tChQygvLwcA1NfXd/fQRETURd0O+f79+yu3a06f\nPo2FCxfi/PnzVvsPHjwY/fr1w5tvvono6GhER0e32+/k/n9WvnY3BOE5Q3B3SyQiElLL7WrIDTVd\n6qvKoiETJkzA9evXcf36dfTp0wctLS3KvsbGRsiyDAcHBxQWFiI3NxfZ2dnYuXMncnNz24wVuuB/\nqVESEZGwdANHAANHKNsttees9lUl5MvLy9Hc3IyhQ4fi+eefR1lZGe7fv4+7d+8iNzcXr7zyCu7c\nuYM7d+5g2rRpCAkJgaenpxqHJiKiDth8Tx4AZFnGvn37IEkS3N3dMW/ePPj4+GD06NEICAgAANy+\nfRszZ85UruxTUlLUOQMiIrKq2yFvsVis7tu0aRM2bdrUpv2bb77p7uGIiKgb+BevREQCY8gTEQmM\nIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDBJlmVZ6yIekiQJ\nX5Z1bY7kJ1HDfevz+fQG/2PwNa1LsI3c0nmfJ1jL/7uqdQk2kfo5al1Ct/33f1RoXYJNHGckwFqU\n80qeiEhgDHkiIoEx5ImIBMaQJyISGEOeiEhgDHkiIoEx5ImIBMaQJyISGEOeiEhgDHkiIoEx5ImI\nBKZayOt0OixcuFDZtlgsGDZsGGbMmAEASEtLQ2JiolqHIyKiLlAt5J2cnHDhwgU0NjYCAI4ePYpR\no0ZBkiQAUP4lIqKeo+rtmunTpyMnJwcAkJ6ejri4OGVmtCdosksioqeGqiEfGxuLjIwMNDU1obS0\nFOPHj1dzeCIiekx91BzMYDCgsrIS6enpiIqK6tYY+3du+Wm84BAYgkPVKo+ISAjHSy/heGnX5sBX\nNeQBICYmBitWrEBBQQGuXXv8RSgWJCSpXRIRkVAmGV7EJMOLyvZv0w9b7at6yC9evBhDhgyBt7c3\n8vPz1R6eiIgeg2r35B9+esbNzQ0JCQlK26OfrklLS4O7uzvc3d3x3HPPoaqqSq3DExFRO7jGq4q4\nxqvGuMarprjGq3a4xisR0VOKIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDCGPBGRwBjyREQCY8gT\nEQmMIU9EJDCGPBGRwFSfhdJW0/U/aF1Ct7399/O0LsEm8cHdWwPgSVG/fYrWJdhE5/qc1iXYxPLD\nRa1L6Lb6snKtS7AbXskTEQmMIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDCGPBGRwBjyREQCY8gT\nEQmMIU9EJDCGPBGRwDoN+Y8++gg+Pj7w8/OD0WhEYWEhtm3bhnv37nU6eGpqapf6ERGRfXQY8qdO\nnUJOTg7MZjOKi4uRm5uLUaNGITU1FXfv3u108G3btnWpHxER2UeHIV9TU4Nnn30Wffv2BQC4uLgg\nOzsbVVVVCAsLQ0REBADgrbfeQlBQEHx8fJCcnAwA2L59e6t+LS0teOONN2AwGODr64vU1FT7nhkR\nEXU81fDUqVOxbt06vPTSS3j11VcRGxuLZcuWISUlBfn5+XBxcQEA/Pa3v8WQIUPQ3NyMV199FefP\nn2/Tr6ioCFVVVSgtLQUA3Lp1y/5nR0T0lOsw5J2cnFBUVIQTJ04gLy8PsbGx2LBhAwBAlmWlX2Zm\nJnbv3g2LxYLq6mqUlZXBx8en1Vienp64cuUKli1bhqioKEydOrXdY36wbbfytWl8ACZPCOz2yRER\niejkv/+Iv135zy717XTREJ1OB5PJBJPJBIPBgLS0NACAJEkAgO+//x5bt27FmTNnMHjwYMTHx6Ox\nsbHNOM7OzigpKcHXX3+NTz75BFlZWfjss8/a9Hv/H5d0qXAioqdVqOcohHqOUra3/uVbq307vCd/\n6dIlVFRUKNtmsxkeHh4YOHAg6uvrAQD19fVwcnLCoEGDUFtbi8OHDyv9H+1348YNWCwWzJ49G+vX\nr8fZs2e7d3ZERNRlHV7JNzQ0IDExEXV1dejTpw/GjBmDXbt2Yf/+/YiMjISbmxtyc3NhNBoxduxY\nuLu7Y+LEicrjly5dqvRLSUlBfHw8WlpaAAAbN26075kREREk+dGb6xqTJAnNl7/Ruoxue9urd6/x\nupdrvGqKa7xqp67wtNYl2OTv/vdOWIty/sUrEZHAGPJERAJjyBMRCYwhT0QkMIY8EZHAGPJERAJj\nyBMRCYwhT0QkMIY8EZHAGPJERAJjyBMRCeyJm7tmtf4Frcvotn/a/5bWJdik7ygvrUuwyaBlR7Uu\nwSYNmb17mu2TsW9rXUK33b3eu9einnblHOeuISJ6GjHkiYgExpAnIhIYQ56ISGAMeSIigTHkiYgE\nxpAnIhIYQ56ISGAMeSIigTHkiYgExpAnIhJYpyGv0+mwcOFCZdtisWDYsGGYMWNGh48rLi7G4cOH\nle3k5GRs3brVhlKJiOhxdRryTk5OuHDhAhobGwEAR48exahRoyBJUoePM5vN+Oqrr5TtzvoTEZH6\nunS7Zvr06cjJyQEApKenIy4uTpnxrLCwECEhIQgICEBoaCguXbqE+/fvY+3atcjMzITRaERWVhYA\noKysDGFhYfD09MSOHTvsdEpERPRQl0I+NjYWGRkZaGpqQmlpKcaPH6/s+8UvfoETJ07g7Nmz+OCD\nD7BmzRro9XqsX78e8+fPh9lsxrx58yDLMsrLy3HkyBEUFhbigw8+QHNzs91OjIiIgD5d6WQwGFBZ\nWYn09HRERUW12ldXV4df/epXuHz5MiRJgsViAQDIstxqfmNJkhAdHY2+ffti6NChcHV1RW1tLUaO\nHNlqvBOWm8rXz+n643ld/26fHBGRiEru3UbJvYYu9e1SyANATEwMVqxYgYKCAly7dk1pf++99xAR\nEYGDBw/ihx9+wOTJk62Oodfrla8dHByUHwiPeqWPS1dLIiJ6Kvn2Hwjf/gOV7f9bV2u1b5dDfvHi\nxRgyZAi8vb2Rn5+vtNfX1ytX43v27FHaBw0ahNu3bz9O3UREpLJO78k//FSMm5sbEhISlLaH7StX\nrsTq1asREBCA5uZmpT0sLAxlZWWt3njlJ2yIiHoW13hVEdd41RbXeNUW13jVDtd4JSJ6SjHkiYgE\nxpAnIhIYQ56ISGAMeSIigTHkiYgExpAnIhIYQ56ISGAMeSIigTHkiYgExpAnIhLYEzd3TW1d1+ZI\nfhINKe/dc6f894+XtS7BJu/O3al1CTaZfO6k1iXYZF7/Sq1L6LZmZzetS7CJfvhozl1DRPQ0YsgT\nEQmMIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDDVQ76mpgbz\n58+Hl5cXxo0bh6ioKFRUVGDZsmUwGAzw9fVFcHAwKisr1T40ERH9TB81B5NlGbNmzUJ8fDwyMjIA\nACUlJcjIyEB1dTVKS0sBAFVVVXB0dFTz0ERE1A5Vr+Tz8vKg1+uxdOlSpc3X1xcDBgzAiBEjlLaR\nI0fC2dlZzUMTEVE7VA358+fPIzAwsE37vHnz8OWXX8JoNGLFihU4d+6cmoclIiIrVL1dI0lSu+1u\nbm747rvvcOzYMRw7dgwRERH4/e9/j/Dw8DZ9t2z4SPk6ZOIrCH1lkpolEhH1egUnT6Pgb6e71FfV\nkPf29kZ2dna7+/R6PSIjIxEZGYnhw4fj0KFD7YZ80ur/o2ZJRETCMYVOgCl0grL94e+2We2r6u2a\n8PBwNDU1Yffu3UpbSUkJjh8/jqqqKgBAS0sLiouL4eHhoeahiYioHap/hPLgwYP4y1/+Ai8vL/j4\n+GDNmjUoKSlBTEwMDAYD/Pz8oNfrkZCQoPahiYjoZ1S9XQMAI0aMQGZmZpt2hjoRUc/jX7wSEQmM\nIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDCGPBGRwFSfu8ZW\nQ8qPal1Ct/3X2Clal2CT1Em/0boEm/zL5SytS7DJV/efuJfjY1n1Xe9d7S3Wv/fW3hleyRMRCYwh\nT0QkMIY8EZHAGPJERAJjyBMRCYwhT0QkMIY8EZHAGPJERAJjyBMRCYwhT0QkMFVCPjw8HEeOHGnV\nlpqaihdeeAGbN29W4xBERNQNqoR8XFwcMjIyWrVlZmZi3759WLlypRqHICKiblAl5OfMmYOcnBxY\nLBYAQGVlJaqqqnD58mUkJiYCAPz9/WE0GmE0GuHo6IgTJ06ocWgiIuqAKiHv4uKC4OBgfPXVVwCA\njIwMxMbGQpIkpc+5c+dgNpuxbt06BAUFISQkRI1DExFRB1R74/XRWzaZmZmIi4tr06eiogIrV65E\nVlYWHBwc1Do0ERFZodoE1jExMXjnnXdgNptx9+5dGI1GFBcXK/sbGhoQGxuLTz/9FMOHD7c6zrrd\nP93bNwX4wBToo1aJRERCOHPqryg69dcu9VUt5AcMGICwsDDEx8djwYIFbfYvXrwY8fHxCA0N7XCc\ntUvmq1USEZGQxr08EeNenqhs707dZLWvqp+Tj4uLQ2lpqXKrRpIkSJKEq1ev4sCBA/j888+VN1/P\nnj2r5qGJiKgdqq43NnPmTDQ3NyvbixYtwqJFiwCgVTsREfUM/sUrEZHAGPJERAJjyBMRCYwhT0Qk\nMIY8EZHAGPJERAJjyBMRCYwhT0QksKcq5AuKzmtdgk1OnjiudQk2+aHlntYl2CT/dJHWJdiktPCk\n1iXY5GppodYldNuZLs4zYw9PV8if7d0h/7e/9u45+K/28pAv+KZ3T8VRWvg3rUuwyX+Ufqt1Cd3W\n1cnE7OGpCnkioqcNQ56ISGCSLMuy1kU8NHnyZBQUFGhdBhFRr2IymZCfn9/uvicq5ImISF28XUNE\nJDCGPBGRwIQL+fDwcBw5cqRVW2pqKl544QVs3rxZo6oeX01NDebPnw8vLy+MGzcOUVFRqKiowLJl\ny2AwGODr64vg4GBUVlZqVqNOp8PChQuVbYvFgmHDhmHGjBkdPq64uBiHDx9WtpOTk7F161a71dmR\njz76CD4+PvDz84PRaERhYSG2bduGe/c6/7hnampql/r1pM6ek7S0NCQmJmpVnlUODg4wGo3w9/dH\nYGAgTp06pXVJXWbttfqkUHVlqCdBXFwcMjIyMHXqVKUtMzMT+/btw8SJEzt45JNDlmXMmjUL8fHx\nyMh4sLB5SUkJMjIyUF1djdLSUgBAVVUVHB0dNavTyckJFy5cQGNjI/r164ejR49i1KhRkCSpw8eZ\nzWYUFRVh2rRpANBpf3s5deoUcnJyYDab0bdvX9y8eRONjY1ITU3FL3/5S/Tv37/Dx2/btg0LFy7s\ntF9P6uw50er/ujOOjo4wm80AgCNHjmD16tVW30h8klh7rdbW1mLMmDEaV/eAcFfyc+bMQU5ODiwW\nCwCgsrISVVVVuHz5snIF4+/vr6w16+joiBMnnqw/MsrLy4Ner8fSpUuVNl9fXwwYMAAjRoxQ2kaO\nHAlnZ2ctSlRMnz4dOTk5AID09HTExcXh4Xv5hYWFCAkJQUBAAEJDQ3Hp0iXcv38fa9euRWZmJoxG\nI7KysgAAZWVlCAsLg6enJ3bs2NEjtdfU1ODZZ59F3759AQAuLi7Izs5GVVUVwsLCEBERAQB46623\nEBQUBB8fHyQnJwMAtm/f3qpfS0sL3njjDeW3rNTU1B45h/Z09Jz0hs9Z3Lp1Cy4uLgCA/Pz8Vr8Z\nJiQkYO/evQCAVatWwdvbG35+fkhKStKkVmuv1YkTJyIpKUn5fnj4fZ6fnw+TyYTXX38dnp6eWLVq\nFb744gsEBwfD19cXV65cUb9IWUDR0dHyH/7wB1mWZXnDhg1yUlKSnJaWJickJLTq98c//lGeNGmS\nbLFYtCjTqm3btsnvvPNOm/Yff/xR9vDwkP39/eXf/OY3stls1qC6nwwYMEAuKSmR586dKzc2Nsr+\n/v5yfn6+HB0dLcuyLNfX1yv/t0ePHpXnzJkjy7Isp6WlyYmJico477//vhwSEiLfv39fvn79ujx0\n6NAeeU4aGhpkf39/+cUXX5TffvttuaCgQJZlWfbw8JBv3Lih9Lt586Ysy7JssVjkyZMny6WlpW36\nnTlzRp4yZYrymLq6OrvX357OnpM9e/a0eR08CRwcHGR/f3957Nix8uDBg+WzZ8/KsizLeXl5Su2y\nLMsJCQny3r175Rs3bsgvvfSS0n7r1q0er1mWrb9Ws7Oz5SlTpsgtLS1ybW2t/Nxzz8nV1dVyXl6e\n7OzsLNfU1MhNTU3yyJEj5ffff18Za/ny5arXKNyVPPDTLRvgwa2auLi4Nn0qKiqwcuVKZGVlwcHB\noadL7JC1X6nd3Nzw3XffYcOGDdDpdIiIiMCxY8d6uLrWDAYDKisrkZ6ejqioqFb76urqMHfuXBgM\nBrz77rsoKysD8OBqUn7kilKSJERHR6Nv374YOnQoXF1dUVtba/fanZycUFRUhF27dmHYsGGIjY1F\nWlqaUuNDmZmZCAwMREBAAC5cuKCcx6M8PT1x5coVLFu2DH/+858xaNAgu9dvTUfPyZOqf//+MJvN\nuHjxIr7++utW7yu0Z/DgwejXrx/efPNNHDx4ULNbZtZeqydPnsSCBQsgSRJcXV1hMpnw7bffQpIk\nBAUFYfjw4dDr9fDy8sJrr70GAPDx8bHLe2xChnxMTAxyc3NhNptx9+5dGI3GVi/ahoYGxMbG4tNP\nP8Xw4cM1rLR93t7eKCpqfzIsvV6PyMhIbN68GWvWrMGhQ4d6uLq2YmJisGLFila3BQDgvffeQ0RE\nBEpLS/Hll192+CalXq9XvnZwcFBut9mbTqeDyWRCcnIydu7ciQMHDgD46cX7/fffY+vWrTh27BiK\ni4sRFRWFxsbGNuM4OzujpKQEkydPxieffIJf//rXPVK/Ndaek95gwoQJuH79Oq5fv44+ffqgpaVF\n2dfY2AhZluHg4IDCwkLMnTsXf/rTnxAZGalJrR29Vn/+//7we+qZZ55R2nQ6nbKt0+ns8n0vZMgP\nGDAAYWFhiI+Px4IFC9rsX7x4MeLj4xEaGqpBdZ0LDw9HU1MTdu/erbSVlJTg+PHjqKqqAgC0tLSg\nuLgYHh4eGlX5k8WLFyM5ORne3t6t2uvr6zFy5EgAwJ49e5T2QYMG4fbt2z1aY3suXbrU6lMQZrMZ\nHh4eGDhwIOrr6wE8OAcnJycMGjQItbW1rT4V9Gi/GzduwGKxYPbs2Vi/fj3OntV2MjNrz0lvUF5e\njubmZgwdOhTPP/88ysrKcP/+fdTV1SE3NxeSJOHOnTuoq6vDtGnT8PHHH6O4uFiTWq29Vp2dnZGZ\nmYmWlhZcu3YNx48fR3BwsCY/cIX7dM1DcXFxmD17tvKGhyRJkCQJV69exYEDB1BRUYHPP/8cAPDZ\nZ58hICBAy3LbOHjwIJYvX45NmzahX79+8PDwQGRkJN599100NTUBAMaPH4+EhATNanx4ZeLm5qbU\n8fD/GQBWrlyJRYsW4cMPP0RUVJTSHhYWho0bN8JoNGL16tWtxupJDQ0NSExMRF1dHfr06YMxY8Zg\n165d2L9/PyIjI+Hm5obc3FwYjUaMHTsW7u7urT6htXTpUqVfSkoK4uPjlavOjRs39vj5AJ0/J5Ik\nIS0tTfkNUJIknD59WvlhrJV79+7BaDQCeHAFvG/fPkiSBHd3d8ybNw8+Pj4YPXq08jq9ffs2Zs6c\nqVzZp6SkaFb7z1+ro0ePRkpKChoaGuDn5wdJkrBlyxa4urri4sWLVr/XH32e1MRpDYiIBCbk7Roi\nInqAIU9EJDCGPBGRwBjyREQCY8gTEQmMIU9EJDCGPBGRwBjyREQC+//Y0zMdNjrHHAAAAABJRU5E\nrkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x108e2a050>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from matplotlib import pyplot as plt\n",
"\n",
"#Get correlation and 0 out the diagonal (for plotting purposes)\n",
"c_mat = dpro.corr()\n",
"for i in range(c_mat.shape[0]):\n",
" c_mat.iloc[i,i] = 0\n",
"\n",
"fig, ax = plt.subplots()\n",
"heatmap = ax.pcolor(c_mat, cmap=plt.cm.RdBu)\n",
"\n",
"#Set the tick labels and center them\n",
"ax.set_xticks(np.arange(c_mat.shape[0])+0.5, minor=False)\n",
"ax.set_yticks(np.arange(c_mat.shape[1])+0.5, minor=False)\n",
"ax.set_xticklabels(c_mat.index.values, minor=False)\n",
"ax.set_yticklabels(c_mat.index.values, minor=False)\n",
"\n",
"#heatmap.set_label('Pearson Correlation')\n",
"\n",
"#plt.legend()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>We can see a range of correlations, while we also see two sets of strongly correlated features. These relationships are between \"business/communication\" and \"math/stats.\" It is perhaps not surprising that students that rank themselves on one aspect of each of these sets might rank themselves highly on the other. We can also see that these two groups of correlated features are uncorrelated with each other. From this we can certainly sense that distinct segments of the student population might be exist.\n",
"<br><br>\n",
"\n",
"##Latent Variables\n",
"\n",
"With this range of correlation in the data, we might wonder whether certain latent features might exist that can explain the above observations. A latent feature (or variable) is described by <a href=\"http://en.wikipedia.org/wiki/Latent_variable\">Wikipedia</a> as: \"...latent variables (or hidden variables, as opposed to observable variables), are variables that are not directly observed but are rather inferred (through a mathematical model) from other variables that are observed (directly measured). As we see above, students rank themselves very similarly in \"math\" and \"stats\". Both \"math\" and \"stats\" are the observed feature. The latent feature might be some sort of intellectual capacity for abstraction and logic. This hidden feature is of course manifested, and thus observed, in the form of skill in two related academic disciplines. <br><br>\n",
"\n",
"One way to detect and define the latent features is through a decomposition of the observed features. Our student survey results are stored in a matrix $X$. Ideally, latent features will all be independent, and each observed feature might be a linear combination of the latent features. One straightfoward mechanism to to mathematically arrive at the properties just described is via the singular value decomposition. Here is a <a href=\"http://nbviewer.ipython.org/github/briandalessandro/DataScienceCourse/blob/master/ipython/Lecture3_PhotoSVD.ipynb\">review</a> of the SVD and an example of a potential use case.<br><br>\n",
"For our exploratory analysis of the survey data, we'll use the SVD to define independent features (basically, latent features).<br><br>\n",
"This starts with the basic decomposition. We'll also generate a scree plot to get a sense of how important the various latent features are to the overall distribution of the data.\n"
]
},
{
"cell_type": "code",
"execution_count": 115,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEZCAYAAACKF66QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVOX+wPEPCG7sqKCARmmyqAiKIlqK5sJ1odxSNEUz\nbrfyppVp2SJ6u4lbXc1bmbvWRW1zl0wNt1T0iqhXBTURFEGRHVQEzu+P+TExrDNsw+j3/XqdF3Pm\nnPOc7wzMl2eec57nMVIURUEIIYTBMtZ3AEIIIapHErkQQhg4SeRCCGHgJJELIYSBk0QuhBAGThK5\nEEIYOEnkotasW7eOZ599tkbK+u677xg0aFCNlFWRuLg4jI2NKSwsrPVzVSYkJIQJEyboOwxhACSR\nC605Ozuzf/9+9fqmTZuwtbXl8OHDNZIAjxw5Qs+ePbG2tqZZs2Y888wznDp1CoDx48fzyy+/VPs1\n1KXjx49jbm5OTk5OqW1eXl58+eWXFR5vZGRUW6GJR4wkcqE1IyMjdXJZv349U6dOZffu3Rq17qr2\nL8vMzGTo0KFMmzaNtLQ0bt68yZw5c2jUqFGNxF4X8vPzNdZ79OiBk5MTP/zwg8bz58+f5+LFiwQG\nBlZYnvTVE9qSRC50oigKK1asYMaMGezdu5cePXoA0Lt3bwCsra2xtLTk+PHj6qT/7rvvYmtry1NP\nPUV4eHiZ5cbGxmJkZMSYMWMwMjKicePGDBgwgE6dOgGlm2mMjY1ZsWIF7du3x8bGhqlTp6q3FRYW\n8s4779CiRQueeuopli9frvFtoeQ3i4qaMNauXYu7uzuWlpa0bduWb775Rr0tIiICJycnFi5cSKtW\nrZgyZUqp44OCgtiwYYPGcxs2bGDIkCHY2Ngwbdo02rRpg5WVFd7e3hw5cqTMOCIiImjdurXGc8Vf\nh6IohIaG0q5dO5o3b86YMWNIS0srsyzx6JFELnTy5ZdfMmfOHA4cOECXLl3Uzx8+fBiAjIwMMjMz\n6dGjB4qicOLECVxdXbl79y4zZ84sM9kBuLi40KBBAyZNmkR4eLhWSWjXrl2cOnWKs2fPsmXLFnXT\nyzfffEN4eDjR0dGcPn2arVu3ajRTFP9mUbReHnt7e3bt2kVmZiZr167lrbfeIioqSr09OTmZtLQ0\n4uPjWbFiRanjX3rpJQ4dOsSNGzcA1T+ZsLAwgoKCAOjevTvR0dGkpaUxbtw4Ro8eTV5eXqWvveTr\nWLZsGdu3b+fQoUPcunULGxsb3njjDa3KEYZPErnQmqIo7Nu3D19fXzp27FhqW1meeOIJpkyZgpGR\nERMnTuTWrVvcvn271H4WFhYcOXIEIyMjgoODsbOz4/nnny9z3yLvvfcelpaWtG7dmr59+xIdHQ3A\nli1bmD59Og4ODlhbW/P+++9X2ExR0bbBgwfz5JNPAqpvHQMHDlT/0wLVN4O5c+diampK48aNSx3f\nunVr/Pz82LhxIwD79+/nwYMHDBkyBFC1/dvY2GBsbMzbb7/NgwcPiImJKTee8qxYsYJPPvkEBwcH\nTE1NmTNnDj/88EO9uGgrap8kcqE1IyMjvv76a2JiYnjllVe0OqZly5bqx02bNgUgOzu7zH1dXV1Z\nu3YtCQkJnD9/nsTERKZPn6512UXl3rp1S6MZwsnJSatYy7Jnzx569OhBs2bNsLGxYffu3dy9e1e9\nvUWLFjRs2LDCMoKCgtSJfOPGjQQGBtKgQQMAFi9ejLu7O9bW1tjY2JCRkUFKSorOccbFxTF8+HBs\nbGywsbHB3d0dExMTkpOTdS5LGB5J5EIn9vb27N+/n8OHD/P666+rn6/pOyxcXFwICgri/PnzOh/b\nqlUrEhIS1OvFHwOYmZlp3EmSlJRUZjkPHjxg5MiRzJw5k9u3b5OWlsbgwYM1avDavO7hw4dz48YN\nfvvtN37++Wd1s8rhw4dZtGgR33//Penp6aSlpWFlZVXmNwQzMzNyc3PV6wUFBdy5c0e93qZNG3WT\nVNGSm5tLq1atKo1PGD5J5EJnrVq1Yv/+/YSHh/P2228DqpqpsbExV69erVKZMTExfPbZZ9y8eRNQ\nJd+wsDB8fX21Ol5RFHUCfPHFF1m6dCmJiYmkp6ezYMECjYTr6enJpk2byM/P59SpU/z4449lJuS8\nvDzy8vJo3rw5xsbG7Nmzh7179+r82szMzBg1ahSTJ0/G2dlZfW0hKysLExMTmjdvTl5eHvPmzSMz\nM7PMMtq3b8/9+/fZvXs3Dx8+5JNPPuHBgwfq7X/729+YPXs28fHxANy5c4ft27frHKswTJLIRZW0\nbt2aAwcO8MMPP/DBBx/QtGlTPvjgA3r16oWtrS0nTpwodVERyq/BWlhYcOLECXx8fDA3N8fX1xcP\nDw+WLFmiPq6iC5TFtwcHBzNw4EA8PDzo2rUrQ4YMoUGDBhgbq/7c//GPf3D16lVsbGwICQlh/Pjx\nZcZoYWHBsmXLePHFF7G1tSUsLIznn39eq9dTUlBQEPHx8UycOFH9nL+/P/7+/rRv3x5nZ2eaNGlC\nmzZtynxNVlZWfPnll7zyyis4OTlhbm6u0Xw0bdo0AgICGDhwIJaWlvj6+hIZGalVbMLwGek6scTS\npUtZtWoViqIQHBzMtGnTSE1NZcyYMVy/fh1nZ2e2bNmCtbV1bcUshE727NnDa6+9RlxcnL5DEaJW\n6FQjP3/+PKtWreLkyZNER0ezc+dOrl69SmhoKAMGDCA2NpbnnnuO0NDQ2opXiEoVNUHk5+dz8+ZN\n5s6dy4gRI/QdlhC1RqdEfunSJXx8fGjcuDENGjSgT58+/Pjjj2zfvl19AScoKIitW7fWSrBCaENR\nFEJCQrC1taVLly506NCBefPm6TssIWqNTk0rly5d4vnnn+fYsWM0btyY/v374+3tzcaNG9UdOBRF\nwdbWVnqVCSFEHTHRZWdXV1dmzZrFwIEDMTMzw9PTU30/bJGyLnAJIYSoRUo1zJ49W/nyyy8VFxcX\n5datW4qiKEpiYqLi4uJSat/OnTsrgCyyyCKLLDosnTt3rjQX61QjB7h9+zZ2dnbEx8fz008/cfz4\nca5du8b69euZNWsW69ev54UXXih1XFH36eqwsLAhMzO12uUIIYSh0KaFQ+fbD3v37s3du3cxNTXl\n888/p2/fvqSmpvLiiy8SHx9f7u2HqmB0OlVZ4crQnkKIx4qRUeV5T+dEXp1gJJELIYRutEnk0rNT\nCCEMnCRyIYQwcDon8vnz59OhQwc6derEuHHjePDgAampqQwYMID27dszcOBA0tPTayNWIYQQZdAp\nkcfFxbFy5UpOnz7NuXPnKCgoYNOmTdJFXwgh9EinRG5paYmpqSm5ubnk5+eTm5uLg4ODdNEXQgg9\n0imR29ra8s4779CmTRv1NFoDBgwgOTkZe3t7QDXxgMxKIoQQdUenRH716lX+9a9/ERcXR2JiItnZ\n2Xz77bca+0gXfSGEqFs69ew8deoUPXv2pFmzZgCMGDGCY8eO0bJlS5KSkmjZsiW3bt3Czs6unBJC\nij32+/9FCCFEkYiICCIiInQ6RqcOQdHR0YwfP56TJ0/SuHFjJk2aRPfu3bl+/TrNmjVj1qxZhIaG\nkp6eXuqCp3QIEkII3dVKz86FCxeyfv16jI2N6dKlC6tWrSIrK0u66AshRC2QLvpCCGHgpIu+EEI8\nBiSRCyGEgZNELoQQBk6nRB4TE4OXl5d6sbKyYtmyZTLWihBC6FGVL3YWFhbi6OhIZGQkX3zxBc2b\nN2fmzJksWLCAtLQ0uf1QCCFqQK1e7Ny3bx/t2rWjdevWMtaKEELoUZUT+aZNmwgMDASQsVaEEEKP\ndJ58GSAvL48dO3awYMGCUtsqHmslpNhjP6SLvhBCaKr1LvpFtm3bxldffUV4eDgArq6uREREqMda\n6du3L5cuXdI8kbSRCyGEzmqtjTwsLEzdrAIQEBDA+vXrAVi/fj0vvPBCVYoVQghRBTrXyHNycnji\niSe4du0aFhYWAKSmpspYK0IIUQtkrBUhhDBwMtaKEEI8BiSRCyGEgdM5kaenpzNq1Cjc3Nxwd3fn\nxIkT0kVfCCH0SOdEPm3aNAYPHszFixc5e/Ysrq6uhIaGMmDAAGJjY3nuuedKdc8XQghRe3S62JmR\nkYGXlxd//PGHxvOurq4cPHgQe3t7kpKS8PPzk/vIhRCiBtT4xc5r167RokULJk+eTJcuXQgODiYn\nJ0e66AshhB7p1EU/Pz+f06dPs3z5crp168b06dPLHOVQuugLIUTV1HoX/aSkJHx9fbl27RoAR44c\nYf78+fzxxx/89ttv0kVfCCFqWI03rbRs2ZLWrVsTGxsLqIay7dChA8OGDZMu+kIIoSc69+yMjo7m\nlVdeIS8vj7Zt27J27VoKCgqki74QQtQC6aIvhBAGTrroCyHEY0ASuRBCGDidZwhydnbG0tKSBg0a\nYGpqSmRkJKmpqYwZM4br16+X20YuhBCiduhcIzcyMiIiIoKoqCgiIyMBpIu+EELoUZWaVko2vG/f\nvp2goCAAgoKC2Lp1a/UjE0IIoZUq1cj79++Pt7c3K1euBJAu+kIIoUc6t5EfPXqUVq1acefOHQYM\nGICrq6vG9oq76AshhKhpOifyVq1aAdCiRQuGDx9OZGSketTDoi76dnZ25RwdUuyxHzLWihBCaKr1\nsVZyc3MpKCjAwsKCnJwcBg4cyJw5c9i3bx/NmjVj1qxZhIaGkp6eXuZgWtIhSAghdFPjPTuvXbvG\n8OHDAdVIiOPHj+f9998nNTVVuugLIUQtkC76Qghh4KSLvhBCPAYkkQshhIGTRC6EEAZO50ReUFCA\nl5cXw4YNAyA1NZUBAwbQvn17Bg4cSHp6eo0HKYQQonw6J/KlS5fi7u6u7vQj46wIIYR+6ZTIb9y4\nwe7du3nllVfUV1FlnBUhhNAvnRL5W2+9xaJFizA2/vMwGWdFCCH0S+su+jt37sTOzg4vL69yu49W\nPs5KSLHHfkgXfSGE0FSrXfRnz57Nxo0bMTEx4f79+2RmZjJixAhOnjxJRESEepyVvn37cunSpdIn\nkg5BQgihs1rr2Xnw4EEWL17Mjh07mDlzZqXjrBQFI4lcCCF0U6s9O4uaUN577z1+/fVX2rdvz4ED\nB3jvvfeqWqQQQogqkLFWhBCiHpOxVoQQ4jEgiVwIIQycTon8/v37+Pj44Onpibu7O++//z4g3fSF\nEEKfdG4jz83NpWnTpuTn5/PMM8+wePFitm/fTvPmzZk5cyYLFiwgLS1NZggSQogaUCtt5E2bNgUg\nLy+PgoICbGxspJu+EELokc6JvLCwEE9PT+zt7enbty8dOnSQbvpCCKFHWnfRL2JsbMyZM2fIyMhg\n0KBB/PbbbxrbK+6mH1LssR/SRV8IITTVahf9svzjH/+gSZMmrFq1qtJu+tJGLoQQuqvxNvKUlBT1\nHSn37t3j119/xcvLi4CAANavXw/A+vXreeGFF6oYshBCCF3pVCM/d+4cQUFBFBYWUlhYyIQJE3j3\n3XdJTU3lxRdfJD4+HmdnZ7Zs2YK1tbXmiaRGLoQQOqu1QbOqGowkciGE0I100RdCiMeAJHIhhDBw\nOiXyhIQE9b3jHTt2ZNmyZYB00RdCCH3SqY08KSmJpKQkPD09yc7OpmvXrmzdupW1a9dKF30hhKgF\nNd5G3rJlSzw9PQEwNzfHzc2NmzdvShd9IYTQoyrftRIXF0efPn04f/48bdq0IS0tDQBFUbC1tVWv\nq08kNXIhhNBZrd21kp2dzciRI1m6dCkWFhalTlp+F30hhBA1TeexVh4+fMjIkSOZMGGCugenvb09\nSUlJ6i76dnZ25RwdUuyxH4Y01oqlpS1ZWWmV71gBCwsbMjNTaygiIcSjqNbHWlEUhaCgIJo1a8bn\nn3+ufn7mzJk0a9aMWbNmERoaSnp6+iN3sdPQ4xdCGKYa79l55MgRevfujYeHh7r5ZP78+XTv3v2R\n76Jv6PFX9xuFfJsQQj+ki34Nkvjl24QQ+iBd9IUQ4jEgiVwIIQycTon85Zdfxt7enk6dOqmfk+75\norZZWtqqb2ut6mJpaavvlyFErdEpkU+ePJnw8HCN50JDQxkwYACxsbE899xzpe5WEaK6VBdplWot\n1b11VIj6TOeLnXFxcQwbNoxz584B4OrqysGDB9X3kvv5+ZWa5g3kYuH/l2LA8Rty7CAXa4WhqpOL\nncnJydjb2wOqjkHJycnVLVKIR4o0DYnapnPPzopI93whSvuzaag6ZcjnSpSv2olc++75YMhd9IV4\nXMnwFHWr1rvoQ+k2cm2654Pht3NK/IYcO0j81Tizgcdv6Gq8Z2dgYCAHDx4kJSUFe3t75s2bx/PP\nP19p9/yiYAz5j0HiN+TYQeKvxpkNPH5DJ130a5DEb8ixg8RfjTMbePyGTrroCyEee4/DXUNSI9f2\nzI99/IYcO0j81TizxI++45cauRBCPOJqLJGHh4fj6urK008/zYIFC2qqWCGEEJWokaaVgoICXFxc\n2LdvH46OjnTr1o2wsDDc3Nz+PNEj8PXm8Y7fkGMHib8aZ5b40Xf8ddK0EhkZSbt27XB2dsbU1JSx\nY8eybdu2mihaCCFEJWokkd+8eZPWrVur152cnLh582ZNFC2EEKISNTLWijbjq3Tu3Jno6OqPF6Hf\nsVwe7/gNOXaQ+Kt59uqXIPFXSefOnSvdp0YSuaOjIwkJCer1hIQEnJycNPY5c+ZMTZxKCCFECTXS\ntOLt7c3ly5eJi4sjLy+PzZs3ExAQUBNFCyGEqESN1MhNTExYvnw5gwYNoqCggClTpmjcsSKEEKL2\n1FnPTiGEELVDenYKIYSBk0ReS/z8/Fi9ejUA3333HYMGDarR8uPi4jA2NqawsLBGy60LP//8M61b\nt8bCwoLo6Gh9h6N3X331Ffb29lhaWpKWVv1JoidNmsRHH31UA5HBa6+9xieffFIjZVVk3bp1PPvs\ns7V+Hm0U/+waCoNN5M7Oztjb25Obm6t+btWqVfTt21ePUf2p+LR348eP55dffqnT8x85coSePXti\nbW1Ns2bNeOaZZzh16lSdxlCeGTNm8OWXX5KVlVXmrVWrV6/Gzc0NS0tLWrZsyZAhQ8jOztZDpLXv\n4cOHvPPOO+zfv5/MzExsbGw0tpf8h60oCn//+99xc3MjMTGxzASo65SLFb3fX331FR9++GE1X2Xd\nCg0NpU+fPqWeT0lJoWHDhly4cKHC4w1xykqDTeQAhYWFLF26tNrlKIrySI2VnJmZydChQ5k2bRpp\naWncvHmTOXPm0KhRI32HhqIoxMfH4+7uXub2gwcP8sEHH7Bp0yYyMzO5ePEiY8eOreMoIT8/v07O\nk5SUxP3797W6OaCwsJBXX32VQ4cOcejQIRwcHKp9/vryfldHQUGBxvqECRP4/fffiYuL03h+06ZN\ndO7cudy/PUNmsIncyMiIGTNmsHjxYjIyMsrc5/fff6dbt25YW1vTvXt3jh07pt7m5+fHhx9+SK9e\nvTA3N+ePP/7A2NiYr776iqeffhpLS0s+/vhjrl69iq+vL9bW1owdO5aHDx8CkJ6eztChQ7Gzs8PW\n1pZhw4aV25u1eK1p4cKFWFhYqBdTU1MmT54MQEZGBlOmTMHBwQEnJyc++ugjdU2ssLCQGTNm0KJF\nC9q2bcuuXbvKfW9iY2MxMjJizJgxGBkZ0bhxYwYMGECnTp0ACAkJYcKECer9S9b6/Pz8+Oijj+jV\nqxcWFhYEBASQkpLC+PHjsbKyonv37ly/fr3c8yuKwieffKL+1hQUFERmZiYPHjzAwsKCgoICOnfu\nzNNPP13q2JMnT+Lr66uuqdvY2DBhwgTMzc0BuHv3LgEBAVhZWeHj48NHH32kfm/Lam4q/jX56tWr\n9OvXj+bNm9OiRQteeukljb8dZ2dnFi5ciIeHBxYWFhQWFnL8+HF69uyJjY0Nnp6eHDx4UOP32rZt\nWywtLXnqqaf4z3/+U+b78eDBA6ZPn46joyOOjo689dZb5OXlERsbq07g1tbW9O/fv9z3ND8/n8mT\nJ3P69GkiIiJo0aIFFy9e5LXXXuPYsWNYWFhga/vnmNmpqakMHToUS0tLevTowR9//FFmuZW938Wb\naSIiInBycuKzzz7D3t4eBwcH1q1bpy7r7t27DBs2TP038uGHH2r9uylp2rRptGnTBisrK7y9vTly\n5Ih6W0hICKNGjWLChAlYWVmxfv16jWMdHR3p168fGzdu1Hh+w4YNTJw4UafPbmWflYo+s3XJYBM5\nqO5f9/PzY/HixaW2paamMmTIEKZPn05qaipvv/02Q4YM0WiD/Pbbb1m1ahVZWVm0adMGgL179xIV\nFcXx48dZsGABwcHBhIWFER8fz7lz5wgLCwNUiXXKlCnEx8cTHx9PkyZNmDp1aqUxz5w5k6ysLLKy\nsrh48SJ2dnbqGtCkSZNo2LAhV69eJSoqir1797Jq1SoAvvnmG3bt2sWZM2c4deoUP/zwQ7lf/1xc\nXGjQoAGTJk0iPDy8VLurNl8bN2/ezLfffsvNmzfV/8ymTJlCamoqbm5uzJ07t9xj165dy/r164mI\niOCPP/4gOzubqVOn0qhRI/VX9rNnz3L58uVSx/bo0YNffvmFkJAQjh49yoMHDzS2v/HGGzRt2pSk\npCTWrFnD2rVrK3w9Jb8mf/DBB9y6dYuLFy+SkJBASEiIxv6bNm1iz549pKenc+vWLYYOHcrHH39M\nWloaixcvZuTIkdy9e5ecnBymTZtGeHg4mZmZHDt2DE9PzzJj+Oc//0lkZCTR0dFER0cTGRnJJ598\nQvv27fnf//4HqBLCvn37yn0d48aN4/Llyxw4cEDd/OLm5sbXX3+Nr68vWVlZpKaqJjdWFIVNmzYR\nEhJCWloa7dq144MPPiiz3Mre75LvX3JyMpmZmSQmJrJ69WreeOMN9T/DN954AwsLC5KTk1m/fj0b\nNmzQ6XdTXPfu3YmOjiYtLY1x48YxevRo8vLy1Nu3b9/O6NGjycjIYNy4caWODwoK0kjkMTExREdH\nM27cOJ0+u5V9Vir6zNYpxUA5Ozsr+/fvV86fP69YWVkpd+7cUVauXKn4+fkpiqIoGzZsUHx8fDSO\n8fX1VdatW6coiqL4+fkpc+bM0dhuZGSk/P777+r1rl27KgsXLlSvv/POO8r06dPLjCcqKkqxsbFR\nr/v5+SmrV69WFEVR1q5dqzzzzDMa++fm5ipdunRRl5+UlKQ0atRIuXfvnnqf//znP0rfvn0VRVGU\nvn37KitWrFBv27t3r2JkZKQUFBSUGc/FixeVSZMmKU5OToqJiYkSEBCgJCcnK4qiKHPmzFFeeukl\n9b7Xrl3TKMvPz0/59NNPNV734MGD1es7duxQPD09yzyvoihKv379lK+++kq9HhMTo5iamqrLNzIy\nUq5evVru8Xv27FGGDRumWFtbK+bm5srbb7+tFBQUKPn5+YqpqakSExOj3nf27Nnq97bk6yh6LUW/\nh5J+/vlnxcvLS73u7OysrF27Vr0eGhqqTJgwQeOYQYMGKevXr1dycnIUa2tr5ccff1Ryc3PLfS2K\noiht27ZV9uzZo17/5ZdfFGdn53JjLq5ou5WVlbJkyZJS28v625o0aZISHBysXt+9e7fi6upabnzl\nvd9FZX344YeKoijKb7/9pjRp0kQjVjs7O+XEiRPq301sbKx624cffqj176as11GcjY2NcvbsWUVR\nVH+/ffr0KXdfRVGUnJwcxdLSUv15nj17tvLCCy+UuW9Fn92KPiuVfWbrkkHXyAE6dOjA0KFDCQ0N\n1fjvmZiYqK5lF3niiSdITExUrxcf6KuIvb29+nGTJk1KrRfVKHNzc3n11VdxdnbGysqKPn36kJGR\noXVbe1GnqXfffReA69ev8/DhQ1q1aoWNjQ02Njb87W9/486dOwDcunVLI96Sr60kV1dX1q5dS0JC\nAufPnycxMZHp06drFRtovg+NGzfGzs5OY73offj000/VzUSvv/66OtYnnnhCI9b8/HySk5NLncfc\n3BwLCwssLS25ceMGAP7+/mzfvp20tDS2bdvGunXrWLVqFXfu3CE/P1+n96G45ORkxo4di5OTE1ZW\nVkyYMIG7d+9q7FO87OvXr/P999+rfx82NjYcPXqUpKQkmjZtyubNm/n6669xcHBg6NChxMTElHne\nxMTEUu9H8b9DbezcuZO5c+eydu1arfYv7++2LOW932Vp1qwZxsZ/po2mTZuSnZ1d5u+m5DAduli8\neDHu7u5YW1tjY2NDRkYGKSkpWpfdtGlTRo8ezYYNGwDVnWMTJ04Eqv/ZLVLZZ7YuGXwiB5g7dy4r\nV67UaOdydHQs1Y57/fp1HB0d1evVuTK9ZMkSYmNjiYyMJCMjg4MHD2p90TQ0NJQrV65otA+2bt2a\nRo0acffuXdLS0khLSyMjI4Nz584B0KpVK+Lj49X7F39cGRcXF4KCgjh//jwAZmZmGnf7JCUlVXh8\nRe/T7Nmz1U1FX375JQAODg4aF5ri4+MxMTHRSC5FsrOzycrKIjMzs8wPZ79+/ejXrx//+9//sLOz\nw8TEpNz3wczMDKDc1zZ79mwaNGjA+fPnycjIYOPGjaXaM4u/1jZt2jBhwgT17yMtLY2srCxmzpwJ\nwMCBA9m7dy9JSUm4uroSHBxc5ntU1vuh64XKnj17smPHDqZNm6Zu3isZb00o/n7rco4WLVpgYmJS\nasylIpX9boo7fPgwixYt4vvvvyc9PZ20tDSsrKw0PlvaxBQUFMSWLVvYu3cv2dnZDBs2DNDts2tu\nbl5uzJV9ZuvSI5HI27Zty5gxYzTuYPnLX/5CbGwsYWFh5Ofns3nzZi5dusTQoUPV+2iTdIvvU/xx\ndnY2TZo0wcrKitTU1ArbjIvbs2cPX3zxBT/99JPGXSStWrVi4MCBvP3222RlZVFYWMjVq1c5dOgQ\nAC+++CLLli3j5s2bpKWlERoaWu45YmJi+Oyzz9T/2BISEggLC8PX1xcAT09PDh06REJCAhkZGcyf\nP1/r162NwMBAPv/8c+Li4sjOzmb27NmMHTtWoyZXnu3bt7N582bS0tJQFIXIyEgOHjxIjx49MDY2\nZsSIEYSneCWcAAAa20lEQVSEhHDv3j0uXLig0Q7bokULHB0d2bhxIwUFBaxZs4arV6+qy87OzsbM\nzAxLS0tu3rzJokWLKozlpZdeYseOHezdu5eCggLu379PREQEN2/e5Pbt22zbto2cnBxMTU0xMzOj\nQYMG5b4fn3zyCSkpKaSkpDBv3jyNC2ja6t27Nz/99BN//etf+emnnwBVzfvGjRvqi/Cg2++rove7\nqCxtymvQoIHG7+bSpUts3LhR699NcVlZWZiYmNC8eXPy8vKYN28emZmZWr+mIs8++yzW1ta8+uqr\nBAYGYmKiGpFEl89uRZ+Vyj6zdemRSOQAH3/8Mbm5ueo/nGbNmrFz506WLFlC8+bNWbx4MTt37tS4\nsl/yv3pZ/+WLP1f84sz06dO5d+8ezZs3p2fPnvzlL38pt5ZQ/LgtW7aQkpKCm5tbqSaJDRs2kJeX\nh7u7O7a2towePVpdAwgODmbQoEF07twZb29vRo4cWe75LCwsOHHiBD4+Ppibm+Pr64uHhwdLliwB\nYMCAAYwZMwYPDw+6devGsGHDKnwvyrooVVGN6OWXX2bChAn07t2bp556iqZNm/LFF19odayNjQ0r\nV66kffv26uaPmTNnEhgYCMDy5cvJzs6mZcuWvPzyy0yePFkj0axcuZJFixbRvHlzLly4QK9evdTb\n5syZw+nTp7GysmLYsGEVvoeg+vq+bds2Pv30U+zs7GjTpg1LlixBURQKCwv5/PPPcXR0pFmzZhw+\nfJivvvqqzHI+/PBDvL298fDwwMPDA29vb417syurXRbf3r9/fzZv3kxQUBC7du3iueeeo0OHDrRs\n2VLd/KXL76uy97tkWRXFunz5cjIyMmjZsiVBQUEEBgbSsGFD9faKfjfFz+Pv74+/vz/t27fH2dmZ\nJk2aaDSh6XKf98SJE4mPj1c3q4Bun93+/ftX+Fmp6DNblyoda+Xll19m165d2NnZlfuV4c0332TP\nnj00bdqUdevW4eXlVSvBClHSunXrWL16NYcPH9Z3KKKEWbNmcfv2ba3b9UXVVVojnzx5MuHh4eVu\n3717N1euXOHy5ct88803vPbaazUaoBDCMMTExHD27Fl1E82aNWsYPny4vsN6LFQ6jO2zzz5bqodU\ncdu3bycoKAgAHx8f0tPTSU5OLvPClhA1zRC7Uz+qsrKyCAwMJDExEXt7e2bMmCHzEtSRao9HXtZ8\nnTdu3JBELupEUFCQuiIh9KtoghlR92pkYomSzexl1ZDatWtX7lVqIYQQZWvbti1XrlypcJ9qJ/KS\n83XeuHFD417tIlevXjXogalCQkJKdec2JIYcf23FrihQUAB5earlwYM/H1f3ueLrJ06E0KlTCPn5\n1NpibAwmJmUvpqblb9NmOXcuhK5dQ2jQAPViYoLGenWW2i5rwYIQZs8OwcgI9WJsjMZ6Wc/XF9o0\nHVY7kQcEBLB8+XLGjh3L8ePHsba2lmYVUaGCArh3D3Jzy/9Z/PHRo/DPf9Zsoi16ztgYGjbUXBo1\nqtpzRevm5mBr++dzt27BgAE1k1TLWho0UL2O2hISoloMVePGYGmp7yhqV6WJPDAwkIMHD5KSkkLr\n1q2ZO3euuvPBq6++yuDBg9m9ezft2rXDzMxMbjUyUIoC9++XnUhr+rmHD6FJE2ja9M+fxR+XfC4n\nR3Vcw4ZgZVX1RFvWc+X04alRt25BsduYhahxlSby4t2By7N8+fIaCaY+8/Pz03cIgKoWmZ0NWVll\n/yzvufh4P44cKT/J3r+vSmwVJdSynjMzg+bNK9+v+M9GjXT76hoR4Uc9efurpL787VSVxF//1dnk\ny0ZGRgbdRl4VBQWq2mRFSVabRFx8m6KAhYVqMTcv/2fJx2ZmFSfZxo3rpnYqhNCNNrlTErkWzp+H\nU6d0T8T37qkSqDbJVtttxXo8CyEeA5LIa0BODrRtq7pYZWWlfSK2sFDVdmvzIpQQ4tGnTe6skfvI\nH2X//jf07g0lZo0SQoh6Q2rkFcjKgnbt4MAB6NBB39EIIR5H2uRO+eJfgeXLoV8/SeJCiPpNauTl\nyMxU1cYPHoT/n+hcCCHqnNTIq+GLL2DgQEniQoj6T2rkZcjIUNXGjx6F9u31HY0Q4nEmNfIqWroU\nBg+WJC6EMAxSIy8hPV1VGz9+XPVTCCH0qUZq5OHh4bi6uvL000+zYMGCUttTUlLw9/fH09OTjh07\nsm7duioHXB98/jkMGyZJXAhhOCqskRcUFODi4sK+fftwdHSkW7duhIWF4VbsCmBISAgPHjxg/vz5\npKSk4OLiQnJyMiYmmn2NDKFGnpqqak6JjISnntJ3NEIIUQM18sjISNq1a4ezszOmpqaMHTuWbdu2\naezTqlUrMjMzAcjMzKRZs2alkrih+OwzeOEFSeJCCMNSYcYtaz7OEydOaOwTHBxMv379cHBwICsr\niy1bttROpLXs7l346iv473/1HYkQQuimwhq5NlMMffrpp3h6epKYmMiZM2d44403yMrKqrEA68qS\nJTBqFDg76zsSIYTQTYU18pLzcSYkJODk5KSxz++//84HH3wAqCYJffLJJ4mJicHb27tUecXnXfTz\n86s3A77fuQMrVkBUlL4jEUI87iIiIoiIiNDpmAovdubn5+Pi4sL+/ftxcHCge/fupS52vv3221hZ\nWTFnzhySk5Pp2rUrZ8+exdbWVvNE9fhi56xZqgGyvvxS35EIIYSmag9ja2JiwvLlyxk0aBAFBQVM\nmTIFNzc3VqxYAajm7Jw9ezaTJ0+mc+fOFBYWsnDhwlJJvD67fRtWroSzZ/UdiRBCVM1j3yFoxgzV\nfJWPwbSjQggDJDMEVSIpCdzd4dw5cHTUdzRCCFGaJPJKvPUWFBaqxlYRQoj6SBJ5BRIToWNH+N//\noFUrfUcjhBBlk0RegTffBBMTVW9OIYSorySRl+PmTejUCS5eBHt7fUcjhBDlk0RejqlToUkTWLRI\n35EIIUTFJJGXISEBOneGS5fAzk7f0QghRMVkhqAyfPopBAdLEhdCPDoeqxr59evQpQvExEDz5noN\nRQghtCI18hL++U949VVJ4kKIR8tjUyO/dg28vSE2Fpo101sYQgihE6mRF/PJJ/D665LEhRCPnmpP\nvgyq8XO9vLzo2LFjvRljvLirV2HbNnj7bX1HIoQQNa/aky+np6fTq1cvfvnlF5ycnEhJSaF5GY3Q\n+mxamTwZnngCis1rIYQQBqHa45EXn3wZUE++XDyR/+c//2HkyJHqmYPKSuL6dPky7NgBV67oOxIh\nhKgdFTatlDX58s2bNzX2uXz5MqmpqfTt2xdvb282btxYO5FW0T/+oRpXxdpa35EIIUTtqLBGrs3k\nyw8fPuT06dPs37+f3NxcfH196dGjB08//XSpfet6zs6YGNizR2rjQgjDUZU5O6s9+XLr1q1p3rw5\nTZo0oUmTJvTu3Zvo6OhKE3ldmDcPpk8HK6s6Pa0QQlRZyUru3LlzKz2mwqYVb29vLl++TFxcHHl5\neWzevJmAgACNfZ5//nmOHDlCQUEBubm5nDhxAnd396q9ghp08SL8+quqWUUIIR5l1Z582dXVFX9/\nfzw8PDA2NiY4OLheJPK5c1W3G1pY6DsSIYSoXY9kz87//Q/69VPdP25uXienFEKIWvHY9uycOxdm\nzJAkLoR4PDxyNfKzZ2HgQFVt3Mys1k8nhBC16rGskc+dC+++K0lcCPH4eKRq5GfOwODBqvvGmzat\n1VMJIUSdeOxq5CEhMHOmJHEhxOPlkamR//e/EBCgqo03aVJrpxFCiDr1WNXIQ0LgvfckiQshHj+P\nRI385EkYPlxVG2/cuFZOIYQQevHY1MhDQmD2bEniQojHU4Vd9A3B8eNw7hz89JO+IxFCCP0w+Bp5\nUW28USN9RyKEEPpRI3N2Apw8eRITExN+qsOq8e+/w6VL8PLLdXZKIYSodypM5AUFBUydOpXw8HAu\nXLhAWFgYFy9eLHO/WbNm4e/vX6fzcs6ZAx98AA0b1tkphRCi3qkwkRefs9PU1FQ9Z2dJX3zxBaNG\njaJFixa1FmhJhw+rxlOZNKnOTimEEPVStefsvHnzJtu2beO1114DtJseribMmQMffgimpnVyOiGE\nqLeqPWfn9OnTCQ0NVd/rWFHTSk3N2XnwIFy/DhMmVOlwIYSot6oyZ2eFHYKOHz9OSEgI4eHhAMyf\nPx9jY2NmzZql3uepp55SJ++UlBSaNm3KypUrS00JV1MdghQF/PxUFziDgqpdnBBC1Gva5M4KE3l+\nfj4uLi7s378fBwcHunfvTlhYGG5ubmXuP3nyZIYNG8aIESOqFIw2DhyAv/0NLlwAE4O/C14IISqm\nTe6s9pyddUlRVG3jH38sSVwIIYoY1Fgr+/bB1KmqOTkbNKihwIQQoh57pMZaURRVTfzjjyWJCyFE\ncQaTyPfuhfR0GDNG35EIIUT9YhCJvKg2PmeO1MaFEKIkg0jke/ZATg6MHq3vSIQQov6p94m86E6V\nkBAwrvfRCiFE3av3qXHnTsjLgzJuTRdCCEE9T+SKoqqJS21cCCHKV6/T4/btUFgIL7yg70iEEKL+\nqrf9IwsLVW3jc+dCHQ2oKIQQBqne1si3blXdalhi7C0hhBAl1MsaeVFtfP58qY0LIURltKqRVzZv\n53fffUfnzp3x8PCgV69enD17tlpB/fgjNGkCQ4ZUqxghhHgsVDpoVkFBAS4uLuzbtw9HR0e6detW\naijbY8eO4e7ujpWVFeHh4YSEhHD8+HHNE2k5aFZBAXh4wOLF8Je/VPFVCSHEI6JGBs3SZt5OX19f\nrKysAPDx8eHGjRtVDvr778HCAvz9q1yEEEI8VipN5NrM21nc6tWrGTx4cJWCKShQ3aUyb560jQsh\nhLYqvdipy2TKv/32G2vWrOHo0aNVCmbTJrC1hQEDqnS4EEI8lipN5I6OjiQkJKjXExIScHJyKrXf\n2bNnCQ4OJjw8HBsbmzLLqmjy5fx8VU383/+W2rgQ4vFV45Mvg3bzdsbHx9OvXz++/fZbevToUfaJ\nKmmw37gRVq6EgwclkQshRJFqz9kJ2s3bOW/ePNLS0njttdcAMDU1JTIyUutAi2rj33wjSVwIIXRV\nL+bsXLdOtej4bUIIIR552tTI9Z7IHz4EV1dYswb69KmLSIQQwnAYxOTLGzaAs7MkcSGEqCq91sjz\n8sDFRXWh85ln6iIKIYQwLPW+Rr5uHTz9tCRxIYSoDr3VyPPyVEl80ybw9a2LCIQQwvDU6xr5mjXg\n7i5JXAghqksvNfIHD6BdO9Vwtd2718XZhRDCMNXbGvmqVaqhaiWJCyFE9dV5jfz+fVVtfOtW8Pau\nizMLIYThqpc18m++gS5dJIkLIURNqdMaeW6uQtu2sGsXeHnVxVmFEMKw1UiNvLL5OgHefPNNnn76\naTp37kxUVFS5ZX39Nfj4SBIXQoiaVGEiLygoYOrUqYSHh3PhwgXCwsK4ePGixj67d+/mypUrXL58\nmW+++UY9AmJZFi6EYkOSGxRdxweubww5fkOOHSR+fTP0+LVRYSLXZr7O7du3ExQUBKjm60xPTyc5\nObnM8nr1gs6dayjyOmbofwyGHL8hxw4Sv74ZevzaqDCRazNfZ1n7lDf58pw51QlVCCFEWSpM5NrO\n11myIb684zp10jIqIYQQ2lMqcOzYMWXQoEHq9U8//VQJDQ3V2OfVV19VwsLC1OsuLi5KUlJSqbLa\ntm2rALLIIosssuiwtG3btqI0rSiKolQ41Zu3tzeXL18mLi4OBwcHNm/eTFhYmMY+AQEBLF++nLFj\nx3L8+HGsra2xt7cvVdaVK1cqOpUQQogqqjCRazNf5+DBg9m9ezft2rXDzMyMtWvX1kngQgghVOqs\nQ5AQQojaUetd9LXpUFSfvfzyy9jb29PJAK/UJiQk0LdvXzp06EDHjh1ZtmyZvkPSyf379/Hx8cHT\n0xN3d3fef/99fYdUJQUFBXh5eTFs2DB9h6IzZ2dnPDw88PLyoruBjXKXnp7OqFGjcHNzw93dnePH\nj+s7JK3FxMTg5eWlXqysrCr+/Fbail4N+fn5Stu2bZVr164peXl5SufOnZULFy7U5ilr3KFDh5TT\np08rHTt21HcoOrt165YSFRWlKIqiZGVlKe3btze49z8nJ0dRFEV5+PCh4uPjoxw+fFjPEeluyZIl\nyrhx45Rhw4bpOxSdOTs7K3fv3tV3GFUyceJEZfXq1YqiqP5+0tPT9RxR1RQUFCgtW7ZU4uPjy92n\nVmvk2nQoqu+effZZbGxs9B1GlbRs2RJPT08AzM3NcXNzIzExUc9R6aZp06YA5OXlUVBQgK2trZ4j\n0s2NGzfYvXs3r7zySqXjZdRXhhh3RkYGhw8f5uWXXwZU1/usrKz0HFXV7Nu3j7Zt22r01ympVhO5\nNh2KRN2Ii4sjKioKHx8ffYeik8LCQjw9PbG3t6dv3764u7vrOySdvPXWWyxatAhjY71Oj1tlRkZG\n9O/fH29vb1auXKnvcLR27do1WrRoweTJk+nSpQvBwcHk5ubqO6wq2bRpE+PGjatwn1r969K2Q5Go\nXdnZ2YwaNYqlS5dibm6u73B0YmxszJkzZ7hx4waHDh0yqO7WO3fuxM7ODi8vL4Os1QIcPXqUqKgo\n9uzZw7///W8OHz6s75C0kp+fz+nTp3n99dc5ffo0ZmZmhIaG6jssneXl5bFjxw5Gjx5d4X61msgd\nHR1JSEhQryckJODk5FSbpxQlPHz4kJEjR/LSSy/xwgsv6DucKrOysmLIkCGcOnVK36Fo7ffff2f7\n9u08+eSTBAYGcuDAASZOnKjvsHTSqlUrAFq0aMHw4cOJjIzUc0TacXJywsnJiW7dugEwatQoTp8+\nreeodLdnzx66du1KixYtKtyvVhN58Q5FeXl5bN68mYCAgNo8pShGURSmTJmCu7s706dP13c4OktJ\nSSE9PR2Ae/fu8euvv+JlQGMgf/rppyQkJHDt2jU2bdpEv3792LBhg77D0lpubi5ZWVkA5OTksHfv\nXoO5e6tly5a0bt2a2NhYQNXO3KFDBz1HpbuwsDACAwMr3a/CDkHVVV6HIkMSGBjIwYMHuXv3Lq1b\nt2bevHlMnjxZ32Fp5ejRo3z77bfq28cA5s+fj7+/v54j086tW7cICgqisLCQwsJCJkyYwHPPPafv\nsKrM0Joak5OTGT58OKBqqhg/fjwDBw7Uc1Ta++KLLxg/fjx5eXm0bdvW4Dor5uTksG/fPq2uTUiH\nICGEMHCGeSldCCGEmiRyIYQwcJLIhRDCwEkiF0IIAyeJXAghDJwkciGEMHCSyEWNKz4MwO7du3Fx\ncSE+Pp5t27Zx8eJF9TY/Pz/++9//VlhWYWEhb775Jp06dcLDw4Pu3btz/fp1AIYMGUJmZmaNx+/s\n7ExqamqVjl2/fn2pcTFSUlKws7Pj4cOHZR6zbt06/v73v1fpfEKAJHJRC4o6vuzfv59p06YRHh5O\nmzZt+Pnnn7lw4UKp/SqyefNmbt26xblz5zh79ixbt25Vj2K3a9cuLC0tay1+bRUWFqofjxgxgl9/\n/ZV79+6pn/vhhx8ICAjA1NS0Rs4nREmSyEWtOHToEH/961/ZtWsXTz75JL///js7duzg3XffpUuX\nLvzxxx8AfP/99/j4+ODi4sKRI0dKlZOUlKQe7wPAwcEBa2tr4M+ac1xcHG5ubvz1r3+lY8eODBo0\niPv37wNw8uRJdc/Wd999V93FvGQteOjQoRw6dKjU+YcPH463tzcdO3bU6GFnbm7OjBkz8PT01Jiw\nwMLCgj59+rBjxw71c5s3byYwMJCdO3fSo0cPunTpwoABA7h9+3ap802aNIkff/xR4zxFFi1aRPfu\n3encuTMhISHlvPPicSSJXNS4+/fvM3z4cLZt20b79u0B6NmzJwEBASxevJjTp0/z1FNPAarZc06c\nOMG//vUv5s6dW6qsF198kR07duDl5cWMGTM4c+aMelvxmuyVK1eYOnUq58+fx9raWp0MJ0+ezMqV\nK4mKisLExKTc2m95z69Zs4ZTp05x8uRJli1bRlpaGqAah6RHjx6cOXOGnj17ahwTGBjIpk2bAEhM\nTCQ2NpZ+/frxzDPPcPz4cU6fPs2YMWNYuHAhoDned8k4itb37t3LlStXiIyMJCoqiv/+978GMxKh\nqH2SyEWNa9iwIb169WLVqlWltpUcEWLEiBEAdOnShbi4uFL7Ozo6EhMTw/z58zE2Nua5557jwIED\npfZ78skn8fDwAKBr167ExcWRkZFBdna2egz2cePG6Tyc7NKlS/H09MTX15eEhAQuX74MQIMGDRg5\ncmSZxwwePJijR4+SlZXFli1bGDVqFEZGRiQkJDBw4EA8PDxYvHixRjNTZfbu3cvevXvx8vKia9eu\nxMTEcOXKFZ1ei3h0SSIXNc7Y2JgtW7YQGRnJ/PnzNbaVrHE2atQIUCXG/Pz8Mstr2LAh/v7+LFy4\nkNmzZ7N169ZS+xSVU1FZxZO4iYmJRtt2UVNMcREREezfv5/jx49z5swZvLy81Ps1bty43Fp8kyZN\n8Pf356efflI3qwD8/e9/58033+Ts2bOsWLFCox29rLgKCwvJy8tTb3v//feJiooiKiqK2NhYgxm8\nTdQ+SeSiVjRu3Jhdu3bx3XffsWbNGkDVfqzrXSZRUVHq6ekKCwuJjo7G2dlZq2OtrKywsLBQj6Fd\n1NwBqvb1M2fOoCgKCQkJZY6znZmZiY2NDY0bN+bSpUs6Td4bGBjIZ599xu3bt+nRo4e6PAcHB0DV\nRl8WZ2dn9Z0827dvV9/pMmjQINasWUNOTg6gmn3rzp07WscjHm21OoyteDwV1VRtbGwIDw+nd+/e\n2NnZMXbsWIKDg/niiy/4/vvvyz2uuNu3bxMcHMyDBw8A8PHxYerUqaX2L69tefXq1QQHB2NsbEyf\nPn3Ud7w888wzPPnkk7i7u+Pm5kbXrl1Lndvf35+vv/4ad3d3XFxc8PX1rTDW4vr378+tW7d45ZVX\n1M+FhIQwevRobGxs6Nevn/o2SiMjI3V5wcHBPP/883h6euLv76++2DlgwAAuXryojsHCwoJvv/22\n0gkHxONBhrEVj7ScnBzMzMwACA0NJTk5mc8//1zPUQlRs6RGLh5pu3btYv78+eTn5+Ps7Fxuk4YQ\nhkxq5EIIYeDkYqcQQhg4SeRCCGHgJJELIYSBk0QuhBAGThK5EEIYOEnkQghh4P4P23WB7XEKEEsA\nAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x10d8086d0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import course_utils as bd\n",
"reload(bd)\n",
"U, sig, Vt = np.linalg.svd(dpro, full_matrices=0)\n",
"bd.plotSVD(sig)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>This is a fairly extreme outcome. Most of the data can be explained by the first singular vector and value. We can guess that this first (and very dominant) latent feature might be related to having self-reported business/communication skills or math/stats skills. Remember from the above correlations, that this is almost an either/or scenario.<br><br>\n",
"So while we have good evidence that skills based segments exist, we have no principled way to identify them right now. Fortunately, there are tools to solve this problem.\n",
"</p>\n",
"\n",
"\n",
"##Clustering Examples\n",
"<p><a href=\"http://scikit-learn.org/stable/modules/clustering.html\">Clustering</a> can be performed using the <a href=\"http://scikit-learn.org/stable/modules/classes.html#module-sklearn.cluster\">sklearn.cluster</a> library. We'll show two examples below, one using <a href=\"http://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans\">sklearn.cluster.kmeans</a> for K-Means clustering, and one using <a href=\"http://docs.scipy.org/doc/scipy-0.14.0/reference/cluster.hierarchy.html\">scipy.cluster.hierarchy</a> for hierarchical clustering. Note, in the latter case we'll demo scipy not sklearn here because scipy supports plotting the cluster dendrograms better.\n",
"\n",
"\n",
"</p>\n",
"\n",
"\n",
"###K-means clustering\n",
"<p>\n",
"One decision we have to make is what data to use. We have our original feature matrix $X$, but we have also computed the SVD of $X$, which gives us an orthonormal matrix of user latent features $U$. If we use $U$, the features are normalized and independent. The normalization is important because clustering methods use distance metrics that are sensitive to scale. The indepedence means each feature will hold equal weight in the clustering. This may or may not be a good thing. For example, if we use $U$, we know that the fist singular vector is by far the most important. We might want this feature to dominate the clustering process. The good news is, we can weight the columns in $U$ using the singular values, i.e., cluster on $U\\Sigma$ instead of $U$.<br><br>\n",
"A subtle corollary of this last point is using $U$ or $U\\Sigma$ gives us a great tool to overcome the curse of dimensionality. If $X$ happend to be very high dimensional, but most of the sum-of-squares can be explained by a smaller first-$k$ subset of the singular vectors, then we might be better off clustering on $U_k$ or $U_k\\Sigma_k$ (the rank-$k$ approximations). \n",
"<br><br>\n",
"We start with a basic clustering. It is fairly easy to implement. In general, you'll always get a result, and a major question is always how do you know if it is a good fit? Ultimately, this becomes both a qualitative and quantitative issue. Some criteria might be:<br>\n",
"<ul>\n",
" <li>Do the clusters make sense? (this is decidedly qualitative)</li>\n",
" <li>Are the clusters well balanced? (a quantitative attribution to a potentially arbitrary need).</li>\n",
"</ul><br>\n",
"A similar question on quality, is what is the optimal $k$. For choosing $k$, we think of the above two questions, but we can also see how well the clusters minimize the within cluster sum of squares. This criteria is also called 'inertia' and is defined as:<br><br>\n",
"\n",
"<center>$inertia = \\sum\\limits_{j=1}^k\\:\\sum\\limits_{x_i \\in C_j}|x_i-\\mu_j|^2$\n",
"</center>\n",
"\n",
"\n",
"\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"KMeans(copy_x=True, init='k-means++', max_iter=300, n_clusters=2, n_init=5,\n",
" n_jobs=1, precompute_distances=True, random_state=None, tol=0.0001,\n",
" verbose=0)"
]
},
"execution_count": 64,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sklearn import cluster\n",
"\n",
"#Note - most of these input parms, except the first, help ensure stability of the fit\n",
"km = cluster.KMeans(n_clusters=2, init = 'k-means++', n_init = 5)\n",
"km.fit(dpro)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>Let's loop through different values of k to get the inertia as a function of k. We'll also compute another metric that has been used to evaluate clusters where true cluster labels are not known (which is usually the case). This is called the Silhoette Coefficient. More details can be found <a href=\"http://scikit-learn.org/stable/modules/clustering.html#clustering-evaluation\">here</a></p>"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.text.Text at 0x10d327ad0>"
]
},
"execution_count": 70,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEKCAYAAAAVaT4rAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtclHW+B/DPoBgWyP1+CUUNQUVdLmpppEArJUfxkoKC\nSdbqupvZmtuuHbO2wFxrtU6ttSp5WbXT6ZQvReJlilqK4HrpbOhCBHITLS4KojDA9/xhzIIyMMNl\nZuD5vF+veb1mnsvv+T7DzJdnnuf3/L4qEREQEZFimBk7ACIiMiwmfiIihWHiJyJSGCZ+IiKFYeIn\nIlIYJn4iIoVh4ie6y9KlS/GnP/2pU+t6e3vjq6++6nIM6enp8PT07HI7RG1h4ieT113JtC3JycmY\nNGlSq2kffPAB1qxZ06n2VCoVVCpVd4RG1GOY+Mnk9VQybWho6PY2iXoDJn7qVZKTk/HII49g1apV\nsLOzw5AhQ5CamqqZf/36dSQkJMDNzQ0eHh545ZVX0NTUpFn34YcfxsqVK+Hg4IB58+Zh6dKlOHXq\nFKysrGBnZwcAWLRoEV555RUAQGVlJZ588kk4OTnBzs4O06dPR0lJSbsxZmZmwt/fH3Z2dli8eDHq\n6uoAACNHjsSBAwc0y6nVajg4OODChQsd7vfmzZvh7++P0tJS/d4wojYw8VOvk5mZCV9fX5SXl+Ol\nl15CQkKCZt6iRYswYMAA5OXl4dy5c0hLS8Pf/va3Vuv6+Pjg2rVr2LVrF/76179iwoQJqK6uRkVF\nBYDWvzBEBAkJCSgsLERhYSEGDhyI5cuXa41NRPD3v/8daWlpyMvLQ05OjuZ6QXx8PHbt2qVZNiUl\nBe7u7ggICGh3f1977TXs2LEDx48fh5ubm/5vGNFdmPip13nwwQeRkJAAlUqFuLg4XLlyBdeuXcPV\nq1dx6NAhvPPOOxg4cCAcHR2xYsUK7N27V7Oum5sbfv3rX8PMzAwWFhbQNlRV83Q7OzvMnDkTFhYW\nsLS0xB/+8AccO3ZMa2wqlQrLly+Hu7s7bG1t8cc//hF79uwBAMTGxuLgwYOoqakBAOzcuRMLFy7U\n2paIYOXKlTh8+DCOHj0Ke3t7vd8rorb0N3YARPpycXHRPL///vsBADU1Nfjpp5+gVqvh6uqqmd/U\n1AQvLy/Na317ytTW1uKFF17Al19+icrKSs22RETrdYeW2/Dy8tKcnnFzc8PDDz+MTz/9FDNmzEBq\naireffddrduuqqrC3/72N+zduxdWVlZ6xU3UHiZ+6jM8PT1x3333oby8HGZmbf+YvTtZa0vezdM3\nbtyInJwcZGZmwsnJCefPn8e4cePaTfyFhYWtnrc8PRMfH4+tW7dCrVZj4sSJrf5J3c3W1ha7d+/G\nnDlz8L//+7+YOHGi1mWJ9MFTPdRnuLq6IiIiAitXrkR1dTWampqQl5eH48ePa13HxcUFxcXFUKvV\nmmkiojnVU1NTg4EDB8La2hoVFRVYt25duzGICP7rv/4LJSUlqKiowBtvvIF58+Zp5s+cORNnz57F\n5s2bERcX1+E+TZ48Gbt370Z0dDSysrI6XJ5IF0z81Ku01bWz5esdO3agvr4efn5+sLOzw5w5c1BW\nVqZ13SlTpsDf3x8uLi5wcnK6Z7kVK1bg1q1bcHBwwMSJEzFt2rR2u5aqVCrExsYiIiICPj4+GDZs\nWKt7AiwsLBAdHY2CggJER0d3uK8AEBYWhm3btmH69Ok4f/58R28RUYdU7RViWbx4MQ4ePAgnJyf8\n3//9HwBg1apVOHDgAAYMGAAfHx9s374d1tbWAIDExERs27YN/fr1w+bNmxEREQEA+Mc//oFFixbh\n9u3biIyMxKZNmwywa0Sm6fXXX0dubi527Nhh7FBIodo94n/66adb9ZEGgIiICHz33Xe4cOEChg8f\njsTERABAdnY29u3bh+zsbKSmpmLZsmWan8tLly7F1q1bkZubi9zc3HvaJFKKiooKbNu2Dc8++6yx\nQyEFazfxT5o0Cba2tq2mhYeHay6chYSEoLi4GADwxRdfYP78+TA3N4e3tzeGDh2K06dP48qVK6iu\nrkZwcDAAIC4uDp9//nlP7AuRSfvoo4/g5eWFadOm4ZFHHjF2OKRgXTrHv23bNkRGRgIASktL4eHh\noZnn4eGBkpKSe6a7u7t3eOcjUV+0ZMkS1NTU4P333zd2KKRwnU78b7zxBgYMGICYmJjujIeIiHpY\np/rxJycnIyUlpdWIie7u7igqKtK8Li4uhoeHB9zd3TWng5qnu7u7t9numDFjdBq3hIiI/i0gIECv\nHl96J/7U1FRs2LABx44dg4WFhWZ6VFQUYmJisHLlSpSUlCA3NxfBwcFQqVQYNGgQTp8+jeDgYOzc\nuRO//e1v22z7woULWm+hJyKituk7em27iX/+/Pk4duwYfvrpJ3h6emLdunVITExEfX09wsPDAQAT\nJkzA+++/Dz8/P8ydOxd+fn7o378/3n//fU0w77//PhYtWoRbt24hMjISv/zlLzu5e0RE1FXt9uM3\nNJVKxSN+IiI96Zs7eecuEZHCMPETESkMEz8RkcIw8RMRKQwTPxGRwjDxExEpDBM/EZHCmFziby6C\n0fIxaJCdscMiIuoz2k38ixcvhrOzM0aNGqWZVlFRgfDwcAwfPhwRERGoqqrSzEtMTMSwYcPg6+uL\ntLQ0zfR//OMfGDVqFIYNG4bnn3++g5Dknkd1daX+e0ZERG3SuxBLUlISwsPDkZOTg6lTpyIpKQkA\nC7EQEfUWehdi2b9/P+Lj4wEA8fHxmqIqLMRCRNQ76H2O/+rVq3B2dgYAODs74+rVqwBYiIWIqLfo\n0sXd5ouvRETUe+g9Hr+zszPKysrg4uKCK1euwMnJCUD3FGK549UWz0N/fhARUbP09HSkp6d3vgHp\nQH5+vowcOVLzetWqVZKUlCQiIomJibJ69WoREfnuu+8kICBA6urq5IcffpAhQ4ZIU1OTiIgEBwdL\nRkaGNDU1ybRp0+TQoUNtbguAANLGo8MwiYgUS98cqVchltdeew2///3vMXfuXGzduhXe3t745JNP\nAICFWIiIegmTK8Ryp+/+PXNYoIWISAsWYiEionYx8RMRKQwTPxGRwvTJxD9okF2bg71xwDcioj56\ncVd7O/q3RURk6nhxl4iI2sXET0SkMEz8REQK0+nEn5iYCH9/f4waNQoxMTGoq6vrVJEWIiIyrE5d\n3C0oKMCUKVNw8eJF3HfffXjqqacQGRmJ7777Dg4ODnjppZewfv16VFZWIikpCdnZ2YiJiUFWVhZK\nSkoQFhaGnJwcmJm1/r/Di7tERPozyMXdQYMGwdzcHLW1tWhoaEBtbS3c3Nz0KtKSmZnZmU0TEVEX\ndSrx29nZ4cUXX4SXlxfc3NxgY2OD8PBwvYu09Aa8J4CI+hq9x+MHgLy8PPzlL39BQUEBrK2tMWfO\nHOzatavVMh0VadE+79UWz0Nh7PH47xR6b/snVHU1i9AQkeF1dTz+TiX+M2fOYOLEibC3twcAREdH\n49SpU3BxcdG5SIv2YiyvdiYkIiLFCA0NRWhoqOb1unXr9Fq/U6d6fH19kZGRgVu3bkFEcPjwYfj5\n+WH69On4+OOPAQAff/wxZsyYAQCIiorC3r17UV9fj/z8fOTm5mqKrxMRkWF16og/ICAAcXFxCAwM\nhJmZGcaNG4dnn30W1dXVehdpISIiw+JYPQZsi4ioJ3CsHiIiahcTPxGRwjDxG5C2ewJ4PwARGRLP\n8ZtEW7xWQESdx3P8RETULiZ+IiKFYeInIlIYJn4iIoXpdOKvqqrC7NmzMWLECPj5+eH06dMsxGIg\nHDGUiLqi04n/+eefR2RkJC5evIhvv/0Wvr6+SEpKQnh4OHJycjB16lQkJSUBALKzs7Fv3z5kZ2cj\nNTUVy5YtQ1NTU7fthNL8e8TQex935hERadepxH/9+nWcOHECixcvBgD0798f1tbWLMRCRNQLdCrx\n5+fnw9HREU8//TTGjRuHJUuW4ObNm32yEAsRUV/TqdE5GxoacPbsWbz33nsICgrCihUrNKd1mvWV\nQix93aBBdm2eHrKyssWNGxVGiIiIOmKUQiweHh7w8PBAUFAQAGD27NlITExkIZZeSFuFMVYXIzJd\nRinE4uLiAk9PT+Tk5AAADh8+DH9/fxZiISLqBTp1xA8A7777LmJjY1FfXw8fHx9s374djY2NLMRC\nRGTiOEibSbTV+2MiIuPhIG1kFLypjKj34BG/SbTFmIio83jET0RE7WLiJyJSGCZ+IiKFYeInk8ML\nxUQ9q9P9+Il6ira7ie/M4/0fRF3FI37q07T9euAvB1KyLiX+xsZGjB07FtOnTwcAFmIhk6OtdgHr\nFpCSdSnxb9q0CX5+fprhF1iIhYjI9HU68RcXFyMlJQXPPPOM5sYBFmIhIjJ9nU78L7zwAjZs2AAz\ns383wUIs1FexpxH1JZ3q1XPgwAE4OTlh7NixWosBsBAL9SXsaUSmxCiFWE6ePIn9+/cjJSUFt2/f\nxo0bN7Bw4UI4OzuzEAtRB1j1jLrKKIVY3nzzTRQVFSE/Px979+7FlClTsHPnTkRFRbEQC1EH2NOI\njK1bbuBqPm3z+9//noVYiAxE2y8HgL8eqH0cltkk2mJMhm+LMVHfwWGZiYioXUz8RMTuqgrDQdqI\niN1VFYZH/ERECsPET0TdiiOimj6e6iGibqXttBFPGZkOHvETkUniBeeewyN+IjJJvODcczp1xF9U\nVITHHnsM/v7+GDlyJDZv3gyAhViIyDTxukNrnbpzt6ysDGVlZRgzZgxqamrwi1/8Ap9//jm2b98O\nBwcHvPTSS1i/fj0qKyuRlJSE7OxsxMTEICsrCyUlJQgLC0NOTk6rIZ0B07yjsa/fsdm7YurOthiT\n4dvq/TF15zAZ3dmWQe7cdXFxwZgxYwAAlpaWGDFiBEpKSliIhYj6NG0D7HVmkL3ubEtfXb64W1BQ\ngHPnziEkJISFWIiIeoEuXdytqanBrFmzsGnTJlhZWbWax0IsREQ9wyiFWABArVZj1qxZWLhwoWbc\nfRZiISLqeUYpxCIiSEhIgJ+fH1asWKGZzkIsRESmr1O9er7++mtMnjwZo0eP1pyySUxMRHBwMObO\nnYvCwkJNIRYbGxsAd6p2bdu2Df3798emTZvw+OOP3xtMr+oN0J1tMSbDt8WYDN8WY+rJtvRanoVY\nTKEtxmT4thiT4dtiTD3ZFguxEBGRVkz8REQKw8RPRKQwTPxERArDxE9EpDBM/ERECsPET0SkMEz8\nREQKY9DEn5qaCl9fXwwbNgzr16835KaJiOhnBkv8jY2NWL58OVJTU5GdnY09e/bg4sWLhto8ERH9\nzGCJPzMzE0OHDoW3tzfMzc0xb948fPHFF4baPBER/cxgib+kpASenp6a1yzGQkRkHF0qxKKP9oqy\nNAsICMCFC20vp8v6d63RpVgM3RZjMnxbjMnwbTGmnmkrICBAr60aLPHfXYylqKioVTlGADh//ryh\nwiEiUiyDneoJDAxEbm4uCgoKUF9fj3379iEqKspQmyciop8Z7Ii/f//+eO+99/D444+jsbERCQkJ\nGDFihKE2T0REPzOpQixERNTzeOcuEZHCMPGTVrt3725VG9nMzAw//PADAGDRokV45ZVXjBWaybh1\n6xamT58OGxsbPPXUUwCANWvWwNHREW5ubigqKoKVlVWHZfFOnDgBX19fQ4SM9PT0Vl2ru4Kfg96J\niV/hvv76a0ycOBE2Njawt7fHI488gjNnzgAAYmNj8eWXX7a5nkql6kTXtY4lJydj0qRJraZ1R3LJ\nzMxEZGQkbG1tYW9vj5CQECQnJ3epTQD49NNPce3aNVRUVGDfvn0oLCzE22+/jUuXLqG0tBSenp6o\nrq7u8L2aNGkSLl261OV4AMDb2xtHjhzplrY60lOfA+pZTPwKduPGDTz55JN4/vnnUVlZiZKSEqxd\nuxb33XefsUPrVqdOncLUqVPx2GOPIS8vD+Xl5fjggw+Qmpra5bYvX76M4cOHw8zszlepsLAQ9vb2\nsLe373LbnaVv4e2u4mXCXkhIsbKyssTGxkbr/O3bt8sjjzyiea1SqSQvL09ERBYtWiS//vWv5Ykn\nnhArKysJCQnRzBMR+eabbyQwMFCsra0lKChITp48qZlXVVUlixcvFldXV3F3d5c1a9ZIY2OjZGdn\ni4WFhfTr108sLS3FxsZGPvzwQzE3N5cBAwaIpaWlREVFiYhISUmJREdHi6OjowwePFg2b96sdT8e\nfvhhWb58ebvvxYcffihDhw4VOzs7iYqKktLSUs28ixcvSlhYmNjZ2clDDz0kn3zyiYiI/Od//qcM\nGDBAzM3NxdLSUrZs2SIDBw4UMzMzsbS0lKefflry8/NFpVJJY2OjiIiUl5fLokWLxM3NTWxtbWXG\njBkiInL06FHx8PDQbLO9/Vu7dq3MmTNH4uLixMrKSvz9/eXMmTMiIrJgwQIxMzOTgQMHiqWlpWzY\nsOGefW3e1ptvvikODg7i7e0tu3fvFhGRzMxMcXZ2lqamJs3y//M//yMBAQFtvm+LFi2SNWvWiIjI\njRs3JDQ0VJ5//vl232syPiZ+Bbtx44bY29tLfHy8HDp0SCoqKlrNby/xx8fHi729vWRlZUlDQ4PE\nxsbKvHnzROROcrOxsZFdu3ZJY2Oj7NmzR2xtbTXtz5gxQ371q19JbW2tXLt2TYKDg2XLli0iIpKc\nnNxqmyJ3kssrr7yied3Y2Cjjxo2T119/XdRqtfzwww8yZMgQ+fLLL+/Zx5s3b0q/fv0kPT1d6/vw\n1VdfiYODg5w7d07q6urkN7/5jUyePFlERGpqasTDw0OSk5OlsbFRzp07Jw4ODpKdnS0iIq+++qos\nXLhQ01Z6enqrBH534o+MjJR58+ZJVVWVqNVqOX78uIi0Tvwd7d/atWvFwsJCDh06JE1NTfLyyy/L\n+PHjNdv09vaWr776Suv+Hj16VPr37y8vvvii1NfXy7Fjx+SBBx6QnJwcERHx8/OTQ4cOaZafMWOG\nvP3222221fy3+emnnyQoKKjV34lMF0/1KJiVlRW+/vprqFQqLFmyBE5OTviP//gPXLt2rcN1VSoV\noqOjERgYiH79+iE2NlZz5/XBgwfx0EMPITY2FmZmZpg3bx58fX2xf/9+XL16FYcOHcI777yDgQMH\nwtHREStWrMDevXsBaD9t0HJ6VlYWfvrpJ6xZswb9+/fH4MGD8cwzz2jaaKmyshJNTU1wdXXVui+7\nd+9GQkICxowZgwEDBiAxMRGnTp3C5cuXceDAAQwePBjx8fEwMzPDmDFjEB0djf/+7//WxNUyNm3x\nA8CVK1eQmpqKv/71r7C2tkb//v3vuZ6h6/5NmjQJv/zlL6FSqbBgwQJcuHBB63a1ef3112Fubo7J\nkyfjiSeewL59+wAAcXFx2LVrFwCgoqICaWlpiImJ0dpOSUkJQkND8dRTT+G1117TOw4yPIPdwEWm\nydfXF9u3bwcA/Otf/8KCBQuwYsUK/P3vf+9wXWdnZ83zgQMHoqamBgBQWloKLy+vVss++OCDKCkp\nQWFhIdRqdatE3NTUdM/y7bl8+TJKS0tha2urmdbY2IjJkyffs6ytrS3MzMxw5coVDB8+vM32rly5\ngsDAQM3rBx54APb29igpKcHly5dx+vTpVttqaGhAXFyczvE2Kyoqgp2dHaytrdtdTpf9a/ne33//\n/bh9+zaampo01xo6Ymtri4EDB2peP/jggygtLQVw56K+v78/amtr8cknn2Dy5MmttteSiODgwYOw\nsrLCc889p9O2yfiY+EnjoYceQnx8PD788MMutePu7o7PPvus1bTLly9j2rRp8PT0xH333Yfy8vI2\nk1RbPUTunubl5YXBgwcjJyenw1juv/9+TJgwAZ9++ikeffTRNpdxc3NDQUGB5vXNmzdRXl4ODw8P\neHl54dFHH0VaWlqb6+rTo8XT0xMVFRW4fv16u8nf09Oz3f3raJu6xFRZWYna2lrcf//9AO78fUaP\nHg3gzsi548ePx2effYZdu3Zh2bJl7W5ryZIlqKysRGRkJFJTUzVtkuniqR4F+9e//oW3335bMzx2\nUVER9uzZgwkTJnS4bnunNKZNm4acnBzs2bMHDQ0N2LdvHy5duoQnn3wSLi4uiIiIwMqVK1FdXY2m\npibk5eXh+PHjAO4cyRYXF0OtVmvac3Z21tw/AADBwcGwsrLCW2+9hVu3bqGxsRH//Oc/Nd1Q7/bW\nW28hOTkZf/7zn1FeXg4AuHDhAubPnw8AmD9/PrZv344LFy6grq4Of/jDHzB+/Hh4eXnhiSeeQE5O\nDnbt2gW1Wg21Wo2srCxN18v23oe7ubq6Ytq0aVi2bBmqqqqgVqs1+91SR/vX0TadnZ2Rl5fXYTxr\n166FWq3GiRMncPDgQcyZM0czLy4uDuvXr8c///lPREdHa22jOZb33nsPDz30EKZPn47bt293uG0y\nLiZ+BbOyssLp06cREhICS0tLTJgwAaNHj8bGjRsB3NtH++7ndx9ZNr+2t7fHgQMHsHHjRjg4OODP\nf/4zDhw4ADs7OwDAjh07UF9fDz8/P9jZ2WHOnDkoKysDAEydOhX+/v5wcXGBk5MTACAhIQHZ2dmw\ntbVFdHQ0zMzMcODAAZw/fx5DhgyBo6Mjnn32Wdy4caPN/ZwwYQKOHDmCI0eOwMfHB/b29njuuefw\nxBNPaLb5+uuvY9asWXBzc0N+fr7mfLqVlRXS0tKwd+9euLu7w9XVFS+//DLq6+s7fB/aer1z506Y\nm5vD19cXzs7O2Lx58z3L9evXr93962ibL7/8Mv70pz/B1tYWb7/99j3vh0qlgqurK2xtbeHm5oaF\nCxdiy5YtrU6FRUdHo7CwEDNnzoSFhUWb7+vdsXz44Yfw8PDAjBkzUFdXp3UdMj69xupJTU3FihUr\n0NjYiGeeeQarV69uNX/37t146623ICKwsrLCBx98oPn56O3tjUGDBqFfv34wNzdHZmZm9+4JEXWr\nYcOGYcuWLZgyZYqxQ6FupvM5/uaauYcPH4a7uzuCgoIQFRXVaoTNIUOG4Pjx47C2tkZqaiqeffZZ\nZGRkALhzZJCenq456iMi0/XZZ59BpVIx6fdROif+ljVzAWhq5rZM/C3PDYeEhKC4uLhVG/qcDyUi\n4wgNDcWlS5ewc+dOY4dCPUTnxN9WzdzTp09rXX7r1q2IjIzUvFapVAgLC0O/fv3w3HPPYcmSJZ0M\nmYh6Unp6urFDoB6mc+LXp9va0aNHsW3bNnzzzTeaad988w1cXV3x448/Ijw8HL6+vvfcvDJ06FCd\neiMQEdG/+fj44Pvvv9d5eZ179ehSMxcAvv32WyxZsgT79+9vdQNK8w07jo6OmDlzZpsXd/Py8jR3\nQprKY+3atUaPgTH1rbgYE2Pq7oe+B8w6J35dauYWFhYiOjoau3btwtChQzXTa2trUV1dDeDOzTFp\naWkYNWqUXoESEVH30PlUj7aauVu2bAEAPPfcc3jttddQWVmJpUuXAoCm22ZZWZnmJpCGhgbExsYi\nIiKiB3aHiIg6JCbExMIRkTsjGZoaxqQ7U4yLMemGMelO39xpUsXWDV1AgoioL9A3d3LIBiIihWHi\nJyJSGCZ+IiKFYeInIlIYJn4iIoVh4iciUhgmfiIihWHiJyJSGCZ+IiKFYeInIlIYvRJ/amoqfH19\nMWzYMKxfv/6e+bt370ZAQABGjx6Nhx9+GN9++63O6xJR3zBokJ2mCHvLx6BBLLtqMnQd1KehoUF8\nfHwkPz9f6uvrJSAgQLKzs1stc/LkSamqqhIRkUOHDklISIjO6zYPNNTWw8rKVq8BiIjIeO58b6WN\nh+kNwthX6Pve6nzE37Lmrrm5uabmbksTJkyAtbU1gNY1d3VZt8W/onse1dWVOv8jIyKi9umc+Nuq\nuVtSUqJ1+ZY1d/Vdt6u0/dTkz00iIgPV3NVn3e5w5xdC20OUVlcbNhYiMr5Bg+zaPHNgZWWLGzcq\njBCRcemc+PWtuZuamqqpuavrune82uJ56M8P49H2gQGU+6Eh6m20HQz21gPB9PR0pKend74BXS8G\nqNVqGTJkiOTn50tdXV2bF2gvX74sPj4+curUKb3Xbb5A0R0XhbS3Y9y2SDdWVra80N+Lddf3uK/H\n1J303Q+D1NzVtq7S8OembniqjgD+2u5JJld6se0vvH5lxbS3Y6ptseRkS935nncXJiHd9a7vcd/4\n7rH0IlEP+PevEHY3VrK+0mOQiZ+I+kxC62l95QCAib8X4peUultfSWikGyb+Xqg7v6QcV4VIeXTu\n1UN9U1/r30xEHeMRPxGRwjDxExEpDBM/EZHCMPFTt2BPI9111wV1vufUWbxz1yTaYkw91VZ3DZPB\n97yv7p/xYupO+t65y1491Kex1xLRvXiqh4hIYbq12PqlS5cwYcIEWFhYYOPGja3meXt7Y/To0Rg7\ndiyCg4O7FjURUS9nzGs0Op/qaWxsxPLly3H48GG4u7sjKCgIUVFRrYZXtre3x7vvvovPP//8nvVV\nKhXS09NhZ8eLTkRExhx+vFuLrTs6OiIwMBDm5uZttmFC15GJiBSrx4qt302lUiEsLAyBgYH46KOP\n9IuSiIi6TY8UW2/LN998A1dXV/z4448IDw+Hr68vJk2a1MaSr7Z4Hgpj19wlIjI1Xa252+3F1rVx\ndXUFcOd00MyZM5GZmalD4icioruFhoYiNDRU83rdunV6ra/zqZ7AwEDk5uaioKAA9fX12LdvH6Ki\notpc9u5z+bW1taiurgYA3Lx5E2lpaRg1apRegRIRUffo1mLrZWVlCAoKwo0bN2BmZoZNmzYhOzsb\n165dQ3R0NACgoaEBsbGxiIiI6Jk9IiKidnHIBpNoizEZvi3GZPi2GFNPtsVi60REpBUTPxGRwjDx\nExEpDBM/EZHCMPETESkMEz8RkcIw8RMRKQwTPxGRwjDxExEpDBM/EZHCMPETESmMwWrudrQuEREZ\nhs6Jv7nmbmpqKrKzs7Fnzx5cvHix1TLNNXd/97vf6b0uEREZhkFq7uqyLhERGYZBau52tV4vERF1\nH4PU3NU2q7bdAAAJWUlEQVRv3VdbPA8Fa+4SEbXWK2ru6rfuq7qGRESkSL2i5q4+6xIRUc8ySM1d\nS0vLNtclIiLDY81dk2iLMRm+LcZk+LYYU0+2xZq7RESkFRM/EZHCMPETESkMEz8RkcIw8RMRKQwT\nPxGRwjDxExEpDBM/EZHCMPETESkMEz8RkcIw8RMRKUy31twFgN/+9rcYNmwYAgICcO7cOc10b29v\njB49GmPHjkVwcHDXoiYiok7TeXTO5rq5hw8fhru7O4KCghAVFdVqlM2UlBR8//33yM3NxenTp7F0\n6VJkZGQAuDOIUHp6Ouzs7Lp/L4iISGfdWnN3//79iI+PBwCEhISgqqoKV69e1cw3oYFAiYgUq1tr\n7ra3jEqlQlhYGAIDA/HRRx91NW4iIuqkbq+5q+2o/uuvv4abmxt+/PFHhIeHw9fXF5MmTdJ180RE\n1E26tebu3csUFxfD3d0dAODm5gYAcHR0xMyZM5GZmakl8b/a4nkoWGydiKi1rhZbh+hIrVbLkCFD\nJD8/X+rq6iQgIECys7NbLXPw4EGZNm2aiIicOnVKQkJCRETk5s2bcuPGDRERqampkYkTJ8qXX355\nzzYACCBtPHQOs4N2TLUtxmT4thiT4dtiTD3Zlj66teZuZGQkUlJSMHToUDzwwAPYvn07AKCsrAzR\n0dEAgIaGBsTGxiIiIkK//1BERNQtWHPXJNpiTIZvizEZvi3G1JNt6bM879wlIlIYJn4iIoVh4ici\nUhgmfiIihWHiJyJSGCZ+IiKFYeInIlIYJn4iIoVh4iciUhgmfiIihWHiJyJSGIPV3NVlXSIiMgBd\nh/FsaGgQHx8fyc/Pl/r6+g6HZc7IyNAMy6zLuj8PFmeSw50qc2hYU4ypr++fKcbU1/fPFGPqXFv6\n6PGau2VlZTqtS0REhmGQmrulpaUdrktERIahc+Lvas1dIiIyDT1ec9fDwwNqtbrDdQHAx8cHeXlt\n/4PR9R9PizW0zzHBthiT4dtiTIZvizH1TFs+Pj56bVXnxB8YGIjc3FwUFBTAzc0N+/btw549e1ot\nExUVhffeew/z5s1DRkYGbGxs4OzsDHt7+w7XBYDvv/9er+CJiEh/Bqm5q21dIiIyPJOquUtERD3P\nJO7cLSoqwmOPPQZ/f3+MHDkSmzdvNnZIAIDGxkaMHTsW06dPN3YoGlVVVZg9ezZGjBgBPz8/ZGRk\nGDskJCYmwt/fH6NGjUJMTAzq6uoMHsPixYvh7OyMUaNGaaZVVFQgPDwcw4cPR0REBKqqqkwirlWr\nVmHEiBEICAhAdHQ0rl+/bvSYmm3cuBFmZmaoqKgwiZjeffddjBgxAiNHjsTq1auNHlNmZiaCg4Mx\nduxYBAUFISsry6AxacuVen/W9er130OuXLki586dExGR6upqGT58eJs3eBnaxo0bJSYmRqZPn27s\nUDTi4uJk69atIiKiVqulqqrKqPHk5+fL4MGD5fbt2yIiMnfuXElOTjZ4HMePH5ezZ8/KyJEjNdNW\nrVol69evFxGRpKQkWb16tUnElZaWJo2NjSIisnr1aoPH1VZMIiKFhYXy+OOPi7e3t5SXlxs9piNH\njkhYWJjU19eLiMi1a9eMHtOjjz4qqampIiKSkpIioaGhBo1JW67U97NuEkf8Li4uGDNmDADA0tIS\nI0aMQGlpqVFjKi4uRkpKCp555hmT6aJ6/fp1nDhxAosXLwZw59qJtbW1UWMaNGgQzM3NUVtbi4aG\nBtTW1sLd3d3gcUyaNAm2tratprW8oTA+Ph6ff/65ScQVHh4OM7M7X72QkBAUFxcbPSYAWLlyJd56\n6y2DxtKsrZg++OADvPzyyzA3NwcAODo6Gj0mV1dXzS+0qqoqg3/W28qVJSUlen/WTSLxt1RQUIBz\n584hJCTEqHG88MIL2LBhg+YLagry8/Ph6OiIp59+GuPGjcOSJUtQW1tr1Jjs7Ozw4osvwsvLC25u\nbrCxsUFYWJhRY2p29epVODs7AwCcnZ1x9epVI0d0r23btiEyMtLYYeCLL76Ah4cHRo8ebexQNHJz\nc3H8+HGMHz8eoaGhOHPmjLFDQlJSkubzvmrVKiQmJhotlpa5Ut/PuulkNQA1NTWYPXs2Nm3aBEtL\nS6PFceDAATg5OWHs2LEmc7QPAA0NDTh79iyWLVuGs2fP4oEHHkBSUpJRY8rLy8Nf/vIXFBQUoLS0\nFDU1Ndi9e7dRY2qLSqXqRB/rnvXGG29gwIABiImJMWoctbW1ePPNN7Fu3TrNNFP43Dc0NKCyshIZ\nGRnYsGED5s6da+yQkJCQgM2bN6OwsBDvvPOO5te3odXU1GDWrFnYtGkTrKysWs3T5bNuMolfrVZj\n1qxZWLBgAWbMmGHUWE6ePIn9+/dj8ODBmD9/Po4cOYK4uDijxgTcGerCw8MDQUFBAIDZs2fj7Nmz\nRo3pzJkzmDhxIuzt7dG/f39ER0fj5MmTRo2pmbOzM8rKygAAV65cgZOTk5Ej+rfk5GSkpKSYxD/J\nvLw8FBQUICAgAIMHD0ZxcTF+8Ytf4Nq1a0aNy8PDA9HR0QCAoKAgmJmZoby83KgxZWZmYubMmQDu\nfP8yMzMNHkNzrly4cKEmV+r7WTeJxC8iSEhIgJ+fH1asWGHscPDmm2+iqKgI+fn52Lt3L6ZMmYId\nO3YYOyy4uLjA09MTOTk5AIDDhw/D39/fqDH5+voiIyMDt27dgojg8OHD8PPzM2pMzaKiovDxxx8D\nAD7++GOjH1A0S01NxYYNG/DFF1/AwsLC2OFg1KhRuHr1KvLz85Gfnw8PDw+cPXvW6P8oZ8yYgSNH\njgAAcnJyUF9fD3t7e6PGNHToUBw7dgwAcOTIEQwfPtyg29eWK/X+rPfgBWidnThxQlQqlQQEBMiY\nMWNkzJgxcujQIWOHJSIi6enpJtWr5/z58xIYGCijR4+WmTNnGr1Xj4jI+vXrxc/PT0aOHClxcXGa\nXhiGNG/ePHF1dRVzc3Px8PCQbdu2SXl5uUydOlWGDRsm4eHhUllZafS4tm7dKkOHDhUvLy/NZ33p\n0qVGiWnAgAGa96qlwYMHG7xXT1sx1dfXy4IFC2TkyJEybtw4OXr0qFFiavmZysrKkuDgYAkICJDx\n48fL2bNnDRqTtlyp72edN3ARESmMSZzqISIiw2HiJyJSGCZ+IiKFYeInIlIYJn4iIoVh4iciUhgm\nfiIihWHiJyJSmP8HtBf3m06XRO0AAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x10d12add0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from sklearn.metrics import pairwise_distances\n",
"from sklearn import metrics\n",
"\n",
"inert_k = []\n",
"sil_k = []\n",
"\n",
"for k in range(2,20):\n",
" km = cluster.KMeans(n_clusters=k, init = 'k-means++', n_init = 5)\n",
" km.fit(dpro)\n",
" inert_k.append(km.inertia_)\n",
" sil_k.append(metrics.silhouette_score(dpro, km.labels_, metric='euclidean')) \n",
"\n",
"fix = plt.figure()\n",
"ax1 = plt.subplot(211)\n",
"plt.bar(range(2,20), inert_k, 0.35)\n",
"plt.title('Inertia by k')\n",
"plt.tick_params(axis='x',which='both',bottom='off',top='off',labelbottom='off')\n",
"ax1 = plt.subplot(212)\n",
"plt.bar(range(2,20), sil_k, 0.35)\n",
"plt.title('Silhoette Coefficient by k')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>We can see that increasing $k$ tends to continually decrease the inertia. But at the same time, our Silhouette Coefficient does not increase. We should always expect inertia to decrease as $k$ increases, but the other metric shows that this does not exactly result in good clusters.\n",
"<br><br>\n",
"One method for determining a good value of $k$ is called the <a href=\"http://en.wikipedia.org/wiki/Determining_the_number_of_clusters_in_a_data_set#The_Elbow_Method\">elbow method</a>. This amounts to looking at the first plot above, and determining at what $k$ the within cluster sum of squares starts to level off. This is somewhat subjective, but tends to be useful.\n",
"</p>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>Next we'll look at a qualitative evaluation. For a small range of $k$, we first look at the distribution of clusters.\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEaCAYAAABEsMO+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlUFFe+B/BvCxgXFsHIomBYlMiiDYqSGEnaUdwhJGpc\nEuEJmjka4zNmJk9Pnk98zyg+J2OMJuo40RjjuCR5UcYgGVFbHY0xCmRxX8CALDEuASTKdt8fRgaU\npamlq4Dv55w6p2mqq77VXT8uXXXrlkEIIUBERKQzbbQOQEREVBc2UEREpEtsoIiISJfYQBERkS6x\ngSIiIl1iA0VERLrEBkpnvL29sW/fPq1jEDVrrKOWgQ2UzhgMBhgMBkmvLSsrQ0JCAry9veHo6IjQ\n0FCkpqYqnJBI/+TUEQC89NJL8PDwgKOjI3x9ffHWW28pmI4sxQaqBamoqED37t1x6NAhFBUVYfHi\nxXjhhRdw5coVraMRNSvz589HVlYWioqKsGfPHqxatYr/7GmADZSOnTlzBr6+vti+fbtF83fo0AEL\nFy5E9+7dAQCjR4+Gj48P0tPT1YxJpGtNrSMACAoKQrt27ap/trW1haurqxrxqAFsoHQqPT0dI0aM\nwOrVqzFhwgSMGTMGzs7OdU7R0dF1LqOwsBDnz59HUFCQldMT6YOcOpo5cyY6duyIoKAg/Od//if6\n9u2r0Va0XgaOxacvPj4+iI2NxYYNG7BlyxY8/fTTkpZTXl6OkSNHomfPnlizZo3CKYn0Tak6EkLg\n4MGDGDduHFJSUjBgwACFk1JD2EDpjLe3N+7cuQOTyYRt27ZJWkZVVRUmT56MkpIS7Nq1CzY2Ngqn\nJNI3JeqophkzZqBdu3ZYsWKFAunIUjzEpzMGgwHr1q3DlStXMHfu3OrnR44cCQcHhzqn0aNHV88n\nhEBCQgKuXbuGzz77jI0TtUpy6+hB5eXl6NixozWiUw22Wgeghzk4OCA1NRVDhgzB/PnzsXTpUuzZ\ns8ei186YMQNnz55FWloaHnnkEZWTEumX1Dq6du0a9u3bh6ioKLRr1w5paWn45JNPkJaWZoXUVBO/\nQemUk5MT9u7diz179mDhwoUWvebKlSv4y1/+gm+//Rbu7u7V/xlu3bpV5bRE+iSljgwGA9auXQtP\nT0907twZCxYswObNm9G/f3+V09KDGjwHlZOTg9jYWPz0008wGAx4+eWXMXv2bCQmJuKvf/0runTp\nAgBYunQpRowYYbXQRM0Na4mo6RpsoAoKClBQUICQkBCUlJSgX79+2LlzJ3bs2AEHB4dax3aJqH6s\nJaKma/AclLu7O9zd3QEA9vb2CAgIwNWrVwHcOxlPRJZhLRE1ncXnoLKzs5GRkYEnnngCALBq1SoY\njUYkJCTg1q1bqgUkamlYS0QWEhYoLi4W/fr1E59//rkQQojCwkJRVVUlqqqqxJtvvini4+Mfeo2f\nn58AwIlTs578/PwsKRGLsZY4tcZJah012kCVlZWJYcOGiRUrVtT5+6ysLBEcHPzwgmFR22c1Cxcu\n1DpCNT1lEUJfefSURQhl9+OWUkuN0dtnaInmlrm55ZW6Dzd4iE/8dtFnYGAg5syZU/18fn5+9ePP\nP/8cvXv3bmgxRK0ea4mo6RrsJHHkyBF8/PHH6NOnD0JDQwEAS5YswdatW5GZmQmDwQAfHx+sW7fO\nKmGJmivWElHTNdhADRo0CFVVVQ89P3LkSIsW3r9/pLRUNUyeHIPXXntF9nJMJpPsZShFT1kAfeXR\nUxYlya2l5qQ5fobNLXNzyyuVaoPF3rub5T9kLuUghg07hy+//ESJSNSKODq6oLj4piLLUqlELGYw\nGDTPQK2T1nWk8lh8cr9B3QJwTokg1MrcKyol/qhLv204UXOndR012EkiJycHgwcPRlBQEIKDg/Hu\nu+8CAG7cuIHIyEj4+/tj2LBhvHaDqBGsJaKma7CBsrOzw4oVK3Dq1CkcO3YM7733Hs6cOYOkpCRE\nRkbi/PnzGDJkCJKSkqyVl6hZYi0RNV2DDZS7uztCQkIA1B6eJTk5GXFxcQCAuLg47Ny5U/2kRM0Y\na4mo6Zo81FF4eDgKCwvh5uYGAHBzc0NhYaFqAYlaGtYSkWUs6iRRUlKCsWPHYuXKlXBwcKj1O4PB\n8FuPvbok1nhs+m0i0jPzb5M6pNZSYmJi9WOTydRquhlTc2WGEnXUaANVXl6OsWPHYsqUKYiJiQFw\n7z+9goICuLu7Iz8/H66urvW8OlF2QCLrMqH2P1KLFFuynFqq2UAR6Z8JStSRpKGOoqOjsWnTJgDA\npk2bqouNiOrGWiJqugYv1P3nP/+Jp59+Gn369Kk+9LB06VIMGDAAL7zwAn788Ud4e3tjx44d6NSp\nU+0FGwyQ33/+EwwbtoMX6lKTKbP/AYAyF8nKrSVeqEta0LqOVB5Jgg0UaUPrwlISGyjSitZ1ZHEv\nPiIiImtiA0VERLrUaAMVHx8PNze3WvepSUxMhKenJ0JDQxEaGorU1FRVQxI1d6wjoqZrtIGaOnXq\nQ4VjMBgwd+5cZGRkICMjAyNGjFAtIFFLwDoiarpGG6iIiAg4Ozs/9DxP2hJZjnVE1HSSz0GtWrUK\nRqMRCQkJHIGZSCLWEVH9JDVQM2bMQFZWFjIzM+Hh4YHXX3+9njkTa0xmKasisjIzau+36rG8ju6d\nr7o/mc1mVXMRyWeGEnUk6YaFNYdjmTZtGqKiouqZM1HK4ok0ZIJaQx09yPI64lBH1NyYoPpQR/XJ\nz8+vfvz555/X6plERJZhHRE1rNFvUJMmTcLBgwfx888/w8vLC4sWLYLZbEZmZiYMBgN8fHywbt06\na2QlarZYR0RNx6GOqEXSeogWJXGoI9KK1nXEkSSIqNVzdHSpvh+X1MnR0UXrzWhxJHWSICJqSYqL\nb0LuN4Xi4vpu3EpSSRrq6MaNG4iMjIS/vz+GDRvG6zeIGsE6Imo6SUMdJSUlITIyEufPn8eQIUOQ\nlJSkWkCiloB1RNR0koY6Sk5ORlxcHAAgLi4OO3fuVCcdUQvBOiJqOknnoAoLC+Hm5gYAcHNzQ2Fh\noaKhiFqDptTR9u3bZa2rbdu2iI6Oho2NjazlEFmT7E4S93uw1C2xxmMTal9ZTKRHZmgxLFfDdQTE\nxv539WMbG1fY2rrWO29d7tz5AqdOZaBnz56SMxJZzgwl6khSA+Xm5oaCggK4u7sjPz+/1pAttSVK\nT0akCROsNdSR5XUElJWdkrUuBwd/XktFVmSCZkMdRUdHY9OmTQCATZs2ISYmRtLKiVoz1hFRwxpt\noCZNmoSBAwfi3Llz8PLywsaNGzFv3jzs3bsX/v7+2L9/P+bNm2eNrETNFuuIqOk41BG1SFoP0aIk\nJbbFwcEfJ07shr+/vzKhWhhl9hft9xWlaV1HHOqIiIh0iQ0UERHpkqxu5t7e3nB0dISNjQ3s7Oxw\n/PhxpXIRtRqsI6K6yWqgDAYDzGYzXFw4ii+RVKwjorrJPsTX0k4KEmmBdUT0MFkNlMFgwNChQxEW\nFob169crlYmoVWEdEdVN1iG+I0eOwMPDA9euXUNkZCR69eqFiIiIGnMk1nhsAoc6Iv0zw9pDHTVe\nRwBriZoXMzQb6ug+Dw8PAECXLl3w3HPP4fjx4w00UETNgQnWGurovsbrCGAtUfNigmZDHQFAaWkp\niouLAQC3b9/GP/7xj1o3YyOixrGOiOon+RtUYWEhnnvuOQBARUUFXnzxRQwbNkyxYEStAeuIqH6S\nGygfHx9kZmYqmYWo1WEdEdWPI0kQEZEusYEiIiJdktxApaamolevXujZsyeWLVumZCZVmM1mrSNU\n01MWQF959JTFWppbLTWmNX6G1tZa3mNJDVRlZSVmzZqF1NRUnD59Glu3bsWZM2eUzqYoPX2gesoC\n6CuPnrJYQ3Ospca0ts9QC63lPZbUQB0/fhw9evSAt7c37OzsMHHiROzatUvpbEQtHmuJqH6SevFd\nvXoVXl5e1T97enri66+/rmPORImx7jsl8/VE+matWrp797qs1xNpQkjw6aefimnTplX/vHnzZjFr\n1qxa8/j5+QncuxUjJ07NdvLz85NSIqwlTpxqTFLrSNI3qG7duiEnJ6f655ycHHh6etaa5+LFi1IW\nTdSqsJaI6ifpHFRYWBguXLiA7OxslJWVYfv27YiOjlY6G1GLx1oiqp+kb1C2trZYvXo1hg8fjsrK\nSiQkJCAgIEDpbEQtHmuJqH4GIXinNCIi0h/ZI0lYcpHh7Nmz0bNnTxiNRmRkZMhdpaw8W7ZsgdFo\nRJ8+ffDUU0/hu+++0yzLfd988w1sbW3xf//3f5pmMZvNCA0NRXBwMEwmk2pZLMnz888/Y8SIEQgJ\nCUFwcDA+/PBD1bLEx8fDzc2twVHErbEP662WGtNYXrPZDCcnJ4SGhiI0NBSLFy/WIOU9evmMm6Kx\nzHp6f4F7508HDx6MoKAgBAcH4913361zvia9zzI6IImKigrh5+cnsrKyRFlZmTAajeL06dO15vni\niy/EyJEjhRBCHDt2TISHh8tZpew8R48eFbdu3RJCCLFnzx7V8liS5f58gwcPFqNHjxaffvqpZllu\n3rwpAgMDRU5OjhBCiGvXrqmSxdI8CxcuFPPmzavO4uLiIsrLy1XJc+jQIZGeni6Cg4Pr/L019mG9\n1VJjLMl74MABERUVpVHC2vTwGTdVY5n19P4KIUR+fr7IyMgQQghRXFws/P39Ze/Dsr5BWXKRYXJy\nMuLi4gAA4eHhuHXrFgoLC+WsVlaeJ598Ek5OTtV5cnNzNcsCAKtWrcK4cePQpUsXVXJYmuVvf/sb\nxo4dW92D7NFHH9U0j4eHB4qKigAARUVF6Ny5M2xtZd1fs14RERFwdnau9/fW2If1VkuNsXT/Fjo5\ng6CHz7ipGssM6Of9BQB3d3eEhIQAAOzt7REQEIC8vLxa8zT1fZbVQNV1keHVq1cbnUetRsGSPDV9\n8MEHGDVqlGZZrl69il27dmHGjBkAAIPBoFmWCxcu4MaNGxg8eDDCwsKwefNmVbJYmmf69Ok4deoU\nunbtCqPRiJUrV6qWpzHW2If1VkuNsSSvwWDA0aNHYTQaMWrUKJw+fdraMS2mp/fWUnp+f7Ozs5GR\nkYHw8PBazzf1fZb1L6mlf1AfbOXV+kPclOUeOHAAGzZswJEjRzTLMmfOHCQlJcFgMEAIodp/Q5Zk\nKS8vR3p6Ovbt24fS0lI8+eSTeOKJJ9CzZ09N8ixZsgQhISEwm824dOkSIiMj8e2338LBwUHxPJZQ\nex/WWy01xpL19u3bFzk5OejQoQP27NmDmJgYnD9/3grppNHLe2spvb6/JSUlGDduHFauXAl7e/uH\nft+U91nWNyhLLjJ8cJ7c3Fx069ZNzmpl5QGA7777DtOnT0dycnKjX6HVzHLy5ElMnDgRPj4++Oyz\nzzBz5kwkJydrksXLywvDhg1D+/bt0blzZzz99NP49ttvFc9iaZ6jR49i/PjxAAA/Pz/4+Pjg3Llz\nquRpjDX2Yb3VUmMsyevg4IAOHToAAEaOHIny8nLcuHHDqjktpaf31lJ6fH/Ly8sxduxYvPTSS4iJ\niXno901+n+WcFCsvLxe+vr4iKytL3L17t9ETu1999ZWqJx8tyXPlyhXh5+cnvvrqK9VyWJqlpn/7\nt38Tn332mWZZzpw5I4YMGSIqKirE7du3RXBwsDh16pRmeV577TWRmJgohBCioKBAdOvWTVy/fl2V\nPEIIkZWVZdEJdLX2Yb3VUmMsyVtQUCCqqqqEEEJ8/fXX4rHHHtMg6b9o/RlL0VBmvb2/VVVVYsqU\nKWLOnDn1ztPU91lWAyWEECkpKcLf31/4+fmJJUuWCCGEWLt2rVi7dm31PK+88orw8/MTffr0ESdP\nnpS7Sll5EhIShIuLiwgJCREhISGif//+mmWpSc0GytIsy5cvF4GBgSI4OFisXLlStSyW5Ll27ZoY\nM2aM6NOnjwgODhZbtmxRLcvEiROFh4eHsLOzE56enuKDDz7QZB/WWy01prG8q1evFkFBQcJoNIon\nn3xS9X8KG6KXz7gpGsusp/dXCCEOHz4sDAaDMBqN1X9fU1JSZL3PvFCXiIh0ibd8JyIiXWIDRURE\nusQGioiIdIkNFBER6RIbKCIi0iU2UEREpEtsoIiISJfYQBERkS6xgSIiIl1iA0VERLrEBkpnvL29\nsW/fPq1jEDVrrKOWgQ2UzhgMBkXuQ3PhwgW0a9cOU6ZMUSAVUfMit45MJhPat28PBwcHODg4ICAg\nQMF0ZCk2UC3UK6+8ggEDBuj+pmtEemQwGPDee++huLgYxcXFOHPmjNaRWiU2UDp25swZ+Pr6Yvv2\n7U163bZt2+Ds7IwhQ4aodpdeouZCah2xdrTHBkqn0tPTMWLECKxevRoTJkzAmDFj4OzsXOcUHR1d\n/bqioiIsXLgQK1asYIFRqye1jgBg/vz56NKlCwYNGoSDBw9qtAWtm63WAehhBw8exIYNG7BlyxY8\n/fTTAIDdu3db9NoFCxZg2rRp6Nq1Kw/vUasmp46WLVuGoKAgtG3bFlu3bkVUVBQyMzPh6+urZmR6\nAL9B6YwQAuvWrcNTTz1VXVSWyszMxL59+zBnzpzqZRG1RnLqCAAGDBiAjh07ws7ODrGxsXjqqaeQ\nkpKiQlJqCBsonTEYDFi3bh2uXLmCuXPnVj8/cuTI6h5FD06jR48GAJjNZmRnZ6N79+7w8PDA22+/\njc8++wxhYWFabQ6RJuTUEekHb/muMz4+Pvjggw/Qr18/DBkyBJGRkVi6dKlFr/31119RXFwM4N5/\nkH/605+QnZ2NtWvXonPnzmrGJtIVOXX0yy+/4NixY3jmmWdga2uL7du34/e//z0yMzPRo0cPlZNT\nTTwHpVNOTk7Yu3cvBg8ejLZt22LRokWNvqZ9+/Zo37599c/29vZo3749GydqtaTUUXl5ORYsWICz\nZ8/CxsYGAQEB2LVrFxsnDTT4DSonJwexsbH46aefYDAY8PLLL2P27Nm4ceMGJkyYgCtXrsDb2xs7\nduxAp06drJmbqFmJj4/HF198AVdXV3z//ffVz69atQrvv/8+bGxsMHr0aCxbtkzDlET60mADVVBQ\ngIKCAoSEhKCkpAT9+vXDzp07sXHjRjz66KN44403sGzZMty8eRNJSUnWzE3UrBw+fBj29vaIjY2t\nbqAOHDiAJUuWICUlBXZ2drh27Rq6dOmicVIi/Wiwk4S7uztCQkIA3DtcFBAQgKtXryI5ORlxcXEA\ngLi4OOzcuVP9pETNWEREBJydnWs9t2bNGsyfPx92dnYAwMaJ6AEW9+LLzs5GRkYGwsPDUVhYCDc3\nNwCAm5sbCgsLVQtI1FJduHABhw4dwhNPPAGTyYQTJ05oHYlIVyzqJFFSUoKxY8di5cqVcHBwqPW7\n+gZl7NGjBy5duqRMSiKN+Pn54eLFi6osu6KiAjdv3sSxY8fwzTff4IUXXsDly5cfms9gcARQrEoG\nImuQWkeNfoMqLy/H2LFjMWXKFMTExAC4962poKAAAJCfnw9XV9eHXnfp0iUIIaw+LVy4sNWsl9uq\n/qTmP1menp54/vnnAQD9+/dHmzZtcP369TrmLAYgZE0ODj1x7tw5q7ynzW25zTFzc1uu1DpqsIES\nQiAhIQGBgYHVoxMAQHR0NDZt2gQA2LRpU3XDRUR1i4+PR1hYGC5cuFD9XExMDPbv34+3334bBoMB\nd+7c4SUBRDU02EAdOXIEH3/8MQ4cOIDQ0FCEhoYiNTUV8+bNw969e+Hv74/9+/dj3rx51spLKnB0\ndKk+VGvJtGjRooeec3R00XozdO3HH3+EwWBAWVkZvLy8sHHjRsTHx+P7779HYmIi2rZti/fff1/r\nmES60uA5qEGDBqGqqqrO36WlpakSSC6TydRq1qvUOouLb+LeoSBLmQHUXndxsboD02r1uSolLS0N\n2dnZiIqKqnUdlI2NDY4cOYJnn30WgwYNsmomtd7T5rZcNZfN5cqj2lBHBoMBKi2aFHavk4vcz6pl\nft5K7scPNlC7du2C2WzGihUr4OPjg5MnT8LF5eFvokp8Pg4O/jhxYjf8/f1lLYdICql1xKGOiDRQ\nWlqKJUuWYO/evdXPtcQGnkgONlBEGrh06RKys7NhNBoBALm5uejXrx+OHz9eZ69YILHGYxMePMRK\npCdmsxlms1n2cniIj3iIrwFqHuKriYf4qCWTWkeNXgcVHx8PNzc39O7du/q5xMREeHp61urZR0T1\n8/X1hZ+fH06dOlXdi++Pf/wjAgICYDQa8dNPP6GoqEjrmES60mgDNXXq1IcaIIPBgLlz5yIjIwMZ\nGRkYMWKEagGJWoJNmzbhxIkTCAoKQk5ODqZOnYphw4bh1KlT+Pbbb/Hqq69i7dq1Wsck0pVGG6i6\nBrkEeEKXqCnqqqPIyEi0aXOvBMPDw5Gbm6tFNCLdknzL91WrVsFoNCIhIQG3bt1SMhNRq7NhwwaM\nGjVK6xhEuiKpF9+MGTPwX//1XwCABQsW4PXXX8cHH3zw0HyJiYnVj00mk+4uAiN6kFK9j5rirbfe\nQtu2bTF58uQG5kqs8dgE9uIjPbNqL76Geh/V9zv24ms+2Iuvfmr34vvwww+xfv167Nu3D+3atas3\nA3vxUXOmWi++uuTn51c//vzzz2v18CMiy6SmpmL58uXYtWtXvY0TUWvW6CG+SZMm4eDBg/j555/h\n5eWFRYsWwWw2IzMzEwaDAT4+Pli3bp01shI1W76+vrhy5QqEENV19NZbbyEvLw+enp5o27Ytxo8f\nX+ehcqLWqtEGauvWrQ89Fx8fr0oYopZq06ZNsLe3R2xsbPUhvrNnz+LRRx/FG2+8gWXLluHmzZsa\npyTSF8m9+IjIcnV1M09OTkZcXBwAIC4uDjt37tQiGpFusYEi0khhYSHc3NwA3LtLdWFhocaJiPSF\ng8US6cD9Gz/WL7HGYxPYzZz0TKlu5mygiDTi5uaGgoICuLu7Iz8/v55RzO9LtFYsItkevO510aJF\nkpbDQ3xEGomOjsamTZsA3OtEERMTo3EiIn2RNJr5jRs3EBkZCX9/fwwbNoxDHRE1YtKkSRg4cCDO\nnTtXPZr5vHnzsHfvXjz66KNYvHgxkpOTMXnyZNy9e1fruES6IGk086SkJERGRuL8+fMYMmQIkpKS\nVAuoN46OLtXnC6ROjo4P3/OHWratW7ciLy8PZWVl1aOZu7i44K9//SscHR1x/fp1nD59GpWVldi2\nbZvWcVsd1rU+SRrNvDV3jy0uvol7w85In+4tgwhwdHSEnZ0dSktLUVFRgdLSUnTr1k3rWK0O61qf\nJHWSsLR77F/+8hfpyX7Tr18/9OvXT/ZyiPTIxcUFr7/+Orp374727dtj+PDhGDp0qNaxiHRBdi++\nhrrHvvLKv4ZAatPGAzY2XZu07MrKyxg8eC9SUz+RlZHIUtYezfzSpUt45513kJ2dDScnJ4wfPx5b\ntmzBiy+++MCciTUem6BVN3NHRxdFvik4ODijqOiGAolIjzTtZm5p99iKipOywgGfQIgdMpdBZDml\nusda6sSJExg4cCA6d+4MAHj++edx9OjRRhoo7fzrUJjc5TR0zRc1d5p2M2f3WCJl9OrVC8eOHcOv\nv/4KIQTS0tIQGBiodSwiXWi0gWqoe6y/vz/279+PefPmWSMrUYtjNBoxfvx4uLm5oX379ti1axeM\nRqPWsYh0QdJo5gCQlpameBii1ujy5ct45513EB8fj4qKCty+fVvrSEQAlDvnKBWHOiLS0C+//ILD\nhw9XHzK3tbWFk5OTxqmI7lHqnCMg7Zwjhzoi0lBWVha6dOmCqVOnom/fvpg+fTpKS0u1jkWkC2yg\niDRUUVGB9PR0zJw5E+np6ejYsWM9I7Mk1pjM1gtIVqfEqBbaj2xhRu19Vhoe4iPSkKenJzw9PdG/\nf38AwLhx4xpooKg1aBld+U2ofa0eRzMnanbc3d3h5eWF8+fPA7jX+SgoKEjjVET6IOsblLe3Nxwd\nHWFjYwM7OzscP35cqVxErcaqVavw4osv4ocffkCnTp1w9uxZrSMR6YKsBspgMMBsNsPFhaP4kr5o\n3T22KYxGIyZNmoSTJ0+iuLiYvfiIfiP7EJ8QSnRBJFKWEqNTK9O9tnG5ublISUnBtGnTWE9ENchq\noAwGA4YOHYqwsDCsX79eqUxErcprr72G5cuXo00bnhImqknWIb4jR47Aw8MD165dQ2RkJHr16oWI\niIgacyTWeGyCViMwE1nODGt24969ezdcXV0RGhrayOjPiTUem8BaIn0zQ4k6ktVAeXh4AAC6dOmC\n5557DsePH2+ggSJqDkxQonuspY4ePYrk5GSkpKTgzp07KCoqQmxsLD766KMH5kxUNQeRskzQtJt5\naWkpiouLAQC3b9/GP/7xD/Tu3Vvq4ohapRkzZqBHjx7o0KED2rVrBx8fnzoaJ6LWSXIDVVhYiIiI\nCISEhCA8PBxjxozBsGHDlMxG1OLZ2dlhxYoVOHXqFNasWYPs7GycOXNG61hEuiD5EJ+Pjw8yMzOV\nzELU6ri7u8Pd3R0AMGLECAwdOhR5eXkICAjQOBmR9thtiEgnsrOzkZGRgfDwcK2jEOkCGygiHSgp\nKcG4ceOwcuVK2Nvbax2HSBc4WCyRxsrLyzF27Fi89NJLiImJqWeuxBqPTWA3c9I3MzTvZk5E8ggh\nkJCQgMDAQMyZM6eBOROtFYlIASZo2s08NTUVvXr1Qs+ePbFs2TKpi1Fcwxc7qrpm66+R29rsHTly\nBB9//DHWrFmDRx55BF27dkVqaqpV1q3ee6rOcpvjPsD3WB5JDVRlZSVmzZqF1NRUnD59Glu3btVN\n11j+0bbKmq2/Rp0VjlKefPJJ+Pr64uzZsygpKYGrqysee+wxq6ybfzzVx/dYHkkN1PHjx9GjRw94\ne3vDzs4OEydOxK5du5TORtTisZaI6ifpHNTVq1fh5eVV/bOnpye+/vrrOubMk5rrNzdkvp5I36xV\nS1VVFbJeT6QJIcGnn34qpk2bVv3z5s2bxaxZs2rN4+fnp8S9Djhx0nTy8/OTUiKsJU6cakxS60jS\nN6hu3brv2R6WAAAZTUlEQVQhJyen+uecnBx4enrWmufixYtSFk3UqrCWiOon6RxUWFgYLly4gOzs\nbJSVlWH79u2Ijo5WOhtRi8daIqqfpG9Qtra2WL16NYYPH47KykokJCRw7DAiCVhLRPUzCMF7TBMR\nkf7IHovPkgt2Z8+ejZ49e8JoNCIjI0PuKi1ar9lshpOTE0JDQxEaGorFixfLWl98fDzc3NwavOeV\nGtvZ2HqV3k7g3nmQwYMHIygoCMHBwXj33XfrnE/p7bVkvUpv7507dxAeHo6QkBAEBgZi/vz5dc6n\nxmf7ILVqSY1aUase1Nrf1dqn1dxn1do3LVmunDqrrKxEaGgooqKi5OeV0QFJVFRUCD8/P5GVlSXK\nysqE0WgUp0+frjXPF198IUaOHCmEEOLYsWMiPDxcziotXu+BAwdEVFSU7HXdd+jQIZGeni6Cg4Pr\n/L0a22nJepXeTiGEyM/PFxkZGUIIIYqLi4W/v79VPldL1qvG9t6+fVsIIUR5ebkIDw8Xhw8frvV7\ntT7bmtSqJbVqRa16UGt/V2ufVnufVWvfbGy5cjK//fbbYvLkyXW+vql5ZX2DsuQiw+TkZMTFxQEA\nwsPDcevWLRQWFspZrcUXNwoFj15GRETA2dm53t+rsZ2WrBdQdjuBe/coCgkJAQDY29sjICAAeXm1\nr8NRY3stWS+g/PZ26NABAFBWVobKykq4uLjU+r1an21NatWSWrWiVj2otb+rtU+rvc+qtW82tlyp\nmXNzc5GSkoJp06bV+fqm5pXVQNV1keHVq1cbnSc3N1fOai1ar8FgwNGjR2E0GjFq1CicPn1a1jql\nZJK7nZZQezvru0eR2ttb33rV2N6qqiqEhITAzc0NgwcPRmBgYK3fW+OzVauWtKoVtd4zJbKqtU+r\nsc+qtW82tlypmV977TUsX74cbdrU3bQ0Na+s0cwNBoNF8z3Yklr6Ojnr7du3L3JyctChQwfs2bMH\nMTExOH/+vKz1Nkbp7bSEmtvZ2D2K1Nrehtarxva2adMGmZmZ+OWXXzB8+HCYzWaYTKZa86j92apV\nS1rWihrvmdysau3Tau2zau2bjS1XSubdu3fD1dUVoaGhDY7p15S8sr5BWXKR4YPz5Obmolu3bnJW\na9F6HRwcqr/Gjhw5EuXl5bhxQ72hk9TYTkuotZ2N3aNIre1tbL1qfq5OTk4YPXo0Tpw4Uet5a3y2\natWSVrWi1nsmJ6ta+7Q19lm19s36lisl89GjR5GcnAwfHx9MmjQJ+/fvR2xsrLy8ks6C/aa8vFz4\n+vqKrKwscffu3UZP7H711VeKnGC2ZL0FBQWiqqpKCCHE119/LR577DHZ683KyrLopLBS22nJetXY\nzqqqKjFlyhQxZ86ceudRY3stWa/S23vt2jVx8+ZNIYQQpaWlIiIiQqSlpdWaR83P9j61aknNWlGr\nHtTY39Xap9XcZ9XaNy1Zrtw6M5vNYsyYMQ8939S8sg7x1XeR4bp16wAAv//97zFq1CikpKSgR48e\n6NixIzZu3ChnlRav99NPP8WaNWtga2uLDh06YNu2bbLWOWnSJBw8eBA///wzvLy8sGjRIpSXl6u6\nnZasV+ntBP51j6I+ffogNDQUALBkyRL8+OOP1etVY3stWa/S25ufn4+4uDhUVVWhqqoKU6ZMwZAh\nQ1Tfhx+kVi2pVStq1YNa+7ta+7Sa+6xa+6Yly1Wizu4fupOTlxfqEhGRLsm+UJeIiEgNbKCIiEiX\n2EAREZEusYEiIiJdYgNFRES6xAaKiIh0iQ0UERHpEhsoIiLSJTZQRESkS2ygiIhIl9hA6Yy3tzf2\n7dundQyiZo111DKwgdIZg8Eg+74527ZtQ0BAAOzt7dGjRw/885//VCgdUfMgt47s7e3h4OBQPdna\n2mL27NkKJiRLyBrNnPRn7969mDdvHnbs2IEBAwYgPz9f8VukE7V0JSUl1Y9v374Nd3d3vPDCCxom\nap34DUrHzpw5A19fX2zfvt3i1yxcuBALFy7EgAEDAAAeHh7o2rWrWhGJdE9KHdX06aefws3NDYMG\nDVI4GTWGDZROpaenY8SIEVi9ejUmTJiAMWPGwNnZuc4pOjoaAFBZWYmTJ0/ip59+Qs+ePeHl5YVX\nX30Vd+7c0XhriLQhpY4etGnTpofuDEvWwftB6YyPjw9iY2OxYcMGbNmyBU8//bTFr83Ly4OnpyfC\nwsLw97//Hba2tnj22WdhMpmwePFiFVMT6YucOqrpypUr8PPzw6VLl/DYY48pnJIaw29QOiOEwLp1\n6/DUU081uajat28PAHj11Vfh5uaGzp07Y+7cuUhJSVEjKpFuyamjmjZv3oyIiAg2ThphA6UzBoMB\n69atw5UrVzB37tzq50eOHFmrV1HNafTo0QAAZ2dneHp6ahWdSDfk1FFNH330EeLi4qwZnWpgLz4d\ncnBwQGpqKoYMGYL58+dj6dKl2LNnj0WvnTp1KlatWoURI0bA1tYWK1asQFRUlMqJifRHTh0BwNGj\nR5GXl4fx48ermJIawgZKp5ycnLB3714MHjwYbdu2xaJFiyx63YIFC/Dzzz/D398f7dq1w4QJE/Dm\nm2+qnJZIn6TWEXDv29PYsWPRsWNHFRNSQxrsJBEfH48vvvgCrq6u+P7776ufX7VqFd5//33Y2Nhg\n9OjRWLZsmVXCEjVXddXSH//4R+zevRtt27aFn58fNm7cCCcnJ42TEulHg+egpk6ditTU1FrPHThw\nAMnJyfjuu+/www8/4A9/+IOqAYlagrpqadiwYTh16hS+/fZb+Pv7Y+nSpRqlI9KnBhuoiIgIODs7\n13puzZo1mD9/Puzs7AAAXbp0US8dUQtRVy1FRkaiTZt7JRgeHo7c3FwtohHpVpN78V24cAGHDh3C\nE088AZPJhBMnTqiRi6hV2bBhA0aNGqV1DCJdaXIniYqKCty8eRPHjh3DN998gxdeeAGXL19+aL4e\nPXrg0qVLioQk0oqfnx8uXryo6jreeusttG3bFpMnT67z96wlau6k1lGTv0F5enri+eefBwD0798f\nbdq0wfXr1x+a79KlSxBC6GZauHCh5hn0mEVvefSURQihesPw4YcfIiUlBVu2bKl3Hi1qSavPoTWt\ntzVtq9Q6anIDFRMTg/379wMAzp8/j7KyMnTu3FnSyolas9TUVCxfvhy7du1Cu3bttI5DpDsNNlC+\nvr7w8/PDqVOn4OXlhY0bNyI+Ph6XL1+Gh4cHHn/8caxevdpaWYmarUmTJmHgwIE4d+4cvLy8sGHD\nBrz66qsoKSlBZGQkQkNDMXPmTK1jEumLaMChQ4dEenq6CA4OrvX8jz/+KIYPHy68vb3F9evX63xt\nI4u2ugMHDmgdoZqesgihrzzt29sLALInBwdnRfLoYT/WIoNW+0RrWm9r2lap+3Cjo5lnZ2cjKiqq\n1oW648ePx4IFC/Dss8/i5MmTcHFxeeh1BoMBjSya6CH37oKqxH6jzP6n1H5c14W6N27cwIQJE3Dl\nyhV4e3tjx44d6NSpk2oZSF2Oji4oLr4pezkODs4oKrqhQCL9kLoPN/kc1K5du+Dp6Yk+ffo0eWVE\nrVVdF+omJSUhMjIS58+fx5AhQ5CUlKRROlLCvcZJ/gEAJRq5lqJJ3cxLS0uxZMkS7N27t/q5hlrF\nxMTE6scmkwkmk6nJAYmsyWw2w2w2K77ciIgIZGdn13ouOTkZBw8eBADExcXBZDKxkSKqoUmH+L7/\n/nsMHToUHTp0AADk5uaiW7duOH78OFxdXWsvmIclSIKWeogPePhwubOzM27evPffshACLi4u1T+r\nlYHUo7d9V0+k7sNN+gbVu3dvFBYWVv/s4+NT7zkoIrKcwWD47Q9c3Xg0gpoTxY5ENNSDYuLEiaJ9\n+/YCgLC1tRUbNmwQQgjxhz/8QfTq1Uu0bdtWjB49Wty6dUuxXhvUugEQgFBgUmb/U3I/zsrKqtUj\n9vHHHxf5+flCCCHy8vLE448/rnoGUo/e9l09kbpNDXaS2Lp1K7788kukp6ejV69emDp1KoB/jcJ8\n9+5dBAcHcxRmIgmio6OxadMmAMCmTZsQExOjcSIifWm0Fx9HYSaS78ELdTdu3Ih58+Zh79698Pf3\nx/79+zFv3jytYxLpiuw76m7YsAGTJk1SIgtRi7V169Y6n09LS7NyEqLmQ1YD1dgozDyxS82NWt3M\nG7J06VJ8/PHHaNOmDXr37o2NGzfikUcesWoGIj1qtJs5UPdoEh9++CHWr1+Pffv21TnQJbvGkhR6\n66qr9n6cnZ2N3/3udzhz5gweeeQRTJgwAaNGjUJcXJzVMpAy9Lbv6olqI0nEx8cjLCwMFy5cqH7u\nk08+waxZs1BQUIDo6GjcunWrySsmIsDR0RF2dnYoLS1FRUUFSktL0a1bN61jEelCo4f4fvzxRxgM\nBpSVlcHLywuLFi3C66+/Djs7Ozg6OuKHH37A7373O6Snp1sjL1GL4uLigtdffx3du3dH+/btMXz4\ncAwdOvSh+fLy8mStx87ODl26dJG1DCJrk3SIr1evXjh48CDc3NxQUFAAk8mEs2fP1l4wD0uQBHo7\nTKL2fnzp0iVERUXh8OHDcHJywvjx4zFu3Di8+OKLtTLY2tpX/2xj0xY2Nk07R3Xnzk/IyrqM7t27\nK5adatPbvqvE4LVSB6598FzuokWL1B9J4r7CwkK4ubkBANzc3GqNLkFEljtx4gQGDhxYfdPP559/\nHkePHq3VQAFARUVxjcdNX4+Dgz/u3LkjKys1L/8avFbOMuof3aQhD3aKW7RokaTlNHk08wc1NkQL\nEdWvV69eOHbsGH799VcIIZCWlobAwECtYxHpgqRvUPcP7bm7uyM/P/+hgWLvYzdzam6s3c3caDQi\nNjYWYWFhaNOmDfr27YuXX37Zausn0jNJ56DeeOMNdO7cGf/xH/+BpKQk3Lp166HbBPAcFEmht+P4\netiPlXhPHBz8ceLEbvj7+ysTih6ix31Xfh5t66jRQ3z1DdGyfv16PPLII1i8eDEuXryIu3fvSgpO\n1NrdunUL48aNQ0BAAAIDA3Hs2DGtIxHpQqMN1NatW5GXl4eysjLk5ORg6tSpKCoqQlVVFYqKilBS\nUgIbGxts27bNGnmJWpx///d/x6hRo3DmzBl89913CAgI0DoSkS5IOgdV8+JCGxsbXlxIJNEvv/yC\nw4cPV49qbmtrCycnJ41TEemDpF58NS8u7Nq1Kzp16lTnxYVE1LCsrCx06dIFU6dORd++fTF9+nSU\nlpZqHYtIFyQ1UJcuXcI777yD7Oxs5OXloaSkBFu2bHlovvtd0OVMsbG/l72RRHpVUVGB9PR0zJw5\nE+np6ejYseNDHY6IWitJh/gsvbgQWFjjsem3qSk+QWHhDikRH6LEVdWA9CurqXmwdjdzT09PeHp6\non///gCAcePG1dNAJdZ4bELTa4lICltNr3OV1ED16tUL//M//4Nff/0V7dq1Q1paGgYMGFDHnIny\n0ilIiauq7y2HFyW3ZEpdAW8pd3d3eHl54fz58/D390daWhqCgoLqmDNR1RxEdauAUl3npZDUQBmN\nRowfPx5ubm4oKytDu3btMH36dEkBiFq7VatW4cUXX0RZWRn8/PywceNGrSMR6YLkGxZevnwZ77zz\nDuLj41FRUYHbt28rmYuo1TAajfjmm2+0jkGkO5I6SdzvGhsfHw+AXWOJ5KqsrERoaCiioqJUW0ff\nvgNkd1pydHRRLR/RgyQ1UOwaS6SslStXIjAwUNUT0rdv/4J75xOkT0p0NCKylKRDfPe7xq5evRr9\n+/fHnDlzkJSUhP/+7/9+YM7EGo9NYM+je9ijUL+s3YsPAHJzc5GSkoI333wTf/7zn626biI9k9RA\nSesaS/exR6F+WbsXHwC89tprWL58OYqKilRfF1FzIukQX82usQAa6BpLRA3ZvXs3XF1dERoaqvmo\n6UR6I7kX3/2usT/88AM6der00C3fiahxR48eRXJyMlJSUnDnzh0UFRUhNjYWH3300QNzJtZ4bEJz\nP1yut8PcSuWh+8y/TfJYdD+o+vz5z3/GyZMnUVxcjOTk5NoLVuReJJ9g2LAd+PLLT2QuR1/3atFT\nFr1R7r2xw72LDOWz1nt88OBB/OlPf8Lf//73Ws8rdT+o4uILspejr3sVAfrKw21qaDmq3A+qPvdP\n7E6bNq3F/ZGkluD+FfByJ+vSclgZIr2RfIiPJ3aJlPXMM8/gmWee0ToGkW5IaqBqnthtuEtuYo3H\nJjT34+YtlRLH31tOl3czlDh2TkTySWqgpJ3YJb1Sott7y+nybkLtf6TU7Waek5OD2NhY/PTTTzAY\nDHj55Zcxe/ZsVddJ1FzI6iQBqHtil50k1M8CKHcyVT9ZAK1P7lqqoKAABQUFCAkJQUlJCfr164ed\nO3fWuu07O0k0uCQd5eE2NbQcq3aSyMnJweDBgxEXF4eDBw/i3XfflbooolbL3d0dISEhAAB7e3sE\nBAQgLy9P41RE+iC5gbKzs8OKFSuq76r73nvv4cyZM0pmI2pVsrOzkZGRgfDwcK2jEOmC5F587u7u\ncHd3B1D7P7+ahyaIyDIlJSUYN24cVq5cCXt7+zrmSKzx2AR2OCJ9M0OJzkaSG6ia+J8fkXTl5eUY\nO3YsXnrpJcTExNQzV6I1IxHJZIISnY1kN1AN/+eXWOOxCfyvj/TPDGt2MxdCICEhAYGBgZgzZ47V\n1kvUHMhqoBr/zy9RzuKJNGCCNbuZHzlyBB9//DH69OmD0NBQAMDSpUsxYsQIVddL1BxIbqD4nx+R\nfIMGDUJVVZXWMYh0SXIvvrfffhubN2/G+++/j65duyI0NBSpqalKZiNqFVJTU9GrVy/07NkTy5Yt\n0zrOb8ytbL1aMLey9TadpAaqsrISa9euRVZWFkpKSuDq6oq//e1vPCxB1ESVlZWYNWsWUlNTcfr0\naWzdulUnl2uYW9l6tWBuZettOkkN1PHjx9GjRw94e3vDzs4OEydOxK5du5TORtTisZaI6ifpHNTV\nq1fh5eVV/bOnpye+/vrrh+ZzdIySngxAeXkebG17yFoGkZ5Zq5Z+/fWqrNcTaUFSA2XJPWv8/Pxw\n6dJuKYuvJSUlHQbDDtnLuUeZAU2VuWePnrIASuTRUxalluPn56dAjvpZs5Z+W2MT5q27B6P6n3PT\nek7qa79r6jKaw3usXR1JaqC6deuGnJyc6p9zcnLg6elZa56LFy9KCkTUmrCWiOon6RxUWFgYLly4\ngOzsbJSVlWH79u2Ijo5WOhtRi8daIqqfpG9Qtra2WL16NYYPH47KykokJCRwDD4iCVhLRPWTfT8o\nIiIiNUi+ULcherrwMD4+Hm5ubujdu7emOYB/3UMrKCgIwcHBmt5D686dOwgPD0dISAgCAwMxf/58\nzbLUVFlZidDQUERFyeu1Jpe3t3f18EMDBgzQJIMWdaRVvWhRG1rWgBb7uVb79K1btzBu3DgEBAQg\nMDAQx44ds/zFQmEVFRXCz89PZGVlibKyMmE0GsXp06eVXo3FDh06JNLT00VwcLBmGe7Lz88XGRkZ\nQgghiouLhb+/v6bvze3bt4UQQpSXl4vw8HBx+PBhzbLc9/bbb4vJkyeLqKgoTXN4e3uL69eva7Z+\nrepIq3rRqja0qgEt9nOt9unY2FjxwQcfCCHuvc+3bt2y+LWKf4PS24WHERERcHZ21mz9Nent7qkd\nOnQAAJSVlaGyshIuLi6aZQGA3NxcpKSkYNq0aareZt1SWmbQqo60qhetakOLGtByP7f2+n755Rcc\nPnwY8fHxAO6dc3VycrL49Yo3UHVdeHj1Ki8SfJAe7qFVVVWFkJAQuLm5YfDgwQgMDNQsCwC89tpr\nWL58Odq0UeXIc5MYDAYMHToUYWFhWL9+vdXX35rryJq1oUUNaLWfa7FPZ2VloUuXLpg6dSr69u2L\n6dOno7S01OLXK/4OKXeBWcvV+N1TraNNmzbIzMxEbm4uDh06BLPZrFmW3bt3w9XVFaGhobr49nTk\nyBFkZGRgz549eO+993D48GGrrr+11pG1a8PaNaDlfq7FPl1RUYH09HTMnDkT6enp6NixI5KSkix+\nveINlCUXHrZmlt091bqcnJwwevRonDhxQrMMR48eRXJyMnx8fDBp0iTs378fsbGxmuXx8PAAAHTp\n0gXPPfccjh8/btX1t8Y60rI2rFUDWu7nWuzTnp6e8PT0RP/+/QEA48aNQ3p6uuULUPqEWHl5ufD1\n9RVZWVni7t27mneSEEKIrKwsXXSSqKqqElOmTBFz5szROoq4du2auHnzphBCiNLSUhERESHS0tI0\nTnWP2WwWY8aM0Wz9t2/fFkVFRUIIIUpKSsTAgQPFl19+adUMWtaRFvWiRW1oXQPW3M+13KcjIiLE\nuXPnhBBCLFy4ULzxxhsWv1bxBkoIIVJSUoS/v7/w8/MTS5YsUWMVFps4caLw8PAQbdu2FZ6enmLD\nhg2aZTl8+LAwGAzCaDSKkJAQERISIvbs2aNJlu+++06EhoYKo9EoevfuLf73f/9Xkxx1MZvNmvbi\nu3z5sjAajcJoNIqgoCDN9mEt6kiretGiNrSuAWvu51ru05mZmSIsLEz06dNHPPfcc03qxccLdYmI\nSJe07y5FRERUBzZQRESkS2ygiIhIl9hAERGRLrGBIiIiXWIDRUREusQGioiIdOn/AY+MuvHOMl6u\nAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x10e0c1a10>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"fig = plt.figure()\n",
"for k in range(2,8):\n",
" km = cluster.KMeans(n_clusters=k, init = 'k-means++', n_init = 5)\n",
" km.fit(dpro)\n",
" ax = fig.add_subplot(3, 2 , k-1)\n",
" plt.hist(km.labels_)\n",
" plt.title('k={}'.format(k))\n",
"\n",
"fig.tight_layout()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>We can see that the clusters are actually pretty well balanced for all values of $k$. Each choice of $k$ certainly has a clear maximum, but the differences aren't too extreme.<br><br>\n",
"The next thing we might try to do then is define the clusters based on the centroids. The centroid essentially describes the average student within each cluster. We can use that to better understand and then define the clusters. For comparison's sake, we'll also show this for clusters derived using $U$ and $U\\Sigma$. When doing this though, we have to remember to report the cluster means in the original $X$ space for us to be able to interpret it. To project the $U\\Sigma$ centroids back into the $X$ space, we just right multiply the centroid by $V^T$ from the SVD.<br><br>\n",
"Another thing we do here is we subtract the mean of $X$ from each centroid. This is because we're more interested in how each cluster differs from the average student profile.\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 148,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x117e35890>"
]
},
"execution_count": 148,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XdcFGf+B/DP0FTK0kSlmGAATRQFrHeoAUQstCTGHrGg\nd2piS6JG4yXiyxijnj9LvJhoVGI8URNPYzk9jUKMvUDsBQtGFBtKk84+vz845yCwuyjLFvfzfr32\n9dqdnZnn+8zufPfZZ2aekYQQAkRE9MIz03cARESkG0z4REQmggmfiMhEMOETEZkIJnwiIhPBhE9E\nZCKY8Im0JC0tDWZmZlAqlS9UWfTiYMKnWgkODoaTkxOKi4vrZP3nz59Hjx494OzsDEdHR7Rv3x67\ndu2qk7IMzZUrV9CvXz+4uLjAwcEBfn5+WLRokVaTfHBwMFatWqW19ZFhY8Kn55aWlobjx4+jUaNG\n2LZtW52UERUVhZ49e+LevXu4f/8+li5dCoVCodUyysrKtLo+bbh27Ro6deqEl19+GefOnUNWVhZ+\n+OEHnDp1Cnl5eVorR5KkWi3PfxhGRhA9p1mzZomoqCjx2WeficjISCGEEIWFhcLe3l6cO3dOnu/+\n/fuiQYMG4sGDB0IIIebNmydcXV2Fu7u7WLlypZAkSVy7dq3K+h88eCAkSRLZ2dkqY9i6davw8/MT\nCoVCeHl5id27dwshhLh9+7aIiooSTk5OwtvbW6xcuVJeZubMmeLtt98WQ4YMEQqFQqxatUpkZWWJ\n2NhYOa6//e1voqysTAghRGpqqnj99deFvb29aNiwoRgwYEC1sdy4cUNIkiRWrFgh3NzchKurq/j7\n3/8uhBAiIyNDWFtbi8zMTHn+U6dOCRcXF1FaWlplXe+88468TdWV9TTGl19+Wfz888+V6jhkyBAh\nhBAFBQXinXfeEc7OzsLBwUF06NBB3Lt3T3z88cfC3Nxc1K9fX9ja2orx48cLIYS4ePGi6N69u3By\nchItWrQQmzZtktc7bNgwMWbMGNG7d29hY2Mj9u3bpzJGMjxM+PTcvLy8xLp168SVK1eEpaWluH//\nvhBCiNjYWDFjxgx5vmXLlonevXsLIYTYtWuXaNKkibhw4YLIz88X77zzjjAzM6s24SuVSuHj4yMi\nIyPF1q1bxd27dyu9f+zYMWFvby8nutu3b4tLly4JIYTo2rWreO+990RRUZH47bffhIuLi9i/f78Q\nojwZWlpaip9++kkIUZ4Q33zzTTFmzBiRn58v7t+/Lzp27Ci++eYbIYQQAwcOFJ9//rkQQoiioiJx\n6NCharfH0yQ8ePBgkZ+fL86ePStcXFzk+MLDw8Xy5cvl+SdNmiQmTJhQ7bqaNGki4uPjVW77PyZ8\nT0/PSsk3Li5OxMTECCGE+Prrr0VUVJQoKCgQSqVSJCcni5ycHCGEEMHBwWLVqlXycnl5ecLDw0PE\nx8eLsrIykZKSIho2bCguXLgghChP+Pb29uLw4cNCiPIfeDIe7NKh53Lw4EHcvn0b0dHR8PHxQcuW\nLfHPf/4TADB48GBs2LBBnnf9+vUYPHgwAGDTpk2IjY3Fa6+9hgYNGmDWrFkQKoZzkiQJiYmJ8PT0\nxIcffgg3NzcEBQXh6tWrAIBVq1Zh5MiRCA0NBQC4ubmhRYsWuHXrFg4fPox58+bBysoKfn5+GDVq\nFNauXSuvOzAwENHR0QCA7Oxs7Nq1C4sWLUKDBg3g4uKCSZMmyXWwsrJCWloabt++DSsrKwQGBqrd\nNjNnzkSDBg3g6+uLESNGICEhAQAwdOhQrFu3DkB5N9KGDRsQExNT7ToyMzPh6uqqthx1RHljTo4/\nMzMTqampkCQJAQEBsLOzqzTvUzt27ECzZs0wbNgwmJmZwd/fH3369MEPP/wgz/Pmm2/iz3/+MwCg\nXr16zx0j6R4TPj2X7777Dj169JATR79+/fDdd98BKD8QmJ+fj+PHjyMtLQ2nT5/GW2+9BQDIyMhA\n06ZN5fV4eHioLcfd3R1ffvklrl69ips3b8LGxgZDhw4FAKSnp8PLy6vKMnfu3IGTkxNsbGzkaS+9\n9BJu375dbbk3b95ESUkJXF1d4ejoCEdHR4wZMwYPHjwAAMyfPx9CCHTs2BG+vr5Ys2aN2pgr1u+l\nl17CnTt3AABvvPEGLly4gLS0NOzduxf29vZo3759tetwdnaWl3seFfvmY2Ji0LNnTwwcOBDu7u74\n6KOPUFpaWu28N2/exLFjx+Tt4OjoiPXr1+PevXvyvBXrR8bFQt8BkPEpKCjApk2boFQq5VZoUVER\nsrKycObMGbRp0wb9+/dHQkICGjVqhKioKDn5urq64tatW/K6Kj7XxMPDA++++678b6Fp06Zya78i\nNzc3PHr0CHl5ebC1tQUA/P7775WSfMUk17RpU9SrVw+ZmZkwM6vaBmrcuDFWrFgBADh06BC6d++O\noKAgvPLKK9XG+fvvv6NFixbyc3d3dwBA/fr10a9fP6xbtw6XLl2Sf7iq0717d2zevBnDhw9Xt0lk\nNjY2ePLkifw6IyNDfm5hYYFPP/0Un376KW7evInw8HC0aNECsbGxVQ7avvTSSwgKCsKePXtqVC4Z\nF7bw6Zlt3boVFhYWuHjxIk6fPo3Tp0/j4sWL6Nq1q9xt8rRbp2J3DgD0798fa9aswaVLl5Cfn4/Z\ns2erLCcrKwszZ87EtWvXoFQq8fDhQ6xevVruThg5ciTWrFmD/fv3Q6lU4vbt27h8+TKaNm2KwMBA\nTJ8+HUVFRThz5gxWr16NIUOGVFuOq6srevTogQ8++AC5ublQKpW4du0aDhw4AAD44YcfkJ6eDgBw\ncHCAJEnV/jA89dlnn6GgoADnz59HfHw8BgwYIL83dOhQrFmzBtu2bVPZnQMAs2bNwuHDhzF16lS5\ndX316lXExMQgJyenyvz+/v7YsGEDSktLcfLkSWzevFlO5klJSTh79izKyspgZ2cHS0tLmJubAyj/\nMbt27Zq8nsjISFy5cgXr1q1DSUkJSkpKcOLECVy6dAkAVHa/kZHQ5wEEMk69evUSkydPrjJ906ZN\nwtXVVT6Q6O3tLZydnUVJSUml+ebOnSuaNGki3N3dxfLly4UkSSI9Pb3K+p48eSKGDRsmPD09ha2t\nrWjSpIkYPHiwuHPnjjzPli1bRJs2bYSdnZ3w9vYWe/bsEUIIkZ6eLiIjI4WTk5Pw8vKSD8AKUfmA\n5lPZ2dli7NixwsPDQ9jb24uAgACxceNGIYQQU6dOFe7u7sLW1lZ4eXlVOuOnohs3bggzMzOxcuVK\n4ebmJpo0aSIWLFhQZT5vb28RHBxc7Toqunz5sujXr59wdnYW9vb2ws/PTyxZskSUlZXJZT3d1tev\nXxedOnUStra2IiIiQkycOFGuY0JCgmjRooWwsbERjRs3FhMnTpSXO3LkiGjevLlwdHQUEydOlMuN\niIgQLi4uwtnZWYSGhorTp08LIYQYPny4+OSTTzTGToZJEkK/P9llZWVo3749PDw8sH37dn2GQnpw\n8eJFtG7dGsXFxWpbzS+S7t27Y/DgwYiNjdV3KGRi9L6HLVmyBC1btqz1BSBkPLZs2YKioiI8fvwY\nH330EaKjo00m2Z84cQLJycmVunmIdEWve1l6ejr+/e9/Y9SoUewbNCErVqxA48aN4e3tDUtLSyxf\nvlzfIenEsGHDEBYWhsWLF1c6g4hIV/R6ls7777+PBQsWVHsQil5cpjIWzh89PW2VSF/01sLfsWMH\nGjVqhICAALbuiYh0QV9Hi6dPny48PDyEp6enaNKkibC2tq5y5oSfn58AwAcffPDBxzM8/Pz8qs27\nBnFaZlJSUrUDRQHPH97MmTNrEZFhYV0Mz4tSDyFYF0NVm7qoyp0Gc2oEz9IhIqpbBjG0QlBQEIKC\ngvQdBhHRC81gWvjaFhwcrO8QtIZ1MTwvSj0A1sVQ1UVd9H6lrTqSJPEMHiKiZ6Qqd76wLXwiIqqM\nCZ+IyEQw4RMRmQgmfCIiE8GET0RkIpjwiYhMBBM+EZGJYMInIjIRTPhERCaCCZ+IyEQw4RMRmQgm\nfCIiE8GET0RkIpjwiYhMBBM+EZGJYMInIjIRTPhERCaCCZ+IyEQw4b8AFA4KSJJU7UPhoNB3eERk\nIHhP2xeAJElAnIo348BtSGRieE9bIiITp7eEX1hYiE6dOsHf3x8tW7bE9OnT9RUKEb0gFAon1d2b\nCid9h6d3FvoquH79+khMTIS1tTVKS0vRpUsXHDx4EF26dNFXSERk5HJzHwOovgszN1fSbTAGSK9d\nOtbW1gCA4uJilJWVwcmJv8BERHVFrwlfqVTC398fjRs3RkhICFq2bKnPcIiIXmh669IBADMzM/z2\n22/Izs5Gz549kZSUhODg4ErzxMXFyc+Dg4OrvE9EZOqSkpKQlJSkcT6DOS1z9uzZaNCgASZPnixP\n42mZNcPTMonKSZIEVX34gOnkE4M7LfPhw4fIysoCABQUFGDv3r0ICAjQVzhERC88vXXpZGRkYNiw\nYVAqlVAqlYiJiUFoaKi+wiEieuHpLeG3bt0aycnJ+iqeiMjk8EpbIiITwYRPRGQimPCJiEwEEz4R\nkYlgwiciMhFM+EREJoIJn4jIRDDhExGZCCZ8IiITwYRPRGQimPCJiEwEEz4RkYlgwiciMhFM+ERE\nJsKoE76jowKSJFX7cHRU6Ds8IiKDotd72tZWVlYuEhOrfy8kJFe3wRARGTijbuETEVHNMeETEZkI\nJnwiIhPBhE9EZCKY8ImITAQTPhGRiWDCJyIyEXpL+Ldu3UJISAhatWoFX19fLF26VF+hEBGZBI0J\n/8mTJ5g9ezb+8pe/AABSU1OxY8eOWhdsaWmJRYsW4fz58zh69Cj+8Y9/4OLFi7VeLxERVU9jwh8x\nYgSsrKxw+PBhAICbmxtmzJhR64KbNGkCf39/AICtrS1ee+013Llzp9brJSKi6mlM+NeuXcNHH30E\nKysrAICNjY3Wg0hLS0NKSgo6deqk9XUTEVE5jWPp1KtXDwUFBfLra9euoV69eloLIC8vD3379sWS\nJUtga2tb5f24uDj5eXBwMIKDg7VWNhGRoVIonJCb+7ja9+zsHJGT80h+nZSUhKSkJI3rlIQQQt0M\ne/bswZw5c3DhwgWEhYXh0KFDiI+PR0hIyLNFX42SkhJERkaid+/emDRpUtXgJAnqwpMkSc3gaVC7\n7ItEkiQgTsWbcaazHahuKBwUyM2ufjBCO3s75GTl6Dgi1SRJAqDq+64+nxia2tRFVe5U28JXKpV4\n/PgxNm/ejKNHjwIAlixZAhcXlxoHrYoQAiNHjkTLli2rTfZEZBhys3NVNihy4zgqrTFR24dvZmaG\n+fPno2HDhoiMjERkZKRWkj0AHDp0COvWrUNiYiICAgIQEBCA3bt3a2XdRERUlcY+/LCwMPz973/H\ngAEDKh2wdXJyqlXBXbp0gVKprNU6iIio5jQm/A0bNkCSJPzjH/+Qp0mShOvXr9dpYHXNmPoliYi0\nQWPCT0tL00EYusd+SSIyNRoTfnFxMZYvX44DBw5AkiQEBQVhzJgxsLS01EV8RESkJRoT/tixY1Fa\nWor33nsPQgh8//33GDt2LL799ltdxEdERFqiMeGfOHECZ86ckV+HhoaiTZs2dRoUERFpn8ahFSws\nLHD16lX59bVr12BhofF3goiIDIzGzL1gwQJ069YNzZo1A1B+EHfNmjV1HhgREWmXxoQfGhqKK1eu\n4PLly5AkCc2bN0f9+vV1ERsREWmRxi6dZcuWoaCgAH5+fmjTpg0KCgrw1Vdf6SI2IiLSIo0Jf+XK\nlXB0dJRfOzo6YsWKFXUaFBERaZ/GhK9UKisNgVBWVoaSkpI6DYqISOvMykcJUPVQOCj0HWGd09iH\n37NnTwwcOBCjR4+GEALffPMNevXqpYvYiIi0RwnVw4jDNK6w15jw582bhxUrVmD58uUAygdTGzVq\nVJ0HRkRE2qUx4Zubm2Ps2LEYOXIkzp07Bw8PD5ibm+siNiIi0iKVffijR4/GuXPnAADZ2dnw8/PD\nsGHD4O/vj/Xr1+ssQCIi0g6VCf/XX3+Fr68vAGDNmjVo0aIFzp49i+TkZMyfP19nARIRkXaoTPgV\nb1S+Z88evPHGGwCAJk2a1H1URESkdSoTvr29PbZv347k5GQcPnxYPjOnpKQEhYWFOguQiIi0Q+VB\n22+++QYTJkzA3bt3sXjxYri6ugIA9u/fj4iICJ0FSERE2qEy4bdo0QL/+c9/qkzv2bMnevbsWadB\nERGR9mm80paIiF4MTPhERCaCCZ+IyESo7MNfuHCh/FySJAgh5OcA8MEHH9S68NjYWOzcuRONGjXC\n2bNna70+IiJSTWULPzc3F3l5eTh16hSWL1+OO3fu4Pbt2/j666+RnJyslcJHjBiB3bt3a2VdRESk\nnsoWflxcHACga9euSE5Ohp2dHQBg1qxZCA8P10rhXbt2RVpamlbWRURE6mnsw79//z4sLS3l15aW\nlrh//36dBkVERNqncbTMoUOHomPHjujTpw+EENi6dSuGDRumi9gA/O+fBgAEBwcjODhYZ2UTERmD\npKQkJCUlaZxPY8KfMWMGevXqhYMHDwIA4uPjERAQUOsAa6piwifSJUdHBbKyqr8phoODHR4/ztFx\nRETV+2NjeNasWdXOpzHhA0B+fj7s7OwQGxuLBw8e4MaNG2jWrJlWAiUyVFlZuUhMrP69kJAX/+5I\n9OLR2IcfFxeH+fPn44svvgAAFBcXY8iQIVopfNCgQQgMDMSVK1fQtGlTrFmzRivrJSKiqjS28Lds\n2YKUlBS0a9cOAODu7o7cXO20bhISErSyHiIik2L2v2uinoXGhF+vXj2Ymf3vj8CTJ0+euRAiItIi\nDTdkV/Wexi6dfv36YfTo0cjKysKKFSsQGhrKm5gTERkhjS38KVOmYM+ePbCzs8OVK1cwe/ZshIWF\n6SI2IiLSIo0J/6OPPsK8efPQo0ePKtOIiMh4aOzS2bNnT5Vp//73v+skGCIiqjsqW/jLly/HV199\nhWvXrqF169by9NzcXHTu3FknwRERVaTuYjjSTGXCHzx4MHr37o1p06Zh3rx58vDICoUCTk5OOguQ\niOgpdRfDAUBIiO5iMUYqu3Ts7e3h6emJzz77DI0bN4anpydu3LiBdevWISsrS5cxEhGRFmjsw+/b\nty8sLCxw9epVjB49Grdu3cLgwYN1ERsREWmRxoQvSRIsLCzwr3/9C+PHj8eCBQuQkZGhi9iIiEiL\nNCZ8KysrrF+/HmvXrkVkZCQAoKSkpM4DIyIi7dKY8FevXo0jR45gxowZaNasGa5fv661wdOIiEh3\nNF541apVK3z55Zfy61deeQXTpk2r06CIiEj7NCb86sa9lyQJ169fr5OAiIiobmhM+CdOnJCfFxYW\n4scff0RmZmadBkWGy0mhwGM1w2M72tnhUQ7vBEVkiDT24Tds2FB+eHh4YNKkSdi5c6cuYiMD9Dg3\nFwJQ+VD3Y0BE+qWxhX/q1Cl5oH2lUomTJ0+irKyszgMjIiLt0pjwP/zwQznhW1hYwNPTE5s2barz\nwIiISLs0JvykpCQdhEFERHVNZcL//vvvERMTg4ULF1a6d6IQApIkwdnZGdHR0XB0dNRJoEREVDsq\nD9rm5+cDKB8OueIjLy8Pubm5OHnyJHr37q2zQImIqHZUtvBHjx4NAIiLi1O58CeffKL1gIiIqG6o\nTPjjx4+Xn0uSJI+H/7R7Z+nSpZg9e3Ydh0dERNqiskunXbt2aNeuHYqKipCcnIzmzZvDx8cHKSkp\nKC4u1krhu3fvxquvvgofHx/eI5eIqI6pbOEPHz4cQPmtDg8ePAhLS0sAwNixY9GlS5daF1xWVoZx\n48bh559/hru7Ozp06IDo6Gi89tprtV43UU1oumqY6EWj8UrbrKws5FS4VD43N1crd7w6fvw4vL29\n4enpCUtLSwwcOBA//fRTrddLVFOarhometFoPA9/2rRpaNu2LUJCQiCEwC+//KL2QG5N3b59G02b\nNpVfe3h44NixY7VeLxERVU9jwh8xYgR69eqFY8eOQZIkzJs3D87OzrUuuOK5/c87n4W5upsWW6ov\nwwxAXPVv2dnb1Si2Z6Gu+8DCHChVO1qFJQA1N53RYV0c7ewgqekGsTDX9NmqqYsZAKXqJe3s7ZCT\npb2B2WpSl1p9v3RYF3Xfr/pWVihUc9zNysoGxcVPVK/cwPYV9TcqtwSg4nNRUw9A+3XR1GVYs/3+\n+eqissyazOTq6oro6Gjs378fM2bMwI4dO3Dv3r1nL60Cd3d33Lp1S35969YteHh4VJlP3V9rqQzy\n2UNV3pMk9UsrJZXL1oWn3QfVkcqAxETVy4aElMBQ6qJpJExJkp6/LkpJ7Zc4N067/e21GdWzJt8v\nXdZF7feruFjt98PY9hVdxlIb6uoB1Gy/f966qmqMaOzDP3LkCCZMmABPT0+8+eab6Nq1Ky5evPhc\nQVTUvn17pKamIi0tDcXFxdi4cSOio6NrvV4iIqqeyhb+9OnTsXnzZrzyyivo378/4uLi0K5dO/ns\nnVoXbGGBZcuWoWfPnigrK8PIkSN5hg4RmQxzDd1TDg7a7y5TmfC//fZbtGvXDmPHjkXv3r1hZWWl\n9cJ79+7N4RmIyCSV6aF7SmWXTkZGBiZOnIh//etf8PLyQkxMDAoKClBSoubgIRERGSyVLXwLCwu5\nBV5YWIgdO3YgPz8fHh4eCA0Nxfr163UZJxkJBwc7hITwYiYiQ1Sjs3Tq16+Pvn37om/fvsjJycHW\nrVvrOi4yUo8faz6Lh4j0Q+NZOn+kUCgwdOjQuoiFiIjqUI1a+EQ6oeMLY14k6i4ic7TjdqNyTPhk\nOJTGc1GNoanNRWRkOmqU8A8dOoS0tDSUlpYCKO+HZbfOs9F0GT8RUV3TmPCHDBmC69evw9/fH+bm\n5vJ0Jvxno64FxgOZxsfOzhG5ufzcyLhoTPinTp3ChQsXmJT0ygIqB1FCefIh3crJeaT2fe4vZIg0\nnqXj6+uLjIwMXcRCKpVCCKHyoSn5EBEBNWjhP3jwAC1btkTHjh1Rr149AOWtl23bttV5cEREpD0a\nE742bnZC9JS6vm92TRHVLY0JPzg4WAdhkKlg9xOR/tRoPPwOHTrA1tYWlpaWMDMzg0Kh0EVsRESk\nRRoT/rhx47B+/Xr4+PigsLAQq1atwrvvvquL2IiISItqNJaOj48PysrKYG5ujhEjRmD37t11HRcR\nEWmZxj58GxsbFBUVwc/PD1OnTkWTJk14+TsRkRHS2MJfu3YtlEolli1bBmtra6Snp2Pz5s26iI2I\niLRIYwvf09MT+fn5uHv3Lk/RJCIyYhpb+Nu2bUNAQAB69uwJAEhJSUF0dHSdB0ZERNqlMeHHxcXh\n2LFjcHQsvygmICAA169fr/PAiIhIuzQmfEtLSzg4OFReyOyZb5RFRER6pjFzt2rVCv/85z9RWlqK\n1NRUjB8/HoGBgbqIjYiItEhjwv/yyy9x/vx51KtXD4MGDYJCocDixYtrVegPP/yAVq1awdzcHMnJ\nybVaFxER1Ywk9HBS/aVLl2BmZobRo0dj4cKFaNu2bbXzSZIEdcFJUH1LPIXCCbm5j1Uua2fnaDDj\nukiShMRE1e+HhPDWf8ZG4aBAbrbqO5zZ2dshJ8swbktYPna/+j1Nl98/dfu9un3e0NQmf2ml7GrW\nrfK0zKioKJUL1XZ45FdfffW5l60pQ0nmZJoMJZkTVaQy4R89ehQeHh4YNGgQOnXqBOB/v0a8mw8R\nkfFRmfAzMjKwd+9eJCQkICEhARERERg0aBBatWpVoxWHhYXh7t27VaZ//vnniIqKqnGAcRWeB//3\nQURE/5OUlISkpCSN89WoD7+oqAgJCQmYPHky4uLiMG7cOG3EiJCQkDrrwzcm7MMnfWIfft0wqj58\nACgsLMTOnTuxYcMGpKWlYeLEiXjrrbe0GpixfHhERMZOZcKPiYnB+fPnER4ejk8//RStW7fWWqFb\ntmzBhAkT8PDhQ0RERCAgIAC7du3S2vqJiKgqlV06ZmZmsLGxqX4hSUJOTt2fhcAunXLs0qG6ZGin\nMLNLR0tlP0uXjlKprJNAiMiw8BRm08FBcYiITAQTPhFRHXC0s4MEqHw42tnpPCaNN0AhIqJn90gH\nxzmfFVv4REQmggmfiMhEMOETEZkIJnwiIhPBhE9EZCIM/iwddQMx6+O0JiIiY2XwCd9YLqMmIjJ0\n7NIhIjIRTPhERCaCCZ+IyEQw4RMRmQgmfCIiE8GET0RkIpjwiYhMhMpbHBoCVbfpetE4OiqQlZWr\n8n0HBzs8fmx4Q60S1YUX5RaH+qQqdzLhE5FBYcKvPVW5k106REQmggmfiMhEMOETEZkIvST8KVOm\n4LXXXoOfnx/69OmD7OxsfYRBRGRS9JLwe/TogfPnz+P06dNo3rw55s6dq48wiIhMil4SflhYGMzM\nyovu1KkT0tPT9REGEZFJ0Xsf/urVqxEeHq7vMIiIXnh1dgOUsLAw3L17t8r0zz//HFFRUQCAOXPm\nwMrKCoMHD1a5nri4OPl5cHAwgoODtR0qEZFRS0pKQlJSksb59HbhVXx8PFauXIl9+/ahfv361c7D\nC6+ITA8vvKo9VblTL7c43L17NxYsWIBffvlFZbInIiLt0ksL38fHB8XFxXBycgIA/PnPf8ZXX31V\nNTi28IlMDlv4tcexdIjIKDDh1x7H0iEiMnFM+EREJoIJn4jIRDDhExGZCCZ8IiITwYRPRGQi9HLh\n1fNycnLC48eP9R0GEdUxSd17krp3TYOjoyMePXr0zMsZ1Xn4PC+fiEhzLuR5+EREJo4Jn4jIRBh1\nwndSKCBJUq0fTgqF2nLOnDmD8PBwhISEoHPnzli0aBEAwNvb+5ljXrp06XPV9andu3cjMDAQgYGB\n2LNnT63W9UeOjtrZno6OxrM9Y2Nj4ebmhr/85S+1Ws8fKRROWtmWCoWT2nIMZVump6cjODgYr7/+\nOrp06YJTp04997r+SOGgne+lwsF4vpdhYWEICQlBp06dsHXr1lqtqyKj7sNXN+bGM5UD1eNzZGdn\n4/XXX8dx+MmVAAAN0ElEQVTWrVvRrFkzAMDevXsRFhYGHx8fpKamPlNZz7qMUqmU7w5WVlaGtm3b\n4tdff4UQAkFBQUhJSdHaQSxJkpCYWPv1hIQYx/YEgDt37iA1NRXr1q3DypUrn6lsdco/E+18O41h\nW+bk5KC4uBgNGzbExYsXMXr0aBw4cOCZyldFkiQgTgsrijOe72VpaSksLCyQm5uLdu3a4cqVK5Xm\nZx9+Hdm5cyeio6PlLwFQ/utbUXx8PObMmQOgvKUTEhICAFi0aBH+9Kc/oVu3bli6dCkSEhJw+/Zt\nhISEYO7cuSgtLcWoUaPQrVs3dO3aFSdOnAAADB8+HGPHjkVUVBR+/fVXuZyrV6+iWbNmUCgUsLe3\nh6en5zN/EfXNkLYnALi5uRntiQCGtC0VCgUaNmwIALCysoKFhVGdAGhQ2xKAvP3y8vLg4OCgtXoa\n16eiB+np6fDw8FA7j6oW9vr165GUlAQbGxsIISBJEj799FMk/rcZ/fXXX8PHxwfffvst7t27h7ff\nfhsHDx6EJEnw9PTE8uXLK60vMzMTjo6O8msHB4fnOjVLnwxpexo7Q9yWZWVlmDhxIv72t7/VrnI6\nZmjbUqlUIiQkBOfPn8eKFStqX8H/YsLXoGnTpjh37pzaeSp+ESq2FhcvXozx48ejpKQEY8aMQefO\nnSstd/bsWRw5cgS7d+8GUP63+KnAwMAq5Tg7OyMrK0t+nZ2dDWdn52erkJ4Z0vasrjxjYojbcvTo\n0QgPD0e3bt2eqS76Zmjb0szMDL/88gsePXqErl27Ijw8XCs3i2KXjgYRERHYvn07rl+/Lk/7+eef\nK83j5OSE9PR0AKh0sCogIACrV6/G3LlzMXHiRADlf9Wefll8fX0xdOhQJCYmIjExsdKyFfvznvL2\n9saNGzeQm5uLnJwc3Lhx47kOKOmTIW3Pp4y1S8fQtuXkyZPh5uaGd999VzsV1CFD2palpaUoKysD\nAFhbW6OsrExr31G28DVQKBRYt24d3nvvPRQWFqK4uBj9+/dH9+7d5V/8sLAwLFq0CD169IC/v788\nPSYmBpmZmSgsLMS4ceMAAH379kVERATCw8MxZswYjB8/Xm4NdejQAfPmzQNQfavT3Nwcc+fORc+e\nPQEAX3zxhdG1Tg1pewLAJ598gl27duHu3bvo0aMHtm7dCmtr67reDFphSNvy5MmTWLp0Kbp06YKQ\nkBC4uLhg06ZNutgMWmFI2/L+/fsYNGgQzMzMUFhYiI8//hgNGjTQSj2N+iwdJ4UCj3Nza12Oo50d\nHlX4m2WqHB0VyMqq/fZ0cLDD48emvT0VCifk5tZ+GBA7O0fk5BjXcRptUzgokJtd+++lnb0dcrJe\njO/l856lY9QJn4jIFPG0TCIiUosJn4jIRDDhExGZCL2cpfPJJ59g27ZtkCQJzs7OiI+PR9OmTTUu\n5+joaHRnpRARaVvFCzCfidCDnJwc+fnSpUvFyJEjq52vNuElJiY+97KGhnUxPC9KPYRgXQxVbeqi\nKnfqpUvHzs5Ofp6XlyePwaFNSUlJWl+nvrAuhudFqQfAuhiquqiL3i68mjFjBr7//ntYW1vj6NGj\n+gqDiMhk1FkLPywsDK1bt67y2L59OwBgzpw5+P333zF8+HC8//77dRUGERE99dydRFpy8+ZN0apV\nq2rf8/PzEygfVJwPPvjgg48aPvz8/KrNqXrp0klNTYWPjw8A4KeffkJAQEC18/3222+6DIuI6IWm\nl6EV+vbti8uXL8Pc3BxeXl5Yvnw5GjVqpOswiIhMikGPpUNERNpj1FfaduvWrcqNvBcvXoxXXnkF\n8+fP11NUtXP37l0MHDgQ3t7eaN++PSIiIpCamooJEyagdevWaNOmDTp27Ii0tDR9hwqgfDzvmJgY\n+XVpaSlcXFwQFRWldrnTp09j165d8uu4uDgsXLiwzuKsiTlz5sDX1xd+fn4ICAjA8ePHsWTJEhQU\nFGhcdvHixTWaT580fVbx8fEYP368vsLTyNzcHAEBAfD390e7du1w5MgRfYf0XFTt47pg1OPhDxo0\nCBs2bECPHj3kaRs3bsTatWvRpUsXPUb2fIQQeOuttzBixAhs2LABAHDmzBls2LABGRkZOHv2LIDy\nG28bypjtNjY2OH/+PAoLC1G/fn3s3bsXHh4eGq+ITklJwalTp9C7d28A+r/r1JEjR7Bz506kpKTA\n0tISjx49QmFhIRYvXowhQ4ZoHI98yZIliImJ0dq45XVB02el789AE2tra6SkpAAA9uzZg+nTpxvd\nefeq9vF79+7JxzXrklG38N9++23s3LkTpaWlAIC0tDTcuXMHV69elVsq/v7+CAgIQEBAAKytravc\nLNiQJCYmwsrKCn/961/laW3atIGtrS1cXV3laW5ublq9sXFthYeHY+fOnQCAhIQEDBo0SB6a9fjx\n4wgMDETbtm3RuXNnXLlyBcXFxfj000+xceNGBAQEyDfKuHDhAkJCQuDl5YUvv/xSp3W4e/cuGjZs\nCEtLSwDldzf68ccfcefOHYSEhCA0NBQAMHbsWHTo0AG+vr6Ii4sDACxdurTSfEqlEsOHD5f/kS1e\nvFindVFH3WdlTL272dnZcHJyAlB+gVLFf5Tjxo3Dd999BwCYNm0aWrVqBT8/P0yZMkUvsVakah/v\n0qULpkyZIn9nnu4TSUlJCAoKwptvvgkvLy9MmzYN33//PTp27Ig2bdpUukNXjdTBmZY6FRkZKX76\n6SchhBBz584VU6ZMEfHx8WLcuHGV5tu2bZt4/fXXRWlpqT7CrJElS5aI999/v8r09PR04enpKfz9\n/cWHH34oUlJS9BBd9WxtbcWZM2dE3759RWFhofD39xdJSUkiMjJSCFE+jMbTbb53717x9ttvCyGE\niI+PF+PHj5fXM3PmTBEYGCiKi4vFw4cPhbOzs04/q7y8POHv7y+aN28u3n33XfHLL78IIYTw9PQU\nmZmZ8nyPHj0SQghRWloqgoODxdmzZ6vMd/LkSREWFiYvk5WVpatqqKXps1qzZk2V/caQmJubC39/\nf/Hqq68Ke3t7kZycLIQoH4LgaR2EEGLcuHHiu+++E5mZmaJFixby9OzsbJ3H/Eeq9vEff/xRhIWF\nCaVSKe7duydeeuklkZGRIRITE4WDg4O4e/euKCoqEm5ubmLmzJnyuiZNmvRM5Rt1Cx/4X7cOUN6d\nM2jQoCrzpKamYurUqdi0aRPMzc11HWKNqfpL7e7ujsuXL2Pu3LkwMzNDaGgo9u/fr+PoVGvdujXS\n0tKQkJCAiIiISu9lZWWhb9++aN26NT744ANcuHABQHlrUvzh5jaRkZGwtLSEs7MzGjVqhHv37ums\nDjY2Njh16hRWrFgBFxcXDBgwAPHx8XKsT23cuBHt2rVD27Ztcf78ebk+FXl5eeH69euYMGEC/vOf\n/0ChUOiqGhqp+6wMXYMGDZCSkoKLFy9i9+7dlY5HVMfe3h7169fHyJEjsWXLFoPoblO1jx86dAiD\nBw+GJElo1KgRgoKCcOLECUiShA4dOqBx48awsrKCt7e3fItTX1/fZz6WZ/QJPzo6Gvv27UNKSgry\n8/MREBBQaQfNy8vDgAED8O2336Jx48Z6jFSzVq1aVbrBcUVWVlbo1asX5s+fj48//hhbt27VcXTq\nRUdHY/LkyZW6CIDykVFDQ0Nx9uxZbN++Xe2BTSsrK/m5ubm53FWnK2ZmZggKCkJcXByWLVuGzZs3\nA/jfTnrjxg0sXLgQ+/fvx+nTpxEREYHCwsIq63FwcMCZM2cQHByMr7/+GqNGjdJpPTRR9VkZkz/9\n6U94+PAhHj58CAsLCyiVSvm9wsJCCCFgbm6O48ePo2/fvtixYwd69eqlx4jLqdvH//hZPP3e1atX\nT55mZmYmvzYzM3vmfcToE76trS1CQkIwYsQIDB48uMr7sbGxGDFiBDp37qyH6J5Nt27dUFRUhJUr\nV8rTzpw5gwMHDuDOnTsAAKVSidOnT8PT01NPUVYvNjYWcXFxaNWqVaXpOTk5cHNzAwCsWbNGnq5Q\nKJCrhfsRa8uVK1cqnSmRkpICT09P2NnZIee/9zvOycmBjY0NFAoF7t27V+kso4rzZWZmorS0FH36\n9MHs2bORnJys28pooOqzMiaXLl1CWVkZnJ2d8fLLL+PChQsoLi5GVlYW9u3bB0mS8OTJE2RlZaF3\n7974v//7P5w+fVrfYavcxx0cHLBx40YolUo8ePAABw4cQMeOHbX+g2zUZ+k8NWjQIPTp00c+0CFJ\nEiRJwu+//47NmzcjNTUVq1evBgCsWrUKbdu21We4am3ZsgWTJk3CvHnzUL9+fXh6eqJXr1744IMP\nUFRUBADo1KkTxo0bp+dIyz1thbi7u8sxPd3+ADB16lQMGzYMn332GSIiIuTpISEh+OKLLxAQEIDp\n06dXWpc+5OXlYfz48cjKyoKFhQV8fHywYsUKrF+/Hr169YK7uzv27duHgIAAvPrqq2jatGmlM8H+\n+te/yvMtWrQII0aMkFudX3zxhb6qVYmmz0qSJMTHx8v/HiVJwtGjR+UfbH0rKCiQr8oXQmDt2rWQ\nJAlNmzZF//794evri2bNmsn7d25uLt544w25xb9o0SJ9hi/74z7erFkzLFq0CHl5efDz84MkSViw\nYAEaNWqEixcvqtwvKn52NcULr4iITITRd+kQEVHNMOETEZkIJnwiIhPBhE9EZCKY8ImITAQTPhGR\niWDCJyIyEUz4REQm4v8BU4CzJtA65r0AAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1172a5b90>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVNX/P/DXZROBGbZc2AwTs9yANO3jBoi4gJIlLpCK\n2ye13OpTlvUxx2+LYfVzabHUBJcENdNc0jQDzdJMJTUlRZQUBReUTUGWOb8/+HCDYGZAZpjBeT0f\nj3k8Zu7ce+773DvznjNnzpwrCSEEiIjogWdh7ACIiKhhMOETEZkJJnwiIjPBhE9EZCaY8ImIzAQT\nPhGRmWDCpwdCeno6LCwsoFarH6h9NaTQ0FCsXbvW2GGQATHhN2KBgYFwcXFBcXGxQco/ffo0+vfv\nD1dXVzg7O6Nr167YtWuXQfZlas6dO4fhw4ejWbNmcHJygq+vLxYtWqTXJB8YGIgvv/xSb+UBgLe3\nN/bt21dlWVxcHHr37q1z2++++w5jxozRazwV3nvvPTzyyCNQKBTw8vLCqFGjAABTpkxBdHR0tfVP\nnDgBW1tb3L59GyqVCtbW1lAqlVAqlWjXrh2mT5+OrKwsg8T6IGPCb6TS09Nx5MgRNG/eHNu2bTPI\nPoYMGYIBAwbg2rVruH79OpYuXQqlUqnXfZSVlem1PH1IS0tD9+7d8fDDD+OPP/5ATk4ONm3ahGPH\njqGgoEBv+5EkqV7b1/ThI0lSvcvVt9WrV2PdunXYt28f8vPzcfToUfTr1w8AMG7cOHzzzTe4e/du\nlW3Wrl2LIUOGwNnZGQAQGRmJvLw83L59G1u2bEFWVha6dOnCpF9Xghql+fPniyFDhoh33nlHDB48\nWAghRFFRkXB0dBR//PGHvN7169dF06ZNxY0bN4QQQsTExAg3Nzfh4eEhVqxYISRJEmlpadXKv3Hj\nhpAkSeTm5mqMYevWrcLX11colUrRpk0bsXv3biGEEFeuXBFDhgwRLi4uwsfHR6xYsULeZt68eWLY\nsGFi9OjRQqlUii+//FLk5OSICRMmyHH997//FWVlZUIIIVJTU0WfPn2Eo6OjeOihh8TIkSNrjOXi\nxYtCkiSxfPly4e7uLtzc3MSHH34ohBAiMzNT2NnZiezsbHn9Y8eOiWbNmonS0tJqZT333HPyMdW2\nr4oYH374YfHDDz9UqePo0aOFEEIUFhaK5557Tri6ugonJyfx5JNPimvXrok33nhDWFpaCltbW+Hg\n4CCmT58uhBAiJSVF9OvXT7i4uIh27dqJjRs3yuVGR0eLKVOmiEGDBgl7e3uxb9++arF5e3tXWx4b\nGyt69eqlMZ7r168LIYQICAgQK1eulLfp2bOneOWVV4Szs7No3bq12LVrl1zmhQsXRO/evYVCoRD9\n+vUTL7zwglznf5o2bZqYNWuWxuPZrl07sWbNGvlxaWmpcHd3F9u2bat2PCuUlZUJX19f8corr2gs\nl6pjC7+RWrNmDUaOHIkRI0bg+++/x40bN9CkSRMMGzYM8fHx8nobN25EYGAgHnroIezevRuLFi3C\nvn37kJqaiqSkJI2tQVdXV/j4+OC5557Dt99+i2vXrlV5/siRI4iOjsZHH32E3NxcHDhwAN7e3gCA\nUaNGoVWrVsjMzMTXX3+NN954A4mJifK227Ztw/Dhw5Gbm4uoqCiMGzcONjY2SEtLQ3JyMvbs2YOV\nK1cCAObOnYuBAwciJycHV65cwYwZM7Qel6SkJJw/fx579uxBTEwM9u3bh5YtWyIwMBAbN26U11u7\ndi0iIyNhaWlZrYx9+/YhIiJC+wmo5J+t6sqPV69ejby8PGRkZODWrVv44osv0LRpU7z77rvo3bs3\nPv30U+Tn52Pp0qW4c+cOQkJCMHr0aNy4cQMJCQl44YUXkJKSIpcdHx+PuXPnoqCgAD179qx1jBVq\nisfW1rbGehw5cgSPPfYYsrOzMXv2bEycOFF+LioqCk899RRu3boFlUqFdevWaXwtPfXUU1izZg0+\n/PBDHD16tNq3urFjx2LNmjXy4x9++AElJSUIDQ3VWA8LCws8/fTT+Omnn+p8DMwZE34jdPDgQVy5\ncgXh4eFo27Yt2rdvj6+++gpA+RsxISFBXnf9+vWIiooCUJ78J0yYgMcffxxNmzbF/PnzITRMpSRJ\nEhITE+Ht7Y3//Oc/cHd3R0BAAM6fPw8A+PLLLzFx4kQEBwcDANzd3dGuXTtcvnwZv/zyC2JiYmBj\nYwNfX19MmjSpyhu6R48eCA8PBwDk5uZi165dWLRoEZo2bYpmzZph1qxZch1sbGyQnp6OK1euwMbG\nBj169NB6bObNm4emTZuiY8eOGD9+vPzhN3bsWKxbtw5AeTdSQkKCxv7q7OxsuLm5ad2PNkII+bja\n2NggOzsbqampkCQJ/v7+UCgUVdatsGPHDrRu3RrR0dGwsLCAn58fnn32WWzatEleZ+jQofjXv/4F\nAGjSpEmdY9MVT2UPP/wwJk6cCEmSMHbsWGRmZuL69eu4dOkSjh49iv/7v/+DlZUVevbsifDwcI2v\npeeeew4ff/wxvv/+ewQGBqJFixZYuHCh/Pzo0aOxf/9+XL16FUB5Y+a5556r8cO4Mjc3N9y6davO\nx8CcMeE3QqtXr0b//v3lN+rw4cOxevVqAOU/BN69exdHjhxBeno6Tpw4gWeeeQYAkJmZCS8vL7kc\nT09Prfvx8PDAxx9/jPPnz+Ovv/6Cvb09xo4dCwDIyMhAmzZtqm1z9epVuLi4wN7eXl7WqlUrXLly\npcb9/vXXXygpKYGbmxucnZ3h7OyMKVOm4MaNGwCAhQsXQgiBbt26oWPHjoiNjdUac+X6tWrVSk4i\nTz/9NM6cOYP09HTs3bsXjo6O6Nq1a41luLq6ytvdj8ot3TFjxmDAgAEYNWoUPDw88Nprr6G0tLTG\ndf/66y/8+uuv8nFwdnbG+vXr5W9XkiRVqV9NrKysUFJSUmVZSUkJrK2taxVPZS1btpTv29nZAQAK\nCgrkc1zxzQCAzriioqKwd+9e5Obm4vPPP8fcuXOxZ88eAOXnqU+fPli7di0KCgrw7bffyq8zba5c\nuQJXV1ed69HfrIwdANVNYWEhNm7cCLVaLbdC7927h5ycHJw8eRKdO3fGiBEjEB8fj+bNm2PIkCFy\n8nVzc8Ply5flsirf18XT0xMvvPCC/G3By8tLbu1X5u7ujlu3bqGgoAAODg4AgEuXLlVJ8pWTnJeX\nF5o0aYLs7GxYWFRvf7Ro0QLLly8HAPz888/o168fAgIC8Mgjj9QY56VLl9CuXTv5voeHBwDA1tYW\nw4cPx7p16/Dnn39qTSj9+vXD5s2bMW7cOG2HRGZvb487d+7IjzMzM+X7VlZWeOutt/DWW2/hr7/+\nQmhoKNq1a4cJEyZU6wJp1aoVAgIC5ER4P1q1aoWLFy9WWXbx4kW5u01bPLVV0bIuLCxE06ZNAZQf\n69r8WGxpaYmIiAjExMTIo8AAIDo6GjExMWjZsiVat24Nf39/eZuaylWr1di+fbu8PdUOW/iNzNat\nW2FlZYWUlBScOHECJ06cQEpKCnr37i13m1R061TuzgGAESNGIDY2Fn/++Sfu3r2Lt99+W+N+cnJy\nMG/ePKSlpUGtVuPmzZtYtWqV3J0wceJExMbG4scff4RarcaVK1dw9uxZeHl5oUePHpgzZw7u3buH\nkydPYtWqVRg9enSN+3Fzc0P//v3x8ssvIz8/H2q1GmlpaThw4AAAYNOmTcjIyAAAODk5QZKkGj8Y\nKrzzzjsoLCzE6dOnERcXh5EjR8rPjR07FrGxsdi2bZvW4Yfz58/HL7/8gtmzZ8ut6/Pnz2PMmDHI\ny8urtr6fnx8SEhJQWlqKo0ePYvPmzXKSSkpKwqlTp1BWVgaFQgFra2u5q6JFixZIS0uTyxk8eDDO\nnTuHdevWoaSkBCUlJfjtt9/w559/AoDGLpPKRo4cicWLF+Ps2bMQQuDo0aOIjY2Vh0Fqi6e2Hn74\nYXTt2hUqlQolJSU4dOgQduzYoTHhr169Gt999518fnft2oXTp0+je/fu8jrDhg3DpUuXoFKpqn3Q\nVq53aWkpUlJSEBkZievXr+Pll1+uU+xmz2g/F9N9GThwYI0jEzZu3Cjc3NzkkSM+Pj7C1dVVlJSU\nVFlvwYIFomXLlsLDw0MsW7ZMSJIkMjIyqpV3584dER0dLby9vYWDg4No2bKliIqKElevXpXX2bJl\ni+jcubNQKBTCx8dH7NmzRwghREZGhhg8eLBwcXERbdq0EV988YW8jUqlEmPGjKmyr9zcXDF16lTh\n6ekpHB0dhb+/v9iwYYMQQojZs2cLDw8P4eDgINq0aVNlxE9lFy9eFBYWFmLFihXC3d1dtGzZUnzw\nwQfV1vPx8RGBgYE1llHZ2bNnxfDhw4Wrq6twdHQUvr6+YsmSJaKsrEzeV8WxvnDhgujevbtwcHAQ\nYWFhYubMmXId4+PjRbt27YS9vb1o0aKFmDlzprzdoUOHxKOPPiqcnZ3FzJkz5f2GhYWJZs2aCVdX\nVxEcHCxOnDghhBBi3LhxYu7cuVrjVqvV4v333xdt27YVSqVStG/fXqxatUp+Xls8gYGB4ssvvxRC\nCBEXFyd69+5dpWwLCwt5RFdaWpo8Sic4OFg8//zzYuLEiTXG9M0334iePXsKZ2dnoVQqRefOncXq\n1aurrTdu3DhhbW0tMjMzqyxXqVTC2tpaODg4CHt7e9G2bVvx4osvVnktUu1IQhj3AihlZWXo2rUr\nPD09sX37dmOGYnZSUlLQqVMnFBcXa201P0j69euHqKioOnVhkG4jR45E+/btMW/ePGOHQloY/V2+\nZMkStG/f3uT+LPKg2rJlC+7du4fbt2/jtddeQ3h4uNkk+99++w3Hjx+v0s1D9+fo0aNyd9+uXbuw\nbds2DB061NhhkQ5GfadnZGTgu+++w6RJk2rVP0n1t3z5crRo0QI+Pj6wtrbGsmXLjB1Sg4iOjkZI\nSAgWL15cZQQR3Z+srCwEBQVBoVDgpZdewueffw5fX19jh0U6GLVLZ/jw4XjjjTeQl5eHDz/8kF06\nREQGZLQW/o4dO9C8eXP4+/uzdU9E1BCM9WvxnDlzhKenp/D29hYtW7YUdnZ21UZv+Pr6CgC88cYb\nb7zV4ebr61tj3jWJYZlJSUk1TlYF3H948+bNq0dEpoV1MT0PSj2EYF1MVX3qoil3mszwDI7SISIy\nLJOYWiEgIAABAQHGDoOI6IFmMi18fQsMDDR2CHrDupieB6UeAOtiqgxRF6P/01YbSZI4goeIqI40\n5c4HtoVPRERVMeETEZkJJnwiIjPBhE9EZCaY8ImIzAQTPhGRmWDCJyIyE0z4RERmggmfiMhMMOET\nEZkJJnwiIjPBhE9EZCaY8InILCidlJAkSeNN6aQ0dogGx9kyGwGl0gX5+bc1Pq9QOCMv71YDRkTU\n+EiSBKi0rKDCA5NvNOVOk7gACmlXnuw1vxDz83m1MCLSjV06RERmggmfiMhMMOETEZkJJnwiIjPB\nhE9EZCaY8ImIzITREn5RURG6d+8OPz8/tG/fHnPmzDFWKEREZsFo4/BtbW2RmJgIOzs7lJaWolev\nXjh48CB69eplrJCIiB5oRu3SsbOzAwAUFxejrKwMLi4uxgyHiOiBZtSEr1ar4efnhxYtWiAoKAjt\n27c3ZjhERA80o06tYGFhgd9//x25ubkYMGAAkpKSEBgYWGUdlUol3w8MDKz2PBGRuUtKSkJSUpLO\n9Uxm8rS3334bTZs2xSuvvCIv4+Rp5SRJgra5dAAeJyJdOHmaEbt0bt68iZycHABAYWEh9u7dC39/\nf2OFQ0T0wDNal05mZiaio6OhVquhVqsxZswYBAcHGyscIqIHntESfqdOnXD8+HFj7Z6IyOzwn7ZE\nRGaCCZ+IyEww4RMRmQkmfCIiM8GET0RkJpjwiYjMBBM+EZGZYMInImpklE5KSJKk8aaJUSdPIyKi\nusvPzdc5L1BN2MInIjITTPhERGaCCZ+IyEww4RMRmQkmfCIiM8GET0RkJpjwiYjMBBM+EZGZYMIn\nIjITTPhERGaCCZ+IyEww4RMRmQkmfCIiM8GET0RkJoyW8C9fvoygoCB06NABHTt2xNKlS40VChGR\nWdCZ8O/cuYO3334b//73vwEAqamp2LFjR713bG1tjUWLFuH06dM4fPgwPv30U6SkpNS7XCIiqpnO\nhD9+/HjY2Njgl19+AQC4u7vjzTffrPeOW7ZsCT8/PwCAg4MDHn/8cVy9erXe5RIRUc10Jvy0tDS8\n9tprsLGxAQDY29vrPYj09HQkJyeje/fuei+biIjK6bzEYZMmTVBYWCg/TktLQ5MmTfQWQEFBASIi\nIrBkyRI4ODhUe16lUsn3AwMDERgYqLd9ExE9EC4CSNe9ms6Er1KpMHDgQGRkZCAqKgo///wz4uLi\n6h0fAJSUlGDYsGEYPXo0hg4dqnH/RESkRev/3Srsr3k1rQlfrVbj9u3b2Lx5Mw4fPgwAWLJkCZo1\na1bv+IQQmDhxItq3b49Zs2bVuzwiItJOEkIIbSt06dIFx44d0/uODx48iD59+qBz586QJAkAsGDB\nAgwcOPDv4CQJOsIzC+XHR9tx4HEi0kWSJEClZQUVGs376H7rorNLJyQkBB9++CFGjhxZ5QdbFxeX\n+wjzb7169YJara5XGUREVHs6E35CQgIkScKnn34qL5MkCRcuXDBoYEREpF86E356enoDhEFERIam\nM+EXFxdj2bJlOHDgACRJQkBAAKZMmQJra+uGiI+IqNaUShfk5982dhgmS2fCnzp1KkpLS/Hiiy9C\nCIG1a9di6tSpWLlyZUPER0RUa+XJXtMPr1JDhmKSdCb83377DSdPnpQfBwcHo3PnzgYNioiI9E/n\n1ApWVlY4f/68/DgtLQ1WVjo/J4iIyMTozNwffPAB+vbti9aty//GlZ6ejtjYWIMHRkRE+qUz4QcH\nB+PcuXM4e/YsJEnCo48+Cltb24aIjYiI9Ehnl84nn3yCwsJC+Pr6onPnzigsLMRnn33WELEREZEe\n6Uz4K1asgLOzs/zY2dkZy5cvN2hQRESkfzoTvlqtrjIFQllZGUpKSgwaFBER6Z/OPvwBAwZg1KhR\nmDx5MoQQ+OKLL6pMcEZERI2DzoQfExOD5cuXY9myZQDKJ1ObNGmSwQMjIiL90jk9coXi4mL88ccf\n8PT0RPPmzQ0dFwBOj1yB0yMT1Y729wqnR9bYhz958mT88ccfAIDc3Fz4+voiOjoafn5+WL9+fT3D\nJSKihqYx4f/000/o2LEjACA2Nhbt2rXDqVOncPz4cSxcuLDBAiQiIv3QmPArX6h8z549ePrppwEA\nLVu2NHxURESkdxoTvqOjI7Zv347jx4/jl19+kUfmlJSUoKioqMECJCIi/dA4SueLL77AjBkzkJWV\nhcWLF8PNzQ0A8OOPPyIsLKzBAiQiIv2o9SgdY+AonXIcpWMczs5K5OTk1/ick5MCt2/nNXBEpAtH\n6fyP6j4vYk5krnJy8pGYWPNzQUE1fxAQmTKdUysQEdGDgQmfiMhMaOzS+eijj+T7lfvSy/vIgJdf\nfrneO58wYQJ27tyJ5s2b49SpU/Uuj4iINNPYws/Pz0dBQQGOHTuGZcuW4erVq7hy5Qo+//xzHD9+\nXC87Hz9+PHbv3q2XsoiISDuNLXyVSgUA6N27N44fPw6FQgEAmD9/PkJDQ/Wy8969eyM9PV0vZRER\nkXY6+/CvX78Oa2tr+bG1tTWuX79u0KCIiEj/dA7LHDt2LLp164Znn30WQghs3boV0dHRDREbgL+/\naQBAYGAgAgMDG2zfRESNwkUA6bpX05nw33zzTQwcOBAHDx4EAMTFxcHf37+e0dVe5YRPREQ1aP2/\nW4X9Na9Wq2GZd+/ehUKhwMyZM+Hp6YmLFy/WP0AiImpQOhO+SqXCwoUL8f777wMovxDK6NGj9bLz\nyMhI9OjRA+fOnYOXlxdiY2P1Ui4REVWns0tny5YtSE5ORpcuXQAAHh4eyM/Xz9/K4+Pj9VJOTZRK\nF+Tn39b4vELhjLy8WwbbPxGRqdHZwm/SpAksLP5e7c6dOwYNSF/Kk73QeNP2YUDU2LgolZAkqcab\ni1Jp7PDIROhM+MOHD8fkyZORk5OD5cuXIzg4mBcxJzIxt/PzNTZvbuvpGzk1fjq7dF599VXs2bMH\nCoUC586dw9tvv42QkJCGiI2IyGzp6pa+HzoT/muvvYaYmBj079+/2jIiIjKMv7ulayLdV5k6u3T2\n7NlTbdl33313XzsjIiLj0djCX7ZsGT777DOkpaWhU6dO8vL8/Hz07NmzQYIjIiL90Zjwo6KiMGjQ\nILz++uuIiYmRp0dWKpVwcXFpsACJiEg/NHbpODo6wtvbG++88w5atGgBb29vXLx4EevWrUNOTk5D\nxkhERHqgsw8/IiICVlZWOH/+PCZPnozLly8jKiqqIWIjE6RtvDfHfBOZNp0JX5IkWFlZ4ZtvvsH0\n6dPxwQcfIDMzsyFiIxOkbbw3x3wTmTadCd/Gxgbr16/HmjVrMHjwYABASUmJwQMjIiL90pnwV61a\nhUOHDuHNN99E69atceHCBb1NnkZERA1HEhXDb0xQ5Yun38+2mv+0AAD3X3ZDM6W6SJKkIxI0muOq\niyRJSEys+bmgINOqp7bz8iCdE120v1ckQKVlY5VpHSdD1EXjsMzhw4dj06ZNVcbgVw7ExcUFs2bN\nwtChQ7UGTUREpkFjwl+yZAkAYPv27TU+n52djaioKCZ8IqJGQmPCd3d3BwB4e3vX+Ly3tze++uor\ngwRFRET6p3PyNAcHh//1JZVf7aqkpAQODg7Iy8tD165dDR4gERHph86EX1BQIN9Xq9XYtm0bDh8+\nbNCgiIhI/2p1EXN5ZQsLDB06FLt37zZUPEREZCA6W/ibN2+W76vVahw7dgxNmzY1aFBERKR/OhP+\n9u3b5T58KysreHt749tvvzV4YETUMHRdWUmhcEZe3q0GjIgMRWfCj4uLa4AwiMhYtF9ZCcjPv7+r\nK5Hp0Zjwp0+fLt+v/I/Xitb+0qVLDRwaNUaWln+/Rmri5KTA7dt5DRgREVXQmPC7dOkCAPjll19w\n5swZjBw5EkIIbNq0CR06dNDLznfv3o1Zs2ahrKwMkyZNwmuvvaaXcsl4ysqgcToCAAgK4myaRMai\nMeGPGzcOQPmlDg8ePAhra2sAwNSpU9GrV69677isrAzTpk3DDz/8AA8PDzz55JMIDw/H448/Xu+y\niYioOp3DMnNycpCX9/dX8Pz8fL1c8erIkSPw8fGBt7c3rK2tMWrUKP4YTERkQDp/tH399dfxxBNP\nICgoCEII7N+/HyqVqt47vnLlCry8vOTHnp6e+PXXX+tUhrOzEjk57CIgIqoNnQl//PjxGDhwIH79\n9VdIkoSYmBi4urrWe8faftir7Xp2djZatrRG+cSwGlhoLlvhqEBejn5/WHRRKjVeDcrKEigt07a1\n6dTFWaGApOWqVlaW5VMHa2at+ZxaAFBr3lLfddF2TgBdddFSD6DB66LtvFjp+CHdlF5fgAHfKxbQ\nOqWwwlFRuwBrqTavL0PVReM+a7OSm5sbwsPD8eOPP+LNN9/Ejh07cO3atbrvrRIPDw9cvnxZfnz5\n8mV4enpWW89Q81NLkub5pPNV+v/WUHFpwBpj0flDZ4nW49CQdbmVV783t9Y5vtXa5/jWd120nROg\n/LxoOu46r1HQwHXRdl60zesPmNbrCzDse6Uh1eb1Zai6aPqA1tmHf+jQIcyYMQPe3t4YOnQoevfu\njZSUlPsKorKuXbsiNTUV6enpKC4uxoYNGxAeHl7vcomIqGYaE/6cOXPw6KOPYv78+fDz88Pvv/+O\nZs2aYdy4cXBxcan3jq2srPDJJ59gwIABaN++PUaOHMkROkREBqSxS2flypXo0qULpk6dikGDBsHG\nRlt/+f0ZNGgQBg0apPdyiYioOo0JPzMzE3v37kVCQgKmTZuGwMBAFBYWoqSkRB6TT0QPPoWjQmNf\nvb5/6CTD0pjwrays5BZ4UVERduzYgbt378LT0xPBwcFYv359Q8ZJREai71E4ZDy1GqVja2uLiIgI\nREREIC8vD1u3bjV0XEREjZquIczGUKuEX5lSqcTYsWMNEQsR0QND1xDm2v4XSZ/qdMUrIiJqvJjw\niYjMRK26dH7++Wekp6ejtLQUQPlXEXbrEBE1LjoT/ujRo3HhwgX4+fnB0tJSXs6ET0TUuOhM+MeO\nHcOZM2eM8gMDUaPVwBN1EdWGzj78jh07IjMzsyFiIWpQzgoFJEDjzVlRj6SsLp94TdONY9s103Ze\nqH50tvBv3LiB9u3bo1u3bmjSpAmA8j78bdu2GTw4IkOq78yfZBi6Zv6k+6cz4evjYidERGR8OhN+\nYGBgA4RBRESGpjPhV8yHn5KSgnv37qGsrAwODg5VrnNL9WOp4ypRTk78gY+I6k9nwp82bRoSEhIw\nYsQIHD16FGvWrMHZs2cbIjazUablykpERPpSq3/atm3bFmVlZbC0tMT48eOxe/duQ8dFRER6prOF\nb29vj3v37sHX1xezZ89Gy5Yt2RolImqEdLbw16xZA7VajU8++QR2dnbIyMjA5s2bGyI2IiLSI50t\nfG9vb9y9exdZWVkcokn1plA4Iz+fY6mJjEFnC3/btm3w9/fHgAEDAADJyckIDw83eGD0YMrLu6Xx\n36dEZFi1+uPVr7/+iqD/jRv09/fHhQsXDB4YEemHk5MCQUGar7zEYb/mQ2fCt7a2hpOTU5VlFhac\nRp+osbh9m/+ZMUXG+CDWmfA7dOiAr776CqWlpUhNTcXSpUvRo0cPvQdCRGROjPFBrLOp/vHHH+P0\n6dNo0qQJIiMjoVQqsXjx4nrtdNOmTejQoQMsLS1x/PjxepVFRES1U6tx+O+99x7ee+89ve20U6dO\n2LJlCyZPnqy3MomISDuNCX/IkCGQJKnG0RP1nR75scceu+9tiYjo/mhM+IcPH4anpyciIyPRvXt3\nAH/P98I5qevOWaGAlF/zDzT1utAGEVEtaUz4mZmZ2Lt3L+Lj4xEfH4+wsDBERkaiQ4cOtSo4JCQE\nWVlZ1ZYOFQ4NAAARqElEQVS/9957GDJkSK0DrPxnr8DAwEY7XTMvtkFEhpKUlISkpCSd62lM+FZW\nVhg0aBAGDRqEe/fuIT4+HgEBAVCpVJg2bZrOgvfu3VungDXhv3uJiLT7Z2N4/vz5Na6n9UfboqIi\n7Ny5EwkJCUhPT8fMmTPxzDPP6DVQ/sOSiKhhaEz4Y8aMwenTpxEaGoq33noLnTp10ttOt2zZghkz\nZuDmzZsICwuDv78/du3apbfyiYioOkloaGJbWFjA3t6+5o0kqUGueKVplJC+yoZKw5OqxvXN40Gp\ni9Z6ACZVl/KBC9piMdxr15xJkoTERM3PBwWZzmvEmDTlTo0tfLVabdCAiIioYXFSHCIiM8GET0Rk\nJpjwiYjMBBM+EZGZ0Dl5GhGRqeDFXOqHCZ+IGg1ezKV+2KVDRGQm2MInk6FwVCBfpfnrusKRX9eJ\n6oMJn0xGXg6/rhMZErt0iIjMBFv4DwBtXSHsBiGiCkz4DwB2hRBRbbBLh4jITDDhE90HhcIZgKTx\nVv48kWlhlw7RfcjLu2XsEIjqjC18IiIzwYRPRGQmmPCJiMwEEz4RkZlgwiciMhNM+EREZoIJn4jI\nTBgl4b/66qt4/PHH4evri2effRa5ubnGCIOIyKwYJeH3798fp0+fxokTJ/Doo49iwYIFxgiDiMis\nGCXhh4SEwMKifNfdu3dHRkaGMcIgIjIrRu/DX7VqFUJDQ40dBhHRA89gc+mEhIQgKyur2vL33nsP\nQ4YMAQC8++67sLGxQVRUlMZyVCqVfD8wMBCBgYH6DpWIqFFLSkpCUlKSzvUkIYQwfDjVxcXFYcWK\nFdi3bx9sbW1rXEeSJBgqPEmSAJWGJ1Uw2H6JiAxNU+40ymyZu3fvxgcffID9+/drTPZERKRfRmnh\nt23bFsXFxXBxcQEA/Otf/8Jnn31WPTi28ImI6kxT7jRal05tGDLhK52UyM/VfB1YXjaQiBorJnwi\nIjOhKXcafVgmERE1DCZ8IiIzwYRPRGQmmPCJiMwEEz4RkZkwyh+v7peLiwtu375t7DCIiIzK2dkZ\nt27dqvN2jWpYJodpEhHpzoUclklEZOaY8ImIzESjTvguSiUkSar3zUWp1LqfkydPIjQ0FEFBQejZ\nsycWLVoEAPDx8alzzEuXLr2vulbYvXs3evTogR49emDPnj31KuufnJ31czydnRvP8ZwwYQLc3d3x\n73//u17l/JNS6aKXY6lUumjdj6kcy4yMDAQGBqJPnz7o1asXjh07dt9l/ZPSST+vS6VT43ldhoSE\nICgoCN27d8fWrVvrVVZljboPX5Ik6CN4CZonS8vNzUWfPn2wdetWtG7dGgCwd+9ehISEoG3btkhN\nTa3Tvuq6jVqtlq8OVlZWhieeeAI//fQThBAICAhAcnJy+URweiBJEhIT619OUFDjOJ4AcPXqVaSm\npmLdunVYsWJFnfatTfk50c+rszEcy7y8PBQXF+Ohhx5CSkoKJk+ejAMHDtRp/5poneiwLlSN53VZ\nWloKKysr5Ofno0uXLjh37lyV9dmHbyA7d+5EeHi4/CIAyj99K4uLi8O7774LoLylExQUBABYtGgR\nnnrqKfTt2xdLly5FfHw8rly5gqCgICxYsAClpaWYNGkS+vbti969e+O3334DAIwbNw5Tp07FkCFD\n8NNPP8n7OX/+PFq3bg2lUglHR0d4e3vX+YVobKZ0PAHA3d290Q4EMKVjqVQq8dBDDwEAbGxsYGXV\nqAYAmtSxBCAfv4KCAjg5Oemtno3rrBhBRkYGPD09ta6jqYW9fv16JCUlwd7eHkIISJKEt956C4n/\na0Z//vnnaNu2LVauXIlr165h2LBhOHjwICRJgre3N5YtW1alvOzsbDg7O8uPnZyc7mtoljGZ0vFs\n7EzxWJaVlWHmzJn473//W7/KNTBTO5ZqtRpBQUE4ffo0li9fXv8K/g8Tvg5eXl74448/tK5T+YVQ\nubW4ePFiTJ8+HSUlJZgyZQp69uxZZbtTp07h0KFD2L17N4Dyr8UVevToUW0/rq6uyMnJkR/n5ubC\n1dW1bhUyMlM6njXtrzExxWM5efJkhIaGom/fvnWqi7GZ2rG0sLDA/v37cevWLfTu3RuhoaF6uVgU\nu3R0CAsLw/bt23HhwgV52Q8//FBlHRcXF2RkZABAlR+r/P39sWrVKixYsAAzZ84EUP5VreLF0rFj\nR4wdOxaJiYlITEyssm3l/rwKPj4+uHjxIvLz85GXl4eLFy/e1w9KxmRKx7NCY+3SMbVj+corr8Dd\n3R0vvPCCfirYgEzpWJaWlqKsrAwAYGdnh7KyMr29RtnC10GpVGLdunV48cUXUVRUhOLiYowYMQL9\n+vWTP/FDQkKwaNEi9O/fH35+fvLyMWPGIDs7G0VFRZg2bRoAICIiAmFhYQgNDcWUKVMwffp0uTX0\n5JNPIiYmBkDNrU5LS0ssWLAAAwYMAAC8//77ja51akrHEwDmzp2LXbt2ISsrC/3798fWrVthZ2dn\n6MOgF6Z0LI8ePYqlS5eiV69eCAoKQrNmzbBx48aGOAx6YUrH8vr164iMjISFhQWKiorwxhtvoGnT\npnqpZ6MepeOiVOJ2fs1XraoLZ4UCt/J4hStnZyVycup/PJ2cFLh927yPp1Lpgvz8+k8DolA4Iy+v\ncf1Oo2/ark5XFw/Slezud5ROo074RETmiMMyiYhIKyZ8IiIzwYRPRGQmjDJKZ+7cudi2bRskSYKr\nqyvi4uLg5eWlcztnZ+dGNyqFiEjfKv8Bs06EEeTl5cn3ly5dKiZOnFjjevUJLzEx8b63NTWsi+l5\nUOohBOtiqupTF0250yhdOgqFQr5fUFAgz8GhT0lJSXov01hYF9PzoNQDYF1MlSHqYrQ/Xr355ptY\nu3Yt7OzscPjwYWOFQURkNgzWwg8JCUGnTp2q3bZv3w4AePfdd3Hp0iWMGzcOL730kqHCICKiCvfd\nSaQnf/31l+jQoUONz/n6+gqUTyrOG2+88cZbLW++vr415lSjdOmkpqaibdu2AIBvv/0W/v7+Na73\n+++/N2RYREQPNKNMrRAREYGzZ8/C0tISbdq0wbJly9C8efOGDoOIyKyY9Fw6RESkP436n7Z9+/at\ndiHvxYsX45FHHsHChQuNFFX9ZGVlYdSoUfDx8UHXrl0RFhaG1NRUzJgxA506dULnzp3RrVs3pKen\nGztUAOXzeY8ZM0Z+XFpaimbNmmHIkCFatztx4gR27dolP1apVPjoo48MFmdtvPvuu+jYsSN8fX3h\n7++PI0eOYMmSJSgsLNS57eLFi2u1njHpOldxcXGYPn26scLTydLSEv7+/vDz80OXLl1w6NAhY4d0\nXzS9xxtCo54PPzIyEgkJCejfv7+8bMOGDVizZg169eplxMjujxACzzzzDMaPH4+EhAQAwMmTJ5GQ\nkIDMzEycOnUKQPmFt01lznZ7e3ucPn0aRUVFsLW1xd69e+Hp6anzH9HJyck4duwYBg0aBMD4V506\ndOgQdu7cieTkZFhbW+PWrVsoKirC4sWLMXr0aJ3zkS9ZsgRjxozR27zlhqDrXBn7HOhiZ2eH5ORk\nAMCePXswZ86cRjfuXtN7/Nq1a/LvmobUqFv4w4YNw86dO1FaWgoASE9Px9WrV3H+/Hm5peLn5wd/\nf3/4+/vDzs6u2sWCTUliYiJsbGzw/PPPy8s6d+4MBwcHuLm5ycvc3d31emHj+goNDcXOnTsBAPHx\n8YiMjJSnZj1y5Ah69OiBJ554Aj179sS5c+dQXFyMt956Cxs2bIC/v798oYwzZ84gKCgIbdq0wccf\nf9ygdcjKysJDDz0Ea2trAOVXN/r6669x9epVBAUFITg4GAAwdepUPPnkk+jYsSNUKhUAYOnSpVXW\nU6vVGDdunPyNbPHixQ1aF220navG1Lubm5sLFxcXAOV/UKr8jXLatGlYvXo1AOD1119Hhw4d4Ovr\ni1dffdUosVam6T3eq1cvvPrqq/JrpuI9kZSUhICAAAwdOhRt2rTB66+/jrVr16Jbt27o3LlzlSt0\n1YoBRlo2qMGDB4tvv/1WCCHEggULxKuvviri4uLEtGnTqqy3bds20adPH1FaWmqMMGtlyZIl4qWX\nXqq2PCMjQ3h7ews/Pz/xn//8RyQnJxshupo5ODiIkydPioiICFFUVCT8/PxEUlKSGDx4sBCifBqN\nimO+d+9eMWzYMCGEEHFxcWL69OlyOfPmzRM9evQQxcXF4ubNm8LV1bVBz1VBQYHw8/MTjz76qHjh\nhRfE/v37hRBCeHt7i+zsbHm9W7duCSGEKC0tFYGBgeLUqVPV1jt69KgICQmRt8nJyWmoamil61zF\nxsZWe9+YEktLS+Hn5ycee+wx4ejoKI4fPy6EKJ+CoKIOQggxbdo0sXr1apGdnS3atWsnL8/NzW3w\nmP9J03v866+/FiEhIUKtVotr166JVq1aiczMTJGYmCicnJxEVlaWuHfvnnB3dxfz5s2Ty5o1a1ad\n9t+oW/jA3906QHl3TmRkZLV1UlNTMXv2bGzcuBGWlpYNHWKtafpK7eHhgbNnz2LBggWwsLBAcHAw\nfvzxxwaOTrNOnTohPT0d8fHxCAsLq/JcTk4OIiIi0KlTJ7z88ss4c+YMgPLWpPjHxW0GDx4Ma2tr\nuLq6onnz5rh27VqD1cHe3h7Hjh3D8uXL0axZM4wcORJxcXFyrBU2bNiALl264IknnsDp06fl+lTW\npk0bXLhwATNmzMD3338PpVLZUNXQSdu5MnVNmzZFcnIyUlJSsHv37iq/R9TE0dERtra2mDhxIrZs\n2WIS3W2a3uM///wzoqKiIEkSmjdvjoCAAPz222+QJAlPPvkkWrRoARsbG/j4+MiXOO3YsWOdf8tr\n9Ak/PDwc+/btQ3JyMu7evQt/f/8qb9CCggKMHDkSK1euRIsWLYwYqW4dOnSocoHjymxsbDBw4EAs\nXLgQb7zxBrZu3drA0WkXHh6OV155pUoXAVA+M2pwcDBOnTqF7du3a/1h08bGRr5vaWkpd9U1FAsL\nCwQEBEClUuGTTz7B5s2bAfz9Jr148SI++ugj/Pjjjzhx4gTCwsJQVFRUrRwnJyecPHkSgYGB+Pzz\nzzFp0qQGrYcums5VY/LUU0/h5s2buHnzJqysrKBWq+XnioqKIISApaUljhw5goiICOzYsQMDBw40\nYsTltL3H/3kuKl53TZo0kZdZWFjIjy0sLOr8Hmn0Cd/BwQFBQUEYP348oqKiqj0/YcIEjB8/Hj17\n9jRCdHXTt29f3Lt3DytWrJCXnTx5EgcOHMDVq1cBAGq1GidOnIC3t7eRoqzZhAkToFKp0KFDhyrL\n8/Ly4O7uDgCIjY2VlyuVSuTr4XrE+nLu3LkqIyWSk5Ph7e0NhUKBvP9d7zgvLw/29vZQKpW4du1a\nlVFGldfLzs5GaWkpnn32Wbz99ts4fvx4w1ZGB03nqjH5888/UVZWBldXVzz88MM4c+YMiouLkZOT\ng3379kGSJNy5cwc5OTkYNGgQ/t//+384ceKEscPW+B53cnLChg0boFarcePGDRw4cADdunXT+wdy\nox6lUyEyMhLPPvus/EOHJEmQJAmXLl3C5s2bkZqailWrVgEAvvzySzzxxBPGDFerLVu2YNasWYiJ\niYGtrS28vb0xcOBAvPzyy7h37x4AoHv37pg2bZqRIy1X0Qrx8PCQY6o4/gAwe/ZsREdH45133kFY\nWJi8PCgoCO+//z78/f0xZ86cKmUZQ0FBAaZPn46cnBxYWVmhbdu2WL58OdavX4+BAwfCw8MD+/bt\ng7+/Px577DF4eXlVGQn2/PPPy+stWrQI48ePl1ud77//vrGqVYWucyVJEuLi4uRvj5Ik4fDhw/IH\ntrEVFhbK/8oXQmDNmjWQJAleXl4YMWIEOnbsiNatW8vv7/z8fDz99NNyi3/RokXGDF/2z/d469at\nsWjRIhQUFMDX1xeSJOGDDz5A8+bNkZKSovF9Ufnc1Rb/eEVEZCYafZcOERHVDhM+EZGZYMInIjIT\nTPhERGaCCZ+IyEww4RMRmQkmfCIiM8GET0RkJv4/SeiOK/c9HUoAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x118916310>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#Clustering on original X space\n",
"km = cluster.KMeans(n_clusters=4, init = 'k-means++', n_init = 10)\n",
"km.fit(dpro)\n",
"cols = ['r','y','b','g']\n",
"\n",
"fig = plt.figure()\n",
"for i, cc in enumerate(km.cluster_centers_):\n",
" w = 0.15\n",
" ax = fig.add_subplot(1,1,1)\n",
" plt.bar(np.arange(len(cc))+i*w, cc - dpro.mean(), w, color = cols[i], label='Cluster {}'.format(i))\n",
" ax.set_xticklabels(dpro.columns.values)\n",
" ax.set_xticks(np.arange(len(cc))+2*w)\n",
"\n",
"plt.ylabel('Mean Adjusted Score')\n",
"plt.title('Avg Scores by Cluster')\n",
"plt.legend(loc = 3, ncol = 4, prop={'size':9})\n",
" \n",
"#Now cluster on the SVD space\n",
"km_u = cluster.KMeans(n_clusters=4, init = 'k-means++', n_init = 10)\n",
"#km_u.fit(pd.DataFrame(U.dot(diag(sig))))\n",
"km_u.fit(pd.DataFrame(U))\n",
"cols = ['r','y','b','g']\n",
"\n",
"fig = plt.figure()\n",
"for i, cc in enumerate(km_u.cluster_centers_):\n",
" w = 0.15\n",
" ax = fig.add_subplot(1,1,1)\n",
" #plt.bar(np.arange(len(cc))+i*w, cc.dot(Vt), w, color = cols[i], label='Cluster {}'.format(i))\n",
" plt.bar(np.arange(len(cc))+i*w, cc.dot(diag(sig).dot(Vt)) - dpro.mean(), w, color = cols[i], label='Cluster {}'.format(i))\n",
" ax.set_xticklabels(dpro.columns.values)\n",
" ax.set_xticks(np.arange(len(cc))+2*w)\n",
"\n",
"plt.ylabel('Mean Adujsted Score')\n",
"plt.title('Avg Scores by Cluster Using SVD')\n",
"plt.legend(loc = 3, ncol = 4, prop={'size':9})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>In the above we chose $k=4$. Let's assume we did this because we want to assign students to groups of 4 to work together in a study group. Our goal would be to maximize the skill diversity of each group, so we'd cluster into 4 clusters and assign one student per cluster into each group. If we look at the above plots, we can get a sense of how the average person within each group differs from the average student in the class. When we make our profiles, we'll use the bottom chart, as the clusters using $U\\Sigma$ seem to be more pronounced.\n",
"<br><br>\n",
"<b><u>Student Profiles</u></b>\n",
"<ul>\n",
" <li><u>Cluster 0</u> doesn't have any defining strong properties and tend to be consistent with the average student profile. They do tend to be the strongest at math and machine learning though.</li>\n",
" <li><u>Cluster 1</u> looks like our best stats group.</li>\n",
" <li><u>Cluster 2</u> looks like our CS students. These the strong programmers in the group.</li>\n",
" <li><u>Cluster 3</u> are the business and strategy minded students. </li>\n",
"</ul>\n",
"</p>\n",
"\n",
"###Hierarchical Clustering\n",
"<p>\n",
"In the above we showed how to compute and evaluate K-Means, and we also came up with a use case for the clustering. The above use case (i.e., putting students into study groups of size 4) essentially dictated the choice of $k$. In a more general use case, we might not have such an application specific best $k$. One way we can be more general is to use hiearchical clustering. In this type of clustering, the individual clusters are embedded in a taxonomy. We can use this taxonomy to see if there are any natural values of $k$ that make the most sense. We can also use it to ensure that each final cluster we choose is well balanced in size. Additionally, we can use this to get a sense of any outlier clusters (those with very small counts).<br><br>\n",
"\n",
"Using Scipy isn't as straightforward as using Sklearn, but again, scipy has a good procedure for displaying the dendrogram.\n",
"\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEgCAYAAACgiViPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVOXCB/DfGXaU3R3FXZBNUFFxYXGB1FS0sry5a95K\nLW/5dvNzuwmV93rf6+v22nuvlWVqWmmK5oKCMCiogYgiamoKN/cVNGUdeN4/dE4gzAwgh+X4+34+\nlMw551nOnPnN4TlnnpGEEAJERKQamvpuABER1S4GOxGRyjDYiYhUhsFORKQyDHYiIpVhsBMRqQyD\nnRqUkJAQaDQahIaG1ndTAAAajQYajQZRUVH13ZRq6dChAzQaDaZNm1bfTaF6wGBXueLiYixevBie\nnp5o0qQJ7O3t0blzZ0RERCA1NbW+m2eQJEn13YRylGjP1KlTodFo0LFjx1ovu2fPnujXrx+6dOlS\n62VTw2de3w0gZb333ntYsWIFAKBr166wsbFBdnY2fvzxR4wbNw4BAQH13EKqzTeN/Px82NjYYOvW\nrbVWJjU+PGNXuU2bNgEAPvzwQ5w9exbHjx9Hbm4uDh8+XC7U169fjz59+qBZs2awsLCAk5MTnnvu\nuXJn9VqtVh6a+PLLLxESEgIbGxv06dMHJ0+eRHR0NNzd3WFnZ4fnn38eN27ckLctO8Tyv//7v3Bz\nc4ONjQ1GjhyJK1euGO1DXl4eFixYgM6dO8PS0hItWrTAzJkzcefOHXmdGzduYNKkSWjTpg2srKzQ\nokULBAUF4ZtvvjFa9o0bN/D666/Dzc1NLnv06NEG1y+7DxITE+XHnxyyKSkpwQcffIAuXbrAxsYG\nTk5O8PPzw4IFCwA8GipZt24dACA7O7tCmdnZ2Zg0aRJatWoFS0tLdO7cGZ988glKSkoq3af/+Mc/\n0KZNG7i6usrllx2KKVvH0qVLMXHiRNjZ2aFt27ZYtGhRuT6eOnUKgwYNgrW1Nbp3745t27ZxaKex\nEaRqLVq0EJIkicDAQPHjjz+Ka9euVbrenDlzhK2trfDw8BD+/v7CxsZGSJIk7O3txfXr14UQQiQk\nJAhJkoQkScLa2lp4eHgIKysrIUmScHV1FVZWVsLT01NoNBohSZJ49dVX5fKDg4Pl7WxtbYWXl5e8\nXt++fSusFxoaKoQQoqSkRAwcOFBIkiQsLS2Fn5+fcHBwEJIkCS8vL5Gfny+EEGLcuHFCkiRhZ2cn\nevfuLTp27CjMzc3Fa6+9ZnDf3L59W7Rv317uU7du3UTHjh2FRqOR19Evi4qKKrcPNBqNSExMNLje\nypUrhSRJwsLCQvj5+QkPDw9hY2MjunbtKoQQYuzYsaJ58+ZCkiRhZWUlAgMDRWBgoEhPTxeXLl2S\nnzdHR0fh5+cnLCwshCRJYtq0aRX2lZWVlTA3Nxfe3t6ic+fOQggh90u/flZWltxGS0tL4erqKtch\nSZKIjY0VQgiRn58v2rVrJ6/n5eUlbG1thbW1dYX6qeFisKtcZGSk/OLV/7i7u4sPP/xQ5OXlyeud\nO3eu3O/nz5+X11+zZo0Qonywz5o1SwghxAcffCA/9re//U0IIcSkSZOEJEmidevWcnn6ELK0tBRn\nz54VQgixfPlyeduEhIRy6+mDPTo6Wg7SlJQUIYQQV69eld94Vq9eLYQQwtvbW0iSJDZs2CDXefv2\nbZGRkWFw30RFRcllb9q0SX48PT1d/ndNg33OnDlCkiQxc+ZMeZ38/Hxx+PBh+fepU6cKSZJEx44d\ny7Vr3rx5QpIk0a5dO3H37t1y+0GSJHHu3Lly+0qj0Yi9e/cKIYQoLS0VQhgP9gEDBoji4mJx+/Zt\n+Q3j/fffF0IIsWbNGnm9rVu3CiGE2Lt3r/wYg71x4FCMyi1cuBBbt27FmDFj4ODgAEmScO7cOXz8\n8ceYPHmyvF5OTg5Gjx4NZ2dnaDQadOvWTV527dq1CuWOGjUKANC+fXsAj8aJ9Y/pLwaWHYrR8/X1\nlct++eWX5cdPnTpVaftTUlIAAEII9O3bFxqNBq6urigoKAAA/PTTTwAgD59MmTIFnTt3xogRI7B6\n9Wq0bt3a4L7Rb9uhQwe88sor8uN+fn4Gt6mqUaNGQZIkrFmzBq1bt0ZwcDD+8pe/wM7OTl5HGJh/\nT9/ny5cvw8XFBRqNBmPHjpWXHzlypNz67u7uCAsLA1C18frx48fD3NwcLi4uaNGiBQDg5s2bAIDM\nzEwAgJWVlVxnWFgYHB0dq9Rvahh48fQZEBERgYiICADAsWPHMGvWLBw7dgw7d+4EADx48ADh4eG4\nd+8ebGxs0KtXL1hYWMgBUnZcF3gUHvb29gAAc/PfDyH9Y1W9GGgo2Cqj0WjQp0+fCo/rg3vRokUY\nMGAA9u3bh8zMTCQnJyMmJgabN29Genp6lesxpWzf9Pvl3r17FdYLCwvDsWPHsGXLFpw4cQLp6ek4\nePAgPv/8c5w+fRpt27Y1WZejoyM8PDwqPN6kSZNyv7ds2bJafSgb0vrn78nnoqHdlUTVwzN2lfvg\ngw9w4sQJ+feePXvC3d0dAOSzx7Nnz8rh9OWXXyI1NRVLly5VpD0nTpzAuXPnAACbN2+WH/f29q50\nfX2Yl5aWIioqCocOHcKhQ4eQlJSETz75RP6rIykpCcHBwVi+fDni4uKwevVqAEBGRgZycnIqLbtf\nv34AgKysrHJtOX78uMH2689whRD45ZdfAADbtm2rsF5GRgaaNWuGjz/+GDt27JAvQj948ED+t62t\nLYBHF4cr67OVlRU2b94s9zkuLg5vvfUWnn/++XLr12YI+/r6AgAKCgrkN/69e/ciNze31uog5THY\nVe6LL76Av78/mjdvjl69eqF9+/bynTITJkwAAHTq1Ek+C5w+fTp8fX3L/en/pMrOtKt69m1lZYWe\nPXvCy8sL8+bNAwAEBAQgODi40vVHjx6NAQMGAADCw8Ph4eEBLy8v2NvbY+jQofj5558BAO+//z6c\nnZ3RpUsX9OrVC9OnTwcAtG3bFk5OTpWWPXv2bHko6eWXX0bXrl3RuXNn9OrVy2D7u3btCjc3NwDA\nO++8g9DQULz55psV1vv+++/Rrl07uLm5oVevXvDx8QHw6AzZy8sLANC9e3cAj4ZB3N3dERgYiIKC\nAsyfPx/NmzfHjRs30LlzZ/j5+aFz585wdHTEH/7wB3kYSgkTJkxAu3btAAAvvPACvLy8MHbsWFhZ\nWSlWJ9U+BrvKLVq0SB5fP3fuHG7evIlu3brhL3/5C5YsWQIAcHJywubNm+Hp6QkhBKytrfHjjz8C\nqPxssOxj+n9X9lhlAgICsHz5cjx8+BDW1tYYPnx4uXuuJUmqUNbevXuxYMECdO3aFdnZ2bhx4wa6\nd++OBQsWyGe3r7zyCvr27YuHDx/i1KlTcHBwQEREBPbs2WOwLc7Ozjhy5Aj++Mc/ws3NDb/++ise\nPHiAkSNHGuyvubk5vvvuO/j7+6O0tBS5ubly+8uuFxISgpEjR0KSJJw+fRqlpaUYMGAAfvjhB/ka\nw/Tp0/HCCy/A0dER58+fR0pKCkpLS+Hq6oqUlBRMnjwZLi4uOHPmDAoKCjBo0CAsWbJEfhN+cl89\n2WZTz11l61lZWWHPnj0YOHAgNBoNdDodNmzYgKZNmwIAbGxsDO5PajgkUZ2BTqIaCgkJwYEDBxAS\nEoL4+Pj6bg4Z8csvv5T7xKr+eQOA1atX47XXXqunllFV8eIpEZUzf/58HD9+HD4+Pnj48CEOHjwI\nAPD09MSkSZPquXVUFRyKoTphbNiAGpbBgwejSZMmSEhIwOHDh9GpUye8++67SE5OhrW1dX03j6qA\nQzFERCrDM3YiIpVhsBMRqQyDnYhIZRjsREQqw2AnIlIZBjsRkcow2ImIVKZeP3kaEhJS7uvFiIjI\ntODgYGi1WoPL6/UDSpIkVWtObiIiMp2dHIohIlIZBjsRkcow2ImIVIbBTkSkMgx2IiKVYbATEakM\ng52ISGUY7EREKsNgJyJSmQYV7M7OgCQp9+PsXN89JCJSXoOaUkCSACVbo3T5RER1gVMKEBE9Yxjs\nREQqw2AnIlIZBjsRkcow2ImIVEbRYM/NzcWLL76I7t27w9PTE0eOHFGyOiIigsJfjff2229jxIgR\n2LJlC3Q6HR4+fKhkdUREBAXvY7937x78/f1x8eJFw5XzPnYiomqrt/vYs7Ky0Lx5c0ybNg09e/bE\na6+9hry8PKWqIyKixxQbitHpdDh27BhWrVqFgIAAzJs3D4sXL8ZHH31Ubr3IyMgyv4U8/iEiIj2t\nVgutVlvl9RUbirl+/ToCAwORlZUFAEhKSsLixYuxc+fO3yvnUAwRUbXV21BMq1at0K5dO5w7dw4A\nEBcXBy8vL6WqIyKixxSdBOzEiROYOXMmioqK0LlzZ3z11VdwcHD4vXKesRMRVZupM3bO7khE1Mhw\ndkciomcMg52ISGUY7EREKsNgJyJSGQY7EZHKMNiJiFSGwU5EpDIMdiIilWGwExGpTKMJdmfnR58c\nfZof4OnLcHau3/1ARGRKo5lSoKFMB9BQ2kFEzy5OKUBE9IxhsBMRqQyDnYhIZRjsREQqw2AnIlIZ\nBjsRkcow2ImIVIbBTkSkMgx2IiKVYbATEakMg52ISGUY7EREKsNgJyJSGXOlK+jQoQPs7e1hZmYG\nCwsLpKSkKF0lEdEzTfFglyQJWq0WzpzInIioTtTJUEw9TvlORPTMUTzYJUnC0KFD0bt3b3z++edK\nV0dE9MxTfCgmOTkZrVu3xq1btzBs2DB4eHhg0KBB8vLIyMgya4c8/iEiIj2tVgutVlvl9ev0q/Gi\noqLQtGlTvPvuu48q51fjERFVW71+NV5eXh5+++03AMDDhw+xb98++Pj4KFklEdEzT9GhmBs3bmDs\n2LEAAJ1Oh1dffRVhYWFKVklE9Myr06GYCpXX0VCMszOQk1OzbZXm5ATcvVvfrSCixsTUUMwzEewN\neVy8IbeNiBqmeh1jJyKiuqf47Y7PquoM/0iSsm2pDg4NETV+HIpRyNP0pT6HZzg0RNTwcSiGiOgZ\nw2AnIlIZjrGrQG3fzllbY/4cryeqHwx2FcjJaZjj4g3pojDRs4RDMUREKsMzdgKg3Kdz1XTWzqEl\naiwY7ASg+sM5z+JtkWp6kyJ141AMEZHKMNiJiFRGNUMxpsaIq/tnNMdT619DnJWzIQ3H8BglQ1Qz\npUBtT0fwtGPIdTmlQG2Md9dHnQ2hjsaM++fZxSkFiIieMQx2IiKVUc0YOzU+Sc5J0OXoDC6fgg7Q\nStl10hZzJ3MMvDtQkbKTkpyh09X+xYIpUxZCq42q9XIBwNzcCQMHcgC/sWKwU73R5egQIkIqXaaV\ntFgrOgDoYLIcraQ1WE5VaSXtU21vjE6Xg5CQ2h8MDwkBgMhaLxcAtNoGdJWYqo1DMUREKsNgJyJS\nGQ7FENUjpcbfawOHY+pObV/TYLAT1SMlxt+1WkmRMX1STm2/iXIohohIZRjsREQqo3iwl5SUwN/f\nH6NGjVK6qmeDs/Ojz5KX+VmIyAqPQZIerUtEzxzFx9hXrFgBT09P/Pbbb0pX9WyoZOL0yDL/Lach\nzVhFRHVG0TP2y5cvY/fu3Zg5c6bRCWuIiKj2KBrsf/rTn/DPf/4TGg2H8omI6opiQzE7d+5EixYt\n4O/vD61Wa3C9yMjIMr+FPP4hIlIfY59bePKWx7L3tmu1WqM5+iTFgv3QoUPYsWMHdu/ejYKCAty/\nfx+TJ0/GunXryq1XNtijlJnPiIioQajO5xbKBn1ISAhCHk0OBACIMhGWio2R/O1vf8OlS5eQlZWF\nb7/9FoMHD64Q6kREVPuqFOzZ2dmIi4sDAOTl5eH+/fvVrkjiHRpERHXC5FDMZ599hs8//xx3797F\nhQsXcPnyZbzxxhvYv39/lSsJDg5GcHDwUzVUKc7/cEZOQSVjXsELIT3x546TtRPu/plzVDdmxuaA\nNzR1r5JztZtS07lkavIRdc7Brh4mg/3TTz9FSkoK+vXrBwDo1q0bbt68qXjD6kpOQQ7EwkrGvBYC\nT94bLkXxr47Gztgc8IYoOVe7KUrN5V4ZTvqlHiaHYqysrGBlZSX/rtPpOKxCRNSAmQz24OBgLFq0\nCHl5eYiNjcVLL73E6QGIiBowk8G+ePFiNG/eHD4+Pli9ejVGjBiBTz75pC7aRkRENWByjL2goAAz\nZszArFmzADya1Cs/Px+2traKN64hePLiatlxdl5MVZfqXFitzwuqRKaYDPbBgwdj//79aNq0KYBH\ntzuGh4fj0KFDijeuITB4cRW8mKo21bmwWp8XVIlMMTkUU1hYKIc6ANjZ2SEvL0/RRhERUc2ZDPYm\nTZogLS1N/v3o0aOwsbFRtFFERFRzJodili9fjvHjx6N169YAgGvXruG7775TvGFERFQzJoM9ICAA\nZ86cwdmzZyFJEtzd3WFhYVEXbSMiohqo0uyOR48eRVZWFnQ6HY4dOwYAmDx5sqINIyKimjEZ7BMn\nTsTFixfh5+cHMzMz+XEGOxFRw2Qy2NPS0nD69GlOI0BE1EiYvCvG29sb165dq4u2EBFRLTB5xn7r\n1i14enqiT58+8mRgkiRhx44dijeOiIiqz2Swl/9OUiIiauhMBnvZ79kjIqKGz+QY++HDhxEQEICm\nTZvCwsICGo0G9vb2ddE2IiKqAZNn7HPmzMG3336L8ePH4+jRo1i3bh3Onj1bF22jBsI5KQk5uidm\nPZzSAZI22+S2TubmuDuQsyDWhZp+jV5Z9fktSvxqvtpTpQ8ode3aFSUlJTAzM8O0adPg5+eHxYsX\nK902aiBydDqIJ4fkQgCgAySttuKyMiStVrF2UXm18TV6Wq1UZ1/FV1ndVDtMBnuTJk1QWFiIHj16\n4L333kOrVq0gRP088UREZJrJMfb169ejtLQUq1atgq2tLS5fvowffvihLtpGREQ1YDLYo6OjYWNj\nAwcHB0RGRmLp0qXYtWtXXbSNiIhqwORQzNq1a/H222+Xe+yrr76q8Bg1AM7OQM4TF8/0U0E4OQF3\neWGK6k9VLu4+zTh7Y7/4Wtn+Kbs/qtM/g8G+adMmbNy4EVlZWRg1apT8+G+//QYXF5fqtpnqQk4O\nYOj6B+f6oXpm7OJubVy0bewXX01d/K5O/wwGe//+/dG6dWvcunUL8+fPly+Y2tvbw9fXt0qFFxQU\nIDg4GIWFhSgqKsKYMWPw97//vcqNIyKi6jMY7O3bt0f79u0RFxcHGxsbmJmZ4ezZszh79ix8fHyq\nVLi1tTUSEhJga2sLnU6HgQMHIikpCQN5XzMRkWJMXjzVn3FfuXIF4eHhWL9+PaZOnVrlCmxtbQEA\nRUVFKCkpgbOzc40bS0REppkM9tLSUtja2mLr1q148803sXnzZmRmZla5gtLSUvj5+aFly5YIDQ2F\np6fnUzWYiIiMq9InTw8fPoxvvvkGa9asAfAorKtKo9Hg+PHjuHfvHsLDw6HVastNLFZ+9siQxz9E\nRI9UZ6qEqlxgbIx3z2i1WqxdC2i1kVVa32SwL1++HH//+98xduxYeHl54cKFCwgNDa12wxwcHDBy\n5EgcPXrUYLBHRVW7WCJSudqYKqGsxnj3jD4zQ0IiAQBRJsLSZLAHBwcjODhY/r1z585YuXJllRpz\n+/ZtmJubw9HREfn5+YiNjcXChQurtC0REdWMwWB/++23sWLFinL3sOtV9RuUrl27hilTpqC0tBSl\npaWYNGkShgwZ8nQtJiIiowwG++TJkwEA7777boVlVf1iax8fHxw7dqyGTSMiopowGOy9evUC8Ghs\n59atWwCA5s2b102riIxIck6CLqf8/PBaSSv/29zJHAPv8rMShpi6GFmtTzg2sAuRVb3QaqyPDa1P\nNWEw2IUQiIqKwqpVq1BSUgIAMDMzw9y5czlOTvVKl6NDiAgxuLxsyFNFVfnoelUvVja0C5G1NSd9\nY2fwPvZly5YhOTkZqampyMnJQU5ODlJSUpCcnIylS5fWZRuJiKgaDAb7unXrsHHjRnTs2FF+rFOn\nTvjmm2+wbt26OmkcERFVn8Fg1+l0lY6pN2/eHLonv/+SiIgaDIPBbmFhYXAjY8uIiKh+Gbx4mpGR\nATs7u0qX5efnK9YgIiJ6OgaDXX8nDBE1TMa+cUcNt+wpzditkYbujGks+7VKk4ARUcNj6huJyLia\n3BrZWParyWl7iYiocWGwExGpDIOdiEhlGOxERCrDi6dE1GgZuzMIqPldLNW5Y6Yh3inDYCeiRqsq\nE5opUW5t1KEkDsUQEakMg52ISGUY7EREKsNgJyJSGQY7EZHK8K4Yoip68rtW+T2r1FAx2ImqyNh3\nrfJ7Vqkh4VAMEZHKMNiJiFRG0WC/dOkSQkND4eXlBW9vb6xcuVLJ6oiICAqPsVtYWGDZsmXw8/PD\ngwcP0KtXLwwbNgzdu3dXsloiomeaomfsrVq1gp+fHwCgadOm6N69O65evapklUREz7w6G2PPzs5G\neno6+vbtW1dVEhE9k+rkdscHDx7gxRdfxIoVK9C0adNyyyIjI8v8FvL4h4gaIn6B9iNP7ofamCrY\nGK1Wi7VrAa02skrrKx7sxcXFeOGFFzBx4kRERERUWF422KOilG4NET0NfoH2I3W9H0JCQh7/PxIA\nEGUiLBUdihFCYMaMGfD09MS8efOUrIqIiB5TNNiTk5OxYcMGJCQkwN/fH/7+/oiJiVGySiKiZ56i\nQzEDBw5EaWmpklUQEdET+MlTIiKVYbATEakMg52ISGUY7EREKsNgJyJSGQY7EZHKMNiJiFSGwU5E\npDIMdiIilWGwExGpDIOdiEhlGOxERCrDYCciUhkGOxGRyjDYiYhUhsFORKQyDHYiIpVhsBMRqQyD\nnYhIZRjsREQqw2AnIlIZBjsRkcow2ImIVIbBTkSkMgx2IiKVUTTYp0+fjpYtW8LHx0fJaoiIqAxF\ng33atGmIiYlRsgoiInqCosE+aNAgODk5KVkFERE9gWPsREQqY17fDYiMjCzzW8jjHyIi0tNqtVi7\nFtBqI6u0foMK9qio+msHEVFDFRIS8vj/kQCAKBNhyaEYIiKVUTTYJ0yYgP79++PcuXNo164dvvrq\nKyWrIyIiKDwUs2nTJiWLJyKiSnAohohIZRjsREQqw2AnIlIZBjsRkcow2ImIVIbBTkSkMgx2IiKV\nYbATEakMg52ISGUY7EREKsNgJyJSGQY7EZHKMNiJiFSGwU5EpDIMdiIilWGwExGpDIOdiEhlGOxE\nRCrDYCciUhkGOxGRyjDYiYhUhsFORKQyDHYiIpVhsBMRqYyiwR4TEwMPDw907doV//jHP5SsioiI\nHlMs2EtKSjBnzhzExMTg9OnT2LRpE86cOaNUdURE9JhiwZ6SkoIuXbqgQ4cOsLCwwCuvvILt27cr\nVR0RET2mWLBfuXIF7dq1k39v27Ytrly5olR1RET0mLlSBUuSZHKdHj16VFjP2GamiqzptlKk8YKN\nLTe6TKG+1LyjJvppbFNj2xkt1cTymhb8NJXWR7kNrjONbQc2tDqVKrdqdfbo0cPoWooFu6urKy5d\nuiT/funSJbRt27bcOsePH1eqeiKiZ5ZiQzG9e/fG+fPnkZ2djaKiInz33XcYPXq0UtUREdFjip2x\nm5ubY9WqVQgPD0dJSQlmzJiB7t27K1UdERE9JgkhRH03goiIao9ZZGRkZH1VnpGRgW3btiEuLg6/\n/PILWrZsCTs7O1y9ehV2dnYoLS1FdHQ0tm3bhitXrsDDwwMaTe2PHqWkpMDV1bXWyyUqq6ioCLt2\n7cKdO3fQtm1bbNiwAUlJSejWrRusra3ru3m1KjMzE3fv3kWzZs3kx44cOVLhOlt1paeno6CgAE5O\nToiNjcXPP/+MLl26lLsJ4+TJk4iPj0dBQQFcXV2xY8cOtGvXDhYWFgbLPXr0KH744Qfs27cPGRkZ\nMDMzQ5s2bZ6qrfWp3s7Y//znPyM/Px9+fn6Ij4+HtbU1zMzMMGDAAKxduxbx8fF46623YGtri8GD\nByM9PR1paWn4/vvvDZZZXFyMnTt3wsXFBQMGDMCXX36JoqIiTJ06FU2aNEFpaWmFbYQQCA8PR0xM\nDKKjo3H48GHk5ubC0dERgYGBiIiIgEajwe7du2Fubo6wsDD5zSU6OhoRERHIycmBk5MTAODHH39E\nZmYmunTpghdffLFKdwfVlg8//BAfffQRVqxYgVGjRqFTp04V1iktLTXal8p8+umnmD17No4dO4ae\nPXsiLy8Pq1evxpkzZ9CpUye8/vrrcHR0NNguU/untLQUO3fuxKFDh5Cbm4tWrVph5MiRCAgIqNF+\n2LFjBwYMGIBvvvkGzZo1w9ixY7FkyRLcu3cPs2fPRseOHY3uh5EjRxo8jmxsbGrc1oiICPTp0we5\nublIS0vDiBEj4OLigk2bNmHEiBEGnzNj/Rw9ejTu3LljsK9t27Y12JcvvvjCaJ3Gym3fvr3B/Xfg\nwAHcvHkTFhYWuHXrFr788ku0aNECoaGh2Lp1a41eKzt27MCePXtQWFiI/Px8WFtbw87ODvb29rh8\n+TKuX7+OmJgYLF++HHFxcXj++eeRnJyMtm3b4uuvv4abmxtatmyJcePGYfTo0XIbAGDevHkoKirC\n0KFD4eDggHv37mH//v0wNzfHsmXLavx863Q6g5libl75KPiOHTvQtm3bGr3Oyqq3YB88eDDi4+Pl\n34cOHYq4uDgMGTIEkiQhLi5OfkzP1IHx0ksvwdfXFw8ePEB6ejqGDx8OOzs77Nq1C9HR0bCxsUG/\nfv0qtOXEiRMYMWIEfH19MXToUNjb2+P+/fuIi4tDRkYGhBDo2LEjLCwsEBsbiy+++AIeHh4IDQ1F\nQkKC3JcFCxYgJycHERERSEpKwpUrVzB37txqPUn6cDb1JuXm5gY3N7dyL4ZTp07B29sbly9fhq+v\nL65fv47hw4dj3Lhx8PHxAQC8+uqrRvsyaNAgSJKEsoeFvlxzc3PEx8dj8uTJ6N+/v/yG+/XXX2P3\n7t0G+2Js/3z11VeYMWMGvLy80KNHD8THx+O3336Di4sLrKysMH/+fIMvjl9//bVcO/XtnjJlCmxs\nbDBt2jTPaaTyAAAOIUlEQVTk5ORg9erVWLhwIVxcXBAZGQmtVmt0P7i4uBg8jlxcXAy29f3336/0\nWNfvB/0+BgBvb29kZmbKx/V//vMfg8/ZxYsXDfYzKSkJw4YNM9jXZs2aGexLRkaGwToBGC3X1dXV\n4P7T6XQ4ePAggEd/lc+dOxdLlizBe++9B0mSjB4Lxvqq0Whw4MABAICPjw9OnjwJAAgODoZGo0FC\nQgKCgoKQkJAAMzMzAMCAAQNgaWmJhIQEXLx4EVu3bsXOnTthaWmJiIgIvPnmmwgKCpLLLSsoKAhd\nu3at0bFpbm6OiRMnGsyUjz76yGA/LS0tq/U6q0y9Bfsrr7wCf39/+Pj4IDExEfn5+Vi5ciVCQ0Mx\nbdo0HDhwACUlJSguLkZQUBAyMjJgY2ODtLQ0gwdGVlYWtFotACAgIACpqakAIL+gevbsifj4+AqB\nOnToUBQWFsoHY1mDBg2CRqNBYmIiAODq1auYOnUqZs+ejeXLlyMhIUEu/8kDJDg4GGZmZgafpMzM\nTIPh3KJFC6NvUlu3bsWWLVswbNgwTJw4ERYWFhg+fDj27Nkjt+fhw4fYs2cPoqOjcebMGQwZMgQ/\n/fST0b4sW7YMJ06cwJQpUxAaGgoAcrlDhgxBbGwshg8fjpiYGLndwcHByMrKMtgXMzMzg/snMTGx\nXOABv7/pDx06FK1atTL44ti+fTtefPHFCs/Zvn370LVrV/lY8PT0xOnTpwEAISEh0Gq1ct2V7Qch\nhMHjCIDBtsbFxRl9w3VyckJAQAByc3Nx7NgxjBw5Ek5OTvj2229RXFxs8Dn717/+ZbCfV65ckftU\nWV8BmOxLZXX+93//t9FyhRAG919RURESEhJgaWkJALh79y4mTpyIo0ePwsvLy+ixYGdnZ7CvHTp0\nQHJyMoDf/1rRt+fMmTMICwtDQkICzp8/DxsbGwCP7s6zs7Mr95wBwPXr17Fjxw7MmjULf/rTn/Dw\n4UOEhYXBzs4O9+/fx/79+2FtbY0TJ07U6NjcsGEDBg0aZDBTjh8/brCfHh4eBl9n+n1uSr0Fu06n\nw7Zt25CVlQV3d3eMGjUKGo0GV69eRZs2bXDlyhXs3bsXN27cgKOjI/r3748ePXoYDVGdTic/8Skp\nKejTpw+A31/M165dg7OzM6ysrCq0ZdmyZdBqtQgJCZGfpMTERAQFBWHnzp3YtWsX7OzsAACFhYWY\nNWsWtm3bhvv378PBwQG+vr44c+YMfvnlFzg6OqKkpAR9+vSBo6OjwSfp7bffNhjOZV9Ulb1J6e3e\nvRvr1q1DYGAgduzYgf3791dYB4AcHJ988onRvugfW7NmDRITE/GHP/wB//rXvxATE4PY2Fh89tln\n0Gg0uHbtGgYOHIgzZ84gODgYbm5uBvtibP+kpaVh7ty5KC4uhq+vLxITE+Ht7Y2//vWvFc7+yho0\naBB0Oh22b9+OFi1alFs2fvx4NG3aFJIk4cGDB7CysoK9vT2cnZ1x6tQp/PDDDwgKCjK4H3x8fAwe\nRz4+PpW2Vf98GXvDLSwsRExMDJo1a4bAwECsX78e9+/fx8SJEzFu3DiDz9nChQsN9vP777/H9OnT\nDfb1+vXrBvsiSZLBOsPCwsqVqx/60Jd769Ytg/svNjYWHTp0QMuWLcu9xjZv3ozXX3+9wrFQWlqK\ngIAApKWlITAwsNK+vvzyy/jwww/h7u5ebhijqKgIMTEx8PX1lR9r06YNLC0t8eDBAxw8eBAajQbh\n4eEVjqGyjh07hp9++gm5ublwcHBAYGAg/P39a3xsHjx4EP/85z8NZsq2bdsM9nPmzJkGX2fz5s0z\n2g+9RndXTGUhoT8w9u7dC2dn53IXWIuLi5Geni4f0MYcOHAAp06dgqOjIxwcHNC7d29cvHgRrVq1\ngpOTExwcHOR1V61ahd69e1c6tAMAeXl5yMzMxL1790w+SZWF84ABA4y+ST0pPj4emZmZCAwMhKWl\nJczMzGBubg4PDw95nSNHjqBly5ZwcnKq8FfLgQMHEBQUVO6x4uJirF+/HufOncPixYsBPLoodvDg\nQXl/9O7dGxcuXJD3Q2V90W9nZmYm3/Kal5eHjIwMebukpCTEx8djzJgx8qfqDh06hOTkZIMvjnfe\neafCWKX+egDwaBggNzcXQUFBSExMlN+Eg4KCkJ2djbt378LJyQkdO3ZEbGwsCgsL4ejoiO7du+M/\n//lPuWX5+flo3bo1AgICkJKSAq1Wi/DwcNy8eRNFRUUICwsrd3HO0H4w5Pjx4/Dz86vweEpKCnr2\n7ImMjIxy7SkqKsJzzz0nDznoT3KCgoLw0Ucfwc/PD8OHD8f9+/dx5coVWFpaysdCcXExDh8+DAcH\nB5SWlsLR0dFguSdOnICTkxPc3NzkfdiyZUu0aNECly9fRrNmzSocY4ZeE3rp6elwcHBAp06d5H3r\n6uqKXr16QafT4eeff6702DVVrlJSU1PlE1D9sZmSkoLExESDx+Z7770HALh58yaOHj2Ke/fuya+X\n7Oxs+Pv7V3oxV/9az8/Px+HDh+UT2969eyMrK6tKOQY0wmCvzMqVKxEcHFxufFBPCIHnnnsOsbGx\nRst45513DF7w0el0BsedDxw4YPCirL7eQ4cO4eeff4atrS2aNGmCe/fuYcKECfKLRy8hIQEnT57E\nW2+9hTt37sDZ2bncn/QrV65EYGAgAgICjF4I9vX1NdiXygKmNvZRQkJChRfsmTNnMGfOHMyfP9/o\ndqbKvXXrFo4ePSqfTR09ehQffvih0esBvXv3lsu8ffs21qxZU67MN954w+CFOBsbmxotW7t2LYDy\nb2Lx8fE4deoU+vTpg759+xrct8aOoS5duhit01hfnJ2dDe5bDw8Po+Uaa5OPj4/R58wQY21du3at\nyWOhrpWWlsrHl/61WPb1kpqaitTUVDg5OaFJkybIzc3Fq6++CjMzM3n/ld1ev+3evXsr1PXkssq2\nM/Ua1VPsA0pKMfZiTk1NNXhx1JTU1NRyF3xeeuklLFmyBAAwbtw4g+POANCkSROD9Ro6kGfMmIEL\nFy5U2pctW7ZACGH0zcRYnWWvFzzZF2PbPc0+MtTP6dOn48KFCwa3M1Wuoec7Li7O6PNSdnxTX+b/\n/M//lCvD0IU4IUSNlgHl3/z0byiDBw82GUymnk9jdRrri7F9b2w7U20qKCgw+pwaYqpOY8dCfajq\na1s/BKZ/ba9du9botjVdVmWikVm6dKmYMmWKiI+Plx977rnnhBBC+Pv7i5ycnArbDBkyxGS5/fv3\nF4WFhfLvd+7cEcOHDxfNmzcXQghRUFAgPv30UzF+/HgRHR0twsPD5XWN1Tto0CD5d29vb/nfQUFB\nRvtibJmpOo31Ral9ZKyfpvatseWm9oOh56Uqdept375d/ndwcHCNlwkhxMCBA+XHTpw4IYKCgkRq\naqoICQkRxph6Po3Vaaq9hvaDqXJreowZU5W+1KRcpdT0tW1q25ouq6pGF+xCGH4xX716VRQUFFRY\nv7i42GSZR44cEdevX6+w3caNG8s9VlRUJNasWSP+/Oc/y48Zq9fUgWzsDcPYMmN1GuuLUvvIWD9N\n7VtTy43tB70nnxdTZWZmZlboc2Fhodi+fXuNlwlR82Ay9ryYqtPYcmP7wVS5NT3GjDFVZ03LVcrT\nvLaNbVvTZVXVKINdr7KQbWhMHch6xvqipn4+jcawHxpaMJFy6uKYrylVXDwlIqLfKfpl1kREVPcY\n7EREKsNgJyJSGQY7qcaiRYvg7e2NHj16wN/fHykpKYrVFRISgrS0NMXKJ3oaje4DSkSVOXz4MHbt\n2oX09HRYWFjg7t27KCwsVKw+SZLqdEpmourgGTupwvXr19GsWTN5/g1nZ2e0bt0aH3/8Mfr06QMf\nHx/88Y9/lNcPCQnBO++8g4CAAHTv3h2pqakYO3YsunXrhr/+9a8AgOzsbHh4eGDixInw9PTESy+9\nhPz8/Ap179u3D/3790evXr0wfvx4PHz4EADw/vvvy1O+/td//Vcd7AWiRxjspAphYWG4dOkS3N3d\nMXv2bPlj63PmzEFKSgpOnjyJ/Px87Ny5E8CjM24rKyukpqbijTfewJgxY/Dvf/8bmZmZWLt2LXJy\ncgAA586dw+zZs3H69GnY29vj//7v/8rVe/v2bSxatAj79+9HWloaevXqhaVLl+Lu3buIjo7GqVOn\ncOLECfnNgqguMNhJFZo0aYK0tDR89tlnaN68OV5++WV8/fXXiI+PR79+/eDr64v4+Hh5TnEA8nze\n3t7e8Pb2RsuWLWFpaYlOnTrh0qVLAIB27dohMDAQADBx4kQkJSXJ2wshcOTIEZw+fRr9+/eHv78/\n1q1bh19//RUODg6wtrbGjBkzsG3bNnl+cKK6wDF2Ug2NRoPg4GB5ps9///vfOHnyJNLS0uDq6oqo\nqCgUFBTI6+vn5ddoNOXm6NdoNNDpdABQbhxdPJ6Y7UnDhg3Dxo0bKzyekpKC/fv3Y8uWLVi1apXJ\nqXuJagvP2EkVzp07h/Pnz8u/p6enw8PDA5IkwcXFBQ8ePMDmzZurXe6vv/6KI0eOAAA2btyIQYMG\nycskSUK/fv2QnJyMCxcuAAAePnyI8+fP4+HDh8jNzcXw4cOxdOnS6s3MR/SUeMZOqvDgwQPMnTsX\nubm5MDc3R9euXbF69Wo4OjrC29sbrVq1MjgfurE7XNzd3fHpp59i+vTp8PLywhtvvFFuebNmzbB2\n7VpMmDBBvgtn0aJFsLOzw5gxY1BQUAAhBJYtW1a7HSYygnPFEBmQnZ2NUaNGyXOGEzUWHIohMoL3\nqlNjxDN2IiKV4Rk7EZHKMNiJiFSGwU5EpDIMdiIilWGwExGpzP8DKdK6O6L2hfYAAAAASUVORK5C\nYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x107a45e90>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from scipy.spatial.distance import pdist, squareform\n",
"from scipy.cluster.hierarchy import linkage, dendrogram\n",
"from sklearn import cluster\n",
"import pandas as pd\n",
"\n",
"d = pd.read_csv('/Users/briand/Desktop/ds course/datasets/survey.csv',header=0,index_col=0)\n",
"dpro = d[['profile_{}'.format(k+1) for k in range(7)]]\n",
"dpro.columns = ['Viz','CS','Math','Stats','ML','Bus','Com']\n",
"\n",
"\n",
"#This function gets pairwise distances between observations in n-dimensional space.\n",
"dists = pdist(dpro)\n",
"\n",
"#This function performs hierarchical/agglomerative clustering on the condensed distance matrix y.\n",
"links = linkage(dists)\n",
"\n",
"p = 46\n",
"#Now we want to plot the dendrogram\n",
"den = dendrogram(links, truncate_mode = 'lastp', p = p)\n",
"plt.xlabel('Samples')\n",
"plt.ylabel('Distance')\n",
"plt.suptitle('Samples clustering', fontweight='bold', fontsize=14);\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>Let's try to understand the output of the linkage function. The first few records look like:\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 177,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 19. , 30. , 2. , 2. ],\n",
" [ 18. , 22. , 2. , 2. ],\n",
" [ 33. , 39. , 2.23606798, 2. ],\n",
" [ 3. , 5. , 2.44948974, 2. ],\n",
" [ 0. , 46. , 2.44948974, 3. ]])"
]
},
"execution_count": 177,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"links[:5,:]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>The way we read this is as follows:\n",
"<ul>\n",
" <li>At the lowest level, each record is assigned to its own cluster, and the cluster number is just its original index.</li>\n",
" <li>The linkage function returns an (n-1) by 4 matrix where:</li>\n",
" <ul>\n",
" <li>The ith row corresponds to a new cluster, whose id=n+i</li>\n",
" <li>The L[i,0] and L[i,1] columns are the two clusters that are joined to make cluster (n+i)</li>\n",
" <li>The column L[i, 2] is the distance between L[i,0] and L[i,1]</li>\n",
" <li>The last column is the final size of cluster (n+i)</li>\n",
" </ul>\n",
"</ul><br>\n",
"In the above few rows, we see that user 19 and 30 were combined to make cluster 46. Then, we see in the 5th row that this cluster 46 was joined with user 0 to make cluster 50. If we look up at the dendrogram we can see these 3 students being combined into a single cluster.\n",
"\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 191,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"#Note code to follow"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>If we want to use a hiearchical clustering technique to get a specific number of clusters, we can use <a href=\"http://scikit-learn.org/stable/modules/generated/sklearn.cluster.Ward.html#sklearn.cluster.Ward\">sklearn.cluster.Ward</a> for a concise process that returns exactly what we need. We'll continue with our student study group example and choose $k=4$.\n",
"\n",
"\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"Ward(compute_full_tree='auto', connectivity=None, copy=True,\n",
" memory=Memory(cachedir=None), n_clusters=4, n_components=None)"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ka = cluster.Ward(n_clusters = 4)\n",
"ka.fit(dpro)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>Let's do another plot of mean adjusted centroids to get a sense of what each cluster represents. Sklearn.cluster.Ward does not return the centroids, so we'll have to compute this step ourselves.\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.legend.Legend at 0x10fc58810>"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#Clustering on original X space using Hierarchical clustering\n",
"\n",
"cols = ['r','y','b','g']\n",
"\n",
"fig = plt.figure()\n",
"for i in range(4):\n",
" w = 0.15\n",
" ax = fig.add_subplot(1,1,1)\n",
" cc = dpro[(ka.labels_==i)].mean()\n",
" plt.bar(np.arange(len(cc))+i*w, cc - dpro.mean(), w, color = cols[i], label='Cluster {}'.format(i))\n",
" ax.set_xticklabels(dpro.columns.values)\n",
" ax.set_xticks(np.arange(len(cc))+2*w)\n",
"\n",
"plt.ylabel('Mean Adjusted Score')\n",
"plt.title('Avg Scores by Cluster')\n",
"plt.legend(loc = 3, ncol = 4, prop={'size':9})\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>When we plot the 4 clusters using hierarchical clustering, we get similar conceptual groupings as we did with k-means. However, I find this latter plot more easy to interpret (remember, this is a bit subjective).\n",
"<br><br>\n",
"<b><u>Student Profiles using Hierarchical Clustering</u></b>\n",
"<ul>\n",
" <li><u>Cluster 0:</u> This group has most of the math/stats experience</li>\n",
" <li><u>Cluster 1:</u> This group is a little below average in skill across all categories (or at least is the group that underrates their own skill levels).</li>\n",
" <li><u>Cluster 2:</u> These are the strong programmers in the group.</li>\n",
" <li><u>Cluster 3:</u> These are the business and strategy minded students. </li>\n",
"</ul>\n",
"<br><br>\n",
"So now we have to make a choice - which clustering method to use for the student profiles. One last thing to compare is the distribution of students across clusters for each method.\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 199,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAEKCAYAAADgl7WbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGlhJREFUeJzt3XtwVPXdx/HPAdYqISEXNHJNMGAwXJIUp4iKQhEobbUg\nHcXWFkRFbTsWDXVqLRWnVtEOtRar8UIhth1rbavIiDwKZZVBLF5wnOBUqwYN5AJK8MmlmEC+zx/A\nPokku2c3m2zyy/s1c2ayu+f8ft9zvruf3ZzsbjwzMwEAerQ+iS4AANBxhDkAOIAwBwAHEOYA4ADC\nHAAcQJgDgAN6bZiPGzdOL7/8cqLLAOCwu+++W9dee22XzOVkmGdnZ2vz5s2trlu7dq2mTJkSulxa\nWqoLLrigU+YvKirSrFmzWl23ZMkSXXzxxZ0yX0/zxf785S9/UXp6urZu3ZrAqjpHT74v9KY+LVy4\nUMuWLWt13e7du9WnTx81NzeHrispKdFLL73Uar1169apoKBAAwcO1Kmnnqrp06dr9+7dkqRbb71V\njz76aKfXL0n9umSWLuZ5njzP65SxDx8+rH79wh+2O++8UxMmTNDatWu1cOFCbd++XY8//rhKS0s7\npaaepmV/SkpKVFRUpA0bNuicc85JcGXx15PvC72pT+Eyw8z0yCOP6LTTTmt1OTMzU2PHjtWCBQv0\n9NNPa9q0aaqrq9MLL7ygvn37dmX5oUKdk52dbZs3b2513dq1a+38888PXc7KyrJNmzaZmVlzc7Pd\nfffdlpOTYxkZGXbZZZfZgQMHzMysrKzMPM+z1atX24gRI+zCCy/0VcOWLVssIyPDdu/ebWPGjLFH\nHnkkPjvngOzsbNu0aZMVFxfboEGD7I033mh3Xc/z7MEHH7RRo0ZZcnKyLVu2zN5//30755xzbODA\ngXb55ZdbY2NjaP3169dbfn6+paam2rnnnmtvv/126LbjPU5OTra8vDx7+umnQ7etWbPGzjvvPFu6\ndKmlpaXZyJEj7fnnn291+xlnnGHJyck2cuRI+/Of/+x7f3vqfaE39WnhwoX285//vNV1xx/7R44c\nsaamJvv1r39tBQUFNnnyZFuzZo2ZmT311FNWUFDQ7ri33367XXnllaHLJSUlNmLECMvIyLBf/vKX\nrXKoo5wN8y8eoDVr1rQK85aB/9vf/tYmT55se/futcbGRrvuuuvsiiuuMLP/b+iCBQusoaHBDh06\nZB999JGlpqZaeXl52Dquu+46y8jIsK9+9atx3sOeLTs72y699FLLzMxs9SBui+d5NmfOHKutrbVd\nu3bZSSedZNOmTbOysjL77LPPLC8vz0pKSszM7M0337TTTjvNduzYYc3NzVZSUmLZ2dmhEHnqqaes\nsrLSzMyefPJJS0pKsqqqKjM7ev8IBAL22GOPWXNzsz300EM2ZMgQMzOrq6uzlJQUe++998zMrKqq\nynbt2mVm5vR9oTf1KVKYNzY22sqVK62goMDOO++8UJh/+OGHdvLJJ9tNN91kW7Zssdra2lZjLF++\nPBTmu3btsgEDBti2bdussbHRli5daoFA4IQXnrFyMsyzsrJswIABlpqaGlr69+9vU6ZMCa3TMszP\nOuusVge0oqLCAoGAHTlyJNTQsrKyqOv44x//aJ7n2WOPPdbhfXJJVlaWpaSk2Jw5c6y5uTnsup7n\n2SuvvBK6PHHiRLv33ntDl4uKimzJkiVmZnb99dfbsmXLWm2fm5trL730UptjFxQU2Lp168zsaEiM\nGjUqdFt9fb15nmfV1dVWV1dnqamp9ve//90aGhqi29ljeuJ9oTf1KVyYHz582B5++GH7xz/+YWvX\nrrUtW7ZYcXFx6DeGV1991S677DI79dRT7eSTT7aFCxdaXV2dmbV+ZX7HHXfYd77zndD4DQ0NdtJJ\nJ8UtzJ38A6jneVq3bp1qampCy4MPPihr5zvFdu/erblz5yotLU1paWnKy8tTv379VF1dHVpn+PDh\nUdXw6aef6ic/+YluuukmLVu2TJ999lmH9sklnuepuLhY7777rq655prQ9WPHjlVycrKSk5O1bdu2\n0PWZmZmhn0855ZRWl08++WTV19dLkj766COtXLky1Me0tDTt2bNHlZWVkqTHH39chYWFodtKS0v1\n6aefhsY6/fTTQz/3799fklRXV6ekpCQ9+eSTKi4u1pAhQ/TNb35T7777ru/97an3hd7Up379+qmp\nqanVdU1NTerTp4/69u2rxYsXa+7cuaHjct1112nOnDmSpEmTJunJJ5/Uvn37tHXrVr388sv61a9+\ndcIcFRUVGjZsWKtjlJGR4as+P5wM87a0F+SSNGLECG3cuLFV+Dc0NGjw4MGhdaL9g+qSJUs0e/Zs\nrVy5UhdccIGWLl0ac+0uyszM1ObNm7V161b94Ac/kCTt2rVLtbW1qq2t1XnnnedrnJZ9GTFihG67\n7bZWfayrq9Pll1+ujz76SIsXL9bvf/97HThwQDU1NRo3blzY+0VLM2fO1AsvvKCqqiqNGTMmqreb\n9eT7Qm/p04gRI0LvQDmurKzshBdxCxYs0IUXXtjuOGeffbbmzp2rXbt2nXDbkCFDtGfPntDl//73\nv62epDqq14R5ONdff71+9rOf6eOPP5Yk7d+/X88++2zM423YsEGbNm3Sb37zG0nSqlWr9MwzzygY\nDMajXGcMHjxYmzdv1saNG3XzzTf73q7lA9uOniqUJF177bUqLi7Wjh07ZGaqr6/Xc889p7q6OtXX\n18vzPA0aNEjNzc1as2aN73eU7Nu3T+vWrVN9fb0CgYCSkpJ8v1vBhftCb+jTvHnz9Nxzz+nFF1/U\nkSNHVFFRoTvvvFNXXHFF2O22bdumxx57TPv375ck/fvf/9b69evbfMfPvHnztH79em3fvl2NjY1a\nvny57ycpP3pNmId769GPf/xjXXLJJZo5c6ZSUlI0efJk7dixo9W2LX388cdKTk5u9Sx7XG1trW64\n4QatWrVKqampkqRTTz1VK1eu1OLFi/X555/Hca96vuHDh+uf//yn/va3v+m222474fa2etbyupZ9\nnThxoh599FH96Ec/Unp6ukaPHq3HH39ckpSXl6eioiJNnjxZp59+ukpLS3X++ee3Oc4X52lubtZ9\n992noUOHKiMjQ1u3btVDDz0kqffcF1zu0/F5n3jiCd16663KyMjQueeeq8mTJ+v2228Pe1xSU1P1\n7LPPavz48UpOTtbs2bN16aWX6pZbbjmh3rFjx2rVqlWaP3++hgwZouTkZJ122mn60pe+FHYOvzwL\n89RQXl6u73//+9q3b588z9PixYt144036sCBA6FfibKzs/XXv/41dGdF90df3UVve466ujqlpaXp\n/fffV1ZWVofHCxvmVVVVqqqqUkFBgerq6jRx4kQ988wzWrNmjQYNGqRbbrlF99xzj2pqarRixYoO\nF4OuQV/dRW+7t/Xr12v69OkyMxUVFem1117TG2+8EZ/Bo3nry7e+9S178cUXLTc3N/S+z8rKSsvN\nzY3DG2uQKPTVXfS2e7nmmmssNTXVBg4caBdddFHoPfHxEPaVeUu7d+/WhRdeqNLSUo0YMUI1NTXH\nnwyUnp4euoyehb66i972Lr7+AFpXV6d58+bp/vvvV3JycqvbOvN7UNC56Ku76G0vFOmle2Njo82c\nOdPuu+++0HW5ubmhj9tWVFS0+StbTk6OSWLpBktOTk7c+nr0N7mhCd8nlrb7ymO25y/t9TWSsK/M\nzUxXX3218vLytGTJktD1l1xyiUpKSiQd/Ta145+EaumDDz4Ivbc01uX2229P6Pau1PDBBx/Era9H\n7Y1wf6xXIHCK08e0O9Twxb4m+jHbHY5JorfvrL76Efa7XLdt26Y//elPmjBhggoLCyUd/bL1n/70\np7rsssu0evXq0Nuc0HPQV3fR294rbJiff/75rb6YvaVNmzZ1SkHofPTVXfS29+rWnwCdOnVqQrd3\npYbuxoVj2h1q6G66wzFJ9PbxGiMWvt+aGPXAnqdOGhpRincvjr4TItx4DQoEBqmxsSFuc+JEndFX\nHrOJF2sfuvUrcwCAP4Q5ADiAMAcABxDmAOAAwhwAHECYA4ADCHMAcABhDgAOIMwBwAGEOQA4gDAH\nAAcQ5gDgAMIcABxAmAOAAwhzAHAAYQ4ADiDMASRMSkq6PM+LuKSkpCe61G4v7P8ABYDOVFtbo/D/\nter4el7nF9PD8cocABxAmAOAAwhzAHAAYQ4ADiDMAcABhDkAOIAwBwAHEOYA4ADCHAAcQJgDgAMI\ncwBwAGEOAA4gzAHAAYQ5ADiAMAcABxDmAOAAwhwAHECYA4ADCHMAcABhDgAOIMwBwAGEOQA4gDAH\nAAcQ5gDgAMIcABxAmAOAAwhzAHBAxDBftGiRMjMzNX78+NB1y5cv17Bhw1RYWKjCwkJt3LixU4tE\n/NFXN9HX3itimF911VUnNN/zPN18883auXOndu7cqa997WudViA6B311E33tvSKG+ZQpU5SWlnbC\n9WbWKQWha9BXN9HX3ivmc+arVq1Sfn6+rr76ah08eDCeNSGB6Kub/Pa1oqIi7PLJJ590YdWIRr9Y\nNrrhhhv0i1/8QpK0bNkyFRUVafXq1Sest3z58tDPU6dO1dSpU2MqMhopKemqra2JuF5ycpr+938P\ndHo9iRAMBhUMBqPezm9fj1re4uepxxZ0pq7oa1ZWbujnvn1PUt++X2p1+6FD+7RnT7kGDx4cdR1o\nW6x9PYH5UFZWZuPGjYvqNp9Dx50kk8zHkpj6EqG9fY2lr8fHC39s6y0QOCVu9aNtXd9Xs6SkLCsr\nK4vXLvB4bUOs+xrTaZbKysrQz08//XSrv5yj56KvbqKvvUPE0yxXXHGFXnrpJX3yyScaPny47rjj\nDgWDQb311lvyPE8jR47Uww8/3BW1Io7oq5voa+/lHXtZH/+BPS8hf0H3PE+Sn3kTU18ixLsXkY9x\ngwKBQWpsbIjbnDhR1/dVSkrKVmlpUNnZ2V0257E1ebxGwCdAAcABhDkAOIAwBwAHEOYA4ADCHAAc\nQJgDgANi+ji/X6ecMjDiOhdcMF3/8z//6MwyAMB5nRrmhw59HGGNnfrww6WdWQIA9AqdGuZSpFfm\nAzp3egDoJThnDgAOIMwBwAGEOQA4gDAHAAcQ5gDgAMIcABxAmAOAAwhzAHAAYQ4ADiDMAcABhDkA\nOIAwBwAHEOYA4ADCHAAcQJgDgAMIcwBwAGEOAA4gzAHAAYQ5ADiAMAcABxDmAOAAwhwAHECYA4AD\nCHMAcABhDgAOIMwBwAGEOQA4gDAHAAcQ5gDgAMIcABxAmAOAAwhzAHAAYQ4ADiDMAcABhDkAOIAw\nBwAHEOYA4ADCHAAcEDHMFy1apMzMTI0fPz503YEDBzRjxgydeeaZmjlzpg4ePNipRSL+6Kub6Gvv\nFTHMr7rqKm3cuLHVdStWrNCMGTP03nvvafr06VqxYkWnFYjOQV/dRF97MfOhrKzMxo0bF7qcm5tr\nVVVVZmZWWVlpubm5J2wjySSLsLxmo0ZN9FOCb/7mPbpeb9HevsbS1+PjhT+29RYInBL/HUErXd9X\ns6SkLCsrK4vrPvB4bS3WfY3pnHl1dbUyMzMlSZmZmaquro7D0woSjb66ib72Dv06OoDnefI8r51b\nl7f4eeqxBZ0tGAwqGAx2aIzwfZXobdejr26KR18lxX6apbKy0szMKioqOM3SzbW3r7H09fh4nGZJ\nvK7vK6dZukKs+xrTaZZLLrlEJSUlkqSSkhLNmTOn488qSDj66ib62ktESvv58+fb4MGDLRAI2LBh\nw+wPf/iDffrppzZ9+nQbPXq0zZgxw2pqatp8duGVeffQ1r7G2tfj4/HKPPG6vq+8Mu8Kse6rd2zj\nuDt6Xi7S0K9r1Kjr9Z//vN7F80qSp07a9W7H8+K7r5GPcYMCgUFqbGyI25w4Udf3VUpKylZpaVDZ\n2dldNuexNXm8RsAnQAHAAYQ5ADiAMAcABxDmAOAAwhwAHECYA4ADCPM4SUlJD31Uur0lJSU90WX2\nWH6Ob2cc40TNi87lYl87/N0sOKq2tkaR3i9bWxvuOzEQjp/je3S9+B7jRM2LzuViX3llDgAOIMwB\nwAGEOQA4gDAHAAcQ5gDgAMIcABxAmAOAAwhzAHAAYQ4ADiDMAcABhHkP5vf7JdCz0FfEgu9m6cH8\nfr+ExAO/J6GviAWvzAHAAYQ5ADiAMAcABxDmAOAAwhwAHECYA4ADCHMAcABhDgAOIMwBwAGEOQA4\ngDAHAAcQ5gDgAMIcABxAmAOAAwhzAHAAYQ4ADiDMAcABhDkAOIAwBwAHEOYA4ADCHAAcQJgDgAMI\ncwBwAGEOAA4gzAHAAYQ5ADiAMAcAB/TryMbZ2dlKSUlR3759FQgEtGPHjnjVhQSir26ir27rUJh7\nnqdgMKj09PR41YNugL66ib66rcOnWcwsHnWgm6GvbqKv7upQmHuep4suukhnn322Hn300XjVhASj\nr26ir27r0GmWbdu2afDgwdq/f79mzJihMWPGaMqUKS3WWN7i56nHFnS+4LElNpH7KtHbRAiKvron\nGAwqGAx2eBzP4vR71x133KEBAwaoqKjo6MCeJynS0K9r1Kjr9Z//vB6PEqKYV5K8uP7K6W/eRMzZ\nsXm/2Fd/8zYoEBikxsaGmOZsS/fua3zn7b59lZKSslVaGlR2dnZM835Rb+qrX54X25wxn2ZpaGhQ\nbW2tJKm+vl4vvPCCxo8fH+tw6Cboq5voq/tiPs1SXV2tuXPnSpIOHz6s7373u5o5c2bcCkNi0Fc3\n0Vf3xRzmI0eO1FtvvRXPWtAN0Fc30Vf38QlQAHAAYQ4ADiDMAcABhDkAOIAwBwAHEOYA4ADCHAC6\nQEpKujzPi7jEqkPfzQIA8Ke2tkZ+v0IgFrwyBwAHEOYA4ADCHAAcQJgDgAMIcwBwAGEOAA4gzAHA\nAYQ5ADiAMAcABxDmAOAAwhwAHECYA4ADCHMAcABhDgAOIMwBwAGEOQA4gDAHAAcQ5gDgAMIcABxA\nmAOAAwhzAHAAYQ4ADiDMAcABhDkAOIAwBwAHEOYA4ADCHAAcQJgDgAMIcwBwAGEOAA4gzAHAAYQ5\nADiAMAcABxDmAOAAwhwAHECYA4ADCHMAcABhDgAOiDnMN27cqDFjxmj06NG655574lkTEozeuom+\nOs5icPjwYcvJybGysjJrbGy0/Px8e+edd1qtI8kki7C8ZqNGTWx3ni1btkRdW+t5t4SZ29+u+62h\n/f3dEvWcfmvwd4yjmzc+va23QOCUqPcnnO7d19h62/P6apaUlGVlZWVR7U84J87ZXm/payQxvTLf\nsWOHRo0apezsbAUCAc2fP1/r1q3rwFNK24LBYEdHoIYodUVv3Tim3aEG/3pGX6WOHtfe1teWYgrz\nvXv3avjw4aHLw4YN0969e+NWFBKH3rqJvrqvXywbeZ7na72UlIvD3n7kyEEFAvwNtjuJT28P6/PP\n6Wt3Eq/HbENDtfr0obfdUiznZrZv326zZs0KXb7rrrtsxYoVrdbJyck5do6IJdFLTk4OvXVwoa9u\nLtH0tSXPzExROnz4sHJzc7V582YNGTJEX/nKV/TEE0/orLPOinYodDP01k301X0xnWbp16+fHnjg\nAc2aNUtHjhzR1VdfzZ3CEfTWTfTVfTG9MgcAdC8d/kuGnw8i3HjjjRo9erTy8/O1c+fOqMcIBoMa\nOHCgCgsLVVhYqDvvvDN026JFi5SZmanx48e3W2Ok+SONEW5+SSovL9e0adM0duxYjRs3Tr/73e+i\nqsPP9pFqOHTokCZNmqSCggLl5eXp1ltvjelYHEdf6Wtn9FXqeG8T3Ve/Y4SrI959laTY3p1+jJ8P\nIjz33HM2e/ZsMzN79dVXbdKkSVGPsWXLFrv44ovbrOHll1+2N99808aNG9fm7ZHm9zNGuPnNzCor\nK23nzp1mZlZbW2tnnnlmVMfBz/aRajAzq6+vNzOzpqYmmzRpkm3dutV3DS3R16Poa/z7atbx3ia6\nr37HiFRHvPp6XIdemfv5IMKzzz6rBQsWSJImTZqkgwcPqrq6Oqoxjj3ptFnDlClTlJaW1m6Nkeb3\nM0a4+SXp9NNPV0FBgSRpwIABOuuss1RRUeG7Dj/bR6pBkvr37y9Jamxs1JEjR5Senu67hpbo61H0\nNf59lTre20T31e8YkeqIV1+P61CY+/kgQlvr7NmzJ6oxPM/TK6+8ovz8fH3961/XO++806EaW87v\nRzTz7969Wzt37tSkSZNiqqO97f3U0NzcrIKCAmVmZmratGnKy8uLqQb6eiL66n+MjvQ1mv1pT1f2\nNdwYkeqIV1+Pi+ndLC2L9eOLz04tt/Mzxpe//GWVl5erf//+ev755zVnzhy99957vusMN78ffuev\nq6vTt7/9bd1///0aMGBA1HWE295PDX369NFbb72lzz77TLNmzVIwGNTUqVOjqqG969pCX/3VQV+j\n62ukGuI1f0f7GmmMSHXEq6+h8dq9xYehQ4eqvLw8dLm8vFzDhg0Lu86ePXs0dOjQqMZITk4O/Uoy\ne/ZsNTU16cCBAzHV+MX5/fAzf1NTk+bNm6crr7xSc+bMibqOSNtHcwwGDhyob3zjG3r99dejqqG9\n9egrfY1mjI70NZr9aU9X9NXPGH6PQ0f7GhL2jHoETU1NdsYZZ1hZWZl9/vnnEf+gsn379hNO4vsZ\no6qqypqbm83M7F//+pdlZWW1ur2srMzXH1Pamt/PGJHmb25utu9973u2ZMmSNrePVIef7SPVsH//\nfqupqTEzs4aGBpsyZYpt2rTJdw0t0dej6Gvn9NWs471NZF/9jhGujnj29bgOhbmZ2YYNG+zMM8+0\nnJwcu+uuu8zMrLi42IqLi0Pr/PCHP7ScnBybMGGCvfHGG1GP8cADD9jYsWMtPz/fJk+ebNu3bw9t\nO3/+fBs8eLAFAgEbNmyYrV69Our5I40Rbn4zs61bt5rneZafn28FBQVWUFBgGzZs8F2Hn+0j1fD2\n229bYWGh5efn2/jx4+3ee++NqRd+e+JnLPpKX9vap472NtF99TtGuDri3VezGD/ODwDoXvj6MwBw\nAGEOAA4gzAHAAYQ5ADiAMAcABxDmAOAAwhwAHECYA4AD/g/6L1FHiVfykQAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x119d1dd10>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"plt.subplot(1,3,1)\n",
"plt.hist(ka.labels_)\n",
"plt.ylim([0,20])\n",
"plt.title('Hier: X')\n",
"\n",
"plt.subplot(1,3,2)\n",
"plt.hist(km.labels_)\n",
"plt.ylim([0,20])\n",
"plt.title('K-means: X')\n",
"\n",
"plt.subplot(1,3,3)\n",
"plt.hist(km_u.labels_)\n",
"plt.ylim([0,20])\n",
"plt.title('K-means: U*Sig')\n",
"\n",
"fig.tight_layout()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>Comparing the above methods, we see that (objectively), hierarchical based clustering gives more even clusters, and that in our subject opinion, the results are easier to interpret. Given these properties, for a final analysis, we would go with hierarchical clustering for the application of putting students into study groups of size 4. \n",
"</p>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python [Root]",
"language": "python",
"name": "Python [Root]"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.12"
}
},
"nbformat": 4,
"nbformat_minor": 0
}