{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 05 - Logistic Regression\n",
    "by [Alejandro Correa Bahnsen](albahnsen.com/)\n",
    "\n",
    "version 0.2, May 2016\n",
    "\n",
    "## Part of the class [Machine Learning for Security Informatics](https://github.com/albahnsen/ML_SecurityInformatics)\n",
    "\n",
    "\n",
    "This notebook is licensed under a [Creative Commons Attribution-ShareAlike 3.0 Unported License](http://creativecommons.org/licenses/by-sa/3.0/deed.en_US). Special thanks goes to [Kevin Markham](https://github.com/justmarkham)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Review: Predicting a Continuous Response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type\n",
       "id                                                                        \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1\n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6\n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1\n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1\n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import zipfile\n",
    "with zipfile.ZipFile('../datasets/glass.csv.zip', 'r') as z:\n",
    "    f = z.open('glass.csv')\n",
    "    glass = pd.read_csv(f, sep=',', index_col=0)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** Pretend that we want to predict **ri**, and our only feature is **al**. How could we do it using machine learning?\n",
    "\n",
    "**Answer:** We could frame it as a regression problem, and use a linear regression model with **al** as the only feature and **ri** as the response.\n",
    "\n",
    "**Question:** How would we **visualize** this model?\n",
    "\n",
    "**Answer:** Create a scatter plot with **al** on the x-axis and **ri** on the y-axis, and draw the line of best fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f443a6c2ef0>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEPCAYAAABlZDIgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuQXOV55/HvI2l6pqXRjCQySI6xZrgaCBZCBBzKVKkV\nhzJ4HWAd1liOyxcUB+OoYImTIFM4Ixt7KzJrvHYcIivMIthFF1fW5hLiDKGYiUtJhdECinAGmWTN\nCDtZNJMAXhNkI8Kzf/Q5M2d6+j7ndJ/u+X2qutRzrm+/De/T7/WYuyMiIjJfi5qdABERaQ8KKCIi\nEgsFFBERiYUCioiIxEIBRUREYqGAIiIisUg0oJjZkJkdM7PDJfZvNLNXzOyp4HVbwf5FwfaHItsG\nzexHkXMuT/IziIhIdZYkfP17gD8E7itzzHfd/coS+24CxoGegu13uvudMaRPRERikmgNxd0PAC9X\nOMyKbjQ7BXgvcHe154iISPOkoQ/lEjM7ZGaPmNm5ke1fAX4XKDaVf2twzt1m1tuYZIqISDnNDihP\nAmvdfT3wdeABADN7H3DM3Q+Rr41EayR3AacF57wIqOlLRCQFLOm1vMysH3jY3ddVcewPgF8Efgf4\nMPAGkAWWA99y94/Ucm0z00JlIiJ1cPeauxYaUUMprGHM7DBbHXl/MbDI3V9y91vdfa27nwZ8EHg8\nDCZmtiZyifcD3yt3c3dP/WtwcLDpaVA6lUalU+kMX/VKdJSXme0BcsBJZvYCMAhkAHf3XcA1ZnYD\ncAI4DlxbxWW/ZGbrgTeBCeD6BJIuIiI1SjSguPuHKuz/I+CPKhzzV8BfRf7+SJnDRUSkSZrdKS9A\nLpdrdhKqonTGpxXSCEpn3FolnfVKvFO+mczM2/nziYgkwczwlHbKi4jIAqCAIiIisVBAERGRWCig\niIhILBRQREQkFgooIiISCwUUERGJhQKKiIjEQgFFRERioYAiIiKxUEAREZFYKKCIiEgsFFBERCQW\nCigiIhKLRAOKmQ2Z2TEzO1xi/0Yze8XMngpetxXsXxRsfyiybaWZPWpm3zezYTPrTfIziIhIdZKu\nodwDvKfCMd919w3B6wsF+24Cxgu2bQMec/e3A48Dn4knqSIiMh+JBhR3PwC8XOGwog9xMbNTgPcC\ndxfsugq4N3h/L3D1fNIoIiLxSEMfyiVmdsjMHjGzcyPbvwL8LlD4yMWT3f0YgLu/CJzcoHSKiEgZ\nS5p8/yeBte7+mpldATwAnGVm7wOOufshM8tRohYTKPuM3+3bt0+/z+Vybf9MZxGRWo2OjjI6Ojrv\n6yT+THkz6wcedvd1VRz7A+AXgd8BPgy8AWSB5cC33P0jZvYskHP3Y2a2Bhhx93NKXE/PlBcRqVGa\nnylvlO4nWR15fzGwyN1fcvdb3X2tu58GfBB43N0/Ehz6EPCx4P1HgQcTS7mIiFQt0SYvM9sD5ICT\nzOwFYBDIAO7uu4BrzOwG4ARwHLi2isvuAL5pZtcBR4EPJJF2ERGpTeJNXs2kJi8RkdqluclLREQW\nAAUUERGJhQKKiIjEQgFFRERioYAiIiKxUEAREZFYKKCIiEgsFFBERCQWCigiIhILBRQREYmFAoqI\niMRCAUVERGKhgCIiIrFQQBERkVgooIiISCwUUEREJBaJBhQzGzKzY2Z2uMT+jWb2ipk9FbxuC7Z3\nmtkTZva0mT1jZoORcwbN7EeRcy5P8jOIiEh1En0EMHAP8IfAfWWO+a67Xxnd4O4/M7NN7v6amS0G\n/trMvuPuY8Ehd7r7nQmlWURE6pBoDcXdDwAvVzis6GMm3f214G0n+cAXfZZvzY+mFBGRZKWhD+US\nMztkZo+Y2bnhRjNbZGZPAy8Cf+nuByPnbA3OudvMehueYhERmSPpJq9KngTWBk1bVwAPAGcBuPub\nwAVm1gM8YGbnuvs4cBfweXd3M/sCcCewpdQNtm/fPv0+l8uRy+WS+iwiIi1pdHSU0dHReV/H3L3y\nUfO5gVk/8LC7r6vi2OeBC939pYLtnwX+rbDfpNK1zcyT/nwiIu3GzHD3mrsWGtHkZZTo8zCz1ZH3\nF5MPcC+Z2c+FTVlmlgUuA44Ef6+JXOL9wPeSSriIiFQv0SYvM9sD5ICTzOwFYBDIAO7uu4BrzOwG\n4ARwHLg2OPUtwL1mtoh80Nvv7n8e7PuSma0H3gQmgOuT/AySNzU1xcTEBAMDA/T19TU7OSKSQok3\neTWTmrzisXfvfrZs+RSZzACvvz7B0NBdbN58beUTRaQl1dvkpYAiZU1NTdHffzbHj48A64DDZLOb\nOHr0iGoqIm0qzX0o0sImJibIZAbIBxOAdXR09DMxMdG8RIlIKimgSFkDA/lmLghXzznMiRNHGRgY\naF6iRCSVFFCkrL6+PoaG7iKb3URPzway2U0MDd2l5i4RmUN9KFIVjfISWTjUKV+EAoqISO3UKS8i\nIk2lgCIiIrFQQBERkVgooIiISCwUUEREJBYKKCIpNDU1xcGDB5mammp2UkSqpoAikjJ79+6nv/9s\nLrvsk/T3n83evfubnSSRqmgeikiKaDFOSQPNQxFpA1qMU1qZAopIimgxTmlliQYUMxsys2NmdrjE\n/o1m9oqZPRW8bgu2d5rZE2b2tJk9Y2aDkXNWmtmjZvZ9MxsOHxUs0g60GKe0skT7UMzsUuBV4D53\nX1dk/0bg0+5+ZZF9S939NTNbDPw1cKO7j5nZDuBf3f1LZnYLsNLdt5W4v/pQpCVpMU5ppnr7UBJ9\npry7HzCz/gqHFU20u78WvO0kn84wMlwFbAze3wuMAkUDikir6uvrUyCRlpOGPpRLzOyQmT1iZueG\nG81skZk9DbwI/KW7Hwx2nezuxwDc/UXg5MYnWURECiVaQ6nCk8DaoGnrCuAB4CwAd38TuMDMeoAH\nzOxcdx8vco2ybVrbt2+ffp/L5cjlcjElXUSkPYyOjjI6Ojrv6yQ+DyVo8nq4WB9KkWOfBy5095cK\ntn8W+Dd3v9PMngVy7n7MzNYAI+5+TonrqQ9FRKRGaZ6HYpToJzGz1ZH3F5MPcC+Z2c+Fo7fMLAtc\nBhwJDn0I+Fjw/qPAgwmlW0REapBok5eZ7QFywElm9gIwCGQAd/ddwDVmdgNwAjgOXBuc+hbgXjNb\nRD7o7Xf3Pw/27QC+aWbXAUeBDyT5GUREpDpaekVERGZJc5OXiIgsAAooIiISCwUUERGJhQJKi9ID\nmEQkbRRQWpAewCQiaaRRXi2mmgcwaWFBEZkPjfJaICYmJliypJ9SD2BS7UVEmkUBpcU89dQhfvKT\nIxR7ANPU1BRbtnyK48dH+PGPn+T48RG2bPmU+llEpCEUUFrI1NQUN9+8DdgObALOB36Jr3zlD+jr\n69PjY0WkqRRQWshMwPg98kub3U139+mcemo/Bw8epLu7O/L42ClgD6+//rweHysiDaGA0kJmP2+8\nD+jkZz87ytVXb+ayyz7JhRdeypYtH6aj41JgABjkzTedxx57vImpFpGFQqO8WszevfvZsuVTdHT0\n8/rrz/Pmm87rr3+XcMRXV9dGwPjpT0cpNQqslWjEmkjjaZTXArF587UcPXqExx77Bg8+uJ9s9nSi\nfSaLF5/M4sVraYd+FI1YE2ktqqG0sGJzUrq6NmK2qOw8lVZQzXwbEUlGvTWUZj8CWOahr6+PoaG7\n2LJlEx0d/Zw4cZShoZ0ABdvuarlCOByAcPz43JpWq30WkYUi0RqKmQ0B7wOOFXsEsJltJP/ExR8E\nm77l7l8ws1OA+4DVwJvAn7j714JzBoFPAJPBObe6+1+UuH9b11BCxfoZWr3vQTUUkeapt4aSdEC5\nFHgVuK9MQPm0u19ZsH0NsMbdD5lZN/AkcJW7HwkCyk/c/c4q7r8gAkq7ig5ACGtamzdfW/lEEZmX\nVDZ5ufsBM+uvcNicRLv7i8CLwftXzexZ4K3MPFe+5g8qrWfz5mv5lV/55ZauaYksJGkY5XWJmR0y\ns0fM7NzCnWY2AKwHnohs3hqcc7eZ9TYondIEfX19XHTRRS0RTPRIAVnomh1QngTWuvt64OvAA9Gd\nQXPXnwI3ufurwea7gNOCc14EKjZ9iSRNQ5xFGjBsOGjyerhYH0qRY58HLnT3l8xsCfBnwHfc/av1\nXNvMfHBwcPrvXC5HLper41OIlKYBBNLqRkdHGR0dnf77c5/7XPo65WG6yephd39HkX2r3f1Y8P5i\n4JvuPhD8fR/wL+7+2wXnrAn6WDCzm4GL3P1DJe6tTvkma/XRZtU4ePAgl132SX784yent/X0bOCx\nx77BRRdd1MSUidQnlTPlzWwP8DfAWWb2gpl93MyuN7PfDA65xsy+Z2ZPA/8NuDY4713ArwO/bGZP\nm9lTZnZ5cM6XzOywmR0CNgI3J/kZpH4LpRlo9hprEH2kgMhCopnykoh6m4GSqtEkXVPSEGdpJ6ms\nocjCVc+zWZKq0TSiphRdY+3o0SMKJrIgqYYiiai1hpJUx7Y6zEVqpxpKC2vH+QvhOmPZ7CZ6ejaQ\nzW6aXlOs2OdN6mmTeoqlSOMooDRZIzuuGx24ijUDlfq8SXVsq8NcpIHcvW1f+Y+XXpOTk57NrnL4\nOwd3+DvPZlf55ORk7Pfas2efZ7OrvLd3g2ezq3zPnn2x36OSSp83TGNPzwWxpjGp64q0q6DsrLnM\nVR9KEzVq/kJa+hGq+bytOsprvtKePllYUrk4pJQ3uzkmX9DH2RwTFlIvv/xyKp4tUs3n7evrSyRN\nSV03DuGQ40wmnz8aciwtq1z1BegJ/l1V7FVPlaiRL1La5DU+Pu67d+/28fHx2JpjJicnfWxsbE7z\n0fLl7/BMpts7OnrKNq0Vnl/p+vVS89NsjWz2FKkWdTZ5VSqQ/4z8UvFvkn8I1vOR1w/quWEjX2kM\nKFu33uSQdTjLIetbt94478K6sH9k585dQSG1w2GVw/kOGe/o6JlTkE9OTvrtt3+xbP9K3P0vcQWn\ndjA2Nua9vRuCYJJ/9fRc4GNjY81OmixgiQQUnymYv1fPxZv9SltAGR8fD4LJzK9RyPr4+Hjd1yz2\nC7ezs8eXLTsvCCYz2zOZHh8eHp5Vi+nqWuGwtOQv5Hp/QccVNOK4TpoDmGookkb1BpRqhw0/aWZa\n5W6exsbGgLcRnRMBpwTb61N8nsXaoK9i9r06O09j5cqV03NBPvrR3+SnP/0w8POUmqfRzBnvcVwn\nvMamTZ9I5Xpi5ebriLScaqIO+SclvgH8H/I9qs8Ah+uJYI18sUBrKNnsKr/jji/PuVdX14rpGsqV\nV17tMOCwbM5x86mhxPWLO47rTE5OeibTW1BL603lr/8016Jk4SHhJq/+Yq96btjIV9oCirv71q03\nBgX4mdN9KMXUUsCU6ujeuXOXd3au8OXL13tHx3LPZHq9t3eDd3b2OGSC19JIX8s6h6xv2/aZqq5f\nTFx9AvVeJ5pvw8PDDmfMugac7sPDwzWlRWShSTSgtOorjQHFffYor2Lq6QQvFYDCgnX2r/3PO5zq\n0OVwerBt0mHMYaBogVvtKLADBw54Z+eKptRQCvNt27Zb5/QPwVIFFJEKFFBaKKCUk0Qn7dxf+2HT\n22fnNHfV0yQUFuTZ7GkOWe/oWOuQ9Wz2vHmNCitXMyoMcMXyratrhXd0dDusdLjAYaV3dHQX/XzN\naHJSM5eklQJKmwSUJIaRFitsFy3qCoLJyQ5ZX7Lk7XUV/jPXHvHZo8pGvLOzZ179Q+H1w0I3fB8O\ni47W4Erl2+23f9G7ulb4smVneVfXiqKfrxnL0tR7TwUhaYRUBhRgCDhWqgOf/BMXXwGeCl63BdtP\nAR4H/j4YAHBj5JyVwKPA94FhoLfM/WPO5uQlNYy02K/9sOntwIEDdRdSMwX5mENy8ymiEzWLDSIY\nHx8vmW/lCuFmDNut955pWI9NFoa0BpRLgfUVAspDRbavAdYH77uD4HF28PcO4PeC97cAf1Dm/jFm\nceMkNZs8iV+3pWso8RXMswvgMc9P1JwbuGrJtzAvhoeHGz6xsJ5aqOarSCOlMqDk00V/hYDycBXX\neAB4d/D+CLDaZwLPkTLnxZS9jddKTRthQd7VNRBL30mh2QXwZNnAVU2+RX/pd3WtmDO0uNZlaWpV\nT3DQjHpppFYOKP8CHAIeAc4tcswAMAEsC/5+qWD/S2XuHVsGF0qywG+lYBIK0zw+Pj6vtBf77HML\n4B0OWV++fH3NgWvutUZ88eKsd3WtLFqzSaqZqdZaqGoo0kitGlC6gaXB+yuA54rs/9/AVZFthQHl\nX8vcO678nSXJtuyF3E5e7rMXFsA7d+6qK3DN/qW/L6jtnOWZTI/ffvsXKwSyeAvxWn84aGFNaZR6\nA0riz0Mxs37yzVrrqjj2eeBCd3/JzJaQX5zyO+7+1cgxzwI5dz9mZmuAEXc/p8T1fHBwcPrvXC5H\nLpeb1+dJ8tkicV07jc/WKJemqakpnn76aa6+enPZzx7H55rJ4/8F/BpQ+n6Nel5NLdL43UrrGx0d\nZXR0dPrvz33uc3gdz0NpRA1lAHimxL7VkfcXAxORv+8D7ixyzg7gluB9wzvlk2zLjuPazazhlPrF\nPTNjv3ST0rJlb/f8rPZwguXk9GefbxNgsaX986sFnFU2r2upocTZTNmKTZ7SXkhjkxewB/hn4GfA\nC8DHgeuB3wz2/xbwPeBp4G+Adwbb3wX8O/m+lafJDym+PNi3CniM/MivR4EVZe4fe0Yn2Qwy32s3\ns529VCDbuXOXl1pTbPas+kmHcBLiBoeVvmTJMr/lls/MK0CWStf4+HhVM/qjy9eUun+cQXwhN3lK\neqQyoDT7lURAcU+2LXs+127WSKBSgSxfaPf47GG++xyWeiYz4NAZqSVMOkRHW+3w/NIwpZfWrz5d\nI0GtZ2TW+ZXyOjr3pbOzx3fu3FX1Z68niKvjXdJCAaWBAcU9naO8Gl0gheuEfe1rX/Plyy+Y01y1\ne/du7+o6J+j4HnEYdljhM3NWwn8L55eMB8fd76UmS1aTR2NjY8FyMKuC6+SHNhc2aw0PD896Tkwt\neRlnENfQYEkLBZQGB5S0atRIoD179nlHx/KgBnFqUNuYaa7q6Oj2AwcOBM1dHwr+Xev5xSjHPL+2\n1r6gmWupw9uCY3Y4hP0bc+ecdHWtqLoZbOb+cx8XUG4ZF/fKhXt0mHQzayjqb5EkKKC0UUCJuxM6\nrutGr5N/0uPKSM1j+ayCMJPp9eHhYe/qWhsp1CeDcz4bNGetDAJI+EyWTPDvSCSQ7AuOO907OpZH\ngthI0Was0EzH+5mzgkI2e970I49LLeMSLtdSqnAv7OfYuvXG2IJ4LT8I1N8iSVFAaZOAkvREujiu\nOzY2FozKCpuTwhFaPt3s1d19nu/fvz9YhPL0SKEedtK/x/MP+AoD0/1BreSCyHErHM7wTKbHt227\nNSjg74/ct3QzVv7YbwfXmF3DmelX2e35Z8DMBJxoLaRY4V6uv6iRo7zU3yJJUkBpg4BSTSFRTy0j\n7sJncnIy+PVfWPMIH9R1vkOnL1nSHdREosd9PhJIuoPgsMHh9yK1ls9Gai1rfPHibBBQ3uEzS++X\nfurl8PBw0Om/yvPNcVmHM7yzc0VQOwkDUukaSqn8Tks/R1rSIe1JAaUNAkqlQqLeWkYShU/+4VXR\nmsdvRArnA55vAvtjn+lfyXq+qao3+Hudw68HQSRs6vqQwxKf3Ry2yuEch47gmPuDv2c+SyZzznSh\nf/vtXywIdh7URjr9wIEDRR7DnF/Gpbv7/JZaAiUt6ZD2pIDSBgGlXCExnwIk7sJnz559QR9KOKR3\nMggg6xxuCgLGW4JaRiYIIlnPN0FlHd4fqbmsCo45NVJrOTMIGr2RoHKmw2rPd9jPraHccceXI2m6\nP6h9zASdbPY8HxsbC0Z+vcOjI9K6us6efoJmNbW/auamNEI9qysr4Eg1FFDaIKC4ly4k5lvLiGv0\n1+zgFHaYvzUICGFh/9uRWsYngwBzvuf7LMLmpi3BseHosM5IIAmfeX+qz3TOh81qf+zQH2y/wGGV\nd3a+LZikGA4zLhwdNvOwr8nJyaBjf/aItOhor66uFXPW9Qrt3LnLM5kez2ZP80ymu+jclGrysNzz\nWWop+GtdXVmd91INBZQ2CSju1a64W3stI45fqXMD26QvXXpaUKD/p0hgCftGdkcCw7cjwWXY8x35\nYd/JUp9pGtsaCTKFkyLDprMRD0d5dXb2eHf3+oJAEga7NZ5fUv8d04tKdnT0zMrHjo7lBUGy1+GU\nOU94zM/6nzs8upb8DGs3S5euK7kMTZwFv5rGpB4KKG0UUEpJw2qzpQqofEHZExS4v+Azo7fCZq6w\n6WqVzwwN7glqFW8PCvKM55u1VkQCR3hsPnh0da3wbds+M+uxvlu2fMJnmsHCQDLgS5Ys9cWLu2el\ndfHipV44Ii2bPc2XLTvfZ5ruZgeM8fHxoKM/nDMzu29m//79VeVdsWVoMpneeTdplqPOe6mHAsoC\nCCju6WgLLxXYJicn/corr4oEkLAADvtQTnXocLOuIPBkIrWTkUgQGnH4YnDOkuDfMx2yvnXrjdN9\nOF1dZ3pnZ09Q4wgD1rrgHmFzWnQeymRwrejx5zt0Bdf4Yy9c6gU6vbNzRTBM+ucjwShc+v5M7+xc\n4Xfc8eXpfpio6ATIfEA6w6OFO5zuw8PDiRX8aaqhpOG/XamOAsoCCShpUa5wuOOOL3sm0+vLlv2C\nZzLd/qu/+h+9s7PXOzvDIbwjQcG90vOz5zORwj8sqE/1Ymt5dXWtKOgDyfrMaLMDDu8Lzuvx2Uu8\njPnMqLNwSHK02as7KPCjI9cmffbAg96C64bnhysBnDUd9NxnN2F1dvYEQ5kLA9bS6WVfkir401Cz\nLdacVynAKAA1jwKKAkqqFF+aZCyoQeyP1ErCfeM+szjkuM80h83+1b506WmRQnncZ5qofi0IJJ2e\n778JawI3+UwNpyto8jrTZyZQ5l/Llq3z/fv3F/Sv3F9Qo9gXXD9a8yk+L+bAgQMFAWLEZ2plK4P7\nr3TITNdq4ij4q1kloZaCOo5CvViwDPutSvUXaSBBcymgKKCk0uymnLAP4RafWdMrGjDyKxEvWbIm\n2D93tNaSJdmgkN/nM2t+fdhn5qicF6mBRAPWpMP9QUDp8sIZ9IXLqixbtq7I8+bzKyB3dfVHgshu\nL3yuCpzqW7ZsCRbMnNmeybwl8rnyQ5bD4cyhegt+9+oK4VoK6rgK9WIDOcqtIp2mZrqFSgFFASWV\nZi8hv8pn+lYKC/x8wZFvGuoJfr3PHa2VX704HJI84vkaytKgUA8D1od8ZsTY6T7TjLbBYalv2HBx\nUFNY6nC6L1myfFZhGS3Iw1FZy5adV1ATyU+IXLr07KLb5x7/d0G/z0qvpqCstTCvdpWFagvqOAv1\nudcqrPnN7i/SQILmU0BRQEmtPXv2BTWDsH9i2POrC6/yfB/K7KVR8oVJGAQu8HxTVbj45GQQUMKn\nO3YG1+ryfJPZsiBQRJd3iRbiI5GCPl9r6epaUbZQX778Hd7Rscy7us6bVch1d5/nu3fv9uuuC0eZ\nnerFgkt0AmQ1zVr1FObVFMK1FNTFjg0/bz1NZdHPPbfmpxpK2qQyoABDwDHgcIn9G4FXgicyPgXc\nVulcYBD4UeScy8vcP9ZMlvrMXp04OklxxPNNP9+eNfFwpjDJF/iZTHek+WjM87PgV3m+kz0bBI73\neH6C5QbP99GEw5fDJjKPnF/+0b9hmuf2gZRe92t8fNx///d/P5gPU74QrtSUVc8v9ORrKGFwrNy/\nU+qRz9HPXe3DzZo5kGAhS2tAuRRYXyGgPFTLuUFA+e0q7x9T9sp8zBSQ0VpHp3d09BQtMAoLk3AW\n+9yFKMNRWft8Zu5KGKiWRYJK9YFhbppnCvWurgHv7FxRspCL65d1PdcJ1zGrVAiXWkG5WIALj+3u\nntt8Vyo9xebalAtaC2mUVyt9nlQGlHy66K8QUB6u5dwgoHy6ynvHkrkyP8VqHV1dK8qunVWuySQ/\nxLfXly2L9l9Men6Gfdg3stpn5rBE56hk/brrPlF3s1Ol9b7i+mVd73NRyi0bE/1shTWFUn01k5OT\nvnv37jkDDErV6uY+8tl9+fL1C77/o9VGrbVyQPkX4BDwCHBupXODgPJ8cM7dQG+Ze8eYxTIfcQ+J\nDd+HtZdly9ZN12bCR/qOj4/7tm2fme5Ujz4XvpY1sGpNc1y/RKu5znxqRdWeW+1xY2NjwUPLCgda\nFO+jWihasU+oVQNKN7A0eH8F8Fylc4E+wIL3XwCGytzbBwcHp18jIyNx5bfUIakqf5JNJ2lvppjP\niKhazq1tMMHsGmE9C2i2k1YYtTYyMjKrrGzJgFLk2OeBVdWeW8X+WDJbJK0aUUOJHl9tra67e3aN\ncCFTDSXegDIAPFNi3+rI+4uBiUrnAmsi728G9pS5d0zZK9IY9dSI5tOcmMRoqrTX6pqh1Uat1RtQ\nwqajRJjZHiAHnER+CPAgkAkSu8vMfgu4ATgBHAdudvcnSp3r7veY2X3kR3+9CUwA17v7sRL39yQ/\nn0ic9u7dz5YtnyKTGeD11ycYGrqLzZuvrercqakpJiYmGBgYoK+vr6b7zudcqV4r5bOZ4e5W83nt\nXOAqoEirmJqaor//bI4fHwHWAYfJZjdx9OiR1Bc+0n7qDSiLkkiMiNRmYmKCTGaAfDABWEdHRz8T\nExPNS5RIjRRQRFJgYCDfzAWHgy2HOXHiKAMDA81LVIuZmpri4MGDTE1NNTspC5YCirStVipg+vr6\nGBq6i2x2Ez09G8hmNzE0dJeau6q0d+9++vvP5t3v3sLb3nYG3/jGnzQ7SQuS+lCkLc2ng7uZWqnj\nNi1m+p9uAXYAbwOeY+fOr3L99Z9ocupakzrli1BAWZjUwb2wHDx4kHe/ews/+ck/ATPfeWfnRn74\nw+f0nddBnfIiAXVwLyz5/qej5GsmM995JjOg77zBFFCk7aiDe2Hp6+vjq1/9r8BzRL/zN954Qd95\ngy1pdgKkvaShDyDs4N6yZRMdHf2cOHFUHdxtLuwruemmjWQyA7zxxgv6zptAfSgSm7R1hKchuElj\n6TuPhzqRDWzyAAAJx0lEQVTli1BAaRx1hIu0D3XKS1OpI1xEFFBi0kqT6JKwkDrCF/p3LVKKAkoM\nwlm6l132Sfr7z2bv3v3NTlLDLZSZ3vquRUpTH8o8qe9gtnbuFNV3LQuF+lCaRH0Hs/X19XHRRRe1\nZQGr71qkPAWUeVpIfQcLnb5rkfISDShmNmRmx8zscIn9G83sFTN7KnjdVulcM1tpZo+a2ffNbNjM\nepP8DJUslL4D0XctUknSjwC+FHgVuM/d1xXZvxH4tLtfWe25ZrYD+Fd3/5KZ3QKsdPdtJe7fsHko\n7dx3ILPpu5Z2l9qJjWbWDzxcJqD8jrv/arXnmtkRYKO7HzOzNcCou59d4nxNbBQRqVErd8pfYmaH\nzOwRMzu3iuNPdvdjAO7+InBysskTEZFqNHtxyCeBte7+mpldATwAnFXjNcpWQbZv3z79PpfLkcvl\nary8iEh7Gx0dZXR0dN7XaWqTV5FjnwcudPeXSp1rZs8CuUiT14i7n1PiemryEhGpUZqbvCx4zd1h\ntjry/mLyAe6lCuc+BHwseP9R4MHYUioiInVLepTXHiAHnAQcAwaBDODuvsvMfgu4ATgBHAdudvcn\nSp3r7veY2Srgm+Qfz3YU+IC7v1Li/qqhiIjUKLWjvJpJAUVEpHZpbvISEZEFQAFFRERioYAiIiKx\nUEAREZFYKKCIiEgsFFBERCQWCigiIhILBRQREYmFAoqIiMRCAUVERGKhgCIiIrFQQBERkVgooIiI\nSCwUUEREJBYKKCIiEgsFFBERiUWiAcXMhszsmJkdLrF/o5m9YmZPBa/bIvsuN7MjZvacmd0S2T5o\nZj+KnHN5kp9BRESqsyTh698D/CFwX5ljvuvuV0Y3mNki4OvAu4F/Bg6a2YPufiQ45E53vzOJBIuI\nSH0SraG4+wHg5QqHFXvM5MXAP7j7UXc/AewDrqpwjoiINFEa+lAuMbNDZvaImZ0bbHsr8MPIMT8K\ntoW2BufcbWa9DUupiIiUlHSTVyVPAmvd/TUzuwJ4ADirwjl3AZ93dzezLwB3AltKHbx9+/bp97lc\njlwuN980i4i0ldHRUUZHR+d9HXP3+aem3A3M+oGH3X1dFcc+D1xIPqhsd/fLg+3bAHf3HbVc28w8\n6c8nItJuzAx3r7lroRFNXkaJPg8zWx15fzH5APcScBA4w8z6zSwDfBB4KDhuTeQS7we+l1TCRUSk\neok2eZnZHiAHnGRmLwCDQIZ8bWMXcI2Z3QCcAI4D15Lf+e9mthV4lHzQG3L3Z4PLfsnM1gNvAhPA\n9Ul+BhERqU7iTV7NpCYvEZHapbnJS0REFgAFFBERiYUCioiIxEIBRUREYqGAIiIisVBAERGRWCig\niIhILBRQREQkFgooIiISCwUUERGJhQKKiIjEQgFFRERioYAiIiKxUEAREZFYKKCIiEgsEg0oZjZk\nZsfM7HCJ/RvN7BUzeyp43RbZd7mZHTGz58zslsj2lWb2qJl938yGzaw3yc8gIiLVSbqGcg/wngrH\nfNfdNwSvLwCY2SLg68G5vwBsNrOzg+O3AY+5+9uBx4HPJJP0xhkdHW12EqqidManFdIISmfcWiWd\n9Uo0oLj7AeDlCocVeyrYxcA/uPtRdz8B7AOuCvZdBdwbvL8XuDqOtDZTq/xHpnTGpxXSCEpn3Fol\nnfVKQx/KJWZ2yMweMbNzg21vBX4YOeZHwTaA1e5+DMDdXwROblxSRUSklCVNvv+TwFp3f83MrgAe\nAM6q8Rp6aLyISAqYe7LlsZn1Aw+7+7oqjn0euJB8UNnu7pcH27cB7u47zOxZIOfux8xsDTDi7ueU\nuJ6CjYhIHdy9WHdEWY2ooRjF+0kws+nmKzO7mHyAe8nMDgJnBMHo/wIfBDYHpz0EfAzYAXwUeLDU\njevJEBERqU+iAcXM9gA54CQzewEYBDLkaxu7gGvM7AbgBHAcuJb8zn83s63Ao+T7eYbc/dngsjuA\nb5rZdcBR4ANJfgYREalO4k1eIiKyMKRhlNe8lZoEWXDM18zsH4IRZevTlsZykzwbnM6yk1GDY5qa\nl0Ea6p4028A0nmJmj5vZ35vZM2Z2Y4njmv3fZsV0piQ/O83sCTN7OkjnYInjmp2fFdOZhvwM0rEo\nuP9DJfbXlpfu3tIv8kHxH4F+oAM4BJxdcMwVwCPB+3cCf5vCNG4EHkpBfl4KrAcOl9jf1LysIZ1N\nz09gDbA+eN8NfD9t/23WkM6m52eQjqXBv4uBvwUuTlt+VpnOtOTnzcD/LJaWevKyHWoo5SZBhq4C\n7gNw9yeAXjNbnbI0QonBC43klSejNjsvCe5d76TZhnH3F939UPD+VeBZZuZThZqen1WmE9Lx3+dr\nwdtO8n3AhW32Tc/P4N6V0glNzk8zOwV4L3B3iUNqzst2CCjlJkGWOuafihyTpGrSCMUneaZNs/Oy\nFqnJTzMbIF+jeqJgV6rys0w6IQX5GTTRPA28CPylux8sOCQV+VlFOqH5+fkV4HcpPZev5rxsh4DS\nLsJJnuvJr2P2QJPT0+pSk59m1g38KXBTUANIpQrpTEV+uvub7n4BcArwzmb/UCilinQ2NT/N7D8A\nx4KaacmpHbVqh4DyT8DayN+nBNsKj3lbhWOSVDGN7v5qWE129+8AHWa2qnFJrFqz87IqaclPM1tC\nvpD+H+5ebM5UKvKzUjrTkp+R9Pw/YAS4vGBXKvIzVCqdKcjPdwFXmtkPgL3AJjO7r+CYmvOyHQLK\n9CRIM8uQnwRZOGLhIeAjAGb2S8ArHkyoTEsao22TFpnk2cA0zkoOpX+xNDsvo8pOmo28b2Z+/ndg\n3N2/WmJ/WvKzbDrTkJ9m9nMWPK7CzLLAZcCRgsOanp/VpLPZ+enut7r7Wnc/jXx59Li7f6TgsJrz\nstlrec2bl5gEaWbXE0ygdPc/N7P3mtk/Av8GfDxtaaTEJM9GswqTUZudl9WmkxTkp5m9C/h14Jmg\nPd2BW8mP9ktNflaTTlKQn8BbgHst/3iLRcD+IP9S8/96tekkHfk5x3zzUhMbRUQkFu3Q5CUiIimg\ngCIiIrFQQBERkVgooIiISCwUUEREJBYKKCIiEgsFFJEmMrPnU7oigkjNFFBEmksTwaRtKKCINIiZ\nfdvMDgYPXfqNcHNTEyUSo5ZfekWkhXzc3V8xsy7goJl9q9kJEomTAopI4/xnM7s6eH8KcCZq8pI2\nooAi0gBmthH4ZeCd7v4zMxsBupqcLJFYqQ9FpDF6gZeDYHI28EvBdvWhSNtQQBFpjL8g/xClvwf+\nC/A3wXY1eUnb0PL1IiISC9VQREQkFgooIiISCwUUERGJhQKKiIjEQgFFRERioYAiIiKxUEAREZFY\nKKCIiEgs/j9JKCk3iDdoEgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f443a6c24a8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot using Pandas\n",
    "glass.plot(kind='scatter', x='al', y='ri')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f44380e6ef0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEPCAYAAABlZDIgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuQXOV55/HvI2l6pqXRjCQySI6xZrgaCBZCBBzKVKkV\nhzJ4HWAd1liOyxcUB+OoYImTIFM4Ixt7KzJrvHYcIivMIthFF1fW5hLiDKGYiUtJhdECinAGmWTN\nCDtZNJMAXhNkI8Kzf/Q5M2d6+j7ndJ/u+X2qutRzrm+/De/T7/WYuyMiIjJfi5qdABERaQ8KKCIi\nEgsFFBERiYUCioiIxEIBRUREYqGAIiIisUg0oJjZkJkdM7PDJfZvNLNXzOyp4HVbwf5FwfaHItsG\nzexHkXMuT/IziIhIdZYkfP17gD8E7itzzHfd/coS+24CxoGegu13uvudMaRPRERikmgNxd0PAC9X\nOMyKbjQ7BXgvcHe154iISPOkoQ/lEjM7ZGaPmNm5ke1fAX4XKDaVf2twzt1m1tuYZIqISDnNDihP\nAmvdfT3wdeABADN7H3DM3Q+Rr41EayR3AacF57wIqOlLRCQFLOm1vMysH3jY3ddVcewPgF8Efgf4\nMPAGkAWWA99y94/Ucm0z00JlIiJ1cPeauxYaUUMprGHM7DBbHXl/MbDI3V9y91vdfa27nwZ8EHg8\nDCZmtiZyifcD3yt3c3dP/WtwcLDpaVA6lUalU+kMX/VKdJSXme0BcsBJZvYCMAhkAHf3XcA1ZnYD\ncAI4DlxbxWW/ZGbrgTeBCeD6BJIuIiI1SjSguPuHKuz/I+CPKhzzV8BfRf7+SJnDRUSkSZrdKS9A\nLpdrdhKqonTGpxXSCEpn3FolnfVKvFO+mczM2/nziYgkwczwlHbKi4jIAqCAIiIisVBAERGRWCig\niIhILBRQREQkFgooIiISCwUUERGJhQKKiIjEQgFFRERioYAiIiKxUEAREZFYKKCIiEgsFFBERCQW\nCigiIhKLRAOKmQ2Z2TEzO1xi/0Yze8XMngpetxXsXxRsfyiybaWZPWpm3zezYTPrTfIziIhIdZKu\nodwDvKfCMd919w3B6wsF+24Cxgu2bQMec/e3A48Dn4knqSIiMh+JBhR3PwC8XOGwog9xMbNTgPcC\ndxfsugq4N3h/L3D1fNIoIiLxSEMfyiVmdsjMHjGzcyPbvwL8LlD4yMWT3f0YgLu/CJzcoHSKiEgZ\nS5p8/yeBte7+mpldATwAnGVm7wOOufshM8tRohYTKPuM3+3bt0+/z+Vybf9MZxGRWo2OjjI6Ojrv\n6yT+THkz6wcedvd1VRz7A+AXgd8BPgy8AWSB5cC33P0jZvYskHP3Y2a2Bhhx93NKXE/PlBcRqVGa\nnylvlO4nWR15fzGwyN1fcvdb3X2tu58GfBB43N0/Ehz6EPCx4P1HgQcTS7mIiFQt0SYvM9sD5ICT\nzOwFYBDIAO7uu4BrzOwG4ARwHLi2isvuAL5pZtcBR4EPJJF2ERGpTeJNXs2kJi8RkdqluclLREQW\nAAUUERGJhQKKiIjEQgFFRERioYAiIiKxUEAREZFYKKCIiEgsFFBERCQWCigiIhILBRQREYmFAoqI\niMRCAUVERGKhgCIiIrFQQBERkVgooIiISCwUUEREJBaJBhQzGzKzY2Z2uMT+jWb2ipk9FbxuC7Z3\nmtkTZva0mT1jZoORcwbN7EeRcy5P8jOIiEh1En0EMHAP8IfAfWWO+a67Xxnd4O4/M7NN7v6amS0G\n/trMvuPuY8Ehd7r7nQmlWURE6pBoDcXdDwAvVzis6GMm3f214G0n+cAXfZZvzY+mFBGRZKWhD+US\nMztkZo+Y2bnhRjNbZGZPAy8Cf+nuByPnbA3OudvMehueYhERmSPpJq9KngTWBk1bVwAPAGcBuPub\nwAVm1gM8YGbnuvs4cBfweXd3M/sCcCewpdQNtm/fPv0+l8uRy+WS+iwiIi1pdHSU0dHReV/H3L3y\nUfO5gVk/8LC7r6vi2OeBC939pYLtnwX+rbDfpNK1zcyT/nwiIu3GzHD3mrsWGtHkZZTo8zCz1ZH3\nF5MPcC+Z2c+FTVlmlgUuA44Ef6+JXOL9wPeSSriIiFQv0SYvM9sD5ICTzOwFYBDIAO7uu4BrzOwG\n4ARwHLg2OPUtwL1mtoh80Nvv7n8e7PuSma0H3gQmgOuT/AySNzU1xcTEBAMDA/T19TU7OSKSQok3\neTWTmrzisXfvfrZs+RSZzACvvz7B0NBdbN58beUTRaQl1dvkpYAiZU1NTdHffzbHj48A64DDZLOb\nOHr0iGoqIm0qzX0o0sImJibIZAbIBxOAdXR09DMxMdG8RIlIKimgSFkDA/lmLghXzznMiRNHGRgY\naF6iRCSVFFCkrL6+PoaG7iKb3URPzway2U0MDd2l5i4RmUN9KFIVjfISWTjUKV+EAoqISO3UKS8i\nIk2lgCIiIrFQQBERkVgooIiISCwUUEREJBYKKCIpNDU1xcGDB5mammp2UkSqpoAikjJ79+6nv/9s\nLrvsk/T3n83evfubnSSRqmgeikiKaDFOSQPNQxFpA1qMU1qZAopIimgxTmlliQYUMxsys2NmdrjE\n/o1m9oqZPRW8bgu2d5rZE2b2tJk9Y2aDkXNWmtmjZvZ9MxsOHxUs0g60GKe0skT7UMzsUuBV4D53\nX1dk/0bg0+5+ZZF9S939NTNbDPw1cKO7j5nZDuBf3f1LZnYLsNLdt5W4v/pQpCVpMU5ppnr7UBJ9\npry7HzCz/gqHFU20u78WvO0kn84wMlwFbAze3wuMAkUDikir6uvrUyCRlpOGPpRLzOyQmT1iZueG\nG81skZk9DbwI/KW7Hwx2nezuxwDc/UXg5MYnWURECiVaQ6nCk8DaoGnrCuAB4CwAd38TuMDMeoAH\nzOxcdx8vco2ybVrbt2+ffp/L5cjlcjElXUSkPYyOjjI6Ojrv6yQ+DyVo8nq4WB9KkWOfBy5095cK\ntn8W+Dd3v9PMngVy7n7MzNYAI+5+TonrqQ9FRKRGaZ6HYpToJzGz1ZH3F5MPcC+Z2c+Fo7fMLAtc\nBhwJDn0I+Fjw/qPAgwmlW0REapBok5eZ7QFywElm9gIwCGQAd/ddwDVmdgNwAjgOXBuc+hbgXjNb\nRD7o7Xf3Pw/27QC+aWbXAUeBDyT5GUREpDpaekVERGZJc5OXiIgsAAooIiISCwUUERGJhQJKi9ID\nmEQkbRRQWpAewCQiaaRRXi2mmgcwaWFBEZkPjfJaICYmJliypJ9SD2BS7UVEmkUBpcU89dQhfvKT\nIxR7ANPU1BRbtnyK48dH+PGPn+T48RG2bPmU+llEpCEUUFrI1NQUN9+8DdgObALOB36Jr3zlD+jr\n69PjY0WkqRRQWshMwPg98kub3U139+mcemo/Bw8epLu7O/L42ClgD6+//rweHysiDaGA0kJmP2+8\nD+jkZz87ytVXb+ayyz7JhRdeypYtH6aj41JgABjkzTedxx57vImpFpGFQqO8WszevfvZsuVTdHT0\n8/rrz/Pmm87rr3+XcMRXV9dGwPjpT0cpNQqslWjEmkjjaZTXArF587UcPXqExx77Bg8+uJ9s9nSi\nfSaLF5/M4sVraYd+FI1YE2ktqqG0sGJzUrq6NmK2qOw8lVZQzXwbEUlGvTWUZj8CWOahr6+PoaG7\n2LJlEx0d/Zw4cZShoZ0ABdvuarlCOByAcPz43JpWq30WkYUi0RqKmQ0B7wOOFXsEsJltJP/ExR8E\nm77l7l8ws1OA+4DVwJvAn7j714JzBoFPAJPBObe6+1+UuH9b11BCxfoZWr3vQTUUkeapt4aSdEC5\nFHgVuK9MQPm0u19ZsH0NsMbdD5lZN/AkcJW7HwkCyk/c/c4q7r8gAkq7ig5ACGtamzdfW/lEEZmX\nVDZ5ufsBM+uvcNicRLv7i8CLwftXzexZ4K3MPFe+5g8qrWfz5mv5lV/55ZauaYksJGkY5XWJmR0y\ns0fM7NzCnWY2AKwHnohs3hqcc7eZ9TYondIEfX19XHTRRS0RTPRIAVnomh1QngTWuvt64OvAA9Gd\nQXPXnwI3ufurwea7gNOCc14EKjZ9iSRNQ5xFGjBsOGjyerhYH0qRY58HLnT3l8xsCfBnwHfc/av1\nXNvMfHBwcPrvXC5HLper41OIlKYBBNLqRkdHGR0dnf77c5/7XPo65WG6yephd39HkX2r3f1Y8P5i\n4JvuPhD8fR/wL+7+2wXnrAn6WDCzm4GL3P1DJe6tTvkma/XRZtU4ePAgl132SX784yent/X0bOCx\nx77BRRdd1MSUidQnlTPlzWwP8DfAWWb2gpl93MyuN7PfDA65xsy+Z2ZPA/8NuDY4713ArwO/bGZP\nm9lTZnZ5cM6XzOywmR0CNgI3J/kZpH4LpRlo9hprEH2kgMhCopnykoh6m4GSqtEkXVPSEGdpJ6ms\nocjCVc+zWZKq0TSiphRdY+3o0SMKJrIgqYYiiai1hpJUx7Y6zEVqpxpKC2vH+QvhOmPZ7CZ6ejaQ\nzW6aXlOs2OdN6mmTeoqlSOMooDRZIzuuGx24ijUDlfq8SXVsq8NcpIHcvW1f+Y+XXpOTk57NrnL4\nOwd3+DvPZlf55ORk7Pfas2efZ7OrvLd3g2ezq3zPnn2x36OSSp83TGNPzwWxpjGp64q0q6DsrLnM\nVR9KEzVq/kJa+hGq+bytOsprvtKePllYUrk4pJQ3uzkmX9DH2RwTFlIvv/xyKp4tUs3n7evrSyRN\nSV03DuGQ40wmnz8aciwtq1z1BegJ/l1V7FVPlaiRL1La5DU+Pu67d+/28fHx2JpjJicnfWxsbE7z\n0fLl7/BMpts7OnrKNq0Vnl/p+vVS89NsjWz2FKkWdTZ5VSqQ/4z8UvFvkn8I1vOR1w/quWEjX2kM\nKFu33uSQdTjLIetbt94478K6sH9k585dQSG1w2GVw/kOGe/o6JlTkE9OTvrtt3+xbP9K3P0vcQWn\ndjA2Nua9vRuCYJJ/9fRc4GNjY81OmixgiQQUnymYv1fPxZv9SltAGR8fD4LJzK9RyPr4+Hjd1yz2\nC7ezs8eXLTsvCCYz2zOZHh8eHp5Vi+nqWuGwtOQv5Hp/QccVNOK4TpoDmGookkb1BpRqhw0/aWZa\n5W6exsbGgLcRnRMBpwTb61N8nsXaoK9i9r06O09j5cqV03NBPvrR3+SnP/0w8POUmqfRzBnvcVwn\nvMamTZ9I5Xpi5ebriLScaqIO+SclvgH8H/I9qs8Ah+uJYI18sUBrKNnsKr/jji/PuVdX14rpGsqV\nV17tMOCwbM5x86mhxPWLO47rTE5OeibTW1BL603lr/8016Jk4SHhJq/+Yq96btjIV9oCirv71q03\nBgX4mdN9KMXUUsCU6ujeuXOXd3au8OXL13tHx3LPZHq9t3eDd3b2OGSC19JIX8s6h6xv2/aZqq5f\nTFx9AvVeJ5pvw8PDDmfMugac7sPDwzWlRWShSTSgtOorjQHFffYor2Lq6QQvFYDCgnX2r/3PO5zq\n0OVwerBt0mHMYaBogVvtKLADBw54Z+eKptRQCvNt27Zb5/QPwVIFFJEKFFBaKKCUk0Qn7dxf+2HT\n22fnNHfV0yQUFuTZ7GkOWe/oWOuQ9Wz2vHmNCitXMyoMcMXyratrhXd0dDusdLjAYaV3dHQX/XzN\naHJSM5eklQJKmwSUJIaRFitsFy3qCoLJyQ5ZX7Lk7XUV/jPXHvHZo8pGvLOzZ179Q+H1w0I3fB8O\ni47W4Erl2+23f9G7ulb4smVneVfXiqKfrxnL0tR7TwUhaYRUBhRgCDhWqgOf/BMXXwGeCl63BdtP\nAR4H/j4YAHBj5JyVwKPA94FhoLfM/WPO5uQlNYy02K/9sOntwIEDdRdSMwX5mENy8ymiEzWLDSIY\nHx8vmW/lCuFmDNut955pWI9NFoa0BpRLgfUVAspDRbavAdYH77uD4HF28PcO4PeC97cAf1Dm/jFm\nceMkNZs8iV+3pWso8RXMswvgMc9P1JwbuGrJtzAvhoeHGz6xsJ5aqOarSCOlMqDk00V/hYDycBXX\neAB4d/D+CLDaZwLPkTLnxZS9jddKTRthQd7VNRBL30mh2QXwZNnAVU2+RX/pd3WtmDO0uNZlaWpV\nT3DQjHpppFYOKP8CHAIeAc4tcswAMAEsC/5+qWD/S2XuHVsGF0qywG+lYBIK0zw+Pj6vtBf77HML\n4B0OWV++fH3NgWvutUZ88eKsd3WtLFqzSaqZqdZaqGoo0kitGlC6gaXB+yuA54rs/9/AVZFthQHl\nX8vcO678nSXJtuyF3E5e7rMXFsA7d+6qK3DN/qW/L6jtnOWZTI/ffvsXKwSyeAvxWn84aGFNaZR6\nA0riz0Mxs37yzVrrqjj2eeBCd3/JzJaQX5zyO+7+1cgxzwI5dz9mZmuAEXc/p8T1fHBwcPrvXC5H\nLpeb1+dJ8tkicV07jc/WKJemqakpnn76aa6+enPZzx7H55rJ4/8F/BpQ+n6Nel5NLdL43UrrGx0d\nZXR0dPrvz33uc3gdz0NpRA1lAHimxL7VkfcXAxORv+8D7ixyzg7gluB9wzvlk2zLjuPazazhlPrF\nPTNjv3ST0rJlb/f8rPZwguXk9GefbxNgsaX986sFnFU2r2upocTZTNmKTZ7SXkhjkxewB/hn4GfA\nC8DHgeuB3wz2/xbwPeBp4G+Adwbb3wX8O/m+lafJDym+PNi3CniM/MivR4EVZe4fe0Yn2Qwy32s3\ns529VCDbuXOXl1pTbPas+kmHcBLiBoeVvmTJMr/lls/MK0CWStf4+HhVM/qjy9eUun+cQXwhN3lK\neqQyoDT7lURAcU+2LXs+127WSKBSgSxfaPf47GG++xyWeiYz4NAZqSVMOkRHW+3w/NIwpZfWrz5d\nI0GtZ2TW+ZXyOjr3pbOzx3fu3FX1Z68niKvjXdJCAaWBAcU9naO8Gl0gheuEfe1rX/Plyy+Y01y1\ne/du7+o6J+j4HnEYdljhM3NWwn8L55eMB8fd76UmS1aTR2NjY8FyMKuC6+SHNhc2aw0PD896Tkwt\neRlnENfQYEkLBZQGB5S0atRIoD179nlHx/KgBnFqUNuYaa7q6Oj2AwcOBM1dHwr+Xev5xSjHPL+2\n1r6gmWupw9uCY3Y4hP0bc+ecdHWtqLoZbOb+cx8XUG4ZF/fKhXt0mHQzayjqb5EkKKC0UUCJuxM6\nrutGr5N/0uPKSM1j+ayCMJPp9eHhYe/qWhsp1CeDcz4bNGetDAJI+EyWTPDvSCSQ7AuOO907OpZH\ngthI0Was0EzH+5mzgkI2e970I49LLeMSLtdSqnAv7OfYuvXG2IJ4LT8I1N8iSVFAaZOAkvREujiu\nOzY2FozKCpuTwhFaPt3s1d19nu/fvz9YhPL0SKEedtK/x/MP+AoD0/1BreSCyHErHM7wTKbHt227\nNSjg74/ct3QzVv7YbwfXmF3DmelX2e35Z8DMBJxoLaRY4V6uv6iRo7zU3yJJUkBpg4BSTSFRTy0j\n7sJncnIy+PVfWPMIH9R1vkOnL1nSHdREosd9PhJIuoPgsMHh9yK1ls9Gai1rfPHibBBQ3uEzS++X\nfurl8PBw0Om/yvPNcVmHM7yzc0VQOwkDUukaSqn8Tks/R1rSIe1JAaUNAkqlQqLeWkYShU/+4VXR\nmsdvRArnA55vAvtjn+lfyXq+qao3+Hudw68HQSRs6vqQwxKf3Ry2yuEch47gmPuDv2c+SyZzznSh\nf/vtXywIdh7URjr9wIEDRR7DnF/Gpbv7/JZaAiUt6ZD2pIDSBgGlXCExnwIk7sJnz559QR9KOKR3\nMggg6xxuCgLGW4JaRiYIIlnPN0FlHd4fqbmsCo45NVJrOTMIGr2RoHKmw2rPd9jPraHccceXI2m6\nP6h9zASdbPY8HxsbC0Z+vcOjI9K6us6efoJmNbW/auamNEI9qysr4Eg1FFDaIKC4ly4k5lvLiGv0\n1+zgFHaYvzUICGFh/9uRWsYngwBzvuf7LMLmpi3BseHosM5IIAmfeX+qz3TOh81qf+zQH2y/wGGV\nd3a+LZikGA4zLhwdNvOwr8nJyaBjf/aItOhor66uFXPW9Qrt3LnLM5kez2ZP80ymu+jclGrysNzz\nWWop+GtdXVmd91INBZQ2CSju1a64W3stI45fqXMD26QvXXpaUKD/p0hgCftGdkcCw7cjwWXY8x35\nYd/JUp9pGtsaCTKFkyLDprMRD0d5dXb2eHf3+oJAEga7NZ5fUv8d04tKdnT0zMrHjo7lBUGy1+GU\nOU94zM/6nzs8upb8DGs3S5euK7kMTZwFv5rGpB4KKG0UUEpJw2qzpQqofEHZExS4v+Azo7fCZq6w\n6WqVzwwN7glqFW8PCvKM55u1VkQCR3hsPnh0da3wbds+M+uxvlu2fMJnmsHCQDLgS5Ys9cWLu2el\ndfHipV44Ii2bPc2XLTvfZ5ruZgeM8fHxoKM/nDMzu29m//79VeVdsWVoMpneeTdplqPOe6mHAsoC\nCCju6WgLLxXYJicn/corr4oEkLAADvtQTnXocLOuIPBkIrWTkUgQGnH4YnDOkuDfMx2yvnXrjdN9\nOF1dZ3pnZ09Q4wgD1rrgHmFzWnQeymRwrejx5zt0Bdf4Yy9c6gU6vbNzRTBM+ucjwShc+v5M7+xc\n4Xfc8eXpfpio6ATIfEA6w6OFO5zuw8PDiRX8aaqhpOG/XamOAsoCCShpUa5wuOOOL3sm0+vLlv2C\nZzLd/qu/+h+9s7PXOzvDIbwjQcG90vOz5zORwj8sqE/1Ymt5dXWtKOgDyfrMaLMDDu8Lzuvx2Uu8\njPnMqLNwSHK02as7KPCjI9cmffbAg96C64bnhysBnDUd9NxnN2F1dvYEQ5kLA9bS6WVfkir401Cz\nLdacVynAKAA1jwKKAkqqFF+aZCyoQeyP1ErCfeM+szjkuM80h83+1b506WmRQnncZ5qofi0IJJ2e\n778JawI3+UwNpyto8jrTZyZQ5l/Llq3z/fv3F/Sv3F9Qo9gXXD9a8yk+L+bAgQMFAWLEZ2plK4P7\nr3TITNdq4ij4q1kloZaCOo5CvViwDPutSvUXaSBBcymgKKCk0uymnLAP4RafWdMrGjDyKxEvWbIm\n2D93tNaSJdmgkN/nM2t+fdhn5qicF6mBRAPWpMP9QUDp8sIZ9IXLqixbtq7I8+bzKyB3dfVHgshu\nL3yuCpzqW7ZsCRbMnNmeybwl8rnyQ5bD4cyhegt+9+oK4VoK6rgK9WIDOcqtIp2mZrqFSgFFASWV\nZi8hv8pn+lYKC/x8wZFvGuoJfr3PHa2VX704HJI84vkaytKgUA8D1od8ZsTY6T7TjLbBYalv2HBx\nUFNY6nC6L1myfFZhGS3Iw1FZy5adV1ATyU+IXLr07KLb5x7/d0G/z0qvpqCstTCvdpWFagvqOAv1\nudcqrPnN7i/SQILmU0BRQEmtPXv2BTWDsH9i2POrC6/yfB/K7KVR8oVJGAQu8HxTVbj45GQQUMKn\nO3YG1+ryfJPZsiBQRJd3iRbiI5GCPl9r6epaUbZQX778Hd7Rscy7us6bVch1d5/nu3fv9uuuC0eZ\nnerFgkt0AmQ1zVr1FObVFMK1FNTFjg0/bz1NZdHPPbfmpxpK2qQyoABDwDHgcIn9G4FXgicyPgXc\nVulcYBD4UeScy8vcP9ZMlvrMXp04OklxxPNNP9+eNfFwpjDJF/iZTHek+WjM87PgV3m+kz0bBI73\neH6C5QbP99GEw5fDJjKPnF/+0b9hmuf2gZRe92t8fNx///d/P5gPU74QrtSUVc8v9ORrKGFwrNy/\nU+qRz9HPXe3DzZo5kGAhS2tAuRRYXyGgPFTLuUFA+e0q7x9T9sp8zBSQ0VpHp3d09BQtMAoLk3AW\n+9yFKMNRWft8Zu5KGKiWRYJK9YFhbppnCvWurgHv7FxRspCL65d1PdcJ1zGrVAiXWkG5WIALj+3u\nntt8Vyo9xebalAtaC2mUVyt9nlQGlHy66K8QUB6u5dwgoHy6ynvHkrkyP8VqHV1dK8qunVWuySQ/\nxLfXly2L9l9Men6Gfdg3stpn5rBE56hk/brrPlF3s1Ol9b7i+mVd73NRyi0bE/1shTWFUn01k5OT\nvnv37jkDDErV6uY+8tl9+fL1C77/o9VGrbVyQPkX4BDwCHBupXODgPJ8cM7dQG+Ze8eYxTIfcQ+J\nDd+HtZdly9ZN12bCR/qOj4/7tm2fme5Ujz4XvpY1sGpNc1y/RKu5znxqRdWeW+1xY2NjwUPLCgda\nFO+jWihasU+oVQNKN7A0eH8F8Fylc4E+wIL3XwCGytzbBwcHp18jIyNx5bfUIakqf5JNJ2lvppjP\niKhazq1tMMHsGmE9C2i2k1YYtTYyMjKrrGzJgFLk2OeBVdWeW8X+WDJbJK0aUUOJHl9tra67e3aN\ncCFTDSXegDIAPFNi3+rI+4uBiUrnAmsi728G9pS5d0zZK9IY9dSI5tOcmMRoqrTX6pqh1Uat1RtQ\nwqajRJjZHiAHnER+CPAgkAkSu8vMfgu4ATgBHAdudvcnSp3r7veY2X3kR3+9CUwA17v7sRL39yQ/\nn0ic9u7dz5YtnyKTGeD11ycYGrqLzZuvrercqakpJiYmGBgYoK+vr6b7zudcqV4r5bOZ4e5W83nt\nXOAqoEirmJqaor//bI4fHwHWAYfJZjdx9OiR1Bc+0n7qDSiLkkiMiNRmYmKCTGaAfDABWEdHRz8T\nExPNS5RIjRRQRFJgYCDfzAWHgy2HOXHiKAMDA81LVIuZmpri4MGDTE1NNTspC5YCirStVipg+vr6\nGBq6i2x2Ez09G8hmNzE0dJeau6q0d+9++vvP5t3v3sLb3nYG3/jGnzQ7SQuS+lCkLc2ng7uZWqnj\nNi1m+p9uAXYAbwOeY+fOr3L99Z9ocupakzrli1BAWZjUwb2wHDx4kHe/ews/+ck/ATPfeWfnRn74\nw+f0nddBnfIiAXVwLyz5/qej5GsmM995JjOg77zBFFCk7aiDe2Hp6+vjq1/9r8BzRL/zN954Qd95\ngy1pdgKkvaShDyDs4N6yZRMdHf2cOHFUHdxtLuwruemmjWQyA7zxxgv6zptAfSgSm7R1hKchuElj\n6TuPhzqRDWzyAAAJx0lEQVTli1BAaRx1hIu0D3XKS1OpI1xEFFBi0kqT6JKwkDrCF/p3LVKKAkoM\nwlm6l132Sfr7z2bv3v3NTlLDLZSZ3vquRUpTH8o8qe9gtnbuFNV3LQuF+lCaRH0Hs/X19XHRRRe1\nZQGr71qkPAWUeVpIfQcLnb5rkfISDShmNmRmx8zscIn9G83sFTN7KnjdVulcM1tpZo+a2ffNbNjM\nepP8DJUslL4D0XctUknSjwC+FHgVuM/d1xXZvxH4tLtfWe25ZrYD+Fd3/5KZ3QKsdPdtJe7fsHko\n7dx3ILPpu5Z2l9qJjWbWDzxcJqD8jrv/arXnmtkRYKO7HzOzNcCou59d4nxNbBQRqVErd8pfYmaH\nzOwRMzu3iuNPdvdjAO7+InBysskTEZFqNHtxyCeBte7+mpldATwAnFXjNcpWQbZv3z79PpfLkcvl\nary8iEh7Gx0dZXR0dN7XaWqTV5FjnwcudPeXSp1rZs8CuUiT14i7n1PiemryEhGpUZqbvCx4zd1h\ntjry/mLyAe6lCuc+BHwseP9R4MHYUioiInVLepTXHiAHnAQcAwaBDODuvsvMfgu4ATgBHAdudvcn\nSp3r7veY2Srgm+Qfz3YU+IC7v1Li/qqhiIjUKLWjvJpJAUVEpHZpbvISEZEFQAFFRERioYAiIiKx\nUEAREZFYKKCIiEgsFFBERCQWCigiIhILBRQREYmFAoqIiMRCAUVERGKhgCIiIrFQQBERkVgooIiI\nSCwUUEREJBYKKCIiEgsFFBERiUWiAcXMhszsmJkdLrF/o5m9YmZPBa/bIvsuN7MjZvacmd0S2T5o\nZj+KnHN5kp9BRESqsyTh698D/CFwX5ljvuvuV0Y3mNki4OvAu4F/Bg6a2YPufiQ45E53vzOJBIuI\nSH0SraG4+wHg5QqHFXvM5MXAP7j7UXc/AewDrqpwjoiINFEa+lAuMbNDZvaImZ0bbHsr8MPIMT8K\ntoW2BufcbWa9DUupiIiUlHSTVyVPAmvd/TUzuwJ4ADirwjl3AZ93dzezLwB3AltKHbx9+/bp97lc\njlwuN980i4i0ldHRUUZHR+d9HXP3+aem3A3M+oGH3X1dFcc+D1xIPqhsd/fLg+3bAHf3HbVc28w8\n6c8nItJuzAx3r7lroRFNXkaJPg8zWx15fzH5APcScBA4w8z6zSwDfBB4KDhuTeQS7we+l1TCRUSk\neok2eZnZHiAHnGRmLwCDQIZ8bWMXcI2Z3QCcAI4D15Lf+e9mthV4lHzQG3L3Z4PLfsnM1gNvAhPA\n9Ul+BhERqU7iTV7NpCYvEZHapbnJS0REFgAFFBERiYUCioiIxEIBRUREYqGAIiIisVBAERGRWCig\niIhILBRQREQkFgooIiISCwUUERGJhQKKiIjEQgFFRERioYAiIiKxUEAREZFYKKCIiEgsEg0oZjZk\nZsfM7HCJ/RvN7BUzeyp43RbZd7mZHTGz58zslsj2lWb2qJl938yGzaw3yc8gIiLVSbqGcg/wngrH\nfNfdNwSvLwCY2SLg68G5vwBsNrOzg+O3AY+5+9uBx4HPJJP0xhkdHW12EqqidManFdIISmfcWiWd\n9Uo0oLj7AeDlCocVeyrYxcA/uPtRdz8B7AOuCvZdBdwbvL8XuDqOtDZTq/xHpnTGpxXSCEpn3Fol\nnfVKQx/KJWZ2yMweMbNzg21vBX4YOeZHwTaA1e5+DMDdXwROblxSRUSklCVNvv+TwFp3f83MrgAe\nAM6q8Rp6aLyISAqYe7LlsZn1Aw+7+7oqjn0euJB8UNnu7pcH27cB7u47zOxZIOfux8xsDTDi7ueU\nuJ6CjYhIHdy9WHdEWY2ooRjF+0kws+nmKzO7mHyAe8nMDgJnBMHo/wIfBDYHpz0EfAzYAXwUeLDU\njevJEBERqU+iAcXM9gA54CQzewEYBDLkaxu7gGvM7AbgBHAcuJb8zn83s63Ao+T7eYbc/dngsjuA\nb5rZdcBR4ANJfgYREalO4k1eIiKyMKRhlNe8lZoEWXDM18zsH4IRZevTlsZykzwbnM6yk1GDY5qa\nl0Ea6p4028A0nmJmj5vZ35vZM2Z2Y4njmv3fZsV0piQ/O83sCTN7OkjnYInjmp2fFdOZhvwM0rEo\nuP9DJfbXlpfu3tIv8kHxH4F+oAM4BJxdcMwVwCPB+3cCf5vCNG4EHkpBfl4KrAcOl9jf1LysIZ1N\nz09gDbA+eN8NfD9t/23WkM6m52eQjqXBv4uBvwUuTlt+VpnOtOTnzcD/LJaWevKyHWoo5SZBhq4C\n7gNw9yeAXjNbnbI0QonBC43klSejNjsvCe5d76TZhnH3F939UPD+VeBZZuZThZqen1WmE9Lx3+dr\nwdtO8n3AhW32Tc/P4N6V0glNzk8zOwV4L3B3iUNqzst2CCjlJkGWOuafihyTpGrSCMUneaZNs/Oy\nFqnJTzMbIF+jeqJgV6rys0w6IQX5GTTRPA28CPylux8sOCQV+VlFOqH5+fkV4HcpPZev5rxsh4DS\nLsJJnuvJr2P2QJPT0+pSk59m1g38KXBTUANIpQrpTEV+uvub7n4BcArwzmb/UCilinQ2NT/N7D8A\nx4KaacmpHbVqh4DyT8DayN+nBNsKj3lbhWOSVDGN7v5qWE129+8AHWa2qnFJrFqz87IqaclPM1tC\nvpD+H+5ebM5UKvKzUjrTkp+R9Pw/YAS4vGBXKvIzVCqdKcjPdwFXmtkPgL3AJjO7r+CYmvOyHQLK\n9CRIM8uQnwRZOGLhIeAjAGb2S8ArHkyoTEsao22TFpnk2cA0zkoOpX+xNDsvo8pOmo28b2Z+/ndg\n3N2/WmJ/WvKzbDrTkJ9m9nMWPK7CzLLAZcCRgsOanp/VpLPZ+enut7r7Wnc/jXx59Li7f6TgsJrz\nstlrec2bl5gEaWbXE0ygdPc/N7P3mtk/Av8GfDxtaaTEJM9GswqTUZudl9WmkxTkp5m9C/h14Jmg\nPd2BW8mP9ktNflaTTlKQn8BbgHst/3iLRcD+IP9S8/96tekkHfk5x3zzUhMbRUQkFu3Q5CUiIimg\ngCIiIrFQQBERkVgooIiISCwUUEREJBYKKCIiEgsFFJEmMrPnU7oigkjNFFBEmksTwaRtKKCINIiZ\nfdvMDgYPXfqNcHNTEyUSo5ZfekWkhXzc3V8xsy7goJl9q9kJEomTAopI4/xnM7s6eH8KcCZq8pI2\nooAi0gBmthH4ZeCd7v4zMxsBupqcLJFYqQ9FpDF6gZeDYHI28EvBdvWhSNtQQBFpjL8g/xClvwf+\nC/A3wXY1eUnb0PL1IiISC9VQREQkFgooIiISCwUUERGJhQKKiIjEQgFFRERioYAiIiKxUEAREZFY\nKKCIiEgs/j9JKCk3iDdoEgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f4437bfb048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# equivalent scatter plot using Matplotlib\n",
    "plt.scatter(glass.al, glass.ri)\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('ri')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fit a linear regression model\n",
    "from sklearn.linear_model import LinearRegression\n",
    "linreg = LinearRegression()\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.ri\n",
    "linreg.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  \n",
       "id             \n",
       "22   1.521227  \n",
       "185  1.521103  \n",
       "40   1.520781  \n",
       "39   1.520781  \n",
       "51   1.520682  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# make predictions for all values of X\n",
    "glass['ri_pred'] = linreg.predict(X)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f442dd03fd0>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEPCAYAAABlZDIgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VPWdN/DPlyRzSULCpRGslAQVvCNixdpaGarUy7bo\nWltKdW1r2qotq0trK/rYBsU+K7rFaisiKyu6lUu3W1FL2ygr2S7draEIRRspWglq+0BiAVcLCsLn\n+eOcSU4mM5OZyZmZM8nn/XrNi5lz/c2J/r7zuxtJiIiI9NeQYidAREQGBgUUERHxhQKKiIj4QgFF\nRER8oYAiIiK+UEARERFf5DWgmNlSM9tlZltS7J9qZnvN7Dn3dUvC/iHu9ic825rM7HXPORfk8zuI\niEhmyvN8/YcA/ADAI2mO+RXJGSn2XQ+gDUBNwvaFJBf6kD4REfFJXksoJNcD2NPHYZZ0o9kYABcB\neDDTc0REpHiC0IZylpltNrM1ZnaiZ/vdAL4JINlQ/tnuOQ+aWW1hkikiIukUO6BsBDCW5CQAPwSw\nGgDM7BMAdpHcDKc04i2RLAJwtHvOTgCq+hIRCQDL91xeZlYP4EmSEzM49hUAHwRwA4ArALwHIApg\nKICfkrwym2ubmSYqExHJAcmsmxYKUUJJLGF07zAb5Xk/BcAQkrtJ3kxyLMmjAXwWwDPxYGJmoz2X\nuBTAC+luTjLwr6ampqKnQelUGpVOpTP+ylVee3mZ2XIAMQAjzexVAE0AQgBIcgmAy8zsWgAHAewH\nMDODy95pZpMAHAbQDuDqPCRdRESylNeAQvJzfey/D8B9fRzznwD+0/P5yjSHi4hIkRS7UV4AxGKx\nYichI0qnf0ohjYDS6bdSSWeu8t4oX0xmxoH8/URE8sHMwIA2youIyCCggCIiIr5QQBEREV8ooIiI\niC8UUERExBcKKCIi4gsFFBER8YUCioiI+EIBRUREfKGAIiIivlBAERERXyigiIiILxRQRETEFwoo\nIiLii7wGFDNbama7zGxLiv1TzWyvmT3nvm5J2D/E3f6EZ9twM3vKzP5gZs1mVpvP7yAiIpnJdwnl\nIQDn93HMr0hOdl+3J+y7HkBbwra5ANaSPA7AMwBu8iepIiLSH3kNKCTXA9jTx2FJF3ExszEALgLw\nYMKuiwE87L5/GMAl/UmjiIj4IwhtKGeZ2WYzW2NmJ3q23w3gmwASl1w8guQuACC5E8ARBUqniIik\nUV7k+28EMJbkPjO7EMBqABPM7BMAdpHcbGYxpCjFuNKu8Ttv3ryu97FYbMCv6Swikq2Wlha0tLT0\n+zp5X1PezOoBPElyYgbHvgLggwBuAHAFgPcARAEMBfBTklea2YsAYiR3mdloAOtInpDielpTXkQk\nS0FeU96Qup1klOf9FABDSO4meTPJsSSPBvBZAM+QvNI99AkAX3Dffx7A43lLuYiIZCyvVV5mthxA\nDMBIM3sVQBOAEACSXALgMjO7FsBBAPsBzMzgsgsA/NjMrgKwA8Bn8pF2ERHJTt6rvIpJVV4iItkL\ncpWXiIgMAgooIiLiCwUUERHxhQKKiIj4QgFFRER8oYAiIiK+UEARERFfKKCIiIgvFFBERMQXCigi\nIuILBRQREfGFAoqIiPhCAUVERHyhgCIiIr5QQBEREV8ooIiIiC/yGlDMbKmZ7TKzLSn2TzWzvWb2\nnPu6xd0eNrNnzWyTmT1vZk2ec5rM7HXPORfk8zuIiEhm8roEMICHAPwAwCNpjvkVyRneDSTfNbNp\nJPeZWRmAX5vZL0i2uocsJLkwT2kWEZEc5LWEQnI9gD19HJZ0mUmS+9y3YTiBz7uWb9ZLU4qISH4F\noQ3lLDPbbGZrzOzE+EYzG2JmmwDsBPA0yQ2ec2a75zxoZrUFT7GIiPSS7yqvvmwEMNat2roQwGoA\nEwCA5GEAp5lZDYDVZnYiyTYAiwDcRpJmdjuAhQAaU91g3rx5Xe9jsRhisVi+vouISElqaWlBS0tL\nv69jJPs+qj83MKsH8CTJiRkcux3A6SR3J2z/NoC/Jrab9HVtM2O+v5+IyEBjZiCZddNCIaq8DCna\nPMxslOf9FDgBbreZvS9elWVmUQDTAWx1P4/2XOJSAC/kK+EiIpK5vFZ5mdlyADEAI83sVQBNAEIA\nSHIJgMvM7FoABwHsBzDTPfVIAA+b2RA4QW8VyZ+7++40s0kADgNoB3B1Pr+DODo7O9He3o6GhgbU\n1dUVOzkiEkB5r/IqJlV5+WPFilVobPwqQqEGHDjQjqVLF2HWrJl9nygiJSnXKi8FFEmrs7MT9fXH\nY//+dQAmAtiCaHQaduzYqpKKyAAV5DYUKWHt7e0IhRrgBBMAmIiKinq0t7cXL1EiEkgKKJJWQ4NT\nzQXEZ8/ZgoMHd6ChoaF4iRKRQFJAkbTq6uqwdOkiRKPTUFMzGdHoNCxdukjVXSLSi9pQJCPq5SUy\neKhRPgkFFBGR7KlRXkREikoBRUREfKGAIiIivlBAERERXyigiIiILxRQRAKos7MTGzZsQGdnZ7GT\nIpIxBRSRgFmxYhXq64/H9OnXoL7+eKxYsarYSRLJiMahiASIJuOUINA4FJEBQJNxSilTQBEJEE3G\nKaUsrwHFzJaa2S4z25Ji/1Qz22tmz7mvW9ztYTN71sw2mdnzZtbkOWe4mT1lZn8ws+b4UsEiA4Em\n45RSltc2FDM7G8DbAB4hOTHJ/qkAvkFyRpJ9lST3mVkZgF8DuI5kq5ktAPAXknea2Y0AhpOcm+L+\nakORkqTJOKWYcm1Dyeua8iTXm1l9H4clTTTJfe7bMJx0xiPDxQCmuu8fBtACIGlAESlVdXV1CiRS\ncoLQhnKWmW02szVmdmJ8o5kNMbNNAHYCeJrkBnfXESR3AQDJnQCOKHySRUQkUV5LKBnYCGCsW7V1\nIYDVACYAAMnDAE4zsxoAq83sRJJtSa6Rtk5r3rx5Xe9jsRhisZhPSRcRGRhaWlrQ0tLS7+vkfRyK\nW+X1ZLI2lCTHbgdwOsndCdu/DeCvJBea2YsAYiR3mdloAOtInpDiempDERHJUpDHoRhStJOY2SjP\n+ylwAtxuM3tfvPeWmUUBTAew1T30CQBfcN9/HsDjeUq3iIhkIa9VXma2HEAMwEgzexVAE4AQAJJc\nAuAyM7sWwEEA+wHMdE89EsDDZjYETtBbRfLn7r4FAH5sZlcB2AHgM/n8DiIikhlNvSIiIj0EucpL\nREQGAQUUERHxhQKKiIj4QgGlRGkBJhEJGgWUEqQFmEQkiNTLq8RksgCTJhYUkf5QL69Bor29HeXl\n9Ui1AJNKLyJSLAooJea55zbjrbe2ItkCTJ2dnWhs/Cr271+HN9/ciP3716Gx8atqZxGRglBAKSGd\nnZ2YM2cugHkApgE4FcCHcPfdd6Curk7Lx4pIUSmglJDugPEtOFObPYjq6mMwblw9NmzYgOrqas/y\nsZ0AluPAge1aPlZECkIBpYT0XG+8DkAY7767A5dcMgvTp1+D008/G42NV6Ci4mwADQCacPgwsXbt\nM0VMtYgMFurlVWJWrFiFxsavoqKiHgcObMfhw8SBA79CvMdXJDIVgOGdd1qQqhdYKVGPNZHCUy+v\nQWLWrJnYsWMr1q59AI8/vgrR6DHwtpmUlR2BsrKxGAjtKOqxJlJaVEIpYcnGpEQiU2E2JO04lVKQ\nyXgbEcmPXEsoxV4CWPqhrq4OS5cuQmPjNFRU1OPgwR1YunQxACRsW1RymXC8A8L+/b1LWqX2XUQG\ni7yWUMxsKYBPANiVbAlgM5sKZ8XFV9xNPyV5u5mNAfAIgFEADgP4Z5L3uuc0AfgygA73nJtJ/jLF\n/Qd0CSUuWTtDqbc9qIQiUjy5llDyHVDOBvA2gEfSBJRvkJyRsH00gNEkN5tZNYCNAC4mudUNKG+R\nXJjB/QdFQBmovB0Q4iWtWbNm9n2iiPRLIKu8SK43s/o+DuuVaJI7Aex0379tZi8COArd68pn/UWl\n9MyaNRPnnfexki5piQwmQejldZaZbTazNWZ2YuJOM2sAMAnAs57Ns91zHjSz2gKlU4qgrq4OZ5xx\nRkkEEy0pIINdsQPKRgBjSU4C8EMAq7073equnwC4nuTb7uZFAI52z9kJoM+qL5F8UxdnkQJ0G3ar\nvJ5M1oaS5NjtAE4nudvMygH8DMAvSN6Ty7XNjE1NTV2fY7EYYrFYDt9CJDV1IJBS19LSgpaWlq7P\nt956a/Aa5YGuKqsnSZ6SZN8okrvc91MA/Jhkg/v5EQBvkPx6wjmj3TYWmNkcAGeQ/FyKe6tRvshK\nvbdZJjZs2IDp06/Bm29u7NpWUzMZa9c+gDPOOKOIKRPJTSBHypvZcgD/DWCCmb1qZl80s6vN7Cvu\nIZeZ2QtmtgnA9wHMdM/7CIDLAXzMzDaZ2XNmdoF7zp1mtsXMNgOYCmBOPr+D5G6wVAP1nGMN8C4p\nIDKYaKS85EWu1UD5KtHku6SkLs4ykASyhCKDVy5rs+SrRFOIkpJ3jrUdO7YqmMigpBKK5EW2JZR8\nNWyrwVwkeyqhlLCBOH4hPs9YNDoNNTWTEY1O65pTLNn3zddqk1rFUqRwFFCKrJAN14UOXMmqgVJ9\n33w1bKvBXKSASA7Yl/P1gqujo4PR6AgCvyNAAr9jNDqCHR0dvt9r+fKVjEZHsLZ2MqPREVy+fKXv\n9+hLX983nsaamtN8TWO+risyULl5Z9Z5rtpQiqhQ4xeC0o6Qyfct1V5e/RX09MngEsjJISW9ntUx\nTkbvZ3VMPJPas2dPINYWyeT71tXV5SVN+bquH+JdjkMh5/moy7GUrHTFFwA17r8jkr1yKRIV8oWA\nVnm1tbVx2bJlbGtr8606pqOjg62trb2qj4YOPYWhUDUrKmrSVq0lnt/X9XOl6qeeClntKZIp5Fjl\n1VeG/DM4U8UfhrMI1nbP65VcbljIVxADyuzZ1xOIEphAIMrZs6/rd2ad2D6yePESN5NaQGAEgVMJ\nhFhRUdMrI+/o6OD8+d9N277id/uLX8FpIGhtbWVt7WQ3mDivmprT2NraWuykySCWl4DC7oz5hVwu\nXuxX0AJKW1ubG0y6f40CUba1teV8zWS/cMPhGlZVnewGk+7toVANm5ube5RiIpFhBCpT/kLO9Re0\nX0HDj+sEOYCphCJBlGtAybTb8EYz0yx3/dTa2grgA/COiQDGuNtzk3ycxVi3raLnvcLhozF8+PCu\nsSCf//xX8M47VwB4P1KN0yjmiHc/rhO/xrRpXw7kfGLpxuuIlJxMog6clRLfA/BHOC2qzwPYkksE\nK+QLg7SEEo2O4F13fa/XvSKRYV0llBkzLiHQQKCq13H9KaH49Yvbj+t0dHQwFKpNKKXVBvLXf5BL\nUTL4IM9VXvXJXrncsJCvoAUUkpw9+zo3Ax/f1YaSTDYZTKqG7sWLlzAcHsahQyexomIoQ6Fa1tZO\nZjhcQyDkvio9bS0TCUQ5d+5NGV0/Gb/aBHK9jve5NTc3Ezi2xzWAY9jc3JxVWkQGm7wGlFJ95TWg\nHDpEHj6c06neXl7J5NIInioAxTPWnr/2byMwjkCEwDHutg4CrQQakma4mfYCW79+PcPhYUUpoSQ+\nt7lzb+7VPgRUKqCI9EEBpdAB5amnyOpq8rTTyJkzyVtuIR95hPzNb8i//CXny+ajkbb3r/141du3\ne1V35VIlFM/Io9GjCURZUTGWQJTR6Mn96hWWrmSUGOCSPbdIZBgrKqoJDCdwGoHhrKioTvr9ilHl\npGouCSoFlEIHFJLcs4dsbSV/9COyqYmcNYv84AfJmhpy5EjyQx8ir7ySnD+fXLWK3LSJfOuttJfM\nRzfSZJntkCERN5gcQSDK8vLjcsr8u6+9jj17la1jOFzTr/ah+PXjmW78fbxbtLcEl+q5zZ//XUYi\nw1hVNYGRyLCk368Y09Lkek8FISmEQAYUAEsB7ErVgA9nxcW9AJ5zX7e428cAeAbA790OANd5zhkO\n4CkAfwDQDKA2zf19fswZOnyY3LmT/K//IpcuJefOJS+9lDz5ZDIaJd//fnLqVPLLXybvuotcvZps\nayPfeSdv3UiT/dqPV72tX78+50yqOyNvJZC/8RTegZrJOhG0tbWlfG7pMuFidNvN9Z5BmI9NBoeg\nBpSzAUzqI6A8kWT7aACT3PfVbvA43v28AMC33Pc3Argjzf19fMQ+OXSI3LGDXLuWXLSInDOH/Ju/\nIcePJ8Nhctw4/nniqVxUFuYNkTGcEarm43ffQ773Xr9vnY9ft6lLKP5lzD0z4FY6AzV7B65sOg/E\nn0Vzc3PBBxbmUgrVeBUppEAGFCddqO8joDyZwTVWAzjXfb8VwCh2B56tac7z6fEWyIED5LZt5Jo1\nfGv+fO761Kf47jnnkGPHOsHm+OPJGTPIb3yDfOAB8plnyNdfz7lzgF/iGXkk0uBL20minhlwR9rA\nlUnQ9P7Sj0SG9epanO20NNnKJThoRL0UUikHlDcAbAawBsCJSY5pANAOoMr9vDth/+409/btASfK\nZ1120mvv20c+/zz57/9O3nEHedVV5Ec/So4aRVZWkqeeSn760+TNN5PLlpG//jXZ2VmwYBNPc1tb\nW7+eS7Lv3jsDXkAgyqFDJ2UduHpfax3LyqKMRIYnLdnkq5op2znNVEKRQirVgFINoNJ9fyGAbUn2\n/xbAxZ5tiQHlL2nu7dfz7SGfddk5XfvNN8nf/pZcvpy89Vby8svJKVPIYcPI4cOd91dc4exbsYLc\nuJH83//1Lc1+SffdEzPgxYuX5BS4ev7SX+mWdiYwFKrh/Pnf7SOQ+ZuJZ/ujRBNrSqHkGlDyvh6K\nmdXDqdaamMGx2wGcTnK3mZXDmZzyFyTv8RzzIoAYyV1mNhrAOpInpLgem5qauj7HYjHEYrF+fZ98\nri3i17W71taor0edGfDSS8C2bd3/btsGvPwyUFMDjB8PTJjQ/e+ECcAxxwCRSL++S8o0JVnvo7Oz\nE5s2bcIll8xK+939WDOk+xn/O4BPAUh9v0KtV5MNrZsi+dDS0oKWlpauz7feeiuYw3oohSihNAB4\nPsW+UZ73UwC0ez4/AmBhknMWALjRfV/wRvl81mX7ce2MSziHDpGvvea0wyxe7LTLfPKT5HHHOe01\n9fXkeeeRX/0qeffd5Jo15EsvkQcPprx3ql/c3SP2U1cpVVUdR2dUe3yAZUfXd+9v9WKyqf2d2QIm\npH3W2ZRQ/KwCVddgKTYEscoLwHIAfwbwLoBXAXwRwNUAvuLu/xqAFwBsAvDfAM50t38EwCE4bSub\n4HQpvsDdNwLAWjg9v54CMCzN/X1/0PmsBunvtX1L28GD5Msvk7/4BXnPPeTXvkZ+/ONkQ4MTbCZM\ncHqmzZlD3n8/uXYtV//gh6yMDO8VyBYvXsJUc4r1HFXfQSA+CHEygeEsL6/ijTfe1K/qxVQBtq2t\nLaMR/d7pa1Ld388qUHUNliAIZEAp9isfAYXMb112f65dkJ5A+/eTv/89+dhj5J13kl/6Et/98If5\nZxj/igh/h1P4E5zLO8sj/NPtt3NaRSXrcCKBw552i0qGQg0Ewp5SQgcBb2+rBXSmhkk9tX5fenZp\nbiWwrsf5fT1r79iXcLiGixcvSXOP/v/AUMO7BIUCSgEDClmEXl4ZnlfIDCk+T9i9997LoUNPYzVe\n4ST8Kz+DJbwtfCRf/vCH+RuL8g0Y96KKrZjAH6GCTfg8Z6GKH8T9rEW8lOAdX9JGYBiBR5lqsGQm\nz6i1tdWdDmaEex2na3NitVZzc3OPdWKyeZZ+BnF1DZagUEApcEAJqkL1BFq+fCUrKoa6JYhxbmmj\nu7qqoqKa69evd6u7PscRiPBMjOLf4Qjehqu4EsO4EQ18C+AugP+FEJeijHNxAS9FlKegnlG0M3HM\nSSQyLONqsO77914uIN00LmTfmbu3m3QxSyhqb5F8UEAZQAHF70Zov67rvY6z0uNwtzqpmcDQHhlh\nKFTL5uZmRiJjPZl6h3vOt93qrOEE7uCRiPIchPgllHEByvkYPsLfYwj3IcwdGMm1KOci1PCGIWFe\nXBblBERYgaeSVmPFdTe8j+8RFKLRk7uWPE41jUt8upZUmXtiO8fs2df5FsSz+UGg9hbJFwWUARJQ\n8j2Qzo/rtra2ur2y4tVJ8R5aZLyXVnX1yVy1apU7CeUxnkw93kh/Pp0FvuKB6VE67SmndR03BLVs\nQD0vKq/k49M/zvvKwvw5JvJllHM/wG0Icw0q+IOyoWz/5jedGaDb29mxc6cbDB5zr9+zhNPdrrKM\nzhow3QHHWwpJlrmnCjT9HdDplUngV3uL5JMCygAIKJlkErmUMvzOfDo6Otxf/4klj/hCXacSCLO8\nvNotiXiPu80TSKrdoDSZwLc8pZZvs3slydEsK4ty7tybGY2ewvjU+xX4LY/Di/wk7uHXUc7dn/40\nOW0aedRRfC8U4gtWwZ+ignegllehgh/FURwbquH82273tKukLqGket5BaecISjpkYFJAGQABpa9M\nItdSRj4yH2fxKm/J40uezHk9nSqw+9ndvhKl095S636eSOByN4iEGG9rAco9gSUeoE4gUOEe86j7\nufu7hEIndGX68+d/l8NDQzkRYV6Gu3gTvsuHcD7Xw3hg2DC+V1nJ38K4AufzVnybl2MmpyDMMVUn\nldQUKEFJhwxMCigDIKCkyyT6k4H4nfksX77SbUOJd+ntcAPIRALXuwHjSLeUEXKDSJROFVSUwKWe\nkssI95hxnlLLeDdo1HqCyngCowh4S0ZkvKH9rru+50nTo27pozvoRKMns7W1lRvXruVHw8fwctzH\nefgSl+Nv+VuL8N1IhAeHDeNbp5zC/Z/5DHn77eSPf+ysYfP22z2+fyZjUwohl9mVFXAkEwooAyCg\nkKkzif6WMvzq/dUzOK10SxJHuQEhntl/3VPKuMYNMKfSabOIVzc1usfGe4eFPYEkvub9OHb38opX\nq91PoN7dfhqBEQyHP+AOUox3M06ckbh7sa+Ojg63d1rPHmmL73+ADZFhvKBqAq8uj3L9R8/hOxdd\nRJ50EhmJOGvYxGL8/Uc/yhvLwrwsNIoTKyq55Af35fQM063Pkk3Gn+3symq8l0wooAyQgEJmOuNu\n9qUMP36l9g5sHaysPNrN0D/tCSzxtpFlnsDwmCe4NNNpyI+3nVSyu2pstifIeNc+WcnuqrN1jPfy\nCodrWF09KSGQxIPdaDpT6p/SNalkRUVNj+dYUTE0IUjWEhjTvcLjoUNkezufvP4feC3KuRBh/gw1\n3IYh3A/wvfp68vzzyb//e/Lee8lf/pL84x+TrmETL91UVk5MOQ2Nnxm/qsYkFwooAyigpBKE2WZT\nZVBORlnjBoGT2N17K17NFa+6GuF+XucGnkfp9BJb6ZZKRrnnxgNH/FgneEQiwzh37k09lvVtbPwy\nu6vB4oGkgeXllSwrq+6R1rKySib2SItGj2ZV1ansrrrrWXppa2tjc3MzQ6Fq9hy5v47lCHHN3XeT\nP/sZuXAhec015Lnnkh/4gDNNzQknOGvY3HAD//PyKzgVIb4fT9GZOcDpXt3fKs101HgvuVBAGQQB\nhQxGXXiqwNbR0cEZMy72BJB4BhxvQxlHoIJmETfwhDylk3WeILSOwHfdc8rdf8cTiHL27Ou62nAi\nkfEMh2vcEkc8YE107xGvTvOOQ+lwr+U9/lQCEfca9zNxqhcgzHB4mNtN+v2eYBSf+n48w+FhvOuu\n73HZsmVsa2vrflB//Sv/0tLCl+64g7vmzOFDQ8r5K0S4E0fwLVTxOUziKlTxj7Nm8Y/f+Q7Pq5rA\nEXjD14w/SCWUIPy3K5lRQBkkASUo0mUOd931PYZCtayqOomhUDU/+cm/ZThcy3B4nKfEUekGj7Fu\nYIln/vGMehyTzeUViQxLaAOJsru32XoCn3DPq2H3ssTr3BJOvNdZvEuyt9qr2i2BeHuudbBnx4Pa\nhOvGz/+ce70JXUGP7FmFFQ7XuPOXOderwV6ejuWchRBf+ru/4/5LL+UGK+MeDOVfMJz/g1P4aFmI\nb994Y7/XsAlCyTZZdV5fAUYBqHgUUBRQAiX51CStbgliladUEt/Xxu7JIdvYXR3Ws7qmsvJoTybf\nxu4qqk+5gSRMp/0mXpK43lPCibhVXuPZPYDSeVVVTeSqVasS2lce9VyHbrALs2fJp61XcAKiXL9+\nfULJYB27S2XD3fsPJxDqKtUsX76S0chwHl19MqeFhvJ/vnINedNN5Kc+RZ5yChmNkqNHk+ecQzY2\nkgsWOJN0vvCCM2knM5slIZuM2o9MPVkpKd5ulaq9SB0JiksBRQElkHrW4cdHyd/olgRaEwKGMxNx\neflod3/v3lrl5VE3k1/pBp0JBK5g9xiVkz0lEG/A6iDwqBtQIkwcQZ84rUpV1cQk6807MyBHIvWe\nILKMieuqAOPY2NjIoUN7Bq1Q6EjP93LWfIl3Z45Lm/EfOkS++ir5H//hLBswZw75iU84ywmEw3zr\nfe/j00PKuST0Pn6jopLrvnVjrzVsssmo/crUk3XkSDeLdJCq6QYrBRQFlEDqOYX8CHa3rSRm+E7G\n4VQN1bi/3nv31opETmB3l+R1dEoolW6mHg9Yn2N3j7Fj2F2NNplAJSdPnuKWFCoJHMPy8qE9Mktv\nRh7vlVVVdXJCScRZ176y8vik23sf/zu33Wc4M8kos83MO/78Z54UruUFuI/X4fv8IWby6SHlfC/e\nOWDCBL7z8Y/z++URXo3/w2n4D45BMysjw1N2X87fpJeJJb+e7UXqSFB8CigKKIG1fPlKt2QQb59o\nJvABN5OPTx55LMPhYZw//7tuZhIPAqfRqaqKTz7Z4QaU+OqOYfdaETpVZlVuoPBO7+LNxNd5Mnqn\n1BKJDEubqQ8degorKqoYiZzcI5Orrj6Zy5Yt41VXxXuZjWOy4OIdAJlJe0YumXnaTHj/fvKFF7ht\nwQJ+O/J+/jMa2YJz+GeM5tsYwr8ee6xTrXbTTeS//Au5fj2f++UvWVtzWtLvm0tVmfd79y75qYQS\nNIEMKACWAtgFYEuK/VMB7HVXZHwOwC19nQugCcDrnnMuSHN/Xx+y5Kbn7MTeQYrr6FT9PNZj4GF3\nZuJk+KHkw6mwAAAQL0lEQVRQtaf6qJXOKPgRdBrZo27gOJ/OAMvJdNpo4t2X41Vk9JyffunfeJp7\nt4Gknverra2N3/nOd9zxMOkz4b7aJXL5hZ7pPHCJxxwRGc7da9c6Df+33UZecQV55pk8VFvLPTA+\ni5P4I3yO38F5nIUKnlN5HEfFx+ekkGrJZ+/3znRxs2J2JBjMghpQzgYwqY+A8kQ257oB5esZ3t+n\nxyv90Z1BeksdYVZU1CTNMBIzk/iaJb0nooz3ylrJ7rEr8UBV5QkqmQeG3mnuztQjkQaGw8NSZnJ+\n/bLOdV2U+LT86TLhVDMo9wpwhw/zJ4uXcGpoKL8SPorzUc5V+Dg34VS+jQh3wnjgzDPJL36R/Md/\nJH/yE3LLFv7zvT/M6PnG0zyYenmV0vcJZEBx0oX6PgLKk9mc6waUb2R4b18ervRPslJHJDIs7ZTv\n6apMnC6+tayq8rZfdNAZYR9vGxnF7jEs3jEqUV511Zdzrnbqa5p6v35Z57ouSiTiVBv2FXwSSwqp\n2mo6Ojq4bNmyhA4Gh3lc9Ul88f77yQceIG+4gZwxgwfHj+d+gDtQwadxLhfhGv4DFvLT0XH83b/9\nG3ngQE7PYiAotV5rpRxQ3gCwGcAaACf2da4bULa75zwIoDbNvX18xNIffmS0yXpAxUsvVVUTu0oz\n8SV929raOHfuTV2N6t514bOZAyvbNPv1SzST6xRi0tBMj2ttbeWw6pM5DrX8OBZxNu7lvfgsm62c\n740d63QOOPZY8qKLyOuvJ++7j3z6abK93enBNkCVYptQrgHFnHPzx8zq3VLIxCT7qgEcJrnPzC4E\ncA/JCenONbM6AG+QpJndDuBIko0p7s2mpqauz7FYDLFYzK+vJlnq7OxEe3s7GhoaUFdXV7Dr9ue+\n+UqzXzZs2IDp06/Bm29u7NpWUzMZa9c+gDPOOMO3c1esWIXGxq+ioqIeBw/uwNKlizBr1swex3R2\ndqK+/njs338jgAUAxgB4CYsX34Orr/4y8O67wCuvANu2AS+91PPfPXuAo48GJkxwXuPHd/87ahRg\n1t9HVTT9+RsVSktLC1paWro+33rrrSCZ9UMvakBJcux2AKeT3J3JuRnsZ76/n0gxdWfi6wBMBLAF\n0eg07Nixtc8AmO25mQTXeOApK3s/Dh58Fffc809OMOnL228DL7+cPNgcONAdYBKDzfDhfV+7yPrz\nNyoWM8spoJTnIzEJzH313mE2iuQu9/0UOAFud7pzzWw0yZ3ux0sBvOB/kkWKI9sSUV1dHZYuXYTG\nxmk9Sg/5OLeurq7P686aNRPnnfex7Et11dXApEnOK9Hu3T2DzM9+1v0+EkkebI49Fqiqyuzeedaf\nv1GpyWsJxcyWA4gBGAmnC3ATgBCc+rklZvY1ANcCOAhgP4A5JJ9NdS7Jh8zsETi9vw4DaAdwdTwo\nJbm/SihSMuK/7kOhBhw40J60WimVgVytlxIJ7NzZu0SzbZtTtTZyZO8SzYQJTtVaKFTw5JbSc861\nhJL3Kq9iUkCRUlGK1SKBdugQ8NprPYNM/P1rrwFjxiQPNmPHAmVlxU590QW5yktE+tDe3o5QqAH7\n98ebAyeioqIe7e3tCii5KCsDGhqc1/TpPfcdOABs394dYF54AXjsMef9G28A48YlDzZHHlnSnQMK\nQSUUkQBQCaX/fKlS2rcvdeeAfft6B5n4+5Ej/f0yRaYqryQUUAa3UqqzBjLrmivJxZ9deflROHBg\nR+a9y7Kxd2/vIBP/t7w8dbCprvY3HQWggJKEAsrg1Z8G7mIqtSAYBL3Hv3wAwLbu8S/5RgIdHck7\nB/zxj8CwYd1BprER+NCH8p+mflJASUIBZXBS9dHgsmHDBpx7biPeeutPALr/5uHwVLz22rbi/s0P\nHwZef707yHz4w8CppxYvPRlSo7yISw3cg0tDQwMOHNgBYBycYAIAExEKNRT/bz5kiNNzbOxY4Nxz\ni5eOAhlS7ASI+M3JYNoBbHG3bMHBgzvQ0NBQvERJ3tTV1eGee/4JwDZ4/+bvvfeq/uYFphKK+CoI\nbQCDaWSyOOJtJddfPxWhUAPee+9V/c2LQG0o4pugNYQHIbhJYelv7g81yiehgFI4aggXGThyDShq\nQxFfxBvCvY2i8YZwERkcFFB80tnZiQ0bNqCzs7PYSSmKwdQQPtj/1iKpKKD4YMWKVaivPx7Tp1+D\n+vrjsWLFqmInqeDiDeHR6DTU1ExGNDptQDaK6m8tkpraUPpJbQc9DeRGUf2tZbBQG0qRqO2gp7q6\nOpxxxhkDMoPV31okPQWUfhpMbQeDnf7WIunlNaCY2VIz22VmW1Lsn2pme83sOfd1S1/nmtlwM3vK\nzP5gZs1mVpvP79CXwdJ2IPpbi/Ql30sAnw3gbQCPkJyYZP9UAN8gOSPTc81sAYC/kLzTzG4EMJzk\n3BT3L9g4lIHcdiA96W8tA11gBzaaWT2AJ9MElBtIfjLTc81sK4CpJHeZ2WgALSSPT3G+BjaKiGSp\nlBvlzzKzzWa2xsxOzOD4I0juAgCSOwEckd/kiYhIJoo9OeRGAGNJ7jOzCwGsBjAhy2ukLYLMmzev\n630sFkMsFsvy8iIiA1tLSwtaWlr6fZ2iVnklOXY7gNNJ7k51rpm9CCDmqfJaR/KEFNdTlZeISJaC\nXOVl7qv3DrNRnvdT4AS43X2c+wSAL7jvPw/gcd9SKiIiOct3L6/lAGIARgLYBaAJQAgASS4xs68B\nuBbAQQD7Acwh+Wyqc0k+ZGYjAPwYzsLROwB8huTeFPdXCUVEJEuB7eVVTAooIiLZC3KVl4iIDAIK\nKCIi4gsFFBER8YUCioiI+EIBRUREfKGAIiIivlBAERERXyigiIiILxRQRETEFwooIiLiCwUUERHx\nhQKKiIj4QgFFRER8oYAiIiK+UEARERFfKKCIiIgv8hpQzGypme0ysy0p9k81s71m9pz7usWz7wIz\n22pm28zsRs/2JjN73XPOBfn8DiIikpnyPF//IQA/APBImmN+RXKGd4OZDQHwQwDnAvgzgA1m9jjJ\nre4hC0kuzEeCRUQkN3ktoZBcD2BPH4clW2ZyCoCXSO4geRDASgAX93GOiIgUURDaUM4ys81mtsbM\nTnS3HQXgNc8xr7vb4ma75zxoZrUFS6mIiKSU7yqvvmwEMJbkPjO7EMBqABP6OGcRgNtI0sxuB7AQ\nQGOqg+fNm9f1PhaLIRaL9TfNIiIDSktLC1paWvp9HSPZ/9Sku4FZPYAnSU7M4NjtAE6HE1TmkbzA\n3T4XAEkuyObaZsZ8fz8RkYHGzEAy66aFQlR5GVK0eZjZKM/7KXAC3G4AGwAca2b1ZhYC8FkAT7jH\njfZc4lIAL+Qr4SIikrm8VnmZ2XIAMQAjzexVAE0AQnBKG0sAXGZm1wI4CGA/gJlwdh4ys9kAnoIT\n9JaSfNG97J1mNgnAYQDtAK7O53cQEZHM5L3Kq5hU5SUikr0gV3mJiMggoIAiIiK+UEARERFfKKCI\niIgvFFBERMQXCigiIuILBRQREfGFAoqIiPhCAUVERHyhgCIiIr5QQBEREV8ooIiIiC8UUERExBcK\nKCIi4gsFFBER8UVeA4qZLTWzXWa2JcX+qWa218yec1+3ePZdYGZbzWybmd3o2T7czJ4ysz+YWbOZ\n1ebzO4iISGbyXUJ5CMD5fRzzK5KT3dftAGBmQwD80D33JACzzOx49/i5ANaSPA7AMwBuyk/SC6el\npaXYSciI0umfUkgjoHT6rVTSmau8BhSS6wHs6eOwZKuCTQHwEskdJA8CWAngYnffxQAedt8/DOAS\nP9JaTKXyH5nS6Z9SSCOgdPqtVNKZqyC0oZxlZpvNbI2ZnehuOwrAa55jXne3AcAokrsAgOROAEcU\nLqkiIpJKeZHvvxHAWJL7zOxCAKsBTMjyGlo0XkQkAIzMb35sZvUAniQ5MYNjtwM4HU5QmUfyAnf7\nXAAkucDMXgQQI7nLzEYDWEfyhBTXU7AREckByWTNEWkVooRiSN5OAjPrqr4ysylwAtxuM9sA4Fg3\nGP0/AJ8FMMs97QkAXwCwAMDnATye6sa5PBAREclNXgOKmS0HEAMw0sxeBdAEIASntLEEwGVmdi2A\ngwD2A5gJZ+chM5sN4Ck47TxLSb7oXnYBgB+b2VUAdgD4TD6/g4iIZCbvVV4iIjI4BKGXV7+lGgSZ\ncMy9ZvaS26NsUtDSmG6QZ4HTmXYwqntMUZ+lm4acB80WMI1jzOwZM/u9mT1vZtelOK7Y/232mc6A\nPM+wmT1rZpvcdDalOK7Yz7PPdAbhebrpGOLe/4kU+7N7liRL+gUnKL4MoB5ABYDNAI5POOZCAGvc\n92cC+E0A0zgVwBMBeJ5nA5gEYEuK/UV9llmks+jPE8BoAJPc99UA/hC0/zazSGfRn6ebjkr33zIA\nvwEwJWjPM8N0BuV5zgHwo2RpyeVZDoQSSrpBkHEXA3gEAEg+C6DWzEYFLI1Ais4LhcS+B6MW+1nC\nvXeug2YLhuROkpvd928DeBHd46niiv48M0wnEIz/Pve5b8Nw2oAT6+yL/jzde/eVTqDIz9PMxgC4\nCMCDKQ7J+lkOhICSbhBkqmP+lOSYfMokjUDyQZ5BU+xnmY3APE8za4BTono2YVegnmeadAIBeJ5u\nFc0mADsBPE1yQ8IhgXieGaQTKP7zvBvAN5F6LF/Wz3IgBJSBIj7IcxKcecxWFzk9pS4wz9PMqgH8\nBMD1bgkgkPpIZyCeJ8nDJE8DMAbAmcX+oZBKBuks6vM0s78BsMstmaYc2pGtgRBQ/gRgrOfzGHdb\n4jEf6OOYfOozjSTfjheTSf4CQIWZjShcEjNW7GeZkaA8TzMrh5NJ/yvJZGOmAvE8+0pnUJ6nJz3/\nC2AdgAsSdgXiecalSmcAnudHAMwws1cArAAwzcweSTgm62c5EAJK1yBIMwvBGQSZ2GPhCQBXAoCZ\nfQjAXroDKoOSRm/dpHkGeRYwjT2Sg9S/WIr9LL3SDpr1vC/m8/wXAG0k70mxPyjPM206g/A8zex9\n5i5XYWZRANMBbE04rOjPM5N0Fvt5kryZ5FiSR8PJj54heWXCYVk/y2LP5dVvTDEI0syuhjuAkuTP\nzewiM3sZwF8BfDFoaUSKQZ6FZn0MRi32s8w0nQjA8zSzjwC4HMDzbn06AdwMp7dfYJ5nJulEAJ4n\ngCMBPGzO8hZDAKxyn19g/l/PNJ0IxvPspb/PUgMbRUTEFwOhyktERAJAAUVERHyhgCIiIr5QQBER\nEV8ooIiIiC8UUERExBcKKCJFZGbbAzojgkjWFFBEiksDwWTAUEARKRAze8zMNriLLn0pvrmoiRLx\nUclPvSJSQr5Icq+ZRQBsMLOfFjtBIn5SQBEpnH8ws0vc92MAjIeqvGQAUUARKQAzmwrgYwDOJPmu\nma0DEClyskR8pTYUkcKoBbDHDSbHA/iQu11tKDJgKKCIFMYv4Syi9HsA/xfAf7vbVeUlA4amrxcR\nEV+ohCIiIr5QQBEREV8ooIiIiC8UUERExBcKKCIi4gsFFBER8YUCioiI+EIBRUREfPH/AQRnCS0I\nnJdYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f442dcf6b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# put the plots together\n",
    "plt.scatter(glass.al, glass.ri)\n",
    "plt.plot(glass.al, glass.ri_pred, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('ri')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Refresher: interpreting linear regression coefficients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear regression equation: $y = \\beta_0 + \\beta_1x$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.51699012])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=2 using the equation\n",
    "linreg.intercept_ + linreg.coef_ * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.51699012])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=2 using the predict method\n",
    "linreg.predict(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['al'] [-0.00247761]\n"
     ]
    }
   ],
   "source": [
    "# examine coefficient for al\n",
    "print(feature_cols, linreg.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** A 1 unit increase in 'al' is associated with a 0.0025 unit decrease in 'ri'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.5145125136125304"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# increasing al by 1 (so that al=3) decreases ri by 0.0025\n",
    "1.51699012 - 0.0024776063874696243"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.51451251])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=3 using the predict method\n",
    "linreg.predict(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting a Categorical Response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    70\n",
       "2    76\n",
       "3    17\n",
       "5    13\n",
       "6     9\n",
       "7    29\n",
       "Name: glass_type, dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine glass_type\n",
    "glass.glass_type.value_counts().sort_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "      <th>household</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  household  \n",
       "id                        \n",
       "22   1.521227          0  \n",
       "185  1.521103          1  \n",
       "40   1.520781          0  \n",
       "39   1.520781          0  \n",
       "51   1.520682          0  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# types 1, 2, 3 are window glass\n",
    "# types 5, 6, 7 are household glass\n",
    "glass['household'] = glass.glass_type.map({1:0, 2:0, 3:0, 5:1, 6:1, 7:1})\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's change our task, so that we're predicting **household** using **al**. Let's visualize the relationship to figure out how to do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f442dc8a748>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHDhJREFUeJzt3X2Q3HWd4PH3J5MZ0mTChEgILAmMEgR25Sl3DlF0HVQ0\neHhYq6f4uLLqUlubk7ut8kDrLOLWPpxVd7e7LGt5rOiia+SuZD1BDxdZM3fHCSanAXImEVgYjHHJ\nZJGwhGV5iJ/7o3+d/DKZ6en5Tc90T3y/qrr69/B9+Py+3dOf+T11R2YiSdJ0Leh0AJKk+ckEIkmq\nxAQiSarEBCJJqsQEIkmqxAQiSaqk4wkkIm6KiD0R8cAk698TEfcXj7sj4py5jlGSdKSOJxDgC8Cb\nm6x/BPjVzDwP+D3gz+ckKklSUws7HUBm3h0RpzVZf29p9l7glNmPSpI0lW7YA5mODwN3dDoISVIX\n7IG0KiIuBq4EXtPpWCRJ8ySBRMS5wI3Ausx8skk5v9hLkqYpM6NKvW45hBXF48gVEacCtwLvz8y/\nnaqhzOzqx3XXXdfxGIzTOI3TOBuPmej4HkhEbASGgZdExI+B64A+IDPzRuCTwDLgMxERwAuZOdSp\neCVJdR1PIJn5ninWfwT4yByFI0lqUbccwvqFMTw83OkQWmKc7WWc7WWc3SFmegysm0REHk3bI0mz\nLSLIeX4SXZI0z5hAJEmVmEAkSZWYQCRJlZhAJEmVmEAkSZWYQCRJlZhAJEmVmEAkSZWYQCRJlZhA\nJEmVmEAkSZWYQCRJlZhAJEmVmEAkSZWYQCRJlZhAJEmVmEAkSZWYQCRJlZhAJEmVmEAkSZV0PIFE\nxE0RsSciHmhS5vqIeCgi7ouI8+cyPknSxBZ2OgDgC8CfAl+caGVEXAqcnplnRMSFwGeBtXMY34T2\n7t3L6Ogog4ODLF++vOPtdJMdO3awefNmhoaGOPvssyfdxqm2vZWxaZTp7+9n//797Nmzhy1btvDK\nV76SFStWMDg4CHCwHYCtW7cCcMEFF0wYT6Ot/v5+du3axb59+w6WWbp0KatWrWLXrl1N22j0tWnT\nJh5++GFWr17NOeecw/79+w+LqdHHVG0tX778iLbLsZbHqJV6U435RO22Mv6txlH19a4aZyccjX/b\nR8jMjj+A04AHJln3WeBdpfkdwIpJyuZc2LjxlqzVluXAwJqs1Zblxo23dLSdbrJ+/dUJtYSXJ9Ty\nkksunXAbp9r2VsamUaZWe1nR50DxfHJCLXt6zsze3iXZ1zeQAwNrsrd3Sfb0LE44NmF19vUNHBFP\no63e3lMTjhn3OLZo+5imbTT6ilhUKntSQi1rtXMOxlTva+q2arVluX791Ye1fah+vc3GGLVSb6ox\nn6jdiTQr3yyOqq931Tg7YT79bRefm9U+u6tWbOdjigRyO/Dq0vxdwJpJyrZnRJsYGxvLWm1Zwv0J\nmXB/1mrLcmxsrCPtdJPt27cXH+CNbdo0br6+jdu3b2+67a2MzaEymxKWJfxF0Vdj/v6EsYTjS9MD\npfmJ4mnU3ZSwtCg/UKrXWN6sjUZfSxIWleqNj6nVtsaP49gEbdbrLlq0tIV6U435ke1O9L5sVr55\nHNVe71b67Za/n/n2tz2TBNINh7DaasOGDQenh4eHGR4ebmv7o6Oj9PUN8uyz5xZLzqW39zRGR0en\ntZvarna6yebNm4FVQGObFgMrS/P1bdy8eXPTbW9lbA6VWQwMAo8UfTXmzwW2AC8tTZ9UrD/U7oIF\nK0vxNOouBlaM27rFpeXN2mj0dXyx/oRJYmq1rUbfqyaoP3hY3Z6eE4HauPEfX2+qMT+y3Ynel83K\nN4+jWd9T/y1MN85O6Pa/7ZGREUZGRtrTWNXM084H0zuEtZMOHsJyD2Ry7oG4B+IeyPz72+YoOIQ1\nCGybZN1bgG8W02uBe5u0054RnULj+OZxx13QlnMgM22nm6xf/9Hiw+KMhFq+6U2XTriNU217K2PT\nKLNo0WDR55LiuX6+oafn5dnb2599fQN53HEXZG9vf/b0HJv1cw6nT3jOodFWb++qhL488hzIimL5\n5G00+oo4plR2RdaP17/iYEz1vqZuq34O4aOHtX2ofr3N8ecemtWbaswnancizco3i6Pq6101zk6Y\nT3/bM0kgUa/fORGxERgGXgLsAa4D+qhv1I1FmRuAdcAzwJWZ+YNJ2sq52h6vwpqcV2F5FdZUcXgV\nVveICDIzKtXtdAJpp7lMIJJ0NJhJAun4jYSSpPnJBCJJqsQEIkmqxAQiSarEBCJJqsQEIkmqxAQi\nSarEBCJJqsQEIkmqxAQiSarEBCJJqsQEIkmqxAQiSarEBCJJqsQEIkmqxAQiSarEBCJJqsQEIkmq\nxAQiSarEBCJJqsQEIkmqxAQiSarEBCJJqqTjCSQi1kXEzoh4MCKumWD9cRFxW0TcFxHbIuKDHQhT\nkjROZGbnOo9YADwIvAH4KbAFuCIzd5bKfBw4LjM/HhEnAD8CVmTmixO0l53cHkmabyKCzIwqdTu9\nBzIEPJSZj2XmC8AtwOXjyiSwpJheAjwxUfKQJM2tTieQU4BdpfmfFMvKbgB+OSJ+CtwPXD1HsUmS\nmljY6QBa8GZga2a+PiJOB74dEedm5v6JCm/YsOHg9PDwMMPDw3MSpCTNByMjI4yMjLSlrU6fA1kL\nbMjMdcX8tUBm5qdLZb4B/GFm/p9i/m+AazLz/07QnudAJGka5vM5kC3A6og4LSL6gCuA28aVeQx4\nI0BErABeDjwyp1FKko7Q0UNYmXkgItYDd1JPZjdl5o6IuKq+Om8Efg/4i4h4oKj27zLzZx0KWZJU\n6OghrHbzEJYkTc98PoQlSZqnTCCSpEpMIJKkSkwgkqRKTCCSpEpMIJKkSkwgkqRKTCCSpEpMIJKk\nSkwgkqRKTCCSpEpMIJKkSkwgkqRKTCCSpEpMIJKkSkwgkqRKTCCSpEpMIJKkSkwgkqRKTCCSpEpM\nIJKkShY2WxkR24CcbH1mntv2iCRJ80LTBAJcVjz/dvH8peL5vbMTjiRpvojMSXcwDhWK2JqZF4xb\n9oPMXDPjACLWAX9M/XDaTZn56QnKDAN/BPQCezPz4knayla2R5JUFxFkZlSp2+o5kIiIi0ozr55G\n3WaNLgBuAN4M/Arw7og4a1yZAeDPgMsy8xXAv5ppv5KkmZvqEFbDh4DPFx/mATwJ/EYb+h8CHsrM\nxwAi4hbgcmBnqcx7gFszczdAZv59G/qVJM1QSwkkM78PnFckEDLzqTb1fwqwqzT/E+pJpezlQG9E\nbAL6gesz80tIkjpqqquwfmeS5QBk5n+ehZjGWwisAV4PLAbuiYh7MvPhiQpv2LDh4PTw8DDDw8Nz\nEKIkzQ8jIyOMjIy0pa2mJ9Ej4rpmlTPzUzPqPGItsCEz1xXz19abPXQiPSKuARY1+oqIzwF3ZOat\nE7TnSXRJmoaZnERv6Sqs2RIRPcCPgDcAfwdsBt6dmTtKZc4C/hRYBxwDfA94V2Zun6A9E4gkTcOs\nX4UVESsj4msRMVY8bo2IlVU6LMvMA8B64E7gh8AtmbkjIq6KiN8syuwE/hp4ALgXuHGi5CFJmlut\n3gfybWAjh24kfB/w3sy8ZBZjmzb3QCRpemb9EFZE3JeZ50+1rNNMIJI0PXNxI+ETEfG+iOgpHu8D\nnqjSoSTp6NBqAvkN4J3A49RPdr8DuHK2gpIkdb+OXoXVbh7CkqTpmckhrJbuRI+I5cBHgMFyncxs\nx9eZSJLmoVa/C+vrwP8G7gIOzF44kqT5ovJVWN3IQ1iSND1zcRXWNyLiLVU6kCQdnab6Lqynqf+k\nbVD/IsPni0dQ/86q4+YiyFa5ByJJ0zNrJ9Ezc0m1kCRJR7tWvwsrihsJP1nMr4qI8b/bIUn6BdLq\nOZDPAK+i/uuAAPup/8ysJOkXVKuX8V6YmWsiYitAZj4ZEX2zGJckqcu1ugfyQvHbHQkHbyz8+axF\nJUnqeq0mkOuBrwEnRsTvA3cDfzBrUUmSul7L34VV/DLgG6hfwvs35V8N7BZexitJ0zMXv0h4OvBo\nZv4Z8P+ASyJiaZUOJUlHh1YPYd0KHIiI1cB/AVZR/4VCSdIvqFYTyM8z80Xg14AbMvNjwMmzF5Yk\nqdtN5yqsdwMfAL5RLOudnZAkSfNBqwnkSuo3Ev5+Zj4aES8FvjR7YUmSup2/SChJv8Dm4hcJH6W4\nibAsM19WpVNJ0vzX6iGsfw68sni8lvqNhX/ZjgAiYl1E7IyIByPimiblXhkRL0TEr7WjX0nSzFQ+\nhBUR38/MfzajziMWAA9Sv0Hxp8AW4IrM3DlBuW8DzwKfz8y/mqQ9D2FJ0jTMxSGsNaXZBdT3SFr9\nIsZmhoCHMvOxop9bgMuBnePK/Wvgq9T3gCRJXaDVJPCfStMvAqPAO9vQ/ynArtL8T6gnlYMi4peA\nt2Xmxf4GiSR1j5YSSGZePNuBNPHHQPncSNNdrQ0bNhycHh4eZnh4eFaCkqT5aGRkhJGRkba01dI5\nkIgYAK4DfrVY9D+B383Mp2bUecRaYENmrivmr6X+W+ufLpV5pDEJnAA8A/xmZt42QXueA5GkaZjJ\nOZBWE8it1L9E8eZi0fuB8zJzRldEFb8x8iPqJ9H/DtgMvHuyb/qNiC8At3sSXZLaY9ZPogOnZ+bb\nS/Ofioj7qnRYlpkHImI9cCf1k/M3ZeaOiLiqvjpvHF9lpn1Kktqj1T2Qe4CPZebdxfxFwH/MzFfN\ncnzT4h6IJE3PXOyB/BZwc3EuBOBJ4NerdChJOjq0ugdyDPAO4HRgKfAU9UNMvzu74U2PeyCSND1z\nsQfydWAf8ANgd5WOJElHl1YTyMrGpbaSJEHrX6b43Yg4Z1YjkSTNK03PgUTENuqXzi4EzgAeAZ6j\nflNfZua5cxFkqzwHIknTM5vnQC6r0qgk6ejnLxJK0i+wmeyBtHoORJKkw5hAJEmVmEAkSZWYQCRJ\nlZhAJEmVmEAkSZWYQCRJlZhAJEmVmEAkSZWYQCRJlZhAJEmVmEAkSZWYQCRJlZhAJEmVmEAkSZV0\nPIFExLqI2BkRD0bENROsf09E3F887vandSWpO3T0B6UiYgHwIPAG4KfAFuCKzNxZKrMW2JGZT0XE\nOmBDZq6dpD1/UEqSpmE+/6DUEPBQZj6WmS8AtwCXlwtk5r2Z+VQxey9wyhzHKEmaQKcTyCnArtL8\nT2ieID4M3DGrEUmSWrKw0wG0KiIuBq4EXtOs3IYNGw5ODw8PMzw8PKtxSdJ8MjIywsjISFva6vQ5\nkLXUz2msK+avBTIzPz2u3LnArcC6zPzbJu15DkSSpmE+nwPZAqyOiNMiog+4AritXCAiTqWePN7f\nLHlIkuZWRw9hZeaBiFgP3Ek9md2UmTsi4qr66rwR+CSwDPhMRATwQmYOdS5qSRJ0+BBWu3kIS5Km\nZz4fwpIkzVMmEElSJSYQSVIlJhBJUiUmEElSJSYQSVIlJhBJUiUmEElSJSYQSVIlJhBJUiUmEElS\nJSYQSVIlJhBJUiUmEElSJSYQSVIlJhBJUiUmEElSJSYQSVIlJhBJUiUmEElSJSYQSVIlJhBJUiUm\nEElSJR1PIBGxLiJ2RsSDEXHNJGWuj4iHIuK+iDh/rmOUJB1pYSc7j4gFwA3AG4CfAlsi4uuZubNU\n5lLg9Mw8IyIuBD4LrJ3rWPfu3cvo6CiDg4MsX758WnV37NjB5s2bGRoa4oQTTjisnUa7/f397N+/\n/4j29+7dy9atW9m3bx9Lly7lggsuAGDr1q0ALF68mIcffpjVq1fzzDPPHCy3atUqtm3bxp49e3jj\nG98IwF133UWtVgNgbGyM1atXc/HFFx+xPeV4zz777MNiAFi6dCl79+7lK1/5Cv39/bz1rW/lscce\nY2xsjN27d7Nz506WLFnCa1/7WgYGBnj00Ue555576O/vZ9myZTz33HPs3r2bp59+mjPPPJOBgYGD\nMf34xz9maGiI4eFh1qxZw+7du7n99tt58MEHOfPMM7nssst44okn+PKXv8zy5cv50Ic+xOOPP87n\nPvc5du3axdq1aznrrLMAOO+88zjnnHPYtm0b3/3ud3nqqae46KKLuOiii9i/f//BMX/++ecPjmFf\nX9/B5eOfy69NY0wAVq1aNeFr14p2tSN1RGZ27EE9EdxRmr8WuGZcmc8C7yrN7wBWTNJezoaNG2/J\nWm1ZDgysyVptWW7ceEvLddevvzqhlvDyhGNy4cIlB9tZv/7qrNWWZa32soRa1mrnHNb+xo23ZG/v\nkoRjEo5NWJ09PYuzp2dxMX980fYvFWUa5U5O6C3WnVFMN9b3lsqdngsXLjlsew6Pt5aXXHJpKYZG\nvVrCwuL5+KLN3oSe4lHup6co23g0yvSWlvWOq1crbUNPHh7zMaW+Ty612ahbLntSqZ/yslr29p6a\nUMsFC5aXxrCWPT0rD1vfeC6/Nodel8ZYH/natfq+akc70kwUn5vVPsOrVmzHA3g7cGNp/n3A9ePK\n3A68ujR/F7BmkvbaNaYHjY2NZa22LOH+hEy4P2u1ZTk2NjZl3e3btxcfTvcnjBUfto12NhXrNiUc\n2f727dtz0aKlCQOlemOl+a+V6pfLbUo4rtTv9oRFxfolxfTxh/W3aNHxOTY2Ni7eRoyNuo32r0/o\nK8p9rZRYGolgUamfRUXZxqM3YUFRvrHsmIQo1Wts03GlD/5GzL9f6ntTQv+4uuWyjTb6xi1bVnou\nj+GyJs/lsVpavC7HT7i+1ffG2NhYW9qRZmomCaSjh7Bmw4YNGw5ODw8PMzw8PKP2RkdH6esb5Nln\nzy2WnEtv72mMjo5OeZhh8+bNwCrgXGAL8NJiGmBxsW4xMFhaXm9/8+bN9PSsKJVttHFSMf9UqX65\n3GJgKbCoqHMzcCJQA54typ1wWH8RpzA6Osr27dtL8TbaWw4cW5q/v3g+oYhhabFuP/VTakuBKJY9\nUfTb8CLwT8BAMZ1AL/Czop8o4m5sw4FielHR3/dKfTe29dlS3eeAY0rrlxZxNbajMdaN5/IYDjZ5\nPjRWPT0ncuBAAksmXN/qe2N0dLR4fRvbUa0dabpGRkYYGRlpS1udTiC7gVNL8yuLZePLrJqizEHl\nBNIOg4ODPP/8KPAA9T/wB3jhhccYHBycsu7Q0BCwq6g7CDxaaueZYt0zwJHtDw0NceDAHuofsgtK\nbTxezA+U6pfLPQPsA14o6gwBY9Q/WH9eLH/6sP4ydzM4OEh/f38p3kaMe4u6FO1/EPgShxLBvmLd\nAeof4nupJ4UXiuXPlEYki3JPFdMUdQ6U6r1Y2obnivl/KGJ+O/Ctou9nisfPS3WfL9p7utTGPwH/\nSD2RNMa68Vwew9Emz4fG6sCBsSLuv59wfavvjcHBweL1jRm1I03X+H+sP/WpT1VvrOquSzseQA/w\nMHAa0AfcB5w9rsxbgG8W02uBe5u01779upLGOZDjjrugwjmQj+ahcxF9uXDhkoPtrF//0azVluWi\nRYNZP/79ignOgfRn+ZxFT8+x2dNzbDE/kIfOBfSVyq3IQ+cJVhfTjfW9xfRk50DK8dbyTW+6tBRD\n+TxET1FuaR46lxHFIaq+Uj+Rh8579BTrIw8/H9Ko26hXy/q5ioWlso2Ye0t9n1RMl+uWy64o9VNe\nVsve3lVZPwfyktIY1rKn55TD1jeey6/NodflUHvjX7tW31ftaEeaCWZwCCvq9TsnItYBf0L939ub\nMvM/RMRVxUbdWJS5AVhH/V+1KzPzB5O0lbO1PV6F5VVYXoWlo1FEkJkxdckJ6nY6gbTTbCYQSToa\nzSSBdPxGQknS/GQCkSRVYgKRJFViApEkVWICkSRVYgKRJFViApEkVWICkSRVYgKRJFViApEkVWIC\nkSRVYgKRJFViApEkVWICkSRVYgKRJFViApEkVWICkSRVYgKRJFViApEkVWICkSRVYgKRJFViApEk\nVdKxBBIRx0fEnRHxo4j464gYmKDMyoj4TkT8MCK2RcRHOxGrJOlIndwDuRa4KzPPBL4DfHyCMi8C\nv5OZvwK8CvjtiDhrDmNsu5GRkU6H0BLjbC/jbC/j7A6dTCCXAzcX0zcDbxtfIDMfz8z7iun9wA7g\nlDmLcBbMlzeUcbaXcbaXcXaHTiaQEzNzD9QTBXBis8IRMQicD3xv1iOTJE1p4Ww2HhHfBlaUFwEJ\n/PsJimeTdvqBrwJXF3sikqQOi8xJP7dnt+OIHcBwZu6JiJOATZl59gTlFgLfAO7IzD+Zos3ObIwk\nzWOZGVXqzeoeyBRuAz4IfBr4deDrk5T7PLB9quQB1QdBkjR9ndwDWQb8N2AV8BjwzszcFxEnA3+e\nmZdFxEXA/wK2UT/ElcAnMvNbHQlaknRQxxKIJGl+m3d3okfEuojYGREPRsQ1k5S5PiIeioj7IuL8\nuY6xiKFpnBHxuojYFxE/KB4TXVgw2zHeFBF7IuKBJmW6YSybxtkNY1nE0dKNr50e01bi7PSYRsQx\nEfG9iNhaxHjdJOU6PZZTxtnpsRwXy4IihtsmWT+98czMefOgnvAeBk4DeoH7gLPGlbkU+GYxfSFw\nb5fG+Trgtg6P52uoXxr9wCTrOz6WLcbZ8bEs4jgJOL+Y7gd+1KXvz1bi7PiYAscWzz3AvcBQt41l\ni3F2fCxLsfxb4C8niqfKeM63PZAh4KHMfCwzXwBuoX5DYtnlwBcBMvN7wEBErGButRIn1C9r7pjM\nvBt4skmRbhjLVuKEDo8ltHzja8fHtMU4ofPvz38sJo+hfsHP+OPtHR/Lou+p4oQueH9GxErgLcDn\nJiky7fGcbwnkFGBXaf4nHPnGH19m9wRlZlsrcQK8qthV/GZE/PLchDYt3TCWreqqsWxy42tXjekU\nN+h2dEyLwy1bgceBb2fmlnFFumIsW4gTuuP9+UfAx5j8nrtpj+d8SyBHk+8Dp2bm+cANwH/vcDzz\nWVeN5Xy58XWKODs+ppn588y8AFgJXNgN/xhMpIU4Oz6WEfEvgD3FnmfQpj2i+ZZAdgOnluZXFsvG\nl1k1RZnZNmWcmbm/seubmXcAvcWlzd2kG8ZySt00lsWNr18FvpSZE93b1BVjOlWc3TSmmfkPwCZg\n3bhVXTGWDZPF2SVjeRHwLyPiEeArwMUR8cVxZaY9nvMtgWwBVkfEaRHRB1xB/YbEstuADwBExFpg\nXxbfuTWHpoyzfGwxIoaoX1L9s7kNs949k/830g1j2TBpnF00ljD1ja/dMqZN4+z0mEbECVH8xENE\n1IBLgJ3jinV8LFuJs9NjCZCZn8jMUzPzZdQ/j76TmR8YV2za49nJO9GnLTMPRMR64E7qye+mzNwR\nEVfVV+eNmfk/IuItEfEw8AxwZTfGCbwjIn4LeAF4FnjXXMcZERuBYeAlEfFj4Dqgjy4ay1bipAvG\nsojzIuC9wLbimHgCn6B+NV7XjGkrcdL5MT0ZuDkiFlD/G/qvxdh11d96K3HS+bGc1EzH0xsJJUmV\nzLdDWJKkLmECkSRVYgKRJFViApEkVWICkSRVYgKRJFViApHmUEQ82oXfOCBVYgKR5pY3XumoYQKR\nZklEfC0ithQ/NPThxuKOBiW10bz6KhNpnrkyM/dFxCJgS0T8VacDktrJBCLNnn8TEW8rplcCZ+Ah\nLB1FTCDSLIiI1wGvBy7MzOciYhOwqMNhSW3lORBpdgwATxbJ4yxgbbHccyA6aphApNnxLeo/HPRD\n4A+A7xbLPYSlo4Zf5y5JqsQ9EElSJSYQSVIlJhBJUiUmEElSJSYQSVIlJhBJUiUmEElSJSYQSVIl\n/x8JyZImob0tlwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f442dcf6e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(glass.al, glass.household)\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's draw a **regression line**, like we did before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# fit a linear regression model and store the predictions\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.household\n",
    "linreg.fit(X, y)\n",
    "glass['household_pred'] = linreg.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f442dcf6ba8>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcVOWV//HPoemGErARRTQBbRVFTABBbVBMbKMSXBLN\noiYZXybGqDE/RrPMjE4mM2q2ySyJWzRKJMYlxCSTuMWYKEq7QxNZE0Ak2oCoNFFANgG7z++P5xZc\nml6qq6vqVlV/369XvWq5z33uuU919+lzn1u3zN0RERHpql5JByAiIqVJCURERLKiBCIiIllRAhER\nkawogYiISFaUQEREJCuJJxAzm2Zma8xsYTvLTzKz9WY2N7p9q9AxiojInnonHQBwJ3AzcHcHbZ52\n948XKB4REclA4hWIuz8LrOukmRUiFhERyVziCSRDx5vZfDN7xMyOSjoYEREpjkNYnXkROMjdt5jZ\n6cADwBEJxyQi0uMVfQJx902xx4+a2a1mNsjd327d1sx0YS8RkS5y96ymCYrlEJbRzjyHmQ2JPa4F\nrK3kkebuRX275pprEo9BcSpOxak407fuSLwCMbPpQB2wr5mtBK4BqgB396nAp83scmAHsBU4P6lY\nRURkl8QTiLt/rpPltwC3FCgcERHJULEcwuox6urqkg4hI4oztxRnbinO4mDdPQZWTMzMy2l/RETy\nzczwEp9EFxGREqMEIiIiWVECERGRrCiBiIhIVpRAREQkK0ogIiKSFSUQERHJihKIiIhkRQlERESy\nogQiIiJZUQIREZGsKIGIiEhWlEBERCQrSiAiIpIVJRAREcmKEoiIiGRFCURERLKiBCIiIllRAhER\nyYf165OOIO+UQEREcqm5Gf77v+EDH4B33kk6mrzqnXQAIiJlY+lS+MIXoF8/eO452HvvpCPKK1Ug\nIiLd1dwM//M/cOKJcOGF8PjjUFOTdFR5pwpERKQ7li6Fiy6Cvn1hzhw45JCkIyoYVSAiItlobob/\n/d9QdVxwATzxRI9KHqAKRESk6156KVQdVVXQ0ACHHpp0RIlQBSIikqnmZvjhD2HiRPjc5+DJJ3ts\n8gBVICIimVm2LFQdvXvD7Nlw2GFJR5Q4VSAiIh1pbobrr4cTToDzz4eZM5U8IqpARETa8/LLoero\n1QtmzYLhw5OOqKioAhERaa2lBW64AY4/Hs47D+rrlTzaoApERCRu+fJQdYCqjk6oAhERgVB13Hgj\nTJgAn/oUPPWUkkcnVIGIiCxfDl/8YkgiL7wAhx+edEQlQRWIiPRcLS1w002h6vjEJ0LVoeSRMVUg\nItIz/e1voep47z14/nk44oikIyo5qkBEpGdpaYGbb4bx4+Hss+Hpp5U8sqQKRER6jldeCVXH9u3h\n+zpGjEg6opKmCkREyl9LC9xyC9TWwllnwTPPKHnkgCoQESlvr74aqo5334Vnn4Ujj0w6orKhCkRE\nylNLC9x6a6g6zjxTySMPVIGISPlpbAxVx5Yt4XCVEkdeqAIRkfLR0gI/+QkcdxxMnhwmypU88kYV\niIiUh8ZGuPhi2LQpnJo7cmTSEZU9VSAiUtrc4bbbQtUxaVKoOpQ8CkIViIiUrhUrQtXxzjvhMiRH\nHZV0RD1K4hWImU0zszVmtrCDNjeZ2ctmNt/Mji5kfCJShNzh9tvh2GPh1FPDpUiUPAquGCqQO4Gb\ngbvbWmhmpwOHufvhZjYeuA2YUMD42rR27VoaGxupqalh8ODBifdTTJYsWUJDQwO1tbWMHDmy3X3s\nbN8zGZt0m/79+7Np0ybWrFnDnDlzOO644xgyZAg1NTUAO/sBmDdvHgBjx45tM550X/3792fVqlWs\nX79+Z5uBAwcybNgwVq1a1WEf6W3NnDmT5cuXM3z4cEaNGsWmTZt2iym9jc76Gjx48B59x2ONj1Em\n63U25m31m8n4ZxpHtu83ACtXsv3CC9nR1MSb06bx9oEHUrNuXdH9/pTj7/Ye3D3xG3AwsLCdZbcB\n58eeLwGGtNPWC2H69Ps8lRrk1dXjPJUa5NOn35doP8VkypQrHVIORzik/LTTTm9zHzvb90zGJt0m\nlTo02mZ1dH+gQ8orKkZ4ZeUAr6qq9urqcV5ZOcArKvo57OUw3KuqqveIJ91XZeVBDn1a3faK+u7T\nYR/pbZn1jbU9wCHlqdSonTGFbXXeVyo1yKdMuXK3vnetH/pMj1Em63U25m3125aO2ncUR7bvt7e0\nuE+d6lsHDPD/6J3y/n0PySjOJJTS73b0dzO7v93ZrpjLWycJ5GHghNjzGcC4dtrmZkQ70NTU5KnU\nIIcFHuroBZ5KDfKmpqZE+ikmixcvjv6Ap/dpZqvnYR8XL17c4b5nMja72sx0GOTw82hb6ecLHJoc\n9ok9ro49byue9LozHQZG7atj66Vf76iP9LYGOPSNrdc6pkz7aj2OTW30Gdbt23dgBut1NuZ79tvW\nz2VH7TuOI7v321escJ80ybePGePH9Nk74ziTUGq/291JIMVwCCunrr322p2P6+rqqKury2n/jY2N\nVFXVsHXr6OiV0VRWHkxjY2OXytRc9VNMGhoagGFAep/6AUNjz8M+NjQ0dLjvmYzNrjb9gBrglWhb\n6eejgTnAIbHHB0TLd/Xbq9fQWDzpdfsBQ1rtXb/Y6x31kd7WPtHy/dqJKdO+0tse1sb6NbutW1Gx\nP5BqNf6t1+tszPfst62fy47adxxHR9tuY/l++8G0afCv/wpf+xrzTz6Z5adPgW2ZxZmEYv/drq+v\np76+PjedZZt5cnmja4ewlpLgISxVIO1TBaIKJJcVyN/nzXP/6EfdjznGfdGirOJMQqn9blMGh7Bq\ngEXtLDsDeCR6PAGY1UE/uRnRTqSPb+6999iczIF0t59iMmXKFdEfi8MdUj5p0ult7mNn+57J2KTb\n9O1bE21zQHQf5hsqKo7wysr+XlVV7XvvPdYrK/t7RcVeHuYcDmtzziHdV2XlMIcq33MOZEj0evt9\npLdl1ifWdoiH4/Uf3BlT2FbnfYU5hCt263vX+qHP1nMPHa3X2Zi31W9bOmrfURydvt999/FZl1zq\nvt9+7t/5jvv27d2KMwml9LvdnQRiYf3kmNl0oA7YF1gDXANUEXZqatTmx8BkYDNwkbvPbacvL9T+\n6Cys9uksLJ2F1Vkc7W179ezZHHX99VStWwc//zmMHr1Hu2ziTEKp/G6bGe5uWa2bdALJpUImEBHJ\nIXe480646iq44gq4+mqorEw6qh6hOwmk7CbRRaTEvPYaXHopvPEGPPFEu1WHFJ/EP4kuIj1UuuoY\nOxYmTICGBiWPEqMKREQKb/XqUHW8/jrMmAFjxiQdkWRBFYiIFI473HVXqDqOOw5mz1byKGGqQESk\nMF5/PVQdr70Gjz0GR+u6qKVOFYiI5Jc73H13SBjHHhvmOpQ8yoIqEBHJn9dfh8sug5Ur4U9/Coeu\npGyoAhGR3HOHe+4JlcbYsTBnjpJHGVIFIiK59cYboepobIQ//hHGjUs6IskTVSAikhvucO+94ayq\nMWPgz39W8ihzqkBEpPvefDNUHa+8Ao8+Cscck3REUgCqQEQke+7wi1+EimPUqFB1KHn0GKpARCQ7\nb74Jl18OL78MjzwSTtGVHkUViIh0jTv88peh6hg5El58Ucmjh1IFIiKZW7MmVB0vvQS//324HIn0\nWKpARKRz7nDffaHqOPJImDtXyUNUgYhIJ9asga98BZYsgYcegtrapCOSIqEKRETa5g6/+lWoOg4/\nPFQdSh4SowpERPbU1BSqjr/+FR58EMaPTzoiKUKqQERkd7/+dfhmwMMOg3nzlDykXapARCRYuzZU\nHX/5CzzwQPiaWZEOqAIREfjNb8InyQ85JMx1KHlIBlSBiPRka9fClCmwYAHcfz8cf3zSEUkJUQUi\n0lP99rdhrmPYsDDXoeQhXaQKRKSn+fvfQ9Uxb15IIieckHREUqJUgYj0JL/7Xag6hg6F+fOVPKRb\nVIGI9AR//zv84z+GCx/+5jcwcWLSEUkZUAUiUu7uvz9UHQceGKoOJQ/JEVUgIuXqrbdC1TFnTvhw\n4IknJh2RlJkOE4iZLQK8veXuPjrnEYlI9z3wQPhQ4Pnnh1N099or6YikDHVWgZwV3f+/6P6e6P4f\n8hOOiHTLW2/BFVdAQ0O4EOKHPpR0RFLGzL3dAmNXI7N57j621Wtz3X1c3iLLgpl5JvsjUpYefDBU\nHeeeC9//vqoOyYiZ4e6WzbqZzoGYmU109+eiJyegCXiR4vD223DllfDCC+GrZj/84aQjkh4i0yRw\nMXCrmTWa2QrgVuCL+QtLRDLy8MPhGlb77BPmOpQ8pIAyOoS1s7FZNYC7b8hbRN2gQ1jSY6xbF6qO\n556Dn/0MTjop6YikROXtEJaZfb29DQK4+4+y2aiIdMPvfw9f/jJ88pOwcCH065d0RNJDdTYHMqAg\nUYhI59atg69+FZ59Fu69F+rqko5IergOE4i7X1eoQESkA488ApddBuecE+Y6+vdPOiKRzM7CMrOh\nwM1A+hoIzwBXuvtr+QpMRID160PV8fTTcM89cPLJSUckslOmZ2HdCTwEvC+6PRy9JiL58oc/hDOs\n+vULcx1KHlJkMv0g4Xx3P7qz15Kms7CkLKxfD1/7GtTXw7Rp8JGPJB2RlLHunIWVaQXylpldYGYV\n0e0C4K1sNigiHXj00VB1pFKh6lDykCKWaQVyMGEO5HjCxRWfB65w95X5Da9rVIFIydqwAb7+dXji\niVB1nHJK0hFJD5H3S5m4+wrg49lsQEQ68ac/wSWXwBlnwKJFMEBnz0tpyPQsrMHAJUBNfB131+VM\nRLK1YQN84xswY0b4NPmppyYdkUiXZHoxxQcJp+7OAJrzF45ID/HYY/ClL6nqkJKW9VlYxUhzIFL0\n3nknVB2PPQZ33AGnnZZ0RNLDFeIsrN+b2RnZbKAzZjbZzJaa2TIzu6qN5SeZ2XozmxvdvpWPOETy\n7rHHwhlWZqHqUPKQEtdhBWJmGwlnXRnQD9ge3Qxwd9+7Wxs36wUsA04BXgfmAJ9x96WxNicB33D3\nTifxVYFIUXrnHfinfwqT5T/9KUyalHREIjvlrQJx9wHuvnd038vd+8aedyt5RGqBl919hbvvAO4D\nzm6jXVY7J5K4xx8PVYd7+FyHkoeUkYwOYVlwgZn9e/R8mJnV5mD77wdWxZ6/Fr3W2vFmNt/MHjGz\no3KwXZH82rgxXPzw4oth6tRQeVRXJx2VSE5lehbWrUAL8BHgO8Am4BbguDzFFfcicJC7bzGz04EH\ngCPaa3zttdfufFxXV0edLnkthTZjRjjD6tRTw1yHEocUkfr6eurr63PSV6ZnYc1193FmNs/dx0av\nLXD3Md3auNkE4Fp3nxw9v5owt/JfHazzKnCMu7/dxjLNgUhyNm6Ef/mX8IVPU6fC6acnHZFIpwpx\nFtYOM6sgTKinP1jYks0GW5kDDDezg82sCvgM4aq/O5nZkNjjWkLS2yN5iCTqySdh9GjYti1UHUoe\n0gNkegjrJuB+YH8z+x7waaDbp9O6e7OZTQEeIySzae6+xMwuC4t9KvBpM7sc2AFsBc7v7nZFcmbT\nplB1PPww3H57+GCgSA+R0SEsADM7knC6rQFPuPuSfAaWDR3CkoKaOTNMktfVwY9+BAMHJh2RSJd1\n5xBWpnMghwGvufs2M6sDRgN3u/v6bDaaL0ogUhCbNsFVV8GDD4a5DlUdUsIKMQfyW6DZzIYDtwPD\ngOnZbFCkpNXXh7mOzZvDXIeSh/Rgmc6BtLj7e2b2SeDH7n6zmc3LZ2AiRWXTJrj6anjggTDXceaZ\nSUckkriunIX1WeBC4PfRa5X5CUmkyDz1FIwZE07TXbRIyUMkkmkFchHwZeB77v6qmR0C3JO/sESK\nwObNoer43e/gttvgYx9LOiKRopLxWVilQJPokjNPPw0XXQQTJ8INN8CgQUlHJJIXef9K2+jT33v8\nZXb3Q7PZqEjR2rwZvvlN+L//g5/8BD6ub3IWaU+mh7COjT3uC5wL6F8yKS/PPBOqjgkTwlyHqg6R\nDmV9CMvMXnT3Y3IcT7foEJZkZcuWUHX8+teh6ji7rW8UEClPhTiENS72tBehIsm0ehEpXs8+G6qO\n8ePhL39R1SHSBZkmgR/GHr8HNALn5TwakULZsgX+7d/gV7+CW2+Fc85JOiKRkpNRAnH3k/MdiEjB\nPPdcqDqOPTbMdey7b9IRiZSkTA9hVQPXAB+OXnoK+La7b8hXYCI5t2ULfOtbcN99cMst8IlPJB2R\nSEnL9JPoPwM2Eg5bnQe8A9yZr6BEcu755+Hoo+GNN8J3kyt5iHRbplfjne/uR3f2WtJ0FpbsYevW\nUHVMnx6qjk9+MumIRIpKIa7Gu9XMToxtcCLhy51EitcLL4SqY/XqMNeh5CGSU5mehXU5cFc0FwKw\nDvh8fkIS6aatW+E//gPuvRduvhk+/emkIxIpS5kmkCXAfwOHAQOBDcA5wMI8xSWSnVmz4AtfCN/Z\nsXAhDB6cdEQiZSvTBPIgsB6YC6zOXzgiWXr33VB13H13qDrOPTfpiETKXqYJZKi7T85rJCLZmj07\nVB2jRoW5DlUdIgWR6ST682Y2Kq+RiHTVu++G7yY/+2y47rpwLSslD5GC6bACMbNFhMu49wYuMrNX\ngG2AAe7uo/MfokgbGhpC1XHUUWGuY//9k45IpMfp7BDWWQWJQiRT774L114LP/853HgjnHceWFan\nsItIN3WYQNx9RaECEenUnDmh6jjySFiwAIYMSToikR5Nl2SX4rdtW6g6fvazUHWcf76qDpEioAQi\nxS1ddYwYEeY6VHWIFI1Mz8ISKaxt28L3dZx1Vrj/7W+VPESKjCoQKT4vvhiqjsMOC3MdBxyQdEQi\n0gZVIFI8tm0LV8494wy4+mq4/34lD5EipgpEisPcufD5z6vqECkhqkAkWdu3w7//O0yeHD5VrqpD\npGSoApHkzJ0b5jpqakLVceCBSUckIl2gCkQKb/v2cOXcyZPhn/8ZHnxQyUOkBKkCkcKaNy9UHQcd\nBPPnw/vel3REIpIlVSBSGNu3wzXXwEc/Ct/4Bjz0kJKHSIlTBSL5N39+qDqGDg0VyPvfn3REIpID\nqkAkf3bsCN/TMWkSfPWr8PDDSh4iZUQViOTHggWh6jjwQFUdImVKFYjk1o4d8O1vw2mnwRVXwCOP\nKHmIlClVIJI7CxeGquOAA8JnPIYOTToiEckjVSDSfTt2wHe+A6ecAlOmhKpDyUOk7KkCke5ZtChU\nHYMHh6pj2LCkIxKRAlEFItnZsQO++134yEfgK1+BRx9V8hDpYVSBSNf95S/hyrn77aeqQ6QHUwUi\nmXvvPfje9+Dkk+Hyy+GPf1TyEOnBVIFIZv761zDXsc8+4RsDDzoo6YhEJGGqQKRj770H//mfUFcH\nl1wCf/qTkoeIAEWQQMxsspktNbNlZnZVO21uMrOXzWy+mR1d6Bh7rMWL4YQT4Mkn4c9/hksvBbOk\noxKRImHuntzGzXoBy4BTgNeBOcBn3H1prM3pwBR3P9PMxgM3uvuEdvrzfO3P2rVraWxspKamhsGD\nB3dp3SVLltDQ0EBtbS377bffbv2k++3fvz+bNm3ao/+1a9cyb9481q9fz8CBAxk7diwA8+bNA6Bf\nv34sX76c4cOHs3nz5p3thg0bxqJFi1izZg2nnnoqADNmzCCVSgHQ1NTE8OHDOfnkk/fYnyWLFrHl\n299mzBNP0PsHP2DtOecwb/581q9fD8DAgQNZu3Ytv/zlL+nfvz8f+9jHWLFiBU1NTaxevZqlS5cy\nYMAAPvShD1FdXc2rr77KCy+8QP/+/Rk0aBDbtm1j9erVbNy4kREjRlBdXb0zppUrV1JbW0tdXR3j\nxo1j9erVPPzwwyxbtowRI0Zw1lln8dZbb/GLX/yCwYMHc/HFF/Pmm29yxx13sGrVKiZMmMCRRx4J\nwJgxYxg1ahSLFi3i+eefZ8OGDUycOJGJEyeyadOmnWO+ffv2nWNYVVW18/XW9/H3Jv2+AAwbNqzN\n9y4TuepHJFtmhrtn95+huyd2AyYAj8aeXw1c1arNbcD5sedLgCHt9Of5MH36fZ5KDfLq6nGeSg3y\n6dPvy3jdKVOudEg5HOHQx3v3HrCznylTrvRUapCnUoc6pDyVGrVb/9On3+eVlQMc+jjs5TDcKyr6\neUVFv+j5PlHf74vapNsd6FAZLTs8epxeXhlrd5j37j1gt/357ucu8FmYP85efhB9/LTTTo/FkF4v\n5dA7ut8n6rPSoSK6xbdTEbVN39JtKmOvVbZaLxXbhwrfPeY+sW0fGOszvW687QGx7cRfS3ll5UEO\nKe/Va3BsDFNeUTF0t+Xp+/h7s+t9SY/1nu9dpj9XuehHpDuiv5vZ/Q3PdsVc3IBPAVNjzy8AbmrV\n5mHghNjzGcC4dvrL1Zju1NTU5KnUIIcFDu6wwFOpQd7U1NTpuosXL47+OC1waIr+2Kb7mRktm+mw\nZ/+LFy/2vn0HOlTH1muKPb8/tn683UyHvWPbXezQN1o+IHq8z27b69t3H296/XVf8/WvexP4Zfyb\nQ0vUV3rddP83OVRF/d8fSyzpRNA3tp2+Udv0rdKhV9Q+/VofB4utl96nvWN/+NMxfy+27ZkO/Vut\nG2+b7qOq1WuDYvfxMRzUwX18rAZG78s+bS7P9GejqakpJ/2IdFd3EkjZnYV17bXX7nxcV1dHXV1d\nt/prbGykqqqGrVtHR6+MprLyYBobGzs9zNDQ0AAMA0YTjs4dEj0G6Bct6wfUxF4P/Tc0NFBRMSTW\nNt3HAdHzDbH14+36AQOBvtE6dwH7Aylga9Ruv922d6TvS+q009jY0sLxHMIKvhvrbzCwV+z5guh+\nvyiGgdGyTYQptYFAuhp+K9pu2nvAu0B19NiBSuDtaDsWxZ3eh+bocd9oe7Nj207v69bYutuAPrHl\nA6O40vuRHuv0fXwMazq43zVWFRX709zswIA2l2f6s9HY2Bi9v+n9yK4fka6qr6+nvr4+J30lnUBW\nA/FTeoZGr7VuM6yTNjvFE0gu1NTUsH17I7CQ8Au+kB07VlBTU9PpurW1tcCqaN0a4NVYP5ujZZuB\nPfuvra2luXkN4Y9sr1gfb0bPq2Prx9ttBtYDO6J1aoEmwh/Wluj1jcBCevEBvsE/8S/b/oZf+APW\nn3kmKz54XKsY10brEvX/BeAediWC9dGyZsIf8bWEpLAjen1zbEQ8archeky0TnNsvfdi+7Atev5O\nFPOngD9G294c3Vpi626P+tsY6+NdYAshkaTHOn0fH8PGDu53vTfNzU1R3H9vc3mmPxs1NTXR+2vd\n6kekq1r/Y33ddddl31m2pUsubkAFsBw4GKgC5gMjW7U5A3gkejwBmNVBf7mr62LScyB77z02izmQ\nK3zXXESV9+49YGc/U6Zc4anUIO/bt8bD8e8PtjEH0t/jcxYVFXt5RcVe0fNq3zUXUBVrN8R3zRMM\njx6nl1c6VPkI+voL9PEnrcIfvOGmduJN+aRJp8diiM9DVETtBvquuQyLDlFV7dxOeK0idusVvRaf\nD0mvm14v5WGuonesbVW07crYtg+IHsfXjbcdEttO/LWUV1YO8zAHsm9sDFNeUfH+3Zan7+Pvza73\nZVd/rd+7TH+uctGPSHfQjUNYiZ6FBeE0XuBGwr+309z9B2Z2WbRTU6M2PwYmE/5Vu8jd57bTl+dr\nf8rlLKy9+vThg48/zqg//IEln/kMB33/+wweMqTdeEeOHLlbDKCzsOLvC+gsLClt3TkLK/EEkkv5\nTCBl4aWX4KKLoE8fmDYNDj006YhEJGHdSSCJf5BQCqC5GX74Q5g4ET73OXjiCSUPEem2pCfRJd+W\nLQtVR2UlzJ4Nhx2WdEQiUiZUgZSr5mb40Y9C1fHZz4bLkSh5iEgOqQIpR8uWwRe/CBUVMGuWEoeI\n5IUqkHLS3AzXXx8ugHjeeTBzppKHiOSNKpBy8fLLYa6jV69QdQwfnnREIlLmVIGUupYWuOEGOP54\nOPdcqK9X8hCRglAFUsqWLw9VB8ALL8Dhhycbj4j0KKpASlFLC9x4I0yYAJ/6VKg6lDxEpMBUgZSa\nv/0tnGHV3KyqQ0QSpQqkVLS0wM03w/jxcPbZ8NRTSh4ikihVIKXglVdC1bFjBzz3HIwYkXREIiKq\nQIpaSwv8+Meh6vj4x+Hpp5U8RKRoqAIpVq+8AhdfDNu2wbPPKnGISNFRBVJsWlrgllugthbOPBOe\neUbJQ0SKkiqQYvLqq2Gu4913Q9URfTGSiEgxUgVSDFpa4NZb4bjj4IwzlDxEpCSoAklaY2OoOrZs\nCYerRo5MOiIRkYyoAklKSwv85Cdw7LEweXKoOpQ8RKSEqAJJwooV4QyrjRvDqblHHZV0RCIiXaYK\npJDc4fbbQ9Vx6qnhQ4FKHiJSolSBFMqKFfClL8GGDeHihx/4QNIRiYh0iyqQfHOHqVND1XHKKfD8\n80oeIlIWVIHk08qVoepYt05Vh4iUHVUg+eAOP/0pHHMM1NWFy64reYhImVEFkmsrV8Ill8Bbb8HM\nmfDBDyYdkYhIXqgCyRV3uOOOUHV8+MOh6lDyEJEypgokF1atClXH2rXw5JMwalTSEYmI5J0qkO5w\nh2nTYNw4OPFEmDVLyUNEegxVINl67bVQdaxZA088AaNHJx2RiEhBqQLpKne4804YOxaOPx5mz1by\nEJEeSRVIV6xeDZdeCq+/DjNmwJgxSUckIpIYVSCZmj49VB3jx0NDg5KHiPR4qkAyVVkJjz0GRx+d\ndCQiIkXB3D3pGHLGzLyc9kdEJN/MDHe3bNbVISwREcmKEoiIiGRFCURERLKiBCIiIllRAhERkawo\ngYiISFaUQEREJCtKICIikhUlEBERyYoSiIiIZEUJREREspLYxRTNbB/gV8DBQCNwnrtvaKNdI7AB\naAF2uHttAcMUEZF2JFmBXA3McPcRwJPAv7bTrgWoc/ex5ZA86uvrkw4hI4oztxRnbinO4pBkAjkb\nuCt6fBdwTjvtjDI61FYqP1CKM7cUZ24pzuKQ5B/m/d19DYC7vwns3047Bx43szlmdknBohMRkQ7l\ndQ7EzB4eISd+AAAE6UlEQVQHhsRfIiSEb7XRvL0v8pjo7m+Y2WBCIlni7s/mOFQREemixL5QysyW\nEOY21pjZAcBMdx/ZyTrXABvd/UftLNe3SYmIdFG2XyiV5FfaPgR8Afgv4PPAg60bmNleQC9332Rm\n/YBJwHXtdZjtIIiISNclWYEMAn4NDANWEE7jXW9mBwI/dfezzOwQ4H7C4a3ewC/c/QeJBCwiIrsp\nq+9EFxGRwim502PNbLKZLTWzZWZ2VTttbjKzl81svpkdXegYoxg6jNPMTjKz9WY2N7q1dWJBvmOc\nZmZrzGxhB22KYSw7jLMYxjKKY6iZPWlmfzWzRWZ2RTvtEh3TTOJMekzNrI+ZzTazeVGM17TTLumx\n7DTOpMeyVSy9ohgeamd518bT3UvmRkh4ywmfXq8E5gNHtmpzOvBI9Hg8MKtI4zwJeCjh8TwROBpY\n2M7yxMcywzgTH8sojgOAo6PH/YGXivTnM5M4Ex9TYK/ovgKYBdQW21hmGGfiYxmL5WvAvW3Fk814\nlloFUgu87O4r3H0HcB/hA4lxZwN3A7j7bKDazIZQWJnECeG05sR4OB16XQdNimEsM4kTEh5LCJ9n\ncvf50eNNwBLg/a2aJT6mGcYJyf98boke9iHMgbY+3p74WEbb7ixOKIKfTzMbCpwB3NFOky6PZ6kl\nkPcDq2LPX2PPH/zWbVa30SbfMokT4PioVHzEzI4qTGhdUgxjmamiGkszqyFUTbNbLSqqMe0gTkh4\nTKPDLfOAN4HH3X1OqyZFMZYZxAnF8fN5PfDPtP+Zuy6PZ6klkHLyInCQux8N/Bh4IOF4SllRjaWZ\n9Qf+D7gy+g+/KHUSZ+Jj6u4t7j4WGAqML4Z/DNqSQZyJj6WZnQmsiSpPI0cVUaklkNXAQbHnQ6PX\nWrcZ1kmbfOs0TnfflC593f1RoDI6tbmYFMNYdqqYxtLMehP+KN/j7nt8tokiGdPO4iymMXX3d4CZ\nwORWi4piLNPai7NIxnIi8HEzewX4JXCymd3dqk2Xx7PUEsgcYLiZHWxmVcBnCB9IjHsIuBDAzCYA\n6z265lYBdRpn/NiimdUSTql+u7Bhhs3T/n8jxTCWae3GWURjCfAzYLG739jO8mIZ0w7jTHpMzWw/\nM6uOHqeA04ClrZolPpaZxJn0WAK4+zfd/SB3P5Tw9+hJd7+wVbMuj2eSn0TvMndvNrMpwGOE5DfN\n3ZeY2WVhsU919z+Y2RlmthzYDFxUjHECnzazy4EdwFbg/ELHaWbTgTpgXzNbCVwDVFFEY5lJnBTB\nWEZxTgT+AVgUHRN34JuEs/GKZkwziZPkx/RA4C4z60X4HfpVNHZF9bueSZwkP5bt6u546oOEIiKS\nlVI7hCUiIkVCCURERLKiBCIiIllRAhERkawogYiISFaUQEREJCtKICIFZGavFuEVB0SyogQiUlj6\n4JWUDSUQkTwxs/vNbE70RUNfSr+caFAiOVRSlzIRKTEXuft6M+sLzDGz3yUdkEguKYGI5M9Xzeyc\n6PFQ4HB0CEvKiBKISB6Y2UnAR4Dx7r7NzGYCfRMOSySnNAcikh/VwLooeRwJTIhe1xyIlA0lEJH8\n+CPhi4P+CnwfeD56XYewpGzocu4iIpIVVSAiIpIVJRAREcmKEoiIiGRFCURERLKiBCIiIllRAhER\nkawogYiISFaUQEREJCv/H4Tf3Ppqdi5dAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f442dc4d7b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot that includes the regression line\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If **al=3**, what class do we predict for household? **1**\n",
    "\n",
    "If **al=1.5**, what class do we predict for household? **0**\n",
    "\n",
    "We predict the 0 class for **lower** values of al, and the 1 class for **higher** values of al. What's our cutoff value? Around **al=2**, because that's where the linear regression line crosses the midpoint between predicting class 0 and class 1.\n",
    "\n",
    "Therefore, we'll say that if **household_pred >= 0.5**, we predict a class of **1**, else we predict a class of **0**.\n",
    "\n",
    "## $$h_\\beta(x) = \\beta_0 + \\beta_1x_1 + \\beta_2x_2 + ... + \\beta_nx_n$$\n",
    "\n",
    "- $h_\\beta(x)$ is the response\n",
    "- $\\beta_0$ is the intercept\n",
    "- $\\beta_1$ is the coefficient for $x_1$ (the first feature)\n",
    "- $\\beta_n$ is the coefficient for $x_n$ (the nth feature)\n",
    "\n",
    "### if $h_\\beta(x)\\le 0.5$ then $\\hat y = 0$ \n",
    "\n",
    "### if $h_\\beta(x)> 0.5$ then $\\hat y = 1$ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['small', 'big', 'small'], \n",
       "      dtype='<U5')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# understanding np.where\n",
    "import numpy as np\n",
    "nums = np.array([5, 15, 8])\n",
    "\n",
    "# np.where returns the first value if the condition is True, and the second value if the condition is False\n",
    "np.where(nums > 10, 'big', 'small')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "      <th>household</th>\n",
       "      <th>household_pred</th>\n",
       "      <th>household_pred_class</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.340495</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.315436</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.250283</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.250283</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.230236</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  household  household_pred  household_pred_class  \n",
       "id                                                              \n",
       "22   1.521227          0       -0.340495                     0  \n",
       "185  1.521103          1       -0.315436                     0  \n",
       "40   1.520781          0       -0.250283                     0  \n",
       "39   1.520781          0       -0.250283                     0  \n",
       "51   1.520682          0       -0.230236                     0  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transform household_pred to 1 or 0\n",
    "glass['household_pred_class'] = np.where(glass.household_pred >= 0.5, 1, 0)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f442dbe74e0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH5dJREFUeJzt3X+UHHWZ7/H3k56eTCeTTBIIgU1CokQIixDgSIgbf3Rc\nkaC4eNdVwRV28Rdnr7lw756zwuWsh8Gju+s9u/fusqxHUfy5htxVdEFdfsiaUbgYk9VAokmECBND\nkEwWEzQIZJI894+untR0erpraqqmurs+r3P6dFfVt77fp74zmSf1VHW3uTsiIiLjNSXrAEREpD0p\ngYiISCxKICIiEosSiIiIxKIEIiIisSiBiIhILJknEDO73cz2mtmWMba/28weDR4PmdnZkx2jiIgc\nL/MEAnweuLjB9ieA17n7MuBjwGcmJSoREWmoK+sA3P0hM1vUYPuG0OIGYH76UYmISDOtcAYyHu8H\n7sk6CBERaYEzkKjMbBVwNfCarGMREZE2SSBmdg5wG7Da3fc3aKcP9hIRGSd3tzj7tUoJy4LH8RvM\nTgXuBK50958368jdW/px0003ZR6D4lScilNxVh8TkfkZiJmtBcrACWb2C+AmoBtwd78N+AgwB/ik\nmRkw7O7Ls4pXREQqMk8g7v7uJts/AHxgksIREZGIWqWElRvlcjnrECJRnMlSnMlSnK3BJloDayVm\n5p10PCIiaTMzvM0voouISJtRAhERkViUQEREJBYlEBERiUUJREREYlECERGRWJRAREQkFiUQERGJ\nRQlERERiUQIREZFYlEBERCQWJRAREYlFCURERGJRAhERkViUQEREJBYlEBERiUUJREREYlECERGR\nWJRAREQkFiUQERGJRQlERERiyTyBmNntZrbXzLY0aHOLmT1uZo+Y2bmTGZ+IiNTXlXUAwOeBfwS+\nVG+jmV0CnOburzCzC4FPASsmMb669u3bx+DgIIsXL2bu3LmZ99NKtm/fzsaNG1m+fDlnnnnmmMfY\n7NijzE21TW9vLwcPHmTv3r1s2rSJCy64gHnz5rF48WKAkX4ANm/eDMB5551XN55qX729vezevZsD\nBw6MtJk1axYLFy5k9+7dDfuojrV+/Xp27tzJkiVLOPvsszl48OComKpjNOtr7ty5x/UdjjU8R1H2\nazbn9fqNMv9R44j7844bZxY68d/2cdw98wewCNgyxrZPAe8KLW8H5o3R1ifD2rXrvFSa431953up\nNMfXrl2XaT+tZM2a6xxKDqc7lPyiiy6pe4zNjj3K3FTblEovD8bsC55PcSh5oXCGF4szvLu7z/v6\nzvdicYYXCtMdpjks8e7uvuPiqfZVLJ7qMLXmMS3oe2rDPqpjmfWE2p7sUPJS6eyRmCpjNe+rVJrj\na9ZcN6rvY/tX+qzOUZT9ms15vX7radS+URxxf95x48xCO/3bDv5uxvvbHXfHJB9NEsg3gd8LLT8A\nnD9G22RmtIGhoSEvleY4POrgDo96qTTHh4aGMumnlWzbti34A149pvU1y5Vj3LZtW8NjjzI3x9qs\nd5jj8IVgrOryow5DDrNDr/tCy/Xiqe673mFW0L4vtF91faM+qmPNcOjxKczyAg94gdle4Ede4Gkv\njKzrC17/yAsMe4EfeW/PbN+2ZYv39swOrX/AC/TU2X/2qH2nT+2LsN/osYaeftp9eNiHnn462Pf4\nfsPtqo9G7RvH0WjsxmPGiTOLR/h44GjL/9ueSAJphRJWovr7+0del8tlyuVyov0PDg7S3b2YF144\nJ1hzDsXiIgYHB8d1mppUP61k48aNwEKgekzTgQWh5coxbty4seGxR5mbY22mA4uBJ4KxqsvnAJuA\nl4VenxxsP9bvlCkLQvFU950OzKs5uumh9Y36qI41m9MZ5qf8ErgYOAIsD/Y5ElpHaD3w4hGmnHsu\n+48eHb1+zP1DbV6q099x+40eq7BwIQAnAvuPjNFvqF1Vw/ZN4xhr7MZjxokzC+HjmcZvGW6xf9sD\nAwMMDAwk01nczJPkg/GVsHaQYQlLZyBj0xnI6DOQFXT7wxTGiClqX7XzGN5/9Bz19MyKsF+zOT++\n33q/l43aN44j3s87yrit8u+n3f5t0wElrMXA1jG2vRn4dvB6BbChQT/JzGgT1frmzJnnJXINZKL9\ntJI1a64N/li8wqHkb3rTJXWPsdmxR5mbapuensXBmDOC58r1hkLhdC8We727u89nzjzPi8VeLxSm\neeWaw2l1rzlU+yoWFzp0+/HXQOYF68fuozrW6yn697DQfiUvlV45ElNlrOZ9Va4hXDuq72P7V/qs\nvfbQaL9mc16v33oatW8UR9yfd9w4s9BO/7YnkkCssn92zGwtUAZOAPYCNwHdVA7qtqDNrcBq4Hng\nanf/8Rh9+WQdj+7CGpvuwqqMVfze93jp5pu5/YordBeW7sLKOpwxmRnubrH2zTqBJGkyE4hIU/fc\nA7fcUnkWaVETSSCZv5FQpGMdOgTFYtZRiKRGCUQkLcPD0N2ddRQiqVECEUmLzkCkwymBiKRFZyDS\n4ZRARNIyPKwzEOloSiAiaVEJSzqcEohIWlTCkg6nBCKSFpWwpMMpgYikRSUs6XBKICJpUQlLOpwS\niEhadAYiHU4JRCQtOgORDqcEIpIWXUSXDqcEIpIWlbCkwymBiKRFJSzpcEogImlRCUs6nBKISFpU\nwpIOpwQikhaVsKTDKYGIpEVnINLhlEBE0qIzEOlwSiAiadFFdOlwSiAiaVEJSzqcEohIWlTCkg6X\neQIxs9VmtsPMHjOz6+tsn2lmd5vZI2a21cz+NIMwRcZPJSzpcJkmEDObAtwKXAycBVxhZktrmn0I\n+Km7nwusAv7OzLomN1KRGFTCkg6X9RnIcuBxd9/l7sPAOuCymjYOzAhezwCedffDkxijSDwqYUmH\nyzqBzAd2h5afCtaF3Qr8rpk9DTwKXDdJsYlMjM5ApMO1QynoYmCzu7/BzE4DvmNm57j7wXqN+/v7\nR16Xy2XK5fKkBClyHJ2BSAsaGBhgYGAgkb7M3RPpKNbgZiuAfndfHSzfALi7fyLU5lvAX7v7/wuW\n/x243t3/o05/nuXxiIwyfz5s3Fh5FmlRZoa7W5x9sy5hbQKWmNkiM+sGLgfurmmzC3gjgJnNA04H\nnpjUKEXiUAlLOlymJSx3P2Jma4D7qSSz2919u5ldU9nstwEfA75gZluC3T7s7r/KKGSR6FTCkg6X\naQkraSphSUuZPh2GhirPIi2qnUtYIp1LJSzpcEogImlwh8OHlUCkoymBiKTh8GHo6gKLVRkQaQtK\nICJpOHRIF9Cl4ymBiKRBH6QoOaAEIpIGXUCXHFACEUmD3gMiOaAEIpIGlbAkB5RARNKgEpbkgBKI\nSBpUwpIcUAIRSYNKWJIDSiAiaVAJS3JACUQkDSphSQ4ogYikQWcgkgNKICJp0BmI5IASiEgadBFd\nckAJRCQNKmFJDiiBiKRBJSzJASUQkTSohCU5oAQikgaVsCQHlEBE0qASluRAV6ONZrYV8LG2u/s5\niUck0gl0BiI50DCBAJcGzx8Knr8cPP9xOuGIdAidgUgONCxhufsud98FXOTuH3b3rcHjBuBNSQRg\nZqvNbIeZPWZm14/Rpmxmm83sJ2a2PolxRVKli+iSA1GvgZiZrQwt/N449m3U6RTgVuBi4CzgCjNb\nWtOmD/gn4FJ3fyXwjomOK5I6lbAkB5qVsKreB3wu+GNuwH7gvQmMvxx4PDjLwczWAZcBO0Jt3g3c\n6e57ANz9PxMYVyRdKmFJDkRKIO7+I2BZkEBw9+cSGn8+sDu0/BSVpBJ2OlAMSle9wC3u/mVEWtnw\nMPT2Zh2FSKqa3YX152OsB8Dd/3cKMdXqAs4H3gBMB35gZj9w9531Gvf394+8LpfLlMvlSQhRpIZK\nWNKiBgYGGBgYSKSvZmcgMxIZZWx7gFNDywuCdWFPAf/p7i8CL5rZ94FlQNMEIpIZlbCkRdX+x/rm\nm2+O3VfDBOLu8XuOZhOwxMwWAb8ELgeuqGlzF/CPZlYApgIXApNx5iMSn85AJAci3UllZgvM7Btm\nNhQ87jSzBRMd3N2PAGuA+4GfAuvcfbuZXWNmHwza7ADuA7YAG4Db3H3bRMcWSZXOQCQHot6F9Xlg\nLcduoX1PsO6iiQbg7vcCZ9Ss+3TN8t8CfzvRsUQmjd4HIjkQ9b0cc9398+5+OHh8AZibYlwi7U0l\nLMmBqAnkWTN7j5kVgsd7gGfTDEykramEJTkQNYG8F3gn8AyVi91/BFydVlAibU8lLMmBqG8k3AX8\nQcqxiHQOlbAkByIlEDObC3wAWBzex92T+DgTkc6jEpbkQNS7sO4CHgQeAI6kF45Ih9AZiORA1AQy\nzd3rftS6iNShMxDJgagX0b9lZm9ONRKRTqKL6JIDzT5M8TdUvtLWgBvN7BBwKFh2d5+ZfogibUgl\nLMmBZp+FlfaHKYp0JpWwJAeifhaWBW8k/EiwvNDMar+3Q0SqVMKSHIh6DeSTwKupfDsgwEEqXzMr\nIvWohCU5EPUurAvd/Xwz2wzg7vvNTOfnImNRCUtyIOoZyHDwfRwOI28sPJpaVCLtTmcgkgNRE8gt\nwDeAk8zs48BDwF+lFpVIu9MZiOSAuXu0hmZLgd+ncgvvv7v79jQDi8PMPOrxiKSqVIJf/aryLNLC\nzAx3tzj7Rr0L6zTgSXf/J+AnwEVmNivOgCK5oBKW5EDUEtadwBEzWwJ8GlhI5RsKRaTW0aPgDoVC\n1pGIpCpqAjnq7oeBPwRudfe/AE5JLyyRNlZ9D4jFqgqItI3x3IV1BXAV8K1gnc7PRepR+UpyImoC\nuZrKGwk/7u5PmtnLgC+nF5ZIG9MdWJITke/Cage6C0tawjPPwLJlsHdv1pGINDWRu7CifiPhkwRv\nIgxz95fHGVSko+kMRHIiagnrVcAFweO1VN5Y+M9JBGBmq81sh5k9ZmZjfmmVmV1gZsNm9odJjCuS\nGn2QouREpATi7s+GHnvc/e+Bt0x0cDObAtwKXAycBVwRvGGxXru/Ae6b6JgiqdNFdMmJqCWs80OL\nU6ickUT9IMZGlgOPu/uuYJx1wGXAjpp2/w34GpUzIJHWphKW5ETUJPB3odeHgUHgnQmMPx/YHVp+\nikpSGWFmvwO8zd1X6TtIpC2ohCU5ESmBuPuqtANp4O+B8LWRhncL9Pf3j7wul8uUy+VUghIZk0pY\n0sIGBgYYGBhIpK9It/GaWR9wE/C6YNX3gI+6+3MTGtxsBdDv7quD5RuofNf6J0Jtnqi+BE4Engc+\n6O531+lPt/FK9h58EG68sfIs0uJS/zBF4HPAb6iUrd4J/Br4fJwBa2wClpjZouALqi4HRiUGd395\n8HgZlesg/7Ve8hBpGToDkZyIeg3kNHd/e2j5ZjN7ZKKDu/sRM1sD3E8lmd3u7tvN7JrKZr+tdpeJ\njimSOl1El5yImkBeMLPXuPtDAGa2EnghiQDc/V7gjJp1nx6j7XuTGFMkVbqILjkRNYH8GfDF4FoI\nwH7gT9IJSaTNqYQlORE1gWwH/hdwGjALeA54G7AlpbhE2pdKWJITURPIXcAB4MfAnvTCEekAKmFJ\nTkRNIAuqt9qKSBMqYUlORL2N92EzOzvVSEQ6hUpYkhMNz0DMbCuVW2e7gKuDN/W9ROVNfe7u56Qf\nokibUQlLcqJZCevSSYlCpJMcOqQzEMmFhgmk+im5IjIOOgORnIh6DUREotJFdMkJJRCRpOkiuuSE\nEohI0lTCkpxQAhFJmkpYkhNKICJJUwlLckIJRCRpKmFJTiiBiCRNJSzJCSUQkaSphCU5oQQikjSd\ngUhOKIGIJE1nIJITSiAiSdNFdMkJJRCRpKmEJTmhBCKSNJWwJCeUQESSphKW5IQSiEjSVMKSnMg8\ngZjZajPbYWaPmdn1dba/28weDR4P6at1peWphCU5kWkCMbMpwK3AxcBZwBVmtrSm2RPA69x9GfAx\n4DOTG6XIOOkMRHIi6zOQ5cDj7r7L3YeBdcBl4QbuvsHdnwsWNwDzJzlGkfHRGYjkRNYJZD6wO7T8\nFI0TxPuBe1KNSGSidBFdcqLhd6K3EjNbBVwNvKZRu/7+/pHX5XKZcrmcalwix1EJS1rYwMAAAwMD\nifRl7p5IR7EGN1sB9Lv76mD5BsDd/RM17c4B7gRWu/vPG/TnWR6PCADz5sGWLZVnkRZnZri7xdk3\n6xLWJmCJmS0ys27gcuDucAMzO5VK8riyUfIQaRkqYUlOZFrCcvcjZrYGuJ9KMrvd3beb2TWVzX4b\n8BFgDvBJMzNg2N2XZxe1SBMqYUlOZFrCSppKWNISpk6FX/+68izS4tq5hCXSWdx1BiK5oQQikqQj\nR6BQgCn6pyWdT7/lIknSBXTJESUQkSSpfCU5ogQikiR9jInkiBKISJJUwpIcUQIRSZJKWJIjSiAi\nSVIJS3JECUQkSToDkRxRAhFJks5AJEeUQESSpIvokiNKICJJUglLckQJRCRJKmFJjiiBiCRJJSzJ\nESUQkSSphCU5ogQikiSVsCRHlEBEkqQzEMkRJRCRJOkMRHJECUQkSbqILjmiBCKSJJWwJEeUQESS\npBKW5IgSiEiSVMKSHFECEUmSSliSI5knEDNbbWY7zOwxM7t+jDa3mNnjZvaImZ072TGKRKYSluRI\nV5aDm9kU4Fbg94GngU1mdpe77wi1uQQ4zd1fYWYXAp8CVkx2rPv27WNwcJDFixczd+7cce27fft2\nNm7cyPLlyznxxBNH9VPtt7e3l4MHDx7X/759+9i8eTMHDhxg1qxZnHfeeQBs3rwZgOnTp7Nz506W\nLFnC888/P9Ju4cKFbN26lb179/LGN74RgAceeIBSqQTA0NAQS5YsYdWqVccdTzjeM888c1QMALNm\nzWLfvn3ccccd9Pb28ta3vpVdu3YxNDTEnj172LFjBzNmzOC1r30tfX19PPnkk/zgBz+gt7eXOXPm\n8NJLL7Fnzx5+85vfcMYZZ9DX1zcS0y9+8QuWL19OuVzm/PPPZ8+ePXzzm9/kscce44wzzuDSSy/l\n2Wef5Stf+Qpz587lfe97H8888wyf/exn2b17NytWrGDp0qUALFu2jLPPPputW7fy8MMP89xzz7Fy\n5UpWrlzJwYMHR+b80KFDI3PY3d09sr72Ofyzqc4JwMKFC49tH+cZyJj9jPN3TCQT7p7Zg0oiuCe0\nfANwfU2bTwHvCi1vB+aN0Z+nYe3adV4qzfG+vvO9VJrja9eui7zvmjXXOZQcTneY6l1dM0b6WbPm\nOi+V5nip9HKHkpdKZ4/qf+3adV4sznCY6jDNYYkXCtO9UJgeLM8O+v6doE213SkOxWDbK4LX1e3F\nULvTvKtrxqjjGR1vyS+66JJQDNX9Sg5dwfPsoM+iQyF4hMcpBG2rj2qbYmhdsWa/UugYCj465qmh\nsU8J9VndN9z25NA44XUlLxZPdSj5lClzQ3NY8kJhwajt1efwz+bYz6U618e2/+Syt7l/7GORf6/G\n6mc8v2MiExH83Yz1N9wq+2fDzN4OXOzuHwyW3wMsd/drQ22+Cfy1uz8cLD8AfNjdf1ynP0/6ePbt\n28eiRUu5/IVreRV7gWfpKtzFlVdeMfK/+bHs37+fO9Z9FXg7MA24A7gMOAHYA3wbeAtwX2h9pf93\nvOO/8C9f/TpHjgBYsH0asDZYXgV8J9j/3mBEAy4G7gGOBuNOAb4KFAAP1neNGq9QuIurrryCF198\nMRRvOMZCqP8LgIeD1xcB9wfbPFhnwZhHQ+vDjnJ85fRoMEZ1v7cEx3A41GcXsAz4j2D5LcC/BW2q\n+x4Jta3OQ+26+0LP4Tm8r8726nN4rv4VgCNHrO72lXYnp/3ljfR+9KM0sm/fPk499XRefNGAr1OZ\n8/XAOcAWSqVV7Nq1Q2cikjozw90tzr6ZlrDS0N/fP/K6XC5TLpcn1N/g4CDd3Yt56oVX08NOAErF\nh9l70kksXrSo4b4/27CBn3AC8HpgF3BS8BpgBrABWApsDq2v9P+zYpEdhVm8dARgaqiPWcHyXCp/\ntJYG/VTbLQUeBIrBPhuAmcHycNCud9R43VMeZO9JJ/HML38Zirca4/eBak1/KpU/yFODPuYC04Nt\nL1H5Q10KngGe51jygUpyGKaSCI9SSS4F4LfBWBbEWT2G54Oxi8F4L4TGXkrlD+7B0L6HqfxK94b6\neCnoY0ZorqvP4Tnc3OD52FxNLQxw9KgzTE/d7T+f+jBXvepVLKOxwcFBCoV5wfxNBxZTSR4A51As\nLmJwcFAJRBI3MDDAwMBAMp3FPXVJ4kGlhHVvaDlKCWsHk1jCGhoa8lJpjsOjDu7wqJdKc3xoaKjp\nvtu2bQvKI486DAXlnmo/64Nt6x2O73/btm3e0zPLoS+031Bo+Ruh/cPt1jvMDI27zaEn2D4jeD17\n1Hg9PbN9aGioJt5qjNV9q/3f4tAdtPuGHyttVUtRPaFxeoK21UfRYUrQvrpuqoOF9qse08xQ6aka\n88dDY6936K3ZN9y22kd3zbo5oefwHM5p8Byeq1nBz2V23e1RfzeGhoYS6UdkophACSvrBFIAdgKL\nqPw38RHgzJo2bwa+7ccSzoYG/SU3qyHVayAzZ54X4xrItX7sWkS3d3XNGOlnzZprvVSa4z09i71S\n/35lnWsgvR6+ZlEoTPNCYVqw3OfHrgV0h9rN82PXCZYEr6vbi8Hrsa6BhOMt+ZvedEkohvB1iELQ\nbpYfu5ZhQYLoDo1jfuy6RyHYbj76ekh13+p+Ja9cq+gKta3GXAyNfXLwOrxvuO280DjhdSUvFhd6\n5RrICaE5LHmhMH/U9upz+Gdz7OdyrL/an13U36sk+hGZiIkkkEyvgUDlNl7gH6gUsW93978xs2uC\ng7otaHMrsJpKTeNqr3P9I2jnaR2P7sLSXViR7sIa5++G7sKSrE3kGkjmCSRJaSYQEZFONJEEkvkb\nCUVEpD0pgYiISCxKICIiEosSiIiIxKIEIiIisSiBiIhILEogIiISixKIiIjEogQiIiKxKIGIiEgs\nSiAiIhKLEoiIiMSiBCIiIrEogYiISCxKICIiEosSiIiIxKIEIiIisSiBiIhILEogIiISixKIiIjE\nogQiIiKxKIGIiEgsmSUQM5ttZveb2c/M7D4z66vTZoGZfdfMfmpmW83s2ixiFRGR42V5BnID8IC7\nnwF8F/ifddocBv7c3c8CXg18yMyWTmKMiRsYGMg6hEgUZ7IUZ7IUZ2vIMoFcBnwxeP1F4G21Ddz9\nGXd/JHh9ENgOzJ+0CFPQLr9QijNZijNZirM1ZJlATnL3vVBJFMBJjRqb2WLgXOCHqUcmIiJNdaXZ\nuZl9B5gXXgU48Jd1mnuDfnqBrwHXBWciIiKSMXMf8+92ugObbQfK7r7XzE4G1rv7mXXadQHfAu5x\n939o0mc2ByMi0sbc3eLsl+oZSBN3A38KfAL4E+CuMdp9DtjWLHlA/EkQEZHxy/IMZA7wL8BCYBfw\nTnc/YGanAJ9x90vNbCXwfWArlRKXAze6+72ZBC0iIiMySyAiItLe2u6d6Ga22sx2mNljZnb9GG1u\nMbPHzewRMzt3smMMYmgYp5m93swOmNmPg0e9GwvSjvF2M9trZlsatGmFuWwYZyvMZRBHpDe+Zj2n\nUeLMek7NbKqZ/dDMNgcx3jRGu6znsmmcWc9lTSxTghjuHmP7+ObT3dvmQSXh7QQWAUXgEWBpTZtL\ngG8Hry8ENrRonK8H7s54Pl9D5dboLWNsz3wuI8aZ+VwGcZwMnBu87gV+1qK/n1HizHxOgWnBcwHY\nACxvtbmMGGfmcxmK5X8A/1wvnjjz2W5nIMuBx919l7sPA+uovCEx7DLgSwDu/kOgz8zmMbmixAmV\n25oz4+4PAfsbNGmFuYwSJ2Q8lxD5ja+Zz2nEOCH738/fBi+nUrnhp7benvlcBmM3ixNa4PfTzBYA\nbwY+O0aTcc9nuyWQ+cDu0PJTHP+LX9tmT502aYsSJ8Crg1PFb5vZ705OaOPSCnMZVUvNZYM3vrbU\nnDZ5g26mcxqUWzYDzwDfcfdNNU1aYi4jxAmt8fv5f4C/YOz33I17PtstgXSSHwGnuvu5wK3Av2Yc\nTztrqblslze+Nokz8zl196Pufh6wALiwFf5jUE+EODOfSzN7C7A3OPM0EjojarcEsgc4NbS8IFhX\n22ZhkzZpaxqnux+snvq6+z1AMbi1uZW0wlw21UpzGbzx9WvAl9293nubWmJOm8XZSnPq7r8G1gOr\naza1xFxWjRVni8zlSuAPzOwJ4A5glZl9qabNuOez3RLIJmCJmS0ys27gcipvSAy7G7gKwMxWAAc8\n+MytSdQ0znBt0cyWU7ml+leTG2ZleMb+30grzGXVmHG20FxC8ze+tsqcNowz6zk1sxMt+IoHMysB\nFwE7applPpdR4sx6LgHc/UZ3P9XdX07l79F33f2qmmbjns8s34k+bu5+xMzWAPdTSX63u/t2M7um\nstlvc/d/M7M3m9lO4Hng6laME/gjM/szYBh4AXjXZMdpZmuBMnCCmf0CuAnopoXmMkqctMBcBnGu\nBP4Y2BrUxB24kcrdeC0zp1HiJPs5PQX4oplNofJv6P8Gc9dS/9ajxEn2czmmic6n3kgoIiKxtFsJ\nS0REWoQSiIiIxKIEIiIisSiBiIhILEogIiISixKIiIjEogQiMonM7MkW/MQBkViUQEQml954JR1D\nCUQkJWb2DTPbFHzR0PurqzMNSiRBbfVRJiJt5mp3P2BmPcAmM/t61gGJJEkJRCQ9/93M3ha8XgC8\nApWwpIMogYikwMxeD7wBuNDdXzKz9UBPxmGJJErXQETS0QfsD5LHUmBFsF7XQKRjKIGIpONeKl8c\n9FPgr4CHg/UqYUnH0Me5i4hILDoDERGRWJRAREQkFiUQERGJRQlERERiUQIREZFYlEBERCQWJRAR\nEYlFCURERGL5/8IISwT5yDZDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f442dcc8be0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the class predictions\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_class, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$h_\\beta(x)$ can be lower 0 or higher than 1, which is countra intuitive\n",
    "\n",
    "## Using Logistic Regression Instead\n",
    "\n",
    "Logistic regression can do what we just did:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# fit a logistic regression model and store the class predictions\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.household\n",
    "logreg.fit(X, y)\n",
    "glass['household_pred_class'] = logreg.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f442dbf45c0>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHytJREFUeJzt3X10HPV97/H3V6uVtbZsGYMxFBs7YLAp8ROnCCcmyToN\n2DykpE2TQBrSkoRweuML9/acBpp7c1Bukzb00NyWujmBBMhDY3ybEAqE8hAS6yZccOwQYzuxHXBB\nxphgudSmmAdHtr/3j5ldz65Wu7OjXc3K/rzO2bM7M7/5zXd+0uijmdmVzN0RERGpV1vaBYiIyNik\nABERkUQUICIikogCREREElGAiIhIIgoQERFJJPUAMbPbzWy3mW0aZvmHzWxj+HjMzOaNdo0iIjJU\n6gEC3Aksq7L8WeCd7r4A+Dzw1VGpSkREqmpPuwB3f8zMZlZZvjYyuRY4pflViYhILa1wBlKPTwAP\npl2EiIi0wBlIXGa2FLgKOD/tWkREZIwEiJnNB24Dlrv73irt9Ie9RETq5O6WZL1WuYRl4WPoArNT\ngbuBK93932p15O4t/bjxxhtTr0F1qk7VqToLj5FI/QzEzFYBeeB4M3seuBHoANzdbwM+C0wBvmxm\nBgy6e09a9YqISCD1AHH3D9dYfjVw9SiVIyIiMbXKJaxjRj6fT7uEWFRnY6nOxlKdrcFGeg2slZiZ\nH037IyLSbGaGj/Gb6CIiMsYoQEREJBEFiIiIJKIAERGRRBQgIiKSiAJEREQSUYCIiEgiChAREUlE\nASIiIokoQEREJBEFiIiIJKIAERGRRBQgIiKSiAJEREQSUYCIiEgiChAREUlEASIiIokoQEREJBEF\niIiIJKIAERGRRBQgIiKSSOoBYma3m9luM9tUpc0tZvaMmT1lZgtHsz4REamsPe0CgDuBfwC+WWmh\nmV0EnO7uZ5jZecBXgMWjWF9Fe/bsob+/n1mzZjF16tTU+2klW7duZd26dfT09HDWWWcNu4+19j3O\n2BTadHV1sX//fnbv3s369es599xzmTZtGrNmzQIo9gOwYcMGABYtWlSxnkJfXV1d7Ny5k3379hXb\nTJ48mRkzZrBz586qfRS2tWbNGrZv387s2bOZN28e+/fvL6mpsI1afU2dOnVI39Fao2MUZ71aY16p\n3zjjH7eOpF/vpHWm4Wg8todw99QfwExg0zDLvgJ8KDK9FZg2TFsfDatWrfZcbop3d5/judwUX7Vq\ndar9tJIVK65zyDmc6ZDzCy64qOI+1tr3OGNTaJPLnRZuszt8Ptkh55nMHM9mJ3pHR7d3d5/j2exE\nz2QmOIx3mO0dHd1D6in0lc2e6jCu7DE+7Htc1T4K2zLrjLQ9ySHnudy8Yk3Btmr3lctN8RUrrivp\n+8j6QZ+FMYqzXq0xr9RvJdXaV6sj6dc7aZ1pGEvHdvhzM9nP7qQrNvJRI0DuB94emX4UOGeYto0Z\n0SoGBgY8l5visNHBHTZ6LjfFBwYGUumnlWzZsiX8AV7YpzVl08E+btmypeq+xxmbI23WOExx+Hq4\nrcL0RocBh+Mir7sj05XqKay7xmFy2L47sl5h/nGe4UnPMOgZnvSuzuN8y6ZN3tVZmP+iZ+jyDOM8\nw2TP8KhniC4rzOsOX1frazBs21lh/dI6JozrjrFe6bYGXnzRfXDQB158MVx3aL/RdoVHtfbV66i2\n7erbTFJnGo/o/sDhlj+2RxIgrXAJq6F6e3uLr/P5PPl8vqH99/f309ExizfemB/OmU82O5P+/v66\nTlMb1U8rWbduHTADKOzTBGB6ZDrYx3Xr1lXd9zhjc6TNBGAW8Gy4rcL0fGA98JbI65PC5Uf6bWub\nHqmnsO4EYFrZ3k0ozl+I8TPOxbFg0ZuHaFu4kL2HDwM9YftD4fMBYFk4HV22LNKmMH+4vqiyfqTN\ngQr9DVmvdFuZGTMAOAHYe2iYfiPtCqq2r1nHcNuuvs0kdaYhuj/jeZ3BFju2+/r66Ovra0xnSZOn\nkQ/qu4S1jRQvYekMZHjHyhnIu+nyR+mpsk8DDhMdOiPrldd05Gym9vhEx3GgQp/Bup2dk2OsV2vM\nh/Zb6fuyWvvqdST7esfZbqscP2Pt2OYouIQ1C9g8zLKLgQfC14uBtVX6acyI1lC4vjlp0qKG3AMZ\naT+tZMWKa8MfFmc45PzCCy+quI+19j3O2BTadHbOCrc5MXwO7jdkMmd6NtvlHR3dPmnSIs9muzyT\nGe/BPYfTK95zKPSVzc5w6PCh90Cm+YW0+0O0DdtHYVtm48I+gvWC6/VvLdYUbKujaj2F/V+x4tqS\nvo+sH/RZfu+h2nq1xrxSv5VUa1+tjqRf76R1pmEsHdsjCRAL1k+Pma0C8sDxwG7gRqCDYKduC9us\nBJYDrwFXufvPh+nLR2t/9C6s4R3t78I669ln6bzzTjb85V/qXVh6F9awxsqxbWa4uyVaN+0AaaTR\nDBA5ht17L9xxR/AsMsaNJEBS/yChyJhz8CC0H3XvPxGpmwJEpF4KEBFAASJSPwWICKAAEamfAkQE\nUICI1E8BIgIoQETqpwARARQgIvVTgIgAChCR+ilARAAFiEj9FCAigAJEpH4KEBFAASJSPwWICKAA\nEamfAkQEUICI1E8BIgIoQETqpwARARQgIvVTgIgAChCR+ilARAAFiEj9FCAigAJEpH4KEBFAASJS\nPwWICKAAEamfAkQEUICI1E8BIgIoQETqpwARARQgIvUbHFSAiNACAWJmy81sm5k9bWbXV1g+yczu\nM7OnzGyzmf1JCmWKHKEzEBEg5QAxszZgJbAMOBu4wszmljX7FPBLd18ILAX+1sx09Ep6FCAiQPpn\nID3AM+6+w90HgdXAZWVtHJgYvp4IvOzuB0exRpFSBw9CNpt2FSKpSztATgF2RqZfCOdFrQR+28xe\nBDYC141SbSKV6QxEBICxcBQsAza4+7vN7HTgB2Y23933V2rc29tbfJ3P58nn86NSpBxDFCAyhvX1\n9dHX19eQvszdG9JRoo2bLQZ63X15OH0D4O5+U6TN94G/dvf/F07/ELje3X9WoT9Pc3/kGHH++fDF\nLwbPImOcmeHulmTdtC9hrQdmm9lMM+sALgfuK2uzA3gPgJlNA84Enh3VKkWidAYiAqR8CcvdD5nZ\nCuARgjC73d23mtk1wWK/Dfg88HUz2xSu9ml3/4+UShZRgIiEUj8K3P0hYE7ZvFsjr39NcB9EpDUo\nQESA9C9hiYw9ChARQAEiUj8FiAigABGpnwJEBFCAiNRPASICKEBE6qcAEQEUICL1U4CIAAoQkfop\nQEQABYhI/RQgIoACRKR+ChARQAEiUj8FiAigABGpnwJEBFCAiNTHHQ4dgkwm7UpEUqcAEalHITws\n0b9PEDmqKEBE6qHLVyJFChCReihARIoUICL1UICIFClAROqhABEpUoCI1EMBIlKkABGphwJEpEgB\nIlIPBYhIkQJEpB4KEJGiqkeCmW0GfLjl7j6/4RWJtLLBQQWISKjWkXBp+Pyp8Plb4fMfNacckRan\nMxCRoqqXsNx9h7vvAC5w90+7++bwcQNwYSMKMLPlZrbNzJ42s+uHaZM3sw1m9gszW9OI7YokcvAg\nZLNpVyHSEuLeAzEzWxKZeHsd61brtA1YCSwDzgauMLO5ZW26gX8ELnX3twIfGOl2RRLTGYhIUdwj\n4ePAHeEPcwP2Ah9rwPZ7gGfCsxzMbDVwGbAt0ubDwN3uvgvA3f+9AdsVSUYBIlIU60hw9yeBBWGA\n4O6vNGj7pwA7I9MvEIRK1JlANrx01QXc4u7fQiQNChCRolrvwvqzYeYD4O5fakJN5dqBc4B3AxOA\nJ8zsCXffXqlxb29v8XU+nyefz49CiXLMUIDIGNfX10dfX19D+qp1JExsyFaGtws4NTI9PZwX9QLw\n7+7+JvCmmf0YWADUDBCRhlOAyBhX/ov15z73ucR9VT0S3D15z/GsB2ab2Uzg18DlwBVlbe4F/sHM\nMsA44DxgNM58RIZSgIgUxXonlZlNN7N7zGwgfNxtZtNHunF3PwSsAB4BfgmsdvetZnaNmX0ybLMN\neBjYBKwFbnP3LSPdtkgiChCRorhHwp3AKo68hfYj4bwLRlqAuz8EzCmbd2vZ9M3AzSPdlsiIKUBE\niuJ+lmOqu9/p7gfDx9eBqU2sS6Q1KUBEiuIGyMtm9hEzy4SPjwAvN7MwkZakABEpihsgHwM+CLxE\ncLP7D4GrmlWUSMtSgIgUxf0g4Q7g95pci0jrU4CIFMU6EsxsKnA1MCu6jrs34s+ZiIwdChCRorhH\nwr3AT4BHgUPNK0ekxSlARIriHgnj3b3in1oXOaYoQESK4t5E/76ZXdzUSkTGAgWISFGtP6b4KsG/\ntDXgM2b2G+A34bS7+6TmlyjSQhQgIkW1/hZWs/+YosjYogARKYr7t7As/CDhZ8PpGWZW/n87RI5+\nChCRorj3QL4MvI3gvwMC7Cf4N7MixxYFiEhR3AA5z90/BbwJ4O57gY6mVSXSqhQgIkVxA2Qw/H8c\nDsUPFh5uWlUirUoBIlIUN0BuAe4BTjSzLwCPAX/VtKpEWpUCRKQo7t/C+raZPQn8LsFbeN/n7lub\nWplIK1KAiBTFfRfW6cBz7v6PwC+AC8xsclMrE2lFChCRoriXsO4GDpnZbOBWYAbBfygUObYoQESK\n4gbIYXc/CPwBsNLd/xw4uXllibQoBYhIUT3vwroC+Cjw/XBetjklibSwwUEFiEgoboBcRfBBwi+4\n+3Nm9hbgW80rS6RF6QxEpCjuu7C2ANdGpp8DbmpWUSIt6+BByOrkWwTi/0fC5wg/RBjl7qc1vCKR\nVqYzEJGiuJewfgc4N3y8g+CDhf/UiALMbLmZbTOzp81s2H9aZWbnmtmgmf1BI7YrkogCRKQoVoC4\n+8uRxy53/zvgkpFu3MzagJXAMuBs4AozmztMuy8CD490myIjogARKYp7CeucyGQbwRlJI46iHuAZ\nd98Rbmc1cBmwrazdfwW+S3AGJJIeBYhIUdwj4W8jrw8C/cAHG7D9U4CdkekXCEKlyMx+i+BPpyzV\n/yCR1ClARIrivgtrabMLqeLvgOi9EavWuLe3t/g6n8+Tz+ebUpQcoxQgMsb19fXR19fXkL7Mfcib\nq4Y2MusGbgTeGc76v8D/cvdXRrRxs8VAr7svD6dvIPhf6zdF2jxbeAmcALwGfNLd76vQn8fZH5HE\n3v52uPnm4FnkKGBmuHvVX8yHE/ddWHcArxJctvog8J/AnUk2WGY9MNvMZppZB3A5UBIM7n5a+HgL\nwX2Q/1IpPERGhc5ARIriHgmnu/v7I9OfM7OnRrpxdz9kZiuARwjC7HZ332pm1wSL/bbyVUa6TZER\nUYCIFMU9Et4ws/Pd/TEAM1sCvNGIAtz9IWBO2bxbh2n7sUZsUyQxBYhIUdwj4U+Bb4T3QgD2An/c\nnJJEWpgCRKQo7pGwFfgb4HRgMvAK8D5gU5PqEmlNChCRorhHwr3APuDnwK7mlSPS4hQgIkVxj4Tp\nhbfaihzTFCAiRXHfxvu4mc1raiUiY4ECRKSo6pFgZpsJ3jrbDlwVfqjvAMGH+tzd5ze/RJEWogAR\nKap1JFw6KlWIjBUKEJGiqkdC4a/kikhIASJSFPceiIiAAkQkQgEiUg8FiEiRAkSkHgoQkSIFiEhc\nhw8Hz206bERAASISn84+REooQETiUoCIlFCAiMSlABEpoQARiUsBIlJCASISlwJEpIQCRCSuwUEF\niEiEAkQkLp2BiJRQgIjEpQARKaEAEYnr4EHIZtOuQqRlKEBE4tIZiEgJBYhIXAoQkRKpB4iZLTez\nbWb2tJldX2H5h81sY/h4TP9aV1KjABEpkWqAmFkbsBJYBpwNXGFmc8uaPQu8090XAJ8Hvjq6VYqE\nFCAiJdI+A+kBnnH3He4+CKwGLos2cPe17v5KOLkWOGWUaxQJKEBESqQdIKcAOyPTL1A9ID4BPNjU\nikSGowARKTFmjgYzWwpcBZxfrV1vb2/xdT6fJ5/PN7UuOYYoQOQo0NfXR19fX0P6MndvSEeJNm62\nGOh19+Xh9A2Au/tNZe3mA3cDy93936r052nujxzlHn4YvvSl4FnkKGFmuLslWTftS1jrgdlmNtPM\nOoDLgfuiDczsVILwuLJaeIg0nc5AREqkejS4+yEzWwE8QhBmt7v7VjO7JljstwGfBaYAXzYzAwbd\nvSe9quWYpQARKZH60eDuDwFzyubdGnl9NXD1aNclMoQCRKRE2pewRMYOBYhICQWISFwKEJESChCR\nuBQgIiUUICJxKUBESihAROJSgIiUUICIxKUAESmhABGJSwEiUkIBIhKXAkSkhAJEJC4FiEgJBYhI\nXAoQkRIKEJG4FCAiJRQgInEpQERKKEBE4lKAiJRQgIjEpQARKaEAEYlLASJSQgEiEpcCRKSEAkQk\nrsFBBYhIhAJEJC6dgYiUUICIxHXwIGSzaVch0jIUICJx6QxEpIQCRCQuBYhICQWISFwKEJESChCR\nuBQgIiVSDxAzW25m28zsaTO7fpg2t5jZM2b2lJktHO0aRQAFiEiZVI8GM2sDVgK/C7wIrDeze919\nW6TNRcDp7n6GmZ0HfAVYPNq17tmzh/7+fmbNmsXUqVPrWnfr1q2sW7eOnp4eTjjhhJJ+Cv12dXWx\nf//+If3v2bOHDRs2sG/fPiZPnsyiRYsA2LBhAwATJkxg+/btzJ49m9dee63YbsaMGWzevJndu3fz\nnve8B4BHH32UXC4HwMDAALNnz2bp0qVD9ida71lnnVVSA8DkyZPZs2cPd911F11dXbz3ve9lx44d\nDAwMsGvXLrZt28bEiRN5xzveQXd3N8899xxPPPEEXV1dTJkyhQMHDrBr1y5effVV5syZQ3d3d7Gm\n559/np6eHvL5POeccw67du3i/vvv5+mnn2bOnDlceumlvPzyy3z7299m6tSpfPzjH+ell17ia1/7\nGjt37mTx4sXMnTsXgAULFjBv3jw2b97M448/ziuvvMKSJUtYsmQJ+/fvL475b37zm+IYdnR0FOeX\nPy94/XU6wgApjAnAjBkzKn7t4mhUPyKpcPfUHgRB8GBk+gbg+rI2XwE+FJneCkwbpj9vhlWrVnsu\nN8W7u8/xXG6Kr1q1Ova6K1Zc55BzONNhnLe3Tyz2s2LFdZ7LTfFc7jSHnOdy80r6X7VqtWezEx3G\nOYx3mO2ZzATPZCaE08eFff9W2KbQ7mSHbLjsjPB1YXk20u50b2+fWLI/pfXm/IILLorUUFgv59Ae\nPh8X9pl1yISP6HYyYdvCo9AmG5mXLVsvF9mHjJfWPC6y7ZMjfRbWjbY9KbKd6LycZ7OnOuS8rW1q\nZAxznslML1leeM7l5vkP29r9h3/xPyJfl8JYD/3axf2+akQ/IiMR/txM9DPcgvXTYWbvB5a5+yfD\n6Y8APe5+baTN/cBfu/vj4fSjwKfd/ecV+vNG78+ePXuYOXMul79xLb/DbuBl2jP3cuWVVxR/mx/O\n3r17uWv1d4D3A+OBu4DLgOOBXcADwCXAw5H5Qf8f+MDv88/f+R6HDgFYuHw8sCqcXgr8IFz/oXCL\nBiwDHgQOh9ttA74DZAAP57eXbC+TuZePXnkFb775ZqTeaI2ZSP/nAo+Hry8AHgmXeTjPwm0ejsyP\nOszQK6eHw20U1rsk3IeDkT7bgQXAz8LpS4B/DdsU1j0UaVsYh/J5D0eeo2P4cIXlhedgrC7ju3ws\n+zo/zmR5800DvheO1RpgPrCJXG4pO3Zsq3kGsWfPHk499cwR9yMyUmaGu1uSdY+6C7q9vb3F1/l8\nnnw+P6L++vv76eiYxQtvvI1OtgOQyz7O7hNPZNbMmVXX/dXatfyC44F3ATuAE8PXABOBtcBcYENk\nftD/r7JZtmUmc+AQwLhIH5PD6akEP+Tnhv0U2s0FfgJkw3XWApPC6cGwXVfJ9jrafsLuE0/kpV//\nOlJvocYfAx2R/g+Fz11hDRPCZQcIflDnwmeA1zgSPhCEwyBBEB4mCJcM8Hq4LQvrLOzDa+G2s+H2\n3ohsey7BD9z9kXUPEnxLd0X6OBD2MTEy1oXn6BhuqPIcjMdGFvBU9mY4bGF/E4BZBD/0AeaTzc6k\nv7+/5g/+/v5+MplpYR/J+xGpV19fH319fY3pLOmpSyMeBJewHopMx7mEtY1RvIQ1MDDgudwUh40O\n7rDRc7kpPjAwUHPdLVu2hJdHNjoMhJd7Cv2sCZetcRja/5YtW7yzc7JDd2S9gcj0PZH1o+3WOEyK\nbHeLQ2e4fGL4+riS7XV2HucDAwNl9RZqLKxb6P8Wh46w3T1+5NJW4VJUZ2Q7nWHbwiPr0Ba2L8wb\n52CR9Qr7NCly6alQ8xci217j0FW2brRtoY+OsnlTIs/RMZxS5Tk6VpPDr8txFZfH/d4YGBhoSD8i\nI8UILmGlHSAZYDswk+DXxKeAs8raXAw84EcCZ22V/ho3qhGFeyCTJi1KcA/kWj9yL6LD29snFvtZ\nseJaz+WmeGfnLA+uf7+1wj2QLo/es8hkxnsmMz6c7vYj9wI6Iu2m+ZH7BLPD14Xl2fD1cPdAovXm\n/MILL4rUEL0PkQnbTfYj9zIsDIiOyHbMj9z3yITLzUvvhxTWLayX8+BeRXukbaHmbGTbJ4Wvo+tG\n206LbCc6L+fZ7AwP7oEcHxnDnGcyp5QsLzxHvzZHvi5H+iv/2sX9vmpEPyIjMZIASfUeCARv4wX+\nnuAi9u3u/kUzuybcqdvCNiuB5QTXNK7yCvc/wnberP3Ru7D0Lqzo10bvwpKjxUjugaQeII3UzAAR\nETkajSRAUv8goYiIjE0KEBERSUQBIiIiiShAREQkEQWIiIgkogAREZFEFCAiIpKIAkRERBJRgIiI\nSCIKEBERSUQBIiIiiShAREQkEQWIiIgkogAREZFEFCAiIpKIAkRERBJRgIiISCIKEBERSUQBIiIi\niShAREQkEQWIiIgkogAREZFEUgsQMzvOzB4xs1+Z2cNm1l2hzXQz+5GZ/dLMNpvZtWnUKiIiQ6V5\nBnID8Ki7zwF+BPxFhTYHgT9z97OBtwGfMrO5o1hjw/X19aVdQiyqs7FUZ2OpztaQZoBcBnwjfP0N\n4H3lDdz9JXd/Kny9H9gKnDJqFTbBWPmGUp2NpTobS3W2hjQD5ER33w1BUAAnVmtsZrOAhcBPm16Z\niIjU1N7Mzs3sB8C06CzAgf9ZoblX6acL+C5wXXgmIiIiKTP3YX9uN3fDZluBvLvvNrOTgDXuflaF\ndu3A94EH3f3va/SZzs6IiIxh7m5J1mvqGUgN9wF/AtwE/DFw7zDt7gC21AoPSD4IIiJSvzTPQKYA\n/wzMAHYAH3T3fWZ2MvBVd7/UzJYAPwY2E1zicuAz7v5QKkWLiEhRagEiIiJj25j7JLqZLTezbWb2\ntJldP0ybW8zsGTN7yswWjnaNYQ1V6zSzd5nZPjP7efio9MaCZtd4u5ntNrNNVdq0wlhWrbMVxjKs\nI9YHX9Me0zh1pj2mZjbOzH5qZhvCGm8cpl3aY1mzzrTHsqyWtrCG+4ZZXt94uvuYeRAE3nZgJpAF\nngLmlrW5CHggfH0esLZF63wXcF/K43k+wVujNw2zPPWxjFln6mMZ1nESsDB83QX8qkW/P+PUmfqY\nAuPD5wywFuhptbGMWWfqYxmp5b8D/1SpniTjOdbOQHqAZ9x9h7sPAqsJPpAYdRnwTQB3/ynQbWbT\nGF1x6oTgbc2pcffHgL1VmrTCWMapE1IeS4j9wdfUxzRmnZD+9+fr4ctxBG/4Kb/envpYhtuuVSe0\nwPenmU0HLga+NkyTusdzrAXIKcDOyPQLDP3GL2+zq0KbZotTJ8DbwlPFB8zst0entLq0wljG1VJj\nWeWDry01pjU+oJvqmIaXWzYALwE/cPf1ZU1aYixj1Amt8f35v4E/Z/jP3NU9nmMtQI4mTwKnuvtC\nYCXwLynXM5a11FiOlQ++1qgz9TF198PuvgiYDpzXCr8YVBKjztTH0swuAXaHZ55Gg86IxlqA7AJO\njUxPD+eVt5lRo02z1azT3fcXTn3d/UEgG761uZW0wljW1EpjGX7w9bvAt9y90mebWmJMa9XZSmPq\n7v8JrAGWly1qibEsGK7OFhnLJcDvmdmzwF3AUjP7ZlmbusdzrAXIemC2mc00sw7gcoIPJEbdB3wU\nwMwWA/s8/Jtbo6hmndFri2bWQ/CW6v8Y3TKDzTP8byOtMJYFw9bZQmMJtT/42ipjWrXOtMfUzE6w\n8F88mFkOuADYVtYs9bGMU2faYwng7p9x91Pd/TSCn0c/cvePljWrezzT/CR63dz9kJmtAB4hCL/b\n3X2rmV0TLPbb3P1fzexiM9sOvAZc1Yp1An9oZn8KDAJvAB8a7TrNbBWQB443s+eBG4EOWmgs49RJ\nC4xlWOcS4I+AzeE1cQc+Q/BuvJYZ0zh1kv6Yngx8w8zaCI6h/xOOXUsd63HqJP2xHNZIx1MfJBQR\nkUTG2iUsERFpEQoQERFJRAEiIiKJKEBERCQRBYiIiCSiABERkUQUICKjyMyea8G/OCCSiAJEZHTp\ng1dy1FCAiDSJmd1jZuvDfzT0icLsVIsSaaAx9adMRMaYq9x9n5l1AuvN7HtpFyTSSAoQkeb5b2b2\nvvD1dOAMdAlLjiIKEJEmMLN3Ae8GznP3A2a2BuhMuSyRhtI9EJHm6Ab2huExF1gcztc9EDlqKEBE\nmuMhgn8c9Evgr4DHw/m6hCVHDf05dxERSURnICIikogCREREElGAiIhIIgoQERFJRAEiIiKJKEBE\nRCQRBYiIiCSiABERkUT+P5BmRMt+um/xAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f442db67860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the class predictions\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_class, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we wanted the **predicted probabilities** instead of just the **class predictions**, to understand how confident we are in a given prediction?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# store the predicted probabilites of class 1\n",
    "glass['household_pred_prob'] = logreg.predict_proba(X)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f442db362e8>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8FPW9//HXJyEhgYQAFUEBiRXvgkpLRKk2WLVotWpv\nXlu1XmgVtbanao+2xlpb6emv3mu15bRqi7ZHagvanqqVtFoVUBFQbqIEEIUARyighQCf3x8zC0NI\nNpvN7s5u8n4+HvPYnZ3vfOez3+zuJ9/5zsXcHRERkfYqijsAEREpTEogIiKSFiUQERFJixKIiIik\nRQlERETSogQiIiJpiT2BmNlEM1tlZnNaWX6umc0Op+fNbFiuYxQRkd3FnkCAXwGfTrL8beA4dz8c\n+AHwi5xEJSIiSXWLOwB3f97MhiRZ/lJk9iVgYPajEhGRtuRDD6Q9LgH+EncQIiKSBz2QVJnZGOAi\n4BNxxyIiIgWSQMxsOPAAMNbd309SThf2EhFpJ3e3dNbLl11YFk67LzDbB5gMfNnd32qrInfP6+mm\nm26KPQbFqTgVp+JMTB0Rew/EzCYBtcBHzGwZcBNQCri7PwB8F+gL/MzMDGhy95q44hURkUDsCcTd\nz21j+aXApTkKR0REUpQvu7C6jNra2rhDSInizCzFmVmKMz9YR/eB5RMz8870fkREss3M8AIfRBcR\nkQKjBCIiImlRAhERkbQogYiISFqUQEREJC1KICIikhYlEBERSYsSiIiIpEUJRERE0qIEIiIiaVEC\nERGRtCiBiIhIWpRAREQkLUogIiKSFiUQERFJixKIiIikRQlERETSogQiIiJpUQIREZG0KIGIiEha\nlEBERCQtsScQM5toZqvMbE6SMneZ2Ztm9pqZHZHL+EREpGXd4g4A+BVwN/BQSwvN7GRgP3ff38yO\nAn4OjMphfC1avXo1DQ0NVFdX069fv9jrySfz589nxowZ1NTUcPDBB7f6Htt676m0TaJMRUUFGzdu\nZNWqVcycOZORI0fSv39/qqurAXbUAzBr1iwAjjzyyBbjSdRVUVHB8uXLWbdu3Y4yvXv3ZvDgwSxf\nvjxpHYltTZs2jcWLFzN06FCGDRvGxo0bd4kpsY226urXr99udUdjjbZRKuu11eYt1ZtK+6caR7p/\n73TjjENn/G7vxt1jn4AhwJxWlv0cOCsyPx/o30pZz4VJkx718vK+XlU1wsvL+/qkSY/GWk8+GT/+\naodyhwMcyv3EE09u8T229d5TaZtEmfLyj4bbrAof93Io9+LiA72kpNJLS6u8qmqEl5RUenFxT4ce\nDkO9tLRqt3gSdZWU7OPQvdnUI6y7e9I6EtsyK4uUHeBQ7uXlw3bEFGyr7brKy/v6+PFX71L3zvWD\nOhNtlMp6bbV5S/W2JFn5ZHGk+/dON844FNJ3O/zdTO+3O90VMzm1kUCmAsdE5p8BRrRSNjMtmkRj\nY6OXl/d1mO3gDrO9vLyvNzY2xlJPPpk3b174A554T9OazQfvcd68eUnfeypts7PMNIe+Dr8Ot5WY\nn+3Q6NAn8rwqMt9SPIl1pzn0DstXRdZLvJ6sjsS2Kh3KIus1jynVupq3Y2MLdQbrlpX1TmG9ttp8\n93pb+lwmK588jvT+3qlsN1++P4X23e5IAsmHXVgZVVdXt+N5bW0ttbW1Ga2/oaGB0tJqPvxwePjK\ncEpKhtDQ0NCubmqm6sknM2bMAAYDiffUExgUmQ/e44wZM5K+91TaZmeZnkA18Ha4rcT8cGAmsG/k\n+YBw+c56i4oGReJJrNsT6N/s3fWMvJ6sjsS2+oTL92glplTrSmx7cAvrV++ybnHxnkB5s/Zvvl5b\nbb57vS19LpOVTx5Hsm23/V1Itt3SbvuwdNEi+pWVwZYtyafNm9suk+Y6Pdat45XNH9CNz3MI89ia\nZ9/t+vp66uvrM1JXISSQFQSfvoRB4WstiiaQbKiurmbLlgZgDsGHdw5NTUt37F/OdT35pKamBljO\nzve0CXiH5u+xpqYm6XtPpW12ltkENAAfDbeVmJ9D8AOzJPJ8JcFxIzvr3b79nUg8iXU3AasAj7y7\nosjrlqSOxLbeB5qADa3ElGpdiXZc3sL6u7bRtm2NmBW1sV5bbb57vS19LpOVTx5H8r93EYfSl+cY\nuPlthq5aBVOmwJo1sHYtrF3LYStW8NCGufTmEj7CG3yEPajkQ0rZQsmGrfiYMdC9O5SWtm9Ktk5F\nRbvKb9m0ifNPOZN/bf4p2yhutQ3j0vwf65tvvjn9ytLtumRyIvh0z21l2SnAk+HzUcBLSerJTJ+u\nDYn9m716HZmRMZCO1pNPxo+/Ktxdsb9DuZ900sktvse23nsqbZMoU1ZWHW6zMnwMxhuKiw/wkpIK\nLy2t8l69jvSSkgovLu7hwZjDfi2OOSTqKikZ7FDqu4+B9A9fb72OxLbMukfK9vdgf/1hO2IKttV2\nXcEYwlW71L1z/aDO5mMPydZrq81bqrclycq3FsegyuE+snsv//u3/sP9zjvdr7nG/XOf87XV+/pq\nzJvA12C+fsAA96OPdj/1VPcLLnD/5jfdb73V/f77/R/fuMZPKq30j5Xu7QMo835lB3uvsj4+6beP\ndOizm0mF9N2mA7uwLFg/PmY2CagFPkLwL9lNQCnBm3ogLHMPMJbgX5mL3P3VVuryXL0fHYXVOh2F\npaOw2LCBdc8+y/rnn2fPNWsoX7KE7W+8gX/wAT5kCN0++lGorg6mIUOgupo1PXqwZMMGqvfbT0dh\n5ZCZ4e6W1rpxJ5BMymUCEZHQli3w6qswfTq8/HIwLVsGw4cH06GHwiGHBI8DBoCl9VslWaIEElIC\nEcmR1ath6lR4/HGor4f99oOjj4aPfzyYDjkESkrijlJSoAQSUgIRyaIlS+CPfwySxuzZcNJJcOaZ\ncPLJ0KdP2+tLXlICCSmBiGSQO8yZszNpvPsufPazcMYZcMIJUFYWd4SSAUogISUQkQxYsQJ+8Qt4\n6KEgiZx5ZjAdcwwUF8cdnWRYRxJIIZwHIiK58PrrcMst8PTTcM45MHkyHHGEBr2lVbFfjVdEYrZ4\nMZx3HnzqUzByJCxdCvfeC0ceqeQhSSmBiHRVy5bBpZfCqFFw8MFBIvmP/4DKyrgjkwKhBCLS1axc\nCVdfHfQw+vWDRYvgxhuVOKTdlEBEuoqmJrj11uCEvqIimD8ffvhD6Ns37sikQGkQXaQreOUV+OpX\nYeDA4KzxIUPijkg6AfVARDqzpia44QY45RT49rfhySeVPCRj1AMR6ayWLYOzz4bevYMTAvs3v8eJ\nSMeoByLSGU2ZEhySe+aZ8MQTSh6SFeqBiHQm7nDbbXDffcElSI4+Ou6IpBNTAhHpLLZtgyuvhH/+\nE156CfbeO+6IpJNTAhHpDD78EM49FzZsgOeeg1694o5IugCNgYgUujVrgsuQ9OwJf/6zkofkjBKI\nSCFbsgRGj4ZPfjK4em5padwRSReiXVgiherNN2HMGPjOd+CKK+KORrog3Q9EpBAtXw7HHhtcw+qS\nS+KORgpYR+4Hol1YIoVm1argjoBXX63kIbFSAhEpJO+/H9yL/Jxz4Jpr4o5GujjtwhIpFJs2BT2P\nUaPgpz/VzZ4kIwp6F5aZjTWzBWa2yMyua2F5LzObYmavmdlcM7swhjBF4uUOF10E++2n5CF5I9aj\nsMysCLgH+BTwLjDTzP7k7gsixa4A3nD3z5rZHsBCM/uNu2+NIWSReNx6a3Cr2b//XclD8kbch/HW\nAG+6+1IAM3sUOB2IJhAHErdKqwTWKnlIl/L443D//TBjBpSVxR2NyA5xJ5CBwPLI/DsESSXqHmCK\nmb0LVABn5Sg2kfjNmQOXXRacYb7XXnFHI7KLuBNIKj4NzHL3481sP+BpMxvu7htbKlxXV7fjeW1t\nLbW1tTkJUiTj1qyB00+HO+8MLs0ukgH19fXU19dnpK5Yj8Iys1FAnbuPDeevB9zdJ0TKPAH8yN3/\nGc7/DbjO3V9uoT4dhSWdgzucdhocfDD813/FHY10YoV8FNZMYKiZDTGzUuBsYEqzMkuBEwDMrD9w\nAPB2TqMUybW774bVq+GHP4w7EpFWxX4eiJmNBe4kSGYT3f02MxtH0BN5wMz2An4NJHYA/8jdH2ml\nLvVApPDNmhWcLPjSS8FhuyJZ1JEeSOwJJJOUQKTgbdwIH/sY3HRTcH8PkSxTAgkpgUjBu/ji4M6C\nv/513JFIF9GRBFIIR2GJdA2//31wN8FXX407EpGUqAcikg8aG2H4cJgyBWqanwolkj3ahRVSApGC\n9aUvwb77woQJbZcVySDtwhIpZJMnB2ecP/hg3JGItIt6ICJxWrMGhg0Lksgxx8QdjXRB2oUVUgKR\ngnP++dCvH9x+e9yRSBelXVgihWjatOCoq3nz4o5EJC1xX8pEpGtqaoIrroA77oCePeOORiQtSiAi\ncbjzThgyBM44I+5IRNKmMRCRXFuxAg4/HF58EfbfP+5opIvTIHpICUQKwjnnBBdJ/MEP4o5ERIPo\nIgVj2rSg5zFxYtyRiHSYxkBEciU6cN6jR9zRiHSYEohIriQGzk8/Pe5IRDJCYyAiubByJRx2mAbO\nJe9oED2kBCJ569JLoaoKfvKTuCMR2YUG0UXy2ezZwWXaFy6MOxKRjNIYiEi2XXstfPe70Lt33JGI\nZJQSiEg2/e1v8NZbcNllcUciknFKICLZsn07XHcd3HorlJbGHY1IxiUdAzGzuUCro9LuPjzjEYl0\nFo89Fjx+8YvxxiGSJW0Nop8aPl4RPj4cPp6XnXBEOommJrjhBrjvPihSR186p5QO4zWzWe5+ZLPX\nXnX3ER0OwGwscAfB7rSJ7r7bTaHNrBa4HSgBVrv7mFbq0mG8kh/uuw8efxyeeiruSESSysVhvGZm\no939n+HMMWRg/MTMioB7gE8B7wIzzexP7r4gUqYKuBc4yd1XmNkeHd2uSFZt3Ai33AJPPBF3JCJZ\nlWoCuRj47/DH3ID3ga9mYPs1wJvuvhTAzB4FTgcWRMqcC0x29xUA7r4mA9sVyZ477oDaWhjR4Q66\nSF5LKYG4+yvA4WECwd3XZ2j7A4Hlkfl3CJJK1AFAiZlNAyqAu9z9YUTy0erVQQKZPj3uSESyrq2j\nsL7ZyusAuPtPsxBTc92AEcDxQE/gRTN70d0Xt1S4rq5ux/Pa2lpqa2tzEKJI6Ec/grPPDu73IZKH\n6uvrqa+vz0hdSQfRzeymZCu7+80d2rjZKKDO3ceG89cH1e4cSDez64CyxLbM7JfAX9x9cgv1aRBd\n4vPuuzBsGLzxBgwYEHc0Iikp2IspmlkxsJBgEP09YAZwjrvPj5Q5CLgbGAt0B6YDZ7n7vBbqUwKR\n+Fx1VXDCoC6YKAUk60dhmdkggh/x0eFLzwFXu/s76Ww0wd23mdl44Cl2HsY738zGBYv9AXdfYGZ/\nBeYA24AHWkoeIrF65x347W9h/vy2y4p0EqmeB/I0MImdJxKeD5zn7idmMbZ2Uw9EYnP55VBZCRN2\nO41JJK9lfReWmb3m7ke09VrclEAkFkuXBofsLlwIe+g0JSksHUkgqZ4MuNbMzjez4nA6H1ibzgZF\nOp1bb4WvfU3JQ7qcVHsgQwjGQI4muLjiC8BV7r4su+G1j3ogknNvvw01NbBoEfTtG3c0Iu1WsEdh\nZZoSiOTcV78KgwfDzR06ol0kNrk4CqsfcClQHV3H3TNxORORwrR4MUydCm++GXckIrFI9VpYfyI4\ndPcZgkNpReSWW4JzP3SrWumi0j4KKx9pF5bkzMKFcOyxQS+kV6+4oxFJWy6OwnrCzE5JZwMindL3\nvw/f+IaSh3RpbV0LawPBUVdGcCHDLeFkBGeK59W3Rz0QyYl582DMmKD3UVkZdzQiHZK1QXR317dD\npLmbb4ZvfUvJQ7q8VMdAjOA+6Pu6+y1mNhjYy91nZDvA9lAPRLJu7lw48UR46y3o2TPuaEQ6LBdj\nID8jOInw3HB+I8FtZkW6lro6uPZaJQ8RUj+M9yh3H2FmswDc/X0zK81iXCL5Z9YsePFFeFg3xBSB\n1HsgTeG9Oxx2nFi4PWtRieSjujq4/nro0SPuSETyQqo9kLuAx4E9zexW4AvAjVmLSiTfvPwyvPIK\n/O53cUcikjdSvhZWeGfATxEcwvu36F0D84UG0SVrPvOZYLr88rgjEcmoXNwPZD/gHXffbGa1wHDg\nIXdfl85Gs0UJRLJi+nT44heDa1517x53NCIZlYujsCYD28xsKHA/MJjgDoUind93vws33KDkIdJM\nqglku7tvBT4H3OPu3wb2yl5YInniH/8Izjj/qi48LdJce47COgf4CvBE+FpJdkISyRPuQe/jppug\nRB93keZSTSAXEZxIeKu7LzGzfQEdDC+d29/+BitXwnnnxR2JSF7SHQlFWuIORx8dXHH37LPjjkYk\na3JxR8IlhCcRRrn7R9PZqEjee/JJ2LQJvvSluCMRyVup7sL6ODAynI4lOLHwN5kIwMzGmtkCM1tk\nZtclKTfSzJrM7HOZ2K5Iq7Zvh+99L7jnR1GqXxGRrielb4e7r41MK9z9DuAzHd24mRUB9wCfBg4F\nzglPWGyp3G3AXzu6TZE2Pf44mMEZZ8QdiUheS3UX1ojIbBFBjyTVy6AkUwO86e5Lw+08CpwOLGhW\n7krgMYIekEj2bNsWHHX14x8HSUREWpVqEvh/kedbgQYgEzuHBwLLI/PvECSVHcxsb+AMdx9jZrss\nE8m43/0uuE3tySfHHYlI3kspgbj7mGwHksQdQHRsJOm/hXV1dTue19bWUltbm5WgpBPaujW44u59\n96n3IZ1WfX099fX1Gakr1WthVQE3AceFL/0d+L67r+/Qxs1GAXXuPjacv57gXusTImXeTjwF9gA2\nAZe5+5QW6tNhvJK+iRPhN7+BZ59VApEuIxcXU5wMvA48GL70ZeBwd+/QEVHhPUYWElzl9z1gBnBO\na1f6NbNfAVPd/Q+tLFcCkfR88AEccABMngxHHRV3NCI5k/XzQID93P3zkfmbzey1dDYY5e7bzGw8\n8BTB4PxEd59vZuOCxf5A81U6uk2RFt11F4wapeQh0g6p9kBeBL7t7s+H86OBn7j70VmOr13UA5G0\nrF0LBx4IL7wQ9EJEupBc7MI6gmD3VVX40vvABe4+J52NZosSiKTlW98KdmHdd1/ckYjkXC4SSHeC\n29juB/QG1hPsYvp+OhvNFiUQabeGBvjYx+CNN2DAgLijEcm5XIyB/AlYB7wKrEhnQyJ56Xvfgyuu\nUPIQSUOqCWRQ4lBbkU5j9mx46ilYtCjuSEQKUqpXinvBzIZlNRKRXHIPxj5uvDE481xE2i1pD8TM\n5hIcOtsNuCg8qW8zwUl97u7Dsx+iSBZMnQrvvgvjxsUdiUjBSjqIbmZDkq2cuAhivtAguqRk82Y4\n7DC45x749KfjjkYkVlkbRM+3BCGSEXffHZz3oeQh0iG6pa10LY2NcMgh8M9/BklEpIvL+nkghUIJ\nRNo0bhz06AG33x53JCJ5IRfngYgUvpkzYcoUmDcv7khEOgXd8Fm6hm3b4OtfhwkToE+fuKMR6RSU\nQKRr+PnPoWdP+PKX445EpNPQGIh0fitXwrBhUF8Phx4adzQieUWD6CElEGnR+efDoEFw221xRyKS\ndzSILtKap5+G558PrrYrIhmlMRDpvDZsgEsvhfvvD8Y/RCSjtAtLOq/LLw8uWzJxYtyRiOQt7cIS\naW7atOCcj9dfjzsSkU5Lu7Ck89m0CS65JDh0t3fvuKMR6bS0C0s6n/HjYf16ePjhuCMRyXvahSWS\n8MQTwfTaa3FHItLpKYFI5/Hee8Guq8ce064rkRyIfQzEzMaa2QIzW2Rm17Ww/Fwzmx1Oz+vWutKi\n7dvhggvga1+DT3wi7mhEuoRYE4iZFQH3AJ8GDgXOMbODmhV7GzjO3Q8HfgD8IrdRSkG4/fZg8PzG\nG+OORKTLiHsXVg3wZuLOh2b2KHA6sCBRwN1fipR/CRiY0wgl/z3/PPz4xzB9OnSL+yMt0nXEvQtr\nILA8Mv8OyRPEJcBfshqRFJZ334WzzoJf/xqqq+OORqRLKZh/18xsDHARkHQHd11d3Y7ntbW11NbW\nZjUuidGWLfCFLwT3+Tj55LijESkI9fX11NfXZ6SuWM8DMbNRQJ27jw3nrwfc3Sc0KzccmAyMdfe3\nktSn80C6kq9/PTjy6g9/gKK4O9MihamQzwOZCQw1syHAe8DZwDnRAma2D0Hy+HKy5CFdzM9+Flyu\nZMYMJQ+RmMSaQNx9m5mNB54iGI+Z6O7zzWxcsNgfAL4L9AV+ZmYGNLl7TXxRS+ymTIFbbgkGz3v1\nijsakS5LlzKRwjJ9Opx6Kvz5zzByZNzRiBS8juzCUt9fCsdbb8EZZ8CvfqXkIZIHlECkMCxdCiec\nAHV1QQ9ERGKnBCL5b/lyOP54+MY3YNy4uKMRkZASiOS3FStgzBi44gq4+uq4oxGRCCUQyV8NDUHy\nuOwy+OY3445GRJpRApH8NHt2cFXdK6+Ea6+NOxoRaUHcJxKK7G7atOD6VvfeC1/8YtzRiEgr1AOR\n/PLgg0Hy+P3vlTxE8px6IJIftmwJxjmefhrq6+GQQ+KOSETaoAQi8XvvvaC30bdvcG2rqqq4IxKR\nFGgXlsTrj3+EESPgxBOD50oeIgVDPRCJx7/+BddcE+yueuwxGD067ohEpJ3UA5Hce+IJOPzw4DLs\nr72m5CFSoNQDkdxpaAjOJp8/H+6/H046Ke6IRKQD1AOR7Fu3Dm68ET7+caipgblzlTxEOgElEMme\nDz6ACRNg//3h3XfhlVfghhuge/e4IxORDNAuLMm8NWuCW87eey8cdxw89xwcdFDcUYlIhqkHIpkz\nezZcfnnQ41i2LDjC6n/+R8lDpJNSD0Q65v334ZFHYOJEWL0aLrooGCQfMCDuyEQky3RPdGm/detg\n6lSYPDnoZYwdCxdfHNz0qbg47uhEpB06ck90JRBp29at8PLL8OyzwTRjRnCfjs9/Hk47Dfr0iTtC\nEUmTEkhICSRDtm6F11/fmTCeew6qq4MexvHHQ20tVFbGHaWIZIASSEgJJA3r1sGcOcEAeGKaNw8G\nDw56GYmE0a9f3JGKSBYUdAIxs7HAHQRHhE109wktlLkLOBnYBFzo7q+1UpcSSEs2bYKlS4OpoQHe\nfhsWLgwSx5o1MGxYcGmRxDRsmHoYIl1EwSYQMysCFgGfAt4FZgJnu/uCSJmTgfHu/hkzOwq4091H\ntVJf1hLI6tWraWhooLq6mn7t/G98/vz5zJgxg5qaGvbYY49d6knUW1FRwcaNG3erf/Xq1cyaNYt1\n69bRu3dvjjzySABmvfIKJRs2UPXvf7P6jTeo7tEDX7mS7StX0uvDD6navJmmpUspa2ykrKmJpr32\nYmVZGR/268eGPfZgWXk55SNHMvLss+nX7IipaLwHH3zwLjEA9O7dm9WrV/PII49QUVHBaaedxtKl\nS2lsbGTFihUsWLCAyspKjj32WKqqqliyZAkvvvgiFRUV9O3bl82bN7NixQo2bNjAgQceSFV4Bd7G\nxkaWLVtGTU0NtbW1jBgxghUrVjB16lQWLVrEgQceyKmnnsratWv57W9/S79+/bj44otZuXIlv/zl\nL1m+fDmjRo3ioPCw4cMPP5xhw4Yxd+5cXnjhBdavX8/o0aMZPXo0Gzdu3NHmW7ZsYfHixQwdOpTS\n0tIdrzd/jP5tEm0CMHjw4Bb/dql+rjJRj0i6OpJAcPfYJmAU8JfI/PXAdc3K/Bw4KzI/H+jfSn2e\nDZMmPerl5X29qmqEl5f39UmTHm298Pbt7lu2uH/wgfv69X7tJeN8T8p8EPv6/pT4kcU9fEzPA/2E\n0kq/57Nn+BdKK/z8kj39K5T4VSV7+w3dyv31089wv/JKf+u44/xx6+ZPY/4SRT6PEn8H83+BbwVf\ng/kbmE+j1B+lyO+iyG+kxC+lyk+n2Gvo7v3Z141uDt3DqSR87OGwn3frVrnL+xk//mqHcocDHMr9\nxBNP9pKSysj6PcLl3cLHPmGdJQ7F4RTdTnFYNjElypREXitptl65w16R16Mxd49se69InYl1o2UH\nRLYTfa3cS0r2cSj3oqJ+YV17O5R7cfGgXZYnHsvLh+3420+a9GjYJj3CGHZd3p7PVSbqEemI8Hcz\nvd/wdFfMxAR8HnggMn8+cFezMlOBYyLzzwAjWqkvU226Q2Njo5eX9/Ufc4HP4yB/k8HeYEW+dcAA\n9z33dO/Tx72y0r2szL24OGjS4mL3sjLf2qOHrwVfRR9fxt6+iCKfw1Cfzkj/O8P9rxT5nzjGf0ep\nP8ipfh/jfAIXel23cn/vO9/xS7qV++fo4SdQ4TU87AfxvO9NpVfS243J4Q/fNIfeDlXhj/k0h17h\nstkO8xzKwuWV4fM+4TJ3mO1lZX28sbHR582bF1nPw7oS6ybqv8uhNCz3eCSxJBJBWWQ7ZWHZxFTi\nUBSWT7zW3cEi6yXeU6/ID38i5lsj257mUNFs3WjZRB2lzV7rG3l8PFJX3ySP0bbq7WVlvZvVt3N5\neXlfb2xsTOlzlYl6RDqqIwmk051IWFdXt+N5bW0ttbW1HaqvoaGB0tJq7v7w+/yKjTRRQlnPM3j4\nFxM4YuRI6NYNSkp2fbSgN/ibBx/kwgt/CCwk2Dv3NeCVsOaZBPnyjvD1qTu22avHbKoPPJBHuu/D\npq0APcOyM4G9w/kNwODwef9wzZ7h1BsoA4YDDwJ7AuXAh2G5PcJlAMMxG0hDQwPz5s0L6xweqa8f\n0CMyPzt83ANYH24LYCPBMFZvINEbXhtuN2Er8G+gKnzuQAnwf+F2LIw78R62hc/Lwu1Nj2w78V4/\njKy7GegeWd47jCvxPnoC1ZHH9ZE2rE7yuLOtiov3ZNs2BypbXF5SMoSGhoY2d0E1NDRQXNw/8j7S\nq0ekverr66mvr89IXXEnkBXAPpH5QeFrzcsMbqPMDtEEkgnV1dVs2dLActYRfMHnUL5tJQOPOqrN\nI5NqamqA5cAcgh+IJeHz4QTHAywPHxsir8+hqWkpNTU1bNu2iuBHtihSx8pwviqyfrTcJmAd0BSu\nUwM0EvzOweclAAAHv0lEQVSwbg9f37DL9txXUF1dTUVFRSTeRIyrw3UJ678QeJidiWBduGwbwY/4\naoKk0BS+vinSIh6WWx8+J1xnW2S9rZH3sDmc/1cY8+eB/w23vSmctkfW3RLWtyFSx7+BDwgSSaKt\nE4/RNmxI8rizrbZtawzjXtPi8qampVRXV9OW6urq8O9rHapHpL2a/2N98803p19Zul2XTExAMbAY\nGAKUAq8BBzcrcwrwZPh8FPBSkvoy16+LSIyB9Op1ZLv3T48ff1W4m2R/h1Lv1q1yRz3jx1/l5eV9\nvaysOtz/fdgu9Qf7yCs8OmZRXNzDi4t7hPNVvnMsoDRSrr/vHCcYGj5PLC8Jn7c2BhKNt9xPOunk\nSAzRcYjisFxv3zmWYeEuqtLIdsx3jnsUh8vNdx0PSaybWK/cg7GKbpGyiZhLItseED6Prhst2z+y\nnehr5V5SMtiDMZCPRNqw3IuLB+6yPPEY/dvs/LvsrK/53y7Vz1Um6hHpCDqwCytfDuO9k52H8d5m\nZuPCN/VAWOYeYCzBv2oXufurrdTl2Xo/eXcUVnjkTs+ePXccQbRp06Yd5QYPHszcuXNZtWoVJ5xw\nAgDPPPMM5eXBLqXGxkaGDh3KmDFjdns/OgpLR2FJ11Gwh/Fmms4DERFpn44kEF3OXURE0qIEIiIi\naVECERGRtCiBiIhIWpRAREQkLUogIiKSFiUQERFJixKIiIikRQlERETSogQiIiJpUQIREZG0KIGI\niEhalEBERCQtSiAiIpIWJRAREUmLEoiIiKRFCURERNKiBCIiImlRAhERkbQogYiISFqUQEREJC1K\nICIikpbYEoiZ9TGzp8xsoZn91cyqWigzyMyeNbM3zGyumV0VR6wiIrK7OHsg1wPPuPuBwLPAd1oo\nsxX4prsfChwNXGFmB+Uwxoyrr6+PO4SUKM7MUpyZpTjzQ5wJ5HTgwfD5g8AZzQu4+0p3fy18vhGY\nDwzMWYRZUCgfKMWZWYozsxRnfogzgezp7qsgSBTAnskKm1k1cAQwPeuRiYhIm7pls3IzexroH30J\ncODGFop7knoqgMeAq8OeiIiIxMzcW/3dzu6GzeYDte6+yswGANPc/eAWynUDngD+4u53tlFnPG9G\nRKSAubuls15WeyBtmAJcCEwALgD+1Eq5/wbmtZU8IP1GEBGR9ouzB9IX+D0wGFgKfMnd15nZXsAv\n3P1UMxsN/AOYS7CLy4H/dPf/jSVoERHZIbYEIiIiha3gzkQ3s7FmtsDMFpnZda2UucvM3jSz18zs\niFzHGMaQNE4z+6SZrTOzV8OppQMLsh3jRDNbZWZzkpTJh7ZMGmc+tGUYR0onvsbdpqnEGXebmll3\nM5tuZrPCGG9qpVzcbdlmnHG3ZbNYisIYprSyvH3t6e4FMxEkvMXAEKAEeA04qFmZk4Enw+dHAS/l\naZyfBKbE3J6fIDg0ek4ry2NvyxTjjL0twzgGAEeEzyuAhXn6+UwlztjbFOgRPhYDLwE1+daWKcYZ\ne1tGYrkG+E1L8aTTnoXWA6kB3nT3pe7eBDxKcEJi1OnAQwDuPh2oMrP+5FYqcUJwWHNs3P154P0k\nRfKhLVOJE2JuS0j5xNfY2zTFOCH+z+cH4dPuBAf8NN/fHntbhttuK07Ig8+nmQ0CTgF+2UqRdrdn\noSWQgcDyyPw77P7Bb15mRQtlsi2VOAGODruKT5rZIbkJrV3yoS1TlVdtmeTE17xq0zZO0I21TcPd\nLbOAlcDT7j6zWZG8aMsU4oT8+HzeDnyb1s+5a3d7FloC6UxeAfZx9yOAe4A/xhxPIcurtiyUE1/b\niDP2NnX37e5+JDAIOCof/jFoSQpxxt6WZvYZYFXY8zQy1CMqtASyAtgnMj8ofK15mcFtlMm2NuN0\n942Jrq+7/wUoCQ9tzif50JZtyqe2DE98fQx42N1bOrcpL9q0rTjzqU3d/V/ANGBss0V50ZYJrcWZ\nJ205Gvismb0NPAKMMbOHmpVpd3sWWgKZCQw1syFmVgqcTXBCYtQU4CsAZjYKWOfhNbdyqM04o/sW\nzayG4JDq/8ttmMHmaf2/kXxoy4RW48yjtoS2T3zNlzZNGmfcbWpme1h4iwczKwdOBBY0KxZ7W6YS\nZ9xtCeDu/+nu+7j7Rwl+j5519680K9bu9ozzTPR2c/dtZjYeeIog+U109/lmNi5Y7A+4+5/N7BQz\nWwxsAi7KxziBL5jZ14Em4EPgrFzHaWaTgFrgI2a2DLgJKCWP2jKVOMmDtgzjHA2cB8wN94k78J8E\nR+PlTZumEifxt+lewINmVkTwHfpd2HZ59V1PJU7ib8tWdbQ9dSKhiIikpdB2YYmISJ5QAhERkbQo\ngYiISFqUQEREJC1KICIikhYlEBERSYsSiEgOmdmSPLzigEhalEBEcksnXkmnoQQikiVm9riZzQxv\nNHRJ4uVYgxLJoIK6lIlIgbnI3deZWRkw08z+EHdAIpmkBCKSPd8wszPC54OA/dEuLOlElEBEssDM\nPgkcDxzl7pvNbBpQFnNYIhmlMRCR7KgC3g+Tx0HAqPB1jYFIp6EEIpId/0tw46A3gB8CL4SvaxeW\ndBq6nLuIiKRFPRAREUmLEoiIiKRFCURERNKiBCIiImlRAhERkbQogYiISFqUQEREJC1KICIikpb/\nD83g+mCZg3q1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f442db18048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the predicted probabilities\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_prob, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.97161726  0.02838274]]\n",
      "[[ 0.34361555  0.65638445]]\n",
      "[[ 0.00794192  0.99205808]]\n"
     ]
    }
   ],
   "source": [
    "# examine some example predictions\n",
    "print(logreg.predict_proba(1))\n",
    "print(logreg.predict_proba(2))\n",
    "print(logreg.predict_proba(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first column indicates the predicted probability of **class 0**, and the second column indicates the predicted probability of **class 1**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Probability, odds, e, log, log-odds\n",
    "\n",
    "$$probability = \\frac {one\\ outcome} {all\\ outcomes}$$\n",
    "\n",
    "$$odds = \\frac {one\\ outcome} {all\\ other\\ outcomes}$$\n",
    "\n",
    "Examples:\n",
    "\n",
    "- Dice roll of 1: probability = 1/6, odds = 1/5\n",
    "- Even dice roll: probability = 3/6, odds = 3/3 = 1\n",
    "- Dice roll less than 5: probability = 4/6, odds = 4/2 = 2\n",
    "\n",
    "$$odds = \\frac {probability} {1 - probability}$$\n",
    "\n",
    "$$probability = \\frac {odds} {1 + odds}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>probability</th>\n",
       "      <th>odds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.111111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.20</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.25</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.60</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.80</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.90</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   probability      odds\n",
       "0         0.10  0.111111\n",
       "1         0.20  0.250000\n",
       "2         0.25  0.333333\n",
       "3         0.50  1.000000\n",
       "4         0.60  1.500000\n",
       "5         0.80  4.000000\n",
       "6         0.90  9.000000"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a table of probability versus odds\n",
    "table = pd.DataFrame({'probability':[0.1, 0.2, 0.25, 0.5, 0.6, 0.8, 0.9]})\n",
    "table['odds'] = table.probability/(1 - table.probability)\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is **e**? It is the base rate of growth shared by all continually growing processes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.7182818284590451"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# exponential function: e^1\n",
    "np.exp(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is a **(natural) log**? It gives you the time needed to reach a certain level of growth:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.99989631572895199"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# time needed to grow 1 unit to 2.718 units\n",
    "np.log(2.718)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also the **inverse** of the exponential function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.log(np.exp(5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>probability</th>\n",
       "      <th>odds</th>\n",
       "      <th>logodds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.111111</td>\n",
       "      <td>-2.197225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.20</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>-1.386294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.25</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>-1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.60</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>0.405465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.80</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.386294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.90</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>2.197225</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   probability      odds   logodds\n",
       "0         0.10  0.111111 -2.197225\n",
       "1         0.20  0.250000 -1.386294\n",
       "2         0.25  0.333333 -1.098612\n",
       "3         0.50  1.000000  0.000000\n",
       "4         0.60  1.500000  0.405465\n",
       "5         0.80  4.000000  1.386294\n",
       "6         0.90  9.000000  2.197225"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add log-odds to the table\n",
    "table['logodds'] = np.log(table.odds)\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  What is Logistic Regression?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Linear regression:** continuous response is modeled as a linear combination of the features:\n",
    "\n",
    "$$y = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "**Logistic regression:** log-odds of a categorical response being \"true\" (1) is modeled as a linear combination of the features:\n",
    "\n",
    "$$\\log \\left({p\\over 1-p}\\right) = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "This is called the **logit function**.\n",
    "\n",
    "Probability is sometimes written as pi:\n",
    "\n",
    "$$\\log \\left({\\pi\\over 1-\\pi}\\right) = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "The equation can be rearranged into the **logistic function**:\n",
    "\n",
    "$$\\pi = \\frac{e^{\\beta_0 + \\beta_1x}} {1 + e^{\\beta_0 + \\beta_1x}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In other words:\n",
    "\n",
    "- Logistic regression outputs the **probabilities of a specific class**\n",
    "- Those probabilities can be converted into **class predictions**\n",
    "\n",
    "The **logistic function** has some nice properties:\n",
    "\n",
    "- Takes on an \"s\" shape\n",
    "- Output is bounded by 0 and 1\n",
    "\n",
    "We have covered how this works for **binary classification problems** (two response classes). But what about **multi-class classification problems** (more than two response classes)?\n",
    "\n",
    "- Most common solution for classification models is **\"one-vs-all\"** (also known as **\"one-vs-rest\"**): decompose the problem into multiple binary classification problems\n",
    "- **Multinomial logistic regression** can solve this as a single problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6: Interpreting Logistic Regression Coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f442daeacc0>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEPCAYAAABsj5JaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8FPW9//HXJyEhgYQAFUEBiRXvgkpLRKk2WLVotWpv\nXlu1XmgVtbanao+2xlpb6emv3mu15bRqi7ZHagvanqqVtFoVUBFQbqIEEIUARyighQCf3x8zC0NI\nNpvN7s5u8n4+HvPYnZ3vfOez3+zuJ9/5zsXcHRERkfYqijsAEREpTEogIiKSFiUQERFJixKIiIik\nRQlERETSogQiIiJpiT2BmNlEM1tlZnNaWX6umc0Op+fNbFiuYxQRkd3FnkCAXwGfTrL8beA4dz8c\n+AHwi5xEJSIiSXWLOwB3f97MhiRZ/lJk9iVgYPajEhGRtuRDD6Q9LgH+EncQIiKSBz2QVJnZGOAi\n4BNxxyIiIgWSQMxsOPAAMNbd309SThf2EhFpJ3e3dNbLl11YFk67LzDbB5gMfNnd32qrInfP6+mm\nm26KPQbFqTgVp+JMTB0Rew/EzCYBtcBHzGwZcBNQCri7PwB8F+gL/MzMDGhy95q44hURkUDsCcTd\nz21j+aXApTkKR0REUpQvu7C6jNra2rhDSInizCzFmVmKMz9YR/eB5RMz8870fkREss3M8AIfRBcR\nkQKjBCIiImlRAhERkbQogYiISFqUQEREJC1KICIikhYlEBERSYsSiIiIpEUJRERE0qIEIiIiaVEC\nERGRtCiBiIhIWpRAREQkLUogIiKSFiUQERFJixKIiIikRQlERETSogQiIiJpUQIREZG0KIGIiEha\nlEBERCQtsScQM5toZqvMbE6SMneZ2Ztm9pqZHZHL+EREpGXd4g4A+BVwN/BQSwvN7GRgP3ff38yO\nAn4OjMphfC1avXo1DQ0NVFdX069fv9jrySfz589nxowZ1NTUcPDBB7f6Htt676m0TaJMRUUFGzdu\nZNWqVcycOZORI0fSv39/qqurAXbUAzBr1iwAjjzyyBbjSdRVUVHB8uXLWbdu3Y4yvXv3ZvDgwSxf\nvjxpHYltTZs2jcWLFzN06FCGDRvGxo0bd4kpsY226urXr99udUdjjbZRKuu11eYt1ZtK+6caR7p/\n73TjjENn/G7vxt1jn4AhwJxWlv0cOCsyPx/o30pZz4VJkx718vK+XlU1wsvL+/qkSY/GWk8+GT/+\naodyhwMcyv3EE09u8T229d5TaZtEmfLyj4bbrAof93Io9+LiA72kpNJLS6u8qmqEl5RUenFxT4ce\nDkO9tLRqt3gSdZWU7OPQvdnUI6y7e9I6EtsyK4uUHeBQ7uXlw3bEFGyr7brKy/v6+PFX71L3zvWD\nOhNtlMp6bbV5S/W2JFn5ZHGk+/dON844FNJ3O/zdTO+3O90VMzm1kUCmAsdE5p8BRrRSNjMtmkRj\nY6OXl/d1mO3gDrO9vLyvNzY2xlJPPpk3b174A554T9OazQfvcd68eUnfeypts7PMNIe+Dr8Ot5WY\nn+3Q6NAn8rwqMt9SPIl1pzn0DstXRdZLvJ6sjsS2Kh3KIus1jynVupq3Y2MLdQbrlpX1TmG9ttp8\n93pb+lwmK588jvT+3qlsN1++P4X23e5IAsmHXVgZVVdXt+N5bW0ttbW1Ga2/oaGB0tJqPvxwePjK\ncEpKhtDQ0NCubmqm6sknM2bMAAYDiffUExgUmQ/e44wZM5K+91TaZmeZnkA18Ha4rcT8cGAmsG/k\n+YBw+c56i4oGReJJrNsT6N/s3fWMvJ6sjsS2+oTL92glplTrSmx7cAvrV++ybnHxnkB5s/Zvvl5b\nbb57vS19LpOVTx5Hsm23/V1Itt3SbvuwdNEi+pWVwZYtyafNm9suk+Y6Pdat45XNH9CNz3MI89ia\nZ9/t+vp66uvrM1JXISSQFQSfvoRB4WstiiaQbKiurmbLlgZgDsGHdw5NTUt37F/OdT35pKamBljO\nzve0CXiH5u+xpqYm6XtPpW12ltkENAAfDbeVmJ9D8AOzJPJ8JcFxIzvr3b79nUg8iXU3AasAj7y7\nosjrlqSOxLbeB5qADa3ElGpdiXZc3sL6u7bRtm2NmBW1sV5bbb57vS19LpOVTx5H8r93EYfSl+cY\nuPlthq5aBVOmwJo1sHYtrF3LYStW8NCGufTmEj7CG3yEPajkQ0rZQsmGrfiYMdC9O5SWtm9Ktk5F\nRbvKb9m0ifNPOZN/bf4p2yhutQ3j0vwf65tvvjn9ytLtumRyIvh0z21l2SnAk+HzUcBLSerJTJ+u\nDYn9m716HZmRMZCO1pNPxo+/Ktxdsb9DuZ900sktvse23nsqbZMoU1ZWHW6zMnwMxhuKiw/wkpIK\nLy2t8l69jvSSkgovLu7hwZjDfi2OOSTqKikZ7FDqu4+B9A9fb72OxLbMukfK9vdgf/1hO2IKttV2\nXcEYwlW71L1z/aDO5mMPydZrq81bqrclycq3FsegyuE+snsv//u3/sP9zjvdr7nG/XOf87XV+/pq\nzJvA12C+fsAA96OPdj/1VPcLLnD/5jfdb73V/f77/R/fuMZPKq30j5Xu7QMo835lB3uvsj4+6beP\ndOizm0mF9N2mA7uwLFg/PmY2CagFPkLwL9lNQCnBm3ogLHMPMJbgX5mL3P3VVuryXL0fHYXVOh2F\npaOw2LCBdc8+y/rnn2fPNWsoX7KE7W+8gX/wAT5kCN0++lGorg6mIUOgupo1PXqwZMMGqvfbT0dh\n5ZCZ4e6W1rpxJ5BMymUCEZHQli3w6qswfTq8/HIwLVsGw4cH06GHwiGHBI8DBoCl9VslWaIEElIC\nEcmR1ath6lR4/HGor4f99oOjj4aPfzyYDjkESkrijlJSoAQSUgIRyaIlS+CPfwySxuzZcNJJcOaZ\ncPLJ0KdP2+tLXlICCSmBiGSQO8yZszNpvPsufPazcMYZcMIJUFYWd4SSAUogISUQkQxYsQJ+8Qt4\n6KEgiZx5ZjAdcwwUF8cdnWRYRxJIIZwHIiK58PrrcMst8PTTcM45MHkyHHGEBr2lVbFfjVdEYrZ4\nMZx3HnzqUzByJCxdCvfeC0ceqeQhSSmBiHRVy5bBpZfCqFFw8MFBIvmP/4DKyrgjkwKhBCLS1axc\nCVdfHfQw+vWDRYvgxhuVOKTdlEBEuoqmJrj11uCEvqIimD8ffvhD6Ns37sikQGkQXaQreOUV+OpX\nYeDA4KzxIUPijkg6AfVARDqzpia44QY45RT49rfhySeVPCRj1AMR6ayWLYOzz4bevYMTAvs3v8eJ\nSMeoByLSGU2ZEhySe+aZ8MQTSh6SFeqBiHQm7nDbbXDffcElSI4+Ou6IpBNTAhHpLLZtgyuvhH/+\nE156CfbeO+6IpJNTAhHpDD78EM49FzZsgOeeg1694o5IugCNgYgUujVrgsuQ9OwJf/6zkofkjBKI\nSCFbsgRGj4ZPfjK4em5padwRSReiXVgiherNN2HMGPjOd+CKK+KORrog3Q9EpBAtXw7HHhtcw+qS\nS+KORgpYR+4Hol1YIoVm1argjoBXX63kIbFSAhEpJO+/H9yL/Jxz4Jpr4o5GujjtwhIpFJs2BT2P\nUaPgpz/VzZ4kIwp6F5aZjTWzBWa2yMyua2F5LzObYmavmdlcM7swhjBF4uUOF10E++2n5CF5I9aj\nsMysCLgH+BTwLjDTzP7k7gsixa4A3nD3z5rZHsBCM/uNu2+NIWSReNx6a3Cr2b//XclD8kbch/HW\nAG+6+1IAM3sUOB2IJhAHErdKqwTWKnlIl/L443D//TBjBpSVxR2NyA5xJ5CBwPLI/DsESSXqHmCK\nmb0LVABn5Sg2kfjNmQOXXRacYb7XXnFHI7KLuBNIKj4NzHL3481sP+BpMxvu7htbKlxXV7fjeW1t\nLbW1tTkJUiTj1qyB00+HO+8MLs0ukgH19fXU19dnpK5Yj8Iys1FAnbuPDeevB9zdJ0TKPAH8yN3/\nGc7/DbjO3V9uoT4dhSWdgzucdhocfDD813/FHY10YoV8FNZMYKiZDTGzUuBsYEqzMkuBEwDMrD9w\nAPB2TqMUybW774bVq+GHP4w7EpFWxX4eiJmNBe4kSGYT3f02MxtH0BN5wMz2An4NJHYA/8jdH2ml\nLvVApPDNmhWcLPjSS8FhuyJZ1JEeSOwJJJOUQKTgbdwIH/sY3HRTcH8PkSxTAgkpgUjBu/ji4M6C\nv/513JFIF9GRBFIIR2GJdA2//31wN8FXX407EpGUqAcikg8aG2H4cJgyBWqanwolkj3ahRVSApGC\n9aUvwb77woQJbZcVySDtwhIpZJMnB2ecP/hg3JGItIt6ICJxWrMGhg0Lksgxx8QdjXRB2oUVUgKR\ngnP++dCvH9x+e9yRSBelXVgihWjatOCoq3nz4o5EJC1xX8pEpGtqaoIrroA77oCePeOORiQtSiAi\ncbjzThgyBM44I+5IRNKmMRCRXFuxAg4/HF58EfbfP+5opIvTIHpICUQKwjnnBBdJ/MEP4o5ERIPo\nIgVj2rSg5zFxYtyRiHSYxkBEciU6cN6jR9zRiHSYEohIriQGzk8/Pe5IRDJCYyAiubByJRx2mAbO\nJe9oED2kBCJ569JLoaoKfvKTuCMR2YUG0UXy2ezZwWXaFy6MOxKRjNIYiEi2XXstfPe70Lt33JGI\nZJQSiEg2/e1v8NZbcNllcUciknFKICLZsn07XHcd3HorlJbGHY1IxiUdAzGzuUCro9LuPjzjEYl0\nFo89Fjx+8YvxxiGSJW0Nop8aPl4RPj4cPp6XnXBEOommJrjhBrjvPihSR186p5QO4zWzWe5+ZLPX\nXnX3ER0OwGwscAfB7rSJ7r7bTaHNrBa4HSgBVrv7mFbq0mG8kh/uuw8efxyeeiruSESSysVhvGZm\no939n+HMMWRg/MTMioB7gE8B7wIzzexP7r4gUqYKuBc4yd1XmNkeHd2uSFZt3Ai33AJPPBF3JCJZ\nlWoCuRj47/DH3ID3ga9mYPs1wJvuvhTAzB4FTgcWRMqcC0x29xUA7r4mA9sVyZ477oDaWhjR4Q66\nSF5LKYG4+yvA4WECwd3XZ2j7A4Hlkfl3CJJK1AFAiZlNAyqAu9z9YUTy0erVQQKZPj3uSESyrq2j\nsL7ZyusAuPtPsxBTc92AEcDxQE/gRTN70d0Xt1S4rq5ux/Pa2lpqa2tzEKJI6Ec/grPPDu73IZKH\n6uvrqa+vz0hdSQfRzeymZCu7+80d2rjZKKDO3ceG89cH1e4cSDez64CyxLbM7JfAX9x9cgv1aRBd\n4vPuuzBsGLzxBgwYEHc0Iikp2IspmlkxsJBgEP09YAZwjrvPj5Q5CLgbGAt0B6YDZ7n7vBbqUwKR\n+Fx1VXDCoC6YKAUk60dhmdkggh/x0eFLzwFXu/s76Ww0wd23mdl44Cl2HsY738zGBYv9AXdfYGZ/\nBeYA24AHWkoeIrF65x347W9h/vy2y4p0EqmeB/I0MImdJxKeD5zn7idmMbZ2Uw9EYnP55VBZCRN2\nO41JJK9lfReWmb3m7ke09VrclEAkFkuXBofsLlwIe+g0JSksHUkgqZ4MuNbMzjez4nA6H1ibzgZF\nOp1bb4WvfU3JQ7qcVHsgQwjGQI4muLjiC8BV7r4su+G1j3ogknNvvw01NbBoEfTtG3c0Iu1WsEdh\nZZoSiOTcV78KgwfDzR06ol0kNrk4CqsfcClQHV3H3TNxORORwrR4MUydCm++GXckIrFI9VpYfyI4\ndPcZgkNpReSWW4JzP3SrWumi0j4KKx9pF5bkzMKFcOyxQS+kV6+4oxFJWy6OwnrCzE5JZwMindL3\nvw/f+IaSh3RpbV0LawPBUVdGcCHDLeFkBGeK59W3Rz0QyYl582DMmKD3UVkZdzQiHZK1QXR317dD\npLmbb4ZvfUvJQ7q8VMdAjOA+6Pu6+y1mNhjYy91nZDvA9lAPRLJu7lw48UR46y3o2TPuaEQ6LBdj\nID8jOInw3HB+I8FtZkW6lro6uPZaJQ8RUj+M9yh3H2FmswDc/X0zK81iXCL5Z9YsePFFeFg3xBSB\n1HsgTeG9Oxx2nFi4PWtRieSjujq4/nro0SPuSETyQqo9kLuAx4E9zexW4AvAjVmLSiTfvPwyvPIK\n/O53cUcikjdSvhZWeGfATxEcwvu36F0D84UG0SVrPvOZYLr88rgjEcmoXNwPZD/gHXffbGa1wHDg\nIXdfl85Gs0UJRLJi+nT44heDa1517x53NCIZlYujsCYD28xsKHA/MJjgDoUind93vws33KDkIdJM\nqglku7tvBT4H3OPu3wb2yl5YInniH/8Izjj/qi48LdJce47COgf4CvBE+FpJdkISyRPuQe/jppug\nRB93keZSTSAXEZxIeKu7LzGzfQEdDC+d29/+BitXwnnnxR2JSF7SHQlFWuIORx8dXHH37LPjjkYk\na3JxR8IlhCcRRrn7R9PZqEjee/JJ2LQJvvSluCMRyVup7sL6ODAynI4lOLHwN5kIwMzGmtkCM1tk\nZtclKTfSzJrM7HOZ2K5Iq7Zvh+99L7jnR1GqXxGRrielb4e7r41MK9z9DuAzHd24mRUB9wCfBg4F\nzglPWGyp3G3AXzu6TZE2Pf44mMEZZ8QdiUheS3UX1ojIbBFBjyTVy6AkUwO86e5Lw+08CpwOLGhW\n7krgMYIekEj2bNsWHHX14x8HSUREWpVqEvh/kedbgQYgEzuHBwLLI/PvECSVHcxsb+AMdx9jZrss\nE8m43/0uuE3tySfHHYlI3kspgbj7mGwHksQdQHRsJOm/hXV1dTue19bWUltbm5WgpBPaujW44u59\n96n3IZ1WfX099fX1Gakr1WthVQE3AceFL/0d+L67r+/Qxs1GAXXuPjacv57gXusTImXeTjwF9gA2\nAZe5+5QW6tNhvJK+iRPhN7+BZ59VApEuIxcXU5wMvA48GL70ZeBwd+/QEVHhPUYWElzl9z1gBnBO\na1f6NbNfAVPd/Q+tLFcCkfR88AEccABMngxHHRV3NCI5k/XzQID93P3zkfmbzey1dDYY5e7bzGw8\n8BTB4PxEd59vZuOCxf5A81U6uk2RFt11F4wapeQh0g6p9kBeBL7t7s+H86OBn7j70VmOr13UA5G0\nrF0LBx4IL7wQ9EJEupBc7MI6gmD3VVX40vvABe4+J52NZosSiKTlW98KdmHdd1/ckYjkXC4SSHeC\n29juB/QG1hPsYvp+OhvNFiUQabeGBvjYx+CNN2DAgLijEcm5XIyB/AlYB7wKrEhnQyJ56Xvfgyuu\nUPIQSUOqCWRQ4lBbkU5j9mx46ilYtCjuSEQKUqpXinvBzIZlNRKRXHIPxj5uvDE481xE2i1pD8TM\n5hIcOtsNuCg8qW8zwUl97u7Dsx+iSBZMnQrvvgvjxsUdiUjBSjqIbmZDkq2cuAhivtAguqRk82Y4\n7DC45x749KfjjkYkVlkbRM+3BCGSEXffHZz3oeQh0iG6pa10LY2NcMgh8M9/BklEpIvL+nkghUIJ\nRNo0bhz06AG33x53JCJ5IRfngYgUvpkzYcoUmDcv7khEOgXd8Fm6hm3b4OtfhwkToE+fuKMR6RSU\nQKRr+PnPoWdP+PKX445EpNPQGIh0fitXwrBhUF8Phx4adzQieUWD6CElEGnR+efDoEFw221xRyKS\ndzSILtKap5+G558PrrYrIhmlMRDpvDZsgEsvhfvvD8Y/RCSjtAtLOq/LLw8uWzJxYtyRiOQt7cIS\naW7atOCcj9dfjzsSkU5Lu7Ck89m0CS65JDh0t3fvuKMR6bS0C0s6n/HjYf16ePjhuCMRyXvahSWS\n8MQTwfTaa3FHItLpKYFI5/Hee8Guq8ce064rkRyIfQzEzMaa2QIzW2Rm17Ww/Fwzmx1Oz+vWutKi\n7dvhggvga1+DT3wi7mhEuoRYE4iZFQH3AJ8GDgXOMbODmhV7GzjO3Q8HfgD8IrdRSkG4/fZg8PzG\nG+OORKTLiHsXVg3wZuLOh2b2KHA6sCBRwN1fipR/CRiY0wgl/z3/PPz4xzB9OnSL+yMt0nXEvQtr\nILA8Mv8OyRPEJcBfshqRFJZ334WzzoJf/xqqq+OORqRLKZh/18xsDHARkHQHd11d3Y7ntbW11NbW\nZjUuidGWLfCFLwT3+Tj55LijESkI9fX11NfXZ6SuWM8DMbNRQJ27jw3nrwfc3Sc0KzccmAyMdfe3\nktSn80C6kq9/PTjy6g9/gKK4O9MihamQzwOZCQw1syHAe8DZwDnRAma2D0Hy+HKy5CFdzM9+Flyu\nZMYMJQ+RmMSaQNx9m5mNB54iGI+Z6O7zzWxcsNgfAL4L9AV+ZmYGNLl7TXxRS+ymTIFbbgkGz3v1\nijsakS5LlzKRwjJ9Opx6Kvz5zzByZNzRiBS8juzCUt9fCsdbb8EZZ8CvfqXkIZIHlECkMCxdCiec\nAHV1QQ9ERGKnBCL5b/lyOP54+MY3YNy4uKMRkZASiOS3FStgzBi44gq4+uq4oxGRCCUQyV8NDUHy\nuOwy+OY3445GRJpRApH8NHt2cFXdK6+Ea6+NOxoRaUHcJxKK7G7atOD6VvfeC1/8YtzRiEgr1AOR\n/PLgg0Hy+P3vlTxE8px6IJIftmwJxjmefhrq6+GQQ+KOSETaoAQi8XvvvaC30bdvcG2rqqq4IxKR\nFGgXlsTrj3+EESPgxBOD50oeIgVDPRCJx7/+BddcE+yueuwxGD067ohEpJ3UA5Hce+IJOPzw4DLs\nr72m5CFSoNQDkdxpaAjOJp8/H+6/H046Ke6IRKQD1AOR7Fu3Dm68ET7+caipgblzlTxEOgElEMme\nDz6ACRNg//3h3XfhlVfghhuge/e4IxORDNAuLMm8NWuCW87eey8cdxw89xwcdFDcUYlIhqkHIpkz\nezZcfnnQ41i2LDjC6n/+R8lDpJNSD0Q65v334ZFHYOJEWL0aLrooGCQfMCDuyEQky3RPdGm/detg\n6lSYPDnoZYwdCxdfHNz0qbg47uhEpB06ck90JRBp29at8PLL8OyzwTRjRnCfjs9/Hk47Dfr0iTtC\nEUmTEkhICSRDtm6F11/fmTCeew6qq4MexvHHQ20tVFbGHaWIZIASSEgJJA3r1sGcOcEAeGKaNw8G\nDw56GYmE0a9f3JGKSBYUdAIxs7HAHQRHhE109wktlLkLOBnYBFzo7q+1UpcSSEs2bYKlS4OpoQHe\nfhsWLgwSx5o1MGxYcGmRxDRsmHoYIl1EwSYQMysCFgGfAt4FZgJnu/uCSJmTgfHu/hkzOwq4091H\ntVJf1hLI6tWraWhooLq6mn7t/G98/vz5zJgxg5qaGvbYY49d6knUW1FRwcaNG3erf/Xq1cyaNYt1\n69bRu3dvjjzySABmvfIKJRs2UPXvf7P6jTeo7tEDX7mS7StX0uvDD6navJmmpUspa2ykrKmJpr32\nYmVZGR/268eGPfZgWXk55SNHMvLss+nX7IipaLwHH3zwLjEA9O7dm9WrV/PII49QUVHBaaedxtKl\nS2lsbGTFihUsWLCAyspKjj32WKqqqliyZAkvvvgiFRUV9O3bl82bN7NixQo2bNjAgQceSFV4Bd7G\nxkaWLVtGTU0NtbW1jBgxghUrVjB16lQWLVrEgQceyKmnnsratWv57W9/S79+/bj44otZuXIlv/zl\nL1m+fDmjRo3ioPCw4cMPP5xhw4Yxd+5cXnjhBdavX8/o0aMZPXo0Gzdu3NHmW7ZsYfHixQwdOpTS\n0tIdrzd/jP5tEm0CMHjw4Bb/dql+rjJRj0i6OpJAcPfYJmAU8JfI/PXAdc3K/Bw4KzI/H+jfSn2e\nDZMmPerl5X29qmqEl5f39UmTHm298Pbt7lu2uH/wgfv69X7tJeN8T8p8EPv6/pT4kcU9fEzPA/2E\n0kq/57Nn+BdKK/z8kj39K5T4VSV7+w3dyv31089wv/JKf+u44/xx6+ZPY/4SRT6PEn8H83+BbwVf\ng/kbmE+j1B+lyO+iyG+kxC+lyk+n2Gvo7v3Z141uDt3DqSR87OGwn3frVrnL+xk//mqHcocDHMr9\nxBNP9pKSysj6PcLl3cLHPmGdJQ7F4RTdTnFYNjElypREXitptl65w16R16Mxd49se69InYl1o2UH\nRLYTfa3cS0r2cSj3oqJ+YV17O5R7cfGgXZYnHsvLh+3420+a9GjYJj3CGHZd3p7PVSbqEemI8Hcz\nvd/wdFfMxAR8HnggMn8+cFezMlOBYyLzzwAjWqkvU226Q2Njo5eX9/Ufc4HP4yB/k8HeYEW+dcAA\n9z33dO/Tx72y0r2szL24OGjS4mL3sjLf2qOHrwVfRR9fxt6+iCKfw1Cfzkj/O8P9rxT5nzjGf0ep\nP8ipfh/jfAIXel23cn/vO9/xS7qV++fo4SdQ4TU87AfxvO9NpVfS243J4Q/fNIfeDlXhj/k0h17h\nstkO8xzKwuWV4fM+4TJ3mO1lZX28sbHR582bF1nPw7oS6ybqv8uhNCz3eCSxJBJBWWQ7ZWHZxFTi\nUBSWT7zW3cEi6yXeU6/ID38i5lsj257mUNFs3WjZRB2lzV7rG3l8PFJX3ySP0bbq7WVlvZvVt3N5\neXlfb2xsTOlzlYl6RDqqIwmk051IWFdXt+N5bW0ttbW1HaqvoaGB0tJq7v7w+/yKjTRRQlnPM3j4\nFxM4YuRI6NYNSkp2fbSgN/ibBx/kwgt/CCwk2Dv3NeCVsOaZBPnyjvD1qTu22avHbKoPPJBHuu/D\npq0APcOyM4G9w/kNwODwef9wzZ7h1BsoA4YDDwJ7AuXAh2G5PcJlAMMxG0hDQwPz5s0L6xweqa8f\n0CMyPzt83ANYH24LYCPBMFZvINEbXhtuN2Er8G+gKnzuQAnwf+F2LIw78R62hc/Lwu1Nj2w78V4/\njKy7GegeWd47jCvxPnoC1ZHH9ZE2rE7yuLOtiov3ZNs2BypbXF5SMoSGhoY2d0E1NDRQXNw/8j7S\nq0ekverr66mvr89IXXEnkBXAPpH5QeFrzcsMbqPMDtEEkgnV1dVs2dLActYRfMHnUL5tJQOPOqrN\nI5NqamqA5cAcgh+IJeHz4QTHAywPHxsir8+hqWkpNTU1bNu2iuBHtihSx8pwviqyfrTcJmAd0BSu\nUwM0EvzOweclAAAHv0lEQVSwbg9f37DL9txXUF1dTUVFRSTeRIyrw3UJ678QeJidiWBduGwbwY/4\naoKk0BS+vinSIh6WWx8+J1xnW2S9rZH3sDmc/1cY8+eB/w23vSmctkfW3RLWtyFSx7+BDwgSSaKt\nE4/RNmxI8rizrbZtawzjXtPi8qampVRXV9OW6urq8O9rHapHpL2a/2N98803p19Zul2XTExAMbAY\nGAKUAq8BBzcrcwrwZPh8FPBSkvoy16+LSIyB9Op1ZLv3T48ff1W4m2R/h1Lv1q1yRz3jx1/l5eV9\nvaysOtz/fdgu9Qf7yCs8OmZRXNzDi4t7hPNVvnMsoDRSrr/vHCcYGj5PLC8Jn7c2BhKNt9xPOunk\nSAzRcYjisFxv3zmWYeEuqtLIdsx3jnsUh8vNdx0PSaybWK/cg7GKbpGyiZhLItseED6Prhst2z+y\nnehr5V5SMtiDMZCPRNqw3IuLB+6yPPEY/dvs/LvsrK/53y7Vz1Um6hHpCDqwCytfDuO9k52H8d5m\nZuPCN/VAWOYeYCzBv2oXufurrdTl2Xo/eXcUVnjkTs+ePXccQbRp06Yd5QYPHszcuXNZtWoVJ5xw\nAgDPPPMM5eXBLqXGxkaGDh3KmDFjdns/OgpLR2FJ11Gwh/Fmms4DERFpn44kEF3OXURE0qIEIiIi\naVECERGRtCiBiIhIWpRAREQkLUogIiKSFiUQERFJixKIiIikRQlERETSogQiIiJpUQIREZG0KIGI\niEhalEBERCQtSiAiIpIWJRAREUmLEoiIiKRFCURERNKiBCIiImlRAhERkbQogYiISFqUQEREJC1K\nICIikpbYEoiZ9TGzp8xsoZn91cyqWigzyMyeNbM3zGyumV0VR6wiIrK7OHsg1wPPuPuBwLPAd1oo\nsxX4prsfChwNXGFmB+Uwxoyrr6+PO4SUKM7MUpyZpTjzQ5wJ5HTgwfD5g8AZzQu4+0p3fy18vhGY\nDwzMWYRZUCgfKMWZWYozsxRnfogzgezp7qsgSBTAnskKm1k1cAQwPeuRiYhIm7pls3IzexroH30J\ncODGFop7knoqgMeAq8OeiIiIxMzcW/3dzu6GzeYDte6+yswGANPc/eAWynUDngD+4u53tlFnPG9G\nRKSAubuls15WeyBtmAJcCEwALgD+1Eq5/wbmtZU8IP1GEBGR9ouzB9IX+D0wGFgKfMnd15nZXsAv\n3P1UMxsN/AOYS7CLy4H/dPf/jSVoERHZIbYEIiIiha3gzkQ3s7FmtsDMFpnZda2UucvM3jSz18zs\niFzHGMaQNE4z+6SZrTOzV8OppQMLsh3jRDNbZWZzkpTJh7ZMGmc+tGUYR0onvsbdpqnEGXebmll3\nM5tuZrPCGG9qpVzcbdlmnHG3ZbNYisIYprSyvH3t6e4FMxEkvMXAEKAEeA04qFmZk4Enw+dHAS/l\naZyfBKbE3J6fIDg0ek4ry2NvyxTjjL0twzgGAEeEzyuAhXn6+UwlztjbFOgRPhYDLwE1+daWKcYZ\ne1tGYrkG+E1L8aTTnoXWA6kB3nT3pe7eBDxKcEJi1OnAQwDuPh2oMrP+5FYqcUJwWHNs3P154P0k\nRfKhLVOJE2JuS0j5xNfY2zTFOCH+z+cH4dPuBAf8NN/fHntbhttuK07Ig8+nmQ0CTgF+2UqRdrdn\noSWQgcDyyPw77P7Bb15mRQtlsi2VOAGODruKT5rZIbkJrV3yoS1TlVdtmeTE17xq0zZO0I21TcPd\nLbOAlcDT7j6zWZG8aMsU4oT8+HzeDnyb1s+5a3d7FloC6UxeAfZx9yOAe4A/xhxPIcurtiyUE1/b\niDP2NnX37e5+JDAIOCof/jFoSQpxxt6WZvYZYFXY8zQy1CMqtASyAtgnMj8ofK15mcFtlMm2NuN0\n942Jrq+7/wUoCQ9tzif50JZtyqe2DE98fQx42N1bOrcpL9q0rTjzqU3d/V/ANGBss0V50ZYJrcWZ\nJ205Gvismb0NPAKMMbOHmpVpd3sWWgKZCQw1syFmVgqcTXBCYtQU4CsAZjYKWOfhNbdyqM04o/sW\nzayG4JDq/8ttmMHmaf2/kXxoy4RW48yjtoS2T3zNlzZNGmfcbWpme1h4iwczKwdOBBY0KxZ7W6YS\nZ9xtCeDu/+nu+7j7Rwl+j5519680K9bu9ozzTPR2c/dtZjYeeIog+U109/lmNi5Y7A+4+5/N7BQz\nWwxsAi7KxziBL5jZ14Em4EPgrFzHaWaTgFrgI2a2DLgJKCWP2jKVOMmDtgzjHA2cB8wN94k78J8E\nR+PlTZumEifxt+lewINmVkTwHfpd2HZ59V1PJU7ib8tWdbQ9dSKhiIikpdB2YYmISJ5QAhERkbQo\ngYiISFqUQEREJC1KICIikhYlEBERSYsSiEgOmdmSPLzigEhalEBEcksnXkmnoQQikiVm9riZzQxv\nNHRJ4uVYgxLJoIK6lIlIgbnI3deZWRkw08z+EHdAIpmkBCKSPd8wszPC54OA/dEuLOlElEBEssDM\nPgkcDxzl7pvNbBpQFnNYIhmlMRCR7KgC3g+Tx0HAqPB1jYFIp6EEIpId/0tw46A3gB8CL4SvaxeW\ndBq6nLuIiKRFPRAREUmLEoiIiKRFCURERNKiBCIiImlRAhERkbQogYiISFqUQEREJC1KICIikpb/\nD83g+mCZg3q1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f442dac1e10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the predicted probabilities again\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_prob, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.64722323])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted log-odds for al=2 using the equation\n",
    "logodds = logreg.intercept_ + logreg.coef_[0] * 2\n",
    "logodds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.91022919])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert log-odds to odds\n",
    "odds = np.exp(logodds)\n",
    "odds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.65638445])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert odds to probability\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.65638445])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted probability for al=2 using the predict_proba method\n",
    "logreg.predict_proba(2)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['al'], array([ 4.18040386]))"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the coefficient for al\n",
    "feature_cols, logreg.coef_[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** A 1 unit increase in 'al' is associated with a 4.18 unit increase in the log-odds of 'household'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.99205808391674566"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# increasing al by 1 (so that al=3) increases the log-odds by 4.18\n",
    "logodds = 0.64722323 + 4.1804038614510901\n",
    "odds = np.exp(logodds)\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.99205808])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted probability for al=3 using the predict_proba method\n",
    "logreg.predict_proba(3)[:, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Bottom line:** Positive coefficients increase the log-odds of the response (and thus increase the probability), and negative coefficients decrease the log-odds of the response (and thus decrease the probability)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-7.71358449])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the intercept\n",
    "logreg.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** For an 'al' value of 0, the log-odds of 'household' is -7.71."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.00044652])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert log-odds to probability\n",
    "logodds = logreg.intercept_\n",
    "odds = np.exp(logodds)\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That makes sense from the plot above, because the probability of household=1 should be very low for such a low 'al' value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Logistic regression beta values](images/logistic_betas.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changing the $\\beta_0$ value shifts the curve **horizontally**, whereas changing the $\\beta_1$ value changes the **slope** of the curve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparing Logistic Regression with Other Models\n",
    "\n",
    "Advantages of logistic regression:\n",
    "\n",
    "- Highly interpretable (if you remember how)\n",
    "- Model training and prediction are fast\n",
    "- No tuning is required (excluding regularization)\n",
    "- Features don't need scaling\n",
    "- Can perform well with a small number of observations\n",
    "- Outputs well-calibrated predicted probabilities\n",
    "\n",
    "Disadvantages of logistic regression:\n",
    "\n",
    "- Presumes a linear relationship between the features and the log-odds of the response\n",
    "- Performance is (generally) not competitive with the best supervised learning methods\n",
    "- Can't automatically learn feature interactions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}