{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 04 - Logistic Regression\n",
    "\n",
    "by [Alejandro Correa Bahnsen](http://www.albahnsen.com/) & [Iván Torroledo](http://www.ivantorroledo.com/)\n",
    "\n",
    "version 1.3, May 2018\n",
    "\n",
    "## Part of the class [Applied Deep Learning](https://github.com/albahnsen/AppliedDeepLearningClass)\n",
    "\n",
    "\n",
    "This notebook is licensed under a [Creative Commons Attribution-ShareAlike 3.0 Unported License](http://creativecommons.org/licenses/by-sa/3.0/deed.en_US). Special thanks goes to [Kevin Markham](https://github.com/justmarkham)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Review: Predicting a Continuous Response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type\n",
       "id                                                                        \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1\n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6\n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1\n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1\n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import zipfile\n",
    "with zipfile.ZipFile('../datasets/glass.csv.zip', 'r') as z:\n",
    "    f = z.open('glass.csv')\n",
    "    glass = pd.read_csv(f, sep=',', index_col=0)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** Pretend that we want to predict **ri**, and our only feature is **al**. How could we do it using machine learning?\n",
    "\n",
    "**Answer:** We could frame it as a regression problem, and use a linear regression model with **al** as the only feature and **ri** as the response.\n",
    "\n",
    "**Question:** How would we **visualize** this model?\n",
    "\n",
    "**Answer:** Create a scatter plot with **al** on the x-axis and **ri** on the y-axis, and draw the line of best fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('ggplot')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fcf7e2ffdd8>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAENCAYAAAD34uk0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHoJJREFUeJzt3X+wXOV93/H33rt7dWVJIOQFyxcIAoOhKS5kmpK4aj0KTccEFPxH6RfS1KnTTplh6tZY8ZA6tquWsWeK26AS41+qYTCxh/CtQ2wMtJrGGX5YNDQ2Y+JgjGNLSpGu+HElBPcKXd0f2v6xu9LuuWd/PHv3/Lr6vGYY7u6ePfvdc6/O9zzP9znPU6rVaoiIiIQYyToAEREpHiUPEREJpuQhIiLBlDxERCSYkoeIiART8hARkWBKHiIiEkzJQ0REgil5iIhIsHLWASRIt86LiIQr9bPRSk4eTE5OZh0CANVqlampqazDGEiRY4dix6/Ys1Hk2GF58U9MTPS9rbqtREQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwcppfIiZ3QtsBV5198tjXt8CfAvY23jqIXe/veX1UeB7wAF335p8xCIi0k0qyQO4D7gbuL/LNk91SQwfAV4AzhhyXCIiMoBUuq3c/Ung8CDvNbPzgOuArww1KBERGVhaLY9+vNfMngMmgY+5+/ON5/8bcBuwrtcOzOxm4GYAd6darSYVa5ByuZybWEIVOXYodvyKPRtFjh3Siz8vyeNZ4AJ3nzGza4FvApeYWbNO8v1GXaQrd98J7Gw8rE1NTSUWcIhqtUpeYglV5Nih2PEr9mwUOXZYXvwTExN9b5uL0Vbu/qa7zzR+fgyomFkV2Axcb2b7gD8Crjazr2UXqYiIQE5aHma2EXjF3WtmdhX1pHbI3T8OfLyxzRbq3Vn/PLtIRUQE0huq+wCwBaia2X5gO1ABcPcvATcAt5jZAnAMuMnda2nEJiIi4Uq12oo9R9cmJyezjgEodh9qkWOHYsev2LNR5NhhKDWPUj/b5qLmISIixaLkISIiwZQ8REQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmJKHiIgEK2cdgOTTwek5djw9ydGFfawpw7bNE2xcO5Z1WCKSE2p5SKwdT0/y4tQs+4/M8uLULDt2T2YdkojkiJKHxJo+vtj2+M3IYxE5vSl5SKx1q0a7PhaR05uSh8TatnmCy6rjnLd+nEur42zbPJF1SCKSIyqYS6yNa8e44/2bqFarTE1NZR2OiOSMWh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmIbqihRQc+6x6eOLrFs1qrnHJHVqeYgUUHPuscnpec09JplQ8hApIM09JllT8hApIM09JllLpeZhZvcCW4FX3f3ymNe3AN8C9jaeesjdbzezceBJYFUj1m+4+/Y0YhbJs22bJ9ixe5I3W2oeImlKq2B+H3A3cH+XbZ5y962R544DV7v7jJlVgO+a2f909z9PKE6RQmjOPSaSlVS6rdz9SeDwAO+ruftM42Gl8V9tmLGJiEi4PA3Vfa+ZPQdMAh9z9+cBzGwU+D5wMfB5d38mwxhFRIT8JI9ngQsa3VPXAt8ELgFw90XgSjNbD/yJmV3u7n8VtxMzuxm4ufE+qtVqOtH3UC6XcxNLqCLHDsWOX7Fno8ixQ3rxl2q1dHqBzGwT8EhcwTxm233AL7r7VOT57cBRd/+vfXxkbXIyH2Pfi7wmRpFjh2LHr9izUeTYYXnxT0xMAJT62TYXQ3XNbKOZlRo/X0U9rkNmdnajxYGZrQZ+FfhxdpGKiAikN1T3AWALUDWz/cB26sVv3P1LwA3ALWa2ABwDbnL3mpm9E/hqo+4xUt/cH0kjZhER6Sy1bqsMqNtqCIocOxQ7fsWejSLHDqdZt5WIiBSLkoeIiART8hARkWB5uc9DYmjNBhHJK7U8ckxrNohIXqnlkWO91mxQy0REsqKWR04dnJ7j8LGFtueiazaoZSIiWVHyyKkdT08yu3DqHpzxcmnJmg1aTU5EsqJuq5yKJoYNq8vUanDbrn0nu6nGy+25/7Wj89y2a5+6r0QkcWp55FTcMqPRbipqNS6rjlNp/BbnT6DuKxFJhZJHTm3bPMFl1XEm1lW4tDrOts0TS1ojs4s17nj/Js5eU2l7Xt1XIpI0dVvlVNwyo+tWjcL0fPvjLs+vBBpRJpJPankUSFxrpNvzK4FGlInkk1oeBRLXGun2/EqgEWUi+aSWh+Ra3MABEcmekofk2krukhMpMnVbSa6t5C45kSJT8hBJgUaNyUqjbiuRFGjUmKw0Sh4iKdCoMVlp1G0liVFXzSkr+UZOOT2p5SGJUVfNKRo1JiuNWh6SmEG6atJorWTRItKoMVlp1PKQxAxyg18arRW1iESWTy2PhB2cnuMT33mOw0dnT7t+/22bJ9ixe5I3W67we131p1FYVvFaZPmUPBLWvMoFYHqeHbsnE+++yEuhOq6r5rZd+7oejzQKyypeiyyfuq0SlsVVbp67ZXodjzQKyypeiyyfWh4Jy+IqN8/dMr2ORxqF5aIWr/PSohQBtTwSt23zBJdvXJf4Ve7B6TlufXQPNzzwYyZbTs6Qr24ZXfUPLs8tSjn9dG15mNn73P3Jxs9Xd9rO3f9s2IEV1V++PMNnnjjA/GKNymiJT245ly/feAVTU1ND2X+nq88dT0+y98hc27aVEbhoQ/wJutdVbPP1owv7WFNmaFe5Rb3qz4M8tyjl9NOr2+oLwOWNn78KLMRsUwMuGmZQRfaZJw4wu1ADYHGhxqcfP8Cv/O1NQ9t/pwJ89MQCcPaaCp+NnKibSWHP4VnmT7BkP7GfA6kU+qU7FfolT7omD3e/HMDMRoGzgTPd/XgagRXV/GKt6+Pl6nT1GT2xnHyuxcHpOW59bO/J5Ba3n16f049h9s3ndV9ZiBv6LJKVvmoe7r4IvAi8Pdlwiq8yWur6eLk63Xi3bfMEF60fozLS6K46a1XbyeUvX57hlof3xCaObvvt9LibYfbND3Nfn33qQNu+7njywMD7ykKzy++L17+Lz75/U6ESn6w8IaOtvg48YmZ3Afupd1cBqnm0+uSWc/n04+01j04GuRLudPW5ce0YO6471Xt4cHqOO3ef2ve+14/TrQ0UvYptfs7MAidrHv0aZt/8cltArTdo/r832hvNL72hRrTIoEKSxy2N///HyPOqebR4zzvW8uCNl/a17SA3EPZbcI7uu5vKCEuSVvNzqtVq12J/MwEefmue6bkTnLFqdMkJfjl988vp5w85BiISpu/k4e4XJhnI6SjJ0TNxBfROzj9z1cCfEy2szy7Ux1SMl0tsWF1edt98t37+0KlORoHWZ847Q90+IoPSTYIZSnL0THTf560rM3VskfnFGqMleMe6MRZP1JZ9cu+UpDasLvPF69818H6burW0erXcosfg59aPsao80lfBOQ/F9QNHjvGpXfuCY8hD7LLypZI8zOxeYCvwanMEV+T1LcC3gL2Npx5y99vN7HzgfmAjcALY6e53pRFzGpIcPRO37yROIHGjvE4+n4DWE+NrR9s/N26qk88989pAk1JmMSdZ1O27fjJQDHmIXVa+tFoe9wF3U08EnTzl7lsjzy0Av+Puz5rZOuD7Zva/3f1HCcWZqiRvmEvrZrxmkjrUUvM4a3U5sWGk0W6yVnFTnfR7g2b0av31Y+23NGVxQ96R2e7JsRPdTChpSCV5uPuTZrZpgPcdBA42fp42sxeAc4EVkTxWgrTvGI+eGCsj9Zshl9tyi16tj5fbh1jH3TOTdNfQ+vEK+zmVKPttzelmQklDnmoe7zWz54BJ4GPu/nzri43k8wvAM0kFkFZfsfqku+t2fKInxos2jC+5i34Q0aR0fKHGeLnUsSWVRtfQ9mvezaceeT64W1M3E0oa8pI8ngUucPcZM7sW+CZwSfNFM1sL/DFwq7u/2WknZnYzcDOAu1OtVoOC+MR3nms7IXzumdf48o1XBH6VpcrlclssSX1OEqKxp6Hb8fn01jX8p//1E47MzrN+vML2a95N9czVHffVb/wb1hxom1CyBswu1Li4ujr2d3N0YV/b45kFhn6cyuUy9/zmLwa/r1qFezZlmzCy+LsZliLHDunFn4vk0ZoQ3P0xM/uCmVXdfcrMKtQTx9fd/aEe+9kJ7Gw8rIVORnj46OySx8OY0DB6r8SwPieNFkyv+zx66SfGXvWGV958i3/59e/F72P+KFNTR5cd/7/9pbPZsXuevz40S+uMMp1+N2vKSx8Pa/LLpk6xF6Hluty/mywVOXZYXvwTE/1fdOQieZjZRuAVd6+Z2VXUp005ZGYl4B7gBXe/M+k40uorHtbn5GVUTaeT2ZK5tKbnueOJ/VTKI23bRr/HWGTSnKm3FnntrcWTrw/6PbuddJu1m7aVDun8u+m3ayiJE31efu9yektrqO4DwBagamb7ge1ABcDdvwTcANxiZgvAMeCmRiL5B8AHgR+a2Q8au/s9d38siTjT6ise1ufkZVRNp5PZjqcnl8yl9dKbc22z+X7k0b1LJo88QbvotCqDfs9+Trq9fjfRZLD96vO7JoMkTvR5+b3L6S2t0Va/0eP1u6kP5Y0+/11guDMLdpHWyKFhfU5Wo2r6HdYafT5O3ESNvX7hze8ZelV/+K32oa+H3lp6f8rGtWPc+vcnTu73zt2TbfsNTQZJnOg1mkryIBfdVjKYLEbVxHVFRbuZVjVmEo47UW5cN8ZLb8wteb71vRtWlzk4Ez8X1Xi5xAevrHLbrn091yRpjXnH05Onur6ab5k70fZ6MwnNL9bY8/rx2P32SgbRfY2X2w/OME70g/zei1AnkWJR8sjAsP4hd2vBJHWyiOuKinYzlUr15LF6FGYjjY9arT4EttPU8McXa7GJo3VVxDt3x98oGNeSaMYct/0ZjRN5tDVRiSTD1gTR66o/uq+L1o9xWXV8qAl+kJar6iQybEoeGUjjH3JSnxE3l9VCJHu89MZxXp6Z41hMD83L0/Ocs7ayZJ31Xk605JpO3WHNlkRUtLuq6azV9T//6Heaj+ymNUH0uuqP7mt2scaO6zbFfn6aVCeRYVPyyEBo18cgrYakThad5rJqNX8C7nhiPwsxqygu1ODgANOjL9Y4uRhUp++yJtpkoD654NRbS7e/tHpqbfe479RpVuBeV/15rUfkNS4pLiWPDIR2fQzSakjiZHFweo65hRNURpZenUftOdK5rrGchXlfnJrtuPzl0fmlNYzXZxeXfN45a8ptd6Vv2zzBh7+9p+07DTorcF7v7s5rXFJcSh4ZCO36GKTVkMTJYsfTk+ztkhTSUKN9TY5W6xqV+26TJ8Kp7qqmjWvHOO+Msbbv1iz699sKDB3C201zX0cX9p1cxXG59aqQOomK69IPJY8MpNH1kcSw406TEh4+ttCxAJ6UZg9VW2vhbRWg+0JYYyMwv1jjlod/1nZibBb5m5qPo63AD397z8nCffOEGjcC7SOP7uWu6y4c6KQbTX6tLc80Tuwqrks/OvUASIa2bZ7gsuo4E+sqbX3zWYsmsYs2jPPF69/FXdddeDLetG7KuWjDOHf/+kVLjtPB6TkORwrq4+XSyW3OO3MVe14/zuT0PC9OzXLHE/u5bdc+/uZI+3rms41RAHHF9Ob7muJGoM0u1NixezLoOx2cnuPWR/csaTW1tjybJ/Zm/KGf0Q8V16UfannkUNrTnPerU1dYa7w/fGWGT/3p/o51jerqEd6YPcF8Y4NyqT6SqkcJ5aQSsKpxr8fGtWP85hVVPvPEAV6ZqV/tn72m0nYiX10Z4Za/dzZf+L+v8srMPNEaftsd7y0OH1vg5Zm5jgME/ubIHLc1VvmLLkrV9NeHZrlt176+WwedugVbk3YaJ3YV16UfpVot3e6GFNUmJ4d/VTaIIk+0NkjsL8/MnUwy0S6tbvd4xGkmi4UTtbYhwZdVx7nj/Zu48cEXu+7vHevGeHV6rmMyK5fqI8DiXFYd56ONhPnjLjWUXlqXAK6MlvjklnN5zzvWxt6p/+rRpcOQv/yBi04mn+jcW5dWhzMlfavW39+gXWNJ/M2nVYsp8r9XGMrEiH11IKjlIUPX2hK55eGftd3TMUhtJO49zSvu6LxYUa9Mdy/wj46UYocUNz+j+V0++tjeU3edA+WRpfe3lFr+3/rS/ulTCWFxocanHz/Agzde2nMBKqjXdlpPkMMaCNHPJJGDOjg9xye+89xAy/92Ez1ecZNs9pqxWcX/4VHykET1c19IVPMei26F+NeOznPro3vabh4cxMJijUqJk91orVq7a373fee2nbTbpjBpaO5iVY/W1exCjdt27Vtys+O6sRFqtRMcb0lm55+5qm2b6In94PSp7rOQk2NIUTz0BJzWDarRSTbjPkfF/+SoYC6Jahb/z35b737z8ki9G+au6y7ki9e/iw2rO1/bzJ+AvUc6d0f1axEYGWm/4m/GEXdz4H/4lfMBmD6+wHi5xDlrykumM1k3NtJWyG8O+2314tTsknrFhrdV+IOtpwYfXL5xHb/7vnO7xj9oAT2kdhL6GYneoNpF3Oeo+J8ctTwkUa3rZEQnJmy1arTEH2xtH9oabbU0l4WNqwssx0Kk+XLOmkrHOkJ0GO2m9WXOWl1ue27D2yptV7c/fGWGTz9+YElrZN3YCJvWV5bUFprv7afvetCTY0hRPPQzovs+fGxhydDobjq1dKJddtHWX9x3UPE/OUoekoroCWgUuOCsVcwunOh4Uonr379z9+TQk0dltMRiy4k99ES6/erzu9Yh3vOOtTx446VLit3RJDOIQU+OIbWT0M/YtnmCzz3zGoePzp7sepycnu+7e6zTbMnRLru4wv5yvqeEUfKQVERPQBf3MUoornDbejJYNVqiVCoxfXyB6bkTzC/W2obinndGhdVjFfZMvRV7V3pzpt4PXlnlaz+YGvhE2m+BOYkTWeg+B7kTPvoZzSnxO9VANq4d48s3XsHU1NSSARP9dI9FdXpPP8c9r8Pek5DEzATdaKhuCoo89G9YsQ9j+Ocgn3H5pomT8Q8jhjS+R1MSfzfR1k9zyPOw99GMPWRocTTR9POeJBT13+swfrcaqiu5k8YVYK/PGEYMRb+SHUYBOWQfy+kea13DRXpLe3CAkofIaWQYBeSQfYQk27hEo3sy+pf24AAlD5HTyDDqLkkVoYveqsta8/cys8DJmkeSlDxECmS5d0yr627lav5e0qrZ6CZBkQJJY1ZdkX4oeYgUiO6YlrxQ8hApkGgRVHdMS1ZU8xApEN0xnbwDR47xqV37OPzWPNNzJzhj1ShnrS5r9FeEkoeseCtpWm4Vq5N3+66ftN1sN7tQX2dFM/K2U7eVrHgqMkuII7PxSwiovtROyUNWPBWZJcT68Urs86ovtVO3lQxFnruGNC23hNh+zbv51CPPcyim5iGnKHnIUOR5xTYVmSXExJmrc/O3m2dKHjIUee4aUpFZZPhU85Ch0P0HIqcXtTxkKNQ1lO+6j8iwKXn0oBNCf9Q1lO+6j8iwqduqB90jIP3Kc91HZNiUPHrQCUH6pbqPnE6UPHrQCUH6tW3zBJdVx5lYV+HSqpZPlZUtlZqHmd0LbAVedffLY17fAnwL2Nt46iF3v72f9yZNhWDpl+o+cjpJq2B+H3A3cH+XbZ5y960DvjcxOiGIiCyVSreVuz8JHE77vSIikow8DdV9r5k9B0wCH3P350N3YGY3AzcDuDvVanXIIQ6mXC7nJpZQRY4dih2/Ys9GkWOH9OLPS/J4FrjA3WfM7Frgm8AloTtx953AzsbDWhqLwPcjrQXpk1Dk2KHY8Sv2bBQ5dlhe/BMT/dd0czHayt3fdPeZxs+PARUzK27qFxFZ4XKRPMxso5mVGj9fRT2uQ9lGJSIinaQ1VPcBYAtQNbP9wHagAuDuXwJuAG4xswXgGHCTu9c6vdfd70kjbhERiVeq1WpZx5CU2uRkPqYSKXIfapFjh2LHr9izUeTYYSg1j1I/2+ai20pERIpFyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCabkISIiwZQ8REQkmJKHiIgEU/IQEZFgSh4iIhJMyUNERIIpeYiISDAlDxERCVZO40PM7F5gK/Cqu18e8/oW4FvA3sZTD7n77Y3XrgHuAkaBr7j7f04jZhER6SyV5AHcB9wN3N9lm6fcfWvrE2Y2Cnwe+MfAfuAvzOxhd/9RUoGKiEhvqXRbufuTwOEB3noV8FN33+Puc8AfAR8YanAiIhIsrZZHP95rZs8Bk8DH3P154FzgpZZt9gO/lEVwIiJySl6Sx7PABe4+Y2bXAt8ELgFKMdvWOu3EzG4GbgZwd6rVahKxBiuXy7mJJVSRY4dix6/Ys1Hk2CG9+HORPNz9zZafHzOzL5hZlXpL4/yWTc+j3jLptJ+dwM7Gw9rU1FQS4QarVqvkJZZQRY4dih2/Ys9GkWOH5cU/MTHR97a5SB5mthF4xd1rZnYV9VrMIeAIcImZXQgcAG4C/ll2kYqICKQ3VPcBYAtQNbP9wHagAuDuXwJuAG4xswXgGHCTu9eABTP7MLCL+lDdexu1EBERyVCpVutYQii62uRkxx6uVBW5GVzk2KHY8Sv2bBQ5dhhKt1VcrXkJ3WEuIiLBVnTLI+sAREQK6LRveZTy8p+ZfT/rGE7H2Isev2JX7BnF35eVnDxERCQhSh4iIhJMySMdO3tvkltFjh2KHb9iz0aRY4eU4l/JBXMREUmIWh4iIhIsF9OTrBS9Fq4ysw8B/4X6VCsAd7v7V1INsoM+FuwqUf9u1wJvAR9y92fTjTLechYby5qZnU99nZuNwAlgp7vfFdkml8e+z9i3kMNjb2bjwJPAKurnwW+4+/bINquof7+/S326pBvdfV/KoS7RZ+wfIuFzjVoeQ9KycNWvAT8P/IaZ/XzMpg+6+5WN/3KROBruA67p8vqvUZ/p+BLqMxd/MYWY+nUf3WOH+mJjzeOe+cmrxQLwO+7+t4BfBv5NzN9NXo99P7FDPo/9ceBqd78CuBK4xsx+ObLNvwJed/eLgR3AHSnH2Ek/sUPC5xolj+Ep9MJVfSzY9QHgfnevufufA+vN7J3pRNfdMhYby5y7H2y2Itx9GniB+jo2rXJ57PuMPZcax3Km8bDS+C9aAP4A8NXGz98A/lGjFZipPmNPnLqthqffhav+iZm9D/gJ8FF3fylmmzyK+37nAgezCSdY3GJjuWJmm4BfAJ6JvJT7Y98ldsjpsW/0FnwfuBj4vLt3PO7uvmBmbwBvBzKf+KqP2CHhc41aHsMTd0USvRr4NrDJ3f8O8Kecuqopgn6+X141Fxu7Avgc9cXGcsXM1gJ/DNzaur5NQ66PfY/Yc3vs3X3R3a+kvk7QVWYWrZfl9rj3EXvi5xolj+HpuXCVux9y9+ONh/+deiGuKIIW5soTd3+z2cx398eASmOxsVwwswr1k+/X3f2hmE1ye+x7xZ73Yw/g7keAx1laNzt53M2sDJxJzrpHO8WexrlGyWN4/oLGwlVmNkZ94aqHWzeI9FNfT72PuCgeBn7LzEqN4twb7p6bbpNuzGxjs686sthY5hpx3QO84O53dtgsl8e+n9jzeuzN7GwzW9/4eTXwq8CPI5s9DPyLxs83AH/WWGcoU/3Ensa5RjWPIWn0iS5ZuMrMbge+5+4PA//OzK6nPkrlMPChzAKO6GPBrseoDxX9KfXhor+dTaRLLWOxsTzYDHwQ+KGZ/aDx3O8BPwe5P/b9xJ7XY/9O4KuN2sEI4O7+SOTf6z3AH5rZT6n/e70pu3Db9BN74uca3WEuIiLB1G0lIiLBlDxERCSYkoeIiART8hARkWBKHiIiEkzJQyQjZvYhM/tu1nGIDELJQ0REgil5iIhIMN1hLpIwM/v3wL8GzqE+S+sn3P1Pso1KZHmUPESS9zPgHwIvA/8U+JqZXZxtSCLLo+QhkjB3/x8tDx80s49TXzxMpLCUPEQSZma/BWwDNjWeWgtUgcWsYhJZLhXMRRJkZhdQX0/hw8Db3X098FfELzQkUhhKHiLJWkN99bnXAMzst4Hoqm8ihaPkIZIgd/8R8PvA/wFeAd4D7M40KJEh0HoeIiISTC0PEREJpuQhIiLBlDxERCSYkoeIiART8hARkWBKHiIiEkzJQ0REgil5iIhIMCUPEREJ9v8BspFSpMA6rQ8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf7e2ff780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot using Pandas\n",
    "glass.plot(kind='scatter', x='al', y='ri')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'ri')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAENCAYAAAD34uk0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X2QHPWd3/H37K6QhBASaATL8iDBmYAdxGGHCOPzHYrPcXFEZf0R7mscYsxdHFVRMYcJ8pVtHIRVfro6dByF7MOKTMk4hOMbH0aOQ/l8KVcQTiU628QYCxDGSDbSaqVdCaEHJMHudP7o2dXsbM9M9zx09+x+XlUq7fb0dH+nd/f37d9jF4IgQEREJImerAMQEZHuo+QhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIokpeYiISGJ9aZzEzB4GVgL73f2KiNdXAFuAneVNT7j7uvJru4AjwBgw6u5XpxGziIjUlkryADYDG4BH6uzzjLuvrPHav3D3kbZHJSIiTUklebj7VjNbmsa5KmjqvIhIcoU4O6VV84jjWjN7DhgE1rj79vL2APihmQXAN9x9Y60DmNlqYDWAu/PWW291OuZY+vr6GB0dzTqMpnVz/Io9G4o9G63Gftppp8U/V9Nnaa9ngSXuftTMbgCeBC4tv/Z77j5oZucA/2BmL7n71qiDlBPLeHIJRkby0dJVLBbJSyzN6Ob4FXs2FHs2Wo19YGAg9r65GG3l7ofd/Wj566eAWWZWLH8/WP5/P/BdYHlmgYqICJCT5GFm/WZWKH+9nDCuA2Y2z8zml7fPAz4E/DK7SEVEBNIbqvsYsAIomtluYC0wC8DdHwJuBG4zs1HgOHCTuwdmdi7wXTMbj/W/uvsP0ohZRERqK0zj53kEg4ODWccAdHcbKnR3/Io9G4o9G23q84g12ioXzVYiItJdlDxERCQxJQ8REUlMyUNERBJT8hARkcSUPEREJDElDxERSUzJQ0REElPyEBGRxJQ8REQkMSUPERFJTMlDREQSU/IQEZHElDxERCQxJQ8REUlMyUNERBJT8hARkcSUPEREJDElDxERSUzJQ0REElPyEBGRxJQ8REQkMSUPERFJTMlDREQSU/IQEZHElDxERCQxJQ8REUlMyUNERBJT8hARkcSUPEREJLG+NE5iZg8DK4H97n5FxOsrgC3AzvKmJ9x9XcXrvcBPgT3uvrLzEYuISD2pJA9gM7ABeKTOPs/USQx3AC8CZ7Y5LhERaUIqzVbuvhU42Mx7zewC4F8Bm9oalIiINC2tmkcc15rZc8AgsMbdt5e3/zXw58D8zCITEZFJ8pI8ngWWuPtRM7sBeBK41MzG+0l+Vu4XqcvMVgOrAdydYrHYyZhj6+vry00szejm+BV7NhR7NtKMvRAEQSonMrOlwPejOswj9t0FXA3cBXwMGAXmEPZ5POHu/zbGKYPBwcGm422nYrHIyMhI1mE0rZvjV+zZUOzZaDX2gYEBgEKcfXNR8zCzfmCfuwdmtpywL+aAu38W+Gx5nxWEzVlxEoeIiHRQWkN1HwNWAEUz2w2sBWYBuPtDwI3AbWY2ChwHbnL3dKpEIiKSWGrNVhlQs1WbdHP8ij0bij0baTZbaYa5iIgkpuQhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIokpeYiISGJKHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiuXiSoORTaXgItjzKwWNHKM2bD6tupmdxf9ZhiUgOKHlIpNLwEMH998DwEG+Pb3x1B6U71ymBiIiaraSGLY/C8NDkbeWaiIiIkodECg4dTLRdRGYWJQ+JVFh4dqLtIjKzKHlItFU3Q3XfxuL+cLuIzHjqMJdIPYv7Kd25DrY8St+xI4xqtJWIVFDykJp6FvfDJ+7i7GKRkZGRrMMRkRxRs5WIiCSm5CEiIokpeYiISGLq8xDpQuNLxwSHDobDpzWYQVKm5CHSZSqXjgEIQEvHSOrUbCXSbbR0jOSAkodIl9HSMZIHqTRbmdnDwEpgv7tfEfH6CmALsLO86Ql3X2dmc4CtwOxyrN9x97VpxCySV4WFZ4dNVRHbRdKSVs1jM3B9g32ecferyv/WlbedBD7g7r8LXAVcb2bv7WCcIvmnpWMkB1Kpebj7VjNb2sT7AuBo+dtZ5X9RN10iM0bl0jEabSVZydNoq2vN7DlgEFjj7tsBzKwX+BnwDuBr7r4twxhFcmF86RiRrOQleTwLLHH3o2Z2A/AkcCmAu48BV5nZQuC7ZnaFu/8y6iBmthpYXX4fxWIxnegb6Ovry00szejm+BV7NhR7NtKMvRAE6bQClZutvh/VYR6x7y7gancfqdq+Fjjm7vfFOGUwODjYTKhtV+zyhQW7OX7Fng3Fno1WYx8YGAAoxNk3F0N1zazfzArlr5cTxnXAzBaXaxyY2Vzgg8BL2UUqIiKQ3lDdx4AVQNHMdgNrCTu/cfeHgBuB28xsFDgO3OTugZmdB3yr3O/RE+7u308jZhERqS21ZqsMqNmqTbo5fsWeDcWejRnXbCUiIt0lL6OtJIJWThWRvFLyyKk4K6cquYhIVtRslVcNVk4dTy7Btqdhx/ME254muP+eMKGIiHSYkkdOBfujk0Awnhy0LLeIZEjJI68Ovx69ffcuxu67m+CFn0e+HLzwc9U+RKTjlDzy6syF0dvfOgk7nocjb0S/fuQNNV+JSMcpeeRU4Zzzmn/zNGu+Kg0PUdq0nrH77qa0ab0So0gOaLRVXq26GV7dMbVfo1pPL5TGpmyeLk+V0/O6RfJJNY+c6lncT+HOdRSuuQ4uWwaLzone8axFkZunzVPlNDBAJJeUPHKsZ3E/PZ+4i941X6Jw1xejnx536x3T+qlyel63SD6p2apL1Ht63HR+qpye1y2ST0oeXaTW0+Om9VPlovp+urBmpdUAZLpR8pBcmw41K3X6y3Sk5CG51/U1q3qd/t38uWRGU/KQjlFTTUid/jIdKXlIR6ip5hR1+st0pOQhndFkU00atZXUa0TTpNNfpJKSR4eVhod449sbGNu3d0Y13TTTVJNGbSWLGtF06PQXqabk0UHjBdWJDJpusu5vqNVUw5y5lDatj44rjY7ljDqvu77TX6SKkkcnZVRQ5aK/Iaqp5qwivLaT4OBwZFxpdCyr81qkPbQ8SQdlVlDlYD2o6rW5CtdcBxddAuXEERVXrQ7kdnYsp3GOTtHqwpInqnl0UJqjbErDQwSPbwrv9o8djdwnadIab/o6eOwIpXnzEzd9VTfVjN13d/240uhY7tLO61zUJkUqKHl0wNhLz8PmB+DoYejpgVLp1IstFlRRfRkAwV9+Dl4fqfvewsKzT71//144fAjOPIvCOf1TEkNlYfX2+MYWC6tGyTSNjuWu7bzWREPJmbrJw8z+wN23lr/+QK393P1H7Q6sW4299Dzcf8/kZ2wUCnDBUgoDF7VUUNW6+2TgooaJg8X9BMuuhi/8GZw8cWr7gf0EO3dMTQwtFFY1O+tj3PVHdSy3q/O/+jiFj9+e/6RRpr4ayZtGNY+vA1eUv/4WMBqxTwBc0s6gutrmB6Y+nCkI4M1j9LR6h1irQD9xvPZ75s6jcOXVBEsuhU3ra+9XlRiaLawaNa8kvetvV3NNaXiI4L67J/pcAoBfvUBpzZe6IoFooqHkTd3k4e5XAJhZL7AYWODuJ9MIrGu9eSz29qR31E3dZZ4+j2DPb2Db0w13rTx+04VVgxpL4iGrLdaAxufYsG8QDh2YvMPB4bCf6JOfjx9PVrq0r0amr1h9Hu4+ZmY7gEXAYGdD6nKnz4PjEQnk9HmTvm3mjrrm3IlLLoPfvjq16aqnBw7sjx36pMQQs7Ca0ody9EjksZttXmm1BnSiOvFUe3VHU3GlrWv7amTaStJh/ijwfTN7ANgNp8ox9XlUuPWOqX0evb3h9krN3FHXKNALH/kEwKnRVgC9fVPvtOspFKb0PYwXVn3HjjAaMdqqOgHWPXyTzSv1akB1a25R1zeBrCdZAowODVLa/ODkGNQ5LjmRJHncVv7/3qrt6vOo0Hv5MsbuXBf2fbx5DE6fx4I71nL0vAsn7dfMHXXDu8+K5pex++5OljwuWzalcBxvYjq7WGRkJKJDPm4B3UrzSlTCnD2HYM9vJ3X+x51wOMUll03ZlIdhsaXhIQ498AWCfXsSx5CHxCfTX+zk4e4XN3sSM3sYWAnsH+9HqXp9BbAF2Fne9IS7rzOzC4FHgH6gBGx09weajSMtvZcvg69umvh+TrHI0arCt9k+hbh9BjWbuKKcvZjCLZ+Mu/eEhgV0ubO+lcJrUsLcvxcGfxsmjN07p+5cUXOL9fnPKk7U2ioLXEb2TW3uS3tY7JZHGSsnjiQx5CHxycyQ1jyPzcAGwkRQyzPuvrJq2yhwl7s/a2bzgZ+Z2T+4+wsdijM9ne4AjTp+Tw+cNhtmz4XzLoSg1NKdaaMCunDl1W1pZhlPmKVN6wl2vlx337oTDs8qhrPcTxyf/Az4mM1vlcmy03f3TQ/N1XwQSUkqycPdt5rZ0ibetxfYW/76iJm9CJwPdCx5pLkkOGecGU4gXHAWhcVTJ+q1IpUO1qgCelwLibDWzyBOU1T1hMPZP/gOJxqtaByz+W382Gnc3TdbM9V8EElLnmaYX2tmzxGO5lrj7tsrXywnn3cD2zoVQBZLggNhjWB8GGsbdXol1ylNSocPtZwI6/0MGjZFRUw4XHDnvbwd1V9TIVbBWnnsNO7uV91M765XJjddxUjImg8iaclL8ngWWOLuR83sBuBJ4NLxF83sDODvgE+5++FaBzGz1cBqAHenWCwmCuKNb2+YOrRzeIjZP/gOC+68N9GxKvX19U3E0s5zjA4NcuyxjYwdHKH37CLzPrqavv6BpuOspTL+KYpFeOdXWoqxcp9g/yBBxPUp/PU9FBYsYmzO3EmTIgtz5tJ70e/Q1z8Qeey6sZe9ce55nNjxfM3Xe87p56wvbJg49sFjR04t2VJ5rmNHOLvqXE3/jIpFgi9u4PC3/2bKe+sdc/TW2zlUlXR6zz2fhbfeTl/Cv4dWxLnueaXYY54rlbM0UJkQ3P0pM/u6mRXdfcTMZhEmjkfd/YkGx9kIbCx/G0SOEKpjbN/eyO0n9u1tePdaT7FitFK7zlF9h/42cOLFX1DoQMdosdZoq4pYopqZqmd1vw2c+PlP4OJLwwQwZ27Y+f3rF+HtqOK44hz7h2B/OanMngPnL5mo4QSL+3kbOARQFWdl7DXjvP5GePEXNZuuSmct5lDfaTAyEn6mvbsj9xudN3/SdWr1Z1QsnsvbHwsHMpTKn6/04i/rH7PvNEp3rKVQ8TlLq26eiD8tjX5n8mwmxz4wEP/mMxfJw8z6gX3uHpjZcsKl4g+YWQH4JvCiu/9Vp+NIo8rftnPkpGO0XjNT8PimqUuwH34dnvvH1k568gSFxf2JOuNLw0OTFo8MAF7eTunTX55ofgvWfz5yUuWUvo6oiZdRTUqd+BnFOKYePCVpSCV5mNljwAqgaGa7gbXALAB3fwi4EbjNzEaB48BN5UTyfuBjwPNm9vPy4T7n7k91JNAuWhI8q47RSUu/Q/RkxPHC7JUXOxZHcOhgosENweObps7Af31kYnmSnsX9lO764tT+qNlzCN7/ofDrWh3ri86JrE104mfUyox7zf2QdkprtNVHG7y+gXAob/X2HwOFTsVVLa0lwcduuX3SJEJuSb66a5waTLsLjOq793qC/Xvh+JtNn6uhoT0E/+k2GAtn8lfXJKaolche3TF5jseic+CN1+Gt8hJuJ0/AIw+GNZNaBXTx3IlzTpkvEqGVmmwzNVfN/ZBOyEWzVZ50uspfGh6CRx481fRx/NhE4ZToD7lBDaYjBcaWR2MlDgAOdrjN+I2IgryiJlGpNDwEb0Y/IIuxscZzPMoJoVHBHT2SrnfyUjWt1mSbqbnmpIlTphclj7TF/ENuVGtoWEvqQIGRqLnlyBtTl6ZPQ9VCh6NDgwR/8ZlwWfwop82ONccjOHSQwsdvr19wR13z0lhYmyme25baXzO1Y839kE5Q8khZnD/kuLWGerWkjhQYc+bG3zeLxAFQKk1a6ffA3tdqP++kUAhnncdZA2zwt2FyuOV2Cj/+YWTBXa9Zq3fNl5r8QFMlrR1r7od0gpJHymL9Ibeh1tDuAqM0PBQu+553PT2xV/qlpxfOXBDvuEfeINj2dFjzuHMdvdWrCz++CX79UuRbk17zes+Ob6ofK2FTlzrXJQ4lj7TF+ENuS62h3SPHkvR3ZGl0NP5S7GOjUG+trN6+cJ9Kw0ME997O2O330Hv5ssaDCBJc89LwEMEjG2DH8xAEU54dDzTVj5WkqUud6xKXkkfK4vwht6PW0O6RY13TPn4i4Qivw4dqv3babDge8eTlt07C+s8zdvmVsGdX2L9Ty6JzY4VRNwmN1zrHv456rUGNNHZTlzrXJSYljww0/ENuU62hnSPHaq4rdfoZ4d15+bkamavVMd6MuokogJeea3yMl54j+OqfM9bTG16j0+fBrXdM1FomDeutU7Orl7zbmdjz3Lmu5rR8UfLIoVw+crTWUwzLzSlseTR8VvruXenFNG8+HIt+7G1bBEG4FEqribGydnP8GNx/D2N/+qlETzscr3W2qx+rVkHcjlpv5bPj2/W7G9mc9vJ2xiKW2K9+X67+jqaRQtDOO7V8CQYH8/G49W5eKwdOxR/nD3Fs29Pw8P3hUvPTwcX/BPb85tSkwXbp6Yl/jQoFCl/6BsDUwQA1nlFST+R8lIobgVbOUe/YLU1Q3bQ+HLBQT9V5mo2lm/9e27S2VayJ2b333ntv0yfKuXuPHOngXWkCp59+Om++2cHZ1h02Hn9h3hkU3vM+et73hxTe8z4K886Ysm/PBUvhvSsoHD0cThQcrb/gYd4VLrsC/vhPYdvT7W0SS3Ksy6+k94MfDq/3lf88vLZnnAkXLA1rNbt+FU463fMb+MVPwn0ifjYTp37sG/Dy9skb3zxK4ehhen7/Qy2do9ax+cet8NpOggsvqRtbLaUf/Y/oNcUiPkPhPe9r+DnH94nSzX+vrcY+f/58gC/E2ben6bOI1NBTXrSwcOXVLRykt30BNavcz9R7+TK4cx0sOJuOr5ZTqPqTrHpE8Pi17V3zJQpz5k5deLKyc72GRv0arZyjZt9IeahzcP89YQ02Qml4iNKm9Yx9eQ1jn/kEY1/+NKVN6ykND8VuNqs8f577b6YD9XlI59R70mA9i/th4KLWV99txfwFk5o3ei9fRunyZY2bTsaVZ5VTKMDLv4zXRLW4f9IkxDnnnsfJ62+svdhjE4VjaXgo0ZpbNc/xi59S2rQ+9kjBCTVGbkU2MR3YT7BzR/g7dEvE7P4Gn2EmTY6sNzeoU5Q8pGMmljr/yqfrD2eFsA/g/CUUBi6aGFUWDP526hMXU+pLKbzrqtir5DJr1uTnkVS1q4+99Dw8uC663yRq6ZLLlwGwoEH7ddLCMfGS8nXOwfFjE5MmJ80BiXHDEHkd6w0eGB6i8OMfhrW/8dFpc+bCazsn14qqP0Maq2TnQGXirZ4b1MkEouQhHdWzuJ/Su66KvmNfuAjOHag9UqZqxFnw/g+Fi0omrck0DDLe4oU1C9J3vZvCnLk1BxL0Xr6M0r0Ptr8jOWnhmHBJ+ZrnqBTxLJGJZ8f/v22RNw1JajiVr/dWDT1vef236SKjuTlKHtJ5dYb5NpoZPaV5o/ruE8IO3YMjcPTw5BnhZ55F35LfYXTXr+DECXi76s6/bxb803fDB1fVXK8q1uf4yCcaFkidKMiSHjPOkvJ1z/GLn4ZDjRscd/zZ8SernnoIJE/MFa9Hxda2yZFdLKu+HSUP6bh2Fpz1CoOoO9FF77yi4WNogYmmok5+jnYUZJGfocPrnY3HXWu4bK33J7pe9Wo407CpqZ2y6tvRPI8UdPO4ceju+KdT7K3OoUjz/c1c98rVkDl8CBacNfGc+jSbmrrtd6adc2uSzPNQ8khBt/0yVuvm+KdT7DXv/K+5Lnbto9UZ13HfP52uezcY/7n0HTvCaAujrZIkDzVbiXSJdrRtt9p0NhP6ELrR+M/l7BQTn5KHSJeYSfMWsjQ6NEhp84Onms/OPIvCOek3n+WdkofMCNNigbwZMm8hS6XhIQ498AWCfXtObayYrKjnmpyi5CHT3nR5wNGMmbeQpS2PMlaZOCrpuSaTKHnI9DeNHnCkPofOijNZUUJKHtIWeW4W0gJ5ElczkxVnKiUPaVnem4XU0SyxrbqZ3l2vRDddqX9pEi3JLq2r1yyUB6tuDv/wK6kgkAg9i/tZeO8DFK65LnwQ2KJz4JLLKFxzXcsPtJpuVPOQluW9WSitjuY8N91JfH39A7EnXc5kSh4NqEBorBuahTrd0Zz3pjuRdlOzVR3jBUKw7WnY8XzDJ6HNWGoWyn/TnUibKXnUowIhlp7yImyFa66Dy5bNyPbhvDfdibSbmq3qUIEQ30yff9ANTXci7ZRK8jCzh4GVwH53vyLi9RXAFmBnedMT7r4uzns7SQWCxKalQ2SGSavmsRnYADxSZ59n3H1lk+/tDBUIEpOWDpGZJpXk4e5bzWxp2u9tlQoESWKmN93JzJKnPo9rzew5YBBY4+7bkx7AzFYDqwHcnWKx2HpUxSK88ystHaKvr689sWSkm+NX7NlQ7NlIM/a8JI9ngSXuftTMbgCeBC5NehB33whsLH8b5OVpYN34ZLJK3Ry/Ys+GYs9Gq7GXnyQYSy6G6rr7YXc/Wv76KWCWmXVn6hcRmQFykTzMrN/MCuWvlxPGdSDbqEREpJa0huo+BqwAima2G1gLzAJw94eAG4HbzGwUOA7c5O5Brfe6+zfTiFtERKIVgqDe6vVdLRgcHMw6BqC721Chu+NX7NlQ7NloU59HIc6+uWi2EhGR7qLkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIokpeYiISGJKHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIokpeYiISGJKHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJ9aVxEjN7GFgJ7Hf3KyJeXwFsAXaWNz3h7uvKr10PPAD0Apvc/atpxCwiIrWlkjyAzcAG4JE6+zzj7isrN5hZL/A14F8Cu4GfmNn33P2FTgUqIiKNpdJs5e5bgYNNvHU58Iq7v+rubwF/C6xqa3AiIpJYWjWPOK41s+eAQWCNu28Hzgdeq9hnN3BNFsGJiMgpeUkezwJL3P2omd0APAlcChQi9g1qHcTMVgOrAdydYrHYiVgT6+vry00szejm+BV7NhR7NtKMPRfJw90PV3z9lJl93cyKhDWNCyt2vYCwZlLrOBuBjeVvg5GRkU6Em1ixWCQvsTSjm+NX7NlQ7NloNfaBgYHY++YieZhZP7DP3QMzW07YF3MAOARcamYXA3uAm4B/k12kIiIC6Q3VfQxYARTNbDewFpgF4O4PATcCt5nZKHAcuMndA2DUzD4J/D3hUN2Hy30hIiKSoUIQ1OxC6HbB4GDNFq5UdXM1GLo7fsWeDcWejTY1W0X1NU+hGeYiIpKYkoeIiCSm5CEiIokpeYiISGJKHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIolN6+d5ZB2AiEgXmvHP8yjk5Z+Z/SzrGGZq/IpdsSv2xP9imc7JQ0REOkTJQ0REElPySMfGrANoUTfHr9izodizkVrs07nDXEREOkQ1DxERSawv6wCmEzO7HngA6AU2uftXq16/FfhLYE950wZ335RqkDWY2cPASmC/u18R8XqB8LPdALwJ3Oruz6YbZbQYsa8AtgA7y5uecPd16UVYm5ldCDwC9AMlYKO7P1C1Ty6vfczYV5DDa29mc4CtwGzCcvA77r62ap/ZhJ/vnwEHgI+4+66UQ50iZuy30uGyRjWPNjGzXuBrwB8B7wI+ambvitj1cXe/qvwvF4mjbDNwfZ3X/wi4tPxvNfA3KcQU12bqxw7wTMV1z7zwqjAK3OXu7wTeC/yHiN+bvF77OLFDPq/9SeAD7v67wFXA9Wb23qp9/h3wuru/A7gf+IuUY6wlTuzQ4bJGyaN9lgOvuPur7v4W8LfAqoxjis3dtwIH6+yyCnjE3QN3/7/AQjM7L53o6osRe265+97xWoS7HwFeBM6v2i2X1z5m7LlUvpZHy9/OKv+r7gBeBXyr/PV3gD8s1wIzFTP2jlOzVfucD7xW8f1u4JqI/f61mf0B8DJwp7u/FrFPHkV9vvOBvdmEk9i1ZvYcMAiscfftWQdUzcyWAu8GtlW9lPtrXyd2yOm1L7cW/Ax4B/A1d6953d191MzeABYBI6kGGiFG7NDhskY1j/aJuiOpvhv478BSd78S+J+cuqvpBnE+X149CywpV/MfBJ7MOJ4pzOwM4O+AT7n74aqXc33tG8Se22vv7mPufhVwAbDczKr7y3J73WPE3vGyRsmjfXYDF1Z8fwHhndYEdz/g7ifL3/5nwo64btHw8+WVux8er+a7+1PALDMrZhzWBDObRVj4PuruT0Tskttr3yj2vF97AHc/BPwvpvabTVx3M+sDFpCz5tFasadR1ih5tM9PgEvN7GIzOw24Cfhe5Q5V7dQfJmwj7hbfA24xs0K5c+4Nd89Ns0k9ZtY/3lZtZssJf+8PZBtVqBzXN4EX3f2vauyWy2sfJ/a8XnszW2xmC8tfzwU+CLxUtdv3gI+Xv74R+JG7Z17ziBN7GmWN+jzapNwm+kng7wmH6j7s7tvNbB3wU3f/HvBnZvZhwlEqB4FbMwu4ipk9BqwAima2G1hL2BGHuz8EPEU4VPQVwuGif5JNpFPFiP1G4DYzGwWOAzfloRAo+z3gY8DzZvbz8rbPARdB7q99nNjzeu3PA75V7jvoAdzdv1/19/pN4Ntm9grh3+tN2YU7SZzYO17WaIa5iIgkpmYrERFJTMlLJrSJAAABD0lEQVRDREQSU/IQEZHElDxERCQxJQ8REUlMyUMkI2Z2q5n9OOs4RJqh5CEiIokpeYiISGKaYS7SYWb2GeDfA+cQrtJ6t7t/N9uoRFqj5CHSeb8Gfh8YAv4Y+C9m9o5sQxJpjZKHSIe5+3+r+PZxM/ss4cPDRLqWkodIh5nZLcB/BJaWN50BFIGxrGISaZU6zEU6yMyWED5P4ZPAIndfCPyS6AcNiXQNJQ+RzppH+PS5YQAz+xOg+qlvIl1HyUOkg9z9BWA98H+AfcAy4H9nGpRIG+h5HiIikphqHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iIpLY/wfkOD+N+67eLQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf75d58588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# equivalent scatter plot using Matplotlib\n",
    "plt.scatter(glass.al, glass.ri)\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('ri')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# fit a linear regression model\n",
    "from sklearn.linear_model import LinearRegression\n",
    "linreg = LinearRegression()\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.ri\n",
    "linreg.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  \n",
       "id             \n",
       "22   1.521227  \n",
       "185  1.521103  \n",
       "40   1.520781  \n",
       "39   1.520781  \n",
       "51   1.520682  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# make predictions for all values of X\n",
    "glass['ri_pred'] = linreg.predict(X)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'ri')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAENCAYAAAD34uk0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X2QHPV95/F3T88+6PlpVhqthBAE2RiDgxMKQuUBnS+XIxwV/jjuZ2yfCakjqqIMsTlIKg6JsXXOw1XAhAPHHCcoGR9H/DsbI8eHH3J2HcKJLdtgsBBCgBEGsdonPa+kXe109/3RvbuzMz2z07vz0LP6vKq2mO3pmflOo+1v/76/h3aCIEBERCSJTKsDEBGR9qPkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhi2WZ8iDHmUeBaYNBae3HM85uBHcD+aNOT1tqt0XNvAicADyhYay9rRswiIlJZU5IHsB14EHisyj7PWmuvrfDcv7LWDtc9KhERmZWmJA9r7U5jzMZmfFYRTZ0XEUnOqWWnZrU8anGlMeZFoA+401q7J9oeAN8xxgTAf7fWPlzpDYwxW4AtANZazpw50+iYa5LNZikUCq0OY9baOX7F3hqKvTXmGntnZ2ftnzXrT6mv54FzrbUjxphrgKeATdFzv26t7TPGrAb+yRjzirV2Z9ybRIllIrkEw8PpqHTlcjnSEststHP8ir01FHtrzDX23t7emvdNxWgra+1xa+1I9PhpoMMYk4t+74v+Owh8Dbi8ZYGKiAiQkuRhjMkbY5zo8eWEcR0yxiwyxiyJti8Cfgd4qXWRiogING+o7hPAZiBnjDkA3A10AFhrHwKuB24xxhSA08AN1trAGLMG+JoxZiLW/2Wt/VYzYhYRkcqceXw/j6Cvr6/VMQDtXUOF9o5fsbeGYm+NOvV51DTaKhVlKxERaS9KHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIokpeYiISGJKHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIokpeYiISGJKHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSWLYZH2KMeRS4Fhi01l4c8/xmYAewP9r0pLV2a9HzLvAT4B1r7bWNj1hERKppSvIAtgMPAo9V2efZKonh48BeYGmd4xIRkVloStnKWrsTODyb1xpj1gP/DthW16BERGTWmtXyqMWVxpgXgT7gTmvtnmj73wF/AixpWWQiIjJNWpLH88C51toRY8w1wFPAJmPMRD/Jc1G/SFXGmC3AFgBrLblcrpEx1yybzaYmltlo5/gVe2so9tZoZuxOEARN+SBjzEbgG3Ed5jH7vglcBtwBfBQoAN2EfR5PWmv/Yw0fGfT19c063nrK5XIMDw+3OoxZa+f4FXtrKPbWmGvsvb29AE4t+6ai5WGMyQMD1trAGHM5YV/MIWvtJ4FPRvtsJixn1ZI4RESkgZo1VPcJYDOQM8YcAO4GOgCstQ8B1wO3GGMKwGngBmttc5pEIiKSWNPKVi2gslWdtHP8ir01FHtrNLNspRnmIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIokpeYiISGJKHiIikpiSh4iIJKbkISIiiSl5iIhIYkoeIiKSmJKHiIgkpuQhIiKJpeJOgpJO/lA/7HicwydP4C9aAtd9hExPvtVhiUgKKHlILH+on+C+T8FQP+MTG9/Yh3/7ViUQEVHZSirY8TgM9U/fFrVERESUPCRWcPRwou0icnZR8pBYzvKVibaLyNlFyUPiXfcRKO3b6MmH20XkrKcOc4mV6cnj374VdjxO9uQJChptJSJFlDykokxPHm6+g5W5HMPDw60OR0RSRGUrERFJTMlDREQSU/IQEZHE1Och0oYmlo4Jjh4Oh09rMIM0mZKHSJspXjoGIAAtHSNNp7KVSLvR0jGSAkoeIm1GS8dIGjSlbGWMeRS4Fhi01l4c8/xmYAewP9r0pLV2qzGmG9gJdEWxfsVae3czYhZJK2f5yrBUFbNdpFma1fLYDlw9wz7PWmsvjX62RtvGgA9Ya38ZuBS42hjzaw2MUyT9tHSMpEBTWh7W2p3GmI2zeF0AjES/dkQ/cRddImeN4qVjNNpKWiVNo62uNMa8CPQBd1pr9wAYY1zgOeAC4PPW2l0tjFEkFSaWjhFplbQkj+eBc621I8aYa4CngE0A1loPuNQYsxz4mjHmYmvtS3FvYozZAmyJXkcul2tO9DPIZrOpiWU22jl+xd4air01mhm7EwTNqQJFZatvxHWYx+z7JnCZtXa4ZPvdwElr7T01fGTQ19c3m1DrLtfmCwu2c/yKvTUUe2vMNfbe3l4Ap5Z9UzFU1xiTN8Y40ePLCeM6ZIzpiVocGGMWAL8NvNK6SEVEBJo3VPcJYDOQM8YcAO4m7PzGWvsQcD1wizGmAJwGbrDWBsaYtcAXo36PTLi7/UYzYhYRkcqaVrZqAZWt6qSd41fsraHYW+OsK1uJiEh7SctoK4mhlVNFJK2UPFKqlpVTlVxEpFVUtkqrGVZOnUguwa5nYN9ugl3PENz3qTChiIg0mJJHSgWD8UkgmEgOWpZbRFpIySOtjh+J337gTbx77iJ4+YXYp4OXX1DrQ0QaTskjrZYuj99+Zgz27YYTx+KfP3FM5SsRaTglj5RyVq+d/YvnWfnKH+rH33Yv3j134W+7V4lRJAU02iqtrvsIvLGvvF+jVMYF3yvbPF/uKqf7dYukk1oeKZXpyePcvhXniqvg3ZfAqtXxO65YFbt53txVTgMDRFJJySPFMj15MjffgXvnX+Lc8dn4u8fd9PF5fVc53a9bJJ1UtmoT1e4eN5/vKqf7dYukk5JHG6l097h5fVe5uL6fNmxZaTUAmW+UPCTV5kPLSp3+Mh8peUjqtX3Lqlqnfzt/LzmrKXlIw6hUE1Knv8xHSh7SECrVTFGnv8xHSh7SGLMs1TSjtdL0FtE86fQXKabk0WD+UD/HvvQg3sDBs6p0M5tSTTNaK61oEc2HTn+RUkoeDTRxohptQemm1f0NlUo1dC/A33ZvfFzN6FhuUed123f6i5RQ8mikFp2oUtHfEFeqWZGDt/cTHB6KjasZHcvqvBapDy1P0kAtO1GlYD2o0rW5nCuugg3nQ5Q44uKq1IFcz47lZnxGo2h1YUkTtTwaqJmjbPyhfoIvbwuv9k+OxO6TNGlNlL4OnzyBv2hJ4tJXaanGu+eu6nE1o2O5TTuvU9GaFCmi5NEA3iu7Yfv9MHIcMhnw/akn53iiiuvLAAj+9s/gyHDV1zrLV069fvAgHD8KS1fgrM6XJYbik9X4xMY5nqxmSqbN6Fhu285rTTSUlKmaPIwxv2Wt3Rk9/kCl/ay136t3YO3Ke2U33Pep6ffYcBxYvxGnd8OcTlSVrj7p3TBj4qAnT3DJZfCZP4Kx0anthwYJ9u8rTwxzOFlV7Kyv4ao/rmO5Xp3/pe/j/P5t6U8aEfXVSNrM1PL4e+Di6PEXgULMPgFwfj2Damvb7y+/OVMQwKmTZOZ6hVjphD56uvJrFizCed9lBOdugm33Vt6vJDHM9mQ1U3kl6VV/vco1/lA/wT13Tfa5BACvvYx/51+2RQLRRENJm6rJw1p7MYAxxgV6gGXW2rFmBNa2Tp2seXvSK+pZXWUuXETwzi9g1zMz7lr8/rM+Wc3QYkk8ZHWOLaCJOTYM9MHRQ9N3ODwU9hPd+ue1x9MqbdpXI/NXTX0e1lrPGLMPWAX0NTakNrdwEZyOSSALF037dTZX1BXnTpz/bnjrjfLSVSYDhwZrDn1aYqjxZFXWhzJyIva9Z1temWsLaLQ08ZR6Y9+s4mq2tu2rkXkrSYf548A3jDH3Awdg6jw2n/o8nNOnWXLvvXj5PN6aNXj5PH4+j7d6NXR1zfwGN328vM/DdcPtxWZzRV3hhO588GaAqdFWAG62/Eq7Gscp63uYOFllT56gEDPaqjQBVn37WZZXqrWAqrbc4o5vAq2eZAlQ6O/D3/7A9BjUOS4pkSR53BL999Ml2+dVn0dmaIhFjzyCc+ZM2XPeypX4a9bgrV2Lt2ZN+DhKMn60jU0X4d2+Nez7OHUSFi5i2cfvZmTtOdPeazZX1DNefRaVX7x77kqWPN59SdnJcaLEtDKXY3g4pkO+1hP0XMorcQmzq5vgnbemdf7XOuGwzPnvLtuUhmGx/lA/R+//DMHAO4ljSEPik/mv5uRhrT1vth9ijHkUuBYYnOhHKXl+M7AD2B9tetJau9UYcw7wGJAHfOBha+39s42jFt6GDRx84w2cI0dw+/txBwbIDAzgHjyIOzCA299PZmCAjj17yAwO4gTTr4sD18VfvTpMKvk8/pI1dD39HbwlS8IWTJRsnGUrZtWnUGufQcUSV5yVPTg33lrr3pNmPEFHnfVzOXlNS5iDB6HvrTBhHNhfvnNRy62m778iN9lqKz7hMjxQXu5r9rDYHY/jRYkjSQxpSHxydmjWPI/twIOEiaCSZ62115ZsKwB3WGufN8YsAZ4zxvyTtfblBsUZchyClSsprFxJ4aKLKu9XKJAZGppKMv390x5n33gD9wc/IHP0KCtKXup3d+N1ZMKfrg687k68VSvxu3sIdu0KS2Zr1sCCBbP7DnFX7JkMdHZB1wJYew4E/pyuTGc6QTvvu6wuZZaJhOlvu5dg/6tV96064XBFLpzlPnp6+j3gayy/FSfLRl/dz3poruaDSJM0JXlYa3caYzbO4nUHgYPR4xPGmL3AOqBhySPRSSGbxV+7Fn/t2qmJdDFyCxdy5OWXJ1stmZ+/hvsvO8kMD+EeO0HnqXGyQ8dx9g/ATz45PZ7ly6dKYzFlMm/NGvyeHshO/1/ZlA7WuBP0hDmUqir9P6ilFFU64bDrW19hdKYVjWssv028dzOu7mc72k3zQaRZ0jTD/EpjzIuEo7nutNbuKX4ySj7vB3Y1KoCGnRQWLsTbuBFv48bwM3b9H1gBrOgBesJO7098BrdzQVgaGxggU1ImcwcG6Ni3j8zQEI43fR5JkMng9/SUJ5l8Hv+CX51MMsGK0vbP3JSVlI4fhWUrcHrKZ6zXqtr/gxlLUTETDpfd/mnG4/pritR0Yi1+72Zc3V/3Edw3X59euqohIWs+iDRLWpLH88C51toRY8w1wFPApoknjTGLga8Cn7DWHq/0JsaYLcAWAGstuVwuURDHvvRg+dDOoX66vvUVlt3+6UTvVSybzU7GUvEzvv3V8DMuuKDi+3iA53kwOIhz8CD+yy9x5h+/gjMwiDvu0bloOdmDB3Geew7nUHlnedDVBWvXEqxbR7B2bfi4txd6ewl6e8Ntvb2waPqw4uL4y+Ry8J6/rhhzob+Pk088jHd4GHdljkUf2kI231txn2CwjyDm+Dh/9ymcZavwuhdMmxTpdC/A3fBLZPO9se9dNfbIsTVrGd23u+LzmdV5Vnzmwcn3PnzyRGxLM3vyBCtLPquW7x8rlyP47IMc/9IXyl5b7T0LN93G0ZKk465Zx/KbbiOb8O9hLmo57mml2Gv8rKZ8ygyKE4K19mljzN8bY3LW2mFjTAdh4njcWvvkDO/zMPBw9GsQO0KoCm/gYOz20YGDM169VpMrGq1Ul8/o6MBf0Emw8yk4MwwrMkAGehbj3P+58Ip/dBR3cDC2L8bt78d9/vnw8alTZW/vL10atmCilox73nmcWrZsetls9Wro6Aj3r1BmKp3VPQ6MvvBjOG9TmAC6F4Sd3z/fC+PVCn/gD/bDYJRUurph3bmTLZygJ884cBSg5BgWH/uKcV59Pez9WcXSlb+ih6PZThgeDr/TwQOx+xUWLZk2Kq20FTUOjO79GU6NLdlcbg3jHw0HMvjR9/P3vlT9PbOd+B+/G6foe/rXfWQy/mbJVRqh1wbO5th7e2u4sImkInkYY/LAgLU2MMZcTrhU/CFjjAM8Auy11n6u0XE0o8lft8+YqXTS3Y23YQPehg2V3yMIcEZGwtJYlFwmy2RRkun8wQ/IPPUUywvTV6YJHAc/l8NbtRJv5Aie4+N1Rx3/O5/F/8PbCX7wHYJDg+EckgnHj8CLP0r2XUuNjeL05BN1xvtD/dMWjwwAXt2D/8d/NVl+C+7989hJlWV9HXETL+NKSo0ob9XwnrrxlDRDU5KHMeYJYDOQM8YcAO4GOgCstQ8B1wO3GGMKwGnghiiR/AbwUWC3MeaF6O3+zFr7dEMCbaMlwevSMeo4BEuWUFiyBDZtqrhbbuVKDr/6Kpn+fjL79pL55tdw3/oF7ukzuMcP4Y6cpHN0HPfMRIL5BXz3w2E8jhMmla6OqeQy8XhilFl3B0HWrT3u6HsmGdwQfHlb+Qz8I8OTy5NkevL4d3y2fNRVVzfBb/xO+LhSx/qq1bGtiUZ0Xs9lxr3mfkg9NWu01YdmeP5BwqG8pdu/Dzjlr2iMZi0J7t1427RJhNyYfHXXWlowdTthZDL4uRyFoEDw2DfBGYFzV5Xv5/u4YwXc0TO4y3pw334L9/QY7tg47ug4HSdG6R4+Tqbgl7/UzVRNLl5X+IMb3b+s/x2Cv7gFosEDpS2JMq/vjf9ub+ybPsdj1Wo4dgTOREu4jY3CYw+ELZNKJ+jcmsnPLJsvEmMuLdnZtFw190MaIRVlqzRpdJPfH+qHxx6YKn2cPjl5ckr0hzxDC6YhJ4wdj1df+j2TwVvQibegE5YtgGy+fIVhwCl4uKPjUVI5M5lcwm1n6Doygjs2juOXnya9zmzl5HLsFN4j/w3+5LPhnJaIP9QPp+JvkIXnzTzHI0oIM524Y+eLZNzpx2CuLdnZtFw190MaQMmj2Wr8Q56p1TBjK6kBJ4xE5ZYTx2ITB0CQdSksdiks7q7yYQGZca88uRT93nn8FJmxwvSm6T/vJfjC45NDlzOrV7PsjZfxHG96opkolXV21TTHIzh6GOf3b6t+4o475r4XtmZya+rSkp1N61hzP6QRlDyarJY/5FpbDdVaSQ05YXQnmO1eIXHUzHHwO7P4ndmqEzDxgzCZTCSYIIP7K7+J+9wPcYcP4vx8H4tOjZEplMfjuxm8xQvxXMpbMdFjv6uDwM2Ey6LseBxuvA3n+9+JPXFXK2u5d/7l3I5HkaStY839kEZQ8miymv6Q69BqqPcJwx/qD5d9T5uMM1UqA1iyDAoD0LsAetcRLkgAjudPtloyo+Nkow5+t2sxmf5+Oo+dwh04SiauVNbhhgnluz/B+5LF3/xv8c9/VziUuW+A8cEhvO9+DX7+SmyISY95tXvHz6ofK2GpS53rUgslj2ar4Q+5Lq2Geo8cm6m/Iy0KhdgyVOBmKCzqprCopFS2dDmctyzaKcApeGRHwwTjjnu4p0Yn+2Lc0XE6DgzgPvIITkmOCQhbL353B4XuDvyuqBWTy+H1nEewdy9ePk+wfPn0octF/KF+gscehH27IQjK7h0PzKofK0mpS53rUisljyar5Q+5Hq2Geo8ca5v6+Gj5pMeqjh+deuw4BB1ZxjuysGQBLKhwY68gIDNWwM1vxH1rP+6RY5PJxR0dJ3vqDO7hEdxxD17tg3+5beql3d1Ta5IVLSNTWNiN992n8MZP4XV3hqWyCROtzonHxWpskdZc6lLnutRIyaMFZvxDrlOroZ4jxyquK7VwMXiFyftqtFww40LstauUiBwHv7sD/+g7jC/thKU98ft5ftgX43aFieXECG4hwF27keypUbIvvkjXt75JZrxQ9lI/606fHzN0JuyfOXZkajhzVwdknLom9jR3rqucli5KHimUyluOVrqLYVROYcfj4b3SD7zZvJgWLYGT8be9rYsgCJdCmW1idDN4C7vwALo6YFm0MGXmJHzsE+HV/GAOp+BPa7lMDmGOHncdOoF7cA+OP31+TAD4nVm8n76Nv/tA2KJZu3b6MjL5PP7KlWWlskon4nq0eovvHV+vf7ux5bRX9+DFLLFf+rpU/R3NI05Qzyu1dAn6+tJxu/V2XisHpuKv5Q/R2/UMPHof+OUTAdvSee+Cd34xNWmwXjKZ2o+R4+D8ly+QOXKEzH1bw6VjJpJLkMHFxT0+gnvqNO7J02UvDzo78VavnlyrrLB0Cd6rL+AVRqdaMevXw5/8Vbh/6VyVCvdBiRM71yW6yJjLSdvfdi/Brmeq71TyObONpZ3/Xuu0tlVNE7OVPJqgnf8xQvL4J5PMz34S32fQRpwrrgqXJym9L30zXfg+3Ds+C5TMYO9eAG/vn1x4EoCVq8l++GNkx73pa5VFS/xnBgZw336LzFj5bZb9rk689eeE65WNjuB1unjLl+GNncQrnJ4so7Gmt+IJuOJJfskynIsunfWVv3fPXeFAghk4V1w1ueZZpViK94nTzn+vzUweKltJ3U27899MV4sV38Rt3cl6QtTP5Pbkw/vSb7s3XLqk9hv8JudkIChqkZTcIri4H8vfdi9BceIAODyI9+PvEdx8R8X5Md49d+HseaG8RLZwKdne83EHBsgeGcEdGMA5U55kvM7deN/+Ef5Fl4T3jCkqk43vew1vbBy/Mzu9VHbiWPhvocrIrcnEOHFvmKUrcFaH/w9qva1ycd9Mmvtv5gMlD2mcancarKYnD70b5r767lwsWTbt6tq98BL8Cy+pPRlGs8pxHHj1pdpKVD35aZMQu9esZezq6ysv9jiLk6M/1A/DA7Gz/MuuyIOA4LN34r70YnmiGfdw975Ex89eJHPoME5JBSNwiiZeFi8nc+AQ/uf+iuAPbg2HLi9ZEg5AiCsxHRok2L8v/Dd0Y8zs/hjFfTNn0+TIanODGkXJQxpmcqnzv/7jcLmSqjtnwnt09G6YHFUW9L1Vfg/2JvWlOBddWvMquXR0TL8fSUld3XtlNzywNb7fJG7pkgsvAWDZDCWIpCfHxEvKOw7B2l7G33qN8aUVgujJ49z6F2Rxcfv7cV57BffL23GHD00uJ9NxcpTuQyemZvn/7E147KthTAsWhH0xwTiFsZNTs/y7O/Am5sr09+F8/ztQNIgktmRX+h2asUp2ChQn3tK5QY1MIEoe0lCZnjz+RZfGX7EvXxXWzyuNlCkZcRb8xu+Ei0ombcnMGGRtixdWLJ1c9H6c7gUVBxK4F16C/+kH6t+RnPTkmHBJ+YqfUWyoH562+Dffgd/bC7/yK/gf+EB47/if7pp20eAUvLD1cv5FdFy2efpNyn7yQ7qOncQdOBq/IOYzL+P/wzeL5scso7DhUjxvLwU8/N5e/A/fnGz9t/miRXNzlDyk8aoM851pZnTpP36/9OoTwvr44WEYOR7OOZmwdAXZc3+JwpuvwegojJdc+Wc74L3vh9++ruJ6VTV9jw/ePOMJqREnsqTvWcuS8lU/o8IAiNL3nbh3/FjJXQ+DrEth7Tq8j/8phdILhYn+sYkFMYtKZJmxcbLLenBX5nEHBuh45RUyg4PlQ5fv24a/evXUMOXJSZjr8TddFg5lznYRBEHFWf7tqFV9O0oe0nD1PHFWm/gYN5R41XsunvE2tMBkqaiR36MekzZjv0OD1zubaQBEpdcnOl5FiXlyQcwl0cVB3IWG55EZGpoaQRaNJpsYXZZ96y3cXbvIHD1a9lHTZvlHicbL58PH+TxceCFOVxfBggQLgbZQq/p2NFS3Cdp56B+0d/zzKfa5zqFo5utnc9zLRlstWzF5n/pZt9BGR8OkMjAwvUxWfNvl/n4yp8vnx/jLlk0llijZTE7CnNje0xP2ebVQPefWaJ5HSMmjTto5/vkU+2znLRSb64zrWl/fVsc9CHBOnJhMJMtPneLUa6+FLZoo4WQGBnAHB3EK05eSCRwnvHdMyVplky2aNWvw167FX7GioaWyif8v2ZMnKMxhtJXmeYjMQ/Wobc+1dNboO222hOMQLF1KYelSeNe78HM5RuISn++TOXRoMslMm4TZ34/b10fHT3+Ke+hQ2UuDzs6pBFNSJitOMsGiRbP6ChP/X1Y2MWkreYi0ibNp3kIrFfr78Lc/EDtZkZ6oVHVJlT6ysTHcoSEyE/0wxSWz/n6yr7xC1zPPkBkpvzWyv3hxfJmsaK0yb/Vq6Oxs4BGojZKHnBXmxQJ5Z8m8hVbyh/o5ev9nCAbemdpYNFmxprkTXV1469fjrV9f9S6YzsjItH6Y0iTT+aMfhbP8x8vfxVu1qryzf80anHe9C973PuiucovnOlHykHlvvtzg6KyZt9BKOx7HK04cxeo8dyJYvBjvggvwLrig8k6+Hy6IWdzZX7JWWcfu3WSGhydn+Tt79xIoeYjUwTy6wdG87HNIkZn6j5q+LlYmg79qFf6qVRTe+97K+42PkxkcZNXYWLjkSxMoeUhdpLkspAXypFYzLcCY2v6ljg78desIcjlQh7m0i7SXhdTRLDW77iO4b74eX7pS/9I0mZl3EZlBtbJQGkSjZKbRiUBiZHryLP/0/ThXXBXeCGzVajj/3ThXXDXnG1rNN2p5yJylvSzUrI7mNJfupHbZfG/Nky7PZkoeM9AJYWbtUBZqdEdz2kt3IvWmslUVEyeEYNczsG83wa5nCO77VJhQZIrKQukv3YnUmZJHNToh1CQTLcLmXHEVvPuSs7I+nPbSnUi9qWxVhU4ItTvb5x+0Q+lOpJ6akjyMMY8C1wKD1tqLY57fDOwA9kebnrTWbq3ltY2kE4LUTEuHyFmmWS2P7cCDwGNV9nnWWnvtLF/bGDohSI20dIicbZqSPKy1O40xG5v92rnSCUGSONtLd3J2SVOfx5XGmBeBPuBOa+2epG9gjNkCbAGw1pLL5eYeVS4H7/nrOb1FNputTywt0s7xK/bWUOyt0czY05I8ngfOtdaOGGOuAZ4CNiV9E2vtw8DD0a9BWu5k1lZ3VYvRzvEr9tZQ7K0x19ijOwnWJBVDda21x621I9Hjp4EOY0x7pn4RkbNAKpKHMSZvjHGix5cTxlV+L0cREUmFZg3VfQLYDOSMMQeAu4EOAGvtQ8D1wC3GmAJwGrjBWhtUeq219pFmxC0iIvGcIKi2en1bC/r6+lodA9DeNVRo7/jE7LowAAAHoUlEQVQVe2so9taoU5+HU8u+qShbiYhIe1HyEBGRxJQ8REQkMSUPERFJTMlDREQSU/IQEZHElDxERCQxJQ8REUlMyUNERBJT8hARkcSUPEREJDElDxERSUzJQ0REElPyEBGRxJQ8REQkMSUPERFJTMlDREQSU/IQEZHElDxERCQxJQ8REUlMyUNERBJT8hARkcSUPEREJDElDxERSUzJQ0REElPyEBGRxJQ8REQkMSUPERFJTMlDREQSU/IQEZHEss34EGPMo8C1wKC19uKY5zcDO4D90aYnrbVbo+euBu4HXGCbtfZvmhGziIhU1pTkAWwHHgQeq7LPs9baa4s3GGNc4PPAvwEOAD82xnzdWvtyowIVEZGZNaVsZa3dCRyexUsvB1631r5hrT0D/ANwXV2DExGRxJrV8qjFlcaYF4E+4E5r7R5gHfB20T4HgCtaEZyIiExJS/J4HjjXWjtijLkGeArYBDgx+waV3sQYswXYAmCtJZfLNSLWxLLZbGpimY12jl+xt4Zib41mxp6K5GGtPV70+GljzN8bY3KELY1zinZdT9gyqfQ+DwMPR78Gw8PDjQg3sVwuR1pimY12jl+xt4Zib425xt7b21vzvqlIHsaYPDBgrQ2MMZcT9sUcAo4Cm4wx5wHvADcAH25dpCIiAs0bqvsEsBnIGWMOAHcDHQDW2oeA64FbjDEF4DRwg7U2AArGmFuBbxMO1X006gsREZEWcoKgYhdCuwv6+ipWuJqqnZvB0N7xK/bWUOytUaeyVVxfcxnNMBcRkcSUPEREJDElDxERSUzJQ0REElPyEBGRxJQ8REQkMSUPERFJTMlDREQSU/IQEZHElDxERCQxJQ8REUlMyUNERBJT8hARkcSUPEREJDElDxERSWxe38+j1QGIiLShs/5+Hk5afowxz7U6hrM1fsWu2BV74p+azOfkISIiDaLkISIiiSl5NMfDrQ5gjto5fsXeGoq9NZoW+3zuMBcRkQZRy0NERBLLtjqA+cQYczVwP+AC26y1f1Py/E3A3wLvRJsetNZua2qQFRhjHgWuBQattRfHPO8QfrdrgFPATdba55sbZbwaYt8M7AD2R5uetNZubV6ElRljzgEeA/KADzxsrb2/ZJ9UHvsaY99MCo+9MaYb2Al0EZ4Hv2Ktvbtkny7C7/erwCHgg9baN5scapkaY7+JBp9r1PKoE2OMC3we+F3gIuBDxpiLYnb9srX20ugnFYkjsh24usrzvwtsin62AF9oQky12k712AGeLTruLT95FSkAd1hr3wP8GvCxmH83aT32tcQO6Tz2Y8AHrLW/DFwKXG2M+bWSff4TcMRaewFwH/BfmxxjJbXEDg0+1yh51M/lwOvW2jestWeAfwCua3FMNbPW7gQOV9nlOuAxa21grf0hsNwYs7Y50VVXQ+ypZa09ONGKsNaeAPYC60p2S+WxrzH2VIqO5Uj0a0f0U9oBfB3wxejxV4B/HbUCW6rG2BtOZav6WQe8XfT7AeCKmP3+vTHmt4BXgduttW/H7JNGcd9vHXCwNeEkdqUx5kWgD7jTWrun1QGVMsZsBN4P7Cp5KvXHvkrskNJjH1ULngMuAD5vra143K21BWPMMWAVMNzUQGPUEDs0+Fyjlkf9xF2RlF4N/COw0Vr7PuD/MnVV0w5q+X5p9TxwbtTMfwB4qsXxlDHGLAa+CnzCWnu85OlUH/sZYk/tsbfWetbaS4H1wOXGmNL+stQe9xpib/i5Rsmjfg4A5xT9vp7wSmuStfaQtXYs+vV/EHbEtYsZv19aWWuPTzTzrbVPAx3GmFyLw5pkjOkgPPk+bq19MmaX1B77mWJP+7EHsNYeBf4f5f1mk8fdGJMFlpGy8mil2JtxrlHyqJ8fA5uMMecZYzqBG4CvF+9QUqf+PcIacbv4OnCjMcaJOueOWWtTUzapxhiTn6hVG2MuJ/x3f6i1UYWiuB4B9lprP1dht1Qe+1piT+uxN8b0GGOWR48XAL8NvFKy29eB348eXw98z1rb8pZHLbE341yjPo86iWqitwLfJhyq+6i1do8xZivwE2vt14E/Msb8HuEolcPATS0LuIQx5glgM5AzxhwA7ibsiMNa+xDwNOFQ0dcJh4v+QWsiLVdD7NcDtxhjCsBp4IY0nAQivw58FNhtjHkh2vZnwAZI/bGvJfa0Hvu1wBejvoMMYK213yj5e30E+JIx5nXCv9cbWhfuNLXE3vBzjWaYi4hIYipbiYhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iLWKMuckY8/1WxyEyG0oeIiKSmJKHiIgkphnmIg1mjPlT4A+B1YSrtN5lrf1aa6MSmRslD5HG+znwm0A/8B+A/2mMuaC1IYnMjZKHSINZa/930a9fNsZ8kvDmYSJtS8lDpMGMMTcC/xnYGG1aDOQAr1UxicyVOsxFGsgYcy7h/RRuBVZZa5cDLxF/oyGRtqHkIdJYiwjvPjcEYIz5A6D0rm8ibUfJQ6SBrLUvA/cCPwAGgEuAf25pUCJ1oPt5iIhIYmp5iIhIYkoeIiKSmJKHiIgkpuQhIiKJKXmIiEhiSh4iIpKYkoeIiCSm5CEiIokpeYiISGL/H9dBp3A/ZjzrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf6b645d68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# put the plots together\n",
    "plt.scatter(glass.al, glass.ri)\n",
    "plt.plot(glass.al, glass.ri_pred, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('ri')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Refresher: interpreting linear regression coefficients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear regression equation: $y = \\beta_0 + \\beta_1x$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.51699012])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=2 using the equation\n",
    "linreg.intercept_ + linreg.coef_ * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.51699012])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=2 using the predict method\n",
    "linreg.predict(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['al'] [-0.00247761]\n"
     ]
    }
   ],
   "source": [
    "# examine coefficient for al\n",
    "print(feature_cols, linreg.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** A 1 unit increase in 'al' is associated with a 0.0025 unit decrease in 'ri'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.5145125136125304"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# increasing al by 1 (so that al=3) decreases ri by 0.0025\n",
    "1.51699012 - 0.0024776063874696243"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.51451251])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute prediction for al=3 using the predict method\n",
    "linreg.predict(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting a Categorical Response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    70\n",
       "2    76\n",
       "3    17\n",
       "5    13\n",
       "6     9\n",
       "7    29\n",
       "Name: glass_type, dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine glass_type\n",
    "glass.glass_type.value_counts().sort_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "      <th>household</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  household  \n",
       "id                        \n",
       "22   1.521227          0  \n",
       "185  1.521103          1  \n",
       "40   1.520781          0  \n",
       "39   1.520781          0  \n",
       "51   1.520682          0  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# types 1, 2, 3 are window glass\n",
    "# types 5, 6, 7 are household glass\n",
    "glass['household'] = glass.glass_type.map({1:0, 2:0, 3:0, 5:1, 6:1, 7:1})\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's change our task, so that we're predicting **household** using **al**. Let's visualize the relationship to figure out how to do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAENCAYAAAAVPvJNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAGlFJREFUeJzt3X20XXV95/H3Ta7yTEg4F0NCBJTUis+rFnTamdJqXeA4ZLkGvwRRCj5kbEt9bqvWQVdm2TK6FDPKaFNkApYRv1KnoMMUl8u6bB3tgFaH4UHFVCSEQEJ4MiCQ5M4f+9xwcvK795ydnMfk/Vrrrty9z2/v/dn77rO/2Xv/zj4T09PTSJLUbt6wA0iSRpMFQpJUZIGQJBVZICRJRRYISVKRBUKSVGSBkCQVWSAkSUUWCElS0eSwA+wjPwYuSXtnolODcS8QbNy4cdgRdmk0GmzZsmXYMfaK2YdjnLPDeOc/kLMvWbKkq3ZeYpIkFVkgJElFFghJUpEFQpJUZIGQJBVZICRJRRYISVLRQD4HERGXA68B7svM5xdenwDWAK8GHgXOz8zvDyKbJKlsUB+UWwd8GrhyltfPAJY3f04FPtP8d2B2bt4E117F9INbmThqEaw4l3lTi0dunqNqx+03w7o18Og2OPQweO0bmbj5puK6d7Ndut12Ozdv4qHPf5od994DBx9Sjdx0N2y+B3buhHnz4BlLmXjms5j+zVcx8Y9fY/q+e2DrFtj+ZPX6s57DxNlv2W3+u5Z/3z3w8INw5EI4cgE88hD87A7YuQMmJuDgQ+H4Z1cT3X1n9W/b/FrXhYMPaZnHTu6dNwGHHwlHP4OJYxZ3nbG0fYDdlwNV9mb+iWMW79qO7dPvWu4sw+3bf2b6rdseYcf85mHkl491tZ/v3LyJ6S9eBut/tMf26rRec2Wp+z7bvmkjOz770WKOUdW63XcedkTfjykT09ODeVpFRJwAfHWWM4i/BL6ZmV9oDv8IOC0z7+kw2+lefJJ65+ZNTF9yEWze9NTIqcVMvGt1rY3f+unGXs1zUPblk5k7br8ZLrmoOmjOprnuQMft0u22K7aby7z5s2dcNMXEez+y6yBVa75zzA8K67uPGYvzXNioCtbWzXPPf2oxnPdHcOWndp++fbntwy3bv+P2mWM/37l5E9Mf+wA80LavLZqCC965Z67SenXK0sX7bOfmTfDxDzJ9/3175JjZD0ZNL48pzU9Sd3zUxqjcg1gK3NUyvKE5bjCuvWrPnb1ZqUdqnqNq3Zq5iwM8te7dbJdut12p3Vzmyrh181PzrzvfuebXj4yleT6wpXNxgGq6dWv2nL59ue3Drdu/0zrNtZ9fe9WexQGq7KVcpfXqlKWb99m1V+1ZHGZyjOp7dAjHlFF5FlOpkhVPbSJiFbAKIDNpNBr7vPCt2x7hycL4yW2PsKjG/CcnJ3fl6dU8B6U1e133PvZod8vY9ghAx+3S7babrd3empl/r+Y71/oOdZ5d/r1Ky+52+8y2n885bY1cnbJ0ep/NlWNU36PDOKaMSoHYACxrGT4OKF47ysy1wNrm4HQvHra187AjiuO3H3ZErcsuu11i6tE8B2WfHv51yKHw6C86Nts+yzaZea3utput3d6amX+v5jvX+g51nl3+vUrL7nb7zLafzzltjVydsnR6n82VY1Tfo708pozbw/quA86LiImIeBnwUBf3H3pnxbnVtdlWU4ufukE2KvMcVee/o7pmPZeZde9mu3S77Urt5jJXxkVTT82/7nznml8/MpbmubBRvd7J1OLq79U+ffty24dbt3+ndZprP19xbpW13aKpcq7SenXK0s37bMW5TBx9TDnHqL5Hh3BMGchN6oj4AnAa0ADuBT4EPA0gMz/b7Ob6aeB0qm6uF2TmTV3Muic3qaE3PY7a/xc+Tr2Y9vXxwcPsxXTQ313DL/emF9OOJ2Giy15MCxbCEb3vxcRML6bGM5iYKvRimiVj7V5MCxYyMdX7XkyT2x7hyTHtxXTU9ie4f0x7MU1ue6Q6m9zLY0q3N6kH1oupT3pWIHrhQH6+/DCZfXjGOf+BnH3cejFJkkaMBUKSVGSBkCQVWSAkSUUWCElSkQVCklRkgZAkFVkgJElFFghJUpEFQpJUZIGQJBVZICRJRRYISVKRBUKSVGSBkCQVWSAkSUUWCElSkQVCklRkgZAkFVkgJElFFghJUpEFQpJUZIGQJBVZICRJRRYISVKRBUKSVGSBkCQVWSAkSUUWCElSkQVCklQ0OagFRcTpwBpgPnBZZl7c9vozgSuAo5pt3peZ1w8qnyRpdwM5g4iI+cClwBnAycA5EXFyW7MPApmZLwFWAv91ENkkSWWDusR0CnBHZq7PzCeAq4EVbW2mgSObvy8ANg4omySpYFCXmJYCd7UMbwBObWvzYeBrEfFHwGHAK0sziohVwCqAzKTRaPQ87N6anJwcqTx1mH04xjk7jHd+s3exnL4voTJRGDfdNnwOsC4zPx4RLwc+HxHPz8ydrY0ycy2wdmYeW7Zs6X3avdRoNBilPHWYfTjGOTuMd/4DOfuSJUu6ajeoS0wbgGUtw8ex5yWkNwMJkJnfAQ4GxrO8S9J+YFBnEDcCyyPiROBuqpvQr29r83PgFcC6iHguVYHYPKB8kqQ2AzmDyMztwIXADcBt1ai8JSJWR8SZzWbvAd4aET8EvgCcn5ntl6EkSQMyMT091sfg6Y0bR6ez04F8TXOYzD4845z/QM7evAdRuje8Gz9JLUkqskBIkoosEJKkIguEJKnIAiFJKrJASJKKLBCSpCILhCSpyAIhSSqyQEiSiiwQkqQiC4QkqcgCIUkqskBIkoosEJKkIguEJKnIAiFJKrJASJKKLBCSpCILhCSpyAIhSSqyQEiSiiwQkqQiC4QkqWhyrhcjoqsCkpk7exNHkjQq5iwQwHZguov5zO9BFknSCOlUIE5s+f3fAmcBfwHcCRwP/CnwN/2JJkkapjkLRGbeOfN7RLwbeGlmPtgc9eOIuAm4CfhM/yJKkoahzk3qBcChbeMObY6XJO1nOl1ianUF8PWI+CRwF7AMeHtzfEcRcTqwhup+xWWZeXGhTQAfprrv8cPMfH2NfJKkHqpzBvEnwH8BzgY+AawEPt0cP6eImA9cCpwBnAycExEnt7VZDrwf+I3MfB7wzhrZJEk91vUZRLMr62ebP3WdAtyRmesBIuJqYAVwa0ubtwKXZuYDzeXdtxfLkST1SKfPQbypm5lk5uUdmiyluiw1YwNwalubX2ku89tUl6E+nJl/183yJUm91+kM4o1dzGMa6FQgJmaZrj3LcuA04DjgHyLi+S29pgCIiFXAKoDMpNFodBFxMCYnJ0cqTx1mH45xzg7jnd/sXSxnrhcz87d7tJwNVDe1ZxwHbCy0+W5mPgn8S0T8iKpg3NiWaS2wtjk4vWXLlh5F3HeNRoNRylOH2YdjnLPDeOc/kLMvWbKkq3Z1ejEREQuBf0d1yehu4Csz9ww6uBFYHhEnNqdbCbT3UPpb4BxgXUQ0qC45ra+TT5LUO133YoqIlwM/Bd4GvBD4D8BPm+PnlJnbgQuBG4DbqlF5S0Ssjogzm81uAO6PiFuBvwf+ODPvr7U2kqSeqXMG8UngDzLz6pkREXE2VdfXX+80cWZeD1zfNu6ilt+ngXc3fyRJQ1bncxC/AmTbuGuAk3oXR5I0KuoUiJ9Q3Tto9Tqqy06SpP1MnUtM7wS+GhFvp3qa6wlUvYxe04dckqQh6/oMIjP/N/BsqsdrfA/4FHBSc7wkaT9Tq5trs0vrX/cpiyRphHRdIJqfYfgI8GLg8NbXMvOZPc4lSRqyOmcQ/53qhvR7gEf7E0eSNCrqFIjnUT2Ke2e/wkiSRkedbq7fAl7SryCSpNHS6XHfq1sGfwbcEBFfBja1tmv9RLQkaf/Q6RLTsrbhrwBPK4yXJO1nOj3u+4JBBZEkjZa6j/t+LnAW8IzMvDAingMclJn/ty/pJElDU+dx36+julG9FDivOfoI4BN9yCVJGrI6vZhWA7+bmW8DdjTH/RB4Uc9TSZKGrk6BOIaqIMBT3yc9zZ7fLS1J2g/UKRDfA97YNm4l8H96F0eSNCrq3KR+O/C1iHgzcFhE3ED1JUKv6ksySdJQ1Xnc9+3ArwKXAh8E/hvwgsz8SZ+ySZKGqO7jvh+l+bWjEfEs4GjgF33IJUkasjrdXL8QEf+q+fsFwC3Arc1LTpKk/Uydm9SvAG5q/v5u4JXAKcD7eh1KkjR8dS4xPT0zn4iIpcCizPw2QEQ8oz/RJEnDVKdA/CAi3g8cD/xPgGaxeLgfwSRJw1XnEtObgRcAh1D1YgJ4OXBVr0NJkoav6zOIzPwp8Pq2cdcA1/Q6lCRp+LouEBHxptley8zLexNHkjQq6tyDaH/MxmLg2cC3AQuEJO1n6lxi+u32cc2ziuf2NJEkaSTUuUldso7q5rUkaT9T5x5EezE5FHgD8GCX058OrAHmA5dl5sWztDsL+BLw65l5U6mNJKn/6pxBbAeebPl5CPgA8PudJoyI+VQP+TsDOBk4JyJOLrQ7guqpsf9UI5ckqQ/q3KQ+sW14W2Zu6XLaU4A7MnM9QERcDawAbm1r95+AjwLvrZFLktQHdW5S3wkQEc+k+l7qDTWWsxS4q2V4A3Bqa4OIeAmwLDO/GhGzFoiIWAWsamai0WjUiNFfk5OTI5WnDrMPxzhnh/HOb/YultNtw4g4Fria6tPT9wNHR8R3gZWZubHD5BOFcbu+qrR5f+MS4PxOOTJzLbB2Zh5btnR7EtN/jUaDUcpTh9mHY5yzw3jnP5CzL1mypKt2de5BfIbqO6kXZuaxwELgn4HPdjHtBmBZy/BxQGtROQJ4PvDNiPgZ8DLguoh4aY18kqQeqnMP4jeBYzPzSYDM3BYRfwLc3cW0NwLLI+LEZvuVtDy2IzMfAnadL0XEN4H32otJkoanzhnEA1Q9kFo9hy66uWbmduBC4AbgtmpU3hIRqyPizBoZJEkDUucM4qPA1yPic8CdwAlU9wz+YzcTZ+b1wPVt4y6ape1pNXJJkvqg6zOIzPwrIKguBb0GWER1g3rtnBNKksZSne+kfjqwnOpDcluBg4ALIuLKPmWTJA1RnUtMVwAvAr4CbOpPHEnSqKhTIE4HTszMrp69JEkab3V6Mf2c6rKSJOkAMOcZRET8TsvglcC1EbEGuLe1XWZ+ow/ZJElD1OkS0+cK4/68bXgaeFZv4kiSRsWcBSIz25/gKkk6QOzrN8pJkvZTFghJUpEFQpJUZIGQJBVZICRJRRYISVKRBUKSVGSBkCQVWSAkSUUWCElSkQVCklRkgZAkFVkgJElFFghJUpEFQpJUZIGQJBVZICRJRRYISVKRBUKSVGSBkCQVWSAkSUWTg1pQRJwOrAHmA5dl5sVtr78beAuwHdgMvCkz7xxUPknS7gZyBhER84FLgTOAk4FzIuLktmb/DLw0M18IXAN8dBDZJEllgzqDOAW4IzPXA0TE1cAK4NaZBpn59y3tvwu8YUDZJEkFg7oHsRS4q2V4Q3PcbN4M/K++JpIkzWlQZxAThXHTpYYR8QbgpcBvzfL6KmAVQGbSaDR6lXGfTU5OjlSeOsw+HOOcHcY7v9m7WE7fl1DZACxrGT4O2NjeKCJeCfwZ8FuZ+XhpRpm5FljbHJzesmVLj6PuvUajwSjlqcPswzHO2WG88x/I2ZcsWdJVu0EViBuB5RFxInA3sBJ4fWuDiHgJ8JfA6Zl534BySZJmMZB7EJm5HbgQuAG4rRqVt0TE6og4s9nsY8DhwJci4gcRcd0gskmSyiamp4u3AsbF9MaNe1ypGpoD+ZR1mMw+POOc/0DO3rzEVLo3vBs/SS1JKrJASJKKLBCSpCILhCSpyAIhSSqyQEiSiiwQkqQiC4QkqcgCIUkqskBIkoosEJKkIguEJKnIAiFJKrJASJKKLBCSpCILhCSpyAIhSSqyQEiSiiwQkqQiC4QkqcgCIUkqskBIkoosEJKkIguEJKnIAiFJKrJASJKKLBCSpCILhCSpyAIhSSqyQEiSiiYHtaCIOB1YA8wHLsvMi9tePwi4Evg14H7g7Mz8WT+y7Ny8Ca69iukHtzJx1CJYcS7zphbXmseO22+GdWvg0W1w6GHw2jfy0I9vZse99+yaJ8D0Fy+D9T+qJnrWc5g4+y27lrVz86bq9Z/cCk88DgcdDCc9F165Ar5+7VPTTS2GzZvgFw/D9DTMnw8HHwrLTqym+/l62LkTjjwKXvVauOHL8PADVdsZ8ydh+fOYOO8Pd1vXmfW497FH4ZBD4fx3MHH01FO5fvkYTExU89q5Y+83+t6YmAfzJmDH3Mu9t9N85s+H45fDQw/A/S2tp46FJcvg4Qern8mnwwObq+UefAgcuwwef6x67ciFcOQCePyXcPed1fQtf89df8v1P6ryPv0gWNhg4pjFe7V/7dpH77unWv4hh8Fj2+DIhXs9T6muienWg0ifRMR84MfA7wIbgBuBczLz1pY2fwC8MDPfFhErgddm5tkdZj29cePGWll2bt7E9CUXVQfcGVOLmXjX6q7fcDtuvxkuuWjuA+bCRnWgePiB3ccvmmLivR+pwn/sA/DAllr599mCRUz86cXMm1o8y3pMwOFHVMVInS2aggveCZdfMvvfco79q9FosGXL7tMV99Ea8xykUv5xcSBnX7JkCcBEp3aDusR0CnBHZq7PzCeAq4EVbW1WAFc0f78GeEVEdFyB2q69as83XvN/a11bt6bz/6Yf2LJncQDYurla1rVXDb44ADy09al1La7HtMWhjq2bq+0419+y7v5V2kf3dZ7SXhjUJaalwF0twxuAU2drk5nbI+Ih4Ghgt3deRKwCVjXb0Wg0agXZuu0RniyMn9z2CIu6nNe9jz1aa5mlZQHFHIMws677uh5q6mI7zrZ/TU5O7rEPz7aPdjvPQSrlHxdm72I5fV9CpXQm0H5tq5s2ZOZaYO3M63VPs3YedkRx/PbDjuj+lO2QQ+HRX9RabvuyhmnXuu7jeqipi+042/5VvMTU5f5Ra5/tkwP5Ms0w9egSU0eDusS0AVjWMnwc0H7zYFebiJgEFgBbe55kxbnVTd9WU4t33VTuyvnvgHnz526zsFHd2Gy3aKpa1opzqzaDtuCpG+jl9ZiAw48ceKyxtWiq2o5z/S3r7l+lfXRf5ynthUGdQdwILI+IE4G7gZXA69vaXAf8HvAd4CzgG5nZ8zvo86YWs/Ndq/epF9P8X30BO961eo9eTAf/+GZ+WacX0x//+VB7Me22HmPci6mjur2Y5s2Dg9p6MS1YCEfM0Ytp5m/Z2otpUYOJqfo9jnbbR9t7MS1YuFfzlPbGQHoxAUTEq4FPUnVzvTwzPxIRq4GbMvO6iDgY+DzwEqozh5WZub7DbGv3YuqnA/mUdZjMPjzjnP9Azt5tL6aBFYg+sUD0iNmHY5yzw3jnP5Czj1o3V0nSmLFASJKKLBCSpCILhCSpyAIhSSqyQEiSiiwQkqSisf8cxLADSNKY2u8/BzExSj8R8b1hZzD7eP2Mc/Zxz2/2zsa9QEiS+sQCIUkqskD01trOTUaW2YdjnLPDeOc3ewfjfpNaktQnnkFIkooG9YVB+5WIOB1YQ/XdFpdl5sVtr58PfIzqy5EAPp2Zlw00ZEFEXA68BrgvM59feH2Car1eDTwKnJ+Z3x9syrIusp8GXAv8S3PUlzNz9eASzi4ilgFXAouBncDazFzT1mYkt32X2U9jdLf9wcC3gIOojnfXZOaH2tocRLWOvwbcD5ydmT8bcNQ9dJn9fPp4rPEMoqaImA9cCpwBnAycExEnF5p+MTNf3PwZenFoWgecPsfrZwDLmz+rgM8MIFO31jF3doB/aNnmI3GAatoOvCcznwu8DPjDwj4zqtu+m+wwutv+ceB3MvNFwIuB0yPiZW1t3gw8kJknAZcA/3nAGWfTTXbo47HGAlHfKcAdmbk+M58ArgZWDDlTVzLzW8z9Pd8rgCszczozvwscFRHHDibd3LrIPrIy856Zs4HMfAS4DVja1mwkt32X2UdWc3v+ojn4tOZP+43XFcAVzd+vAV7RPKMbqi6z95WXmOpbCtzVMrwBOLXQ7t9HxL8Bfgy8KzPvKrQZNaV1WwrcM5w4tb08In4IbATem5m3DDtQu4g4geprdf+p7aWR3/ZzZIcR3vbNs/7vAScBl2bmrNs+M7dHxEPA0cDQv26ui+zQx2ONZxD1lf5n0V7VvwKckJkvBL7OU/87GXXdrNuo+j5wfPN0/FPA3w45zx4i4nDgb4B3ZubDbS+P9LbvkH2kt31m7sjMFwPHAadERPs9rJHd9l1k7+uxxgJR3wZgWcvwcVT/a9olM+/PzMebg39FdfNrHHRct1GVmQ/PnI5n5vXA0yKiMeRYu0TE06gOsFdl5pcLTUZ223fKPurbfkZmPgh8kz3vZe3a9hExCSxgxC5nzpa938caC0R9NwLLI+LEiHg6sBK4rrVB27XjM6mu246D64DzImKieTPsocwcmUscc4mIxTPXjSPiFKp9+/7hpqo0c30OuC0zPzFLs5Hc9t1kH/FtPxURRzV/PwR4JXB7W7PrgN9r/n4W8I3MHPoZRDfZ+32s8R5ETc1rlBcCN1B1c708M2+JiNXATZl5HfD2iDiTqgfIVuD8oQVuERFfAE4DGhGxAfgQ1Y0vMvOzwPVU3SzvoOpqecFwku6pi+xnAb8fEduBx4CVo/Amb/oN4I3AzRHxg+a4DwDPhJHf9t1kH+VtfyxwRfNa/jwgM/Orbe/XzwGfj4g7qN6vK4cXdzfdZO/rscZPUkuSirzEJEkqskBIkoosEJKkIguEJKnIAiFJKrJASH0WEedHxD8OO4dUlwVCklRkgZAkFflJaqlHIuJ9wFuBY6ieDvpnmfk/hptK2nsWCKl3fgr8a2AT8DrgryPipOFGkvaeBULqkcz8UsvgFyPi/VRfMCWNJQuE1CMRcR7wbuCE5qjDgQawY1iZpH3hTWqpByLieKrn8V8IHJ2ZRwH/j/KX0UhjwQIh9cZhVN9CthkgIi4A2r/9SxorFgipBzLzVuDjwHeAe4EXAN8eaihpH/l9EJKkIs8gJElFFghJUpEFQpJUZIGQJBVZICRJRRYISVKRBUKSVGSBkCQVWSAkSUX/H59wNk1UW1UIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf6af34198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(glass.al, glass.household)\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's draw a **regression line**, like we did before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# fit a linear regression model and store the predictions\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.household\n",
    "linreg.fit(X, y)\n",
    "glass['household_pred'] = linreg.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAENCAYAAAAhRzNRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xt8VPWd//HX5H5PgHARUFGhlnrZWpWuVq2XSlnXFbvqV2y1xRvaQstiEEEEEaqgQBErVYECohT9irph1a2shS5ui1tUtvoTxaJS5Sa5J+SezPz+mEHjkMvMZGbOmeT9fDx4kDnzzTnv+WZyPjmX+X49Pp8PERGRcCU5HUBERBKTCoiIiEREBURERCKiAiIiIhFRARERkYiogIiISERUQEREJCIqICIiEhEVEBERiUiK0wFiTB+zFxGJjKerBj29gLB//36nI3yhsLCQ0tJSp2NEJJGzQ2LnV3Zn9ObsgwcPDqmdTmGJiEhEVEBERCQiKiAiIhIRFRAREYmICoiIiEREBURERCKiAiIiIhFRARER6UFS336bpAcfjMu2VEBERHoAT309effdR+HYsSSvXImnujrm21QBERFJcGl//jP9v/c9cpYvp+7662l+6y18eXkx364KiIhIgvLU1JB/110UXnMNAKUbNlA1fz7EoXhALxgLS0SkJ0r/wx8ouOsukj7/nMO33UbNnXfiy8yMawYVEBGRBOIpLyd/zhyynn+e5pNPpnzFCprPOMORLK4pIMaYVcDlwCFr7antPP8j4K7Aw8PAT621f41jRBERR2W89BL5M2eSVFlJzZQp1Pz855Ce7lgeN10DWQOM6eT5T4DvWmtPB+YBy+MRSkTEaUmHDtHn1lvpe9tttA4eTMkrr1AzdaqjxQNcdARird1qjBnWyfN/bvPwDWBozEOJiDjJ5yNzwwby58zBU19P9d13c/i22yDFHbtud6QI383AfzodQkQkVpL37SP/rrvI2LKFxrPPpnLRIlqHD3c61lckXAExxlyEv4Cc18HzE4AJANZaCgsL45iucykpKa7KE45Ezg6JnV/ZneFYdq+XpBUrSL77bvD5aFmyBM/tt9MnKfQrDvHKnlAFxBhzOrAS+CdrbVl7bay1y/ny+ojPTVNS9uYpMp2WyPmV3RlOZE/+5BMK7ryTlG3baDz/fCoXLqT12GOhvDys9cRrStuEKSDGmOOAF4AbrLUfOp1HRCRqWlvJXrGC3IULIS2NisWLqb/2WvB4nE7WKdcUEGPMeuBCoNAYsxe4F0gFsNY+DswG+gG/McYAtFhrz3ImrYhIdKTs2kVBURFpO3ZQ//3vU/XAA3gHDXI6VkhcU0Cstdd18fwtwC1xiiMiEltNTeQsW0bu0qV4c3Mp/81vaLjiCtcfdbTlmgIiItJbpL7zDgV33EHq++9Td+WVVM+di7dfP6djhU0FREQkXurryV2yhJzHH8fbvz9lq1fTOHq006kipgIiIhIHaX/5CwVFRaR8/DG1P/wh1ffcgy8/3+lY3aICIiISQ57aWnLnzyd7zRpahw6ldP16mi64wOlYUaECIiISI+lbt5I/bRrJe/dSe9NN1Nx1F77sbKdjRY0KiIhIlHmqqsibO5fsZ56h+aSTKHvxRZrOPtvpWFGnAiIiEkUZr75K/owZJJWWUjNpEjVTpkBGhtOxYkIFREQkCpLKysibNYus4mKaR46k/MknaT7tNKdjxZQKiIhId/h8ZBYXkzdrFkk1NVRPncrhiRMhLc3pZDGnAiIiEqGkAwcomDGDjP/6L5rOOIOyxYtpOflkp2PFjQqIiEi4fD6y1q8nb948aGqiavZsam+5BZKTnU4WVyogIiJhSP70UwqmTSP99ddpPOcc/5DrJ5zgdCxHqICIiITC6yV79Wpy58+H5GQq58+n7vrrIYyJnnoaFRARkS6k7N7tH3L9zTdpuPhiKhcswDtkiNOxHKcCIiLSkZYWch5/nNxf/QpfZiYVS5dSf9VVCTXkeiypgIiItMPzzjsU3nQTae++S/1ll1F1//14BwxwOparuKaAGGNWAZcDh6y1p7bzvAdYClwG1AHjrbVvxyuft+QgFK/DV1mOp6AvjP0RSf27N2tYLNbpVq0fvAtrlkJdLWRlww9uwPPumx2+9q76Jpy+85YcpOqpR2n9bA9UVwIeqCgBr9d//nrgEDzHnYjvvNF4/mcTvspyyMiExgbY93f/Sk48Gc+1t3yxjS+2f+iAf515fSAv3/89n34MDfXg8/nXn5EJAwdDean/+axsGD+Z5K+f1u5r8Z03Gl5+Fj58D7xePk/yQE4e9Bvo3wb4t1leCi3N/m0E5etovV95fUfWE8jvGTDoK/0Y0vc31HfY/1/0++cHQmp/VO42fds2W2e5OssS8u9aYyO5S5eSsmwZ3ow0SsecR30/H6xceFQfuc2R11leW4M3OzfmWT0+ny9mKw+HMeYC4DCwtoMCchnwc/wF5NvAUmvtt7tYrW///v3dzuYtOYhvyWwoOfjlwv6D8EyZG9YPp+1E99FaZ7y0zR6u1g/ehSWzwdvacaM2r72rvgmn79pt25Gk5M4z9u2PZ+r9AKGvs7NtTZmLp1//7q8rKF+HfdTV64Mv+hHaeY2dfX9Q/3fZ75H8vPoPgh//HNb+uvNcoWTpYPupb79NQVERqR9+SN1JQ6k4oR/etKC/s136exrNfcrgwYMBujxP55rbB6y1W4HyTpqMxV9cfNbaN4ACY8wxcQlXvO7oN3Sg0rtqnW61ZmnXO662r72rvgmn79pr25GuMpaX+NcXzjo729aapdFZV3A+aH+9Xb0++LIfw/3+4P7v6nVF8vMqOejvs65yhZIlqI2nvp68++6jcOxYkmpqKL3+B5SdPPDo4tFZdqc5sE9xzSmsEAwBPmvzeG9g2YG2jYwxE4AJANZaCgsLu73h8toamttZnlJbQ98w1p+SkvJFnmitM17aZg/X5/V1oW0j8Nq76ptw+q6jtpFKqa0BiM466+tIiUG+zvoo1HVA+K+xbf+Hsv2Ifl5hvpc6W+eRNp7//m9Sbr8dz8cf03rrrbQ+8AAtC++G8k/Dyu40J/YpiVRA2jucOur8m7V2ObD8yPORnnZpy5ud2+7yluzcsE7rfOUUVpTWGS/dOYVFZhbUHe6y2ZHX3lXfhNN3HbWNVEs015eZFd310XUfhbqO7mwbQuv3iH5eYb6XOltna3IazTffTPbTT9MybBiVzz1H07nnQlNTlznc+HsazX1K4BRWl1xzCisEe4Fj2zweCnT/Akcoxv7If/61rf7+i2muWqdbjZ/sP0/dmbavvau+Cafv2mvbka4y9u3vX1846+xsW+MnR2ddwfmg/fV29frgy34M9/uD+7+r1xXJz6v/IH+fdZUrhCwZjUkMXLWBrN/9jsO33UbJa6/5i0eb70ke2MHnPNz6e+rAPsU1F9EBjDHDgJc6uIj+z8AkvryI/oi1dlQXq4zKRXSIzh1TwX/FJ9JdWN06AsH5u7DSf7+BhljehZXfB3JjcxcWR+7CKhzo3wZ8eRdWazN4onAXVn4fPP2jfxdW+u830NDdu7CCsnXnLizPgf0UbPsr2X99n+avfY3KxYtp/ta32s1R0NJE+Zpfd5jDjY68zpTaGv+RZIRZQ72I7poCYoxZD1wIFAKfA/cCqQDW2scDt/E+CozBfxvvjdbaN7tYbdQKSDR0dyfspETODomdX9mjI+Oll8ifOZOkykoOT5pEzS9+AenpHbZ3U/ZwdTd7qAXENddArLXXdfG8D5gYpzgi0kMkHTpE/syZZL7yCk2nnUbZ735HyymnOB2rR3BNARERiSqfj8wNG8ifMwdPfT3Vd9/N4dtugxTt9qJFPSkiPU7yvn3k33UXGVu20Hj22VQuWkTr8OFOx+pxVEBEpOfwesl66iny7r8ffD6q5s2jdvz4Xj3keiypgIhIj5D8yScU3Hkn6du20Xj++f6Jno49tutvlIipgIhIYmttJXvFCnIXLoS0NCoXLaJu3DgNuR4HKiAikrBSdu3yT/S0Ywf1o0dTNX8+3kHu/IxGT6QCIiKJp6mJnGXLyF26FG9uLuW/+Q0NV1yho444UwERkYSS+s47FNxxB6nvv0/dlVdSPXcu3n79nI7VK6mAiEhiqK8nd8kSch5/HG9hIWWrV9M4erTTqXo1FRARcb20v/yFgqIiUj7+mNrrrqN61ix8+flOx+r1VEBExLU8tbXkzp9P9po1tA4dSun69TRdcIHTsSRABUREXCl961byp00jee9eam+8kZrp0/FlZzsdS9pQARERV/FUVZE3dy7ZzzxD80knUfbiizSdfbbTsaQdKiAi4hoZr75K/owZJJWWUjNpEjVTpkBGhtOxpAMqICLiuKSyMvJmzSKruJjmkSMpX7OG5tNPdzqWdME1BcQYMwZYCiQDK621C4KePw54EigItJlurX0l7kFFJHp8PjKLi8mbNYukmhqqp07l8MSJkJbmdDIJgSuGqDTGJAPLgH8CvgFcZ4z5RlCzewBrrT0DGAf8Jr4pRSSakg4coO+NN9Jn4kRajz+ekldf5fCUKSoeCcQtRyCjgN3W2o8BjDHPAGOBnW3a+IC8wNf5gHvmqhWR0Pl8ZD3zDHlz50JTE1WzZ1N7yy2QnOx0MgmTWwrIEOCzNo/3At8OajMH2GSM+TmQDXwvPtFEJFqSP/2UgmnTSH/9dRrPOcc/5PoJJzgdSyLklgLS3ghovqDH1wFrrLWLjTHnAE8ZY0611nrbNjLGTAAmAFhrKSwsjEngSKSkpLgqTzgSOTskdv4ekd3rJemxx0ieNQuSkmj59a/x3HILfVw80VOP6PdYbyfmWwjNXqDtzC9DOfoU1c3AGABr7TZjTAZQCBxq28hauxxYHnjoKy0tjUngSBQWFuKmPOFI5OyQ2PkTPXvFG29QMHUqKdu303DxxVQuWIB3yBAoL3c6XqcSvd+7k33w4MEhtXNLAdkOjDDGnADsw3+R/IdBbT4FLgHWGGNGAhlASVxTikjoWlpIWriQAfPm4cvMpGLpUuqvukpDrvcgrjh+tNa2AJOAV4H3/Yvse8aYucaYKwLNioBbjTF/BdYD4621wae5RMQFUt57j8LLLyflnntouOQSDm3ZQv3VV6t49DAen69H74N9+/e752at3nxI7LREzp9Q2RsbyX3kEXIefRRvnz54f/1rSs4/3+lUEUmofg8SpVNYXVZ7VxyBiEjiS337bfqPGUPuww9Tf+WVHNqyBd8PfuB0LIkht1wDEZEE5amvJ/ehh8heuRLvwIGUPfUUjRdf7HQsiQMVEBGJWNq2bf47rPbsofaGG6ieORNfbq7TsSROVEBEJGyemhry7r+f7KeeomXYMEqfe46mc891OpbEmQqIiIQlffNm8u+6i+SDBzl8223U3HknvsxMp2OJA1RARCQknooK8u+9l6znn6f5a1+jtLiY5m99y+lY4iAVEBHpUsbLL5M/cyZJFRXU/Nu/UfOLX0B6utOxxGEqICLSoaRDh8ifOZPMV16h6bTTKFu3jpZTTnE6lrhEpwXEGBPS50SCBzQUkQTn85G5YQP5c+bgqa+n+u67OXzbbZCivznlS129G1o4elTc9mggf5EeImnfPgqmTydj82Yazz6bykWLaB0+3OlY4kJdFZC2A/X/M3A1MB/4O3A8cBfwfGyiiUhceb1kPf00efffD14vVfPmUTt+PLh4yHVxVqcFxFr79yNfG2PuAM6y1lYGFn1ojHkTeBN4LHYRRSTWkj/5hII77yR92zYazz/fP9HTscd2/Y3Sq4VzQjMfyAIq2yzLCiwXkUTU2kr2ihXkLlwIaWlULlpE3bhxGjVXQhJOAXkSeM0Y8zD+6WePBX4RWC4iCSZl1y4KiopI27GD+tGjqZo/H++gQU7HkgQSTgGZBuwGrgUGAweAR4EVMcglIrHS3EzOo4+Su3Qp3txcyn/zGxquuEJHHRK2kAtI4FbdxwP/RCQBpb7zDgV33EHq++9Td+WVVM+di7dfP6djSYLq6nMgN4WyEmvtqu4GMcaMAZbivyV4pbV2QTttDDAH/63Ff7XWBk97KyLtaWggd8kSch57DG9hIWWrV9M4erTTqSTBdXUEckMI6/AB3SogxphkYBlwKbAX2G6M2Wit3dmmzQhgBvAda22FMWZAd7Yp0lukbd9OflERqR99RO1111E9axa+fN37It3X1W28F8Upxyhgt7X2YwBjzDPAWGBnmza3AsustRWBbIfilE0kIXlqa8ldsIDs1atpHTqU0vXrabrgAqdjSQ8S1rgExpg+wL8AQ4B9wH8c2aF30xD8d3YdsRf4dlCbrwUy/An/aa451trft5NxAjABwFpLYWFhFOJFR0pKiqvyhCORs0Ni548ku+cPfyDlpz+FTz/F+9Of4p03j7ycnBgl7Fhv63e3iFf2kAuIMeYc4GXgA/yfRL8ceNgY88/W2m3dzNHe7R/BQ6ikACOAC4GhwOvGmFPbfLARAGvtcmD5kXV0Z2L5aOvuRPdOSuTskNj5w8nuqaoib948stevp/mkk6h68UWazj4bGhr8/+Kst/S723Q3++DBg0NqF84RyMPAz6y1zxxZYIy5FngEODusdEfbi/9zJUcMBfa30+YNa20z8IkxZhf+grK9m9sW6RHSN22iYMYMkkpKqJk0iZopUyAjw+lY0oOFU0C+BtigZRuIzm2924ERxpgT8J8aGwcE32H178B1wBpjTGEgz8dR2LZIQksqKyNv1iyyiotpHjmS8tWraT79dKdjSS8Qzihpf8O/Y2/rGuCj7oaw1rYAk4BXgff9i+x7xpi5xpgrAs1eBcqMMTuBLcCd1tqy7m5bJGH5fGQUF9P/wgvJfOUVqqdOpeSVV1Q8JG48Pl8oo7WDMeZc4CXgQ/zXQIbhP4V0ubX2z7EK2E2+/fuDz4Q5pzefU3VaIudvL3vSwYPkz5hB5qZNNJ1xBpWLF9Ny8skOJexYT+v3RBGlayBdDk0Q8hFIoEichH/4kreAXwPDXVw8RHoen4+s9esZcNFFpG/dStXs2ZQWF7uyeEjPF9ZtvIFbdp+OURYR6UTyp59SMG0a6a+/TuM55/iHXD/hhK6/USRGwrmN9wTgfuCbwFduKLfWHhflXCJyhNdL9qpV5M6fDx4PlfPnU3f99ZroSRwXzhHI7/BfMC8C6mITR0TaSt69m5QZM8j/859puOgiqh58kNYhQ5yOJQKEV0BOwT8OlTdWYUQkoKWFnMcfJ/dXv4KsLCoefpj6q6/WkOviKuEUkK3AGfgvoItIjKS8955/oqd336X+sstIfuwx6lPCulwpEhddDec+t83DPcCrxpgXgINt21lrZ0c/mkgv09hI7iOPkPPoo3gLCih/4gkaLr/cP6ZRgt5OKj1bV3/WHBv0+D+A1HaWi0g3pL79NgVFRaR++CF1V11F1Zw5+Pr2dTqWSKe6Gs79xngFEemNPPX15D70ENkrV+IdOJCytWtpvOQSp2OJhCTc4dxHAlcDA621k4wxJwPp1tp3YpJOpAdL27aNgqlTSdmzh9obbqB65kx8ublOxxIJWcg3khtjrsF/IX0I8OPA4lzgVzHIJdJjeWpqyJ8+ncKrrwag9LnnqFqwQMVDEk44n0SaC1xqrb0daA0s+yvwD1FPJdJDpW/eTP+LLyZr3ToOT5hAyWuv0XTuuU7HEolIOKewBuAvGPDlZE8+jp74SUSCeCoqyL/3XrKef57mr32N0uJimr/1LadjiXRLOEcgbwE3BC0bB/wlenFEep6Ml19mwEUXkVlcTM3kyZT8/vcqHtIjhHME8gtgkzHmZiDbGPMq/kmdRsckmUiCSzp0iPyZM8l85RWaTjuNsnXraDnlFKdjiURNyAXEWvuBMebr+OdCfwn4DHjJWns4GkGMMWOApUAysNJau6CDdlcDzwFnW2vfjMa2RaLK5yNzwwby58zBU19P9YwZHL79dtCnyaWHCXc49zoC09oaY04E+gHdLiDGmGRgGXAp/rnPtxtjNlprdwa1y8V/JPS/3d2mSCwk7dtHwfTpZGzeTNNZZ/kneho+3OlYIjERzm286wOzEmKMuRF4D9gZOKXVXaOA3dbaj621TcAzwNh22s0DHgIaorBNkejxeslau5YBF19M2rZtVM2dS+kLL6h4SI8WzkX0S4Ajp4zuAL6Hf8c/PQo5huA/JXbE3sCyLxhjzgCOtda+FIXtiURN8ief0M8YCmbMoPmb36Rk82Zqb74ZkpOdjiYSU+Gcwkqz1jYZY4YAfa21fwIwxgyMQo72xqj+4vZgY0wSsAQY39WKjDETgAkA1lr/QHQukZKS4qo84Ujk7BCj/K2tJD3yCMn33QdpabQ8/jie8ePpE+Uh1xO575XdGfHKHk4B+T9jzAzgeOBlgEAxqY5Cjr18dYDGocD+No9zgVOBPxpjAAYBG40xVwRfSLfWLgeWBx76ujOxfLR1d6J7JyVydoh+/pRduygoKiJlxw4aLr2Uyvnz8R5zDJSVRW0bRyRy3yu7M7qbffDgwSG1C6eA3Iz/GkQzcGdg2TnAurCStW87MCIwbe4+/J8v+eGRJ621VcAX5dQY80dgqu7Ckrhrbibn0UfJXboUb24uFcuWUT92rCZ6kl4pnNt4P6LNTj2wbAOwobshrLUtxphJwKv4b+NdZa19LzAfyZvW2o3d3YZId6W+8w4Fd9xB6vvvUzd2LNXz5uHt18/pWCKO8fh8oY1EYoy5qaPnrLWropYounz79+/vulWc9OZDYqd1K39DA7lLlpDz2GN4CwupXLCAxtHx+/xsIve9sjsjSqewujysDucUVvAwJoOAk4A/AW4tICLdkrZ9O/lFRaR+9BG148ZRPXs2vvx8p2OJuEI4p7AuCl4WOCoZGdVEIi7gqa0ld8ECslevpnXIEMrWr6fxggucjiXiKt0dW2ENUMqXF9VFEl7a1q0UTJtG8t691N54IzXTp+PLznY6lojrhFxAAp/FaCsLuB6ojGoiEYd4qqrImzeP7PXraTnxRMpeeIGmUaOcjiXiWuEcgbRw9Nwf+4BboxdHxBnpmzZRMGMGSSUl1EycSM2UKZCZ6XQsEVcLp4CcEPS41lqbmLcoiAQklZWRN2sWWcXFNI8cSfnq1TSffrrTsUQSQjgX0f8OYIw5Dv84VXtjFUok5nw+MjZuJP+ee0iqqaF66lQOT5wIaWlOJxNJGOFcAzkG/yi55wBlQD9jzBvAOGutez5sIdKFpIMHyZ8xg8xNm2g64wzKFi2i5etfdzqWSMIJZzTex/DPid7HWnsM0AfYATwei2AiUefzkbV+PQMuuoiMrVupmjWL0uJiFQ+RCIVTQM4Diqy1tQCB/6cB58YimEhUffIJ/a67joKpU2k+5RQOvfYatbffriHXRbohnAJSAXwjaNnJ6DZecTOvl+xVq0g980xS336byvnzKbOW1hOC7wkRkXCFcxfWQ8BrxpjfAn8HhuGfn2NW9GOJdF/y7t0UTJ1K+vbteEePpuSXv6R1yJCuv1FEQhLyEYi1dgVg8A+rfjnQF/8F9OWdfqNIvLW0kLNsGQNGjyb1b3+j4uGHadm4UcVDJMrCmRM9DRiBfz6QciAduNEYszZG2UTClrJzJ4X/8i/kPfAADZdcwqEtW6i/5hrN1yESA+GcwnoS+AfgP4CDsYkjEqHGRnIfeYScRx/FW1BA+RNP0HD55U6nEunRwikgY4ATrLW6aC6ukrpjBwVFRaTu2kXdVVdRNWcOvr59nY4l0uOFU0A+xX/aKiaMMWOApfhnJFxprV0Q9PwdwC34x+QqAW468ul46Z089fXkLlxI9ooVeAcOpGztWhovucTpWCK9RqcFxBhzcZuHa4FiY8xS4PO27ay1m7sTwhiTDCwDLsU/RMp2Y8xGa+3ONs12AGdZa+uMMT/Ff1fYtd3ZriSutG3bKJg6lZQ9e6i94QaqZ87El5vrdCyRXqWrI5DftrPsgaDHPuDEbuYYBey21n4MYIx5BhgLfFFArLVb2rR/A/9Q8tLLeGpqyHvgAbLXrqVl2DBKn3uOpnP1WVYRJ3RaQKy18fq01RDgszaP9wLf7qT9zcB/xjSRuE76li3kT5tG8sGDHJ4wgZpp0/BpyHURx3R3RsJoae8ey+C5RwAwxlwPnAV8t4PnJwATAKy1FBYWRitjt6WkpLgqTzgczV5eTvKdd5L89NN4R46k5dlnSRs1in5hrEJ97wxld0a8srulgOwFjm3zeChw1Ai/xpjvATOB71prG9tbUeCDjUc+3OgrLXXPlCWFhYW4KU84nMqe8fLL5M+cSVJFBTWTJ1MzeTKkp0OYWdT3zlB2Z3Q3++DBg0Nq55YCsh0YYYw5Af8sh+OAH7ZtYIw5A3gCGGOtPRT/iBJPSSUl5M+cSebLL9N02mmUrVtHyymnOB1LRNoIZzDFmLHWtgCTgFeB9/2L7HvGmLnGmCsCzRYCOcBzxpj/M8ZsdCiuxJLPR+aGDQy48EIyXnuN6hkzKH3pJRUPERfy+HztXmroKXz797tnrqvefEgciqR9+yiYPp2MzZtpOussKhcvpmX48KisW33vDGV3RpROYXU5/o9bTmFJb+b1krVuHXm//CW0tlI1dy6148drrg4Rl1MBEUcl79njH3J92zYazzuPyoULaT3uOKdjiUgIVEDEGa2tZK9cSe5DD0FaGpWLFlE3bpxGzRVJICogEncpu3ZRUFRE2o4dNFx6KZXz5+M95hinY4lImFRAJH6am8lZtozcpUvx5uRQsWwZ9WPH6qhDJEGpgEhcpL77LgV33EHqzp3UjR1L9bx5ePuF81lyEXEbFRCJrYYGcpcsIeexx/AWFlK2ejWNo0c7nUpEokAFRGImbft28ouKSP3oI2rHjaN69mx8+flOxxKRKFEBkajz1NaS++CDZK9aReuQIZStX0/jBRc4HUtEokwFRKIqbetWCqZNI3nvXmpvvJGa6dPxZWc7HUtEYkAFRKLCU1VF3rx5ZK9fT8uJJ1L2wgs0jRrldCwRiSEVEOm29E2bKJgxg6SSEmomTqRmyhTQRE8iPZ4KiEQsqayMvNmzyfr3f6d55EjKV6+m+fTTnY4lInGiAiLh8/nI2LiR/HsU2U0KAAAQz0lEQVTuIammhuqpUzk8cSKkpTmdTETiSAVEwpJ08CD5M2aQuWkTTWecQdmiRbR8/etOxxIRB7imgBhjxgBLgWRgpbV2QdDz6cBa4EygDLjWWrsn3jl7LZ+PzGeeIf+++/A0NVE1axa1t96qIddFejFXFBBjTDKwDLgU//zo240xG621O9s0uxmosNYON8aMAx4Ero1VJm/JQSheh6+yHE9BXxj7I5L6Dwr5+1s/eBfWLIW6WsjKhh/cQNWH79L6+YEv1gf4t3HoAFRXQl4fPAMGfbEtb8lBfM+uhL/thKZGSM+A4SPhe2PhtWL4eJd/Hf0HQXkp1NdCa6t/p97aCimpkJQEjQ3g80FeAYz+Abz6AlRX+JcdkZwCI07B8+OJX3mdrR+8S/Kj8/H+6R36HKqg4R9Oo2LePFq2vQp33AAN9f6xrFLT/OtrrI9G94fGkwRJHv9r7cLnnT2ZnAzHjwB8X/bpEceeCH0L/T+fkoNQX+d/ndk5MGSY//UGfnbkBT4kWV151M8T8P8sP97lz5uWDn0Kv/LzDscX788j753MbP/PP+g9JBJLrpiR0BhzDjDHWvv9wOMZANba+W3avBpos80YkwIcBPpbazt7ARHNSOgtOYhvyWz/DuOI/oPwTJkb0i9l6wfvwpLZ4O1kx9an0L/jLS85+rn+g+DHP4dVS6AizjOi5ffFc9cCkvoPonXnX8n5t9vJ/2AvAJVfH0rtcf0hNw8OV8c3VyLrU+gvGtUV7T/fxXsreHa5dt+fYa4zXnrzrH5OiteMhK6YEx0YAnzW5vHewLJ22wTmUK8CYjMaX/G6o385A3/xhWTN0s6LB/gLQ3vF48i21iyNf/EAqCqH4nUk797NgB/9iD7vfUpj3xwOXnAKtcf397+lVDzCU1HacfGA8N5b0P77s7vrFImAK05h0X6lCz6yCKUNxpgJwAQAay2FhYVhhymvraG5neUptTX0DWF9n9fXhb3No0RjHZHw+sjf/Do5DyzD622l7PRh1A3pqyHXY6yz91ZKSspX3scdvT/DWWe8BGdPJMoewnZivoXQ7AWObfN4KBB87ulIm72BU1j5QHnwiqy1y4HlgYe+SA7jvNm57S5vyc4N7bAwMwvqDoe93aivI0yp1XX0fefvpFXXUX/ZZZRnNeCtq4prht6qs/fWUaewOnh/hrPOeOnNp4GcFKVTWF1yyyms7cAIY8wJxpg0YBywMajNRuAnga+vBjZ3cf0jcmN/5L8O0Vb/Ly+Gdmn8ZEjq4u6kPoXQt3/7z/Uf5F9Hnzj99dPqJe/D/Qz80/skN7ZQtvBBKlaswPvTae28Dg/k5MUnV0/Rp9B/kb0j4by3oP33Z3fXKRIBV1xEBzDGXAY8jP823lXW2vuNMXOBN621G40xGcBTwBn4jzzGWWs/7mK1EV1Eh9jchZXx4bs0dHYXVn4fPP1jcBdWUwN4278LK62ylr7v7CH1cAO1I0+i6rHlMOLrR7+O+jr/UdH4yXj69f8yV4LchdWpcO7Caqjz92XwXVj5fSA36C6sNj9PaOcurL6FX/l5d6S9vyY7vAsr6D3ktN78V7yT4nUR3TUFJEYiLiCx4KY3pKe+ntyFC8lesQLvwIFUPvggjZdc0mF7N2WPRCLnV3Zn9ObsoRYQt1wDkThK27aNgqlTSdmzh9rrr6f6nnvw5YZ2Xl1E5AgVkF7EU1ND3gMPkL12LS3HH0+ptTR95ztOxxKRBKUC0kukb9lC/rRpJB84wOFbb6Vm2jR8WVlOxxKRBKYC0sN5KirInzOHrA0baB4xgtLiYprPPNPpWCLSA6iA9GAZL79M/syZJFVUUDN5MjWTJ0N6utOxRKSHUAHpgZJKSsifOZPMl1+m6dRTKXv6aVpOPdXpWCLSw6iA9CQ+H5nPP0/+vffiqa+nevp0Dt9+O6SmOp1MRHogFZAeImnfPgqmTydj82aazjqLysWLaRk+3OlYItKDqYAkOq+XrHXryPvlL6G1laq5c6kdP14TPYlIzKmAJLDkPXsomDqV9G3baDzvPCofeojW4493OpaI9BIqIImotZXslSvJfeghSE2lcuFC6q67TkOui0hcqYAkmJRduygoKiJtxw4aLr2Uyvnz8R5zjNOxRKQXUgFJFM3N5CxbRu7SpXizs6lYtoz6sWN11CEijlEBSQCp775LwR13kLpzJ/VXXEHVvHl4E3SmNBHpOVRA3KyhgdwlS8h57DG8hYWUr1pFw/e/73QqERFABcS10rZvJ7+oiNSPPqJ23DiqZ83CV1DgdCwRkS84XkCMMX2BZ4FhwB7AWGsrgtp8E3gMyANagfuttc/GN2l8eGpryX3wQbJXraJ1yBDKfvc7Gr/7XadjiYgcxQ1zok8H/mCtHQH8IfA4WB3wY2vtKcAY4GFjTI/7czxt61b6X3IJOb/9LbXjx1OyebOKh4i4luNHIMBY4MLA108CfwTuatvAWvthm6/3G2MOAf2ByvhEjC1PVRV58+aRvX49LSeeSOmLL9I0apTTsUREOuWGI5CB1toDAIH/B3TW2BgzCkgDPopDtphL37SJARdfTNazz1IzcSKHNm1S8RCRhBCXIxBjzGvAoHaemhnmeo4BngJ+Yq31dtBmAjABwFpLoYtud01JSfkyT0kJyUVFJD/7LN5TT6XlhRdIP/NM3Dpbx1eyJ6BEzq/szlD2rnl8Pl/MN9IZY8wu4EJr7YFAgfijtfbkdtrl4T+9Nd9a+1yIq/ft378/emG7qbCwkNKSEjI2biT/nntIqqmhZvJkDk+cCGlpTsfrVGFhIaWlpU7HiFgi51d2Z/Tm7IMHDwbo8lPKbjiFtRH4SeDrnwDFwQ2MMWnAi8DaMIqH++zfT5+bbqLvz35G63HHUfL733N4yhTXFw8Rkfa44SL6AsAaY24GPgWuATDGnAXcbq29BTDABUA/Y8z4wPeNt9b+nwN5w+fzkfnss6TOnUtqYyNVs2ZRe8stkOKG7hcRiYzjp7BizPFTWMmffUb+tGlkbN2K9/zzKXngAVpPPNHRTJFI5MN5SOz8yu6M3pw9kU5h9UxeL9mrVtH/4otJe+stKh94gJZNmxKyeIiItEfnUGIgefduCu68k/S//IWGiy6i6sEHaR0yhKwk1WsR6TlUQKKppYWcJ54gd/FifJmZVCxZQv0112jIdRHpkVRAoiRl507/RE/vvEP9ZZdRdf/9eAd0+plIEZGEpgLSXY2N5D7yCDmPPoq3oIDyJ56g4fLLnU4lIhJzKiDdkLpjBwVFRaTu2kXdv/4rVffdh69vX6djiYjEhQpIBDz19eQuWkT28uV4Bw6kbO1aGi+5xOlYIiJxpQISprQ33qCgqIiUPXuovf56qu+5B19urtOxRETiTgUkRJ7Dh8m7/36y166l5fjjKbWWpu98x+lYIiKOUQEJQfqWLeRPm0bygQMcvvVWaqZNw5eV5XQsERFHqYB0wlNRQf5995H13HM0jxhBaXExzWee6XQsERFXUAHpQOpbb9H35ptJqqigZvJkaiZPhnS3ztYhIhJ/KiAdaB02jOaRI6meOZOWU091Oo6IiOuogHTA268f5evXOx1DRMS1NLqfiIhERAVEREQi4vgpLGNMX+BZYBiwBzDW2ooO2uYB7wMvWmsnxSujiIgczQ1HINOBP1hrRwB/CDzuyDzgv+OSSkREOuWGAjIWeDLw9ZPAle01MsacCQwENsUpl4iIdMLxU1jAQGvtAQBr7QFjzFGTaBhjkoDFwA1Ap6MWGmMmABMC66OwsDD6iSOUkpLiqjzhSOTskNj5ld0Zyh7CdmK+BcAY8xowqJ2nZoa4ip8Br1hrPzPGdNrQWrscWB546OvOxPLR1t2J7p2UyNkhsfMruzN6c/bBgweH1C4uBcRa+72OnjPGfG6MOSZw9HEMcKidZucA5xtjfgbkAGnGmMPW2s6ul4iISAy54RTWRuAnwILA/8XBDay1PzrytTFmPHBWqMUj1EoaL27LE45Ezg6JnV/ZnaHsnXPDRfQFwKXGmL8BlwYeY4w5yxizspvr9rjpnzHmLacz9MbsiZ5f2ZXdoexdcvwIxFpbRjsXxq21bwK3tLN8DbAm5sFERKRTbjgCERGRBKQCEl/Lu27iWomcHRI7v7I7Q9m74PH5fPHYjoiI9DA6AhERkYg4fhG9JzLGjAGWAsnASmvtgqDnxwMLgX2BRY9aa7t7x1lUGGNWAZcDh6y1R82kZYzx4H9tlwF1wHhr7dvxTdm+ELJfiP828U8Ci16w1s6NX8KOGWOOBdbi/8CtF1hurV0a1MaVfR9i9gtxYd8bYzKArUA6/v3hBmvtvUFt0vG/vjOBMuBaa+2eOEc9SojZxxPDfY2OQKLMGJMMLAP+CfgGcJ0x5hvtNH3WWvvNwD9XFI+ANcCYTp7/J2BE4N8E4LE4ZArVGjrPDvB6m353fAfWRgtQZK0dCfwjMLGd941b+z6U7ODOvm8ELrbW/gPwTWCMMeYfg9rcDFRYa4cDS4AH45yxI6Fkhxjua1RAom8UsNta+7G1tgl4Bv+AkQnBWrsVKO+kyVhgrbXWZ619AygIjCDguBCyu5a19sCRowlrbQ3+aQuGBDVzZd+HmN2VAn15OPAwNfAv+MJw2wFfNwCXBI4GHRVi9pjSKazoGwJ81ubxXuDb7bS7yhhzAfAhMMVa+1k7bdyovdc3BDjgTJywnWOM+SuwH5hqrX3P6UDBjDHDgDOA/w16yvV930l2cGnfB84avAUMB5ZZazvsd2ttizGmCugHOD5QVgjZIYb7Gh2BRF97f5kE/1XwH8Awa+3pwGt8+ddNIgjl9bnV28DxgUP+XwP/7nCeoxhjcoDngX+z1lYHPe3qvu8iu2v73lrbaq39JjAUGGWMCb5+5tp+DyF7TPc1KiDRtxc4ts3jofj/4vqCtbbMWtsYeLgC/8W5RNHl63Mra231kUN+a+0rQKoxxjXjdRtjUvHvgNdZa19op4lr+76r7G7vewBrbSXwR46+jvZFvxtjUoB8XHaqtKPssd7XqIBE33ZghDHmBGNMGjAO/4CRXwg6b30F/nPGiWIj8GNjjCdwwa7qyHwubmeMGXTk3LUxZhT+93+Zs6n8Arl+C7xvrf1VB81c2fehZHdr3xtj+htjCgJfZwLfAz4IanZkwFeAq4HN1lrHj0BCyR7rfY2ugURZ4BzpJOBV/LfxrrLWvmeMmQu8aa3dCPzCGHMF/rtXyoHxjgUOYoxZD1wIFBpj9gL34r84h7X2ceAV/LeR7sZ/K+mNziQ9WgjZrwZ+aoxpAeqBcW7YEQR8B/+Eae8aY/4vsOxu4Dhwfd+Hkt2tfX8M8GTgWkISYK21LwX9vv4WeMoYsxv/7+s45+J+RSjZY7qv0SfRRUQkIjqFJSIiEVEBERGRiKiAiIhIRFRAREQkIiogIiISERUQEYcZY8YbY/7H6Rwi4VIBERGRiKiAiIhIRPRJdJE4McZMB24FBuAf3XWmtfZFZ1OJRE4FRCR+PgLOBw4C1wBPG2OGOxtJJHIqICJxYq19rs3DZ40xM/BPQCaSkFRAROLEGPNj4A5gWGBRDlAItDqVSaQ7dBFdJA6MMcfjn49hEtDPWlsA/D/an6xIJCGogIjERzb+WexKAIwxNwLBs8eJJBQVEJE4sNbuBBYD24DPgdOAPzkaSqSbNB+IiIhEREcgIiISERUQERGJiAqIiIhERAVEREQiogIiIiIRUQEREZGIqICIiEhEVEBERCQiKiAiIhKR/w/iucQE48TBngAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf6a2947b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot that includes the regression line\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If **al=3**, what class do we predict for household? **1**\n",
    "\n",
    "If **al=1.5**, what class do we predict for household? **0**\n",
    "\n",
    "We predict the 0 class for **lower** values of al, and the 1 class for **higher** values of al. What's our cutoff value? Around **al=2**, because that's where the linear regression line crosses the midpoint between predicting class 0 and class 1.\n",
    "\n",
    "Therefore, we'll say that if **household_pred >= 0.5**, we predict a class of **1**, else we predict a class of **0**.\n",
    "\n",
    "## $$h_\\beta(x) = \\beta_0 + \\beta_1x_1 + \\beta_2x_2 + ... + \\beta_nx_n$$\n",
    "\n",
    "- $h_\\beta(x)$ is the response\n",
    "- $\\beta_0$ is the intercept\n",
    "- $\\beta_1$ is the coefficient for $x_1$ (the first feature)\n",
    "- $\\beta_n$ is the coefficient for $x_n$ (the nth feature)\n",
    "\n",
    "### if $h_\\beta(x)\\le 0.5$ then $\\hat y = 0$ \n",
    "\n",
    "### if $h_\\beta(x)> 0.5$ then $\\hat y = 1$ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['small', 'big', 'small'], dtype='<U5')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# understanding np.where\n",
    "import numpy as np\n",
    "nums = np.array([5, 15, 8])\n",
    "\n",
    "# np.where returns the first value if the condition is True, and the second value if the condition is False\n",
    "np.where(nums > 10, 'big', 'small')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ri</th>\n",
       "      <th>na</th>\n",
       "      <th>mg</th>\n",
       "      <th>al</th>\n",
       "      <th>si</th>\n",
       "      <th>k</th>\n",
       "      <th>ca</th>\n",
       "      <th>ba</th>\n",
       "      <th>fe</th>\n",
       "      <th>glass_type</th>\n",
       "      <th>ri_pred</th>\n",
       "      <th>household</th>\n",
       "      <th>household_pred</th>\n",
       "      <th>household_pred_class</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.51966</td>\n",
       "      <td>14.77</td>\n",
       "      <td>3.75</td>\n",
       "      <td>0.29</td>\n",
       "      <td>72.02</td>\n",
       "      <td>0.03</td>\n",
       "      <td>9.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.521227</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.340495</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>1.51115</td>\n",
       "      <td>17.38</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.34</td>\n",
       "      <td>75.41</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6.65</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>1.521103</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.315436</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.250283</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>1.52213</td>\n",
       "      <td>14.21</td>\n",
       "      <td>3.82</td>\n",
       "      <td>0.47</td>\n",
       "      <td>71.77</td>\n",
       "      <td>0.11</td>\n",
       "      <td>9.57</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520781</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.250283</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>1.52320</td>\n",
       "      <td>13.72</td>\n",
       "      <td>3.72</td>\n",
       "      <td>0.51</td>\n",
       "      <td>71.75</td>\n",
       "      <td>0.09</td>\n",
       "      <td>10.06</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>1</td>\n",
       "      <td>1.520682</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.230236</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ri     na    mg    al     si     k     ca   ba    fe  glass_type  \\\n",
       "id                                                                           \n",
       "22   1.51966  14.77  3.75  0.29  72.02  0.03   9.00  0.0  0.00           1   \n",
       "185  1.51115  17.38  0.00  0.34  75.41  0.00   6.65  0.0  0.00           6   \n",
       "40   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "39   1.52213  14.21  3.82  0.47  71.77  0.11   9.57  0.0  0.00           1   \n",
       "51   1.52320  13.72  3.72  0.51  71.75  0.09  10.06  0.0  0.16           1   \n",
       "\n",
       "      ri_pred  household  household_pred  household_pred_class  \n",
       "id                                                              \n",
       "22   1.521227          0       -0.340495                     0  \n",
       "185  1.521103          1       -0.315436                     0  \n",
       "40   1.520781          0       -0.250283                     0  \n",
       "39   1.520781          0       -0.250283                     0  \n",
       "51   1.520682          0       -0.230236                     0  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transform household_pred to 1 or 0\n",
    "glass['household_pred_class'] = np.where(glass.household_pred >= 0.5, 1, 0)\n",
    "glass.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAENCAYAAAAVPvJNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHqdJREFUeJzt3X20XHV97/H3zjkJeSDkaU4ICRFQUiuCD9UG1BZpoa7g9ZLFuvglgFAQzLUtWJ7aqvWiK3fZy9WlmCtcbYpcxHLFr5RbkBuL1+UDLVULWpXyoGAKEvJAThKSkAQyc87cP/ack8lkn5m9T2Z+M7+Tz2uts3Jmz2/2/szOPvOd7957ZifVahUREZFGk7odQEREepMKhIiIZFKBEBGRTCoQIiKSSQVCREQyqUCIiEgmFQgREcmkAiEiIplUIEREJFN/twMcIn0MXERkfJJWA2IvEGzYsKHbEUaVSiUGBwe7HWNclL07Ys4Ocec/nLMvXLgw1zjtYhIRkUwqECIikkkFQkREMqlAiIhIJhUIERHJpAIhIiKZVCBERCRTkM9BmNltwHuAF9z95Iz7E2A18G5gD3Cpu/8kRDYREckW6oNytwM3A3eMcf/ZwJLaz6nAF2r/BjO8ZRPceyfVF7eRzJ4Lyy9i0sCCnptnrxp68lG4fTXs2Q3TZ8C5F5M8+kjmc8+zXvKuu+Etm9jxlZsZ2rwRpk5LJ256HrZshOFhmDQJjl5E8qpXU/2dd5H807eovrARtg1CpZze/+rXkpx/xQHzH13+Cxth54tw1Bw4ahbs2gHPPA3DQ5AkMHU6HPea9EHPP5v+2zC/+ufC1Gl18xhm86QEjjwK5h1NMn9B7oxZ6wc4cDmQZq/lT+YvGF2PjY8fXe4YtxvX/8jjt+3exVBf7WXk5b25tvPhLZuofu1WWPeLg9ZXq+fVLEvRv7PKpg0MffFTmTl6Vf16H54xs+OvKUm1GubbKszseOD+MTqIvwa+5+5frd3+BXCGu29sMdtqOz5JPbxlE9WbboAtm/ZPHFhAcs2qQiu//tON7ZpnKIfyycyhJx+Fm25IXzTHUnvuQMv1knfdZY5rZlLf2BnnDpBc/8nRF6lC820yP8h4voeYMXOec0ppwdq2pfn8BxbAJVfBHZ8/8PGNy228Xbf+W66fJtv58JZNVD/9UdjesK3NHYDLrj44V9bzapUlx9/Z8JZN8JmPUd36wkE5RraDXtPO15TaJ6mj+aqNRcBzdbfX16a1KhDtce+dB2/stUrNFdf1zjx71e2rmxcH2P/cR37Pum9kveRdd1njmmmWcduW/fMvOt9m84OD5tW3dx+lHz9NUhkuPt9vnpn+u3fvOIP9G9zzXRhq8f+V9bj7HyI5ag7VndtbLH//2EZNH3vvgzlztcoy9vJz5fjmmU0f2y31mXcfO49dJx7T8deUXikQWZUss7Uxs5XASgB3p1QqHfLCt+3eRTljev/uXcwtMP/+/v7RPO2aZyj12YvavHdPvmXs3gXQcr3kXXdjjRuvkfm3a75jPd/JL+1lys69vDxvJkNTiv0JTpo9F4DhF7cdQrDJ6a6rgibNnsuUN7yV8s8fabn8kbGNmj62QK5WWcZafp4crR7bLfWZh6ZOGZ3eydeUXikQ64HFdbePBTL3Hbn7GmBN7Wa1HV+2NTxjZub0yoyZhXa7HLCLqU3zDOWQvvxr2nTY81LLYZUx1snIfUXX3Vjjxmtk/u2a75jPdzh977PjtYvYN3tGoXkmp74TgOqPvj/+YPPmQ+OulZzLnnTFdQzf+pmWyx8Z26jpYwvkapVlrOXnydHqsd0yVubxvKbE9mV99wGXmFliZqcBO3Icf2if5Rel+2brDSzYf4CsV+bZqy7903SfdTMjzz3Pesm77rLGNdMs49yB/fMvOt9m88uYV1I77ldNMhrnPBmz8s0ppfe3MrAg/f9qfHzjchtv16//Vuun2Xa+/KI0a6O5A9m5sp5Xqyx5/s6WX0Qyb352jl79G+3Ca0qQg9Rm9lXgDKAEbAY+DkwGcPcv1k5zvRlYRnqa62Xu/kiOWbflIDW054yjxnfhMZ3FdKhfH9zNs5iO+Ie7eXk8ZzENlSHJeRbTrDkwsz1nMU37yc8pPfAQm3739ZRnTU/PYiodTTKQcRbTGBkLn8U0aw7JQPvPYurfvYtypGcxza7sY2ukZzH1796VdqjjfE3Je5A62FlMHdK2AtEOh/P3y3dTbNmn3XMPc666is3f/z5zTjstquyNYlv39Q7n7HkLRK/sYhI5fJRrB2InT+5uDpEWVCBEAksqFQCq/b1yjohINhUIkdDUQUgkVCBEAlMHIbFQgRAJrVYg1EFIr1OBEAksqe1iUgchvU4FQiQ0dRASCRUIkcCScjn9FHVfi0+fi3SZCoRIaJVK2j1kfdWGSA9RgRAJLCmXdfxBoqACIRLaSAch0uNUIEQCUwchsVCBEAlNHYREQgVCJDB1EBILFQiR0CoVUIGQCKhAiASWlMtUtYtJIqACIRKaOgiJhAqESGDqICQWKhAioamDkEioQIgEpg5CYqECIRKaOgiJhAqESGBJpaIOQqKgAiESWrmsDkKioAIhEpiOQUgsVCBEAkvUQUgkVCBEQtMxCImECoRIYOogJBYqECKhqYOQSKhAiASmDkJioQIhElqloutBSBSCbaVmtgxYDfQBt7r7jQ33vwr4MjC7NubD7r42VD6RUBJdUU4iEaSDMLM+4BbgbOAk4AIzO6lh2McAd/c3AyuA/xkim0hwuqKcRCLULqalwNPuvs7d9wF3AcsbxlSBo2q/zwI2BMomEk61SjI0pA5CohDqbcwi4Lm62+uBUxvGfAL4lpldBcwAzsqakZmtBFYCuDulUqntYcerv7+/p/IUoeyB7NsHwLRZsziiVIore4aY8yt7juV0fAmpJGNateH2BcDt7v4ZM3sb8BUzO9ndh+sHufsaYM3IPAYHB9ufdpxKpRK9lKcIZQ8j2bOHY4A9+/bx0uBgVNmzxJz/cM6+cOHCXONC7WJaDyyuu30sB+9CuhxwAHf/ATAViLO8i4ylXAbQMQiJQqit9GFgiZmdADxPehD6woYxvwbOBG43s9eRFogtgfKJBJFUKukvOgYhEQjSQbh7BbgSeAB4Ip3kj5nZKjM7pzbsOuADZvYz4KvApe7euBtKJG7qICQiwbbS2mca1jZMu6Hu98eBd4TKI9IN6iAkJvoktUhI6iAkIioQIgGpg5CYqECIhKQOQiKiAiESkDoIiYkKhEhI6iAkIioQIgGpg5CYqECIhKQOQiKiAiESkDoIiYkKhEhI6iAkIioQIgGNdBBVdRASARUIkZBqHQTqICQCKhAiAY12ECoQEgEVCJGQRjoI7WKSCKhAiASkDkJiogIhEpI6CImICoRIQOogJCYqECIh6YNyEhEVCJGAEn1QTiKiAiES0kgHMWVKd3OI5KACIRJQUi5TTRLo6+t2FJGWVCBEQqpUdPxBoqECIRJQUi7r+INEQwVCJCR1EBIRFQiRgNRBSEyabqlmlquAuPtwe+KITHDqICQird7KVIBqjvnolAyRHNRBSExabakn1P3+H4DzgP8GPAscB/wF8HediSYyAVUquhaERKPpluruz478bmbXAm919xdrk35pZo8AjwBf6FxEkYkjKZd1NTmJRpGD1LOA6Q3Tptemi0ge6iAkIkW21C8D3zazzwHPAYuBD9Wmt2Rmy4DVpMcrbnX3GzPGGPAJ0uMeP3P3CwvkE+l56iAkJkU6iD8H/gdwPvBZYAVwc216U2bWB9wCnA2cBFxgZic1jFkCfAR4h7u/Hri6QDaROKiDkIjk3lJrp7J+sfZT1FLgaXdfB2BmdwHLgcfrxnwAuMXdt9eW98I4liPS09RBSExafQ7i/Xlm4u63tRiyiHS31Ij1wKkNY36jtsyHSHdDfcLd/yHP8kWiUanom1wlGq06iItzzKMKtCoQyRiPa8yyBDgDOBb4RzM7ue6sKQDMbCWwEsDdKZVKOSKG0d/f31N5ilD2MPoBpk8fzRtT9iwx51f2HMtpdqe7/16blrOe9KD2iGOBDRljfujuZeDfzewXpAXj4YZMa4A1tZvVwcHBNkU8dKVSiV7KU4Syh1Hau5fhWbPYVssbU/YsMec/nLMvXLgw17hCR8vMbA7wH0l3GT0PfGPkmEELDwNLzOyE2uNWAI1nKP09cAFwu5mVSHc5rSuST6TXJZWKjkFINHKfxWRmbwN+BXwQeAPwn4Ff1aY35e4V4ErgAeCJdJI/ZmarzOyc2rAHgK1m9jjwXeDP3H1roWcj0uvKZZ3FJNEosqV+Dvhjd79rZIKZnU966utvt3qwu68F1jZMu6Hu9ypwbe1HZEJSByExKfI5iN8AvGHa3cCJ7YsjMsGpg5CIFCkQT5EeO6j3XtLdTiKSgzoIiUmRtzJXA/eb2YdIv831eNKzjN7TgVwiE5M6CIlI7g7C3f8ZeA3p12v8GPg8cGJtuojkkFQquh6ERKPQllo7pfVvO5RFZOIrl3VFOYlG7gJR+wzDJ4E3AUfW3+fur2pzLpEJSR2ExKTIlvq/SQ9IXwfs6UwckQmsWiXRNaklIkUKxOtJv4p7uFNhRCa0SgVAHYREo8hprg8Cb+5UEJGJLqkVCHUQEotWX/e9qu7mM8ADZnYPsKl+XP0nokVkDOUyoA5C4tFqS13ccPsbwOSM6SLSgjoIiU2rr/u+LFQQkQlPHYREpujXfb8OOA842t2vNLPXAke4+887kk5kAlEHIbEp8nXf7yU9UL0IuKQ2eSbw2Q7kEpl49u0D1EFIPIqcxbQK+AN3/yAwVJv2M+CNbU8lMgGpg5DYFCkQ80kLAuy/nnSVg68tLSJZdAxCIlOkQPwYuLhh2grgX9oXR2TiUgchsSnyVuZDwLfM7HJghpk9QHoRoXd1JJnIRKMOQiJT5Ou+nwR+E7gF+Bjwv4BT3P2pDmUTmVDUQUhsin7d9x5qlx01s1cD84CXOpBLZOJRByGRKXKa61fN7O213y8DHgMer+1yEpEW1EFIbIocpD4TeKT2+7XAWcBS4MPtDiUyIamDkMgU2VKnuPs+M1sEzHX3hwDM7OjORBOZWEY6iKo6CIlEkQLxUzP7CHAc8H8BasViZyeCiUw4tQ4CdRASiSK7mC4HTgGmkZ7FBPA24M52hxKZiBJdMEgik3tLdfdfARc2TLsbuLvdoUQmpJEOQruYJBK5C4SZvX+s+9z9tvbEEZm41EFIbIpsqY1fs7EAeA3wEKACIdKKOgiJTJFdTL/XOK3WVbyurYlEJih1EBKbIgeps9xOevBaRFrRB+UkMkWOQTQWk+nA+4AXcz5+GbAa6ANudfcbxxh3HvB14Lfd/ZGsMSIxSkY+KKcCIZEo0kFUgHLdzw7go8AftXqgmfWRfsnf2cBJwAVmdlLGuJmk3xr7owK5ROKgDkIiU2Rn6AkNt3e7+2DOxy4Fnnb3dQBmdhewHHi8Ydx/BT4FXF8gl0gUknKZapJAX1+3o4jkUuQg9bMAZvYq0utSry+wnEXAc3W31wOn1g8wszcDi939fjMbs0CY2UpgZS0TpVKpQIzO6u/v76k8RSh75/VNngyTJx+QNZbsY4k5v7LnWE7egWZ2DHAX6aentwLzzOyHwAp339Di4UnGtNFLldaOb9wEXNoqh7uvAdaMzGNwMG8T03mlUoleylOEsnfeUTt3Mr2//4CssWQfS8z5D+fsCxcuzDWuyDGIL5Bek3qOux8DzAH+FfhijseuBxbX3T4WqC8qM4GTge+Z2TPAacB9ZvbWAvlEelulouMPEpUixyB+BzjG3csA7r7bzP4ceD7HYx8GlpjZCbXxK6j72g533wGM9ktm9j3gep3FJBNJUi7rMxASlSIdxHbSM5DqvZYcp7m6ewW4EngAeCKd5I+Z2SozO6dABpF4qYOQyBR5O/Mp4Ntm9iXgWeB40mMG/yXPg919LbC2YdoNY4w9o0AukSiog5DY5O4g3P1vACPdFfQeYC7pAeo1TR8oIqlKRdeCkKgUuSb1FGAJ6YfktgFHAJeZ2R0dyiYyoSTlsj5FLVEp8nbmy8AbgW8AmzoTR2QCUwchkSmytS4DTnD3XN+9JCIHUgchsSlyFtOvSXcrich4qIOQyDTdWs3s9+tu3gHca2argc3149z9Ox3IJjKhqIOQ2LR6O/OljGl/1XC7Cry6PXFEJrBKBaZM6XYKkdyaFgh3b/wGVxEZp6RSYXjGjG7HEMntUK8oJyJ5lcs6BiFRUYEQCSSpVHQMQqKiAiESijoIiYwKhEgg6iAkNioQIqGog5DIqECIBKIOQmKjAiESijoIiYwKhEggSaWi60FIVFQgREIpl3VFOYmKCoRIIOogJDYqECIhVKskuia1REYFQiSESgVAHYRERQVCJICkViDUQUhMVCBEQiiXAXUQEhcVCJEA1EFIjFQgREJQByERUoEQCUAdhMRIBUIkBHUQEiEVCJEA1EFIjFQgREJQByERUoEQCUAdhMQo2NsZM1sGrAb6gFvd/caG+68FrgAqwBbg/e7+bKh8Ih2lDkIiFKSDMLM+4BbgbOAk4AIzO6lh2L8Cb3X3NwB3A58KkU0kBHUQEqNQb2eWAk+7+zoAM7sLWA48PjLA3b9bN/6HwPsCZRPpPHUQEqFQxyAWAc/V3V5fmzaWy4FvdjSRSEDqICRGod7OJBnTqlkDzex9wFuBd45x/0pgJYC7UyqV2pXxkPX39/dUniKUvbOSadMAmFUqUa3LGkP2ZmLOr+w5ltPxJaTWA4vrbh8LbGgcZGZnAX8JvNPdX8makbuvAdbUblYHBwfbHHX8SqUSvZSnCGXvrKnbtjEX2L57N5W6rDFkbybm/Idz9oULF+YaF6pAPAwsMbMTgOeBFcCF9QPM7M3AXwPL3P2FQLlEwqgdg0DHICQiQY5BuHsFuBJ4AHgineSPmdkqMzunNuzTwJHA183sp2Z2X4hsIiEkumCQRCjY1urua4G1DdNuqPv9rFBZRIIb6SB0kFoiok9SiwSgDkJipAIhEoI6CImQCoRIAKMdhAqEREQFQiQEdRASIRUIkQASfdWGREgFQiQEfdWGREgFQiSApFymmiTQ19ftKCK5qUCIhFCpqHuQ6KhAiASQlMs6/iDRUYEQCUEdhERIBUIkAHUQEiMVCJEQ1EFIhFQgRAJQByExUoEQCaFS0bUgJDoqECIBJOWyvodJoqMCIRKCOgiJkAqESADqICRGKhAiIaiDkAipQIgEoA5CYqQCIRKCOgiJkAqESABJpaIOQqKjAiESQrmsDkKiowIhEoA6CImRCoRICOogJEIqECIBqIOQGKlAiISgDkIipAIhEoA6CImRCoRICOogJEIqECIBJJWKrgch0VGBEAmhXNYV5SQ6wd7SmNkyYDXQB9zq7jc23H8EcAfwFmArcL67P9OJLMNbNsG9d1J9cRvJ7Lmw/CImDSwoNI+hJx+F21fDnt0wfQacezE7fvkoQ5s3js4ToPq1W2HdL9IHvfq1JOdfMbqs4S2b0vufehz2vQJHTIUTXwdnLYdv37v/cQMLYMsmeGknVKvQ1wdTp8PiE9LH/XodDA/DUbPhXefCA/fAzu3p2BF9/bDk9SSX/MkBz3XkeWzeuwemTYdL/5Rk3sD+XC/vhSRJ5zU8NP6VPh7JJJiUwFDz5W5uNZ++PjhuCezYDlvrRg8cAwsXw84X05/+KbB9S7rcqdPgmMXwyt70vqPmwFGz4JWX4fln08fX/X+O/l+u+0Wad8oRMKdEMn8BLL+ocAcxuo2+sDFd/rQZsHc3HDVndJ5Ft1mRopJq/YtIh5hZH/BL4A+A9cDDwAXu/njdmD8G3uDuHzSzFcC57n5+i1lXN2zYUCjL8JZNVG+6IX3BHTGwgOSaVbn/4IaefBRuuqH5C+acUvpCsXP7gdPnDpBc/8k0/Kc/CtsHC+U/ZLPmkvzFjUwaWDDG80jgyJlpMZLW5g7AZVfDbTeN/X9ZOprFd6xl1zXXsOv66w+8q1RicPDAx2Vuo40KbrOdkpU/Fodz9oULFwIkrcaF6iCWAk+7+zoAM7sLWA48XjdmOfCJ2u93AzebWeLu7a1g994JWzYxdfOLTNm5N5321Eb49VUkb3l7rllU/9+9aefQ1Maxp19zVfrr+mdyLa+90uUnb3l7zuchzW2ER69qvh6fSt/E5O4gattoU7UOgyuuy5lTpLhQBWIR8Fzd7fXAqWONcfeKme0A5gEHlEkzWwmsrI2jVCoVCrJt9y7KwLTNL3Lk+q3773hqA3z3nwvNa9yeKtb1dGT5oZ6rAFBNEqb91m8xtWF77e/vP2gbHtlGW+nfvYu5Bbf/dsvKHwtlz7Gcji8hldXKNHYGecbg7muANSP3F22zhmfMBGD7Kcex/ZTj9i986elMuvzaXPMY+shK2PZCoeXWS5aeDkD1Xx4c9zwOxchzPdTnITVz57dcj8nS05l0+unQsL1m7mKqbaOtVGbM7PouksN5N003tWkXU0uhzmJaDyyuu30s0Pg2enSMmfUDs4BtbU+y/KL0oG+S7P+Zfwyce3F6MDPPz/uvTg/61s+j8WfuAMyae/D0efPTZZ17cTqm2Tw68TN73v7nmvk8JsHMWeFzxfozb366Hpv9X45sX0W30WYGFoyeCCHSKaE6iIeBJWZ2AvA8sAK4sGHMfcAfAj8AzgO+0/bjD5CecXLNqkM6i6nvN09h6JpVB53FNPWXj/JykbOY/uyvunoW0wHPI+KzmFoqehbTpElwRMNZTLPmpIVzrLOYRv4v689imlsiGSh+xtEB22jjWUyz5oxrniLjEeQsJgAzezfwOdLTXG9z90+a2SrgEXe/z8ymAl8B3kzaOawYOajdROGzmDrpcG5Zu0nZuyfm/Idz9rxnMQUrEB2iAtEmyt4dMWeHuPMfztnzFgh9klpERDKpQIiISCYVCBERyaQCISIimVQgREQkkwqEiIhkUoEQEZFM0X8OotsBREQiNeE/B5H00o+Z/bjbGZQ9rp+Ys8eeX9lbi71AiIhIh6hAiIhIJhWI9lrTekjPUvbuiDk7xJ1f2VuI/SC1iIh0iDoIERHJFOqCQROKmS0DVpNe2+JWd7+x4f5LgU+TXhwJ4GZ3vzVoyAxmdhvwHuAFdz854/6E9Hm9G9gDXOruPwmbMluO7GcA9wL/Xpt0j7uvCpdwbGa2GLgDWAAMA2vcfXXDmJ5c9zmzn0HvrvupwIPAEaSvd3e7+8cbxhxB+hzfAmwFznf3ZwJHPUjO7JfSwdcadRAFmVkfcAtwNnAScIGZnZQx9Gvu/qbaT9eLQ83twLIm958NLKn9rAS+ECBTXrfTPDvAP9at8554gaqpANe5++uA04A/ydhmenXd58kOvbvuXwF+393fCLwJWGZmpzWMuRzY7u4nAjcB/z1wxrHkyQ4dfK1RgShuKfC0u69z933AXcDyLmfKxd0fpPl1vpcDd7h71d1/CMw2s2PCpGsuR/ae5e4bR7oBd98FPAEsahjWk+s+Z/aeVVufL9VuTq79NB54XQ58ufb73cCZtY6uq3Jm7yjtYipuEfBc3e31wKkZ4/6TmZ0O/BK4xt2fyxjTa7Ke2yJgY3fiFPY2M/sZsAG43t0f63agRmZ2POlldX/UcFfPr/sm2aGH132t6/8xcCJwi7uPue7dvWJmO4B5QNcvN5cjO3TwtUYdRHFZ7ywaq/o3gOPd/Q3At9n/7qTX5XluveonwHG1dvzzwN93Oc9BzOxI4O+Aq919Z8PdPb3uW2Tv6XXv7kPu/ibgWGCpmTUew+rZdZ8je0dfa1QgilsPLK67fSzpu6ZR7r7V3V+p3fwb0oNfMWj53HqVu+8cacfdfS0w2cxKXY41yswmk77A3unu92QM6dl13yp7r6/7Ee7+IvA9Dj6WNbruzawfmEWP7c4cK3unX2tUIIp7GFhiZieY2RRgBXBf/YCGfcfnkO63jcF9wCVmltQOhu1w957ZxdGMmS0Y2W9sZktJt+2t3U2VquX6EvCEu392jGE9ue7zZO/xdT9gZrNrv08DzgKebBh2H/CHtd/PA77j7l3vIPJk7/RrjY5BFFTbR3kl8ADpaa63uftjZrYKeMTd7wM+ZGbnkJ4Bsg24tGuB65jZV4EzgJKZrQc+TnrgC3f/IrCW9DTLp0lPtbysO0kPliP7ecAfmVkF2Aus6IU/8pp3ABcDj5rZT2vTPgq8Cnp+3efJ3svr/hjgy7V9+ZMAd/f7G/5evwR8xcyeJv17XdG9uAfIk72jrzX6JLWIiGTSLiYREcmkAiEiIplUIEREJJMKhIiIZFKBEBGRTCoQIh1mZpea2T91O4dIUSoQIiKSSQVCREQy6ZPUIm1iZh8GPgDMJ/120L909//T3VQi46cCIdI+vwJ+F9gEvBf4WzM7sbuRRMZPBUKkTdz963U3v2ZmHyG9wJRIlFQgRNrEzC4BrgWOr006EigBQ93KJHIodJBapA3M7DjS7+O/Epjn7rOBfyP7YjQiUVCBEGmPGaRXIdsCYGaXAY1X/xKJigqESBu4++PAZ4AfAJuBU4CHuhpK5BDpehAiIpJJHYSIiGRSgRARkUwqECIikkkFQkREMqlAiIhIJhUIERHJpAIhIiKZVCBERCSTCoSIiGT6//lSBwDrn71kAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf6a20d7f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the class predictions\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_class, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$h_\\beta(x)$ can be lower 0 or higher than 1, which is countra intuitive\n",
    "\n",
    "## Using Logistic Regression Instead\n",
    "\n",
    "Logistic regression can do what we just did:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# fit a logistic regression model and store the class predictions\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "feature_cols = ['al']\n",
    "X = glass[feature_cols]\n",
    "y = glass.household\n",
    "logreg.fit(X, y)\n",
    "glass['household_pred_class'] = logreg.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAENCAYAAAAVPvJNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHqpJREFUeJzt3XvUHXV97/H3PHkgCSHktp8QEiIXSa0oKksbvPQordQVPB6yWAe/BBEaBHNsixfAtmo96MpZthxdijnC0aZIAYviV+opaFE8LvXQUrXgrZSLgClIyIU8uZuEy76cP2aeJzs78zx7Zs++/p7Pa629smf2b37z3ZN59nf/LjM7qtVqiIiINBrqdQAiItKflCBERCSVEoSIiKRSghARkVRKECIikkoJQkREUilBiIhIKiUIERFJpQQhIiKphnsdQEG6DFxEpDVRswKDniDYtGlTr0MYVyqVGB0d7XUYLVHsvTHIscNgxz+VY1+8eHGmcupiEhGRVEoQIiKSSglCRERSKUGIiEgqJQgREUmlBCEiIqmUIEREJFVXroMwsxuBtwHPuPvLU16PgHXAW4H9wGp3/2k3YhMRkXTdulDuJuA64JYJXj8bWJY8zgA+n/zbNdVtW+COW6nt2kE0dz6svJChkUV9V2e/qjzyANy0Dvbvg6NmwbkXET1wf+p7z3Jcsh676rYt7P7SdVS2boYZM+OVW56GbZuhWoWhITh2CdGLTqb2u28h+ufvUHtmM+wYhfIL8esnv4To/MsOqX98/89shj274Jh5cMwc2LsbnngcqhWIIphxFJzw4nijp5+M/22or/69MGNmXR1Vtg5FcPQxsOBYooWLMseYdnyAQ/cDcexJ/NHCRePHsXH78f1OsNx4/Me237FvL5VpycfIswcynefVbVuoffUG2PDLw45Xs/c1WSx5/87KWzZR+cInU+PoV/XHvTprdsc/U6JarTt3qzCzE4FvTtCC+GvgB+7+lWT5l8CZ7r65SbW1dlxJXd22hdq1V8O2LQdXjiwiumJtroNff3Vju+rsliJXZlYeeQCuvTr+0JxI8t6Bpscl67FLLTeZoWkTxzh/hOiDnxj/kMpV7yT1Qcr7LRhjap3zSnHC2rFt8vpHFsHF74VbPnfo9o37bVyuO/5Nj88k53l12xZqn/oI7Gw41+aPwCUfODyutPfVLJYMf2fVbVvg0x+ltv2Zw+IYOw/6TTs/U5IrqQfmVhtLgKfqljcm65oliPa449bDT/YkU3PZVf1TZ7+6ad3kyQEOvvex52mvjR2XrMcurdxkJotxx7aD9eetF5i5eQdzHt106N3BvvXm+N8DB3LVNanCdf47fP37UGny/5W23TfvJTpmHrU9O5vs/2DZRpNue8c9GeNqFsvE+88Ux7fePOm2vVIf877jF7D3lOM6/pnSLwkiLZOlNm3MbA2wBsDdKZVKhXe+Y99eXkhZP7xvL/Nz1D88PDweT7vq7Jb62PPaemB/tn3s2wvQ9LhkPXYTlWvVWP2t1Dtj+28YPvA8+4+dO75uaO58AKq7drQtxrbUOXxE3HXVwr6PfMVreOHf7m+6/7GyjSbdNkdczWKZaP9Z4mi2ba/Ux1yZceT4+k5+pvRLgtgILK1bPh5I7Tty9/XA+mSx1o6bbVVnzU5dX541O1e3yyFdTG2qs1sK3fxr5lGw/zdNi5UnOCZjr+U9dhOVa9VY/S3VW6tRPWKYHaefPL4qOuNN8Us//n/tCrE9dS5YCI1dKxn3PXTZVVRv+HTT/Y+VbTTptjniahbLRPvPEkezbXtlophb+UwZtJv13QlcbGaRmb0W2J1h/KF9Vl4Y983WG1l0cICsX+rsV6vfH/dZT2bsvWc5LlmPXVq5yUwW4/yRg/XnrRegVqNW3w4eq68TMabVOa8Uv97MyKL4/6tx+8b9Ni7XH/9m72my83zlhXGsjeaPpMeV9r6axZLl72zlhUQLFqbH0a9/oz34TOnKILWZfQU4EygBW4GPAUcAuPsXkmmu1wEriKe5XuLu92eoui2D1NCeGUeN38IHaRZT0dsH93IW0/Rv386zrcxiqrwAUcZZTHPmweyJZzHN//Y9TH/yaTa/7fW5ZjExNoupdCzRSMospglizD2Lac48opH2z2Ia3reXFwZ0FtPc8vNsH9BZTMP79sYt8hY/U7IOUndtFlOHtC1BtMNUvr98L/VD7HMvv5wjf/Yznrn33lzb9UPsRQxy/FM59qwJol+6mEQGWlQuUxvulyE9kfZQghBph0oFpjUZhxEZMEoQIu2gBCEBUoIQaQN1MUmIlCBE2kEtCAmQEoRIG0TlshKEBEcJQqQdKhV1MUlwlCBE2kFdTBIgJQiRNtAgtYRICUKkHdSCkAApQYi0gxKEBEgJQqQNIg1SS4CUIETaQdNcJUBKECJtoEFqCZEShEg7VCrxb0+IBERntEg7VCqgFoQERglCpA00SC0hUoIQaQcNUkuAlCBE2kA365MQKUGItEO1Sk0JQgKjBCHSDuWyBqklOEoQIm2gQWoJkRKESDuUy7oOQoKjM1qkqFqNSNdBSICUIESKqlYBNEgtwVGCECmqXI7/VQtCAqMEIVJQVKnET9SCkMAoQYgUlSQIdTFJaJQgRIpSF5MESglCpKBILQgJVNe+8pjZCmAdMA24wd2vaXj9RcDNwNykzIfc/a5uxSfSMrUgJFBdaUGY2TTgeuBs4FTgAjM7taHYRwF399OBVcD/7kZsIkVpkFpC1a0upuXA4+6+wd2fB24DVjaUqQHHJM/nAJu6FJtIMepikkB1q028BHiqbnkjcEZDmY8D3zGz9wKzgLPSKjKzNcAaAHenVCq1PdhWDQ8P91U8eSj2AnbuBGD2vHnMyhlHz2MvaJDjV+wZ9tPxPcSilHW1huULgJvc/dNm9jrgS2b2cnev1hdy9/XA+rE6RkdH2x9ti0qlEv0UTx6KvXXDo6MsBPbs38+zOePodexFDXL8Uzn2xYsXZyrXrS6mjcDSuuXjObwL6VLAAdz9h8AMYDDTu0wtY4PU6mKSwHSrBXEfsMzMTgKeJh6EfkdDmV8DbwZuMrOXEieIbV2KT6R1Y4PUmsUkgelKC8Ldy8DlwN3Aw/Eqf9DM1prZOUmxq4B3m9kvgK8Aq929sRtKpO/oOggJVde+8iTXNNzVsO7quucPAW/oVjwibaPrICRQupJapCBdByGhUoIQKUpdTBIoJQiRotTFJIFSghApSIPUEiolCJGidB2EBEoJQqSgSNdBSKCUIESKUheTBEoJQqQoDVJLoJQgRAoaH6Qe0p+ThEVntEhRakFIoJQgRIqqJnekV4KQwChBiBQUJS0IDVJLaJQgRIpSF5MESglCpCDdrE9CpQQhUpSug5BAKUGIFKUrqSVQShAiBWmQWkKlBCFSlG7WJ4FSghApSIPUEiolCJGiKpX4Nhu61YYERme0SFGVigaoJUhKECIFReWybtQnQdJZLVJUuawWhARJCUKkqGpVCUKCNOlZbWaZEoi7V9sTjsjgicplXQMhQWr2tacM1DLUo78OmboqFU1xlSA1SxAn1T3/z8B5wF8BTwInAH8O/H1nQhMZDFG5rAQhQZo0Qbj7k2PPzexK4DXuvitZ9aiZ3Q/cD3y+cyGK9LlKhZrGICRAeQap5wBHNaw7KlkvMnWpi0kCledrz83Ad83ss8BTwFLgfcn6psxsBbCOeLziBne/JqWMAR8nHvf4hbu/I0d8Ij2hQWoJVZ4WxJ8B/ws4H/gMsAq4Llk/KTObBlwPnA2cClxgZqc2lFkGfBh4g7u/DPhAjthEekdXUkugMp/VyVTWLySPvJYDj7v7BgAzuw1YCTxUV+bdwPXuvjPZ3zMt7Eek+zRILYFqdh3Eu7JU4u43NimyhLhbasxG4IyGMr+V7PNe4m6oj7v7t7PsX6SXIg1SS6CandUXZaijBjRLENEE2zXGsgw4Ezge+Ccze3ndrCkAzGwNsAbA3SmVShlC7I7h4eG+iicPxV5g/0NDMGNGSzH0OvaiBjl+xZ5hP5O96O6/16b9bCQe1B5zPLAppcyP3P0F4D/M7JfECeO+hpjWA+uTxdro6GibQiyuVCrRT/Hkodhbt+DAAaJqtaUYeh17UYMc/1SOffHixZnK5WoXm9k84L8Qdxk9DXxjbMygifuAZWZ2UrLdKqBxhtI/ABcAN5lZibjLaUOe+ER6Ql1MEqjMs5jM7HXAr4D3AK8A/hvwq2T9pNy9DFwO3A08HK/yB81srZmdkxS7G9huZg8B3wf+1N2353o3Ir2g6yAkUHm+9nwW+GN3v21shZmdTzz19XeabezudwF3Nay7uu55DbgyeYgMjKhcpjpzZq/DEGm7PNdB/BbgDetuB05pXzgiA0gtCAlUngTxGPHYQb23E3c7iUxZulmfhCpPF9MHgG+a2fuI7+Z6IvEso7d1IC6RwVGtapBagpS5BeHu/wK8mPj2Gj8BPgeckqwXmbrUgpBA5frak0xp/bsOxSIykHSzPglV5gSRXMPwCeBVwNH1r7n7i9ocl8jg0CC1BCpPC+LLxAPSVwH7OxOOyADS3VwlUHnO6pcR34q72qlgRAaRbtYnocozzfUe4PROBSIysMplGMrzpyQyGJrd7ntt3eITwN1m9nVgS325+iuiRaaaqFxWF5MEqdlZvbRh+RvAESnrRaauSkWzmCRIzW73fUm3AhEZWBqklkDlvd33S4HzgGPd/XIzewkw3d3/rSPRiQwADVJLqPLc7vvtxAPVS4CLk9Wzgc90IC6RwaFBaglUnrN6LfAH7v4eoJKs+wXwyrZHJTIoajUidTFJoPIkiIXECQEO/p50jcN/W1pk6qjGlwVpkFpClCdB/AS4qGHdKuBf2xeOyIApl+N/1YKQAOU5q98HfMfMLgVmmdndxD8i9JaORCYyAKJK0tuqFoQEKM/tvh8Bfhu4Hvgo8LfAae7+WIdiE+l/SQtCXUwSory3+95P8rOjZnYysAD4TQfiEhkMYy0IdTFJgPJMc/2Kmb0+eX4J8CDwUNLlJDIljXUxqQUhIcozSP1m4P7k+ZXAWcBy4EPtDkpkYIwNUitBSIDytIuPdPfnzWwJMN/d7wUws2M7E5pI/4vUxSQBy3NW/9zMPgycAPwjQJIs9nQiMJGBoEFqCVieLqZLgdOAmcSzmABeB9za7qBEBoZaEBKwzGe1u/8KeEfDutuB29sdlMig0CC1hCxzgjCzd030mrvf2J5wRAaMBqklYHnaxY232VgEvBi4F1CCkKlJXUwSsDxdTL/XuC5pVby0rRGJDBB1MUnIit7E/ibiwWuRqUldTBKwPGMQjcnkKOCdwK6M268A1gHTgBvc/ZoJyp0HfA34HXe/P62MSL/QdRASsjwtiDLwQt1jN/AR4I+abWhm04hv8nc2cCpwgZmdmlJuNvFdY3+cIy6R3tF1EBKwPF97TmpY3ufuoxm3XQ487u4bAMzsNmAl8FBDuf8BfBL4YI64RHpHLQgJWJ5B6icBzOxFxL9LvTHHfpYAT9UtbwTOqC9gZqcDS939m2Y2YYIwszXAmiQmSqVSjjA6a3h4uK/iyUOxtyaaNQuAOQsWUGshhkE+7jDY8Sv2DPvJWtDMjgNuI756ejuwwMx+BKxy901NNo9S1o3/VGkyvnEtsLpZHO6+Hlg/VsfoaNZGTOeVSiX6KZ48FHtrpu/YwQJg1969vNBCDIN83GGw45/KsS9evDhTuTxjEJ8n/k3qee5+HDAP+BnwhQzbbgSW1i0fD9QnldnAy4EfmNkTwGuBO83sNTniE+k6DVJLyPKc1b8LHOfuLwC4+z4z+zPg6Qzb3gcsM7OTkvKrqLtth7vvBsbbS2b2A+CDmsUkfU+D1BKwPC2IncQzkOq9hAzTXN29DFwO3A08HK/yB81srZmdkyMGkf6iFoQELM9Z/Ungu2b2ReBJ4ETiMYP/nmVjd78LuKth3dUTlD0zR1wiPTN+JfVQ0WtORfpP5rPa3f8GMOKuoLcB84kHqNdPuqFIyMaupFYLQgKU5zepjwSWEV8ktwOYDlxiZrd0KDaR/qcuJglYnrP6ZuCVwDeALZ0JR2Sw6GZ9ErI8CWIFcJK7Z7r3ksiUoJv1ScDyjKz9mrhbSUQSug5CQjbpWW1mv1+3eAtwh5mtA7bWl3P373UgNpH+p+sgJGDNvvZ8MWXdXzYs14CT2xOOyICpVuN/1YKQAE16Vrt74x1cRaRONNaC0HUQEiCd1SJF6DoICZgShEgB44PUGoOQAClBiBRRLsfdS+pikgDprBYpolpV60GCpQQhUkBULmuKqwRLCUKkiHJZA9QSLCUIkSIqFSUICZYShEgBUaWiLiYJlhKESBGVigapJVhKECIFROWyEoQESwlCpIhymZrGICRQShAiReg6CAmYEoRIAboOQkKmBCFShKa5SsCUIESK0CC1BEwJQqSAqFLRILUESwlCpAhdByEBU4IQKUDXQUjIlCBEilAXkwRMCUKkCLUgJGBKECIF6GZ9ErKutY3NbAWwDpgG3ODu1zS8fiVwGVAGtgHvcvcnuxWfSEt0HYQErCstCDObBlwPnA2cClxgZqc2FPsZ8Bp3fwVwO/DJbsQmUoQGqSVk3frqsxx43N03AJjZbcBK4KGxAu7+/bryPwLe2aXYRFqnQWoJWLfGIJYAT9Utb0zWTeRS4FsdjUikHSoVGNJQnoSpW199opR1tbSCZvZO4DXAmyZ4fQ2wBsDdKZVK7YqxsOHh4b6KJw/F3uK+azWGZs1qef+DfNxhsONX7Bn20/E9xDYCS+uWjwc2NRYys7OAvwDe5O7PpVXk7uuB9clibXR0tM2htq5UKtFP8eSh2Fuz8Pnneb5cZleL+x/k4w6DHf9Ujn3x4sWZynUrQdwHLDOzk4CngVXAO+oLmNnpwF8DK9z9mS7FJVJMuaxZTBKsrnSeunsZuBy4G3g4XuUPmtlaMzsnKfYp4Gjga2b2czO7sxuxiRQRVasapJZgde3Mdve7gLsa1l1d9/ysbsUi0jblsgapJVg6s0UKiNTFJAFTghApQrfakIApQYgUoRaEBEwJQqSAqFpVC0KCpQQhUoTuxSQBU4IQaVWtRqS7uUrAlCBEWlWpAKiLSYKlBCHSqiRBqItJQqUEIdKiaCxBqItJAqUEIdKqchlQF5OESwlCpFVJglALQkKlBCHSoqhaBdSCkHApQYi0aqwFoQQhgVKCEGmRBqkldEoQIq3SILUETglCpFUapJbAKUGItEiD1BI6JQiRVmmQWgKnBCHSKg1SS+CUIERaFGmQWgKnBCHSKt2sTwKnBCHSIl0HIaFTghBplbqYJHBKECKt0nUQEjglCJEWjV8HMaQ/IwmTzmyRVqkFIYFTghBpkQapJXRKECKt0iC1BE4JQqRVutWGBE4JQqRFulmfhK5rnadmtgJYB0wDbnD3axpenw7cArwa2A6c7+5PdCKW6rYtcMet1HbtIJo7H1ZeyNDIolx1VB55AG5aB/v3wVGz4NyL2P3oA1S2bh6vE6D21Rtgwy/jjU5+CdH5l43vq7ptS/z6Yw/B88/B9BlwykvhrJXw3TsObjeyCLZtgd/sgVot/sY64yhYelK83a83QLUKx8yFt5wLd38d9uyMy46ZNgzLXkZ08Z8c8l7H3sfWA/th5lGw+v1EC0YOxvXsAYiiuK5qpfWD3opoCIaig1csT2Brs3qmTYMTlsHunbC9rvTIcbB4KezZFT+Gj4Sd2+L9zpgJxy2F5w7Erx0zD46ZA889C08/CUB1X1LP8PDB/8sNv4zjPXI6zCsRLVzU0vk1fo4+szne/8xZcGAfHDOv5TpF8opq9R8iHWJm04BHgT8ANgL3ARe4+0N1Zf4YeIW7v8fMVgHnuvv5Taqubdq0KVcs1W1bqF17dfyBO2ZkEdEVazP/wVUeeQCuvXryD8x5pfiDYs/OQ9fPHyH64Cfi4D/1Edg5miv+wubMJ/rzaxgaWTTB+4jg6NlxMpJJzXpqlPkPPMmmL99M5R+/PPH/5STnV6lUYnT00O1Sz9EcdXZTWvyDYirHvnjxYoCoWblutSCWA4+7+wYAM7sNWAk8VFdmJfDx5PntwHVmFrl7ezPYHbfCti3M2LqLI/cciNc9thl+/V6iV78+UxW1/3tH3HKY1OaJ11/x3vjpxicy7a+94v1Hr359xvchEzlyd3Lsbv9beHbvxAWT1gCXXZWt4uQcnVTeOkVa0K0EsQR4qm55I3DGRGXcvWxmu4EFwCFp0szWAGuScpRKpVyB7Ni3lxeAmVt3cfTG7QdfeGwTfP9fctXVssfytXo6sv9uvdfAVY4cplotNy03vG8v81PO1eHh4cPO4bFztNU6uykt/kGh2DPsp+N7iKU1ZRpbBlnK4O7rgfVjr+dtZlVnzQZg52knsPO0Ew7ufPkbGbr0ykx1VD68BnY8k2u/9aLlbwSg9q/3tFxHEWPvtej7kMTs2bD9wKRFyrNmp3YJpHYxJedoMxPV2U1TuZuml9rUxdRUt2YxbQSW1i0fDzR+jR4vY2bDwBxgR9sjWXlhPOgbRQcfC4+Dcy+KBzOzPN71gXjQt76Oxsf8EZgz//D1CxbG+zr3orjMZHV04jF3wcH3mvo+hmD2nO7HNaiPBQth9fvjMaeJjCwan7SQ6xydTN46RVrQrRbEfcAyMzsJeBpYBbyjocydwB8CPwTOA77X9vEHYGhkEdUr1haaxTTtt0+jcsXaw2YxzXj0AZ7NM4vpT/+yp7OYDnkfAzyLqam8s5iGhmB6wyymOfPixFk3i6n+/3P8/7J+FtP8EtFI/hlHh5yjjbOY5sxrqU6RVnRlFhOAmb0V+CzxNNcb3f0TZrYWuN/d7zSzGcCXgNOJWw6rxga1J5F7FlMnTeUmay8p9t4Z5PincuxZZzF1LUF0iBJEmyj23hjk2GGw45/KsWdNELqSWkREUilBiIhIKiUIERFJpQQhIiKplCBERCSVEoSIiKRSghARkVQDfx1ErwMQERlQwV8HEfXTw8x+0usYFPtgPQY59kGPX7E3N+gJQkREOkQJQkREUilBtNf65kX6lmLvjUGOHQY7fsXexKAPUouISIeoBSEiIqm69YNBQTGzFcA64t+2uMHdr2l4fTXwKeIfRwK4zt1v6GqQKczsRuBtwDPu/vKU1yPi9/VWYD+w2t1/2t0o02WI/UzgDuA/klVfd/e13YtwYma2FLgFWARUgfXuvq6hTF8e+4yxn0n/HvsZwD3AdOLPu9vd/WMNZaYTv8dXA9uB8939iS6HepiMsa+mg581akHkZGbTgOuBs4FTgQvM7NSUol9191clj54nh8RNwIpJXj8bWJY81gCf70JMWd3E5LED/FPdMe+LD6hEGbjK3V8KvBb4k5Rzpl+PfZbYoX+P/XPA77v7K4FXASvM7LUNZS4Fdrr7KcC1wP/scowTyRI7dPCzRgkiv+XA4+6+wd2fB24DVvY4pkzc/R4m/53vlcAt7l5z9x8Bc83suO5EN7kMsfctd9881hpw973Aw8CShmJ9eewzxt63kuP5m2TxiOTROPC6Erg5eX478OakRddTGWPvKHUx5bcEeKpueSNwRkq5/2pmbwQeBa5w96dSyvSbtPe2BNjcm3Bye52Z/QLYBHzQ3R/sdUCNzOxE4p/V/XHDS31/7CeJHfr42Cet/p8ApwDXu/uEx97dy2a2G1gA9Pzn5jLEDh38rFELIr+0bxaNWf0bwInu/grguxz8dtLvsry3fvVT4ISkOf454B96HM9hzOxo4O+BD7j7noaX+/rYN4m9r4+9u1fc/VXA8cByM2scw+rbY58h9o5+1ihB5LcRWFq3fDzxt6Zx7r7d3Z9LFv+GePBrEDR9b/3K3feMNcfd/S7gCDMr9TiscWZ2BPEH7K3u/vWUIn177JvF3u/Hfoy77wJ+wOFjWePH3syGgTn0WXfmRLF3+rNGCSK/+4BlZnaSmR0JrALurC/Q0Hd8DnG/7SC4E7jYzKJkMGy3u/dNF8dkzGzRWL+xmS0nPre39zaqWBLXF4GH3f0zExTry2OfJfY+P/YjZjY3eT4TOAt4pKHYncAfJs/PA77n7j1vQWSJvdOfNRqDyCnpo7wcuJt4muuN7v6gma0F7nf3O4H3mdk5xDNAdgCrexZwHTP7CnAmUDKzjcDHiAe+cPcvAHcRT7N8nHiq5SW9ifRwGWI/D/gjMysDB4BV/fBHnngDcBHwgJn9PFn3EeBF0PfHPkvs/XzsjwNuTvryhwB39282/L1+EfiSmT1O/Pe6qnfhHiJL7B39rNGV1CIikkpdTCIikkoJQkREUilBiIhIKiUIERFJpQQhIiKplCBEOszMVpvZP/c6DpG8lCBERCSVEoSIiKTSldQibWJmHwLeDSwkvjvoX7j7/+ltVCKtU4IQaZ9fAf8J2AK8Hfg7MzultyGJtE4JQqRN3P1rdYtfNbMPE//AlMhAUoIQaRMzuxi4EjgxWXU0UAIqvYpJpAgNUou0gZmdQHw//suBBe4+F/h30n+MRmQgKEGItMcs4l8h2wZgZpcAjb/+JTJQlCBE2sDdHwI+DfwQ2AqcBtzb06BECtLvQYiISCq1IEREJJUShIiIpFKCEBGRVEoQIiKSSglCRERSKUGIiEgqJQgREUmlBCEiIqmUIEREJNX/B1whAUke3p8NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf6a2ac438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the class predictions\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_class, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we wanted the **predicted probabilities** instead of just the **class predictions**, to understand how confident we are in a given prediction?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# store the predicted probabilites of class 1\n",
    "glass['household_pred_prob'] = logreg.predict_proba(X)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAENCAYAAAAVPvJNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X2cVGX9//HXmZu932VZBkQQEdIUb8tUspuvfPtWSr9y1exSKhXS8F7TtDRTDLVMUUPBGwRDzdTLuwS1TEorJQ2z8gbNO7xBWGDZZdn73Zk5vz9mdl2GYXcGZufM7Lyfj8c8ds6Za855z7Wz89nrnDPnOK7rIiIiksjndQAREclNKhAiIpKUCoSIiCSlAiEiIkmpQIiISFIqECIikpQKhIiIJKUCISIiSalAiIhIUgGvA+wgfQ1cRGT7OAM1yPcCwZo1a7yO0CsUClFfX+91jO2i7N7I5+yQ3/kLOfuYMWNSaqdNTCIikpQKhIiIJKUCISIiSalAiIhIUioQIiKSlAqEiIgkpQIhIiJJZeV7EMaYO4CvA+uttfsmedwB5gJfA9qA6dbal7KRTUREksvWF+UWA/OAu7bx+FRgj/htMnBL/GfWRDfUwaP34G5qwKmugdrv4Bs5OueWmasib7wCi+dCWyuUlcPRJ+C88mLS155Kv6Tad9ENdTTdPY/IurVQUhqbWfcRbFgL0Sj4fLDTWJxdJ+J+4as4z/4Rd/1aaKiHcHfs8Yl74hx3yhbL713/+rWweRNUDYeqYdDcBO+9DdEIOA6UlMH4T8Se9NH7sZ8Jy+v7Wigp7bOMKOt8DlRUwYidcEaNTjljsv4BtlwPxLLH8zujRvf2Y+Lze9e7jenE/u95fkNrMxF//GOkoz2l93l0Qx3u/Qvh3f9u1V8Dva7+sqT7dxauW0Pk1muS5shVffs9Wl456J8pjutm52wVxpjdgMe2MYK4DXjGWntvfPq/wBRr7doBFutm4pvU0Q11uDdcBhvqPp45cjTOebPT6vy+327M1DKzZUe+mRl54xW44bLYh+a2xF87MGC/pNp3Sdv1x+ffdsaakTgXXNX7IZXWcvtZHiR5vTuYMekyh4diBathQ//LHzkaTjwb7rppy+cnrjdxuk//D9g//bzPoxvqcK/9CTQmvNdqRsKMH2ydK9nrGihLCn9n0Q11cN1PcTeu3ypHz/sg12TyMyX+Teq8OdXGWODDPtOr4/MGKhCZ8eg9W7/Z45WaU36YO8vMVYvn9l8c4OPX3nM/2WM9/ZJq3yVr15/+MjZs+Hj56S63v+VB5jMmW2biB+62bKiL/b4SPxgT15s43bf/B+qf/t7nj96TPGvDhuS5krUdKEsqf2eP3rN1cejJsb1/o64L4TBOVxd0deF0deF0d0NnJ053d+rze+53dUF3N068nfvyizjr1uBEXdpHVtG2S2jQP1NypUAkq2RJhzbGmJnATABrLaFQaIdX3tDaTHeS+YHWZmrSWH4gEOjNk6llZkvf7Ola196W2jpamwEG7JdU+25b7bZXz/Iztdz+Xq+ny0zx95Vs3an2z7be5/0+N41cA2VJuv5wGBoacOrr2fz6m7hrG/F1hfF3hfF1h3GiUYi6+N9roujl93E6OyH+4U1XF8Sn+52fYW5RERQVQXEx0Y523GgY1+fQVVna/2vNkFwpEKuBcX2mdwGSbjuy1i4AFsQn3UycbCtaXpl0fri8Mq3NLltsYsrQMrNlh07+VVoGbS0DNgtvo096Hku377bVbnv1LD9Ty+3v9Xq6zBR/X8nWnWr/bOt93u9z08jVm6W0An9HF/62TgLtXR9/4K/rIvJyLb6NG/E1NODfuBGnqQknvkl9RGIuvw/X78P1OdAaJtr1Hygqwo3fCAZxhw3DDQZj84NB3OLiLe8Hgx+3H2h+z/34NMXFW80nGIxtXuvJuPA63Bf+knJf9yfVk/XlSoFYApxljLmP2M7pphT2P2RO7XdiO6oStu317iDLlWXmqunnprQPove1D9QvqfZdsnb9GWD7fu/y013uQMsbjIyJy8zCPoiU+6e/93ntd+DN15Lvg5h+bvJ9EK5L4MPVBNo6YzfXT2DzCwTuPozABx/ENsX04ToO0RHdRFu7idbUEJ40ia4RI4iOGEGkpoZoTQ0Rv0N06W8ItzUTLfLHDgSI58jVfRBefKZkZSe1MeZeYAoQAtYBs4AggLX21vhhrvOAI4gd5jrDWvtiCovOyE5qyMwRR4n/hefTUUw7evpgL49iKv7Dg3Rsz1FMkW5wUjyKadhwqMz8UUz0HMUU2glnZJKjmLaRMe2jmIYNxxmZ+aOYAq3NdGfwKCb33bcouv0mgu+somhzG4HOCMFV78W2zfc8v6Kc8G4TiOy6K5Hx4+muqSb8wX8J+1yiO++Me9z38I3aeZvr71Ed7mJjnh7FFGhtjo0mt/MzJdWd1Fk7immQZKxAZEIhn1/eS8runR3KH40SeOMNil56ieC//kXRSy8ReOut3s1A4TFjCO+1F92TJhHec0/Cu+9OeNddcaurt9j04kl2j2XoehB5cxSTiBQAp6WFkmXLKHn8cYqXL8e3aRMAkZoaug88kPbaWroOPJDuAw7AHTbM47SiAiEig8ppa6P4qacofewxSv78Z5yODiI77UTH4YfTeeihdB18MJHx4zMyKpDMUoEQkYxz2tsp/tOfKF26lOJly/B1dBAZNYrWadPoOPJIug466OMdw5KzVCBEJDM6Oih5+mlKli6l5Kmn8LW1EQmFaDeG9iOPpOuQQ8Dv9zqlpEEFQkR2iP+dd6hYuJDShx/G19JCpKaG9mOOof0b36Drs5+FgD5m8pV+cyKyXQKvvkrgxhvZ6fHHcYuKaK+tpf2YY+j83OdUFIYI/RZFJC2+NWuo+uUvKX3oIRg+nObzzqP1pJOIjhzpdTTJMBUIEUmJ09JCxfz5VCxYAK5Ly+mnUzxrFs3hsNfRZJCoQIhI/8Jhyn77Wyqvuw5/fT1tRx9N849/TGTcOIqrqyFPv2wmA1OBEJHkXJfiZcuouuoqgm+9RefkyTTceSfdn/qU18kkS1QgRGQrvrVrqb7gAkqeeYbwxIk03HEHHV/9qr7MVmBUIERkCyWPPkr1T34CnZ00/exntJ50UuzU01JwVCBEBACnqYlhl1xC2SOP0PXpT9N4441EJk70OpZ4SAVCRAi88QY1J5+Mf/VqNl9wAS1nn63vMogKhEihK1myhOrzz8etqqL+wQfpPvhgryNJjtDZskQKVSRC5VVXUXP66YT32YcNv/+9ioNsQSMIkULU1cXwc86hdOlSWk88kaaf/QyKirxOJTlGBUKk0HR0UHPqqZQsW0bTpZfSetppXieSHKUCIVJAnNZWambMoGj5cjZdfTVtJ5zgdSTJYSoQIgXCaWpixAknEPz3v9k0dy7t3/ym15Ekx6lAiBQA38aNjJg2jcCbb9J42210TJ3qdSTJAyoQIkOcb+1aRkybhv/DD2lYvJjOKVO8jiR5QgVCZAjz1dcTOvZYfPX1NNxzT+wKbyIpUoEQGaKctjZqTjoJX10dG++/n+6DDvI6kuQZFQiRoSgcZvgZZxB8+WUaFi1ScZDtogIhMtS4LsMuvZSSp55i01VX0fnVr3qdSPKUTrUhMsRUzJ9P+V130XzGGbRNn+51HMljKhAiQ0jJo49S9Ytf0FZbS/PFF3sdR/KcCoTIEBF4/XWqzz+fzsmT2XTDDeDTn7fsGL2DRIYAp7mZmpkzcauqaLz1Vigu9jqSDAFZ20ltjDkCmAv4gYXW2qsTHt8VuBOojre5yFr7RLbyieQt16X6ggvwv/8+G++/n+ioUV4nkiEiKyMIY4wfmA9MBfYGphlj9k5o9lPAWms/DRwP3JyNbCL5rvyOOyh97DGaL7qIrkMP9TqODCHZ2sR0CPC2tfZda20XcB9Qm9DGBari94cBa7KUTSRvBV98karZs2k//HBaTj/d6zgyxGRrE9NY4MM+06uByQltLgf+aIw5GygHvpxsQcaYmcBMAGstoVAo42G3VyAQyKk86VB2b+xQ9vp6gmeeCePG4b/rLkLV1ZkNl4KC7XuPZSt7tgqEk2SemzA9DVhsrb3OGHMocLcxZl9rbbRvI2vtAmBBzzLq6+szn3Y7hUIhcilPOpTdG9ud3XUZPnMmwfXr2bB0KeFwGDzog4Ls+xywo9nHjBmTUrtsbWJaDYzrM70LW29COhmwANbavwMlQH6Wd5FBVvq731H6xBM0X3gh4X339TqODFHZGkGsAPYwxkwAPiK2E/rbCW0+AP4PWGyMmUSsQGzIUj6RvOGrq2PYJZfQ9ZnP0HLqqV7HkSEsKyMIa20YOAt4Eng9Nsu+ZoyZbYw5Mt7sh8D3jTH/Ae4FpltrEzdDiRQ216X6wguhs5PGX/0K/H6vE8kQlrXvQcS/0/BEwrzL+txfCXw+W3lE8lHZffdR8uc/03TFFUQmTvQ6jgxx+ia1SJ7wf/ghVbNm0fm5z9Gqk/BJFqhAiOSDaJTq888HYNP11+s8S5IVuh6ESB4ofeABipcvZ9Mvf0lk3LiBnyCSAfo3RCTHOQ0NVF1xBZ0HH0zbtxMP/hMZPCoQIjmu6uc/x7d5M02/+IU2LUlW6d0mksOKVqyg/N57aZ05k/CkSV7HkQKjAiGSq7q7GXbRRYTHjKH5vPO8TiMFSDupRXJU+aJFBN94g4Y77sAtL/c6jhQgjSBEcpBv/Xoqr7+eji9/mY7DD/c6jhQoFQiRHFQ5Zw5OZydNs2Z5HUUKmAqESI4JvP46ZffeS+tJJ+l0GuIpFQiRHFN1xRW4VVXaMS2eU4EQySHFTz9NyV/+QvO55+IOH+51HClwKhAiuSIcpmr2bMK77aaT8UlO0GGuIjmi7Le/JfjmmzTcfjsUFXkdR0QjCJFc4DQ3UzlnDp2TJ9MxdarXcUQAjSBEckLFvHn4N26k4e67wXG8jiMCaAQh4jn/6tVU3H47bcccQ/cBB3gdR6SXCoSIxyp/+UtwHDZfdJHXUUS2oAIh4iHn1VcpfeQRWmfMIDp2rNdxRLagAiHiIf/ll+NWVNB8xhleRxHZigqEiEeCL72Eb+lSWk47Dbemxus4IltRgRDxSOWcObihEK3f/77XUUSSUoEQ8UBwxQpK/vIXIuedp2s9SM7q93sQxpiUCoi1NpqZOCKFofL664mMGEH09NOhvd3rOCJJDVQAwkB3CjcRSVHRihWU/PWvtJxxBmj0IDlsoG9ST+hz//8BxwK/AN4HxgM/Bh4anGgiQ1PlddcRCYVoO/FESr0OI9KPfguEtfb9nvvGmPOBg6y1m+Kz3jTGvAi8CNwyeBFFho6if/yD4r/9jaZLL8UtK/M6jki/0tlJPQxIfEeXxeeLSAp6Rw8nneR1FJEBpXOyvjuBZcaYXwEfAuOAc+LzB2SMOQKYC/iBhdbaq5O0McDlgAv8x1r77TTyieS0ohdeoPjZZ2m67DLcUm1cktyXzgjiR8CNwHHA9cDxwLz4/H4ZY/zAfGAqsDcwzRizd0KbPYCLgc9ba/cBfpBGNpGcV3nddURGjqTtxBO9jiKSkpRHEPFDWW+N39J1CPC2tfZdAGPMfUAtsLJPm+8D8621jfH1rd+O9YjkpKLnn6f4uedomjVLowfJGwN9D+J7qSzEWnvHAE3GEtss1WM1MDmhzSfj63yO2Gaoy621f0hl/SK5rvK664iMGkXrCSd4HUUkZQONIFJ5N7vAQAUi2RVQ3CRZ9gCmALsAfzPG7NvnqCkAjDEzgZkA1lpCoVAKEbMjEAjkVJ50KPvgcZYvJ7h8OeFrryU0btwWj+V69oHkc35lT2E9/T1orf3fDK1nNbGd2j12AdYkafO8tbYbWGWM+S+xgrEiIdMCYEF80q2vr89QxB0XCoXIpTzpUPbBU3PllfiGD2fDUUfhJuTM9ewDyef8hZx9zJgxKbVL65KjxpjhwDeIbTL6CFjas89gACuAPYwxE+LPOx5IPELpd8A0YLExJkRsk9O76eQTyTWBlSsp+dOf2Hzhhfreg+SdlI9iMsYcCrwDnAbsD5wKvBOf3y9rbRg4C3gSeD02y75mjJltjDky3uxJYKMxZiXwNHChtXZjWq9GJMdUzJ9PtLyc1unTvY4ikrZ0RhC/As6w1t7XM8MYcxyxQ18PHujJ1tongCcS5l3W574LnB+/ieQ9/6pVlC5ZQuupp+JWV3sdRyRt6XwP4pOATZj3ILB75uKIDB0Vt9wCwSAtut6D5Kl0CsRbxPYd9PUtYpudRKQPX10dZQ88QJsxRHfayes4ItslnU1MPwAeM8acQ+xsrrsRO8ro64OQSySvVdx+O4TDtJx+utdRRLZbyiMIa+1y4BPETq/xT+AmYPf4fBGJcxobKbv7btpra4mMH+91HJHtltZhrvFDWn8zSFlEhoTyxYvxtbbScuaZXkcR2SEpF4j4dxiuAj4FVPR9zFq7a4ZzieQlp62N8kWL6PjKVwhPmuR1HJEdks4I4rfEdkj/EGgbnDgi+a3snnvwNzbScNZZXkcR2WHpFIh9iJ2KOzpYYUTyWlcXFbfeSuehh9J90EFepxHZYekc5vpX4NODFUQk35U99BD+ujpazj7b6ygiGTHQ6b5n95l8D3jSGPMwUNe3Xd9vRIsUpEiEivnz6dpvPzr/53+8TiOSEQNtYhqXML0UCCaZL1LQSp54gsCqVTTcdhs4yc5uL5J/Bjrd94xsBRHJW65Lxbx5hCdOpGPqVK/TiGRMuqf7ngQcC+xkrT3LGLMnUGytfXlQ0onkgeJnnqHo1VdpvP568Pu9jiOSMemc7vtbxHZUjwV6rrpeCVw/CLlE8kbFvHlEdt6Z9qOP9jqKSEalcxTTbOAr1trTgEh83n+AAzKeSiRPBFesoPj552k57TQoKvI6jkhGpVMgRhErCPDx9aRdtr62tEjBqJw3j8jw4bR9O/ECiSL5L50C8U/ghIR5xwP/yFwckfwRWLmSkmXLaD3lFF1OVIakdHZSnwP80RhzMlBujHmS2EWEvjooyURynC4nKkNdOqf7fgPYC5gP/BT4NbCftfatQcomkrP8771H6ZIltJ14oi4nKkNWuqf7biN+2VFjzERgBNAyCLlEcpouJyqFIJ3DXO81xnwufn8G8BqwMr7JSaRg+Nato8xaXU5Uhrx0dlL/H/Bi/P75wJeBQ4CLMh1KJJdV3HYbRCK6nKgMeelsYiqy1nYZY8YCNdba5wCMMfoXSgqG09Cgy4lKwUinQPzbGHMxMB54HCBeLDYPRjCRXFSxaBG+tjad0lsKQjqbmE4G9gNKiR3FBHAocE+mQ4nkIqe5mfJf/5r2qVMJf/KTXscRGXQpjyCste8A306Y9yDwYKZDieSi8jvvxNfURMs553gdRSQrUi4Qxpjvbesxa+0dmYkjkpuc9nbKFyygY8oUuvff3+s4IlmRzj6IxNNsjAY+ATwHqEDIkFZ27734N26kUaMHKSDpbGL638R58VHFpIwmEsk1XV1U3HwznZMn0zV5stdpRLImnZ3UySwmtvNaZMgqe+gh/GvXat+DFJx09kEkFpMy4LvAphSffwQwF/ADC621V2+j3bHAA8DB1toXk7URyZpwmIp58+jaf386DzvM6zQiWZXOCCIMdPe5NQE/AQb8Oqkxxk/sJH9Tgb2BacaYvZO0qyR21tgX0sglMmhKH3uMwHvvxUYPjuN1HJGsSmcn9YSE6VZrbX2Kzz0EeNta+y6AMeY+oBZYmdDuCuAa4II0cokMjmiUiptuovuTn6Tj8MO9TiOSdenspH4fwBizK7HrUq9OYz1jgQ/7TK8GttjbZ4z5NDDOWvuYMWabBcIYMxOYGc9EKBRKI8bgCgQCOZUnHcq+NWfJEoJvvEH4178mNGpUxpcP+d3vkN/5lT2F9aTa0BizM3AfsW9PbwRGGGOeB4631q4Z4OnJxua9lyqN79+4AZg+UA5r7QJgQc8y6utTHcQMvlAoRC7lSYeyJ3BdQlddRXj8eNZ/6UswSH2Tz/0O+Z2/kLOPGTMmpXbp7IO4hdg1qYdba3cGhgP/Am5N4bmrgXF9pncB+haVSmBf4BljzHvAZ4ElxpiD0sgnkjHFf/oTRf/+Ny1nnQWBtC6bIjJkpPPO/wKws7W2G8Ba22qM+RHwUQrPXQHsYYyZEG9/PH1O22GtbQJ6x0vGmGeAC3QUk3jCdam8/nrCu+5K27e+5XUaEc+kM4JoJHYEUl97ksJhrtbaMHAW8CTwemyWfc0YM9sYc2QaGUQGXfFTT1H0n//QfO65EAx6HUfEM+mMIK4BlhljFgHvA7sR22dwaSpPttY+ATyRMO+ybbSdkkYukczpGT2MH0/7N7/pdRoRT6U8grDW3g4YYpuCvg7UENtBvaDfJ4rkkZI//pGiV16h+Qc/0OhBCl4616QuAvYg9iW5BqAYmGGMuWuQsolkVzRK5Zw5hCdMoP2YY7xOI+K5dDYx3QkcACwF6gYnjoh3Sv7wB4IrV9I4d66OXBIhvQJxBDDBWpvSuZdE8ko0Gtv3MHEi7Ucd5XUakZyQToH4gNhmJZEhp+SJJwi+/jqNN92k0YNIXL9/CcaYL/WZvAt41BgzF1jXt5219s+DkE0kO8JhKq+5hu7dd6e9ttbrNCI5Y6B/lRYlmffzhGkXmJiZOCLZV/bAAwTfeYeGhQvB7/c6jkjO6LdAWGsTz+AqMrS0t1M5Zw5dBx5IxxFHeJ1GJKdoY6sUtPLFi/HX1dE4b56u9yCSYEcvOSqSt5ymJirnzaPjS1+i69BDvY4jknNUIKRgVdx8M05TE5svusjrKCI5SQVCCpKvro7yhQtpP/powvvs43UckZykAiEFqeqaa3AiEZov0NVtRbZFBUIKTvDllym1ltZTTiEyfrzXcURylgqEFBbXperyy4nW1NB8zjlepxHJaSoQUlBKHnuM4hdeoPlHP8KtqvI6jkhOU4GQwtHRQdWVV9I9aRJt06Z5nUYk5+mLclIwKm6/ncDq1dRbq1NqiKRAIwgpCL66Oipuuon2I46g6/Of9zqOSF5QgZCCMOzyy3HCYTZfmtIl1EUEFQgpAMXPPEPp0qU0n302kd128zqOSN5QgZChrb2dYZdcQnjiRFrOOMPrNCJ5RTupZUirnDePwHvvUX///VCsCyKKpEMjCBmy/G+/TcXNN9N2zDF0feELXscRyTsqEDI0uS7VP/kJbkkJmy+7zOs0InlJm5hkSCr7zW8ofu45Nl19NdGRI72OI5KXNIKQIcf/wQdUzZ5N5xe/SNt3v+t1HJG8pQIhQ0s0SvUPfwg+H5vmzNFlREV2gDYxyZBSdtddFC9fzqZrryWyyy5exxHJa1krEMaYI4C5gB9YaK29OuHx84FTgDCwAfietfb9bOWT/Od//32qrrySjilTdDI+kQzIyiYmY4wfmA9MBfYGphlj9k5o9i/gIGvt/sCDwDXZyCZDRDhM9bnnQjDIpmuv1aYlkQzI1gjiEOBta+27AMaY+4BaYGVPA2vt033aPw9o76KkrHLOHIpXrKBx/nyiY8Z4HUdkSMjWTuqxwId9plfH523LycDvBzWRDBnOsmVUzJtH67RptB91lNdxRIaMbI0gko333WQNjTHfBQ4CDtvG4zOBmQDWWkKhUKYy7rBAIJBTedKRt9nr6gjMmIG7114Eb76ZUFmZ14nSkrf9HpfP+ZU9hfUM+hpiVgPj+kzvAqxJbGSM+TJwCXCYtbYz2YKstQuABfFJt76+PsNRt18oFCKX8qQjL7NHo4z47nehuZn6++4j3NYGbW1ep0pLXvZ7H/mcv5Czj0lxM2y2CsQKYA9jzATgI+B44Nt9GxhjPg3cBhxhrV2fpVySxypvuIHiv/2N8C23EN5zT6/jiAw5WdkHYa0NA2cBTwKvx2bZ14wxs40xR8abXQtUAA8YY/5tjFmSjWySn0oef5zK66+n7VvfIjpjhtdxRIYkx3WT7grIF+6aNVttqfJMIQ9Zsynw2muEamsJT5pE/QMPENpll7zJniif+j2ZfM5fyNnjm5gGPBZcp9qQvOKrr6dmxgzc6moaFi2CkhKvI4kMWTrVhuSPri6Gf//7+DdupP6RR4iOGuV1IpEhTQVC8kMkwvBzzqH4H/+g4eab6d5/f68TiQx52sQkuc91GXbJJZQuXUrTpZfSUVvrdSKRgqACITmvcs4cyu++m+Yzz6T1tNO8jiNSMFQgJKeVL1xI5a9+Reu0aTRffLHXcUQKigqE5KyyxYsZNmsW7VOn0nT11TpDq0iWqUBITiq//XaqL7mE9sMPp3H+fAjoeAqRbNNfneSciptvpuqqq2j/2tdovPlmCAa9jiRSkFQgJHdEo1RdeSUVt91GW20tm+bOVXEQ8ZAKhOSGjg6qL7iAskceoWXGDDbPng0+bQEV8ZIKhHjOt349NSefTNFLL7H5ootoOess7ZAWyQEqEOKpwKuvUjNjBr7GRhpuv52Or33N60giEqcxvHjDdSn7zW8YeeSROK5L/e9+p+IgkmM0gpCsc1paGHbxxZQ9/DAdU6aw6cYbiY4Y4XUsEUmgAiFZVfT3v1N93nn4P/qIzT/6ES1nn62d0SI5SgVCssJpa6PymmsoX7iQyPjxbHz4YboOPtjrWCLSDxUIGVyuS8nvf0/VrFkE1qyh9cQT2fzTn+KWl3udTEQGoAIhg8a/ahXDLr2UkqefpnvSJOrnzaNr8mSvY4lIilQgJON869ZReeONlN1zD25xMU2XX07rjBk6n5JIntFfrGSM09hIxS23UL5oEU53N23TptF83nlER4/2OpqIbAcVCNlh/g8+oHzhQsruvRenvZ32o4+m+fzziUyY4HU0EdkBKhCyfVyXohUrKL/jDkoefxx8PtqPOoqW008nvNdeXqcTkQxQgZC0+DZsoPTBBym7916C77xDtLKSltNOo3XGDKJjxngdT0QySAVCBuQ0NlLy5JOULl1K8bPP4oTDdB58MI1nnknHN76BW1bmdUQRGQQqELI11yXwxhsU//WvFP/lLxQ/9xxOOEx4/HhaTj2VdmMI77671ylFZJCpQEjM2rWUPvporCg8+yz+desA6P7EJ2iZOZOOb3yD7v3202m4RQqICkQBcpqaCL4Rg49vAAAK50lEQVTySu+t6OWXCaxaRREQGT6cri9+kY7DDqPzi18kOnas13FFxCMqEENZNIpv7VqC77wTKwYvv0zwlVcIvP9+b5Pw2LGxkcHJJ9N40EF077OPTp4nIoAKRH5zXZxNm/B/9BGBjz7Cv3o1/vjPwKpV+Fetwtfe3ts8vOuudO+3H23TptG9//5077tv72m2Q6EQ3fX1Xr0SEclBWSsQxpgjgLmAH1horb064fFi4C7gM8BG4Dhr7XuDkSW6oQ4evQd3UwNOdQ3UfgffyPS+7Rt54xVYPBfaWqGsHI4+gaY3XyGybm3vMgHc+xfCu/+NPWninjjHndK7ruiGutjjb62Erk4oKsbZbQ+cyV/C99QSfO++ja+zC3+wFF/9BvzNLfg6w/i74rf2Lnzd4S1fWzBIpCRIuCRAx06VdJePJFxRQnd1JdF9DsA58cwtXmvP61jX3galZTD9XJwRIz/O1dEe2+/guhCNbH+nbw/HBz4HIv2vd91Ay/H7Yfwe0NQIG/u0HrkzjBkHmzfFboEiaNwQW29JKew8DjrbY49VDYeqYdDZAR/FR2B9fp+9v8t3/xvLW1QMw0M4o0Zv1/ur9z26fm1s/aXl0N4KVcO3e5ki6XJc1x30lRhj/MCbwFeA1cAKYJq1dmWfNmcA+1trTzPGHA8cba09boBFu2vWrEkrS3RDHe4Nl8GGuo9njhyNc97s/v/gXDf2h9/dTfT1l3FuuhIiYZyoixOJ4kSj8Z/x6dIKnO5unNaW2HQkihOO4AsU4dtjP3wtLTgrX8bX0YEvHMHpjsR+9vPriPp9RIsDRIqCRIqDREqLCJcWESmJ/ywtIloU6H9H8rAanB9fjW/k6FhxuOGyhA9+ByoqoWVzyn1a0GpGwowfwB03QOM2RmD9vL9CoRD1CSO3pO/RNJaZTcny54tCzj4m9p2lAY84ydYI4hDgbWvtuwDGmPuAWmBlnza1wOXx+w8C84wxjrU2sxXs0XtgQx2Vb6+ldN0mHNeF6Gs4jy/HKa+C7m6ccBjC4djP+LTT3b3Dq3aBaNCPu24T0eIiot3dREqK6A74cQN+ogE/0WDsp9tzvyhApChAtDiI68/AvoGmhlgfnPLD2Ahoq1GBq+KQjoYNsX7cVnGA2Ad9T5+nIv4e7Ve6yxTZDtkqEGOBD/tMrwYSz/vc28ZaGzbGNAEjgC3+8owxM4GZ8XaEQqG0gjS0NtMNuD4f0WAAfA6u4+BUD6fowMkQDMbOOhoI4AaDEAxu+TMQoMX+GjfcFXuuz4frc3D9vth9fz/Tfh84DsF9DwSg+9WX0sqeKYHWZmpCodhmJdlxKfRjT59vNT8Q2Oo93PMe3d5lZlOy/PlC2VNYz6CvISbZUCZxZJBKG6y1C4AFPY+nO8yKllcC0DJxJ1om7vTxyicfhi/F/8Yiby6HjevTWm9f4XgGr4TLK2PD09IyaGvxNMuQkEI/9vZ5gqSbmFJ8f2xrmdlUyJtpvJShTUwDytbxjKuBcX2mdwESdx70tjHGBIBhQEPGk9R+BxK3244c3btTOSXTzwWfv/82w0OxHZuJakbG1lX7nVibbBv28Q705K/DgYqqrMfKWzUjY/3Y3+8y3fdXsvfoji5TZDtkawSxAtjDGDMB+Ag4Hvh2QpslwEnA34FjgT9nfP8DxI44OW/2Dh3F5N9rPyLnzd7qKKaSN1+hI52jmC78+ZZHMRWXwO6T4Mu1sOzRj583cnRsm3PL5tjOcr8fSspg3ITY8z54F6JRqKqGrx4NTz4MmxtjbXtDB2CPfbY4immL15HHRzENKN2jmHw+KE44imnYcKjs5yimnt9l36OYakI4I9M/4miL92jiUUzDhm/XMkW2R1aOYgIwxnwN+BWxw1zvsNZeZYyZDbxorV1ijCkB7gY+TWzkcHzPTu1+pH0U02Aq5CGrl5TdO/mcv5Czp3oUU9YKxCBRgcgQZfdGPmeH/M5fyNlTLRA6p4KIiCSlAiEiIkmpQIiISFIqECIikpQKhIiIJKUCISIiSalAiIhIUnn/PQivA4iI5Kkh/z0IJ5duxph/ep1B2fPrls/Z8z2/sg8s3wuEiIgMEhUIERFJSgUisxYM3CRnKbs38jk75Hd+ZR9Avu+kFhGRQaIRhIiIJJWtCwYNKcaYI4C5xK5tsdBae3XC49OBa4ldHAlgnrV2YVZDJmGMuQP4OrDeWrtvkscdYq/ra0AbMN1a682FsxOkkH0K8CiwKj7rYWvt7Owl3DZjzDjgLmA0EAUWWGvnJrTJyb5PMfsUcrfvS4C/AsXEPu8etNbOSmhTTOw1fgbYCBxnrX0vy1G3kmL26QziZ41GEGkyxviB+cBUYG9gmjFm7yRN77fWfip+87w4xC0Gjujn8anAHvHbTOCWLGRK1WL6zw7wtz59nhMfUHFh4IfW2knAZ4Ezk7xncrXvU8kOudv3ncCXrLUHAJ8CjjDGfDahzclAo7V2d+AG4JdZzrgtqWSHQfysUYFI3yHA29bad621XcB9QK3HmVJirf0r/V/nuxa4y1rrWmufB6qNMTtnJ13/Usies6y1a3tGA9baZuB1YGxCs5zs+xSz56x4f7bEJ4PxW+KO11rgzvj9B4H/i4/oPJVi9kGlTUzpGwt82Gd6NTA5SbtvGmP+B3gTOM9a+2GSNrkm2WsbC6z1Jk7aDjXG/AdYA1xgrX3N60CJjDG7Ebus7gsJD+V83/eTHXK47+Oj/n8CuwPzrbXb7HtrbdgY0wSMADy/3FwK2WEQP2s0gkhfsv8sEqv6UmA3a+3+wDI+/u8k16Xy2nLVS8D4+HD8JuB3HufZijGmAngI+IG1dnPCwznd9wNkz+m+t9ZGrLWfAnYBDjHGJO7Dytm+TyH7oH7WqECkbzUwrs/0LsT+a+plrd1ore2MT95ObOdXPhjwteUqa+3mnuG4tfYJIGiMCXkcq5cxJkjsA/Yea+3DSZrkbN8PlD3X+76HtXYT8Axb78vq7XtjTAAYRo5tztxW9sH+rFGBSN8KYA9jzARjTBFwPLCkb4OEbcdHEttumw+WACcaY5z4zrAma23ObOLojzFmdM92Y2PMIcTe2xu9TRUTz7UIeN1ae/02muVk36eSPcf7fqQxpjp+vxT4MvBGQrMlwEnx+8cCf7bWej6CSCX7YH/WaB9EmuLbKM8CniR2mOsd1trXjDGzgRettUuAc4wxRxI7AqQBmO5Z4D6MMfcCU4CQMWY1MIvYji+stbcCTxA7zPJtYodazvAm6dZSyH4scLoxJgy0A8fnwh953OeBE4BXjDH/js/7CbAr5Hzfp5I9l/t+Z+DO+LZ8H2CttY8l/L0uAu42xrxN7O/1eO/ibiGV7IP6WaNvUouISFLaxCQiIkmpQIiISFIqECIikpQKhIiIJKUCISIiSalAiAwyY8x0Y8yzXucQSZcKhIiIJKUCISIiSemb1CIZYoy5CPg+MIrY2UEvsdY+4m0qke2nAiGSOe8AXwTqgG8BvzHG7O5tJJHtpwIhkiHW2gf6TN5vjLmY2AWmRPKSCoRIhhhjTgTOB3aLz6oAQkDEq0wiO0I7qUUywBgzntj5+M8CRlhrq4FXSX4xGpG8oAIhkhnlxK5CtgHAGDMDSLz6l0heUYEQyQBr7UrgOuDvwDpgP+A5T0OJ7CBdD0JERJLSCEJERJJSgRARkaRUIEREJCkVCBERSUoFQkREklKBEBGRpFQgREQkKRUIERFJSgVCRESS+v83daF9IuIb9AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf6a120208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the predicted probabilities\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_prob, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.97161726 0.02838274]]\n",
      "[[0.34361555 0.65638445]]\n",
      "[[0.00794192 0.99205808]]\n"
     ]
    }
   ],
   "source": [
    "# examine some example predictions\n",
    "print(logreg.predict_proba(1))\n",
    "print(logreg.predict_proba(2))\n",
    "print(logreg.predict_proba(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first column indicates the predicted probability of **class 0**, and the second column indicates the predicted probability of **class 1**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Probability, odds, e, log, log-odds\n",
    "\n",
    "$$probability = \\frac {one\\ outcome} {all\\ outcomes}$$\n",
    "\n",
    "$$odds = \\frac {one\\ outcome} {all\\ other\\ outcomes}$$\n",
    "\n",
    "Examples:\n",
    "\n",
    "- Dice roll of 1: probability = 1/6, odds = 1/5\n",
    "- Even dice roll: probability = 3/6, odds = 3/3 = 1\n",
    "- Dice roll less than 5: probability = 4/6, odds = 4/2 = 2\n",
    "\n",
    "$$odds = \\frac {probability} {1 - probability}$$\n",
    "\n",
    "$$probability = \\frac {odds} {1 + odds}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>probability</th>\n",
       "      <th>odds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.111111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.20</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.25</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.60</td>\n",
       "      <td>1.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.80</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.90</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   probability      odds\n",
       "0         0.10  0.111111\n",
       "1         0.20  0.250000\n",
       "2         0.25  0.333333\n",
       "3         0.50  1.000000\n",
       "4         0.60  1.500000\n",
       "5         0.80  4.000000\n",
       "6         0.90  9.000000"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a table of probability versus odds\n",
    "table = pd.DataFrame({'probability':[0.1, 0.2, 0.25, 0.5, 0.6, 0.8, 0.9]})\n",
    "table['odds'] = table.probability/(1 - table.probability)\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is **e**? It is the base rate of growth shared by all continually growing processes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.718281828459045"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# exponential function: e^1\n",
    "np.exp(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is a **(natural) log**? It gives you the time needed to reach a certain level of growth:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.999896315728952"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# time needed to grow 1 unit to 2.718 units\n",
    "np.log(2.718)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also the **inverse** of the exponential function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.log(np.exp(5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>probability</th>\n",
       "      <th>odds</th>\n",
       "      <th>logodds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.10</td>\n",
       "      <td>0.111111</td>\n",
       "      <td>-2.197225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.20</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>-1.386294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.25</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>-1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.50</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.60</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>0.405465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.80</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.386294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.90</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>2.197225</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   probability      odds   logodds\n",
       "0         0.10  0.111111 -2.197225\n",
       "1         0.20  0.250000 -1.386294\n",
       "2         0.25  0.333333 -1.098612\n",
       "3         0.50  1.000000  0.000000\n",
       "4         0.60  1.500000  0.405465\n",
       "5         0.80  4.000000  1.386294\n",
       "6         0.90  9.000000  2.197225"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add log-odds to the table\n",
    "table['logodds'] = np.log(table.odds)\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  What is Logistic Regression?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Linear regression:** continuous response is modeled as a linear combination of the features:\n",
    "\n",
    "$$y = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "**Logistic regression:** log-odds of a categorical response being \"true\" (1) is modeled as a linear combination of the features:\n",
    "\n",
    "$$\\log \\left({p\\over 1-p}\\right) = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "This is called the **logit function**.\n",
    "\n",
    "Probability is sometimes written as pi:\n",
    "\n",
    "$$\\log \\left({\\pi\\over 1-\\pi}\\right) = \\beta_0 + \\beta_1x$$\n",
    "\n",
    "The equation can be rearranged into the **logistic function**:\n",
    "\n",
    "$$\\pi = \\frac{e^{\\beta_0 + \\beta_1x}} {1 + e^{\\beta_0 + \\beta_1x}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In other words:\n",
    "\n",
    "- Logistic regression outputs the **probabilities of a specific class**\n",
    "- Those probabilities can be converted into **class predictions**\n",
    "\n",
    "The **logistic function** has some nice properties:\n",
    "\n",
    "- Takes on an \"s\" shape\n",
    "- Output is bounded by 0 and 1\n",
    "\n",
    "We have covered how this works for **binary classification problems** (two response classes). But what about **multi-class classification problems** (more than two response classes)?\n",
    "\n",
    "- Most common solution for classification models is **\"one-vs-all\"** (also known as **\"one-vs-rest\"**): decompose the problem into multiple binary classification problems\n",
    "- **Multinomial logistic regression** can solve this as a single problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6: Interpreting Logistic Regression Coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'household')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAENCAYAAAAVPvJNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X2cVGX9//HXmZu932VZBkQQEdIUb8tUspuvfPtWSr9y1exSKhXS8F7TtDRTDLVMUUPBGwRDzdTLuwS1TEorJQ2z8gbNO7xBWGDZZdn73Zk5vz9mdl2GYXcGZufM7Lyfj8c8ds6Za855z7Wz89nrnDPnOK7rIiIiksjndQAREclNKhAiIpKUCoSIiCSlAiEiIkmpQIiISFIqECIikpQKhIiIJKUCISIiSalAiIhIUgGvA+wgfQ1cRGT7OAM1yPcCwZo1a7yO0CsUClFfX+91jO2i7N7I5+yQ3/kLOfuYMWNSaqdNTCIikpQKhIiIJKUCISIiSalAiIhIUioQIiKSlAqEiIgkpQIhIiJJZeV7EMaYO4CvA+uttfsmedwB5gJfA9qA6dbal7KRTUREksvWF+UWA/OAu7bx+FRgj/htMnBL/GfWRDfUwaP34G5qwKmugdrv4Bs5OueWmasib7wCi+dCWyuUlcPRJ+C88mLS155Kv6Tad9ENdTTdPY/IurVQUhqbWfcRbFgL0Sj4fLDTWJxdJ+J+4as4z/4Rd/1aaKiHcHfs8Yl74hx3yhbL713/+rWweRNUDYeqYdDcBO+9DdEIOA6UlMH4T8Se9NH7sZ8Jy+v7Wigp7bOMKOt8DlRUwYidcEaNTjljsv4BtlwPxLLH8zujRvf2Y+Lze9e7jenE/u95fkNrMxF//GOkoz2l93l0Qx3u/Qvh3f9u1V8Dva7+sqT7dxauW0Pk1muS5shVffs9Wl456J8pjutm52wVxpjdgMe2MYK4DXjGWntvfPq/wBRr7doBFutm4pvU0Q11uDdcBhvqPp45cjTOebPT6vy+327M1DKzZUe+mRl54xW44bLYh+a2xF87MGC/pNp3Sdv1x+ffdsaakTgXXNX7IZXWcvtZHiR5vTuYMekyh4diBathQ//LHzkaTjwb7rppy+cnrjdxuk//D9g//bzPoxvqcK/9CTQmvNdqRsKMH2ydK9nrGihLCn9n0Q11cN1PcTeu3ypHz/sg12TyMyX+Teq8OdXGWODDPtOr4/MGKhCZ8eg9W7/Z45WaU36YO8vMVYvn9l8c4OPX3nM/2WM9/ZJq3yVr15/+MjZs+Hj56S63v+VB5jMmW2biB+62bKiL/b4SPxgT15s43bf/B+qf/t7nj96TPGvDhuS5krUdKEsqf2eP3rN1cejJsb1/o64L4TBOVxd0deF0deF0d0NnJ053d+rze+53dUF3N068nfvyizjr1uBEXdpHVtG2S2jQP1NypUAkq2RJhzbGmJnATABrLaFQaIdX3tDaTHeS+YHWZmrSWH4gEOjNk6llZkvf7Ola196W2jpamwEG7JdU+25b7bZXz/Iztdz+Xq+ny0zx95Vs3an2z7be5/0+N41cA2VJuv5wGBoacOrr2fz6m7hrG/F1hfF3hfF1h3GiUYi6+N9roujl93E6OyH+4U1XF8Sn+52fYW5RERQVQXEx0Y523GgY1+fQVVna/2vNkFwpEKuBcX2mdwGSbjuy1i4AFsQn3UycbCtaXpl0fri8Mq3NLltsYsrQMrNlh07+VVoGbS0DNgtvo096Hku377bVbnv1LD9Ty+3v9Xq6zBR/X8nWnWr/bOt93u9z08jVm6W0An9HF/62TgLtXR9/4K/rIvJyLb6NG/E1NODfuBGnqQknvkl9RGIuvw/X78P1OdAaJtr1Hygqwo3fCAZxhw3DDQZj84NB3OLiLe8Hgx+3H2h+z/34NMXFW80nGIxtXuvJuPA63Bf+knJf9yfVk/XlSoFYApxljLmP2M7pphT2P2RO7XdiO6oStu317iDLlWXmqunnprQPove1D9QvqfZdsnb9GWD7fu/y013uQMsbjIyJy8zCPoiU+6e/93ntd+DN15Lvg5h+bvJ9EK5L4MPVBNo6YzfXT2DzCwTuPozABx/ENsX04ToO0RHdRFu7idbUEJ40ia4RI4iOGEGkpoZoTQ0Rv0N06W8ItzUTLfLHDgSI58jVfRBefKZkZSe1MeZeYAoQAtYBs4AggLX21vhhrvOAI4gd5jrDWvtiCovOyE5qyMwRR4n/hefTUUw7evpgL49iKv7Dg3Rsz1FMkW5wUjyKadhwqMz8UUz0HMUU2glnZJKjmLaRMe2jmIYNxxmZ+aOYAq3NdGfwKCb33bcouv0mgu+somhzG4HOCMFV78W2zfc8v6Kc8G4TiOy6K5Hx4+muqSb8wX8J+1yiO++Me9z38I3aeZvr71Ed7mJjnh7FFGhtjo0mt/MzJdWd1Fk7immQZKxAZEIhn1/eS8runR3KH40SeOMNil56ieC//kXRSy8ReOut3s1A4TFjCO+1F92TJhHec0/Cu+9OeNddcaurt9j04kl2j2XoehB5cxSTiBQAp6WFkmXLKHn8cYqXL8e3aRMAkZoaug88kPbaWroOPJDuAw7AHTbM47SiAiEig8ppa6P4qacofewxSv78Z5yODiI77UTH4YfTeeihdB18MJHx4zMyKpDMUoEQkYxz2tsp/tOfKF26lOJly/B1dBAZNYrWadPoOPJIug466OMdw5KzVCBEJDM6Oih5+mlKli6l5Kmn8LW1EQmFaDeG9iOPpOuQQ8Dv9zqlpEEFQkR2iP+dd6hYuJDShx/G19JCpKaG9mOOof0b36Drs5+FgD5m8pV+cyKyXQKvvkrgxhvZ6fHHcYuKaK+tpf2YY+j83OdUFIYI/RZFJC2+NWuo+uUvKX3oIRg+nObzzqP1pJOIjhzpdTTJMBUIEUmJ09JCxfz5VCxYAK5Ly+mnUzxrFs3hsNfRZJCoQIhI/8Jhyn77Wyqvuw5/fT1tRx9N849/TGTcOIqrqyFPv2wmA1OBEJHkXJfiZcuouuoqgm+9RefkyTTceSfdn/qU18kkS1QgRGQrvrVrqb7gAkqeeYbwxIk03HEHHV/9qr7MVmBUIERkCyWPPkr1T34CnZ00/exntJ50UuzU01JwVCBEBACnqYlhl1xC2SOP0PXpT9N4441EJk70OpZ4SAVCRAi88QY1J5+Mf/VqNl9wAS1nn63vMogKhEihK1myhOrzz8etqqL+wQfpPvhgryNJjtDZskQKVSRC5VVXUXP66YT32YcNv/+9ioNsQSMIkULU1cXwc86hdOlSWk88kaaf/QyKirxOJTlGBUKk0HR0UHPqqZQsW0bTpZfSetppXieSHKUCIVJAnNZWambMoGj5cjZdfTVtJ5zgdSTJYSoQIgXCaWpixAknEPz3v9k0dy7t3/ym15Ekx6lAiBQA38aNjJg2jcCbb9J42210TJ3qdSTJAyoQIkOcb+1aRkybhv/DD2lYvJjOKVO8jiR5QgVCZAjz1dcTOvZYfPX1NNxzT+wKbyIpUoEQGaKctjZqTjoJX10dG++/n+6DDvI6kuQZFQiRoSgcZvgZZxB8+WUaFi1ScZDtogIhMtS4LsMuvZSSp55i01VX0fnVr3qdSPKUTrUhMsRUzJ9P+V130XzGGbRNn+51HMljKhAiQ0jJo49S9Ytf0FZbS/PFF3sdR/KcCoTIEBF4/XWqzz+fzsmT2XTDDeDTn7fsGL2DRIYAp7mZmpkzcauqaLz1Vigu9jqSDAFZ20ltjDkCmAv4gYXW2qsTHt8VuBOojre5yFr7RLbyieQt16X6ggvwv/8+G++/n+ioUV4nkiEiKyMIY4wfmA9MBfYGphlj9k5o9lPAWms/DRwP3JyNbCL5rvyOOyh97DGaL7qIrkMP9TqODCHZ2sR0CPC2tfZda20XcB9Qm9DGBari94cBa7KUTSRvBV98karZs2k//HBaTj/d6zgyxGRrE9NY4MM+06uByQltLgf+aIw5GygHvpxsQcaYmcBMAGstoVAo42G3VyAQyKk86VB2b+xQ9vp6gmeeCePG4b/rLkLV1ZkNl4KC7XuPZSt7tgqEk2SemzA9DVhsrb3OGHMocLcxZl9rbbRvI2vtAmBBzzLq6+szn3Y7hUIhcilPOpTdG9ud3XUZPnMmwfXr2bB0KeFwGDzog4Ls+xywo9nHjBmTUrtsbWJaDYzrM70LW29COhmwANbavwMlQH6Wd5FBVvq731H6xBM0X3gh4X339TqODFHZGkGsAPYwxkwAPiK2E/rbCW0+AP4PWGyMmUSsQGzIUj6RvOGrq2PYJZfQ9ZnP0HLqqV7HkSEsKyMIa20YOAt4Eng9Nsu+ZoyZbYw5Mt7sh8D3jTH/Ae4FpltrEzdDiRQ216X6wguhs5PGX/0K/H6vE8kQlrXvQcS/0/BEwrzL+txfCXw+W3lE8lHZffdR8uc/03TFFUQmTvQ6jgxx+ia1SJ7wf/ghVbNm0fm5z9Gqk/BJFqhAiOSDaJTq888HYNP11+s8S5IVuh6ESB4ofeABipcvZ9Mvf0lk3LiBnyCSAfo3RCTHOQ0NVF1xBZ0HH0zbtxMP/hMZPCoQIjmu6uc/x7d5M02/+IU2LUlW6d0mksOKVqyg/N57aZ05k/CkSV7HkQKjAiGSq7q7GXbRRYTHjKH5vPO8TiMFSDupRXJU+aJFBN94g4Y77sAtL/c6jhQgjSBEcpBv/Xoqr7+eji9/mY7DD/c6jhQoFQiRHFQ5Zw5OZydNs2Z5HUUKmAqESI4JvP46ZffeS+tJJ+l0GuIpFQiRHFN1xRW4VVXaMS2eU4EQySHFTz9NyV/+QvO55+IOH+51HClwKhAiuSIcpmr2bMK77aaT8UlO0GGuIjmi7Le/JfjmmzTcfjsUFXkdR0QjCJFc4DQ3UzlnDp2TJ9MxdarXcUQAjSBEckLFvHn4N26k4e67wXG8jiMCaAQh4jn/6tVU3H47bcccQ/cBB3gdR6SXCoSIxyp/+UtwHDZfdJHXUUS2oAIh4iHn1VcpfeQRWmfMIDp2rNdxRLagAiHiIf/ll+NWVNB8xhleRxHZigqEiEeCL72Eb+lSWk47Dbemxus4IltRgRDxSOWcObihEK3f/77XUUSSUoEQ8UBwxQpK/vIXIuedp2s9SM7q93sQxpiUCoi1NpqZOCKFofL664mMGEH09NOhvd3rOCJJDVQAwkB3CjcRSVHRihWU/PWvtJxxBmj0IDlsoG9ST+hz//8BxwK/AN4HxgM/Bh4anGgiQ1PlddcRCYVoO/FESr0OI9KPfguEtfb9nvvGmPOBg6y1m+Kz3jTGvAi8CNwyeBFFho6if/yD4r/9jaZLL8UtK/M6jki/0tlJPQxIfEeXxeeLSAp6Rw8nneR1FJEBpXOyvjuBZcaYXwEfAuOAc+LzB2SMOQKYC/iBhdbaq5O0McDlgAv8x1r77TTyieS0ohdeoPjZZ2m67DLcUm1cktyXzgjiR8CNwHHA9cDxwLz4/H4ZY/zAfGAqsDcwzRizd0KbPYCLgc9ba/cBfpBGNpGcV3nddURGjqTtxBO9jiKSkpRHEPFDWW+N39J1CPC2tfZdAGPMfUAtsLJPm+8D8621jfH1rd+O9YjkpKLnn6f4uedomjVLowfJGwN9D+J7qSzEWnvHAE3GEtss1WM1MDmhzSfj63yO2Gaoy621f0hl/SK5rvK664iMGkXrCSd4HUUkZQONIFJ5N7vAQAUi2RVQ3CRZ9gCmALsAfzPG7NvnqCkAjDEzgZkA1lpCoVAKEbMjEAjkVJ50KPvgcZYvJ7h8OeFrryU0btwWj+V69oHkc35lT2E9/T1orf3fDK1nNbGd2j12AdYkafO8tbYbWGWM+S+xgrEiIdMCYEF80q2vr89QxB0XCoXIpTzpUPbBU3PllfiGD2fDUUfhJuTM9ewDyef8hZx9zJgxKbVL65KjxpjhwDeIbTL6CFjas89gACuAPYwxE+LPOx5IPELpd8A0YLExJkRsk9O76eQTyTWBlSsp+dOf2Hzhhfreg+SdlI9iMsYcCrwDnAbsD5wKvBOf3y9rbRg4C3gSeD02y75mjJltjDky3uxJYKMxZiXwNHChtXZjWq9GJMdUzJ9PtLyc1unTvY4ikrZ0RhC/As6w1t7XM8MYcxyxQ18PHujJ1tongCcS5l3W574LnB+/ieQ9/6pVlC5ZQuupp+JWV3sdRyRt6XwP4pOATZj3ILB75uKIDB0Vt9wCwSAtut6D5Kl0CsRbxPYd9PUtYpudRKQPX10dZQ88QJsxRHfayes4ItslnU1MPwAeM8acQ+xsrrsRO8ro64OQSySvVdx+O4TDtJx+utdRRLZbyiMIa+1y4BPETq/xT+AmYPf4fBGJcxobKbv7btpra4mMH+91HJHtltZhrvFDWn8zSFlEhoTyxYvxtbbScuaZXkcR2SEpF4j4dxiuAj4FVPR9zFq7a4ZzieQlp62N8kWL6PjKVwhPmuR1HJEdks4I4rfEdkj/EGgbnDgi+a3snnvwNzbScNZZXkcR2WHpFIh9iJ2KOzpYYUTyWlcXFbfeSuehh9J90EFepxHZYekc5vpX4NODFUQk35U99BD+ujpazj7b6ygiGTHQ6b5n95l8D3jSGPMwUNe3Xd9vRIsUpEiEivnz6dpvPzr/53+8TiOSEQNtYhqXML0UCCaZL1LQSp54gsCqVTTcdhs4yc5uL5J/Bjrd94xsBRHJW65Lxbx5hCdOpGPqVK/TiGRMuqf7ngQcC+xkrT3LGLMnUGytfXlQ0onkgeJnnqHo1VdpvP568Pu9jiOSMemc7vtbxHZUjwV6rrpeCVw/CLlE8kbFvHlEdt6Z9qOP9jqKSEalcxTTbOAr1trTgEh83n+AAzKeSiRPBFesoPj552k57TQoKvI6jkhGpVMgRhErCPDx9aRdtr62tEjBqJw3j8jw4bR9O/ECiSL5L50C8U/ghIR5xwP/yFwckfwRWLmSkmXLaD3lFF1OVIakdHZSnwP80RhzMlBujHmS2EWEvjooyURynC4nKkNdOqf7fgPYC5gP/BT4NbCftfatQcomkrP8771H6ZIltJ14oi4nKkNWuqf7biN+2VFjzERgBNAyCLlEcpouJyqFIJ3DXO81xnwufn8G8BqwMr7JSaRg+Nato8xaXU5Uhrx0dlL/H/Bi/P75wJeBQ4CLMh1KJJdV3HYbRCK6nKgMeelsYiqy1nYZY8YCNdba5wCMMfoXSgqG09Cgy4lKwUinQPzbGHMxMB54HCBeLDYPRjCRXFSxaBG+tjad0lsKQjqbmE4G9gNKiR3FBHAocE+mQ4nkIqe5mfJf/5r2qVMJf/KTXscRGXQpjyCste8A306Y9yDwYKZDieSi8jvvxNfURMs553gdRSQrUi4Qxpjvbesxa+0dmYkjkpuc9nbKFyygY8oUuvff3+s4IlmRzj6IxNNsjAY+ATwHqEDIkFZ27734N26kUaMHKSDpbGL638R58VHFpIwmEsk1XV1U3HwznZMn0zV5stdpRLImnZ3UySwmtvNaZMgqe+gh/GvXat+DFJx09kEkFpMy4LvAphSffwQwF/ADC621V2+j3bHAA8DB1toXk7URyZpwmIp58+jaf386DzvM6zQiWZXOCCIMdPe5NQE/AQb8Oqkxxk/sJH9Tgb2BacaYvZO0qyR21tgX0sglMmhKH3uMwHvvxUYPjuN1HJGsSmcn9YSE6VZrbX2Kzz0EeNta+y6AMeY+oBZYmdDuCuAa4II0cokMjmiUiptuovuTn6Tj8MO9TiOSdenspH4fwBizK7HrUq9OYz1jgQ/7TK8GttjbZ4z5NDDOWvuYMWabBcIYMxOYGc9EKBRKI8bgCgQCOZUnHcq+NWfJEoJvvEH4178mNGpUxpcP+d3vkN/5lT2F9aTa0BizM3AfsW9PbwRGGGOeB4631q4Z4OnJxua9lyqN79+4AZg+UA5r7QJgQc8y6utTHcQMvlAoRC7lSYeyJ3BdQlddRXj8eNZ/6UswSH2Tz/0O+Z2/kLOPGTMmpXbp7IO4hdg1qYdba3cGhgP/Am5N4bmrgXF9pncB+haVSmBf4BljzHvAZ4ElxpiD0sgnkjHFf/oTRf/+Ny1nnQWBtC6bIjJkpPPO/wKws7W2G8Ba22qM+RHwUQrPXQHsYYyZEG9/PH1O22GtbQJ6x0vGmGeAC3QUk3jCdam8/nrCu+5K27e+5XUaEc+kM4JoJHYEUl97ksJhrtbaMHAW8CTwemyWfc0YM9sYc2QaGUQGXfFTT1H0n//QfO65EAx6HUfEM+mMIK4BlhljFgHvA7sR22dwaSpPttY+ATyRMO+ybbSdkkYukczpGT2MH0/7N7/pdRoRT6U8grDW3g4YYpuCvg7UENtBvaDfJ4rkkZI//pGiV16h+Qc/0OhBCl4616QuAvYg9iW5BqAYmGGMuWuQsolkVzRK5Zw5hCdMoP2YY7xOI+K5dDYx3QkcACwF6gYnjoh3Sv7wB4IrV9I4d66OXBIhvQJxBDDBWpvSuZdE8ko0Gtv3MHEi7Ucd5XUakZyQToH4gNhmJZEhp+SJJwi+/jqNN92k0YNIXL9/CcaYL/WZvAt41BgzF1jXt5219s+DkE0kO8JhKq+5hu7dd6e9ttbrNCI5Y6B/lRYlmffzhGkXmJiZOCLZV/bAAwTfeYeGhQvB7/c6jkjO6LdAWGsTz+AqMrS0t1M5Zw5dBx5IxxFHeJ1GJKdoY6sUtPLFi/HX1dE4b56u9yCSYEcvOSqSt5ymJirnzaPjS1+i69BDvY4jknNUIKRgVdx8M05TE5svusjrKCI5SQVCCpKvro7yhQtpP/powvvs43UckZykAiEFqeqaa3AiEZov0NVtRbZFBUIKTvDllym1ltZTTiEyfrzXcURylgqEFBbXperyy4nW1NB8zjlepxHJaSoQUlBKHnuM4hdeoPlHP8KtqvI6jkhOU4GQwtHRQdWVV9I9aRJt06Z5nUYk5+mLclIwKm6/ncDq1dRbq1NqiKRAIwgpCL66Oipuuon2I46g6/Of9zqOSF5QgZCCMOzyy3HCYTZfmtIl1EUEFQgpAMXPPEPp0qU0n302kd128zqOSN5QgZChrb2dYZdcQnjiRFrOOMPrNCJ5RTupZUirnDePwHvvUX///VCsCyKKpEMjCBmy/G+/TcXNN9N2zDF0feELXscRyTsqEDI0uS7VP/kJbkkJmy+7zOs0InlJm5hkSCr7zW8ofu45Nl19NdGRI72OI5KXNIKQIcf/wQdUzZ5N5xe/SNt3v+t1HJG8pQIhQ0s0SvUPfwg+H5vmzNFlREV2gDYxyZBSdtddFC9fzqZrryWyyy5exxHJa1krEMaYI4C5gB9YaK29OuHx84FTgDCwAfietfb9bOWT/Od//32qrrySjilTdDI+kQzIyiYmY4wfmA9MBfYGphlj9k5o9i/gIGvt/sCDwDXZyCZDRDhM9bnnQjDIpmuv1aYlkQzI1gjiEOBta+27AMaY+4BaYGVPA2vt033aPw9o76KkrHLOHIpXrKBx/nyiY8Z4HUdkSMjWTuqxwId9plfH523LycDvBzWRDBnOsmVUzJtH67RptB91lNdxRIaMbI0gko333WQNjTHfBQ4CDtvG4zOBmQDWWkKhUKYy7rBAIJBTedKRt9nr6gjMmIG7114Eb76ZUFmZ14nSkrf9HpfP+ZU9hfUM+hpiVgPj+kzvAqxJbGSM+TJwCXCYtbYz2YKstQuABfFJt76+PsNRt18oFCKX8qQjL7NHo4z47nehuZn6++4j3NYGbW1ep0pLXvZ7H/mcv5Czj0lxM2y2CsQKYA9jzATgI+B44Nt9GxhjPg3cBhxhrV2fpVySxypvuIHiv/2N8C23EN5zT6/jiAw5WdkHYa0NA2cBTwKvx2bZ14wxs40xR8abXQtUAA8YY/5tjFmSjWySn0oef5zK66+n7VvfIjpjhtdxRIYkx3WT7grIF+6aNVttqfJMIQ9Zsynw2muEamsJT5pE/QMPENpll7zJniif+j2ZfM5fyNnjm5gGPBZcp9qQvOKrr6dmxgzc6moaFi2CkhKvI4kMWTrVhuSPri6Gf//7+DdupP6RR4iOGuV1IpEhTQVC8kMkwvBzzqH4H/+g4eab6d5/f68TiQx52sQkuc91GXbJJZQuXUrTpZfSUVvrdSKRgqACITmvcs4cyu++m+Yzz6T1tNO8jiNSMFQgJKeVL1xI5a9+Reu0aTRffLHXcUQKigqE5KyyxYsZNmsW7VOn0nT11TpDq0iWqUBITiq//XaqL7mE9sMPp3H+fAjoeAqRbNNfneSciptvpuqqq2j/2tdovPlmCAa9jiRSkFQgJHdEo1RdeSUVt91GW20tm+bOVXEQ8ZAKhOSGjg6qL7iAskceoWXGDDbPng0+bQEV8ZIKhHjOt349NSefTNFLL7H5ootoOess7ZAWyQEqEOKpwKuvUjNjBr7GRhpuv52Or33N60giEqcxvHjDdSn7zW8YeeSROK5L/e9+p+IgkmM0gpCsc1paGHbxxZQ9/DAdU6aw6cYbiY4Y4XUsEUmgAiFZVfT3v1N93nn4P/qIzT/6ES1nn62d0SI5SgVCssJpa6PymmsoX7iQyPjxbHz4YboOPtjrWCLSDxUIGVyuS8nvf0/VrFkE1qyh9cQT2fzTn+KWl3udTEQGoAIhg8a/ahXDLr2UkqefpnvSJOrnzaNr8mSvY4lIilQgJON869ZReeONlN1zD25xMU2XX07rjBk6n5JIntFfrGSM09hIxS23UL5oEU53N23TptF83nlER4/2OpqIbAcVCNlh/g8+oHzhQsruvRenvZ32o4+m+fzziUyY4HU0EdkBKhCyfVyXohUrKL/jDkoefxx8PtqPOoqW008nvNdeXqcTkQxQgZC0+DZsoPTBBym7916C77xDtLKSltNOo3XGDKJjxngdT0QySAVCBuQ0NlLy5JOULl1K8bPP4oTDdB58MI1nnknHN76BW1bmdUQRGQQqELI11yXwxhsU//WvFP/lLxQ/9xxOOEx4/HhaTj2VdmMI77671ylFZJCpQEjM2rWUPvporCg8+yz+desA6P7EJ2iZOZOOb3yD7v3202m4RQqICkQBcpqaCL4Rg49vAAAK50lEQVTySu+t6OWXCaxaRREQGT6cri9+kY7DDqPzi18kOnas13FFxCMqEENZNIpv7VqC77wTKwYvv0zwlVcIvP9+b5Pw2LGxkcHJJ9N40EF077OPTp4nIoAKRH5zXZxNm/B/9BGBjz7Cv3o1/vjPwKpV+Fetwtfe3ts8vOuudO+3H23TptG9//5077tv72m2Q6EQ3fX1Xr0SEclBWSsQxpgjgLmAH1horb064fFi4C7gM8BG4Dhr7XuDkSW6oQ4evQd3UwNOdQ3UfgffyPS+7Rt54xVYPBfaWqGsHI4+gaY3XyGybm3vMgHc+xfCu/+NPWninjjHndK7ruiGutjjb62Erk4oKsbZbQ+cyV/C99QSfO++ja+zC3+wFF/9BvzNLfg6w/i74rf2Lnzd4S1fWzBIpCRIuCRAx06VdJePJFxRQnd1JdF9DsA58cwtXmvP61jX3galZTD9XJwRIz/O1dEe2+/guhCNbH+nbw/HBz4HIv2vd91Ay/H7Yfwe0NQIG/u0HrkzjBkHmzfFboEiaNwQW29JKew8DjrbY49VDYeqYdDZAR/FR2B9fp+9v8t3/xvLW1QMw0M4o0Zv1/ur9z26fm1s/aXl0N4KVcO3e5ki6XJc1x30lRhj/MCbwFeA1cAKYJq1dmWfNmcA+1trTzPGHA8cba09boBFu2vWrEkrS3RDHe4Nl8GGuo9njhyNc97s/v/gXDf2h9/dTfT1l3FuuhIiYZyoixOJ4kSj8Z/x6dIKnO5unNaW2HQkihOO4AsU4dtjP3wtLTgrX8bX0YEvHMHpjsR+9vPriPp9RIsDRIqCRIqDREqLCJcWESmJ/ywtIloU6H9H8rAanB9fjW/k6FhxuOGyhA9+ByoqoWVzyn1a0GpGwowfwB03QOM2RmD9vL9CoRD1CSO3pO/RNJaZTcny54tCzj4m9p2lAY84ydYI4hDgbWvtuwDGmPuAWmBlnza1wOXx+w8C84wxjrU2sxXs0XtgQx2Vb6+ldN0mHNeF6Gs4jy/HKa+C7m6ccBjC4djP+LTT3b3Dq3aBaNCPu24T0eIiot3dREqK6A74cQN+ogE/0WDsp9tzvyhApChAtDiI68/AvoGmhlgfnPLD2Ahoq1GBq+KQjoYNsX7cVnGA2Ad9T5+nIv4e7Ve6yxTZDtkqEGOBD/tMrwYSz/vc28ZaGzbGNAEjgC3+8owxM4GZ8XaEQqG0gjS0NtMNuD4f0WAAfA6u4+BUD6fowMkQDMbOOhoI4AaDEAxu+TMQoMX+GjfcFXuuz4frc3D9vth9fz/Tfh84DsF9DwSg+9WX0sqeKYHWZmpCodhmJdlxKfRjT59vNT8Q2Oo93PMe3d5lZlOy/PlC2VNYz6CvISbZUCZxZJBKG6y1C4AFPY+nO8yKllcC0DJxJ1om7vTxyicfhi/F/8Yiby6HjevTWm9f4XgGr4TLK2PD09IyaGvxNMuQkEI/9vZ5gqSbmFJ8f2xrmdlUyJtpvJShTUwDytbxjKuBcX2mdwESdx70tjHGBIBhQEPGk9R+BxK3244c3btTOSXTzwWfv/82w0OxHZuJakbG1lX7nVibbBv28Q705K/DgYqqrMfKWzUjY/3Y3+8y3fdXsvfoji5TZDtkawSxAtjDGDMB+Ag4Hvh2QpslwEnA34FjgT9nfP8DxI44OW/2Dh3F5N9rPyLnzd7qKKaSN1+hI52jmC78+ZZHMRWXwO6T4Mu1sOzRj583cnRsm3PL5tjOcr8fSspg3ITY8z54F6JRqKqGrx4NTz4MmxtjbXtDB2CPfbY4immL15HHRzENKN2jmHw+KE44imnYcKjs5yimnt9l36OYakI4I9M/4miL92jiUUzDhm/XMkW2R1aOYgIwxnwN+BWxw1zvsNZeZYyZDbxorV1ijCkB7gY+TWzkcHzPTu1+pH0U02Aq5CGrl5TdO/mcv5Czp3oUU9YKxCBRgcgQZfdGPmeH/M5fyNlTLRA6p4KIiCSlAiEiIkmpQIiISFIqECIikpQKhIiIJKUCISIiSalAiIhIUnn/PQivA4iI5Kkh/z0IJ5duxph/ep1B2fPrls/Z8z2/sg8s3wuEiIgMEhUIERFJSgUisxYM3CRnKbs38jk75Hd+ZR9Avu+kFhGRQaIRhIiIJJWtCwYNKcaYI4C5xK5tsdBae3XC49OBa4ldHAlgnrV2YVZDJmGMuQP4OrDeWrtvkscdYq/ra0AbMN1a682FsxOkkH0K8CiwKj7rYWvt7Owl3DZjzDjgLmA0EAUWWGvnJrTJyb5PMfsUcrfvS4C/AsXEPu8etNbOSmhTTOw1fgbYCBxnrX0vy1G3kmL26QziZ41GEGkyxviB+cBUYG9gmjFm7yRN77fWfip+87w4xC0Gjujn8anAHvHbTOCWLGRK1WL6zw7wtz59nhMfUHFh4IfW2knAZ4Ezk7xncrXvU8kOudv3ncCXrLUHAJ8CjjDGfDahzclAo7V2d+AG4JdZzrgtqWSHQfysUYFI3yHA29bad621XcB9QK3HmVJirf0r/V/nuxa4y1rrWmufB6qNMTtnJ13/Usies6y1a3tGA9baZuB1YGxCs5zs+xSz56x4f7bEJ4PxW+KO11rgzvj9B4H/i4/oPJVi9kGlTUzpGwt82Gd6NTA5SbtvGmP+B3gTOM9a+2GSNrkm2WsbC6z1Jk7aDjXG/AdYA1xgrX3N60CJjDG7Ebus7gsJD+V83/eTHXK47+Oj/n8CuwPzrbXb7HtrbdgY0wSMADy/3FwK2WEQP2s0gkhfsv8sEqv6UmA3a+3+wDI+/u8k16Xy2nLVS8D4+HD8JuB3HufZijGmAngI+IG1dnPCwznd9wNkz+m+t9ZGrLWfAnYBDjHGJO7Dytm+TyH7oH7WqECkbzUwrs/0LsT+a+plrd1ore2MT95ObOdXPhjwteUqa+3mnuG4tfYJIGiMCXkcq5cxJkjsA/Yea+3DSZrkbN8PlD3X+76HtXYT8Axb78vq7XtjTAAYRo5tztxW9sH+rFGBSN8KYA9jzARjTBFwPLCkb4OEbcdHEttumw+WACcaY5z4zrAma23ObOLojzFmdM92Y2PMIcTe2xu9TRUTz7UIeN1ae/02muVk36eSPcf7fqQxpjp+vxT4MvBGQrMlwEnx+8cCf7bWej6CSCX7YH/WaB9EmuLbKM8CniR2mOsd1trXjDGzgRettUuAc4wxRxI7AqQBmO5Z4D6MMfcCU4CQMWY1MIvYji+stbcCTxA7zPJtYodazvAm6dZSyH4scLoxJgy0A8fnwh953OeBE4BXjDH/js/7CbAr5Hzfp5I9l/t+Z+DO+LZ8H2CttY8l/L0uAu42xrxN7O/1eO/ibiGV7IP6WaNvUouISFLaxCQiIkmpQIiISFIqECIikpQKhIiIJKUCISIiSalAiAwyY8x0Y8yzXucQSZcKhIiIJKUCISIiSemb1CIZYoy5CPg+MIrY2UEvsdY+4m0qke2nAiGSOe8AXwTqgG8BvzHG7O5tJJHtpwIhkiHW2gf6TN5vjLmY2AWmRPKSCoRIhhhjTgTOB3aLz6oAQkDEq0wiO0I7qUUywBgzntj5+M8CRlhrq4FXSX4xGpG8oAIhkhnlxK5CtgHAGDMDSLz6l0heUYEQyQBr7UrgOuDvwDpgP+A5T0OJ7CBdD0JERJLSCEJERJJSgRARkaRUIEREJCkVCBERSUoFQkREklKBEBGRpFQgREQkKRUIERFJSgVCRESS+v83daF9IuIb9AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcf6a06c710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the predicted probabilities again\n",
    "plt.scatter(glass.al, glass.household)\n",
    "plt.plot(glass.al, glass.household_pred_prob, color='red')\n",
    "plt.xlabel('al')\n",
    "plt.ylabel('household')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.64722323])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted log-odds for al=2 using the equation\n",
    "logodds = logreg.intercept_ + logreg.coef_[0] * 2\n",
    "logodds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.91022919])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert log-odds to odds\n",
    "odds = np.exp(logodds)\n",
    "odds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.65638445])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert odds to probability\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.65638445])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted probability for al=2 using the predict_proba method\n",
    "logreg.predict_proba(2)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['al'], array([4.18040386]))"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the coefficient for al\n",
    "feature_cols, logreg.coef_[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** A 1 unit increase in 'al' is associated with a 4.18 unit increase in the log-odds of 'household'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9920580839167457"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# increasing al by 1 (so that al=3) increases the log-odds by 4.18\n",
    "logodds = 0.64722323 + 4.1804038614510901\n",
    "odds = np.exp(logodds)\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.99205808])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# compute predicted probability for al=3 using the predict_proba method\n",
    "logreg.predict_proba(3)[:, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Bottom line:** Positive coefficients increase the log-odds of the response (and thus increase the probability), and negative coefficients decrease the log-odds of the response (and thus decrease the probability)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-7.71358449])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the intercept\n",
    "logreg.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interpretation:** For an 'al' value of 0, the log-odds of 'household' is -7.71."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.00044652])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert log-odds to probability\n",
    "logodds = logreg.intercept_\n",
    "odds = np.exp(logodds)\n",
    "prob = odds/(1 + odds)\n",
    "prob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That makes sense from the plot above, because the probability of household=1 should be very low for such a low 'al' value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Logistic regression beta values](images/logistic_betas.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changing the $\\beta_0$ value shifts the curve **horizontally**, whereas changing the $\\beta_1$ value changes the **slope** of the curve."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparing Logistic Regression with Other Models\n",
    "\n",
    "Advantages of logistic regression:\n",
    "\n",
    "- Highly interpretable (if you remember how)\n",
    "- Model training and prediction are fast\n",
    "- No tuning is required (excluding regularization)\n",
    "- Features don't need scaling\n",
    "- Can perform well with a small number of observations\n",
    "- Outputs well-calibrated predicted probabilities\n",
    "\n",
    "Disadvantages of logistic regression:\n",
    "\n",
    "- Presumes a linear relationship between the features and the log-odds of the response\n",
    "- Performance is (generally) not competitive with the best supervised learning methods\n",
    "- Can't automatically learn feature interactions"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}