{
"metadata": {
"name": "",
"signature": "sha256:2e18e95b9da8cd7ac4c8e6d7888419f11d68ff0aef0d266b39ee02b6bbad8f59"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "code",
"collapsed": false,
"input": [
"# special IPython command to prepare the notebook for matplotlib\n",
"%matplotlib inline \n",
"\n",
"import numpy as np\n",
"import pandas as pd\n",
"import scipy.stats as stats\n",
"import matplotlib.pyplot as plt\n",
"import statsmodels.api as sm\n",
"\n",
"# To load in the baseball data\n",
"import requests\n",
"import StringIO\n",
"import zipfile\n",
"\n",
"# special matplotlib argument for improved plots\n",
"from matplotlib import rcParams\n",
"\n",
"#colorbrewer2 Dark2 qualitative color table\n",
"dark2_colors = [(0.10588235294117647, 0.6196078431372549, 0.4666666666666667),\n",
" (0.8509803921568627, 0.37254901960784315, 0.00784313725490196),\n",
" (0.4588235294117647, 0.4392156862745098, 0.7019607843137254),\n",
" (0.9058823529411765, 0.1607843137254902, 0.5411764705882353),\n",
" (0.4, 0.6509803921568628, 0.11764705882352941),\n",
" (0.9019607843137255, 0.6705882352941176, 0.00784313725490196),\n",
" (0.6509803921568628, 0.4627450980392157, 0.11372549019607843)]\n",
"\n",
"rcParams['figure.figsize'] = (10, 6)\n",
"rcParams['figure.dpi'] = 150\n",
"rcParams['axes.color_cycle'] = dark2_colors\n",
"rcParams['lines.linewidth'] = 2\n",
"rcParams['axes.facecolor'] = 'white'\n",
"rcParams['font.size'] = 14\n",
"rcParams['patch.edgecolor'] = 'white'\n",
"rcParams['patch.facecolor'] = dark2_colors[0]\n",
"rcParams['font.family'] = 'StixGeneral'"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Recall from from lab last week 10/10/2014\n",
"\n",
"Previously discussed: \n",
"\n",
"* Boston Housing data set\n",
" * Predict housing prices\n",
"* Linear regression\n",
" * Multiple ways to do this in python: `numpy`, `scipy`, `statsmodels`, `sklearn`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Today, we will discuss the following:\n",
"\n",
"* More linear regression using the `statmodels` python module\n",
" * Old Faithful Geyser data set\n",
" * Oakland baseball data set\n",
"\n",
" Download this notebook from Github \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# `statsmodels`\n",
"\n",
"[statsmodels](http://statsmodels.sourceforge.net) is python module specifically for estimating statistical models (less machine learning compared to `sklearn`). It can estimate many types of statistical models, but today we will focus on linear regression. \n",
"\n",
"### Recap of linear regression and least squares\n",
"Last week we learned, [linear regression](http://en.wikipedia.org/wiki/Linear_regression) is a method to model the relationship between a set of independent variables $X$ (also knowns as explantory variables, features, predictors) and a dependent variable $Y$. This method assumes the relationship bewteen each predictor $X$ is linearly related to the dependent variable $Y$. \n",
"\n",
"$$ Y = \\beta_0 + \\beta_1 X + \\epsilon$$\n",
"\n",
"On Tuesday in lecture, we learned how to write this model using matrix multiplication \n",
"\n",
"$$ Y = \\beta X + \\epsilon$$ \n",
"\n",
"where $Y$ has dimensions $n \\times 1$, $X$ has dimensions $n \\times p$ and $\\epsilon$ has dimensions $n \\times 1$. On Tuesday, we also derived the [least squares](http://en.wikipedia.org/wiki/Least_squares) estimates of the coefficients of a linear model. These estimates minimize the difference between the following: \n",
"\n",
"$$ S = \\sum_{i=1}^n r_i = \\sum_{i=1}^n (y_i - (\\beta_0 + \\beta_1 x_i))^2 $$\n",
"\n",
"where $n$ is the number of observations. \n",
"\n",
"> The least squares estimates $\\hat{\\beta}_0$ and $\\hat{\\beta}_1$ minimize the sum of the squared residuals $r_i = y_i - (\\beta_0 + \\beta_1 x_i)$ in the model (i.e. makes the difference bewteen the observed $y_i$ and linear model $\\beta_0 + \\beta_1 x_i$ as small as possible). "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Old Faithful Geyser Data Set\n",
"\n",
"The [Old Faithful Geyser](https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/faithful.html) data set is a well-known data set that depicts the relationship of the waiting time between eruptions and the duration of the eruption for the Old Faithful geyser in Yellowstone National Park, Wyoming, USA [[webcam]](http://yellowstone.net/webcams/). This data set is found in the base installation of the [R programming language](http://cran.r-project.org). \n",
"\n",
"`faithful` is a data set with 272 observations on 2 variables.\n",
"\n",
"Column name| Description \n",
"--- | --- \n",
"eruptions | Eruption time (in mins)\n",
"waiting\t| Waiting time to next eruption (in mins)\n",
"\n",
"There is a function in `statsmodels` (or `sm` for short) called `sm.datasets.get_rdataset` which will download and return a data set found in [R](http://cran.r-project.org). \n",
"\n",
"Let's import the `faithful` dataset. "
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"faithful = sm.datasets.get_rdataset(\"faithful\")"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 8
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"# Let's look at the help file\n",
"# sm.datasets.get_rdataset?\n",
"# faithful?"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 4
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"faithful.title"
],
"language": "python",
"metadata": {},
"outputs": [
{
"metadata": {},
"output_type": "pyout",
"prompt_number": 5,
"text": [
"'Old Faithful Geyser Data'"
]
}
],
"prompt_number": 5
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"faithful = faithful.data\n",
"faithful.head()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"\n",
"
\n",
" \n",
" \n",
" | \n",
" eruptions | \n",
" waiting | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 3.600 | \n",
" 79 | \n",
"
\n",
" \n",
" | 1 | \n",
" 1.800 | \n",
" 54 | \n",
"
\n",
" \n",
" | 2 | \n",
" 3.333 | \n",
" 74 | \n",
"
\n",
" \n",
" | 3 | \n",
" 2.283 | \n",
" 62 | \n",
"
\n",
" \n",
" | 4 | \n",
" 4.533 | \n",
" 85 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" const | \n",
" waiting | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 1 | \n",
" 79 | \n",
"
\n",
" \n",
" | 1 | \n",
" 1 | \n",
" 54 | \n",
"
\n",
" \n",
" | 2 | \n",
" 1 | \n",
" 74 | \n",
"
\n",
" \n",
" | 3 | \n",
" 1 | \n",
" 62 | \n",
"
\n",
" \n",
" | 4 | \n",
" 1 | \n",
" 85 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" yearID | \n",
" teamID | \n",
" salary | \n",
" W | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 1985 | \n",
" ATL | \n",
" 14807000 | \n",
" 66 | \n",
"
\n",
" \n",
" | 1 | \n",
" 1985 | \n",
" BAL | \n",
" 11560712 | \n",
" 83 | \n",
"
\n",
" \n",
" | 2 | \n",
" 1985 | \n",
" BOS | \n",
" 10897560 | \n",
" 81 | \n",
"
\n",
" \n",
" | 3 | \n",
" 1985 | \n",
" CAL | \n",
" 14427894 | \n",
" 90 | \n",
"
\n",
" \n",
" | 4 | \n",
" 1985 | \n",
" CHA | \n",
" 9846178 | \n",
" 85 | \n",
"
\n",
" \n",
"
\n",
"