{
"metadata": {
"name": ""
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "heading",
"level": 1,
"metadata": {},
"source": [
"Homework 2: Desperately Seeking Silver"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Due Thursday, Oct 3, 11:59 PM"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"![](\"http://www.scribewise.com/Portals/202647/images/photo.jpg\")
\n",
"\n",
"
\n",
"\n",
"In HW1, we explored how to make predictions (with uncertainties) about upcoming elections based on the Real Clear Politics poll. This assignment also focuses on election prediction, but we are going to implement and evaluate a number of more sophisticated forecasting techniques. \n",
"\n",
"We are going to focus on the 2012 Presidential election. Analysts like Nate Silver, Drew Linzer, and Sam Wang developed highly accurate models that correctly forecasted most or all of the election outcomes in each of the 50 states. We will explore how hard it is to recreate similarly successful models. The goals of this assignment are:\n",
"\n",
"1. To practice data manipulation with Pandas\n",
"1. To develop intuition about the interplay of **precision**, **accuracy**, and **bias** when making predictions\n",
"1. To better understand how election forecasts are constructed\n",
"\n",
"The data for our analysis will come from demographic and polling data. We will simulate building our model on October 2, 2012 -- approximately one month before the election. \n",
"\n",
"### Instructions\n",
"\n",
"The questions in this assignment are numbered. The questions are also usually italicised, to help you find them in the flow of this notebook. At some points you will be asked to write functions to carry out certain tasks. Its worth reading a little ahead to see how the function whose body you will fill in will be used.\n",
"\n",
"**This is a long homework. Please do not wait until the last minute to start it!**\n",
"\n",
"The data for this homework can be found at [this link](https://www.dropbox.com/s/vng5x10b837ahnc/hw2_data.zip). Download it to the same folder where you are running this notebook, and uncompress it. You should find the following files there:\n",
"\n",
"1. us-states.json\n",
"2. electoral_votes.csv\n",
"3. predictwise.csv\n",
"4. g12.csv\n",
"5. g08.csv\n",
"6. 2008results.csv\n",
"7. nat.csv\n",
"8. p04.csv\n",
"9. 2012results.csv\n",
"10. cleaned-state_data2012.csv"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Setup and Plotting code"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"from ggplot import *"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 1
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"%matplotlib inline\n",
"from collections import defaultdict\n",
"import json\n",
"\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import pandas as pd\n",
"\n",
"from matplotlib import rcParams\n",
"import matplotlib.cm as cm\n",
"import matplotlib as mpl\n",
"\n",
"#colorbrewer2 Dark2 qualitative color table\n",
"dark2_colors = [(0.10588235294117647, 0.6196078431372549, 0.4666666666666667),\n",
" (0.8509803921568627, 0.37254901960784315, 0.00784313725490196),\n",
" (0.4588235294117647, 0.4392156862745098, 0.7019607843137254),\n",
" (0.9058823529411765, 0.1607843137254902, 0.5411764705882353),\n",
" (0.4, 0.6509803921568628, 0.11764705882352941),\n",
" (0.9019607843137255, 0.6705882352941176, 0.00784313725490196),\n",
" (0.6509803921568628, 0.4627450980392157, 0.11372549019607843)]\n",
"\n",
"rcParams['figure.figsize'] = (10, 6)\n",
"rcParams['figure.dpi'] = 150\n",
"rcParams['axes.color_cycle'] = dark2_colors\n",
"rcParams['lines.linewidth'] = 2\n",
"rcParams['axes.facecolor'] = 'white'\n",
"rcParams['font.size'] = 14\n",
"rcParams['patch.edgecolor'] = 'white'\n",
"rcParams['patch.facecolor'] = dark2_colors[0]\n",
"rcParams['font.family'] = 'StixGeneral'\n",
"\n",
"\n",
"def remove_border(axes=None, top=False, right=False, left=True, bottom=True):\n",
" \"\"\"\n",
" Minimize chartjunk by stripping out unnecesasry plot borders and axis ticks\n",
" \n",
" The top/right/left/bottom keywords toggle whether the corresponding plot border is drawn\n",
" \"\"\"\n",
" ax = axes or plt.gca()\n",
" ax.spines['top'].set_visible(top)\n",
" ax.spines['right'].set_visible(right)\n",
" ax.spines['left'].set_visible(left)\n",
" ax.spines['bottom'].set_visible(bottom)\n",
" \n",
" #turn off all ticks\n",
" ax.yaxis.set_ticks_position('none')\n",
" ax.xaxis.set_ticks_position('none')\n",
" \n",
" #now re-enable visibles\n",
" if top:\n",
" ax.xaxis.tick_top()\n",
" if bottom:\n",
" ax.xaxis.tick_bottom()\n",
" if left:\n",
" ax.yaxis.tick_left()\n",
" if right:\n",
" ax.yaxis.tick_right()\n",
" \n",
"pd.set_option('display.width', 500)\n",
"pd.set_option('display.max_columns', 100)"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 2
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"#this mapping between states and abbreviations will come in handy later\n",
"states_abbrev = {\n",
" 'AK': 'Alaska',\n",
" 'AL': 'Alabama',\n",
" 'AR': 'Arkansas',\n",
" 'AS': 'American Samoa',\n",
" 'AZ': 'Arizona',\n",
" 'CA': 'California',\n",
" 'CO': 'Colorado',\n",
" 'CT': 'Connecticut',\n",
" 'DC': 'District of Columbia',\n",
" 'DE': 'Delaware',\n",
" 'FL': 'Florida',\n",
" 'GA': 'Georgia',\n",
" 'GU': 'Guam',\n",
" 'HI': 'Hawaii',\n",
" 'IA': 'Iowa',\n",
" 'ID': 'Idaho',\n",
" 'IL': 'Illinois',\n",
" 'IN': 'Indiana',\n",
" 'KS': 'Kansas',\n",
" 'KY': 'Kentucky',\n",
" 'LA': 'Louisiana',\n",
" 'MA': 'Massachusetts',\n",
" 'MD': 'Maryland',\n",
" 'ME': 'Maine',\n",
" 'MI': 'Michigan',\n",
" 'MN': 'Minnesota',\n",
" 'MO': 'Missouri',\n",
" 'MP': 'Northern Mariana Islands',\n",
" 'MS': 'Mississippi',\n",
" 'MT': 'Montana',\n",
" 'NA': 'National',\n",
" 'NC': 'North Carolina',\n",
" 'ND': 'North Dakota',\n",
" 'NE': 'Nebraska',\n",
" 'NH': 'New Hampshire',\n",
" 'NJ': 'New Jersey',\n",
" 'NM': 'New Mexico',\n",
" 'NV': 'Nevada',\n",
" 'NY': 'New York',\n",
" 'OH': 'Ohio',\n",
" 'OK': 'Oklahoma',\n",
" 'OR': 'Oregon',\n",
" 'PA': 'Pennsylvania',\n",
" 'PR': 'Puerto Rico',\n",
" 'RI': 'Rhode Island',\n",
" 'SC': 'South Carolina',\n",
" 'SD': 'South Dakota',\n",
" 'TN': 'Tennessee',\n",
" 'TX': 'Texas',\n",
" 'UT': 'Utah',\n",
" 'VA': 'Virginia',\n",
" 'VI': 'Virgin Islands',\n",
" 'VT': 'Vermont',\n",
" 'WA': 'Washington',\n",
" 'WI': 'Wisconsin',\n",
" 'WV': 'West Virginia',\n",
" 'WY': 'Wyoming'\n",
"}"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 3
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is some code to plot [State Chloropleth](http://en.wikipedia.org/wiki/Choropleth_map) maps in matplotlib. `make_map` is the function you will use."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"#adapted from https://github.com/dataiap/dataiap/blob/master/resources/util/map_util.py\n",
"\n",
"#load in state geometry\n",
"state2poly = defaultdict(list)\n",
"\n",
"data = json.load(file(\"data/us-states.json\"))\n",
"for f in data['features']:\n",
" state = states_abbrev[f['id']]\n",
" geo = f['geometry']\n",
" if geo['type'] == 'Polygon':\n",
" for coords in geo['coordinates']:\n",
" state2poly[state].append(coords)\n",
" elif geo['type'] == 'MultiPolygon':\n",
" for polygon in geo['coordinates']:\n",
" state2poly[state].extend(polygon)\n",
"\n",
" \n",
"def draw_state(plot, stateid, **kwargs):\n",
" \"\"\"\n",
" draw_state(plot, stateid, color=..., **kwargs)\n",
" \n",
" Automatically draws a filled shape representing the state in\n",
" subplot.\n",
" The color keyword argument specifies the fill color. It accepts keyword\n",
" arguments that plot() accepts\n",
" \"\"\"\n",
" for polygon in state2poly[stateid]:\n",
" xs, ys = zip(*polygon)\n",
" plot.fill(xs, ys, **kwargs)\n",
"\n",
" \n",
"def make_map(states, label):\n",
" \"\"\"\n",
" Draw a cloropleth map, that maps data onto the United States\n",
" \n",
" Inputs\n",
" -------\n",
" states : Column of a DataFrame\n",
" The value for each state, to display on a map\n",
" label : str\n",
" Label of the color bar\n",
"\n",
" Returns\n",
" --------\n",
" The map\n",
" \"\"\"\n",
" fig = plt.figure(figsize=(12, 9))\n",
" ax = plt.gca()\n",
"\n",
" if states.max() < 2: # colormap for election probabilities \n",
" cmap = cm.RdBu\n",
" vmin, vmax = 0, 1\n",
" else: # colormap for electoral votes\n",
" cmap = cm.binary\n",
" vmin, vmax = 0, states.max()\n",
" norm = mpl.colors.Normalize(vmin=vmin, vmax=vmax)\n",
" \n",
" skip = set(['National', 'District of Columbia', 'Guam', 'Puerto Rico',\n",
" 'Virgin Islands', 'American Samoa', 'Northern Mariana Islands'])\n",
" for state in states_abbrev.values():\n",
" if state in skip:\n",
" continue\n",
" color = cmap(norm(states.ix[state]))\n",
" draw_state(ax, state, color = color, ec='k')\n",
"\n",
" #add an inset colorbar\n",
" ax1 = fig.add_axes([0.45, 0.70, 0.4, 0.02]) \n",
" cb1=mpl.colorbar.ColorbarBase(ax1, cmap=cmap,\n",
" norm=norm,\n",
" orientation='horizontal')\n",
" ax1.set_title(label)\n",
" remove_border(ax, left=False, bottom=False)\n",
" ax.set_xticks([])\n",
" ax.set_yticks([])\n",
" ax.set_xlim(-180, -60)\n",
" ax.set_ylim(15, 75)\n",
" return ax"
],
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 4
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Today: the day we make the prediction"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"# We are pretending to build our model 1 month before the election\n",
"import datetime\n",
"today = datetime.datetime(2012, 10, 2)\n",
"today"
],
"language": "python",
"metadata": {},
"outputs": [
{
"metadata": {},
"output_type": "pyout",
"prompt_number": 5,
"text": [
"datetime.datetime(2012, 10, 2, 0, 0)"
]
}
],
"prompt_number": 5
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Background: The Electoral College\n",
"\n",
"US Presidential elections revolve around the Electoral College . In this system, each state receives a number of Electoral College votes depending on it's population -- there are 538 votes in total. In most states, all of the electoral college votes are awarded to the presidential candidate who recieves the most votes in that state. A candidate needs 269 votes to be elected President. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Thus, to calculate the total number of votes a candidate gets in the election, we add the electoral college votes in the states that he or she wins. (This is not entirely true, with Nebraska and Maine splitting their electoral college votes, but, for the purposes of this homework, we shall assume that the winner of the most votes in Maine and Nebraska gets ALL the electoral college votes there.) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is the electoral vote breakdown by state:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*As a matter of convention, we will index all our dataframes by the state name*"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"electoral_votes = pd.read_csv(\"data/electoral_votes.csv\").set_index('State')\n",
"electoral_votes.head()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Votes | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | California | \n",
" 55 | \n",
"
\n",
" \n",
" | Texas | \n",
" 38 | \n",
"
\n",
" \n",
" | New York | \n",
" 29 | \n",
"
\n",
" \n",
" | Florida | \n",
" 29 | \n",
"
\n",
" \n",
" | Illinois | \n",
" 20 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Obama | \n",
" Romney | \n",
" Votes | \n",
"
\n",
" \n",
" | States | \n",
" | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" 0.000 | \n",
" 1.000 | \n",
" 9 | \n",
"
\n",
" \n",
" | Alaska | \n",
" 0.000 | \n",
" 1.000 | \n",
" 3 | \n",
"
\n",
" \n",
" | Arizona | \n",
" 0.062 | \n",
" 0.938 | \n",
" 11 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" 0.000 | \n",
" 1.000 | \n",
" 6 | \n",
"
\n",
" \n",
" | California | \n",
" 1.000 | \n",
" 0.000 | \n",
" 55 | \n",
"
\n",
" \n",
"
\n",
"
![](\"http://i.imgur.com/uCOFXHp.png\")
"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"plot_simulation(result)"
],
"language": "python",
"metadata": {},
"outputs": [
{
"metadata": {},
"output_type": "display_data",
"png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAGDCAYAAACBTdwmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4FFW+//FPdQIJWUhCCEsEElCI4BVZRRYhODggowio\nDIIiXkFccB+X36iMC8plGJfxGXUcdMbRERz3jSDOqBdwQSUGFRRkCQTC1gmETgIhIfn+/uCmh04n\nAUIgRfr9ep48D33O6apTh+7KJ1WnqhwzMwEAAKDBeRq6AwAAADiIYAYAAOASBDMAAACXIJgBAAC4\nBMEMAADAJQhmAAAALhHe0B0A3C43N1cLFy5UmzZtdOGFFzZ0d4AA+fn5Ki4uVocOHRq6K41KcXGx\nPvnkE+Xk5OjGG288pmWZmTIyMvTxxx+rffv26tq1q0aMGFFPPUVjwxEz1KuMjAxddNFF8ng88ng8\nOvvss3XuuecqLS1NvXv31j333KPc3Nx6Xednn32m6667Ttdcc40SExN1xx131No+Ly9Pv/vd79S/\nf38NHz5c559/vnr27KnJkyfr22+/DWj75Zdf6sorr9S1114bVBcqZsyYocTERH3//feNYj1HY9Om\nTZoyZYpuv/12TZ48WRdeeGG1/cvPz9cNN9ygW265RdOmTdP555+vxYsX12mdixYtUlhYWLV1q1at\n8n+3Kn86d+6sli1bBrRbvny5pk2bpmXLlmnOnDkaNWqU/vCHP2jmzJl64YUXdPnllyslJUU+n69O\nfayNmelPf/qTzjzzTHXv3l1t27b19/U3v/lNva/veNi8ebNuu+02XXzxxXrzzTePaVmrVq1Snz59\n9Pe//1133323brvttqBQtmXLFk2ZMkX333+/7rjjDo0dO1Y//vjjMa0XJzED6tnu3bvNcRzzeDwB\n5RkZGdasWTOLjY217777rl7WtXHjRouOjrbMzEwzM3v22Wdt0qRJNbb/5JNPLDEx0YYNG2a5ubn+\n8uLiYrv77rvN4/HY/fffH/Cef//73+Y4jj344IP10ueTzW9/+1tLTEy0FStWNIr1HKnc3Fxr27at\n/etf//KXPfvssxYTE2PLly/3lxUWFtoZZ5xhc+fO9ZctXLjQmjRpYh988MFRrdPr9Vrbtm2DvjuV\nJkyYYNdff73deeed/p8333wzoM3ixYstKirKsrKy/GUPPPCA/9/ffPONhYeH28KFC4+qb0dqzpw5\nFhERYcuWLfOXvfHGG9a8eXObMGHCcVnn8VBeXm6O49jQoUPrvIylS5daXFyc3XHHHTW22bRpkyUn\nJwd8ft577z2Li4uzn376qc7rxsmLYIbjorpgZmZ22223meM4dtFFF9XLembOnGmO49imTZsO2/b7\n77+36Oho69Kli+3bt6/aNpX9+/3vf+8v+/TTT0M6mIWqK664wpKTkwPKysvLrU2bNtarVy9/2X33\n3WdhYWFBn6m+ffta27Ztbf/+/Ue8zkmTJtnQoUOr/e6sXbvWfvnLXx52GT179rQxY8YElM2aNcvM\nzA4cOGA9e/a0a6655oj7dLQ6depkffr0CSpfsGCBDR48+Lit93g4lmCWk5NjLVu2POw2X3zxxXbK\nKacElffq1cuGDBlSp3Xj5MapTJxQnTp1kiRt2LChXpa3adMmSQdPnxzOrbfeqr179+qOO+5QZGRk\ntW1+97vfqVmzZpoxY4a2bt1aL33EyefAgQN6++23ddpppwWUezweDRgwQFlZWVqxYoUk6fXXX9cp\np5wS9Jk699xztX37di1YsOCI1vn8889r2LBh6tChQ7Wf51mzZmndunWaNm2aXnvtNe3du7fa5axZ\ns0YDBgzwv/7kk080bNgwSdIf//hH5efn64knnjiiPtXFzp07tWrVKm3cuDGgfOTIkUpOTj5u63Wb\ne+65R/n5+Xr44YdrbLN161a99957Ou+884Lqzj//fC1ZskSrVq06nt2ECxHMcEJ9+eWXkqTevXvX\n2m7FihW66qqrdPvtt2v8+PHq2bOnnnnmGX/9ypUrdfXVV+vTTz+VJP3mN7/R1VdfrTfeeKPa5eXm\n5urTTz+V4zi1TuCPi4vTueeeq/3792v+/PkBdRUVFZo9e7batWun6OhoXXDBBVq7dm1Am4yMDF19\n9dX6wx/+oEmTJumKK67Qrl27JEn79+/Xq6++qssvv1y//OUvtW7dOl100UWKjY1VSkqKMjIyVFpa\nqpkzZ6pTp05KSEjQLbfcEvBLev/+/XrwwQd1xx136NFHH9V5552np59+utax/Oqrr9SqVSt5PB7d\nfvvt2r9/vz755BOlpKTI4/Fo/PjxWr9+vSTpp59+Uo8ePTRixAj/ejds2KA5c+b450wVFxfrpZde\n0tixYzVp0iStXLlSw4YNU0xMjPr16+cfkwMHDujNN9/UFVdcoWHDhiknJ0djx45VbGyszjjjDH31\n1VcB/azreg79/3niiSc0ffp0TZ48WZGRkfJ4PEpLS9PQoUP9QWbWrFmKj4/XkiVLahwzr9ervXv3\nqlmzZkF1KSkpkqRvvvlGkpSdnV1ru+XLl9f23yNJWrt2rT7//HNdeeWV1dbv27dPmZmZ2rVrl+bO\nnavx48crJSVFb731VlDb9PR0/9yxbdu26ccff1SfPn20efNmPfDAA3r++ecVGxt72D7V1dChQ1VS\nUqKhQ4fqs88+C6j705/+5P/30qVLdeONN+qcc87RunXr9Ktf/UoxMTHq0KGDnnrqKX+7devWaebM\nmTr99NNVWFioCy+8ULGxsXr//fclSTk5OZoyZYrGjh2rLl26aPjw4VqzZk3Aev/xj39o6tSpmjNn\nji699FLddNNN2rdvX0CbdevW6fLLL9c111yju+66S3PmzAnattLSUvXs2VO9e/fWgQMHahyD/Px8\nvfrqq4qJidHnn3+uvn37KioqSqeffnrA97Vyf3j66acHLaNr166S5N/HIYQ07AE7NFZVT2Xu2rXL\nZsyYYY7jWFpamm3durXG93766acWExMTMNfoww8/NI/HY1OmTAloe9VVVx3Rqcz333/fHMex+Pj4\nw/b9pptuMsdxbPz48f7+VPb70UcftUWLFvlPeSYnJ1teXp6ZmX377bfm8Xjs1VdfNbODp706dOhg\nl112mZkdnMf21VdfWdOmTa1NmzZ2//3324YNG2zLli3WsWNHa9Omjd100022YsUK27Nnj11//fXm\nOI59+OGH/r7deuutFhkZ6X/90UcfmeM4tmDBglq3afbs2eY4jr333nv+svnz55vjOPb3v/89oO3Y\nsWNt3bp1Zmb28ssvW48ePQLaFRYW2rJly8xxHOvcubP99re/tdWrV9ubb75pHo/HLrjgAjMzKykp\nsTVr1lhUVJS1bdvWbr31Vvv+++/tf//3fy0mJsa6du3qX+exrKfSgw8+aGeffXbAMh3Hseuuuy6g\n3bRp0wL+n6pTUlJiHo/HzjjjjKC6+++/3xzH8Z8eTE5OttjY2KB2L7zwgjmOY9OmTatxPWZmZWVl\nNmbMGNu9e7eZ/eczXZMVK1bYlClTzOPxVDtXLD8/326//Xb77W9/a3PnzrWKigozO3jK7Nprr621\nL/Vh8+bNdtppp/n3ARMnTrTNmzcHtCkoKLB58+ZZRESExcfH2/Tp023p0qX29ttvW6dOncxxHJs3\nb54VFxdbRkaGtWrVyhzHsYceesheffVV69Wrl33wwQe2fft269ixo39fUVRUZB06dLC2bduaz+cz\nM7N33nnHHMfxz3krLCy0yMhIu/POO/392bBhg7Vu3TpgPuG8efOCTmUWFhZaYmKitWzZ0oqLi2sc\ng9dff90cx7EOHTr459Pm5eXZ2LFjzXEcmz17tpmZPf744+Y4jv3lL38JWsaCBQvMcRy77bbbjmr8\ncfIjmOG4cBzHHMexgQMHWs+ePa1Lly6Wnp5uM2fOrHWHZmbWpUsXGzlyZFD5JZdcYo7j2NKlS/1l\nRxrMXnnlFXMcx9q3b3/Yvt97773mOI4NHz7czP4TzG6++eaAdtOmTTPHcex3v/udmZktW7bM2rdv\nb59++qm/zZAhQywtLS3gfR06dLAOHToElN18883mOI79+9//9pdlZWWZ4zh27733+svuuusu69Gj\nh/91dnZ2QEioSV5enkVGRgbMPSopKbFmzZrZ+eef7y/Lz8+30aNHB7x37ty5QQGuoqLCHMexQYMG\nBbTt3r27tWjRImh7q477qFGjzHEc/y/P+lhPcnJywOTy8vJyS0pKshEjRgS0Ky8vDwoK1akMimvW\nrAkov+eee8xxHHvmmWfMzGz06NHmOI4tWrQooN2f//xncxzH7rrrrlrXM2PGjIBAcLhgVunDDz+0\npk2bVhseq3rrrbcsNTXVCgsLbc+ePTZjxgybNm2aPfroo0c1B+5IFRUV2V133WXNmjUzx3EsNjbW\nnn766aB2lSHqUKtWrfL/IVRp8ODB5vF4bMuWLQFtb7jhBhs3blxA2a233mqO49gLL7xgZmZvv/22\ndejQwVavXu1vk5KS4v9+m5mNHDnS0tPTA5ZTVlZW7RyzPXv22J49e2rd/j/84Q/mOI798Y9/DCgv\nLCy02NhYi4uLs9LSUnvkkUfMcRx78cUXg5bx8ccfm+M4NnXq1FrXhcaHU5k4bhzH0WeffaZvv/1W\na9as0aeffqp7771XUVFRNb7n559/1tq1a9WxY8eguosvvljSwdOFR6tFixaSdES3B9izZ48kKSEh\nIaA8MTEx4PW1114rSf7TNf369VNOTo7S09OVk5Ojp59+Wrm5uSotLQ14n+M4QbdDqFxXkyZN/GXx\n8fGSDs7ZqTR79mxlZWVJkv73f/9Xzz//vCQFraOqxMREXXrppfrggw+0Y8cOSdKyZcsUFRWljz/+\n2D9X7/nnn9c111wT8N7w8ODbHTqOE9Tfyu0oKCgIalt1GZXbe2jbY11PaWmpVq9e7X/t8XjUsWNH\n/ynFQ8vbtWsXtK6qHn30UTmOoxtuuMG/rs8//1yvvvqqpP/Ml3zooYfUpEkT/eY3v/HfCmblypV6\n7rnnAtpV54svvlBRUZF/DtjRGD58uKZNm6Yff/xRxcXFNbYrLCzUrbfequeff1779+/XOeecoxUr\nVuiZZ57R+PHj9cILL9T43oqKCpWUlAT81HYKr1J0dLRmz56tNWvWaPTo0SoqKtL06dN17733BrRz\nHCdobl63bt3Uu3dvrVu3zn+6sfJzcMoppwS0/eCDD7RmzRpdffXV/p+1a9eqR48e2r9/vyRp9OjR\n2rRpk9LS0rRmzRo99dRTKiws9H9n8vPz9eGHH6pHjx4By67u8yhJzZs3V/PmzWvdfvu/aQBV59TF\nxMRo8ODB8vl8+vHHH9WqVStJUllZWdAyKvsXHR1d67rQ+BDM4Cp5eXmSpJKSkqC61NRUSdLu3buP\nermVc9oKCwu1ffv2WtuuW7cu4D01qfyFe+gk7PXr12vKlClatGiRpk6dGvSLpC6q/iJ84403NHny\nZDVr1kxTp0494uXccMMNOnDggP72t79Jkh577DG9++678ng8+utf/yoz08KFC0/oTXTtCC7aOFJ3\n3XWXsrKy9Pbbb0s6+DnJz8/X7bffXqflXXDBBXrrrbe0f/9+9enTR8OGDdPXX3+tqKgoRUZGasiQ\nIZKkM888Ux9//LHatGmjc889V0OHDtUbb7zh/2Ng+PDh1S6/sLBQDz30kO6///6A4FNeXi7p4HzC\n6r4Hh6oMdLW1u++++/SrX/1Kv/jFL3TzzTdr48aNeu655/zBtTKoV+ell15SVFRUwE/lHyRHon37\n9nrrrbf02GOPSZL+53/+J2j+V3U6duyoiooKFRUV1dpu27ZtGjlypP72t7/5fz744AN9++23uv76\n6/3tsrKyNHnyZH333XeaPn16wBy7tWvXyszUtGnTI96uw2nfvr2k/+zPDlUZ1vbt2+f/A6G6dpVl\nVS9AQePHnf/hKpU7rZ9++imorjKgHMnRjqqSkpI0YsQIffjhh1qwYEHQUaFKxcXF+uKLLxQREaHx\n48fXuszKv+Ird5xZWVkaMmSIXnjhBV122WVH3ccjcd999+mFF17Qzz//rNjY2KAr32rTv39/de/e\nXc8//7zOP/98JSUlaeDAgRo5cqRefPFF9erVq8YQcTK488475fP59Ne//lVZWVmKi4vT559/rtat\nW9d5mRdffLH/SK10cCL/HXfcoeuvvz7gSM+gQYP00Ucf+V9v375djzzyiC644AL/HxRVZWZm6qOP\nPvIHuKqaNWsmx3H8Qa06+/fvV0JCQtDR3EPXkZGRoe+++0779u3T66+/rjFjxqhNmzb+99dm1KhR\nQRcvVL2Z7aHmzJmjsWPH6tRTTw0ov+222/TGG2/oyy+/1A8//KC0tLRa17t3714lJCQoKSmp1nax\nsbFauXJltXU+n0/NmzdXRkaGxowZo88++0x9+/YNald5NLbyApj6cM4558hxHGVnZwfVVR4J69Ch\ng7p06aKwsDD9/PPPQe0qA2x6enq99QsnB4IZ6t2xHAVJTU1Vz5499fXXX2vr1q0BpwIq73p+ySWX\n1GnZjz32mD777DPNmjVLv/71rxUTExPU5oknnlBhYaEefvjhwwbAyjvAX3XVVZIOHg0oKirSueee\n62+zd+/eejsqVFhYqNmzZ+sXv/iF/y/+yqN1R7qO66+/Xtdff72uuOIK/1Vt06ZN04UXXqhbbrnl\niK4gdKsPPvgg4Gq9mlRUVGjr1q1HHfDLysp08803q02bNpoxY0atbW+66SY1a9Ys6Mq+ytPkcXFx\n6t27t5YtWxZQb2Z6+OGHlZGREVRXnffee0///d//XW1deXm5rrvuOs2dO1dRUVHyer06cOCA+vfv\n72/z1ltv1fpooBYtWtQYHKvTvn173XLLLXr//ff9f7hUqgx0VUNbVQcOHNCKFSs0YcKEw65v4MCB\n/kcd/eIXv/CXb9u2TU8++aRmz56tBx98UI7jBISyQ7+XZ5xxhmJiYrRw4ULl5+f7Q25lfdXvVkFB\ngRzHUVxcXI39SklJ0bBhw/TOO+9o9uzZAXU//fSTevfu7d+3jRo1Sh9//HHQMhYtWqSzzz5bZ5xx\nxmHHAY3MkUxEKy0trfGGnEBVh975v/KKxaOxfPlyi4mJsUsuucQOHDhgZgev4jr99NODbvJ66aWX\nmuM4tnLlyiNa9uLFiy0pKckGDx5sGzZs8JeXlZXZ448/buHh4Xb33XcHvGfp0qXmOI5df/31/rLi\n4mJLT08PuKP3+PHjzXEcu/POO+3HH3+0p556ytq3b2+RkZH29ddf+9fXtm1bS0lJCVhH5QUHh16B\nuWbNGnMcx/8kg6KiIgsPD7fY2FhbtGiRLV++3G644Qb/DXuXLFly2O0vKiqy5s2b2+WXX+4vq6io\nsA4dOtgVV1xR7XueeeYZcxzHnn32WX/Zvn37zHGcoJtn9u7d2xzH8f+/mZm1bt066GKHygs5Kq/+\nrI/1JCcn26BBg+zuu++2e++91+6991576KGHgq5avPbaa81xHHvttddqHKeqKioq7LrrrrOkpCT7\n9ttva207a9Ysi4qKClpvUVGRtWzZ0lq1alXr/rS6yf/XXHONpaSk2LPPPuu/yvKFF16wESNG1Lis\nxx9/3KZPnx5Q1q1bN3vsscfM7OB36p577ql1W47WN998Y47j2IQJE8zr9frLly1bZlFRUUET9VNS\nUiwqKipgYv7s2bOtc+fOVlBQ4C8bMGCAOY5jJSUlAe9ftmyZNWnSxJo0aWLXXXedPffcczZjxgzr\n0aOH//t2zjnnmOM49sQTT9iqVats5syZ1qJFCzvllFMsMzPTtm/f7p+En56ebj/++KPt2LHDX9ay\nZUt79dVXbf/+/VZUVGQtWrSwVq1a2d69e2sdi6ysLIuMjAyY2L906VKLjo62r7/+2l+2fv16S0hI\nsL/+9a/+sg8++MAiIyPtm2++OZJhRyNTazCrqKiwb7/91h577DFbv369vzw7O9ueeeYZe+SRR+yl\nl14K+ALt2bPH3n//ffv666/trbfesh07dhxRHRqHhQsX2uWXX+4PZhdeeKH97W9/O+rlrFy50i6+\n+GI799xz7YYbbrBf//rXNn/+fH/99u3b7c9//rMlJCSYx+Ox0aNHH/Ev2vz8fHvggQfsnHPOsYED\nB9p5551nvXr1squuuqraHWFFRYU98cQT1q9fPzv//PNt4sSJdskll9grr7wS1OezzjrLmjVrZgMG\nDLAlS5bYiy++aLGxsTZq1CjbuXOn/f73vzfHcaxJkyb23HPP2e7du23RokXWq1cv83g8NnLkSPvy\nyy9tw4YNNmnSJP8l95W3dnjqqaesZcuWlpCQYFOmTLG8vDwbPny4tW7d2v8L93Duuece+/HHHwPK\n5syZU23geOWVV6xPnz7m8XisV69e9t5771leXp7/KtLY2Fj785//bOXl5fbEE09YeHi4eTweu+uu\nuyw3N9cefvhhcxzHwsPDbc6cOVZYWGj/+Mc/LC4uzjwej1199dWWk5NzTOvJz883M7MXX3zRTj31\nVEtNTbXo6GgLCwvzXx18aKCfNWuWxcfHB1w9W5tNmzbZqFGj7LzzzrOcnJwa2+Xl5dk111xjvXr1\nsu+//z6ofv/+/da9e3fr2bOnlZWV1bicyZMnB935/7XXXrO0tDSLiIiwAQMG2LXXXmvz58/3h7Sq\nNm/ebGeddVZQePj+++/t/PPPt7vvvtseeOCBw14hfbQKCgr83/2oqCjr27evDR482Hr37m1PP/20\nlZeXB7RPSUmx5ORku/HGG23ChAn2q1/9yiZPnuz/3VBYWGiPPPKIRUZGmsfjsalTpwZclW128DFr\n55xzjkVGRlqbNm1s4sSJAX90LV682E477TSLioqy4cOH2w8//GAPP/ywNW/e3CZPnuwPe48//rh1\n7NjRIiIirE+fPvbZZ59Zly5d7P7777dVq1aZ2cGDFD179jzs/2Gl5cuX24gRI2zixIl23XXX2aWX\nXup/fNyh1qxZY7/+9a/tlltusTvuuMPGjBlTbTuEBses5nMgxcXFOnDggJ544glNmjRJnTp1UlFR\nkf71r39pwIABKiws1Pvvv6/ExERNmjRJZqa//OUvGjZsmE499VR5vV698soruvnmm+U4To11Hg/X\nIACou+3bt+vGG2/Uiy++GDCxu/JKzTvvvFOLFi06qmUuXbpUixcvVkFBgS6++OKAU9SH+u6777Rw\n4UJt375dw4cP1wUXXHBM2xJKUlNT5fF46u1JIEBjUOscs+ou083OztbIkSMVERGh1q1bKz093f/I\nkQ0bNsjr9fonuyYlJSksLEyrV69WREREjXXdunWr360CEFKmTp2qlJSUoDvaN23aVJ07dz7sFbbV\nOffcc2sMY4c666yzdNZZZx318gGgOkc9+f/MM88MeB0TE+OfBJmTk6OEhISAezQlJiYqOztb0dHR\nNdYRzAAci7179+rFF19U+/bt9ctf/lLx8fHatWuXvvrqK2VlZQVNwIY7lJWVqaKioqG7AbjKMZ9D\n3LZtm/r06SNJKioqUkREREB9ZGSkfD5ftXURERFHdMNPAKjNP//5T02ZMkXPPfec+vfvr969e+vO\nO+9UZGSk/vKXvxzVlYU4/rZt26b/9//+n7Zt26adO3fqrrvu0tKlSxu6W4ArHNPtMkpLS7Vjxw7/\n7Qs8Hk/QHc3t4AUGNdYBwLFq2bKlnnzyST355JMN3RUcgbZt22rWrFmaNWtWQ3cFcJ1jCmZffPGF\nRo4c6Z+8Hxsbq5ycnIA2JSUliouLU0xMjP+xL4fWVT52pjqZmZlBj10BAABwo/j4+DrNaT1UnYNZ\nZmamunfv7r9AoLy8XB07dvQ/N7BSXl6ezjrrLMXFxQXV5efnBz2f7FAFBQUBNw0E3KjyPpocAAZO\nAL5wcLHqbhZ8tA47x6y6iZlZWVkKDw9XeXm5vF6vNm7cqB9++EHt27dXfHy8/zEUXq9XpaWlSktL\nU7t27YLqysrKDvtoDgAAgFBR6xGz4uJiZWZmynEc/fDDD4qNjVVBQYHef//9gMDmOI6mT58uSRo/\nfrwWL14sr9er3NxcTZw40f8ssqp1EyZM8NcBAACEulpvMNvQqj7/DHAjzqwAJxBfOLhYfeQWbrkP\nAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAwPU2+fL1xbb1/tdfbFuvTb78\nBuwRcHwQzAAArpdbXKBxH871vx734VzlFvMsZTQ+BDMAAACXIJgBAAC4BMEMAADAJQhmAAAALkEw\nAwAAcAmCGQAAgEsQzAAAAFyCYAYAAOASBDMAAACXIJgBAAC4BMEMAADAJQhmAAAALkEwAwAAcAmC\nGQAAgEsQzAAAAFyCYAYAAOASBDMAAACXIJgBAAC4BMEMAADAJQhmAAAALkEwAwAAcAmCGQAAgEsQ\nzAAAAFyCYAYAAOASBDMAAACXIJgBAAC4BMEMAADAJQhmAAAALkEwAwAAcAmCGQAAgEsQzAAAAFyC\nYAYAAOASBDMAAACXIJgBAAC4BMEMAADAJQhmAAAALnFEwaysrEwlJSXHuy8AAAAhLby2SjPTihUr\n9Omnn2r06NHq1KmTJMnn82nJkiVq3bq1tmzZooEDB6pVq1bHVAcAABDqaj1itnfvXnXq1Ek+n89f\nZmaaP3++unbtqr59+2rQoEGaN2+eKioq6lwHAACAwxwxi46ODirbsGGDvF6vUlNTJUlJSUkKCwvT\n6tWrFRERUae6bt261etGAQAAnIyOevJ/Tk6OEhISFBYW5i9LTExUdna2Nm/eXKc6AAAAHOaIWXWK\niooUERERUBYZGSmfzyczO6q6iIiIgNOkAAAAoeyoj5h5PJ6Ao17SwXlnZlanOgAAABx01MEsNjY2\n6NYZJSUlat68uWJiYo66LjY2tg7dBgAAaHyOOpilpqZq9+7dAWV5eXlKTU1Vx44dj6ouPz/ffzEA\nAABAqDtsMKt6O4v27dsrPj7eP2nf6/WqtLRUaWlpateu3VHVlZWVKS0trb63CQAA4KRU6+T/4uJi\nZWZmynEJ+5JOAAAZpElEQVQc/fDDD4qNjVVSUpLGjx+vxYsXy+v1Kjc3VxMnTlSTJk0k6ajqJkyY\n4K8DAAAIdYe9j9ngwYM1ePDggPIWLVpozJgx1b6nrnUAAAChjoeYAwAAuATBDAAAwCUIZgAAAC5B\nMAMAAHAJghkAAIBLEMwAAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJ\nghkAAIBLEMwAAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAAIBL\nEMwAAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAAIBLEMwAAABc\ngmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAAIBLEMwAAABcgmAGAADg\nEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHCJ8Lq+cdOmTVq/fr2aNWumrVu3asiQIWrZ\nsqV8Pp+WLFmi1q1ba8uWLRo4cKBatWolSbXWAQAAhLo6HTGrqKjQO++8o/T0dPXv31+9e/dWRkaG\nJGn+/Pnq2rWr+vbtq0GDBmnevHmqqKiQmdVYBwAAgDoGs3379qmwsFBlZWWSpMjISO3bt0/r16+X\n1+tVamqqJCkpKUlhYWFavXq1NmzYUGMdAAAA6hjMoqOjlZycrLffflslJSX66quvdN555yknJ0cJ\nCQkKCwvzt01MTFR2drY2b95cYx0AAACOYfL/ZZddpry8PD322GPq1KmTOnfurKKiIkVERAS0i4yM\nlM/nq7YuIiJCPp+vrl0AAABoVOo8+b+oqEidOnVSUVGR3nnnHXk8HoWFhQUcEZMkM5OZ+eur1gEA\nAOCgOh0xKy0t1SuvvKIhQ4Zo3LhxGjBggN59911FRUWppKQkoG1JSYmaN2+umJiYautiY2Pr3nsA\nAIBGpE7BbOfOnTIzRUdHS5KGDh0qx3GUmpqq3bt3B7TNy8tTamqqOnbsGFSXn5/vvxgAAAAg1NUp\nmCUmJqq8vFyFhYWSpPLycjVt2lRt2rRRfHy8f0K/1+tVaWmp0tLS1K5du6C6srIypaWl1dOmAAAA\nnNzqNMesWbNmGjdunBYtWqTk5GT5fD6NGTNGkZGRGj9+vBYvXiyv16vc3FxNnDhRTZo0kaSgugkT\nJvjrAAAAQl2dJ/936tRJnTp1Cipv0aKFxowZU+17aqsDAAAIdTwrEwAAwCUIZgAAAC5BMAMAAHAJ\nghkAAIBLEMwAAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAAIBL\nEMwAAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAAIBLEMwAAABc\ngmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAAIBLEMwAAABcgmAGAADg\nEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAAIBLEMwAAABcgmAGAADgEgQzAAAA\nlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJghkAAIBLhB/rAnbv3q1Vq1YpOjpaXbp0UXR0dH30\nCwAAIOQcUzBbuXKlli1bpksuuUQJCQmSJJ/PpyVLlqh169basmWLBg4cqFatWh22DgAAINTV+VRm\ndna2MjIyNG7cOH8oMzPNnz9fXbt2Vd++fTVo0CDNmzdPFRUVtdYBAACgjsHMzLRgwQL169dPzZs3\n95dv2LBBXq9XqampkqSkpCSFhYVp9erVtdYBAACgjqcyN2/erLy8PBUUFOif//ynvF6vzj77bBUX\nFyshIUFhYWH+tomJicrOzlZ0dHSNdd26dTv2LQEAADjJ1SmYbdu2TRERERo2bJiio6O1detWzZ07\nV6eeeqoiIiIC2kZGRsrn88nMguoiIiLk8/nq3nsAAIBGpE6nMktLS5WYmOi/AjM5OVnJyclq0aJF\nwBEx6eBpTzOTx+Optg4AAAAH1SmYxcTEqKysLKCsefPm+vrrr7V///6A8pKSEjVv3lwxMTEqKSkJ\nqouNja1LFwAAABqdOgWzdu3aac+ePSovL/eXlZeXKz09Xbt27Qpom5eXp9TUVHXs2FG7d+8OqMvP\nz/dfDAAAABDq6hTMkpKS1LZtW/3888+SpAMHDmjHjh3q3bu34uPjlZ2dLUnyer0qLS1VWlqa2rVr\nF1RXVlamtLS0etoUAACAk1udbzA7duxYffTRR8rLy5PP59NFF12k2NhYjR8/XosXL5bX61Vubq4m\nTpyoJk2aSFJQ3YQJE/x1AAAAoa7OwSwuLk6XXXZZUHmLFi00ZsyYat9TWx0AAECo4yHmAAAALkEw\nAwAAcAmCGQAAgEsQzAAAAFyCYAYAAOASdb4qEwAA1N0mX75yiwv8r0+JjldK88QG7BHcgGAGAEAD\nyC0u0LgP5/pfvzZiKsEMnMoEAABwC4IZAACASxDMAAAAXIJgBgAA4BIEMwAAAJcgmAEAALgEt8sA\nABwXVe/TJXGvLuBwCGYAgOOi6n26JO7VBRwOpzIBAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAA\nwCUIZgAAAC5BMAMAAHAJghkAAIBLEMwAAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAA\nAC5BMAMAAHAJghkAAIBLEMwAAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC4R3tAd\nAAA32uTLV25xgf/1KdHxSmme2IA9AhAKCGYAUI3c4gKN+3Cu//VrI6YSzAAcd5zKBAAAcAmCGQAA\ngEsQzAAAAFyCOWYAEKKqXuAgcZED0NAIZgAQoqpe4CBxkQPQ0DiVCQAA4BLHfMSsoqJCL730ktLT\n05Wamiqfz6clS5aodevW2rJliwYOHKhWrVpJUq11AAAAoe6Yj5gtX75cO3bskCSZmebPn6+uXbuq\nb9++GjRokObNm6eKiopa6wAAAHCMwWzTpk2Kj49XRESEJGnDhg3yer1KTU2VJCUlJSksLEyrV6+u\ntQ4AAADHcCpz79692rx5swYNGqSMjAxJUk5OjhISEhQWFuZvl5iYqOzsbEVHR9dY161bt2PYBADA\nicTVnMDxU+dgtmzZMg0ePDigrLi42H/0rFJkZKR8Pp/MLKguIiJCPp+vrl0AADQAruYEjp86ncrM\nzMzUmWeeqfDwwFznOE7AETHp4LwzM5PH46m2DgAAAAfV6YhZZmamFi5c6H994MABvfzyyzKzoKss\nS0pKFBcXp5iYGG3atCmoLj4+vi5dAAAAaHTqFMyuvfbagNdPPvmkRo8erbCwML388ssBdXl5eTrr\nrLMUFxenzz77LKAuPz9fPXr0qEsXAAAAGp16vcFsu3btFB8fr+zsbEmS1+tVaWmp0tLSqq0rKytT\nWlpafXYBAADgpFWvj2RyHEfjx4/X4sWL5fV6lZubq4kTJ6pJkyaSFFQ3YcIEfx0AAECoq5dgduut\nt/r/3aJFC40ZM6badrXVAQAAhDqelQkAAOASBDMAAACXIJgBAAC4BMEMAADAJQhmAAAALkEwAwAA\ncAmCGQAAgEsQzAAAAFyCYAYAAOAS9fpIJgAA3G6TL1+5xQUBZadExyuleWID9Qj4D4IZACCk5BYX\naNyHcwPKXhsxlWAGV+BUJgAAgEtwxAwAXIpTbkDoIZgBgEtxyg0IPZzKBAAAcAmCGQAAgEtwKhMA\n0GhUnZdXn3PymPOHE4FgBgBoNKrOy6vPOXnM+cOJwKlMAAAAlyCYAQAAuATBDAAAwCWYYwYAxxmT\nxgEcKYIZABxnTBoHcKQIZgCAEybc8eiLbev9rzlyCAQimAEATphd+/dqyicv+19z5BAIxOR/AAAA\nlyCYAQAAuATBDAAAwCUIZgAAAC7B5H8ACAHV3Uttf/mBBuoNgJoQzAAgBFR3L7Xnz7uygXoDoCac\nygQAAHAJghkAAIBLEMwAAABcgmAGAADgEgQzAAAAlyCYAQAAuATBDAAAwCUIZgAAAC5BMAMAAHAJ\nghkAAIBLEMwAAABcgmdlAnCV6h62fUp0vFKaJzZQj9wl3PHoi23rA8oYH6DxIJgBcJXqHrb92oip\nBI//s2v/Xk355OWAMsYHaDw4lQkAAOASdT5itnHjRi1cuFC7d+9W+/btNWrUKMXFxcnn82nJkiVq\n3bq1tmzZooEDB6pVq1aSVGsdAMCdqp5e3l9+oAF7AzRudTpiVlRUpKysLI0dO1bjxo1TXl6e3n33\nXUnS/Pnz1bVrV/Xt21eDBg3SvHnzVFFRITOrsQ4A4F6Vp5crfwhmwPFTpyNm2dnZGjlypCIiItS6\ndWulp6drwYIFWr9+vbxer1JTUyVJSUlJCgsL0+rVqxUREVFjXbdu3eprewCEsKpHdpgUD+BkU6dg\nduaZZwa8jomJUVxcnDZv3qyEhASFhYX56xITE5Wdna3o6Oga6whmAOpD1QsHmBQP4GRTL1dlbtu2\nTX369FF+fr4iIiIC6iIjI+Xz+WRmQXURERHy+Xz10QUAAICT3jFflVlaWqodO3aoX79+chwn4IiY\nJJmZzEwej6faOgAATlaV95U79GeTL7+hu4WT2DEfMfviiy80cuRIeTwexcbGKicnJ6C+pKREcXFx\niomJ0aZNm4Lq4uPjj7ULAAA0CO4rh/p2TEfMMjMz1b17d0VHR0uSOnTooN27dwe0ycvLU2pqqjp2\n7BhUl5+f778YAABQN1WP2nDEBjh51fmIWVZWlsLDw1VeXi6v16vi4mLt3r1b8fHxys7OVseOHeX1\nelVaWqq0tDSFh4cH1ZWVlSktLa0+tweASzS2Ryu5+VFIVY/acMQGOHnVKZitXbtW77//fsA9yBzH\n0fTp05WSkqLFixfL6/UqNzdXEydOVJMmTSRJ48ePD6ibMGGCvw5A49LYHq3EKSsAJ0Kdglnnzp01\nY8aMGuvHjBlTbXmLFi1qrAMAAAh1PCsTAADAJQhmAAAALkEwAwAAcIl6ufM/AAB14earXYGGQDAD\nADQYrnYFAhHMAACuUt1RtP3lBxqoN8CJRTADALhKdUfRnj/vygbqDXBiMfkfAADAJQhmAAAALkEw\nAwAAcAnmmAEIKY3t4eoAGheCGYCQ0tgerg6gceFUJgAAgEtwxAxAo8X9sACcbAhmABot7ocF4GTD\nqUwAAACXIJgBAAC4BKcyAaCRYW4dcPIimAFAA6ganurzXmoNMbeOMAjUD4IZADSAquHpZL+XGhda\nAPWDOWYAAAAuQTADAABwCYIZAACASxDMAAAAXIJgBgAA4BIEMwAAAJfgdhkAUEfV3burPu9HBiD0\nEMwAoI6qu3fXyX4/MgANi1OZAAAALsERMwC12uTLV25xgf81p+oA4PghmAGoVW5xgcZ9ONf/mlN1\nAHD8EMwA+FU9OibxIGoAOJEIZgD8qh4dk3gQNVAf6vpHT3XvYzpB40YwAwDgOKvrHz3VvY/pBI0b\nwQzAUeHeXY1bdf+/bj2dfTL1FThSBDMAR4V7dzVu1f3/uvV09snUV7eq7lRpXNNm2lO6L6CMP75O\nHIIZANfjKB1wfNR0ipU/vhoOwQzACVM1YB1puDqZjtLV9fQap+WOD8YVJxuCGXCSq+sNYBvi1hhV\nA5Zbw9WxqOvpNU7LHR+MK042BDPApY70MvkjuQFsTSHsyn/9LaDsZPqFxZEQHG9Vvzd8vnAiEMwA\nl6rPy+SP9/3JGuL0HUdCUJ9q+iwe+seLmz9fPDqt8SCYAThmnL7Dyc4Nn8Vj+UOFR6c1HgQzACGP\n06KoT3X9PLkhHKLhEcyA44xHqrgfvxBRn/g84VgQzIDj7EQ/UoWjPwBw8jrhwczn82nJkiVq3bq1\ntmzZooEDB6pVq1YnuhtAo8Vf6wBw8jqhwczMNH/+fA0bNkynnnqqUlNT9corr+jmm2+Wx+M5kV0B\nAKDR4mkZJ68TGsw2bNggr9er1NRUSVJSUpLCwsK0evVqdevW7UR2BTgqbpknxmlKAEeyHzjeT8tw\nyz6xMTqhwSwnJ0cJCQkKCwvzlyUmJio7O5tghnp3LA/nre7GklVvxno0N3KtS1+rex+nKQG4YT9w\noufOhpITGsyKiooUERERUBYRESGfz3ciu4FG6kjC1JE+nLfqTqe6nd5//mo9VZL0xbb1R3w3/ap/\n8Z7sd+EHgOrU9chaKB+Rc8zMTtTKFixYoJ07d+rqq6/2l73xxhsqKyvT5ZdfHtQ+MzNTBQUFQeUA\nAABuEx8fr969ex/TMk7oEbPY2Fjl5OQElJWUlCg+Pr7a9se6cQAAACeTE3opZMeOHbV79+6Asvz8\nfP/FAAAAAKHshAazdu3aKT4+XtnZ2ZIkr9ersrIypaWlnchuAAAAuNIJnWMmSbt27dLixYt1yimn\nKDc3V/369VNycvKJ7AIAAIArnfBgBgAAgOrxrEw0Crt379aqVasUHR2tLl26KDo6uqG7BACNFvvc\n46fBgtnGjRu1cOFC7d69W+3bt9eoUaMUFxdXY7kUGs/ZrG37JamiokIvvfSS0tPT/RdNhPq4rFy5\nUsuWLdMll1yihIQE/3sa+7jUNCabNm3S+vXr1axZM23dulVDhgxRy5YtJTX+MZGkbdu2KSMjQ16v\nV8nJybr00ksVFRVV67aH8riE+j63pnGpFKr73NrGJVT3uTWNSX3vcxvkAZVFRUXKysrS2LFjNW7c\nOOXl5endd99VcXFxteXSf56z2bVrV/Xt21eDBg3SvHnzVFFR0RCbcFzUNC6HWr58uXbs2OF/Herj\nkp2drYyMDI0bNy5gB9HYx6WmMamoqNA777yj9PR09e/fX71791ZGRoakxj8mknTgwAGtWrVKkyZN\n0u23367S0lJ9+eWXklTjtofyuIT6Pre2z0ulUNzn1jYuobrPrWlMjsc+t0GCWXZ2tkaOHKnWrVvr\ntNNOU3p6unJycmosl2p/zmZjUdv2S9KmTZsUHx8f8PSEUB+XBQsWqF+/fmrevHnAexr7uNQ0Jvv2\n7VNhYaHKysokSZGRkdq37+AjqBr7mEgH74uYnp6uJk2aqGnTpkpJSZHjOFq/fn2N2x7K47Jhw4aQ\n3ufWNC6VQnWfW9u4hOo+t6YxOR773AYJZmeeeWbABz0mJkZxcXH6r//6r2rLpdqfs9lY1DQukrR3\n715t3rxZXbp0CXhPKI/L5s2blZeXp4KCAv3zn//Un/70J3399deSGv+41DQm0dHRSk5O1ttvv62S\nkhJ99dVXOu+88yQ1/jGRDo5DePjBGRoHDhxQcXGxzjnnnFq3ffPmzSE5Lv379691nxOqn5f+/ftL\nCu19bk3jkpOTE7L73JrG5Hjsc10x+X/btm3q06dPreWh+JzNQ7d/2bJlGjx4cFCbUB6XrVu3KiIi\nQsOGDVN0dLS2bt2quXPnKjk5OeTG5dDPymWXXaa///3veuyxxzRq1Ch17txZUmh9VtasWaNPPvlE\n+/btk9frrXbbIyMj5fP5ZGYhOS47d+5USkpKQH2o7nOrGxf2ucHjsn379pDf51b3WanvfW6DHDE7\nVGlpqXbs2KF+/frVWu7xeAJSp3Tw/G1jdej2Z2Zm6swzz/Sn9UOF8riUlpYqMTHRfzVQcnKykpOT\n9fPPPyssLCxkxqXqd6WoqEidOnVS586d9c4772jVqlWSQuuzkpaWpvHjxyslJUVvvfVWjZ8HMwvp\ncTlUKO9zq44L+9yDqo4L+9zqv0P1vc9t8GD2xRdfaOTIkfJ4PLWWx8bGqqSkJKBNSUmJYmNjT1hf\nT6RDtz8zM1PPPfecZs6cqZkzZ6qgoEAvv/yyXn/99ZAel5iYGP95/UpxcXHat2+fYmJiQmZcDh2T\n0tJSvfLKKxoyZIjGjRunAQMG6N133/Vve6iMiSQlJCRo1KhR2rt3r6Kioqrd9ubNm4fUZ0UKHJe9\ne/f6y0N9n3vouCxdupR97v85dFwcx2Gfq8AxKSgoqPd9boMGs8zMTHXv3t2fvsvLy2ssD6XnbFbd\n/muuuUb33Xef/yc+Pl5XXnmlLrvsMqWmpobsuLRt21Z79uzxf24kqaysTAkJCSHzeak6Jjt37pSZ\n+V8PHTpUjuNo165dITMmh2rSpImaNWumTp06BW17Xl6eUlNTQ3pcmjVrJol9bqXKcbnlllvY5x6i\ncly6dOkS8vvcSpVjUlRUVO/73AYLZllZWQoPD1d5ebm8Xq82btyoH374ocby9u3bh8RzNmva/pqE\n8rhs377dfxhdOjghc+fOnerevXtIPJe1ujHZvHmzysvLVVhYKOngL9gmTZooMTExJMZk7969WrNm\njf/1xo0bddZZZ6lDhw5B215aWqq0tLSQHhfHcUJ6n1vbuNQklMelVatWatu2bUjuc2sak8TExHrf\n5zbII5nWrl2r+fPnB9zLw3EcjRgxQosWLQoqnz59uhITExv9czZrGpfK7a/05JNPavTo0f7UHcrj\nEh4ero8++kht2rSRz+dTWlqaTjvtNEmNe1xqG5M9e/bo22+/VXJysnw+n7p06aJOnTpJatxjIkm5\nubmaN2+eWrZsqW7duqlp06bq2bOnpNq3PRTHpUePHlq3bl2t+5xQHJfKz8uhQm2fW9u47NmzJyT3\nubWNyYYNG+p1n8uzMgEAAFyiwSf/AwAA4CCCGQAAgEsQzAAAAFyCYAYAAOASBDMAAACXIJgBAAC4\nBMEMAADAJQhmAAAALkEwAwAAcIn/D4l/lZe1oZ5bAAAAAElFTkSuQmCC\n",
"text": [
""
]
}
],
"prompt_number": 15
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Evaluating and Validating our Forecast\n",
"\n",
"The point of creating a probabilistic predictive model is to simultaneously make a forecast and give an estimate of how certain we are about it. \n",
"\n",
"However, in order to trust our prediction or our reported level of uncertainty, the model needs to be *correct*. We say a model is *correct* if it honestly accounts for all of the mechanisms of variation in the system we're forecasting.\n",
"\n",
"In this section, we **evaluate** our prediction to get a sense of how useful it is, and we **validate** the predictive model by comparing it to real data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**1.4** Suppose that we believe the model is correct. Under this assumption, we can **evaluate** our prediction by characterizing its **accuracy** and **precision** (see [here](http://celebrating200years.noaa.gov/magazine/tct/accuracy_vs_precision_556.jpg) for an illustration of these ideas). *What does the above plot reveal about the **accuracy** and **precision** of the PredictWise model?*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Your Answer Here*\n",
"\n",
"Spread represents the precision and probability represents the accuracy."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**1.5** Unfortunately, we can never be *absolutely sure* that a model is correct, just as we can never be absolutely sure that the sun will rise tomorrow. But we can test a model by making predictions assuming that it is true and comparing it to real events -- this constitutes a hypothesis test. After testing a large number of predictions, if we find no evidence that says the model is wrong, we can have some degree of confidence that the model is right (the same reason we're still quite confident about the sun being here tomorrow). We call this process **model checking**, and use it to **validate** our model.\n",
"\n",
"*Describe how the graph provides one way of checking whether the prediction model is correct. How many predictions have we checked in this case? How could we increase our confidence in the model's correctness?*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Your Answer Here*\n",
"\n",
"We did 10000 predictions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Gallup Party Affiliation Poll"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we will try to **estimate** our own win probabilities to plug into our predictive model.\n",
"\n",
"We will start with a simple forecast model. We will try to predict the outcome of the election based the estimated proportion of people in each state who identify with one one political party or the other.\n",
"\n",
"Gallup measures the political leaning of each state, based on asking random people which party they identify or affiliate with. [Here's the data](http://www.gallup.com/poll/156437/heavily-democratic-states-concentrated-east.aspx#2) they collected from January-June of 2012:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
"gallup_2012 = pd.read_csv(\"data/g12.csv\").set_index('State')\n",
"gallup_2012[\"Unknown\"] = 100 - gallup_2012.Democrat - gallup_2012.Republican\n",
"gallup_2012.head()"
],
"language": "python",
"metadata": {},
"outputs": [
{
"html": [
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Democrat | \n",
" Republican | \n",
" Dem_Adv | \n",
" N | \n",
" Unknown | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
" | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" 36.0 | \n",
" 49.6 | \n",
" -13.6 | \n",
" 3197 | \n",
" 14.4 | \n",
"
\n",
" \n",
" | Alaska | \n",
" 35.9 | \n",
" 44.3 | \n",
" -8.4 | \n",
" 402 | \n",
" 19.8 | \n",
"
\n",
" \n",
" | Arizona | \n",
" 39.8 | \n",
" 47.3 | \n",
" -7.5 | \n",
" 4325 | \n",
" 12.9 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" 41.5 | \n",
" 40.8 | \n",
" 0.7 | \n",
" 2071 | \n",
" 17.7 | \n",
"
\n",
" \n",
" | California | \n",
" 48.3 | \n",
" 34.6 | \n",
" 13.7 | \n",
" 16197 | \n",
" 17.1 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Dem_Adv | \n",
" Dem_Win | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" -0.8 | \n",
" -21.58 | \n",
"
\n",
" \n",
" | Alaska | \n",
" -10.6 | \n",
" -21.53 | \n",
"
\n",
" \n",
" | Arizona | \n",
" -0.4 | \n",
" -8.52 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" 12.5 | \n",
" -19.86 | \n",
"
\n",
" \n",
" | California | \n",
" 19.4 | \n",
" 24.06 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Dem_Adv | \n",
" Dem_Win | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Arkansas | \n",
" 12.5 | \n",
" -19.86 | \n",
"
\n",
" \n",
" | Georgia | \n",
" 3.6 | \n",
" -5.21 | \n",
"
\n",
" \n",
" | Kentucky | \n",
" 13.5 | \n",
" -16.23 | \n",
"
\n",
" \n",
" | Louisiana | \n",
" 9.4 | \n",
" -18.63 | \n",
"
\n",
" \n",
" | Mississippi | \n",
" 1.1 | \n",
" -13.18 | \n",
"
\n",
" \n",
" | Missouri | \n",
" 10.9 | \n",
" -0.14 | \n",
"
\n",
" \n",
" | Montana | \n",
" 3.9 | \n",
" -2.26 | \n",
"
\n",
" \n",
" | North Dakota | \n",
" 0.6 | \n",
" -8.63 | \n",
"
\n",
" \n",
" | Oklahoma | \n",
" 5.6 | \n",
" -31.30 | \n",
"
\n",
" \n",
" | South Carolina | \n",
" 0.1 | \n",
" -8.97 | \n",
"
\n",
" \n",
" | South Dakota | \n",
" 1.3 | \n",
" -8.41 | \n",
"
\n",
" \n",
" | Tennessee | \n",
" 5.0 | \n",
" -15.07 | \n",
"
\n",
" \n",
" | Texas | \n",
" 2.4 | \n",
" -11.77 | \n",
"
\n",
" \n",
" | West Virginia | \n",
" 18.8 | \n",
" -13.12 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Dem | \n",
" Rep | \n",
"
\n",
" \n",
" | Year | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | 2004 | \n",
" 48 | \n",
" 51 | \n",
"
\n",
" \n",
" | 2008 | \n",
" 53 | \n",
" 46 | \n",
"
\n",
" \n",
" | 2012 | \n",
" 51 | \n",
" 47 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Dem | \n",
" Rep | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" 37 | \n",
" 63 | \n",
"
\n",
" \n",
" | Alaska | \n",
" 34 | \n",
" 62 | \n",
"
\n",
" \n",
" | Arizona | \n",
" 44 | \n",
" 55 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" 45 | \n",
" 54 | \n",
"
\n",
" \n",
" | California | \n",
" 54 | \n",
" 45 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" pvi | \n",
" Dem_Adv | \n",
" Dem_Win | \n",
" obama_win | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" -23 | \n",
" -13.154902 | \n",
" -21.58 | \n",
" 0 | \n",
"
\n",
" \n",
" | Alaska | \n",
" -25 | \n",
" -22.954902 | \n",
" -21.53 | \n",
" 0 | \n",
"
\n",
" \n",
" | Arizona | \n",
" -8 | \n",
" -12.754902 | \n",
" -8.52 | \n",
" 0 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" -6 | \n",
" 0.145098 | \n",
" -19.86 | \n",
" 0 | \n",
"
\n",
" \n",
" | California | \n",
" 12 | \n",
" 7.045098 | \n",
" 24.06 | \n",
" 1 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Dem_Adv | \n",
" pvi | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" -14.684314 | \n",
" -28.58 | \n",
"
\n",
" \n",
" | Alaska | \n",
" -9.484314 | \n",
" -28.53 | \n",
"
\n",
" \n",
" | Arizona | \n",
" -8.584314 | \n",
" -15.52 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" -0.384314 | \n",
" -26.86 | \n",
"
\n",
" \n",
" | California | \n",
" 12.615686 | \n",
" 17.06 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Winner | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" 0 | \n",
"
\n",
" \n",
" | Alaska | \n",
" 0 | \n",
"
\n",
" \n",
" | Arizona | \n",
" 0 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" 0 | \n",
"
\n",
" \n",
" | California | \n",
" 1 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Obama | \n",
" Votes | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" 0.004234 | \n",
" 9 | \n",
"
\n",
" \n",
" | Alaska | \n",
" 0.008093 | \n",
" 3 | \n",
"
\n",
" \n",
" | Arizona | \n",
" 0.069288 | \n",
" 11 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" 0.031901 | \n",
" 6 | \n",
"
\n",
" \n",
" | California | \n",
" 0.994956 | \n",
" 55 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Pollster | \n",
" State | \n",
" MoE | \n",
" Obama (D) | \n",
" Romney (R) | \n",
" Sample | \n",
" obama_spread | \n",
" poll_date | \n",
" age_days | \n",
" Votes | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" Rasmussen Reports | \n",
" Washington | \n",
" 4.5 | \n",
" 52 | \n",
" 41 | \n",
" 500 | \n",
" 11 | \n",
" 2012-09-26 00:00:00 | \n",
" 6.0 | \n",
" 12 | \n",
"
\n",
" \n",
" | 1 | \n",
" Gravis Marketing | \n",
" Washington | \n",
" 4.6 | \n",
" 56 | \n",
" 39 | \n",
" 625 | \n",
" 17 | \n",
" 2012-09-21 12:00:00 | \n",
" 10.5 | \n",
" 12 | \n",
"
\n",
" \n",
" | 2 | \n",
" Elway Poll | \n",
" Washington | \n",
" 5.0 | \n",
" 53 | \n",
" 36 | \n",
" 405 | \n",
" 17 | \n",
" 2012-09-10 12:00:00 | \n",
" 21.5 | \n",
" 12 | \n",
"
\n",
" \n",
" | 3 | \n",
" SurveyUSA | \n",
" Washington | \n",
" 4.4 | \n",
" 54 | \n",
" 38 | \n",
" 524 | \n",
" 16 | \n",
" 2012-09-08 00:00:00 | \n",
" 24.0 | \n",
" 12 | \n",
"
\n",
" \n",
" | 4 | \n",
" SurveyUSA | \n",
" Washington | \n",
" 4.4 | \n",
" 54 | \n",
" 37 | \n",
" 524 | \n",
" 17 | \n",
" 2012-08-01 12:00:00 | \n",
" 61.5 | \n",
" 12 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" N | \n",
" poll_mean | \n",
" poll_std | \n",
" Votes | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" 9 | \n",
"
\n",
" \n",
" | Alaska | \n",
" NaN | \n",
" NaN | \n",
" NaN | \n",
" 3 | \n",
"
\n",
" \n",
" | Arizona | \n",
" 20 | \n",
" -5.500000 | \n",
" 4.559548 | \n",
" 11 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" 3 | \n",
" -20.333333 | \n",
" 4.041452 | \n",
" 6 | \n",
"
\n",
" \n",
" | California | \n",
" 20 | \n",
" 18.950000 | \n",
" 5.548589 | \n",
" 55 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" N | \n",
" poll_mean | \n",
" poll_std | \n",
" Votes | \n",
"
\n",
" \n",
" | State | \n",
" | \n",
" | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | Alabama | \n",
" NaN | \n",
" -100.00 | \n",
" 0.100000 | \n",
" 9 | \n",
"
\n",
" \n",
" | Alaska | \n",
" NaN | \n",
" -100.00 | \n",
" 0.100000 | \n",
" 3 | \n",
"
\n",
" \n",
" | Arizona | \n",
" 20 | \n",
" -5.50 | \n",
" 4.559548 | \n",
" 11 | \n",
"
\n",
" \n",
" | Arkansas | \n",
" 3 | \n",
" -100.00 | \n",
" 0.100000 | \n",
" 6 | \n",
"
\n",
" \n",
" | California | \n",
" 20 | \n",
" 18.95 | \n",
" 5.548589 | \n",
" 55 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"
\n",
" \n",
" \n",
" | \n",
" Pollster | \n",
" State | \n",
" MoE | \n",
" Obama (D) | \n",
" Romney (R) | \n",
" Sample | \n",
" obama_spread | \n",
" poll_date | \n",
" age_days | \n",
" Votes | \n",
"
\n",
" \n",
" \n",
" \n",
" | 427 | \n",
" SurveyUSA | \n",
" Kansas | \n",
" 4.4 | \n",
" 39 | \n",
" 48 | \n",
" 510 | \n",
" -9 | \n",
" 2011-11-19 12:00:00 | \n",
" 317.5 | \n",
" 6 | \n",
"
\n",
" \n",
" | 428 | \n",
" SurveyUSA | \n",
" Kansas | \n",
" 3.5 | \n",
" 31 | \n",
" 56 | \n",
" 800 | \n",
" -25 | \n",
" 2011-11-10 00:00:00 | \n",
" 327.0 | \n",
" 6 | \n",
"
\n",
" \n",
"
\n",
"