{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Learning objectives\n",
    "\n",
    "The learning objectives today are quite simple. We will learn what IID (independent and identically distributed) means and how it applies to samples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Second most important acronym\n",
    "\n",
    "So we come to the second most important acronym in statistics: iid. And the most important assumption. This will be important not only for the bootstrapping (up next) but throughout the class. \n",
    "\n",
    "This assumption has two parts:\n",
    "\n",
    "1. Independent\n",
    "2. Identically distributed\n",
    "\n",
    "The second assumption is pretty easy, so we will start there."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Identically distributed\n",
    "\n",
    "This means that the samples come from the same r.v. More specifically, this means that we sample from the same function each time. Below would be some identically distributed samples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def rv1():\n",
    "    return 1\n",
    "\n",
    "identically_distributed_samples = [rv1() for _ in range(10)]\n",
    "\n",
    "identically_distributed_samples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In this case the samples are all the same (remember that a r.v. does not have to be random). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "And below is an example of samples that are not:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 1, 1, 1, 1, 2, 2, 2, 2, 2]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def rv2():\n",
    "    return 2\n",
    "\n",
    "not_identically_distributed_samples = [rv1() for _ in range(5)] + [rv2() for _ in range(5)]\n",
    "\n",
    "not_identically_distributed_samples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now there is something subtle about this. Say we are measuring the height of pigs at a farm. We randomly pick a pig and then we measure the height. This is a real life r.v. Now let's say that we accidentally picked up a chicken and measured it and put it in the data set. Then that is not identically distributed right!\n",
    "\n",
    "But if instead you are measuring the height of farm animals, then picking a pig then a chicken would be right up your alley. So notice that the answer will greatly depend on how your problem is framed.\n",
    "\n",
    "The most important thing to remember here is to always know what you are sampling from.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Independent\n",
    "\n",
    "The next property that we will discuss is very hard to wrap your mind around, but this is independence. Two r.v. are independent if they do not depend on each other. In function speak, this would mean that the independent variables don't take as parameters each other (or any r.v. that is dependent on the other). Okay, confusing right, well, let's look at an example.\n",
    "\n",
    "These r.v. are independent:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 5)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def ind1():\n",
    "    return 4\n",
    "\n",
    "def ind2():\n",
    "    return 5\n",
    "\n",
    "ind1(), ind2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "These are not:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 5, 6, 7, 8]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 3\n",
    "\n",
    "def not_ind():\n",
    "    global x\n",
    "    x += 1\n",
    "    return x\n",
    "\n",
    "not_ind_samples = [not_ind() for _ in range(5)]\n",
    "not_ind_samples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Notice that one depends on the other's outcome. Another way of thinking about it is: if I know the outcome of one r.v., does that give me information as to the outcome of another?\n",
    "\n",
    "Okay, let's try a more complicated one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[3, 3, 3, 3, 3]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from numpy.random import rand\n",
    "\n",
    "x = 3\n",
    "\n",
    "def not_ind_adv():\n",
    "    global x\n",
    "    if rand() < .01:\n",
    "        x += 1\n",
    "    return x\n",
    "\n",
    "not_ind_samples = [not_ind_adv() for _ in range(5)]\n",
    "\n",
    "not_ind_samples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Are these samples independent of each other? \n",
    "\n",
    "Actually not! One way of thinking about it is that the sample values will always be either x or 3 + x, therefore they are dependent on each other.\n",
    "\n",
    "This is one of the most confusing parts of statistics and rightly so, and we would skip it if we could, but this assumption is so vitally important that there is no way to get around it.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Why this matters\n",
    "\n",
    "This is probably one of the most important lessons that you will learn here, because this is one of the ones that is most often messed up. Knowing what distribution you are sampling from and whether two things are independent are crucial assumptions that you will often make in statistics, and getting this wrong can often ruin entire projects (think of the Truman election or the recent facebook CL scandal)! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Learning objectives\n",
    "\n",
    "The learning objectives today are quite simple. We will learn what IID (independent and identically distributed) means and how it applies to samples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Comprehension\n",
    "\n",
    "1.\tWhy is it so important to define your population of interest?\n",
    "2.\tIf two r.v. are not correlated, can they be dependent?\n",
    "3.\tWhy do we care about the sample values being independent rather than the r.v. that made them being independent?\n",
    "4.\tHow can we tell if our samples are IID? \n",
    "5.\tWhat is a common way to get IID samples from a population?\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}