{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Bayes Theorem\n",
    "\n",
    "### Rohan L. Fernando\n",
    "\n",
    "### June 2015"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Motivation\n",
    "\n",
    "\n",
    "In whole-genome analyses, the number $k$ of marker covariates typically\n",
    "exceeds the number of $n$ of observations. In this situation, least\n",
    "squares methods cannot be used to simultaneously estimate the effects of\n",
    "all the $k$ marker covariates. One of the most widely used methods to\n",
    "overcome this problem is Bayesian inference, where prior information\n",
    "about marker effects is combined with the data to make inferences about\n",
    "the marker effects. In Bayesian inference, inferences are based on\n",
    "conditional probabilities, and the Bayes theorem is a statement on\n",
    "conditional probability."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##Conditional Probability of $X$ Given $Y$\n",
    "\n",
    "\n",
    "Suppose $X$ and $Y$are two random variables with joint probability\n",
    "distribution $\\Pr(X,Y)$. Then, the conditional probability of $X$ given\n",
    "$Y$ is given by Bayes theorem as\n",
    "\n",
    "$$\\Pr(X|Y)  =  \\frac{\\Pr(X,Y)}{\\Pr(Y)} \\tag{1}$$\n",
    "\n",
    "where $\\Pr(Y)$ is the probability distribution of $Y$. \n",
    "\n",
    "Similarly, thethe conditional probability of $Y$ given $X$ is\n",
    "\n",
    "$$\\Pr(Y|X)=\\frac{\\Pr(X,Y)}{\\Pr(X)},$$ \n",
    "\n",
    "which upon rearranging gives\n",
    "\n",
    "$$\\Pr(X,Y)=\\Pr(Y|X)\\Pr(X). \\tag{2}$$\n",
    "\n",
    "Then, substituting (2) in (1) gives\n",
    "\n",
    "$$\\begin{eqnarray} \n",
    "\\Pr(X|Y)  &= &\\frac{\\Pr(X,Y)}{\\Pr(Y)}\\\\\n",
    "          &= &\\frac{\\Pr(Y|X)\\Pr(X)}{\\Pr(Y)},\n",
    "\\end{eqnarray}$$\n",
    "\n",
    "which is the form of the formula that is used for inference of $X$ given\n",
    "$Y.$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Bayes Theorem by Example\n",
    "----------------------------------------------\n",
    "\n",
    "Here we consider a simple example to justify the formula\n",
    "(1). The following table gives the joint distribution of\n",
    "smoking and lung cancer in a hypothetical population of 1,000,000\n",
    "individuals.\n",
    "\n",
    "$$\n",
    "\\begin{array}{c|lcr}\n",
    "\\text{Cancer} & \\text{Yes} & \\text{No} & \\text{} \\\\\n",
    "\\hline\n",
    "\\text{Yes} & 42,500 & 7,500 & 50,000 \\\\\n",
    "\\text{No}  & 207,500 & 742,500 & 950,000 \\\\\n",
    " & 250,000 & 750,000 \n",
    "\\end{array}\n",
    "$$\n",
    "\n",
    "Given these numbers, consider how you would compute the relative\n",
    "frequency of lung cancer among smokers. There are a total of 250,000\n",
    "smokers in this population, and among these 250,000 individuals, 42,500\n",
    "have lung cancer. So, relative frequency of lung cancer among smokers is\n",
    "$\\frac{42,500}{250,000}$. As we reason below, this relative frequency is\n",
    "also the conditional probability of lung cancer given the individual is\n",
    "a smoker."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "1.  The frequentist definition of probability of an event is the\n",
    "    limiting value of its relative frequency in a “large” number of\n",
    "    trials.\n",
    "\n",
    "2.  Suppose we sample with replacement individuals from the 250,000\n",
    "    smokers and compute the relative frequency of the incidence of lung\n",
    "    cancer.\n",
    "\n",
    "3.  It can be shown that as the sample size goes to infinity, this\n",
    "    relative frequency will approach $\\frac{42,500}{250,000}=0.17$.\n",
    "\n",
    "4.  This ratio can also be written as\n",
    "    $$\\frac{42,500/1,000,000}{250,000/1,000,000}=0.17.$$\n",
    "\n",
    "5.  The ratio in the numerator is the joint probability of smoking and\n",
    "    lung cancer, and the ratio in the denominator is the marginal\n",
    "    probability of smoking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Using existing profile dir: u'/Users/rohan/.ipython/profile_default'\n",
      "[NbConvertApp] Converting notebook BayesTheorem.ipynb to slides\n",
      "[NbConvertApp] Support files will be in BayesTheorem_files/\n",
      "[NbConvertApp] Loaded template slides_reveal.tpl\n",
      "[NbConvertApp] Writing 203424 bytes to BayesTheorem.slides.html\n"
     ]
    }
   ],
   "source": [
    ";ipython nbconvert --to slides BayesTheorem.ipynb"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Julia 0.3.7",
   "language": "julia",
   "name": "julia 0.3"
  },
  "language_info": {
   "name": "julia",
   "version": "0.3.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}