{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Probability Distributions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resources\n",
    "\n",
    "- [Second Chapter of Yoshua Bengio's Deep Learning book](http://www.deeplearningbook.org/contents/prob.html)\n",
    "- [QA Site](http://stats.stackexchange.com/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![boxes](images/boxes.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*\"Patterm Recognition and Machine Learning\", Bishop, 2006*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Random Variables: **_Boxes_**, and **_fruits_**.\n",
    "\n",
    "As we draw random elements we can gain an insight on the population in general."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Probability of choosing the red box.\n",
    "\n",
    "$p(box = red) = \\frac{\\# Red boxes}{\\# boxes} =  \\frac{1}{2}$\n",
    "\n",
    "$p(box = blue) = 1 - p(box = red)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Probability of choosing an apple in the red box\n",
    "\n",
    "$p(fruit = apple | box = red) = \\frac{2}{8}$\n",
    "\n",
    "Probability of choosing an apple in the blue box\n",
    "\n",
    "$p(fruit = apple | box = blue) = \\frac{3}{4}$\n",
    "\n",
    "Probability of choosing an orange in the red box\n",
    "\n",
    "$p(fruit = orange | box = red) = \\frac{6}{8}$\n",
    "\n",
    "Probability of choosing an apple in the blue box\n",
    "\n",
    "$p(fruit = orange | box = blue) = \\frac{1}{4}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Some generalization\n",
    "\n",
    "$p(fruit = apple, box = red) = p(fruit = apple | box = red)p(box = red)$ This is probability product rule.\n",
    "\n",
    "$p(fruit = orange, box = red) = p(fruit = orange | box = red)p(box = red)$ \n",
    "\n",
    "$p(fruit = apple, box = blue) = p(fruit = apple | box = blue)p(box = blue)$ This is probability product rule.\n",
    "\n",
    "$p(fruit = orange, box = blue) = p(fruit = orange | box = blue)p(box = blue)$ \n",
    "\n",
    "$p(fruit = orange) = p(fruit = orange, box = blue) + p(fruit = orange, box = red) $ This is probability sum rule\n",
    "\n",
    "$p(fruit = apple) = p(fruit = apple, box = blue) + p(fruit = apple, box = red) $ This is probability sum rule\n",
    "\n",
    "## The golden rules of probability\n",
    "\n",
    "$p(x) = \\int p(x,y) dy$\n",
    "\n",
    "$p(x,y) = p(x|y)p(y)$\n",
    "\n",
    "$p(x,y) = p(x)p(y)$ If event x and y are unrelated (i.e. drawing a fruit is independent from the box)\n",
    "\n",
    "\n",
    "## Bayes rule\n",
    "\n",
    "$p(x|y) = \\frac{p(y|x)p(x)}{p(y)}$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the moment this is it for probability distributions, we will revisit in further chapters."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}