{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Recursive State Estimation\n",
    "\n",
    "- Assume a state space model $p(x_t,z_t|z_{t-1})=p(x_t|z_t) p(z_t|z_{t-1})$. \n",
    "\n",
    "- Find a recursive state update $p(z_t|x^t)$ from a prior estimate $p(z_{t-1}|x^{t-1})$ and a new observation $x_t$\n",
    "\n",
    "$$\\begin{align*}\n",
    "\\underbrace{p(z_t|x^t)}_{\\text{posterior}} &= \\left(1/p(x^t)\\right) p(z_t,x^t) \\\\\n",
    "  &= \\left(1/p(x^t)\\right) \\int p(z_t,x_t,x^{t-1},z_{t-1}) \\mathrm{d} z_{t-1} \\\\\n",
    "  &= \\left(1/p(x^t)\\right) \\int p(z_t,x_t|x^{t-1},z_{t-1}) p(x^{t-1},z_{t-1}) \\mathrm{d} z_{t-1} \\\\\n",
    "    &= \\frac{p(x^{t-1})}{p(x^t)} \\int p(z_t,x_t|x^{t-1},z_{t-1}) p(z_{t-1}|x^{t-1}) \\mathrm{d} z_{t-1} \\\\\n",
    "     &= \\underbrace{\\frac{1}{p(x_t|x^{t-1})}}_{\\text{normalization}} \\underbrace{p(x_t|z_t)}_{\\text{observation}}\\int \\underbrace{p(z_t|z_{t-1})}_{\\text{transition}} \\underbrace{p(z_{t-1}|x^{t-1})}_{\\text{prior}} \\mathrm{d} z_{t-1}    \n",
    "\\end{align*}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gaussian Mixture Models\n",
    "\n",
    "- Assume a model $p(x_n,\\mathcal{C}_k) = \\pi_k \\mathcal{N}(x_n|\\mu_k,\\Sigma_k)$ \n",
    "- Let's rewrite this with a one-hot coding variable $z_k$:\n",
    "$$\\begin{align*}\n",
    "p(x_n|z_{nk}) &= \\mathcal{N}(x_n|\\mu_k,\\Sigma_k) \\\\\n",
    "p(z_{nk}) &= \\pi_k\n",
    "\\end{align*}$$\n",
    "- This leads to\n",
    "$$\\begin{align*}\n",
    "p(x_n|z_{n}) &= \\prod_k \\mathcal{N}(x_n|\\mu_k,\\Sigma_k)^{z_{nk}} \\\\\n",
    "p(z_{n}) &= \\prod_k \\pi_k^{z_{nk}}\n",
    "\\end{align*}$$\n",
    "- and generative model\n",
    "$$\\begin{align*}\n",
    "p(x,z) &= \\prod_n p(z_{n}) p(x_n|z_{n}) \\\\\n",
    "&= \\prod_n \\prod_k \\left( \\pi_k \\mathcal{N}(x_n|\\mu_k,\\Sigma_k)\\right)^{z_{nk}}\n",
    "\\end{align*}$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}