{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework 8.1: Hierarchical models are hiding in plain sight (20 pts)\n",
    "\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Say we have a set of measurements, $\\mathbf{x} = \\{x_1, x_2,\\ldots\\}$. Each measurement has associated with is some measurement error such that $x_i \\sim \\text{Norm}(\\mu_i, s)$. Furthermore, there is a natural variability from measurement to measurement such that $\\mu_i \\sim \\text{Norm}(\\mu, \\sigma)$.\n",
    "\n",
    "**a)** Write a mathematical expression for the joint generative probability density, $\\pi(\\mathbf{x}, \\boldsymbol{\\mu}, \\mu, s, \\sigma)$, where $\\mathbf{x} = \\{x_1, x_2,\\ldots\\}$ and $\\boldsymbol{\\mu} = \\{\\mu_1, \\mu_2, \\ldots\\}$. You may assume that the prior may be separated such that $g(\\mu, \\sigma, s) = g(\\mu) g(\\sigma) g(s)$.\n",
    "\n",
    "**b)** Calculate $\\pi(\\mathbf{x}, \\mu, s, \\sigma)$. *Hint*: It may help to use the relation\n",
    "\n",
    "\\begin{align}\n",
    "\\frac{(x_i-\\mu_i)^2}{2s^2} + \\frac{(\\mu_i-\\mu)^2}{2\\sigma^2} = \\frac{(x_i-\\mu)^2}{2(s^2 + \\sigma^2)} + \\frac{\\left(\\mu_i - b\\right)^2}{2a^2},\n",
    "\\end{align}\n",
    "\n",
    "where\n",
    "\n",
    "\\begin{align}\n",
    "a^2 = \\frac{s^2\\sigma^2}{s^2+\\sigma^2}\\;\\;\\text{and}\\;\\;\n",
    "b = \\frac{x_i/s^2 + \\mu/s^2}{s^{-2} + \\sigma^{-2}},\n",
    "\\end{align}\n",
    "\n",
    "which may be found by completing the square. *Another hint*: It is useful to know about Gaussian integrals, namely that\n",
    "\n",
    "\\begin{align}\n",
    "\\int_{-\\infty}^\\infty \\mathrm{d}x\\,\\mathrm{e}^{-(x-b)^2 / 2 a^2} = \\sqrt{2\\pi a^2}.\n",
    "\\end{align}\n",
    "\n",
    "**c)** Write an expression fro $\\pi(\\mathbf{x}, \\mu, s, \\sigma)$ in the limit where the natural variability is much greater than the measurement error ($s \\ll \\sigma$). \n",
    "\n",
    "**d)** Finally, in this limit, write the expression for $\\pi(\\mathbf{x}, \\mu, \\sigma)$ and show that for this hierarchical model, the limit of $s\\ll \\sigma$ is equivalent to having a likelihood of $x_i \\sim \\text{Norm}(\\mu, \\sigma)\\;\\forall i$.\n",
    "\n",
    "This problem shows how a hierarchical model can reduce to a non-hierarchical one. In this case, it helps you see what is being neglected when you choose not to use a hierarchical model (since so many models actually are hierarchical if you don't make approximations like the one derived here)."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}