{ "cells": [ { "cell_type": "markdown", "id": "4d05d6b8", "metadata": {}, "source": [ "# Continuous Data and the Gaussian Distribution\n", "\n", "\n", "- **[1]** (##) We are given an IID data set $D = \\{x_1,x_2,\\ldots,x_N\\}$, where $x_n \\in \\mathbb{R}^M$. Let's assume that the data were drawn from a multivariate Gaussian (MVG),\n", "\\begin{align*}\n", "p(x_n|\\theta) = \\mathcal{N}(x_n|\\,\\mu,\\Sigma) = |2 \\pi \\Sigma|^{-\\frac{1}{2}} \\exp\\left\\{-\\frac{1}{2}(x_n-\\mu)^T\n", "\\Sigma^{-1} (x_n-\\mu) \\right\\}\n", "\\end{align*} \n", " (a) Derive the log-likelihood of the parameters for these data. \n", " (b) Derive the maximum likelihood estimates for the mean $\\mu$ and variance $\\Sigma$ by setting the derivative of the log-likelihood to zero.\n", "\n", "\n", "- **[2]** (#) Shortly explain why the Gaussian distribution is often preferred as a prior distribution over other distributions with the same support?\n", "\n", "\n", "- **[3]** (###) Proof that the Gaussian distribution is the maximum entropy distribution over the reals with specified mean and variance. \n", "\n", "\n", "- **[4]** (##) Proof that a linear transformation $z=Ax+b$ of a Gaussian variable $\\mathcal{N}(x|\\mu,\\Sigma)$ is Gaussian distributed as\n", "$$\n", "p(z) = \\mathcal{N} \\left(z \\,|\\, A\\mu+b, A\\Sigma A^T \\right) \n", "$$ \n", "\n", "\n", "- **[5]** (#) Given independent variables\n", "$x \\sim \\mathcal{N}(\\mu_x,\\sigma_x^2)$ and $y \\sim \\mathcal{N}(\\mu_y,\\sigma_y^2)$, what is the PDF for $z = A\\cdot(x -y) + b$? \n", "\n", "\n", "\n", "- **[6]** (###) Compute\n", "\n", "\\begin{equation*}\n", " \\int_{-\\infty}^{\\infty} \\exp(-x^2)\\mathrm{d}x \\,.\n", " \\end{equation*}\n", " \n", "\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "9ca67ac1", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Julia 1.5.2", "language": "julia", "name": "julia-1.5" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.5.2" } }, "nbformat": 4, "nbformat_minor": 5 }