{ "cells": [ { "cell_type": "markdown", "id": "d3027ced", "metadata": {}, "source": [ "# Bayesian Machine Learning\n", "\n", "\n", "- **[1]** (#) (a) Explain shortly the relation between machine learning and Bayes rule. \n", " (b) How are Maximum a Posteriori (MAP) and Maximum Likelihood (ML) estimation related to Bayes rule and machine learning?\n", "\n", "\n", "- **[2]** (#) What are the four stages of the Bayesian design approach?\n", "\n", "\n", "- **[3]** (##) The Bayes estimate is a summary of a posterior distribution by a delta distribution on its mean, i.e., \n", "$$\n", "\\hat \\theta_{bayes} = \\int \\theta \\, p\\left( \\theta |D \\right)\n", "\\,\\mathrm{d}{\\theta}\n", "$$\n", "Proof that the Bayes estimate minimizes the expected mean-squared error, i.e., proof that\n", "$$\n", "\\hat \\theta_{bayes} = \\arg\\min_{\\hat \\theta} \\int_\\theta (\\hat \\theta -\\theta)^2 p \\left( \\theta |D \\right) \\,\\mathrm{d}{\\theta}\n", "$$\n", "\n", "\n", "- **[4]** (###) We make $N$ IID observations $D=\\{x_1 \\dots x_N\\}$ and assume the following model\n", "$$\n", "x_k = A + \\epsilon_k \n", "$$\n", " where $\\epsilon_k = \\mathcal{N}(\\epsilon_k | 0,\\sigma^2)$ with known $\\sigma^2=1$. We are interested in deriving an estimator for $A$. \n", " (a) Make a reasonable assumption for a prior on $A$ and derive a Bayesian (posterior) estimate. \n", " (b) (##) Derive the Maximum Likelihood estimate for $A$. \n", " (c) Derive the MAP estimates for $A$. \n", " (d) Now assume that we do not know the variance of the noise term? Describe the procedure for Bayesian estimation of both $A$ and $\\sigma^2$ (No need to fully work out to closed-form estimates). \n", "\n", " \n", "- **[5]** (##) We consider the coin toss example from the notebook and use a conjugate prior for a Bernoulli likelihood function. \n", " (a) Derive the Maximum Likelihood estimate. \n", " (b) Derive the MAP estimate. \n", " (c) Do these two estimates ever coincide (if so under what circumstances)? \n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "id": "27db69a5", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Julia 1.6.3", "language": "julia", "name": "julia-1.6" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.6.3" } }, "nbformat": 4, "nbformat_minor": 5 }