{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Maximum Likelihood Estimation\n",
    "\n",
    "**Functions**\n",
    "\n",
    "`np.log`, `scipy.special.gamma`, `scipy.special.gammaln`, `scipy.stats.norm.cdf`, `scipy.optimize.minimize`,\n",
    "`scipy.stats.t`, `np.var`, `np.std`, `scipy.stats.norm.pdf`\n",
    "\n",
    "### Exercise 20\n",
    "\n",
    "Simulate a set of i.i.d. Student's t random variables with degree of freedom\n",
    "parameter $\\nu=10$. Standardize the residuals so that they have unit variance\n",
    "using the fact that $V\\left[x \\right]=\\frac{\\nu}{\\nu-2}$. Use these to estimate\n",
    "the degree of freedom using maximum likelihood. Note that the likelihood of\n",
    "a standardized Student's t is\n",
    "\n",
    "$$f(x;\\nu,\\mu,\\sigma^{2})=\\frac{\\Gamma\\left(\\frac{\\nu+1}{2}\\right)}{\\Gamma\\left(\\frac{\\nu}{2}\\right)}\\,\\frac{1}{\\sqrt{\\pi(\\nu-2)}}\\,\\frac{1}{\\sigma}\\,\\frac{1}{\\left(1+\\frac{\\left(x-\\mu\\right)^{2}}{\\sigma^{2}(\\nu-2)}\\right)^{\\frac{\\nu+1}{2}}}$$\n",
    "\n",
    "where $\\Gamma\\left(\\right)$ is known as the gamma function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-22T10:07:07.404380Z",
     "iopub.status.busy": "2021-09-22T10:07:07.404380Z",
     "iopub.status.idle": "2021-09-22T10:07:07.490383Z",
     "shell.execute_reply": "2021-09-22T10:07:07.491380Z"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 21\n",
    "\n",
    "Repeat the previous exercise using daily, weekly and monthly S&P 500 and Hang Seng data.\n",
    "Note that it is necessary to remove the mean and standardize by the standard deviation\n",
    "error before estimating the degree of freedom. What happens over longer horizons?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-22T10:07:07.652380Z",
     "iopub.status.busy": "2021-09-22T10:07:07.652380Z",
     "iopub.status.idle": "2021-09-22T10:07:08.267380Z",
     "shell.execute_reply": "2021-09-22T10:07:08.267380Z"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 22\n",
    "\n",
    "Repeat the previous problem by estimating the mean and variance simultaneously with\n",
    "the degree of freedom parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-22T10:07:08.429252Z",
     "iopub.status.busy": "2021-09-22T10:07:08.428253Z",
     "iopub.status.idle": "2021-09-22T10:07:08.439273Z",
     "shell.execute_reply": "2021-09-22T10:07:08.439273Z"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 23\n",
    "\n",
    "Simulate a set of Bernoulli random variables $y_{i}$ where\n",
    "\n",
    "$$p_{i}=\\Phi\\left(x_{i}\\right)$$\n",
    "\n",
    "where $X_{i}\\sim N\\left(0,1\\right)$. (Note: $p_{i}$ is the probability of\n",
    "success and $\\Phi\\left(\\right)$ is the standard Normal CDF). Use this simulated data to\n",
    "estimate the Probit model where $p_{i}=\\Phi\\left(\\alpha_{0}+\\alpha_{1}x_{i}\\right)$ \n",
    "using maximum likelihood. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-22T10:07:08.928140Z",
     "iopub.status.busy": "2021-09-22T10:07:08.928140Z",
     "iopub.status.idle": "2021-09-22T10:07:09.049141Z",
     "shell.execute_reply": "2021-09-22T10:07:09.049141Z"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 24\n",
    "\n",
    "Estimate the asymptotic covariance of the estimated parameters in the previous.\n",
    "\n",
    "The derivative of the log-likelihood for a single observation is\n",
    "\n",
    "$$ \\frac{\\partial \\left\\{y_i \\ln\\left(\\Phi\\left(\\alpha_0+\\alpha_1 x_i \\right)\\right) + \\left(1-y_i\\right) \\ln\\left(1-\\Phi\\left(\\alpha_0+\\alpha_1 x_i \\right)\\right)\\right\\}}{\\partial \\alpha_j} $$\n",
    "\n",
    "which is\n",
    "\n",
    "$$ y_i \\frac{\\phi\\left(\\alpha_0+\\alpha_1 x_i \\right)}{\\Phi\\left(\\alpha_0+\\alpha_1 x_i \\right)} - \\left(1-y_i\\right)\\frac{\\phi\\left(\\alpha_0+\\alpha_1 x_i \\right)}{1-\\Phi\\left(\\alpha_0+\\alpha_1 x_i \\right)} $$\n",
    "\n",
    "for $\\alpha_0$ and \n",
    "\n",
    "$$ y_i x_i \\frac{\\phi\\left(\\alpha_0+\\alpha_1 x_i \\right)}{\\Phi\\left(\\alpha_0+\\alpha_1 x_i \\right)} - \\left(1-y_i\\right)x_i\\frac{\\phi\\left(\\alpha_0+\\alpha_1 x_i \\right)}{1-\\Phi\\left(\\alpha_0+\\alpha_1 x_i \\right)} $$\n",
    "\n",
    "for $\\alpha_1$ where $\\Phi(\\cdot)$ is the cdf of a standard Normal random variable and\n",
    "$\\phi(\\cdot)$ is the pdf of a standard Normal random variable.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-22T10:07:09.101192Z",
     "iopub.status.busy": "2021-09-22T10:07:09.101192Z",
     "iopub.status.idle": "2021-09-22T10:07:09.113197Z",
     "shell.execute_reply": "2021-09-22T10:07:09.113197Z"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}