\n",
" | Generative | Discriminative (ML) |
\n",
"\n",
" | 1 | Like density estimation, model joint prob.\n",
"$$p(\\mathcal{C}_k) p(x|\\mathcal{C}_k) = \\pi_k \\mathcal{N}(\\mu_k,\\Sigma)$$ | Like (linear) regression, model conditional\n",
"$$p(\\mathcal{C}_k|x,\\theta)$$ |
\n",
"\n",
" | 2 | Leads to softmax posterior class probability\n",
"$$ p(\\mathcal{C}_k|x,\\theta ) = e^{\\theta_k^T x}/Z$$\n",
"with structured $\\theta$ | Choose also softmax posterior class probability\n",
"$$ p(\\mathcal{C}_k|x,\\theta ) = e^{\\theta_k^T x}/Z$$\n",
"but now with 'free' $\\theta$ |
\n",
"\n",
" | 3 | For Gaussian $p(x|\\mathcal{C}_k)$ and multinomial priors,\n",
"$$\\hat \\theta_k = \\left[ {\\begin{array}{c}\n",
" { - \\frac{1}{2} \\mu_k^T \\sigma^{-1} \\mu_k + \\log \\pi_k} \\\\\n",
" {\\sigma^{-1} \\mu_k } \\\\\n",
"\\end{array}} \\right]$$\n",
"in one shot. | Find $\\hat\\theta_k$ through gradient-based adaptation\n",
"$$\\nabla_{\\theta_k}\\mathrm{L}(\\theta) = \\sum_n \\Big( y_{nk} - \\frac{e^{\\theta_k^T x_n}}{\\sum_{k^\\prime} e^{\\theta_{k^\\prime}^T x_n}} \\Big)\\, x_n$$ |
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## ![](\"./figures/Figure4.9.png\")
![](\"./figures/Figure4.14a.png\")