Classification | Clustering | |
1 | Class label $y_n$ is observed | Class label $z_n$ is latent |
2 | log-likelihood conditions on observed class $$\\propto \\sum_{nk} y_{nk} \\log \\mathcal{N}(x_n|\\mu_k,\\Sigma_k)$$ | log-likelihood marginalizes over latent classes $$\\propto \\sum_{n}\\log \\sum_k \\pi_k \\mathcal{N}(x_n|\\mu_k,\\Sigma_k)$$ |
3 | 'Hard' class selector $$\n", "y_{nk} = \\begin{cases} 1 & \\text{if $y_n$ in class $\\mathcal{C}_k$}\\\\\n", " 0 & \\text{otherwise} \\end{cases}\n", "$$ | 'Soft' class responsibility $$\\gamma_{nk} = p(\\mathcal{C}_k|x_n)$$ |
4 | \n", "Estimation: \n", "$$\\begin{align*}\n", "\\hat{\\pi}_k &= \\frac{1}{N}\\sum_n y_{nk} \\\\\n", "\\hat{\\mu}_k &= \\frac{\\sum_n y_{nk} x_n}{\\sum_n y_{nk}} \\\\\n", "\\hat{\\Sigma}_k &= \\frac{\\sum_n y_{nk} (x_n-\\hat\\mu_k)(x_n-\\hat\\mu_k)^T}{\\sum_n y_{nk}}\n", "\\end{align*}$$\n", " | \n",
"Estimation (1 update of M-step!) \n", "$$\\begin{align*}\n", "\\hat{\\pi}_k &= \\frac{1}{N}\\sum_n \\gamma_{nk} \\\\\n", "\\hat{\\mu}_k &= \\frac{\\sum_n \\gamma_{nk} x_n}{\\sum_n \\gamma_{nk}} \\\\\n", "\\hat{\\Sigma}_k &= \\frac{\\sum_n \\gamma_{nk} (x_n-\\hat\\mu_k)(x_n-\\hat\\mu_k)^T}{\\sum_n \\gamma_{nk}}\n", "\\end{align*}$$\n", " | \n",
"