{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kernel Smoothing Methods\n",
    "\n",
    "Chapter 6 Kernel Smoothing Methods\n",
    "\n",
    "*好吧，看到我照抄标题，就知道我没看懂这章节。假设，如果听课的话，老师讲得也差不多是书上的内容，但是，和自己看书完全是两回事。前者是别人读完书（还不止这一本）给你讲重点；后者是自己漫无目的地搜寻重点。本章节中，有好多与后面具体实现方法的联系，感觉作者是为后续方法做铺垫。那么问题就来了，在完全不知道后续方法的情况下，看这个章节就莫名其妙了，也不太懂里面的细节、技巧。看着公式和之前章节都差不多。好吧，等我自学完后面章节后，会再回来读这一章节的。*\n",
    "\n",
    "这一章中讲的 Kernel 方法，与后面高维空间的 Kernel 不是一回事。至于区别，后面的我还没学，不知道。\n",
    "\n",
    "*本章节，和[上一章节](./Splines.ipynb) 都是讲 smoothing，根据给定的数据，设计一个平滑的拟合。由于数据太复杂，不能直接套用 linear regression。所以需要分情况处理，比如分成多个 splines，或者应用 kernel。本章节和前一章节，分别对应第二章节提到的两个经典算法，即 KNN 和 linear regression。kernel smoothing 只是相对于 KNN 采用了更加复杂的函数对相邻一部分的变量进行处理； splines 是对一段一段的变量分别应用 linear regression。*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 内容概览\n",
    "\n",
    "1. *Nadaraya-Watson kernel-weighted average* \n",
    "\\begin{align}\n",
    "\\hat{f}(x_0) = \\frac{\\sum_{i=0}^{N}K_{\\lambda}(x_0,x_i)y_i}{\\sum_{i=1}^{N}K_{\\lambda}(x_0,x_i)}\n",
    "\\end{align}\n",
    "with the Epanechnikov quadratic kernel\n",
    "\\begin{align}\n",
    "K_{\\lambda}(x_0,x_i) = D\\left(\\frac{|x - x_0|}{\\lambda}\\right)\n",
    "\\end{align}\n",
    "with\n",
    "\\begin{align}\n",
    "D(t) = \\begin{cases}\\tfrac{3}{4}(1-t^2) & if\\ |t|\\leq 1 \\\\ 0 & \\textrm{otherwise}\\end{cases}\n",
    "\\end{align}\n",
    "\n",
    "1. *Structured Kernels*，其实就是给各个维度加上一个权重，引入正定矩阵 positive semidefinite matrix $\\mathbf{A}$，得到\n",
    "\\begin{align}\n",
    "K_{\\lambda,A}(x_0,x_i) = D\\left(\\frac{(x - x_0)^T\\mathbf{A}(x - x_0)}{\\lambda}\\right)\n",
    "\\end{align}\n",
    "\n",
    "1. *Parzen kernel density estimate*，核密度估计，\n",
    "\\begin{align}\n",
    "\\hat{f}_{X}(x_0) = \\frac{1}{N\\lambda}\\sum_{i=1}^{N}K_{\\lambda}(x_0,x_i)\n",
    "\\end{align}\n",
    "\n",
    "1. *Kernel Density Classification* \n",
    "\\begin{align}\n",
    "\\hat{\\Pr}(G=j|X=x_0) = \\frac{\\hat{\\pi}_j\\hat{f}_j(x_0)}{\\sum_{k=1}^J\\hat{\\pi}_k\\hat{f}_k(x_0)}\n",
    "\\end{align}\n",
    "有意思是，如果核分类是最终目标的话，没有必要先进行和密度估计，那样做甚至会有副作用。why？ 没想过。\n",
    "\n",
    "1. *Radial Basis Functions* 第5章有介绍过，将函数表达成 $M$ 个基函数，$f(x) = \\sum_{j=1}^M \\beta_jh_j(x)$。在 kernel 中，我们引入 *Renormalized* radial basis functions,\n",
    "\\begin{align}\n",
    "h_j(x) = \\frac{D(\\|x-\\xi_j\\|/\\lambda)}{\\sum_{k=1}^MD(\\|x-\\xi_k\\|/\\lambda)}.\n",
    "\\end{align}\n",
    "\n",
    "1. *Gaussian Mixture Models* \n",
    "\\begin{align}\n",
    "f(x) = \\sum_{m=1}^{M}\\alpha_m\\phi(x;\\mu_m,\\mathbf{\\Sigma}_m)\n",
    "\\end{align}\n",
    "with $\\sum_{m=1}^M \\alpha_m = 1$. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 具体内容\n",
    "\n",
    "没有。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}