{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3 向量正交"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 3.1 正交子向量\n",
    "\n",
    "### 3.1.1正交性\n",
    "向量x,y,如果$x^{T}y=x_1y_1+x_2y_2+\\cdots+x_ny_n=0$,则称之为两个向量正交（orthogonal) \n",
    "### 3.1.2 四个基本空间关系\n",
    "秩r的$m \\times n$矩阵,其行空间$\\dim C(A^T)=r$和零空间$\\dim N(A)=n-r$同属于$R^{n}$空间，而列空间$\\dim C(A)=r$和左零空间$\\dim N(A^T)=m-r$同属于$R^{m}$空间。 \n",
    "+ Ax=0  \n",
    "$\\begin{bmatrix}row_1 \\\\ row_2 \\\\ \\vdots \\\\ row_m \\end{bmatrix} [x]=0 \\Rightarrow \\begin{cases}row_1[x]=0 \\\\ row_2[x]=0 \\\\ \\vdots \\\\row_m[x]=0 \\end{cases}$  \n",
    "\n",
    "表明**行空间**与**零空间**将$R^{n}$拆分成两个正交的子空间；同理**列空间**与**左零空间**将$R^{m}$拆分成两个正交的子空间。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 投影矩阵"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2.1向量投影\n",
    "向量$\\mathbf{b}$投影到向量$\\mathbf{a}$上，那么投影的向量为$\\mathbf{p}=x\\mathbf{a}$，那么剩余的向量$\\mathbf{e}=\\mathbf{b}-x\\mathbf{a}$，由投影关系可知 $\\mathbf{e} \\bot \\mathbf{a}$  \n",
    "\n",
    "所以： \n",
    "$a^{T}e=a^{T}(b-p)=a^{T}(b-xa)=0 \\Rightarrow x=\\frac{a^{T}b}{a^{T}a}$ 投影后的向量$\\mathbf{p}=a\\frac{a^{T}b}{a^{T}a}$,由此可知，投影向量为$P=\\frac{aa^{T}}{a^{T}a}$  \n",
    "\n",
    "*性质*\n",
    "+ $p=p^T$  \n",
    "$(p^T)^T=\\frac{(aa^T)^T}{a^Ta}=\\frac{aa^T}{a^Ta}=p$ \n",
    "+ $P=p^2$  \n",
    "如果对一个向量作一次投影后再作一次投影，那么结果不变。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  3.2.2 空间投影 \n",
    "对于方程Ax=b不一定有解，但是向量Ax一定在A的列空间中，所以可以将b投影到A的列空间中，将$Ax=b$转换为$Ax=p$。 \n",
    "$p=Ax \\Rightarrow e=b-Ax \\Rightarrow A^{T}e=A^{T}(b-Ax)=0 \\Rightarrow A^{T}Ax=A^{T}b$   \n",
    "\n",
    "**所以**  \n",
    "+ $\\hat{x} = (A^TA)^{-1}A^Tb$  \n",
    "+ $P=A\\hat{x}=A(A^TA)^{-1}A^Tb$ \n",
    "+ 投影矩阵  \n",
    "$P=A(A^TA)^{-1}A^T$  \n",
    "\n",
    "如果A的各个列向量线性无关，则$A^TA$可逆"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 正交矩阵和Gram-schmit正交化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3.1 标准正交向量\n",
    "$q_i^{T}q_j=\\begin{cases}0 & i \\ne j  \\\\ 1 & i = j \\end{cases}$  \n",
    "将标准正交向量放入到矩阵中   \n",
    "$Q=\\Bigg[ q_1q_2\\ldots q_n \\Bigg]$ \n",
    "所以\n",
    "$$Q^TQ=\\Bigg[q_1^Tq_2^T\\ldots q_n^T \\Bigg]\\Bigg[ q_1q_2\\ldots q_n \\Bigg] = \\begin{bmatrix}1 & 0 & \\cdots & 0  \\\\ 0 & 1 & \\cdots & 0 \\\\ \\vdots & \\vdots & \\ddots & \\vdots \\\\ 0 & 0 & \\ldots & 1\\end{bmatrix} = I$$ 将$Q$称之为**标准正交矩阵**。若$Q$为方阵，则$Q^{T}=Q^{-1}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 3.3.2 Gram-schmit 正交化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "若干线性无关向量 $a,b,c,\\cdots$，求解出正交向量 $A,B,C,\\cdots$\n",
    "+ 确定A向量  \n",
    "令 $A=a$  \n",
    "+ 确定B向量  \n",
    "取 $b$ 在A上的法向量即为向量B，通过投影一章可知，法向量$e$等于$b-p$，其中$p$为$b$在$A$上的投影。所以$B=b-p=b-\\frac{A^Tb}{A^TA}A$  \n",
    "+ 确定C向量  \n",
    "$C$向量为分别垂直于$A$和$B$向量的法向量，用$c$向量分别减去$c$在$A$和$B$上的投影，可得到$C=c-\\frac{A^Tc}{A^TA}A-\\frac{B^Tc}{B^TB}B$ \n",
    "+ $\\ldots$   \n",
    "剩下的同理"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}