{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7 奇异值分解, 左右逆和伪逆 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 7.1 奇异值分解\n",
    "### 7.1.1 定义 \n",
    "对于任意一矩阵$A$,都可以分解成$A=U\\Sigma V^T$,其中$U_{m \\times m},\\Sigma_{m \\times n},V_{n \\times n}$分别为正交矩阵，对角矩阵和正交矩阵。 \n",
    "### 7.1.2 证明 \n",
    "在$A$的列空间中找到一组正交基$v_1,v_2,\\ldots ,v_r$,在矩阵$A$的作用下转换为$A$的行空间的一组正交基$u_1,u_2,\\ldots,u_r$,用矩阵表达为$$AV=U\\Sigma \\Rightarrow A=U\\Sigma V^T$$ \n",
    "### 7.1.3 求解 \n",
    "+ 求解$V$  \n",
    "\n",
    "$A^TA=(V\\Sigma^TU^T)(U\\Sigma V^T)=V\\Sigma^T\\Sigma V^T$ ,通过矩阵的对角化将$V$和$\\Sigma^T\\Sigma=\\Sigma^2$ 元素求解出来，其中$V$为单位话的特征向量组成的矩阵，$\\Sigma^2$为特征值的平方。 \n",
    "+ 求解$U$  \n",
    "\n",
    "$AA^T=(U\\Sigma V^T)(V\\Sigma^TU)=U \\Sigma^2 U^T$,计算过程如上"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.1.4 实例  \n",
    "$A=\\begin{vmatrix}4 & 4 \\\\ -3 &3 \\end{vmatrix}$  \n",
    "+ $A^TA$   \n",
    "$A^TA=\\begin{vmatrix}4 & -3 \\\\ 4 &3 \\end{vmatrix}\\begin{vmatrix}4 & 4 \\\\ -3 &3 \\end{vmatrix}=\\begin{vmatrix}25 & 7 \\\\ 7 &25 \\end{vmatrix}$,特征值和标准化特征向量分别为:$\\sigma_1 = 32, v_1=\\begin{bmatrix}\\frac{1}{\\sqrt{2}} \\\\ \\frac{1}{\\sqrt{2}} \\end{bmatrix};\\sigma_2 = 18, v_2=\\begin{bmatrix}\\frac{1}{\\sqrt{2}} \\\\ -\\frac{1}{\\sqrt{2}} \\end{bmatrix}$   \n",
    "+ $AA^T$ \n",
    "$AA^T=\\begin{vmatrix}4 & 4 \\\\ -3 &3 \\end{vmatrix}\\begin{vmatrix}4 & -3 \\\\ 4 &3 \\end{vmatrix}=\\begin{vmatrix}32 & 0 \\\\ 0 &18 \\end{vmatrix}$, 特征值和标准化特征向量分别为$\\sigma_1 = 32, u_1=\\begin{bmatrix}1 \\\\ 0 \\end{bmatrix};\\sigma_2 = 18, u_2=\\begin{bmatrix}0 \\\\ 1 \\end{bmatrix}$\n",
    "+ 选择符号  \n",
    "在$\\Sigma ^2$开根号处理过程中，由于会出现正负两种情况，所以具体一个符号取值需要根据$A=U\\Sigma V^T$实际情况来判定。当$\\lambda_1=\\sqrt{32},v_1=\\begin{bmatrix}\\frac{1}{\\sqrt{2}} \\\\ \\frac{1}{\\sqrt{2}} \\end{bmatrix}$时，$u_1=\\begin{bmatrix}1 \\\\ 0 \\end{bmatrix}$,当$\\lambda_2=\\sqrt{8},v_2=\\begin{bmatrix}\\frac{1}{\\sqrt{2}} \\\\ -\\frac{1}{\\sqrt{2}} \\end{bmatrix}, u_2=\\begin{bmatrix}0 \\\\ \\underline{-1} \\end{bmatrix}$ \n",
    "+ 综合   \n",
    "$U=\\begin{bmatrix} 1 & 0 \\\\ 0 & -1 \\end{bmatrix},\\Sigma=\\begin{bmatrix} \\sqrt{32} & 0 \\\\ 0 & \\sqrt{8} \\end{bmatrix},V=\\begin{bmatrix} \\frac{1}{\\sqrt{2}} &\\frac{1}{\\sqrt{2}} \\\\ \\frac{1}{\\sqrt{2}} & -\\frac{1}{\\sqrt{2}} \\end{bmatrix}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7.2 左右逆"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.2.1 左逆\n",
    "对于矩阵$A_{m \\times n}$,其秩$Rank(A)=n \\lt m$,矩阵$rank(A^TA)=n$，因此$(A^TA)^{-1}$存在，所以$$(A^TA)^{-1}A^TA=I$$,则$(A^TA)^{-1}A^T$为矩阵$A$的左逆"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7.2.2 右逆 \n",
    "对于矩阵$A_{m\\times n}$,其秩$Rank(A)=m<n$,矩阵$rank(AA^T)=m$，因此$(AA^T)^{-1}$存在，所以$$AA^T(AA^T)^{-1}=I$$,则$A^T(AA^T)^{-1}$为矩阵$A$的右逆"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7.3 伪逆\n",
    "对于矩阵$A_{m \\times n}$,其秩$Rank(A)\\lt min(m,n)$,伪逆记为$A^+$   \n",
    "将矩阵$A$进行奇异值分解，$A=U\\Sigma V^T$,其中对角矩阵为$$\\Sigma=\\begin{bmatrix} \\sigma_1 & 0 & \\ldots & 0 & 0 \\\\ 0 & \\sigma_2 & \\ldots & 0 & 0 \\\\ \\vdots & \\vdots & \\ddots & \\vdots & \\vdots \\\\ 0 & 0 & \\ldots & \\sigma_r & 0 \\\\ 0 & 0 & \\ldots & 0 & [ 0 ]\\end{bmatrix}$$  \n",
    "则 $$\\Sigma^{+}=\\begin{bmatrix} \\frac{1}{\\sigma_1} & 0 & \\ldots & 0 & 0 \\\\ 0 & \\frac{1}{\\sigma_2} & \\ldots & 0 & 0 \\\\ \\vdots & \\vdots & \\ddots & \\vdots & \\vdots \\\\ 0 & 0 & \\ldots & \\frac{1}{\\sigma_r} & 0 \\\\ 0 & 0 & \\ldots & 0 & [ 0 ]\\end{bmatrix}$$ \n",
    "其中$\\Sigma$为$m \\times n$矩阵，$\\Sigma^+$为$n \\times m$矩阵   \n",
    "$A^+=V\\Sigma^{+}U^T$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}