{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 行向量对元素求导\n",
    "\n",
    "设$y^T=[y_1 ... y_n]$是n维行向量，$x$是元素\n",
    "\n",
    "$\\frac{\\partial y^T}{\\partial x}=[ \\frac{\\partial y_1}{\\partial x} ... \\frac{\\partial y_n}{\\partial x} ] $\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 列向量对元素求导\n",
    "\n",
    "设$y=\\begin{bmatrix}y_1\\\\ ...\\\\ y_n\\end{bmatrix}$是m维列向量，$x$是元素\n",
    "\n",
    "则$\\frac{\\partial y}{\\partial x}=\\begin{bmatrix} \\frac{\\partial y_1}{\\partial x}\\\\ ... \\\\\n",
    "\\frac{\\partial y_m}{\\partial x} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 矩阵对元素求导\n",
    "\n",
    "设$Y=\\begin{bmatrix}y_1 & ... & y_{1n}\\\\...&...& ...\\\\ y_{m1} & ... & y_{mn}\\end{bmatrix}$是$m*n$维矩阵，$x$是元素\n",
    "\n",
    "则$\\frac{\\partial Y}{\\partial x}=\\begin{bmatrix} \\frac{\\partial y_{11}}{\\partial x} & ... &\\frac{\\partial y_{1n}}{\\partial x}\\\\...&...& ... \\\\\n",
    "\\frac{\\partial y_{m1}}{\\partial x}& ... & \\frac{\\partial y_{mn}}{\\partial x} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 元素对行向量求导\n",
    "\n",
    "设$y$是元素，$x^T=[x_1 ... x_q]$是q维行向量\n",
    "\n",
    "则$\\frac{\\partial y}{\\partial x^T}=[ \\frac{\\partial y}{\\partial x_1} ... \\frac{\\partial y}{\\partial x_q} ] $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 元素对列向量求导\n",
    "\n",
    "设$y$是元素，$x=\\begin{bmatrix}x_1\\\\ ...\\\\ x_n\\end{bmatrix}$是p维列向量\n",
    "\n",
    "则$\\frac{\\partial y}{\\partial x}=\\begin{bmatrix} \\frac{\\partial y}{\\partial x_1}\\\\ ... \\\\\n",
    "\\frac{\\partial y}{\\partial x_p} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 元素对矩阵求导\n",
    "\n",
    "设$y$是元素，$X=\\begin{bmatrix}x_1 & ... & x_{1q}\\\\...&...& ...\\\\ x_{p1} & ... & x_{pq}\\end{bmatrix}$是$p*q$维矩阵\n",
    "\n",
    "则$\\frac{\\partial y}{\\partial X}=\\begin{bmatrix} \\frac{\\partial y}{\\partial x_{11}} & ... &\\frac{\\partial y}{\\partial x_{1q}}\\\\...&...& ... \\\\\n",
    "\\frac{\\partial y}{\\partial x_{p1}}& ... & \\frac{\\partial y}{\\partial x_{pq}} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 行向量对列向量求导\n",
    "\n",
    "设$y^T=[y_1 ... y_n]$是n维行向量，$x=\\begin{bmatrix}x_1\\\\ ...\\\\ x_p\\end{bmatrix}$是p维列向量\n",
    "\n",
    "则$\\frac{\\partial y^T}{\\partial x}=\\begin{bmatrix} \\frac{\\partial y_{1}}{\\partial x_{1}} & ... &\\frac{\\partial y_{n}}{\\partial x_{1}}\\\\...&...& ... \\\\\n",
    "\\frac{\\partial y_1}{\\partial x_{p}}& ... & \\frac{\\partial y_n}{\\partial x_{p}} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 列向量对行向量求导\n",
    "\n",
    "设$y=\\begin{bmatrix}y_1\\\\ ...\\\\ y_m\\end{bmatrix}$是m维列向量，$x^T=[x_1 ... x_q]$是q维行向量\n",
    "\n",
    "则$\\frac{\\partial y}{\\partial x^T}=\\begin{bmatrix} \\frac{\\partial y_{1}}{\\partial x_{1}} & ... &\\frac{\\partial y_{1}}{\\partial x_{q}}\\\\...&...& ... \\\\\n",
    "\\frac{\\partial y_m}{\\partial x_{1}}& ... & \\frac{\\partial y_m}{\\partial x_{q}} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 行向量对行向量求导\n",
    "\n",
    "设$y^T=[y_1 ... y_n]$是n维行向量，$x^T=[x_1 ... x_q]$是q维行向量\n",
    "\n",
    "则$\\frac{\\partial y^T}{\\partial x^T}=[ \\frac{\\partial y^T}{\\partial x_1} ... \\frac{\\partial y^T}{\\partial x_q} ] $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 列向量对列向量求导\n",
    "\n",
    "设$y=\\begin{bmatrix}y_1\\\\ ...\\\\ y_m\\end{bmatrix}$是m维列向量，$x=\\begin{bmatrix}x_1\\\\ ...\\\\ x_p\\end{bmatrix}$是p维列向量\n",
    "\n",
    "则$\\frac{\\partial y}{\\partial x}=\\begin{bmatrix} \\frac{\\partial y_1}{\\partial x} \\\\...\\\\\n",
    "\\frac{\\partial y_m}{\\partial x} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 矩阵对行向量求导\n",
    "\n",
    "设$Y=\\begin{bmatrix}y_1 & ... & y_{1n}\\\\...&...& ...\\\\ y_{m1} & ... & y_{mn}\\end{bmatrix}$是$m*n$维矩阵,$x^T=[x_1 ... x_q]$是q维行向量\n",
    "\n",
    "则$\\frac{\\partial Y}{\\partial x^T}=[ \\frac{\\partial Y}{\\partial x_1} ... \\frac{\\partial Y}{\\partial x_q} ] $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 矩阵对列向量求导\n",
    "\n",
    "设$Y=\\begin{bmatrix}y_1 & ... & y_{1n}\\\\...&...& ...\\\\ y_{m1} & ... & y_{mn}\\end{bmatrix}$是$m*n$维矩阵,$x=\\begin{bmatrix}x_1\\\\ ...\\\\ x_n\\end{bmatrix}$是p维列向量\n",
    "\n",
    "则$\\frac{\\partial Y}{\\partial x}=\\begin{bmatrix} \\frac{\\partial y_{11}}{\\partial x} & ... &\\frac{\\partial y_{1n}}{\\partial x}\\\\...&...& ... \\\\\n",
    "\\frac{\\partial y_{m1}}{\\partial x}& ... & \\frac{\\partial y_{mn}}{\\partial x} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 行向量对矩阵求导\n",
    "\n",
    "设$y^T=[y_1 ... y_n]$是n维行向量，$X=\\begin{bmatrix}x_1 & ... & x_{1q}\\\\...&...& ...\\\\ x_{p1} & ... & x_{pq}\\end{bmatrix}$是$p*q$维矩阵\n",
    "\n",
    "则$\\frac{\\partial y^T}{\\partial X}=\\begin{bmatrix} \\frac{\\partial y^T}{\\partial x_{11}} & ... &\\frac{\\partial y^T}{\\partial x_{1q}}\\\\...&...& ... \\\\\n",
    "\\frac{\\partial y^T}{\\partial x_{p1}}& ... & \\frac{\\partial y^T}{\\partial x_{pq}} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 列向量对矩阵求导\n",
    "\n",
    "设$y=\\begin{bmatrix}y_1\\\\ ...\\\\ y_m\\end{bmatrix}$是m维列向量，$X=\\begin{bmatrix}x_1 & ... & x_{1q}\\\\...&...& ...\\\\ x_{p1} & ... & x_{pq}\\end{bmatrix}$是$p*q$维矩阵\n",
    "\n",
    "则$\\frac{\\partial y}{\\partial X}=\\begin{bmatrix} \\frac{\\partial y_1}{\\partial X}\\\\ ... \\\\\n",
    "\\frac{\\partial y_m}{\\partial X} \\end{bmatrix} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 矩阵对矩阵求导\n",
    "\n",
    "设$Y=\\begin{bmatrix}y_1 & ... & y_{1n}\\\\...&...& ...\\\\ y_{m1} & ... & y_{mn}\\end{bmatrix}=\\begin{bmatrix}y_1^T\\\\ ...\\\\ y_m^T\\end{bmatrix}$是$m*n$维矩阵，$X=\\begin{bmatrix}x_1 & ... & x_{1q}\\\\...&...& ...\\\\ x_{p1} & ... & x_{pq}\\end{bmatrix}=[x_1 ... x_q]$是$p*q$维矩阵\n",
    "\n",
    "则$\\frac{\\partial Y}{\\partial X}\n",
    "=[ \\frac{\\partial Y}{\\partial x_1} ... \\frac{\\partial Y}{\\partial x_q} ]\n",
    "=\\begin{bmatrix}\n",
    "\\frac{\\partial y_1^T}{\\partial X}\\\\\n",
    "... \\\\\n",
    "\\frac{\\partial y_m^T}{\\partial X} \n",
    "\\end{bmatrix} \n",
    "=\\begin{bmatrix}\n",
    "\\frac{\\partial y_{1}^T}{\\partial x_{1}} & ... &\\frac{\\partial y_{1}^T}{\\partial x_{q}}\\\\\n",
    "...&...& ... \\\\\n",
    "\\frac{\\partial y_m^T}{\\partial x_{1}}& ... & \\frac{\\partial y_m^T}{\\partial x_{q}} \\end{bmatrix} $"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "toc": {
   "colors": {
    "hover_highlight": "#DAA520",
    "navigate_num": "#000000",
    "navigate_text": "#333333",
    "running_highlight": "#FF0000",
    "selected_highlight": "#FFD700",
    "sidebar_border": "#EEEEEE",
    "wrapper_background": "#FFFFFF"
   },
   "moveMenuLeft": true,
   "nav_menu": {
    "height": "282px",
    "width": "252px"
   },
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 4,
   "toc_cell": false,
   "toc_section_display": "block",
   "toc_window_display": false,
   "widenNotebook": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}