{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Theano 符号图结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用 `Theano`，首先要定义符号变量，然后是利用这写符号变量进行计算，这些符号被称为 **`variables`**，而操作 `+, -, **, sum(), tanh()` 被称为 **`ops`**，一个 `op` 操作接受某些类型的输入，并返回某些类型的输出。\n",
    "\n",
    "`Theano` 利用这些来构建一个图结构，一个图结构包括：\n",
    "- **`variable`** 节点\n",
    "- **`op`** 节点\n",
    "- **`apply`** 节点\n",
    "\n",
    "其中，`apply` 节点用来表示一个特定的 `op` 作用在一些特定的 `variables` 上，例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using gpu device 0: GeForce GTX 850M\n"
     ]
    }
   ],
   "source": [
    "import theano\n",
    "import theano.tensor as T\n",
    "\n",
    "x = T.dmatrix('x')\n",
    "y = T.dmatrix('y')\n",
    "z = x + y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "要显示这个图结构可以用 `pydotprint`，先安装 [graphviz](http://www.graphviz.org)。\n",
    "\n",
    "`Windows` 下：\n",
    "\n",
    "在环境变量 path 后加上：\n",
    "\n",
    "- path \n",
    "  - C:\\Program Files (x86)\\Graphviz2.38\\bin\n",
    "\n",
    "然后要先安装 `pydot` 包：\n",
    "\n",
    "如果你的 `pyparsing >= 2.0` ，则将其降为 `1.5.7`，下载并安装 `pydot-1.0.28`。\n",
    "    \n",
    "安装完之后，找到 `pydot.py` 将其中：\n",
    "\n",
    "    graph.append( '%s %s {\\n' % (self.obj_dict['type'], self.obj_dict['name']) )\n",
    "\n",
    "修改为：\n",
    "    \n",
    "    graph.append( '%s %s {\\n' % (self.obj_dict['type'], quote_if_necessary(self.obj_dict['name'])) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output file is available at apply1.png\n"
     ]
    }
   ],
   "source": [
    "theano.printing.pydotprint(z, outfile='apply1.png', var_with_name_simple=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "它的图结构如下：\n",
    "\n",
    "![图结构1](apply1.png)\n",
    "\n",
    "`z` 的 `owner` 是一个 `apply` 结构，其 `op` 为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Elemwise{add,no_inplace}'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z.owner.op.name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个 `apply` 结构的输入值有两个，输出值有一个："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "print z.owner.nin\n",
    "print z.owner.nout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "查看它的输入："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[x, y]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z.owner.inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以用 pprint 来显示它："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(x + y)\n"
     ]
    }
   ],
   "source": [
    "print theano.printing.pprint(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用 `debugprint` 显示图结构："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elemwise{add,no_inplace} [@A] ''   \n",
      " |x [@B]\n",
      " |y [@C]\n"
     ]
    }
   ],
   "source": [
    "theano.printing.debugprint(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再看另一个稍微复杂的例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y = x * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "查看 `y` 的图谱： "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elemwise{mul,no_inplace} [@A] ''   \n",
      " |x [@B]\n",
      " |DimShuffle{x,x} [@C] ''   \n",
      "   |TensorConstant{2} [@D]\n"
     ]
    }
   ],
   "source": [
    "theano.printing.debugprint(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们看到，`y` 对应的第二个 `input` 并不是 `2`，而是一个 `DimShuffle` 的操作："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<theano.tensor.elemwise.DimShuffle at 0x1b816390>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.owner.inputs[1].owner.op"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "它的输入才是常数 2："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[TensorConstant{2}]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.owner.inputs[1].owner.inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output file is available at apply2.png\n"
     ]
    }
   ],
   "source": [
    "theano.printing.pydotprint(y, outfile='apply2.png', var_with_name_simple=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其图结构为\n",
    "![结构2](apply2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## function 对图的优化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a = T.dscalar('a')\n",
    "b = a + a ** 10\n",
    "\n",
    "f = theano.function([a], b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output file is available at apply_no_opti.png\n",
      "The output file is available at apply_opti.png\n"
     ]
    }
   ],
   "source": [
    "theano.printing.pydotprint(b, outfile='apply_no_opti.png', var_with_name_simple=True)\n",
    "theano.printing.pydotprint(f, outfile='apply_opti.png', var_with_name_simple=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "比较一下 `function` 函数对图结构进行的优化：\n",
    "\n",
    "未优化前：\n",
    "\n",
    "![没有优化](apply_no_opti.png)\n",
    "\n",
    "优化后：\n",
    "\n",
    "![优化](apply_opti.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 图结构的作用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 计算按照图结构来计算 \n",
    "- 优化，求导"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}