{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Theano tensor 模块：操作符和逐元素操作" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 操作符" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Using gpu device 1: Tesla C2075 (CNMeM is disabled)\n" ] } ], "source": [ "import theano\n", "from theano import tensor as T" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`tensor` 类型支持很多基本的操作：" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# 两个整形三维张量\n", "\n", "a, b = T.itensor3(\"a\"), T.itensor3(\"b\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 算术操作" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(a + TensorConstant{3})\n", "(TensorConstant{3} - a)\n", "(a * TensorConstant{3.5})\n", "(TensorConstant{2.20000004768} / a)\n", "(TensorConstant{2.20000004768} // a)\n", "(TensorConstant{2.20000004768} ** a)\n", "mod(b, a)\n" ] } ], "source": [ "print theano.pp(a + 3) # T.add(a, 3) -> itensor3\n", "print theano.pp(3 - a) # T.sub(3, a)\n", "print theano.pp(a * 3.5) # T.mul(a, 3.5) -> ftensor3 or dtensor3 (depending on casting)\n", "print theano.pp(2.2 / a) # T.truediv(2.2, a)\n", "print theano.pp(2.2 // a) # T.intdiv(2.2, a)\n", "print theano.pp(2.2**a) # T.pow(2.2, a)\n", "print theano.pp(b % a) # T.mod(b, a)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 比特操作" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "and_(a, b)\n", "xor(a, TensorConstant{1})\n", "or_(a, b)\n", "invert(a)\n" ] } ], "source": [ "print theano.pp(a & b) # T.and_(a,b) bitwise and (alias T.bitwise_and)\n", "print theano.pp(a ^ 1) # T.xor(a,1) bitwise xor (alias T.bitwise_xor)\n", "print theano.pp(a | b) # T.or_(a,b) bitwise or (alias T.bitwise_or)\n", "print theano.pp(~a) # T.invert(a) bitwise invert (alias T.bitwise_not)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 原地操作" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`Theano` 不支持原地操作如 `+=` 等，`Theano` 的图优化解构会自动决定是否使用原地操作。如果需要更新变量的值，可以考虑使用共享变量 `theano.shared`。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 逐元素操作" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 类型转换" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`T.cast(x, dtype)` 用于类型转换：" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = T.matrix()\n", "x_as_int = T.cast(x, 'int32')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`T.cast(x, dtype)` 的机制与 `numpy.asarray(x, dtype)` 的机制类似，只有 `dtype` 不同时才会创建新的变量： " ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "False\n", "True\n" ] } ], "source": [ "print x_as_int is x\n", "print T.cast(x, theano.config.floatX) is x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "复数取实部，虚部，角度，模：\n", "\n", "- `T.real(a)`\n", "- `T.imag(a)`\n", "- `T.angle(a)`\n", "- `T.abs_(a)`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 比较" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`Theano` 的比较操作也是逐元素的：\n", "\n", "- `T.lt(a, b)` : <\n", "- `T.gt(a, b)` : >\n", "- `T.le(a, b)` : <=\n", "- `T.ge(a, b)` : >=\n", "- `T.eq(a, b)` : ==\n", "- `T.neq(a, b)` : !=\n", "\n", "`Theano` 中没有 `bool` 类型，所有的 `bool` 类型都用 `int8` 表示。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "le(x, y)\n" ] } ], "source": [ "x, y = T.dmatrices('x','y')\n", "\n", "print theano.pp(T.le(x, y))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "除此之外，还有另一些与 `numpy` 类似的用法：\n", "\n", "- `T.isnan(a)` : 是否 NAN\n", "- `T.isinf(a)` : 是否 INF\n", "- `T.isclose(a, b)` ：浮点数是否接近\n", "- `T.allclose(a, b)` ：浮点数是否很接近" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 条件" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`T.switch(cond, ift, iff)` 选择 `ift (if ture)` 和 `iff (if false)`。\n", "\n", "`T.where(cond, ift, iff)` 与 `switch` 一致。\n", "\n", "`T.clip(x, min, max)` 低于 `min` 的部分变成 `min`，超过 `max` 的部分变成 `max`。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 数学操作" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "maximum(a, b)\n", "minimum(a, b)\n", "(-a)\n", "inv(a)\n", "exp(a)\n", "log(a) log2(a) log10(a)\n", "sgn(a)\n", "floor(a)\n", "ceil(a)\n", "round_half_away_from_zero(a)\n", "int64(round_half_away_from_zero(a))\n", "sqr(a)\n", "sqrt(a)\n", "cos(a) sin(a) tan(a)\n", "cosh(a) sinh(a) tanh(a)\n", "erf(a) erfc(a)\n", "erfinv(a) erfcinv(a)\n", "gamma(a)\n", "gammaln(a)\n", "psi(a)\n" ] } ], "source": [ "a, b = T.matrices(\"a\", \"b\")\n", "\n", "\n", "print theano.pp(T.maximum(a, b)) # max(a, b)\n", "print theano.pp(T.minimum(a, b)) # min(a, b)\n", "\n", "print theano.pp(T.neg(a)) # -a\n", "print theano.pp(T.inv(a)) # 1.0/a\n", "\n", "print theano.pp(T.exp(a)) \n", "print theano.pp(T.log(a)), theano.pp(T.log2(a)), theano.pp(T.log10(a)) # log10(a)\n", "\n", "print theano.pp(T.sgn(a)) # sgn(a)\n", "print theano.pp(T.floor(a)) # floor(a)\n", "print theano.pp(T.ceil(a)) # ceil(a)\n", "print theano.pp(T.round(a)) # round(a)\n", "print theano.pp(T.iround(a)) # iround(a)\n", "\n", "print theano.pp(T.sqr(a)) # sqr(a)\n", "print theano.pp(T.sqrt(a)) # sqrt(a)\n", "\n", "print theano.pp(T.cos(a)), theano.pp(T.sin(a)), theano.pp(T.tan(a))\n", "print theano.pp(T.cosh(a)), theano.pp(T.sinh(a)), theano.pp(T.tanh(a)) # tan(a)\n", "\n", "print theano.pp(T.erf(a)), theano.pp(T.erfc(a)) # erf(a), erfc(a)\n", "print theano.pp(T.erfinv(a)), theano.pp(T.erfcinv(a))\n", "\n", "print theano.pp(T.gamma(a)) # gamma(a)\n", "print theano.pp(T.gammaln(a)) # log(gamma(a))\n", "print theano.pp(T.psi(a)) # digamma(a)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "其中 `erf, erfc` 定义如下：\n", "https://en.wikipedia.org/wiki/Error_function\n", "\n", "$$\n", "\\operatorname{erf}(x) = \\frac{2}{\\sqrt\\pi} \\int_0^x e^{-t^2} dt\n", "$$\n", "$$\n", "\\begin{align}\n", " \\operatorname{erfc}(x) & = 1-\\operatorname{erf}(x) \\\\\n", " & = \\frac{2}{\\sqrt\\pi} \\int_x^{\\infty} e^{-t^2}\\,\\mathrm dt \\\\\n", " & = e^{-x^2} \\operatorname{erfcx}(x)\n", "\\end{align} \n", "$$\n", "\n", "`erfinv, erfcinv` 为其反函数：1\n", "https://en.wikipedia.org/wiki/Error_function#Inverse_functions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Broadcasting" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![](http://deeplearning.net/software/theano/_images/bcast.png)\n", "\n", "图示如上。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 线性代数" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "矩阵乘法：`T.dot(x, y)`\n", "\n", "向量外积：`T.outer(x, y)`\n", "\n", "张量乘法：`tensordot(a, b, axes=2)`\n", "\n", "`axes` 参数表示 `a` `b` 对应要去掉的维度。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(2, 3, 4) (5, 6, 4, 3)\n", "(2, 5, 6)\n" ] } ], "source": [ "import numpy as np\n", "\n", "a = np.random.random((2,3,4))\n", "b = np.random.random((5,6,4,3))\n", "\n", "#tensordot\n", "c = np.tensordot(a, b, [[1,2],[3,2]])\n", "\n", "\n", "#loop replicating tensordot\n", "a0, a1, a2 = a.shape\n", "b0, b1, _, _ = b.shape\n", "cloop = np.zeros((a0,b0,b1))\n", "\n", "#loop over non-summed indices -- these exist\n", "#in the tensor product.\n", "for i in range(a0):\n", " for j in range(b0):\n", " for k in range(b1):\n", " #loop over summed indices -- these don't exist\n", " #in the tensor product.\n", " for l in range(a1):\n", " for m in range(a2):\n", " cloop[i,j,k] += a[i,l,m] * b[j,k,m,l]\n", "\n", "assert np.allclose(c, cloop)\n", "\n", "print a.shape, b.shape\n", "print c.shape" ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 0 }