{ "cells": [ { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "# Working With Multiple Layers\n", "\n", "First we start with loading the necessary libraries and resetting the computational graph." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import tensorflow as tf\n", "import os\n", "from tensorflow.python.framework import ops\n", "ops.reset_default_graph()" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Create a Graph Session" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [ "sess = tf.Session()" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Create Tensors\n", "\n", "Here we will create a small image of size 4x4 pixels and propagate it through multiple layers." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [ "# Create a small random 'image' of size 4x4\n", "x_shape = [1, 4, 4, 1]\n", "x_val = np.random.uniform(size=x_shape)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Create the Data Placeholder" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [ "x_data = tf.placeholder(tf.float32, shape=x_shape)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### First Layer: Moving Window (Convolution)\n", "\n", "Our first layer will be a spatial moving window of size [2x2] with stride 2 (in both height and width directions)\n", "\n", "To make this a moving window average, the value of the filter will be all 0.25." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [ "# Create a layer that takes a spatial moving window average\n", "# Our window will be 2x2 with a stride of 2 for height and width\n", "# The filter value will be 0.25 because we want the average of the 2x2 window\n", "my_filter = tf.constant(0.25, shape=[2, 2, 1, 1])\n", "my_strides = [1, 2, 2, 1]\n", "mov_avg_layer= tf.nn.conv2d(x_data, my_filter, my_strides,\n", " padding='SAME', name='Moving_Avg_Window')" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Second Layer: Custom\n", "\n", "Our second layer will be a custom layer. Given an input, x, this layer flattens out x and computes sigmoid(Ax+b). Here, A and b will be predetermined constants.\n", "\n", "We then add the custom layer to the graph under the name 'Custom_Layer'. This is for visualizing the graph in Tensorboard later." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [ "# Define a custom layer which will be sigmoid(Ax+b) where\n", "# x is a 2x2 matrix and A and b are 2x2 matrices\n", "def custom_layer(input_matrix):\n", " input_matrix_sqeezed = tf.squeeze(input_matrix)\n", " A = tf.constant([[1., 2.], [-1., 3.]])\n", " b = tf.constant(1., shape=[2, 2])\n", " temp1 = tf.matmul(A, input_matrix_sqeezed)\n", " temp = tf.add(temp1, b) # Ax + b\n", " return(tf.sigmoid(temp))\n", "\n", "# Add custom layer to graph\n", "with tf.name_scope('Custom_Layer') as scope:\n", " custom_layer1 = custom_layer(mov_avg_layer)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Run Output\n", "\n", "The output should be an array that is 2x2, but size (1,2,2,1)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[[[ 0.39856768]\n", " [ 0.85437918]]\n", "\n", " [[ 0.2236276 ]\n", " [ 0.35883212]]]]\n" ] } ], "source": [ "print(sess.run(mov_avg_layer, feed_dict={x_data: x_val}))" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "After custom operation, size is now 2x2 (squeezed out size 1 dims), see below:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[ 0.8636359 0.92904055]\n", " [ 0.78113878 0.77243596]]\n" ] } ], "source": [ "print(sess.run(custom_layer1, feed_dict={x_data: x_val}))" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Save summaries for viewing in Tensorboard:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [ "merged = tf.summary.merge_all(key='summaries')\n", "\n", "if not os.path.exists('tensorboard_logs/'):\n", " os.makedirs('tensorboard_logs/')\n", "\n", "my_writer = tf.summary.FileWriter('tensorboard_logs/', sess.graph)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "![multiple_layers_tensorboard](https://github.com/nfmcclure/tensorflow_cookbook/raw/master/02_TensorFlow_Way/images/03_Multiple_Layers.png)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.5.2" } }, "nbformat": 4, "nbformat_minor": 2 }