{
  "nbformat_minor": 0,
  "cells": [
    {
      "source": [
        "#### Copyright 2017 Google LLC."
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "JndnmDMp66FL"
      }
    },
    {
      "source": [
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": [],
      "metadata": {
        "colab_type": "code",
        "id": "hMqWDc_m6rUC",
        "colab": {
          "autoexec": {
            "wait_interval": 0,
            "startup": false
          }
        },
        "cellView": "both"
      }
    },
    {
      "source": [
        "# TensorFlow Programming Concepts"
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "rFpcvnCJ4Xkf"
      }
    },
    {
      "source": [
        "**Learning Objectives:**\n",
        "  * Learn the basics of the TensorFlow programming model, focusing on the following concepts:\n",
        "    * tensors\n",
        "    * operations\n",
        "    * graphs\n",
        "    * sessions\n",
        "  * Build a simple TensorFlow program that creates a default graph, and a session that runs the graph"
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "9IkBOsrNzahz"
      }
    },
    {
      "source": [
        "**Note:** Please read through this tutorial carefully. The TensorFlow programming model is probably different from others that you have encountered, and thus may not be as intuitive as you'd expect."
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "YMG9mHPdzahz"
      }
    },
    {
      "source": [
        "## Overview of Concepts\n",
        "\n",
        "TensorFlow gets its name from **tensors**, which are arrays of arbitrary dimensionality. Using TensorFlow, you can manipulate tensors with a very high number of dimensions. That said, most of the time you will work with one or more of the following low-dimensional tensors:\n",
        "\n",
        "  * A **scalar** is a 0-d array (a 0th-order tensor).  For example, `\"Howdy\"` or `5`\n",
        "  * A **vector** is a 1-d array (a 1st-order tensor).  For example, `[2, 3, 5, 7, 11]` or `[5]`\n",
        "  * A **matrix** is a 2-d array (a 2nd-order tensor).  For example, `[[3.1, 8.2, 5.9][4.3, -2.7, 6.5]]`\n",
        "\n",
        "TensorFlow **operations** create, destroy, and manipulate tensors.  Most of the lines of code in a typical TensorFlow program are operations.\n",
        "\n",
        "A TensorFlow **graph** (also known as a **computational graph** or a **dataflow graph**) is, yes, a graph data structure.  A graph's nodes are operations (in TensorFlow, every operation is associated with a graph).  Many TensorFlow programs consist of a single graph, but TensorFlow programs may optionally create multiple graphs. A graph's nodes are operations; a graph's edges are tensors. Tensors flow through the graph, manipulated at each node by an operation. The output tensor of one operation often becomes the input tensor to a subsequent operation. TensorFlow implements a **lazy execution model,** meaning that nodes are only computed when needed, based on the needs of associated nodes.\n",
        "\n",
        "Tensors can be stored in the graph as **constants** or **variables**. As you might guess, constants hold tensors whose values can't change, while variables hold tensors whose values can change. However, what you may not have guessed is that constants and variables are just more operations in the graph. A constant is an operation that always returns the same tensor value. A variable is an operation that will return whichever tensor has been assigned to it.\n",
        "\n",
        "To define a constant, use the `tf.constant` operator and pass in its value. For example:\n",
        "\n",
        "```\n",
        "  x = tf.constant(5.2)\n",
        "```\n",
        "\n",
        "Similarly, you can create a variable like this:\n",
        "\n",
        "```\n",
        "  y = tf.Variable([5])\n",
        "```\n",
        "\n",
        "Or you can create the variable first and then subsequently assign a value like this (note that you always have to specify a default value):\n",
        "\n",
        "```\n",
        "  y = tf.Variable([0])\n",
        "  y = y.assign([5])\n",
        "```\n",
        "\n",
        "Once you've defined some constants or variables, you can combine them with other operations like `tf.add`. When you evaluate the `tf.add` operation, it will call your `tf.constant` or `tf.Variable` operations to get their values and then return a new tensor with the sum of those values.\n",
        "\n",
        "Graphs must run within a TensorFlow **session**, which holds the state for the graph(s) it runs:\n",
        "\n",
        "```\n",
        "with tf.Session() as sess:\n",
        "  initialization = tf.global_variables_initializer()\n",
        "  print(y.eval())\n",
        "```\n",
        "\n",
        "When working with `tf.Variable`s, you must explicitly initialize them by calling `tf.global_variables_initializer` at the start of your session, as shown above.\n",
        "\n",
        "**Note:** A session can distribute graph execution across multiple machines (assuming the program is run on some distributed computation framework). For more information, see [Distributed TensorFlow](https://www.tensorflow.org/deploy/distributed).\n",
        "\n",
        "### Summary\n",
        "\n",
        "TensorFlow programming is essentially a two-step process:\n",
        "\n",
        "  1. Assemble constants, variables, and operations into a graph.\n",
        "  2. Evaluate those constants, variables and operations within a session.\n"
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "NzKsjX-ufyVY"
      }
    },
    {
      "source": [
        "## Creating a Simple TensorFlow Program\n",
        "\n",
        "Let's look at how to code a simple TensorFlow program that adds two constants. "
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "VL0yWNNdgBpG"
      }
    },
    {
      "source": [
        "### Provide import statements\n",
        "\n",
        "As with nearly all Python programs, you'll begin by specifying some `import` statements.\n",
        "The set of `import` statements required to run a TensorFlow program depends, of course, on the features your program will access. At a minimum, you must provide the `import tensorflow` statement in all TensorFlow programs:"
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "mN4R0gmMzah3"
      }
    },
    {
      "source": [
        "import tensorflow as tf"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": [],
      "metadata": {
        "colab_type": "code",
        "id": "SDbi6heigEGA",
        "colab": {
          "autoexec": {
            "wait_interval": 0,
            "startup": false
          }
        }
      }
    },
    {
      "source": [
        "**Don't forget to execute the preceding code block (the `import` statements).**\n",
        "\n",
        "Other common import statements include the following:\n",
        "\n",
        "```\n",
        "  import matplotlib.pyplot as plt # Dataset visualization.\n",
        "  import numpy as np              # Low-level numerical Python library.\n",
        "  import pandas as pd             # Higher-level numerical Python library.\n",
        "```"
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6RRT4YIA4fQd"
      }
    },
    {
      "source": [
        "TensorFlow provides a **default graph**. However, we recommend explicitly creating your own `Graph` instead to facilitate tracking state (e.g., you may wish to work with a different `Graph` in each cell)."
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "1aNpvufRgbTD"
      }
    },
    {
      "source": [
        "from __future__ import print_function\n\n",
        "import tensorflow as tf\n",
        "\n",
        "# Create a graph.\n",
        "g = tf.Graph()\n",
        "\n",
        "# Establish the graph as the \"default\" graph.\n",
        "with g.as_default():\n",
        "  # Assemble a graph consisting of the following three operations:\n",
        "  #   * Two tf.constant operations to create the operands.\n",
        "  #   * One tf.add operation to add the two operands.\n",
        "  x = tf.constant(8, name=\"x_const\")\n",
        "  y = tf.constant(5, name=\"y_const\")\n",
        "  my_sum = tf.add(x, y, name=\"x_y_sum\")\n",
        "\n",
        "\n",
        "  # Now create a session.\n",
        "  # The session will run the default graph.\n",
        "  with tf.Session() as sess:\n",
        "    print(my_sum.eval())"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": [],
      "metadata": {
        "colab_type": "code",
        "id": "Md8ze8e9geMi",
        "colab": {
          "autoexec": {
            "wait_interval": 0,
            "startup": false
          }
        }
      }
    },
    {
      "source": [
        "## Exercise: Introduce a Third Operand\n",
        "\n",
        "Revise the above code listing to add three integers, instead of two:\n",
        "\n",
        "  1. Define a third scalar integer constant, `z`, and assign it a value of `4`.\n",
        "  2. Add `z` to `my_sum` to yield a new sum.\n",
        "  \n",
        "  **Hint:** See the API docs for [tf.add()](https://www.tensorflow.org/api_docs/python/tf/add) for more details on its function signature.\n",
        "  \n",
        "  3. Re-run the modified code block. Did the program generate the correct grand total?"
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "7mSz5GVqggZy"
      }
    },
    {
      "source": [
        "### Solution\n",
        "\n",
        "Click below for the solution."
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Zep4W80H9awM"
      }
    },
    {
      "source": [
        "# Create a graph.\n",
        "g = tf.Graph()\n",
        "\n",
        "# Establish our graph as the \"default\" graph.\n",
        "with g.as_default():\n",
        "  # Assemble a graph consisting of three operations. \n",
        "  # (Creating a tensor is an operation.)\n",
        "  x = tf.constant(8, name=\"x_const\")\n",
        "  y = tf.constant(5, name=\"y_const\")\n",
        "  my_sum = tf.add(x, y, name=\"x_y_sum\")\n",
        "  \n",
        "  # Task 1: Define a third scalar integer constant z.\n",
        "  z = tf.constant(4, name=\"z_const\")\n",
        "  # Task 2: Add z to `my_sum` to yield a new sum.\n",
        "  new_sum = tf.add(my_sum, z, name=\"x_y_z_sum\")\n",
        "\n",
        "  # Now create a session.\n",
        "  # The session will run the default graph.\n",
        "  with tf.Session() as sess:\n",
        "    # Task 3: Ensure the program yields the correct grand total.\n",
        "    print(new_sum.eval())"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": [],
      "metadata": {
        "colab_type": "code",
        "id": "4OTZPqhS9kzu",
        "colab": {
          "autoexec": {
            "wait_interval": 0,
            "startup": false
          }
        }
      }
    },
    {
      "source": [
        "## Further Information\n",
        "\n",
        "To explore basic TensorFlow graphs further, experiment with the following tutorial:\n",
        "\n",
        "  * [Mandelbrot set](https://www.tensorflow.org/tutorials/non-ml/mandelbrot)"
      ],
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "BrlnLTesgywL"
      }
    }
  ],
  "metadata": {
    "colab": {
      "default_view": {},
      "version": "0.3.2",
      "collapsed_sections": [
        "JndnmDMp66FL",
        "Zep4W80H9awM"
      ],
      "name": "tensorflow_programming_concepts.ipynb",
      "views": {}
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "nbformat": 4
}