{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![book](https://raw.githubusercontent.com/ageron/tensorflow-safari-course/master/images/intro_to_tf_course.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Try not to peek at the solutions when you go through the exercises. ;-)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First let's make sure this notebook works well in both Python 2 and Python 3:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "tf.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*__From notebook 2 on variables:__*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> graph = tf.Graph()\n",
    ">>> with graph.as_default():\n",
    "...     x = tf.Variable(100)\n",
    "...     c = tf.constant(5)\n",
    "...     increment_op = tf.assign(x, x + c)\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Collections"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> graph.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf.GraphKeys.GLOBAL_VARIABLES"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> graph.add_to_collection(\"my_collection\", c)\n",
    ">>> graph.get_collection(\"my_collection\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Navigating the Graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> graph = tf.Graph()\n",
    ">>> with graph.as_default():\n",
    "...     a = tf.constant(3)\n",
    "...     b = tf.constant(5)\n",
    "...     s = a + b\n",
    "...\n",
    ">>> graph.get_operations()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> graph.get_operation_by_name(\"add\") is s.op"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> graph.get_tensor_by_name(\"add:0\") is s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> list(s.op.inputs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> list(s.op.outputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Naming Operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> graph = tf.Graph()\n",
    ">>> with graph.as_default():\n",
    "...     a = tf.constant(3, name='a')\n",
    "...     b = tf.constant(5, name='b')\n",
    "...     s = tf.add(a, b, name='s')\n",
    "...\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ">>> graph.get_operations()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Exercise](https://c1.staticflickr.com/9/8101/8553474140_c50cf08708_b.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "3.1) Create a graph with four variables named `\"x1\"`, `\"x2\"`, `\"x3\"` and `\"x4\"`, with initial values 1.0, 2.0, 3.0 and 4.0 respectively, then write some code that prints the name of every operation in the graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "3.2) Notice that for each `Variable`, TensorFlow actually created 4 operations:\n",
    "* the variable itself,\n",
    "* its initial value,\n",
    "* an assignment operation to assign the initial value to the variable,\n",
    "* and a read operation that you can safely ignore for now (for details, check out mrry's great answer to [this question](http://stackoverflow.com/questions/42783909/internals-of-variable-in-tensorflow)).\n",
    "\n",
    "Get the collection of global variables in the graph, and for each one of them use `get_operation_by_name()` to find its corresponding `/Assign` operation (just append `\"/Assign\"` to the variable's name).\n",
    "\n",
    "Hint: each object in the collection of global variables is actually a `Tensor`, not an `Operation` (it represents the variable's output, i.e., its value), so its name ends with `\":0\"`. You can get the `Operation` through the `Tensor`'s `op` attribute: its name will not end with `\":0\"`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.3) Add a `tf.group()` to your graph, containing all the assignment operations you got in question 3.2. Congratulations! You have just reimplemented `tf.global_variables_initializer()`.\n",
    "\n",
    "Start a `Session()`, run your group operation, then evaluate each variable and print out the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.4) For each assignment operation you fetched earlier, get its second input and store it in a list. Next, start a session and evaluate that list (using `sess.run()`). Print out the result: you should see `[1.0, 2.0, 3.0, 4.0]`. Can you guess why?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try not to peek at the solution below before you have done the exercise! :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![thinking](https://upload.wikimedia.org/wikipedia/commons/0/06/Filos_segundo_logo_%28flipped%29.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3 - Solution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "graph = tf.Graph()\n",
    "with graph.as_default():\n",
    "    x1 = tf.Variable(1.0, name=\"x1\")\n",
    "    x2 = tf.Variable(2.0, name=\"x2\")\n",
    "    x3 = tf.Variable(3.0, name=\"x3\")\n",
    "    x4 = tf.Variable(4.0, name=\"x4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gvars = graph.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)\n",
    "init_assign_ops = [graph.get_operation_by_name(gvar.op.name + \"/Assign\")\n",
    "                   for gvar in gvars]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "init_assign_ops"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with graph.as_default():\n",
    "    init = tf.group(*init_assign_ops)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with tf.Session(graph=graph):\n",
    "    init.run()\n",
    "    print(x1.eval())\n",
    "    print(x2.eval())\n",
    "    print(x3.eval())\n",
    "    print(x4.eval())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "init_val_ops = [init_assign_op.inputs[1]\n",
    "                for init_assign_op in init_assign_ops]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with tf.Session(graph=graph) as sess:\n",
    "    print(sess.run(init_val_ops))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Explanation: in the case of assignment operations, the first input is a reference to the variable, and the second is the assignment value. The assignment operations we have here are used to initialize the variables, so their assignment values correspond to the initial values: 1.0 for `x1`, 2.0 for `x2`, 3.0 for `x3` and 4.0 for `x4`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}