{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Eight Bullet Friday\n",
    "## TensorFlow Gentle Mental Calisthenics\n",
    "\n",
    "I am hoping that this Python Notebook will help you examine the code we ran last week.\n",
    "\n",
    "### Bullet 1. Import the libraries\n",
    "In the code space below import the tensorflow library and the mnist example data. Also print the tensorflow version and set the random_set\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bullet 2: Download the data \n",
    "Download both the MNIST data (images) and their labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting data/train-images-idx3-ubyte.gz\n",
      "Extracting data/train-labels-idx1-ubyte.gz\n",
      "Extracting data/t10k-images-idx3-ubyte.gz\n",
      "Extracting data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bullet 3: Grab the first 10 images and their labels\n",
    "You will create 2 variables-- one for the image data and the other for the labels\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bullet 4: Get one image\n",
    "Create a variable `image1` that contains the 2d array that is the first image in the first 10 images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A cool thing about numpy\n",
    "Numpy has a cool feature called broadcasting. It is easiest just to show you what it does. Suppose we have an array:\n",
    "\n",
    "     [[1, 2, 3, 4],\n",
    "      [5, 6, 7, 8]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 2, 3, 4],\n",
       "       [5, 6, 7, 8]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample = np.array([[1,2,3,4],[5,6,7,8]])\n",
    "sample"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and let's suppose we want to add 10 to each element in the array. We can do so by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[11, 12, 13, 14],\n",
       "       [15, 16, 17, 18]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample + 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or if we wanted to mulitply it by 2:\n",
    "2 * sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2,  4,  6,  8],\n",
       "       [10, 12, 14, 16]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "2 * sample"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bullet 5 - display the sample image\n",
    "In the previous step we set `image` to hold the 2d array of the first image. Let's go ahead and display that image:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "first_image = np.array(image1, dtype='float')\n",
    "pixels = first_image.reshape((28, 28))\n",
    "plt.imshow(pixels, cmap='gray')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's cool, but we would much rather see a black number on a white background. Can you display the second image of our initial ten in that way?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bullet 6: The labels\n",
    "When we downloaded the data we also downloaded the labels for the data. Print out the label for the first image of our batch of 10. And then print out the label of the second."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Maybe you were expecting a single number like **3** or **6**. How do you interpret the label value?\n",
    "\n",
    "### Step 7: Matrix Multiplication\n",
    "\n",
    "In our discussion we had customers rate different genres:\n",
    "\n",
    "**P**\n",
    "\n",
    "Customers | dream pop | neo | R&B\n",
    ":---: | :---: |  :---: |  :---: | \n",
    "Sam | 5 | 2 | 4\n",
    "Mary | 1 | 5 | 3\n",
    "Ben | 1 | 3 | 5\n",
    "Julie | 1 | 1 | 5\n",
    "\n",
    "**Q**\n",
    "\n",
    "artist | dream pop | neo | R&B\n",
    ":---: | :---: |  :---: |  :---: | \n",
    "Dua Lipa | 5 | 1 | 4\n",
    "Midland | .5 | 5 | 1\n",
    "Bruno Mars | 3 | 1 | 5\n",
    "Lorde | 4 | 2 | 2\t\n",
    "Bebe Rexha | 3 | 1 | 3\n",
    "\n",
    "And by hand we computed\n",
    "\n",
    "$$PQ^T$$\n",
    "\n",
    "Let's see how we can do this in numpy. First, let's create a numpy array for the customers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5., 2., 4.],\n",
       "       [1., 5., 3.],\n",
       "       [1., 3., 5.],\n",
       "       [1., 1., 5.]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p = np.array([[5., 2., 4.], [1., 5., 3.], [1., 3., 5.], [1., 1., 5.]])\n",
    "p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cool. Can you do the same (create a q variable this time) for the artist table?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok.\n",
    "\n",
    "We can transpose a matrix in numpy using\n",
    "\n",
    "     np.transpose()\n",
    "     \n",
    "For example, if we want to transpose a matrix *a* and assign it to the variable *aT* we could do\n",
    "\n",
    "     aT = np.transpose(a)\n",
    "     \n",
    "     \n",
    "You can multiply two matrices, `a` and `b`, together using:\n",
    "\n",
    "    np.matmul(a, b)\n",
    "    \n",
    "Can you compute\n",
    "\n",
    "$$PQ^T$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So you should get the same values we did when we did it by hand.\n",
    "\n",
    "Tensorflow also has a matrix multiplication function, called `matmul` so for example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = tf.matmul(p, qT)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Go ahead and print the value of `predictions`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will see that it does not print the nice matrix that the `numpy.matmul` did. \n",
    "\n",
    "###  Bullet 8 Sessions\n",
    "In tensorflow when we write and execute code like:\n",
    "\n",
    "     prediction = tf,matmul(a, b)\n",
    "     \n",
    " or\n",
    " \n",
    "     Y = tf.nn.softmax(tf.matmul(XX, W) + b)\n",
    "     ...\n",
    "     cross_entropy = -tf.reduce_mean(Y_ * tf.log(Y)) * 1000.0 \n",
    "     \n",
    "Tensorflow doesn't evaluate the results of that assignment. What it does is construct a graph where operations like matrix multiplications are nodes in a graph. For example:\n",
    "\n",
    "\n",
    "![](https://photos-1.dropbox.com/t/2/AADclR2gSB6Qxoytibc0QoZ4otrzIBys81_yLBgPO2lBhw/12/27548/png/32x32/3/1517000400/0/2/Screenshot%202017-12-17%2014.34.25.png/EL74IBizlLPlBCAHKAc/EWMhasd9gzVkXl9kob1_dZNFk2nJ2zPyNifNgdaTWIo?dl=0&preserve_transparency=1&size=1600x1200&size_mode=3)\n",
    "\n",
    "\n",
    "You can **deploy** (think run or execute) the graph by first binding the graph to a particular **session**. A session is a particular execution environment, which includes whether you are running it on a CPU or a cluster of GPUs.\n",
    "\n",
    "In the code we executed last week we had the lines:\n",
    "\n",
    "     init = tf.global_variables_initializer()\n",
    "     sess = tf.Session()\n",
    "     sess.run(init)\n",
    "     \n",
    "So we created a session and that ran it (*deployed the graph*). In this Python notebook we are playing around with things rather than setting up a high-falutin graph, so we can do something like:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "sess = tf.InteractiveSession()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which, as the name suggests, starts an interactive session. Now we can evaluate the graph we created with\n",
    "\n",
    "     predictions = tf.matmul(p, qT)\n",
    "\n",
    "above with"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[43. , 16.5, 37. , 32. , 29. ],\n",
       "       [22. , 28.5, 23. , 20. , 17. ],\n",
       "       [28. , 20.5, 31. , 20. , 21. ],\n",
       "       [26. , 10.5, 29. , 16. , 19. ]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions.eval()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### cross entropy\n",
    "\n",
    "Returning to our data. set a variable `y_` equal to the first label of our group of ten.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the label states that this image is of *3*. Suppose we ran the image through our classifier and it predicted:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = [0.01, 0.01, 0.5, 0.5, 0.001, 0.0001, 0.001, 0.023, 0.014, 0.94]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can you compute and print:\n",
    "\n",
    "### $$\\bar{y} \\log(y)$$\n",
    "\n",
    "where y_ is y bar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Congratulations!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}