{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction to TensorFlow\n", "> Before you can build advanced models in TensorFlow 2.0, you will first need to understand the basics. In this chapter, you’ll learn how to define constants and variables, perform tensor addition and multiplication, and compute derivatives. Knowledge of linear algebra will be helpful, but not necessary. This is the Summary of lecture \"Introduction to TensorFlow in Python\", via datacamp.\n", "\n", "- toc: true \n", "- badges: true\n", "- comments: true\n", "- author: Chanseok Kang\n", "- categories: [Python, Datacamp, Tensorflow-Keras, Deep_Learning]\n", "- image: images/tf.png" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'2.1.0'" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import tensorflow as tf\n", "import pandas as pd\n", "import numpy as np\n", "\n", "tf.__version__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Constants and variables\n", "- Tensorflow\n", " - Open-source library for graph-based numerical computation\n", " - Low and high level API\n", " - Addition, multiplication, differentiation\n", " - Machine Learning models\n", " - In v2.0\n", " - Eager execution by default\n", " - Model building with Keras and Estimators\n", "- Tensor\n", " - Generalization of vectors and matrices\n", " - Collection of numbers\n", " - Specific shape" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Defining data as constants\n", "Throughout this course, we will use `tensorflow` version 2.1 and will exclusively import the submodules needed to complete each exercise. \n", "\n", "you will use it to transform a numpy array, `credit_numpy`, into a tensorflow constant, `credit_constant`. This array contains feature columns from a dataset on credit card holders.\n", "\n", "Note that tensorflow version 2.0 allows you to use data as either a numpy array or a tensorflow constant object. Using a constant will ensure that any operations performed with that object are done in tensorflow." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
0123
02.01.024.03913.0
12.02.026.02682.0
22.02.034.029239.0
32.01.037.046990.0
42.01.057.08617.0
\n", "
" ], "text/plain": [ " 0 1 2 3\n", "0 2.0 1.0 24.0 3913.0\n", "1 2.0 2.0 26.0 2682.0\n", "2 2.0 2.0 34.0 29239.0\n", "3 2.0 1.0 37.0 46990.0\n", "4 2.0 1.0 57.0 8617.0" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "credit = pd.read_csv('./dataset/credit.csv')\n", "credit_numpy = credit.to_numpy()\n", "credit.head()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The datatype is: \n", "The shape is: (30000, 4)\n" ] } ], "source": [ "# Convert the credit_numpy array into a tensorflow constant\n", "credit_constant = tf.constant(credit_numpy)\n", "\n", "# Print constant datatype\n", "print('The datatype is:', credit_constant.dtype)\n", "\n", "# Print constant shape\n", "print('The shape is:', credit_constant.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Defining variables\n", "Unlike a constant, a variable's value can be modified. This will be quite useful when we want to train a model by updating its parameters. Constants can't be used for this purpose, so variables are the natural choice.\n", "\n", "Let's try defining and working with a variable. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "[1 2 3 4]\n" ] } ], "source": [ "# Detine the 1-dimensional variable A1\n", "A1 = tf.Variable([1, 2, 3, 4])\n", "\n", "# Print the variable A1\n", "print(A1)\n", "\n", "# Convert A1 to a numpy array and assign it to B1\n", "B1 = A1.numpy()\n", "\n", "# Print B1\n", "print(B1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Basic operations\n", "- TensorFlow operation\n", " - model of computation that revolves around the use of graphs\n", " - Graph contains edges and nodes, where edges are tensors and the nodes are operations\n", "- Performing tensor addition\n", " - The `add()` operation performs **element-wise addition** with two tensors\n", " - Element-wise addition requires both tensors to have the same shape:\n", " - Scalar addition: $1 + 2 = 3$\n", " - Vector addition: $[1, 2] + [3, 4] = [4, 6]$\n", " - Matrix addition: $\\begin{bmatrix} 1 & 2 \\\\ 3 & 4 \\end{bmatrix} + \\begin{bmatrix} 5 & 6 \\\\ 7 & 8 \\end{bmatrix} = \\begin{bmatrix} 6 & 8 \\\\ 10 & 12 \\end{bmatrix}$\n", " - Operator is overloaded with $+$\n", "- Performing tensor multiplication\n", " - **Element-wise multiplication** performed using `multiply()` operation\n", " - Matrix multiplication performed with `matmul()` operator\n", " - The `matmul(A, B)` operation multiplies A by B\n", " - Number of columns of A must equal the number of rows of B\n", "- Summing over tensor dimensions\n", " - The `reduce_sum()` operator sums over the dimensions of a tensor\n", " - `reduce_sum(A)` sums over all dimensions of A\n", " - `reduce_sum(A, i)` sums over dimension i\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Performing element-wise multiplication\n", "Element-wise multiplication in TensorFlow is performed using two tensors with identical shapes. This is because the operation multiplies elements in corresponding positions in the two tensors. An example of an element-wise multiplication, denoted by the $\\odot$ symbol, is shown below:\n", "\n", "$$ \\begin{bmatrix} 1 & 2 \\\\ 2 & 1 \\end{bmatrix} \\odot \\begin{bmatrix} 3 & 1 \\\\ 2 & 5 \\end{bmatrix} = \\begin{bmatrix} 3 & 2 \\\\ 4 & 5 \\end{bmatrix} $$\n", "\n", "In this exercise, you will perform element-wise multiplication, paying careful attention to the shape of the tensors you multiply." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "C1: [1 2 3 4]\n", "C23: [[1 2 3]\n", " [1 6 4]]\n" ] } ], "source": [ "# Define tensors A1 and A23 as constants\n", "A1 = tf.constant([1, 2, 3, 4])\n", "A23 = tf.constant([[1, 2, 3], [1, 6, 4]])\n", "\n", "# Define B1 and B23 to have the correct shape\n", "B1 = tf.ones_like(A1)\n", "B23 = tf.ones_like(A23)\n", "\n", "# Perform element-wise multiplication\n", "C1 = tf.multiply(A1, B1)\n", "C23 = tf.multiply(A23, B23)\n", "\n", "# Print the tensors C1 and C23\n", "print('C1: {}'.format(C1.numpy()))\n", "print('C23: {}'.format(C23.numpy()))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice how performing element-wise multiplication with tensors of ones leaves the original tensors unchanged." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Making predictions with matrix multiplication\n", "In later chapters, you will learn to train linear regression models. This process will yield a vector of parameters that can be multiplied by the input data to generate predictions. In this exercise, you will use input data, `features`, and a target vector, `bill`, which are taken from a credit card dataset we will use later in the course.\n", "\n", "$$ \\text{features} = \\begin{bmatrix} 2 & 24 \\\\ 2 & 26 \\\\ 2 & 57 \\\\ 1 & 37 \\end{bmatrix}, \n", "\\text{bill} = \\begin{bmatrix} 3913 \\\\ 2682 \\\\ 8617 \\\\ 64400 \\end{bmatrix}, \\text{params} = \\begin{bmatrix} 1000 \\\\ 150 \\end{bmatrix} $$\n", "\n", "The matrix of input data, `features`, contains two columns: education level and age. The target vector, `bill`, is the size of the credit card borrower's bill.\n", "\n", "Since we have not trained the model, you will enter a guess for the values of the parameter vector, `params`. You will then use `matmul()` to perform matrix multiplication of features by `params` to generate predictions, `billpred`, which you will compare with `bill`." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[-1687]\n", " [-3218]\n", " [-1933]\n", " [57850]]\n" ] } ], "source": [ "# Define features, params, and bill as constants\n", "features = tf.constant([[2, 24], [2, 26], [2, 57], [1, 37]])\n", "params = tf.constant([[1000], [150]])\n", "bill = tf.constant([[3913], [2682], [8617], [64400]])\n", "\n", "# Compute billpred using features and params\n", "billpred = tf.matmul(features, params)\n", "\n", "# Compute and print the error\n", "error = bill - billpred\n", "print(error.numpy())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Summing over tensor dimensions\n", "You've been given a matrix, `wealth`. This contains the value of bond and stock wealth for five individuals in thousands of dollars.\n", "$$\\text{wealth} = \\begin{bmatrix} 11 & 50 \\\\ 7 & 2 \\\\ 4 & 60 \\\\ 3 & 0 \\\\ 25 & 10 \\end{bmatrix} $$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first column corresponds to bonds and the second corresponds to stocks. Each row gives the bond and stock wealth for a single individual." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "wealth = tf.constant([[11, 50], [7, 2], [4, 60], [3, 0], [25, 10]])" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tf.reduce_sum(wealth)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tf.reduce_sum(wealth, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Advanced operations\n", "- Finding th optimum\n", " - In many problems, we will want to find the optimum of a function\n", " - Minimum: Lowest value of a loss function\n", " - Maximum: Highest value of objective function\n", " - We can do this using the `gradient()` operation\n", " - Optimum: Find a point where gradient = 0\n", " - Minimum: Change in gradient > 0\n", " - Maximum: Change in gradient < 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Reshaping tensors\n", "Later in the course, you will classify images of sign language letters using a neural network. In some cases, the network will take 1-dimensional tensors as inputs, but your data will come in the form of images, which will either be either 2- or 3-dimensional tensors, depending on whether they are grayscale or color images.\n", "\n", "The figure below shows grayscale and color images of the sign language letter A. The two images have been imported for you and converted to the numpy arrays `gray_tensor` and `color_tensor`. Reshape these arrays into 1-dimensional vectors using the `reshape` operation, which has been imported for you from tensorflow. Note that the shape of `gray_tensor` is 28x28 and the shape of `color_tensor` is 28x28x3.\n", "![sign](image/sign_1_10.jpg)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "gray_tensor = pd.read_csv('./dataset/gray_tensor.csv').to_numpy()\n", "\n", "color_tensor = pd.read_csv('./dataset/color_tensor.csv').to_numpy().reshape(28, 28, 3)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAUcElEQVR4nO3dW4xVZZYH8P8SBKFEiuJaUKV4ITo4ibfSaDSGiUx7eVD7oUd5aJnEDB1tku6kH8YwD+2DMWYybYeHsQ09EHHSo+mkvfBgVAKdEDVeCqyBUhhBBJqmBLG4FNfisuahtpNqrb3+5d5nn3Omv/8vIVV1Vn3nfGefszjn1Nrr+8zdISJ//S5o9AREpD6U7CKJULKLJELJLpIIJbtIIsbW88bGjx/vLS0tufEylQE2tpFVh6pv+/9rRaXqeZtZ4bFl58ZuO7r+MvM+deoUzpw5M+IVlEp2M7sHwHIAYwD8h7s/E/1+S0sL7rrrrtz4uXPnwtuLDhAbe/bs2TDOxpdx/vz5UuPL3Df2pB07Nn4KsONWBpsbu99jxowJ4xdcUPyNa9n7zeZWVbL39PTkxgofDTMbA+DfAdwLYD6ARWY2v+j1iUi1ynxmvwXADnff6e6DAF4G8EBtpiUitVYm2ecA+NOwn/dml/0FM1tiZt1m1n369OkSNyciZZRJ9pE+WHzng4i7r3D3LnfvGj9+fImbE5EyyiT7XgCdw37uALCv3HREpCplkv0jAPPM7HIzGwfgYQBrajMtEam1wqU3dz9rZksBvIWh0tsqd/+EjAnLKaxEFY1lZRp23WXq9GVKJUD5sl+ZmjD7O8q4ceMKXzcwVPfNw44bi5cpaZZ9zJgqzxkpOrZUnd3d3wDwRpnrEJH60OmyIolQsoskQskukgglu0gilOwiiVCyiySirv3sTJV1dqZMS2PZVk02ntWEo7mzVktWRz958mQYZ3OLrr9s2zG7b9Hcyq5/wOLsuRy135Ztic69zUquVUSajpJdJBFKdpFEKNlFEqFkF0mEkl0kEXUtvbl7ZWWFsu2QbJXVwcHB3BhbxZSViFibKRsfxasuSbLVh86cOZMbY4/JwMBAGJ82bVoYjx6XLVu2hGPvvvvuML5jx44wXqZsyB7vovTKLpIIJbtIIpTsIolQsoskQskukgglu0gilOwiiah7i2tVbaqsZlt2V86ojs+um7VDsjZSFm9vbw/jka1bt4bxqVOnhvHZs2eH8QsvvDA3tnHjxnAse0znzp0bxmfNmpUbY3X2EydOhPGLLroojLNzBKJaepnncvRc0yu7SCKU7CKJULKLJELJLpIIJbtIIpTsIolQsoskou519qgOWGZJZjY26qsG4nowu21WF500aVIYf+edd8L49OnTw3hUb7755pvDsZ9//nkYL7ucc3T77DHbvHlzGI+2gwbifvdLLrkkHNvX1xfG77jjjjC+fv36MF60Vg7w9RPylEp2M9sFYADAOQBn3b2rzPWJSHVq8cr+d+5+sAbXIyIV0md2kUSUTXYH8LaZbTSzJSP9gpktMbNuM+uO1nETkWqVfRt/u7vvM7MZANaa2TZ33zD8F9x9BYAVANDa2hr/5UFEKlPqld3d92VfDwB4FcAttZiUiNRe4WQ3sxYzm/TN9wB+AKC3VhMTkdoq8zZ+JoBXsz7vsQD+y93fjAawdeOr7ElndXRW2/ziiy9yY2z98osvvjiMt7S0hPEJEyaE8f379xcey7B14fv7+8P4wYP5hRp2v9la/qznPFqP//LLLw/HsjUE2POF9drv2bMnN8aeq0XzoHCyu/tOANcVHS8i9aXSm0gilOwiiVCyiyRCyS6SCCW7SCKaqsW1DNbuyJb+3bRpUxiPWjlZ6e3qq68O46zFlbWZRmWiqCw3GqxseOzYsTAezX337t2F5vQN1qY6ZcqU3NgjjzwSjl25cmUY7+2NTym56qqrwvhll12WG9uwYUNuDCjeJq5XdpFEKNlFEqFkF0mEkl0kEUp2kUQo2UUSoWQXSUTd6+xRGytb7nncuHGFrhcAWltbwzhbMitqO2RtnqyGz7A6e9QKeuDAgVK3HW1VDfB2y46OjtzYmjVrwrHR482um1mwYEEYf+WVV8L4oUOHwvj27dvD+MKFC3Nj110XN5P29PSE8Tx6ZRdJhJJdJBFKdpFEKNlFEqFkF0mEkl0kEUp2kUTUtc7OlpJmve7Hjx8vfNvvvfdeGGf98FGc9VWznu8bb7wxjG/ZsiWMR7Vwtiwx8+WXX4ZxtlR1dGzYcstsOedZs2aF8WirbPaYsDUKdu7cGcY7OzvD+JEjR3JjbIns6LmofnYRUbKLpELJLpIIJbtIIpTsIolQsoskQskukoi697NHvdlsi96onsy2/2V93awuGvUvHz58uNR133bbbWGc1dmjWjo7LqyeHG25DAAzZ84M49Hj/dBDD4VjP/jggzA+ceLEMB6t18/uV9m1/NlzIlqP/9NPPw3HsjUG8tBXdjNbZWYHzKx32GVtZrbWzLZnX/NX4xeRpjCat/EvALjnW5c9AWCdu88DsC77WUSaGE12d98A4NvrLj0AYHX2/WoAD9Z4XiJSY0U/s8909z4AcPc+M5uR94tmtgTAEgAYP358wZsTkbIq/2u8u69w9y5372ILCIpIdYom+34zaweA7Gu5JUxFpHJFk30NgMXZ94sBvF6b6YhIVehndjN7CcACANPMbC+AXwJ4BsDvzexRAHsA/KgWk2FrkFc1FgDmzJkTxgcGBnJjrJY9derUMM7Gs73lo57xaF95ALj33nvDOKs3s57yqN586623hmPZevusV3/+/Pm5saVLl4ZjT58+HcbZOSHsMY96+dm6DdH9jvrZabK7+6Kc0F1srIg0D50uK5IIJbtIIpTsIolQsoskQskukoimWkqate5FSwt3d3eHY1kJirUkXnvttbkxVjrr6uoK42yL3iuvvDKMX3PNNbkx1gZ6//33h3FW0mT3PWrvZVt0s/ju3bvD+FNPPZUb27ZtWziWldZmz54dxm+44YYwvmvXrtwYO6asLJhHr+wiiVCyiyRCyS6SCCW7SCKU7CKJULKLJELJLpKIui8lHWHL80Y148HBwXAsWxKLraIT1U3ZUs/stlk9ua2tLYxfcEH+/9ns/AK2FPTkyZPDeLTENhA/Zi+++GI49t133w3jrMU1Oi7sfrW2toZx1hIdbRcNAG+++WZujG1dHp37oC2bRUTJLpIKJbtIIpTsIolQsoskQskukgglu0gi6l5nj/rZoxgQ11VZ3zWrwx89ejSMf/bZZ7kxVqsuu+3xsWPHwviePXtyY3feeWc4ltXw2ZLI7ByBxx57LDfGHhN27kO07TEATJmSv7lwtMw0wM/5uPTSS8N4VOMH4n52htXh8+iVXSQRSnaRRCjZRRKhZBdJhJJdJBFKdpFEKNlFElH3deOj2iqrH0bbJrMaPat7HjlyJIzv27cvNxZtmQzwdb6j6wZ4rfvrr7/OjUVr7QPAxx9/HMbffvvtML5+/fowfuLEidwYq5OzfnW2TkDUs87q6B0dHWGc7XEQbckMxI8L26I7mnupfnYzW2VmB8ysd9hlT5rZn82sJ/t3H7seEWms0byNfwHAPSNc/mt3vz7790ZtpyUitUaT3d03AOivw1xEpEJl/kC31Mw2Z2/zc09CNrMlZtZtZt3sPGoRqU7RZP8NgCsBXA+gD8Cv8n7R3Ve4e5e7d7E/uIhIdQolu7vvd/dz7n4ewG8B3FLbaYlIrRVKdjNrH/bjDwH05v2uiDQHWmc3s5cALAAwzcz2AvglgAVmdj0AB7ALwE9Ge4NRvZvVPov28QK8Ds/q0VEdfseOHeHYU6dOhfGoFg3w/bqjOvtbb70Vjl2+fHkYZ/vWs8ckqoWzejJbe52dfxD1s8+aNavw2NF4+umnw3j0kZY9V6MafxSjye7ui0a4eCUbJyLNRafLiiRCyS6SCCW7SCKU7CKJULKLJKLuS0lHpRpWxqnyDDy2rHHUXsu2RWalFFaC6uvrC+NfffVVbmz37t3hWHa/2X1jrcMTJkzIjbGS4rRp08J4Z2dnGJ8+fXpujLXXMqzcyrayLnPqOFtiO49e2UUSoWQXSYSSXSQRSnaRRCjZRRKhZBdJhJJdJBF1r7MXbc8DyrW4MmVaYFn9n7XuRlsuA3w76qiFlh0zFmfHhS3nHNXS2RLcUZ0c4FtdR22qUf0f4EtJP/vss2GcPSeiWnmZx6zUUtIi8tdByS6SCCW7SCKU7CKJULKLJELJLpIIJbtIIpqqn53VkyNV1+ij/mO2VDTrPz569Gip8VE/PKv3sjo66/tmvfgTJ07Mjc2ZMyccy/rZWZ0+Gj9jxoxw7LJly8L42LFx6rB+9Wh8VeeT6JVdJBFKdpFEKNlFEqFkF0mEkl0kEUp2kUQo2UUSUfc6e4TVF6PaZVTPBYDjx4+HcVanj7CaK1sfnfVWs7lFte7W1tZwLMPWjWfbJke1btYLz3rK2fjouD733HPhWLaNNlujgD0nonNK2DGPbrtUP7uZdZrZH81sq5l9YmY/yy5vM7O1ZrY9+1puQ2sRqdRo3safBfALd/8bALcC+KmZzQfwBIB17j4PwLrsZxFpUjTZ3b3P3Tdl3w8A2ApgDoAHAKzOfm01gAermqSIlPe9PrOb2VwANwD4AMBMd+8Dhv5DMLMRTzY2syUAlgDF96gSkfJG/dd4M7sYwB8A/Nzd486NYdx9hbt3uXtXlRszikhsVMluZhdiKNF/5+6vZBfvN7P2LN4O4EA1UxSRWqBv422o7rMSwFZ3H75+7hoAiwE8k319nV2Xu4ctlWVa+1i5gmG3HZXPWGmNzY21iZ4+fTqMR3NnJUm25TJbznnSpElhfO7cubkxVhZkS0Wz0tuHH36YG+vt7Q3Hln0+lWnXZmW9okbzmf12AD8GsMXMerLLlmEoyX9vZo8C2APgR5XMUERqgia7u78DIO+sjrtqOx0RqYpOlxVJhJJdJBFKdpFEKNlFEqFkF0lEXVtc3Z0usRuJavTz588Px77//vthnJ3KG9XSWTsjW46ZLYnM2nOjY8rqvey22dyjOjoAzJ49OzfGzk+ItlwGgIMHD4bxl19+OTfGavRVi86NqGpZdL2yiyRCyS6SCCW7SCKU7CKJULKLJELJLpIIJbtIIqyq7WFH0tbW5gsXLsyNs5puVE+eN29eOJZtq8y2Lm5ra8uNtbe3h2PLLkvc398fxqPjwvqy2epBrBbOzjGI+unZlszsnIzHH388jEePKevjZ48JG1+lKGf7+/tx5syZEQv1emUXSYSSXSQRSnaRRCjZRRKhZBdJhJJdJBFKdpFE1LWffeLEibjpppty46ymG9VNWb2X9W0fPnw4jHd2dubG2NrqrCbLat2sJ/2KK67IjbG+bXb+AevzHxwcDONRnZ2df3Do0KEw/vDDD4fx559/PjfG1qxnjxk7P4U9Zuz5Gila49cru0gilOwiiVCyiyRCyS6SCCW7SCKU7CKJULKLJGI0+7N3AngRwCwA5wGscPflZvYkgH8C8FX2q8vc/Q1yXWHddmBgIJxLtBc469tm9WRWu5w8eXJujNWLWU2Vza1MTZhdN1ujnJ0DwOJRTzqr4bN+92XLloXx6NyJ1157LRzL7ldZZdaRKLp/+2gq+2cB/MLdN5nZJAAbzWxtFvu1u/9boVsWkboazf7sfQD6su8HzGwrgDlVT0xEaut7fWY3s7kAbgDwQXbRUjPbbGarzGzEvXrMbImZdZtZN9vGSESqM+pkN7OLAfwBwM/d/SiA3wC4EsD1GHrl/9VI49x9hbt3uXsXO/ddRKozqmQ3swsxlOi/c/dXAMDd97v7OXc/D+C3AG6pbpoiUhZNdhv6c+1KAFvd/dlhlw9fUvWHAHprPz0RqZXR/DX+dgA/BrDFzHqyy5YBWGRm1wNwALsA/GQ0N8iWbI6UKeOwUgeb14QJEwrNC+BtoGW2iwaAkydP5sZY2Y+VmFiZhx23qCzI7jdrE2VLj7/wwgu5MdaWzG6blSxZKZiNjxQt243mr/HvABhpZmFNXUSai86gE0mEkl0kEUp2kUQo2UUSoWQXSYSSXSQRdV1K2t3DmjOrCUe1y6jWDPBadUdHRxiPapus7slqqqxezI5LmZptmbFAuceM1fDZ+Qnbtm0L4zNmzAjjZbDHnN23qrZKj65Xr+wiiVCyiyRCyS6SCCW7SCKU7CKJULKLJELJLpIIq6reN+KNmX0FYPewi6YBOFi3CXw/zTq3Zp0XoLkVVcu5XebuIzbr1zXZv3PjZt3u3tWwCQSadW7NOi9AcyuqXnPT23iRRCjZRRLR6GRf0eDbjzTr3Jp1XoDmVlRd5tbQz+wiUj+NfmUXkTpRsoskoiHJbmb3mNn/mNkOM3uiEXPIY2a7zGyLmfWYWXeD57LKzA6YWe+wy9rMbK2Zbc++jrjHXoPm9qSZ/Tk7dj1mdl+D5tZpZn80s61m9omZ/Sy7vKHHLphXXY5b3T+zm9kYAJ8B+HsAewF8BGCRu39a14nkMLNdALrcveEnYJjZnQCOAXjR3f82u+xfAfS7+zPZf5RT3P2fm2RuTwI41uhtvLPditqHbzMO4EEA/4gGHrtgXv+AOhy3Rryy3wJgh7vvdPdBAC8DeKAB82h67r4BQP+3Ln4AwOrs+9UYerLUXc7cmoK797n7puz7AQDfbDPe0GMXzKsuGpHscwD8adjPe9Fc+707gLfNbKOZLWn0ZEYw0937gKEnD4Dq1l4qhm7jXU/f2ma8aY5dke3Py2pEso+06Fkz1f9ud/cbAdwL4KfZ21UZnVFt410vI2wz3hSKbn9eViOSfS+AzmE/dwDY14B5jMjd92VfDwB4Fc23FfX+b3bQzb4eaPB8/k8zbeM90jbjaIJj18jtzxuR7B8BmGdml5vZOAAPA1jTgHl8h5m1ZH84gZm1APgBmm8r6jUAFmffLwbwegPn8heaZRvvvG3G0eBj1/Dtz9297v8A3Iehv8h/DuBfGjGHnHldAeC/s3+fNHpuAF7C0Nu6Mxh6R/QogKkA1gHYnn1ta6K5/SeALQA2Yyix2hs0tzsw9NFwM4Ce7N99jT52wbzqctx0uqxIInQGnUgilOwiiVCyiyRCyS6SCCW7SCKU7CKJULKLJOJ/AbIkutK9QFBtAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "plt.imshow(gray_tensor, cmap=plt.cm.binary);\n", "# plt.imshow(image, interpolation='nearest')" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "# Reshape the grayscale image tensor into a vector\n", "gray_vector = tf.reshape(gray_tensor, (784, 1))\n", "\n", "# Reshape the color image tensor into a vector\n", "color_vector = tf.reshape(color_tensor, (2352, 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Optimizing with gradients\n", "You are given a loss function, `y=x^2`, which you want to minimize. You can do this by computing the slope using the `GradientTape()` operation at different values of `x`. If the slope is positive, you can decrease the loss by lowering `x`. If it is negative, you can decrease it by increasing `x`. This is how gradient descent works.\n", "![gradient](image/gradient_plot.png)\n", "\n", "In practice, you will use a high level tensorflow operation to perform gradient descent automatically. In this exercise, however, you will compute the slope at x values of -1, 1, and 0. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-2.0\n", "2.0\n", "0.0\n" ] } ], "source": [ "def compute_gradient(x0):\n", " # Define x as a variable with an initial value of x0\n", " x = tf.Variable(x0)\n", " with tf.GradientTape() as tape:\n", " tape.watch(x)\n", " # Define y using the multiply operation\n", " y = x * x\n", " # Return the gradient of y with respect to x\n", " return tape.gradient(y, x).numpy()\n", "\n", "# Compute and print gradients at x = -1, 1, and 0\n", "print(compute_gradient(-1.0))\n", "print(compute_gradient(1.0))\n", "print(compute_gradient(0.0))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that the slope is positive at x = 1, which means that we can lower the loss by reducing x. The slope is negative at x = -1, which means that we can lower the loss by increasing x. The slope at x = 0 is 0, which means that we cannot lower the loss by either increasing or decreasing x. This is because the loss is minimized at x = 0." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Working with image data\n", "You are given a black-and-white image of a letter, which has been encoded as a tensor, `letter`. You want to determine whether the letter is an X or a K. You don't have a trained neural network, but you do have a simple model, `model`, which can be used to classify `letter`.\n", "\n", "The 3x3 tensor, `letter`, and the 1x3 tensor, `model`, are available in the Python shell. You can determine whether letter is a K by multiplying letter by model, summing over the result, and then checking if it is equal to 1. As with more complicated models, such as neural networks, `model` is a collection of weights, arranged in a tensor." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "letter = np.array([[1.0, 0, 1.0], [1., 1., 0], [1., 0, 1.] ])\n", "model = np.array([[1., 0., -1.]])" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.0\n" ] } ], "source": [ "# Reshape model from a 1x3 to a 3x1 tensor\n", "model = tf.reshape(model, (3, 1))\n", "\n", "# Multiply letter by model\n", "output = tf.matmul(letter, model)\n", "\n", "# Sum over output and print prediction using the numpy method\n", "prediction = tf.reduce_sum(output)\n", "print(prediction.numpy())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Your model found that `prediction`=1.0 and correctly classified the letter as a K. In the coming chapters, you will use data to train a model, `model`, and then combine this with matrix multiplication, `matmul(letter, model)`, as we have done here, to make predictions about the classes of objects." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }