{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# CS 20 : TensorFlow for Deep Learning Research\n", "## Lecture 04 : Eager execution\n", "### Custon training walkthrough\n", "Categorizing Iris flowers by species by using Tensorflow's eager execution.\n", "\n", "This guide uses these high-level TensorFlow concepts:\n", "\n", "* Enable an [eager execution](https://www.tensorflow.org/guide/eager?hl=ko) development environment,\n", "* Import data with the [Datasets API](https://www.tensorflow.org/guide/datasets?hl=ko)\n", "* Build models and layers with TensorFlow's [Keras API](https://keras.io/getting-started/sequential-model-guide/) \n", " \n", " \n", "* Reference\n", " + https://www.tensorflow.org/tutorials/eager/custom_training_walkthrough?hl=ko" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setup" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.12.0\n" ] } ], "source": [ "from __future__ import absolute_import, division, print_function\n", "import os, sys\n", "import numpy as np\n", "import tensorflow as tf\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "\n", "tf.enable_eager_execution()\n", "\n", "print(tf.__version__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a model" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "model = tf.keras.Sequential()\n", "model.add(tf.keras.layers.Dense(10, activation = tf.nn.relu, input_shape = (4,)))\n", "model.add(tf.keras.layers.Dense(10, activation = tf.nn.relu))\n", "model.add(tf.keras.layers.Dense(3))" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def loss_fn(model, features, label):\n", " score = model(features)\n", " return tf.losses.sparse_softmax_cross_entropy(labels = label, logits = score)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[, , , , , ]\n" ] } ], "source": [ "print(model.trainable_variables) # different from tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import and parse the training dataset" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['iris_test.csv', 'iris_training.csv']" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "os.listdir('../data/lecture04/')" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# define parsing function\n", "def parse_single_example(record):\n", " decoded = tf.decode_csv(record, [[.0],[.0],[.0],[.0],[]])\n", " features = decoded[:4]\n", " label = tf.cast(x = decoded[4], dtype = tf.int32)\n", " return features, label" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "epochs = 10\n", "batch_size = 8\n", "learning_rate = .03\n", "tr_dataset = tf.data.TextLineDataset(filenames = '../data/lecture04/iris_training.csv')\n", "tr_dataset = tr_dataset.map(parse_single_example)\n", "tr_dataset = tr_dataset.shuffle(200).batch(batch_size = batch_size)\n", "opt = tf.train.GradientDescentOptimizer(learning_rate = learning_rate)\n", "global_step = tf.Variable(0.)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train the model" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "epoch : 1, ce_loss : 1.154\n", "epoch : 2, ce_loss : 0.829\n", "epoch : 3, ce_loss : 0.706\n", "epoch : 4, ce_loss : 0.591\n", "epoch : 5, ce_loss : 0.553\n", "epoch : 6, ce_loss : 0.553\n", "epoch : 7, ce_loss : 0.449\n", "epoch : 8, ce_loss : 0.451\n", "epoch : 9, ce_loss : 0.514\n", "epoch : 10, ce_loss : 0.395\n" ] } ], "source": [ "for epoch in range(epochs):\n", " avg_loss = 0\n", " tr_step = 0\n", " \n", " for mb_x, mb_y in tr_dataset:\n", " with tf.GradientTape() as tape:\n", " tr_loss = loss_fn(model, mb_x, mb_y)\n", " grads = tape.gradient(tr_loss, model.variables)\n", " opt.apply_gradients(zip(grads, model.variables), global_step = global_step)\n", " \n", " avg_loss += tr_loss\n", " tr_step += 1\n", " else:\n", " avg_loss /= tr_step\n", " \n", " print('epoch : {:3}, ce_loss : {:.3f}'.format(epoch + 1, avg_loss))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate the model on the test dataset" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "tst_dataset = tf.data.TextLineDataset(filenames = '../data/lecture04/iris_test.csv')\n", "tst_dataset = tst_dataset.map(parse_single_example)\n", "tst_dataset = tst_dataset.batch(batch_size = 30)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "tst_x, tst_y = next(iter(tst_dataset))\n", "tst_yhat = tf.argmax(model(tst_x), axis = -1, output_type = tf.int32) " ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "test accuracy : 96.67%\n" ] } ], "source": [ "print('test accuracy : {:.2%}'.format(np.mean(tf.equal(tst_y, tst_yhat))))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" } }, "nbformat": 4, "nbformat_minor": 2 }