{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "SB93Ge748VQs" }, "source": [ "##### Copyright 2019 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "id": "0sK8X2O9bTlz" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "HEYuO5NFwDK9" }, "source": [ "# Get started with TensorBoard\n", "\n", "\n", " \n", " \n", " \n", "
\n", " View on TensorFlow.org\n", " \n", " Run in Google Colab\n", " \n", " View source on GitHub\n", "
" ] }, { "cell_type": "markdown", "metadata": { "id": "56V5oun18ZdZ" }, "source": [ "In machine learning, to improve something you often need to be able to measure it. TensorBoard is a tool for providing the measurements and visualizations needed during the machine learning workflow. It enables tracking experiment metrics like loss and accuracy, visualizing the model graph, projecting embeddings to a lower dimensional space, and much more.\n", "\n", "This quickstart will show how to quickly get started with TensorBoard. The remaining guides in this website provide more details on specific capabilities, many of which are not included here. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "6B95Hb6YVgPZ" }, "outputs": [], "source": [ "# Load the TensorBoard notebook extension\n", "%load_ext tensorboard" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "_wqSAZExy6xV" }, "outputs": [], "source": [ "import tensorflow as tf\n", "import datetime" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Ao7fJW1Pyiza" }, "outputs": [], "source": [ "# Clear any logs from previous runs\n", "!rm -rf ./logs/ " ] }, { "cell_type": "markdown", "metadata": { "id": "z5pr9vuHVgXY" }, "source": [ "Using the [MNIST](https://en.wikipedia.org/wiki/MNIST_database) dataset as the example, normalize the data and write a function that creates a simple Keras model for classifying the images into 10 classes." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "j-DHsby18cot" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n", "11493376/11490434 [==============================] - 0s 0us/step\n" ] } ], "source": [ "mnist = tf.keras.datasets.mnist\n", "\n", "(x_train, y_train),(x_test, y_test) = mnist.load_data()\n", "x_train, x_test = x_train / 255.0, x_test / 255.0\n", "\n", "def create_model():\n", " return tf.keras.models.Sequential([\n", " tf.keras.layers.Flatten(input_shape=(28, 28)),\n", " tf.keras.layers.Dense(512, activation='relu'),\n", " tf.keras.layers.Dropout(0.2),\n", " tf.keras.layers.Dense(10, activation='softmax')\n", " ])" ] }, { "cell_type": "markdown", "metadata": { "id": "XKUjdIoV87um" }, "source": [ "## Using TensorBoard with Keras Model.fit()" ] }, { "cell_type": "markdown", "metadata": { "id": "8CL_lxdn8-Sv" }, "source": [ "When training with Keras's [Model.fit()](https://www.tensorflow.org/api_docs/python/tf/keras/models/Model#fit), adding the `tf.keras.callbacks.TensorBoard` callback ensures that logs are created and stored. Additionally, enable histogram computation every epoch with `histogram_freq=1` (this is off by default)\n", "\n", "Place the logs in a timestamped subdirectory to allow easy selection of different training runs." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "WAQThq539CEJ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train on 60000 samples, validate on 10000 samples\n", "Epoch 1/5\n", "60000/60000 [==============================] - 15s 246us/sample - loss: 0.2217 - accuracy: 0.9343 - val_loss: 0.1019 - val_accuracy: 0.9685\n", "Epoch 2/5\n", "60000/60000 [==============================] - 14s 229us/sample - loss: 0.0975 - accuracy: 0.9698 - val_loss: 0.0787 - val_accuracy: 0.9758\n", "Epoch 3/5\n", "60000/60000 [==============================] - 14s 231us/sample - loss: 0.0718 - accuracy: 0.9771 - val_loss: 0.0698 - val_accuracy: 0.9781\n", "Epoch 4/5\n", "60000/60000 [==============================] - 14s 227us/sample - loss: 0.0540 - accuracy: 0.9820 - val_loss: 0.0685 - val_accuracy: 0.9795\n", "Epoch 5/5\n", "60000/60000 [==============================] - 14s 228us/sample - loss: 0.0433 - accuracy: 0.9862 - val_loss: 0.0623 - val_accuracy: 0.9823\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 6, "metadata": { "tags": [] }, "output_type": "execute_result" } ], "source": [ "model = create_model()\n", "model.compile(optimizer='adam',\n", " loss='sparse_categorical_crossentropy',\n", " metrics=['accuracy'])\n", "\n", "log_dir = \"logs/fit/\" + datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n", "tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)\n", "\n", "model.fit(x=x_train, \n", " y=y_train, \n", " epochs=5, \n", " validation_data=(x_test, y_test), \n", " callbacks=[tensorboard_callback])" ] }, { "cell_type": "markdown", "metadata": { "id": "asjGpmD09dRl" }, "source": [ "Start TensorBoard through the command line or within a notebook experience. The two interfaces are generally the same. In notebooks, use the `%tensorboard` line magic. On the command line, run the same command without \"%\"." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "A4UKgTLb9fKI" }, "outputs": [], "source": [ "%tensorboard --logdir logs/fit" ] }, { "cell_type": "markdown", "metadata": { "id": "MCsoUNb6YhGc" }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "id": "Gi4PaRm39of2" }, "source": [ "A brief overview of the dashboards shown (tabs in top navigation bar):\n", "\n", "* The **Scalars** dashboard shows how the loss and metrics change with every epoch. You can use it to also track training speed, learning rate, and other scalar values.\n", "* The **Graphs** dashboard helps you visualize your model. In this case, the Keras graph of layers is shown which can help you ensure it is built correctly. \n", "* The **Distributions** and **Histograms** dashboards show the distribution of a Tensor over time. This can be useful to visualize weights and biases and verify that they are changing in an expected way.\n", "\n", "Additional TensorBoard plugins are automatically enabled when you log other types of data. For example, the Keras TensorBoard callback lets you log images and embeddings as well. You can see what other plugins are available in TensorBoard by clicking on the \"inactive\" dropdown towards the top right." ] }, { "cell_type": "markdown", "metadata": { "id": "nB718NOH95yG" }, "source": [ "## Using TensorBoard with other methods\n" ] }, { "cell_type": "markdown", "metadata": { "id": "IKNt0nWs-Ekt" }, "source": [ "When training with methods such as [`tf.GradientTape()`](https://www.tensorflow.org/api_docs/python/tf/GradientTape), use `tf.summary` to log the required information.\n", "\n", "Use the same dataset as above, but convert it to `tf.data.Dataset` to take advantage of batching capabilities:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "nnHx4DsMezy1" }, "outputs": [], "source": [ "train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))\n", "test_dataset = tf.data.Dataset.from_tensor_slices((x_test, y_test))\n", "\n", "train_dataset = train_dataset.shuffle(60000).batch(64)\n", "test_dataset = test_dataset.batch(64)" ] }, { "cell_type": "markdown", "metadata": { "id": "SzpmTmJafJ10" }, "source": [ "The training code follows the [advanced quickstart](https://www.tensorflow.org/tutorials/quickstart/advanced) tutorial, but shows how to log metrics to TensorBoard. Choose loss and optimizer:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "H2Y5-aPbAANs" }, "outputs": [], "source": [ "loss_object = tf.keras.losses.SparseCategoricalCrossentropy()\n", "optimizer = tf.keras.optimizers.Adam()" ] }, { "cell_type": "markdown", "metadata": { "id": "cKhIIDj9Hbfy" }, "source": [ "Create stateful metrics that can be used to accumulate values during training and logged at any point:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "jD0tEWrgH0TL" }, "outputs": [], "source": [ "# Define our metrics\n", "train_loss = tf.keras.metrics.Mean('train_loss', dtype=tf.float32)\n", "train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy('train_accuracy')\n", "test_loss = tf.keras.metrics.Mean('test_loss', dtype=tf.float32)\n", "test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy('test_accuracy')" ] }, { "cell_type": "markdown", "metadata": { "id": "szw_KrgOg-OT" }, "source": [ "Define the training and test functions:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "TTWcJO35IJgK" }, "outputs": [], "source": [ "def train_step(model, optimizer, x_train, y_train):\n", " with tf.GradientTape() as tape:\n", " predictions = model(x_train, training=True)\n", " loss = loss_object(y_train, predictions)\n", " grads = tape.gradient(loss, model.trainable_variables)\n", " optimizer.apply_gradients(zip(grads, model.trainable_variables))\n", "\n", " train_loss(loss)\n", " train_accuracy(y_train, predictions)\n", "\n", "def test_step(model, x_test, y_test):\n", " predictions = model(x_test)\n", " loss = loss_object(y_test, predictions)\n", "\n", " test_loss(loss)\n", " test_accuracy(y_test, predictions)" ] }, { "cell_type": "markdown", "metadata": { "id": "nucPZBKPJR3A" }, "source": [ "Set up summary writers to write the summaries to disk in a different logs directory:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3Qp-exmbWf4w" }, "outputs": [], "source": [ "current_time = datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n", "train_log_dir = 'logs/gradient_tape/' + current_time + '/train'\n", "test_log_dir = 'logs/gradient_tape/' + current_time + '/test'\n", "train_summary_writer = tf.summary.create_file_writer(train_log_dir)\n", "test_summary_writer = tf.summary.create_file_writer(test_log_dir)" ] }, { "cell_type": "markdown", "metadata": { "id": "qgUJgDdKWUKF" }, "source": [ "Start training. Use `tf.summary.scalar()` to log metrics (loss and accuracy) during training/testing within the scope of the summary writers to write the summaries to disk. You have control over which metrics to log and how often to do it. Other `tf.summary` functions enable logging other types of data." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "odWvHPpKJvb_" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1, Loss: 0.24321186542510986, Accuracy: 92.84333801269531, Test Loss: 0.13006582856178284, Test Accuracy: 95.9000015258789\n", "Epoch 2, Loss: 0.10446818172931671, Accuracy: 96.84833526611328, Test Loss: 0.08867532759904861, Test Accuracy: 97.1199951171875\n", "Epoch 3, Loss: 0.07096975296735764, Accuracy: 97.80166625976562, Test Loss: 0.07875105738639832, Test Accuracy: 97.48999786376953\n", "Epoch 4, Loss: 0.05380449816584587, Accuracy: 98.34166717529297, Test Loss: 0.07712937891483307, Test Accuracy: 97.56999969482422\n", "Epoch 5, Loss: 0.041443776339292526, Accuracy: 98.71833038330078, Test Loss: 0.07514958828687668, Test Accuracy: 97.5\n" ] } ], "source": [ "model = create_model() # reset our model\n", "\n", "EPOCHS = 5\n", "\n", "for epoch in range(EPOCHS):\n", " for (x_train, y_train) in train_dataset:\n", " train_step(model, optimizer, x_train, y_train)\n", " with train_summary_writer.as_default():\n", " tf.summary.scalar('loss', train_loss.result(), step=epoch)\n", " tf.summary.scalar('accuracy', train_accuracy.result(), step=epoch)\n", "\n", " for (x_test, y_test) in test_dataset:\n", " test_step(model, x_test, y_test)\n", " with test_summary_writer.as_default():\n", " tf.summary.scalar('loss', test_loss.result(), step=epoch)\n", " tf.summary.scalar('accuracy', test_accuracy.result(), step=epoch)\n", " \n", " template = 'Epoch {}, Loss: {}, Accuracy: {}, Test Loss: {}, Test Accuracy: {}'\n", " print (template.format(epoch+1,\n", " train_loss.result(), \n", " train_accuracy.result()*100,\n", " test_loss.result(), \n", " test_accuracy.result()*100))\n", "\n", " # Reset metrics every epoch\n", " train_loss.reset_states()\n", " test_loss.reset_states()\n", " train_accuracy.reset_states()\n", " test_accuracy.reset_states()" ] }, { "cell_type": "markdown", "metadata": { "id": "JikosQ84fzcA" }, "source": [ "Open TensorBoard again, this time pointing it at the new log directory. We could have also started TensorBoard to monitor training while it progresses." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "-Iue509kgOyE" }, "outputs": [], "source": [ "%tensorboard --logdir logs/gradient_tape" ] }, { "cell_type": "markdown", "metadata": { "id": "NVpnilhEgQXk" }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "id": "ozbwXgPIkCKV" }, "source": [ "That's it! You have now seen how to use TensorBoard both through the Keras callback and through `tf.summary` for more custom scenarios. " ] }, { "cell_type": "markdown", "metadata": { "id": "vsowjhkBdkbK" }, "source": [ "## TensorBoard.dev: Host and share your ML experiment results\n", "\n", "[TensorBoard.dev](https://tensorboard.dev) is a free public service that enables you to upload your TensorBoard logs and get a permalink that can be shared with everyone in academic papers, blog posts, social media, etc. This can enable better reproducibility and collaboration.\n", "\n", "To use TensorBoard.dev, run the following command:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Q3nupQL24E5E" }, "outputs": [], "source": [ "!tensorboard dev upload \\\n", " --logdir logs/fit \\\n", " --name \"(optional) My latest experiment\" \\\n", " --description \"(optional) Simple comparison of several hyperparameters\" \\\n", " --one_shot" ] }, { "cell_type": "markdown", "metadata": { "id": "lAgEh_Ow4EX6" }, "source": [ "Note that this invocation uses the exclamation prefix (`!`) to invoke the shell\n", "rather than the percent prefix (`%`) to invoke the colab magic. When invoking this command from the command line there is no need for either prefix.\n", "\n", "View an example [here](https://tensorboard.dev/experiment/EDZb7XgKSBKo6Gznh3i8hg/#scalars).\n", "\n", "For more details on how to use TensorBoard.dev, see https://tensorboard.dev/#get-started" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "get_started.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }