{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "tensorflow/datasets", "version": "0.3.2", "provenance": [], "collapsed_sections": [] } }, "cells": [ { "metadata": { "colab_type": "text", "id": "6XvCUmCEd4Dm" }, "cell_type": "markdown", "source": [ "# TensorFlow Datasets\n", "\n", "TensorFlow Datasets provides a collection of datasets ready to use with TensorFlow. It handles downloading and preparing the data and constructing a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset)." ] }, { "metadata": { "colab_type": "text", "id": "J8y9ZkLXmAZc" }, "cell_type": "markdown", "source": [ "Copyright 2018 The TensorFlow Datasets Authors, Licensed under the Apache License, Version 2.0" ] }, { "metadata": { "colab_type": "text", "id": "OGw9EgE0tC0C" }, "cell_type": "markdown", "source": [ "\n", " \n", " \n", " \n", "
\n", " View on TensorFlow.org\n", " \n", " Run in Google Colab\n", " \n", " View source on GitHub\n", "
" ] }, { "metadata": { "colab_type": "text", "id": "_7hshda5eaGL" }, "cell_type": "markdown", "source": [ "## Installation\n", "\n", "`pip install tensorflow-datasets`\n", "\n", "Note that `tensorflow-datasets` expects you to have TensorFlow already installed, and currently depends on `tensorflow` (or `tensorflow-gpu`) >= `1.12.0`." ] }, { "metadata": { "cellView": "both", "colab_type": "code", "id": "boeZp0sYbO41", "colab": {} }, "cell_type": "code", "source": [ "!pip install -q tensorflow tensorflow-datasets matplotlib" ], "execution_count": 0, "outputs": [] }, { "metadata": { "colab_type": "code", "id": "TTBSvHcSLBzc", "colab": {} }, "cell_type": "code", "source": [ "from __future__ import absolute_import\n", "from __future__ import division\n", "from __future__ import print_function\n", "\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import tensorflow as tf\n", "\n", "import tensorflow_datasets as tfds" ], "execution_count": 0, "outputs": [] }, { "metadata": { "colab_type": "text", "id": "8-ZBEd6Ie28N" }, "cell_type": "markdown", "source": [ "## Eager execution\n", "\n", "TensorFlow Datasets is compatible with both TensorFlow [Eager mode](https://www.tensorflow.org/guide/eager) and Graph mode. For this colab, we'll run in Eager mode." ] }, { "metadata": { "colab_type": "code", "id": "o9H2EiXzfNgO", "colab": {} }, "cell_type": "code", "source": [ "tf.enable_eager_execution()" ], "execution_count": 0, "outputs": [] }, { "metadata": { "colab_type": "text", "id": "VZZyuO13fPvk" }, "cell_type": "markdown", "source": [ "## List the available datasets\n", "\n", "Each dataset is implemented as a [`tfds.core.DatasetBuilder`](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/DatasetBuilder) and you can list all available builders with `tfds.list_builders()`.\n", "\n", "You can see all the datasets with additional documentation on the [datasets documentation page](https://github.com/tensorflow/datasets/blob/master/docs/datasets.md)." ] }, { "metadata": { "colab_type": "code", "id": "FAvbSVzjLCIb", "outputId": "79f354d8-746a-4fac-ab34-86225904ee46", "colab": { "base_uri": "https://localhost:8080/", "height": 181 } }, "cell_type": "code", "source": [ "tfds.list_builders()" ], "execution_count": 4, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['bair_robot_pushing',\n", " 'celeb_a',\n", " 'cifar10',\n", " 'cifar100',\n", " 'diabetic_retinopathy_detection',\n", " 'fashion_mnist',\n", " 'image_label_folder',\n", " 'imdb_reviews',\n", " 'mnist']" ] }, "metadata": { "tags": [] }, "execution_count": 4 } ] }, { "metadata": { "colab_type": "text", "id": "VjI6VgOBf0v0" }, "cell_type": "markdown", "source": [ "## `tfds.load`: A dataset in one line\n", "\n", "[`tfds.load`](https://www.tensorflow.org/datasets/api_docs/python/tfds/load) is a convenience method that's the simplest way to build and load and `tf.data.Dataset`.\n", "\n", "Below, we load the MNIST training data. Setting `download=True` will download and prepare the data. Note that it's safe to call `load` multiple times with `download=True` as long as the builder `name` and `data_dir` remain the same. The prepared data will be reused." ] }, { "metadata": { "colab_type": "code", "id": "dCou80mnLLPV", "outputId": "0ee326c1-ea25-4c4b-b3dc-a7c29e9b2156", "colab": { "base_uri": "https://localhost:8080/", "height": 108 } }, "cell_type": "code", "source": [ "mnist_train = tfds.load(name=\"mnist\", split=tfds.Split.TRAIN)\n", "assert isinstance(mnist_train, tf.data.Dataset)\n", "mnist_train" ], "execution_count": 5, "outputs": [ { "output_type": "stream", "text": [ "INFO:tensorflow:Reusing dataset mnist (/root/tensorflow_datasets/mnist/1.0.0)\n", "WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py:423: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n", "Instructions for updating:\n", "Colocations handled automatically by placer.\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 5 } ] }, { "metadata": { "colab_type": "text", "id": "u-GAxR79hGTr" }, "cell_type": "markdown", "source": [ "## Feature dictionaries\n", "\n", "All `tfds` datasets contain feature dictionaries mapping feature names to Tensor values. A typical dataset, like MNIST, will have 2 keys: `\"image\"` and `\"label\"`. Below we inspect a single example." ] }, { "metadata": { "colab_type": "code", "id": "YHE21nkHLrER", "outputId": "2cae50b0-4f36-44b3-d858-71c36403826a", "colab": { "base_uri": "https://localhost:8080/", "height": 365 } }, "cell_type": "code", "source": [ "mnist_example, = mnist_train.take(1)\n", "image, label = mnist_example[\"image\"], mnist_example[\"label\"]\n", "\n", "plt.imshow(image.numpy()[:, :, 0].astype(np.float32), cmap=plt.get_cmap(\"gray\"))\n", "print(\"Label: %d\" % label.numpy())" ], "execution_count": 6, "outputs": [ { "output_type": "stream", "text": [ "Label: 0\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUsAAAFKCAYAAACU6307AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEj9JREFUeJzt3V9IVPn/x/HX/HSlpMJ0VQj2H2Gs\npF0sFFmUWtLiwtKfXdjNtVjooliK3IgQyTSCLIuWrIvUrS6ShQGvugiUNtuNMCMvQr3RugiJ1rSV\ntsh2TeZ7sfxkqxnnPePMnHOm5wPmwnNm5rzfe06vPX/mc44vEAgEBACY0f85XQAAeAFhCQAGhCUA\nGBCWAGBAWAKAAWEJABaBBJAU9NXX1xdynldfydhTsvZFT955JaqvmfgS8TtLn88XdHogEAg5z6uS\nsScpOfuiJ+9IVF8zxWFqtF969OhR3b17Vz6fTzU1NVq2bFm0XwUArhdVWN6+fVsPHjyQ3+/X/fv3\nVVNTI7/fH+vaAMA1orrA093drbKyMknS4sWL9fTpUz1//jymhQGAm0S1Zzk2NqalS5dO/52ZmanR\n0VHNmzcv6Pv7+vpUUFAQdF4CTpkmXDL2JCVnX/TkHU73FfU5y/8K10RhYWHIzyXbyehk7ElKzr7o\nyTvccIEnqsPwnJwcjY2NTf/9+PFjZWdnR/NVAOAJUYXl6tWr1dHRIUkaGBhQTk5OyENwAEgGUR2G\nf/bZZ1q6dKm+/fZb+Xw+1dXVxbouAHAVfpQeY8nYk5ScfdGTd3j2nCUAvGsISwAwICwBwICwBAAD\nwhIADAhLADAgLAHAgLAEAAPCEgAMCEsAMCAsAcCAsAQAA8ISAAwISwAwICwBwICwBAADwhIADAhL\nADAgLAHAgLAEAAPCEgAMCEsAMCAsAcCAsAQAA8ISAAwISwAwICwBwICwBAADwhIADAhLADAgLAHA\ngLAEAAPCEgAMCEsAMCAsAcCAsAQAA8ISAAwISwAwICwBwICwBAADwhIADAhLADBIdboAJL/6+nrz\ne4uLi2O+/MOHD4ecV1JS8trf169fj/nykRzYswQAg6j2LHt6erR3717l5eVJkpYsWaLa2tqYFgYA\nbhL1YfiKFSvU1NQUy1oAwLU4DAcAg6jD8t69e9q1a5e2bt2qmzdvxrImAHAdXyAQCET6oZGREfX2\n9qq8vFzDw8Pavn27Ojs7lZaWFvT9/f39KigomHWxAOCUqMLyTV9//bV++uknffDBB8EX4vMFnR4I\nBELO86pk7EmaXV9u/elQV1eXSktLX5vm9Z8Osf3NfjmhRHUYfvnyZZ0/f16SNDo6qidPnig3Nze6\n6gDAA6K6Gr5u3Trt379fv/76qyYnJ1VfXx/yEBwAkkFUYTlv3jydO3cu1rUAgGvF5Jxl2IVwztLz\n3uyrq6vL/Nk3hxS62UxDI/8rkvOwifSubH/xXE4o/M4SAAwISwAwICwBwICwBAADwhIADAhLADAg\nLAHAgLAEAAPCEgAMCEsAMODpjnjNTEMT/zvPS0MYI1FXVxfz73Tr0EhEhj1LADAgLAHAgLAEAAPC\nEgAMCEsAMCAsAcCAsAQAA8ISAAwISwAw4IFlMeb1nhKwOczI+sCwSBQXFwedXlJSouvXr781LdZK\nS0vN732znkh5ffsLhQeWAYBHEJYAYEBYAoABYQkABoQlABgQlgBgQFgCgAFhCQAGhCUAGBCWAGDA\ncMcYc2NPkQzh6+rqivnyIxnCF8nQwNkKtq6cHu45223HjdtfLDDcEQA8grAEAAPCEgAMCEsAMCAs\nAcCAsAQAA8ISAAwISwAwICwBwICwBACDVKcLQPzV1dXF/DvdOoRxtqxD6iIZFhrJcNNIhlsm47BG\nNzPtWQ4ODqqsrExtbW2SpEePHmnbtm2qqKjQ3r179c8//8S1SABwWtiwfPHihY4cOaKioqLpaU1N\nTaqoqNAvv/yijz76SO3t7XEtEgCcFjYs09LS1NraqpycnOlpPT09Wr9+vaR/D7G6u7vjVyEAuEDY\nc5apqalKTX39bRMTE0pLS5MkZWVlaXR0ND7VAYBLzPoCj+WEdF9fnwoKCqL+vNckY09vitdFi0Rz\nc23hhKrdyz3NxOm+ogrL9PR0vXz5UnPmzNHIyMhrh+jBFBYWBp2ejDcqdWNP8bhymwxXw2ezruJ1\nNTwSwWp34/YXC569+e+qVavU0dEhSers7NSaNWuiqwwAPCLsnmV/f7+OHz+uhw8fKjU1VR0dHTp5\n8qSqq6vl9/u1aNEibdq0KRG1AoBjwoZlQUGBLl269Nb0ixcvxqUgAHAjRvB4VCTnweJxzuy3336L\n+Xd6SST9x+ucZajvfXN6JOeXERpjwwHAgLAEAAPCEgAMCEsAMCAsAcCAsAQAA8ISAAwISwAwICwB\nwICwBAADXyABN4kLdWulZLydlBt7iscqftdv0RaJ+vp683vj8XA5t22P0fDsLdoA4F1DWAKAAWEJ\nAAaEJQAYEJYAYEBYAoABYQkABoQlABgQlgBgQFgCgAFPd0RU3vWnO+Ldw54lABgQlgBgQFgCgAFh\nCQAGhCUAGBCWAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoABYQkABoQlABgQlgBgQFgCgAFhCQAGhCUA\nGBCWAGBAWAKAAWEJAAY8sAyIs+LiYqdLQAywZwkABqawHBwcVFlZmdra2iRJ1dXV+vLLL7Vt2zZt\n27ZN169fj2eNAOC4sIfhL1680JEjR1RUVPTa9H379qm0tDRuhQGAm4Tds0xLS1Nra6tycnISUQ8A\nuJIvEAgELG88c+aMFi5cqMrKSlVXV2t0dFSTk5PKyspSbW2tMjMzQ362v79fBQUFMSsaABItqqvh\nGzduVEZGhvLz89XS0qKzZ8/q0KFDId9fWFgYdHogEJDP54umBNdyY0/G/x9G5PDhw+b31tfXx3z5\nsZCoddXV1WV+b0lJScyX77btMRqJWlcz/VuJ6mp4UVGR8vPzJUnr1q3T4OBgdJUBgEdEFZZ79uzR\n8PCwJKmnp0d5eXkxLQoA3CbsYXh/f7+OHz+uhw8fKjU1VR0dHaqsrFRVVZXmzp2r9PR0NTQ0JKJW\nAHBM2LAsKCjQpUuX3pr++eefx6UgAHAjhjsCcRaPizZIPIY7AoABYQkABoQlABgQlgBgQFgCgAFh\nCQAGhCUAGBCWAGBAWAKAAWEJAAYMd0RUInliYST307TeJ9Ppe2S6YQhjsP9WdXV1Ed1rFHbsWQKA\nAWEJAAaEJQAYEJYAYEBYAoABYQkABoQlABgQlgBgQFgCgIEvEMnwimgX4vMFnR4IBELO8yo39pSA\nVRwziRzBE2xdWUfmdHV1zXr5wVy/ft383tLS0remuXH7i4VE9TXTvxX2LAHAgLAEAAPCEgAMCEsA\nMCAsAcCAsAQAA8ISAAwISwAwICwBwICwBAADHlj2DohkCJ3TD+KKpFarmYZGvjmvrq4u5suPpCce\nNuZe7FkCgAFhCQAGhCUAGBCWAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoABYQkABjzdMcbc2FMkQxjj\n9dRCK+twv+LiYvN3xmMI52yfwhgvbtz+YsENT3c0jQ1vbGxUb2+vXr16pZ07d6qwsFAHDhzQ1NSU\nsrOzdeLECaWlpcWsYABwm7BheevWLQ0NDcnv92t8fFybN29WUVGRKioqVF5erlOnTqm9vV0VFRWJ\nqBcAHBH2nOXy5ct1+vRpSdKCBQs0MTGhnp4erV+/XtK/hxjd3d3xrRIAHBY2LFNSUpSeni5Jam9v\n19q1azUxMTF92J2VlaXR0dH4VgkADjPfz/Lq1atqb2/XhQsXtGHDhunplutDfX19KigoCDovAdeX\nEi4Ze0qUeNxPMh4iuWiU6O0hWbc/p/syheWNGzd07tw5/fzzz5o/f77S09P18uVLzZkzRyMjI8rJ\nyZnx84WFhUGnJ+OVOzf2xNXwkiirCY2r4YnlhqvhYQ/Dnz17psbGRjU3NysjI0OStGrVKnV0dEiS\nOjs7tWbNmhiVCgDuFHbP8sqVKxofH1dVVdX0tGPHjungwYPy+/1atGiRNm3aFNciAcBpYcPym2++\n0TfffPPW9IsXL8alIABwI0bwxJjXe3L6JLrTrOciE3keMhJe3/5C8cQ5SwAAYQkAJoQlABgQlgBg\nQFgCgAFhCQAGhCUAGBCWAGBAWAKAAWEJAAbm+1ni3RBquF9JSclr8+Jx27N4CTU0saur6615kdx6\nDe8W9iwBwICwBAADwhIADAhLADAgLAHAgLAEAAPCEgAMCEsAMCAsAcCAsAQAA57uGGNe78krT3eM\n5OmKoYYwen1dBZOMPUk83REAPIOwBAADwhIADAhLADAgLAHAgLAEAAPCEgAMCEsAMCAsAcCAB5bh\nNdaHe9XV1Zm/8/Dhw+b38sAwuBV7lgBgQFgCgAFhCQAGhCUAGBCWAGBAWAKAAWEJAAaEJQAYEJYA\nYEBYAoABDyyLsWTsSUrOvujJO9zwwDLT2PDGxkb19vbq1atX2rlzp65du6aBgQFlZGRIknbs2KGS\nkpKYFAsAbhQ2LG/duqWhoSH5/X6Nj49r8+bNWrlypfbt2xfR40gBwMvChuXy5cu1bNkySdKCBQs0\nMTGhqampuBcGAG4S0TlLv9+vO3fuKCUlRaOjo5qcnFRWVpZqa2uVmZkZeiGcs/S8ZOyLnrzDDecs\nzWF59epVNTc368KFC+rv71dGRoby8/PV0tKiP/74Q4cOHQr52f7+fhUUFEReOQC4RcDg999/D3z1\n1VeB8fHxt+YNDQ0Fvvvuuxk/Lynoa6Z5Xn0lY0/J2hc9eeeVqL5mEvZ3ls+ePVNjY6Oam5unr37v\n2bNHw8PDkqSenh7l5eWF+xoA8LSwF3iuXLmi8fFxVVVVTU/bsmWLqqqqNHfuXKWnp6uhoSGuRQKA\n0/hReowlY09ScvZFT96RqL5mikOGOwKAAWEJAAaEJQAYEJYAYEBYAoABYQkABoQlABgQlgBgQFgC\ngAFhCQAGhCUAGBCWAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoABYQkABoQlABgQlgBgQFgCgAFhCQAG\nCXkULgB4HXuWAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoBBqhMLPXr0qO7evSufz6eamhotW7bMiTJi\nqqenR3v37lVeXp4kacmSJaqtrXW4qugNDg7qhx9+0Pfff6/Kyko9evRIBw4c0NTUlLKzs3XixAml\npaU5XWZE3uypurpaAwMDysjIkCTt2LFDJSUlzhYZocbGRvX29urVq1fauXOnCgsLPb+epLf7unbt\nmuPrKuFhefv2bT148EB+v1/3799XTU2N/H5/osuIixUrVqipqcnpMmbtxYsXOnLkiIqKiqanNTU1\nqaKiQuXl5Tp16pTa29tVUVHhYJWRCdaTJO3bt0+lpaUOVTU7t27d0tDQkPx+v8bHx7V582YVFRV5\nej1JwftauXKl4+sq4Yfh3d3dKisrkyQtXrxYT58+1fPnzxNdBmaQlpam1tZW5eTkTE/r6enR+vXr\nJUmlpaXq7u52qryoBOvJ65YvX67Tp09LkhYsWKCJiQnPrycpeF9TU1MOV+VAWI6NjWnhwoXTf2dm\nZmp0dDTRZcTFvXv3tGvXLm3dulU3b950upyopaamas6cOa9Nm5iYmD6cy8rK8tw6C9aTJLW1tWn7\n9u368ccf9eeffzpQWfRSUlKUnp4uSWpvb9fatWs9v56k4H2lpKQ4vq4cOWf5X8ky2vLjjz/W7t27\nVV5eruHhYW3fvl2dnZ2ePF8UTrKss40bNyojI0P5+flqaWnR2bNndejQIafLitjVq1fV3t6uCxcu\naMOGDdPTvb6e/ttXf3+/4+sq4XuWOTk5Ghsbm/778ePHys7OTnQZMZebm6svvvhCPp9PH374od5/\n/32NjIw4XVbMpKen6+XLl5KkkZGRpDicLSoqUn5+viRp3bp1GhwcdLiiyN24cUPnzp1Ta2ur5s+f\nnzTr6c2+3LCuEh6Wq1evVkdHhyRpYGBAOTk5mjdvXqLLiLnLly/r/PnzkqTR0VE9efJEubm5DlcV\nO6tWrZpeb52dnVqzZo3DFc3enj17NDw8LOnfc7L//0sGr3j27JkaGxvV3Nw8fZU4GdZTsL7csK4c\nuevQyZMndefOHfl8PtXV1enTTz9NdAkx9/z5c+3fv19//fWXJicntXv3bhUXFztdVlT6+/t1/Phx\nPXz4UKmpqcrNzdXJkydVXV2tv//+W4sWLVJDQ4Pee+89p0s1C9ZTZWWlWlpaNHfuXKWnp6uhoUFZ\nWVlOl2rm9/t15swZffLJJ9PTjh07poMHD3p2PUnB+9qyZYva2tocXVfcog0ADBjBAwAGhCUAGBCW\nAGBAWAKAAWEJAAaEJQAYEJYAYEBYAoDB/wDAUXSm7wcXiwAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": { "tags": [] } } ] }, { "metadata": { "colab_type": "text", "id": "EW-kEK_mhbhy" }, "cell_type": "markdown", "source": [ "## `DatasetBuilder`\n", "\n", "`tfds.load` is really a thin conveninence wrapper around `DatasetBuilder`. We can accomplish the same as above directly with the MNIST `DatasetBuilder`." ] }, { "metadata": { "colab_type": "code", "id": "9FDDJXmBhpQ4", "outputId": "2be38dd6-1aa3-4b63-fe8a-f4668147fb14", "colab": { "base_uri": "https://localhost:8080/", "height": 54 } }, "cell_type": "code", "source": [ "mnist_builder = tfds.builder(\"mnist\")\n", "mnist_builder.download_and_prepare()\n", "mnist_train = mnist_builder.as_dataset(split=tfds.Split.TRAIN)\n", "mnist_train" ], "execution_count": 7, "outputs": [ { "output_type": "stream", "text": [ "INFO:tensorflow:Reusing dataset mnist (/root/tensorflow_datasets/mnist/1.0.0)\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": { "tags": [] }, "execution_count": 7 } ] }, { "metadata": { "colab_type": "text", "id": "7tlVOAzLlKqc" }, "cell_type": "markdown", "source": [ "## Input pipelines\n", "\n", "Once you have a `tf.data.Dataset` object, it's simple to define the rest of an input pipeline suitable for model training by using the [`tf.data` API](https://www.tensorflow.org/guide/datasets).\n", "\n", "Here we'll repeat the dataset so that we have an infinite stream of examples, shuffle, and create batches of 32." ] }, { "metadata": { "colab_type": "code", "id": "9OQZqGZMlSE8", "colab": {} }, "cell_type": "code", "source": [ "mnist_train = mnist_train.repeat().shuffle(1024).batch(32)\n", "\n", "# prefetch will enable the input pipeline to asynchronously fetch batches while\n", "# your model is training.\n", "mnist_train = mnist_train.prefetch(tf.data.experimental.AUTOTUNE)\n", "\n", "# Now you could loop over batches of the dataset and train\n", "# for batch in mnist_train:\n", "# ..." ], "execution_count": 0, "outputs": [] }, { "metadata": { "colab_type": "text", "id": "uczpNuc_A7wE" }, "cell_type": "markdown", "source": [ "## DatasetInfo\n", "\n", "After generation, the builder contains useful information on the dataset:" ] }, { "metadata": { "colab_type": "code", "id": "mSamfFznA9Ph", "outputId": "a3203f4b-a1ca-4290-c7da-d4a148096807", "colab": { "base_uri": "https://localhost:8080/", "height": 272 } }, "cell_type": "code", "source": [ "info = mnist_builder.info\n", "print(info)" ], "execution_count": 9, "outputs": [ { "output_type": "stream", "text": [ "tfds.core.DatasetInfo(\n", " name='mnist',\n", " version=1.0.0,\n", " description='The MNIST database of handwritten digits.',\n", " urls=[u'http://yann.lecun.com/exdb/mnist/'],\n", " features=FeaturesDict({'image': Image(shape=(28, 28, 1), dtype=tf.uint8), 'label': ClassLabel(shape=(), dtype=tf.int64)}),\n", " num_examples=70000,\n", " splits=[u'test', u'train'],\n", " examples_per_split=[10000L, 60000L],\n", " supervised_keys=(u'image', u'label'),\n", " citation='Y. Lecun and C. Cortes, \"The MNIST database of handwritten digits,\" 1998.\n", "[Online]. Available: http://yann.lecun.com/exdb/mnist/',\n", ")\n", "\n" ], "name": "stdout" } ] }, { "metadata": { "id": "cspsneov2VbC", "colab_type": "text" }, "cell_type": "markdown", "source": [ "`DatasetInfo` also contains useful information about the features:" ] }, { "metadata": { "id": "u1wL14QH2TW1", "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 72 }, "outputId": "262f4857-7253-4a16-af00-1ff404b69963" }, "cell_type": "code", "source": [ "print(info.features)\n", "print(info.features[\"label\"].num_classes)\n", "print(info.features[\"label\"].names)" ], "execution_count": 10, "outputs": [ { "output_type": "stream", "text": [ "FeaturesDict({'image': Image(shape=(28, 28, 1), dtype=tf.uint8), 'label': ClassLabel(shape=(), dtype=tf.int64)})\n", "10\n", "[u'0', u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9']\n" ], "name": "stdout" } ] }, { "metadata": { "colab_type": "text", "id": "xbrm0WBmBLEI" }, "cell_type": "markdown", "source": [ "You can also load the `DatasetInfo` directly with `tfds.load` using `with_info=True`." ] }, { "metadata": { "colab_type": "code", "id": "tvZYujQwBL7B", "outputId": "46fbcb43-697d-460b-da80-02e58bed8b3e", "colab": { "base_uri": "https://localhost:8080/", "height": 290 } }, "cell_type": "code", "source": [ "dataset, info = tfds.load(\"mnist\", split=\"test\", with_info=True)\n", "print(info)" ], "execution_count": 11, "outputs": [ { "output_type": "stream", "text": [ "INFO:tensorflow:Reusing dataset mnist (/root/tensorflow_datasets/mnist/1.0.0)\n", "tfds.core.DatasetInfo(\n", " name='mnist',\n", " version=1.0.0,\n", " description='The MNIST database of handwritten digits.',\n", " urls=[u'http://yann.lecun.com/exdb/mnist/'],\n", " features=FeaturesDict({'image': Image(shape=(28, 28, 1), dtype=tf.uint8), 'label': ClassLabel(shape=(), dtype=tf.int64)}),\n", " num_examples=70000,\n", " splits=[u'test', u'train'],\n", " examples_per_split=[10000L, 60000L],\n", " supervised_keys=(u'image', u'label'),\n", " citation='Y. Lecun and C. Cortes, \"The MNIST database of handwritten digits,\" 1998.\n", "[Online]. Available: http://yann.lecun.com/exdb/mnist/',\n", ")\n", "\n" ], "name": "stdout" } ] } ] }