{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "TBFXQGKYUc4X" }, "source": [ "##### Copyright 2018 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "cellView": "form", "colab": {}, "colab_type": "code", "id": "1z4xy2gTUc4a" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "FE7KNzPPVrVV" }, "source": [ "# Image classification" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "KwQtSOz0VrVX" }, "source": [ "\u003ctable class=\"tfo-notebook-buttons\" align=\"left\"\u003e\n", " \u003ctd\u003e\n", " \u003ca target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/images/classification\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" /\u003eView on TensorFlow.org\u003c/a\u003e\n", " \u003c/td\u003e\n", " \u003ctd\u003e\n", " \u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/images/classification.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /\u003eRun in Google Colab\u003c/a\u003e\n", " \u003c/td\u003e\n", " \u003ctd\u003e\n", " \u003ca target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/tutorials/images/classification.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /\u003eView source on GitHub\u003c/a\u003e\n", " \u003c/td\u003e\n", " \u003ctd\u003e\n", " \u003ca href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/tutorials/images/classification.ipynb\"\u003e\u003cimg src=\"https://www.tensorflow.org/images/download_logo_32px.png\" /\u003eDownload notebook\u003c/a\u003e\n", " \u003c/td\u003e\n", "\u003c/table\u003e" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "gN7G9GFmVrVY" }, "source": [ "This tutorial shows how to classify cats or dogs from images. It builds an image classifier using a `tf.keras.Sequential` model and load data using `tf.keras.preprocessing.image.ImageDataGenerator`. You will get some practical experience and develop intuition for the following concepts:\n", "\n", "* Building _data input pipelines_ using the `tf.keras.preprocessing.image.ImageDataGenerator` class to efficiently work with data on disk to use with the model.\n", "* _Overfitting_ —How to identify and prevent it.\n", "* _Data augmentation_ and _dropout_ —Key techniques to fight overfitting in computer vision tasks to incorporate into the data pipeline and image classifier model.\n", "\n", "This tutorial follows a basic machine learning workflow:\n", "\n", "1. Examine and understand data\n", "2. Build an input pipeline\n", "3. Build the model\n", "4. Train the model\n", "5. Test the model\n", "6. Improve the model and repeat the process" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "zF9uvbXNVrVY" }, "source": [ "## Import packages" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "VddxeYBEVrVZ" }, "source": [ "Let's start by importing the required packages. The `os` package is used to read files and directory structure, NumPy is used to convert python list to numpy array and to perform required matrix operations and `matplotlib.pyplot` to plot the graph and display images in the training and validation data." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "rtPGh2MAVrVa" }, "outputs": [], "source": [ "from __future__ import absolute_import, division, print_function, unicode_literals" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Jlchl4x2VrVg" }, "source": [ "Import Tensorflow and the Keras classes needed to construct our model." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "E82grprdYPI0" }, "outputs": [], "source": [ "try:\n", " # %tensorflow_version only exists in Colab.\n", " %tensorflow_version 2.x\n", "except Exception:\n", " pass\n", "import tensorflow as tf" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "L1WtoaOHVrVh" }, "outputs": [], "source": [ "from tensorflow.keras.models import Sequential\n", "from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D\n", "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n", "\n", "import os\n", "import numpy as np\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "UZZI6lNkVrVm" }, "source": [ "## Load data" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "DPHx8-t-VrVo" }, "source": [ "Begin by downloading the dataset. This tutorial uses a filtered version of \u003ca href=\"https://www.kaggle.com/c/dogs-vs-cats/data\" target=\"_blank\"\u003eDogs vs Cats\u003c/a\u003e dataset from Kaggle. Download the archive version of the dataset and store it in the \"/tmp/\" directory." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "C1nqr-CYY6uw" }, "outputs": [], "source": [ "_URL = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip'\n", "\n", "path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=_URL, extract=True)\n", "\n", "PATH = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs_filtered')" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Giv0wMQzVrVw" }, "source": [ "The dataset has the following directory structure:\n", "\n", "\u003cpre\u003e\n", "\u003cb\u003ecats_and_dogs_filtered\u003c/b\u003e\n", "|__ \u003cb\u003etrain\u003c/b\u003e\n", " |______ \u003cb\u003ecats\u003c/b\u003e: [cat.0.jpg, cat.1.jpg, cat.2.jpg ....]\n", " |______ \u003cb\u003edogs\u003c/b\u003e: [dog.0.jpg, dog.1.jpg, dog.2.jpg ...]\n", "|__ \u003cb\u003evalidation\u003c/b\u003e\n", " |______ \u003cb\u003ecats\u003c/b\u003e: [cat.2000.jpg, cat.2001.jpg, cat.2002.jpg ....]\n", " |______ \u003cb\u003edogs\u003c/b\u003e: [dog.2000.jpg, dog.2001.jpg, dog.2002.jpg ...]\n", "\u003c/pre\u003e" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "VpmywIlsVrVx" }, "source": [ "After extracting its contents, assign variables with the proper file path for the training and validation set." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "sRucI3QqVrVy" }, "outputs": [], "source": [ "train_dir = os.path.join(PATH, 'train')\n", "validation_dir = os.path.join(PATH, 'validation')" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "Utv3nryxVrV0" }, "outputs": [], "source": [ "train_cats_dir = os.path.join(train_dir, 'cats') # directory with our training cat pictures\n", "train_dogs_dir = os.path.join(train_dir, 'dogs') # directory with our training dog pictures\n", "validation_cats_dir = os.path.join(validation_dir, 'cats') # directory with our validation cat pictures\n", "validation_dogs_dir = os.path.join(validation_dir, 'dogs') # directory with our validation dog pictures" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "ZdrHHTy2VrV3" }, "source": [ "### Understand the data" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "LblUYjl-VrV3" }, "source": [ "Let's look at how many cats and dogs images are in the training and validation directory:" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "vc4u8e9hVrV4" }, "outputs": [], "source": [ "num_cats_tr = len(os.listdir(train_cats_dir))\n", "num_dogs_tr = len(os.listdir(train_dogs_dir))\n", "\n", "num_cats_val = len(os.listdir(validation_cats_dir))\n", "num_dogs_val = len(os.listdir(validation_dogs_dir))\n", "\n", "total_train = num_cats_tr + num_dogs_tr\n", "total_val = num_cats_val + num_dogs_val" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "g4GGzGt0VrV7" }, "outputs": [], "source": [ "print('total training cat images:', num_cats_tr)\n", "print('total training dog images:', num_dogs_tr)\n", "\n", "print('total validation cat images:', num_cats_val)\n", "print('total validation dog images:', num_dogs_val)\n", "print(\"--\")\n", "print(\"Total training images:\", total_train)\n", "print(\"Total validation images:\", total_val)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "8Lp-0ejxOtP1" }, "source": [ "For convenience, set up variables to use while pre-processing the dataset and training the network." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "3NqNselLVrWA" }, "outputs": [], "source": [ "batch_size = 128\n", "epochs = 15\n", "IMG_HEIGHT = 150\n", "IMG_WIDTH = 150" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "INn-cOn1VrWC" }, "source": [ "## Data preparation" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "5Jfk6aSAVrWD" }, "source": [ "Format the images into appropriately pre-processed floating point tensors before feeding to the network:\n", "\n", "1. Read images from the disk.\n", "2. Decode contents of these images and convert it into proper grid format as per their RGB content.\n", "3. Convert them into floating point tensors.\n", "4. Rescale the tensors from values between 0 and 255 to values between 0 and 1, as neural networks prefer to deal with small input values.\n", "\n", "Fortunately, all these tasks can be done with the `ImageDataGenerator` class provided by `tf.keras`. It can read images from disk and preprocess them into proper tensors. It will also set up generators that convert these images into batches of tensors—helpful when training the network." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "syDdF_LWVrWE" }, "outputs": [], "source": [ "train_image_generator = ImageDataGenerator(rescale=1./255) # Generator for our training data\n", "validation_image_generator = ImageDataGenerator(rescale=1./255) # Generator for our validation data" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "RLciCR_FVrWH" }, "source": [ "After defining the generators for training and validation images, the `flow_from_directory` method load images from the disk, applies rescaling, and resizes the images into the required dimensions." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "Pw94ajOOVrWI" }, "outputs": [], "source": [ "train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,\n", " directory=train_dir,\n", " shuffle=True,\n", " target_size=(IMG_HEIGHT, IMG_WIDTH),\n", " class_mode='binary')" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "2oUoKUzRVrWM" }, "outputs": [], "source": [ "val_data_gen = validation_image_generator.flow_from_directory(batch_size=batch_size,\n", " directory=validation_dir,\n", " target_size=(IMG_HEIGHT, IMG_WIDTH),\n", " class_mode='binary')" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "hyexPJ8CVrWP" }, "source": [ "### Visualize training images" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "60CnhEL4VrWQ" }, "source": [ "Visualize the training images by extracting a batch of images from the training generator—which is 32 images in this example—then plot five of them with `matplotlib`." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "3f0Z7NZgVrWQ" }, "outputs": [], "source": [ "sample_training_images, _ = next(train_data_gen)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "49weMt5YVrWT" }, "source": [ "The `next` function returns a batch from the dataset. The return value of `next` function is in form of `(x_train, y_train)` where x_train is training features and y_train, its labels. Discard the labels to only visualize the training images." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "JMt2RES_VrWU" }, "outputs": [], "source": [ "# This function will plot images in the form of a grid with 1 row and 5 columns where images are placed in each column.\n", "def plotImages(images_arr):\n", " fig, axes = plt.subplots(1, 5, figsize=(20,20))\n", " axes = axes.flatten()\n", " for img, ax in zip( images_arr, axes):\n", " ax.imshow(img)\n", " ax.axis('off')\n", " plt.tight_layout()\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "d_VVg_gEVrWW" }, "outputs": [], "source": [ "plotImages(sample_training_images[:5])" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "b5Ej-HLGVrWZ" }, "source": [ "## Create the model" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "wEgW4i18VrWZ" }, "source": [ "The model consists of three convolution blocks with a max pool layer in each of them. There's a fully connected layer with 512 units on top of it that is activated by a `relu` activation function. The model outputs class probabilities based on binary classification by the `sigmoid` activation function." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "F15-uwLPVrWa" }, "outputs": [], "source": [ "model = Sequential([\n", " Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),\n", " MaxPooling2D(),\n", " Conv2D(32, 3, padding='same', activation='relu'),\n", " MaxPooling2D(),\n", " Conv2D(64, 3, padding='same', activation='relu'),\n", " MaxPooling2D(),\n", " Flatten(),\n", " Dense(512, activation='relu'),\n", " Dense(1, activation='sigmoid')\n", "])" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "PI5cdkMQVrWc" }, "source": [ "### Compile the model\n", "\n", "For this tutorial, choose the *ADAM* optimizer and *binary cross entropy* loss function. To view training and validation accuracy for each training epoch, pass the `metrics` argument." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "6Mg7_TXOVrWd" }, "outputs": [], "source": [ "model.compile(optimizer='adam',\n", " loss='binary_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "2YmQZ3TAVrWg" }, "source": [ "### Model summary\n", "\n", "View all the layers of the network using the model's `summary` method:" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "Vtny8hmBVrWh" }, "outputs": [], "source": [ "model.summary()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "N06iqE8VVrWj" }, "source": [ "### Train the model" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "oub9RtoFVrWk" }, "source": [ "Use the `fit_generator` method of the `ImageDataGenerator` class to train the network." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "KSF2HqhDVrWk" }, "outputs": [], "source": [ "history = model.fit_generator(\n", " train_data_gen,\n", " steps_per_epoch=total_train // batch_size,\n", " epochs=epochs,\n", " validation_data=val_data_gen,\n", " validation_steps=total_val // batch_size\n", ")" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "ojJNteAGVrWo" }, "source": [ "### Visualize training results" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "LZPYT-EmVrWo" }, "source": [ "Now visualize the results after training the network." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "K6oA77ADVrWp" }, "outputs": [], "source": [ "acc = history.history['accuracy']\n", "val_acc = history.history['val_accuracy']\n", "\n", "loss = history.history['loss']\n", "val_loss = history.history['val_loss']\n", "\n", "epochs_range = range(epochs)\n", "\n", "plt.figure(figsize=(8, 8))\n", "plt.subplot(1, 2, 1)\n", "plt.plot(epochs_range, acc, label='Training Accuracy')\n", "plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n", "plt.legend(loc='lower right')\n", "plt.title('Training and Validation Accuracy')\n", "\n", "plt.subplot(1, 2, 2)\n", "plt.plot(epochs_range, loss, label='Training Loss')\n", "plt.plot(epochs_range, val_loss, label='Validation Loss')\n", "plt.legend(loc='upper right')\n", "plt.title('Training and Validation Loss')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "kDnr50l2VrWu" }, "source": [ "As you can see from the plots, training accuracy and validation accuracy are off by large margin and the model has achieved only around **70%** accuracy on the validation set.\n", "\n", "Let's look at what went wrong and try to increase overall performance of the model." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "rLO7yhLlVrWu" }, "source": [ "## Overfitting" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "hNyx3Lp4VrWv" }, "source": [ "In the plots above, the training accuracy is increasing linearly over time, whereas validation accuracy stalls around 70% in the training process. Also, the difference in accuracy between training and validation accuracy is noticeable—a sign of *overfitting*.\n", "\n", "When there are a small number of training examples, the model sometimes learns from noises or unwanted details from training examples—to an extent that it negatively impacts the performance of the model on new examples. This phenomenon is known as overfitting. It means that the model will have a difficult time generalizing on a new dataset.\n", "\n", "There are multiple ways to fight overfitting in the training process. In this tutorial, you'll use *data augmentation* and add *dropout* to our model." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "UOoVpxFwVrWy" }, "source": [ "## Data augmentation" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "Wn_QLciWVrWy" }, "source": [ "Overfitting generally occurs when there are a small number of training examples. One way to fix this problem is to augment the dataset so that it has a sufficient number of training examples. Data augmentation takes the approach of generating more training data from existing training samples by augmenting the samples using random transformations that yield believable-looking images. The goal is the model will never see the exact same picture twice during training. This helps expose the model to more aspects of the data and generalize better.\n", "\n", "Implement this in `tf.keras` using the `ImageDataGenerator` class. Pass different transformations to the dataset and it will take care of applying it during the training process." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "2uJ1G030VrWz" }, "source": [ "### Augment and visualize data" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "hvX7hHlgVrW0" }, "source": [ "Begin by applying random horizontal flip augmentation to the dataset and see how individual images look like after the transformation." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "rlVj6VqaVrW0" }, "source": [ "### Apply horizontal flip" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "xcdvx4TVVrW1" }, "source": [ "Pass `horizontal_flip` as an argument to the `ImageDataGenerator` class and set it to `True` to apply this augmentation." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "Bi1_vHyBVrW2" }, "outputs": [], "source": [ "image_gen = ImageDataGenerator(rescale=1./255, horizontal_flip=True)" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "zvwqmefgVrW3" }, "outputs": [], "source": [ "train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,\n", " directory=train_dir,\n", " shuffle=True,\n", " target_size=(IMG_HEIGHT, IMG_WIDTH))" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "zJpRSxJ-VrW7" }, "source": [ "Take one sample image from the training examples and repeat it five times so that the augmentation is applied to the same image five times." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "RrKGd_jjVrW7" }, "outputs": [], "source": [ "augmented_images = [train_data_gen[0][0][0] for i in range(5)]" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "EvBZoQ9xVrW9" }, "outputs": [], "source": [ "# Re-use the same custom plotting function defined and used\n", "# above to visualize the training images\n", "plotImages(augmented_images)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "i7n9xcqCVrXB" }, "source": [ "### Randomly rotate the image" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "qXnwkzFuVrXB" }, "source": [ "Let's take a look at a different augmentation called rotation and apply 45 degrees of rotation randomly to the training examples." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "1zip35pDVrXB" }, "outputs": [], "source": [ "image_gen = ImageDataGenerator(rescale=1./255, rotation_range=45)" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "kVoWh4OIVrXD" }, "outputs": [], "source": [ "train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,\n", " directory=train_dir,\n", " shuffle=True,\n", " target_size=(IMG_HEIGHT, IMG_WIDTH))\n", "\n", "augmented_images = [train_data_gen[0][0][0] for i in range(5)]" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "wmBx8NhrVrXK" }, "outputs": [], "source": [ "plotImages(augmented_images)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "FOqGPL76VrXM" }, "source": [ "### Apply zoom augmentation" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "NvqXaD8BVrXN" }, "source": [ "Apply a zoom augmentation to the dataset to zoom images up to 50% randomly." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "tGNKLa_YVrXR" }, "outputs": [], "source": [ "# zoom_range from 0 - 1 where 1 = 100%.\n", "image_gen = ImageDataGenerator(rescale=1./255, zoom_range=0.5) # " ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "VOvTs32FVrXU" }, "outputs": [], "source": [ "train_data_gen = image_gen.flow_from_directory(batch_size=batch_size,\n", " directory=train_dir,\n", " shuffle=True,\n", " target_size=(IMG_HEIGHT, IMG_WIDTH))\n", "\n", "augmented_images = [train_data_gen[0][0][0] for i in range(5)]" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "-KQWw8IZVrXZ" }, "outputs": [], "source": [ "plotImages(augmented_images)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "usS13KCNVrXd" }, "source": [ "### Put it all together" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "OC8fIsalVrXd" }, "source": [ "Apply all the previous augmentations. Here, you applied rescale, 45 degree rotation, width shift, height shift, horizontal flip and zoom augmentation to the training images." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "gnr2xujaVrXe" }, "outputs": [], "source": [ "image_gen_train = ImageDataGenerator(\n", " rescale=1./255,\n", " rotation_range=45,\n", " width_shift_range=.15,\n", " height_shift_range=.15,\n", " horizontal_flip=True,\n", " zoom_range=0.5\n", " )" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "K0Efxy7EVrXh" }, "outputs": [], "source": [ "train_data_gen = image_gen_train.flow_from_directory(batch_size=batch_size,\n", " directory=train_dir,\n", " shuffle=True,\n", " target_size=(IMG_HEIGHT, IMG_WIDTH),\n", " class_mode='binary')" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "AW-pV5awVrXl" }, "source": [ "Visualize how a single image would look five different times when passing these augmentations randomly to the dataset." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "z2m68eMhVrXm" }, "outputs": [], "source": [ "augmented_images = [train_data_gen[0][0][0] for i in range(5)]\n", "plotImages(augmented_images)" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "J8cUd7FXVrXq" }, "source": [ "### Create validation data generator" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "a99fDBt7VrXr" }, "source": [ "Generally, only apply data augmentation to the training examples. In this case, only rescale the validation images and convert them into batches using `ImageDataGenerator`." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "54x0aNbKVrXr" }, "outputs": [], "source": [ "image_gen_val = ImageDataGenerator(rescale=1./255)" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "1PCHKzI8VrXv" }, "outputs": [], "source": [ "val_data_gen = image_gen_val.flow_from_directory(batch_size=batch_size,\n", " directory=validation_dir,\n", " target_size=(IMG_HEIGHT, IMG_WIDTH),\n", " class_mode='binary')" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "yQGhdqHFVrXx" }, "source": [ "## Dropout" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "2Iq5TAH_VrXx" }, "source": [ "Another technique to reduce overfitting is to introduce *dropout* to the network. It is a form of *regularization* that forces the weights in the network to take only small values, which makes the distribution of weight values more regular and the network can reduce overfitting on small training examples. Dropout is one of the regularization technique used in this tutorial\n", "\n", "When you apply dropout to a layer it randomly drops out (set to zero) number of output units from the applied layer during the training process. Dropout takes a fractional number as its input value, in the form such as 0.1, 0.2, 0.4, etc. This means dropping out 10%, 20% or 40% of the output units randomly from the applied layer.\n", "\n", "When appling 0.1 dropout to a certain layer, it randomly kills 10% of the output units in each training epoch.\n", "\n", "Create a network architecture with this new dropout feature and apply it to different convolutions and fully-connected layers." ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "DyxxXRmVVrXy" }, "source": [ "## Creating a new network with Dropouts" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "1Ba2LjtkVrXy" }, "source": [ "Here, you apply dropout to first and last max pool layers. Applying dropout will randomly set 20% of the neurons to zero during each training epoch. This helps to avoid overfitting on the training dataset." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "2fjio8EsVrXz" }, "outputs": [], "source": [ "model_new = Sequential([\n", " Conv2D(16, 3, padding='same', activation='relu', \n", " input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),\n", " MaxPooling2D(),\n", " Dropout(0.2),\n", " Conv2D(32, 3, padding='same', activation='relu'),\n", " MaxPooling2D(),\n", " Conv2D(64, 3, padding='same', activation='relu'),\n", " MaxPooling2D(),\n", " Dropout(0.2),\n", " Flatten(),\n", " Dense(512, activation='relu'),\n", " Dense(1, activation='sigmoid')\n", "])" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "tpTgIxWAVrX0" }, "source": [ "### Compile the model" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "1osvc_iTVrX1" }, "source": [ "After introducing dropouts to the network, compile the model and view the layers summary." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "OkIJhS-WVrX1" }, "outputs": [], "source": [ "model_new.compile(optimizer='adam',\n", " loss='binary_crossentropy',\n", " metrics=['accuracy'])\n", "\n", "model_new.summary()" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "7KiDshEUVrX6" }, "source": [ "### Train the model" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "NFj0oVqVVrX6" }, "source": [ "After successfully introducing data augmentations to the training examples and adding dropouts to the network, train this new network:" ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "GWxHs_luVrX7" }, "outputs": [], "source": [ "history = model_new.fit_generator(\n", " train_data_gen,\n", " steps_per_epoch=total_train // batch_size,\n", " epochs=epochs,\n", " validation_data=val_data_gen,\n", " validation_steps=total_val // batch_size\n", ")" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "bbdyqZdxVrYA" }, "source": [ "### Visualize the model" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "OgvF2nt7OtR7" }, "source": [ "Visualize the new model after training, you can see that there is significantly less overfitting than before. The accuracy should go up after training the model for more epochs." ] }, { "cell_type": "code", "execution_count": 0, "metadata": { "colab": {}, "colab_type": "code", "id": "7BTeMuNAVrYC" }, "outputs": [], "source": [ "acc = history.history['accuracy']\n", "val_acc = history.history['val_accuracy']\n", "\n", "loss = history.history['loss']\n", "val_loss = history.history['val_loss']\n", "\n", "epochs_range = range(epochs)\n", "\n", "plt.figure(figsize=(8, 8))\n", "plt.subplot(1, 2, 1)\n", "plt.plot(epochs_range, acc, label='Training Accuracy')\n", "plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n", "plt.legend(loc='lower right')\n", "plt.title('Training and Validation Accuracy')\n", "\n", "plt.subplot(1, 2, 2)\n", "plt.plot(epochs_range, loss, label='Training Loss')\n", "plt.plot(epochs_range, val_loss, label='Validation Loss')\n", "plt.legend(loc='upper right')\n", "plt.title('Training and Validation Loss')\n", "plt.show()" ] } ], "metadata": { "accelerator": "GPU", "colab": { "collapsed_sections": [], "name": "classification.ipynb", "private_outputs": true, "provenance": [], "toc_visible": true, "version": "0.3.2" }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }