{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "image_classification.ipynb",
"version": "0.3.2",
"provenance": [],
"private_outputs": true,
"collapsed_sections": [
"LrO5SdNoVrYH"
],
"toc_visible": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"metadata": {
"id": "TBFXQGKYUc4X",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"##### Copyright 2018 The TensorFlow Authors."
]
},
{
"metadata": {
"id": "1z4xy2gTUc4a",
"colab_type": "code",
"cellView": "form",
"colab": {}
},
"cell_type": "code",
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "FE7KNzPPVrVV"
},
"cell_type": "markdown",
"source": [
"# Image Classification using tf.keras"
]
},
{
"metadata": {
"colab_type": "text",
"id": "KwQtSOz0VrVX"
},
"cell_type": "markdown",
"source": [
""
]
},
{
"metadata": {
"colab_type": "text",
"id": "gN7G9GFmVrVY"
},
"cell_type": "markdown",
"source": [
"In this tutorial, we will discuss how to classify cats vs dogs from images. We'll build an image classifier using `tf.keras.Sequential` model and load data using `tf.keras.preprocessing.image.ImageDataGenerator`.\n",
"\n",
"## Specific concepts that will be covered:\n",
"In the process, we will build practical experience and develop intuition around the following concepts\n",
"\n",
"* Building _data input pipelines_ using the `tf.keras.preprocessing.image.ImageDataGenerator`class — How can we efficiently work with data on disk to interface with our model? \n",
"* _Overfitting_ - what is it, how to identify it, and how can we prevent it? \n",
"* _Data Augmentation_ and _Dropout_ - Key techniques to fight overfitting in computer vision tasks that we will incorporate into our data pipeline and image classifier model. \n",
"\n",
"## We will follow the general machine learning workflow:\n",
"\n",
"1. Examine and understand data\n",
"2. Build an input pipeline \n",
"3. Build our model\n",
"4. Train our model\n",
"5. Test our model\n",
"6. Improve our model/Repeat the process\n",
"\n",
"**Audience:** This post is geared towards beginners with some Keras API and ML background. To get the most out of this post, you should have some basic ML background and know what CNNs are."
]
},
{
"metadata": {
"colab_type": "text",
"id": "zF9uvbXNVrVY"
},
"cell_type": "markdown",
"source": [
"# Importing packages"
]
},
{
"metadata": {
"colab_type": "text",
"id": "VddxeYBEVrVZ"
},
"cell_type": "markdown",
"source": [
"Let's start by importing required packages. **os** package is used to read files and directory structure, **numpy** is used to convert python list to numpy array and to perform required matrix operations and **matplotlib.pyplot** is used to plot the graph and display images in our training and validation data."
]
},
{
"metadata": {
"colab_type": "code",
"id": "rtPGh2MAVrVa",
"colab": {}
},
"cell_type": "code",
"source": [
"import os\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "TI3XEQuJVrVd"
},
"cell_type": "markdown",
"source": [
"The data we are using is initially available as \".zip\" archive file. We are using **zipfile** to extract its contents."
]
},
{
"metadata": {
"colab_type": "code",
"id": "19eGwtQ4VrVe",
"colab": {}
},
"cell_type": "code",
"source": [
"import zipfile"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "Jlchl4x2VrVg"
},
"cell_type": "markdown",
"source": [
"Let's import **Tensorflow as tf** and from **tf.keras**, we need to import different methods to construct our model. Uses of all these methods will be explained as we progress with the tutorial."
]
},
{
"metadata": {
"colab_type": "code",
"id": "L1WtoaOHVrVh",
"colab": {}
},
"cell_type": "code",
"source": [
"import tensorflow as tf\n",
"from tensorflow.keras.models import Sequential\n",
"from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "UZZI6lNkVrVm"
},
"cell_type": "markdown",
"source": [
"# Data Loading"
]
},
{
"metadata": {
"colab_type": "text",
"id": "DPHx8-t-VrVo"
},
"cell_type": "markdown",
"source": [
"In order to build our image classifier, we can begin by downloading the dataset. The dataset we are using is a filtered version of Dogs vs Cats dataset from Kaggle. We first need to download the archive version of the dataset and after the download we are storing it to \"/tmp/\" directory."
]
},
{
"metadata": {
"colab_type": "code",
"id": "rpUSoFjuVrVp",
"colab": {}
},
"cell_type": "code",
"source": [
"!wget --no-check-certificate \\\n",
" https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip \\\n",
" -O /tmp/cats_and_dogs_filtered.zip"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "_lPjfOmNVrVs"
},
"cell_type": "markdown",
"source": [
"After downloading the dataset, we need to extract its contents."
]
},
{
"metadata": {
"colab_type": "code",
"id": "OYmOylPlVrVt",
"colab": {}
},
"cell_type": "code",
"source": [
"local_zip = '/tmp/cats_and_dogs_filtered.zip' # local path of downloaded .zip file\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('/tmp') # contents are extracted to '/tmp' folder\n",
"zip_ref.close()"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "Giv0wMQzVrVw"
},
"cell_type": "markdown",
"source": [
"The dataset we have downloaded has following directory structure. \n",
"\n",
"\n",
"cats_and_dogs_filtered\n",
"|__ train\n",
" |______ cats: [cat.0.jpg, cat.1.jpg, cat.2.jpg ....]\n",
" |______ dogs: [dog.0.jpg, dog.1.jpg, dog.2.jpg ...]\n",
"|__ validation\n",
" |______ cats: [cat.2000.jpg, cat.2001.jpg, cat.2002.jpg ....]\n",
" |______ dogs: [dog.2000.jpg, dog.2001.jpg, dog.2002.jpg ...]\n",
""
]
},
{
"metadata": {
"colab_type": "text",
"id": "VpmywIlsVrVx"
},
"cell_type": "markdown",
"source": [
"After extracting its contents, we need to assign variables with the proper file path for training and validation set."
]
},
{
"metadata": {
"colab_type": "code",
"id": "sRucI3QqVrVy",
"colab": {}
},
"cell_type": "code",
"source": [
"base_dir = '/tmp/cats_and_dogs_filtered'\n",
"train_dir = os.path.join(base_dir, 'train')\n",
"validation_dir = os.path.join(base_dir, 'validation')"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "Utv3nryxVrV0",
"colab": {}
},
"cell_type": "code",
"source": [
"train_cats_dir = os.path.join(train_dir, 'cats') # directory with our training cat pictures\n",
"train_dogs_dir = os.path.join(train_dir, 'dogs') # directory with our training dog pictures\n",
"validation_cats_dir = os.path.join(validation_dir, 'cats') # directory with our validation cat pictures\n",
"validation_dogs_dir = os.path.join(validation_dir, 'dogs') # directory with our validation dog pictures"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "ZdrHHTy2VrV3"
},
"cell_type": "markdown",
"source": [
"### Understanding our data"
]
},
{
"metadata": {
"colab_type": "text",
"id": "LblUYjl-VrV3"
},
"cell_type": "markdown",
"source": [
"Let's look at how many cats and dogs images we have in our training and validation directory"
]
},
{
"metadata": {
"colab_type": "code",
"id": "vc4u8e9hVrV4",
"colab": {}
},
"cell_type": "code",
"source": [
"num_cats_tr = len(os.listdir(train_cats_dir))\n",
"num_dogs_tr = len(os.listdir(train_dogs_dir))\n",
"\n",
"num_cats_val = len(os.listdir(validation_cats_dir))\n",
"num_dogs_val = len(os.listdir(validation_dogs_dir))\n",
"\n",
"total_train = num_cats_tr + num_dogs_tr\n",
"total_val = num_cats_val + num_dogs_val"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "g4GGzGt0VrV7",
"colab": {}
},
"cell_type": "code",
"source": [
"print('total training cat images:', num_cats_tr)\n",
"print('total training dog images:', num_dogs_tr)\n",
"\n",
"print('total validation cat images:', num_cats_val)\n",
"print('total validation dog images:', num_dogs_val)\n",
"print(\"--\")\n",
"print(\"Total training images:\", total_train)\n",
"print(\"Total validation images:\", total_val)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "tdsI_L-NVrV_"
},
"cell_type": "markdown",
"source": [
"# Setting Model Parameters"
]
},
{
"metadata": {
"colab_type": "text",
"id": "8Lp-0ejxOtP1"
},
"cell_type": "markdown",
"source": [
"For convenience, let us set up variables that will be later used while pre-processing our dataset and training our network."
]
},
{
"metadata": {
"colab_type": "code",
"id": "3NqNselLVrWA",
"colab": {}
},
"cell_type": "code",
"source": [
"batch_size = 100\n",
"epochs = 15\n",
"IMG_SHAPE = 150 # Our training data consists of images with width of 150 pixels and height of 150 pixels"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "INn-cOn1VrWC"
},
"cell_type": "markdown",
"source": [
"# Data Preparation "
]
},
{
"metadata": {
"colab_type": "text",
"id": "5Jfk6aSAVrWD"
},
"cell_type": "markdown",
"source": [
"Images should be formatted into appropriately pre-processed floating point tensors before being fed into the network. The steps involving preparing these images are:\n",
"\n",
"1. Read images from the disk\n",
"2. Decode contents of these images and convert it into proper grid format as per their RGB content\n",
"3. Convert them into floating point tensors\n",
"4. Rescale the tensors from values between 0 and 255 to values between 0 and 1, as neural networks prefer to deal with small input values.\n",
"\n",
"Fortunately, all these tasks can be done using a single class provided in **tf.keras** preprocessing module, called **ImageDataGenerator**. Not only it can read images from the disks and preprocess images into proper tensors, but it will also set up generators that will turn these images into batches of tensors, which will be very helpful while training our network as we need to pass our input to the network in the form of batches.\n",
"\n",
"We can easily set up this using a couple of lines of code."
]
},
{
"metadata": {
"colab_type": "code",
"id": "syDdF_LWVrWE",
"colab": {}
},
"cell_type": "code",
"source": [
"train_image_generator = ImageDataGenerator(rescale=1./255) # Generator for our training data\n",
"validation_image_generator = ImageDataGenerator(rescale=1./255) # Generator for our validation data"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "RLciCR_FVrWH"
},
"cell_type": "markdown",
"source": [
"After defining our generators for training and validation images, **flow_from_directory** method will load images from the disk and will apply rescaling and will resize them into required dimensions using single line of code."
]
},
{
"metadata": {
"colab_type": "code",
"id": "Pw94ajOOVrWI",
"colab": {}
},
"cell_type": "code",
"source": [
"train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size, \n",
" directory=train_dir, \n",
" # Its usually best practice to shuffle the training data\n",
" shuffle=True, \n",
" target_size=(IMG_SHAPE,IMG_SHAPE), #(150,150) \n",
" class_mode='binary')"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "2oUoKUzRVrWM",
"colab": {}
},
"cell_type": "code",
"source": [
"val_data_gen = validation_image_generator.flow_from_directory(batch_size=batch_size, \n",
" directory=validation_dir, \n",
" target_size=(IMG_SHAPE,IMG_SHAPE), #(150,150)\n",
" class_mode='binary')"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "hyexPJ8CVrWP"
},
"cell_type": "markdown",
"source": [
"### Visualizing Training images"
]
},
{
"metadata": {
"colab_type": "text",
"id": "60CnhEL4VrWQ"
},
"cell_type": "markdown",
"source": [
"We can visualize our training images by using following lines of code which will first extract a batch of images from training generator, which is 32 images in our case and then we will plot 5 of them using **matplotlib**"
]
},
{
"metadata": {
"colab_type": "code",
"id": "3f0Z7NZgVrWQ",
"colab": {}
},
"cell_type": "code",
"source": [
"sample_training_images, _ = next(train_data_gen) "
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "49weMt5YVrWT"
},
"cell_type": "markdown",
"source": [
"**next** function returns a batch from the dataset. The return value of **next** function is in form of (x_train, y_train) where x_train is training features and y_train, its labels. We are discarding the labels in above situation because we only want to visualize our training images."
]
},
{
"metadata": {
"colab_type": "code",
"id": "JMt2RES_VrWU",
"colab": {}
},
"cell_type": "code",
"source": [
"# This function will plot images in the form of a grid with 1 row and 5 columns where images are placed in each column.\n",
"def plotImages(images_arr):\n",
" fig, axes = plt.subplots(1, 5, figsize=(20,20))\n",
" axes = axes.flatten()\n",
" for img, ax in zip( images_arr, axes):\n",
" ax.imshow(img)\n",
" plt.tight_layout()\n",
" plt.show()"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "d_VVg_gEVrWW",
"colab": {}
},
"cell_type": "code",
"source": [
"plotImages(sample_training_images[:5])"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "b5Ej-HLGVrWZ"
},
"cell_type": "markdown",
"source": [
"# Model Creation"
]
},
{
"metadata": {
"colab_type": "text",
"id": "wEgW4i18VrWZ"
},
"cell_type": "markdown",
"source": [
"The model consists of 3 convolution blocks with max pool layer in each of them. We have a fully connected layer with 512 units on top of it, which is activated by **relu** activation function. Model will output class probabilities based on binary classification which is done by **sigmoid** activation function. "
]
},
{
"metadata": {
"colab_type": "code",
"id": "F15-uwLPVrWa",
"colab": {}
},
"cell_type": "code",
"source": [
"model = Sequential()\n",
"model.add(Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_SHAPE,IMG_SHAPE, 3,))) \n",
"model.add(MaxPooling2D(pool_size=(2, 2)))\n",
"model.add(Conv2D(32, 3, padding='same', activation='relu'))\n",
"model.add(MaxPooling2D(pool_size=(2, 2)))\n",
"\n",
"model.add(Conv2D(64, 3, padding='same', activation='relu'))\n",
"model.add(MaxPooling2D(pool_size=(2, 2)))\n",
"\n",
"model.add(Flatten())\n",
"model.add(Dense(512, activation='relu'))\n",
"model.add(Dense(1, activation='sigmoid'))"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "PI5cdkMQVrWc"
},
"cell_type": "markdown",
"source": [
"### Compiling the model\n",
"\n",
"We will use **ADAM** optimizer as our choice of optimizer for this task and **binary cross entropy** function as a loss function. We would also like to look at training and validation accuracy on each epoch as we train our network, for that we are passing it in the metrics argument."
]
},
{
"metadata": {
"colab_type": "code",
"id": "6Mg7_TXOVrWd",
"colab": {}
},
"cell_type": "code",
"source": [
"model.compile(optimizer='adam', \n",
" loss='binary_crossentropy', \n",
" metrics=['accuracy']\n",
" )"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "2YmQZ3TAVrWg"
},
"cell_type": "markdown",
"source": [
"### Model Summary\n",
"\n",
"Let's look at all the layers of our network using **summary** method."
]
},
{
"metadata": {
"colab_type": "code",
"id": "Vtny8hmBVrWh",
"colab": {}
},
"cell_type": "code",
"source": [
"model.summary()"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "N06iqE8VVrWj"
},
"cell_type": "markdown",
"source": [
"### Train the model"
]
},
{
"metadata": {
"colab_type": "text",
"id": "oub9RtoFVrWk"
},
"cell_type": "markdown",
"source": [
"Its time we train our network. We will use **fit_generator** function to train our network instead of **fit** function, as we are using **ImageDataGenerator** class to generate batches of training and validation data for our network. "
]
},
{
"metadata": {
"colab_type": "code",
"id": "KSF2HqhDVrWk",
"colab": {}
},
"cell_type": "code",
"source": [
"history = model.fit_generator(\n",
" train_data_gen,\n",
" steps_per_epoch=int(np.ceil(total_train / float(batch_size))),\n",
" epochs=epochs,\n",
" validation_data=val_data_gen,\n",
" validation_steps=int(np.ceil(total_val / float(batch_size)))\n",
")"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "ojJNteAGVrWo"
},
"cell_type": "markdown",
"source": [
"### Visualizing results of the training"
]
},
{
"metadata": {
"colab_type": "text",
"id": "LZPYT-EmVrWo"
},
"cell_type": "markdown",
"source": [
"Let us now visualize the results we get after training our network."
]
},
{
"metadata": {
"colab_type": "code",
"id": "K6oA77ADVrWp",
"colab": {}
},
"cell_type": "code",
"source": [
"acc = history.history['acc']\n",
"val_acc = history.history['val_acc']\n",
"\n",
"loss = history.history['loss']\n",
"val_loss = history.history['val_loss']\n",
"\n",
"epochs_range = range(epochs)\n",
"\n",
"plt.figure(figsize=(8, 8))\n",
"plt.subplot(1, 2, 1)\n",
"plt.plot(epochs_range, acc, label='Training Accuracy')\n",
"plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n",
"plt.legend(loc='lower right')\n",
"plt.title('Training and Validation Accuracy')\n",
"\n",
"plt.subplot(1, 2, 2)\n",
"plt.plot(epochs_range, loss, label='Training Loss')\n",
"plt.plot(epochs_range, val_loss, label='Validation Loss')\n",
"plt.legend(loc='upper right')\n",
"plt.title('Training and Validation Loss')\n",
"plt.show()"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "kDnr50l2VrWu"
},
"cell_type": "markdown",
"source": [
"As we can see from the plots, training accuracy and validation accuracy are off by large margin and our model has achieved only around **70%** accuracy on the validation set, let us analyse what went wrong there and try to increase overall performance of the model."
]
},
{
"metadata": {
"colab_type": "text",
"id": "rLO7yhLlVrWu"
},
"cell_type": "markdown",
"source": [
"# Overfitting"
]
},
{
"metadata": {
"colab_type": "text",
"id": "hNyx3Lp4VrWv"
},
"cell_type": "markdown",
"source": [
"If we look at the plots above, we can see that training accuracy is increasing linearly over time, whereas validation accuracy stalls around 70% after some time in our training process. Moreover, the difference in accuracy between training and validation accuracy is noticeably large. This is a sign of overfitting. \n",
"\n",
"When we have small number of training examples, our model sometimes learns from noises or unwanted details from our training examples to an extent that it negatively impacts the performance of the model on new examples. This phenomenon is known as overfitting. It simply means that our model will have a hard time generalizing well on a new dataset.\n",
"\n",
"There are multiple ways to fight overfitting in our training process. Two of them is **Data Augmentation** and adding **Dropout** to our model. Let's start with Data Augmentation and see how it will help us to fight overfitting in our model.\n",
"\n",
"To begin, we can clear our previous session and start with a new one."
]
},
{
"metadata": {
"colab_type": "code",
"id": "_L8qd6IxVrWw",
"colab": {}
},
"cell_type": "code",
"source": [
"# Clear resources\n",
"tf.keras.backend.clear_session()\n",
"epochs = 80"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "UOoVpxFwVrWy"
},
"cell_type": "markdown",
"source": [
"# Data Augmentation"
]
},
{
"metadata": {
"colab_type": "text",
"id": "Wn_QLciWVrWy"
},
"cell_type": "markdown",
"source": [
"Overfitting generally occurs when we have small number of training examples. One way to fix this problem is to augment our dataset so that it has sufficient number of training examples. Data augmentation takes the approach of generating more training data from existing training samples, by augmenting the samples via a number of random transformations that yield believable-looking images. The goal is that at training time, your model will never see the exact same picture twice. This helps expose the model to more aspects of the data and generalize better.\n",
"\n",
"In **tf.keras** we can implement this using the same **ImageDataGenerator** class we used before. We can simply pass different transformations we would want to our dataset as a form of arguments and it will take care of applying it to the dataset during our training process. "
]
},
{
"metadata": {
"colab_type": "text",
"id": "2uJ1G030VrWz"
},
"cell_type": "markdown",
"source": [
"## Augmenting and visualizing data"
]
},
{
"metadata": {
"colab_type": "text",
"id": "hvX7hHlgVrW0"
},
"cell_type": "markdown",
"source": [
"We can begin by applying random horizontal flip augmentation to our dataset and see how individual images will look like after the transformation."
]
},
{
"metadata": {
"colab_type": "text",
"id": "rlVj6VqaVrW0"
},
"cell_type": "markdown",
"source": [
"### Applying Horizontal Flip"
]
},
{
"metadata": {
"colab_type": "text",
"id": "xcdvx4TVVrW1"
},
"cell_type": "markdown",
"source": [
"We can simply pass **horizontal_flip** as an argument to our ImageDataGenerator class and set it to **True** to apply this augmentation."
]
},
{
"metadata": {
"colab_type": "code",
"id": "Bi1_vHyBVrW2",
"colab": {}
},
"cell_type": "code",
"source": [
"image_gen = ImageDataGenerator(rescale=1./255, horizontal_flip=True)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "zvwqmefgVrW3",
"colab": {}
},
"cell_type": "code",
"source": [
"train_data_gen = image_gen.flow_from_directory(\n",
" batch_size=batch_size, \n",
" directory=train_dir, \n",
" shuffle=True, \n",
" target_size=(IMG_SHAPE,IMG_SHAPE)\n",
" )"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "zJpRSxJ-VrW7"
},
"cell_type": "markdown",
"source": [
"Let's take 1 sample image from our training examples and repeat it 5 times so that the augmentation can be applied to the same image 5 times over randomly, to see the augmentation in action."
]
},
{
"metadata": {
"colab_type": "code",
"id": "RrKGd_jjVrW7",
"colab": {}
},
"cell_type": "code",
"source": [
"augmented_images = [train_data_gen[0][0][0] for i in range(5)]"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "EvBZoQ9xVrW9",
"colab": {}
},
"cell_type": "code",
"source": [
"# Here, we are simply re-using the same custom plotting function \n",
"# we defined and used above to visualize our training images\n",
"plotImages(augmented_images)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "i7n9xcqCVrXB"
},
"cell_type": "markdown",
"source": [
"### Randomly rotating the image"
]
},
{
"metadata": {
"colab_type": "text",
"id": "qXnwkzFuVrXB"
},
"cell_type": "markdown",
"source": [
"Let's take a look at different augmentation called rotation and apply 45 degrees of rotation randomly to our training examples. "
]
},
{
"metadata": {
"colab_type": "code",
"id": "1zip35pDVrXB",
"colab": {}
},
"cell_type": "code",
"source": [
"image_gen = ImageDataGenerator(rescale=1./255, rotation_range=45)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "kVoWh4OIVrXD",
"colab": {}
},
"cell_type": "code",
"source": [
"train_data_gen = image_gen.flow_from_directory(\n",
" batch_size=batch_size, \n",
" directory=train_dir, \n",
" shuffle=True, \n",
" target_size=(IMG_SHAPE, IMG_SHAPE)\n",
" )\n",
"\n",
"augmented_images = [train_data_gen[0][0][0] for i in range(5)]"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "wmBx8NhrVrXK",
"colab": {}
},
"cell_type": "code",
"source": [
"plotImages(augmented_images)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "FOqGPL76VrXM"
},
"cell_type": "markdown",
"source": [
"### Applying Zoom"
]
},
{
"metadata": {
"colab_type": "text",
"id": "NvqXaD8BVrXN"
},
"cell_type": "markdown",
"source": [
"Let's apply Zoom augmentation to our dataset to zoom images up to 50% randomly."
]
},
{
"metadata": {
"colab_type": "code",
"id": "tGNKLa_YVrXR",
"colab": {}
},
"cell_type": "code",
"source": [
"image_gen = ImageDataGenerator(rescale=1./255, zoom_range=0.5)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "VOvTs32FVrXU",
"colab": {}
},
"cell_type": "code",
"source": [
"train_data_gen = image_gen.flow_from_directory(\n",
" batch_size=batch_size, \n",
" directory=train_dir, \n",
" shuffle=True, \n",
" target_size=(IMG_SHAPE, IMG_SHAPE)\n",
" )\n",
"\n",
"augmented_images = [train_data_gen[0][0][0] for i in range(5)]"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "-KQWw8IZVrXZ",
"colab": {}
},
"cell_type": "code",
"source": [
"plotImages(augmented_images)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "usS13KCNVrXd"
},
"cell_type": "markdown",
"source": [
"### Putting it all together"
]
},
{
"metadata": {
"colab_type": "text",
"id": "OC8fIsalVrXd"
},
"cell_type": "markdown",
"source": [
"We can apply all the augmentations we saw above and even more with just one line of code. We can simply pass the augmentations as arguments with proper values and that would be all.\n",
"\n",
"Here, we have applied rescale, rotation of 45 degrees, width shift, height shift, horizontal flip and zoom augmentation to our training images."
]
},
{
"metadata": {
"colab_type": "code",
"id": "gnr2xujaVrXe",
"colab": {}
},
"cell_type": "code",
"source": [
"image_gen_train = ImageDataGenerator(\n",
" rescale=1./255, \n",
" rotation_range=45, \n",
" width_shift_range=.15, \n",
" height_shift_range=.15, \n",
" horizontal_flip=True, \n",
" zoom_range=0.5\n",
" )"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "K0Efxy7EVrXh",
"colab": {}
},
"cell_type": "code",
"source": [
"train_data_gen = image_gen_train.flow_from_directory(\n",
" batch_size=batch_size, \n",
" directory=train_dir, \n",
" shuffle=True, \n",
" target_size=(IMG_SHAPE,IMG_SHAPE),\n",
" class_mode='binary'\n",
" )"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "AW-pV5awVrXl"
},
"cell_type": "markdown",
"source": [
"Let's visualize how a single image would look like 5 different times, when we pass these augmentations randomly to our dataset. "
]
},
{
"metadata": {
"colab_type": "code",
"id": "z2m68eMhVrXm",
"colab": {}
},
"cell_type": "code",
"source": [
"augmented_images = [train_data_gen[0][0][0] for i in range(5)]\n",
"plotImages(augmented_images)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "J8cUd7FXVrXq"
},
"cell_type": "markdown",
"source": [
"### Creating Validation Data generator"
]
},
{
"metadata": {
"colab_type": "text",
"id": "a99fDBt7VrXr"
},
"cell_type": "markdown",
"source": [
"Generally, we only apply data augmentation to our training examples. So, in this case we are only rescaling our validation images and converting them into batches using ImageDataGenerator."
]
},
{
"metadata": {
"colab_type": "code",
"id": "54x0aNbKVrXr",
"colab": {}
},
"cell_type": "code",
"source": [
"image_gen_val = ImageDataGenerator(rescale=1./255)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "code",
"id": "1PCHKzI8VrXv",
"colab": {}
},
"cell_type": "code",
"source": [
"val_data_gen = image_gen_val.flow_from_directory(batch_size=batch_size, \n",
" directory=validation_dir, \n",
" target_size=(IMG_SHAPE, IMG_SHAPE),\n",
" class_mode='binary')"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "yQGhdqHFVrXx"
},
"cell_type": "markdown",
"source": [
"# Dropout "
]
},
{
"metadata": {
"colab_type": "text",
"id": "2Iq5TAH_VrXx"
},
"cell_type": "markdown",
"source": [
"Another technique we can use to reduce overfitting is to introduce something called **Dropout** to our network. If you are not familiar with the term **regularization**, it simply means forcing the weights in your network to take only small values, which makes the distribution of weight values more regular and the network can reduce overfitting on small training examples. Dropout is one of the regularization technique we will be using in this tutorial.\n",
"\n",
"When we apply Dropout to a layer it will randomly drop out (set to zero) number of output units from the applied layer during the training process. Dropout takes fraction number as its input value, in the form such as 0.1, 0.2, 0.4 etc. which simply means dropping out 10%, 20% or 40% of the output units randomly from the applied layer. \n",
"\n",
"When we apply 0.1 value as a Dropout value to a certain layer, it will kill 10% of its output units randomly in each training epoch.\n",
"\n",
"Let's create a network architecture with this new Dropout feature and apply it to different Convolutions and Fully Connected layers."
]
},
{
"metadata": {
"colab_type": "text",
"id": "DyxxXRmVVrXy"
},
"cell_type": "markdown",
"source": [
"# Creating a new network with Dropouts"
]
},
{
"metadata": {
"colab_type": "text",
"id": "1Ba2LjtkVrXy"
},
"cell_type": "markdown",
"source": [
"Here, we have applied Dropouts to first and last max pool layers and to a fully connected layer which has 512 output units. 30% of the first and last max pool layer and 10% of fully connected layer output units will be set to zero randomly during each epoch while training. "
]
},
{
"metadata": {
"colab_type": "code",
"id": "2fjio8EsVrXz",
"colab": {}
},
"cell_type": "code",
"source": [
"model = Sequential()\n",
"model.add(Conv2D(16, 3, padding='same', activation='relu', input_shape=(150,150,3,))) \n",
"model.add(MaxPooling2D(pool_size=2))\n",
"model.add(Dropout(0.3))\n",
"model.add(Conv2D(32, 3, padding='same', activation='relu'))\n",
"model.add(MaxPooling2D(pool_size=2))\n",
"\n",
"model.add(Conv2D(64, 3, padding='same', activation='relu'))\n",
"model.add(MaxPooling2D(pool_size=2))\n",
"\n",
"model.add(Dropout(0.3))\n",
"model.add(Flatten())\n",
"model.add(Dense(512, activation='relu'))\n",
"model.add(Dropout(0.1))\n",
"model.add(Dense(1, activation='sigmoid'))"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "tpTgIxWAVrX0"
},
"cell_type": "markdown",
"source": [
"### Compiling the model"
]
},
{
"metadata": {
"colab_type": "text",
"id": "1osvc_iTVrX1"
},
"cell_type": "markdown",
"source": [
"After introducing Dropouts to our network, let us compile the model and see the layers summary."
]
},
{
"metadata": {
"colab_type": "code",
"id": "OkIJhS-WVrX1",
"colab": {}
},
"cell_type": "code",
"source": [
"model.compile(optimizer='adam', \n",
" loss='binary_crossentropy', \n",
" metrics=['accuracy']\n",
" )\n",
"\n",
"model.summary()"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "7KiDshEUVrX6"
},
"cell_type": "markdown",
"source": [
"### Training the model"
]
},
{
"metadata": {
"colab_type": "text",
"id": "NFj0oVqVVrX6"
},
"cell_type": "markdown",
"source": [
"After successfully introducing Data Augmentations to our training examples and adding Dropouts to our network, let us now train this new network."
]
},
{
"metadata": {
"colab_type": "code",
"id": "GWxHs_luVrX7",
"colab": {}
},
"cell_type": "code",
"source": [
"history = model.fit_generator(\n",
" train_data_gen,\n",
" steps_per_epoch=int(np.ceil(total_train / float(batch_size))),\n",
" epochs=epochs,\n",
" validation_data=val_data_gen,\n",
" validation_steps=int(np.ceil(total_val / float(batch_size)))\n",
")"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "bbdyqZdxVrYA"
},
"cell_type": "markdown",
"source": [
"### Visualizing the model"
]
},
{
"metadata": {
"colab_type": "text",
"id": "OgvF2nt7OtR7"
},
"cell_type": "markdown",
"source": [
"Let us visualize our new model after the training and see if we can find any sign of overfitting in it."
]
},
{
"metadata": {
"colab_type": "code",
"id": "7BTeMuNAVrYC",
"colab": {}
},
"cell_type": "code",
"source": [
"acc = history.history['acc']\n",
"val_acc = history.history['val_acc']\n",
"\n",
"loss = history.history['loss']\n",
"val_loss = history.history['val_loss']\n",
"\n",
"epochs_range = range(epochs)\n",
"\n",
"plt.figure(figsize=(8, 8))\n",
"plt.subplot(1, 2, 1)\n",
"plt.plot(epochs_range, acc, label='Training Accuracy')\n",
"plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n",
"plt.legend(loc='lower right')\n",
"plt.title('Training and Validation Accuracy')\n",
"\n",
"plt.subplot(1, 2, 2)\n",
"plt.plot(epochs_range, loss, label='Training Loss')\n",
"plt.plot(epochs_range, val_loss, label='Validation Loss')\n",
"plt.legend(loc='upper right')\n",
"plt.title('Training and Validation Loss')\n",
"plt.show()"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"colab_type": "text",
"id": "LrO5SdNoVrYH"
},
"cell_type": "markdown",
"source": [
"### Evaluating our model\n",
"As we can see, our learning curves are much better than before and we don't see nearly as much overfitting as we did previously, and our model is able to achieve an accuracy of ~**75%**!"
]
},
{
"metadata": {
"colab_type": "text",
"id": "MC71agTfVrYI"
},
"cell_type": "markdown",
"source": [
"# Key Takeaways:\n",
"\n",
"## What we covered in this tutorial:\n",
"* **Building data input pipelines using the ImageDataGenerator** - The ImageDataGenerator class allows us to easily load our data from the disk and efficiently add any arbitrary transformation to it. \n",
" * Training with a ImageDataGenerator is extremely easy as well\n",
" * Working with ImageDataGenerator allows very powerful and fast transformation to be applied to your data\n",
"* **Overfitting** - Overfitting is a key problem that can severely harm your model's performance. Crucial techniques to know when combatting overfitting are **Dropout** and **Data Augmentation**\n",
" * Dropout - is a powerful regularization technique that is found in almost every model \n",
" * Data Augmentation - is a cheap and powerful way to augment your dataset and allow your model to generalize to unseen data. \n"
]
},
{
"metadata": {
"colab_type": "code",
"id": "3DVg5Nl-VrYJ",
"colab": {}
},
"cell_type": "code",
"source": [
""
],
"execution_count": 0,
"outputs": []
}
]
} ![](\"https://www.tensorflow.org/images/tf_logo_32px.png\"){kind=logo}
View on TensorFlow.org\n",
" ![](\"https://www.tensorflow.org/images/colab_logo_32px.png\"){kind=logo}
Run in Google Colab\n",
" ![](\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\")
View source on GitHub\n",
"