{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Inception Network Tutorial on Fashion MNIST Data Set\n", "\n", "\n", "This turorial gives a breif intro on using CNN for train and prediction (i.e. inference)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Let's Load Some Packages" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Plots.GRBackend()" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using MLDatasets\n", "using NumNN\n", "using Plots\n", "gr()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Temp for ProgressMeter.jl Package\n", "\n", "**Uncomment the following line if you run this code for the first time***" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# ] add https://github.com/timholy/ProgressMeter.jl.git ;" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "using ProgressMeter\n", "ProgressMeter.ijulia_behavior(:clear);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Load the Train/Test Data/Labels" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "X_train, Y_train = FashionMNIST.traindata(Float64);\n", "X_test, Y_test = FashionMNIST.testdata(Float64);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Let's Prepare the data/labels\n", "\n", "Since the shape of the MNIST data is `(28,28,size)` and to use it in CNN 2D it must be as 4D Array" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "X_train = reshape(X_train, (size(X_train)[1:2]..., 1, size(X_train)[end]))\n", "X_test = reshape(X_test, (size(X_test)[1:2]...,1,size(X_test)[end]))\n", "\n", "Y_train = oneHot(Y_train)\n", "Y_test = oneHot(Y_test);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define the Layers" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "X_Input = Input(X_train)\n", "Xc = [\n", " Conv2D(3, (3,3), padding=:same)(X_Input),\n", " Conv2D(4, (5,5), padding=:same)(X_Input),\n", " Conv2D(10, (1,1), padding=:same)(X_Input),\n", " MaxPool2D((2,2); padding=:same)(X_Input),\n", " AveragePool2D((3,3); padding=:same)(X_Input),\n", "]\n", "\n", "X = ConcatLayer()(Xc)\n", "X = BatchNorm(dim=3)(X) #to normalize across the channels\n", "X = Activation(:relu)(X)\n", "X = MaxPool2D((2,2))(X);" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "Xc = [\n", " Conv2D(6, (3,3), padding=:same)(X),\n", " Conv2D(8, (5,5), padding=:same)(X),\n", " Conv2D(10, (1,1), padding=:same)(X),\n", " MaxPool2D((2,2); padding=:same)(X),\n", " AveragePool2D((3,3); padding=:same)(X),\n", "]\n", "X = ConcatLayer()(Xc)\n", "X = BatchNorm(dim=3)(X) #to normalize across the channels\n", "X = Activation(:relu)(X)\n", "X = AveragePool2D((2,2))(X);" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "X = Flatten()(X)\n", "X_Output = FCLayer(10, :softmax)(X);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Let's Define Our Model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This will also initialize the `Layer`s' parameters" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "model = Model(X_train,Y_train,X_Input,X_Output, 0.005; optimizer=:adam);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Let's use `predict` to see the current Accuracy\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:00:25\u001b[39m\n", "\u001b[34m Instances 10000: 10000\u001b[39m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "The accuracy of Test Data before the training process 0.0222\n", "The cost of Test Data before the training process 2.3156\n" ] } ], "source": [ "TestP = predict(model, X_test, Y_test);\n", "\n", "println()\n", "println(\"The accuracy of Test Data before the training process $(round(TestP[:accuracy], digits=4))\")\n", "println(\"The cost of Test Data before the training process $(round(TestP[:cost], digits=4))\")" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:01:08\u001b[39m\n", "\u001b[34m Instances 60000: 60000\u001b[39m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "The accuracy of Train Data before the training process 0.0212\n", "The cost of Train Data before the training process 2.3156\n" ] } ], "source": [ "TrainP = predict(model, X_train, Y_train);\n", "\n", "println()\n", "println(\"The accuracy of Train Data before the training process $(round(TrainP[:accuracy], digits=4))\")\n", "println(\"The cost of Train Data before the training process $(round(TrainP[:cost], digits=4))\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train the model" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 1:01:25\u001b[39m\n", "\u001b[34m Epoch 10: 10\u001b[39m\n", "\u001b[34m Instances 60000: 60000\u001b[39m\n", "\u001b[34m Train Cost: 0.2657\u001b[39m\n", "\u001b[34m Train Accuracy: 0.9039\u001b[39m\n" ] } ], "source": [ "TrainD = train(X_train, Y_train, model, 10);# testData = X_test, testLabels = Y_test);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`train` function provides an extra `kwargs` to use test Data/Labels to get the Costs and Accuracies during each training epoch. \n", "\n", "**Note** This will take extra time to do the training\n", "\n", "Instead it can be used as follows:\n", "\n", "```julia\n", "TrainD = train(X_train, Y_train, model, 10)\n", "```" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "plot(1:10, TrainD[:trainAccuracies], label=\"Training Accuracies\")\n", "plot!(1:10, TrainD[:trainCosts], label=\"Training Costs\")\n", "# plot!(1:10, TrainD[:testAccuracies], label=\"Test Accuracies\")\n", "# plot!(1:10, TrainD[:testCosts], label=\"Test Costs\")\n", "ylabel!(\"Epochs\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Predict After Training" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:01:07\u001b[39m\n", "\u001b[34m Instances 60000: 60000\u001b[39m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "The accuracy of Train Data after the training process 0.9041\n", "The cost of Train Data after the training process 0.2655\n" ] } ], "source": [ "TrainP = predict(model, X_train, Y_train);\n", "\n", "println()\n", "println(\"The accuracy of Train Data after the training process $(round(TrainP[:accuracy], digits=4))\")\n", "println(\"The cost of Train Data after the training process $(round(TrainP[:cost], digits=4))\")" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32mProgress: 100%|█████████████████████████████████████████| Time: 0:00:11\u001b[39m\n", "\u001b[34m Instances 10000: 10000\u001b[39m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "The accuracy of Test Data after the training process 0.8905\n", "The cost of Test Data after the training process 0.313\n" ] } ], "source": [ "TestP = predict(model, X_test, Y_test);\n", "\n", "println()\n", "println(\"The accuracy of Test Data after the training process $(round(TestP[:accuracy], digits=4))\")\n", "println(\"The cost of Test Data after the training process $(round(TestP[:cost], digits=4))\")" ] } ], "metadata": { "kernelspec": { "display_name": "Julia 1.4.1", "language": "julia", "name": "julia-1.4" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.4.1" } }, "nbformat": 4, "nbformat_minor": 4 }