{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Handwriting classification\n", "\n", "\"MNIST\n", "\n", "In this sample, you will create a deep neural network using Deeplearning4j and train a model capable of classifying random handwriting digits. \n", "\n", "This example use the [MNIST](https://en.wikipedia.org/wiki/MNIST_database) dataset.\n", "\n", "This sample is written in Kotlin, a Java-based language that is well suited for notebooks like this one.\n", "\n", "### What you will learn\n", "\n", "1. Load a dataset for a neural network.\n", "2. Format EMNIST for image recognition.\n", "3. Create a deep neural network.\n", "4. Train a model.\n", "5. Evaluate the performance of your model." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%use deeplearning4j-cuda(cuda=10.2)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "//number of rows and columns in the input pictures\n", "val numRows = 28\n", "val numColumns = 28\n", "val outputNum = 10 // number of output classes\n", "val batchSize = 64 // batch size for each epoch\n", "val rngSeed = 123 // random number seed for reproducibility\n", "val numEpochs = 5 // number of epochs to perform\n", "val rate = 0.0015 // learning rate" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator\n", "\n", "//Get the DataSetIterators:\n", "val mnistTrain = MnistDataSetIterator(batchSize, true, rngSeed)\n", "val mnistTest = MnistDataSetIterator(batchSize, false, rngSeed)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "=======================================================================\n", "LayerName (LayerType) nIn,nOut TotalParams ParamsShape \n", "=======================================================================\n", "layer0 (DenseLayer) 784,500 392 500 W:{784,500}, b:{1,500}\n", "layer1 (DenseLayer) 500,100 50 100 W:{500,100}, b:{1,100}\n", "layer2 (OutputLayer) 100,10 1 010 W:{100,10}, b:{1,10} \n", "-----------------------------------------------------------------------\n", " Total Parameters: 443 610\n", " Trainable Parameters: 443 610\n", " Frozen Parameters: 0\n", "=======================================================================\n", "\n" ] } ], "source": [ "import org.nd4j.linalg.learning.config.Nesterovs\n", "import org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction\n", "import org.nd4j.linalg.activations.Activation\n", "\n", "val conf = NeuralNetConfiguration.Builder()\n", " .seed(rngSeed.toLong()) //include a random seed for reproducibility\n", " // use stochastic gradient descent as an optimization algorithm\n", "\n", " .activation(Activation.RELU)\n", " .weightInit(WeightInit.XAVIER)\n", " .updater(Nesterovs(rate, 0.98)) //specify the rate of change of the learning rate.\n", " .l2(rate * 0.005) // regularize learning model\n", " .list()\n", " .layer(DenseLayer.Builder() //create the first input layer.\n", " .nIn(numRows * numColumns)\n", " .nOut(500)\n", " .build())\n", " .layer(DenseLayer.Builder() //create the second input layer\n", " .nIn(500)\n", " .nOut(100)\n", " .build())\n", " .layer(OutputLayer.Builder(LossFunction.NEGATIVELOGLIKELIHOOD) //create hidden layer\n", " .activation(Activation.SOFTMAX)\n", " .nIn(100)\n", " .nOut(outputNum)\n", " .build())\n", " .build()\n", " \n", "val model = MultiLayerNetwork(conf)\n", "model.init()\n", "println(model.summary())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Launching Deeplearning4j UI" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "import org.deeplearning4j.ui.api.UIServer\n", "import org.deeplearning4j.optimize.listeners.ScoreIterationListener\n", "import org.deeplearning4j.api.storage.StatsStorageRouter\n", "import org.deeplearning4j.api.storage.impl.RemoteUIStatsStorageRouter\n", "import org.deeplearning4j.ui.stats.StatsListener\n", "\n", "val uiServer: UIServer = UIServer.getInstance()\n", "uiServer.enableRemoteListener()\n", "//Create the remote stats storage router - this sends the results to the UI via HTTP, assuming the UI is at http://localhost:9000\n", "val remoteUIRouter: StatsStorageRouter = RemoteUIStatsStorageRouter(\"http://localhost:9000\")\n", "model.setListeners(ScoreIterationListener(100), StatsListener(remoteUIRouter))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "open a new tab in your browser and go to [http://localhost:9000](http://localhost:9000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train model" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "model.fit(mnistTrain, numEpochs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate model" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "========================Evaluation Metrics========================\n", " # of classes: 10\n", " Accuracy: 0,9718\n", " Precision: 0,9722\n", " Recall: 0,9711\n", " F1 Score: 0,9714\n", "Precision, recall & F1: macro-averaged (equally weighted avg. of 10 classes)\n", "\n", "\n", "=========================Confusion Matrix=========================\n", " 0 1 2 3 4 5 6 7 8 9\n", "---------------------------------------------------\n", " 967 0 0 1 1 0 4 2 2 3 | 0 = 0\n", " 0 1123 2 1 0 0 5 1 3 0 | 1 = 1\n", " 4 3 995 0 2 0 4 14 10 0 | 2 = 2\n", " 0 1 2 988 0 1 0 10 6 2 | 3 = 3\n", " 3 0 0 0 956 0 6 4 0 13 | 4 = 4\n", " 7 1 0 19 2 825 20 1 11 6 | 5 = 5\n", " 6 3 0 0 4 1 943 1 0 0 | 6 = 6\n", " 0 8 5 2 0 0 0 1010 1 2 | 7 = 7\n", " 5 0 2 6 4 1 8 7 937 4 | 8 = 8\n", " 3 6 1 6 10 0 2 6 1 974 | 9 = 9\n", "\n", "Confusion matrix format: Actual (rowClass) predicted as (columnClass) N times\n", "==================================================================\n" ] } ], "source": [ "val eval: org.nd4j.evaluation.classification.Evaluation = model.evaluate(mnistTest)\n", "println(eval.stats())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To stop the UI:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "uiServer.stop()" ] } ], "metadata": { "kernelspec": { "display_name": "Kotlin", "language": "kotlin", "name": "kotlin" }, "language_info": { "codemirror_mode": "text/x-kotlin", "file_extension": ".kt", "mimetype": "text/x-kotlin", "name": "kotlin", "pygments_lexer": "kotlin", "version": "1.4.0-dev-7568" } }, "nbformat": 4, "nbformat_minor": 2 }