{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "mlp-mnist.ipynb", "provenance": [], "collapsed_sections": [] }, "kernelspec": { "name": "python3", "display_name": "Python 3" } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "8J3LuF3DTEd8" }, "source": [ "# Multi-Layer Perceptron with MNIST" ] }, { "cell_type": "markdown", "metadata": { "id": "tvHZtL7ETLDp" }, "source": [ "## Pre-requisites\n", "\n", "Install the following Python packages\n", "\n", "1. cloudmesh-installer\n", "2. cloudmesh-common" ] }, { "cell_type": "code", "metadata": { "id": "IE8l7YgQ_z2p", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "2da8a3ef-6e20-4e56-d7b1-12dbe864d998" }, "source": [ "! pip3 install cloudmesh-installer\n", "! pip3 install cloudmesh-common" ], "execution_count": 1, "outputs": [ { "output_type": "stream", "text": [ "Requirement already satisfied: cloudmesh-installer in /usr/local/lib/python3.6/dist-packages (4.4.19)\n", "Requirement already satisfied: cloudmesh-common in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (4.3.42)\n", "Requirement already satisfied: pytest-cov in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (2.11.1)\n", "Requirement already satisfied: coverage in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (3.7.1)\n", "Requirement already satisfied: oyaml in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (1.0)\n", "Requirement already satisfied: humanize in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (0.5.1)\n", "Requirement already satisfied: tabulate in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (0.8.7)\n", "Requirement already satisfied: flake8 in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (3.8.4)\n", "Requirement already satisfied: ordered-set in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (4.0.2)\n", "Requirement already satisfied: docopt in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (0.6.2)\n", "Requirement already satisfied: python-hostlist in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (1.21)\n", "Requirement already satisfied: wheel in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (0.36.2)\n", "Requirement already satisfied: pip-tools in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (4.5.1)\n", "Requirement already satisfied: psutil in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (5.4.8)\n", "Requirement already satisfied: pytest in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (3.6.4)\n", "Requirement already satisfied: bump2version==1.0.0 in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (1.0.0)\n", "Requirement already satisfied: pipdeptree in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (2.0.0)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (2.23.0)\n", "Requirement already satisfied: colorama in /usr/local/lib/python3.6/dist-packages (from cloudmesh-installer) (0.4.4)\n", "Requirement already satisfied: python-dateutil in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common->cloudmesh-installer) (2.8.1)\n", "Requirement already satisfied: pathlib in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common->cloudmesh-installer) (1.0.1)\n", "Requirement already satisfied: pytz in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common->cloudmesh-installer) (2018.9)\n", "Requirement already satisfied: simplejson in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common->cloudmesh-installer) (3.17.2)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from oyaml->cloudmesh-installer) (3.13)\n", "Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from flake8->cloudmesh-installer) (0.6.1)\n", "Requirement already satisfied: pycodestyle<2.7.0,>=2.6.0a1 in /usr/local/lib/python3.6/dist-packages (from flake8->cloudmesh-installer) (2.6.0)\n", "Requirement already satisfied: pyflakes<2.3.0,>=2.2.0 in /usr/local/lib/python3.6/dist-packages (from flake8->cloudmesh-installer) (2.2.0)\n", "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from flake8->cloudmesh-installer) (3.4.0)\n", "Requirement already satisfied: click>=7 in /usr/local/lib/python3.6/dist-packages (from pip-tools->cloudmesh-installer) (7.1.2)\n", "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from pip-tools->cloudmesh-installer) (1.15.0)\n", "Requirement already satisfied: more-itertools>=4.0.0 in /usr/local/lib/python3.6/dist-packages (from pytest->cloudmesh-installer) (8.7.0)\n", "Requirement already satisfied: py>=1.5.0 in /usr/local/lib/python3.6/dist-packages (from pytest->cloudmesh-installer) (1.10.0)\n", "Requirement already satisfied: atomicwrites>=1.0 in /usr/local/lib/python3.6/dist-packages (from pytest->cloudmesh-installer) (1.4.0)\n", "Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.6/dist-packages (from pytest->cloudmesh-installer) (20.3.0)\n", "Requirement already satisfied: pluggy<0.8,>=0.5 in /usr/local/lib/python3.6/dist-packages (from pytest->cloudmesh-installer) (0.7.1)\n", "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from pytest->cloudmesh-installer) (53.0.0)\n", "Requirement already satisfied: pip>=6.0.0 in /usr/local/lib/python3.6/dist-packages (from pipdeptree->cloudmesh-installer) (19.3.1)\n", "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->cloudmesh-installer) (1.24.3)\n", "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->cloudmesh-installer) (3.0.4)\n", "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->cloudmesh-installer) (2.10)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->cloudmesh-installer) (2020.12.5)\n", "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->flake8->cloudmesh-installer) (3.4.0)\n", "Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->flake8->cloudmesh-installer) (3.7.4.3)\n", "Requirement already satisfied: cloudmesh-common in /usr/local/lib/python3.6/dist-packages (4.3.42)\n", "Requirement already satisfied: python-dateutil in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (2.8.1)\n", "Requirement already satisfied: python-hostlist in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (1.21)\n", "Requirement already satisfied: pathlib in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (1.0.1)\n", "Requirement already satisfied: humanize in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (0.5.1)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (2.23.0)\n", "Requirement already satisfied: pytz in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (2018.9)\n", "Requirement already satisfied: oyaml in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (1.0)\n", "Requirement already satisfied: colorama in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (0.4.4)\n", "Requirement already satisfied: psutil in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (5.4.8)\n", "Requirement already satisfied: simplejson in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (3.17.2)\n", "Requirement already satisfied: tabulate in /usr/local/lib/python3.6/dist-packages (from cloudmesh-common) (0.8.7)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil->cloudmesh-common) (1.15.0)\n", "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->cloudmesh-common) (1.24.3)\n", "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->cloudmesh-common) (2.10)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->cloudmesh-common) (2020.12.5)\n", "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->cloudmesh-common) (3.0.4)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from oyaml->cloudmesh-common) (3.13)\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "XYe7k1hvTYB7" }, "source": [ "## Sample MLP with Tensorflow Keras" ] }, { "cell_type": "code", "metadata": { "id": "eu-7aFi0_u10", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "31276960-8d54-4890-b5c0-83e07ca5aca3" }, "source": [ "from __future__ import absolute_import\n", "from __future__ import division\n", "from __future__ import print_function\n", "\n", "import time \n", "\n", "import numpy as np\n", "from keras.models import Sequential\n", "from keras.layers import Dense, Activation, Dropout\n", "from keras.utils import to_categorical, plot_model\n", "from keras.datasets import mnist\n", "#import pydotplus\n", "from keras.utils.vis_utils import model_to_dot\n", "#from keras.utils.vis_utils import pydot\n", "\n", "\n", "from cloudmesh.common.StopWatch import StopWatch\n", "\n", "StopWatch.start(\"data-load\")\n", "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n", "StopWatch.stop(\"data-load\")\n", "\n", "num_labels = len(np.unique(y_train))\n", "\n", "\n", "y_train = to_categorical(y_train)\n", "y_test = to_categorical(y_test)\n", "\n", "\n", "image_size = x_train.shape[1]\n", "input_size = image_size * image_size\n", "\n", "\n", "x_train = np.reshape(x_train, [-1, input_size])\n", "x_train = x_train.astype('float32') / 255\n", "x_test = np.reshape(x_test, [-1, input_size])\n", "x_test = x_test.astype('float32') / 255\n", "\n", "batch_size = 128\n", "hidden_units = 512\n", "dropout = 0.45\n", "\n", "model = Sequential()\n", "model.add(Dense(hidden_units, input_dim=input_size))\n", "model.add(Activation('relu'))\n", "model.add(Dropout(dropout))\n", "model.add(Dense(hidden_units))\n", "model.add(Activation('relu'))\n", "model.add(Dropout(dropout))\n", "model.add(Dense(hidden_units))\n", "model.add(Activation('relu'))\n", "model.add(Dropout(dropout))\n", "model.add(Dense(hidden_units))\n", "model.add(Activation('relu'))\n", "model.add(Dropout(dropout))\n", "model.add(Dense(num_labels))\n", "model.add(Activation('softmax'))\n", "model.summary()\n", "plot_model(model, to_file='mlp-mnist.png', show_shapes=True)\n", "\n", "StopWatch.start(\"compile\")\n", "model.compile(loss='categorical_crossentropy',\n", " optimizer='adam',\n", " metrics=['accuracy'])\n", "StopWatch.stop(\"compile\")\n", "StopWatch.start(\"train\")\n", "model.fit(x_train, y_train, epochs=5, batch_size=batch_size)\n", "StopWatch.stop(\"train\")\n", "\n", "StopWatch.start(\"test\")\n", "loss, acc = model.evaluate(x_test, y_test, batch_size=batch_size)\n", "print(\"\\nTest accuracy: %.1f%%\" % (100.0 * acc))\n", "StopWatch.stop(\"test\")\n", "\n", "StopWatch.benchmark()\n", "\n" ], "execution_count": 2, "outputs": [ { "output_type": "stream", "text": [ "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n", "11493376/11490434 [==============================] - 0s 0us/step\n", "Model: \"sequential\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense (Dense) (None, 512) 401920 \n", "_________________________________________________________________\n", "activation (Activation) (None, 512) 0 \n", "_________________________________________________________________\n", "dropout (Dropout) (None, 512) 0 \n", "_________________________________________________________________\n", "dense_1 (Dense) (None, 512) 262656 \n", "_________________________________________________________________\n", "activation_1 (Activation) (None, 512) 0 \n", "_________________________________________________________________\n", "dropout_1 (Dropout) (None, 512) 0 \n", "_________________________________________________________________\n", "dense_2 (Dense) (None, 512) 262656 \n", "_________________________________________________________________\n", "activation_2 (Activation) (None, 512) 0 \n", "_________________________________________________________________\n", "dropout_2 (Dropout) (None, 512) 0 \n", "_________________________________________________________________\n", "dense_3 (Dense) (None, 512) 262656 \n", "_________________________________________________________________\n", "activation_3 (Activation) (None, 512) 0 \n", "_________________________________________________________________\n", "dropout_3 (Dropout) (None, 512) 0 \n", "_________________________________________________________________\n", "dense_4 (Dense) (None, 10) 5130 \n", "_________________________________________________________________\n", "activation_4 (Activation) (None, 10) 0 \n", "=================================================================\n", "Total params: 1,195,018\n", "Trainable params: 1,195,018\n", "Non-trainable params: 0\n", "_________________________________________________________________\n", "Epoch 1/5\n", "469/469 [==============================] - 14s 29ms/step - loss: 0.7886 - accuracy: 0.7334\n", "Epoch 2/5\n", "469/469 [==============================] - 14s 29ms/step - loss: 0.1981 - accuracy: 0.9433\n", "Epoch 3/5\n", "469/469 [==============================] - 14s 29ms/step - loss: 0.1546 - accuracy: 0.9572\n", "Epoch 4/5\n", "469/469 [==============================] - 14s 29ms/step - loss: 0.1302 - accuracy: 0.9641\n", "Epoch 5/5\n", "469/469 [==============================] - 14s 29ms/step - loss: 0.1168 - accuracy: 0.9663\n", "79/79 [==============================] - 1s 9ms/step - loss: 0.0785 - accuracy: 0.9765\n", "\n", "Test accuracy: 97.6%\n", "\n", "+---------------------+------------------------------------------------------------------+\n", "| Attribute | Value |\n", "|---------------------+------------------------------------------------------------------|\n", "| BUG_REPORT_URL | \"https://bugs.launchpad.net/ubuntu/\" |\n", "| DISTRIB_CODENAME | bionic |\n", "| DISTRIB_DESCRIPTION | \"Ubuntu 18.04.5 LTS\" |\n", "| DISTRIB_ID | Ubuntu |\n", "| DISTRIB_RELEASE | 18.04 |\n", "| HOME_URL | \"https://www.ubuntu.com/\" |\n", "| ID | ubuntu |\n", "| ID_LIKE | debian |\n", "| NAME | \"Ubuntu\" |\n", "| PRETTY_NAME | \"Ubuntu 18.04.5 LTS\" |\n", "| PRIVACY_POLICY_URL | \"https://www.ubuntu.com/legal/terms-and-policies/privacy-policy\" |\n", "| SUPPORT_URL | \"https://help.ubuntu.com/\" |\n", "| UBUNTU_CODENAME | bionic |\n", "| VERSION | \"18.04.5 LTS (Bionic Beaver)\" |\n", "| VERSION_CODENAME | bionic |\n", "| VERSION_ID | \"18.04\" |\n", "| cpu_count | 2 |\n", "| mem.active | 1.2 GiB |\n", "| mem.available | 11.6 GiB |\n", "| mem.free | 9.8 GiB |\n", "| mem.inactive | 1.4 GiB |\n", "| mem.percent | 8.4 % |\n", "| mem.total | 12.7 GiB |\n", "| mem.used | 913.7 MiB |\n", "| platform.version | #1 SMP Thu Jul 23 08:00:38 PDT 2020 |\n", "| python | 3.6.9 (default, Oct 8 2020, 12:12:24) |\n", "| | [GCC 8.4.0] |\n", "| python.pip | 19.3.1 |\n", "| python.version | 3.6.9 |\n", "| sys.platform | linux |\n", "| uname.machine | x86_64 |\n", "| uname.node | 6609095905d1 |\n", "| uname.processor | x86_64 |\n", "| uname.release | 4.19.112+ |\n", "| uname.system | Linux |\n", "| uname.version | #1 SMP Thu Jul 23 08:00:38 PDT 2020 |\n", "| user | collab |\n", "+---------------------+------------------------------------------------------------------+\n", "\n", "+-----------+----------+--------+--------+---------------------+-------+--------------+--------+-------+-------------------------------------+\n", "| Name | Status | Time | Sum | Start | tag | Node | User | OS | Version |\n", "|-----------+----------+--------+--------+---------------------+-------+--------------+--------+-------+-------------------------------------|\n", "| data-load | failed | 0.549 | 0.549 | 2021-02-15 15:24:00 | | 6609095905d1 | collab | Linux | #1 SMP Thu Jul 23 08:00:38 PDT 2020 |\n", "| compile | failed | 0.023 | 0.023 | 2021-02-15 15:24:01 | | 6609095905d1 | collab | Linux | #1 SMP Thu Jul 23 08:00:38 PDT 2020 |\n", "| train | failed | 69.1 | 69.1 | 2021-02-15 15:24:01 | | 6609095905d1 | collab | Linux | #1 SMP Thu Jul 23 08:00:38 PDT 2020 |\n", "| test | failed | 0.907 | 0.907 | 2021-02-15 15:25:10 | | 6609095905d1 | collab | Linux | #1 SMP Thu Jul 23 08:00:38 PDT 2020 |\n", "+-----------+----------+--------+--------+---------------------+-------+--------------+--------+-------+-------------------------------------+\n", "\n", "# csv,timer,status,time,sum,start,tag,uname.node,user,uname.system,platform.version\n", "# csv,data-load,failed,0.549,0.549,2021-02-15 15:24:00,,6609095905d1,collab,Linux,#1 SMP Thu Jul 23 08:00:38 PDT 2020\n", "# csv,compile,failed,0.023,0.023,2021-02-15 15:24:01,,6609095905d1,collab,Linux,#1 SMP Thu Jul 23 08:00:38 PDT 2020\n", "# csv,train,failed,69.1,69.1,2021-02-15 15:24:01,,6609095905d1,collab,Linux,#1 SMP Thu Jul 23 08:00:38 PDT 2020\n", "# csv,test,failed,0.907,0.907,2021-02-15 15:25:10,,6609095905d1,collab,Linux,#1 SMP Thu Jul 23 08:00:38 PDT 2020\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "rkD7IrsbCRIJ" }, "source": [ "# REFERENCES\n", "\n", "1. [Adavnced Keras Deep Learning Book](https://github.com/PacktPublishing/Advanced-Deep-Learning-with-Keras)" ] } ] }