{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "TensorFlow 2.3 on Python 3.6 (CUDA 10.1)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.9"
    },
    "colab": {
      "name": "8-4.intermediate_net_in_keras.ipynb",
      "provenance": []
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nCGY1Tt9yXEX"
      },
      "source": [
        "# 케라스로 중간 깊이 신경망 만들기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-_CQynIXyXEd"
      },
      "source": [
        "이 노트북에서 지금까지 다룬 이론을 적용해 [얕은 신경망](https://github.com/the-deep-learners/deep-learning-illustrated/blob/master/notebooks/5-1.shallow_net_in_keras.ipynb)을 발전시켜 보겠습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dx9A9ov2yXEd"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rickiepark/dl-illustrated/blob/master/notebooks/8-4.intermediate_net_in_keras.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6qfrQhM2yXEe"
      },
      "source": [
        "#### 라이브러리를 적재합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0soySK3DyXEe"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.datasets import mnist\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "from tensorflow.keras.optimizers import SGD"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4csC82MayXEf"
      },
      "source": [
        "#### 데이터를 적재합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0nayoW6VyXEf"
      },
      "source": [
        "(X_train, y_train), (X_valid, y_valid) = mnist.load_data()"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FfrvC7U0yXEf"
      },
      "source": [
        "#### 데이터를 전처리합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QUJaYClIyXEf"
      },
      "source": [
        "X_train = X_train.reshape(60000, 784).astype('float32')\n",
        "X_valid = X_valid.reshape(10000, 784).astype('float32')"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xZs4S5HzyXEg"
      },
      "source": [
        "X_train /= 255\n",
        "X_valid /= 255"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6mX4MhlRyXEg"
      },
      "source": [
        "n_classes = 10\n",
        "y_train = keras.utils.to_categorical(y_train, n_classes)\n",
        "y_valid = keras.utils.to_categorical(y_valid, n_classes)"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "un3qPpxuyXEg"
      },
      "source": [
        "#### 신경망을 만듭니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eU5DiMF8yXEg"
      },
      "source": [
        "model = Sequential()\n",
        "model.add(Dense(64, activation='relu', input_shape=(784,)))\n",
        "model.add(Dense(64, activation='relu'))\n",
        "model.add(Dense(10, activation='softmax'))"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s1XIoz5xyXEg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "38d8932c-8c31-4a42-f9a0-f31f6120475d"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense (Dense)               (None, 64)                50240     \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 64)                4160      \n",
            "                                                                 \n",
            " dense_2 (Dense)             (None, 10)                650       \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 55,050\n",
            "Trainable params: 55,050\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aecR4zeOyXEh"
      },
      "source": [
        "#### 모델을 컴파일합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U6hvCeQyyXEi"
      },
      "source": [
        "model.compile(loss='categorical_crossentropy', optimizer=SGD(learning_rate=0.1), metrics=['accuracy'])"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w0LMR5llyXEi"
      },
      "source": [
        "#### 훈련!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YXaISMyWyXEi",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fb7bc82f-2146-454e-8bcb-316a933df039"
      },
      "source": [
        "model.fit(X_train, y_train, batch_size=128, epochs=20, verbose=1, validation_data=(X_valid, y_valid))"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "469/469 [==============================] - 4s 6ms/step - loss: 0.4607 - accuracy: 0.8681 - val_loss: 0.2505 - val_accuracy: 0.9265\n",
            "Epoch 2/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.2274 - accuracy: 0.9339 - val_loss: 0.1906 - val_accuracy: 0.9416\n",
            "Epoch 3/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.1746 - accuracy: 0.9490 - val_loss: 0.1538 - val_accuracy: 0.9526\n",
            "Epoch 4/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.1428 - accuracy: 0.9583 - val_loss: 0.1344 - val_accuracy: 0.9597\n",
            "Epoch 5/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.1219 - accuracy: 0.9642 - val_loss: 0.1282 - val_accuracy: 0.9619\n",
            "Epoch 6/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.1068 - accuracy: 0.9688 - val_loss: 0.1123 - val_accuracy: 0.9646\n",
            "Epoch 7/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0936 - accuracy: 0.9727 - val_loss: 0.1105 - val_accuracy: 0.9664\n",
            "Epoch 8/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0844 - accuracy: 0.9750 - val_loss: 0.1003 - val_accuracy: 0.9691\n",
            "Epoch 9/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0768 - accuracy: 0.9772 - val_loss: 0.1039 - val_accuracy: 0.9666\n",
            "Epoch 10/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0682 - accuracy: 0.9796 - val_loss: 0.0923 - val_accuracy: 0.9712\n",
            "Epoch 11/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0626 - accuracy: 0.9817 - val_loss: 0.0999 - val_accuracy: 0.9681\n",
            "Epoch 12/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0582 - accuracy: 0.9827 - val_loss: 0.0853 - val_accuracy: 0.9732\n",
            "Epoch 13/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0528 - accuracy: 0.9844 - val_loss: 0.0823 - val_accuracy: 0.9732\n",
            "Epoch 14/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0487 - accuracy: 0.9861 - val_loss: 0.0800 - val_accuracy: 0.9753\n",
            "Epoch 15/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0448 - accuracy: 0.9867 - val_loss: 0.0826 - val_accuracy: 0.9744\n",
            "Epoch 16/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0421 - accuracy: 0.9877 - val_loss: 0.0844 - val_accuracy: 0.9734\n",
            "Epoch 17/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0383 - accuracy: 0.9890 - val_loss: 0.0802 - val_accuracy: 0.9754\n",
            "Epoch 18/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0352 - accuracy: 0.9904 - val_loss: 0.0855 - val_accuracy: 0.9741\n",
            "Epoch 19/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0332 - accuracy: 0.9908 - val_loss: 0.0782 - val_accuracy: 0.9758\n",
            "Epoch 20/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0305 - accuracy: 0.9914 - val_loss: 0.0825 - val_accuracy: 0.9755\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f8860051910>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    }
  ]
}