{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "TensorFlow 2.3 on Python 3.6 (CUDA 10.1)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.9"
    },
    "colab": {
      "name": "10-1.lenet_in_keras.ipynb",
      "provenance": []
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UV3_mVndP6Hi"
      },
      "source": [
        "# 케라스로 심층 합성곱 신경망 만들기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H2icE8zkP6Hm"
      },
      "source": [
        "이 노트북에서 [LeNet-5](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf)과 비슷한 MNIST 손글씨 숫자를 분류하는 심층 합성곱 신경망을 만듭니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G74ktsMRP6Hn"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/rickiepark/dl-illustrated/blob/master/notebooks/10-1.lenet_in_keras.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ivEeMRHrP6Hn"
      },
      "source": [
        "#### 라이브러를 적재합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ma9EMkCAP6Hn"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.datasets import mnist\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, Dropout\n",
        "from tensorflow.keras.layers import Flatten, Conv2D, MaxPooling2D # new!"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SsTnIg6aP6Ho"
      },
      "source": [
        "#### 데이터를 적재합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NRkMN0KJP6Ho",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3131be58-b22f-474b-cf70-d77f5342906b"
      },
      "source": [
        "(X_train, y_train), (X_valid, y_valid) = mnist.load_data()"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11490434/11490434 [==============================] - 0s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hkf9icAJP6Hp"
      },
      "source": [
        "#### 데이터를 전처리합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S_aWRMslP6Hp"
      },
      "source": [
        "X_train = X_train.reshape(60000, 28, 28, 1).astype('float32')\n",
        "X_valid = X_valid.reshape(10000, 28, 28, 1).astype('float32')"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "se51PfthP6Hp"
      },
      "source": [
        "X_train /= 255\n",
        "X_valid /= 255"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KUE_VeFiP6Hp"
      },
      "source": [
        "n_classes = 10\n",
        "y_train = keras.utils.to_categorical(y_train, n_classes)\n",
        "y_valid = keras.utils.to_categorical(y_valid, n_classes)"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5aE2huiNP6Hp"
      },
      "source": [
        "#### 신경망 모델을 만듭니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ozZ3asHxP6Hq"
      },
      "source": [
        "model = Sequential()\n",
        "\n",
        "model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(28, 28, 1)))\n",
        "\n",
        "model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))\n",
        "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
        "model.add(Dropout(0.25))\n",
        "model.add(Flatten())\n",
        "\n",
        "model.add(Dense(128, activation='relu'))\n",
        "model.add(Dropout(0.5))\n",
        "\n",
        "model.add(Dense(n_classes, activation='softmax'))"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A3JA2ahzP6Hq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "da2032c1-0cc9-4713-80f3-87bdf3c4fa41"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " conv2d (Conv2D)             (None, 26, 26, 32)        320       \n",
            "                                                                 \n",
            " conv2d_1 (Conv2D)           (None, 24, 24, 64)        18496     \n",
            "                                                                 \n",
            " max_pooling2d (MaxPooling2D  (None, 12, 12, 64)       0         \n",
            " )                                                               \n",
            "                                                                 \n",
            " dropout (Dropout)           (None, 12, 12, 64)        0         \n",
            "                                                                 \n",
            " flatten (Flatten)           (None, 9216)              0         \n",
            "                                                                 \n",
            " dense (Dense)               (None, 128)               1179776   \n",
            "                                                                 \n",
            " dropout_1 (Dropout)         (None, 128)               0         \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 10)                1290      \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 1,199,882\n",
            "Trainable params: 1,199,882\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RqMEQz_rP6Hq"
      },
      "source": [
        "#### 모델을 설정합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xevWEuNDP6Hq"
      },
      "source": [
        "model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sa8I9xhCP6Hq"
      },
      "source": [
        "#### 훈련!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bsLCVNMPP6Hq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "65a79490-6092-4ef6-f3c0-c448c92d19be"
      },
      "source": [
        "model.fit(X_train, y_train, batch_size=128, epochs=10, verbose=1, validation_data=(X_valid, y_valid))"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "469/469 [==============================] - 15s 11ms/step - loss: 0.2413 - accuracy: 0.9275 - val_loss: 0.0489 - val_accuracy: 0.9849\n",
            "Epoch 2/10\n",
            "469/469 [==============================] - 5s 10ms/step - loss: 0.0879 - accuracy: 0.9739 - val_loss: 0.0349 - val_accuracy: 0.9869\n",
            "Epoch 3/10\n",
            "469/469 [==============================] - 4s 9ms/step - loss: 0.0648 - accuracy: 0.9804 - val_loss: 0.0334 - val_accuracy: 0.9889\n",
            "Epoch 4/10\n",
            "469/469 [==============================] - 4s 9ms/step - loss: 0.0548 - accuracy: 0.9836 - val_loss: 0.0331 - val_accuracy: 0.9901\n",
            "Epoch 5/10\n",
            "469/469 [==============================] - 4s 9ms/step - loss: 0.0445 - accuracy: 0.9861 - val_loss: 0.0292 - val_accuracy: 0.9899\n",
            "Epoch 6/10\n",
            "469/469 [==============================] - 4s 9ms/step - loss: 0.0383 - accuracy: 0.9886 - val_loss: 0.0246 - val_accuracy: 0.9920\n",
            "Epoch 7/10\n",
            "469/469 [==============================] - 4s 9ms/step - loss: 0.0330 - accuracy: 0.9894 - val_loss: 0.0279 - val_accuracy: 0.9906\n",
            "Epoch 8/10\n",
            "469/469 [==============================] - 4s 9ms/step - loss: 0.0313 - accuracy: 0.9900 - val_loss: 0.0270 - val_accuracy: 0.9918\n",
            "Epoch 9/10\n",
            "469/469 [==============================] - 4s 9ms/step - loss: 0.0285 - accuracy: 0.9902 - val_loss: 0.0267 - val_accuracy: 0.9923\n",
            "Epoch 10/10\n",
            "469/469 [==============================] - 4s 9ms/step - loss: 0.0258 - accuracy: 0.9915 - val_loss: 0.0272 - val_accuracy: 0.9916\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f89d037f7c0>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    }
  ]
}