{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "TensorFlow 2.3 on Python 3.6 (CUDA 10.1)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.9"
    },
    "colab": {
      "name": "custom_model_in_keras.ipynb",
      "provenance": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TuVJZkooU5x3"
      },
      "source": [
        "# 케라스 API를 사용한 사용자 정의 모델 만들기 with 텐서플로 2.3+2.4\n",
        "\n",
        "DLD(Daejeon Learning Day) 2020을 위해 작성된 노트북입니다.\n",
        "\n",
        "* 깃허브 주소: https://github.com/rickiepark/handson-ml2/blob/master/custom_model_in_keras.ipynb\n",
        "* 코랩 주소: https://colab.research.google.com/github/rickiepark/handson-ml2/blob/master/custom_model_in_keras.ipynb"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i01nJ_jCU5x5",
        "outputId": "73f2a283-88e4-48b9-9432-b799e8829c15",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "tf.__version__"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.6.0'"
            ]
          },
          "metadata": {},
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T14-oCyRU5x6"
      },
      "source": [
        "### MNIST 손글씨 숫자 데이터 적재"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JOiyYV5vU5x7",
        "outputId": "de65a0cb-3c0a-409b-e5d5-fcfdda280549",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
        "\n",
        "X_train = X_train.reshape(-1, 784) / 255."
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11493376/11490434 [==============================] - 0s 0us/step\n",
            "11501568/11490434 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XlHCNho_U5x7",
        "outputId": "772f3056-75aa-4424-9f23-8416ccfb7ec9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "X_train.shape"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(60000, 784)"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "skabK6kOU5x8"
      },
      "source": [
        "### `Sequential()` 클래스와 함수형 API의 관계"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "thdjt3qKU5x8"
      },
      "source": [
        "`Sequential()`:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "odseYIpgU5x9"
      },
      "source": [
        "시퀀셜 모델에 10개의 유닛을 가진 완전 연결 층을 추가합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dde_dt0VU5x9",
        "outputId": "4872c912-a03a-4453-96dc-ad37cf6033a8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "seq_model = tf.keras.Sequential()\n",
        "\n",
        "seq_model.add(tf.keras.layers.Dense(units=10, \n",
        "                                    activation='softmax',\n",
        "                                    input_shape=(784,)))\n",
        "\n",
        "seq_model.summary()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 10)                7850      \n",
            "=================================================================\n",
            "Total params: 7,850\n",
            "Trainable params: 7,850\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UEO2GA4lU5x-",
        "outputId": "61a45639-1294-476a-b375-c87746ce366d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "seq_model.compile(loss='sparse_categorical_crossentropy',\n",
        "                  metrics=['accuracy'])\n",
        "seq_model.fit(X_train, y_train, batch_size=32, epochs=2)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/2\n",
            "1875/1875 [==============================] - 4s 2ms/step - loss: 0.4404 - accuracy: 0.8830\n",
            "Epoch 2/2\n",
            "1875/1875 [==============================] - 3s 2ms/step - loss: 0.3028 - accuracy: 0.9154\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fa70b327a90>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yQtlaqq7U5x-"
      },
      "source": [
        "### 함수형 API:\n",
        "\n",
        "함수형 API를 사용할 때는 `Input()`을 사용해 입력의 크기를 정의해야 합니다. 하지만 `InputLayer` 층이 추가되어 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sK9N1D7gU5x-",
        "outputId": "45fde0e4-4d8a-456e-8037-716f16c22553",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "inputs = tf.keras.layers.Input(784)\n",
        "\n",
        "outputs = tf.keras.layers.Dense(units=10,\n",
        "                                activation='softmax')(inputs)  # __call()__ 메서드 호출\n",
        "# dense = tf.keras.layers.Dense(units=10, activation='softmax')\n",
        "# outputs = dense(inputs)\n",
        "\n",
        "func_model = tf.keras.Model(inputs, outputs)\n",
        "\n",
        "func_model.summary()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_1 (InputLayer)         [(None, 784)]             0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 10)                7850      \n",
            "=================================================================\n",
            "Total params: 7,850\n",
            "Trainable params: 7,850\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-eSCyp0eU5x_",
        "outputId": "47b25432-29c6-4992-df41-3a4ee28c0310",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "func_model.compile(loss='sparse_categorical_crossentropy',\n",
        "                  metrics=['accuracy'])\n",
        "func_model.fit(X_train, y_train, batch_size=32, epochs=2)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/2\n",
            "1875/1875 [==============================] - 3s 1ms/step - loss: 0.4453 - accuracy: 0.8799\n",
            "Epoch 2/2\n",
            "1875/1875 [==============================] - 3s 2ms/step - loss: 0.3022 - accuracy: 0.9156\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fa70b2b8590>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DxaYANyKU5x_"
      },
      "source": [
        "`Input`의 정체는 무엇일까요? 이 함수는 `InputLayer` 클래스의 객체를 만들어 그 결과를 반환합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LwRzr7ryU5x_",
        "outputId": "62eef073-1f85-402a-ca98-9b421f976e0d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "type(tf.keras.layers.Input)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "function"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IJJtKIhLU5yA"
      },
      "source": [
        "사실 신경망의 입력층은 입력 그 자체입니다. `InputLayer` 객체의 입력 노드 출력을 그대로 `Dense` 층에 주입할 수 있습니다. 모든 층은 입력과 출력 노드를 정의합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "5GDNPCE-U5yA",
        "outputId": "14dce074-e4ca-4f2b-d8df-f815343452b6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "# inputs = tf.keras.layers.Input(784)\n",
        "\n",
        "input_layer = tf.keras.layers.InputLayer(784)\n",
        "inputs = input_layer._inbound_nodes[0].outputs\n",
        "\n",
        "outputs = tf.keras.layers.Dense(units=10,\n",
        "                                activation='softmax')(inputs)\n",
        "\n",
        "input_layer_model = tf.keras.Model(inputs, outputs)\n",
        "\n",
        "input_layer_model.summary()"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_2 (InputLayer)         [(None, 784)]             0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 10)                7850      \n",
            "=================================================================\n",
            "Total params: 7,850\n",
            "Trainable params: 7,850\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "MsuY6-nHU5yA",
        "outputId": "78f71d4a-a340-44e7-f3ab-c4fcaf4302ef",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "input_layer_model.compile(loss='sparse_categorical_crossentropy', \n",
        "                          metrics=['accuracy'])\n",
        "input_layer_model.fit(X_train, y_train, batch_size=32, epochs=2)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/2\n",
            "1875/1875 [==============================] - 3s 1ms/step - loss: 0.4364 - accuracy: 0.8832\n",
            "Epoch 2/2\n",
            "1875/1875 [==============================] - 3s 1ms/step - loss: 0.3020 - accuracy: 0.9161\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fa70b2b5f50>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tTux1i7bU5yA"
      },
      "source": [
        "함수형 API를 사용한 모델은 `layers` 속성에 `InputLayer` 클래스를 포함합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6gHU1vi-U5yB",
        "outputId": "77a4bfd7-ffea-4a66-d053-bd685654691d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "func_model.layers"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<keras.engine.input_layer.InputLayer at 0x7fa709a39410>,\n",
              " <keras.layers.core.Dense at 0x7fa70997ee90>]"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cItMzPQ2U5yB"
      },
      "source": [
        "하지만 시퀀셜 모델은 `layers` 속성에 `InputLayer` 클래스가 보이지 않습니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LbJOms9GU5yB",
        "outputId": "74a61016-37b4-43d6-9a69-aba423ad7682",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "seq_model.layers"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<keras.layers.core.Dense at 0x7fa70f861310>]"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w9M8YtyOU5yB"
      },
      "source": [
        "모델은 감춰진 `_self_tracked_trackables` 속성이 또 있습니다. 여기에서 `InputLayer` 클래스를 확인할 수 있습니다(텐서플로 2.5 이전 버전에서는 `_layers` 속성을 사용합니다)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-1YPNEUyVNMB",
        "outputId": "1f88f3bf-d4f4-405a-fcbb-ccbb539939cb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "seq_model._self_tracked_trackables"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<keras.engine.input_layer.InputLayer at 0x7fa70e41a6d0>,\n",
              " <keras.layers.core.Dense at 0x7fa70f861310>]"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aicQp3fkU5yB"
      },
      "source": [
        "또는 `_input_layers` 속성에서도 확인할 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-W8jyeoMU5yB",
        "outputId": "9dc3b4e5-fe48-45d8-a9dc-186c5cd9f1c6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "seq_model._input_layers, func_model._input_layers"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "([<keras.engine.input_layer.InputLayer at 0x7fa70e41a6d0>],\n",
              " [<keras.engine.input_layer.InputLayer at 0x7fa709a39410>])"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PqRMSkfsU5yC",
        "outputId": "8b56322b-25a9-4619-8c70-0fac2cbc2b1e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "seq_model._output_layers, func_model._output_layers"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "([<keras.layers.core.Dense at 0x7fa70f861310>],\n",
              " [<keras.layers.core.Dense at 0x7fa70997ee90>])"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6-I50i3xU5yC"
      },
      "source": [
        "`Model` 클래스로 만든 `func_model`은 사실 `Functional` 클래스의 객체입니다. `Model` 클래스는 서브클래싱에 사용합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Eia9vw5RU5yC",
        "outputId": "37c38096-3b68-456b-c1f1-e3cc1e0299d3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "func_model.__class__"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "keras.engine.functional.Functional"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iV1mrKLwU5yC"
      },
      "source": [
        "시퀀셜 모델은 함수형 모델의 특별한 경우입니다. (`Model` --> `Functional` --> `Sequential`)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lI4sh1V1U5yC"
      },
      "source": [
        "### 사용자 정의 층 만들기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gmjr-bOhU5yC"
      },
      "source": [
        "`tf.layers.Layer` 클래스를 상속하고 `build()` 메서드에서 가중치를 만든다음 `call()` 메서드에서 연산을 구현합니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XD-Fqp-3U5yC"
      },
      "source": [
        "class MyDense(tf.keras.layers.Layer):\n",
        "    \n",
        "    def __init__(self, units, activation=None, **kwargs):\n",
        "        # units와 activation 매개변수 외에 나머지 변수를 부모 클래스의 생성자로 전달합니다.\n",
        "        super(MyDense, self).__init__(**kwargs)\n",
        "        self.units = units\n",
        "        # 문자열로 미리 정의된 활성화 함수를 선택합니다. e.g., 'softmax', 'relu'\n",
        "        self.activation = tf.keras.activations.get(activation)\n",
        "        \n",
        "    def build(self, input_shape):\n",
        "        # __call__() 메서드를 호출할 때 호출됩니다. 가중치 생성을 지연합니다.\n",
        "        # 가중치와 절편을 생성합니다.\n",
        "        self.kernel = self.add_weight(name='kernel', \n",
        "                                      shape=[input_shape[-1], self.units],\n",
        "                                      initializer='glorot_uniform'   # 케라스의 기본 초기화\n",
        "                                     )\n",
        "        self.bias = self.add_weight(name='bias',\n",
        "                                    shape=[self.units],\n",
        "                                    initializer='zeros')\n",
        "    \n",
        "    def call(self, inputs):  # training=None은 training은 배치 정규화나 드롭아웃 같은 경우 사용\n",
        "        # __call__() 메서드를 호출할 때 호출됩니다.\n",
        "        # 실제 연산을 수행합니다. [batch_size, units]\n",
        "        z = tf.matmul(inputs, self.kernel) + self.bias\n",
        "        if self.activation:\n",
        "            return self.activation(z)\n",
        "        return z"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "Ba4d43vKU5yD",
        "outputId": "3514811d-2a0c-411d-a0bb-35d40a905b71",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "inputs = tf.keras.layers.Input(784)\n",
        "# Layer.__call__() --> MyDense().build() --> Layer.build() --> MyDense().call()\n",
        "outputs = MyDense(units=10, activation='softmax')(inputs)\n",
        "\n",
        "my_dense_model = tf.keras.Model(inputs, outputs)\n",
        "\n",
        "my_dense_model.summary()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_3 (InputLayer)         [(None, 784)]             0         \n",
            "_________________________________________________________________\n",
            "my_dense (MyDense)           (None, 10)                7850      \n",
            "=================================================================\n",
            "Total params: 7,850\n",
            "Trainable params: 7,850\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_DYUeNqeU5yD",
        "outputId": "53f266f1-82c9-4857-8836-321113026955",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "my_dense_model.compile(loss='sparse_categorical_crossentropy', \n",
        "                       metrics=['accuracy'])\n",
        "my_dense_model.fit(X_train, y_train, batch_size=32, epochs=2)"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/2\n",
            "1875/1875 [==============================] - 3s 2ms/step - loss: 0.4409 - accuracy: 0.8838\n",
            "Epoch 2/2\n",
            "1875/1875 [==============================] - 3s 1ms/step - loss: 0.3026 - accuracy: 0.9166\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fa70aeb0c90>"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xwYkLuDPU5yD"
      },
      "source": [
        "### 사용자 정의 모델 만들기"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Byaow6qdU5yD"
      },
      "source": [
        "# fit(), compile(), predict(), evaluate() 등의 메서드 제공\n",
        "class MyModel(tf.keras.Model):\n",
        "    \n",
        "    def __init__(self):\n",
        "        super(MyModel, self).__init__()\n",
        "        self.output_layer = MyDense(units=10, activation='softmax')\n",
        "    \n",
        "    def call(self, inputs):\n",
        "        return self.output_layer(inputs)"
      ],
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "7dvJVGZvU5yD",
        "outputId": "155141a0-a66e-4b08-bfd5-0b71ec34e692",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "my_model = MyModel()\n",
        "\n",
        "my_model.compile(loss='sparse_categorical_crossentropy', \n",
        "                       metrics=['accuracy'])\n",
        "my_model.fit(X_train, y_train, batch_size=32, epochs=2)"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/2\n",
            "1875/1875 [==============================] - 3s 2ms/step - loss: 0.4376 - accuracy: 0.8830\n",
            "Epoch 2/2\n",
            "1875/1875 [==============================] - 3s 1ms/step - loss: 0.3024 - accuracy: 0.9157\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fa70adf0b50>"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8o-Wrt9WU5yD"
      },
      "source": [
        "### 사용자 정의 훈련"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VacWTtsnU5yE"
      },
      "source": [
        "class MyCustomStep(MyModel):\n",
        "    \n",
        "    def train_step(self, data):\n",
        "        # fit()에서 전달된 데이터\n",
        "        x, y = data\n",
        "\n",
        "        # 그레이디언트 기록 시작\n",
        "        with tf.GradientTape() as tape:\n",
        "            # 정방향 계산\n",
        "            y_pred = self(x)\n",
        "            # compile() 메서드에서 지정한 손실 계산\n",
        "            loss = self.compiled_loss(y, y_pred)\n",
        "\n",
        "        # 훈련가능한 파라미터에 대한 그레이디언트 계산\n",
        "        gradients = tape.gradient(loss, self.trainable_variables)\n",
        "        # 파라미터 업데이트\n",
        "        self.optimizer.apply_gradients(zip(gradients, self.trainable_variables))\n",
        "        \n",
        "        # TF 2.4에서는\n",
        "        # self.optimizer.minimize(loss, self.trainable_variables, tape=tape)\n",
        "        \n",
        "        # compile() 메서드에서 지정한 지표 계산\n",
        "        self.compiled_metrics.update_state(y, y_pred)\n",
        "        \n",
        "        # 현재까지 지표와 결괏값을 딕셔너리로 반환\n",
        "        return {m.name: m.result() for m in self.metrics}"
      ],
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xIfS8tJVU5yE",
        "outputId": "fab050e9-0867-4a9f-a7fb-dd3a53ece80a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "my_custom_step = MyCustomStep()\n",
        "\n",
        "my_custom_step.compile(loss='sparse_categorical_crossentropy', \n",
        "                       metrics=['accuracy'])\n",
        "my_custom_step.fit(X_train, y_train, batch_size=32, epochs=2)"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/2\n",
            "1875/1875 [==============================] - 4s 2ms/step - loss: 0.4370 - accuracy: 0.8837\n",
            "Epoch 2/2\n",
            "1875/1875 [==============================] - 3s 2ms/step - loss: 0.3023 - accuracy: 0.9158\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fa70ade5fd0>"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    }
  ]
}