{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'2.6.0'" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from tensorflow import keras\n", "keras.__version__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 뉴스 기사 분류: 다중 분류 문제\n", "\n", "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/케라스-창시자에게-배우는-딥러닝/) 책의 3장 5절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다. 이 노트북의 설명은 케라스 버전 2.2.2에 맞추어져 있습니다. 케라스 최신 버전이 릴리스되면 노트북을 다시 테스트하기 때문에 설명과 코드의 결과가 조금 다를 수 있습니다.\n", "\n", "----\n", "\n", "이전 섹션에서 완전 연결된 신경망을 사용해 벡터 입력을 어떻게 두 개의 클래스로 분류하는지 보았습니다. 두 개 이상의 클래스가 있을 때는 어떻게 해야 할까요?\n", "\n", "이 절에서 로이터 뉴스를 46개의 상호 배타적인 토픽으로 분류하는 신경망을 만들어 보겠습니다. 클래스가 많기 때문에 이 문제는 다중 분류의 예입니다. 각 데이터 포인트가 정확히 하나의 범주로 분류되기 때문에 좀 더 정확히 말하면 단일 레이블 다중 분류 문제입니다. 각 데이터 포인트가 여러 개의 범주(가령, 토픽)에 속할 수 있다면 이런 문제는 다중 레이블 다중 분류의 문제가 됩니다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 로이터 데이터셋\n", "\n", "1986년에 로이터에서 공개한 짧은 뉴스 기사와 토픽의 집합인 로이터 데이터셋을 사용하겠습니다. 이 데이터셋은 텍스트 분류를 위해 널리 사용되는 간단한 데이터셋입니다. 46개의 토픽이 있으며 어떤 토픽은 다른 것에 비해 데이터가 많습니다. 각 토픽은 훈련 세트에 최소한 10개의 샘플을 가지고 있습니다.\n", "\n", "IMDB와 MNIST와 마찬가지로 로이터 데이터셋은 케라스에 포함되어 있습니다. 한 번 살펴보죠:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras.datasets import reuters\n", "\n", "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(num_words=10000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "IMDB 데이터셋에서처럼 num_words=10000 매개변수는 데이터에서 가장 자주 등장하는 단어 10,000개로 제한합니다.\n", "\n", "여기에는 8,982개의 훈련 샘플과 2,246개의 테스트 샘플이 있습니다:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "8982" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(train_data)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2246" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(test_data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "IMDB 리뷰처럼 각 샘플은 정수 리스트입니다(단어 인덱스):" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1,\n", " 245,\n", " 273,\n", " 207,\n", " 156,\n", " 53,\n", " 74,\n", " 160,\n", " 26,\n", " 14,\n", " 46,\n", " 296,\n", " 26,\n", " 39,\n", " 74,\n", " 2979,\n", " 3554,\n", " 14,\n", " 46,\n", " 4689,\n", " 4329,\n", " 86,\n", " 61,\n", " 3499,\n", " 4795,\n", " 14,\n", " 61,\n", " 451,\n", " 4329,\n", " 17,\n", " 12]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_data[10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "궁금한 경우를 위해 어떻게 단어로 디코딩하는지 알아보겠습니다:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/reuters_word_index.json\n", "557056/550378 [==============================] - 0s 0us/step\n", "565248/550378 [==============================] - 0s 0us/step\n" ] } ], "source": [ "word_index = reuters.get_word_index()\n", "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n", "# 0, 1, 2는 '패딩', '문서 시작', '사전에 없음'을 위한 인덱스이므로 3을 뺍니다\n", "decoded_newswire = ' '.join([reverse_word_index.get(i - 3, '?') for i in train_data[0]])" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'? ? ? said as a result of its december acquisition of space co it expects earnings per share in 1987 of 1 15 to 1 30 dlrs per share up from 70 cts in 1986 the company said pretax net should rise to nine to 10 mln dlrs from six mln dlrs in 1986 and rental operation revenues to 19 to 22 mln dlrs from 12 5 mln dlrs it said cash flow per share this year should be 2 50 to three dlrs reuter 3'" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "decoded_newswire" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "샘플에 연결된 레이블은 토픽의 인덱스로 0과 45 사이의 정수입니다." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "3" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_labels[10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 데이터 준비\n", "\n", "이전의 예제와 동일한 코드를 사용해서 데이터를 벡터로 변환합니다:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "\n", "def vectorize_sequences(sequences, dimension=10000):\n", " results = np.zeros((len(sequences), dimension))\n", " for i, sequence in enumerate(sequences):\n", " results[i, sequence] = 1.\n", " return results\n", "\n", "# 훈련 데이터 벡터 변환\n", "x_train = vectorize_sequences(train_data)\n", "# 테스트 데이터 벡터 변환\n", "x_test = vectorize_sequences(test_data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "레이블을 벡터로 바꾸는 방법은 두 가지입니다. 레이블의 리스트를 정수 텐서로 변환하는 것과 원-핫 인코딩을 사용하는 것입니다. 원-핫 인코딩이 범주형 데이터에 널리 사용되기 때문에 범주형 인코딩이라고도 부릅니다. 원-핫 인코딩에 대한 자세한 설명은 6.1절을 참고하세요. 이 경우 레이블의 원-핫 인코딩은 각 레이블의 인덱스 자리는 1이고 나머지는 모두 0인 벡터입니다:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "def to_one_hot(labels, dimension=46):\n", " results = np.zeros((len(labels), dimension))\n", " for i, label in enumerate(labels):\n", " results[i, label] = 1.\n", " return results\n", "\n", "# 훈련 레이블 벡터 변환\n", "one_hot_train_labels = to_one_hot(train_labels)\n", "# 테스트 레이블 벡터 변환\n", "one_hot_test_labels = to_one_hot(test_labels)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "MNIST 예제에서 이미 보았듯이 케라스에는 이를 위한 내장 함수가 있습니다:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras.utils import to_categorical\n", "\n", "one_hot_train_labels = to_categorical(train_labels)\n", "one_hot_test_labels = to_categorical(test_labels)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 모델 구성\n", "\n", "이 토픽 분류 문제는 이전의 영화 리뷰 분류 문제와 비슷해 보입니다. 두 경우 모두 짧은 텍스트를 분류하는 것이죠. 여기에서는 새로운 제약 사항이 추가되었습니다. 출력 클래스의 개수가 2에서 46개로 늘어난 점입니다. 출력 공간의 차원이 훨씬 커졌습니다.\n", "\n", "이전에 사용했던 것처럼 `Dense` 층을 쌓으면 각 층은 이전 층의 출력에서 제공한 정보만 사용할 수 있습니다. 한 층이 분류 문제에 필요한 일부 정보를 누락하면 그 다음 층에서 이를 복원할 방법이 없습니다. 각 층은 잠재적으로 정보의 병목이 될 수 있습니다. 이전 예제에서 16차원을 가진 중간층을 사용했지만 16차원 공간은 46개의 클래스를 구분하기에 너무 제약이 많을 것 같습니다. 이렇게 규모가 작은 층은 유용한 정보를 완전히 잃게 되는 정보의 병목 지점처럼 동작할 수 있습니다.\n", "\n", "이런 이유로 좀 더 규모가 큰 층을 사용하겠습니다. 64개의 유닛을 사용해 보죠:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras import models\n", "from tensorflow.keras import layers\n", "\n", "model = models.Sequential()\n", "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n", "model.add(layers.Dense(64, activation='relu'))\n", "model.add(layers.Dense(46, activation='softmax'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이 구조에서 주목해야 할 점이 두 가지 있습니다:\n", "\n", "* 마지막 `Dense` 층의 크기가 46입니다. 각 입력 샘플에 대해서 46차원의 벡터를 출력한다는 뜻입니다. 이 벡터의 각 원소(각 차원)은 각기 다른 출력 클래스가 인코딩된 것입니다.\n", "* 마지막 층에 `softmax` 활성화 함수가 사용되었습니다. MNIST 예제에서 이런 방식을 보았습니다. 각 입력 샘플마다 46개의 출력 클래스에 대한 확률 분포를 출력합니다. 즉, 46차원의 출력 벡터를 만들며 `output[i]`는 어떤 샘플이 클래스 `i`에 속할 확률입니다. 46개의 값을 모두 더하면 1이 됩니다.\n", "\n", "이런 문제에 사용할 최선의 손실 함수는 `categorical_crossentropy`입니다. 이 함수는 두 확률 분포의 사이의 거리를 측정합니다. 여기에서는 네트워크가 출력한 확률 분포와 진짜 레이블의 분포 사이의 거리입니다. 두 분포 사이의 거리를 최소화하면 진짜 레이블에 가능한 가까운 출력을 내도록 모델을 훈련하게 됩니다." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "model.compile(optimizer='rmsprop',\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 훈련 검증\n", "\n", "훈련 데이터에서 1,000개의 샘플을 따로 떼어서 검증 세트로 사용하겠습니다:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "x_val = x_train[:1000]\n", "partial_x_train = x_train[1000:]\n", "\n", "y_val = one_hot_train_labels[:1000]\n", "partial_y_train = one_hot_train_labels[1000:]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이제 20번의 에포크로 모델을 훈련시킵니다:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/20\n", "16/16 [==============================] - 1s 35ms/step - loss: 2.6304 - accuracy: 0.5178 - val_loss: 1.7735 - val_accuracy: 0.6440\n", "Epoch 2/20\n", "16/16 [==============================] - 0s 20ms/step - loss: 1.4333 - accuracy: 0.7046 - val_loss: 1.2986 - val_accuracy: 0.7130\n", "Epoch 3/20\n", "16/16 [==============================] - 0s 20ms/step - loss: 1.0527 - accuracy: 0.7747 - val_loss: 1.1326 - val_accuracy: 0.7560\n", "Epoch 4/20\n", "16/16 [==============================] - 0s 20ms/step - loss: 0.8278 - accuracy: 0.8254 - val_loss: 1.0336 - val_accuracy: 0.7940\n", "Epoch 5/20\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.6590 - accuracy: 0.8661 - val_loss: 1.0011 - val_accuracy: 0.7900\n", "Epoch 6/20\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.5263 - accuracy: 0.8966 - val_loss: 0.9220 - val_accuracy: 0.8180\n", "Epoch 7/20\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.4251 - accuracy: 0.9152 - val_loss: 0.8995 - val_accuracy: 0.8180\n", "Epoch 8/20\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.3422 - accuracy: 0.9278 - val_loss: 0.8942 - val_accuracy: 0.8110\n", "Epoch 9/20\n", "16/16 [==============================] - 0s 18ms/step - loss: 0.2834 - accuracy: 0.9386 - val_loss: 0.9177 - val_accuracy: 0.8130\n", "Epoch 10/20\n", "16/16 [==============================] - 0s 20ms/step - loss: 0.2433 - accuracy: 0.9420 - val_loss: 1.0039 - val_accuracy: 0.7990\n", "Epoch 11/20\n", "16/16 [==============================] - 0s 18ms/step - loss: 0.2069 - accuracy: 0.9476 - val_loss: 0.9506 - val_accuracy: 0.8120\n", "Epoch 12/20\n", "16/16 [==============================] - 0s 18ms/step - loss: 0.1840 - accuracy: 0.9515 - val_loss: 0.9389 - val_accuracy: 0.8130\n", "Epoch 13/20\n", "16/16 [==============================] - 0s 22ms/step - loss: 0.1681 - accuracy: 0.9521 - val_loss: 0.9823 - val_accuracy: 0.8140\n", "Epoch 14/20\n", "16/16 [==============================] - 0s 18ms/step - loss: 0.1509 - accuracy: 0.9540 - val_loss: 1.0148 - val_accuracy: 0.8070\n", "Epoch 15/20\n", "16/16 [==============================] - 0s 18ms/step - loss: 0.1441 - accuracy: 0.9548 - val_loss: 0.9849 - val_accuracy: 0.8150\n", "Epoch 16/20\n", "16/16 [==============================] - 0s 18ms/step - loss: 0.1310 - accuracy: 0.9550 - val_loss: 0.9869 - val_accuracy: 0.8090\n", "Epoch 17/20\n", "16/16 [==============================] - 0s 18ms/step - loss: 0.1235 - accuracy: 0.9558 - val_loss: 1.0289 - val_accuracy: 0.8020\n", "Epoch 18/20\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.1228 - accuracy: 0.9569 - val_loss: 1.0548 - val_accuracy: 0.8060\n", "Epoch 19/20\n", "16/16 [==============================] - 0s 20ms/step - loss: 0.1162 - accuracy: 0.9579 - val_loss: 1.0817 - val_accuracy: 0.7980\n", "Epoch 20/20\n", "16/16 [==============================] - 0s 18ms/step - loss: 0.1126 - accuracy: 0.9573 - val_loss: 1.0717 - val_accuracy: 0.8010\n" ] } ], "source": [ "history = model.fit(partial_x_train,\n", " partial_y_train,\n", " epochs=20,\n", " batch_size=512,\n", " validation_data=(x_val, y_val))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "손실과 정확도 곡선을 그려 보죠:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAgAElEQVR4nO3deZhU5bXv8e9itmlEGZyYGqKIInMDKg6oGRQNKGqUcFWCihANUXM0RBLlmHDvyYnJMTjjPKDoMYbjgMcJEdQ4AKKCYkQFbUWDINDIIMO6f7y7oWiququHXVXd9fs8Tz21a0+1anf1XvUO+93m7oiISP5qkO0AREQku5QIRETynBKBiEieUyIQEclzSgQiInlOiUBEJM8pEUitMrOnzey82l43m8xsmZl9P4b9upkdGE3fama/S2fdarzPSDN7trpxVrDfwWZWUtv7lcxrlO0AJPvMbH3CywJgM7Aten2Ru09Ld1/uflIc69Z37j62NvZjZkXAJ0Bjd98a7XsakPbfUPKPEoHg7oVl02a2DLjA3Z8vv56ZNSo7uYhI/aGqIUmprOhvZr82sy+Bu81sbzN70sxWmtk30XT7hG1mm9kF0fQoM3vZzK6L1v3EzE6q5rqdzWyOmZWa2fNmdpOZPZAi7nRi/L2ZvRLt71kza5Ow/BwzW25mq8xsYgXHZ6CZfWlmDRPmnWZm70TTA8zsH2a2xsxWmNmNZtYkxb7uMbM/JLy+ItrmCzMbXW7dk83sLTNbZ2afmdmkhMVzouc1ZrbezI4oO7YJ2x9pZm+a2dro+ch0j01FzOyQaPs1ZrbYzIYmLBtiZu9F+/zczP4tmt8m+vusMbPVZjbXzHReyjAdcKnMfkAroBMwhvCduTt63RHYCNxYwfYDgQ+ANsB/AneamVVj3QeBN4DWwCTgnAreM50Yfwr8DNgHaAKUnZgOBW6J9n9A9H7tScLdXwe+BY4vt98Ho+ltwGXR5zkCOAH4eQVxE8VwYhTPD4CDgPLtE98C5wJ7AScD48zs1GjZMdHzXu5e6O7/KLfvVsBTwJTos/0FeMrMWpf7DLsdm0pibgw8ATwbbfcLYJqZHRytciehmrEFcBgwK5r/K6AEaAvsC1wFaNybDFMikMpsB65x983uvtHdV7n739x9g7uXApOBYyvYfrm73+7u24B7gf0J//Bpr2tmHYH+wNXu/p27vww8nuoN04zxbnf/p7tvBB4BekfzzwCedPc57r4Z+F10DFJ5CBgBYGYtgCHRPNx9vru/5u5b3X0ZcFuSOJL5SRTfInf/lpD4Ej/fbHd/1923u/s70fuls18IieNDd78/iushYAnw44R1Uh2bihwOFAL/Ef2NZgFPEh0bYAtwqJnt6e7fuPuChPn7A53cfYu7z3UNgJZxSgRSmZXuvqnshZkVmNltUdXJOkJVxF6J1SPlfFk24e4bosnCKq57ALA6YR7AZ6kCTjPGLxOmNyTEdEDivqMT8apU70X49T/czJoCw4EF7r48iqNrVO3xZRTH/yWUDiqzSwzA8nKfb6CZvRhVfa0Fxqa537J9Ly83bznQLuF1qmNTaczunpg0E/d7OiFJLjezl8zsiGj+n4ClwLNm9rGZTUjvY0htUiKQypT/dfYr4GBgoLvvyc6qiFTVPbVhBdDKzAoS5nWoYP2axLgicd/Re7ZOtbK7v0c44Z3ErtVCEKqYlgAHRXFcVZ0YCNVbiR4klIg6uHtL4NaE/Vb2a/oLQpVZoo7A52nEVdl+O5Sr39+xX3d/092HEaqNZhBKGrh7qbv/yt27AEOBy83shBrGIlWkRCBV1YJQ574mqm++Ju43jH5hzwMmmVmT6NfkjyvYpCYxPgqcYmZHRQ2711L5/8mDwC8JCee/y8WxDlhvZt2AcWnG8AgwyswOjRJR+fhbEEpIm8xsACEBlVlJqMrqkmLfM4GuZvZTM2tkZmcBhxKqcWridULp4Uoza2xmgwl/o+nR32ykmbV09y2EY7IdwMxOMbMDo7agtYR2lYqq4iQGSgRSVdcDewBfA68B/5uh9x1JaHBdBfwBeJhwvUMy1Y7R3RcDFxNO7iuAbwiNmRUpq6Of5e5fJ8z/N8JJuhS4PYo5nRiejj7DLEK1yaxyq/wcuNbMSoGriX5dR9tuILSJvBL1xDm83L5XAacQSk2rgCuBU8rFXWXu/h3hxH8S4bjfDJzr7kuiVc4BlkVVZGMJf08IjeHPA+uBfwA3u/uLNYlFqs7ULiN1kZk9DCxx99hLJCL1nUoEUieYWX8z+56ZNYi6Vw4j1DWLSA3pymKpK/YDHiM03JYA49z9reyGJFI/qGpIRCTPqWpIRCTP1bmqoTZt2nhRUVG2wxARqVPmz5//tbu3TbasziWCoqIi5s2bl+0wRETqFDMrf0X5DqoaEhHJc0oEIiJ5TolARCTP1bk2AhHJvC1btlBSUsKmTZsqX1myqlmzZrRv357GjRunvY0SgYhUqqSkhBYtWlBUVETq+wpJtrk7q1atoqSkhM6dO6e9XV5UDU2bBkVF0KBBeJ6m23iLVMmmTZto3bq1kkCOMzNat25d5ZJbvS8RTJsGY8bAhuiWJsuXh9cAI0em3k5EdqUkUDdU5+9U70sEEyfuTAJlNmwI80VEJA8SwaefVm2+iOSeVatW0bt3b3r37s1+++1Hu3btdrz+7rvvKtx23rx5jB8/vtL3OPLII2sl1tmzZ3PKKafUyr4ypd4ngo7lb/JXyXwRqbnabpdr3bo1CxcuZOHChYwdO5bLLrtsx+smTZqwdevWlNsWFxczZcqUSt/j1VdfrVmQdVi9TwSTJ0NBwa7zCgrCfBGpfWXtcsuXg/vOdrna7qQxatQoxo4dy8CBA7nyyit54403OOKII+jTpw9HHnkkH3zwAbDrL/RJkyYxevRoBg8eTJcuXXZJEIWFhTvWHzx4MGeccQbdunVj5MiRlI3SPHPmTLp160a/fv0YP358pb/8V69ezamnnkrPnj05/PDDeeeddwB46aWXdpRo+vTpQ2lpKStWrOCYY46hd+/eHHbYYcydO7d2D1gF6n1jcVmD8MSJoTqoY8eQBNRQLBKPitrlavv/rqSkhFdffZWGDRuybt065s6dS6NGjXj++ee56qqr+Nvf/rbbNkuWLOHFF1+ktLSUgw8+mHHjxu3W5/6tt95i8eLFHHDAAQwaNIhXXnmF4uJiLrroIubMmUPnzp0ZMWJEpfFdc8019OnThxkzZjBr1izOPfdcFi5cyHXXXcdNN93EoEGDWL9+Pc2aNWPq1Kn86Ec/YuLEiWzbto0N5Q9ijOp9IoDw5dOJXyQzMtkud+aZZ9KwYUMA1q5dy3nnnceHH36ImbFly5ak25x88sk0bdqUpk2bss8++/DVV1/Rvn37XdYZMGDAjnm9e/dm2bJlFBYW0qVLlx3980eMGMHUqVMrjO/ll1/ekYyOP/54Vq1axbp16xg0aBCXX345I0eOZPjw4bRv357+/fszevRotmzZwqmnnkrv3r1rdGyqot5XDYlIZmWyXa558+Y7pn/3u99x3HHHsWjRIp544omUfembNm26Y7phw4ZJ2xfSWacmJkyYwB133MHGjRsZNGgQS5Ys4ZhjjmHOnDm0a9eOUaNGcd9999Xqe1ZEiUBEalW22uXWrl1Lu3btALjnnntqff8HH3wwH3/8McuWLQPg4YcfrnSbo48+mmlR48js2bNp06YNe+65Jx999BE9evTg17/+Nf3792fJkiUsX76cfffdlwsvvJALLriABQsW1PpnSCW2RGBmHczsRTN7z8wWm9kvk6wz2MzWmtnC6HF1XPGISGaMHAlTp0KnTmAWnqdOjb969sorr+Q3v/kNffr0qfVf8AB77LEHN998MyeeeCL9+vWjRYsWtGzZssJtJk2axPz58+nZsycTJkzg3nvvBeD666/nsMMOo2fPnjRu3JiTTjqJ2bNn06tXL/r06cPDDz/ML3+52ykzNrHds9jM9gf2d/cFZtYCmA+c6u7vJawzGPg3d0+7021xcbHrxjQimfX+++9zyCGHZDuMrFu/fj2FhYW4OxdffDEHHXQQl112WbbD2k2yv5eZzXf34mTrx1YicPcV7r4gmi4F3gfaxfV+IiJxu/322+nduzfdu3dn7dq1XHTRRdkOqVZkpNeQmRUBfYDXkyw+wszeBr4glA4WJ9l+DDAGoKOuBBORLLnssstysgRQU7E3FptZIfA34FJ3X1du8QKgk7v3Am4AZiTbh7tPdfdidy9u2zbpvZdFRKSaYk0EZtaYkASmuftj5Ze7+zp3Xx9NzwQam1mbOGMSEZFdxdlryIA7gffd/S8p1tkvWg8zGxDFsyqumEREZHdxthEMAs4B3jWzhdG8q4COAO5+K3AGMM7MtgIbgbM9rm5MIiKSVJy9hl52d3P3nu7eO3rMdPdboySAu9/o7t3dvZe7H+7u+Tv8n4ikdNxxx/HMM8/sMu/6669n3LhxKbcZPHgwZV3NhwwZwpo1a3ZbZ9KkSVx33XUVvveMGTN4770dvd65+uqref7556sSflK5NFy1riwWkZw3YsQIpk+fvsu86dOnpzXwG4RRQ/faa69qvXf5RHDttdfy/e9/v1r7ylVKBCKS88444wyeeuqpHTehWbZsGV988QVHH30048aNo7i4mO7du3PNNdck3b6oqIivv/4agMmTJ9O1a1eOOuqoHUNVQ7hGoH///vTq1YvTTz+dDRs28Oqrr/L4449zxRVX0Lt3bz766CNGjRrFo48+CsALL7xAnz596NGjB6NHj2bz5s073u+aa66hb9++9OjRgyVLllT4+bI9XHVejD4qIrXn0kth4cLK16uK3r3h+utTL2/VqhUDBgzg6aefZtiwYUyfPp2f/OQnmBmTJ0+mVatWbNu2jRNOOIF33nmHnj17Jt3P/PnzmT59OgsXLmTr1q307duXfv36ATB8+HAuvPBCAH77299y55138otf/IKhQ4dyyimncMYZZ+yyr02bNjFq1CheeOEFunbtyrnnnsstt9zCpZdeCkCbNm1YsGABN998M9dddx133HFHys+X7eGqVSIQkTohsXoosVrokUceoW/fvvTp04fFixfvUo1T3ty5cznttNMoKChgzz33ZOjQoTuWLVq0iKOPPpoePXowbdo0Fi/e7drWXXzwwQd07tyZrl27AnDeeecxZ86cHcuHDx8OQL9+/XYMVJfKyy+/zDnnnAMkH656ypQprFmzhkaNGtG/f3/uvvtuJk2axLvvvkuLFi0q3Hc6VCIQkSqp6Jd7nIYNG8Zll13GggUL2LBhA/369eOTTz7huuuu480332Tvvfdm1KhRKYefrsyoUaOYMWMGvXr14p577mH27Nk1irdsKOuaDGM9YcIETj75ZGbOnMmgQYN45plndgxX/dRTTzFq1Cguv/xyzj333BrFqhKBiNQJhYWFHHfccYwePXpHaWDdunU0b96cli1b8tVXX/H0009XuI9jjjmGGTNmsHHjRkpLS3niiSd2LCstLWX//fdny5YtO4aOBmjRogWlpaW77evggw9m2bJlLF26FID777+fY489tlqfLdvDVatEICJ1xogRIzjttNN2VBGVDdvcrVs3OnTowKBBgyrcvm/fvpx11ln06tWLffbZh/79++9Y9vvf/56BAwfStm1bBg4cuOPkf/bZZ3PhhRcyZcqUHY3EAM2aNePuu+/mzDPPZOvWrfTv35+xY8dW63OV3Uu5Z8+eFBQU7DJc9YsvvkiDBg3o3r07J510EtOnT+dPf/oTjRs3prCwsFZuYBPbMNRx0TDUIpmnYajrlpwZhlpEROoGJQIRkTynRCAiaalr1cj5qjp/JyUCEalUs2bNWLVqlZJBjnN3Vq1aRbNmzaq0nXoNiUil2rdvT0lJCStXrsx2KFKJZs2a0b59+ypto0QgIpVq3LgxnTt3znYYEhNVDYmI5DklAhGRPKdEICKS55QIRETynBKBiEieUyIQEclzSgQiInlOiUBEJM8pEYiI5DklAhGRPKdEICKS55QIRETynBKBiEieUyIQEclzSgQiInlOiUBEJM8pEYiI5LnYEoGZdTCzF83sPTNbbGa/TLKOmdkUM1tqZu+YWd+44hERkeTivFXlVuBX7r7AzFoA883sOXd/L2Gdk4CDosdA4JboWUREMiS2EoG7r3D3BdF0KfA+0K7casOA+zx4DdjLzPaPKyYREdldRtoIzKwI6AO8Xm5RO+CzhNcl7J4sMLMxZjbPzOatXLkyrjBFRPJS7InAzAqBvwGXuvu66uzD3ae6e7G7F7dt27Z2AxQRyXOxJgIza0xIAtPc/bEkq3wOdEh43T6aJyIiGRJnryED7gTed/e/pFjtceDcqPfQ4cBad18RV0wiIrK7OHsNDQLOAd41s4XRvKuAjgDufiswExgCLAU2AD+LMR4REUkitkTg7i8DVsk6DlwcVwwiIlI5XVksIpLnlAhERPKcEoGISJ5TIhARyXNKBCIieU6JQEQkzykRiIjkOSUCEZE8p0QgIpLnlAhERPJc3iSCLVtg5kxwz3YkIiK5JW8SwX33wcknwz/+ke1IRERyS94kgrPPhpYtYcqUbEciIpJb8iYRNG8OF1wAjz4KJSXZjkZEJHfkTSIAuPhi2L4dbr0125GIiOSOvEoEnTvD0KFw222waVO2oxERyQ15lQgAxo+Hr7+G6dOzHYmISG7Iu0Rw3HHQvXtoNFZXUhGRPEwEZqFU8NZb8Mor2Y5GRCT78i4RAIwcCXvvra6kIiKQp4mgrCvpY4/BZ59lOxoRkezKy0QA8POfhzaCW27JdiQiItmVt4mgqAiGDYOpU2HjxmxHIyKSPXmbCCA0Gq9aBQ89lO1IRESyJ68TwbHHQo8e6koqIvktrxNBWVfSt9+GuXOzHY2ISHbkdSIA+OlPoVUrdSUVkfyV94mgoAAuvBD+/nf49NNsRyMiknl5nwggdCUFuPnm7MYhIpINSgRAx45w2mlw++2wYUO2oxERyazYEoGZ3WVm/zKzRSmWDzaztWa2MHpcHVcs6Rg/HlavhgcfzGYUIiKZF2eJ4B7gxErWmevuvaPHtTHGUqmjj4ZevdSVVETyT2yJwN3nAKvj2n9tK+tK+u678NJL2Y5GRCRzst1GcISZvW1mT5tZ91QrmdkYM5tnZvNWrlwZWzAjRkDr1upKKiL5JZuJYAHQyd17ATcAM1Kt6O5T3b3Y3Yvbtm0bW0B77AFjxsD//A8sWxbb24iI5JS0EoGZNTezBtF0VzMbamaNa/LG7r7O3ddH0zOBxmbWpib7rA3jxoVqosSupNOmhUHqGjQIz9OmZSs6EZHal26JYA7QzMzaAc8C5xAag6vNzPYzM4umB0SxrKrJPmtDhw4wfHjoSvrtt+GkP2YMLF8eGpGXLw+vlQxEpL5INxGYu28AhgM3u/uZQMo6fQAzewj4B3CwmZWY2flmNtbMxkarnAEsMrO3gSnA2e650V9n/HhYsyac7CdO3P3agg0bwnwRkfqgUZrrmZkdAYwEzo/mNaxoA3cfUcnyG4Eb03z/jBo0CPr0CY3Gy5cnX0fDUYhIfZFuieBS4DfA3919sZl1AV6ML6zsKutKungx7LNP8nU6dsxsTCIicUkrEbj7S+4+1N3/GDUaf+3u42OOLavOPhvatAltBgUFuy4rKIDJk7MTl4hIbUu319CDZranmTUHFgHvmdkV8YaWXc2awUUXwYIF8Ic/QKdOoaTQqVO4veXIkdmOUESkdqRbNXSou68DTgWeBjoTeg7Va+PGhS6jn38erivYvj08KwmISH2SbiJoHF03cCrwuLtvAXKih0+c2rWDM86AO+6A9euzHY2ISDzSTQS3AcuA5sAcM+sErIsrqFwyfjysXQsPPJDtSERE4mHV7bpvZo3cfWstx1Op4uJinzdvXsbezx369w/XDixeHNoJRETqGjOb7+7FyZal21jc0sz+Ujbwm5n9mVA6qPfKupK+/z688EK2oxERqX3pVg3dBZQCP4ke64C74woq15x1VrieQKOSikh9lG4i+J67X+PuH0ePfwe6xBlYLmnaNHQlffLJcJN7EZH6JN1EsNHMjip7YWaDgI3xhJSbLr0UiovDgHTXXhu6koqI1AfpjjU0FrjPzFpGr78BzosnpNzUqhXMmRNGHr3mGnj7bbj3XigszHZkIiI1k+4QE29HN5DpCfR09z7A8bFGloOaNQsn/z//GWbMgCOPhI8/znZUIiI1U6U7lEU3kym7fuDyGOLJeWZw+eXw9NPw2Weha+msWdmOSkSk+mpyq8q87lH/wx/Cm2/CvvuG6RtuCNcciIjUNTVJBHl/2jvwQHjtNRgyJFxrcOGFsHlztqMSEamaChOBmZWa2bokj1LggAzFmNP23DO0F/z2t3DnnXDccfDll9mOSkQkfRUmAndv4e57Jnm0cPd0exzVew0awO9/D488EnoTFReHaiMRkbqgJlVDUs6ZZ8Irr0CjRnD00RqoTkTqBiWCWta7dygNHH44nHMOXHEFbNuW7ahERFJTIohB27bw3HNw8cVw3XVw8snwzTfZjkpEJDklgpg0bgw33hhuazlrFgwcGEYwFRHJNUoEMbvwwpAI1q4NyeDf/12lAxHJLUoEGXDUUTBvHhx/PEyaBJ06wW9+AytXZjsyERElgozp0CFcb/D22+ECtD/+MSSEyy+HL77IdnQiks+UCDKsZ0+YPh3eey90N50yBTp3hp//HJYvz3Z0IpKPqn3P4mzJ9D2La8O0aTBxInz6KXTsCJMnw8iRYdnHH4fSwd13h7GKzj0XJkyAgw7KbswiUnXbt4eRBT7+eNfHJ5+EB0CLFmH4+sTnZPOSLdtnH9h77+rFVtE9i5UIYjZtWriHwYYNO+cVFITeRGXJAMJIpn/6E9x+O3z3HZx9Nlx1FXTvnvmYRSS10tJwUk88ySdOJ443Zgbt20OXLlBUFEYhWL8+7KPsOXF6YyW3+7ryyvDDsTqUCLKoqCh5lU+nTrBs2e7zv/wS/vIXuPlm+PZbOP30UJro0yfuSEUEYNOmUHr/5JPwP5r4/Mknu3fyaNkynOi7dAnVvGXTXbqEGoCmTdN/723bQlJIliTWr4du3aBv3+p9LiWCLGrQIPnw1GYV3+5y1Sr4619DG8LateGitAkTwhXLjTTKk0i1bdkSSuDJTvTLlu3eeaNJk/DDrago+cm+ulU1mZaVRGBmdwGnAP9y98OSLDfgr8AQYAMwyt0XVLbfupYIqloiKG/NGrjpJviv/wrJoXHj0H7QrRsccsjOx8EHQ/PmtR29SN3lHv7H5s0Lw77Mnw9Ll0JJya4/who2DL36OncO/6/lnw84IPygq+uylQiOAdYD96VIBEOAXxASwUDgr+4+sLL91rVEkG4bQWXWrw/dTxctgiVLwlXKH3206zhGnTrtniAOOQTatKm9zyP5Y9Mm+Oc/d37fli4N36WuXcPjoINC/XeunCRXrAgn/LIT/7x58PXXYVmTJqHHXrdu4QSfeLJv3z4/StkVJYLYPr67zzGzogpWGUZIEg68ZmZ7mdn+7r4irpiyoexkn6rXULoKC+H//J9d523eHP45338/PMr+YefM2bXRqU2bkBC6dw9dVgcPzp1/3rqgtBTuvx8efTTcjOiEE8LFgW3bZjuy2rFmzc7vTuLjk092/nI2g3btYPXqXX/UNGsWjklZYkh83mefsF0cVq/eecIvO+l//nlY1qBB+K4PHRpuJVtcDD16VK2uPt/E2kYQJYInU5QIngT+w91fjl6/APza3Xf7uW9mY4AxAB07duy3XB3uK7R9e0g6if/US5aEi9lKS0O95ujRMGpU+OeW5D74IFTL3XNPOG7duoWTTWlpWN6zZ0gKJ5wAxxwTuvflsi+/hMWLdz/hJ95IqUmTcBJPLFF26xbmFRSE6pYvvgglhX/+Ez78cOfzRx+F+vcyLVrsmhgOPDAkju3bQ0l2+/Zdpyt7/u67EP+bb4YeOmW6dt15wu/fP3SsKCjI3HGtK7LWWFxbiSBRXasayiUbN8Jjj8Edd8Ds2eGX04knwvnnw49/HNof8t22bfDUU2HAwOeeC8fkrLPgkktgwICwfN68MH7UCy+E+09s3hzqmQcM2JkYjjgiu79AS0tDnfjrr8Mbb4Tnsl/MEO6sV74KsazapLrVJFu3hvawxORQljCWL6+de3p37BhO9mUn/n79YK+9ar7ffJCrieA2YLa7PxS9/gAYXFnVkBJB7Vi6NFzEds894RfePvuEi9nOPz+cEPLNqlVw112h2+6yZaGkNHZsGDRw331Tb7dxI7z66s7E8Oab4dfrHnuEMaaOPz4khr59Q7KIw9atoe2o7KT/xhvhl3PZv/b3vheS1IABoYrkkENg//3jq7ZJZtOmcFy3bAnHoUGD6j2reqf6cjURnAxcws7G4inuPqCyfSoR1K6tW+GZZ0Ip4cknw+sjj4QLLgjtCYWF2Y4wXm+9FX79P/hgOFkde2z49T9sWPVKSGvXwksv7UwMixaF+XvtFaqPOnQIv8ZbtgyPVNMtWiRPHO7h13XZr/w33gi//MvahFq3Dif8gQN3nvxbt67+8ZH6I1u9hh4CBgNtgK+Aa4DGAO5+a9R99EbgREL30Z9VVi0ESgRx+uoruO8+uPPOUD9eWBiucD7//HBiyeQvyDh9912oIrvhhvBrvqAg3E3u4ovDL+ba9NVXISnMmgVz54aLkdauTe+udYWFuyaJPfYIv/T/9a+wvGnTUNJIPOl36VJ//k5Su3RBmVSJe6j7vvNOeOSR0Euke/fQuNyjR+huV/bLti754ovQbfe220ID6YEHhpP/qFGZrWd2D7/g164Nj3Xrkk+Xf/3tt6FhtOzE36NHaNwVSYcSgVTbunXw8MMhKbz++q7LWrQISaHs0aHDrq/btw8n2Ez8Qt2yJZzoP/ss9JhK9rx6dYhlyJBQ/fPDH6obreQPJQKpFSUlocGvpGT3x2efhQt6yn+dmjffmRQOOCBUwzRtGh5Nmuw+nWxe2XTDhuE9Ek/uZdMrVuw+ZMfee4fk1LFjeC4qguHDQ0lAJN9k5YIyqX/KTuipbNkSqlzKJ4iy6bIL3b77LnS53HaJRM8AAAvdSURBVLy54vGWKtKs2c6T/A9+sPNkn3jir+8N3SK1RYlAak3jxjtPxunatm1nUkhMEMmmt26F/fYL+2/TRo2iIrVFiUCyqmHDUF2kK0FFskdNZXXAtGk7b2pRVBRei4jUFpUIclz50UuXLw+voeoD14mIJKMSQY6bOHHX0R4hvJ44MTvxiEj9o0SQ4z79tGrzRUSqSokgx3XsWLX5IiJVpUSQ4yZP3r1HTUFBmC8iUhuUCHLcyJFhfJxOnUK/+U6dqn6bSxGRiqjXUB0wcqRO/CISH5UIRETynBKBiEieUyIQEclzSgQiInlOiUBEJM8pEeQBDVonIhVR99F6ToPWiUhlVCKo5zRonYhURomgntOgdSJSGSWCek6D1olIZZQI6jkNWicilVEiqOc0aJ2IVEa9hvKABq0TkYqoRCAikueUCERE8pwSgaRFVyeL1F9qI5BK6epkkfot1hKBmZ1oZh+Y2VIzm5Bk+SgzW2lmC6PHBXHGI9Wjq5NF6rfYSgRm1hC4CfgBUAK8aWaPu/t75VZ92N0viSsOqTldnSxSv8VZIhgALHX3j939O2A6MCzG95OY6OpkkfotzkTQDvgs4XVJNK+8083sHTN71Mw6xBiPVJOuThap37Lda+gJoMjdewLPAfcmW8nMxpjZPDObt3LlyowGKLo6WaS+M3ePZ8dmRwCT3P1H0evfALj7/0uxfkNgtbu3rGi/xcXFPm/evNoOV0SkXjOz+e5enGxZnCWCN4GDzKyzmTUBzgYeLxfY/gkvhwLvxxiPZJGuQxDJXbH1GnL3rWZ2CfAM0BC4y90Xm9m1wDx3fxwYb2ZDga3AamBUXPFI9ug6BJHcFlvVUFxUNVT3FBWFk395nTrBsmWZjkYkP2WrakgE0HUIIrlOiUBip+sQRHKbEoHETtchiOQ2JQKJna5DEMltSgSSESNHhobh7dvDc1WTgLqfisRHw1BLzlP3U5F4qUQgOU/DYIvES4lAcp66n4rES4lAcp66n4rES4lAcl5tdD9VY7NIakoEkvNq2v20rLF5+XJw39nYrGQgEmisIan3NNaRiMYakjxXG43NqlqS+kyJQOq9mjY2q2pJ6jslAqn3atrYrOsYpL5TIpB6r6aNzapakvpOiUDyQk3GOsqFqiUlEomTEoFIJbJdtaQ2CombEoFIJbJdtVQbbRQqUUhFlAhE0pDNqqWaJpJcqJpSIsptSgQiMatp1VJNE0m2q6ZyIRFJJdy9Tj369evnInXNAw+4d+rkbhaeH3igatsWFLiH02h4FBSkvw+zXbcte5ilt32nTsm379QpM9vX9POX7aO6x782ts8FwDxPcV7N+om9qg8lAslHNTkR1fREXNNEku1EVNNEUl8SkRKBSB6r6Yks2yWCbCeS+pCI3JUIRPJeNqumsp2I6nqJpqbbl1EiEJEayWbVRrYTSV1PRGWUCESkTsvnEk0mSgTqPioiOa8m13HU9ILAmm5f0+7DtXGHvsroxjQiIjGbNi1ct/Hpp+H6j8mTq5bMaro9VHxjGiUCEZE8oDuUiYhISrEmAjM70cw+MLOlZjYhyfKmZvZwtPx1MyuKMx4REdldbInAzBoCNwEnAYcCI8zs0HKrnQ984+4HAv8F/DGueEREJLk4SwQDgKXu/rG7fwdMB4aVW2cYcG80/ShwgplZjDGJiEg5cSaCdsBnCa9LonlJ13H3rcBaoHX5HZnZGDObZ2bzVq5cGVO4IiL5qVG2A0iHu08FpgKY2UozW57lkFJpA3yd7SAqkOvxQe7HqPhqRvHVTE3i65RqQZyJ4HOgQ8Lr9tG8ZOuUmFkjoCWwqqKdunvb2gyyNpnZvFTds3JBrscHuR+j4qsZxVczccUXZ9XQm8BBZtbZzJoAZwOPl1vnceC8aPoMYJbXtQsbRETquNhKBO6+1cwuAZ4BGgJ3uftiM7uWMObF48CdwP1mthRYTUgWIiKSQbG2Ebj7TGBmuXlXJ0xvAs6MM4YMm5rtACqR6/FB7seo+GpG8dVMLPHVuSEmRESkdmmICRGRPKdEICKS55QIqsjMOpjZi2b2npktNrNfJllnsJmtNbOF0ePqZPuKMcZlZvZu9N67DdVqwZRojKd3zKxvBmM7OOG4LDSzdWZ2abl1Mn78zOwuM/uXmS1KmNfKzJ4zsw+j571TbHtetM6HZnZesnViiu9PZrYk+hv+3cz2SrFthd+HGOObZGafJ/wdh6TYtsIxyWKM7+GE2JaZ2cIU28Z6/FKdUzL6/Ut1xxo9UtzSDfYH+kbTLYB/AoeWW2cw8GQWY1wGtKlg+RDgacCAw4HXsxRnQ+BLoFO2jx9wDNAXWJQw7z+BCdH0BOCPSbZrBXwcPe8dTe+dofh+CDSKpv+YLL50vg8xxjcJ+Lc0vgMfAV2AJsDb5f+f4oqv3PI/A1dn4/ilOqdk8vunEkEVufsKd18QTZcC77P70Bm5bhhwnwevAXuZ2f5ZiOME4CN3z/qV4u4+h9CFOVHiWFj3Aqcm2fRHwHPuvtrdvwGeA07MRHzu/qyHoVkAXiNctJkVKY5fOtIZk6zGKoovGt/sJ8BDtf2+6ajgnJKx758SQQ1Ew2b3AV5PsvgIM3vbzJ42s+4ZDQwceNbM5pvZmCTL0xkHKhPOJvU/XzaPX5l93X1FNP0lsG+SdXLlWI4mlPKSqez7EKdLoqqru1JUbeTC8Tsa+MrdP0yxPGPHr9w5JWPfPyWCajKzQuBvwKXuvq7c4gWE6o5ewA3AjAyHd5S79yUMAX6xmR2T4fevVHS1+VDgv5Mszvbx242HcnhO9rU2s4nAVmBailWy9X24Bfge0BtYQah+yUUjqLg0kJHjV9E5Je7vnxJBNZhZY8IfbJq7P1Z+ubuvc/f10fRMoLGZtclUfO7+efT8L+DvhOJ3onTGgYrbScACd/+q/IJsH78EX5VVmUXP/0qyTlaPpZmNAk4BRkYni92k8X2Ihbt/5e7b3H07cHuK98328WsEDAceTrVOJo5finNKxr5/SgRVFNUn3gm87+5/SbHOftF6mNkAwnGucDC9WoyvuZm1KJsmNCguKrfa48C5Ue+hw4G1CUXQTEn5Kyybx6+cxLGwzgP+J8k6zwA/NLO9o6qPH0bzYmdmJwJXAkPdfUOKddL5PsQVX2K702kp3jedMcni9H1gibuXJFuYieNXwTklc9+/uFrC6+sDOIpQRHsHWBg9hgBjgbHROpcAiwk9IF4DjsxgfF2i9307imFiND8xPiPcPe4j4F2gOMPHsDnhxN4yYV5Wjx8hKa0AthDqWc8n3BvjBeBD4HmgVbRuMXBHwrajgaXR42cZjG8poX647Ht4a7TuAcDMir4PGYrv/uj79Q7hpLZ/+fii10MIPWU+ymR80fx7yr53Cetm9PhVcE7J2PdPQ0yIiOQ5VQ2JiOQ5JQIRkTynRCAikueUCERE8pwSgYhInlMiEImY2TbbdWTUWhsJ08yKEke+FMklsd6qUqSO2ejuvbMdhEimqUQgUoloPPr/jMakf8PMDozmF5nZrGhQtRfMrGM0f18L9wd4O3ocGe2qoZndHo05/6yZ7RGtPz4ai/4dM5uepY8peUyJQGSnPcpVDZ2VsGytu/cAbgSuj+bdANzr7j0JA75NieZPAV7yMGheX8IVqQAHATe5e3dgDXB6NH8C0Cfaz9i4PpxIKrqyWCRiZuvdvTDJ/GXA8e7+cTQ42Jfu3trMviYMm7Almr/C3duY2UqgvbtvTthHEWHc+IOi178GGrv7H8zsf4H1hFFWZ3g04J5IpqhEIJIeTzFdFZsTprexs43uZMLYT32BN6MRMUUyRolAJD1nJTz/I5p+lTBaJsBIYG40/QIwDsDMGppZy1Q7NbMGQAd3fxH4NdAS2K1UIhIn/fIQ2WkP2/UG5v/r7mVdSPc2s3cIv+pHRPN+AdxtZlcAK4GfRfN/CUw1s/MJv/zHEUa+TKYh8ECULAyY4u5rau0TiaRBbQQilYjaCIrd/etsxyISB1UNiYjkOZUIRETynEoEIiJ5TolARCTPKRGIiOQ5JQIRkTynRCAikuf+P29p5vZTo3+HAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "loss = history.history['loss']\n", "val_loss = history.history['val_loss']\n", "\n", "epochs = range(1, len(loss) + 1)\n", "\n", "plt.plot(epochs, loss, 'bo', label='Training loss')\n", "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n", "plt.title('Training and validation loss')\n", "plt.xlabel('Epochs')\n", "plt.ylabel('Loss')\n", "plt.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAgAElEQVR4nO3deZhU1bX38e+ikUmIKIMiII0RRI0ytXjBxOGqESeIihHsqGgMAfUafUOMuajBgRsNGo3ReNPGAQUFpyjmQlRwTJxoEFBQBLERiCKiDIoIDev9Y5+iq5vq7uqhhu76fZ6nnqoz1qrT1XvV3vucfczdERGR3NUk0wGIiEhmKRGIiOQ4JQIRkRynRCAikuOUCEREcpwSgYhIjlMikF2Y2UwzO7++180kMysxs+NTsF83swOi1/9rZtcks24t3qfQzJ6rbZwiVTFdR9A4mNlXcZOtgG+B7dH0z919Svqjyh5mVgJc5O6z6nm/DvRw92X1ta6Z5QMfAbu5e2l9xClSlaaZDkDqh7u3jr2uqtAzs6YqXCRb6PuYHdQ01MiZ2TFmtsrMfm1mnwL3m9meZvZ3M1trZl9Gr7vEbfOSmV0UvR5pZv80s1uidT8ys5NquW53M3vFzDaZ2Swzu8vMJlcSdzIx3mBm/4r295yZtY9bfq6ZrTCzdWY2rorjc4SZfWpmeXHzTjezhdHrAWb2upmtN7NPzOxOM2tWyb4eMLMb46Z/FW3zbzO7sMK6p5jZ22a20cxWmtn4uMWvRM/rzewrMxsYO7Zx2w8yszlmtiF6HpTssanhcd7LzO6PPsOXZvZU3LKhZjY/+gwfmtngaH65ZjgzGx/7O5tZftRE9lMz+xh4IZr/WPR32BB9Rw6J276lmd0a/T03RN+xlmb2f2b2XxU+z0IzOz3RZ5XKKRHkhn2AvYBuwCjC3/3+aHo/4Bvgziq2PwJYArQHfg/ca2ZWi3UfBt4C2gHjgXOreM9kYjwHuADoCDQDxgKY2cHA3dH+943erwsJuPubwNfAf1bY78PR6+3AFdHnGQgcB1xcRdxEMQyO4jkB6AFU7J/4GjgPaAucAowxsx9Fy46Kntu6e2t3f73CvvcC/g+4I/psfwD+z8zaVfgMuxybBKo7zg8RmhoPifZ1WxTDAOBB4FfRZzgKKKnseCRwNHAQcGI0PZNwnDoC84D4psxbgP7AIML3+EpgBzAJ+ElsJTPrDXQmHBupCXfXo5E9CP+Qx0evjwG2Ai2qWL8P8GXc9EuEpiWAkcCyuGWtAAf2qcm6hEKmFGgVt3wyMDnJz5Qoxqvjpi8G/hG9vhaYGrds9+gYHF/Jvm8E7otetyEU0t0qWfdy4G9x0w4cEL1+ALgxen0fcFPcej3j102w39uB26LX+dG6TeOWjwT+Gb0+F3irwvavAyOrOzY1Oc5AJ0KBu2eC9f4Si7eq7180PT72d477bPtXEUPbaJ09CInqG6B3gvVaAF8S+l0gJIw/p/v/rTE8VCPIDWvdfUtswsxamdlfoqr2RkJTRNv45pEKPo29cPfN0cvWNVx3X+CLuHkAKysLOMkYP417vTkupn3j9+3uXwPrKnsvwq//M8ysOXAGMM/dV0Rx9IyaSz6N4vgfQu2gOuViAFZU+HxHmNmLUZPMBmB0kvuN7XtFhXkrCL+GYyo7NuVUc5y7Ev5mXybYtCvwYZLxJrLz2JhZnpndFDUvbaSsZtE+erRI9F7Rd3oa8BMzawKMINRgpIaUCHJDxVPDfgkcCBzh7t+hrCmisuae+vAJsJeZtYqb17WK9esS4yfx+47es11lK7v7YkJBehLlm4UgNDG9T/jV+R3gv2sTA6FGFO9hYDrQ1d33AP43br/Vncr3b0JTTrz9gNVJxFVRVcd5JeFv1jbBdiuB71ayz68JtcGYfRKsE/8ZzwGGEprP9iDUGmIxfA5sqeK9JgGFhCa7zV6hGU2So0SQm9oQqtvro/bm36b6DaNf2MXAeDNrZmYDgdNSFOPjwKlm9v2oY/d6qv+uPwz8glAQPlYhjo3AV2bWCxiTZAyPAiPN7OAoEVWMvw3h1/aWqL39nLhlawlNMvtXsu8ZQE8zO8fMmprZ2cDBwN+TjK1iHAmPs7t/Qmi7/3PUqbybmcUSxb3ABWZ2nJk1MbPO0fEBmA8Mj9YvAIYlEcO3hFpbK0KtKxbDDkIz2x/MbN+o9jAwqr0RFfw7gFtRbaDWlAhy0+1AS8KvrTeAf6TpfQsJHa7rCO3y0wgFQCK1jtHdFwGXEAr3TwjtyKuq2ewRQgfmC+7+edz8sYRCehNwTxRzMjHMjD7DC8Cy6DnexcD1ZraJ0KfxaNy2m4EJwL8snK30HxX2vQ44lfBrfh2h8/TUCnEnq7rjfC6wjVAr+ozQR4K7v0XojL4N2AC8TFkt5RrCL/gvgesoX8NK5EFCjWw1sDiKI95Y4B1gDvAFcDPly64HgUMJfU5SC7qgTDLGzKYB77t7ymsk0niZ2XnAKHf/fqZjaahUI5C0MbPDzey7UVPCYEK78FPVbSdSmajZ7WKgKNOxNGRKBJJO+xBObfyKcA78GHd/O6MRSYNlZicS+lPWUH3zk1RBTUMiIjlONQIRkRzX4Aada9++vefn52c6DBGRBmXu3Lmfu3uHRMsaXCLIz8+nuLg402GIiDQoZlbxavSd1DQkIpLjlAhERHKcEoGISI5TIhARyXFKBCIiOU6JQEQkxaZMgfx8aNIkPE+ZUt0W6aVEICKNXl0L4rpsP2UKjBoFK1aAe3geNarm+0hpIsn0LdJq+ujfv7+LSHpNnuzerZu7WXiePLnhbD95snurVu6hGA6PVq2S30ddt+/Wrfy2sUe3bul5/xig2CspVzNesNf0oUQguSiXC9JMF8R13d4s8fZm6Xn/GCUCkQYs1wvSTBfEmS7I6/r+MVUlAvURiKRYXdt3x42DzZvLz9u8OcxPx/Yff1yz+dm2/X4V7xZdzfz63n7CBGjVqvy8Vq3C/HS8fzKUCERSqD46CnO9IM10QVzX7QsLoagIunUDs/BcVBTmp+P9k1JZVSFbH2oaknSrS/t6fbTvZrppJdNNU/XRWZrpzu66qo/3R30EIrVT10KoPtp3VZBmviBuDJQIJKdl8hd9fZ3xoYJU6qqqRNDgblVZUFDguh+BJCvWRh/fWdqqVfJttE2ahKK7IjPYsSP17y9SX8xsrrsXJFqmzmJp1Op6xkxdOyrr2lEokg5KBNKo1fWMmfo4Y6OwEEpKQg2ipERJQLKPEoE0avpFL1I9JQLJenW5IEu/6EWqp0QgWa2uF2TpF71I9XTWkGS1/PxQ+FfUrVv4dS4iydFZQ9Jg1bWzV0Sqp0QgWS0dA26J5DolAkm5THf2ikjVlAgkpdTZK5L91FksKaXOXpHsoM5iyRh19opkPyUCSSl19opkPyUCSSl19opkPyUCSSl19opkv5QmAjMbbGZLzGyZmV2VYHk3M5ttZgvN7CUz65LKeCQzNFaPSHZLWSIwszzgLuAk4GBghJkdXGG1W4AH3f0w4Hrgd6mKR0REEktljWAAsMzdl7v7VmAqMLTCOgcDL0SvX0ywXLJAXS4IE5Hsl8pE0BlYGTe9KpoXbwFwRvT6dKCNmbWruCMzG2VmxWZWvHbt2pQEK4nV9YIwEcl+me4sHgscbWZvA0cDq4HtFVdy9yJ3L3D3gg4dOqQ7xpxW11s9ikj2a5rCfa8GusZNd4nm7eTu/yaqEZhZa+BMd1+fwpikhnRBmEjjl8oawRygh5l1N7NmwHBgevwKZtbezGIx/Aa4L4XxSC3ogjCRxi9licDdS4FLgWeB94BH3X2RmV1vZkOi1Y4BlpjZB8DegC4zyjK6IEyk8dOgc1KtKVNCn8DHH4eawIQJuhZApKGpatC5VPYRSCNRWKiCX6Qxy/RZQyIikmFKBCIiOU6JQEQkxykRiIjkOCUCEZEcp0SQAzRonIhURaePNnKxQeNi4wXFBo0DnRIqIoFqBI2cBo0TkeooETRyGjRORKqjRNDIadA4EamOEkEjp0HjRKQ6SgSNXGEhFBVBt25gFp6LitRRLCJldNZQDtCgcSJSFdUIRERynBKBiEiOUyIQEclxSgQiIjlOiUBEJMcpEYiI5DglAhGRHKdEICKS45QIRERynBKBiEiOUyIQEclxSgQNgG41KSKppEHnspxuNSkiqaYaQZbTrSZFJNWUCLKcbjUpIqmmRJDldKtJEUk19RFkuQkTyvcRQGZuNTlnDtx+O2zdWvt9NGsG3/se9OsXHh061F98IlJ7SgRZLtYhPG5caA7ab7+QBNLZUXzPPXDppdC6NeyzT+338/XX8PDDZdNdu5YlhX79oH9/6NSp7vFK9lm9GhYuhM6d4cADoXnzTEck8czdMx1DjRQUFHhxcXGmw8gJW7aEBHDvvXDCCfDII9CuXd32+eWXMH8+zJ0L8+aFxwcfQOxruM8+5RNDv34hYZhVv293+OYb2LgxPDZsKP96xw445RTYe++6fYa62LEDvv028WPLluTn12bdHTugV6/yx7Zz5+SObU1s3Rr+xq+/Hh6vvQYrV5Ytz8uDAw4ItcNDDil79OgRao2SGmY2190LEi5TIpBEVqyAYcOguDjURq67LvwDp8KmTaHgiCWGefNg8eJQcEFIPv36wUEHhUKmYgEfe71xI5SWVv1eTZvCkCHws5+F5JaqzxSzdSv84x/w0EMwc2aoFdUHM2jRIvyyTvRItMwd3n0X3n+/7Nh26FCWFGKP/PyaJYc1a8oX+sXFIflASOKDBsHAgdCnD/z737BoUdnjww/LYmnaFHr2LJ8cYgmiqdou6kyJQGpk1iwYPhy2bYMHH4ShQ9Mfw+bNoSkhPjl88EHoH/nOd2CPPcJzTV5v2hQ+z6RJ8PnnoZntwgvhggvqt/PdPfSpPPQQTJ0a3qt9ezjzzFDjqVhIVzedaF7TprX/Jf/117BgQflju2hRWRLdc8/yiaFfv/ALvkmTsM4775QV+q+/DsuXh+122y0klYEDyx5dulQdy5YtITHFJ4dFi8I+Y0VTs2ahOemQQ6Bv3xBP3751r53mGiUCSYo73HxzqAEcdBA8+WT4hdbYbN0KTz8Nf/0rPP98mDd4MFx0EZx2WijQamPFCpg8OSSAJUtCgT1kCJx3Hpx4Yu33mw5btoQCPpYY5s4N07GTA9q0Cb/Mlywpq9Xss0/Zr/1Bg0IB3aJF/cSzefOuCeKdd8IxjsnP3zVhpaLZb9MmWLUq1Hx69oR9963/90iHjCUCMxsM/BHIA/7q7jdVWL4fMAloG61zlbvPqGqfSgSpsXEjjBwJf/sbnH12KCRbt850VKn30Udw//1w332hQ7Njx3AcLrooFHzV2bABHn881DReeSXMO+ooOPfc0LTWtm1Kw0+prVtDE10sOSxZEvoYYgV/t271379QnS++gLffLktW8+bB0qVlyzt33jU5VNUP8vXXoZBfuTI8Er3esKH8NvvtV3YMYk1e2ZzkYzKSCMwsD/gAOAFYBcwBRrj74rh1ioC33f1uMzsYmOHu+VXtV4mg/i1eDGecAcuWwcSJcPnl6f8Hz7TSUnj22XCG1N//Dtu3w9FHh76EM86Ali3L1t22DZ57LhT+06eHX9M9e4bCv7AQunfP3OfIRRs3liWH2CO+H6Rjx5AQvve98Os+vpD/8std99exY2jS6tq17NGlS+hPWbSorFls1aqwfsuWUFBQPjl07Fj/n/Pbb0Otvba1rkwlgoHAeHc/MZr+DYC7/y5unb8Ay9395mj9W919UFX7VSKoX489FtrId98dHn00FH657pNP4IEHQq1o+fLwq/4nP4GTTw7J4uGHYe3a0EY9fHhIAAMG5F7yzGbx/SCxmsPixeFvGV/IV3zduXPyBe3KleU7yd9+O/xIANh//7KkMHAgHHrorh3eW7eG/qPPPgvfp+qeN24MP1Quuqh2xyRTiWAYMNjdL4qmzwWOcPdL49bpBDwH7AnsDhzv7nMT7GsUMApgv/32678ivqFQaqW0FK66Cm69NXxRH3ss/BNImR074KWXQkJ44onwj9usWehHOPdcOOkkne7YkLinNll/801IOLFO9NdeC/0KEH5o9e8fapqxgn39+sT7ado0nFzQsWN4dOhQ9nzyyaGjvDayORH8vyiGW6Mawb3A99x9R2X7VY2g7j77LPQDvPQSXHIJ/OEPKtCqs25d+Oc+8shwVo1IddyhpKQsMcybF2ob8QV7osK+bdtwhlZ9qyoRpPLs3NVA17jpLtG8eD8FBgO4++tm1gJoD3yWwrgapEcfhfHjQyGUqP2ya9dwxkR1X6A33gidmOvWhdMozzsvLeE3eO3awamnZjoKaUjMQn9R9+7ZP2R8KhPBHKCHmXUnJIDhwDkV1vkYOA54wMwOAloAa1MYU4P0zDNwzjnhlM5mzcIvi6efDp1H8Zo2Dc07ido/u3aFN98MHcFduoRfKH36ZObziEh2SVkicPdSM7sUeJZwauh97r7IzK4Hit19OvBL4B4zuwJwYKQ3tAsbUuzFF+Gss8JZD7Nnh/O5IVQ7P/+88lPf3ngjTMc6r2IGDw43u9lrr/R/FhHJTrqgLIu99RYcd1w4X/vll2t+JeWOHaFTKpYkduwIVwmnelgFEck+meojkDp4993w671jx3DOem0up2/SJPQb7L13OM9ZRCQR3ZgmC334YRgQrWXLMO5PQ72kXUQahmoTgZmdZmZKGGmyejUcf3xo23/+eV2lKiKpl0wBfzaw1Mx+b2a9Uh1QYzRlShggq0mT8DxlSuL1Pv881ATWrQtDFx98cDqjFJFcVW0fgbv/xMy+A4wgnObpwP3AI+6+KdUBNnRTppS/1eSKFWEayp9bvGFD6BP46KOQBNSmLyLpklSTj7tvBB4HpgKdgNOBeWb2XymMrVEYN678/YYhTI8bV376tNPC2ChPPKHxfkQkvZLpIxhiZn8DXgJ2Awa4+0lAb8J1AFKFjz+uev7WreFK33/+M4xlf/LJ6YtNRASSO330TOA2d38lfqa7bzazn6YmrMZjv/3K30wjfv727WHwspkzoagojP8jIpJuyTQNjQfeik2YWUszywdw99kpiaoRmTAh3F4xXqtWcOONMHp0GENo4sQw7r2ISCYkkwgeA+JHA90ezZMkFBaGX/uxuzl16wZ/+Uu4Wftf/wpXXw1jx2Y6ShHJZckkgqbuvjU2Eb3WoMU1UFgYhqPdsSM8l5SE+wBceilcf32GgxORnJdMIlhrZkNiE2Y2FPg8dSE1bn/6E1xzTegb+OMfdVcrEcm8ZDqLRwNTzOxOwICVgEaxr4UHH4TLLoMf/SjcLD0VN58QEampZC4o+xD4DzNrHU1/lfKoGqG//z3cG/j442Hq1F3vXyoikilJFUdmdgpwCNDCorYMd1frdpLeeQdGjAj3Gv3b36B580xHJCJSJpkLyv6XMN7QfxGahs4CuqU4rkZj7VoYMiTcUObpp6F160xHJCJSXjKt1IPc/TzgS3e/DhgI9ExtWI3D1q1w5pnw6achCXTunOmIRER2lUzT0JboebOZ7QusI4w3JFVwhzFj4NVX4ZFH4PDDMx2RiEhiySSCZ8ysLTARmEe4t/A9KY2qEfjjH8OZQVdfDcOHZzoaEZHKVZkIohvSzHb39cATZvZ3oIW7b0hLdA3UzJnwy1/CGWfAdddlOhoRkapV2Ufg7juAu+Kmv1USqNp774UawKGHhusGdK2AiGS7ZIqp2WZ2ppmuga3OunXhvgItW8L06bD77pmOSESkesn0Efwc+H9AqZltIZxC6u7+nZRG1sBs2wY//jGsXAkvvRSGmRYRaQiSubK4TToCaeh+8Qt44QWYNAkGDsx0NCIiyas2EZjZUYnmV7xRTS7785/h7rvhyivhPI3CJCINTDJNQ7+Ke90CGADMBf4zJRE1MLNmhYHkTjsN/ud/Mh2NiEjNJdM0dFr8tJl1BW5PWUQNyNKlcNZZcNBBMGUK5OVlOiIRkZqrzcmNq4CD6juQhmb9+lALaNo0nCHURj0pItJAJdNH8CfC1cQQEkcfwhXGOau0NNxofvny0DTUvXumIxIRqb1k+giK416XAo+4+79SFE+DMHYsPPdcuOfwUQm70kVEGo5kEsHjwBZ33w5gZnlm1srdN6c2tOx0zz1hHKHLL4ef/jTT0YiI1F1SVxYDLeOmWwKzUhNOdnv5Zbj4Yhg8GCZOzHQ0IiL1I5lE0CL+9pTR61apCyk7lZSEewsccIBuNSkijUsyieBrM+sXmzCz/sA3qQspO40dC99+G84Q2mOPTEcjIlJ/kkkElwOPmdmrZvZPYBpwaWrDyi433ghPPAFffQUnnBCuGRARaSySuaBsjpn1Ag6MZi1x922pDSt7TJkCv/1t2fSKFTBqVHhdWJiZmERE6lMyN6+/BNjd3d9193eB1mZ2cTI7N7PBZrbEzJaZ2VUJlt9mZvOjxwdmtr7mHyG1xo6FHTvKz9u8GcaNy0w8IiL1LZmmoZ9FdygDwN2/BH5W3UZmlke4qc1JwMHACDM7OH4dd7/C3fu4ex/gT8CTNQk+HT79NPH8jz9ObxwiIqmSTCLIi78pTVTAN0tiuwHAMndf7u5bganA0CrWHwE8ksR+0+aFFypfpvsNiEhjkUwi+AcwzcyOM7PjCIX1zCS26wysjJteFc3bhZl1A7oDCYteMxtlZsVmVrx27dok3rru3MON5/fcM9xxLF6rVjBhQlrCEBFJuWQSwa8JBfTo6PEO5S8wqw/DgcdjVy9X5O5F7l7g7gUdOnSo57dObOZMeP11+N3vwtXE3bqBWXguKlJHsYg0HsmcNbTDzN4Evgv8GGgPPJHEvlcDXeOmu0TzEhkOXJLEPtMiVhvo3h0uuACaNVPBLyKNV6WJwMx6EtrtRwCfE64fwN2PTXLfc4AeZtadkACGA+ckeJ9ewJ7A6zWKPIWefBLefjvcdrJZMr0hIiINWFVNQ+8T7kJ2qrt/393/BCRsuknE3UsJF549C7wHPOrui8zsejMbErfqcGCqu3ui/aTb9u1w7bXQq5dqASKSG6pqGjqDUEi/aGb/IJz1Y1Wsvwt3nwHMqDDv2grT42uyz1SbOhUWL4Zp03THMRHJDZXWCNz9KXcfDvQCXiQMNdHRzO42sx+mK8B02rYtXEXcuzcMG5bpaERE0qPas4bc/Wt3fzi6d3EX4G3CmUSNzqRJ8OGHcMMN0KQ2N/EUEWmALEua5pNWUFDgxcXF1a9YQ99+Cz16QKdO8MYb4VRREZHGwszmuntBomUaVT9SVAQrV8J99ykJiEhuUQMIYRC5CRPg6KPhuOMyHY2ISHqpRgDcdResWQOPP67agIjknpyvEWzcCDfdFO5D/P3vZzoaEZH0y/lEcPvt8MUX4UwhEZFclNOJ4Isv4NZb4Uc/goKEfekiIo1fTieCiRNh0ya4/vpMRyIikjk5mwjWrIE77oDhw+HQQzMdjYhI5uRsIrjppnAR2fjxmY5ERCSzcjIRrFoFd98N558PPXtmOhoRkczKyURw442wYwdcc02mIxERybycSwTLl8O998LPfgb5+ZmORkQk83IuEVx3HTRtCuPGZToSEZHskFOJ4L33YPJkuOQS2HffTEcjIpIdcioRjB8PrVrBrxvl3RRERGonZxLB/Pnw6KNw+eXQoUOmoxERyR45kwhmzYI994Rf/jLTkYiIZJecSQRjx8LSpdC2baYjERHJLjmTCADatct0BCIi2SenEoGIiOxKiUBEJMcpEYiI5DglAhGRHKdEICKS45QIRERynBKBiEiOUyIQEclxSgQiIjlOiUBEJMcpEYiI5DglAhGRHKdEICKS45QIRERyXEoTgZkNNrMlZrbMzK6qZJ0fm9liM1tkZg+nMh4REdlV01Tt2MzygLuAE4BVwBwzm+7ui+PW6QH8BjjS3b80s46pikdERBJLZY1gALDM3Ze7+1ZgKjC0wjo/A+5y9y8B3P2zFMYjIiIJpDIRdAZWxk2viubF6wn0NLN/mdkbZjY40Y7MbJSZFZtZ8dq1a1MUrohIbsp0Z3FToAdwDDACuMfMdrmrsLsXuXuBuxd06NAhzSGKiDRuqUwEq4GucdNdonnxVgHT3X2bu38EfEBIDCIikiapTARzgB5m1t3MmgHDgekV1nmKUBvAzNoTmoqWpzAmERGpIGWJwN1LgUuBZ4H3gEfdfZGZXW9mQ6LVngXWmdli4EXgV+6+LlUxiYjIrszdMx1DjRQUFHhxcXGmwxARaVDMbK67FyRalunOYhERyTAlAhGRHKdEICKS41I2xISIND7btm1j1apVbNmyJdOhSCVatGhBly5d2G233ZLeRolARJK2atUq2rRpQ35+PmaW6XCkAndn3bp1rFq1iu7duye9nZqGRCRpW7ZsoV27dkoCWcrMaNeuXY1rbEoEIlIjSgLZrTZ/HyUCEZEcp0QgIikzZQrk50OTJuF5ypS67W/dunX06dOHPn36sM8++9C5c+ed01u3bq1y2+LiYi677LJq32PQoEF1C7IBUmexiKTElCkwahRs3hymV6wI0wCFhbXbZ7t27Zg/fz4A48ePp3Xr1owdO3bn8tLSUpo2TVysFRQUUFCQ8MLacl577bXaBdeAqUYgIikxblxZEojZvDnMr08jR45k9OjRHHHEEVx55ZW89dZbDBw4kL59+zJo0CCWLFkCwEsvvcSpp54KhCRy4YUXcswxx7D//vtzxx137Nxf69atd65/zDHHMGzYMHr16kVhYSGxIXlmzJhBr1696N+/P5dddtnO/cYrKSnhBz/4Af369aNfv37lEszNN9/MoYceSu/evbnqqnAX32XLlnH88cfTu3dv+vXrx4cffli/B6oKqhGISEp8/HHN5tfFqlWreO2118jLy2Pjxo28+uqrNG3alFmzZvHf//3fPPHEE7ts8/777/Piiy+yadMmDjzwQMaMGbPLufdvv/02ixYtYt999+XII4/kX//6FwUFBfz85z/nlVdeoXv37owYMSJhTB07duT555+nRYsWLF26lBEjRlBcXMzMmTN5+umnefPNN2nVqhVffPEFAIWFhVx11VWcfvrpbNmyhR07dtT/gaqEEoGIpMR++4XmoB3mXTsAAA4oSURBVETz69tZZ51FXl4eABs2bOD8889n6dKlmBnbtm1LuM0pp5xC8+bNad68OR07dmTNmjV06dKl3DoDBgzYOa9Pnz6UlJTQunVr9t9//53n6Y8YMYKioqJd9r9t2zYuvfRS5s+fT15eHh988AEAs2bN4oILLqBVq1YA7LXXXmzatInVq1dz+umnA+GisHRS05CIpMSECRCVdTu1ahXm17fdd9995+trrrmGY489lnfffZdnnnmm0nPqmzdvvvN1Xl4epaWltVqnMrfddht77703CxYsoLi4uNrO7ExSIhCRlCgshKIi6NYNzMJzUVHtO4qTtWHDBjp3DrdHf+CBB+p9/wceeCDLly+npKQEgGnTplUaR6dOnWjSpAkPPfQQ27dvB+CEE07g/vvvZ3PUgfLFF1/Qpk0bunTpwlNPPQXAt99+u3N5OigRiEjKFBZCSQns2BGeU50EAK688kp+85vf0Ldv3xr9gk9Wy5Yt+fOf/8zgwYPp378/bdq0YY899thlvYsvvphJkybRu3dv3n///Z21lsGDBzNkyBAKCgro06cPt9xyCwAPPfQQd9xxB4cddhiDBg3i008/rffYK6Mb04hI0t577z0OOuigTIeRcV999RWtW7fG3bnkkkvo0aMHV1xxRabD2inR30k3phERqUf33HMPffr04ZBDDmHDhg38/Oc/z3RIdaKzhkREauiKK67IqhpAXalGICKS45QIRERynBKBiEiOUyIQEclxSgQi0mAce+yxPPvss+Xm3X777YwZM6bSbY455hhip5yffPLJrF+/fpd1xo8fv/N8/so89dRTLF68eOf0tddey6xZs2oSftZSIhCRBmPEiBFMnTq13LypU6dWOvBbRTNmzKBt27a1eu+KieD666/n+OOPr9W+so1OHxWRWrn8cohuDVBv+vSB22+vfPmwYcO4+uqr2bp1K82aNaOkpIR///vf/OAHP2DMmDHMmTOHb775hmHDhnHdddftsn1+fj7FxcW0b9+eCRMmMGnSJDp27EjXrl3p378/EK4RKCoqYuvWrRxwwAE89NBDzJ8/n+nTp/Pyyy9z44038sQTT3DDDTdw6qmnMmzYMGbPns3YsWMpLS3l8MMP5+6776Z58+bk5+dz/vnn88wzz7Bt2zYee+wxevXqVS6mkpISzj33XL7++msA7rzzzp03x7n55puZPHkyTZo04aSTTuKmm25i2bJljB49mrVr15KXl8djjz3Gd7/73Todd9UIRKTB2GuvvRgwYAAzZ84EQm3gxz/+MWbGhAkTKC4uZuHChbz88sssXLiw0v3MnTuXqVOnMn/+fGbMmMGcOXN2LjvjjDOYM2cOCxYs4KCDDuLee+9l0KBBDBkyhIkTJzJ//vxyBe+WLVsYOXIk06ZN45133qG0tJS777575/L27dszb948xowZk7D5KTZc9bx585g2bdrOu6jFD1e9YMECrrzySiAMV33JJZewYMECXnvtNTp16lS3g4pqBCJSS1X9ck+lWPPQ0KFDmTp1Kvfeey8Ajz76KEVFRZSWlvLJJ5+wePFiDjvssIT7ePXVVzn99NN3DgU9ZMiQncveffddrr76atavX89XX33FiSeeWGU8S5YsoXv37vTs2ROA888/n7vuuovLL78cCIkFoH///jz55JO7bJ8Nw1XnRI2gvu+bKiKZM3ToUGbPns28efPYvHkz/fv356OPPuKWW25h9uzZLFy4kFNOOaXS4aerM3LkSO68807eeecdfvvb39Z6PzGxoawrG8Y6G4arbvSJIHbf1BUrwL3svqlKBiINU+vWrTn22GO58MILd3YSb9y4kd1335099tiDNWvW7Gw6qsxRRx3FU089xTfffMOmTZt45plndi7btGkTnTp1Ytu2bUyJKyjatGnDpk2bdtnXgQceSElJCcuWLQPCKKJHH3100p8nG4arbvSJIF33TRWR9BkxYgQLFizYmQh69+5N37596dWrF+eccw5HHnlkldv369ePs88+m969e3PSSSdx+OGH71x2ww03cMQRR3DkkUeW69gdPnw4EydOpG/fvuXuJ9yiRQvuv/9+zjrrLA499FCaNGnC6NGjk/4s2TBcdaMfhrpJk1ATqMgsjJEuIsnTMNQNg4ahrqCy+6Om4r6pIiINUaNPBOm8b6qISEPU6BNBpu6bKtJYNbTm5FxTm79PTlxHUFiogl+kPrRo0YJ169bRrl07zCzT4UgF7s66detqfH1BShOBmQ0G/gjkAX9195sqLB8JTARWR7PudPe/pjImEam9Ll26sGrVKtauXZvpUKQSLVq0oEuXLjXaJmWJwMzygLuAE4BVwBwzm+7uiyusOs3dL01VHCJSf3bbbTe6d++e6TCknqWyj2AAsMzdl7v7VmAqMDSF7yciIrWQykTQGVgZN70qmlfRmWa20MweN7OuiXZkZqPMrNjMilUlFRGpX5k+a+gZIN/dDwOeByYlWsndi9y9wN0LOnTokNYARUQau1R2Fq8G4n/hd6GsUxgAd18XN/lX4PfV7XTu3Lmfm9mKeomw/rUHPs90EFVQfHWT7fFB9seo+OqmLvF1q2xBKhPBHKCHmXUnJIDhwDnxK5hZJ3f/JJocArxX3U7dPWurBGZWXNkl3NlA8dVNtscH2R+j4qubVMWXskTg7qVmdinwLOH00fvcfZGZXQ8Uu/t04DIzGwKUAl8AI1MVj4iIJJbS6wjcfQYwo8K8a+Ne/wb4TSpjEBGRqmW6s7ixKcp0ANVQfHWT7fFB9seo+OomJfE1uGGoRUSkfqlGICKS45QIRERynBJBDZlZVzN70cwWm9kiM/tFgnWOMbMNZjY/elybaF8pjLHEzN6J3nuX27lZcIeZLYuu6u6XxtgOjDsu881so5ldXmGdtB8/M7vPzD4zs3fj5u1lZs+b2dLoec9Ktj0/WmepmZ2fptgmmtn70d/vb2bWtpJtq/wupDjG8Wa2Ou7veHIl2w42syXR9/GqNMY3LS62EjObX8m2KT2GlZUpaf3+ubseNXgAnYB+0es2wAfAwRXWOQb4ewZjLAHaV7H8ZGAmYMB/AG9mKM484FOgW6aPH3AU0A94N27e74GrotdXATcn2G4vYHn0vGf0es80xPZDoGn0+uZEsSXzXUhxjOOBsUl8Bz4E9geaAQsq/j+lKr4Ky28Frs3EMaysTEnn9081ghpy90/cfV70ehPhIrhEYyhls6HAgx68AbQ1s04ZiOM44EN3z/iV4u7+CuFalnhDKRv2ZBLwowSbngg87+5fuPuXhKFSBqc6Nnd/zt1Lo8k3CFfuZ0wlxy8ZaRmcsqr4LNxY4cfAI/X9vsmookxJ2/dPiaAOzCwf6Au8mWDxQDNbYGYzzeyQtAYGDjxnZnPNbFSC5ckOCJhqw6n8ny+Txy9mby+78v1TYO8E62TDsbyQUMNLpLrvQqpdGjVf3VdJ00Y2HL8fAGvcfWkly9N2DCuUKWn7/ikR1JKZtQaeAC53940VFs8jNHf0Bv4EPJXm8L7v7v2Ak4BLzOyoNL9/tcysGWFYkccSLM708duFh3p41p1rbWbjCFfmT6lklUx+F+4Gvgv0AT4hNL9koxFUXRtIyzGsqkxJ9fdPiaAWzGw3wh9sirs/WXG5u29096+i1zOA3cysfbric/fV0fNnwN8I1e941Q4ImAYnAfPcfU3FBZk+fnHWxJrMoufPEqyTsWNp4Q5/pwKFUUGxiyS+Cynj7mvcfbu77wDuqeS9M/pdNLOmwBnAtMrWSccxrKRMSdv3T4mghqL2xHuB99z9D5Wss0+0HmY2gHCc1yVaNwXx7W5mbWKvCZ2K71ZYbTpwXnT20H8AG+KqoOlS6a+wTB6/CqYDsbMwzgeeTrDOs8APzWzPqOnjh9G8lLJwG9grgSHuvrmSdZL5LqQyxvh+p9Mree+dg1NGtcThhOOeLscD77v7qkQL03EMqyhT0vf9S1VPeGN9AN8nVNEWAvOjx8nAaGB0tM6lwCLCGRBvAIPSGN/+0fsuiGIYF82Pj88ItxH9EHgHKEjzMdydULDvETcvo8ePkJQ+AbYR2ll/CrQDZgNLgVnAXtG6BYR7cMe2vRBYFj0uSFNsywhtw7Hv4P9G6+4LzKjqu5DG4/dQ9P1aSCjUOlWMMZo+mXCmzIepijFRfNH8B2Lfu7h103oMqyhT0vb90xATIiI5Tk1DIiI5TolARCTHKRGIiOQ4JQIRkRynRCAikuOUCEQiZrbdyo+MWm8jYZpZfvzIlyLZJKX3LBZpYL5x9z6ZDkIk3VQjEKlGNB7976Mx6d8yswOi+flm9kI0qNpsM9svmr+3hXsELIgeg6Jd5ZnZPdGY88+ZWcto/cuisegXmtnUDH1MyWFKBCJlWlZoGjo7btkGdz8UuBO4PZr3J2CSux9GGPTtjmj+HcDLHgbN60e4IhWgB3CXux8CrAfOjOZfBfSN9jM6VR9OpDK6slgkYmZfuXvrBPNLgP909+XR4GCfuns7M/ucMGzCtmj+J+7e3szWAl3c/du4feQTxo3vEU3/GtjN3W80s38AXxFGWX3KowH3RNJFNQKR5Hglr2vi27jX2ynrozuFMPZTP2BONCKmSNooEYgk5+y459ej168RRssEKARejV7PBsYAmFmeme1R2U7NrAnQ1d1fBH4N7AHsUisRSSX98hAp09LK38D8H+4eO4V0TzNbSPhVPyKa91/A/Wb2K2AtcEE0/xdAkZn9lPDLfwxh5MtE8oDJUbIw4A53X19vn0gkCeojEKlG1EdQ4O6fZzoWkVRQ05CISI5TjUBEJMepRiAikuOUCEREcpwSgYhIjlMiEBHJcUoEIiI57v8DX3eysbfaES0AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.clf() # 그래프를 초기화합니다\n", "\n", "acc = history.history['accuracy']\n", "val_acc = history.history['val_accuracy']\n", "\n", "plt.plot(epochs, acc, 'bo', label='Training acc')\n", "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n", "plt.title('Training and validation accuracy')\n", "plt.xlabel('Epochs')\n", "plt.ylabel('Accuracy')\n", "plt.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이 모델은 9번째 에포크 이후에 과대적합이 시작됩니다. 9번의 에포크로 새로운 모델을 훈련하고 테스트 세트에서 평가하겠습니다:" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/9\n", "16/16 [==============================] - 1s 30ms/step - loss: 2.5010 - accuracy: 0.5120 - val_loss: 1.6078 - val_accuracy: 0.6510\n", "Epoch 2/9\n", "16/16 [==============================] - 0s 22ms/step - loss: 1.3421 - accuracy: 0.7160 - val_loss: 1.2406 - val_accuracy: 0.7120\n", "Epoch 3/9\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.9974 - accuracy: 0.7863 - val_loss: 1.0940 - val_accuracy: 0.7560\n", "Epoch 4/9\n", "16/16 [==============================] - 0s 20ms/step - loss: 0.7861 - accuracy: 0.8314 - val_loss: 1.0324 - val_accuracy: 0.7650\n", "Epoch 5/9\n", "16/16 [==============================] - 0s 20ms/step - loss: 0.6242 - accuracy: 0.8659 - val_loss: 0.9605 - val_accuracy: 0.7900\n", "Epoch 6/9\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.4980 - accuracy: 0.8928 - val_loss: 0.9147 - val_accuracy: 0.7970\n", "Epoch 7/9\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.4009 - accuracy: 0.9153 - val_loss: 0.8977 - val_accuracy: 0.8180\n", "Epoch 8/9\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.3285 - accuracy: 0.9320 - val_loss: 0.9312 - val_accuracy: 0.8010\n", "Epoch 9/9\n", "16/16 [==============================] - 0s 19ms/step - loss: 0.2728 - accuracy: 0.9379 - val_loss: 0.8983 - val_accuracy: 0.8060\n", "71/71 [==============================] - 0s 2ms/step - loss: 0.9999 - accuracy: 0.7854\n" ] } ], "source": [ "model = models.Sequential()\n", "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n", "model.add(layers.Dense(64, activation='relu'))\n", "model.add(layers.Dense(46, activation='softmax'))\n", "\n", "model.compile(optimizer='rmsprop',\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])\n", "model.fit(partial_x_train,\n", " partial_y_train,\n", " epochs=9,\n", " batch_size=512,\n", " validation_data=(x_val, y_val))\n", "results = model.evaluate(x_test, one_hot_test_labels)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0.9998643398284912, 0.7853962779045105]" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "대략 78%의 정확도를 달성했습니다. 균형 잡힌 이진 분류 문제에서 완전히 무작위로 분류하면 50%의 정확도를 달성합니다. 이 문제는 불균형한 데이터셋을 사용하므로 무작위로 분류하면 19% 정도를 달성합니다. 여기에 비하면 이 결과는 꽤 좋은 편입니다:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.18967052537845058" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import copy\n", "\n", "test_labels_copy = copy.copy(test_labels)\n", "np.random.shuffle(test_labels_copy)\n", "float(np.sum(np.array(test_labels) == np.array(test_labels_copy))) / len(test_labels)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 새로운 데이터에 대해 예측하기\n", "\n", "모델 인스턴스의 `predict` 메서드는 46개 토픽에 대한 확률 분포를 반환합니다. 테스트 데이터 전체에 대한 토픽을 예측해 보겠습니다:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "predictions = model.predict(x_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`predictions`의 각 항목은 길이가 46인 벡터입니다:" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(46,)" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "predictions[0].shape" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이 벡터의 원소 합은 1입니다:" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.99999994" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.sum(predictions[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "가장 큰 값이 예측 클래스가 됩니다. 즉, 가장 확률이 높은 클래스입니다:" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "3" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.argmax(predictions[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 레이블과 손실을 다루는 다른 방법\n", "\n", "앞서 언급한 것처럼 레이블을 인코딩하는 다른 방법은 다음과 같이 정수 텐서로 변환하는 것입니다:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "y_train = np.array(train_labels)\n", "y_test = np.array(test_labels)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이 방식을 사용하려면 손실 함수 하나만 바꾸면 됩니다. 코드 3-21에 사용된 손실 함수 `categorical_crossentropy`는 레이블이 범주형 인코딩되어 있을 것이라고 기대합니다. 정수 레이블을 사용할 때는 `sparse_categorical_crossentropy`를 사용해야 합니다:" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "model.compile(optimizer='rmsprop', loss='sparse_categorical_crossentropy', metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이 손실 함수는 인터페이스만 다를 뿐이고 수학적으로는 `categorical_crossentropy`와 동일합니다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 충분히 큰 중간층을 두어야 하는 이유\n", "\n", "앞서 언급한 것처럼 마지막 출력이 46차원이기 때문에 중간층의 히든 유닛이 46개보다 많이 적어서는 안 됩니다. 46차원보다 훨씬 작은 중간층(예를 들면 4차원)을 두면 정보의 병목이 어떻게 나타나는지 확인해 보겠습니다." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/20\n", "63/63 [==============================] - 1s 11ms/step - loss: 3.3646 - accuracy: 0.0971 - val_loss: 2.8813 - val_accuracy: 0.2550\n", "Epoch 2/20\n", "63/63 [==============================] - 1s 8ms/step - loss: 2.3801 - accuracy: 0.3186 - val_loss: 1.9828 - val_accuracy: 0.5790\n", "Epoch 3/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.7491 - accuracy: 0.5796 - val_loss: 1.6941 - val_accuracy: 0.5860\n", "Epoch 4/20\n", "63/63 [==============================] - 1s 8ms/step - loss: 1.5250 - accuracy: 0.6007 - val_loss: 1.6080 - val_accuracy: 0.5830\n", "Epoch 5/20\n", "63/63 [==============================] - 1s 8ms/step - loss: 1.4201 - accuracy: 0.6156 - val_loss: 1.5845 - val_accuracy: 0.5910\n", "Epoch 6/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.3555 - accuracy: 0.6230 - val_loss: 1.5871 - val_accuracy: 0.5920\n", "Epoch 7/20\n", "63/63 [==============================] - 0s 7ms/step - loss: 1.3057 - accuracy: 0.6329 - val_loss: 1.5993 - val_accuracy: 0.5970\n", "Epoch 8/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.2663 - accuracy: 0.6387 - val_loss: 1.6076 - val_accuracy: 0.6010\n", "Epoch 9/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.2319 - accuracy: 0.6461 - val_loss: 1.6537 - val_accuracy: 0.5990\n", "Epoch 10/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.2008 - accuracy: 0.6478 - val_loss: 1.6804 - val_accuracy: 0.5980\n", "Epoch 11/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.1736 - accuracy: 0.6562 - val_loss: 1.7083 - val_accuracy: 0.6060\n", "Epoch 12/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.1494 - accuracy: 0.6597 - val_loss: 1.7064 - val_accuracy: 0.6030\n", "Epoch 13/20\n", "63/63 [==============================] - 1s 8ms/step - loss: 1.1249 - accuracy: 0.6615 - val_loss: 1.7594 - val_accuracy: 0.6060\n", "Epoch 14/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.1070 - accuracy: 0.6679 - val_loss: 1.8417 - val_accuracy: 0.5970\n", "Epoch 15/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.0878 - accuracy: 0.6704 - val_loss: 1.8655 - val_accuracy: 0.6140\n", "Epoch 16/20\n", "63/63 [==============================] - 1s 8ms/step - loss: 1.0705 - accuracy: 0.6867 - val_loss: 1.8595 - val_accuracy: 0.6080\n", "Epoch 17/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.0523 - accuracy: 0.6966 - val_loss: 1.8809 - val_accuracy: 0.6140\n", "Epoch 18/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.0395 - accuracy: 0.7001 - val_loss: 1.9981 - val_accuracy: 0.6080\n", "Epoch 19/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.0271 - accuracy: 0.7043 - val_loss: 2.0545 - val_accuracy: 0.6060\n", "Epoch 20/20\n", "63/63 [==============================] - 0s 8ms/step - loss: 1.0154 - accuracy: 0.7161 - val_loss: 2.0268 - val_accuracy: 0.6090\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model = models.Sequential()\n", "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n", "model.add(layers.Dense(4, activation='relu'))\n", "model.add(layers.Dense(46, activation='softmax'))\n", "\n", "model.compile(optimizer='rmsprop',\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])\n", "model.fit(partial_x_train,\n", " partial_y_train,\n", " epochs=20,\n", " batch_size=128,\n", " validation_data=(x_val, y_val))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "검증 정확도의 최고 값은 약 71%로 8% 정도 감소되었습니다. 이런 손실의 대부분 원인은 많은 정보(46개 클래스의 분할 초평면을 복원하기에 충분한 정보)를 중간층의 저차원 표현 공간으로 압축하려고 했기 때문입니다. 이 네트워크는 필요한 정보 대부분을 4차원 표현 안에 구겨 넣었지만 전부는 넣지 못했습니다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 추가 실험\n", "\n", "* 더 크거나 작은 층을 사용해 보세요: 32개 유닛, 128개 유닛 등\n", "* 여기에서 두 개의 은닉층을 사용했습니다. 한 개의 은닉층이나 세 개의 은닉층을 사용해 보세요." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 정리\n", "\n", "다음은 이 예제에서 배운 것들입니다.\n", "\n", "* N개의 클래스로 데이터 포인트를 분류하려면 네트워크의 마지막 `Dense` 층의 크기는 N이어야 합니다.\n", "* 단일 레이블, 다중 분류 문제에서는 N개의 클래스에 대한 확률 분포를 출력하기 위해 `softmax` 활성화 함수를 사용해야 합니다.\n", "* 이런 문제에는 항상 범주형 크로스엔트로피를 사용해야 합니다. 이 함수는 모델이 출력한 확률 분포와 타깃 분포 사이의 거리를 최소화합니다.\n", "* 다중 분류에서 레이블을 다루는 두 가지 방법이 있습니다.\n", " * 레이블을 범주형 인코딩(또는 원-핫 인코딩)으로 인코딩하고 `categorical_crossentropy` 손실 함수를 사용합니다.\n", " * 레이블을 정수로 인코딩하고 `sparse_categorical_crossentropy` 손실 함수를 사용합니다.\n", "* 많은 수의 범주를 분류할 때 중간층의 크기가 너무 작아 네트워크에 정보의 병목이 생기지 않도록 해야 합니다." ] } ], "metadata": { "kernelspec": { "display_name": "TensorFlow 2.4 on Python 3.8 & CUDA 11.1", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" } }, "nbformat": 4, "nbformat_minor": 2 }