{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Using TensorFlow backend.\n" ] }, { "data": { "text/plain": [ "'2.2.4'" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import keras\n", "keras.__version__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 영화 리뷰 분류: 이진 분류 예제\n", "\n", "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/케라스-창시자에게-배우는-딥러닝/) 책의 3장 4절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다. 이 노트북의 설명은 케라스 버전 2.2.2에 맞추어져 있습니다. 케라스 최신 버전이 릴리스되면 노트북을 다시 테스트하기 때문에 설명과 코드의 결과가 조금 다를 수 있습니다.\n", "\n", "----\n", "\n", "2종 분류 또는 이진 분류는 아마도 가장 널리 적용된 머신 러닝 문제일 것입니다. 이 예제에서 리뷰 텍스트를 기반으로 영화 리뷰를 긍정과 부정로 분류하는 법을 배우겠습니다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## IMDB 데이터셋\n", "\n", "인터넷 영화 데이터베이스로부터 가져온 양극단의 리뷰 50,000개로 이루어진 IMDB 데이터셋을 사용하겠습니다. 이 데이터셋은 훈련 데이터 25,000개와 테스트 데이터 25,000개로 나뉘어 있고 각각 50%는 부정, 50%는 긍정 리뷰로 구성되어 있습니다.\n", "\n", "왜 훈련 데이터와 테스트 데이터를 나눌까요? 같은 데이터에서 머신 러닝 모델을 훈련하고 테스트해서는 절대 안 되기 때문입니다! 모델이 훈련 데이터에서 잘 작동한다는 것이 처음 만난 데이터에서도 잘 동작한다는 것을 보장하지 않습니다. 중요한 것은 새로운 데이터에 대한 모델의 성능입니다(사실 훈련 데이터의 레이블은 이미 알고 있기 때문에 이를 예측하는 모델은 필요하지 않습니다). 예를 들어 모델이 훈련 샘플과 타깃 사이의 매핑을 모두 외워버릴 수 있습니다. 이런 모델은 처음 만나는 데이터에서 타깃을 예측하는 작업에는 쓸모가 없습니다. 다음 장에서 이에 대해 더 자세히 살펴보겠습니다.\n", "\n", "MNIST 데이터셋처럼 IMDB 데이터셋도 케라스에 포함되어 있습니다. 이 데이터는 전처리되어 있어 각 리뷰(단어 시퀀스)가 숫자 시퀀스로 변환되어 있습니다. 여기서 각 숫자는 사전에 있는 고유한 단어를 나타냅니다.\n", "\n", "다음 코드는 데이터셋을 로드합니다(처음 실행하면 17MB 정도의 데이터가 컴퓨터에 다운로드됩니다):" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from keras.datasets import imdb\n", "\n", "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "매개변수 `num_words=10000`은 훈련 데이터에서 가장 자주 나타나는 단어 10,000개만 사용하겠다는 의미입니다. 드물게 나타나는 단어는 무시하겠습니다. 이렇게 하면 적절한 크기의 벡터 데이터를 얻을 수 있습니다.\n", "\n", "변수 `train_data`와 `test_data`는 리뷰의 목록입니다. 각 리뷰는 단어 인덱스의 리스트입니다(단어 시퀀스가 인코딩된 것입니다). `train_labels`와 `test_labels`는 부정을 나타내는 0과 긍정을 나타내는 1의 리스트입니다:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1,\n", " 14,\n", " 22,\n", " 16,\n", " 43,\n", " 530,\n", " 973,\n", " 1622,\n", " 1385,\n", " 65,\n", " 458,\n", " 4468,\n", " 66,\n", " 3941,\n", " 4,\n", " 173,\n", " 36,\n", " 256,\n", " 5,\n", " 25,\n", " 100,\n", " 43,\n", " 838,\n", " 112,\n", " 50,\n", " 670,\n", " 2,\n", " 9,\n", " 35,\n", " 480,\n", " 284,\n", " 5,\n", " 150,\n", " 4,\n", " 172,\n", " 112,\n", " 167,\n", " 2,\n", " 336,\n", " 385,\n", " 39,\n", " 4,\n", " 172,\n", " 4536,\n", " 1111,\n", " 17,\n", " 546,\n", " 38,\n", " 13,\n", " 447,\n", " 4,\n", " 192,\n", " 50,\n", " 16,\n", " 6,\n", " 147,\n", " 2025,\n", " 19,\n", " 14,\n", " 22,\n", " 4,\n", " 1920,\n", " 4613,\n", " 469,\n", " 4,\n", " 22,\n", " 71,\n", " 87,\n", " 12,\n", " 16,\n", " 43,\n", " 530,\n", " 38,\n", " 76,\n", " 15,\n", " 13,\n", " 1247,\n", " 4,\n", " 22,\n", " 17,\n", " 515,\n", " 17,\n", " 12,\n", " 16,\n", " 626,\n", " 18,\n", " 2,\n", " 5,\n", " 62,\n", " 386,\n", " 12,\n", " 8,\n", " 316,\n", " 8,\n", " 106,\n", " 5,\n", " 4,\n", " 2223,\n", " 5244,\n", " 16,\n", " 480,\n", " 66,\n", " 3785,\n", " 33,\n", " 4,\n", " 130,\n", " 12,\n", " 16,\n", " 38,\n", " 619,\n", " 5,\n", " 25,\n", " 124,\n", " 51,\n", " 36,\n", " 135,\n", " 48,\n", " 25,\n", " 1415,\n", " 33,\n", " 6,\n", " 22,\n", " 12,\n", " 215,\n", " 28,\n", " 77,\n", " 52,\n", " 5,\n", " 14,\n", " 407,\n", " 16,\n", " 82,\n", " 2,\n", " 8,\n", " 4,\n", " 107,\n", " 117,\n", " 5952,\n", " 15,\n", " 256,\n", " 4,\n", " 2,\n", " 7,\n", " 3766,\n", " 5,\n", " 723,\n", " 36,\n", " 71,\n", " 43,\n", " 530,\n", " 476,\n", " 26,\n", " 400,\n", " 317,\n", " 46,\n", " 7,\n", " 4,\n", " 2,\n", " 1029,\n", " 13,\n", " 104,\n", " 88,\n", " 4,\n", " 381,\n", " 15,\n", " 297,\n", " 98,\n", " 32,\n", " 2071,\n", " 56,\n", " 26,\n", " 141,\n", " 6,\n", " 194,\n", " 7486,\n", " 18,\n", " 4,\n", " 226,\n", " 22,\n", " 21,\n", " 134,\n", " 476,\n", " 26,\n", " 480,\n", " 5,\n", " 144,\n", " 30,\n", " 5535,\n", " 18,\n", " 51,\n", " 36,\n", " 28,\n", " 224,\n", " 92,\n", " 25,\n", " 104,\n", " 4,\n", " 226,\n", " 65,\n", " 16,\n", " 38,\n", " 1334,\n", " 88,\n", " 12,\n", " 16,\n", " 283,\n", " 5,\n", " 16,\n", " 4472,\n", " 113,\n", " 103,\n", " 32,\n", " 15,\n", " 16,\n", " 5345,\n", " 19,\n", " 178,\n", " 32]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_data[0]" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_labels[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "가장 자주 등장하는 단어 10,000개로 제한했기 때문에 단어 인덱스는 10,000을 넘지 않습니다:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "9999" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "max([max(sequence) for sequence in train_data])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "재미 삼아 이 리뷰 데이터 하나를 원래 영어 단어로 어떻게 바꾸는지 보겠습니다:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# word_index는 단어와 정수 인덱스를 매핑한 딕셔너리입니다\n", "word_index = imdb.get_word_index()\n", "# 정수 인덱스와 단어를 매핑하도록 뒤집습니다\n", "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n", "# 리뷰를 디코딩합니다. \n", "# 0, 1, 2는 '패딩', '문서 시작', '사전에 없음'을 위한 인덱스이므로 3을 뺍니다\n", "decoded_review = ' '.join([reverse_word_index.get(i - 3, '?') for i in train_data[0]])" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"? this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\"" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "decoded_review" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 데이터 준비\n", "\n", "신경망에 숫자 리스트를 주입할 수는 없습니다. 리스트를 텐서로 바꾸는 두 가지 방법이 있습니다:\n", "\n", "* 같은 길이가 되도록 리스트에 패딩을 추가하고 `(samples, sequence_length)` 크기의 정수 텐서로 변환합니다. 그다음 이 정수 텐서를 다룰 수 있는 층을 신경망의 첫 번째 층으로 사용합니다(`Embedding` 층을 말하며 나중에 자세히 다루겠습니다).\n", "* 리스트를 원-핫 인코딩하여 0과 1의 벡터로 변환합니다. 예를 들면 시퀀스 `[3, 5]`를 인덱스 3과 5의 위치는 1이고 그 외는 모두 0인 10,000차원의 벡터로 각각 변환합니다. 그다음 부동 소수 벡터 데이터를 다룰 수 있는 `Dense` 층을 신경망의 첫 번째 층으로 사용합니다.\n", "\n", "여기서는 두 번째 방식을 사용하고 이해를 돕기 위해 직접 데이터를 원-핫 벡터로 만들겠습니다:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "\n", "def vectorize_sequences(sequences, dimension=10000):\n", " # 크기가 (len(sequences), dimension))이고 모든 원소가 0인 행렬을 만듭니다\n", " results = np.zeros((len(sequences), dimension))\n", " for i, sequence in enumerate(sequences):\n", " results[i, sequence] = 1. # results[i]에서 특정 인덱스의 위치를 1로 만듭니다\n", " return results\n", "\n", "# 훈련 데이터를 벡터로 변환합니다\n", "x_train = vectorize_sequences(train_data)\n", "# 테스트 데이터를 벡터로 변환합니다\n", "x_test = vectorize_sequences(test_data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이제 샘플은 다음과 같이 나타납니다:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0., 1., 1., ..., 0., 0., 0.])" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x_train[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "레이블은 쉽게 벡터로 바꿀 수 있습니다:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# 레이블을 벡터로 바꿉니다\n", "y_train = np.asarray(train_labels).astype('float32')\n", "y_test = np.asarray(test_labels).astype('float32')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이제 신경망에 주입할 데이터가 준비되었습니다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 신경망 모델 만들기\n", "\n", "입력 데이터가 벡터이고 레이블은 스칼라(1 또는 0)입니다. 아마 앞으로 볼 수 있는 문제 중에서 가장 간단할 것입니다. 이런 문제에 잘 작동하는 네트워크 종류는 `relu` 활성화 함수를 사용한 완전 연결 층(즉, `Dense(16, activation='relu')`)을 그냥 쌓은 것입니다.\n", "\n", "`Dense` 층에 전달한 매개변수(16)는 은닉 유닛의 개수입니다. 하나의 은닉 유닛은 층이 나타내는 표현 공간에서 하나의 차원이 됩니다. 2장에서 `relu` 활성화 함수를 사용한 `Dense` 층을 다음과 같은 텐서 연산을 연결하여 구현하였습니다:\n", "\n", "`output = relu(dot(W, input) + b)`\n", "\n", "16개의 은닉 유닛이 있다는 것은 가중치 행렬 `W`의 크기가 `(input_dimension, 16)`이라는 뜻입니다. 입력 데이터와 `W`를 점곱하면 입력 데이터가 16 차원으로 표현된 공간으로 투영됩니다(그리고 편향 벡터 `b`를 더하고 `relu` 연산을 적용합니다). 표현 공간의 차원을 '신경망이 내재된 표현을 학습할 때 가질 수 있는 자유도'로 이해할 수 있습니다. 은닉 유닛을 늘리면 (표현 공간을 더 고차원으로 만들면) 신경망이 더욱 복잡한 표현을 학습할 수 있지만 계산 비용이 커지고 원치 않은 패턴을 학습할 수도 있습니다(훈련 데이터에서는 성능이 향상되지만 테스트 데이터에서는 그렇지 않은 패턴입니다).\n", "\n", "`Dense` 층을 쌓을 때 두 가진 중요한 구조상의 결정이 필요합니다:\n", "\n", "* 얼마나 많은 층을 사용할 것인가\n", "* 각 층에 얼마나 많은 은닉 유닛을 둘 것인가\n", "\n", "4장에서 이런 결정을 하는 데 도움이 되는 일반적인 원리를 배우겠습니다. 당분간은 저를 믿고 선택한 다음 구조를 따라 주세요.\n", "\n", "* 16개의 은닉 유닛을 가진 두 개의 은닉층\n", "* 현재 리뷰의 감정을 스칼라 값의 예측으로 출력하는 세 번째 층\n", "\n", "중간에 있는 은닉층은 활성화 함수로 `relu`를 사용하고 마지막 층은 확률(0과 1 사이의 점수로, 어떤 샘플이 타깃 '1'일 가능성이 높다는 것은 그 리뷰가 긍정일 가능성이 높다는 것을 의미합니다)을 출력하기 위해 시그모이드 활성화 함수를 사용합니다. `relu`는 음수를 0으로 만드는 함수입니다. 시그모이드는 임의의 값을 [0, 1] 사이로 압축하므로 출력 값을 확률처럼 해석할 수 있습니다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "다음이 이 신경망의 모습입니다:\n", "\n", "![3-layer network](https://s3.amazonaws.com/book.keras.io/img/ch3/3_layer_network.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "다음은 이 신경망의 케라스 구현입니다. 이전에 보았던 MNIST 예제와 비슷합니다:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "from keras import models\n", "from keras import layers\n", "\n", "model = models.Sequential()\n", "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n", "model.add(layers.Dense(16, activation='relu'))\n", "model.add(layers.Dense(1, activation='sigmoid'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "마지막으로 손실 함수와 옵티마이저를 선택해야 합니다. 이진 분류 문제이고 신경망의 출력이 확률이기 때문에(네트워크의 끝에 시그모이드 활성화 함수를 사용한 하나의 유닛으로 된 층을 놓았습니다), `binary_crossentropy` 손실이 적합합니다. 이 함수가 유일한 선택은 아니고 예를 들어 `mean_squared_error`를 사용할 수도 있습니다. 확률을 출력하는 모델을 사용할 때는 크로스엔트로피가 최선의 선택입니다. 크로스엔트로피는 정보 이론 분야에서 온 개념으로 확률 분포 간의 차이를 측정합니다. 여기에서는 원본 분포와 예측 분포 사이를 측정합니다.\n", "\n", "다음은 `rmsprop` 옵티마이저와 `binary_crossentropy` 손실 함수로 모델을 설정하는 단계입니다. 훈련하는 동안 정확도를 사용해 모니터링하겠습니다." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "model.compile(optimizer='rmsprop',\n", " loss='binary_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "케라스에 `rmsprop`, `binary_crossentropy`, `accuracy`가 포함되어 있기 때문에 옵티마이저, 손실 함수, 측정 지표를 문자열로 지정하는 것이 가능합니다. 이따금 옵티마이저의 매개변수를 바꾸거나 자신만의 손실 함수, 측정 함수를 전달해야 할 경우가 있습니다. 전자의 경우에는 옵티마이저 파이썬 클래스를 사용해 객체를 직접 만들어 `optimizer` 매개변수에 전달하면 됩니다:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "from keras import optimizers\n", "\n", "model.compile(optimizer=optimizers.RMSprop(lr=0.001),\n", " loss='binary_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "후자의 경우는 `loss`와 `metrics` 매개변수에 함수 객체를 전달하면 됩니다:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "from keras import losses\n", "from keras import metrics\n", "\n", "model.compile(optimizer=optimizers.RMSprop(lr=0.001),\n", " loss=losses.binary_crossentropy,\n", " metrics=[metrics.binary_accuracy])" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "model.compile(optimizer='rmsprop',\n", " loss='binary_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 훈련 검증\n", "\n", "훈련하는 동안 처음 본 데이터에 대한 모델의 정확도를 측정하기 위해서는 원본 훈련 데이터에서 10,000의 샘플을 떼어서 검증 세트를 만들어야 합니다:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "x_val = x_train[:10000]\n", "partial_x_train = x_train[10000:]\n", "\n", "y_val = y_train[:10000]\n", "partial_y_train = y_train[10000:]" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "이제 모델을 512개 샘플씩 미니 배치를 만들어 20번의 에포크 동안 훈련시킵니다(`x_train`과 `y_train` 텐서에 있는 모든 샘플에 대해 20번 반복합니다). 동시에 따로 떼어 놓은 10,000개의 샘플에서 손실과 정확도를 측정할 것입니다. 이렇게 하려면 `validation_data` 매개변수에 검증 데이터를 전달해야 합니다:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train on 15000 samples, validate on 10000 samples\n", "Epoch 1/20\n", "15000/15000 [==============================] - 2s 116us/step - loss: 0.5084 - acc: 0.7813 - val_loss: 0.3797 - val_acc: 0.8684\n", "Epoch 2/20\n", "15000/15000 [==============================] - 1s 70us/step - loss: 0.3004 - acc: 0.9047 - val_loss: 0.3004 - val_acc: 0.8897\n", "Epoch 3/20\n", "15000/15000 [==============================] - 1s 65us/step - loss: 0.2179 - acc: 0.9285 - val_loss: 0.3085 - val_acc: 0.8711\n", "Epoch 4/20\n", "15000/15000 [==============================] - 1s 69us/step - loss: 0.1750 - acc: 0.9437 - val_loss: 0.2840 - val_acc: 0.8832\n", "Epoch 5/20\n", "15000/15000 [==============================] - 1s 67us/step - loss: 0.1427 - acc: 0.9543 - val_loss: 0.2841 - val_acc: 0.8872\n", "Epoch 6/20\n", "15000/15000 [==============================] - 1s 67us/step - loss: 0.1150 - acc: 0.9650 - val_loss: 0.3166 - val_acc: 0.8772\n", "Epoch 7/20\n", "15000/15000 [==============================] - 1s 67us/step - loss: 0.0980 - acc: 0.9705 - val_loss: 0.3127 - val_acc: 0.8846\n", "Epoch 8/20\n", "15000/15000 [==============================] - 1s 66us/step - loss: 0.0807 - acc: 0.9763 - val_loss: 0.3859 - val_acc: 0.8649\n", "Epoch 9/20\n", "15000/15000 [==============================] - 1s 67us/step - loss: 0.0661 - acc: 0.9821 - val_loss: 0.3635 - val_acc: 0.8782\n", "Epoch 10/20\n", "15000/15000 [==============================] - 1s 69us/step - loss: 0.0561 - acc: 0.9853 - val_loss: 0.3843 - val_acc: 0.8792\n", "Epoch 11/20\n", "15000/15000 [==============================] - 1s 68us/step - loss: 0.0439 - acc: 0.9893 - val_loss: 0.4153 - val_acc: 0.8779\n", "Epoch 12/20\n", "15000/15000 [==============================] - 1s 59us/step - loss: 0.0381 - acc: 0.9921 - val_loss: 0.4525 - val_acc: 0.8690\n", "Epoch 13/20\n", "15000/15000 [==============================] - 1s 59us/step - loss: 0.0300 - acc: 0.9928 - val_loss: 0.4698 - val_acc: 0.8729\n", "Epoch 14/20\n", "15000/15000 [==============================] - 1s 66us/step - loss: 0.0247 - acc: 0.9945 - val_loss: 0.5023 - val_acc: 0.8726\n", "Epoch 15/20\n", "15000/15000 [==============================] - 1s 68us/step - loss: 0.0175 - acc: 0.9979 - val_loss: 0.5342 - val_acc: 0.8693\n", "Epoch 16/20\n", "15000/15000 [==============================] - 1s 68us/step - loss: 0.0149 - acc: 0.9983 - val_loss: 0.5710 - val_acc: 0.8698\n", "Epoch 17/20\n", "15000/15000 [==============================] - 1s 69us/step - loss: 0.0151 - acc: 0.9971 - val_loss: 0.6025 - val_acc: 0.8697\n", "Epoch 18/20\n", "15000/15000 [==============================] - 1s 65us/step - loss: 0.0075 - acc: 0.9996 - val_loss: 0.6782 - val_acc: 0.8633\n", "Epoch 19/20\n", "15000/15000 [==============================] - 1s 62us/step - loss: 0.0117 - acc: 0.9975 - val_loss: 0.6693 - val_acc: 0.8673\n", "Epoch 20/20\n", "15000/15000 [==============================] - 1s 68us/step - loss: 0.0041 - acc: 0.9999 - val_loss: 0.6942 - val_acc: 0.8658\n" ] } ], "source": [ "history = model.fit(partial_x_train,\n", " partial_y_train,\n", " epochs=20,\n", " batch_size=512,\n", " validation_data=(x_val, y_val))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "CPU를 사용해도 에포크마다 2초가 걸리지 않습니다. 전체 훈련은 20초 이상 걸립니다. 에포크가 끝날 때마다 10,000개의 검증 샘플 데이터에서 손실과 정확도를 계산하기 때문에 약간씩 지연됩니다.\n", "\n", "`model.fit()` 메서드는 `History` 객체를 반환합니다. 이 객체는 훈련하는 동안 발생한 모든 정보를 담고 있는 딕셔너리인 `history` 속성을 가지고 있습니다. 한 번 확인해 보죠:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "dict_keys(['val_loss', 'val_acc', 'loss', 'acc'])" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "history_dict = history.history\n", "history_dict.keys()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "이 딕셔너리는 훈련과 검증하는 동안 모니터링할 측정 지표당 하나씩 모두 네 개의 항목을 담고 있습니다. 맷플롯립을 사용해 훈련과 검증 데이터에 대한 손실과 정확도를 그려 보겠습니다:" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xt4VNW5x/HvK6LIRUDA4wW5VbQCcjNFrCjeavEGaqmA4F1Rq61WbaVqWw+Vc1CpF5Ra0UqtRJHqQWiF0lqp1rYCARG5iCACRlADCoKgEnjPH2szDmGSTDLZmUny+zzPPJnZs2fvd/ZM9jtrrb3WMndHREQEYK9sByAiIrlDSUFERBKUFEREJEFJQUREEpQUREQkQUlBREQSlBSkSplZPTPbYmZtqnLdbDKzw82syq/dNrPTzGxV0uNlZnZCOutWYl+Pm9ltlX19Gdu9y8x+X9XblezZO9sBSHaZ2Zakhw2BL4Ed0eOr3T2/Ittz9x1A46pety5w9yOrYjtmdiUwzN1PStr2lVWxban9lBTqOHdPnJSjX6JXuvtLpa1vZnu7e3F1xCYi1U/VR1KmqHrgWTN7xsw2A8PM7Dgze93MNprZOjMba2b1o/X3NjM3s3bR44nR8zPMbLOZ/cfM2ld03ej5M8zsHTPbZGYPmdm/zOzSUuJOJ8arzWyFmX1qZmOTXlvPzO43sw1m9i7Qr4zjc4eZTSqxbJyZ3Rfdv9LMlkbv593oV3xp2yo0s5Oi+w3N7KkotsXAMSn2uzLa7mIz6x8tPxp4GDghqppbn3Rs70x6/TXRe99gZi+Y2cHpHJvymNm5UTwbzexlMzsy6bnbzGytmX1mZm8nvdfeZjY/Wv6Rmd2b7v4kBu6um264O8Aq4LQSy+4CvgLOIfyI2A/4FnAsoaTZAXgHuD5af2/AgXbR44nAeiAPqA88C0ysxLoHApuBAdFzNwHbgUtLeS/pxDgVaAq0Az7Z9d6B64HFQGugBfBq+FdJuZ8OwBagUdK2PwbyosfnROsYcAqwDegaPXcasCppW4XASdH9McA/gOZAW2BJiXUvAA6OPpMLoxj+K3ruSuAfJeKcCNwZ3T89irE70AD4DfByOscmxfu/C/h9dP+oKI5Tos/otui41wc6A6uBg6J12wMdovtzgSHR/SbAsdn+X6jLN5UUJB2vufuf3H2nu29z97nuPtvdi919JTAe6FvG659z9wJ33w7kE05GFV33bGCBu0+NnrufkEBSSjPG/3X3Te6+inAC3rWvC4D73b3Q3TcAo8vYz0pgESFZAXwH2OjuBdHzf3L3lR68DPwdSNmYXMIFwF3u/qm7ryb8+k/e72R3Xxd9Jk8TEnpeGtsFGAo87u4L3P0LYATQ18xaJ61T2rEpy2Bgmru/HH1Go4H9Ccm5mJCAOkdVkO9Fxw5Ccu9oZi3cfbO7z07zfUgMlBQkHe8nPzCzb5rZi2b2oZl9BowEWpbx+g+T7m+l7Mbl0tY9JDkOd3fCL+uU0owxrX0RfuGW5WlgSHT/QkIy2xXH2WY228w+MbONhF/pZR2rXQ4uKwYzu9TM3oyqaTYC30xzuxDeX2J77v4Z8ClwaNI6FfnMStvuTsJndKi7LwNuJnwOH0fVkQdFq14GdAKWmdkcMzszzfchMVBSkHSUvBzzUcKv48PdfX/gF4TqkTitI1TnAGBmxu4nsZIyiXEdcFjS4/IumX0WOC36pT2AkCQws/2A54D/JVTtNAP+mmYcH5YWg5l1AB4BrgVaRNt9O2m75V0+u5ZQJbVre00I1VQfpBFXRba7F+Ez+wDA3Se6+/GEqqN6hOOCuy9z98GEKsJfA8+bWYMMY5FKUlKQymgCbAI+N7OjgKurYZ9/Bnqa2TlmtjdwA9AqphgnAzea2aFm1gK4tayV3f0j4DVgArDM3ZdHT+0L7AMUATvM7Gzg1ArEcJuZNbPQj+P6pOcaE078RYT8eCWhpLDLR0DrXQ3rKTwDXGFmXc1sX8LJ+Z/uXmrJqwIx9zezk6J9/4TQDjTbzI4ys5Oj/W2LbjsIb+AiM2sZlSw2Re9tZ4axSCUpKUhl3AxcQviHf5TwSzlW0Yl3EHAfsAH4BvAGoV9FVcf4CKHu/y1CI+hzabzmaULD8dNJMW8EfgxMITTWDiQkt3T8klBiWQXMAP6QtN2FwFhgTrTON4Hkevi/AcuBj8wsuRpo1+v/QqjGmRK9vg2hnSEj7r6YcMwfISSsfkD/qH1hX+AeQjvQh4SSyR3RS88Ellq4um0MMMjdv8o0HqkcC1WzIjWLmdUjVFcMdPd/ZjsekdpCJQWpMcysn5k1jaogfk64omVOlsMSqVWUFKQm6QOsJFRB9APOdffSqo9EpBJUfSQiIgkqKYiISEKNGxCvZcuW3q5du2yHISJSo8ybN2+9u5d1GTdQA5NCu3btKCgoyHYYIiI1ipmV1zMfUPWRiIgkUVIQEZEEJQUREUmItU3BzPoBDxIGv3rc3UeXeP5+4OToYUPgwGhwrwrZvn07hYWFfPHFF5mGLNWgQYMGtG7dmvr1SxuaR0SyJbakEA1DMI4wvnwhMNfMprn7kl3ruPuPk9b/IdCjMvsqLCykSZMmtGvXjjB4puQqd2fDhg0UFhbSvn378l8gItUqzuqjXsCKaIKRr4BJfD0RSSpDCKM3VtgXX3xBixYtlBBqADOjRYsWKtWJ5Kg4k8Kh7D5JSCGljH9vZm0JY6y/XNmdKSHUHPqsRHJXnEkh1X9+aWNqDCZMw7gj5YbMhptZgZkVFBUVVVmAIiI1wYcfws9/DsuWxb+vOJNCIbvPHNWaMNRxKoMpo+rI3ce7e56757VqVW6HvGq3YcMGunfvTvfu3TnooIM49NBDE4+/+iq9YeEvu+wylpXziY8bN478/Pwy10lXnz59WLBgQZVsS0TisXAhXHYZtG0Lo0bBy5WuS0lfnFcfzSVMxt2eMB3fYML8tbsxsyMJE278J8ZYdpOfD7ffDmvWQJs24WAPzWCKkRYtWiROsHfeeSeNGzfmlltu2W0dd8fd2Wuv1Hl4woQJ5e7nuuuuq3yQIlIj7NwJM2fCfffBSy9Bw4Zw1VVwww3QsWP8+4+tpODuxYQpBGcCS4HJ7r7YzEaaWf+kVYcAk7yahmvNz4fhw2H1anAPf4cPD8ur2ooVK+jSpQvXXHMNPXv2ZN26dQwfPpy8vDw6d+7MyJEjE+vu+uVeXFxMs2bNGDFiBN26deO4447j448/BuCOO+7ggQceSKw/YsQIevXqxZFHHsm///1vAD7//HO+973v0a1bN4YMGUJeXl65JYKJEydy9NFH06VLF2677TYAiouLueiiixLLx44dC8D9999Pp06d6NatG8OGDavyYyZSV23bBo89Bl26wJlnwpIl8L//C++/Dw8/XD0JAWLup+Du04HpJZb9osTjO+OMoaTbb4etW3dftnVrWJ5JaaE0S5YsYcKECfz2t78FYPTo0RxwwAEUFxdz8sknM3DgQDp16rTbazZt2kTfvn0ZPXo0N910E0888QQjRozYY9vuzpw5c5g2bRojR47kL3/5Cw899BAHHXQQzz//PG+++SY9e/YsM77CwkLuuOMOCgoKaNq0Kaeddhp//vOfadWqFevXr+ett94CYOPGjQDcc889rF69mn322SexTEQq76OP4De/Cbf166F7d3jqKbjgAthnn+qPp871aF6zpmLLM/WNb3yDb33rW4nHzzzzDD179qRnz54sXbqUJUuW7PGa/fbbjzPOOAOAY445hlWrVqXc9vnnn7/HOq+99hqDBw8GoFu3bnTu3LnM+GbPns0pp5xCy5YtqV+/PhdeeCGvvvoqhx9+OMuWLeOGG25g5syZNG3aFIDOnTszbNgw8vPz1flM6ozly+Hww8MJe8gQGDkS/vhHWLQIvqzkNE+LFsEVV4Qq7JEj4bjjYNYsmD8fhg3LTkKAGjhKaqbatAlVRqmWx6FRo0aJ+8uXL+fBBx9kzpw5NGvWjGHDhqW8Xn+fpG9DvXr1KC4uTrntfffdd491KloLV9r6LVq0YOHChcyYMYOxY8fy/PPPM378eGbOnMkrr7zC1KlTueuuu1i0aBH16tWr0D5FapIvv4TBg+GTT+CII+D11+HZZ0P1M0C9etChAxx1FHTqFP4edRR885vQpMnu23KHv/41tBf89a+w334hMdxwAxx5ZPW/t1TqXFIYNSq0ISRXITVsGJbH7bPPPqNJkybsv//+rFu3jpkzZ9KvX78q3UefPn2YPHkyJ5xwAm+99VbKkkiy3r1785Of/IQNGzbQtGlTJk2axC233EJRURENGjTg+9//Pu3bt+eaa65hx44dFBYWcsopp9CnTx/y8/PZunUrTUp+80VqkZ/+NPx6nzoV+ketoVu3hstDly7d/TZjBmzf/vVrW7f+OkkcdFBou1y8ONwfNQquvhpatMjO+ypNnUsKu9oNqvLqo3T17NmTTp060aVLFzp06MDxxx9f5fv44Q9/yMUXX0zXrl3p2bMnXbp0SVT9pNK6dWtGjhzJSSedhLtzzjnncNZZZzF//nyuuOIK3B0z4+6776a4uJgLL7yQzZs3s3PnTm699VYlBKnVpk6FsWPDL/n+SZfHNGwIPXqEW7Lt22Hlyj2Txe9+B59/Dt26wZNPwqBBEBX0c06Nm6M5Ly/PS06ys3TpUo466qgsRZRbiouLKS4upkGDBixfvpzTTz+d5cuXs/feuZX/9ZlJrluzJrQhtG8P//53Zidx99CI3LIlZKtDv5nNc/e88tbLrTOFZGzLli2ceuqpFBcX4+48+uijOZcQRHJdcTFceGH4++yzmf+qN4Mc7Hebks4WtUyzZs2YN29etsMQqdF++Uv417/g6afDVUd1SZ27JFVEpCwvvRQ6jV1xRbj8tK5RUhARiXz0UegjcNRRoYG5LlL1kYgIYcyhiy6CTZu+HnOoLlJSEBEB7r4b/vY3GD8+jD9UV6n6qAqcdNJJzJw5c7dlDzzwAD/4wQ/KfF3jxo0BWLt2LQMHDix12yUvwS3pgQceYGtSb7wzzzyzSsYluvPOOxkzZkzG2xHJdf/6V5ivYNAguPLKbEeTXUoKVWDIkCFMmjRpt2WTJk1iSJqtVIcccgjPPfdcpfdfMilMnz6dZs2aVXp7InXJJ5+EBuW2bUMpoa5PDKikUAUGDhzIn//8Z76MRsZatWoVa9eupU+fPol+Az179uToo49m6tSpe7x+1apVdInKq9u2bWPw4MF07dqVQYMGsW3btsR61157bWLY7V/+8pcAjB07lrVr13LyySdz8sknA9CuXTvWr18PwH333UeXLl3o0qVLYtjtVatWcdRRR3HVVVfRuXNnTj/99N32k8qCBQvo3bs3Xbt25bzzzuPTTz9N7L9Tp0507do1MRDfK6+8kphkqEePHmzevLnSx1YkTu7hKqMPPwz9EfbfP9sRZV+ta1O48Uao6gnFuneH6HyaUosWLejVqxd/+ctfGDBgAJMmTWLQoEGYGQ0aNGDKlCnsv//+rF+/nt69e9O/f/9S5yl+5JFHaNiwIQsXLmThwoW7DX09atQoDjjgAHbs2MGpp57KwoUL+dGPfsR9993HrFmzaNmy5W7bmjdvHhMmTGD27Nm4O8ceeyx9+/alefPmLF++nGeeeYbHHnuMCy64gOeff77M+REuvvhiHnroIfr27csvfvEL/vu//5sHHniA0aNH895777HvvvsmqqzGjBnDuHHjOP7449myZQsNGjSowNEWqT4PPwwvvBAGqMsrt69v3aCSQhVJrkJKrjpyd2677Ta6du3KaaedxgcffMBHH31U6nZeffXVxMm5a9eudO3aNfHc5MmT6dmzJz169GDx4sXlDnb32muvcd5559GoUSMaN27M+eefzz//+U8A2rdvT/fu3YGyh+eGML/Dxo0b6du3LwCXXHIJr776aiLGoUOHMnHixETP6eOPP56bbrqJsWPHsnHjRvWolpw0fz7ccgucfXb4MSlBrftvLesXfZzOPfdcbrrpJubPn8+2bdsSv/Dz8/MpKipi3rx51K9fn3bt2qUcLjtZqlLEe++9x5gxY5g7dy7Nmzfn0ksvLXc7ZY1rtW9Sv/169eqVW31UmhdffJFXX32VadOm8atf/YrFixczYsQIzjrrLKZPn07v3r156aWX+OY3v1mp7YvEYfPm0KjcqhVMmKB2hGQqKVSRxo0bc9JJJ3H55Zfv1sC8adMmDjzwQOrXr8+sWbNYnWoyhyQnnngi+dHcoIsWLWLhwoVAGHa7UaNGNG3alI8++ogZM2YkXtOkSZOU9fYnnngiL7zwAlu3buXzzz9nypQpnHDCCRV+b02bNqV58+aJUsZTTz1F37592blzJ++//z4nn3wy99xzDxs3bmTLli28++67HH300dx6663k5eXx9ttvV3ifInFxh2uvDaOZPvNMGKROvlbrSgrZNGTIEM4///zdrkQaOnQo55xzDnl5eXTv3r3cX8zXXnstl112GV27dqV79+706tULCLOo9ejRg86dO+8x7Pbw4cM544wzOPjgg5k1a1Ziec+ePbn00ksT27jyyivp0aNHmVVFpXnyySe55ppr2Lp1Kx06dGDChAns2LGDYcOGsWnTJtydH//4xzRr1oyf//znzJo1i3r16tGpU6fELHIiueD3vw/zGowcCZX4jVTraehsyQp9ZpINS5eGBuVjjw0d1erSpIHpDp0da/WRmfUzs2VmtsLM9px5PqxzgZktMbPFZvZ0nPGISN21bRtccAE0agQTJ9athFARsVUfmVk9YBzwHaAQmGtm09x9SdI6HYGfAce7+6dmdmBc8YhI3eUOP/4xLFoUpsw85JBsR5S74iwp9AJWuPtKd/8KmAQMKLHOVcA4d/8UwN0/ruzOalo1WF2mz0qq04IFcNpp8Oij8JOfQBVPi17rxJkUDgXeT3pcGC1LdgRwhJn9y8xeN7OUH5eZDTezAjMrKCoq2uP5Bg0asGHDBp1sagB3Z8OGDerQJrFbty6MY9SzZ0gMDz4Y5kmQssV59VGqK39LnrX3BjoCJwGtgX+aWRd33200N3cfD4yH0NBccqOtW7emsLCQVAlDck+DBg1o3bp1tsOQWmrr1tBDefRo+OqrUG10xx3QvHm2I6sZ4kwKhcBhSY9bA2tTrPO6u28H3jOzZYQkMbciO6pfvz7t27fPJFYRqeF27gz9DkaMgMJCOP/8MBx2XZtOM1NxVh/NBTqaWXsz2wcYDEwrsc4LwMkAZtaSUJ20MsaYRKQWeu016N07zJp24IHwj3/A888rIVRGbEnB3YuB64GZwFJgsrsvNrORZtY/Wm0msMHMlgCzgJ+4+4a4YhKR2mXlSvj+90MntA8+gCefhLlzIRqmSyqhVnReE5G6ZdMmGDUqNB7vvTfceivcfHPogyCppdt5TcNciEiNUVwcJsL55S9hwwa45JKQHNTvoOpoQDwRqRFmzICuXeG668IcygUFYYRTJYSqpaQgIjmtqCgMc33mmbB9e5gU5+WXQ/8DqXpKCiKSk9xh0iTo1Ckkgl/9ChYvhgEDNP9BnNSmICI5Z926MOfB1KnQqxc88QR07pztqOoGlRREJGe4h8tKO3WCmTPh3nvhX/9SQqhOKimISE54/324+urQoHz88aF0cMQR2Y6q7lFJQUSyyh0eeyyUBl55JfQ9ePVVJYRsUUlBRLLmvffgqqvg73+Hk0+Gxx+HDh2yHVXdppKCiFS7nTvhoYdCf4M5c+C3v4WXXlJCyAUqKYhItVq+HC6/PAxi993vhh7KbdpkOyrZRSUFEakWO3bAr38deiUvWhR6I8+YoYSQa1RSEJHYzZsXhqeYPRv694dHHtHwFLlKJQURic3atXDppZCXF4a5zs8PvZOVEHKXSgoiUuW2bQtVRaNHh/GKfvpTuO02aNo025FJeZQURKTKuMOzz4b5DdasCVNi3nMPfOMb2Y5M0qXqIxGpEnPmQJ8+MGQIHHAAzJoVpsRUQqhZ6kRSyM+Hdu1gr73C3/z8bEckUnt88AFcfDEceyy8+27ogFZQACedlO3IpDJqffVRfj4MHw5bt4bHq1eHxwBDh2YvLpGabutWGDMG7r47XG76s5+FW5Mm2Y5MMlHrSwq33/51Qthl69awXEQqzh2efhqOPDJMi3nWWbB0KfzP/ygh1AaxJgUz62dmy8xshZmNSPH8pWZWZGYLotuVVR3DmjUVWy4ipXv9dfj2t0Mp+8ADw8B1kydD+/bZjkyqSmxJwczqAeOAM4BOwBAz65Ri1WfdvXt0e7yq4yitt6R6UYqkr7AQhg2D444LVbATJsDcuXDCCdmOTKpanCWFXsAKd1/p7l8Bk4ABMe4vpVGjoGHD3Zc1bBiWi0jZtm2Du+4KVUXPPReqXd95J3RI26vWVz7XTXF+rIcC7yc9LoyWlfQ9M1toZs+Z2WGpNmRmw82swMwKioqKKhTE0KFhwK22bcO8rm3bhsdqZBYpnXu4nLRTJ/j5z+GMM+Dtt0OCaNw429FJnOJMCqmm1vYSj/8EtHP3rsBLwJOpNuTu4909z93zWrVqVeFAhg6FVavCcL2rVikhiJTlrbfg1FNh4MDQcPzyy6GU0K5dtiOT6hBnUigEkn/5twbWJq/g7hvc/cvo4WPAMTHGIyJl2LAhDFrXvTu8+Sb85jcwf36Y/EbqjjiTwlygo5m1N7N9gMHAtOQVzOzgpIf9gaUxxiMiKRQXw8MPQ8eO8Oij8IMfhDkPrr0W9q71PZmkpNg+cncvNrPrgZlAPeAJd19sZiOBAnefBvzIzPoDxcAnwKVxxSMie3r5ZbjhhjC/wSmnhPmRu3TJdlSSTeZespo/t+Xl5XlBQUG2wxCp0d57D26+GaZMCX0Mfv1rOPfccDGG1E5mNs/d88pbTxeVidQhn38Od9wBRx0FM2eGS7OXLIHzzlNCkEA1hiJ1wJdfwh//CCNGhAHshg4NYxYdmuoicanTlBREaqmNG2H6dJg6NcyFvHkzHHNMGJbi29/OdnSSq5QURGqRNWtCEpg6FV55JVxZdOCBMGhQaDM44wz1RJayKSmI1GDusGDB14lgwYKw/MgjQ0PygAFhngMlAkmXkoJIDbN9eygFTJ0K06aF0oFZqBK65x7o3z8kBZHKUFIQqQF27AiXj06ZAi++CJs2QYMG8J3vhDkNzj47VBOJZEpJQSTHvf02XHZZmMugZctw+eiAASEhNGqU7eiktlFSEMlRO3bA/feHfgUNG8If/gAXXgj16mU7MqnNlBREclBy6WDAAPjtb+Ggg7IdldQFuiZBJIfs2AFjxoSRSt95B/LzQzuCEoJUF5UURHKESgeSC1RSEMkylQ4kl6ikIJJFKh1IrlFJQSQLduyAe+/9unTw9NMqHUhuUElBpJollw7OPRceeUTJQHKHSgoi1SRV6eD//k8JQXKLSgoi1WD+fLjuOpUOJPeppCASozVr4OKLwzwGK1aodCC5L9akYGb9zGyZma0wsxFlrDfQzNzMyp0/VKQm+OwzuO22MFrp5MlhxrMVK2DIEE17KbkttuojM6sHjAO+AxQCc81smrsvKbFeE+BHwOy4YhGpLtu3w2OPwZ13QlERXHQR3HUXtGmT7chE0hNnSaEXsMLdV7r7V8AkYECK9X4F3AN8EWMsIrFyD3MbHH10aDvo3BkKCsIgdkoIUpPEmRQOBd5PelwYLUswsx7AYe7+5xjjEIlVQQGcfHLofAYhObz8cmhHEKlp4kwKqWpOPfGk2V7A/cDN5W7IbLiZFZhZQVFRURWGKFJ5q1fDsGHwrW/BkiXwm9/AW2/BOeeo3UBqrjiTQiFwWNLj1sDapMdNgC7AP8xsFdAbmJaqsdndx7t7nrvntWrVKsaQRcq3aVNoOD7ySHj++dCgvGIFXHst1K+f7ehEMhNnUpgLdDSz9ma2DzAYmLbrSXff5O4t3b2du7cDXgf6u3tBjDFJDeUO774b/mbL9u0wbhwcfniYC3nQoNAJbdQo2H//7MUlUpViSwruXgxcD8wElgKT3X2xmY00s/5x7Vdqp1Gjwsm4U6cwG9knn1TfvouKYOxY6NIFrr8+NCYXFMCTT8Jhh5X/epGaxDybP70qIS8vzwsKKl6YKCyEZ5+Fm8ttwZBc89JLcPrpcNpp4fr/2bNh333h+9+H4cOhT5+qr8P/6it48cVw4n/xRSguhry8cKnpmWeqzUBqHjOb5+7l9gWrMz2an3wSbrkldCSSmuODD8K8xJ06hVFEX38dFiyAK64IV/mceGK4/POBBzIvPbiHEsAPfwiHHALnnx8S0I03hgbkuXPhrLOUEKR2S6ukYGbfAArd/UszOwnoCvzB3TfGHN8eKltS2L4dTjgBli0L/+CtW8cQnFSp7dvhlFPgjTfCCfmoo3Z//vPPQ+lv/PjdSw9XXw3HH5/+yXvtWpg4MfxwWLIkbOfcc+GSS+A734G9NUKY1AJVXVJ4HthhZocDvwPaA09nEF+1q18//ONv3x7+2XfuzHZEUp7bboPXXoPHH98zIQA0agSXXx5KD2+8Ee5PnRqSf5cu8OCD8Omnqbe9bRs88wz06xfaBW69FZo1g0cfhQ8/hEmT4IwzlBCkDnL3cm/A/OjvT4AfRvffSOe1VX075phjPBOPP+4O7mPGZLQZidmUKeFz+sEPKva6LVvCZ9yrV3h9gwbuF1/s/tpr7jt3uv/zn+5XXum+//7h+TZt3O+4w/2dd+J5HyK5AijwNM6x6VYfzQYeAG4HznH398xskbt3iStZlaay1Ue7uIe64unTYc4c6NatCoOTKrFyJfTsCR07hpLCvvtWbjsLFoSqpYkTYfNmaNo09DFo1AgGDgwlxr59Ya8607ImdVm61UfpJoVOwDXAf9z9GTNrDwxy99GZh1oxmSYFgPXrw2WFLVqEuur99qui4CRjX3wB3/42rFoV5iBo1y7zbW7ZEtoeZs0KVzGdfz40bpz5dkVqkipNCiU23JwwXtHCygaXiapICgAzZ4b65B/9KNQ9S264+urw6/5Pf4Kzz852NCK1R5U2NJvZP8xsfzM7AHgTmGBm92UaZDZ997shIYwdGxKEZN9TT4WEMGKEEoJItqRbm9rU3T8AHJpUAAATYUlEQVQDzgcmuPsxwGnxhVU9Ro8O17hfemmoUpLsWbwYrrkm1PH/6lfZjkak7ko3KextZgcDFwC1Zpjr/faD/PzQ6emqq7I7rk5dtnkzfO970KRJuExUl4GKZE+6SWEkYQyjd919rpl1AJbHF1b16dYtjKvzwgvwxBPZjqbucQ9DVSxfHvoGHHxwtiMSqdvqzNhHZdm5M4yrM2dOuIzx8MOrdPNSht/8JsxU9j//Az/7WbajEam9qrqhubWZTTGzj83sIzN73sxqzUARe+0VhjioXz9MmrJ9e7YjqhvmzAnjCp15ZuhRLCLZl2710QTCXAiHEKbU/FO0rNY47LAwxMHs2WGidYnXJ5/ABReEgeeeekodyERyRbr/iq3cfYK7F0e33wO1bgq0Cy6Aiy4KSeE//8l2NLXXzp1w8cVhILrJk+GAA7IdkYjskm5SWG9mw8ysXnQbBmyIM7BsefhhaNMmVCNt3pztaGqnu+8OcxTcfz/06pXtaEQkWbpJ4XLC5agfAuuAgcBlcQWVTfvvH6ozVq2CG27IdjS1zz/+AXfcAYMHww9+kO1oRKSktK4Id/c1wG5TaJrZjYRB8mqdPn3ClTCjRoVJVb73vWxHlH2ffRZmQGvYMFT37Lo1bQr16qW3jXXrQjLo2DH0XNZkNSK5p9KXpJrZGndvU8XxlCuOS1JT2b49DMy2ciUsXAiHHhr7LnPSzp3w+9+HJPnxx3s+bxbmITjgAGjefPeEkXxr3hzuuy9ccTRnTpjvQESqT7qXpGbSd7RW/86rXz/0du7RIwyDMXNmZlfIuIcTbLq/qnPBv/8dxoeaNw9694annw4lhU8+Kfu2cmX4u3HjnpMZPfmkEoJILsskKZRbxDCzfsCDQD3g8ZJDbZvZNcB1wA5gCzDc3ZdkEFOVOuKI0Bh69dXQsmU4ybVpE6qVhg5N/ZqNG0Pv3HfeCX+T73/5ZWjAvuGG3D4xfvBB6DeQnx8uGZ04McyTXNHqnp07Q7XTrmSx335hrCkRyV1lJgUz20zqk78BZc5CYGb1gHHAd4BCYK6ZTStx0n/a3X8brd8fuA/ol3748WvYMPy63zWt4+rVYZyk1atD3XjJk3/ywHpmIYkccURIIl98EU60jz8e5v698cYwfHeuXKP/xRfw61+H3sU7dsDtt4cRSys798Bee4WqpWbNoEOHqo1VROJRZlJw9yYZbLsXsMLdVwKY2SRgAJBICtHIq7s0Io3SR3W7445wgky2bVs4Ye5yyCEhQZx3XvjbsWNIBB06QIMGu7/2nntCI+vDD4dG7COPDCWHiy8OM4JlgztMmQI33xyuujr/fLj3Xp3IReqi2MY+MrOBQD93vzJ6fBFwrLtfX2K964CbgH2AU9x9j4H2zGw4MBygTZs2x6xevTqWmFPZa6/SR099440wTlJlfkl/9RU891yoniooCA2xw4eHcYAOOyyzmCvirbdCieXll0OV1gMPwKmnVt/+RaR6VOnYR5WNIcWyPU6v7j7O3b8B3ArckWpD7j7e3fPcPa9Vq+rtSN2mlOur2raF7t0rX7Wyzz6hnn7OnDAP8amnhl/n7duHyzZnz658zOnYsAGuvz68hzfeCCWXN95QQhCp6+JMCoVA8m/e1sDaMtafBJwbYzyVMmpUaFdI1rBhWF4VzOD44+GPfwxX7dx4I8yYEa72Oe64MAxEcXHV7AvCth5+OFRxPfIIXHttaA+57jrNYyAi8SaFuUBHM2tvZvsAgwmD6iWYWcekh2eRg3M0DB0a2gDatg0n8LZtw+PSrj7KRNu2MGYMFBaGaUKLimDQoFC3f8894df9V1+F25dfhobhbdvCbetW+PzzMEn95s3h9tlnsGlTuCJq48bQ+axHD/jhD8PfBQtCgmjRourfi4jUTLHOp2BmZxJ6PdcDnnD3UWY2Eihw92lm9iBhWs/twKfA9e6+uKxtVlfntVywYwdMnx7aHWbNqppttm8frjA691z1KBapS9JtU9AkOzXEm2+GQeR27gwn810n9F33k2+plkNozB48eM8rokSk9quOHs1Sjbp1CzcRkTjlSLcpERHJBUoKIiKSoKQgIiIJSgoiIpKgpCAiIglKCiIikqCkICIiCUoKIiKSoKQgIiIJSgoiIpKgpCAiIglKCiIikqCkICIiCUoKIiKSoKRQDfLzoV072Guv8Dc/P9sRiYikpvkUYpafD8OHh+kyAVavDo8hnik9RUQyoZJCzG6//euEsMvWrWG5iEiuUVKI2Zo1FVsuIpJNSgoxa9OmYstFRLIp1qRgZv3MbJmZrTCzESmev8nMlpjZQjP7u5m1jTOebBg1Cho23H1Zw4ZhuYhIroktKZhZPWAccAbQCRhiZp1KrPYGkOfuXYHngHviiidbhg6F8eOhbVswC3/Hj1cjs4jkpjivPuoFrHD3lQBmNgkYACzZtYK7z0pa/3VgWIzxZM3QoUoCIlIzxFl9dCjwftLjwmhZaa4AZqR6wsyGm1mBmRUUFRVVYYgiIpIszqRgKZZ5yhXNhgF5wL2pnnf38e6e5+55rVq1qsIQRUQkWZzVR4XAYUmPWwNrS65kZqcBtwN93f3LGOMREZFyxFlSmAt0NLP2ZrYPMBiYlryCmfUAHgX6u/vHMcYiIiJpiC0puHsxcD0wE1gKTHb3xWY20sz6R6vdCzQG/mhmC8xsWimbq9M0dpKIVJdYxz5y9+nA9BLLfpF0/7Q4918baOwkEalO6tGc4zR2kohUJyWFHKexk0SkOikp5DiNnSQi1UlJIcdp7CQRqU5KCjlOYyeJSHXSzGs1gMZOEpHqopKCiIgkKCnUAer8JiLpUvVRLafObyJSESop1HLq/CYiFaGkUMup85uIVISSQi2nzm8iUhFKCrWcOr+JSEUoKdRy6vwmIhWhq4/qAHV+E5F0qaQg5VI/B5G6QyUFKZP6OYjULSopSJnUz0GkblFSkDKpn4NI3RJrUjCzfma2zMxWmNmIFM+faGbzzazYzAbGGYtUjvo5iNQtsSUFM6sHjAPOADoBQ8ysU4nV1gCXAk/HFYdkpir6OaihWqTmiLOk0AtY4e4r3f0rYBIwIHkFd1/l7guBnTHGIRnItJ/Drobq1avB/euGaiUGkdwUZ1I4FHg/6XFhtKzCzGy4mRWYWUFRUVGVBCfpGzoUVq2CnTvD34pcdaSGapGaJc6kYCmWeWU25O7j3T3P3fNatWqVYVhSndRQLVKzxJkUCoHDkh63BtbGuD/JQWqoFqlZ4kwKc4GOZtbezPYBBgPTYtyf5CA1VIvULLElBXcvBq4HZgJLgcnuvtjMRppZfwAz+5aZFQLfBx41s8VxxSPZoYZqkZrF3CtVzZ81eXl5XlBQkO0wpJq0axcSQUlt24ZGbxFJj5nNc/e88tZTj2bJaWqoFqleSgqS06qioVptEiLpU1KQnJZpQ7XaJEQqRklBclqmDdXqPCdSMUoKkvMy6VFdFW0Sqn6SukRJQWq1TNskVP0kdY2SgtRqmbZJqPpJ6holBanVMm2TUPWT1DWao1lqvaFDKz+fdJs2qTvPVbT6SXNcS02hkoJIGXKl+kmlDakuSgoiZciV6ic1dkt1UVIQKUcml8RWRY/sqihtqKQh6VJSEIlRVQwdnmlpQyUNqQglBZEYZVr9BJmXNlTSkIpQUhCJWSbVT5B5aUMlDakIJQWRHJdpaUMlDakIJQWRGiCT0kZtKGlkmlSUlCrA3WvU7ZhjjnERqZiJE93btnU3C38nTkz/tW3buofT+e63tm2r5/UTJ7o3bLj7axs2TP89ZPr6qpDJ8a8qQIGncY7N+km+ojclBZHqlelJ1Sx1UjBL7/XZTkrumZ3UcyEpuaefFFR9JCJlynabRqbVV9mu/qpxbTLpZI7K3oB+wDJgBTAixfP7As9Gz88G2pW3TZUURGqWTH8pZ7ukkOnrMy0pVVVJg2yXFMysHjAOOAPoBAwxs04lVrsC+NTdDwfuB+6OKx4RyY5MSxqZNpRnu6E9F67+qpB0MkdlbsBxwMykxz8DflZinZnAcdH9vYH1gJW1XZUUROqeTBtqs9nQnu02mV3IdkkBOBR4P+lxYbQs5TruXgxsAlqU3JCZDTezAjMrKCoqiilcEclVmXYAzOYlvdluk6moOJOCpVjmlVgHdx/v7nnunteqVasqCU5EJB1VMVRJNpNSRcU5yU4hcFjS49bA2lLWKTSzvYGmwCcxxiQiUmGZTNRUFfuG0IawZk0oIYwaFV88cSaFuUBHM2sPfAAMBi4ssc404BLgP8BA4OWo7ktERCLVmZRiSwruXmxm1xMak+sBT7j7YjMbSWjwmAb8DnjKzFYQSgiD44pHRETKF+scze4+HZheYtkvku5/AXw/zhhERCR96tEsIiIJSgoiIpKgpCAiIglW0y72MbMiYHW24yhFS0Kv7Fyl+DKT6/FB7seo+DKTSXxt3b3cjl41LinkMjMrcPe8bMdRGsWXmVyPD3I/RsWXmeqIT9VHIiKSoKQgIiIJSgpVa3y2AyiH4stMrscHuR+j4stM7PGpTUFERBJUUhARkQQlBRERSVBSqCAzO8zMZpnZUjNbbGY3pFjnJDPbZGYLotsvUm0rxhhXmdlb0b4LUjxvZjbWzFaY2UIz61mNsR2ZdFwWmNlnZnZjiXWq/fiZ2RNm9rGZLUpadoCZ/c3Mlkd/m5fy2kuidZab2SXVFNu9ZvZ29PlNMbNmpby2zO9CzDHeaWYfJH2OZ5by2n5mtiz6Po6oxvieTYptlZktKOW1sR7D0s4pWfv+pTM9m267TSF6MNAzut8EeAfoVGKdk4A/ZzHGVUDLMp4/E5hBmOSoNzA7S3HWAz4kdKrJ6vEDTgR6AouSlt0DjIjujwDuTvG6A4CV0d/m0f3m1RDb6cDe0f27U8WWznch5hjvBG5J4zvwLtAB2Ad4s+T/U1zxlXj+18AvsnEMSzunZOv7p5JCBbn7OnefH93fDCxlz2lGc90A4A8evA40M7ODsxDHqcC77p71Huru/ip7TvA0AHgyuv8kcG6Kl34X+Ju7f+LunwJ/A/rFHZu7/9XDFLYArxMmscqaUo5fOnoBK9x9pbt/BUwiHPcqVVZ8ZmbABcAzVb3fdJRxTsnK909JIQNm1g7oAcxO8fRxZvammc0ws87VGliY0vSvZjbPzIaneD6d+bOrw2BK/0fM5vHb5b/cfR2Ef1zgwBTr5MKxvJxQ8kulvO9C3K6PqrieKKX6IxeO3wnAR+6+vJTnq+0YljinZOX7p6RQSWbWGHgeuNHdPyvx9HxClUg34CHghWoO73h37wmcAVxnZieWeD6tubHjZGb7AP2BP6Z4OtvHryKyeizN7HagGMgvZZXyvgtxegT4BtAdWEeooikp699FYAhllxKq5RiWc04p9WUplmV0/JQUKsHM6hM+vHx3/7+Sz7v7Z+6+Jbo/HahvZi2rKz53Xxv9/RiYQiiiJ0tn/uy4nQHMd/ePSj6R7eOX5KNd1WrR349TrJO1Yxk1Kp4NDPWogrmkNL4LsXH3j9x9h7vvBB4rZd9Z/S5amBv+fODZ0tapjmNYyjklK98/JYUKiuoffwcsdff7SlnnoGg9zKwX4ThvqKb4GplZk133CQ2Si0qsNg24OLoKqTewaVcxtRqV+ussm8evhF1ziBP9nZpinZnA6WbWPKoeOT1aFisz6wfcCvR3962lrJPOdyHOGJPbqc4rZd+Judyj0uNgwnGvLqcBb7t7Yaonq+MYlnFOyc73L64W9dp6A/oQimcLgQXR7UzgGuCaaJ3rgcWEKyleB75djfF1iPb7ZhTD7dHy5PgMGEe46uMtIK+aj2FDwkm+adKyrB4/QoJaB2wn/Pq6AmgB/B1YHv09IFo3D3g86bWXAyui22XVFNsKQl3yru/gb6N1DwGml/VdqMbj91T0/VpIOMEdXDLG6PGZhCtu3o0rxlTxRct/v+t7l7RutR7DMs4pWfn+aZgLERFJUPWRiIgkKCmIiEiCkoKIiCQoKYiISIKSgoiIJCgpiETMbIftPoJrlY3YaWbtkkfoFMlVe2c7AJEcss3du2c7CJFsUklBpBzRePp3m9mc6HZ4tLytmf09GvDt72bWJlr+XxbmOHgzun072lQ9M3ssGjP/r2a2X7T+j8xsSbSdSVl6myKAkoJIsv1KVB8NSnruM3fvBTwMPBAte5gwBHlXwoB0Y6PlY4FXPAzo15PQExagIzDO3TsDG4HvRctHAD2i7VwT15sTSYd6NItEzGyLuzdOsXwVcIq7r4wGLvvQ3VuY2XrC0A3bo+Xr3L2lmRUBrd39y6RttCOMe98xenwrUN/d7zKzvwBbCKPBvuDRYIAi2aCSgkh6vJT7pa2TypdJ93fwdZveWYSxqI4B5kUjd4pkhZKCSHoGJf39T3T/34RRPQGGAq9F9/8OXAtgZvXMbP/SNmpmewGHufss4KdAM2CP0opIddEvEpGv7We7T97+F3ffdVnqvmY2m/BDaki07EfAE2b2E6AIuCxafgMw3syuIJQIriWM0JlKPWCimTUljF57v7tvrLJ3JFJBalMQKUfUppDn7uuzHYtI3FR9JCIiCSopiIhIgkoKIiKSoKQgIiIJSgoiIpKgpCAiIglKCiIikvD/IYufLjtGRKgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "acc = history.history['acc']\n", "val_acc = history.history['val_acc']\n", "loss = history.history['loss']\n", "val_loss = history.history['val_loss']\n", "\n", "epochs = range(1, len(acc) + 1)\n", "\n", "# ‘bo’는 파란색 점을 의미합니다\n", "plt.plot(epochs, loss, 'bo', label='Training loss')\n", "# ‘b’는 파란색 실선을 의미합니다\n", "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n", "plt.title('Training and validation loss')\n", "plt.xlabel('Epochs')\n", "plt.ylabel('Loss')\n", "plt.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8FeXZ//HPFUQCsgfcQBaXWhUBMQJW3C2CWnFrFbFV0VKtaG3r0weXFkvdHuterRWr1iVK+dVioVUUEUWLKEFJEFxARI0gRmQHgcD1++OewCGc5BxytoR836/XvM6ZmXtmrjM5mevMfc/cY+6OiIhITfJyHYCIiNR9ShYiIpKQkoWIiCSkZCEiIgkpWYiISEJKFiIikpCShSTNzBqZ2Woz65TOsrlkZvubWdqvHzezk8xsYcz4h2Z2dDJla7Gtv5rZdbVdXiQZu+Q6AMkcM1sdM9oMWA9sisZ/5u5FO7I+d98ENE932YbA3Q9Mx3rM7FLgAnc/Lmbdl6Zj3SI1UbLYibn7loN19Mv1Und/ubryZraLu1dkIzaRRPR9rFtUDdWAmdlNZvZ3M3vGzFYBF5jZkWY23cyWm9liM7vPzBpH5XcxMzezLtH4U9H8F8xslZm9aWZdd7RsNH+gmX1kZivM7E9m9l8zu6iauJOJ8WdmNt/MlpnZfTHLNjKzu81sqZl9DAyoYf/cYGZjqkx7wMzuit5fambvR5/n4+hXf3XrKjOz46L3zczsySi2OcDhcba7IFrvHDM7PZp+KHA/cHRUxfd1zL69MWb5y6LPvtTMnjOzvZLZNzuynyvjMbOXzewbM/vSzH4Ts53fRvtkpZkVm9ne8ar8zOyNyr9ztD+nRtv5BrjBzA4wsynRZ/k62m+tYpbvHH3G8mj+vWaWH8V8UEy5vcxsrZkVVPd5JQF319AABmAhcFKVaTcBG4AfEH44NAWOAPoQzjr3BT4ChkfldwEc6BKNPwV8DRQCjYG/A0/VouzuwCpgUDTvV8BG4KJqPksyMf4LaAV0Ab6p/OzAcGAO0BEoAKaGf4O429kXWA3sFrPur4DCaPwHURkDTgDWAd2jeScBC2PWVQYcF72/A3gVaAN0BuZWKfsjYK/ob3J+FMMe0bxLgVerxPkUcGP0vn8UY08gH/gz8Eoy+2YH93MrYAnwC6AJ0BLoHc27FigBDog+Q0+gLbB/1X0NvFH5d44+WwVwOdCI8H38DnAisGv0PfkvcEfM53kv2p+7ReWPiuaNBm6O2c6vgXG5/j+sz0POA9CQpT909cnilQTLXQP8v+h9vATwl5iypwPv1aLsUOD1mHkGLKaaZJFkjH1j5v8TuCZ6P5VQHVc575SqB7Aq654OnB+9Hwh8VEPZfwNXRO9rShafxf4tgJ/Hlo2z3veAU6P3iZLF48AtMfNaEtqpOibaNzu4n38MFFdT7uPKeKtMTyZZLEgQwznAjOj90cCXQKM45Y4CPgEsGp8FnJXu/6uGNKgaSj6PHTGz75rZf6JqhZXAKKBdDct/GfN+LTU3aldXdu/YODz8d5dVt5IkY0xqW8CnNcQL8DQwOHp/PrDlogAzO83M3oqqYZYTftXXtK8q7VVTDGZ2kZmVRFUpy4HvJrleCJ9vy/rcfSWwDOgQUyapv1mC/bwPML+aGPYhJIzaqPp93NPMxprZF1EMf6sSw0IPF1Nsw93/SzhL6Wdm3YBOwH9qGZOgNgsJvzRjPUT4Jbu/u7cEfkf4pZ9Jiwm/fAEwM2Pbg1tVqcS4mHCQqZTo0t6/AyeZWUdCNdnTUYxNgX8AtxKqiFoDLyUZx5fVxWBm+wIPEqpiCqL1fhCz3kSX+S4iVG1Vrq8FobrriyTiqqqm/fw5sF81y1U3b00UU7OYaXtWKVP18/0f4Sq+Q6MYLqoSQ2cza1RNHE8AFxDOgsa6+/pqykkSlCykqhbACmBN1ED4syxs899ALzP7gZntQqgHb5+hGMcCV5tZh6ix839rKuzuSwhVJY8BH7r7vGhWE0I9ejmwycxOI9StJxvDdWbW2sJ9KMNj5jUnHDDLCXnzUsKZRaUlQMfYhuYqngEuMbPuZtaEkMxed/dqz9RqUNN+Hg90MrPhZrarmbU0s97RvL8CN5nZfhb0NLO2hCT5JeFCikZmNoyYxFZDDGuAFWa2D6EqrNKbwFLgFgsXDTQ1s6Ni5j9JqLY6n5A4JAVKFlLVr4ELCQ3ODxF+WWdUdEA+F7iL8M+/H/Au4RdlumN8EJgMzAZmEM4OEnma0AbxdEzMy4FfAuMIjcTnEJJeMkYSznAWAi8QcyBz91LgPuDtqMx3gbdilp0EzAOWmFlsdVLl8hMJ1UXjouU7AUOSjKuqavezu68Avg+cTWhQ/wg4Npr9R+A5wn5eSWhszo+qF38KXEe42GH/Kp8tnpFAb0LSGg88GxNDBXAacBDhLOMzwt+hcv5Cwt95g7tP28HPLlVUNv6I1BlRtcIi4Bx3fz3X8Uj9ZWZPEBrNb8x1LPWdbsqTOsHMBhCqFb4lXHpZQfh1LVIrUfvPIODQXMeyM1A1lNQV/YAFhOqJAcAZapCU2jKzWwn3etzi7p/lOp6dgaqhREQkIZ1ZiIhIQjtNm0W7du28S5cuuQ5DRKRemTlz5tfuXtOl6sBOlCy6dOlCcXFxrsMQEalXzCxRLwaAqqFERCQJShYiIpKQkoWIiCSkZCEiIgkpWYiISEIZSxZm9qiZfWVm71Uz36LHJ843s1Iz6xUz70IzmxcNF2YqRhGRVBQVQZcukJcXXouKEi1Rf7efyUtn/0Z4XnB1XQMPJDx28QDCoxsfBPpEXRmPJDx+04GZZjbe3ZdlMFYRkR1SVATDhsHatWH800/DOMCQ2vbzW4e3n7EzC3efSui6uTqDgCc8mA60tvBg+ZOBSe7+TZQgJhH6ChKRnUyqv4xz+cv++uu3HqgrrV0bpicrlfjTsf0dkcs2iw5s+wjFsmhaddO3Y2bDzKzYzIrLy8szFqiIpF/lL+NPPwX3rb+Mkz1gprp85Tpqe7D+rJruCaubHm/bqcSf6vZ3VC6TRbzHT3oN07ef6D7a3QvdvbB9+4R3q4tIHZLqL+NUl0/1YN2pmgfyVje9qlTjT3X7OyqXyaKMbZ9D3JHwwJvqpotIHZPLX+apLp/qwfrmm6FZs22nNWsWpicj1fhT3f6OymWyGA/8JLoqqi+wwt0XAy8C/c2sjZm1AfpH00SkDsn1L/NUl0/1YD1kCIweDZ07g1l4HT06+cblVONPdfs7zN0zMhAeHL8Y2Eg4W7gEuAy4LJpvwAPAx4Tn5BbGLDsUmB8NFyezvcMPP9xFGpqnnnLv3NndLLw+9VT2lu/c2T2kiW2Hzp2T33azZtsu26xZ8jGkunyq8acq1fjTBSj2ZI7pyRSqD4OShTQ0uT7YmsU/2Jrt2GfIVbKrCwfrVD9/OiSbLHaaJ+UVFha6uiiXhqRLl1D1U1XnzrBwYd1fvi4oKgptFJ99Fqp/br45O/dI1CVmNtPdCxOVU3cfIjlUnxuIs93AmglDhoTEtnlzeG1oiWJHKFmI5Eh9byDOegOr5JSShUiO5PrSzXScGeiXecOhZCGSglxWI6X6y15nBrIj1MAtUktVO3KD8Ms82QPuztBALPWfGrhFMizX1Ugi2aRkIQ1afa5GEsmmTD7PQqROS/V5AJ06xa9G2pGO3IYMUXKQ+kFnFtJgqRpJJHlKFtJgqRpJJHmqhpIGS9VIIsnTmYU0WKpGEkmekoXUa6lczaRqJJHkqRpK6q1Ur2aqLKfkIJKYziyk3kr1aiYRSZ6ShdRbqV7NJCLJU7KQeivVLrZFJHlKFpJTqTRQ62omkexRspCcSfXhP7qaSSR71EW55Iy66BbJPXVRLnWeGqhF6g8lC8kZNVCL1B9KFpISNVCLNAxKFlJraqAWaTjUwC21pgZqkfpPDdyScWqgFmk4lCyk1tRALdJwKFlIramBWqThULKQWlMDtUjDoedZSEr0PAiRhkFnFiIikpCSRQOXyk11ItJwqBqqAUvHY0lFpGHQmUUDpseSikiylCwaMN1UJyLJymiyMLMBZvahmc03sxFx5nc2s8lmVmpmr5pZx5h5m8xsVjSMz2ScDZVuqhORZGUsWZhZI+ABYCBwMDDYzA6uUuwO4Al37w6MAm6NmbfO3XtGw+mZirMh0011IpKsTJ5Z9Abmu/sCd98AjAEGVSlzMDA5ej8lznzJIN1UJyLJymSy6AB8HjNeFk2LVQKcHb0/E2hhZgXReL6ZFZvZdDM7I94GzGxYVKa4vLw8nbE3GEOGhB5iN28Or0oUIhJPJpOFxZlWtT/0a4Bjzexd4FjgC6Aimtcp6jb3fOAeM9tvu5W5j3b3QncvbN++fRpDFxGRWJm8z6IM2CdmvCOwKLaAuy8CzgIws+bA2e6+ImYe7r7AzF4FDgM+zmC8IiJSjUyeWcwADjCzrma2K3AesM1VTWbWzswqY7gWeDSa3sbMmlSWAY4C5mYw1npLd2CLSDZk7MzC3SvMbDjwItAIeNTd55jZKKDY3ccDxwG3mpkDU4ErosUPAh4ys82EhHabuytZVKE7sEUkW/RY1XpMjzUVkVTpsaoNgO7AFpFsUbKox3QHtohki5JFPaY7sEUkW5Qs6jHdgS0i2aLnWdRzeqypiGSDzixERCQhJQsREUlIyUJERBJSshARkYSULEREJCElCxERSUjJIsfUa6yI1Ae6zyKH1GusiNQXOrPIoeuv35ooKq1dG6aLiNQlShY5pF5jRaS+ULLIIfUaKyL1hZJFDqnXWBGpL5Qscki9xopIfaGroXJMvcaKSH2gMwsREUlIyUJERBJSshARkYSULEREJCElCxERSUjJQkREElKyEBGRhJQsREQkISULERFJKGGyMLPhZtYmG8GIiEjdlMyZxZ7ADDMba2YDzMwyHZSIiNQtCZOFu98AHAA8AlwEzDOzW8xsvwzHJiIidURSbRbu7sCX0VABtAH+YWa3ZzA2ERGpI5Jps7jKzGYCtwP/BQ5198uBw4GzMxxfnVdUBF26QF5eeC0qynVEIiLpl0wX5e2As9z909iJ7r7ZzE7LTFj1Q1ERDBu29Tnan34axkHdjovIziWZaqjngW8qR8yshZn1AXD39zMVWH1w/fVbE0WltWvDdBGRnUkyyeJBYHXM+JpoWoP32Wc7Nl1EpL5KJllY1MANhOonknzCXnSp7YdmNt/MRsSZ39nMJptZqZm9amYdY+ZdaGbzouHCZLaXbZ067dh0EZH6KplksSBq5G4cDb8AFiRayMwaAQ8AA4GDgcFmdnCVYncAT7h7d2AUcGu0bFtgJNAH6A2MrIs3Bt58MzRrtu20Zs3CdBGRnUkyyeIy4HvAF0AZ4QA+LInlegPz3X2Bu28AxgCDqpQ5GJgcvZ8SM/9kYJK7f+Puy4BJwIAktplVQ4bA6NHQuTOYhdfRo9W4LSI7n4TVSe7+FXBeLdbdAfg8Zrwy0cQqIVx+ey9wJtDCzAqqWbZDLWLIuCFDlBxEZOeXMFmYWT5wCXAIkF853d2HJlo0zjSvMn4NcL+ZXQRMJZy9VCS5LGY2jOgsp5MaCkREMiaZaqgnCf1DnQy8BnQEViWxXBmwT8x4R2BRbAF3X+TuZ7n7YcD10bQVySwblR3t7oXuXti+ffskQhIRkdpIJlns7+6/Bda4++PAqcChSSw3AzjAzLqa2a6EqqzxsQXMrJ2ZVcZwLfBo9P5FoL+ZtYkatvtH00REJAeSSRYbo9flZtYNaAV0SbSQu1cAwwkH+feBse4+x8xGmdnpUbHjgA/N7CNgD+DmaNlvgD8QEs4MYFQ0TUREcsBibqGIX8DsUuBZwtnE34DmwG/d/aGMR7cDCgsLvbi4ONdhiIjUK2Y2090LE5WrsYE7qiJaGV2+OhXYN03xiYhIPVJjNVR0t/bwLMUiIiJ1VDJtFpPM7Boz28fM2lYOGY9MRETqjGSSxVDgCkI11MxoUONAZMkSOPlkGDMm15GIiGROMndwd81GIPXR+vVw1lkwbRpMngy77hrGRUR2Nsncwf2TeNPd/Yn0h1N/uIcHHU2bBo8+Cg8/DOedB+PHw4A614tVfGVl8Oab8M47cNxx4QxJRCSeZLoaPyLmfT5wIvAO0KCTxR//CE88ASNHwsUXw5lnwvHHh9eJE+HYY3Md4bY2boSSkpDcKofPo963zOC220KSu/NOOLhq38Ai0uAlvM9iuwXMWgFPuvvpCQtnUTbvs5gwAQYNgnPOCW0VeVHLT3l5SBKffx6qpXr3zko4cX39dThrqEwMM2bAunVhXqdO8L3vwZFHhteDDgq95Y4aBatWhTOm3/8estWDyrx50LEjNG2ane2JyFbJ3mdRm2TRGCh194NqG1wmZCtZzJ4dDrAHHghTp27/PIsvvoBjjoFly+DVV6F794yHxObNMHfutmcN8+aFeY0bw2GHhZgrE0THjvHXs3RpSBJ//jPsthvccANcdRU0aZL+mNevh3/8Ax54ICS1o4+Gl16C/PzEy4pI+qQtWZjZBLb2+JpHeAbFWHff7sl3uZSNZFFeHs4W1q8Pv9Q7VNNp+sKF4eC3YUNIKAcemLmYJkyAyy6DRVE3i+3bb00M3/seHH74jv9i/+ADuOYa+M9/oGtXuP12OPvsUF2Vqs8+g4ceCm085eWw//6h+uv++8PFAWPHQqNGqW9HRJKTbLLA3WscgGNjhqOAjomWycVw+OGHeyZ9+617v37u+fnub7+duPwHH7jvvrt7hw7uCxakP57ly90vusgd3Hv0cH/8cfd589w3b07fNl56yf3QQ8M2+vVL7nPHs3mz+6RJ7mec4Z6XF4bTT3efONF906ZQ5u67w3Z+/vP0fgYRqRlQ7EkcY5NJFl2B/JjxpkCXZFaezSGTyWLzZveLLw5765lnkl+upMS9TRv3ffd1LytLXzyTJ7t36hQOutdf775+ffrWXVVFhftDD4XEB+4//rH7558nt+zy5e733ut+4IFh2Xbt3EeMcP/kk/jl/+d/Qrmbbkpb+CKSQDqTRTGwa8z4rsCMZFaezSGTyeKOO8Ke+u1vd3zZt95yb97c/bvfdf/qq9TiWLPG/corQyzf+Y779OmprW9HrFjhfu217k2auDdt6v6737mvWhW/bEmJ+7Bh7s2ahVj79HF/4gn3detq3samTSEZgfsjj6T/M6SqoiIkwE8/dZ892/2NN9yff959zBj3f/7T/bXX3N97z33x4swmcJF0SjZZJNNmMcvde1aZVuLuPZKuFMuCTLVZPP88nHba1vr0vGTuea9i6tRQL3/ggTBlCrRuvePrmD4dLrwQPvooNDrfeuv2jevZsHAhjBgBf/877LUX3HIL/OQnUFEB//xnaLB+443QUD14MFxxRWg3SdbGjfCDH8DLL8Nzz4V9n0nuMG5cuHBhxQpYuTIMle9jX1ev3rF1N28OBQXbDm3bbj+toAC6dcvN31MknQ3ck4A/ufv4aHwQcJW7n5iWSNMkE8lizpxw9dD++8Prr4crhGrrxRfDQfDww2HSpHAgScaGDeEKpdtuC1cxPfYYnHBC7eNIlzffhF/+Et56Cw49NDRWf/kl7LsvXH55uPekoKB26169OtyzMmcOvPIK9O2b3tgrrVwJP/vZ1q5amjeHli3D0KpV/Nd401q0CH+npUu3Dt98s+147LRly0KSirX77vCb34R9p6Qh2ZTOBu79gOnAZ9EwjfD0vJxXPcUO6a6GKi9379rVfY893D/7LD3r/Oc/3Rs1cj/+ePe1axOXLykJjdcQ2kyWL09PHOmyaZP700+7d+vmfsop7v/5z9YG61QtWeK+337ubdu6v/9+etYZ69133Q84ILT73HKL+8aN6d9GdSoq3L/+2v2jj9zffNP92Wfdv//98HfefXf3O+8MVY4i2UC62iy2FAwPPWqRbPlsD+lMFuvXux9zTKifT3e7wJNPupu5n3pq9fXaGze633qre+PGIVmNH5/eGOqL+fPDwbNTJ/cvvkjPOjdvdv/LX8Lfdu+93adOTc960+H1191PPDH8V+65Z7hCLJkfFSKpSFuyAG4BWseMtwFuSmbl2RzSlSw2b3a/9NKwZ4qK0rLK7Tz0UFj/D3+4/S/ajz5y79s3zD/nnHCG05DNnBkuEDj0UPdly1Jb18qV7oMHh3178smpX3CQKVOnup9wwtakcc89ShqSOckmi2TaLN5198OqTHvH3XvtQLVYxqWrzeLee+Hqq+G66+Dmm9MQWDXuvht+9avQaP3oo2Han/8c6q3z80ND8XnnpedGuPpu0iQ49dRwk+HEibW7y7u0FH74Q5g/H/7wh9BIX5uLFbLptddCe9WUKeFighEjQlcs6bjL3T1cLDF9emh3WrYMdtkl3PG/yy7Vv69ufocO0KNHeNV3tn5JZ5tFKdAkZrwpMCeZTJTNIR1nFi+8EOqwzzgjfXXvNRk1Kvx6HDp0a/XDgAHpq3LZmRQVbT3bqqhIfrnNm91Hjw43U+61V7i8tb6ZMsX92GPD5997b/f77kt8GXJVy5eHmyx//3v3gQPD/T8hZbi3ahUuxd53X/d99gn7qX1799atw1ldfn5oa6ssX9NQUBC+y7/6VbhcurTUfcOGTOwVSRfSWA31G+AN4JJoeAP4TTIrz+aQarKYO9e9ZcvQoFzd/QPptnnz1hvRdtstVE/p7uXq3Xln2FfDhye3n1audD///LBM//6h0bw+e+UV96OPDp+nQwf3+++PnzQ2bQr3gTz8sPsll7gffHBoJ4Pw2q1bqGp95BH3OXOS/2G0eXOoNl27NuzbpUvDPi0rC/ec3H9/WO8RR4QEU5lAdt3V/bDDwkUa99zj/uqrqVcpSvokmyyS6kjQzAYAJwEGLAP2cvcrdvh8J4NSqYZauhT69Ak9rs6YEXplzRb3cM9Cnz6hHyap2TXXhG7Ub701VMtUJ7baadQouPbaul/tlAz3UC01cmS4n6VDh1Bl2qVLuJx5+nR4++1wWTCE+zr69g2XgPftG/o2a9ky83FWVIRqrlmzwlBSAu++Gy6xrtS5M/TsGYaCgtDnWuzw7bfJT9u8OVR/5eXFf61pXl4efOc7cMEF4bkuDa1vsrT2OmtmPYHzgR8BnwDPuvv9KUeZRrVNFhs3Qv/+oafWV18N/1RSd23eDD/+MTz9dLjn5KKLtp3vDo88AldeCW3awDPP1L1ni6SDe7gHZeRI+O9/w7S8vNDLcWViqLxHqK60IbiHe3Eqk0dlIvnoo+3vO2nSJAz5+Vvf1zSel7e1Mmzz5ppfq07btAmKi0OC7dABzj8/JI5s9BhdF6ScLMzsO8B5wGBgKfB34Bp375zOQNOltsni449Dl+K33RYOQlL3bdgQ7ux+5ZXwZMJTTgnTV68ON7U99RScdFJ43WOP3Maaae7hh87GjVBYmPzNnnXJ2rWwZs3Wg3/jxtlPcOvWwb//DU8+CS+8EM6MuncPSeP886vvYToVK1aEiwvefDO8b98+3Jy5++5b37dvH/6mmdwf6UgWm4HXgUvcfX40bYG775vWSNMklWqoVavCXbhSf6xaFaoMPvggJI1mzUK107x54Qqia69teNUJkh7l5aFrnyefDAdzs9BrwgUXhK76a3OscA9VopXPm3nzTXjvvTA9Ly88RmDNmvjL5udvm0Divd9nn9CTQm2kI1mcSTiz+B4wERgD/NXd62TNejaflCd1w5Il4XLaZcvCL8PWrUO103HH5Toy2VnMmxfOUJ96ChYsCAf1M84IieP73w9nQfGsXRuqtioTw7Rp4emVELqJ6dt36zNnKtuR1q0Lieqrr7YOseNV561fv3V7vXuHxFYb6ewbajfgDEJ11AnA48A4d3+pdqFlhpJFwzR/fqhG7NYt/BLc2audJDfcw0H/qafCBSnffBN+0Q8eHBLHnntue9bw7ruhKgtCB6KVjzCufIxxqhdbuIdq18okYhYukqmNjDxW1czaAj8EznX3OtCd3VZKFg1XRUW4MUwkGzZsCO0aTz4ZnlS5YcPWeU2bhoN2ZXLo2xfatctdrMnI2DO46yolCxHJtmXLQhf3a9aE5NC9e/VVU3VVsslCv8dERGqpTRsYOjTXUWTHTnCbkoiIZJqShYiIJKRkISIiCSlZiIhIQkoWIiKSkJKFiIgkpGQhIiIJZTRZmNkAM/vQzOab2XZPHzCzTmY2xczeNbNSMzslmt7FzNaZ2axo+Esm4xQRkZpl7KY8M2sEPAB8HygDZpjZeHefG1PsBmCsuz9oZgcDzwNdonkfu3vPTMUnIiLJy+SZRW9gvrsvcPcNhF5rB1Up40Dlc7taAYsyGI+IiNRSJpNFB+DzmPGyaFqsG4ELzKyMcFZxZcy8rlH11GtmdnS8DZjZMDMrNrPi8tjnNYqISFplMlnEe7ZT1V4LBwN/c/eOwCnAk2aWBywGOrn7YcCvgKfNbLsnB7v7aHcvdPfC9u3bpzl8ERGplMlkUQbsEzPeke2rmS4BxgK4+5tAPtDO3de7+9Jo+kzgY+A7GYxVRERqkMlkMQM4wMy6mtmuhKfuja9S5jPgRAAzO4iQLMrNrH3UQI6Z7QscACzIYKwiIlKDjF0N5e4VZjYceBFoBDzq7nPMbBRQ7O7jgV8DD5vZLwlVVBe5u5vZMcAoM6sANgGXufs3mYpVRERqpocfiYg0YMk+/Eh3cIuISEJKFiIikpCShYiIJKRkISIiCSlZiIhIQkoWIiKSkJKFiIgkpGQhIiIJKVmIiEhCShYiIpKQkoWIiCSkZCEiIgkpWYiISEJKFiIikpCShYiIJKRkISIiCSlZiIhIQkoWIiKSkJKFiIgkpGQhIiIJKVmIiEhCShYiIpKQkoWIiCSkZCEiIgkpWYiISEJKFiIikpCShYiIJKRkISIiCe2S6wBEpP7buHEjZWVlfPvtt7kORaqRn59Px44dady4ca2WV7IQkZSVlZXRokULunTpgpnlOhypwt1ZunQpZWVldO3atVbrUDWUiKTs22+/paCgQImijjIzCgoKUjrzU7IQkbRQoqjbUv37KFmIiEhCShYiknVFRdClC+TlhdeiotTWt3TpUnr27EnPnj3Zc8896dChw5bxDRs2JLWOiy++mA9N50i3AAARP0lEQVQ//LDGMg888ABFqQZbT6mBW0SyqqgIhg2DtWvD+KefhnGAIUNqt86CggJmzZoFwI033kjz5s255pprtinj7rg7eXnxfyM/9thjCbdzxRVX1C7AnYDOLEQkq66/fmuiqLR2bZiebvPnz6dbt25cdtll9OrVi8WLFzNs2DAKCws55JBDGDVq1Jay/fr1Y9asWVRUVNC6dWtGjBhBjx49OPLII/nqq68AuOGGG7jnnnu2lB8xYgS9e/fmwAMPZNq0aQCsWbOGs88+mx49ejB48GAKCwu3JLJYI0eO5IgjjtgSn7sD8NFHH3HCCSfQo0cPevXqxcKFCwG45ZZbOPTQQ+nRowfXZ2JnJZDRZGFmA8zsQzObb2Yj4szvZGZTzOxdMys1s1Ni5l0bLfehmZ2cyThFJHs++2zHpqdq7ty5XHLJJbz77rt06NCB2267jeLiYkpKSpg0aRJz587dbpkVK1Zw7LHHUlJSwpFHHsmjjz4ad93uzttvv80f//jHLYnnT3/6E3vuuSclJSWMGDGCd999N+6yv/jFL5gxYwazZ89mxYoVTJw4EYDBgwfzy1/+kpKSEqZNm8buu+/OhAkTeOGFF3j77bcpKSnh17/+dZr2TvIylizMrBHwADAQOBgYbGYHVyl2AzDW3Q8DzgP+HC17cDR+CDAA+HO0PhGp5zp12rHpqdpvv/044ogjtow/88wz9OrVi169evH+++/HTRZNmzZl4MCBABx++OFbft1XddZZZ21X5o033uC8884DoEePHhxyyCFxl508eTK9e/emR48evPbaa8yZM4dly5bx9ddf84Mf/AAIN9I1a9aMl19+maFDh9K0aVMA2rZtu+M7IkWZPLPoDcx39wXuvgEYAwyqUsaBltH7VsCi6P0gYIy7r3f3T4D50fpEpJ67+WZo1mzbac2ahemZsNtuu215P2/ePO69915eeeUVSktLGTBgQNx7D3bdddct7xs1akRFRUXcdTdp0mS7MpXVSTVZu3Ytw4cPZ9y4cZSWljJ06NAtccS7xNXdc35pciaTRQfg85jxsmharBuBC8ysDHgeuHIHlsXMhplZsZkVl5eXpytuEcmgIUNg9Gjo3BnMwuvo0bVv3N4RK1eupEWLFrRs2ZLFixfz4osvpn0b/fr1Y+zYsQDMnj077pnLunXryMvLo127dqxatYpnn30WgDZt2tCuXTsmTJgAhJsd165dS//+/XnkkUdYt24dAN98803a404kk8kiXhqsmnIHA39z947AKcCTZpaX5LK4+2h3L3T3wvbt26ccsIhkx5AhsHAhbN4cXrORKAB69erFwQcfTLdu3fjpT3/KUUcdlfZtXHnllXzxxRd0796dO++8k27dutGqVattyhQUFHDhhRfSrVs3zjzzTPr06bNlXlFREXfeeSfdu3enX79+lJeXc9pppzFgwAAKCwvp2bMnd999d9rjTsSSOWWq1YrNjgRudPeTo/FrAdz91pgyc4AB7v55NL4A6AtcElvWzF6M1vVmddsrLCz04uLijHwWEanZ+++/z0EHHZTrMOqEiooKKioqyM/PZ968efTv35958+axyy65v1Mh3t/JzGa6e2GiZTMZ/QzgADPrCnxBaLA+v0qZz4ATgb+Z2UFAPlAOjAeeNrO7gL2BA4C3MxiriEharF69mhNPPJGKigrcnYceeqhOJIpUZewTuHuFmQ0HXgQaAY+6+xwzGwUUu/t44NfAw2b2S0I100UeTnXmmNlYYC5QAVzh7psyFauISLq0bt2amTNn5jqMtMtounP35wkN17HTfhfzfi4Qt9LQ3W8GMnR9hIiI7AjdwS0iIgkpWYiISEJKFiIikpCShYjUe8cdd9x2N9jdc889/PznP69xuebNmwOwaNEizjnnnGrXneiy/HvuuYe1Mb0jnnLKKSxfvjyZ0OsNJQsRqfcGDx7MmDFjtpk2ZswYBg8enNTye++9N//4xz9qvf2qyeL555+ndevWtV5fXVT/L/4VkTrl6qshTo/cKenZE6KeweM655xzuOGGG1i/fj1NmjRh4cKFLFq0iH79+rF69WoGDRrEsmXL2LhxIzfddBODBm3bTd3ChQs57bTTeO+991i3bh0XX3wxc+fO5aCDDtrSxQbA5ZdfzowZM1i3bh3nnHMOv//977nvvvtYtGgRxx9/PO3atWPKlCl06dKF4uJi2rVrx1133bWl19pLL72Uq6++moULFzJw4ED69evHtGnT6NChA//617+2dBRYacKECdx0001s2LCBgoICioqK2GOPPVi9ejVXXnklxcXFmBkjR47k7LPPZuLEiVx33XVs2rSJdu3aMXny5LT9DZQsRKTeKygooHfv3kycOJFBgwYxZswYzj33XMyM/Px8xo0bR8uWLfn666/p27cvp59+erUd8z344IM0a9aM0tJSSktL6dWr15Z5N998M23btmXTpk2ceOKJlJaWctVVV3HXXXcxZcoU2rVrt826Zs6cyWOPPcZbb72Fu9OnTx+OPfZY2rRpw7x583jmmWd4+OGH+dGPfsSzzz7LBRdcsM3y/fr1Y/r06ZgZf/3rX7n99tu58847+cMf/kCrVq2YPXs2AMuWLaO8vJyf/vSnTJ06la5du6a9/yglCxFJq5rOADKpsiqqMllU/pp3d6677jqmTp1KXl4eX3zxBUuWLGHPPfeMu56pU6dy1VVXAdC9e3e6d+++Zd7YsWMZPXo0FRUVLF68mLlz524zv6o33niDM888c0vPt2eddRavv/46p59+Ol27dqVnz55A9d2gl5WVce6557J48WI2bNhA165dAXj55Ze3qXZr06YNEyZM4JhjjtlSJt3dmDf4Not0PwtYRHLjjDPOYPLkybzzzjusW7duyxlBUVER5eXlzJw5k1mzZrHHHnvE7ZY8Vryzjk8++YQ77riDyZMnU1payqmnnppwPTX1vVfZvTlU3w36lVdeyfDhw5k9ezYPPfTQlu3F67I8092YN+hkUfks4E8/BfetzwJWwhCpf5o3b85xxx3H0KFDt2nYXrFiBbvvvjuNGzdmypQpfPrppzWu55hjjqEoOgi89957lJaWAqF78912241WrVqxZMkSXnjhhS3LtGjRglWrVsVd13PPPcfatWtZs2YN48aN4+ijj076M61YsYIOHcLTGR5//PEt0/v378/999+/ZXzZsmUceeSRvPbaa3zyySdA+rsxb9DJIpvPAhaRzBs8eDAlJSVbnlQHMGTIEIqLiyksLKSoqIjvfve7Na7j8ssvZ/Xq1XTv3p3bb7+d3r3Dc9d69OjBYYcdxiGHHMLQoUO36d582LBhDBw4kOOPP36bdfXq1YuLLrqI3r1706dPHy699FIOO+ywpD/PjTfeyA9/+EOOPvrobdpDbrjhBpYtW0a3bt3o0aMHU6ZMoX379owePZqzzjqLHj16cO655ya9nWRkrIvybKtNF+V5eeGMoiqz0M++iCRHXZTXD6l0Ud6gzyyy/SxgEZH6qkEni2w/C1hEpL5q0Mkil88CFtnZ7CxV2jurVP8+Df4+iyFDlBxEUpWfn8/SpUspKCjI6OWbUjvuztKlS8nPz6/1Ohp8shCR1HXs2JGysjLKy8tzHYpUIz8/n44dO9Z6eSULEUlZ48aNt9w5LDunBt1mISIiyVGyEBGRhJQsREQkoZ3mDm4zKwdq7vQlt9oBX+c6iBoovtQovtQovtSkEl9nd2+fqNBOkyzqOjMrTuaW+lxRfKlRfKlRfKnJRnyqhhIRkYSULEREJCEli+wZnesAElB8qVF8qVF8qcl4fGqzEBGRhHRmISIiCSlZiIhIQkoWaWJm+5jZFDN738zmmNkv4pQ5zsxWmNmsaPhdDuJcaGazo+1v92hBC+4zs/lmVmpmvbIY24Ex+2aWma00s6urlMnqPjSzR83sKzN7L2ZaWzObZGbzotc21Sx7YVRmnpldmMX4/mhmH0R/v3Fm1rqaZWv8LmQwvhvN7IuYv+Ep1Sw7wMw+jL6LI7IY399jYltoZrOqWTYb+y/ucSUn30F315CGAdgL6BW9bwF8BBxcpcxxwL9zHOdCoF0N808BXgAM6Au8laM4GwFfEm4Yytk+BI4BegHvxUy7HRgRvR8B/F+c5doCC6LXNtH7NlmKrz+wS/T+/+LFl8x3IYPx3Qhck8Tf/2NgX2BXoKTq/1Om4qsy/07gdzncf3GPK7n4DurMIk3cfbG7vxO9XwW8D3TIbVS1Mgh4woPpQGsz2ysHcZwIfOzuOb0r392nAt9UmTwIeDx6/zhwRpxFTwYmufs37r4MmAQMyEZ87v6Su1dEo9OB2vdLnaJq9l8yegPz3X2Bu28AxhD2e1rVFJ+FB3P8CHgm3dtNVg3Hlax/B5UsMsDMugCHAW/FmX2kmZWY2QtmdkhWAwsceMnMZprZsDjzOwCfx4yXkZukdx7V/5Pmeh/u4e6LIfwzA7vHKVNX9uNQwpliPIm+C5k0PKome7SaKpS6sP+OBpa4+7xq5md1/1U5rmT9O6hkkWZm1hx4Frja3VdWmf0OoVqlB/An4Llsxwcc5e69gIHAFWZ2TJX58R5zltXrq81sV+B04P/FmV0X9mEy6sJ+vB6oAIqqKZLou5ApDwL7AT2BxYSqnqpyvv+AwdR8VpG1/ZfguFLtYnGm1XofKlmkkZk1JvxBi9z9n1Xnu/tKd18dvX8eaGxm7bIZo7svil6/AsYRTvdjlQH7xIx3BBZlJ7otBgLvuPuSqjPqwj4EllRWzUWvX8Upk9P9GDVmngYM8agCu6okvgsZ4e5L3H2Tu28GHq5mu7nef7sAZwF/r65MtvZfNceVrH8HlSzSJKrffAR4393vqqbMnlE5zKw3Yf8vzWKMu5lZi8r3hIbQ96oUGw/8JLoqqi+wovJ0N4uq/UWX630YGQ9UXllyIfCvOGVeBPqbWZuomqV/NC3jzGwA8L/A6e6+tpoyyXwXMhVfbBvYmdVsdwZwgJl1jc40zyPs92w5CfjA3cvizczW/qvhuJL972AmW/Ib0gD0I5zilQKzouEU4DLgsqjMcGAO4cqO6cD3shzjvtG2S6I4ro+mx8ZowAOEK1FmA4VZjrEZ4eDfKmZazvYhIWktBjYSfqldAhQAk4F50WvbqGwh8NeYZYcC86Ph4izGN59QV135PfxLVHZv4PmavgtZiu/J6LtVSjjo7VU1vmj8FMLVPx9nM75o+t8qv3MxZXOx/6o7rmT9O6juPkREJCFVQ4mISEJKFiIikpCShYiIJKRkISIiCSlZiIhIQkoWIgmY2SbbtjfctPWAamZdYns8Famrdsl1ACL1wDp375nrIERySWcWIrUUPc/g/8zs7WjYP5re2cwmRx3lTTazTtH0PSw8X6IkGr4XraqRmT0cPa/gJTNrGpW/yszmRusZk6OPKQIoWYgko2mVaqhzY+atdPfewP3APdG0+wndvHcndOJ3XzT9PuA1D50g9iLc+QtwAPCAux8CLAfOjqaPAA6L1nNZpj6cSDJ0B7dIAma22t2bx5m+EDjB3RdEnb196e4FZvY1oQuLjdH0xe7ezszKgY7uvj5mHV0Izxw4IBr/X6Cxu99kZhOB1YSedZ/zqANFkVzQmYVIarya99WViWd9zPtNbG1LPJXQT9fhwMyoJ1SRnFCyEEnNuTGvb0bvpxF6SQUYArwRvZ8MXA5gZo3MrGV1KzWzPGAfd58C/AZoDWx3diOSLfqlIpJYUzObFTM+0d0rL59tYmZvEX54DY6mXQU8amb/A5QDF0fTfwGMNrNLCGcQlxN6PI2nEfCUmbUi9AR8t7svT9snEtlBarMQqaWozaLQ3b/OdSwimaZqKBERSUhnFiIikpDOLEREJCElCxERSUjJQkREElKyEBGRhJQsREQkof8P/tfw5tN8DFEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.clf() # 그래프를 초기화합니다\n", "acc = history_dict['acc']\n", "val_acc = history_dict['val_acc']\n", "\n", "plt.plot(epochs, acc, 'bo', label='Training acc')\n", "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n", "plt.title('Training and validation accuracy')\n", "plt.xlabel('Epochs')\n", "plt.ylabel('Accuracy')\n", "plt.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "점선은 훈련 손실과 정확도이고 실선은 검증 손실과 정확도입니다. 신경망의 무작위한 초기화 때문에 사람마다 결과거 조금 다를 수 있습니다.\n", "\n", "여기에서 볼 수 있듯이 훈련 손실이 에포크마다 감소하고 훈련 정확도는 에포크마다 증가합니다. 경사 하강법 최적화를 사용했을 때 반복마다 최소화되는 것이 손실이므로 기대했던 대로입니다. 검증 손실과 정확도는 이와 같지 않습니다. 4번째 에포크에서 그래프가 역전되는 것 같습니다. 이것이 훈련 세트에서 잘 작동하는 모델이 처음 보는 데이터에 잘 작동하지 않을 수 있다고 앞서 언급한 경고의 한 사례입니다. 정확한 용어로 말하면 과대적합되었다고 합니다. 2번째 에포크 이후부터 훈련 데이터에 과도하게 최적화되어 훈련 데이터에 특화된 표현을 학습하므로 훈련 세트 이외의 데이터에는 일반화되지 못합니다.\n", "\n", "이런 경우에 과대적합을 방지하기 위해서 3번째 에포크 이후에 훈련을 중지할 수 있습니다. 일반적으로 4장에서 보게 될 과대적합을 완화하는 다양한 종류의 기술을 사용할 수 있습니다.\n", "\n", "처음부터 다시 새로운 신경망을 4번의 에포크 동안만 훈련하고 테스트 데이터에서 평가해 보겠습니다:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/4\n", "25000/25000 [==============================] - 1s 47us/step - loss: 0.4749 - acc: 0.8217\n", "Epoch 2/4\n", "25000/25000 [==============================] - 1s 42us/step - loss: 0.2658 - acc: 0.9097\n", "Epoch 3/4\n", "25000/25000 [==============================] - 1s 42us/step - loss: 0.1982 - acc: 0.9299\n", "Epoch 4/4\n", "25000/25000 [==============================] - 1s 42us/step - loss: 0.1679 - acc: 0.9404\n", "25000/25000 [==============================] - 1s 45us/step\n" ] } ], "source": [ "model = models.Sequential()\n", "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n", "model.add(layers.Dense(16, activation='relu'))\n", "model.add(layers.Dense(1, activation='sigmoid'))\n", "\n", "model.compile(optimizer='rmsprop',\n", " loss='binary_crossentropy',\n", " metrics=['accuracy'])\n", "\n", "model.fit(x_train, y_train, epochs=4, batch_size=512)\n", "results = model.evaluate(x_test, y_test)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0.3231545869159698, 0.87348]" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "아주 단순한 방식으로도 87%의 정확도를 달성했습니다. 최고 수준의 기법을 사용하면 95%에 가까운 성능을 얻을 수 있습니다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 훈련된 모델로 새로운 데이터에 대해 예측하기\n", "\n", "모델을 훈련시킨 후에 이를 실전 환경에서 사용하고 싶을 것입니다. `predict` 메서드를 사용해서 어떤 리뷰가 긍정일 확률을 예측할 수 있습니다:" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.1402615 ],\n", " [0.9997029 ],\n", " [0.29552558],\n", " ...,\n", " [0.07234979],\n", " [0.04342841],\n", " [0.48153415]], dtype=float32)" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.predict(x_test)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "여기에서처럼 이 모델은 어떤 샘플에 대해 확신을 가지고 있지만(0.99 또는 그 이상, 0.01 또는 그 이하) 어떤 샘플에 대해서는 확신이 부족합니다(0.6, 0.4). " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 추가 실험\n", "\n", "* 여기에서는 두 개의 은닉층을 사용했습니다. 한 개 또는 세 개의 은닉층을 사용하고 검증과 테스트 정확도에 어떤 영향을 미치는지 확인해 보세요.\n", "* 층의 은닉 유닛을 추가하거나 줄여 보세요: 32개 유닛, 64개 유닛 등\n", "* `binary_crossentropy` 대신에 `mse` 손실 함수를 사용해 보세요.\n", "* `relu` 대신에 `tanh` 활성화 함수(초창기 신경망에서 인기 있었던 함수입니다)를 사용해 보세요.\n", "\n", "다음 실험을 진행하면 여기에서 선택한 구조가 향상의 여지는 있지만 어느 정도 납득할 만한 수준이라는 것을 알게 것입니다!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 정리\n", "\n", "다음은 이 예제에서 배운 것들입니다:\n", "\n", "* 원본 데이터를 신경망에 텐서로 주입하기 위해서는 꽤 많은 전처리가 필요합니다. 단어 시퀀스는 이진 벡터로 인코딩될 수 있고 다른 인코딩 방식도 있습니다.\n", "* `relu` 활성화 함수와 함께 `Dense` 층을 쌓은 네트워크는 (감성 분류를 포함하여) 여러 종류의 문제에 적용할 수 있어서 앞으로 자주 사용하게 될 것입니다.\n", "* (출력 클래스가 두 개인) 이진 분류 문제에서 네트워크는 하나의 유닛과 `sigmoid` 활성화 함수를 가진 `Dense` 층으로 끝나야 합니다. 이 신경망의 출력은 확률을 나타내는 0과 1 사이의 스칼라 값입니다.\n", "* 이진 분류 문제에서 이런 스칼라 시그모이드 출력에 대해 사용할 손실 함수는 `binary_crossentropy`입니다.\n", "* `rmsprop` 옵티마이저는 문제에 상관없이 일반적으로 충분히 좋은 선택입니다. 걱정할 거리가 하나 줄은 셈입니다.\n", "* 훈련 데이터에 대해 성능이 향상됨에 따라 신경망은 과대적합되기 시작하고 이전에 본적 없는 데이터에서는 결과가 점점 나빠지게 됩니다. 항상 훈련 세트 이외의 데이터에서 성능을 모니터링해야 합니다." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" } }, "nbformat": 4, "nbformat_minor": 2 }