{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.4'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 단어 임베딩 사용하기\n",
    "\n",
    "이 노트북은 [케라스 창시자에게 배우는 딥러닝](https://tensorflow.blog/케라스-창시자에게-배우는-딥러닝/) 책의 6장 1절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다. 이 노트북의 설명은 케라스 버전 2.2.2에 맞추어져 있습니다. 케라스 최신 버전이 릴리스되면 노트북을 다시 테스트하기 때문에 설명과 코드의 결과가 조금 다를 수 있습니다.\n",
    "\n",
    "---\n",
    "\n",
    "단어와 벡터를 연관짓는 강력하고 인기 있는 또 다른 방법은 단어 임베딩이라는 밀집 단어 벡터를 사용하는 것입니다. 원-핫 인코딩으로 만든 벡터는 희소하고(대부분 0으로 채워집니다) 고차원입니다(어휘 사전에 있는 단어의 수와 차원이 같습니다). 반면 단어 임베딩은 저차원의 실수형 벡터입니다(희소 벡터의 반대인 밀집 벡터입니다). 그림 6-2를 참고하세요. 원-핫 인코딩으로 얻은 단어 벡터와 달리 단어 임베딩은 데이터로부터 학습됩니다. 보통 256차원, 512차원 또는 큰 어휘 사전을 다룰 때는 1,024차원의 단어 임베딩을 사용합니다. 반면 원-핫 인코딩은 (20,000개의 토큰으로 이루어진 어휘 사전을 만들려면) 20,000차원 또는 그 이상의 벡터일 경우가 많습니다. 따라서 단어 임베딩이 더 많은 정보를 적은 차원에 저장합니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![word embeddings vs. one hot encoding](https://s3.amazonaws.com/book.keras.io/img/ch6/word_embeddings.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "단어 임베딩을 만드는 방법은 두 가지입니다.\n",
    "\n",
    "* (문서 분류나 감성 예측과 같은) 관심 대상인 문제와 함께 단어 임베딩을 학습합니다. 이런 경우에는 랜덤한 단어 벡터로 시작해서 신경망의 가중치를 학습하는 것과 같은 방식으로 단어 벡터를 학습합니다.\n",
    "* 풀려는 문제가 아니고 다른 머신 러닝 작업에서 미리 계산된 단어 임베딩을 로드합니다. 이를 사전 훈련된 단어 임베딩이라고 합니다.\n",
    "\n",
    "두 가지 모두 살펴보겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `Embedding` 층을 사용해 단어 임베딩 학습하기\n",
    "\n",
    "단어와 밀집 벡터를 연관짓는 가장 간단한 방법은 랜덤하게 벡터를 선택하는 것입니다. 이 방식의 문제점은 임베딩 공간이 구조적이지 않다는 것입니다. 예를 들어 accurate와 exact 단어가 대부분 문장에서 비슷한 의미로 사용되지만 완전히 다른 임베딩을 가지게 됩니다. 심층 신경망이 이런 임의의 구조적이지 않은 임베딩 공간을 이해하기는 어렵습니다.\n",
    "\n",
    "단어 벡터 사이에 조금 더 추상적이고 기하학적인 관계를 얻으려면 단어 사이에 있는 의미 관계를 반영해야 합니다. 단어 임베딩은 언어를 기하학적 공간에 매핑하는 것입니다. 예를 들어 잘 구축된 임베딩 공간에서는 동의어가 비슷한 단어 벡터로 임베딩될 것입니다. 일반적으로 두 단어 벡터 사이의 거리(L2 거리)는 이 단어 사이의 의미 거리와 관계되어 있습니다(멀리 떨어진 위치에 임베딩된 단어의 의미는 서로 다르고 반면 비슷한 단어들은 가까이 임베딩됩니다). 거리외에 임베딩 공간의 특정 방향도 의미를 가질 수 있습니다.\n",
    "\n",
    "[...]\n",
    "\n",
    "실제 단어 임베딩 공간에서 의미 있는 기하학적 변환의 일반적인 예는 '성별' 벡터와 '복수(plural)' 벡터입니다. 예를 들어 'king' 벡터에 'female' 벡터를 더하면 'queen' 벡터가 됩니다. 'plural' 벡터를 더하면 'kings'가 됩니다. 단어 임베딩 공간은 전형적으로 이런 해석 가능하고 잠재적으로 유용한 수천 개의 벡터를 특성으로 가집니다.\n",
    "\n",
    "사람의 언어를 완벽하게 매핑해서 어떤 자연어 처리 작업에도 사용할 수 있는 이상적인 단어 임베딩 공간이 있을까요? 아마도 가능하겠지만 아직까지 이런 종류의 공간은 만들지 못했습니다. 사람의 언어에도 그런 것은 없습니다. 세상에는 많은 다른 언어가 있고 언어는 특정 문화와 환경을 반영하기 때문에 서로 동일하지 않습니다. 실제로 좋은 단어 임베딩 공간을 만드는 것은 문제에 따라 크게 달라집니다. 영어로 된 영화 리뷰 감성 분석 모델을 위한 완벽한 단어 임베딩 공간은 영어로 된 법률 문서 분류 모델을 위한 완벽한 임베딩 공간과 다를 것 같습니다. 특정 의미 관계의 중요성이 작업에 따라 다르기 때문입니다.\n",
    "\n",
    "따라서 새로운 작업에는 새로운 임베딩을 학습하는 것이 타당합니다. 다행히 역전파를 사용해 쉽게 만들 수 있고 케라스를 사용하면 더 쉽습니다. `Embedding` 층의 가중치를 학습하면 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Embedding\n",
    "\n",
    "# Embedding 층은 적어도 두 개의 매개변수를 받습니다.\n",
    "# 가능한 토큰의 개수(여기서는 1,000으로 단어 인덱스 최댓값 + 1입니다)와 임베딩 차원(여기서는 64)입니다\n",
    "embedding_layer = Embedding(1000, 64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Embedding` 층을 (특정 단어를 나타내는) 정수 인덱스를 밀집 벡터로 매핑하는 딕셔너리로 이해하는 것이 가장 좋습니다. 정수를 입력으로 받아 내부 딕셔너리에서 이 정수에 연관된 벡터를 찾아 반환합니다. 딕셔너리 탐색은 효율적으로 수행됩니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Embedding` 층은 크기가 `(samples, sequence_length)`인 2D 정수 텐서를 입력으로 받습니다. 각 샘플은 정수의 시퀀스입니다. 가변 길이의 시퀀스를 임베딩할 수 있습니다. 예를 들어 위 예제의 `Embedding` 층에 `(32, 10)` 크기의 배치(길이가 10인 시퀀스 32개로 이루어진 배치)나 `(64, 15)` 크기의 배치(길이가 15인 시퀀스 64개로 이루어진 배치)를 주입할 수 있습니다. 배치에 있는 모든 시퀀스는 길이가 같아야 하므로(하나의 텐서에 담아야 하기 때문에) 작은 길이의 시퀀스는 0으로 패딩되고 길이가 더 긴 시퀀스는 잘립니다.\n",
    "\n",
    "`Embedding` 층은 크기가 `(samples, sequence_length, embedding_dimensionality)`인 3D 실수형 텐서를 반환합니다. 이런 3D 텐서는 RNN 층이나 1D 합성곱 층에서 처리됩니다(둘 다 이어지는 절에서 소개하겠습니다).\n",
    "\n",
    "`Embedding` 층의 객체를 생성할 때 가중치(토큰 벡터를 위한 내부 딕셔너리)는 다른 층과 마찬가지로 랜덤하게 초기화됩니다. 훈련하면서 이 단어 벡터는 역전파를 통해 점차 조정되어 이어지는 모델이 사용할 수 있도록 임베팅 공간을 구성합니다. 훈련이 끝나면 임베딩 공간은 특정 문제에 특화된 구조를 많이 가지게 됩니다.\n",
    "\n",
    "이를 익숙한 IMDB 영화 리뷰 감성 예측 문제에 적용해 보죠. 먼저 데이터를 준비합니다. 영화 리뷰에서 가장 빈도가 높은 10,000개의 단어를 추출하고(처음 이 데이터셋으로 작업했던 것과 동일합니다) 리뷰에서 20개 단어 이후는 버립니다. 이 네트워크는 10,000개의 단어에 대해 8 차원의 임베딩을 학습하여 정수 시퀀스 입력(2D 정수 텐서)를 임베딩 시퀀스(3D 실수형 텐서)로 바꿀 것입니다. 그 다음 이 텐서를 2D로 펼쳐서 분류를 위한 `Dense` 층을 훈련하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras import preprocessing\n",
    "\n",
    "# 특성으로 사용할 단어의 수\n",
    "max_features = 10000\n",
    "# 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용합니다)\n",
    "maxlen = 20\n",
    "\n",
    "# 정수 리스트로 데이터를 로드합니다.\n",
    "(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "\n",
    "# 리스트를 (samples, maxlen) 크기의 2D 정수 텐서로 변환합니다.\n",
    "x_train = preprocessing.sequence.pad_sequences(x_train, maxlen=maxlen)\n",
    "x_test = preprocessing.sequence.pad_sequences(x_test, maxlen=maxlen)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_2 (Embedding)      (None, 20, 8)             80000     \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 160)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 161       \n",
      "=================================================================\n",
      "Total params: 80,161\n",
      "Trainable params: 80,161\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 1s 74us/step - loss: 0.6759 - acc: 0.6050 - val_loss: 0.6398 - val_acc: 0.6814\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 1s 46us/step - loss: 0.5657 - acc: 0.7427 - val_loss: 0.5467 - val_acc: 0.7206\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 1s 46us/step - loss: 0.4752 - acc: 0.7808 - val_loss: 0.5113 - val_acc: 0.7384\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 1s 45us/step - loss: 0.4263 - acc: 0.8077 - val_loss: 0.5008 - val_acc: 0.7452\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 1s 45us/step - loss: 0.3930 - acc: 0.8258 - val_loss: 0.4981 - val_acc: 0.7538\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 1s 45us/step - loss: 0.3668 - acc: 0.8395 - val_loss: 0.5014 - val_acc: 0.7530\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 1s 45us/step - loss: 0.3435 - acc: 0.8533 - val_loss: 0.5052 - val_acc: 0.7520\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 1s 45us/step - loss: 0.3223 - acc: 0.8657 - val_loss: 0.5132 - val_acc: 0.7486\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 1s 46us/step - loss: 0.3022 - acc: 0.8766 - val_loss: 0.5213 - val_acc: 0.7490\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 1s 45us/step - loss: 0.2839 - acc: 0.8860 - val_loss: 0.5303 - val_acc: 0.7466\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Flatten, Dense, Embedding\n",
    "\n",
    "model = Sequential()\n",
    "# 나중에 임베딩된 입력을 Flatten 층에서 펼치기 위해 Embedding 층에 input_length를 지정합니다.\n",
    "model.add(Embedding(10000, 8, input_length=maxlen))\n",
    "# Embedding 층의 출력 크기는 (samples, maxlen, 8)가 됩니다.\n",
    "\n",
    "# 3D 임베딩 텐서를 (samples, maxlen * 8) 크기의 2D 텐서로 펼칩니다.\n",
    "model.add(Flatten())\n",
    "\n",
    "# 분류기를 추가합니다.\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "model.summary()\n",
    "\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=32,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "약 75% 정도의 검증 정확도가 나옵니다. 리뷰에서 20개의 단어만 사용한 것치고 꽤 좋은 결과입니다. 하지만 임베딩 시퀀스를 펼치고 하나의 Dense 층을 훈련했으므로 입력 시퀀스에 있는 각 단어를 독립적으로 다루었습니다. 단어 사이의 관계나 문장의 구조를 고려하지 않았습니다(예를 들어 이 모델은 “this movie is a bomb”와 “this movie is the bomb”를 부정적인 리뷰로 동일하게 다룰 것입니다). 각 시퀀스 전체를 고려한 특성을 학습하도록 임베딩 층 위에 순환 층이나 1D 합성곱 층을 추가하는 것이 좋습니다. 다음 절에서 이에 관해 집중적으로 다루겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 사전 훈련된 단어 임베딩 사용하기\n",
    "\n",
    "이따금 훈련 데이터가 부족하면 작업에 맞는 단어 임베딩을 학습할 수 없습니다. 이럴 땐 어떻게 해야 할까요?\n",
    "\n",
    "풀려는 문제와 함께 단어 임베딩을 학습하는 대신에 미리 계산된 임베딩 공간에서 임베딩 벡터를 로드할 수 있습니다. 이런 임베딩 공간은 뛰어난 구조와 유용한 성질을 가지고 있어서 언어 구조의 일반적인 측면을 잡아낼 수 있습니다. 자연어 처리에서 사전 훈련된 단어 임베딩을 사용하는 이유는 이미지 분류 문제에서 사전 훈련된 컨브넷을 사용하는 이유와 거의 동일합니다. 충분한 데이터가 없어서 자신만의 좋은 특성을 학습하지 못하지만 꽤 일반적인 특성이 필요할 때입니다. 이런 경우에는 다른 문제에서 학습한 특성을 재사용하는 것이 합리적입니다.\n",
    "\n",
    "단어 임베딩은 일반적으로 (문장이나 문서에 같이 등장하는 단어를 관찰하는) 단어 출현 통계를 사용하여 계산됩니다. 여기에는 여러 가지 기법이 사용되는데 신경망을 사용하는 것도 있고 그렇지 않은 방법도 있습니다. 단어를 위해 밀집된 저차원 임베딩 공간을 비지도 학습 방법으로 계산하는 아이디어는 요슈아 벤지오 등이 2000년대 초에 조사했습니다. 연구나 산업 애플리케이션에 적용되기 시작된 것은 Word2vec 알고리즘이 등장한 이후입니다. 이 알고리즘은 2013년 구글의 토마스 미코로프가 개발하였으며 가장 유명하고 성공적인 단어 임베딩 방법입니다. Word2vec의 차원은 성별 같은 구체적인 의미가 있는 속성을 잡아냅니다.\n",
    "\n",
    "케라스의 `Embedding` 층을 위해 내려받을 수 있는 미리 계산된 단어 임베딩 데이터베이스가 여럿 있습니다. Word2vec은 그 중 하나입니다. 인기 있는 또 다른 하나는 2014년 스탠포드 대학의 연구자들이 개발한 GloVe(Global Vectors for Word Representation)입니다. 이 임베딩 기법은 단어의 동시 출현 통계를 기록한 행렬을 분해하는 기법을 사용합니다. 이 개발자들은 위키피디아 데이터와 커먼 크롤 데이터에서 가져온 수백만 개의 영어 토큰에 대해서 임베딩을 미리 계산해 놓았습니다.\n",
    "\n",
    "GloVe 임베딩을 케라스 모델에 어떻게 사용하는지 알아보죠. Word2vec 임베딩이나 다른 단어 임베딩 데이터베이스도 방법은 같습니다. 앞서 보았던 텍스트 토큰화 기법도 다시 살펴보겠습니다. 원본 텍스트에서 시작해서 완전한 모델을 구성해 보겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 모든 내용을 적용하기: 원본 텍스트에서 단어 임베딩까지\n",
    "\n",
    "앞서 만들었던 것과 비슷한 모델을 사용하겠습니다. 문장들을 벡터의 시퀀스로 임베딩하고 펼친 다음 그 위에 `Dense` 층을 훈련합니다. 여기서는 사전 훈련된 단어 임베딩을 사용하겠습니다. 케라스에 포함된 IMDB 데이터는 미리 토큰화가 되어 있습니다. 이를 사용하는 대신 원본 텍스트 데이터를 다운로딩해서 처음부터 시작하겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 원본 IMDB 텍스트 다운로드하기\n",
    "\n",
    "먼저 http://mng.bz/0tIo 에서 IMDB 원본 데이터셋을 다운로드하고 압축을 해제합니다.\n",
    "\n",
    "훈련용 리뷰 하나를 문자열 하나로 만들어 훈련 데이터를 문자열의 리스트로 구성해 보죠. 리뷰 레이블(긍정/부정)도 `labels` 리스트로 만들겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "imdb_dir = './datasets/aclImdb'\n",
    "train_dir = os.path.join(imdb_dir, 'train')\n",
    "\n",
    "labels = []\n",
    "texts = []\n",
    "\n",
    "for label_type in ['neg', 'pos']:\n",
    "    dir_name = os.path.join(train_dir, label_type)\n",
    "    for fname in os.listdir(dir_name):\n",
    "        if fname[-4:] == '.txt':\n",
    "            f = open(os.path.join(dir_name, fname), encoding='utf8')\n",
    "            texts.append(f.read())\n",
    "            f.close()\n",
    "            if label_type == 'neg':\n",
    "                labels.append(0)\n",
    "            else:\n",
    "                labels.append(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 데이터 토큰화\n",
    "\n",
    "이전 절에서 소개한 개념을 사용해 텍스트를 벡터로 만들고 훈련 세트와 검증 세트로 나누겠습니다. 사전 훈련된 단어 임베딩은 훈련 데이터가 부족한 문제에 특히 유용합니다(그렇지 않으면 문제에 특화된 임베딩이 훨씬 성능이 좋습니다). 그래서 다음과 같이 훈련 데이터를 처음 200개의 샘플로 제한합니다. 이 모델은 200개의 샘플을 학습한 후에 영화 리뷰를 분류할 것입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "88582개의 고유한 토큰을 찾았습니다.\n",
      "데이터 텐서의 크기: (25000, 100)\n",
      "레이블 텐서의 크기: (25000,)\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing.text import Tokenizer\n",
    "from keras.preprocessing.sequence import pad_sequences\n",
    "import numpy as np\n",
    "\n",
    "maxlen = 100  # 100개 단어 이후는 버립니다\n",
    "training_samples = 200  # 훈련 샘플은 200개입니다\n",
    "validation_samples = 10000  # 검증 샘플은 10,000개입니다\n",
    "max_words = 10000  # 데이터셋에서 가장 빈도 높은 10,000개의 단어만 사용합니다\n",
    "\n",
    "tokenizer = Tokenizer(num_words=max_words)\n",
    "tokenizer.fit_on_texts(texts)\n",
    "sequences = tokenizer.texts_to_sequences(texts)\n",
    "\n",
    "word_index = tokenizer.word_index\n",
    "print('%s개의 고유한 토큰을 찾았습니다.' % len(word_index))\n",
    "\n",
    "data = pad_sequences(sequences, maxlen=maxlen)\n",
    "\n",
    "labels = np.asarray(labels)\n",
    "print('데이터 텐서의 크기:', data.shape)\n",
    "print('레이블 텐서의 크기:', labels.shape)\n",
    "\n",
    "# 데이터를 훈련 세트와 검증 세트로 분할합니다.\n",
    "# 샘플이 순서대로 있기 때문에 (부정 샘플이 모두 나온 후에 긍정 샘플이 옵니다) \n",
    "# 먼저 데이터를 섞습니다.\n",
    "indices = np.arange(data.shape[0])\n",
    "np.random.shuffle(indices)\n",
    "data = data[indices]\n",
    "labels = labels[indices]\n",
    "\n",
    "x_train = data[:training_samples]\n",
    "y_train = labels[:training_samples]\n",
    "x_val = data[training_samples: training_samples + validation_samples]\n",
    "y_val = labels[training_samples: training_samples + validation_samples]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GloVe 단어 임베딩 내려받기\n",
    "\n",
    "https://nlp.stanford.edu/projects/glove 에서 2014년 영문 위키피디아를 사용해 사전에 계산된 임베딩을 내려받습니다. 이 파일의 이름은 glove.6B.zip이고 압축 파일 크기는 823MB입니다. 400,000만개의 단어(또는 단어가 아닌 토큰)에 대한 100차원의 임베딩 벡터를 포함하고 있습니다. datasets 폴더 아래에 파일 압축을 해제합니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 임베딩 전처리\n",
    "\n",
    "압축 해제한 파일(.txt 파일)을 파싱하여 단어(즉 문자열)와 이에 상응하는 벡터 표현(즉 숫자 벡터)를 매핑하는 인덱스를 만듭니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "400000개의 단어 벡터를 찾았습니다.\n"
     ]
    }
   ],
   "source": [
    "glove_dir = './datasets/'\n",
    "\n",
    "embeddings_index = {}\n",
    "f = open(os.path.join(glove_dir, 'glove.6B.100d.txt'), encoding=\"utf8\")\n",
    "for line in f:\n",
    "    values = line.split()\n",
    "    word = values[0]\n",
    "    coefs = np.asarray(values[1:], dtype='float32')\n",
    "    embeddings_index[word] = coefs\n",
    "f.close()\n",
    "\n",
    "print('%s개의 단어 벡터를 찾았습니다.' % len(embeddings_index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그다음 `Embedding` 층에 주입할 수 있도록 임베딩 행렬을 만듭니다. 이 행렬의 크기는 `(max_words, embedding_dim)`이어야 합니다. 이 행렬의 `i`번째 원소는 (토큰화로 만든) 단어 인덱스의 `i`번째 단어에 상응하는 `embedding_dim` 차원 벡터입니다. 인덱스 `0`은 어떤 단어나 토큰도 아닐 경우를 나타냅니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_dim = 100\n",
    "\n",
    "embedding_matrix = np.zeros((max_words, embedding_dim))\n",
    "for word, i in word_index.items():\n",
    "    embedding_vector = embeddings_index.get(word)\n",
    "    if i < max_words:\n",
    "        if embedding_vector is not None:\n",
    "            # 임베딩 인덱스에 없는 단어는 모두 0이 됩니다.\n",
    "            embedding_matrix[i] = embedding_vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델 정의하기\n",
    "\n",
    "이전과 동일한 구조의 모델을 사용하겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_3 (Embedding)      (None, 100, 100)          1000000   \n",
      "_________________________________________________________________\n",
      "flatten_2 (Flatten)          (None, 10000)             0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 32)                320032    \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,320,065\n",
      "Trainable params: 1,320,065\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Embedding, Flatten, Dense\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_words, embedding_dim, input_length=maxlen))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(32, activation='relu'))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델에 GloVe 임베딩 로드하기\n",
    "\n",
    "`Embedding` 층은 하나의 가중치 행렬을 가집니다. 이 행렬은 2D 부동 소수 행렬이고 각 `i`번째 원소는 `i`번째 인덱스에 상응하는 단어 벡터입니다. 간단하네요. 모델의 첫 번째 층인 `Embedding` 층에 준비된 GloVe 행렬을 로드하세요:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.layers[0].set_weights([embedding_matrix])\n",
    "model.layers[0].trainable = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "추가적으로 `Embedding` 층을 동결합니다(`trainable` 속성을 `False`로 설정합니다). 사전 훈련된 컨브넷 특성을 사용할 때와 같은 이유입니다. 모델의 일부는 (`Embedding` 층처럼) 사전 훈련되고 다른 부분은 (최상단 분류기처럼) 랜덤하게 초기화되었다면 훈련하는 동안 사전 훈련된 부분이 업데이트되면 안됩니다. 이미 알고 있던 정보를 모두 잃게 됩니다. 랜덤하게 초기화된 층에서 대량의 그래디언트 업데이트가 발생하면 이미 학습된 특성을 오염시키기 때문입니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델 훈련과 평가\n",
    "\n",
    "모델을 컴파일하고 훈련합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 200 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      "200/200 [==============================] - 0s 2ms/step - loss: 1.6733 - acc: 0.5400 - val_loss: 0.6907 - val_acc: 0.5338\n",
      "Epoch 2/10\n",
      "200/200 [==============================] - 0s 974us/step - loss: 0.6077 - acc: 0.6700 - val_loss: 0.8459 - val_acc: 0.5060\n",
      "Epoch 3/10\n",
      "200/200 [==============================] - 0s 988us/step - loss: 0.4950 - acc: 0.7650 - val_loss: 0.6870 - val_acc: 0.5583\n",
      "Epoch 4/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.3349 - acc: 0.8950 - val_loss: 0.8019 - val_acc: 0.5098\n",
      "Epoch 5/10\n",
      "200/200 [==============================] - 0s 966us/step - loss: 0.2251 - acc: 0.9600 - val_loss: 0.7647 - val_acc: 0.5462\n",
      "Epoch 6/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.3254 - acc: 0.8150 - val_loss: 0.6875 - val_acc: 0.5835\n",
      "Epoch 7/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.1144 - acc: 0.9950 - val_loss: 1.0806 - val_acc: 0.5026\n",
      "Epoch 8/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.1141 - acc: 0.9800 - val_loss: 0.7699 - val_acc: 0.5488\n",
      "Epoch 9/10\n",
      "200/200 [==============================] - 0s 982us/step - loss: 0.0671 - acc: 0.9950 - val_loss: 0.7842 - val_acc: 0.5610\n",
      "Epoch 10/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0814 - acc: 0.9850 - val_loss: 1.6252 - val_acc: 0.5076\n"
     ]
    }
   ],
   "source": [
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=32,\n",
    "                    validation_data=(x_val, y_val))\n",
    "model.save_weights('pre_trained_glove_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 모델의 성능을 그래프로 그려 보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl4VdW5x/Hvy2QMs4DXAkJQKaJAIEYcwBkV1EIdqtDYihapA1qHXq8VW6wV7XXWaq3UsTVKqSP1Uq0z2joAiiig4gAYQAREBIJIyHv/WCfJIWQ4CUn2yc7v8zx5cvY+6+z9nn2S96y91tprm7sjIiLx0izqAEREpO4puYuIxJCSu4hIDCm5i4jEkJK7iEgMKbmLiMSQknuMmVlzM9tgZj3qsmyUzGwvM6vz8btmNszMFictf2hmh6RSthb7usfMrqjt60VS0SLqAKSMmW1IWswENgNbE8s/d/f8mmzP3bcCbeq6bFPg7n3qYjtmNg443d0PT9r2uLrYtkhVlNzTiLuXJtdEzXCcuz9fWXkza+HuRQ0Rm0h19PeYXtQs04iY2TVm9jcze8TM1gOnm9lBZvaGmX1tZivM7HYza5ko38LM3MyyEssPJZ7/p5mtN7PXzaxXTcsmnh9hZh+Z2Toz+4OZ/dvMxlYSdyox/tzMPjaztWZ2e9Jrm5vZLWa2xsw+AYZXcXyuNLOp5dbdaWY3Jx6PM7OFiffzSaJWXdm2Cszs8MTjTDP7ayK2+cB+Fez308R255vZyMT6/sAdwCGJJq/VScf2qqTXn5N472vM7Ekz+14qx6Ymx7kkHjN73sy+MrMvzOyypP38OnFMvjGz2WbWtaImMDN7reRzThzPmYn9fAVcaWa9zeylxHtZnThu7ZNe3zPxHlclnr/NzDISMfdNKvc9Mys0s06VvV+phrvrJw1/gMXAsHLrrgG+A35A+GLeGdgfOIBwFrYH8BEwIVG+BeBAVmL5IWA1kAu0BP4GPFSLsrsC64FRiecuAbYAYyt5L6nE+BTQHsgCvip578AEYD7QHegEzAx/thXuZw9gA9A6adtfArmJ5R8kyhhwJLAJGJB4bhiwOGlbBcDhicc3Ai8DHYGewIJyZU8Fvpf4TH6ciOG/Es+NA14uF+dDwFWJx8ckYhwIZAB/BF5M5djU8Di3B1YCvwB2AtoBgxPP/Qp4F+ideA8DgV2Avcofa+C1ks858d6KgHOB5oS/x+8DRwGtEn8n/wZuTHo/7yeOZ+tE+SGJ56YAk5P2cynwRNT/h435J/IA9FPJB1N5cn+xmtf9Evh74nFFCftPSWVHAu/XouxZwKtJzxmwgkqSe4oxHpj0/OPALxOPZxKap0qeO658wim37TeAHycejwA+qqLs08D5icdVJfelyZ8FcF5y2Qq2+z5wfOJxdcn9QeDapOfaEfpZuld3bGp4nH8CzK6k3Ccl8ZZbn0py/7SaGE4BZiUeHwJ8ATSvoNwQ4DPAEstzgZPq+v+qKf2oWabx+Tx5wcz2NrP/S5xmfwNcDXSu4vVfJD0upOpO1MrKdk2Ow8N/Y0FlG0kxxpT2BSypIl6Ah4Exicc/Bko7oc3sBDN7M9Es8TWh1lzVsSrxvapiMLOxZvZuomnha2DvFLcL4f2Vbs/dvwHWAt2SyqT0mVVznHcHPq4kht0JCb42yv897mZm08xsWSKGB8rFsNhD5/023P3fhLOAoWbWD+gB/F8tYxLU5t4YlR8GeDehpriXu7cDfkOoSdenFYSaJQBmZmybjMrbkRhXEJJCieqGav4NGGZm3QnNRg8nYtwZeBS4jtBk0gH4V4pxfFFZDGa2B3AXoWmiU2K7HyRtt7phm8sJTT0l22tLaP5ZlkJc5VV1nD8H9qzkdZU9tzERU2bSut3KlSn//v6XMMqrfyKGseVi6GlmzSuJ4y/A6YSzjGnuvrmScpICJffGry2wDtiY6JD6eQPs82kgx8x+YGYtCO24XeopxmnARWbWLdG59j9VFXb3lYSmg/uBD919UeKpnQjtwKuArWZ2AqFtONUYrjCzDhauA5iQ9FwbQoJbRfieG0eouZdYCXRP7tgs5xHgZ2Y2wMx2Inz5vOrulZ4JVaGq4zwd6GFmE8yslZm1M7PBiefuAa4xsz0tGGhmuxC+1L4gdNw3N7PxJH0RVRHDRmCdme1OaBoq8TqwBrjWQif1zmY2JOn5vxKacX5MSPSyA5TcG79LgTMIHZx3E2qu9SqRQE8Dbib8s+4JvEOosdV1jHcBLwDvAbMIte/qPExoQ384KeavgYuBJwidkqcQvqRSMYlwBrEY+CdJicfd5wG3A28lyuwNvJn02ueARcBKM0tuXil5/TOE5pMnEq/vAeSlGFd5lR5nd18HHA2cTOjA/Qg4LPH0DcCThOP8DaFzMyPR3HY2cAWhc32vcu+tIpOAwYQvmenAY0kxFAEnAH0JtfilhM+h5PnFhM/5O3f/Tw3fu5RT0nkhUmuJ0+zlwCnu/mrU8UjjZWZ/IXTSXhV1LI2dLmKSWjGz4YTT7G8JQ+mKCLVXkVpJ9F+MAvpHHUscqFlGamso8CnhdH048EN1gEltmdl1hLH217r70qjjiQM1y4iIxJBq7iIiMRRZm3vnzp09Kysrqt2LiDRKc+bMWe3uVQ09BiJM7llZWcyePTuq3YuINEpmVt1V2oCaZUREYknJXUQkhpTcRURiqNo2dzO7j3DJ8Jfu3q+C5w24jTAVayFhOtC3axPMli1bKCgo4Ntvv63Ny6WBZGRk0L17d1q2rGy6FBGJWiodqg8Q7iZT2UQ+IwiT/Pcm3CjgrsTvGisoKKBt27ZkZWURvjMk3bg7a9asoaCggF69elX/AhGJRLXNMu4+kzDRUmVGAX/x4A2gQ8ltwmrq22+/pVOnTkrsaczM6NSpk86u0lx+PmRlQbNm4Xd+jW6tHj9N8XjUxVDIbmw7YX9BYt2K8gUTU4aOB+jRo+JpuZXY058+o/SWnw/jx0NhYVhesiQsA+TVdr7JRqypHo+66FCt6D+9wjkN3H2Ku+e6e26XLtWOwReRWpg4sSyRlSgsDOuboqZ6POoiuRew7V1quhOmf2101qxZw8CBAxk4cCC77bYb3bp1K13+7rvvUtrGmWeeyYcfflhlmTvvvJP8pnBeKJFYWsm0W5Wtj7umejzqIrlPB36auIPLgcA6d9+uSaY+1HU7WqdOnZg7dy5z587lnHPO4eKLLy5dbtWqFRA6FIuLiyvdxv3330+fPn2q3M/5559PXpzPByVSlbR4Vro+7tLpeDRk23+1yd3MHiHM293HzArM7Gdmdo6ZnZMoMoMw9evHwJ8Jd4avdyXtaEuWgHtZO1p9HKyPP/6Yfv36cc4555CTk8OKFSsYP348ubm57Lvvvlx99dWlZYcOHcrcuXMpKiqiQ4cOXH755WRnZ3PQQQfx5ZdfAnDllVdy6623lpa//PLLGTx4MH369OE//wk3oNm4cSMnn3wy2dnZjBkzhtzcXObOnbtdbJMmTWL//fcvja9kls+PPvqII488kuzsbHJycli8eDEA1157Lf379yc7O5uJcT8vbaImT4bMzG3XZWaG9U1RuhyPhsxZQKiJRvGz3377eXkLFizYbl1levZ0D4do25+ePVPeRJUmTZrkN9xwg7u7L1q0yM3M33rrrdLn16xZ4+7uW7Zs8aFDh/r8+fPd3X3IkCH+zjvv+JYtWxzwGTNmuLv7xRdf7Nddd527u0+cONFvueWW0vKXXXaZu7s/9dRTfuyxx7q7+3XXXefnnXeeu7vPnTvXmzVr5u+88852cZbEUVxc7KNHjy7dX05Ojk+fPt3d3Tdt2uQbN2706dOn+9ChQ72wsHCb19ZGTT4raXgPPRT+F8zC74ceUhxRx1FXOQuY7Snk2EZ7hWpDt6Ptueee7L///qXLjzzyCDk5OeTk5LBw4UIWLFiw3Wt23nlnRowYAcB+++1XWnsu76STTtquzGuvvcbo0aMByM7OZt99963wtS+88AKDBw8mOzubV155hfnz57N27VpWr17ND37wAyBcdJSZmcnzzz/PWWedxc477wzALrvsUvMDIY1CXh4sXgzFxeF3FK2ADV5TrUI6HI+GzlmNNrk3dDta69atSx8vWrSI2267jRdffJF58+YxfPjwCsd9l7TTAzRv3pyioqIKt73TTjttV8ZTuIlKYWEhEyZM4IknnmDevHmcddZZpXFUNFzR3TWMURpMUx2lUpmGzlmNNrlH2Y72zTff0LZtW9q1a8eKFSt49tln63wfQ4cOZdq0aQC89957FZ4ZbNq0iWbNmtG5c2fWr1/PY4+FG8137NiRzp07849//AMIF4cVFhZyzDHHcO+997Jp0yYAvvqqqmvTRHZMUx2lUpmGzlmNNrnn5cGUKdCzJ5iF31OmNMzpVk5ODvvssw/9+vXj7LPPZsiQIXW+jwsuuIBly5YxYMAAbrrpJvr160f79u23KdOpUyfOOOMM+vXrx4knnsgBB5TN+pCfn89NN93EgAEDGDp0KKtWreKEE05g+PDh5ObmMnDgQG655ZY6j1ukRDqNUkkHDZ2zIruHam5urpe/WcfChQvp27dvJPGkm6KiIoqKisjIyGDRokUcc8wxLFq0iBYtIru/yjb0WUl1yl8ZCqGm2lCVsLgysznunltdufTIFLKdDRs2cNRRR1FUVIS7c/fdd6dNYhdJRUkCnzgxNMX06BGaIJTYG4ayRZrq0KEDc+bMiToMkR2Sl6dkHpVG2+YuUl5TnPlPpDKquUssNNWZ/0Qqo5q7xILGVItsS8ldYkFjqkW2peSe5PDDD9/ugqRbb72V886rei60Nm3aALB8+XJOOeWUSrddfuhnebfeeiuFSdXP4447jq+//jqV0Js8jakW2ZaSe5IxY8YwderUbdZNnTqVMWPGpPT6rl278uijj9Z6/+WT+4wZM+jQoUOtt9eUpMvMfyLpQsk9ySmnnMLTTz/N5s2bAVi8eDHLly9n6NChpePOc3Jy6N+/P0899dR2r1+8eDH9+vUDwtQAo0ePZsCAAZx22mmll/wDnHvuuaXTBU+aNAmA22+/neXLl3PEEUdwxBFHAJCVlcXq1asBuPnmm+nXrx/9+vUrnS548eLF9O3bl7PPPpt9992XY445Zpv9lPjHP/7BAQccwKBBgxg2bBgrV64Ewlj6M888k/79+zNgwIDS6QueeeYZcnJyyM7O5qijjqqTY1vforxiWSQdpe1omYsuggqmL98hAwdCIi9WqFOnTgwePJhnnnmGUaNGMXXqVE477TTMjIyMDJ544gnatWvH6tWrOfDAAxk5cmSlE3HdddddZGZmMm/ePObNm0dOTk7pc5MnT2aXXXZh69atHHXUUcybN48LL7yQm2++mZdeeonOnTtvs605c+Zw//338+abb+LuHHDAARx22GF07NiRRYsW8cgjj/DnP/+ZU089lccee4zTTz99m9cPHTqUN954AzPjnnvu4frrr+emm27id7/7He3bt+e9994DYO3ataxatYqzzz6bmTNn0qtXr0Y1/4zGVIuUUc29nOSmmeQmGXfniiuuYMCAAQwbNoxly5aV1oArMnPmzNIkO2DAAAYMGFD63LRp08jJyWHQoEHMnz+/wknBkr322muceOKJtG7dmjZt2nDSSSfx6quvAtCrVy8GDhwIVD6tcEFBAcceeyz9+/fnhhtuYP78+QA8//zznH/++aXlOnbsyBtvvMGhhx5Kr169AE0LLNJYpW3Nvaoadn364Q9/yCWXXMLbb7/Npk2bSmvc+fn5rFq1ijlz5tCyZUuysrIqnOY3WUW1+s8++4wbb7yRWbNm0bFjR8aOHVvtdqqa/6dkumAIUwZX1CxzwQUXcMkllzBy5EhefvllrrrqqtLtlo9R0wKLxINq7uW0adOGww8/nLPOOmubjtR169ax66670rJlS1566SWWLFlS5XYOPfTQ0ptgv//++8ybNw8I0wW3bt2a9u3bs3LlSv75z3+WvqZt27asX7++wm09+eSTFBYWsnHjRp544gkOOeSQlN/TunXr6NatGwAPPvhg6fpjjjmGO+64o3R57dq1HHTQQbzyyit89tlngKYFFmmslNwrMGbMGN59993SOyEB5OXlMXv2bHJzc8nPz2fvvfeuchvnnnsuGzZsYMCAAVx//fUMHjwYCHdVGjRoEPvuuy9nnXXWNtMFjx8/nhEjRpR2qJbIyclh7NixDB48mAMOOIBx48YxaNCglN/PVVddxY9+9CMOOeSQbdrzr7zyStauXUu/fv3Izs7mpZdeokuXLkyZMoWTTjqJ7OxsTjvttJT3IyLpQ1P+Sq3osxKJRqpT/qrmLiISQ0ruIiIxlHbJPapmIkmdPiOR9JdWyT0jI4M1a9YoeaQxd2fNmjVkZGREHYqIVCGtxrl3796dgoICVq1aFXUoUoWMjAy6d+8edRgiUoW0Su4tW7YsvTJSRERqL62aZUREpG4ouYuIxJCSu4hIDCm5i4jEkJK7iEgMpZTczWy4mX1oZh+b2eUVPN/TzF4ws3lm9rKZaZyciEiEqk3uZtYcuBMYAewDjDGzfcoVuxH4i7sPAK4GrqvrQEVEJHWp1NwHAx+7+6fu/h0wFRhVrsw+wAuJxy9V8LzEWH4+ZGVBs2bhd2IaexGJUCrJvRvwedJyQWJdsneBkxOPTwTamlmn8hsys/FmNtvMZusq1HjIz4fx42HJEnAPv8ePV4IXiVoqyb2ie66Vn/zll8BhZvYOcBiwDCja7kXuU9w9191zu3TpUuNgJf1MnAiFhduuKywM60UkOqlMP1AA7J603B1YnlzA3ZcDJwGYWRvgZHdfV1dBSvpaurRm60WkYaRSc58F9DazXmbWChgNTE8uYGadzaxkW78C7qvbMCVd9ehRs/Ui0jCqTe7uXgRMAJ4FFgLT3H2+mV1tZiMTxQ4HPjSzj4D/AibXU7ySZiZPhszMbddlZob1IhKdtLqHqjRO+fmhjX3p0lBjnzwZ8vKijkoknlK9h2paTfkrjVNenpK5SLrR9AMiIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C5Sh/LzISsLmjULv/Pzo45ImqoWUQcgEhf5+TB+PBQWhuUlS8IyQF5edHFJ06Sau0gdmTixLLGXKCwM60UampK7SB1ZurRm60Xqk5K7SB3p0aNm60XqU0rJ3cyGm9mHZvaxmV1ewfM9zOwlM3vHzOaZ2XF1H6pIeps8GTIzt12XmRnWizS0apO7mTUH7gRGAPsAY8xsn3LFrgSmufsgYDTwx7oOVCTd5eXBlCnQsyeYhd9TpqgzVaKRymiZwcDH7v4pgJlNBUYBC5LKONAu8bg9sLwugxRpLPLylMwlPaSS3LsBnyctFwAHlCtzFfAvM7sAaA0Mq5PoRESkVlJpc7cK1nm55THAA+7eHTgO+KuZbbdtMxtvZrPNbPaqVatqHq2IiKQkleReAOyetNyd7ZtdfgZMA3D314EMoHP5Dbn7FHfPdffcLl261C5iERGpVirJfRbQ28x6mVkrQofp9HJllgJHAZhZX0JyV9VcRCQi1SZ3dy8CJgDPAgsJo2Lmm9nVZjYyUexS4Gwzexd4BBjr7uWbbkREpIGkNLeMu88AZpRb95ukxwuAIXUbmoiI1JauUBURiSEldxGRGFJyFxGJISV3EZEYUnIXEYkhJXcRkRhScm/EdL9OEamM7qHaSOl+nSJSFdXcGyndr1NEqqLk3kjpfp0iUhUl90ZK9+sUkaoouTdSul+niFRFyb2R0v06RaQqGi3TiOl+nSJSGdXcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhlJK7mY23Mw+NLOPzezyCp6/xczmJn4+MrOv6z5UERFJVbX3UDWz5sCdwNFAATDLzKa7+4KSMu5+cVL5C4BB9RCriIikKJWa+2DgY3f/1N2/A6YCo6ooPwZ4pC6CExGR2kkluXcDPk9aLkis246Z9QR6AS9W8vx4M5ttZrNXrVpV01hFRCRFqSR3q2CdV1J2NPCou2+t6El3n+Luue6e26VLl1RjFBGRGkoluRcAuyctdweWV1J2NGqSERGJXCrJfRbQ28x6mVkrQgKfXr6QmfUBOgKv122IIiJSU9Umd3cvAiYAzwILgWnuPt/MrjazkUlFxwBT3b2yJhsREWkg1Q6FBHD3GcCMcut+U275qroLS0REdoSuUBURiSEldxGRGFJyFxGJISX3WsjPh6wsaNYs/M7PjzoiEZFtpdShKmXy82H8eCgsDMtLloRlgLy86OISEUmmmnsNTZxYlthLFBaG9SIi6ULJvYaWLq3ZehGRKCi511CPHjVbLyISBSX3Gpo8GTIzt12XmRnWi4ikCyX3GsrLgylToGdPMAu/p0xRZ6qIpBeNlqmFvDwlcxFJb6q5i4jEkJK7iEgMKbmLiMSQkruISAwpuYuIxJCSu4hIDCm5i4jEkJK7iEgMKbmLiMSQkruISAwpuYuIxJCSu4hIDCm5i4jEkJK7iEgMKbmLiMSQkruISAwpuYuIxJCSu4hIDCm5i4jEkJK7iEgMpZTczWy4mX1oZh+b2eWVlDnVzBaY2Xwze7huwxQRkZpoUV0BM2sO3AkcDRQAs8xsursvSCrTG/gVMMTd15rZrvUVsIiIVC+Vmvtg4GN3/9TdvwOmAqPKlTkbuNPd1wK4+5d1G6aIiNREKsm9G/B50nJBYl2y7wPfN7N/m9kbZja8og2Z2Xgzm21ms1etWlW7iEVEpFqpJHerYJ2XW24B9AYOB8YA95hZh+1e5D7F3XPdPbdLly41jVVERFKUSnIvAHZPWu4OLK+gzFPuvsXdPwM+JCR7ERGJQCrJfRbQ28x6mVkrYDQwvVyZJ4EjAMysM6GZ5tO6DFRERFJXbXJ39yJgAvAssBCY5u7zzexqMxuZKPYssMbMFgAvAf/t7mvqK2gREamauZdvPm8Yubm5Pnv27Ej2LVKf1q+H99+HAw8Eq6jHSmQHmNkcd8+trpyuUBWpI0uWwKWXQvfucPDBcMUVEFHdSaT6i5hEpGqvvw633AKPPRZq6qeeCq1awe9/D4WF4blmqkZJA1NyF6mFoiJ4/PGQuN94Azp0gF/+EiZMgN13DzX2XXYJzxcWwp/+BM2bRx21NCVK7iI1sG4d3HMP3H47LF0Ke+0Fd9wBZ5wBbdqUlTODm26C1q3hmmtg0yZ44AFoof84aSD6UxNJwaefwm23wX33wYYNcNhh8Ic/wPHHV14jN4Pf/Q523hkmToRvv4WHHw5NNiL1TcldpBLu8NproWnlySdDEh8zBi66CHJyUt/OFVdAZiZcfDGceGJom8/IqL+4RUDJXWQ7W7bA3/8ekvrs2aHt/Fe/gvPPh65da7fNiy4KCf6cc+CEE+Cpp0KTjUh9UXIXSVi7FqZMCc0ty5ZBnz5w113w05+GxLyjxo8PTTRjx8Kxx8KMGdCu3Y5vV6QiSu7S5C1aFNrT778/jGw56ii4+24YMaLuhzD+5CehSebHPw77efbZcGYg9ccdVq4M/SZ9+0LHjlFH1DCU3Bu5DRtCAqqLmmVT4g4vvxyaXp5+Glq2DAn3oosgO7t+9/2jH4Ua/MknwxFHwHPPwa66vc0OcYfVq8MXdUU/GzaEcp07h888Ly/+Vw9r+oFGqqgoDMf7zW9CTfCaa2DcOA21q85338HUqeEffO7c8M9+7rlw3nmw224NG8tzz8GoUdCzJzz/PHQrf5cE2c7atdsm7Y8+Knu8bl1ZuebNISsLeveG738//O7aFW68MVx0dvTR4dqDPfaI7K3UWqrTDyi5N0JvvQU//3lITscdBxs3wiuvQP/+cOutcOSRUUeYflavDk0td94JK1bAPvuE0St5eaEWHZVXXw2f4a67wgsvhITU1H3zTeU18DVJ0xGahS/G3r23/+nVK5yNlVdcHP4O/ud/QgXpqqvC30FFZdNVqskdd4/kZ7/99nOpma+/dj/vPHcz965d3R991L24OPw8+qh7VpY7uJ94ovsnn0QdbXpYuND95z93z8gIx+bYY92feSYcs3Tx5pvuHTq47767+0cfRR1Nw9iwwX3uXPe//9392mvdzzzTfcgQ9113DZ9T8k/37u5HHOE+frz7DTe4P/WU+4IF7t9+W/v9FxSE/xNwz852f+utuntv9Q2Y7SnkWCX3RqC42H3qVPfddnNv1sz9wgvd163bvtymTe6TJ7u3bu3eqpX75Ze7f/NNw8cbteJi9+eecx8xIvyF77ST+7hx7u+/H3VklXvnHffOncNnnM5x1sacOe7/+7/hMzjssFAxKZ/Av/c990MPdf/Zz9x//3v3xx5znzfPfePG+o3t8cdDPM2auV90kfv69fW7v7qg5B4Tn3wSapvgvt9+7rNmVf+aZcvcf/rT8JrddnO/7z73rVvrP9aobdoU3mv//uG977qr+29/675yZdSRpWb+/JDkOnd2f/vtqKPZcZ984n7aaWUJvEsX94MPdj/jDPdrrnH/29/Cl1rUCTX5jLhHD/enn442nurEMrk/9JB7z57hQ+jZMyzH1ebNoRaekeHetq37bbe5FxXVbBtvvul+4IFlXwyvvVY/sUZt2TL3iRNDUoSQ3O+/f8dO26OyaFFIMB06uL/xRtTR1M7q1aEW3LKle2am+69/Hdalu3//233ffcPf0Kmnuq9YEXVEFYtdcn/oofCHknwql5kZzwQ/c6b7PvuE93jyyaF9sLaKi8Mx6tYtbG/0aPclS+ou1ii9/rr7mDHuLVqEL/yRI92ffz692tNrY/Fi9z33dG/Txv3ll6OOJnWFhaFJpX370Mwxblz44m1MNm92/93vQrNmhw7uf/5z+p31xi659+zp27XTQVgfF6taJecsAAAKo0lEQVRXh46lkvdVl6eHGzaEGlRGhvvOO7tPmlT/7Zn1YfPm8GU1eHA4Tu3auV98cfw6kJctc+/bN3xWzz4bdTRV27rV/cEHQ4cwuB9/vPt770Ud1Y754IPQPwChL+CDD6KOqEzskrtZxcndrKaHJv0UF7s/8EBoVmjRwv2yy0Iyrg+LF5e1g3bv7v7ww42jprtypfvVV4c2aXD//vfd77gj3h3GX34ZRnK0ahVGiKSjf/0rxFjS9Pfii1FHVHeKi93vvde9Y8fwGVx9dahcRC12yT2uNfeFC8tqCAcfHEYINISZM90HDSrbbyodtVGYMyd0wLVqFWIdPtx9xoz0O1WuL199Fc5SWrQIHZDpYu5c92OOCZ9JVlaoJMT1M/nii7IK0T77hLb5KMUuuVfU5t68ufuECdH3ttdGYaH7lVeGTqcOHdynTGn4f46iIvd77ikbWzx2rPvy5Q0bQ0W2bAnjn4cODXG1bu1+/vnhi7ApWrfO/ZBDQjv2Aw9EG8vSpeHL1izUaG+6qXF2XNfG00+Hzm5wP/fcMMomCrFL7u4hwZcc3LZtQ8cNhHbkE090z8+vePx3uvnXv0KHGbiffnr0Q/XWrQtNQS1bhk68664Lwwob2urVoUOupO22V6+QPNaubfhY0s2GDe7DhoXjctddDb//r78O101kZISzqF/+MpxVNDXr14c+nmbNQhPhY481fAyxTO7lFRWF5oULLyy7MGKnncKoib/+Nbpv1sqsWBFGd4B7795hZEc6WbTIfdSossT6+OMN0x7/3nvuZ58dOg/B/cgj3Z98suZDP+Nu0yb3E04Ix+jmmxtmn5s3h2G4nTqF/ebluX/2WcPsO53NmuU+cGA4Jj/8ofvnnzfcvptEck+2dWtoC7vootBRCKGGcfzx4VQ2ylrG1q2httW+fYhp0qRoasapeu65svG+Rxzh/u67db+PoqLQSXjkkWVnX+PGNVyfQ2O1ebP7KaeEY3bNNfW3n+Li0Ma/xx5lX7hz5tTf/hqj775zv/76UClp2zZ08DdEhaTJJfdkW7eGMdCXXlrWEduyZbgc/d573desqbddb2fu3LILiY48Mr2GVFVlyxb3O+9032WXcAp6zjlh9MaO+vrrUOssSRrdu4dmoMZwkUu62LLF/Sc/Ccfviivq/uxq5syyoab9+oUO7MYwoioqn3zifvTR4XgdeGD9V1CadHJPVlwcJgX67/8OTQ0QRh4ce2y4QGHVqvrZ7/r14culefNw2fVf/tI4/0HWrAnNXs2bhzOPW24JNZaa+uCD0CnaunX4DIYMcZ82LSQqqbmtW8NEWuD+i1/Uzd/WwoVlzXJdu4apHNQ0lpqSiwVLhjNPnFh/Z+dK7hUoLnafPTt0DJV0aDZvHjqq/vSnuuvYnD69rON33LiGPVOoL/Pnl81x06eP+//9X/Wv2brV/Z//LJvAq1WrMOfN7Nn1H29TUFwcEjuERF/b0VYrVoQzs+bNQ/PC5MmN8wK3dLBqVdm8Tr1718+4fyX3ahQXh0mLJk4MF8RAaH444gj3P/6xdvNKLF1aNo1ov37xm8uluDgMBys5XiNGVDw8cf360P7Yp4+XTl7229+G8cJSt4qLQ9MMhKaampwJrV/vftVV4WyqRYswrLgumt4k9FuVND2eeWbdNjsquddAcXFoJ/v1r9333ttLr3w97DD3P/yh+rHfW7aEduQ2bULnyu9/X7umi8Zi8+YwRLFdu5AUfvGL0GH9ySful1xSNkQ1NzeMWkqHq/ri7pprwjE/5ZTqj/eWLe533x2+dEvmL2oq88g3pI0bQytBSdNsXV0NruS+A95/P9RoSkaMmIULam67bfshT2+9VXal53HHuX/6aTQxR2HlytAcYBYSvVlI9qNHu//nP42zj6Exu/nm8Hd4wgkVt/cWF4cmw759vfTK5P/8p+HjbGrmznXff38vvcJ6R4eSKrnXkQULwpwSAwZ46ZWxBx8c/pHOP7/srkh//3vTTWZz54bx+1dcsWMzWMqOu+uu8Dc6bNi28xO99VaYAKtkXp6GuoZBgqKiUDls3Tpcaf/II7XfVp0md2A48CHwMXB5Bc+PBVYBcxM/46rbZmNJ7sk+/DB0NpVcvGDmfsEFjeOqWGk6Hngg9B8NHRr6lUrmRdl119CfFOcmw3S3ZEm46Omdd2q/jVSTe7U3yDaz5sBHwNFAATALGOPuC5LKjAVy3X1CtTdtTWjsN8j+5JNQj99rr6gjEdnetGnh5t9FReEG4JdeCpddBm3bRh2Z7KhUb5DdIoVtDQY+dvdPExueCowCFlT5qpjbc8+oIxCp3KmnQrt28NxzIbF37Rp1RNLQUknu3YDPk5YLgAMqKHeymR1KqOVf7O6fly9gZuOB8QA9evSoebQikrLhw8OPNE3NUihjFawr35bzDyDL3QcAzwMPVrQhd5/i7rnuntulS5eaRSoiIilLJbkXALsnLXcHlicXcPc17r45sfhnYL+6CU9ERGojleQ+C+htZr3MrBUwGpieXMDMvpe0OBJYWHchiohITVXb5u7uRWY2AXgWaA7c5+7zzexqwpCc6cCFZjYSKAK+IgyNFBGRiFQ7FLK+NPahkCIiUUh1KGQqzTIiItLIKLmLiMSQkruISAxF1uZuZquAJZHsvO50BlZHHUQa0fEoo2OxLR2Pbe3I8ejp7tVeKBRZco8DM5udSsdGU6HjUUbHYls6HttqiOOhZhkRkRhSchcRiSEl9x0zJeoA0oyORxkdi23peGyr3o+H2txFRGJINXcRkRhSchcRiSEl91ows93N7CUzW2hm883sF1HHFDUza25m75jZ01HHEjUz62Bmj5rZB4m/kYOijilKZnZx4v/kfTN7xMwyoo6poZjZfWb2pZm9n7RuFzN7zswWJX53rI99K7nXThFwqbv3BQ4EzjezfSKOKWq/QFM9l7gNeMbd9wayacLHxcy6ARcS7rHcjzCz7Ohoo2pQDwDl74d1OfCCu/cGXkgs1zkl91pw9xXu/nbi8XrCP2+3aKOKjpl1B44H7ok6lqiZWTvgUOBeAHf/zt2/jjaqyLUAdjazFkAm5W72E2fuPpMwDXqyUZTdre5B4If1sW8l9x1kZlnAIODNaCOJ1K3AZUBx1IGkgT2AVcD9iWaqe8ysddRBRcXdlwE3AkuBFcA6d/9XtFFF7r/cfQWEiiKwa33sRMl9B5hZG+Ax4CJ3/ybqeKJgZicAX7r7nKhjSRMtgBzgLncfBGyknk67G4NEe/IooBfQFWhtZqdHG1XToOReS2bWkpDY89398ajjidAQYKSZLQamAkea2UPRhhSpAqDA3UvO5B4lJPumahjwmbuvcvctwOPAwRHHFLWVJbcmTfz+sj52ouReC2ZmhDbVhe5+c9TxRMndf+Xu3d09i9BR9qK7N9mambt/AXxuZn0Sq44CFkQYUtSWAgeaWWbi/+YomnAHc8J04IzE4zOAp+pjJ9XeQ1UqNAT4CfCemc1NrLvC3WdEGJOkjwuA/MQN5T8Fzow4nsi4+5tm9ijwNmGU2Ts0oakIzOwR4HCgs5kVAJOA3wPTzOxnhC+/H9XLvjX9gIhI/KhZRkQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhv4fkkgs1JvjB/gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd4VFX6wPHvCwSQzgJKEwNWitQsa0FBbKBSZFFBsLOIDWy7snZZWbEjLhZkLasouokoKoL+MC5YQBJKpIggBo0ghEivGfL+/jgTSGIymSQzc2cm7+d58mTmzpl737mTvPfcc885V1QVY4wx8aWK1wEYY4wJPUvuxhgThyy5G2NMHLLkbowxcciSuzHGxCFL7sYYE4csuZtiiUhVEdklIq1CWdZLInKciIS876+InCMimQWerxaRM4IpW45tTRWRu8v7/gDrfVhEXg31eo13qnkdgAkNEdlV4GktYD9w0P/8elWdVpb1qepBoE6oy1YGqnpiKNYjIiOA4araq8C6R4Ri3Sb+WXKPE6p6KLn6a4YjVPX/SiovItVU1ReJ2IwxkWfNMpWE/7T7bRF5S0R2AsNF5FQRWSAi20Rko4hMEpEEf/lqIqIikuh//ob/9Y9FZKeIfC0ircta1v96XxH5XkS2i8izIvKliFxdQtzBxHi9iKwVka0iMqnAe6uKyNMikiMiPwB9Auyfe0VkepFlk0XkKf/jESKyyv95fvDXqktaV5aI9PI/riUir/tjWwF0K2a76/zrXSEi/f3LTwb+BZzhb/LaUmDfPljg/aP8nz1HRN4TkWbB7JvSiMhAfzzbROQzETmxwGt3i8gGEdkhIt8V+KyniMhi//JNIvJ4sNszYaCq9hNnP0AmcE6RZQ8DB4B+uIP6EcAfgT/hzuDaAN8DN/vLVwMUSPQ/fwPYAiQBCcDbwBvlKHsksBMY4H/tdiAXuLqEzxJMjO8D9YFE4Lf8zw7cDKwAWgKNgHnuT77Y7bQBdgG1C6x7M5Dkf97PX0aA3sBeoKP/tXOAzALrygJ6+R8/AXwONASOAVYWKXsp0Mz/nVzuj+Eo/2sjgM+LxPkG8KD/8Xn+GDsDNYHngM+C2TfFfP6HgVf9j9v64+jt/47u9u/3BKA9sB5o6i/bGmjjf7wIGOp/XBf4k9f/C5X5x2rulcsXqvqBquap6l5VXaSqC1XVp6rrgClAzwDvT1bVNFXNBabhkkpZy14ELFXV9/2vPY07EBQryBgfUdXtqpqJS6T527oUeFpVs1Q1B5gQYDvrgOW4gw7AucA2VU3zv/6Bqq5T5zNgLlDsRdMiLgUeVtWtqroeVxsvuN13VHWj/zt5E3dgTgpivQDDgKmqulRV9wFjgZ4i0rJAmZL2TSBDgJmq+pn/O5oA1MMdZH24A0l7f9Pej/59B+4gfbyINFLVnaq6MMjPYcLAknvl8nPBJyJykoh8JCK/isgOYBzQOMD7fy3weA+BL6KWVLZ5wThUVXE13WIFGWNQ28LVOAN5Exjqf3w57qCUH8dFIrJQRH4TkW24WnOgfZWvWaAYRORqEVnmb/7YBpwU5HrBfb5D61PVHcBWoEWBMmX5zkpabx7uO2qhqquBO3Dfw2Z/M19Tf9FrgHbAahH5RkQuCPJzmDCw5F65FO0G+CKutnqcqtYD7sc1O4TTRlwzCQAiIhRORkVVJMaNwNEFnpfWVfNt4Bx/zXcALtkjIkcAycAjuCaTBsAnQcbxa0kxiEgb4HngBqCRf73fFVhvad02N+CaevLXVxfX/PNLEHGVZb1VcN/ZLwCq+oaqno5rkqmK2y+o6mpVHYJrensSSBGRmhWMxZSTJffKrS6wHdgtIm2B6yOwzQ+BriLST0SqAWOAJmGK8R3gVhFpISKNgLsCFVbVTcAXwCvAalVd43+pBlAdyAYOishFwNlliOFuEWkgbhzAzQVeq4NL4Nm449wIXM093yagZf4F5GK8BVwnIh1FpAYuyc5X1RLPhMoQc38R6eXf9l9x10kWikhbETnLv729/p+DuA9whYg09tf0t/s/W14FYzHlZMm9crsDuAr3j/siruYaVv4EehnwFJADHAsswfXLD3WMz+Paxr/FXexLDuI9b+IukL5ZIOZtwG3ADNxFycG4g1QwHsCdQWQCHwP/KbDeDGAS8I2/zElAwXbqT4E1wCYRKdi8kv/+2bjmkRn+97fCtcNXiKquwO3z53EHnj5Af3/7ew3gMdx1kl9xZwr3+t96AbBKXG+sJ4DLVPVAReMx5SOuydMYb4hIVVwzwGBVne91PMbEC6u5m4gTkT4iUt9/an8frgfGNx6HZUxcseRuvNADWIc7te8DDFTVkppljDHlYM0yxhgTh6zmbowxccizicMaN26siYmJXm3eGGNiUnp6+hZVDdR9GPAwuScmJpKWlubV5o0xJiaJSGkjrQFrljHGmLhkyd0YY+KQJXdjjIlDltyNMSYOWXI3xpg4FFPJfdo0SEyEKlXc72lluuWzMcZUHjFzg+xp02DkSNizxz1fv949BxhW4XnwjDEmvsRMzf2eew4n9nx79rjlxhhjCouZ5P7TT2VbbowxlVnMJPdWJdwgraTlxhhTmcVMch8/HmrVKrysVi233BhjTGExk9yHDYMpU+CYY0DE/Z4yxS6mGmNMcWKmtwy4RG7J3BhjShczNXdjjIkHY8bABx+EfzulJncReVlENovI8gBleonIUhFZISL/C22IxhgTHzIzYdIkWLUq/NsKpub+Ku4+l8USkQbAc0B/VW0PXBKa0IwxJr68+677/ec/h39bpSZ3VZ0H/BagyOXAu6r6k7/85hDFZowxcSU5Gbp0gWOPDf+2QtHmfgLQUEQ+F5F0EbmypIIiMlJE0kQkLTs7OwSbNsaY2PDLL/D115GptUNokns1oBtwIXA+cJ+InFBcQVWdoqpJqprUpEmptwA0xpi4kd8kM3hwZLYXiq6QWcAWVd0N7BaReUAn4PsQrNsYY+JCcjJ06AAnnhiZ7YWi5v4+cIaIVBORWsCfgAhcCzbGmNiwaRPMnx+5JhkIouYuIm8BvYDGIpIFPAAkAKjqC6q6SkRmAxlAHjBVVUvsNmmMMZXNjBmgGrkmGQgiuavq0CDKPA48HpKIjDEmziQnwwknQPv2kdumjVA1xpgw2rIFPv/c1dpFIrddS+7GGBNG778PBw9GtkkGLLkbY0xYJSdD69bQuXNkt2vJ3RhjwmTrVpg7N/JNMmDJ3RhjwuaDDyA3N/JNMmDJ3RhjwiY5GY4+Gv74x8hv25K7McaEwY4d8MknbuBSpJtkwJK7McaExUcfwf793jTJgCV3Y4wJi+RkaNYMTj3Vm+1bcjfGmBDbvRs+/hgGDYIqHmVZS+7GGBNiH38Me/d61yQDltyNMSbkkpOhSRM44wzvYrDkbowxIbR3r7uYevHFULWqd3FYcjfGmBD65BPYtcvbJhmw5G6MMSGVkgING0KvXt7GYcndGGNCZP9+mDkTBg6EhARvYyk1uYvIyyKyWUQC3l1JRP4oIgdFxOOTEWOM8cbcubB9u/dNMhBczf1VoE+gAiJSFXgUmBOCmIwxJialpEC9enD22V5HEkRyV9V5wG+lFLsFSAE2hyIoY4yJNbm58N570L8/1KjhdTQhaHMXkRbAxcALQZQdKSJpIpKWnZ1d0U0bY0zU+Pxz+O236GiSgdBcUJ0I3KWqB0srqKpTVDVJVZOaNGkSgk0bY0x0SEmB2rXhvPO8jsSpFoJ1JAHTxc1p2Ri4QER8qvpeCNZtjDFR7+BBePdduOgiOOIIr6NxKpzcVbV1/mMReRX40BK7MaYymT8fsrPd3O3RotTkLiJvAb2AxiKSBTwAJACoaqnt7MYYE+9SUlyNvW9fryM5rNTkrqpDg12Zql5doWiMMSbG5OW55N63L9Sp43U0h9kIVWOMqYCvv4aNG6OrSQYsuRtjTIWkpED16u5iajSx5G6MMeWk6uZuP/98NzI1mlhyN8aYclq0CH7+OfqaZMCSuzHGlFtKClSr5qYciDaW3I0xphzym2TOOcfN3x5tLLkbY0w5LFsG69ZFZ5MMWHI3xphySU5290gdONDrSIpnyd0YY8oov0mmVy9o3NjraIpnyd0YY8po5UpYvTp6m2TAkrsxxpRZcjKIwMUXex1JySy5G2NMGSUnwxlnQNOmXkdSMkvuxhhTBqtXw/Ll0d0kA5bcjTGmTFJS3O9Bg7yNozSW3I0xpgySk+HUU6FlS68jCazU5C4iL4vIZhFZXsLrw0Qkw//zlYh0Cn2YxhjjvXXrYMmS6G+SgeBq7q8CfQK8/iPQU1U7Av8ApoQgLmOMiTr5TTKxkNyDuRPTPBFJDPD6VwWeLgCi/GTFGGPKJzkZunWDxESvIyldqNvcrwM+LulFERkpImkikpadnR3iTRtjTPj89BN88w0MHux1JMEJWXIXkbNwyf2uksqo6hRVTVLVpCZNmoRq08YYE3bvvut+x0KTDATRLBMMEekITAX6qmpOKNZpjDHRJDkZOnaE44/3OpLgVLjmLiKtgHeBK1T1+4qHZIwx0WXDBvjqq9hpkoEgau4i8hbQC2gsIlnAA0ACgKq+ANwPNAKeExEAn6omhStgY4yJtBkz3EyQcZXcVXVoKa+PAEaELCJjjIkyycnQtq37iRU2QtUYYwLYvBnmzYutWjtYcjfGmIDeew/y8iy5G2NMXElJgeOOg5NP9jqSsrHkbowxJcjJgblzXa3d9ReJHZbcjTGmBDNnwsGDsdckA5bcjTGmRCkpbh6Zrl29jqTsLLkbY0wxtm+HTz5x0w3EWpMMWHI3xphiffAB5ObGZpMMWHI3Jm79/DM8+yz4fF5HEptSUqBFC+je3etIyseSuzFxKDfX3eNz9Gi48UY3dN4Eb+dO+Phj1yRTJUazZIyGbYwJ5B//gLQ0OP98eOklGD/e64hiy6xZsH9/7DbJgCV3Y+LO11+7ZH7lla72ecUVcN998OqrXkcWO1JS4Kij4LTTvI6k/EIyn7sxJjrs2uWS+dFHw6RJrpfH1KmwcSP85S/QrJmrzZuS7dkDH30EV10FVat6HU35Wc3dmDhy222wbh28/jrUr++WVa/uaqLt2rlmhsWLvY0x2s2e7RJ8rNxxqSSW3I2JEzNnulr63/4GZ5xR+LV69VwTTcOGcOGFkJnpSYgxISUFGjWCnj29jqRiLLkbEwc2bYIRI6BzZxg3rvgyzZu7BL9vH/TtC7/9FtkYY8G+fa5/+8UXQ7UYb7QuNbmLyMsisllElpfwuojIJBFZKyIZIhKDA3WNiV2qcN11sGMHvPGGa4YpSfv2bgrbdeugf3+XzMxhn37qukHGepMMBFdzfxXoE+D1vsDx/p+RwPMVD8sYE6wpU9wFwEcfdcm7ND17ujb5L7+E4cPdxFjGSUmBBg2gd2+vI6m4UpO7qs4DAp3ADQD+o84CoIGINAtVgMaYkn3/Pdx+O5xzDtxyS/Dvu/RSePJJl8xuv90GOQEcOADvvw8DBgQ++4kVoWhVagH8XOB5ln/ZxqIFRWQkrnZPq1atQrBpYyqv3FzX7bFGDdeHvawjKW+/3U1RMHEitGoFd9wRljBjxmefwbZt8dEkA6FJ7sXNl1ZsPUBVpwBTAJKSkqyuYEwFjB8P33wDb7/t5kApjyefhKwsuPNOt44hQ0IbYyxJSYG6deHcc72OJDRCkdyzgKMLPG8JbAjBeo0xJVi4EB5+2LWZX3pp+ddTpYprf//1Vzdop2lT6NUrZGHGDJ8PZsyAfv2gZk2vowmNUHSFnAlc6e81cwqwXVV/1yRjjAmN3btdc0yLFvCvf1V8fTVrurbmNm1g4EBYsaLi64w18+a5W+rFS5MMBNcV8i3ga+BEEckSketEZJSIjPIXmQWsA9YCLwE3hi1aYwx33AFr18J//nN4FGpF/eEPrg/8EUdAnz7wyy+hWW+sSE6GWrXcZ48XpTbLqOrQUl5X4KaQRWSMKdGHH8KLL8Jf/xr6EZSJiW42xDPPhAsucLXZUB08otnBg/Duu27kbq1aXkcTOjZC1ZgYsXmzG6zUqZOb0jccunRxFxZXrnRNFAcOhGc70eSrr9wI33hqkgFL7sbEBFU3q+O2bW4Uao0a4dvWeee5OWrmznUHk3jvA5+c7K47XHCB15GEVozPnmBM5fDvf7uJwZ58Ejp0CP/2rrrK9YG/7z43ffA//xn+bXohL8+dqfTp47pBxhNL7sZEubVr4dZb3ZD4W2+N3Hbvuccl+EcecQn+hhsit+1I+eYbd/F4wgSvIwk9S+7GRDGfz3V7TEgo3yjUihCByZNhwwa4+WY3q+SAAZHbfiQkJ7t926+f15GEnrW5GxPFHnkEFiyA555ztedIq1YNpk+Hbt1g6FAXS7xQdcn9vPPis1eQJXdjotSiRfDQQy6pDg3YITm8atd2XTCbN4eLLnKTlcWDxYth/fr46yWTz5K7MVFo9243tUCzZq5pxGtHHuluPyfibvSxaZPXEVVccrI7M4m3pqZ8ltyNiUJ//aurIb/2mrs1XjQ47jhXg9+40dXgd+/2OqLyy2+S6d3bjc6NR5bcjYkys2bB88+7KXmj7aYRf/qTa4NfvNhNWObzeR1R+Xz7reuFFK9NMmDJ3Ziokp0N114LJ5/spvSNRv37u6aiWbPgxhtjc5BTcrLreTRwoNeRhI91hTQmSqjCyJGwdSt88kl0Tz07ahT89NPhPvD33ed1RGWTnOzm0DnySK8jCR+ruZdDXp5rC/3b39yNDowJhVdecTevHj8eOnb0OprSjR/v+uDff7/rgx8rVq6EVatg8GCvIwkvS+5l9PnnkJQEV18Njz8OJ57oJnHau9fryEwsW7cOxoxxN8q4/XavowmOiJuD5pxz3Lw3c+Z4HVFwUlJc7Bdf7HUk4WXJPUhr1rg/hrPOgi1b4M034YcfXLew+++Htm3hv/+NzfZH4638UahVq7ozwkiOQq2o6tVdsmzXztWEFy/2OqLSJSfDaae5fvvxLIb+jLyxdaurSbVvD//3f+5UdPVqN6ikTRv3h/LZZ26E26WXuprX0qVeR21iyWOPuWlnJ092N6qONfXquRt9NGzo5kTPzPQ6opKtWQMZGfHfJANBJncR6SMiq0VkrYiMLeb1ViKSKiJLRCRDRGJ+8szcXHj2Wde3d+JEN0vemjVw993ubjUFnXUWpKe77msrVkDXrnD99a7ngzGBpKfDAw/AZZfB5Zd7HU35NW/uEvy+fe5s9rffvI6oeCkp7vegQd7GERGqGvAHqAr8ALQBqgPLgHZFykwBbvA/bgdklrbebt26aTTKy1P94APVE09UBdXevVWXLg3+/b/9pnrrrarVqqnWr6/61FOq+/eHL95osXq16pIlqjt2eB1J7Ni9W/Wkk1RbtFDNyfE6mtD4/HPV6tVVTz9dde9er6P5vaQk1e7dvY6iYoA0LSW/qmpQNffuwFpVXaeqB4DpQNEBuwrU8z+uD2yoyAHHKxkZbhKhfv1c2/nMma4pplOn4NfRsCE8/bRb15/+5Jp0OnZ0tZp4k5sL77zjbvd24onuLj716rnuZaecAsOGuS5yr74K8+e7qVXz8ryOOnrcdRd8953bP/EySrJnT3j9dfjySzd9wsGDXkd0WGYmpKVVjiYZIKia+2BgaoHnVwD/KlKmGfAtkAVsBbqVsK6RQBqQ1qpVq4gc5YLx66+qf/mLapUqqg0bqj7zjOqBAxVfb/5ZwHHHubOACy5Q/e67iq/Xa7/8ovrAA6rNmrnP1aaN6mOPqf73v6oTJqiOHKl69tmqiYlun7pDpfupWVO1bVvViy5SHTPG7esPP1RduTI6a3rhMnu22x9jxngdSXg8+aT7fKNHu/8Dr+Xmqj7yiIvphx+8jqZiCLLmLlpK9w4RuQQ4X1VH+J9fAXRX1VsKlLkdEFV9UkROBf4NdFDVEutpSUlJmpaWVo7DUejs2+dq2f/8p3t8882uphnqWtSBAzBpEowb57pMjh7tttOgQWi3E06q7obJkyfDjBmuRtanj9tnffqU3MMjN9fNvLdunetd9MMPhR8XnJ9EBFq0cBeqjz328O/8x40auTKxLifHjUBt2NDVJItew4kXt93mrlc98QTccUfZ379/P+zcCTt2uN+BHpf2Wn5X5S5dYqNHTyAikq6qSaWVC2aEahZQcCbplvy+2eU6oA+Aqn4tIjWBxsDm4MKNLFV4+20YO9YlngEDXI+FE04Iz/aqV4c773Td3e65xx1QXn/d9by59lrXBS5a7dzp7tk5ebK7WNywobsb0KhRLumWJiHBXZQ+7rjfv6bqLjoXTfjr1rkZCDduLFy+Xr3fJ/z8361auRn+op2qu9i+ZYsbvh+viR3cLQGzstzf/r597uAcTDLOf5ybG9x2atVyfxt167qfevXcqNn8x/nL69Z1za6VRTA192rA98DZwC/AIuByVV1RoMzHwNuq+qqItAXmAi00wMq9qrkvWOBqFAsWuLb0p56K/ORMixe7AStffAGdO8Mzz7ih0NFk1Sp3g4jXXnP/aF27ulr6kCGRS0h79sCPPxaf/H/80Z0R5atWDY45xiX7005zZxNJSdF34HztNTcAbsIE1+Ye7/btg/PPd2d9+UQKJ9yiCbhoog70uE6d6PuOwy3Ymnup7Tb+/HwBLsH/ANzjXzYO6K+He8h8ietJsxQ4r7R1Rrq3TGam6tChrs2taVPVf/9b1eeLaAiF5OWpTp+uevTRLqZLL3Uxeik3VzUlxfUQAtfrYfhw1a+/jo5204J8PtWfflJNTVWdOlX17rtVL7tMtXNnVREX/x/+oDpkiOqrr6pu3Oh1xKrr1qnWrat6xhne/u1Fms+n+v33qhs2qO7cqXrwoNcRxTaCbHMPKrmH4ydSyX3HDvePX7Om+7n33ujqrrd7t+qDD6oecYSL7/77VXftimwMv/6q+o9/qLZs6f4iWrVS/ec/VTdtimwcoZKdrfrmm6pXXql61FGHL+Z27qw6dqzrrheKC+Zl4fOp9ujhkvuPP0Z22ya+VPrk7vOpvvTS4X/uYcNU168P6yYrZP16V/MEl2TffDO8teW8PNUvvnBnMwkJbrvnnqv63nvxVas8eFB18WJ3sDrzTDf+AFySHThQ9YUXInPGlN9T47XXwr8tE98qdXKfO1e1Uyf36U47TXXBgrBtKuTmzVPt0sXFfvrpqmlpoV3/rl2qU6Yc3j/167vueKtXh3Y70WrbNtV333XdNVu1OlyrP+kkN/hs9mzVPXtCu83Fi90BdPDg6GveMrGnUib3775T7dfPfarERNW3347Nf6b8s44jj3Ttx9deW/E24++/d8mrfn23fzp2VH3xxcg3AUWTvDzXv/6pp1TPO0+1Rg091Be/Tx/ViRPd31RF/ob27HH9+ps1U92yJXSxm8qrUiX3nBw3WKJaNXe6PWFCfAyI2bZN9c47Xa2vbl03UGjfvuDf7/Opvv++S1zg9s+QIarz58fmQS/cdu9WnTXL/S2dcMLhWn1iouqoUa7JqqzXa0aPduuYMyc8MZvKp1Ik9/37VZ9+2o0qrVJF9frr3cXBeLN6tRvRCW6068yZgZPz5s2ujfeYY9x7WrRQHTcuOnqMxJIfflB97jnV/v1Va9d2+zIhQbVXL1eBWLo08PfwySfuPbfcErmYTfyL6+Sel+dqpMcfr4cuBGZklHt1MePjj13bMKief75rUsiXl+euLVxxhevCCKpnnaWanOy6OJqK2b9f9bPPVP/2N9eklV+rb9ZM9eqrXbfWgpN/5eSoNm/ummRC3YZvKrdgk3upg5jCpbyDmJYvd8P3U1PhpJPcKLi+feNjWHowcnPd4KIHHoBdu+Cmm9xgrOeec9PH1qnjpie+8UZ3AwUTHhs2uDsPzZ4Nn37q5v2vUgW6d3cDqNLT3WRxCxe6AWDGhEqwg5hiLrnPnevmvn7oIXcz4YSEMAQXA7Kz3R2gpkxxMy22a+cS/RVXuJF7JnJ8Pli0yCX62bPdY1U3Z9Hf/+51dCbexG1yBzfZVO3aIQ4oRn33nas1nnJK5Tl7iXZbtsC337rpb2PplnkmNoRy4rCoY4n9sJNO8joCU1Tjxu7uXMZ4yeoVxhgThyy5G2NMHLLkbowxcciSuzHGxCFL7uUwbRokJrqeEImJ7rkxxkSToJK7iPQRkdUislZExpZQ5lIRWSkiK0TkzdCGGT2mTXP969evd32Z1693zy3BG2OiSTC32auKuwvTubj7qS4ChqrqygJljgfeAXqr6lYROVJVA94/NRpukF0eiYkuoRd1zDGQmRnpaIwxlU2w/dyDqbl3B9aq6jpVPQBMBwYUKfMXYLKqbgUoLbHHsp9+KttyY4zxQjDJvQXwc4HnWf5lBZ0AnCAiX4rIAhHpU9yKRGSkiKSJSFp2dnb5IvZYq1ZlW26MMV4IJrkXN6i9aFtONeB4oBcwFJgqIg1+9ybVKaqapKpJTZo0KWusUWH8eKhVq/CyWrXccmOMiRbBJPcs4OgCz1sCG4op876q5qrqj8BqXLKPO8OGucm6jjnGzeVyzDHu+bBhXkdmjDGHBZPcFwHHi0hrEakODAFmFinzHnAWgIg0xjXTrAtloNFk2DB38TQvz/22xG6MiTalJndV9QE3A3OAVcA7qrpCRMaJSH9/sTlAjoisBFKBv6pqTriCNsYYE1hMTvlrjDGVVSi7QhpjjIkxltyNMSYOWXI3xpg4ZMndGGPikCX3GGazUxpjShKT91A1h2en3LPHPc+fnRKs370xxmruMeueew4n9nx79rjlxhhjyT1G2eyUxphALLnHKJud0hgTiCX3GGWzUxpjArHkHqNsdkpjTCDWWyaGDRtmydwYUzyruRtjTByy5G6MMXHIkrsxxsQhS+7GGBOHgkruItJHRFaLyFoRGRug3GARUREpdSJ5Y4wx4VNqcheRqsBkoC/QDhgqIu2KKVcXGA0sDHWQxhhjyiaYmnt3YK2qrlPVA8B0YEAx5f4BPAbsC2F8xhhjyiGY5N4C+LnA8yz/skNEpAtwtKp+GGhFIjJSRNJEJC07O7vMwRpjjAlOMMldill26K7aIlIFeBq4o7QVqeoUVU1S1aQmTZoEH6UxxpgyCSa5ZwEnnu/uAAAQM0lEQVRHF3jeEthQ4HldoAPwuYhkAqcAM+2iqjHGeCeY5L4IOF5EWotIdWAIMDP/RVXdrqqNVTVRVROBBUB/VU0LS8TGGGNKVWpyV1UfcDMwB1gFvKOqK0RknIj0D3eAxhhjyi6ofu6qOktVT1DVY1V1vH/Z/ao6s5iyvazWXrnYvVyNiT42QtVUSP69XNevB9XD93KtzAneDnYmGlhyNxVi93ItzA52JlpYcjcVYvdyLcwOdiZaWHI3FWL3ci3MDnYmWlhyNxVi93ItzA52JlpYcjcVYvdyLcwOdiZa2D1UTYXZvVwPy98P99zjmmJatXKJ3faPibSoSu65ublkZWWxb59NLBkLatasScuWLUlISPA6lKhiBzsTDaIquWdlZVG3bl0SExMRKW6+MhMtVJWcnByysrJo3bq11+EYY4qIqjb3ffv20ahRI0vsMUBEaNSokZ1lGROloiq5A5bYY4h9V8ZEr6hL7sYYYyouppN7qOfwyMnJoXPnznTu3JmmTZvSokWLQ88PHDgQ1DquueYaVq9eHbDM5MmTmRai8eg9evRg6dKlIVlXrLM5XYw5LKouqJZF/hwe+UO98+fwgPL3VGjUqNGhRPnggw9Sp04d7rzzzkJlVBVVpUqV4o+Lr7zySqnbuemmm8oXoClROP4ejIllMVtzj+QcHmvXrqVDhw6MGjWKrl27snHjRkaOHElSUhLt27dn3Lhxh8rm16R9Ph8NGjRg7NixdOrUiVNPPZXNmzcDcO+99zJx4sRD5ceOHUv37t058cQT+eqrrwDYvXs3f/7zn+nUqRNDhw4lKSmp1Br6G2+8wcknn0yHDh24++67AfD5fFxxxRWHlk+aNAmAp59+mnbt2tGpUyeGDx8e8n0WaTanizGFBZXcRaSPiKwWkbUiMraY128XkZUikiEic0XkmNCHWlik5/BYuXIl1113HUuWLKFFixZMmDCBtLQ0li1bxqeffsrKlSt/957t27fTs2dPli1bxqmnnsrLL79c7LpVlW+++YbHH3/80IHi2WefpWnTpixbtoyxY8eyZMmSgPFlZWVx7733kpqaypIlS/jyyy/58MMPSU9PZ8uWLXz77bcsX76cK6+8EoDHHnuMpUuXsmzZMv71r39VcO94z+Z0MaawUpO7iFQFJgN9gXbAUBFpV6TYEiBJVTsCycBjoQ60qEjP4XHsscfyxz/+8dDzt956i65du9K1a1dWrVpVbHI/4ogj6Nu3LwDdunUjMzOz2HUPGjTod2W++OILhgwZAkCnTp1o3759wPgWLlxI7969ady4MQkJCVx++eXMmzeP4447jtWrVzNmzBjmzJlD/fr1AWjfvj3Dhw9n2rRpcTEIyeZ0MaawYGru3YG1qrpOVQ8A04EBBQuoaqqq5p8UL8DdRDusIj2HR+3atQ89XrNmDc888wyfffYZGRkZ9OnTp9j+3tWrVz/0uGrVqvh8vmLXXaNGjd+VUdUyxVdS+UaNGpGRkUGPHj2YNGkS119/PQBz5sxh1KhRfPPNNyQlJXHw4MEybS/a2JwuxhQWTHJvAfxc4HmWf1lJrgM+rkhQwfBywqodO3ZQt25d6tWrx8aNG5kzZ07It9GjRw/eeecdAL799ttizwwKOuWUU0hNTSUnJwefz8f06dPp2bMn2dnZqCqXXHIJDz30EIsXL+bgwYNkZWXRu3dvHn/8cbKzs9lTtME6xtgEZsYUFkxvmeJGqhRbTRSR4UAS0LOE10cCIwFaheB82as5PLp27Uq7du3o0KEDbdq04fTTTw/5Nm655RauvPJKOnbsSNeuXenQocOhJpXitGzZknHjxtGrVy9UlX79+nHhhReyePFirrvuOlQVEeHRRx/F5/Nx+eWXs3PnTvLy8rjrrruoW7duyD9DpNmcLsYcJqWd/ovIqcCDqnq+//nfAVT1kSLlzgGeBXqq6ubSNpyUlKRpaYXvo71q1Sratm1bpg8Qr3w+Hz6fj5o1a7JmzRrOO+881qxZQ7Vq0dV71b4zYyJLRNJVNam0csFkikXA8SLSGvgFGAJcXmRjXYAXgT7BJHZTul27dnH22Wfj8/lQVV588cWoS+zGmOhVarZQVZ+I3AzMAaoCL6vqChEZB6Sp6kzgcaAO8F//fCM/qWr/MMYd9xo0aEB6errXYRhjYlRQVUFVnQXMKrLs/gKPzwlxXMYYYyogZkeoGmOMKZkld2OMiUOW3I2JQ9EyQ2a0xFEZWXIvoFevXr8bkDRx4kRuvPHGgO+rU6cOABs2bGDw4MElrrto18+iJk6cWGgw0QUXXMC2bduCCT2gBx98kCeeeKLC6zGxIX+GzPXrQfXwDJmRTqzREkdlZcm9gKFDhzJ9+vRCy6ZPn87QoUODen/z5s1JTk4u9/aLJvdZs2bRoEGDcq/PVE7RMkNmtMRRWUVtx+lbb4VQ34Oic2fwz7RbrMGDB3Pvvfeyf/9+atSoQWZmJhs2bKBHjx7s2rWLAQMGsHXrVnJzc3n44YcZMKDQFDtkZmZy0UUXsXz5cvbu3cs111zDypUradu2LXv37j1U7oYbbmDRokXs3buXwYMH89BDDzFp0iQ2bNjAWWedRePGjUlNTSUxMZG0tDQaN27MU089dWhWyREjRnDrrbeSmZlJ37596dGjB1999RUtWrTg/fff54gjjijxMy5dupRRo0axZ88ejj32WF5++WUaNmzIpEmTeOGFF6hWrRrt2rVj+vTp/O9//2PMmDGAu6XevHnz4mIka7yLlhkyoyWOyspq7gU0atSI7t27M3v2bMDV2i+77DJEhJo1azJjxgwWL15Mamoqd9xxR8DJvZ5//nlq1apFRkYG99xzT6E+6+PHjyctLY2MjAz+97//kZGRwejRo2nevDmpqamkpqYWWld6ejqvvPIKCxcuZMGCBbz00kuHpgBes2YNN910EytWrKBBgwakpKQE/IxXXnkljz76KBkZGZx88sk89NBDAEyYMIElS5aQkZHBCy+8AMATTzzB5MmTWbp0KfPnzw940DDRI1pmyIyWOKJJJK9BRG3NPVANO5zym2YGDBjA9OnTD9WWVZW7776befPmUaVKFX755Rc2bdpE06ZNi13PvHnzGD16NAAdO3akY8eOh1575513mDJlCj6fj40bN7Jy5cpCrxf1xRdfcPHFFx+amXLQoEHMnz+f/v3707p1azp37gwEnlYY3Pzy27Zto2dPN/XPVVddxSWXXHIoxmHDhjFw4EAGDhwIwOmnn87tt9/OsGHDGDRoEC1bhn2yTxMC48cXvisVeDNDZrTEES0ifbcwq7kXMXDgQObOncvixYvZu3cvXbt2BWDatGlkZ2eTnp7O0qVLOeqoo4qd5rcg/2jdQn788UeeeOIJ5s6dS0ZGBhdeeGGp6wl0hpA/XTAEnla4NB999BE33XQT6enpdOvWDZ/Px9ixY5k6dSp79+7llFNO4bvvvivXuk1kRcsMmdESB0RHr51IX4Ow5F5EnTp16NWrF9dee22hC6nbt2/nyCOPJCEhgdTUVNavXx9wPWeeeeahm2AvX76cjIwMwE0XXLt2berXr8+mTZv4+OPDsyPXrVuXnTt3Fruu9957jz179rB7925mzJjBGWecUebPVr9+fRo2bMj8+fMBeP311+nZsyd5eXn8/PPPnHXWWTz22GNs27aNXbt28cMPP3DyySdz1113kZSUZMk9hgwbBpmZkJfnfns1W2Y0xBEtvXYifQ0iaptlvDR06FAGDRpUqOfMsGHD6NevH0lJSXTu3JmTTjop4DpuuOEGrrnmGjp27Ejnzp3p3r074O6q1KVLF9q3b/+76YJHjhxJ3759adasWaF2965du3L11VcfWseIESPo0qVLwCaYkrz22muHLqi2adOGV155hYMHDzJ8+HC2b9+OqnLbbbfRoEED7rvvPlJTU6latSrt2rU7dFcpY2JJoBpzJA82rVq5A0txy8Oh1Cl/w8Wm/I0P9p2ZaFeliquxFyXizigipWibO7hrEGVtqgp2yl9rljHGxLVo6bUT6WsQltyNMXEtmu6vG8lrEFGX3L1qJjJlZ9+ViQXR1GsnkqLqgmrNmjXJycmhUaNGxXYjNNFDVcnJyaFmzZpeh2JMqSrj/XWDSu4i0gd4BncnpqmqOqHI6zWA/wDdgBzgMlXNLGswLVu2JCsri+zs7LK+1XigZs2aNrDJmChVanIXkarAZOBcIAtYJCIzVXVlgWLXAVtV9TgRGQI8ClxW1mASEhJo3bp1Wd9mjDGmiGDa3LsDa1V1naoeAKYDA4qUGQC85n+cDJwt1q5ijDGeCSa5twB+LvA8y7+s2DKq6gO2A42KrkhERopImoikWdOLMcaETzDJvbgaeNFuEsGUQVWnqGqSqiY1adIkmPiMMcaUQzAXVLOAows8bwlsKKFMlohUA+oDvwVaaXp6+hYRCTxBS/RrDGzxOogoYvujMNsfh9m+KKwi++OYYAoFk9wXAceLSGvgF2AIcHmRMjOBq4CvgcHAZ1pKJ2hVjfmqu4ikBTMMuLKw/VGY7Y/DbF8UFon9UWpyV1WfiNwMzMF1hXxZVVeIyDggTVVnAv8GXheRtbga+5BwBm2MMSawoPq5q+osYFaRZfcXeLwPuCS0oRljjCmvqJt+IMZM8TqAKGP7ozDbH4fZvigs7PvDsyl/jTHGhI/V3I0xJg5ZcjfGmDhkyb0cRORoEUkVkVUiskJExngdk9dEpKqILBGRD72OxWsi0kBEkkXkO//fyKlex+QlEbnN/3+yXETeEpFKNZWoiLwsIptFZHmBZX8QkU9FZI3/d8NQb9eSe/n4gDtUtS1wCnCTiLTzOCavjQFWeR1ElHgGmK2qJwGdqMT7RURaAKOBJFXtgOtOXdm6Sr8K9CmybCwwV1WPB+b6n4eUJfdyUNWNqrrY/3gn7p+36Hw7lYaItAQuBKZ6HYvXRKQecCZu7AeqekBVt3kbleeqAUf4R6/X4vcj3OOaqs7j9yP2C062+BowMNTbteReQSKSCHQBFnobiacmAn8DIni74ajVBsgGXvE3U00VkdpeB+UVVf0FeAL4CdgIbFfVT7yNKiocpaobwVUWgSNDvQFL7hUgInWAFOBWVd3hdTxeEJGLgM2qmu51LFGiGtAVeF5VuwC7CcMpd6zwtyUPAFoDzYHaIjLc26gqB0vu5SQiCbjEPk1V3/U6Hg+dDvQXkUzcXP+9ReQNb0PyVBaQpar5Z3LJuGRfWZ0D/Kiq2aqaC7wLnOZxTNFgk4g0A/D/3hzqDVhyLwf/jUj+DaxS1ae8jsdLqvp3VW2pqom4C2WfqWqlrZmp6q/AzyJyon/R2cDKAG+Jdz8Bp4hILf//zdlU4gvMBeRPtoj/9/uh3kBU3SA7hpwOXAF8KyJL/cvu9s/BY8wtwDQRqQ6sA67xOB7PqOpCEUkGFuN6mS2hkk1FICJvAb2AxiKSBTwATADeEZHrcAfAkM/NZdMPGGNMHLJmGWOMiUOW3I0xJg5ZcjfGmDhkyd0YY+KQJXdjjIlDltyNMSYOWXI3xpg49P9ZORrEUAtOsAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 모델은 과대적합이 빠르게 시작됩니다. 훈련 샘플 수가 작기 때문에 놀라운 일은 아닙니다. 같은 이유로 검증 정확도와 훈련 정확도 사이에 차이가 큽니다. 검증 정확도는 50% 후반을 달성한 것 같습니다.\n",
    "\n",
    "훈련 샘플 수가 적기 때문에 어떤 샘플 200개를 선택했는지에 따라 성능이 크게 좌우됩니다. 여기서는 샘플들을 랜덤하게 선택했습니다. 만약 선택한 샘플에서 성능이 나쁘면 예제를 위해서 랜덤하게 200개의 샘플을 다시 추출하세요(실전에서는 훈련 데이터를 고르지 않습니다).\n",
    "\n",
    "사전 훈련된 단어 임베딩을 사용하지 않거나 임베딩 층을 동결하지 않고 같은 모델을 훈련할 수 있습니다. 이런 경우 해당 작업에 특화된 입력 토큰의 임베딩을 학습할 것입니다. 데이터가 풍부하게 있다면 사전 훈련된 단어 임베딩보다 일반적으로 훨씬 성능이 높습니다. 여기서는 훈련 샘플이 200개뿐이지만 한 번 시도해 보죠:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_4 (Embedding)      (None, 100, 100)          1000000   \n",
      "_________________________________________________________________\n",
      "flatten_3 (Flatten)          (None, 10000)             0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 32)                320032    \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 1,320,065\n",
      "Trainable params: 1,320,065\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 200 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.6894 - acc: 0.5200 - val_loss: 0.6953 - val_acc: 0.5117\n",
      "Epoch 2/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.4895 - acc: 0.9650 - val_loss: 0.7149 - val_acc: 0.5138\n",
      "Epoch 3/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.2904 - acc: 0.9800 - val_loss: 0.7029 - val_acc: 0.5150\n",
      "Epoch 4/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.1236 - acc: 1.0000 - val_loss: 0.7255 - val_acc: 0.5194\n",
      "Epoch 5/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0553 - acc: 1.0000 - val_loss: 0.7111 - val_acc: 0.5195\n",
      "Epoch 6/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0275 - acc: 1.0000 - val_loss: 0.7580 - val_acc: 0.5212\n",
      "Epoch 7/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0152 - acc: 1.0000 - val_loss: 0.7334 - val_acc: 0.5197\n",
      "Epoch 8/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0085 - acc: 1.0000 - val_loss: 0.7509 - val_acc: 0.5228\n",
      "Epoch 9/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0052 - acc: 1.0000 - val_loss: 0.7437 - val_acc: 0.5208\n",
      "Epoch 10/10\n",
      "200/200 [==============================] - 0s 1ms/step - loss: 0.0032 - acc: 1.0000 - val_loss: 0.7570 - val_acc: 0.5243\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Embedding, Flatten, Dense\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_words, embedding_dim, input_length=maxlen))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(32, activation='relu'))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy',\n",
    "              metrics=['acc'])\n",
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=32,\n",
    "                    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xt8FPW9//HXRy6GcBewKhFCLfVCSCCuoD9RsVCKVqFeWknxPMQLtFas9fI7Pyr8jjxssT3eqlaPR2q1tqZQflqvx8upFm+1KkEFBQ5CFTGAGhARCQqhn98fMwmbZZOdhA2bTN7PxyOPnct3Zz47Sd47+53ZGXN3REQkXvbLdQEiIpJ9CncRkRhSuIuIxJDCXUQkhhTuIiIxpHAXEYkhhXuMmVkHM/vczAZks20umdnXzCzr5++a2VgzW5M0vtLMTojSthnrutvMrm7u80Wi6JjrAmQ3M/s8aTQf+BLYFY7/wN3Lm7I8d98FdMt22/bA3Q/PxnLM7CLgXHcfnbTsi7KxbJHGKNxbEXevC9dwz/Aid3+mofZm1tHda/ZFbSKZ6O+xdVG3TBtiZj83sz+Z2Twz2wqca2bHmdkrZvapmW0ws9vMrFPYvqOZuZkVhuP3h/OfNLOtZvZ3MxvU1Lbh/FPM7B0z22Jmvzazv5nZlAbqjlLjD8xstZltNrPbkp7bwcx+ZWabzOwfwPhGts8sM5ufMu0OM7s5HL7IzFaEr+cf4V51Q8uqNLPR4XC+mf0hrG0ZcHSa9b4bLneZmU0Ipw8FbgdOCLu8NiZt29lJz/9h+No3mdnDZnZwlG3TlO1cW4+ZPWNmn5jZh2b2r0nr+b/hNvnMzCrM7JB0XWBm9lLt7zncni+E6/kEmGVmg81sYfhaNobbrWfS8weGr7EqnH+rmeWFNR+Z1O5gM6s2sz4NvV7JwN310wp/gDXA2JRpPwd2AKcTvDF3AY4BRhJ8Cvsq8A4wPWzfEXCgMBy/H9gIJIBOwJ+A+5vR9kBgKzAxnHcFsBOY0sBriVLjI0BPoBD4pPa1A9OBZUAB0Ad4IfizTbuerwKfA12Tlv0xkAjHTw/bGPANYDtQHM4bC6xJWlYlMDocvhF4DugNDASWp7T9HnBw+Dv5fljDV8J5FwHPpdR5PzA7HB4X1jgMyAP+A/hrlG3TxO3cE/gIuAzYH+gBjAjn/RRYAgwOX8Mw4ADga6nbGnip9vccvrYa4GKgA8Hf49eBMUDn8O/kb8CNSa/n7XB7dg3bHx/OmwvMSVrPlcBDuf4/bMs/OS9APw38YhoO979meN5VwP8Lh9MF9n8mtZ0AvN2MthcALybNM2ADDYR7xBqPTZr/Z+CqcPgFgu6p2nmnpgZOyrJfAb4fDp8CvNNI28eBS8LhxsJ9bfLvAvhRcts0y30b+HY4nCnc7wOuS5rXg+A4S0GmbdPE7fwvQEUD7f5RW2/K9Cjh/m6GGs4GFoXDJwAfAh3StDseeA+wcPxN4Mxs/1+1px91y7Q9HySPmNkRZvZf4cfsz4Brgb6NPP/DpOFqGj+I2lDbQ5Lr8OC/sbKhhUSsMdK6gPcbqRfgj0BZOPx9oO4gtJmdZmavht0SnxLsNTe2rWod3FgNZjbFzJaEXQufAkdEXC4Er69uee7+GbAZ6J/UJtLvLMN2PhRY3UANhxIEfHOk/j0eZGYLzGxdWMPvUmpY48HB+3rc/W8EnwJGmVkRMAD4r2bWJKjPvS1KPQ3wLoI9xa+5ew/g3wj2pFvSBoI9SwDMzKgfRqn2psYNBKFQK9Opmn8CxppZAUG30R/DGrsADwC/IOgy6QX8d8Q6PmyoBjP7KnAnQddEn3C5/5O03Eynba4n6OqpXV53gu6fdRHqStXYdv4AOKyB5zU0b1tYU37StINS2qS+vn8nOMtraFjDlJQaBppZhwbq+D1wLsGnjAXu/mUD7SQChXvb1x3YAmwLD0j9YB+s83Gg1MxON7OOBP24/VqoxgXAT8ysf3hw7f801tjdPyLoOrgXWOnuq8JZ+xP0A1cBu8zsNIK+4ag1XG1mvSz4HsD0pHndCAKuiuB97iKCPfdaHwEFyQc2U8wDLjSzYjPbn+DN50V3b/CTUCMa286PAgPMbLqZdTazHmY2Ipx3N/BzMzvMAsPM7ACCN7UPCQ7cdzCzaSS9ETVSwzZgi5kdStA1VOvvwCbgOgsOUncxs+OT5v+BoBvn+wRBL3tB4d72XQmcR3CA8y6CPdcWFQboOcDNBP+shwFvEOyxZbvGO4FngbeARQR735n8kaAP/Y9JNX8KXA48RHBQ8myCN6koriH4BLEGeJKk4HH3pcBtwGthmyOAV5Oe+xdgFfCRmSV3r9Q+/ymC7pOHwucPACZHrCtVg9vZ3bcA3wTOIjiA+w5wUjj7BuBhgu38GcHBzbywu20qcDXBwfWvpby2dK4BRhC8yTwKPJhUQw1wGnAkwV78WoLfQ+38NQS/5x3u/nITX7ukqD14IdJs4cfs9cDZ7v5iruuRtsvMfk9wkHZ2rmtp6/QlJmkWMxtP8DH7C4JT6WoI9l5FmiU8fjERGJrrWuJA3TLSXKOAdwk+ro8HvqMDYNJcZvYLgnPtr3P3tbmuJw7ULSMiEkPacxcRiaGc9bn37dvXCwsLc7V6EZE2afHixRvdvbFTj4EchnthYSEVFRW5Wr2ISJtkZpm+pQ2oW0ZEJJYU7iIiMaRwFxGJIYW7iEgMKdxFRGIoY7ib2T1m9rGZvd3AfAtvs7XazJaaWWn2y5TWrLwcCgthv/2Cx/Im3cY7XnW0hhpUh+oAMt+JCTgRKCW8C0+a+acSXCnPgGOBV6PcJeToo492afvuv989P98ddv/k5wfT21sdraEG1RH/OmjgjlqpP5Fu10Rw78aGwv0uoCxpfCVwcKZlKtzjYeDA+n+stT8DB7a/OlpDDaoj/nVEDfds9Ln3p/6ttipp4K48ZjYtvLN6RVVVVRZWLbm2toFLPDU0Pc51tIYaVIfqqJWNcE93m7K0VyNz97nunnD3RL9+Gb89K23AgAZuetfQ9DjX0RpqUB2qo1Y2wr2S+veXLCC4cYO0A3PmQH5+/Wn5+cH09lZHa6hBdaiOOlH6bmi8z/3b1D+g+lqUZarPPT7uvz/oNzQLHvf1garWVEdrqEF1xLsOIva5Z7yeu5nNA0YDfQlu9nsN0Cl8Y/hPMzPgdoIbNlQD57t7xiuCJRIJ14XDRESaxswWu3siU7uMV4V097IM8x24pAm1SZaUl8PMmcEBmQEDgo93k5t7a2URiRXdQ7WNKi+HadOgujoYf//9YBwU8CKiyw+0WTNn7g72WtXVwXQREYV7G9Vazt0VkdZJ4d5GtZZzd0WkdVK4t1Gt5dxdEWmdFO7N0BquMDd5MsydCwMHglnwOHeuDqaKSEBnyzRRazpLZfJkhbmIpKc99ybSWSoi0hYo3JtIZ6mISFugcG8inaUiIm2Bwr2JdJaKiLQFCvcm0lkqItIW6GyZZtBZKiLS2mnPXUQkhhTuIiIxpHAXEYkhhbuISAwp3EVEYkjhLiISQwp3EZEYUriLiMSQwl1EJIYU7iIiMaRwFxGJIYW7iEgMKdxFRGJI4S4iEkMKdxGRGFK4i4jEkMJdRCSGFO4iIjGkcBcRiaFI4W5m481spZmtNrMZaeYPNLNnzWypmT1nZgXZL1VERKLKGO5m1gG4AzgFOAooM7OjUprdCPze3YuBa4FfZLtQERGJLsqe+whgtbu/6+47gPnAxJQ2RwHPhsML08wXEZF9KEq49wc+SBqvDKclWwKcFQ6fAXQ3sz6pCzKzaWZWYWYVVVVVzalXREQiiBLulmaap4xfBZxkZm8AJwHrgJo9nuQ+190T7p7o169fk4sVEZFoOkZoUwkcmjReAKxPbuDu64EzAcysG3CWu2/JVpEiItI0UfbcFwGDzWyQmXUGJgGPJjcws75mVrusnwL3ZLdMERFpiozh7u41wHTgaWAFsMDdl5nZtWY2IWw2GlhpZu8AXwHmtFC9IiISgbmndp/vG4lEwisqKnKybhGRtsrMFrt7IlM7fUNVRCSGFO4iIjGkcBcRiSGFu4hIDCncRURiSOEuIhJDCncRkRhSuIuIxJDCXUQkhhTuIiIxpHAXEYkhhbuISAwp3EVEYkjhLiISQwp3EZEYUriLiMSQwl1EJIYU7iIiMaRwFxGJIYW7iEgMKdxFRGJI4S4iEkMKdxGRGFK4i4jEkMJdRCSGFO4iIjGkcBcRiSGFu4hIDCncRURiSOEuIhJDCncRkRiKFO5mNt7MVprZajObkWb+ADNbaGZvmNlSMzs1+6WKiEhUGcPdzDoAdwCnAEcBZWZ2VEqzWcACdx8OTAL+I9uFiohIdFH23EcAq939XXffAcwHJqa0caBHONwTWJ+9EkVEpKmihHt/4IOk8cpwWrLZwLlmVgk8AVyabkFmNs3MKsysoqqqqhnliohIFFHC3dJM85TxMuB37l4AnAr8wcz2WLa7z3X3hLsn+vXr1/RqRUQkko4R2lQChyaNF7Bnt8uFwHgAd/+7meUBfYGPs1GkiGTPzp07qays5Isvvsh1KdKIvLw8CgoK6NSpU7OeHyXcFwGDzWwQsI7ggOn3U9qsBcYAvzOzI4E8QP0uIq1QZWUl3bt3p7CwELN0H8wl19ydTZs2UVlZyaBBg5q1jIzdMu5eA0wHngZWEJwVs8zMrjWzCWGzK4GpZrYEmAdMcffUrhsRaQW++OIL+vTpo2BvxcyMPn367NWnqyh77rj7EwQHSpOn/VvS8HLg+GZXISL7lIK99dvb35G+oSoi+9SmTZsYNmwYw4YN46CDDqJ///514zt27Ii0jPPPP5+VK1c22uaOO+6gvLw8GyW3SZH23EWk/Sovh5kzYe1aGDAA5syByZObv7w+ffrw5ptvAjB79my6devGVVddVa+Nu+Pu7Ldf+v3Pe++9N+N6LrnkkuYXGQPacxeRBpWXw7Rp8P774B48TpsWTM+21atXU1RUxA9/+ENKS0vZsGED06ZNI5FIMGTIEK699tq6tqNGjeLNN9+kpqaGXr16MWPGDEpKSjjuuOP4+OPgJL1Zs2Zxyy231LWfMWMGI0aM4PDDD+fll18GYNu2bZx11lmUlJRQVlZGIpGoe+NJds0113DMMcfU1Vd7SPGdd97hG9/4BiUlJZSWlrJmzRoArrvuOoYOHUpJSQkzZ87M/saKQOEuIg2aOROqq+tPq64OpreE5cuXc+GFF/LGG2/Qv39/fvnLX1JRUcGSJUv4y1/+wvLly/d4zpYtWzjppJNYsmQJxx13HPfcc0/aZbs7r732GjfccEPdG8Wvf/1rDjroIJYsWcKMGTN444030j73sssuY9GiRbz11lts2bKFp556CoCysjIuv/xylixZwssvv8yBBx7IY489xpNPPslrr73GkiVLuPLKK7O0dZpG4S4iDVq7tmnT99Zhhx3GMcccUzc+b948SktLKS0tZcWKFWnDvUuXLpxyyikAHH300XV7z6nOPPPMPdq89NJLTJo0CYCSkhKGDBmS9rnPPvssI0aMoKSkhOeff55ly5axefNmNm7cyOmnnw4E56Xn5+fzzDPPcMEFF9ClSxcADjjggKZviCxQn7uINGjAgKArJt30ltC1a9e64VWrVnHrrbfy2muv0atXL84999y0pwZ27ty5brhDhw7U1NSkXfb++++/R5soZ2xXV1czffp0Xn/9dfr378+sWbPq6kh3Rou7t4qzkbTnLiINmjMH8vPrT8vPD6a3tM8++4zu3bvTo0cPNmzYwNNPP531dYwaNYoFCxYA8NZbb6X9ZLB9+3b2228/+vbty9atW3nwwQcB6N27N3379uWxxx4Dgu8PVFdXM27cOH7729+yfft2AD755JOs1x2Fwl1EGjR5MsydCwMHglnwOHfu3p0tE1VpaSlHHXUURUVFTJ06leOPz/5XaS699FLWrVtHcXExN910E0VFRfTs2bNemz59+nDeeedRVFTEGWecwciRI+vmlZeXc9NNN1FcXMyoUaOoqqritNNOY/z48SQSCYYNG8avfvWrrNcdheXqi6SJRMIrKipysm6R9mzFihUceeSRuS6jVaipqaGmpoa8vDxWrVrFuHHjWLVqFR07to4e63S/KzNb7O6JTM9tHa9ARCQHPv/8c8aMGUNNTQ3uzl133dVqgn1vxeNViIg0Q69evVi8eHGuy2gR6nMXEYkhhbuISAwp3EVEYkjhLiISQwp3EdmnRo8evccXkm655RZ+9KMfNfq8bt26AbB+/XrOPvvsBped6RTrW265heqkC+aceuqpfPrpp1FKb1MU7iKyT5WVlTF//vx60+bPn09ZWVmk5x9yyCE88MADzV5/arg/8cQT9OrVq9nLa60U7iKyT5199tk8/vjjfPnllwCsWbOG9evXM2rUqLrzzktLSxk6dCiPPPLIHs9fs2YNRUVFQHBpgEmTJlFcXMw555xT95V/gIsvvrjucsHXXHMNALfddhvr16/n5JNP5uSTTwagsLCQjRs3AnDzzTdTVFREUVFR3eWC16xZw5FHHsnUqVMZMmQI48aNq7eeWo899hgjR45k+PDhjB07lo8++ggIzqU///zzGTp0KMXFxXWXL3jqqacoLS2lpKSEMWPGZGXbJtN57iLt2E9+AmkuX75Xhg2DMBfT6tOnDyNGjOCpp55i4sSJzJ8/n3POOQczIy8vj4ceeogePXqwceNGjj32WCZMmNDghbjuvPNO8vPzWbp0KUuXLqW0tLRu3pw5czjggAPYtWsXY8aMYenSpfz4xz/m5ptvZuHChfTt27feshYvXsy9997Lq6++irszcuRITjrpJHr37s2qVauYN28ev/nNb/je977Hgw8+yLnnnlvv+aNGjeKVV17BzLj77ru5/vrruemmm/jZz35Gz549eeuttwDYvHkzVVVVTJ06lRdeeIFBgwa1yPVntOcuIvtcctdMcpeMu3P11VdTXFzM2LFjWbduXd0ecDovvPBCXcgWFxdTXFxcN2/BggWUlpYyfPhwli1blvaiYMleeuklzjjjDLp27Uq3bt0488wzefHFFwEYNGgQw4YNAxq+rHBlZSXf+ta3GDp0KDfccAPLli0D4Jlnnql3V6jevXvzyiuvcOKJJzJo0CCgZS4LrD13kXassT3slvSd73yHK664gtdff53t27fX7XGXl5dTVVXF4sWL6dSpE4WFhWkv85ss3V79e++9x4033siiRYvo3bs3U6ZMybicxq6zVXu5YAguGZyuW+bSSy/liiuuYMKECTz33HPMnj27brmpNe6LywJrz11E9rlu3boxevRoLrjggnoHUrds2cKBBx5Ip06dWLhwIe+nu5h8khNPPLHuJthvv/02S5cuBYLLBXft2pWePXvy0Ucf8eSTT9Y9p3v37mzdujXtsh5++GGqq6vZtm0bDz30ECeccELk17Rlyxb69+8PwH333Vc3fdy4cdx+++1145s3b+a4447j+eef57333gNa5rLACncRyYmysjKWLFlSdyckgMmTJ1NRUUEikaC8vJwjjjii0WVcfPHFfP755xQXF3P99dczYsQIILir0vDhwxkyZAgXXHBBvcsFT5s2jVNOOaXugGqt0tJSpkyZwogRIxg5ciQXXXQRw4cPj/x6Zs+ezXe/+11OOOGEev35s2bNYvPmzRQVFVFSUsLChQvp168fc+fO5cwzz6SkpIRzzjkn8nqi0iV/RdoZXfK37dibS/5qz11EJIYU7iIiMaRwFxGJIYW7SDuUq2NtEt3e/o4U7iLtTF5eHps2bVLAt2LuzqZNm8jLy2v2MvQlJpF2pqCggMrKSqqqqnJdijQiLy+PgoKCZj9f4S7SznTq1Knua+8SX+qWERGJoUjhbmbjzWylma02sxlp5v/KzN4Mf94xs/hd+V5EpA3J2C1jZh2AO4BvApXAIjN71N3rLrHm7pcntb8UiP6dXRERybooe+4jgNXu/q677wDmAxMbaV8GzMtGcSIi0jxRwr0/8EHSeGU4bQ9mNhAYBPy1gfnTzKzCzCp0pF5EpOVECfd0Fx1u6ATZScAD7r4r3Ux3n+vuCXdP9OvXL2qNIiLSRFHCvRI4NGm8AFjfQNtJqEtGRCTnooT7ImCwmQ0ys84EAf5oaiMzOxzoDfw9uyWKiEhTZQx3d68BpgNPAyuABe6+zMyuNbMJSU3LgPmu7zSLiORcpG+ouvsTwBMp0/4tZXx29soSEZG9oW+oiojEkMJdRCSGFO4iIjGkcBcRiSGFu4hIDCncRURiSOEuIhJDCncRkRhSuIuIxJDCXUQkhhTuIiIxpHAXEYkhhbuISAwp3EVEYkjhLiISQwp3EZEYUriLiMSQwl1EJIYU7iIiMaRwFxGJIYW7iEgMKdxFRGJI4S4iEkMKdxGRGFK4i4jEkMJdRCSGFO4iIjGkcBcRiSGFu4hIDCncRURiSOEuIhJDkcLdzMab2UozW21mMxpo8z0zW25my8zsj9ktU0REmqJjpgZm1gG4A/gmUAksMrNH3X15UpvBwE+B4919s5kd2FIFi4hIZlH23EcAq939XXffAcwHJqa0mQrc4e6bAdz94+yWKSIiTREl3PsDHySNV4bTkn0d+LqZ/c3MXjGz8ekWZGbTzKzCzCqqqqqaV7GIiGQUJdwtzTRPGe8IDAZGA2XA3WbWa48nuc9194S7J/r169fUWkVEJKIo4V4JHJo0XgCsT9PmEXff6e7vASsJwl5ERHIgSrgvAgab2SAz6wxMAh5NafMwcDKAmfUl6KZ5N5uFiohIdBnD3d1rgOnA08AKYIG7LzOza81sQtjsaWCTmS0HFgL/2903tVTRIiLSOHNP7T7fNxKJhFdUVORk3SIibZWZLXb3RKZ2+oaqiEgMKdxFRGJI4S4iEkMKdxGRGFK4i4jEkMJdRCSGFO4iIjGkcBcRiSGFu4hIDCncRURiSOEuIhJDCncRkRhSuIuIxJDCXUQkhhTuIiIxpHAXEYkhhbuISAwp3EVEYkjhLiISQwp3EZEYUriLiMSQwl1EJIYU7iIiMaRwFxGJoY65LqApysth5kxYuxYGDIA5c2Dy5FxXJdK+1dTA9u1QXd3w4z//CfvvD507N/yTbv5+Odz93LULtm0L6t+2bc/h5s6rroabboLzz2/Z+ttMuJeXw7RpwYYBeP/9YBwU8LKbe/BPuXPn7p8dO+qP74tpNTXQsSN06rT7MXm4qdOa2x4yB2+Ux8bm7dzZcr/PDh0yvwE0Zf7OndGD+Msvm1arGXTtCvn5wWPycO/e9ccHD26Z7VWvHndv+bWkkUgkvKKiInL7wsIg0FMNHAhr1mStrHbHPfhD3ro1+Pnss+Y9bt0aBFrtMpOXn244aru9GW5JycHaufPu4eSwranZHfTJj8nDu3btm3qj6tw5CKAuXRp/jNIm+XG//YI3wHQ/X37Z9HnNfU6nTnsGb7rxxuY11DYvLwj4lmZmi909kaldm9lzX7u2adPbsn/+M/jn37UreMw0XF3d/GD+/PNgfZl06ADdu0OPHrsfe/aEQw8Nxrt3372nCPX/yBsajtquqcPJIZsuePd2WseO2fsndm88/PdmWu0eddQA7tIl+D1LPLSZcB8wIP2e+4ABu4dr/1GS372T38VTpzV3Xu1w7T9S1BBON5xuXrb2PlMDuXv3IJALCuqHdJTHLl32zV5Je2O2+41DJJvaTLjPmVO/zx2Cf4xPPgkCqDZ0s/2xvGPHPfvyaodrP37X/tT2D+bnB8PJ01tquEOHYH0KZBFJ1mbCvfag6eWXQ1VV0MeVSMCQIXuGbupjU+fVDnfqpI+pItI2RQp3MxsP3Ap0AO5291+mzJ8C3ACsCyfd7u53Z7FOIAh4nRkjIpJZxnA3sw7AHcA3gUpgkZk96u7LU5r+yd2nt0CNIiLSRFG+IjACWO3u77r7DmA+MLFlyxIRkb0RJdz7Ax8kjVeG01KdZWZLzewBMzs0K9WJiEizRAn3dOdbpJ6T8hhQ6O7FwDPAfWkXZDbNzCrMrKKqqqpplYqISGRRwr0SSN4TLwDWJzdw903uXvtl3d8AR6dbkLvPdfeEuyf69evXnHpFRCSCKOG+CBhsZoPMrDMwCXg0uYGZHZw0OgFYkb0SRUSkqTKeLePuNWY2HXia4FTIe9x9mZldC1S4+6PAj81sAlADfAJMacGaRUQkgzZz4TAREYl+4bCchbuZVQFprhbTpvQFNua6iFZE22M3bYv6tD3q25vtMdDdMx60zFm4x4GZVUR5B20vtD1207aoT9ujvn2xPXSbPRGRGFK4i4jEkMJ978zNdQGtjLbHbtoW9Wl71Nfi20N97iIiMaQ9dxGRGFK4i4jEkMK9GczsUDNbaGYrzGyZmV2W65pyzcw6mNkbZvZ4rmvJNTPrFV4d9X/Cv5Hjcl1TLpnZ5eH/ydtmNs/M8nJd075iZveY2cdm9nbStAPM7C9mtip87N0S61a4N08NcKW7HwkcC1xiZkfluKZcuwxdU6jWrcBT7n4EUEI73i5m1h/4MZBw9yKCS5hMym1V+9TvgPEp02YAz7r7YODZcDzrFO7N4O4b3P31cHgrwT9vumvctwtmVgB8G8j6rRXbGjPrAZwI/BbA3Xe4+6e5rSrnOgJdzKwjkE/KVWXjzN1fILjeVrKJ7L4s+n3Ad1pi3Qr3vWRmhcBw4NXcVpJTtwD/Cvwz14W0Al8FqoB7w26qu82sa66LyhV3XwfcCKwFNgBb3P2/c1tVzn3F3TdAsKMIHNgSK1G47wUz6wY8CPzE3T/LdT25YGanAR+7++Jc19JKdARKgTvdfTiwjRb62N0WhP3JE4FBwCFAVzM7N7dVtQ8K92Yys04EwV7u7n/OdT05dDwwwczWENxf9xtmdn9uS8qpSqDS3Ws/yT1AEPbt1VjgPXevcvedwJ+B/5XjmnLto9p7YISPH7fEShTuzWBmRtCnusLdb851Pbnk7j919wJ3LyQ4UPZXd2+3e2bu/iHwgZkdHk4aAyzPYUm5thY41szyw/+bMbTjA8yhR4HzwuHzgEdaYiUZb9YhaR0P/Avwlpm9GU672t2fyGFN0npcCpSHdy57Fzg/x/XkjLu/amYPAK8YnHRsAAAAVElEQVQTnGX2Bu3oUgRmNg8YDfQ1s0rgGuCXwAIzu5Dgze+7LbJuXX5ARCR+1C0jIhJDCncRkRhSuIuIxJDCXUQkhhTuIiIxpHAXEYkhhbuISAz9f3ENODCbvDuFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xt8VPWd//HXh3C/CAqRW4AA4oU7mLL6kwp46WJVYC22YLRq9Ud1tWp19ydV21V+sqvWn1q6/Gypq9uVVOrqWmmrsrvKVl0rEi4NAlIucokgBBQEQSHw2T++EzJJJskkmWQmh/fz8ZjHzDnznXM+M4H3+c73nDnH3B0REYmWFukuQEREUk/hLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwl4TMLMvMDphZ31S2TSczO83MUn7sr5ldZGab46bXmdlXk2lbj3U9ZWb31Pf1NSz3QTP751QvV9KnZboLkNQwswNxk+2BL4GjsenvuntBXZbn7keBjqlueyJw9zNSsRwzuxG42t3Hxy37xlQsW6JP4R4R7n48XGM9wxvd/T+ra29mLd29tClqE5Gmp2GZE0Tsa/evzew5M9sPXG1m55rZu2a218x2mNkcM2sVa9/SzNzMcmPT82PPv2pm+83sj2bWv65tY89fYmZ/NrN9ZvZTM/tvM7uumrqTqfG7ZrbBzD41szlxr80ys8fNbI+ZbQQm1vD53GdmCyrNm2tmj8Ue32hma2PvZ2OsV13dsorNbHzscXszezZW22rg7ATr3RRb7mozmxSbPwz4R+CrsSGv3XGf7f1xr78p9t73mNlvzKxnMp9NbcxsSqyevWb2hpmdEffcPWa23cw+M7MP4t7rOWa2PDZ/p5n9ONn1SSNwd90idgM2AxdVmvcgcBi4nLBRbwd8BfgLwje4AcCfgVtj7VsCDuTGpucDu4E8oBXwa2B+PdqeCuwHJseeuxM4AlxXzXtJpsaXgc5ALvBJ2XsHbgVWAzlAV+DN8E8+4XoGAAeADnHL3gXkxaYvj7Ux4ALgEDA89txFwOa4ZRUD42OPHwX+CzgZ6AesqdT2m0DP2N/kqlgN3WPP3Qj8V6U65wP3xx5/LVbjSKAt8P+BN5L5bBK8/weBf449PitWxwWxv9E9sc+9FTAE2AL0iLXtDwyIPV4KTI897gT8Rbr/L5zIN/XcTyxvu/tv3f2Yux9y96XuvsTdS919EzAPGFfD619w90J3PwIUEEKlrm0vA1a6+8ux5x4nbAgSSrLGf3D3fe6+mRCkZev6JvC4uxe7+x7goRrWswl4n7DRAbgY2OvuhbHnf+vumzx4A3gdSLjTtJJvAg+6+6fuvoXQG49f7/PuviP2N/kVYcOcl8RyAfKBp9x9pbt/AcwExplZTlyb6j6bmkwDFrr7G7G/0UPASYSNbClhQzIkNrT3Yeyzg7CRHmRmXd19v7svSfJ9SCNQuJ9YtsVPmNmZZvZ7M/vYzD4DZgHdanj9x3GPD1LzTtTq2vaKr8PdndDTTSjJGpNaF6HHWZNfAdNjj68ibJTK6rjMzJaY2SdmtpfQa67psyrTs6YazOw6M/tTbPhjL3BmksuF8P6OL8/dPwM+BXrHtanL36y65R4j/I16u/s64C7C32FXbJivR6zp9cBgYJ2ZvWdmX0/yfUgjULifWCofBvhzQm/1NHc/CfgRYdihMe0gDJMAYGZGxTCqrCE17gD6xE3Xdqjmr4GLYj3fyYSwx8zaAS8A/0AYMukC/HuSdXxcXQ1mNgB4ErgZ6Bpb7gdxy63tsM3thKGesuV1Igz/fJREXXVZbgvC3+wjAHef7+7nEYZksgifC+6+zt2nEYbe/h/wopm1bWAtUk8K9xNbJ2Af8LmZnQV8twnW+TtgtJldbmYtgduB7Eaq8XngDjPrbWZdgbtrauzuO4G3gWeAde6+PvZUG6A1UAIcNbPLgAvrUMM9ZtbFwu8Abo17riMhwEsI27kbCT33MjuBnLIdyAk8B9xgZsPNrA0hZN9y92q/CdWh5klmNj627r8l7CdZYmZnmdmE2PoOxW5HCW/gGjPrFuvp74u9t2MNrEXqSeF+YrsLuJbwH/fnhJ5ro4oF6LeAx4A9wEBgBeG4/FTX+CRhbHwVYWffC0m85leEHaS/iqt5L/B94CXCTsmphI1UMv6O8A1iM/Aq8C9xyy0C5gDvxdqcCcSPU/8HsB7YaWbxwytlr3+NMDzyUuz1fQnj8A3i7qsJn/mThA3PRGBSbPy9DfAIYT/Jx4RvCvfFXvp1YK2Fo7EeBb7l7ocbWo/Uj4UhT5H0MLMswjDAVHd/K931iESFeu7S5Mxsopl1jn21/yHhCIz30lyWSKQo3CUdxgKbCF/tJwJT3L26YRkRqQcNy4iIRJB67iIiEZS2E4d169bNc3Nz07V6EZFmadmyZbvdvabDh4E0hntubi6FhYXpWr2ISLNkZrX90hrQsIyISCQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEZS249xFosgd3n4b/vhHOOMMGDUK+vQBa+xLoEjGO3wYduyA7duhXz/o1atx16dwF0mBP/8Znn0W5s+HzZsrPnfKKSHkR44M96NGheDPykpLqZJix45BSUkI7e3b4aOPKt6XPS4pKX/Nk0/CTTc1bl0Kd5F62r0bFiwIof7ee9CiBVx4IcyaBX/5l7BxI6xcCStWhNs//iN8GTv3Zbt2MHx4ediPGgVDh4b5zd3Bg7BpU3j/GzeGx4cOwUknhVvnzjU/7tgxfJbp5g6ffVZ7aO/YAaWlFV9rBqeeCr17Q04OjBkTHvfqFW4jk7lMeQOl7ayQeXl5rtMPSHPzxRfw29+GQH/11fCfesQIuOYamD695q/aR47ABx+Uh/2KFSH89+0Lz2dlwZlnVgz8kSPh5JOb5r0lyz30QuPDu+zxxo3wcaVrRnXuHAJ7//4QlrUxg06dkt8YVPe4Q4fqh8O+/LL20N6+HT7/vOprO3cuD+r4wI5/3KMHtKru4ogNZGbL3D2v1nYKd5GaHTsWxtGffRb+9V9DGPfqBfn5IdSHDav/st3DME584K9YEYKlTG5u1WGd3r0bdxz/yBHYujVxeG/aBAcOVGyfkwMDB8KAAeG+7DZgQBiWKqv12LHw2n37QtB/9ln9HicK3cpatCgP/LLb/v3hs92zp2r7Nm1qDu3evaFnz7DRSCeFu0gDrVtXPo6+ZUv4T/2Nb4RAnzChccfMd+0q79mXBf769WFjANCtW8Ue/qhRcNppdatp//7qw3vLFjh6tLxtmzbQv3/V4B44MMxv2za17782R4+G+uu6YejYsfrwPvnk5rHjW+EuUg8lJeXj6EuXht7fxReHQJ8yJb29tv37oaioYg///fdDLxtCbZXH8bt3D98MKof3xo0Vd/BB6GEnCu+BA0P4ZcI4uCjcRZJ26FD5OPprr4Vx9JEjy8fRe/ZMd4XVO3wY1q6tOo6/f3/Vti1ahMMyE4X3gAHQpUvT1y91p3AXqcGxY/DWW+Xj6J99Fr6el42jDx2a7grr79ix0DtfuTL0zvv3D+GdmwutW6e7OmmoZMNdh0JKnbjDzp2walUYEli1KtzWrg3jrn36hJ1rffpUfJyTE25NPTZb2QcfhEAvKAjjyh07lo+jjx8fjWPPW7QI4++nnZbuSiSdFO5Srf37YfXqqkG+e3d5m+7dQy/3hhvC4WXFxeEoi3fegU8+qbrMbt0SB3/8BqBNm9S+j127ysfRCwtD+H3ta/D3fw+TJ6f/6AeRxqBwr4fS0rCTav/+cDxrdja0bMaf5JEj4ReWZeFdFuYffljepkOHEOKTJ4dD/8pu2TVcyfHzz8Pxwtu2hdCPv//wwzAs8umnVV936qmJg7/svnfv2jcAhw7BwoXl4+hHj4YdjI89FsbRe/So32cl0lxozL0Gu3eHw+Eq3zZuLD9CAcLhU926hcCIv3XvXnX6lFPSd9SBewjW+BBftSoMVZS9n6ys8NP4YcNCmJeFeG5u49T9+edVg7/y/d69VV/XvXvi4O/YEV5+GV54IYyj5+SUj6MPGZL6+kWaWkp3qJrZROAnQBbwlLs/VOn5x4EJscn2wKnuXuO+90wJ9y+/DGGdKMTjhxVatQpjmGecUX7r3Dl85f/44/Lbzp3lj7/4our6WrasGPqVNwDx8zp1qv9xt598UnU45f33K/5CsE+f8vAuC/Izz0z9sEhDHTgQgr4s7BNtAMp+5Qkh4KdOLR9H1yF8EiUpC3czywL+DFwMFANLgenuvqaa9t8DRrn7d2pablOGu3sI20QB/uGH4eiCMj17Vgzwslu/fnUbeik7L0XlwE+0Idi5s+IPRsq0a1f7BqB79zC0UTnI43/h2KVLxaGUoUPDLUqHvu3fH4J+9244+2xo3z7dFYk0jlQeLTMG2ODum2ILXgBMBhKGOzAd+LtkC02lgwfDr/gShXj8cb/t2sHpp4cQuOqq8gA//fTwE+VUMAs9+86dw7JrcuxY+Dl0dRuCnTthw4bwE/j4nZmVtWkDZ50VTl4VH+SN/VP1TNCpU3jvIhIkE+69gW1x08XAXyRqaGb9gP7AG9U8PwOYAdC3b986FQrh8LV77glHY5x6KkycGMK4LMC3bq3Yvm/fEKzXXluxF56Tk1lf1Vu0CDsms7NrP0/JkSNhKCh+I9CpU3jdaac17x27IpI6yURBoj5fdWM504AX3D3BIAO4+zxgHoRhmaQqjCkogBkzQu8cQsD9y7+EXvjQoXD++aHnXRbggwZF86t5q1ahJ967d7orEZFMlky4FwN94qZzgO3VtJ0G3NLQohK5997yYI+XnR3OpS0iIuWSGZxYCgwys/5m1poQ4AsrNzKzM4CTgT+mtsSg8pBLmW3bEs8XETmR1Rru7l4K3AosAtYCz7v7ajObZWaT4ppOBxZ4Ix04X90QfT2G7kVEIi+p3W/u/grwSqV5P6o0fX/qyqpq9uyKY+4QxtRnz27MtYqINE8ZdMxIzfLzYd68cLy5WbifNy/MFxGRiprVgXP5+QpzEZFkNJueu4iIJE/hLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiaCkwt3MJprZOjPbYGYzq2nzTTNbY2arzexXqS1TRETqotaLdZhZFjAXuBgoBpaa2UJ3XxPXZhDwA+A8d//UzE5trIJFRKR2yfTcxwAb3H2Tux8GFgCTK7X538Bcd/8UwN13pbZMERGpi2TCvTewLW66ODYv3unA6Wb232b2rplNTFWBIiJSd8lcQ9USzPMEyxkEjAdygLfMbKi7762wILMZwAyAvn371rlYERFJTjI992KgT9x0DrA9QZuX3f2Iu38IrCOEfQXuPs/d89w9Lzs7u741i4hILZIJ96XAIDPrb2atgWnAwkptfgNMADCzboRhmk2pLFRERJJXa7i7eylwK7AIWAs87+6rzWyWmU2KNVsE7DGzNcBi4G/dfU9jFS0iIjUz98rD500jLy/PCwsL07JuEZHmysyWuXtebe30C1URkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEUFLhbmYTzWydmW0ws5kJnr/OzErMbGXsdmPqSxURkWS1rK2BmWUBc4GLgWJgqZktdPc1lZr+2t1vbYQaRUSkjpLpuY8BNrj7Jnc/DCwAJjduWSIi0hDJhHtvYFvcdHFsXmXfMLMiM3vBzPokWpCZzTCzQjMrLCkpqUe5maGgAHJzoUWLcF9QkO6KREQqSibcLcE8rzT9WyDX3YcD/wn8MtGC3H2eu+e5e152dnbdKs0QBQUwYwZs2QLu4X7GDAW8iGSWZMK9GIjviecA2+MbuPsed/8yNvkL4OzUlJd57r0XDh6sOO/gwTBfRCRTJBPuS4FBZtbfzFoD04CF8Q3MrGfc5CRgbepKzCxbt9ZtvohIOtR6tIy7l5rZrcAiIAt42t1Xm9ksoNDdFwK3mdkkoBT4BLiuEWtOq759w1BMovkiIpnC3CsPnzeNvLw8LywsTMu6G6JszD1+aKZ9e5g3D/Lz01eXiJwYzGyZu+fV1k6/UK2j/PwQ5P36gVm4V7CLSKapdVhGqsrPV5iLSGZTz11EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhKKtzNbKKZrTOzDWY2s4Z2U83MzazWq4SIiEjjqTXczSwLmAtcAgwGppvZ4ATtOgG3AUtSXaSIiNRNMj33McAGd9/k7oeBBcDkBO3+L/AI8EUK6xMRkXpIJtx7A9vipotj844zs1FAH3f/XU0LMrMZZlZoZoUlJSV1LlZERJKTTLhbgnl+/EmzFsDjwF21Lcjd57l7nrvnZWdnJ1+liIjUSTLhXgz0iZvOAbbHTXcChgL/ZWabgXOAhdqpKiKSPsmE+1JgkJn1N7PWwDRgYdmT7r7P3bu5e6675wLvApPcvbBRKhYRkVrVGu7uXgrcCiwC1gLPu/tqM5tlZpMau0AREam7lsk0cvdXgFcqzftRNW3HN7wsERFpCP1CVUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCkgp3M5toZuvMbIOZzUzw/E1mtsrMVprZ22Y2OPWliohIsmoNdzPLAuYClwCDgekJwvtX7j7M3UcCjwCPpbxSERFJWjI99zHABnff5O6HgQXA5PgG7v5Z3GQHwFNXooiI1FUy4d4b2BY3XRybV4GZ3WJmGwk999sSLcjMZphZoZkVlpSU1KdeiVNQALm50KJFuC8oSHdFIpIpkgl3SzCvSs/c3ee6+0DgbuC+RAty93nunufuednZ2XWrVCooKIAZM2DLFnAP9zNmKOBFJEgm3IuBPnHTOcD2GtovAKY0pCip3b33wsGDFecdPBjmi4gkE+5LgUFm1t/MWgPTgIXxDcxsUNzkpcD61JUoiWzdWrf5InJiaVlbA3cvNbNbgUVAFvC0u682s1lAobsvBG41s4uAI8CnwLWNWbRA375hKCbRfBGRWsMdwN1fAV6pNO9HcY9vT3FdUovZs8MYe/zQTPv2Yb6IiH6h2kzl58O8edCvH5iF+3nzwnwRkaR67pKZ8vMV5iKSmHruIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEUFLhbmYTzWydmW0ws5kJnr/TzNaYWZGZvW5m/VJfqoiIJKvWcDezLGAucAkwGJhuZoMrNVsB5Ln7cOAF4JFUFyoiIslLpuc+Btjg7pvc/TCwAJgc38DdF7t72dU83wVyUlumiIjURTLh3hvYFjddHJtXnRuAVxM9YWYzzKzQzApLSkqSr1JEROokmXC3BPM8YUOzq4E84MeJnnf3ee6e5+552dnZyVcpIiJ1kswFsouBPnHTOcD2yo3M7CLgXmCcu3+ZmvJERKQ+kum5LwUGmVl/M2sNTAMWxjcws1HAz4FJ7r4r9WWKiEhd1Bru7l4K3AosAtYCz7v7ajObZWaTYs1+DHQE/tXMVprZwmoWJyIiTSCZYRnc/RXglUrzfhT3+KIU1yUiIg2gX6iKiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcJcGKyiA3Fxo0SLcFxSkuyIRSerEYSLVKSiAGTPgYOwii1u2hGmA/Pz01SVyolPPXRrk3nvLg73MwYNhvoikj8JdGmTr1rrNF5GmoXCXBunbt27zRaRpKNylQWbPhvbtK85r3z7MF5H0SSrczWyima0zsw1mNjPB8+eb2XIzKzWzqakvUzJVfj7Mmwf9+oFZuJ83TztTRdKt1qNlzCwLmAtcDBQDS81sobuviWu2FbgO+JuGFHPkyBGKi4v54osvGrIYaSJt27YlJyeH/PxWCnORDJPMoZBjgA3uvgnAzBYAk4Hj4e7um2PPHWtIMcXFxXTq1Inc3FzMrCGLkkbm7uzZs4fi4mL69++f7nJEpJJkhmV6A9vipotj81Luiy++oGvXrgr2ZsDM6Nq1q75liWSoZMI9UdJ6fVZmZjPMrNDMCktKSqprU59FSxrobyWSuZIJ92KgT9x0DrC9Pitz93nunufuednZ2fVZhIiIJCGZcF8KDDKz/mbWGpgGLGzcspKT6nOa7Nmzh5EjRzJy5Eh69OhB7969j08fPnw4qWVcf/31rFu3rsY2c+fOpSBFJ2AZO3YsK1euTMmyRCQ6at2h6u6lZnYrsAjIAp5299VmNgsodPeFZvYV4CXgZOByM3vA3Yc0ZuGNcU6Trl27Hg/K+++/n44dO/I3f1PxACB3x91p0SLxdvGZZ56pdT233HJL/QoUEUlSUse5u/sr7n66uw9099mxeT9y94Wxx0vdPcfdO7h718YOdmjac5ps2LCBoUOHctNNNzF69Gh27NjBjBkzyMvLY8iQIcyaNet427KedGlpKV26dGHmzJmMGDGCc889l127dgFw33338cQTTxxvP3PmTMaMGcMZZ5zBO++8A8Dnn3/ON77xDUaMGMH06dPJy8urtYc+f/58hg0bxtChQ7nnnnsAKC0t5Zprrjk+f86cOQA8/vjjDB48mBEjRnD11Ven/DMTkfRqtmeFbOpzmqxZs4ZnnnmGn/3sZwA89NBDnHLKKZSWljJhwgSmTp3K4MGDK7xm3759jBs3joceeog777yTp59+mpkzq/wGDHfnvffeY+HChcyaNYvXXnuNn/70p/To0YMXX3yRP/3pT4wePbrG+oqLi7nvvvsoLCykc+fOXHTRRfzud78jOzub3bt3s2rVKgD27t0LwCOPPMKWLVto3br18XkiEh3N9vQDTX1Ok4EDB/KVr3zl+PRzzz3H6NGjGT16NGvXrmXNmjVVXtOuXTsuueQSAM4++2w2b96ccNlXXHFFlTZvv/0206ZNA2DEiBEMGVLzl6ElS5ZwwQUX0K1bN1q1asVVV13Fm2++yWmnnca6deu4/fbbWbRoEZ07dwZgyJAhXH311RQUFNCqVas6fRYikvmabbg39TlNOnTocPzx+vXr+clPfsIbb7xBUVEREydOTHi8d+vWrY8/zsrKorS0NOGy27RpU6WNe92ONq2ufdeuXSkqKmLs2LHMmTOH7373uwAsWrSIm266iffee4+8vDyOHj1ap/WJSGZrtuGeznOafPbZZ3Tq1ImTTjqJHTt2sGjRopSvY+zYsTz//PMArFq1KuE3g3jnnHMOixcvZs+ePZSWlrJgwQLGjRtHSUkJ7s6VV17JAw88wPLlyzl69CjFxcVccMEF/PjHP6akpISDlXdgNEO6IpRIuWY75g4hyNNxTpPRo0czePBghg4dyoABAzjvvPNSvo7vfe97fPvb32b48OGMHj2aoUOHHh9SSSQnJ4dZs2Yxfvx43J3LL7+cSy+9lOXLl3PDDTfg7pgZDz/8MKWlpVx11VXs37+fY8eOcffdd9OpU6eUv4empCtCiVRkdf36nyp5eXleWFhYYd7atWs566yz0lJPpiktLaW0tJS2bduyfv16vva1r7F+/Xpatsys7XGm/M1yc0OgV9avH1Szq0OkWTKzZe6eV1u7zEoKOe7AgQNceOGFlJaW4u78/Oc/z7hgzyS6IpRIRUqLDNWlSxeWLVuW7jKajb59E/fcdUUoOVE12x2qIvF0RSiRihTuEgm6IpRIRRqWkchI19FTIplIPXeRFNKx9pIpFO5xxo8fX+UHSU888QR//dd/XePrOnbsCMD27duZOjXx9cHHjx9P5UM/K3viiScq/Jjo61//ekrO+3L//ffz6KOPNng5UrOyY+23bAH38mPtFfCSDgr3ONOnT2fBggUV5i1YsIDp06cn9fpevXrxwgsv1Hv9lcP9lVdeoUuXLvVenjStpjxTqUhtMnbM/Y47INXXoBg5EmJn2k1o6tSp3HfffXz55Ze0adOGzZs3s337dsaOHcuBAweYPHkyn376KUeOHOHBBx9k8uTJFV6/efNmLrvsMt5//30OHTrE9ddfz5o1azjrrLM4dOjQ8XY333wzS5cu5dChQ0ydOpUHHniAOXPmsH37diZMmEC3bt1YvHgxubm5FBYW0q1bNx577DGefvppAG688UbuuOMONm/ezCWXXMLYsWN555136N27Ny+//DLt2rWr9j2uXLmSm266iYMHDzJw4ECefvppTj75ZObMmcPPfvYzWrZsyeDBg1mwYAF/+MMfuP3224FwSb0333yz2f+StTFl0rH2BQVho7J1azgcdPZs7Y840ajnHqdr166MGTOG1157DQi99m9961uYGW3btuWll15i+fLlLF68mLvuuqvGk3s9+eSTtG/fnqKiIu69994Kx6zPnj2bwsJCioqK+MMf/kBRURG33XYbvXr1YvHixSxevLjCspYtW8YzzzzDkiVLePfdd/nFL37BihUrgHASs1tuuYXVq1fTpUsXXnzxxRrf47e//W0efvhhioqKGDZsGA888AAQTmG8YsUKioqKjp/W+NFHH2Xu3LmsXLmSt956q8aNhjT9mUqro+EhgQzuudfUw25MZUMzkydPZsGCBcd7y+7OPffcw5tvvkmLFi346KOP2LlzJz169Ei4nDfffJPbbrsNgOHDhzN8+PDjzz3//PPMmzeP0tJSduzYwZo1ayo8X9nbb7/NX/3VXx0/M+UVV1zBW2+9xaRJk+jfvz8jR44Eaj6tMITzy+/du5dx48YBcO2113LllVcerzE/P58pU6YwZcoUAM477zzuvPNO8vPzueKKK8jJyUnmIzxhzZ5d8fw2kJ5j7WsaHmpZQpGeAAAGrklEQVTq3numfIPIlDqaknrulUyZMoXXX3+d5cuXc+jQoeMXySgoKKCkpIRly5axcuVKunfvnvA0v/HMrMq8Dz/8kEcffZTXX3+doqIiLr300lqXU9M3hLLTBUPNpxWuze9//3tuueUWli1bxtlnn01paSkzZ87kqaee4tChQ5xzzjl88MEH9Vr2iSJTjrXPlOGhTPkGkSl1lNXSVEdTJRXuZjbRzNaZ2QYzq3IpITNrY2a/jj2/xMxyU11oU+nYsSPjx4/nO9/5ToUdqfv27ePUU0+lVatWLF68mC2Jfuse5/zzzz9+Eez333+foqIiIJwuuEOHDnTu3JmdO3fy6quvHn9Np06d2L9/f8Jl/eY3v+HgwYN8/vnnvPTSS3z1q1+t83vr3LkzJ598Mm+99RYAzz77LOPGjePYsWNs27aNCRMm8Mgjj7B3714OHDjAxo0bGTZsGHfffTd5eXkK9yTk54cTlR07Fu7T0TvMlOGhTNnBnCl1NPVGptZhGTPLAuYCFwPFwFIzW+ju8ScYvwH41N1PM7NpwMPAtxqj4KYwffp0rrjiigpHzuTn53P55ZeTl5fHyJEjOfPMM2tcxs0338z111/P8OHDGTlyJGPGjAHCVZVGjRrFkCFDqpwueMaMGVxyySX07Nmzwrj76NGjue66644v48Ybb2TUqFE1DsFU55e//OXxHaoDBgzgmWee4ejRo1x99dXs27cPd+f73/8+Xbp04Yc//CGLFy8mKyuLwYMHH7+qlGS2TBkeypRvEJlSR5MPl7l7jTfgXGBR3PQPgB9UarMIODf2uCWwm9jphKu7nX322V7ZmjVrqsyTzKa/WWaaP9+9Xz93s3A/f37T19Cvn3voo1a89et3YtZhlrgOs7otByj0WnLb3ZMalukNbIubLo7NS9jG3UuBfUDXygsysxlmVmhmhSUlJcluf0SkjjJheChTTuaWKXU09XBZMuFeda8gVN7Dl0wb3H2eu+e5e152dnYy9YlIM5UpO5gzpY6m3sgkcyhkMdAnbjoH2F5Nm2Izawl0Bj6pT0EeuxycZD5P01W8pPnIlJO5ZUIdZetvqkMyk+m5LwUGmVl/M2sNTAMWVmqzELg29ngq8IbX439+27Zt2bNnj0KjGXB39uzZQ9u2bdNdikiz0ZTDZbX23N291MxuJew0zQKedvfVZjaLMLC/EPgn4Fkz20DosU+rTzE5OTkUFxej8fjmoW3btvphk0iGyqgLZIuISM2SvUC2fqEqIhJBCncRkQhSuIuIRFDaxtzNrASo+QQtma8b4de4EujzKKfPoiJ9HhU15PPo5+61/lAobeEeBWZWmMyOjROFPo9y+iwq0udRUVN8HhqWERGJIIW7iEgEKdwbZl66C8gw+jzK6bOoSJ9HRY3+eWjMXUQkgtRzFxGJIIW7iEgEKdzrwcz6mNliM1trZqvN7PZ015RuZpZlZivM7HfpriXdzKyLmb1gZh/E/o2cm+6a0snMvh/7f/K+mT1nZifMqUTN7Gkz22Vm78fNO8XM/sPM1sfuT26MdSvc66cUuMvdzwLOAW4xs8FprindbgfWpruIDPET4DV3PxMYwQn8uZhZb+A2IM/dhxLOLFuvs8Y2U/8MTKw0bybwursPAl6PTaecwr0e3H2Huy+PPd5P+M9b+dKDJwwzywEuBZ5Kdy3pZmYnAecTToONux92973prSrtWgLtYhfyaU/Vi/1Elru/SdULF00Gfhl7/EtgSmOsW+HeQGaWC4wClqS3krR6Avg/wLF0F5IBBgAlwDOxYaqnzKxDuotKF3f/CHgU2ArsAPa5+7+nt6q06+7uOyB0FIFTG2MlCvcGMLOOwIvAHe7+WbrrSQczuwzY5e7L0l1LhmgJjAaedPdRwOc00tfu5iA2njwZ6A/0AjqY2dXprerEoHCvJzNrRQj2Anf/t3TXk0bnAZPMbDOwALjAzOant6S0KgaK3b3sm9wLhLA/UV0EfOjuJe5+BPg34H+luaZ022lmPQFi97saYyUK93qwcAXvfwLWuvtj6a4nndz9B+6e4+65hB1lb7j7Cdszc/ePgW1mdkZs1oXAmjSWlG5bgXPMrH3s/82FnMA7mGPirzl9LfByY6yk1muoSkLnAdcAq8xsZWzePe7+ShprkszxPaAgdkH5TcD1aa4nbdx9iZm9ACwnHGW2ghPoVARm9hwwHuhmZsXA3wEPAc+b2Q2Ejd+VjbJunX5ARCR6NCwjIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAT9D5UptkgAjSstAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 정확도는 50% 초반에 멈추어 있습니다. 이 예제에서는 사전 훈련된 단어 임베딩을 사용하는 것이 임베딩을 함께 훈련하는 것보다 낫습니다. 훈련 샘플의 수를 늘리면 금새 상황이 바뀝니다. 연습삼아 한 번 확인해 보세요."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훈련 샘플의 수를 2000개로 늘려서 확인해 보겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_samples = 2000\n",
    "x_train = data[:training_samples]\n",
    "y_train = labels[:training_samples]\n",
    "x_val = data[training_samples: training_samples + validation_samples]\n",
    "y_val = labels[training_samples: training_samples + validation_samples]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 2000 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      "2000/2000 [==============================] - 0s 174us/step - loss: 0.6383 - acc: 0.6060 - val_loss: 0.6543 - val_acc: 0.6142\n",
      "Epoch 2/10\n",
      "2000/2000 [==============================] - 0s 174us/step - loss: 0.1578 - acc: 0.9880 - val_loss: 0.6246 - val_acc: 0.6631\n",
      "Epoch 3/10\n",
      "2000/2000 [==============================] - 0s 174us/step - loss: 0.0191 - acc: 0.9995 - val_loss: 0.6568 - val_acc: 0.6787\n",
      "Epoch 4/10\n",
      "2000/2000 [==============================] - 0s 175us/step - loss: 0.0016 - acc: 1.0000 - val_loss: 0.7071 - val_acc: 0.6916\n",
      "Epoch 5/10\n",
      "2000/2000 [==============================] - 0s 171us/step - loss: 1.2005e-04 - acc: 1.0000 - val_loss: 0.7635 - val_acc: 0.6972\n",
      "Epoch 6/10\n",
      "2000/2000 [==============================] - 0s 173us/step - loss: 8.1681e-06 - acc: 1.0000 - val_loss: 0.8398 - val_acc: 0.7043\n",
      "Epoch 7/10\n",
      "2000/2000 [==============================] - 0s 170us/step - loss: 7.4167e-07 - acc: 1.0000 - val_loss: 0.9032 - val_acc: 0.7046\n",
      "Epoch 8/10\n",
      "2000/2000 [==============================] - 0s 172us/step - loss: 1.7394e-07 - acc: 1.0000 - val_loss: 0.9659 - val_acc: 0.7041\n",
      "Epoch 9/10\n",
      "2000/2000 [==============================] - 0s 172us/step - loss: 1.1575e-07 - acc: 1.0000 - val_loss: 0.9996 - val_acc: 0.7038\n",
      "Epoch 10/10\n",
      "2000/2000 [==============================] - 0s 170us/step - loss: 1.1111e-07 - acc: 1.0000 - val_loss: 1.0031 - val_acc: 0.7046\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(x_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=32,\n",
    "                    validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3XuYFOWd9vHvzUERRQQGNYIcNCQeEBAnGBdU1EjQqCRqFNTEM8aIiW6yu8bwLi4e4iauURNfV+IhJhIJr64Gsx5WEaPGE0M4KLgIIuoI6oiIIigO+b1/VA30NHNohoaembo/19XXdFU9Vf3rGri75qnqehQRmJlZNrQpdQFmZrbtOPTNzDLEoW9mliEOfTOzDHHom5lliEPfzCxDHPoZJKmtpNWSehWzbSlJ+qKkol9/LOlrkpbmTC+UdGghbZvwWrdJuryp65sVol2pC7DGSVqdM9kR+AxYn05fEBGTN2d7EbEe2KnYbbMgIr5cjO1IOg84IyKG52z7vGJs26whDv0WICI2hG56JHleRDxeX3tJ7SKielvUZtYY/3tsXty90wpIukrSHyXdI+lj4AxJh0h6XtKHkpZLuklS+7R9O0khqU86fXe6/GFJH0t6TlLfzW2bLj9G0quSVkn6laS/SjqrnroLqfECSYslrZR0U866bSX9UtIKSa8BIxvYP+MlTcmbd7Ok69Pn50l6JX0/r6VH4fVtq1LS8PR5R0m/T2ubDxxUx+suSbc7X9IJ6fwDgF8Dh6ZdZ+/n7Nsrctb/XvreV0h6QNIXCtk3m7Ofa+qR9LikDyS9I+mfc17n/6T75CNJFZL2qKsrTdIzNb/ndH8+lb7OB8B4Sf0kzUjfy/vpfuucs37v9D1WpctvlNQhrXnfnHZfkLRGUrf63q81IiL8aEEPYCnwtbx5VwHrgONJPsh3AL4CHEzy19xewKvAuLR9OyCAPun03cD7QDnQHvgjcHcT2u4KfAyMSpf9I/A5cFY976WQGv8EdAb6AB/UvHdgHDAf6Al0A55K/jnX+Tp7AauBHXO2/R5Qnk4fn7YRcCSwFhiQLvsasDRnW5XA8PT5dcCTQBegN7Agr+0pwBfS38lpaQ27pcvOA57Mq/Nu4Ir0+Yi0xkFAB+D/Ak8Usm82cz93Bt4FfghsD+wMDEmX/QSYC/RL38MgoCvwxfx9DTxT83tO31s1cCHQluTf45eAo4Dt0n8nfwWuy3k/L6f7c8e0/dB02STg6pzX+RFwf6n/H7bkR8kL8GMzf2H1h/4Tjaz3Y+D/pc/rCvL/zGl7AvByE9qeAzyds0zAcuoJ/QJr/GrO8v8Cfpw+f4qkm6tm2bH5QZS37eeB09LnxwCvNtD2z8BF6fOGQv/N3N8F8P3ctnVs92XgG+nzxkL/LuCanGU7k5zH6dnYvtnM/fwdoKKedq/V1Js3v5DQX9JIDScDM9PnhwLvAG3raDcUeB1QOj0HOLHY/6+y9HD3TuvxVu6EpH0k/Xf65/pHwESgrIH138l5voaGT97W13aP3Doi+V9aWd9GCqyxoNcC3migXoA/AGPS56cBG05+SzpO0gtp98aHJEfZDe2rGl9oqAZJZ0mam3ZRfAjsU+B2IXl/G7YXER8BK4EeOW0K+p01sp/3BBbXU8OeJMHfFPn/HneXNFXS22kNv82rYWkkFw3UEhF/JfmrYZik/kAv4L+bWJPhPv3WJP9yxVtJjiy/GBE7A/9KcuS9NS0nORIFQJKoHVL5tqTG5SRhUaOxS0r/CHxNUk+S7qc/pDXuANwL/Iyk62UX4H8KrOOd+mqQtBdwC0kXR7d0u/+bs93GLi9dRtJlVLO9TiTdSG8XUFe+hvbzW8De9axX37JP0po65szbPa9N/vv7d5Krzg5Iazgrr4bektrWU8fvgDNI/iqZGhGf1dPOCuDQb706AauAT9ITYRdsg9f8MzBY0vGS2pH0E3ffSjVOBS6R1CM9qfcvDTWOiHdJuiDuBBZGxKJ00fYk/cxVwHpJx5H0PRdaw+WSdlHyPYZxOct2Igm+KpLPv/NIjvRrvAv0zD2hmuce4FxJAyRtT/Kh9HRE1PuXUwMa2s/TgF6SxknaTtLOkoaky24DrpK0txKDJHUl+bB7h+SCgbaSxpLzAdVADZ8AqyTtSdLFVOM5YAVwjZKT4ztIGpqz/Pck3UGnkXwA2BZw6LdePwLOJDmxeivJke5WlQbrqcD1JP+J9wZmkxzhFbvGW4DpwEvATJKj9cb8gaSP/g85NX8IXArcT3Iy9GSSD69CTCD5i2Mp8DA5gRQR84CbgBfTNvsAL+Ss+xiwCHhXUm43Tc36j5B0w9yfrt8LOL3AuvLVu58jYhVwNHASyYnjV4HD08W/AB4g2c8fkZxU7ZB2250PXE5yUv+Lee+tLhOAISQfPtOA+3JqqAaOA/YlOep/k+T3ULN8KcnveV1EPLuZ793y1JwcMSu69M/1ZcDJEfF0qeuxlkvS70hODl9R6lpaOn85y4pK0kiSP9c/Jbnkr5rkaNesSdLzI6OAA0pdS2vg7h0rtmHAEpI/+0cC3/SJN2sqST8j+a7ANRHxZqnraQ3cvWNmliE+0jczy5Bm16dfVlYWffr0KXUZZmYtyqxZs96PiIYukQaaYej36dOHioqKUpdhZtaiSGrsW+mAu3fMzDLFoW9mliEOfTOzDHHom5lliEPfzCxDGg19SXdIek/Sy/UsVzos2mJJ8yQNzll2pqRF6ePMYhZu9Zs8Gfr0gTZtkp+TN2vYdNfhOrJRQ2braGyUFeAwYDDp6Eh1LD+W5A6DAr4KvJDO70rydfyuJPcBXwJ0aez1DjrooLCmu/vuiI4dI2Djo2PHZL7rcB3NoY7mUENrrIN6RkDLfxQ0vBbJGJz1hf6twJic6YUkIwqNAW6tr119D4f+lundu/Y/nppH796uw3U0jzqaQw2tsY5CQ78Yffo9qD00WmU6r775m5A0VlKFpIqqqqoilJRdb9ZzS6r65rsO17Gt62gONWS5jmKEfl3DykUD8zedGTEpIsojorx790a/RWwN6FXPoIH1zXcdrmNb19EcashyHcUI/UpqjxPak2TgjPrm21Z09dXQsWPteR07JvNdh+toDnU0hxoyXUchfUA03Kf/DWqfyH0xnd8VeJ3kJG6X9HnXxl6rJffp33130g8nJT+39Qkh1+E6WkodzaGG1lYHBfbpN3o/fUn3AMOBMpLBnCcA7dMPjP+UJODXJANmrAHOjoiKdN1zSMbRBLg6Iu5s7EOovLw8WuIN1yZPhrFjYc2ajfM6doRJk+D0po5samZWIEmzIqK80XaNhf621lJDv08feKOOe9z17g1Ll27raswsawoNfX8jt0iay5UAZmYNcegXSXO5EsDMrCEO/SJpLlcCmJk1xKFfJKefnpy07d0bpOSnT+KaWXPT7IZLbMlOP90hb2bNm4/0zcwyxKFvZpYhDn0zswxx6JuZZYhD38wsQxz6ZmYZ4tA3M8sQh76ZWYY49M3MMsShb2aWIQ59M7MMKSj0JY2UtFDSYkmX1bG8t6TpkuZJelJSz5xl6yXNSR/Tilm8mZltnkZvuCapLXAzcDTJYOczJU2LiAU5za4DfhcRd0k6EvgZ8J102dqIGFTkus3MrAkKOdIfAiyOiCURsQ6YAozKa7MfMD19PqOO5WZm1gwUEvo9gLdypivTebnmAielz78FdJLULZ3uIKlC0vOSvlnXC0gam7apqKqq2ozyzcxscxQS+qpjXv5o6j8GDpc0GzgceBuoTpf1SgfrPQ24QdLem2wsYlJElEdEeffu3Quv3szMNkshg6hUAnvmTPcEluU2iIhlwIkAknYCToqIVTnLiIglkp4EDgRe2+LKzcxssxVypD8T6Cepr6TtgNFAratwJJVJqtnWT4A70vldJG1f0wYYCuSeADYzs22o0dCPiGpgHPAo8AowNSLmS5oo6YS02XBgoaRXgd2AmuHA9wUqJM0lOcF7bd5VP2Zmtg0pIr97vrTKy8ujoqKi1GWYmbUokmal508b5G/kmplliEPfzCxDHPpmZhni0DczyxCHvplZhjj0zcwyxKFvZpYhDn0zswxx6JuZZYhD38wsQxz6ZmYZ4tA3M8sQh76ZWYY49M3MMsShb2aWIQ59M7MMKSj0JY2UtFDSYkmX1bG8t6TpkuZJelJSz5xlZ0palD7OLGbxZma2eRoNfUltgZuBY4D9gDGS9strdh3wu4gYAEwEfpau2xWYABwMDAEmSOpSvPLNzGxzFHKkPwRYHBFLImIdMAUYlddmP2B6+nxGzvKvA49FxAcRsRJ4DBi55WWbmVlTFBL6PYC3cqYr03m55gInpc+/BXSS1K3AdZE0VlKFpIqqqqpCazczs81USOirjnn5o6n/GDhc0mzgcOBtoLrAdYmISRFRHhHl3bt3L6AkMzNrinYFtKkE9syZ7gksy20QEcuAEwEk7QScFBGrJFUCw/PWfXIL6jUzsy1QyJH+TKCfpL6StgNGA9NyG0gqk1SzrZ8Ad6TPHwVGSOqSnsAdkc4zM7MSaDT0I6IaGEcS1q8AUyNivqSJkk5Imw0HFkp6FdgNuDpd9wPgSpIPjpnAxHSemZmVgCI26WIvqfLy8qioqCh1GWZmLYqkWRFR3lg7fyPXzCxDHPpmZhni0DczyxCHvplZhjj0zcwyxKFvZpYhDn0zswxx6JuZZYhD38wsQxz6ZmYZ4tA3M8sQh76ZWYY49M3MMsShb2aWIQ59M7MMceibmWVIQaEvaaSkhZIWS7qsjuW9JM2QNFvSPEnHpvP7SForaU76+M9ivwEzMytcowOjS2oL3AwcTTJI+kxJ0yJiQU6z8STDKN4iaT/gIaBPuuy1iBhU3LLNzKwpCjnSHwIsjoglEbEOmAKMymsTwM7p887AsuKVaGZmxVJI6PcA3sqZrkzn5boCOENSJclR/sU5y/qm3T5/kXRoXS8gaaykCkkVVVVVhVdvZmabpZDQVx3z8kdTHwP8NiJ6AscCv5fUBlgO9IqIA4F/BP4gaee8dYmISRFRHhHl3bt337x3YGZmBSsk9CuBPXOme7Jp9825wFSAiHgO6ACURcRnEbEinT8LeA340pYWbWZmTVNI6M8E+knqK2k7YDQwLa/Nm8BRAJL2JQn9Kknd0xPBSNoL6AcsKVbxZma2eRq9eiciqiWNAx4F2gJ3RMR8SROBioiYBvwI+I2kS0m6fs6KiJB0GDBRUjWwHvheRHyw1d6NmZk1SBH53fOlVV5eHhUVFaUuw8ysRZE0KyLKG2vnb+SamWWIQ9/MLEMc+mZmGeLQNzPLEIe+mVmGOPTNzDLEoW9mliEOfTOzDHHom5lliEPfzCxDHPpmZhni0DczyxCHvplZhjj0zcwyxKFvZpYhDn0zswwpKPQljZS0UNJiSZfVsbyXpBmSZkuaJ+nYnGU/SddbKOnrxSzezMw2T6PDJaZj3N4MHE0ySPpMSdMiYkFOs/HA1Ii4RdJ+wENAn/T5aGB/YA/gcUlfioj1xX4jZmbWuEKO9IcAiyNiSUSsA6YAo/LaBLBz+rwzsCx9PgqYEhGfRcTrwOJ0e2ZmVgKFhH4P4K2c6cp0Xq4rgDMkVZIc5V+8Geuamdk2Ukjoq455+aOpjwF+GxE9gWOB30tqU+C6SBorqUJSRVVVVQElmZlZUxQS+pXAnjnTPdnYfVPjXGAqQEQ8B3QAygpcl4iYFBHlEVHevXv3wqs3M7PNUkjozwT6SeoraTuSE7PT8tq8CRwFIGlfktCvStuNlrS9pL5AP+DFYhVvZmabp9GrdyKiWtI44FGgLXBHRMyXNBGoiIhpwI+A30i6lKT75qyICGC+pKnAAqAauMhX7piZlY6SbG4+ysvLo6KiotRlmJm1KJJmRUR5Y+38jVwzswxx6JuZZYhD38wsQxz6ZmYZ4tA3M8sQh76ZWYY49M3MMsShb2aWIQ59M7MMceibmWWIQ9/MLEMc+mZmGeLQNzPLEIe+mVmGOPTNzDLEoW9mliEOfTOzDCko9CWNlLRQ0mJJl9Wx/JeS5qSPVyV9mLNsfc6y/LF1zcxsG2p0jFxJbYGbgaOBSmCmpGkRsaCmTURcmtP+YuDAnE2sjYhBxSvZzMyaqpAj/SHA4ohYEhHrgCnAqAbajwHuKUZxZmZWXIWEfg/grZzpynTeJiT1BvoCT+TM7iCpQtLzkr5Zz3pj0zYVVVVVBZZuZmabq5DQVx3zop62o4F7I2J9zrxe6QjtpwE3SNp7k41FTIqI8ogo7969ewElmZlZUxQS+pXAnjnTPYFl9bQdTV7XTkQsS38uAZ6kdn+/mZltQ4WE/kygn6S+krYjCfZNrsKR9GWgC/BczrwukrZPn5cBQ4EF+euamdm20ejVOxFRLWkc8CjQFrgjIuZLmghURETNB8AYYEpE5Hb97AvcKunvJB8w1+Ze9WNmZtuWamd06ZWXl0dFRUWpyzAza1EkzUrPnzbI38g1M8sQh76ZWYY49M3MMsShb2aWIQ59M7MMceibmWWIQ9/MLEMc+mZmGeLQNzPLEIe+mVmGOPTNzDLEoW9mliEOfTOzDHHom5lliEPfzCxDGh1ExcyK49NPYfny5GfEpo+//73u+YUu35xtAEjQpk3dj4aWFbONtPGRP53/qFleChGwfn3Dj+rqxts09ujcGQ47bOu+l4JCX9JI4EaSkbNui4hr85b/EjginewI7BoRu6TLzgTGp8uuioi7ilG4WXMRAStWwNtvN/xYsaLUlbYeDX0oNPahUdcDGg7jbTXW1MEHw/PPb93XaDT0JbUFbgaOJhkkfaakabnDHkbEpTntLyYd/FxSV2ACUA4EMCtdd2VR34XZVvLpp7BsWcNhvmwZrFtXez0Jdt0VevSA3r3hH/4heb7HHtCxY/FDqynLa47863s0trzQNo21K+ZfO01dDtC27ZY92rXb8m106rT1/00XcqQ/BFgcEUsAJE0BRlH/AOdjSIIe4OvAYxHxQbruY8BI4J4tKdpsS23J0XnHjhsDvCbM8x9f+AK0b7/t35dZYwoJ/R7AWznTlcDBdTWU1BvoCzzRwLo96lhvLDAWoFevXgWUZLapCPjoI3jvPaiq2viomV6+vPbR+Wef1V6/vqPz/EfnzqXrWzbbUoWEfl3/vOvr4RoN3BsR6zdn3YiYBEyCZGD0AmqyDIiAVatqB3f+8/zpzz+ve1s77QS7756E9iGH+OjcsquQ0K8E9syZ7gksq6ftaOCivHWH5637ZOHlWWsSAR9+2Hhw10y//379Id6pE3Tvnjx69YKDDkqe77rrxvm50x06bNv3atZcFRL6M4F+kvoCb5ME+2n5jSR9GegCPJcz+1HgGkld0ukRwE+2qGJr1iKSrpPZs5PHnDmwePHGMK+urnu9nXeuHeLl5ZsGd+7DIW7WNI2GfkRUSxpHEuBtgTsiYr6kiUBFRExLm44BpkRsvLgpIj6QdCXJBwfAxJqTutby/f3vSaDXBHzNo6pqY5t+/WCffeArX6n/KLx7d9h++9K9D7MsUU5GNwvl5eVRUVFR6jIsz7p1MH9+7XCfOxdWr06Wt28P++8PBx648TFw4La5BM3MQNKsiChvrJ2/kWub+PjjJNBzA37+/I396zvuCIMGwVlnbQz4/fbz0bpZS+DQz7j33tu0e2bx4o1fWOnePQn1r399Y8B/8YvJF33MrOVx6GdEBCxdumnAL8u5DqtPnyTUv/OdjQG/xx6+Jt2sNXHot0IRydH688/Xvormww+T5W3awL77wpFHbgz3QYOgS5eGt2tmLZ9DvxVYtw7+9jd45hn461+TR80VNB06wIABcOqpGwP+gANghx1KW7OZlYZDvwX64AN47rmNIT9zZnJjMIC994ZjjoGhQ5PbCOyzT3IjKDMzcOg3exGwZEkS7jUhvyC91V27djB4MFx4YRLyQ4cmtxowM6uPQ7+Z+fzzpA8+N+TffTdZ1rlzcvR+2mlJwA8Zktzx0cysUA79Evvww6SrpibkX3wR1q5NlvXtC0cfvfEofv/9famkmW0Zh/42VHPZZM3J1meeSb70FJEMoHDggTB27MaQ32OPUldsZq2NQ38rqq5OLpXMDfnly5NlnTolXTWnnLKxq2annUpbr5m1fg79Ilu5Em66CZ56Cl54AT75JJnfqxccccTGo/j+/ZOjezOzbcmhX0TPPJOcZH377eRmY+ecszHke/YsdXVmZg79oqiuhquvhokTk5Ovzz+f3ErYrKX5/PPPqays5NOaL35Ys9OhQwd69uxJ+yYO8+bQ30JvvglnnAFPP53cs+bmm307YWu5Kisr6dSpE3369EG+6VKzExGsWLGCyspK+vbt26Rt+ALALXDffUk3zuzZ8Pvfw+9+58C3lu3TTz+lW7duDvxmShLdunXbor/ECgp9SSMlLZS0WNJl9bQ5RdICSfMl/SFn/npJc9LHtLrWbWnWrIHvfQ9OPjm5zfDs2cnRvllr4MBv3rb099No946ktsDNwNEkA53PlDQtIhbktOlHMvbt0IhYKWnXnE2sjYhBW1RlMzJvHowZk9wK4Z//Ga68ErbbrtRVmZkVppAj/SHA4ohYEhHrgCnAqLw25wM3R8RKgIh4r7hlll5E0l8/ZAisWAGPPgr//u8OfMu2yZOTcRjatEl+Tp68ZdtbsWIFgwYNYtCgQey+++706NFjw/S6desK2sbZZ5/NwoULG2xz8803M3lLi22hCjmR2wN4K2e6Ejg4r82XACT9lWTw9Csi4pF0WQdJFUA1cG1EPJD/ApLGAmMBevXqtVlvYFtYsSK5/HLatOQOlr/9bTKot1mWTZ6cfIN8zZpk+o03kmmA009v2ja7devGnDlzALjiiivYaaed+PGPf1yrTUQQEbSp554kd955Z6Ovc9FFFzWtwFagkCP9ujqQ8kdTbwf0A4YDY4DbJO2SLuuVDtZ7GnCDpL032VjEpIgoj4jy7t27F1z8tvDkk8nJ2ocfhl/+Ev78Zwe+GcBPf7ox8GusWZPML7bFixfTv39/vve97zF48GCWL1/O2LFjKS8vZ//992fixIkb2g4bNow5c+ZQXV3NLrvswmWXXcbAgQM55JBDeO+9pBNi/Pjx3HDDDRvaX3bZZQwZMoQvf/nLPPvsswB88sknnHTSSQwcOJAxY8ZQXl6+4QMp14QJE/jKV76yob5Ixxp99dVXOfLIIxk4cCCDBw9m6dKlAFxzzTUccMABDBw4kJ9ujZ3ViEJCvxLYM2e6J7CsjjZ/iojPI+J1YCHJhwARsSz9uQR4EjhwC2veJj7/HMaPT0aX2nHH5Nu1l1ziG56Z1Xjzzc2bv6UWLFjAueeey+zZs+nRowfXXnstFRUVzJ07l8cee4wFCxZsss6qVas4/PDDmTt3Locccgh33HFHnduOCF588UV+8YtfbPgA+dWvfsXuu+/O3Llzueyyy5g9e3ad6/7whz9k5syZvPTSS6xatYpHHkk6OcaMGcOll17K3LlzefbZZ9l111158MEHefjhh3nxxReZO3cuP/rRj4q0dwpXSITNBPpJ6itpO2A0kH8VzgPAEQCSyki6e5ZI6iJp+5z5Q4FNfzPNzNKlcPjhyReuzj4bZs1KboZmZhvV1xO7tXpo9957b76S863He+65h8GDBzN48GBeeeWVOkN/hx124JhjjgHgoIMO2nC0ne/EE0/cpM0zzzzD6NGjARg4cCD7779/netOnz6dIUOGMHDgQP7yl78wf/58Vq5cyfvvv8/xxx8PJF+o6tixI48//jjnnHMOO6RD13Xt2nXzd8QWajT0I6IaGAc8CrwCTI2I+ZImSjohbfYosELSAmAG8E8RsQLYF6iQNDedf23uVT/N0R//mHTnzJ8PU6bA7bf7Rmhmdbn66k3Hc+jYMZm/Ney4444bni9atIgbb7yRJ554gnnz5jFy5Mg6r13fLudKi7Zt21JdXV3ntrfffvtN2tR00zRkzZo1jBs3jvvvv5958+ZxzjnnbKijrksrI6Lkl8QW1FkREQ9FxJciYu+IuDqd968RMS19HhHxjxGxX0QcEBFT0vnPptMD05+3b723smU++QTOPRdGj4b99kvujnnqqaWuyqz5Ov10mDQJevcGKfk5aVLTT+Jujo8++ohOnTqx8847s3z5ch599NGiv8awYcOYOnUqAC+99FKdf0msXbuWNm3aUFZWxscff8x9990HQJcuXSgrK+PBBx8Eki+9rVmzhhEjRnD77bezNh0044MPPih63Y3xbRhIvlw1Zgy8+mpyEmrCBGjibS3MMuX007dNyOcbPHgw++23H/3792evvfZi6NChRX+Niy++mO9+97sMGDCAwYMH079/fzp37lyrTbdu3TjzzDPp378/vXv35uCDN17YOHnyZC644AJ++tOfst1223Hfffdx3HHHMXfuXMrLy2nfvj3HH388V155ZdFrb4gK+RNmWyovL4+Kiopt8loRcOON8C//AmVlcPfdye2PzbLqlVdeYd999y11Gc1CdXU11dXVdOjQgUWLFjFixAgWLVpEu3alP1au6/ckaVZ6pWSDSl99ibz3XnKS9qGH4IQTkr77srJSV2VmzcXq1as56qijqK6uJiK49dZbm0Xgb6mW/w6a4LHH4LvfTQY8+fWv4fvfT/okzcxq7LLLLsyaNavUZRRdpq46X7cu6coZMQK6dk0GIb/oIge+mWVHZo70X3stOVk7cyZccAFcf/2ml5uZmbV2mQj9yZOTWyG3a5fcAz/9HoaZWea06u6djz9O+u7POAMGDYK5cx34ZpZtrTb0Kypg8ODkKP+KK2DGjK339XAzK47hw4dv8kWrG264ge9///sNrrdT+rX5ZcuWcfLJJ9e77cYuB7/hhhtYk3MXuWOPPZYPP/ywkNJbjFYX+n//O/ziF3DIIfDZZ8ldMidMSLp2zKx5GzNmDFOmTKk1b8qUKYwZM6ag9ffYYw/uvffeJr9+fug/9NBD7LLLLg2s0fK0qih8552kO+exx+Ckk+A3v4EuXUpdlVnLdMklye1IimnQIEjvaFynk08+mfHjx/PZZ5+x/fbbs3TpUpYtW8awYcNYvXo1o0aNYuXKlXz++edcddVVjBpVezynpUuXctxxx/ERZnAdAAAI80lEQVTyyy+zdu1azj77bBYsWMC+++674dYHABdeeCEzZ85k7dq1nHzyyfzbv/0bN910E8uWLeOII46grKyMGTNm0KdPHyoqKigrK+P666/fcJfO8847j0suuYSlS5dyzDHHMGzYMJ599ll69OjBn/70pw03VKvx4IMPctVVV7Fu3Tq6devG5MmT2W233Vi9ejUXX3wxFRUVSGLChAmcdNJJPPLII1x++eWsX7+esrIypk+fXrTfQasJ/UWLYOhQWL0abr0Vzj/fl2KatTTdunVjyJAhPPLII4waNYopU6Zw6qmnIokOHTpw//33s/POO/P+++/z1a9+lRNOOKHeG5jdcsstdOzYkXnz5jFv3jwGDx68YdnVV19N165dWb9+PUcddRTz5s3jBz/4Addffz0zZsygLO+bmrNmzeLOO+/khRdeICI4+OCDOfzww+nSpQuLFi3innvu4Te/+Q2nnHIK9913H2fkDZo9bNgwnn/+eSRx22238fOf/5z/+I//4Morr6Rz58689NJLAKxcuZKqqirOP/98nnrqKfr27Vv0+/O0mtDfa6/k6P7ii5MbppnZlmnoiHxrquniqQn9mqPriODyyy/nqaeeok2bNrz99tu8++677L777nVu56mnnuIHP/gBAAMGDGDAgAEblk2dOpVJkyZRXV3N8uXLWbBgQa3l+Z555hm+9a1vbbjT54knnsjTTz/NCSecQN++fRk0KBkGvL7bN1dWVnLqqaeyfPly1q1bR9++fQF4/PHHa3VndenShQcffJDDDjtsQ5ti33651fTpT5mSjG7Vv39xxuo0s9L45je/yfTp0/nb3/7G2rVrNxyhT548maqqKmbNmsWcOXPYbbfd6rydcq66/gp4/fXXue6665g+fTrz5s3jG9/4RqPbaegeZTW3ZYb6b9988cUXM27cOF566SVuvfXWDa9X162Wt/btl1tF6NeM1fnGG8lN1GrG6nTwm7U8O+20E8OHD+ecc86pdQJ31apV7LrrrrRv354ZM2bwxhtvNLidww47bMPg5y+//DLz5s0Dktsy77jjjnTu3Jl3332Xhx9+eMM6nTp14uOPP65zWw888ABr1qzhk08+4f777+fQQw8t+D2tWrWKHj16AHDXXXdtmD9ixAh+/etfb5heuXIlhxxyCH/5y194/fXXgeLffrlVhP62HKvTzLa+MWPGMHfu3A0jVwGcfvrpVFRUUF5ezuTJk9lnn30a3MaFF17I6tWrGTBgAD//+c8ZMmQIkIyCdeCBB7L//vtzzjnn1Lot89ixYznmmGM4Iu92u4MHD+ass85iyJAhHHzwwZx33nkcuBnD6V1xxRV8+9vf5tBDD611vmD8+PGsXLmS/v37M3DgQGbMmEH37t2ZNGkSJ554IgMHDuTUIg/sUdCtlSWNBG4E2gK3RcS1dbQ5BbiCZND0uRFxWjr/TGB82uyqiLgrf91cTbm1cps2yRH+pjUll3CaWWF8a+WWYaveWllSW+Bm4GiSAdBnSpqWO+yhpH7AT4ChEbFS0q7p/K7ABKCc5MNgVrruyoLfXQF69Uq6dOqab2ZmGxXSvTMEWBwRSyJiHTAFGJXX5nzg5powj4j30vlfBx6LiA/SZY8BI4tT+kbbeqxOM7OWqpDQ7wG8lTNdmc7L9SXgS5L+Kun5tDuo0HWRNFZShaSKqqqqwqtPlXKsTrPWprmNpme1benvp5Dr9Ou6dij/VdsB/YDhQE/gaUn9C1yXiJgETIKkT7+AmjZRqrE6zVqTDh06sGLFCrp167ZVLxu0pokIVqxYQYcOHZq8jUJCvxLYM2e6J7CsjjbPR8TnwOuSFpJ8CFSSfBDkrvtkU4s1s62rZ8+eVFZW0pS/uG3b6NChAz179mzy+oWE/kygn6S+wNvAaOC0vDYPAGOA30oqI+nuWQK8BlwjqeYOOCNITviaWTPUvn37Dd8Etdap0dCPiGpJ44BHSS7ZvCMi5kuaCFRExLR02QhJC4D1wD9FxAoASVeSfHAATIyI4n7TwMzMClbQdfrbUlOu0zczy7pCr9NvFd/INTOzwjS7I31JVUDDN9Vo/sqA90tdRDPi/VGb98dG3he1bcn+6B0R3Rtr1OxCvzWQVFHIn1lZ4f1Rm/fHRt4XtW2L/eHuHTOzDHHom5lliEN/65hU6gKaGe+P2rw/NvK+qG2r7w/36ZuZZYiP9M3MMsShb2aWIQ79IpK0p6QZkl6RNF/SD0tdU6lJaitptqQ/l7qWUpO0i6R7Jf1v+m/kkFLXVEqSLk3/n7ws6R5JTb91ZAsk6Q5J70l6OWdeV0mPSVqU/uzS0DaawqFfXNXAjyJiX+CrwEWS9itxTaX2Q+CVUhfRTNwIPBIR+wADyfB+kdQD+AFQHhH9Se7rNbrhtVqd37LpoFKXAdMjoh8wPZ0uKod+EUXE8oj4W/r8Y5L/1JsMGpMVknoC3wBuK3UtpSZpZ+Aw4HaAiFgXER+WtqqSawfsIKkd0JFNb9neqkXEU0D+DShHATXjiN8FfLPYr+vQ30ok9QEOBF4obSUldQPwz4CHp4e9gCrgzrS76zZJO5a6qFKJiLeB64A3geXAqoj4n9JW1SzsFhHLITmIBHYt9gs49LcCSTsB9wGXRMRHpa6nFCQdB7wXEbNKXUsz0Q4YDNwSEQcCn7AV/nRvKdK+6lFAX2APYEdJZ5S2qmxw6BeZpPYkgT85Iv6r1PWU0FDgBElLgSnAkZLuLm1JJVUJVEZEzV9+95J8CGTV14DXI6IqHXHvv4B/KHFNzcG7kr4AkP58r9gv4NAvIiWDit4OvBIR15e6nlKKiJ9ERM+I6ENygu6JiMjskVxEvAO8JenL6ayjgAUlLKnU3gS+Kqlj+v/mKDJ8YjvHNODM9PmZwJ+K/QKFDJdohRsKfAd4SdKcdN7lEfFQCWuy5uNiYLKk7UiGEz27xPWUTES8IOle4G8kV73NJmO3ZJB0D8kY4mWSKoEJwLXAVEnnknwwfrvor+vbMJiZZYe7d8zMMsShb2aWIQ59M7MMceibmWWIQ9/MLEMc+mZmGeLQNzPLkP8P5mhwgtV8IKsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl8VOXZ//HPZQAji4CAGwgBtSKbgFGooIBQi1JBLQoIRX1Uqta6VR+pWutSWvWxaF1+VtRSlVTkgSo8AtJFFG0rshSRRQqyRhABJYKAErh+f9yTZBKyTELCTE6+79drXpk5c88510ySb+7c55z7mLsjIiLRcliyCxARkcqncBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuEuxzCzNzHaaWcvKbJtMZnaSmVX6sb9m1s/M1sY9XmFmZyfStgLbet7M7qro60tZ76/M7I+VvV5JnlrJLkAqh5ntjHtYF/gG2Bd7/GN3zyrP+tx9H1C/stvWBO5+SmWsx8yuAUa4e++4dV9TGeuW6FO4R4S754drrGd4jbv/raT2ZlbL3XMPRW0icuhpWKaGiP3b/aqZvWJmO4ARZvZdM3vfzLab2SYze8LMasfa1zIzN7OM2OMJsednmtkOM/uXmbUub9vY8+eb2X/MLMfMnjSzf5jZlSXUnUiNPzazVWb2pZk9EffaNDN7zMy2mdknQP9SPp97zGxikWVPm9nY2P1rzGx57P18EutVl7SubDPrHbtf18xejtW2FDi9mO2ujq13qZkNjC3vCDwFnB0b8toa99neF/f662LvfZuZvW5mxyXy2ZTFzC6K1bPdzN4ys1PinrvLzDaa2Vdm9nHce+1uZgtjyzeb2f8kuj2pAu6uW8RuwFqgX5FlvwK+BS4k/FE/AjgD6Eb4D64N8B/gxlj7WoADGbHHE4CtQCZQG3gVmFCBtkcDO4BBseduA/YCV5bwXhKpcSrQEMgAvsh778CNwFKgBdAEmBN+5IvdThtgJ1Avbt2fA5mxxxfG2hhwLrAb6BR7rh+wNm5d2UDv2P1HgbeBxkArYFmRtpcBx8W+J5fHajgm9tw1wNtF6pwA3Be7f16sxs5AOvD/gLcS+WyKef+/Av4Yu39qrI5zY9+ju2Kfe22gPbAOODbWtjXQJnZ/HjAsdr8B0C3Zvws1+aaee83ynrv/n7vvd/fd7j7P3ee6e667rwbGAb1Kef1kd5/v7nuBLEKolLftD4BF7j419txjhD8ExUqwxt+4e467ryUEad62LgMec/dsd98GPFTKdlYDSwh/dAC+B2x39/mx5//P3Vd78Bbwd6DYnaZFXAb8yt2/dPd1hN54/HYnufum2PfkT4Q/zJkJrBdgOPC8uy9y9z3AaKCXmbWIa1PSZ1OaocA0d38r9j16CDiS8Ec2l/CHpH1saG9N7LOD8Ef6ZDNr4u473H1ugu9DqoDCvWbZEP/AzNqa2XQz+8zMvgIeAJqW8vrP4u7vovSdqCW1PT6+Dnd3Qk+3WAnWmNC2CD3O0vwJGBa7fznhj1JeHT8ws7lm9oWZbSf0mkv7rPIcV1oNZnalmX0YG/7YDrRNcL0Q3l/++tz9K+BLoHlcm/J8z0pa737C96i5u68Afkb4PnweG+Y7Ntb0KqAdsMLMPjCzCxJ8H1IFFO41S9HDAJ8l9FZPcvcjgXsJww5VaRNhmAQAMzMKh1FRB1PjJuCEuMdlHar5KtAv1vMdRAh7zOwIYDLwG8KQSSPgLwnW8VlJNZhZG+AZ4HqgSWy9H8ett6zDNjcShnry1teAMPzzaQJ1lWe9hxG+Z58CuPsEd+9BGJJJI3wuuPsKdx9KGHr7LTDFzNIPshapIIV7zdYAyAG+NrNTgR8fgm2+AXQ1swvNrBZwM9CsimqcBNxiZs3NrAlwZ2mN3X0z8B4wHljh7itjTx0O1AG2APvM7AdA33LUcJeZNbJwHsCNcc/VJwT4FsLfuWsIPfc8m4EWeTuQi/EKcLWZdTKzwwkh+667l/ifUDlqHmhmvWPbvoOwn2SumZ1qZn1i29sdu+0jvIEfmVnTWE8/J/be9h9kLVJBCvea7WfAFYRf3GcJPdcqFQvQIcBYYBtwIvBvwnH5lV3jM4Sx8Y8IO/smJ/CaPxF2kP4prubtwK3Aa4SdkoMJf6QS8UvCfxBrgZnAS3HrXQw8AXwQa9MWiB+n/iuwEthsZvHDK3mvf5MwPPJa7PUtCePwB8XdlxI+82cIf3j6AwNj4++HA48Q9pN8RvhP4Z7YSy8Alls4GutRYIi7f3uw9UjFWBjyFEkOM0sjDAMMdvd3k12PSFSo5y6HnJn1N7OGsX/tf0E4AuODJJclEikKd0mGnsBqwr/2/YGL3L2kYRkRqQANy4iIRJB67iIiEZS0icOaNm3qGRkZydq8iEi1tGDBgq3uXtrhw0ASwz0jI4P58+cna/MiItWSmZV1pjWgYRkRkUhSuIuIRJDCXUQkglLqSkx79+4lOzubPXv2JLsUSUB6ejotWrSgdu2Spj4RkWRJqXDPzs6mQYMGZGRkECYLlFTl7mzbto3s7Gxat25d9gtE5JBKqWGZPXv20KRJEwV7NWBmNGnSRP9liaSoMsPdzP5gZp+b2ZISnjcL17VcZWaLzazrwRSkYK8+9L0SSV2JDMv8kXBpsJdKeP584OTYrRthmtBulVGciMihtH8/7N1bcPv228KPS7uVp+0PfgBnnFG176XMcHf3ORa7qn0JBgEvxS6X9n7sogTHufumSqrxkNm2bRt9+4ZrMHz22WekpaXRrFk4EeyDDz6gTp06Za7jqquuYvTo0Zxyyikltnn66adp1KgRw4cf9NTb9OzZk6eeeorOnRO5NKZIzfHtt7B0KSxcCAsWwL//DV98UXoQ7z9ElxY5/vgUCPcENKfwNSKzY8sOCHczGwWMAmjZsqwrnpUtKwvuvhvWr4eWLWHMGDiYvGzSpAmLFi0C4L777qN+/frcfvvthdrkX1n8sOJHtMaPH1/mdn7yk59UvEgROcDu3fDRRyHI88L8o49CYAMceSR07gxdukDt2gW3OnUKPy56K+35ir42LQ0OxYhmZYR7cWUWO9Wku48jXL2ezMzMg5qOMisLRo2CXbvC43XrwmM4uIAvzqpVq7jooovo2bMnc+fO5Y033uD+++9n4cKF7N69myFDhnDvvfcCBT3pDh060LRpU6677jpmzpxJ3bp1mTp1KkcffTT33HMPTZs25ZZbbqFnz5707NmTt956i5ycHMaPH89ZZ53F119/zciRI1m1ahXt2rVj5cqVPP/886X20CdMmMDDDz+MuzNw4EB+/etfk5uby1VXXcWiRYtwd0aNGsVNN93EY489xnPPPUft2rXp2LEjEyZMqNwPTaSKfP01LFpUEOQLF4Ye+r594fmjjoKuXeHWW+H008P9Nm2ghP5YZFVGuGdT+ALALQhX1qlSd99dEOx5du0Kyys73AGWLVvG+PHj+f3vfw/AQw89xFFHHUVubi59+vRh8ODBtGvXrtBrcnJy6NWrFw899BC33XYbf/jDHxg9evQB63Z3PvjgA6ZNm8YDDzzAm2++yZNPPsmxxx7LlClT+PDDD+natfT91NnZ2dxzzz3Mnz+fhg0b0q9fP9544w2aNWvG1q1b+eijjwDYvn07AI888gjr1q2jTp06+ctEUk1OTkGQL1gQvn78MeTNVH700SHAL7wwhPjpp4f/4rWvv3LCfRpwo5lNJOxIzTkU4+3r15dv+cE68cQTOSNukOyVV17hhRdeIDc3l40bN7Js2bIDwv2II47g/PPPB+D000/n3XeLv4rcJZdckt9m7dq1ALz33nvceWe4nvNpp51G+/btS61v7ty5nHvuuTRt2hSAyy+/nDlz5nDnnXeyYsUKbr75Zi644ALOO+88ANq3b8+IESMYNGgQF110UTk/DZHK98UXhXvjCxbAqlUFzzdvHgJ8yJDwtWvXMHatIC9emeFuZq8AvYGmZpZNuOBvbQB3/z0wg3Bh3FXALuCqqio2XsuWYSimuOVVoV69evn3V65cye9+9zs++OADGjVqxIgRI4o93jt+B2xaWhq5ubnFrvvwww8/oE15L6JSUvsmTZqwePFiZs6cyRNPPMGUKVMYN24cs2bN4p133mHq1Kn86le/YsmSJaSlpZVrmyIV9fnnhXvjCxdCrF8DQEZGCO8rrwy98S5d4JhjklRsNZXI0TLDynjegUO+h3DMmMJj7gB164blVe2rr76iQYMGHHnkkWzatIlZs2bRv3//St1Gz549mTRpEmeffTYfffQRy5YtK7V99+7dueOOO9i2bRsNGzZk4sSJ3H777WzZsoX09HQuvfRSWrduzXXXXce+ffvIzs7m3HPPpWfPnmRlZbFr1y4aNGhQqe9BBGDbNvjnPwuH+aefFjx/0knQrRtcf30I9C5doEmT5NUbFSk1/UB55I2rV+bRMonq2rUr7dq1o0OHDrRp04YePXpU+jZ++tOfMnLkSDp16kTXrl3p0KEDDRs2LLF9ixYteOCBB+jduzfuzoUXXsiAAQNYuHAhV199Ne6OmfHwww+Tm5vL5Zdfzo4dO9i/fz933nmngl0q1TffwPTp8NJLMGNGOGrFDNq2hd69C3Z0du4MpfxYy0FI2jVUMzMzvejFOpYvX86pp56alHpSTW5uLrm5uaSnp7Ny5UrOO+88Vq5cSa1aqfX3WN8zyeMO778fAv3VV+HLL+HYY0OH66KLQpDXr5/sKqs/M1vg7plltUutpJB8O3fupG/fvuTm5uLuPPvssykX7CIAq1fDhAnw8sthB+gRR8DFF8PIkdC3L+jHNjn0saeoRo0asWDBgmSXIVKs7dth0qQQ6O+9F4Zc+vQJw6Q//CFolC/5FO4ikpC9e+HNN0OgT5sWxtXbtoVf/zoMvVTVkWpSMQp3ESmRezjC5aWXYOJE2LIFmjaFH/8YfvSjsGNUx5mnJoW7iBxgw4aCcfTly+Hww2HgwDCO/v3vhzlSJLUp3EUEgB07YMqUEOizZ4dee8+eMG4cXHopNGqU7AqlPGrYVDql6927N7NmzSq07PHHH+eGG24o9XX1Y8d3bdy4kcGDB5e47qKHfhb1+OOPsyvurKwLLrigUuZ9ue+++3j00UcPej0SPbm5MGtWGDM/5hi46qpw3sh998Enn8C778K11yrYqyOFe5xhw4YxceLEQssmTpzIsGGlnqSb7/jjj2fy5MkV3n7RcJ8xYwaN9FslVWDxYrj9djjhBOjfH2bOhCuuCGeS/uc/cO+9YSZFqb4U7nEGDx7MG2+8wTfffAPA2rVr2bhxIz179sw/7rxr16507NiRqVOnHvD6tWvX0qFDBwB2797N0KFD6dSpE0OGDGH37t357a6//noyMzNp3749v/zlLwF44okn2LhxI3369KFPnz4AZGRksHXrVgDGjh1Lhw4d6NChA48//nj+9k499VSuvfZa2rdvz3nnnVdoO8VZtGgR3bt3p1OnTlx88cV8+eWX+dtv164dnTp1YujQoQC88847dO7cmc6dO9OlSxd27NhR4c9Wkm/TJvjtb+G008LtiSege/cwFLNpEzzzDHz3u9pBGhUpO+Z+yy1hqs/K1LkzxHKxWE2aNOHMM8/kzTffZNCgQUycOJEhQ4ZgZqSnp/Paa69x5JFHsnXrVrp3787AgQNLvI7oM888Q926dVm8eDGLFy8uNGXvmDFjOOqoo9i3bx99+/Zl8eLF3HTTTYwdO5bZs2fnz+yYZ8GCBYwfP565c+fi7nTr1o1evXrRuHFjVq5cySuvvMJzzz3HZZddxpQpUxgxYkSJ73HkyJE8+eST9OrVi3vvvZf777+fxx9/nIceeog1a9Zw+OGH5w8FPfroozz99NP06NGDnTt3kp6eXo5PW1LBrl3w+uvhaJe//jVcaahbN3j66TC7ouZwiS713IuIH5qJH5Jxd+666y46depEv379+PTTT9m8eXOJ65kzZ05+yHbq1IlOnTrlPzdp0iS6du1Kly5dWLp0aZmTgr333ntcfPHF1KtXj/r163PJJZfkTx/cunXr/At4xE8ZXJycnBy2b99Or169ALjiiiuYM2dOfo3Dhw9nwoQJ+WfC9ujRg9tuu40nnniC7du36wzZamL/fnj77TB+fswxYTz944/hrrvC1/ffhxtuULBHXcr+tpbWw65KF110Ebfddlv+VZbyetxZWVls2bKFBQsWULt2bTIyMoqd5jdecb36NWvW8OijjzJv3jwaN27MlVdeWeZ6Spv/J2+6YAhTBpc1LFOS6dOnM2fOHKZNm8aDDz7I0qVLGT16NAMGDGDGjBl0796dv/3tb7Rt27ZC65eqt3Jl6KG//HKYDrtBA7jssjCW3rNnzbsSUU2nb3cR9evXp3fv3vzXf/1XoR2pOTk5HH300dSuXZvZs2ezrrjJ5OOcc845ZGVlAbBkyRIWL14MhOmC69WrR8OGDdm8eTMzZ87Mf02DBg2KHdc+55xzeP3119m1axdff/01r732GmeffXa531vDhg1p3Lhxfq//5ZdfplevXuzfv58NGzbQp08fHnnkEbZv387OnTv55JNP6NixI3feeSeZmZl8/PHH5d6mVK3t2+HZZ+Gss+A73wlni7ZtC3/6E3z2GbzwApxzjoK9JkrZnnsyDRs2jEsuuaTQkTPDhw/nwgsvJDMzk86dO5fZg73++uu56qqr6NSpE507d+bMM88EwlWVunTpQvv27Q+YLnjUqFGcf/75HHfcccyePTt/edeuXbnyyivz13HNNdfQpUuXUodgSvLiiy9y3XXXsWvXLtq0acP48ePZt28fI0aMICcnB3fn1ltvpVGjRvziF79g9uzZpKWl0a5du/yrSkly5R2++OKLBdMAtG8PjzwShmCOPz7ZFUoq0JS/clD0PTt0PvwwBHpWVriSUdOmcPnl4azRrl11lEtNoSl/RSJg8+YQ5i+9FMK9du1wMeiRI+H88yHuSo4ihSjcRVLMnj1huOXFF8Pwy759cOaZ8NRTMHSojnKRxKRcuOddDk5SX7KG9KLIHf71rxDor74KOTnQvDnccUfopWvkS8orpcI9PT2dbdu20aRJEwV8inN3tm3bphObDtLateHQxZdeClcxqlsXLrkkHL7Ypw+kpSW7QqmuUircW7RoQXZ2Nlu2bEl2KZKA9PR0WrRokewyqp0dO2Dy5BDob78dlvXurasYSeVKqXCvXbs2rVu3TnYZIpVu3z54660w7PLnP8Pu3XDyyfDgg+GiF61aJbtCiZqUCneRqFm+PAT6hAnw6adh6tyRI8OwS/fuOnxRqo7CXaSS5eQUjKPPmxfGzfv3h8ceC4cxajeFHAoKd5FKsmYN/O534ZT/nTvDtLpjx8KwYXDsscmuTmoahbvIQcg7hHHsWHjttTCHy9ChYcrq009PdnVSkyncRSogNzfsGB07FubOhcaN4b//G268MRyfLpJsCneRcsjJgeefD1cxWr8eTjopXPjiiiugXr1kVydSQOEukoA1a0KgP/98GE/v1QuefBIGDNCJRpKaEprl2cz6m9kKM1tlZqOLeb6lmc02s3+b2WIzu6DySxU5tNzDBaMHDw499KeegkGDYP78cPLRwIEKdkldZfbczSwNeBr4HpANzDOzae4ef224e4BJ7v6MmbUDZgAZVVCvSJUrOp7eqJHG06X6SWRY5kxglbuvBjCzicAgID7cHTgydr8hsLEyixQ5FIobT3/qqTCeXr9+sqsTKZ9Ewr05sCHucTbQrUib+4C/mNlPgXpAv+JWZGajgFEALVu2LG+tIlVC4+kSRYmMuRd3gnTRuV6HAX909xbABcDLZnbAut19nLtnuntms2bNyl+tSCXReLpEXSI992zghLjHLThw2OVqoD+Au//LzNKBpsDnlVGkSGXReLrUFIn03OcBJ5tZazOrAwwFphVpsx7oC2BmpwLpgObtlZSRkxMC/cQTYcgQ2LYt9NY3bIDf/EbBLtFTZs/d3XPN7EZgFpAG/MHdl5rZA8B8d58G/Ax4zsxuJQzZXOm6TI+kgLzx9BdeCPOoazxdaoqETmJy9xmEwxvjl90bd38Z0KNySxOpuLz5Xv785zDfy5AhcOutmu9Fag6doSqRsXt3uLD0Y48VjKffcUcYT9cFo6SmUbhLtbZrF8ycCf/7v/DGG/D11zo+XQQU7lIN7dwJ06eH65DOmBECvmlTGD48HNp47rkaTxdRuEu18NVXoWc+eXLoqe/ZA8ccA1deGQL97LOhln6aRfLp10FS1vbtYQx98mSYNQu+/RaOPx6uvTYEeo8e6qGLlEThLinliy9g6tQQ6H/9K+zdG3aG3nADXHppuKj0YQnNZSpSsyncJem2bg2XqJs8Gd56K5xFmpEBN98ceuhnnKFAFykvhbskxebNBYH+9tuwb184e/RnPws99K5dwYqb1UhEEqJwl0Nm48ZwUtHkyTBnTpi86zvfgdGjQw/9tNMU6CKVReEuVWrDhoJA/8c/QqC3awe/+EUI9A4dFOgiVUHhLpVu3TqYMiWcWPT++2FZx45w//3wwx+GcBeRqqVwl0rxySch0CdPhnnzwrIuXWDMmBDop5yS3PpEahqFu1TI3r3w3nvhTNHp0+Hjj8PyzEx4+OEQ6CeemNwaRWoyhbskbPPmcHbo9Onwl7+Es0br1AnT6F53XbiSUUZGsqsUEVC4Syn274cFCwp65/Pnh+XHHw+XXRbmRO/XT5NziaQihbsU8tVXoVc+fXropW/eHI5m6dYNHnwwBHrnzjrCRSTVKdxrOHdYsaKgd/7uu+EM0UaN4PvfD2Hevz/oeuYi1YvCvQbaswfeeacg0FevDss7dAhniF5wAZx1lmZZFKnO9OtbQ3z6aUGY/+1vYQ709HTo2xduvz0EeqtWya5SRCqLwj2i9u0Ll5rLC/QPPwzLW7UKc6APGAB9+sARRyS1TBGpIgr3CPniizDv+fTp8OabsG1bmO+8R49w7PmAAeHsUO0MFYk+hXs1tmcPLFkShlmmT4d//jMcvti0aRhmGTAAzjsPGjdOdqUicqhVu3B/5RX4/e/Dzr/4W9QDbOvWMLSyaFHBbfnyMPwC4VT/u+4KgX7GGbpCkUhNV63CPSsLbrklBN2774bD+PI0bx5CvmPHgsBv1676jSnv3w9r1hQO8UWLIDu7oE3z5uFY80GDwtezzgonFomI5Kk24Z6VBaNGhaM8IAR7ejr89KdhGGLJknB78kn45pvQxgxOOunA0D/55NQ4zG/PHli6tHCIf/gh7NgRnk9Lg7Ztw+n9nTuH22mn6ZhzESmbeXz39xDKzMz0+XnnsycgIyNMJVtUq1awdm3B49zcMEPhkiXw0UcFob9yZegVQ5gP5dRTC8I+L/hbtqy6nY1lDavUrx+COy/EO3eG9u2r338eIlK1zGyBu2eW2a66hPthhxUehsljVhDapdm9O8xcWDT0N2woaNOgwYFj+R07lq+nvH9/OCkovide0rBK/K1NG10nVETKlmi4p8DgRGJatiy+596yZWKvP+KIsNOxS5fCy7dvD0Mj8aE/ZQo891xBm6OPPnBop317qF07tC86rLJzZ3hdWlr4D6F374JeuYZVRORQqDbhPmZM4TF3gLp1w/KD0ahROA68R4+CZe7w2WcFvfu80H/uucLbT0srPKzSuXM4QSh+WCU9/eDqExGpiGoT7sOHh6933w3r14ce+5gxBcsrkxkcd1y4fe97BcvzjmTJC/xvvinokWtYRURSSbUZcxcRkcTH3BPqa5pZfzNbYWarzGx0CW0uM7NlZrbUzP5U3oJFRKTylDksY2ZpwNPA94BsYJ6ZTXP3ZXFtTgZ+DvRw9y/N7OiqKlhERMqWSM/9TGCVu69292+BicCgIm2uBZ529y8B3P3zyi1TRETKI5Fwbw7EHQ1OdmxZvO8A3zGzf5jZ+2bWv7gVmdkoM5tvZvO3bNlSsYpFRKRMiYR7cedsFt0LWws4GegNDAOeN7NGB7zIfZy7Z7p7ZjMd7C0iUmUSCfds4IS4xy2AjcW0merue919DbCCEPYiIpIEiYT7POBkM2ttZnWAocC0Im1eB/oAmFlTwjDN6sosVEREEldmuLt7LnAjMAtYDkxy96Vm9oCZDYw1mwVsM7NlwGzgDnffVlVFi4hI6XQSk4hINVKpJzGJiEj1onAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEISijczay/ma0ws1VmNrqUdoPNzM0ss/JKFBGR8ioz3M0sDXgaOB9oBwwzs3bFtGsA3ATMrewiRUSkfBLpuZ8JrHL31e7+LTARGFRMuweBR4A9lVifiIhUQCLh3hzYEPc4O7Ysn5l1AU5w9zdKW5GZjTKz+WY2f8uWLeUuVkREEpNIuFsxyzz/SbPDgMeAn5W1Incf5+6Z7p7ZrFmzxKsUEZFySSTcs4ET4h63ADbGPW4AdADeNrO1QHdgmnaqiogkTyLhPg842cxam1kdYCgwLe9Jd89x96bunuHuGcD7wEB3n18lFYuISJnKDHd3zwVuBGYBy4FJ7r7UzB4ws4FVXaCIiJRfrUQaufsMYEaRZfeW0Lb3wZclIiIHQ2eoiohEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7iIiEaRwFxGJIIW7iEgEKdxFRCJI4S4iEkEKdxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiKKFwN7P+ZrbCzFaZ2ehinr/NzJaZ2WIz+7uZtar8UkVEJFFlhruZpQFPA+cD7YBhZtauSLN/A5nu3gmYDDxS2YWKiEjiEum5nwmscvfV7v4tMBEYFN/A3We7+67Yw/eBFpVbpoguz+bLAAAG3ElEQVSIlEci4d4c2BD3ODu2rCRXAzMPpigRETk4tRJoY8Us82Ibmo0AMoFeJTw/ChgF0LJlywRLFBGR8kqk554NnBD3uAWwsWgjM+sH3A0MdPdviluRu49z90x3z2zWrFlF6hURkQQkEu7zgJPNrLWZ1QGGAtPiG5hZF+BZQrB/XvlliohIeZQZ7u6eC9wIzAKWA5PcfamZPWBmA2PN/geoD/yvmS0ys2klrE5ERA6BRMbccfcZwIwiy+6Nu9+vkusSEZGDoDNUKyArCzIy4LDDwtesrGRXJCJSWEI9dymQlQWjRsGu2FH969aFxwDDhyevLhGReOq5l9PddxcEe55du8JyEZFUoXAvp/Xry7dcRCQZFO7lVNK5VzonS0RSicK9nMaMgbp1Cy+rWzcsFxFJFQr3cho+HMaNg1atwCx8HTdOO1NFJLXoaJkKGD5cYS4iqU09dxGRCFK4i4hEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJBCncRkQhSuIuIRJDCXUQkghTuIiIRpHAXEYkghbuISAQp3EVEIkjhLiISQQp3EZEIUriLiESQwl1EJIIU7tVYVhZkZMBhh4WvWVnJrkhEUoUukF1NZWXBqFGwa1d4vG5deAy6eLeIqOdebd19d0Gw59m1KywXEVG4V1Pr15dvuYjULAmFu5n1N7MVZrbKzEYX8/zhZvZq7Pm5ZpZR2YVKYS1blm95VUqVsf9UqCMValAdqgMAdy/1BqQBnwBtgDrAh0C7Im1uAH4fuz8UeLWs9Z5++ukuFTdhgnvduu5QcKtbNyxXHcmpIxVqUB3RrwOY72Xkq4fVlxnu3wVmxT3+OfDzIm1mAd+N3a8FbAWstPUq3A/ehAnurVq5m4Wvh/qH1T1sN/6HNe/WqlXNqyMValAd0a8j0XC30LZkZjYY6O/u18Qe/wjo5u43xrVZEmuTHXv8SazN1iLrGgWMAmjZsuXp69atK+8/GpJiDjss/IgWZQb799esOlKhBtUR/TrMbIG7Z5a5vUTWVcyyoiUm0gZ3H+fume6e2axZswQ2LakuVcb+U6GOVKhBdaiOPImEezZwQtzjFsDGktqYWS2gIfBFZRQoqW3MGKhbt/CyunXD8ppWRyrUoDpUR76yxm0IY+irgdYU7FBtX6TNTyi8Q3VSWevVmHt0pMLYf6rUkQo1qI5o10FljbkDmNkFwOOEI2f+4O5jzOyB2EammVk68DLQhdBjH+ruq0tbZ2Zmps+fP78Cf45ERGquRMfcE5p+wN1nADOKLLs37v4e4NLyFikiIlVDZ6iKiESQwl1EJIIU7iIiEaRwFxGJoISOlqmSDZttAar7KapNCVMtSKDPo4A+i8L0eRR2MJ9HK3cv8yzQpIV7FJjZ/EQOSaop9HkU0GdRmD6Pwg7F56FhGRGRCFK4i4hEkML94IxLdgEpRp9HAX0WhenzKKzKPw+NuYuIRJB67iIiEaRwFxGJIIV7BZjZCWY228yWm9lSM7s52TUlm5mlmdm/zeyNZNeSbGbWyMwmm9nHsZ+R7ya7pmQys1tjvydLzOyV2CyyNYKZ/cHMPo9drS5v2VFm9lczWxn72rgqtq1wr5hc4GfufirQHfiJmbVLck3JdjOwPNlFpIjfAW+6e1vgNGrw52JmzYGbgEx370CYNnxocqs6pP4I9C+ybDTwd3c/Gfh77HGlU7hXgLtvcveFsfs7CL+8zZNbVfKYWQtgAPB8smtJNjM7EjgHeAHA3b919+3JrSrpagFHxK7SVpcDr+QWWe4+hwOvSjcIeDF2/0XgoqrYtsL9IJlZBuEiJXOTW0lSPQ78N3AILzecstoAW4DxsWGq582sXrKLShZ3/xR4FFgPbAJy3P0vya0q6Y5x900QOorA0VWxEYX7QTCz+sAU4BZ3/yrZ9SSDmf0A+NzdFyS7lhRRC+gKPOPuXYCvqaJ/u6uD2HjyIMJlOo8H6pnZiORWVTMo3CvIzGoTgj3L3f+c7HqSqAcw0MzWAhOBc81sQnJLSqpsINvd8/6Tm0wI+5qqH7DG3be4+17gz8BZSa4p2Tab2XEAsa+fV8VGFO4VYGZGGFNd7u5jk11PMrn7z929hbtnEHaUveXuNbZn5u6fARvM7JTYor7AsiSWlGzrge5mVjf2e9OXGryDOWYacEXs/hXA1KrYSELXUJUD9AB+BHxkZotiy+6KXWtW5KdAlpnVAVYDVyW5nqRx97lmNhlYSDjK7N/UoKkIzOwVoDfQ1MyygV8CDwGTzOxqwh+/Krn+tKYfEBGJIA3LiIhEkMJdRCSCFO4iIhGkcBcRiSCFu4hIBCncRUQiSOEuIhJB/x86Oeahd3GmyQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "훈련 샘플의 수를 늘리니 단어 임베딩을 같이 훈련하는 모델의 검증 정확도가 70%를 넘었습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "마지막으로 테스트 데이터에서 모델을 평가해 보죠. 먼저 테스트 데이터를 토큰화해야 합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_dir = os.path.join(imdb_dir, 'test')\n",
    "\n",
    "labels = []\n",
    "texts = []\n",
    "\n",
    "for label_type in ['neg', 'pos']:\n",
    "    dir_name = os.path.join(test_dir, label_type)\n",
    "    for fname in sorted(os.listdir(dir_name)):\n",
    "        if fname[-4:] == '.txt':\n",
    "            f = open(os.path.join(dir_name, fname), encoding=\"utf8\")\n",
    "            texts.append(f.read())\n",
    "            f.close()\n",
    "            if label_type == 'neg':\n",
    "                labels.append(0)\n",
    "            else:\n",
    "                labels.append(1)\n",
    "\n",
    "sequences = tokenizer.texts_to_sequences(texts)\n",
    "x_test = pad_sequences(sequences, maxlen=maxlen)\n",
    "y_test = np.asarray(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그다음 이 절의 첫 번째 모델을 로드하고 평가합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000/25000 [==============================] - 1s 21us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[1.6611276818060874, 0.50412]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.load_weights('pre_trained_glove_model.h5')\n",
    "model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "테스트 정확도는 겨우 50% 정도입니다. 적은 수의 훈련 샘플로 작업하는 것은 어려운 일이군요!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}