{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "“12_Embeddings_cn”", "version": "0.3.2", "provenance": [], "collapsed_sections": [], "toc_visible": true }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "accelerator": "GPU" }, "cells": [ { "metadata": { "id": "bOChJSNXtC9g", "colab_type": "text" }, "cell_type": "markdown", "source": [ "# 嵌入层" ] }, { "metadata": { "id": "OLIxEDq6VhvZ", "colab_type": "text" }, "cell_type": "markdown", "source": [ "\n", "\n", "如今，我们使用一组one-hot编码来表示文本，它是一个n维数组，其中每个索引对应一个词条（token）。该索引处的值与该单词在句子中出现的次数相对应。这种方法使得我们完全丢失输入文本中的结构性信息。\n", "\n", "```python\n", "[0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0.\n", " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0.\n", " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n", " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]```\n", "\n", " 我们还用一种one-hot编码来表示输入，其中每个词条由一个n维数组表示。\n", " \n", " ```python\n", "[[0. 0. 0. ... 0. 0. 0.]\n", " [0. 0. 1. ... 0. 0. 0.]\n", " [0. 0. 0. ... 0. 0. 0.]\n", " ...\n", " [0. 0. 0. ... 0. 0. 0.]\n", " [0. 0. 0. ... 0. 0. 0.]\n", " [0. 0. 0. ... 0. 0. 0.]]\n", "```\n", "\n", "这种表示方式允许我们保存结构性信息，但有两个主要缺点。如果我们有一个很大的词汇表，每个词条标记的表示长度将是巨大的，从而导致大量计算。虽然我们保留了文本中的结构信息，但是每个词条的实际表示并没有保留着与其他词条的关系。\n", "\n", "在本节笔记中，我们将学习嵌入层以及它们是如何解决我们目前各种表示方法的不足之处。\n", "\n", "\n", "\n" ] }, { "metadata": { "id": "VoMq0eFRvugb", "colab_type": "text" }, "cell_type": "markdown", "source": [ "# 概述" ] }, { "metadata": { "id": "qWro5T5qTJJL", "colab_type": "text" }, "cell_type": "markdown", "source": [ "* **目标：**表示文本中包含内在语义关系的词条。\n", "* **优点：**\n", " * 捕捉关系的同时保持低维度\n", " * 可解释的词条表示\n", "* **缺点：**\n", " * 没有\n", "* **其他方面：**有许多预训练好的嵌入层可供选择，但是您也可以从头开始训练自己的嵌入层。\n", " \n" ] }, { "metadata": { "id": "2thIKTLYwzJd", "colab_type": "text" }, "cell_type": "markdown", "source": [ "# 嵌入层学习" ] }, { "metadata": { "id": "MlxioJLqx2Ls", "colab_type": "text" }, "cell_type": "markdown", "source": [ "嵌入层的主要思想是为文本中的词条提供固定长度的表示，而不考虑词汇表中的词条数量。因此，每个词条的表示形式的形状不再是[1 X V]（V是词汇表的大小），而是[1 X D]（D是嵌入层大小(通常为50,100,200,300)）。表示中的数字不再是0和1，而是在D维潜在空间中表示该词条的浮点小数。如果嵌入层确实捕获了词条之间的关系，那么我们应该能够发现这个潜在空间并确认已知的关系(我们很快就会这么做)。\n", "\n", "但是我们首先应该如何学习嵌入层呢?嵌入层的直观解释是，词条的定义不取决于词条本身，而是取决于它的上下文。下面有几种不同的做法:\n", "\n", "1. 给定上下文中的单词，预测目标单词(CBOW模型 -连续词袋)。\n", "2. 给定目标词，预测上下文单词（除中心单词外窗口内的其他单词）(skip-gram模型)。\n", "3. 给定一个单词序列，预测下一个单词(LM语言建模)。\n", "\n", "这些方法都涉及到创建数据来训练我们的模型。句子中的每个词都成为目标词，上下文词由一个窗口决定。在下面的图(skip-gram)中，窗口大小为2。我们对语料库中的每个句子都重复这个过程，这就产生了针对非监督任务的训练数据。这是一种无监督学习技术，因为我们没有正式的上下文标签。我们的想法是，相似的目标词会出现在相似的语境中，我们可以通过反复训练这种匹配模式(上下文，目标)来学习这种关系。\n", "\n", "\n", "\n", "我们可以使用上述任何一种方法学习嵌入层，各有优劣。确认用哪种方法的最佳方式是验证它在有监督任务上的性能表现。我们可以通过在PyTorch中创建模型来学习嵌入层，但是这里我们将使用一个专门用于嵌入层和主题建模的库[Gensim](https://radimrehurek.com/gensim/)。" ] }, { "metadata": { "id": "uGDGEVvz41LL", "colab_type": "code", "outputId": "205d7ff2-3b1d-4b92-9127-c6e15ad65f99", "colab": { "base_uri": "https://localhost:8080/", "height": 339 } }, "cell_type": "code", "source": [ "!pip install gensim " ], "execution_count": 3, "outputs": [ { "output_type": "stream", "text": [ "Requirement already satisfied: gensim in /usr/local/lib/python3.6/dist-packages (3.6.0)\n", "Requirement already satisfied: smart-open>=1.2.1 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.7.1)\n", "Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.14.6)\n", "Requirement already satisfied: six>=1.5.0 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.11.0)\n", "Requirement already satisfied: scipy>=0.18.1 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.1.0)\n", "Requirement already satisfied: boto>=2.32 in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (2.49.0)\n", "Requirement already satisfied: bz2file in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (0.98)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (2.18.4)\n", "Requirement already satisfied: boto3 in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (1.9.67)\n", "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (3.0.4)\n", "Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (1.22)\n", "Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (2.6)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (2018.11.29)\n", "Requirement already satisfied: s3transfer<0.2.0,>=0.1.10 in /usr/local/lib/python3.6/dist-packages (from boto3->smart-open>=1.2.1->gensim) (0.1.13)\n", "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.6/dist-packages (from boto3->smart-open>=1.2.1->gensim) (0.9.3)\n", "Requirement already satisfied: botocore<1.13.0,>=1.12.67 in /usr/local/lib/python3.6/dist-packages (from boto3->smart-open>=1.2.1->gensim) (1.12.67)\n", "Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= \"2.7\" in /usr/local/lib/python3.6/dist-packages (from botocore<1.13.0,>=1.12.67->boto3->smart-open>=1.2.1->gensim) (2.5.3)\n", "Requirement already satisfied: docutils>=0.10 in /usr/local/lib/python3.6/dist-packages (from botocore<1.13.0,>=1.12.67->boto3->smart-open>=1.2.1->gensim) (0.14)\n" ], "name": "stdout" } ] }, { "metadata": { "id": "9wr9S6965DD7", "colab_type": "code", "outputId": "c8361503-4c69-4310-b97e-641be5b0b16f", "colab": { "base_uri": "https://localhost:8080/", "height": 50 } }, "cell_type": "code", "source": [ "import os\n", "from argparse import Namespace\n", "import copy\n", "import gensim\n", "from gensim.models import Word2Vec\n", "import json\n", "import nltk; nltk.download('punkt')\n", "import numpy as np\n", "import pandas as pd\n", "import re\n", "import urllib\n", "import warnings\n", "warnings.filterwarnings('ignore')" ], "execution_count": 4, "outputs": [ { "output_type": "stream", "text": [ "[nltk_data] Downloading package punkt to /root/nltk_data...\n", "[nltk_data] Unzipping tokenizers/punkt.zip.\n" ], "name": "stdout" } ] }, { "metadata": { "id": "5-sx-n9655TJ", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "args = Namespace(\n", " seed=1234,\n", " data_file=\"harrypotter.txt\",\n", " embedding_dim=100,\n", " window=5,\n", " min_count=3,\n", " skip_gram=1, # 0 = CBOW\n", " negative_sampling=20,\n", ")" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "VaJSRgNEg_V_", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 将数据从github上更新到本地notebook\n", "url = \"https://raw.githubusercontent.com/LisonEvf/practicalAI-cn/master/data/harrypotter.txt\"\n", "response = urllib.request.urlopen(url)\n", "html = response.read()\n", "with open(args.data_file, 'wb') as fp:\n", " fp.write(html)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "irvgngOG5yqk", "colab_type": "code", "outputId": "fa3bd3aa-ce8d-45c7-db57-242881c7405f", "colab": { "base_uri": "https://localhost:8080/", "height": 50 } }, "cell_type": "code", "source": [ "# 将文本分割成每个句子\n", "tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')\n", "with open(args.data_file, encoding='cp1252') as fp:\n", " book = fp.read()\n", "sentences = tokenizer.tokenize(book)\n", "print (len(sentences))\n", "print (sentences[11])" ], "execution_count": 7, "outputs": [ { "output_type": "stream", "text": [ "15640\n", "Snape nodded, but did not elaborate.\n" ], "name": "stdout" } ] }, { "metadata": { "id": "VTREFDg47Vrx", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 数据预处理\n", "def preprocess_text(text):\n", " text = ' '.join(word.lower() for word in text.split(\" \"))\n", " text = re.sub(r\"([.,!?])\", r\" \\1 \", text)\n", " text = re.sub(r\"[^a-zA-Z.,!?]+\", r\" \", text)\n", " text = text.strip()\n", " return text" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "IX7pkCbg7WPK", "colab_type": "code", "outputId": "1aa67d6c-4b75-4abd-c0ce-cc7ca649fe30", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "cell_type": "code", "source": [ "# 对句子进行数据清洗\n", "sentences = [preprocess_text(sentence) for sentence in sentences]\n", "print (sentences[11])" ], "execution_count": 9, "outputs": [ { "output_type": "stream", "text": [ "snape nodded , but did not elaborate .\n" ], "name": "stdout" } ] }, { "metadata": { "id": "dsmEEVzX5ytO", "colab_type": "code", "outputId": "2fcbdc06-0a05-4f3d-e96c-4039707a042c", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "cell_type": "code", "source": [ "# 为gensim对句子进行预处理\n", "sentences = [sentence.split(\" \") for sentence in sentences]\n", "print (sentences[11])" ], "execution_count": 10, "outputs": [ { "output_type": "stream", "text": [ "['snape', 'nodded', ',', 'but', 'did', 'not', 'elaborate', '.']\n" ], "name": "stdout" } ] }, { "metadata": { "id": "Fa-DbjPW-KC3", "colab_type": "text" }, "cell_type": "markdown", "source": [ "当我们有大型的词汇表去学习嵌入层时，事情很快就会变得复杂起来。回想一下，softmax的反向传播更新了正确类和错误类的权重。对于每一次的回传，都需要大量的计算，所以解决方法是使用负采样，它只需要更新正确类和一些随机的错误类(negative_sampling=20)。我们之所以能够这样做，是因为我们有大量的训练数据，在这些数据中，我们将多次看到与目标类相同的单词。\n", "\n" ] }, { "metadata": { "id": "wzkUkl-b5ywC", "colab_type": "code", "outputId": "f9d0adc3-3738-48a2-bbb1-2dea7737dcc0", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "cell_type": "code", "source": [ "# 由于底层是C语言写的所以速度非常快\n", "model = Word2Vec(sentences=sentences, size=args.embedding_dim, \n", " window=args.window, min_count=args.min_count, \n", " sg=args.skip_gram, negative=args.negative_sampling)\n", "print (model)" ], "execution_count": 11, "outputs": [ { "output_type": "stream", "text": [ "Word2Vec(vocab=4837, size=100, alpha=0.025)\n" ], "name": "stdout" } ] }, { "metadata": { "id": "gcMb9HUd7vS_", "colab_type": "code", "outputId": "090ae4f6-f896-4af8-90ec-d0492df571d6", "colab": { "base_uri": "https://localhost:8080/", "height": 370 } }, "cell_type": "code", "source": [ "# 将每个词进行向量化\n", "model.wv.get_vector(\"potter\")" ], "execution_count": 12, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([ 0.57121795, -0.4473022 , -0.24764916, -0.02014627, 0.21817273,\n", " -0.17684223, 0.0971131 , -0.22699513, -0.44662133, 0.34343132,\n", " -0.43126193, 0.14076817, -0.30172673, -0.34286836, -0.01562079,\n", " -0.1676844 , 0.10327088, -0.27377397, 0.21167807, 0.00207133,\n", " 0.08535747, 0.09200649, 0.31590024, -0.00346842, -0.0040499 ,\n", " 0.05679551, -0.03496649, 0.12977482, 0.5481599 , -0.05868822,\n", " 0.22223113, -0.65511256, -0.22511719, 0.01033045, -0.47850764,\n", " 0.27103794, 0.22211818, 0.05265665, 0.08114785, -0.10955557,\n", " -0.23463504, 0.05834079, 0.38490152, 0.3176595 , -0.04837222,\n", " -0.07741249, 0.08143531, 0.08240102, 0.23861782, -0.15033284,\n", " -0.37583345, 0.5298492 , -0.27770242, -0.02436154, 0.6498464 ,\n", " 0.15981446, -0.08876655, -0.11807754, 0.04486049, -0.08496018,\n", " 0.5243146 , 0.05710106, 0.0346731 , -0.50119925, 0.1966134 ,\n", " -0.046356 , -0.22535692, 0.15454814, 0.20338546, -0.169614 ,\n", " 0.12269565, -0.12660046, -0.33274096, -0.07981768, -0.13004518,\n", " 0.17338865, 0.2412951 , -0.3526382 , -0.4982048 , -0.062686 ,\n", " -0.07248274, -0.05320947, -0.4392711 , -0.15316884, -0.05510715,\n", " -0.07104576, 0.16983937, 0.13383926, -0.28093493, -0.06915343,\n", " -0.23352067, 0.27850398, 0.06351388, 0.17225006, 0.19435039,\n", " -0.06836061, 0.06705192, -0.33565933, -0.21312903, -0.32038397],\n", " dtype=float32)" ] }, "metadata": { "tags": [] }, "execution_count": 12 } ] }, { "metadata": { "id": "BozlP5WG70Ak", "colab_type": "code", "outputId": "cd975735-100e-4a06-846a-743e060f3883", "colab": { "base_uri": "https://localhost:8080/", "height": 101 } }, "cell_type": "code", "source": [ "# 找到距离最近的单词(不包括自己)\n", "model.wv.most_similar(positive=\"scar\", topn=5)" ], "execution_count": 13, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "[('pain', 0.9411180019378662),\n", " ('forehead', 0.9267646074295044),\n", " ('prickling', 0.9210630059242249),\n", " ('mouth', 0.9191247820854187),\n", " ('burning', 0.9186864495277405)]" ] }, "metadata": { "tags": [] }, "execution_count": 13 } ] }, { "metadata": { "id": "q-wgfMnH68vQ", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 保存权重值\n", "model.wv.save_word2vec_format('model.txt', binary=False)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "K1OC2smgw1P5", "colab_type": "text" }, "cell_type": "markdown", "source": [ "# 预训练嵌入层" ] }, { "metadata": { "id": "DHUU_-TJz_7K", "colab_type": "text" }, "cell_type": "markdown", "source": [ "我们可以使用上面的任意一种方法从头开始学习嵌入层，另外我们也可以使用已经在数百万文档上经过训练的预训练嵌入层。目前流行的嵌入层包括Word2Vec (skip-gram)或GloVe (global word-word co-occurrence)。我们可以通过确认这些嵌入层捕获的有意义的语义关系来进行验证操作。" ] }, { "metadata": { "id": "vJNngmZjglVg", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "from gensim.scripts.glove2word2vec import glove2word2vec\n", "from gensim.models import KeyedVectors\n", "from io import BytesIO\n", "import matplotlib.pyplot as plt\n", "from sklearn.decomposition import PCA\n", "from zipfile import ZipFile\n", "from urllib.request import urlopen" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "5WZ2bT27gncI", "colab_type": "code", "outputId": "686307fa-1dbd-486f-ff79-8be6b4e2e019", "colab": { "base_uri": "https://localhost:8080/", "height": 84 } }, "cell_type": "code", "source": [ "# 解压缩文件（大约3分钟）\n", "resp = urlopen('http://nlp.stanford.edu/data/glove.6B.zip')\n", "zipfile = ZipFile(BytesIO(resp.read()))\n", "zipfile.namelist()" ], "execution_count": 16, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['glove.6B.50d.txt',\n", " 'glove.6B.100d.txt',\n", " 'glove.6B.200d.txt',\n", " 'glove.6B.300d.txt']" ] }, "metadata": { "tags": [] }, "execution_count": 16 } ] }, { "metadata": { "id": "NNacKuRNi8_x", "colab_type": "code", "outputId": "debec622-f3b7-4294-90e3-acee1f2ac8c9", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "cell_type": "code", "source": [ "# 写入嵌入层\n", "embeddings_file = 'glove.6B.{0}d.txt'.format(args.embedding_dim)\n", "zipfile.extract(embeddings_file)" ], "execution_count": 17, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "'/content/glove.6B.100d.txt'" ] }, "metadata": { "tags": [] }, "execution_count": 17 } ] }, { "metadata": { "id": "SA5Y1BGbbpPo", "colab_type": "code", "outputId": "d5f53c76-1c7a-4c86-b260-b765fac26a5f", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "cell_type": "code", "source": [ "# 保存word2vec格式的GloVe嵌入层到本地词典\n", "word2vec_output_file = '{0}.word2vec'.format(embeddings_file)\n", "glove2word2vec(embeddings_file, word2vec_output_file)" ], "execution_count": 18, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "(400000, 100)" ] }, "metadata": { "tags": [] }, "execution_count": 18 } ] }, { "metadata": { "id": "bQp1pIJLiZw3", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 读取嵌入层（大约一分钟）\n", "glove = KeyedVectors.load_word2vec_format(word2vec_output_file, binary=False)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "HDpHnC0sbpVE", "colab_type": "code", "outputId": "0e75f0cd-ca2f-4043-9b6a-2a2d5e5798e8", "colab": { "base_uri": "https://localhost:8080/", "height": 101 } }, "cell_type": "code", "source": [ "# (king - man) + woman = ?\n", "glove.most_similar(positive=['woman', 'king'], negative=['man'], topn=5)" ], "execution_count": 20, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "[('queen', 0.7698541283607483),\n", " ('monarch', 0.6843380928039551),\n", " ('throne', 0.6755735874176025),\n", " ('daughter', 0.6594556570053101),\n", " ('princess', 0.6520534753799438)]" ] }, "metadata": { "tags": [] }, "execution_count": 20 } ] }, { "metadata": { "id": "_1TatbmvbpYU", "colab_type": "code", "outputId": "8df52b35-5bd7-4fdd-a4fd-6ee5da0e3ccb", "colab": { "base_uri": "https://localhost:8080/", "height": 101 } }, "cell_type": "code", "source": [ "# 找到距离最近的单词(不包括自己)\n", "glove.wv.most_similar(positive=\"goku\", topn=5)" ], "execution_count": 21, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "[('gohan', 0.7246542572975159),\n", " ('bulma', 0.6497020125389099),\n", " ('raistlin', 0.6443604230880737),\n", " ('skaar', 0.6316742897033691),\n", " ('guybrush', 0.6231324672698975)]" ] }, "metadata": { "tags": [] }, "execution_count": 21 } ] }, { "metadata": { "id": "oMljB5MEbpdb", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 降维方便绘图\n", "X = glove[glove.wv.vocab]\n", "pca = PCA(n_components=2)\n", "pca_results = pca.fit_transform(X)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "Aok5pRqXqcCf", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "def plot_embeddings(words, embeddings, pca_results):\n", " for word in words:\n", " index = embeddings.index2word.index(word)\n", " plt.scatter(pca_results[index, 0], pca_results[index, 1])\n", " plt.annotate(word, xy=(pca_results[index, 0], pca_results[index, 1]))\n", " plt.show()" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "NOdLFgTOrrBd", "colab_type": "code", "outputId": "b65ec57d-de4e-47a8-c814-710eb55c2949", "colab": { "base_uri": "https://localhost:8080/", "height": 347 } }, "cell_type": "code", "source": [ "plot_embeddings(words=[\"king\", \"queen\", \"man\", \"woman\"], embeddings=glove, \n", " pca_results=pca_results)" ], "execution_count": 24, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAFKCAYAAADMuCxnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHp5JREFUeJzt3Xt0VPW9/vFnLrkQEsIkmQgmYYEg\noNwairYRFE2DpcrPVoskFMW7h1Vp8YItBA+xItcKugB/aqO1KkhoaUSpHtMqQTlcKioggaOQqMgl\nhgm5wBACJJnzh3WOlGQmmmTmm5n366/M7L3n+5lnsebJ3jNMLB6PxyMAABBU1mAPAAAAKGQAAIxA\nIQMAYAAKGQAAA1DIAAAYgEIGAMAA9kAv6HIdD/SSXg5HjKqr64K2fmdARr6Rj39k5Bv5+BeKGTmd\ncX73CaszZLvdFuwRjEdGvpGPf2TkG/n4F64ZhVUhAwBgKgoZAAADUMgAABiAQgYAwAAUMgAABqCQ\nAQAwAIUMnz788H1lZ//srPv27CnR/fdPDdJEABCaKGR8axdfPFhLliwP9hgAEFIC/k1d6LwaGhp0\n33336LLLLtfatWu0evVaPffcM6qtrZHL5VJp6T517x6v+fOXKCkpSZ988rHy8mZKkq6++id65531\nmjZtuoYPHxHkZwIA5uEMGa32xBO/V1paLw0YMPCs+4uL39a0aQ/oz39eK4cjQa+//qokadGiucrO\nnqSCglcUGxurAwe+CMbYANApUMholVdeWaODBw/o/vt/e862YcPS1aNHT1ksFl144QBVVHypU6fq\n9ckn/6OsrB9Lkm64YYI8Hk+gxwaAToNCRotOnWlUjfuUjh49qqefXqbExETZ7ee+yxEbG+v92Wq1\nqqmpScePH5fFYlFc3FdfqG632+VwJARsdgDobHgPGedobGrS6vWl2r7XpYOf7lGDx6ob75qv9X9Z\npHfeKfaWrC8xMV3l8XhUX1+v6OhoNTQ0qKamOgDTA0DnxBkyzrF6faneev+gjh47JY8kiy1a/yyt\n1yVX364lSxaopqbG72PExMSod+8+Wr/+H5KkV18tlGTp2MEBoBOjkHGW+tMN2r7X1ey2ilNJuvKq\nq7V48fxWPdb99/9WL774R9100wTV15+U0+mUxUIpA0BzLJ4Af9LG5ToeyOXO4nTGBXX9zqDBYtV/\nzH9Lzf2jsFqkeXf/UMmOmFY/nsfj8ZbwuHFZevzx/68LL+zfTtMGHv+G/CMj38jHv1DMyOn0/1Yf\nZ8g4i6NblBK6RTW/LS5a8bHNb2vOQw/9VitXviBJ+uCDbfJ4POrVq1e7zAkAoaZVhbx3715lZWVp\nxYoVLe6zePFi3Xzzze02GIIjOtKu9P7OZrel909SVISt1Y91551T9O67G5STc4OeeOL3euihRxQV\nFd1eowJASPH7Keu6ujrNmTNHGRkZLe5TWlqqbdu2KSIiol2HQ3BkZ/aTJG3fW6nq4/VyxEUrvX+S\n9/7W6t27j/7whz91wIQAEHr8FnJkZKTy8/OVn5/f4j4LFizQfffdp+XL+X7jUGCzWvWLrP76+ei+\nqnWfUnxs1Lc6MwYAfHt+C9lutzf7ZRBfKyws1KWXXqqUlJRWLehwxMhuD96Le2veWA9338woNYhz\nmIp/Q/6RkW/k4184ZtSmLwapqalRYWGhnn/+eVVUVLTqmOrqurYs2Sah+Mm99kZGvpGPf2TkG/n4\nF4oZdfinrLdu3aqqqipNmjRJU6dO1e7duzVv3ry2PCQAAGGpTWfIY8eO1dixYyVJBw8e1MyZM5Wb\nm9sugwEAEE78FnJJSYkWLlyoQ4cOyW63q6ioSJmZmUpNTdWYMWMCMSMAACGPb+rCWcjIN/Lxj4x8\nIx//QjEjvqkLAIBOgkIGAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBC\nBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAw\nAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCED\nAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiA\nQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAZoVSHv\n3btXWVlZWrFixTnbtm7dqgkTJignJ0czZ85UU1NTuw8JAECo81vIdXV1mjNnjjIyMprdPnv2bC1d\nulQFBQU6ceKENm7c2O5DAgAQ6vwWcmRkpPLz85WcnNzs9sLCQvXo0UOSlJCQoOrq6vadEACAMGD3\nu4PdLru95d1iY2MlSUeOHNGmTZs0bdo0n4/ncMTIbrd9yzHbj9MZF7S1Owsy8o18/CMj38jHv3DM\nyG8ht8bRo0c1ZcoU5eXlyeFw+Ny3urquPZb8TpzOOLlcx4O2fmdARr6Rj39k5Bv5+BeKGbXmF4w2\nf8ra7Xbrrrvu0r333qtRo0a19eEAAAhLbS7kBQsW6JZbbtEVV1zRHvMAABCW/F6yLikp0cKFC3Xo\n0CHZ7XYVFRUpMzNTqampGjVqlNauXav9+/drzZo1kqRx48YpOzu7wwcHACCU+C3kwYMH66WXXmpx\ne0lJSbsOBABAOOKbugAAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJAB\nADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxA\nIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGAAe7AHAADAJOXl\nhzVlym2aMOEX+tvfXpXHIz300O/0wgvPat++vbr00h8qNzdP69atVUHBCjU2NioxMUn/+Z+PqEeP\nnnrjjXXavPm/1bVrV+3cuUN2u03Lly/ThRde6HNdzpABAPg3NTU1SkhI1KpVherXr5/y8mZq1qzf\n6YUXVumtt4q0e3eJHn98kR5//EkVFLyilJRU/elPz3qP37p1k66//kYVFBQqPX2EXnjhBb9rUsgA\nAPybxsZGZWZmSZIuuKCfLrroYnXv3l3x8d2VmJikhoYzKip6R8nJ50mShg1L1+HDh7zH9+59gQYO\nvEiSNGDAAJWXl/tdk0vWAAD8S9OpUzpz9KhsVpuioqIlSVarVV26xHj3sVqtOnPmjJ599mlt2vSu\nGhsbVVdXp7S0Xt59unaN/cb+NjU2Nvpdm0IGAIQ9T2OjXH8pkHv7h/ryyy/laWrUkYKVct6Y0+z+\nVVVV2rTpXS1fnq/u3bvrtdde0d///l9tmoFCBgCEPddfClTz1j++uuHxSNL/3W5GTU2VevToqe7d\nu6u2tkbr1/9DJ0+ebNMMvIcMAAhrTadOyb39w2a3ubdvl6eZy82JiUmqra1VdvbP9PDDs3TXXb/U\nkSMVWrbs8e88h8Xj+devAgHich0P5HJncTrjgrp+Z0BGvpGPf2TkG/n4F+iMTh85os9n/dZ7ZnwW\nq1W9H12gyOTkNq3hdMb53YczZABAWLPHx8uekND8NkeC7PHxAZmDQgYAhDVrVJRi04c3uy02PV3W\nqKiAzMGHugAAYe/rT1O7t29XQ3WV7I4Exaant/gp645AIQMAwp7FZlNyziQlXT9eDbW1ssfHB+zM\n+GsUMgAA/2KNimrzB7i+89pBWRUAAJyFQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAK\nGQAAA1DIAAAYoFWFvHfvXmVlZWnFihXnbNu8ebPGjx+v7OxsPfnkk+0+IAAA4cBvIdfV1WnOnDnK\nyMhodvujjz6qZcuWadWqVdq0aZNKS0vbfUgAAEKd30KOjIxUfn6+kpv5bs8DBw4oPj5ePXv2lNVq\n1ejRo7Vly5YOGRQAgFDmt5Dtdruio6Ob3eZyuZTwjT/qnJCQIJfL1X7TAQAQJgL+154cjhjZ7bZA\nL+vldMYFbe3Ogox8Ix//yMg38vEvHDNqUyEnJyersrLSe7uioqLZS9vfVF1d15Yl28TpjJPLdTxo\n63cGZOQb+fhHRr6Rj3+hmFFrfsFo0397Sk1Nldvt1sGDB9XQ0KDi4mKNHDmyLQ8JAEBY8nuGXFJS\nooULF+rQoUOy2+0qKipSZmamUlNTNWbMGD388MN64IEHJEnXXHON+vTp0+FDAwAQaiwej8cTyAWD\neRkiFC+DtDcy8o18/CMj38jHv1DMqMMvWQMAgPZBIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAag\nkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAA\nDEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DI\nAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAG\noJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQA\nAAxAIQMAYAAKGQAAA9hbs9O8efO0c+dOWSwW5ebmaujQod5tK1eu1GuvvSar1arBgwdr1qxZHTYs\nAAChyu8Z8nvvvaf9+/dr9erVmjt3rubOnevd5na79dxzz2nlypVatWqVysrKtGPHjg4dGACAUOS3\nkLds2aKsrCxJUt++fVVbWyu32y1JioiIUEREhOrq6tTQ0KCTJ08qPj6+YycGACAE+S3kyspKORwO\n7+2EhAS5XC5JUlRUlO655x5lZWXpqquu0rBhw9SnT5+OmxYAgBDVqveQv8nj8Xh/drvdeuaZZ/Tm\nm28qNjZWt9xyiz7++GMNHDiwxeMdjhjZ7bbvNm07cDrjgrZ2Z0FGvpGPf2TkG/n4F44Z+S3k5ORk\nVVZWem8fOXJETqdTklRWVqa0tDQlJCRIkkaMGKGSkhKfhVxdXdfWmb8zpzNOLtfxoK3fGZCRb+Tj\nHxn5Rj7+hWJGrfkFw+8l65EjR6qoqEiStHv3biUnJys2NlaSlJKSorKyMtXX10uSSkpK1Lt37zaM\nDABAePJ7hjx8+HANGjRIOTk5slgsysvLU2FhoeLi4jRmzBjdcccdmjx5smw2m9LT0zVixIhAzA0A\nQEixeL75pnAABPMyRCheBmlvZOQb+fhHRr6Rj3+hmFG7XLIGAAAdj0IGAMAAFDIAAAagkAEAMACF\nDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBg\nAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAB7sAcAEPpuuOFaLV36\ntFJT0/T223/XnDmz9eabGxQdHa2CghXav3+/rFaLPvzwfVmtVv3whyP1y1/+WjabTePH/z/l5EzS\nG2+sk8vl0vTpM/T++9v0z39uVvfuDj322FJ169ZNJSUfacmSRaqvPymr1app06brkkt+oPLyw5oy\n5TbddNNtWrfuFR07dky/+tV9+tGPrg52LMBZOEMG0OHS07+vkpKPJEk7dmzXgAEXac+eEknSzp07\n5HQ6deRIhV566c/64x9X6KOPtuutt4q8x3/6aZn++MeVuvXWOzRnzmxdddWPtHr1Wnk8TXr33fWS\npEWL5uoXv7hZL7/8V02adIsee2y+9/iamhpZrRa9+OJq/frXDyg//6kAPnugdShkAB1u+PARKinZ\nJUnavXuXxo37qXbt2ilJ2rNnlzZu3KDrrrtedrtdUVHRGjPmJ3rvva3e4y+//EpJ0gUX9FNUVJSG\nDx8hi8WiPn36qrKyUpL0/PMvKzNzjCRp2LB0HT58yHt8Y2OjrrnmOknSgAEDVVHxZYc/Z+Db4pI1\ngA5z5kyj6tynNXRoutasKdCxY8cUERGh4cNH6PHHF2n//s+VnNxDJ064FRfXzXtcXFycqqurvbdj\nYrpKkqxWm7p0ifHeb7Va1djYKEn6+9//S2vWrFZd3Qk1NTXJ4/F497PZbOrSpYv3mKampg593sB3\nQSEDaHdNTU3avL5Mn+2tlPvYKcV2i1LV0WPaunWTBg8eopSUVJWXH9bOnds1YsSl2rVrp2pra73H\nHztWq4SEhFav53Id0aJFc/WHP/xJF144QAcOfKGJE2/oiKcGdBguWQNod5vXl2nX+4fkPnZKkuQ+\ndkrxXdP0/HMvaMiQYZKkXr166/XXX9P3v3+JLrtslF5//VU1Njbq5MmTKip6QxkZo1q9Xk1NtaKj\nu6hXr95qaGjQa6+9Ikmqq6tr/ycHdBAKGUC7OnO6QZ/trTzn/vMS++nAoVINHDhYkjRkyFDt2/eJ\nhgwZpp//PFvJyefp5psn6M47b9Zll12uzMysVq/Zr19/ZWSM1MSJN2jKlNs1cuTlGjRoiKZOvbvd\nnhfQ0Syeb77REgAu1/FALncWpzMuqOt3BmTkG/n4Z7NYtXz++ma3WSzSxLt/oHhHlwBPZQ7+DfkX\nihk5nXF+9+EMGUC7iusWpdhuUc1ui42LUkxsZIAnAjoHChlAu4qItKtP/6Rmt/Xun6SICFuAJwI6\nBz5lDaDdXZbZV5L0+d5KuY+fUmxclHr3T/LeD+BcFDKAdme1WjUq60L9YPQFqnOfVkxsJGfGgB8U\nMoAOExFhC+sPcAHfBu8hAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxA\nIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAVr195DnzZunnTt3ymKxKDc3V0OH\nDvVuKy8v1/33368zZ87o4osv1iOPPNJhwwIAEKr8niG/99572r9/v1avXq25c+dq7ty5Z21fsGCB\nbr/9dq1Zs0Y2m02HDx/usGEBAAhVfgt5y5YtysrKkiT17dtXtbW1crvdkqSmpiZ98MEHyszMlCTl\n5eXp/PPP78BxAQAITX4LubKyUg6Hw3s7ISFBLpdLklRVVaWuXbtq/vz5mjhxohYvXtxxkwIAEMJa\n9R7yN3k8nrN+rqio0OTJk5WSkqK7775bGzZs0JVXXtni8Q5HjOx223catj04nXFBW7uzICPfyMc/\nMvKNfPwLx4z8FnJycrIqKyu9t48cOSKn0ylJcjgcOv/889WrVy9JUkZGhvbt2+ezkKur69o48nfn\ndMbJ5ToetPU7AzLyjXz8IyPfyMe/UMyoNb9g+L1kPXLkSBUVFUmSdu/ereTkZMXGxkqS7Ha70tLS\n9Pnnn3u39+nTpw0jAwAQnvyeIQ8fPlyDBg1STk6OLBaL8vLyVFhYqLi4OI0ZM0a5ubmaMWOGPB6P\n+vfv7/2AFwAAaD2L55tvCgdAMC9DhOJlkPZGRr6Rj39k5Bv5+BeKGbXLJWsAANDxKGQAAAxAIQMA\nYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGCAb/33kE31/PP5Wrdu\nreLj45WZebVeffWvSk//vlJSUnXrrXdKkmbMmKHExPN066136rPPPtXixQtUWVmpyMgI5ebmaeDA\niyVJr75aqNWrV+r06dMaNGiIcnNnKyoqWnPnPqwePXpq166dOnDgC6Wl9dKCBUsUHR0dzKcOAAgB\nIXGG/OmnpVq9+mXl57+g/PwXtWfPLp/7NzU1aebM6Ro79hoVFBRq+vSZmjHjATU0NGjnzu169tmn\ntXTp01qzZp1iY2OVn/+099ji4rf0yCPztXr1WtXU1Ojdd4s7+ukBAMJASBTyRx/tUHr6cCUmJslu\ntysra6zP/ffv/1w1NVW69tqfSpKGDv2eund3qKTkI23a9K5+9KMxSkpySpJ+9rOf6513/q90MzJG\nqVu3eNntdvXt21cVFV923BMDAISNTn/J+nTjaZUf/VIxXbt673M4HD6PcbuPq76+XpMmjffed+LE\nCdXW1ur4cbc2bizWe+9tlSQ1NXnU0HDGu1/Xb6xjtdrU2NjYXk8FABDGOm0hNzY1qrD0dX3k2q19\nrhLVHziuv+x9TTf0u1Y1NTWSJKvVqqamJu8xtbW1Skw8T0lJTnXt2lUvv/zXcx63tHSvxo4dp6lT\n7w3YcwEAoNNesi4sfV0bDv63qk5Vq0taN1V/5tJbn6zXmk/W6c03/yZJSkxMUmnpPknSoUMH9eGH\nH0qSevToKafzPBUXvyVJqqmpUV5erk6ePKlRo0brnXeKVV1dLUnauHGDVqz4U+CfIAAgrHTKM+TT\njaf1kWu393ZMzzglfv987X1qm/bH7tKNP5mgTz8t03XXXa/c3OnKyble/fsP1I9//GNJksVi0e9+\nN0+///085ec/JavVquzsSerSpYsGDBioyZNv069+9R/yeJrkcCTowQdzg/VUAQBhwuLxeDyBXNDl\nOt72x6g7qt9tXSSPzh3dIotujL1GyxY9pjVr1p21zemMa5f1QxkZ+UY+/pGRb+TjXyhm5HTG+d2n\nU16yjo+KkyOqe7PbEqK7KzYiJsATAQDQNp2ykCNtkRrqHNTstiFJg2S3RgR4IgAA2qZTvocsSTf0\nu1aStKtyt6rqa5QQ3V1Dkgbphn7Xyma1nXO5GgAAk3XaQrZZbbqx/3X6ad+xqj11XPFRcYq0RQZ7\nLAAAvpNOW8hfi7RFyhmTGOwxAABok075HjIAAKGGQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxA\nIQMAYAAKGQAAAwT8rz0BAIBzcYYMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAIVfIJ0+e\n1LRp03TTTTfpxhtvVHFx8Vnby8vLNXHiRI0fP16zZ88O0pTB5S+jlStXKjs7WxMnTtTcuXODNGXw\n1dfXKysrS4WFhWfdv3nzZo0fP17Z2dl68skngzSdGVrKaOvWrZowYYJycnI0c+ZMNTU1BWnC4Gop\nn68tXrxYN998c4CnMktLGYXja3XIFXJxcbEGDx6sFStW6IknntCCBQvO2r5gwQLdfvvtWrNmjWw2\nmw4fPhykSYPHV0Zut1vPPfecVq5cqVWrVqmsrEw7duwI4rTB89RTTyk+Pv6c+x999FEtW7ZMq1at\n0qZNm1RaWhqE6czQUkazZ8/W0qVLVVBQoBMnTmjjxo1BmC74WspHkkpLS7Vt27YAT2SeljIKx9dq\ne7AHaG/XXHON9+fy8nKdd9553ttNTU364IMPtGTJEklSXl5ewOczga+MIiIiFBERobq6OsXExOjk\nyZMtvqCEsrKyMpWWlurKK6886/4DBw4oPj5ePXv2lCSNHj1aW7ZsUb9+/YIwZXC1lJEkFRYWKjY2\nVpKUkJCg6urqAE8XfL7ykb4qnPvuu0/Lly8P7GAGaSmjcH2tDrkz5K/l5ORo+vTpys3N9d5XVVWl\nrl27av78+Zo4caIWL14cxAmDr7mMoqKidM899ygrK0tXXXWVhg0bpj59+gRxyuBYuHChZsyYcc79\nLpdLCQkJ3tsJCQlyuVyBHM0YLWUkyVvGR44c0aZNmzR69OhAjmYEX/kUFhbq0ksvVUpKSoCnMktL\nGYXra3XIFnJBQYGeeuopPfjgg/r620E9Ho8qKio0efJkrVixQnv27NGGDRuCO2gQNZeR2+3WM888\nozfffFNvv/22du7cqY8//jjIkwbW2rVr9b3vfU9paWnBHsVYrcno6NGjmjJlivLy8uRwOAI4XfD5\nyqempkaFhYW67bbbgjCZOXxlFK6v1SF3ybqkpESJiYnq2bOnLrroIjU2NqqqqkqJiYlyOBw6//zz\n1atXL0lSRkaG9u3b1+IlpVDlK6OysjKlpaV5zwJHjBihkpISDRw4MMhTB86GDRt04MABbdiwQV9+\n+aUiIyPVo0cPXXbZZUpOTlZlZaV334qKCiUnJwdx2uDwlZH01S92d911l+69916NGjUqyNMGnq98\ntm7dqqqqKk2aNEmnT5/WF198oXnz5p11pSoc+MooXF+rQ66Q33//fR06dEizZs1SZWWl6urqvL+d\n2+12paWl6fPPP1fv3r21e/duXXvttUGeOPB8ZZSSkqKysjLV19crOjpaJSUlYXe58YknnvD+vGzZ\nMqWkpHiLJjU1VW63WwcPHlSPHj1UXFysxx57LFijBo2vjKSv3h+95ZZbdMUVVwRjvKDzlc/YsWM1\nduxYSdLBgwc1c+bMsCtjyXdG4fpaHXJ/7am+vl6zZs1SeXm56uvrNXXqVNXU1CguLk5jxozR/v37\nNWPGDHk8HvXv318PP/ywrNaQvXLfLH8ZFRQUqLCwUDabTenp6frNb34T7JGD5usXCknefLZt2+Yt\n4auvvlp33HFHMEcMun/PaNSoUbrkkkuUnp7u3WfcuHHKzs4O1ohB1dy/oa99XcgvvfRSsMYzQnMZ\nheNrdcgVMgAAnVFo/7oBAEAnQSEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAH+\nF3Klci4DckjLAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "tags": [] } } ] }, { "metadata": { "id": "6667pZttboaj", "colab_type": "code", "outputId": "3cce2cbd-6988-4756-de17-58790f04913a", "colab": { "base_uri": "https://localhost:8080/", "height": 101 } }, "cell_type": "code", "source": [ "# 输出嵌入层的偏差\n", "glove.most_similar(positive=['woman', 'doctor'], negative=['man'], topn=5)" ], "execution_count": 25, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "[('nurse', 0.7735227346420288),\n", " ('physician', 0.7189429998397827),\n", " ('doctors', 0.6824328303337097),\n", " ('patient', 0.6750682592391968),\n", " ('dentist', 0.6726033687591553)]" ] }, "metadata": { "tags": [] }, "execution_count": 25 } ] }, { "metadata": { "id": "LGOkD_DGIq7t", "colab_type": "text" }, "cell_type": "markdown", "source": [ "# 使用嵌入层" ] }, { "metadata": { "id": "tu3Bl-_yw9LA", "colab_type": "text" }, "cell_type": "markdown", "source": [ "下面有几种不同的方法去使用嵌入层:\n", "\n", "1. 使用自己训练的嵌入层（在非监督数据集上训练）\n", "2. 使用预训练的嵌入层（GlioVe，word2vec等）\n", "3. 随机初始化的嵌入层\n", "\n", "一旦你确定使用嵌入层，你可以在使用监督数据训练的时候，选择冻结（freeze）还是继续（这可能会导致过拟合）。在本例中，我们将使用GloVe嵌入层并且在训练的时候冻结它。我们这次的任务是预测给定标题的文章类型。\n" ] }, { "metadata": { "id": "8OlZB0vc1Hbb", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 设置" ] }, { "metadata": { "id": "11CPv1Z-IrD6", "colab_type": "code", "outputId": "72b9ae60-7f57-49d2-81b3-8f2e66468546", "colab": { "base_uri": "https://localhost:8080/", "height": 138 } }, "cell_type": "code", "source": [ "# 读取PyTorch库\n", "!pip3 install torch" ], "execution_count": 26, "outputs": [ { "output_type": "stream", "text": [ "Collecting torch\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/7e/60/66415660aa46b23b5e1b72bc762e816736ce8d7260213e22365af51e8f9c/torch-1.0.0-cp36-cp36m-manylinux1_x86_64.whl (591.8MB)\n", "\u001b[K 100% |████████████████████████████████| 591.8MB 32kB/s \n", "tcmalloc: large alloc 1073750016 bytes == 0x620e2000 @ 0x7fc78e7402a4 0x591a07 0x5b5d56 0x502e9a 0x506859 0x502209 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x507641 0x502209 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x507641 0x504c28 0x502540 0x502f3d 0x507641\n", "\u001b[?25hInstalling collected packages: torch\n", "Successfully installed torch-1.0.0\n" ], "name": "stdout" } ] }, { "metadata": { "id": "T3daO-e9wpXe", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "import os\n", "from argparse import Namespace\n", "import collections\n", "import json\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "import re\n", "import torch" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "Ye64bXPrwpaQ", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 设置Numpy和PyTorch的种子\n", "def set_seeds(seed, cuda):\n", " np.random.seed(seed)\n", " torch.manual_seed(seed)\n", " if cuda:\n", " torch.cuda.manual_seed_all(seed)\n", " \n", "# 创建目录\n", "def create_dirs(dirpath):\n", " if not os.path.exists(dirpath):\n", " os.makedirs(dirpath)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "m5_jiImZ1NAr", "colab_type": "code", "outputId": "5ac66bc2-782b-46b2-ecb2-aee3e51cab39", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "cell_type": "code", "source": [ "# 配置参数\n", "args = Namespace(\n", " seed=1234,\n", " cuda=True,\n", " shuffle=True,\n", " data_file=\"news.csv\",\n", " split_data_file=\"split_news.csv\",\n", " vectorizer_file=\"vectorizer.json\",\n", " model_state_file=\"model.pth\",\n", " save_dir=\"news\",\n", " train_size=0.7,\n", " val_size=0.15,\n", " test_size=0.15,\n", " cutoff=25, # 词条必须出现至少次才能出现在SequenceVocabulary类\n", " num_epochs=5,\n", " early_stopping_criteria=5,\n", " learning_rate=1e-3,\n", " batch_size=64,\n", " num_filters=100,\n", " embedding_dim=100,\n", " hidden_dim=100,\n", " dropout_p=0.1,\n", ")\n", "\n", "# 设置种子\n", "set_seeds(seed=args.seed, cuda=args.cuda)\n", "\n", "# 创建保存目录\n", "create_dirs(args.save_dir)\n", "\n", "# 拓展文件路径\n", "args.vectorizer_file = os.path.join(args.save_dir, args.vectorizer_file)\n", "args.model_state_file = os.path.join(args.save_dir, args.model_state_file)\n", "\n", "# 检查CUDA\n", "if not torch.cuda.is_available():\n", " args.cuda = False\n", "args.device = torch.device(\"cuda\" if args.cuda else \"cpu\")\n", "print(\"Using CUDA: {}\".format(args.cuda))" ], "execution_count": 29, "outputs": [ { "output_type": "stream", "text": [ "Using CUDA: True\n" ], "name": "stdout" } ] }, { "metadata": { "id": "GkNclM3I1XMg", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 数据" ] }, { "metadata": { "id": "vk6ucvgp1NDD", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "import re\n", "import urllib" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "uVQNGEYa1NFu", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 将数据从github上更新到本地notebook\n", "url = \"https://raw.githubusercontent.com/LisonEvf/practicalAI-cn/master/data/news.csv\"\n", "response = urllib.request.urlopen(url)\n", "html = response.read()\n", "with open(args.data_file, 'wb') as fp:\n", " fp.write(html)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "aTU6ENCu1NIc", "colab_type": "code", "outputId": "ff62ad3e-f0b7-4221-d2fb-326ecdfc1588", "colab": { "base_uri": "https://localhost:8080/", "height": 195 } }, "cell_type": "code", "source": [ "# 原始数据\n", "df = pd.read_csv(args.data_file, header=0)\n", "df.head()" ], "execution_count": 32, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
categorytitle
0BusinessWall St. Bears Claw Back Into the Black (Reuters)
1BusinessCarlyle Looks Toward Commercial Aerospace (Reu...
2BusinessOil and Economy Cloud Stocks' Outlook (Reuters)
3BusinessIraq Halts Oil Exports from Main Southern Pipe...
4BusinessOil prices soar to all-time record, posing new...
\n", "
" ], "text/plain": [ " category title\n", "0 Business Wall St. Bears Claw Back Into the Black (Reuters)\n", "1 Business Carlyle Looks Toward Commercial Aerospace (Reu...\n", "2 Business Oil and Economy Cloud Stocks' Outlook (Reuters)\n", "3 Business Iraq Halts Oil Exports from Main Southern Pipe...\n", "4 Business Oil prices soar to all-time record, posing new..." ] }, "metadata": { "tags": [] }, "execution_count": 32 } ] }, { "metadata": { "id": "xIouG0f71NK4", "colab_type": "code", "outputId": "490303dc-f982-47ef-a90a-5277af322093", "colab": { "base_uri": "https://localhost:8080/", "height": 84 } }, "cell_type": "code", "source": [ "# 按类别划分\n", "by_category = collections.defaultdict(list)\n", "for _, row in df.iterrows():\n", " by_category[row.category].append(row.to_dict())\n", "for category in by_category:\n", " print (\"{0}: {1}\".format(category, len(by_category[category])))" ], "execution_count": 33, "outputs": [ { "output_type": "stream", "text": [ "Business: 30000\n", "Sci/Tech: 30000\n", "Sports: 30000\n", "World: 30000\n" ], "name": "stdout" } ] }, { "metadata": { "id": "YZyFpi791NNP", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 创建划分数据集\n", "final_list = []\n", "for _, item_list in sorted(by_category.items()):\n", " if args.shuffle:\n", " np.random.shuffle(item_list)\n", " n = len(item_list)\n", " n_train = int(args.train_size*n)\n", " n_val = int(args.val_size*n)\n", " n_test = int(args.test_size*n)\n", "\n", " # 给数据指定一个划分属性\n", " for item in item_list[:n_train]:\n", " item['split'] = 'train'\n", " for item in item_list[n_train:n_train+n_val]:\n", " item['split'] = 'val'\n", " for item in item_list[n_train+n_val:]:\n", " item['split'] = 'test' \n", "\n", " # 添加到最终列表\n", " final_list.extend(item_list)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "NLrQTVoO1NP4", "colab_type": "code", "outputId": "229b44c3-09e2-4bab-88b9-6a8e57ca6833", "colab": { "base_uri": "https://localhost:8080/", "height": 84 } }, "cell_type": "code", "source": [ "# 将划分数据集转化为pandas格式\n", "split_df = pd.DataFrame(final_list)\n", "split_df[\"split\"].value_counts()" ], "execution_count": 35, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "train 84000\n", "test 18000\n", "val 18000\n", "Name: split, dtype: int64" ] }, "metadata": { "tags": [] }, "execution_count": 35 } ] }, { "metadata": { "id": "v-HSMGwh1NUu", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 数据预处理\n", "def preprocess_text(text):\n", " text = ' '.join(word.lower() for word in text.split(\" \"))\n", " text = re.sub(r\"([.,!?])\", r\" \\1 \", text)\n", " text = re.sub(r\"[^a-zA-Z.,!?]+\", r\" \", text)\n", " return text\n", " \n", "split_df.title = split_df.title.apply(preprocess_text)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "gvNVV7n11NXX", "colab_type": "code", "outputId": "d457902e-527a-4eae-e015-5f8efc63a9ad", "colab": { "base_uri": "https://localhost:8080/", "height": 195 } }, "cell_type": "code", "source": [ "# 保存为CSV格式\n", "split_df.to_csv(args.split_data_file, index=False)\n", "split_df.head()" ], "execution_count": 37, "outputs": [ { "output_type": "execute_result", "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
categorysplittitle
0Businesstraingeneral electric posts higher rd quarter profit
1Businesstrainlilly to eliminate up to us jobs
2Businesstrains amp p lowers america west outlook to negative
3Businesstraindoes rand walk the talk on labor policy ?
4Businesstrainhousekeeper advocates for changes
\n", "
" ], "text/plain": [ " category split title\n", "0 Business train general electric posts higher rd quarter profit\n", "1 Business train lilly to eliminate up to us jobs\n", "2 Business train s amp p lowers america west outlook to negative\n", "3 Business train does rand walk the talk on labor policy ? \n", "4 Business train housekeeper advocates for changes" ] }, "metadata": { "tags": [] }, "execution_count": 37 } ] }, { "metadata": { "id": "cPDWGdlE5vye", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 词汇表" ] }, { "metadata": { "id": "EMR5Y3D95v6P", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "class Vocabulary(object):\n", " def __init__(self, token_to_idx=None):\n", "\n", " # 词条转换为索引\n", " if token_to_idx is None:\n", " token_to_idx = {}\n", " self.token_to_idx = token_to_idx\n", "\n", " # 索引转换为词条\n", " self.idx_to_token = {idx: token \\\n", " for token, idx in self.token_to_idx.items()}\n", "\n", " def to_serializable(self):\n", " return {'token_to_idx': self.token_to_idx}\n", "\n", " @classmethod\n", " def from_serializable(cls, contents):\n", " return cls(**contents)\n", "\n", " def add_token(self, token):\n", " if token in self.token_to_idx:\n", " index = self.token_to_idx[token]\n", " else:\n", " index = len(self.token_to_idx)\n", " self.token_to_idx[token] = index\n", " self.idx_to_token[index] = token\n", " return index\n", "\n", " def add_tokens(self, tokens):\n", " return [self.add_token[token] for token in tokens]\n", "\n", " def lookup_token(self, token):\n", " return self.token_to_idx[token]\n", "\n", " def lookup_index(self, index):\n", " if index not in self.idx_to_token:\n", " raise KeyError(\"the index (%d) is not in the Vocabulary\" % index)\n", " return self.idx_to_token[index]\n", "\n", " def __str__(self):\n", " return \"\" % len(self)\n", "\n", " def __len__(self):\n", " return len(self.token_to_idx)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "Th-KJnOi_74q", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 序列词汇表" ] }, { "metadata": { "id": "WKnfFKY95v81", "colab_type": "code", "outputId": "df7ad1e5-2155-4984-ebbf-0fc4fb1543dc", "colab": { "base_uri": "https://localhost:8080/", "height": 84 } }, "cell_type": "code", "source": [ "# 词汇表实例\n", "category_vocab = Vocabulary()\n", "for index, row in df.iterrows():\n", " category_vocab.add_token(row.category)\n", "print (category_vocab) # __str__\n", "print (len(category_vocab)) # __len__\n", "index = category_vocab.lookup_token(\"Business\")\n", "print (index)\n", "print (category_vocab.lookup_index(index))" ], "execution_count": 39, "outputs": [ { "output_type": "stream", "text": [ "\n", "4\n", "0\n", "Business\n" ], "name": "stdout" } ] }, { "metadata": { "id": "jxO9-6vhBCSO", "colab_type": "text" }, "cell_type": "markdown", "source": [ "下面，我们将为文章标题创建词汇表类，由一系列词条构成。" ] }, { "metadata": { "id": "ip8ViI2v_7Y2", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "from collections import Counter\n", "import string" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "S2LTR5NjBDCG", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "class SequenceVocabulary(Vocabulary):\n", " def __init__(self, token_to_idx=None, unk_token=\"\",\n", " mask_token=\"\", begin_seq_token=\"\",\n", " end_seq_token=\"\"):\n", "\n", " super(SequenceVocabulary, self).__init__(token_to_idx)\n", "\n", " self.mask_token = mask_token\n", " self.unk_token = unk_token\n", " self.begin_seq_token = begin_seq_token\n", " self.end_seq_token = end_seq_token\n", "\n", " self.mask_index = self.add_token(self.mask_token)\n", " self.unk_index = self.add_token(self.unk_token)\n", " self.begin_seq_index = self.add_token(self.begin_seq_token)\n", " self.end_seq_index = self.add_token(self.end_seq_token)\n", " \n", " # 索引转换为词条\n", " self.idx_to_token = {idx: token \\\n", " for token, idx in self.token_to_idx.items()}\n", "\n", " def to_serializable(self):\n", " contents = super(SequenceVocabulary, self).to_serializable()\n", " contents.update({'unk_token': self.unk_token,\n", " 'mask_token': self.mask_token,\n", " 'begin_seq_token': self.begin_seq_token,\n", " 'end_seq_token': self.end_seq_token})\n", " return contents\n", "\n", " def lookup_token(self, token):\n", " return self.token_to_idx.get(token, self.unk_index)\n", " \n", " def lookup_index(self, index):\n", " if index not in self.idx_to_token:\n", " raise KeyError(\"the index (%d) is not in the SequenceVocabulary\" % index)\n", " return self.idx_to_token[index]\n", " \n", " def __str__(self):\n", " return \"\" % len(self.token_to_idx)\n", "\n", " def __len__(self):\n", " return len(self.token_to_idx)\n" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "DvDPVDbsBDE4", "colab_type": "code", "outputId": "50ec6b81-e2ab-494f-f504-808c92bb2228", "colab": { "base_uri": "https://localhost:8080/", "height": 84 } }, "cell_type": "code", "source": [ "# 得到单词长度\n", "word_counts = Counter()\n", "for title in split_df.title:\n", " for token in title.split(\" \"):\n", " if token not in string.punctuation:\n", " word_counts[token] += 1\n", "\n", "# 创建SequenceVocabulary实例\n", "title_vocab = SequenceVocabulary()\n", "for word, word_count in word_counts.items():\n", " if word_count >= args.cutoff:\n", " title_vocab.add_token(word)\n", "print (title_vocab) # __str__\n", "print (len(title_vocab)) # __len__\n", "index = title_vocab.lookup_token(\"general\")\n", "print (index)\n", "print (title_vocab.lookup_index(index))" ], "execution_count": 42, "outputs": [ { "output_type": "stream", "text": [ "\n", "4400\n", "4\n", "general\n" ], "name": "stdout" } ] }, { "metadata": { "id": "6TuUHvlI6JGh", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 向量化" ] }, { "metadata": { "id": "s8IsOPHT5v_i", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "class NewsVectorizer(object):\n", " def __init__(self, title_vocab, category_vocab):\n", " self.title_vocab = title_vocab\n", " self.category_vocab = category_vocab\n", "\n", " def vectorize(self, title):\n", " indices = [self.title_vocab.lookup_token(token) for token in title.split(\" \")]\n", " indices = [self.title_vocab.begin_seq_index] + indices + \\\n", " [self.title_vocab.end_seq_index]\n", " \n", " # 创建向量\n", " title_length = len(indices)\n", " vector = np.zeros(title_length, dtype=np.int64)\n", " vector[:len(indices)] = indices\n", "\n", " return vector\n", " \n", " def unvectorize(self, vector):\n", " tokens = [self.title_vocab.lookup_index(index) for index in vector]\n", " title = \" \".join(token for token in tokens)\n", " return title\n", "\n", " @classmethod\n", " def from_dataframe(cls, df, cutoff):\n", " \n", " # 创建类别词表\n", " category_vocab = Vocabulary() \n", " for category in sorted(set(df.category)):\n", " category_vocab.add_token(category)\n", "\n", " # 获取词长度\n", " word_counts = Counter()\n", " for title in df.title:\n", " for token in title.split(\" \"):\n", " word_counts[token] += 1\n", " \n", " # 创建标题词表\n", " title_vocab = SequenceVocabulary()\n", " for word, word_count in word_counts.items():\n", " if word_count >= cutoff:\n", " title_vocab.add_token(word)\n", " \n", " return cls(title_vocab, category_vocab)\n", "\n", " @classmethod\n", " def from_serializable(cls, contents):\n", " title_vocab = SequenceVocabulary.from_serializable(contents['title_vocab'])\n", " category_vocab = Vocabulary.from_serializable(contents['category_vocab'])\n", " return cls(title_vocab=title_vocab, category_vocab=category_vocab)\n", " \n", " def to_serializable(self):\n", " return {'title_vocab': self.title_vocab.to_serializable(),\n", " 'category_vocab': self.category_vocab.to_serializable()}" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "p2blo9sT5wCS", "colab_type": "code", "outputId": "d152e0d8-5eb9-49dc-d563-0a8e77bf6409", "colab": { "base_uri": "https://localhost:8080/", "height": 101 } }, "cell_type": "code", "source": [ "# 向量化实例\n", "vectorizer = NewsVectorizer.from_dataframe(split_df, cutoff=args.cutoff)\n", "print (vectorizer.title_vocab)\n", "print (vectorizer.category_vocab)\n", "vectorized_title = vectorizer.vectorize(preprocess_text(\n", " \"Roger Federer wins the Wimbledon tennis tournament.\"))\n", "print (np.shape(vectorized_title))\n", "print (vectorized_title)\n", "print (vectorizer.unvectorize(vectorized_title))" ], "execution_count": 44, "outputs": [ { "output_type": "stream", "text": [ "\n", "\n", "(11,)\n", "[ 2 1 4152 1232 25 1 2393 4077 39 31 3]\n", " federer wins the tennis tournament . \n" ], "name": "stdout" } ] }, { "metadata": { "id": "jTpYV1FkHBTi", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 数据集" ] }, { "metadata": { "id": "_PcKZGbYIVA7", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "from torch.utils.data import Dataset, DataLoader" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "Lrssjuxb1NZy", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "class NewsDataset(Dataset):\n", " def __init__(self, df, vectorizer):\n", " self.df = df\n", " self.vectorizer = vectorizer\n", " \n", " # 最大的标题长度\n", " get_length = lambda title: len(title.split(\" \"))\n", " self.max_seq_length = max(map(get_length, df.title)) + 2 # ( + )\n", "\n", " # 数据划分\n", " self.train_df = self.df[self.df.split=='train']\n", " self.train_size = len(self.train_df)\n", " self.val_df = self.df[self.df.split=='val']\n", " self.val_size = len(self.val_df)\n", " self.test_df = self.df[self.df.split=='test']\n", " self.test_size = len(self.test_df)\n", " self.lookup_dict = {'train': (self.train_df, self.train_size), \n", " 'val': (self.val_df, self.val_size),\n", " 'test': (self.test_df, self.test_size)}\n", " self.set_split('train')\n", "\n", " # 类权重（用于样本失衡）\n", " class_counts = df.category.value_counts().to_dict()\n", " def sort_key(item):\n", " return self.vectorizer.category_vocab.lookup_token(item[0])\n", " sorted_counts = sorted(class_counts.items(), key=sort_key)\n", " frequencies = [count for _, count in sorted_counts]\n", " self.class_weights = 1.0 / torch.tensor(frequencies, dtype=torch.float32)\n", "\n", " @classmethod\n", " def load_dataset_and_make_vectorizer(cls, split_data_file, cutoff):\n", " df = pd.read_csv(split_data_file, header=0)\n", " train_df = df[df.split=='train']\n", " return cls(df, NewsVectorizer.from_dataframe(train_df, cutoff))\n", "\n", " @classmethod\n", " def load_dataset_and_load_vectorizer(cls, split_data_file, vectorizer_filepath):\n", " df = pd.read_csv(split_data_file, header=0)\n", " vectorizer = cls.load_vectorizer_only(vectorizer_filepath)\n", " return cls(df, vectorizer)\n", "\n", " def load_vectorizer_only(vectorizer_filepath):\n", " with open(vectorizer_filepath) as fp:\n", " return NewsVectorizer.from_serializable(json.load(fp))\n", "\n", " def save_vectorizer(self, vectorizer_filepath):\n", " with open(vectorizer_filepath, \"w\") as fp:\n", " json.dump(self.vectorizer.to_serializable(), fp)\n", "\n", " def set_split(self, split=\"train\"):\n", " self.target_split = split\n", " self.target_df, self.target_size = self.lookup_dict[split]\n", "\n", " def __str__(self):\n", " return \" software firm to cut jobs \n", "tensor([3.3333e-05, 3.3333e-05, 3.3333e-05, 3.3333e-05])\n" ], "name": "stdout" } ] }, { "metadata": { "id": "IxjCMr_DINGk", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 模型" ] }, { "metadata": { "id": "202KnN2lLj61", "colab_type": "text" }, "cell_type": "markdown", "source": [ "input（输入层） → embedding（嵌入层） → conv （卷积层）→ FC （全连接层）\n", "我们将使用一维的卷积层计算 ([nn.Conv1D](https://pytorch.org/docs/stable/nn.html#torch.nn.Conv1d))，尽管我们的输入是各个单词，但我们并不是使用字符级来表示它们，我们采用的输入层形状是shape $\\in \\mathbb{R}^{NXSXE}$\n", "* 其中:\n", " * N = batchsize（每个批处理的训练样本数）\n", " * S = max sentence length （最大句子长度）\n", " * E = embedding dim at a word level（词嵌入维度）" ] }, { "metadata": { "id": "j-Xgp0F3INRA", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "import torch.nn as nn\n", "import torch.nn.functional as F" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "Lfw7csveHApA", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "class NewsModel(nn.Module):\n", " def __init__(self, embedding_dim, num_embeddings, num_input_channels, \n", " num_channels, hidden_dim, num_classes, dropout_p, \n", " pretrained_embeddings=None, freeze_embeddings=False,\n", " padding_idx=0):\n", " super(NewsModel, self).__init__()\n", " \n", " if pretrained_embeddings is None:\n", " self.embeddings = nn.Embedding(embedding_dim=embedding_dim,\n", " num_embeddings=num_embeddings,\n", " padding_idx=padding_idx)\n", " else:\n", " pretrained_embeddings = torch.from_numpy(pretrained_embeddings).float()\n", " self.embeddings = nn.Embedding(embedding_dim=embedding_dim,\n", " num_embeddings=num_embeddings,\n", " padding_idx=padding_idx,\n", " _weight=pretrained_embeddings)\n", " \n", " # 卷积层权重\n", " self.conv = nn.ModuleList([nn.Conv1d(num_input_channels, num_channels, \n", " kernel_size=f) for f in [2,3,4]])\n", " \n", " # 全连接层权重\n", " self.dropout = nn.Dropout(dropout_p)\n", " self.fc1 = nn.Linear(num_channels*3, hidden_dim)\n", " self.fc2 = nn.Linear(hidden_dim, num_classes)\n", " \n", " if freeze_embeddings:\n", " self.embeddings.weight.requires_grad = False\n", "\n", " def forward(self, x_in, channel_first=False, apply_softmax=False):\n", " \n", " # 嵌入\n", " x_in = self.embeddings(x_in)\n", "\n", " # 重新调整输入层，使得通道数num_channels为一维（N,C,L）\n", " if not channel_first:\n", " x_in = x_in.transpose(1, 2)\n", " \n", " # 卷积层输出\n", " z1 = self.conv[0](x_in)\n", " z1 = F.max_pool1d(z1, z1.size(2)).squeeze(2)\n", " z2 = self.conv[1](x_in)\n", " z2 = F.max_pool1d(z2, z2.size(2)).squeeze(2)\n", " z3 = self.conv[2](x_in)\n", " z3 = F.max_pool1d(z3, z3.size(2)).squeeze(2)\n", " \n", " # 连接卷积层的输出\n", " z = torch.cat([z1, z2, z3], 1)\n", "\n", " # 全连接层\n", " z = self.dropout(z)\n", " z = self.fc1(z)\n", " y_pred = self.fc2(z)\n", " \n", " if apply_softmax:\n", " y_pred = F.softmax(y_pred, dim=1)\n", " return y_pred" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "d7_4SiLkKji8", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 训练" ] }, { "metadata": { "id": "UQAIfKN2HAtN", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "import torch.optim as optim" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "RwgZASWYHAwB", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "class Trainer(object):\n", " def __init__(self, dataset, model, model_state_file, save_dir, device, shuffle, \n", " num_epochs, batch_size, learning_rate, early_stopping_criteria):\n", " self.dataset = dataset\n", " self.class_weights = dataset.class_weights.to(device)\n", " self.model = model.to(device)\n", " self.save_dir = save_dir\n", " self.device = device\n", " self.shuffle = shuffle\n", " self.num_epochs = num_epochs\n", " self.batch_size = batch_size\n", " self.loss_func = nn.CrossEntropyLoss(self.class_weights)\n", " self.optimizer = optim.Adam(self.model.parameters(), lr=learning_rate)\n", " self.scheduler = optim.lr_scheduler.ReduceLROnPlateau(\n", " optimizer=self.optimizer, mode='min', factor=0.5, patience=1)\n", " self.train_state = {\n", " 'stop_early': False, \n", " 'early_stopping_step': 0,\n", " 'early_stopping_best_val': 1e8,\n", " 'early_stopping_criteria': early_stopping_criteria,\n", " 'learning_rate': learning_rate,\n", " 'epoch_index': 0,\n", " 'train_loss': [],\n", " 'train_acc': [],\n", " 'val_loss': [],\n", " 'val_acc': [],\n", " 'test_loss': -1,\n", " 'test_acc': -1,\n", " 'model_filename': model_state_file}\n", " \n", " def update_train_state(self):\n", "\n", " # 输出详细信息\n", " print (\"[EPOCH]: {0:02d} | [LR]: {1} | [TRAIN LOSS]: {2:.2f} | [TRAIN ACC]: {3:.1f}% | [VAL LOSS]: {4:.2f} | [VAL ACC]: {5:.1f}%\".format(\n", " self.train_state['epoch_index'], self.train_state['learning_rate'], \n", " self.train_state['train_loss'][-1], self.train_state['train_acc'][-1], \n", " self.train_state['val_loss'][-1], self.train_state['val_acc'][-1]))\n", "\n", " # 保存至少一个模型\n", " if self.train_state['epoch_index'] == 0:\n", " torch.save(self.model.state_dict(), self.train_state['model_filename'])\n", " self.train_state['stop_early'] = False\n", "\n", " # 如果表现提高的话，再次保存模型\n", " elif self.train_state['epoch_index'] >= 1:\n", " loss_tm1, loss_t = self.train_state['val_loss'][-2:]\n", "\n", " # 如果损失无法收敛\n", " if loss_t >= self.train_state['early_stopping_best_val']:\n", " # Update step\n", " self.train_state['early_stopping_step'] += 1\n", "\n", " # 如果损失下降\n", " else:\n", " # 保存最佳模型\n", " if loss_t < self.train_state['early_stopping_best_val']:\n", " torch.save(self.model.state_dict(), self.train_state['model_filename'])\n", "\n", " # 重置early stopping step为0\n", " self.train_state['early_stopping_step'] = 0\n", "\n", " # 提前停止\n", " self.train_state['stop_early'] = self.train_state['early_stopping_step'] \\\n", " >= self.train_state['early_stopping_criteria']\n", " return self.train_state\n", " \n", " def compute_accuracy(self, y_pred, y_target):\n", " _, y_pred_indices = y_pred.max(dim=1)\n", " n_correct = torch.eq(y_pred_indices, y_target).sum().item()\n", " return n_correct / len(y_pred_indices) * 100\n", " \n", " def pad_seq(self, seq, length):\n", " vector = np.zeros(length, dtype=np.int64)\n", " vector[:len(seq)] = seq\n", " vector[len(seq):] = self.dataset.vectorizer.title_vocab.mask_index\n", " return vector\n", " \n", " def collate_fn(self, batch):\n", " \n", " # 深拷贝\n", " batch_copy = copy.deepcopy(batch)\n", " processed_batch = {\"title\": [], \"category\": []}\n", " \n", " # 得到最长的序列长度\n", " max_seq_len = max([len(sample[\"title\"]) for sample in batch_copy])\n", " \n", " # 数据填充\n", " for i, sample in enumerate(batch_copy):\n", " seq = sample[\"title\"]\n", " category = sample[\"category\"]\n", " padded_seq = self.pad_seq(seq, max_seq_len)\n", " processed_batch[\"title\"].append(padded_seq)\n", " processed_batch[\"category\"].append(category)\n", " \n", " # 转换为合适的张量数据类型\n", " processed_batch[\"title\"] = torch.LongTensor(\n", " processed_batch[\"title\"])\n", " processed_batch[\"category\"] = torch.LongTensor(\n", " processed_batch[\"category\"])\n", " \n", " return processed_batch \n", " \n", " def run_train_loop(self):\n", " for epoch_index in range(self.num_epochs):\n", " self.train_state['epoch_index'] = epoch_index\n", " \n", " # 对训练集数据进行迭代\n", "\n", " # 初始化批处理生成器，设置loss和acc为0，设置为训练模式\n", " self.dataset.set_split('train')\n", " batch_generator = self.dataset.generate_batches(\n", " batch_size=self.batch_size, collate_fn=self.collate_fn, \n", " shuffle=self.shuffle, device=self.device)\n", " running_loss = 0.0\n", " running_acc = 0.0\n", " self.model.train()\n", "\n", " for batch_index, batch_dict in enumerate(batch_generator):\n", " # 梯度置零\n", " self.optimizer.zero_grad()\n", "\n", " # 计算输出\n", " y_pred = self.model(batch_dict['title'])\n", "\n", " # 计算损失loss\n", " loss = self.loss_func(y_pred, batch_dict['category'])\n", " loss_t = loss.item()\n", " running_loss += (loss_t - running_loss) / (batch_index + 1)\n", "\n", " # 使用损失函数计算梯度\n", " loss.backward()\n", "\n", " # 使用优化器进行梯度计算\n", " self.optimizer.step()\n", " \n", " # 计算准确率accuracy\n", " acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n", " running_acc += (acc_t - running_acc) / (batch_index + 1)\n", "\n", " self.train_state['train_loss'].append(running_loss)\n", " self.train_state['train_acc'].append(running_acc)\n", "\n", " # 对验证集数据进行迭代\n", "\n", " # 初始化批处理生成器，设置loss和acc为0，设置为验证模式\n", " self.dataset.set_split('val')\n", " batch_generator = self.dataset.generate_batches(\n", " batch_size=self.batch_size, collate_fn=self.collate_fn, \n", " shuffle=self.shuffle, device=self.device)\n", " running_loss = 0.\n", " running_acc = 0.\n", " self.model.eval()\n", "\n", " for batch_index, batch_dict in enumerate(batch_generator):\n", "\n", " # 计算输出\n", " y_pred = self.model(batch_dict['title'])\n", "\n", " # 计算损失loss\n", " loss = self.loss_func(y_pred, batch_dict['category'])\n", " loss_t = loss.to(\"cpu\").item()\n", " running_loss += (loss_t - running_loss) / (batch_index + 1)\n", "\n", " # 计算准确率accuracy\n", " acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n", " running_acc += (acc_t - running_acc) / (batch_index + 1)\n", "\n", " self.train_state['val_loss'].append(running_loss)\n", " self.train_state['val_acc'].append(running_acc)\n", "\n", " self.train_state = self.update_train_state()\n", " self.scheduler.step(self.train_state['val_loss'][-1])\n", " if self.train_state['stop_early']:\n", " break\n", " \n", " def run_test_loop(self):\n", " # 初始化批处理生成器，设置loss和acc为0，设置为验证模式\n", " self.dataset.set_split('test')\n", " batch_generator = self.dataset.generate_batches(\n", " batch_size=self.batch_size, collate_fn=self.collate_fn, \n", " shuffle=self.shuffle, device=self.device)\n", " running_loss = 0.0\n", " running_acc = 0.0\n", " self.model.eval()\n", "\n", " for batch_index, batch_dict in enumerate(batch_generator):\n", " # 计算输出\n", " y_pred = self.model(batch_dict['title'])\n", "\n", " # 计算损失loss\n", " loss = self.loss_func(y_pred, batch_dict['category'])\n", " loss_t = loss.item()\n", " running_loss += (loss_t - running_loss) / (batch_index + 1)\n", "\n", " # 计算准确率accuracy\n", " acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n", " running_acc += (acc_t - running_acc) / (batch_index + 1)\n", "\n", " self.train_state['test_loss'] = running_loss\n", " self.train_state['test_acc'] = running_acc\n", " \n", " def plot_performance(self):\n", " # 图片大小\n", " plt.figure(figsize=(15,5))\n", "\n", " # 绘制损失loss\n", " plt.subplot(1, 2, 1)\n", " plt.title(\"Loss\")\n", " plt.plot(trainer.train_state[\"train_loss\"], label=\"train\")\n", " plt.plot(trainer.train_state[\"val_loss\"], label=\"val\")\n", " plt.legend(loc='upper right')\n", "\n", " # 绘制准确率accuracy\n", " plt.subplot(1, 2, 2)\n", " plt.title(\"Accuracy\")\n", " plt.plot(trainer.train_state[\"train_acc\"], label=\"train\")\n", " plt.plot(trainer.train_state[\"val_acc\"], label=\"val\")\n", " plt.legend(loc='lower right')\n", "\n", " # 保存图片\n", " plt.savefig(os.path.join(self.save_dir, \"performance.png\"))\n", "\n", " # 展示图片\n", " plt.show()\n", " \n", " def save_train_state(self):\n", " with open(os.path.join(self.save_dir, \"train_state.json\"), \"w\") as fp:\n", " json.dump(self.train_state, fp)" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "LJfKGc9cHA0y", "colab_type": "code", "outputId": "f665bf35-27ae-4110-d42e-582477df1756", "colab": { "base_uri": "https://localhost:8080/", "height": 202 } }, "cell_type": "code", "source": [ "# 初始化\n", "dataset = NewsDataset.load_dataset_and_make_vectorizer(args.split_data_file, \n", " cutoff=args.cutoff)\n", "dataset.save_vectorizer(args.vectorizer_file)\n", "vectorizer = dataset.vectorizer\n", "model = NewsModel(embedding_dim=args.embedding_dim, \n", " num_embeddings=len(vectorizer.title_vocab), \n", " num_input_channels=args.embedding_dim, \n", " num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n", " num_classes=len(vectorizer.category_vocab), \n", " dropout_p=args.dropout_p, pretrained_embeddings=None, \n", " padding_idx=vectorizer.title_vocab.mask_index)\n", "print (model.named_modules)" ], "execution_count": 52, "outputs": [ { "output_type": "stream", "text": [ "\n" ], "name": "stdout" } ] }, { "metadata": { "id": "wDxPyGIVKoUK", "colab_type": "code", "outputId": "e6ea87e9-0df8-4b11-ce50-f6124c46b215", "colab": { "base_uri": "https://localhost:8080/", "height": 101 } }, "cell_type": "code", "source": [ "# 训练\n", "trainer = Trainer(dataset=dataset, model=model, \n", " model_state_file=args.model_state_file, \n", " save_dir=args.save_dir, device=args.device,\n", " shuffle=args.shuffle, num_epochs=args.num_epochs, \n", " batch_size=args.batch_size, learning_rate=args.learning_rate, \n", " early_stopping_criteria=args.early_stopping_criteria)\n", "trainer.run_train_loop()" ], "execution_count": 53, "outputs": [ { "output_type": "stream", "text": [ "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 0.79 | [TRAIN ACC]: 69.2% | [VAL LOSS]: 0.59 | [VAL ACC]: 78.5%\n", "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 0.50 | [TRAIN ACC]: 81.7% | [VAL LOSS]: 0.51 | [VAL ACC]: 81.7%\n", "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 0.42 | [TRAIN ACC]: 84.7% | [VAL LOSS]: 0.48 | [VAL ACC]: 82.7%\n", "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 0.36 | [TRAIN ACC]: 86.7% | [VAL LOSS]: 0.51 | [VAL ACC]: 82.5%\n", "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 0.32 | [TRAIN ACC]: 88.2% | [VAL LOSS]: 0.50 | [VAL ACC]: 82.5%\n" ], "name": "stdout" } ] }, { "metadata": { "id": "V9Ruxpc8WT17", "colab_type": "code", "outputId": "d66159a8-0809-4856-ddd7-347b9118b71b", "colab": { "base_uri": "https://localhost:8080/", "height": 335 } }, "cell_type": "code", "source": [ "# 性能展示\n", "trainer.plot_performance()" ], "execution_count": 54, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2gAAAE+CAYAAAD4XjP+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlclPe99//XzLDvwyo7CCgRBTVo\n3EUQMWq2ZrUnieck/fVu7vT0vu+T5jY3XUxysrS/u/Y06TlJl5OTbmlim6A2MVEjgrviStyDqICA\n7IvsMDP3HxjUqnEDhsH38xEfmbmu+V7XewZl5jPXdzHYbDYbIiIiIiIiYndGewcQERERERGRXirQ\nREREREREhggVaCIiIiIiIkOECjQREREREZEhQgWaiIiIiIjIEKECTUREREREZIhQgSZyk0aPHs3Z\ns2ftHUNERGRQPPbYY9x77732jiEy7KlAExEREZGv9eWXX+Lt7U1YWBj79++3dxyRYU0Fmkg/6+zs\n5Mc//jFZWVncfffd/OQnP8FisQDwpz/9ibvvvpv58+fz0EMPUVRU9LXbRUREhoKVK1cyf/58Fi1a\nxKpVq/q2r1q1iqysLLKysnj++efp6uq66vZdu3aRmZnZ1/bi+7/85S/54Q9/yEMPPcTvfvc7rFYr\nL730EllZWaSnp/P888/T3d0NQH19Pd/5znfIyMjgnnvuYevWreTn57No0aJLMn/jG99gw4YNA/3S\niPQ7J3sHEBlufv/733P27FnWrFlDT08Pjz/+OJ988gkZGRm88cYb5OXl4eXlxWeffUZ+fj6hoaFX\n3J6QkGDvpyIiIoLFYuHzzz/n2WefxWQysXz5crq6uqiuruanP/0pq1atIjg4mH/+53/mD3/4A/Pn\nz7/i9nHjxn3teTZt2sTq1avx9/dn3bp17Nmzh08++QSr1coDDzzAp59+yn333cfy5cuJi4vjV7/6\nFUeOHOGf/umf2LJlCzU1NRw7dozExEQqKiooLS1l1qxZg/QqifQfFWgi/Sw/P5+nnnoKJycnnJyc\nuOeee9i2bRsLFizAYDDw4YcfsmjRIu6++24Auru7r7hdRERkKNi6dSvjxo3Dy8sLgMmTJ5OXl0dj\nYyMTJkwgJCQEgOXLl2Mymfjoo4+uuH3v3r1fe56UlBT8/f0ByMrKYs6cOTg7OwMwbtw4ysrKgN5C\n7re//S0AY8aMITc3FxcXF7KyslizZg2JiYls2LCBjIwMXFxc+v8FERlg6uIo0s/q6+vx9fXtu+/r\n60tdXR3Ozs787ne/Y9++fWRlZfHNb36T48ePX3W7iIjIUJCTk0N+fj6pqamkpqayfv16Vq5cSUND\nAz4+Pn2Pc3V1xcnJ6arbr+Xi9876+nqWLl1KVlYW8+fPJzc3F5vNBkBjYyPe3t59j/2qcFy4cCFr\n1qwBYMOGDSxYsODWnriInahAE+lngYGBNDY29t1vbGwkMDAQ6P2m780332THjh3MmDGDZcuWfe12\nERERe2pqaqKgoIBdu3axZ88e9uzZw+7duzl48CBGo5GGhoa+x7a0tFBbW4vZbL7idpPJ1DcmG6C5\nufmq5/23f/s3nJyc+Pjjj1m7di2zZ8/u2+fn53fJ8c+cOUN3dzeTJk2ip6eHvLw8ioqKmDZtWn+9\nDCKDSgWaSD9LS0vjww8/xGKx0NbWxurVq5k9ezbHjx/ne9/7Hl1dXbi4uDB27FgMBsNVt4uIiNjb\nmjVrmDJlyiVdBZ2cnJgxYwZdXV3s27ePM2fOYLPZWLZsGR9++CGzZ8++4vagoCBqamqoq6vDYrHw\n8ccfX/W8dXV1jBo1ChcXF44dO8b+/ftpa2sDID09nZUrVwJw4sQJvvGNb2CxWDAajSxYsIB//dd/\nJT09va97pIij0Rg0kVvwxBNPYDKZ+u6/8sorPPHEE5SVlbFw4UIMBgPz58/vG1cWERHBokWLcHZ2\nxtPTkx//+MeMGjXqittFRETsbdWqVSxZsuSy7ZmZmbz11lu8/PLLLFmyBJPJxLhx4/inf/onXF1d\nr7r9wQcf5P777ycsLIz77ruPo0ePXvG8Tz31FEuXLiUnJ4fU1FSWLl3KD37wA5KTk3n++edZunQp\n6enpeHp68rOf/Qw3Nzegt5vju+++q+6N4tAMtq869IqIiIiIOLDa2loeeOAB8vPzL/kCVcSRqIuj\niIiIiAwLb775JosXL1ZxJg5NBZqIiIiIOLTa2loyMjKora3lqaeesncckVuiLo4iIiIiIiJDhK6g\niYiIiIiIDBEq0ERERERERIaIQZ9mv6bm3C0fw2z2oKGhrR/SDA5HyqusA8ORsoJj5VXWgdFfWYOC\nvPshze3jdnuPdKSs4Fh5lXVgKOvAcaS8/ZH1694fr6tAe+211ygsLMRgMJCdnU1ycnLfvvfee4+/\n/e1vGI1Gxo4dyw9+8INbCns9nJwca2YeR8qrrAPDkbKCY+VV1oHhSFnlUo70s3OkrOBYeZV1YCjr\nwHGkvAOd9ZpdHAsKCigpKWHFihW8+uqrvPrqq337WlpaeOedd3jvvfd4//33KS4u5sCBAwMaWERE\nREREZLi6ZoG2Y8cO5s6dC0BcXBxNTU20tLQA4OzsjLOzM21tbfT09NDe3o6vr+/AJhYRERERERmm\nrlmg1dbWYjab++77+/tTU1MDgKurK88++yxz585lzpw5pKSkEBsbO3BpRUREREREhrEbniTk4mXT\nWlpa+PWvf83atWvx8vJiyZIlHDt2jMTExKu2N5s9+qXfpqMNPHekvMo6MBwpKzhWXmUdGI6UVURE\nZLi4ZoEWHBxMbW1t3/3q6mqCgoIAKC4uJjIyEn9/fwBSU1M5dOjQ1xZo/TUrWH/MdDVYHCmvsg4M\nR8oKjpVXWQdGf2VVkSciInJjrtnFcfr06axbtw6Aw4cPExwcjJeXFwDh4eEUFxfT0dEBwKFDh4iJ\niRm4tCIiIiIiIsPYNa+gTZw4kaSkJB577DEMBgPLli0jJycHb29vMjMzefrpp3nyyScxmUxMmDCB\n1NTUwcgtIiIiIiIy7FzXGLTvf//7l9y/uAvjY489xmOPPda/qUREZEDk5+eSlpZxzce9+uqrLFr0\nIGFh4YOQSkRERL5yzS6OIiIyPFRWVrBhw7rreuwPfvADFWciIiJ2cMOzONpbZ5eFj7ecZPxIM24u\nDhdfRMRufv7zn3L06GFmzpzEvHl3U1lZwS9+8Ravv/4yNTXVtLe389RT32b69Jk88cQTfPe7/0Je\nXi6trS2UlpZQXn6G733vOaZOnW7vpyIiIjKounssnD57jlOV55g0NhSz+8DVIQ5X4Rwva+A3qw4y\nZ0I4T2SNtnccERGHsXjxE+Tk/IXY2DhKS0/z1lv/SUNDPZMnT+HuuxdRXn6GH/3oBaZPn3lJu+rq\nKn72szfZuXM7q1d/pAJNRESGvYZznRSXN3Hi/J+Ss+ewWHuXGyutaeFbC+4YsHM7XIE2JsafiGAv\n8g+UkzYhnMhgL3tHEhG5YX/ZeILdx6r79ZiTEoN5JD3+uh57xx1JAHh7+3D06GH+9rccDAYjzc1N\nlz02OXk80LvsSktLS/8FFhERGQJ6LFbKqls4Ud5E8fk/dc2dfftNRgNRIV7EhfkSH+HLnMnRtJ7r\nGLA8DlegOZmMfOu+sbz42528v+FLnl88AYPBYO9YIiIOxdnZGYDPP19Lc3Mz//Ef/0lzczPf+tYT\nlz3WZDL13bbZbIOWUUREZCA0t3WdL8SaOVHexOnKZrp6rH37vT2cGR8fSHyEL3FhPsSE+uDqfOG9\n0MPNWQXa37szMYSUuAAKi+vYe7yG1MRge0cSEbkhj6THX/fVrv5iNBqxWCyXbGtsbCQ0NAyj0cim\nTRvp7u4e1EwiIiIDyWq1UV7b2nd17ER5E9UN7X37DUB4kFdfMRYf4Uuwn7tdLwA5ZIEG8GhGAodO\n1fOXvBMkxwXgclFVKyIil4uOjuX48WOEhobh5+cHQFpaOi+88C8cOXKIhQvvJTg4mHff/a2dk4qI\niNycto5uTlY0940dO1nRTEfXhS8n3V2dGDvSn/gwX+IifBkZ6oO769AqiYZWmhswwt+DzNRI1haU\nsm53GfdMi7F3JBGRIc1sNpOTs+aSbaGhYfz+9x/03Z83724AgoK8qak5x8iRF67yjRwZz7//+28G\nJ6yIiMg12Gw2zta3XTR2rJmK2lYu7owfGuDRN3YsLtyX0AAPjEN8eJTDFmgAi6bFsP1QJWt2nGbG\nuFDM3q72jiQiIsNQa2srS5cupampie7ubp599ll+85sLxWp1dTUPPPAA3/nOd/q2/fKXv+Tjjz8m\nJCQEgHvvvZeHH3540LOLiAwXHV09nKo819dVsbi8idaOnr79rs4mRkf5ne+u2FuQebk72zHxzXHo\nAs3DzYkHZ8fx7mfH+Gv+Cb59T5K9I4mIyDC0cuVKYmNjee6556iqqmLJkiWsXbu2b/+3vvUt7rvv\nvsvaPfnkkzz++OODGVVEZFiw2WzUNnVcMtX9mepWrBdNVhXo68a4uIDeK2ThvkQEe2IyGu2Yun84\ndIEGMD05lI37y9l5uIr0CRHER/jaO5KIiAwzZrOZ48ePA9Dc3IzZbO7bt337dmJiYggNDbVXPBER\nh9fVbaHoTGPfzIrF5U00tXb17XcyGRkZ7tM7dizcl/hwH3y9hmfvOYcv0IwGA9+cm8Drf9rHnzd8\nyQ+XpA75fqUiIuJYFi5cSE5ODpmZmTQ3N/PrX/+6b98f/vAHsrOzr9hu7dq15Obm4uLiwg9/+EMi\nIyMHK7KIyJB28ULQxeVNlFSdo8dy4eqYn5cLqaODiA/vLciiQrxxdnL8q2PXw+ELNICECD/uGhPC\nriNVbD94lhnJ+hZTRET6z+rVqwkLC+Odd97h2LFjZGdnk5OTQ1VVFW1tbURFRV3WZvbs2UyZMoVJ\nkyaxZs0aXnnllUsKuysxmz1wcrr1WYmDgrxv+RiDxZGygmPlVdaBoaw3rsdi5VRFE0dP13P8dANH\nS+qpuWiqe5PRQGy4L3fE+JMYbSYxxp8gO091fy0D+doOiwIN4OG0OPYX1fDhpmLuHB005KbLFBER\nx7Vv3z5mzJgBQGJiItXV1VgsFjZt2sSUKVOu2CY5Obnvdnp6Oj/72c+ueZ6GhrZbzvrVDJyOwJGy\ngmPlVdaBoazX51xbV19XxSstBO3l3rsQdFy4D/HhvsSE+hAR5nchb4+F2toWu2S/Hv3x2n5dgTds\nqhh/HzcWTIlm1ZZTfLL9NA/PGdwFYEVEhouHHrqHTz9dc+0H3kaio6MpLCwkKyuL8vJyPD09MZlM\nHDx4kDlz5lyxzSuvvML8+fNJTU2loKCAhISEQU4tIjLwrFYbFecXgv6qu2LVZQtBe/Z1VYwP9yXY\nPLSvjtnbsCnQAOZPjmJLYQXrd5cxKyWMEH8Pe0cSEZFh4NFHHyU7O5vHH3+cnp4eXnzxRQBqamoI\nCAjoe1xNTQ2//OUvefnll3n44YdZtmwZTk5OGAwGXnnlFTulFxHpPxcvBF1c3kTxlRaCjvXvK8hG\nhg29haCHumH1ark4m3gkPYG3Vx1ixcYTfO+h5Gs3EhG5TTz11D/w2mvLGTFiBGfPVvJ//s9zBAUF\n097eTkdHB//rfz3PmDFj7R1zSPL09OSNN964bPuvfvWrS+4HBQXx8ssvAzB69Gg++OCDy9qIiDiK\nSxeCbqa4vOmyhaBH+HucL8Z6uyuGBnpqwr5bNKwKNIDU0UGMjvTjwIlaDp2qY2xswLUbiYjcBmbN\nmsO2bZt58MFH2LJlE7NmzSEuLoFZs9LYu3c37733e1599f/aO6aIiNhJZ5eFk5XNfbMrnqxopqW9\nu2+/i7OR0VF+fV0VHXUh6KFu2BVoBoOBxXMTeOl3u3l/QxEvPWXGyXR7TMkpIo4j58Qn7K8+2K/H\nnBA8jm/EL7rq/lmz5vDv//4LHnzwEbZu3cR3v/u/+OCDP/L++3+ku7sbNze3fs0jIiJDl81mo66p\n46KxY82UVbdcthD02Fj/voJsuCwEPdQNuwINICrEm9kpYeQfqCBvfzmZqVp3RkRk5Mg46upqqKo6\ny7lz59iyJZ/AwGB+9KN/5dixI/z7v//C3hFFRGSAdPdYKDnb0jd27MRlC0EbGBnm09ddMS7cF79h\nuhD0UDcsCzSA+2eNZNfRalZvOcWUMSF4e7jYO5KISJ9vxC/62qtdA2Xq1Bn85jdvMXPmbBobG4iL\n651ZcNOmPHp6egY9j4iIDIy6pnb2HKu+6kLQvucXgv7q6tjttBD0UDdsCzQfDxfumxHLB7lFrNpy\niieyRts7koiI3c2ePYfvfOcpfve79+noaOeVV5aRl7eBBx98hA0b1rNmzd/sHVFERG6C1WbjVEUz\nB07UUniiljM1rX37jAYDkSFexPeNHfMhwMdNU90PUcO2QANInxjOpgPl5B8oZ/b4MKJChsZq6iIi\n9nLHHUls2rSr7/57733Yd3vGjNkALFx4L56enrS1OcZirCIit6uOrh4On2qg8EQtXxTX0tzWO6GH\nk8nIxMRgYs8XZTEjfHB1Mdk5rVyvYV2gOZmMLM5I4Od/KeSD3CKeXzxB3xSIiIiIiMOqb+6g8EQt\nB07UcbSkgR6LFQAfD2dmJIcyPj6QpBh/IsL9qKnRF22OaFgXaABjRwaQEhdAYXEde4/XkJoYbO9I\nIiIiIiLXxWqzUXL2XG9RVlRLaXVL376IIE9S4gMZnxBIbKiP1h8bJoZ9gQbwWEYCh07V85e8EyTH\nBeDirEu8IiIiIjI0dXZbOHq6oXc8WXEtTS29sy2ajAaSYv0ZHx9ISlwAgX7udk4qA+G2KNBC/D3I\nnBTJ2l2lrCso5Z7psfaOJCIiIiLSp7Glk8ITtRSeqOPI6Xq6enq7Lnq5OzNt7Ijeroux/ri73hYf\n329rt81P+J5pMWw/WMmanSVMHxeKv48WZBURERER+7DZbJRVt3DgfNfF02cvjBcLDfBg/Pmui3Fh\nvhiN6rp4O7ltCjR3VycenB3Hu58d48NNxXz7niR7RxIRERGR20h3j4VjpY0cKOrtuljf3An0dl1M\njPJjfEIQ4+MDCDZ72Dmp2NNtU6ABTE8OZeP+cnYeriJ9QgTxEb72jiQiIiIiw1hzaxeFxb1dFw+f\nqqez2wKAh6sTU8aEkBIfyLiR/ni4Ods5qQwVt1WBZjQY+Ie5o3jtT3v584Yv+eGSVM12IyIiIiL9\nxmazUV7ben4q/FpOljdjO78vxOzeO+tifCDxEb44mYx2zSpD021VoAHER/gyZUwIO49Use1gJTOT\nw+wdSUREREQcWI/FyvGy810XT9RS29QBgMEACZF+vbMuxgcQGuBp56TiCG67Ag3gobQ49hXV8NGm\nk6SODtZsOCIiIiJyQ1rau/miuHfB6MOn6mjv7O266O5qYlJiMOPjAxkXF4CXu7ouyo25LSsTfx83\nFkyJZtWWU3yy/TQPz4m3dyQRERERGcJsNhtlVefI211CYVEtReVN2M73XQz0dWP62FDGJwQyKtJP\nXRflltyWBRrA/MlRbCmsZP3uMmalhBHir9lyREREROSCHouVojNNfePJqhvaATAAceG+pMQHMD4+\nkLBATwya10D6yW1boLk4m3g0PZ63Vh1ixcYTfO+hZHtHEhERERE7a+3o5uDJOgpP1HGwuI62zh4A\nXJ1NTB0Xyh2RfiTHBeDj6WLnpDJc3bYFGsCdo4MYHenHgRO1HDpZx9iRAfaOJCIiIiKDrKqhjcKi\n3qtkX5Y1YT3fd9Hfx5W7kkIYHx9IYpQfYaF+1NScu8bRRG7NbV2gGQwGFs9N4KXf7eb93CJeijar\nz7CIiIjIMGe12jhR3sSBE72zLlbWtfXtiw31Pj/rYiCRwV7quiiD7rYu0ACiQryZnRJG/oEK8vaX\nk5kaae9IIiIiItLP2jt7OHSqngNFtRw8WUdLezcALk5GxscHMj4hkOS4APy8XO2cVG53t32BBnD/\nrJEUHK1m9ZZT3DUmBB8P9SkWERERcXS1je19V8mOlTZisfZ2XfTzcmH2+DBS4gMZE23Gxdlk56Qi\nF6hAA3w8XLhvRizv5xaxasspnswabe9IIiIiInKDrDYbpyqaOXB+1sXymta+fVEhXn1XyqJDvNV1\nUYYsFWjnzZkYTv6BcjYdKCdtfBhRId72jiQiIkNEa2srS5cupampie7ubp599ll+85vf0NbWhodH\n7zItS5cuZezYsX1turu7eeGFF6ioqMBkMvH6668TGalu9CL9raOrh8OnGig8UcsXxbU0t/V2XXQy\nGUmOCyAlPpCUuAD8fdzsnFTk+qhAO8/JZGRxRgI//0shH+QW8fziCfpmRUREAFi5ciWxsbE899xz\nVFVVsWTJEoKCgnj99dcZNWrUFdt88skn+Pj4sHz5crZu3cry5cv5xS9+McjJRYan+uaO82uT1XG0\npIEeixUAHw9nZiSHMj4+kKQYf1xd1HVRHI8KtIuMHdm72OCBE7XsPV5DamKwvSOJiMgQYDabOX78\nOADNzc2YzeZrttmxYwf3338/ANOmTSM7O3tAM4oMZ1abjZKz5zhQ1DuerLS6pW9fRJAn4xN6Z12M\nDfXBqC/YxcFdV4H22muvUVhYiMFgIDs7m+Tk3kWdq6qq+P73v9/3uLKyMp577jnuueeegUk7CB5N\nj+fgyTpWbDxBclyABo2KiAgLFy4kJyeHzMxMmpub+fWvf83y5ct58803aWhoIC4ujuzsbNzcLnSh\nqq2txd/fHwCj0YjBYKCrqwsXl6tPRGU2e+DkdOvvO0FBjtNN35GygmPldfSsHV09fFFUS8GRs+w+\ncpb65k4AnEwGJowKYnLSCCaNGUGIv4fdsw5VjpQVHCvvQGa9ZoFWUFBASUkJK1asoLi4mOzsbFas\nWAFASEgIf/zjHwHo6enhiSeeID09fcDCDoYQfw8yJ0Wydlcp6wpKuWd6rL0jiYiIna1evZqwsDDe\neecdjh07RnZ2Ns888wyjR48mKiqKZcuW8d577/H0009f9Ri28wvffp2GhrZrPuZagoK8HWYhXUfK\nCo6V11GzNrZ09nZdLKrlaEkDXT29XRe93J2ZNnZEb9fFWH/cXc9/hLVYBvV5Ourr6ggcKW9/ZP26\nAu+aBdqOHTuYO3cuAHFxcTQ1NdHS0oKXl9clj1u5ciVZWVl4enreUtih4J5pMWw/dJY1O0uYPi5U\ng0pFRG5z+/btY8aMGQAkJiZSXV1Neno6JlPv1a709HQ+/fTTS9oEBwdTU1NDYmIi3d3d2Gy2r716\nJnI7stlsnCxvIq+ghAMnajl99sKH3tAAD8YnBDI+PpC4MF+MRnVdlNvDNQu02tpakpKS+u77+/tT\nU1NzWYH217/+lf/6r//q/4R24O7qxIOzR/Lup8f4ML+Yb9+bdO1GIiIybEVHR1NYWEhWVhbl5eV4\neHjw9NNP8+abb+Lj48OuXbtISEi4pM306dNZu3YtM2fOJC8vj7vuustO6UWGns5uC9sPVrJ+zxmq\n6nuvHJuMBu6INpMSH8j4+ACCzYPbdVFkqLjhSUKu1EVj//79jBw58rKi7UocpX/9/XNGseWLSnYe\nqeKB9ATGxAbc0vHUp3ZgKOvAcaS8yjowHCnrQHv00UfJzs7m8ccfp6enh5deeomGhgb+8R//EXd3\nd0JCQvjnf/5nAJ555hnefvttFixYwPbt21m8eDEuLi785Cc/sfOzELG/ppZOcveVk7+/nJb2bpxM\nBmaND+eOKD/GjfTHw83Z3hFF7O6aBVpwcDC1tbV996urqwkKCrrkMfn5+UydOvW6TuhI/esfSYvn\ntT/t5a0PC/nRktSbnhXodutTO1iUdeA4Ul5lHRj9lXW4FHmenp688cYbl21fsGDBZdvefvttgL61\nz0QEymtaWLe7jJ2Hz9JjseHl7syiaTFkTAwnPjbQYX43igyGaxZo06dP55e//CWPPfYYhw8fJjg4\n+LIrZQcPHrzim5Sji4/wZcqYEHYeqWLbwUpmJofZO5KIiIiIQ7DZbBwpaWBdQSmHTtYDEGJ2Z96k\nSKaNC8VVM2WLXNE1C7SJEyeSlJTEY489hsFgYNmyZeTk5ODt7U1mZiYANTU1BATcWhfAoeqhtDj2\nFdXw0aaTpI4OvjBrkIiIiIhcpsdiZdeRKtbvLqPs/HployJ8yZocRUpCoNYpE7mG66o2Ll7rDHpn\nsLrYxx9/3H+Jhhh/HzcWTolm5ZZTfLz9NI/Mibd3JBEREZEhp7Wjm/z95eTuPUNjSxdGg4HJdwST\nNTmK2FAfe8cTcRi6HHQdsiZHsbmwks93lzE7JWzQF0QUERERGaqqG9v5fHcZW7+opLPbgpuLiXmT\nIpmbGkGgr7u944k4HBVo18HF2cSj6fG8teoQKzae4HsPJds7koiIiIhdnShvYl1BKfu+rMFmA7O3\nK/fNiGVWShgebvqIKXKz9K/nOt05OojEKD8OnKjl0Mk6xo4cnmPuRERERK7GarWx78sa1u0upbi8\nGYDoEG+yJkeSmhiMk8lo54Qijk8F2nUyGAw8lpHAS7/bzfu5RbwUbdYvIREREbktdHZZ2HqwkvW7\nS6lp7AAgJS6ArMlRjI7yw6CJP0T6jQq0GxAV4s3s8eHk7y8nb185mZMi7R1JREREZMA0nOtk474z\n5O8vp7WjB2cnI7PHhzFvUiShAZ72jicyLKlAu0H3z4yl4EgVq7ae4q6kEHw8XOwdSURERKRflVW3\nsL6glJ1HqrBYbXh7OHPfjFjmTAzXZx+RAaYC7Qb5eLhw34xY3s8tYtWWUzyZNdrekURERERumc1m\n4/CpetYVlHL4dAMAoQEezJsUydSkEbhoYWmRQaEC7SbMmRhO/oFyNh0oJ218GFEh3vaOJCIiInJT\nunus7DxylvW7yyivaQUgMcqPrMlRjIsL0MLSIoNMBdpNcDIZWZyRwM//Usj7G4r439+coMGxIiIi\n4lBa2rvJ21/Oxr1naGrtwmQ0MCUphKxJUUSP0JfPIvaiAu0mjR0ZwPj4QA6cqGXv8RpSE4PtHUlE\nRETkmqoa2li/u4xtX1TS1WPF3dXE/LuimHtnBP4+bvaOJ3LbU4F2Cx5Nj+fgyTpWbDxBclyA+maL\niIjIkGSz2Sg607uw9IGiWmy5SXWzAAAgAElEQVRAgI8bmZMimZkcirurPhKKDBX613gLQvx7B85+\ntquUtQWl3Ds91t6RRERERPpYrFb2Hq9hXUEZpyp7F5aODfUma3IUd44OwmTUmq4iQ40KtFu0aFoM\n2w6d5dMdJcwYF6quASIiImJ3bR3drN9dxoY9ZdQ2dWAAJiQEkjU5ioQIX42dFxnCVKDdIndXJx6c\nPZJ3Pz3Gh/nFfPveJHtHEhERkdtUfXMHG/aeYUthBa0dPbg4GZkzIZx5kyIJ8fewdzwRuQ4q0PrB\n9HGh5O0rZ+eRKuZMDCchws/ekUREROQ2UnL2HOt2l7L7aDUWqw0/b1cemBTJnIkReLk72zueiNwA\nFWj9wGgw8M3MUbz2x738eUMRP1qSqjVDREREZEBZbTYOFtexrqCUY6WNAIQHejJvUiSLZsfT1Nhm\n54QicjNUoPWT+HBfpiSFsPNwFdu+qGRmSpi9I4mIiMgw1N1jYfuh3oWlK+t6i7AxMWayJkcxNtYf\ng8GgmaVFHJgKtH700Ow49n1Zw0ebiklNDNaUtSIiItJvmtu6yNtXzsZ9ZzjX1o3JaGDa2BHMmxRJ\nVIgWlhYZLlRB9CN/HzcWTolm5ZZTfLz9NI/Mibd3JBEREXFwlXWtrN9dxvZDZ+nuseLh6sSCKdFk\n3BmB2dvV3vFEpJ+pQOtnWZOj2PJFJZ/vLmNWShgjNGOSiIiI3CCbzcaXZY2sKyjjwIlaAAJ93Zg3\nKZIZyaG4uegjnMhwpX/d/czF2cQjc+J5a9UhVuQW8T8eTrF3JBERuUWtra0sXbqUpqYmuru7efbZ\nZwkKCuLll1/GaDTi4+PD8uXLcXd372uTk5PDG2+8QVRUFADTpk3jmWeesddTEAfRY7Gy53g16wrK\nKDl7DoC4cB+yJkUxcVQQRuPwmoTMZrNhs9nsHUNkSFGBNgDuHB1EYpQfhcV1HDxZR3qQ+oWLiDiy\nlStXEhsby3PPPUdVVRVLliwhMDCQF154geTkZH7605+Sk5PDP/zDP1zSbsGCBSxdutROqcWRtHX0\nsLmwgg17y6hv7sRg6P08kTU5ivhwX3vHuyVWm5WGjiZq2+uoaa+ltr2emvZaatrrqGmvo8vShdFg\n7PtjuuS26SrbjRjP7/v77Re3MRpMF/Ybr/fYl2//apu505OW5s7LH2e88eOaDCYMBgNGg9HePyIZ\nYlSgDQCDwcDiuaN48d0CPsgtYlZqlL0jiYjILTCbzRw/fhyA5uZmzGYzv/rVr/Dy8gLA39+fxsZG\ne0YUB1XX1MHne8rYXFhBR5cFV2cTGXdGkJkaQbDZcYZJ9Fh7qO9o6C262ur6irGa9nrq2uvosVku\na+NiciHIPQBfdy86u7qx2KxYbZbz/+/989XtHmsPnTbLZdu/+uPIDBguKuRMlxR1fUWe8fJ9V2pj\nuqhodXdzobOzB4PBgIHeK69f3TZ8dWbD+dsGA0YM57f17jNioPe/8y0uum0w9B3hwjHOH/fC+S7s\nu6T9+ba9x7+QxqfRnZaWzr5jcOEol96/2vMxfH02o+HilGAwGC/Pdv75X5wfON/2wvl9zQn9+Dfg\ncirQBkhksBdp48PJ21/Op9tOMfWOYHtHEhGRm7Rw4UJycnLIzMykubmZX//6133FWVtbG6tXr+aN\nN964rF1BQQFPP/00PT09LF26lDFjxgx2dBmiTlU2s66glD3HarDabPh6ubBwajRpE8LxdBuaC0t3\nWbouu/pV29b7//qOBmxc3lXR08mDcK8wgjwCCHIPINA9gCD3QALdA/Bx8cJgMBAU5E1NzbmbzmWz\n2S4r3CyXFXOWywq7C7evXBRarZbLjufu6UzzufZbO67NgsV6tZyXZ+6xWbD2dF/1nDL4plRN5ImE\nxwbs+CrQBtD9M2PZdaSKP687RlK0Hz4eLvaOJCIiN2H16tWEhYXxzjvvcOzYMbKzs8nJyaGtrY1n\nnnmGp556iri4uEvapKSk4O/vT1paGvv372fp0qV8/PHHX3ses9kDJ6dbX78qyIG61jtSVri1vFar\njd1HzrJyUzGHT9YBEBPqwwNpccwcH4GzU/92dbuZrG1d7ZxtqeZsS+35/9dQdf52Q3vTFduY3XwZ\nHTiSEV7BhHgFMsI7qO+2l4vngGWVC8Wp5aKC8uLbNnrH+Nl6H3zJ/Qu3bfT+d+F+77BAG1bbV4/k\non29hfiFfRcd8/w26/nH3PIxbV89kr5i1PZ3z+OSY/7d87jkmDawYb3FY/bumxSeQlDgwP2dVYE2\ngLw9XLhvZizvbyhi1eaTPDk/0d6RRETkJuzbt48ZM2YAkJiYSHV1NV1dXfz3//7fWbRoEd/4xjcu\naxMXF9dXtE2YMIH6+nosFgsm09ULsIaGtlvOeqtXIwaTI2WFm8/b2X1hYemq+t6f8diR/mRNjmJM\ntBmDwUBjQ+ugZLXZbLR0t57vith7Jaz2/NWwmvZaWrsv/ztowIDZzY/R5vgLV8E8AglyDyDAzR83\npytM9W+B9iYr7Vz79XKkvweOk9VIUJBvX9avppYZsClm+uEEjvPaQlDgrWf9ui8lVKANsDkTwtl6\nsJJNBypImxCuhSRFRBxQdHQ0hYWFZGVlUV5ejqenJ++88w6TJ0/m4YcfvmKb3/72t4SGhrJo0SK+\n/PJL/P39v7Y4k+GnqbWLjXvPkLe/nJb2bpxMBmaMC2Xe5EgigrwG7LxWm5Xatnq+bCi9MCnHRcVY\nh6XzsjYmg4kAdzMxPlHnuyFe+OPv7o+zUR8ZRQaL/rUNMCeTkW/dN45lv9nB+xuK+N/fnNA34FBE\nRBzDo48+SnZ2No8//jg9PT28+OKLPP/880RERLBjxw4A7rrrLr773e/yzDPP8Pbbb3PPPffw/PPP\n88EHH9DT08Orr75q52chg6W8tpX1BaXsOFxFj8WKp5sTi6ZFkzExAl+v/llY2mK1UN/R2Dce7OJJ\nOWrb6+ix9lzWxtno3Ft0eQQS6O5PkHtgXxFmdvPTbIIiQ4QKtEEwcXQw4+MDOXCilj3Ha5iUqAlD\nREQciaen52WTgGzduvWKj3377bcBGDFiBH/84x8HPJsMDTabjaMlDawrKOPg+fFlwWZ35k2KZPrY\nUFxdbvzqabelm9qO3qtfF7ohXpiU40oTRLg7uRPmOYIIvxB8jL4EelwownxcvPUlsYgDUIE2SB7N\niOfgyTr+srGIlLgAXJzVzUVERMTR9VisFBytYn1BGaXVLQAkRPiSNTmK8fGB11xYur2no6/46p0R\n8cIMiY2dV56Uw9vFixifqL7Cq3dMWO/siJ7OvVPzO9J4HhG5lAq0QRJi9mDepEg+21XK2oJS7p0e\na+9IIiIicpPaOrrJP1BB7t4zNJzrXVh6UmIwWZOjGBnm0/c4m81Ga3fbpVPTn18rrKa9lpbuyycH\nMWDAz9WXUX5xfYVXYF8x5o+bk9tgPlURGWQq0AbRomkxbDt0lk93lDBjXCj+PvoFKyIi4khqGtv5\nfHcZW76opLPbgquLibmpEUxJ9sXi3MrZ9uMcKq67ZGxYe0/HZccxGowEuvkT5R1BkEfARRNzBBLg\nZsbZNDTXQhORgacCbRC5uzrx0Ow4/uvTo/w1v5j/dm+SvSOJiIjIdSgqb+Ctz3az//QpcG3DPaqT\n0GArBtc2Cjrq2Xao+7I2zkYnAt0DSPCL+7uuiAGYXf0wGTXcQUQupwJtkE0bN4KN+86w60gVcyaE\nMyrSz96RRERE5Cpq2+t5r/AzjrccxOBqxWV073YLUGUBt043RngGXyjAvpoZ0aN3Ug7NjCgiN0oF\n2iAzGgx8M3MUr/1xL+9vKOJH/5iKUTMqiYiIDCnVbTWsO51Hwdl9WLFCtztJIxKI9g3uW6T5q0k5\nNDOiiPQnFWh2EB/uy5SkEHYermLbF5XMTAmzdyQREREBKlurWHs6l71Vhdiw4W00U1cUyZzYSfyP\neamaGVFEBpwKNDt5OC2efV/W8NGmYu4cHYyHm34UIiIi9nLmXAVrT+dyoOYQNmyEe4WSEZHGnz88\nh1OPjUXTNPuyiAwOVQV2YvZ2ZeHUGFZuPskn20/zSHq8vSOJiIjcdkqay/jsdC4Ha48AEOUdzvyY\nuYwLvIM120s419bAfTNi8fFwsXNSEbldqECzo6xJkWwprODzPWXMGh/GCH8Pe0cSERG5LZxsOs1n\np3I5Un8cgFifaO6OzWCM/2gMBgPn2rpYW1CKl7sz8yZF2jmtiNxOVKDZkYuziUfmxPPWqkOsyC3i\nfzycYu9IIiIiw5bNZqOo8SSfnc7ly4YTACT4jeTumLmMMsddMtnHpztLaO+0sDhjJO6u+rgkIoNH\nv3Hs7M7RQSRG+VFYXMfBk3WMGxlg70giIiLDis1m41h9EZ+d3kBx02kA7vAfxfyYDOL9Lh9bVt/c\nQe7ecgJ8XEmbED7IaUXkdqcCzc4MBgOL547ixXcL+CC3iDuizTiZtGaKiIjIrbLZbByqO8pnp3Mp\naS4DYGzAHcyPySDWN+qq7VZvPUWPxcr9M0fi7KT3ZBEZXCrQhoDIYC/SxoeTt7+cjfvK1dddRETk\nFlhtVgprDrP2dC5nWioAGB80lvkxGUR6f/0Vscq6VrYerCQs0JOpSSMGI66IyCVUoA0R98+MZdeR\nKlZvPcWUpBDNFiUiInKDrDYr+6oKWVuykcrWKgwYuDM4hfkxGYR5XV+xlbP5JDYbPDhrJEajFqAW\nkcF3XQXaa6+9RmFhIQaDgezsbJKTk/v2VVZW8i//8i90d3czZswYXn755QELO5x5e7hw38xY3t9Q\nxKrNJ3lyfqK9I4mIiDgEi9XC7qr9rCvZSHVbLUaDkbtG3ElW9BxCPIOv+zinKpvZe7yGuDAfxicE\nDmBiEZGru2aBVlBQQElJCStWrKC4uJjs7GxWrFjRt/8nP/kJTz31FJmZmbz00ktUVFQQFhY2oKGH\nqzkTwtl0oIJNBypImxBOVIi3vSOJiIgMWT3WHnZV7mVdSR51HfWYDCamh01mXvQcAt1vfNKtD/OL\nAXgo7dIZHUVEBtM1C7QdO3Ywd+5cAOLi4mhqaqKlpQUvLy+sVit79+7l5z//OQDLli0b2LTDnJPJ\nyOKMBJavOMCfNxSx9JsT9AYhIiLyd7ot3Wyv3M3nJfk0dDbiZHRiVvg0MqNn4+9mvqljHj5dz9GS\nBsaO9Gd01M0dQ0SkP1yzQKutrSUpKanvvr+/PzU1NXh5eVFfX4+npyevv/46hw8fJjU1leeee25A\nAw93SbH+jI8P5MCJWvYcr2FS4vV3zRARERnOuixdbC3fyYbSTTR1ncPZ6Ex65Ewyombh5+p708e1\n2Wx9V88enBXXX3FFRG7KDU8SYrPZLrldVVXFk08+SXh4ON/+9rfJz88nLS3tqu3NZg+cnEw3FfZi\nQUGO1f3vRvI+83AKz/7/eXy4qZiMKTG4Ot/663UjHOm1VdaB40h5lXVgOFJWGd46ejrYXL6D3NLN\ntHS34mpyITMqjYyoWXi7eN3y8fcer6Hk7Dkm3xFM9Aj9vRcR+7pmgRYcHExtbW3f/erqaoKCggAw\nm82EhYURFdW7lsjUqVMpKir62gKtoaHtFiP3fmioqTl3y8cZLDea1xnInBTBZztL+dMnh7l3xuWL\naA4UR3ptlXXgOFJeZR0Y/ZVVRZ7cirbudjad2UZe2VZae9pwd3Lj7pgM0iJn4OXs2S/nsFitfLT5\nJCajgQdmjeyXY4qI3Iprrr44ffp01q1bB8Dhw4cJDg7Gy6v32yonJyciIyM5ffp03/7Y2MErJoaz\nRVNj8PV04dOdJdQ3d9g7joiIyKBp6W7l45Pr+NH21/nk1HoAFsVm8fLU/8OikVn9VpwBbDt4lqr6\nNmamhBFi9ui344qI3KxrXkGbOHEiSUlJPPbYYxgMBpYtW0ZOTg7e3t5kZmaSnZ3NCy+8gM1mY9So\nUaSnpw9G7mHP3dWJB2fH8V+fHuWv+cX8t3uTrt1IRETEgZ3raiG3dDOby7fTaenC29mL+TELmBk+\nBTcnt34/X1e3hdVbT+HiZOSeaTH9fnwRkZtxXWPQvv/9719yPzHxwhpd0dHRvP/++/2bSgCYNm4E\nefvPsOtIFXMmhDMq0s/ekURERPpdY2cTG0o3sbV8F93WbnxdvFk0MosZYXfhYnIZsPNu3FdOw7lO\nFkyJxuztOmDnERG5ETc8SYgMHqPBwOK5o3jtj3t5f0MRP1qSitGoafdFRAZba2srS5cupampie7u\nbp599lmCgoJ48cUXARg9ejQvvfTSJW26u7t54YUXqKiowGQy8frrrxMZGWmH9ENXbWs9K45/wvaK\nAnpsFsyufsyLTmNq6CScTc4Deu62jm7W7DiNp5sTC6ZEDei5RERuhMMVaJWtVfzffW+S4BNPWuT0\nW5pW1xHEh/syNSmEHYer2HqwklkpWgRcRGSwrVy5ktjYWJ577jmqqqpYsmQJQUFBZGdnk5yczHPP\nPcemTZuYPXt2X5tPPvkEHx8fli9fztatW1m+fDm/+MUv7Pgsho6atjrWl+Sxq2ovFquFQDd/5sXM\n4a4Rd+JkHJyPJmsLSmnt6OHhtDg83Aa2GBQRuREOV6AZMdDQ3sTnjfnklm3mzuDxZETNJNI73N7R\nBsxDafHs/bKGnE3FpI4OxsPN4X5sIiIOzWw2c/z4cQCam5vx8/OjvLyc5ORkAObMmcOOHTsuKdB2\n7NjB/fffD8C0adPIzs4e/OBDTFVrNWtLNrKn6gBWm5Uw7xDmRqSRGjIek3HwlpRpbOlk/e4y/Lxc\nSL8zYtDOKyJyPRzuk36IZzD/cc+rfHpoExtLt7C7ah+7q/Yxyi+O9KiZJAUkYjRcc3JKh2L2dmXh\n1BhWbj7JJ9tP80h6vL0jiYjcVhYuXEhOTg6ZmZk0Nzfz9ttv8/LLL/ftDwgIoKam5pI2tbW1+Pv7\nA2A0GjEYDHR1deHicvUxVcN1rdDSxnJyjnzGjrJ92LAR6RvGg2PuZkrERIzGwX/P/nDzSbq6rfx/\n991BRNiNje8eaq/t11HWgaGsA8eR8g5kVocr0ABcTM5MD7uLqaGTOFpfxMbSzRxrKOLLxmJCPIKY\nEzmTu0ZMHNCBxYNt/uRIthRW8PmeMmaND2OEv6YCFhEZLKtXryYsLIx33nmHY8eO8eyzz+LtfeHN\n2WazXfMY1/OY4bZWaNm5cj47nUthzSEAIr3CmB87l+TAMRgNRoxG46BnrW5oY93OEkLM7qTEmm/o\n/EPptb0WZR0YyjpwHClvf2T9ugLPIQu0rxgNRpICRpMUMJrylsrzV9T288HxHD4+uZaZ4VOZFT4N\nX1fHqcavxtnJxKPp8fzHykN8kFvE/3w4xd6RRERuG/v27WPGjBlA70zGnZ2d9PT09O2vqqoiODj4\nkjbBwcHU1NSQmJhId3c3Npvta6+eDSenmkpZe3oDh+qOARDjE8XdMRkkBSRiMNh3sqtVW05hsdp4\nYNZInEzDq8eNiAwPDl2gXSzcK5QnxjzCvXHz2XxmO1vKd7L2dC4bSvJJHTGB9MiZhHuF2jvmLZk4\nKojEKD++KK7ji+I6kuMC7B1JROS2EB0dTWFhIVlZWZSXl+Pp6Ul4eDh79uwhNTWV9evX88QTT1zS\nZvr06axdu5aZM2eSl5fHXXfdZaf0g+dE4yk+O7WBYw1FAMT5xnJ3bAaJ5gS7F2YApVXn2HmkiugQ\nb1ITg6/dQETEDoZNgfYVX1cf7ombT1ZMOrvO7mVj2RZ2Vu5hZ+UeEs0JZETN4g7/UUPijeJGGc5P\nu//iuwV8kFvEmBizvv0TERkEjz76KNnZ2Tz++OP09PTw4osvEhQUxI9//GOsVispKSlMmzYNgGee\neYa3336bBQsWsH37dhYvXoyLiws/+clP7PwsBobNZuN4wwnWns6lqPEkAKPN8dwdk0GCOc7O6S6V\ns7k334NpIzE64OcAEbk9DLsC7SsuJhdmhk9lethdHK47Ru75cWrHGooI9QwhPXImk0ImDPg6K/0t\nMtiLtAnh5O0rZ+O+cuZN0po6IiIDzdPTkzfeeOOy7X/+858v2/b2228D9K19NlzZbDaO1B/ns1O5\nnGouAWBMwGjujpnLSN9oO6e73PHSBr4oriMxyo+kGH97xxERuaphW6B9xWgwMi5wDOMCx1B67gwb\nS7ewt7qQ9459yN+K1zIrYiozw6fi7eJl76jX7YGZIyk4UsXqraeYkhSCj8ftMaZBRETsz2qzcrD2\nKGtP51J67gwAKYFJZMWkE+0zNL80tNlsfLTpq6tncQ7Zi0ZEbh/DvkC7WJR3BP+YtJj74u5m05nt\nbK3YxZpTn7O+JI/JI+4kPXIGIzxD7B3zmrzcnblvRix/3lDEys0nWTI/0d6RRERkmLParByoOcTa\n07mUt1RiwMDE4GTmx2QM+THehSfqOFHexMRRQcSF+do7jojI17qtCrSvmN38uD9+AfNjMthZuYe8\nsi1sq9jFtopdJAUkkh45k9Hm+CH9DVvahHDyD1Sw+UAFcyaEExXi+DNViojI0GOxWthbXci60xs5\n21aNAQOTQiYyP2aOQ3ypabXa+GhzMQYDPDBrpL3jiIhc021ZoH3FzcmVtMjpzIqYyhe1R8gt3czh\numMcrjtGuFcoGZGzuDMkBSfj0HuZnExGFmcksHzFAf68oYil35wwpAtKERFxLBarhV1n97G+ZCM1\n7XUYDUamhk5iXvQcgj0C7R3vuu08cpbymlZmjAslPNDT3nFERK5p6FUedmA0GBkfNJbxQWM51VRK\nXtkW9tcc5A9HV7C6+FNmR0xnRvgUPJ2H1uLQSbH+TEgIZH9RLbuPVTP5jqH/TaaIiAxt3dYedlbu\nZn1JPvUdDTgZTMwIn8K8qDQC3B1rco3uHiurtpzCyWTgvhmx9o4jInJdVKD9nVjfKGJ9/4G69gby\nz2xle0UBfzu5lrWnc5kSmsqcyBkEewTZO2afR9LjOXiyjr/mnSAlPhBXZ5O9I4mIiAPqsnSzrWIX\nG0o30djZhLPRibSI6cyNmo3Zzc/e8W7KpgPl1DZ1MG9SJAG+bvaOIyJyXVSgXUWAu5kHE+5hQWwm\nOyoKyDuzjc3lO9hSvpNxgWNIj5xJvF+s3bsVhpg9yJwUyWc7S1m3q5R79Q2hiIjcgI6eTrZW7GRD\n6SbOdbXgYnIhI2oWGZGz8XV13PHN7Z09fLz9NG4uJhZMHXrT/ouIXI0KtGtwd3IjPWoWsyOmc6Dm\nELllm/mi9jBf1B4myjuCjMiZTAhOxmS035WrRVNj2H7wLJ/uLGFGcij+PvqWUEREvl57Twebzmxn\nY9lmWrvbcDO5khWdTnrkTLxcHH+s1ud7yjjX1s39M2K1HI2IOBQVaNfJZDRxZ0gKE4OTOdVcQm7p\nZgprDvPukfdZWfwpaRHTmR52Fx7O7oOezd3ViYfS4nhnzVH+kneC79w3dtAziIiIY2jrbiOvbCt5\nZ7bR3tOOu5M7C2IzmRMxHY8hNtb6Zp1r62LtrlK8PZzJnDQ012YTEbkaFWg3yGAwMNI3hpHjYqhp\nq+sdp1a5m1XFn/LZ6Q1MC51MWuQMAgd5IPXUsSPYuO8MBUerSZ/YyKhIxxwvICIiA+NcVwsby7aw\n+cx2OiydeDl7cu/I+cyKmIa70/DqebFmRwkdXRYemDUSd1d91BERx6LfWrcgyCOAh0fdx8LYTLZV\nFJB/Zht5Z7aSf2YbKUFjyYiaxUjfwen3bjQYWDx3FK/9cS9/3vAlP14yCaNR0+6LiNzumjqbyS3d\nzJbyHXRZu/F28eLu2LnMDJ+Kq2n4df2ra+pg475yAnzcSBsfbu84IiI3TAVaP/Bw9iAzOo30yJns\nq/6C3LLNHKg5yIGag8T6RHH/2HnEusQN+Di1+HBfpiaFsONwFVsPVjIrJWxAzyciIkNXQ0cjn5fm\ns62igB5rD36uvtwXnca00Mm4mJztHW/ArN52ih6LlftnxuLsZLR3HBGRG6YCrR+ZjCYmjZhAash4\nTjSeJLdsMwdrj/Jv2/+TADczaZEzmBo6aUC7kjyUFs++L2vJ2VRM6uhgPNz0IxYRuZ1Ut9Ty/rFP\n2Fm5B4vNQoCbmXnRc7grNBVn4/B+T6iobWXbwUrCgzyZmjTC3nFERG7K8P5NbScGg4EEcxwJ5jiq\n2mrYWbOLvFM7+KjoY9ac/JzpYZNJi5yOv5u5389t9nZl4dRocjaf5OPtp3g0PaHfzyEiIkPT+pI8\nPj65DqvNSrB7IPNi0pkcMsGuMw0PppzNJ7HZ4MFZcermLyIOSwXaAAvxCOJbqYvJCJvD1vKdbDqz\nndyyzeSd2cqEoHFkRM0i2qd/Z5jKmhzJ5sIKNuw5w6yUMEIDHH+6ZBERubaGjkaifcOZHTaDiXZe\nAmawFVc0se/LGuLDfUmJD7B3HBGRm6YCbZB4OXsyPyaDjKjZ7K06wMayLeytLmRvdSFxvjFkRM1i\nXOAYjIZb7y/v7GTi0fR4/mPlIVZsPMH/fDilH56BiIgMdY+OfoCgIG9qas7ZO8qgstlsfJRfDMBD\naXEYDLp6JiKOSwXaIHM2OjElNJW7RtzJ8YYT5JZt5kjdcYoPnibQPYA558ep3erMWhNHBXFHtJkv\niuv4oriO5Dh9mygiIsPT4dP1HCttJDkuQMvMiIjD0/RGdmIwGEj0T+DZlKf54V3PMT1sMo2dTfz1\ny9X8cNurrC7+jMbOpls6/uKMBAwG+CC3iB6LtR/Ti4iIDA1Wm42P8k8C8I1ZI+2cRkTk1qlAGwJC\nPUP4ZuJDvDItmwWxmRgNRtaX5PGj7a/z+yMfUHau4qaOGxHsRdqEcM7Wt7Fx75l+Ti0iImJ/e45V\nU1J1jiljQogK8bZ3HBGRW6YujkOIt4sXC2MzmReVxu6q/eSWbaHg7D7+X3t3Hh9Vfe+P/3VmyyST\nTDKTzGTfA9kw7AECSQAFARFZBPFqrVdbq9Sv39t6+9VSLbd+FZcf9qGlVqveelu/VBGIGkWwokBY\nYhK2QBYI2ffJDNn37QWBBIEAACAASURBVPz+CAyELSyZnJnk9Xw88pBz5szwykcyJ+/5bJm1xzDe\nIxzzgxIR6xl1U/PUViSGITPPhC8PlWJmrA+0mtG3KSkREY1NvX39+DytGHKZgOWJoVLHISIaFizQ\n7JBSrkSCXzxm+U5HXn0BfihPw+mGsyhoLIK3iwHzAhMxw2cKVDcwT83VWYnliWHY8l0BPj9QjJ8u\nihqB74CIiMj2Dp6qgamhA/Om+MOoc5E6DhHRsGCBZscEQUCsZyRiPSNR1VqDH8oPIMt0HJ+eScFX\nxbuR5D8Lif4JcHe6/pCOuZP9sO94FdJOVGPuJH8E+3AICBERObaunj6kHiyBSinDvQkhUschIho2\nnIPmIPxdffGTmDX4vwm/xaLg+YAI7Cr9Hr8/vBEf53+Gqtaaaz5XLpNh7V3jIAL4ZE8BRFEcueBE\nREQ28MPRSjS2dmPBtEB4uDpJHYeIaNiwB83BuDtpcW/4ItwdMh8ZtUfxQ8UB/FhzBD/WHEG0fjzm\nByYiWj/+ij1gYkP0mDzOC8fPWpB1ug7x0d4SfQdERI5n27ZtSE1NtR5nZ2dj4sSLe0zW1dVhxYoV\nePLJJ63nNm/ejK+++gre3gPvt8uWLcPq1atHLvQo1tbZg53pZdCoFVg8I0jqOEREw4oFmoNSyVVI\n9J+F2X4zkHvuNL4vT0N+fQHy6wvgq/HG/MAkTPeeBKVcaX3OA/MjcKr4HLbtLcTECC84KeUSfgdE\nRI5j9erV1uIqMzMTu3btwoYNG6yP/+xnP8N99913xfMeeeQRPPzwwyOWc6zYnVGO9q5erJ4XDhe1\ncugnEBE5EA5xdHAyQYY7vGLwH1OexHPTn8F078kwtZux5fQ2vHj4VXxT8h1aulsBAEadCxZOD8K5\n5i7sziiXODkRkWN65513sG7dOuvx4cOHERISAl9fXwlTjR2NrV34LqsCHq4q3DklQOo4RETDjgXa\nKBLkFoBHYx/ES7Oex4KguegV+7Cz5Du8eHgj/nl6B2rbTLhnVjDcNSrs+rEM55o6pY5MRORQTp48\nCV9fXxgMBuu5f/zjH3jkkUeuev3u3bvx7//+7/jFL36BioqKkYo5qn11qBTdvf24b04oVBwJQkSj\nEIc4jkI6tQeWRyzBopA78WPNEeytOIBD1Rk4VJ2BWM8ozEmIxc7vurBtXyGevG+C1HGJiBzG9u3b\nsWLFCuuxyWRCe3s7goKunAeVnJyMmTNnYvr06di5cydefvll/PWvf73u6+t0LlAobr/oMBgcZ7Xe\nm8labWlFWnY1/A0arJg/HnL5yH/OPFrbVmrMahuOlBVwrLy2zMoCbRRTK5wwN3A2kgJm4aQlDz+U\npyH33Gnk4jTcJnngaEUg8st9ER3kKXVUIiKHkJGRgRdeeMF6vH//fsycOfOq18bFxVn/PH/+fGza\ntGnI129oaL/tjAaDG8zmltt+nZFws1n/lpqLvn4Ry2aHor6+zYbJrm40t62UmNU2HCkr4Fh5hyPr\n9Qo8DnEcA2SCDJMME/Drqevwm2lPY6pxIvpUTVCFn8JfTr+N3SU/oK3n9n8pICIazUwmEzQaDVQq\nlfXcqVOnEBUVddXrX375ZRw5cgTAwMIi48aNG5Gco1W5qQUZeSYE+7hhaqRh6CcQETko9qCNMSHa\nIDw24SHc17EEm9NSUSc7ja9KduPbsu8x03ca7pYlwqVPC5VcNfSLERGNIWazGXq9/opznp6eg443\nb96Ml156CatXr8aGDRugUCggCAJefvnlkY48quzYXwwAuD85HLLLtpIhIhpNWKCNUZ7OOvzvhLVY\n/+EByI2V0IRUI60qHWlV6RAgwODiCX+NL/xdL37p1bor9lcjIhorJkyYgA8//HDQuffee2/QscFg\nwEsvvQQAiIyMxKeffjpi+UazM+UNOFV8DtHBOsSE6KSOQ0RkUyzQxjCdmxPumRGBlDQZZvvOQuSE\nblR2VaDIXI7K1hocbz+F4+ZT1uvVcjX8XH0Q4OoLv/NFm5/GG2qFWsLvgsaKnv5e1LWbUdNmQm2b\nCTVtdahtrwOEfqgEFdQKZzgr1HCWq+GsUEOtGPjv5X92lqut1yplCn7oQGTnRFHE9n1FAIBVyeH8\nmSWiUY8F2hh3d3wg0rKr8f2RaiRPjMfCKQkwm1sgiiIau5pQ1Voz6KukqQzFTaWDXsPL2XOgl03j\nc763zQ+ezjrIBE5xpJvX3dcNk7UQqxsoxtpNMLefgwhx0LVquRrOSifUdzeiu7/npv8uuSCHWuF0\nWVHnfI2i7pLjS65VyZT8hZHIhk6ctaCouhlTxxsQ5qeVOg4Rkc2xQBvjlAo5Hpg/Du98fgpbfyhE\nXJQPAEAQBOjUHtCpPTDBK9p6fU9fD2raTahqrUV1aw0qW2tQ1VqNbHMOss051utUchX8NT7WnraB\nLx84K5xH/Hsk+9TZ2wVTe521ELvQM3aus+GKQkyjcEGYewh8NUb4aLzhq/GGj8YId5UWRqMWZnML\n+vr70NHXic7eTnRc8jXouK9j0PGFP3f2daGuw4Kuvu6b/j5kggxqudOVvXZy5ysKOu8OD/S04/zj\nFx9TyVX8QIPoKvr7RexIK4YgACuSwqSOQ0Q0Im6oQNu4cSOys7MhCALWr19/xdLBPj4+kMsH9m3Z\ntGkTvL29bZOWbGLKeC9EB+twsugcjuSbEOzlcs1rlXIlgtwCEOQWYD0niiKau1uu6G0ra6lESXP5\noOfr1Tr4u/rA39XP2utmcPHiL6ejWEdvx/kC7Hxv2Pmvhq7GK651U7oiwiP0fAHmbS3I3JSuQ/ZS\nyWVyuMo0cFVqbjlrv9h/ZYHXd7WCr+N8wTe4ADzXUY/Ovq6b/nsFCFArnAYVbZf36F3ei3d5j56T\n3Ik/RzTqpOfWotrShsQ4X/h53frPNhGRIxmyQMvMzERZWRm2bt2KoqIirF+/Hlu3bh10zQcffACN\nhm+cjkoQBDx45zhs+CgTb3ychQXTArFwehBc1DfWwSoIAtydtHB30iLGM9J6vre/F7VtdQMFW1sN\nqltrUdlajVOWfJyy5FuvU8qU8NP4wN91cI+bRnntQpHsT1tPu7UXzNoj1l6Hxq6mK651V7khSjcO\nPpf2iLkY4aqS9n1EJsjgonSBy2382+sX+9HV13XVXjyFswhzQxM6rEVfxxUFYUNXI2rauq7oRbwR\narnTlUWctehzvnKY5lUKQhZ5ZC96evvxxYFiKOQy3DcnVOo4REQjZsjfwNPT03HXXXcBAMLDw9HU\n1ITW1la4urraPByNnACjKx6/Jxrb9xUj9VApvj9aiSUzgzF/agCclPJbek2FTIEANz8EuPkNOt/c\n3YLq1trLetyqUdZSMeg6Dyf3i8MjNT7wd/OD0dkLctmt5aHh0dLdal2kw7pgR7sJLd2tV1yrc/JA\ntH68dUjihULsdgogeycTZOd7va4cznujG1sOFHndlwzBvM6wzd5OdPZ1DHqsqasZtW11t1TkqeQq\nOMvV8NUa8Hj0I3BRclgySWPf8Sqca+7C3fGB0Gu5GBURjR1DFmgWiwWxsbHWY71eD7PZPKhA27Bh\nA6qqqjB16lQ8++yznDDvoBIm+GJhQhg+3Z2P3Rnl2LavCP/KqsDShBAkTfSDUjE8n6xrVW7Q6t0Q\npb+4aWtffx9M7eaL89rO97jlnjuN3HOnrdcpZAr4uhjh5+qLSJ9QuEMHf1dfuKn4gcFwEkURTV3N\nF+eHtV/sGWvtabviek+1DrGeUYOGJnq7GOHMFT5vyUCRN9CrdasLiouiOFDk9V1Z1HWeH555ZcF3\nsUevracD/WL/sH5fRDeqo6sXXx0uhbOTHPfMCpE6DhHRiLrpRUJEcfAnss888wwSExPh7u6OX/7y\nl/j222+xaNGiaz5fp3OBQnH7PSAGg9ttv8ZIcqS8/37fHVi9MAqf7ytEaloRtnxXgO+OVODBhZGY\nNzUQcrlthkD5wAMTMW7QuZauVpQ1VqG8qWrgv41VKG+uRkVrNTJqj1qv81BrEezhj2CPAAS5+yPY\nwx/+bj5QyO1nHRx7/DcgiiLqOxpR2VyDiqYaVDbXoOr8f9t6OgZdK0CAt6sXogzhCHD3RYDWFwFa\nH/hpfaBWOEn0HQywx7a9FkfKSiSVf2VVoLWjBysSQ+HqrJQ6DhHRiBryt1ej0QiLxWI9rqurg8Fg\nsB4vX77c+uekpCQUFBRct0BraGi/1axWNzpMyF44Ut5Lsy6aFoCEaCN2ppdh7/FKvL31BLZ+V4Dl\niaGYFmWEbIR6Sr1lfvDW+WG6bjqAgeFfde0WtMgacbq6GFVtNahsqUF2bT6yay/ObZMLcvhojPDT\n+J5fmGRgCwCtaugFJ4ab1P8G+sV+NHQ2WueFXbqE/eWLWsgEGXxcDYjwCIevxhu+LgPzxIwuBqjk\nl/2i1Ae0NHSjBTe/+uFwkbptb8ZYzMqClG5Wc3s3dmeWQ+uixILpgVLHISIacUMWaLNnz8bmzZux\ndu1a5Obmwmg0Woc3trS04D/+4z/w7rvvQqVSISsrC3fffbfNQ9PI0WpUePCucbg7PhCph0px8GQN\n3vsyF0HpZViRFIa4cM8RL3Zkggw+GiPuMIRjnPN46/n2ng5Ut9Va57Rd2AqgqrUGWaaLz3dVaqxz\n2/xcfRHg6gsfFyOUlxcfDqhf7Me5jgbUtptQ02oaNDTx8n3C5IIcRhevgSGJLhcX6zC4eMHPW+cw\nhQQRjS47D5ehq7sP9yeHQ62yn1EQREQjZch3vilTpiA2NhZr166FIAjYsGEDUlJS4ObmhgULFiAp\nKQkPPPAAnJycEBMTc93eM3Jceq0ajy6OwuIZQfjyYAky8kx4e/tJRPi7Y2VSGKKCb3WmzPBxUToj\nwiMUER4XV/vqF/th6ai3LkZyoWA701CIMw2F1utkggxGFwP8NT4IcPWD3/keNw8nd7ucU9nX3wdL\nxznUtF9cur62rQ6m9jr09PcOulYhyOGtMcLHZWCRjgvzxAzOnlxwhYjsiqWpA3uPV8LLXY3kSX5D\nP4GIaBS6oY+m/vM//3PQcVRUlPXPP/3pT/HTn/50eFOR3fLWu+CJZbFYMjMYnx8oxvGzFrzxyXHE\nhOiwMikcYX5aqSMOMlB4ecHo4oXJxjus5zt6O1Fj7W2rPd/jVoPaNhOO1mVbr9MoXKzF2oUvX403\nVHLViOTv7e9FXbvlkmGJA4VYXbsZvWLfoGuVMiV8NN7wcfG+ZENnIzzVehZiROQQvjxYgt4+ESsS\nw6Cw0XxnIiJ7x7EDdEsCjK74X6viUFzdjM/TipBb2oC80iOYPM4LKxLDEGC071UVnRVqhLmHIMw9\nxHquX+xHfWejtaet8vx/CxtLcLax2HqdAAFGF6+BPdus89v8oFd73HJvW09fD+o6LBeXrT8/P6yu\nw3LFSnoquQr+rn4Xl63XGOGr8YFe7cE9rIjIYVWZW3E4pxYBBg1mxHhLHYeISDIs0Oi2hPlp8eza\nyThd1oCUtIEetRNnLZgR4437EkPhrXOc/a5kggxeznp4Oesx0XBxa4muvu6B3raWgeX/L/S6mepO\n4jhOWq9Ty9WXLEZyobdt8AqH3X3dqG2vu7iR8/lCzNxx7oo9q9RyNYLdAq29YRd6xDyc3FmIEdGo\nk5JWDFEEViaHQyazv6HlREQjhQUaDYuoYB1++/AUnCw6h8/TivFjngmZ+XWYE+eLZbNDHHqTUSe5\nCiHaIIRog6znRFFEQ1ejtVi70ONW3FSGoqbSQc/3cvaEr9YL1U1m1Hc2XFGIuSicEeYebF2k40Kv\nmLtKa5fz34iIhltRVROOn7UgIsAdE8M9pY5DRCQpFmg0bARBwMQIL9wR7omjZ8z4PK0YadnVOJxT\ni3mT/XHPrGBoNSMzd8vWBEGAXq2DXq3DHV4x1vPdfT2obTMNFG5tNQO9bq01OGU6AzelKyI8Qgdt\n5uyj8YabcuSX/ScisheiKGL7viIAwP3J4Xw/JKIxjwUaDTuZIGB6lBFTxnshPceELw+W4LsjFUjL\nrsZd0wKwaEYQNGrHX9L+alRyJYK0AQjSBljPiaIIrd4JLQ3S7RVGRGSvckvqcaaiEXHhnhgf6CF1\nHCIiybFAI5uRy2SYE+eLGTHeSMuuxteHSwc2vT5WhUUzgnDXtIAxsceNIAhQK5wk3cyZiMge9feL\n2L6/CAKAVcnhUschIrILo/+3Y5KcUiHDnVMDMCfOFz8crcQ3P5YhJa0Ye45U4J5ZIZg72Q9KBZeB\nJyIaaw5lV6Pc1IqZsd4ItPPVf4mIRgqXgqMR46SUY/HMYLz+ZAKWzQ5BV28/Pvn+LH77/o9Iy65G\nb1//0C9CRESjQm9fPz7enQ+5TMDyxDCp4xAR2Q0WaDTiXNQKLE8MwxtPzsKi+CC0tPfgf3adxgsf\nZuDHvFr0i+LQL0JERA7twMka1FjakDzJD0YPZ6njEBHZDRZoJBk3FxXWzI/Aa7+YhXmT/XGuqRPv\np+bhv/6WieNnzRBZqBERjUpdPX1IPVgCJ5Uc9yaESB2HiMiucA4aSU7n5oSf3B2Ju2cEIfVgCdJz\na7F5xymE+mqxKjkMMSF6qSMSEdEw2nOkAk1t3Vhz13i4uzpJHYeIyK6wB43shtHDGT9bGoOXHp+B\nqZEGlNQ0Y9OnJ/DGP4+hsKpJ6nhERDQM2jp7sOvHcmjUCqycGyF1HCIiu8MeNLI7/l4a/HLFHSit\nbUZKWjFyiuux8eOjiAv3xMqkMAR5u0kdkYjGmG3btiE1NdV6nJOTgwkTJqC9vR0uLi4AgOeeew4T\nJkywXtPT04Pnn38e1dXVkMvlePXVVxEYGDji2e3NNz+Wob2rF2vmRUDjrER7a6fUkYiI7AoLNLJb\nIT5a/HrNJBRUNCJlfxFOFp3DyaJzmB5lxPLEUBgMLNSIaGSsXr0aq1evBgBkZmZi165dKCwsxKuv\nvorx48df9Tlff/01tFot3nzzTRw8eBBvvvkm3nrrrZGMbXcaWrqw50gldG5OmD/FX+o4RER2iUMc\nye6ND/TAcw9Nwa/XTESwjxuyTtfhhQ8z8Panx2Fp6pA6HhGNMe+88w7WrVs35HXp6elYsGABACAh\nIQHHjh2zdTS799WhEvT09uO+OaFQKbn/JRHR1bAHjRyCIAiYEOaJ2FA9jhWY8fmBEuzJKsfeoxWY\nO8kfSxOCOdGciGzu5MmT8PX1hcFgAAD86U9/QkNDA8LDw7F+/Xqo1WrrtRaLBXr9wCJHMpkMgiCg\nu7sbKpXqmq+v07lAobj9wsUeRxhUm1uRdrIG/gZXLJ83DnL5wGfE9pj1ehwpL7PaBrPajiPltWVW\nFmjkUARBwNRIIyaPMyC3ogn/b1cevj9WiQMnq3HntAAsnhEMV2el1DGJaJTavn07VqxYAQB45JFH\nEBkZiaCgIGzYsAFbtmzB448/fs3n3sjWIQ0N7bed0WBwg9ncctuvM9z++8sc9PeLuG92COrr2wDY\nb9ZrcaS8zGobzGo7jpR3OLJer8DjEEdySDKZgPnTAvHKz2fikbsjoXFWYteP5XjuvcNIPVSCjq5e\nqSMS0SiUkZGByZMnAwAWLFiAoKAgAMD8+fNRUFAw6Fqj0Qiz2QxgYMEQURSv23s2mpXVtiAzvw4h\nPm6YGmmQOg4RkV1jgUYOTSGXYe5kf7z6xEw8MD8CcpkMXxwowXPvpePbzHJ09/RJHZGIRgmTyQSN\nRgOVSgVRFPHoo4+iubkZwEDhNm7cuEHXz549G7t37wYA7N27FzNmzBjxzPZix/4iAMD9c8MhCILE\naYiI7BsLNBoVVEo57o4PwutPzsLyxFD09fdj6w+FeP6v6dh3vAq9ff1SRyQiB2c2m61zygRBwJo1\na/Doo4/ioYceQm1tLR566CEAwFNPPQUAWLJkCfr7+/Hggw9iy5YtePbZZyXLLqX8sgbklNQjJkSH\nmBC91HGIiOwe56DRqOLspMCy2aGYPyUAuzLK8P2RSvzj2zPYlVGG++aEYmaMD2QyfnpLRDdvwoQJ\n+PDDD63HS5YswZIlS6647t133wUA695nY5koitbes1XJ4RKnISJyDOxBo1HJ1VmJ1XMj8PqTs3Dn\n1ADUN3fhw6/z8fu/ZeLombobmqxPRES35/hZC4qrmzEt0oBQX63UcYiIHAJ70GhUc3d1wkMLxuPu\n+ECkHirFoVM1eOfzHAT7uGFVUhhiQ/WcD0FEZAP9/QO9ZzJBwIqkMKnjEBE5DBZoNCZ4uTvjsSXR\nWDwjCF8eLEFmfh3++Fk2xge4Y2VyOMYHekgdkYhoVDmcU4uac+1ImugLX0+N1HGIiBwGCzQaU3w9\nNXjyvglYMrMFn6cVI7voHF7bcgwTwvRYmRSGEB8OwSEiul09vX348mAxFHIZls0OlToOEZFDYYFG\nY1KQtxv+9+qJKKxqQsr+IuQU1yOnuB5TIw1YnhgGfy9+2ktEdKv2Hq/GueYuLIoPgl6rljoOEZFD\nYYFGY1qEvzv+z79NQV5pPVLSinH0jBnHCsyYFeuDZXNCYfRwljoiEZFD6ejqxdeHS+HsJMeSWcFS\nxyEicjgs0IgAxIToER2sw4lCCz5PK8bhnFpk5JmQNNEPSxNCoHNzkjoiEZFD+DazHK0dPViRFAZX\nZ6XUcYiIHA4LNKLzBEHA5HEGTIzwQlZ+Hb44UIy9x6tw8FQN7pwSgMUzg+DmopI6JhGR3Wpu68a3\nmRXQalRYMC1A6jhERA6JBRrRZWSCgBkx3pgWZcChU7VIPVSC3Znl2HeiCgunB2Lh9CC4qPmjQ0R0\nua8Pl6Krpw/3zw2HWsX3SSKiW8F3T6JrkMtkSJroh1mx3th3oho7D5ci9VApvj9aiSUzgzF/agCc\nlHKpYxIR2QVLYwf2Hq+Cl7sayZP8pI5DROSwZFIHILJ3SoUcC6YF4rUnZ2FVchhEEdi2rwjPv5eO\n749WorevX+qIRESS++JgCfr6RaxICoNCzl8viIhuFd9BiW6QWqXAPbNC8MZTs7A0IRid3X3Y8l0B\nfvvXH3HwZA36+lmoEdHYVGluRXpOLQIMrpgR4y11HCIih8YCjegmuaiVWJkUjtefnIUF0wLR1NaN\nv32Tjxc/zERmvgn9oih1RCKiEZWyvxgigFXJYZAJgtRxiIgcGgs0oluk1ajw4F3j8NovZiJpoh/q\nGjrw3pe5eOmjLGQXWiCyUCOiMaCwsgknCi0YH+COuHBPqeMQETk8LhJCdJv0WjUeXRyFxTOD8OXB\nEmTkmvD29pOI8HfHyqQwRAXrpI5IRGQToihi+75CAMCqueEQ2HtGRHTbWKARDRNvnQueuDcWS2YG\n4/O0Yhw/a8EbnxxHTIgOjy2bAL0LN2wlotHlVHE9CiqbMCnCC+MCPKSOQ0Q0KrBAIxpmAQZX/K9V\ncSiubsbnaUXILW3Af/7pAAKNroiPNiI+2hsGD2epYxIR3ZZ+UcSO/UUQAKxMCpM6DhHRqMECjchG\nwvy0eHbtZJwpb8C+7BocyTdhx/5i7NhfjFBfLWZEGzE92hs6NyepoxIR3bTMfBMq6loxK9YHAUZX\nqeMQEY0aLNCIbCwySIc5U4NQWlGPYwVmZObXIb+0ASU1zdj6QyHGBXpgRrQRU6OM0LqopI5LRDSk\n3r5+fJ5WDLlMwPLEUKnjEBGNKizQiEaIRq1EYpwfEuP80NzWjaNn6pCRX4ezFY0oqGjElu/OIjpE\nh/hoI6aON8BFzTlrRGSfDmRXw9zYiTunBnDINhHRMGOBRiQBrUaFeVMCMG9KAOqbO3Hk9ECxlltS\nj9ySenz87RncEeaJ+GhvTIrwgpNKLnVkIiIAQFd3H1IPlcJJKcfShBCp4xARjTos0IgkpteqsTA+\nCAvjg1DX2IGsfBMy8upw/KwFx89aoFLIMDHCC/HR3ogL10OpYLFGRNLZc7QCTW3duDchBO4aDssm\nIhpuLNCI7IjRwxn3zArBPbNCUGVpO1+smZB1ug5Zp+vg7CTH5HEGxEd7IyZEB4Wce80T0chp7ejB\nNz+Ww9VZibvjg6SOQ0Q0Kt1QgbZx40ZkZ2dDEASsX78ecXFxV1zz5ptv4sSJE/j444+HPSTRWOTv\npYF/YhjumxOKclMrMvNNyMw34XBOLQ7n1MLVWYmpkQPFWmSgB2QybhBLZCvbtm1Damqq9TgnJwef\nfPIJXnrpJchkMmi1Wrz55ptwdr44HyslJQVvv/02goIGCpmEhAQ89dRTI559OO36sQwdXb14YH4E\nXNT8jJeIyBaGfHfNzMxEWVkZtm7diqKiIqxfvx5bt24ddE1hYSGysrKgVHJRA6LhJggCgn3cEOzj\nhvvnhqOouhmZ53vV9p+oxv4T1XDXqDA9yoj4GG+E+2khCCzWiIbT6tWrsXr1agAD98Vdu3bh5Zdf\nxvPPP4+4uDi8/vrrSElJwUMPPTToeUuWLMFzzz0nReRh19DShT1HK6Fzc8L8Kf5SxyEiGrWGLNDS\n09Nx1113AQDCw8PR1NSE1tZWuLpe3PPktddew69+9Sv8+c9/tl1SIoIgCIjwd0eEvzvW3jkOZyoa\nkZlvwpHTddhztBJ7jlbCU6u2bogd5O3KYo1omL3zzjvYtGkTnJ2drfdCvV6PxsZGiZPZ1pcHS9DT\n24/lc0I5F5aIyIaGLNAsFgtiY2Otx3q9Hmaz2XpTSklJQXx8PPz9+Wka0UiSyQREB+sQHazDQwvG\nI6+0AZn5JhwrMGNXRjl2ZZTDW++CGeeLNT8vjdSRiRzeyZMn4evrC4PBYD3X3t6OL7/8Em+//fYV\n12dmZuLxxx9Hb28vnnvuOcTExIxk3GFTc64NB0/WwNfTBQl3+Egdh4hoVLvpAeSiKFr/3NjYiJSU\nFHz00UcwmUw39HydzgWKYfjkzWBwu+3XGEmOlJdZbcPWWX193HHnzBB09/Th6GkT0o5XITPPhNRD\npUg9VIoQXy2SJvsjcZI/fDyHLtbYtrbBrI5t+/btWLFihfW4vb0dTz31FB577DGEh4cPunbixInQ\n6/WYO3cujh8/WrpLagAAGg9JREFUjueeew5fffXVdV/fXu+R/73rNPpFEY8ujYWPt/uwvraj/Ttz\npLzMahvMajuOlNeWWYcs0IxGIywWi/W4rq7O+snhjz/+iPr6ejz00EPo7u5GeXk5Nm7ciPXr11/z\n9Roa2m87tMHgBrO55bZfZ6Q4Ul5mtY2Rzhrh44aIxVH4tzsjcKLQgsy8OpwqPod/fJOPf3yTj1Bf\nLWZEGzE92hs6NyfJ894OZrWN4crqSDfbG5GRkYEXXngBANDb24t169Zh6dKlWLly5RXXhoeHW4u2\nyZMno76+Hn19fZDLr12A2eM9sqSmGYeyqxHq64YIH9dhfW1H+pkAHCsvs9oGs9qOI+UdjqzXuz8O\nWaDNnj0bmzdvxtq1a5Gbmwuj0Wgd3rho0SIsWrQIAFBZWYnf/va31y3OiGhkqVUKzIzxwcwYH7R1\n9uBYgRmZ+XXIL21ASU0ztv5QiHGBHpgRbcTUKCO0LtzTiOhaTCYTNBoNVKqBn5MPPvgA8fHx1sVD\nLvfBBx/A19cXS5cuRUFBAfR6/XWLM3uVsr8IAHB/cjjntBIRjYAhC7QpU6YgNjYWa9euhSAI2LBh\nA1JSUuDm5oYFCxaMREYiGgYatRKJcX5IjPNDc1s3jp6pQ0Z+Hc5WNKKgohFbvjuL6BAd4qONWJgQ\nJnVcIrtjNpuh1+utx1u2bEFAQADS09MBADNmzMDTTz+Np556Cu+++y7uvfde/OY3v8Gnn36K3t5e\nvPLKK1JFv2V5pfXILW1AbIgO0SH6oZ9ARES3TRAvnVQ2AoZryIyjdIECjpWXWW3DnrPWN3fiyOmB\nYq2kphkAoJDLMCFUj/gYIyZFeEGtst/9juy5bS83FrOOtiGOtmZP90hRFPHyP46gpKYFL/50GkJ9\ntbf9mpdzpJ8JwLHyMqttMKvtOFJeyYc4EtHopteqsTA+CAvjg1DX2IGsfBOOFlhwonDgS6WQYWKE\nF+KjvREXrufy2kRjxLECM0pqWjAtymiT4oyIiK6OBRoRWRk9nHHPrBA8uuwOnMivRVa+CRn5dcg6\nPfClVskxZbwB8dHeiAnRQSGXSR2ZiGygr78fKWnFkAkCViZxyDMR0UhigUZEV+XvpYF/YhjumxOK\nclMrMvNNyMw34XBOLQ7n1EKjVmBa1MAea5GBHpDJuHgA0Whx+FQtas61I3mSH3z0LlLHISIaU1ig\nEdF1CYKAYB83BPu44f654SiqbkZmnglZp+uw/0Q19p+ohrtGhelRRsTHeCPcT8uV3ogcWE9vH744\nWAKlQoZls0OljkNENOawQCOiGyYIAiL83RHh7461d45DQUUjMvNNOHLGjD1HK7HnaCU8tWrERw/0\nrAV5u7JYI3IwPxyrQkNLFxbPCLrqPolERGRbLNCI6JbIZAKignWICtbh3xaMR35ZAzLyTDhWYMau\njHLsyiiHt94FM84Xa35eGqkjE9EQ2jt7sTO9DM5OCiyeGSx1HCKiMYkFGhHdNoVchjvCPHFHmCd6\nevtwsqgemfkmZBdakHqoFKmHShFgcMWMGCOmR3vD6OEsdWQiuopvM8vR2tGDVclhcHVWSh2HiGhM\nYoFGRMNKqZBjaqQBUyMN6OzuxYlCC7Ly63Cq+Bx27C/Gjv3FCPXVYkb0QLHGIVRE9qGprRv/yqqA\nu0aFu6YGSh2HiGjMYoFGRDajVikwM8YHM2N80N7Zg2MFFmTmm5BX2oCSmmZs/aEQ4wI9MCPaiKmR\nRmg1KqkjE41ZXx8uRVdPH9bMC4eTivsdEhFJhQUaEY0IF7USc+J8MSfOF83t3Th6xozMPBMKKhpR\nUNGILd+dRXSIDvHRRkwdb4CLmsOriEaKubED+45XwejhjMSJflLHISIa01igEdGI07qoMG+yP+ZN\n9kdDSxeyTtchM9+E3JJ65JbU4x+7z+COME/ExxgxKcILahXfqohs6YsDJejrF7E8KZQb0BMRSYy/\n9RCRpHRuTlg4PRALpweirrEDWfkmZObX4UShBScKLVApZJgY4YX4aG/EheuhVHDoFdFwqqxrxY+5\ntQg0uiI+2lvqOEREYx4LNCKyG0YPZ9wzKwT3zApBtaUNmfkmZOTXIev0wJdaJceU8QbER3sjJkTH\nT/qJhkFKWjFEAKuSwyHjvoVERJJjgUZEdsnPS4PliWG4b04oKupakZFvQmZeHQ7n1OJwTi00agWm\nRhoxb3oQfLROXNSA6BYUVDTiRKEF4wM9cEeYXuo4REQEFmhEZOcEQUCQtxuCvN1wf3I4iqubkZFv\nQlZ+HdKyq5GWXQ25TECEvztiQnSICdEjxNcNchl714iuRxRFbN9fBAC4PzkcAnvPiIjsAgs0InIY\ngiAg3N8d4f7uWDt/HAoqGlFsasWRvFoUVDTiTEUjPj9QAmcnOaKCBoq1mBAdfPQu/OWT6DIni86h\nsLIJkyK8EBHgLnUcIiI6jwUaETkkmUxAVLAOidOCsCQ+EK0dPThd1oC8sgbkldbj+FkLjp+1ABhY\niCQmeKBgiw7RwcOVm2PT2NYvitixvwgCgJXJYVLHISKyC/v2fY+5c+8c8rpXXnkFS5eugp+fv01y\nsEAjolHB1VmJaVFGTIsyAgAsjR3WYi2vtAGHcmpxKKcWAODvpUH0+eGQkYEecHbiWyGNLRl5JlSa\n25AwwQcBBlep4xARSa6mphp79nx7QwXa7373O5jNLTbLwt9KiGhU8vJwRpKHM5Im+qFfFFFZ14q8\n0oGCraCiEVVH2rDnSCXkMgGhflprD1uYn5arQ9Ko1tvXj8/TiiGXCVg+J1TqOEREduGPf3wd+fm5\nSEycjoULF6OmphpvvfUXvPrqSzCb69DR0YHHHnsCs2cn4ic/+QmefvrX2Lv3e7S1taK8vAxVVZV4\n5plnMWvW7NvOwgKNiEY92SULjSyaEYSe3n4UVTUhr2ygd62oqgmFlU1IPVQKJ5UcUYEe1vlrfl4a\nzl+jUWX/iWpYmjpx19QAeHk4Sx2HiOgKn/1QiKzTdcP6mtOjjFgzP+Kajz/44E+QkvIZQkPDUV5e\nir/85UM0NNQjPn4mFi9eiqqqSrz44vOYPTtx0PPq6kzYtOlP+PHHw/jyyx0s0IiIboVSIUNUsA5R\nwTqsTALaO3twurzROhwyu+gcsovOAQDcNSrr6pDRwTrotWqJ0xPdus7uXnx1uBROSjmWJoRIHYeI\nyC5FR8cCANzctMjPz0VqagoEQYbm5qYrro2LmwQAMBqNaG1tHZa/nwUaEY15Lmolpow3YMp4AwCg\nvrlzYDjk+R629FwT0nNNAABfTxfEBA/0rkUG6eCi5tsoOY7vjlSiua0by2aHQKtRSR2HiOiq1syP\nuG5vl60plUoAwHff7UZzczPeeedDNDc342c/+8kV18rlF/dhFUVxWP5+/mZBRHQZvVaNOXG+mBPn\nC1EUUWVpQ15JPfLKGnCmvBHfH6vE98cqIQhAmK8W0SF6xIboEObnDqWC89fIPrV29GB3RhlcnZW4\nOz5I6jhERHZFJpOhr69v0LnGxkb4+vpBJpNh//4f0NPTMyJZWKAREV2HIAgIMLgiwOCKhfFB6O3r\nR3F1s3U4ZHF1M4qqm/H14VKolDKMD/Sw9rAFGF0h4/w1shPfpJeho6sPa+eHcuVSIqLLBAeH4syZ\n0/D19YOHhwcAYO7c+Xj++V8jLy8H99yzDEajER999IHNs/AdmojoJijkA0XY+EAPLE8EOrp6cebC\n/LWyBuQU1yOnuB4AoHVRIjpEb10h0mBwkzg93apt27YhNTXVepyTk4NPPvkE//Vf/wUAiIyMxB/+\n8IdBz+np6cHzzz+P6upqyOVyvPrqqwgMDBzJ2Fb1zZ3Yc7QSnlonzJtim317iIgcmU6nQ0rKzkHn\nfH398Pe/f2o9XrhwMQDAYHCD2dyCsLCLwzDDwiLw5z+/PyxZWKAREd0GZycFJo3zwqRxXgCAhpYu\n5J+fu5ZXWo+MPBMy8gbmr/l5aRAZ6IGYkIEFSjRqpZTR6SasXr0aq1evBgBkZmZi165deOWVV7B+\n/XrExcXh2Wefxf79+5GcnGx9ztdffw2tVos333wTBw8exJtvvom33npLkvyph0rQ29eP++aEQamQ\nD/0EIiKSDAs0IqJhpHNzQsIEXyRMGJi/VnOu3Toc8kxFI/Yer8Le41UQBCDEx21gOf9gHSIC3PmL\ns4N455138Oqrr+Lhhx9GXFwcAGDevHlIT08fVKClp6dj+fLlAICEhASsX79ekrw159pw4GQN/Lw0\nSJjgI0kGIiK6cSzQiIhsRBAE+Hlp4OelwV3TAqHXa5B5qtpasBVVNaGkpgU708ugVMgwPsD9/P5r\negR6c/6aPTp58iR8fX0hl8uh1Wqt5z09PWE2mwdda7FYoNfrAQxMPhcEAd3d3VCprr16ok7nAsUw\nFOqXDqf9729OQxSBR5fGwttbe51nScPRhv46Ul5mtQ1mtR1HymvLrCzQiIhGiFwuQ4S/OyL83bFs\ndig6u3tRUNF0vmCrR25pA3JLGwAUwdVZiahgnXUPNiM3FLYL27dvx4oVK644fyNLK9/INQ0N7beU\n61IX5kYAQElNMw6drEaYnxbh3hrreXtxaVZH4Eh5mdU2mNV2HCnvcGS9XoHHAo2ISCJqlQJx4Z6I\nC/cEADS1diG/rMG6B9uR03U4croOAODlrj7fu6ZDdLAObi7cw0oKGRkZeOGFFyAIAhobG63nTSYT\njEbjoGuNRiPMZjOioqLQ09MDURSv23tmCzv2FwEA7k8Oh8AeWSIih8ACjYjITri7OmFmrA9mxvpA\nFEWYGjqswyHzyxqQll2NtOxqCACCvN2svWvjAtyhUnL+mq2ZTCZoNBprkRUWFoYjR45g2rRp+Ne/\n/oWf/GTwBqazZ8/G7t27kZiYiL1792LGjBkjmjf3/L+dCaF6RAXrRvTvJiKiW8cCjYjIDgmCAB+9\nC3z0Lpg/JQB9/f0oq221DocsrGpCmakFuzLKoZDLMC7A3VqwBXu7QSZjb8lwM5vN1jllALB+/Xr8\n/ve/R39/PyZOnIiEhAQAwFNPPYV3330XS5YsweHDh/Hggw9CpVLhtddeG7Gsoihix76B3rNVyeEj\n9vcSEY12999/L775ZufQF94GFmhERA5ALpMhzE+LMD8tliaEoKunD2crG63L+eeXDfSy7dhfDBcn\nBaIvnb+mc+bwtmEwYcIEfPjhh9bjiIgI/POf/7ziunfffRcArHufSeHoGTNKa1sQH21EsI/jTLon\nIiIWaEREDslJKceEUE9MCB2Yv9bc3o3TZQPFWm5JA44WmHG0YGBVQU+t08CG2SE6xATrodVw/tpo\n1tfXjx1pxZAJAlYkhkkdh4jIITz22EPYuPFN+Pj4oLa2Br/97bMwGIzo6OhAZ2cnfvWr3yAmZsKI\nZGGBRkQ0CmhdVIiP9kZ8tDdEUYS5sWNQ79rBkzU4eLIGABBgcLX2rkUGesBJxflro8merAqY6tsx\nd5IfvPUuUschIrppKYVf43jdqWF9zcnGO7AyYuk1H09KmodDh9KwatUaHDiwH0lJ8xAePg5JSXNx\n9GgWtmz5O1555f8b1kzXwgKNiGiUEQQBRp0LjDoXzJ3sj/5+EeV1LdaCraCiCZXmVvwrqwJymYAI\n/4vz10J83SCXyaT+FugWdff04ZN/nYZSIcO9s0OljkNE5DCSkubhz39+C6tWrcHBg/vx9NO/wqef\nfoxPPvkYPT09UKvVI5aFBRoR0SgnkwkI8dEixEeLJTOD0d3Th8KqpksKtkacqWjE5wdK4OwkR1SQ\nDklTAnBHiI6bZTuYH45V4VxTJxbPDILOzUnqOEREt2RlxNLr9nbZQlhYOM6dM8NkqkVLSwsOHNgH\nLy8jXnzx/+L06Tz8+c9vjVgWFmhERGOMSik/v6eaHkA4Wjt6BuavnZ/DdvysBcfPWvDaL2bCqOMQ\nOUdyqvgcXJ2VWDIzWOooREQOZ9asOXj//b8gMTEZjY0NCA8fBwDYv38vent7RywHCzQiojHO1VmJ\naVFGTIsa2GjZ0tgBQaWAp4tS4mR0sx6/JxruHi6Q9/dLHYWIyOEkJ8/Dk08+hv/5n0/Q2dmBl1/e\ngL1792DVqjXYs+df2LkzdURysEAjIqJBvDycYTC4wWxukToK3SS9Vg2Dp4b/74iIbkF0dCz278+w\nHm/Zst365zlzkgEA99yzDBqNBu3ttnuf5UxwIiIiIiIiO8ECjYiIiIiIyE6wQCMiIiIiIrITLNCI\niIiIiIjsxA0tErJx40ZkZ2dDEASsX78ecXFx1sc+++wzbN++HTKZDFFRUdiwYQME7ptDRERERER0\n04bsQcvMzERZWRm2bt2KV155Ba+88or1sY6ODuzcuRNbtmzBp59+iuLiYhw/ftymgYmIiIiIiEar\nIQu09PR03HXXXQCA8PBwNDU1obW1FQDg7OyMv//971Aqlejo6EBraysMBoNtExMREREREY1SQxZo\nFosFOp3OeqzX62E2mwdd8/7772PBggVYtGgRAgMDhz8lERERERHRGHDTG1WLonjFuSeeeAKPPPII\nfv7zn2Pq1KmYOnXqNZ+v07lAoZDf7F97BYPB7bZfYyQ5Ul5mtQ1Hygo4Vl5mtQ1HykpERDRaDFmg\nGY1GWCwW63FdXZ11GGNjYyPOnj2L6dOnQ61WIykpCceOHbtugdbQ0H7boQ0GN5jNttu9e7g5Ul5m\ntQ1Hygo4Vl5mtY3hysoij4iI6OYI4tW6xC5x7NgxbN68GR999BFyc3Px8ssv45NPPgEwMPzxgQce\nQGpqKjQaDZ555hksW7bMOmeNiIiIiIiIbtyQBRoAbNq0CUeOHIEgCNiwYQPy8vLg5uaGBQsWICUl\nBVu2bIFCoUBkZCT+8Ic/cJl9IiIiIiKiW3BDBRoRERERERHZ3pCrOBIREREREdHIYIFGRERERERk\nJ1igERERERER2QkWaERERERERHbipjeqHmkbN25EdnY2BEHA+vXrERcXZ33s8OHD+OMf/wi5XI6k\npCT88pe/lDDp9bPOnz8fPj4+kMsHNunetGkTvL29pYoKACgoKMC6devw6KOP4uGHHx70mL217fWy\n2lvbvvHGGzh69Ch6e3vxi1/8AgsXLrQ+Zm/ter2s9tSuHR0deP7553Hu3Dl0dXVh3bp1mDdvnvVx\ne2rXobLaU7teqrOzE0uXLsW6deuwcuVK63l7alsazJHuj4Bj3SN5f7Qd3iOHH++RtiXJ/VG0YxkZ\nGeITTzwhiqIoFhYWimvWrBn0+OLFi8Xq6mqxr69PfPDBB8WzZ89KEVMUxaGzzps3T2xtbZUi2lW1\ntbWJDz/8sPjCCy+IH3/88RWP21PbDpXVnto2PT1d/NnPfiaKoijW19eLycnJgx63p3YdKqs9tevO\nnTvF999/XxRFUaysrBQXLlw46HF7atehstpTu17qj3/8o7hy5Upxx44dg87bU9vSRY50fxRFx7pH\n8v5oO7xH2gbvkbYlxf3Rroc4pqenWze9Dg8PR1NTE1pbWwEAFRUVcHd3h6+vL2QyGZKTk5Genm6X\nWe2RSqXCBx98AKPReMVj9ta218tqb6ZPn463334bAKDVatHR0YG+vj4A9teu18tqb5YsWYKf//zn\nAICamppBn6bZW7teL6u9KioqQmFhIebOnTvovL21LV3kSPdHwLHukbw/2g7vkbbBe6TtSHV/tOsh\njhaLBbGxsdZjvV4Ps9kMV1dXmM1m6PX6QY9VVFRIERPA9bNesGHDBlRVVWHq1Kl49tlnJd3QW6FQ\nQKG4+v9+e2vb62W9wF7aVi6Xw8XFBQCwfft2JCUlWbvp7a1dr5f1Antp1wvWrl2L2tpavPfee9Zz\n9tauF1wt6wX21q6vv/46XnzxRXzxxReDzttr25Jj3R8Bx7pH8v5oO7xH2hbvkcNPqvujXRdolxMd\naE/ty7M+88wzSExMhLu7O375y1/i22+/xaJFiyRKN7rYY9vu2bMH27dvx9/+9jdJc9yIa2W1x3b9\n9NNPkZ+fj9/85jdITU2V/GZ4PdfKam/t+sUXX2DSpEkIDAyULAPdPke6PwK8R44Ue21X3iNtg/fI\n4SXl/dGuhzgajUZYLBbrcV1dHQwGw1UfM5lMknbxXy8rACxfvhyenp5QKBRISkpCQUGBFDFviL21\n7VDsrW0PHDiA9957Dx988AHc3Nys5+2xXa+VFbCvds3JyUFNTQ0AIDo6Gn19faivrwdgf+16vayA\nfbUrAOzbtw/ff/891qxZg23btuEvf/kLDh8+DMD+2pYucqT7IzB67pH22LbXY4/tynvk8OM90jak\nvD/adYE2e/ZsfPvttwCA3NxcGI1G63CIgIAAtLa2orKyEr29vdi7dy9mz55tl1lbWlrw+OOPo7u7\nGwCQlZWFcePGSZZ1KPbWttdjb23b0tKCN954A3/961/h4eEx6DF7a9frZbW3dj1y5Ij100uLxYL2\n9nbodDoA9teu18tqb+0KAG+99RZ27NiBzz77DKtXr8a6deuQkJAAwP7ali5ypPsjMHrukfbYttdi\nj+3Ke6Rt8B5pG1LeHwXRzsdFbNq0CUeOHIEgCNiwYQPy8vLg5uaGBQsWICsrC5s2bQIALFy4EI8/\n/rjdZv373/+OL774Ak5OToiJicGLL74oaddzTk4OXn/9dVRVVUGhUMDb2xvz589HQECA3bXtUFnt\nqW23bt2KzZs3IzQ01HpuxowZiIyMtLt2HSqrPbVrZ2cnfve736GmpgadnZ14+umn0djYaJfvBUNl\ntad2vdzmzZvh7+8PAHbZtjSYI90fAce5R/L+aDu8R9oG75G2N9L3R7sv0IiIiIiIiMYKux7iSERE\nRERENJawQCMiIiIiIrITLNCIiIiIiIjsBAs0IiIiIiIiO8ECjYiIiIiIyE6wQCMiIiIiIrITLNCI\niIiIiIjsBAs0IiIiIiIiO/H/Az40SBTjSfuLAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": { "tags": [] } } ] }, { "metadata": { "id": "xRphYPaVWTzY", "colab_type": "code", "outputId": "a9067c20-2b3c-434b-c3a6-983b293fcba0", "colab": { "base_uri": "https://localhost:8080/", "height": 50 } }, "cell_type": "code", "source": [ "# 测试性能\n", "trainer.run_test_loop()\n", "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n", "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))" ], "execution_count": 55, "outputs": [ { "output_type": "stream", "text": [ "Test loss: 0.50\n", "Test Accuracy: 82.6%\n" ], "name": "stdout" } ] }, { "metadata": { "id": "z_Zrw9rIWTwU", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 保存所有结果\n", "trainer.save_train_state()" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "bsWE23n-U-Cy", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 使用GloVe嵌入层" ] }, { "metadata": { "id": "HH_328MjWoM-", "colab_type": "text" }, "cell_type": "markdown", "source": [ "We just used some randomly initialized embeddings and we were able to receive decent performance. Keep in mind that this may not always be the case and we may overfit on other datasets with this approach. We're now going to use pretrained GloVe embeddings to initialize our embeddings. We will train our model on the supervised task and assess the performance by first freezing these embeddings (so they don't change during training) and then not freezing them and allowing them to be trained. \n", "\n", "我们上面使用了一些随机初始化的嵌入层，获得还挺不错的性能。但是这种情况可能并不总是如此，而且我们使用这种方法在其他数据集可能会过拟合。现在我们要使用预训练好的GloVe嵌入层来初始化我们的嵌入层。我们将在监督任务上训练我们的模型，先冻结这些嵌入层(这样它们在训练期间就不会有任何变化)，然后解除冻结并允许它们进行训练，用这种方法来评估性能。\n", "\n", "```python\n", "pretrained_embeddings = torch.from_numpy(pretrained_embeddings).float()\n", "self.embeddings = nn.Embedding(embedding_dim=embedding_dim, \n", " num_embeddings=num_embeddings, \n", " padding_idx=padding_idx, \n", " _weight=pretrained_embeddings)\n", "```" ] }, { "metadata": { "id": "r1EAWPBEKoaS", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "def load_glove_embeddings(embeddings_file):\n", " word_to_idx = {}\n", " embeddings = []\n", "\n", " with open(embeddings_file, \"r\") as fp:\n", " for index, line in enumerate(fp):\n", " line = line.split(\" \")\n", " word = line[0]\n", " word_to_idx[word] = index\n", " embedding_i = np.array([float(val) for val in line[1:]])\n", " embeddings.append(embedding_i)\n", "\n", " return word_to_idx, np.stack(embeddings)\n", "\n", "def make_embeddings_matrix(words):\n", " word_to_idx, glove_embeddings = load_glove_embeddings(embeddings_file)\n", " embedding_dim = glove_embeddings.shape[1]\n", " embeddings = np.zeros((len(words), embedding_dim))\n", " for i, word in enumerate(words):\n", " if word in word_to_idx:\n", " embeddings[i, :] = glove_embeddings[word_to_idx[word]]\n", " else:\n", " embedding_i = torch.zeros(1, embedding_dim)\n", " nn.init.xavier_uniform_(embedding_i)\n", " embeddings[i, :] = embedding_i\n", "\n", " return embeddings" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "FZBLYjtWKoX5", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "args.use_glove_embeddings = True" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "Gj77_BmoHA3s", "colab_type": "code", "outputId": "28b1c760-dec5-43d4-974b-46f349fc1759", "colab": { "base_uri": "https://localhost:8080/", "height": 34 } }, "cell_type": "code", "source": [ "# 初始化\n", "dataset = NewsDataset.load_dataset_and_make_vectorizer(args.split_data_file, \n", " cutoff=args.cutoff)\n", "dataset.save_vectorizer(args.vectorizer_file)\n", "vectorizer = dataset.vectorizer\n", "\n", "# 创建嵌入层\n", "embeddings = None\n", "if args.use_glove_embeddings:\n", " embeddings_file = 'glove.6B.{0}d.txt'.format(args.embedding_dim)\n", " words = vectorizer.title_vocab.token_to_idx.keys()\n", " embeddings = make_embeddings_matrix(words=words)\n", " print (\"\".format(\n", " np.shape(embeddings)[0], np.shape(embeddings)[1]))" ], "execution_count": 59, "outputs": [ { "output_type": "stream", "text": [ "\n" ], "name": "stdout" } ] }, { "metadata": { "id": "CC4ofLswt-D3", "colab_type": "code", "outputId": "69ee3335-7a8b-4c40-cfc4-f356e7826970", "colab": { "base_uri": "https://localhost:8080/", "height": 202 } }, "cell_type": "code", "source": [ "# 初始化模型\n", "model = NewsModel(embedding_dim=args.embedding_dim, \n", " num_embeddings=len(vectorizer.title_vocab), \n", " num_input_channels=args.embedding_dim, \n", " num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n", " num_classes=len(vectorizer.category_vocab), \n", " dropout_p=args.dropout_p, pretrained_embeddings=embeddings, \n", " padding_idx=vectorizer.title_vocab.mask_index)\n", "print (model.named_modules)" ], "execution_count": 60, "outputs": [ { "output_type": "stream", "text": [ "\n" ], "name": "stdout" } ] }, { "metadata": { "id": "44gVZbKbHA7R", "colab_type": "code", "outputId": "178befb1-6fed-4c0a-bdc9-6c6430fd4101", "colab": { "base_uri": "https://localhost:8080/", "height": 101 } }, "cell_type": "code", "source": [ "# 训练\n", "trainer = Trainer(dataset=dataset, model=model, \n", " model_state_file=args.model_state_file, \n", " save_dir=args.save_dir, device=args.device,\n", " shuffle=args.shuffle, num_epochs=args.num_epochs, \n", " batch_size=args.batch_size, learning_rate=args.learning_rate, \n", " early_stopping_criteria=args.early_stopping_criteria)\n", "trainer.run_train_loop()" ], "execution_count": 61, "outputs": [ { "output_type": "stream", "text": [ "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 0.50 | [TRAIN ACC]: 81.9% | [VAL LOSS]: 0.44 | [VAL ACC]: 84.2%\n", "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 0.41 | [TRAIN ACC]: 85.2% | [VAL LOSS]: 0.45 | [VAL ACC]: 84.0%\n", "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 0.37 | [TRAIN ACC]: 86.6% | [VAL LOSS]: 0.44 | [VAL ACC]: 84.5%\n", "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 0.33 | [TRAIN ACC]: 87.8% | [VAL LOSS]: 0.44 | [VAL ACC]: 84.1%\n", "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 0.29 | [TRAIN ACC]: 89.3% | [VAL LOSS]: 0.47 | [VAL ACC]: 83.9%\n" ], "name": "stdout" } ] }, { "metadata": { "id": "98MqlEQ0sfTs", "colab_type": "code", "outputId": "ecadfc5a-b766-422a-d608-9f4121587696", "colab": { "base_uri": "https://localhost:8080/", "height": 335 } }, "cell_type": "code", "source": [ "# 性能展示\n", "trainer.plot_performance()" ], "execution_count": 62, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA24AAAE+CAYAAAD1QEO5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xdg1PXh//HnJbmsyyVc9iKDJBC2\nQFiyA5ERHCAKrtJqh99C+63V/viaqljbot/vV/v9qlSrrRZHv4piABVcTEU2YUOABMgCsvce9/sD\njCArQJLLJa/HP+Qu97l75Uy8e917fAxWq9WKiIiIiIiIdFgOtg4gIiIiIiIiV6biJiIiIiIi0sGp\nuImIiIiIiHRwKm4iIiIiIiIdnIqbiIiIiIhIB6fiJiIiIiIi0sGpuIm0ol69enHmzBlbxxAREWk3\nc+bM4bbbbrN1DJFOT8VNRERERK7L0aNHMZvNBAcHs3v3blvHEenUVNxE2kFtbS1PPfUUkydPZurU\nqTz33HM0NjYC8O677zJ16lSmTJnCrFmzOHbs2BWvFxER6SiWL1/OlClTmD59OitWrGi+fsWKFUye\nPJnJkyfzu9/9jrq6ustev23bNhISEpqPPf/yyy+/zBNPPMGsWbNYsmQJTU1N/OEPf2Dy5MnEx8fz\nu9/9jvr6egCKiop4+OGHmThxIrfeeiubNm1iw4YNTJ8+/YLMM2fOZM2aNW391Ii0OidbBxDpCt56\n6y3OnDnDqlWraGho4P777+fTTz9l4sSJvPjii6xfvx4PDw8+++wzNmzYQFBQ0CWvj4mJsfWPIiIi\nAkBjYyNfffUV8+bNw9HRkRdeeIG6ujry8vL4z//8T1asWIG/vz+/+tWvePvtt5kyZcolr+/fv/8V\nH2fjxo2sXLkSb29vvvjiC3bu3Mmnn35KU1MTM2bMYPXq1dx+++288MILREVF8be//Y1Dhw7xk5/8\nhG+++Yb8/HxSU1OJjY3l1KlTZGZmMnbs2HZ6lkRaj4qbSDvYsGEDDz74IE5OTjg5OXHrrbfy7bff\nMm3aNAwGA8uWLWP69OlMnToVgPr6+kteLyIi0lFs2rSJ/v374+HhAcCwYcNYv349JSUlDBo0iICA\nAABeeOEFHB0d+eijjy55/a5du674OAMHDsTb2xuAyZMnM2HCBIxGIwD9+/cnKysLOFvw/v73vwPQ\np08f1q5di7OzM5MnT2bVqlXExsayZs0aJk6ciLOzc+s/ISJtTFMlRdpBUVERXl5ezZe9vLwoLCzE\naDSyZMkSUlJSmDx5Mvfeey9Hjhy57PUiIiIdRXJyMhs2bCAuLo64uDi+/PJLli9fTnFxMZ6ens23\nc3FxwcnJ6bLXX835r59FRUUsWLCAyZMnM2XKFNauXYvVagWgpKQEs9ncfNvvCmViYiKrVq0CYM2a\nNUybNu3GfnARG1FxE2kHvr6+lJSUNF8uKSnB19cXOPup4EsvvcSWLVsYPXo0CxcuvOL1IiIitlZa\nWsr27dvZtm0bO3fuZOfOnezYsYP9+/fj4OBAcXFx820rKiooKCjAYrFc8npHR8fmdd8AZWVll33c\n//mf/8HJyYlPPvmEzz//nHHjxjV/r1u3bhfcf3Z2NvX19QwdOpSGhgbWr1/PsWPHuPnmm1vraRBp\nVypuIu1g/PjxLFu2jMbGRqqqqli5ciXjxo3jyJEj/PrXv6aurg5nZ2f69euHwWC47PUiIiIdwapV\nqxgxYsQFUw6dnJwYPXo0dXV1pKSkkJ2djdVqZeHChSxbtoxx48Zd8no/Pz/y8/MpLCyksbGRTz75\n5LKPW1hYSM+ePXF2diY1NZXdu3dTVVUFQHx8PMuXLwcgLS2NmTNn0tjYiIODA9OmTeOPf/wj8fHx\nzdMsReyN1riJtLIHHngAR0fH5st/+tOfeOCBB8jKyiIxMRGDwcCUKVOa162FhoYyffp0jEYjJpOJ\np556ip49e17yehERkY5gxYoVzJ0796LrExISeOWVV3jmmWeYO3cujo6O9O/fn5/85Ce4uLhc9vo7\n77yTO+64g+DgYG6//XYOHz58ycd98MEHWbBgAcnJycTFxbFgwQJ+//vfM2DAAH73u9+xYMEC4uPj\nMZlMPP/887i6ugJnp0v+85//1DRJsWsG63cTg0VEREREOqGCggJmzJjBhg0bLvhwVcSeaKqkiIiI\niHRqL730Evfcc49Km9g1FTcRERER6ZQKCgqYOHEiBQUFPPjgg7aOI3JDNFVSRERERESkg9OIm4iI\niIiISAen4iYiIiIiItLBdZjTAeTnl9/wfVgs7hQXV7VCmrZnT1nBvvIqa9uwp6xgX3m7WlY/P3Mr\npekaWuP1Ebre71l7Uda2Y095lbVt2FNWaPvXyE414ubkZD87BdlTVrCvvMraNuwpK9hXXmWV9mBP\n/+2UtW3YU1awr7zK2jbsKSu0fd5OVdxEREREREQ6IxU3ERERERGRDk7FTUREREREpINTcRMRERER\nEengVNxEREREREQ6OBU3ERERERGRDk7FTUREREREpINTcRMRETZsWNui2/35z3/m1KmcNk4jIiIi\nP6TiJiLSxZ0+fYo1a75o0W1///vfExwc0saJRERE5IecWnKjRYsWsXfvXgwGA0lJSQwYMKD5e/Hx\n8QQGBuLoePZM4c8//zwBAQFXPKYtnDxTxtFT5fQMNrfp44iIdDZ/+ct/cvjwQcaMGcott0zl9OlT\n/O//vsKzzz5Dfn4e1dXVPPjgzxk1agwPPPAA8+f/lvXr11JZWUFmZgY5Odn8+tePMnLkKFv/KCIi\nIu2uqKyGXUfzuX18TJs+zlWL2/bt28nIyGDp0qWkp6eTlJTE0qVLL7jN3//+d0wm0zUd09o+35bJ\n9sN5/OzWPozsG9imjyUi0pncc88DJCd/QGRkFJmZJ3nllX9QXFzEsGEjmDp1Ojk52Tz55H8watSY\nC47Ly8vl+edfYuvWzaxc+ZGKm4iIdCm1dY18ti2Dz7dlUtfQRM8IH8J93dvs8a5a3LZs2cKkSZMA\niIqKorS0lIqKCjw8PFr1mBt1++hIDpwo4q3PUgnxNREWoJE3EbE/H6xLY0dqXqve59BYf+6Oj27R\nbXv37guA2ezJ4cMH+fjjZAwGB8rKSi+67YABNwHg7+9PRUVF6wUWERHpwJqsVrYcOMNHG9MpqajD\ny+TMfbf0YEisPwUFbfd6eNU1bgUFBVgslubL3t7e5OfnX3CbhQsXcs899/D8889jtVpbdExrC/Ix\n8dt7BlPX0MTi5P1UVNe36eOJiHRGRqMRgK+++pyysjL++td/sGjR85e87XdT5AGsVmu75BMREbGl\nY9kl/Omtnbyx6jCVNQ1MvzmcZ38xgjEDgjEYDG362C1a43a+H744//rXv2bMmDF4eXkxb948vvji\n4gXuLXlBt1jccXJyvOrtrsTPz8ychF68/9UR3vwslad/NhJHh7Z9Am+En599jQraU15lbRv2lBXs\nK+93WefNHtTuj+3t7YGjowGTyQUPD1f8/Mw0NFQTHR1JQIAXGzZ8TmNjQ3NGi8V0wW2Li004OzvZ\n1fMtIiJyLQpKqvlwQ3rzrJjhfQKYNS4KHy/Xdstw1eLm7+9PQUFB8+W8vDz8/PyaL99xxx3NX48d\nO5ajR49e9ZhLKS6uuqbgl+LnZ2bS4GAOHy9gz9F8XvtoD3eNb9n0oPbm52cmP7/c1jFazJ7yKmvb\nsKesYF95bZ3VyyuA/fsP4OPjj9HoRn5+OXFxo/iP//gtO3bsIjHxNnx9/fiv//oLAMXFlVRW1mI0\n1pCfX05xcSV1dQ0t/hlU8ERExF5U1zawaksGX+7IoqGxiR7BnsyZGEN0iFe7Z7lqcRs1ahQvv/wy\nc+bM4eDBg/j7+zevVSsvL+c3v/kNr776Ks7OzuzYsYPJkycTEBBw2WPamoPBwM9u7cMf39rJZ1sz\niQj0ZGisf7s8toiIPbJYLCQnr7rguqCgYN566/3my7fcMhX4vmT26PH9h2I9ekSzePHr7RNWRESk\nHTQ1Wdm0/zTJXx+nrLIOi9mFWeOjGN4nAIc2nhJ5OVctboMHD6Zv377MmTMHg8HAwoULSU5Oxmw2\nk5CQwNixY5k9ezYuLi706dOHKVOmYDAYLjqmPbm7Gpk/sz9/ensXb646TJCPO6F+7VMcRUREvlNZ\nWcmCBQsoLS2lvr6eefPmYTKZeO655zAajQwZMoTf/va3to4pIiLnOXyyiPfXpZGVV4Gz0YE7xkQy\neVgYLsYbW9Z1o1q0xu2xxx674HJsbGzz13PnzmXu3LlXPaa9hfh58FBib15ZcYDFyft5am4c7q5G\nm2YSEZGuZfny5URGRvLoo4+Sm5vL3LlzcXZ25i9/+QvR0dEkJSWRkpLC4MGDbR1VRKTLyy2q4oP1\naew+dnbJ16h+gcwcF4XF7GLjZGdddVdJexYX68/UEWHkFVfz+ieHaNKuZyIi0o4sFgslJSUAlJWV\nYbFYyM/PJzr67FTT0aNH8+2339oyoohIl1dZU8/7a4/xxD+2sftYATGhXjw5N46HpvfpMKUNrmNX\nSXtz59goMnMr2JdeyMebTnDHmB62jiQiIl1EYmIiycnJJCQkUFZWxmuvvcaf//xnduzYQVxcHJs3\nb77gtAoiItJ+Gpua2LD7FCs3naCiuh5fL1funhDNkF5+bb61//Xo9MXNwcHAL27ryzNLdvDxtycJ\nDzQzKObKO1yKiIi0hpUrVxIcHMwbb7xBamoqSUlJPPfcc/z5z3/G0dGR2NjYq568vDVOl/Mde9rR\nU1nbhj1lBfvKq6xto62y7krN5Y2PD5CVW4GbixM/TuzDrWN64HyD69ja8rnt9MUNwMPt7GYli97Z\nxd8/OcSTc+MI8jHZOpaIiHRyKSkpjB49Gji7PjwvL4+oqCjeeustAN5//33KysqueB+tcbocsP1p\nJ66FsrYNe8oK9pVXWdtGW2TNKahk6bpjHDhehMEA424K5o4xPfAyOVNacmP/v22NvFcqfp16jdv5\nwgLM/HhqLDV1jSxO3k91bYOtI4mI2JVZs26lsrLS1jHsSnh4OHv37gUgJycHk8nEE088QWpqKo2N\njaxcuZLx48fbNqSISBdQXlXHu18eYeEb2zlwvIje4Rae/skw5k6JxcvkbOt4LdIlRty+M6JvICdO\nl/PVzizeWHWYX87oZ7PzMIiISOc3e/ZskpKSuP/++2loaODpp5/G2dmZxx9/HIDp06fTs2dPG6cU\nEem8GhqbWLsrm4+/PUl1bQMB3u7MnhDNwGifDrmO7Uq6VHEDuGtCFFl55aQczWf1lgym3xxh60gi\nIjb14IP3sWjRCwQGBnLmzGkef/xR/Pz8qa6upqamhkce+R19+vSzdUy7ZDKZePHFFy+6fvny5TZI\nIyLSdVitVvYcK2Dp+jTyiqtxd3FizsQY4geH4ORon5MO7TP1DXBydODh2/thMbuw/Ovj7D9eaOtI\nIiI2NXbsBL799msAvvlmI2PHTmD69Dt4+eXXePjh+fzrX2/ZOKGIiEjLZeaW89/v7ebl5P0UlNQw\ncUgozz08kluGdrfb0gZdcMQNwNPkzPyZ/Xn23RReW3mQp34ch7/F3daxRERITvuU3Xn7W/U+B/n3\nZ2b09Mt+f+zYCSxe/L/ceefdbNq0kfnzH+H999/hvffeob6+HldX11bNIyIi0hZKK+tY/nU63+w9\njRUYEOXD3ROiCfbtHJsS2m/lvEGRQZ48MLknVbUNLE7eT21do60jiYjYRI8eURQW5pObe4by8nK+\n+WYDvr7+vPrqGzz22H/YOp6IiMgV1Tc0smrLSR5/bQtf7z1NkK+J3949kN/cNbDTlDbooiNu3xkz\nIJiTp8tZvzuHf352mF/c1tfuFimKSOcyM3r6FUfH2srIkaN5/fVXGDNmHCUlxURFxQCwceN6Ghq0\nC6+IiHQ8VquVnUfy+XB9GgWlNXi4GXnglijG3hSMo0PnG5/q0sUN4J5JMWTlVbD9cB4RgZ5MGR5m\n60giIu1u3LgJPPzwgyxZ8h41NdX86U8LWb9+DXfeeTdr1nzJqlUf2zqiiIhIsxOny3h/7TGOZZfi\n6GBg8rDu3HpzBO6uRltHazNdvrg5OTrwyxn9+MOSHXy4IY3wAA96R3jbOpaISLvq3bsvGzdua778\nr38ta/569OhxACQm3obJZKKqyj5O3CoiIp1PcXktH21MZ/OBMwAMivHl7vhoArrAfhWdbwzxOnTz\ncGHeHf1xMBh4deVBCkqrbR1JRERERETOqa1vZOWmEzz++hY2HzhDmL8Hv7tnEL+6c0CXKG2gEbdm\n0aFe3JvQk3e+OMJfkw/w+P2DcTY62jqWiIiIiEiX1WS1su1gLss2plNcXounyZl7J/VgdP8gHBy6\n1t4UKm7nGX9TMCdOl7Fp32ne/uIIDyX21mYlIiIiIiI2cPhEEa9+tIcTp8txcnQgcWQ400aE4+bS\nNStM1/ypL8NgMPDALT3Jya9g84EzRAZ5MnFIqK1jiYiIiIh0GQWl1SzbkM72w3kADOvtz6xxUfh2\nc7NxMttScfsBo5Mj82b05w9LdvD+2mN09/egZ/duto4lIiIiItKpVdc2sHprBl9sz6KhsYmeYd2Y\nNTaK6FAvW0frELQ5ySV4e7ryyzv6YbXCKysOUFxea+tIIiIiIiKdUlOTla/3nuLx17eyaksGZncj\nP5veh//+1ViVtvNoxO0yeoVZmB0fzXtrj/HX5ftZcO9gjE7quSIiIiIireVwRjFL1x4jM68CZ6MD\nd4yOZPLwMFyMjl1u85GrUXG7gklxoZw4U8bWg7n835qjzJ0Sa+tIIiIiIiJ2L7e4ig/WpbH7WAEA\nN/cL5M5xUVjMLjZO1nGpuF2BwWBg7pRYTuVXsnHPKSICzYy7KcTWsURERERE7FJVTT2fbD7Jmp3Z\nNDZZiQ714p6JMUQGedo6Woen4nYVLkZH5s3szzNLdvCvr44S6u9BVLDm2oqIiIiItFRjUxNf7znF\n8m9OUFFdj4+nK3fHRxPXy0+n32ohLdpqAb9ubjx8ez8am6y8svwApRXarEREREREpCUOHC/k6Td3\n8M6XR6lvbOLOcT1Y9PPhDI31V2m7Bhpxa6G+kd7MGhfFhxvSeXXFAR67ZxBOjuq9IiIiIiKXcqqg\nkg/Wp7EvvRADMHZgEDPG9MDLQ+vYroeK2zWYMjyME2fK2Zmax9J1adyX0NPWkUREREREOpSK6npW\nfnOC9btzaLJaiQ3rxpyJMYQFmG0dza6puF0Dg8HAg9NiOV1Qydpd2UQEmhnVP8jWsUREREREbK6h\nsYl1KTl8vOkEVbUN+FvcmD0hmptifDUlshWouF0jV2cn5s/szzNv7eTtL44Q6udBeKA+PRARERGR\nrslqtbI3rZCl646RW1yNm4sTc+KjiR8SqqVFrUjP5HUI8Hbn57f2oaGhicXJ+yivqrN1JBERERGR\ndpeVV8Hz7+/hpY/2kV9SQ/zgEJ77xQhuGRam0tbKNOJ2nQZG+3L76EhWbDrB31Ye5LezB+LooF9O\nEREREen8SivrWP71cb7ZdwqrFfr18GZ2fAwhviZbR+u0VNxuwPRREZw8U86etAI+2nCcu+OjbR1J\nRERERKTN1Dc08tXObD7dfJKaukaCfNyZMzGG/j18bB2t01NxuwEOBgM/nd6HP769k8+3ZxIRZGZY\n7wBbxxIRERERaVVWq5VdR/L5YH0aBaU1eLgZuS8hivGDgjXrrJ2ouN0gd1cnfjWzP398eydvrj5M\nkI+J7v4eto4lIiIdQGVlJQsWLKC0tJT6+nrmzZtHVVUVb775JkajkYCAAJ599lmcnZ1tHVVE5LJO\nninj/TXHOJpdiqODgVuGdufWURGYXI22jtalqLi1gmBfEz9N7MNfl+9ncfI+npw7FA83/SKLiP1r\nsjZxsiyLQ4WppBYdo29QT6aG3GLrWHZj+fLlREZG8uijj5Kbm8vcuXOprKxk9erVmM1mnnzySb76\n6isSExNtHVVE5CLF5bUkb0zn2wNnABgU48vdE6IJ8Ha3cbKuScWtlQzp5UfiyHBWbcng9U8O8ptZ\nA3Fw0PkqRMT+lNdVcKjwCIeKjnC48CiVDVUAOBgcGBjS28bp7IvFYuHIkSMAlJWVYbFYMBqNlJWV\nYTabm68TEelIausb+WJbJqu3ZVBX30R3fw/mxEfTO8Lb1tG6NBW3VjRjTA8ycss5cLyIFZuOM3Ns\nlK0jiYhc1fmjagcLj5BVnoMVKwDdXLwY5T+MPj6x9LJEExbkR35+uY0T24/ExESSk5NJSEigrKyM\n1157jdraWmbMmIHZbKZPnz7cfPPNto4pIgJAk9XKtkO5LNuQTnF5LZ4mZ+6d1IPR/YM0INEBqLi1\nIgcHA7+4rS/PLNnBp5szCA/wZEgvP1vHEhG5yJVG1aK7RdLXJ5Y+Pr0INgViMOjF+nqtXLmS4OBg\n3njjDVJTU3n88cdpaGhg2bJldO/end/85jesXbuWiRMnXvY+LBZ3nJwcWyWPn5+5Ve6nPShr27Cn\nrGBfee09a+rJIv6+cj9HM0swOjkwKz6GuybG4G7jdWz29LxC2+ZVcWtlJlcj82cO4M/v7OQfqw4R\n5BNHsM5nISI2di2jam5OrjZO23mkpKQwevRoAGJjY0lPTyc8PJywsDAARo4cyYEDB65Y3IqLq1ol\ni5+f2W5GS5W1bdhTVrCvvPactaC0mmUb0tl+OA+AuFh/7hofhV83NyrLa6gsr7FVVLt6XqF18l6p\n+Km4tYHu/h78ZGpvXvv4IIuT9/PEj+Jwd9VTLSLtS6NqthceHs7evXuZPHkyOTk5BAYGUlpaSlFR\nEd7e3uzfv5+hQ4faOqaIdEE1dQ2s3prBF9uzqG9oIiLQzJyJMfTs3s3W0eQy1CbayPA+AZw8U8YX\n27P4x6eHmH9nfxz0xkhE2pBG1Tqe2bNnk5SUxP33309DQwN/+MMfqKys5OGHH8bZ2ZnQ0FDtKCki\n7aqpyco3+06RvPE4pZV1dPNw5s5xUYzsF6j3qh2cilsbmjU+iszcCvakFfDp5pPcNirS1pFEpJPR\nqFrHZjKZePHFFy+6ftKkSTZIIyJdmdVq5VBGMcvf2cXxnFKcnRy4bVQEU4eH4+LcOutopW2puLUh\nRwcHHr797GYlK785QXiAmYHRvraOJSJ2TKNqIiJyLZqsVvYcK2DVlgxOnC4DYGTfAO4cF4W3p14n\n7ImKWxszuzszb2Z/Fr2TwuufHOKpuXE6aaGIXJOWjKr19YklyBSgUTUREQGgsamJ7YfyWLU1g1MF\nlQAM7unHA9P64OWqETZ7pOLWDiICPZk7pRdvrDrM4uT9/P5HQ2wdSUQ6MI2qiYjI9aqrb2TT/tN8\nvi2TgtIaHAwGbu4XyNQR4YT4muxup0b5nopbOxnVP4iTp8tZm5LNm6tTeeqnI2wdSUQ6EI2qiYjI\njaiubWDD7hy+2JFFWWUdRicH4geHMGVYGL7d3GwdT1qBils7mj0xmqy8cnam5pG8Po2x/QNtHUlE\nbOT8UbWju9M4XpypUTUREblmZVV1rNmZxbpdOVTVNuDq7Mi0EeEkDO2Ol8nZ1vGkFam4tSMnRwf+\n7Y5+/GHJDt5efQgfD2f6RnrbOpaItJPLjao5alRNRESuUVFZDZ9vy+Trvaeoa2jC7G5k5tgexA8O\nwd3VaOt40gZU3NqZl4cL82b05z//L4W/rTzAUz8eip+Gr0U6pZauVRsVcxOVJQ02TisiIvbgdGEl\nn23NZMvBMzQ2WfH2dGHKsDDGDAzGxahNRzozFTcbiArx4uGZA1j84V7+mryfxx8Y0mH+0BqbGimp\nLaOopoiCmmKKqosorSsjvCAYbwdfwsyhmIzaFVPkcq5nrZq70Y1KtFBcREQuL+NMOau2nGTXkXys\nQKC3O9NGhDOibwBOjg62jiftoEXFbdGiRezduxeDwUBSUhIDBgy46DYvvPACe/bs4Z133mHbtm38\n+7//OzExMQD07NmTJ598snWT27nJIyLYdzSfr/ee4q3PU/nZ9D7tMjWqydpEaW0ZhTXFFNUUU1hd\nROF5/xbXltBkbbrouG9Pff+1j6s3YeYQwjxDCTOH0t0cojInXVaTtYmMsiwOagdIERFpZVarlaNZ\nJazaksGBE0UAhAeYSRwZzuCefjg4aFp9V3LV4rZ9+3YyMjJYunQp6enpJCUlsXTp0gtuk5aWxo4d\nOzAav59PO2zYMF566aXWT9yJ3JfQk+z8CrYezCUy0JOEod1v+D6tVitldRUU1hRRVH1u1KymiMLq\nYgpriiiuKaHB2njJY72czUR4dsfb1YKPqzc+bmf/9XQ2U2OsYH/2MbLKc8gsz2Z3/n525+9vPtbH\n1ftckQshzHz2X3eVOemkLhhVKzpKZb12gBQRkdZjtVrZl17Iqi0ZpOWUAtCrezcSbw6nb4S3Xlu6\nqKsWty1btjBp0iQAoqKiKC0tpaKiAg8Pj+bbPPfcczzyyCMsXry47ZJ2QkYnB+bN6M8fluxg6bo0\nwgI86BVmueIxVquVivpKCs+VsaKaYgpqiig6V8yKaoqpb7r0Whmz0YMQczC+rt5ny5mbNz6uFnxc\nLXi7WjA6Xn4hq5+fmR4u0c0ZimqKyTxX4prLXN4+duftaz7G19Wb7ipz0gl8P6p2hIOFqRePqgVr\nVE1ERG5cU5OVHal5rNqSQXZ+BQADo3xIHBlBdKiXjdOJrV21uBUUFNC3b9/my97e3uTn5zcXt+Tk\nZIYNG0ZISMgFx6WlpfHwww9TWlrK/PnzGTVqVCtH7xwsZhd+eUc//vu93byy4gBPzY3D1d3aXMy+\nL2jnpjTWFFPXWHfJ+zIZ3QkyBeDj6o33udEyn3MFzdvVgotj62wJazAYzpY+N28G+fcHLlPmylTm\nxH5pVE1ERNpLfUMTmw+c5rOtmeSVVGMwwIg+AUwbEU6ov8fV70C6hGvenMRqtTZ/XVJSQnJyMv/8\n5z/Jzc1tvj4iIoL58+czdepUsrKy+NGPfsSXX36Js/Pli4PF4o6T041v0OHnZ77h+2hrVXXVnCzO\nJq+2gPzKQvIppMfoLDKLclm443OsDvWXPM5kdCPY7I+fyQd/ky9+Jm/8Tb74m3zwNXnjbmzb3Smv\n9tz640ks4c2XrVYr+VVFHC+rit6eAAAgAElEQVTK4HhxJseLMjlenHlRmQsw+RLpHUaUJZwe3mFE\nWrrj4Wxq06wdibK2nWvJ29TURFrRSfacOcjuUwcvOK+aj5uFEd0HMyioL/0CerXJ35o9Pbf2lFVE\npCOrqWtg455TfLE9k5KKOpwcDYy/KZgpw8Pwt+iDbbnQVYubv78/BQUFzZfz8vLw8/MDYOvWrRQV\nFXHfffdRV1dHZmYmixYtIikpiWnTpgEQFhaGr68vubm5dO9++TVcxcVVN/qz4OdnJj/f9juz1TTU\nnt344/xRs3M7NBbWFFPVUH3J45zcnGiodsXiEsig8LCzUxjdvJtHzi77ZrEeKksa2nRXuut9bg04\nE+UaQ1RQDARdPDKXWXZ2dG5rVgpbs1Kaj/NtXjN3dvOTaxmZ6yi/By2hrG2nJXmvNKoW060HfXx6\nXTSq1hZ/a/b03LZGVhU/EenqKqrrWbsrmzU7s6isacDF6MiUYWEkDO2Oxexi63jSQV21uI0aNYqX\nX36ZOXPmcPDgQfz9/ZunSU6ZMoUpU6YAkJ2dzeOPP05SUhIff/wx+fn5PPTQQ+Tn51NYWEhAQEDb\n/iTtqK6x/vupi+cVs8Lqs2vMKuorL3mc0cGIj5s3kV7hhFj8MWHG29Vyds2ZmwWj1YXn/rWbjNxy\nfH17MaFnyCXvx55daZplxnlTLLPKc0jJ20fK+dMs3Xyap1hea5kTAa1VExER2your+XLHZls2H2K\n2vpGTK5O3DE6kvghoXi46aTZcmVXLW6DBw+mb9++zJkzB4PBwMKFC0lOTsZsNpOQkHDJY+Lj43ns\nscdYu3Yt9fX1PP3001ecJtnR1Dc1UHyZUlZQU0R5XcUlj3NycMLbtRvdzSHNa8vObvzhja+bNx5G\nU/On9pf71HrezH48s2Qn//fVUbr7exAd0vkXop5f5gb7nz3VhNVqpbCm+IL1cpnl2S0qc6BP8+V7\n1zOqJiIi0pryiqv4bFsm3+4/TUOjlW4ezswYE8nYm4JxddZplaVlDNbzF63ZUGtME2rpFJ7GpkaK\na0soOFfGCn9wPrOyuvLmT+HP52hwxOLa7bxdGS/cNt/s7IGDoWUnQLxS1sMni3h+6R48Tc4s/PFQ\nunnYfsi8I0zlulyZ++HU0wAPP0Lcg85tfnK20LX1+r/r1RGe15ayh6yNTY2U11dQWF1MZu1Jtmfu\nu2hUra9Prw43qmYPz+13NFWy/bXW70ZX+z1rL8raduwp7+WyZuVVsHprBtsP52K1gn83N6aNDGdk\n30CMTrY5aXZneF47qrZ+jeyUFb+xqZGS2jKKas6dx+y7UnZuzVlJbekli5mDwQGLixfR3SLP2yrf\nu/lrLxfPFhezG9E7wpu7xkfzwfo0Xll+gP937yCcHG3zx92RGAwGfN3Ojl5eamTuuymWWZU5pFRc\nPDIX3jwq17HLnFzouzJWXldBWV0FZXXllNeVn7tcTlldRfPlH05T1qiaiIjYQlp2Kau2nGRveiEA\noX4eJI4MJy7WD0cHvaeT69Npitve/IN8e2ALZ8oKKK4tocnadNFtDBjo5uJFD68IfM9tkX/+lMZu\nLl44Otz4zpatYfKw7pw8U8b2w3m8t/YYD9zSy9aROqRLlTlfXw9SszIvKHOZ5dnsytvLrry9zcf6\nufmcN8VSZa49/bCMldeVnytkF5exyvqqS37Qcj53JzfMzmaCTAF4OpvxdDEzJKwvgY4hHWZUTURE\nOjer1crBE0Ws2pLBkawSAKJDvZg+Mpz+PXz0waHcsE5T3FKLjnEw7yhezp5EeHa/YNOPs7syemNx\n9cLJwT5+ZIPBwE+m9uZUQSXrU3KICDQzZkCwrWPZhcuPzBWd3c3yXJnLuEqZC/c8+6+bk8pcSzQ2\nNVJRX3lR8Sr7QSm7njJmdvbA09mM2dmMZ/PXZ//1cPbAeIm/a3ubXiEiIvapqcnKt3tP8d4XqWTk\nnn3d6dfDm+kjI+jZvZuN00lnYh8tpgVm97qDh0feQ3HRpbfat0cuzo7Mn9mfZ5bs5J0vjhLq50Fk\nkKetY9mls2XOB183n8uWuczybDLLcy5b5sLOnTi8K5W578tYBTkNDWTl511Uxr673JIy5ubkhud5\nZexsETtbxr4rYlcqYyIiIh1FQ2MTWw6e4bOtmZwpqsIAxMX6kzginPBAreWV1tep3hk5OXaqHwcA\nf4s7v7i9L//7wV7+unw/T80diqfJfnbo7Mi6apk7v4xdvF7sesuYh8qYiIh0CbX1jXy99+xJs4vK\nanF0MJAwLIzxA4MI8jHZOp50YnoXZQf69/BhxtgeJH99nL+tPMCjc27SwtY2cqUyl3HeerlLlTl/\nN9+z6+VsUObOL2MXrxe7/jIWaPJvnqIYZPHBsd75vGmLZ0uaypiIiHQFVTX1rE3J4asdWVRU1+Ps\n5MCkuFCmDAujV5SfpudLm9M7LjuRODKck2fKSTmaz4fr05kzMcbWkbqM88vckICBwNkyV1Bd9P2p\nCVpU5kLpbg5ucZn7YRm73HqxGyljPxwVMzt7YDZ6YHS8+CSgWjMmIiJdUWllHV/uyGR9Sg41dY24\nuzgx/eYIJsWF4umuWVDSflTc7ITBYOChxN6cLqzkyx1ZRASaGdE30NaxuiyDwYCfuw9+7pcvcxnn\n/r1SmQuq8OFUYcFFo2ItL2OueDqbCTT5X3aK4pXKmIi0rcrKShYsWEBpaSn19fXMmzeP119/vfn7\neXl5zJgxg4cfftiGKUXkUgpKqvlseyab9p2mvqEJT5Mzt46KYPxNIbi56C20tD/91tkRNxcn5s/s\nzx/f2smSz1IJ9jURFqDFrx3F1crcd6NyWZfYzfJ8F5exC3dRVBkTsR/Lly8nMjKSRx99lNzcXObO\nncvnn3/e/P2f/vSn3H777TZMKCI/lFNQyeotGWw7lEuT1YqvlytTh4cxekAQRqeOcdoo6ZpU3OxM\nkI+Jn93ah5c/2s/i5P089eOheLjpzXtHdeUyl4XRzYChzvj9ujGVMZFOxWKxcOTIEQDKysqwWCzN\n39u8eTMREREEBQXZKp6InOf4qTJWbTnJ7mMFAIT4mpg2Ipxhffy1t4B0CCpudmhQjB+3jYrg429P\n8trHB3nkroE4OOikjvbi/DKndWMinVtiYiLJyckkJCRQVlbGa6+91vy9t99+m6SkJBumExGr1Upq\nRjGfbsngcEYxAJFBnkwfGc7AGF8cdNJs6UBU3OzUbaMjOXmmnH3phSR/fZxZ46NsHUlERH5g5cqV\nBAcH88Ybb5CamkpSUhLJycnk5uZSVVVFWFjYVe/DYnHHqZWmZ/n52c/0emVtG/aUFdoub1OTlW0H\nz7Bs3VGOZpYAcFOMH7MmxjAg2hfDdRQ2e3pulbXttGVeFTc75WAw8PNb+/DMWztZvTWDiEAzcbH+\nto4lIiLnSUlJYfTo0QDExsaSl5dHY2MjGzduZMSIES26j+LiqlbJYk8j/MraNuwpK7RN3samJrYf\nymPV1gxOFVQCMLinH4kjw4kM8gSgoKCiQ2RtK8radloj75WKn4qbHXN3NTJ/Zn/+/PYu3lh1mCAf\nd0L8PGwdS0REzgkPD2fv3r1MnjyZnJwcTCYTjo6O7N+/nwkTJtg6nkiXUd/QyKZ9p/lsWyYFpTU4\nGAzc3C+QqSPCCfHVSbPFPqi42blQPw8eTOzNqysOsDh5P0/OjcPdVZtbiIh0BLNnzyYpKYn777+f\nhoYGnn76aQDy8/Px8fGxbTiRLqC6toENu3P4YkcWZZV1GJ0ciB8cwpRhYfh2a9l5VUU6ChW3TmBo\nrD8nh4fx2bZM/v7JIX41a4AW04qIdAAmk4kXX3zxouv/9re/2SCNSNdRVlXHmp3ZrNuVTVVtA67O\njkwbEU7C0O54mXTSbLFPKm6dxJ3josjMLWdveiEfbzrBHWN62DqSiIiISLsqKqvh8+2ZfL3nFHUN\nTZjdjcwc24P4wSGakSR2T8Wtk3BwMPCL2/vxzJIdfPztScIDzQyK8bN1LBEREZE2d6aoitVbM9hy\n4AyNTVa8PV2YMiyMMQODcTHqpNnSOai4dSIebmc3K1n0zi7+8ekhnvhRHEE+WnArIiIinVPGmXJW\nbTnJriP5WIFAb3emjQhnRN8AnBx10mzpXFTcOpmwADNzp8by908OsTh5P0/8KA43F/1nFhERkc7B\narVyNKuEVVsyOHCiCIDwQDOJI8IZ3NMPBwet85fOSe/oO6GRfQM5cbqMNTuzeXPVYX45o991nUhS\nREREpKOwWq3sSy9k1dYM0rJLAYgN68a0keH0jfDWex3p9FTcOqm7J0STlVvBrqP5rN6aQeLICFtH\nEhEREblmTU1WdqTmsXprBll5Z0+OfVO0L9NGhhMd4mXjdCLtR8Wtk3JydODf7ujHH5bsIHnjccIC\nzPTvoXMGiYiIiH2ob2hk454cPtuaSV5JNQYDjOgTwLQR4YT6e9g6nki7U3HrxDxNzsyf2Z9n303h\n9Y8P8uSPh+Kvk02KiIhIB1bf0Mj6lBy+3JlNUVkNTo4Gxt8UzJThYfhb3G0dT8RmVNw6ucggTx64\npSf//CyVxR/t5/cPDMHFWdviioiISMditVrZfjiPZRvSKSyrwdXZkSnDwkgY2h2L2cXW8URsTsWt\nCxgzMJgTZ8rZsDuHJZ+n8vNb+2gBr4iIiHQY6adKeX/tMdJzynByNDBleBg/mt6XmspaW0cT6TBU\n3LqIeyfFkJ1XwbZDuUQEmpk8LMzWkURERKSLKyqrYdmGdLYeygVgSC8/7hofhb/FHbO7s4qbyHlU\n3LqI7zYreWbJDj5cn06Yvwe9I7xtHUtERES6oJq6BlZvzeSL7ZnUNzQRHmBmzsRoeoVZbB1NpMPS\nKeW7EIvZ5dw53eDVlQcpLK2xdSQRERHpQpqsVr7Zd4rHX9/Kp5tPYnJ14qHE3jz54ziVNpGr0Ihb\nFxMT2o17J8XwzpdHWbx8P4/fNxhnozYrERERkbZ1JLOY99YeIzO3AmcnB24bFcHU4eHaNE2khVTc\nuqDxg0I4cbqcTftP884XR3gwsbc2KxEREZE2kVdcxQfr00k5mg/AyL4B3DkuCm9PVxsnE7EvKm5d\nkMFg4IHJPcnOr+DbA2eICPJk4pBQW8cSERGRTqSqpp5PNp9kzc5sGpusRId4MWdiDD2CPW0dTcQu\nqbh1UUYnR+bP7M8fluzg/bXH6O7vQc/u3WwdS0REROxcY1MTG/ecYsU3J6iorsfH05W7JkQxNNZf\nM3xEboA2J+nCvD1d+bfb+2G1wisrDlBcri13RURE5PrtP17Iwjd38O6XR2lobOLOcT1Y9PPhDOsd\noNImcoM04tbFxYZbuDs+mvfXHuOV5fv5f/cOxuikPi8iIiItl1NQydJ1xzhwvAiDAcbdFMwdY3rg\nZXK2dTSRTkPFTUiIC+Xk6TK2HsrlvTVH+dGUWFtHEhERETtQXlXHik0n2Lj7FE1WK30iLMyOj6G7\nv4eto4l0OipugsFgYO7UWHIKKtmw5xQRQZ6MHRhs61giIiLSQTU0NrFmZzafbD5JdW0DAd7uzI6P\nZmCUj6ZEirQRFTcBwMV4drOSZ5bs4N0vjxDiZyIq2MvWsURERKQDsVqtpBwt4MP1aeSVVGNydeKe\nSTFMGBSCk6OWWoi0Jf2FSTO/bm784va+NDZZeWX5AUor62wdSURERDqIjDPl/Pd7u/nr8v0UltUw\nKS6UZ38xkoS47iptIu1AI25ygX6RPtw5LoplG9J5dfl+HrtnkP5nLCIi0oWVVNSSvPE43+4/jRW4\nKdqXuyZEEeRjsnU0kS5FxU0uMnV4GCdPl7HzSD4frEvj3oSeto4kIiIi7ayuvpEvtmeyemsmtfWN\nhPqZmD0xhr4R3raOJtIlqbjJRQwGAw8m9uZ0YRVrdmUTEWTm9glmW8cSEbE7lZWVLFiwgNLSUurr\n65k3bx433XQTjzzyCKWlpQQEBPCXv/wFZ2dtmS4dh9VqZduhXJZtTKeorBZPdyNzJkYzZkAwDg7a\neETEVlTc5JJcnZ3Oblby1k7e+vwIfj4eRAd6aKcoEZFrsHz5ciIjI3n00UfJzc1l7ty5xMfHM3r0\naH784x+zePFiUlNTGTBggK2jigCQllPK+2uPcfxUGU6ODkwbEU7iyHDcXPSWUcTW9FcolxXg7c7P\nb+3D4uT9PPvWDnp278bs+GgigzxtHU1ExC5YLBaOHDkCQFlZGRaLhfXr1/Puu+8CMH/+fFvGE2lW\nUFrNsg3pbD+cB8DQWH9mjY/Cr5ubjZOJyHdU3OSKBkb78sxDw/h4cwbbDp7hj2/tZESfAGaO64Gv\nl/5nLiJyJYmJiSQnJ5OQkEBZWRmvvfYaP/vZz3jvvffYvHkz0dHRPPHEE5oqKTZTXdvA6q0ZfLE9\ni4bGJiKDzMyZGENMaDdbRxORH1Bxk6sK8jHxxIPD+WZnJkvXp7H1UC47j+STEBdK4shw3F2Nto4o\nItIhrVy5kuDgYN544w1SU1NJSkqitraWUaNGMX/+fJ544gk+/PBD7rvvvsveh8XijpOTY6vk8fOz\nn/XKyto2vsva2GRlzfZM3v38MCXltfh4uTI3sQ/jBoV2qHVs9vjc2gNlbTttmVfFTVosNtzCk3Pj\n2HYol482pvPZtky+2Xea20ZFMF4n3hQRuUhKSgqjR48GIDY2lry8PAIDAxk0aBAAo0aNYtu2bVe8\nj+LiqlbJ4udnJj+/vFXuq60pa9v4Luvhk0W8vy6NrLwKnI0O3DE6ksnDw3AxOlJYWGHrmM3s8bm1\nB8radloj75WKn4qbXBMHg4GRfQMZ0tOPNbuyWbXlJP+35hhrd2Uza3w0g3v6agMTEZFzwsPD2bt3\nL5MnTyYnJweTycTw4cPZunUrI0aM4ODBg0RGRto6pnQRp/IreHXZPvakFQAwql8gM8dFYTG72DiZ\niLSEiptcF2ejI9NGhDN6QBCfbDrJ+t05/HX5fmJCvZgdH0OPYG1gIiIye/ZskpKSuP/++2loaODp\np5+mV69ePPbYY7z00kv4+vryy1/+0tYxpZOrrKnn400nWZeSTWOTlZ6hXsyZFENEoF6rRexJi4rb\nokWL2Lt3LwaDgaSkpEtuW/zCCy+wZ88e3nnnnRYfI/bP092Z+27pSfyQEJZtSGf3sQL+9PZOhvX2\n585x2o1KRLo2k8nEiy++eNH1b775pg3SSFfT0NjEht05rNx0gsqaBgJ93Jk5pgdDevlpdoyIHbpq\ncdu+fTsZGRksXbqU9PR0kpKSWLp06QW3SUtLY8eOHRiNxhYfI51LkI+JX905gCOZxSxdl8b2w3mk\nHM1n0pDuJN4cjkkbmIiIiLQLq9XKvvRCPlifxunCKtxcHLlrQhT3TOlNSSutmRSR9nfV4rZlyxYm\nTZoEQFRUFKWlpVRUVODh4dF8m+eee45HHnmExYsXt/gY6Zx6hVl4Ym4c289tYPL59ky+2XeK20ZF\nMmGwNjARERFpS9n5FSxde4yDJ4sxGGD8oBDuGB2Jp8kZYyvtTioitnHV4lZQUEDfvn2bL3t7e5Of\nn99cwpKTkxk2bBghISEtPkY6NweDgRF9AxnSy481O7P5dEsG7639bgOTKE3REBERaWVllXWs+OY4\nG/eewmqFvpHezI6PJtRP771EOotr3pzEarU2f11SUkJycjL//Oc/yc3NbdExl9Na56mxp3M92FNW\nuL68P7q1G7dPiOH9r47w2eaTvLLiAL0jvHnwtr7Ehnu3Qcqz7Om5Vda2Y095lVVErkd9QxNrdmbx\n6ZaTVNc2EuTjzuz4aPr38NGHpCKdzFWLm7+/PwUFBc2X8/Ly8PPzA2Dr1q0UFRVx3333UVdXR2Zm\nJosWLbriMZfTGuepsadzPdhTVrjxvDNHR3JznwCWbUgn5Wg+v3vpG4bG+nPn+Cj8W3kDE3t6bpW1\n7dhT3q6WVcVP5MZZrVZ2Hcnng/VpFJTWYHJ14r6Enoy7KVjLEkQ6qasWt1GjRvHyyy8zZ84cDh48\niL+/f/OUxylTpjBlyhQAsrOzefzxx0lKSiIlJeWyx0jXFejtzvyZ/TmaVcLSdcfYkXp2A5OJQ0KZ\nfnMEHm7awERERORqTp4p4/01xziaXYqjg4Fbhnbn1lER2ghMpJO7anEbPHgwffv2Zc6cORgMBhYu\nXEhycjJms5mEhIQWHyPynZ7du/H7H8Wx43AeH21M58sdWXy7/zS33hzBhMGhGJ30SaGIiMgPFZfX\n8tHGdDYfOAPAoBhf7p4QTYC3u42TiUh7aNEat8cee+yCy7GxsRfdJjQ0tPkcbpc6RuR8DgYDw/sE\nMLinH2t3ZfPJ5pO8vy6NtSnZzBofTZw2MBEREQGgtr6Rz7dl8tm2DOrqm+ju78GciTH0DrfYOpqI\ntKNr3pxEpDUZnRyYMjyM0QOC+PjbE6xPyeHVFQeICvFk9oQYokO9bB1RRETEJpqsVrYePMNHG49T\nXF6Lp8mZeyf1YHT/IBwc9OGmSFej4iYdgoebkXsn9WTi4FCWbUxn15F8Fr27i7hefswaH4W/RdNA\nRESk6ziWXcL7a49x4nQ5To4OJI4MZ9qIcNxc9NZNpKvSX790KAHe7syb0Z9j2SUsXZfGziP57D5W\noA1MRESkS8gvqebDDensTM0DYFhvf2aNj8LXq3V3YBYR+6PiJh1STGg3fv/AEHak5rFsw9kNTDbt\nO82toyKI1wYmIiLSyVTXNvDplpN8tSObhsYmegR7MmdiDNEhWjIgImepuEmHZTAYGNY7gEExfqxL\nyeaTb0+ydF0aa3dlM2t8FENj/bWBiYiI2LWmJitf7zvFiq+PU1ZVj7enC7PGRzG8d4Be40TkAipu\n0uEZnRyYPCyMUf2D+HTzSdbuyuZvKw/y5Y4sZsdHExPazdYRRURErtnBk0UsXXuM7PxKXIyOzBjb\ng8lDu+NsdLR1NBHpgFTcxG54uBmZMzGG+MEhLNt4nJ2peTz7bgpDzm1gEqANTERExA6cLqzkg3Vp\n7E0vxACMHhDEzLE96ObhYutoItKBqbiJ3fG3uPPLO/qRll3K0vXH2HUknz3HCpgwOITbRkXiZ+uA\nIiIil1BRXc/Hm06wfncOjU1WYsO6MTs+hvBAs62jiYgdUHETuxUd6kXS/UPYdSSfDzeksWZnNt/u\nP8OchF6MiPXF6KSpJiIiYnsNjU2sT8nh429PUFnTgL/FjbsnRDMoxlfr2ESkxVTcxK4ZDAbiYv0Z\nGO3L+t05fPLtCf756UE++caVO8dFMay3NjARERHbsFqt7E0rZOn6NHKLqnBzcWJ2fDQTh4Ti5Kjd\nkUXk2qi4SadgdHLglqHdGdU/kLW7T/HppuO89vH3G5j07K4NTEREpP1k5VXw/tpjHM4oxsFgIH5w\nCLePjsTs7mzraCJip1TcpFMxuRp56LZ+jOjtT/LGdLYfzuO5f6UwuOfZDUwCvbWBiYiItJ3SyjqW\nf32cb/adwmqF/j18uDs+mhBfk62jiYidU3GTTsm/mxsP396PhLhSlq5PI+VoPnvTChg/KITbRkXo\nE08REWlV9Q2NfLkji1VbMqipayTY18Sc+Gj69fCxdTQR6SRU3KRTiwrx4vH7BpNyNJ8PN6Szdlc2\nmw+cZvrICCbFhWoDExERuSFWq5Xth3P5cH06hWU1eLgZeeCWKMbeFIyjg9axiUjrUXGTTs9gMDCk\n1/cbmHy86QQfbkhnXUr22Q1M+gTgoA1MRETkGmXnVfDf7+/h8MkiHB0MTBkWxvSbw3F3Ndo6moh0\nQipu0mU4OTqQENedUf0C+XRLBmt2ZvH6J4eaNzDpFWaxdUQREbEThaU1/Nd7u6mormdITz/umhCF\nv0XrqEWk7ai4SZfj7mrk7gnRTBgUQvLXx9l2KJf//L/dDIrxZdb4KIJ8tIBcREQur66+kcXJ+6mo\nrufhmQMY1tPX1pFEpAtQcZMuy6+bG7+4rS8Jcd1Zuu4Yu48VsDetkPGDgrltdCSe2sBERG5QZWUl\nCxYsoLS0lPr6eubNm8frr79OVVUV7u5nR2cWLFhAv379bJxUWspqtfLW50fIyC1n7MAgpt0cQUFB\nha1jiUgXoOImXV6PYE/+477BpBwt4MMNaaxLyWHzgTMkjgwnIa47zkZtYCIi12f58uVERkby6KOP\nkpuby9y5c/Hz8+PZZ5+lZ8+eto4n12HNzmy2HDxDVLAn9yX0wqA10iLSTrTdkQjfbWDix59+Opx7\nJ8Xg5OjARxuPk/T3rWw5cIYmq9XWEUXEDlksFkpKSgAoKyvDYtFaWnt2OKOYpevS8DI588sZ/TE6\n6W2UiLQf/R9H5DxOjg5MiuvOc78YwdThYZRV1vP3Tw/xxyU7Sc0otnU8EbEziYmJnDp1ioSEBO6/\n/34WLFgAwEsvvcR9993HU089RU1NjY1TSksUlFbz6ooDGAwwb0Z/LGYXW0cSkS5GUyVFLsHd1chd\n521gsvVQLv/13m5uij67gUmwrzYwEZGrW7lyJcHBwbzxxhukpqaSlJTEv/3bv9GrVy/CwsJYuHAh\n//rXv3jooYcuex8WiztOrXTOST8/c6vcT3voSFlr6xv587u7qKiu55ezBjJyUOgF3+9IWa/GnrKC\nfeVV1rZhT1mhbfOquIlcgW83N35+W18ShnZn6bo09qQVsC+9kHE3BXP76Eg8TdrAREQuLyUlhdGj\nRwMQGxtLXl4e8fHxODqeLWLx8fGsXr36ivdRXFzVKln8/Mzk55e3yn21tY6U1Wq18o9PD5GeXcrY\ngcHERftckK0jZb0ae8oK9pVXWduGPWWF1sl7peKnqZIiLRAZ5MmCewfxq5n98bO4sX53Dv/x2hY+\n3XyS2vpGW8cTkQ4qPDycvXv3ApCTk4O7uzsPPfQQZWVlAP+/vTsPa+rM+wb+PUmABBIgQAIiCIgL\nglu1rrhvY106rTNVO+PYVacVp3362L5aZvrazkyd6TKdOvbpY+0ytc/T661Lqdja1m7auqDihoq4\nQBHZSdgDYQmc949AJI0eZbQAACAASURBVLKqhJzA93NdvSA5h/DjrnrzzX2f38Hx48cxePBgZ5ZI\nnfj2ZA6SUgsR2d8bv53LhjJE5DxccSPqIkEQcNcQHUZE+uPHs3lIPJyJhJ9+xoEzuVgybSAmDQ+C\njN3FiKiFZcuWIT4+HitWrIDFYsFLL72E0tJSPPzww1CpVAgMDMQf/vAHZ5dJ7UjLKsXO5mYk97EZ\nCRE5F4Mb0S1SyGWYPTYEk2KC8OWxLHyTnI3396Xh25PZWDZzEIaF+zm7RCKSCC8vL2zevLnV8wsW\nLHBCNXQr2IyEiKSGbx0R3SZPpQK/nhGJTasnYFJMIK4XmvDaJ2fx5q4U5BqrnF0eERHdptr6BryV\ncB4mcz1+O3cIBoX4OLskIiKuuBHdqQAfFVYtbmpg8n06zmUU4/zPxZg+Khi/nDoQPmxgQkTkMkRR\nxPavL+F6oQnTRgVjxl39nV0SEREABjeibhMe5I3/85u7kJJejJ0H0nHwbB6SLhZiwYQBmDd+ADzc\nuqedNxEROc63ydk4xmYkRCRBDG5E3UgQBIweHIDhA/1wKCUPew5n4rNDmTh4Ng/3Tx2IycODIJOx\ngQkRkRSlXSvBzgMZbEZCRJLEf5GIHEAhl2HmmBD8/feTsHBSGEzmenzwZRpe+jAZqddKnF0eERHd\nxFhmxn8nprIZCRFJFoMbkQOpPBT41fRI/G31REweHoTsIhP+8clZ/HNnCnIMJmeXR0REuKkZyTw2\nIyEiaeJWSaIe4OetxOOLojH37lDs+OEqzv9cjAuZxZgzbgDmjumPAF+Vs0skIuqTRFHE9q8u4XqR\nCdNHB2PGaDYjISJpYnAj6kFhQRo89+BdOJdhbWDy7Ynr+OFkNmJHBGHhpHDoGOCIiHrUN8nZOHbR\n2ozkN3PYjISIpIvBjaiHCYKAUYMCMGKgP9JyyvG/X1/CTyn5OHK+AJOHB2HRZAY4IqKecPFaCXYe\nSIeP2h1x97MZCRFJG4MbkZPIZAJmjA3FsBAfnEgrxN4j13DoXD6OXrAGuIWTw6FngCMicghjmRlb\nE1MhEwTE3T8Cvmo2IyEiaWNwI3IymUzAxJggjB8WaBfgjpwvwOQR1hU4Bjgiou7TshnJQ/OHYlB/\nNiMhIuljcCOSCLsAd6kQnx+5hsPn8nHUtoUyDHqtp7PLJCJyaaIo4sOmZiQzRgdjOpuRSE59Qz0M\n5mK4a0SIovUSAyJicCOSHJlMwMToIIyPCkTypSLsPZKJw+dvbKFkgCMiun37T2Tj+MVCDOrvg9/M\nZTMSZzNbapBTmYdsU671Y2UuCqqL0Cg2AgCUciX0nv7QqQKg9wy48dEzAGo3LydXT9SzGNyIJEom\nEzAhOhDjovStAtyk4YFYNDkcgQxwRERdlnqtBLsOpsNX7Y419w+HQs5mJD2poq4S2ZW5yK7MQ05l\nLrJNeTCai+3OcZe5IUwTin5egaiT1SKnrAB5VYW4Xpnb6vU8FSroPAOgVwXYPjaHO083XmJAvQ+D\nG5HEtQxwJy8XYe+RazhyvgBJFwoZ4IiIushQZsbWPRcgl7EZiaOJoojimhK7gJZTmYvyukq787wU\nnhiqHYRQTX+EqoMRoukPvWcAZII1UOt0GhgMlWgUG1FaUw6D2YiiaqPdx5zKPGRVZLeqQe3mddMq\nnb8t3CkVyh4ZB6LuxuBG5CJkMgHjhwXi7ig9Tl66KcDFBGJRLAMcEVFbmpuRVNVY8PA9UYhkM5Ju\n09DYgMJqA7Irc5Fjymv6mA+zxWx3nq+HD0YERNsCWqgmGFoP3y5dvyYTZPBXaeGv0iLKb3Cr719a\nW4aiaiOKzEYYWnzMqsxGZkVWq9fTuKvtVulafvSQu9/ZgBA5EIMbkYuRCW0EuAsFOJpagEkxQVg8\nORyBfgxwRESAdfXn31+mIbvIhBl39ce0UcHOLsll1TXUI68q324lLc+Uj/pGi+0cAQL0ngGI9huC\nUE1/hGiCEaruD7W7Y65Hk8vkCFD5I0Dlj2gMtTvW0NiA4pqSptW5YrvVup/Ls5BRfq3V6/m4e7e6\nlk6vCkCAyh/ucjeH/AxEXcXgRuSiWga4U5cN2Hs4E0cvFCCJAY6IyGb/iWycSCvCoBAf/GbO4M6/\ngAAA1fVm5JhuBLTsylwUVhtsTUMAQC7IEewViJAWAa2/uh+UCmlsQ5XL5NB76qD31LU6Vt9oQbG5\nxBbkbKt11Uakl2XiatnPducLEODr4dMU5PztrqnzV/n31I9EfRyDG5GLkwkCxkXpMXaoDqcvG5B4\n5EaAmxgdhMWx4QhigCOiPig180Yzkrj72IykPeW1FXZbHbMr81BcU2J3jrvcHeHeAxCqCUaI2rrV\nsZ9XIBQy1/xV0k2mQJCXHkFe+lbH6hrqYTQXt3FNXTGulKbjSmm63fkCBAR4+cHf3c9ulU7nGYAA\npR/kMnlP/VjUy7nm3zYiakUmCLg7So8xLQJcUmoBjl0swMRoaxOTfv5snUzU1zSKjfj++k9QGhRw\nsyjh4+ENXw8f+Lh7Q6VQ9tp7ZBnKzNia2NSMZMkI+LAZCURRhKG6GNmmXGtQa2rDX1lnsjtP7eaF\nKO1ga9MQjfWaNJ3K39Y0pLdzl7shWB2EYHVQq2O1DXUwNm+7bFqpK6o2ori2GJdKr+JS6VW782WC\nDH5KbZvX1PkpfRnq6JYwuBH1MjcHuL1HMpGUWohjFwsZ4Ij6oIq6SiRmfAURYqtj7jK3G0HOw9v6\nubs3fJoeWwOeBm4udm1PbV0DtnzaohlJcN9rRtLQ2ICC6iK7gJZblQ9zfY3deVoPX4wMiEGoJth6\nTZo6GL4ePr020N8pD7k7+qv7ob+6n93zOp0G2fmGVtfSNX+8WHIZKLls9zVyQQ5/VduhTqv07TNB\nmbqOwY2ol2od4K7ZAtyE6EAsZoAj6hN8PXzw0qT1MLtVIquoAOW1FSirrUB5bQXKa8tRVleB9LLM\nNoNdMy83T/i4W4Ocb1PA82n5ubsPNO5ekvhFUxRF/PurNOQYTJjZR5qR1DXUIdfU1DTEZN3qmFdV\nAMtNTUOCNYHo5xdkC2ghmmDexLobKRXKplXK/q2OVdebW19P1+K6Otjfzg4KmQIBSr8271Pn4+Et\nib9r1PMY3Ih6uZYB7swVAxIPX8Ox1EIcv1iICcMCsTiWAY6ot/NX+UGnC0OIIqzN4w2NDaioq0RZ\nbbkt1JXVlqO87kbIK6kpRV5VQbvfQybI4O2usa3e+Xp428Ke7bGHN5Ryx27P/PrEdVszkgd7YTOS\n6vpqZDetoFlX0vJQWFVkF7wVghzB6iDbtWghGmvTkJAgfxgMlR28OjmKp5sKYW6hCPMObXXMVF9l\nC3D2K3XFKKguanW+m8wNuhYNUnSe/rZw5+PuzdXSXqxLwW3Tpk1ISUmBIAiIj4/HyJEjbcd27tyJ\n3bt3QyaTISoqChs3bsSJEyfw9NNPY/Bg6z+YQ4YMwQsvvOCYn4CIukQmCBg7VI+7huhw5ooRiYcz\ncexiU4CLZoAj6svkMjm0Sl9olb4dnldjqbGu1LUIdGW15TdW8eqsTS6uVVxv9zXc5e5N2zG97bZp\n2lbz3L3h23B7N0hOzSzB7oMZvaIZiSiKtvFsDmjZlbkoqSm1O89D7o6BPuG2gBaqtjYN4bVTrkPt\n5gW1jxcifOzfWBFFEab6qrZX6czGNt9IcZe7Q6fyb7390jMAGjc1Q52L6zS4nThxAllZWdixYwcy\nMjIQHx+PHTt2AADMZjP27duHjz/+GG5ubli5ciXOnDkDABg/fjz+9a9/ObZ6Irpl1gCnw11DAnDm\nihF7j9wIcOObtlAGBzDAEXWHqqoqrF+/HuXl5aivr0dcXBymTp0KAPjkk0+wbds2/PDDD06usuuU\nCiWUCiUC2+jE16xRbERVfXVTsCtvc/WurLYcRWZjh99L7ebV4ro7+1W75rCndruxPdOVm5E0io0w\nmoubtjo2d3bMham+yu48tZsXhjXfH00djFBNMAL6UNOQvkYQBGjc1dC4qxHpG253TBRFVNRVtnk9\nXVG1Ebmm/Favp5QrofP0R7CPHh6iCt7uGnh7qK0fm/7TuKtdtlNoX9Dp/5mkpCTMmTMHABAZGYny\n8nKYTCao1WqoVCps374dgDXEmUwm6HQ65OXlObZqIrpjLQPc2atG7D2cieMXC3HiYiHGDdNjcWwE\n+jPAEd2Rzz77DBEREVi3bh0KCwvx0EMP4euvv0ZxcTG+/fZbZ5fnEDJBZvtlM1TT/vVllkYLymsr\nmwJdedM1d9ZwVy2aYDSVwmgubvMX0Jbfy8fdG97uGuQXNKJOL8fYgaEwCumoL7kR8FSK21vBc4SG\nxgbkVxXaVtByKnORa8pHTUOt3Xn+Si0ifSMQ2nQtWqimP7fBkY0gCLY3NgZrB9odaxQbUV5bcdM1\ndcUoMhutf/Yqczt8bU+F6kaY87CGuZbhzhrwNJK5rrUv6TS4GY1GxMTE2B77+fnBYDBArVbbntu2\nbRs++ugjrFy5EqGhocjLy0N6ejqeeOIJlJeXY+3atYiNjXXMT0BEd0QmCBgzRIe7BlsDXOLhTJxI\nK0JyWhEDHNEd0mq1uHzZ2kmuoqICWq0WAPDaa6/hqaeewjPPPOPM8pxKIVPAX6WFv0rb6phOp7Fd\ni1VjqWn3urvypmvysipyAC8RCi/gXPV1nEuzfz0PubvtFgh2TVVa3BrBx0PT7SsNtbamIbm2G1nn\nmwpgERts5wgQEOilvxHQ1NabWXu58f6bdHtkgsy29XmIdpDdsUaxER7eAjLz8lFRV4mKukpU1pls\nn1fUVqKi3oTK2so2r69rSYAAtbtXq1Dn7a6GpkXw83bXwFOh4psO3eCW/4USxdZdp1avXo2VK1di\n1apVGDt2LMLDw7F27Vrcc889yM7OxsqVK/HNN9/A3d293dfVaj2hUNz5fmydTnPHr9FTXKlWwLXq\nZa23Z57eG3MnR+B4agH+3zeXrQHuUhGmjOqPZXOHICzI29kl3hIpjW1nWGvvtHDhQiQkJGDu3Lmo\nqKjAO++8g+PHj8PDwwOjRo1ydnkuQalQIkihbPNGyQDw1fEs7EpOx8ABSqxYMAAmi8m+yUrdjdW8\nwmpDh99L7ebVbnOV5sDn5ebZ5iqDqb7Kei2a7UbWeSiqNtg3DZEpEKzu1+Im1v3RXx0Ed3n7vx8R\ndSeZIIOvUoMQTechqr7RAlPLUFdXiYpaEyrrmwJe03PF5pIOV8YB660P7FfurJ9rPDStnvOQezDk\ntaPT4KbX62E03tiHXlRUBJ1OBwAoKyvD1atXMW7cOCiVSkybNg2nT5/G2LFjsWDBAgDAgAEDEBAQ\ngMLCQoSGtu6k06y0tPpOfxa7d+ikzpVqBVyrXtZ65yID1fjjijE4m25dgTt0NheHU3IxLkqPxZPD\n0V+n7vxFnEyqY9uWvlZrXwp+iYmJCA4Oxvvvv49Lly7h+eefh6enJ95+++0uv0Z3vbEJuNbYd6XW\nM5eL8OnBDPj7qPDiI9Oh9e54S2R9Qz1KaypQai5DibkMpeZylJjLUGIutz1nMBcjx9T+JR/WRi4+\n8FP5QqvyQcPlRlwrzYaxusTuPJWbElG6QYjwDUGEdgAitKEI9g6CwslNQ1zpzwDgWvX2zlpbr4i3\npcZSi/KaCpTVVKC8phJlNeUoq6lAmbkCZbWVKDdbH+dXFeB6ZU6Hr+Uhd4ePUgNfpQ98ld7W/1RN\nK+Qqb9tzPkpvuEvwHpOO/HPQaXCLjY3Fli1bsHz5cqSmpkKv19u2SVosFmzYsAF79+6Fl5cXzp8/\nj3vvvRd79+6FwWDAY489BoPBgOLiYgQGBjrshyCi7icIAu4arMPoQQFISS/GvuNZti2Ud0fpsTg2\nHCEuEOCInOn06dOYMmUKACAqKgoZGRkICgrCqlWrAFjfDH3mmWfwz3/+s93X6I43NoHe9wZBUZkZ\nr3yUDJlMwBO/jIGlth4GQ32nry3AHX7Qw0+pB5Ro9XupKIqoaai1bcMsv2nlrvm59JJraBQbAQAa\ndzWi/YZam4Y0bXf0V2ntV+bqgdLi7vl/ebtc6c8A4Fr1slZABiX8oISfhx7wANDGfe+tf79q7Fbs\nKm5a1ausM6GithLp1Tf+jrVHpVDaNVa5udFK81ZNjZu6RzqtOvrNzU6D25gxYxATE4Ply5dDEARs\n3LgRCQkJ0Gg0mDt3LuLi4rBy5UooFAoMHToUs2fPRlVVFZ599ll8//33qK+vx4svvtjhNkkiki5B\nEDB6cADmTArHd0nXkHg4E8mXrFso747S414GOKJ2hYWFISUlBb/4xS+Qm5uLfv36Yf/+/bbjs2bN\n6jC0Udtq6xrw1qfnUFVjwSP3RCEyuI3fEG+TIAhQKZRQKZQI8mr/TedGsRGm+ir4+3mh3sQGDURd\nYf37pYJKoeqwOy1g/Tum8pYhM7/Afrtm85bNFo872wYtQICXm6ddc5WbO2o2h732tkNLQZeucXv2\n2WftHkdFRdk+X7JkCZYsWWJ3XK1WY+vWrd1QHhFJRXOAGzXIHykZxUg8nImTl4pwkgGOqF3Lli1D\nfHw8VqxYAYvFghdffNHZJbk8URTxwZdpyDFUYeaY/pg6qv3OlY5ku+G4SgODyTVWWohciUyQwVup\nQbAaCEZQh+c2NDagst5kt2LXVvOV0tqyNu9/d/P31bip2+mo2eKxhwZKubJHr8fjjRqI6JYIgoDR\ngwIwKtIf524OcEN1uDc2AiF6BjgiAPDy8sLmzZvbPe5K93CTiq+PX0fypSIMDvHBg7MHO7scIpIA\nuUwOXw8f+Hp0vvpe31Bvtz2z8qYtm5VN3TULqoo6vXWCQqawBTl/pRarJiwH4LgtmQxuRHRbBEHA\nqEEBGNkywF024ORlA8Y2BbhQBjgi6kYXMoux+8cMaDUeWHP/CCjk0tzORETS5SZ3a/dWJC2Jooja\nhtp2r8Fr+Ti7MhfXK3NQYJoFndDPYbUzuBHRHWkZ4M7/bA1wpy4bcIoBjoi6UVFpNd5JTIVcJkPc\n/SPg48Vr54nIcQRBgFKhhFKhhN4zoMNzRVGERWxAsF7r0CY1DG5E1C0EQcDIyACMGOiP8z+X2Ae4\nITosjg3HgEDXaZVMRNJRW9eAtxLOW5uRLIjCwGDXuqckEfVugiDATXB8rGJwI6JuZQ1w/hgx0O9G\ngLtiwKkrBowZosO9DHBEdAtaNiOZNaY/po50TjMSIiJnY3AjIodoGeAuZFoD3OkrBpxmgCOiW/BV\nUzOSISE+WM5mJETUhzG4EZFDCYKAEQP9MTzCD6mZJdjTIsDdNTgAv5wSwQBHRG268HMxPj1obUby\nJJuREFEfx+BGRD1CEAQMH+iPmKYAl3g4E2euGnHmqhF3DQ7AvbERCAtigCMiq6LSamxNTIVcLsPa\nJWxGQkTE4EZEPcouwF0rQeIhBjgismeutWBLwnlU11rw6IJhiOjHZiRERAxuROQUgiBgeIQ/YsKb\nAlyLFbjRg6xbKBngiPoeURSx+ZMzyDVUYfaYEEwZ6bh7IhERuRIGNyJyqpYB7uK1UiQezsTZdCPO\npjPAEfVFXx7LwpFzeRgS6otlswc5uxwiIslgcCMiSRAEATERfogO1+JiVikSD9kHuHunhCM8iNul\niHqz8z8XI+HHnxHgo8ST9w1nMxIiohYY3IhIUgRBQEy4H6LDmgJcixW4UZH+uHdKBK93IeqFCkur\n8U5TM5LnHx4PHxV/RSEiaon/KhKRJLUMcGlNAS4loxgpGcUMcES9TE2dBW99eqMZyZABWhgMlc4u\ni4hIUhjciEjSBEFAdLgfhoVpcSmrFHtaBLiRkf74JQMckUsTRREf7EtDrpHNSIiIOsLgRkQuQRAE\nDAv3Q1RTgEs8nIlzGcU41xTg7o2NwMBgBjgiV/PlsSycvGxgMxIiok4wuBGRS7ELcNfLGOCIXNi5\nDGszEq3GA2vYjISIqEMMbkTkkgRBwLAwrd0WyuYAN2KgPx775XB4e8idXSYRtaOwtBrb9lqbkaxd\nMgLeXu7OLomISNIY3IjI5UWFabGhxRbK8z8X4z/++SMmxgRiydSBCPBVObtEImrBXHujGcljC4fx\nOlUioi5gcCOiXiMqTIuoMC1SM0vw2aFMHEstxMlLRZg9NgQLJ4VDrXJzdolEfZ4oivjgy6ZmJGND\nEDuCzUiIiLqCwY2Iep2YCD9Mu3sAvvgpHQk//oz9J7JxKCUfCyeHYc7YELgpuIWSyFn2JWXh1GUD\nhob6YtksNiMhIuoqBjci6pVkMgGTYoJw91Advj+Vi31J17DrQAZ+OJWD+6YOxKSYIMhkgrPLJOpT\nzmUY8dlPP8PP2wNPshkJEdEt4b+YRNSruSnkmD9hAP7+xCTMnzAA5VX1eH9fGl76MBkXMoudXR5R\nn1FYUo139l6EXC5D3P1sRkJEdKsY3IioT/BSumHpzEH42+qJmDw8CDlFJryxIwWvf3IGWQWVzi6P\nqFcz11qwJeE8zLUWPDR/KJuREBHdBgY3IupT/H2UeHxRNDY+Mg7DI/xw8VopXvowGds+T4WxzOzs\n8oh6HVEU8cG+NOQZqzCHzUiIiG4br3Ejoj5pQKAG/7lsNFIzS7DrQDo7UBI5yL6kLJy6Ym1GspTN\nSIiIbhuDGxH1aTERfhgWPg7HLxayAyV1u6qqKqxfvx7l5eWor69HXFwc6urqsG3bNri5ucHPzw+v\nvfYaPDw8nF2qQ7AZCRFR92FwI6I+TyawAyU5xmeffYaIiAisW7cOhYWFeOihhxAYGIj33nsPGo0G\nzz//PL755hssXrzY2aV2u+ZmJAqFDGuXsBkJEdGd4ltfRERN2IGSuptWq0VZWRkAoKKiAlqtFtu3\nb4dGo4HFYoHBYEBgYKCTq+x+NzcjCQ9iMxIiojvF4EZEdBN2oKTusnDhQuTl5WHu3LlYsWIF1q9f\nDwBISEjAnDlzMGDAAIwfP97JVXYvu2Ykd4dg8nA2IyEi6g6CKIqis4sAAIPhzn8Z0uk03fI6PcGV\nagVcq17W6hiuVCvQvfVeL6zEroMZSM0sAQBMjAnEkqkDEeCr6pbXd6Wx7Y5adTpNN1UjfYmJiTh5\n8iT+8pe/4NKlS4iPj0dCQgIAwGKxYP369ZgxY0aHWyUtlgYoXOhayx3fXcb/fnUJIyID8OffT+J1\nbURE3YTXuBERdWJAoAbr2IGSbsPp06cxZcoUAEBUVBRycnJw8OBBzJgxAwqFArNnz8aJEyc6DG6l\npdXdUktPvEGQkm7Ex19dgr+3Bx5bGIXSkqrbep2+9mZGT3GlWgHXqpe1OoYr1Qo4/s1Nvg1GRNRF\nMRF++L+PjMOqxdHw8fLA/hPZ2LA1CV8dz0K9pcHZ5ZEEhYWFISUlBQCQm5sLjUaDjRs3orCwEABw\n7tw5REREOLPEblNYUo1tnzc3IxkJb082IyEi6k5ccSMiugXsQEm3YtmyZYiPj8eKFStgsVjw17/+\nFXV1dYiLi4O7uzsCAgLw9NNPO7vMO9ayGcmqRdEIC+o722GJiHoKgxsR0W1o7kA5dVQ/7EvKwncn\nc/D+vjR8k5yNB2ZGYniEv7NLJAnw8vLC5s2bWz0/ffp0J1TjGI2iiPebmpHMvTsUk4YHObskIqJe\niVsliYjuADtQUl+37+g1nL5iQNQAXzwwM9LZ5RAR9VoMbkRE3cDfR4nHF0Vj4yPjEBPhh4vXSvHS\nh8nY9nkqjGVmZ5dH5BAp6UbsOZQJf28PPHHfcHaQJCJyIG6VJCLqRuxASX1FQUk1tn2eymYkREQ9\nhMGNiMgBYiL8MCx8HI6nFiLhp5+x/0Q2DqXkY+HkMMwZGwI3F7ovF9HNzLUWbPn0HMy1DWxGQkTU\nQxjciIgcRCYImDQ8CHdHsQMl9R6Nooj3vriI/OJqNiMhIupB3IxORORgzR0o//7EJMyfMADlVfV4\nf18aXvowGRcyi51dHtEt+eLoNZy5akTUAF8sncVmJEREPYXBjYiohzR3oNy0egImxdzoQPnC1qPs\nQEku4Wy6EYktmpHIZfw1goiop3CrJBFRDwvwUWHV4mj8Ynwodh3MwNmrBpy9asCkmEDcP20gAnxU\nzi6RqJX84iq8y2YkREROw+BGROQkzR0oc0rMeG/PeSSlFiKZHShJgsy1FryVcN7ajGQxm5EQETkD\n9zgQETnZXUP1+L+PjMOqRdHw8XLH/hPZ2LA1CV8dz0K9pcHZ5VEf17IZybxxoZgUw2YkRNS3HDz4\nfZfOe/nll5GXl+uwOhjciIgkoLkD5abVE7F05iAIArDrQAbitx3D0Qv5aBRFZ5dIfdQXR6zNSIaF\nafHATDYjIaK+JT8/D999t79L5/7xj39EcHB/h9XCrZJERBLS3IFy6qh+2JeUhe9O5uC9L9Kw/0Q2\nHpgZieER/s4ukfqQs1eN2HM4E/7eSjzxyxg2IyGiPueNN15BWloqpk4dh3nz7kF+fh7efPNt/O1v\nf4bBUASz2YxHH12N2Nip+N3vfoe1a/8TBw58j6oqE65fz0Jubg6eemodJk2KveNaGNyIiCSouQPl\nrDH98dlPmTiWWoA3dqQgOlyLB2YM4jVG5HD5xVV494tUuCtkWLtkBDRsRkJETrTzh3QkXyrq1tcc\nF6XH0lmDOjznwQd/h4SEnYiIiMT169fw9tvvobS0BOPHT8Q99yxCbm4OXnhhA2Jjp9p9XVFRIV5/\n/V84duwoEhM/ZXAjIurtbu5AmZpZgpc+TGYHSnKols1IVrMZCRERAGDYsBgAgEbjjbS0VOzdmwBB\nkKGiorzVuSNHjgYA6PV6mEymbvn+XQpumzZtQkpKCgRBQHx8PEaOHGk7tnPnTuzevRsymQxRUVHY\nuHEjBEHo8GuIiOjWNHegTM0swa4D6exASQ7TKIp49/MbzUgmshkJEUnA0lmDOl0dczQ3N+tc++23\nX6OiogL/9V/voaKiAo8//rtW58rlctvnYjddp95pcDtx4gSysrKwY8cOZGRkID4+Hjt27AAAmM1m\n7Nu3Dx9//DHcyWzPNgAADdhJREFU3NywcuVKnDlzBhaLpd2vISKi2xcT4Ydh4eNwPLUQCT9lYP+J\nbBxKycfCyWGYMzYEbgp55y9C1IHPj1zD2XQ2IyEiAgCZTIaGBvsOz2VlZejXLxgymQw//vgD6uvr\ne6aWzk5ISkrCnDlzAACRkZEoLy+3LfepVCps374dbm5uMJvNMJlM0Ol0HX4NERHdGXagJEc5c9WA\nRDYjISKyCQuLwOXLl1BVdSPLzJgxC0ePHsLTTz8JlUoFvV6Pf//7XYfX0umKm9FoRExMjO2xn58f\nDAYD1Gq17blt27bho48+wsqVKxEaGtqlryEiojvDDpTUnfKLq/Du5xfhrpDhD79iMxIiIgDQarVI\nSNhn91y/fsHYvv0T2+N58+4BAOh0GhgMlRg48MaWzoEDB+Gtt7Z1Sy233JykrT2aq1evxsqVK7Fq\n1SqMHTu2S19zM63WE4pu2OKj07nOBdSuVCvgWvWyVsdwpVoB16r3TmrVAYgL9cMDc4bif79Ow8HT\nOXhjRwpGD9bh4UXRiAzx7b5C4VrjSl1TXWPBlk/Po6auAavvjcaAQP4/JiKSmk6Dm16vh9FotD0u\nKiqCTqcDYN3fefXqVYwbNw5KpRLTpk3D6dOnO/ya9pSWVt/uz2DTnHJdgSvVCrhWvazVMVypVsC1\n6u2uWgUAv5s7BNNH9sOugxk4e9WA//jnj93agbI7amXwk5ZGUcR7X1xEQUk1fjE+FBOj2YyEiEiK\nOt28Hhsbi/37rXcLT01NhV6vt215tFgs2LBhA6qqqgAA58+fR0RERIdfQ0REjtXcgXLdstEYoFcj\nKbUQ8duOYccPV2Ey98wF1OQ69h7OtDUj+fUMNiMhIpKqTlfcxowZg5iYGCxfvhyCIGDjxo1ISEiA\nRqPB3LlzERcXh5UrV0KhUGDo0KGYPXs2BEFo9TVERNSz2IGSOnPmigF7j1xDgI8ST943nM1IiIgk\nrEvXuD377LN2j6OiomyfL1myBEuWLOn0a4iIqOc1d6C8O0qH70/l4ouj17DrQAZ+OJWD+6cNxMSY\nIMgEwdllkhPkF1fh3S+szUjWLhnBewESEUkc31ojIuoDmjtQ/v2JSZg/fgDKq+rx3hdpeOnfybiQ\nWezs8qiHtWxG8vCCKDYjISJyAQxuRER9iFrlhqWzBmHT6gmYFBOEnCIT3tiRgtc/OYOsAtdo5kJ3\nhs1IiIi6169/vRjV1XfeaLEzt3w7ACIicn0BPiqsWhyNX4wPxa6DGUjNLMFLHyZ3awdKAqqqqrB+\n/XqUl5ejvr4ecXFx0Ol0+POf/wyZTAZvb2/84x//gErVc+PNZiRERK6JwY2IqA9r7kB5IbMYuw5k\nICm1EMmXijB7bAgWTgrndU936LPPPkNERATWrVuHwsJCPPTQQwgICMCGDRswcuRIvPLKK0hISMBv\nf/vbHqmHzUiIiLru0Ud/i02b/oGgoCAUFOTj+efXQafTw2w2o6amBs888xyio4f3WD0MbkREhOER\n/ogO92MHym6m1Wpx+fJlAEBFRQW0Wi22bt1qu0WOn58fysrKeqSW7MJKNiMhIpeVkP4FzhSd79bX\nvEs/AksGLWr3+LRpM3HkyE/41a+W4tChHzFt2kxERg7GtGkzcOpUMj7+eDtefvm1bq2pI3yrjYiI\nANzoQLlp9UQsnTkIALDrQAbitx3D0Qv5aBRFJ1foehYuXIi8vDzMnTsXK1aswPr1622hrbq6GomJ\niZg/f77D66iuseDlfx9HTV0DHlkwjM1IiIi6wBrcDgEADh/+EVOmTMePP36PJ598DP/931tQXl7e\no/VwxY2IiOw0d6CcMrIfvkzKwnensvHeF2nYfyIbTy2/C/6eXKnpqsTERAQHB+P999/HpUuXEB8f\nj4SEBFRXV+PJJ5/Eo48+isjIjq8z02o9obiDFU9RFPGXD44j11CF+2cMwqLpg277tXqSTuc64ZK1\nOo4r1ctaHaO51t/rHgTwYA9/79H461+LYbGYUFNTjdOnkzBgQAj+9a83cf78ebz66qvQ6TSQy2UI\nCFDb1esIDG5ERNSm5g6Us8b2x2c/ZeJYagE+2JuK55aPdnZpLuP06dOYMmUKAOs9UIuKilBXV4c1\na9Zg0aJFbd4H9WalpXfWqaykogbJFwsxerAOC8aHwGCQfvdQnU7jEnUCrNWRXKle1uoYUqh1/PjJ\n+NvfXsXEiVOQl1eIyMjBMBgqkZi4D9XVNTAYKtHQ0Aij0QQvL687rrej4MetkkRE1KHmDpQvr56I\n51bc7exyXEpYWBhSUlIAALm5ufDy8sL777+P8ePH44EHHuiRGvy8lfjTyrvxwmMT2IyEiOgWTZ8+\nE999tx8zZszG/PkLsWPHx3jmmTjExAxHcXEx9u3b22O1cMWNiIi6JMjPEzqtyunvfrqSZcuWIT4+\nHitWrIDFYsGLL76I5557DiEhIUhKSgIATJgwAWvXrnVoHQODveHuxgYzRES3atiwGPz443Hb448/\n3m37fMqU6QCAhQvv7ZFaGNyIiIgcxMvLC5s3b7Z77vDhw06qhoiIXBn3TBAREREREUkcgxsRERER\nEZHEMbgRERERERFJHIMbERERERGRxDG4ERERERERSRyDGxERERERkcQxuBEREREREUkcgxsRERER\nEZHEMbgRERERERFJnCCKoujsIoiIiIiIiKh9XHEjIiIiIiKSOAY3IiIiIiIiiWNwIyIiIiIikjgG\nNyIiIiIiIoljcCMiIiIiIpI4BjciIiIiIiKJUzi7gNuxadMmpKSkQBAExMfHY+TIkbZjR48exRtv\nvAG5XI5p06YhLi7OiZVadVTvrFmzEBQUBLlcDgB4/fXXERgY6KxSceXKFaxZswYPP/wwVqxYYXdM\nimPbUb1SG9tXX30Vp06dgsViwe9//3vMmzfPdkxqY9tRrVIaV7PZjA0bNqC4uBi1tbVYs2YNZs6c\naTsupXHtrFYpjWuzmpoaLFq0CGvWrMGSJUtsz0tpXKk1V5ojXWl+BFxrjuT86BicHx2Dc2QXiS7m\n+PHj4urVq0VRFMX09HRx6dKldsfvueceMS8vT2xoaBAffPBB8erVq84o06azemfOnCmaTCZnlNZK\nVVWVuGLFCvFPf/qT+D//8z+tjkttbDurV0pjm5SUJD7++OOiKIpiSUmJOH36dLvjUhrbzmqV0rju\n27dP3LZtmyiKopiTkyPOmzfP7riUxrWzWqU0rs3eeOMNccmSJeKnn35q97yUxpXsudIc6Urzoyi6\n1hzJ+dExOD86DufIrnG5rZJJSUmYM2cOACAyMhLl5eUwmUwAgOzsbPj4+KBfv36QyWSYPn06kpKS\nnFluh/VKjbu7O959913o9fpWx6Q4th3VKzXjxo3D5s2bAQDe3t4wm81oaGgAIL2x7ahWqVmwYAFW\nrVoFAMjPz7d7901q49pRrVKUkZGB9PR0zJgxw+55qY0r2XOlOdKV5kfAteZIzo89X6vUuNL8CHCO\n7CqX2yppNBoRExNje+zn5weDwQC1Wg2DwQA/Pz+7Y9nZ2c4o06ajeptt3LgRubm5GDt2LNatWwdB\nEJxRKhQKBRSKtv9ISHFsO6q3mVTGVi6Xw9PTEwCwe/duTJs2zbbcL7Wx7ajWZlIZ12bLly9HQUEB\ntm7dantOauParK1am0lpXF955RW88MIL2LNnj93zUh1XsnKlOdKV5kfAteZIzo+OwfnR8ThHdszl\ngtvNRFF0dgm35OZ6n3rqKUydOhU+Pj6Ii4vD/v37MX/+fCdV17tIcWy/++477N69Gx988IFT6+iK\n9mqV4rh+8sknSEtLw3PPPYe9e/c6faLsSHu1Smlc9+zZg9GjRyM0NNQp35+6jyvNkZwfe44Ux5bz\no2O40vwIcI7sjMttldTr9TAajbbHRUVF0Ol0bR4rLCx0+jaBjuoFgPvuuw/+/v5QKBSYNm0arly5\n4owyOyXFse2M1Mb20KFD2Lp1K959911oNBrb81Ic2/ZqBaQ1rhcuXEB+fj4AYNiwYWhoaEBJSQkA\n6Y1rR7UC0hrXgwcP4vvvv8fSpUuxa9cuvP322zh69CgA6Y0r2XOlObK3zI+A9Ma2M1IbW86P3c+V\n5keAc2RXuVxwi42Nxf79+wEAqamp0Ov1tm0VISEhMJlMyMnJgcViwYEDBxAbG+vMcjust7KyEo89\n9hjq6uoAAMnJyRg8eLDTau2IFMe2I1Ib28rKSrz66qt455134Ovra3dMamPbUa1SG9eTJ0/a3vE0\nGo2orq6GVqsFIL1x7ahWqY3rm2++iU8//RQ7d+7EAw88gDVr1mDy5MkApDeuZM+V5sjeMj8C0hvb\njkhtbDk/OoYrzY8A58iuEkRX2kfR5PXXX8fJkychCAI2btyIixcvQqPRYO7cuUhOTsbrr78OAJg3\nbx4ee+wxJ1fbcb3bt2/Hnj174OHhgejoaLzwwgtOW8a+cOECXnnlFeTm5kKhUCAwMBCzZs1CSEiI\nJMe2s3qlNLY7duzAli1bEBERYXtuwoQJGDp0qOTGtrNapTSuNTU1+OMf/4j8/HzU1NRg7dq1KCsr\nk+S/B53VKqVxbWnLli3o378/AEhyXKk1V5ojXWV+BFxrjuT86BicH51Xr5TGtqWeniNdMrgRERER\nERH1JS63VZKIiIiIiKivYXAjIiIiIiKSOAY3IiIiIiIiiWNwIyIiIiIikjgGNyIiIiIiIoljcCMi\nIiIiIpI4BjciIiIiIiKJY3AjIiIiIiKSuP8PUVNsIo0sMk4AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": { "tags": [] } } ] }, { "metadata": { "id": "wYGUHK0GsfhP", "colab_type": "code", "outputId": "56837ce9-5982-42a9-9c83-8b7cc84b71f6", "colab": { "base_uri": "https://localhost:8080/", "height": 50 } }, "cell_type": "code", "source": [ "# 测试性能\n", "trainer.run_test_loop()\n", "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n", "print(\"Test Accuracy: {0:.2f}%\".format(trainer.train_state['test_acc']))" ], "execution_count": 72, "outputs": [ { "output_type": "stream", "text": [ "Test loss: 0.45\n", "Test Accuracy: 84.32%\n" ], "name": "stdout" } ] }, { "metadata": { "id": "GO8v_uUisfd9", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 保存所有结果\n", "trainer.save_train_state()" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "VbQdvtK3twoW", "colab_type": "text" }, "cell_type": "markdown", "source": [ "## 冻结嵌入层" ] }, { "metadata": { "id": "XrMm4K61t21h", "colab_type": "text" }, "cell_type": "markdown", "source": [ "现在我们要冻结GloVe嵌入层并且在监督任务上进行训练。模型里唯一需要改变的是打开`freeze_embeddings`:\n", "\n", "```python\n", "if freeze_embeddings:\n", " self.embeddings.weight.requires_grad = False\n", "```" ] }, { "metadata": { "id": "ilV_QbtktzH3", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "args.freeze_embeddings = True" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "7NdD-iP6tzFQ", "colab_type": "code", "outputId": "b59e9dd4-8378-4506-c514-9056da832355", "colab": { "base_uri": "https://localhost:8080/", "height": 202 } }, "cell_type": "code", "source": [ "# 初始化模型\n", "model = NewsModel(embedding_dim=args.embedding_dim, \n", " num_embeddings=len(vectorizer.title_vocab), \n", " num_input_channels=args.embedding_dim, \n", " num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n", " num_classes=len(vectorizer.category_vocab), \n", " dropout_p=args.dropout_p, pretrained_embeddings=embeddings,\n", " freeze_embeddings=args.freeze_embeddings,\n", " padding_idx=vectorizer.title_vocab.mask_index)\n", "print (model.named_modules)" ], "execution_count": 66, "outputs": [ { "output_type": "stream", "text": [ "\n" ], "name": "stdout" } ] }, { "metadata": { "id": "6NJhm2EOtzCV", "colab_type": "code", "outputId": "a441d5d2-8d71-44de-e1f8-dbe6bc62b985", "colab": { "base_uri": "https://localhost:8080/", "height": 101 } }, "cell_type": "code", "source": [ "# 训练\n", "trainer = Trainer(dataset=dataset, model=model, \n", " model_state_file=args.model_state_file, \n", " save_dir=args.save_dir, device=args.device,\n", " shuffle=args.shuffle, num_epochs=args.num_epochs, \n", " batch_size=args.batch_size, learning_rate=args.learning_rate, \n", " early_stopping_criteria=args.early_stopping_criteria)\n", "trainer.run_train_loop()" ], "execution_count": 67, "outputs": [ { "output_type": "stream", "text": [ "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 0.53 | [TRAIN ACC]: 80.7% | [VAL LOSS]: 0.48 | [VAL ACC]: 82.6%\n", "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 0.45 | [TRAIN ACC]: 83.7% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.3%\n", "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 0.42 | [TRAIN ACC]: 85.0% | [VAL LOSS]: 0.45 | [VAL ACC]: 83.6%\n", "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 0.39 | [TRAIN ACC]: 85.8% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.7%\n", "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 0.36 | [TRAIN ACC]: 86.9% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.6%\n" ], "name": "stdout" } ] }, { "metadata": { "id": "oojDHLowty_9", "colab_type": "code", "outputId": "48034d83-b5b3-44c4-c05a-9a38991e5101", "colab": { "base_uri": "https://localhost:8080/", "height": 335 } }, "cell_type": "code", "source": [ "# 性能展示\n", "trainer.plot_performance()" ], "execution_count": 68, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3MAAAE+CAYAAAAnEEnhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlA1HX+x/HnHNyX3Id4cIqgmGeH\nQoqampbapXZZ+eu33b/tV627bJu2R3vV7i+37dyy7diyDI8OMzU1vG9QVAQ8QQQGOUSQc35/YBR5\nKzAMvB7/MN/vfOc7rxmRmff3cxmsVqsVERERERERsStGWwcQERERERGRS6diTkRERERExA6pmBMR\nEREREbFDKuZERERERETskIo5ERERERERO6RiTkRERERExA6pmBNpA7169eLYsWO2jiEiItJmpk6d\nys0332zrGCIdmoo5EREREWlR+/btw8PDg5CQELZv327rOCIdloo5ERuqrq7mueeeY8yYMYwbN44/\n/elP1NfXA/DBBx8wbtw4xo4dy2233UZWVtZ594uIiLQXCxYsYOzYsUyYMIGFCxc27V+4cCFjxoxh\nzJgxPPPMM9TU1Jxz/8aNGxk9enTTY3+8/Y9//INnn32W2267jXfffZeGhgaef/55xowZQ1JSEs88\n8wy1tbUAHD9+nIceeoiRI0dy0003sWbNGlatWsWECROaZb7llltYvnx5a781Ii3KbOsAIp3Zv//9\nb44dO8aXX35JXV0dd999N1988QUjR47k5ZdfZuXKlbi7u7NkyRJWrVpFcHDwWfdHRUXZ+qWIiIgA\nUF9fz7Jly3j00UcxmUy89NJL1NTUUFhYyJ///GcWLlxIQEAAjz/+OO+99x5jx4496/6+ffue93lW\nr17NokWL8PHxYenSpWzZsoUvvviChoYGJk+ezFdffcXEiRN56aWXiIiI4PXXX2f37t3cf//9pKam\nUlRUxN69e4mJieHo0aMcPnyYxMTENnqXRFqGijkRG1q1ahUPPPAAZrMZs9nMTTfdxNq1a7nxxhsx\nGAzMnz+fCRMmMG7cOABqa2vPul9ERKS9WLNmDX379sXd3R2AIUOGsHLlSkpLS+nfvz+BgYEAvPTS\nS5hMJj777LOz7t+6det5n6dfv374+PgAMGbMGEaMGIGDgwMAffv25ciRI0Bj0ffWW28BEBsby4oV\nK3B0dGTMmDF8+eWXxMTEsHz5ckaOHImjo2PLvyEirUjdLEVs6Pjx43h5eTVte3l5UVxcjIODA+++\n+y7btm1jzJgx3HnnnWRmZp5zv4iISHuRkpLCqlWrGDRoEIMGDeKbb75hwYIFlJSU4Onp2XSck5MT\nZrP5nPsv5Mefn8ePH2fmzJmMGTOGsWPHsmLFCqxWKwClpaV4eHg0Hft9kTl+/Hi+/PJLAJYvX86N\nN954ZS9cxAZUzInYkJ+fH6WlpU3bpaWl+Pn5AY1XD+fMmcP69esZNmwYs2bNOu9+ERERWysrK2PT\npk1s3LiRLVu2sGXLFjZv3szOnTsxGo2UlJQ0HVtRUYHFYsHb2/us+00mU9M4coDy8vJzPu/f//53\nzGYzn3/+OV9//TXXX399031dunRpdv7c3Fxqa2sZPHgwdXV1rFy5kqysLK677rqWehtE2oyKOREb\nGj58OPPnz6e+vp7KykoWLVrE9ddfT2ZmJk888QQ1NTU4OjrSp08fDAbDOfeLiIi0B19++SXXXHNN\ns+6KZrOZYcOGUVNTw7Zt28jNzcVqtTJr1izmz5/P9ddff9b9/v7+FBUVUVxcTH19PZ9//vk5n7e4\nuJjo6GgcHR3Zu3cv27dvp7KyEoCkpCQWLFgAQHZ2Nrfccgv19fUYjUZuvPFGfve735GUlNTURVPE\nnmjMnEgbueeeezCZTE3bv//977nnnns4cuQI48ePx2AwMHbs2KZxcKGhoUyYMAEHBwfc3Nx47rnn\niI6OPut+ERGR9mDhwoVMnz79jP2jR4/m1Vdf5be//S3Tp0/HZDLRt29f7r//fpycnM65/9Zbb2XS\npEmEhIQwceJE9uzZc9bnfeCBB5g5cyYpKSkMGjSImTNn8utf/5r4+HieeeYZZs6cSVJSEm5ubrz4\n4os4OzsDjV0t586dqy6WYrcM1u87FIuIiIiIdCIWi4XJkyezatWqZhdcReyFulmKiIiISKc0Z84c\npk2bpkJO7JaKORERERHpVCwWCyNHjsRisfDAAw/YOo7IZVM3SxERERERETukljkRERERERE7pGJO\nRERERETEDrXrpQmKik60yHm8vV0pKalskXO1NmVtHfaUFewrr7K2DnvKCi2T19/fo4XSdA4t8RnZ\nGX/P2oqytg5lbT32lLezZT3f52OnaJkzm+1nhiJlbR32lBXsK6+ytg57ygr2l1ca2du/mz3lVdbW\noaytx57yKusPOkUxJyIiIiIi0tGomBMREREREbFDKuZERERERETsULueAEVERKSjOXnyJDNnzqSs\nrIza2loeffRR3nzzzab7CwsLmTx5Mg899JANU4qIiD1QMSciItKGFixYQFhYGE899RQFBQVMnz6d\nr7/+uun+//qv/2LixIk2TCgiIvZC3SxFRETakLe3N6WlpQCUl5fj7e3ddN+6devo2bMnwcHBtoon\nIiJ2RC1zIiIibWj8+PGkpKQwevRoysvLeeONN5rue++990hOTr6o83h7u7bIlNf2tr6fPeVV1tah\nrK3HnvIqayMVcyIick6rVq1g+PCRFzzuD3/4AxMm3EpISNc2SGXfFi1aREhICG+//TZ79+4lOTmZ\nlJQUCgoKqKyspHv37hd1npZYMNff36NFFh9vK/aUV1lbh7K2HnvK29mydvpFw0VE5NLl5x9l+fKl\nF3Xsr3/9axVyF2nbtm0MGzYMgJiYGAoLC6mvr2f16tVcc801Nk4nIiL2pMO3zKWmH+Xq+K442jqI\niIid+dvf/syePRkkJAzmhhvGkZ9/lP/7v1f54x9/S1FRIVVVVTzwwH8zdGgC99xzD4899r+sXLmC\nkycrOHz4EHl5uTzxxFNce+1QW7+UdqVHjx6kpaUxZswY8vLycHNzw2QysXPnTkaMGGHreCIi0gKs\nViu7DhzH8Wg5vUI8W+15OnQxV11Tz7tL9vL5ukM8N30Q7i4Oto4kImI3pk27h5SUTwgLi+Dw4YO8\n+uq/KCk5zpAh1zBu3ATy8nL5zW9+ydChCc0eV1hYwIsvzmHDhnUsWvSZirmfmDJlCsnJydx9993U\n1dUxe/ZsAIqKivD19bVtOBERuSJWq5U9h0pY8N1+co6W4+XuyN8fG9Zqz9ehizknRxOThoWxIPUA\nb36ewc9v74fRYLB1LBGRS/bJt9ls3lvYouccHBPAHUmRF3Vs795xAHh4eLJnTwaLF6dgMBgpLy87\n49j4+KsACAgIoKKiouUCdxBubm68/PLLZ+x//fXXbZBGRERayr4jpSxM3c/ew40zFg+M9ueBiX1a\n9Tk7dDEHMP66nhwuOsnWvYV8ue4gNw0Ns3UkERG74+DQ2LNh2bKvKS8v55///Bfl5eX813/dc8ax\nJtMPMyxardY2yygiImILB/LLWfDdfnYdOA5AfIQvkxPC6RHk0eqTtXT4Ys5oMPC/dw7k8Re/ZWHq\nAcK7ehHX08fWsURELskdSZEX3YrWUoxGI/X19c32lZaWEhwcgtFoZPXqb6mtrW3TTCIiIu3FkcIK\nFqbuZ3uWBYDePbyZnBhOZFevNsvQKWaz9HRz5OFJfTAaDby5OIOSE9W2jiQi0u716BFGZuZeTp78\noavk8OFJrFuXyv/8z8O4uLgQEBDA3Llv2TCliIhI28ovPsnri3Yx651NbM+yEBnqxTPT+vPMtP5t\nWsjBRbbMvfDCC6SlpWEwGEhOTiY+Pr7pvqSkJIKCgpq61bz44osEBgbyl7/8ha1bt1JXV8fPfvYz\nbrjhBn75y1+SkZFBly5dAJgxYwbDhw9v+Vd1FhEhXkwdGcWHy/bx2qJd/GJaf8ymTlHLiohcFm9v\nb1JSvmy2Lzg4hH//++Om7RtuGAf8sI5OePgPrYfh4ZG88sqbbRNWRESklRWWVrF4zQHWZxzDaoUe\nQR7ckhhOnzAfDDaal+OCxdymTZs4dOgQ8+bNIycnh+TkZObNm9fsmLfeegs3N7em7Q0bNpCVlcW8\nefMoKSlh8uTJ3HDDDQD87//+r82mXk4a0JWs3FI27Snks9U5TEmKskkOERERERGxD8fLT/H5uoOs\nSc+nvsFKqL8bkxPCuSrKz2ZF3PcuWMytX7+eUaNGARAREUFZWRkVFRW4u7uf8zGDBw9uar3z9PSk\nqqrqjHEXtmAwGJg+NobDBRUs3XSEyK5eDOwVYOtYIiIiIiLSzpRVVPPl+kOs2pFHXb2VIB9XJiWE\nMSgmoN3MkH/BYs5isRAXF9e07ePjQ1FRUbNibtasWeTl5TFw4ECeeuopTCYTrq6uAMyfP5/ExMSm\nbpgffPABc+fOxdfXl9/85jf4+LTtZCQuTmYendyH3723hXe+2kNogDuB3q5tmkFERERERNqnE5U1\nLNl4mG+35lJT14CflzMTh4VxTVwgJmP7GqZ1ybNZ/nSa6SeeeIKEhAS8vLx49NFHWbp0KWPHjgVg\n+fLlzJ8/n3feeQeAiRMn0qVLF3r37s2bb77JK6+8wnPPPXfO5/L2dsVsNp3z/kvh7+/R7Pajt13F\n3z/axpuf7+avTyTi5NAyz9MSfpy1vVPW1mNPeZW1ddhTVrC/vCIiIj9WeaqWpZuO8M2WI1TX1OPt\n4cTUoT0Z1je43c61ccFiLiAgAIvF0rRdWFiIv79/0/akSZOabicmJrJv3z7Gjh1Lamoqr7/+Ov/6\n17/w8Gj8gL/22mubjk1KSmL27Nnnfe6SksqLfiHnc7b1Hfr26MLwq0JYteMoL/9nK/ff2LtFnutK\ntfZaFC1JWVuPPeVV1tZhT1mhZfKqGBQREVuoqq5j+dZclm48TGV1HZ5ujtySGM7wq0JwaKGGpdZy\nwRJz6NChLF26FICMjAwCAgKaulieOHGCGTNmUFNTA8DmzZuJiorixIkT/OUvf+GNN95omrkS4PHH\nH+fIkSMAbNy4kago205AMm1UFD0CPUhNzyc1/ahNs4iIiIiISNupqa3n642Hmfn6ehZ8tx+DAW4f\nHsGff3Ytowd1a/eFHFxEy9yAAQOIi4tj6tSpGAwGZs2aRUpKCh4eHowePZrExESmTJmCk5MTsbGx\njB07lk8++YSSkhJ+/vOfN53nz3/+M3fddRc///nPcXFxwdXVlT/+8Y+t+uIuxMFs4pHJfXh+7mY+\n+GYfPQI96B6oK8MiIpfitttu4quvvrzwgSIiIu1AbV0D36Ud5Yv1BymrqMHFycSkhDBGD+qGi9Ml\nj0KzqYtK+/TTTzfbjomJabo9ffp0pk+f3uz+KVOmMGXKlDPOExISwmeffXY5OVuNfxcXZkzozT8+\n28mrC3fx3PTBuDrb1z+iiIiIiIicX119A+t2HePztQcoLq/GycHE+Gt7MGZId9xdHGwd77KoagH6\nR/kz7pruLNlwmLlL9vDIpD42XzNCRMTWHnjgLl544SWCgoI4diyfX/3qKfz9A6iqquLUqVM8+eQz\nxMb2sXVMERGR82posLJxdwGL1hygsLQKB7ORGwZ348ZreuDp5mjreFdExdxptySGk5NXztbMIpZt\nyeWGwd1sHUlExKYSE0ewdu133HrrHaSmriYxcQQREVEkJg5n69bNfPjhv/nDH/5q65giIiJn1WC1\nsi2ziAWp+8kvrsRkNJA0oCvjr+2Jt4eTreO1CBVzp5mMRh6aGMfsuZv5dGU24cGeRIZ62TqWiAgA\nKdlfsL1wZ4ues39AX26JnHDO+xMTR/DKK//HrbfewZo1q3nssSf5+OP3+eij96mtrcXZ2blF84iI\niLQEq9VKWnYxC1L3c6SwAqPBQEJ8MDcN7Ymfl4ut47Wo9rlggo10cXfioZvjaLBaeW3RLsora2wd\nSUTEZsLDIyguLqKg4BgnTpwgNXUVfn4BvPba2zz99C9tHU9ERKQZq9VKxoHj/P69rcz5LJ3cwgqu\njQvkD/99Nfff2LvDFXKglrkzxPTw5pbEcD5bvZ+3Fmfw5B1XYTRq/JyI2NYtkRPO24rWWq69dhhv\nvvkqCQnXU1paQkRE45Iyq1evpK6urs3ziIiInE3m4RIWpB5g35FSAAb18mfisDC6+rvbOFnrUjF3\nFuOu6UF2bhlpOcUsXnuASQnhto4kImIT118/goceeoB33/2IU6eq+P3vZ7Fy5XJuvfUOli//hi+/\nXGzriCIi0ontP1rOgtT9ZBw4DkC/CF8mJYTTI6hzLDemYu4sjAYDMybE8tt3N/P52oNEdvWiT7iv\nrWOJiLS53r3jWL16Y9P2hx/Ob7o9bNj1AIwffzNubm5UVp5o83wiItI5HS44wcLUA+zItgAQ29Ob\nyQnhRHTtXHNeqJg7B3cXBx6e1Ic/frCVNz/fzez7B+PjqcH+IiIiIiK2ctRykreX7GVt2lEAokK9\nuCUxnF7dvW2czDZUzJ1HWLAn00ZF8/7STF5buIuZdw3AbNKcMSIiIiIibamgpJLFaw6wYXcBViuE\nBXswOTGcuJ4+nXp9aBVzFzD8qhCyckvZkFHAJyuzuXNUtK0jiYiIiIh0CsVlp/h83QHWpB+jwWql\nW4A708fHEhbg1qmLuO+pmLsAg8HAvWN6cbigguVbcokK7cLgmABbxxIRERER6bBKK6r5ct0hVqfl\nUVdvJdjXlUkJ4Qzs5U9ggCdFRRqnDSrmLoqzo5lHJvXhd//ewjtf7SHU341gXzdbxxIRERER6VDK\nK2tYsuEQ327Lo7auAf8uzkwcFsY1sUFaLuwsVMxdpBA/N+4bF8MbizN4deEunr13EE4OJlvHEhER\nERGxeydP1bJ002GWbc6lurYeH08nbrquJ0P7BmvOivNQMXcJro4NJCu3lG+35fH+0kxmjO+tvroi\nIiIiIpepqrqO5VuO8PWmI1RV1+Hl5shtwyNI7BeCg1lF3IWomLtEU5KiOJBfzrpdx4ju1oXEfiG2\njiQiIiIiYleqa+v5dlsuSzYcpqKqFncXB+4YEcmIAV3V++0SqJi7RA5mIw9P6sPzczfzwTf76BHo\n0WlWmBcRERERuRK1dQ2s3pHHF+sPUX6yBhcnM5MTwhg1qBsuTipNLpXescvg5+XCgzfF8n+fpvPq\nwp3Mum8wrs4Oto4lIiIiItIu1dU3sGZnPl+sO8jx8mqcHE1MuK4nY4Z0w03foy+birnLFB/hx4Tr\nevDFukO8/eUeHrulr8bPiYiIiIj8SEODlfUZx1i89gBFpadwMBsZO6Q7Y6/pjqero63j2T0Vc1dg\n0rBwcvLK2Z5lYemmI4y9urutI4mIiIiI2FyD1cqWvYUsWnOA/OJKzCYDIweGMv7aHnRxd7J1vA5D\nxdwVMBoN/PfNccyeu4n5q3IID/EkulsXW8cSEZF27OTJk8ycOZOysjJqa2t59NFHueqqq3jyyScp\nKysjMDCQv/3tbzg66oq1iNgfq9XKjmwLC747QG5RBUaDgcR+Idx0XU98vZxtHa/DUTF3hbzcHHl4\nYh/+8p/tvLZoF7PvH4KXmz6ARUTk7BYsWEBYWBhPPfUUBQUFTJ8+naSkJIYNG8Z9993HK6+8wt69\ne4mPj7d1VBGRi2a1Wsk4cJwFqfs5kH8CA3BtXBA3D+tJoLerreN1WCrmWkB0ty7cNjyCT1Zm88ai\nXTw9tb9WqBcRkbPy9vYmMzMTgPLycry9vVm5ciUffPABAI899pgt44mIXLLMwyWkfLefrNwyAAbF\nBDBxWBhd/dxsnKzjUzHXQsYM6UZWbinbsywsXLOfWxIjbB1JRETaofHjx5OSksLo0aMpLy/njTfe\n4MEHH+Sjjz5i3bp1REZG8uyzz6qbpYi0ezl5ZaR8t589h0oAuCrSj0kJYXQP1LJdbUXFXAsxGAzM\nGN+b59/dzBfrDhER4kW/SD9bxxIRkXZm0aJFhISE8Pbbb7N3716Sk5Oprq5m6NChPPbYYzz77LN8\n+umn3HXXXec9j7e3K2bzlS+s6+9vX1+67CmvsrYOZW09F5s3J7eUD77ey5Y9BQD0j/bn7nG9ie7u\n3ZrxmrGn97Y1s6qYa0Guzg48Mqkvf3h/K//6Yjez7h+Mn5eLrWOJiEg7sm3bNoYNGwZATEwMhYWF\nBAUF0b9/fwCGDh3Kxo0bL3iekpLKK87i7+9BUdGJKz5PW7GnvMraOpS19VxM3ryiChauOcDWzCKg\ncajRLYnhTRMAttXrtaf3tiWynq8YNF7RmeUMPYI8uPuGaE6equO1hbuorWuwdSQREWlHevToQVpa\nGgB5eXm4ublxzTXXsGHDBgAyMjIICwuzZUQRkWYKjlfy5ucZPPf2JrZmFhEe4slTU69i5p39NZO7\njallrhUkxAeTdaSUtbuOMe/bLO6+oZetI4mISDsxZcoUkpOTufvuu6mrq2P27Nn06tWLp59+mjlz\n5uDn58cjjzxi65giIlhKq1i87iDrdh6jwWqle4A7kxLD6Rfhi8Ggyf7ag4sq5l544QXS0tIwGAwk\nJyc3my45KSmJoKAgTKbGfvsvvvgigYGBZ31Mfn4+v/jFL6ivr8ff35+//vWvHXKAt8Fg4O4xvThU\ncIJvt+URGerFNbFBto4lIiLtgJubGy+//PIZ+9955x0bpBEROVPJiWq+WH+Q73Ycpb7BSoifG5OG\nhTGglz9GFXHtygWLuU2bNnHo0CHmzZtHTk4OycnJzJs3r9kxb731Fm5ubhd8zJw5c7jzzjsZN24c\nf/vb35g/fz533nlny7+qdsDJwcQjk/vy23c38+8lmXQP8CBE07OKiIiISDtVfrKGrzYcYuX2PGrr\nGgjo4sLEhDCu7h2oZbfaqQuOmVu/fj2jRo0CICIigrKyMioqKi7rMRs3bmTkyJEAjBgxgvXr119p\n/nYtyMeV+2/sTXVtPf9csJNTNXW2jiQiIiIi0kxFVS3vfbWbma+v55vNR/B0deC+cTH8/sGruTYu\nSIVcO3bBljmLxUJcXFzTto+PD0VFRbi7uzftmzVrFnl5eQwcOJCnnnrqnI+pqqpq6lbp6+tLUVFR\nS76WdmlwTABZg0JZviWX95Zm8uCEWPUxFhERERGbslqt7M8vJzUtn417CqiuqcfL3ZHbhkeQ2C8E\nB7PmSbQHlzwBitVqbbb9xBNPkJCQgJeXF48++ihLly694GPOte+nWmoNHbDtWhSP3N6fI0Un2ZBR\nwICYQMZdd/5ZyrRuRuuwp6xgX3mVtXXYU1awv7wiIp1ReWUNG3YdIzU9nzzLSQB8PJ24a0wMQ6L9\ncHRome/e0jYuWMwFBARgsViatgsLC/H392/anjRpUtPtxMRE9u3bd87HuLq6curUKZydnSkoKCAg\nIOC8z90Sa+hA+1iL4sHxvZk9dzNvLtyJr7sjYcGeZz2uPWS9WMraeuwpr7K2DnvKCq2/jo6IiFy+\nhgYruw4cJzX9KDuyLNQ3WDGbDAyOCSAhPpjYnj4EBnra1eeONLpg++nQoUObWtsyMjIICAho6mJ5\n4sQJZsyYQU1NDQCbN28mKirqnI+57rrrmvZ/8803JCQktMqLao98PJ3575tiqa+38uqCXVRU1do6\nkoiIiIh0YIWlVaR8t59nXlvH/32axtbMIoJ9XZk2MoqXHh3Kw5P60CfcV2Pi7NgFW+YGDBhAXFwc\nU6dOxWAwMGvWLFJSUvDw8GD06NEkJiYyZcoUnJyciI2NZezYsRgMhjMeA/D4448zc+ZM5s2bR0hI\nSLNWvc6gT7gvNw3tyeK1B3n7i908flu8pncVERERkRZTU1vP1n1FrEnPZ8+hEgBcnEwMvyqEhH4h\n9Azy0PwNHchFjZl7+umnm23HxMQ03Z4+fTrTp0+/4GOgscvm3LlzLzVjh3Lz0DBy8spIyylmyYZD\njL+2p60jiYiIiIidO3TsBN+lH2VjRgGV1Y0zqPfq1oWEfsEM7BWAk8bCdUiXPAGKXBmj0cCDN8fx\n/NzNpHy3n4gQL2J6eNs6loiIiIjYmZOnatmQUUBq2lEOFzYuHebl7sj4AT0YFh9MoLerjRNKa1Mx\nZwOero48PLEPf/7PNl5fnMHs+wfTxd3J1rFEREREpJ1rsFrZc6iE1LSjbNtnoa6+AZPRQP8oPxL6\nhdA33AeTUcsKdBYq5mwkMtSL20dE8vGKLF5flMEz067SfzwREREROavj5adYk57Pmp35WMpOARDk\n40pCv2Cu6xOMl5ujjROKLaiYs6HRg0LJyi1la2YRKd/t5/bhkbaOJCIiIiLtRG1dAzuyLaSmHSXj\nwHGsgJODiWHxwSTGhxDR1VOTmXRyKuZsyGAw8MCNvcktrGDJhsNEdvXiBq2zJCIiItKp5RZWkJqe\nz/qMY03LWUV09SQhPoTBMQG4OOkrvDTSb4KNuTiZeWRyX37/3hbe/mIP8b0C0VxDIiIiIp1L5ak6\nNu0pIDX9KAfyGxfv9nR1YOyQ7gyLDybEz83GCaU9UjHXDnQLcOeeG3rxzld7+NN7m/nF1KtwMKuk\nExEREenIrFYr+46Ukpqez5a9hdTUNWAwQHyELwnxIfSL9MVs0pwKcm4q5tqJYfHBZOU2/mf+aHkW\n946NufCDRERERMTulJyoZt2ufFLT8yksqQIgoIsLw+KDGdo3GG8PzXIuF0fFXDty1+hoci0nWbXj\nKJGhXlzXJ9jWkURERESkBdTVN5CeU0xq2lF27j9Og9WKo9nItXFBJMQHE929C0ZNZiKXSMVcO+Lo\nYOKX0wfz87+t4r2lmXQP9CDU393WsURERETkMuUXnyQ1PZ91u45RfrIGgJ5BHiT0C+Hq3oG4Ouvr\nuFw+/fa0MyF+7jxwYyz/XLCTVxfs4jfTB2nGIhERERE7cqqmjmUbD/HV2gNk55UB4OZsZtTAUBL6\nhdAtQBfrpWWoSmiHBvbyZ8yQbizddIR3l+zloYlxWkNEREREpB2zWq3kHC0nNe0om/YWUl1TjwGI\nC/MhIT6Y/lH+OJg1mYm0LBVz7dSt10eQc7SczXsLie7WhZEDQ20dSURERER+ovxkDet2HSM1/Sj5\nxZUA+Ho6c+vwSK6K8MHPy8UUHJhTAAAgAElEQVTGCaUjUzHXTplNRh6e2IfZczfx8YosegZ7EBHi\nZetYIiIiIp1efUMDu/YfJzU9n7RsC/UNVswmA0N6B5AQH0Lvnt4EBnhSVHTC1lGlg1Mx1455ezjx\ns5vjeOnjHby2cBez7x+Cu4uDrWOJiIiIdEqFJZWkpuezdmc+pRWNk5mE+ruT0C+Ya+OC9D1N2pyK\nuXYutqcPkxLCWJB6gDc/z+Dnt/fTtLUiIiIibaS6tp5tmUWkph9l7+FSAFyczIzo35WEfsH0CPTQ\n3AZiMyrm7MD463qSnVfOzv3FfLnuIDcNDbN1JBEREZEOy2q1cvDYCVLT89m4u4Cq6joAYrp3ISE+\nhAG9/HFyMNk4pYiKObtgNBh48KZYZs/dxMLUA4R39SKup4+tY4mIiIh0KBVVtazPOEZqWj65RRUA\ndHF3JGlADxLigwnwdrVxQpHmVMzZCXcXBx6e1Ic/fbCNNxdnMPv+IXh7ONk6loiIiIhda7Ba2X3w\nOKlp+WzPKqKu3orJaGBgtD8J/YKJC/PBZNSSAtI+qZizIxEhXkwdGcWHy/bx2qJd/GJaf8wm/XER\nERERuVSWsirWnJ7MpLi8GoBgX1cS4kO4rk8Qnm6ONk4ocmEq5uxM0oCuZOWWsmlPIZ+tzmFKUpSt\nI4mIiIjYhdq6erZnWUhNO8rugyVYASdHEwnxwST0CyEixFOTmYhd6dDFXE19LS9ufQV3JxdiuvQi\n3i+WQNcAu/5PajAYmD42hsMFFSzddITIrl4M7BVg61giIiIi7dbhgsbJTDZkHOPkqcbJTCJDvUiI\nD2ZwTADOjh36K7F0YB36N9dkMOLj7M2u4j1kFu9nUc4SAlz86OsXS1+/WMK9emAy2t9MRC5OZh6d\n3IffvbeFd77aQ2iAO4EakCsiIiLSpPJULRt3F/Bdej6HjjUu3u3p6sDYq7uTEB9MsK+bjROKXLmO\nXcwZTTwUfx/OngZWZ24h3bKb3cczWXHkO1Yc+Q43B1f6+Pamr18svX2icDY72zryRevq7869Y3rx\nry/28OqCXfz6noE4aopcEZF27+TJk8ycOZOysjJqa2t59NFHefPNN6msrMTVtfHC3MyZM+nTp4+N\nk4rYnwarlczDpaxJP8qWzCJq6xowGKBfhC8J/UKIj/DVfAPSoXToYu57Hk7uXB08kKuDB1JbX8u+\n0v2kWzLYWbSbjce2svHYVswGE9E+kcSfbrXr4uRl69gXdF2fYLJzy1i14ygfLtvH/Tf2tnUkERG5\ngAULFhAWFsZTTz1FQUEB06dPx9/fnz/+8Y9ER0fbOp6IXSo5Uc2anfmsST9KUekpAAK8XUiID+a6\nPsGaAVw6rE5RzP2Yg8mBON9exPn2Ymr0ZI6cyCPdktHYalecye7iTD7OXEB3j9Cmwq6re3C7HWc3\nbVQUB/Ib+4E39v0OsXUkERE5D29vbzIzMwEoLy/H29vbxolE7FNdfQNp2RZS0/PZub8YqxUczUau\n6xNEQnww0d26tNvvbyItpdMVcz9mMBjo7hlKd89QJoSPobiqhJ2W3ey07GZfaQ6HT+TyxYFv8HH2\npq9fLPF+sUR2CcNsbD9vm4PZxCOT+/D83M188M0+egR60D3Qw9axRETkHMaPH09KSgqjR4+mvLyc\nN954g5deeok5c+ZQUlJCREQEycnJODvbT9d/kbaUZzlJatpR1mcc40RlLQBhwZ4kxAczpHcgrs7t\n53uaSGszWK1W64UOeuGFF0hLS8NgMJCcnEx8fPwZx7z00kvs2LGD999/n08//ZTFixc33bdr1y62\nb9/OPffcc0ljAoqKTlzOazqDv7/HJZ+rqq6K3cWZpFt2k1G8l6q6xiZ7F7MzsT6NM2PG+sbg6uDS\nIhmvJCvA9qwi/vHZTgK8XXhu+uA2+UN2uVltwZ6ygn3lVdbWYU9ZoWXy+vt3jgtRixYtYsuWLfzu\nd79j7969JCcn8/DDD9OrVy+6d+/OrFmz6N69OzNmzDjveerq6jGbNVZaOger1cqGXfl8tjKbzEMl\nAHi4OjJiUCijh/SgZ7CnjROK2MYFv/Fv2rSJQ4cOMW/ePHJyckhOTmbevHnNjsnOzmbz5s04ODgA\ncPvtt3P77bc3PX7JkiVNx9rLmAAXswsDA69iYOBV1DfUk116oHGcnWU3WwvT2FqYhtFgJKpLeFOr\nna+Lj83y9o/yZ9w13Vmy4TBzl+zhkUl91LVARKQd2rZtG8OGDQMgJiaGwsJCkpKSMJkaC7OkpCS+\n+uqrC56npKTyirN0xosGbUVZW05uUQUfLc9iz6ESDAboE+ZDQr8Qror0w8HcOJlJe8zf3t/Xn7Kn\nvJ0t6/kudl6wmFu/fj2jRo0CICIigrKyMioqKnB3d2865k9/+hNPPvkkr7zyyhmP/+c//8mLL754\nObnbDZPRRC+fSHr5RHJb1M0cPXmM9KLG7piZJdlklmQzP2sxIW5BxPvHEe8XSzePrhgNbTtb0i2J\n4eTklbM1s4hlW3K5YXC3Nn1+ERG5sB49epCWlsaYMWPIy8vD1dWVGTNmMGfOHDw9Pdm4cSNRUVG2\njilicxVVtSxKPcDK7Xk0WK3ER/jy8G39cNK1apEmFyzmLBYLcXFxTds+Pj4UFRU1FXMpKSkMGTKE\nrl27nvHY9PR0goOD8ff3b9pn72MCDAYDXd2D6eoezLiwkZRWl7HLsof004Xd1wdX8PXBFXg5etLX\nr3HZg17ekTiYHFo9m8lo5KGJccyeu5lPV2YTHuxJZGj7n5VTRKQzmTJlCsnJydx9993U1dXx/PPP\nU1JSwn333YeLiwuBgYE8/vjjto4pYjMNDVZW78hjQeoBKqpqCfR2YdqoKOIj/OyqRUakLVzywKof\nD7ErLS0lJSWFuXPnUlBQcMax8+fPZ/LkyU3b9957b7MxAR9++OF5xwR4e7u22HiA1hqL4Y8HUaGh\nTGY0p2pPkV6wl815aWw7upM1Rzey5uhGnMxO9AvqzaCQeAaE9MXTyf3857yCrP7+Hvzy3sE8+/pa\n3vg8g5f/dzhe7q03Ha89jXGxp6xgX3mVtXXYU1awv7y24ubmxssvv3zG/htvvNEGaUTal8zDJXy4\nLIvcogqcHU3cMSKSUYNCtTacyDlcsJgLCAjAYrE0bRcWFja1tG3YsIHjx49z1113UVNTw+HDh3nh\nhRdITk4GYOPGjTz77LNNjx09enTT7YsZE9AS4wGgbfvVhjlFEBYewW1hk9hfdqhpPbtNuTvYlLsD\nAwbCvXoQ7x9HX79YAl39mz2+JbIGeTkxOTGcz1bv549zN/LkHVdhNLZ8nwR7ujpmT1nBvvIqa+uw\np6ygCVBE5MpYyqr4ZGUOW/YWAjCsbzC3Xh/eqhekRTqCCxZzQ4cO5R//+AdTp04lIyODgICApi6W\nY8eOZezYsQDk5ubyq1/9qqmQKygowM3NDUdHR6CxRe/+++/vNGMCjAYjkV3CiOwSxi2REyg4WUi6\nZTfplt3sLztETtlBFmR/SaCrP/F+jYVdmFf3Fnv+cdf0IDu3jLScYhavPcCkhPAWO7eIiIhIS6iu\nrefrjYf5asMhausaiAjx5M7R0YRpdkqRi3LBYm7AgAHExcUxdepUDAYDs2bNIiUlBQ8Pj2YtbT9V\nVFSEj88PszsaDAbuuOOOTjsmINAtgNFuAYzuMZwTNRXssuxhp2U3e47vY9nhVSw7vAp3BzcGhcYT\n7R5FjE80TibHy34+o8HAjAmx/PbdzXy+9iCRXb3oE+7bgq9IRERE5PJYrVa2ZBbxybdZFJdX4+Xm\nyG1jIri2TxBGzcYtctEuap05W7HlOnNtpaa+lsySrNOLle+hvKYxp9loJsY7kni/OPr49cbL6fKu\nUB3IL+ePH2zF2dHM7PsH4+PZchPOtOf39afsKSvYV15lbR32lBXUzdIWWuL3ozP+nrUVZT23I4UV\n/GfZPjKPlGI2GRg9uBsTru2Ji9OFp3LQ+9p67ClvZ8t6RUsTSOtyNDnQ1y+Wvn6xNFgbOGEqYXXW\nZnZadrOreC+7ivdCJvTw7Ea8X+OyB8FugRe9hlxYsCfTRkXz/tJMXlu4i5l3DdAgYhEREWlzFVW1\nLPhuP6t25GG1wlWRfkwZGUmgt6uto4nYLRVz7YjRYCTStydeDb7cHDEWS1Ux6Zbd7CzaTXbZAQ6V\nH+Hz/V/j6+xDvF8s8f6xRHiFYTKef8bP4VeFkJVbyoaMAj5Zmc2do9r/ou0iIiLSMdQ3NLBq+1EW\npu7n5Kk6gn1dmTYySsM/RFqAirl2zM/Fl6RuCSR1S6CytpKM4kzSLRnsLs5kZe4aVuauwcXsQpxv\nL+L9Yon17YWL2eWM8xgMBu4d04vDBRUs35JLVGgXBscE2OAViYiISGey5+Bx/rMii7yik7g4mZia\nFEnSQC01INJSVMzZCVcHVwYH9WdwUH/qGurIKt1PetFudlp2s6VgB1sKdmAymIjqEk5f/1ji/WLx\ncfZueryzo5lHJvXhd//ewjtf7SHU341gXzcbviIRERHpqCylVcz7Nput+4owAIn9grklMQJPt8uf\n3E1EzqRizg6ZjWZ6+0TT2yeaO6InkluR37ienWU3e0uy2FuSxaf7FhHqHkJfv8bCrptHV0L83Lhv\nXAxvLM7g1YW7ePbeQTg5tMyi7CIiIiLVNfV8teEQSzYepq6+gciuXtw5OoqeQVpqQKQ1qJizcwaD\ngW4eIXTzCGF82GhKTpWy07KHdEsG+0pyyK04ypKDy+ni5NU40UpgLMMHBLFq2zHeX5rJjPG9L3oy\nFREREZGzsVqtbNpTyCcrsyk5UU0Xd0duHxHJNbEXP2mbiFw6FXMdjLdzFxJDryUx9Fqq6k6x5/g+\n0ot2k1G8h9S89aTmrcfJyZEufQLYmJ9Hjx3OjO6vBcVFRETk8hw6doKPlu9jX24ZZpOR8df2YPy1\nPXB21NdMkdam/2UdmIvZmQEB8QwIiKe+oZ6csoPstOwmvSgDi2sujhG5LDi+i60bujM4pC99/WIJ\ncPWzdWwRERGxA+WVNSz4bj/f7TiKFegf5ceUkVEEdDlzMjYRaR0q5joJk9FEtHcE0d4R3BI5gWOV\nhXyTuZkNuWkc4RBHsg+Rkv0FQW6BxJ9e966nZzeMBs02JSIiIj+oq29g5fY8FqUeoLK6jhA/N6aN\niiKup4+to4l0OirmOiGDwUCwWyDTB0zAo6I3X2zeR1h0JX7dy8ksyeKbQyv55tBKPBzc6evXm75+\nscT4ROFo0gxUIiIinVnGweN8tDyLo5aTuDqZmTYqihH9u2qpAREbUTHXyU0aFk5OXjl7MkoYHDCY\nGQl3sed4FjstjcserMvfzLr8zTgYHYjxiSLeL5Y+fr3xdPSwdXQRERFpI4WlVcxbkcX2LAsGYPhV\nIUxKDMfTVRd6RWxJxVwnZzQa+O+b45g9dxPzV+UQHuJJv25x9POPo8HawMHyI6QXZTQVdzstuzFg\noKdndwZ164MHXvi7+hHg4oez2dnWL0dERERa0KmaOr5cf4ilmw5TV28lKtSLO0dF0yNIF3VF2gMV\nc4KXmyMPT+zDX/6zndcW7WL2/UPwcnPEaDAS7tWDcK8eTIq8kcLKItJPF3Q5pQc5kHGo2Xk8HT3w\nd/Ej4HRx5+/aeNvfxVddNEVEROyI1Wplw+4CPl2ZTWlFDd4eTtwxIpIhvQO01IBIO6JiTgCI7taF\n24ZH8MnKbN5YtIunp/bHaGz+xzrA1Z9R3a9nVPfrqag5SYnBQvaxIxRWWiiqslBYaWF/2UFyyg6c\ncf4uTl7NCryA00Wfr4svDkb9GoqIiLQXB4+V859lWWTnleFgNnLTdT258ZoeODmabB1NRH5C36Kl\nyZgh3cjKLWV7loWFa/ZzS2LEOY91d3QjzD+Ibg49mu2vbaijuKqYwkoLhVUWiiotFFYVU1hZxL7S\nHPaV5jQ73oABH+cuBLj6N7Xq+bv4NhZ6zj6YjPrgEBERaQvlJ2v4bHUOa9LzsQIDe/kzZUQkflpq\nQKTdUjEnTQwGAzPG9+b5dzfzxbpDRIR40S/y0tadczCaCXILJMgt8Iz7auprKKoqPl3gNbbkfd+q\nt+f4Pvawr9nxRoMRP2efpjF5P27V83buomUTREREWkBdfQPfbDrMorUHqKqup6u/G3eOjKK3lhoQ\nafdUzEkzrs4OPDKpL394fyv/+mI3s+4fjJ9Xy1yRczQ50tU9mK7uwWfcd6ruFEXft+j9qNtmUZWF\njOK9ZPzkeLPBhJ+Lb1OhF9A0Ps8PLydPFXoiIiIXYdf+Yj55ZxO5hRW4OZu5a3Q0w/uHYDLqc1TE\nHqiYkzP0CPLg7huieXfJXl5buItf3jUQB3Pr/lF3NjvTzaMr3Ty6nnFfZW1lU0teUVP3zWIKq4o4\nVll4xvEORofTXTX9mwq8aLrhUO2Gp6O7Bm6L/ITVaqW2oY7q+mqq62vO/FlXTU1DTdPt5sfUYKWB\n6QNvxQVPW78UEblIBSWVzFuRzY5sC0YDjOjflcmJ4bi7ONg6mohcAhVzclYJ8cFkHSll7a5jzPs2\ni7tv6GWzLK4OrvR06E5Pz+7N9lutVipqT/7QildpoaDqh4Lv6MljPxy8t/GHs8mpebfN78fpufrh\n7uDWhq9K5PLUN9Sfp7D6ocA6+/5qqutqqDnLcVasl53JZDBRVHmc7g4q5kTau6rqOr5Yf5Blm49Q\nV28lpnsXHrn9Ktwd1BInYo9UzMlZGQwG7h7Ti0MFJ/h2Wx5RoV24OvbMcXC2ZDAY8HB0x8PRnXCv\nns3us1qtlNecOD0RSxEVnOBQ8VGKKi0cO1nAkRN5Z5zP1ezSvNvmj8bpuZg1+FsuzcW0dn2/XXNG\n8VWD1VTHiaqqZo+rqa+htqHuinIZMOBocsDJ5ISTyREPR3ecTI5N204mJ5zMP2w7mhzPvP8sP4MC\nu1BUdKKF3j0RaWkNVivrdx1j/uocyipq8PV0YkpSFAN7+RMQ4Kn/vyJ2SsWcnJOTg4lHJvflt+9u\n5t0le+kW4E6In320XhkMBrycPPFy8iTKOxx/f4+mD6oGawOl1WVnjM0rrLSQe+Ioh8qPnHE+dwe3\nxm6bzdbPa/zppDX07F59Qz3V9TWNLV7nbe062+2ftnb9uPvh5bd2QePYUCeTE44mRzwcPX5SRJ2t\nsDp3Ufb9Twejg7oai3QyB/LL+XDZPvYfLcfRbGTisDDGXt0dJwfNGC1i71TMyXkF+bhy/429eW3h\nLl5duIvf3DvI7teZMRqM+Dh74+PsTQxRze6rb6inpLq0+dIKp28fLD/M/rKDZ5zPy9GzWXH3/W1/\nF18cTBp7cKmsVit11nrqGmqpbaijtr6OuoZaahrqftj3/e36n2yfvl17+nazffW1GMwNZ7R2VdfX\nUNfirV0/LbwaCytH44+LrPO3dnUN8qWkuLKF3lUR6YzKKqqZvzqHtTsbhx0MjgngjhGR+Ho52ziZ\niLQUFXNyQYNjAsgaFMryLbn8e+leHpwQ22Gv7JuMjbNk+rn4EkvzcYJ1DXUUnyo5XeAVUfijZRay\nSw+QVbq/2fEGDI2Lpbv6nW7V+2H2TV8XH8zteLH0BmsDdQ0/Kqh+9LPudIH1/T6Xk2aOl5344bj6\nM4uumu8f99PCq/7Moquuoe6KW7TO56etXX5ntGxdoEvhT1q7HE2OOBjNLT6DqllrLIrIZaqrb2DZ\nliN8vvYgp2rq6Rbgzp2joujV3dvW0USkhbXfb5PSrtwxIpIDR8vZkFFAdGgXhvc/c9bJjs5sNBPo\n6k+gqz/Qu9l9tfW1jWvo/aTbZmGlhcySbDJLspsd/33r4I8nYvF39SPQ1Q9vpy5AY0H1Q4FU21To\nnFFYnS6K6hrqflQ0/fj474umHxdUZzlHU6FWS521vlXfS6PBiIPRjIPRAbPRjKPJETcHt6ZtB6MZ\nB5MZs9Gh6bgfH994f+O+H4754bgf7zMbHXA0NT4uJMCbkuNVrfraRERsKT3HwkfLsygoqcLdxYF7\nxkRyfb8QjMaOeRFWpLNTMScXxWwy8vCkPsyeu5n/LN9Hz2AP/P09bB2r3XAwORDiHkSIe9AZ91XX\n1/xoSQVLsy6cu49nwvHMZscbDUYMQL21oVUzmw2mpqLHbDTjYnY+XQz9sK9ZEWUyNy/ATv/08XLn\n1Mn600WZQ7PHmZsVWT8UYiYbtTqZTfqTJyId07HjlXy8Iov0nGKMBgMjB4YycViYlhoQ6eD0zUYu\nmo+nM/99Uyx//ySNVxfs4h8R/raOZBecTI6EeoQQ6hFyxn1VdVWn18w73XWzspjiU8WYzSaoN5wu\noBx+0ur0Q3HU2HrVvGD6aSF2riKrpboF/nhyGRERaVtV1XV8vvYgy7Ycob7BSu8e3kwbFUWov7ut\no4lIG7ioYu6FF14gLS0Ng8FAcnIy8fHxZxzz0ksvsWPHDt5//302btzI//zP/xAV1Ti5RHR0NL/5\nzW/Iz8/nF7/4BfX19fj7+/PXv/4VR0fNBGhP+oT7ctPQnixee5AXP9zKA+NicHHSNYHL5WJ2obtn\nKN09Q5vtV4EkIiLn02C1snZnPp+t3k/5yRr8vJyZkhTFgGi/DjuuXUTOdMFv4Zs2beLQoUPMmzeP\nnJwckpOTmTdvXrNjsrOz2bx5Mw4OPzTlDxkyhDlz5jQ7bs6cOdx5552MGzeOv/3tb8yfP58777yz\nhV6KtJWbh4aRc7ScrXsLycktZdqoaAb18teHh4jIRTh58iQzZ86krKyM2tpaHn30URISEgD4+OOP\nefPNN/n2229tnFLas5y8Mv6zfB8H8k/g6GBkckIYY4Z0x1FLDYh0OhfsZ7V+/XpGjRoFQEREBGVl\nZVRUVDQ75k9/+hNPPvnkBZ9s48aNjBw5EoARI0awfv36y8ksNmY0Gnji1r7ceUMvKqrqeG3hLv7+\nSRoFJZpGXUTkQhYsWEBYWBjvv/8+L7/8Mn/4wx8AKC4uZtmyZTZOJ+1ZyYlq3vp8N394fysH8k9w\ndWwgLzx4DTcNDVMhJ9JJXbCYs1gseHv/MJWtj48PRUVFTdspKSkMGTKErl2bz26YnZ3NQw89xLRp\n01i7di0AVVVVTd0qfX19m51H7IuD2cS0MTH87r+G0CfMh10HjvObf21iYep+autadyZEERF75u3t\nTWlpKQDl5eVNn7F//etfeeKJJ2wZTdqp2roGvlx/kOS3NrA+4xjdA9355V0D+NnNcfh4as04kc7s\nkgc7Wa0/rP9UWlpKSkoKc+fOpaCgoGl/z549eeyxxxg3bhxHjhzh3nvv5Ztvvjnnec7F29u1cSKI\nFmBPMy/aU9Y+0YG8EBXAuvR83lq0k8VrD7J5bxE/u6UvA2MCbR2vGXt6X8G+8ipr67CnrGB/eW1l\n/PjxpKSkMHr0aMrLy3njjTfYuHEjTk5O9OvXz9bxpB2xWq2kZRfz8YosCksblxqYOjaShHgtNSAi\njS5YzAUEBGCxWJq2CwsL8fdvnMVww4YNHD9+nLvuuouamhoOHz7MCy+8QHJyMjfeeCMA3bt3x8/P\nj4KCAlxdXTl16hTOzs4UFBQQEBBw3ucuaaFue/Y0mYS9Zo0O8eC3Dwxh0ZoDLN+Sy+y3NjColz9T\nR0a1i6uG9vS+gn3lVdbWYU9ZoWXydpZicNGiRYSEhPD222+zd+9efvWrX+Hq6sqrr756SedpqQue\n9va+21PeK8l6pOAE/1qUwbbMQoxGAzcnhjPthphWW2qgs7yvbc2esoJ95VXWRhcs5oYOHco//vEP\npk6dSkZGBgEBAbi7N053O3bsWMaOHQtAbm4uv/rVr0hOTmbx4sUUFRUxY8YMioqKKC4uJjAwkOuu\nu46lS5cyceJEvvnmm6YB39IxuDiZmToyiqF9g3l/aSZbMovYeeA4k4aFMXJgKGZTy0yFLyJiz7Zt\n28awYcMAiImJIScnh6CgIB588EGg8aLpk08+yd///vfznqclLnh2xosGbeVys1aeqmPx2gOs2JpL\nfYOVuJ7eTB0VTVc/N6oqTlFVcardZLUFZW099pS3s2U9XzF4wWJuwIABxMXFMXXqVAwGA7NmzSIl\nJQUPDw9Gjx591sckJSXx9NNPs2LFCmpra5k9ezaOjo48/vjjzJw5k3nz5hESEsKkSZMu/1VJu9Ut\nwJ1f3j2Aten5fLoqh3nfZrN2Zz73jOlFVGgXW8cTEbGpHj16kJaWxpgxY8jLyyM4OJilS5c23Z+U\nlHTBQk46noYGK2t25vPZ6hxOVNbi38WZqUlRXBWlpQZE5Nwuaszc008/3Ww7JibmjGNCQ0N5//33\nAXB3d+f1118/45iAgADmzp17OTnFzhgNBhL6hdA/2p/5q7L5Li2fP36wjWHxwdw+PAIPV60vKCKd\n05QpU0hOTubuu++mrq6O2bNn2zqS2FhWbin/WZbFoYITODmYuPX6cG4Y3A2HFpo3QEQ6Lq32LK3K\n3cWB+8b1Zlh8CO8vzWRNej7b9xVx+4hIhsUHY9TVRhHpZNzc3Hj55ZfPeb/WmOs8Sk5U8+mqbDZk\nNE4id21cILcNj8Tbw8nGyUTEXqiYkzYR2dWL5+4bxIqteSxI3c+7S/aSmn6Ue27oRfdA+xnAKiIi\ncqVq6+r5etMRvlx/kJraBnoEeXDX6Ggiu3rZOpqI2BkVc9JmTEYjNwzuxuCYAD5ekcXmvYU8/+5m\nRg3sxqSEMFyc9OsoIiIdl9VqZXuWhY9XZGEpO4WnqwN3jYpmqHqqiMhl0rdnaXPeHk48PKkPCQeK\n+eCbfSzbcoTNewuYOjKKwTEBGugtIiIdTp7lJB8t38fugyWYjAbGDOnGTdeF4eqsr2Iicvn0F0Rs\npk+YL7+bMYQlGw7zxfpDvL4og9T0fO4eHU2gj6ut44mIiFyxk6dqWZR6gG+35dFgtdIn3IdpI6P4\n//buO77K8v7/+Ous7ABZJ4MMQtiBsBP2DkNwV0VBtO6qndbRfH+WftuKdbTfOtqK1VoHbUGMQkEB\nQUSEEEaYYa8EQvYO2TT+D9oAACAASURBVMn5/RGIREZAk5xzkvfz8egDzn3f55z3fRXPfT7nuu7r\nCvbztHc0EWkHVMyJXVnMJm4YE0lcdCCL1hxm34kCnn07metGRHDdiAhcLJrJS0REnE9dvY0vd2aQ\n+NVxyipqsPq4M3tyTwZG+WkEioi0GBVz4hACfTz4+e0D2XEol3+vO8LyTSfZkprNnKm9GNDdz97x\nRERErtrhU0UseW8Hx88U4+pi4rYJUUwZFobFbLR3NGnHauprqaytpLK2isq6yoa/11Vd8LiKytpK\nKuqqqLpgW0VdJVW1VVgsZow2Ey5GCxajBYvJgsVoxsXogsVkObfdjIvJ5dx+MxajpfGxi9F87jmW\nc88xN76W2WjWjxitRMWcOAyDwcCwPlaiI31ZvukEn287zf8t2c3Q3gHcObknvp3c7B1RRETkig6m\nFfLSf3Zis8Ho/kHcOiGKLl5aakAurd5WT1Vd9QWFV9MC6/y2iguKsaq6KiouLNDOHVtrq/tOGSxG\nM64mV2y1Nqprq6mpr23hswQDBixNij3LBQXiBX+/1P5z2xoKx4Y/A+o6UV5ae8n9LkYLJmPHGdml\nYk4cjrurmTsm9WRU/2DeX32IHYdy2Xe8gBvHRDJlWChmk37ZFBERx1NaXs2b/03FgIHfPjyCrj7u\n9o4krcBms1FbX/utXq+GYqziXLFVWVuFMaue/JLSxn1V54q080VZZV0lVXXV3ymDAQNuZlfcTG50\ncvHGzd0fN7MbbibXhj/P7Wv407XJPnezG64m18Z9ZmNDORAQ4E1ubin1tnpq62uprq+hpq6m8c+a\n+ob/VV+0rZbquuqGffU11NTVUl1ffW5/beNzauqrqa6vpaauhsq6Kkqqy6ipr6HuOxahV2I0GBuK\nx0v0HJ7vNfymAHQ51wN5QTF5iQLywm3f3m402O+7qYo5cVhhVi+emTuETXsz+XD9MZasP8rmfZnc\nPa03PUO72DueiIhII5vNxjufHqSorJpbxnVnUC8rubml9o4lF2joBTtfgDXtBbuwh6tpwVV1yeGK\n37UAsRjNjUVWJ1fvcwXW+cLL7VuF1zd/dz/3p+u5Y11NLq02bNFoMOJicsHF5AKWVnmLJupt9eeK\nvZrGPy8sHL+9vbquBhd3I0UlZQ3F4wXFZM23isnqc8VkeU05RfUNvY71tvoWPweTwdS02Lug1zE6\nuAfTQuJb/D3PUzEnDs1oMDA2JoTBPQNY+uUxvtp9huc/SGHMgGBumxiFt4eLvSOKiIjwRUoGu47m\n0TfCh+tGRNg7Trtjs9korSmjrLCIzML8xl6wJveD1VVdNFzxm20t1wtmdQ9o2uN1mV6wYH9fKsvq\nLyjCvukFk28YDcaGtuPqhyOf70X8Lurq66j+duF4QW9j9be3fevvjc/7VqFZfUExWVpd2tgzWUs1\nU4OntFrxrX9R4hS83C3cO6MPY2Iahl5+vTeTnUdy+cGEKMYODNFiqyIiYjfp2aUs/uIoXu4WHpjV\nD6NR16TvoqqumvyKAvIq8smvLCSvIp+8igLyKwvIryigur7mml7vqnvBvlWEuV+wz9Xk+p17wQIC\nvMlFvbOOxmQ04W404W5u/bkYbDYbAQHe5OWVtdp7qJgTp9Kja2d+fe8wvtiRwccbj/PuqkN8vadh\n6GV4oLe944mISAdTVV3HwuWp1NbVc9/M/vh4a7KTy6mrr6Ooqpj8yoKGIq2igLxzhVpeRQGlNZf+\nwutudiPQIwA/d1+CuwRAjQk3syvuJrfGHq/z94JdWJR1pEkwxDEZDIZWn8VTxZw4HZPRSPzwMIb1\nsbL4iyNsPZDD//5zG5OHhnLz2O64u+qftYiItI1/rztCZn45U4aGMqiHv73j2JXNZuNsbXljcdak\nWKssoKCy8JL3KxkNRvzcfAj1DsHPzQd/dz/83H3xd/PF390XD4tH47HfZ3idSHukb73itHy8XXnk\nxv6MjSnggzWHWLv9NNsO5nDn5J4M72PVeiYiItKqth3M4avdZwi3enHbxB72jtMmaupqyK8sbOxd\nu3BIZH5FIZV1lZd8nreLFxHeYfi5NxRr/m6+DQWbuy9dXDvbdTZAEWemYk6cXnSkL7+9P5bPktNZ\nsTmNN5alsnH3GeZM7U2Qr0fzLyAiInKN8ooq+OdnB3GxGHn4xuh2syB4va2ekurSb3rWvnX/WnF1\nySWf52K0NOlRO1+o+Z37u6tJE5aJtAYVc9IuWMwmbhgdyYh+gSz6/Ah7j+fz67eTmREXwcyRmlVM\nRERaTl19PQv/m0pFVS0/nNGHYD9Pe0e6JhW1FeRVFJJfkU9eZQFn00s5XZhNfkUB+ZWF1F5i0WgD\nBnzcutCrS1RjofZN0eaHl8VTI2JE7EDFnLQrVh8PfnZbDCmHc/nX2iP8d/NJtuzP4tEfDCLCX710\nIiLy/S37+gTHMkqI7WtlTEywveNcpK6+joLKIvIq8xvvXzt/71p+RQFna8sv+TxPswchnkGNPWr+\n5wo1PzdffN26aEIREQekYk7aHYPBwNDeVqIjfVn+9UnWbDvF/761haG9A7hzck98O7X+VLQiItI+\nHUgrZOXmNPw7uzFvWh+79EadX3OtcaKRb92/VlhZhA3bRc8zG834ufkQ0TmsSa9az+BQjJWuuJvd\n2/xcROT7UTEn7Zabi5nbJ/VgVP8g/rP+KDsO5bLveAE3jolkyrBQzKb2cX+DiIi0jdLyav7+31QM\nBgMP3xCNh1vrfY06v+Za02n8GyYZyavIv+yaa11cO9O9c8S5HrULZoZ096WTi/clJxoJ8NEMkSLO\nSsWctHuhVi+ef3QMy9YfYcn6oyxZf5RN+zK5e2pveoV1sXc8ERFxAjabjXc+PUhRWTW3ju9OVNfO\n3+v16m31FFZeuOZafpNp/EurL73mmpvJlQAP/4tmhPRz88XPzQeLyfK9comIc1ExJx2C0WhgTEww\ng3r689GGY2zYdYY/LEphzIBgfjAxik4emmVLREQub92O0+w6mkffCB9mjGh+Yi2bzcbZmvKLZoM8\nX6wVVhZRZ6u76HlGgxFfNx+6+gRfPDOkuy+eZg9NNCIijVTMSYfi5W7hnul9GDMgmPdXH+LrvZns\nPJLLrROiGDcwBKMukCIi8i3p2aUsWX8UL3cLD8zqd9lrRV5FPik5e9iTm0pWRQ4VNZdZc83iRZh3\n12/NCNnQu9bFtbMmGhGRq6ZiTjqkqK6defbeYXyRksHHXx3nvVWH+HpPw9DLiCBve8cTEREHUVVd\nx8LlqdTW2bh/Zl98vF2b7M8tz2dnzh5ScvdwqjQDaOhd6+odSOdOXRp71PzdGiYb8XXzwc3seqm3\nEhG5ZirmpMMyGY3EDwtjWG8ri784wtYDOfz23W1MHhrKzWO74+6q/zxERDq6f609TGZ+OfHDwhjY\nwx+AnPI8dubsYWfOHk6VnQEaCrh+vr0ZbI0hJqAfkSFBmlRERFqdvq1Kh+fj7cojN/ZnbEwBH6w5\nxNrtp9l2MIfZk3oS29eqexNERDqorQey2bgnk3CrF+PjOrPq5BfszNnD6QsLOL/eDAmIISYgGk+L\n1jMVkbalYk7knOhIX357fxyrktNYkZTGwuWpbNxzhrlTexPkqwu0iEhHkltUwbtfbMc1NBNj92Ke\n27YUAJPBRLRfHwZbYxjo3w8PFXAiYkdXVcwtWLCA3bt3YzAYSEhIICYm5qJj/vjHP7Jr1y7ef/99\nAF588UV27NhBbW0tDz/8MFOnTuWZZ54hNTWVLl0apoO///77mTBhQsudjcj3ZDEbuX50JHHRQSxa\nc5i9x/P59dvJTI+LYNbICFwsuildRKQ9yzqbw47s3aw5vBX6FmME8qpM9D9XwMWogBMRB9JsMbd1\n61bS0tJYvHgxx44dIyEhgcWLFzc55ujRo2zbtg2LpWFtky1btnDkyBEWL15MYWEhN998M1OnTgXg\nF7/4BRMnTmyFUxFpOdYu7vzsthhSDufyr7VHWLH5JFtSs5g7tRcxUf72jiciIi0o62w2KTl72Jmz\nlzNnswCwmQ10qg3lpgGjiPGPxsPibueUIiIXa7aYS0pKYsqUKQBERUVRXFxMWVkZXl5ejcf84Q9/\n4Oc//zmvv/46AMOHD2/svevUqRMVFRXU1V28loqIIzMYDAztbSU60pflm07y+bZT/PnDPQzpFcBd\nU3ri28nN3hFFROQ7ymws4PaQeTYbALPBRKRHDw7t9aBLfRi/vmc0Hm66I0VEHFezn1B5eXlER0c3\nPvb19SU3N7exmEtMTCQ2NpauXbs2HmMymfDwaBiCsHTpUsaNG4fJ1DA87YMPPuCdd97Bz8+PZ599\nFl9f3xY9IZGW5uZi5vaJPRjVP4j3Vx8i5XAuqScKuGFMN+KHhWE2Ge0dUURErsKZsqxzywjsJet8\nAWc0E+MfzWDrACI9erDgvT1QXsMjcwepkBMRh3fNn1I2m63x70VFRSQmJvLOO++QnZ190bFr165l\n6dKl/OMf/wDgxhtvpEuXLvTt25c333yT119/nV//+teXfS8fHw/M5pa5RykgwHnWDlPW1vF9swYE\neDOobxDrtp3inRWpfLj+GMkHcnj01oFEd/droZRN389ZKGvrcKas4Hx5pf2z2WxNeuCyynOAhgJu\noH80g60x9Pfvi7vZDZvNxqtL91BcVs0PJkQRFdLZzulFRJrXbDFntVrJy8trfJyTk0NAQADQcG9c\nQUEBc+bMobq6mvT0dBYsWEBCQgIbN27kjTfe4K233sLbu+ECP3LkyMbXmTRpEr/5zW+u+N6FheXf\n5ZwuEhDg7TRrvShr62jJrAMjffj9A3EkbjjGhl1neOYvXzO6fxC3TepBJw+XFnmPjtq2rU1ZW09L\n5O0oxeDZs2d5+umnKS4upqamhscee4zq6mrefPNNLBYLvr6+vPTSS7i6amHp78Jms3Hm7LkeuJy9\nZJ8r4CxGMwMD+jMkYAD9/fviZm46VH7tjtPsPpZPv24+TI8Lt0d0EZFr1mwxN3r0aF577TVmz55N\namoqVqu1cYjl9OnTmT59OgCnT5/mV7/6FQkJCZSWlvLiiy/yz3/+s3HmSoAf//jHPPXUU4SFhZGc\nnEzPnj1b6bREWpeXu4V50/swekAw768+xKZ9Wew6mset46MYNygEo9amE5HL+Pjjj4mMjOSJJ54g\nOzube+65h8DAwMYfP3/1q1+xZs0arr/+entHdRrnC7jzPXDZ5blAQwE3KKB/Qw+cX5+LCrjz0rNL\n+XD9Ubw9LDwwq58+w0XEaTRbzA0ZMoTo6Ghmz56NwWBg/vz5JCYm4u3tTXx8/CWf8+mnn1JYWMjP\nfvazxm0vvPACc+bM4Wc/+xnu7u54eHjw/PPPt9yZiNhBVNfOPHvvML5IyeDjr47z3upDbNyTybxp\nvYkI6hi9DCJybXx8fDh06BAAJSUl+Pj48O677wJQW1tLbm4ugYGB9ozoFGw2GxllmefugdtDTnnD\nKCKL0cKggAEMsQ4g2q8vbuYr93BWVdfxxrJUauts3D+zL1281CMqIs7jqu6Z++Uvf9nkcZ8+fS46\nJjQ0tHGNuTvuuIM77rjjomNCQkL46KOPvktOEYdlMhqJHxbG8D5W/rPuCFsP5PDbd7cxaUgoN4/t\nrhvoRaSJmTNnkpiYSHx8PCUlJSxcuBBomFDs1VdfZdKkScTGxto5pWOy2WycKs0gJWcPu3L2klPx\nTQE3OGBA4z1wrqarH/K+aO1hsgrKmTo8TEvPiIjTMdgunNHEwbTU/SLOdO+JsraOtsyaerKAD9Yc\nJrugnM6eLtwxuQdxfQMxXMOwHbVt61DW1qN75q7esmXL2L59O7/73e84ePAgCQkJJCYmAg09c08/\n/TQTJkxodphlbW1di00S5shsNhsnCk+x5XQKW06lkFXWMITS1eTCkJABjAgbzODg/s32wF3KVztP\n89IHO+jetTMv/2Qslg7QniLSvqjLQKSFRXfz5bf3xbIqOY0VSWm8uXw/G3dnMndqL4L9PO0dT0Ts\nLCUlhTFjxgANI11Onz7Nl19+yYQJEzCbzUyePJmtW7c2W8y1xCRhjvqjwYU9cDtz95JXkQ+Aq9mV\nodaBDLbGEO3XG5dzPXClhdWUUn1N75FbVMHrH+7C1WLigZl9KWqhSdfOc9S2vRRlbR3OlBWcK29H\ny3qlHztVzIm0AovZyPWjI4mLDuJfnx9mz7F85v9jK9PjIpg1MgIXi379FemoIiIi2L17N9OmTSMj\nIwNvb2/mz5/PkiVLCAwMZM+ePURGRto7Zpuz2Wykl55mZ85edubsIa+yAAAXkwtDrQMZYo1hXO9h\nlBRWfe/3qq2rZ+HyVCqq6rh/Zl+CfD2+92uKiNiDijmRVmTt4s5PfxBDyuE8/rX2MCs2n2RLahZz\n4nsxsIfuzRDpiO644w4SEhKYO3cutbW1/P73v6e6uprHHnsMFxcX/P39+elPf2rvmG3ifAHXMAvl\nXvLPFXCuJheGBQ5isDWGfr69cTFZGrabXYDvX8wt+/oEx8+UMKJfIKP6B33v1xMRsRcVcyKtzGAw\nMLR3ANGRPizfdJLPt53ilaV7GNIrgDsn98Sv86WnyhaR9snT05NXXnnlou3jx4+3Q5q2Z7PZSCs9\n1TiJSX5lIQBuJleGBQ5iiDWGvhcUcC1t/8kCPk1KI6CLG3dP631N9zOLiDgaFXMibcTNxcztE3sw\nqn8QH6w+RMrhXPadyOfGMZHEDwvDbDLaO6KISKuw2WycLDnFznP3wBVcUMANDxx8rgeuF5ZWKuDO\nKymv5u8r9mM0Gnj4hv64u+prkIg4N32KibSx0AAvnp4zhM37slj8xVE+XH+MzXuzmDu1F73Dfewd\nT0SkRTQUcOmNQygLq4oAcDO5MTxwCEOsA+jbBgXchXn+sfIAxWXV3DYhiu4hndrkfUVEWpOKORE7\nMBgMjB4QzMAe/iRuOMaGXWd44V87Gd0/iNsm9iAgwN4JRUSuXb2tvrGA25Wzr7GAcze7ERc0lMHW\nAfTx7YXF2PZfP9ZuP82eY/lEd/NhWlx4m7+/iEhrUDEnYkde7hbmTe/D6Jhg3l99iE37sth5JI87\n4nsxrKe/hgCJiMOrt9Vzoji9cQhlUVUx8E0BN8QaQ2/fnnYp4M5Lyyrlwy+P4u1h4YFZ/TDqPjkR\naSf0TVHEAUSFdObZe4axPiWDjzce550V+1niZmbKsDAmDw3Fy71thiGJiFyNels9x4vT2Jmzh125\n+y4o4NwZETTsXA9cT8x2LODOq6yu5Y3lqdTW2bh/Zj86e1374uIiIo7K/p+yIgKAyWhkyrAwRvYP\nYsvBXD758ijLvj7Bqq3pTBrclamx4XT2dLF3TBHpoM4XcOdnoSyuLgHAw+zOiOBhDT1wPj0cooC7\n0L8+P0J2QTlTh4cRE+Vn7zgiIi3KsT5xRQRPNwuz43szup+VL3eeYfXWdD5LTmftjtOMiwlhely4\nljMQkTZRb6vnWNFJduaeL+BKAfA0ezAyeDiDrTH09olyuALuvC37s/h6byYRQd78YEKUveOIiLQ4\nx/z0FRHcXMxMjwtn8tCubNyTyWdb0liXcpovd2Uwqn8Q142MINDHw94xRaSdaSjgTpCSs5dduXsp\nuaCAG9VYwPXAZDTZOemV5RRV8N6qQ7haTDxyQ7SWfxGRdknFnIiDs5hNTBoSyriBIWxJzWblljQ2\n7snk672ZxPYNZObICEIDvOwdU0ScWL2tnqNFJxonMSmtLgPA0+LB6JBYBgfE0MsnyuELuPNq6+pZ\nuCyVyuo67p/Zl0Bf/fAlIu2TijkRJ2E2GRkTE8yo/kFsP5TDis1pJO/PJnl/NoN7+jNrVDcig7Vu\nkohcHZvNxr7sQ6w/ksyuCwo4L4sno0PiGGKNoWeX7k5TwF3ok40nOJFZwsjoQEYPCLZ3HBGRVqNi\nTsTJGI0GYvsGMryPld3H8lmx+SQ7j+Sx80ge0ZG+zBoZocXHRaRZ609t5KOjK4CGAm5MSByDnbiA\nOy/1ZAGfbUnD2sWduVN72zuOiEirUjEn4qQMBgODevgzMMqPA2mFrNh8ktQTBaSeKKBnaGdmjepG\n/0hfDFpPSUQuoY9vL27qO41w1wh6dIl06gLuvJKz1bz13/0YjQYevjFaa3WKSLunTzkRJ2cwGOjX\nzZd+3Xw5mlHMis0n2XMsn/9bspuIIG9mjYxgcK8ALZIrIk2EeAUxMLInubml9o7SIuptNt5eeYDi\ns9XcNjFKw85FpENQMSfSjvTo2pmf3TaQ9OxSViSlseNgDn/5eB8h/p7MHBlBbF8rJqNmdBOR9mft\ntlPsPZ5PdKQv02LD7R1HRKRNqJgTaYfCA7159Kb+ZOaf5dOkNJJSs/n7f/fzycbjXDciglH9g7GY\nVdSJSPuQllXKh18eo5OHhQdm9tVIBBHpMPRtTqQdC/bz5P5Z/Xj+4RFMGNyVwtIq3l11iGcWJvH5\ntlNU1dTZO6KIyPdSWV3LG8v2UVdv44FZ/ejs5WrvSCIibUbFnEgHENDFnXnTevPCI6OYOjyMs5U1\n/HvdEZ7622ZWJp2koqrW3hFFRL6TRZ8fJruwgumx4fTv7mfvOCIibUrFnEgH4uPtyuzJPXnpR6OY\nNaobtXU2PtpwnF/+dTOJXx2ntLza3hFFRK7altQsNu3NIiLIm1vGd7d3HBGRNqd75kQ6IG8PF24Z\n153pseGs33ma1VtPsWLzST7fdooJg0OYFhtOFw1VEhEHllNYznurD+HqYuKRG6Mxm/T7tIh0PCrm\nRDowDzczM0d2Y8rQMDbsPsOq5DRWbz3Fuh0ZjI0JZkZcOP5d3O0dU0Skidq6ehYuT6Wyuo4HZ/Uj\n0MfD3pFEROxCxZyI4OpiYurwMCYO7sqmfZl8tiWN9Tsz+Gr3GUb0C+S6kREE+3naO6aICAAff3Wc\nE5mljIwOYmT/IHvHERGxGxVzItLIYjYyYVBXxsYEs3V/DiuSTrJpXxab92UxtI+VWSMjCA/0tndM\nEenA9p3I57PkdKw+7syd2svecURE7OqqirkFCxawe/duDAYDCQkJxMTEXHTMH//4R3bt2sX7779/\n2edkZmby1FNPUVdXR0BAAC+99BIuLi4te0Yi8r2ZjEZG9g8iLjqQnYdzWbE5je0Hc9h+MIeBUX7M\nGtWNqK6d7R1TRDqYotIq3lpxAJPRwMM3ROPuqt+kRaRja/Zu4a1bt5KWlsbixYt57rnneO655y46\n5ujRo2zbtq3Z57z66qvcdddd/Otf/yIiIoKlS5e24KmISEszGgwM7W3l1/cO4+e3D6RnaGd2H8vn\nufd38NK/d3LgZAE2m83eMUWkA6i32fi//6RQcraaW8dHERncyd6RRETsrtliLikpiSlTpgAQFRVF\ncXExZWVlTY75wx/+wM9//vNmn5OcnMzkyZMBmDhxIklJSS12IiLSegwGAwO6+/GruUN5+q7BRHfz\n4UBaIS/9ZxdPvraRXUfzVNSJSKv6fNspUg7m0D/Sl6mxYfaOIyLiEJodn5CXl0d0dHTjY19fX3Jz\nc/Hy8gIgMTGR2NhYunbt2uxzKioqGodV+vn5kZub22InIiJto3e4D73DfTiRWcKKzSfZeSSPQ2mF\nhFm9mDkygmG9rRiNBnvHFJF25GRWCUu/PEYXb1fun9UPo0GfMSIi8B0mQLnw1/eioiISExN55513\nyM7OvqrnXGnbt/n4eGA2m6414iUFBDjPpA3K2jqcKSs4ft6AAG9iY7pyMrOED9cd5utdGbyxLJWu\nAWncNrkn44eEOuS6T47erhdypqzgfHnFOVRU1fLGslTq6m38/M4hdPbUvfYiIuc1W8xZrVby8vIa\nH+fk5BAQEADAli1bKCgoYM6cOVRXV5Oens6CBQsu+xwPDw8qKytxc3MjOzsbq9V6xfcuLCz/rufV\nRECAN7m5pS3yWq1NWVuHM2UF58rbLbgT907rzYzhYXy6JY3N+7L483928sFnB5gRF86YmGAsLfSj\nzPflTO3qTFmhZfKqGJRLWfT5YXIKK5geF86Q3lan+u9CRKS1Nfuz+ejRo1m9ejUAqampWK3WxiGW\n06dP59NPP2XJkiW8/vrrREdHk5CQcNnnjBo1qnH7mjVrGDt2bGudl4i0sUBfD354XV/+8PBIJg8N\npfhsNe+vOcxTbySxKjmdyupae0cUESeTdG5plMhgb24Z193ecUREHE6zPXNDhgwhOjqa2bNnYzAY\nmD9/PomJiXh7exMfH3/VzwH48Y9/zNNPP83ixYsJCQnhpptuatmzERG78+vsxpz4Xswa1Y01W9P5\nYmcGS9Yf5dMtaUwZFsqUoaF4uFnsHVNEHFxOYTnvrTmEm4uJh2+Idshh2yIi9nZV98z98pe/bPK4\nT58+Fx0TGhrauMbcpZ4DDUM233nnnWvNKCJOqLOnC7dN7MGMERGs23GatdtP8cnGE6zems6kIaHE\nDw+jk4fufRGRi9XW1fPGslSqqut48Pp+WH087B1JRMQhabVNEWlVXu4WbhwTydThYXy5K4PVW0+x\nMimNz7edYtygEKbHhuPbyc3eMUXazNmzZ3n66acpLi6mpqaGxx57jICAAH77299iNBrp1KkTf/zj\nH3F3d7d3VLtJ/Oo4J7NKGdU/iJHRQfaOIyLisFTMiUibcHc1MyMugslDQtm4J5NVyWms3X6a9SkZ\njB4QzHUjwvXru3QIH3/8MZGRkTzxxBNkZ2dzzz334O/vzzPPPENMTAwvvPACiYmJzJkzx95R7WLf\niXxWJacT6OPOnPhe9o4jIuLQVMyJSJtysZiYPDSU8YNCSNqXxadb0vhq9xk27jlDXL9AZo7sRld/\nT3vHFGk1Pj4+HDp0CICSkhJ8fHx44403GicX8/X1paioyJ4R7ab4bDVvrTiAyWjg4RujcXfV1xQR\nkSvRp6SI2IXZZGTswBBGDwhm28EcViadZEtqNltSsxnaK4BZo7oREaSp6qX9mTlzJomJicTHx1NS\nUsLChQsbC7ny4svgMAAAHX1JREFU8nKWLVvGK6+8YueUba/eZuPtFfspOVvNHZN60C2ok70jiYg4\nPBVzImJXRqOBuH6BDO9rZffRPFZsTmPH4Vx2HM6lf3dfZo3sRq+wLvaOKdJili1bRkhICG+//TYH\nDx4kISGBxMREysvL+dGPfsR9991HVFRUs6/j4+OBuQXWcHSU9f0+/vIo+04UMKSPlbtm9MNoNFzy\nOEfJezWUtXUoa+txprzK2kDFnIg4BKPBwOCeAQzq4c/+tEJWbj7JvuMF7DteQK+wLswaFUF0N18M\nhkt/wRNxFikpKYwZMwZomB06JyeH6upqHn30UWbNmsUtt9xyVa9TWFj+vbM4yuL0JzJLeHflfjp5\nujAvvhf5+WWXPM5R8l4NZW0dytp6nClvR8t6pWJQxZyIOBSDwUB0N1+iu/ly5HQRKzansfd4Pn9a\nXERksDezRnZjYE9/jCrqxElFRESwe/dupk2bRkZGBp6enrz99tvExsZy22232Ttem6uoqmXh8lTq\n6m08OKsfnTy1ZImIyNVSMSciDqtnaBd+fnsX0rJKWZF0kpRDubyWuJeuAZ7MHBlBbJ/Ayw7FEnFU\nd9xxBwkJCcydO5fa2lp+85vf8OSTTxIaGkpSUhIAcXFxPP7443ZO2jY+WHOYnMIKZsSFEx3pa+84\nIiJORcWciDi8iCBvHrt5AGfyzrIyKY3k/dm8uXw/n2w8wXUjIhjVPwizyWjvmCJXxdPT86IJTr7+\n+ms7pbGvzfsySUrNIjK4EzeP627vOCIiTkfffkTEaYT4e/Lg9f1Y8PAIJgwKoaCkkn9+dpBnFiax\ndvspqmvq7B1RRK5SdkE57685jJuLiYdv6KcfZEREvgN9coqI07F2cWfe9D688Mgopg4Po6y8hn+t\nPcJTf9vMZ1vSqKiqtXdEEbmC2rp63lieSlV1HfOm9cbq42HvSCIiTknFnIg4LR9vV2ZP7smLj45i\n5sgIaurq+fDLYzz1t818svE4ZRU19o4oIpeQuOE4aVmljB4QxIjoIHvHERFxWrpnTkScXicPF24d\nH8WMuHDWpWTw+bZTLN90ktXbTjFxcFemDQ+js5ervWOKCLDveD6rtqYT6OPOnPhe9o4jIuLUVMyJ\nSLvh4Wbh+lHdmDosjA27Mli1NZ1Vyems23GasTHBzLmuH5r7UsR+isuqeGvFfkxGA4/c2B83F30N\nERH5PvQpKiLtjquLiamx4UwcEsqmvZl8uiWNL1Iy2LDrDLF9A5keF06Y1cveMUU6lHqbjbdWHqCk\nvIbZk3oQEXT5RXBFROTqqJgTkXbLYjYyYXBXxsQEk7w/m893nCYpNYuk1Cz6d/dlRlwEfcK7YNAC\n5CKtbs3WU6SeKCAmyo/44WH2jiMi0i6omBORds9sMjJ6QDA3TuzJF8kn+WxLOvuOF7DveAERQd7M\niAtnaO8ATEbNCSXSGk5klvDRhmN09nThvuv66gcUEZEWomJORDoMg8FATJQ/MVH+HD9TwqrkNHYc\nyuWNZakEdHFj6vBwxsQE42ox2TuqSLtRUVXLwmWp1NfbeOD6fnTydLF3JBGRdkPFnIh0SN1DOvHo\nzQPILixnzdZTfL03k0WfH2bZ1yeYNKQrk4eG4u2hL50i39cHaw6RU1TBdSMiiO7ma+84IiLtioo5\nEenQAn08uHtab24cE8m6Haf5IuU0yzedZFVyOmNigpkaG461i7u9Y4o4pU17M0lKzSYyuBM3jY20\ndxwRkXZHxZyICNDJ04Wbx3XnuhERbNxzhtVbT/FFSgbrd2YwrLeV6XHhRAZ3sndMEaeRVVDOB2sO\n4+5q4uEbozGbdE+qiEhLUzEnInIBVxcTU4aFMXFIV7YfzOWz5DS2Hcxh28Ec+oR3YcaICPpH+moC\nB5ErqK2rZ+GyVKpq6njohn7q3RYRaSUq5kRELsFkNBLXL5DYvlb2pxWyKjmd1BMFHEwvIjTAk+lx\n4cT2DVRvg8glfLThGGnZpYwZEMyIfkH2jiMi0m6pmBMRuQKDwUB0N1+iu/mSnl3KquR0th7I4a0V\nB0j86jhTh4UxdmAI7q76OBUB2HMsn9VbTxHo68Fd8T3tHUdEpF3TT8oiIlcpPNCbh26I5g+PjGDK\nsFDKKmr4zxdHefKvm/lowzGKy6rsHVHErorLqnh75X7MJgOP3BCNm4t+5BARaU36lBURuUb+nd25\na0ovbhgdyfqdGazbfoqVSWms3prOqP5BTIsNJ9jP094xRdpUvc3GWyv2U1pew52TexIR5G3vSCIi\n7Z6KORGR78jL3cL1o7oxbXgYm/dlsXprOl/tzmTj7kwG9fRnRlwEPUI72zumSJtYnZxO6slCYqL8\nmDIs1N5xREQ6hKsq5hYsWMDu3bsxGAwkJCQQExPTuG/JkiUsXboUo9FInz59mD9/PkuXLmX58uWN\nx+zbt4+dO3dy9913U15ejoeHBwBPP/00/fv3b+FTEhFpWy4WExMGd2XcwBB2Hsnls+R0dh7JY+eR\nPHqEdmZGXDgDe/hj1AyY0k4dP1NC4lfH6ezlwn0z+2q2VxGRNtJsMbd161bS0tJYvHgxx44dIyEh\ngcWLFwNQUVHBypUrWbRoERaLhXnz5rFz505uu+02brvttsbnf/bZZ42v9/zzz9OrV69WOh0REfsx\nGg0M7W1lSK8Ajpwu5rMtaew+ls9rp/cS7OfBtNhwRkYHYTHrdmVpPyqqalm4fB/19TYenNWPTh4u\n9o4kItJhNFvMJSUlMWXKFACioqIoLi6mrKwMLy8v3N3deffdd4GGwq6srIyAgIAmz//LX/7Cyy+/\n3ArRRUQck8FgoFdYF3qFdSEj7yyrk9NJSs3in58d5OOvjjNlWCgTB3fFw81i76gi34vNZuP91YfI\nLapk5sgI+nXztXckEZEOpdliLi8vj+jo6MbHvr6+5Obm4uXl1bjtzTff5L333mPevHmEhYU1bt+z\nZw/BwcFNCrxXX32VwsJCoqKiSEhIwM3N7bLv7ePjgdlsuuaTupSAAOe5EVtZW4czZQXnyqusV36/\nQX2DyC+uYPlXx1m15SQfbTjOp1vSmDaiGzeOi8L/MgsqO1O7gvPlle9v874stuzPJiqkEzeOibR3\nHBGRDueaJ0Cx2WwXbXvooYeYN28eDz74IEOHDmXo0KEALF26lJtvvrnxuHnz5tG7d2/Cw8OZP38+\nixYt4v7777/sexUWll9rvEsKCPAmN7e0RV6rtSlr63CmrOBceZX16s0aEc6kQSFs2J3B59tO8cmG\nY/x343Hi+gUyPTacUOs3P5LZO+u1aom8KgadS1ZBOR+sOYy7q4mHbojGbNLwYRGRttbsJ6/VaiUv\nL6/xcU5OTmNPW1FREdu2bQPAzc2NcePGkZKS0nhscnIygwcPbnwcHx9PeHg4AJMmTeLw4cMtcxYi\nIk7Cw83MjLgIXvzRKO67ri+Bvh5s3pfFr/+xlf9bspuDaYWX/NFMxJHU1NbzxrJ9VNXUcc/0PgRc\npndZRERaV7PF3OjRo1m9ejUAqampWK3WxiGWtbW1PPPMM5w9exaAvXv3EhnZMMwiOzsbT09PXFwa\nboS22Wzce++9lJSUAA2FXs+ePVv+jEREnIDZZGRMTDC/vT+Wn/wghl6hndl7PJ8X/72T37+3na93\nZ1Bfr6JOHNNHG46Rnl3GmJhgYvsG2juOiEib+/LLdVd13HPPPceZMxmtlqPZYZZDhgwhOjqa2bNn\nYzAYmD9/PomJiXh7exMfH89jjz3GvHnzMJvN9O7dm8mTJwOQm5uLr+83N0IbDAZuv/127r33Xtzd\n3QkMDOTHP/5xq52YiIgzMBoMDOrhz6Ae/hzLKGZVcjoph3N54b3tWLu4My02jNEDgnGxtMz9wyLf\n155jeazZdoogXw/mTNHs1CLS8WRmnmHt2tVMmDC52WP/53/+p1VvmzDYHHg8T0uduDPde6KsrcOZ\nsoJz5VXWlpdVUM5Xe7NYuzWd2rp6vNwtTBkayqShoXi5O+YMmLpnru21xL/la/3/raisivn/2EpF\nVS3/b94wwgPbfkIhZ/hvGJS1tShr63GmvPbO+uSTP+XAgVSKi4uZOnUGmZln+POf/8rzz/+W3Nwc\nKioquO++hxg9eiy/+MWjPP74L1i/fh1nz5aRnp5GRsZpfvKTJxg5cvRVvd+Vro/XPAGKiIi0riBf\nDx77wUCmDQtl3Y5TrE/J4JOvT/BpchpjB4QwNTZM9yhJm6u32XhrxX5Ky2u4c0rPNi/kREQuZckX\nR9l2MKdFX3N4Hyu3T+px2f133nk3iYlLiIyMIj39JH/961sUFhYQGzuCGTNmkZFxmmeffYbRo8c2\neV5OTjYvv/wqW7ZsZtmyj666mLsSFXMiIg6qs6cLt4yL4roREWzcncmabemsSznNFztPM7yPlelx\n4XQL6mTvmNJBrEpOZ//JQgZG+TFlaKi944iIOIS+fRuWcPP27sSBA6ksX56IwWCkpKT4omNjYgYB\nDRNMlpWVtcj7q5gTEXFwbi5m4oeHMXFIV7YfzOGz5HS2Hshh64Ec+kb4MCMunOhIXwwGg72jSjt1\n7EwxH391nM5eLtw3s6/+rYmIw7h9Uo8r9qK1Noul4faHzz9fRUlJCX/5y1uUlJTwwAN3X3SsyfTN\n/e8tdaebijkRESdhNhkZER1EXL9AUk8WNPaUHEgrJMzqxfS4cIb3sWq9L2lR5ZW1LFyWSn29jYdm\n9cPbw8XekURE7MpoNFJXV9dkW1FREcHBIRiNRjZs+IKampq2ydIm7yIiIi3GYDDQP9KPX84ezPx7\nhxPb18rp3DL+/t/9/GphEmu2naKyutbeMeUyzp49y+OPP87dd9/N7Nmz2bhxI/X19bz88suMGDHC\n3vGasNlsvL/mEHnFlVw3MoK+3Xybf5KISDsXERHJoUMHOXv2m6GSEyZMYvPmjfz0pz/C3d0dq9XK\nO+/8vdWzqGdORMSJRQR588iN/bl1fAVrtp1i454z/GfdEf676QQTBndlyrAwOnuqJ8WRfPzxx0RG\nRvLEE0+QnZ3NPffcw0033URwcLDDLRi/aW8WyfuzieraiRvHRNo7joiIQ/Dx8SExcWWTbcHBIbz7\n7n8aH0+dOgP4ZubN7t2/GQravXsPXn/9zRbJop45EZF2IKCLO3Pie/Hyo6O5aWwkBoOBlUlpPPnX\nzby76iBZBeX2jijn+Pj4UFRUBEBJSQk+Pj7MnTuXOXPm2DlZU5n5Z1n0+WHcXU08fH20hu+KiDgg\n9cyJiLQjXu4WbhgdyfTYcDbty2J1cjobdp3hq11nGNwrgBlx4UR17WzvmB3azJkzSUxMJD4+npKS\nEhYuXIiXl5e9YzVRU1vPwuWpVNXU8ciN0fhrKQwREYekYk5EpB1ysZiYOLgr4weGkHI4l8+S00g5\nnEvK4Vx6hnZmRlwEMT38MGpWwja3bNkyQkJCePvttzl48CAJCQkkJiZe8+v4+HhgNpuaP7AZl1qM\n9u/L9pKeXUZ8bDgzx9lvlrhLcabF5ZW1dShr63GmvMraQMWciEg7ZjQaGNbHytDeARw+VcRnyens\nOZbPkdN7CPbzYHpsOCOig7CYNYSuraSkpDBmzBgA+vTpQ05ODnV1dU2mrL4ahYXff+js+Xs5LrT7\naB7LvzpOsJ8Ht4yJvGi/PV0qr6NS1tahrK3HmfJ2tKxXKgZVzImIdAAGg4He4T70DvfhdG4Zq5PT\n2bI/m3c+O0jixuNMHRbG+EFd8XDTZaG1RUREsHv3bqZNm0ZGRgaenp7XXMi1lsLSKt5eeQCzycDD\nN0Tj6uIYuURE5NJ01RYR6WBCA7y4f1Y/bh7XnbXbT/Plrgw+/PIY/918kgmDujJlWCi+ndzsHbPd\nuuOOO0hISGDu3LnU1tbym9/8ht/97nccPnyYsrIy7r77biZNmsQPf/jDNs1VX2/jrRX7Kauo4a4p\nPQkPdJ4hTCIiHZWKORGRDsq3kxu3T+rBrFERfLnrDJ9vP8Wqrel8vv0UI/oFMi0unNAAx5qYoz3w\n9PTklVdeabJt5MiRdkrzjc+S0ziQVsigHv5MHhpq7zgiIk7vBz+4nk8/Xdn8gd+DijkRkQ7Ow83C\ndSMiiB8WxpbULFZtTWfTviw27csiJsqPGXHh9ArrgkGTpbRbxzKK+firE3TxcuGH1/XR/9ciIk5C\nxZyIiABgMRsZOzCE0THB7Dmaz2fJaew5ls+eY/lEBndiRlw4Q3oFYDTqi357Ul5Zy8LlqdhsNh68\nPhpvDy0yLyJyJffdN4cFC/5IUFAQWVmZ/OpXTxAQYKWiooLKykp+/vMn6devf5tkUTEnIiJNGA0G\nBvX0Z1BPf45mFLMqOZ2dh3P56yf7sPq4My02nNH9g3CxaHIMZ2ez2Xhv9UHyiiuZNaobfSN87B1J\nROSaJB5dwc6cvS36moOtA7ilx6zL7h83biKbNn3FrbfezsaNGxg3biJRUT0ZN24CO3ZsY9Gid3nu\nuZdaNNPlaC5qERG5rB5dO/P4LQP4/YNxjBsYQkFJFe+vPsSTf9vM8k0nKKuosXdE+R7WbUtn64Ec\norp24sYx3ewdR0TEKTQUcxsB+PrrDYwZM54NG9bxox/dz9/+9hrFxcVtlkU9cyIi0qxgP0/undGH\nm8dGsnbHadanZPDJxhN8uiWNsTEhPHRLjL0jyjXKzD/LGx/vxd3VzMPXR2My6vddEXE+t/SYdcVe\ntNbQvXsU+fm5ZGdnUVpaysaNX+Lvb+XZZ3/HwYP7ef31P7dZFn1yi4jIVevs5cqt46N46dFRzJ7c\nEy93C+t2nGbjrjP2jibXaPvBHKqq67h3Rh/8u7jbO46IiFMZOXIMb775V8aOHU9xcRFduzbMArxh\nw3pqa2vbLId65kRE5Jq5u5qZOjyMSUO6cvxMCXEDu1JYcNbeseQaTBkWxoThEXi76HddEZFrNX78\nRB555D7++c9/U1lZwe9/P5/169dy6623s3btGlauXN4mOVTMiYjId2Y2GekV1gWzSQWBs3F3NRMQ\n4E1ubqm9o4iIOJ2+faPZsCG58fGiRUsb/z5mzHgAZs68AU9PT8rLW+9zVldfERERERERJ6RiTkRE\nRERExAmpmBMREREREXFCKuZERERERESckIo5ERERERERJ3RVs1kuWLCA3bt3YzAYSEhIICbmm8Vh\nlyxZwtKlSzEajfTp04f58+ezdetWfvrTn9KzZ08AevXqxbPPPktmZiZPPfUUdXV1BAQE8NJLL+Hi\n4tI6ZyYiIiIiItKONVvMbd26lbS0NBYvXsyxY8dISEhg8eLFAFRUVLBy5UoWLVqExWJh3rx57Ny5\nE4DY2FheffXVJq/16quvctdddzFjxgz+9Kc/sXTpUu66665WOC0REREREZH2rdlhlklJSUyZMgWA\nqKgoiouLKSsrA8Dd3Z13330Xi8VCRUUFZWVlBAQEXPa1kpOTmTx5MgATJ04kKSmpJc5BRERERESk\nw2m2mMvLy8PHx6fxsa+vL7m5uU2OefPNN4mPj2f69OmEhYUBcPToUR555BHuvPNONm3aBDT05J0f\nVunn53fR64iIiIiIiMjVuap75i5ks9ku2vbQQw8xb948HnzwQYYOHUq3bt14/PHHmTFjBqdOnWLe\nvHmsWbOm2df5Nh8fD8xm07VGvKSAAO8WeZ22oKytw5mygnPlVdbW4UxZwfnyioiIOLtmizmr1Upe\nXl7j45ycnMahlEVFRRw5coThw4fj5ubGuHHjSElJYejQoVx33XUAhIeH4+/vT3Z2Nh4eHlRWVuLm\n5kZ2djZWq/XK4VqokBMREWlvWqp4drYi3JnyKmvrUNbW40x5lbVBs8MsR48ezerVqwFITU3FarXi\n5eUFQG1tLc888wxnz54FYO/evURGRrJ8+XLefvttAHJzc8nPzycwMJBRo0Y1vtaaNWsYO3Zsq5yU\niIiIiIhIe2ewXcV4x5dffpnt27djMBiYP38++/fvx9vbm/j4eBITE1m0aBFms5nevXvzv//7v5w9\ne5Zf/vKXlJSUUFNTw+OPP8748ePJycnh6aefpqqqipCQEJ5//nksFktbnKeIiIiIiEi7clXFnIiI\niIiIiDiWZodZioiIiIiIiONRMSciIiIiIuKEVMyJiIiIiIg4oWteZ86RLViwgN27d2MwGEhISCAm\nJqZx3+bNm/nTn/6EyWRi3LhxPPbYY3ZMeuWskyZNIigoCJOpYWmGl19+mcDAQHtFBeDw4cM8+uij\n3HvvvcydO7fJPkdr2ytldbS2ffHFF9mxYwe1tbU8/PDDTJ06tXGfo7XrlbI6WrtWVFTwzDPPkJ+f\nT1VVFY8++igTJ05s3O9IbdtcVkdrW4DKykpmzZrFo48+yi233NK43ZHaVS6ma2TrcKbrI+ga2Vqc\n5Rqp62Prssv10dZOJCcn2x566CGbzWazHT161Hb77bc32T9jxgzbmTNnbHV1dbY777zTduTIEXvE\ntNlszWedOHGirayszB7RLuns2bO2uXPn2v7f//t/tvfff/+i/Y7Uts1ldaS2TUpKsj3wwAM2m81m\nKygosI0fP77Jfkdq1+ayOlK72mw228qVK21vvvmmzWaz2U6fPm2bOnVqk/2O1LbNZXW0trXZbLY/\n/elPtltuucX20UcfNdnuSO0qTeka2Tqc6fpos+ka2Vqc6Rqp62Prssf1sd0Ms0xKSmLKlCkAREVF\nUVxcTFlZGQCnTp2ic+fOBAcHYzQaGT9+PElJSQ6Z1RG5uLjw97///ZKLvDta214pq6MZPnw4r7zy\nCgCdOnWioqKCuro6wPHa9UpZHdF1113Hgw8+CEBmZmaTX+ocrW2vlNURHTt2jKNHjzJhwoQm2x2t\nXaUpXSNbhzNdH0HXSHtkdTS6PrYee10f280wy7y8PKKjoxsf+/r6kpubi5eXF7m5ufj6+jbZd+rU\nKXvEBK6c9bz58+eTkZHB0KFDeeKJJzAYDPaICoDZbMZsvvQ/FUdr2ytlPc9R2tZkMuHh4QHA0qVL\nGTduXONQAUdr1ytlPc9R2vVCs2fPJisrizfeeKNxm6O17XmXynqeI7XtCy+8wLPPPssnn3zSZLuj\ntqs00DWydTjT9RF0jWwtzniN1PWx5dnr+thuirlvsznR8nnfzvqTn/yEsWPH0rlzZx577DFWr17N\n9OnT7ZSufXHEtl27di1Lly7lH//4h11zXI3LZXXEdgX4z3/+w4EDB3jyySdZvny53S+eV3K5rI7U\ntp988gmDBg0iLCzMLu8vLUfXSLkUR2xbXSNbh66PLcue18d2M8zSarWSl5fX+DgnJ4eAgIBL7svO\nzrbrEIMrZQW46aab8PPzw2w2M27cOA4fPmyPmFfF0dq2OY7Wths3buSNN97g73//O97e3o3bHbFd\nL5cVHK9d9+3bR2ZmJgB9+/alrq6OgoICwPHa9kpZwbHa9ssvv2TdunXcfvvtfPjhh/z1r39l8+bN\ngOO1qzSla2Tbc7R2vRqO1ra6RrY8XR9bhz2vj+2mmBs9ejSrV68GIDU1FavV2jgkIzQ0lLKyMk6f\nPk1tbS3r169n9OjRDpm1tLSU+++/n+rqagC2bdtGz5497Za1OY7WtlfiaG1bWlrKiy++yMKFC+nS\npUuTfY7WrlfK6mjtCrB9+/bGX0bz8vIoLy/Hx8cHcLy2vVJWR2vbP//5z3z00UcsWbKE2267jUcf\nfZRRo0YBjteu0pSukW3P0dq1OY7WtrpGtg5dH1uHPa+PBpszjbVoxssvv8z27dsxGAzMnz+f/fv3\n4+3tTXx8PNu2bePll18GYOrUqdx///0Om/Xdd9/lk08+wdXVlX79+vHss8/atft73759vPDCC2Rk\nZGA2mwkMDGTSpEmEhoY6XNs2l9WR2nbx4sW89tprREZGNm6Li4ujd+/eDteuzWV1pHaFhqmB/+d/\n/ofMzEwqKyt5/PHHKSoqcsjPg+ayOlrbnvfaa6/RtWtXAIdsV7mYrpEtz5muj1eT15HaVtfI1qHr\nY+tr6+tjuyrmREREREREOop2M8xSRERERESkI1ExJyIiIiIi4oRUzImIiIiIiDghFXMiIiIiIiJO\nSMWciIiIiIiIE1IxJyIiIiIi4oRUzImIiIiIiDghFXMiIiIiIiJO6P8DwYftnkVtln4AAAAASUVO\nRK5CYII=\n", "text/plain": [ "" ] }, "metadata": { "tags": [] } } ] }, { "metadata": { "id": "rFYs9dH3ui66", "colab_type": "code", "outputId": "bb093304-5a6a-4ccd-92f5-d9abcc6a764a", "colab": { "base_uri": "https://localhost:8080/", "height": 50 } }, "cell_type": "code", "source": [ "# 测试性能\n", "trainer.run_test_loop()\n", "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n", "print(\"Test Accuracy: {0:.2f}%\".format(trainer.train_state['test_acc']))" ], "execution_count": 71, "outputs": [ { "output_type": "stream", "text": [ "Test loss: 0.45\n", "Test Accuracy: 84.29%\n" ], "name": "stdout" } ] }, { "metadata": { "id": "vKDmGXAyukpV", "colab_type": "code", "colab": {} }, "cell_type": "code", "source": [ "# 保存所有结果\n", "trainer.save_train_state()" ], "execution_count": 0, "outputs": [] }, { "metadata": { "id": "P4yiDxOavmJy", "colab_type": "text" }, "cell_type": "markdown", "source": [ "通过对比我们可以发现，使用没有冻结的GloVe嵌入层在测试集上取得了最好的成绩。不同的任务会产生不同的结果，所以我们需要根据实验结论来选择是否使用冻结技术。" ] }, { "metadata": { "id": "F0yKVPF8xBQK", "colab_type": "text" }, "cell_type": "markdown", "source": [ "# TODO" ] }, { "metadata": { "id": "A0e8p2LkxD6x", "colab_type": "text" }, "cell_type": "markdown", "source": [ "* 什么时候使用skip-gram/CBOW\n", " * 根据Mikolov的观点：\n", " * Skip-gram：能够很好地处理少量的训练数据，甚至能很好地表示罕见的单词或短语。\n", " * CBOW：训练速度比skip-gram要快几倍，对频繁出现的单词的准确率高。\n", "* 读取word2vec\n", "* 可解释的卷积层过滤器（我们在处理单词的时候应用了conv卷积层）\n", "* [基于上下文的词嵌入（contextualized word embeddings）](https://arxiv.org/abs/1607.00578)" ] } ] }