{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "オリジナルの作成: 2016/05/03" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\tHiroshi TAKEMOTO\n", "\t(take.pwave@gmail.com)\n", "\t\n", "\t

SageでTheanoの手書き数字認識（MNIST）を試す

\n", "\t

参考サイト

\n", "\t

\t\n", "\t\tここでは、\n", "\t\t 人工知能に関する断創録\n", "\t\t のTheanoに関連する記事をSageのノートブックで実装し、Thenoの修得を試みます。\t\n", "\t

\n", "\t

\n", "\t\t今回は、TheanoのTutorialからMNISTの手書き数字認識を以下のページを参考にSageのノートブックで試してみます。\n", "\t\t

\n", "\t\t\t\tTheanoによるロジスティック回帰の実装\n", "\t\t\t

\t\t\n", "\t

\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

処理系をSageからPythonに変更

\n", "\t

\n", "\t\tSageでTheanoのtutorialのMNISTを実行すると、以下のy_predがベクトルではなくスカラーになり、うまく処理できません。\n", "\t\t

\n",
    "\tself.y_pred = T.argmax(self.p_y_given_x, axis=1)\t\t\t\n",
    "\t\t

\t\t\n", "\t

\n", "\t

\n", "\t\tそこで、ノートブックの処理系をSageからPythonに切り替えます。上部の左から4つめのプルダウンメニューから\n", "\t\t「python」を選択してください。\n", "\t

\n", "\t

\n", "\t\t\n", "\t

\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

必要なライブラリをインポート

\n", "\t

\n", "\t\t今回は、ちょっと多くのライブラリを使用します。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# 必要なライブラリのインポート\n", "import six.moves.cPickle as pickle\n", "import gzip\n", "import os\n", "import sys\n", "import timeit\n", "import urllib\n", "\n", "import numpy as np\n", "import matplotlib.pyplot as plt \n", "%matplotlib inline\n", "\n", "import theano\n", "import theano.tensor as T" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

データのセットアップ

\n", "\t

データのダウンロード

\n", "\t

\n", "\t\tTheanoのTutorialで使用しているMNISTは手書き数字の画像データセットは、\n", "\t\tcPickleモジュールでロードできる形式に圧縮したmnist.pkl.gzが以下のサイトから\n", "\t\tダウンロードできます。\n", "\t\t

http://deeplearning.net/data/mnist/mnist.pkl.gz

\n", "\t

\n", "\t\tMNISTのデータには、70000の手書き数字データが収録されおり、それを以下の様に分割して使用します。\n", "\t\t

50000の訓練セット
10000のバリデーションセット
10000のテストセット

\n", "\t\t\n", "\t\t各サンプル画像は、28x28=784個の配列に格納され、データの値は0.0から1.0に正規化されています。\n", "\t

\n", "\t

\n", "\t\tTensorFlowのMNIST for ML BeginnersのMNISTデータの説明が分かりやすいので、引用します。\n", "\t

\n", "\t

\n", "\t\t\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# データのダウンロード(最初１回だけ実行)\n", "# urllib.urlretrieve(\"http://deeplearning.net/data/mnist/mnist.pkl.gz\", \"data/mnist.pkl.gz\")\n", "# Sageサーバではメモリ不足のため、1/10のmini_mnist_pkl.gzを作成してこれで計算することにした" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# 以下の手順でmini_mnist.pk.gzを作成（コメントを外す）\n", "#mini_train_set = (train_set[0][0:5000], train_set[1][0:5000])\n", "#mini_valid_set = (valid_set[0][0:500], valid_set[1][0:500])\n", "#mini_test_set = (test_set[0][0:500], test_set[1][0:500])\n", "#f = gzip.open(DATA+\"mini_mnist.pkl.gz\", 'wb')\n", "#pickle.dump((mini_train_set, mini_valid_set, mini_test_set), f)\n", "#f.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

1/10のサブセットを作成

\n", "\t

\n", "\t\tSageサーバのメモリは1Gと少ないため、MNISTのデータをそのまま使用するとメモリ不足になります。\n", "\t\tそこで、1/10のサイズのサブセットデータmini_mnist.pkl.gzを作成し、これを使用して数字を認識することにします。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# データセットをロードする\n", "#f = gzip.open(\"data/mnist.pkl.gz\", 'rb')\n", "f = gzip.open(\"data/mini_mnist.pkl.gz\", 'rb')\n", "train_set, valid_set, test_set = pickle.load(f)\n", "f.close()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2 (5000, 784) (5000,) \n" ] } ], "source": [ "# データは、(画像データとラベル）のタプルで、画像データは5000サンプルで、各画像データは28x28=784、ラベルはint64の型\n", "print len(train_set), train_set[0].shape, train_set[1].shape, type(train_set[1][0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

サンプルデータについて

\n", "\t

\n", "\t\tどのような画像データが入っているのか、訓練用データ（train_set）の最初の100個を表示してみましょう。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWkAAAEACAYAAABxgIfcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXdUFdfa/589HgReUdDQ/En1KuK1BFzWawNeY+QVC15r\n1CheY0nR6LJeOy5L7Mo1id3IjbHEnms0GkW9EksExaiIxoKCBRFBMCDC9/eHmckZOGU4ZwpJ5rPW\nrKVzhrM/e8+cZ/bsmdkPA0A6Ojo6OpUTTmsBHR0dHR3z6EFaR0dHpxKjB2kdHR2dSowepHV0dHQq\nMXqQ1tHR0anE6EFaR0dHpxKjB2kdHR2dSowepHV0dHQqMXqQ1tHR0anE6EFaR0dHpxKjB2kdHR2d\nSoxBhTLknhyE2fh3uocY3UOM7iFG9xCjmYfek9bR0dGpxOhB2gQxMTHEcRzFxMRoraKjUw7GGDVp\n0kRrDZ0KEBERYfPfahakb968SVu2bKERI0YQx3Gixd/fnziOoxo1alBCQoJqTufOnSNfX1/asmUL\nubq60sGDB+mHH36gly9fKlLeyZMnJW+7cOFCioqKUsRDCps3byYXFxfiOK5C3n8kbt++TaNHj6Yq\nVaoIi9oUFBQQY4yuXr2qetlERD/99BNNnDiRAgICqEqVKsRxnKg9qlSpQsOGDVPViTFGtWvXVrVM\nqUyZMoWcnZ1p1KhRtn8JAKWXciQlJYExZnIxGAzYsmULdu3ahR9++MHUn8vmYUxBQQH8/f3BcRw4\njkOLFi2wc+dOMMYwb948RTzmzJljTUtgxIgRaNKkiSIeUpg9ezYMBgMYYzhx4oSqHh988AGaNGkC\nxhiWLVuGbdu24cyZM+Y2V8Tj2rVrqFatGjiOg7+/Pxo3bgyO43Dt2jVVPQCAMQaO46RsKrtHt27d\nhN8Ix3GCS9lFaQ/RF78eL5ayqaIepujYsaPd7WGrsN2VGzZsGOLi4sAYQ2hoKPLz83H58mW89957\n1uoteyP/+OOPcHNzA2MM4eHhYIyhTp06uHjxIogI/fr1U8QjICDAWl1/K4wIsbGxinhYo06dOjAY\nDDAYDBYV5fQoLi5GYmIiqlevjvDwcBw7dgzPnz9HZGQkOI7D4sWLVfHYtGmTKPgEBQUBANLS0sBx\nHE6dOqWKB09GRoYQGO/cuWNtc9k9iouLkZ6ejvT0dNH63Nxc+Pr6guM49OrVS3EP0RdXkiDNGCt3\nPBARdu/ebZeHrcJ2Vy43NxelpaV477338OWXX0ppgwpXTopHcnIy3NzcwHEcoqKi8Pz5c8yfPx+P\nHz8G8LrhXVxcZPe4dOkSnJ2dpVeaSNUrC2P4q5zw8HCLinJ6GAfH3NxcAEB8fLywjt8/SnvwJwWO\n49C/f3/cvXsXALB//35NgjTwW086Li5OyuaKHx8AsGPHDri4uIDjOIwdO1ZVj8oSpDmOw5EjR4T/\nJycngzGGrVu32uVhq7BslZswYQLCw8NRUlIipR0qVDkpHgMGDABjDJ6enti5c2e5zy1cWtrlsWDB\nAskH1sOHD0FE5XovcnhYIysrCwaDAe7u7vj+++8tbSqrB9/uH330kbAuODjY0uW0Ih4ZGRmYNWsW\nTp8+LVq/bt06PUj/yldffYXw8PByJ1W1PNzc3DQP0tOnT0ejRo2EzkN+fj769++PNm3a4OXLl3Z5\n2CosayPXqFEDfn5+iIuLQ2lpqbXNZfNITEwEx3FISEgwW5hSQXro0KFWD6zt27ejW7ducHR0RHx8\nvLnNFNsvn332GQwGA95++208e/bM2uayeOTk5MDLyws9evQQ1t24cQMtW7YEx3Ho27cv8vPzFfcw\nxcqVKzF//nzMnz8fLVq0AMdxePXqleoeWgbp27dvIzY2Fh06dECHDh1EY9Jubm64d++eKh7GdOvW\nTbMgnZKSgnr16olixOLFi1G1alXZOhO2CsvayLt374arqysYY1i4cCEyMzNlqZw1j9atW1u7hAcR\ngTEmu0fZIH3x4kUkJSVh8eLFGD16NGrUqAEPDw9ERUWhRo0amtygcnV1hcFgwMOHD6VsLovHo0eP\nwHEc7t69i0ePHmHhwoWoUaOGcFN5//79qngYU1BQgHPnzpW7yV2nTh1VPXi0DNIBAQFmbxx2795d\nNQ9jtArSxcXFYIwJMWLevHkoLCxEy5Yt4eDggJiYGFk8bBWWtZF5tmzZIux4OSpnyWPHjh3gOM7i\nCaGwsBCMMbz11luKeCxcuBDdu3cXlg0bNpQbd16zZo21A1CR/TJq1CgpNwsV8Thz5gwYYwgKCsLE\niRPx4MEDdOzYER9//LGqHkVFRUIA4m/aZmdni4LUtm3bUFhYqKhHuS/+NSj06dNH0uZyety5cwdz\n587F+fPncfnyZVy+fBkff/yx6sNQxixbtkz1IL1t2zYYDAZ4e3vj2LFjSE5OFh0XBoMBPj4+snjY\nKixrIxtTtWpVMMZw/PhxuytnyYMP0uYoLCzE9OnT4efnh0OHDinmYY2+ffuqHqSTk5MRGBgIg8Fg\n7k69Kh48J06cABFh1apVqnm8fPkSU6ZMAcdx6Nq1K3JycvD48WM0b94cTk5OmDlzJqKjo8FxHN5+\n+20kJSUhKSlJlfYw7r1euXJFlfawxLNnzzQN0l9//TWISNWnXcLDwxEYGCha17ZtW1GgHjx4sCwe\ntgrLvrMvXbqELl26CJeSFm4kyuJhKUgnJydjwIABmh10xmgRpD08PGAwGNC2bVs8f/5cqqpi7XHo\n0CEwxiw90SG7x+TJk8FxHFavXo2nT58CAFq1agWO43Ds2DEAr59QGjRoEGrUqAGO48o+UqlYe4we\nPVoIBGaepFDFg2f79u2aBum9e/eCiHD9+nUpm8visWLFinI38vmnxK5cuYIrV66Yu4FaYQ9bhWVp\n5AcPHmDp0qWicb6WLVti3759slTOkgcfpD/66CMkJycjPT0d3bp1g5+fHwICAtC/f39zj7zJ6mEN\ntYM0P1bOGJPaM1HEwxgtgjTHcXB2dkaDBg2EADR37lxLNwoV8TBH/fr1hfHQmzdvKu7RpUuXckE4\nOztb9FikmUdVZfUwx9ixY/HLL79I2VQRj/nz58Pb29vSjVObPdSYBc8kx44dow8//JBSU1OFdbt2\n7aIePXoQx6n3tvrq1avp66+/JldXV0pLSyMiomnTplFsbKxqDpWJI0eOEGOMHB0dycvLS2sdIiL6\n+eefNSm3qKiIbty4QV27dqUOHTrQxx9/rMmr4KZo1KiRqu3y3XffERHRP//5T2HdkSNH6MKFC8QY\no7CwMHr//fdV8zFF1apVNSn37t27tG7dOhoxYgT5+PjIX4AdZ5YKn4HOnDkDX19fUc+5WrVq1h6p\nsvkMZM6DJzMzU3SHOi0tTRMPS6jdk+Zf/a5bt25FVRVtj/r161u7slHFowIo6nHnzh3h2FWjJx0T\nE1Pu1e+QkBDRc+xWULQ9vL29LT0BpajHvHnzKvKafoU9bBW2qXKTJ08WgvOUKVMwbdo05OTkKFY5\ncx4yoYrHpk2bVA3S/LOvlS1Ib9q0CeHh4ZXiRplE/lAehYWF+Pzzz4XXvj///HNNPMzRr18/TV6T\ntwPJZTNA7rmsy3fWZf6+392k3WXQPcRI8sjLy6OaNWvS3//+d9q0aRNVq1ZNE48KoHuI0T3ESPbQ\ng7Tt6B5idA8xuocY3UNMpQrSOjo6Ojo2omdm0dHR0anE6EFaR0dHpxKjZwu3nT+NR1paGr399ttU\nWlpKd+/e1cxDIrqHGN1DzO/OQ+9J61ilXbt2lJ6eTqGhoVqr6OiY5datW9SvXz/NXmpRCs2D9NWr\nV+kvf/kLDRgwgBYtWqRY0ledivPo0SP629/+RtnZ2dSkSRPasGGD1ko6OmYJCQmh8+fP0wcffKC1\nirzY8XC33Q+BZ2dnw9PTU0h/Q0Rwc3PD4cOHZXkIXKqHjdjlMXHiRFG9TS1NmjTBvHnzcOvWLcU8\nLMG/WWYm/Y8qHgUFBSAirFu3ToqDYh42IKtHVlYWevToAcYYunbtiq5du4KIEBwcrKqHKS5evIiL\nFy/i7t27lpwU80hLS4Ozs7PU7E6/u+PDVmHZKvfZZ5+BiODn5ycEp3HjxslSuYp4mOPOnTu4du0a\nPDw84OHhgaFDh8riUbduXaG+7u7uCA8PFy2hoaHC5wcOHFC9Pa5fvy68HSoRxfYLEZnL2K6qB8/8\n+fMxePBgMMbQsGFDVTyaNWuGWrVqYcqUKcK66tWrg+M4a1nnFWmPlJQUdO7cGZ07dxaOk6lTp4KI\nEBERoZoH8Dojd6tWraRsqqhHdnY2Pv30U6E9rByzksu2VVjWyoWEhIh6kFevXpWlchX14Bk2bBha\ntmwp+NSoUQMbNmwoO8uWoh5ffvkliAhOTk44d+6cpU1l92jdujWcnJwqzVwmRARHR0dNPDp27Ahf\nX18hE7bxXM784ufnp7hHUFAQ9u7dK1p36dIlEBG8vb1Vaw8AmDVrFpydnRETE4NJkyYBeD29r6en\nJzw9Pc3NRqfI8VFYWIguXbpY20xxj6ysLHAcBy8vLxw8eBD9+/cHx3EoKiqy28NWYVl/hA8ePECT\nJk2EoGhlonnFPADgyZMnICLUqlULzZs3R2pqqpAhWi2PkSNHwsnJCURUdiJ5xT327t0rBKIKTLso\nu4foizXKBn3kyJFyATk1NRWpqak4duwY/P39wXGcuSAha3sUFBSUW5eamiqlbWTfLxMmTAARCVmN\nbty4gT59+oCIsHr1atU8ACA2NhYXLlywtpniHm3atEFkZKTw/7S0NLi7u+PixYt2e9gqLOuPsOz4\n7PLlyy1trpgH8Hpe2jFjxkiZ7F4Rj++//15oh88//1zKHLmyeowfP95k8t0VK1Zg/PjxqnmIvlij\nIP3WW28JwdnZ2Vl0RcNPvF+3bl1kZWUp6mGOly9folGjRqoH6TNnzqBRo0Z49913kZ+fj27duoEx\nhlq1aqnqAQDt2rWztokqHhEREaJJp/ggff/+fbs9bBWWpXK7d++Go6NjuRtmH374oaU/U6SRp06d\niv79+0udOFwRj/Hjx5drixYtWgj59dTw4C/p+SwjxrnbqlevDsYYWrdurUp7CF+sQZDu3LkzOI5D\n+/btcfToUdFnX375JVxdXcFxXLnP5PawRGFhIYKDgzXJgTlu3DjRlMPvvfdeuUwlSnuMHDkSTZo0\nAfDbbJFOTk7WEvQq0h5eXl5o1aqVkNV+586d1v5Ectm2CstWuZUrV5YLTFoMdxCRtYwwinucPn0a\nkZGRJp/0GD9+PB49eqSoR15eHjiOQ506dTBjxgxcv34dAwYMgKenJwYOHIikpCQwxlS/e69FkP76\n668RGhqKBw8elCuAP2k1bdpUcQ9LmBruOHr0KObOnYvU1FRFPVauXCkE6KioKCmpq2T3cHBwwM6d\nO1FUVISAgADk5eVh27ZtcHJyUtUDABo2bCh0cKpXry7rlLq2City0M2cORNEZG0uY1k9+IH97Oxs\nMMZQo0YNqbqKt0d8fDxatWolCtZKeixcuFA0zGGcty4xMREcx1ka8lCsPbQa7jAFPxS0Zs0aVT3u\n3LmDXbt2YdKkSWjevLkwzMEvAQEBCAgIwPnz58s+sil7e/Tu3VvUi5aIrB5Hjx4VHjLIy8vDkydP\nhM+sBEjFjo+UlBQwxmTPtWirsOwHP/Dbj9HKs5+yeGRmZiI0NBTx8fHCusryyJkxxcXF6NixoypB\nesqUKYiOjgbwepiDMYbly5fj+vXrCAgIsNY2irVHZQnS/CNmjDHjnqqiHi9evMCkSZNEAdnV1RV1\n6tQR/v/ee+8p7sEzd+5coQ2GDRumeZDOy8sr95lcCWCleBizdetWEMmfEFfzNw5N8Y9//EPxMpo1\na0apqak0aNAg0fq33npL8bIrgsFgoGbNmmlSNmOMUlJSqF27duTp6Un37t3TxKOykJycTIwxWrVq\nFdWvX1+VMnv27EmLFi2iqKgo+uijj+jo0aP0008/0f3794mIqG7durRs2TJVXIiIZs6cSURE8+bN\no7CwMNXKLQtgfiqNEydOqGjyG87OzsQYk/+1dDvOLDadgQYMGAAiQkJCAm7cuCGcgfhnpcePH2/t\n5p1sZ8LFixcLz3Z6eHggNTVVk2zQDx48ABHh2bNnovUPHz4U9aLMjIHK5vHjjz+afBb4gw8+ULU9\nyn2xxj3pvLw89OvXDxzHYeXKlap6EBFCQkJE66KiokBEOH78uGoewOsribCwMACvnyzx8PDQbL9k\nZmaidu3a+PTTT4V1L1++xLBhwyy9YCS7B8+zZ8/AcRyWLFlibdMKe9gqbHPlEhMT0bp1a2EczfhG\nWXBwsJSktIoFgwoim0e/fv1AREhOTsaLFy8AvL7MrVGjhtA21atXR0JCgqIeABAeHi4E6ZiYGGuP\nQyrmYYyEx8wU9bh69So4jkNQUJBUB9k8GjRoACJC9+7dMWPGDGzatAkcx5l7wkYxDwCYNm0awsPD\n8fLlS+zYsQO1atWyNtSiiAdPXFwcnJyckJubi6SkJAQFBaFp06aWHntTxAMARowYgYEDB1rbzCYP\nW4XtrtyiRYuwaNEioWft6uoqe+WkeNiBbB63bt0yO3+Hg4MDJk6cqIqHnSjmMXz4cM2C9IABA0Q3\nUCvAH26/LF++HIwx4WUvvkOhtoedyOpx+/Zt1KxZE87Ozop5qDGftEkmTpwo/Hvr1q1aaVQKAgMD\n6auvvqJjx45RUFAQ7d+/n4KDgykiIoIaNGjwp58idNq0afTTTz9pUvbz58+JiGjkyJGalF+ZGDJk\nCM2YMYOaN29O3bt3J2dnZ62VNGfJkiWUm5tLvXv3VqwMPRGt7egeYv6QHlWqVCF/f3/69ttvqUGD\nBpp52IHuIUZWD47j6G9/+xt9//335OjoqIiHZj1pHZ3fAyUlJVor6FRiSktLFS9Dzxauo6OjU4mp\nlM9J6+jo6Oi8Rg/SOjo6OpUYPUjr6OjoVGLUuHFYqe/O2oHuIUb3EKN7iNE9xEj20HvSOr8bAgMD\n6ezZs1pr6OioSqUL0mlpaRQREaG1BiUkJBDHcZpN1qJTnjt37tCgQYOouLhYk/L//e9/07hx4ygi\nIoL+/e9/U1JSkiYeOn8y7HhNskKvU06bNk3Sq7UnT54Ex3HC5Ev2vE5pykMK4eHhMBgMokULj7Fj\nx8LBwUGTzBtlycvLg7e3NxwdHbF+/XrVPVJSUkBE6NmzJ0pKSixtKpvHpUuXhNRQlhYz2bpV2S93\n795F7dq1LW0ii0dqaipiYmIQExODfv36mW2LJk2amJuq026PPXv2mC23QYMGCAkJQatWraxlDldl\nv0hActm2Cle4cnyQLpvxuCw7d+7ULEgfP34cAQEBQmBu3LixZkF6w4YNaNasGTZu3Ig5c+bg2rVr\n5jZV/KArLi7GJ598AsYY/Pz8zE2qrphHcXGxMMfLjh07rG0um0dcXJwoENStW1dYnJ2drU18L3t7\nXL9+vVxS5MjISCHVmZLt4evra/VkxS9mEmfY7bF27VrJDsb5KJVoDxmQXLatwhWu3Pr164WJaoyz\n6hqTlJQkbDNv3jy7K2fKwxTx8fHCzqVfJzRnjGHz5s2q/gh5GGNYtWqV8P8FCxZY2lzxgy4kJERo\nBwuT6ijmwWdOVztIP336FNeuXcPt27fLfZaYmCi0SfXq1RX1AIBevXph8ODBonWTJ08GYwyXLl0y\n92eyeaSmpuLnn3/Gzz//LJwoPvjgA/z8889wd3cvFyRNTKNqt8e8efMQHByMmJgYnDx5EsuWLUNw\ncDAcHBzMBmul2sMUO3fuRGRkJDp06CAcr3PnzjW3ueSybRWucOVevXqF2NhYizOKZWRkoH79+qr2\npOPj41GrVi0YDAbUrVsXnp6eMBgMcHFxwa5du1TvSWdnZ6Nv374oLi4G8DptkprpxEzBH/Bubm6q\ne2zcuBFEhNDQUNWDtDkmTJgAf39/oV2M5zRWysPb27vcbIiDBw8GEakSpI0zGJWldevWosBYrVo1\nxTxM8emnn2LJkiWoWbOm6kH60aNHiI+Px4wZM4TAHBsbi5YtWwqJcc0guWxbhW2qHD8xNsdxopxk\nPMY9abWCNGNMFIg3bdoEg8Eg9GTVDNLFxcUIDg5Gdna2sK5du3aaj0nzB/zUqVMV93j+/DlOnTqF\nkSNHws3NDUSEVatW4ebNm5UiSB87dkwUBOrVq2cyWa3cHm5ubpg0aZJonbOzM6pVq2YtXZMi7ZGf\nn48WLVqgRYsW5YY6tm7dqpqHMXl5eXjw4IFozFxpDz5bO7/w6ec2bNgAIsLIkSPN/anksm0Vtrly\nfBDmex+nT5/GrFmzRJlAzAXxilbOkgcABAYGgjGG+fPnC+vKnhz4m4hKevB06dJFlN/R09NTteBo\nitGjR6N27doICQnB48ePrW0um0dhYSEuX74sGn+9ffs2iAibNm1SzcOYS5cuiS6rnZ2d0bt3b9U8\nJk6cCMYY+vbt+1sBv87DzmdLUcMDAO7fv1+R4QXFPEyRk5MjuLRp00Zxj2+//bZczOCH5+S6V2Cr\nsM2VGzhwYLkdazwOzC9mniCoUOUseSQnJ8PDw8NSIwIAwsLCVBvbMh7Dys3NBWMMo0eP1iyxZu3a\ntS1dzqvmAQBPnjwRnu5Q0+Ply5e4evUqgoKCwBiDg4MDnJ2dpaRJktXj6dOniI6OBhGhb9++ePfd\nd4Xe27Jly1TzAF4PybVs2dJkkE5KSlLNoyyPHj1CZGSkcHW8du1a1T0WLVoEIoKfnx/S0tIsbSq5\nbFuFba6c8ZAGvwQFBaFBgwaYM2eOsG748OF2V86Sh4VhDKnbybqzU1JS4OTkhODgYDg6OqJGjRpY\nsWKFVT+5PXj4LBxWekeKe4gKUHm4Iz8/H3Xr1hXaITMzs0K6cnmYLYAIDx8+1Nxj5MiRQhs5ODjg\n1KlTqnusX79e8x599erVQUTWgnOFPTR9meWNN96gkJAQun79OqWmplK3bt201ClHamoqEREFBAQo\nXlaTJk1ozZo15OnpSTt27CBXV1fNsoHcu3eP1q9fTwaDgUaNGqWJgzk8PT1VK2vTpk10+/ZtIiIK\nCwuj2rVrq1b274l33nmHXF1diYjo1atXqpcfEBBAY8eOJSIiV1dXunPnjuoO33zzDT1//pzq1asn\nfyZ5O84sNp2BcnNzERMTg1GjRiElJUX0mXEv28/PD0+fPrXrDGTJw1pP+tq1awgKCoK3tzcuXryo\nmIcpxowZg6pVq0rZVHaPGzduIDg4GIyxcjeq1PQAgLS0NJw9exYpKSnIyMhAQEAAdu7cicePH2PQ\noEHYt2+fYh5Xr16Fn58fGGN466238OjRI9Hnd+7cwZUrVxAaGorQ0FB06NABp0+fVrQ9jCksLEST\nJk2k5BlU1AN4fb+Av38yevRoFBUVqeJx+vRp9O3bVzQGbe09DCU89u/fL/SiLbzPYLOHrcKK7Oxb\nt26hTp06QqCOiYmxq3KWPPjH7sLCwhAeHo4VK1YgPDwcYWFhwiN4xjcUlfIwhY+PD54/fy5lU9k9\ntmzZIhz0Zk6Sino8fPhQyJ7OL7Vr18Zbb72FCRMmiJbLly8r5mF8Cc8YQ61atRAQEACO40yOxZp4\nrl/R38v69evx448/StnUbo/vv/8ejRs3RuPGjXHjxg3cuHEDjx8/xo0bNzBq1ChRO1hImix7ezRo\n0EAod9y4cfjll19UaQ9jfH19QUTw9/eXUrZNHrYKy37Q8Zw5cwa1a9e29Dy1LB4JCQmoVauW6BE8\ng8EAxhgiIiLwxRdfWFNVpD3Onz+Pzz77zNpminkYP/9bQWTxWLZsmRCcu3btauqlCFU8CgsL0b9/\nf0lvt82dOxeFhYWKeJhj9OjRUje128PU69jt2rUz2Rb79+9XzMOYly9fws3NTVQ2/8LP4MGDcfz4\ncXNBW1YPIoKDg0PZqygpSC7bVmHZD7qyhISEYPr06XZVTorH5s2b0ahRIxgMBnzxxRfmbnoo7gG8\nfihfq+DI4+joCMYYFi5cqKmHHcjqsXXrVsyfPx+MMYwaNQpbt27FmTNnVPcw5vjx46re0E1KSoKT\nk5PZk9SwYcOQk5OjuEdZNm/eXO5FGn6pW7cuvL294e7urqjH3bt3rT19ZQ7JZevZwm1Hdo+GDRuS\nk5MTJScna+bh7+9PL168oO+++45CQ0M187CDP7zHpEmTaPPmzfT48WPVPJYsWUKTJk0SbbB3714i\nIurevbtqHqbYsGEDERG99957wrpr167R06dP6erVq/SPf/xDFY8KItlDD9K2I7uHt7c3zZw5k95/\n/31NPWxE9xCjaJBOTEyk//73v5p6VBDdQ4wepE2ge4jRPcToHmJ0DzF/6CCto6Ojo2MjlS4zi46O\njo7Ob+hBWkdHR6cSowdpHR0dnUqMQYUy/vQD/2WQ7MEYIz8/P7py5Qq5uLho5iER3UOM7iFG9xAj\n2UONIK1TQUpKSmju3LnUpUsXatWqFd26dYuaNm2qtZZOJeLw4cN04cIFat68OXXu3FlrHR0F0YN0\nJeSrr76iOXPmkP7kjY45vvnmG/rXv/5FYWFh1LZtW6pWrZpmLhzHUZ06dWjt2rUUGRmpmYfWlJaW\nUmxsLDk5OdGUKVNk+159TNqI8ePHE8dxouWNN96gNWvWqOoxZMgQWrBggaplVnaioqLIwcFBWMaO\nHUsnT57UWovu3btH48ePJ8YYtWnTRpUya9asSatXryYiIl9fX3J0dFSl3LI8e/aM3nnnHapSpQo9\nfPhQ6puHf1gMBgPFxsYK07bK9r2yfpsEsrOz6ZtvvqGLFy+W+2zMmDEUGBiotpLAzp07iYgoNDSU\nfHx8iIjo2LFjtG3bNtXmdj5x4gT169dP1jOxPVy+fFk01HLw4EHVe0sNGjSghw8fitbFxcVR/fr1\nqUOHDqq68OzYsYPOnDlDy5cvF9b5+vqqUnZubi4REXXq1IlmzJhBBoM2F8QpKSl09uxZ0bp9+/ZR\njx49FCvz3r17RET03//+l27cuCH6LCYmhv7f//t/VKVKFcXKN0daWprw79GjR8v75XZMOFKhiUno\n1xRZAQGdsQDgAAAgAElEQVQBCAgIwPjx47F+/Xph+fjjj8EYs5YKqEITk5jysISPj49o4pqnT58K\nE7b06dNHFY/IyMiKzEmrmEd8fDwYYxgyZAg2b94MLy8voS1iY2NV8wAgmqVw06ZN+OCDD8AYQ1xc\nnKoeANCnTx8QEfr06YPt27db2lQRj06dOgkzN0pEEY8xY8aYnNiIiBTdL82aNUObNm1MLvTr7IkH\nDhxQtT2uX78umh1QIpLLVu0UHBQURI6OjpSSklLus3v37gmXrv7+/mopmeXkyZPUuHFjUY+xZ8+e\nipebk5NDSUlJFBwcrHhZ1vjiiy/ok08+oTFjxhBjjK5du0affvop/fLLL1S9enVVHL799lvq2rWr\n8P/S0lIiIsrLyyNA/fH68ePH05kzZygxMVG1oQ1jfvrpJ7py5QoREQ0dOlT18o1hjAk91r///e/U\nrl07OnXqFO3cuZN2795NH374oSLlnjlzhhwcHEx+1rFjRzp58iSdOHGCoqKiFCnfFLt37yYiorZt\n29LXX38tfwF2nFlk66E0b94cjDFER0cjLy9PtjNQRT34nnTZJTAwUBWPqVOn4vUuqRCKtEerVq2w\nbNkylJSU4MCBA2jfvj0YYxg6dKhqHgcPHhT1onlWrVqlSU+aiNC6dWspmyriYdx7NVN3xT3u37+P\nrVu3olatWnB3d0dwcDCePXuGoqIizJw501IvX7H9IhTwa0/63LlzFjeT2+Pdd98FYwwPHjwQ1uXk\n5GDEiBG4c+eO3R62CstSOT5NPb907NgRXbt2RdeuXXH79m27KyfVg4dPAsAfaBs3brS0uawe2dnZ\n8PHxQfv27aWoKubBc//+fZMnLDU9Ro8eLQToQ4cOAQCWLFkiJGdQM0j36dMHrVu3xtKlS9GnTx+k\np6db+xNZPVJTU+Hj4yMEoh49emDevHnmUlUp5sG3vcFgwLhx44T1+fn5wjDUnylIz507F4wxuLi4\nCOuioqJEvxnj4G2Lh63CsjRydHS02YnE+/bti1evXtlVOakexcXF+O6770RlX7p0ydKfyO5x+vRp\nEBGWLFkirMvMzMSECRMwfPhwuLu7Y9asWYp7GJObm4tnz54J7WKhF62IB/+Dj46ORlZWFgAgKChI\n1SCdmJiI1q1bg4iEwGz8b7XaY/fu3SbHf6tXr46wsDBVPPLz84VyGzduXO6z0aNHm/xMifYwWcCv\naaxycnJw4MABfP3118KilMfw4cPBGEOTJk2Edfz+adu27e8/SBtz6tQp3L17V/g/PwRiT+WkePDJ\nRiX2FhXz4IM0nzpr165dCAoKEjLFEBEWLFiguIcxd+/exYgRI+Do6Cgln56sHoGBgeV6ZTt37hRy\nU6qVpglAuYCsRZA+c+YMatasKRyjISEhws14xhg6deqEx48fK+oRHh4unCDLdhjS0tJgMBgsHSey\neeTn52P58uXo2rWr0A78wnEcXF1d8fHHH+O7775T1AP4LSDzN5F//PFH1K5dG4mJicLnf4gg/cMP\nP5S7K7p8+XLFg3RycrLQyA4ODpUiSH/77bfIzs6Gk5OTkJansLAQRKTKQWfMF198AcYYFixYICUp\nrawexpfVPEOGDLGa5V1uj7IsXbpUdJmvpkfLli2FY3TMmDE4d+4c3nzzTWHdmjVrFPUwHmqKiIgQ\n1q9atQozZ86EwWDA559/rmh7ZGVlwdPTUxSYiQh+fn4gIuzZs8ds+8ndHsBvQTo8PBxZWVmIiIjA\n5MmTAQCLFy9GvXr1zKUWk1y2rcKyHvy9evVCs2bNROtOnz4NR0dHuypnzaNhw4ZgjCEqKgoJCQma\nBumbN2/Czc0NH374IV6+fInhw4cDeD0U06FDB0vjj4rtFwcHB3PJgBX3MBWkTd1EVNrDmKVLl8LX\n11dKL1p2j1OnTsHV1VV0xXf27Fk0bdpU+L+Zm5qyefBDTS1bthRd9aq5X3Jzc1G/fn1MmTIFS5Ys\nwaNHj/DkyRPcvXsXRIQnT55YcpC1Pfg2YYzByckJP/zwg3CVMW3aNDg5OSEtLc1uD1uFZT34x44d\nKwy+889RWwiYsnjwZRjDGIO3t7cUZdk8jNm7dy+qVasGX19fIVW8i4uLaJxaDQ/gdSLgunXr4sKF\nC9Y2VcSDH/tkjGH06NHC8TBw4EBVPIhItB8kDnHI7mFMZmYm2rRpIxqT5v/dq1cvxT3Gjx8vOnka\nL97e3jh58qSq7VGuAA2C9N27dxEcHGzyvtrcuXNl8bBVWNZGLiwsxMiRI8tVcuTIkXZVzpKHi4uL\nEKSvX7+OqVOnCgFBIoq0x9KlS4WgMHLkSFy+fFlVj1evXiE+Ph4cx2H16tXWylbMw1wwuHnzpioe\n/M3CPn36YNy4ccIYYwVQ5Ph4+PAhatSoUS5IP3r0SHEP4zFp430SHBwsJYO6YvFDKECDIA28DtTj\nxo0T9sXq1auRnZ1t7sGHCnvYKqxII0tEFg9vb+9yJwUrbyop4iEDsnq4u7uDMYZJkyZp6vH555+X\nCwZaeNjBH9KjVatWwn45dOiQ8NSN2h4mC9AoSPN07doVHTt2lKwrdbFV+Hd/0KWkpIgCdWRkpCYe\nMiCbB/+4XXh4eEUv7WX14Jk7d64QDPjnpLXwsBHdQ2WPU6dOWeu9quIhEcll69nCbecP5ZGenk6d\nOnWiLl260JIlS6hq1aqaeMiA7iFG9xDzu/PQg7Tt6B5idA8xuocY3UNMpQrSOjo6Ojo2ok/6r6Oj\no1OJ0YO0jo6OTiVGzxZuO7qHGN1DjO4hRvcQI9lD70nr6OjoVGL0IG0FFxcXWrZsmdYaOjo6f1Iq\nRZB++fIl/e///i8FBwdTXl6eZh6DBg0iZ2dnqlKlCnl6etLt27fpxYsX9N///lczJy25e/cudezY\nkTiOI8YY/fWvf1W8zMGDB9P06dPpwoUL5Rad16SmplKvXr0oIiKCvLy8yMPDQ2slIiKaPXs2McZE\nS0JCgiJlXbx4kd544w0hjdfBgwcVKUcKGzduJH9/f+I4jqpUqSIsHMeRv78/zZw5074C7HgDR7Y3\ndXJzc9GiRQswxvDw4UPZ3tSpiMepU6fAcZxoSUlJgYeHh7lpOhVrj9LSUmRkZKBdu3bw9vYWzTim\npkd0dLTQFowxazPiyeLBzyPOl2f87+bNm+Pq1avWtBVrj86dOwvzZXh7e2PatGmypEeS6vH06VPs\n2rVLmJDs6NGjuHnzJnx9fS39mWLtwTNr1qxyU4caL0p4rFq1CkSEmjVrYvTo0XB1dZWqa5dHRkaG\naDbKo0ePwtXVVfQbKfubMfO7kVy2rcKy7uwrV66AMYaYmBgpm8vuwf/w+EnTx48fj759+8LT09OS\nk6wely5dQv/+/eHo6IiAgAB89NFHICJER0dbaQ5l9ounpyd2794NAFizZg06dOiA3r17q+5hDMdx\nGDx4sLXNZPW4d+8eBgwYAI7j4O7ujvj4eOTm5mLcuHHgOM7ShFyyenAch0aNGiEuLg43btzAggUL\n4OTkhA0bNqBevXqqtYfJAogQFhaGWbNm4fjx46p48EF60qRJmDNnTkVyg8reHpmZmVi/fj2WL1+O\nCxcuiIJ03bp1zU1XKrlsW4Vl3dlaB2n+jPfixQsAwJEjRyydAWX3OHbsGJycnEBEGDFiBAoKCl4X\nQISZM2dacpDVg2fu3Lmig37evHkgIowaNUpVj7IwxrBr1y5rm8nq0axZM3AcBw8PD2FGwuTkZPj5\n+YHjOHMTusvqcfHiRfTv37/c+suXLyM8PBwNGzY05yCrhzEWgrEqHpUpSPMsXboUbm5uoqlkV65c\nabeHrcKyVo7PE6ZVkJ40aRI4jkPbtm2RnJyMOnXqgOM4+Pj4qOKxY8cODBw4EHv27EFJSclvBZhP\nmaWIB0/Z1GVdunQBYwwrVqxQ1cOYuXPnYsaMGVI2lc3j+fPn4DgOXl5eormSo6KiVD2JX7t2DcHB\nwUKmHh4+Ue/69etV8eAJCwvTPKv9unXrQEQIDw9HkyZNQER49uyZ6h4A8OTJE6xdu1Y0vNGsWTNr\nU/1KLttWYVl/hEOGDAFjTMp4Y4UqVxGPZ8+eCQ3s7e2N5ORkTTxEBRBZ6qkp6tGrVy/RQde8eXNN\nPABIGQ9XxGPr1q2oU6eOkLl+/fr18Pb2Rt26daW0iaztcfnyZbRt2xZEhAcPHmDGjBlS5taW3YMf\nZ7aS/FZxDwAoKirChQsXkJmZiS+//BJhYWF4+fKl6h6RkZGiMeiMjAxrDhXysFVY1h8hnwzWwk0Y\nmypXUQ/jJJ9aevC0aNFC06kXBw0aJPSqzeTPU9xj2bJl8Pf3t5b1QxGPvn37IiQkBBs3bsTGjRvh\n4eEBFxcXxMTEgOM4jBkzRhUPY9auXVv2hpw1ZPUwviFoZQxaUY+yvHjxAkSEw4cPq+6RmZmJ5cuX\nCzfa3dzckJCQIJuHrcKyNrKWuQUBICkpCXXq1IGPjw+Sk5PBcZy5rDCKehgTFxcnNTAp6rFr1y54\nenpq5nHixAk0b95cyKTD3+RVwyMjIwP+/v7C1UR+fj4ACJezVpC9PUpKSjB8+HBs2LABYWFhQmZ5\ntT0A8RMdEgO1oscpAISGhiIkJMRab1pxj+7duws9azk8bBWWtXJaB2lPT09wHCfk8+M4Dm+++aa1\nDNmK7uwBAwZYe/ROcY+rV6/CxcVFSu9EUQ/g9bjsihUr0KtXL9XGgk2Rl5cHxpiUjOGye8THxws9\n6Ly8PPj6+koJkIq1x/HjxysyPq3ofgFe31siImRmZmrq8ejRI/j4+MgWpNWYu6NSM2zYMMrKyiLG\nGDVr1kxYn5eXR4WFhZp5PXjwgPz8/DQrn+j1yywFBQXUuXNnTT2IiIKDgyk4OJgaNGhAe/bs0czj\nyJEjFBgYSO+//76q5T558oQ++ugj4f/Vq1enJ0+e0PTp02nv3r3k7u6uqg8RUVhYmGIvq9hDSkoK\n1a5dW7Pyq1SpQi4uLvJ9oR1nFtnOQFr2pDmOQ9WqVYX8hpcvX0ZAQIC1s7HsHsbk5ubCyclJyqaK\nehCR1KEORT141qxZAyLS5JEzAHjw4AEYY9i3b5+UzWX14IcXFi9eLKx7+PAhqlatijlz5qjiUXYM\nmozGpiWg+PHB96Tj4uIU94iPjwdjDPfv3xfWLV26FMuWLRPimZUhMcllV4rXwrWmWrVqVLVqVVqx\nYgW1aNGCQkNDNT0T79+/X9NePBEJVxf//Oc/NfUwZt26dcQYo969e2tS/qZNm4iIqHv37pqUT0TU\nokUL4d+ZmZlUXFysSrmzZ8+mOXPmUHh4uPDKt/FnWnDkyBH6y1/+QmvXrlW97NGjRxNjjL755hva\nuXMnde/enSZMmEATJkwQ2mfs2LHyFGbHmUW2MyFjDC1atJCyaYXOQFI8Jk2aJDw606RJE+FxK7U9\njOnZs6dqr7maYteuXWCMVSQTtCIexgwaNMjaG4+Ke3Achxo1akhVltUjPT0dHh4eaNSoEZYsWYLJ\nkyfD2dkZ06dPV8WD78nPmjVLeLuwgsi+X37++We8fPkSrVu3Fm4ou7u7WztuZfHIyMiAm5ubcFO5\natWq8PPzs/bMuk0etgrLevAzxjBhwgTZK1dRjwqimEdwcDACAgI08cjPz0fDhg01f1mBZ9myZZg2\nbVpFThqye9y6dQutWrUCx3GIj4+X4qCIx+HDh1GzZk1hmOH69euaeNiIYh4lJSVYtGgRiAhLly5V\nzSMhIQEcx2HcuHEVCc4V9rBV+A+5syuLBxFJeQRQEY8ff/wRHMehY8eOUstXxMPYZ9SoUVKfMFHE\nY8eOHcKPsQL84Y9T3UMdDz1buO3oHmJ0DzG6hxjdQ4yeLVxHR0fnj4D+dIeOjo5OJUYP0jo6OjqV\nGD1I6+jo6FRi1Hgt/E8/8F8G3UOM7iFG9xDzp/fQe9I6Ojo6lRhNg3R+fj6dOXOGYmNjKTo6mjZv\n3kylpaVaKuno6OjYTVFREfXs2ZMCAgLs/i7NgvT//d//0T//+U86fPgwDR06lGJjY+mTTz6hv//9\n75r43LhxgwIDA4kxRhzHiZYmTZpo4qRTubh37x7NnDmT/P39hfkZioqKaMmSJdS6dWt6+fKlYmU/\nffqU3nnnHfLy8hKOyzfffFNYnj59qljZlZFLly6VW+fi4lJuXhEtePXqFQ0ePJj2799Pzs7O9n+h\nHW/gyP6mTkhICJycnMrlcrP1TR2pHrm5uWjVqpVoUvmyi5IepaWlWLVqlWiZOXMmVq5ciVWrVlnL\nWKP4frly5QoGDhyo6jzOSUlJiI2Nhbu7O4gI8+bNE32el5eHqVOnIjExUVEPnl9++QVBQUEmjw3G\nGBwdHYWkAEp47Ny502zZjDH06tULv/zyi6k/VfT4uHr1Kjw9PVWbnfDp06fYvHkzPD09y03uT0ZZ\nY5T2sMT06dOF/ZKammq3h63CslcuPz8fXl5eUuZnkNWj7MHOB+mGDRti/fr1igdpfoIWU4txjsGA\ngABz85sotl/Cw8Ph5+cHIkLnzp2Rl5cnfLZ582ZFPM6fP49+/foJ++D27dtISkrCqlWrRNvx+6Wo\nqEjR9rh//z4aNmwIxhg+/fRT3LlzB4WFhRgxYoTgUK1aNVN/KptHbm6uycDctm1b0f/d3NwU9eB5\n/Pgxpk2bJgTmkydP4s6dO/D09LQ04ZMsHj/88IMQiMvu+8oSpF1dXS3tjwp72Cosa+WysrIQHh6u\nerqozMxM0UE+ffp0bNy4EYwxbNmyRdUgPWvWLMTFxZVbvLy8RAFbyfYwpqioCESENm3agIhw6tQp\n0edKBenevXuDMYYnT57g+fPnJt34uXzd3d1NfSxre4wePRqMMXh7ewvrvvvuO9Fxs3fvXkU9bt26\nVa7XPnXqVGRnZ+Prr79GzZo1VbniA14HaD7v5YULF1BQUICrV69i2rRp4DgOBQUF5v5UFo/fQ5Bm\njMHZ2dlankPJZdsqLFvlHjx4gJCQEBARxo0bh7i4OGRnZ8tSOWseISEhYIzBYDDA09MTL168AABc\nv34dr169wrNnzxQ/+IcOHSpMk2puWCMtLU3VIP3ixQtMmzYNzZo1w8OHD0FEuHfvnmib/fv3K+Lh\n6OhoNgHE48ePER4eDmdnZ7i4uJhykL09+CAdGBgorBs2bJhwXPTo0aPsMIfsHgMHDhQF6bLThCYm\nJiI4OFiVIM1fQQwaNEhYxydjUCP3pHGQLrv/K0OQHjduHBhj6Ny5s7VNJZdtq7AslTt//jyio6Ox\nefNm5OTkICcnB6tXr0bVqlVlqZwljx49eggH/ezZs00WpEZPmqddu3Zo166dyUA9fvx4MMYQExOj\nuMeBAweEA33Hjh2YOHEiIiIihM+Tk5OxY8cOxTyys7PRunVr1KtXD8HBwcLi7OwsClTPnj0zVwXZ\nj9NDhw6BMYbo6GhcuHBBcKhZs6alP5PFo2fPnmCMwdXVFeHh4eVOljxRUVGCo5LtQUQYNWqUaB0/\nTGgli48sHh06dBAFY29vbzg5OYnWaRWka9euLQyLSUBy2bYKy1q5skRFRclSOUsefJCePn260IMu\ni5pB2hxXrlxBYGAgwsLCzAUmWT1evXqFoUOHig74OnXqYNSoURg1ahQcHBzM/Qhk88jKykJoaKjQ\n9mFhYUKwKturVbo9gNdXFvwQi3GA/s9//qO4R9OmTcEYQ7du3SyVJfi9+eabinjwMMbg5eWFLl26\nYM2aNejSpYswHLdixQpLirJ4uLi4lAvIphalPUx+8a9lZ2RkSNpc6mKrsKyVK8uePXss3UCUxcNC\n8JW6neLt8cUXX8DJyUmz7NizZ882e8AfOnRINQ8exhjq16+PJ0+eWNpMEY/8/HzhWCjbk1TSgw/S\nX375pdUCp06dqniQBl5nbh81ahTi4+NRUFAAjuMwePBga3qyeGRlZSEuLg4+Pj7CEhsbi88++0wI\nkn5+fop7GFNSUiIMcwwfPtxaO1TYw1ZhRX6EPOvWrZOlcpY8fg9BulGjRuA4Dr6+vpY2U8zD0dER\nLVu2NPnZjRs3VPMAXp+4JfTWFPEoKCjA2rVrhR60mWEnRTyk9qT5QKFGkDZm0KBBUhNEKP570aon\nfffuXeEK48yZM5J1pS6V8rVwpZOfpqSkWN2Gf0mB6PWLN1pw9epVIiJ69913NSm/qKjI7MtF9erV\nU9VlyZIl1KBBAxowYICq5RIRLViwgEaOHEk9e/akM2fOUEZGhuoON27coJ9//tns5/fv36cVK1ao\naPSa1NTUSpWsWAv4eNKpUydq1aqV/AXYcWax+Qx0/PhxREZGlsvRFhERASJC06ZNZTkDWfKw1JNe\nuXIlnJychBs2R48eVczDFI8fP0bLli3BGJNyZlbEY+nSpfj888+l6CrqAQBbtmyRdNWjhIebm5vw\nFAfwet9IuHMvm0d2djZ69eolHK/BwcEoLCwsV5iXlxcYY3j69Kmi7WGMhSeOTKGYh1CABj1pf39/\nyVfltnrYKmxX5Z4/fw4iQkBAAL766ivMmjULMTExICKEhITg4cOHslTOkofxQZ+UlISsrCxkZWXh\n+++/R40aNYRnHY8fP66ohym++OILcByHnj17WnruVDGPy5cvw9fX19wbbKp58Li7u2sSpPlxaF9f\nX+ExuwEDBmDy5Mmqevzyyy+im5aDBw/GDz/8IHx+/Phx1Yflvv32WxCRpol5jXn06JEmQZo/Nhlj\nePz4sVTdCnnYKmx35a5evYqZM2fCzc0NUVFRiI+Px6tXr2StnCUP40fwjBf69XnPuLg4VTxMwbus\nXLkS+/btU9UjIyMD1atXx/jx46WoKuZhjJQxWSU8Jk+ejG7dumH58uVYvny5xWe4lfQAXveoT506\nhYkTJ5o8bjdt2qSKBwDMnTu30mST58nIyBACdIsWLVTz8PPz+2P2pO1EFo9r164Jl7Jlg3RiYiKK\ni4tV8TCF8Wvhag93TJ8+HX379sWjR4+kqCrmYQxjDEeOHFHdY/LkyeWOjwpkDFesPT777DN07NhR\ncIqMjFTNY9euXXBxcbH2TLTiHiYL+DVIb9y4UVMPqbpSF1uFfxeV+7168EH6vffeU324w9PTE8eO\nHZOiqaiHMRXsqcjmYfzGqbu7O1JSUlBSUqK6h53I6sG/cdilSxdNPezgd+ehRrbwP31mhTLoHmJ0\nDzG6h5g/vYcaQVpHR0dHx0Yq5XPSOjo6Ojqv0YO0jo6OTiVGD9I6Ojo6lRg1grSkWasqsOgeuofu\nYbSEh4dXCg8ri+5ho4fBDmkdHR2NYRonXdVRHn24oxITFBREjDHy9PTUWkWnEhIeHq61QqWgb9++\nxBij8ePHa62iCJoF6TfeeIMaNmxIxcXF5T7btm0b9e3bVwOr8tSqVYv69eunaplHjhwhFxcXunnz\nJrVt25aePHmiavk8n332GfXs2ZMcHByERW2mT59OjDFq2bKlaH1hYSF169aNtmzZorpTZSA8PJwS\nEhIoLCyMKstjtNnZ2aqVNX78eGKMEWOMli5dSgBo+fLlmgXrkydPUpUqVahq1apUtWpV4d+y/Hbt\neAPH5jd1+MlZiMonkwSA3bt3w83NzdxE4qq8MVRSUoKBAwfC29sbly5dUtWDz1cXGhqKFy9eCDOw\nmUERj9OnTwsTTRkMBmFZtmwZXr58qZqHv78/atasieTkZNH6goICNG3aFF988YUqHtYoKipCWloa\n0tLSFPc4fvy48PuRgKLtkZ6ejq1btwJ4PdWCGh7p6ekgIrRu3RqJiYnCel9fX6Fdli5dqrgHz4kT\nJ9C4cWNwHAcHBwc4ODgI/05KSrLbw1ZhmyvHp2ayNnG6hYNQsYNu6NChyMrKwvPnzxEcHAwiQk5O\njllFJTzmzZsHxsTZqa0gu8fYsWOFoFw2SBsMBgQHB6vikZSUBFdXV5NZej755BPVjw9jzp07J8zt\nQUTC6+NG+SAV86hAgFbUA4CQTT4+Pl6VIM0HYlOkp6cLQXvcuHGqHB9EJEzjYOrfFjLqSC7bVmGb\nK+fu7g4iwoULF8zJC5VX80eYnJwMNzc3pKeno1u3biAiDB061KKi3B779u0TZlobO3asNWVFPGJj\nY03OCmi8+Pj4KO7x5MkTeHt7m/1BhoaGqh6kExISsGfPHtFd+sjISERGRmLKlCk4f/684h5hYWEg\nIktT6JZFsfY4fvw4nJycUKtWLVy4cEGVIE1E6NOnjxQ9LF26FK1bt1bEg2f06NEIDw8X9Z779u2L\n3r17C73qXbt2mfpTyWXbKmxT5X788UfJvQA1g3ROTg6GDBkiTBrj5uYGNzc3S71oRTz4ZKLt27cX\n5i+WgKweZXvOpnrSBoNBcY9+/fqBiFCjRg3TlVb5JJ6TkwMHBwfhxBUZGYns7GxLfyK7Bz/MERYW\nZk1XUQ+et99+W+hFAzA5dCm3RwWuIITtjYdE5PLguXPnDsLCwoQgHR4ejqysLPTq1UsI0nv27DGp\nJnWxVdimynl7e8PDwwOHDx82V2eBVq1agYhw+/ZtmytnzsOYNWvWgOM4uLu7A3jdm+3Zs6eU2c5k\n9TDOTJ6YmIh169ahbdu2aNu2LXr27KmKx8WLF4Wg7O7ujs8//xyMMUyaNEnYhg/SSiaiffXqFYgI\ngYGBJgPhggUL4OTkVG6cWm4P4PXYPB+YFy9eLGUKW0U8Zs2aZfXZWzO9a1k9jKlfvz6IyNSxoJiH\n1F40j4netKztER8fLwxxGPPjjz8K65s3b27qTyWXbauwTZUjIquTtz948ABJSUlo3LgxiAhBQUE2\nV86chzH8JXW3bt0wdepUVKtWTRjsNxMEFPFo166daH7gskMMFrLVyOJx//59NG7cWAjS27dvB/D6\nJq/xdKl8kDYxTixbewwePBhubm64e/euyQrHxMSgUaNGirYHz/Xr1+Hj4wMigo+PD7Zt22ZuU0U9\nygbkWbNmCQs/BDJr1izFPUw5nTt3TtLmcnhYScpcju3bt5ftfcvaHsY3C42ZNm2asH7+/Pmm/lRy\n2S8CNVkAACAASURBVLYK21S56OjoCr+ZY2LsWtZG3rx5MzZv3ozZs2eDiODt7Y2IiAhERERgyJAh\n5v5Mdo86deoIAdnV1RXR0dEAgEuXLlkaB5bNo1atWqLhjYkTJ5osTOmedOvWrUFEKC0tFdbduHED\ncXFxqFq1qtCrzcvLU7Q9TMGnW5OIrB6Whgn5XraaQfrEiRMWh6OU8jAxfGERJYP03bt3hSAdGBgo\n+uzLL78UgvSgQYNM/bnksm0VtquRLS0dOnTAjBkzUKdOHVXHHFNSUuDm5oYrV65Y21QRD+MgbZxp\n4/vvvwdjDBMmTFDMIyEhQQi+Pj4+MBgMcHJyEo2L5+TkIC0tTfjMRDICWdqDD9IFBQXYunUrNm3a\nBGdn53LHiQUUOT6A10Ha2lNJSnmYq7eEx/EUaQ8++IWHhwvr7t27Z+lPZIsfFelNm3jKQ7b2uHPn\nDogIXl5e5cad+WGQRo0a2R2kVX8tPC8vjz777DO6ffs2ERG1bduWsrKyaPjw4UREVLVqVXJ0dKTv\nvvuOMjIyVHECQLt376aZM2fSX//6V1XKNEerVq3onXfeISKiW7du0dChQ4mIyr3MIScvXrwQ/n34\n8GFav349ERGlp6dTw4YNiYgoPj5eeElgzJgx9D//8z+K+RAR1a5dm/Ly8oT/BwQEUHp6OpWWlqry\notPUqVOpUaNGNGjQINH6wMBAxcuWyuzZs2nOnDlE9PoYVhP+JQ3jt2Hz8/MVL7dPnz60c+dOydt/\n/fXX1Lp1a0VcFi5cSBzHUa9evahnz56iz4YMGUJVqlSh+Ph4Cg0Nta8gO84ssvVQTMHfOLTnDCTF\n48WLF4iOjoabm1tFFWX1GDVqlNCTvnTpEs6ePSv8v27duop6HDx4EIwxfPzxx+W+/PHjx6KxcTXa\nIzMzEwsXLkRGRgZyc3OF9QaDAUSEzMxMxT369OkDIkJwcLBwVTd16lRL5SriIXxZxW4WKubB06BB\nA6xZs8baZop48C+zWMs3mZiYaKrXLWt7mBruaNGixe/7ZRapdOvWzVzWX1k9YmNjQUTm3m60hKwe\nly9fRu3atcvdMDQYDOYe4ZHN4+DBgzAYDAgNDS335T169IDBYEBERIS1DOqKHx9EhPbt25t741FW\njzt37mDIkCHCOPT48eOtPZKpiAeP8bAGmR9/VtwDAM6cOQM3N7eyz4Wr6sEPYxCRcJObJz09XXjp\nJT09XTEPPmu6pZdZ5GiPP32OwzfeeIPeffddWr58uaYexsTExJCjoyO1bNmShg0bporHsGHDKD4+\nnkpKSqhKlSrC+pKSEiotLZXyfYrnjmOM0YABA2jr1q2aekjkD+sRGBhI6enpVFJSoqkHz7Jly+jM\nmTOidTt27FDF48mTJzRhwgTaunWr6LdTUlJCHTt2pGPHjtnt8aefqrS0tJQKCwu11hCxadMm1cuc\nOnUqxcfHi9bNmTOHmjVrprqLORhjmt8z0HlN7969tVYQ0HL2O3d3d1qyZAm9ePGCdu3aRUREZ8+e\nFT6Tgz99T7pZs2b017/+lf79739r6mEHuocY3UOM7iHmd+fxpw/SdqB7iNE9xOgeYnQPMZUqSOvo\n6Ojo2IiemUVHR0enEqMHaR0dHZ1KjJ4tXPewuBQXF4MxhuXLl+vtoXvoHhp4VIqedEJCgp5Us5Ky\nZ88eIZecjo6O+lSKIN2tWzc6ceIEnThxQmuVSoGbmxsdOnRIaw0iImGejAsXLmhsosOzd+9e2rt3\nLyUmJmqtUqn4wz5Hb8drkna/Xnrw4EG4ublJmRfCptcppXrYiGIeTZs2rchMX4q2x1tvvYU333wT\nT548UdWDMSa8YstxHBhjOHbsmBRlxdrj8OHD8Pb2Fr22P3LkSBQWFirmkZqaihMnTghTtPKLh4cH\niAhOTk44evSoJu3BY5wTU0sPIhLaSUsPqbpSF1uF7a7cwYMHUatWLamT99hUOSkedqCYR9OmTa0d\naKp4nD17FrVr1xbSI6nlkZGRYTJIt2jRwtI80rJ7lMXJyanc3CqMMaxevVoxj4YNGwqBuVOnTli6\ndCmWLl2K//znP8IcEWbmuFG8PXj4ucj57EZaefDHjFoTgVni9u3bmDlzJjp27Ihnz57Z5WGrsN2V\nGzdunDDzGj+Zu0QUaeSCggJhLuOyi5k5hBXb2QkJCZUiSDPG8NZbb2niYTyX9blz54QfYEpKiqoe\nxpgK0O3atVPUY9q0aVi3bp3pAn49PrUM0l27dhV60VZmpVPUA9AuSBcVFSEtLQ0tW7YUyjfuXBw4\ncMAuD1uF7a7cixcvcOPGDRQWFsLT01OzIH3v3j3s27cP7dq1Ew76+vXro379+kLaJLWDdFxcnOZB\n+tSpU2CM4ciRI8K6rKws3Lx5U1UP4LcUZ3y2Giso4vHVV18JgTkwMFC1IG2OQ4cOCcfryZMnLW2q\nmMft27fRokULMMbQoEEDa5sr2h58kmvGGPz9/VXzKC0txYIFC8pd9Rn/28xvRnLZtgrL/iPUIkgX\nFBTA3d1d1Gt2cnISPtuwYQOIzM4jrFh7fPXVV5oG6YKCAkRERAgZsRMSEjB48GA0bdoUPj4+qiSA\nBV6fyC9evIhq1aqBMYY1a9bgxYsXqrcHANFY9LVr19C9e3fNgnRxcTFiYmJA9DqTkZnLacU9Vq1a\nJfSily9fbm1zxTyA11ccfHC0Mu2wbB4JCQnw8vISBejo6GhcvHhRWD927Fi7PWwVlrWR+Z6SRGTx\nePHiBcaNG4evvvoKp06dEmUlT0hIEIL3tWvXFPUwxZw5czQL0pmZmaIrm6FDh4Ixhg4dOmDy5Mmo\nVq0aPvjgA8U9Tpw4gVq1aoHjOBw/fhyNGzcGx3Fo3769tUAtq0fDhg2F4Pzq1SvRZ6GhoYqPSZv8\n4l87FHPnzpW0uVIefIC2Ms+44h67du0S9hHHcapcWXz77beoVasW4uPjcejQIdy5c0f4jPfgO3z2\netgqLGsja9WTNkd4eDiICAMHDtTEQ8sgfejQITDG0K1bN+zYsQOurq6YPHmykNPQ0dFRlSB94MAB\nNG7cGIGBgSgpKcH27duF3oqVCe9l9eAvWadMmVLuMz5JsJpBOjs7G0SENm3alDtpmEGx45QP0uay\nuqvl0bx5c+HE5enpqZrHiRMnyq3jMxk1adLE2g13yWXbKixrI5t6uuP58+d2V66iHgBw7NgxuLm5\n4dSpU9Y2VcxDy6c7+GzlfLB+7733hM9u3bqFVatWqeJhCuPLSgvI5tGmTRt4eHjg8OHDJgu6cuWK\nqkGazwSydu1aS5sp7gH89tjdhx9+qKkHAOGxu+bNm2vq8emnn6J69erIysqS1cNWYdkbmQ/S/DDE\nRx99ZHflKupx6tQpODg44IcffpCyuWIeWgfpBg0aYMuWLfD29sbVq1cBALt370bDhg0t9Q4Uaw8e\nDw8P1YJ0fn4+WrRogU6dOpkt6PTp02jSpAnu37+vmAfP/v374eLiAgcHBwtVN4ns+yUhIUF47O7Q\noUOaeQDA1atXhasdC0OTinsAQPXq1a0dmzZ52Cosa+X4IF1YWIj169eDMYauXbvaXbmKejg4OFTk\nR6CYBz8eroUHH6QXLVqEwMBArF69GuPGjRMek8zOzlbFwxRffvmlakE6Li7O4g//3Llz8PHxUe3G\nYUBAAIgItWvXtlB15T0AYPTo0TAYDGjWrJlwEtfCIz8/X7hnoPXTUMDrKz1vb2/ZPWwVlqVy9+7d\nQ0hISLlnT93d3S3dHJLd49y5c2jXrh2ioqIsbaa4h/DFROjcubMmHvyNQuOlXr16uHnzJkpKSlTz\n2LZtm2ioBXgdqBhjcHFxUdzDXJB++fKl4GHlBylre6SkpAjjrosXL8aBAwewfPly4UmbnJwcLF68\nWFiU8gAgPOpWQWT1ePz4MYKDg4VjVMuTxb59+8BxHCZOnCjVoUIetgrLUrk5c+aUCwj9+vWz9gqy\n7B79+/cHEVlKv66Kh/DFRIiNjdXE4+effxbtj4CAABQXF6vuMXHiRNSqVQv79u3Do0ePkJiYCI7j\n4O7ubu1+gSweubm5aNasGXx9fREQEICDBw/i9OnT6NOnDxhj8PHxwenTp1Vrj/z8fAwfPlwI1PzV\nVt26ddGuXTs0a9YMRAQ3Nze4ubkp5vHpp58KWewriKwe/MtwHMehd+/emnkMGzZMeNyOv7kut4et\nwnZXzg5k9WjUqBG8vb1x9uxZTT3KOl2+fFlzjwoiq8fGjRvLvSCwdu1aVZ+TvnXrFmrUqCE6aTk4\nOCA3N9d6ayi4X/bt21dusfC6vKweXbt2rRRBevr06fD09MSKFSs08ygtLQVjDEFBQRV1qJCHrcJ/\niGDw9OlTEBFmzpypqYed/CE98vLyhGejW7VqhUmTJqGoqEh1jwMHDiAqKgqMMWzduhXbt2+X4iC7\nhx3I6lFZgrQdyOYRGxsLb2/vitw8tcnjT52INjk5mTZv3kyLFi0iR0dHzTzsRPcQo3uI0T3EyOKR\nkZFB7du3p1WrVlFUVJSiHn/aID1y5Ejav38/PXjwQFMPGdA9xOgeYnQPMb87Dz1buI6Ojk4lplJk\nZtHR0dHRMY0epHV0dHQqMXqQ1tHR0anEqBGkNUuFrnvI48EYw+zZszX3kLjoHrrHH8pD70lXIgoL\nC2n//v305ptv0jvvvEMFBQVaK+no6GiM5kE6NzeXWrVqRRzHkbe3N/3rX/+i4uJirbVUZ8mSJfT+\n++9TdHQ0/fTTT7R9+3Zq3749XbhwQVOvhIQETcvX0TFHYWEhffPNN8QYI8YY3bhxg0pLS7XWkh87\n3sCx+42hiRMnguM4bN26FUVFRcjLy0OLFi1UndTdmLCwMJPJRs3MFSGrR+PGjctlyOaXtm3batIe\nx48fR1hYGI4fPy5l8z/cG2XG8K8AM8YQHR2NdevWwcHBAYWFhfjwww9NvSpul0dhYSEKCwvx8ccf\nl7tU3rBhA16+fKlpe9iArB78W6CMMcTHx6N+/fpgjCEqKsrSnOeye0jBy8sLjDFs3rzZJg9bhe2u\n3OXLl4WgZMzQoUORnp5uqc6KNHJGRoYoMLu6ugpBW+0g7evri1OnTiEyMlLVSe7LffGvQUEidnnw\ns5qNHDnS5MK79OrVCyNHjlTMwxxFRUXw8/PDrl27hHVVq1ZFp06d4OrqKrsHX1/GGKpXr45q1aqJ\nAnXdunWtKSvaHsDr16L9/PxEXmbyX8ruwf9OhwwZgmHDhpWbX8UCirTHq1evcOnSpXLr4+Pj4eTk\nBC8vLyQmJtrkYauw3ZVbvHgxiAheXl6i9YWFhcKkR2aQ1ePRo0f4/vvvRTu5adOmuHz5Mh4+fKha\nkD537hwSEhKQk5MjWp+cnAyO4zBu3DhV2oNn1qxZICKpvWjZPNasWSMsxgHRmPPnz///9s4+pqnr\n/+PnHA2wABWqVIxIwIgpm4oaEQi6oEumZuimUZSpQzd1JjNumgkuUdii00VCHGPTMHFGyTCiMtFM\nt0UDY4luzKnRTB4UkCkgKD5UoPj4/v3h755vn1vae287d17JTWh76X2d29t3zz09vR8UFhYq6mFJ\nUFAQGhsbATy/UmBSUhK/8JOd6iSKeDx8+BBDhgxR7cPTkr179yIgIAABAQGglCI5ORm5ubk4c+YM\nf/+EhYUp6rFz505QSpGfn4+nT5+CUorvv/8ewPPOVmJiIvz9/VFZWan4/gAAo9GIGTNmICAgwKqU\nGGMMoaGhtt5HLm/bXWGPGyeVHmKM8Yb19PRg2bJlzi6eLavHpk2brIY3JDZs2KBaSNujq6sLo0eP\nhl6vt3eBIdk9KioqrAK6oqICOTk5jkJblf0BPK8MrXZIv/rqq0hMTAQAJCQk8GNl1KhRMBqNqnj0\n9vZizZo1IIRgzJgxzpQV8Xj//fd522/fvs2vSvj48WP8/vvvoJQiPT1dMY9nz55h9uzZoJTyM+6h\nQ4eitraWr3PixAlQSqHX6xXfHwaDAXPmzAFjDDt37jR77NGjR2CM4eTJkx55uCssy0EXFRUFxhgi\nIiIwbdo03jthjPE3hCeNc8Wjrq4OI0eOBKUUgYGBZpc+9IWQBsCr1ajVM0hJSTHrqUm9atNFDQ97\nEEJUD+muri4wxpCRkcF70C+99JKqxSk2btzI939ERITN02ulPUJCQqw6Mx0dHZgyZQofe922bZti\nHleuXAGlFBMmTHB0eVabdVOV2B9SQPv7+1t9L/HVV19h+vTpyMzM9MjDXWFZDrqlS5cCAKqqqlBa\nWor29nZ+uu9gzE1WD9Oe9MWLF/n91dXVoJTCz8/P3sXdFQkDu40mRJWQlgJZ+vJWup2SkmJ2W2kP\neyxatMjZqb5iHtJxEhMTY++1UNyjpqYGVVVV2LZtGw/sadOmqeJRV1eHefPmobW1Fbdu3cKxY8dQ\nW1tr8yxUKQ8ppO0NhUmsW7cOlFJcvXpVEY8HDx5g9uzZiIqKwv79+622P2bMGDDGkJqaau+D3OVt\nuyssSxh0dXXZe1i14Y758+fzun6mSCHtoCSO7G/Ce/fuobS0lC+mXzRQSlFQUKC4h/TGl4Y1UlJS\neEAD/xsKUWN/WHL58mUEBQWpEgaWNDc3IzAwEFqtFps2bXJFV/H9sW3bNgQEBIAQgqamJsU9tm/f\nDq1Wa3X6LgV0dHS0I11ZPHJyckApxYULFxxtC+fPn1c0pOfOnQvGGKqqqqy2fejQIWg0GjDGHBWI\ncHnb7gp7fNAxxixrsZkRGhqK1tZWjxrnzCMrK4uXQrKc0iSF9LfffmtPUTaP4uJifgptawqe9Ngn\nn3yiqAcAs4C21Wu2HApRysMWOp0OjDGUlZU5Wk1Wj4aGBnz44YcYN24cAKC7uxuUUnvjjIp5OOKP\nP/4AIQTz589X3MN0PN500Wg0zs4uZPEoLCwEpRRFRUWOtqV4SNvaB6bLsGHDsGLFCln2h7vCHr/Y\njDGHL2pYWBju3LnjUeOceaxduxaUUps9I2lajwNk89Dr9S6F9IQJExT1sAxly9tSL9rOPHZZjw+b\nGyAEOp3O6WpyepSUlIBSala9h1LqaLaNIh7O8PPzU+XDs7W1Fdu3b0dgYCBCQ0N5rUEXyljJ4iEN\ndyxcuNDhxuLi4kAptZUhsngcPXrU5rJixQoMGjTI0ZlNnz3cFfb4xXbWk9ZoNPj77789apwjD4PB\ngIULF2LNmjVWveimpibodDpVQrq0tJR/+k6ePBnffPMNf2zr1q38w0KaM6uUB2C75yyFstSDdvBD\nI1mPD0s6OjpUDQOJIUOGwN/f3+xLKl8M6aSkJFXPcKqqqnD27FksWbIEU6dORXt7u7N/kcWjqakJ\ngYGBeOuttxwWSFbri0NTfvzxR+j1euTl5bmyusvbdlfY48ZJMyq0Wi0fWG9ubuZT8OwEdJ8a58jj\n119/5XNcr1y5wpeVK1fyb+4//fRTew6yeZSWlpr1lqWJ735+fmY96erqapw9e1YxD/5kNi4G42Tq\nnSIeluh0OixevNiVVWXz6O7uxqRJk6yqQEtfeKvl4Qrbt29XfRiKUoqgoCBXFWXzmDt3rt1ZV7/9\n9hvPlp9//llRD1OMRiMYY/jiiy+crdpnD3eFPW7cwYMH+el9cnIyfvjhBwwdOpRPyZOjcY48Ghoa\nEB8fb3dMKSEhwZGDbB4VFRUICAhwONyh1WoV9+BPZhHQpl8aqrE/bFFYWAhCCC5fvqyqx8GDB82+\nB+jq6sIHH3zg7LIFsnu4gtohLc1F3rFjh6uKsnmUl5eDUorw8HCzXzjm5OTAz88PlFLs3bsX9+7d\nU9TDlKysLKxYsQK9vb3OVu2zh9drHN6/f58sWrSIREVFkcjISJKRkUG0Wi3p37+/vX+RrUbZgQMH\nSHp6utWKLl6k5V9XK82Cf43Hyy+/TGpqaoiLx6osHh0dHSQxMZF0dnYSnU5H7ty5QwCQjz76iGRn\nZ6vmYcmNGzfIzJkzSUZGhtn9a9eu5W9qJT16e3vJ4sWLyeHDh0l7ezsJCwtz9flk9QgPDycdHR1W\n98fHx5OtW7eSqVOnquKxefNmkpOTQ/z8/IjRaOzL87nu4cEni6I9Azk+gVzxKC8vx549e3gP2s7E\nc8U9POCF92CMQafTqd6Trq+v58fFpEmTHE2nUtTDkuTkZKsznvj4eJSXlyvusWPHDlBKHf3YzB4v\n5HGanp4OxpizoVGPPLzek3aDF77n2EdeeA/GGAFApk+fTk6cOOE1jz6imMeTJ0+I9L69ePEiKS8v\nJzk5OaRfv36Ke2g0GtLV1UVqa2vJyJEj+/J8L+Tr8vbbbxOj0UgOHTpkb/977CFC2n2EhznCwxzh\nYY7wMMenQlogEAgEbuL1yiwCgUAgsI8IaYFAIPBhREgLBAKBD6NGSLtS3pwvbW1tiI2NRUlJib11\nVPFwYREewkN4CA/FPXyqJ7169Wry2muvkdraWhIZGeltHcH/M3DgQEIpJZs3b/a2ikDwn8OnQrqg\noIDU1NSQgIAAEhMT420dASHkzz//JHfv3iWUUhISEuJtHa9z8uRJQikllFLy9ddfe1tH8B/AJ0Ja\nq9USSinZtGkTuX79OjEajWTw4MHe1lKVx48fk/Pnz1vd39jYSIKCgrxgRMh7771HEhISyMqVK0l9\nfT1ZtWqVVzz2799P4uPjSb9+/fhy/PhxVR1u3LhBGGPk9ddf5yG9ZcsWYjAYVPXwFTo7OwljjPTr\n14/ExsaStLQ0UlZWRuLi4ghjysfK1atXSUxMDD8eGGNk2LBh5Ny5c4pv2xnjx48njDH59oMHP5OU\n5Wedu3btAiEEI0aMkP3nlH3xcANZPUpKSrB+/Xqr+2tqarxSiQT43+Ue7RRaVdzj3LlzmDZtGr/Q\nVH5+Pnbt2oW4uDjVamBKTJ482eaFuKqrq1X1AJ4XpD158iTGjRvHxzgdoIiHVHbO9AqOAwYMwOjR\no1W5RKhUH1Wn0+GXX37BkSNHeA1KJyieH+PGjZO1nJi7wh43rq2tDTExMQgODkZeXh6ePn3q6j6Q\nfSffuXMHBQUFGDBggM0r0dn5AJHVgxDbF7Q/evSoV0JaehM6KB+mqMfw4cNBKcXMmTOtrht86dIl\nBAUF4a+//lLco7u7G++88w5/0wUEBOD06dMIDw/n1/RQY38kJiZCo9GYfUCkpqaqVltQ4u7du3y7\npqSnp6O4uBiUUkycOFFxj5ycHHR2dvLb7e3tYIzZKzGnmIctpNdk165dsni4K+xx4+Lj43kv4MmT\nJ33ZB7J6fP755wgPDzfrEViGdEhIiOIehBBkZWVZ3X/06FHVe0rd3d0IDAxEeHg4Ghoa0NPTg7a2\nNty8eVM1j+joaBBC8MYbb5gVZejt7UVGRoa9UlGye0jVe6Rl+fLlaG1tNbvwkhoepg65ubm4fv06\njhw5onpIX7hwAcnJyWCMmd1/9+5dXmFIDQ9LvvvuO2cFeVXxAMDPKo4dOyaLh7vCHjdOCmi9Xt+X\n9vepcY48srOzERsba7NMFWOMV/tljCEyMlIxD9P9YWu4Q+rRKr0/TJkxY4ZZWbFJkybxyul2rtEr\nu0dubi5/LUzrCUr1IJUuCiERGxvLg3D9+vUwGo148uQJL4iqVkjn5eUhLy/PrELMmjVrQAjB2LFj\n7f2b7B4AsGHDBjDGUFdXx+8bP368s6vBye4hUVRUBEopdu/e7crqinkAz6uIU0qRnp4um4e7wh41\nrrOzk4e06SnBsWPHkJubyx+bNWuWR41z5DF48GAkJiZi4cKFKCgowKlTp8weLysrA2MMUVFRaGtr\nU8yDP5mdkM7KykJoaKi9f5Pd4+OPPwalFLGxsTAYDLzw6PTp08EYw/Dhw+2VSpL94L937x5mzZoF\nxhhSU1Oh1WoRERHhrPqFbB5dXV08oMPDw/Hs2TP+WHl5OUJDQ1ULaZsbIARhYWGor693uJoSHqWl\npSCEYMKECaCUQq/Xm4W2Gh5FRUVWnSu9Xo/bt2+r6iFx//595OfnY9CgQWhsbHS2usvbdlfYo8Zd\nvnwZhBCMGTOGjyvt27cPOp0OMTExiIyMBCEEwcHBHjXOkYed8UyOVLLdwRiXrC+2aUjX1dWhqKgI\nu3fvxogRI7Bs2TJHqrJ6SKGUnZ2NAwcO8L8fP37MryWsZlmilpYWszOdjRs3OvsX2TwaGxv5/pgx\nY4bV4wkJCV4P6aSkJKerKeFx/fp1Hozx8fEu6crt0dLSgujoaGRmZuLmzZuIjo4GY8xZ5RzFXpfK\nykpERUW5coz2ycNdYY8al52dzXvLycnJ8Pf3x4IFC/jj9fX1IIQgLS3No8Y587BFb28vNBoN/P39\nzUonKe1ByPNCsyEhIVi/fj1qa2tRW1uL0NBQzJs3TzUPKZRMe5ESz549Uz2kAUCr1YIxhuLiYldW\nl80jKSmJ7wNbZ1OLFi3yWkhnZGTY+4JOcY+WlhYsWbKEh/TMmTO94mFJa2srn/XhDY/Vq1eDUoqr\nV6+6srrL23ZX2KPGSSEsLVlZWWhqakJvby82bNiA48ePgxBiLyQVfbG3bNkCxhjmzJnjbFVZPTIz\nM1FcXGwVBmFhYar3pDUaDbq7u61C+vDhw3jllVfQ3NysuIfEqVOneC/aWyE9ZcoUs7FgAKiurkZE\nRAS2bNmiuIclFy9ehEajkb0qtaseW7duBaUUeXl5fGpiZWWl6h62GDt2rNdCevjw4QgPD3dV1eVt\nuyvsUeN6e3vNQlqqRD1q1Ch+3/Lly3kVcXcb58zDkoqKCj4M09HR4Wx1xQ86g8GA0NBQ1UN61apV\n/G9KKZqbm3Hp0iXExcXh2rVrqngAwO3bt3kv2pshbRmGHR0diIiIAKUUN27cUNzDktOnT9v0UsNj\n3rx5iIyM5NNFbX2JqIaHPbwV0p2dnaCU4s0333RV1eVtuyvsceO+/PJLDBkyxOrCI2lpaV4Z+O/u\n7uazOlxE8YPu2rVrCA4O9sqY9OrVqxEWFsZvf/bZZ2ZfnCntkZqaCkIIYmJisG/fPlBKcfDgpqI/\nRgAAAqdJREFUQUfbl91DCmmtVovjx49j5MiRiIqK4vtk9uzZqu0PU1yYdqeIR11dHSilKCwsxMOH\nD2EwGDB27FgUFhaq6mGPdevWqT4lEQB++ukn6PV6V2tw9tnDXWFZGtfT04P8/Hwe0JWVlWZzYj1t\nnKseBoMB8+fP70tvTREPS6qqquDv7++VkDZd0tLSXCnCKqtHYGAgQkJC0NjYiLy8PGe9I0U8TMek\nbS3nz59XxcMSb4X0gQMH0L9/fz7/d+fOnaCUWg0FKe0BAI8ePcKFCxfQ0NDAb0tnXEuXLlXNAwAW\nLFgASqkrMzrc8nBXWNaDro/I6iFV+x0/frxXPeyh9pi0B8jqYfnDotGjR6vuIc15NV38/f2d/ahH\ndg+rJyYEZ86ccWVVWT3y8/PNXpPAwEDs27dPdQ/g+S+WLX/fkJOTg3/++UdVDz8/P+h0Opw7d87Z\ndt32cFf4hQgDg8GAoKAgpKSkuPLiKubhiOTk5P9kSAcHB/NA2LNnDx48eOAVDw9QxKOlpQWUUpSV\nlanuIX2ZLE27a21tddVBVg/g+Xt31KhRPKQzMzO94kEpRUlJiavbdsvjP1stvKenh7z77rvEaDSS\n8vJyr3nIgPAw54X2aGtrI0OHDiUDBw4kt27d8pqHGwgPc3yqWrhP7uSJEyeSEydOkIEDB3rVQwaE\nhznCwxzhYc6/zkONkBYIBAKBm/jERf8FAoFAYBsR0gKBQODDiJAWCAQCH0aEtEAgEPgwIqQFAoHA\nhxEhLRAIBD6MCGmBQCDwYURICwQCgQ8jQlogEAh8GBHSAoFA4MOIkBYIBAIfRoS0QCAQ+DAipAUC\ngcCHESEtEAgEPowIaYFAIPBhREgLBAKBDyNCWiAQCHwYEdICgUDgw4iQFggEAh9GhLRAIBD4MP8H\n4zoYljtMywwAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# 訓練用データを100個表示\n", "for index in range(100):\n", " plt.subplot(10, 10, index + 1)\n", " plt.axis('off')\n", " plt.imshow(train_set[0][index].reshape(28, 28), cmap=plt.cm.gray_r, interpolation='nearest')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

データをTheanoの共有変数に格納

\n", "\t

\n", "\t\tTheanoの共有変数を使用すると、GPUのメモリ領域に保存され、学習時（update）に高速に読み書きできます。\n", "\t\t共有変数の値を取り出すときに、get_value()を使用するのは、GPUにセットされたデータをCPUのメモリにコピー\n", "\t\tするためと考えられます。\n", "\t

\n", "\t

\n", "\t\tまた、GPUで使用するデータは必ずfloat型で格納しなければなりません。\n", "\t\tそのため、ラベルshared_yは、T.cast(shared_y, 'int32')でキャストして返しています。\n", "\t

\n", "\t

\n", "\t\tこれでdatasets変数に訓練用、検証用、テスト用のデータがセットされました。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# データセットをGPUの共有変数に格納\n", "def shared_dataset(data_xy, borrow=True):\n", " data_x, data_y = data_xy\n", "\n", " # 共有変数には必ずfloat型で格納\n", " shared_x = theano.shared(\n", " np.asarray(data_x, dtype=theano.config.floatX), borrow=borrow)\n", " shared_y = theano.shared(\n", " np.asarray(data_y, dtype=theano.config.floatX), borrow=borrow)\n", "\n", " # ラベルはint型なのでキャストして返す\n", " return shared_x, T.cast(shared_y, 'int32')\n", "\n", "test_set_x, test_set_y = shared_dataset(test_set)\n", "valid_set_x, valid_set_y = shared_dataset(valid_set)\n", "train_set_x, train_set_y = shared_dataset(train_set)\n", "\n", "datasets = [(train_set_x, train_set_y),\n", " (valid_set_x, valid_set_y),\n", " (test_set_x, test_set_y)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

多クラス分類

\n", "\t

\n", "\t\tニューラルネットの入力$x_i$と出力$u_i$の関係は、重み$W_i$とバイアス$b_i$を使って以下の関係になります。\n", "$$\n", "\tu_i = \\sum_j W_{i,j} x_j + b_i\n", "$$\t\t\n", "\t

\n", "\t

\n", "\t\tこれを分かりやすく説明した図を再度、TensorFlowのMNIST for ML Beginnersから引用します。\n", "\t

\n", "\t

\n", "\t\t\n", "\t

\n", "\t

\n", "\t\tこの重みWとバイアスbをTheanoの共有変数でセットしているのが、以下の箇所です。\n", "\t\t

\n",
    "        # 重み行列を初期化\n",
    "        self.W = theano.shared(value=np.zeros((n_in, n_out),\n",
    "                                              dtype=theano.config.floatX),\n",
    "                               name='W',\n",
    "                               borrow=True)\n",
    "\n",
    "        # バイアスベクトルを初期化\n",
    "        self.b = theano.shared(value=np.zeros((n_out,),\n",
    "                                              dtype=theano.config.floatX),\n",
    "                               name='b',\n",
    "                               borrow=True)\t\t\t\n",
    "\t\t

\t\t\n", "\t

\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

ソフトマック関数

\n", "\t

\n", "\t\t多クラス分類の活性化関数として使用されるのが、ソフトマック関数（Softmax）で、出力層のクラスiに分類される確率は、以下の様に表されます。\n", "$$\n", "\ty_i = \\frac{exp(u_i)}{\\sum_j exp(u_j)}\n", "$$\t\t\n", "\t

\n", "\t

\n", "\t\t\n", "\t

\n", "\t

\n", "\t\tこの部分をTheanoのシンボルで表現している箇所が以下の部分です。\n", "\t\t

\n",
    "        # 各サンプルが各クラスに分類される確率を計算するシンボル\n",
    "        # 全データを行列化してまとめて計算している\n",
    "        # 出力は(n_samples, n_out)の行列\n",
    "        self.p_y_given_x = T.nnet.softmax(T.dot(input, self.W) + self.b)\t\t\t\n",
    "\t\t

\n", "\t

\n", "\t\t人工知能に関する断創録の貴重なコメントがあります。\n", "\t\t

T.dot(self.W, input)+ self.bは、サンプルxを1つだけ与えたときの確率を計算している
T.dot(input, self.W)+ self.bは、複数のサンプル（ミニバッチ単位）を行列化してinputに与え、それらのサンプルの確率をまとめて計算している

\n", "\t

\n", "\t\t確率が求まったら最終的に一番高い確率が得られるクラスをy_predにセットします。\n", "$$\n", "\t\ty_{pred} = argmax(y)\n", "$$\t\t\n", "\t\tTheanoではT.argmax()にaxis=1を指定することでp_y_given_xの各行（サンプルに相当）において一番確率が高いインデックス（クラスに相当）がまとめて取得できるそうです。\n", "\t\t

\n",
    "        # 確率が最大のクラスのインデックスを計算\n",
    "        # 出力は(n_samples,)のベクトル\n",
    "        self.y_pred = T.argmax(self.p_y_given_x, axis=1)\t\t\t\n",
    "\t\t

\n", "\t

\t\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

コスト関数

\n", "\t

\n", "\t\t分類されたクラスkだけ１で、他は0のベクトルを$d_n$とすると、事後分布は、以下の様になります。\n", "$$\n", "\tp(d | x) = \\prod_{k=1}^K p(C_k | x)^{d_k}\n", "$$\n", "\t\t訓練データ${ (x_n, d_n)}(n=1,...,N)$に対するwの尤度は、以下の様になります。\n", "$$\n", "\tL(W) = \\prod_{n=1}^N p(d_n | x_n; W) = \\prod_{n=1}^N \\prod_{k=1}^K p(c_k|x_n)^{d_{nk}} = \\prod_{n=1}^N \\prod_{k=1}^K (y_k(x; W))^{d_{nk}}\n", "$$\t\t\n", "\t

\n", "\t

\n", "\t\t負の対数尤度を誤差関数とすると\n", "$$\n", "\tE(W) = - \\sum_{n=1}^N \\sum_{k=1}^K d_{nk} log y_k(x_n; W)\n", "$$\t\t\n", "\t

\n", "\t

\n", "\t\t人工知能に関する断創録では、Sumの代わりに平均meanを使っていることに注意！\n", "\t\t

\n",
    "        # 式通りに計算するとsumだがmeanの方がよい\n",
    "        return -T.mean(T.log(self.p_y_given_x)[T.arange(y.shape[0]), y])\t\t\t\n",
    "\t\t

\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "class LogisticRegression(object):\n", "\n", " def __init__(self, input, n_in, n_out):\n", " \"\"\"ロジスティック回帰モデルの初期化\n", " input: ミニバッチ単位のデータ行列（n_samples, n_in）\n", " n_in : 入力の次元数\n", " n_out: 出力の次元数\n", " \"\"\"\n", " # 重み行列を初期化\n", " self.W = theano.shared(value=np.zeros((n_in, n_out),\n", " dtype=theano.config.floatX),\n", " name='W',\n", " borrow=True)\n", "\n", " # バイアスベクトルを初期化\n", " self.b = theano.shared(value=np.zeros((n_out,),\n", " dtype=theano.config.floatX),\n", " name='b',\n", " borrow=True)\n", "\n", " # 各サンプルが各クラスに分類される確率を計算するシンボル\n", " # 全データを行列化してまとめて計算している\n", " # 出力は(n_samples, n_out)の行列\n", " self.p_y_given_x = T.nnet.softmax(T.dot(input, self.W) + self.b)\n", "\n", " # 確率が最大のクラスのインデックスを計算\n", " # 出力は(n_samples,)のベクトル\n", " self.y_pred = T.argmax(self.p_y_given_x, axis=1)\n", "\n", " # ロジスティック回帰モデルのパラメータ\n", " self.params = [self.W, self.b]\n", "\n", " # モデルの入力値を保持\n", " self.input = input\n", " \n", " def negative_log_likelihood(self, y):\n", " \"\"\"誤差関数である負の対数尤度を計算するシンボルを返す\n", " yにはinputに対応する正解クラスを渡す\n", " \"\"\"\n", " # 式通りに計算するとsumだがmeanの方がよい\n", " return -T.mean(T.log(self.p_y_given_x)[T.arange(y.shape[0]), y])\n", "\n", " def errors(self, y):\n", " \"\"\"分類の誤差率を計算するシンボルを返す\n", " yにはinputに対応する正解クラスを渡す\"\"\"\n", " if y.ndim != self.y_pred.ndim:\n", " raise TypeError('y should have the same shape as self.y_pred',\n", " ('y', y.type, 'y_pred', self.y_pred.type))\n", "\n", " if y.dtype.startswith('int'):\n", " return T.mean(T.neq(self.y_pred, y))\n", " else:\n", " raise NotImplementedError()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

モデルの訓練

\n", "\t

\n", "\t\tモデルの訓練には、ミニバッチ確率的勾配降下法（MSGD）を使用しています。\n", "\t\t確率的勾配降下法（SGD）はただ1つのサンプルで1回だけパラメータを更新するのに対し、\n", "\t\t少数のサンプルをひとまとめにしてその単位で重みを更新します。\n", "\t\tこのひとまとめにしたサンプル集合をミニバッチ（minbatch）と呼びます。\n", "\t

\n", "\t

\n", "\t\t何番目のミニバッチを使用するかを示すのがシンボルindexです。\n", "\t\tコードのfunction()のgivensで定義されている部分がミニバッチの設定箇所です。\n", "\t\t

\n",
    "        givens={\n",
    "            x: train_set_x[index * batch_size: (index + 1) * batch_size],\n",
    "            y: train_set_y[index * batch_size: (index + 1) * batch_size]\n",
    "        }\t\t\t\n",
    "\t\t

\n", "\t

\n", "\t\t確率的勾配降下法で使用するコスト関数とその微分は、負の対数尤度を計算する\n", "\t\tnegative_log_likelihood関数とT.gradの箇所で計算しています。\n", "\t\t

\n",
    "    # 誤差（コスト）を計算 => 最小化したい\n",
    "    cost = classifier.negative_log_likelihood(y)\n",
    "    # コスト関数のtheta = (W,b)の微分を計算\n",
    "    g_W = T.grad(cost=cost, wrt=classifier.W)\n",
    "    g_b = T.grad(cost=cost, wrt=classifier.b)\t\t\t\n",
    "\t\t

\n", "\t

\n", "\t\tパラメータの更新式は、Wとbの2個をタプルとして指定します。\n", "\t\t

\n",
    "    # パラメータ更新式\n",
    "    updates = [(classifier.W, classifier.W - learning_rate * g_W),\n",
    "               (classifier.b,  classifier.b - learning_rate * g_b)]\t\t\t\n",
    "\t\t

\n", "\t

\n", "\t\t関数の定義は以下の通りです。\n", "\t\t

inputs：何番目のミニバッチかを示すインデックスをindexで与え
outputs：負の対数尤度を計算するcostを渡す
updates：update式を渡す
givens：関数を実行する際にシンボルを置き替えるオプションで、ここではじめてxとyに値をセットしています

\n", "\t

\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

モデルの評価

\n", "\t

\n", "\t\tモデルの当てはまりの良さをエラー率で評価しています。LogisticRegressionのerrors関数で計算します。\n", "\t\t

\n",
    "    def errors(self, y):\n",
    "        \"\"\"分類の誤差率を計算するシンボルを返す\n",
    "        yにはinputに対応する正解クラスを渡す\"\"\"\n",
    "        return T.mean(T.neq(self.y_pred, y))\n",
    "\t\t

\n", "\t\tT.neq(self.y_pred, y)で予測クラスと正解クラスが異なる要素の数を求め、\n", "\t\tその平均を取ることでエラー率を計算しています。\n", "\t

\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

Early-Stoppingによる収束判定

\n", "\t

\n", "\t\t検証用データvalid_setのエラー率と訓練用データtrain_setのエラー率から適合（overfitting）を防ぐために、\n", "\t\t検証用データのエラー率が増加した時点で学習を打ち切る早期終了（Early-Stopping）というテクニックを使用します。\n", "\t

\n", "\t

\n", "\t\t人工知能に関する断創録に解説されているEarly-Stoppingの説明を引用します。\n", "\t\t

モデルの訓練は訓練データのミニバッチ単位で行う。
エポック epoch は訓練データのミニバッチを使い切ったら+1される。\n", "\t\t\t\t訓練データを一通り使いきっても収束していなければまた最初のミニバッチに戻って同じデータを繰り返し使ってパラメータ更新する。
最初の while ループは、最大n_epochs 回エポックを繰り返されたら更新を終えることを意味する。\n", "\t\t\t\tこれによりたとえ収束しなくてもプログラムはいつかは終わることが保証される。
iterはこれまで訓練に使用したミニバッチの数が入る。つまり、パラメータ更新回数を意味する。
パラメータ更新回数がpatienceを超えたらもうこれ以上更新してもモデルは改善しないと判断されてループを抜ける。\n", "\t\t\t\t逆に言うと、patienceの値が上がるとまだまだパラメータ更新によるモデル改善の余地があると判断されてことになる。
patienceを上げるかどうかの判断はバリデーションセットを用いて決められる。
validation_frequency回更新されるたびにモデルの検証が行われる。
バリデーションセットを用いて平均エラー率が計算され、\n", "\t\t\t\tこの平均エラー率がこれまでの最良の平均エラー率より十分な程度下がっていればまだまだモデル改善の余地ありと判断してpatienceを引き上げる。\n", "\t\t\t\t十分かどうかはimprovement_thresholdで調整できる。
この検証の際にテストセットを用いたモデルのエラー率の評価も合わせて行う。

\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def sgd_optimization_mnist(learning_rate=0.13, n_epochs=100,\n", " dataset='mnist.pkl.gz',\n", " batch_size=60):\n", " # 先にセットした学習データを利用\n", " # datasets = load_data(dataset)\n", "\n", " train_set_x, train_set_y = datasets[0]\n", " valid_set_x, valid_set_y = datasets[1]\n", " test_set_x, test_set_y = datasets[2]\n", "\n", " # ミニバッチの数\n", " n_train_batches = train_set_x.get_value(borrow=True).shape[0] // batch_size\n", " n_valid_batches = valid_set_x.get_value(borrow=True).shape[0] // batch_size\n", " n_test_batches = test_set_x.get_value(borrow=True).shape[0] // batch_size\n", "\n", " print('... building the model')\n", "\n", " # シンボルの割り当て\n", " # ミニバッチのインデックスを表すシンボル \n", " index = T.lscalar() # index to a [mini]batch\n", " \n", " # ミニバッチの学習データとラベルを表すシンボル\n", " x = T.matrix('x') # data, presented as rasterized images\n", " y = T.ivector('y') # labels, presented as 1D vector of [int] labels\n", "\n", " # MNISTの手書き数字を分類する多クラス分類モデル\n", " # 入力は28ピクセルx28ピクセルの画像、出力は0から9のラベル\n", " # 入力はシンボルxを割り当てておいてあとで具体的なデータに置換する\n", " classifier = LogisticRegression(input=x, n_in=28 * 28, n_out=10)\n", "\n", " # 誤差（コスト）を計算 => 最小化したい\n", " cost = classifier.negative_log_likelihood(y)\n", "\n", " # index番目のテスト用ミニバッチを入力してエラー率を返す関数を定義\n", " test_model = theano.function(\n", " inputs=[index],\n", " outputs=classifier.errors(y),\n", " givens={ # ここで初めてシンボル x, y を具体的な値で置き換える\n", " x: test_set_x[index * batch_size: (index + 1) * batch_size],\n", " y: test_set_y[index * batch_size: (index + 1) * batch_size]\n", " }\n", " )\n", " \n", " # index番目のバリデーション用ミニバッチを入力してエラー率を返す関数を定義\n", " validate_model = theano.function(\n", " inputs=[index],\n", " outputs=classifier.errors(y),\n", " givens={\n", " x: valid_set_x[index * batch_size: (index + 1) * batch_size],\n", " y: valid_set_y[index * batch_size: (index + 1) * batch_size]\n", " }\n", " )\n", "\n", " # コスト関数のtheta = (W,b)の微分を計算\n", " g_W = T.grad(cost=cost, wrt=classifier.W)\n", " g_b = T.grad(cost=cost, wrt=classifier.b)\n", "\n", " # パラメータ更新式\n", " updates = [(classifier.W, classifier.W - learning_rate * g_W),\n", " (classifier.b, classifier.b - learning_rate * g_b)]\n", "\n", " # index番目の訓練バッチを入力し、パラメータを更新する関数を定義\n", " # 戻り値としてコストが返される\n", " # この関数の呼び出し時にindexに具体的な値が初めて渡される\n", " train_model = theano.function(\n", " inputs=[index],\n", " outputs=cost,\n", " updates=updates,\n", " givens={\n", " x: train_set_x[index * batch_size: (index + 1) * batch_size],\n", " y: train_set_y[index * batch_size: (index + 1) * batch_size]\n", " }\n", " )\n", "\n", " print('... training the model')\n", " \n", " # eary-stoppingのパラメータ\n", " # patience = 5000 \n", " patience = 500\n", " patience_increase = 2 \n", " improvement_threshold = 0.995 \n", " validation_frequency = min(n_train_batches, patience // 2)\n", "\n", " best_validation_loss = np.inf\n", " test_score = 0.\n", " start_time = timeit.default_timer()\n", "\n", " done_looping = False\n", " epoch = 0\n", " while (epoch < n_epochs) and (not done_looping):\n", " epoch = epoch + 1\n", " for minibatch_index in range(n_train_batches):\n", " # minibatch_index番目の訓練データのミニバッチを用いてパラメータ更新\n", " minibatch_avg_cost = train_model(minibatch_index)\n", " \n", " # validation_frequency回の更新ごとにバリデーションセットによるモデル検証が入る\n", " iter = (epoch - 1) * n_train_batches + minibatch_index\n", "\n", " if (iter + 1) % validation_frequency == 0:\n", " # バリデーションセットの平均エラー率を計算\n", " validation_losses = [validate_model(i)\n", " for i in range(n_valid_batches)]\n", " this_validation_loss = np.mean(validation_losses)\n", "\n", " print(\n", " 'epoch %i, minibatch %i/%i, validation error %f %%' %\n", " (\n", " epoch,\n", " minibatch_index + 1,\n", " n_train_batches,\n", " this_validation_loss * 100.\n", " )\n", " )\n", "\n", " # エラー率が十分改善したならまだモデル改善の余地があるため\n", " # patienceを上げてより多くループを回せるようにする\n", " if this_validation_loss < best_validation_loss:\n", " #improve patience if loss improvement is good enough\n", " if this_validation_loss < best_validation_loss * \\\n", " improvement_threshold:\n", " patience = max(patience, iter * patience_increase)\n", "\n", " best_validation_loss = this_validation_loss\n", " \n", " # テストセットを用いたエラー率も求めておく\n", " test_losses = [test_model(i)\n", " for i in range(n_test_batches)]\n", " test_score = np.mean(test_losses)\n", "\n", " print(\n", " (\n", " ' epoch %i, minibatch %i/%i, test error of'\n", " ' best model %f %%'\n", " ) %\n", " (\n", " epoch,\n", " minibatch_index + 1,\n", " n_train_batches,\n", " test_score * 100.\n", " )\n", " )\n", "\n", " # save the best model\n", " with open('data/best_model.pkl', 'wb') as f:\n", " pickle.dump(classifier, f)\n", " \n", " # patienceを超えたらループを終了\n", " if patience <= iter:\n", " done_looping = True\n", " break\n", "\n", " end_time = timeit.default_timer()\n", " print(\n", " (\n", " 'Optimization complete with best validation score of %f %%,'\n", " 'with test performance %f %%'\n", " )\n", " % (best_validation_loss * 100., test_score * 100.)\n", " )\n", " print('The code run for %d epochs, with %f epochs/sec' % (\n", " epoch, 1. * epoch / (end_time - start_time)))\n", " print('Ran for %.1fs' % ((end_time - start_time)))" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "... building the model\n", "... training the model\n", "epoch 1, minibatch 83/83, validation error 16.458333 %\n", " epoch 1, minibatch 83/83, test error of best model 15.833333 %\n", "epoch 2, minibatch 83/83, validation error 14.375000 %\n", " epoch 2, minibatch 83/83, test error of best model 12.916667 %\n", "epoch 3, minibatch 83/83, validation error 13.333333 %\n", " epoch 3, minibatch 83/83, test error of best model 11.666667 %\n", "epoch 4, minibatch 83/83, validation error 13.541667 %\n", "epoch 5, minibatch 83/83, validation error 13.750000 %\n", "epoch 6, minibatch 83/83, validation error 13.125000 %\n", " epoch 6, minibatch 83/83, test error of best model 10.833333 %\n", "epoch 7, minibatch 83/83, validation error 12.916667 %\n", " epoch 7, minibatch 83/83, test error of best model 10.208333 %\n", "epoch 8, minibatch 83/83, validation error 13.125000 %\n", "epoch 9, minibatch 83/83, validation error 13.125000 %\n", "epoch 10, minibatch 83/83, validation error 12.916667 %\n", "epoch 11, minibatch 83/83, validation error 12.916667 %\n", "epoch 12, minibatch 83/83, validation error 12.291667 %\n", " epoch 12, minibatch 83/83, test error of best model 8.750000 %\n", "epoch 13, minibatch 83/83, validation error 12.291667 %\n", "epoch 14, minibatch 83/83, validation error 12.500000 %\n", "epoch 15, minibatch 83/83, validation error 12.500000 %\n", "epoch 16, minibatch 83/83, validation error 12.500000 %\n", "epoch 17, minibatch 83/83, validation error 12.500000 %\n", "epoch 18, minibatch 83/83, validation error 12.500000 %\n", "epoch 19, minibatch 83/83, validation error 12.708333 %\n", "epoch 20, minibatch 83/83, validation error 12.708333 %\n", "epoch 21, minibatch 83/83, validation error 12.708333 %\n", "epoch 22, minibatch 83/83, validation error 12.708333 %\n", "epoch 23, minibatch 83/83, validation error 13.125000 %\n", "Optimization complete with best validation score of 12.291667 %,with test performance 8.750000 %\n", "The code run for 24 epochs, with 28.264020 epochs/sec\n", "Ran for 0.8s\n" ] } ], "source": [ "sgd_optimization_mnist()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

学習結果を使った予測

\n", "\t

\n", "\t\tテスト用データtest_setの最初の25個に対して認識を行い、予測結果と画像を合わせて表示してみます。\n", "\t

\n", "\t

\n", "\t\t1/10の学習データでもそこそこ良い結果が得られています。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Predicted values for the first 10 examples in test set:\n", "['(7, 0.988)', '(2, 0.733)', '(1, 0.951)', '(0, 0.995)', '(4, 0.936)', '(1, 0.979)', '(4, 0.957)', '(9, 0.912)', '(6, 0.912)', '(9, 0.844)', '(0, 0.904)', '(6, 0.668)', '(9, 0.917)', '(0, 0.983)', '(1, 0.991)', '(5, 0.832)', '(9, 0.790)', '(7, 0.986)', '(3, 0.659)', '(4, 0.979)', '(9, 0.775)', '(6, 0.952)', '(6, 0.801)', '(5, 0.955)', '(4, 0.862)']\n" ] } ], "source": [ "# 学習結果をテスト用データで確かめる\n", "classifier = pickle.load(open('data/best_model.pkl'))\n", "test_set_x, test_set_y = datasets[2]\n", "test_set_x = test_set_x.get_value()\n", "\n", "# compile a predictor function\n", "predict_model = theano.function(\n", " inputs=[classifier.input],\n", " outputs=classifier.p_y_given_x\n", " )\n", " \n", "predicted_values = predict_model(test_set_x[:25])\n", "print(\"Predicted values for the first 10 examples in test set:\")\n", "print([\"(%d, %.3f)\" % (np.argmax(value), max(value)) for value in predicted_values])" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAEKCAYAAACxA4b4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXd4FUXXwH8TAiRAKFIEpSolFGm+SK8C0hGEIC/VEhTU\nD0FBEaQoLyoCFkBpIggq0ksERUCadBIElF4CSOglCYFAwnx/rDvkkhCScLO792Z+z3Mf7u7dnT0Z\nZs/OnjlFSCnRaDQajTX42C2ARqPRZCS00tVoNBoL0UpXo9FoLEQrXY1Go7EQrXQ1Go3GQrTS1Wg0\nGgvRSlej0WgsxPFKVwgRJYSI/PcTJYSIE0J8YbdcdiOEyCKEmCaEOC6EuCqE2CmEaGa3XHYihHhN\nCLFdCHFDCDHdbnmcghAijxBikRAiWghxTAjR2W6ZnIIQopQQ4roQ4jurrulr1YXSipQywPwuhMgG\nnAHm2ieRY/AFTgB1pZQnhRAtgblCiApSyhM2y2YX/wAfAs8A/jbL4iS+Am4A+YGqwM9CiF1Syn32\niuUIJgDbrLyg42e6d9EROCel/MNuQexGShkjpfxASnny3+2fgWPAk/ZKZh9SysVSyqXAJbtlcQr/\nTlTaA0OklNf/vXeWAt3slcx+hBDPA5eB1VZe19OUbnfAstcAT0II8TBQCvjLblk0jqI0ECelPJJg\n359AeZvkcQRCiJzACOAtQFh5bY9RukKIokA9YKbdsjgNIYQvMBuYIaU8aLc8GkeRA7h6176rQEAS\nx2YkPgCmSin/sfrCjrfpJqA7sFFKGW63IE5CCCEwFG4s8IbN4micRzSQ8659OYEoG2RxBEKIykBj\noLId1/ckpdsNGGW3EA7kGyAf0EJKGW+3MBrHcRDwFUI8nsDEUImMbYaqDxQDTvw7ackBZBJClJNS\n/ie9L+4RSlcIUQt4BJhvtyxOQggxCQgEGkspb9otj90IITIBmYFMGIomK4Y9M8M+jKSUMUKIhcAH\nQohgoArQBqhlr2S2Mhn4McH2AAwl/KoVF/cUm253YIGU8prdgjiFf23cvTBekc4m8GfOyD6YQ4AY\n4B2gy7/fB9sqkTN4DcgGnAO+B17NyO5iUsobUspz5gfDBHNDSmmJ14vQScw1Go3GOjxlpqvRaDRe\ngVa6Go1GYyFa6Wo0Go2FaKWr0Wg0FmKFy5iTV+osDf9LgJP7BOzpF90nSePkftF9kjTJ9oue6Wo0\nGo2FaKWr0Wg0FuIREWkagzFjxgBw/fp1du/eDcD8+UaQXu/evQGoWbMm3bpl+Kx9Go1j0TNdjUaj\nsRIpZXp/nIwVf79b+iQoKEgKIe77KVWqlAwPD5fh4eFpuYyJR/RJSjlw4IDEWHyRX375ZVqb8Zix\ncjfR0dGyd+/eaoxUq1ZNHj9+XB4/fvxBm/bYPklnkpVfmxccTqdOnQCYN2+ey/7AwEAAmjVrxtGj\nR1m6dCkAhw8fZvbs2QC89957FkrqXMLCwvDxMV7qHn30UZulsZ7Tp08zdepUMmXKBMCOHTtYtmwZ\nAK+//rqdollKaGgoAO3bt+f48eMpOmflypUAlC1bliJFirhFDq10HcyOHTtYtGiR2q5QoQIAS5cu\nJV++fADkyJGDmzdvUr16dQD+/PNPLl68aL2wDmbXrl3kyJEDMG64jML58+cB6NGjh82SOINff/0V\ngNjY2BSfY05mpk+fzpw5c9wih7bpajQajYXYPtM1V9+nTp3KI488AoCfnx9dunShYMGCAJQsWdI2\n+ewkIiICKQ0/8AoVKqgndaFChVyOGzNmDPv23cnU16pVK+uEdDB79uwBYPz48XTv3t1maazlyy+/\nZPHixQBs37490e8bNmwAjDWdSpUqUa9ePUvls5q4uDiWL1+e6vP+8x8jp/m4ceO4du0a2bNnf2BZ\nbFe6AwYMAEhkY5k0aRI5cxpVRsqVK5fi9ooUKcLAgQOBOx3mqbRu3ZrDhw8DEBAQwEMPPZTkcT/9\n9BM3b2b4HOaJOHDgAADXrl1TtvGMwptvvqlsuEmxcOFC9W/RokWZO3cuAE8+6Z3FpH///Xc2bdoE\nwDvvvJPi8y5dMlLs/vXXX8TExLhF6Wrzgkaj0ViI7TPdadOmAcYCkDmj/fvvvwkLC2Pt2rUAbNmy\nhaJFi3LixIlE52fOnJl8+fIRERGhjjVXGT19pgtQrFixZH//9NNPOXjwTgHg6tWrq0W1jM7o0aMB\nKF68uFeMhZTSokULpJTExyddpShfvnxqxhYeHs6xY8eoVq0aALdv37ZMTiswTUzPP/+8MlOmxqvH\nXEhzJ7Yr3aefftrlXzDcoAAuX74MGC4///nPf5K0TWXNmpUyZcooF6pLly7x+OOPp7fYthMSEgLA\n0KFDiY2N5eGHHwbg448/Jlu2bHaK5giOHz+uxkuZMmXc8lrodNatWwfA/v37EUIkMi+8+qpRAqxp\n06bkypULgDVr1vC///1PHfP111+r6EZvwPzbYmJilCul6clyPy5duqT61Khf6R5sV7rJkSdPHgAa\nNWoEuCrmhCxYsEAp6IoVK/L8889bI6CN7NixA7jj/mLaLOvXr2+bTE7CvFkA8ufPb6Mk1nD8+HE1\n7i9cuODyW9GiRenQoQPDhg0DcHkoFytWjMmTJ6tzBg4cyI0bNwDDhzdz5sxWiJ8uzJ8/Xy2elSxZ\nUs3mU8rIkSOVsm3QoAG5c+d2i1zapqvRaDRWcr+QNTd80pWzZ8/KAgUKqDDP+fPnp+Z0jwxjbNu2\nrfTz85N+fn5SCCF79uwpo6KiZFRU1IM2beJxfXI3/fv3V2Ni6dKl7mjS0WPl4MGD0tfXV32EELJR\no0ayUaNG8vz588me++WXX7qcZ34/fPiwR/dJUFCQ9PHxkT4+PnLixIkpPU0eO3ZMHjt2TD788MMy\nc+bMMnPmzHL16tUpPv9+8jvavJASJk6cyLlz59TUv0yZMjZLlH6Yi4WbNm1SZoX8+fMzZMiQFNup\nMgKbN2/m22+/pUqVKgA0adLEZomsp1q1anz77bcAKnrxXrRp04bvv/8egG3btqW7bFZw9epVtmzZ\norb79OmT4nOnTJkCGBF95uK+aeJ0B9q8oNFoNBbisTPdjRs3AsZqPcCSJUuAO/kJvBEzb0DChZIu\nXbpkCG+N1LB69WouX76svGD8/PxslsgaErqIbd26NcXnSSmVq5iUd1zNhg0bplb8PY3Y2FhOnTpF\n586dU33ukSNH1Pf00Cceq3TNVcmbN2/SuHFjatasabNE6cvSpUsJCwtT2w0aNADggw8+sEki5/Ln\nn38C0LFjR5slsY5JkyYlG4GWHMuWLVNjK6Gr2YgRI9wmn9UEBARQuXJl5ad76dKle0Z0JuTcuXMu\nGf1q167tdtk8Uulev36dX375BTD8dEeMGOHRri3JYWYMGzVqlEuob+XKlYGU+xxmBM6cOQMYeQUC\nAwNp166dzRJZh+m3nVLMDGR///03o0aNcvnNtAF78j3l7+9PyZIlVW6Xli1b0r9//ySP3bt3r5rd\nhoeHu/jkmilB3Ym26Wo0Go2FeORM99NPP1WvQ82bN6dWrVo2S5R+jB07FnBdVX722We1WSEJZsyY\nAcDZs2dp3ry5vcI4HDNSa+LEiS77ixcvzsyZMwEjqMKTGT58OFIaWfpCQkLuGTSVP39+Nbu9O7Dk\nhRdecLtcHqV0zVeoDz/8UIUxvv/++3aKlO6MGzcu0b6JEydqs0IShIeHq+9mNKMmMS1atGD//v1J\n/lauXDnq1q1rsUTpQ9myZVX2tLCwMJcFsoR06NBBfe/Ro4fL4qG/v7/b5fIYpXvx4kX+7//+DzBy\nY7Zo0QLA6xfQkuLixYv3tLflypVL/Xbr1i2uXr2qfrt8+TKfffZZonMyZcrEJ598AuDReRvMEjSQ\n8XIKJ/Q6AFixYoX6HhwczOnTp12OvVcugdTahj2FKlWqKL/t5Hjsscdcts2FuCeeeMJtsmibrkaj\n0ViIR8x04+PjadasGceOHQOM5BUffvihzVLZR8WKFe/5W1BQkKoscfbs2RTXdTKzlA0ZMuTBBbSB\nDRs2cPbsWbvFsI3evXur5P1grNYndCFL+D0+Pv6eGcgyOmaorok7Z7gmHqF0jxw5orJqgWHnzCgB\nAaYZxSy9cj9MG9bdZM6c2cX9pU2bNi45ZuvUqfMAUtrPokWLiIuLA4xXyYyWba19+/Yqf/Ddi0FJ\nYbqFlS1blqlTpyYqAZVREUK4NY1jUmjzgkaj0ViIo2e65mp006ZNAaMAI2SsRRKzltXo0aOTrIP2\n999/JzIhvPTSS4Br1YnnnnuOsmXLpqOk9hATEwO4Lhx17NgxzdFZnkqxYsX46aefAOOt6PPPP0/2\n+MGDBwNGzlzNHcxcwpA+ngsAIqH9Ip1I8wXMshofffQRcKeqqRtLr6Tve8S9SfdOf0Ds6Jc09cmt\nW7cAqFevnrJL//DDD+nhheFRY+WXX35R2bKWLVtG69ateeWVV4wGpVTZsx7QF9ej+iQlFCxYUI2p\noUOH0rdv37Q0k3y/3C/3oxs+aWL9+vUyICBABgQEqLyoWbJkka1bt05rk0nh6HygKeHgwYPSz89P\nduvWzZ3Nemyf7Nu3TzZq1EjmypVLlipVSi5atMhdTXv0WPnxxx9l2bJlZfbs2WXJkiXlxo0b3dGs\nx/bJvcZJq1at5OrVq1ObP/dukpXfo2y6bdq0sVsEx/H666/z1FNP2S2GI4iPj6dt27a0adOGy5cv\nM3nyZLp27arK2GdUfvvtNwYNGsTMmTOJjo5m/fr1ifxRMxJ2jxPH2nQ3btxIVFSU2i5ZsiT58uXj\n3LlzNkrlLObMmUOePHkoV65chlcsYBRkjIiIUK+EDRs2pHbt2syaNcujM2Y9KMOHD2fo0KGqRlhG\n91RIbpwkDLBJLzxiplu5cmW2bt1Kvnz5VPhvRicyMpJhw4YxduxYpHS6idgakuoHKSV79+61QRpn\ncPv2bXbs2MG5c+coVaoURYsW5Y033lCVRzIido8Tj1C6msQMHTqU4OBgHn30UbtFcQyBgYEUKFCA\nMWPGEBcXx8qVK1m3bp3ycMiInD17llu3brFgwQL++OMPdu3aRVhYGCNHjrRbNNuwe5w4VukOGjRI\nGZ7DwsJSlIA4o7Br1y5WrVrFm2++abcojsLX15fFixcTEhJCoUKF+Oyzz+jUqROFCxe2WzTbMN2e\n/u///o8CBQrw0EMP0b9/f1UEICNi9zixwmXMbQghPgQelVK+aLcsdiKE6AuMBKIw3FNyAJmAv6WU\nbvOn8waEEH8AM6SUU+2WxS6EECeA96SUs//dbg8MllI+aa9kzsHKceLYmW5ChBCZhBB+GIrFVwiR\nVQiRsbzfXZkMPA5UBioBk4AQoKmdQjkBIcQT/46PbEKIt4GCwAybxbKbb4E3hBD5hRB5gDeB9F8x\ncjB2jhOPULrAECAGeAfo8u/3wbZKZCNSyhtSynPmB4gGbkgpL9ktmwPoBkQAZ4CGQBMp5S17RbKd\nD4EdwEHgL2AnMCrZM7wf28aJR5kXNBqNxtPxlJmuRqPReAVa6Wo0Go2FaKWr0Wg0FqKVrkaj0ViI\nFbkXnLxS53Wp6dyEx6R2tBA9VhKj+yRpku0XPdPVaDQaC9FKV6PRaCxEK12NRqOxEK10NRqNxkIc\nm8Rco9GkP5cvX+bEiRMu+8yCpp999hkVKlSgdOnSAFSqVMly+bwRj1W6Zob3Nm3aMH78eHr37g3g\ntVVgz507R1BQEAC1atWiV69eABQvXjzFbVy9epX169cD0KxZMzJnzux2OTXOJyQkRN0/a9eu5dCh\nQy6/lylTBoDjx4+7JDu/ffu2dUJ6MY6uBnwvLl68qJ66//zzD3CnFHcqyyY73uXl8uXLAJQuXZqr\nV68C0K5dO1VuO6VcvXqVqlWrcuHCBQB27NhBqVKl7nW4x7mMRUZG8u677wLw119/sWrVKnc/VBw/\nVu7FkSNHmDhxoqoOfP369TRVG0lC6Xpsn6Qz2mVMo9FonIJHmhfWr1+vZrgAnTt3xs/Pz0aJ0ocL\nFy4ok8LFixd57bXXABg/fnyq2xo5ciTHjh1Ts51kZrkexezZswEYMmSIi20yMjKSvHnz2iWWozh1\n6hSff/55io4NDAykQoUK6SyRczh8+DAXLlxg0aJFgGFu8fEx5qKvvvoqtWrVAtx7v3iUecG0L9Wq\nVYvQ0FC1f/ny5TRv3jwtTTr69WjlypU0a9ZMbZ89exaA/Pnzp/hCZrG9J554gnbt2jFz5kwAAgIC\nkjvNI8wLp06dokqVKoDxgBLijtjPP/88EyZMAHBXqSdHjxVAmY4+//xz6tSpo8bO5s2badGiBTly\n5AAgOjqaZ555BoAKFSpQvXp11Y/+/v5kz549pZd0fJ8kxZ49e5g4cSIACxcu5Pz58/c81jRRlSlT\nhjp16vDFF18AkCVLluQuoc0LGo1G4xjM4o/p+HEb27Ztk9u2bZMYTzoJSF9f3wdp0oq/P019cvbs\nWdmrVy8phJBCCPntt9+m+o/bs2ePLFiwoCxYsKAE5KxZs1J6qiP75G769u2r+gdQ381P7ty5Ze7c\nueWYMWNkbGysjI2NTctlTBw7VqSUMjo6WlauXFlWrlxZCiHkkiVLXH4/duyY+h4eHi7j4+NlfHx8\nav7+pHB0nyTkzz//lL169ZK9evWSuXLlctEhhQsXlp07d5adO3eWgwYNkr6+vtLX11dWr15dHVOo\nUCFZtGhR+fXXX8uvv/76gfrFkR10L95991357rvvunRY8+bNH6RJxw6arl27SkBWrVpVVq1aVUZH\nR6f6j/v6669VP73wwgupOdWRfWJy/Phxefz4cZkzZ06lYCtVqiSbNm0qmzZtmkj5FixYUEZERMiI\niIjUXMYJfXLffjEfJm3atFF/7+DBg+W1a9ce5G9NKY7sk4SYijZ//vwueqNx48aycePGsl+/fvL6\n9esu5zRo0EA2aNBA7tu3T9aoUUPWqFFD+vn5yeLFi6s+PnfuXJr7xaMW0tatW6e+mzaVUaO8s9ST\nEAIhBI8++ihwXxuS4vr166pPJk6cqOyc06dPTx9BbWDXrl2AsVhWr149wBgbN27cAOCHH37go48+\n4vDhwwCcOXOGtm3bArBixQp32XhtJzo6Wv1fL1u2TNn6BwwYQLZs2ewUzXZu3LjB6NGjmTrVKO4r\npaRAgQIA9O7dmwEDBgAkab++ePEiAHFxcYwYMQKAZ555huPHj7tFNm3T1Wg0GgvxmJnupk2b2Lx5\ns9o2n+SVK1e2SyRLCAkJAaBp06bkzp0bQEXfJWTt2rXq3y1btqj9HTt2TH8hLcb0YhFC0K9fP7Xf\ndBt88cUXmT9/PkeOHAGMWY45XlL6xuAJLF68mI8//hgwQnc3bNgAQK5cuewUyxGsXbuWTz/9FCkN\nR4dHH32UhQsXAvDUU08lOj4+Ph6AkydP0r17dwBatmypgpNMunXrBqDuxbTgMUp3+/btLttJKR5v\nom/fvqxZs4bTp08DxuuzOYCWLFmS6Hjzt4RuU48//rhXml9+/PFH9f3nn38G4Nlnn3U5ZseOHS7b\nNWrUAFBuU97Apk2b1PcqVapQuHBhG6VxFnFxcS4pATJnzszWrVsBmD9/Pvv371e/+fv7s2/fPgD2\n7dtHvnz5AMMslZCHH36YIUOGqPbSijYvaDQajYV4THBE165d+f777wFjar9nzx6AB326O9q5+/Ll\ny2rR6JdffmH06NGA8cTt0aOHy7Hma0/FihVd9pnBEKnE0cERc+fOBYwACPPvnTNnjhoTixYtYt68\neSoA5PLly2rxbMOGDZQrVy4t8jlurBQoUEAFRGTNmlXlnmjTpo0KdkhnHNcnJtevX+e///0vv/32\nG2DkZklK1/n6+hIXF3fPdszotPbt2/Pll19SqFChlMiXbL94hNLduHEj9erVU51WrFgxd60kOnbQ\npJajR48ChknBtHOvXLkyVdFrCXC00r106RJg/K1mEiAppYtppUmTJirqqFWrVhw8eBCAXr16MWnS\npLTI57ixYnq43E2mTJl49dVXqV69OmDYKUuWLEn58uXVMX/99RcANWvWfJCJi+P65G6uXLkCwMcf\nf8wff/wBQN68eSlatChgrA/8+eefyvRwN6YZc9SoUamx4ybfL07yqbsXixcvdvGxe/nll93RrJQe\n4GeYUnr06CF79OghhRBy5cqVcuXKlQ/SnEf0yW+//SZz5cqVyNn9//7v/1x8LwcNGqR+K168uDx8\n+LA8fPhwai/nuLHy9ttvJ/JJTu3n4Ycflp06dUptXzi2T9JCt27dXMZPzpw5Zc6cOeW0adNkXFyc\njIuLS22TycqvbboajUZjIR5hXjDtueb0fuXKlVSrVu2BBcMDXo9Swrx581Q2spw5c/L7778DULVq\n1bQ26WjzQkJWrVoFGAER5vj44IMPXLwUTPseGJ4fpv07lfZux42V+Ph4lfipS5cu3Lp1CzASAZku\nUClBCKGCAMzV+ZSempqD3Yjb7p/Ro0czZMgQ1XeAWjsyx0wa8Fyb7qlTpwAoWrQoUkqVcs5cMHED\nHj9owPBL/fbbbwEjzeUPP/zwoE16jNJNKXPmzAGMG8m0Ye7atSs10WkeM1ZWr17NrVu3GD58OADb\ntm277zlmxJ6Z4jCFeEyf3M20adMA6N+/P1FRUWp/hQoVlLth1qxZ09p8sv3iaD9d0w/RfDCYA0Pj\nyooVK1Q449tvv22zNM7EfBNYunSpUsATJkxg6NChdoqVLjz99NPAnXDpbdu2kTlzZl544QUAgoOD\n+eyzzwDc8YD2OLZt28Zbb70FoBSu6eny9ddfP4iyTRHapqvRaDQW4uiZrpl4AiBfvny8+eabNkrj\nTCZNmsSZM2d4+OGHgQey43o1pr/lwIEDWbx4MQDDhw/n+eefV9VuvY2mTZsC8N5773Hr1i1VNeTQ\noUMqbNzETKyUEVi2bBmRkZFqO3v27CxduhSAOnXqpL8A93NvcMMnzbRt21a2bdtWArJKlSry5s2b\n8ubNmw/S5N14vMtLpUqVpBBCvvDCCyp9Y2RkpIyMjJTh4eFpbdaj++R+jBkzRo4ZM0YCsn379jIm\nJkbGxMTc7zSPGyvm3/X888/f02Usc+bM8tlnn5XR0dFpSR/qUX1i3hdZsmRxcRF75ZVX0trkvUhW\nfm1e0Gg0GgtxrHnh1q1bKh8qGBmk3FxS26vw9TX+K2fPnq0WSSpUqJDWMGCvxswiNXnyZBYuXMih\nQ4cA1xBqb8Df3x8waqZFRUWxc+dOwKi1V7x4ccDoC9PLwZuJjo6mbNmyANy8eVPtr1SpUoqLdroL\nxypdHx8f5Yv7119/eU312vTCTNY8bdo0Xn75ZQDef/99O0VyLGZo9KpVqyhWrJhKj+itK/kPP/ww\nISEhzJo1CzAKVZqK1kzs7e2sWbPGpYK4ybhx4yyvJO5oP10zreGQIUOoWrUqr7/+utuE+heP9TM0\n2bBhA8OGDVMVFHr37k2ePHmAB8od63V+uveiadOmyjVx27ZtySXD8fixkg54TJ9UqlSJ3bt3q+2B\nAwcC8Mknn7hPqjt4fjXg2NhYpk6dSo4cOShRooRLPtWMzurVq9m6dSujRo1i/PjxKXKE92b279/P\n008/Te7cuSldurTyVMjIXL58mXbt2un7JwGhoaF89913+Pn58eKLL1p6bceaFwAeeeQRAG7fvk25\ncuXYsmULoaGhtGzZksqVKysbTUblt99+Y+bMmaxdu5Zq1aoREREBeFd1hNQQHx9P27Zt6dOnD6tW\nrWLt2rW0bt2aXbt2UbJkySTPmT9/PpUqVQLg8OHDaU376Gj69OmDn58f58+fz7D3j5mZDgyTSufO\nnfHx8WH79u1cv37dWmHu597ghs8Dce3aNZklSxaXrFDdunWTgwYNetCmpfQwl5e7qVWrlpw+fbq7\nmkuIR/bJ3r17ZUBAgMu+pk2byqFDh7qjeY8cK/r+MRg3bpxyERs/frzaP2TIkNRWyk4JycrvePPC\nwYMH8fX15fHHH1f7KlWqpPKBZlRu377Njh07OHfuHKVKlaJo0aK88cYbqn5YRkTKxKY+KSV79+61\nQRpnoO8f5+F4pRsdHZ2o0F6uXLlcklRkRM6ePcutW7dYsGABf/zxB7t27SIsLIyRI0faLZptBAYG\nUqBAAcaMGUNcXBwrV65k3bp1xMTE2C2abej7x6Bfv35qppkOC/KpwgrvhQdCCFEZ2CilzJFgX3+g\nvpQyw2bAEULkBi4B3aWUs//d1x4YLKV80lbhbEQIUQGYAJQHdgDngVgpZbCtgtmEvn+SRwjxIfCo\nlNKy1TTHz3SBg4CvEOLxBPsqARn6/UhKeQU4ZbccTkNKuVdK2UBKmV9K2Rx4HMjILh36/nEYjle6\nUsoYYCHwgRAimxCiNtAGmGWvZI7gW+ANIUR+IUQe4E1gmc0y2YoQ4gkhRNZ/x8rbQEFghs1i2Ya+\nf5JGCJFJCOEHZMJ4KGUVQmS633nuwPFK919eA7IB54DvgVellPvsFckRfIjxCn0QY+ayExhlq0T2\n0w2IAM4ADYEmUspbyZ/i9ej7JzFDgBjgHaDLv98HW3Fhx9t0NRqNxpvwlJmuRqPReAVa6Wo0Go2F\naKWr0Wg0FmJF7gUnG409JkuSxWSYLGOpQI+VxOg+SRrPzzKm0Wg03oJWuhqNRmMhWul6ELGxscTG\nxvLkk0/i4+ODj48Pzz77rN1iaTSaVKCVrkaj0ViIo5OYb9iwAYBatWpx4MABQkJCAPj5559p2bKl\nOq5mzZrUrVvXFhmtIjY2ln79+gGwa9cuhDBs9U8+mWFz22g0D8zw4cMZMWIEDRo0AOD3339P/4ve\nL+GuGz6p5urVq7JVq1bS399f+vv7y4ceekgGBAS41KpP+PH395f58uWT+fLlk/PmzUvNpTwmCfPo\n0aOlj4+P9PHxkU2aNJGbN2+WmzdvTktTKcEj+sRiPGas3M2lS5fk6tWr5YABA+SAAQMkIIUQUggh\ng4KC5IABA+SZM2fkmTNnUtu0x/aJSYMGDVx0ye+//+6OZpOV35Ez3XfeeUfNagGuX79O2bJlVeXS\nnDlzqt8vtzhTAAAgAElEQVRu377Nzz//rEpuvPTSS5QuXRrwrpLaZikegMaNG1OjRg0bpdE4nVu3\nbjF27FgAJkyY4DJ+hBDqTWn+/PkAXLhwAYDp06dbLKm9rF27NtG2OetNL7RNV6PRaKzkflNhN3xS\nzJ49e+SePXtkvnz5JCCLFCkiixQpIn///Xd58uRJGRUVJaOiolzOiY+Pl8OGDVOv3oBs166dbNeu\nnbx06dIDvQY4oU9MgoODpZ+fn/Tz85M7d+5MSxOpwSP65G5CQ0NlaGiobNmypRRCqFdG8zU6KChI\nHjp0SEZHR8vo6Gi5bNkyGRMTk9LmPWasSCnl+PHj1T1x96dhw4b3/C2VeFSfJPkH3GWqdFezyX2s\nyDKW4gts2bIFMBbGhBB8+eWXACkqr/Hee+8BMGbMGG7dMjL5LVu2jFatWiV3muMjak6fPg1A4cKF\nqVWrFgAbN25MH6nu4DERaeb/9bp16+jZsydwxxRjjm3zVRqgW7dunDhxAjBeJb/77ju6du2akks5\nfqyY7N27l6efflqZDEw++eQTAPr27cvQoUMBGD16tMsx8fHxqbmUx/TJvUg4NiDpOntpaTa5H7V5\nQaPRaCzEUQtpCSvZ9uzZM1UF5EaNMnJ3z5kzh2PHjgGwcOHC+810Hc+DFJrcvHkzp07dqehTqVIl\ntcjoLYSGhgLwzDPPqH2PPPIIEyZMIFu2bGpfeHg4ANmyZeONN94AIGvWrBQqVMhCadMXs+rxe++9\nx/nz59UsrlixYixdupRy5coB4OPjwwcffABAu3btaNOmDefPnweMxefdu3fbIL09DBs2jBEjRqjt\n4cOHM3z48HS9pqOU7vvvv6++V69ePU1tNGvWjK+//hq4Y67wZH7++Wf1/eWXX77v8b1791bnXL58\n2aUSbs6cOenfvz/g2teeyt69e2nTpo3abty4MQAfffQRVatWdTnWNNO0bduWK1euADBw4ECefvpp\ni6RNf8LCwgAICQlBSknmzJkBeO2116hQoYLLseZvTz31FD179lSeDnv27KFXr14ATJkyxSrRbSOh\nwrUKxyjdo0eP8s8//wCQO3dunnjiiTS106hRI6V0PZ2YmBhlsyxcuLCyWSYkLi6O0NBQFQ585swZ\nZZfKnz+/UkShoaGcOHGCyZMnA9C9e3eKFStmwV+RfowcOVLN0Fq1aqUUR6lSpRIda84CzZkxGA9o\nb2LFihXAHTul6fr01ltvJXvexx9/rM7ds2cP27dvTz8hNdqmq9FoNFbimJnu7NmzOXr0KAAdOnRQ\nK/UZmWnTpnH27FkAXnnlFZffzNflKVOm8OGHH6r9jz76KN26dQOgT58+FC5cWP3Wpk0bZXqIiIjw\n2JlucHAwAHPnziVHjhyAMVtLaoYLhofDRx99BBir0+YMsH79+ukvrEVcvHiRrVu3uuwzx0FKMI8d\nOHCgW+XSJMYxSvfHH38kd+7cgOHSorljo4PEr8zmAtukSZMQQijb5Lhx4xLZ70xKliyZTpJay44d\nOwDjNTp79uwAapEoIaZp5v3332f9+vXqHNNdypvYuXMnx48fV9v16tVzyU+SGkybd0REhFctNDoF\nbV7QaDQaC3HMTBcgMDAQgDp16tgsiTMwTQh3c/DgQebMmaO2e/XqxRdffAFAlixZkm3TzEp29+q+\nt3H8+HG++uorALXABoY7WeXKle0SK90wZ/8mI0aMIE+ePGlqywwe2bt3b4ab6aa3uxg4QOleu3YN\nMFbhNa5ERkYqT4SEkTLjx49Xr4BdunRJsbdGdHQ0vr7Gf/n9lLOTKVu2LAC7d+/m0qVLAFSpUsXl\nmPPnz6uHVsKoo6efflqZsbyJmJgYlzGSVnu1BRGqGR7ble5PP/0EwOHDh8mXL98Dt7d06VL13fRF\n9FQSZoNKqDhOnz6ttu81G76b06dPM23aNJ577jn3C2ox33zzDQBRUVFqYTAph35zLMyaNUtl03r1\n1VctktJaduzYkSikNS0kHHOa9EHbdDUajcZCbJ/puoudO3cCRpIbk//97392iZOuTJkyhU2bNgGw\nadMmRo0apVzK8ubNm+Q57du3J1u2bPd1lPcE/P39AeP/2syHmtCmWa5cOVq0aEGfPn0AmDdvHmXK\nlAHg8ccft1ZYDyQgIAC491jSPBheoXR37typFkuuXLmiFuI8OeLo9OnTLomnE5I3b14VWdWmTRve\nf/99fv31V8AIATVvmpCQEOVaFhYWxpAhQ7wu+bnpc5tU4ulJkyYBxitztWrVACNKT5OY7777Tn03\nF5O8fbEVjHGTMJF5hsi9ULx4ccC1GkRqiI+PZ8yYMWo1v3DhwowZMwZALRp5Io888gilS5dWiVrW\nrFmjZrPZsmVTq8rbt28nJCRELS5duXJFzWanTZumkr4MGTLEK/ItpJSEPqsBAQG8+eab9gljAR9/\n/DG7du0CjEXEF198MVVVIMxw6gIFCnit3dspaJuuRqPRWIjtU8FGjRoBxszu6tWrgFGvKTlPht27\ndysfzNDQUJcEHbNnz05zhjKn8c0336ioop9//pmmTZsC0L9/fxf/ya1bt6rUllu3blVuP2XKlFH7\n27VrZ6XotmOmLgQjGY63vypXrlyZTz/9FIAePXowd+5clRr1fn97cHCwCjcPCgrCz88vfYV1AKZJ\n4e4aaVZgu9JNyL59+wAjN2pyTtlbt251yYqfP39+WrduDaBsd95A4cKF+eWXXwBo2LAhmzdvBqBj\nx47qGCllIhefF154ATCqAmTExZC9e/eycOFCte3Jtv3UULt2bQD++9//8sMPP7Bu3TogeaW7Zs0a\nFi5cyMMPPwzglSHSSWFHSkcTbV7QaDQaC3HMTHfUqFEqW1bCnKf3wsfHeF7kzZuX/v378+6776ar\nfHZhzvi3bNniEkgydepUwCg5b/aFuW2GU2dUwsLCiIyMVG8AGeF1GeCxxx4DjGRIf/zxh5rNnT9/\nXpmZwAgj37ZtG2CYqq5cucLbb78NJJ04yNtYu3ZtIrPC77//DiTtBeNuHFWY0oyuatasGXv27Lnn\ncb169VJhnw+40urxhfXSCY8pTJkU48aNY8CAASrb2p9//umOZj1qrERERKh7Y926dZQoUUJtDx06\n1MU817p1a+VymUo/Zo/qE5O1a9fSsGFDwCjXkw4uYsn2i6OUrg145KCxAI9WulWqVGH37t2q+q05\ni3tAPG6smAvTBw4c4MMPP2T58uWAa38899xzVK1aNa3ulR7XJxbh+dWAGzRogL+/Pzlz5iQgIED5\npGZk9u/fr5K3lC5dmsWLF9stku0EBASQM2dOdu/eze3bt3nnnXdYsmSJ3WLZzrBhw/jtt9+4ffs2\nt2/fzlCFJ+/HoUOH8Pf3p3v37tZdVEqZ3p8HpkGDBnL69OnuaOpurPj73d4ncXFxsnTp0vLzzz+X\nt2/flmvWrJHZs2eXhw4detCmTTyuTxLSuXNnKYSQWbNmlX369HFXsx45VqSU8u+//5Y3b96UUkp5\n4MABWbBgQRkaGuqOpj22T0yaNm0q69WrJ7t16+bOZpOV3yNmuqBTziVk//79RERE0LdvX4QQNGzY\nkNq1azNr1iy7RXMEzZs3p0SJEuTNm5egoCC7xbGdsmXLqox7UhouhkeOHLFZKvuZM2cOefLksbwi\ntMco3UGDBlGgQAHq1q2r/A8zKkk9gKSUquKtxgiweeaZZ+wWwzG89tprZM+enbJly/LII4/QokUL\nu0WylcjISIYNG8bYsWMtn9B5hNIdPXq0KtEeHBxM69atOXbsmN1i2UZgYCAFChRgzJgxxMXFsXLl\nStatW0dMTIzdojmC+vXrExMTw7Bhw7yq+OSDMHHiRKKjo9m4cSPt27cna9asdotkK0OHDiU4OJhH\nH33U8mtb4b3gdoQQK4AQKeVEu2WxCyFEBWACUB7YAZwHYqWUwbYK5gCEEEOAp6WUDe2WxYkIIb4G\n/pJSTrBbFjsQQlQGZgOVpZRxQohhwONSSktW0xwTHJFKJPa5qzgCKeVeoIG5LYT4A5hhlzwOoxsw\n6r5HZVx8gYycWLg+UAw4IYwImhxAJiFEOSnlf9L74o43LwghcgkhmgohsgohMgkhugB1gV/tls1O\nhBBP/Nsn2YQQbwMF0UoXIUQt4BFgvt2yOAEhRH4hRCchRHYhhI8Q4hngeWC13bLZyGSMh05loBIw\nCQgBmlpxcU+Y6WYGRgJlgHhgP9BWSnnIVqnspxvwMsb/4QagiZTylr0iOYLuwAIp5TW7BXEIEugN\nfI0xyQoH+kopQ2yVykaklDeAG+a2ECIauCGlvGTF9T3SpqvRaDSeiuPNCxqNRuNNaKWr0Wg0FqKV\nrkaj0ViIFQtpTjYa6yxJSePRWcbSCT1WEqP7JGk8P8uYRqPReAta6Wo0Go2FaKWr0Wg0FqKVrkaj\n0ViIJ0SksXPnThYtWsSCBQsAo/yIGdQhhODJJ59U1SQGDRqkK0toMgzR0dEAnDx5kq+//lrtf/HF\nF6lcubJdYmmSwVE10qZMmQIYSbo3bNig9u/cuRMhhIui7dWrFwDt2rWjadM0h0zr1dek0d4LiXHc\nWImOjubTTz8FUJW0TXx9fenUqRMAX3zxBQ899FB6yOa4PnEnzz//PK1atQKga9euqTlVey9oNBqN\nU3DUTNfIsmb8my1bNmUmqFu3LoGBgeTLlw+A9u3bu0s2j3xSr127loULFwIwf/58IiIiVEn6oKAg\n3n333QeVT890E+O4sfLee+/x8ccf37eBggULMmPGjAd5I7wXjusTd3D79m3A6LfXX38dMJKep4Jk\n+8VRNl1TmS5evJiyZcuyfft2myVyDmfOnKFdu3YAbNu2TZlaihQpQpkyZTh58iQAgwcPplixYgB0\n7tzZHmHdyPLly9XfffPmTZff/P39adu2rdo2/+6+ffuydetW9ZCuU6eORdJaS4kSJdR3IYRSEOXL\nl+fmzZtKUZw5c4a2bdvyzjvvADBw4ECyZctmvcAeQlhYGADnz59Pl/YdpXQnTZoEQGhoKOHh4Zw4\ncQKAokWL2imWrVy4cAGAFi1asGvXLsBQLpMnTwagevXq5MqVSyndNm3aMG/ePAA6deqkvlepUoVS\npUqptwlP4cSJE4mUrcn169eZM2dOov2fffYZN2/exMfHsJ5Vr16djh07AlCuXDmKFy8OQJkyZdJH\naItYtGiR+h4UFMQXX3zh8nulSpUAYzJz8eJFPvjgAwCOHDnC9OnTAVTBSm/n4MGDvP322wCMHz9e\nPaDvxxNPPOF2WbRNV6PRaCzEUTPd/PnzAxAcHMyQIUPULC8jz3TN1eldu3apInoHDhwgS5YsLscV\nKVIEMGy8ZtHB5cuXu5gYrl27hr+/vxViu42XXnpJzcYOHz7sMhauX7/O0qVLE52zb98+zp07p2xz\nmzdvZvPmzep3Pz8/wHjNHjFiRHqKn66sWLFCvbkMHjw40e9169YFYMmSJQwaNEh5BH3//ffKPDVj\nxgx8fR2lBtKFLVu2sGzZMgB69OiR7Ez38OHD6nt6FK501EKayciRI3n//feZNWsWQCK/27Jly7rL\nJuXohYA5c+bQrVs3AHLnzq0qIOfIkSPZ8/7++28AateuzdWrVwF49tlnWbBgQUrNCx69kLZnzx5+\n++03tf3jjz+yY8eORMflypWL8PBwcuXKlZJmHTdWmjRpwurVRtWdo0ePKrNJUmzdulWVXb98+bLa\nP2fOHIKCgtIqm+P65F707NmTmTNnAsZDuEaNGvc8tlGjRoBh2zXtuql8MGmXMY1Go3EKjnqvMJ8q\n33zzDUIIevToAYCU0iU4ol27dnTp0gVwq/uY49i9ezfx8fGAsSJ9vxmuSeHChRPtCwgI8LhFtLTy\nxBNPuCyA9OnTh3/++QeAjz/+mGnTpgFw9epVxo4dqxaYPI2yZcuqmW5Cpk2bxg8//MArr7zisv+/\n//0vABMnTlT7Dh48mL5C2kxUVBQAq1evVsEiTz31VLLnxMXFAeDj45MuphfHKN3z589Tr149AMLD\nw11Ce02Xn6lTpwKGd4PppyqEYPv27epYb3KFOXLkiPo+cODAFJ/3669GoeQbN1TtPbV6nxHx8/Pj\n8ceNiuPvvPOOUro5c+akZ8+eNkr2YPznP3eqhe/evVv9f7/++uvcvHmTtWvX3reNb775hsDAQJo0\naQKQUlOLx2Ca2k6dOqWUrenVkhRXrlxh3759AOnh12xcP11a1Wg0Gk2SOGamu3//fg4cOADAc889\np/xLE2LmW7hw4QKzZ88GjECKatWqUa5cOQDmzZvnFQlvYmJiXPwwU7qKevPmTd577z0AYmNjCQgI\nANLH39ATWbJkifoeGRnJ/PnzU/UW4SSeffZZvvvuO8BY/Dl79ixgzOzv5dt8N+Hh4QQFBak3xKlT\np6qAE294a9y4caP63qBBg/se/9NPPymvKfPN29040nshtUyZMkWZHsLDw1mxYgUATz755P1Odezq\na0xMjIsNNywsTDm7J8WtW7cAw3ZlrlKDEZ0FRsBAKvBo74WkOHr0KGAEDJiZuXLlysWxY8fIkydP\nSppw7Fi5m5CQEObNm8elS5cA+Pnnn1N1vvmA/v7776lQoUJyhzq6T2JjY9UE7OrVq8p7YdGiReoB\nFRMTw7p16+40LKVaOxo7diz9+/dPi3zJ9otXKF3A5elkDravvvrqfgttjh00N2/eJDAwkOPHjwPw\nySefMGDAgCSPjYiIUO51d+ddMEMak1PYSeB1SteM1nrzzTfVvrffflv5QacAx46Ve2EuwpqLSaai\nEUJQoEABddywYcNUhNq1a9fU/iZNmvDJJ58klyLS0X1y9epVcufOnWi/j4+PejO+281u1apVyjbu\n5+enIj+7d++eGvm0y5hGo9E4Ba+Z6ZqsX7+et956CzBMDe+9957L7OYuHP2kPnXqlHoiR0VF0bhx\nY8CweZurslFRUaxfv17NYjJnzsyVK1cAI0dDaGgoQGrzqXrVTPfQoUNUrVoVMHLQZs+eHYAdO3YQ\nGBiY0mYcPVYScuHCBQ4ePEitWrVSfM6mTZsA6N27N3v27FH7n3nmGWWuSwJH98mNGzfUG965c+fU\nWkePHj1cZvoJKVq0qMpjEhAQoMwrZv+kkIxhXkhIQlPDgQMH1GtWEjh60AAqzPV///ufS9Y1Mwy4\nRIkS1KpVS/lgtmrVitjYWMCIwjFfG1OJVyhdcxy8/PLLLgto48aNA6Bfv36pac7xY8UMc+3bty8R\nEREqGVDCTGz3IyoqSj2gjhw5Qs6cOVU7zZo1u/twx/dJZGQkYPjeJjfxMP24AwMDKVmyJAAzZ85U\nYfOlSpVKjXzavKDRaDROwTEuY+7EzKNat25d9u/fb7M0D0abNm0AaN68OTt37lT7zZmuOSsxI4vM\nWS5Ahw4drBLTkXz00UeAq5vYY489pjw6vA1zwSwiIoLY2Fi1iLxx40Zq1qyZojYCAgL44YcfAKhV\nqxaRkZF88sknQJIzXceTM2fOFB33yy+/AIb5qWXLlgBUrFgxXWTySqVrRpQsXrxY2UQ9ncyZMyeb\npOPUqVOJ9iV3vLczZ84cFzc50/1u8eLFyUYkeTKmien06dMMHDhQuT4lY15Lkt27dwN3Kiikl/Jx\nEgmTADVs2DBdr+Wdo0+j0WgciqNmuubMJH/+/KmtvqkIDw9XuUWvXbvm4vjszcyfP99uERzDunXr\neOWVV0i4SDxjxgwgY0Tm9erVixUrVvD7778Dho9pgwYNlA936dKlE51j+jFPmzZN5ZO1YJHdkdyd\nq9rdOEbpLly4ULl6vfLKK/dVumZGMjNU1vw3NDRUJUOfNWtWalyCPJYTJ07w448/qu369esDKbdn\neQumq1yrVq1U1BkYCWBM23hGIGfOnCxZskSZBSIiIpgxY4YKoEnKvGJGNN7NU089ldqijJr74Bil\nC3eerJMnT2bBggVqIUBKyf79+8mbNy9g2OXMY82Uj2a4X5cuXZQ/nrmg5u0cPnxYJSuHOy5CGaEi\ngMnt27dVmKepcM0sXOPGjcswtcBMcuTIoUKfZ86cyZw5c5T/7enTp5M9t3bt2oDhoxscHKzuO28m\noR+umQPGrLzhbrRNV6PRaCzEMVOh9u3bK7eNxYsXA3dMBufOnaNcuXIqCfcrr7yiZrFmeW7TjOAN\nmZFSS8JS0dmyZeONN96wURp72LJlS6LIQ7PkeEab5d5Njx496NGjB2fOnAEM1zIzQVSDBg3YsWOH\nsvM++eSTqg6dWWsvI2C62wEpTYCUdsysOun4eWD27dsnGzVqJHPlyiVLlSolFy1a5I5mpUz/vz3d\n+sTkxx9/lAEBARIjSkeWL1/eHc16TJ9cvXpVXr16VebJk0f1ASD9/f1l9uzZZcmSJeXGjRvT2nxC\nPHas1K9fX/r5+cmAgACZI0cOGRgY6I5mpfTgPjE5ePCg9PPzk08++aQcM2aMbNu2rbx27Zq8du3a\ngzSbrPyONy/Ex8fTtm1b2rRpw+XLl5k8eTJdu3Z1qdiZUfntt98YNGgQTz31lHoLyEizk+QoU6YM\nV65cYc2aNTz22GN2i2MrQgi++uorIiMjiYqKUn7sGmOR9X7le9yN45Xu/v37iYiIoG/fvgghaNiw\nIbVr11YrsRmZ4cOHM3ToUFUnSwhB9erVbZbKWlatWsWqVatcnNtLlSpFSEgIvr6+FClShEKFCtko\noTOQMmO6fyXHnDlzyJMnD08//TTlypXjrbfeYvHixWTLli1dzZSOV7pJDRYpJXv37rVBGudw+/Zt\nduzYwblz5yhZsiSFCxfmtddeS22yco+nfPnylC9fnoIFC1K3bl2EEHTo0IEGDRpQtGhR3njjDZfQ\n6IzKoEGDKFCgAHXr1s0wvuvJERkZybBhwxg7dqzlDyTHK93AwEAKFCjAmDFjiIuLY+XKlaxbt46Y\nmBi7RbOVs2fPcuvWLRYsWMAff/zBrl27CAsLY+TIkXaLZhs3b95ESsny5ct1nyRg9OjRHD16lH/+\n+Yfg4GBat27NsWPH7BbLVoYOHUpwcHCKy2C5lfsZfd3weWD27Nkj69evL/PlyyebNWsmu3XrJl9+\n+WV3NO2xCwGXL1+WQgg5a9YstW/BggWyatWq7mhe90liPHas3E2zZs3khAkT3NGUR/ZJWFiYLF++\nvLx165aUUsrhw4fLbt26PWizCUlWfivy6bodIcQfwAwp5VS7ZbETIcQJ4D0p5ex/t9sDg6WU9y0O\n563oPrk/QojlwHIp5QS7ZbEDIURfYCQQhZH7NgeQCfhbSvmf5M51B443LwAIIZ4QQmQVQmQTQrwN\nFARm2CyWE/gWeEMIkV8IkQd4E1hms0x2o/skAUKIXEKIpv/eP5mEEF2AusCvdstmI5OBx4HKQCVg\nEhACNLXi4o4JjrgP3YCXMeTdADSRUiYdLJ6x+BDIBxwErgM/AaNslch+dJ+4khljVlcGiAf2A22l\nlIdslcpGpJQ3gBvmthAiGrghpbxkxfU90ryg0Wg0nopHmBc0Go3GW9BKV6PRaCxEK12NRqOxECsW\n0pxsNHZ8CWmb8IoS7G5Gj5XE6D5JGl2CXaPRaJyCVroajUZjIVrpeihhYWGEhYXRqlUrfHx8yJEj\nBzly5CA0NNRu0Sxj3LhxCCEQQrB582a7xdFoUoRWuhqNRmMhVgRHONno7bELAc2aNQOMROaAqoDc\nuHFjZs+e/aDNe8RCWtGiRTl58iRgFBasWbOm24VKgMeOlXRE90nSJNsvnhIGrEnAmjVrXMwIAwYM\n4MUXXwTg0iVLIhltxVS0J0+eZOzYsQDprXC9DvNhvWDBAubOneuSBD4hPj4+bN26FbhTXdnTqVmz\nJp9++ikAderUsfz62ryg0Wg0FuKRM93w8HC+/PJLAHbs2MHEiROpUKGCzVKlPxcvXgSgY8eOXLly\nBYDWrVszcuRIfH098r8yTcybN099L1y4sI2SeA4//fQTy5YZydaWL1+uxo+UklKlSvHyyy8DUL16\ndXUvjR07lqlTp7JgwQLA82e65tvhnj17eOihh2yTw2Pu1IMHDzJhgpH+87vvvuPq1avqt2bNmhES\nEgIYr5zFihUDoGLFitYLmo6YK/TmDQPw7rvvZiiFC65KV5sV7s3AgQMBGD9+PLGxsZjrN2XKlOGZ\nZ54BoF+/flSpUiXJMvVPPfUUhw4d8orKG7dv3+add94BIEuWLGoN5G4GDRqkHi7PPfdcusji6Lv1\n9u3bAPz99980adKEM2fOJHncP//8Q/369QGj9pF5I27cuBEfH++xoJi1raSUPPvsswDUqFHDTpEs\n5+TJk2zZsgWAIkWKUKRIEZslci4zZ84E4MaNGwQFBfH2228DUKlSJbJkyXLf8xs2bEjXrl3JlClT\nusppBXPnzuXUqVMA7N27955KN3PmzMyZMwdIP6XrPRpJo9FoPADHznTPnz/P+PHjAfjwww9dfsud\nOzeRkZHAndmwuQ1G2XbzN2+Z6Z47d44VK1YARqn13r172yyRPSSsdtyhQ4cUnbN582bl8QComXLH\njh292jxRu3ZtABYtWkTLli2pVq1aqs5//PHH00MsW9i3bx89evQA4JFHHrnncaVLl2bx4sXpKotj\nle7gwYOZOvVOCbQsWbLwxRdfAFCiRAmGDx8O3LmBTPLnz8+SJUsAvMrW+d133/H3338DkDNnTvLm\nzWuzRPZgviLC/U0rpg28U6dOLkrX5LPPPrPCv9cWDh48yC+//AIY90u7du1slsgejh8/DsBXX33F\n4MGDU3SOWSn5xIkTFC1a1O0yecc0UKPRaDwER00FTVNBhw4dWLJkiTINVKxYkWnTpimH7jfffFOZ\nEO6matWqXjlzMWe5YMxcqlataqM0noFpikg4yx07dqwaH506daJ///5embdh0qRJXL9+HTC8ewIC\nAmyWyB7mz58PQI4cOejSpct9j1+yZAnx8fGA8VaVHjNdRyld0/d20aJFAAQGBgKGW1SdOnW4cePG\nPdWXwzAAAAevSURBVM8tXbo0AJMnT05nKe3BtOcCGdaemxo2b97s4lr2008/ARAUFKT21ahRw+UY\nb8JUuHDn3shonDhxQq0HDRgw4J4eC3BnHWjJkiX07dsXgFq1aqWLXNq8oNFoNBbimJnurVu3+OST\nT1z2mU+f559/HkBFkbzxxhusWrUKgD/++ANA5R4wAyO8EdO5fenSpRw+fBgwzA7Lly9XvwkhVB8M\nHTqU7t27A3iFr2VqSOjl8NNPP7nMcO9m7ty5AMke42n8/PPPyqRg+nRnJOLj45k+fbq6L/r06ZPs\n8YUKFQIgb9686b4A7xil6+PjQ4kSJQBUEIS/vz8AWbNm5fXXX6d///6AYaNLqKBr1KiRIV65hTCS\nF61YscLF3CCEoHz58oChhE+cOAHAyy+/zIULFwDj9cobSBj2m9CTISEnT55k3rx5yrvBm5RpSoiO\njiYyMpIyZcqo7b1797ocU7JkSQD8/Pwsl88KoqKiGDFiBC1btgS4b9hvrly5ACMgJL1xVGpHM7w1\nJCQEX19fKleuDNyx7UZHRwPQvXt3ZfcNCAhgx44dabVbeUxqukKFCnH27FnAcBkzF4O6d+9Ovnz5\nqFevHgDr169nypQpACxcuFCdP2fOHDp27JjSyzk2tePJkyddFjeSGr9BQUHMmzdPZSAzH9Z3U7Nm\nTf755x/1kLoPHjNWVq5cqcJ870WVKlUAI+y1devWaVW+ju2T69ev07x5c7UA/cILL6gHTbt27ciX\nL1+S53322WdqXejXX391+a1IkSIp9fvXNdI0Go3GKTjGvABGpBlA165dk/zdzHZkznLBcPvJCKuz\nPXv2VCaVTp063dNLo0mTJuq1eu/evRw8eBAwMrN5A0WKFFF/35YtWxg3bhyQ9Gz2XhnITBvuli1b\n1GzYmyhRogTPPPOMemU2I8tMc0xoaChhYWGA8VbQuXNnpk+fDniPucHf359ffvlF6Yr9+/fz3Xff\nATB69GiXv1NKqUx3V65cUf1UvHhx9UYQFBREv379yJo16wPL5iilmxyXLl1yuUHMV8yJEyfaJZKl\nJIxA27ZtW7LHmgsoderUUUrXm+jXrx9gPHzeeustwFCg5n7TDSwpf+25c+eqxC9FihRJjcnFYyhV\nqpSKRkuK69evq9fuDz74gB9//JGyZcsC8P7771sioxX4+fnRuXNntT1ixAgAYmJiuHXrFmCE12/f\nvl0dM2HCBJXBcNeuXSqhUlJZ2NKKNi9oNBqNhXjMTLdVq1bs2bNHbQ8dOhQgRSnqvIFs2bKpRaO4\nuDhiY2MBknzdMV8dly5dmuRCk6djeiOcOnWKzz//HDBmt3cHOpiRZps3b1aRSQmP2bRpU4ZMDenv\n70/BggWBO7kJkgsc8DayZcumvufKlYtSpUqp7ZCQEOWO+dBDD7l1hquQUqb354E5cuSIDAgIkBir\nlrJVq1YyLi5OxsXFPWjTVvz9buuTnj17yp49e0ofHx8ZHBwsg4ODZUxMjMsx4eHhsmbNmrJmzZrS\nx8dHfWbMmJGaS3lMn5w4cUKeOHFC9uvXT42P5D41atRQ56QSjxorybF27VpZsWJFWbFiRQnIunXr\nysjISBkZGZnaprymT0w6d+4s+/btK/v27fsgzSQrvzYvaDQajYU4yk/3bv755x/AWBA6fvy4Wjzb\nsGGDuxJRONbPMCnMiq0VK1bk9OnTALz00kt06NCBa9euAUa0XkREBGDkDX3hhRcAY8EkFTjWTzcl\nJCzNDsbCmxsCJTxqrADcvHkTMExwMTExaiFp4sSJarzUqFGD8ePHp7X+mcf1yf2YPXu28uRYs2ZN\nWpvx3BLsZiE50+5khvqmR+YfTyBPnjyAkZSjbdu2AHzzzTd88803mA9PIQSNGzcG4KOPPsqQ2cg6\ndOjgEgZ86tQpZdMtUqSIV2ahS0hUVBTLly/nyJEjAERERBASEqLuIz8/P5WPun///hk2A9m9MKP3\njh49ymOPPeb29h2rdLdt26byBoCxYNSiRQsbJXIOVatWVZVd33//fZYvX06DBg0AaN68ucqSlFEW\nGe/G9N01FS3cSXjurdWD4+PjlUvlkCFDCAwMVO6CsbGx+Pj4qD746quvlP+pxpVmzZrRrVs3wHjT\nTg+lq226Go1GYyGOtOleu3aNDh06uDh4FyxYUNlYTEduN+B1Nik34dE23XTC0WNl4MCBfPrppy77\nTHenKlWqMGLECJo1a+Zu2RzdJ2khLi5OmeSuXbumTDSpxPNsulOmTEmkcFesWOFOZavReBW1a9dW\ni4fh4eH07dtXpXR0R+hqRsHX15devXoBd9LGuhttXtBoNBoLcaR54csvv2TYsGEqlj44OFglGXYz\nXvd65Ca0eSExeqwkRvdJ0iTbL45UuhaiB03SaKWbGD1WEqP7JGl0Pl2NRqNxClrpajQajYVYYV7Q\naDQazb/oma5Go9FYiFa6Go1GYyFa6Wo0Go2FaKWr0Wg0FqKVrkaj0ViIVroajUZjIVrpajQajYVo\npavRaDQWopWuRqPRWIhWuhqNRmMhWulqNBqNhWilq9FoNBaila5Go9FYiFa6Go1GYyFa6Wo0Go2F\naKWr0Wg0FqKVrkaj0ViIVroajUZjIVrpajQajYVopavRaDQW8v+EIX2thTZ9IgAAAABJRU5ErkJg\ngg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# 画像と認識結果を一緒に表示\n", "for index in range(25):\n", " plt.subplot(5, 5, index + 1)\n", " plt.axis('off')\n", " plt.imshow(test_set_x[index].reshape(28, 28), cmap=plt.cm.gray_r, interpolation='nearest')\n", " plt.title('%i' % predicted_values[index].argmax())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

重みWのパターン

\n", "\t

\n", "\t\t最後に各クラスを識別した重みWがどのように形に求まったのかみてみましょう。\n", "\t

\n", "\t

\n", "\t\t赤い部分で正で、葵部分が負の値です。各数字の特徴的な部分に赤の部分が見られることが分かります。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAADeCAYAAADy3YFwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztfXmYXUW1/aqeu9PdGUggYAJNGAyEUcAHgShhEMGnT3wg\nODCon3FA8lMxPhXCKKBGfT4Qh6gIgj4CKAhqVJBE0cgDEUhoDARCQ8IUQobuTk/p7vP7o9a6p27l\nNumQ27l9kr2+r7/dZ7h16tSpU2ftXXvvckmSwGAwGAzZQlmpK2AwGAyGLYcN3gaDwZBB2OBtMBgM\nGYQN3gaDwZBB2OBtMBgMGYQN3gaDwZBB2OBtMBgMGcR2NXg750Y75+5wzrU75551zn2g1HUqNZxz\n5znnHnLOdTnnri91fUoN51yVc+7HzrkW59x659zDzrl3lrpewwHOuZuccy+yXZY65z5W6joNFzjn\n9nHOdTrnflbquggVpa5AkfE9AF0AxgF4C4DfOuceTZLkX6WtVknxAoArAJwEoLbEdRkOqADwPIBp\nSZKscM69C8CtzrkDkiR5vsR1KzWuAvDRJEk2Ouf2BfBn59w/kyR5pNQVGwb4LoAHS12JENsN83bO\n1QF4H4CLkiTpTJLkbwDuAnBWaWtWWiRJcmeSJHcBWFPqugwHJEnSkSTJ5UmSrOD2bwE8C+Cw0tas\n9EiS5F9JkmzkpgOQANirhFUaFnDOnQlgLYA/lbouIbabwRvAvgB6kyR5Jtj3GIApJaqPIQNwzu0C\nYB8AzaWuy3CAc+4659wGAP8C8CKA35W4SiWFc64RwGUALoD/oA0bbE+Ddz2A9dG+9QAaSlAXQwbg\nnKsAcDOAG5IkearU9RkOSJLkPPh36RgAvwLQXdoalRyXA/hRkiQvlLoiMbanwbsdQGO0rxFAWwnq\nYhjmcM45+IG7G8D5Ja7OsELisQjARACfKnV9SgXn3CEATgDwnVLXpRC2pwnLpwBUOOf2CkwnB8PU\nYUNh/ATAWACnJEnSV+rKDFNUYMe2eb8dwB4AnufHvh5AuXNu/yRJDi9t1bYj5p0kSQe8mne5c67O\nOXc0gPcAuKm0NSstnHPlzrkaAOXwH7dq51x5qetVSjjnfgBgMoD3JEnSU+r6DAc458Y5585wzo1w\nzpU5504CcCaG2STdNsYP4T9eh8ATwR8A+A2Ad5SyUsJ2M3gT5wGoA7AKwM8BfHIHdxMEgIsAdAD4\nLwAf4v8XlrRGJYRzbncAM+BfyFecc23OuVaLCUACbyJZAe+Z9A0A/y9Jkt+UtFYlRJIkXUmSrNIf\nvGm2K0mSYeG55WwxBoPBYMgetjfmbTAYDDsEbPA2GAyGDMIGb4PBYMgghtxV8Ey4HcaofguSQUVg\nuduu3WHaJDn9/MG1SfOO00+SKYPrJ8CO0y5b1CZLd4w2AYBk8sDtsj35eRt2dMhbu59SeuVgHCNH\nUCoe9+ViVcowrBB79CuTS2yDUJ8pFAGwIdqOQwMHQpEddM1sYjAYDBlEpph3a7StD94KyvUDnFfo\nmH47kXJkdP7I6LzsofJ1jsXUdHPnZSQAUdVV2E0NparfFO1vCX77WvRb/UYMXKxJzLyKsizaDpGR\nZtuknv0DHI+Z40BsNQsQ49a9iU3Hr432h0k2dJ810bG4bwz0ChapnYx5GwwGQwYxrJm3WLJY8yId\n6KIUW9KXT5+ikC7ra8iv4FR+FReRUTUPxCrGeDElYFS7Fyh+20MV1c1yfYWERlunG1IjrfWid2la\nRAezfDYqH9PuyIfKWBXtr4qOlxhqijpKMqEZO3s5mx1ot+Ve9vCZd+7k5c37pkXNbOE/alb1LTG0\ndZRiU3F/GSZNkqtXGPQfM+l4LoDtMJXtt0gvHn83dZSXn+Duc1Q22zXX1eLrFLrWtkb8XLSte1Dd\nX462dV7t570sf1NaRi8TUK5p8bK/g2Xc7yX7l8aQ3LawMfh/K9rFmLfBYDBkECVn3qF9Ok7G/bjO\nkd1J7EdfRVLgRn4Un+TxEcFscDUN4q27eLm618uPjfdykc6NCS1xdPC/7ODbdq2suGL6bJMiJGO9\ndKISbISuu/izP3gZzpDL0N+uHWTYZTO9jMm7tt1bvKx9V1pWGWnZJgbmIYSahJeeyGc5i7s/tdjL\nfioKZXxgNf/Py6TFy/NVZQDnM/3S0wd5OZPKyPy/8QT1vZaoDnosATHLea4MxNKLCZWta42KtsP6\nUJu4hprJfnS4O+Gh/CIfZRusY/v9gPt/qBPUFi9Rrr6O/wR0v+NXXu4asdGgzYcMhSip+rLaJVYi\n46z//WTcZUfxvKDiFVT1ytiQG5m4NDmJZfGd2y+6phB6Mun6eq23oK8Y8zYYDIYMwgZvg8FgyCBK\nbjYJTSWbTEjKptIWSWkwVAdP5uZ4FfDVoFD+Zizn5qqp8f+SGl4X1ZTVVG16Gc/0HW7LwgCkZhzJ\nA+KbGRJI52Kj9LV4Wc4c+Zqg3MBZJ6l3DVLpKEcfkRa5hnpyVaRzS+t9K6XUO14S/eO8LAt1TP2v\nsmJbS5EwIv13YpOX11Pt35vPuIkaOv5BqQd0gRerW7yUhvqCqgpgL9rHNHX7u1NZ1C1eHqE+p4lM\ntU0nZ6WWBVlC96d8PW/NrYXqw0dyBk2I32f9Rgfz0+uavKzivFrdp3ngw1500qT48B5eTtN7NDm6\n5vxpXtayod3VXi57p5ejgnPHTvBSk3jqDrtSbou2ATZ164tNWZrslhnlYconaAqqp72x54m0zDL+\naCrNIzKZ/Z2SZqdZbI+p3D2Z7b96j7SoaTTjQousqU8OwnxizNtgMBgyiJIxb5HqFeHOnuggv1S5\nr5LYFyeo9NW8qJOnk/GU3ZAWWUMG0vMfXjac6GX1NV5W8Vo78evZ8JyXlx7i5Tv0ZQawIprIG8n6\nhOy8OAg/u6IE5Ivl/Kz3sMKdZDgv8Tyt9X0SJxf34yNecVBa5G+ofizgZ/9ynjPhzV72PemlJqZw\nlhc1x/N44Pskpl8lt8IiL0wTB8cgnZisIPOe8KyXnaz+t07xUt1oBtc9WV3tZQ2f46Tc/QHP8Vk+\nyuucw/254K1RkRReoUZTOSXd1/JtL0Xj1W+LSZVYzxlk3FfSI3T0u3l8WXrqqHn857uUj3jRP8PL\nPrLRenaHiW/zUgoslQ/M/4Bn0Wewq83r+rL/Zynlo29JL9r2qpe6Z73o8rON27GYCLS0HOPWs2Z/\nmsUxRJOwrVp+einf5kYOEM/wvbkwN7sPjKn38gec5f4n380OttQ6ahvU3g5nR5zAsahd4xeQ9hH1\n7y1QXI15GwwGQwaxzZm32JDc7fKYt5j2QDbv2AbHD96d/NKeS/PjyCCUteb3+UWtusfL/ejG07bS\nS7GPdhJZ2b6vD/KXnRi70IVf+KKgkKGL1KGDtLJGkSas6EOkQXP9jZTRGNtfy/VRJ9GVcLyoBYB9\naB9f0MLf7u3lVU1evkDm3UGbZiWPt1G2BmFKjWzZHPPWPRSK2NgCxK5bAdmfxvYfxeffRRb3Mp//\nbD6nG1mV3VZ7eVGTl/PUGbRMNeCXrwaAPnUiv1Jc6xgazMXgxBhZFl6m3bP8vWlZKj/uJ1vjMihb\nLd+RiZzKmEVCWK13h6QQH0l/2sauUzHHy8fI9o5i336VZl1pMs+f7WXvF7zcl+/KL/my1tJ98hZe\nq3+Sl+VH/jN3zal8VxfN545VtJdvICuNp0qKiTCUXe+smDe76RxE+/9Gxl39My8fYSN93c9wnYU/\n5Iq8ac33/D/vJ/P+0W5e1p7nJac/5rR4eQqLWneglwtqg/qpr0jBVj8L5mMGgjFvg8FgyCBK7m2S\n94VRIIncAWg2y31iHqCk6VVf1bncPJey4bG0yORjXu4kSUKqueNdaNMex6/hQ2RWe/OLXBEw7yk8\nVyH10hripFZbj2AqvoNT0xtZsU5SLhnrF+d/svtBdnywdpOpbzg9LfMEsos308Winiy5a6GXq+7w\n8lna+9RYYgzyGACABnahhPVwohJb6W2iPsCmuCd4Dvvylp5nU6ykcnEan49s4u+Jovu/QHP8fLL4\n1jACSx4Dj5A2HXpB3vVzjFssWmWrnrWBplHFYI2ulK1tNaLUpXKkGUvGXSMGSUN1WxA01CgNQ5I2\nbTC2ZNyROtGLBTd4+W+cSziSClgvlYsO/r7uYlaJ+694X3rNOYhQ0cT7uD8+svUYKAQeSL2C1H56\nXf5C2UmNoJETJetpkM5pBP592j2YRJjBHjb3x+yAjRpw2FAbb/PyKa+JTJeNW11Ec3hAaoKIAs/y\nQugHgDFvg8FgyCC2OfPeJKlTIecEfYVk5+RMeo4d8fjJZEG/IAsaRfs2VqdFPXacl4tpS3ovQ3rH\nkD2OlW/mCV40kf2PYkxw3fvTsr5LU/H0EQPcS9EQGAIr2vMPVfCR9dLgeAb3H0T6uSe3xRAd6dOp\nQRk9NGreye3R0l0u9eJeMu65ujaveQobce8CVc4lxNpKW7cgLYyX/GywnsjRTayeWJXs1SRHzft4\nOUdtEHsxkTDlaX1irrSXypdcWMHfyqekWW7dYtn9QcyzUhKItReTIrFriLB9kPdy8ej8044K36vn\nSYmrP+7lbezIX37ayx/RKFvxIQDA9GmeMSbf8ruf/L6X+7zHy0VUBo9TBDlN3Z/UewpgtrrBSrr4\nVJLmF9O/uy+SumboyaJnLLaruYENnAiopbdQLz1mKsia2Ydwg/f3rjw3rfhcfNL/sz9jLVp4oJve\nNuP+mV8v+Y5rW7ECQMqwNbapfoNoJ2PeBoPBkEGU3uYd2nbElPSFGpe/rSgyBYDdQrLTyEREPYwM\n/OdeaZFHkXiKWS2jL/jemtVlGWvIuEb/2ctnL/Fyz5+mZb2FLF2JkLY+wnKgfLSB0U5eHFWkDvKx\nln91GRn3Lmycy/k7EgmM9hMFSRAgpnt2H+L2XaQGbXQvmKuwLzLvI3nDmmvYKTAm19ExPo9OFBF8\nLs1BYq1m2W5FCddSbVpJLqK5EplXZdtWVaVo/G/QKHV0kKYXxIpHuZ8+0UqbegDLahYzqifL3hjY\nt0XPB3q8RUAzm7uZ/Xq+iL/q9dzc9OT/9Q/ujN95Q+9d+E8AQC1fuPd+/GsAgOvn+U5zx15+fqSZ\nzcsATJRRwzlBj5/XXniul9NDts93EmO7vWwkGxUrLmabqG8U8pGWtvUCvYgcB5lx9Kg6jFLeL6qn\nuvN9vo9cikPSMm9lxxJbVkTy7/lbtYPGtnjtE7FrAKCnTm4+aQtS6BrzNhgMhgyi5BGWeeAXcwHt\nd4qE+ynZhLw6vsQvbCMD2Vpme7mnPBJ+MS0ts5xUtNsz1H2WemfLV1/0u2XzHk3y2eNJCdYccwwA\noP+vf80VtQu/qGLcqk+cynbwGOjzGlIHfb5JL8rZcmU02rfx0836T+r0THv5xWQKf/CpOVcFnhW5\noFFGpqKC1GG9vuW8xhgybjmq7ExLa/WLQf26IlmknJ9xqpSW4NhD9Mn9sw+dPO4m/4z2ZtaZBfgt\nAOBdrHjd7zzb6iMF+jq+5H//ARkjAdxOKea6jJ4D1WzHHp9BZ54Ymezssn0Hkbi5Muj+m9MEioF4\neTYZv9tZ3z72zov3z/3kwmXee+hecuhZdEm5HN5ef/3PWchEf48TJ3hKOeUor+31Kiescsb8FyXf\nu+li4vPfnNZzBBmttOk4t0gxoD4SBSHnxV/oei18UG/18h42z9tYTUctopLP90x61MzruMH/M+OG\ntMz+ewEAszhV8EH20UObeFws/o9k+128tuzoYX8QC9/c0mkFYMzbYDAYMoiSMe+cp0Zg8z6DRO9w\nEoF2nnQA7WRKHj+FNs/2z3i5Jye0cRtzcDx7aVroZZ6NHUeKct9/+x+Pe9EbxhcwIdrhjLSsv8zL\n8ks8mwvXMKjjB3RivGpY0RF6bERhnVqCqYKNwq/4pN95xv1hvAIAeAQ+c2Ad83qsuPfeXIljTqCN\nWAsNPHOsl8qpL2PmabzGSNKDKrnxhJpBnJ5tEA6qbwTNwf8113t5kzdM1+A3AIBlfFofhE/a0Q1v\nb13NRqpjHT8LP6HxnUsCt4T+n3vZTrb0Ao3c4wL/eCBlTbrNOi4OVvXD9Bx5rsj2Le0hjPzbWkRr\ncuSo2xJSyiC3yZVQRkmvff7fJaTBk/w7sISeSAfQvR/7e8bd8ku/2cQklL2MlajgfncMz/8z84DU\nB/2inXS9jdR2KGzdQqzshdG5aidGfLayHiPomt17qJfyq5Ii9U2q0/OUQfGEP6ZljrkCAPDVIHMj\nANzI531ObrEYlqZtLRsXviKiz7HX3SCit415GwwGQwYxrGze36WfaD3dT9vpNnkEv1R78/NY+XUv\nO7/BH+oLWElH1MuUcRu4CL6wVTQS3ldFukA73fQmL2eQTX+XNu+R9DZZU69kEUDZYl+B9cHEc3FR\nyD0h+iTLPr2WyzNd7Ov0Aa5JdTm8x8hxuBEAcCNteMtHpSxzz0/xn5fo/0vPlF3wIADglVpeQz7j\njt4n/aQ45YVWty2Sf3cMVTv0yV6pKXr/bBvgo+OWkn1eEXXrfjCEcAbzwcg0G3bCF3jOTkx2vTO1\njWPoQSAVTDZvbfeTccszCkgpkezRQ6GM6Bpqn1fojbQnn80lwcrZ63nfarYxl3p5mBefoKfDb+iy\nPpoa8G7yjeZjb6HmsI8e9R+oqpUxp0f/Dek165mysPU7XnblZTEqLuLXJszax+d0RZOXDbdyP7Uj\ndYHnvvlN/8+dPvjhMM11iQG/dGFaJvtPFbvGw/TaOpvTDacy3uTh07yczvEsN08ROmapLQvZ6zcD\nY94Gg8GQQdjgbTAYDBlEycwmOfe6wDVmLNO1ws8HYPznvByhRZk5ydBzk5dahSlnNnnR21kuhGZe\ngKVYAgB4M5iPUbOO0WLncym/STV0zyYv17YE4enbYuXrTSB9isE5nYxAas//7ibUvw7i5NR9E7x5\naFSLP155+OG5c9sVrPJDpQ/yDfiK4sbl0qT7TV6vm8iEMsSrxgcpYXPLwV/p4/TnTWIdehlRVS2/\nSEZRdF3l5b8xkEaqqSZsgU1SruZMK5ooXcWJTCVYquYkYMIZzGcD/696TnJJJQ5NKlsLmQbi1caV\nVKyR5rSXgoURprG99p3uJfv4FM7JahnBDWyDu5lSIjfxprQTXBAeShlRQXOJJmb//In0mg08VkNz\niewTaotiTlzKQsRXYmpQ9pl0ejj/Pu5gtcDQ/7FcqKLqDG9uXCYzKR0YwEla3BUsc6d1TbjASy7D\nK80iDUwfUaN7loODvDTCyXe18RtwoTTmbTAYDBlEyZi3AlxaQzbLebFVnG/cwK/enj/hcQYJVFJW\nyyWo32fIKfucP3AlJ+0A4Evws5t3aRajMlj7Csj5Bs3iV3E9y2zggsW7X5Ce2k+moqmX4i9/Vgis\nWEK6qElDfbFrPVMY0enlyXSTW/xNH5/syLI3HpAG8zfszga+yjf4JM7eLBfVFusQC63TNTVhGTpQ\nDjHUP4KUB/g7GSQTJP2dk8xvJaNZ1eQp4o/JMC+il1c/76eMoe8PBMm6TiLzab2LOx6j/+lxDO8e\nxY7yMuUozlY9x/PKlO0KaXAO3dByFClw3ysa9CgU9r2uwGz6S6J+nKAu9/S3eaOfbL2FzFlrbMzR\nRKtimMQY1eE1kan2U8CS2goAuhRBx+3cu0op7WcQiw4MGix7UTCfvois/HxlfOAzSbjOnSPzrmca\nhMPu85p2O12Ic+07PS0zofvt4y1eHsB26PJZBlBOregoBcHJMlAok52GozgVbFWBcyMY8zYYDIYM\nomTLoOUch0IirLV0udn7D8+ka2ivzUXazuQ/+oq3+1SXX4SnTV8LinyQjHvJJbQFapViMrlrSJhE\nKsbKnYcm4rG/TMtavUd+3YeOeZdt+n9/xH4j8vs1eJts21m0xa73N5jQHezAx1P3SdzmXbyuxgIA\nwJ1s2eVaQ0vJ+Qe1lrAMjDLaFckvTpqFGEjg/jWRARdq/yb2IS3s+jKbKExRCgBLyLoOpiZ35CPp\nsWaVqU62f3feRRbQbfI/+DhkJ57Xw/PaUle4+6lWjier3Cdc9mprES8fJhvyw7Szd3OVgbe+Nf3N\nnxlA9UVOJknFhX+Rmn/kb6p5JMtoYx+qJnt+m59kmsJ5h7+TeVeTUVZzOqAxUD5a1T9jn+B4KcFi\nIF7qLFjxT/mk/ko7/jEX5J86mmOOlkF8gm60T0fP7KVgnHqKc3PdHJ+WcnyazKwLd6rpVQ91DfXh\nMIgoTla1BVNHxrwNBoMhgyh5eHxrOBNPm6VIRV/gIQGkK0/tzejoh7jk0xFkDL+ld8G7cUPuN3fX\nMihnCg2fR3impOXOamiXqicZyS0npcCEE9Pr/0XpQge8q2IhnIon9WzX4r9VebvxLi/appAtNdEm\nS7vrRjLJqj8FRT7vqUE3k+t2iHFP4DUUTa1r9PMbL9bfF0QSlMfeJv3R9htEFPMzNbAB/oQsZTIj\n/pcy2n9XPp9r2HxHUEqbypGpH/N3acYA3K7ofnraKOnQVO6mYwEeY6BKE0nqLcoYEK77daUXK4/l\ndqycbA34CLQA8SIy2/sYn3bOTzlR1PiT9DdvJxW8yVPBfZhRahnXt7v4455hO3K5y2QYvt673Ewl\n49asSQO9dFZqIRC+O08G7PRxah8nRu9TfB9FRZy0C8i9rNP43FrpOTOa2kMLF1feUxoCFak72Fk6\nmIqj9sm0SDHeXo5PezAIx2kuQPMmr1JtG8EUC8sWejkxXag5h3jB7UG8Psa8DQaDIYMoXXi8CFuY\nrIdsWDnOl7+J/73gP2m5fFBPIx/t3mlzydW/8zIMMR3HEN5dPLu4o8lvinErV/wI2ui6WId/MJT4\ng0ELSVuYgqFCeSQBbKBx/nlST2WolS82TfmYuNDLPqovvI81tLNdMjm4zM1+Rv1bTBHaNoZUmxHN\nuZnvkUz9qiXXykixykNDpmzc8ZpUW8m8RSsYSjwxsHnvxsv3kg5Pps9uz0n5RSi9J2jnbKONEo95\nxrRLZ3ru42Tejexk72f/PJfN3srUwX9hezfR/b+ZXguhArkz+8x96jAtlMXIIMD35UHWazwzA5/F\nfnHOB3je/I+lv2n8b/7j6flHaeu+h7pIJQvt4zO7iE7QX2Wiqf/p+SIAoJdL0fXQz3kCvSyepSY0\nVl4qACpE06WmyltoKBJTxV2ubEx67DVqouwzjfKgUX30Pr1Cv/gGz4pX0INJmmuNVqQAsLPPPIEe\nLrxcJz94xwUw2vkilfHm6d2DEUqc15IWNmJNfn00dtliDAaDwbB9Ypsz75zPg2abAzvguiYvR9P3\nctQZnnHvyv2dLTyR9sb9tSbA2TRI/YSfr9CeRgY9i2a/w/kFvoKs6HHathaRed1Isn+OZssD11Xh\n5C1IHrNleJ1vqRjCghYvVzZ5+TbmmUzItLQgBW11u+r3od9o4v1x2r7/PV6WlHC00l5Grix9NAwq\nQVVeZp0hWusrStgTZuGtZPUqZnEHI217ybBzXYqeQu20Sx+sOZQ/b3o56RKtZI9HqL3Ul6huvVVe\nAfSFzy0/8Fha1lJOHeSSxBYzCFW2WjZIO9+j9QyCSL7h7/HRL6Y/OfQ1hipf7Zn3l+dwImeNd3i/\n70jOC52nxTbokH6cZ5Bd7Ja9lC/yHdmDuZqUMrZzj/SaZ6srV0ZyKJi3+oqemdgskNqdG5lwTHT5\ndp4sRj2GhbR5I/3M27wn1sx92JBNaZEbOAcwRZqVnms7g0OquBhMBQeqVi1uzP39Qf3koVNA6d4c\njHkbDAZDBrHNmXfOFCZGE3xpnua+CYxg2k22bUYJ1nKpKi2MOoVRTwnz6D9Ok9KBgTtII6Mi30PS\neAe/lnMf5AlLKPmBPkcG7UKeAaxrI5n31i+DFiNOWIF0AYT30453apOX1WQKVWwkPUm1qxjHs+d6\n2ZZbaQGoJH0cw1nvzgn51RDTLiOtK5cxU40SqB79LTx3VHTOVlKsKEl9sNQB3kOGdwy9TLQMnrAT\naXQDzf/19N3tpymykzP7Dwem0flix2o/3vo9ZGYnkF3mVEey614qfT1B9Nw91OIWydNCx4qxRjPL\nkv15b9qdZcrVwtkHvSf9yQLaqKcfSM10Dh3cW+kEPZ4LMZf5RZhn0TniInptvMi2kUfWSNrd17OL\njeLFa/RCAFghNhqsdwEgcCWLtrcG6nJ6dmEkY/+x/IeO+v18UbSAi16LlXTKPoWTQ6dyQepl7CzP\nK48yMEL2acWASC1cRTV9TYuXeq1yGheZeKHFGN6A940xb4PBYMggtjnz1kdxaoHY/Xp+uGr4pU/4\nVZYtrZaRf28m85Y/Zzu/cEoqf+O+aZnvIPt5gAxg5m95QEZOeTFMjrbFmkLz75swtOglVVTmOgCo\nZCPU/8jLWqZ262EGN5mhlfywhvtfoa1O5rWJQcRdJVlFD6NN66jCbOA5HaSk9ZxUqBUtVaMFjZLw\n/1564VcUyRcnYmStoSeD1l7mpVbyuU+j3flZPv8GtcltXnSxSUbIp/sXZ6WFyk45xtPzxuP95tGc\nOxDj7qXZvJN2zzvYJ0KyN1PzORsiWQyQCL6Lb+76D3t52HOsGNlgZ+A3fABVwysYEzD7Dvpxl9HD\noocaGM3Bn5ZPNNtpApW/1VwYXO9lLxnvy8wy2BU+s9i/W20wlMuhiYGHPtMvMwVpFe383eSremBc\nSAE7cQBQfXs4QPXzHXgtWFy5+1wvy2jLlquRxodVVP3F0JVwUmUrhBx4Q7ZuwZi3wWAwZBAl8/NW\nXorHCxwbyQ/XkzS9VtC8uzdZcQXNT71k4G1k5n8bywI+kpbV81EvT9UXv4mSX8Ep/DrKXLdIDEFf\nwpBSDQVbCFFRyBlWDugLvdTSZBtoBV7PteDqafhf6tWTsou990G/DG8/Dx51FxlnJylpBQ268kMd\noeTZubW+KMuibaQ+rEVNWh0g9voAcBRtiPcwivR4Esc7mMdlNRWFGuazaCF5Okq5Jv7G+6x6V3Ah\nGqR568pM3U0WVzcV+ZCGJjt2+Mi0Lw46LQZYZiuZ7WoqTRW0oy6k98KpgX39RjJm5XqZ/TYdYcOx\nXU5mX6+oXwvFAAAgAElEQVSggjWBHk4LmWFPNu+D2Sa/5n3p/Zwm7y8g5+2UY9qDyJL3hhG/l6E3\nmDI8rqLmOp4NtxMDJSqonjklwmGfT/jerGM+mPIgk1EPGTenDGZQ0/sA22G6To0XiVY99Q4Dm+Z+\n2YK+YszbYDAYMoiSMW+hOfB9vZ42tqlkUEoeKNLzUxLTPjJuOaP0feYzAICO734XADBJORcA3CJb\nNq8zhXbSM7l7drywrL7aZAyNwVf9aOSjeF4mguhlAVcX2cGXrsk/pY85LMpu8PJib6DsV76SGaSI\niRyUkbL4Q1nWU5RqC+ViyCFORBxOjRcjYcfrQPQinKFv8eJE2a5lZtfsfz+1kG7SLrGrcj7kKtZ5\n3TlpmdWccKHpfs5iSjFsMaR10bbmV1QXIF14OFrhpSh+3hEzK+e2ckj/O+vdGTyii6lNvIX72miD\nbZCCRWb+DUUdcy7pWr+uc2rDZ5SpvCumS1H8NWUYLS1FrCaS2wIhE895uyhjorIvcrd80zWtEycu\nUllNwYHo3uawve5kX9Hw8zh/26x71xhTKK/3G9DOjHkbDAZDBmGDt8FgMGQQJTebTAlUHGmbc6S+\nUw2bR7VtnkwaSihE7feacm8uueVav50u+BWYOlhmM1XZ2VKTpBJKtWnIr1cYll1I2xl68KZH+AUn\n0MFZkpWcmRpP/V1rWGm5pgmcvVVbPTkpLXLMQi+Xc6KyQrksNaNXKNoBSBtriBcbDiF1MgxwketV\n7ILXyWW+KlhvhfMrMKOc9e5mzs76c9Pftt/g5QvUp+MgEnUENY1MBM9E+0MUKUdXwTK1Jgf7cQXr\nKfNJ3a/Sn3xVaQOYl0tujhufyf+N4z0+zhdx5nwW8Fr+8mmo4mR53F3Cibicy93mbmiIIdOVnp+e\nl/qO3ApbKGUSWckoHmWuC90PH+KM5ERvShmpEHeZ0nRNmdg0F6rXqUiOD8a8DQaDIYNwSZJs/qyt\nwJlwg76A3AZzS6WJZegruTHa1hdOk0X6ooWTDpqwUFn6guq3lfnHJ3K/vH2CiN8cBmLgtyBxAxzK\ng7vt2s20Sfhp1veVkSg9jARQmtaErHIdA2vEVNUm2q4OJiyrGXVRLd8uNejmfNsKxTS/Pq1MTj9/\ncG3SvJl+El5GjEZJ91t571qEt4w3r0WBe5gLob5AljEhZs7qH/Fad9H61bnfha5wm5nDTaYMrp8A\nr9MuegTjo/2sz4zgfrRk2wSy4L2oNWxgPXeiJqPgmxFin0ohIU1HipfU2TgAJ2xD1S9urwGwRW2y\ndAv6isYMDSrqM3HdK67xsof+yVp4pJZUvSNY3KKek569zMtQpplbznZ3MhpM7ah207PKZYvDZpeD\nSyYP3C7GvA0GgyGDKLnNO8TISObc9Eh19fFcvxMKIufvHiw8MNAiwXLz2+Rar4PS2LwFGq+rnqWM\nKjyC2wnZp5Ox8fViswdi2psz0m5Dm7dQyP1LD6RHOQDWROcGa1cNVJYQ35LOiZtG14xpz1AHcMVQ\nfV+ItlnfucH9zI1t77HtNU7bKqa9a3Se5pxiBUzPI8h0WoouUhBx0qrxA+zvomOyNFbVX3MbyVvS\nMnPprBnAozQMGpcUGFQZnS/tLAzM2YrgJWPeBoPBkEEMK+Yds+SBWHOMHCMvcKyQzboQcgsiD7B/\n26JvgP+B/GiVAnDxHWyHiJP5FCNOaIhjjYYcb6RNYkY+kJIWa7pikpvpiiVDIS1oRCSFgV7wTdbT\nDhYNHmxfUT3iYK0ipQow5m0wGAwZxJB7mxgMBoOh+DDmbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4G\ng8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmE\nDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAY\nDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjg\nbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQ\nQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4G\ng8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmEDd4Gg8GQQdjgbTAYDBmE\nDd4Gg8GQQWxXg7dzbqFzrtM51+qca3PO/avUdRoOcM6d6Zx7wjnX7pxb5pw7utR1KiXYN1qDftLr\nnPufUter1HDO7eGc+61zbo1z7kXn3LXOue1qjNhSOOcmO+f+5Jxb55x7yjn33lLXSdjeHkwC4NNJ\nkjQmSdKQJMl+pa5QqeGcOxHA1QDOSZKkHsDbACwvba1KC/aNxiRJGgHsAqADwK0lrtZwwPcAvALf\nJocAeDuAT5e0RiWEc64cwK8B3AVgNIBPALjZObd3SStGbG+DNwC4UldgmOFSAJcnSfIQACRJ8lKS\nJC+VtkrDCqcDWJUkyd9KXZFhgD0B3JokycYkSVYB+D2AKSWuUykxGcCuSZL8T+KxAMDfAJxV4noB\n2D4H76udc6ucc/c7595e6sqUElR5DwewM80lz1MVri513YYRzgbws1JXYpjgOwA+4Jyrdc69CcDJ\nAOaXuE6lRCEi6AAcsK0rUgjb2+D9RQCTALwJwI8A3O2c27O0VSopdgFQCeA/ARwNrwofCuCiUlZq\nuMA5tzu8GenGUtdlmOAv8Ey7FcDzAB5KkuSu0lappFgKYJVz7gvOuQrn3DvgTUl1Ja4XgO1s8E6S\n5KEkSTZQ7fsZvIpzSqnrVUJ0Ul6TJMmqJEnWAPg2duw2CXE2gL8mSfJcqStSajjnHIA/ALgdfnAa\nC2CMc+7rJa1YCZEkSS+A9wL4dwAvAfgcgHkAVpayXsJ2NXgXQIId2AaeJMk6DJOONkxxFoAbSl2J\nYYIxACYAuI7kZy2An8KbTnZYJEnyeJIkxyZJMi5JkpMB7AXgwVLXC9iOBm/n3Ejn3Ducc9XOuXLn\n3IcATINnEzsyfgrgfOfcOOfcaACfBXB3ietUcjjnpgLYDZ5p7vBIkuQ1AM8C+BTfn1EAzgHwaGlr\nVlo45w7kmFLnnPsCgPEYJh/87WbwhrftfhXAKgCvAjgPwH8kSbKspLUqPa4A8A8ATwFoBvAwgKtK\nWqPhgbMB/DJJkg2lrsgwwvvgmfar8P1lI4DPl7RGpcdZ8CaTlwFMB3BikiQbS1slD5ckSanrYDAY\nDIYtxPbEvA0Gg2GHgQ3eBoPBkEHY4G0wGAwZRMVQX8D9ye0wRvXk+GRQbonumR2oTfYaZJss3oHa\n5KDBtQkAuNt+sEO0S3L6JwffJvPu3CHaBACSM947YLsM+eBtKDLKKSspayg1/90XyfJIAqm+1RFt\nC/3R/v7oeB8MmYQ6wWAfYHm0PdDvtrRcQzFgZhODwWDIILYP5q0Pfsw+C50jFGKkQPo5G+j4tsZA\n1xfjHhWd10PZln98mQLlAdT3etnLMkaw3bpZRssILy8gu1/UxR+uoxzuntGx5hDv3xpkku7EnWig\n7VitI/qZyqOsO/qdXqouDB7DlJ0nAwyF/RUDH+/jC1TGF8pFMt5fZGSyKxoMBsOOjmww75hZd0X7\nR0TbIXEQA43JhvY3UIrBjo7KCImCrr8OpYM+t5Gtewrvb0WVl628nzu4f9LTQRFP8J+XKRkg/vzf\nvdyDuxc+7uVZzOg8TwWEREtt0oDSI2bWA80DqP7h+TGNGYic1kTbw47+FFLVBmDUOVTlb3bvkr+9\navf87RpE261e1gedzEnVez11uISImbRYtPaLcffWe7mhMT1XmmfcJ6rZocrZwSra86X2F4mJD7uu\nZzAYDIbNY3gxb9lr5QXxGqXWDpDNLRnj5Yg1Xq5SAW/m/ifTMsWSx1O+iVLsS2Rkr2hbbL4lqJ9I\nRFzPoYS+6qqXvvpizWTYu/O85hbuf8aLUyd6OeWQoEz+fz/bd/RklnGLl6038ZJcW+az+3r5ONuw\nOSRwul7s3bItIOYcM+tYQ5NUf+p/i5cVTWlZ1UeyDDZs90Msq9nLKvY1aWiS6ieFvHm2CeIGF4vu\nG/ictgO9fJ5s8jIdYEONYWf6Bjt4PV+O0dfxPN7gind4uYrl7DkhvYYYaw3bs+qFqF492CaIGbYY\ndT/vsS+SYsk6rjZaH5ShVGYrlbBTHYz3fxDnCP6DcredvaxhH6oMVHex8PItmTfwMOZtMBgMGYQN\n3gaDwZBBlM5sIq0pnPzroI7feL6Xo6nu9VzA4zSfVFP90KTj6G/x+J1e9gVmE2mMUnN5iZzKTW3l\njp28nKB6Ubs7QuaU4NxNtovhghYi1Hhj7VKmmrX5u+fLnPIspcwtr3rRHGZl5r2OoVb37DQvm/ib\n2v/y8k5OVM7hz5pj80RQ1iY0YCiTZqrd9fxj987K6LxVNKfVz/Syh3YgqcgA0M0O8hQn5+7h/g9S\nNv7Ty/U/9nID08TLHBdCphRZMIaEIulmeZH+kV52UnVfMSY9lWawnNlI1o2LHvBy7ke9rPBmyYc+\n5De7WO9eygvLzgMALFrK37fyZWqg3bL8XUH1+A7LLLCRk6CV7JC5hzMEjROaSmS+6eHzbaUZRH1H\nlo/fUV7A4w180S7z9b8Sf8oV2cOX8jJ8FgBwCk4HAExAEwDgpcX+xWle7DvicnWS0/xxHBQ8m914\nnVpWZAvMJ8a8DQaDIYMoHfMWM2sL9tXuz3/4lexf4SXnV9DUnffbk8n65reSmXOOCVOCMsXCdB3O\nP2niciJJmBh3BVn0eH4A2zQxCKCGWsLXDvJydjwR9kYxUIBNWLbYnFiPmLaYOCcoc0wwPh4umfo8\nJe91T5WtNeUfn+3lE+O8XH+pl1Oo8exa4B70PEdF28VCofKkRXE+KPestfyGNIR+aWJ0aatazO1A\nralmhziQbHTvY7xcs5uXXWzY2v/0sp3MW1pX2Afitohd64oK8q/usV6+TFYXrg9US3ke5YGcid7w\nHS/f7t+rR6h0/IZkXsvGL+I7M4vv1SI1X/25rEKTl+0fS69ZwQ64kR2w7kwv+3fLq3Z+Z99KiHH3\nBQ3eQTXjRV5QY8TvVvMfPqSr+dtdeXNlfNFu9Y1xYfUFaZnynFxyki+qkh2xnA209mgvr/PXPH/x\nAgDAuNu9K+XFtx+ZlnUlNYNdWE8x8LLNuxMa8zYYDIYMovTMu3rapse6fuml85TwDDJdfs9yXjvH\n8uP0FbLKaUfwwP1BWatZfjd3lvMrudrb5FZ0fNFvk/SLcY9f5OXTgRlPPO3Cf3g5W/berWXeYogi\nIaHZqyw61kE3t3qynGdIp8Texbj73udl44letn9n0+vWv9/LJ8kqb30nAKDhdk9P2s5ig76bTGb5\njPy6AJu6Mm4uHmRLofJ2DvaRXC5g7z2SsSGHU4tqlgI3Ob+oiTVfBgCskGvp0uAg+9gZJGKfYd+a\nJs1FjE3PWvbs5VRXQrfDjU/m172obRLZujdKOyKjVQDWzMCPte67Xrb90EuSPexD+U+/0tmhD3Gt\n4T9M8lLupTv5BpvzGFVP2a3bzvaygtFdFSel1+S7iwnUUNac62UupJxaddE6ClI3wM7AZVGM+yfc\nXilVWh2XL9tEdoqyFi9bfV/51wf85i7STgEs/4cfAPY/+3AAwBGJv8fm2zj3NvoOL89/NwDg2o97\n08HVeAQAMAmPp2U9QhZ+GAexnfku1m1+3XBj3gaDwZBBlI55iyFWH5Hu61roZaWM1n6aXIx7OiNu\nDySpWE/2fqPsvGLcHd9Ky5Shuu7j3EHamJCxkrDsxd2jaQ4E2VxrUOWEX9yuw/0XNzdTXSybpkhI\n6GEiokVzM0bQFaCDETW697XUMPrIxNSuvZ/yct+gzCfIFjce5+VZMoh715rPw7P0y25i2739KC9r\nzvJyxcK0rD4yKNmfFUW8pYRqoOAeMZ6gjWeRdB5MFlxD8+UDPGc1UxysrEUeDiC5alEd35oeOyRe\nZpeOKK1Xe/kkm/d9vPYK2X3XcR7mhQIeTnFEeLG9koCUZUqTfTe9InpmpOec6sUSZsFezcc//Q6d\nMN2L58i4byIz/O0BXl5BtecFykOWeymbfhcbKwno6cm/8lJt8XdqiuvvRn6FizA5kkSh7J1BuP9f\nKFfKhsy5AT3g0+Lf/MyLJi/2Y1vdGGQQfxz+/Z/zL+74u47w3etnroiR93l5kNdo+xYXSBFwO+s1\nhfcwCFu3YMzbYDAYMojSMe+cjfbb6b5+fuYq6VNLVnwoP0brIjZXRVo8UyxeTOClK9OTamg/V3Kd\nib/3Uh/833oxRjPIYmO0dW88IS1q+cH+i1v7Q5VNWeycO4VcPcXGOznrfTC35e9dQ7VjFW3dFd5m\nB5KpnIcJAIy6wsvP+5s+CN4TYCecAgC4DIf6419hg2qSoZFuP9KQgE2T88T7Bwu1oQiQyiOZaQxI\n3SlksM10bT5g7/yilNJ2VOQjv5SM/CheqzWYq2j9vpfLLroIAFD9/a8CAN5EH+lDHuM1Oc/xcdrI\n5+lZydsHSFMXxP7nW0WVBshd3EhXkHrKhM/26PRMMe5R7PNPSyPRfM7PvW0W+9zg5Te8mnHhFz01\nv/I8qh23vOjlhZ5JQsy9xc+r/Gt6es3JVAylGTr104RMt1wTRlsRJh8nkcp5mwTn6DLX8dgIql/P\nUouYQlV7HTXzyu78at3k55jOSYI8yH20+a/lREiTfMD5jF5j5xjHjnCaF9cv9u/TctF6ADiN9cp5\nTFHbHUTyKmPeBoPBkEGUjnkXSiG6gV89BSCRAHbxgzZWeanIghSduZHMiwFicKfJQAzgniVe9pBa\nP0zWsBOp0td43hpSqNNI+U7yn95K9460rF5v350lEqSoxaGMJhQDlc2a7TZRigR3L3qK//TR1rhf\n9PuQeS/1vspXr/RhhF3wdvTL5H5wCe2oTWzHEWyrbvpAN8lTIK3PJox5a0HSfz+VsbGB84Qi/37B\na40lmdudpGoJj8/h8Wuiol8iiXphdLpvMs27eO97AQDdLS0AgFduvhkAMNpvooHyS2J0vMa8MLYg\nXqxiKClS+yVe7kubO6c6pgZm3wOjSOC1sbfN6ZR3n+vla/55X4kv+O3P8vk/T3uxGDa1IqUfbgh9\ny4mfncJ//vcTXm7UkKNGKYLaKsbdSvoaxGbg3+VFcqmX0honc86rlZr/v/H8Jso/0D2l61gvw3dc\nQaJi57uzISo4uaCMutLsJvm5ouWXXLppWbIW1FOrCaN+NwNj3gaDwZBBlN7mHTI1OT2QVU7hV+kA\nMuw3ywQrP1Xa7CqYqfJpui1jUYHr7cov8Jf99+py0uYF+DWlT25wxe3eJ3b27bTv/vRXaRk/87Rm\njuzixV4OrKrAPjK8RjLvVrIoEcVvk5Hup1S38miIc26Evtl0+f6yKNRZcpjm8XELveyhnEbDL5/V\nLLEFADL/t4a5U4oBspoJ9DAaGZhGD2abrCDDmsP6KJ2HcrGseNjLU2WPph14FtvqHYEHweSPeFlJ\nT6KN9fVhNdBFn/GNtKe3sC0UhQhlPAXSfhEw+63HADl3G8m4NR9EVpj3CuinNNGOPhB56GdEah/3\nV+YUK9qD6TQBOifhCNJ7tkUDze13np2WWc+2PeeP3LHhNi9Hvo07hkBdlYYRNlEvPaUY7pDzXvqb\nZ9wTGfgp35zZC/hPX8TidwtchQ5gZ0/ocvR2zjOoD5CI/52RyEe9xPenO3ZpQupdUrXlK7wY8zYY\nDIYMonTMu5AfsBgCDbm0kuE/yaxu1Hn6woqBMwve7lO5HdrT17Kwp/glvdyLi0E3kovpsw3v0Du7\nloVexN0ut9IDkDA73SqynWL77cbeCUhzr8huu4KMTw4g/01tRSbXZnro/J1lHCUWWBtYfr/Ne6q6\nwcsO8md5+3RQtTjDe7AsIzmQt0JVYFM8mx4fB8aeN1u7KAM1ikf52OoDlrxCfrURWZkoDwv59r7y\n3152UqX4k7+RObSRzql7Iv3x//N+yImiMml7bKXjwMtkbPJ4msvTSNwwN2TeQ5gwL0XU+XTN3/EZ\njvpacJAPp+MXXj7MENS3+ee+gc/s0d10PlNpKgTjVTLG5bQT1/NNXOy9VJzmIx4MLil7r1j86Ev5\nT7yiSREgzwy997stT49xGucKavOz2WemvMfLZtr9Z8sr60XGMvRx0MnN++ScuYExzE5JTWUN+/oY\nvoTP8hZXOp7PaTd0McCgOpjAKdPiD/Lzju7pdWDM22AwGDKI0i+DFtq8xeg48TqTHypxxvfIrtri\nxVJG1k2WPUsIs94toXWYJAw/4le55zNe3nqulx3HeunoPN7DmfWagGZ2kn6JPBR7yS+RqUAruZ6M\n84/8iisrgpSM03lcDgNjaTcfL8adm/GemRYqmq5bE4Pto7vOeM+4X6W9eSxtorL3VgbBhG+mMpLL\nkxE4orwhiJCxX5xKRnlPdXCO6i9viWZW4lYafEfO8nKJp1eTrvBeMmfRILyezvHfgeyvAG7y/vH3\nfsjPcdzG9p7LNjqDrE5+3bKRzlU7h37e0eLQxUzfkYJ0uZ/RxC/Q3WgUtaaNu6Wnyoj9zA359dnd\nM+8RTKB4mNYPpvZxB7WOU2kfxgd87pKLmbbxZvwcALB8BjWbExam12z/hpejL/aym/WplrZTBN4o\ndqqiGpZves4Sr3nP3sg2YL6WZr0X6m/ySy8PDPcAUPO/Xq4N4lE0Z8B37WaSdI1TFSxzGtO6oI2a\ni7TSNUGKzxr+P+6J/HsaBIx5GwwGQwZhg7fBYDBkEKU3m4ToofrbTL2ciV9mHublhzn3OJqTjpP/\n7KXy4VQruOfXgY4tF5xro+gRpfBcd6mXSmE58kte1sqnKAgiGPPP/F3bYKX0B6i+K5eSJsgOUura\nKJS+nU/0o3Lnozb9rwPSc/bg5NoJnJdaJPOJonwV/EJXO1zoRaVsM5rjBdBOl0QFhRTy0nxDkDmC\nVqyPhilhdc8KllAis366dfawgld4O8AHaR+6DD61wHQociTITFXtbW8POF/GXKq+D7FP1XOO6Uxq\nubmkni2UekBA2j+GJH1C1Pn6OKO91gcXQa5uYSKyq/zGJHjz0fJ9aOb4tBfdfN3UHxT1fqr6hVxI\nr/PRcH89zx84G95u+fhc/0LePjcwzX2f64rV0Z2umrbQnK9etN7g1kCmBsnKIEiviu9zCwPMVtFt\nr4MTk1XyN+bM+x2cuTyRNsxa2gPr3peW+azvI9fQvHQq++hE9pGZem/auahJGe1PjXQUaAs6s0xY\nW2AuEYx5GwwGQwZRelfB0ONJSex79cUio7rPT340neuZ70ucOxih2HC5KHH9YYwOJhfuJXXag9t7\nKdfnQi8P4CTd7pQPcnGGXuYCXa+QD+Q8qHJ1HioGHsQvzOZXfRaZnYJi/iBvv8DrCABWkjjkmCGZ\ncXvwpB3rfR0Z7KGaP9GkI0Ppn2ZQxt5i3JzQSoIFKq5jwMwiTXoqGKhQ+oMtQdS2K8LcvGofBkPg\nZTLuRk7erWd4/5FeHbjqgXWskp+V6lABVwarNXT6m5rNicdEk7KRW+q+JE2TyWxnqx8HgUtFT1T2\nepCLmVJG3N6yySljqK+dCT+Re9WE/OOak7+Xbf2w3hXd28+oEdf4h3DfBD/Ded9K/y59lcNId84n\nDrj7+0xmdQEn/mvlaqdoMvHGIka6beRDqAxmzft5c1XHe9nBCUklgFrGxU32odp5EB/sT1m/CUyP\ncV+QJuN7nrWf+4IPh29UQNxv6dy8mmOINJffUHJhh5zGCKSJ4xTibxOWBoPBsH1j2zNvsZJCn40q\nfgUT2iJbGEzwmg+tbf2Zt5uNSL7CMkgZZa8eEdvTABxNW9vTWviUDFtBDCs+56Xcz2hfxwNMqRqk\nIs2Fr/dFslhQm4TmeX6lc/yfzDbHuJmAqop5dA5n0yz+gZfHMDghWPIiXdJrIaVYstgjNYu5ZJtf\n+SQP02z4ijQQALMVwyQ2USw6IG2GJHnGTumhuWLe2ifm08d0uSPZgJ/ytsb+g7wNvI0E8v8mkmXV\npzdyBVcAe6e6jlzJuOBGL6/RQqa0nxbiiJMUAZtqZkVFpOYl8SvsH+K7AxZcB6+5XvVZukb+u7/v\nTt7jtdz9cc6PTBP5k+12CtWQL3n1r2ylX5Sknx31ohnv3LSac+nH2873rpaMu5/vbNmrm/7mjUJt\nUC41KXhpHe3NnVJBWZ862sUnMbdGd4uXk9l3xjHiSxrYmsBluN3b9idIifgpZTcZ90zvw9rAl7VN\nKsxi+imeFtSdpnaM5stW1hXdy8Aw5m0wGAwZROls3oVCyzcyw00Dk8PvyhneazzvvHKlZxMX4sd+\n/5Ws/iP8/bv4Na1sSctsoO2NzApKRK8ljMSmG6NtfVVD9iRGXOyEVDFCUqKwaxEumu6quN6EVuNK\nuMpU74c/DAA47HifyvRmsuxDw/BthcX30K4/ivSCwU1XkCB8gUzsr7SFtzOkOM+jRIFTxaYBancy\n+7ljgmOyryvNrVhywmddxwclG+jx9BJqpU1yH99P1mjOBEANbd11TdxBojaRSuAKXUO25adI42s5\nAdATzLNII1AfHxKKRDYn1qmOQOP83f8VLOx9gLd1z/qwz751Jj2YKngvJ1KLqxXTfpDv3Vrqe7uw\n/ZjHbNwCf2MfYZTU126nK9MFQfW+T1t3C7fHlefXdyhctcRaNwSuNh3sOOoz91Keyv2aOCtv4vn7\n5J9/OAN/blXOZeQWT27limmoZRBOPVd2ucb3ibZr/eaBy7y9f4mWXlMKDmCrUigb8zYYDIYMYtsz\n7/iDGzLbad4evZ6EafFY70Uwrdr7sF54jg/PRSfp3oWH5Jf1fn5WtQQYkGOquZlfMqp7uIzYifyw\nisbEKYEAAAx7SURBVHi36ourmfbA1pq3GnGhe9laFGJqoocNZHZ/pOsH7Wb9XL6rj4sjg4sItHMR\ngRqWOSWwUzdvPBcAcAadLT5LG/L+Ypdsq3/I5VU2ZrnqBow1V1fVPYj83SpIA1Kbh2H3VNDQzZuv\novtEN9neErqEiNVU8qGX0z1lN8+8nwk8Yo7QPcqzotxrIyukga3RotbsGHVUj8pYSNgXxmHooUVu\nWzmpoRQRHXylD/tT7tRZp/v3aAYZtx5VG+8tZ7+vYEj7B6lNcJ2HXK4FphJ+lQ/9a3L410ImOwcd\nd+RCL3fim9VHWS61tYgTAjkPDd57V6CmPR+dK82bqyHiQKqTmjdRn2FUPDr5ElwcDJXVjKWvDZLW\nAcAytts+XJLw0177WfI5riqjhV6agt+M4dOQ1mALEBsMBsP2jW3PvMXaNJkapvWkza2RzGUvEu3+\nf3jfzDImSMql/GxnJKXsjgfT3zdc1OAeXY8RVSf6CCt5+E5VlGHsZSDGFSKu8zaIsIQWPnWsYK9P\nBuSO80uXJZ/yu5dyEYENB3j745OUh33Se3w/JBYFoJlkYv8WLyt5ifWciD+WiwjkFA3ZQsVUw2cW\nJ18qVpvE6XGbg2Ntio5jRbtIm5aTcYk9vYnMqJZe7xNo26dmcURLUKaY/WSeoyC9NWSjLzOCcTwn\nEcrEHNkAoZ93URYc3gz6eJOay5hCm+zesqemwQJzqKi+n45WR+g9UjKt55hS6f3+ZbgSPnT5wsuo\nhfyQnheMtOw/kIx7P7bv51mXkXel9esite1hwqUR31PFN3dnWw7Z0cVaQ0cNRmHn5ioUyyCN+us+\neKEsyi98Kbcvlnbx0u7pwYWRC5f65u288K18oTQOTWf7kIDn9YvGpciD+XkbDAbD9o1tz7wVgVdo\nGTTiTAYzKcXiM/z4JfO8bOWs96zxnkrM/UvEuLX0EgCMpl10vKeoU2kOnSg2Kcak2XrZmA8jFQuX\njJIXSEweis3AwzZxooDiwTTacYWqhX7VNhz7YR7+kGeZj7X4zXaSzpqb0iKlddRxvdQOmtOVI/5o\nEtq5IgXx8lKFVmySa22x0qDqmmrrQuUlUcM3sY0a6OO8gUbaTnpL0OU3x7LDhWonRvukgdUpt4Uc\n65/1ov0GL8vYocI3SXUfklSwRG99/nY33ankaBF6RJGiHaE+v4w22ZVUtXp0mp83uYsq1hk0GM/7\nBOcUrmNBI6muKhfRtWxnsW0gjb2oJj0tl324PJJFWJRBbFUMvC50ZWOdlZ+FjPvdt/s2OAJ+DOnn\nQ+uFL2sxB4RdOBv2ys1BHpzFLV4uaPJSc3DTNQfHfljBehzDOnCuJe/ZlbOj9YWTa4ODMW+DwWDI\nILY98xZr0wc3/EjS1j2PtrijudDAefywLSRzUia9nAOCXEXku62vLAC8RqMw2eQipVgQwxIBEAMf\nz1KVyz60n8WeFYUWDN4aqPwwqlPeG/109ehflVev6X5dAdzB1RmO/6uXu5MsiS/Wfyotsp6aTA9/\n87zIoyIrNUMv19ZW+v1upIYTLm4cZvsDis821cahNtJFNaKKD3oEV42QAbiM3kYbOSfy9qhuL1GG\n7SymKgK0nhMu6+nD3M+H00f1Swv/rmnJr2d4naH0865mBGO5Ihjpa1xDNSr0eNHz+r2PVC47h5HK\nbK+2Wz3767/ZL3P2f1Xegfn/ljD0Us4bZ5Bx6535k5YDZN+slGsXgHXMT6TsndU6NiSpFvMRLuZ7\nHCuv956eNXfT+H03lPfGmwQ+z4Hhfvio0VdqA8YtTG8CAExawCyNSnwje3pu1XCqruMYKa58Klxi\njzu90BxG3rHXhzFvg8FgyCBKn887ZLZiPy1ezCQzmqkvP1mmGNP9NHX9jaamVnpRTAxyK68Qi4zz\nTWgFrHixWE0qiz2F0Y6xD3ixIUYTMkKxpuU35J+ruvzKs8tTlR1RngTKZCYGFhKdoyhlA51PGbdF\nhwzBEeMOPStEpIaaBoTMu4nyNS6Iq3kUZWiTp9Aytom0LLFq3WdYpnLb5Hzama+jjw1dS8rWQ9cC\naUSy7WoJOWDbeJuU8ab1TMrJEB/lM2v8fHrug3xBVvrO3X/rQgBA21ouusz8GsrfUrmMmfeaGLrc\nQI+RX7I8ZZVU93hO8zFBJ9uF7dH9sejYUCUGClAWDCqqoyKB1Qe+xZt9zcs2RktexkyJGEM2rXuV\ncgeg4XbvcbQcVG9nUGpV6uP40pXTFLBzlG88Lx9Nef6xLYAxb4PBYMggSs+8ywb4H8jltdAM+slk\nSl8kiz6ME/8ruf8LZFwfCfJhX04mPV8+rfrgt0TbikAUiZCxOAyiGqqFh2OEzJ62euxGW7wM/T3M\n0zGKlPEpzurLgK11XsVCQ8+Kf/EmlblMrLKLbK2OF5H9X89F9u2QsW5FboZBQdcObcrSrHjryqes\nVYbmylNELEvPTTZ8lRW47k7kva1QO7Uz22Q9/bylbnQ/lF9mLT0Iwj4xpP1DHZYvQRUnj17j8yer\nxoKn05/oJZjM3BvTfFa85D6/+fBO3stk/4P9ZMgUzjU199AxWVko/43y1Uh203ul9lfpNbujdtuk\n/kOIkNnW0R5/LOcGFKswmu5lXZxIW0n79ATvi9Ww0t90201yzl6dK7KJHilLzooSo9PMj7IWL1uZ\n3KSOKzf1c1CpCuYG9AINIotgDGPeBoPBkEHY4G0wGAwZROnMJoUm5wS5Eb6Qv3s+VdupVJMnUy0e\nSTV5Pc+7PDA7zJeWpgk+qcpRdHNu0kvnaXI01GaK7Ro4EEK1WxNjTNeaa691XBBNqms5F7LYm+pz\nhexAbLTxgX2jj+vGKQXvuj94KTODVEtdK5fgqUD9hhqFFjagJj6L/eDSFi8V5v8d1ne9PLb4bN/K\niVpZnlpkXgFQyYntk9g/FjFfEX7PBPsb2b79NJNoEr1A/v9tQ4l4EbmftfI5c8J6l1x8P/AKM//i\n+j29vM/PUDuZF5q9ueQKmkvulUcbt1/mrR+oXMAyLXVqdeYmL/uVIxXARk7aVb7CHUUIxtkctCRc\naDbRGm9v4kRkpbKr8eHvwYnJWxWl52++rZtmljlKuZuaTZZ8hR1p/0u91KIwFZwYXn+Fl3vyGWzg\nu6oxBkFCvSSw3W0hjHkbDAZDBuGSJBnaC/zJvf4F+gb4H9g01DpapivHkhui84IJy03c+sQa9RUU\nc9JEZRwUEv5+M4wzOT5xr3+Gh3tmM21SCLqnnmg79ryKmapkuFCx2LzutTLajpNClUfnbQGSvQbZ\nJos30yZh2zO+/37+4hgqEj0kgmLg7s/IQ9sZrBPLavxnemw1yVQd1bflu3h5oAisNBxNYL+e2+Fm\n2ik5aHBtAgDuth9spq8wXUIPZ6bX0F/yL8Ept+ufFkq9OJxwu5ypdHUPe7BhtFJ1XyH1GEC5JkXV\nGKFqqo6qB9cV7c9HcvonB98m8+7czJgSPAwFvTj5IfMBrv1i/m+0/KFcC6WeydV4dXVQJt0gm7hN\nhSZ3aw9EZbZQrmWwW7Xy9wI5j4T+oPwAyRnvHbBdjHkbDAZDBlF6V8HB2E/FMsWCxGz06dFHVawz\n/LiLNMQEYCCmXRPtHy6I7zmu30Cf4ULLzQ2ErHzKyYKnqb4M99e9yqX0KroSijT9kRyGOb2w6IS0\nyJPZnhfTBPoy23sGCe0tlK0K9pAWOARrCwwO0YtTxQmi8Yz9P702PXYqb6qfDSMbbcKXQMnPZCvW\n8nFireVhlisgvVkx7kJJptTxhnrNwAAKdKkMI+v0oijKRotpkAVXspM8TcP+ctnEpXW0eDk2WOau\nl26Qq9kpVkuTYYdUSoAn1FnYPtUajIIowgEY92CQldfVYDAYDAFKz7xDxCxc2wOx4IGWVCtkc4yP\niTnFdt1tzqC2EHF9B3t+1hE+F3mJxClqSfYU7T8/1lK0Le0qmBuZT4I4PzbvbojOjTWgbbUo9SZQ\ng3RG+xVuHbhJVa7F0EJ1KS+wbxtCmkPSUOBYFFBTsTf3c1ueKn2cM8gtR0aD9kbllAggDUUeXVW0\np1dKk+E1lAK2LAqT30oY8zYYDIYMYsi9TQwGg8FQfBjzNhgMhgzCBm+DwWDIIGzwNhgMhgzCBm+D\nwWDIIGzwNhgMhgzCBm+DwWDIIGzwNhgMhgzCBm+DwWDIIGzwNhgMhgzCBm+DwWDIIGzwNhgMhgzC\nBm+DwWDIIGzwNhgMhgzCBm+DwWDIIGzwNhgMhgzCBm+DwWDIIGzwNhgMhgzCBm+DwWDIIGzwNhgM\nhgzCBm+DwWDIIP4/p6Yz/vERnHkAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Wにどのような値がセットされたか表示する\n", "w = classifier.W.get_value()\n", "for index in range(10):\n", " plt.subplot(2, 5, index + 1)\n", " plt.axis('off')\n", " plt.imshow(w[:, index].reshape(28, 28), cmap=plt.cm. spectral, interpolation='nearest')\n", " plt.title('%i' % index)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.13" } }, "nbformat": 4, "nbformat_minor": 0 }