{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "オリジナルの作成: 2016/05/03" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\tHiroshi TAKEMOTO\n", "\t(take.pwave@gmail.com)\n", "\t\n", "\t

SageでTheanoのロジスティック回帰を試す

\n", "\t

参考サイト

\n", "\t

\t\n", "\t\tここでは、\n", "\t\t 人工知能に関する断創録\n", "\t\t のTheanoに関連する記事をSageのノートブックで実装し、Thenoの修得を試みます。\t\n", "\t

\n", "\t

\n", "\t\t今回は、Theanoのロジスティック回帰を以下のページを参考にSageのノートブックで試してみます。\n", "\t\t

\t\t\n", "\t

\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

2値分類

\n", "\t

\n", "\t\t2値分類とは、入力xの値によって2種類に区別する問題です。2値の変数$y \\in {0, 1}$で表現した場合、\n", "\t\t入力xからyの値を推定します。\n", "\t

\n", "\t

\n", "\t\t入力xを指定したとき、y=1となる事後確率$p(y=1|x)$をニューラルネットを使ってモデル化します。活性化関数gにはロジスティック関数を使用します。\n", "$$\n", "\th(x) = p(y=1|x) \\approx g(x; w)\n", "$$\t\t\n", "$$\n", "\tg(z) = \\frac{1}{1 + e^{-z}}\n", "$$\n", "\t

\n", "\t

\n", "\t\tべき乗の性質を使ったトリックを使うと事後分布$p(y|x; w)$をy=1とy=0の分布を使って\n", "$$\n", "\t\tp(y|x) = p(y=1|x)^y p(y=0|x)^{1-y}\n", "$$\t\t\n", "\t\tと表すことができます。ここで$p(y=0|x) = 1 - h(x)$であることを使うとwの尤度は、\n", "$$\n", "\t\tL(w) = \\prod_{n=1}^N p(y_n | x_n; w) = \\prod_{n=1}^N \\{ h(x_n)\\}^{y_n} \\{1 - h(x_n) \\}^{1-y_n}\n", "$$\t\t\n", "\t\tで与えられ、負の対数尤度は、以下のようになります。\n", "$$\n", "\t\tE(w) = - \\sum_{n=1}^N [y_n log\\, h(x_n) + (1-y_n) log\\{1 - h(x_n)\\}]\n", "$$\t\t\n", "\t

\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

必要なライブラリをインポート

\n", "\t

\n", "\t\t最初に使用するライブラリをインポートします。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import theano\n", "import theano.tensor as T" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

訓練データのロード

\n", "\t

\n", "\t\t訓練データex2data1.txtをdataディレクトリから読み込みます。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# 訓練データをロード\n", "data = np.genfromtxt(\"data/ex2data1.txt\", delimiter=\",\")\n", "data_x = data[:, (0, 1)]\n", "data_y = data[:, 2]\n", "\n", "# 訓練データ数\n", "m = len(data_y)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "\n", "\t

データの分布

\n", "\t

\n", "\t\ty=1を赤、y=0を青の●(point)でプロットし、データの分布をみます。\n", "\t

\n", "\t

\n", "\t\tSageのGraphicsとpointを使うと簡単にデータの分布が可視化することができます。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEeCAYAAACDq8KMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XlcVPX6B/DPwICAAuI2MG6gKKLmhlcDt1IMtdQWNMyl\nBb2llKLlUl2XbnrVMvlZWlnm0nVB73XJJXAhFSUEUUFNU0gMvTrggjOCMizz/f3xDYYBF5Bz5pzD\nPO/Xi1fnzBzOeRr1PHO+y/NVMcYYCCGE2Bw7qQMghBAiDUoAhBBioygBEEKIjaIEQAghNooSACGE\n2ChKAIQQYqMoARBCiI2iBEAIITaKEgAhhNgoSgCEEGKjqp0Ajhw5gmHDhqFp06aws7PDzp07Kx0z\nZ84caLVauLi4YODAgcjIyLB4Pzc3F6NHj4a7uzs8PDwwfvx45OfnP/n/BSGEkGqrdgLIz89Hly5d\nsGLFCqhUqkrvL168GMuXL8fKlSuRnJyMunXrIiQkBIWFhWXHvPbaazh//jzi4uKwZ88exMfH4+23\n367Z/wkhhJBqUdWkGJydnR127NiBYcOGlb2m1Woxffp0TJ06FQBgMBig0Wiwbt06jBw5EufPn0eH\nDh1w4sQJdO3aFQCwd+9ePP/887h69So8PT1r+L9ECCGkKgTtA8jMzIROp8OAAQPKXnNzc0PPnj2R\nmJgIADh27Bg8PDzKbv4AEBwcDJVKhaSkJCHDIYQQ8giCJgCdTgeVSgWNRmPxukajgU6nKzumSZMm\nFu/b29ujQYMGZcdUxBiDwWAAVa4mhBDhWGUUEGPsgf0FVT3m7t27cHd3x927d8UIjxBCbJJayJN5\nenqCMYbs7GyLp4CcnJyyJh9PT0/k5ORY/F5JSQlyc3MrPTlUFBYWBrXaMuRRo0Zh1KhRAv0fEEKI\n7RA0Afj4+MDT0xNxcXHo1KkTAN4JnJSUhIiICABAYGAg7ty5g1OnTpUlhbi4ODDG0LNnz0eePzo6\nGm5ubkKGXG1XrwLjxwOXLwNjxwIffyxpOIQQ8sSqnQDy8/ORkZFR1h5/6dIlpKWloUGDBmjevDki\nIyMxf/58+Pr6wtvbG7Nnz0azZs0wfPhwAEC7du0QEhKCCRMm4JtvvkFhYSHee+89jBo1ShEjgN54\nA4iL49v/+AfQsSPw1/8aIYQoC6umQ4cOMZVKxezs7Cx+3nzzzbJj5s6dy7y8vJizszN77rnnWHp6\nusU5cnNz2ejRo5mbmxurX78+Gz9+PMvPz3/oNfV6PQPA9Hp9dcMVnLc3Y4D554svpI6IEEKeTI3m\nAViLwWCAu7s79Hq95E1As2YBixfz7Xr1gJQUwM9P0pBIbXPxIlCnDtCypdSRkFqOagFV06JFwKZN\nwMKFQHKyFW7+hYU840REAEeOiHwxIrnwcP6Xytsb+PRTqaMhtZyingAGDx4MtVptWyN/3noLWLOG\nbzs48KzTpYu0MRFxpKYC5SZIQqUCbt8G6teXLiZSqwk6CkhschgFZHX795u3i4qAw4cpAYjNZOLD\nvBo0sO7N90HzYB4zf4aQmqAmILmr+I2Qbv7Cu3YN2LYNOHcOKCgAgoOB1q0BrRbYtct6cXTuDEyc\nyLdVKt7e6O5uvetXxcKFQL9+wHvvAffvSx0NqSFFNQHJoRPY6nJzgenTgawsPvFg7FipI6pdLlwA\ngoJ4U4taDbz9NrBihfn91q2BCuXMRffnn4CjI+DlVbXjMzOB2FigVSsgJES8uFav5n0Upd59F/jq\nK/GuR0SnqCYgm+ThAaxaJXUUtdfq1fzmDwDFxZZNbgBQUmL9mKoz+ic9HejRA7hzh+8vWgTMnClO\nXKdPP3qfKA41AT3CmTPAt98Cx45JHQkRjYeH5X6bNvyJAOBDMZcssX5M1bF9u/nmD/CEJpaQEMs+\niUGDxLsWsQpFPQGU1gKyxiigo0eBAQP4KEw7Oz70c+RIUS9JpDBlChAfD+zdC/j7A19+CbRoAfz+\nO9C4MfCY+lSSa9bs0ftCGjwY2L0b2LcPeOopy+YgokjUB/AQEyfyb/+lBg0CYmKscmkiBZOJZ3ql\nYQyYNg2IjuZ9AOvXAz4+UkdFFEJRTwDWpNVa7jdtKk0cxEqUePMHeJNMVBT/IaSaKAE8xPTpfFRg\nXBzQrZu5/AMhhNQW1ARECCE2SqHPvYQQQmqKEgAhhNgoRSWAsLAwDBs2DJs2bZI6FEIIUTzqAyBE\n7vLy+Jj7xESgTx/g++8BFxepoyK1gKKeAAixSXPnAlu2AFeuABs3AgsWSB0RqSUoARAid1lZj96X\ni5s3gT17+IpmRBEoARAid6NHmyeq2dsDclwMKTOTl4d44QWgY0dgxw6pIyJVQAmAELl78UW+HOjS\npUBCAjBkiNQRVbZ6NaDT8e2iIuCzz6SNh1SJomYCW7MYHCGyEhRkrlIqRxUHZ8htIRvyQKI8AeTl\n5SEyMhLe3t5wcXFB7969kZKSYnHMnDlzoNVq4eLigoEDByKjCotuREdHY+fOnXTzJ0Ru3n0X6NuX\nb7u5ATNmSBsPqRJREkB4eDji4uKwYcMGnD17FgMHDkRwcDCuX78OAFi8eDGWL1+OlStXIjk5GXXr\n1kVISAgKCwvFCIcQIjaVCrh6lW8bDHzYKi0ZKX9MYPfv32dqtZrFxMRYvB4QEMBmz57NGGPMy8uL\nLV26tOw9vV7PnJyc2ObNmx94Tr1ezwAwvV4vdLiE2KZDhxhr354xb2/G1q2r+fnOn2eMF6c2/5w7\nV/PzElEJ/gRQXFyMkpIS1KlTx+J1Z2dnHD16FJmZmdDpdBgwYEDZe25ubujZsycSExOFDocQUlFR\nEe9YPncOuHwZeOutmq973KwZ0KSJeb9JE6B585qdk4hO8ARQr149BAYG4tNPP8X169dhMpmwfv16\nJCYm4vr169DpdFCpVNBUWGlJo9FAVzqKgBAijpISvgZy+WUkS0qAa9dqdt569fh6yi+/DLz0Et+u\nV69m5ySiE6UPYP369WCMoWnTpnBycsLy5cvx2muvwd7e/qG/wxiDqvx6o8T2MMZX5iLi2LCB35S1\nWqBtW/PrHToA3bvX/PydOgFbtwLbtvFtInuiDAP18fHBwYMHcf/+fRgMBmg0GoSFhcHHxweenp5g\njCE7O9viKSAnJwddu3Z95HlLh4GWR0NCa4n//Id3HBqNwLx5wIcfSh1R7VJQwJt6SgdaXLzIS0o0\naMAnllFtIZtklWJwubm5aNWqFZYsWYLw8HBotVpMnz4dU6dOBYCyJPHjjz9ixIgRlX6fisHVcgUF\ngIcH/2+p334D2reXLqbaxmCoPDY/IUHecwusLT+fz2B2cQGGD1fuMqHVIMr/4b59+7B3715cvnwZ\n+/fvR//+/eHv74833ngDABAZGYn58+dj165dOHPmDMaNG4dmzZph+PDhYoRDqquoiH8LDw3lMzzF\nVlBgefMHLNuoSc25ufGx+qUGDgR69pQuHrkpKODzGMaM4f0YY8ZIHZF1iDG0aMuWLax169bMycmJ\nabVaNnnyZGYwGCyOmTt3LvPy8mLOzs7sueeeY+np6Q89Hw0DtbL337cczrdli/jXnDDBfL1+/Rgr\nLBT/mrbo118ZO3CAsaIiqSORl8OHKw9jvXNH6qhER+sBkMp69QJ+/dW8P3Uqr0MjtoMH+eSh4GDA\n0VH86xFS6vffeZNj6e3Q1RW4dQtwcJA2LpHV/kYuUn19+jx6XyzPPssLndHN37r0emD3buDkSakj\nsR69HvjHP4DJk/l8iHbtgOXLgYYNgRYt+KCEWn7zBxS2ItjgwYOpGFxFJhNw6BCfiv/MM/y/NVVc\nDHz+OXD2LL8hjx5d83MSebp1i/cF/PEH3//yS+C996SNyRr69AGOHuXbDRrwQQeentLGJAFFJQBq\nAqqAMWDECD72GgBefRWIjpY2JqIs330HvP22eb9ZM77yWG1mNAJOTpav7d4NPP+89WIoLuYL6DRu\nzNd4kAg1ASnZxYvmmz8AbN5s/iZHSFVUHBpav740cVhTnTq8yaeUkxPg72+962dkAG3aAF5efMKc\nhBUQKAEoWb16lmOV7e1p+n1VLFsG9OsHTJjAx8fbshEjgNdf53+PtFpg1SqpI7KOmBj+xDxoELBz\nJ9CqlfWuPXcur8EE8P6HRYusd+0KFLUgDKmgaVPeZjttGt9ftgyoUGOJVLBtGxAZybfj4/n473//\nW9qYpFJSwr80rF3Lb/xqG7odeHtL11xqNFruV5wDY0X0BKB0ERHAvXv85513pI5G/tLSLPfPnJEm\njlJGI/DNN8CSJUBOjnWuaTIBb7zBm0KaNweOH7etm7/UZs40N7VpNHyYtUQU1QlMo4BIjSUk8Oaf\nkhK+P2MGsHixdPEMHgzExvLtVq2AU6cqL68otOhoy4XlO3cGUlPFvSaxdOMGkJ7O+x48PCQLQ1EJ\ngEYBEUEcPAj89BPg58dHwEhV8+VB9XkOHADKrZUhqNxc/q1/3Tpg0iTz6y1aAH/+Kc41iaxRExCx\nPc8+C/zf/wETJ0pb8KtePcs+G7UaaNlSnGtNmsTHu5d+2/TxMb/3wQfiXJPIHj0BECKllBQ+8Sov\nj89MffVV4a+RkAD07m3ed3YGsrL4682bA926CX9NogjU80OIlLp3B8ReCrXiKJPCQt70RNV3bR41\nARFS2/XrBzz3nHl/3jybqHNDHo+eAAip7dRq4OefeXOTqysttEPKKKoPgIaBEkKIcBSVAKgTmBBC\nhEN9AIQQYqMoARBCiI2iBEAIITaKEgAhhNgoRSWAsLAwDBs2DJs2bZI6FEIIUTzBE4DJZMLs2bPR\nqlUruLi4wNfXF/Pnz6903Jw5c6DVauHi4oKBAwciIyPjseeOjo7Gzp07aQgokd6ZM8Ds2cDKlebK\nooQojOATwRYtWoSVK1fixx9/RPv27ZGSkoI33ngD9evXx7vvvgsAWLx4MZYvX45169bBx8cH//jH\nPxASEoLz58/D0dFR6JAIEdbFi0BgIJCfz/dTU3lNf0IURvB5AEOHDoWnpye+//77stdCQ0Ph4uKC\nH3/8EQCg1Woxffp0TP1rIQSDwQCNRoN169Zh5MiRlc5J8wCIrHz9NV+Ip5SnJ3D9unTxEPKEBG8C\nCgoKQlxcHNLT0wEAaWlpSEhIwJAhQwAAmZmZ0Ol0GFCu5rmbmxt69uyJRLGLYhEihPILij9onxCF\nELwJaNasWTAYDGjXrh3s7e1hMpmwYMEChIWFAQB0Oh1UKhU0Fdau1Wg00Ol0QodDiPD69wdWrADW\nrAGaNQOWL5c6IkKeiOAJYPPmzdi4cSOio6PRvn17pKamYsqUKdBqtRg7duxDf48xBpVKJXQ4hIhj\n0iTLVbUIUSDBE8CMGTPw0UcfYcSIEQCADh064PLly1i4cCHGjh0LT09PMMaQnZ1t8RSQk5ODrl27\nPvLcYWFhUFdYvJoKwxFCyJMRPAHcu3ev0jd5Ozs7mEwmAICPjw88PT0RFxeHTp06AeCdvElJSYgo\n37H2ANHR0dQJXM7x48DYscDNm8C77/Iy74QQUlWCJ4ChQ4diwYIFaN68OTp06ICTJ08iKioK48eP\nLzsmMjIS8+fPh6+vL7y9vTF79mw0a9YMw2mFomoJCwMuXeLbn3wCPPMM/yE24P59nvEvXgRefpl/\nEyCkmgRPAMuXL8fs2bMRERGBnJwcaLVaTJw4EbNnzy47ZsaMGbh37x7efvtt3LlzB3369EFMTIwi\n5wCkpQGZmUCvXkDjxta9dsU+cxqJaEW//ALk5gIhIXxxd2ubPBlYtYpv79gBNGwI/DXSjpCqovUA\namDtWiA8HDCZAK0WSErig0KsZdo0ICqKb/v48Cahhg2td32b9f77wNKlfLtTJ+DXX4G6da0bQ6dO\nfDZyqblzqQ2QVJuiagHJzZIl/OYPANeuARs3Wvf6S5cCu3fz0YjJyXTztwqTCfjyS/P+6dP8acDa\n+vY1b6tUQJ8+1o+BKJ6i1gQuHQUkl5E/Hh6P3reG55+3/jVtmp0d/4O+ccP8mhSZNyqKz0C+eBF4\n6SWg3MRKYgXHjwObNwMtWvDhwGpF3UrLUBNQDZw9CwwfDly+DLzyCn8CUOjfA1IdcXG80/XOHeCD\nD4B//lPqiIg1nT0L/O1vQEEB3//733lRQAWi21UNdOwI/PEHbxWwo8Y02zFgAG/zI9X3yy986FpI\nCNC8udTRPJm4OPPNHwB+/lm6WGqIEoAA6OZPSBV88QV/YgJ4s1lyMtCqlbQxPYmnnrLc79hRmjgE\nQLcumcnI4AM8nJyA114DiouljqiaTp4Ejh0D5NCyePcu7yGPjqaa/XLw3Xfm7Vu3gG3bpIulJvr3\n5/8vffoAo0cDf1U5ViJKADIzaRIf3Wc0Aps2AeWqasvflClAQACvlT9ihLRJ4P59/g/0rbeAUaOA\nB5QZJ1bm5fXofSWZMAGIjwfWr7f+BCABKSoByGlJyIQEXgTy9Glhz1t+cAnAyzwowo0blsMjt24F\nTp2SLp4TJ/gsvVLbtvGJW6Tmbt8Gxo3jQ1G//bbqv7dqFdC9O2/+efdd/ohLJEWjgJ7Apk38yY8x\nwNEROHBAuGHYa9bwyWWMAY0a8daU1q2FObeo9Hr+D7t8U8v589LVyk9P59cunajh5sazqYODNPHU\nJi++CPz0k3k/NpZ36hLFUdQTgFysW2du3Sgs5AlBKG++yfvGNm3iX2AVcfMHAHd3XiO/dBzsnDnS\nLpTSpg0fmqfV8g9x61a6+Qul/AzkB+0TxaBRQE+g4ug1ocs/dO/OfxTn7bd500BJiTT1cSoaP57/\nEGENHsyTPcAfgUsnoTHGZyUTxaAE8AQWL+aF106e5H/3339f6ohkxNlZ6giI2JYtA/z8eBXE0FAg\nLw9o2pQ3sUVEmOskEdmjPgBCSM20bAlkZZn39+0DBg6ULh5SZdQHQAipmdu3Lfdv3ZImDlJtikoA\nchoGShTu9Gk+b2HePN6EQZ5c+TZQf3/eR0AUgZqABJaXx+cc/fIL78jduhUot/QxkYMrV/j0fYOB\n7z/3HL+JrV7NK2x+8gkf1USq7uhRPhdkwAA+5JYoAnUCC2zRIiAmhm8nJAAffsjvK0RGkpLMN3+A\nF/f65Rdz3Y30dGDPHmliU6revaWOoPY5f56PNunZU7QFhxTVBKQEFWfyVtwnMtChg+WcgKZNLYsu\nJSRYPyZCyvv+e/6UOmAA0KMHLz0uAkoAAgsPB1xc+LaDA/DOO9LGQx7A3x/Yvp3PXg0L4yUKyicE\n+jZLpPbpp+ZZ7OfOAf/5jyiXoSYggfXowfsXk5J4VU8FV4qt3Z5/3nI5tZgYcx/A3LnSxUUIULnJ\nR6QmIEV1Ag8ePFhWS0JaQ0wMb5IeNAho21bqaAghVnHkCK+5dPs2r6y7aRNgby/4ZQRPAD4+Pvjz\nzz8rvR4REYGvvvoKRqMR06ZNw+bNm2E0GhESEoKvv/4aTZo0eeg5lTQKSEhLlgDTp/PtevX4U0X7\n9tLGRAixEpMJuHdP1LIqgvcBpKSkQKfTlf3s378fKpUKI/+qxx4ZGYk9e/Zg69atiI+Px7Vr1/DK\nK68IHUatsG6deTsvT7nrZxAFMhiA337j6yoQadjZiV5TS/A+gIYNG1rs79q1C61bt0afPn1gMBiw\nevVqREdHo1+/fgCANWvWwN/fH8nJyejRo4fQ4ShaixZ8/eny+4SI7uRJPjfi1i1eSfXwYT5SitQ6\noo4CKioqwoYNGxAeHg6APx0UFxdjQGn1QAB+fn5o0aIFEhMTxQxFkVauBJ59llcfnToVGDtW6ohs\nSEwM/9DXrJE6kppjjBduq+rqQvPmmcs5/PEHEBUlWmiiY4z/f5eOqCEWRB0FtH37duj1erz++usA\ngOzsbDg6OlZqx9doNNDpdGKGokjNmvH5ScTKfv4ZeOEF86IPOTnAzJnSxvSkTCbeibhtGx/qumoV\nL9n9KBVLOiu1xPOtW7wo3alTgLc3L1LXpo3UUcmKqE8Aq1evxuDBg+Hp6fnI4xhjUFXhL1lpLaDy\nP1QXSCGWL+dVI7t0AVJSpI7m0WJjLdczjo2VLpaaio01dx4VFfGlGB/nk0/M69z6+vInISX67DPz\nsqSXLwMffSRpOHIk2hNAVlYWDhw4gB07dpS95unpicLCQhgMBoungJycHGiqUDAnOjrapkYB1Ron\nTgDvvce3s7KAl1+2LB8sN506We4reTJHxaaPqjSFdOkCXLoEXL0K+PjwkgRbtwLdugG9eokTpxgq\ndmDLqUM7L4+vnufkJG0cTCRz585lWq2WlZSUlL2m1+uZo6Mj27ZtW9lrFy5cYCqViiUlJT30XHq9\nngFger1erHCJmHbsYIx/p+Y/9vaMFRdLHdXDmUyMzZ/PWO/ejEVEMHbvntQRPbmiIsaGDDF/7t99\nV73fP3SIMQcH/vsqFWObN4sTpxguXmRMo+Gx16vH2JEjUkfEzZrFY3JwYGzVKsv3Tpxg7NlnGQsK\nYmzvXtFDESUBmEwm1rJlS/bRRx9Vem/ixInM29ubHTx4kKWkpLCgoCDWu3fvR56PEoDC3b7NmI+P\nOQG8/rrUEdmWkhLGzp1j7H//q/7vTphgmbyHDBE+PjHdusXY4cOMXbsmdSRcWprl5+ngwFh+Pn/P\naGSsSRPze87OosctShPQgQMHcOXKFbz55puV3ouKioK9vT1CQ0NhNBoxaNAgrChdX5TUTh4efBbb\nf//Lt/+aE0KsxM6O1z96EhUXwK64L3cNGgB9+0odhVnFZqiiIv4D8Fm/OTmWx/75J+DlJVo4iioF\nYWszgQmRXEEBMGECH44WEAD8+CNQv77UUSmXycRLPOzaxfc/+AD4/HO+zRgQGMi/LAG8/+X0aVEn\ng1ECIIQQazKZgGPHAGdnoGtXy/cMBuDrr3niffttUb/9AwpLALZYDI4QQsSiqARATwCEECIcWhDG\nhhw+zAvMXb8udSQ2Kj2d19jp2rV2lJggikdPADbiiy94fxPAF6k/flx5AzoU76mnzNX9VCr+hxAQ\nIG1MxKbRE4CN+Ppr83Z2Np/YSawsPd28zZjlPhHf/fu8yF/pKBtCCcBWVKy0UYXKG0RoL79s3m7U\nSF7j02u7e/f4Ws9DhgBPP80rntaU0ciHcL7/PpCaWvPzSUBRTUA0CujJnT/P1z+/fBkYM4bXZlNq\nkUfFKiri1Thv3gRGjwZatZI6ItuxYwfw0kvm/Tp1+BNBTf4RjBoFREfz7bp1eRLw9a1ZnFamqARQ\nW/sAMjOBWbP4EOAPPgDKLZdArl/nCyM4OvJKlrXwz59YwcGDQP/+5v3GjS1n3T6J+vUBvd68v3o1\n8IDqB3Im6noApGoGDwYuXODbhw7xb+ve3lJGJBN5ebz6ZGYm3//pJyAxkZc2IKQ6nn0WmDIF+Oor\n/iXi3/+u+Tmfego4epRv29kBHTrU/JxWRk8AErt3jz89lhcTAwwaJE08snLsGJ8aX961a6LPjiS1\nWFERL8MsRPvn9evAtGmATsfLZbz2Ws3PaWX0BCAxFxcgKAj49Ve+37Bh5dnhNqtlS/4B3bvH9zUa\n/gER8qQcHIQ7l5cXoPAFqSgByMCePcCSJbwPYNIkGqFTxsuLN/t88gnvtPv8c94XQAgRBDUBEUKI\njVJUb1rpmsC0DrDyXLjA11l/5hm+5johRHr0BECswseHz0EAeGvO+fP8NUKIdBT1BECUKS/PfPMH\n+ATKjAzJwiGE/IUSABFdvXqWVQ80GhrpRIgc0CggYhV79gDLlgF37wJ//zsvhUMIkRb1ARBCiI1S\nVBMQjQIihBDhiJIArl27hrFjx6JRo0ZwcXFB586dcfLkSYtj5syZA61WCxcXFwwcOBAZVegVjI6O\nxs6dO6kSqIL99huf9VxcLHUkVsIYMHkyrz/ToQNw5ozUERE5W7SId5p5egKxsaJfTvAEcOfOHfTq\n1Qt16tTB3r17cf78eXzxxRfw8PAoO2bx4sVYvnw5Vq5cieTkZNStWxchISEoLCwUOhzR3b/Py4uQ\nx5s/H+jYkdd3Cwmxkc9t2zZegOzuXeDcOcVViyRWdOoU8OGHQH4+X7UpLAwwmcS9JhPYzJkzWd++\nfR95jJeXF1u6dGnZvl6vZ05OTmzz5s0PPF6v1zMATK/XCxprTc2YwZhKxZiTE2MbN0odjbwVFzOm\nVjPGvxLzn9hYqaOyghUrLP+nmzWTOiIiV/v2Wf5dUakYu39f1EsK/gSwa9cudO/eHSNHjoRGo0G3\nbt2watWqsvczMzOh0+kwoFzRezc3N/Ts2ROJiYlChyOalBTgs8/4n1RBAfDWWzbyjbaCmzd5q4bR\n+OjjVCo+Aaw8Z2fx4pKNF18EtFrzfkSEdLEQeevd23KN6L//HXByEvWSgieAS5cu4ZtvvoGfnx/2\n7duHd955B5MnT8b69esBADqdDiqVCpoKFc80Gg10Op3Q4YgmP99y32i0oXbtv+zbxwt2duoE9OgB\n3Lnz8GPt7IAffjAngXfeEXBFxLQ0IDmZZ2O50WqBkyeBH3/ki5LMmiV1RESunJ2B+Hjgv/8F9u4F\nvv1W9EsKPgy0Tp066NGjB44cOVL22pQpU5CSkoKEhAQkJiaid+/euHbtmkUSGDlyJNRqNTZu3Fjp\nnBWXhCxPquUhi4v5Qi4HDvD9mTN5/40t6d4dOHHCvL9kCV8e9VEKCniydHcXKIgPPgC++IJvh4YC\nW7bQWpeEVJHgE8G8vLzg7+9v8Zq/vz+2bdsGAPD09ARjDNnZ2RYJICcnB10fMz00OjpaNvMA1Gq+\ncEtCAu+0L//kZisqLsxVlYW6nJwEfKq9fdt88wf4N6eTJ23zD4OQJyB4E1CvXr1woXR9w79cuHAB\nLVu2BAD4+PjA09MTcXFxZe8bDAYkJSUhKChI6HBEpVYD/frZ7v3ms8/MS/T+7W/A+PFWDkCtBuzt\nLV+ziY4FQgQidK/y8ePHmaOjI/vXv/7FMjIy2IYNG1i9evXYpk2byo5ZvHgxa9CgAdu5cyc7ffo0\nGz58OPP19WVGo/GB55TrKCDCmF7PWEYGY0VFEgXw9deM2dvzURMffihREIQokyilIH7++WfMmjUL\nGRkZ8PFIS1PzAAAdB0lEQVTxwfvvv4+33nrL4ph58+bhu+++w507d9CnTx+sWLECvr6+DzwflYIg\nj5SfzztlBOtYIMQ2UC0gQgixUYqqBUQIIUQ4ikoAVAyOEBt04QKwZg0vlUAERU1AhBD5SkwE+vfn\nE0js7YGtW4Hhw6WOqtZQ1BMAEUZJiW2WrSAKULH42dq1/OYP8L+4339v9ZBqM0oANubHH/nENWdn\nYOFCqaMh5C+7dwMNGvBZgnPnml/39LQ8rkIJGVIz1ARkQ/Ly+L+x8t/+f/8d8POTLiZCYDIB9evz\nktmlkpP57ML8fGDMGOCXX/iMy82bgcaNpYu1lqE1gW1IQUHlpp/y/+YIkURxMf92Ul5pZcG6dYHt\n260fk41QVBMQjQKqmUaNgAkTzPuDBgGPKb9EiPgcHfmqaaV69gT69JEuHhtCTUA26OhRXpHzmWcq\nl9IhRDLx8YDBAAQHi14Hn3CUAAghxEYpqgmIEEKIcCgBEEKIjaIEQAghNooSACGE2ChFJQAaBkoI\nIcKhUUCEyMXPP/NiZ61bA9OnAw4OUkdEajmaCUyIHMTHA0OHmouhXb0KfP21tDGRWk9RTUDENnz3\nHa/51bIlEBsrTQx79wJffAGcPm2lC8bHW1bCPHTIShcmtowSAKm2pCReQqJ1a2DVKmHPnZEBTJwI\n5OQAWVnAiBF81rI1ff01L5PxwQdAjx68Lpnound/9D4hIqAEQKrtxReB1FTg0iXg7beB334T7tw5\nOZZfhPPyKtcJE9v69eZtoxH473+tcNFBg4B164AXXgCmTAG++cYKFyW2TlEJgEYBSc9oBHQ6877J\nBFy5Itz5AwIsv/yGhgINGwp3/qrw9n70vmjGjQN27QL+7/94FUxCRCZ4Avjkk09gZ2dn8dO+ffuy\n941GIyIiItCoUSO4uroiNDQUOTk5VTp3dHQ0du7ciVGjRgkdNqmiOnV4s0yp1q2BoCBhz3/oEPDv\nf/Nv3tHRwp27qpYt41/Evb2BiAj+lENIbST4MNBPPvkEW7duRVxcHEpPrVar0aBBAwDAxIkTERMT\ng3Xr1sHNzQ0RERGwt7fHkSNHHnpOGgYqL8XFwMaNvHBjWBgvM00IUR5RhoGq1Wo0fsCqPQaDAatX\nr0Z0dDT69esHAFizZg38/f2RnJyMHj16iBEOEZhazVsraqMtW4BNm/i3/08/5ctnElJbidIHkJ6e\njqZNm6J169YYM2YMrvzVSHzixAkUFxdjwIABZcf6+fmhRYsWSExMFCMUYkWZmXw0o1JXGTt0iD/R\n7NjBm+HLL55DSG0keAJ4+umnsXbtWuzduxfffvstMjMz0bdvX+Tn50On08HR0bFSM45Go4GufM+i\niE6eBCIj+YLoBQVWuaRN2LqVry3crx8fIlrFbh1ZOX4cKN8gapXhn4RISPAmoJCQkLLtjh07okeP\nHmjZsiW2bNkCp4es8sMYg0qleuy5w8LCoFZbhjxq1KgqdwpnZAB9+/J1pgGeDP7znyr9KnmMf/7T\nvN7wH38Aa9cCM2ZIGlK19e4N2NmZh6H27SttPISITfRSEO7u7mjbti0yMjIQHByMwsJCGAwGi6eA\nnJwcaDSax54rOjq6Rp3ACQnmmz/AZ3sSYbi4WO7bKWqAMRcYCOzeDWzeDPj4ADNnSh0RIeIS/Z9p\nXl4e/vjjD2i1WgQEBECtViMuLq7s/YsXLyIrKwuBgYFih4KOHS1vTJ07i35Jm/Hll0D9+ub9r74C\n/vc/6eJ5UoMH86eXuXNpWVpS+wmeAKZPn474+Hj8+eef+PXXX/HSSy9BrVYjLCwMbm5uCA8Px7Rp\n03Do0CGcOHECb775Jnr16mWVEUABAXyER//+wKhRfMQHEcbf/gZ06mTez8qiyaykCu7f55VPhw4V\nvq4IeSzBm4CuXr2K1157Dbdu3ULjxo3Ru3dvHDt2DA3/ms4ZFRUFe3t7hIaGwmg0YtCgQVixYoXQ\nYTzUyJH8hwiv4jdmR0dp4iAKMnUqsHIl3969m08qefFFaWOyIbQeABFMaiovaZOdzZ+2DhywbBYi\nCmE08vY7rVb8drCAAD4ao9RHHwELFoh7TVJGgV11RK66dOFNP//7Hx9CSTd/BcrMBNq14zU+/Pz4\nkC4xPfOMeVul4uOIidUo6glg8ODBUKvV1Rr6aU2xsUBuLvD88wA9qBBFmjDBsi3+9dd5r7hYiouB\nqCjg/Hlg2DBq/rEyRa0IVtNhoGJ6912gtCujQwfg2DEqIwDwCXfr1/PFXb77DmjWTOqI5KGoiA9L\ndnXlrSCyUVLy6H2hqdW8E5hIgpqABFBSAnz7rXn/t9+Agweli0cudu3iTbrnzgExMcAbb0gdkTwU\nFQEDBwLPPstLX//jH1a8+MmTfMhWmzY8I1c0cybg5cW3NRrgww+tGByxNkoAArC3B/4qdlqmSRNp\nYpGTjAzLfbGbk5Xi8GH+U2rhQvMsatG9+CKQksL/cN55h/fcl+fnB1y4AJw6BVy8CJQr5U5qH0oA\nAtmyBWjRgrf9z58P9OwpdUTS2r+fPwnVqWN+rfw6Aras4lovzs78S4ToiostZ+cxxnvtK3J15T36\nMm1ulaX//Q+4dUvqKKqNEoBAnnkG+PNPQK8HPv5Y6mik9fPPQEgI8MMPfEThkCG8H2DxYqkjk4fA\nQGDaNL7t7AysWWOl0hlqNS93WqplS14AydYYDMKeb8IE3rml0Shu9iONAiKCi4jgC6uX6t8fKFf9\ng/zl/n3AwYHfl62mpATYsAG4cwd49VV+07IVt27xbyYnTvA+kH37ar7e57FjPKOXUqt5wTGFzIKk\nUUBEcBWbjTt0kCYOuXN2luCi9vbireaTkMCbQoKDK3eKycGiRfzmDwDp6XyEwsaNNTtnxVFSJpO5\nnKwCKCoBEGWYNIkvHL9/P29KXrRI6oiI6D77zFw+1dubzwR8wKqAkipfCvhB+08iKAgIDeULWAN8\nFrOCqggqqglITqUgzp/nX3i6dOFD+QixaV5ePOuX+u47+S2pdu4cX+Th1i3eEx8bK0wfCGN8xEPd\nuryOuILY5BNAdjbvqNRqeZNgdf36KzBgAF9RzN6ejwB6+WXh4yREMRo1skwAjRpJF8vDtG/Pv7md\nPs3LXTRtKsx5VSpea16BbG4UkE7HZ16+9RYvXDZ7dvXPsW6deTnJkhI+2oUQm7Z2LdCqFW/+mDSp\nZiUdrl/nj9d6vWDhlWncmH97E+rmr3CKSgBhYWEYNmwYNm3a9MTn2LPHcih0aSXaBykqApYvB+bN\ns5zUpNVaHlc6cZIQmxUQwGf63b/Pa6JUYYnXBzp4EPD15U0zHTs+eJ6CEEwmPl47MBB47z3bXSCc\nKYBer2cAmF6vr/G5du9mjDfa8Z+nnnr4sa++aj6uYUPGrl7lr9+7x9grrzDm4cFYcDBjN27UOCxi\nQ4qKpI5Axvr3t/wHOnOmONeJirK8zvTp4lxH5hT1BCCE55/ngxUaNOBfMNavf/ix27ebt2/dAo4c\n4dvOzrzT//ZtPtJFjs2dRH4uX+ZDYh0c+EjJvDypI7Kuixd5v/CkScCVKw85qOIIGrFG1Jw9++h9\nG2FzCQDgwxJv3QLOnLFcxrAiPz/ztp0d0Lat+LER65g7l1c68PEBjh61zjU/+IAPRAH4xLilS61z\nXTkwGHip/1Wr+GTZ/v0fUv9o8WJzG2v37kBkpDgBPf/8o/dthE2OAqqqbdt4meebN3kzYbduUkdE\nhBAfD/zzn3z77l2+PvRDv5EK6M6dR+/XZunploOEMjJ4X2+LFhUO7NiR11S5fZt32D5pX8LjvPQS\n7xA8eJD3X5QvkWFDaB4AsTlbt/K5O6Xq1LFOH+DPP/P7TmEhb4I8ehTw9xf/unKQm8ufoG/e5Pst\nWvCkoJCKCbWWTTYBEdsWHGzZvBcRYZ3rDhnCmx1/+ok3OVv75n/gAG9dqVsX+PRT617bw4M3e736\nKjBmDN+mm7/0FPUEQMXgiFDu3OGL1DRsCDz3nHjX0et5uRlHR37jK18e29oaNuQtK6WSkoAePaSL\nh0hP9CeAhQsXws7ODtNK698CMBqNiIiIQKNGjeDq6orQ0FDk5OQ89lzR0dHYuXMn3fxJjdWvz9v+\nxbz5FxTwjs9Jk4Dx44GhQ/mYQykUF1fucyhtjiG2S9QEcPz4cXz//ffo3LmzxeuRkZHYs2cPtm7d\nivj4eFy7dg2vvPKKmKEQYnWnTwNpaeb9/ft5GRIpqNWWTV2dO/M1LKypoIA3ATVsyJvhbtyw7vVJ\nZaIlgLy8PIwZMwarVq1C/fr1y143GAxYvXo1oqKi0K9fP3Tt2hVr1qxBQkICkpOTxQqHEKvz8uJj\n/ku5uQHu7tLF8+WXPAn95z+8A9rFxbrXX7KE1826fZv3AcyYYd3rk8pESwAREREYOnQo+vfvb/F6\nSkoKiouLMWDAgLLX/Pz80KJFCyQmJooVDiFW17w5X3ulTRs+AWzHDonWACgnOJiPgKpXz/rXvnbN\ncv/6devHQCyJMg8gOjoaqampSElJqfRednY2HB0dKw3n1Gg00JUfKExILTBiBK2FXGrcOL78ZUEB\nn1gZHi51RETwBHD16lVERkZi//79cCj//PsYjDGoHjPpIywsDOoK6+fRiCDbcvo0L+Dn4cGbEGha\niHI8/TRw8iQv9Nm5M/C3v0kdERE8AZw4cQI3btxAQEAASkeYlpSUID4+HsuXL0dsbCyMRiMMBoPF\nU0BOTg40j1mflJaEtG1Xr/L1PEqrBCck8ImcRDn8/ZU5+a2wkDdheXlJO5RXaIL3AQQHB+PMmTNI\nTU1FWloa0tLS0L17d4wZM6Zs28HBAXHlVgm/ePEisrKyEFh+cWVCKkhJsSwRf/gwH95IiJiuXuV9\nOD4+vFL1xYtSRyQcwZ8A6tati/YVVgWvW7cuGjZsCP+/Un94eDimTZsGDw8PuLq6YvLkyejVqxd6\n0KwU8ggdOvBvX0Yj3+/UiQ9vJERMixaZ1wO5epWvD/K4teTPnuX1pQID+ZwTubLKP5+KbftRUVGw\nt7dHaGgojEYjBg0ahBUrVlgjFKJgbdoAO3cCy5bxPoCFC6WOiNiCkhLL/cc9df7wA/D3v/M1Z3x8\ngGPHgCZNxIuvJhRVCoKKwRFCrC0jg8/ovnaNr/1x4ADvxH6Ytm15obtSy5YBkyeLH+eToAdoQgh5\nBF9f4PffeSLw8Xl8k46rq+W+nL+zKuoJgIrBEULk7vhxYPhwPtEtNBTYtEm+fVWKSgDUBEQIUQqj\nUf5DRmk9AEIIEYHcb/4AJQBCCLFZlAAIIcRGUQIgslZcDERG8mF34eHAvXtSR0RI7SHTvukHKy0G\nR6OAbMfSpXwcNcALwbm5AVFR0sZESG2hqARAxeBsT8W6K7WpDgshUqMmICJrL74IlK8k8vLL0sVC\nSG2jqCcAYnteeIEvH3j4MNC9O98nhAiDJoIRQoiNoiYgQgixUYpKAGFhYRg2bBg2bdokdSiEEKJ4\n1ARECCE2SlFPAIQQQoRDCYAQQmwUJQBCCLFRlAAIIcRGUQIghBAbJXgC+Pbbb9G5c2e4u7vD3d0d\nQUFBiI2NLXvfaDQiIiICjRo1gqurK0JDQ5GTk1Olc9MwUEIIEY7gw0D37NkDe3t7+Pr6AgDWrl2L\nzz//HKmpqfD398fEiRMRExODdevWwc3NDREREbC3t8eRI0ceek4aBkoIIcKzyjyAhg0bYsmSJXjl\nlVfQuHFjREdH46WXXgIAXLhwAf7+/jh27Bh69OjxwN+nBECIMGJi+LoK+fnA3LnAtGlSR0SkJGof\ngMlkQnR0NO7du4fAwECcOHECxcXFGDBgQNkxfn5+aNGiBRITE8UMhRCbV1wMvPoqcP06YDAA778P\nnD0rdVRESqJUAz179iwCAwNRUFAAV1dXbN++He3atcOpU6fg6OhY6Vu8RqOBTqcTIxRCyF8KCoC7\ndy1fu3FDmliIPIjyBNCuXTukpaUhKSkJEydOxLhx4/D7778/9HjGGFTli74TQgRXrx7wxhvm/e7d\ngcBAycIhMiDKE4BarUarVq0AAN26dUNycjKWLVuGkSNHorCwEAaDweIpICcnBxqN5rHnLV0Ssjxa\nHpKQqlu9GggN5X0AL7wAODlJHRGRklUWhDGZTDAajQgICIBarUZcXFxZJ/DFixeRlZWFwCp8FaEl\nIQmpGZUKeP55qaMgciF4Avj4448xePBgNG/eHHfv3sWGDRtw+PBh7Nu3D25ubggPD8e0adPg4eEB\nV1dXTJ48Gb169XroCCBCCCHiEDwBZGdnY9y4cbh+/Trc3d3RqVMn7Nu3D/379wcAREVFwd7eHqGh\noTAajRg0aBBWrFghdBiEEEIeQxHrATDGcPfuXbi6ulJnMSGECEQRCYAQQojwqBgcIYTYKEoAhBBi\noygBEEKIjaIEQAghNspmE4BS1hRQSpwAxSoWpcSqlDgBirUUJQCZU0qcAMUqFqXEqpQ4AYq1lM0m\nAEIIsXWUAB6iKln3cccIcY6qsFYcSonVGnEKdYxcYpXLn39VjpFLrHL5868JSgAPQX9Rn+yYmp5D\nLv+oKAE82TFCnEMpscrlz78mrFINVCqlJSQepLi4GAaD4aG/+7j3be0cSopVLudQUqxyOYeSYpX6\nHEKUxqnVpSBK1xImhJDaRog10mt1AnjUEwAhhCgZPQEQQgh5YtQJTAghNooSACGE2ChKAIQQYqMo\nARBCiI2qtQng22+/RefOneHu7g53d3cEBQUhNja27H2j0YiIiAg0atQIrq6uCA0NRU5OjoQRcwsX\nLoSdnR2mTZtW9ppcYv3kk09gZ2dn8dO+fXvZxVnq2rVrGDt2LBo1agQXFxd07twZJ0+etDhmzpw5\n0Gq1cHFxwcCBA5GRkWH1OH18fCp9rnZ2dnjvvfcAyOtzNZlMmD17Nlq1agUXFxf4+vpi/vz5lY6T\nw+eal5eHyMhIeHt7w8XFBb1790ZKSorkcR45cgTDhg1D06ZNYWdnh507d1Y65nFx5ebmYvTo0XB3\nd4eHhwfGjx+P/Pz86gfDaqndu3ezmJgYlp6eztLT09nHH3/MHB0d2blz5xhjjL3zzjusZcuW7NCh\nQ+zkyZMsMDCQ9e7dW9KYk5OTmY+PD+vSpQubOnVq2etyiXXevHnsqaeeYjk5OSw7O5tlZ2ezW7du\nyS5OxhjLzc1l3t7eLDw8nKWkpLDLly+z/fv3s0uXLpUds2jRIubh4cF27tzJzpw5w4YPH85atWrF\njEajVWO9efNm2eeZnZ3NDhw4wOzs7Fh8fDxjTF6f64IFC1jjxo1ZTEwM+/PPP9nWrVuZq6sr++qr\nr8qOkcvnOnLkSNaxY0d29OhR9scff7B58+Yxd3d3du3aNUnjjImJYbNnz2bbt29ndnZ27KeffrJ4\nvypxDRo0iHXt2pUdP36cJSQksDZt2rDRo0dXO5ZamwAepEGDBmz16tVMr9czR0dHtm3btrL3fv/9\nd6ZSqVhSUpIksd29e5e1bduWxcXFsWeeeaYsAcgp1nnz5rGuXbs+8D05xckYYzNnzmR9+/Z95DFe\nXl5s6dKlZft6vZ45OTmxzZs3ix3eI02ZMoW1adOmLCY5fa4vvPACGz9+vMVrr7zyChs7dmzZvhw+\n1/v37zO1Ws1iYmIsXg8ICGCzZ8+WTZwqlapSAnhcXOfOnWMqlYqdPHmy7JjY2Fhmb2/Prl+/Xq3r\n19omoPJMJhOio6Nx7949BAYG4sSJEyguLsaAAQPKjvHz80OLFi2QmJgoSYwREREYOnQo+vfvb/F6\nSkqKrGJNT09H06ZN0bp1a4wZMwZXrlwBANl9prt27UL37t0xcuRIaDQadOvWDatWrSp7PzMzEzqd\nziJeNzc39OzZU7K/AwBQVFSEDRs2IDw8HID8/vyDgoIQFxeH9PR0AEBaWhoSEhIwZMgQAPL5XIuL\ni1FSUoI6depYvO7s7IyjR4/KJs6KqhLXsWPH4OHhga5du5YdExwcDJVKhaSkpGpdr1bXAjp79iwC\nAwNRUFAAV1dXbN++He3atcOpU6fg6OhYaRq1RqOBTqezepzR0dFITU2t1D4JANnZ2bKJ9emnn8ba\ntWvh5+eH69evY968eejbty/Onj0LnU4nmzgB4NKlS/jmm2/w/vvv4+OPP0ZSUhImT54MJycnjBkz\nBjqdDiqVChqNRhbxltq+fTv0ej1ef/11APL68weAWbNmwWAwoF27drC3t4fJZMKCBQsQFhYGALL5\nXOvVq4fAwEB8+umnaNeuHTQaDTZu3IjExES0adNGNnFWVJW4dDodmjRpYvG+vb09GjRoUO3Ya3UC\naNeuHdLS0nDnzh1s3boV48aNQ3x8/EOPZ4zVeGp1dV29ehWRkZHYv38/HBwcqvx7UsQaEhJStt2x\nY0f06NEDLVu2xJYtW+Dk5PTA35EiToA/9fXo0QOffvopAKBz58747bff8M0332DMmDEP/T2p4i21\nevVqDB48GJ6eno88Tqo4N2/ejI0bNyI6Ohrt27dHamoqpkyZAq1Wi7Fjxz7096SId/369XjrrbfQ\ntGlTqNVqdOvWDa+99lqlgQDlSf3n/zBVietJYq/VTUBqtRqtWrVCt27dsGDBAnTu3BnLli2Dp6cn\nCgsLK1XYy8nJqZR5xXbixAncuHEDAQEBcHBwgIODAw4fPoxly5bB0dERGo0GRqNRFrFW5O7ujrZt\n2yIjI0NWnykAeHl5wd/f3+I1f39/ZGVlAQA8PT3BGEN2drbFMVJ+rllZWThw4AAmTJhQ9prcPtcZ\nM2bgww8/xIgRI9ChQweMHj0aU6dOxcKFC8vilcvn6uPjg4MHDyI/Px9XrlzBsWPHUFhYCB8fH1nF\nWV5V4vL09Kw0CqykpAS5ubnVjr1WJ4CKTCYTjEYjAgICoFarERcXV/bexYsXkZWVhcDAQKvGFBwc\njDNnziA1NRVpaWlIS0tD9+7dMWbMmLJtBwcHWcRaUV5eHv744w9otVpZfaYA0KtXL1y4cMHitQsX\nLqBly5YAUHYTKB+vwWBAUlISgoKCrBprqdWrV0Oj0ZS1pwOQ3ed67969St8y7ezsYDKZAMjzc3V2\ndoZGo0Fubi727t2LF198UZZxAlX7/AIDA3Hnzh2cOnWq7Ji4uDgwxtCzZ8/qXbBaXcYK8tFHH7Ej\nR46wy5cvszNnzrBZs2Yxe3t7FhcXxxhjbOLEiczb25sdPHiQpaSksKCgIMmHgZYqPwqIMfnE+sEH\nH7DDhw+zy5cvs4SEBBYcHMyaNGnCbt68Kas4GWPs+PHjzNHRkf3rX/9iGRkZbMOGDaxevXps06ZN\nZccsXryYNWjQgO3cuZOdPn2aDR8+nPn6+lp9uCJjjJlMJtayZUv20UcfVXpPTp/rG2+8wZo3b872\n7NnDLl++zLZt28YaN27MPvzww7Jj5PK57t27l8XGxrLMzEy2b98+1qVLFxYYGMiKi4sljTMvL4+l\npqayU6dOMZVKxaKiolhqairLysqqclyDBw9mAQEBLDk5mR09epS1bduWjRkzptqx1NoEEB4eznx8\nfJiTkxPTaDRs4MCBZTd/xhgrKChg7777LmvYsCGrV68eCw0NZdnZ2RJGbPbss89aJAC5xBoWFsaa\nNm3KnJycWPPmzdmoUaMsxtXLJc5Se/bsYU899RRzdnZm7du3Zz/88EOlY+bOncu8vLyYs7Mze+65\n51h6eroEkTK2b98+Zmdn98Dry+lzzcvLY1OnTmXe3t7MxcWF+fr6sjlz5rCioiKL4+TwuW7ZsoW1\nbt2aOTk5Ma1WyyZPnswMBoPkcR46dIipVCpmZ2dn8fPmm29WOa7c3Fw2evRo5ubmxurXr8/Gjx/P\n8vPzqx0LlYMmhBAbZVN9AIQQQswoARBCiI2iBEAIITaKEgAhhNgoSgCEEGKjKAEQQoiNogRACCE2\nihIAIYTYKEoAhBBioygBEEKIjaIEQAghNooSACGE2Kj/B+wFKZLN4q8LAAAAAElFTkSuQmCC\n", "text/plain": [ "Graphics object consisting of 100 graphics primitives" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# データのプロット\n", "data_plt = Graphics()\n", "for i in range(m):\n", " if data_y[i] == 1:\n", " data_plt += point(data_x[i], rgbcolor='red')\n", " else:\n", " data_plt += point(data_x[i], rgbcolor='blue')\n", "show(data_plt, figsize=4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

コスト関数の自動微分

\n", "\t

\n", "\t\tバイアス項もwに追加するために、入力データxの1列目に1の列を追加します。\n", "\t

\n", "\t

\n", "\t\t従って重みwは、$ w = [w_0, w_1, w_2]$となります。\n", "\t

\n", "\t

\n", "\t\tTheanoでコスト関数で定義します。\n", "\t\t

\n", "\t\t人工知能に関する断創録のコメントは鋭く、T.dot(X, w)のXとwの順序が式と逆ではないかと指摘しており、\n", "\t\tこれはXの行がサンプル、列が特徴をデータをまとめた行列なので、この順番になるのだそうです。\n", "\t

\n", "\t

\n", "\t\tg_wは、コスト関数をパラメータwで微分した数式を表している。\n", "\t\tこのようにとても簡単に微分を定義することができるのがTheanoの特徴です。\n", "" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# 訓練データの1列目に1を追加\n", "data_x = np.hstack((np.ones((m, 1)), data_x))\n", "\n", "# 訓練データを共有変数にする\n", "X = theano.shared(np.asarray(data_x, dtype=theano.config.floatX), borrow=True)\n", "y = theano.shared(np.asarray(data_y, dtype=theano.config.floatX), borrow=True)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# パラメータを共有変数にし、0で初期化\n", "# 訓練データに1を加えたのでバイアスもwに含めてしまう\n", "w = theano.shared(np.zeros(3, dtype=theano.config.floatX), name='w', borrow=True)\n", "# コスト関数として負の対数尤度を定義\n", "h = T.nnet.sigmoid(T.dot(X, w))\n", "# 通常のコスト関数とは異なり1/mがファクターとして掛けてある。これがないと計算がすぐにnanとなり求まらない\n", "cost = - (1.0/m)*T.sum(y * T.log(h) + (1 - y) * T.log(1 - h))\n", "\n", "# コスト関数の微分\n", "g_w = T.grad(cost=cost, wrt=w)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

Theanoによる勾配降下法の実装

\n", "\t

\n", "\t\t学習率learning_rateと更新updatesにwの更新を定義し、theanoのfunctionとしてtran_modelを定義して、\n", "\t\tこれを繰り返し実行するだけで勾配降下法が実現できてしまうところが本当にすごいです。\n", "\t\t

\n",
    "\t# 訓練用の関数を定義\n",
    "\ttrain_model = theano.function(inputs=[], outputs=cost, updates=updates)\n",
    "\t\t
\t\n", "\t\tfunctionで指定したupdatesは、関数の実行後に指定された更新が自動的に行われる仕組みになっており、\n", "\t\tupdatesには(共有変数, 更新式のシンボル)のペアが並ぶリストを指定します。\t\n", "\t

\n", "\t

\n", "\t\ttrain_modelの戻り値には、cost関数の戻り値が返されます、これを収束の判定に使うことができます。\n", "\t

\n", "\t

\n", "\t\t今回は収束判定はしないで、30万回パラメータの更新を繰り返します。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# モデルでnanが返された時に、以下のdetect_nanを使うことで、エラーになったデータの場所が特定できます!\n", "# train_model = theano.function(inputs=[], outputs=cost, updates=updates, mode=theano.compile.MonitorMode(post_func=detect_nan))\n", "# \n", "def detect_nan(i, node, fn):\n", " for output in fn.outputs:\n", " if (not isinstance(output[0], np.random.RandomState) and\n", " np.isnan(output[0]).any()):\n", " print '*** NaN detected ***'\n", " theano.printing.debugprint(node)\n", " print 'Inputs : %s' % [input[0] for input in fn.inputs]\n", " print 'Outputs: %s' % [output[0] for output in fn.outputs]\n", " break" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# 勾配降下法\n", "# パラメータ更新式\n", "learning_rate = 0.001\n", "updates = [(w, w - learning_rate * g_w)]\n", "\n", "# 訓練用の関数を定義\n", "train_model = theano.function(inputs=[], outputs=cost, updates=updates)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0 0.69314718056\n", "20000 0.547295463694\n", "40000 0.488295709188\n", "60000 0.445118809496\n", "80000 0.41259440171\n", "100000 0.387389522711\n", "120000 0.367353248365\n", "140000 0.351070505109\n", "160000 0.337586093857\n", "180000 0.326237734121\n", "200000 0.316554335209\n", "220000 0.308192817261\n", "240000 0.300897944917\n", "260000 0.294476164256\n", "280000 0.288778154733\n" ] } ], "source": [ "# 高度な収束判定はせずにiterations回だけ繰り返す\n", "iterations = 300000\n", "for iter in range(iterations):\n", " current_cost = train_model()\n", " if iter%20000 == 0:\n", " print iter, current_cost" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

結果をプロット

\n", "\t

\n", "\t\t求められた重みwからy=0とy=1の境界を求めます。\n", "$$\n", "\tw_0 + w_1 x_1 + w_2 x_2 = 0\n", "$$\t\n", "\t\tから\n", "$$\n", "\tx_2 = -w_0/w_2 - w_1/w_2 x_1\n", "$$\t\t\n", "\t\tとなります。これをSageのPlot関数を使って表示してみましょう。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "w: [-9.25573205 0.07960975 0.07329322]\n" ] } ], "source": [ "# 更新されたパラメータを表示\n", "t = w.get_value()\n", "print \"w:\", t" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEeCAYAAACDq8KMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XlcVHX3B/DPwIiAAuI2gCngvm+QhnuKJrnlTzNMNEst\nFfdssXIr1yezejK1xbVU9Mkl0nCJ3FMUF8Q0RcXQdMAFAReQ5fz+OME4qAhyZ+4Mc96vly+5M5d7\nD8Mw597vcr4aIiIIIYSwOXZqByCEEEIdkgCEEMJGSQIQQggbJQlACCFslCQAIYSwUZIAhBDCRkkC\nEEIIGyUJQAghbJQkACGEsFGSAIQQwkYVOQHs3bsXPXv2RJUqVWBnZ4fw8PCH9pkyZQq8vLzg7OyM\nzp0749y5c0bPJycnY8CAAXBzc4O7uzuGDh2KO3fuPP1PIYQQosiKnADu3LmDpk2b4uuvv4ZGo3no\n+blz52LBggX45ptvcOjQIZQpUwYvvPAC7t+/n7fPq6++itOnTyMyMhJbtmzBnj178NZbbxXvJxFC\nCFEkmuIUg7Ozs8OmTZvQs2fPvMe8vLzwzjvvYPz48QCA1NRU6HQ6rFixAv369cPp06fRoEEDHDly\nBM2aNQMAbNu2Dd26dcPly5fh4eFRzB9JCCFEYSjaBxAfHw+9Xo9OnTrlPebq6oqWLVviwIEDAICD\nBw/C3d0978MfAAIDA6HRaBAVFfXI4xIRUlNTIYVLhRBCOYomAL1eD41GA51OZ/S4TqeDXq/P26dy\n5cpGz9vb26N8+fJ5++SXlpYGNzc3pKWlKRmuEELYNLOMAiKiR/YXFHUfIYQQytEqeTAPDw8QERIT\nE43uApKSkvKafDw8PJCUlGT0fdnZ2UhOTn7oziG/4OBgaLXGIffv3x/9+/dX6CconJ9/Bi5eBLp3\nB2rUMOuphRBCMYomAF9fX3h4eCAyMhKNGzcGwJ3AUVFRCA0NBQAEBATg1q1bOHbsWF5SiIyMBBGh\nZcuWBR4/LCwMrq6uSoZcZNOmAdOnG76OjpYkIISwTk81DDQmJgbHjx8HAFy4cAExMTG4dOkSAGDc\nuHGYMWMGfvnlF8TGxmLQoEF45pln0KtXLwBA3bp18cILL2DYsGE4fPgw9u/fj9GjR6N///5WMQJo\n5UrD17duAY+YBiGEENaBimjXrl2k0WjIzs7O6N/rr7+et8/UqVPJ09OTnJycqEuXLhQXF2d0jOTk\nZBowYAC5urpSuXLlaOjQoXTnzp3HnjMlJYUAUEpKSlHDVVz79kSA4d9PP6kdkShRMjKIfv6ZaOtW\nopwctaMRJVyx5gGYS2pqKtzc3JCSkqJ6E1B8PDB4MPcBhIQAM2ea+ISXLgFvvgkkJACDBgHvvWfi\nEwrVZGUBgYHA7t28HRIC/PCDujGJEs2qEkBQUBC0Wq0qHb+q6dgR2LnTsL15M9Ctm3rxCNM5dAjI\n3w+WlARUqqROPKLEU7QT2NQsoRPY7PLVUcL58+rEUdJdvw64uwP29sCSJcCyZcAzzwBffgk8YXSa\nYtzdAY2GWxcBwNERcHY2z7kL6/hx4PffgcaN+W5FWDWpBmrp+vUzfO3iAnTtql4sJVFKCvDcc3yV\n7e0NLF8ODB0K7N8PrF3LzTDmUqsW8MUX/KHv7s7NP2XKPPn7btzgsckxMaaN78ABvkN5+22gc2fg\n++9Nez5hclbVBGQJfQBmRwSsWsV9AL17A/XqqR1RyfLxx8DUqYbt+vWBU6cM2x4ewNWr5o+rsK5c\n4Q/ly5f57uH774E33jDNucaP5wSVq317YNcu05xLmIVVNQGZ2+rVwG+/Ac2bA6Gh/PdldhqNea9C\nbc29e8bbZcoATk6Gxy29v2X1av7wB/hiYd480yUAb++Ct4XVsaoEkDsT2BydwGvWAAMG8NfLlgG3\nbwPvv2/SUwo1jBgB/Pgjf4g6OQGzZnHzS1gYULUqMHKk2hEWzN3deLtcOdOda9Qo4K+/gG3bgEaN\ngPnzTXcuYRbSBPQYb70FfPutYbtLF37fixLo1i0gNhaoXh2oUkXtaIomK4uvVH76CahWjfsC/p2F\nL8STWNUdgDn5+xsngGefVS8WYWLlygFt26odxdPRarmzOjubRzAJUQSSAB5j2DBu9omM5D6AyZPV\njkiIAsiHv3gK0gQkhBA2yibnAWRmqh2BEEKoz6oSQHBwMHr27Ik1a9Y89TEyM4E2bYCPPgIeWKde\nCMt2+DCwcCFw9KjakYgSxOaagLKygDlzuKZ/w4Zc3rlRI4UCFcIUNm8GXnqJO3q1WmDLFh6WJkQx\nWdUdgBK0Wr76j4riuwF/f2DuXP7bEsIirVhheINmZVluhdC0NOCPP3h2srAKNpcAcjVvzqt5jR0L\nTJrEs9rz110TwiI884zxtiXOVbhyBWjSBGjdGqhZE9ixQ+2IRCHYbAIAuNjif/7D5ddz37+LFhmK\nMQphEaZPB3r1AipXBvr0scwxyd98w4tlAFxGI3fdVGHRTJIAbt++jXHjxsHHxwfOzs5o06YNoqOj\njfaZMmUKvLy84OzsjM6dO+NcIS6/legEfpS2bbmQYkgIz/zv2tVQXkUI1bm6Aps2AYmJPOO3MBVC\nza10aeNtR0d14hBFY4plxvr160cNGzakffv20fnz52natGnk5uZGV65cISKiOXPmkLu7O4WHh1Ns\nbCz16tWLqlevThkZGY88njmXhPz1VyJPT6Jy5Yh+/FFW5ROiUFJSiJ55xrBW6ogRakckCkHxBHDv\n3j3SarUUERFh9Lifnx9NnjyZiIg8PT1p/vz5ec+lpKSQo6MjrV279pHHNPeawDduEPXvz+/jPn2I\nkpLMclohzOfSJaKhQ4kGDCA6ckSZ4z24WDZAlJBQ/OMKk1K8CSgrKwvZ2dkone+W0MnJCfv27UN8\nfDz0ej06deqU95yrqytatmyJAwcOKB3OUylfnqvsrl3LqzE2bAiEh6sdlRAKITIs6LJqFdCpEy89\nWRyPml0pMy4tnuIJoGzZsggICMAnn3yCq1evIicnBz/++CMOHDiAq1evQq/XQ6PRQJdvmT2dTge9\nXq90OMXSrx9w8iQXguvVi8usp6aqHZUQxXD0KBARwWWdc926ZbwIztPw9eXS2rmGD+fqqsKimaQT\n+McffwQRoUqVKnB0dMSCBQvw6quvwr6AglVEBI0qK64UzNMT+OUXvlj63/+40u6Da7QLheTkAOvX\nc23+O3fUjqZkeucdwM+PF7kpW9bwuLs7r4RWXAsXclnt2FgeTicsnkmqgfr6+mLnzp24d+8eUlNT\nodPpEBwcDF9fX3h4eICIkJiYaHQXkJSUhGbNmhV43NwFYR5kjsVhNBpgyBCgY0fg9df5/7Fjgdmz\neQ0RoYD+/YF16/hrPz9g3z4ZSaKk1FReLSzX7ds8u9jFhdf4rVxZmfM0bKjMcdSSkwPY2dDoeHN0\nNNy8eZPKlStH33//PRE9vhN43bp1j/x+c3cCFyQ7m2j+fKLSpYnq1CGKilI7IhPZs4do4UKi06dN\nf66bNx/uQNy71/TntSV37xKVKmX8Gh8/rnZUlmXWLCIHByJXV6KfflI7GrMwSQLYtm0bbd26leLj\n42n79u3UtGlTCggIoKysLCIimjt3LpUvX57Cw8PpxIkT1KtXL6pZs6ZFDAMtrFOniPz9ieztiSZP\nJnpM6NZp6VLDh4STkzKjRAqSkUFUtqzhnHZ2RGfOmPactmjpUkMSeO89taOxLLGxxsnRyamE/VE/\nmkkSwLp166hGjRrk6OhIXl5eNGbMGEpNTTXaZ+rUqeTp6UlOTk7UpUsXiouLe+zxLDEBEBHdv080\nbRqRVkvUrBm/h0qENm2M/xgmTjT9ObduJapWjahSJaJFi0x/Plt17x5RWpraUViePXsevgvN95lV\nEtlcNVBTOHIEGDgQOH8emDEDmDDByhdoCg7mMbC5Pv8cGDdOvXiEaR06BKxZwzWHxowBSpVSOyLT\nO3WKO/JSU7lzvFcvIDAQ2LOHn3/rLWDxYnVjNAOrSgBBQUHQarVm6fgtqnv3uMro559zPawVK8w0\nCu7ECeDjj7mneupUZTrh9Hqui3HyJBAUxIsj28KHgi36808uiZueztuDBwPLlqkakln4+gIXL/LX\nWi2/1319uYidszPw/POqhmcuVpUALPUO4EG7d/Pf0LVrwGefAW++yZ/NJpGWBtSowScDAA8PLmlq\nibVihGVauBAIDTVse3gAV6+qF485ZGQ8PMJsyxbgxRfNc/7sbGDmTC5H3KEDMH68CT8kCmZD453M\no317vijv35/nwrz4IvDPPyY6WUKC4cMf4Ct3qWL3ZDt3cpPWwoU87M+WNWpk/OHTuLF6sZhL6dJc\n8TGXpyfQooX5zj9zJt+t//ILD8FVsalJEoAJuLgA333HCzkdP85/Y2vWmKDMtK8v4O1t2K5e3Xhb\nPGz/fi6D8OWXfOU7aZLaEann2jX+4Fu2jK9cQkJ4Ip4t2LgRmD+fP4gPHAAqVjTfuQ8fLnjbjCQB\nmFC3bty02KUL8Oqr3Ld644aCJ3B25janUaO4827XLpk89STbtxsv/7Ztm3qxAFziefRorjMSE2Oe\nc2ZmAt278+SvypWBqlX5vfPDD0ClSuaJQW2Ojtz0Mm2a+S+a2rcveNuMrKoPwJI7gZ8kLIzXGihd\nmstKdOumdkQ2asMGXlQlV0iIukssNmnCbYYAl2Q4fRrIVydLcStXAq+9ZtiuW5fPK8yDiEtlHD7M\nfQAP/i7MzKoSgDV0AhfkyhVg6FCuxTVkCN+BWvGPY72+/JKbAGrX5vIIav0SUlKAcuWMH/vtN67O\nqbSYGODrr/ln9fLitudcPj6G1byETZEEYGZEfAcwYQI3Oy5fruodoFBb/fqGq29XV/7ay0vZc1y9\nyue5dYu3/f2B+/f5zkOr5T6AkBBlzymsgvQBmJlGAwwbxhdkVavycOMJE3gegbBB27dz+3+/fjwG\nXekPf4BHIuR++AM8/HDXLu4QP3tWPvxtmNwBqCg7G/jiC+DDD3kAz8qVfHEmhKL+/pvvAO7e5e2G\nDblks7B5cgegInt7boo9coTLSj/3HA9KkIWUhKK8vbnj6aWXgEGD+GshYGV3ANY8CuhJMjO5jtDM\nmUDTpnw3oMQaHUII8ThWlQBKWhPQoxw+zBdp8fGcDMaNs/LCckIIiyVNQBbm2Wd52dYRI4CJE3n1\nMRmhJ4QwBUkAFsjJiauK7tzJ/XeNG3NpCcu/VxNCWBNJABasQwceqt2vH1cV7d695BdqFEKYj+IJ\nICcnB5MnT0b16tXh7OyMmjVrYsaMGQ/tN2XKFHh5ecHZ2RmdO3fGuXPnnnjs4OBg9OzZE2vWrFE6\nbIvl6gosWQKEh/NooYYNDWunC5XZeiVRYf2UXmJs5syZVKlSJYqIiKC///6b1q9fTy4uLvTVV1/l\n7TNnzhxyd3en8PBwio2NpV69elH16tWtak1gNVy7RtS3L69WFxxMdOOG2hHZqOxsojfe4AWhn3mG\n6NAhtSMS4qkongC6d+9OQ4cONXqsT58+NHDgwLxtT09Pmj9/ft52SkoKOTo60tq1ax95TEkABjk5\nRKtWEZUrR+TpSfTrr2pHZIPWrjVeO7ZRI7UjEuKpKN4E1KpVK0RGRiIuLg4AEBMTg/379+PFf1fb\niY+Ph16vR6cHCl65urqiZcuWOHDggNLhlDgaDZeWPnmSO4dffJH7B9LS1I7MhiQnF7wthJVQPAG8\n//77eOWVV1C3bl04ODjAz88P48aNQ3BwMABAr9dDo9FAl6/krU6ng16vVzqcEqtKFZ7QuXgxsHo1\nVxXeu1ftqGxEnz7GCz5PnKheLEIUg+IJYO3atVi9ejXCwsJw7NgxrFixAp9++il+eELNdSKCRqV1\nMa2VRgO89RYXlvPy4qqiEyca1vcWJlKxIvfIb9rEM/fGjlU7IiGeilbpA7777rv44IMP8PLLLwMA\nGjRogIsXL2L27NkYOHAgPDw8QERITEw0ugtISkpCs2bNCjx2cHAwtFrjkEtiWYjCIuKFZq5d47uA\ntWuBjz7iO4OVKwE/P7UjLMHKlQN69VI7CiGKRfEEcPfu3Yeu5O3s7JDz75A5X19feHh4IDIyEo3/\nXYA6NTUVUVFRCA0NLfDYYWFhJb4URFEMHw58+y1/PWcOcOwYEBQEDBzIheUmT+Ylb0uVUjdOYSJ7\n9wJxcTxd3MdH7WiENVK6V3nw4MFUtWpV2rJlC128eJE2bNhAlSpVokmTJuXtM3fuXCpfvjyFh4fT\niRMnqFevXlSzZk2rHAaakEC0fz/R7dvmP7eTk/FglNWr+fGMDKKPPiKysyPy9yc6dcr8sZV4J08S\n7dtHdP++OudfuNDwiy9Xjuivv9SJQ1g1xRPA7du3afz48eTj40POzs5Us2ZNmjJlCmVmZhrtN3Xq\nVPL09CQnJyfq0qULxcXFPfaYlpoANm4kcnDgv8E6dYiuXzfv+evWNU4A+/YZP3/wIFHt2kSOjkSf\nf87D14UCZs0yvOjt23PGNbcmTYx/+R9/bP4YhNVTPAGYgqUmgMaNjf8G580z7/lPnCBq0YLIx4fo\ns88evc+dO0RjxnB8HToQxcebNcSSJzvbkPVz/23ebP44XnjBOIZvvzV/DMLqWVUtIEsrBeHoaLxd\nurR5z9+oERAVxdVCJ0x49D7OzrwGemQkcOECzx1YskQKyz01O7uHf/HOzuaPY/FioEUL7oweMoSX\nlRTm89FHvKZr69ZAIcrYWCy1M1BhWOodwB9/EFWoYLi6vntX7YgKdusW0eDBHG/37kRXr6odkZX6\n6SdDB8ybb6odjXXJzCS6eVPtKIpn40bju6/WrdWO6KlZ1R2ApQkIAK5cARITgd9/5zLOlszNDVi2\nDPj5Z+DQIS4s99NPakdlhfr04dm/t24B33yjdjTW4+BBwNMTKF+eh6tlZKgd0dO5dMl4OyFBnTgU\nIAmgmBwcgMqVeVKWEq5eBbp2BWrUAN55xzRNNT17cimJ9u2Bl18GBgxQsJpBdDTwxx+WUSnz9m3g\nhx84y2VnK3vs0qU5o4rCGz0auH6dv966FVixQt14nlaPHpzEcg0erFooxSUJwMKMGAFs28bt9fPm\n8YQuU6hUiT8Xf/gB2LKF7wa2bSvmQUeP5iXNWrcG+vZVt6MhPZ0z3KBBnOVsdLKgSRU1yd+9a7x9\n545ysZiTjw/PBF+wgOu0f/yx2hE9NatKAJbSCZyZCXz4Id/FfvaZssfOv/zjxYvKHv9BGg0QEsJ3\nAw0a8J3HiBF84Vxk167xH0SujRt5ZppaoqN5bc1c//ufFG1TyvHjgK8v3/4OGFD4u6spU4Dcmfy1\na3NytlY+PkBoKN8NWDO1OyEKw9I6gSdNMt0IvP/8x3BcZ2eimBjljl2QnByeW+TsTFS9OtHevUU8\nwK1bXB//wRdGzclJZ87wTLjcWFxc1BmvXxK1bGn8e16+vPDfe/480a5dRGlppotPFJpV3QFYiuho\n4+0jR5Q79jvv8F3lvHncUftvtQyT02j46j8mBvDwANq1A959twiF5dzcgIUL+QpPowGmTQPq1DFl\nyAWrXZvjqVwZ8Pbm9i4HB/XiKUmKUw67enVumitbVtmYxFPREFn+iPDU1FS4ubkhJSXFImoBzZnD\nNXZyrV3L6/aWFNnZnIAmT+bP0R9+AJ5Qp88gPZ0PUKaMSWMUKvr2Wy5ERcRj4deu5YR/7RowapTM\nSbAikgCeAhHPwzl6FOjUCfh3qYMS58QJLix36hQwdSrw/vuGJlxh444dA/7+G2jbFggM5H4BgO/+\nDh7kSWrC4kkCEAW6fx+YPp3vevz9eVSSmi07wgKVKWM8wufHH7lzWFg8q+oDsJRRQE8SEwN8/TVQ\nEla4dHAAZs4E9u3jpt6mTYH//tcyhvkXy+zZvLBL7dr8i8rOBs6c4WYMUTS9exu+Ll+e2/iFVZA7\nAIXt2QN07sxXznZ2vGDLv2vjWL07d7gZaMEC4PnneVaxt7faUT2FAweAVq0M21Wq8G3N779zxlu5\nEnjlFfXiszZZWcB333HyfPVVoGZNtSMShWRVdwDW4Mcf+cMf4Kvk5ctVDUdRZcoAX30F7NjB65A0\nasQ/n+VfQuSTf+1pvZ4//AH+5Y0fb/6YrJlWy0PIpkyRD3+lHD3K5QBKl+bX1kR/ZJIAFPbMM8bb\nVauqE4cpBQYCsbF85//668BLL3E9JKvRsSNQq5bx9oNkbWqhtjfe4HIA9+/ziJMNG0xyGkkACnv3\nXb4L9vAAunXjpuaSqFw5LuWycSO3qDRsCKxfr3ZUheTmxnW0V6zgSRebN/NwLoCbgD7/XN34hMit\nmfS4baUoPbPMx8eHNBrNQ/9GjRpFRETp6ek0cuRIqlChApUtW5b69OlDiYmJBR4zdyZwUFAQ9ejR\ng1bnrn1YwqWkEL31FlGnTkSLFqkdzeMlJhL17s2TQkNCiJKT1Y7oKWRn8+zha9fUjkQIok8/Ncy0\n9vbmPzITULwT+MaNG8h+oDZIbGwsunTpgl27dqFt27YYMWIEIiIisGLFCri6uiI0NBT29vbYu3fv\nY49pTZ3AShowAFi92rAdHm65pUeIeMLY6NGAiwuwdCnQpYvaUQlhxf74A7h8me9OK1QwzTlMklYe\nMHbsWKpVqxYR8ZW8g4MDbdiwIe/5v/76izQaDUVFRT32GJZWC8hc6te3vmVfExKIAgM53pEjiW7f\nVjsiUWSpqXwr17Ah0bhxRFlZakckTMSkfQCZmZlYtWoVhgwZAgCIjo5GVlYWOuW2twKoU6cOqlWr\nhgMlYdC8wh68gra3f7iv0hJVrcplpRcs4GGiTZvyhYzVOXyYZ79FRKgdiTKuXAFSUgq373vv8XC2\nkyeBL77goV/W7NYtKxyqZh4mTQAbN25ESkoKXnvtNQBAYmIiHBwcHmrG0el00Ocfmicwbx7//YWG\n8odq69ZqR1Q4dnYc8/HjPNeqbVueP5Bx4zbX0rZ0e/fyPIFJk4AXX7TuVb9ycngthCpVuDBeYSZR\nxsUVvG0tbt7k9Snc3Xl46vnzakdkcUyaAJYuXYqgoCB4eHgUuB8RQSND7x5ibw+MHctX0w/cNFmN\n2rX5s/STT4D5n2bh2YoXEOPalocOWbKffuLJTbnCwtSLpbi2bzfEf/8+jyl/kr59DV/b2fE4X2v0\n6aeG0r0XLgAffKBuPBbIZKW9EhIS8Ntvv2HTpk15j3l4eOD+/ftITU01ugtISkqCTqd74jGDg4Oh\nzVeNrH///ugvqz1ZLK0W+KDVLnTLGYuB+AHPpu/BtP5z8O7t3pZbWK569YK3rUn+O67C3IG99Rbg\n5cWTkdq3BzZtAt5+m4tBffWV9VR6zb/imKWsQHblCs+cdnICRo58uDT2vn28XnKHDnwVaEqm6lyY\nOnUqeXl5UXZ2dt5jj+oEPnPmjHQCl3SbNxMBlA4Heh+zyA5Z9NxzOXTmjNqBPUZWFtGoUUS1ahH1\n6UN086baET29+/cNvfJ2dkQLFhTt+x9coQggGjvWNHGawpkzRJUqcdxlyvBCNGpLSeFhnbmvZ+vW\nxs+/+abhuaAgHp5sQiZJADk5OeTt7U0ffPDBQ8+NGDGCfHx8aOfOnRQdHU2tWrWiNm3aFHg8SQBW\nLj2dqE2bvDf2/jeXUY0aRE5ORF99ZfL3uMjKIjp2jCg+vujfO2yYcQIIClI8PJO6do0oMpLo0iW1\nI2F79hi/noBh7sm1aw8/d/iwScMxSQLYvn072dnZUVxc3EPPpaen06hRo/ImgvXt27fQE8EkAVix\njAyi3bvz1ri8fZuHiQI80S0hQeX4xKNt2UKk0Rg+kL77Tu2IrFtCApGjo+H19PAgyszk59LSiEqV\nMuuyqlINVKhq+3Yue3L7NpeZHjhQSvFYnF27+J+fn+XORLQmERE8MsLJCfjsMx4rnWvlSu6Dyczk\nfR5cetAErCoBBAUFQavVSsdvCZOcDIwZw0PPX3qJR11Wrqx2VEKoJDubh++WKmXyU1lVApA7gOLL\nzDTL++qpbNjAFz8aDSeBB9cZKTF++YXr5vfsyZMkhFCRVAO1EUeO8Czd0qV5XtAD5Zosxv/9H08+\nbdWKv37tNZ7EWWKMHcsf/EOG8Jq5N2+qHZGwcZIAbMRbb3FdKSKeF7RqldoRPZpOx/PEli3j4eeN\nGgG//aZ2VAr59lvD1/HxJegHsxIZGdyXceKE2pFYDEkANiL/lbQlX1lrNMDgwbzoTO3avMTm6NHG\n645bJU/PgreF6aSn88Sq558HmjThOk/FlZbGC4AMGsSJxRqZdIyRQmxhPYC0NNOWov/2W8NoPh8f\nk5UXV1x2NtF//8sj52rVIjpwQO2IiuHQIaJ69Xhy0owZakdjWzZtMh5e6ehIlJNTvGN262Y4noMD\n0cmTysRqRlaVAErqPIDlyw3Df4cPN915YmKIfvnFyia2LltG5OlJp706UovayWRnRzRpEk8rEKLQ\nduwwTgAVKhT/mG5uxsdcurT4xzQzGQWksqwsLgWSkWF4bP9+7gi1eRcvchXHf3uss5xdMWfidUyf\nVQoNGvCQ6caN1Q1RWJG33uJ+mDJluCpqcec0dO3KZXoBHloXHW11b0jpA1BZTo5x4UnAOBnYtKQk\no+FK2rup+GjkTRw6xA/7+3NTriWOaBIW6JtveMbhrVvKTGhbswYYNYqrp4aHW92HPyAJQHUODsD0\n6YbtF18E2rVTLx6L0qQJ13PP1bMnoNOhWTO+2Bo/niv8tm1rvSXrhZmVKQPFytC6u3N11P/9j+8G\nrJA0AVmIv/7iQQV+flyCvSS6f5/vdpydi/BNd+4A69fzBIY+fR764923j+cL6PVc/n3ECCklIURh\nWVUCkFIQ1mvNGq75k57OpeXnzVPu2LdvA++8AyxezMtoLlkCPPOMcscXoqSyqgRQku8ASrKsLMDF\nhT/8cx06ZNy6o4StW3mS7Z07vIragAFyNyBEQUpoY4OwJFlZD3ds376t/Hm6duVSEt26cVXRvn25\n7I4Q4tEkAQiTc3TkReFzderEHbem4O7OZS7+9z9g926gYUMeoCGEeJg0AQmzOXqUr/xbtVJuIEZB\n9Hpg2DBLR34QAAAgAElEQVRg82YuLfHFF4Cbm+nPK4S1MMkdwJUrVzBw4EBUrFgRzs7OaNKkCY4e\nPWq0z5QpU+Dl5QVnZ2d07twZ586de+Jxg4OD0bNnT6xZs8YUYQsTa96ch7iaazF4Dw+++l+yhAcS\nNW4M/P67ec5tJCUF2LIFiIlR4eTCqqSnc8lwc9UWUnpqcXJyMvn4+NCQIUMoOjqaLl68SDt27KAL\nFy7k7TNnzhxyd3en8PBwio2NpV69elH16tUp4zHz+0t6KQhbcfky0WuvEb30EtHOneY9d3w8Ufv2\nPGN/zBiiO3fMdOLr14lq1OATazREX39tphMLq5OeTvTcc4bSEqNHm/yUiieA9957j9q1a1fgPp6e\nnjR//vy87ZSUFHJ0dKS1a9c+cn9LTQB//EHUpw/RoEGypm1hNG5seG87ORE9cE1gFtnZRJ9/znXA\natcmOnjQDCddtMi4Xky1amY4qbBK+esVaTRE9+6Z9JSKNwH98ssv8Pf3R79+/aDT6dC8eXN8//33\nec/Hx8dDr9ejU6dOeY+5urqiZcuWOHDggNLhmMw///CY8/XruSbNCy+oHZE67t3j1yInp+D9MjON\ny7Dfu8cjdszJzg4YN477IlxduS/io494gprJ5O90kE4I8Tju7sbbZctyqQATUjwBXLhwAYsWLUKd\nOnWwfft2DB8+HGPGjMGPP/4IANDr9dBoNNDpdEbfp9PpoNfrlQ7HZE6dMh7KePo0z+S1Jfv3A1Wq\n8KSrtm15/P3jlCplXODO1RVo1kyBIIiAyEjg1185yxRCvXrAH38AU6cCc+cCLVuaMBm98ophQoKn\nJ/DddyY6kbB6fn7AtGn8oZ87nM3UZQGUvqVwcHCgNm3aGD02ZswYatWqFRER/fHHH2RnZ0d6vd5o\nn5dffpn69+//yGNaYhOQXk/k7m64W2vWTO2IzK9FC+M71i++KHj/mzeJJk4kGjaM6OhRhYIYPNgQ\nQKdORJmZRfr26Gii+vW5nPvcuURZWQrFld/9+yY6sChxsrPNdirFx2N4enqiXr16Ro/Vq1cPGzZs\nAAB4eHiAiJCYmGh0F5CUlIRmT7gkDA4OhjbfEBK1ykLodDzO/L//5Tu1Dz4wewiqy3/B/aSmFHd3\nrtejmGvXgOXLDduRkcCxY0WaYuznx+slf/QRz1UIDwdWrABq1FAwToBvgYQoDDMWA1M8AbRu3Rpn\nzpwxeuzMmTPw9vYGAPj6+sLDwwORkZFo/G/51NTUVERFRSE0NLTAY4eFhVnUPIBGjWz7jv6TT3i2\nbXo6N6u88YaZA3B25iJxudOMNRqgXLkiH8bRkWsT9ezJ8wWaNOHtt96SUhKihFP6luLw4cPk4OBA\ns2bNonPnztGqVauobNmytGbNmrx95s6dS+XLl6fw8HA6ceIE9erVi2rWrCnDQK3QlSu80qHZhlXm\nt2YNUdmyRKVLE82bV+zDpaYSvfkmtyh17cpDV4UoqUyyJOSWLVuoUaNG5OTkRPXr16clS5Y8tM/U\nqVPJ09OTnJycqEuXLhQXF/fY40kCEAXKyVG83XTLFiJPT6Jy5YhWrSr+8rFCWCIpBSHEY9y8CYSG\nAmFh3NS1aBFQsaLaUQmhHKsqBielIIQ5lS/P6xisXcslJBo25LpCwsy+/poXA5o9W9b/VJjcAQhR\nCFevAkOH8nSDN94APv+c5zIIE1uyhF/4XNOnA1OmqBdPCWNVdwCi+M6fB0JCgH79ePijKBxPT776\n/+47YN06LixnrnpdNuHqVR521b8/cPCg4fEHvwYAK6oWYA0kAdiQrCwgMNBQLz8wELh+Xe2orIdG\nwxejJ04A3t7A88/zwvT37qkdWQkQFAR8+y13uHTuDFy6xI+3aWO8n6kWkrBRZirMKyzBtWvAxYuG\n7Vu3gLg46dgsKl9fYOdOXl/ggw94KcqVK5Vf4tJmpKcbl8q+fRuIjQWqVgVee42vXH7/nWftjRun\nXpwlkFX1Acii8MWTnc0dmX/9xds6Hdcwyl+DShTeqVO8/GRMDCeDyZNl0u9T8fPjKn0Ad66cOsWF\npoRJWVUCkE7g4rtyBZgzhy+6JkwA6tZVOyLrl5kJzJwJzJjBs4hXrgQaNFA7KiuTlAR8/DGQmgqM\nGQP4+6sdkU2QBCCEQqKjgUGDgAsXOCGMGwfY26sdlRCPJ53AQijE359HVo0cCbzzDncSX7igdlRC\nPJ4kACEU5OQEzJ/PncSXLvFw0W+/5XrVQlgaSQBCmED79jxctH9/Ht7erRv3vwhhSawqAUgpCGFN\nXFx44tjmzbxMQcOGPMy9QLdvy+2CMBvpBBbCDG7c4L6Bdet4FvbChUCFCg/skJEB9O4NREQYph03\nb65avMI2WNUdgLANp04Bw4YBo0dzhQA1JCZyFYIH130ujgoVuKjcmjXAjh18N/Drrw/s8P33/OEP\n8A89erQyJxaiAJIARJHFx/NVbNeu/GGmpBs3uP38+++BBQu4XEVOjrLneJKdO3lJyIAAXvVNybb7\n4GBegL5pU+4XGDYMSEsDcOeO8Y5KZR4hCiAJQBRZUBDXEtq2jZdRjI9X7tinTxvXJzp1ipOCOX38\nseHz+OJFbq5RkpcXX/0vXsx3BI0bA7trDgF8fHgHrdY2F5kWZqd4Apg+fTrs7OyM/tWvXz/v+YyM\nDISGhqJixYpwcXFB3759kZSUVKhjSyew+tLTgQeXfE5PN5SWUELt2oCbm2G7Ro18beVm4OBQ8LYS\nNBoeHRQTAzzzDPB83wp4u9tfSN8SyVnvlVeUP6kQ+Sm9xNi0adOoUaNGlJSURImJiZSYmEg3btzI\ne3748OHk7e1Nu3btoqNHj1JAQAC1adOmwGPKkpCWpU0bXjMXIKpQgUivV/b4hw8Tvfwy0cCBRPHx\nyh67MI4dI9Lp+Ofz8yNKTjbt+bKyiD79lMjBgahePf75hTAHxUcBTZ8+HT///DOO5hZ2ekBqaioq\nVaqEsLAw9O7dGwBw5swZ1KtXDwcPHkSLFi0eeUwZBWRZUlKAzz7jsi3Dh5esekKJiVx48pln+O7D\nwwOwM1ND6cmTXErixAngo4+ADz+UwnLCtEzy1o6Li0OVKlVQo0YNhISE4NK/tb2PHDmCrKwsdOrU\nKW/fOnXqoFq1ajggCz1YDTc3bif/4gvjD/+0NO4PsNZV+y5fBpo1A159FWjXDli/3nwf/gCPDDp4\nkJv/Z8zgTujTp813fmF7FH97P/fcc1i+fDm2bduGxYsXIz4+Hu3atcOdO3eg1+vh4ODw0FW8TqeD\nXq9XOpRHOn4cePtt4D//4aHXQhm7d/NVc/XqQOvW1jmIZd0642GnX31l/hgcHDi5/vEHv4bNmvHy\nk+YeCSVsg+ILwrzwwgt5Xzds2BAtWrSAt7c31q1bB0dHx0d+DxFBo9EoHcpDzp/nBYVyP5yio/mP\nXhTfxIncJAQAUVE8jNPa1u6oVMl4W82Fclq04NnDkyZx2e6ffwaWLzcMFBJCCSZfEczNzQ21a9fG\nuXPnEBgYiPv37yM1NdXoLiApKQk6ne6JxwoODoZWaxxyURaH2bfP+Mp069bC/QziyfI3+/z5pzpx\nFMeAAbzO7+rVvOrXd9+pG4+TEzez9eoFDB7McxK++IIXpTfD9ZKwBabuZU5LS6Py5cvTggULKCUl\nhRwcHGjDhg15z585c4Y0Gg1FRUU99hhKjQI6fJjIzs4wgqVVq2IdTjwgIoJIqzW8tgDRtm1qR1Vy\npKQQvfEGv67duhFduaJ2RAo6c4Zoyxaia9fUjsTmKJ4AJk6cSLt376aLFy/S/v37KTAwkCpXrkzX\nr18nIqIRI0aQj48P7dy5k6Kjo6lVq1ZmHQa6Zg1Rhw5Er7xC9M8/xT6ceICfn3ECmDhR7YhKnvBw\nHqJavjzR2rVqR6OA9esNVw6enkQXL6odkU1RPAEEBwdTlSpVyNHRkapWrUr9+/enCxcu5D2fnp5O\no0aNogoVKlDZsmWpb9++lJiYWOAxZR6AdRg+3DgB/PCD2hGVTNeuEfXty69xcDDRA9NslJGRwRMw\nMjIUPvAjtGxp/KaZMsX05xR5pBqoUMzdu8B77/F49u7debSVMA0iLiMRGsp9BUuWcImOYvv7b17K\nLD4e8PbmSRHVqytw4Mfo0sW4oNRnn3GvtzALq0oAQUFB0Gq1Rer4NZfkZF4H9tYtYMQIwM9P7Ygs\nQ2Ymj2X38AAqV1Y7mpLn8mVgyBBg+3bgzTf587Ns2WIc8M03jXu/Bw8Gli0rbpiPd/o0V8WLjwde\neAHYuJEzmjAPdW9ACscamoAeLI/g4kL0999qR6S+tDQif39+TRwdiX7+We2ILMeuXUQvvcTlLhIS\ninesnByihQuJnJ2JfH2J9ux5wjds2UK0ahX3LOc3eLBxk0xISPGCK6z0dPOcRxiRBKCAzEzjvxmA\naNMmtaNS38KFxq9JnTpqR2QZLl4kcnIyvC4NGihz3Lg4Htmm0XAH/L17j9hp5EjDievXJ0pNNX7+\n9GlDIaTKlYlOnlQmOGGRpBy0ArRa48WbnJx4zLYtO30a2L/f+DF7e3VisTR//gncu2e8nZ5e/OPW\nrAns2QPMmQP897+Avz9gVJIrOxv45hvD9qlTPPHhQXXrcrnX6Gjg7FmgQYPiB2YL7t3j9k4rIwlA\nIZs38wSd//s/rvVuyn4zSxcVxQlx1SrDY2XKAPPnqxeTJWnaFChXzrDdsiXwmEnyRWZvD7z7Ln9+\na7V87E8+AbKy/n0yf23tR03AdHPjTqwH63KXFBkZQFwcj1hQyqRJ/AZ3deWeeWui9i1IYeQ2AQUF\nBVGPHj1o9erVaockCjBqlHHTT+vWj25utmUxMURvvslNNYoP4/xXRgbRhx/y5Mdnn+XWHdq9m8jH\nh6hcOaJZs0xzYkt16RJRzZr8pvTwUKZ5Kzra+M3u6Eh0/37xj2smJi8FoaSwsDAZBmoFPD2Nt2vU\n4IsjYdC4sXFrjCk4OHBV0R49uMx0s2bAnDntMPp8vPJVTm/d4vrV//zDI4deeknhEyhg3jzg3Dn+\nWq8Hpk3jpe2KI/9Snunp3BRkJXW8pQmoGKyx4qU5vP02r31boQLQqRPw6adqR2TbWrbkwnLDhnGB\nvsBAHu6vqJAQXjvz55+BPn2AQ4cUPoEC8pdUVaLEaqtW/ILmeucdwNm5+Mc1E6uaB6DURLDMTJ6s\nVLkyUKVK0b//5k2edHPoEFCnDheVkyqNwhpERgKvv84X7F9+yRfrihSWq1wZuHbNsL1oEa8WZEn+\n/psXekhI4FKvO3Zwh0xxZWVxpckyZYBnny3+8czI5u4A7t0D2rfnTkofH+OOysKaO9dwgXPmjKzf\nLaxHp05AbCxfpL/xBlcaVWQpjg4dDF+XKsWr2TwtvR44cMBQX1wp3t48PO3ECa4Nr8SHP8C97R06\nWN2HP2CDCeB//+P3FsCJ+913H79vWho3a775JnD4sPHj+fcTwlq4ufHk3k2beAWyhg2Bn34q5kGX\nLwcmT+assn070KTJ0x1n1y4ez9qqFY+l/nc1QcU4O/NxnZy4jka9esDAgbbbnqt2L3RhKDkKaPVq\n4057b+/H7/vCC4b9ypQhOn+eH//zT67GCPDsy127njocYWNiYvg9qMZi94+SmEjUuze/lwcMILp5\n07Tny8wkysoqYIdOnYz/QN97zzSBzJljfJ7Ro01zHgtnVXcAYWFhCA8PL1YdoL59ga5d+Wtn54KX\n/fv9d8PXd+4Ymn3q1+c5NNu28R1l+/ZPHY5QyeXLPC9hxQrzrWG8YQM3Pb76Ko8Ciokxz3kLUrky\nr328ciXPZWnUiN/XpjBvHl94lykDLF36mJ0cHIy3lZogkV/uaKBc58+b5jyWTu0MVBhKl4LIyeH6\nK/lnwecXEGC4QChViq/8hfVLTCTy8jL8bgcNMs95O3Y0vugcN8485y2shASiwECObfhwruWklPPn\nuURF7s+u1RIlJz9ixxMneF0AgKh5c9PdkkREGK8OtWyZac5j4azqDkApGg1QtSrg4lLwfhs38oiJ\nnj15dFv9+uaJT5jW7t3AlSuGbXNN3rSkNYcfpWpVvvpfsIDvjJo2fbicx9NKS+NP2lxZWY+ZjNuo\nEY/WuXqVO97c3ZUJIL+uXfmNMGMGEBHBw6FskdoZqDAsvRicsC6HDhlfideqZZ7zXr5M1KIFUenS\nRD17Et29a57z5tq9m6h6daIKFYg+/bTgfc+e5TtgjYbo3XeLX6wzO5uoRw/Daz54cPGOJ5Rh8gQw\na9Ys0mg0NH78+LzH0tPTaeTIkXmrgvXp06fAVcGkFIRQ2sKFRLVr84dcbKzpznPqFNGQIUQjRnAl\nAjVVqmSc+KKjC94/K4to9mxu/mzUiOjYseKdPyuL6LffOBEJy2DSiWCHDx/GK6+8Ajc3Nzz//POY\n/281sBEjRiAiIgIrVqyAq6srQkNDYW9vj7179z7yOLIimLBGyclcXDMpibdr1+bKn1oVCrBkZnJ/\n6oOTXzdv5rVYniQmhktJnD4NTJ3Kq7497c9w7hwPw27c+OlHigoFmSqzpKWlUe3atSkyMpI6dOiQ\ndweQkpJCDg4OtGHDhrx9//rrL9JoNBQVFfXIY0kTkLBGBw48vE7EP/+oF8/QoYY46tV78iCIB6Wn\nE02axP2mLVsSnTlT9PMfPszDqQEie3tZM8MSmKwTODQ0FD169EDHjh2NHo+OjkZWVhY6deqU91id\nOnVQrVo1HMidoSVECVCrFlC+vGHb11fdZTG//ZYHM6xYwVfhTxoE8aDSpYFZs7jiwY0b3EH81VdF\nK6ezbJmhdlp2NleLEOoyyc1oWFgYjh8/jujo6IeeS0xMhIODw0NNOTqdDnpF5qQLYRkqVODaO3Pn\n8vD2adPUaf7JpdHwiLbiCAgAjh/nZqAxY3g28bJlQLVqT/7e/MnvUUsRCPNS/O14+fJljBs3Djt2\n7ECpIpREJSJonlCVKjg4GNp8f0GWuEC8MJ3ff+fVrsqV4ytSLy+1IypY06bWt0bIk5Qpw0NFX3qJ\nh0k3asS/k0GDCi4s9847vELZb79xaeq5c80Xs3gMpduUNm3aRHZ2dlSqVCnSarWk1WpJo9HkPRYZ\nGUkajeah9nxvb2/64osvHnlM6QMQRLzmraOjoR27WTO1IxLJybywPUDUqxdPsiuJ9u8nmjuXKDJS\n7UiUpXgfQGBgIGJjY3H8+HHExMQgJiYG/v7+CAkJyfu6VKlSiIyMzPues2fPIiEhAQHFqSAoSryT\nJ43Xzj127N+lDoVqypXjMhLr1/OksQYNuORFSbJ1K9C2LTd7BQaWrDs6xRNAmTJlUL9+faN/ZcqU\nQYUKFVCvXj24urpiyJAhmDBhAnbt2oUjR47g9ddfR+vWrdGiRQulwxEliJ+f8cpibduq26YuDP7v\n/zhBt27NpaYHDeI1B0qCtWsNnd1EQFhYwfufOwc89xzg4QFMnGj6+IrDLKUg8rftf/755+jevTv6\n9u2LDh06wMvLC+vXrzdHKMKKVa0K7NzJK1u9+y4QHq52ROJBOh2XT1m+nEcbNWrEa65YO1/fgrfz\ne+MNICoKSEwEPvsMWLfOdLEVl02uCCaEMK2EBP4gjIwERo4E/vMf7jy2RhkZ/DPs3s1rvnz3HVC2\n7OP3r1EDuHDBsD1/PjB+vOnjfBpWlQCCgoKg1Wpl5I8QViAnB/j6a247r1KF+wpsoZtv5kzgo4/4\n6/LlgejoJ981qMWqEoDcAQhhfc6e5T6Bw4c5GUydyhPLSrKtW4GLF3ntcG9vtaN5PEkAQgiTy8ri\nZqBp03gVxh9+4HpAQl02uR6AEMK8tFrggw94Vb2cHMDfH5g9W4bxqk0SgLBoOTnA9OlAhw48pO7+\nfbUjEsXRtCm3iU+YwO3k7doBcXFqR2W7rCoBBAcHo2fPnlhTkmZiiAItWMDNBrt385C6qVPVjkgU\nV+nSwJw5wJ49XCq7aVPuLC5KYTmhDKuaRhMWFiZ9ADYm/8LpJ06oE4dQXuvWXFju3XeBUaO4sNzS\npTzfQ5iHVd0BCNvTtWvB28K6lS0LLFzIo2ZOneLJYz/8YLx+sDAdGQUkLN6mTTwD2N8fGDhQ7WiE\nqSQnA6NHA6tWAb17A998A1SqpHZUJZskACGERfnpJ2D4cMDOjhexeekltSMquaQJSAhhUfr25cJy\nzz3HdwKDBwMpKWpHVTJZVQKQUUBC2AYPDy4ot3Qpl5du1IjrCgllSROQEMKi/f033wXs2sV9BHPm\nAM7OakdVMljVHYAQwvZ4e/PV/xdfcCXOZs243LIoPkkAQgiLZ2cHjB3Lq8C5uQGtWvFMYpkZXjyS\nAIQQVqNuXeCPP7g8yNy5QMuWQGys2lFZL8UTwOLFi9GkSRO4ubnBzc0NrVq1wtatW/Oez8jIQGho\nKCpWrAgXFxf07dsXSUlJhTq2dAILIbRavvqPigIyM3l+yNy5QHa22pFZH8U7gbds2QJ7e3vUrFkT\nALB8+XJ8+umnOH78OOrVq4cRI0YgIiICK1asgKurK0JDQ2Fvb4+9e/c+9pjSCSyEMoiAX38F7twB\nune3/s7U9HRgyhRg3jxuFlq+HPj3o0cUBplB+fLlaenSpZSSkkIODg60YcOGvOf++usv0mg0FBUV\n9djvT0lJIQCUkpJijnCFKLEGDybiNEDk7090757aESljzx6i6tWJnJ2JFi4kyslROyLrYNI+gJyc\nHISFheHu3bsICAjAkSNHkJWVhU6dOuXtU6dOHVSrVg0HDhwwZShC2Lzbt/kKOVd0NFBS/uzatuXC\ngQMH8vq9XbsCly+rHZXlM0kCOHnyJFxcXFC6dGmMHDkSGzduRN26daHX6+Hg4PBQM45Op4NerzdF\nKEKIfzk6Ai4uxo+VpFo7ZcsCixdzE1dsLE8eW7VKCssVxCTloOvWrYuYmBjcunUL69evx6BBg7Bn\nz57H7k9E0Gg0TzxucHAwtFrjkGWBeCEKR6sF1q0D3niD+wCmTQMaNlQ7KuUFBXEpiVGjgJAQYONG\nYNGikpXslGKWmcCdO3dGzZo10a9fPwQGBiI5OdnoLsDHxwfjx4/H2LFjH/n90gkshHga69YBI0Zw\n8vvuO6BnT7UjsixmmQeQk5ODjIwM+Pn5QavVIvKBoh5nz55FQkICAgICzBGKEMKG9OvHdwPPPgv0\n6sV3P6mpakdlORRvAvrwww8RFBSEqlWrIi0tDatWrcLu3buxfft2uLq6YsiQIZgwYQLc3d3h4uKC\nMWPGoHXr1mjRooXSoQghBDw9gV9+4cJy48ZxWYnly4Hnn1c7MvUpfgeQmJiIQYMGoW7duggMDMSR\nI0ewfft2dOzYEQDw+eefo3v37ujbty86dOgALy8vrF+/XukwhBAij0YDDBnCS4r6+gIdO3IyuHdP\n7cjUJdVAhRA2JScH+PJLYNIkwMcHWLkSsNUGCKuqBSSlIIQQxWVnB4wfz4XlXFx4BvGUKbZZWE7u\nAIQQNiszE5g1C5gxg+cNrFxZMofGPo5V3QEIIYSSSpUCpk4FDh4EMjIAPz/g009tp7CcJAAhhM3z\n8wOOHOEVx957D+jQAbhwQe2oTE8SgBBCgEtlzJvHS09evgw0bgx8803JLiVhVQlAOoGFEKbWrh0P\nF+3fHxg+HHjxReCff9SOyjSkE1gIIR5jyxZg6FDuH/j6ayA4mOcUlBRWdQcghBDm1K0bl5Lo0gV4\n9VXglVeA69fVjko5kgCEEKIAFSoAYWHAmjXAb7/xMNHNm9WOShmSAIQQohCCg/luoHlzoEcPbhqy\n9sJykgCEEKKQvLy4X+Dbb4G1a4EmTYDdu9WO6ulZVQKQUUBCCLVpNMCwYbwEZdWqXFV0wgTrLCwn\no4CEEOIpZWcDX3wBfPghUL06l5Lw91c7qsKzqjsAIYSwJPb2wNtv8yxiJyfgued4qc3MTLUjKxxJ\nAEIIUUwNGnA9oQ8/5MJyAQHAqVNqR/VkkgCEEEIBpUoB06cDBw4Ad+7waKHPPrPswnKKJ4DZs2ej\nRYsWcHV1hU6nQ+/evXH27FmjfTIyMhAaGoqKFSvCxcUFffv2RVJS0hOPLZ3AQghL9+yzwNGjwMiR\nwMSJvPpYfLzaUT2a4p3AL774Ivr37w9/f39kZWVh0qRJOHnyJE6fPg0nJycAwIgRIxAREYEVK1bA\n1dUVoaGhsLe3x969ex95TOkEFkJYo127gMGDgRs3gPnzee6AJZWSMPkooOvXr6Ny5crYs2cP2rRp\ng9TUVFSqVAlhYWHo3bs3AODMmTOoV68eDh48+MjF4SUBCCGsVWoqr0C2dCkXlvv+e16o3hKYvA/g\n1q1b0Gg0KF++PADgyJEjyMrKQqdOnfL2qVOnDqpVq4YDBw6YOhwhhDArV1dgyRIgPJxHCzVsyJPI\nLIFJEwARYdy4cWjTpg3q168PANDr9XBwcHjoSl6n00Gv15syHCGEUE2PHlxKomNHLisRHMxNQ2rS\nmvLgI0eOxKlTp7Bv374n7ktE0DyhcSw4OBharXHI/fv3R//+/YsVpxBCmEPFisC6dVxYLjSU1yFe\nsgQIClInHpMlgFGjRuHXX3/F3r174eXllfe4h4cH7t+/j9TUVKO7gKSkJOh0ugKPGRYWJn0AQgir\nptFwaen27YEhQ7hfYNgwHjLq4mLeWEzSBDRq1Cj8/PPP2LlzJ6pVq2b0nJ+fH7RaLSIjI/MeO3v2\nLBISEhAQEGCKcIQQwuJUqQJERACLFwOrV3NhuccMhDQZxUcBjRw5EmvWrEF4eDhq166d97ibmxsc\nHR3z9omIiMCyZcvg4uKCMWPGwM7OToaBCiFs0vnzwGuvAX/8wYXlZszgNYpNTfEEYGdn98i2/GXL\nlmHQoEEAeCLYxIkTsWbNGmRkZKBr1674+uuvUbly5UceUxKAEKKky87muQIffQTUrMmF5fz8THtO\nqQYqhBAW5ORJYNAgIDYWmDwZmDSJy0yYglXVApJSEEKIkq5hQy4s9/77PF/AlJVF5Q5ACCEsVEYG\nUJVFHf4AAAquSURBVLq06Y5vVXcAQghhS0z54Q9IAhBCCJslCUAIIWyUJAAhhLBRVtEJTERIS0uD\ni4vLE+sFCSGEKByrSABCCCGUJ01AQghhoyQBCCGEjZIEIIQQNkoSgBBC2CibTQDWUk/IWuIEJFZT\nsZZYrSVOQGLNJQnAwllLnIDEairWEqu1xAlIrLlsNgEIIYStkwTwGIXJuk/aR4ljFIa54rCWWM0R\np1L7WEqslvL7L8w+lhKrpfz+i0MSwGPIG/Xp9inuMSzlj0oSwNPto8QxrCVWS/n9F4dW1bObWG4J\niUfJyspCamrqY7/3Sc/b2jGsKVZLOYY1xWopx7CmWNU+hhKlcUp0KYjchWSEEKKkUWKBrBKdAAq6\nAxBCCGsmdwBCCCGemnQCCyGEjZIEIIQQNkoSgBBC2ChJAEIIYaNKbAJYvHgxmjRpAjc3N7i5uaFV\nq1bYunVr3vMZGRkIDQ1FxYoV4eLigr59+yIpKUnFiNns2bNhZ2eHCRMm5D1mKbFOnz4ddnZ2Rv/q\n169vcXHmunLlCgYOHIiKFSvC2dkZTZo0wdGjR432mTJlCry8vODs7IzOnTvj3LlzZo/T19f3odfV\nzs4Oo0ePBmBZr2tOTg4mT56M6tWrw9nZGTVr1sSMGTMe2s8SXtfbt29j3Lhx8PHxgbOzM9q0aYPo\n6GjV49y7dy969uyJKlWqwM7ODuHh4Q/t86S4kpOTMWDAALi5ucHd3R1Dhw7FnTt3ih4MlVCbN2+m\niIgIiouLo7i4OPrwww/JwcGBTp06RUREw4cPJ29vb9q1axcdPXqUAgICqE2bNqrGfOjQIfL19aWm\nTZvS+PHj8x63lFinTZtGjRo1oqSkJEpMTKTExES6ceOGxcVJRJScnEw+Pj40ZMgQio6OposXL9KO\nHTvowoULefvMmTOH3N3dKTw8nGJjY6lXr15UvXp1ysjIMGus169fz3s9ExMT6bfffiM7Ozvas2cP\nEVnW6zpz5kyqVKkSRURE0N9//03r168nFxcX+uqrr/L2sZTXtV+/ftSwYUPat28fnT9/nqZNm0Zu\nbm505coVVeOMiIigyZMn08aNG8nOzo5+/vlno+cLE1fXrl2pWbNmdPjwYdq/fz/VqlWLBgwYUORY\nSmwCeJTy5cvT0qVLKSUlhRwcHGjDhg15z/3111+k0WgoKipKldjS0tKodu3aFBkZSR06dMhLAJYU\n67Rp06hZs2aPfM6S4iQieu+996hdu3YF7uPp6Unz58/P205JSSFHR0dau3atqcMr0NixY6lWrVp5\nMVnS69q9e3caOnSo0WN9+vShgQMH5m1bwut679490mq1FBERYfS4n58fTZ482WLi1Gg0DyWAJ8V1\n6tQp0mg0dPTo0bx9tm7dSvb29nT16tUinb/ENgE9KCcnB2FhYbh79y4CAgJw5MgRZGVloVOnTnn7\n1KlTB9WqVcOBAwdUiTE0NBQ9evRAx44djR6Pjo62qFjj4uJQpUoV1KhRAyEhIbh06RIAWNxr+ssv\nv8Df3x/9+vWDTqdD8+bN8f333+c9Hx8fD71ebxSvq6srWrZsqdp7AAAyMzOxatUqDBkyBIDl/f5b\ntWqFyMhIxMXFAQBiYmKwf/9+vPjiiwAs53XNyspCdnY2SpcubfS4k5MT9u3bZzFx5leYuA4ePAh3\nd3c0a9Ysb5/AwEBoNBpERUUV6XwluhbQyZMnERAQgPT0dLi4uGDjxo2oW7cujh07BgcHh4emUet0\nOuj1erPHGRYWhuPHjz/UPgkAiYmJFhPrc889h+XLl6NOnTq4evUqpk2bhnbt2uHkyZPQ6/UWEycA\nXLhwAYsWLcLbb7+NDz/8EFFRURgzZgwcHR0REhICvV4PjUYDnU5nEfHm2rhxI1JSUvDaa68BsKzf\nPwC8//77SE1NRd26dWFvb4+cnBzMnDkTwcHBAGAxr2vZsmUREBCATz75BHXr1oVOp8Pq1atx4MAB\n1KpVy2LizK8wcen1elSuXNnoeXt7e5QvX77IsZfoBFC3bl3ExMTg1q1bWL9+PQYNGoQ9e/Y8dn8i\nKvbU6qK6fPkyxo0bhx07dqBUqVKF/j41Yn3hhRfyvm7YsCFatGgBb29vrFu3Do6Ojo/8HjXiBPiu\nr0WLFvjkk08AAE2aNMGff/6JRYsWISQk5LHfp1a8uZYuXYqgoCB4eHgUuJ9aca5duxarV69GWFgY\n6tevj+PHj2Ps2LHw8vLCwIEDH/t9asT7448/4o033kCVKlWg1WrRvHlzvPrqqw8NBHiQ2r//xylM\nXE8Te4luAtJqtahevTqaN2+OmTNnokmTJvjyyy/h4eGB+/fvP1RhLykp6aHMa2pHjhzBtWvX4Ofn\nh1KlSqFUqVLYvXs3vvzySzg4OECn0yEjI8MiYs3Pzc0NtWvXxrlz5yzqNQUAT09P1KtXz+ixevXq\nISEhAQDg4eEBIkJiYqLRPmq+rgkJCfjtt98wbNiwvMcs7XV99913MWnSJLz88sto0KABBgwYgPHj\nx2P27Nl58VrK6+rr64udO3fizp07uHTpEg4ePIj79+/D19fXouJ8UGHi8vDweGgUWHZ2NpKTk4sc\ne4lOAPnl5OQgIyMDfn5+0Gq1iIyMzHvu7NmzSEhIQEBAgFljCgwMRGxsLI4fP46YmBjExMTA398f\nISEheV+XKlXKImLN7/bt2zh//jy8vLws6jUFgNatW+PMmTNGj505cwbe3t4AkPch8GC8qampiIqK\nQqtWrcwaa66lS5dCp9PltacDsLjX9e7duw9dZdrZ2SEnJweAZb6uTk5O0Ol0SE5OxrZt2/DSSy9Z\nZJxA4V6/gIAA3Lp1C8eOHcvbJzIyEkSEli1bFu2EReoytiIffPAB7d27ly5evEixsbH0/vvvk729\nPUVGRhIR0YgRI8jHx4d27txJ0dHR1KpVK9WHgeZ6cBQQkeXEOnHiRNq9ezddvHiR9u/fT4GBgVS5\ncmW6fv26RcVJRHT48GFycHCgWbNm0blz52jVqlVUtmxZWrNmTd4+c+fOpfLly1N4eDidOHGCevXq\nRTVr1jT7cEUiopycHPL29qYPPvjgoecs6XUdPHgwVa1albZs2UIXL16kDRs2UKVKlWjSpEl5+1jK\n67pt2zbaunUrxcfH0/bt26lp06YUEBBAWVlZqsZ5+/ZtOn78OB07dow0Gg19/vnndPz4cUpISCh0\nXEFBQeTn50eHDh2iffv2Ue3atSkkJKTIsZTYBDBkyBDy9fUlR0dH0ul01Llz57wPfyKi9PR0GjVq\nFFWoUIHKli1Lffv2pcTERBUjNnj++eeNEoClxBocHExVqlQhR0dHqlq1KvXv399oXL2lxJlry5Yt\n1KhRI3JycqL69evTkiVLHtpn6tSp5OnpSU5OTtSlSxeKi4tTIVKi7du3k52d3SPPb0mv6+3bt2n8\n+PHk4+NDzs7OVLNmTZoyZQplZmYa7WcJr+u6deuoRo0a5OjoSF5eXjRmzBhKTU1VPc5du3aRRqMh\nOzs7o3+vv/56oeNKTk6mAQMGkKurK5UrV46GDh1Kd+7cKXIsUg5aCCFslE31AQghhDCQBCCEEDZK\nEoAQQtgoSQBCCGGjJAEIIYSNkgQghBA2ShKAEELYKEkAQghhoyQBCCGEjZIEIIQQNkoSgBBC2ChJ\nAEIIYaP+Hzzcb5KDyeu6AAAAAElFTkSuQmCC\n", "text/plain": [ "Graphics object consisting of 101 graphics primitives" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# 決定境界を描画\n", "xmin, xmax = min(data_x[:,1]), max(data_x[:,1])\n", "x1 = var('x1')\n", "x2 = -t[0]/t[2] - t[1]/t[2]*x1\n", "line_plt = plot(x2, [x1, xmin, xmax])\n", "(data_plt + line_plt).show(figsize=4)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false }, "source": [ "\n", "\t

Theanoによる確率的勾配降下法の実装

\n", "\t

\n", "\t\t上記の勾配降下法は、誤差関数に各サンプルの誤差$E_n(w)$の和を求めています。この方法をバッチ学習(batch leaning)と呼びます。\n", "$$\n", "\tE(w) = \\sum_{n=1}^N E_n(w)\n", "$$\t\t\n", "\t

\n", "\t

\n", "\t\tこれに対してサンプルからランダムに1個を抽出し、パラメータの更新を行う方式を確率的勾配降下法(stochastic gradient descent)\n", "\t\tと呼び、SGDと略します。SGDでのwの更新は以下のように行われます。\n", "$$\n", "\tw^{(t+1)} = w^{(t)} - \\epsilon \\Delta E_n\n", "$$\t\t\t\t\n", "\t

\n", "\t

\n", "\t\tTheanoでSGDを実装するには、更新式を少し書き替えるだけで実現できます。\n", "\t\t

\n",
    "\t# 訓練データのインデックスを表すシンボルを定義\n",
    "\tindex = T.lscalar()\t\t\t\n",
    "\t# コスト関数の微分をindex番目のデータのみ使うように修正\n",
    "\th = T.nnet.sigmoid(T.dot(w, X[index,:]))\n",
    "\tcost = -y[index] * T.log(h) - (1 - y[index]) * T.log(1 - h)\n",
    "\t# 訓練用の関数の定義で、inputにindexを指定し、index番目の訓練データのみを使ってパラメータ更新する\n",
    "\ttrain_model = theano.function(inputs=[index], outputs=cost, updates=updates)\n",
    "\t\t
\t\t\n", "\t

\n", "\t

\n", "\t\t人工知能に関する断創録では、5000で計算していましたが、解がばらばなので、10000に増やして計算してみました。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# 確率的勾配降下法\n", "# 訓練データを再度設定\n", "data_x = data[:, (0, 1)]\n", "data_y = data[:, 2]\n", "\n", "# 訓練データの1列目に1を追加\n", "data_x = np.hstack((np.ones((m, 1)), data_x))\n", "\n", "# データをシャッフル\n", "p = np.random.permutation(m)\n", "data_x = data_x[p, :]\n", "data_y = data_y[p]" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# 訓練データを共有変数にする\n", "X = theano.shared(np.asarray(data_x, dtype=theano.config.floatX), borrow=True)\n", "y = theano.shared(np.asarray(data_y, dtype=theano.config.floatX), borrow=True)\n", "\n", "# パラメータを共有変数にし、0で初期化\n", "# 訓練データに1を加えたのでバイアスもwに含めてしまう\n", "w = theano.shared(np.zeros(3, dtype=theano.config.floatX), name='w', borrow=True)\n", "\n", "# 訓練データのインデックスを表すシンボルを定義\n", "index = T.lscalar()" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# コスト関数の微分を構築\n", "# 確率的勾配降下法なので全データの和ではなく、index番目のデータのみ使う\n", "h = T.nnet.sigmoid(T.dot(w, X[index,:]))\n", "cost = -y[index] * T.log(h) - (1 - y[index]) * T.log(1 - h)\n", "\n", "# コスト関数の微分\n", "g_w = T.grad(cost=cost, wrt=w)\n", "\n", "# 更新式\n", "learning_rate = 0.0005\n", "updates = [(w, w - learning_rate * g_w)]\n", "\n", "# 訓練用の関数を定義\n", "# index番目の訓練データを使ってパラメータ更新\n", "train_model = theano.function(inputs=[index], outputs=cost, updates=updates)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0 0.0516469108332\n", "500 0.0511148075267\n", "1000 0.0420380848334\n", "1500 0.0324276124006\n", "2000 0.0249896127378\n", "2500 0.0196073042107\n", "3000 0.0156910643233\n", "3500 0.0127780559225\n", "4000 0.0105645722052\n", "4500 0.00885267037499\n", "5000 0.00750901336009\n", "5500 0.00644070000985\n", "6000 0.00558131061815\n", "6500 0.00488244644119\n", "7000 0.00430831303458\n", "7500 0.00383210612443\n", "8000 0.0034335309684\n", "8500 0.00309706548768\n", "9000 0.00281072555632\n", "9500 0.00256517615274\n", "w: [-13.86585403 0.11494973 0.13224085]\n" ] } ], "source": [ "# 確率的勾配降下法\n", "# 5000だと解がばらばらなので、10000に増やしてみた\n", "max_epoch = 10000\n", "for epoch in range(max_epoch):\n", " for i in range(m):\n", " current_cost = train_model(i)\n", " if epoch%500 == 0:\n", " print epoch, current_cost\n", "\n", "# 更新されたパラメータを表示\n", "t = w.get_value()\n", "print \"w:\", t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\t

決定境界の表示

\n", "\t

\n", "\t\t解は勾配降下法の線の当たりをばらつきます。\n", "\t

\n", "" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEeCAYAAACDq8KMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XlcVGUXB/DfACI7gssA5oKiguZuGmpmKiqVmom+kEuL\nWirulZXmlnulvZaVlVmWC5pL7ivuvoiKuG+4awqEIago63n/OMEwgAhyZ+4Mc76fDx+5M3fuPQ4z\n97nPdh4NERGEEEJYHCu1AxBCCKEOKQCEEMJCSQEghBAWSgoAIYSwUFIACCGEhZICQAghLJQUAEII\nYaGkABBCCAslBYAQQlgoKQCEEMJCFbsA2LdvH7p27YrKlSvDysoK69aty7fPhAkT4OXlBQcHBwQE\nBODixYt6zycmJqJ3795wdXWFm5sbBgwYgAcPHjz9/0IIIUSxFbsAePDgARo1aoRvv/0WGo0m3/Oz\nZs3CvHnz8MMPP+DQoUNwdHREp06dkJaWlrPPG2+8gbNnzyI8PBwbN27E3r178d5775XsfyKEEKJY\nNCVJBmdlZYU///wTXbt2zXnMy8sLH374IUaNGgUASE5OhlarxaJFi9CrVy+cPXsW9erVQ1RUFBo3\nbgwA2Lp1K1555RXcvHkTHh4eJfwvCSGEKApF+wCuXLmC2NhYtG/fPucxFxcXtGjRAhEREQCAgwcP\nws3NLefiDwAdOnSARqNBZGRkgcclIiQnJ0MSlwohhHIULQBiY2Oh0Wig1Wr1HtdqtYiNjc3Zp1Kl\nSnrPW1tbw93dPWefvO7duwdXV1fcu3dPyXCFEMKiGWUUEBEV2F9Q3H2EEEIox0bJg3l4eICIEBcX\np1cLiI+Pz2ny8fDwQHx8vN7rMjMzkZiYmK/mkFdwcDBsbPRDDgkJQUhIiEL/g6JZvx64ehV49VXA\n29uopxZCCMUoWgB4e3vDw8MD4eHhaNCgAQDuBI6MjERoaCgAwN/fH3fv3kV0dHROoRAeHg4iQosW\nLQo9flhYGFxcXJQMudgmTwYmTeLfJ04EDh8GatZUNSQhhHgqTzUM9Pjx4zh27BgA4PLlyzh+/Dhu\n3LgBABg5ciSmTp2K9evX4+TJk+jXrx+eeeYZdOvWDQDg6+uLTp06YeDAgTh8+DAOHDiAYcOGISQk\nxCxGAC1apPs9MREoYBqEEEKYByqm3bt3k0ajISsrK72ft99+O2efiRMnkqenJ9nb21PHjh0pJiZG\n7xiJiYnUu3dvcnFxoXLlytGAAQPowYMHjz1nUlISAaCkpKTihqu4Nm2IAN3PypVqRyRKlbQ0oo0b\niXbsIMrKUjsaUcqVaB6AsSQnJ8PV1RVJSUmqNwFdvgy8+Sb3AfTpA8yYYeAT/vUXMHgwcP060K8f\nMHq0gU8oVJORAXTuDISH8/ZbbwG//KJqSKJ0M6sCIDAwEDY2Nqp0/KqmQwfdBQEANm0CAgPVi0cY\nzuHDQPPm+o/9/TdQoYI68YhST9FOYEMzhU5go7twQX87JkYKAEO4dw9wcgI0GuD334FffwUqVwZm\nzwYqVjRODK6ufP7se7KyZQF7e+Ocu6hOnwZ27wbq1wfatFE7GlFCkg3U1L3+uu53R0egY0f1YimN\n7t0DXngBcHHhMb2LF3Mb386dXBD07Wu8WGrXBr74gi/8zs5cCDk6Pvl1d+8CW7YAZ88aNr5Dh4Bm\nzYChQ4EXX+T4hFkzqyYgU+gDMDoiHnp0/ToXBs8+q3ZEpcvUqcD48brtevX4Ljebhwdw+7ZxY8rK\nAqyKeG8WFwc8/zx3Sllb80W5Tx/DxDVqFPDf/+q2X3yRawPCbJlVE5CxrVrFN4KNGwMDBqgUhEbD\nnYHCMPKmIbe3B+zsgEePeLtTJ+PHVNSLP8A1lqtX+ffMTGD6dMMVAFWqFL4tzI5ZFQDZM4GN0Qm8\nciXQs6duOykJeP99g55SqOHdd/muOTaWm16mTAHc3IClS/kCN3y42hEWLm+N2JA15GHDgFOnuLmp\nQQNgzhzDnUsYhTQBPcZ77wE//qjb7tgR2LrVKKcWxnbnDhAdDfj4ANWrqx1N8aSn853K2rWAlxfn\nKWnSRO2ohJkwqxqAMTVqpL+dK3u1KG3Kl+fhtuaoTBngzz+Bhw9Nb8SQMHlSADzGoEE8uCI8nG+o\nPvtM7YiEKIRc/MVTkCYgIYSwUDIPQAghLJRZFQDBwcHo2rUrli1bpnYoQhjXmTM85PPcObUjEaWI\nNAEJYep27ABeeQVIS+Ohqtu2SRoGoQizqgEIYZF+/JEv/gCQmgosWKBuPI+TmsqzqO/eVTsSUURS\nAAhh6vImozNWcrriSEjg4XLPPstzKfbvVzsiUQRSAAhh6qZMAdq146GeHTsCEyaoHVF+333H/RQA\nT5sfN07deESRGKQAuH//PkaOHInq1avDwcEBrVu3xpEjR/T2mTBhAry8vODg4ICAgABcvHjxiceV\nTmBhkdzdeUJKSgpPR3d1VTui/DSawreFSTJIAdC/f3+Eh4djyZIlOHXqFAICAtChQwfc/jer4qxZ\nszBv3jz88MMPOHToEBwdHdGpUyekZbdzPkZYWBjWrVtnOYvBCGEuhgwBPD112zJ13iwoPgro0aNH\ncHZ2xvr169G5c+ecx5s1a4aXX34Zn332Gby8vPDhhx9i1KhRAHiUj1arxaJFi9CrV698x5RRQEIo\n7M4dXnsgJYWTvNWqVbLj3b4NPPMMp7IGOKPpjRucn0iYLMVrABkZGcjMzETZsmX1Hre3t8f+/ftx\n5coVxMbGon379jnPubi4oEWLFoiIiFA6HCFEXkTclzBrFvDNN7wgzj//lOyYKSm6iz/Av+dNtS1M\njuIFgJOTE/z9/TFlyhTcvn0bWVlZWLx4MSIiInD79m3ExsZCo9FAq9XqvU6r1SI2NlbpcIQQuV27\nBkRGAkeP6h6LiwNOnizZcWvWBHI3zYaElLxWIQzOIH0AixcvBhGhcuXKsLOzw7x58/DGG2/A2tr6\nsa8hImik48iy7d3LnZzp6WpHUjrNnMlDNP399ZeadHJS5mK9ZAn/Dffs4d+FyTNINlBvb2/s2rUL\nDx8+zGnfDw4Ohre3Nzw8PEBEiIuL06sFxMfHo/ETOo6yF4TJzRiLwwgjGDwYmD+ff2/XjgsCG0lW\nq5j794GxY3XbDx4A7dvz6mcffaRMW71Gw81JwnyQEfzzzz9Urlw5WrBgAREReXp60pw5c3KeT0pK\nIjs7O1qxYkWBr09KSiIAlJSUZIxwBRHR6dNEK1cS/fWX4c+VmEjELdO6nwMHDH9eS3L/PpGVlf57\nfPiw2lGZlh9/JPL0JKpZk2jHDrWjMQqDFABbt26lLVu20JUrV2jbtm3UqFEj8vf3p4yMDCIimjVr\nFrm7u9O6devoxIkT1K1bN/Lx8aHU1NQCjycFgJGtXk1kY8MXCXd3orNnDXu+hw+JypbVvzidOmXY\nc1qi2bOJNBp+f/v3Vzsa03LunH4B6epKlJ6udlQGZ5ACYMWKFVSzZk2ys7MjLy8vGj58OCUnJ+vt\nM3HiRPL09CR7e3vq2LEjxcTEPPZ4UgAYWZs2+hfjMWMMf86wMCInJy54pk0z/Pks1e3bRFeuqB2F\n6dm9O38t9O5dtaMyOMkGKvJ77TVeYzbbjBnAxx8b/rxEPHywkMECwgBiYoDVq3kc/xtvWMYs3hs3\nuE8kORkYNQpo0YI7x48f5+eDgoA//lA3RiMwqwIgMDAQNjY20vGb29WrwJdf8pf2ww+BqlVLfswr\nV4Bu3TizY+fOwMqVsuRgaXX5MidxS0ri7ZEjga++UjcmY6hfHzh1in+3t+c8RuXL82fd0RHo0cMi\nbkTMqgCQGkAeKSmAnx9w/Tpve3vzB9nOTpnjE1nG3aAl+/57TuOQzdMTuHVLvXiMITU1/3dkwwZe\nc8FYFiwADh8GXnoJCA423nnzkGyg5uzKFd3FP3v72jXljl9aL/4nTwKffw78+afakaivRo3Ct0uj\nsmW5uSdbuXLGzV303/8CAwfyOg8hIarOmZACwJxVraqfG97Dg9txxeMdO8btvR99BHTvzukQLFmn\nTvwe1KkDdOjAy05ago0bgTFjgPfe44lrxsxZFB5e+LYRSQFgzpyd+cMTFAT07MlLB+ae4Sny+/NP\n4OFD3XZYmHqxADxBa+ZMYPx4/dqcIRHxha9sWcDHBwgI4LWGt2/nmcKWwM2NC77584EGDYx77ry1\njSZNjHv+XGSqpbmrX98iRisoJm8Th7e3OnFke+UVTp8AAL/8ws1Tbm6GPecff3DzAwBcugT076+f\nG0gY1vjxPNrt8GGgbVsgNFS1UMyqE1hGAYkSI+IhratXA7VrAwsXAnkSExpNUhK3P+e2YwenaFDa\njRvAr78CLi7ctzNihO65KlWMV/sQJsWsCgAZBSRKFSLux7l5k7fLluWmGKWbYf75h5s5/vqLt9u1\n47H/N27w9uzZwOjRyp5TmAVpAhJCLRoNsGULX3wfPAA++cQwbfCHDuku/gCwcydf/CMieNBA7hEx\nwqJIDUCI0u78eeDZZ4GMDN6uVo0nEAqLJ6OAhCjt6tQBli4FmjbliUcbNqgdkTARZlUDkE5gIYRQ\njlkVANIEJIQQypEmICGEsFBSAAghhIWSAkAIISyU4gVAVlYWxo8fjxo1asDBwQE+Pj6YOnVqvv0m\nTJgALy8vODg4ICAgABcvXnzisYODg9G1a1csW7ZM6bCFEMLyKL3E2LRp06hixYq0efNmunbtGq1a\ntYqcnZ3pm2++ydln5syZ5ObmRuvWraOTJ09St27dqEaNGrImsDAf06cT+fgQtW1LdOmS2tEI8VQU\nHwXUpUsXeHh44Keffsp5LCgoCA4ODvjtt98AAF5eXvjwww8xatQoADzKR6vVYtGiRejVq1e+Y8oo\nIGFSNm8GXn5Zt92yJXDggHrxCPGUFG8CatmyJcLDwxETEwMAOH78OA4cOICX//3CXLlyBbGxsWif\nK+GVi4sLWrRogYiICKXDEUJ5V64Uvi2EmVA8F9DHH3+M5ORk+Pr6wtraGllZWZg2bRqC/132LDY2\nFhqNBto8GRi1Wi1iY2OVDkcI5QUGcsrmxETefuMNdeMR4ikpXgAsX74cS5cuRVhYGOrWrYtjx45h\nxIgR8PLyQt++fR/7OiKCprQuQShKF29vzuW+Zg2nUv7Pf9SOSIinongBMGbMGIwdOxY9e/YEANSr\nVw9Xr17FjBkz0LdvX3h4eICIEBcXp1cLiI+PR+MnrMsZHBwMGxv9kC09LUR0NJCQALRqBTg4qB2N\nBalZE/jgA7WjEKJEFC8AUlJS8t3JW1lZISsrCwDg7e0NDw8PhIeHo8G/S7ElJycjMjISoU9YGScs\nLEw6gXOZNYvXNgGARo2A/ftlRUiL8tdfvJBLgwbyhxdPRfFO4C5dumDatGnYtGkTrl27hjVr1uCr\nr77C66+/nrPPyJEjMXXqVKxfvx4nT55Ev3798Mwzz6Bbt25Kh1OgOXOAI0eUOVZmJnD3rjLHKq5p\n03S/HzvG61wLI3n4kBdaUcvGjVwLadmSS//4ePViEeZL6XGl9+/fp1GjRlH16tXJwcGBfHx8aMKE\nCZSenq6338SJE8nT05Ps7e2pY8eOFBMT89hjKjkP4OFDogYNiACikBCiy5ef/liHDhFVqsTH6tiR\nj21MXl587uyfTZuMe36LtXw5Udmy/Kb3769ODM2a6f/xp09XJw5h1hQvAAxB6YlgGRlECxYQeXoS\n2doSjR5NdOdO8Y/TooX+d/DbbxUJr8i2biUqV45IoyEaONC457Zozs76f/jwcOPH0KqVfgyzZxs/\nBmH2zCoXkFKpIKytgf79eVnU8eOBH3/k2vSXXwKPHhX9OCkphW8bWseOwJ07fN4ffzTuuS1WVhaQ\nmqr/WHE+NEqZMweoUIF/b9UKeO8948dgyZYsATp3Bt59Vzcc2BypXQIVhaFTQcTGEg0ZQmRtTVSt\nGtGSJUSZmU9+3YoVRGXK8A1YzZpEcXEGCU+YmlmzdHfeL71ElJamThypqUS3bxNlZalzfku1Zw9X\nu7M/A127qh3RUzOrGoChaLXAt98Cp08DjRsDvXsDzZsDu3YV/rqePYFz53i/6GigUiXjxCtUNmYM\ncPYsL6q+bRtQpow6cdjaAh4evLi8Obh6FWjThheiHzNG7WieXnQ0X/qzHT2qXiwlJAVALnXq8Nye\nffv4O92uHfDqq1wwPE6NGkDbtoCzszIxPHzI343XXgP+TZ1kXlJSgHv31I6CEQHHjwMXLih/bF9f\n4PnnARvFR1KXXv3785frr7+AL74Ali9XO6Kn06aNfqGfK62NuZECoACtWwP/+x/wxx98h9+gATBw\nIHDrluHPPXw4fzfWrgXefBPYtMnw51TM998DLi78M2GCurEQ8QzdRo24ZJ80Sd14BHDjhv729evq\nxFFSjRsDO3YAgwYB06ebdwec2m1QRaFmOujUVKK5c4nKlydycCAaP54oOZlHAn70EdG2bcqer359\n/cEdEycqe3yDuXuXO1FyB3/2rHrxHDyoHwvAfzhRcrdvE3XrRtSwIfeHFNXUqbq/hasr0YULhotR\nFIlZLQofGBgIGxsbVdI/JCUBM2cC//0v1/6yWzk0GmDdOm4qUsKwYcC8ebpjb99uJjXMv//O3wkS\nFQU0aaJOPEePAk2b6ratrYHkZMmXoYTAQGDLFt322rVA165Fe+3GjZw9NTCQh94JdaldAhWFKS0I\nc/06zx/IfWP53nvKHT8tjW+U+vUjWrNGueMaxYgRujclKEj90SnDhnEs1tbGn6RRmtWsqf8F+OIL\ntSMST8msagCmsiDMyJHA3Lm67Zo1eVhwixbqxWQyoqOBtDQeRmUKo1Pi4ni0jJub2pGUHmPGcEcV\nANjbA7t3A6tXcy1wwADA31/V8ETRSQHwFFJSgNGjuZWhRg0eEXjiBNCrF/cJSc1WlGpEwO+/c1PO\na69xRsLsJiF7ex55VauWujGKIpECQAGZmcDixcCnn/IN55Ah/Hv2RE0hSjUHBx6/nG3xYp5MI0ye\nWQ0DVSoVhNKsrXnI5oULwOTJwMKFvGbIhAn63wthQjZu5Kn8ISH5hyeK4nnuOd3vZcrw0FthFqQG\noLCLF3keQVwcb2u1wOefA336AFZmVdyWYmfPAg0bAunpvN24Mc/6++EHnlm7bBnQrJm6MZqTv/8G\nxo3jlNSDBnHBKsyCFAAKGzqU00pk8/LiCWSNGnG/WYcO6sUm/rVyJefxyGZlxUnesvn5AWfOGD8u\nIbLFxQEjRgA3bwJvv82zqA1A7kkVljctTPasYgcHICCAb45OnFAnNvGvFi0AV1fdtp+f/vPZ1Tch\n1NKnD6fKOHCAR1bt2WOQ00gBoLCPPtJdT6pUAaZM4VFx+/fzSLnLl7k28M47XLgLFVSpAuzdy3k3\nJk4ENmwAqlfXPT9smGqhCQEAOHVKf7uwhGQlofTEgurVq5NGo8n3M3ToUCIievToEQ0ZMoTKly9P\nTk5O1KNHD4p7Qh7l7IlggYGB1KVLF1q6dKnSYSsqI4Po5s2CswSnpRHNm0dUsSKRvT3R2LFEj5vf\nlpVFtHAh73PwoGFjtngJCZwHfOdOtSMRgleay55oZ29PdOaMQU6jeB/AnTt3kJmZmbN98uRJdOzY\nEbt378YLL7yAwYMHY/PmzVi0aBFcXFwQGhoKa2tr7Nu377HHNKc+gKJKTubO4TlzACcnvhF99139\nJqSxY4EZM/j3MmW4KUn6JoWwABkZwHffcTPBf/6jn9ZESQYpVnIZMWIE1apVi4j4Tt7W1pZWr16d\n8/y5c+dIo9FQZGTkY49hSqkglHbjBtE77/D6ErVqEa1apcugUK+e/oz7zz5TN1ZhIbKyOHXGgAGc\n9VCUWgbtA0hPT8eSJUvQ/98e7CNHjiAjIwPtc2U3q1OnDqpWrYqIiAhDhmKynnkG+PlnnjxZsybQ\no4eu4zhv32TebWFAiYncT/DXX2pHYnwzZwKhocCCBXz3uXKl2hEJAzFoAbBmzRokJSXhzTffBADE\nxcXB1tY2XzOOVqtFbGysIUMxefXrA5s3c/bPlBRe5jU1lbOMPvccMGsWEBSkdpQW4soVoF494MUX\ngdq1n7w0nKmbOxeoVo3nOxRl9arwcP3tnTsNE5ehZWRwR3/dujwz2VQWKjIhBi0AFi5ciMDAQHh4\neBS6HxFBYwqJw0xAhw6cRfm334BjxzjFSosWPBTYrP32G0++evFFw41oUMq33wK3b/PvKSm6jhhz\ndOgQZy+8fp0/UEW5i8jb3myo9mdDmzsX+OYbnvi3dCkP0RN6DLae3fXr17Fjxw78+eefOY95eHgg\nLS0NycnJerWA+Ph4aLXaJx4zODgYNnmW4FNjbQBDs7IC+vbluUpff80J5hYt4pxbI0eaYUr7Eye4\nBMuebNWtG0+ZNlX29oVvm5O8aS5u3uQupcJuuKZM4fwm0dE8ecXFBRg8mEcgGGhCkkHk/YxduqRO\nHHllZPC4fjs7rurnlZbGCcaM8bkzVOfCxIkTycvLizIzM3MeK6gT+Pz58xbdCVwUCQlEI0cSlSlD\nVLky0S+/8FBTs7FmjX5vtpWVaf8HEhOJmjXjWKtUUXdls5K6c4eoWjXde9+/f/FeHxam/7ebPdsg\nYRrE9u38WcuO/ccf1Y6IP/cBAbqYQkP1n1+wgL/oGg3RpEkGD8cgBUBWVhZVq1aNxo4dm++5wYMH\nU/Xq1WnXrl105MgRatmyJbVu3brQ41l6AZDt4kWi//yHPzcNGhBt2aJ2REUUH0/k5aX70PfooXZE\nRZOQQJTrBsZsxcbyqJ6wsOL/fwYM0C8AOnc2TIyGsn8/0ZQpRBs2qB0Ji4jIv1RpYiI/l5xMZGNj\n1GVVDVIAbNu2jaysrCgmJibfc48ePaKhQ4fmTAQLCgoq8kQwSy8Ash08SPTCC/z5CAggio5WO6Ii\nuH6daMYMovnzC54hJ0zTvHn6F6QCbupEMZw6pf9+2toSpaTwcwkJ+QuHw4cNGo4kgzNTRMD69dyv\ndf489xlMmQJUrap2ZKJUIeJhoTt3cmfwlCn5E16J4pkyhfPG29ryUNs33tA9l3tR8K5dgTVrDJpG\n2KwKADUXhTdVGRn8GZo4kReuHzkS+OQT/VxnQggTk57OHe0FXdwPHwYePeIOYgPnkDerAkBqAI93\n7x7w5Zf8Y28PjB/PAzdsbdWOTOhJSOA8IN7eprFmsrBokg20lHB25lplTAzw+uu8ZnHdusAff3At\nPiGBM8y2bg3Mn692tBZq8WJeIKJmTV5LN1fOLCHUIDWAUur0aZ43sGEDTySztub0Etm2bgU6dlQv\nPovk5gbcvavbXruW23mF8WRm8pdBAJAaQKlVrx53Eu/cyf0EuS/+QP5048II8t5rmf69V+ny/vs8\n+apCBc65ooQtW4Bff+XlMM2QFACl3EsvcTaAgADdY1ZWsm63KubM0d19duoEvPKKuvFYkt27+f3P\nyADu3OH20JL65BMgMJBnuTdrxmsjmxmzKgCCg4PRtWtXLFu2TO1QFHXsGPDCC5wqJyxM+eNbWXGi\nuTlzOBWPoyNnY5gyBXjwQPnzKerkSR7jOmAAcO2a2tGUzDvvcGqGM2eATZsAG4NlYhF55W56A3jI\nXElrYD/8oPv9xg3+m5oZ6QMwAc88o8s6bG3N7fd16hjufP/8w/mFvvkGKF8e+OwzvokxuabRf/7h\nbJx37vC2jw8n9pILpyiuBw94BMSxY7z96ad8B1QSdevy5zHbli1cszMjZlUDKI0ePtRPOZ+ZCVy9\nathzurvzcNFz54C2bYGBA7n2sWmTiTVLnz+vu/gDnNzLTNtahcocHXmB9fXruUOspBd/AFiyhAsB\nd3devs/MLv6A1ABMwquvAhs38u9Vq/JNipub8c5/5Ajw4YfcTNquHfDFF0CTJsY7/2P98w9XhRIS\neLtWLW4+kRqAEIqQGoAJWLWKU9DPnAkcPGjciz/A/Vc7d/LN0e3bPOO/Tx8TaHJ3d+fFWPr142rK\njh1y8RdCQWZVA5BUEIaXkQH88gswYQLfgA8fzrXbkhZKZ8/y5LR793h+wquvKhOvEOLpmVUBUFqb\ngEzR/fs8aujzzzmdxPjxwJAhQNmyT3e86tV1NQpbWy4QatRQLFwhxFOQJiBRICcnrgVcvMjrgn/4\nIS9KHxamW9irqO7f129OSksz7QXBhLAUUgCIQnl4AN9/z8Px69cHQkKA55/nFe2KysmJRxtl8/Q0\n32VmhShNpAAQReLnx6lr9uzhJJZt23IamzNnivb6DRu4k3vcOB6FV768QcMVQhSBQQqAW7duoW/f\nvqhQoQIcHBzQsGFDHD16VG+fCRMmwMvLCw4ODggICMDFIrQJlNaZwOakTRseqRQWxvmE6tcH3nsP\niI0t/HWOjrx4zdSp3B9gcSTzpyiq9PTit7M+JcULgLt376JVq1YoW7Ystm7dirNnz2L27NlwyzWM\nZNasWZg3bx5++OEHHDp0CI6OjujUqRPS0tIKPXZYWBjWrVsnI4BUptFwv8DZs8Ds2cDKlTxJd9Ik\nbu9/nKwsYMUK4KefeISRRcjKAt58k3u+n3mGF/sQ4nHGjuWEda6uwOrVhj+f0mtMfvTRR9SmTZtC\n9/H09KQ5c+bkbCclJZGdnR0tX768wP1NdU3gpCSihQuJVqwoHWuHP63ERKIxY4jKliXy8CD64Qei\n9PT8+/Xrp1vqtFYtfv9KvWXL9Nd4bdBA7YiEqTp0SP+z4uhIlJFh0FMqXgNYv349mjVrhl69ekGr\n1aJJkyZYsGBBzvNXrlxBbGws2rdvn/OYi4sLWrRogYiICKXDMZiUFF6x7Z13gF69gN691Y5IPeXK\nAbNmceaGgABuEmrQgCeWZQ8yTk8Hfv9d95qYGGDfPnXiNaq8ScjybguRLTlZfzslhb84BqR4AXD5\n8mV8//33qFOnDrZt24ZBgwZh+PDhWLx4MQAgNjYWGo0GWq1W73VarRaxT2pINiEREfo59cPCzCCz\npsISEoCePYHGjTm5XLVqwG+/AUeP8sJXXbtyOurDh3kd8UqV9F/v5aVQIESm28beowcv/5jtgw/U\ni0WYthfA6GJQAAAgAElEQVRe4J9s2esXGJLSVQpbW1tq3bq13mPDhw+nli1bEhHR//73P7KysqLY\n2Fi9fXr27EkhISEFHtMUm4BOnybSaHS1NTc3y2sG6t5dv8b6xx+657KyiDZvJnr2WX4uOJibyvz8\niLy8iObOVSiIJUu4qlymDNHnnyt0UIX98w/R2rVEUVFqRyJMXWoq0datRBERRjmd4olVPD094efn\np/eYn58fVv/boeHh4QEiQlxcnF4tID4+Ho0bNy702MHBwbDJkwtGrbQQdesC333HqZSdnIAff+S8\n+5bk/Hn97XPndL9rNEDnztwktGgRzyRetQoYOpQz8bq7KxDAgwecxzp78MCYMbzQQe3aChxcQW5u\nsvSjKBpbW6Ou1ap4AdCqVSucz3NlOH/+PKpVqwYA8Pb2hoeHB8LDw9GgQQMAnOohMjISoaGhhR47\nLCzMpFJBDBrEP5aqe3fdPICyZYGXX86/j7U195MEBwNffcV9Bb/8woMdhg0rYQ03JUV38c+WlFSC\nAwphYZSuUhw+fJhsbW1p+vTpdPHiRVqyZAk5OTnRsmXLcvaZNWsWubu707p16+jEiRPUrVs38vHx\nodTU1AKPaYpNQIL99hvRhAlEhw8Xbf+4OKIhQ4isrYmqVSNavLiETWfvvKNrgwoIMPioCSFKE8UL\nACKijRs3Uv369cne3p7q1q1LP//8c759Jk6cSJ6enmRvb08dO3akmJiYxx5PCoDS59w5XR9CkyZE\n4eElONiuXdzhUNDYUyHEY0k2UKGq/fs50dzBg9yE9PnnQL16akclhGUwq25LSQVR+rRuzbmB/viD\nO5UbNOD132/dUjsyYTIuXuTlF0+eVDuSUkdqAMJkpKUBP/wATJ7MayW//z7XDpyd1Y5MqObQIZ5M\nkpLCq8GtWSOrCSnIrGoAouQyM/k7tHw5X2RNia0tjwy6dAkYMYLXJvbx4XTUBp4QKUzBtm08Vjj3\njMqFC/niD/BydfPnqxNbKSUFgIUJDgZef53/bd8+/yhKU+DqyjOLL1wAAgOB0FDOOvrnn7rUEqKU\nGTIE6NQJCArilLPZdycVK+rvl3dblIg0AVmQuDhe4CW3yEigeXN14imq48d5jte2bdxn8MUXvCiN\nKCXS0nhCSO5L0datPCHq/n2+W9m5k1cRWrUqf04R8dSkBmBBnJ0Be3vdtpUVUKGCevEUVcOGfD3Y\nupXzZfn7cwK+S5fUjkwookwZrvbllv3BdHLi1YRSUjh7oFz8FWVWBYCMAioZBwdg2TJektHdnVNZ\nmNPC7B07cqK5X3/lZHx+ftxXkJCgdmSiRDQa7pTy8OAP6fTpQJMmakdlEaQJSJilhw+BuXOBGTN4\ne+xYYPhw/RqOEKJwZlUDECKbvT3w8cc8RPyttzjBXJ06nI7aSKvpCWH2pAAQZq1iRa4JnD0LtGjB\nqy82aQJs3652ZEKYPikARKng48OziSMiuN+wY0dOR33ihNqRCWG6zKoAkE5g8STPP8+DRVavBq5c\nARo14iUDbt5UOzIhTI90AotSKz0d+OknYNIk4N49YNQo4KOP8o84NBm//87VGB8fYMoUwNFR7YhE\nKScFgDA5mZl8F29np8yEr+Rknjw2ezZfUydOBN59l1NPmIytW7nNKlu/fryUmhAGZFZNQMI0EAFr\n1wKLF/PFVUmZmZzr66WXeMLX8OElP6aLC99Qx8TwipEjRnDK6VWrCk4tkZDA+9WsyfsaZVTR4cOF\nbwthAFIAiGLr3x947TWgb1+gVSv93F0ldeQIsGWLbvubb4DERGWOXbkysGABcOwYUKsWp51p1YrT\nUec2YgSwbh1w+TLw9decodTg2rTRX1T6xReNcFJh6RQvACZPngwrKyu9n7p16+Y8n5qaitDQUFSo\nUAHOzs4ICgpCfHx8kY4tncDqS03lmbjZTp3iRV2U4uSkv21ry+sNK6l+fWDTJmDHDuDRIy4EevTg\n5HMAdx7ndvWqsucvUJs2XOq8/TYwdSqPbRXC0JReYmzSpElUv359io+Pp7i4OIqLi6M7d+7kPD9o\n0CCqVq0a7d69m44ePUr+/v7UunXrQo8pS0KajqwsInd33TK8AFF0tLLnmDSJyMqKqGxZokWLlD12\nXpmZRL//TlS1KpGNDVFoKNH06br/m60t0cGDho1BCLUo3gk8efJkrF27FkePHs33XHJyMipWrIiw\nsDB0794dAHD+/Hn4+fnh4MGDaP6YtJTSCWxadu8G3nmH2//HjePRNUpLTQWsrXkNEGN49Iibm6ZN\n4zb/7t2B2rWBV17hoaRClEYG6QOIiYlB5cqVUbNmTfTp0wc3btwAAERFRSEjIwPt27fP2bdOnTqo\nWrUqIiIiDBGKMIC2bbl9PCHBMBd/gJt9jHXxB3jE0ejRPBDnwQNOKfH110B0NHdMC1EaKV4APP/8\n8/j111+xdetWzJ8/H1euXEGbNm3w4MEDxMbGwtbWNt9dvFarRWxsrNKhFCgjg2eH/vWXUU5nMe7e\n5VpBmzbAvHlqR/N0lizhpJTZo34yM/n/1Lgxd0yb/oBpIYpH8QKgU6dO6NGjB5599lkEBARg06ZN\nSExMxIoVKx77GiKCRqNROpR8UlOBDh04v3z16nyXJ5Tx3nvAL7/w+P1hw7g/09zcuaO/7e7OC+aU\nK8crk3XsyCOIhCgtDD4M1NXVFbVr18bFixfh4eGBtLQ0JOcZPB4fHw+tVvvEY2WPAsr9U5wRQX/+\nCezZw79nZAAffFCs/4ooRN6cOwV0AZm8Xr0ALy/d9ogRvFranj087+HmTU40168fcP26enEKoRhD\n9zLfu3eP3N3dad68eZSUlES2tra0evXqnOfPnz9PGo2GIiMjH3sMpUYB/fGH/ugVD48SHU7kMnq0\n/nv7zDNEt2+rHVXxxcURLV9OVNDHMT2daP58Iq2WRyiNGUOUmGj8GIVQiuIFwAcffEB79uyhq1ev\n0oEDB6hDhw5UqVIlSkhIICKiwYMHU/Xq1WnXrl105MgRatmypdGGgaanE738Ml+gypYlWrmyRIcT\nuWRmEtWqpV8IfPqp2lEZRnIy0cSJRA4OROXLE/33v0SpqWpHZaZu3iR68UUuVd99lygjQ+2ILIri\nBUBwcDBVrlyZ7OzsqEqVKhQSEkKXL1/Oef7Ro0c0dOhQKl++PDk5OVFQUBDFxcUVekwl5wFkZRFd\nvSp3bobQoYN+ATBpktoRGdatW0QDB/KchRo1uOaQlaV2VGbmtdf0PzTff692RBZFksEJxURF8TDK\nhATuaN+5kztSS7szZzjL6IYN3Gfw5ZfACy+oHdVTyszkxEn793MypkmTeEKGoTz3HOf/yPbpp3x+\nYRRmlQvI1FNBnDvHo0YyMtSORB1Nm3Ln6JUr/J22hIs/ANStC6xfD+zaxdfPNm04V9K5c2pH9hRm\nzwYmTwbCwzklxaxZhj3fO+/ofnd05J54YTRSA1DI55/zXSAAtGvH48bLlFE3JlMwbx6nua9enWfa\nVqqkdkSGlZXFcwnGjgVu3AAGDuSb6LyD3B494hqSiwvQurUqoRYsJAQIC9Nt9+gBrFxp2HPu2sWl\nZfv2PP1aGI+6LVBFY+q5gDIzOWdM7qbMTZvUjkp9mzfrvyeBgWpHZDyPHhHNnk3k5kbk5EQ0eTLR\n/fu6555/Xve+jB5txMDCw4kqV+Ye7MmT8z//88/6f7T5840YnDA2qQEogIjv5O7f1z22cyfntLdk\ns2frz7WoVs1ImTVNSGIiMH06p5UoXx747DNOS/3yy7p9rKyAhw+NtEBNxYrcSZMtIiL/qjtLl+r6\nAPr2NUJQQi1m1QdgqjQazjOfnba4f3/Lvvjfvw906gSMGcPvTbYuXdSLSS1ubrwa2fnz/JkYOBAY\nOlR/H0dHI+U9yszMv7hC7sIg2xtvAN99Jxf/ooqOBt56CwgNBYyU0kYxaldBisLUm4CypaQQ5cp8\nbbEmTtRvRfDz49F9mZlqR6a+I0eIXnqJ3xeNhpuH1q0zYgDDh+v+MPXr69qlLEFWFtHUqUTt2vEs\nPiUmb9y+TVSunO49bdCg5Mc0IrOqAZj6KCB7e8sZ+VKYvDeZVaoAgwbpL3hlqZo25QE2GzYAfn5c\nWwoLM2LT2Ny5wLZtwIoVvBSakgvPnzjBaxvfu6fcMZX03Xc8zHTnTh61MWlSyY956hRnQsx24oTy\n66QaktolUFGYSw1AsBMniFxd+YaoTBmi9evVjsg0pacT/fQTkacnDyJ4/32if/5RO6qnNG8eV2kA\nIl9f05xpOWCA8qMSbt0icnHRHbNu3ZIf04jknuwpLVrEScHmzpU0wXnVrw+cPMmjB48f50XeRX42\nNsCAAbxY/bhxwPz5vBD9nDmcudasTJ+u+yKcO2f4oaNPIyCg8O2n4enJa4sGB3Pn37ZtJT+mEVnk\nKKBffgF++okzP379tX4GyKJYvFi/f2z6dOCTT0oclrBwcXHcKvHTT9xsNn068J//mEnTmZ+f/sy3\n5ctNc1LXypU876BpU/1JaJZK7SpIUSjZBLR/v66mChC1aVP4/tevEx0/rp+jKm9NsnPnEoclRI6z\nZ4m6dePPVrNmRLt3qx1REezbR1SxIn+5+vR5+h7/pCSifv2IGjcmGjfOsMmVjh3jpqv//c9w5zBx\n5nBvkUOJTuDTp/WbbE6ffvy+P/0EeHtzXpvOnYH0dH68RQv9/R6zlLEQT8XXV7d2hZUVL8HZpQvn\nHDJZrVtzFebhQ576/bTVlvff55WaoqN5geafflI2zmx79/IXd+hQjv2PPwxzHlOndglUFErWAC5c\nIHJ01N29v/nm4/fN3bcDEK1Zo3tu7ly+S/vsM+7ME+YnNdW42TtjY3kIqLs70Rtv8IzgJ8nK4iyj\nNWpw1tGBA7nf0Rxdvkw0dCjRqFGFrBXRurX+l27UKMMEM3Cg/nleftkw5zFxFlcAEHHN76OP+CJe\n2MXbzU3/MyKjWUqP0FBurXB15ZQVxtC7t/7nadasor/20SNed8DdnW9gJk4kunfPYKEqLjmZFwnK\n/r/7+j7mu/fll7qdrK2Jdu40TECTJ+v/Md57zzDnMXEWWQAU1dKluhw/3bvLWhWlxfbt+t/9SpWM\nc97sCWDZPyNGFP8YiYk8h6lsWV5DZf5886iBHjmi/38HiK5de8zOYWHc/r93r+ECeviQq2EeHkSv\nvGLG429LxiJHARXH3bs8r+WZZ/TTGgjztWYN8Prrum07O266NrSlS4E+ffjyZ2/PbfzPPfd0x7p+\nnec0LV4M1KnDWZu7dCn8M5qaCvz4I3+m33wTqFr16c79NP75B/Dx0U0SrFwZuHRJlz5FqMPgncAz\nZsyAlZUVRo8enfNYamoqQkNDUaFCBTg7OyMoKAjx8fGGDuWplCvHQ/Lk4l96dOrEi7tnGzvWsOc7\ndQq4cIFT7Pzvf3wRPnr06S/+AF+8f/uNF+GpXBno1o07iw8ffvxrevYEhg8HJkzg/G8FpQEyFHd3\nYPt2jjMoiIfOy8XfBBiyenHo0CHy9vamRo0a0ahcnTmDBg2iatWq0e7du+no0aPk7+9f6LrA2U1A\ngYGB1KVLF1q6dKkhwxYWICWFaOtWosOHDXuevn11TR5jxxrmHFlZ3I9Rvz6f5z//Ibp0SX+ftLT8\nTTBGzUEkTJLBCoB79+5R7dq1KTw8nNq2bZtTACQlJZGtrS2tXr06Z99z586RRqOhyMjIAo8lqSCE\nOYqOzn/RTUgw3PkyMogWLiTy8uIUHCNH6p/P21u/f/X0acPFUpCHD4mCg4nKlycKCCD6+2/jnl/k\nZ7AmoNDQUHTp0gXt2rXTe/zIkSPIyMhA+/btcx6rU6cOqlatioiICEOFI4TR5V0RzsrKsMvrWlsD\nb7/NqSUmTQJ+/plTS3zxBa9AtmEDr1bXpAkP1a9b13CxFOTLLznx3Z073Bw0Zoxxzy/yM0gBEBYW\nhmPHjmHGjBn5nouLi4OtrW2+zlytVotYc8ulLUQh6tUDsru+rKz4QlyunOHP6+DA/RoXL3Kn89ix\n3FF89ChfeKOieOVHY7t1q/BtYXyKFwA3b97EyJEjsXjxYpQpxqK4RASN9LSKJ0hPB/bt42Rz5mD2\nbOD2bSA+XlcYGEulSrwm8+nTQLNmnL+qWTNOR62Gvn15xBXABaKk4lGf4usQRUVF4e+//0bTpk1B\n/44wzczMxN69ezFv3jxs2bIFqampSE5O1qsFxMfHQ5t35ew8goODYZNn6aSQkBCEqHE7I4wuLY0T\nOO7dy9vTphl+BI8SPDzUPX/t2sCqVcCBA8CHHwIdOgCBgZwS/9lnjReHvz/XPvbv5/QqeVOqmLrM\nTMM24alB8XkADx48wLVr1/Qee+utt+Dn54ePP/4YlStXRsWKFREWFobu3bsDAC5cuABfX18cPHgQ\nzQtIrGPqawIL49i0CXjlFd12mTLctm0W2TJNBBGwejXw8cfA5cu8kmH2OsWiYImJPHx1/37uP1m/\nnrNAlwaKf3UcHR1Rt25dvR9HR0eUL18efn5+cHFxQf/+/TF69Gjs3r0bUVFRePvtt9GqVasCL/5C\nZHNw0N+2t5eLf3FpNECPHpxYbu5cYN06oFYtnlRmTgtZGdP06dzsSMQ1mE8/ffJrVq3idR1iYgwf\nX0kY5euTt23/q6++wquvvoqgoCC0bdsWXl5eWLVqlTFCEWasbVteWhLgi//ChaqGY9bKlOFEmBcv\nAiNHcl+Fjw+vmpid9VawvEuc5t3Oa/x4nuz2/vvc53LhguFiKylJBSHMTnIydyba2qodSelx8yZf\nuBYt4hrBzJnAa6/JDHiA7/pfeolTwtjZcVPkSy89fv8aNYArV3Tbs2cbfwBAUZlVBdrUF4UXxuHi\nIhd/pT3zDK+UFx3Na2C8/jrQpg1w8KDakamvaVNO57FmDf9b2MUfAKpXL3zblEgNQAiRz/btPGLo\n+HFuzpgxg5uIxJNdu8ZDXK9eBXr35k52UyUFgBCiQJmZwJIlvGB9XBwweDA3E1WooHZkQilm1QQk\nhDAea2ugXz/uxPzsM+DXXzm1xMyZxkmfLQxPCgBh8uLigG3bgBs31I7EMtnb87yBS5d43sD48Ty5\nbNEiriUI8yUFgDBpp09z0rJOnQA/Px6PLdRRoQLPHTh7lmf1vvUWd5Bu3652ZOJpmVUBIKOALM83\n3/BqUgDw4AEPqRPq8vEBVqwAIiIAJyegY0cuoI8fVzsyUVyK5wIypLCwMOkEtjBOToVvC/U8/zzX\nyNauBT76CGjcmPsMpk7lYaXC9JlVDUBYnk8+4dmUAKc0njZN3XiEPo2GJ4ydOgV8+y2weTNPJBs7\nFkhKUjs68SQyDFSYhfv35e7fHNy7x1lGZ88GHB15/eH33pOJe6ZKagDCLMjF3zw4OwNTpnCOoW7d\nOM9QvXrAypWcTE2YFrMqAKQTWAjz4OUFLFjAHcO1awM9ewKtWvGaBMJ0SBOQEMLgdu7k1BJHjwLd\nu/Nkstq11Y5KmFUNQAhhntq1Aw4fBhYv5uyadesCoaG8VKZQjxQAQgijsLLi5Gjnz3MNYOlSnlMw\nbRqQkqJ2dJZJCgAhhFHZ2QEffMAdxQMGAJMn89DRhQsltYSxKV4AzJ8/Hw0bNoSrqytcXV3RsmVL\nbNmyJef51NRUhIaGokKFCnB2dkZQUBDipR4ohMUpX56XTTx/ntce6N8faNSI5xKYfs9k6aB4AVCl\nShXMmjULUVFRiIqKQrt27dCtWzecPXsWADBy5Ehs3LgRq1atwt69e3Hr1i306NGjSMeWUUBClD7e\n3sCyZcChQ4C7O/Dyy0BAAHcYCwMjI3B3d6eFCxdSUlIS2dra0urVq3OeO3fuHGk0GoqMjHzs65OS\nkggAJSUlGSNcIUqtw4eJnn+eqH59ouXL1Y4mv6wsonXriHx9iQCiPn2Irl1TO6rSy6B9AFlZWQgL\nC0NKSgr8/f0RFRWFjIwMtG/fPmefOnXqoGrVqoiIiDBkKEJYvKws4NVXeZnHkye5Q/biRbWj0qfR\nAF26cHzz53Om0dq1OdfQ3btqR1f6GKQAOHXqFJydnVG2bFkMGTIEa9asga+vL2JjY2Fra5tvLL9W\nq0VsbKwhQhFC/Ov+fV5bIVtGBnD9unrxFMbGhlNIXLzIaxHMm8eL0fz3v0BamtrRlR4GKQB8fX1x\n/PhxREZGYvDgwejXrx/OnTv32P2JCBqNxhChCCH+5eICdO6s265RQ5doz1Q5OQGTJnFBEBQEvP8+\nrwuxfLl0FCvBKDOBAwIC4OPjg169eqFDhw5ITEzUqwVUr14do0aNwogRIwp8ffZM4MDAQNjY6Gew\nDgkJQUhIiEHjF6K0SE0Ffv6Z11bo1w/QatWOqHjOnOEawfr1QPPmwJdfAi+8oHZU5sso6wFkZWUh\nNTUVTZs2hY2NDcLDw9G9e3cAwIULF3D9+nX4+/s/8TiyHoAQJVO2LDBkiNpRPL26dYF164Dduzm1\nRJs2QNeuwKxZgK+v2tGZH8ULgHHjxiEwMBBVqlTBvXv3sGTJEuzZswfbtm2Di4sL+vfvj9GjR8PN\nzQ3Ozs4YPnw4WrVqhebNmysdihCilGrbFoiM5JXJPvkEePZZYOBAYOJEwMND7ejMh+J9AHFxcejX\nrx98fX3RoUMHREVFYdu2bWjXrh0A4KuvvsKrr76KoKAgtG3bFl5eXli1apXSYQghSjkrKyA4GDh3\njtcgWL6cU0t89hk3cYknk2ygQohSITERmD4d+PprnlD22WfA22/ziCJRMLPKBSQzgYUQj+PmBnzx\nBaeWaN8eePddoGFDYMMGGTH0OFIDEEKUSlFR3FG8axf3GXzxhekPezU2s6oBCCFEUTVtCoSHAxs3\nAn//DTz3HPDGG8DVq2pHZjqkABBClFoaDSeXO3aMl6jcvRuoU4fTUf/zj9rRqU8KACFEqWdjw+mm\nY2KATz8FfviBRwzNns2T4yyVFABCCIvh6AiMH8+pJYKDOcmcry+vTpaVpXZ0xmdWBYCMAhJCKEGr\nBb77Djh9mheh6d2bU0vs2qV2ZMYlo4CEEBZv3z4eMRQZCbzyCqeWqFdP7agMz6xqAEIIYQgvvABE\nRHBqibNngQYNOLXE7dtqR2ZYUgAIIQR4xFDPnlwAzJkDrF7NHcUTJ/JaCqWRFABCCJGLrS0wYgRw\n6RIwbBg3B/n48AplGRlqR6cssyoApBNYCGEs5coBM2cCFy4AnTpxGu369Tkdten3nBaNdAILIUQR\nREcDY8YAO3bwOgRffMEjh8yZWdUAhBBCLY0bA9u2AVu2cObRFi14LsHly2pH9vSkABBCiCLSaLg5\nKDoaWLgQ2L+fJ5KNGgXcuaN2dMUnBYAQQhSTtTWvNXDhAi9a//PPQM2avDDNo0dqR1d0ihcAM2bM\nQPPmzeHi4gKtVovu3bvjwoULevukpqYiNDQUFSpUgLOzM4KCghAfH690KEIIYVAODsDYsTxiqG9f\nYNw4Tjb3++/mkVpC8QJg3759GDZsGCIjI7Fjxw6kp6ejY8eOePjwYc4+I0eOxMaNG7Fq1Srs3bsX\nt27dQo8ePZ54bBkFJIQwRRUrAt98w6klnnsO6NeP1x4ID1c7ssIZfBRQQkICKlWqhL1796J169ZI\nTk5GxYoVERYWhu7duwMAzp8/Dz8/Pxw8eLDAxeFlFJAQwpz873+ccjoiAujcmZuG6tdXO6r8DN4H\ncPfuXWg0Gri7uwMAoqKikJGRgfbt2+fsU6dOHVStWhURERGGDkcIIQyuZUvgwAFg1SpuHmrUiNNR\n//WX2pHpM2gBQEQYOXIkWrdujbp16wIAYmNjYWtrm+9OXqvVIjY21pDhCCGE0Wg0wOuvc7PQ11/z\nBLJatXg9guRktaNjBi0AhgwZgjNnzhSpzZ6IoNFoDBmOEEIYXZkyQGgo1wRGjeJFaHx8gG+/BdLT\n1Y3NxlAHHjp0KDZt2oR9+/bBy8sr53EPDw+kpaUhOTlZrxYQHx8PrVZb6DGDg4NhY6MfckhICEJC\nQpQNXgghFObiAkybBgweDEyYwHmG5s7ldBPdu3ONwdgM0gk8dOhQrF27Fnv27EGNGjX0niuoE/jC\nhQvw9fWVTmAhhMU4cYJXJNuyBWjVilNL+PsbNwbFm4CGDBmCJUuWYOnSpXB0dERcXBzi4uLw6N/Z\nES4uLujfvz9Gjx6N3bt3IyoqCm+//TZatWpV4MVfCCFKowYNgM2bge3bgQcPuOM4KIjXLTYWxWsA\nVlZWBbbl//LLL+jXrx8Angj2wQcfYNmyZUhNTUXnzp3x7bffolKlSgUeU2oAQojSLCsLWLKEJ5Ld\nvs3NROPH8/wCQ5JsoEIIYSIePuQRQ9On8/bHHwOjRwNlyxrmfGZVAAQGBsLGxkY6foUQpVpCAjB1\nKjcPRUfzIjWGYFYFgNQAhBCWJC3NcBd/QLKBCiGEyTLkxR+QAkAIISyWFABCCGGhpAAQQggLZRad\nwESEe/fuwdnZWfIFCSGEQsyiABBCCKE8aQISQggLJQWAEEJYKCkAhBDCQkkBIIQQFspiC4CirFJm\nCswlTkBiNRRzidVc4gQk1mxSAJg4c4kTkFgNxVxiNZc4AYk1m8UWAEIIYemkAHiMopS6T9pHiWMU\nhbHiMJdYjRGnUvuYSqym8vcvyj6mEqup/P1LQgqAx5AP6tPtU9JjmMqXSgqAp9tHiWOYS6ym8vcv\nCRtVz25g2SkkCpKRkYHk5OTHvvZJz1vaMcwpVlM5hjnFairHMKdY1T6GEqlxSnUqiOyFZIQQorRR\nYoGsUl0AFFYDEEIIcyY1ACGEEE9NOoGFEMJCSQEghBAWSgoAIYSwUFIACCGEhSq1BcD8+fPRsGFD\nuLq6wtXVFS1btsSWLVtynk9NTUVoaCgqVKgAZ2dnBAUFIT4+XsWI2YwZM2BlZYXRo0fnPGYqsU6e\nPBlWVlZ6P3Xr1jW5OLPdunULffv2RYUKFeDg4ICGDRvi6NGjevtMmDABXl5ecHBwQEBAAC5evGj0\nOIJhmJoAAAj/SURBVL29vfO9r1ZWVhg2bBgA03pfs7KyMH78eNSoUQMODg7w8fHB1KlT8+1nCu/r\n/fv3MXLkSFSvXh0ODg5o3bo1jhw5onqc+/btQ9euXVG5cmVYWVlh3bp1+fZ5UlyJiYno3bs3XF1d\n4ebmhgEDBuDBgwfFD4ZKqQ0bNtDmzZspJiaGYmJiaNy4cWRra0tnzpwhIqJBgwZRtWrVaPfu3XT0\n6FHy9/en1q1bqxrzoUOHyNvbmxo1akSjRo3KedxUYp00aRLVr1+f4uPjKS4ujuLi4ujOnTsmFycR\nUWJiIlWvXp369+9PR44coatXr9L27dvp8uXLOfvMnDmT3NzcaN26dXTy5Enq1q0b1ahRg1JTU40a\na0JCQs77GRcXRzt27CArKyvau3cvEZnW+zpt2jSqWLEibd68ma5du0arVq0iZ2dn+uabb3L2MZX3\ntVevXvTss8/S/v376dKlSzRp0iRydXWlW7duqRrn5s2bafz48bRmzRqysrKitWvX6j1flLg6d+5M\njRs3psOHD9OBAweoVq1a1Lt372LHUmoLgIK4u7vTwoULKSkpiWxtbWn16tU5z507d440Gg1FRkaq\nEtu9e/eodu3aFB4eTm3bts0pAEwp1kmTJlHjxo0LfM6U4iQi+uijj6hNmzaF7uPp6Ulz5szJ2U5K\nSiI7Oztavny5ocMr1IgRI6hWrVo5MZnS+/rqq6/SgAED9B7r0aMH9e3bN2fbFN7Xhw8fko2NDW3e\nvFnv8aZNm9L48eNNJk6NRpOvAHhSXGfOnCGNRkNHjx7N2WfLli1kbW1Nt2/fLtb5S20TUG5ZWVkI\nCwtDSkoK/P39ERUVhYyMDLRv3z5nnzp16qBq1aqIiIhQJcbQ0FB06dIF7dq103v8yJEjJhVrTEwM\nKleujJo1a6JPnz64ceMGAJjce7p+/Xo0a9YMvXr1glarRZMmTbBgwYKc569cuYLY2Fi9eF1cXNCi\nRQvVPgMAkJ6ejiVLlqB///4ATO/v37JlS4SHhyMmJgYAcPz4cRw4cAAvv/wyANN5XzMyMpCZmYmy\nZcvqPW5vb4/9+/ebTJx5FSWugwcPws3NDY0bN87Zp0OHDtBoNIiMjCzW+Up1LqBTp07B398fjx49\ngrOzM9asWQNfX19ER0fD1tY23zRqrVaL2NhYo8cZFhaGY8eO5WufBIC4uDiTifX555/Hr7/+ijp1\n6uD27duYNGkS2rRpg1OnTiE2NtZk4gSAy5cv4/vvv8f777+PcePGITIyEsOHD4ednR369OmD2NhY\naDQaaLVak4g325o1a5CUlIQ333wTgGn9/QHg448/RnJyMnx9fWFtbY2srCxMmzYNwcHBAGAy76uT\nkxP8/f0xZcoU+Pr6QqvVYunSpYiIiECtWrVMJs68ihJXbGwsKlWqpPe8tbU13N3dix17qS4AfH19\ncfz4cdy9exerVq1Cv379sHfv3sfuT0QlnlpdXDdv3sTIkSOxfft2lClTpsivUyPWTp065fz+7LPP\nonnz5qhWrRpWrFgBOzu7Al+jRpwA1/qaN2+OKVOmAAAaNmyI06dP4/vvv0efPn0e+zq14s22cOFC\nBAYGwsPDo9D91Ipz+fLlWLp0KcLCwlC3bl0cO3YMI0aMgJeXF/r27fvY16kR7+LFi/HOO++gcuXK\nsLGxQZMmTfDGG2/kGwiQm9p//8cpSlxPE3upbgKysbFBjRo10KRJE0ybNg0NGzbE3Llz4eHhgbS0\ntHwZ9uLj4/OVvIYWFRWFv//+G02bNkWZMmVQpkwZ7NmzB3PnzoWtrS20Wi1SU1NNIta8XF1dUbt2\nbVy8eNGk3lMA8PT0hJ+fn95jfn5+uH79OgDAw8MDRIS4uDi9fdR8X69fv44dO3Zg4MCBOY+Z2vs6\nZswYfPLJJ+jZsyfq1auH3r17Y9SoUZgxY0ZOvKbyvnp7e2PXrl148OABbty4gYMHDyItLQ3e3t4m\nFWduRYnLw8Mj3yiwzMxMJCYmFjv2Ul0A5JWVlYXU1FQ0bdoUNjY2CA8Pz3nuwoULuH79Ovz9/Y0a\nU4cOHXDy5EkcO3YMx48fx/Hjx9GsWTP06dMn5/cyZcqYRKx53b9/H5cuXYKXl5dJvacA0KpVK5w/\nf17vsfPnz6NatWoAkHMRyB1vcnIyIiMj0bJlS6PGmm3hwoXQarU57ekATO59TUlJyXeXaWVlhays\nLACm+b7a29tDq9UiMTERW7duxWuvvWaScQJFe//8/f1x9+5dREdH5+wTHh4OIkKLFi2Kd8JidRmb\nkbFjx9K+ffvo6tWrdPLkSfr444/J2tqawsPDiYho8ODBVL16ddq1axcdOXKEWrZsqfow0Gy5RwER\nmU6sH3zwAe3Zs4euXr1KBw4coA4dOlClSpUoISHBpOIkIjp8+DDZ2trS9OnT6eLFi7RkyRJycnKi\nZcuW5ewza9Yscnd3p3Xr1tGJEyeoW7du5OPjY/ThikREWVlZVK1aNRo7dmy+50zpfX3rrbeoSpUq\ntHHjRrp69SqtXr2aKlasSJ988knOPqbyvm7dupW2bNlCV65coW3btlGjRo3I39+fMjIyVI3z/v37\ndOzYMYqOjiaNRkNfffUVHTt2jK5fv17kuAIDA6lp06Z06NAh2r9/P9WuXZv69OlT7FhKbQHQv39/\n8vb2Jjs7O9JqtRQQEJBz8ScievToEQ0dOpTKly9PTk5OFBQURHFxcSpGrPPSSy/pFQCmEmtwcDBV\nrlyZ7OzsqEqVKhQSEqI3rt5U4sy2ceNGql+/Ptnb21PdunXp559/zrfPxIkTydPTk+zt7aljx44U\nExOjQqRE27ZtIysrqwLPb0rv6/3792nUqFFUvXp1cnBwIB8fH5owYQKlp6fr7WcK7+uKFSuoZs2a\nZGdnR15eXjR8+HBKTk5WPc7du3eTRqMhKysrvZ+33367yHElJiZS7969ycXFhcqVK0cDBgygBw8e\nFDsWSQcthBAWyqL6AIQQQuhIASCEEBZKCgAhhLBQUgAIIYSFkgJACCEslBQAQghhoaQAEEIICyUF\ngBBCWCgpAIQQwkJJASCEEBZKCgAhhLBQUgAIIYSF+j88P7rQIQkmuAAAAABJRU5ErkJggg==\n", "text/plain": [ "Graphics object consisting of 101 graphics primitives" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# 決定境界を描画\n", "xmin, xmax = min(data_x[:,1]), max(data_x[:,1])\n", "x1 = var('x1')\n", "x2 = -t[0]/t[2] - t[1]/t[2]*x1\n", "line_plt = plot(x2, [x1, xmin, xmax])\n", "(data_plt + line_plt).show(figsize=4)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "SageMath 7.5.1", "language": "", "name": "sagemath" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.13" } }, "nbformat": 4, "nbformat_minor": 0 }