{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Graph Structures\n", "## Theano Graph\n", "- 왜 갑자기 graph? \n", "- Theano로 만든 code를 debuggin 하거나 profiling 하는 건 쉽지 않다.\n", "- 블랙박스 속에서 무슨 일이 벌어지고 있는 지 모른다면\n", "- 이해하기 위해 필요한 만큼만 들여다 보자. 자세한 건 Extending Theano를 보고...\n", "\n", "- Theano code를 쓰는 첫번째 단계는 관련된 모든 수학적 관계식을 symbolic placeholder (variable)를 이용하여 기술하는 것이다.\n", "- 이러한 expression에는 사칙연산, 삼각함수 등 여러가지가 있다.\n", "- 이러한 모든 expression은 내부적으로는 *ops* 로 나타내어진다.\n", "- op.라는 것은 특정한 input을 받아 연산하고 정해진 type의 output을 내는 것을 의미한다.\n", "- 대부분의 프로그래밍 언어에서 이야기하는 function definitions으로 생각해도 되겠다.\n", "- Theano는 내부적으로 _variable node_, _op node_, _apply node_ 가 연결된 graph structure를 만든다.\n", "- apply node는 vaiable에 뭔가를 하는 op을 나타낸다고 생각하자.\n", "- op에 의해 나타내어지는 연산의 정의와 특정한 data type에 적용하는 것 그래서 apply node로 나타나는 것 사이의 차이를 그려보는 것은 중요한 일이다.\n", "- 우선 아래 graph의 예를 보자." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Using gpu device 0: GeForce GTX 980 Ti\n" ] } ], "source": [ "from theano import *\n", "import theano.tensor as T\n", "from theano import function" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "x = T.dscalar('x')\n", "y = T.dscalar('y')\n", "z = x + y\n", "f = function([x, y], z)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "<matplotlib.image.AxesImage at 0x7fde103ba450>" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": [ "iVBORw0KGgoAAAANSUhEUgAAARUAAAEACAYAAACd9eLKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", "AAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4VMXXgN/Zkk1PCARIgSSAlNA70qUoqICIgIqKBT4U\n", "UOzITwVRQRFFVEREEBtSRJQiVZogvRMSOiGQCmkk2Wyf74+ElQDpZVPu+zz75JYpZzZ7z51y5hwh\n", "pURBQUGhpFA5WgAFBYXKhaJUFBQUShRFqSgoKJQoilJRUFAoURSloqCgUKIoSkVBQaFEKRWlIoTo\n", "J4Q4JYQ4K4SYWBp1KCgolE9ESdupCCHUwGmgDxANHAAek1JGlGhFCgoK5ZLS6Kl0AM5JKSOllGZg\n", "KTCoFOpRUFAoh5SGUgkALt90fiX7moKCQhWgNJSKYvevoFCF0ZRCmdFAnZvO65DVW7EjhFAUj4JC\n", "OUZKKYqatzR6KgeBu4QQwUIIJ2A4sPrWRFLKMv1MmTKlStbZvXt31q9fT7169TAajbz//vusXLkS\n", "KSUTJkxg0qRJrFmzhhYtWhAfH8/ly5cJCAjgtddeY86cOQQGBpKcnExKSgpt27bliy++YMaMGQwY\n", "MKBctTO3zyOPPMLOnTupX78+6enp/PLLL8yaNQspJR9//DFjx45lw4YNtGvXjqioKPR6PQ0aNGDs\n", "2LH88MMP1K5dm5iYGDIyMmjTpg29evVizpw5dO3aFaPRWK7aWlKf4lLiPRUppUUIMR7YCKiBhVJZ\n", "+XEobdu2pUmTJpw4cSLH9UOHDrFz504ADh48SEREBPXr18dsNvP+++/j4uLC7NmzSUpKIiYmBg8P\n", "D3r16gXAqlWr0Ov1uLq6lnl7CktwcDD33Xcfe/fuzXF95cqV7N27FyEEMTEx/PPPPzz88MOYzWam\n", "T5+Ol5cXCxYsID4+ntjYWPR6PR06dKBHjx6sXr2axMRE/Pz8HNSq8kup2KlIKddLKRtJKRtIKT8q\n", "jToUCo5KpWLp0qW8/PLLGI1GAAwGAz4+PvY0TZs2xWw2A9CgQQNcXV0RQhAYGIjNZuP48eNotVq2\n", "bt3K1q1bGT58OHq93iHtKSwqlYrPP/+cd999F4PBAIDFYkGn0yFEVi+/UaNG7Nq1C4DAwEC8vLwA\n", "uOuuu7DZbBiNRjw8PIiMjGTr1q088MADWK1WxzSonFMacyrlkp49e1bpOt3d3QkICGDx4sW0bdsW\n", "Z2dnIiMjMRgM6HQ6Fi1axOTJk3Mtt3379ixbtozx48ejUuV8F5WnduaGk5MTXbt2ZebMmYwZMwaN\n", "RoPJZCIjIwM3Nzd++uknBg8enGt+Ly8vEhIS+PDDD7n33nuLKX3hcMT3WxyqjJl+Rfjhl0adQUFB\n", "qNVqAL744gu8vb3x8PAAYP78+QwZMoTBgwdz991307ZtWzQaDcHBwTnya7Va2rZty+OPP87w4cMZ\n", "OHAgo0ePzrXOsqCgdQYEBKDRZL0733rrLTw8POw9tO+++45hw4bx0EMP4eXlRc+ePRFC5Gi/n58f\n", "Wq2WJk2aMH36dBYsWMCAAQMYOXKkvddT2lQ0pVLiFrUFqlQI6Yh6FRQU8kcIgSxnqz8KCgpVGEWp\n", "KCgolCiKUlGwc+3aNSwWi6PFcBjXr1/HZDI5WowKj6JUFOycOXPGbrdSFbl48SIrVqxwtBgVHkWp\n", "KNj55ZdfWLx4saPFcBjr16/no48Us6riUmXsVBTyZ8uWLZjNZgwGA87Ozo4Wp8z5/fffuXjxIomJ\n", "iVSvXt3R4lRYlJ6KApDV9T9z5gxxcXGkpaU5WpwyJzY2lkOHDmEwGEhISHC0OBUaRakoALBp0yZU\n", "KhWZmZmsW7fO0eKUOWFhYQghsFqtrFq1ytHiVGgUpaKAzWZj4cKF2Gw2AGbOnOlgicqeefPm2du/\n", "YMECB0tTsVGUigIWi4WoqCj7eXh4OLGxsQ6UqOw5duyY/fjixYtcuHDBgdJUbBSlosDp06eJj4/P\n", "ce3y5cu5pK58REVFcf78efu5lDKHklUoHIpSUeCjjz6yuwCArIfq999/d6BEZcv8+fNva78yBCo6\n", "yobCKk56ejo+Pj5YrVZ8fX2pVasWx48fp1GjRpw6dcrR4pU6RqORkJAQ4uPj8fHxITg4mIMHD1Kn\n", "Tp0q21tRNhQqFIvLly9jtVoZMGAAs2bNYty4cQwcOJDTp09XiXmFa9eukZqaSvfu3Vm6dCmDBw/m\n", "hRde4PLly4SHhztavAqJolSqOHv37qVNmzYsX76cOnXqULNmTVatWsWDDz7IoUOHHC1eqRMREYG/\n", "vz9r167F09MTZ2dn5s6dy6hRo1i5cqWjxauQKEqlimMwGPj3339xcnLKcX3JkiWkpqY6SKqyIyws\n", "jMOHD+Pm5pbj+hdffIGXl1eV3mBZVJQ5lSqOxWKxe0bbuXMniYmJPPTQQ0CW/cqtriMrG2azGa1W\n", "C8CBAwfYuXMnr776KpA1YXvzBG5VobhzKsXa+yOEiASuA1bALKXsIITwAZYBQUAkMExKmVKcehRK\n", "jxsK5U5UdoUC2BXKnaiKCqUkKO6GQgn0lFIm3XTtLWCzlPITIcTE7PO3illPucZisZQLz+oajcbu\n", "j7YssVqt5WKY4Kj2K+SkJHYp36rOBwI9so9/BLZTCZXKwYMHWfLdb/hSG1eNOyrh+Le6TdrQywyu\n", "WmN5dNQjtG/fvtTqOnHiBD98/TO+wg9XtTuqcjA9Z5M29NYMrsk4hjw9iE6dOim9DQdQEj2Vv4UQ\n", "VuBbKeV3QC0p5Q3zzHigVjHrKFfYbDZ+WvQz0bsTGdZ8pKPFuSMSyaZvNvDP9p28/OqEEn97/7Vm\n", "Hf/+eoBhHZ8u0XJLkj0/7WLT+r95e/KkPId4CiVPcV8vXaSUrYH+wDghRLebb2bPxlaqGdnjx4+j\n", "Pu1Bn+b9HC1KrggEfVr0p0ZcIAcOHCjRsiMjIzm97hKDOg4t0XJLmrsbdaWJtQ27d+92tChVjmKp\n", "cCllbPbfq0KIP4AOQLwQoraUMk4I4Qfc0TnFe++9Zz/u2bNnhYhtkpKSwqeTZjPu3tfLtN6lO36m\n", "S9Pu1KkRVKh8jes046fZ39JpaacSkSMzM5Mpr0xlTLdXSqS8grLh4Brq1W5Aw8AmhcoX5BvCHwuX\n", "0r1791KSrHKwfft2tm/fXmLlFVmpCCFcAbWUMk0I4QbcC0wlKxj7SGBG9t8/75T/ZqVSUYiOjqZP\n", "6AMlPn+Skp7Mj3/PZ8JDE+9430njVOQ6/dwCiYmJwd/fvzgiApCYmEinOt1Rq0p+OPHR8ilMGjb1\n", "jvc0Gi0qVdGGcP4edbhw4QL16tUrjnjExsZy9epV6tevj5ubGxkZGVgsFry8vEhOTsZsNlOzZk1S\n", "UlKwWq14eHiQlpaG2Wzm2rVrNGrUyL7SZLVaOX36NC4uLoSEhABw9epVdDodV69exdPTE19f32LJ\n", "WxhufalPnXrn/0NBKc6voxbwR/ZEmAZYLKXcJIQ4CCwXQjxH9pJysSQsR+j1eprUaXrHe6kZKfx9\n", "ZAM1vGpyNuYUfVv1xyqtbD6ynrv8G9OrZV8A9p3ezeFz+3HS6BjQ8WFqetdi3cFVnLh0jBW7fqV1\n", "/fYkpyfiqnPj6IVDhNSqj4erF1q1lvjkWI5dPEyf1v0xWUz8tf8PHuwwGJ02d9ePd9UM5eyZcyWi\n", "VPR6PQ38Gt/xXqZRz+p9KwmpVY+jFw/Ts3kfXJxc2HxkPbW8a3N/+0EIITh24TB7Tu1ELdT0bz+Q\n", "wBp1+evAn5yKOsmKXb8SWrc5UtowWcycjTmNj0d13J3dcdI6kZZ5nX9P7qBvm/ux2Wys3L2U/u0G\n", "4unqlavMITXqs2vnv8VSKjt37mTs2LE8/PDDrFy5kg0bNpCWlsbMmTNZuHAhkydPJiwsjK1btzJ9\n", "+nTuv/9+AgMDefHFF8nMzESv19O6dWvmzZtHSkoKvXv3pm3bthw8eJAxY8YwZswYPvjgA65cuUJy\n", "cjIzZswoU6VS0hT5lSulvCilbJX9aXYjELuUMklK2UdK2VBKeW9lslHJy2AvMe0an//5MULA3Y27\n", "8sznw1m7/w+GdHmMFbsWk3j9GgC1vGvxZO/n6NO6H+/+lDWM6t6sF4HV69Cv7QDq+gbzz4mtfPr7\n", "NLqE9qBNgw78sXsZ0YlXqO7py49/L+D4hcP8smUhKekpOGl1ecrs6epJWur1Emt/bqspGcZ05q6d\n", "RXxKHH1a9WPU7MdYvG0RAzsNYfuJv4lKuAiAm4sHT/R6lvs7DGLKL1k9s57N+1DLx49+bQfQwL8h\n", "B87u45MVH9Cmfju6hvZgy9GNnIs5g5uzO5sOr2PrsY2sO7iKqIRIPFw885TZzdmdxKuJxWr3hAkT\n", "2L59O1OnTuW9995j/vz5VK9end9++w2TyUR0dDRxcXGkpKRw7NgxOnfuDGRt1tyyZQt///03a9as\n", "4fr166xYsYKnnnqK+fPns2PHDt59911SUrIekW7durF161Y6dOhQLHkdjTItXoK46ly5u3FXtBot\n", "Nb1q8nSf/8PT1YsuoT0IjzpOt2a9SEhN4NdtP5Bh1HMh7hzphnRcdW5oNVrcXTzsZQ3t9jhBNUNy\n", "lK9Ra5j/0s9M+Pb/8PWsyfSnP0fctqLvOASCPq364aJzJbBGXR7p+jg1PH25v91AjkceJahWPQxG\n", "PV+t/pSMzHTikmJITk+imrsPWnXO9vdt3Z8G/o1ylK8SKqY9PYs3Fo7HZrXy9bhFZbJk7OTkZHeE\n", "HRoaytq1a/Hx8SEoKIg1a9bg4eHBY489xtWrV9FqtfYtD127dkWtVqPT6XB2diYzM5PDhw+j1Wr5\n", "3//+B8CQIUPsRobt27evFEvgilIpQXRaHVpN1g/Ky90brSZrDO3i5ILJYua6PpVv133J60PeJrhm\n", "PV74+mmMZgMCcVsvqIbnnbu/maZMAKzSVootKRoqlQoXnSsAHq6euDi5AODq7E5SWiJ6o54Zv01l\n", "4tDJhNRqwLs/v06mMZNq7rf3An29at6xDqPZgMViJmvhXJaJUhVCEB0dTUBAALt376Zjx46o1WpG\n", "jhzJrFmzmDFjBgEBAbzxxhu0atXKnu9OS/ktW7bEarUyduxYhBCYzWb7knde1r0VCcdbLFUgsvdE\n", "FDl/qj4FL1dvQmrVJyk9ici48whArVJzKSESvTEDizV3y1SD2cDIz4by5iOTaRwYyvz1XyHzWbFP\n", "z7yOu4dHnmkKSnHbn56ZhlbrRH3/RqQb0zh+8Yj9XkJKPBmGDMxWc675LVYLr377PE/3HcODHR/m\n", "o+Xv5StPhiEDH59qRZYZYNasWfTq1YuxY8cyffp0+96oZ599liNHjtCoUSNq1KjBvn37GDhwIJCl\n", "YG+2j3F2dkYIwWOPPcZXX33FsGHDGDVqFK1btyYjIwOtVltptkUoPZVC4Obmxr7LYYTWbXbbPQ8X\n", "T+5t84D9vGvoPWiyV0mCa9XDVedGYI26dG7anY+Xv0frBu15qs8odFpnXHSuDO02gu83zaN7s140\n", "C25FjZve1F1Cu1PdswanLp/k5Ycmcpd/I+rWDOb7jd+QkZmeY9hwK6fiTtK1yZMl1v4zsbtoV+/2\n", "JWpnrQv3t3/ov/Y37Ymrc9bOX1+vmmjVGmp612Jo1xFMW/oOreq1Y1S/cfY0I/uOZtHmebRr2JGG\n", "AY1x07nby2pVvy3+1QOJSrjIw10epWOjzlisFi7GnSchNZ5a3rVzlfni1XM8+FTvYrX77rvvZteu\n", "Xej1embOnGnf0ezj48OFCxfw9fVFSsnBgwepVi1LgdWtW9e+MdHJyYnt27dTvXp11Go1YWFhxMXF\n", "YbVa8fb2xt3dncmTJ6PT5T0/VlFQdikXgtTUVEYNGcurD0wqF2b5BeGXA9/x1eLPC5T21l3Kt2Iw\n", "GHjqoaeZ0Pd/aNQV43208sivzPjxwwKlvXWXclVF8fxWhnh5efHeF/8j/MpxR4tSII5HHubh5waW\n", "WHnOzs58PHc6e87tKLEyS5Nzcadp07+5o8WocihKpZCEhoZyQRPOjpObS7Tc/OZGCsueUzvJDE6i\n", "5z09S7TcevXqobnLxh/7lpZouSXdcz11+STndWEMHVa+txNURipGH7YcIYRg2owPOX78OD98sYi0\n", "q3oCfYJw1hR9PJxpyuS6PjXPuYGCYLZZuJJ0CfdaLjw59nFatGhRKkuUL786gYiICBZ+/gNpCXpq\n", "e/rh6uSWf8Y8uBh3npDa9YtVhsVm4XJSJC4+Op4Y9xhPtHq40kx+ViQUpVJEWrRowayFWZH8Ll26\n", "hMFgKHJZy5YtY9/Gf/h+2vfFkkmr1RIcHFwmD1KTJk34dP4MAK5cuUJGRkaRy9q2bRt75/3DK5+N\n", "K5ZMWq2WunXrKruSHYzy7ZcAQUGF2+h3K6tXr+bYsWOEhITc5iu2IhAYGFis/K+++iphYWEEBwdX\n", "mhWQqozSN3Qw8fHxHD58GClllYizcyupqals2bIFKWWO0KMKFRelp+JgoqKiEEJgtVqJiIigRYsW\n", "pVKP0WjMdzI0OTmZtLS0fIdyGo2mxIYYN+w1bDYb27ZtK7V9LyaTyR6APTdsNhtGo7FM218ZUexU\n", "HMw777zDtGnTAOjfvz/r1q0rdplSSuLi4ti8aRMrF/9M5KnTZOrznvOwSknM9XRsSOp4eqDKZ4LX\n", "28eHxm3aMeLZZ+nQoQPe3t5FknXevHm88MILAHTs2JG9e/cWqZxbSUhIYNvWraz45WfOh50gPT0t\n", "T5N+m5TEpmdgtFip6+WBJq95KQGenl7Ub96Cx599js6dO1OjRo0Skbs8UFw7FUWpOJjmzZsTFhYG\n", "gE6nIz09vdhvwXffeYedK3/judB6uGm1aFQiXyWx5Phplp08jUAwqXt7OgTkvhIlyXoILVYbMel6\n", "vj8dydTPPrebqBeGzp07s2fPHiCrB5Cenl7seZWvvvySH+d8wQtNG1DN2QmNSpVv+zeei2TegRPY\n", "pGRcx5bcWz/veTKrlFhsNq7qDSyMuMCL70zhqaeeKpbc5QXF+K0Cc+nSJU6ePGk/N5vN7Nq1q8jl\n", "2Ww2hj/8ENq923i9TROqOetwUuf/QAFcSE7Fast6UH4/eTbPtAJQC4FOoybE24N32oaybsYHvPbK\n", "K4Xyqh8bG8u+fftyyP/bb78VOP+tSCl5e+JEYlYu5r0Ozanl5oKTWl2g9kempGGx2bBJybozF/NN\n", "rxYCnVpNoIcb/2vblCPfzWHcmDGYTKYiy19ZUJSKAzl27FgOOxKbzcaKFSuKVJZer2fI4Ifobkmn\n", "Te3CO/i5u44fPi7O+Lg480yb2/c25YVOo+bBkAAy9/zDd99+W+B858+fz3Fus9lYsGBBvnMfd8Js\n", "NvPayy/jdmQPXQL9Cp2/rX9NfN1c8NA5MaJF4dxWatUqegf543HqGNM/LNiWgMqMMvzJByklV65c\n", "wd/fv8S90vfr14+NGzei0+mQUmIymWjUqBEnT54sdF2LFy8m4Zf53FW9eDtyi8vs/cf4dv3f1K+f\n", "vyHbqFGjWLhwITqdDpvNhtlsxtfXl9OnT9s35hWUf//9l/Vvv0qnIiiUkuTnE6eZsuR3QkNDHSpH\n", "cVDmVEoZg8FA3bp1OXHiBLVqlWy0kbp162K1Wvnhhx84dOgQv//+O8ePHycpKem22L55kZ6eTr/2\n", "bXmrbeHesKVBqsHEvzXqMnfevHzThoaGEhcXx/r161m8eDGnTp1i69atnDp1igYNGhS4TqPRSOcW\n", "zXm7TSOcHBxMLMNkZlmmihWrVlVYa15lTqWUOXToEOnp6bzzzjvMmTOHmTNnEhERAWT9mB944AFi\n", "Y2OZOXMm33//PWPHjmX48OGcO3cOgIsXL/L444/z2GOP8cQTTxAfnxUS6cyZM0gpWb9+PQ0aNMDT\n", "05M///yTli1b8tNPPxVKxhMnTtDS2z3/hGWAl7MTyUcP5psuKiqKxMREVq1aRcuWLXFzc2Px4sV0\n", "7dqVH374oVB1RkRE0MzD2eEKBcDNSUu1xFiMRqOjRXEYilLJh7Zt2+Lu7s6HH37I+PHjefLJJ5kx\n", "4z/z9OTkZPz8/Lh48SInTpxg7ty5TJw4kaeeegqbzcbYsWN57bXXWLJkCYMHD+aLL74A4KeffmLT\n", "pk057FICAgL466+/+PPPPws1r7D4l1/oXqd4+4ZKkiBnDceP572T+++//2bFihV06/ZfqChfX1/+\n", "+usvTp06Vagwslu2bKFfSPGsekuSpt4exZpwr+jku3YphPgeeABIkFI2z76WaxB2IcQk4Fmygra/\n", "JKXcVDqiOwZfX1+OHj3KlStXmDdvHt9884393iuvZMXDad68OTExMZhMJk6dOsXjjz+Ou7s7KSkp\n", "NG2a5Y1/4sSJeNzBI5uvry+rVq0q1K5dQ0YGbk7lxxWht7OO8PDwPA35hg4desf23+ixFKb9yYmJ\n", "NNKVn+0NLloNYSdO0LdvX0eL4hAK0lNZBNwaju9GEPaGwJbsc4QQocBwIDQ7z1whKog3ozwQQth7\n", "Dmq1milTpvD555+zdu1aGjX6zznzjdWM5ORknJycUKlUeHt7s337drZs2cKhQ4f49ddfAe74QN3A\n", "2dm5UBO1VmPRNzOWBk4C+zAvN/Jqv06nK5StTmZGeoHTlgVOQhAbF+toMRxGvg+8lHInkHzL5YFk\n", "BV8n++8NV2GDgCVSSrOUMhI4R1bUwgqLk5MTrVu3Zs6cORw5kuVT9cEHH2T58uUMGjQIZ+f/Yu7M\n", "mTOH/fv388477zB69GicnJwYO3Ysb775JmfPnmXXrl1s27bNUU25I6kGI5Ep19Gbb7cvSTOZiEy5\n", "TlohbS8EFGlZuKgUZ9I/+no6kSnXsd2hDJuURKWmEZNWeKVVlu0vbxS1F5FbEHZ/4MpN6a4AAUWs\n", "o1ygUqnYsGEDI0eOtFt6ajQaQkNDmTRpUo608+fPJzExkXHjxvHGG28AMHr0aD755BPOnDmDTqej\n", "f//+Zd6GvLiakclL67YzddserDc9WFabZMrWPby6YQdpxtydUVd0Vp06z4t/beNSyu2xka7pM3ll\n", "/XaWh51xgGQVl2IPTQoQhL1irB3nQ8OGDQkNDSU1NZX/+7//o0mTJnh55YyMp1ar6d+/Py1btsxx\n", "3c/PjxEjRtC3b99ytxGtfnVvJnRqTcTVJH499t8u6fmHTnAuKZXXOrfFz6N4DpjKMw82qodGpWLf\n", "lbjb7u2/EofJauOekDoOkKziUtRfeG5B2KOBm/8DgdnXbqMiBmiHLEdA48aNo2HDhjmuv/zyy4Wy\n", "LSktbFJittnQFXBORgC9QgLZezmWP0+dp4mvDxabjc3nLnFPSCBd6hY/XGpZYpMSo9WKSwGVt5+7\n", "GzVcXdh47hJDQu9Cq856zxqtVn47eRYPnRONaviUpsgOp7wEaM8tCPtq4FchxCyyhj13AfvvVEBF\n", "DNAO4OrqmiNg1A1uVTKOIN1k5o2N/5BpsTC+Yyva+RfMWE8Iwetd2vLSum3M2HUAi03i7+HGuA4t\n", "889cjtCbLby1eRcpBgPPtWlGj+D8l5m1ahUjWjbms38PsSsq2t4rORJ7laRMA2M7tMRZ43j7l9Kk\n", "zAO0CyGWAD2AGkKIy8Bk4GPuEIRdShkuhFgOhAMWYGyFMZ11MB999FGRN9OdOH6Mf6SNa/pMrhuz\n", "JlWXnjhNa7+aqAvoo1anUTOuYyve/vtfAF7t3KZMjcm+//77Irs9CA8LY5XVTLLBQHJmltHZoiPh\n", "dAr0Q1cAhdAjKJCfj0aw5cJlu1LZdSkaDyenAikmhZzkq1SklI/lcqtPLumnA9OLI1RVIzAwkB07\n", "ih724o0XX6Sv2sD5pFQ+33MYKaGul2eBFcoNNp27ZD8+HJtAfZ+i+UgpLDqdzu7+oCh8OGUKHVJj\n", "iEpN59NdB7FKSR0vd5wK2MMQImtD4baLV8g0W7BKyf4rcTT29SnwMErhP5RvrByg1WqpV69ekfN7\n", "enoQqNYS6OmBTqMm3WSmdyEnF9ecvsCOyCvcf1cIUalpLD5+ikbVfWhRu/SdDwkhitV+72reBMo0\n", "Aj09mHJPJy5fT+P+u0IKFWW5f8MQNp67xM/HInDRajBYrQxv1pBKEC+9zFGUSiWjKLt0Uw1Gfjoa\n", "TotaNXihQwuklDy3ajNf7jvCvAG98/aCVs5oWduXlkVw/RDi7Unr2jVZfzYSNyct9ap50rRm9VKQ\n", "sPJTcX4tCqXCdaOJ8X9tAwQvdWoNZPUcXurUmoR0PW9v2Y2lihhy9W8YjMVmI9VgpFtQhTavciiK\n", "UqkUFL2Pvu7MRXRqNRO7taOWu6v9eqvavoxs3ZTEjEwORudtcn8rUogytcdRqUpmjBLqm7V0rFaJ\n", "YikViSx39khlSdVteSVC7ewM5swi5X20eSMebd7ojveGhDZgSGjB/ZrcINNio16dsjMYc/P0hmtR\n", "xS4nxZC1ctQ9KJCabq75pM4do00SVLd4saAqMkpPpRLg6e3N1YyiKZXSIFGfeUdbntKiV+/exKXr\n", "i13OmtMXUAnB4CbFC7+aYTLTtl27YstTUVGUSiXgiSefZFtU+dkVexkNwcHBZVZfzZo1WXvTcnhh\n", "MFgszN5zhLc272LT+Uu0qu1LHa/cd1AXhGOp+tu2alQllOFPJaB58+bEap3zT1gGXEpNo2nvsvUj\n", "0rBhQzKq+aI3W3DVFu4nbZOQYjCgUanof1cwz7VpjqYYczRxGXqcGzWtkOFrSwrFR20lYfv27fz2\n", "1iv0axBUjGnb4iElfLT3GH8dPFxox9XF5dy5c7w/fDBDG9d3mG2JBL4+dJIft+8qcX/GZYnio1YB\n", "gG7dulHzvoEcjEnIP3EpYJOSOYfC+HLZb2WuUADq1atHjxcm8HfklfwTlwIS+DnsDO98t6hCK5SS\n", "QFEqlQS1Ws2UqVPxemAIf5y+iNladrYl1/SZLIi4yJTvf6Jt27ZlVu/NqFQqnhs1inZjXmJx2BlM\n", "Zdj+FIORhcdPM/7LeXTp0qXM6i2vKMOfSsiO7dt567VX6eWpI9jLAzetBid1/u+PdJMZF40GdT5z\n", "CjYJJquVNJOF44nJ6EMaM/XDD8t0cjYvDh8+zPOjR9PTXUMDHy/c1Gqc1Op8h0USiE3LwD8f/zE3\n", "2p9hsRDdCZLtAAAgAElEQVR+LYUY3wCmzfikXOxULwmUuD8Kd8RkMnHt2jUWzJ/PgT3/EnnhPNKa\n", "93d+Xa9HAB6uedtouLi60qBxE3r07s3QYcPw9vYu8UBrxcVisXDt2jV+XLSIvbv+4cK581jNeXuw\n", "s9psRMXFEeKftw8ZnbMz9e5qSOeePRkxYgQ+Pj6VythNUSoKJcb48eM5f/4869atyxGOtapw8OBB\n", "unfvzqVLl/D1Lfz+ocqCMlGrUGJs376d8PBwMjPLjyFdWbJr1y6MRiOxseXH5qcioigVBQCio6M5\n", "efIksbGxpKWlOVoch7Bu3Tp7kHiFoqMoFQUAtm7dikqlwmKxsHjxYkeLU+akp6fb/bQuWrSoSoct\n", "LS6KUlFASsk333yDzWZDSsns2bMLFXa0MhAeHm5vs16v5+TJkw6WqOKiKBUFLBYLV678ZzQWHR1N\n", "VFTxd/1WJA4ePGgPAGaz2Vi2bJmDJaq4KEpFgfj4+BxKxWazcfDgQQdKVPbcqkRKMmRFVSNfpSKE\n", "+F4IES+EOHHTtfeEEFeEEEeyP/1vujdJCHFWCHFKCHFvaQmuUHLMnDnTfnxjKXndunWOEschHD9+\n", "PMf5oUOHcihahYJT1ADtEpglpWyd/VkPlTdAe2XGZDLZVzvq1q1LkyZNUKvVrFmzBlMhYyhXVPbu\n", "3UtqaipeXl6o1Wpq1KiBlJLVq1c7WrQKSVEDtMOdfRhWugDtlZ3Lly+j1+vp3r07H3/8McOGDWPw\n", "4MEkJycTFhbmaPHKhEWLFqHRaFi/fj3u7u5s3bqV6tWrM3fuXCyW2wPXK+RNcXoRLwohjgkhFgoh\n", "bgSIqXQB2is7Z86coWPHjqxZs4agoCBq1arF0qVLGT16NL/88oujxSt1DAYDf/zxB4sWLeLuu+/G\n", "1dWVWrVqsX//fsxmM3p98T3KVTWKqlS+AUKAVkAs8FkeaRV7/HJMUlISO3fuxMPDAyEENpsNtVrN\n", "nDlzaNOmjaPFK3XS0tL47LPPePzxxwFwcXHBZrMRHBzMli1buHSpaB7lqjJF2gUlpbQ77RBCLADW\n", "ZJ9W+gDtlY0RI0bkOL+xrKrRaHjiiSccIVKZ4uvry5NPPmk/v6FUICtyZGBg5Q97Wi4CtAsh/KSU\n", "NzZIDAZurAxV+gDtlZkbPZWqjKura5X7DspDgPYpQE8hRCuyhjYXgTGgBGivDFT1f5eLi0uV/w6K\n", "S1EDtH+fR3olQHsFRemp5Bz+KBQNxYZEIQdV/YFSlErxUZSKQg6q+gPl6uqqDH+KiaJUFOwowx+l\n", "p1ISVB7HmpWcb76Zx7WklFKt48yZ05w7exaTpeq+qU+Gn+KLr+ZSs2bNUq1H2qwMfeRhmjRpUqr1\n", "OAJFqVQQftsSgXf3D0q1jnjXTcSr1nPEY2yp1lOeSXSNJ9xlCNEezUu1nqtnd9Dz6lVFqSg4DqHS\n", "oHH2LNU61E5uZVJPeUbr4oVK41L637X29ogFR48eLdPA9qWFMqeicBMqkFV7PkHt5OKw72DevHkl\n", "Okm8Z88e9uzZU2LlFRSlp6JgR9osWC3GrKDIVTBEB0iEUGMx6cmy6yzb7+C9995DCIFer8dgMGAy\n", "mbh06RKtWrVCp9NhNBoxGAykpaURGxtLq1at0Gq1pKeno1KpcHV1xWq1EhsbS0BAAGvXrsVms+Hv\n", "74+/vz9ms5nDhw9js9lo164drvnEdyoqSk9FAYDrseEcWTKeK4d/59TGGY4WxyHEh28mfO1U9n47\n", "lOgjf5Z5/Q8//DBSSo4dO0a7du1YuHAhGzdutO9NOnToEO3bt2fnzp0cPnyYfv36YbPZWLFiBevX\n", "rweyNoh2794dk8lEcnIyiYmJnDt3DpPJxP33309CQgI2m41Dhw6VWjuUnooCAOe2zSEjMTLrePtc\n", "Gt33JlXNv9b5Hd9gMaZjMaaTcGYbAa0HO0yWli1b8vbbbwPQsWNH+/UePXrw2GOPYbFY+Oabbzh7\n", "9myuZQwYMACTyUTv3r2BLJOBgIAAWrRogYuLS6nJXrV+NQq5UqfdcJzcfFBrnfFvMaDKKRSAuh1G\n", "oNa6oHFywyeovUNluXn3uJOTk/24Xr16QNYucn9/fwwGA0IIuzOpvJxK/fjjj6xcuZK2bduydu3a\n", "UpJcUSoK2fg27MH90y7Q/8NztH70S0eL4xACWg/mwU+ieeDjS9TtOCL/DA5gzZosLyNpaWkcOnSI\n", "mjVr0qRJE7tP4ZujIOh0OpKT/3PaWLduXWbMmMGBAweYN29eqcmoDH8U7GhdvNC6eDlaDIchVGqc\n", "XKs5rP7g4GAgSxl4eHjYrwcFBdmPmzVrxnPPPUdsbCxfffUVfn5+6HQ6oqOjuf/++2ndujX169dH\n", "CEGTJk1YuHAhW7ZsYerUqbz++uuYzWaMRiMfffRRqbVDCdBeQeg97DWq98nLwZ5CRSLh1Bbef0hL\n", "9+7dC5xn9+7dHDx4kJdeeqkUJVMCtCsoVBl0Oh2enuXfMFHpqVQQ2t19D7HX1Y4WQ6GEMGUk8/tP\n", "nxeqp1JWFLenosypVBA8q9XEUr3y+0utKhhT4+yB2yobilKpIAj3QBoqcyqVhoRTWyqt3xZlTkVB\n", "QaFEUZSKgoJCiZKnUhFC1BFCbBNCnBRChAkhXsq+7iOE2CyEOCOE2HRThEIlQLuCQhUnvzkVM/CK\n", "lPKoEMIdOCSE2Aw8A2yWUn4ihJgIvAW8dUuA9gDgbyFEQymLt5fcZDJhsViwWq3YbDaEEKjVatRq\n", "NTqdrtJOeCkoVETyVCpSyjggLvs4XQgRQZayGEhWLCCAH4HtZCkWe4B2IFIIcSNA+96iCHf06FEW\n", "//ora9esQefjhlN1d1xqeWJON2K6lo4xMR03JxfeeOMN+vTpg5dX1bUGVVAoLxR49UcIEQy0BvYB\n", "taSU8dm34oFa2cf+5FQghQ7QnpGRwaFDh3h98lvoGlTnri4tGPHcu2ictLellVKScC6aH3eu4cNv\n", "P6VX885MmDCBunXrFqZKBQWFEqRASiV76PM7MEFKmXbzcENKKYUQea2NFXjdLCIighFPP4l7Mz8G\n", "zhuPRne7IrlFLmrdFUituwLh2Qc4vzuMjvd24/XRLzF+/Hh0Ol1Bq1ZQUCghChL2VEuWQvlZSnnD\n", "c028EKK2lDJOCOEH3AjYXuQA7SaTiTFvvsgTCyfi5uNxpyz5Ur9zM8av/phN365m68MP89dffxWp\n", "HAWFqkRJB2jP00xfZHVJfgQSpZSv3HT9k+xrM4QQbwHeUsobE7W/kjWPEgD8DTS41Sb/VjP9Tz/9\n", "lOX719P/f0/g4uVe/FZJSdiGfUR8v4P169aVeriFskDZUFi5KMqGwrKitM30uwBPAMeFEEeyr00C\n", "PgaWCyGeAyKBYVC0AO27d+9m4fqlPPntG6jUeZvNWIxmMpKuA+BW3fOO8ywACEGz/p1w9nRj/IQX\n", "+WHholLzx3kzZrOZU6dOoVaradiwIRqNhqSkJDIzM8nIyMBkMlG/fv1S9bqloOBo8lv92UXutix9\n", "cslT4ADt0dHRPPXiKJ777b0CLQtf2HuSRc9kFT1q8WTq390sz/QNujRny4EI/vf2/5j9+eyCiFRk\n", "TCYT7dq1o1u3bsTHx2M0Glm1ahUnTpxg1KhRDBo0iAsXLlC9enW+++67UpVFQcGROMyi1mKx8OST\n", "T9J/8lOlamdyz4uPsPPsIa5cuVJqdQCEhYURHBzMV199xdKlSzEYDKSnpwMQGhrKp59+yu+//05U\n", "VFQOb1wKCpUNhymVM2fOYKnrQkCL+qVaj0qtosPIexk7bmye/juLi9FopGXLlqhUKjQaDUFBQRgM\n", "BiDLSzpkjVV9fX25evVqqcmhoOBoHKZUBj48iM6jHiyTuoLaNSZaphATE1Nqdbi5ubF9+3bMZjOZ\n", "mZmEhYXZ504WL16MlBKTyUR0dDQBAYUy3VHIxpyZSnrCWdITzmLKSHK0OAq54DClkm4x4OVfvczq\n", "6/r8QH755ZdSK79Zs2aEhobSvXt32rVrR9++fXF3z1rJ0uv1NGvWjKZNm9K3b1/c3NxKTQ6HIG2k\n", "XDlG4vl/Sbq4F5vFVCrVRO75gU3vt2Dje804tvyV/DMoOASH+VPp/sKgMt2z4+1fnfmvfMibb76J\n", "RlOwZkspSUpKonr1/JWfSqVi3rx5xMbGolKpqF27tv3e+PHj6du3L0ajMcf1yoIh7Srntn5J4oW9\n", "6JMvU6fdcNo9WfKT0Xf1msBdvSYQvvZ90uIiSrx8hZLBYUql1aBuZVqfs6cbFo2NyMhIGjRocMc0\n", "NpuN6OhoYmJiWLp0KZGRkYVaqRFC4O/vn+PaDc/oBVFMFRVnz1q0GTGPjVOa0HLITCI2fIS0WRGq\n", "LPeX187vJunCHgLbDiM9/jTX4yJwrR6Mb8MeaLMDoZ/bNge36sF4+jfl2vl/kTYr1eq2xSsg7xW+\n", "G5j0yUTu/gGkFY/aTfBr/gAJp7eSEnUElVpLSLfRqLXKUn5Z4DClckebFAkyD6v+HCYvkjw9Z93a\n", "CxJCULtRkH1F5kZ5RqORtLQ0fv75Z2bPns2VK1eQUhIQEMDx48fx8fEpeKPuQKdOnYqVv6KgT4zE\n", "kJZAUOeRRB9bRdS+Xwi6eyQAao0TJ9e+x6mNM9Do3KlRvwsR66bh7OVH70n7Uam1XI8NJ2zVO9is\n", "Zmo2ugeTPpmUy0fp8vxKajW9L9/6hUpD/MmNpCWcpu0TWS8Cjc6DqANL0Xn4Uq/7mFJtv8J/lCt3\n", "kptnL+PMjqO53jek6e3HqyYvQOee+5un5YOd6TpqQI5rNer5k5mZyfXr19mxYwffffcdJ06c4MqV\n", "K1itVqSU9t7GwYMHi61QShPD9TiOLX8Vq8VA80HT8PBr4lB5oo/+SeN+b6HWOFOv6yiOLn+VOu0f\n", "RaXRUS2oHWqtK/4tB9D60Tmotc5YTBnsnjuYY7+9RutHv8SvxYNE7V/CfVNO4lYjGKQk5vhq/v3m\n", "Ie6fdgFnL78869c6e3D38ytY/24jnFyz3Pu41wjBlH6NHq9uQaVR9oGVFeVKqSRfTuDK8fMFSnv1\n", "Qt4rOUFtGuU4lzaJISOTBwcMIDUlBZvNdseejlqtplWrVrz88ssIIRBCoFKp7vg3t+OC3r/TvZ49\n", "e96xPVFnD3PNNhOEQKAi6dJ+Lh9YBoDW2Yv2Ty8q0PdWGljNBs78/TnV63Vi//dPYjZcx5AaQ3rC\n", "eTz9Q+3pAlsPQa11BkDj5EbNxr2IOboKsv8P3oEtshQKgBD4hHTEZjGRmRKTr1IB0Ojcqdv+UcL/\n", "+oAuY1dxfOVEAtoMsQ+xFMqGcqVUPHyrUaOef673zXoDqXFZS4leftXRuuT+9nGrnvOHJFQCFw9X\n", "vp4zBycnJz755BNiYmKIiYnJoWAsFgudO3fmtddeQ61WY7PZsNlsdgdRUkp7r+bGvRvXbz2+9drN\n", "n5vLyKusG+curl64VQ9BSitS2lDHuIAQIKV97sJRJJzagpNrNRr2fc0eg9nZy5/z/3xD60e/sqfT\n", "J13KkS8z+QoaZ/esdgCGtASk1YJQZ/0sTRnJSCRqp4LPhTS693U2fdAKfWIUceGb6PXmv8VtnkIh\n", "KVdK5b43H+fe1x/N9f7Zncf4cdQMAIZ8/Dz1OjXNNa1Q3T5nc/V8DI0ea0Tr1q0ZPHgwmZmZxMfH\n", "88UXXzBv3jy7d7l3330XV1dXXnml/Cxb+gaspnrbR+znAS0H4eIdSEbiRVoMmeFAyeDKkZWEdB1F\n", "jfpd7NfcfRuw/p36hD4wGZ2HLwBHlr2MSuNM3Q6PcW7711zYOZ8er22z50mLO82O2X3oOGoJlswU\n", "9nw7BO86rfGo1ei2OnPD2cuPhn1fZcN7TWhy/zu4+ii+dcoahykVabPd9uCr1CrIY1OhSv3fG1ml\n", "UaPWFlx8KSXxZ6LstiNCCFxdXQkJCWH27Nl88sknXLhwgX/++Yc1a9YwadIkXF1dGTOmfE7wqZ1c\n", "aTboA0eLwYV/5pF8cT8pUYeRNguN7n2D5EuHOLT4eVyqBRK2+l3ajsgKBt760S9IunSAs9u+xNUn\n", "iK7j1+ZQRL53dSO48zMcWDQSizGdwHbDaNT39ayemJTsmjuIzOQrSJsZKSWbP2yDSq2h46gluPv+\n", "Z5lds9E9hP35NsGdnizz70PBgUolbP0+mj9wd5nVZ0jTI/WWHMGub8bJyYnGjRvTuHFj/u///o/U\n", "1FROnjxJVFSU4kkuD+p1f5563Z/Pca1aUFv6/O/AbWndfRtQr1vuSlql1RF890iCs1eNciAEXcet\n", "LpBMUfsW07DPq7hUU4KvOQKHWdTu+PqPQviEKz5p8Uk88/hTODk5FSi9l5cXnTt3VhRKCWC4Ho+0\n", "mkm6dIj4iC1YTfrb0qQnnMWUnkR8+GZSruS+Apgb0mbh6pkdXD64jIu7F1Gn7VCkzVoS4isUEsfN\n", "qegtpMYn4VW7bJZt//3uL5bNXFgmdSnkJOXyEeq0fwx9YiSZyVE4e9XCyz+nUVtm8hV8QjoQffQP\n", "nD1q4R3YqlB1WE2ZxBxfjdWUSVDHJ4g6uBQPv1A0ukq2JaIC4DClsvinX5j642zunfh4qdcVfewc\n", "Xqnq26xdFcqG2k37UbtpvzzTtBgys1h1aJw9aPmI4hmvPOCw4U/nzp0xnbxK3OmoUq1HSsmeHzYw\n", "d+7cAg99FBQUio7DlIpWq+WnH39i3Xs/lGo9u75bQ5tajXPd76OgoFCyONROpV69esyb/gUvj3+b\n", "x758+Y62JTcT1K4RE9ZldZN96tbKMy1A1JEziPBUPvtlbonIq6CgkD8OD9Deq1cvBrbpw/ppP2NM\n", "z8wzrc7NhdqNg6jdOAgnV+c8057ZcZTt7y7mu3nf4uFRtJAfCgoKhcfhSgXgww8+4P96Pcri//sU\n", "Y0beiiU/bFYbe3/cwJmFOwg/GY6fX/57RhQUFEqOPJWKEKKOEGKbEOKkECJMCPFS9vX3hBBXhBBH\n", "sj/9b8ozSQhxVghxSghxb0EFeeKJJ/ht3s8seWIG279eidVceH+yMRGRfDPkbZpZ/dm0cVOh8yso\n", "KBSf/OZUzMArUsqj2aFPDwkhNpNltjZLSjnr5sTZwcSGA6FkBxMTQjSUUtoKIkzLli3Zt2M3386f\n", "z+LnPqNm23rU79KcoDaNEKo7e4lLibnG+V3HOb8rjKbewWz5bR0NGzYsSHUKCgqlQH5xf+KAuOzj\n", "dCFEBP8FXL/TUz4IWCKlNAORQohzZEUr3HuHtHfEy8uLN994gzdef50dO3aw+NfF/DD1V9xqe+Na\n", "2xu32t4Y0zLJjE/FkHAdMiy8+OKLPPjdlErpqlFBoaJR4NUfIUQw0JosBdEFeFEI8RRwEHhNSpkC\n", "+JNTgVzhPyVUKG74FunRowdGoxGz2YzNZsNsNqNWq1Gr1Wg0GnQ6HWq1Y7f+Kygo/EeBlEr20GcF\n", "MCG7x/IN8H727Q+Az4Dncsl+xx0+twZoz805kRACZ2dnnJ3zXu1RUFAoGiUdoD1fpSKE0AK/A79I\n", "Kf8EkFIm3HR/AbAm+zQaqHNT9sDsa7dxs1JRUFBwHLe+1KdOnVqs8vJb/RHAQiBcSjn7pus3r9MO\n", "Bk5kH68GHhVCOAkhQoC7gP3FklBBQaFCkV9PpQvwBHBcCHEk+9r/gMeEEK3IGtpcBMYASCnDhRDL\n", "gXDAAoyVebm8V1BQqHQIRzzzQghF1xSSu3sNJtW1YDFwFMo/Gdci+fmT0XTv3t3RotyGEAIpZZEj\n", "/ZUrH7UKuWPSJ3H1kmLQV1kwG66jUpVPV6XFRVEqFQTPOu3o+UzxfI4olB/iTm7IMxheRUZRKhUF\n", "gT38hULFp7j/SyklZ8+eJSQkBK1WW+j8aWlpJCcnl4q7VOVXqqBQATEajfTs2ZPExMQi5T979iwT\n", "J04sYamyUJSKgkI5Qa/XM3z4cPz8/Bg3bhwWS9am2uee+8+udPPmzezcuZMtW7aQmJjIAw88wJgx\n", "YzCbzUycOJEXXniBgIAA1q5di5SS8PBwpkyZYs8/dOhQMjIyeP/999m8eTOdOnXixIkTHDlyhAYN\n", "GuDr61vsdihKRaHcYcxIJGLdNAypcY4WpUyZMmUKQ4YMISIignr16jF58mQALly4YE+TkpKCyWSi\n", "e/fuVKtWjWXLlvHpp58ipWTBggW8+eabHD16lAkTJnD69GlMJhMxMf+FCD537hw2m41JkybRo0cP\n", "1q9fT5MmTXjrrbfYuXMn588XLOxwXihKRaFcYc5M5e8P2xK26h22zexCATe4Vwr27NnDI488gre3\n", "N4MGDWLJkiX23sqtaLVaVCoV7u7udidknTt3JiQkBF9fX5599lmSkpJyrUur1eLk5ES1atXQaDTc\n", "d999jBkzhgULFhS7HYpSUShXWE16jGnxABhS45EWk4MlKjtcXFwwm80AmEwmPD09UalUWK1WjEYj\n", "AFFRWY7iRXb86ZtXkFJTU+3nSUlJqFQqhBAkJGTtqtHr9Vy7ds2e32b7T2G/+uqrLFmyhDp1bt5l\n", "UzQUpaJQrtB51KR20/4IlRr/Vg+h0uocLVKZMXLkSB555BGuXr3K9OnTefvtt1GpVNSpU4cPP/yQ\n", "rVu3MmfOHCAroqaXlxfLly/n7NmzAJw4cYLly5cTERHBggULCAkJoVq1amzcuJEDBw7w3nvv2RWJ\n", "j48Px44dY9++fWRkZLBkyRKMRmOJOIhXlpQVyhVCpabjqF9Z+aIrzQdP585ueyonw4cPp2HDhvz8\n", "88+88sortGnTBoDZs2fz+++/Ex0dzdq1a9HpdAgh2Lp1K5GRkVy/fh2AAQMGUK9ePTZs2MCRI0eo\n", "VSvLOfy+ffvYsmULTz75JEOHDsXZ2ZmgoCB+++03rl+/TmZmJg0aNGDhwoW4uLgUux2KmX4F4Z5h\n", "r+Hbp2oEy7py6Df2zB9G0wFTCX1wsqPFKRXiTm5g2iNudOvWrUTKM5lMjB49mh9//LHYZSlm+lUE\n", "abNhNRfPKXhF4cKurMnC2LC/aHTv6yAqX2/FZjECJReSVaPR8Morr5RYecVBUSoVhLsbuXP1ROkY\n", "K5UnLBYr6VFZzgPToo/jfXxirv6JKzJeVgueniW390elUtGqVeHiT5cWyvBHoVxx9epV6tevT1pa\n", "GkII1q5dy/333+9osaoUxR3+KKs/CuWKo0ePkpaWBmQtl06aNKnSbryrrChKRaFc8d1339ltMADC\n", "wsKIj493oEQKhUVRKgrlBr1ez+rVq5FS5jDuunz5soMlUygMilJRKDdERkZisVhQqVRUq1aN4OBg\n", "pJR8//33jhZNoRAoSkWh3LB7926klHz77bd07tyZOXPm4OPjw48//mg3X1co/yhKRaHcsGPHDmbN\n", "msUzzzyDs7Mzfn5+XLhwgdDQUI4ePepo8RQKSH4hOpyFEPuEEEeFEOFCiI+yr/sIITYLIc4IITYJ\n", "IbxvylOkAO0KVRubzUb//v2ZMGECarUaZ2dnDAYDXl5ebNq0CYPB4GgRFQpInkpFSmkA7pFStgJa\n", "APcIIboCbwGbpZQNgS3Z57cGaO8HzBWKD0SFAvL444/bj11cXOyKxMfHp8TM2RVKn3wfeCmlPvvQ\n", "CVADycBA4MYmgx+Bh7KP7QHapZSRwI0A7QoKeaJS5fwp6nQ6pXdSQclXqQghVEKIo0A8sE1KeRKo\n", "JaW8YTwQD9TKPvYnKyj7DYocoF2hatO8eXNFqVRQ8t37I7Ncb7USQngBG4UQ99xyXwoh8jJ5LFaA\n", "doW8sdls7N+/n42rNhN9Ohasjh9tCgFaVw3Bzepw74N9adGiRaHLqF27Nnq9Pv+ECsWmpAO0F2rv\n", "jxDiXSATGAX0lFLGZcdV3ialbCyEeAtASvlxdvoNwBQp5b5bylH2/pQA169f58lhT/FU++cJrF7y\n", "oRZKgl0R24h2vsCnn8+8bYiTF6tXryY5OZmRI0eWonQKd6JU9/4IIWrcWNkRQrgAfYEjZAViv/Hf\n", "Hgn8mX2sBGgvIywWC8+NGM2Ee94utwoFoGuTe+js0Yf3p3xQqHxCiFxtU5KSkpg2bRoTJkzg8OHD\n", "ABw4cMDuIHrlypXs35/1s9u0aRMXLlzgyJEj7Nmzh8mTJ/PGG2/YnUFLKVm7di2PPPIIs2fPtu8z\n", "WrFiBadOnWLixIkcP368SG2vquT36vADtmbPqewD1kgptwAfA32FEGeAXtnnSCnDgRsB2tejBGgv\n", "NXbv2s1jrZ7FTVdyPjnyI9OUSdTVyELnq1MjCKcYD06fPl3gPEIITKY7+6cdMWIELVu25MUXX2TU\n", "qFGEhYXh7e3N119/jZSSKVOm8NJLLwHw1VdfERAQQHh4OC+//DIDBw7koYceYty4cUBWj2jZsmW8\n", "//77hISEMHToUGw2GytWrGDWrFmMHDmSoKCgQre5KpPfkvIJKWUbKWUrKWULKeXM7OtJUso+UsqG\n", "Usp7pZQpN+WZLqVsIKVsLKXcWNoNqKps+H0zAdWL76T4Vub99QUG850nSK+mxLPuwKoildu7eT+O\n", "Hiq4AZtKpbpjT0VKidls5sEHH6RBgwa88847nDp1Ch8fH/7880/OnDlD69atOX36NDExMRiNRnS6\n", "LD+3zzzzDO3ataNdu3acO3cOgHnz5tG1a1eOHTtGfHw8mzdvtu+Sfv755wkNDcXLy6tIba6qOH5W\n", "T6FIpMflPolpzFYKlxIiMZmN9msJqTl3+8anxHEm+pT93Gw1sztiJ6kZKWSaMrFJG3qjHpu0cSnh\n", "IgE16vBU71FIKdEb9TnyGUz5e6ULOxhe4PblNvwRQuTYxaxSqdDpdHh7eyOl5LPPPuOZZ57h6aef\n", "Zv78+TRv3tyetmnTpvYybjiAllLSr18/+vXrx9ChQ7l06RKenp4AJeJZviqiKJUKiiqPhbvJP7/J\n", "tKXvsjt8Oy/MGcl3G77mh7+/Y9qSd1i24ycAth7bxI7jmzl4di+PfTyQ+ORYLl+9xHV9KtuP/82h\n", "s/s4H3OGwR/0ZfrSyZy+Ek5k/AW+XPUJQgh+2Pwty/75GaPZwMvfjuFqakK+Ml+/ml7w9qlUuca8\n", "UReYqlIAAA7MSURBVKlUbN68meTkZN5//32Cg4NRq9WMGTOG5cuX06pVK8aOHcvXX3/NsGHD8qzn\n", "ueeeY+rUqVitVsxmMydOnMihtBQKj+JOspLSr90A2jboQHJ6MhfizvHR059z+eolvlw9k+E9nqJn\n", "iz6cuRJBSkYKTeo05VzsGbqE9sDbvRoPdHgId2d3zkafwmQx8crgt3Bzdud87Fl7+U/1HsXYr59h\n", "67FNDOw4hDq++c87FGZ6Ta1W56pUFi5cyOLFi5k7dy5z586190aGDx9OnTp1qFatGp6enkyfPt0e\n", "ciI0NNSuLFQqFW+//TYADz/8MD4+PowePRohBKNHjwayhko3hk0KhUNRKpWUtg2yDJl9PKrjpMl6\n", "OPx8/DFmD4c++PVtUvUp9Gp5L37VA0m8fu2O5dT2ro2bs/tt191c3BnUaQjfrv+Kni37lrj8KpUq\n", "R7CrmwkMDLxjcPHg4GCCg4OBLKV0Q0EAtG7d2n6s0WjsWwLUajW9e/emd+/eOcq67777ituEKosy\n", "/KmgSIoeDtRkMbH/9G5eHjSRBzsM5kx0hP2eVqPFZrPmW8bxi0f4Y/dyxg94jVkrpxeoXo2TOs/7\n", "N6LwQdbDbrX+J0dqamqB6lBwPIpSqaBo3XJ/QKu5+9iP3Zzd8XDxyD4T+LhXR6vR8ki3Ecz6Yzqf\n", "//ER9f3uwjV7afqhu4fyyW/vs3jbIjQaLTW8atnL0qg1eLl5k2nKZMWuJXz5wgIGdhqCh4snu05u\n", "z1fmkGZ5D5FWrFhBRMR/Cu5GTyUhIYFp06blW75C+UDxpl9B+eTDmXRz7V8oK1VHkmFIR9fVTJeu\n", "XXJNc+zYMbp27fr/7Z15cFRVFoe/kx0ImM2EjqAE0QTGUYK7YMANcLAcqZqFGtdxn1KcKssNxhlF\n", "q9RyQ0ZLLEctAUUGRw1hcUGGUoQRNSQSDMEEExI3IiEhbCHd6Tt/9AsmkO708l4nMeerSuXlvX73\n", "d0+6c3Lf3X58/vnn1NfX8/777zN79mwKCgqYNm0aDz30UBRr3H/R3fT7Kb+/8ne8svb5nq5GULR5\n", "23jgzXvIH5cf8HV5eXm0tLSQn5/P4sWLWb9+PWPGjGHz5s1MmzYtSrVVIkVbKn2YJW8s4ev3dzD5\n", "tMtsGwbtuOm0HRxyt7Dk0wXMeuauoOZ9zJw5k+eee47ExETcbjfGGGJiYmhoaNBJaFEi0paKJpU+\n", "TklJCQvnLebEwbmkJ2cwMOHokZpQ+LD0PS46bUpEicUYQ9OB3ew52MT21grum3M32dnZQd1bVVVF\n", "Xl5ep07aiRMn2rqKVgmMein3c/Lz88l/NR+3283GjRvZUb0jbPOtAwcOsGDui5w2LY+kpKSw6xQT\n", "F8tZY09j9OjRISenzMxMsrKyDi/4A5g+fXrYdVGij7ZUlMMUFxdz9tlnU1dXh8vl6pE6GGOYMWMG\n", "S5cuBXzzVUpLSztNt1ecRTtqFdsoLCykra2NhQsX9lgd2me1dmzhDB06tMfqo4SOtlSUwxQUFLBu\n", "3TpcLhd1dXXExgaerOYUxhjS09NpbGzkhBNOoKampkfq0V/RjlrFFpqbm0lNTcXr9RIbG0tlZSU5\n", "OTm2lO12u1mxYgXL3n6Lqu3b2b+/+20it1ZUcKi1lcHJyZw4cmSXr4kRITvbxbjTz+CmW25h2LBh\n", "ttS3v6MdtYotrF279vCx1+ulqqoq4qRSX1/PC88/z7ZVheQdk8z4QQOYfFwqcTHp3d67MsbNa19W\n", "8Owl4xkQ7/9jetDtoWnTep6c8S5V8cnMvPc+pkydGlG9lcjQpKJgjGHevHmd9hhZunQpl1wS/kLB\n", "LVu2cMeVM/jDCBfjTh4R8v2nZGaQFBdLcmI8iQEewwbGx5E+MIkTU4dwMbD07/fywXvv8fhTT/XY\n", "41t/RztqFTwez1H9FoWFhWEPTVdWVjJjysXc8auRDBsc+naXBnjmf5s44PZwa9GHtPlZrdwV03Nz\n", "yKoo4ZqrrvK7yllxFk0qCo2NjdTW1nZaR9TQ0EBZWVnIZXk8Hm6/8k/MKTiTuDDXJe1oambXAd/u\n", "dQ0HWqjbE/zmTgCjM1IZtXMHa9asCUtfiQxNKsrhXeSvuOIKsrKyuPDCCwFYtWpVyGUtXLCAs5ID\n", "P7J0x4iUIeSkDiEpLo68Y9PIHhJaa0eAM1zH8sjdd+qWCT1AwNEfEUkCPgIS8dmeLjPGzBKRB/F5\n", "//xkvXS2MeZd655ZwPVAG3CHMeaDLsrV0Z9egsfjISUlhXPPPZcVK1aQm5vL1q1bue6666irq2PD\n", "hg0hlTd9yiXccKw9O/zvOnCQjIEDwr7/4x3fcfmjc5kwYYIt9ekvODr5LYBBuwGeNsbkW1/tCUUN\n", "2vsYtbW1jBkzhqKiok7bJy5atChkawpjDOmN3e9VGyyRJBSA8cdn8+a/l9hUGyVYwjVoB18r80jU\n", "oL2PMWTIED755BMGDOj8B5yQkMD8+fNpaGgIuqyamhoyI0wEdhIrQnO9fUlOCY5wDdoBZorIlyLy\n", "cruLIWrQ3ufIyMggISGhy2spKSmkp3c/p6Sd6upqkgLMKekJWpu1TyXahGPQPgmYD7Rvw/Uw8BRw\n", "g78iujqpBu2/PNxuN9JlAzY4Wtv8740bIxLWaJIJUKbiw26D9qD/rRhj9ojISuAMY8zhGojIS8By\n", "68fvgI478Qyzzh1Fx6SiKPta3dy+cq3f66dkpnPX+NOjWKP+w5H/1OfMmRNReQGTiohkAB5jTFMH\n", "g/Y5IjLUGPOj9bLpQPuEhiJgsYg8je+xRw3alaBIjIvlqlPzjjpf/tNuVm/fgfp79R26a6m4gAXW\n", "CE4MsMgYs0ZEForIWHyPNtXALeAzaBeRdoN2D2rQ3q8JZWvK+JgYLj7x+E7n6vbs5dmNJbgGD+LG\n", "03U/lb5CwKRijCkDxnVx/poA9zwCBGcEo/xiWVVZw3uVNfw2byQXjTy++xuOYG9rK39bsx6vgQcv\n", "OJdjErvuTFZ6H72rq17pcfbs2cPy5cvDsvxctmwZdTV1lHz3I8u3fYPH6+XlTV9xzjAXgxLigy7H\n", "awyPfvwZrR4vcy+dSHYY64eUnkOTitKJd955J+x7U1NTaUpM5JiEuMN9ILEixIYwauPxennik2Iq\n", "fmrkljNPZVRaSvc3Kb0KTSpKJyIZ2j906BCfbf6M/Kx0TkpPZVFpOVePHUNSXHDrgIwxvF1exYa6\n", "77lm7GimjAptRq/SO9CkojjCqVkZPDGlIKR7Vm+v5bUvtzLt5Bymjx7lUM0Up9GkovQKqnY38WJx\n", "GUOSEhnnyuSr+s7LAwYnJDAyTc3E+gKaVBTbiMSA7Me9+4kRobWtjSc3FB91/dSsY7l/YujLyOx0\n", "W1SCQ5OKYhsulyvgVPtAnDPcxbjsLL/XY8NMDnHJg8O6TwkfTSqKbYwaNYrGgy1h3RsXExP2TnGB\n", "SDxGR4+ije51otjGgAED+D6+92x9UL6rkfPOP7+nq9Hv0KSi2ErqqFy+37u/p6sBwPLKWi6wtsZU\n", "oocmFcVWHn3sMZZuq+7parD7YAtTr7s+5N3rlMjpN0nFzv0iVNO/ZmZmJve/8BJvbfsGr4NrSct2\n", "7vJ7bdfBFl6oqecvt91mu25P/377AppUVNN2zUmTJnHejbexsvpbWjzObJLkL6nUNu/j9Z3NrP54\n", "HWlpabbr9obfb2+n3yQVJbrcePPN3Prcv3j4868oqw9+n9twOdTWxj+/2EJx+nCKVq9h0CBdhNhT\n", "6JCy4ggiQv64cXxUvIny8nLeWPAq1ZuK2d+0G4/bHXH5O5qaqWveR1JyMmnDhnPmRZOZ94/Hyc3N\n", "7WSKpkSfgL4/jomK6MZNitKLicT3p0eSiqIov1y0nagoiq1oUlEUxVainlREZKqIVIhIpYjc66BO\n", "jYhsFpESEfnMOpcmIqtF5GsR+aCDCVq4Gq+IyE4RKetwzq+GiMyy4q4Qkck2aj4oIt9asZaIyKU2\n", "aw4XkbUi8pWIbBGRO5yONYCmY7GKSJKIbBSRUhEpF5FHoxCnP01H31OrnFir7OW2xmmMidoXPtvU\n", "KmAEEA+UAqMd0qoG0o449zhwj3V8L/BYhBrnA/lAWXca+PylS624R1i/hxibNB8A7uzitXZpDgXG\n", "WsfJwDZgtJOxBtB0OtaB1vc44FNgQhTe0640HY3TKutO4HWgyM7PbrRbKmcBVcaYGmOMG1iCz3/Z\n", "KY7swb4cWGAdLwCuiKRwY8w6fvaW7k7DFp9pP5rgoLe1MeZHY0ypdbwP2IrP18mxWANogrOxduUd\n", "7vR7GnW/chEZBvwGeKmDji1xRjupHAfUdfjZSa9lA3woIl+IyE3WuSxjzE7reCfgfwOP8PGn4bTP\n", "dFS8rUVkBL6W0kaiFGsHzU+tU47FKl17hzsapx9NcPY9nQvcDXg7nLMlzmgnlWiOX483xuQDlwK3\n", "iUinNfDG165ztD5BaNilPx/IAcYCP+DztrZdU0SSgbeAvxpj9nYq1KFYLc3/WJr7cDhWY4zXGDMW\n", "n2VvgYhccMR12+PsQnMSDsYpIpcB9caYErpuDUUUZ7STypFey8PpnAFtwxjzg/X9J+AdfM21nSIy\n", "FEBEXEC9A9L+NIL2mQ4VY0y9scDXnG1vmtqmKSLx+BLKImNMoXXa0Vg7aL7WrhmNWC2dPcBK4HSi\n", "9J520DzD4TjPAy4XkWrgDeBCEVmETXFGO6l8AZwkIiNEJAH4Iz7/ZVsRkYEiMtg6HgRMxuf3XARc\n", "a73sWqCw6xIiwp9GETBDRBJEJAcbfaatD0A7R3pbR6wpIgK8DJQbY57pcMmxWP1pOhmriGS0P2bI\n", "z97hJTgbZ5ea7X/cTsRpjJltjBlujMkBZgD/NcZcbVuc4fQaR/KF73FkG77OnlkOaeTg660uBba0\n", "6wBpwIfA18AHQEqEOm8A3wOt+PqK/hxIA5htxV0BTLFJ83pgIbAZ+NL6IGTZrDkB37N3Kb4/shJg\n", "qpOx+tG81MlYgV8DmyzNzcDd3X1uHNR09D3tUNZEfh79sSVOnaavKIqt6IxaRVFsRZOKoii2oklF\n", "URRb0aSiKIqtaFJRFMVWNKkoimIrmlQURbEVTSqKotjK/wFy06hxF2ON/QAAAABJRU5ErkJggg==\n" ], "text/plain": [ "<matplotlib.figure.Figure at 0x7fde64636190>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import Image\n", "im = Image.open('apply1.png')\n", "plt.imshow(im)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- 위 그림에서 화살표는 Python 객체에 대한 참조를 나타내고 있다.\n", "- 청색 상자는 _Apply node_이고 (Computation graph를 표시하기 위해 쓰는 내부 노드 (_Variable node_ 와는 달리 사용자가 직접 조작하지 않는다.)\n", "- 붉은색 상자는 _Variable node_ 이다. \n", "- 초록색 원은 _Ops_ 이고 보라색 상자는 _Type_이라고 한다.\n", "- 주) 그런데 z = x + y 에 대한 정보는 없다?\n", "- 이 그래프는 output (여기서는 어떤 연산)에서 시작하여 input 까지 따라가며 조사할 수 있는데 owner filed를 이용하여 가능하다고 한다.\n", "- 아래 코드를 보자." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import theano\n", "x = theano.tensor.dmatrix('x')\n", "y = x * 2." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- `type(y.owner)` 하면 `<class 'theano.gof.graph.Apply'>` 의 결과를 얻을 수 있는데, 이것은 Apply node와 input을 이어주는 것이라 한다." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "theano.gof.graph.Apply" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(y.owner)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- 이제 y를 얻기 위해 적용되는 op 의 이름을 출력해보자." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "'Elemwise{mul,no_inplace}'" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y.owner.op.name" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- 위의 출력값으로 y 를 계산하기 위해서 elementwise multiplication 이 사용된다는 것을 알았다.\n", "- 이 연산(op)는 어떤 input 에 대해 적용되고 있는지 알아보자." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "2" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(y.owner.inputs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- 두 개의 input 을 받아 연산(op)하는 것이며" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "x" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y.owner.inputs[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- 그 중 첫번째 것은 x 이고" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "DimShuffle{x,x}.0" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y.owner.inputs[1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- 두 번째 것은 예상과는 달리 2가 나오지 않았다.\n", "- 이것은 우선 2가 x와 동일한 모양의 matrix로 broadcast 되기 때문인데\n", " - broadcast: 차원(dimension)이 다른 텐서가 elementwise computations 될 수 있도록 해주는 것이라 한다.\n", " - 이것은 차원이 낮은 텐서를 차원이 높은 텐서로 가상적으로 복제함으로써 이루어진다.\n", " - 자세한 것은 Broadcasting in Theano vs. Numpy 를 보고\n", " - SciPy documentation about numpy's broadcasting 도 \n", " - OnLamp article about numpy’s broadcasting 도 보면 좋겠다.\n", "- 이것은 `DimShuffle` 이라는 op를 통해 이루어진다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "그래서 각각의 type을 한번 알아보면" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "theano.tensor.var.TensorVariable" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(y.owner.inputs[1])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- 이미 텐서이고..." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "theano.gof.graph.Apply" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(y.owner.inputs[1].owner)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "<theano.tensor.elemwise.DimShuffle at 0x7fde0c33e550>" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y.owner.inputs[1].owner.op" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- 이렇게 바꾸어준 범인은 DimShuffle 임!" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "[TensorConstant{2.0}]" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y.owner.inputs[1].owner.inputs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Dimshuffle 이 받은 원래의 input 값은 텐서 상수 2.0 임을 알 수 있다." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Automatic Differentiation\n", "- 이 그래프 구조를 안다면 자동화된 미분(automatics differentiation)은 쉽다고 한다.\n", "- chain rule 에 따라 진행 가능.\n", "- `tensor.grad()` 가 하는 일은 output에서부터 input까지 그래프를 역탐색하는데, 모든 apply node를 둘러보아야 하는 것이다.\n", " - (apply node는 어떤 연산(computation)을 할지 정의해두고 있는 node이다.)\n", "- 각 apply node에 대해, op는 gradient의 ...\n", "\n", "- 자세한 내용은 differentiation 에 있으니 읽어 보세요." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "'((fill((x ** TensorConstant{2}), TensorConstant{1.0}) * TensorConstant{2}) * (x ** (TensorConstant{2} - TensorConstant{1})))'" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from theano import pp\n", "x = T.dscalar('x')\n", "y = x ** 2\n", "gy = T.grad(y, x)\n", "pp(gy) # print out the gradient prior to optimization\n", "# '((fill((x ** 2), 1.0) * 2) * (x ** (2 - 1)))'" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "array(8.0)" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "f = function([x], gy)\n", "f(4)\n", "# array(8.0)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "array(188.4)" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "f(94.2)\n", "# array(188.40000000000001)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Optimizations\n", "- Theano function을 컴파일 할 때, theano.function에 넘겨주는 것은 확실히 그래프이다.\n", "- 이 그래프 구조는 input 으로부터 output을 어떻게 계산할지를 보여주고 있는데,\n", "- 이와 더불어 계산방법에 대한 개선을 할 여지가 있는지도 볼 수 있다.\n", "- Theano 에서의 이 계산 과정 최적화는 그래프에서 특정한 패턴을 찾고 대치하는 식으로 이루어지는데\n", "- 이렇게 대치된 것은 동일한 결과를 주지만 더 빠르고 안정적이라고 한다.\n", "- 최적화에는 또한 동일한 서브그래프를 찾고 동일한 결과를 다시 계산하지 않도록 하거나,\n", "- GPU 에 최적화된 연산으로 식을 바꾸는 것도 포함된다.\n", "- 예를 들어 xy / y 를 x 로 대치하는 식이다.\n", "- 여기도 한번 읽어보세요.\n", " - the optimization process and the specific optimizations that are applicable" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 2. 1026.]\n" ] } ], "source": [ "import theano\n", "a = theano.tensor.vector(\"a\") # declare symbolic variable\n", "b = a + a ** 10 # build symbolic expression\n", "f = theano.function([a], b) # compile function\n", "print f([0, 1, 2]) # prints `array([0,2,1026])`" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The output file is available at ./symbolic_graph_unopt.png\n" ] } ], "source": [ "theano.printing.pydotprint(b, outfile=\"./symbolic_graph_unopt.png\", var_with_name_simple=True)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The output file is available at ./symbolic_graph_opt.png\n" ] } ], "source": [ "theano.printing.pydotprint(f, outfile=\"./symbolic_graph_opt.png\", var_with_name_simple=True)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "<matplotlib.image.AxesImage at 0x7fde4d599210>" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": [ "iVBORw0KGgoAAAANSUhEUgAAATIAAAEACAYAAAAwdPpjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", "AAALEgAACxIB0t1+/AAAIABJREFUeJzsnXlgTVfXh5+TCQkS8zzPs5qnV1CqShUtoTUrSilVaqgW\n", "raqhZq1PW4pqKYqqWQhqjLGpMYkpiUgikfHm5o7r++Mmt4kkCIkryXne99QZ9ll7nXPv/WWdffbe\n", "SxERVFRUVLIzdrZ2QEVFReV5UYVMRUUl26MKmYqKSrZHFTIVFZVsjypkKioq2R5VyFRUVLI9WSJk\n", "iqK8rijKdUVR/BRFmZwVdaioqKgkoWR2PzJFUeyBG0BH4B5wFugnItcytSIVFRWVRLIiImsG+IvI\n", "HRExAJuAt7KgHhUVFRUga4SsDBCYbDsocZ+KiopKlpAVQqaOeVJRUXmhOGSBzXtAuWTb5bBEZVYU\n", "RVHFTkVF5ZkQEeXRfVkhZOeAaoqiVASCAQ+gXxrOZEHVmc/MmTOZOXOmrd14ItnNT+UN5eVucDgP\n", "NLa1E0/Ji/BVA9t6b6Nnz57PbCIzvqOKkkrDgCwQMhExKooyBtgP2AOr1TeWKqmYgOW99svKzMQl\n", "OzCTrPc1CMT75Q0+siIiQ0T2AnuzwraKiorKo6g9+59Au3btbO3CU6H6mcm0s7UDGaCdrR14OrLy\n", "s8/0DrFPVamiSHZpI1PJGhRP5eV+tFRJSRD84f0HvXr1sqkbiqKk2divRmQqKirZHlXIVFRUsj2q\n", "kKm8/JiAbsDVdI6HAUOTbS8Hpj9SZi/QFKiRaKtiot3nITDR3qtAa6A6EPUMdn4B/ADvx5QxAuOB\n", "ic9gPwkBej2hnlXPYd+GqEKm8vJjD3QAzOkcLw78nrh+DougzAZck5XJi2X6goLArsQlPXtPixH4\n", "B5gLjAR8AW0GbXwMvAdUwzJK+VEWJP7bDvgUuPUsjiaiAF15vIAPBhY+Rx02QhUylZeDCP7rDPQ+\n", "YADGYfnhJccj2TIoDTsTgTaJ63HJ9rd/pFxd4A/gENAb+BeLkHwDTMIiUN8DzYGTwG5gLbAF6IFl\n", "KgQvLAKZnAtYxAlgVKKPa4E3gSPAT1h6WPYHwhOv2Q7YDvwKtE2soyUWoZyJ5d7sBN5J55rBItze\n", "wI+Jfh8FumOJwkYknpucUcBxYCCwLfHfXkAe4ASgS6eel5Qs6UemopJhimCJbO5jETJHYAAw5pFy\n", "v/N4amERwadhCXAYi2g4Y4ngpmIRlo+wdNo9hkU8LieeUwFogUVI0+pk3jXRhhlYlmhjMBaRaAW4\n", "AQeA/2EZzFc28byywDUs0VJvLIJowhKNFsEiLvOxzCMTkUa9DYDXgStAJSAhsQ4F+BAY+0j5ZVge\n", "wX8HvsQi5DMSj5UGNFhELZugRmQqLw8TsUROzbFEBKuxDGOK5L+pCHolW95Nw8ZsYDMWEWiYxvHk\n", "vX4igZtYIq5HI5AwIB6LGHTAIhAmLBHTo4+kpkf2zcESATkmq9OQuO8aEAB4YhGK2EdsJD32mRMX\n", "JfHcZUAVLGIIlsdkfbI6TwB3E6/ZBUt0djTRxnwsbYIPk/lZmf/uX8Ij1xNPtgtxspm7KjmePVh+\n", "vE5YHo3qYJmmswmWH+G2NM6JwNJgHgKUBNyBS1jGICYnHsuj2nWgJpbHx4NYxineBz5JtLEFS7vb\n", "RSxi0AyLMPgm1rMfiyDVTrRrxiIyZiyhQZfE7STuYGmbm8F/UV5boEDitYHll1gDSzR3F0t05Qf8\n", "jaXdbRNwCotQA4Qm+lM3cbs9lpcPZ4GiWB57Gyf68x5QL9Hnmlgeh72wiPVuLI/gSQJtB0Qn+paN\n", "UDvEqtiEHNshdhWWiOwOFkEeiuURziWd8mFADFA1g/X4P8M5T8NxLFFd/kf2qx1iVVRyESOxRFRJ\n", "P7U1pC9iYIn8nkWQskLEwPKi5FERywaoQqaiopLtUYVMRUUl26MKmYqKSrZHFTIVFZVsjypkKioq\n", "2R61H5mKbZhGthzTl2tJAMcJjk8uZyPUfmQqKplInz592Lx5s63dyLGo/chUVFRyLKqQqaioZHtU\n", "IVNRUcn2qEKmoqKS7VGFTEVFJdujCpmKikq2RxUyFRWVbI8qZCoqKtkeVchUVFSyPaqQqaioZHtU\n", "IVNRyUQMBgPNmjWjf//+dOjQAXd3d8LDw5/bbsOGDfHw8KBJkyb06NGDr776KsM2bt++DcCsWbMe\n", "W27+/Pl8+OGHaLUZTdL5H1u2bKF58+bpHvfx8UGv16d7PMOIyAtfLNWqqOQ83nrrLQkJCRERkbFj\n", "x4qIiL+//3PbPX78uIiIlCtXTkREDhw4kKHzL168KFFRUeke9/f3l4CAALl27Zp89tlnYjQapXfv\n", "3s/usIhUrVr1sccHDBiQYZuJ2pFKU9SITEUlE3FycqJEiRIp9lWsWJGdO3cyYMAA4uLiqFq1KuvX\n", "r6dWrVoANG/enLVr1xIXF8f06dPZvXs3Y8eO5fjx43Tq1AkPDw9at26dwmajRo1wdXXFbDYzZcoU\n", "jhw5QoMGDZgwYQJ9+/bl/v377Nq1i9dffx2DwcC7776Lq6sr4eHhvPLKK3h5eeHh4cGECRP47rvv\n", "mDNnDl5eXlStWpWrV6/y0Ucf8dtvv6W6PqPRyPDhw+nVqxcajYYlS5Ywffp0/Pz86NevH3/99Rcr\n", "Vqywlr958yYLFy6kdu3ahIaGUq5cOWbMmIFWq2X06NHs3bs3c258WuqW1QtqRKaSQ0kexSRFZEuW\n", "LJHAwEAJDAwUrVYrxYsXFxGRihUriohIhQoVZNiwYfL999/LzZs3RUSkV69eotfrpVevXinsJ0Vk\n", "IiKzZs2SsLAwWbNmjYiING3aVEREChcuLJ9++qm1Tp1OJyVLlkxhIzw8XCZMmCAiItWrV5cZM2bI\n", "nTt3RKPRyJgxY+TVV1+VTZs2pbo+o9Eo+fPnl40bN0rXrl1Fq9WKiIjJZJKIiAhZsmSJ1Y+qVauK\n", "0WiUffv2SYUKFUREJH/+/FZb8fHx0rlz5wzdX541IlMUZY2iKKGKovybbF9hRVEOKoriqyjKAUVR\n", "3JIdm6ooip+iKNcVRXktc+RWRSV7YTQak/5oU6tWLTZv3ozZbGb//v3W/WazGRHh5MmTNGrUiEuX\n", "LjF37lzAEmDY29unsCkimM3/ZQL+4osvqF27Nu+8806Kcvny5aNUqVKcOnUKo9HI2bNnrXUm2ZD/\n", "ggoA7O3tiYuLIzg4GDc3N/bs2YOnpycRERHcv3/fWs5sNhMVFcXChQtp3Lgxc+fO5e7du5w5c4Z6\n", "9erRt29fTCYTBoMBs9nMqVOnCA4OJk+ePMTFxaXwMyYmhooVKz7nnU52cx63YMm1/Arwb7J984FP\n", "E9cnA3MT12tjSY3qiCW3sT9gl4bNDKmwikp2ISkiO3r0qPz++++SkJAgIiLbt2+XAwcOiMFgkFWr\n", "VklUVJSsWrVKAgICZMOGDXLs2DHR6/Vy7tw5OXXqlDx48EAuXLggmzZtEp1OJyIiV65ckVWrVklY\n", "WJi1vm7dulnX69atK1qtVjQajRiNRtm8ebMcPXpURERGjhwpIiIhISGyatUq8fb2lh07doivr6+s\n", "XbtWwsLCrO1wR44ckYMHD4rRaBRfX1+pWbOmtQ6j0Si7d+8WT09PMZvNsmPHDtm/f7+IiHh6esqt\n", "W7fk119/tfoaFRUlmzZtkqtXr8q+fftk1apVEh4eLiIiS5culZiYmAzdX9KJyJ72UbDiI0J2HSiR\n", "uF4SuJ64PhWYnKzcPqBFGvYy5LyKSnbheRvIn5agoCCpV69eClFr0qRJuuXNZrMMHTo0w/UEBgaK\n", "0Wh8Jh8fh0ajkT179mT4vPSE7Fmnui4hIqGJ66FAUutmaeB0snJBQJlnrENFRSUdypQpg4+PT4p9\n", "Z8+eTbe8oiisXr06w/WULVs2w+c8Dc7OznTp0iXT7D33W8sklXxckeetQ0VFReVxPGtEFqooSkkR\n", "CVEUpRQQlrj/HlAuWbmyiftSMXPmTOt6u3btaNeu3TO6oqKiklM5cuQIR44ceWK5p0o+oihKReAv\n", "EamXuD0fiBCReYqiTAHcRGSKoii1gd+AZlgeKT2BqvJIJWryEZWcipp8JGtJL/nIEyMyRVE2Au5A\n", "UUVRAoEvgLnAZkVRhgF3gD4AInJVUZTNwFXACIxWFUtFRSWrUdPBqahkAv369WPbtm3WbaPRiMlk\n", "sqFHORM1HZyKShby0UcfodfrrUvyYToqWY8akamoZBIFCxYkNjYWe3t7EhIScHB41ndpKumhRmQq\n", "KlmMu7s7AFWqVFFF7AWjCpmKSiaxceNGFEVh+vTptnYl16E+WqqoZCIODg5ER0fj4uJia1dyJOk9\n", "WuY6IRs6dCg/9//ZJrHooLWDWLt27YuvWOWpePjwIeHh4URERHD16lWuX79OQEAAAQEBaDQaXFxc\n", "KFy4MGXKlKFEiRIUK1aMIkWKULBgQasNnU5Hnjx5rNtarZbw8HDCwsJ48OABwcHBhIWFodFocHBw\n", "oFy5clSoUIFq1apRt25dihcvTpEiRShSpEiq2S9UnqMfWY6kHbZ5qF5rgzpV0uTatWvs2bOH06dP\n", "Ex8fT/HixWnQoAFVqlShYsWKvP3227i5uT3ZUCYRFxfHvXv3uHPnDseOHePSpUsEBwdjNptp3Lgx\n", "Xbp0oVmzZihKqt+wCuS+iRWHDBkimBBb/G/QoEE2u+7cTkhIiLz33nvi7OwsY8eOFT8/P9HpdGIy\n", "mWzt2mMxm82i1+slMjJSvvzySylatKi0a9dOLly4YGvXbAKZPPuFispLTWxsLF5eXvzyyy/Y2dkx\n", "btw4NmzYYGu3MoyiKDg6OuLm5sbnn3/O559/DoCfnx8jR47k7t279O7dm+7du1OsWDEbe2s7VCFT\n", "yVFERUUxZMgQqlatypAhQ9iyZYutXcoSqlWrxqpVqwAIDAzk559/Zt++fSxatIiGDRva2LsXj9r9\n", "QiVHkJCQQKNGjfjjjz/Yvn07CxYsoHbt2rZ264VQrlw5Pv30Uw4fPkx8fDyurq5ERUXZ2q0XihqR\n", "qWR71q5dy5kzZ/D29s71HVFbtWpFdHQ0S5YsITQ0lDlz5uSKFwRqRKaSrZk0aRI1atRg5cqVTyVi\n", "165do0OHDnz88cdMmjQJg8FgPbZr167Hnrtz504GDRpEhw4diI2NpW/fvo8tHxISwowZMwgICGDr\n", "1q306dOHevXqpetX8kHnyQkODk4xf9/TMH78eEaOHEnTpk0zdF62Ja03AFm9oL61VMkEfHx80kxZ\n", "9iScnJys61WrVhWNRvNU5xUtWlRELMk5RESKFCny2PLVq1cXEZGEhASZNm2aiIg4ODikWbZq1api\n", "NpvTtZXY9zLD6HQ6mT59+jOd+zKCmqD3OfEE/gDa29oRlSSmTJmCh4fHc9lYv349W7du5cKFC4we\n", "PZrLly9To0YN3n33XdavX8+IESM4fPgwAE2aNKFLly4pGtOXL19OkyZNuHv3LnZ2doSFhZEnTx5r\n", "Z9pt27axY8cObty4QXBwsPW8rVu38s0333Dw4EEADAaD9RFw4sSJbN26lYkTJz7XtYElYfCVK1ee\n", "287LTu5uUMgILYE9QM7/TmQbQkNDn1zoCdjZ2aEoCo0aNUKr1VK3bl0ABg0ahEajYfny5YwdO5YO\n", "HTqwd+9eLly4QPny5YmOjgZg7NixzJ8/nwoVKmBnZ0fx4sXJly8fNWvWxNHRkV69epGQkMC///5L\n", "6dKlAfDx8cHOzo433niDQoUKkZCQkCK/47x583jllVe4dOmSdd/KlSvZt28fr7/+eoav8dHM5zkR\n", "NSJ7WhYCLbBIv87GvqgA4OHhQVhY2JMLpkNCQgKjRo2ib9++qZLWJg6FAcBkMnHnzh3u3r1Lo0aN\n", "aNSoEUCqsoqiEB8fj8lkSmHr0XVXV1d+++03ypcvz/nz58mbNy///POP1a8bN26wYsUKhg0bZt33\n", "5Zdf0qlTp2e6zmvXrj3TedkJVcielveBA8AGLJN7q9icsWPHMmTIkAzNxBoWFsasWbPYtGkTe/fu\n", "xdvbG0dHR06fPk2rVq3w8fHh/fffJ1++fIgIPj4+tG3blvLly7Np0yY2bdrEwoUL0Wq1TJo0iejo\n", "aEaPHk1AQACbNm3Cy8uLdevWERYWxvTp0wkICMDb25uaNWsSGRnJ7NmzMZvNjBw5ks2bN9OtWzcA\n", "XF1drT6ePHmStm3bUqtWLe7evQvA/fv3n2nspYeHB5s2bcrwedmOtBrOsnpBbexXySTi4uKkdOnS\n", "1oze2ZWOHTs+Nuu2vb19huwZjUZp2rRpigS+OQHUxn6VnIiLiwv37t1j5MiRzJkzB50uez73Hzx4\n", "kHz58qV5zGg0YjQan8qO0Whk27ZtvPrqqxw7dizXDFtShUwlR7B27VoGDBjAyJEj+eCDD4iIiLC1\n", "SxkmvX5wT9M/zmg08tVXX9G7d28qVarEkSNHyJs3b2a7+NKiCplKjqFcuXKsXbuWFStWsG3bNpyd\n", "nXP8/G9Hjx6lfPnyTJkyhcmTJ7N9+3ZeeeUVW7v1wlG7X6jkOBwcHBg+fDjDhw9Hq9WyaNEiNm7c\n", "SP369RkwYAD169encOHCtnYzw2g0Gq5evcrGjRs5ceIEbdu2ZfLkyQQEBNjaNZujCplKjiZfvnxM\n", "mDCBCRMmYDAYuHLlCjt37uTEiRP4+flRpUoV2rVrR8eOHSlVqpSt3bUSHR2Nl5cXhw8f5sqVK5Qq\n", "VYqmTZvSrFkzZs+ejbOzs61dfKlQhUwl1+Do6EjDhg1p2LAhgwcPtu4XEQ4cOMAXX3zB7t27iYiI\n", "oFatWrRo0YIWLVrQsmVLa4dXRVFSLUCaA7PlkX5kjy6hoaGcOXOGU6dOcfr0aS5duoSjoyOdO3em\n", "Z8+e9OnThx49eryQe5PdyXVz9i9dtpSdf+5Mtf/WrVtUrFgRO7vMazY8ceIErVu3tm6/+eabjB8/\n", "PtPsq2Q9IsLt27cJCgoiODiY4OBgHjx4QEREBBqNhvj4eHQ6HWazGYPBwM2bN63p4Ozt7XF0dMTZ\n", "2Zn8+fPj5uZGsWLFKFWqFKVLl6ZMmTJUqlQpxRz/Ko9HTT7yGO7fv8/nn3/OTz/9lKl2fX19GTly\n", "JAcPHsz108vkFvr06cPmzZtt7UaORU0+kg4iwltvvcWZM2cy3Xb16tXx9PSkUqVK+Pj4vNBkFioq\n", "uYlc3/3izTffZN++fVk2+Zy9vT0BAQF89tln/Pbbb1lSh4pKbidXC9mKFSuYO3fuC3kV/91331Gy\n", "ZEm6du2a5XWpqOQ2cq2QBQYGcuvWLeu0LS+CDh06sH37dho2bJgte56rqLys5No2soEDB+Lp6fnC\n", "63VycuLcuXOMGzeO5s2bM3DgwBfug4pKTiNXRmRdu3Zl586dNktJ7+DgwHfffUfjxo1p164dL9Mb\n", "XBWV7EiuE7KTJ08yZMgQChQoYGtXqFOnDjt37uR///tfrpj8TkUlq3iikCmKUk5RFC9FUa4oinJZ\n", "UZSPEvcXVhTloKIovoqiHFAUxS3ZOVMVRfFTFOW6oiivZeUFZISEhATmzZvHO++8Y2tXrBQsWJDj\n", "x49z8uRJxowZg9lstrVLKirZjqdpIzMAH4vIJUVR8gPnFUU5CAwBDorIfEVRJgNTgCmKotQGPIDa\n", "QBnAU1GU6iJi81/ouHHjXtrZEIYNG0Z0dDQVKlTg1q1bODo62tolFZVswxMjMhEJEZFLietxwDUs\n", "AtUdWJdYbB2QNCjsLWCjiBhE5A7gDzTLZL8zTFBQEDVr1qRQoUK2diVdXF1dCQwMZNGiRalyLuYm\n", "DAYDvr6+tnZDJRuRoTYyRVEqAq8AZ4ASIpKUxiYUSErVUhoISnZaEBbhsynt2rXj448/trUbT8Xk\n", "yZP55JNP6NKlC0eOHMnw+UePHsXLy4uFCxdas/2A5R4cOnSIQoUK4enpSffu3Z/b1+XLl7N7926c\n", "nJw4fPjwMyXI8PLyIioqioMHD+Lt7c3MmTPp3bv3E8+LiooCYNasWUyfPp3+/fsDlsi7Vq1a1jRu\n", "WYWIsGDBglT7DQYDlStXTrEvKCiIefPmsW3bNoYPH86mTZtSpId7Vtzd3Tl06BClSpViz549zJ8/\n", "P8M2lixZAsDUqVPTPJ40wH3WrFls376d3bt3p1kuLCwMb29vXnvtNQ4fPkyZMmXYvXs33333XYb8\n", "ERFWrlwJwNmzZ4mMjHy6k55mAfID54EeiduRjxx/mPjvcuC9ZPt/Ano9UlZmzJhhXby8vDJrSu80\n", "mTt3rkRGRmZpHVnFjh07pG3btqLT6Z76nMqVK4uISHBwsHz44YfW/Xq9XkREihcvLiKWxLFGo1F0\n", "Op2YzWYxGAxiNput89+bTCYxGAxiMpmsx5Ns6HQ60el0Eh8fLyIiefLksdah1WrFZDKJyWSy2tfr\n", "9dZreLTO9evXi9lsFjc3N6v9+vXri4iIwWCwnpdU3mAwSFBQkPTq1ct6bSaTSbp06SInT560li1W\n", "rFiqe2MymazXl/Rv8jpMJpPo9XoxGo0ybNgwqz/JyyTdE4PBICIiX3/9tdV+7969U9yPJJLubUhI\n", "iHzzzTfWe5Vk12w2W+/To/c46Xy9Xm+9rwaDwfr5iYhUqlRJRES0Wq1otVrr+UnXk5CQICaTyVpv\n", "Uh3vvvuu9X7pdDoxGo1W22azWT766COJi4uTkydPyogRIyQgIEBWrFiR6r6azWbp16+f9fMVsSQd\n", "TvIpISFBdDqd9Z4nXUtSvUm+JeHn5ycHDx4ULy8vqV27tlUrSGfO/qcVMUdgPzA+2b7rQMnE9VLA\n", "9cT1KcCUZOX2Ac0fsZfqRmQVMTExMnLkyBdWX1ag1+tl6NChMmfOHOsX8HEULFhQRCxf/jp16qQ6\n", "niRkOp1ORo4cKT/++KO89dZb8t5778nu3btl9uzZcvLkSalQoYIEBwfLtWvXpG7duiIiMmbMGDl2\n", "7JjkzZtXfHx8rP4k/+GuWbNGvL295dy5c6LT6cTFxUUuXrwozZo1kzNnzsiUKVNk9uzZMnPmTOnV\n", "q5f1S5zcRv369WXmzJkSEhIiQUFBMnjwYJk0aZKYzWYpVaqUiIi888471vIffPCBdOjQwZqte+fO\n", "nXL79u0070/58uVFRGTRokXy999/y9KlS6VJkyZy9epV63flww8/tAplcj8GDhwobdq0kd27d8u5\n", "c+dERKREiRJW2+kJWRLJhez8+fOyePFiadWqldy8eVOcnZ3l/v37UqhQIRERyZ8/v1y9elUuXbok\n", "mzdvFr1eL9WqVRNvb2/p3r27bNmyxWo3SchExHr+kiVLxMfHR5o1ayYXLlyQvHnzyoYNG+S7776T\n", "GjVqSFhYmBQuXNj6XahUqZIcPXpUhg8fLpcvX5ZBgwbJkCFDrGI6adIk6dy5c5rXFRgYKF9++WWK\n", "fUlCdv/+fVEURY4fPy4ffPCBiIhs3rxZPD09xd3dXX788cdUWdu1Wq24u7uLiOX7dObMGRFJP/nI\n", "04iYAqwHFj+yfz4wWf4Tr7mJ67WBS4ATUAm4SeIsG2IDIRs+fPhjs9NkJ27evClt2rSRPXv2PLZc\n", "lSpVRETkwYMHMnjw4FTHk4Ts9u3bsnbtWhGx/EUdM2aMiIhs2bJFPD09JSgoSCpVqiQ//fST9YcS\n", "ExMjY8aMsf5Ykkj+wzWZTNKqVSvrDzapvvPnz8v7778vV69etdaZJCqP2qhfv77UrFlT4uPjxWw2\n", "S4UKFcTPz0/++usv648vuZAlbev1egkPD39slL9v3z7ZsGGDREZGytixY62Rlbe3t8yZM8daLknI\n", "HvXj7bffTmHP2dnZup4RIZs4caK1bpPJZL2n5cqVExGRjRs3SunSpWXEiBHi7+8vIiLu7u4SHh4u\n", "EyZMSGE3uZCdPn1a1q9fb41Ok/y1s7OTvn37Wv9wmM1mKV26dAobPj4+snjxYhGxZHZKErJz587J\n", "999/Lzt37pQZM2akui4/Pz/reUkkCZmISOKMN6m+Y9WrV0/zPplMJilbtqyIiOzatUv+/PNPEXm+\n", "LEqtgf5Ae0VRLiYurwNzgU6KovgCHRK3EZGrwGbgKrAXGJ3owAsnISGBokWLvhR9xjKDypUr8/ff\n", "f1OjRg0qVKjAlStppz2fPXs2QUFBbNy4kblz56Y4ptFo0Ol06HQ6KlSowLRp0/juu+9YsGABGo2G\n", "2NhYYmNjiYmJwd3dnYsXL7JhwwZKly5NUFAQS5cuZeHChRiNRvR6PWBJfCEiaDQawJK9u3v37imy\n", "YhuNRo4dO8aKFSto2bIlS5cuZceOHfTr14+YmBhrglytVktkZCQajYY5c+bw559/cv/+faZPn86E\n", "CROsnYdv3brFzZs3AVi6dCmRkZEULVoUEWHEiBGcOHGCIUOGcOnSJSZMmJDiHrz66qts3rwZNzc3\n", "hg8fTuvWrZk+fTrVq1dn0aJFzJkzh2+//ZbSpUvj7e3NN998k8IPjUZjbZ8DKFq0aAr7SfdDq9Wm\n", "+mxCQkKIjY1FRBg1ahTNmjXjiy++ID4+HoPBgMlkIi4ujoSEBGrVqsWiRYvo168f3bt3JyYmhpIl\n", "SxIbG4tWq7Xei8jISOv5AM2bN+fTTz+lZcuWgKXNzmw2065dOzw8POjcuTOTJ09Gr9dbszPFxsYS\n", "FxdHXFwcOp0OjUaDwWCgRIkSnDlzhmLFinH16lXKli1LgwYN2LBhAz/++KP1ugoWLEhISIh1++HD\n", "h2g0GoxGIwkJCYAl0XGNGjWYMGECnp6enDt3joIFCzJp0iQGDhxIREQEHh4eAGi1Wpo0aQLAnTt3\n", "qFKlSprfdStpqVtWL7ygiKxJkyYvpB5bsWHDBunSpYucOnXK1q6k4ocffrA+5iVFZOkxderULPPD\n", "ZDLJV199lWX2Q0NDZe/evdbtpIjM1kyfPt26/mgEmZxNmzZZ2zkzgsFgSBG9ioiMGzcuVVtXRjCb\n", "zbJgwQIRETl69Khcu3ZNREQ8PDysZXieNrLMXl6EkN24cUN+//33LK/H1phMJtm5c6fUqVPH+sHb\n", "mho1algfa8LDw8Xe3v6xj3qhoaHi6+ubJb4YjUaroGY2JpNJvL29U+yztZCFhoZKtWrVrA3uS5cu\n", "lZo1a0p7LWoeAAAgAElEQVRUVFS65xw+fDjD9aT18inpxc2zYjabrS+I/Pz8RERk3bp11sdvkfSF\n", "LMfOENujRw+2bt2aq2Zm3b9/Px9//DGLFy+mffv2ODk52dqlXIc6Q2zWkt4MsTlyrGV4eDitW7fO\n", "VSIG0LlzZ65evUrlypX54IMPGDZsGEFBQU8+UUUlm5MjhezTTz/NNp1fs4Jq1aqxZs0avv/+e7y8\n", "vChbtiwnTpywtVsqKllGjhMys9nMw4cPc100lhZ58uRhwIABBAUFUbFiRd58802GDBnClStXsEWT\n", "gopKVpHjhGzBggVs2rTJ1m68dJQpU4a//vqLH3/8kRs3btC9e3cmT57MgwcPbO1ajkCr1RIeHo5O\n", "pyM8PPzphtWoZBo5rrG/fv36+Pj4ZIntnIbBYODQoUP07t2bsWPHMnv27EzN65mbGD16tHV8IFiS\n", "ASf1s1PJPHJFXstr165x+/Zt3njjjUy3ndPR6/UsX76czZs389Zbb9G7d2+qVatma7eyDbdu3UrR\n", "afPw4cO0b9/ehh7lTHKFkI0bN445c+bg4uKS6bZzE1FRUXh5ebFy5UpcXV2ZMmUKjRs3trVbLz1u\n", "bm5ER0ejKAoGg8FmU6nnZHJ89wsRwdfXVxWxTMDNzY2ePXty4MABtmzZgrOzM61ataJixYrs27cv\n", "186T9iQ+/PBDAF577TVVxF4wOSYii4qKYsuWLQwfPjxT7aqkJDQ0lJ9++okTJ05Qq1YtunbtSsuW\n", "LcmXL5+tXbM5IoKDgwO7du2iS5cutnYnR5LjHy1/+ukn+vbtS/78+TPVrkr6mEwm/P392bZtG1u3\n", "bqVBgwZ8/PHH1KtXz9auPTcmk4mIiAiioqKsA6qTD6yOiYmxlk1ISLBGqfPmzeOTTz7BwcEBOzu7\n", "FE8I+fPnp0CBAhQoUMC6nj9/fvLnz0+RIkXImzfvC7/O7EaOF7KqVavi7++fqTZVMs6DBw+YP38+\n", "69ato23btnz22WdUq1YNFxcXFOW/7190dDQNGjTg/PnzFClSJFPqFhEMBgN6vR69Xo/BYCA8PJzL\n", "ly/j6+uL340b3PT1IzAggPj4eBzs7Mnj6EilkqUp5VaIkq6FKOVWmJKFClOsgBsl3ApRNH9BXF1c\n", "KJjPOYX/mUF0vIaYeA3R8fGERD8kNCqSkKiHhERFEhYTxf3oSO6EhhCj1WAymbF3dKB8uXJUrFqF\n", "6jVqUL1GDerUqUOZMmVwdHTEycnJumS2ry8LOV7IGjVqxIULFzLVpsrzc+HCBfbs2cOFCxfInz8/\n", "DRo0oE2bNuTPn5+6detiZ2fHkCFDWLZsGc7Ozk+0p9Pp8Pf359atW9y6eYsbvjeIuB+CITIa0eoo\n", "5VaYckWLU7FYCcoWLkqpQoUpVagILnlyXrSjNxoJjowgJCqCoIhw7oaHcfdBKPceRiBODji4FSB/\n", "0SLUrFmTKpUrU7lKFapUqULBggVt7fozk6OFzGw2s2bNGt5///1Ms6mS+YgIDx48ICAggKlTp6bI\n", "9F6gQAG++eYba4M5gLe3N5t//x1Pz0NoNXE0K1+F9jXr0aJabVydnSmQ1xmXvHmxV/u+pYuIEK/T\n", "EaONJ1anxefOLQ5ducSJO76Y7O1o1aIFvT086NixY7boQ5ijhezw4cO0bt2aPHnyZJpNlazF3d2d\n", "Y8eOYWdnZ83l6ejgQJH8BaleuizvNP8fvVu6U9Lt5c16lVPQ6nX8ceY4v586yo3gQNw7dWTA0CFU\n", "rVqV4sWLv1TD/XK0kE2fPp3Zs2dnmj2VrCU+Pp5Gr7xCQEAAHWo34I1GzelUrxHVStk82ZZKIsGR\n", "ERy79i9/nTtFeF47unV7kz59PShRosSTT85CcrSQde7cmf3792eaPZWsYd26dXw2dSoTO/dkkHsn\n", "XJ1dsMuhjdI5jRhtPIf+vcCodd8zYuRIZsycaZO+cjlayJo1a4a3t3em2bMlkZGRL3US4Yzy4MED\n", "Jn48Ae2tQFaPmkCBvE9u0H8aImJjKFIg+zZaZxYJBj06gwFX5xfXEVxvNDJ142rOPbjHj+vWUr16\n", "9RdWd47u2Z+8p/mIESM4ffp0ikSl169fZ/HixcyaNYspU6awbNmyTJn1oVu3bmzfvp2KFSuyfv16\n", "tm7dmmEb69evB6BLly588cUXNG/e/KnPffjwIUuWLKFly5b4+fnx22+/MWDAAGvSiechNjb2scd9\n", "fX0JDAx8bJk5X3/NwuFjWdmlL5s/nv7MIjbvz98ZuGIe+y6dZfneHdyPekjDSSOfydaj/OCZdrLZ\n", "JExmM72+nYkhMUnHi+SL39c9scyW08fY8PehDNn96/wp1h09+Kxu4eTgwMIBI/H6eBaHv/2ewQMG\n", "PrOtzCJHCFlSJ9izZ8/SsmVLWrRowfXr161TqVSqVInx48fTunVrmjVrxkcffYSbm9tz17tjxw56\n", "9uyJq6srAwcOpGvXrhk6393dnYEDLV+C4OBgvvzyS8qXL//E85IGI7u6ujJ+/HgmTZpEbGws//vf\n", "//jll18oWbJkxi8mGfHx8bRo0eKxZapXr860adPSPT5lyhR6upRi7nvv4+z0fC9hBrd7jdplK/B6\n", "w6a80agZpdwKky+TXuyM6Pj4z8xoNlE4fwEcbdDg/aXHoCeWaV61Zobtvtm4JT8+QcCfBjtF4YNO\n", "3fjhjX4vNCpL0xeb1p5JJD2m+vr6Wm9oqVKliIuLAywTDD7aQfDAgQNs3LiRBg0aEBQUhJOTEyEh\n", "IVYRqFu3Lp6envj4+LBr1y4SEhIoW7YsGzZsYNSoUaxcuTLV25x8+fJZO3fOmjXLKnQnTpygZs2a\n", "fPvtt/zxxx/UrFmT2NhYzpw5k+b1GAwG+vTpA0CnTp24d+8eXbp04dy5c6xevZqrV68CYG9vz4kT\n", "J5g2bRrh4eGUK1fusffJy8uLzZs3c/PmTTQaDR4eHmzfvp1ChQqxfv16rl27xu7duwkODiY2NjaV\n", "HxcvXrTWaTQa+fzzz/nhhx9S1WMymdD43KBWmSeL8tOy4W9Pen47kz6L/3upYzSZGLf2e7Z7n6Dz\n", "11OI0sRRcngffjnmyYzN67gRHMTsbb8CUHJEH0xmM//74mOMJhPbzhxHo0ug96KvmP3Hr3j7X+f/\n", "Du7iy62/sN37ONXHDUajSwABRwdHAA76nKfz11M56XuV8qPe5ZP1/0eMNp6Lt/2Zvulnvt+/kzth\n", "IUz59Sfi9Tr2XTpr9TXBoMfp3S6ERD2k5PA+XLsXwHbv4yQY9FQY/V6q600w6Kkxbggbj3sx8ZdV\n", "nPK9yvRNa3n1y0lsOHaI2X/8ypLd26zlh69ahCCUGdmXmyHB/Pr3IS4H3uHoVR/6L/uG7d4nKD68\n", "t7V8vE6XaZ+Nk4Mjv7//iU2bd3KEkOkSP5QqVaoQHBwMWNqaHjf+748//sDDw4N//vnH2jO6ZMmS\n", "VlHs378/48aNY9asWbzyyivkzZsXvV5PlSpVaNasGaNGjUrT7qpVq9izZw+DBg2iYcOGNGzYkNat\n", "WxMcHMzBgwfp1asX169fJ3/+/BQvXjxNG3FxcdZOi9988w1eXl7UqlXLKjDJad26NWfPnmXy5MnW\n", "69qwYUOadtu3b8/06dM5c+YMLi4u6PV6evbsSWRkJPPmzaNcuXJ07dqVqlWrpunHlStXcHBwsOZD\n", "KF++PBcvXkz3Hmcm/f/Xke0TZ7J7ytfWx7wHMdFUL1WWns1as++zubi55MfV2YUSroVoULEKFYuX\n", "4E6YJdfiBx274fnvBYq7unEl6A49mrW2dpId26UHY9d8x7mbvhy+fImezdrgu3QtznnyMuXXnxj/\n", "Rk8AGlWqRt1yFWhVvTYRsTFsO3OcfE5O1K9Qia2njzG6c3dG/LCEGqXLMnD5PO6Gh1n9z+vohKO9\n", "AyXdCiMiLN2znSZVapDX0QmHNBrN8zpaEscUKVCQVjXq0KhyNUKiIihftDj9277KpO59+DXZI+WU\n", "t/qy6/wZNLoEPlm/ivZ1GlK3XEXca9fH3s6ens1aE/bjFmt55zx50GXm4H8bv7PJEUJmMpkAaNGi\n", "BYsXL0av1xMREZEqcWpcXJx1srvevXvTqVMnpk2bhlartSaITUhIwGw2M3jwYN59913mzZvH0KFD\n", "MRqNNGzYEIPBkKJNLi4ujvj4eKsA9ujRg9GjR1OxYkWrbyJC165dcXd3p1+/fowbNw6j0WgVXZ1O\n", "h16vR0QwGo0UKFCAkydPEh8fz+rVq3n33XdZsWIFly5d4vbt25QsWRJ/f3/27dtHaGgoZ8+eZc2a\n", "NezcuZOAgABWrVrFvHnz6Nu3b6q2wIULF1rnGbt06RJz586lW7duTJs2jZYtW/LZZ5+h0+mIjo7G\n", "zc0thR99+/a13icAPz8/3N3dU30e9vb2ONerztWgu8/70QKWoTw6gx690YhGl8Cf505iNgvFCroy\n", "c8t6fvTcw9Rff0JnMGAWwWQ2oUlIwGQyYTJbfJ30Vh9+P3mErRO+4AfPPdgpCjHxGjS6BOb/+Tt/\n", "f7mYgz7naVWjDgNXzGPM6uWYTCbmvvc+C3b+JwAmsxkBOtRryIQ3e3PW35eA8Ad85TEYgApFi9Oz\n", "WRtuht5nZOJja5WxAxERy71DSDDoWTTwA/ot+RqjyUT1kqm7nWh0CcQmaDGYjJhMJowmE0aTpW69\n", "0Yh/yD1+GDmeOG0CeqOB95Z9g5ODA0aTic969aP+xBHM3vYr0fEajlz9h293bqHHgplW+yJCHkfH\n", "TPl8DEYjfX74lmbNmmWKvWchR7y17NixY4pe4rbEaDSyevVqRo4cyZ07d1i3bh0zZsxIs+yCBQsY\n", "N25clqVtu3LlCgULFnziI+ez8uabb/LXX3+le3z27Nlozl/hi3f6k+8528lshc5oYPSPy1g96hMi\n", "YmOYs/03Fg78IEM2fjnmyYC2HR9b5qfDe1NsD2zbCac02uWGfL+An0dPylD9jyIiuM/4hGNfLnou\n", "O2YRfjq0hxPx4az75ZfnsvW0pPfW8uXpsvsc1K9f39YuABAQEED37t05f/48AL/88gvHjx9Hr9en\n", "KVaTJk1i165ddOvWLUv8qVy5cpZNr3P69Gk2btz42DLTp08nIiKCiZ9NJ+Lf6/w48mMK5Muc7hcv\n", "Ckd7B+49DEer1/Hzkf14+99Aq9dlSJg9WrV7Ypn3Ozx52h+T2cT5W34s2vUHE7q9/dT1P8q8P39n\n", "7nvPPpxPbzQybdMarmmiWLZqJSOSzYxrK3JERObl5UWLFi3UObFectauXctn0z5jdLvOjHi1K24u\n", "+XFUJyB86RERIjVx7Ll4hk82rWHEiBHMnDVL7RCb2UJmMplYunQpEyZMyDSbKllHfHw827dv58D+\n", "/ZjuhdG0fBVa16hDkyq2fYWv8h/XggI44XuF0/7XiSqYl/btO9C7T+90X1C9KHK0kIEl3dm9e/cy\n", "1abKiyE8PJzAwEA2/vYbe7dup2iBArzdtA29W7alhGvOGeXwsqLV69h25jibz/zNjXsBuHd+jYFD\n", "h1C5cmWKFy/+Uk3bneOFbMCAAaxfvz7HTiiXG/H29ub333/n8OHDaDUaGperTMuK1WhVow5uLi7k\n", "z5uPAnmdyZdFL0tyAnqjkVhtPHEJWmITtJy/5ceJm9c5eccXO0dHmjVtRu8+ljf46jQ+GXcm04Us\n", "MDCQy5cvq3Ol53B0Oh1+fn7cunWLm/43ueF7g4choRijY0Gro7RbYcoVK0GFosUpW7goJd0KU7pQ\n", "YZxz6MSK9yMjuB/1kHsPI7gbEUZAWAj3HkZgcnLAvmB+8hctTI0aNaiSOKli1apV1YkVM9GZTBcy\n", "EaF79+6P7Q6gkrNJmupap9Oh1+nQJ051feXKFfx8ffG74cstPz/u3L6DXq/D3s6evE5OVCpRyjrV\n", "dUm3wpR0K0yxgq6UcC1EkQIFcXXOyqmu44nWagiNekhodFTqqa7DQoiNj8doMmHvYE/5cuWoUKUK\n", "1WpUp1r16immus6TJw9OTk5pjmTJKeTo7hdgucCWLVvy4MEDihUrZmt3VGyAoijWOespUACwDFWr\n", "V68ed+7cYenSpUTFxvDX/n1PnGEkKflIZGQkD5IlHHlS8pHkPCn5SMGCZXFxcaFQgQJUVZOPPBeP\n", "jcgURckLHAXyAE7AnyIyVVGUwsDvQAXgDtBHRKISz5kKDAVMwEciciANu1mSoFej0TBlyhSWL1+e\n", "6bZVsifr169n8+bNeHh40Lt3b1UssjnPFJGJSIKiKO1FJF5RFAfguKIobYDuwEERma8oymRgCjBF\n", "UZTagAdQGygDeCqKUl1EzJl+RWng4uJCWFjYkwuq5GiMRiODBg3i4cOHbNmyxTrDiErO5YmvKUQk\n", "PnHVCbAHIrEIWdJkSeuAHonrbwEbRcQgIncAf+CFDsDatGkTvXr1epFVqrwEGI1GDhw4QKdOnVi0\n", "aBHr169n7969ap7TXMIT28gURbEDLgBVgJUickVRlBIiEppYJBRImsi7NHA62elBWCKzF4aiKDRp\n", "0oSgoCDKli37IqtWsQGxsbF88sknmM1mPvroIw4efPYJA1WyL08UssTHwoaKorgC+xVFaf/IcVEU\n", "5XENXi/8teinn35Kjx492LVr14uuWuUFICLcuXOH1q1bM3Xq1DTnRFPJXTz1W0sRiVYUZTfQGAhV\n", "FKWkiIQoilIKSGqYugckn2qhbOK+VMycOdO63q5dO9q1a5cxzx+Dg4MDw4YN4/Tp00+c6VQl+/Dg\n", "wQPWrFnD6dOn+fzzz63TIKnkXI4cOcKRI0eeWO5Jby2LAkYRiVIUJR+wH5gFdAYiRGSeoihTADcR\n", "SWrs/w1Lu1gZwBOo+ugryqx6a/ko9evXx8fHJ8vrUclaTp48yaRJk+jfvz99+vSxzsKrkvt4pg6x\n", "iqLUw9KYb5e4/CIiCxK7X2wGypO6+8U0LN0vjMA4EUmVp+1FCRlYknrs3bv3yQVVXio0Gg1r165l\n", "6dKl7N+/n0qVKtnaJZWXgBzfsz89duzYgYuLC506dXoh9ak8H8eOHWPlypVUqFCByZMn56jUeCrP\n", "T64VMgAPDw/Wrl2rzlf2kiIifPXVV+zatYsffviBevXqvVQzLqi8PORqIRMRqlatir+/f44dg5bd\n", "EBFu3brFxx9/TKFChfjhhx/Ik0kp3mxJTkuw/LKRoxP0PglFUThz5gzDhg2ztSu5Hr1ez/Lly3nn\n", "nXe4cuUKf/75J+vWrbOKWHBwcKqpy7du3cqyZcuoXLky69at44033sD4nAlzb968ybRp05g5cyYj\n", "Roxg9uzZGR4VEhkZiZeXF3FxcSxevJg2bdo8Vd/FpKxfN27c4IsvvqBRo0YALF68mI4dOzJ+/PiM\n", "X9AjaDSaxx7fuXMn8fHxjy2TrRCRF75Yqn3xHD16VH7++Web1J3biYmJkRYtWsi4cePk4cOHjy1b\n", "pkyZFNsREREiItKkSRMREdFoNGI2m5/Ln/j4eDGZTOLt7S3r1q0TEZGEhIQM2ejfv7+IiJw/f17W\n", "rl0rIiLOzs6PPcdkMknTpk1FRCQuLk5EROrXry+xsbHW+1KoUKEM+fEoISEhMnTo0CeWGzhw4HPV\n", "YwsStSOVpuSKiCyJtm3bcvPmTa5du2ZrV3IFWq2W3bt306pVKw4dOsSpU6dYsmRJhh+9ChcunGLb\n", "2dmZIUOGcPToUSpVqkRYWBh169bl8uXL1KtXj/379/Pzzz8zceJEFi1axL1797hz5w6jR4+mRYsW\n", "bNq0CW9v71QTCYaFhTFx4kTAkp+0a9eu/N///R9ff/013377LW+99RZHjx6lVKlS3Lt3z5r/02Qy\n", "pWiyCA4O5vvvv8dgMFCzZk30ej0LFiygZ8+e2NnZ4e/vD1jGBi9dupSIiAgSEhIoVKgQIsLrr7+e\n", "5n0YO3Ys9+7d4+jRo+h0OpYsWcLWrVv57LPP6N69OyLC8OHDKVy4MGazOZUfFy5cwN3dnZ9++gmA\n", "EiVK5JxZldNSt6xesFFElsTQoUPl33//takPOZmAgAB57733ZPz48eLr65vh8x+NyJJIishERKpX\n", "r57iWMeOHUXEMsokLi5OOnfuLL1795YaNWqIVqsVk8kk5cuXl7fffjvFeckjMhGR1157Tfz9/cVs\n", "NsuYMWNExBKpNWjQQGrXrm0t5+vrK4sXL05lw9nZWebNm2e97m7duonZbJavv/5a3nrrLRFJHXFd\n", "v37dWtfXX3+d7n0JCwuTiRMnyrJly+TMmTPi7e0tIiJms1ny5cuXouzgwYNT+ZGQkCDDhg2zlpk7\n", "d65cuXIl3fpeRlAjsv9YvXo1U6dOTZW8VuX58PT0pEiRIgQFBbFu3ToWL15sTQb8tJhMJszmtCdL\n", "SUrEDFC2bFk+/fRThg0bliJpMFh6g+/cuRNPT0+mT5+Op6cnGo2GcePGYTAYUtjRarUp2ttWr17N\n", "0KFDrRGWyWQiMDCQhQsXotVq+frrr+nTpw8FChSwfn+MRqPVptlsZuzYsbz22muAJVDQ6/V4e3vz\n", "4MEDzp8/j6IoaLVaVqxYgU6nY+nSpSxevJhp06ZRuHBhvv32Wy5cuECdOnVSXH+xYsU4deoUY8eO\n", "pVGjRrRv355ffvmFn3/+merVq/Puu+8yfPhwRITbt2+n8iPJvyTCw8NzzouJtNQtqxdsHJGJWP6K\n", "tWnTRrRara1dydb4+/vLJ598It26dZPbt2/b2p3n5v79+xIbGysiYo2S0mPUqFHP3Vb3OHbs2JFl\n", "tk0mk7z//vtZZj+rIJ2ILNcKmYiITqeTunXrSnR0tK1dyXZs3LhR2rRpIxs3bpT4+Hhbu5Mp/Pbb\n", "bzJ8+HARETEajdKqVStZvnx5uuW1Wq1s3bo1y/zJyj+yGzduFL1en2X2s4r0hCxX9CN7HCJCvXr1\n", "OH36tDp31RPQ6/WMGjWKCxcucOTIEVxdXW3tkkouI1d3iH0SRqORtm3bsm/fvmydYSYrMJvNHDhw\n", "gO+//x53d3fGjx+v9rpXsRmqkD0Bo9FI+/bt2bBhAxUqVLC1OzYnKfrSaDRMmzYtVSdVFRVboArZ\n", "U9KzZ0/mz5+f4bdtOQGz2UxgYCBvvPEGgwcPZtKkSbZ2SUUlBaqQZYBRo0bRo0cPOnfubGtXXgix\n", "sbEsWbIEHx8fxowZg7u7u61dUlFJk1w91jKjrFy5ktjYWL766ivA8kJgzZo1NvYq87l27RoNGzbk\n", "119/ZezYsWzZskUVMZVsiRqRPYYLFy4wZ84cChcuzOrVqwkICKBMmReaS+W5MJvNqYbhREVFsXHj\n", "RpYtW8b69etp2rSpjbxTUck46qPlM7Jo0SImTpyIiFCwYEGio6Nt7dJTMXHiRDQaDStXrgTA29ub\n", "OXPmUL9+fcaNG6dOF62SLVGF7BkICgqiQoUKKYZ1jBkz5qXPZN6uXTuOHj2Kg4MDixcvZuHChWze\n", "vJnGjRunitBUVLITqpA9I+fPn2fChAlcuHDBOseTv78/lStXtrFnqTEYDDRp0iRFwpWkpLUqKjkB\n", "VciegwcPHhAaGsqWLVv4448/MBgMfPLJJ0RFRhITHY0mPp54rRadTodOp0Ov11snzwNLn6y0rtfO\n", "zg5HR0frdt68eXFycsLJyYm8efOSL29eXFxcKFjQFVc3V9zc3HBzc6NQoUK4urpStGhR6yNibGws\n", "VapUSTUQvlixYhmeMFBF5WVFFbJkJCQkcPLkSby9vfnnn3/4959/uH33LmazmUL5C1C9VFmqFi9F\n", "tZKlqVqyNNVLlaVisZI42NtZZ0WwUxRAQVEUFMs1JV0bCpDsP0+PCIKQOBwVEUt2Y0ncb/m/ZWyZ\n", "0WTiZmgwvvfv4R8SzJGr/+Afep8H0VHEaeNRFIUqlSrRsFEjmjVrRvPmzWnRogVOTk6ZdBdVVF48\n", "uULIYmJiCAoKIjAwkEuXLnHjylVuX7uBLiaWckWLU79iZWqVKketMuWpVqoMDrloqI3eaMT3fhDX\n", "ggK4dj+QS7f9uR/5kILFilC6ciVq1K5Fk8aNKVuuHGXLllXHnaq8lOQ4Ibt06RJ79uzh5MmTGDXx\n", "VC9QmHqly1O1ZBnKFylGCbfC5M+bN5M8zvnEauMJiYokICIM//vB/BN8l9vx0eQpUIA2bVrTtVs3\n", "atWqZWs3VXI52VbIRASDwcCBAweYM3s2/1y6xJjXe/BBp25UKl4yiz1VSeLavQD+78BfrD68jxat\n", "WjJt+nTatGmDo6OjmplK5YWRrYQsPj6e/fv2sW3rH0T63qJ745b0at6GogXUmSleFkKiHrLl1DH2\n", "XDpL6Qa16dmrF6917qy2walkKdlCyA4fPszUqVPpUKYKQ9t1pmrJ0upf+2yAyWzG934QKz13czEq\n", "lCVLl9K4cWNbu6WSA3mpx1oeO3aMyiVLU+5yIGcmfs03/YZSrVSZDIlYSNRDQqMjeRgXS2hUJEaz\n", "ifVHD2aKfxpdwhPLPIyLTbOLBVjePIZEPbRuG00mboeFPJdP8U/h0+HLF7n3MDxDdiPiYtK9jvSw\n", "t7OjVpnyLBs0ir/HzcTu0DnKlyiFt7d3huyoqDwrNhUyjUbDGx07wZHz3Fq+nmqlnn0co1avZ43X\n", "PgrnL4CDvT2ahAS++uPXTPHzWlDAY4/rjUbGrlmRrvAqQIOJI63b9nZ21P1k+HP55Hv/yWm8Ltz2\n", "J1ITlyG7moQElu3d/qxuAfBKpSoErPiF0G0HeLNbNwwGw3PZU1F5EjYVsm6dXmPL4HG0rVXvuW05\n", "58mDgoKI8M2Ojbg6u5CkK39f+5cBy+dy72E4F277M+i7+XSbOx2/+/do8/l4HsREcf1eIK/NnsKh\n", "fy/Sb+kc9EYjU379iePXL7PnojcRsTEs2f0Hw1ctQm80MGPzOr79awtg6duVL7FtyGQ2M3Pzekb+\n", "sASA037XWLZnu3Vo0BqvfSzfuyOF6F0OuMMb33xG/+VzOfDPOUSEJbv/YNza74lLSB15HbnyDwd9\n", "LrDq4G5Co6Po9NVk4hK0LNi5mX2XzvL2t7MAsFMsdV66c5M52zdy2u86ZhEW7drK4O8WICIc+vci\n", "7wwia6UAABcpSURBVC6bw/1IS8RY3NWNBzGZM570zcYt2NB7OL169swUeyoq6WEzITMYDPSs+Qou\n", "eTK/i0SsVmtdj4nX8O6yb/jn7i3Gr13JKxWrcNr3Gh92fot/A29z6IsFzNryCzXLlOOfuzdpV6cB\n", "XpcvkmDQM6BtR1rXqMO/AbdZ47WPm6H3mdTdg17fzmK79wnWHz1IeGw0IZEPqVKitOW6TEZef6Up\n", "gREPiNLE8dvfh/jojZ44OThw7Nq/1Cxdjo/e6Im93X9CVrd8RbwuX2LD2Cn0/HYmH/ywhGEd3mDp\n", "4NGUG9Uv1fW5167P1aC73HkQgpuzC+O79sLz3wv84LmbTvUb8/6rbzB29Qpreb/7Qbz/ahfaz/yE\n", "0T8uZViHLvw0agJBD8MZtvJbLgfcYfKvPwKWjr7BkRFkVsupq7MLrV1LEhUVlUkWVVRSYzMhs7Oz\n", "4+Jtv0y3qygKq0aM5/wti22d0UC3Rs3x+fYHFvQfbo2E7JMGTyuQYNADsH3iTKb+tpr5/Ufww8Fd\n", "1ClX0Vq+R9PWtK/TEPcZE4iO13BpwSrOfvMdCgr/BtymQrESABiMRjafOko+RyfiErTEJbZlKYrC\n", "lcA71mhHSbfXv0JsgpZbYfcfe43J/3W0d8Asgp1ih72dHW1q1iFvsreHey+dxdHeHpOY8QsJ5mZo\n", "MA529tx9EMrbLdri8+0PzOg9MNGmHTqDIcPtZI/jcnAAzs7OmWZPReVRbCZk9v/f3pmHV1Wde/j9\n", "MiAJCUMSpgABbHECGVUoaglWKkXE29pHtBUUUSlWsSpKqDwXIoKAA0VQWocWtFUBB5QQchk04EWQ\n", "OZAEKDR4JYEkkCA5EJKcnPPdP/ZOOAkJ85lgvc+zn7PHb//O3tm/rLX2OusLDSXxod/xwZqVFyXe\n", "hI/nMXf5EobPmc5PnhxO0bESihwlrNm5g9YxsVz11EOEhoaycN1q9hcVsvE/u/n4f7/m/dUr2Jn7\n", "A06Xi96drqVnx04M7zeAn11tJUedu3wJW/btYdayz/n3wVx+2+dW/vlkEgmj72dO2hfERjdmQNde\n", "fJW51fpeIaGkbFpH/y7deXnxRzRrFE2vpMfp0LwVA7vfyOy0xQyeNoF+13Wj0uWi8YN383VWBpVu\n", "Fxn/l8OJ8nLeGfU0b6R+zpTPPmTnzFMHdPzL0s/Y+J/drM7OYPayxXyx8Vs+WbcGgNXZGXy5aR0v\n", "/34kads2sODbdOKbxTJr6ee0bRbHPx4fyyNzX2f47OncfHVnrggP5+qnRtAwvKpq7KJDi1b2T7Au\n", "nCmffcjQZ5403TIMXsXv3S9WrVrF5OeSmPfIn+hgl2qCjYpKJ4/9bSbz/vj8OR/79sqlPHb7nafd\n", "J3HSszWWV/33KydLlB789MkH2fPGvAvqslJ0rIS/rUjhz7/+3XnHANh9IJdH58/h7Q//yTXXXHNB\n", "sQyGKi6oH5mIhAKbgFxVvUtEYoAFQHvge+BeVf3R3nc88DDgAsao6vI64tXoR+Z2u/n7e++xYO47\n", "jPz5AIb2TQy6/mMpm7/jl9160SAszC/nd7ndvPfVMm766TV07/CT846Tsnk9A7r24gqPUTnOlkqX\n", "i/fXrOT9b79i1LixDB061Ix/ZrioXKiRPQP0AqJVdYiIzAAOq+oMERkHNFPVJBG5DvgQuBFoA6wE\n", "rlJVd614dXaIVVUcDgfDhw1j87frefuxP/Gr7mYo5kBn4brVjH7nDQb/+m7mvPkm0dHR/pZkuEQ5\n", "byMTkbbAPGAK8IxdItsF9FPVAhFpBaSr6jV2acytqtPtY9OASaq6vlbMs/qtZWFhIW/OmcNXq1bR\n", "rVlLBl7fi+sTOtA+LjiroMGOqrKvMJ8d+/exbPtmshxFDBo0iD+MHk2zZs38Lc9wGXAhRrYImAo0\n", "BsbaRnZEVZvZ2wUoVtVmIjIbWK+q/7K3vQssU9VPa8U859EvVJX9+/ezPSODlJQUdm7YTOOQcO7o\n", "cSO3d+nBNW3anVM8w5nJ3P89K3ZsYWXGZk6ECp1738DgwYO5vmtX4uPj/S3PcBlSn5GdtkFHRAYD\n", "haq6VUQS69pHVVVETudKF+VtgoiQkJBAQkICg++6q8a2nJwcpi1cyIKPPiZ7106ubdeegV16ckf3\n", "G+h7VWdCQoQQCSFEJOja3ryBquJWxa1unJUu1u7OJG3bRpbt2EJO/gG6de3K0Pvu49577+XpZx7h\n", "aX8LNhjOwGlLZCIyFRgGVAINsUpln2G1gSWqar6ItAa+tquWSQCqOs0+Pg2YqKrf1YqrEydOrF5O\n", "TEwkMTHxYn4vAA4fPsz27dvZvn07O7OzOZifj7P0BI01hOiQMNo2i6VdbHNiopsQ2yiK2OjGxERF\n", "0ySyEZFe6KjrbY6VlVFSetzqenKshOJjxzhc8iO5RYfJ/bGIEnXhwEVEdBStW7Wic+fOdO3Wja5d\n", "u9K0aVN/yzcYTiE9PZ309PTq5eTk5Asb/UJE+nGyajkDKFLV6bZ5Na3V2H8TJxv7f1q7Hunvoa6r\n", "KC0txeFwcPz4cRwOB8eOHaOwsJDc3FwOHjjAkcNFFB86RFFBIUeLj1BeVsYV4eE0CAvnirAwYhs3\n", "JSY6muiGkUQ3bEijKxrSJCKSRg0jiGoYcc59sY6WHrc60ZaVUXKilOMV5ThOlFJ0rIQjjhLKnE7K\n", "nRWUO51EREbSNDaGmBYtiGkeR0zzOFrHx9O2bVuaN29OVFQUUVFRREdHExUVRUREhJeuosHgO86r\n", "alkHVe4zDVgoIiOxu18AqGq2iCwEsrFKcY8HhGPVQ2RkpM97nDdt2tT8XMdguMj4vUPs5YYxMoPh\n", "/Ano8cgMBoPhQjBGZjAYgh5jZAaDIegxRmYwGIIeY2QGgyHoMUZmMBiCHmNkBoMh6DFGZjAYgh5j\n", "ZAaDIegxRuYHysvLKS8vp6KigrI60r2dK6pKWVkZFRUV1bHP55cTLpcLgMrKSsrLy5k/f/45HV9R\n", "UUFFRUW1pvLyctxu9xmOunAqKyu9fg5DYGOMzA+EhIQwevRoANatW8exY+eWRLc2+fn5ZGVl4Xa7\n", "mTx5Mm63m3379p1TjE2bNpGTk4PT6WTs2LGkpqbyyCOPnPG4nJwcANLS0sjNzeWWW24BYMKECSxd\n", "uvSijBa7aNGi024vLy9nyZIlF3weQ/BijMwPhHuMh5+YmEh4eDjTpk3jvffeIzMzk5EjRzJq1CjW\n", "rVuH0+lk+vTp3H333bjdbt5++22SkpIoLS1l0aJFFBQU8P7779OrV6/qmBEREWRkZDBmzBhUlaFD\n", "h5KamkpWVhb33HMPhw8fpqysjEmTJpGamgpAUlISnTp1YubMmWRlZXH99SeTJi9atIhZs2axdu1a\n", "AF566SXGjx+Py+Wie/fuZGVlMXDgQK688kruvNNKpJKcnMxvfvObekfdGDx4MFlZWaSlpbF+/Xq+\n", "+eYbRo8eTUFBAU6nkxkzZpCcnExqairPP//8KTp27tzJuHHjePjhh4mMjCQ1NZXS0tKLe6MMwYOq\n", "+nyyTnt50qRJE1VVHTFihJaXl6uqamJiosbHx2t8fLwWFRVpy5YtVVU1IiJCc3JytH///pqZmakP\n", "PPCAHj9+XFVVExISNCkpSfPz86tjnzhxQl944QVVVXW5XNqzZ091u926du1anTVrlmZnZ2tBQYHe\n", "dNNNGhcXV31Ol8ulw4cPPyVGWFiYrl69WtPS0lRV9Z577tHCwkJNSUnRCRMmaEFBgUZGRlafPz8/\n", "X3/xi1/oW2+9paqqJSUl+sUXX9R5HbZs2aKLFi3SyZMna3FxscbExGh8fLwOGTJEO3furKqqe/bs\n", "UafTqX379q1TR48eParjLViwQJcvX37e98UQHNjecYqnmBKZn1CPNiyHw0FeXh45OTmEeWRhUlVa\n", "t25NcnIyffr0obi4mPz8fMDKPCUi9Q6IGBISwqhRo5g2bRp9+/ZFRAgLCyM2NhYRweVykZeXx+7d\n", "uxERQkND64xTWlpKVlYWAB06dEBVef311+ssabVs2ZKFCxeyd+9eHA4H33zzDUOGDKlztI9u3box\n", "Z84cBgwYwPHjx3n00UfJy8tj9uzZHDlyBFUlISGBiooKVLVOHZ7XqkGDBjidzjNddsMlijEyP3D0\n", "6FG2bt3K7NmzAZg3bx4dO3Zk7ty5bN68mRMnTpCRkUF5eTl79+7l008/pX///ixYsICpU6cyc+ZM\n", "1qxZw9q1a3n11Ver406aNIn09PTqNreRI0fWeJmQlpZGTk4OixcvJiUlhU6dOrFkyRJEpLqta+7c\n", "uWzYsIH9+/fjcrlo3LgxRUVFvPbaa/Tp04cWLVqwe/du4uLiGDNmDMOGDSMzM5MXX3yRoUOHkpaW\n", "xiuvvEKXLl147LHHaNOmDQ6Hg3btauZUCAkJYcSIEdxwww20bduWiooKevTogdPpJCUlhSuvvJLZ\n", "s2cTERHBDz/8wB133FFDx6pVqzh48CDFxcWA1dbYu3dvr943QwBTVzHN2xOmaul1jh49qosXL66u\n", "es6aNUt3795d7/5Lly7V4uJir+mZPn2612I7nU597rnnvBbfEDhgqpaXFydOnCAyMpKWLVuiqsTE\n", "xFRXzepi0KBBHDhwwGt6qhrsvcG2bduYMWOG1+IbAh8zQqyPMSPEGgznjxkh1mAwXLIYIzMYDEGP\n", "MTKDwRD0nGs6OMN58uyzz1JUVITb7eahhx6iR48ePPXUU/6WZTBcEpgSmY+Ii4tj/vz5OBwO5s+f\n", "T8OGwZfJ3GAIVMxbSx+Rl5dH+/btq0eYKCoqIiYmxs+qDIbgwry19DNt2rShffv2AHTt2tWYmMFw\n", "ETFG5kMefPBBAJ544gk/KzEYLi1M1dKHOBwOmjRp4pPBBg2GS5H6qpYBZWRTp07lhRe+87ke31IG\n", "XNoN/cuXP8GAAQP8LcNwCVKfkQVU9wuHwwF84W8ZhgukpORTf0swXGaYNjKDwRD0GCMzGAxBjzEy\n", "g8EQ9BgjMxgMQY8xMoPBEPSclZGJyPcisl1EtorIBntdjIisEJF/i8hyEWnqsf94EdkjIrtE5Jfe\n", "Eh94rPK3AIPhsuRsS2QKJKpqD1W9yV6XBKxQ1auwnuAkABG5DhgKXAcMBN4SkQso+WUCh4GVwGeA\n", "Azh4/uGqecv+7AY4gZRzPN5h6wFoC2wEJpzFcVW5F7Ps6WEgHXgc+BS4GD9dWn2G7eXA+7XW7QY8\n", "h7r+7WmOV+B+oPDcpRkMXuBc+pHV7oQ2BOhnz8/HehqTgLuBj1TVCXwvInuBm4D15yfxOmr6bTQQ\n", "AbwL5AOjgGlALNAR6wGbhmU0LwEfAEXAcKyHtRT4BJjrETMcaAM8YO8/BngE+DfwP8ALWGb1LtYl\n", "exT4CdaDvNCO4QAa2fMZwDYgCrjHPlepfdy1wNvAnfa+3YDuQB+sjrK1DaaKp+xjEmxdPYG/A8OA\n", "9sA8YC8wAsuEDtXScQvwsn3tXrSv3RGgmR1/r339YoEPgWIsw/I0t1CgFbAU6FyPToPB95ytkSmw\n", "UkRcwN9U9R2gpaoW2NsLgJb2fDw1TSsXyyXOk7oKc9OAKfb83cBfgeNYD+X9wOvAVuBVrAc7DmgC\n", "LMMys7/WEbMHkIbl1z+398/EMq8WQCKwxN73V1ilOIB7bS23AVOBY1jG9Vf7/CuABkBvLLOp5KSJ\n", "/YhlSuPtY1xA33quw6vAWOCXwCAs0wvHMqd7gTexbqcby6xq62iMVQqtsOM9aK+vuo5lQB6WQZZh\n", "/V8STr11h+3vvx3rz8Jg8D9na2Q3q+pBEWkOrBCRXZ4bVVVF5HR/1adsmzRpUvV8YmIiiYmJZykF\n", "4Aesh66Z/VlFA/vzW+BW4GYso4iztwkQeZq464GnsaqIDk6aqBurhFJix3AAV9QTw8XJKtetWCWj\n", "14GP7TieNAVeA/4L6xJ9Boyzj29Ra99wYAOWyYYBNwLrsMxwAFapsz2W0Wg9OjwL1RFY5l+bqlsV\n", "gmXW0z22NbbPcwCrpJcN/LOOGAbDxSE9PZ309PQz71hXjrjTTcBE4FlgF9DKXtca2GXPJwFJHvun\n", "Ab1rxagzZ11SUpKC1jM9bE9uhTyFLgpPKuQohCgUKYQp5Co8qPAHhRKFEQpzFdYojFf4vR1DFd5R\n", "iLKPUXv9/fb8Pnv+R4UNCjsUrlZItvcLt/f7UqGJwg8KVyl8oTBH4XWF6fa+7RQWKvRTmKrwmcLn\n", "CoMV3rV13qjQyZ6+U7hNIbvWNchSOGTPv6HQS+ErW2tnhdEKLvvaaC0dGxXE1qkKn9rxqmI/pjDO\n", "3v92hUG2jvrux1SFgjq3ffLJJxeSutBgqBfqyWt5NsYVCUTb842AtVj1mxnAOD1pXtPs+euw/l03\n", "wGp0+Q/2j9M9YtYp8vRG5u2pQuF7hRUeRjbpNPsfUstQvaXnS4VSL8V22WblHe3GyAzeoj4jO5uq\n", "ZUvgcxEBq07zL1VdLiKbgIUiMhL4HquhBlXNFpGFWPWOSuBxW0CAU4b1g/Ux9vImrHaySuqugccB\n", "+7yo5y4vxt4LLPdifIPBt5zRyFR1H9Zrtdrri4Hb6zlmKlbLdxARzUkTg9N3P6iio5e0eJur/C3A\n", "YLiomJ79BoMh6DFGZjAYgh5jZAaDIegxRnZG0v0t4CxJ97eAs+Ks+gQFAMGiE4JHqzd1BtRQ19ab\n", "0VH+llGLTcAN/hZxFgSOzpCQgfVuS09PP8fOz/4hWHRC8Gj1ps6AMrIpU6YweXJgZRhKTk5m4sSJ\n", "/pZxRgJJZ2hoqL8lGC4zAsrIRCTgHoKQkJCA01QXwaLTYPAGfksH5/OTGgyGSwINlLyWBoPBcDEx\n", "by0NBkPQY4zMYDAEPT43MhEZaI/lv0dExvn6/LW0/F1ECkRkh8e6gMtFICLtRORrEckSkUwRGROI\n", "WkWkoYh8JyLbRCRbRF4ORJ0e5w6181AsCXCdQZEzQ0SaisgnIrLTvv+9faazriExvDVhjZW8F+iA\n", "NVLgNuBaX2qopedWrKFhd3ismwE8b8+P49ThicJt/XuBEB/pbAV0t+ejsIa5vTZAtUban2FYI1Xe\n", "Eog67fM/A/wL+DJQ7719/n1ATK11AacVa8j7hz3ufxNf6fTJjfD4oj8D0jyWawzC6I/JvoieRrYL\n", "axjvKgOpGjByPPb4a/ZyGtDHT5oXY408ErBascax24g1uH/A6cRKwrAS6A8sCeR7bxtZbK11AaXV\n", "Nq2cOtb7RKevq5ZtgP0eyxc4nr9XOF0uglyP/fyiXUQ6YJUivyMAtYpIiIhss/V8rapZgagTmAk8\n", "R83xxwNRJ1CdM2OTiDxqrws0rR2BQyLyDxHZIiLviEgjX+n0tZEFVV8Ptf5VnE6zT7+PiERh5Yx7\n", "SlUdNYQEiFZVdatqd6wSz89FpH+t7X7XKSKDgUJV3cqp2cEsEQGg04ObVbUHVtaXP4rIrTWEBIbW\n", "MKzMNW+pak+shBBJNUR4UaevjSwPaOex3I6arhwIFIhIKwARac3JDB61tbelZuYTryIi4Vgm9oGq\n", "Lg5krQCqehQrb1yvANTZFxgiIvuAj4DbROSDANQJgKoetD8PAZ9jpVcMNK25QK6qbrSXP8Eytnxf\n", "6PS1kW0COolIBxFpgJXI90sfazgTX2LlSsP+XOyx/j4RaSAiHYFOWGmNvI5Yv6Z/D8hW1b8EqlYR\n", "iat6KyUiEVjpnbYGmk5V/bOqtlPVjsB9wFeqOizQdAKISKSIRNvzjbDyZewINK2qmg/sF5Gq4Ydv\n", "x8pAvcQnOn3VYOnRqPcrrLdue4Hxvj5/LS0fYeU2q8BquxuBlep7JVYeueVAU4/9/2zr3gXc4UOd\n", "t2C15WzDMoatWFncA0orcD2wxda5HXjOXh9QOmtp7sfJt5YBpxOr7WmbPWVWPTMBqrUb1gueDKzc\n", "hk18pdP8RMlgMAQ9pme/wWAIeoyRGQyGoMcYmcFgCHqMkRkMhqDHGJnBYAh6jJEZDIagxxiZwWAI\n", "eoyRGQyGoOf/ATh9k8HAHrT6AAAAAElFTkSuQmCC\n" ], "text/plain": [ "<matplotlib.figure.Figure at 0x7fde4d59f190>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "im = Image.open('symbolic_graph_unopt.png')\n", "plt.imshow(im)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "<matplotlib.image.AxesImage at 0x7fde4cb45d50>" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": [ "iVBORw0KGgoAAAANSUhEUgAAAXIAAACmCAYAAADH/ASEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", "AAALEgAACxIB0t1+/AAAIABJREFUeJztnXd4FNX6xz9nk2yym+wmoSUBQu8oIlWaFFFEUa/XctWr\n", "14YV9Co20KvixY5dL1bgh10UBStSIyAdQg8ltJCQQtpms9m+5/fHTJYkEGpCsng+zzNPzpyZOfPO\n", "zOY7Z95TXiGlRKFQKBShi6GuDVAoFArF6aGEXKFQKEIcJeQKhUIR4ighVygUihBHCblCoVCEOErI\n", "FQqFIsSpFSEXQlwqhNguhNglhHiiNs6hUCgUCg1R0/3IhRBhwA5gOJAFrAFulFKm1eiJFAqFQgHU\n", "To28D5AupdwnpfQCXwNX1cJ5FAqFQkHtCHkz4ECF9Uw9T6FQKBS1QG0IuRrzr1AoFGeQ8FooMwtI\n", "rrCejFYrDyKEUGKvUCgUp4CUUlTNq40a+VqgvRCilRDCCPwD+PEoxoTc8uyzz9a5DX8Fm0/H7osu\n", "ugij0Vgni8FgwGg0Mnz48Dq/f+r3Uf+WmrC5Omq8Ri6l9AkhxgK/A2HAVKl6rCjOEPYGdjxuT92c\n", "fCJ4JnqwX2+vm/Mr/rLUhmsFKeVvwG+1UbZCoVAoKqNGdp4EQ4YMqWsTTppQtBlC1O4hdW3AyROS\n", "95nQtLs2ba7xAUEndFIhZF2cV3H20/f6vqyeubrObVg5c2Wd2qA4OxFCIM9QY6dCoVAoziBKyBUK\n", "hSLEUUKuUCgUIY4ScoVCoQhxlJArFOV4gbnAobo2RKE4OZSQKxTlXKIv99S1IQrFyaGEXKEo53tg\n", "MpBd14YoFCeHEnKFArQ5O7sDDwFuwF+35igUJ4MScoUCQACPANOAW4CMujVHoTgZamWuFYUiJHmw\n", "rg1QKE4NVSNXKBSKEEcJuUKhUIQ4p+VaEULsA0rQmoa8Uso+QogGwDdAS2AfcL2Usvg07VQoFApF\n", "NZxujVwCQ6SU50sp++h544H5UsoOwEJ9XaFQKBS1RE24VqpOqXglMENPzwD+VgPnUCgUCkU11ESN\n", "fIEQYq0Q4i49L0FKmaunc4GE0zyHQqFQKI7B6XY/HCClzBZCNAbmCyG2V9wopZRCiKNGkJg4cWIw\n", "PWTIkJCM+KFQKBS1SUpKCikpKcfdr8YiBAkhngVKgbvQ/OY5QogkYLGUslOVfVWEIEWtcPfdd/Pd\n", "d9+d8vFerxeAiIiIUy7juuuu48MPPzzl4xWK6qguQtApC7kQwgyESSntQohoYB7wHDAcKJBSviKE\n", "GA/ESSnHVzlWCbmiXjJ//nyEEAwfPryuTVEojqA6IT8d10oC8IMQorycL6SU84QQa4GZQog70bsf\n", "nsY5FAqFQnEcTlnIpZR70aYZqppfiFYrVygUCsUZQI3sVCgUihBHCblCoVCEOErIFQqFIsRRQq5Q\n", "KBQhjhJyhUKhCHGUkCsUCkWIo4RcoVAoQhwl5AqFQhHiqJidCkUVCgoKyMrKCq4LIWjatGmNlV+x\n", "bACDwUBSUtIplVVSUoLVaqWoqIjc3FxiY2NPqSy/38+BAwdITk4mLCwMgP3799OyZctTsutksdvt\n", "WCyWM3KusxFVI1coqjBz5kz27t1Ly5YtycjI4IUXXqjR8qdNm8a6desYNWoU6enpXHjhhadUzpIl\n", "S/B4PAA8//zzpKWlceutt57QsYcOHQqmCwoKGDduHAkJCUERT01NPeGyToSysrJjbrfb7fz+++81\n", "dr6/GkrIFYoqjB49moEDByKEoF+/frz88ssArF69mg0bNgDg8XjYuXMnCxcuxOl0ArB27VqWLFkC\n", "gNvtZubMmdhstuC6zWZj9uzZPP300/Ts2ROr1crgwYOZO3cuixcvZuvWrRQWFrJixQoAFi5cyMqV\n", "K9m6dSsANpuNb7/9Nphev349jRo1YsmSJURGRtK4ceNK17FkyRLWrl0bXP/zzz9Zt24dpaWlDB8+\n", "nD///BOATp068cYbb2AymQCtdl5+TUdj5cqV/PrrrwD89NNPgFZ7X7hwYXCfP/74I3ivnn32WWbM\n", "mIGUkqKiImbPnh0sf8GCBaxZs4bExEQ2b95MUVHRiT4mRQWUkCsUVag6ha3FYuH111+nc+fOvPvu\n", "u2zdupWNGzcyadIkzjnnHC655BKKiooQQuD3+/H7/TzyyCNcc801XHvtteTl5fHhhx/y/vvvExMT\n", "c8T52rZtS2pqKvn5+URGRgLgcDi47rrraNOmDffffz87duxg69atXHzxxfTr14/ff/+dbt26AdCi\n", "RQs6duzI+eefHyxz1KhR9OrVC7vdzn/+8x/GjRtH586deeaZZzCbzZhMJnr16kV+fj7NmjVj//79\n", "tGnTBqfTybvvvkv//v2rvT8dO3bkzTffBGDevHnk5OQQGRlJcXExkydP5q677qJfv36sWLGCbdu2\n", "0ahRI3r06IHP5+Pjjz9m5MiRDBw4kIKCAi6//HIaNWpEIBCgf//+LF269LSf318RJeQKxQmQkpKC\n", "xWJh6tSpdO3aFavVSkJCAgkJCezfvx+r1coNN9zA6tWr+fXXX+nduzdhYWGMGTOGmTNnkpyczLBh\n", "w6qdHveee+7hiSee4IcffuCCCy4gOjqayMhImjRpwk033cQrr7xCYmIicXFxrFixgv379xMbGwtA\n", "ZGQkERERwZcAwKJFizCbzfTt25eZM2fyyiuv8MYbbxAVFYXBYCAsLIzIyEgyMjK46667aNOmDXfd\n", "dRejR48mNjaWr776itzcXDZt2nSErfHx8SQlJZGRkcGzzz7LpEmTSExM5JprruGxxx5j3rx5GI1G\n", "7rvvPrp06UJ4eDhRUVG89dZbDB06lMjISHr27ElmZiYGg4HWrVsTHh5OXFwcGzdurJ0HeJajhFyh\n", "qIaKc+abTCZWrFhBfn4+v/76K1LK4PZAIEBpaSm7du0iJyeHRo0aMWXKFEDzRffv359AIIDf7w+W\n", "FwgECAQCwfXo6GgGDBgQrNmX7wOay+amm27iv//9L2VlZSxbtoyuXbtSWFhYqayKNjVr1gyPx4PT\n", "6aRPnz6MHDmSRx55BLvdHrTD5XLRo0cP3nvvPUDzY7/44ovcfvvt3HjjjSQkJNCtWzfy8/ODLqJy\n", "HnnkEa699loaNmzI4MGDeeutt7DZbPz2229ER0eTnp7O3r172blzJxEREbhcLq666ireeecdAJxO\n", "5xGNstnZ2XTvfsSEqooT4LhCLoSYJoTIFUJsrpDXQAgxXwixUwgxTwgRV2HbBCHELiHEdiHEJbVl\n", "uEJRm6xfv577778/6POeOnUqn376KR988AGXXXYZ2dnZJCYmsmPHDq655hqklHz11VeMHDmSfv36\n", "8dxzzzFnzhzi4+Pp0aMHhYWFrF+/Pii0q1atonfv3uzduzd4zhtvvJHLL788uO7xePj000+57bbb\n", "GD58OL169WLs2LF06NCBSy65JOj/XrRoEUVFRaSnp9O7d2/KyspIS0vjtddeY+nSpXzyySc8/fTT\n", "vPXWW0yaNIk5c+YwduxYZs2aBWjukVmzZjFkyJBKvVT+/ve/A7Bu3brgi6mc8847jw8//BAhRNB9\n", "9NxzzzF48GBWrVrF5MmTWblyJR06dODKK6/kq6++on379owYMYJffvmFMWPGkJ+fz9133x3sxfPn\n", "n38ydOjQmn6UfwmOGyFICDEILYTbp1LKc/W8V4F8KeWrQogngHgp5XghRBfgS6A30AxYAHSQUgaq\n", "lKkiBCnqJXUVIejNN9+ktLSUp59+OpjXpEkT8vLyqj0mLS0Np9NJjx49atW2gwcPYjKZiI+Pr7Vz\n", "pKamEhUVRefOnWvtHGcDpxwhSEq5VAjRqkr2lcBgPT0DSAHGA1cBX0kpvcA+IUQ60AdYecqWKxR/\n", "AQYNGkSnTodD22ZlZfHtt99SWlp61AZSgM6dOwdjjNYmNdmHvjrOPfdcwsPVsJZT5VTvXIKUMldP\n", "56KFfQNoSmXRzkSrmSsUimPQq1evSuvNmjWjWbPj/+ucTpDo+oQS8dPjtO+elFIKIY7lJznqtokT\n", "JwbTQ4YMYciQIadrikKhUJxVpKSkkJKSctz9jusjB9BdKz9V8JFvB4ZIKXOEEEnAYillJyHEeAAp\n", "5cv6fnOBZ6WUq6qUp3zkinpJXfnIFYoToTof+al2P/wRKB+/eyswu0L+DUIIoxCiNdAeWH2K51Ao\n", "FArFCXBc14oQ4iu0hs1GQogDwDPAy8BMIcSdwD7gegAp5TYhxExgG+AD7ldVb0UoEAgE+Pbbb9my\n", "ZQtCCAoKCrj++uuDfboVivrMCblWavykyrWiqGeUlpZisViCwi2lxOFwYDab69gyheIw1blWlJAr\n", "FGjCHRERERz1GBYWhtfrVTVyRb2ipn3kCsVZhRCCRx55JLj+2GOPKRFXhAyqRq5Q6OzatSs4KGfn\n", "zp20bdu2ji1SKCqjXCsKxXEIBALBUYwHDx7EYFAfrIr6xSkP0Vco/ioYDAYefPBBhBBKxBUhhaqR\n", "n4WU39uK05oeNS0lssK60+mkuLiY0tJSSktLKS4upri4GLfbTWlpKXa7HZvNRonNhtvtwW4voaTY\n", "hr3ERondjsftoaTEjsvtxO32gF5vEHpCczlXTENcjNZTJC46hoiwcKIjowCIiYoiPCwMAcSZogFN\n", "aK0mrReJMTwcs76v1WQmzGAABHHR0bV3Y0+CYocDkPgCAexOLcxZmduFx+cDwOYsQ+rT1BY7HUjA\n", "5/dT6nIBUOp24vP7KXaUIqWkuNQOwOF/GxlMS4IJIiMjiYqMwmq1YIw0YrVYsVitWONisVisREYa\n", "iY2NxRobi8ViISYmhsjISOLi4oiLiyMmJoaYmBji4uIwmUwIIbQFoDwtyp9h9WlF7aBcK/UUv9+P\n", "x+PB4/Hgdrvxer243W4tz+3G7fHg9XopKiqisLCQoqIiigqLsJfYKC2xU2q3U2qzUVpSQmmJnZKS\n", "EpCSCEMYYWFhhBvCiAgPxxodjdUUjdVkIibKhCXKRKzJTKw5Gqs5GqvJjCXKjMVkwhwZidkYRYzJ\n", "FBROxdlNibOMUqcTh8eF0+2mxOnE7iqjxFlGSZkDm76UulzYXU7sLiclZQ5Kykrx+vz4An58fm3B\n", "ILBarcSUL7FWYiwWYqwWLLGxxMfHEx8fT4MGDYiPj9eCYhiNGCMjMRqNlQJlGI1GjEZjMJboXx0l\n", "5LWEy+XCbrdjt2siWlJSgr38r91OfkEBBQUFFBcV43E68ZY58TqceJ1leF1uDBIsJjNmYySx0Zqo\n", "Noi2EBcdTXy0hfjoGKKjTFh0UbVEmTFXiASjUIQaZW43dqf2krC7nDhcToocdoocpRQ7HBQ67NjK\n", "HJQ4HDg8LkqdTgICIqIiiTCZiYg2EWE2YTSZiIuPo1GjRjRs2BBLTAxWq/4Foi8WiwWr1VopelIo\n", "o4T8BPD7/WRmZpKVlUVubi452dnk5OSQk5tLcXExJTYbhoAkKgCRUhCFIDbKTEOLlUbWWL1mawrW\n", "bC1RZk18TUp8FYraoNJLwVlGiauMUpeTEqeTEkcp+fYSCuw2bC4XbhHAJcAlJIEwA7GxscTFxpKQ\n", "kEBiYiKJSUkkJibStGlTmjdvXi+/Av7yQp6WlsaGDRvYkbadHdu3s2vPbux2OxFCkBzbgOTYhrRu\n", "3IQOic1pk9iUKKORqPAI7W+EkZioKCLCVNuwQnG24PX7KHW5cHk9uDweXD4vLo+H3TlZ7MrOYk9+\n", "LgdshRwoLsSHxGKx0L5tWzp26kSnzp0577zzznggjLNayP1+P8uXL2f+/PksX7qU1WvX4vZ4aNkk\n", "kQHtOtOvY2f6d+hK+8RmWo8EfanYQKNQKBTVUd4hIKAvUkp25WSxfMdWlu9MY/mubew/lEuk0Uif\n", "Xr0YcOGFDB8+nP79+9dozT5khVxKSUlJCTabjfT0dH7/bS5rV6wgfcdOLCYzgzqfy5Au3RjS9TwS\n", "YmsvFJVCoVCcDDnFRaRs3UDKtk0s274Fu7OMdh070Kt/P0Zceint2rUjLi6u0hw/xyNkhNztdrNl\n", "yxbWr1vHqmXL2bd5Ky0aNGJgh64M7HwOHZKan2FrNbKLCvH4fZXymsY3ZFd2Jl2at6zmqJNj3Z6d\n", "9GzT4YT2TcvMoHPzFgB4fD725GVjNkYSH2PBEmWqEXuOhtPjwRfwn9A5dh7MpENT7XlJwO31sCs7\n", "i3NbtAYgIz+PRtZYzMaTaz8oc7vYeyiXxpZYIsLDiY8+eii0mqDAXoLFZMIYfuxIPAEp2Xkwk07N\n", "kgGtAnIqX3tlbvdJt6fk2200ssSe9HkMBoHT4yY+2oLb6yW7qIBWTRIpcZYFeyt5fNp8MxFh4RQ7\n", "Somr5l7vy8uhVZPEk7KhKg63izCDAbfXS6w5GpfHQ66tiJaNEyhxOrDq3VBtZQ5izYe7me44eICO\n", "TZMrleXz+0FAZkE+rRonUN/YcTCTZds3s2zHVg4UFdCqW1f6DhxAz5496dq1a7WNs6c814oQYpoQ\n", "IlcIsblC3kQhRKYQIlVfRlbYNkEIsUsIsV0IccmJXFRhYSGTJ0/m76OuYOzlV7P901mM8MfwyRU3\n", "s+DJl5h27yPcMezSOhNx0Po9d3/sHuzOMoodpdz38dtkFR7in++8XGPn+C11zQntd9WrzxAdpfWh\n", "3rBvN7f971VkQHKwqIAej99XY/YcjXx7MZkFhwAoLnNUu99jn32Ew631ifYHAsxcnoIxLJxFW1KR\n", "wMSZn+IPBPjP19MpsJec8Pk//WM+E76ciinCSMrWjdz38duncznHJT0nS+8TDnans9r9+kwYQ7we\n", "W9PpcTNv07qTOk9ASr5ctojIkwzdNmvl0pN+ce/MziSz8BBFpXb25uXg8/u54/3XAPjvd59jiTIx\n", "adYXAESEhTN98Tw8Pi9WczRv/jLriPJ+TV3NxG8/4/OlC07Kjopsy9xPTlEhBfYSMvJzcXrc3D/1\n", "HXx+Py/P/pqYKDMvfP8loP3uFm/dGDy21OXiiS8+Ca7bnWWsTt+By+Nh/Z5dp2xTbdKxaXPuHDaS\n", "6fc9yoInX+KTUTdziddM2oxvGXPZ1fx91BW8NnkyRUVFJ1TeiQxfmw5cWiVPAm9IKc/Xl98AhBBd\n", "gH8AXfRjpgghqj2H1+tlxCWX8OqdY7jS2Ijv/vUgH98zjn8OHEaLRk1O6ALOFPExFozh4ZyT3Irz\n", "WrZh+v2P0qxBo+DAFrvLydfLU4L778vLochRyrLtWwCYu2ENHp+P+ZvW8eeOrfy0bgU/rl2B2+vl\n", "x7UrALi6zwBAq3HMWbMcmy6UczeuYXfOwWDZG/fvoUWjJvj8fi5/6Sneu/MBOjdvwQXtO/PDYxMB\n", "8Pp8fLdyCTnF2g8hp7gQW5mDP7ZtCtrj1gP3rtyZxs6DmfyyXgvk5PP7mb3mT/Yf0sKyujwe5qxZ\n", "Tp6tmOYNGhNmMODx+Rj23KOk6P9QKds2siVjX9DGDft2B79Ufl63kqFdu2MwGLisR19A8sWyRbRu\n", "ksgN/Yfy2ZIjBeC2KZOPyHO4XYyZ+i5v3z6GNglJXN9/MLcN0eoKS9M2syRNuzZfwM8hWzFzN6wh\n", "ICV78rLZnnUAgMJSOz6/ny+WLsQf0GY6XJO+g/kVhDezIJ8f165ASkm3lm0JyADPffcZH87/GV/A\n", "T3ZxIT+tqxxPfHfOwaBrb+zU9xjatXuw7EVbUgFw+7x8tmQBB4sK9Oe4mx0HD+DyePhz+xaGdj2P\n", "MIOBfXk5wd9EIBDgx7UrSMvKoMztosRZhsPlYu2eneTaioiMiCAywljpGZXfj1/Wr2bLgX14fD5y\n", "bUX8sPpP/IEAE76YSoek5iTGNyQ6MooXf/iSSf+4jVZNEknZupFSl5P7L7mC1L3pCCH456BhfLF0\n", "EQYhGHFer+C9LCciLBxfwEenpi2OeF5z1iwn327TfiNbN/L7hrWkZWXg9nnJt5cwa9VSvH4/L3z/\n", "JW0Tm9K0QSOM4RG88P2XPHPNzbRNbMpP61bi8ri5Y9ilbNq/h5aNmrBw83pKXdqLtXOzFmzavyd4\n", "3gH/eYj+HbsQE2UKfiGFAi0bJ/DPgRfxyb3j+O5fDzLK2IiXb7+PS0eMwOfzHfPY4wq5lHIpcLTX\n", "wtG+G68CvpJSeqWU+4B0oE91Zbdv0ZIf/vVvXr7pTjo2TcZQzxsebWUOrntjEn2fHMuhElsw3+vz\n", "sXDzev7Wuz9N7/kHASm575O3ySkuJGXrRr5ftYzmDRvz+OcfYxCCjPw8RnbvwwfzfyYyIgKnx40/\n", "EKDff/4NwK3vvcqonheQXVTIyBef5KJzzue+T94hqzC/kj2H7DbCw8JoEGMJ5p2T3IqAlNw2ZTJ/\n", "6zOAxz//iD252Tz62Yek5xxkW+Z+Ppz/C91atuG6Nyfh8/u54e0XiIuJYd7Gdfy6fhXXvzWJUT0u\n", "4L3f57Atcz/D/vsYV/Tqx66cLLZm7ufndauICA/HEmVmQMeuPDj9f/Rr34X/fvcZu7KzAO2fOEIP\n", "qPv891/QJDaOQCDA3159FikJug8S4+KDAlyR8pdIRZakbaZH63aV8i7t3ps73n+NHm3akxAbzy3v\n", "vszCzam8O3c2Azp2peX9N9Eg2sI9H7+JBJ76ahob9++mU7MWXD15Is/MnEFifAN6t+vE0OceBeDD\n", "+T9z2fl9CEjJ96uWsjVzP83iG9G7XUc8Xi8GYaCRxcoD094L2mHWv5BAe2kaw8O1F6UAc2QUUkoe\n", "mj6FWy4czvVvPo8/EOD6N54PTgXw4PT/kRTfEIBZq5Zy2fnav821r/+Xy3v0YdaqpRwoyOe2/73K\n", "nzu2Yo0y8/68n4Ivy4rP6PHPP6Zbyzac26IVr86Zybcr/uDNX74nKa4BdmcZvdt1BGDXwUy+XLaY\n", "H9eupE1CEgBx0dEUltppaLEy+sM3AIiOjOL1n78FoHnDxjz9zf9VegZN4xtiEIZgxaCcUS//hyt6\n", "XcDevFwemPYefdt3plWTBN75dTafL1nIe3PnkBTXkEMlxfRuq9m0PSuDWauW8ev61UFXjcVkosRZ\n", "RhNrHGP1ez6k63lsObAPgMiIiKCoA3Rt0QqAQyW2SjX1UMIgBJ2aJvPKP0fz/S0P0C752O7b05lQ\n", "4gEhxEYhxFQhRJye1xTIrLBPJlBtKPAWDRuf9KdkXRJrjubbcU+z6sX3aBIbFwwrvXBLKnHmGKIi\n", "jBz88BsMQmAxmWnWoBE92rSne+u2JMU1INdWyEXn9mDizE/ZeyiHPFsRf+7YyqgefQkzGII+1Wsu\n", "GESnh26naYOGFJbaiQgLZ95/XqZZg0Y8MPU91r0yBYBoYxTFZQ6qtjekZWVgNZkJN4Tx4o13Mun7\n", "L4iPttCqcQLnJLeiS3ILGlms5JdoLwJTRCRNrHG8evNdzPhjAT6fn/CwMB4ZdR2PfvYRz153Cy3v\n", "/ycdmyYTp/smBRBmMBARHs7WA/uJjIhg5rinaZ/UjLFT3+Wbh/+DQQgkkKt/FRgMBsL1FnyfPu+3\n", "0+OmR+v2Qduf+no6nR++k9Xp2+n88J0Mevrh4LaWjZqwNy/niOeyYNN6zMZI2iQksXjrRuKjY2ib\n", "2BSLyYyUkrjoGBpb43C4nMTHWOjZpgM927QnszCfn9eupEGMhViTmY37dgOaX7jvUw9o0wPoPuHw\n", "MG2E7PPff0FCbBz9OnTh3TvGAvDgtP+R/s4MQPM9RxmNgDaNwJ3vv8bP61by+8a1XNq9FwDx0TGE\n", "GQzEmqNpn9iMKKOx0otrZ3YWA595iDK3i7ySYsIMYcSZNTvaJjTlkvN60qFpc/bl5QT/fyo+o+9X\n", "LQuO0AVoEhtP/w5duKBDZ3JtRTTVXxhWPWhG5+YtcOkiXO6fBii024M2CQQ+v59wg4Fs/YuinK7J\n", "LZkx5nEGdOpaKf/hy6+hzZhbaJuQxJw1y4kyGoPTKiTGxXNh53Pp37ELOcWFJMZpXzPl5+7YLDn4\n", "YvD4fERHmhBAvl6Bio+2sG635jYJMxj4/MHx/Hv6FIodpbTUv+ZN+nMIdSIjjCQ3aHTMfU5VyN8H\n", "WgPdgWzg9WPsW21r6vQ539P/1SeZuWJJ0I1QX6k4VwmAzeHAF/AjpaR9YjPe+Pk7SpxlLNuuNSWU\n", "d1EKBAJ4fT59ThPt2GsvGMTbv3zPB3c/xPTFvxMd9HFqx5iMkfzx3Bs89+2nZBcVsP9QLnvzcthy\n", "YB/v3jmWoRO1mqPVbOaOISP4cMHPQbu2ZOylZaMmLNu+hYAMkJGfy439h+IPBIL2+Pz+Sg9FonWp\n", "mrVqKU9cdT07s7PwBwJkFxVw57BLycjPY8PkD3jo/6YQkBKv3uhrMAjKPG68fh+b9u/hYGEBf27f\n", "ynt3PsB9H7+NPxBAAN1bHZ4OVruHkmhdZNbt2cXVfQYGt79ww+2kvTmVPu06kfbmVJZOejO4rUvz\n", "ljS0WNmWuR8Af8DPhr3pdGiajEOfx6Rr85bBe0+Fe17x2TncLnJtRfRq04HzWrUhs+AQEkhu1AR/\n", "IMANA4by+QPjmfHHfAJS4vcHCA8z4HS7Gdr1fJ78ahq2MgfzNmrumHfuGEO3R+8GtC8Nvz6HitPj\n", "ZtNrHwMCt9fLlHk/E5Ay+FuXHLZzQKdzAO0F9+gV1/HRPeOYtWoZ+/NzcbhdFDnswWsufwn2bteR\n", "MrcboNIzahBjIauggCJHqX7tgaAbqWl8Q/bkZQd/oz6/j9duuZvPly5ASkkDizX48mre8LB4lL+E\n", "PT4frY/SqHm0ht0ih501L0/h8c8/ItYczSFbMcW6TQEp8ek2NW/YmH36i6z89/XSTXfwzYoUpJQ0\n", "a9AQi8lEAEmyLtIHCvLo3U7rGODx+bhjyuu8ffv9xEXHsOPg4bpk+bPYk6tdc5nbTW5xEdnFhUd8\n", "QdQ3issczFzxB/1ffZJPf559zH1PaYSLlDKvPC2E+AT4SV/NAio6pZrreUcwceJEAEZeMQpn69aM\n", "X/EbB7duZ0jbzgzu0o3urdrVK1fLuj27uH3ICN6f9yNev5+swnzuHDaSK3v1Iy46hpsGDuOJzz/m\n", "2etuweX10L1lW7IK88mzFVNQWkL3lm05v3U7XF4Pz11/KwaDAZ/fz0OXXw1Anq2YMZdcye7cbPYd\n", "yiEhLp6ti2V1AAAcrElEQVTxf7uBCVffxH++ns6gTufwz0EXAQQnYQJ487b7+HzJAv773ecIIRg7\n", "4kpiokx88eAEvlmegsfn45+DLmL93l3sO5RLjq0IW5mD5IaNGXl+72CD5Hcrl+D3++nRpj2/THie\n", "z5cuIDwsnH8OHMbLs78mLTODF264nU379xCuuwPGX3UDM5en8NMTk3jqq+l0apbM2EuvArRardfn\n", "I8xo5IUb72Dtnp20aZLE3/sOZHvWARY+M5mvl6fQolETuh7ls/GGAUOP+hxSX/2AF3/4ijlrlmOO\n", "jOSe4aP4/akXmfzjt3Rv1ZYfHnuORVtT8QcC7M3L4bahI8guKqBn6/Zk5GuNtPM3raPY4WDKaG2m\n", "w7d//Z7zWh7iz0lvIYRgxc5tnNuiNbcNuYSpi+ZiMZkY2b0Pr/44kxduuJ21u3cwadYXPHvtzUG7\n", "KrraBnU+l5IyB8aICL5dsYSLzu3O0K7dOVhUwKTvPiMyPJwCewmXdu/NlgP7OLdFa6aMfpBl27fQ\n", "v2MXFm3dQOdmLbjlwuG0S2rGs9/MwGDQ/hcaWmL5NXU1V/bqx90XXc4ni36lfVIzCuwlwWcUGWHk\n", "v999xsXdegJau0CRoxR/wE+MyRz0o2/J2EesOYaE2HgaW2KZt2kdb956LwDbMjP44sEJgPZFNfG6\n", "fwGwaf9eXr35rhP5lyGnuIgdBw8w8bp/EWYI45mZMxh2znmA5qZ0uJwEpKSxJZaDRZrbcPP+vZiN\n", "kbRqnMiGfbuZt2kdb982BoCtB/bzyb3jAM3N9uKNdwDawJ6KX/YWk1Yx2pWdRb/2nSl1Ofltw2ru\n", "v+RK8kqKSMvMwOv30bd953rVZTkQCJC6bzcp2zaxZM92mp7TiQsuuICR1lHMmDHjmMeeUPdDIUQr\n", "4Ccp5bn6epKUMltPPwz0llLepDd2fonmF28GLADaVe1reKzuh+np6WzcuJHFCxaSnrqRGMIY2Okc\n", "hnQ9r1LN7q/MS7O/omNSMn/vO/D4O58A7R+8jZ1vT6/RwVEz/piP3+/njmFaO/m3K5YwonuvOp+E\n", "69HPPuSFG+6ocZfeg9P/xxU9LwiK53tz5wRfalXp/59/s/z5I3vbzF79J4O7diM+2nLEthd/+JJr\n", "+g46opvd+r3pWE1m2iU2PeKYXFsRj376IZ89ML5SfpHDzpJtm7mqd/+j2lfsKGX5zm1cdn4f3F4v\n", "36xI4ZZBwykstbMpY0+wIfdU2JuXwytzvuGDu/5dKT/fbmPVru1c3qPvUY8rcpSyatd2Lu3ei2Xb\n", "t5DcsDEt9W6FHy34BUuUmRsHai9/CUyZO4cx1dz/+kLq3nRStm3kz+1bKRUB2p3fjWEXDafbed1o\n", "167dUY855X7kQoivgMFAIyAXeBYYguZWkcBe4B4pZa6+/5PAHYAP+LeU8vejlHnCA4JKS0spKioi\n", "IyODn3/6iWULF5O5fz+NY+MY1OlcBnfpxqDO59ZqX+L6SCAQqJE5s+0uJ2mZ+2mb0JSGFmsNWHaY\n", "gJSVvqqqrtcFqXvTsZhMtEustunmlPH5/cE2gOrweL2k7kundZMkrZ2lCkd7rv5AgK0H9mE2RtIu\n", "6Ui7vX7fUaeP2JObTYnTwXkt2x7xkg7IAIZqOpT5AwF9WuDK/eFP5PqOx+6cg5S6nHRr2eZIm47x\n", "m65oU8W0di1H/q4kR++NcaYpKrWzJG0zf2zbxNLtW8gvKaZ5y5YMvGgoV1x5JcnJycTHxxMTc2L6\n", "FTIDgk4Wr9fLokWL+PXXX5n/+zx279mDMAh6te3AgPadGdjxHAZ0OgeLyYxB6PMmo4bmKxSKk0dK\n", "GWzbCEiJ3elkWdoWlu3YwvL0NNam70QiadumDZeMGMFll13G0KFDiaihL8CzVsiPR0lJCZs2bSI1\n", "NZUNqans2rWLYpsNl8tFkjWe5Nh4msfG07pxIh2TmtPAYsWsT5QVZYzEZDRiMhqrrb0oFIrQJRAI\n", "4PR6cHo8uDxuPe2mwG5n58ED7DmUS1ZJEQdsRWTbCokymYiLjaV9+/Z0P/98evToQbdu3bBYjnSH\n", "1QZ/WSE/UWw2GwcOHCAjI4ODBw+SfTCb7JzsYFSccAlRwoDR48dsCMNijCIprgExJi1Ig9UcjSnC\n", "iNVsJtakzStuMZmIijg7ukApFPUZl9eD3akFu7A5HZSUleH0eigpc2iBMJxOcooLsXtcOAJ+vMYw\n", "nDKAX6BFTLJYSExKIikpiWbNmpGcnEyLFi2wWmvW3Xi6KCGvYRwOB0VFRVroMz2IRInNhq2khOLi\n", "Ygr0gBJOhwNvmQuvw6EFlHC5CEcQHaVH6jGZaBBjxWo20yBa688cH6MFlogrDywRGXV8gxSKEEXr\n", "XllKscNOscNBUakdm9NBYWkpNmcZRaUl2J1OSp1lONwufFISYYoiItpMRLQZo9lEVLSZhg0b0rBh\n", "Q+Li4oi1WoPh7KxWK7F6ZKLoehIK8FRRQl7PqBjerWKYN5fTidfrpdhmIy8vj7y8PGw2G7biYux6\n", "ODe7zUZJUTGOUjsOuwO/z0dEeBjGsAhtpKfVqn0R6CHdrCYzsXqAC6vJrH9FmCt9PVhNZozhar51\n", "xWHcXm8waMPhUG9OvYarh4GrEOnH7nRiKyulyK5Ng+Dxe/H6/ISFhxNjicEcE0NsfBwxsbHEWC1Y\n", "9dihsbGxNGnShISEBGKtViIiIogymSqFe6sY9u2vjBLys5jywUpVl+q2Afh8PoqKiigoKAjGArXZ\n", "bBQXF+NwOHCUlgbXPS43pcFwdjZsthI8Xg8OR1kwoHLwL4cHh1ijYzAIQVyMhXCDAUuUCZMxksiI\n", "cKxRJgzCQJw5BiEEseZojOHhwcnAYqJMhBu0HhLlA1QMQgRHIxrDIw4HX44yBXsxxMUc21cZFRFx\n", "2u4ul9cTHAlZHeXBkn2BAHZ9+HiZ2xkMvlxSVkZAfxblg2R8AX9wqHmp04nX78emj9wtLislIAOU\n", "uJy4PF7NleBy4gsEKC61a9v0gM9Slo/CKw/QrP2NjjZjjDASG2vFatWDL1ssGKO04MuxsbFEx8QQ\n", "HR0dDMYcFxdHgwYNaNiwIfHx8YTrL/tgUOZqAjIfbZvi9FFCrgh5AoEAdn3YuNfrxanPRlhaWopf\n", "H+1YUnLsmRRdLhdufTTk0di2bRtCiGNGfomMjCQq6tjurnLfalhYWLBrmUmvZQJYLJYa6T6q+GtR\n", "nZCrb2lFyGAwaHEWa5Pw8HCEEAwfPrxWz6NQ1CSqSqBQKBQhjhJyhUKhCHGUkCsUCkWIo4RcoVAo\n", "Qhwl5AqFQhHiKCFXKBSKEEcJuUKhUIQ4xxRyIUSyEGKxEGKrEGKLEOJBPb+BEGK+EGKnEGJehZid\n", "CCEmCCF2CSG2CyEuqe0LUCgUir86x6uRe4GHpZRdgQuAMUKIzsB4YL6UsgOwUF9HjxD0D6ALcCkw\n", "RQg1/6tCoVDUJscUWSlljpRyg54uBdLQQrhdCZQHkZsB/E1PXwV8JaX0Sin3AeloYd8UCoVCUUuc\n", "8BB9PW7n+cAqIKE8tBta+LcEPd0UWFnhsEw04VcoQoapU6eyb98+1q1bR+/evdm1axcvvfRSjZU/\n", "fvx4WrduTXp6Ou3bt2fHjh28/vrrJ12O3+9nx44ddOnShenTp7N161YaNmzIhAkTTrqsBQsW4PV6\n", "GTZsGJGRkQBMmDChRq/7WGzatIk2bdqccMgzRWVOyO0hhIgBZqHF4LRX3KbPfnWsGbDU7FiKkOLi\n", "iy9m9OjRfPLJJ9xxxx2MGjWqRsufMGECo0aNYvXq1dx9993HnMTrWLzyyivByb0cDgcvvvgiixcv\n", "PqFjp02bFkw//fTTREVFMXLkyKCIP/fcc6xYseKU7KpKXl4e6enpx9ynW7duvPnmmzVyvr8ixxVy\n", "IUQEmoh/JqWcrWfnCiES9e1JQJ6enwVUDPPdXM87gokTJwaXlJSUUzRfoah5WrRoUWl9wIABZGVl\n", "8dNPPzF48GAApk+fzr333svdd9/NlClTcLvd/Prrr0ExeuSRR0hNTeXmm28GYNy4cbzzzju0bdv2\n", "iIm/3nvvPS6++GL27t2L2+3myy+/ZOnSpfTq1YuHHnqIiy66CCkl77zzDtOmTePrr79m+fLldOzY\n", "ESEEP/zwA9nZ2SxdujRY5t69e5kzZw7jx49n9+7d/Pnnn+zYsYNRo0ZRUlLCiy++yJIlSwB49dVX\n", "+fzzz/noo48AmDNnDk8++WS192fJkiU89NBDAPztb3/D5/MxZcoUHn74YbZv3052djYrVqzg7rvv\n", "xmaz8cILLzB37lyklIwdO5Z58+YxadIk1qxZQ5cuXRg1ahTZ2dl07tyZNWvWnM6jO+tISUmppJXV\n", "Ut1c1vo0swL4FHizSv6rwBN6ejzwsp7uAmwAjEBrYDf6VLlVjpcKRX1k3rx5cv78+VJKKcPDw4P5\n", "119/vXzkkUfklVdeKWfPni3T0tLkuHHjpJRStmzZUhYXF8sRI0ZIt9stU1NT5ZQpU6SUUj711FPy\n", "hx9+kN99951ctmyZ9Pv9UkopMzMz5YUXXhgsf9++ffKhhx6SK1eulAcPHpRSSpmQkCCllHLSpEny\n", "5ZdflqNHj5bjxo2TF154oZwyZYpcuXKllFLKgwcPyi+++EJ6vV558cUXSymltFqtwXIHDRokMzMz\n", "pc/nk7169ZJSStmvXz8ppZR//PGHfPbZZ6WUUt56661y6dKlcvPmzVJKKQcPHnzUexQIBGSPHj1k\n", "IBCQCxculP/73//kuHHj5O233y4ff/xx2apVKymllC6XS/p8PjllyhS5YcMG+dtvv8mUlBQppZQj\n", "RoyQ6enpMioqKlhuxfumODq6dh6h1cerkQ8AbgaGCiFS9eVS4GXgYiHETmCYvo6UchswE9gG/Abc\n", "r59coQhpsrKyeOmll5gzZw4XXXRRpYAJgUAAi8XC66+/TsuWLcnNzWXbtm0AjBgxgpKSEgwGAxER\n", "EdXOQd6iRQsWLVrEjh07SEpKAg4H6AgLC6OgoIDRo0fz+uuvs3DhQsxmM4FAoFp7y+dnT05Opqys\n", "jJ9//pns7Gxat25dqexzzjmHffv2AdC3b19mzJjBXXfdRb9+/Vi/fj0ffPDBEWULIRg9ejSvvPIK\n", "ffv2ZdeuXUycOJFp06YxceJEysrKADAajUE7AA4dOkRaWhoAXbt2xePxVCrX5/OpOdpPkeP1Wlkm\n", "pTRIKbtLKc/Xl7lSykIp5XApZQcp5SVSyuIKx7wopWwnpewkpfy99i9Boah5srKyCAQC7NmzB4C7\n", "7rqLq6++mueff57w8HAOHjxIXl4ehw4dwqaH5fP7/TzxxBP07NmT1NRUMjIy+OWXX7j55pvZs2cP\n", "a9euDZaflpZGbm4uhYWFgCaOr7zyCiaTKbiP0+lk06ZNHDx4kJdeeolrr72W9957j48//pi+ffsG\n", "/c6pqans2rWL7OxsioqK8Hq9TJ48mdmzZ7Ns2TKeeuoplixZwuLFiyksLGTNmjXExcUxd+5c4uPj\n", "KSgoYPv27ezdu5ePPvqIFStWsGLFCnr06MG9997L//3f//HFF19Uuj933nknGRkZREdH88wzz9Cz\n", "Z09ee+01tmzZwpgxY7jsssuYMGECYWFhtGrVilmzZnHttdfyzTffkJGRgdfrJT4+Hr/fj81mA2D7\n", "9u3079+/Vp/rWcvRqum1vaBcK4p6SkXXSlXsdrsMBALVHut2uyutZ2RknPS5vV5vcD0xMVGWlJRU\n", "2qe4uDiYfu2116TH46m2PIfDIR0OR3C9ou0+ny/o5pFSu7bqcLvdcunSpUfkVzze6/VKp9MZXK+Y\n", "llJzs5TbkJ2dfdRzvPXWW9XaoNDgFF0rCoVCJyYm5pjxJ6sGBk5OTq5mzyNp1aoVBQUFwZiYq1at\n", "wm6343A4Ku1XsaF03LhxwS+Go2E2mzHrMU6BSraHhYVVcmMcq9ufx+NhwIABR+RXPD48PLxS+Luq\n", "ofDKe8MIIUhMTDyirMzMTMaOHVutDYpjo2J2KhQVmD9/fr0I9Vbu/1bBixUVUTE7FYoQQjX6KU4G\n", "9WtRKBSKEEcJuUKhUIQ4SsgVCoUixFE+coUCyMjI4LLLLgv2EomOjmbu3Lk0b968ji1TKI6P6rWi\n", "UABerxeTyRQciRgWFobL5Qp2B1Qo6gPV9VpRrhWFAoiIiCA+Pj643qBBAyXiipBBCblCofPWW28F\n", "0++8804dWqJQnBzKtaJQ6OTl5QUnrMrJyaFx48Z1bJFCURk1IEihOA5NmjShb9++CCGUiCtCClUj\n", "V5wyxcXF3HnnnXVtRo2yb98+hBC0bNmyrk2pUaZOnUpcXFxdm6E4TaqrkR9TyIUQyWiBJZqghWz7\n", "SEr5jhBiIjAaOKTv+qSU8jf9mAnAHYAfeFBKOe8o5SohPwtYs2YNffrsB66ta1MUx2Qma9a0oVev\n", "XnVtiOI0OVXXihd4WEq5QY/buU4IMR9N1N+QUr5R5SRdgH+gRQpqBiwQQnSQUlY/A75CoVAoTovj\n", "BZbIkVJu0NOlQBqaQIMWBq4qVwFfSSm9Usp9QDrQp+bMVSgUCkVVTrj7oRCiFXA+sFLPekAIsVEI\n", "MVUIUe58awpkVjgsk8PCr1AoFIpa4IR6rehule+Af0spS4UQ7wP/1TdPAl4Hqmv1OqozvGJE6CFD\n", "hjBkyJATs1ihUCj+IqSkpJCSknLc/Y7ba0UIEQH8DPwmpXzrKNtbAT9JKc8VQowHkFK+rG+bCzwr\n", "pVxV5RjV2HkWoBo7QwXV2Hm2cEpD9IUWmmQqsK2iiAshkirsdjWwWU//CNwghDAKIVoD7YHVp2u8\n", "QqFQKKrneK6VAcDNwCYhRKqe9yRwoxCiO5rbZC9wD4CUcpsQYiawDfAB96uq91+R54HGwAagN1ob\n", "+eQaLP9BoCta/aEbsBV4+xTK8aPZdg7wP+AAEAa8cJLluNF66fYGulfIvwX47BTsOhV2oPUSjj/e\n", "joqzkGMKuZRyGUevtf92jGNeBF48TbsUIU0nNHdLS+B9tM5LNUl588wU4D1OXnjLmQw8rqcL0F5A\n", "F5xged8DfwcCaH0AtlXZ/ibwxynaVZWPgLuPs09HtH+7J2vonIpQQk2apagFqvrM26GNHfsGuAxN\n", "/AagifwAIFvf7ztgmp6eiFajvwjIR6vpPggsAKqOUHwKuBTYifYh+DbwLzTBvxIo9wp+AHwJvItW\n", "g41G+xeYCxQCSyuU+QiwBa1HLUAqWoetSwGPfs4/gIeBYfpyQN93IXB7tXdH80ZuA1zALD3vFeBZ\n", "YDmQBywC7kX74J2s27Yf7QXyjG7bZqCzfo37gcHAD8c4r+JsRc21ojhDPA0YARPwOVot9h9ALLAb\n", "SAKeQxPvnWheu+5otd7v0cT7bY4+fAHgF7QXyIvA5Wgvi1v149oAF6IJbBKwGOgAtNWP7QHsQRNC\n", "gE1AMprLpQ/wFZqAxwBO/ToS9f2fAH4HItHGwaWhuWeONRz+TbSOXlcAF+v3IwsoA6ajifQyYIh+\n", "vY2BQWhfOPv19MX6vbKhNU2B5ir6Ee1FofgroWrkijNEFlqtcxaagIEmUgY00ZbAPKA1kIMm5gB/\n", "A0r1/aoTcdDEcy+aCLerUD6AGShCE7h30GrXJrTa+9EoQ/O7gybgDmA+sBHtBVCx7CS0AdBRaC+P\n", "JWgvpKFoNeuXjlJ+K7Q+APmAFdiHVhv/BO1lVYLmdzegvTjKz+XX/zasxna/fh8UfzWUkCtqiWw0\n", "QSz3Hf8buARN5IxotfDdwC40H7obrbb5MpoQO4DtaH7wu9FEcefh4tmK5g7JrpD3BYen/wGtKWc+\n", "2tfAUDQf+ItoteH2HO5stVG3Mx9NRM9DE9fdaO6eO9DEfyFQrP9tgfZS+hCtRp2K5o65GK3Gvxit\n", "8XECmuumamPvoxx+KYxHa7R9CViB5q4ZCTyGVtNvAPwEvKqfa7V+zCE090y+Xs5+/R4r/mqo2Q8V\n", "p8zJ9yMvd0tUV2v0Udnbl4PmwjhR1gPnAhFo7ovb0Pzgxgr7lKDVgkHrWXsTWu38aBxEG6xcToDD\n", "dZ8A2ldE+bW40UT3aPjQfN4VRbZiWeXrLrSvh6rl+Tn89eLW181Uxod2zY8d5fyqH/nZggr1pqgH\n", "mDj2p3/VJpuTEfFWaEIeoa8vQWvcNFbZz1ohfQeQcYwym1ZZN1RJV7yW6kQcNJG9+Bhlla9XFOeK\n", "5YVV2D+SI0UctJfOuGPYoDibUY2dirOEfVXWvz+BYwRat73aJuoMnKPFGTiHor6ihFxxlnKshlGF\n", "4uxCuVYUCoUixFFCrlAoFCGOEnKFQqEIcZSQnwQnMi9wfSMUbdZIqWsDToGUujbgpAnV30co2l2b\n", "NqvGzpMgJSUl5AJg1KbNQgjCwp5Hm9SpZgkEdmMwtD3+jvWI+mtzHkJMPeqWUPxNQ2jaXZs2KyFX\n", "nDK9evXC59tQK2VPnDixUhSpUCAUbVacHSjXikKhUIQ4dTZE/4yfVKFQKM4CjjZEv06EXKFQKBQ1\n", "h3KtKBQKRYijhFyhUChCnDMu5EKIS4UQ24UQu4QQT5zp81eHEGKaECJXCLG5Ql4DIcR8IcROIcQ8\n", "IURchW0T9GvYLoSok0mghRDJQojFQoitQogtQogHQ8TuKCHEKiHEBiHENiHES6Fgt25HmBAiVQjx\n", "UwjZvE8IsUm3e3Uo2C2EiBNCfCeESNN/I31DwOaO+j0uX2xCiAfPiN1SyjO2oM3HmY4252gEWqyq\n", "zmfShmPYNggt/tjmCnmvAo/r6SeAl/V0F932CP1a0gFDHdicCHTX0zFogSg713e7dVvM+t9wtGCY\n", "A0PE7nFoESx+DIXfiG7LXqBBlbx6bTcwA7ijwm8ktr7bXMV+A1rUk+QzYfeZvrh+wNwK6+OB8XV5\n", "w6vY14rKQr4dSNDTicB2PT0BeKLCfnOBC+qB/bOB4aFkN9rk2muArvXdbqA5WvTnocBPofIb0YW8\n", "YZW8emu3Ltp7jpJfb20+iq2XAEvPlN1n2rXSjMOhxgEy9bz6SoKUMldP5wIJeropmu3l1Pl1CCFa\n", "oX1RrCIE7BZCGIQQG9DsWyyl3Er9t7s8BE+gQl59txm0UEYLhBBrhRB36Xn12e7WwCEhxHQhxHoh\n", "xMdCiGjqt81VuQEtajecAbvPtJCHbF9Hqb0yj2V/nV2bECIGLYDkv6WU9orb6qvdUsqAlLI7Wi33\n", "QiHE0Crb65XdQohRQJ6UMpVqJjuvbzZXYICU8ny0QKBjhBCDKm6sh3aHAz2AKVLKHmgBXMdXMqj+\n", "2RxECGFEizD+bdVttWX3mRbyLDSfUTnJVH4j1TdyhRCJAEKIJLQIwHDkdTTX8844QogINBH/TEo5\n", "W8+u93aXI6W0Ab8APanfdvcHrhRC7EWraQ0TQnxG/bYZAClltv73EPAD0If6bXcmkCmlXKOvf4cm\n", "7Dn12OaKjATW6fcbzsC9PtNCvhZoL4Ropb+1/gH8eIZtOBl+BG7V07ei+aDL828QQhiFEK3RQrKv\n", "PtPGCSEEWgThbVLKtypsqu92NypvuRdCmNACWqZSj+2WUj4ppUyWUrZG+2xeJKW8pT7bDCCEMAsh\n", "LHo6Gs13u5l6bLeUMgc4IITooGcNB7YCP1FPba7CjRx2q8CZuNd10AgwEq13RTowoS4bJKrY9RVa\n", "BFsPmh//dqABWuPWTmAeEFdh/yf1a9gOjKgjmwei+Ws3oAlhKnBpCNh9Llqk5A3AJuAxPb9e213B\n", "lsEc7rVSr21G8zdv0Jct5f9zIWD3eWiN4BvRArDG1nebdTuigXzAUiGv1u1WQ/QVCoUixFEjOxUK\n", "hSLEUUKuUCgUIY4ScoVCoQhxlJArFApFiKOEXKFQKEIcJeQKhUIR4ighVygUihBHCblCoVCEOP8P\n", "xTewrYD3a3AAAAAASUVORK5CYII=\n" ], "text/plain": [ "<matplotlib.figure.Figure at 0x7fde0c33ec50>" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "im = Image.open('symbolic_graph_opt.png')\n", "plt.imshow(im)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "# Printing / Drawing Theano graphs" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": true }, "outputs": [], "source": [ ">>> import numpy\n", ">>> import theano\n", ">>> import theano.tensor as T\n", ">>> rng = numpy.random\n", ">>> # Training data\n", ">>> N = 400\n", ">>> feats = 784\n", ">>> D = (rng.randn(N, feats).astype(theano.config.floatX), rng.randint(size=N,low=0, high=2).astype(theano.config.floatX))\n", ">>> training_steps = 10000\n", ">>> # Declare Theano symbolic variables\n", ">>> x = T.matrix(\"x\")\n", ">>> y = T.vector(\"y\")\n", ">>> w = theano.shared(rng.randn(feats).astype(theano.config.floatX), name=\"w\")\n", ">>> b = theano.shared(numpy.asarray(0., dtype=theano.config.floatX), name=\"b\")\n", ">>> x.tag.test_value = D[0]\n", ">>> y.tag.test_value = D[1]\n", ">>> # Construct Theano expression graph\n", ">>> p_1 = 1 / (1 + T.exp(-T.dot(x, w)-b)) # Probability of having a one\n", ">>> prediction = p_1 > 0.5 # The prediction that is done: 0 or 1\n", ">>> # Compute gradients\n", ">>> xent = -y*T.log(p_1) - (1-y)*T.log(1-p_1) # Cross-entropy\n", ">>> cost = xent.mean() + 0.01*(w**2).sum() # The cost to optimize\n", ">>> gw,gb = T.grad(cost, [w,b])\n", ">>> # Training and prediction function\n", ">>> train = theano.function(inputs=[x,y], outputs=[prediction, xent], updates=[[w, w-0.01*gw], [b, b-0.01*gb]], name = \"train\")\n", ">>> predict = theano.function(inputs=[x], outputs=prediction, name = \"predict\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Pretty Printing" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "'gt((TensorConstant{1} / (TensorConstant{1} + exp(((-(x \\\\dot HostFromGpu(w))) - HostFromGpu(b))))), TensorConstant{0.5})'" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ ">>> theano.printing.pprint(prediction) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Debug Print\n", "The pre-compilation graph:" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Elemwise{gt,no_inplace} [@A] '' \n", " |Elemwise{true_div,no_inplace} [@B] '' \n", " | |DimShuffle{x} [@C] '' \n", " | | |TensorConstant{1} [@D]\n", " | |Elemwise{add,no_inplace} [@E] '' \n", " | |DimShuffle{x} [@F] '' \n", " | | |TensorConstant{1} [@D]\n", " | |Elemwise{exp,no_inplace} [@G] '' \n", " | |Elemwise{sub,no_inplace} [@H] '' \n", " | |Elemwise{neg,no_inplace} [@I] '' \n", " | | |dot [@J] '' \n", " | | |x [@K]\n", " | | |HostFromGpu [@L] '' \n", " | | |w [@M]\n", " | |DimShuffle{x} [@N] '' \n", " | |HostFromGpu [@O] '' \n", " | |b [@P]\n", " |DimShuffle{x} [@Q] '' \n", " |TensorConstant{0.5} [@R]\n" ] } ], "source": [ ">>> theano.printing.debugprint(prediction) " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "The post-compilation graph:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Elemwise{gt,no_inplace} [@A] '' 7\n", " |HostFromGpu [@B] '' 6\n", " | |GpuElemwise{Composite{scalar_sigmoid((-((-i0) - i1)))}}[(0, 0)] [@C] '' 5\n", " | |GpuGemv{inplace} [@D] '' 4\n", " | | |GpuAlloc{memset_0=True} [@E] '' 3\n", " | | | |CudaNdarrayConstant{0.0} [@F]\n", " | | | |Shape_i{0} [@G] '' 2\n", " | | | |x [@H]\n", " | | |TensorConstant{1.0} [@I]\n", " | | |GpuFromHost [@J] '' 1\n", " | | | |x [@H]\n", " | | |w [@K]\n", " | | |TensorConstant{0.0} [@L]\n", " | |GpuDimShuffle{x} [@M] '' 0\n", " | |b [@N]\n", " |TensorConstant{(1,) of 0.5} [@O]\n" ] } ], "source": [ ">>> theano.printing.debugprint(predict) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Picture Printing of Graphs\n", "The pre-compilation graph:" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The output file is available at logreg_pydotprint_prediction.png\n" ] } ], "source": [ ">>> theano.printing.pydotprint(prediction, outfile=\"logreg_pydotprint_prediction.png\", var_with_name_simple=True)\n", "# The output file is available at pics/logreg_pydotprint_prediction.png" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<img src=\"logreg_pydotprint_prediction.png\"/>\n", "The post-compilation graph:" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The output file is available at logreg_pydotprint_predict.png\n" ] } ], "source": [ ">>> theano.printing.pydotprint(predict, outfile=\"logreg_pydotprint_predict.png\", var_with_name_simple=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<img src=\"logreg_pydotprint_predict.png\"/>\n", "The optimized training graph:" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The output file is available at logreg_pydotprint_train.png\n" ] } ], "source": [ ">>> theano.printing.pydotprint(train, outfile=\"logreg_pydotprint_train.png\", var_with_name_simple=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "<img src=\"logreg_pydotprint_train.png\"/>" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 0 }