{ "cells": [ { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "[wasserstein GAN and the kantorovich-Rubinstein Duality](https://vincentherrmann.github.io/blog/wasserstein/)\n", "# Wasserstein GAN(WGAN) and the Kantorovich-Rubinstein Duality\n", "\n", "## 1. Earth Mover's Distance" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "$l$ states로 이루어진 두 discrete probability distribution $P_r, P_\\theta$ 이 주어져있다.\n", "\n", "$EMD(P_r, P_\\theta) = \\inf_{\\gamma \\in \\prod} \\sum_{x,y} ||x-y||\\gamma(x,y)= \\inf_{\\gamma \\in \\prod}\\mathbb{E}_{(x,y)\\sim \\gamma}||x-y||$ \n", " \n", "$\\sum_x \\gamma(x,y) = P_r(y), \\sum_y \\gamma(x,y) = P_\\theta(x)$(Marginal Probability)\n", "\n", "$D=||x-y||, \\Gamma =\\gamma(x,y)$ 라고 하면, $\\Gamma, D\\in \\mathbb{R}^{l\\times l}$ 이게 되고 \n", "\n", "$EMD(P_r, P_\\theta)= \\inf_{\\gamma \\in \\prod}\\langle D, \\Gamma \\rangle_{F}$ \n", "\n", "$\\langle , \\rangle_F$ 은 the Frobenius inner product이며, 두 matrix를 element-wise하게 곱한 후 다 더한 값이다." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## 2. Linear Programming\n", "\n", "$A \\in \\mathbb{R}^{m\\times n}, b\\in \\mathbb{R}^m, c\\in \\mathbb{R}^n$이 고정되어 있을 때,\n", "\n", "$Ax=b, x\\geq 0$ 을 만족하면서 $z=c^Tx$를 최소화하는 $x\\in \\mathbb{R}^n$ 을 찾는 문제이다. \n", "\n", "위 문제를 mapping 해보면, $n= l^2, m=2l$ ($2l$ 개의 constraint가 존재하며, $l^2$ 의 변수를 찾는 문제이다. )\n", "\n", "$x= vec(\\Gamma), c=vec(D)$ 로 대응시킬 수 있으며, $b = \\begin{bmatrix} P_r \\\\\\ P_\\theta \\end{bmatrix}$\n", "\n", "$A = \\begin{bmatrix} 1&&1&&\\cdots &&0&&0&&\\cdots &&\\cdots&&0 &&0&& \\cdots\\\\ 0 && 0 &&\\cdots &&1 && 1 &&\\cdots &&\\cdots && 0 && 0 && \\cdots \\\\\\vdots&&\\vdots&&\\vdots &&\\vdots&&\\vdots&&\\vdots &&\\vdots&&\\vdots &&\\vdots&& \\vdots\\\\ 0 && 0 &&\\cdots &&0 && 0 &&\\cdots &&\\cdots && 1 && 1 && \\cdots \\\\ 1 && 0 &&\\cdots &&1 && 0 &&\\cdots &&\\cdots && 1 && 0 && \\cdots \\\\ 0 && 1 &&\\cdots &&0 && 1 &&\\cdots &&\\cdots && 0 && 1 && \\cdots \\\\ \\vdots && \\vdots &&\\ddots &&\\vdots && \\vdots &&\\ddots &&\\cdots && \\vdots && \\vdots && \\ddots \\\\ 0 && 0 &&\\cdots &&0 && 0 &&\\cdots &&\\cdots && 0 && 0 && \\cdots \\end{bmatrix}\\in \\mathbb{R}^{2l\\times l^2}$, \n", "$x = \\begin{bmatrix}\\gamma(x_1, y_1) \\\\ \\gamma(x_1,y_2)\\\\\\vdots \\\\ \\gamma(x_2, y_1)\\\\\\gamma(x_2,y_2) \\\\ \\vdots \\\\ \\vdots \\\\ \\gamma(x_n,y_1) \\\\ \\gamma(x_n,y_2) \\\\\\vdots \\end{bmatrix}\\in \\mathbb{R}^{l^2}$" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## 3. Dual Form\n", "\n", "실제 GAN에서 $$\\gamma$$를 구하여 earth mover distance를 구하는 것은 불가능하다. 하지만, **EMD**(earth mover distance)를 구하도록 학습시킬 수는 있다.\n", "Duality를 이용해서 **EMD**를 쉽게 구해보자.\n", "모든 **LP**(linear programming) 문제들은 다음과 같이 문제를 정의할 수 있다.\n", "\n", "**primal form**: \n", "\n", "minimize $z = c^Tx$ when $Ax=b$ and $x\\geq 0$ \n", "\n", "**dual form:**\n", "\n", "maximize $\\widetilde{z}=b^Ty$ when $A^Ty\\leq c$\n", "\n", "$z = c^T x \\geq (A^Ty)^T x = y^T A x = y^Tb=\\widetilde{z}^T = \\widetilde{z}$\n", "\n", "$A$ : fixed matrix\n", "\n", "$b$ : marginal probability\n", "\n", "$c$ : point-to-point distance\n", "\n", "$z$ : **EMD**" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## 4. Farkas Theorem" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "1. $\\hat{A}= [a_1 | a_2 | \\cdots | a_n ]\\in \\mathbb{R}^{d\\times n} $, $x\\in\\mathbb{R}^n_{\\geq 0}$ 일 때 $\\hat{A}x$ 가 표현할 수 있는 영역은 $\\mathbb{R}^d$에서 원점을 꼭지점으로 가지는 어떤 도형으로 생각해 볼 수 있다.\n", "\n", "2. $\\hat{b}\\in \\mathbb{R}^d$ 에 대해서, $\\hat{A}x = \\hat{b}$으로 표현이 되는 $x \\in \\mathbb{R}^n_{\\geq 0}$가 존재하는 경우와 존재하지 않는 경우를 생각해 볼 수 있다.\n", "\n", "3. 존재하지 않는 경우, 다음과 같은 조건을 만족하는 hyperplane $h$가 존재한다.\n", "\n", " > * 원점을 지난다.\n", " > * $\\hat{b}$ 와 $\\hat{A}x(x \\in \\mathbb{R}^n_{\\geq 0})$인 도형 사이에 존재한다.\n", "\n", "4. hyperplane $h$에 수직인 $\\hat{y}$ 를 생각해 볼 수 있고, $\\hat{b}^T\\hat{y}>0$이면서 $\\hat{A}^T\\hat{y} \\leq 0$이게 잡을 수 있다. \n", "\n", " 이는 $h$를 중심으로 $\\hat{b}$와 $a_i$ 들이 다른 위치에 있기 때문이다. " ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## 5. Strong Duality" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "**primal form**: \n", "\n", "minimize $z = c^Tx$ when $Ax=b$ and $x\\geq 0$ " ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "* 위 문제를 만족하는 최소의 solution 을 $z^* = c^Tx^*, Ax^*=b$ 라고 하자." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "$\\hat{A} = \\begin{bmatrix} A \\\\ -c^T \\end{bmatrix}$, $\\hat{b_{\\epsilon}} = \\begin{bmatrix} b \\\\-z^*+\\epsilon \\end{bmatrix}$ , $\\hat{y} = \\begin{bmatrix} y \\\\ \\alpha\\end{bmatrix}$" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "만약 $\\epsilon = 0 $ 이면, $\\hat{A}x^* = \\hat{A} = \\begin{bmatrix} Ax^* \\\\ -c^Tx^* \\end{bmatrix}= \\begin{bmatrix} b \\\\ -z^* \\end{bmatrix}=\\hat{b_0}$\n", "\n", "만약 $\\epsilon > 0 $ 이면, $\\hat{A}x' = \\hat{b_\\epsilon}$ 이 존재한다면,\n", "\n", "$Ax'=b$, and $c^Tx' = z^*-\\epsilon$ 이 되어서 $x^*$의 최소성에 모순이 된다.\n", "\n", "고로 $\\hat{A} x = \\hat{b_\\epsilon}$의 해가 없게 되고 **Farkas theorem**에 의해\n", "\n", "$\\hat{y}$ 가 존재하여 $\\hat{b_\\epsilon}^T\\hat{y}>0$이면서 $\\hat{A}^T\\hat{y}\\leq 0$\n", "\n", "$\\Leftrightarrow b^T y - z^* \\alpha + \\epsilon \\alpha > 0$ 이고 $A^Ty-c\\alpha \\leq 0 $\n", "\n", "\n", "\n", "$\\Leftrightarrow b^T y > \\alpha(z^* - \\epsilon)$, $c^T\\alpha \\geq A^Ty $" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### $\\alpha >0$\n", "$\\alpha z^*=\\alpha c^Tx^*\\geq (A^Ty)^Tx^*=y^TAx^*=y^Tb>\\alpha(z^*-\\epsilon)$ \n", "\n", "$\\alpha\\epsilon >0$ \n", "$\\therefore \\alpha>0$" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "$y_\\epsilon = \\frac{y}{\\alpha}$ 라고 하자.\n", "\n", "그러면, $b^Ty_\\epsilon > z^*-\\epsilon$ 이고 $A^Ty_\\epsilon\\leq c$ 인 $y_\\epsilon$이 모든 $\\epsilon>0$에 대해서 존재한다고 말할 수 있다.\n", "\n", "한편 \n", "$z^* = c^T x^* \\geq (A^Ty_\\epsilon)^T x^* = y_\\epsilon^T A x^* = y_\\epsilon^Tb=b^Ty_\\epsilon>z^*-\\epsilon$ \n", "\n", "$\\epsilon \\rightarrow 0$ 일 때, $z^*>b^Ty_\\epsilon > z^*-\\epsilon$ 이고 $A^Ty_\\epsilon\\leq c$ 인 $y_\\epsilon$이 존재한다.\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "**primal form**: \n", "\n", "minimize $z = c^Tx$ when $Ax=b$ and $x\\geq 0$ \n", "\n", "**dual form:**\n", "\n", "maximize $\\widetilde{z}=b^Ty$ when $A^Ty\\leq c$\n", "\n", "이 두 문제는 결국 같은 $z$를 구한다." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## 6. Dual implementation" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "이제 Strong duality를 이용해서 primal form 문제의 답을 dual form 문제의 답으로 해결해도 된다. \n", "### 6.1 Recall\n", "$A = \\begin{bmatrix} 1&&1&&\\cdots &&0&&0&&\\cdots &&\\cdots&&0 &&0&& \\cdots\\\\ 0 && 0 &&\\cdots &&1 && 1 &&\\cdots &&\\cdots && 0 && 0 && \\cdots \\\\\\vdots&&\\vdots&&\\vdots &&\\vdots&&\\vdots&&\\vdots &&\\vdots&&\\vdots &&\\vdots&& \\vdots\\\\ 0 && 0 &&\\cdots &&0 && 0 &&\\cdots &&\\cdots && 1 && 1 && \\cdots \\\\ 1 && 0 &&\\cdots &&1 && 0 &&\\cdots &&\\cdots && 1 && 0 && \\cdots \\\\ 0 && 1 &&\\cdots &&0 && 1 &&\\cdots &&\\cdots && 0 && 1 && \\cdots \\\\ \\vdots && \\vdots &&\\ddots &&\\vdots && \\vdots &&\\ddots &&\\cdots && \\vdots && \\vdots && \\ddots \\\\ 0 && 0 &&\\cdots &&0 && 0 &&\\cdots &&\\cdots && 0 && 0 && \\cdots \\end{bmatrix}\\in \\mathbb{R}^{2l\\times l^2}$ \n", "\n", "\n", "$c^T = \\begin{bmatrix} D_{1,1} && D_{1,2}&&\\cdots&&|&&D_{2,1}&&D_{2,2}&&\\cdots&&|&&\\cdots \\end{bmatrix} $ \n", "**Note**\n", "1. $D_{i,i}=0$\n", "2. $D_{i,j}=D_{j,i}$\n", "\n", "$$b = \\begin{bmatrix} P_r \\\\ P_\\theta \\end{bmatrix}$$\n", "$$y^* = \\begin{bmatrix} f \\\\ g\\end{bmatrix} \\in \\mathbb{R}^{2l}$$\n", "\n", "* $f, g$는 $l$ dim vector이지만, 함수로 보기로 한다.\n", "$$f_1,f_2,\\cdots,f_l =f(x_1),f(x_2),\\cdots f(x_l)$$\n", "$$g_1,g_2,\\cdots,g_l =g(x_1),g(x_2),\\cdots g(x_l)$$\n", "* $D_{i,j}=||x_i-x_j||$" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### 6.2 Dual form problem" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "$EMD(P_r,P_\\theta)=\\max f^TP_r+g^TP_\\theta$ when $$A^Ty \\leq c\\Leftrightarrow f(x_i)+g(x_j)\\leq D_{i,j}$$\n", " \n", "$$f(x_i)+g(x_i) \\leq D_{i,i}=0$$" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "여기에서도 애매하게 설명하긴 했는데, nonnegative 한 $P_r, P_\\theta$를 이용하여 $f_TP_r+g^T_\\theta$를 최대화해야 하기 때문에 $f(x_i)+g(x_i)$가 크면 클수록 좋다. 그렇기 때문에 $f(x_i)=-g(x_i)$이어야 한다.\n" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "$D_{i,j}\\geq f(x_i)+g(x_j) =f(x_i)-f(x_j)$ \n", "$D_{i,j}=D_{j,i}\\geq f(x_j)-f(x_i)$\n", "$\\Rightarrow |f(x_i)-f(x_j)| \\leq D_{i,j}=||x_i-x_j||$" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "#### 6.2.1 Lipschitz continuity" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "**Def)** $f: \\mathbb{R}\\rightarrow \\mathbb{R}$ 가 lipschitz continuous\n", "$\\Leftrightarrow$ 만약에 양수인 $K$가 존재하여 모든 실수 $x_1, x_2$에 대해\n", "$|f(x_1)-f(x_2)|\\leq K|x_1-x_2|$\n", "\n", "\n", "위의 $f$는 $K=1$인 경우의 Lipschitz continuous 를 만족하고, $||f||_{L\\leq1}$ 로 쓴다. \n", "모든 constraint의 조건들이 Lipschitz continuous를 만족하는 함수로 함축되게 된다." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "$$EMD(P_r, P_\\theta) = \\sup_{||f||_{L\\leq1}} \\mathbb{E}_{x\\sim P_r}f(x)- \\mathbb{E}_{x\\sim P_\\theta} f(x)$$\n", "\n", "직관적으로 봤을 때, $P_r>P_\\theta$ 일 때 $f$가 크고, $P_r" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "x1 = np.linspace(-10, 10, 100)\n", "x2 = np.linspace(0, 20, 100)\n", "y1 = gaussian_function(x1, mu = pd1['mu'] , sigma = pd1['sigma'])\n", "y2 = gaussian_function(x2, mu = pd2['mu'], sigma = pd2['sigma'])\n", "plt.plot(x1, y1, 'b-', label = pd_spec(pd1))\n", "plt.plot(x2, y2, 'r-', label = pd_spec(pd2)) \n", "plt.title(\"Two gaussian distributions\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHylJREFUeJzt3Xl0nHd97/H3dxbNaF8s2bEteUucNG7IKtKEQJMCaUOg\nyeVAW9KNctL6trfcQ2+57Q0XTnqh5/YUek4ptCk0vaXccmlYulBTTNNsFAoNxCHOYie2FceLvEmy\ntUuzf+8fM1JkWZqZ2CONn9HndY6O5nmen2a+P2n00U+/+T3zmLsjIiK1JVTtAkREpPIU7iIiNUjh\nLiJSgxTuIiI1SOEuIlKDFO4iIjVI4S4iUoMU7iIiNUjhLiJSgyLVeuDOzk7ftGlTtR5eRCSQnn76\n6SF37yrVrmrhvmnTJnbt2lWthxcRCSQzO1xOO03LiIjUIIW7iEgNUriLiNQghbuISA1SuIuI1CCF\nu4hIDVK4i4jUoKqtc5fyZLI5IuHK/Q2eTmXJutMUe/VHPzSRZDyRYXVzjIa6MGY2eyybc14ZmmBo\nIkXvxvZzahmeTHF6MsW6tjjhkDGZzNLeECWZyRGLhEhnnbpICHdn74kxYpEQl61uJpdzzJh9rHQ2\nx8B4klzOaWuIsu/kOLFImMZYmJAZJ0YTtDdGWd9WT1MswsGhSdxhS2cjXujDmpb4gn1OZ3NEQsbg\neJKB8SR7jo/y7ht6ODmWYGw6zRVrmgmF8nW4+1n9nzEylSIeDROLhMjknEQ6S3M8ypHTU0wkM2zp\naiQeDQOQSGeZe/XKoYkkXc0xJpMZOhrrGJxI0tkYO6v/M3I55+jwFMeGp7l+YzvTqSwNsTBj0xma\nYhHq68Jl/YxfOD7K5s5GBseT9HQ00BSLMJXKcPj0FFu6GtlzfIzNqxoJhw3PQSwaYiyRZjKZJRIy\nQiGjtT7KEy8N0NkU47oNbbP9G0+kqYuEiEVerWV0Ks3AeILmeJRjI9P86LoWJpMZBieSrGqM0dUc\nI5nJcmx4ms2djew7NU4m61y1vhWAqVSGiWSGrqZY/vvgcHoySVdTbMGfx1wzP7NMNkc4ZOc8fwHC\nIaNvYJwjZ6ZY39bA5WuaGJlK8697T/KOq9eRyTl7jo3S09HA+rb62edUczxCQ12EXM45MDBBe2OU\nrqYYY9MZTo4l2NzZSDRsuDP7HFpMNucY+XaLPc8qSeFeBe7O/lMTpLM5vv7scZ45OkI8GqajIcp/\nHDxNa32U9oY6XhmaZHAiyVXrWgkZNNRFGJ5K0dUcoy4cor2xjoHxJP3DU/S0N5DNOVOpDL2bOmio\nC5PNOY+/NMB0OsvmVY2MJdI8dWgYgOZYhNu3reHg0CS7j47M1hYyuKQlTjhsDE+mSaSzZAq/IJ1N\nMTK5fGi3N9QRi4R46eQ4yUzurP6tbo4xMJ4kEjIcaG+oI53NMTqdBsAMoqEQ9XVhLlvdxMhUimMj\n0yTSZ9/PYlriEcYSmdntzqY6hiZSrGrMP87a1nomkhk6m2McHJhgMpWhsS7CePLVr/kff//8WfcZ\nCRlXrm3h8OlJ1rXVMzSRJBwywmZMprKMJdKz/T4xmgDg8jVNHBiYwB0a6sLEo2HikRCnJ1PnfE9m\n1IVDpLL5Yw11Ya7pbqMxFubg0CQGvDw4uWi/m2IRbr50FQdOjRe+j4YB6VyOrqYYXmiz+8jIWX2N\nR0OsaoxxciwxG3avVUdjHR2NdQAcHJygPhqmszlGMp2jpT7CoaGp2X4tZG1rnNOTKVKZHKubYwxO\nJHGHWCTEpV1NHDmT/yNZHw2TzGSJhkMkMzm2rm5iS1cj3+s7TXtjHSdGp+npaKAuHKKrOUYm6zzb\nP8J1G9p4+vAwbfV1xKMhejoaSGdzPH14mGg4RE97A/sK37f5/uyJPlKZHKfGkgsen/szA6iPhklk\n8n/Am+MRDEhkcrzh0lXctGUVN2xsZ2AsyYGBcU6OJjg5lmAqleWVoUlGp9KEQ8a9b9zMf/+pK87r\nZ1Euq9YFsnt7e72Wz1B1d6ZSWU6MJmiJRzg9meIbz53g5FiC7/UNcbwQEJGQcW1PG8lMjoHxBJd2\nNTGRzBCLhNjQ0Uh9XYiv7OpnTUuMsBmXtMY5ODhJR2MdI1Npkpns7NeYGfFoiOf6R8/6JV7dnB8l\nZnPwc6/vJh4J88rQJDtfOEF9NMyv/fgWVjXWceDUBBPJzGwwxSIhRqbS3LK1k6ZYmEdfHGB0Kk0q\nmyMeDbP3+BhvuHQVo9NpxhNprulpY3QqzanxBJd1NTE0mWJNc5yRqRRTqSw/tqWD0ek0/cPTNMcj\nJDM5+gYmaG+I0t3ewGWrmwgZ7Ds5wXUb2jCDRDpHMpNlbWuclwcmGZ1O0zcwwZsu72QymWHP8THO\nTKbYuKqBk6NJsrn8fzqTyQwHBia49fIuVjfHOD2RYm1bnJAZz/WPkMk5q5tjXL+hneMj04xMp/n2\n/kFikTA5d67ubiNkMJXK0t4YJR4Jk8k5gxNJutvrweHpw8PcuLmDK9e28J0Dg4RDIcYT+V/ekak0\n0XCILV2NtNZHSaSztNZHOTmaYO+JMYan0mxb28IrQxNMJrNEI0Z9NMy2tS2saY2ztjXO8ZH8c+T0\nRIrO5joe2XuKw6enWN9Wz+bORhLpbP6/gKYYLw9OcElLnPFEhq1rmrh5yypOjibo6Wjgey8Pkc7m\nnw9XrW/l+f4RDp+Z4tbLu3CHnDuxSIjxZIbxRIb2hijRcIgdzx6nOR5l06oGUpkcITMS6SyXrWli\nKvnqH7wzkynWttZz1foWnu0f5ZG9p7imu43eTe2sbo5xZjLFM0dHWNsSZ3VLjGf7R1nfVk9zLMLL\ngxMcG5mmp72BpniEkBmRkHFqPMmPrmvh688e5+iZKXo3dRAyo6ejnl2HhulorOPkaIKJZGb2j/Il\nrXHi0fyg5vjINMNTKX766nVMp7McGJjgrVeuZm1rPYeGJnnspQFeGZpk+49v4d/2D9I3MMFH3n4l\nI1NpXjg+Sk97AyEzptIZjp6ZorU+ymWrm4mEjCOF7Z6Oer69fwiz/GDp8X0DHD0zfVYOtMQjbFzV\nSENdmEQ6SyhkXLWulbdcuZrbrlh9XtliZk+7e2/Jdgr3yklmsnx7/xDfeO44/953mqGJc0cCZvDm\nK1bzlivXEAkbN29ZRU9HQ9H7HZlK0RSLvKbpmWzOGZ1O01YfXfDf/5l6I6EQ4RL/TorUokpPeQIc\nGprk8Jmp/B/B1U10t9efNX1VCeWGu6ZlKsDdeegHR/nTxw9wYjSBGbx+Ywc/fc1aNq1q5PjINBtW\nNXD1+jZe1936mu+/raHuNX9NOGSz/0YvptJPOpEgqXSwA2zqbGRTZ2PF7/d8KNwr4NOP9fHJR/dz\nw8Z2Pvz2K7lhYztrW+urXZaIrGAK9wt0cHCCP338AHdfu44/+blrl/wVcBGRcmid+wX6g50vEY+G\n+cjbtynYReSiUTLczexzZjZgZi8scvwXzOw5M3vezL5nZtdUvsyL0+6jIzz64il+/dYtdDXHql2O\niMisckbunwfuKHL8FeBWd38d8PvAgxWoKxA+9eh+2hui/Motm6tdiojIWUqGu7t/GzhT5Pj33H24\nsPkk0F2h2i5qu4+O8MS+QX71TVvOOttTRORiUOk593uBb1b4Pi9Kn37sAG0NUd77hk3VLkVE5BwV\nG3Ka2U+QD/c3FmmzHdgOsGHDhko99LJ79ugIj780wO/81BUatYvIRakiI3czuxr4P8Dd7n56sXbu\n/qC797p7b1dXyYt3X7Q+/dgBWuuj/PLNG6tdiojIgi443M1sA/APwC+5+/4LL+niduT0FI+9NMD7\nbtlEczxa7XJERBZUck7BzB4CbgM6zawf+D0gCuDunwXuB1YBf15Y550p530Pgupru48B8DO9PVWu\nRERkcSXD3d3vKXH8V4FfrVhFFzF35x+fOcZNWzpY36a3FxCRi5fOUH0NHtl7ileGJnnndeurXYqI\nSFEK9zLlcs5Hv76XK9e2cPe1CncRubgp3Mv0wyPDHBuZ5j//+JbZy42JiFysFO5l+n9PHqahLsxb\nt62pdikiIiUp3MswMJ7g68+d4J4bN+ikJREJBIV7Gb7x3AmyOeeeG7X8UUSCQeFewuB4kgee6OOa\nnjYuW91c7XJERMqicC/hL79zkDOTKT7xrqurXYqISNkU7kWMTqX54pOHecfV67jiEo3aRSQ4FO5F\nfOHJQ0ymsvz6rZdWuxQRkddE4b6I6VSWv/7uIW67oott61qqXY6IyGuicF/EV58+yunJFL+hUbuI\nBJDCfQHpbI6/+LeDXL+hjRs3d1S7HBGR10zhvoB/+GE/x0am+Y3bLqPwNsYiIoGicJ9nIpnhjx7e\nz3Ub2njrlaurXY6IyHlRuM/zmW/1MTSR5P53bNOoXUQCS+E+x8nRBH/5nVd453XruW5De7XLERE5\nbwr3OT7/vUNksjl++/bLq12KiMgFUbgXHD49yV9/9xXefvU6ejoaql2OiMgFUbiTvzbqR772AnXh\nEB++88pqlyMicsEU7sCTB8/wnQND/LfbL+eS1ni1yxERuWAlw93MPmdmA2b2wiLHzcw+bWZ9Zvac\nmV1f+TKXTi7nfPxfXqKrOcbP/9iGapcjIlIR5YzcPw/cUeT424CthY/twGcuvKzl8+VdR9l9dIT/\neeeP6NqoIlIzSoa7u38bOFOkyd3A33jek0Cbma2tVIFL6fn+UT769T3ctKWD/3Tt+mqXIyJSMZWY\nc18PHJ2z3V/Ydw4z225mu8xs1+DgYAUe+vy5Ox/9+h5a4lH+7Oev1wlLIlJTlvUFVXd/0N173b23\nq6trOR/6HI+/NMCuw8N84K1b6WyKVbUWEZFKq0S4HwPmXjm6u7DvopXNOZ/4l31s7mzkZ3t10WsR\nqT2VCPcdwC8XVs3cBIy6+4kK3O+S+afdx9h3apwP/uTlRMNaDSoitSdSqoGZPQTcBnSaWT/we0AU\nwN0/C+wE7gT6gCngfUtVbCUkM1n++JH9XLW+hTuvCsTrviIir1nJcHf3e0ocd+A3K1bREnvo+0fo\nH57mD975OkIhvYgqIrVpRc1JTCQz/Onjfbzh0lW8aWtntcsREVkyKyrcP/utlzk9meJ37/gRLX0U\nkZq2YsL9h0eGeeBbfbzzuvVc29NW7XJERJbUign3Lz55hKa6CP/7nVdVuxQRkSW3IsJ9Mpnhmy+c\n4B3XrKWhruRryCIigbciwn3Hs8eZSmV59w3d1S5FRGRZ1Hy4D0+m+MNvvsQ13a1cr+uiisgKUfPh\n/rc/OMLodJo/fNfVWiEjIitGTYd7Opvjb/7jEG/a2smVa1uqXY6IyLKp6XDf+fwJTo0led8tm6pd\niojIsqrpcP/r7x5ic2cjt12+utqliIgsq5oN9x8eGWb30RHed8smvYeMiKw4NRnu7s6nHj1AczzC\nu67X8kcRWXlqMtz/vW+If9s/yAfespXGmE5aEpGVp+bC3d35k0cPsLY1zi/dvLHa5YiIVEXNhft3\n+07z9OFh/sttlxKLhKtdjohIVdRUuLs7n3psP5e0xPnZ1+vaqCKyctVUuH9t9zGeOjTM+998mUbt\nIrKi1Uy4pzI5/uhf9nFNdyv33Lih2uWIiFRVzYT7zudPcHw0wW/dfjlhrWsXkRWurHA3szvMbJ+Z\n9ZnZfQsc32BmT5jZM2b2nJndWflSi3voB0fYuKqBW7d2LfdDi4hcdEqGu5mFgQeAtwHbgHvMbNu8\nZh8BvuLu1wHvAf680oUWc3Bwgu+/coafe32PzkYVEaG8kfuNQJ+7H3T3FPAl4O55bRyYedvFVuB4\n5Uos7bEXBwB0NqqISEE54b4eODpnu7+wb67/BfyimfUDO4H/utAdmdl2M9tlZrsGBwfPo9yFDU4k\niUdDrGmJV+w+RUSCrFIvqN4DfN7du4E7gS+Y2Tn37e4Punuvu/d2dVVubvzMZIqOhrqK3Z+ISNCV\nE+7HgLlnBHUX9s11L/AVAHf/DyAOdFaiwHIMT6Zob1S4i4jMKCfcnwK2mtlmM6sj/4LpjnltjgBv\nATCzK8mHe+XmXUo4M5WiQ+EuIjKrZLi7ewZ4P/Aw8CL5VTF7zOxjZnZXodkHgV8zs2eBh4BfcXdf\nqqLnG55M0a5pGRGRWWW9H6677yT/QuncfffPub0XuKWypZXv9KRG7iIicwX+DNV0Nsd4IqNwFxGZ\nI/DhPjKVBqCtIVrlSkRELh6BD/exRD7cW+sV7iIiMwIf7qPT+XBvUbiLiMwKfLiPzYR7XOEuIjIj\n8OE+M3LXtIyIyKsCH+5jiQwALfVlreoUEVkRgh/umpYRETlHTYR7LBIiHtU1U0VEZgQ/3BNpzbeL\niMwT+HAfmUprGaSIyDyBD/cTowku0UU6RETOUgPhPs26NoW7iMhcgQ73VCbHwHiSta311S5FROSi\nEuhwPzWWwB2N3EVE5gl0uB8fmQZgXZtG7iIicwU73EcV7iIiCwl0uPefyYf7eoW7iMhZAh3ux0am\n6WyK6exUEZF5Ah/u69s1ahcRma+scDezO8xsn5n1mdl9i7T5WTPba2Z7zOxvK1vmwvqHp+nWlIyI\nyDlKvk+umYWBB4DbgX7gKTPb4e5757TZCnwIuMXdh81s9VIVPCOXc46NTHP7tjVL/VAiIoFTzsj9\nRqDP3Q+6ewr4EnD3vDa/Bjzg7sMA7j5Q2TLPNTSZJJXJ0a1pGRGRc5QT7uuBo3O2+wv75rocuNzM\nvmtmT5rZHZUqcDH9w1opIyKymEpdvigCbAVuA7qBb5vZ69x9ZG4jM9sObAfYsGHDBT3gsZlw18hd\nROQc5YzcjwE9c7a7C/vm6gd2uHva3V8B9pMP+7O4+4Pu3uvuvV1dXedbc76oEY3cRUQWU064PwVs\nNbPNZlYHvAfYMa/N18iP2jGzTvLTNAcrWOc5zkymiEdDNOvyeiIi5ygZ7u6eAd4PPAy8CHzF3feY\n2cfM7K5Cs4eB02a2F3gC+B13P71URUP+8noKdhGRhZU15+7uO4Gd8/bdP+e2A79d+FgW44kMLfFK\nvWQgIlJbAnuG6lhCI3cRkcUEONwzunaqiMgiAhvu49NpTcuIiCwisOE+lshoWkZEZBGBDffxRJqW\neo3cRUQWEshwT2ayJDM5WjRyFxFZUCDDfTyRAaBZc+4iIgsKZLhPp7IANNQp3EVEFhLIcE9mcgBE\nw1blSkRELk6BDPd0Nh/usUggyxcRWXKBTMdUYeRep3AXEVlQINMxVRi514XDVa5EROTiFMxw15y7\niEhRwQz3rKZlRESKCWQ6as5dRKS4QKbjbLiHA1m+iMiSC2Q6auQuIlJcINMxrTl3EZGiApmOry6F\nDGT5IiJLLpDpOLsUUiN3EZEFlZWOZnaHme0zsz4zu69Iu3eZmZtZb+VKPFdSL6iKiBRVMh3NLAw8\nALwN2AbcY2bbFmjXDHwA+H6li5wvrWkZEZGiyknHG4E+dz/o7ingS8DdC7T7feDjQKKC9S0olckR\nCRmhkM5QFRFZSDnhvh44Ome7v7BvlpldD/S4+zcqWNuiUpmcVsqIiBRxwQlpZiHgj4EPltF2u5nt\nMrNdg4OD5/2YqazCXUSkmHIS8hjQM2e7u7BvRjNwFfAtMzsE3ATsWOhFVXd/0N173b23q6vrvItO\nZ3OabxcRKaKchHwK2Gpmm82sDngPsGPmoLuPununu29y903Ak8Bd7r5rSSomv1omqnAXEVlUyYR0\n9wzwfuBh4EXgK+6+x8w+ZmZ3LXWBC0llcroKk4hIEWVdYdrddwI75+27f5G2t114WcXpBVURkeIC\nmZBpvaAqIlJUIBMynXUiWuMuIrKoQIZ7zp2wwl1EZFGBDXczhbuIyGKCGe450MBdRGRxwQx3TcuI\niBQVyHDPuhPStIyIyKICGe45R+EuIlJEMMM955pzFxEpIpjhrjl3EZGiAhnu2ZyWQoqIFBPIcHeH\nsMJdRGRRgQz3rDuhQFYuIrI8AhmROkNVRKS4QIa7pmVERIoLZLhntRRSRKSoQIZ7zp2Q0l1EZFHB\nDPec3n5ARKSYYIa75txFRIoKZLhrKaSISHFlRaSZ3WFm+8ysz8zuW+D4b5vZXjN7zsweM7ONlS/1\nVa53hRQRKapkuJtZGHgAeBuwDbjHzLbNa/YM0OvuVwN/B3yi0oXOldWcu4hIUeWM3G8E+tz9oLun\ngC8Bd89t4O5PuPtUYfNJoLuyZZ4t5+iNw0REiign3NcDR+ds9xf2LeZe4JsXUlQp+TNUl/IRRESC\nLVLJOzOzXwR6gVsXOb4d2A6wYcOG834cLYUUESmunJH7MaBnznZ3Yd9ZzOytwIeBu9w9udAdufuD\n7t7r7r1dXV3nUy+gaRkRkVLKCfengK1mttnM6oD3ADvmNjCz64C/IB/sA5Uv82xZTcuIiBRVMtzd\nPQO8H3gYeBH4irvvMbOPmdldhWZ/BDQBXzWz3Wa2Y5G7qwh310lMIiJFlDXn7u47gZ3z9t0/5/Zb\nK1xXUVoKKSJSXCDP88w5euMwEZEiAhfuuZwD6C1/RUSKCF64ez7cNecuIrK4wIV7thDumpYREVlc\n4MK9kO1aCikiUkTgwl3TMiIipQUu3LOzL6gq3EVEFhO4cC9ku+bcRUSKCF64aymkiEhJwQv3mTl3\npbuIyKICF+4zSyFNc+4iIosKXLjPLIXUahkRkcUFLtyzmnMXESkpcOGe0xmqIiIlBS7cZ6ZltM5d\nRGRxgQt3TcuIiJQWuHDXUkgRkdICG+5aCikisrgAhnv+s5ZCiogsLnDhrjl3EZHSAhfuWgopIlJa\nWeFuZneY2T4z6zOz+xY4HjOzLxeOf9/MNlW60Bm5XP6zlkKKiCyuZLibWRh4AHgbsA24x8y2zWt2\nLzDs7pcBnwQ+XulCZ7y6WmapHkFEJPjKicgbgT53P+juKeBLwN3z2twN/N/C7b8D3mJLtJxFq2VE\nREorJ9zXA0fnbPcX9i3Yxt0zwCiwav4dmdl2M9tlZrsGBwfPq+DZOXeFu4jIopZ1csPdH3T3Xnfv\n7erqOq/70FJIEZHSygn3Y0DPnO3uwr4F25hZBGgFTleiwPm0FFJEpLRywv0pYKuZbTazOuA9wI55\nbXYA7y3cfjfwuPvMW3xVlpZCioiUFinVwN0zZvZ+4GEgDHzO3feY2ceAXe6+A/gr4Atm1gecIf8H\nYEloKaSISGklwx3A3XcCO+ftu3/O7QTwM5UtbWFaCikiUlrgIlLXUBURKS1w4T4zla/VMiIiiwtc\nuGc15y4iUlLgwv3V1TJVLkRE5CIWuIh0naEqIlJS4MJd0zIiIqUFLtwvaY3x9tetpaW+rFWcIiIr\nUuAS8oaNHdywsaPaZYiIXNQCN3IXEZHSFO4iIjVI4S4iUoMU7iIiNUjhLiJSgxTuIiI1SOEuIlKD\nFO4iIjXIluhqeKUf2GwQOHyeX94JDFWwnCBQn1cG9XlluJA+b3T3rlKNqhbuF8LMdrl7b7XrWE7q\n88qgPq8My9FnTcuIiNQghbuISA0Karg/WO0CqkB9XhnU55VhyfscyDl3EREpLqgjdxERKSJw4W5m\nd5jZPjPrM7P7ql1PpZjZ58xswMxemLOvw8weMbMDhc/thf1mZp8ufA+eM7Prq1f5+TOzHjN7wsz2\nmtkeM/tAYX/N9tvM4mb2AzN7ttDnjxb2bzaz7xf69mUzqyvsjxW2+wrHN1Wz/vNlZmEze8bM/rmw\nXdP9BTCzQ2b2vJntNrNdhX3L9twOVLibWRh4AHgbsA24x8y2Vbeqivk8cMe8ffcBj7n7VuCxwjbk\n+7+18LEd+Mwy1VhpGeCD7r4NuAn4zcLPs5b7nQTe7O7XANcCd5jZTcDHgU+6+2XAMHBvof29wHBh\n/ycL7YLoA8CLc7Zrvb8zfsLdr52z7HH5ntvuHpgP4Gbg4TnbHwI+VO26Kti/TcALc7b3AWsLt9cC\n+wq3/wK4Z6F2Qf4A/gm4faX0G2gAfgj8GPkTWiKF/bPPc+Bh4ObC7UihnVW79tfYz+5CkL0Z+GfA\narm/c/p9COict2/ZntuBGrkD64Gjc7b7C/tq1Rp3P1G4fRJYU7hdc9+Hwr/f1wHfp8b7XZii2A0M\nAI8ALwMj7p4pNJnbr9k+F46PAquWt+IL9ifA7wKFy9uzitru7wwH/tXMnjaz7YV9y/bcDtw1VFcq\nd3czq8mlTWbWBPw98FvuPmZms8dqsd/ungWuNbM24B+BH6lySUvGzN4BDLj702Z2W7XrWWZvdPdj\nZrYaeMTMXpp7cKmf20EbuR8DeuZsdxf21apTZrYWoPB5oLC/Zr4PZhYlH+xfdPd/KOyu+X4DuPsI\n8AT5aYk2M5sZbM3t12yfC8dbgdPLXOqFuAW4y8wOAV8iPzXzKWq3v7Pc/Vjh8wD5P+I3sozP7aCF\n+1PA1sIr7XXAe4AdVa5pKe0A3lu4/V7yc9Iz+3+58Ar7TcDonH/1AsPyQ/S/Al509z+ec6hm+21m\nXYURO2ZWT/41hhfJh/y7C83m93nme/Fu4HEvTMoGgbt/yN273X0T+d/Xx939F6jR/s4ws0Yza565\nDfwk8ALL+dyu9osO5/EixZ3AfvLzlB+udj0V7NdDwAkgTX6+7V7yc42PAQeAR4GOQlsjv2roZeB5\noLfa9Z9nn99Ifl7yOWB34ePOWu43cDXwTKHPLwD3F/ZvAX4A9AFfBWKF/fHCdl/h+JZq9+EC+n4b\n8M8rob+F/j1b+Ngzk1XL+dzWGaoiIjUoaNMyIiJSBoW7iEgNUriLiNQghbuISA1SuIuI1CCFu4hI\nDVK4i4jUIIW7iEgN+v/Dt/2dCMY9vAAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sess= tf.Session()\n", "sess.run(tf.global_variables_initializer())\n", "\n", "target_tract = []\n", "for i in range(500):\n", " sample1 = sampling(100, pd1)\n", " sample2 = sampling(100, pd3)\n", " sess.run(train_op, feed_dict = {X1 : sample1, X2 : sample2} )\n", " sess.run(clip_op)\n", " target_tract.append(sess.run(target, feed_dict = {X1 : sample1, X2 : sample2}))\n", "\n", "plt.plot(target_tract)\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "'''\n", "i=0 \n", "while True:\n", " try:\n", " temp = tf.trainable_variables()[i]\n", " print(\"{} : {}\".format(temp.op.name, sess.run(temp)))\n", " i+=1\n", " except IndexError:\n", " break\n", "'''" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(200,)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4FGXXx/HvAUJH6UWqChbgVdAAIjwKShekiRRBFBAV\nsItdH7sUu9gQFRCQIiC9ClIVEpCOIFWIoYNUCSTn/WMGnzUmZBM2mS3nc117ZXfqubO7v529Z3ZG\nVBVjjDGRI5vXBRhjjMlaFvzGGBNhLPiNMSbCWPAbY0yEseA3xpgIY8FvjDERxoI/g0TkShFZJSLH\nROThLFxvORE5LiLZM2HZdUTkN3f5rS5gOSoiFVMZd5eIzPZjGS+LyIiM1pBZROQzEXkxg/M+JyJD\nAl3Tedb3oIjsdZ/PIu6wbzP63KbnORGRh0Skf0bW47OMEiKy0H2PveMOqywisSIiGVxmqq/NZNPl\nEpFfRaRYRtYT7Cz4M+4pYL6qFlDVDzNrJSKyQ0QanHusqr+ran5VTcyE1b0KDHKX/30mLB9VHamq\njTJj2VlBVR9Q1dcyOO+bqtoj0DWlRESigHeBRu7zeVBErgGuBSYFYPlXiMgkEdkvIodEZJaIXOkz\nyRfAXSJS/AJW0xM4AFykqk+4w14D3tYA/ABJROqLyHwR+VNEdviOU9XTwFfAMxe6nmBkwZ9x5YH1\nXhcRYOHYpkhVAsjNP5/P+4GRgQhNoCAwGbjSXddyfD5QVPUvYAZw9wWsozyw4Vy9IlIKqA8EaqPk\nBE64901l/Cigq4jkCtD6goeq2i2dN2AekAj8BRwHrgB+BHr4THMPsNjnsQIPAL8BR4CPAfEZfx+w\nETgGbACuA74BkoBT7nqeAiq4y8rhzncJzhvwELAFuM9nmS8DY4Hh7nLXA9GptGlrsnXlSjb+XmCK\nz+PfgHE+j3cB1dJqawr/lyrAHLf+vcBz6a3dZ5293HUew9kyvBxYChx1l5XTnbYQMBXYDxx275dx\nxxUGdgMt3Mf53f/r3e7jocDr7v167rRPAfuAeKAV0AzY7LbpuWTPxwj3/iD3/3zudhZ42ec5He/W\ntx142GcZNYFYt017gXdT+F9cgRNq6i57njt8G1A32Wt0iVvLn8CvwK0+4y8FFrj/zznudCNS+f8X\ndtdXxGfYXTjfijPyHhsKnAES3DY0wPkQmZtsuh3AszjvmcPA10Bun/F93eflD6CbW2PFZMtoAOxI\npY7fgJu9zpxA3zwvIFRv/Dvokz++h38H/1ScLaVy7pu6iTuuHRAH1AAEqAiUd8ftABr4LKcC/wz+\nhcAnOFt31dzl3uKOexnnw6kZkB14C/j5PG36x7qSjbsMJ8Sz4QTTTmC3z7jDQDY/2vr3/wUo4L4p\nn3DrLwDUymDtirPFeRHOh8lp4Ae3tovdYOjqTlsEaAvkddc5DvjeZ1mNgD1AcZwui+98xg3ln8F/\nFngJiML58N6Ps6VYwK3jFHCpT5v+FZw+z1t19/+7wl1mTrf+bUBjd9qfgC7u/fzADan8P5K/TvK5\nj4sle42eBR5z62+P8wFQ2Gdd7wK5gJtwPgBSC/5WQHyyYdcBhy7gPfb3/9p9PBD4OIXX7DqgLM6H\nzxKf56cJzodjVbf9o0h/8E/G54M3XG7W1ZO1+qnqEVX9HZiP84YH6AEMUNUYdWxR1Z1pLUxEygJ1\ngKdV9S9VXQUM4Z9frxer6nR19gl8g9PHm26qug3njV8NJwRmAX+IyFXAzcAiVU3yo62+mgN7VPUd\nt/5jqrrsAmofoKpHVXU9ThjMVtVtqvonTrdDdbctB1V1vKqeVNVjwBtuG861dTbOh8EPOB88959n\nnWeAN1T1DDAaKAp84LZlPc4HTqp1uzsPvwceUtVfcD78i6nqq6qa4P7fvwA6+KyvoogUVdXjqvpz\nGv+Tcwq6f48lG74PeF9Vz6jqGGATcJuIlHNreVFVT6vqQmBKKm0og/Ot7vFko47hfOgGSsEU6gdn\nv9QuVT2E81x2dIffCXytqutU9QTOB296HeN//7uwYcGftfb43D+Js8UGztbK1gws7xKcLSrfN8NO\noPR51plbRHJkYF3gfO2vhxP8C3C+5dzs3hYkmza1tvpKq93prX2vz/1TKTzODyAieUXkcxHZKSJH\ncb41FUx2pNRgnC3Foap68DzrPKj/29F+KpU6Umr7uR2w3wGjVHW0O7g8cImIHDl3A57D6UcH6I7T\nlfOriMSISPPz1ObriPu3QLLhcepu2rp24ryuLgEOu4HpOy55G4oBs4FPVPXbZKML4HyD+Bf3CKfj\n7u0zP9twOIX6welmTF4/7t/k49KrAP/734UNC/7AOYHTdXBOyXTMuwunPzol59sR9wdQWER83wzl\ncLqNMsO54P+Pe38BqQe/P3bhdGVktSdwdkrWUtWLcD7IwOlmw/0AGIyzf6GXP4f/ZdBHOH31L/gM\n2wVsV9WCPrcCqtoMQFV/U9WOON1Q/YHvRCRfWityA3wrzoeGr9LJDo0sh/O6igcKJVt2Od8ZRaQQ\nTuhPVtU3Uljt1cDqVOp5U52jjfKr6gNp1e9ak0L94GxAJK8ftw3Jx6VXqm0IZRb8gbMKaONuTVbE\n2TLz1xDgSRG5XhwVRaS8O24vqYSjqu7C2Xn5lojkdg/X6w5k1vHvC3COqsijqruBRTj9qEWAXzKw\nvKlAKRF51D1uuoCI1ApcuakqgLMlfkRECgP/TTb+OZwP3G44/crDA/27CRG5H+cD865kXWTLgWMi\n8rSI5BGR7CJSVURquPN1FpFi7jzntkST8M90fLq0XMWBh0UkSkTa4QTddLerMRZ4RURyikhdoIVP\n/RfhdPctUdXUDnm8GaeLLVDmANeJSO5kw3uLSBn3uXweGOMOHwvc4x77n5dkz7OIZHOXFeU8lNwi\nktNnfGmc/Qb+dqeFDAv+wHkP5wiEvcAwYKS/M6rqOJy+yVE4fYrf47zgwNmp+YL7tf/JFGbviLMj\n7w9gIvBfVZ2bwTakVedmnCMsFrmPj+LseFyiGfhdgdtF1RAnUPbgHEFRP2AFp+59IA/OMeI/AzPP\njRCR63H6qu9229Qf50Mg0Mdzd8T5QP/Dp8vjOXedzXH2iWx3axzC//rKmwDrReQ48AHQQVVP/Xvx\nKRqMc2y97xb+MqCSu543gDt8urY6AbVwjk76L843oHNa4+wDuNen/uPuvgHcQG2G814ICFXdi3NE\nXctko0bhfPPYhvOt5nV3+hk4z/U8nCOz5iWb7yacDYDpON8GTrnLOacTMEydY/rDyrlD7IwxEUBE\nRgFjVfV7EbkH50i0upmwnoeAsqr6VICXWxnnw6Smqqr7w6segd7YcY/dXw3cpKr7ArnsYJDRnXzG\nmBCkqp2yaD0fZdJyN+B808hU7lb+VZm9Hq9YV48xxkQY6+oxxpgIY1v8xhgTYYKyj79o0aJaoUIF\nr8swxpiQsWLFigOq6tdppIMy+CtUqEBsbKzXZRhjTMgQEb9/mWxdPcYYE2Es+I0xJsJY8BtjTISx\n4DfGmAhjwW+MMRHGgt8YYyKMBb8xxkSYNI/jd0+vuhDnups5cK4/mvy81o/jXD7wLM61Q7udu3Sg\niCQCa91Jf1fV2wNXvjHGhK4ziWfY+edOthzawpZDWzh55iRP1QnoCU1T5M8PuE7jXLz7uHupuMUi\nMiPZtT5/AaJV9aSIPAgMwLlwM8ApVU3peqvGGBP2ziadZdvhbWw+uPnvgD9323FkB4k+l7Iomb8k\nfW/syz8vmRB4aQa/ez3O4+7DKPemyaaZ7/PwZ6BzoAo0xphQcDbpLFsPbWX9/vVs2L/h77+bDmzi\ndOL/ruVyUa6LqFS4EtGXRNOhagcqFq74961EvhKZHvrg5ykb3MvOrQAqAh+r6rLzTN6df15uLbeI\nxOJ0A/VT1e8zWqwxxgSDQ6cO8Uv8L6yMX8kve35h3b51bDq4iYTEhL+nqVCwApWLVabx5Y2pXKwy\nVxa5kkpFKlEkT5EsCffz8Sv43cvBVRORgsBEEamqquuSTycinYFo/nldz/KqGicilwHzRGStqm5N\nYd6eQE+AcuUyck1kY4wJvH0n9hETF/N3yK+MX8nOP/93WpyyF5XlmhLX0LRiUyoXq0yV4lW4quhV\n5M+Z38Oqzy/d5+MXkZeAk6r6drLhDYCPgJtTu1SZiAwFpqrqd+dbR3R0tNpJ2owxWe1M4hnW7F3D\nT7t/cm67fmL7ke1/j7+iyBVcV+o6qpesznWlrqNayWoUzVvUw4r/R0RWqGq0P9P6c1RPMeCMqh4R\nkTw4F8fun2ya6sDnQBPf0BeRQjgfEqdFpChQB2fHrzHGeO7Y6WMs+n0RC3Ys4KfdPxH7RyynzjrX\nri+VvxS1y9amV41e1Cpdi2olq1EgVwGPKw4Mf7p6SgHD3H7+bDgXap4qIq8Csao6GRgI5AfGuX1X\n5w7bvBr4XESS3Hn7udfMNMaYLHci4QSLf1/Mjzt+ZP6O+cT+EUuiJhKVLYrqparT8/qe1C5Tm9pl\na1P2orKe98VnlqC89KJ19RhjAiExKZHlccuZsWUGP2z/geVxyzmbdJYc2XJQs3RN6leoT/0K9ald\ntjZ5o/J6Xe4FCWhXjzHGhJIDJw8wa8sspm+ZzswtMzl06hDZJBs1LqnBk7WfpF6FetQpVyeod75m\nNgt+Y0xIU1VW713N5E2Tmf7bdJbHLUdRiuUtRvMrmtOsYjMaXt6QwnkKe11q0LDgN8aEnCRNIiYu\nhvEbxzN+43i2Hd6GINQsXZOX671M04pNuf6S68kmdjqylFjwG2NCgqqydNdSxq4fy4RfJ7D76G6i\nskVx62W38mzdZ7n9ytspnq+412WGBAt+Y0xQ27B/AyPXjGTUulHsOLKD3Dly06RiE9669S2aX9Gc\ngrkLel1iyLHgN8YEnfhj8YxaO4qRa0fyy55fyCbZaHhZQ16r/xqtrmoV0TtmA8GC3xgTFBKTEpmx\nZQZfrPyCaZunkaiJ1LikBu83fp/2VdtTMn9Jr0sMGxb8xhhP7Tyyky9/+ZKvfvmKuGNxFM9XnCdq\nP0G36t24suiVXpcXliz4jTFZLkmTmL11Nh8u+5CZW2YC0LhiYz5s+iEtrmhBVPYojysMbxb8xpgs\nczzhOMNXD+fDZR+y6eAmSuYvyQs3vUD36t0pX7C81+VFDAt+Y0ym23FkB4OWD2LIyiH8efpPalxS\ngxGtR9CuSjtyZs/pdXkRx4LfGJNp1u5dS78l/Ri9bjSCcEflO3ik1iPcUOaGsD0BWiiw4DfGBNzP\nu3/mzUVvMmXzFPJF5eOxGx7j0RsepcxFZbwuzWDBb4wJEFVl7ra5vLn4TX7c8SOF8xTmlXqv0Kdm\nHztPTpCx4DfGXLCFOxfy/LznWfz7YkoXKM27jd7lvuvvsx9aBSkLfmNMhsXExfD8vOeZs20OpfKX\n4uNmH9O9endy5cjldWnmPCz4jTHptnbvWl6c/yKTNk2iSJ4ivN3wbXrV6EWeqDxel2b8YMFvjPFb\n/LF4np/3PENXDaVArgK8Wu9VHr3h0bC5Fm2ksOA3xqTp5JmTvLP0Hfov6U9CYgKP136c5/7znO20\nDVFpBr+I5AYWArnc6b9T1f8mmyYXMBy4HjgItFfVHe64Z4HuQCLwsKrOCmQDjDGZJ0mT+Hbttzzz\nwzPsPrqbNle3YUCDAVxe+HKvSzMXwJ8t/tPALap6XESigMUiMkNVf/aZpjtwWFUrikgHoD/QXkQq\nAx2AKsAlwFwRuUJVEwPcDmNMgMXExdBnRh+Wxy3nulLXMbLNSG4qf5PXZZkASPO6ZOo47j6Mcm+a\nbLKWwDD3/nfAreL8LK8lMFpVT6vqdmALUDMglRtjMsXhU4fpNa0XtYbUYtefuxjacigx98VY6IcR\nv/r4RSQ7sAKoCHysqsuSTVIa2AWgqmdF5E+giDvc95vBbndYSuvoCfQEKFeuXDqaYIwJBFVl5NqR\nPDH7CQ6cPMDDtR7m1fqvclGui7wuzQSYX8Hvds1UE5GCwEQRqaqq6wJZiKoOBgYDREdHJ/9GYYzJ\nRBv3b6TX9F78uONHapWuxcy7ZlK9VHWvyzKZJF1H9ajqERGZDzQBfIM/DigL7BaRHMDFODt5zw0/\np4w7zBgTBBISE3hj4Ru8tfgt8uXMx2e3fcZ9199HNkmzF9iEsDSfXREp5m7pIyJ5gIbAr8kmmwx0\nde/fAcxTVXWHdxCRXCJyKVAJWB6o4o0xGbcyfiU1vqjBqwtfpV2Vdmzqs4n7o++30I8A/mzxlwKG\nuf382YCxqjpVRF4FYlV1MvAl8I2IbAEO4RzJg6quF5GxwAbgLNDbjugxxlunz57mtYWv0W9xP4rn\nK87kDpNpcWULr8syWUicDfPgEh0drbGxsV6XYUzYiYmL4d5J97J+/3q6XtuV9xq/R6E8hbwuywSA\niKxQ1Wh/prVf7hoTAc4knuGVBa/w1uK3KJW/FNM6TaNZpWZel2U8YsFvTJjbcmgLncZ3IuaPGO6p\ndg/vNX6PgrkLel2W8ZAFvzFhSlUZumooD814iKjsUYy9YyztqrTzuiwTBCz4jQlDh08d5v6p9zNu\nwzjqVajH8FbDKXtx2bRnNBHBgt+YMLNgxwI6T+zMnuN76HdrP5688UmyZ8vudVkmiFjwGxMmkjSJ\n/ov788L8F7i80OX81P0noi/x6yAPE2Es+I0JA4dOHaLLxC5M/206Hap2YHDzwXZxFJMqC35jQtzy\nuOXcOe5O/jj2B4OaDqJXjV44J8c1JmX222xjQpSqMmj5IOp+VReAJd2W0Ltmbwt9kybb4jcmBJ1I\nOEGPKT0YvW40za9ozrBWw+wyiMZvFvzGhJidR3bSakwrVu9ZzZu3vMnTdZ+2E6uZdLHgNyaELNy5\nkLZj23Im8QxTO0210y6YDLHNBGNCgKryacyn3Dr8VorkKcKyHsss9E2G2Ra/MUEuITGBh6Y/xOCV\ng2lWqRmj2ozi4twXe12WCWEW/MYEsQMnD9B6TGsW/76YZ+s+y2v1X7Nf4ZoLZsFvTJDafHAzzUY2\nY/fR3YxqM4qO/9fR65JMmLDgNyYILdq5iFZjWpFNsjGv6zxuLHuj1yWZMGI7d40JMiPXjKTBNw0o\nlrcYP3f/2ULfBJwFvzFBQlV5bcFrdJ7YmdplarO0+1IuL3y512WZMJRmV4+IlAWGAyUABQar6gfJ\npukL3OWzzKuBYqp6SER2AMeAROCsv9eENCaSnEk8w31T7mPY6mHcfe3dfNHiC3Jmz+l1WSZM+dPH\nfxZ4QlVXikgBYIWIzFHVDecmUNWBwEAAEWkBPKaqh3yWUV9VDwSycGPCxfGE49wx9g5mbZ3Fyze/\nzEs3v2Tn2zGZKs3gV9V4IN69f0xENgKlgQ2pzNIR+DZgFRoTxg6cPMBto24j9o9YhrQYQvfruntd\nkokA6erjF5EKQHVgWSrj8wJNgPE+gxWYLSIrRKTneZbdU0RiRSR2//796SnLmJC088hO6n5VlzV7\n1zCx/UQLfZNl/D6cU0Ty4wT6o6p6NJXJWgBLknXz1FXVOBEpDswRkV9VdWHyGVV1MDAYIDo6Wv1u\ngTEhaO3etTQZ2YSTZ04yp8sc6par63VJJoL4tcUvIlE4oT9SVSecZ9IOJOvmUdU49+8+YCJQM2Ol\nGhMeFu1cxH++/o9z/95FFvomy6UZ/OLsZfoS2Kiq755nuouBm4FJPsPyuTuEEZF8QCNg3YUWbUyo\nmv7bdBqNaETJ/CVZ2m0pVYtX9bokE4H86eqpA3QB1orIKnfYc0A5AFX9zB3WGpitqid85i0BTHSP\nUMgBjFLVmYEo3JhQ892G7+g0vhPXlLiGmZ1nUjRvUa9LMhHKn6N6FgNpHlumqkOBocmGbQOuzWBt\nxoSN4auHc++ke6ldpjbTOk2zs2saT9kvd43JZJ/GfErX77tyy6W3MKvzLAt94zkLfmMy0dtL36bX\n9F60uKIFUzpOIV/OfF6XZIwFvzGZQVV5+ceX6TunL+2rtGf8nePJnSO312UZA9hpmY0JOFWl75y+\nvPPTO9xT7R6GtBhiF08xQcWC35gAStIk+kzvw6exn9K7Rm8+bPoh2cS+WJvgYq9IYwIkSZPoPa03\nn8Z+St8b+/JR048s9E1QslelMQFwLvQ/W/EZT9d5mv4N+tsZNk3QsuA35gIlD/23bn3LQt8ENQt+\nYy6Ahb4JRRb8xmSQhb4JVRb8xmSAhb4JZRb8xqSThb4JdRb8xqSDqvLwjIct9E1Is+A3xk+qylNz\nnuLjmI95svaTFvomZFnwG+OnVxa8wts/vU3vGr0Z0HCAhb4JWRb8xvih/+L+vLLgFbpV68aHTT+0\n0DchzYLfmDR8tOwjnvnhGTpW7cjgFoPtNAwm5Nkr2JjzGLJyCA/PfJhWV7ViWKthdpZNExb8udh6\nWRGZLyIbRGS9iDySwjT1RORPEVnl3l7yGddERDaJyBYReSbQDTAms4xcM5KeU3rSpGITRrcdTVT2\nKK9LMiYg/Dkt81ngCVVdKSIFgBUiMkdVNySbbpGqNvcdICLZgY+BhsBuIEZEJqcwrzFBZfyG8XT9\nviv1KtRjwp0TyJUjl9clGRMwaW7xq2q8qq507x8DNgKl/Vx+TWCLqm5T1QRgNNAyo8UakxWmbZ5G\nx/EdqVm6JpM7TiZPVB6vSzImoNLVxy8iFYDqwLIURtcWkdUiMkNEqrjDSgO7fKbZTSofGiLSU0Ri\nRSR2//796SnLmID5YdsPtB3blmtKXMOMu2aQP2d+r0syJuD8Dn4RyQ+MBx5V1aPJRq8EyqvqtcBH\nwPfpLURVB6tqtKpGFytWLL2zG3PBlu1eRsvRLalUpBKzOs/i4twXe12SMZnCr+AXkSic0B+pqhOS\nj1fVo6p63L0/HYgSkaJAHFDWZ9Iy7jBjgsq6fetoOrIpJfKXYHbn2RTJW8TrkozJNP4c1SPAl8BG\nVX03lWlKutMhIjXd5R4EYoBKInKpiOQEOgCTA1W8MYGw/fB2Gn3TiNw5cjO3y1xKFSjldUnGZCp/\njuqpA3QB1orIKnfYc0A5AFX9DLgDeFBEzgKngA6qqsBZEekDzAKyA1+p6voAt8GYDIs/Fk+Dbxpw\nOvE0C+9ZyKWFLvW6JGMyXZrBr6qLgfP+Pl1VBwGDUhk3HZieoeqMyUSHTx2m8YjG7D2+lx/u/oEq\nxaukPZMxYcCfLX5jws6JhBPcNuo2Nh3cxLRO06hVppbXJRmTZSz4TcQ5ffY0bca2YVncMsa1G0eD\nyxp4XZIxWcqC30SUxKREOk/szOyts/ny9i9pc3Ubr0syJsvZSdpMxFBVHpj6AN9t+I53Gr1Dt+rd\nvC7JGE9Y8JuI8czcZxjyyxCe/8/zPF77ca/LMcYzFvwmIvRf3J8BSwfwYPSDvFb/Na/LMcZTFvwm\n7A1eMfjvC6kMajbIrp5lIp4FvwlrY9aN4YGpD9CsUjOGtRpmV88yBgt+E8ZmbplJl4ldqFOuDuPa\njbMLqRjjsuA3YWnprqW0GdOGKsWrMKXjFPJG5fW6JGOChgW/CTtr9q7htlG3UeaiMszqPIuCuQt6\nXZIxQcWC34SVrYe20nhEY/JF5WNOlzkUz1fc65KMCTr2y10TNuKPxdNoRCMSEhNYdO8iyhcs73VJ\nxgQlC34TFg6fOkyjEY3Ye3wv87rOo3Kxyl6XZEzQsuA3Ie/cmTY3H9zMtE7TqFm6ptclGRPULPhN\nSEtITKDt2LYsi1vG2DvG2pk2jfGDBb8JWYlJidw98W5mbZ3FFy2+oG3ltl6XZExIsKN6TEhSVfpM\n78OY9WPo36A/Pa7r4XVJxoQMC34Tkl6a/xKfrfiMp258iqfqPOV1OcaElDSDX0TKish8EdkgIutF\n5JEUprlLRNaIyFoRWSoi1/qM2+EOXyUisYFugIk87//8Pq8vep0e1XvQr0E/r8sxJuT408d/FnhC\nVVeKSAFghYjMUdUNPtNsB25W1cMi0hQYDPhexLS+qh4IXNkmUg1fPZzHZj1Gm6vb8Fnzz+xMm8Zk\nQJrBr6rxQLx7/5iIbARKAxt8plnqM8vPQJkA12kMkzdNptukbtx66a2MajOK7Nmye12SMSEpXX38\nIlIBqA4sO89k3YEZPo8VmC0iK0Sk53mW3VNEYkUkdv/+/ekpy0SABTsWcOe4O7mu1HVMbD+RXDly\neV2SMSHL78M5RSQ/MB54VFWPpjJNfZzgr+szuK6qxolIcWCOiPyqqguTz6uqg3G6iIiOjtZ0tMGE\nuZXxK2nxbQsuK3QZM+6aQYFcBbwuyZiQ5tcWv4hE4YT+SFWdkMo01wBDgJaqevDccFWNc//uAyYC\n9rNK47fNBzfTZEQTCuUpxOwusymSt4jXJRkT8vw5qkeAL4GNqvpuKtOUAyYAXVR1s8/wfO4OYUQk\nH9AIWBeIwk342/XnLhp+0xCAOV3mUOYi23VkTCD409VTB+gCrBWRVe6w54ByAKr6GfASUAT4xD3K\n4qyqRgMlgInusBzAKFWdGdAWmLC078Q+Go1oxJG/jvBj1x+5osgVXpdkTNjw56iexcB5j5lT1R7A\nv346qarbgGv/PYcxqTvy1xEaj2jMziM7mdl5JtVLVfe6JGPCip2rxwSV4wnHaTqyKRv2b2Byh8nc\nVP4mr0syJuxY8Jug8dfZv2g5uiUxcTGMazeOxhUbe12SMWHJgt8EhTOJZ2g3rh3zts9jeKvhtL66\ntdclGRO27CRtxnOJSYl0mdiFqZun8kmzT+hybRevSzImrFnwG08laRI9p/RkzPoxDGgwgAdrPOh1\nScaEPQt+4xlV5fFZj/PVqq948aYX6Vunr9clGRMRLPiNZ16a/xIfLPuAR2s9yiv1XvG6HGMihgW/\n8cSAJQP+Pqf+u43ftdMrG5OFLPhNlnv/5/d5eu7TdKjawc6pb4wHLPhNlhq0fBCPzXqMtle3ZXir\n4XZOfWM8YMFvssznsZ/z0IyHaHllS75t+y1R2aO8LsmYiGTBb7LElyu/5IFpD9D8iuaMbTfWQt8Y\nD1nwm0w3bNUw7ptyH00qNuG7dt+RM3tOr0syJqJZ8JtMNWLNCO6ddC+3XnYrE+6cYJdMNCYIWPCb\nTDN63WijfLfZAAAPaklEQVS6ft+VehXqManDJPJE5fG6JGMMFvwmk4xbP47OEzpTt1xdpnScQt6o\nvF6XZIxxWfCbgBu1dhQdxnfghjI3MLXjVPLlzOd1ScYYHxb8JqCGrRpG5wmduan8TczsPJMCuQp4\nXZIxJhl/LrZeVkTmi8gGEVkvIo+kMI2IyIciskVE1ojIdT7juorIb+6ta6AbYILHFyu++HtH7rRO\n08ifM7/XJRljUuDPhVjOAk+o6koRKQCsEJE5qrrBZ5qmQCX3Vgv4FKglIoWB/wLRgLrzTlbVwwFt\nhfHcJzGf0Ht6b5pWbMqE9hPInSO31yUZY1KR5ha/qsar6kr3/jFgI1A62WQtgeHq+BkoKCKlgMbA\nHFU95Ib9HKBJQFtgPPf+z+/Te3pvbr/ydia2n2ihb0yQS1cfv4hUAKoDy5KNKg3s8nm82x2W2vCU\nlt1TRGJFJHb//v3pKct4aMCSAX+fe2dcu3F2nL4xIcDv4BeR/MB44FFVPRroQlR1sKpGq2p0sWLF\nAr14kwleX/j632fZHH3HaPtFrjEhwq/gF5EonNAfqaoTUpgkDijr87iMOyy14SaEqSpPz3maF+e/\nSJdrujCi9QhyZPNnd5ExJhj4c1SPAF8CG1X13VQmmwzc7R7dcwPwp6rGA7OARiJSSEQKAY3cYSZE\nJSYlcv/U+xmwdAAPXP8AX7f82k6tbEyI8WczrQ7QBVgrIqvcYc8B5QBU9TNgOtAM2AKcBO51xx0S\nkdeAGHe+V1X1UODKN1kpITGBzhM6M27DOJ6r+xyv3/K6XUTFmBCUZvCr6mLgvO9uVVWgdyrjvgK+\nylB1JmicSDhB27FtmbV1FgMbDuTJG5/0uiRjTAZZx6xJ0+FTh2n+bXN+3v0zQ1oMoft13b0uyRhz\nASz4zXnFH4unycgmbNy/kTF3jOGOynd4XZIx5gJZ8JtU/XrgV5qMaML+k/uZ1mkaDS9v6HVJxpgA\nsOA3KVry+xJuH307ObLlYME9C4i+JNrrkowxAWJn5zT/MnHjRBp804AieYrwU/efLPSNCTMW/OYf\nPl7+MW3HtuXaEteypNsSLit0mdclGWMCzILfAJCkSTwz9xn6zOhDiytbMK/rPIrls1NnGBOOrI/f\ncOrMKe6ddC9j1o/h/uvvZ1CzQXYKBmPCmL27I1z8sXhajWlFTFwM/W7tx1N1nrJf4xoT5iz4I9iq\nPato8W0LDp06xIT2E2h1VSuvSzLGZAHr449Qk36dRN2v6gKwpNsSC31jIogFf4RRVQYsGUDrMa2p\nUrwKy3ssp1rJal6XZYzJQtbVE0FOnTnF/VPv55s139C+Snu+bvk1eaLyeF2WMSaLWfBHiB1HdtB6\nTGtW71nNq/Ve5YWbXrCduMZEKAv+CDBn6xw6jO9AkiYxtdNUmlVq5nVJxhgPWR9/GFNV+i3uR5OR\nTShdoDQx98VY6BtjbIs/XB07fYx7Jt3DhI0TaF+lPV/e/iX5cubzuixjTBCw4A9Dq/asov137dl6\naCvvNHqHx254zPrzjTF/s+API6rKp7Gf8visxymcpzBz755LvQr1vC7LGBNk0gx+EfkKaA7sU9Wq\nKYzvC9zls7yrgWLuhdZ3AMeAROCsqtr5fTPJkb+O0GNyD8ZvHE/Tik0Z1mqYnWTNGJMif3buDgWa\npDZSVQeqajVVrQY8CyxQ1UM+k9R3x1voZ5Jlu5dR/fPqTNo0iQENBjC101QLfWNMqtIMflVdCBxK\nazpXR+DbC6rI+C0xKZGBSwZS9+u6qCqL7l1E3zp9ySZ2sJYxJnUB6+MXkbw43wz6+AxWYLaIKPC5\nqg4+z/w9gZ4A5cqVC1RZYWvHkR10/b4rC3cupM3VbRjSYgiF8hTyuixjTAgI5M7dFsCSZN08dVU1\nTkSKA3NE5Ff3G8S/uB8KgwGio6M1gHWFFVVl6KqhPDLzEQCGthzK3dfebUftGGP8Fsjg70Cybh5V\njXP/7hORiUBNIMXgN2nbd2IfPaf0ZNKmSdSrUI+hLYdSvmB5r8syxoSYgHQGi8jFwM3AJJ9h+USk\nwLn7QCNgXSDWF4m+//V7qn5SlZlbZvJOo3f44e4fLPSNMRniz+Gc3wL1gKIishv4LxAFoKqfuZO1\nBmar6gmfWUsAE90uiBzAKFWdGbjSI8Oe43t4eMbDjNswjmolqzGv9TyqFv/XUbXGGOO3NINfVTv6\nMc1QnMM+fYdtA67NaGGRTlX5etXXPDH7CU6dOcUbt7xB3xv7EpU9yuvSjDEhzn65G4S2HNpCzyk9\nmb9jPjeVv4nBzQdzZdErvS7LGBMmLPiDSEJiAu/99B4vL3iZnNlz8nnzz+lxXQ87Lt8YE1AW/EFi\n9tbZPDzjYTYd3ETrq1ozqNkgLilwiddlGWPCkAW/x3Yc2cHjsx5n4q8TqVi4ItM6TbNz5htjMpUF\nv0dOnTnFgCUD6LekH9kkG2/e8iaP136cXDlyeV2aMSbMWfBnsSRNYsy6MTw37zl2HNlB+yrtGdhw\nIGUvLut1acaYCGHBn4XmbZ/HU3OeYkX8Cq4tcS3z7p5H/Uvre12WMSbCWPBngTV71/D03KeZuWUm\n5S4ux/BWw7nrmrvsaB1jjCcs+DPR9sPbeXXhqwxbNYyLc1/MwIYD6VOzD7lz5Pa6NGNMBLPgzwTb\nD2/njUVvMGz1MLJLdp6o/QTP/udZCucp7HVpxhhjwR9I2w5v442FbzB8zXCyS3YejH6Qp+s8TemL\nSntdmjHG/M2CPwA2H9xM/8X9GbZ6GDmy5aBXdC+ervu0/QDLGBOULPgvwNJdSxm4dCCTfp1Ezuw5\n6V2jtwW+MSboWfCnU5ImMXnTZAYuHcjSXUsplLsQz//nefrU7EOJ/CW8Ls8YY9Jkwe+no6eP8s3q\nb/hg2Qf8dug3KhSswIdNPqRb9W7ky5nP6/KMMcZvFvxpWLdvHZ/EfMI3a77heMJxalxSgzF3jKHN\n1W3Ikc3+fcaY0GPJlYKExAQmbpzIxzEfs+j3ReTKnouO/9eRXtG9qFG6htflGWPMBbHg97F6z2q+\nXvU1I9eO5MDJA1xW6DIGNhzIvdXupUjeIl6XZ4wxAeHPNXe/ApoD+1T1Xxd7FZF6OBdZ3+4OmqCq\nr7rjmgAfANmBIaraL0B1B8yBkwcYtXYUQ1cN5Zc9v5Aze05uv/J2ulXrRuOKje20CsaYsOPPFv9Q\nYBAw/DzTLFLV5r4DRCQ78DHQENgNxIjIZFXdkMFaA+ZEwgmmbp7K6PWjmbZ5GmeSznB9qev5qOlH\ndKza0bbujTFhzZ+LrS8UkQoZWHZNYIt70XVEZDTQEvAk+P86+xczt8xk9LrRTNk8hZNnTlIqfyn6\n1OzDPdXu4ZoS13hRljHGZLlA9fHXFpHVwB/Ak6q6HigN7PKZZjdQK0Dr88ux08eYtXUWkzZNYvKm\nyRw9fZSieYvS9dqutK/Snrrl6pI9W/asLMkYYzwXiOBfCZRX1eMi0gz4HqiU3oWISE+gJ0C5cuUy\nXEz8sXimbJ7CpE2TmLttLgmJCRTOU5i2V7elQ9UO3HLpLXYYpjEmol1wAqrqUZ/700XkExEpCsQB\nvpeVKuMOS205g4HBANHR0ZreOk6eOcktw25hWdwyAC4rdBm9a/Sm5ZUtqVOujoW9Mca4LjgNRaQk\nsFdVVURqAtmAg8ARoJKIXIoT+B2AThe6vtTkjcpLxcIVaX5Fc1pd1YoqxaogIpm1OmOMCVn+HM75\nLVAPKCoiu4H/AlEAqvoZcAfwoIicBU4BHVRVgbMi0geYhXM451du33+mGdFmRGYu3hhjwoI4GR1c\noqOjNTY21usyjDEmZIjIClWN9mda+3WSMcZEGAt+Y4yJMBb8xhgTYSz4jTEmwljwG2NMhLHgN8aY\nCGPBb4wxESYoj+MXkf3AzgzOXhQ4EMByvBQubQmXdoC1JRiFSzvgwtpSXlWL+TNhUAb/hRCRWH9/\nxBDswqUt4dIOsLYEo3BpB2RdW6yrxxhjIowFvzHGRJhwDP7BXhcQQOHSlnBpB1hbglG4tAOyqC1h\n18dvjDHm/MJxi98YY8x5WPAbY0yECYvgF5F2IrJeRJJEJDrZuGdFZIuIbBKRxl7VmB4i0sStd4uI\nPON1PekhIl+JyD4RWeczrLCIzBGR39y/hbys0R8iUlZE5ovIBve19Yg7PBTbkltElovIarctr7jD\nLxWRZe7rbIyI5PS6Vn+JSHYR+UVEprqPQ7ItIrJDRNaKyCoRiXWHZfprLCyCH1gHtAEW+g4Ukco4\nl3ysAjQBPhGR7Flfnv/c+j4GmgKVgY5uO0LFUJz/ta9ngB9UtRLwg/s42J0FnlDVysANQG/3eQjF\ntpwGblHVa4FqQBMRuQHoD7ynqhWBw0B3D2tMr0eAjT6PQ7kt9VW1ms/x+5n+GguL4FfVjaq6KYVR\nLYHRqnpaVbcDW4CaWVtdutUEtqjqNlVNAEbjtCMkqOpC4FCywS2BYe79YUCrLC0qA1Q1XlVXuveP\n4YRMaUKzLaqqx92HUe5NgVuA79zhIdEWABEpA9wGDHEfCyHallRk+mssLIL/PEoDu3we73aHBbNQ\nrDktJVQ13r2/ByjhZTHpJSIVgOrAMkK0LW7XyCpgHzAH2AocUdWz7iSh9Dp7H3gKSHIfFyF026LA\nbBFZISI93WGZ/hpL82LrwUJE5gIlUxj1vKpOyup6TMaoqopIyBxDLCL5gfHAo6p61Nm4dIRSW1Q1\nEagmIgWBicBVHpeUISLSHNinqitEpJ7X9QRAXVWNE5HiwBwR+dV3ZGa9xkIm+FW1QQZmiwPK+jwu\n4w4LZqFYc1r2ikgpVY0XkVI4W51BT0SicEJ/pKpOcAeHZFvOUdUjIjIfqA0UFJEc7pZyqLzO6gC3\ni0gzIDdwEfABodkWVDXO/btPRCbidPVm+mss3Lt6JgMdRCSXiFwKVAKWe1xTWmKASu5RCjlxdk5P\n9rimCzUZ6Ore7woE/Tc0t9/4S2Cjqr7rMyoU21LM3dJHRPIADXH2WcwH7nAnC4m2qOqzqlpGVSvg\nvDfmqepdhGBbRCSfiBQ4dx9ohHOgSua/xlQ15G9Aa5x+vdPAXmCWz7jncfozNwFNva7Vz/Y0Aza7\ndT/vdT3prP1bIB444z4n3XH6YH8AfgPmAoW9rtOPdtTF6X9dA6xyb81CtC3XAL+4bVkHvOQOvwxn\nQ2gLMA7I5XWt6WxXPWBqqLbFrXm1e1t/7r2eFa8xO2WDMcZEmHDv6jHGGJOMBb8xxkQYC35jjIkw\nFvzGGBNhLPiNMSbCWPAbY0yEseA3xpgI8/8bl31bBoDTJgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "test = np.linspace(-10, 50, 200)\n", "test_y = np.reshape(sess.run(h1_2, feed_dict = {X1 : test}), [-1])\n", "print(test_y.shape)\n", "plt.plot(test, test_y,'g-')\n", "plt.title(\"function f which maximizes f(pd2) - f(pd1)\")\n", "plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.4.5" } }, "nbformat": 4, "nbformat_minor": 2 }