{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# gradient descent (경사 하강법)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Gradient descent 설명\n", " - http://nobilitycat.tistory.com/entry/Gradient-Descent" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Univariate Linear Regression 에서 gradient descent (경사 하강법) 설명" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Hypothesis Function\n", "$$h_{\\theta_0, \\theta_1}(x) = \\theta_0 + \\theta_1 \\cdot x$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Cost Function $J(\\theta_0, \\theta_1)$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\\begin{eqnarray}\n", " J(\\theta_0, \\theta_1) &=& \\dfrac{1}{2m} \\sum_{i=1}^m \\big( h_{\\theta_0, \\theta_1}(x^i) - y^i \\big)^2 \\\\\n", " &=& \\dfrac{1}{2m}\\sum_{i=1}^m \\big( \\theta_0 + \\theta_1\\cdot x^i - y^i \\big)^2\n", "\\end{eqnarray}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- We need derivatives for both $\\theta_0$ and $\\theta_1$:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\\begin{eqnarray}\n", " \\frac{\\partial}{\\partial \\theta_0}J(\\theta_0, \\theta_1) &=& \\frac{1}{m} \\sum_{i=1}^m (\\theta_0 + \\theta_1 x^{i}-y^{i}) \\\\\n", " \\frac{\\partial}{\\partial \\theta_1}J(\\theta_0, \\theta_1) &=& \\frac{1}{m} \\sum_{i=1}^m (\\theta_0 + \\theta_1 x^{i}-y^{i})\\cdot x^{i}\n", "\\end{eqnarray}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Batch Gradient Descent\n", " - 1) Pick an initial value for $\\hat \\theta $.\n", " - 2) Compute\n", " $$ \\frac{\\partial J}{\\partial \\theta} = \\big( \\frac{\\partial J(\\theta_0, \\theta_1)}{\\partial \\theta_0}, \\frac{\\partial J(\\theta_0, \\theta_1)}{\\partial \\theta_1} \\big) $$\n", " - 3) Compute with a proper learning rate $\\alpha$\n", " $$temp_0 := \\theta_0 -\\alpha \\frac{\\partial}{\\partial \\theta_0}J(\\theta_0, \\theta_1)\\\\\n", " temp_1 := \\theta_1 -\\alpha \\frac{\\partial}{\\partial \\theta_1}J(\\theta_0, \\theta_1)$$\n", " $$\\theta_0 := temp_0\\\\\n", " \\theta_1 := temp_1.$$\n", "\n", " - 4) Repeat 2) and 3) until update is small or reaches iteration maximum." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Stochastic (or Incremental) Gradient Descent\n", " - 1) Pick an initial value for $\\hat \\theta $.\n", " - 2) Compute\n", " $$ \\frac{\\partial J}{\\partial \\theta} = \\big( \\frac{\\partial J(\\theta_0, \\theta_1)}{\\partial \\theta_0}, \\frac{\\partial J(\\theta_0, \\theta_1)}{\\partial \\theta_1} \\big) $$\n", " - 3) Compute with a proper learning rate $\\alpha$\n", " $$\\theta_0 := \\theta_0 -\\alpha \\frac{\\partial}{\\partial \\theta_0}J(\\theta_0, \\theta_1)\\\\\n", " \\theta_1 := \\theta_1 -\\alpha \\frac{\\partial}{\\partial \\theta_1}J(\\theta_0, \\theta_1).$$\n", "\n", " - 4) Repeat 2) and 3) until update is small or reaches iteration maximum.\n", " - This method converges to the minimum more rapidly, but has the potential of overshooting the minimum and then oscillating around it." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Learning Rate control\n", " - By slowly letting the learning rate $\\alpha$ decrease to zero as the algorithm runs, it is also possible to ensure that the parameters will converge to the global minimum rather then merely oscillate around the minimum." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Local Optimum vs. Global optimum" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- For linear regression, the cost function $J(\\theta)$ does not have a local optimum other than the global optimum." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- However, we need to be susceptible to local optima in general cases." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Barch Gradient Descent - Python Code.\n", "- We get $\\theta_0$ and $\\theta_1$ as its output:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import random\n", "import sklearn\n", "from sklearn.datasets.samples_generator import make_regression \n", "import pylab\n", "from scipy import stats\n", "\n", "def gradient_descent(alpha, x, y, ep=0.0001, max_iter=10000):\n", " converged = False\n", " iter = 0\n", " m = x.shape[0] # number of samples\n", "\n", " # initial theta\n", " t0 = np.random.random(x.shape[1])\n", " t1 = np.random.random(x.shape[1])\n", "\n", " # total error, J(theta)\n", " J = sum([(t0 + t1*x[i] - y[i])**2 for i in range(m)])\n", "\n", " # Iterate Loop\n", " while not converged:\n", " # for each training sample, compute the gradient (d/d_theta j(theta))\n", " grad0 = 1.0/m * sum([(t0 + t1*x[i] - y[i]) for i in range(m)]) \n", " grad1 = 1.0/m * sum([(t0 + t1*x[i] - y[i])*x[i] for i in range(m)])\n", "\n", " # update the theta_temp\n", " temp0 = t0 - alpha * grad0\n", " temp1 = t1 - alpha * grad1\n", " \n", " # update theta\n", " t0 = temp0\n", " t1 = temp1\n", "\n", " # mean squared error\n", " e = sum( [ (t0 + t1*x[i] - y[i])**2 for i in range(m)] ) \n", "\n", " if abs(J-e) <= ep:\n", " print 'Converged, iterations: ', iter, '!!!'\n", " converged = True\n", " \n", " J = e # update error \n", " iter += 1 # update iter\n", " \n", " if iter == max_iter:\n", " print 'Max interactions exceeded!'\n", " converged = True\n", "\n", " return t0,t1" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Data Preparation\n", " - [Note]: http://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_regression.html" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "x.shape = (100, 1), y.shape = (100,)\n", "[[-2.04952781]\n", " [-0.43153213]\n", " [ 0.09858666]\n", " [-1.01328069]\n", " [ 1.93555916]]\n", "[ -82.10317641 -118.249632 -11.85958046 -111.97082092 72.50576718]\n" ] } ], "source": [ "from sklearn.datasets.samples_generator import make_regression \n", "x, y = make_regression(n_samples=100, n_features=1, n_informative=1, noise=35) \n", "print 'x.shape = %s, y.shape = %s' % (x.shape, y.shape)\n", "print x[0:5]\n", "print y[0:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Do gradient descent!" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Converged, iterations: 712 !!!\n", "theta0 = [ 3.29752334], theta1 = [ 51.72813087]\n", "intercept = 3.31458577872, slope = 51.8034956124\n" ] } ], "source": [ "alpha = 0.01 # learning rate\n", "ep = 0.01 # convergence criteria\n", "\n", "# call gredient decent, and get intercept(=theta0) and slope(=theta1)\n", "theta0, theta1 = gradient_descent(alpha, x, y, ep, max_iter=10000)\n", "print ('theta0 = %s, theta1 = %s') %(theta0, theta1) \n", "\n", "# check with scipy linear regression \n", "slope, intercept, r_value, p_value, slope_std_error = stats.linregress(x[:,0], y)\n", "print ('intercept = %s, slope = %s') %(intercept, slope) " ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAHfCAYAAACLeXg0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2cXGV99/HvLyRLRiCUwECUhx0UMWClJnQjFK0bJCje\nFbC3GlYrVdcWDFZRyw2IbVKLFPCh1baRqEtFgSXc9qUBpTysZK0okAXEAEkglnuWB4GMCMGYJdkk\n1/3HzGbObmZ2ZnbOmXOdM5/367UvZs7Ow7WzS+Y7v+t3XceccwIAAPDFtLgHAAAAEEQ4AQAAXiGc\nAAAArxBOAACAVwgnAADAK4QTAADglabDiZkdZmZ3mtkjZvaQmX2idPwAM7vdzB41s9vMbP/AfS42\ns41mtt7MTm12DAAAID2s2X1OzGyOpDnOuQfNbF9J90s6Q9KHJT3vnLvSzC6UdIBz7iIzO1bSdZK6\nJB0maUDSax0brgAAAIVQOXHOPeuce7B0eYuk9SqGjjMkXVO62TWSzixdPl3SDc65Hc65vKSNkhY0\nOw4AAJAOofacmFlO0hsl3SPpEOfcc1IxwEg6uHSzQyU9Gbjb06VjAAAAmh7WA5WmdL4n6ZPOuS1m\nNnGapuFpmwqPAQAAEsw5Z7VuE0rlxMymqxhMvuucW1U6/JyZHVL6/hxJm0rHn5Z0eODuh5WOVeSc\n46vC19KlS2Mfg89fvD68Prw+vDa8Pv591SusaZ2rJa1zzn01cOwmSR8qXf5LSasCx88ysw4zO1LS\nUZLWhDQOAACQcE1P65jZSZI+IOkhM/uFitM3n5V0haQbzewjkoYlvU+SnHPrzOxGSeskjUpa4hqJ\nUwAAINWaDifOuZ9J2qvKt0+pcp9/kvRPzT53O+vu7o57CF7j9Zkcr8/keH2q47WZHK9POJre5yRK\nZkZRBQCAlDAzuVY1xAIAAISFcAIAALxCOAEAAF4hnAAAAK8QTgAAgFcIJwAAwCuEEwAA4BXCCQAA\n8ArhBAAAeIVwAgAAvEI4AQAAXiGcAAAArxBOAACAVwgnAADAK4QTAADgFcIJAADwCuEEAAB4hXAC\nAAC8QjgBAABeIZwAAACvEE4AAIBXCCcAAMArhBMAAOAVwgkAAPAK4QQAAHiFcAIAALxCOAEAAF4h\nnAAAAK8QTgAAgFcIJwAAwCuEEwAA4BXCCQAA8ArhBAAAeIVwAgAAvEI4AQAAXiGcAAAArxBOAACA\nVwgnAADAK4QTAADgFcIJAADwCuEEAAB4hXACAAC8QjgBAABeIZwAAACvEE4AAIBXCCcAAMAroYQT\nM+szs+fMbG3g2FIze8rMHih9vSPwvYvNbKOZrTezU8MYAwAASIewKif/IentFY5/xTk3v/R1qySZ\n2TGS3ifpGEmnSVpuZhbSOAAAQMKFEk6cc3dJeqHCtyqFjjMk3eCc2+Gcy0vaKGlBGOMAAADJF3XP\nycfN7EEz+5aZ7V86dqikJwO3ebp0DAAAQNMjfOzlkj7vnHNmdqmkL0v6aKMPsmzZst2Xu7u71d3d\nHdb4AABAhAYHBzU4ONjw/cw5F8oAzKxT0s3OueMm+56ZXSTJOeeuKH3vVklLnXP3VrifC2t8AAAg\nXmYm51zNPtMwp3VMgR4TM5sT+N6fS3q4dPkmSWeZWYeZHSnpKElrQhwHAABTcttt0rZtcY8CoUzr\nmNn1krolHWhmT0haKmmhmb1R0i5JeUnnSJJzbp2Z3ShpnaRRSUsojwAA4rRjhzRjRvHys89KhxwS\n73jaXWjTOlFgWgcAELUnnpA6O4uX166V3vCGeMeTZnFM6wAAkCirVpWDyZYtBBNfEE4AIAUKhYKG\nhoZUKBTiHkpinHOOdOaZ0gknSM5J++wT94gwhnACAAnX379SnZ1ztWjRuersnKv+/pVxD8lrO3dK\ne+8tfeMb0le+It19d9wjwkT0nABAghUKBXV2ztXIyGpJx0laq0xmoYaHNyibzcY9PO8884z0qlcV\nL993n3T88fGOp93QcwIAbSCfz6ujI6diMJGk4zRjRqfy+Xx8g/LUrbeWg8nmzQQTnxFOACDBcrmc\ntm/PSxo7KfxajY4OK5fLxTcoD51/vnTaacWG1127pFmz4h4RJkM4AYAEy2az6utbrkxmoWbNmq9M\nZqH6+pYzpVOya5c0e7b01a9KX/hCcamw1ZxUQNzoOQGAFCgUCsrn88rlcgSTkkJBOvjg4uWf/1w6\n8cR4x4P6e04IJwCA1Fm9Wjr55OLl3/5WOuCAeMeDIhpiAQBt6ZJLisHkyCOL0zoEk+QhnAAAvNDs\nRnLOSbmcdNll0uc+Jz3+OP0lSUU4AQDErtmN5H77W2naNGl4uDil84//GNFA0RL0nABAm/GtebbZ\njeR+9jPpzW8eeyzpoIMiHS6aQM8JAGAPPm5138xGcpdeWgwm2Wyxv4Rgkg5UTgCgTfi61f1UxuWc\n9PrXS+vXS5/+tPTlL7d0yJgiKicAgHF83eq+0Y3kXnqp2F+yfr10220EkzSicgIAbcLXysmYenph\n7rtP6uoqXn7mGWnOnBYOEE2jcgIAGMf3re6z2ay6urqqjucrXykGk0xG2rmTYJJmVE4AoM34tlqn\nHgsWSEND0rnnSl//etyjwVSxfT0AIPG2bJH22694edUq6fTT4x0PmlNvOJneisEAANCoX/5SeuMb\ni5efeEI6/PB4x4PWoecEAOCd5cvLwWR0lGDSbggnAACvnHyydN550tlnF/czmU6Nv+3wKwcAeOGF\nF6TZs4uXb7xReu974x0P4kPlBAAQux/8oBxMHn+cYNLuqJwAAGL1p38q/fSnxcu//730ilfEOx7E\nj3ACAIiNBRaVsnMExjCtA6BhhUJBQ0NDKhQKcQ8FCbVlSzmYnHoqwQTjEU4ANKS/f6U6O+dq0aJz\n1dk5V/39K+MeEhLm9tvLG6v98IfFk/f5guDtB3aIBVA3308cB/+dfrp0883Fy5s3S7NmxTueoP7+\nlertXaKOjpy2b8+rr2+5enoWxz2sVOHEfwBCl8/n1dGRUzGYSNJxmjGjU/l8Pr5BITHMysHEOb+C\nSaFQUG/vEo2MrNbmzfdrZGS1enuXUEGJCeEEQN1yueInSmlt6chajY4OK5fLxTcoeG9kpNxfsmCB\nn/0lBG+/EE4A1C2bzaqvb7kymYWaNWu+MpmF6utbzpQOqvrpT8tLg1eulO69N97xVEPw9gs9JwAa\nVigUlM/nlcvlCCao6uyzpe9+t3j5N7+RDjww3vHUMtZzMmNGp0ZHh+k5iUC9PSeEEwBA6JK6fwnB\nO1qEEwBAy23fLu29d/Hya18rPfZYvOOBX1itAwBoqfvuKweTq68mmGDq2L4eANC0j39c+vd/L15+\n5hlpzpx4x4NkI5wAAJqS1P4S+ItpHQDAlOzcWQ4mBx5IMEF4CCcAgIY9/LA0vVR7/9rXikuFgbAw\nrQMAaMgll0iXXVa8nM9LnZ2xDgcpRDgBANQt2F+ya9f460BYmNYBANQUDCLTphX7SwgmiArhBAAw\nqY0bpb32Kl6+7LJiIywQJaZ1AABVXX65dPHFxcuPPVbc9RWIGuEEAFBRcNpm587idA7QCqH8qZlZ\nn5k9Z2ZrA8cOMLPbzexRM7vNzPYPfO9iM9toZuvN7NQwxgAACMfEfhLnCCZorbD+3P5D0tsnHLtI\n0oBz7nWS7pR0sSSZ2bGS3ifpGEmnSVpuRlsVAPjgoYfKQaSri43VEI9Qwolz7i5JL0w4fIaka0qX\nr5F0Zuny6ZJucM7tcM7lJW2UtCCMcQAApu7ss6XjjiteHhiQ1qyJdzxoX1H2nBzsnHtOkpxzz5rZ\nwaXjh0q6O3C7p0vHAAAxCdavt2+XZsyIbyxAKxtip1QcXLZs2e7L3d3d6u7uDmk4AACJE/chOoOD\ngxocHGz4fuZC+ks0s05JNzvnjitdXy+p2zn3nJnNkbTaOXeMmV0kyTnnrijd7lZJS51z91Z4TBfW\n+AAA4/3qV+WlwUceKT3+eLzjQfqZmZxzNftMw+y/ttLXmJskfah0+S8lrQocP8vMOszsSElHSWJm\nEwBa6Pzzy8Hk+98nmMAvoUzrmNn1krolHWhmT0haKulySf/XzD4iaVjFFTpyzq0zsxslrZM0KmkJ\n5REAaJ3gNM7WrVImE99YgEpCm9aJAtM6ABAu+ksQpzimdQAAntq4kWCC5CCcAEDKLV4sHX108fKl\nlxJM4D/OrQMAKRaslvz2t9IBB8Q3FqBehBMASCmmcZBUTOsAQMo8/TTBBMlGOAGAFDnvPOmww4qX\nzz+fYIJkYloHAFIiWC359a+lV74yvrEAzSCcAEAKMI2DNGFaBwAS7NlnCSZIH8IJAG8VCgUNDQ2p\nUCjEPRQvfehD5ambri6CCdKDcALAS/39K9XZOVeLFp2rzs656u9fGfeQWqpWMDOTrrmmePmRR6Q1\nnD4VKcK5dQB4p1AoqLNzrkZGVks6TtJaZTILNTy8QdlsNu7hRa6/f6V6e5eooyOn7dvz6utbrp6e\nxbu/zzQOkopz6wBIrHw+r46OnIrBRJKO04wZncrn8/ENqkUKhYJ6e5doZGS1Nm++XyMjq9Xbu0SF\nQkEvvkgwQXsgnADwTi5XrBhIa0tH1mp0dFi5XK7px/a9j6VaMPv0p3fs3np+zhyCCdKNcALAO9ls\nVn19y5XJLNSsWfOVySxUX9/ypqd0ktDHUimYvfTSA7r22mLn6z33SM88E9fogNag5wSAtwqFgvL5\nvHK53KTBpJ7bJamPZaznZMaMTr300gO7j/PPIZKOnhMAiZfNZtXV1TVpeKhUDak0dZOkPpaensVa\nt24DwQRti8oJgKbVW+GI4nknVkM6Ov5U06aZ9t771eNWuiSpcvLBD0rXXlu+zj+DSAsqJwBaIs4+\njkrVkO3bD9LLL//7HitdoupjCZtZOZhcey3BBO2JygmAKYu7GlHp+aUTJeUlFZ9/1qz5GhhYoa6u\nrt33qVblabYC1Oz9g8uEd+0afx1IAyonACIXdx9HpWrIjBnTJI0tZ9lzCXK1PpZmK0DN3H/79j33\nLyGYoJ1ROQEwZXFXToLjGKtYDAzcuXuly+jo8B67q1a7fzM/RzP3v/BC6cory9f5Jw9pVm/lZHor\nBgMgncYqF729C8eFgVb3cWSz2d3P2dOzWKeccnJD0ytjFaCRkT0rQLXuXygUdMstt2j69ENVqYI0\n2f2D1ZHLLy8GFQBUTgCEIK7VOmGZauVjbD+S6dM79bvfbZC0TNL/kbRWM2e+VatWrdS8efMqPkYw\nmOzYIe21V7g/UxiS/nuFf+qtnBBOAMTOhzfB4MZn9UwHVW7GPUH77vsabds2LLNpymRes8eJ+3bu\nlKYHata+/hNX6+SDwFQQTgAkgk9vgsGQJGnSwDQ0NKRFi87V5s337z62337z9IUvfEQXXrisYhXm\n3/4tq89/vvwYvv7z5ksvEdKH1ToAvDfZGXjjMLaSZ2DgzporbyqdA2fHjif0ute9ruIKpoMPLgeT\n97/f32Aixb8KCyCcAIiNL2+Cwe3u6w1M1TZ1mzdvXsUT943ZulW67rqW/WhTEuVZoYF6EE4AxMaH\nN8GJ+5OsWPHNugNTT89iDQ9v0MDACg0Pb1BPz+JxoWW//eYHHqdYLclkWvBDNSkpu+kiveg5AVLA\nh4bSqWq0ETVMlXorZs58q8ymNd1v8aUv/U4XXLDf7utJ/KcsyX9X8BMNsUCb8KmhdKriehOs1NQ6\na9Z8XXDBe3TZZV+ecmAKLhM+6STprrvCHDWQXIQToA2wqqI5k71+0uSrdaoJBpPnn5dmzw53zECS\nsUMs0Aaa2dkUtXe4bfQ1nHh+HABTQ+UESDDfKydJ6VlodpzXXit98IPl6/yzBVTGPidAG/B5VUWz\nZ/ltpWpnKq6HGcEECBuVEyAFfKtQ+F7RCUtwGufxx6Ujj4xvLEAS0HMCtJHgWXl90A69MPSXANFh\nWgdA6HzYXC0qN99MMAGiRuUEQOhqrYJJKptQjCaYANGg5wRAZHzrhWlGMJjce6+0YEF8YwGSik3Y\nACAkTOMA4WApMYC2EjyzcFjuuYdgAsSBcAIg8aLYU8VMOvHE8nWCCdA6TOsASLQo9lQJVkt+9CPp\nne8MZahA22OfEwCJ0UzjbNh7qjCNA8SPaR0AsWp2SiasPVWGhggmgC+Y1gEQm7CmZPr7V6q3d8m4\nPVV6ehbXfX/2LwFaw5vVOmaWN7NfmtkvzGxN6dgBZna7mT1qZreZ2f5RjwOAf8amZIrBRApOyTSi\np2exhoc3aGBghYaHN0w5mPzt3xJMAB9EXjkxs8clHe+ceyFw7ApJzzvnrjSzCyUd4Jy7qMJ9qZwA\nKRb3CQKZxgFay5vKiSSr8DxnSLqmdPkaSWe2YBwAPDO2zX0ms1CzZs1XJrOwJdvc/+pXBBPAZ62q\nnLwoaaekFc65b5nZC865AwK3+a1zbnaF+1I5AdpAK7e5p78EiI9PS4lPcs49Y2ZZSbeb2aOSJv5z\nUPWfh2XLlu2+3N3dre7u7ijGCCBG2Wy25dM4CxYUz5EDIDqDg4MaHBxs+H4tXa1jZkslbZH0UUnd\nzrnnzGyOpNXOuWMq3J7KCYBQBIPJrl17VlAARM+LnhMze4WZ7Vu6vI+kUyU9JOkmSR8q3ewvJa2K\nchwA2tfzz+/ZX0IwAfwWaeXEzI6U9H0Vp22mS7rOOXe5mc2WdKOkwyUNS3qfc+7FCvencgJgyugv\nAfxSb+WETdiABGplA2lSBYNJJiNt3RrfWAAUeTGtAyB8UZyBN22CwWR0lGACJA2VEyBB4t60zHfb\ntkkzZ5av888H4BcqJ0AKhbXdexqZEUyAtCCcAAkS1hl441IoFDQ0NKRCoRDq49L4CqQL4QRIkLi2\new9DVL0ywWDy0ksEEyAN6DkBEihpq3Um65WRNKWfZdcuaa+9ytf5pwLwHz0nQIpls1l1dXUlIphI\n1XtlVqz45pSqKZkMwQRIMyonACK3Z+VkUB0d79K0aTP08suDamTlUZj9JUmrQAFJR+UEgDeCvTIz\nZ75a0jtldohefvlANbLyKBhM8vnmggn7xQD+onICoGXWr1+vefP+RNu2/UTSKyW9TtKg6qmcTDw/\nTjPq2S+GqgoQPionQJ2iWt6KPW3ZsqVUOTlOUlbS1yWdqH32+aOqK4/e8pZwg4lUe78YqipAvKic\noK31969Ub+8SdXQU9w/p61uunp7FcQ8rtSpVLGbOfKtWrVqpefPm7RFMotq/pNbqIXbhBaJB5QSo\noVAoqLd3iUZGVmvz5vs1MrJavb1LqKBEqNI+LVdffZVOPfXUSYPJ0FC4K3Im2y+GXXiB+FE5Qdsa\nGhrSokXnavPm+3cfmzVrvgYGVqirqyvGkaVfrX6OWtM4YfWDVHoczl8ERIfKCVBD0reCj0sYPTrV\n9mn5yEdqB5Mw+0EqjSPJu/ACaUHlBG1trOdkxoxOjY4O03NSQ5Q9OvX0l7SyquHTah2fxgI0o97K\nCeEEbY9/+OsTZTAIBpMbb5Te+97Kt2vHqTiatpEm9YaT6a0YDOCzbDZLKKnDWKPoyMiejaLNvH6N\nLBMePxVXDEhpnooLNm0XX/e16u1dqFNOOZm/WaQaPSdAxKbSo+Hj3ith9+h88Yvjg8mmTbV/5nbr\nB2HlENqWc87br+LwgOS6/vobXCYz2+2//3yXycx2119/QyT3aZWxsc2aNa+psRVrJOWvRn/mTZs2\nuTVr1rhNmzZN6fmTYtOmTS6Tme2kX5Zeq1+6TGZ26n9upFfpfb3m+z89J0BEptKjEXXDZxj9Nc0+\nRrBacuml0l//NUt3J0PTNtKEnhMgZlPp0Yiqr0MKr7GymR6dSv0lQ0PR/cw+qzfk9fQs1imnnEzT\nNtoKPSdARKbSoxHV3itx74b7gx9Ub3xtx/1mGt2rpdq+MEBaEU6AiEyleTOMhs9KzbRxNlaaSe9+\nd/n6xJnadmtyjTsoAklAzwkQsan0aEy1r6Pa1E1cW7IHqyUf+IB07bXVb9su+820414twBg2YQNa\nyIc31loBZGJj5Wc/+xmdc85fRTbeRvYvaSecuwftjHPrAC0S5rlemlFr6qanZ7GGhzfoggveI+d2\n6Utf+s9IxnvPPQSTybTbNBYwFVROgCb49Cm4nrFEPd56zo+DIh+qbUCrUTkBWsCnHTzr+UQexXjH\nGnCDweSIIwgmtbACB6iOygnQBJ8qJ8ExVftEHvZ4x/pYRkae332M/2UBVEPlBGgBH/sHJvtEHuZ4\nC4WCPvzhK8YFk0zmwMQvifXxvEZAu6FyAjSoUmUiaf0DYYx3Yn+JlPwlsWHtogugMpYSAxFo5M2r\nWgBIWpCppFIw8WFKqxk+TtEBacO0DhCyRnb2rLa8OKxlx3FOPQSDybXX3qhM5kBvprSa4VNzM9Du\nqJwAdap3Z89qn8Dvv/8uHX/8m5v+ZB7X1MOLL0oHHFC+Pva/ZhoqQRKVE6AVqJwAIav3BHXVPoGv\nWbOm6U/mcZ2XxaxyMJHSsyTWx+ZmoF0RToA61fvmVS3ELFiwoOmz78Yx9dBOG6uN7aI7MLBCw8Mb\naIYFYsK0DtCgeqYxJp7HZmzqpdrxRp672tSDpNCnV4LB5He/k/bdN5SHBdCmWK0DhKzR3oqoVutU\nCjiSQu1D2bFDmjGjfJ3/DQGEgXAChMi3/S+CAUdSqI2caZ/GSUsDL5BENMQCIYmrCXUywSbUMPtQ\n0h5MfDmDNIDJEU6AGpp582/FfiT1riKqJRhMnnhC2rQpXdu4+xgyAVRGOAFqmOqbf6s+pYexBDYY\nTJyT7rorfRUGNlkDkoOeE6AOja6yiWNDr6n0UlSaxknrZmRp/bmAJKm352R6KwYDJF1Pz2KdcsrJ\ndb/5j31KHxnZ81N6VG+E2Wx2ytUSqdxfEsfYW2GswtTbu3BcyEzyzwSkFZUTIAK+f0oPBpO775ZO\nOKF83fexN4vVOkB8vF+tY2bvMLMNZvaYmV0Y1ziAKPi2FXqwMXdif0kwmEj+jT1sadluH0izWCon\nZjZN0mOS3ibp15KGJJ3lnNsw4XZUTpBoPnxKH+uXefnljXJu9u7jtf7X8mHsANLF603YzOwESUud\nc6eVrl8kyTnnrphwO8IJ0ITyFM3z445v2lRoWeAg5AAY4/u0zqGSngxcf6p0DECI8vn8HsFk1qz5\nLVs+y6ZnAKaCfU6AFFuwoGvCkalt0DYVbHoGYKriWkr8tKQjAtcPKx3bw7Jly3Zf7u7uVnd3d5Tj\nAlLhYx+TrrqqfD2TObDly2fTuiQZQP0GBwc1ODjY8P3i6jnZS9KjKjbEPiNpjaQe59z6Cbej5wRo\nULWN1Vrd95H2JckAGud1z4lzbqekj0u6XdIjkm6YGEwANC4YTC69NN4T96V9STKA6LAJG5ASE/cv\nGTO2lLijo3iOoFpb74eN1ToAxni9lLhehBOgtq99TfrkJ8vXg//LMLUCwCecWwexivvTctzP3yrV\nzo8zJs6m1Hb5HQAIH0uJEbq497aI+/lbJRhM3vOeyv0luVxxKkdaWzrSmqXE7fI7ABANpnUQqrin\nEeJ+/lap1l9SyVjPSXApcZQ9J+3yOwDQOK9X6yC9xqYRim9KUnAaoR2eP2p33NFYMJGknp7FGh7e\noIGBFRoe3hB5M2zafwcAokfPCUI1fhqh+Km5VTuS+vD8UarVX1JNq3s/0vw7ANAaVE4Qqrj3toj7\n+aMSDCbHHFN/MFmx4ps6/PCj9ba3nVNX70ehUNDQ0FBTW8yn9XcAoHXoOUEk4l6pEffzh6naNE6t\nn3HFim/q3HM/Keke1dP7EfZ+KGn6HQAIB/ucIDa8KYVj3Trp9a8vX29kY7VCoaDDDz9K27YdKenB\n3cf322+efvzjb6ira/wJAWliBdAKNMQiFiwhDYdZ9WBSz9l+i02pnZKeVHAZ8fbt+Yq9HzSxAvAJ\n4QShqedNE7XVu7HaZEEil8tpx46nJV0oaaGkP5J0gr761SsrVkLi2g8FACohnCA0fPpuXjCY7No1\n9Y3Vyk2pV2jffV+lvffO66qrvqpzzvmris9LEysAn9BzgtDQtzB1v/mNFHyJwtpYrdH+nyT3CyV5\n7EC7oCEWsWj1bqRpkJT9S3wW95mXAdSHcILYJOlNM+6xTjWYoIyKHZAcrNZBbLLZrLq6urx/Y4h7\nZVEwmLz8MsFkquh1AtKHygnaUpyftrdtk2bOLF/nT7w5VE6A5KByAkwirk/bZgSTsLHSCEgfKido\nS3F82qa/JFpx9w8BqI3KCdpKoyesa/Wn7WAw+c1visEkjJPsoSwpvU4AaqNygsRrZhlp1J+2nZOm\nTRt/vdkxA0BSsZQYbcHnZsjXvEZ6/PHy9bE/ZZ/HDABRYloHbcHXZaRmlYOJ5O+YAcAXhBMkmo8n\nrAv2l+Tzeza++jhmAPAJ4QSJ5tsy0mAwcU7q7NzzNr6NGQB8Q88JUiHuZaR/8RfSddeVr9fzZxv3\nmKOW9p8PQONoiAVahP1L9sRqJACVEE6AFggGk3vukd70pvjG4gtWIwGohtU6QMQm9pe0Kpj4vnkb\nq5EANItwAjToc5/bM5i0StxnUq4Hq5EANItpHaABcfaXJGm6ZKznZMaMTo2ODtNzAkAS0zpIobin\nM4LB5Hvfa33j64oV39TIyGwlYbqkp2exhoc3aGBghYaHNxBMADSEygkSIe7VH3FN44wpFAo64oij\n9fLLJmlQvldOAKASKidIjUKhoN7eJRoZWa3Nm+/XyMhq9fYuaUkF5dvfjj+YSMUm0733frWkr0ta\nKGm+pBP12c9+hmACIHWmxz0AoJax1R8jI3tOZ0T5xuzT/iXlJtNjJG2QdIdmzjxP55zzV/ENCgAi\nQuUE3otj9UcwmHzxi/FvrDZ+y/u3K5P5G1199VVUTQCkEj0nSIRWrv7wYRqnGraEB5Bk7BCL1In6\njfnnP5dOOql8nT89AAgX4QRogE/9JQCQVqzWAeoUDCYXXND6YBL3/i0A4BvCCdraxP6SK69s7fMn\nYTt6AGg1pnXQlh59VJo7t3w9ro3VkrIdPQCEgWkdoAqz+IOJxNl7AaAawgnaSnAa58wzfdlYrbx/\ny7Zt/08jv4taAAAUAElEQVT77rtvfIMCAA8QTtA2JvaXfP/78Y1FGr+xWibzBkknaNq0A3T88W+u\nu/eEZloAaUTPCVLv+eelgw4qX/ftT2r9+vWaN+8Ebdu2SlK36u09iftkiADQKHpOABWrJT4HE0na\nsmWLZs48SsVgItXTexLnyRABIGqRhRMzW2pmT5nZA6WvdwS+d7GZbTSz9WZ2alRjQHsLTuO84Q2j\nWrPGz+mPqZw7qJXNtEwdAWi1qCsnX3HOzS993SpJZnaMpPepeHrV0yQtN5u4PyfQnOBf1LXX3qhf\n/WqOt3uJjD+p33xlMgvV17d80imdVp0MkX1YAMQhsp4TM1sqaYtz7ssTjl8kyTnnrihd/y9Jy5xz\n91Z4DHpO0JBt26SZM8vXN21Kzl4ijZ47KOqTIbIPC4Cw+dJz8nEze9DMvmVm+5eOHSrpycBtni4d\nA5oyf/74YOJcsvYSyWaz6urqqvuNv6dnsYaHN2hgYIWGhzeE3gybpNcOQLpMb+bOZnaHpEOChyQ5\nSZdIWi7p8845Z2aXSvqypI82+hzLli3bfbm7u1vd3d1NjBhpFZzGOfZY6ZFHipfHT38UP/1HMf0R\nl2w2G1kVI+2vHYDoDQ4OanBwsOH7tWQpsZl1SrrZOXdchWmdWyUtZVoHUxUMJlu3SpnM+O9HPf2R\nZrx2AMJU77ROlD0nc5xzz5Yuf0pSl3Pu/WZ2rKTrJL1JxemcOyS9tlIKIZy0r3r6L3btkvbaq3x9\nsj+VRvs50iKMn7tdXzsA4fOh5+RKM1trZg9KequkT0mSc26dpBslrZN0i6QlJBAE1bNC5P3vrz+Y\nSI33c6RBWCtt2vG1AxAvdohFQ6L+FF3PCpGJC8+n8ieS9moAK20A+MiHyglSphV7XtRaIRIMJs8/\nP7Vg0g57d7DSBkCSUTlBXVr1SXyy5zn44PLzTPXPol0qCu3ycwJIFionCFWrPolX2i31ve+9I5Rg\nIoXzcyRhO/ep7DoLAL6gcoK6tPqT+FhPyIIFXbuPTZsm7dzZ/OPW83NU60lJ2pmA095bAyBZYl9K\nHAbCiV9avedFsL/kqaekQ0PaR7jWz1EtgEQR0AgPANoJ4QSRaNWbaTCYRPEnUO3nmCyA5PN5LVp0\nrjZvvn/37WfNmq+BgRXq6ura80lqSFoVBgCaRThBIv3oR9Kf/Vn5eqt//UNDQ1UDSC6XC61yQsMq\ngHZEQywSxyzeYCJNPJ+MFDyfzFSbTCs10LLUFwCqI5zAC8FpnMceiyeYSLVXuTR6JuBqe6pMFoIA\noN0xrYPYRd1fMhVhnZNmsqkbTqoHoN3UO60zvRWDASq5+27pT/6kfN2XYBKWX/ziF5o27XBVmrrJ\nZrPq6VmsU045mdU6ADAB0zqIhZm/wSSM7e37+1fqjDMW6/e/36jJpm44qR4A7IlpHbRccBpnzRpp\nCqtwI1PPKppaUz7jH2O9pI9Jmq1M5gWmbgC0NVbrwEsT+0t8CiZS7VU09VRVxj/GYkmPap999tIP\nftBPMAGAOhBO0BL/8z/jg8mmTX6en2ayVTSFQkG9vUs0MrJamzffr5GR1ertXbLHz7DnYzyjXbt+\no3nz5rXs5wCAJCOcIHIHHigddVT5+vXXN9/TEZXJlhLXuzcJJ90DgObQc4JIBaslg4PSsccmY2fU\nSn0lje7qynlzAGA8lhIjdpX2LxkaKlYfRkYqL6/1RTab3WM8YxWR3t6F4/YmqTbuSo8BAKiNyglC\n98IL0uzZ5evBX2EazilDRQQApobVOojFkiXVg4mUjn4M9iYBgGhROUFogtM4//3f0lveUv22VB8A\noP3UWzkhnCAUPp4fpxYCEgC0FtM6aImtW5MZTMLYoh4AEA0qJ5iyZcukf/iH8vWk/KrS0JQLAElE\n5QSRMisHk5tuSk4wkWpvUe+jQsHPHXUBIAqEEzQsOI2za5f0rnfFN5apmGyLeh8xBQWg3TCtg7rt\n3ClND2zbl+RfTX//SvX2Lhm3mVoYJ+ULu8mWKSgAacK0DkL14x+nJ5hIUk/PYg0Pb9DAwAoND28I\nJZhEUeFI4hQUADSLyklCxLns9XWvkx57rHi51v4l7SqqCgeVEwBpQuUkReLsOTArB5OdOwkm1URV\n4UjDjroA0CgqJ56L8hP5ZJUY56Rp08ZfR3VRVzjYMA5AGlA5SYkoPpHXqsTcc085mBx2GMGkHlFX\nODifD4B2QuXEc2F/Iq/1eG95i3TXXcXb/uhH0jvfGeIP0waocABAdfVWTqbXugHiNfaJvLd34bhl\nr1N94xurxIyM7FmJOfjg8mM+/XRBr3oVb66NymazhBIAaBKVk4QI6xN5pcrJzJkL9fLLz+++zf77\nH6/t2/Oh7f0BAIDEWYkxieAGZNu27dK2bQ9KkqZNu027dr1SLFkFAESBhlhUNbYBWU/PTbuDyVVX\nbdB++31WbPYFAIgb4aRNfeITWa1YcZgkads26c///MBEnW8GAJBeNMS2obET9+2zj7RlS/Fy2I23\nAABMFT0nbWTzZukP/qB4+fLLpQsv3PM2LIUFAESFhliMc+ed0tveVrz80EPSH/5hvOMBALQf9jnB\nbuecI33jG8XL27ZJHR3xjgcAgMkQTlJurL8kk5G2bo13LAAA1IPVOim1ZUs5mPzd3xFMAADJQeUk\nhX72M+nNby5evu8+6fjj4x0PAACNoHKSMp/+dDmYbN1KMAEAJA+VkxSxQP8zi5wAAEnVVOXEzN5j\nZg+b2U4zmz/hexeb2UYzW29mpwaOzzeztWb2mJn9SzPPj6KRkXIwOf98ggkAINmandZ5SNK7Jf0k\neNDMjpH0PknHSDpN0nKz3Z/rvy6p1zl3tKSjzeztTY6hrd1/v/SKVxQv33WX9M//HO94AABoVlPh\nxDn3qHNuo6SJG6qcIekG59wO51xe0kZJC8xsjqT9nHNDpdt9R9KZzYyhnf3930t//MfFy7/7nXTS\nSfGOBwCAMETVc3KopLsD158uHdsh6anA8adKx9GgV7yiOJ0jMY0DAEiXmuHEzO6QdEjwkCQn6RLn\n3M1RDWzMsmXLdl/u7u5Wd3d31E/pte3bpb33Ll7+6Eelb34z3vEAAFDN4OCgBgcHG75fKOfWMbPV\nkj7jnHugdP0iSc45d0Xp+q2SlkoalrTaOXdM6fhZkt7qnPtYlcfl3DoBDz8sveENxcsDA+Vz5QAA\nkAT1nlsnzH1Ogk92k6SzzKzDzI6UdJSkNc65ZyVtNrMFpQbZsyWtCnEMqXXFFeVg8uKLBBMAQHo1\n1XNiZmdK+ldJB0n6oZk96Jw7zTm3zsxulLRO0qikJYESyHmSvi1ppqRbnHO3NjOGdhA8cR+FJABA\n2oUyrRMVpnWK9tpL+tKXpE99Ku6RAAAwdfVO67BDbALs3Bn3CAAAaB3OrQMAALxCOAEAAF4hnAAA\nAK8QTgAAgFcIJwAAwCuEE0SmUChoaGhIhUIh1c8JAAgX4QSR6O9fqc7OuVq06Fx1ds5Vf//KVD4n\nACB8bMKG0BUKBXV2ztXIyGpJx0laq0xmoYaHNyibzabmOQEAjYnj3DqAJCmfz6ujI6diSJCk4zRj\nRqfy+XyqnhMAEA3CCUKXy+W0fXte0trSkbUaHR1WLpdL1XMCAKJBOEHostms+vqWK5NZqFmz5iuT\nWai+vuWRTq/E8ZwAgGjQc4LIFAoF5fN55XK5loWEOJ4TAFCfentOCCdADAhRANoRDbGIBPuINI8l\nzwAwOSonqFt//0r19i5RR0ex+bSvb7l6ehbHPaxEYckzgHZG5QShKhQK6u1dopGR1dq8+X6NjKxW\nb+8SKigNYskzANRGOEFdeFMNB0ueAaA2wgnqwptqOFjyDAC10XOSAL6s7BjrOZkxo1Ojo8P0nDTB\nl98pALQSS4lTwrcmVN5UAQBTRThJAVZ2AADShNU6KUATKgCgHRFOPEYTKgCgHRFOPMbKDgBAO6Ln\nJAFoQgUApAENsQAAwCs0xAIAgEQinAAAAK8QTgAAgFcIJwAAwCuEEwAA4BXCCQAA8ArhBAAAeIVw\nAgAAvEI4AQAAXiGcAAAArxBOAACAVwgnAADAK4QTAADgFcIJAADwCuEEAAB4hXACAAC8QjgBAABe\nIZwgMQqFgoaGhlQoFOIeCgAgQoQTJEJ//0p1ds7VokXnqrNzrvr7V8Y9JABARMw5F/cYqjIz5/P4\n0BqFQkGdnXM1MrJa0nGS1iqTWajh4Q3KZrNxDw8AUCczk3POat2uqcqJmb3HzB42s51mNj9wvNPM\ntprZA6Wv5YHvzTeztWb2mJn9SzPPj/aQz+fV0ZFTMZhI0nGaMaNT+Xw+vkEBACLT7LTOQ5LeLekn\nFb73K+fc/NLXksDxr0vqdc4dLeloM3t7k2NAyuVyOW3fnpe0tnRkrUZHh5XL5eIbFAAgMk2FE+fc\no865jZIqlWj2OGZmcyTt55wbKh36jqQzmxkD0i+bzaqvb7kymYWaNWu+MpmF6utbzpQOAKTU9Agf\nO2dmD0jaLOnvnHN3STpU0lOB2zxVOgZMqqdnsU455WTl83nlcjmCCQCkWM1wYmZ3SDokeEiSk3SJ\nc+7mKnf7taQjnHMvlHpRfmBmx05lgMuWLdt9ubu7W93d3VN5GKRANpsllABAggwODmpwcLDh+4Wy\nWsfMVkv6jHPugcm+r2JoWe2cO6Z0/CxJb3XOfazK/VitAwBASrRktc7E5ww8+UFmNq10+dWSjpL0\nuHPuWUmbzWyBmZmksyWtCnEMAAAg4ZpdSnymmT0p6QRJPzSz/yp9608lrS31nNwo6Rzn3Iul750n\nqU/SY5I2OudubWYMAAAgXdiEDQAAtEQc0zoAAABNI5wAAACvEE4AAIBXCCcAAMArhBMAAOAVwgkA\nAPAK4QQAAHiFcAIAALxCOAEAAF4hnAAAAK8QTgAAgFcIJwAAwCuEEwAA4BXCCQAA8ArhBAAAeIVw\nAgAAvEI4AQAAXiGcAAAArxBOAACAVwgnAADAK4QTAADgFcIJAADwCuEEAAB4hXACAAC8QjgBAABe\nIZwAAACvEE4AAIBXCCcAAMArhBMAAOAVwgkAAPAK4QQAAHiFcAIAALxCOAEAAF4hnAAAAK8QTgAA\ngFcIJwAAwCuEEwAA4BXCCQAA8ArhBAAAeIVwAgAAvEI4AQAAXiGcAAAArxBOAACAVwgnAADAK4QT\nAADgFcIJAADwCuEEAAB4palwYmZXmtl6M3vQzP7TzGYFvnexmW0sff/UwPH5ZrbWzB4zs39p5vnb\n2eDgYNxD8Bqvz+R4fSbH61Mdr83keH3C0Wzl5HZJr3fOvVHSRkkXS5KZHSvpfZKOkXSapOVmZqX7\nfF1Sr3PuaElHm9nbmxxDW+J/gMnx+kyO12dyvD7V8dpMjtcnHE2FE+fcgHNuV+nqPZIOK10+XdIN\nzrkdzrm8isFlgZnNkbSfc26odLvvSDqzmTEAAIB0CbPn5COSbildPlTSk4HvPV06dqikpwLHnyod\nAwAAkCSZc27yG5jdIemQ4CFJTtIlzrmbS7e5RNJ859z/Ll3/V0l3O+euL13/lorBZVjSPznnTi0d\nf7Ok/+OcO73Kc08+OAAAkCjOOat1m+l1PMiiyb5vZh+S9E5JJwcOPy3p8MD1w0rHqh2v9tw1fwAA\nAJAuza7WeYekCySd7pzbFvjWTZLOMrMOMztS0lGS1jjnnpW02cwWlBpkz5a0qpkxAACAdKk5rTPp\nnc02SuqQ9Hzp0D3OuSWl710sqVfSqKRPOuduLx0/XtK3Jc2UdItz7pNTHgAAAEidpsIJAABA2Lze\nIdbMPm9mvzSzX5jZraWlyCiZbBM8SGb2HjN72Mx2mtn8uMfjAzN7h5ltKG2CeGHc4/GNmfWZ2XNm\ntjbusfjGzA4zszvN7BEze8jMPhH3mHxiZnub2b2l96uHzGxp3GPyjZlNM7MHzOymWrf1OpxIutI5\n90fOuXmSfiSJX/Z4FTfBw24PSXq3pJ/EPRAfmNk0Sf8m6e2SXi+px8zmxjsq7/yHiq8P9rRD0qed\nc6+XdKKk8/j7KSv1XS4svV+9UdJpZrYg5mH55pOS1tVzQ6/DiXNuS+DqPpJ2VbttO5pkEzxIcs49\n6pzbqOLyd0gLJG10zg0750Yl3SDpjJjH5BXn3F2SXoh7HD5yzj3rnHuwdHmLpPVin6pxnHNbSxf3\nVnE1LH0TJWZ2mIore79Vz+29DieSZGaXmtkTkt4v6e/jHo/HPiLpv+IeBLw2cXNENkHElJhZTsXq\nwL3xjsQvpWmLX0h6VtIdgd3QIf2ziqt76wpssYcTM7ujdCLAsa+HSv99lyQ55z7nnDtC0nWS/ibe\n0bZerdendJtLJI2ObXrXTup5fQCEx8z2lfQ9FVdhbql1+3binNtVmtY5TNKbSueZa3tm9r8kPVeq\nvJnqqGbX3IQtarU2eQu4XsVdZpdFNxr/THETvLbRwN8PihseHhG4PukmiMBEZjZdxWDyXecce1RV\n4Zx7ycxWS3qH6uyxSLmTJJ1uZu+UlJG0n5l9xzl3drU7xF45mYyZHRW4eqaKc5womWQTPOyJvhNp\nSNJRZtZpZh2SzlJxw0SMV9cnuzZ1taR1zrmvxj0Q35jZQWa2f+lyRtIiSRviHZUfnHOfdc4d4Zx7\ntYr/7tw5WTCRPA8nki4vlegflHSKip2+KPtXSftKuqO0PGt53APyiZmdaWZPSjpB0g/NrK17cpxz\nOyV9XMVVXo+oeOZwAn+AmV0v6eeSjjazJ8zsw3GPyRdmdpKkD0g6ubRc9oHSByQUvVLS6tL71b2S\nbnPO3VLjPqiCTdgAAIBXfK+cAACANkM4AQAAXiGcAAAArxBOAACAVwgnAADAK4QTAADgFcIJAADw\nyv8H8Ujfi8HxciYAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "fig = plt.figure(figsize=(9, 8))\n", "ax1 = fig.add_subplot(111)\n", "ax1.scatter(x[:,0], y)\n", "\n", "y_predict = theta0 + theta1*x \n", "ax1.plot(x[:,0], y_predict)" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python [Root]", "language": "python", "name": "Python [Root]" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.12" } }, "nbformat": 4, "nbformat_minor": 0 }