{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Gradient vs. Newton step" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/pavanramkumar/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.\n", " warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')\n" ] } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "% matplotlib inline" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "def simpleaxis(ax):\n", " ax.spines['top'].set_visible(False)\n", " ax.spines['right'].set_visible(False)\n", " ax.get_xaxis().tick_bottom()\n", " ax.get_yaxis().tick_left()\n", " ax.tick_params(labelsize=14)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Compare two functions: $y = x^2$ and $y = x^6$" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "collapsed": false }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcYAAAF2CAYAAAD0qDhcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl03OWd5/v3U4v20m4tlrV4kyzbYGNs47DFJhAITmDI\nQpzczISePqc7DZ1eTnpI7mQ6zenbt3M7PU1Ck8uFdCaZ3oBAQ9pJSNjaEIwXvGDM4g1bsnbZllVa\nq0q1PfeP8q9cNt5kV+mp+v2+r3N8bElVqo/kqvr+nl1prRFCCCFEgst0ACGEECKbSGEUQgghUkhh\nFEIIIVJIYRRCCCFSSGEUQgghUkhhFEIIIVJIYRRCCCFSZKwwKqXuV0p1KKWCSqldSqkbL/F+C5VS\n40qpsUxlE0IIIc4nI4VRKfVF4AfAXwHLga3Ab5RScy5yPy/wFPB6JnIJIYQQF6MysfONUmo78I7W\n+mspnzsEPKu1/vYF7vd9oBR4A3hUa12a9nBCCCHEBaS9xXiq1Xct8MpZX3oZuP4C91sP3Al8Pd2Z\nhBBCiEuVia7UasANHDvr88eAunPdQSk1G/gR8H9orQMZyCSEEEJckmyZlfrPwGNa612nPlYmwwgh\nhHAuTwa+5xAQA2rP+nwtMHie+6wDblJKPXTqYwW4lFJh4H6t9Y9Tb6yU0n/xF3+R/Hjt2rWsXbv2\nypOfZSI8wf/c+j8p8hbx4A0Ppv37p5vWmr/9278lEAjwR3/0R1RWVpqOJIRIg+HhYf7+7/+eoqIi\n/tt/+28olf1th795828IRoP82fV/RkleyUw9bFp+MWkvjFrriFJqN3Ab8FzKl24Dnj3P3Zae9fF/\nAv47sAroP9cdHnrooSsLegmKvcW4lZtAJEAkFsHr9mb8Ma+EUoqmpiYOHDhAT0+PFEYhbKK7uxuA\npqamnCiKkViEYDSIW7kp9habjjNtmepKfRi4Tyn1u0qpRUqpR4B64HEApdR3lVKvWjfWWu9L/QP0\nAXGt9X6t9WiGMl6UUgpfvg+A8fC4qRjT0tjYCJx+IQkhcl9qYcwFY1OJZeil+aU5UcjPlomuVLTW\nzyilKoFvkyiI7wOf0lr3nrpJHTA3E4+dbqX5pYyERhgNjVJZmP0tMOuFI4VRCPvo6ekBcrMw5qKM\nTb7RWj+utZ6ntS7UWq/SWm9J+drvaK3nX+C+/5gtaxjL8suA0//R2a6+vh6Px8OJEycIBoOm4wgh\nrlAgEODEiRN4PB7q6+tNx7kkUhhtzvqPzZXC6PF4aGhoAE5fZQohcpf1Om5oaMDtdhtOc2mkMNpc\nrhVGkO5UIewk18YXAUanElNDpDDalBRGIYRJuVgYpcVoc7lYGOfMSezV3tfXRzQaNZxGCHG5otEo\n/f39KKWSM85zgfV+WVZQZjjJ5ZHCeBG5WBgLCwupqakhFovR33/OZaBCiBzQ399PLBajpqaGgoIC\n03EumbQYba44rxiXcjEZmSQaz53Wl9XtIhNwhMhdVjdqLrUWI7EIgUggZxf3gxTGi3IpV062GmWc\nUYjcl4vji9ZmKL58X04u7gcpjJck1wtjJs7cFEJkltY65xb2Q+53o4IUxkuSi4WxrKyM0tJSgsEg\nQ0NDpuMIIabJ2qSjtLSUsrLcmcQyGsrtpRoghfGS5GJhtDYUB+jq6jKcRggxXbm2cbhFWowOkYuF\nEaC5uRmQwihELrJet9brOFckl2rk504r92xSGC+BVRitLoJckdpilHFGMdN+8pOf8L/+1//innvu\nYe/evabj5BStdc4XxlxuMWbkdA27ydUWY01NDYWFhYyNjTE6Okp5ebnpSMIhXnzxRVavXs3SpUup\nqqriv/yX/yLFcRpGRkYYGxujsLCQWbNmmY4zLXYojNJivAS5dsKGRcYZhSmHDh3iiSeeAGDhwoXy\n/Jsm6/eVa+OLYI/CKC3GS3D2In+PK3d+bc3NzRw8eJCuri6WLVtmOo5wiAceeICJiQkAtm7dyh13\n3GE4UW6xJt7kWjdqNB5lMjKJS7kozsvNxf0gLcZL4lIufHk+AManxg2nmR6ZgCNMcLvdlJWVMTY2\nxrPPPsujjz5qOlJOyfXxRV+eD5fK3fKSu8lnWK6OM9bV1eH1ejl58mTyCl6ImaC15q//+q/5p3/6\np5wbJzNpYmKCkydPkpeXlzMHE1vs0I0KUhgvWa4WRrfbndxnUbaHEzPpxz/+MX/6p39KXV0dTz75\npOk4OcNqLTY2NuJy5dZbdK6fqmHJrd+6QcklG1O5tWQDpDvVKR577DH+/M//nIceeij5uU2bNvHp\nT386bY8RDAb53ve+x9///d/zn//zf2bLli383d/9HV/4whfYv39/8nY///nP+cY3vsFVV11FTU0N\n//zP/5y2DHaXOvEm19ilxZg7s0gMy9UWI0hhvBKpRSabH2Pz5s20traycOFCvvnNbya/58aNG6mr\nq0veLhqNcv/99yfP6Tx7fatSCq01Sik2bNjAJz/5yTO+/sgjj/DHf/zHFBYWcs899/DEE0/w05/+\nlKqqKn7v936P9vZ2AO655x7uueeeK/65nChXJ96AFEbHsboGcrEwNjQ04Ha7OXbsGKFQKKfOdROX\nRinFrbfeyn333cddd92V/Pxvf/tbHnzwweTHHo+HH/3oR5f1GFprbr75ZgoLCwE4cOAA3//+93G7\n3YyMjFzZDyAACIVCHDt2DLfbTUNDg+k40yaF0WFyucXo9XqZPXs2PT099PT0sHDhQtORcsZMtBjT\n4cYbb2RiYoJnn32W9957DwC/38/777/PLbfckpbHUEpx/fXXA4kDdDs6OrjpppvS8r1FgnUaTkND\nA16v13ScabPDBuIghfGS5XJhhES3TE9PD11dXVIYbWrLli00NDQwb948AN544w1aW1vP6EqNRCI8\n8MADya7Uc7lQV6r1tVdffZUVK1ZQXFycfOwbbrghAz+Vs+TqMg2LtBgdpiSvBIViIjxBLB7D7XKb\njjQtzc3NvPnmmzLOaGOBQIDS0tNvSE8//fRHWoter/eyu1Kfe+45HnjgAQYHB9m4cSNtbW0ATE5O\nsm3bNimMaZDLE29SF/eX5JWYjnNFZFbqJXIpF778U4v8w7m1yB8SU7+VUvT39xOJREzHERlw5513\n0trayqOPPsojjzzCr371q7R1o0JirPrmm2/m4Ycf5hvf+AZTU1M8/vjjPPHEE/zhH/5h2h7HqSKR\nCP39/Silkkuscom1+UmuL+4HaTFOS2l+KWNTY4yGRikvyK0NuQsKCqirq2NgYIDe3l7mzp1rOpJI\ns1AolFwvuGvXLrxeb1q3YluzZg3PPPNM8mNrvFGkR09PD/F4nPr6+pycIGeXblSQFuO05Po4Y0tL\nCwBHjx41mkOk36FDh6iurubgwYNEIhG+/vWv89hjj1FUVGQ6mrhE1uvSep3mGimMDpWrp2xYrAF9\nKYz2U19fz+///u+zadMmvvWtb/Gd73yHDRs2mI4lpkEKY/aQrtRpyPUWY3NzM0opent7iUQiOTkd\nXJybz+fjhz/8oekY4jJFIhH6+vrOOCou11i7gtmhMEqLcRpyvTAWFhZSW1tLLBajt7fXdBwhxCm9\nvb3EYjHq6uqSGyjkGju1GKUwTkMu75dqkXFGIbJPrnejghRGx7Jmoo6Ecnf7KymMQmQf6/WYqwv7\n4fT7YkVhheEkV04K4zSU5JXgcXkIRAKEY2HTcS6LNc7Y19d3wd1PhBAzIxKJ0Nvbi1IqZwvjVHSK\nQCSAx+Wh2FtsOs4Vk8I4DUqp5MzUXG01WuOM0WhUxhmFyALW+GJtbW3Oji9aw0tl+WUopQynuXJS\nGKdJulOFEOlkh/FF6/0w1zY+OR8pjNNkh8Io6xmFyB7W/qhSGLOHFMZpslNh7O3tlXFGIQyyhjRy\neXwRpDA6nvUfb507louKiopknFGILGBdnOby+CJIYXQ8O7QY4XS3jRxDJYQ5dhhfBCmMjme3wijj\njEKYY4f1i2C/wih7pU5TSV4JbuVmMjJJOBYmz51nOtJlsV6IPT09RKNRPB55Koj06uzs5Cc/+QnL\nly8nHo/zhS98wXSkrJI6lJHLhTEcCyfXMOb6AcUWaTFOk1JKxhmFuAitNf/1v/5XvvnNb/Lxj3+c\nn/70p6YjZZ3U8cVcPh7Mai3aZQ0jSGG8LHbpTrUOK+7s7DScRNjNb3/7WyoqKigpKaG6upp///d/\nNx0p61ivu3nz5hlOcmXs1o0KUhgvixRGIS5s9+7dTE1N8cILL/AP//APbNu2zXSkrGO97qzXYa6y\nY2GUgaXLYJfCmHo+YzgcJi8vN8dLRfaJxWIEAgHWr19PPB5n2bJlvPfee6ZjZY1wOExvby8ulyun\nxxfBnoVRWoyXwS6FsaCggNmzZxOPx+nu7jYdR9hIY2Nj8sBdl8vF6OgoIyO5/XpJp+7ubuLxOLNn\nzyY/P990nCsihVEAUFaQ2xuJp5LuVJEJt9xyCwMDA0Ci9VhRUUF5uX3eOK9UR0cHkPvdqHB6EqKd\nCqN0pV4Gu7QYIfHCfPPNN6Uw2sBjjz3GwMAAbrebhx56CIBNmzbx8MMP86tf/SotjxEMBnn00Ucp\nKChg586dfO1rX2P79u1s376dv/zLv6S9vR2A2tpavvzlL/Poo48yPj7OE088kZbHtwu7jC+CPVuM\nUhgvgy/Pl1zLGIlF8Lq9piNdtqamJtxuNwMDAwSDwZzelioTHnr9ocw/xtorf4zNmzfT2trKwoUL\n+eY3v5ksjBs3bqSuri55u2g0yv3335/cI1drfcb3UUqhtUYpxYYNG/jkJz95xtcfeeQR/viP/5jC\nwkLuuecennjiCX76059SVVXF7/3e7yULI8B99913xT+XHQWDQQYHB3G73TQ2NpqOc0XCsTCTkUnc\nym2bNYwghfGyKKUoKyhjODjMSGiEWcWzTEe6bF6vlzlz5tDV1UVXVxeLFi0yHUlcBqUUt956K/fd\ndx933XVX8vO//e1vefDBB5MfezwefvSjH13WY2itufnmm5MXTwcOHOD73/8+brdbxg+n4ejRo2it\naWxsxOvN3YtqOLMb1S5rGEEK42UrLyi3RWGERHdOV1cXnZ2dUhjPko7W3Ey48cYbmZiY4Nlnn03O\n/vT7/bz//vvccsstaXkMpRTXX389AP39/XR0dHDTTTel5Xs7iXSjZj8pjJfJbuOMr7/+uowz5rgt\nW7bQ0NCQXDD+xhtv0NraekZXaiQS4YEHHrjgcWMX6kq1vvbqq6+yYsUKiouLk499ww03ZOCnsh8p\njNlPCuNlslNhbGhowOv1cvz4cSYmJigpsc9YgZMEAgFKS0uTHz/99NMfaS16vd7L7kp97rnneOCB\nBxgcHGTjxo20tbUBMDk5ybZt26QwXoKJiQlOnDiB1+uloaHBdJwrZtfCKMs1LpOdCqPH40muOZPT\nNnLXnXfeSWtrK48++iiPPPIIv/rVr9LWjQqJC6ibb76Zhx9+mG984xtMTU3x+OOP88QTT/CHf/iH\naXscO7Nai9akt1xn18IoLcbLZKfCCIlunSNHjtDZ2cnSpUtNxxGXIRQK8eSTTwKwa9cuvF4vd9xx\nR9q+/5o1a3jmmWeSH1vjjeLS2akbFexbGKXFeJmSJ2xM5e4JG6msF6q18FjklkOHDlFdXc3BgweJ\nRCJ8/etf57HHHsvpUxvsSApjbshYYVRK3a+U6lBKBZVSu5RSN17gtu1KqU1KqcFTtz+ilPq/lVJZ\nO5fZWss4EZ4gEouYjnPF6uvrKSgowO/3y9T7HFRfX8/v//7vs2nTJr71rW/xne98hw0bNpiOJVL4\n/X78fj8FBQXU19ebjnPFIrGILdcwQoa6UpVSXwR+AHwN2AI8APxGKdWutT7X4X9h4H8De4ARYBnw\nY8ANfCsTGa9U6lrG0alRqouqTUe6Ii6Xi5aWFg4cOEBHRwcrVqwwHUlMg8/n44c//KHpGOICUreB\nc7lyv7MutbVopzWMkLkW458CP9Fa/0RrfVBr/UfAAPAH57qx1vqI1vqftNbvaa17tNa/Av4VyOpF\nUnYbZ7Sm+Ut3qhDpZ72ucv38RUvygOJTe0fbSdoL46nuz2uBV8760svAJY3WK6UWAHcAr6c1XJrZ\nrTDOnz8fSLyAz94qTAhx+bTWtjmY2GLX8UXITIuxmkQX6LGzPn8MqPvozU9TSm1RSgWBg8BmrfW3\nM5AvbexWGCsrKykrKyMQCDA4OGg6jhC2MTg4SCAQoKysjMrKStNx0kIK48y5F7gG+DKwXimVleOL\nFusJ4Q/6DSdJD6WUdKcKkQFHjhwBEr0ydhmPs3NhzMTkmyEgBtSe9fla4ILNEK1136l/HlBKeYAf\nK6W+p7WOn31b6/QAgLVr17J27doriHx5KgoqAPu0GCHRzbNnzx46OjpkJxMh0sRu44sA/lCiQWC9\nD9pJ2guj1jqilNoN3AY8l/Kl24Bnp/Gt3Cl/LlgYTaksTHSJnAyeTO4hmeusF25XVxfRaBSPR/aA\nEOJKRCIRuru7AfusX9RaMxwcBk6/D9pJprpSHwbuU0r9rlJqkVLqEaAeeBxAKfVdpdSr1o2VUl9R\nSn1eKdWmlJqrlLoX+GvgWa111i4SLPIWke/OJxQNEYwGTcdJi+LiYurq6ohGo8kXsxDi8vX09BCN\nRqmrq0tuup7rgtEgoWiIfHc+RV77bSKRkeaA1voZpVQl8G0SBfF94FMpaxjrgNRLpyjwfwILAAV0\nAY+SWAuZtZRSVBZWMjAxwHBw2DZPkHnz5jE4OEhHR4etun6EMMHqRrVmfdtBamvRDj1lZ8vY5But\n9eNa63la60Kt9Sqt9ZaUr/2O1np+ysdPa62v1VqXaa1LtdZXaa3/Rms9lal86WJ1I1hPFDuQCThC\npI818cZOF5l27kaF7JuVmnPsWBibm5txu90MDAwQCARMxxEiZ1lLn1JPsLEDKYziguxYGL1eL01N\nTWit5RgqIa5AZ2cnWmsaGxvxerN26+dpk8IoLsiOhRFOd/tY3UBCiOmz4zINkMIoLsLuhVHGGYW4\nPFprW068ASmM4iJK8krwurwEIgFC0ZDpOGlTX19PYWEhfr+f4WF7FX0hZsLw8DB+v5/CwkLq6i64\nG2ZOCUaCBCIB8tx5tjtuyiKF8QpZSzbAXq1Gl8uVbDUePnzYcBohck/qNnB2OGbKYvelGiCFMS3s\nWBgBFixYAMg4oxCXw7qglG7U3COFMQ3sWhitF3RnZyexWMxwGiFyRywWS87olsKYe6QwpoFdC2Np\naSk1NTWEw2F6enpMxxEiZ3R3dxMOh6mpqaG0tNR0nLSSwiguiV0LI5y+2pVxRiEunTX8YA1H2IkU\nRnFJ7FwYZZxRiOmz6/giSGEUl6g0vxSPy8NEeIKpaNZv7zotzc3NeL1eBgYGmJiYMB1HiKw3MTHB\n4OAgXq+X5uZm03HSaio6xWRkEo/Lgy/PZzpOxkhhTAOlVPKwTuvwTrvweDzJF7e0GoW4OOt10tLS\nYrvzTJ2wVAOkMKZN8tDiwEnDSdJPulOFuHTSjZr7pDCmiZ3HGa0X+JEjR9BaG04jRPbSWsvEGxuQ\nwpgmdi6M1dXVlJWVMTk5yeDgoOk4QmQt66i2srIyqqqqTMdJOymMYlrsXBiVUrJsQ4hLkNpatOMY\nnBRGMS12Lowg44xCXAo7jy+CFEYxTWUFZbiVm/HwOOFY2HSctJs3bx4ul4vu7m6mpuy1JEWIdAiF\nQvT09JyxAb+dhGNhxsPjuJWb0nx77eZzNimMaeJSLioKTy3ZCNpryQZAQUEBjY2NxONxOaNRiHPo\n6OggHo/T2NhIQUGB6ThpZ72vVRRW4FL2Lh32/ulmmN27UxcuXAjAhx9+aDiJENnHel1YrxO7cUo3\nKkhhTCsnFUZZtiHEaVpr2xfGk8HEGm0pjGJakov8g/Zb5A8kTwoYHx/n2LFjpuMIkTUGBweZmJhI\nnkhjR9JiFJfFzrvfQGLZhnSnCvFRqa1FOy7TABgKDAFQVWi/9Zlnk8KYRrOKZgFwInDCcJLMkcIo\nxEfZvRsVThfG6qJqw0kyTwpjGpXml5LnziMQCRCIBEzHyYi5c+fidrvp6ekhGAyajiOEcYFAgN7e\nXtxuN3PnzjUdJyMmw5MEIgHy3Hm2X6oBUhjTSimVvJqyrq7sJj8/n6ampjP2hBTCyaw9hJubm8nP\nzzcdJyNSW4t27SpOJYUxzazCeGJSulOFcAIndaNaw0V2J4Uxzawnjl1bjCDLNoSwxOPx5DZwdi6M\n1rwJJ4wvghTGtEu2GG08Aae6upry8nICgQD9/f2m4whhTH9/P4FAgIqKCluepmFJthiLpcUoLoP1\nxLFzi1GWbQiR4IRlGnB6aEhajOKyVBQk9hEcCY3YcjNxi1UYDx06ZDiJEOY4YXwxHAszOjWKW7kd\nsbgfpDCmndvlTi6AtetCf0gs2/B4PPT39zM+Pm46jhAzbnx8nP7+frxeLy0tLabjZIzV+1VZWGn7\nzcMtzvgpZ5gTxhm9Xm/yaB3pThVOZPWWzJs3D6/XazhN5jhtfBGkMGaEE8YZAdra2gA4ePCg4SRC\nzDyrMLa2thpOkllOG18EKYwZ4YS1jHD6DaGjo4NIJGI4jRAzJxKJJM8ltXthdNoaRpDCmBFOWMsI\n4PP5mD17NpFIhM7OTtNxhJgxnZ2dRCIRZs+ejc/nMx0no5y2hhGkMGZEVdGpyTfBk8TiMcNpMsu6\nWpbZqcJJrOEDazjBrmLxGMPBYRRKCqO4MnnuPMoLyonrOP6Q33ScjEotjLILjnACrbVjxheHg8PE\ndZyygjK8bvtOMDqbFMYMcco4Y319PT6fj7GxMQYHB03HESLjBgYGGB8fp7S0lLq6OtNxMsqJ44sg\nhTFjnDLOqJRKXjXL7FThBKmtRTvvdgPOHF8EKYwZ44S1jBZrnEXGGYUTWBeAdu9GBWeuYQQpjBnj\nlLWMkNgFx+v1yi44wvbGxsYYGBjA6/Xa9lDiVE5cwwhSGDMm9cBiu09KSd0FR1qNws6cstsNJCYZ\nyRijSKsibxHF3mLCsTBjU2Om42ScjDMKJ7AKo92XaQCMTo0SiUco9hZT6C00HWdGSWHMICeNM1oT\nETo6OgiH7XuqiHCuqakpOjo6zphwZmdOHV8EKYwZ5aRxRp/Px5w5c4hGo8kTzYWwkyNHjhCNRpkz\nZw4lJSWm42ScU8cXQQpjRjllLaNl0aJFABw4cMBwEiHSz3peW89zu3Pq+CJIYcyo2uJaAI5NHjOc\nZGZYbxiHDh0iFrP3VnjCWWKxWHJ80SmF0XrfqimuMZxk5klhzKDakkRhPD553PYzUwGqqqqYNWsW\noVCIrq4u03GESJujR48SCoWoqamhqqrKdJyMi+s4xyYShdF6H3MSKYwZVOQtwpfnIxwLMxwcNh1n\nRkh3qrAjp3Wj+oN+IvEIpfmlFHmLTMeZcVIYM6yuJLGXotO6Uw8cOOCIVrKwP6214wqj9X5lDQc5\njRTGDLO6IaxuCbubPXs2paWljI2N0d/fbzqOEFfM2tGptLSU+vp603FmxOBE4kAA68LeaaQwZph1\nxWU90exOKSXdqcJWUluLdt803OLk8UWQwphxTutKBRlnFPayf/9+wDndqCBdqVIYM6yqqAqPy8NI\naIRQNGQ6zoxobm6moKCAEydOMDRk/80NhH0NDQ0xNDREYWEhzc3NpuPMiFA0xEhoBI/LQ1WR/Wfg\nnkvGCqNS6n6lVIdSKqiU2qWUuvECt/24UurflVL9SqlJpdRepdTvZCrbTHIpV3KBrFPGGd1ut+yd\nKmzB6vVobW3F7XYbTjMzrPepmuIaXMqZbaeM/NRKqS8CPwD+ClgObAV+o5Sac567XA+8C3wOWAL8\nf8CPlFIbMpFvpjm5O3Xfvn2Gkwhx+aQb1ZkydTnwp8BPtNY/0Vof1Fr/ETAA/MG5bqy1/q7W+jta\n621a66Na68eB50kUypzntJmpAAsXLsTr9dLX18fo6KjpOEJM28jICH19fXi9XhYsWGA6zoyxJgo6\ndeINZKAwKqW8wLXAK2d96WUSLcNLVQr405XLJKfNTIXEGY0LFy4ETl91C5FLrOdta2ur7c9eTGVd\nwDt1qQZkpsVYDbiBs5tHx4BL+k0rpT4N3AI8kd5oZqRuDRfXccNpZs7ixYsB6U4Vucl63lrPYyeI\n6zjHJ48D0pWaVZRSNwD/Cnxda73bdJ50KPIWUZpfSiQewR+0RSP4kixcuBCPx0N3dzdjY/Y/rFnY\nx9jYGD09PXg8nmTPhxMMB4eTW8E57XDiVJ4MfM8hIAacfblRC1ywL/HUzNUXgP+htf7RhW770EMP\nJf+9du1a1q5dexlRZ05tcS1jU2McmzzmmCnQ+fn5LFiwgAMHDnDgwAFWr15tOpIQl8TqRl24cCF5\neXmG08wc6UZNSHth1FpHlFK7gduA51K+dBvw7Pnup5S6GfgV8Oda60cv9jiphTEX1JXU8eHwhwxO\nDLJ4lnO6ZhYvXsyBAwfYt2+fFEaRM5zYjQoyI9WSqa7Uh4H7lFK/q5RapJR6BKgHHgdQSn1XKfWq\ndWOl1Frg1ySWaTytlKo99cc2R0c7cWYqnF7/1dXVxcTEhOk4QlzUxMQE3d3dZ6zHdQqZkZqQkcKo\ntX4G+BPg28AeErNRP6W17j11kzpgbspdvgoUAn8G9Kf82ZGJfCY47dBiS0FBAfPnzz/jhAIhstn+\n/fvRWrNgwQLy8/NNx5lR0pWakLHJN1rrx7XW87TWhVrrVVrrLSlf+x2t9fyzPnaf48+8TOWbaU7c\nGs4is1NFLnFqN2owEmR0ahSPy0NlYaXpOEZl3axUu3IpFzXFNYDzulPb2tpwuVwcPXqUyclJ03GE\nOK/JyUmOHj3qyG5UqzfLyVvBWZz9088wp3anFhYWMm/ePOLxuOydKrKadcD2vHnzKCx01nIF6UY9\nTQrjDLKecAPjA4aTzDyrW+qDDz4wnESI87O6Udvb2w0nmXkDE4n3JafPSAUpjDNqtm82AH3jfYaT\nzLz29nZcLhednZ3SnSqy0uTkJB0dHbjdbkcWxv7xfgAaShsMJzFPCuMMqiupw6VcnJg8QTgWNh1n\nRhUWFrL2YcvPAAAgAElEQVRgwQLi8bhMwhFZad++fWitmT9/vuO6UaeiU5yYPIFLuaQrFSmMM8rr\n9lJbXItGO7I7denSpQC8//77hpMI8VHW89J6njrJwMQAGk1tcS0eVyY2RMstUhhnmJO7U9va2mTv\nVJGVxsbG6OrqwuPx0NbWZjrOjJNu1DNJYZxh1hOvb8x5hTE/P5/W1la01jIJR2QV6/nY2trquEX9\ncPr9qMEnhRGkMM4464lnXaE5jXSnimzk5G5UON2DZfVoOZ0Uxhk2q3gWXpcXf8hPIBIwHWfGWacV\n9PX14fc75wgukb2Gh4fp6+sjLy/PUUdMWSbDk4yERvC6vMwqnmU6TlaQwjjDXMpFva8ecGZ3qtfr\nZdGiRYC0GkV2sLpRFy1ahNfrNZxm5lm9V7N9sx2/441FfgsGSHeqdKeK7CHdqNKNejYpjAY4eWYq\nwPz58ykoKODYsWOcOHHCdBzhYMePH+fYsWMUFhYyf/78i9/BhmRG6kdJYTQgdWaq1tpwmpnndruT\nW8RJq1GYZD3/2tvbcbvdhtPMPK11ckhHWoynSWE0oKKggkJPIZORScamnLmez+q2evfddx15cSDM\n01rz3nvvAc7tRh2dGmUyMkmhp5CKggrTcbKGFEYDlFKO705taWnB5/Ph9/vp7e29+B2ESLPe3l78\nfj+lpaW0tLSYjmNEajeqUspwmuwhhdEQJy/0B3C5XFx11VVAotUoxEzbu3cvAFdddRUulzPfCqUb\n9dyc+WzIAk6fmQpw9dVXA4lxnlgsZjiNcJJYLJZcpmE9D53I6rGSHW/OJIXREOsKrX+837FjbHV1\nddTW1hIMBjl8+LDpOMJBPvzwQ4LBILW1tdTWOvP8Qa1PH2YgM1LPJIXREF++j9L8UqZiU5wMnjQd\nxxjrat3q1hJiJljd905uLQ4FhpiKTVGaX0pJXonpOFlFCqNBVveFU8cZITG+o5Ti0KFDhEIh03GE\nA4RCIQ4ePIhSKjnO7UTSjXp+UhgNsrovesZ6DCcxx5oRGI1G5QBjMSP27dtHLBZj7ty5lJaWmo5j\nTO9YYja4dKN+lBRGg5rKmgDoHu02nMQsqztLZqeKmWB12zu5GxWga6QLgOayZsNJso8URoNm+2bj\ncXk4PnnckSdtWBYvXozH4+Ho0aOMjo6ajiNsbGRkhK6uLrxeL+3t7abjGBOIBDgROIHH5ZGlGucg\nhdEgj8uT7N93cqsxPz8/eeKGtBpFJlk73bS1tTnyQGKL9X4zp3QObpfztsK7GCmMhjWXJ7oxnFwY\n4XS31jvvvOPY5Ssis7TW7NmzB4Bly5YZTmOW1Y1qDeeIM0lhNMx6YlpPVKdasGABPp+PkydP0tPj\n3MlIInN6enoYHh7G5/M59iQNi3UhLuOL5yaF0bDG0kYUioGJAcKxsOk4xrhcruRV/DvvvGM4jbCj\n1NaiU7eAAwjHwgxMDKBQzCmdYzpOVnLusyNL5HvyqffVE9fx5PRpp1q+fDmQ2CIuHHbuRYJIv3A4\nnNwC7pprrjGcxqzesV7iOk69r558j3PHWS9ECmMWkGUbCdXV1TQ2NhIOh2VNo0irffv2EQ6HaWxs\npKqqynQco2SZxsVJYcwC1hPU6eOMcPpqXrpTRTpZ3ahOby3C6QtwmXhzflIYs4D1BO0d6yUWd/Yp\nE0uWLMHr9XL06FGGh4dNxxE2MDw8nFy7uGTJEtNxjIrFY8khGymM5yeFMQsU5xVTXVRNJB5hYGLA\ndByj8vPzWbx4MSCtRpEe1vNo8eLFjl67CDAwMUAkHqG6qJrivGLTcbKWFMYsIeOMp1mTcPbu3Us8\nHjecRuSyeDyeLIzSjSrji5dKCmOWkHHG01paWqioqGB0dJTOzk7TcUQO6+zsZGxsjIqKCpqbpRh0\njcrC/kshhTFLpLYYnb7zi1Iq2Wp8++23DacRuWz37t1AohdCKWU4jVlaa3pGE5tnWDtuiXOTwpgl\nygvKKc0vJRgNciJwwnQc46655hqUUuzfv5+JiQnTcUQOmpiY4MCBAyilpBsVOD55nGA0SFl+GWX5\nZabjZDUpjFlCKZXsTj06ctRsmCxQWlpKa2vrGWNEQkzHO++8Qzwep7W11dHnLlpSu1Gd3nq+GCmM\nWWRuxVwAOvwdhpNkh2uvvRZIdIc5vXtZTI/WOtmNunLlSsNpsoP1vmK9z4jzk8KYReZXJDY27vR3\nOn49IyQ2Fi8rK8Pv98skHDEtHR0d+P1+ysrKHL9hOCTWL3b6E68h631GnJ8UxixSVlBGdVE1U7Ep\n+sb7TMcxzuVysWLFCgB27dplOI3IJVZr8dprr3X0huGW3rFepmJTzCqaRVmBjC9ejDxjsox1NXdk\n+IjhJNnhmmuuweVyceDAAZmEIy6JNenG5XLJpJtTjvgT7yfzK6W1eCmkMGYZ64lrPZGdTibhiOna\ns2dPctKNz+czHScrWBfa0o16aaQwZpmW8hbcyk3fWB/BSNB0nKwgk3DEpdJaJ9e+yqSbhEAkQP94\nP27llvWLl0gKY5bJc+fRVNaERsvs1FPmz5+fnITT0SG/E3F+R44cwe/3U15ezrx580zHyQqd/k40\nmqayJvLceabj5AQpjFlIulPP5HK5kq3GHTt2GE4jstnOnTsBWLFihUy6OUXGF6dPnjlZKHUCjnQd\nJqxYsQK3282hQ4cYGRkxHUdkIb/fz6FDh3C73ckLKafTWnN4+DAg44vTIYUxC9WV1FHsLWZ0apST\nwZOm42SFkpISlixZgtY62SoQItXOnTvRWrN06VKKi+VIJYChwBBjU2MUe4upK6kzHSdnSGHMQkop\n5lUkxkdk2cZp1113HZDYWDwSiRhOI7JJJBJhz549AKxevdpwmuyR2o0q28BdOimMWcoaD7C6QQQ0\nNDTQ0NBAMBjkvffeMx1HZJH33nuPYDDInDlzaGhoMB0na8gyjcsjhTFLWU/koyNHicajhtNkD6s1\nsGPHDhl/FUBiHO2tt94CpLWYKhqPJg8ksHqgxKWRwpilfPk+aotricQjyTPUBCxZsoTi4mIGBwfp\n7u42HUdkge7ubo4dO0ZxcTGLFy82HSdrdI92E4lHqC2uxZcvGx1MhxTGLLagcgEAB08eNJwke3g8\nHlm6Ic5gtRZXrlyJx+MxnCZ7HBxKvG9Y7yPi0klhzGKLqhcBsP/Efuk2TLFy5UpcLhf79+9nbGzM\ndBxh0NjYWHJfVFmicZrWmgNDBwBon9VuOE3ukcKYxeaUzsGX52N0apTBiUHTcbJGaWkp7e3txONx\nWbrhcDt27CAej9Pe3i6HEacYmBhgdGoUX56PBp9MRpouKYxZTCl1utU4tN9wmuyyZs0aIHEcVTgc\nNpxGmBAOh5PHkX3sYx8znCa77D+ReL9YVL1IlmlchowVRqXU/UqpDqVUUCm1Syl14wVum6+U+qlS\naq9SKqyU2pSpXLkmtTtVnNbY2EhjYyPBYDC5fk04y549ewiFQjQ1NTFnzhzTcbKKdKNemYwURqXU\nF4EfAH8FLAe2Ar9RSp3v2esGgsCjwK8ykSlXtZS3UOAp4ETgBEOBIdNxssr1118PwPbt24nH44bT\niJkUj8fZvn07IK3Fsw0FhjgROEGhp5DmMjlN43JkqsX4p8BPtNY/0Vof1Fr/ETAA/MG5bqy1Dmit\n79da/xiQo+tTuF1u2qragNNXgSKhra2NyspK/H4/Bw7I78ZJ9u/fj9/vp7Kykra2NtNxsorVu9Ra\n1Yrb5TacJjelvTAqpbzAtcArZ33pZeD6dD+eE0h36rm5XK7kWOPWrVtl5q5DaK3ZunUrkGgtyika\nZ5Ju1CuXiWdUNYmu0WNnff4YILvYXoYFlQvwurz0jfcxNiXLE1ItX76cwsJCent76emRjRCcoKen\nh76+PoqKili+fLnpOFllNDRK33gfXpdXtoG7AnKplQO8bm9yka50p54pLy+PVatWAbBt2zbDacRM\nsFqLK1euxOv1Gk6TXaz3hwWVC/C65XdzuTKxTcQQEANqz/p8LZC2xXgPPfRQ8t9r165l7dq16frW\nWal9Vjv7h/az/8R+VjfIfpCpVq9ezZYtWzhw4AAnT56kqqrKdCSRISdPnuTgwYO43W7ZF/UcpBs1\nPdJeGLXWEaXUbuA24LmUL90GPJuux0ktjE6wsHIhLuWia7SLQCRAkbfIdKSsUVJSwrJly3j77bfZ\nsmULd911l+lIIkO2bNmC1pply5ZRUlJiOk5WCUQCHB05iku5aK1qNR0np2WqK/Vh4D6l1O8qpRYp\npR4B6oHHAZRS31VKvZp6B6VUu1JqOYkxyhKl1DKl1LIM5cs5hd5C5pbPJa7jMgnnHG644QaUUuzd\nu5fR0VHTcUQGjI6OsnfvXpRS3HDDDabjZJ39J/aj0cwtn0uBp8B0nJyWkcKotX4G+BPg28AeErNR\nP6W17j11kzpg7ll3+zWwG7iXxKzWPcDbmciXq66qvQqAd4+9azhJ9qmqqmLJkiXEYrHkGJSwl61b\ntxKLxViyZIl0l5/D3mN7Abi69mrDSXJfxibfaK0f11rP01oXaq1Xaa23pHztd7TW88+6/VyttTvl\nj0trLYtwUrRXt+N1eeka7cIf9JuOk3VuuukmAHbv3s3ExIThNCKdJiYm2L17N3D6/1mcNhwcpnu0\nmzx3nowvpoHMSs0h+Z58Fs9KnDdnXR2K02pra2lrayMajSZ3RRH2sG3bNqLRKIsWLaK29ux5fcLq\nRWqvbifPnWc4Te6TwphjltUlhl33Du6VBe3ncPPNNwOJUxeCwaDhNCIdgsFg8hQVaS1+lNaavYOJ\nC2Xr/UFcGSmMOaalvIXS/FL8IT89Y7Kg/WwNDQ3MmzePcDicPMBW5La33nqLcDjM/PnzaWiQI5TO\n1j3ajT/kpzS/lJbyFtNxbEEKY45xKVdycP2dwXcMp8lOVqvxrbfeYmpqynAacSWmpqaSFzjSWjw3\na1hlWe0yXEre0tNBfos5aFltorvkg+MfEIlFDKfJPs3NzTQ1NREMBtmxY4fpOOIKvPXWWwSDQZqa\nmmhulpMizhaJRfjg+AeAzEZNJymMOWhW8Sxm+2YzFZvi4MmDpuNkHaVUciekLVu2EAqFzAYSlyUY\nDCaX3qxbt04O3D2HA0MHmIpN0eBrYFbxLNNxbEMKY45aXpfYPNkadBdnmjt3Li0tLYRCIdlDNUdt\n27aNUCjE3LlzmTv37GXPAlK6UWXSTVpJYcxRS2uW4lZuDg8fZnxq3HScrKOU4pZbbgESb7CBQMBw\nIjEdgUAgueRm3bp1htNkp/GpcY4MH8Gt3CytWWo6jq1IYcxRRd4iWqta0WjeHpANgs6lqamJBQsW\nEA6H2bJly8XvILLGli1bCIfDLFy4kKamJtNxstLbA2+j0bRWtcreyWkmhTGHWads7OzfSSweM5wm\nO1mtxh07dshuODlifHw8OWlKWovnFovH2NmfWNspp+2knxTGHNZS3kJNcQ0T4Qn2ndhnOk5Wmj17\nNosWLSISibB582bTccQlePPNN4lEIrS3tzN79mzTcbLSByc+YCI8QU1xjaxdzAApjDlMKcV1DdcB\n8FafLGY/H2tG465duxgZGTEdR1zAyMgIu3btOmNmsfiot3oTr/frGq6T2boZIIUxx11VexWFnkJ6\nx3rpHeu9+B0cqLa2lqVLlxKLxdi0aZPpOOICNm3aRCwWY+nSpbIn6nn0jvXSN95HoadQ1i5miBTG\nHJfnzmNF/QoAdvTJYvbzueWWW3C73bz77rv09/ebjiPOob+/n3fffRePx8MnPvEJ03GyltVaXFG/\nAq/baziNPUlhtIFVDatQKD44/oEs3TiPiooK1qxZA8DLL78sG7BnGa01L7/8MgDXXXcd5eXlhhNl\np/GpcT448QEKxaqGVabj2JYURhsoLyhnUfUiYjrG7oHdpuNkrZtuuonCwkKOHj3KoUOHTMcRKQ4d\nOsTRo0cpKiqSPVEvYFf/LuI6TvusdsoL5OIhU6Qw2sR1cxKTcHb27SQajxpOk50KCgqSEzpeeeUV\nYjFZ4pINYrEYr7zyCgAf//jHKSgoMJwoO0XjUXb17wJITroTmeExHUCkR3NZM7XFtRybPMY7g++w\ncvZK05Gy0sqVK3nrrbfo6enhr/7qrygrK6O6upr169dTUVFhOp4jvf322wwNDVFVVcXKlfK8PZ93\nBt9hMjJJXUkdTWWy6UEmSYvRJpRS3NycOG5pc9dmaTWeh9vtZvXq1WzZsoU33niDoaEhDh48yPe+\n9z38fr/peI4TCoV47bXXALj11ltxu92GE2WnaDzKG11vAHBT002yRCPDpDDayOJZi6kprmF0apQ9\nA3tMx8laH374IRUVFcRiMY4ePYrb7cbj8fDCCy+YjuY4r732GoFAgObmZhYtWmQ6TtbaM7CHsakx\naotrWTxrsek4tieF0UaUUqxtWQvA5m5pNZ7PyZMnaWtrA6C3t5eJiQncbjdDQ0OGkznL4OAgO3bs\nwOVyceedd0or6DxSW4sfb/m4/J5mgBRGm2mvbqe2uJaxqTHZXPw8qqurKSwspKGhAUi0IKPRKNXV\n1YaTOYfWml//+tdorVm9erUs5r+A3f27GQ+PU1tcS3t1u+k4jiCF0WbOaDXKWOM5rV+/nmg0SlNT\nE3l5efj9fvr7+1m/fr3paI6xd+9euru7KSkpka3fLiASi/Bm95sArG1ZK63FGSKF0YYWVS+irqSO\n8fA4u/tlXePZKioqePDBB1myZAnLly+nqqqK5uZmCgsLTUdzhFAolFye8clPflKWZ1zA7oFEa7Gu\npI5F1TIGO1OkMNrQ2WONkVjEbKAsVFFRwVe+8hX+5m/+hs985jNorZOzI0Vmbdq0icnJSZqbm7nq\nqqtMx8la0lo0RwqjTbVVtVFfUs9EeCL54hIfpZRi/fr1uFwuduzYQV9fn+lIttbX18fOnTtlws0l\n2Ny9mYnwBLN9s2mrajMdx1GkMNqUUopPLfwUAFt6tjAcHDacKHvV1tayZs0atNZs3LiRaFTGZTMh\nGo2yceNGtNasWbNGJtxcwHBwmC3dWwC4Y8EdcgExw6Qw2lhTWRNX115NNB7lpcMvmY6T1datW0dV\nVRXHjx+XA40zZPPmzRw/fpyqqirWrVtnOk5We/Hwi8R0jGW1y2SXGwOkMNrcbfNuI9+dz8GTBzl0\nUjbOPh+v18tdd90FJN7ABwcHDSeyl8HBQTZv3oxSirvvvhuvV45LOp9DJw9x6OQh8t353Db/NtNx\nHEkKo8358n3JiTgvHn5Rlm9cQHNzM6tXryYej7Nx40bZZDxNYrEYGzduJB6Ps3r1apqapAV0PtF4\nlBcPvwgkJtyU5JUYTuRMUhgdYHXDamYVzWI4OMy2nm2m42S1W2+9lfLycgYGBti6davpOLawdetW\nBgYGKC8vlwOIL2JbzzaGg8PMKprF6obVpuM4lhRGB3C73MmJOG90vSETcS4gLy8v2aX6+uuvS5fq\nFRocHOT1118H4K677iIvL89soCx2MnAyufXbnQvvxO2SDdVNkcLoEPMq5nFVzVVE4hGe3/88cR03\nHSlrzZs3j5UrVxKLxfi3f/s3IhFZB3o5wuEw//Zv/0YsFmPVqlXMmzfPdKSsFYvHeH7/80TiEa6u\nvZq5FXNNR3I0KYwOcufCOynLL6N3rDd5ZSrO7fbbb2fWrFkMDQ3x4osvmo6Tk1566SWGhoaYNWsW\nn/zkJ03HyWpvdL1B33gfZfll3LnwTtNxHE8Ko4MUegv5T4v+EwrFG11v0DPaYzpS1vJ6vXz+85/H\n4/Gwe/du9u/fbzpSTtm3bx+7d+/G4/Hw+c9/XmahXkDPaA9vdL2BQnFP+z0UeGSLPNOkMDrM3Iq5\nXN94PXEd5+cHfs5UdMp0pKxVW1vLbbclpsv/4he/YHR01HCi3DA6OsovfvELILEXqizkP7+p6BTP\n738ejeb6xutpKW8xHUkghdGR1s1dR11JHcPB4eTUcHFuq1evprW1lWAwyPPPP088LmOzFxKLxXj+\n+ecJhUK0trayatUq05Gy2ouHX8Qf8lNXUse6ubLpQbaQwuhAHpeHz7Z/Fo/Lw57BPezq32U6Utay\nFqT7fD66urp4+eWXTUfKaq+88gpdXV34fD7uvvtu2crsAnb27WTP4B48Lg+fa/8cHpfHdCRxihRG\nh6opruHTrZ8G4Ncf/poOf4fhRNmruLiYL3zhC7jdbrZv387evXtNR8pKe/fuZfv27bjdbu69916K\ni4tNR8paHf4OfnP4NwB8uvXTzCqeZTiRSCWF0cGW1y3nxqYbies4z3zwDEOBIdORslZTUxOf+lRi\nLegvf/lL+vv7DSfKLv39/fzyl78E4M4776SxsdFwouw1FBjimQ+eIa7j3Nh0I8vrlpuOJM4ihdHh\nPjH3EyyqXkQoGuKp954iGAmajpS1Vq5cybXXXks0GuVnP/sZk5OTpiNlhcnJSZ5++mmi0WjydyTO\nLRgJ8uR7TxKKhlhUvYhPzJWdgLKRFEaHU0rx2fbPUldSx8ngSX72wc/kYOML+NSnPkVjYyOjo6M8\n88wzjj+iKhqN8swzzzA2NkZjY2OyVS0+KhKL8LMPfsZwcJi6kjo+2/5ZGYPNUlIYBXnuPL609EuU\n5JVwdOQoT73/lBTH8/B4PNx7773JyThOnqkaj8d5/vnnk5Nt7r33Xtxu2cbsXCKxCE+9/xRHR45S\nklfCl5Z+iTy3bI+XraQwCgDKCsr46rKvUpJXQoe/Q4rjBfh8Pr7yla9QUFDAvn37ePHFF9Fam441\no7TW/OY3v2Hfvn0UFBTwla98BZ/PZzpWVorEIjz53pN0+DsoySvhq8u+SllBmelY4gKkMIqkWcWz\nziiOT773pBTH86itrWXDhg243W527NjhuMONN2/ezM6dO/F4PGzYsEEW8Z9HOBbmyfeepHOkk5K8\nEu5bfp/MQM0BUhjFGWYVz+K+5fdRkldC50gn//LuvxCIBEzHykotLS187nOfQynFpk2bePvtt01H\nmhFvv/02mzZtSoxPf/aztLS0mI6UlQKRAP/67r/SOdKJL8/Hfcvvo7qo2nQscQlULnYBKaV0LubO\nJUOBIf7xnX9kPDxORUEFX7rqS9QU15iOlZV27tzJCy+8gFKKz3zmM6xYscJ0pIx5++23+eUvf4nW\nmvXr18vONudxfPI4T733FP6QH1+ej68u/6oUxZmRltlMUhjFeY1NjfH0+0/TP95PnjuPz7V/jrbq\nNtOxstKbb77Jq6++CmDbgrFjxw5+/etfA4kDnW+88UbDibLTwaGDPLf/OcKxMLN9s9mwdAOl+aWm\nYzmFFEaReZFYhI0HN/L+8fdRKNa2rOWm5ptwKemFP9v27duTR1TdfvvtfOxjHzOcKH22bdvGSy+9\nBMAdd9zBmjVrDCfKPnEdZ3PXZl4/+joazdKapdzddjdet5wsMoOkMIqZobVmc/dmNnVuAqDB18A9\n7fdI19A5WN2qAOvWrePmm2/O6bVqWmveeOMNXnvtNQA+/elPs3LlSsOpss9QYIif7/85feN9QGLj\njBubbszp//scJYVRzKwjw0fYeHAjY1NjeFwePjH3E6yZs0Ze/GfZs2cPv/jFL9Bac/XVV3PXXXfh\n8eTeBtHRaJRf/OIXvPvuuyiluOuuu7jmmmtMx8oqcR3nrd63+I/O/yAaj1KaX8rdbXczv3K+6WhO\nJYVRzLxQNMSLh1/kncF3gETr8fYFt9NU1mQ4WXY5cOAAzz//POFwmDlz5rBhwwZKSkpMx7pkExMT\nPP300/T29pKXl8dnP/tZFi1aZDpWVuke7ealwy8lW4nX1F3D7Qtul4OGzZLCKMw5OHSQXx76JRPh\nCQDaq9u5dd6tVBVVGU6WPQYHB3nqqacYHR2ltLSUL33pS9TX15uOdVH9/f08/fTTjI2NUVZWxpe/\n/GVZp5jiZOAkr3a8yv6h/QD48nx8pu0ztFa1Gk4mkMKYe7ntZio6xdaerWzt2UokHsGlXCyvW87H\n5nxMFjGfMjExwc9+9jN6enpwu92sW7eO66+/Hpcr+yYvxeNxtm7dymuvvUYsFqOpqYkvfvGLcnzU\nKScmT7CtdxvvDL5DXMfxurxc33g91zdeT74n33Q8kSCFUWSH8alxXjv6GnsG9qBJ/L8srFzIxxo/\nxtzyuY4fg4xGo7z00kvs3LkTSBxhdc8991BRUWE42Wl+v5+f//zndHd3A7Bq1Spuv/32nBwbTSet\nNZ0jnWzt2crh4cMAKBTX1F/DupZ1+PJlG7wsI4VRZJehwBDbe7ezd3AvkXhiK7nKwkqurr2aq2uv\nprKw0nBCsw4fPszGjRsZHx8nLy+PT3ziE6xcudLoxtuxWIxdu3bxH//xH4TDYXw+H3fffTcLFiww\nlikbnAyc5L3j7/HusXcZDg4D4HV5WV63nDVz1siQQfbK7sKolLof+DOgHvgA+BOt9ZsXuP1S4IfA\nauAk8COt9f91nttKYcxigUiAXf272Nm3k/HwePLzc0rnsKh6EQsrF1JTXGO0Jen3+3nhjRcYCgxR\nXVTN+pvXz0gLLhAI8MILL/DBBx8AUFVVxW233UZbW9uM/j601hw8eJBXXnmFkydPArBkyRLWr19P\nUVHRjOXIFlprjk8e58PhDzkwdIDesd7k13x5PlY1rGLl7JUUeZ33u8kx2VsYlVJfBP4Z+BqwBXgA\n+B2gXWvde47b+4BDwOvAXwLtwP8G/kJr/f1z3F4KYw6I6zid/k7ePfYu+4f2E46Fk1/z5flYULmA\n5vJmGksbqSysnLHC4Pf7+d4z38Mzz4Pb6yYWiRHtiPLgvQ/OSHE8V1Fqbm5m7dq1tLS0ZPT3oLXm\n6NGjvP7663R1dQHmirNJWmuGg8P0jPXQNdLF4eHDZ1zE5bnzaK9u5+raq5lbMVc2tMgdWV0YtwPv\naK2/lvK5Q8CzWutvn+P2fwB8F6jRWodPfe7bwNe01o3nuL0UxhwTiUX4cPhDPjz54UfehACKvEU0\n+PkNg2MAAAxhSURBVBqoK6mjpriGmuIaqouqcbvS3834Lxv/hYNFB3F7T3/vWCRGW6CNr9z9lbQ/\n3vnEYjF2797N66+/TiCQ2Ki9urqaVatWsWzZMgoK0jftPxQKsXfvXnbu3MnQ0BAARUVFrF27lmuv\nvdbW5yjG4jGGAkMcnzzO8cnjDE4M0jvWSzAaPON21sXawqqFLKxcKDvW5KbsLIxKKS8QADZorZ9L\n+fwPgSVa63XnuM8/ApVa68+kfG4l8BYwT2vdddbtpTDmMKvb6oj/CD2jPfSM9SSXfaRyKRdl+WVU\nFFZQXlBORUEFvnwfvjwfJXkl+PJ9FHgKpn01/4OnfsBI/chHPl8+UM6ffOlPLvvnulyhUIi33nqL\nXbt2MT6euGDIy8tjwYIFzJ8/nwULFlBWNv3z+0ZHRzl8+DBHjhzh8OHDhMOJFrvP52PlypVcd911\naS2+JsR1nFA0xPjUOBPhCcbD44xPjeMP+RkJjeAP+hmdGiWuP3qYdEleCY2ljTSWNTK/Yr7x7n2R\nFmn5D8zElLNqwA0cO+vzx4BPnOc+dUDPOW6vTn2t6yP3EDlLKUVtSS21JbXQmCiUo1Oj9I71Jq/q\nj08exx/04w8l/pz3e6Eo8BRQ6C2k0FNIviefPHce+e588j35eF1ePC4PXrcXr8uL2+VmIjJBv78f\nT54Hl3KhUMSjcXz46PR3Jj6nFAr1kb+t/Crl9Xe+N1M1jdfo4pWLabumjcNHDrNnzx66u7vZuW8n\nO/clZrJWVlZSM6uG8opyKsorKC0rxeM+/fKNxqKMjY7hH/Ez4h/h+InjDA8Pn/EYTU1NXHPNNSyY\nvwC32814bJzxyTNb7udjzTb+yOdTLlA1Ovmx9e+z/47rOJrE39afWDxGXMeJxqPEdIxYPEYkHiES\nixCNR4nEI0xFp5iKTRGOhZmKThGMBglGgoSiofNmsygUlYWVyZ6ImuIa5pTOoSy/TAqhOCdnz8UW\nWUEpRXlBOeUF5Wd8PhqPJq/6R0Ij+EP+RKvgVOtgIjyReIM89edSBSuC7Nu+D9dcFy6Pi3g0Trwz\nTsmaEv5x7z+m+8ebvnngqncxPDyM3+/H7/cTG47B8MXvmsrtdlNRUUFFRQWVlZWcLDzJq6Ov8urb\nr2YmtwEKRaGnkJK8kmQvQkleCRUFp3oZTvU2eFzyVicuXSaeLUNADDh7q4xaYPA89xk8z+31+e7z\n0EMPJf+9du1a1q5dO/2kIqt5XB6qi6ovuFm51ZUWjCSK41T0VKsiNsVUdCrZ4rBaH9F4lFhdjMXV\ni9n59k7GxsYozivm6juupshXlGzNnKu1Y7VMUv9tuVjX/sVaNR9RBI1VieH1eDzO2NgYk5OTBAIB\nJicnCYVCZzymUoqCggKKi4spKiqiuLiY0tLSK9pI4GIt3rNbW1bLOvXfZ/9ttdBdyoVLuXC73CgU\nHpcHt8uNW7nxuDxntPI9Lg/5nnzy3ad6Azz5FHoKKfQWXlZXuhAXM5OTbw6SmHzzP85x+68B/w9n\nTr7578AfyOQbIYQQlygtfeOZutR6GLhPKfW7SqlFSqlHSKxnfBxAKfVdpVRqf86TJCbs/G+l1BKl\n1GeBbwJ/l6F8QgghxDllpONda/2MUqoS+DaJgvg+8KmUNYx1wNyU248ppW4D/l9gJ+AH/lZr/YNM\n5BNCCCHOR7aEE0IIYRdZ3ZUqhBBC5CQpjEIIIUQKKYxCCCFECimMQgghRAopjEIIIUQKKYxCCCFE\nCimMQgghRAopjEIIIUQKKYxCCCFECimMQgghRAopjEIIIUQKKYxCCCFECimMQgghRAopjEIIIUQK\nKYxCCCFECimMQgghRAopjEIIIUQKKYxCCCFECimMQgghRAopjEIIIUQKKYxCCCFECimMQgghRAop\njEIIIUQKKYxCCCFECimMQgghRAopjEIIIUQKKYxCCCFECimMQgghRAopjEKI/7+dewuV66rjOP79\n0TZYLwFtMdHGBhFvrZhoVIxWrZcE+uKlBdsGTNEieMFasVYktUWrKGK9IIgvrQ9FiQRFUShpmxgv\nNUFbLbRRFBK1Ny3RWmNsYjVZPuwdWJ2cmTNnLmfOTL4fWA+zZu09/zVr7/mfvc/aS1LFxChJUsXE\nKElSxcQoSVLFxChJUsXEKElSxcQoSVLFxChJUsXEKElSxcQoSVLFxChJUsXEKElSxcQoSVLFxChJ\nUsXEKElSxcQoSVLFxChJUsXEKElSxcQoSVLFxChJUsXEKElSxcQoSVLFxChJUsXEKElSxcQoSVLF\nxChJUsXEKElSxcQoSVJl5IkxybIkX0tyIMmhJD9IctY825yTZFuSfUmOJbl21HFJktSPcVwxfhV4\nB3AxcB6wHPhRkvTY5snAH4EtwP4xxCRJUl9GmhiTLAfeA1xVStlZSrkbeBfwUuAt3bYrpdxZSrm6\nlLIVODzKmDQ+u3btmnQIwnFYChyDpSHJ+aPYz6ivGNcBpwK3Ha8opTwA/A54zYg/SxPmj8HS4DhM\nnmOwZJw/ip2MOjGuBI6WUv7eUf9w+54kSUtaX4kxyfXtpJhu5WiS1487WEmSxi2llPkbJc8Azpyn\n2X3AeuB24Jn1VWOSe4FtpZRP9fFZ97RtP92jzfxBS5JOOqWUXhM9+3Jqnx/0CPDIfO2S3AX8D9gA\nbG3rVgEvBu4YPMwT4hm645IkzaWvxNivUsrBJDcCX0hygCaZ3gDcDew43i7JDmBPKWVL+/o04Bwg\nwJOAlUnWAIdKKftGGaMkSb30dSt1QTtsktwXgU3A6TS3Vj9YSnmwarMf+HEp5fL29Wqa5xg7g/lJ\nKeVNIw1QkqQeRp4YJUmaZlOxVmqS9ybZmeQf7SzYs/vc7qIke5McSXJvkrePO9ZZNeBSf5dVs5br\nGczLFivuaZfkA0n2Jzmc5M4k583T/iVJdiV5LMn9ST65WLHOsoWMQ5LVXWbub1zMmGdJkte1vzkP\ntN/n5j62GfhcmIrESLNk3HbgOk683TqnJOtpJgDdDKwBvg1sS/LKcQU54wZZ6g/g3zTPsB4vzyql\nPD7OQGdFkouBrwCfAdYCvwBuaSe0zdX+aTSLa/yFZrGNDwMfS/KRxYl4Ni10HFoF2Eh13AM7xxzq\nLHsqcA9wBfDYfI2HPhdKKVNT2g4eBc7uo+1WYHtH3W3Atybdj2krNEnwP8AlVd2qdiw29NjuMuDg\npOOf1gLsAb7RUfcH4LNd2r8feBRYVtVtAe6fdF+muQwwDquBY8DLJx37LBbgX8DmedoMdS5MyxXj\nINYDt3bUbcel6QYxzFJ/pyf5U3sr44dJ1o4xzpnRTmJbR/Wdt26l+3f+auBn5YlX5NuBZ7cT3LRA\nA47Dcd9L8nCSnye5aCwBqpuhzoVZTowraZaiq7k03WAGXerv9zSLyr8VuAQ4AtyR5HljiXK2nAmc\nwsKO4W7HfHpso94GGYdDwEeBdwIX0Dyq9p0km8YVpE4w1Lkw0ucYFyLJ9TSXtt0U4I2llJ8uUkgn\nnX7HYND9l1L20NyGOv55u4HfAB8Crhx0v9JS1v4B+eWq6tdJzgCuppnroCVuYomR5sC5eZ429w2x\n/78CKzrqVrT1avQ7BuuBU5Kc0XHVuALo+w+XUsqxdnWk5y840pPP32j+h7uQY7jbMV96bKPeBhmH\nufwSePeogtK8hjoXJpYYS5/LzA1hN83SdDdUdRtoZpSJiS31t4bmqlE9lFL+237vG4DvVm9tALZ1\n2Ww38Pkky6r/rWwEHiql/Hl80c6uAcdhLi+jmSGpxTHcuTDpGUZ9zkJaQfODuolmttcF7eunV212\nUM0So7nKeRz4OPBC4BM0MytfMen+TGMBvk5z9fhmmpN8J3AX7SIRXcbg2vZgfG47Xje1Y7Bu0v2Z\nhkLzP6ojwOXAi2gemTkIrGrf/xxwe9V+OfAQze26c4ELgX8CV066L9NcBhiHzcClbdsXAFe1218x\n6b5MawGe0v6GrKV5BOya9vVzuozBUOfCxDvc55dyXZsQj3aUzVWb/cCNHdtdCPy2PSj3Am+bdF+m\ntQCntT8IB2gmF3wfOKujzRPGAPgSzVJ/h2luX9wCvGrSfZmmAryv/V4PA78CXlu9901gX0f7c4Fd\nNM96PQhcM+k+zEJZyDi0iXEvzWMFj9LcRr100n2Y5gK8oUsOuGmuMWjrBj4XXBJOkqTKLD+uIUnS\ngpkYJUmqmBglSaqYGCVJqpgYJUmqmBglSaqYGCVJqpgYJUmqmBglSar8HzQRMIww2RP4AAAAAElF\nTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "x = np.linspace(-1, 1, 100)\n", "y1 = x ** 2\n", "y2 = x ** 6\n", "\n", "plt.figure(figsize=(7,6))\n", "ax = plt.subplot(111)\n", "simpleaxis(ax)\n", "plt.plot(x, y1, 'k', alpha=0.5, lw=2)\n", "plt.plot(x, y2, 'g', alpha=0.5, lw=2)\n", "plt.plot(-0.25, 0.25 ** 2, 'ko', alpha=0.5, lw=2)\n", "plt.plot(-0.25, 0.25 ** 4, 'go', alpha=0.5, lw=2)\n", "plt.ylim([-0.1, 0.4])\n", "plt.legend(['$y = x^2$', '$y = x^6$'], frameon=False, fontsize=16, loc='upper center')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's consider $x_0 = -0.25$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Gradient step" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For $y = x^2$, the gradient $y'(x_0) = 2x_0 = -0.5$.\n", "\n", "For $y = x^6$, the gradient $y'(x_0) = 6x_0^5 = -0.0002$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For a learning rate of $\\zeta = 0.1$, the corresponding steps are:\n", "\n", "$0.05$ for $y = x^2$ and $0.00002$ for $y = x^6$.\n", "\n", "Thus, even though $x_0$ is the same distance away from the minimum $x^* = 0$ in both functions, we take an extremely small step for $y = x^6$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Newton step" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Gradients\n", "For $y = x^2$, the gradient $y'(x_0) = 2x_0 = -0.5$.\n", "\n", "For $y = x^6$, the gradient $y'(x_0) = 6x_0^5 = -0.0002$.\n", "\n", "#### Second derivatives\n", "For $y = x^2$, the second derivative $y''(x_0) = 2$.\n", "\n", "For $y = x^6$, the second derivative $y''(x_0) = 30x_0^4 = 0.117$.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The corresponding steps are:\n", "\n", "$0.25$ for $y = x^2$ and $0.001$ for $y = x^6$.\n", "\n", "Thus, the Newton step takes the local curvaturve into account. It proposes larger steps for regions that are relatively flat and smaller steps for regions where the curvature is high. " ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.11" } }, "nbformat": 4, "nbformat_minor": 0 }